2024-09-20T18:32:50.607 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-20T18:32:50.622 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-20T18:32:50.761 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/gabrioux-2024-09-20_12:24:58-orch:cephadm-wip-guits-main-2024-09-20-1028-distro-default-smithi/7912715 branch: wip-guits-main-2024-09-20-1028 description: orch:cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root} email: gabrioux@ibm.com first_in_suite: false job_id: '7912715' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: gabrioux-2024-09-20_12:24:58-orch:cephadm-wip-guits-main-2024-09-20-1028-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-guits-main-2024-09-20-1028 ceph: conf: client.0: debug ms: 1 debug objecter: 20 debug rados: 20 global: ms bind msgr1: false ms bind msgr2: true ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 paxos service trim min: 10 osd: debug ms: 1 debug osd: 20 osd debug reject backfill probability: 0.3 osd delete sleep: 1 osd max backfills: 3 osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true osd snap trim sleep: 2 flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - CEPHADM_STRAY_DAEMON - CEPHADM_FAILED_DAEMON - but it is still running - objects unfound and apparently lost - \(POOL_APP_NOT_ENABLED\) log-only-match: - CEPHADM_ sha1: 88deadb8082f4a33832e228ccefc59cedba158a5 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: cephadm_mode: root install: ceph: flavor: default sha1: 88deadb8082f4a33832e228ccefc59cedba158a5 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 workunit: branch: wip-guits-main-2024-09-20-1028 sha1: 88deadb8082f4a33832e228ccefc59cedba158a5 owner: scheduled_gabrioux@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 9945 sha1: 88deadb8082f4a33832e228ccefc59cedba158a5 sleep_before_teardown: 0 subset: 1/8 suite: orch:cephadm suite_branch: wip-guits-main-2024-09-20-1028 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_88deadb8082f4a33832e228ccefc59cedba158a5/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 88deadb8082f4a33832e228ccefc59cedba158a5 targets: smithi083.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO/Z8QJR1dTwxQaDoEyaLSQH/xisT+3W6oQhVJC9y94D72b4AbnVeNTKxWDCgWPNeAuxN0j3G+zzEfWkJjYs3Qo= smithi142.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFFrC07NemLu2CRO1t41whMTtSkmDO87ECsEOsJkU33Zx4ZnlvnxOlIMMbvrtvSl9K/hQ8GEAymOdHkOquQt3VA= tasks: - pexec: all: - sudo dnf remove nvme-cli -y - sudo dnf install nvmetcli nvme-cli -y - install: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - thrashosds: chance_pgnum_grow: 1 chance_pgnum_shrink: 1 chance_pgpnum_fix: 1 timeout: 1200 - full_sequential: - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-20_12:24:58 tube: smithi user: gabrioux verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2902 2024-09-20T18:32:50.761 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_88deadb8082f4a33832e228ccefc59cedba158a5/qa; will attempt to use it 2024-09-20T18:32:50.762 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_88deadb8082f4a33832e228ccefc59cedba158a5/qa/tasks 2024-09-20T18:32:50.762 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-20T18:32:50.763 INFO:teuthology.task.internal:Checking packages... 2024-09-20T18:32:50.784 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '88deadb8082f4a33832e228ccefc59cedba158a5' 2024-09-20T18:32:50.784 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-20T18:32:50.784 INFO:teuthology.packaging:ref: None 2024-09-20T18:32:50.784 INFO:teuthology.packaging:tag: None 2024-09-20T18:32:50.784 INFO:teuthology.packaging:branch: wip-guits-main-2024-09-20-1028 2024-09-20T18:32:50.784 INFO:teuthology.packaging:sha1: 88deadb8082f4a33832e228ccefc59cedba158a5 2024-09-20T18:32:50.785 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-guits-main-2024-09-20-1028 2024-09-20T18:32:50.928 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-5099.g88deadb8 2024-09-20T18:32:50.929 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-20T18:32:50.935 INFO:teuthology.task.internal:no buildpackages task found 2024-09-20T18:32:50.935 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-20T18:32:50.942 INFO:teuthology.task.internal:Saving configuration 2024-09-20T18:32:50.953 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-20T18:32:50.960 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-20T18:32:50.984 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi083.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-20_12:24:58-orch:cephadm-wip-guits-main-2024-09-20-1028-distro-default-smithi/7912715', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-20 18:27:53.089122', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO/Z8QJR1dTwxQaDoEyaLSQH/xisT+3W6oQhVJC9y94D72b4AbnVeNTKxWDCgWPNeAuxN0j3G+zzEfWkJjYs3Qo='} 2024-09-20T18:32:51.004 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi142.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-20_12:24:58-orch:cephadm-wip-guits-main-2024-09-20-1028-distro-default-smithi/7912715', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-20 18:27:53.087347', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFFrC07NemLu2CRO1t41whMTtSkmDO87ECsEOsJkU33Zx4ZnlvnxOlIMMbvrtvSl9K/hQ8GEAymOdHkOquQt3VA='} 2024-09-20T18:32:51.004 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-20T18:32:51.016 INFO:teuthology.task.internal:roles: ubuntu@smithi083.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-09-20T18:32:51.016 INFO:teuthology.task.internal:roles: ubuntu@smithi142.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-09-20T18:32:51.016 INFO:teuthology.run_tasks:Running task console_log... 2024-09-20T18:32:51.161 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f65aaab0280>, signals=[15]) 2024-09-20T18:32:51.161 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-20T18:32:51.166 INFO:teuthology.task.internal:Opening connections... 2024-09-20T18:32:51.167 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi083.front.sepia.ceph.com 2024-09-20T18:32:51.168 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-20T18:32:51.243 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi142.front.sepia.ceph.com 2024-09-20T18:32:51.243 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-20T18:32:51.314 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-20T18:32:51.321 DEBUG:teuthology.orchestra.run.smithi083:> uname -m 2024-09-20T18:32:51.338 INFO:teuthology.orchestra.run.smithi083.stdout:x86_64 2024-09-20T18:32:51.338 DEBUG:teuthology.orchestra.run.smithi083:> cat /etc/os-release 2024-09-20T18:32:51.395 INFO:teuthology.orchestra.run.smithi083.stdout:NAME="CentOS Stream" 2024-09-20T18:32:51.396 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION="9" 2024-09-20T18:32:51.396 INFO:teuthology.orchestra.run.smithi083.stdout:ID="centos" 2024-09-20T18:32:51.396 INFO:teuthology.orchestra.run.smithi083.stdout:ID_LIKE="rhel fedora" 2024-09-20T18:32:51.396 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION_ID="9" 2024-09-20T18:32:51.396 INFO:teuthology.orchestra.run.smithi083.stdout:PLATFORM_ID="platform:el9" 2024-09-20T18:32:51.396 INFO:teuthology.orchestra.run.smithi083.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-20T18:32:51.396 INFO:teuthology.orchestra.run.smithi083.stdout:ANSI_COLOR="0;31" 2024-09-20T18:32:51.396 INFO:teuthology.orchestra.run.smithi083.stdout:LOGO="fedora-logo-icon" 2024-09-20T18:32:51.396 INFO:teuthology.orchestra.run.smithi083.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-20T18:32:51.396 INFO:teuthology.orchestra.run.smithi083.stdout:HOME_URL="https://centos.org/" 2024-09-20T18:32:51.396 INFO:teuthology.orchestra.run.smithi083.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-20T18:32:51.396 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-20T18:32:51.396 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-20T18:32:51.397 INFO:teuthology.lock.ops:Updating smithi083.front.sepia.ceph.com on lock server 2024-09-20T18:32:51.420 DEBUG:teuthology.orchestra.run.smithi142:> uname -m 2024-09-20T18:32:51.438 INFO:teuthology.orchestra.run.smithi142.stdout:x86_64 2024-09-20T18:32:51.438 DEBUG:teuthology.orchestra.run.smithi142:> cat /etc/os-release 2024-09-20T18:32:51.495 INFO:teuthology.orchestra.run.smithi142.stdout:NAME="CentOS Stream" 2024-09-20T18:32:51.495 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION="9" 2024-09-20T18:32:51.495 INFO:teuthology.orchestra.run.smithi142.stdout:ID="centos" 2024-09-20T18:32:51.495 INFO:teuthology.orchestra.run.smithi142.stdout:ID_LIKE="rhel fedora" 2024-09-20T18:32:51.496 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION_ID="9" 2024-09-20T18:32:51.496 INFO:teuthology.orchestra.run.smithi142.stdout:PLATFORM_ID="platform:el9" 2024-09-20T18:32:51.496 INFO:teuthology.orchestra.run.smithi142.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-20T18:32:51.496 INFO:teuthology.orchestra.run.smithi142.stdout:ANSI_COLOR="0;31" 2024-09-20T18:32:51.496 INFO:teuthology.orchestra.run.smithi142.stdout:LOGO="fedora-logo-icon" 2024-09-20T18:32:51.496 INFO:teuthology.orchestra.run.smithi142.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-20T18:32:51.496 INFO:teuthology.orchestra.run.smithi142.stdout:HOME_URL="https://centos.org/" 2024-09-20T18:32:51.496 INFO:teuthology.orchestra.run.smithi142.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-20T18:32:51.496 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-20T18:32:51.496 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-20T18:32:51.496 INFO:teuthology.lock.ops:Updating smithi142.front.sepia.ceph.com on lock server 2024-09-20T18:32:51.520 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-20T18:32:51.529 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-20T18:32:51.557 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-20T18:32:51.557 DEBUG:teuthology.orchestra.run.smithi083:> test '!' -e /home/ubuntu/cephtest 2024-09-20T18:32:51.560 DEBUG:teuthology.orchestra.run.smithi142:> test '!' -e /home/ubuntu/cephtest 2024-09-20T18:32:51.576 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-20T18:32:51.582 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-20T18:32:51.582 DEBUG:teuthology.orchestra.run.smithi083:> test -z $(ls -A /var/lib/ceph) 2024-09-20T18:32:51.616 DEBUG:teuthology.orchestra.run.smithi142:> test -z $(ls -A /var/lib/ceph) 2024-09-20T18:32:51.672 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-20T18:32:51.726 INFO:teuthology.run_tasks:Running task kernel... 2024-09-20T18:32:51.741 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-20T18:32:51.741 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-20T18:32:51.741 DEBUG:teuthology.orchestra.run.smithi083:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-20T18:32:51.741 DEBUG:teuthology.orchestra.run.smithi142:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-20T18:32:51.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:32:51.757 DEBUG:teuthology.orchestra.run.smithi083:> uname -r 2024-09-20T18:32:51.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:32:51.758 DEBUG:teuthology.orchestra.run.smithi142:> uname -r 2024-09-20T18:32:51.813 INFO:teuthology.orchestra.run.smithi083.stdout:5.14.0-503.el9.x86_64 2024-09-20T18:32:51.813 INFO:teuthology.task.kernel:Running kernel on smithi083: 5.14.0-503.el9.x86_64 2024-09-20T18:32:51.813 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum install -y kernel 2024-09-20T18:32:51.814 INFO:teuthology.orchestra.run.smithi142.stdout:5.14.0-503.el9.x86_64 2024-09-20T18:32:51.815 INFO:teuthology.task.kernel:Running kernel on smithi142: 5.14.0-503.el9.x86_64 2024-09-20T18:32:51.815 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum install -y kernel 2024-09-20T18:32:55.184 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - BaseOS 4.8 MB/s | 8.3 MB 00:01 2024-09-20T18:32:55.307 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.3 MB 00:01 2024-09-20T18:32:57.133 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - AppStream 28 MB/s | 20 MB 00:00 2024-09-20T18:33:01.684 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - CRB 8.5 MB/s | 6.5 MB 00:00 2024-09-20T18:33:04.409 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - AppStream 2.5 MB/s | 20 MB 00:08 2024-09-20T18:33:05.030 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - Extras packages 8.9 kB/s | 19 kB 00:02 2024-09-20T18:33:06.182 INFO:teuthology.orchestra.run.smithi142.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-20T18:33:09.926 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - CRB 3.8 MB/s | 6.5 MB 00:01 2024-09-20T18:33:11.427 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - Extras packages 71 kB/s | 19 kB 00:00 2024-09-20T18:33:11.545 INFO:teuthology.orchestra.run.smithi142.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-09-20T18:33:12.240 INFO:teuthology.orchestra.run.smithi083.stdout:Extra Packages for Enterprise Linux 33 MB/s | 23 MB 00:00 2024-09-20T18:33:13.443 INFO:teuthology.orchestra.run.smithi142.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-20T18:33:13.443 INFO:teuthology.orchestra.run.smithi142.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-20T18:33:13.536 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-09-20T18:33:13.548 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:33:13.548 INFO:teuthology.orchestra.run.smithi142.stdout: Package Architecture Version Repository Size 2024-09-20T18:33:13.549 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:33:13.549 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-09-20T18:33:13.549 INFO:teuthology.orchestra.run.smithi142.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-20T18:33:13.549 INFO:teuthology.orchestra.run.smithi142.stdout:Installing dependencies: 2024-09-20T18:33:13.549 INFO:teuthology.orchestra.run.smithi142.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-20T18:33:13.549 INFO:teuthology.orchestra.run.smithi142.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-20T18:33:13.549 INFO:teuthology.orchestra.run.smithi142.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-20T18:33:13.549 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:33:13.549 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-09-20T18:33:13.549 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:33:13.549 INFO:teuthology.orchestra.run.smithi142.stdout:Install 4 Packages 2024-09-20T18:33:13.549 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:33:13.550 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 79 M 2024-09-20T18:33:13.550 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 126 M 2024-09-20T18:33:13.550 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-09-20T18:33:13.871 INFO:teuthology.orchestra.run.smithi142.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 260 kB/s | 36 kB 00:00 2024-09-20T18:33:17.229 INFO:teuthology.orchestra.run.smithi142.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 4.5 MB/s | 16 MB 00:03 2024-09-20T18:33:17.654 INFO:teuthology.orchestra.run.smithi083.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-20T18:33:19.088 INFO:teuthology.orchestra.run.smithi142.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 5.5 MB/s | 29 MB 00:05 2024-09-20T18:33:19.560 INFO:teuthology.orchestra.run.smithi083.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-20T18:33:19.560 INFO:teuthology.orchestra.run.smithi083.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-20T18:33:19.654 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-20T18:33:19.670 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:33:19.670 INFO:teuthology.orchestra.run.smithi083.stdout: Package Architecture Version Repository Size 2024-09-20T18:33:19.670 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:33:19.670 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2024-09-20T18:33:19.670 INFO:teuthology.orchestra.run.smithi083.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-20T18:33:19.670 INFO:teuthology.orchestra.run.smithi083.stdout:Installing dependencies: 2024-09-20T18:33:19.670 INFO:teuthology.orchestra.run.smithi083.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-20T18:33:19.670 INFO:teuthology.orchestra.run.smithi083.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-20T18:33:19.670 INFO:teuthology.orchestra.run.smithi083.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-20T18:33:19.671 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:33:19.671 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-09-20T18:33:19.671 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:33:19.671 INFO:teuthology.orchestra.run.smithi083.stdout:Install 4 Packages 2024-09-20T18:33:19.671 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:33:19.671 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 79 M 2024-09-20T18:33:19.671 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 126 M 2024-09-20T18:33:19.671 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-09-20T18:33:20.039 INFO:teuthology.orchestra.run.smithi083.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 402 kB/s | 36 kB 00:00 2024-09-20T18:33:22.595 INFO:teuthology.orchestra.run.smithi142.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 3.9 MB/s | 35 MB 00:08 2024-09-20T18:33:22.596 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-09-20T18:33:22.596 INFO:teuthology.orchestra.run.smithi142.stdout:Total 8.7 MB/s | 79 MB 00:09 2024-09-20T18:33:23.284 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-09-20T18:33:23.637 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-09-20T18:33:23.637 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-09-20T18:33:24.080 INFO:teuthology.orchestra.run.smithi083.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 3.8 MB/s | 16 MB 00:04 2024-09-20T18:33:24.652 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-09-20T18:33:24.652 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-09-20T18:33:27.101 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-09-20T18:33:27.505 INFO:teuthology.orchestra.run.smithi083.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 3.8 MB/s | 29 MB 00:07 2024-09-20T18:33:27.807 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-20T18:33:27.895 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-20T18:33:29.356 INFO:teuthology.orchestra.run.smithi083.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 3.7 MB/s | 35 MB 00:09 2024-09-20T18:33:29.356 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-09-20T18:33:29.356 INFO:teuthology.orchestra.run.smithi083.stdout:Total 8.1 MB/s | 79 MB 00:09 2024-09-20T18:33:30.042 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-09-20T18:33:30.391 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-09-20T18:33:30.391 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-09-20T18:33:30.656 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-20T18:33:30.787 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-20T18:33:31.355 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-09-20T18:33:31.355 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-09-20T18:33:33.802 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-09-20T18:33:34.465 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-20T18:33:34.559 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-20T18:33:35.624 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-20T18:33:35.745 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-20T18:33:37.304 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-20T18:33:37.436 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-20T18:33:40.560 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-20T18:33:42.251 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-20T18:33:42.378 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-20T18:33:47.235 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-20T18:34:18.929 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-20T18:34:18.968 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-20T18:34:19.247 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-20T18:34:19.247 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-20T18:34:19.247 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-20T18:34:19.247 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-20T18:34:20.071 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-20T18:34:20.071 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:34:20.071 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-09-20T18:34:20.071 INFO:teuthology.orchestra.run.smithi142.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-20T18:34:20.072 INFO:teuthology.orchestra.run.smithi142.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-20T18:34:20.072 INFO:teuthology.orchestra.run.smithi142.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-20T18:34:20.072 INFO:teuthology.orchestra.run.smithi142.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-20T18:34:20.072 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:34:20.072 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-09-20T18:34:20.408 DEBUG:teuthology.orchestra.run.smithi142:> rpm -q kernel | sort -rV | head -n 1 2024-09-20T18:34:20.496 INFO:teuthology.orchestra.run.smithi142.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-20T18:34:20.497 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-20T18:34:20.497 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-20T18:34:20.497 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-20T18:34:20.497 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-20T18:34:20.497 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-20T18:34:20.497 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi142.front.sepia.ceph.com, path=None, version=distro) 2024-09-20T18:34:20.498 DEBUG:teuthology.orchestra.run.smithi142:> rpm -q kernel | sort -rV | head -n 1 2024-09-20T18:34:20.556 INFO:teuthology.orchestra.run.smithi142.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-20T18:34:20.556 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-20T18:34:20.556 DEBUG:teuthology.orchestra.run.smithi142:> sudo rpm -qi grub2-tools 2024-09-20T18:34:20.588 INFO:teuthology.orchestra.run.smithi142.stdout:Name : grub2-tools 2024-09-20T18:34:20.588 INFO:teuthology.orchestra.run.smithi142.stdout:Epoch : 1 2024-09-20T18:34:20.588 INFO:teuthology.orchestra.run.smithi142.stdout:Version : 2.06 2024-09-20T18:34:20.588 INFO:teuthology.orchestra.run.smithi142.stdout:Release : 82.el9 2024-09-20T18:34:20.588 INFO:teuthology.orchestra.run.smithi142.stdout:Architecture: x86_64 2024-09-20T18:34:20.588 INFO:teuthology.orchestra.run.smithi142.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-20T18:34:20.588 INFO:teuthology.orchestra.run.smithi142.stdout:Group : Unspecified 2024-09-20T18:34:20.588 INFO:teuthology.orchestra.run.smithi142.stdout:Size : 8274670 2024-09-20T18:34:20.588 INFO:teuthology.orchestra.run.smithi142.stdout:License : GPLv3+ 2024-09-20T18:34:20.588 INFO:teuthology.orchestra.run.smithi142.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-20T18:34:20.589 INFO:teuthology.orchestra.run.smithi142.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-20T18:34:20.589 INFO:teuthology.orchestra.run.smithi142.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-20T18:34:20.589 INFO:teuthology.orchestra.run.smithi142.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-20T18:34:20.589 INFO:teuthology.orchestra.run.smithi142.stdout:Packager : builder@centos.org 2024-09-20T18:34:20.589 INFO:teuthology.orchestra.run.smithi142.stdout:Vendor : CentOS 2024-09-20T18:34:20.589 INFO:teuthology.orchestra.run.smithi142.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-20T18:34:20.589 INFO:teuthology.orchestra.run.smithi142.stdout:Summary : Support tools for GRUB. 2024-09-20T18:34:20.589 INFO:teuthology.orchestra.run.smithi142.stdout:Description : 2024-09-20T18:34:20.589 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:34:20.589 INFO:teuthology.orchestra.run.smithi142.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-20T18:34:20.589 INFO:teuthology.orchestra.run.smithi142.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-20T18:34:20.589 INFO:teuthology.orchestra.run.smithi142.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-20T18:34:20.589 INFO:teuthology.orchestra.run.smithi142.stdout:hardware devices. 2024-09-20T18:34:20.589 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:34:20.589 INFO:teuthology.orchestra.run.smithi142.stdout:This subpackage provides tools for support of all platforms. 2024-09-20T18:34:20.590 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-20T18:34:20.590 INFO:teuthology.task.kernel:Updating grub on smithi142 to boot 5.14.0-508.el9.x86_64 2024-09-20T18:34:20.590 DEBUG:teuthology.orchestra.run.smithi142:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-20T18:34:21.276 INFO:teuthology.orchestra.run.smithi142.stderr:Generating grub configuration file ... 2024-09-20T18:34:22.651 INFO:teuthology.orchestra.run.smithi142.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-20T18:34:22.676 INFO:teuthology.orchestra.run.smithi142.stderr:done 2024-09-20T18:34:22.678 DEBUG:teuthology.orchestra.run.smithi142:> mktemp 2024-09-20T18:34:22.693 INFO:teuthology.orchestra.run.smithi142.stdout:/tmp/tmp.wV9aBqvI6R 2024-09-20T18:34:22.694 DEBUG:teuthology.orchestra.run.smithi142:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.wV9aBqvI6R 2024-09-20T18:34:22.759 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 0666 /tmp/tmp.wV9aBqvI6R 2024-09-20T18:34:22.903 DEBUG:teuthology.orchestra.remote:smithi142:/tmp/tmp.wV9aBqvI6R is 6KB 2024-09-20T18:34:22.924 DEBUG:teuthology.orchestra.run.smithi142:> rm -fr /tmp/tmp.wV9aBqvI6R 2024-09-20T18:34:22.939 DEBUG:teuthology.orchestra.run.smithi142:> sudo /bin/ls /boot/loader/entries || true 2024-09-20T18:34:23.006 INFO:teuthology.orchestra.run.smithi142.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-20T18:34:23.006 INFO:teuthology.orchestra.run.smithi142.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-20T18:34:23.006 INFO:teuthology.orchestra.run.smithi142.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-20T18:34:23.007 INFO:teuthology.orchestra.run.smithi142.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-20T18:34:23.008 DEBUG:teuthology.orchestra.run.smithi142:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-20T18:34:23.161 DEBUG:teuthology.orchestra.run.smithi142:> sudo shutdown -r now 2024-09-20T18:34:25.162 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-20T18:34:25.207 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-20T18:34:25.580 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-20T18:34:25.580 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-20T18:34:25.580 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-20T18:34:25.580 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-20T18:34:26.423 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-20T18:34:26.423 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:34:26.423 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2024-09-20T18:34:26.423 INFO:teuthology.orchestra.run.smithi083.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-20T18:34:26.424 INFO:teuthology.orchestra.run.smithi083.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-20T18:34:26.424 INFO:teuthology.orchestra.run.smithi083.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-20T18:34:26.424 INFO:teuthology.orchestra.run.smithi083.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-20T18:34:26.424 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:34:26.424 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-20T18:34:26.778 DEBUG:teuthology.orchestra.run.smithi083:> rpm -q kernel | sort -rV | head -n 1 2024-09-20T18:34:26.860 INFO:teuthology.orchestra.run.smithi083.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-20T18:34:26.861 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-20T18:34:26.861 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-20T18:34:26.861 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-20T18:34:26.861 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-20T18:34:26.861 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-20T18:34:26.861 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi083.front.sepia.ceph.com, path=None, version=distro) 2024-09-20T18:34:26.862 DEBUG:teuthology.orchestra.run.smithi083:> rpm -q kernel | sort -rV | head -n 1 2024-09-20T18:34:26.920 INFO:teuthology.orchestra.run.smithi083.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-20T18:34:26.921 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-20T18:34:26.921 DEBUG:teuthology.orchestra.run.smithi083:> sudo rpm -qi grub2-tools 2024-09-20T18:34:26.959 INFO:teuthology.orchestra.run.smithi083.stdout:Name : grub2-tools 2024-09-20T18:34:26.959 INFO:teuthology.orchestra.run.smithi083.stdout:Epoch : 1 2024-09-20T18:34:26.959 INFO:teuthology.orchestra.run.smithi083.stdout:Version : 2.06 2024-09-20T18:34:26.959 INFO:teuthology.orchestra.run.smithi083.stdout:Release : 82.el9 2024-09-20T18:34:26.959 INFO:teuthology.orchestra.run.smithi083.stdout:Architecture: x86_64 2024-09-20T18:34:26.959 INFO:teuthology.orchestra.run.smithi083.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-20T18:34:26.959 INFO:teuthology.orchestra.run.smithi083.stdout:Group : Unspecified 2024-09-20T18:34:26.959 INFO:teuthology.orchestra.run.smithi083.stdout:Size : 8274670 2024-09-20T18:34:26.959 INFO:teuthology.orchestra.run.smithi083.stdout:License : GPLv3+ 2024-09-20T18:34:26.959 INFO:teuthology.orchestra.run.smithi083.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-20T18:34:26.960 INFO:teuthology.orchestra.run.smithi083.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-20T18:34:26.960 INFO:teuthology.orchestra.run.smithi083.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-20T18:34:26.960 INFO:teuthology.orchestra.run.smithi083.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-20T18:34:26.960 INFO:teuthology.orchestra.run.smithi083.stdout:Packager : builder@centos.org 2024-09-20T18:34:26.960 INFO:teuthology.orchestra.run.smithi083.stdout:Vendor : CentOS 2024-09-20T18:34:26.960 INFO:teuthology.orchestra.run.smithi083.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-20T18:34:26.960 INFO:teuthology.orchestra.run.smithi083.stdout:Summary : Support tools for GRUB. 2024-09-20T18:34:26.960 INFO:teuthology.orchestra.run.smithi083.stdout:Description : 2024-09-20T18:34:26.960 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:34:26.960 INFO:teuthology.orchestra.run.smithi083.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-20T18:34:26.960 INFO:teuthology.orchestra.run.smithi083.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-20T18:34:26.960 INFO:teuthology.orchestra.run.smithi083.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-20T18:34:26.960 INFO:teuthology.orchestra.run.smithi083.stdout:hardware devices. 2024-09-20T18:34:26.960 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:34:26.960 INFO:teuthology.orchestra.run.smithi083.stdout:This subpackage provides tools for support of all platforms. 2024-09-20T18:34:26.961 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-20T18:34:26.961 INFO:teuthology.task.kernel:Updating grub on smithi083 to boot 5.14.0-508.el9.x86_64 2024-09-20T18:34:26.961 DEBUG:teuthology.orchestra.run.smithi083:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-20T18:34:27.599 INFO:teuthology.orchestra.run.smithi083.stderr:Generating grub configuration file ... 2024-09-20T18:34:28.925 INFO:teuthology.orchestra.run.smithi083.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-20T18:34:28.984 INFO:teuthology.orchestra.run.smithi083.stderr:done 2024-09-20T18:34:28.986 DEBUG:teuthology.orchestra.run.smithi083:> mktemp 2024-09-20T18:34:29.001 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/tmp.aFLOzi1hIQ 2024-09-20T18:34:29.001 DEBUG:teuthology.orchestra.run.smithi083:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.aFLOzi1hIQ 2024-09-20T18:34:29.068 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0666 /tmp/tmp.aFLOzi1hIQ 2024-09-20T18:34:29.211 DEBUG:teuthology.orchestra.remote:smithi083:/tmp/tmp.aFLOzi1hIQ is 6KB 2024-09-20T18:34:29.260 DEBUG:teuthology.orchestra.run.smithi083:> rm -fr /tmp/tmp.aFLOzi1hIQ 2024-09-20T18:34:29.274 DEBUG:teuthology.orchestra.run.smithi083:> sudo /bin/ls /boot/loader/entries || true 2024-09-20T18:34:29.337 INFO:teuthology.orchestra.run.smithi083.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-20T18:34:29.337 INFO:teuthology.orchestra.run.smithi083.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-20T18:34:29.337 INFO:teuthology.orchestra.run.smithi083.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-20T18:34:29.337 INFO:teuthology.orchestra.run.smithi083.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-20T18:34:29.339 DEBUG:teuthology.orchestra.run.smithi083:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-20T18:34:29.496 DEBUG:teuthology.orchestra.run.smithi083:> sudo shutdown -r now 2024-09-20T18:34:53.185 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-20T18:34:53.185 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-09-20T18:34:53.186 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-20T18:34:59.503 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-20T18:34:59.503 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-09-20T18:34:59.504 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-20T18:35:11.727 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.142 2024-09-20T18:35:17.871 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.83 2024-09-20T18:35:20.731 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-09-20T18:35:20.732 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-20T18:35:23.791 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.142 2024-09-20T18:35:26.874 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-09-20T18:35:26.875 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-20T18:35:29.935 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.83 2024-09-20T18:35:35.792 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-09-20T18:35:35.793 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-20T18:35:41.941 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-09-20T18:35:41.941 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-20T18:35:54.223 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.142 2024-09-20T18:36:00.367 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.83 2024-09-20T18:36:09.230 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-09-20T18:36:09.231 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-20T18:36:09.524 DEBUG:teuthology.orchestra.run.smithi142:> true 2024-09-20T18:36:09.863 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-09-20T18:36:09.863 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-508.el9.x86_64"... 2024-09-20T18:36:09.863 DEBUG:teuthology.orchestra.run.smithi142:> uname -r 2024-09-20T18:36:09.878 INFO:teuthology.orchestra.run.smithi142.stdout:5.14.0-508.el9.x86_64 2024-09-20T18:36:09.878 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-20T18:36:09.878 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-20T18:36:09.878 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-20T18:36:10.879 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-20T18:36:10.880 DEBUG:teuthology.orchestra.run.smithi142:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-20T18:36:10.999 INFO:teuthology.orchestra.run.smithi142.stdout:ttyS1 2024-09-20T18:36:11.022 DEBUG:teuthology.parallel:result is None 2024-09-20T18:36:15.368 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-09-20T18:36:15.368 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-20T18:36:16.026 DEBUG:teuthology.orchestra.run.smithi083:> true 2024-09-20T18:36:16.579 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-09-20T18:36:16.579 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-508.el9.x86_64"... 2024-09-20T18:36:16.580 DEBUG:teuthology.orchestra.run.smithi083:> uname -r 2024-09-20T18:36:16.620 INFO:teuthology.orchestra.run.smithi083.stdout:5.14.0-508.el9.x86_64 2024-09-20T18:36:16.621 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-20T18:36:16.621 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-20T18:36:16.621 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-20T18:36:17.621 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-20T18:36:17.622 DEBUG:teuthology.orchestra.run.smithi083:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-20T18:36:17.803 INFO:teuthology.orchestra.run.smithi083.stdout:ttyS1 2024-09-20T18:36:18.020 DEBUG:teuthology.parallel:result is None 2024-09-20T18:36:18.020 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-20T18:36:18.035 INFO:teuthology.task.internal:Creating test directory... 2024-09-20T18:36:18.035 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-20T18:36:18.037 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-20T18:36:18.056 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-20T18:36:18.062 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-20T18:36:18.070 INFO:teuthology.task.internal:Creating archive directory... 2024-09-20T18:36:18.071 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-20T18:36:18.093 DEBUG:teuthology.orchestra.run.smithi142:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-20T18:36:18.130 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-20T18:36:18.137 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-20T18:36:18.137 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-20T18:36:18.157 DEBUG:teuthology.orchestra.run.smithi142:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-20T18:36:18.182 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-20T18:36:18.192 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-20T18:36:18.195 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-20T18:36:18.207 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-20T18:36:18.208 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-20T18:36:18.215 INFO:teuthology.task.internal:Configuring sudo... 2024-09-20T18:36:18.215 DEBUG:teuthology.orchestra.run.smithi083:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-20T18:36:18.237 DEBUG:teuthology.orchestra.run.smithi142:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-20T18:36:18.275 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-20T18:36:18.284 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-20T18:36:18.284 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-20T18:36:18.302 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-20T18:36:18.333 DEBUG:teuthology.orchestra.run.smithi083:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-20T18:36:18.391 DEBUG:teuthology.orchestra.run.smithi083:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-20T18:36:18.449 DEBUG:teuthology.orchestra.run.smithi083:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-20T18:36:18.530 DEBUG:teuthology.orchestra.run.smithi083:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-20T18:36:18.570 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-20T18:36:18.571 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-20T18:36:18.635 DEBUG:teuthology.orchestra.run.smithi142:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-20T18:36:18.680 DEBUG:teuthology.orchestra.run.smithi142:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-20T18:36:18.733 DEBUG:teuthology.orchestra.run.smithi142:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-20T18:36:18.814 DEBUG:teuthology.orchestra.run.smithi142:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-20T18:36:18.858 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-20T18:36:18.858 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-20T18:36:18.924 DEBUG:teuthology.orchestra.run.smithi083:> sudo service rsyslog restart 2024-09-20T18:36:18.927 DEBUG:teuthology.orchestra.run.smithi142:> sudo service rsyslog restart 2024-09-20T18:36:18.964 INFO:teuthology.orchestra.run.smithi083.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-20T18:36:19.010 INFO:teuthology.orchestra.run.smithi142.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-20T18:36:19.525 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-20T18:36:19.537 INFO:teuthology.task.internal:Starting timer... 2024-09-20T18:36:19.537 INFO:teuthology.run_tasks:Running task pcp... 2024-09-20T18:36:19.589 INFO:teuthology.run_tasks:Running task selinux... 2024-09-20T18:36:19.598 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-20T18:36:19.598 DEBUG:teuthology.orchestra.run.smithi083:> sudo service auditd rotate 2024-09-20T18:36:19.675 INFO:teuthology.orchestra.run.smithi083.stdout:Rotating logs: 2024-09-20T18:36:19.676 DEBUG:teuthology.orchestra.run.smithi142:> sudo service auditd rotate 2024-09-20T18:36:19.716 INFO:teuthology.orchestra.run.smithi142.stdout:Rotating logs: 2024-09-20T18:36:19.718 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-20T18:36:19.718 DEBUG:teuthology.orchestra.run.smithi083:> /usr/sbin/getenforce 2024-09-20T18:36:19.753 INFO:teuthology.orchestra.run.smithi083.stdout:Permissive 2024-09-20T18:36:19.753 DEBUG:teuthology.orchestra.run.smithi142:> /usr/sbin/getenforce 2024-09-20T18:36:19.789 INFO:teuthology.orchestra.run.smithi142.stdout:Permissive 2024-09-20T18:36:19.789 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi083.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi142.front.sepia.ceph.com': 'permissive'} 2024-09-20T18:36:19.790 DEBUG:teuthology.orchestra.run.smithi083:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-20T18:36:19.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:36:19.823 DEBUG:teuthology.orchestra.run.smithi142:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-20T18:36:19.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:36:19.855 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-20T18:36:19.855 DEBUG:teuthology.orchestra.run.smithi083:> sudo /usr/sbin/setenforce permissive 2024-09-20T18:36:19.895 DEBUG:teuthology.orchestra.run.smithi142:> sudo /usr/sbin/setenforce permissive 2024-09-20T18:36:19.930 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-20T18:36:19.939 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-20T18:36:19.942 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-20T18:36:20.048 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-20T18:36:20.062 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-20T18:36:20.063 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi083.front.sepia.ceph.com,smithi142.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-20T18:40:03.676 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi083.front.sepia.ceph.com'), Remote(name='ubuntu@smithi142.front.sepia.ceph.com')] 2024-09-20T18:40:03.677 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-09-20T18:40:03.678 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-20T18:40:03.756 DEBUG:teuthology.orchestra.run.smithi083:> true 2024-09-20T18:40:03.829 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-09-20T18:40:03.829 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-09-20T18:40:03.830 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-20T18:40:03.902 DEBUG:teuthology.orchestra.run.smithi142:> true 2024-09-20T18:40:03.985 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-09-20T18:40:03.985 INFO:teuthology.run_tasks:Running task clock... 2024-09-20T18:40:03.995 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-20T18:40:03.995 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-20T18:40:03.996 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-20T18:40:03.998 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-20T18:40:03.998 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-20T18:40:04.024 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-20T18:40:04.038 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-20T18:40:04.055 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-20T18:40:04.062 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: ntpd: command not found 2024-09-20T18:40:04.069 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-20T18:40:04.074 INFO:teuthology.orchestra.run.smithi083.stdout:506 Cannot talk to daemon 2024-09-20T18:40:04.089 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-20T18:40:04.094 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: ntpd: command not found 2024-09-20T18:40:04.103 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-20T18:40:04.105 INFO:teuthology.orchestra.run.smithi142.stdout:506 Cannot talk to daemon 2024-09-20T18:40:04.118 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-20T18:40:04.132 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-20T18:40:04.163 INFO:teuthology.orchestra.run.smithi083.stderr:bash: line 1: ntpq: command not found 2024-09-20T18:40:04.166 INFO:teuthology.orchestra.run.smithi083.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-20T18:40:04.167 INFO:teuthology.orchestra.run.smithi083.stdout:=============================================================================== 2024-09-20T18:40:04.167 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-20T18:40:04.167 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-20T18:40:04.167 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-20T18:40:04.167 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-20T18:40:04.188 INFO:teuthology.orchestra.run.smithi142.stderr:bash: line 1: ntpq: command not found 2024-09-20T18:40:04.191 INFO:teuthology.orchestra.run.smithi142.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-20T18:40:04.191 INFO:teuthology.orchestra.run.smithi142.stdout:=============================================================================== 2024-09-20T18:40:04.191 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-20T18:40:04.191 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-20T18:40:04.191 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-20T18:40:04.191 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-20T18:40:04.192 INFO:teuthology.run_tasks:Running task pexec... 2024-09-20T18:40:04.201 INFO:teuthology.task.pexec:Executing custom commands... 2024-09-20T18:40:04.202 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi083.front.sepia.ceph.com 2024-09-20T18:40:04.202 DEBUG:teuthology.orchestra.run.smithi083:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-20T18:40:04.202 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi142.front.sepia.ceph.com 2024-09-20T18:40:04.202 DEBUG:teuthology.orchestra.run.smithi142:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-20T18:40:04.555 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-20T18:40:04.558 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:40:04.558 INFO:teuthology.orchestra.run.smithi083.stdout: Package Architecture Version Repository Size 2024-09-20T18:40:04.558 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:40:04.558 INFO:teuthology.orchestra.run.smithi083.stdout:Removing: 2024-09-20T18:40:04.558 INFO:teuthology.orchestra.run.smithi083.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-20T18:40:04.558 INFO:teuthology.orchestra.run.smithi083.stdout:Removing unused dependencies: 2024-09-20T18:40:04.558 INFO:teuthology.orchestra.run.smithi083.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-20T18:40:04.558 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:40:04.558 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-09-20T18:40:04.558 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:40:04.558 INFO:teuthology.orchestra.run.smithi083.stdout:Remove 2 Packages 2024-09-20T18:40:04.559 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:40:04.559 INFO:teuthology.orchestra.run.smithi083.stdout:Freed space: 5.5 M 2024-09-20T18:40:04.559 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-09-20T18:40:04.562 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-09-20T18:40:04.562 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-09-20T18:40:04.582 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-09-20T18:40:04.585 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:40:04.585 INFO:teuthology.orchestra.run.smithi142.stdout: Package Architecture Version Repository Size 2024-09-20T18:40:04.585 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:40:04.585 INFO:teuthology.orchestra.run.smithi142.stdout:Removing: 2024-09-20T18:40:04.585 INFO:teuthology.orchestra.run.smithi142.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-20T18:40:04.585 INFO:teuthology.orchestra.run.smithi142.stdout:Removing unused dependencies: 2024-09-20T18:40:04.585 INFO:teuthology.orchestra.run.smithi142.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-20T18:40:04.585 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:40:04.585 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-09-20T18:40:04.585 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:40:04.585 INFO:teuthology.orchestra.run.smithi142.stdout:Remove 2 Packages 2024-09-20T18:40:04.585 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:40:04.585 INFO:teuthology.orchestra.run.smithi142.stdout:Freed space: 5.5 M 2024-09-20T18:40:04.586 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-09-20T18:40:04.589 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-09-20T18:40:04.589 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-09-20T18:40:04.628 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-09-20T18:40:04.628 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-09-20T18:40:04.648 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-09-20T18:40:04.648 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-09-20T18:40:04.768 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-09-20T18:40:04.789 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-09-20T18:40:04.847 INFO:teuthology.orchestra.run.smithi142.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-20T18:40:04.862 INFO:teuthology.orchestra.run.smithi083.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-20T18:40:04.903 INFO:teuthology.orchestra.run.smithi142.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-20T18:40:04.919 INFO:teuthology.orchestra.run.smithi083.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-20T18:40:05.400 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-20T18:40:05.401 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-20T18:40:05.432 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-20T18:40:05.432 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-20T18:40:05.646 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-20T18:40:05.646 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:40:05.646 INFO:teuthology.orchestra.run.smithi142.stdout:Removed: 2024-09-20T18:40:05.646 INFO:teuthology.orchestra.run.smithi142.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-20T18:40:05.646 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:40:05.646 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-09-20T18:40:05.687 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-20T18:40:05.688 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:40:05.688 INFO:teuthology.orchestra.run.smithi083.stdout:Removed: 2024-09-20T18:40:05.688 INFO:teuthology.orchestra.run.smithi083.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-20T18:40:05.688 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:40:05.688 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-20T18:40:06.515 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:00:48 ago on Fri 20 Sep 2024 06:39:18 PM UTC. 2024-09-20T18:40:06.529 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:00:47 ago on Fri 20 Sep 2024 06:39:19 PM UTC. 2024-09-20T18:40:06.646 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-20T18:40:06.647 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:40:06.647 INFO:teuthology.orchestra.run.smithi083.stdout: Package Architecture Version Repository Size 2024-09-20T18:40:06.647 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:40:06.647 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2024-09-20T18:40:06.647 INFO:teuthology.orchestra.run.smithi083.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-20T18:40:06.647 INFO:teuthology.orchestra.run.smithi083.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-20T18:40:06.647 INFO:teuthology.orchestra.run.smithi083.stdout:Installing dependencies: 2024-09-20T18:40:06.647 INFO:teuthology.orchestra.run.smithi083.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-20T18:40:06.648 INFO:teuthology.orchestra.run.smithi083.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-20T18:40:06.648 INFO:teuthology.orchestra.run.smithi083.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-20T18:40:06.648 INFO:teuthology.orchestra.run.smithi083.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-20T18:40:06.648 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:40:06.648 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-09-20T18:40:06.648 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:40:06.648 INFO:teuthology.orchestra.run.smithi083.stdout:Install 6 Packages 2024-09-20T18:40:06.648 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:40:06.655 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 2.0 M 2024-09-20T18:40:06.655 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 9.2 M 2024-09-20T18:40:06.655 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-09-20T18:40:06.663 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-09-20T18:40:06.664 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:40:06.664 INFO:teuthology.orchestra.run.smithi142.stdout: Package Architecture Version Repository Size 2024-09-20T18:40:06.664 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:40:06.664 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-09-20T18:40:06.664 INFO:teuthology.orchestra.run.smithi142.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-20T18:40:06.665 INFO:teuthology.orchestra.run.smithi142.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-20T18:40:06.665 INFO:teuthology.orchestra.run.smithi142.stdout:Installing dependencies: 2024-09-20T18:40:06.665 INFO:teuthology.orchestra.run.smithi142.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-20T18:40:06.665 INFO:teuthology.orchestra.run.smithi142.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-20T18:40:06.665 INFO:teuthology.orchestra.run.smithi142.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-20T18:40:06.665 INFO:teuthology.orchestra.run.smithi142.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-20T18:40:06.665 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:40:06.665 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-09-20T18:40:06.665 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:40:06.665 INFO:teuthology.orchestra.run.smithi142.stdout:Install 6 Packages 2024-09-20T18:40:06.665 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:40:06.672 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 2.0 M 2024-09-20T18:40:06.672 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 9.2 M 2024-09-20T18:40:06.672 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-09-20T18:40:06.909 INFO:teuthology.orchestra.run.smithi083.stdout:(1/6): nvmetcli-0.7-3.el9.noarch.rpm 385 kB/s | 44 kB 00:00 2024-09-20T18:40:06.951 INFO:teuthology.orchestra.run.smithi083.stdout:(2/6): libnvme-1.9-3.el9.x86_64.rpm 673 kB/s | 106 kB 00:00 2024-09-20T18:40:06.984 INFO:teuthology.orchestra.run.smithi083.stdout:(3/6): nvme-cli-2.9.1-6.el9.x86_64.rpm 4.6 MB/s | 906 kB 00:00 2024-09-20T18:40:07.010 INFO:teuthology.orchestra.run.smithi083.stdout:(4/6): python3-configshell-1.1.30-1.el9.noarch. 712 kB/s | 72 kB 00:00 2024-09-20T18:40:07.043 INFO:teuthology.orchestra.run.smithi083.stdout:(5/6): python3-kmod-0.9-32.el9.x86_64.rpm 912 kB/s | 84 kB 00:00 2024-09-20T18:40:07.076 INFO:teuthology.orchestra.run.smithi142.stdout:(1/6): nvmetcli-0.7-3.el9.noarch.rpm 363 kB/s | 44 kB 00:00 2024-09-20T18:40:07.085 INFO:teuthology.orchestra.run.smithi083.stdout:(6/6): python3-urwid-2.1.2-4.el9.x86_64.rpm 8.1 MB/s | 837 kB 00:00 2024-09-20T18:40:07.085 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-09-20T18:40:07.086 INFO:teuthology.orchestra.run.smithi083.stdout:Total 4.6 MB/s | 2.0 MB 00:00 2024-09-20T18:40:07.109 INFO:teuthology.orchestra.run.smithi142.stdout:(2/6): libnvme-1.9-3.el9.x86_64.rpm 680 kB/s | 106 kB 00:00 2024-09-20T18:40:07.151 INFO:teuthology.orchestra.run.smithi142.stdout:(3/6): python3-configshell-1.1.30-1.el9.noarch. 953 kB/s | 72 kB 00:00 2024-09-20T18:40:07.160 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-09-20T18:40:07.173 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-09-20T18:40:07.174 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-09-20T18:40:07.218 INFO:teuthology.orchestra.run.smithi142.stdout:(4/6): nvme-cli-2.9.1-6.el9.x86_64.rpm 3.3 MB/s | 906 kB 00:00 2024-09-20T18:40:07.243 INFO:teuthology.orchestra.run.smithi142.stdout:(5/6): python3-kmod-0.9-32.el9.x86_64.rpm 630 kB/s | 84 kB 00:00 2024-09-20T18:40:07.294 INFO:teuthology.orchestra.run.smithi142.stdout:(6/6): python3-urwid-2.1.2-4.el9.x86_64.rpm 5.8 MB/s | 837 kB 00:00 2024-09-20T18:40:07.294 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-09-20T18:40:07.294 INFO:teuthology.orchestra.run.smithi142.stdout:Total 3.2 MB/s | 2.0 MB 00:00 2024-09-20T18:40:07.367 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-09-20T18:40:07.381 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-09-20T18:40:07.381 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-09-20T18:40:07.465 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-09-20T18:40:07.466 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-09-20T18:40:07.656 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-09-20T18:40:07.656 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-09-20T18:40:07.897 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-09-20T18:40:07.988 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/6 2024-09-20T18:40:08.022 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/6 2024-09-20T18:40:08.051 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/6 2024-09-20T18:40:08.073 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-09-20T18:40:08.163 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/6 2024-09-20T18:40:08.205 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/6 2024-09-20T18:40:08.234 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/6 2024-09-20T18:40:08.258 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/6 2024-09-20T18:40:08.299 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-20T18:40:08.438 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/6 2024-09-20T18:40:08.465 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-20T18:40:08.741 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-20T18:40:08.768 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/6 2024-09-20T18:40:08.915 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-20T18:40:08.955 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/6 2024-09-20T18:40:09.710 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/6 2024-09-20T18:40:09.710 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/6 2024-09-20T18:40:09.710 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/6 2024-09-20T18:40:09.710 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/6 2024-09-20T18:40:09.710 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/6 2024-09-20T18:40:09.710 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/6 2024-09-20T18:40:09.937 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/6 2024-09-20T18:40:09.937 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/6 2024-09-20T18:40:09.938 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/6 2024-09-20T18:40:09.938 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/6 2024-09-20T18:40:09.938 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/6 2024-09-20T18:40:09.938 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/6 2024-09-20T18:40:10.052 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/6 2024-09-20T18:40:10.052 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:40:10.052 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2024-09-20T18:40:10.052 INFO:teuthology.orchestra.run.smithi083.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-20T18:40:10.052 INFO:teuthology.orchestra.run.smithi083.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-20T18:40:10.052 INFO:teuthology.orchestra.run.smithi083.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-20T18:40:10.052 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:40:10.052 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-20T18:40:10.311 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/6 2024-09-20T18:40:10.311 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:40:10.311 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-09-20T18:40:10.311 INFO:teuthology.orchestra.run.smithi142.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-20T18:40:10.312 INFO:teuthology.orchestra.run.smithi142.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-20T18:40:10.312 INFO:teuthology.orchestra.run.smithi142.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-20T18:40:10.312 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:40:10.312 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-09-20T18:40:10.399 DEBUG:teuthology.parallel:result is None 2024-09-20T18:40:10.651 DEBUG:teuthology.parallel:result is None 2024-09-20T18:40:10.652 INFO:teuthology.run_tasks:Running task install... 2024-09-20T18:40:10.660 DEBUG:teuthology.task.install:project ceph 2024-09-20T18:40:10.660 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '88deadb8082f4a33832e228ccefc59cedba158a5'}} 2024-09-20T18:40:10.660 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '88deadb8082f4a33832e228ccefc59cedba158a5'} 2024-09-20T18:40:10.660 INFO:teuthology.task.install:Using flavor: default 2024-09-20T18:40:10.665 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-20T18:40:10.666 INFO:teuthology.task.install:extra packages: [] 2024-09-20T18:40:10.666 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '88deadb8082f4a33832e228ccefc59cedba158a5', 'tag': None, 'wait_for_package': False} 2024-09-20T18:40:10.666 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=88deadb8082f4a33832e228ccefc59cedba158a5 2024-09-20T18:40:10.668 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '88deadb8082f4a33832e228ccefc59cedba158a5', 'tag': None, 'wait_for_package': False} 2024-09-20T18:40:10.668 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=88deadb8082f4a33832e228ccefc59cedba158a5 2024-09-20T18:40:10.809 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-20-1028/88deadb8082f4a33832e228ccefc59cedba158a5/centos/9/flavors/default/ 2024-09-20T18:40:10.809 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5099.g88deadb8 2024-09-20T18:40:10.813 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-20-1028/88deadb8082f4a33832e228ccefc59cedba158a5/centos/9/flavors/default/ 2024-09-20T18:40:10.813 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5099.g88deadb8 2024-09-20T18:40:10.921 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-20-1028/88deadb8082f4a33832e228ccefc59cedba158a5/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-20-1028/88deadb8082f4a33832e228ccefc59cedba158a5/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-20-1028/88deadb8082f4a33832e228ccefc59cedba158a5/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-20T18:40:10.921 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-20T18:40:10.922 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-20T18:40:10.927 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-20-1028/88deadb8082f4a33832e228ccefc59cedba158a5/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-20-1028/88deadb8082f4a33832e228ccefc59cedba158a5/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-20-1028/88deadb8082f4a33832e228ccefc59cedba158a5/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-20T18:40:10.928 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-20T18:40:10.928 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-20T18:40:10.952 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-20T18:40:10.967 DEBUG:teuthology.orchestra.run.smithi083:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/88deadb8082f4a33832e228ccefc59cedba158a5/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-20T18:40:10.969 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-20T18:40:10.969 DEBUG:teuthology.orchestra.run.smithi142:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/88deadb8082f4a33832e228ccefc59cedba158a5/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-20T18:40:11.019 DEBUG:teuthology.orchestra.run.smithi083:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-20T18:40:11.034 DEBUG:teuthology.orchestra.run.smithi142:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-20T18:40:11.095 DEBUG:teuthology.orchestra.run.smithi083:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-20T18:40:11.109 DEBUG:teuthology.orchestra.run.smithi142:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-20T18:40:11.159 INFO:teuthology.orchestra.run.smithi083.stdout:check_obsoletes = 1 2024-09-20T18:40:11.161 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum clean all 2024-09-20T18:40:11.173 INFO:teuthology.orchestra.run.smithi142.stdout:check_obsoletes = 1 2024-09-20T18:40:11.175 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum clean all 2024-09-20T18:40:11.377 INFO:teuthology.orchestra.run.smithi083.stdout:45 files removed 2024-09-20T18:40:11.399 INFO:teuthology.orchestra.run.smithi142.stdout:45 files removed 2024-09-20T18:40:11.404 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-radosgw 2024-09-20T18:40:11.426 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install ceph-radosgw 2024-09-20T18:40:11.959 INFO:teuthology.orchestra.run.smithi083.stdout:ceph packages for x86_64 310 kB/s | 84 kB 00:00 2024-09-20T18:40:11.982 INFO:teuthology.orchestra.run.smithi142.stdout:ceph packages for x86_64 308 kB/s | 84 kB 00:00 2024-09-20T18:40:12.258 INFO:teuthology.orchestra.run.smithi142.stdout:ceph noarch packages 104 kB/s | 24 kB 00:00 2024-09-20T18:40:12.259 INFO:teuthology.orchestra.run.smithi083.stdout:ceph noarch packages 94 kB/s | 24 kB 00:00 2024-09-20T18:40:12.525 INFO:teuthology.orchestra.run.smithi142.stdout:ceph source packages 8.4 kB/s | 1.9 kB 00:00 2024-09-20T18:40:12.535 INFO:teuthology.orchestra.run.smithi083.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-09-20T18:40:12.977 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - BaseOS 20 MB/s | 8.3 MB 00:00 2024-09-20T18:40:14.441 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - BaseOS 4.4 MB/s | 8.3 MB 00:01 2024-09-20T18:40:16.084 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - AppStream 35 MB/s | 20 MB 00:00 2024-09-20T18:40:20.336 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-20T18:40:22.878 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - AppStream 2.3 MB/s | 20 MB 00:08 2024-09-20T18:40:26.100 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - Extras packages 4.2 kB/s | 19 kB 00:04 2024-09-20T18:40:27.016 INFO:teuthology.orchestra.run.smithi142.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-20T18:40:29.220 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - CRB 2.5 MB/s | 6.5 MB 00:02 2024-09-20T18:40:31.389 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - Extras packages 24 kB/s | 19 kB 00:00 2024-09-20T18:40:32.350 INFO:teuthology.orchestra.run.smithi083.stdout:Extra Packages for Enterprise Linux 27 MB/s | 23 MB 00:00 2024-09-20T18:40:32.630 INFO:teuthology.orchestra.run.smithi142.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-09-20T18:40:34.855 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-09-20T18:40:34.858 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:40:34.858 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repository Size 2024-09-20T18:40:34.858 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:40:34.858 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-09-20T18:40:34.858 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-radosgw x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 11 M 2024-09-20T18:40:34.858 INFO:teuthology.orchestra.run.smithi142.stdout:Upgrading: 2024-09-20T18:40:34.858 INFO:teuthology.orchestra.run.smithi142.stdout: librados2 x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 3.5 M 2024-09-20T18:40:34.858 INFO:teuthology.orchestra.run.smithi142.stdout: librbd1 x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 3.2 M 2024-09-20T18:40:34.858 INFO:teuthology.orchestra.run.smithi142.stdout:Installing dependencies: 2024-09-20T18:40:34.858 INFO:teuthology.orchestra.run.smithi142.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-20T18:40:34.858 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-base x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 5.5 M 2024-09-20T18:40:34.858 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-common x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 22 M 2024-09-20T18:40:34.859 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-selinux x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 25 k 2024-09-20T18:40:34.859 INFO:teuthology.orchestra.run.smithi142.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-20T18:40:34.859 INFO:teuthology.orchestra.run.smithi142.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-20T18:40:34.859 INFO:teuthology.orchestra.run.smithi142.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-20T18:40:34.859 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs2 x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 756 k 2024-09-20T18:40:34.859 INFO:teuthology.orchestra.run.smithi142.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-20T18:40:34.859 INFO:teuthology.orchestra.run.smithi142.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-20T18:40:34.859 INFO:teuthology.orchestra.run.smithi142.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-20T18:40:34.859 INFO:teuthology.orchestra.run.smithi142.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-20T18:40:34.859 INFO:teuthology.orchestra.run.smithi142.stdout: libradosstriper1 x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 508 k 2024-09-20T18:40:34.859 INFO:teuthology.orchestra.run.smithi142.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-20T18:40:34.859 INFO:teuthology.orchestra.run.smithi142.stdout: librgw2 x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 5.5 M 2024-09-20T18:40:34.860 INFO:teuthology.orchestra.run.smithi142.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-20T18:40:34.860 INFO:teuthology.orchestra.run.smithi142.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-20T18:40:34.860 INFO:teuthology.orchestra.run.smithi142.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-20T18:40:34.860 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-argparse x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 45 k 2024-09-20T18:40:34.860 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-common x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 148 k 2024-09-20T18:40:34.860 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cephfs x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 162 k 2024-09-20T18:40:34.860 INFO:teuthology.orchestra.run.smithi142.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-20T18:40:34.860 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rados x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 320 k 2024-09-20T18:40:34.860 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rbd x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 307 k 2024-09-20T18:40:34.860 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rgw x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 99 k 2024-09-20T18:40:34.860 INFO:teuthology.orchestra.run.smithi142.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-20T18:40:34.860 INFO:teuthology.orchestra.run.smithi142.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-20T18:40:34.860 INFO:teuthology.orchestra.run.smithi142.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-20T18:40:34.861 INFO:teuthology.orchestra.run.smithi142.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-20T18:40:34.861 INFO:teuthology.orchestra.run.smithi142.stdout:Installing weak dependencies: 2024-09-20T18:40:34.861 INFO:teuthology.orchestra.run.smithi142.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-20T18:40:34.861 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:40:34.861 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-09-20T18:40:34.861 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:40:34.861 INFO:teuthology.orchestra.run.smithi142.stdout:Install 31 Packages 2024-09-20T18:40:34.861 INFO:teuthology.orchestra.run.smithi142.stdout:Upgrade 2 Packages 2024-09-20T18:40:34.861 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:40:34.862 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 62 M 2024-09-20T18:40:34.862 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-09-20T18:40:35.706 INFO:teuthology.orchestra.run.smithi142.stdout:(1/33): ceph-base-19.3.0-5099.g88deadb8.el9.x86 9.7 MB/s | 5.5 MB 00:00 2024-09-20T18:40:35.765 INFO:teuthology.orchestra.run.smithi142.stdout:(2/33): ceph-selinux-19.3.0-5099.g88deadb8.el9. 432 kB/s | 25 kB 00:00 2024-09-20T18:40:35.856 INFO:teuthology.orchestra.run.smithi142.stdout:(3/33): libcephfs2-19.3.0-5099.g88deadb8.el9.x8 8.1 MB/s | 756 kB 00:00 2024-09-20T18:40:35.965 INFO:teuthology.orchestra.run.smithi142.stdout:(4/33): ceph-radosgw-19.3.0-5099.g88deadb8.el9. 13 MB/s | 11 MB 00:00 2024-09-20T18:40:35.990 INFO:teuthology.orchestra.run.smithi142.stdout:(5/33): libradosstriper1-19.3.0-5099.g88deadb8. 3.7 MB/s | 508 kB 00:00 2024-09-20T18:40:36.049 INFO:teuthology.orchestra.run.smithi142.stdout:(6/33): python3-ceph-argparse-19.3.0-5099.g88de 774 kB/s | 45 kB 00:00 2024-09-20T18:40:36.141 INFO:teuthology.orchestra.run.smithi142.stdout:(7/33): python3-ceph-common-19.3.0-5099.g88dead 1.6 MB/s | 148 kB 00:00 2024-09-20T18:40:36.233 INFO:teuthology.orchestra.run.smithi142.stdout:(8/33): librgw2-19.3.0-5099.g88deadb8.el9.x86_6 21 MB/s | 5.5 MB 00:00 2024-09-20T18:40:36.258 INFO:teuthology.orchestra.run.smithi142.stdout:(9/33): python3-cephfs-19.3.0-5099.g88deadb8.el 1.4 MB/s | 162 kB 00:00 2024-09-20T18:40:36.283 INFO:teuthology.orchestra.run.smithi142.stdout:(10/33): python3-rados-19.3.0-5099.g88deadb8.el 6.3 MB/s | 320 kB 00:00 2024-09-20T18:40:36.486 INFO:teuthology.orchestra.run.smithi142.stdout:(11/33): ceph-common-19.3.0-5099.g88deadb8.el9. 16 MB/s | 22 MB 00:01 2024-09-20T18:40:36.511 INFO:teuthology.orchestra.run.smithi142.stdout:(12/33): python3-rgw-19.3.0-5099.g88deadb8.el9. 437 kB/s | 99 kB 00:00 2024-09-20T18:40:36.536 INFO:teuthology.orchestra.run.smithi142.stdout:(13/33): python3-rbd-19.3.0-5099.g88deadb8.el9. 1.1 MB/s | 307 kB 00:00 2024-09-20T18:40:36.678 INFO:teuthology.orchestra.run.smithi142.stdout:(14/33): boost-program-options-1.75.0-8.el9.x86 557 kB/s | 107 kB 00:00 2024-09-20T18:40:36.703 INFO:teuthology.orchestra.run.smithi142.stdout:(15/33): libnbd-1.20.2-2.el9.x86_64.rpm 887 kB/s | 171 kB 00:00 2024-09-20T18:40:36.728 INFO:teuthology.orchestra.run.smithi142.stdout:(16/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 833 kB/s | 160 kB 00:00 2024-09-20T18:40:36.754 INFO:teuthology.orchestra.run.smithi142.stdout:(17/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 599 kB/s | 45 kB 00:00 2024-09-20T18:40:36.812 INFO:teuthology.orchestra.run.smithi142.stdout:(18/33): python3-prettytable-0.7.2-27.el9.noarc 723 kB/s | 42 kB 00:00 2024-09-20T18:40:36.837 INFO:teuthology.orchestra.run.smithi142.stdout:(19/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-09-20T18:40:36.871 INFO:teuthology.orchestra.run.smithi142.stdout:(20/33): librdkafka-1.6.1-102.el9.x86_64.rpm 3.9 MB/s | 662 kB 00:00 2024-09-20T18:40:36.888 INFO:teuthology.orchestra.run.smithi142.stdout:(21/33): qatlib-service-24.02.0-1.el9.x86_64.rp 712 kB/s | 36 kB 00:00 2024-09-20T18:40:36.922 INFO:teuthology.orchestra.run.smithi142.stdout:(22/33): qatlib-24.02.0-1.el9.x86_64.rpm 2.0 MB/s | 221 kB 00:00 2024-09-20T18:40:36.947 INFO:teuthology.orchestra.run.smithi142.stdout:(23/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 628 kB/s | 47 kB 00:00 2024-09-20T18:40:36.989 INFO:teuthology.orchestra.run.smithi142.stdout:(24/33): gperftools-libs-2.9.1-3.el9.x86_64.rpm 3.0 MB/s | 308 kB 00:00 2024-09-20T18:40:37.014 INFO:teuthology.orchestra.run.smithi142.stdout:(25/33): liboath-2.6.7-2.el9.x86_64.rpm 2.1 MB/s | 49 kB 00:00 2024-09-20T18:40:37.039 INFO:teuthology.orchestra.run.smithi142.stdout:(26/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 306 kB/s | 26 kB 00:00 2024-09-20T18:40:37.064 INFO:teuthology.orchestra.run.smithi142.stdout:(27/33): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-20T18:40:37.090 INFO:teuthology.orchestra.run.smithi142.stdout:(28/33): re2-20211101-20.el9.x86_64.rpm 7.5 MB/s | 191 kB 00:00 2024-09-20T18:40:37.140 INFO:teuthology.orchestra.run.smithi142.stdout:(29/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.2 MB/s | 839 kB 00:00 2024-09-20T18:40:37.235 INFO:teuthology.orchestra.run.smithi142.stdout:(30/33): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-20T18:40:37.307 INFO:teuthology.orchestra.run.smithi142.stdout:(31/33): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-20T18:40:37.408 INFO:teuthology.orchestra.run.smithi142.stdout:(32/33): librados2-19.3.0-5099.g88deadb8.el9.x8 13 MB/s | 3.5 MB 00:00 2024-09-20T18:40:37.474 INFO:teuthology.orchestra.run.smithi142.stdout:(33/33): librbd1-19.3.0-5099.g88deadb8.el9.x86_ 14 MB/s | 3.2 MB 00:00 2024-09-20T18:40:37.479 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-09-20T18:40:37.479 INFO:teuthology.orchestra.run.smithi142.stdout:Total 24 MB/s | 62 MB 00:02 2024-09-20T18:40:37.592 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-09-20T18:40:37.627 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-09-20T18:40:37.628 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-09-20T18:40:37.765 INFO:teuthology.orchestra.run.smithi083.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-20T18:40:38.136 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-09-20T18:40:38.136 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-09-20T18:40:39.608 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-09-20T18:40:39.688 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-20T18:40:39.713 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/35 2024-09-20T18:40:39.738 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/35 2024-09-20T18:40:39.761 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-20T18:40:39.784 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-20T18:40:39.930 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-20T18:40:39.982 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-20T18:40:40.043 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-20T18:40:40.045 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:40:40.045 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repository Size 2024-09-20T18:40:40.045 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:40:40.045 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2024-09-20T18:40:40.045 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-radosgw x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 11 M 2024-09-20T18:40:40.045 INFO:teuthology.orchestra.run.smithi083.stdout:Upgrading: 2024-09-20T18:40:40.045 INFO:teuthology.orchestra.run.smithi083.stdout: librados2 x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 3.5 M 2024-09-20T18:40:40.046 INFO:teuthology.orchestra.run.smithi083.stdout: librbd1 x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 3.2 M 2024-09-20T18:40:40.046 INFO:teuthology.orchestra.run.smithi083.stdout:Installing dependencies: 2024-09-20T18:40:40.046 INFO:teuthology.orchestra.run.smithi083.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-20T18:40:40.046 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-base x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 5.5 M 2024-09-20T18:40:40.046 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-common x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 22 M 2024-09-20T18:40:40.046 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-selinux x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 25 k 2024-09-20T18:40:40.046 INFO:teuthology.orchestra.run.smithi083.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-20T18:40:40.046 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-20T18:40:40.046 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-20T18:40:40.046 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs2 x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 756 k 2024-09-20T18:40:40.046 INFO:teuthology.orchestra.run.smithi083.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-20T18:40:40.046 INFO:teuthology.orchestra.run.smithi083.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-20T18:40:40.047 INFO:teuthology.orchestra.run.smithi083.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-20T18:40:40.047 INFO:teuthology.orchestra.run.smithi083.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-20T18:40:40.047 INFO:teuthology.orchestra.run.smithi083.stdout: libradosstriper1 x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 508 k 2024-09-20T18:40:40.047 INFO:teuthology.orchestra.run.smithi083.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-20T18:40:40.047 INFO:teuthology.orchestra.run.smithi083.stdout: librgw2 x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 5.5 M 2024-09-20T18:40:40.047 INFO:teuthology.orchestra.run.smithi083.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-20T18:40:40.047 INFO:teuthology.orchestra.run.smithi083.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-20T18:40:40.047 INFO:teuthology.orchestra.run.smithi083.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-20T18:40:40.047 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-argparse x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 45 k 2024-09-20T18:40:40.047 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-common x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 148 k 2024-09-20T18:40:40.047 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cephfs x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 162 k 2024-09-20T18:40:40.047 INFO:teuthology.orchestra.run.smithi083.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-20T18:40:40.047 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rados x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 320 k 2024-09-20T18:40:40.048 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rbd x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 307 k 2024-09-20T18:40:40.048 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rgw x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 99 k 2024-09-20T18:40:40.048 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-20T18:40:40.048 INFO:teuthology.orchestra.run.smithi083.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-20T18:40:40.048 INFO:teuthology.orchestra.run.smithi083.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-20T18:40:40.048 INFO:teuthology.orchestra.run.smithi083.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-20T18:40:40.048 INFO:teuthology.orchestra.run.smithi083.stdout:Installing weak dependencies: 2024-09-20T18:40:40.048 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-20T18:40:40.048 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:40:40.048 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-09-20T18:40:40.048 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:40:40.048 INFO:teuthology.orchestra.run.smithi083.stdout:Install 31 Packages 2024-09-20T18:40:40.049 INFO:teuthology.orchestra.run.smithi083.stdout:Upgrade 2 Packages 2024-09-20T18:40:40.049 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:40:40.049 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 62 M 2024-09-20T18:40:40.049 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-09-20T18:40:40.318 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/35 2024-09-20T18:40:40.431 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : librados2-2:19.3.0-5099.g88deadb8.el9.x86_64 7/35 2024-09-20T18:40:40.532 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librados2-2:19.3.0-5099.g88deadb8.el9.x86_64 7/35 2024-09-20T18:40:40.619 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rados-2:19.3.0-5099.g88deadb8.el9.x86_64 8/35 2024-09-20T18:40:40.650 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephfs2-2:19.3.0-5099.g88deadb8.el9.x86_64 9/35 2024-09-20T18:40:40.704 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libcephfs2-2:19.3.0-5099.g88deadb8.el9.x86_64 9/35 2024-09-20T18:40:40.732 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/35 2024-09-20T18:40:40.795 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-ceph-argparse-2:19.3.0-5099.g88deadb8.el9. 11/35 2024-09-20T18:40:40.799 INFO:teuthology.orchestra.run.smithi083.stdout:(1/33): ceph-base-19.3.0-5099.g88deadb8.el9.x86 11 MB/s | 5.5 MB 00:00 2024-09-20T18:40:40.866 INFO:teuthology.orchestra.run.smithi083.stdout:(2/33): ceph-selinux-19.3.0-5099.g88deadb8.el9. 379 kB/s | 25 kB 00:00 2024-09-20T18:40:40.887 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cephfs-2:19.3.0-5099.g88deadb8.el9.x86_64 12/35 2024-09-20T18:40:40.910 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libradosstriper1-2:19.3.0-5099.g88deadb8.el9.x86_6 13/35 2024-09-20T18:40:40.974 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5099.g88deadb8.el9.x86_6 13/35 2024-09-20T18:40:41.008 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/35 2024-09-20T18:40:41.050 INFO:teuthology.orchestra.run.smithi083.stdout:(3/33): ceph-radosgw-19.3.0-5099.g88deadb8.el9. 14 MB/s | 11 MB 00:00 2024-09-20T18:40:41.092 INFO:teuthology.orchestra.run.smithi083.stdout:(4/33): libcephfs2-19.3.0-5099.g88deadb8.el9.x8 3.3 MB/s | 756 kB 00:00 2024-09-20T18:40:41.117 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/35 2024-09-20T18:40:41.117 INFO:teuthology.orchestra.run.smithi083.stdout:(5/33): libradosstriper1-19.3.0-5099.g88deadb8. 7.4 MB/s | 508 kB 00:00 2024-09-20T18:40:41.160 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/35 2024-09-20T18:40:41.176 INFO:teuthology.orchestra.run.smithi083.stdout:(6/33): python3-ceph-argparse-19.3.0-5099.g88de 774 kB/s | 45 kB 00:00 2024-09-20T18:40:41.211 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/35 2024-09-20T18:40:41.250 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/35 2024-09-20T18:40:41.268 INFO:teuthology.orchestra.run.smithi083.stdout:(7/33): python3-ceph-common-19.3.0-5099.g88dead 1.6 MB/s | 148 kB 00:00 2024-09-20T18:40:41.282 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/35 2024-09-20T18:40:41.312 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-20T18:40:41.343 INFO:teuthology.orchestra.run.smithi083.stdout:(8/33): librgw2-19.3.0-5099.g88deadb8.el9.x86_6 22 MB/s | 5.5 MB 00:00 2024-09-20T18:40:41.348 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/35 2024-09-20T18:40:41.397 INFO:teuthology.orchestra.run.smithi083.stdout:(9/33): python3-cephfs-19.3.0-5099.g88deadb8.el 1.2 MB/s | 162 kB 00:00 2024-09-20T18:40:41.427 INFO:teuthology.orchestra.run.smithi083.stdout:(10/33): python3-rados-19.3.0-5099.g88deadb8.el 3.7 MB/s | 320 kB 00:00 2024-09-20T18:40:41.460 INFO:teuthology.orchestra.run.smithi083.stdout:(11/33): python3-rbd-19.3.0-5099.g88deadb8.el9. 4.7 MB/s | 307 kB 00:00 2024-09-20T18:40:41.486 INFO:teuthology.orchestra.run.smithi083.stdout:(12/33): python3-rgw-19.3.0-5099.g88deadb8.el9. 1.7 MB/s | 99 kB 00:00 2024-09-20T18:40:41.673 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-20T18:40:41.703 INFO:teuthology.orchestra.run.smithi083.stdout:(13/33): ceph-common-19.3.0-5099.g88deadb8.el9. 15 MB/s | 22 MB 00:01 2024-09-20T18:40:41.717 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : librbd1-2:19.3.0-5099.g88deadb8.el9.x86_64 23/35 2024-09-20T18:40:41.810 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librbd1-2:19.3.0-5099.g88deadb8.el9.x86_64 23/35 2024-09-20T18:40:41.836 INFO:teuthology.orchestra.run.smithi083.stdout:(14/33): boost-program-options-1.75.0-8.el9.x86 284 kB/s | 107 kB 00:00 2024-09-20T18:40:41.839 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rbd-2:19.3.0-5099.g88deadb8.el9.x86_64 24/35 2024-09-20T18:40:41.862 INFO:teuthology.orchestra.run.smithi083.stdout:(15/33): libnbd-1.20.2-2.el9.x86_64.rpm 454 kB/s | 171 kB 00:00 2024-09-20T18:40:41.875 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-20T18:40:41.887 INFO:teuthology.orchestra.run.smithi083.stdout:(16/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 872 kB/s | 160 kB 00:00 2024-09-20T18:40:41.928 INFO:teuthology.orchestra.run.smithi083.stdout:(17/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 599 kB/s | 45 kB 00:00 2024-09-20T18:40:41.963 INFO:teuthology.orchestra.run.smithi083.stdout:(18/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-09-20T18:40:42.013 INFO:teuthology.orchestra.run.smithi083.stdout:(19/33): librdkafka-1.6.1-102.el9.x86_64.rpm 5.2 MB/s | 662 kB 00:00 2024-09-20T18:40:42.038 INFO:teuthology.orchestra.run.smithi083.stdout:(20/33): python3-prettytable-0.7.2-27.el9.noarc 336 kB/s | 42 kB 00:00 2024-09-20T18:40:42.072 INFO:teuthology.orchestra.run.smithi083.stdout:(21/33): qatlib-24.02.0-1.el9.x86_64.rpm 2.0 MB/s | 221 kB 00:00 2024-09-20T18:40:42.097 INFO:teuthology.orchestra.run.smithi083.stdout:(22/33): qatlib-service-24.02.0-1.el9.x86_64.rp 428 kB/s | 36 kB 00:00 2024-09-20T18:40:42.122 INFO:teuthology.orchestra.run.smithi083.stdout:(23/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 565 kB/s | 47 kB 00:00 2024-09-20T18:40:42.172 INFO:teuthology.orchestra.run.smithi083.stdout:(24/33): gperftools-libs-2.9.1-3.el9.x86_64.rpm 3.0 MB/s | 308 kB 00:00 2024-09-20T18:40:42.198 INFO:teuthology.orchestra.run.smithi083.stdout:(25/33): liboath-2.6.7-2.el9.x86_64.rpm 2.1 MB/s | 49 kB 00:00 2024-09-20T18:40:42.223 INFO:teuthology.orchestra.run.smithi083.stdout:(26/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 301 kB/s | 26 kB 00:00 2024-09-20T18:40:42.248 INFO:teuthology.orchestra.run.smithi083.stdout:(27/33): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-20T18:40:42.282 INFO:teuthology.orchestra.run.smithi083.stdout:(28/33): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-20T18:40:42.332 INFO:teuthology.orchestra.run.smithi083.stdout:(29/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.5 MB/s | 839 kB 00:00 2024-09-20T18:40:42.388 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-20T18:40:42.391 INFO:teuthology.orchestra.run.smithi083.stdout:(30/33): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-20T18:40:42.466 INFO:teuthology.orchestra.run.smithi083.stdout:(31/33): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-20T18:40:42.528 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librgw2-2:19.3.0-5099.g88deadb8.el9.x86_64 27/35 2024-09-20T18:40:42.575 INFO:teuthology.orchestra.run.smithi083.stdout:(32/33): librados2-19.3.0-5099.g88deadb8.el9.x8 14 MB/s | 3.5 MB 00:00 2024-09-20T18:40:42.583 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librgw2-2:19.3.0-5099.g88deadb8.el9.x86_64 27/35 2024-09-20T18:40:42.633 INFO:teuthology.orchestra.run.smithi083.stdout:(33/33): librbd1-19.3.0-5099.g88deadb8.el9.x86_ 13 MB/s | 3.2 MB 00:00 2024-09-20T18:40:42.638 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-09-20T18:40:42.638 INFO:teuthology.orchestra.run.smithi083.stdout:Total 24 MB/s | 62 MB 00:02 2024-09-20T18:40:42.665 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rgw-2:19.3.0-5099.g88deadb8.el9.x86_64 28/35 2024-09-20T18:40:42.689 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-ceph-common-2:19.3.0-5099.g88deadb8.el9.x8 29/35 2024-09-20T18:40:42.746 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-09-20T18:40:42.783 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-09-20T18:40:42.783 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-09-20T18:40:43.314 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-09-20T18:40:43.314 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-09-20T18:40:44.800 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-09-20T18:40:44.917 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-20T18:40:44.956 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/35 2024-09-20T18:40:44.989 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/35 2024-09-20T18:40:45.012 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-20T18:40:45.017 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-common-2:19.3.0-5099.g88deadb8.el9.x86_64 30/35 2024-09-20T18:40:45.038 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-20T18:40:45.060 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-common-2:19.3.0-5099.g88deadb8.el9.x86_64 30/35 2024-09-20T18:40:45.231 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-20T18:40:45.283 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-20T18:40:45.627 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-common-2:19.3.0-5099.g88deadb8.el9.x86_64 30/35 2024-09-20T18:40:45.637 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/35 2024-09-20T18:40:45.663 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-base-2:19.3.0-5099.g88deadb8.el9.x86_64 31/35 2024-09-20T18:40:45.754 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-base-2:19.3.0-5099.g88deadb8.el9.x86_64 31/35 2024-09-20T18:40:45.754 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-20T18:40:45.755 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-20T18:40:45.755 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:40:45.789 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-selinux-2:19.3.0-5099.g88deadb8.el9.x86_64 32/35 2024-09-20T18:40:45.808 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : librados2-2:19.3.0-5099.g88deadb8.el9.x86_64 7/35 2024-09-20T18:40:45.920 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librados2-2:19.3.0-5099.g88deadb8.el9.x86_64 7/35 2024-09-20T18:40:46.010 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rados-2:19.3.0-5099.g88deadb8.el9.x86_64 8/35 2024-09-20T18:40:46.046 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephfs2-2:19.3.0-5099.g88deadb8.el9.x86_64 9/35 2024-09-20T18:40:46.108 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libcephfs2-2:19.3.0-5099.g88deadb8.el9.x86_64 9/35 2024-09-20T18:40:46.137 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/35 2024-09-20T18:40:46.182 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-ceph-argparse-2:19.3.0-5099.g88deadb8.el9. 11/35 2024-09-20T18:40:46.251 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cephfs-2:19.3.0-5099.g88deadb8.el9.x86_64 12/35 2024-09-20T18:40:46.281 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libradosstriper1-2:19.3.0-5099.g88deadb8.el9.x86_6 13/35 2024-09-20T18:40:46.328 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5099.g88deadb8.el9.x86_6 13/35 2024-09-20T18:40:46.363 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/35 2024-09-20T18:40:46.493 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/35 2024-09-20T18:40:46.531 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/35 2024-09-20T18:40:46.581 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/35 2024-09-20T18:40:46.612 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/35 2024-09-20T18:40:46.641 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/35 2024-09-20T18:40:46.675 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-20T18:40:46.712 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/35 2024-09-20T18:40:47.039 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-20T18:40:47.097 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : librbd1-2:19.3.0-5099.g88deadb8.el9.x86_64 23/35 2024-09-20T18:40:47.180 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librbd1-2:19.3.0-5099.g88deadb8.el9.x86_64 23/35 2024-09-20T18:40:47.218 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rbd-2:19.3.0-5099.g88deadb8.el9.x86_64 24/35 2024-09-20T18:40:47.254 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-20T18:40:47.743 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-20T18:40:47.774 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librgw2-2:19.3.0-5099.g88deadb8.el9.x86_64 27/35 2024-09-20T18:40:47.846 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librgw2-2:19.3.0-5099.g88deadb8.el9.x86_64 27/35 2024-09-20T18:40:47.927 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rgw-2:19.3.0-5099.g88deadb8.el9.x86_64 28/35 2024-09-20T18:40:47.960 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-ceph-common-2:19.3.0-5099.g88deadb8.el9.x8 29/35 2024-09-20T18:40:50.327 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-common-2:19.3.0-5099.g88deadb8.el9.x86_64 30/35 2024-09-20T18:40:50.373 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-common-2:19.3.0-5099.g88deadb8.el9.x86_64 30/35 2024-09-20T18:40:50.966 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-common-2:19.3.0-5099.g88deadb8.el9.x86_64 30/35 2024-09-20T18:40:51.002 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-base-2:19.3.0-5099.g88deadb8.el9.x86_64 31/35 2024-09-20T18:40:51.097 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-base-2:19.3.0-5099.g88deadb8.el9.x86_64 31/35 2024-09-20T18:40:51.097 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-20T18:40:51.097 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-20T18:40:51.097 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:40:51.137 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-selinux-2:19.3.0-5099.g88deadb8.el9.x86_64 32/35 2024-09-20T18:40:55.615 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5099.g88deadb8.el9.x86_64 32/35 2024-09-20T18:40:55.616 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /sys 2024-09-20T18:40:55.616 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /proc 2024-09-20T18:40:55.616 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /mnt 2024-09-20T18:40:55.616 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /var/tmp 2024-09-20T18:40:55.616 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /home 2024-09-20T18:40:55.616 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /root 2024-09-20T18:40:55.616 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /tmp 2024-09-20T18:40:55.616 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:40:56.656 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-radosgw-2:19.3.0-5099.g88deadb8.el9.x86_64 33/35 2024-09-20T18:40:56.683 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5099.g88deadb8.el9.x86_64 33/35 2024-09-20T18:40:56.683 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-20T18:40:56.683 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-20T18:40:56.683 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-20T18:40:56.683 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-20T18:40:56.683 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:40:56.685 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-20T18:40:56.775 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-20T18:40:56.775 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-20T18:40:57.922 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-20T18:40:57.923 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-base-2:19.3.0-5099.g88deadb8.el9.x86_64 1/35 2024-09-20T18:40:57.923 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-common-2:19.3.0-5099.g88deadb8.el9.x86_64 2/35 2024-09-20T18:40:57.923 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-radosgw-2:19.3.0-5099.g88deadb8.el9.x86_64 3/35 2024-09-20T18:40:57.923 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-selinux-2:19.3.0-5099.g88deadb8.el9.x86_64 4/35 2024-09-20T18:40:57.923 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephfs2-2:19.3.0-5099.g88deadb8.el9.x86_64 5/35 2024-09-20T18:40:57.923 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libradosstriper1-2:19.3.0-5099.g88deadb8.el9.x86_6 6/35 2024-09-20T18:40:57.923 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librgw2-2:19.3.0-5099.g88deadb8.el9.x86_64 7/35 2024-09-20T18:40:57.923 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5099.g88deadb8.el9. 8/35 2024-09-20T18:40:57.923 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-ceph-common-2:19.3.0-5099.g88deadb8.el9.x8 9/35 2024-09-20T18:40:57.924 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cephfs-2:19.3.0-5099.g88deadb8.el9.x86_64 10/35 2024-09-20T18:40:57.924 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rados-2:19.3.0-5099.g88deadb8.el9.x86_64 11/35 2024-09-20T18:40:57.924 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rbd-2:19.3.0-5099.g88deadb8.el9.x86_64 12/35 2024-09-20T18:40:57.924 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rgw-2:19.3.0-5099.g88deadb8.el9.x86_64 13/35 2024-09-20T18:40:57.924 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-20T18:40:57.924 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-20T18:40:57.924 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-20T18:40:57.924 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-20T18:40:57.924 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-20T18:40:57.924 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-20T18:40:57.924 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-20T18:40:57.924 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-20T18:40:57.924 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-20T18:40:57.924 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-20T18:40:57.925 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 24/35 2024-09-20T18:40:57.925 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-20T18:40:57.925 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-20T18:40:57.925 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-20T18:40:57.925 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-20T18:40:57.925 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-20T18:40:57.925 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-20T18:40:57.925 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-20T18:40:57.925 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados2-2:19.3.0-5099.g88deadb8.el9.x86_64 32/35 2024-09-20T18:40:57.925 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-20T18:40:57.925 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librbd1-2:19.3.0-5099.g88deadb8.el9.x86_64 34/35 2024-09-20T18:40:58.784 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-20T18:40:58.784 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:40:58.784 INFO:teuthology.orchestra.run.smithi142.stdout:Upgraded: 2024-09-20T18:40:58.784 INFO:teuthology.orchestra.run.smithi142.stdout: librados2-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:40:58.784 INFO:teuthology.orchestra.run.smithi142.stdout: librbd1-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:40:58.784 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-09-20T18:40:58.785 INFO:teuthology.orchestra.run.smithi142.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-20T18:40:58.785 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-base-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:40:58.785 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-common-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:40:58.785 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-radosgw-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:40:58.785 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-selinux-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:40:58.785 INFO:teuthology.orchestra.run.smithi142.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-20T18:40:58.785 INFO:teuthology.orchestra.run.smithi142.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-20T18:40:58.785 INFO:teuthology.orchestra.run.smithi142.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-20T18:40:58.785 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs2-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:40:58.785 INFO:teuthology.orchestra.run.smithi142.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-20T18:40:58.785 INFO:teuthology.orchestra.run.smithi142.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-20T18:40:58.785 INFO:teuthology.orchestra.run.smithi142.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-20T18:40:58.785 INFO:teuthology.orchestra.run.smithi142.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-20T18:40:58.785 INFO:teuthology.orchestra.run.smithi142.stdout: libradosstriper1-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:40:58.786 INFO:teuthology.orchestra.run.smithi142.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-20T18:40:58.786 INFO:teuthology.orchestra.run.smithi142.stdout: librgw2-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:40:58.786 INFO:teuthology.orchestra.run.smithi142.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-20T18:40:58.786 INFO:teuthology.orchestra.run.smithi142.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-20T18:40:58.786 INFO:teuthology.orchestra.run.smithi142.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-20T18:40:58.786 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-argparse-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:40:58.786 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-common-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:40:58.786 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cephfs-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:40:58.786 INFO:teuthology.orchestra.run.smithi142.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-20T18:40:58.786 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rados-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:40:58.786 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rbd-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:40:58.786 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rgw-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:40:58.787 INFO:teuthology.orchestra.run.smithi142.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-20T18:40:58.787 INFO:teuthology.orchestra.run.smithi142.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-20T18:40:58.787 INFO:teuthology.orchestra.run.smithi142.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-20T18:40:58.787 INFO:teuthology.orchestra.run.smithi142.stdout: re2-1:20211101-20.el9.x86_64 2024-09-20T18:40:58.787 INFO:teuthology.orchestra.run.smithi142.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-20T18:40:58.787 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:40:58.787 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-09-20T18:40:59.162 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install ceph-test 2024-09-20T18:40:59.750 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:00:27 ago on Fri 20 Sep 2024 06:40:32 PM UTC. 2024-09-20T18:40:59.882 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-09-20T18:40:59.882 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:40:59.882 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repository Size 2024-09-20T18:40:59.882 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:40:59.882 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-09-20T18:40:59.882 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-test x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 46 M 2024-09-20T18:40:59.883 INFO:teuthology.orchestra.run.smithi142.stdout:Installing dependencies: 2024-09-20T18:40:59.883 INFO:teuthology.orchestra.run.smithi142.stdout: libcephsqlite x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 161 k 2024-09-20T18:40:59.883 INFO:teuthology.orchestra.run.smithi142.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-20T18:40:59.883 INFO:teuthology.orchestra.run.smithi142.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-20T18:40:59.883 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:40:59.883 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-09-20T18:40:59.883 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:40:59.883 INFO:teuthology.orchestra.run.smithi142.stdout:Install 4 Packages 2024-09-20T18:40:59.883 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:40:59.884 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 46 M 2024-09-20T18:40:59.884 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 201 M 2024-09-20T18:40:59.884 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-09-20T18:41:00.115 INFO:teuthology.orchestra.run.smithi142.stdout:(1/4): libcephsqlite-19.3.0-5099.g88deadb8.el9. 768 kB/s | 161 kB 00:00 2024-09-20T18:41:00.682 INFO:teuthology.orchestra.run.smithi142.stdout:(2/4): socat-1.7.4.1-6.el9.x86_64.rpm 392 kB/s | 304 kB 00:00 2024-09-20T18:41:00.756 INFO:teuthology.orchestra.run.smithi142.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 99 kB/s | 64 kB 00:00 2024-09-20T18:41:01.030 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5099.g88deadb8.el9.x86_64 32/35 2024-09-20T18:41:01.030 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /sys 2024-09-20T18:41:01.030 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /proc 2024-09-20T18:41:01.030 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /mnt 2024-09-20T18:41:01.030 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /var/tmp 2024-09-20T18:41:01.030 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /home 2024-09-20T18:41:01.030 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /root 2024-09-20T18:41:01.030 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /tmp 2024-09-20T18:41:01.030 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:41:01.274 INFO:teuthology.orchestra.run.smithi142.stdout:(4/4): ceph-test-19.3.0-5099.g88deadb8.el9.x86_ 33 MB/s | 46 MB 00:01 2024-09-20T18:41:01.276 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-09-20T18:41:01.276 INFO:teuthology.orchestra.run.smithi142.stdout:Total 33 MB/s | 46 MB 00:01 2024-09-20T18:41:01.305 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-09-20T18:41:01.326 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-09-20T18:41:01.326 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-09-20T18:41:01.692 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-09-20T18:41:01.692 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-09-20T18:41:02.022 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-radosgw-2:19.3.0-5099.g88deadb8.el9.x86_64 33/35 2024-09-20T18:41:02.047 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5099.g88deadb8.el9.x86_64 33/35 2024-09-20T18:41:02.047 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-20T18:41:02.047 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-20T18:41:02.047 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-20T18:41:02.047 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-20T18:41:02.047 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:41:02.049 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-20T18:41:02.139 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-20T18:41:02.139 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-20T18:41:02.210 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-09-20T18:41:02.301 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-20T18:41:02.348 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-20T18:41:02.375 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephsqlite-2:19.3.0-5099.g88deadb8.el9.x86_64 3/4 2024-09-20T18:41:03.293 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-20T18:41:03.293 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-base-2:19.3.0-5099.g88deadb8.el9.x86_64 1/35 2024-09-20T18:41:03.293 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-common-2:19.3.0-5099.g88deadb8.el9.x86_64 2/35 2024-09-20T18:41:03.293 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-radosgw-2:19.3.0-5099.g88deadb8.el9.x86_64 3/35 2024-09-20T18:41:03.294 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-selinux-2:19.3.0-5099.g88deadb8.el9.x86_64 4/35 2024-09-20T18:41:03.294 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephfs2-2:19.3.0-5099.g88deadb8.el9.x86_64 5/35 2024-09-20T18:41:03.294 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libradosstriper1-2:19.3.0-5099.g88deadb8.el9.x86_6 6/35 2024-09-20T18:41:03.294 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librgw2-2:19.3.0-5099.g88deadb8.el9.x86_64 7/35 2024-09-20T18:41:03.294 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5099.g88deadb8.el9. 8/35 2024-09-20T18:41:03.294 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-ceph-common-2:19.3.0-5099.g88deadb8.el9.x8 9/35 2024-09-20T18:41:03.294 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cephfs-2:19.3.0-5099.g88deadb8.el9.x86_64 10/35 2024-09-20T18:41:03.294 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rados-2:19.3.0-5099.g88deadb8.el9.x86_64 11/35 2024-09-20T18:41:03.294 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rbd-2:19.3.0-5099.g88deadb8.el9.x86_64 12/35 2024-09-20T18:41:03.294 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rgw-2:19.3.0-5099.g88deadb8.el9.x86_64 13/35 2024-09-20T18:41:03.294 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-20T18:41:03.294 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-20T18:41:03.295 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-20T18:41:03.295 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-20T18:41:03.295 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-20T18:41:03.295 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-20T18:41:03.295 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-20T18:41:03.295 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-20T18:41:03.295 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-20T18:41:03.295 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-20T18:41:03.295 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 24/35 2024-09-20T18:41:03.295 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-20T18:41:03.295 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-20T18:41:03.295 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-20T18:41:03.296 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-20T18:41:03.297 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-20T18:41:03.297 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-20T18:41:03.297 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-20T18:41:03.297 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados2-2:19.3.0-5099.g88deadb8.el9.x86_64 32/35 2024-09-20T18:41:03.297 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-20T18:41:03.297 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librbd1-2:19.3.0-5099.g88deadb8.el9.x86_64 34/35 2024-09-20T18:41:04.302 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-20T18:41:04.302 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:41:04.302 INFO:teuthology.orchestra.run.smithi083.stdout:Upgraded: 2024-09-20T18:41:04.302 INFO:teuthology.orchestra.run.smithi083.stdout: librados2-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:41:04.302 INFO:teuthology.orchestra.run.smithi083.stdout: librbd1-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:41:04.302 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2024-09-20T18:41:04.303 INFO:teuthology.orchestra.run.smithi083.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-20T18:41:04.303 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-base-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:41:04.303 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-common-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:41:04.303 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-radosgw-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:41:04.303 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-selinux-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:41:04.303 INFO:teuthology.orchestra.run.smithi083.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-20T18:41:04.303 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-20T18:41:04.303 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-20T18:41:04.303 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs2-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:41:04.303 INFO:teuthology.orchestra.run.smithi083.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-20T18:41:04.303 INFO:teuthology.orchestra.run.smithi083.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-20T18:41:04.303 INFO:teuthology.orchestra.run.smithi083.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-20T18:41:04.303 INFO:teuthology.orchestra.run.smithi083.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-20T18:41:04.304 INFO:teuthology.orchestra.run.smithi083.stdout: libradosstriper1-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:41:04.304 INFO:teuthology.orchestra.run.smithi083.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-20T18:41:04.304 INFO:teuthology.orchestra.run.smithi083.stdout: librgw2-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:41:04.304 INFO:teuthology.orchestra.run.smithi083.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-20T18:41:04.304 INFO:teuthology.orchestra.run.smithi083.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-20T18:41:04.304 INFO:teuthology.orchestra.run.smithi083.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-20T18:41:04.304 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-argparse-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:41:04.304 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-common-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:41:04.304 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cephfs-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:41:04.304 INFO:teuthology.orchestra.run.smithi083.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-20T18:41:04.304 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rados-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:41:04.304 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rbd-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:41:04.304 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rgw-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:41:04.304 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-20T18:41:04.305 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-20T18:41:04.305 INFO:teuthology.orchestra.run.smithi083.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-20T18:41:04.305 INFO:teuthology.orchestra.run.smithi083.stdout: re2-1:20211101-20.el9.x86_64 2024-09-20T18:41:04.305 INFO:teuthology.orchestra.run.smithi083.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-20T18:41:04.305 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:41:04.305 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-20T18:41:04.694 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-test 2024-09-20T18:41:05.282 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:00:28 ago on Fri 20 Sep 2024 06:40:37 PM UTC. 2024-09-20T18:41:05.414 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-20T18:41:05.415 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:41:05.415 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repository Size 2024-09-20T18:41:05.415 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:41:05.415 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2024-09-20T18:41:05.415 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-test x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 46 M 2024-09-20T18:41:05.415 INFO:teuthology.orchestra.run.smithi083.stdout:Installing dependencies: 2024-09-20T18:41:05.415 INFO:teuthology.orchestra.run.smithi083.stdout: libcephsqlite x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 161 k 2024-09-20T18:41:05.415 INFO:teuthology.orchestra.run.smithi083.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-20T18:41:05.415 INFO:teuthology.orchestra.run.smithi083.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-20T18:41:05.415 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:41:05.416 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-09-20T18:41:05.416 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:41:05.416 INFO:teuthology.orchestra.run.smithi083.stdout:Install 4 Packages 2024-09-20T18:41:05.416 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:41:05.416 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 46 M 2024-09-20T18:41:05.416 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 201 M 2024-09-20T18:41:05.416 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-09-20T18:41:05.873 INFO:teuthology.orchestra.run.smithi083.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.5 MB/s | 304 kB 00:00 2024-09-20T18:41:05.907 INFO:teuthology.orchestra.run.smithi083.stdout:(2/4): libcephsqlite-19.3.0-5099.g88deadb8.el9. 696 kB/s | 161 kB 00:00 2024-09-20T18:41:05.932 INFO:teuthology.orchestra.run.smithi083.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-20T18:41:06.721 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5099.g88deadb8.el9.x86_64 3/4 2024-09-20T18:41:06.800 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-test-2:19.3.0-5099.g88deadb8.el9.x86_64 4/4 2024-09-20T18:41:07.133 INFO:teuthology.orchestra.run.smithi083.stdout:(4/4): ceph-test-19.3.0-5099.g88deadb8.el9.x86_ 31 MB/s | 46 MB 00:01 2024-09-20T18:41:07.135 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-09-20T18:41:07.135 INFO:teuthology.orchestra.run.smithi083.stdout:Total 27 MB/s | 46 MB 00:01 2024-09-20T18:41:07.163 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-09-20T18:41:07.183 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-09-20T18:41:07.184 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-09-20T18:41:07.550 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-09-20T18:41:07.550 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-09-20T18:41:08.054 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-09-20T18:41:08.137 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-20T18:41:08.182 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-20T18:41:08.209 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephsqlite-2:19.3.0-5099.g88deadb8.el9.x86_64 3/4 2024-09-20T18:41:09.155 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-test-2:19.3.0-5099.g88deadb8.el9.x86_64 4/4 2024-09-20T18:41:09.155 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-test-2:19.3.0-5099.g88deadb8.el9.x86_64 1/4 2024-09-20T18:41:09.155 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephsqlite-2:19.3.0-5099.g88deadb8.el9.x86_64 2/4 2024-09-20T18:41:09.155 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-20T18:41:09.504 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-20T18:41:09.504 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:41:09.504 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-09-20T18:41:09.504 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-test-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:41:09.504 INFO:teuthology.orchestra.run.smithi142.stdout: libcephsqlite-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:41:09.504 INFO:teuthology.orchestra.run.smithi142.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-20T18:41:09.504 INFO:teuthology.orchestra.run.smithi142.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-20T18:41:09.505 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:41:09.505 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-09-20T18:41:09.770 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install ceph 2024-09-20T18:41:10.352 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:00:38 ago on Fri 20 Sep 2024 06:40:32 PM UTC. 2024-09-20T18:41:10.488 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-09-20T18:41:10.491 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:41:10.491 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repository Size 2024-09-20T18:41:10.491 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:41:10.491 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-09-20T18:41:10.491 INFO:teuthology.orchestra.run.smithi142.stdout: ceph x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 6.6 k 2024-09-20T18:41:10.491 INFO:teuthology.orchestra.run.smithi142.stdout:Installing dependencies: 2024-09-20T18:41:10.491 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mds x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 2.4 M 2024-09-20T18:41:10.492 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 1.5 M 2024-09-20T18:41:10.492 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5099.g88deadb8.el9 ceph-noarch 292 k 2024-09-20T18:41:10.492 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mon x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 4.8 M 2024-09-20T18:41:10.492 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-osd x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 17 M 2024-09-20T18:41:10.492 INFO:teuthology.orchestra.run.smithi142.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-20T18:41:10.492 INFO:teuthology.orchestra.run.smithi142.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-20T18:41:10.492 INFO:teuthology.orchestra.run.smithi142.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-20T18:41:10.492 INFO:teuthology.orchestra.run.smithi142.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-20T18:41:10.492 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-20T18:41:10.492 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-20T18:41:10.492 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-20T18:41:10.492 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-20T18:41:10.492 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-20T18:41:10.493 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-20T18:41:10.493 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-collections 2024-09-20T18:41:10.493 INFO:teuthology.orchestra.run.smithi142.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-20T18:41:10.493 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-20T18:41:10.493 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-20T18:41:10.493 INFO:teuthology.orchestra.run.smithi142.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-20T18:41:10.493 INFO:teuthology.orchestra.run.smithi142.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-20T18:41:10.493 INFO:teuthology.orchestra.run.smithi142.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-20T18:41:10.493 INFO:teuthology.orchestra.run.smithi142.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-20T18:41:10.493 INFO:teuthology.orchestra.run.smithi142.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-20T18:41:10.493 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-20T18:41:10.493 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-20T18:41:10.493 INFO:teuthology.orchestra.run.smithi142.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-20T18:41:10.494 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-20T18:41:10.494 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-20T18:41:10.494 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-20T18:41:10.494 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-20T18:41:10.494 INFO:teuthology.orchestra.run.smithi142.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-20T18:41:10.494 INFO:teuthology.orchestra.run.smithi142.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-20T18:41:10.494 INFO:teuthology.orchestra.run.smithi142.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-20T18:41:10.494 INFO:teuthology.orchestra.run.smithi142.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-20T18:41:10.494 INFO:teuthology.orchestra.run.smithi142.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-20T18:41:10.494 INFO:teuthology.orchestra.run.smithi142.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-20T18:41:10.494 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:41:10.494 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-09-20T18:41:10.494 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:41:10.495 INFO:teuthology.orchestra.run.smithi142.stdout:Install 37 Packages 2024-09-20T18:41:10.495 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:41:10.495 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 31 M 2024-09-20T18:41:10.495 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 109 M 2024-09-20T18:41:10.495 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-09-20T18:41:11.075 INFO:teuthology.orchestra.run.smithi142.stdout:(1/37): ceph-19.3.0-5099.g88deadb8.el9.x86_64.r 49 kB/s | 6.6 kB 00:00 2024-09-20T18:41:11.225 INFO:teuthology.orchestra.run.smithi142.stdout:(2/37): ceph-mgr-19.3.0-5099.g88deadb8.el9.x86_ 5.4 MB/s | 1.5 MB 00:00 2024-09-20T18:41:11.283 INFO:teuthology.orchestra.run.smithi142.stdout:(3/37): ceph-mds-19.3.0-5099.g88deadb8.el9.x86_ 6.9 MB/s | 2.4 MB 00:00 2024-09-20T18:41:11.384 INFO:teuthology.orchestra.run.smithi142.stdout:(4/37): ceph-mgr-modules-core-19.3.0-5099.g88de 2.9 MB/s | 292 kB 00:00 2024-09-20T18:41:11.459 INFO:teuthology.orchestra.run.smithi142.stdout:(5/37): ceph-mon-19.3.0-5099.g88deadb8.el9.x86_ 12 MB/s | 4.8 MB 00:00 2024-09-20T18:41:11.593 INFO:teuthology.orchestra.run.smithi142.stdout:(6/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-09-20T18:41:11.652 INFO:teuthology.orchestra.run.smithi142.stdout:(7/37): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-09-20T18:41:11.818 INFO:teuthology.orchestra.run.smithi142.stdout:(8/37): ceph-osd-19.3.0-5099.g88deadb8.el9.x86_ 29 MB/s | 17 MB 00:00 2024-09-20T18:41:11.846 INFO:teuthology.orchestra.run.smithi142.stdout:(9/37): python3-pycparser-2.20-6.el9.noarch.rpm 704 kB/s | 135 kB 00:00 2024-09-20T18:41:11.877 INFO:teuthology.orchestra.run.smithi142.stdout:(10/37): python3-cryptography-36.0.1-4.el9.x86_ 2.9 MB/s | 1.2 MB 00:00 2024-09-20T18:41:11.902 INFO:teuthology.orchestra.run.smithi142.stdout:(11/37): python3-urllib3-1.26.5-6.el9.noarch.rp 3.6 MB/s | 215 kB 00:00 2024-09-20T18:41:11.961 INFO:teuthology.orchestra.run.smithi142.stdout:(12/37): python3-requests-2.25.1-8.el9.noarch.r 876 kB/s | 125 kB 00:00 2024-09-20T18:41:12.086 INFO:teuthology.orchestra.run.smithi142.stdout:(13/37): python3-markupsafe-1.1.1-12.el9.x86_64 278 kB/s | 35 kB 00:00 2024-09-20T18:41:12.111 INFO:teuthology.orchestra.run.smithi142.stdout:(14/37): lua-5.4.4-4.el9.x86_64.rpm 806 kB/s | 188 kB 00:00 2024-09-20T18:41:12.137 INFO:teuthology.orchestra.run.smithi142.stdout:(15/37): python3-mako-1.1.4-6.el9.noarch.rpm 735 kB/s | 172 kB 00:00 2024-09-20T18:41:12.162 INFO:teuthology.orchestra.run.smithi142.stdout:(16/37): python3-packaging-20.9-5.el9.noarch.rp 1.0 MB/s | 77 kB 00:00 2024-09-20T18:41:12.187 INFO:teuthology.orchestra.run.smithi142.stdout:(17/37): python3-toml-0.10.2-6.el9.noarch.rpm 830 kB/s | 42 kB 00:00 2024-09-20T18:41:12.213 INFO:teuthology.orchestra.run.smithi142.stdout:(18/37): python3-pytz-2021.1-5.el9.noarch.rpm 506 kB/s | 51 kB 00:00 2024-09-20T18:41:12.255 INFO:teuthology.orchestra.run.smithi142.stdout:(19/37): luarocks-3.9.2-5.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-09-20T18:41:12.288 INFO:teuthology.orchestra.run.smithi142.stdout:(20/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 5.3 MB/s | 172 kB 00:00 2024-09-20T18:41:12.313 INFO:teuthology.orchestra.run.smithi142.stdout:(21/37): lua-devel-5.4.4-4.el9.x86_64.rpm 147 kB/s | 22 kB 00:00 2024-09-20T18:41:12.339 INFO:teuthology.orchestra.run.smithi142.stdout:(22/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 346 kB/s | 43 kB 00:00 2024-09-20T18:41:12.372 INFO:teuthology.orchestra.run.smithi142.stdout:(23/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 321 kB/s | 11 kB 00:00 2024-09-20T18:41:12.398 INFO:teuthology.orchestra.run.smithi142.stdout:(24/37): python3-jaraco-collections-3.0.0-8.el9 990 kB/s | 23 kB 00:00 2024-09-20T18:41:12.423 INFO:teuthology.orchestra.run.smithi142.stdout:(25/37): python3-jaraco-classes-3.2.1-5.el9.noa 211 kB/s | 18 kB 00:00 2024-09-20T18:41:12.448 INFO:teuthology.orchestra.run.smithi142.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 388 kB/s | 19 kB 00:00 2024-09-20T18:41:12.473 INFO:teuthology.orchestra.run.smithi142.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 391 kB/s | 20 kB 00:00 2024-09-20T18:41:12.498 INFO:teuthology.orchestra.run.smithi142.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 922 kB/s | 46 kB 00:00 2024-09-20T18:41:12.524 INFO:teuthology.orchestra.run.smithi142.stdout:(29/37): python3-cherrypy-18.6.1-2.el9.noarch.r 1.5 MB/s | 358 kB 00:00 2024-09-20T18:41:12.549 INFO:teuthology.orchestra.run.smithi142.stdout:(30/37): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-09-20T18:41:12.574 INFO:teuthology.orchestra.run.smithi142.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 328 kB/s | 16 kB 00:00 2024-09-20T18:41:12.595 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5099.g88deadb8.el9.x86_64 3/4 2024-09-20T18:41:12.599 INFO:teuthology.orchestra.run.smithi142.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.8 MB/s | 90 kB 00:00 2024-09-20T18:41:12.625 INFO:teuthology.orchestra.run.smithi142.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 711 kB/s | 36 kB 00:00 2024-09-20T18:41:12.650 INFO:teuthology.orchestra.run.smithi142.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-09-20T18:41:12.666 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-test-2:19.3.0-5099.g88deadb8.el9.x86_64 4/4 2024-09-20T18:41:12.675 INFO:teuthology.orchestra.run.smithi142.stdout:(35/37): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-20T18:41:12.700 INFO:teuthology.orchestra.run.smithi142.stdout:(36/37): python3-zc-lockfile-2.0-10.el9.noarch. 400 kB/s | 20 kB 00:00 2024-09-20T18:41:12.726 INFO:teuthology.orchestra.run.smithi142.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.1 MB/s | 427 kB 00:00 2024-09-20T18:41:12.729 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-09-20T18:41:12.729 INFO:teuthology.orchestra.run.smithi142.stdout:Total 14 MB/s | 31 MB 00:02 2024-09-20T18:41:12.877 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-09-20T18:41:12.916 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-09-20T18:41:12.916 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-09-20T18:41:13.354 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-09-20T18:41:13.355 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-09-20T18:41:14.640 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-09-20T18:41:14.695 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-20T18:41:14.727 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-20T18:41:14.764 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-20T18:41:14.836 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-20T18:41:14.888 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-20T18:41:14.920 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-20T18:41:15.011 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-20T18:41:15.098 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-20T18:41:15.113 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-test-2:19.3.0-5099.g88deadb8.el9.x86_64 4/4 2024-09-20T18:41:15.113 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-test-2:19.3.0-5099.g88deadb8.el9.x86_64 1/4 2024-09-20T18:41:15.113 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephsqlite-2:19.3.0-5099.g88deadb8.el9.x86_64 2/4 2024-09-20T18:41:15.113 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-20T18:41:15.139 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-20T18:41:15.175 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-20T18:41:15.205 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-20T18:41:15.239 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-20T18:41:15.272 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-20T18:41:15.304 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-20T18:41:15.339 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-20T18:41:15.374 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-20T18:41:15.422 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-20T18:41:15.454 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-20T18:41:15.526 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-20T18:41:15.527 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-20T18:41:15.527 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:41:15.527 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2024-09-20T18:41:15.527 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-test-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:41:15.527 INFO:teuthology.orchestra.run.smithi083.stdout: libcephsqlite-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:41:15.527 INFO:teuthology.orchestra.run.smithi083.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-20T18:41:15.527 INFO:teuthology.orchestra.run.smithi083.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-20T18:41:15.528 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:41:15.528 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-20T18:41:15.666 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-20T18:41:15.745 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-20T18:41:15.799 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-20T18:41:15.806 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph 2024-09-20T18:41:15.837 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-20T18:41:15.886 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-20T18:41:15.941 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-20T18:41:16.126 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-20T18:41:16.183 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-20T18:41:16.252 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-20T18:41:16.341 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-20T18:41:16.392 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-20T18:41:16.408 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:00:39 ago on Fri 20 Sep 2024 06:40:37 PM UTC. 2024-09-20T18:41:16.519 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-20T18:41:16.547 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-20T18:41:16.550 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:41:16.550 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repository Size 2024-09-20T18:41:16.550 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:41:16.550 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2024-09-20T18:41:16.550 INFO:teuthology.orchestra.run.smithi083.stdout: ceph x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 6.6 k 2024-09-20T18:41:16.550 INFO:teuthology.orchestra.run.smithi083.stdout:Installing dependencies: 2024-09-20T18:41:16.550 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mds x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 2.4 M 2024-09-20T18:41:16.550 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 1.5 M 2024-09-20T18:41:16.551 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5099.g88deadb8.el9 ceph-noarch 292 k 2024-09-20T18:41:16.551 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mon x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 4.8 M 2024-09-20T18:41:16.551 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-osd x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 17 M 2024-09-20T18:41:16.551 INFO:teuthology.orchestra.run.smithi083.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-20T18:41:16.551 INFO:teuthology.orchestra.run.smithi083.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-20T18:41:16.551 INFO:teuthology.orchestra.run.smithi083.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-20T18:41:16.551 INFO:teuthology.orchestra.run.smithi083.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-20T18:41:16.551 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-20T18:41:16.551 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-20T18:41:16.551 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-20T18:41:16.551 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-20T18:41:16.551 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-20T18:41:16.551 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-20T18:41:16.551 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-collections 2024-09-20T18:41:16.552 INFO:teuthology.orchestra.run.smithi083.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-20T18:41:16.552 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-20T18:41:16.552 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-20T18:41:16.552 INFO:teuthology.orchestra.run.smithi083.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-20T18:41:16.552 INFO:teuthology.orchestra.run.smithi083.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-20T18:41:16.552 INFO:teuthology.orchestra.run.smithi083.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-20T18:41:16.552 INFO:teuthology.orchestra.run.smithi083.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-20T18:41:16.552 INFO:teuthology.orchestra.run.smithi083.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-20T18:41:16.552 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-20T18:41:16.552 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-20T18:41:16.552 INFO:teuthology.orchestra.run.smithi083.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-20T18:41:16.552 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-20T18:41:16.552 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-20T18:41:16.552 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-20T18:41:16.552 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-20T18:41:16.553 INFO:teuthology.orchestra.run.smithi083.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-20T18:41:16.553 INFO:teuthology.orchestra.run.smithi083.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-20T18:41:16.553 INFO:teuthology.orchestra.run.smithi083.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-20T18:41:16.553 INFO:teuthology.orchestra.run.smithi083.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-20T18:41:16.553 INFO:teuthology.orchestra.run.smithi083.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-20T18:41:16.553 INFO:teuthology.orchestra.run.smithi083.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-20T18:41:16.553 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:41:16.553 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-09-20T18:41:16.553 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:41:16.553 INFO:teuthology.orchestra.run.smithi083.stdout:Install 37 Packages 2024-09-20T18:41:16.553 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:41:16.554 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 31 M 2024-09-20T18:41:16.554 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 109 M 2024-09-20T18:41:16.554 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-09-20T18:41:16.693 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5099.g88deadb8.el9. 32/37 2024-09-20T18:41:16.718 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-2:19.3.0-5099.g88deadb8.el9.x86_64 33/37 2024-09-20T18:41:16.744 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5099.g88deadb8.el9.x86_64 33/37 2024-09-20T18:41:16.744 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-20T18:41:16.744 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-20T18:41:16.744 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-20T18:41:16.744 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-20T18:41:16.745 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:41:17.621 INFO:teuthology.orchestra.run.smithi083.stdout:(1/37): ceph-19.3.0-5099.g88deadb8.el9.x86_64.r 53 kB/s | 6.6 kB 00:00 2024-09-20T18:41:17.804 INFO:teuthology.orchestra.run.smithi083.stdout:(2/37): ceph-mgr-19.3.0-5099.g88deadb8.el9.x86_ 5.0 MB/s | 1.5 MB 00:00 2024-09-20T18:41:17.871 INFO:teuthology.orchestra.run.smithi083.stdout:(3/37): ceph-mds-19.3.0-5099.g88deadb8.el9.x86_ 6.4 MB/s | 2.4 MB 00:00 2024-09-20T18:41:17.963 INFO:teuthology.orchestra.run.smithi083.stdout:(4/37): ceph-mgr-modules-core-19.3.0-5099.g88de 3.1 MB/s | 292 kB 00:00 2024-09-20T18:41:18.218 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-osd-2:19.3.0-5099.g88deadb8.el9.x86_64 34/37 2024-09-20T18:41:18.246 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-osd-2:19.3.0-5099.g88deadb8.el9.x86_64 34/37 2024-09-20T18:41:18.246 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-20T18:41:18.246 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-20T18:41:18.246 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-20T18:41:18.246 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-20T18:41:18.246 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:41:18.355 INFO:teuthology.orchestra.run.smithi083.stdout:(5/37): ceph-osd-19.3.0-5099.g88deadb8.el9.x86_ 31 MB/s | 17 MB 00:00 2024-09-20T18:41:18.422 INFO:teuthology.orchestra.run.smithi083.stdout:(6/37): ceph-mon-19.3.0-5099.g88deadb8.el9.x86_ 6.0 MB/s | 4.8 MB 00:00 2024-09-20T18:41:18.647 INFO:teuthology.orchestra.run.smithi083.stdout:(7/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 370 kB/s | 253 kB 00:00 2024-09-20T18:41:18.694 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mon-2:19.3.0-5099.g88deadb8.el9.x86_64 35/37 2024-09-20T18:41:18.719 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mon-2:19.3.0-5099.g88deadb8.el9.x86_64 35/37 2024-09-20T18:41:18.719 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-20T18:41:18.719 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-20T18:41:18.719 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-20T18:41:18.719 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-20T18:41:18.719 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:41:18.773 INFO:teuthology.orchestra.run.smithi083.stdout:(8/37): python3-pycparser-2.20-6.el9.noarch.rpm 1.1 MB/s | 135 kB 00:00 2024-09-20T18:41:18.882 INFO:teuthology.orchestra.run.smithi083.stdout:(9/37): python3-requests-2.25.1-8.el9.noarch.rp 1.1 MB/s | 125 kB 00:00 2024-09-20T18:41:18.907 INFO:teuthology.orchestra.run.smithi083.stdout:(10/37): python3-ply-3.11-14.el9.noarch.rpm 220 kB/s | 106 kB 00:00 2024-09-20T18:41:18.945 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mds-2:19.3.0-5099.g88deadb8.el9.x86_64 36/37 2024-09-20T18:41:18.968 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mds-2:19.3.0-5099.g88deadb8.el9.x86_64 36/37 2024-09-20T18:41:18.969 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-20T18:41:18.969 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-20T18:41:18.969 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-20T18:41:18.969 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-20T18:41:18.969 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:41:19.007 INFO:teuthology.orchestra.run.smithi083.stdout:(11/37): python3-urllib3-1.26.5-6.el9.noarch.rp 1.7 MB/s | 215 kB 00:00 2024-09-20T18:41:19.034 INFO:teuthology.orchestra.run.smithi083.stdout:(12/37): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-09-20T18:41:19.035 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-2:19.3.0-5099.g88deadb8.el9.x86_64 37/37 2024-09-20T18:41:19.065 INFO:teuthology.orchestra.run.smithi083.stdout:(13/37): python3-markupsafe-1.1.1-12.el9.x86_64 1.0 MB/s | 35 kB 00:00 2024-09-20T18:41:19.108 INFO:teuthology.orchestra.run.smithi083.stdout:(14/37): python3-packaging-20.9-5.el9.noarch.rp 1.8 MB/s | 77 kB 00:00 2024-09-20T18:41:19.141 INFO:teuthology.orchestra.run.smithi083.stdout:(15/37): python3-pytz-2021.1-5.el9.noarch.rpm 1.5 MB/s | 51 kB 00:00 2024-09-20T18:41:19.167 INFO:teuthology.orchestra.run.smithi083.stdout:(16/37): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-09-20T18:41:19.217 INFO:teuthology.orchestra.run.smithi083.stdout:(17/37): python3-toml-0.10.2-6.el9.noarch.rpm 553 kB/s | 42 kB 00:00 2024-09-20T18:41:19.259 INFO:teuthology.orchestra.run.smithi083.stdout:(18/37): luarocks-3.9.2-5.el9.noarch.rpm 3.6 MB/s | 151 kB 00:00 2024-09-20T18:41:19.284 INFO:teuthology.orchestra.run.smithi083.stdout:(19/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 1.7 MB/s | 43 kB 00:00 2024-09-20T18:41:19.309 INFO:teuthology.orchestra.run.smithi083.stdout:(20/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 6.7 MB/s | 172 kB 00:00 2024-09-20T18:41:19.335 INFO:teuthology.orchestra.run.smithi083.stdout:(21/37): lua-devel-5.4.4-4.el9.x86_64.rpm 133 kB/s | 22 kB 00:00 2024-09-20T18:41:19.377 INFO:teuthology.orchestra.run.smithi083.stdout:(22/37): python3-cryptography-36.0.1-4.el9.x86_ 1.2 MB/s | 1.2 MB 00:01 2024-09-20T18:41:19.419 INFO:teuthology.orchestra.run.smithi083.stdout:(23/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 128 kB/s | 11 kB 00:00 2024-09-20T18:41:19.444 INFO:teuthology.orchestra.run.smithi083.stdout:(24/37): python3-cherrypy-18.6.1-2.el9.noarch.r 2.6 MB/s | 358 kB 00:00 2024-09-20T18:41:19.469 INFO:teuthology.orchestra.run.smithi083.stdout:(25/37): python3-jaraco-classes-3.2.1-5.el9.noa 208 kB/s | 18 kB 00:00 2024-09-20T18:41:19.494 INFO:teuthology.orchestra.run.smithi083.stdout:(26/37): python3-jaraco-collections-3.0.0-8.el9 314 kB/s | 23 kB 00:00 2024-09-20T18:41:19.522 INFO:teuthology.orchestra.run.smithi083.stdout:(27/37): python3-jaraco-functools-3.5.0-2.el9.n 251 kB/s | 19 kB 00:00 2024-09-20T18:41:19.547 INFO:teuthology.orchestra.run.smithi083.stdout:(28/37): python3-jaraco-text-3.2.0-6.el9.noarch 254 kB/s | 20 kB 00:00 2024-09-20T18:41:19.572 INFO:teuthology.orchestra.run.smithi083.stdout:(29/37): python3-logutils-0.3.5-21.el9.noarch.r 596 kB/s | 46 kB 00:00 2024-09-20T18:41:19.598 INFO:teuthology.orchestra.run.smithi083.stdout:(30/37): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-09-20T18:41:19.623 INFO:teuthology.orchestra.run.smithi083.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 327 kB/s | 16 kB 00:00 2024-09-20T18:41:19.648 INFO:teuthology.orchestra.run.smithi083.stdout:(32/37): python3-tempora-5.0.0-2.el9.noarch.rpm 1.4 MB/s | 36 kB 00:00 2024-09-20T18:41:19.673 INFO:teuthology.orchestra.run.smithi083.stdout:(33/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-09-20T18:41:19.707 INFO:teuthology.orchestra.run.smithi083.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-09-20T18:41:19.732 INFO:teuthology.orchestra.run.smithi083.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 803 kB/s | 20 kB 00:00 2024-09-20T18:41:19.757 INFO:teuthology.orchestra.run.smithi083.stdout:(36/37): python3-webob-1.8.8-2.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-09-20T18:41:19.783 INFO:teuthology.orchestra.run.smithi083.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.8 MB/s | 427 kB 00:00 2024-09-20T18:41:19.787 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-09-20T18:41:19.787 INFO:teuthology.orchestra.run.smithi083.stdout:Total 9.5 MB/s | 31 MB 00:03 2024-09-20T18:41:19.944 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-09-20T18:41:19.985 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-09-20T18:41:19.985 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-09-20T18:41:20.463 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-09-20T18:41:20.464 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-09-20T18:41:20.708 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-2:19.3.0-5099.g88deadb8.el9.x86_64 37/37 2024-09-20T18:41:20.708 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-2:19.3.0-5099.g88deadb8.el9.x86_64 1/37 2024-09-20T18:41:20.708 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mds-2:19.3.0-5099.g88deadb8.el9.x86_64 2/37 2024-09-20T18:41:20.708 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-2:19.3.0-5099.g88deadb8.el9.x86_64 3/37 2024-09-20T18:41:20.708 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mon-2:19.3.0-5099.g88deadb8.el9.x86_64 4/37 2024-09-20T18:41:20.708 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-osd-2:19.3.0-5099.g88deadb8.el9.x86_64 5/37 2024-09-20T18:41:20.708 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5099.g88deadb8.el9. 6/37 2024-09-20T18:41:20.708 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-20T18:41:20.708 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-20T18:41:20.708 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-20T18:41:20.709 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-20T18:41:20.709 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-20T18:41:20.709 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-20T18:41:20.709 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-20T18:41:20.709 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-20T18:41:20.709 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-20T18:41:20.709 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-20T18:41:20.709 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-20T18:41:20.709 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-20T18:41:20.709 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-20T18:41:20.709 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-20T18:41:20.709 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-20T18:41:20.709 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-20T18:41:20.709 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-20T18:41:20.710 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-20T18:41:20.710 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-20T18:41:20.711 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-20T18:41:20.711 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-20T18:41:20.711 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-20T18:41:20.711 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-20T18:41:20.711 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-20T18:41:20.711 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-20T18:41:20.711 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-20T18:41:20.711 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-20T18:41:20.711 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-20T18:41:20.712 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-20T18:41:20.712 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-20T18:41:21.592 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-20T18:41:21.593 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:41:21.593 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-09-20T18:41:21.593 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:41:21.593 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mds-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:41:21.593 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:41:21.593 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-modules-core-2:19.3.0-5099.g88deadb8.el9.noarch 2024-09-20T18:41:21.593 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mon-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:41:21.593 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-osd-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:41:21.593 INFO:teuthology.orchestra.run.smithi142.stdout: lua-5.4.4-4.el9.x86_64 2024-09-20T18:41:21.593 INFO:teuthology.orchestra.run.smithi142.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-20T18:41:21.593 INFO:teuthology.orchestra.run.smithi142.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-20T18:41:21.594 INFO:teuthology.orchestra.run.smithi142.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-20T18:41:21.594 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-20T18:41:21.594 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-20T18:41:21.594 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-20T18:41:21.594 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-20T18:41:21.594 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-20T18:41:21.594 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-20T18:41:21.594 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-20T18:41:21.594 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-20T18:41:21.594 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-20T18:41:21.594 INFO:teuthology.orchestra.run.smithi142.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-20T18:41:21.594 INFO:teuthology.orchestra.run.smithi142.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-20T18:41:21.594 INFO:teuthology.orchestra.run.smithi142.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-20T18:41:21.595 INFO:teuthology.orchestra.run.smithi142.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-20T18:41:21.595 INFO:teuthology.orchestra.run.smithi142.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-20T18:41:21.595 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-20T18:41:21.595 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ply-3.11-14.el9.noarch 2024-09-20T18:41:21.595 INFO:teuthology.orchestra.run.smithi142.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-20T18:41:21.595 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-20T18:41:21.595 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-20T18:41:21.595 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-20T18:41:21.595 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-20T18:41:21.595 INFO:teuthology.orchestra.run.smithi142.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-20T18:41:21.595 INFO:teuthology.orchestra.run.smithi142.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-20T18:41:21.596 INFO:teuthology.orchestra.run.smithi142.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-20T18:41:21.596 INFO:teuthology.orchestra.run.smithi142.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-20T18:41:21.596 INFO:teuthology.orchestra.run.smithi142.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-20T18:41:21.596 INFO:teuthology.orchestra.run.smithi142.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-20T18:41:21.596 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:41:21.596 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-09-20T18:41:21.745 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-09-20T18:41:21.811 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-20T18:41:21.843 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-20T18:41:21.871 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-20T18:41:21.942 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-20T18:41:21.975 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install ceph-base 2024-09-20T18:41:21.987 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-20T18:41:22.019 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-20T18:41:22.111 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-20T18:41:22.182 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-20T18:41:22.231 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-20T18:41:22.266 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-20T18:41:22.296 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-20T18:41:22.330 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-20T18:41:22.363 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-20T18:41:22.404 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-20T18:41:22.448 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-20T18:41:22.482 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-20T18:41:22.530 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-20T18:41:22.565 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:00:50 ago on Fri 20 Sep 2024 06:40:32 PM UTC. 2024-09-20T18:41:22.570 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-20T18:41:22.644 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-20T18:41:22.655 INFO:teuthology.orchestra.run.smithi142.stdout:Package ceph-base-2:19.3.0-5099.g88deadb8.el9.x86_64 is already installed. 2024-09-20T18:41:22.697 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-09-20T18:41:22.698 INFO:teuthology.orchestra.run.smithi142.stdout:Nothing to do. 2024-09-20T18:41:22.698 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-09-20T18:41:22.761 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install cephadm 2024-09-20T18:41:22.799 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-20T18:41:22.887 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-20T18:41:22.940 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-20T18:41:22.979 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-20T18:41:23.036 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-20T18:41:23.099 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-20T18:41:23.287 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-20T18:41:23.333 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-20T18:41:23.339 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:00:51 ago on Fri 20 Sep 2024 06:40:32 PM UTC. 2024-09-20T18:41:23.402 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-20T18:41:23.469 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-09-20T18:41:23.470 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:41:23.470 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repository Size 2024-09-20T18:41:23.470 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:41:23.470 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-09-20T18:41:23.470 INFO:teuthology.orchestra.run.smithi142.stdout: cephadm noarch 2:19.3.0-5099.g88deadb8.el9 ceph-noarch 784 k 2024-09-20T18:41:23.470 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:41:23.470 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-09-20T18:41:23.470 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:41:23.470 INFO:teuthology.orchestra.run.smithi142.stdout:Install 1 Package 2024-09-20T18:41:23.470 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:41:23.470 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 784 k 2024-09-20T18:41:23.470 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 791 k 2024-09-20T18:41:23.470 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-09-20T18:41:23.499 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-20T18:41:23.551 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-20T18:41:23.735 INFO:teuthology.orchestra.run.smithi142.stdout:cephadm-19.3.0-5099.g88deadb8.el9.noarch.rpm 2.9 MB/s | 784 kB 00:00 2024-09-20T18:41:23.735 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-09-20T18:41:23.735 INFO:teuthology.orchestra.run.smithi142.stdout:Total 2.9 MB/s | 784 kB 00:00 2024-09-20T18:41:23.736 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-09-20T18:41:23.740 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-09-20T18:41:23.741 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-09-20T18:41:23.747 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-09-20T18:41:23.747 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-09-20T18:41:23.847 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-09-20T18:41:23.878 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-20T18:41:24.064 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5099.g88deadb8.el9. 32/37 2024-09-20T18:41:24.093 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-2:19.3.0-5099.g88deadb8.el9.x86_64 33/37 2024-09-20T18:41:24.119 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5099.g88deadb8.el9.x86_64 33/37 2024-09-20T18:41:24.119 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-20T18:41:24.119 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-20T18:41:24.119 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-20T18:41:24.119 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-20T18:41:24.119 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:41:24.264 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: cephadm-2:19.3.0-5099.g88deadb8.el9.noarch 1/1 2024-09-20T18:41:24.347 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephadm-2:19.3.0-5099.g88deadb8.el9.noarch 1/1 2024-09-20T18:41:24.709 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: cephadm-2:19.3.0-5099.g88deadb8.el9.noarch 1/1 2024-09-20T18:41:24.945 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephadm-2:19.3.0-5099.g88deadb8.el9.noarch 1/1 2024-09-20T18:41:24.945 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:41:24.945 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-09-20T18:41:24.945 INFO:teuthology.orchestra.run.smithi142.stdout: cephadm-2:19.3.0-5099.g88deadb8.el9.noarch 2024-09-20T18:41:24.946 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:41:24.946 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-09-20T18:41:25.133 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install ceph-immutable-object-cache 2024-09-20T18:41:25.585 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-osd-2:19.3.0-5099.g88deadb8.el9.x86_64 34/37 2024-09-20T18:41:25.613 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-osd-2:19.3.0-5099.g88deadb8.el9.x86_64 34/37 2024-09-20T18:41:25.613 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-20T18:41:25.613 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-20T18:41:25.613 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-20T18:41:25.613 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-20T18:41:25.613 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:41:25.723 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:00:53 ago on Fri 20 Sep 2024 06:40:32 PM UTC. 2024-09-20T18:41:25.858 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-09-20T18:41:25.859 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:41:25.859 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repo Size 2024-09-20T18:41:25.859 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:41:25.859 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-09-20T18:41:25.860 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 145 k 2024-09-20T18:41:25.860 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:41:25.860 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-09-20T18:41:25.860 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:41:25.860 INFO:teuthology.orchestra.run.smithi142.stdout:Install 1 Package 2024-09-20T18:41:25.860 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:41:25.860 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 145 k 2024-09-20T18:41:25.860 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 432 k 2024-09-20T18:41:25.860 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-09-20T18:41:26.074 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-immutable-object-cache-19.3.0-5099.g88dead 679 kB/s | 145 kB 00:00 2024-09-20T18:41:26.075 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-09-20T18:41:26.075 INFO:teuthology.orchestra.run.smithi142.stdout:Total 674 kB/s | 145 kB 00:00 2024-09-20T18:41:26.075 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-09-20T18:41:26.082 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-09-20T18:41:26.082 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-09-20T18:41:26.086 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mon-2:19.3.0-5099.g88deadb8.el9.x86_64 35/37 2024-09-20T18:41:26.109 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mon-2:19.3.0-5099.g88deadb8.el9.x86_64 35/37 2024-09-20T18:41:26.110 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-20T18:41:26.110 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-20T18:41:26.110 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-20T18:41:26.110 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-20T18:41:26.110 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:41:26.140 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-09-20T18:41:26.140 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-09-20T18:41:26.283 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-09-20T18:41:26.337 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mds-2:19.3.0-5099.g88deadb8.el9.x86_64 36/37 2024-09-20T18:41:26.350 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5099.g88deadb8. 1/1 2024-09-20T18:41:26.360 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mds-2:19.3.0-5099.g88deadb8.el9.x86_64 36/37 2024-09-20T18:41:26.360 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-20T18:41:26.360 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-20T18:41:26.360 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-20T18:41:26.360 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-20T18:41:26.361 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:41:26.375 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5099.g88deadb8. 1/1 2024-09-20T18:41:26.375 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-20T18:41:26.375 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-20T18:41:26.375 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:41:26.433 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-2:19.3.0-5099.g88deadb8.el9.x86_64 37/37 2024-09-20T18:41:27.268 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5099.g88deadb8. 1/1 2024-09-20T18:41:27.268 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:41:27.268 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-09-20T18:41:27.268 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-immutable-object-cache-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:41:27.268 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:41:27.268 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-09-20T18:41:27.466 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install ceph-mgr 2024-09-20T18:41:28.057 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-2:19.3.0-5099.g88deadb8.el9.x86_64 37/37 2024-09-20T18:41:28.057 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-2:19.3.0-5099.g88deadb8.el9.x86_64 1/37 2024-09-20T18:41:28.058 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mds-2:19.3.0-5099.g88deadb8.el9.x86_64 2/37 2024-09-20T18:41:28.058 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-2:19.3.0-5099.g88deadb8.el9.x86_64 3/37 2024-09-20T18:41:28.058 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mon-2:19.3.0-5099.g88deadb8.el9.x86_64 4/37 2024-09-20T18:41:28.058 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-osd-2:19.3.0-5099.g88deadb8.el9.x86_64 5/37 2024-09-20T18:41:28.058 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5099.g88deadb8.el9. 6/37 2024-09-20T18:41:28.058 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-20T18:41:28.058 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-20T18:41:28.058 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-20T18:41:28.058 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-20T18:41:28.058 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-20T18:41:28.058 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-20T18:41:28.059 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-20T18:41:28.059 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-20T18:41:28.059 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-20T18:41:28.059 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-20T18:41:28.059 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-20T18:41:28.059 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-20T18:41:28.059 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-20T18:41:28.059 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-20T18:41:28.059 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-20T18:41:28.059 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-20T18:41:28.059 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-20T18:41:28.059 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-20T18:41:28.059 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-20T18:41:28.059 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-20T18:41:28.060 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-20T18:41:28.060 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-20T18:41:28.060 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-20T18:41:28.061 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-20T18:41:28.061 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-20T18:41:28.061 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-20T18:41:28.061 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-20T18:41:28.061 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-20T18:41:28.061 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-20T18:41:28.061 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-20T18:41:28.063 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:00:56 ago on Fri 20 Sep 2024 06:40:32 PM UTC. 2024-09-20T18:41:28.159 INFO:teuthology.orchestra.run.smithi142.stdout:Package ceph-mgr-2:19.3.0-5099.g88deadb8.el9.x86_64 is already installed. 2024-09-20T18:41:28.201 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-09-20T18:41:28.202 INFO:teuthology.orchestra.run.smithi142.stdout:Nothing to do. 2024-09-20T18:41:28.202 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-09-20T18:41:28.266 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install ceph-mgr-dashboard 2024-09-20T18:41:28.839 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:00:56 ago on Fri 20 Sep 2024 06:40:32 PM UTC. 2024-09-20T18:41:28.970 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-09-20T18:41:28.971 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:41:28.972 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repository Size 2024-09-20T18:41:28.972 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:41:28.972 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-09-20T18:41:28.972 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5099.g88deadb8.el9 ceph-noarch 14 M 2024-09-20T18:41:28.972 INFO:teuthology.orchestra.run.smithi142.stdout:Installing dependencies: 2024-09-20T18:41:28.972 INFO:teuthology.orchestra.run.smithi142.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-20T18:41:28.972 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5099.g88deadb8.el9 ceph-noarch 34 k 2024-09-20T18:41:28.972 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5099.g88deadb8.el9 ceph-noarch 16 k 2024-09-20T18:41:28.973 INFO:teuthology.orchestra.run.smithi142.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-20T18:41:28.973 INFO:teuthology.orchestra.run.smithi142.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-20T18:41:28.973 INFO:teuthology.orchestra.run.smithi142.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-20T18:41:28.973 INFO:teuthology.orchestra.run.smithi142.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-20T18:41:28.973 INFO:teuthology.orchestra.run.smithi142.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-20T18:41:28.973 INFO:teuthology.orchestra.run.smithi142.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-20T18:41:28.973 INFO:teuthology.orchestra.run.smithi142.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-20T18:41:28.973 INFO:teuthology.orchestra.run.smithi142.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-20T18:41:28.973 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:41:28.973 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-09-20T18:41:28.973 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:41:28.974 INFO:teuthology.orchestra.run.smithi142.stdout:Install 12 Packages 2024-09-20T18:41:28.974 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:41:28.974 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 19 M 2024-09-20T18:41:28.974 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 159 M 2024-09-20T18:41:28.974 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-09-20T18:41:29.173 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-20T18:41:29.173 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:41:29.173 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2024-09-20T18:41:29.173 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:41:29.173 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mds-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:41:29.174 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:41:29.174 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-modules-core-2:19.3.0-5099.g88deadb8.el9.noarch 2024-09-20T18:41:29.174 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mon-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:41:29.174 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-osd-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:41:29.174 INFO:teuthology.orchestra.run.smithi083.stdout: lua-5.4.4-4.el9.x86_64 2024-09-20T18:41:29.174 INFO:teuthology.orchestra.run.smithi083.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-20T18:41:29.174 INFO:teuthology.orchestra.run.smithi083.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-20T18:41:29.174 INFO:teuthology.orchestra.run.smithi083.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-20T18:41:29.174 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-20T18:41:29.174 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-20T18:41:29.174 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-20T18:41:29.174 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-20T18:41:29.174 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-20T18:41:29.175 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-20T18:41:29.175 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-20T18:41:29.175 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-20T18:41:29.175 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-20T18:41:29.175 INFO:teuthology.orchestra.run.smithi083.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-20T18:41:29.175 INFO:teuthology.orchestra.run.smithi083.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-20T18:41:29.175 INFO:teuthology.orchestra.run.smithi083.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-20T18:41:29.175 INFO:teuthology.orchestra.run.smithi083.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-20T18:41:29.175 INFO:teuthology.orchestra.run.smithi083.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-20T18:41:29.175 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-20T18:41:29.175 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ply-3.11-14.el9.noarch 2024-09-20T18:41:29.175 INFO:teuthology.orchestra.run.smithi083.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-20T18:41:29.175 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-20T18:41:29.175 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-20T18:41:29.176 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-20T18:41:29.176 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-20T18:41:29.176 INFO:teuthology.orchestra.run.smithi083.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-20T18:41:29.176 INFO:teuthology.orchestra.run.smithi083.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-20T18:41:29.176 INFO:teuthology.orchestra.run.smithi083.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-20T18:41:29.176 INFO:teuthology.orchestra.run.smithi083.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-20T18:41:29.176 INFO:teuthology.orchestra.run.smithi083.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-20T18:41:29.176 INFO:teuthology.orchestra.run.smithi083.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-20T18:41:29.176 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:41:29.176 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-20T18:41:29.612 INFO:teuthology.orchestra.run.smithi142.stdout:(1/12): ceph-grafana-dashboards-19.3.0-5099.g88 212 kB/s | 34 kB 00:00 2024-09-20T18:41:29.627 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-base 2024-09-20T18:41:29.637 INFO:teuthology.orchestra.run.smithi142.stdout:(2/12): ceph-prometheus-alerts-19.3.0-5099.g88d 88 kB/s | 16 kB 00:00 2024-09-20T18:41:30.130 INFO:teuthology.orchestra.run.smithi142.stdout:(3/12): ceph-mgr-dashboard-19.3.0-5099.g88deadb 21 MB/s | 14 MB 00:00 2024-09-20T18:41:30.212 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:00:53 ago on Fri 20 Sep 2024 06:40:37 PM UTC. 2024-09-20T18:41:30.288 INFO:teuthology.orchestra.run.smithi142.stdout:(4/12): python3-protobuf-3.14.0-13.el9.noarch.r 414 kB/s | 269 kB 00:00 2024-09-20T18:41:30.301 INFO:teuthology.orchestra.run.smithi083.stdout:Package ceph-base-2:19.3.0-5099.g88deadb8.el9.x86_64 is already installed. 2024-09-20T18:41:30.338 INFO:teuthology.orchestra.run.smithi142.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 11 MB/s | 548 kB 00:00 2024-09-20T18:41:30.342 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-20T18:41:30.343 INFO:teuthology.orchestra.run.smithi083.stdout:Nothing to do. 2024-09-20T18:41:30.343 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-20T18:41:30.363 INFO:teuthology.orchestra.run.smithi142.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 780 kB/s | 19 kB 00:00 2024-09-20T18:41:30.413 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install cephadm 2024-09-20T18:41:30.430 INFO:teuthology.orchestra.run.smithi142.stdout:(7/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 34 MB/s | 2.0 MB 00:00 2024-09-20T18:41:30.455 INFO:teuthology.orchestra.run.smithi142.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 5.6 MB/s | 144 kB 00:00 2024-09-20T18:41:30.481 INFO:teuthology.orchestra.run.smithi142.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 1.2 MB/s | 31 kB 00:00 2024-09-20T18:41:30.506 INFO:teuthology.orchestra.run.smithi142.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 7.3 MB/s | 188 kB 00:00 2024-09-20T18:41:30.581 INFO:teuthology.orchestra.run.smithi142.stdout:(11/12): protobuf-3.14.0-13.el9.x86_64.rpm 1.0 MB/s | 1.0 MB 00:00 2024-09-20T18:41:30.987 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:00:53 ago on Fri 20 Sep 2024 06:40:37 PM UTC. 2024-09-20T18:41:31.048 INFO:teuthology.orchestra.run.smithi142.stdout:(12/12): protobuf-compiler-3.14.0-13.el9.x86_64 940 kB/s | 863 kB 00:00 2024-09-20T18:41:31.052 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-09-20T18:41:31.052 INFO:teuthology.orchestra.run.smithi142.stdout:Total 9.3 MB/s | 19 MB 00:02 2024-09-20T18:41:31.101 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-09-20T18:41:31.118 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-20T18:41:31.119 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:41:31.119 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repository Size 2024-09-20T18:41:31.119 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:41:31.119 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2024-09-20T18:41:31.119 INFO:teuthology.orchestra.run.smithi083.stdout: cephadm noarch 2:19.3.0-5099.g88deadb8.el9 ceph-noarch 784 k 2024-09-20T18:41:31.119 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:41:31.120 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-09-20T18:41:31.120 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:41:31.120 INFO:teuthology.orchestra.run.smithi083.stdout:Install 1 Package 2024-09-20T18:41:31.120 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:41:31.120 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 784 k 2024-09-20T18:41:31.120 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 791 k 2024-09-20T18:41:31.120 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-09-20T18:41:31.120 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-09-20T18:41:31.121 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-09-20T18:41:31.379 INFO:teuthology.orchestra.run.smithi083.stdout:cephadm-19.3.0-5099.g88deadb8.el9.noarch.rpm 3.0 MB/s | 784 kB 00:00 2024-09-20T18:41:31.379 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-09-20T18:41:31.380 INFO:teuthology.orchestra.run.smithi083.stdout:Total 2.9 MB/s | 784 kB 00:00 2024-09-20T18:41:31.380 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-09-20T18:41:31.385 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-09-20T18:41:31.385 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-09-20T18:41:31.391 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-09-20T18:41:31.391 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-09-20T18:41:31.408 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-09-20T18:41:31.409 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-09-20T18:41:31.469 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-09-20T18:41:31.950 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: cephadm-2:19.3.0-5099.g88deadb8.el9.noarch 1/1 2024-09-20T18:41:32.001 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-09-20T18:41:32.058 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephadm-2:19.3.0-5099.g88deadb8.el9.noarch 1/1 2024-09-20T18:41:32.082 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-20T18:41:32.125 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-20T18:41:32.172 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-20T18:41:32.203 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-20T18:41:32.295 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-20T18:41:32.416 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: cephadm-2:19.3.0-5099.g88deadb8.el9.noarch 1/1 2024-09-20T18:41:32.421 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-20T18:41:32.515 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-20T18:41:32.557 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-20T18:41:32.586 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-20T18:41:32.618 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5099.g88deadb8.el9 10/12 2024-09-20T18:41:32.668 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephadm-2:19.3.0-5099.g88deadb8.el9.noarch 1/1 2024-09-20T18:41:32.668 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:41:32.668 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2024-09-20T18:41:32.668 INFO:teuthology.orchestra.run.smithi083.stdout: cephadm-2:19.3.0-5099.g88deadb8.el9.noarch 2024-09-20T18:41:32.668 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:41:32.668 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-20T18:41:32.900 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-immutable-object-cache 2024-09-20T18:41:33.488 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:00:56 ago on Fri 20 Sep 2024 06:40:37 PM UTC. 2024-09-20T18:41:33.623 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-20T18:41:33.624 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:41:33.624 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repo Size 2024-09-20T18:41:33.624 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:41:33.624 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2024-09-20T18:41:33.624 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 145 k 2024-09-20T18:41:33.624 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:41:33.624 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-09-20T18:41:33.624 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:41:33.624 INFO:teuthology.orchestra.run.smithi083.stdout:Install 1 Package 2024-09-20T18:41:33.624 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:41:33.624 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 145 k 2024-09-20T18:41:33.625 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 432 k 2024-09-20T18:41:33.625 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-09-20T18:41:33.819 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-immutable-object-cache-19.3.0-5099.g88dead 747 kB/s | 145 kB 00:00 2024-09-20T18:41:33.820 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-09-20T18:41:33.820 INFO:teuthology.orchestra.run.smithi083.stdout:Total 740 kB/s | 145 kB 00:00 2024-09-20T18:41:33.820 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-09-20T18:41:33.828 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-09-20T18:41:33.828 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-09-20T18:41:33.885 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-09-20T18:41:33.885 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-09-20T18:41:34.027 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-09-20T18:41:34.087 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5099.g88deadb8. 1/1 2024-09-20T18:41:34.112 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5099.g88deadb8. 1/1 2024-09-20T18:41:34.112 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-20T18:41:34.112 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-20T18:41:34.112 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:41:35.066 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5099.g88deadb8. 1/1 2024-09-20T18:41:35.067 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:41:35.067 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2024-09-20T18:41:35.067 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-immutable-object-cache-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:41:35.067 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:41:35.067 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-20T18:41:35.288 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-mgr 2024-09-20T18:41:35.800 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5099.g88deadb8.el 11/12 2024-09-20T18:41:35.856 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5099.g88deadb8.el9.noa 12/12 2024-09-20T18:41:35.879 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:00:58 ago on Fri 20 Sep 2024 06:40:37 PM UTC. 2024-09-20T18:41:35.968 INFO:teuthology.orchestra.run.smithi083.stdout:Package ceph-mgr-2:19.3.0-5099.g88deadb8.el9.x86_64 is already installed. 2024-09-20T18:41:36.009 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-20T18:41:36.011 INFO:teuthology.orchestra.run.smithi083.stdout:Nothing to do. 2024-09-20T18:41:36.011 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-20T18:41:36.076 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-mgr-dashboard 2024-09-20T18:41:36.650 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:00:59 ago on Fri 20 Sep 2024 06:40:37 PM UTC. 2024-09-20T18:41:36.782 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-20T18:41:36.783 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:41:36.783 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repository Size 2024-09-20T18:41:36.783 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:41:36.783 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2024-09-20T18:41:36.783 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5099.g88deadb8.el9 ceph-noarch 14 M 2024-09-20T18:41:36.783 INFO:teuthology.orchestra.run.smithi083.stdout:Installing dependencies: 2024-09-20T18:41:36.784 INFO:teuthology.orchestra.run.smithi083.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-20T18:41:36.784 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5099.g88deadb8.el9 ceph-noarch 34 k 2024-09-20T18:41:36.784 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5099.g88deadb8.el9 ceph-noarch 16 k 2024-09-20T18:41:36.784 INFO:teuthology.orchestra.run.smithi083.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-20T18:41:36.784 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-20T18:41:36.784 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-20T18:41:36.784 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-20T18:41:36.784 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-20T18:41:36.784 INFO:teuthology.orchestra.run.smithi083.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-20T18:41:36.784 INFO:teuthology.orchestra.run.smithi083.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-20T18:41:36.784 INFO:teuthology.orchestra.run.smithi083.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-20T18:41:36.784 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:41:36.784 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-09-20T18:41:36.784 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:41:36.784 INFO:teuthology.orchestra.run.smithi083.stdout:Install 12 Packages 2024-09-20T18:41:36.785 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:41:36.785 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 19 M 2024-09-20T18:41:36.785 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 159 M 2024-09-20T18:41:36.785 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-09-20T18:41:37.382 INFO:teuthology.orchestra.run.smithi083.stdout:(1/12): ceph-grafana-dashboards-19.3.0-5099.g88 222 kB/s | 34 kB 00:00 2024-09-20T18:41:37.398 INFO:teuthology.orchestra.run.smithi083.stdout:(2/12): ceph-prometheus-alerts-19.3.0-5099.g88d 96 kB/s | 16 kB 00:00 2024-09-20T18:41:37.574 INFO:teuthology.orchestra.run.smithi083.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.5 MB/s | 269 kB 00:00 2024-09-20T18:41:37.633 INFO:teuthology.orchestra.run.smithi083.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 4.0 MB/s | 1.0 MB 00:00 2024-09-20T18:41:37.683 INFO:teuthology.orchestra.run.smithi083.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 11 MB/s | 548 kB 00:00 2024-09-20T18:41:37.708 INFO:teuthology.orchestra.run.smithi083.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 785 kB/s | 19 kB 00:00 2024-09-20T18:41:37.759 INFO:teuthology.orchestra.run.smithi083.stdout:(7/12): protobuf-compiler-3.14.0-13.el9.x86_64. 4.6 MB/s | 863 kB 00:00 2024-09-20T18:41:37.797 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5099.g88deadb8.el9.noa 12/12 2024-09-20T18:41:37.797 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5099.g88deadb8.el 1/12 2024-09-20T18:41:37.798 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5099.g88deadb8.el9.noa 2/12 2024-09-20T18:41:37.798 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5099.g88deadb8.el9 3/12 2024-09-20T18:41:37.798 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-20T18:41:37.798 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-20T18:41:37.798 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-20T18:41:37.798 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-20T18:41:37.798 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-20T18:41:37.798 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-20T18:41:37.798 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-20T18:41:37.798 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-20T18:41:37.809 INFO:teuthology.orchestra.run.smithi083.stdout:(8/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 20 MB/s | 2.0 MB 00:00 2024-09-20T18:41:37.968 INFO:teuthology.orchestra.run.smithi083.stdout:(9/12): ceph-mgr-dashboard-19.3.0-5099.g88deadb 19 MB/s | 14 MB 00:00 2024-09-20T18:41:37.993 INFO:teuthology.orchestra.run.smithi083.stdout:(10/12): python3-repoze-lru-0.7-16.el9.noarch.r 174 kB/s | 31 kB 00:00 2024-09-20T18:41:38.026 INFO:teuthology.orchestra.run.smithi083.stdout:(11/12): python3-grpcio-tools-1.46.7-10.el9.x86 540 kB/s | 144 kB 00:00 2024-09-20T18:41:38.059 INFO:teuthology.orchestra.run.smithi083.stdout:(12/12): python3-routes-2.5.1-5.el9.noarch.rpm 2.0 MB/s | 188 kB 00:00 2024-09-20T18:41:38.063 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-09-20T18:41:38.063 INFO:teuthology.orchestra.run.smithi083.stdout:Total 15 MB/s | 19 MB 00:01 2024-09-20T18:41:38.112 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-09-20T18:41:38.122 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-20T18:41:38.122 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:41:38.122 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-09-20T18:41:38.122 INFO:teuthology.orchestra.run.smithi142.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-20T18:41:38.122 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-grafana-dashboards-2:19.3.0-5099.g88deadb8.el9.noarch 2024-09-20T18:41:38.123 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-dashboard-2:19.3.0-5099.g88deadb8.el9.noarch 2024-09-20T18:41:38.123 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-prometheus-alerts-2:19.3.0-5099.g88deadb8.el9.noarch 2024-09-20T18:41:38.123 INFO:teuthology.orchestra.run.smithi142.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-20T18:41:38.123 INFO:teuthology.orchestra.run.smithi142.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-20T18:41:38.123 INFO:teuthology.orchestra.run.smithi142.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-20T18:41:38.123 INFO:teuthology.orchestra.run.smithi142.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-20T18:41:38.123 INFO:teuthology.orchestra.run.smithi142.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-20T18:41:38.123 INFO:teuthology.orchestra.run.smithi142.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-20T18:41:38.123 INFO:teuthology.orchestra.run.smithi142.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-20T18:41:38.123 INFO:teuthology.orchestra.run.smithi142.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-20T18:41:38.123 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:41:38.123 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-09-20T18:41:38.131 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-09-20T18:41:38.131 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-09-20T18:41:38.382 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-20T18:41:38.427 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-09-20T18:41:38.428 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-09-20T18:41:38.960 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:01:06 ago on Fri 20 Sep 2024 06:40:32 PM UTC. 2024-09-20T18:41:39.025 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-09-20T18:41:39.094 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-20T18:41:39.094 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-09-20T18:41:39.096 INFO:teuthology.orchestra.run.smithi142.stdout:======================================================================================= 2024-09-20T18:41:39.096 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repository Size 2024-09-20T18:41:39.096 INFO:teuthology.orchestra.run.smithi142.stdout:======================================================================================= 2024-09-20T18:41:39.096 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-09-20T18:41:39.096 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5099.g88deadb8.el9 ceph-noarch 7.4 M 2024-09-20T18:41:39.096 INFO:teuthology.orchestra.run.smithi142.stdout:Installing dependencies: 2024-09-20T18:41:39.096 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-20T18:41:39.096 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-20T18:41:39.096 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-20T18:41:39.097 INFO:teuthology.orchestra.run.smithi142.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-20T18:41:39.097 INFO:teuthology.orchestra.run.smithi142.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-20T18:41:39.097 INFO:teuthology.orchestra.run.smithi142.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-20T18:41:39.097 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-20T18:41:39.097 INFO:teuthology.orchestra.run.smithi142.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-20T18:41:39.097 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-20T18:41:39.097 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-20T18:41:39.097 INFO:teuthology.orchestra.run.smithi142.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-20T18:41:39.097 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:41:39.097 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-09-20T18:41:39.097 INFO:teuthology.orchestra.run.smithi142.stdout:======================================================================================= 2024-09-20T18:41:39.097 INFO:teuthology.orchestra.run.smithi142.stdout:Install 12 Packages 2024-09-20T18:41:39.097 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:41:39.098 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 38 M 2024-09-20T18:41:39.098 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 212 M 2024-09-20T18:41:39.098 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-09-20T18:41:39.146 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-20T18:41:39.200 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-20T18:41:39.239 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-20T18:41:39.341 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-20T18:41:39.475 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-20T18:41:39.554 INFO:teuthology.orchestra.run.smithi142.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 985 kB/s | 191 kB 00:00 2024-09-20T18:41:39.560 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-20T18:41:39.611 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-20T18:41:39.656 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-20T18:41:39.671 INFO:teuthology.orchestra.run.smithi142.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 2.5 MB/s | 801 kB 00:00 2024-09-20T18:41:39.688 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5099.g88deadb8.el9 10/12 2024-09-20T18:41:39.805 INFO:teuthology.orchestra.run.smithi142.stdout:(3/12): ceph-mgr-diskprediction-local-19.3.0-50 17 MB/s | 7.4 MB 00:00 2024-09-20T18:41:39.872 INFO:teuthology.orchestra.run.smithi142.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 103 kB/s | 33 kB 00:00 2024-09-20T18:41:40.072 INFO:teuthology.orchestra.run.smithi142.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 192 kB/s | 38 kB 00:00 2024-09-20T18:41:40.097 INFO:teuthology.orchestra.run.smithi142.stdout:(6/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 62 kB/s | 18 kB 00:00 2024-09-20T18:41:40.556 INFO:teuthology.orchestra.run.smithi142.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 534 kB/s | 245 kB 00:00 2024-09-20T18:41:40.848 INFO:teuthology.orchestra.run.smithi142.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.5 MB/s | 3.0 MB 00:01 2024-09-20T18:41:40.973 INFO:teuthology.orchestra.run.smithi142.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.0 MB/s | 250 kB 00:00 2024-09-20T18:41:41.140 INFO:teuthology.orchestra.run.smithi142.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 4.6 MB/s | 4.9 MB 00:01 2024-09-20T18:41:41.365 INFO:teuthology.orchestra.run.smithi142.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 6.3 MB/s | 5.1 MB 00:00 2024-09-20T18:41:42.897 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5099.g88deadb8.el 11/12 2024-09-20T18:41:42.960 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5099.g88deadb8.el9.noa 12/12 2024-09-20T18:41:45.015 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5099.g88deadb8.el9.noa 12/12 2024-09-20T18:41:45.015 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5099.g88deadb8.el 1/12 2024-09-20T18:41:45.015 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5099.g88deadb8.el9.noa 2/12 2024-09-20T18:41:45.015 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5099.g88deadb8.el9 3/12 2024-09-20T18:41:45.015 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-20T18:41:45.015 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-20T18:41:45.015 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-20T18:41:45.015 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-20T18:41:45.015 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-20T18:41:45.016 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-20T18:41:45.016 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-20T18:41:45.016 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-20T18:41:45.341 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-20T18:41:45.341 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:41:45.342 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2024-09-20T18:41:45.342 INFO:teuthology.orchestra.run.smithi083.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-20T18:41:45.342 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-grafana-dashboards-2:19.3.0-5099.g88deadb8.el9.noarch 2024-09-20T18:41:45.342 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-dashboard-2:19.3.0-5099.g88deadb8.el9.noarch 2024-09-20T18:41:45.342 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-prometheus-alerts-2:19.3.0-5099.g88deadb8.el9.noarch 2024-09-20T18:41:45.342 INFO:teuthology.orchestra.run.smithi083.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-20T18:41:45.342 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-20T18:41:45.342 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-20T18:41:45.342 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-20T18:41:45.342 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-20T18:41:45.342 INFO:teuthology.orchestra.run.smithi083.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-20T18:41:45.343 INFO:teuthology.orchestra.run.smithi083.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-20T18:41:45.343 INFO:teuthology.orchestra.run.smithi083.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-20T18:41:45.343 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:41:45.343 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-20T18:41:45.589 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-20T18:41:46.166 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:09 ago on Fri 20 Sep 2024 06:40:37 PM UTC. 2024-09-20T18:41:46.298 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-20T18:41:46.300 INFO:teuthology.orchestra.run.smithi083.stdout:======================================================================================= 2024-09-20T18:41:46.300 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repository Size 2024-09-20T18:41:46.300 INFO:teuthology.orchestra.run.smithi083.stdout:======================================================================================= 2024-09-20T18:41:46.300 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2024-09-20T18:41:46.300 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5099.g88deadb8.el9 ceph-noarch 7.4 M 2024-09-20T18:41:46.300 INFO:teuthology.orchestra.run.smithi083.stdout:Installing dependencies: 2024-09-20T18:41:46.300 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-20T18:41:46.300 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-20T18:41:46.300 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-20T18:41:46.300 INFO:teuthology.orchestra.run.smithi083.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-20T18:41:46.300 INFO:teuthology.orchestra.run.smithi083.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-20T18:41:46.300 INFO:teuthology.orchestra.run.smithi083.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-20T18:41:46.300 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-20T18:41:46.301 INFO:teuthology.orchestra.run.smithi083.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-20T18:41:46.301 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-20T18:41:46.301 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-20T18:41:46.301 INFO:teuthology.orchestra.run.smithi083.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-20T18:41:46.301 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:41:46.301 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-09-20T18:41:46.301 INFO:teuthology.orchestra.run.smithi083.stdout:======================================================================================= 2024-09-20T18:41:46.301 INFO:teuthology.orchestra.run.smithi083.stdout:Install 12 Packages 2024-09-20T18:41:46.301 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:41:46.302 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 38 M 2024-09-20T18:41:46.302 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 212 M 2024-09-20T18:41:46.302 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-09-20T18:41:46.910 INFO:teuthology.orchestra.run.smithi083.stdout:(1/12): ceph-mgr-diskprediction-local-19.3.0-50 16 MB/s | 7.4 MB 00:00 2024-09-20T18:41:46.977 INFO:teuthology.orchestra.run.smithi083.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 491 kB/s | 33 kB 00:00 2024-09-20T18:41:47.001 INFO:teuthology.orchestra.run.smithi083.stdout:(3/12): libquadmath-11.5.0-2.el9.x86_64.rpm 346 kB/s | 191 kB 00:00 2024-09-20T18:41:47.068 INFO:teuthology.orchestra.run.smithi083.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 273 kB/s | 18 kB 00:00 2024-09-20T18:41:47.152 INFO:teuthology.orchestra.run.smithi083.stdout:(5/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 17 MB/s | 3.0 MB 00:00 2024-09-20T18:41:47.177 INFO:teuthology.orchestra.run.smithi083.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 354 kB/s | 38 kB 00:00 2024-09-20T18:41:47.286 INFO:teuthology.orchestra.run.smithi083.stdout:(7/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 37 MB/s | 4.9 MB 00:00 2024-09-20T18:41:47.311 INFO:teuthology.orchestra.run.smithi083.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.8 MB/s | 245 kB 00:00 2024-09-20T18:41:47.345 INFO:teuthology.orchestra.run.smithi083.stdout:(9/12): libgfortran-11.5.0-2.el9.x86_64.rpm 895 kB/s | 801 kB 00:00 2024-09-20T18:41:47.387 INFO:teuthology.orchestra.run.smithi083.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 3.2 MB/s | 250 kB 00:00 2024-09-20T18:41:47.462 INFO:teuthology.orchestra.run.smithi083.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 29 MB/s | 5.1 MB 00:00 2024-09-20T18:41:47.879 INFO:teuthology.orchestra.run.smithi083.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 30 MB/s | 16 MB 00:00 2024-09-20T18:41:47.879 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-09-20T18:41:47.880 INFO:teuthology.orchestra.run.smithi083.stdout:Total 24 MB/s | 38 MB 00:01 2024-09-20T18:41:48.232 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-09-20T18:41:48.257 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-09-20T18:41:48.257 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-09-20T18:41:48.683 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-09-20T18:41:48.683 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-09-20T18:41:49.442 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-09-20T18:41:49.507 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-20T18:41:49.550 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-20T18:41:49.642 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-20T18:41:49.679 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-20T18:41:49.863 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-20T18:41:49.894 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-20T18:41:49.969 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-20T18:41:50.635 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-20T18:41:50.774 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-20T18:41:52.021 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-20T18:41:53.047 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-20T18:41:53.070 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5099.g88dea 12/12 2024-09-20T18:41:55.714 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5099.g88dea 12/12 2024-09-20T18:41:55.714 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5099.g88dea 1/12 2024-09-20T18:41:55.714 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-20T18:41:55.714 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-20T18:41:55.714 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-20T18:41:55.714 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-20T18:41:55.715 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-20T18:41:55.715 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-20T18:41:55.715 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-20T18:41:55.715 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-20T18:41:55.715 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-20T18:41:55.715 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-20T18:41:56.174 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-20T18:41:56.175 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:41:56.175 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2024-09-20T18:41:56.175 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5099.g88deadb8.el9.noarch 2024-09-20T18:41:56.175 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-20T18:41:56.175 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-20T18:41:56.175 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-20T18:41:56.175 INFO:teuthology.orchestra.run.smithi083.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-20T18:41:56.175 INFO:teuthology.orchestra.run.smithi083.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-20T18:41:56.175 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-20T18:41:56.175 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-20T18:41:56.175 INFO:teuthology.orchestra.run.smithi083.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-20T18:41:56.175 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-20T18:41:56.175 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-20T18:41:56.176 INFO:teuthology.orchestra.run.smithi083.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-20T18:41:56.176 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:41:56.176 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-20T18:41:56.526 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-mgr-rook 2024-09-20T18:41:57.111 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:20 ago on Fri 20 Sep 2024 06:40:37 PM UTC. 2024-09-20T18:41:57.244 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-20T18:41:57.245 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:41:57.245 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repository Size 2024-09-20T18:41:57.245 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:41:57.245 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2024-09-20T18:41:57.245 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-rook noarch 2:19.3.0-5099.g88deadb8.el9 ceph-noarch 49 k 2024-09-20T18:41:57.245 INFO:teuthology.orchestra.run.smithi083.stdout:Installing dependencies: 2024-09-20T18:41:57.245 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-20T18:41:57.246 INFO:teuthology.orchestra.run.smithi083.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-20T18:41:57.246 INFO:teuthology.orchestra.run.smithi083.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-20T18:41:57.246 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-20T18:41:57.246 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-20T18:41:57.246 INFO:teuthology.orchestra.run.smithi083.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-20T18:41:57.246 INFO:teuthology.orchestra.run.smithi083.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-20T18:41:57.246 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-20T18:41:57.246 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-20T18:41:57.246 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-20T18:41:57.246 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-20T18:41:57.246 INFO:teuthology.orchestra.run.smithi083.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-20T18:41:57.246 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:41:57.246 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-09-20T18:41:57.246 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:41:57.247 INFO:teuthology.orchestra.run.smithi083.stdout:Install 13 Packages 2024-09-20T18:41:57.247 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:41:57.247 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 2.2 M 2024-09-20T18:41:57.247 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 26 M 2024-09-20T18:41:57.247 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-09-20T18:41:57.613 INFO:teuthology.orchestra.run.smithi083.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 175 kB/s | 26 kB 00:00 2024-09-20T18:41:57.630 INFO:teuthology.orchestra.run.smithi083.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 118 kB/s | 19 kB 00:00 2024-09-20T18:41:57.647 INFO:teuthology.orchestra.run.smithi083.stdout:(3/13): ceph-mgr-rook-19.3.0-5099.g88deadb8.el9 274 kB/s | 49 kB 00:00 2024-09-20T18:41:57.739 INFO:teuthology.orchestra.run.smithi083.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-09-20T18:41:57.764 INFO:teuthology.orchestra.run.smithi083.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.6 MB/s | 222 kB 00:00 2024-09-20T18:41:57.781 INFO:teuthology.orchestra.run.smithi083.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.2 MB/s | 54 kB 00:00 2024-09-20T18:41:57.815 INFO:teuthology.orchestra.run.smithi083.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 640 kB/s | 32 kB 00:00 2024-09-20T18:41:57.840 INFO:teuthology.orchestra.run.smithi083.stdout:(8/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.4 MB/s | 279 kB 00:00 2024-09-20T18:41:57.865 INFO:teuthology.orchestra.run.smithi083.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 168 kB/s | 14 kB 00:00 2024-09-20T18:41:57.899 INFO:teuthology.orchestra.run.smithi083.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.2 MB/s | 59 kB 00:00 2024-09-20T18:41:57.924 INFO:teuthology.orchestra.run.smithi083.stdout:(11/13): python3-google-auth-2.34.0-1.el9.noarc 2.0 MB/s | 220 kB 00:00 2024-09-20T18:41:57.958 INFO:teuthology.orchestra.run.smithi083.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.5 MB/s | 90 kB 00:00 2024-09-20T18:41:58.000 INFO:teuthology.orchestra.run.smithi083.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.4 MB/s | 1.0 MB 00:00 2024-09-20T18:41:58.002 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-09-20T18:41:58.002 INFO:teuthology.orchestra.run.smithi083.stdout:Total 2.9 MB/s | 2.2 MB 00:00 2024-09-20T18:41:58.065 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-09-20T18:41:58.080 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-09-20T18:41:58.081 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-09-20T18:41:58.172 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-09-20T18:41:58.172 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-09-20T18:41:58.688 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-09-20T18:41:58.864 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-20T18:41:58.917 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-20T18:41:58.981 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-20T18:41:59.019 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-20T18:41:59.060 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-20T18:41:59.171 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-20T18:41:59.280 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-20T18:41:59.329 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-20T18:42:00.043 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-20T18:42:00.093 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-20T18:42:00.135 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-20T18:42:00.183 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-20T18:42:00.205 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-rook-2:19.3.0-5099.g88deadb8.el9.noarch 13/13 2024-09-20T18:42:00.918 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5099.g88deadb8.el9.noarch 13/13 2024-09-20T18:42:00.919 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5099.g88deadb8.el9.noarch 1/13 2024-09-20T18:42:00.919 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-20T18:42:00.919 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-20T18:42:00.919 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-20T18:42:00.919 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-20T18:42:00.919 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-20T18:42:00.919 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-20T18:42:00.919 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-20T18:42:00.919 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-20T18:42:00.919 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-20T18:42:00.919 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-20T18:42:00.920 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-20T18:42:01.255 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-20T18:42:01.255 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:42:01.256 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2024-09-20T18:42:01.256 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-rook-2:19.3.0-5099.g88deadb8.el9.noarch 2024-09-20T18:42:01.256 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-20T18:42:01.256 INFO:teuthology.orchestra.run.smithi083.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-20T18:42:01.256 INFO:teuthology.orchestra.run.smithi083.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-20T18:42:01.256 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-20T18:42:01.256 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-20T18:42:01.256 INFO:teuthology.orchestra.run.smithi083.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-20T18:42:01.256 INFO:teuthology.orchestra.run.smithi083.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-20T18:42:01.256 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-20T18:42:01.256 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-20T18:42:01.256 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-20T18:42:01.256 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-20T18:42:01.256 INFO:teuthology.orchestra.run.smithi083.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-20T18:42:01.257 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:42:01.257 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-20T18:42:01.514 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-mgr-cephadm 2024-09-20T18:42:01.946 INFO:teuthology.orchestra.run.smithi142.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 794 kB/s | 16 MB 00:20 2024-09-20T18:42:01.946 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-09-20T18:42:01.947 INFO:teuthology.orchestra.run.smithi142.stdout:Total 1.7 MB/s | 38 MB 00:22 2024-09-20T18:42:02.100 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:25 ago on Fri 20 Sep 2024 06:40:37 PM UTC. 2024-09-20T18:42:02.233 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-20T18:42:02.234 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:42:02.234 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repository Size 2024-09-20T18:42:02.234 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:42:02.234 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2024-09-20T18:42:02.234 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5099.g88deadb8.el9 ceph-noarch 154 k 2024-09-20T18:42:02.234 INFO:teuthology.orchestra.run.smithi083.stdout:Installing dependencies: 2024-09-20T18:42:02.234 INFO:teuthology.orchestra.run.smithi083.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-20T18:42:02.235 INFO:teuthology.orchestra.run.smithi083.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-20T18:42:02.235 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-20T18:42:02.235 INFO:teuthology.orchestra.run.smithi083.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-20T18:42:02.235 INFO:teuthology.orchestra.run.smithi083.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-20T18:42:02.235 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:42:02.235 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-09-20T18:42:02.235 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:42:02.235 INFO:teuthology.orchestra.run.smithi083.stdout:Install 6 Packages 2024-09-20T18:42:02.235 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:42:02.236 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 7.0 M 2024-09-20T18:42:02.236 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 33 M 2024-09-20T18:42:02.236 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-09-20T18:42:02.305 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-09-20T18:42:02.330 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-09-20T18:42:02.331 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-09-20T18:42:02.611 INFO:teuthology.orchestra.run.smithi083.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.2 MB/s | 248 kB 00:00 2024-09-20T18:42:02.636 INFO:teuthology.orchestra.run.smithi083.stdout:(2/6): ceph-mgr-cephadm-19.3.0-5099.g88deadb8.e 675 kB/s | 154 kB 00:00 2024-09-20T18:42:02.686 INFO:teuthology.orchestra.run.smithi083.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 7.1 MB/s | 548 kB 00:00 2024-09-20T18:42:02.712 INFO:teuthology.orchestra.run.smithi083.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 767 kB/s | 58 kB 00:00 2024-09-20T18:42:02.737 INFO:teuthology.orchestra.run.smithi083.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.4 MB/s | 74 kB 00:00 2024-09-20T18:42:02.757 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-09-20T18:42:02.758 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-09-20T18:42:03.004 INFO:teuthology.orchestra.run.smithi083.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 10 MB/s | 6.0 MB 00:00 2024-09-20T18:42:03.006 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-09-20T18:42:03.006 INFO:teuthology.orchestra.run.smithi083.stdout:Total 9.1 MB/s | 7.0 MB 00:00 2024-09-20T18:42:03.077 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-09-20T18:42:03.089 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-09-20T18:42:03.089 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-09-20T18:42:03.191 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-09-20T18:42:03.192 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-09-20T18:42:03.511 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-09-20T18:42:03.515 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-09-20T18:42:03.586 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-20T18:42:03.632 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-20T18:42:03.672 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-20T18:42:03.733 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-20T18:42:03.739 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-20T18:42:03.769 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-20T18:42:03.953 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-20T18:42:03.985 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-20T18:42:04.061 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-20T18:42:04.271 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-20T18:42:04.368 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-20T18:42:04.436 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-20T18:42:04.460 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5099.g88deadb8.el9.noarch 6/6 2024-09-20T18:42:04.725 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-20T18:42:04.847 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-20T18:42:05.252 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5099.g88deadb8.el9.noarch 6/6 2024-09-20T18:42:05.252 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5099.g88deadb8.el9.noarch 1/6 2024-09-20T18:42:05.252 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-20T18:42:05.252 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-20T18:42:05.252 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-20T18:42:05.252 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-20T18:42:05.554 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-20T18:42:05.555 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:42:05.555 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2024-09-20T18:42:05.555 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-cephadm-2:19.3.0-5099.g88deadb8.el9.noarch 2024-09-20T18:42:05.555 INFO:teuthology.orchestra.run.smithi083.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-20T18:42:05.555 INFO:teuthology.orchestra.run.smithi083.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-20T18:42:05.555 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-20T18:42:05.555 INFO:teuthology.orchestra.run.smithi083.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-20T18:42:05.555 INFO:teuthology.orchestra.run.smithi083.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-20T18:42:05.555 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:42:05.555 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-20T18:42:05.815 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-fuse 2024-09-20T18:42:06.077 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-20T18:42:06.389 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:29 ago on Fri 20 Sep 2024 06:40:37 PM UTC. 2024-09-20T18:42:06.522 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-20T18:42:06.522 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:42:06.522 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repository Size 2024-09-20T18:42:06.522 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:42:06.522 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2024-09-20T18:42:06.523 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-fuse x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 899 k 2024-09-20T18:42:06.523 INFO:teuthology.orchestra.run.smithi083.stdout:Installing dependencies: 2024-09-20T18:42:06.523 INFO:teuthology.orchestra.run.smithi083.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-20T18:42:06.523 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:42:06.523 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-09-20T18:42:06.523 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:42:06.523 INFO:teuthology.orchestra.run.smithi083.stdout:Install 2 Packages 2024-09-20T18:42:06.523 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:42:06.524 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 979 k 2024-09-20T18:42:06.524 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 2.9 M 2024-09-20T18:42:06.524 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-09-20T18:42:07.073 INFO:teuthology.orchestra.run.smithi083.stdout:(1/2): ceph-fuse-19.3.0-5099.g88deadb8.el9.x86_ 3.2 MB/s | 899 kB 00:00 2024-09-20T18:42:07.114 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-20T18:42:07.143 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5099.g88dea 12/12 2024-09-20T18:42:07.265 INFO:teuthology.orchestra.run.smithi083.stdout:(2/2): fuse-2.9.9-16.el9.x86_64.rpm 172 kB/s | 80 kB 00:00 2024-09-20T18:42:07.265 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-09-20T18:42:07.266 INFO:teuthology.orchestra.run.smithi083.stdout:Total 1.3 MB/s | 979 kB 00:00 2024-09-20T18:42:07.277 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-09-20T18:42:07.298 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-09-20T18:42:07.298 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-09-20T18:42:07.380 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-09-20T18:42:07.381 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-09-20T18:42:07.570 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-09-20T18:42:07.717 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-20T18:42:07.790 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-fuse-2:19.3.0-5099.g88deadb8.el9.x86_64 2/2 2024-09-20T18:42:08.504 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-fuse-2:19.3.0-5099.g88deadb8.el9.x86_64 2/2 2024-09-20T18:42:08.504 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-fuse-2:19.3.0-5099.g88deadb8.el9.x86_64 1/2 2024-09-20T18:42:08.780 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-20T18:42:08.780 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:42:08.780 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2024-09-20T18:42:08.780 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-fuse-2:19.3.0-5099.g88deadb8.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-20T18:42:08.780 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:42:08.780 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-20T18:42:09.080 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-volume 2024-09-20T18:42:09.674 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:32 ago on Fri 20 Sep 2024 06:40:37 PM UTC. 2024-09-20T18:42:09.719 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5099.g88dea 12/12 2024-09-20T18:42:09.720 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5099.g88dea 1/12 2024-09-20T18:42:09.720 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-20T18:42:09.720 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-20T18:42:09.720 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-20T18:42:09.720 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-20T18:42:09.720 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-20T18:42:09.720 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-20T18:42:09.720 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-20T18:42:09.720 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-20T18:42:09.720 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-20T18:42:09.720 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-20T18:42:09.812 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-20T18:42:09.813 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:42:09.813 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repository Size 2024-09-20T18:42:09.813 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:42:09.813 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2024-09-20T18:42:09.813 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-volume noarch 2:19.3.0-5099.g88deadb8.el9 ceph-noarch 287 k 2024-09-20T18:42:09.813 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:42:09.813 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-09-20T18:42:09.813 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:42:09.814 INFO:teuthology.orchestra.run.smithi083.stdout:Install 1 Package 2024-09-20T18:42:09.814 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:42:09.814 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 287 k 2024-09-20T18:42:09.814 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 1.4 M 2024-09-20T18:42:09.814 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-09-20T18:42:10.046 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-volume-19.3.0-5099.g88deadb8.el9.noarch.rp 1.2 MB/s | 287 kB 00:00 2024-09-20T18:42:10.047 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-09-20T18:42:10.047 INFO:teuthology.orchestra.run.smithi083.stdout:Total 1.2 MB/s | 287 kB 00:00 2024-09-20T18:42:10.047 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-09-20T18:42:10.055 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-09-20T18:42:10.055 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-09-20T18:42:10.092 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-09-20T18:42:10.093 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-09-20T18:42:10.126 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-20T18:42:10.126 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:42:10.126 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-09-20T18:42:10.126 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5099.g88deadb8.el9.noarch 2024-09-20T18:42:10.126 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-20T18:42:10.126 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-20T18:42:10.126 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-20T18:42:10.126 INFO:teuthology.orchestra.run.smithi142.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-20T18:42:10.126 INFO:teuthology.orchestra.run.smithi142.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-20T18:42:10.127 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-20T18:42:10.127 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-20T18:42:10.127 INFO:teuthology.orchestra.run.smithi142.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-20T18:42:10.127 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-20T18:42:10.127 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-20T18:42:10.127 INFO:teuthology.orchestra.run.smithi142.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-20T18:42:10.127 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:42:10.127 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-09-20T18:42:10.310 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-09-20T18:42:10.373 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-volume-2:19.3.0-5099.g88deadb8.el9.noarch 1/1 2024-09-20T18:42:10.390 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-volume-2:19.3.0-5099.g88deadb8.el9.noarch 1/1 2024-09-20T18:42:10.390 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-20T18:42:10.390 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-20T18:42:10.390 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:42:10.496 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install ceph-mgr-rook 2024-09-20T18:42:11.089 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:01:39 ago on Fri 20 Sep 2024 06:40:32 PM UTC. 2024-09-20T18:42:11.223 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-09-20T18:42:11.224 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:42:11.225 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repository Size 2024-09-20T18:42:11.225 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:42:11.225 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-09-20T18:42:11.225 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-rook noarch 2:19.3.0-5099.g88deadb8.el9 ceph-noarch 49 k 2024-09-20T18:42:11.225 INFO:teuthology.orchestra.run.smithi142.stdout:Installing dependencies: 2024-09-20T18:42:11.225 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-20T18:42:11.225 INFO:teuthology.orchestra.run.smithi142.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-20T18:42:11.225 INFO:teuthology.orchestra.run.smithi142.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-20T18:42:11.225 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-20T18:42:11.225 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-20T18:42:11.225 INFO:teuthology.orchestra.run.smithi142.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-20T18:42:11.226 INFO:teuthology.orchestra.run.smithi142.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-20T18:42:11.226 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-20T18:42:11.226 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-20T18:42:11.226 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-20T18:42:11.226 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-20T18:42:11.226 INFO:teuthology.orchestra.run.smithi142.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-20T18:42:11.226 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:42:11.226 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-09-20T18:42:11.226 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:42:11.226 INFO:teuthology.orchestra.run.smithi142.stdout:Install 13 Packages 2024-09-20T18:42:11.226 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:42:11.227 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 2.2 M 2024-09-20T18:42:11.227 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 26 M 2024-09-20T18:42:11.227 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-09-20T18:42:11.463 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-volume-2:19.3.0-5099.g88deadb8.el9.noarch 1/1 2024-09-20T18:42:11.463 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:42:11.463 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2024-09-20T18:42:11.463 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-volume-2:19.3.0-5099.g88deadb8.el9.noarch 2024-09-20T18:42:11.463 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:42:11.463 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-20T18:42:11.544 INFO:teuthology.orchestra.run.smithi142.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 223 kB/s | 26 kB 00:00 2024-09-20T18:42:11.569 INFO:teuthology.orchestra.run.smithi142.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 137 kB/s | 19 kB 00:00 2024-09-20T18:42:11.594 INFO:teuthology.orchestra.run.smithi142.stdout:(3/13): ceph-mgr-rook-19.3.0-5099.g88deadb8.el9 298 kB/s | 49 kB 00:00 2024-09-20T18:42:11.719 INFO:teuthology.orchestra.run.smithi142.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.4 MB/s | 222 kB 00:00 2024-09-20T18:42:11.739 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install librados-devel 2024-09-20T18:42:11.744 INFO:teuthology.orchestra.run.smithi142.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 906 kB/s | 159 kB 00:00 2024-09-20T18:42:11.778 INFO:teuthology.orchestra.run.smithi142.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 916 kB/s | 54 kB 00:00 2024-09-20T18:42:11.803 INFO:teuthology.orchestra.run.smithi142.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.3 MB/s | 279 kB 00:00 2024-09-20T18:42:11.837 INFO:teuthology.orchestra.run.smithi142.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 349 kB/s | 32 kB 00:00 2024-09-20T18:42:11.871 INFO:teuthology.orchestra.run.smithi142.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 156 kB/s | 14 kB 00:00 2024-09-20T18:42:11.913 INFO:teuthology.orchestra.run.smithi142.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.1 MB/s | 220 kB 00:00 2024-09-20T18:42:11.938 INFO:teuthology.orchestra.run.smithi142.stdout:(11/13): python3-kubernetes-26.1.0-2.el9.noarch 11 MB/s | 1.0 MB 00:00 2024-09-20T18:42:11.963 INFO:teuthology.orchestra.run.smithi142.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 3.7 MB/s | 90 kB 00:00 2024-09-20T18:42:11.988 INFO:teuthology.orchestra.run.smithi142.stdout:(13/13): python3-rsa-4.9-2.el9.noarch.rpm 512 kB/s | 59 kB 00:00 2024-09-20T18:42:11.991 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-09-20T18:42:11.991 INFO:teuthology.orchestra.run.smithi142.stdout:Total 2.9 MB/s | 2.2 MB 00:00 2024-09-20T18:42:12.056 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-09-20T18:42:12.070 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-09-20T18:42:12.071 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-09-20T18:42:12.160 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-09-20T18:42:12.161 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-09-20T18:42:12.327 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:35 ago on Fri 20 Sep 2024 06:40:37 PM UTC. 2024-09-20T18:42:12.464 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-20T18:42:12.465 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:42:12.465 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repo Size 2024-09-20T18:42:12.465 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:42:12.465 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2024-09-20T18:42:12.465 INFO:teuthology.orchestra.run.smithi083.stdout: librados-devel x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 126 k 2024-09-20T18:42:12.465 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:42:12.465 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-09-20T18:42:12.465 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:42:12.465 INFO:teuthology.orchestra.run.smithi083.stdout:Install 1 Package 2024-09-20T18:42:12.465 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:42:12.466 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 126 k 2024-09-20T18:42:12.466 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 456 k 2024-09-20T18:42:12.466 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-09-20T18:42:12.660 INFO:teuthology.orchestra.run.smithi083.stdout:librados-devel-19.3.0-5099.g88deadb8.el9.x86_64 652 kB/s | 126 kB 00:00 2024-09-20T18:42:12.661 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-09-20T18:42:12.661 INFO:teuthology.orchestra.run.smithi083.stdout:Total 646 kB/s | 126 kB 00:00 2024-09-20T18:42:12.661 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-09-20T18:42:12.667 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-09-20T18:42:12.667 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-09-20T18:42:12.678 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-09-20T18:42:12.731 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-09-20T18:42:12.732 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-09-20T18:42:12.844 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-20T18:42:12.885 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-09-20T18:42:12.898 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-20T18:42:12.953 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-20T18:42:12.975 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librados-devel-2:19.3.0-5099.g88deadb8.el9.x86_64 1/1 2024-09-20T18:42:12.990 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-20T18:42:13.032 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-20T18:42:13.133 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-20T18:42:13.244 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-20T18:42:13.300 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-20T18:42:13.421 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librados-devel-2:19.3.0-5099.g88deadb8.el9.x86_64 1/1 2024-09-20T18:42:13.671 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados-devel-2:19.3.0-5099.g88deadb8.el9.x86_64 1/1 2024-09-20T18:42:13.671 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:42:13.671 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2024-09-20T18:42:13.671 INFO:teuthology.orchestra.run.smithi083.stdout: librados-devel-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:42:13.671 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:42:13.671 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-20T18:42:13.898 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install libcephfs2 2024-09-20T18:42:14.016 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-20T18:42:14.072 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-20T18:42:14.106 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-20T18:42:14.154 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-20T18:42:14.184 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-rook-2:19.3.0-5099.g88deadb8.el9.noarch 13/13 2024-09-20T18:42:14.492 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:37 ago on Fri 20 Sep 2024 06:40:37 PM UTC. 2024-09-20T18:42:14.583 INFO:teuthology.orchestra.run.smithi083.stdout:Package libcephfs2-2:19.3.0-5099.g88deadb8.el9.x86_64 is already installed. 2024-09-20T18:42:14.629 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-20T18:42:14.630 INFO:teuthology.orchestra.run.smithi083.stdout:Nothing to do. 2024-09-20T18:42:14.630 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-20T18:42:14.696 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install libcephfs-devel 2024-09-20T18:42:14.995 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5099.g88deadb8.el9.noarch 13/13 2024-09-20T18:42:14.995 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5099.g88deadb8.el9.noarch 1/13 2024-09-20T18:42:14.995 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-20T18:42:14.995 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-20T18:42:14.995 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-20T18:42:14.995 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-20T18:42:14.995 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-20T18:42:14.995 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-20T18:42:14.995 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-20T18:42:14.995 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-20T18:42:14.995 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-20T18:42:14.995 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-20T18:42:14.996 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-20T18:42:15.280 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:38 ago on Fri 20 Sep 2024 06:40:37 PM UTC. 2024-09-20T18:42:15.309 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-20T18:42:15.309 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:42:15.309 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-09-20T18:42:15.309 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-rook-2:19.3.0-5099.g88deadb8.el9.noarch 2024-09-20T18:42:15.309 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-20T18:42:15.309 INFO:teuthology.orchestra.run.smithi142.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-20T18:42:15.309 INFO:teuthology.orchestra.run.smithi142.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-20T18:42:15.309 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-20T18:42:15.310 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-20T18:42:15.310 INFO:teuthology.orchestra.run.smithi142.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-20T18:42:15.310 INFO:teuthology.orchestra.run.smithi142.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-20T18:42:15.310 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-20T18:42:15.310 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-20T18:42:15.310 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-20T18:42:15.310 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-20T18:42:15.310 INFO:teuthology.orchestra.run.smithi142.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-20T18:42:15.310 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:42:15.310 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-09-20T18:42:15.414 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-20T18:42:15.415 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:42:15.415 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repo Size 2024-09-20T18:42:15.415 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:42:15.415 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2024-09-20T18:42:15.415 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs-devel x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 32 k 2024-09-20T18:42:15.415 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:42:15.415 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-09-20T18:42:15.415 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:42:15.415 INFO:teuthology.orchestra.run.smithi083.stdout:Install 1 Package 2024-09-20T18:42:15.415 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:42:15.416 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 32 k 2024-09-20T18:42:15.416 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 142 k 2024-09-20T18:42:15.416 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-09-20T18:42:15.566 INFO:teuthology.orchestra.run.smithi083.stdout:libcephfs-devel-19.3.0-5099.g88deadb8.el9.x86_6 213 kB/s | 32 kB 00:00 2024-09-20T18:42:15.566 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-09-20T18:42:15.566 INFO:teuthology.orchestra.run.smithi083.stdout:Total 211 kB/s | 32 kB 00:00 2024-09-20T18:42:15.566 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-09-20T18:42:15.569 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install ceph-mgr-cephadm 2024-09-20T18:42:15.570 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-09-20T18:42:15.570 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-09-20T18:42:15.583 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-09-20T18:42:15.583 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-09-20T18:42:15.673 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-09-20T18:42:15.779 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephfs-devel-2:19.3.0-5099.g88deadb8.el9.x86_64 1/1 2024-09-20T18:42:16.152 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:01:44 ago on Fri 20 Sep 2024 06:40:32 PM UTC. 2024-09-20T18:42:16.163 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-5099.g88deadb8.el9.x86_64 1/1 2024-09-20T18:42:16.285 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-09-20T18:42:16.286 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:42:16.286 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repository Size 2024-09-20T18:42:16.286 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:42:16.286 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-09-20T18:42:16.287 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5099.g88deadb8.el9 ceph-noarch 154 k 2024-09-20T18:42:16.287 INFO:teuthology.orchestra.run.smithi142.stdout:Installing dependencies: 2024-09-20T18:42:16.287 INFO:teuthology.orchestra.run.smithi142.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-20T18:42:16.287 INFO:teuthology.orchestra.run.smithi142.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-20T18:42:16.287 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-20T18:42:16.287 INFO:teuthology.orchestra.run.smithi142.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-20T18:42:16.287 INFO:teuthology.orchestra.run.smithi142.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-20T18:42:16.287 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:42:16.287 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-09-20T18:42:16.287 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:42:16.287 INFO:teuthology.orchestra.run.smithi142.stdout:Install 6 Packages 2024-09-20T18:42:16.287 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:42:16.288 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 7.0 M 2024-09-20T18:42:16.288 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 33 M 2024-09-20T18:42:16.288 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-09-20T18:42:16.403 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephfs-devel-2:19.3.0-5099.g88deadb8.el9.x86_64 1/1 2024-09-20T18:42:16.403 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:42:16.403 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2024-09-20T18:42:16.403 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs-devel-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:42:16.403 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:42:16.403 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-20T18:42:16.584 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install librados2 2024-09-20T18:42:16.731 INFO:teuthology.orchestra.run.smithi142.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.6 MB/s | 248 kB 00:00 2024-09-20T18:42:16.781 INFO:teuthology.orchestra.run.smithi142.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 11 MB/s | 548 kB 00:00 2024-09-20T18:42:16.806 INFO:teuthology.orchestra.run.smithi142.stdout:(3/6): ceph-mgr-cephadm-19.3.0-5099.g88deadb8.e 682 kB/s | 154 kB 00:00 2024-09-20T18:42:16.832 INFO:teuthology.orchestra.run.smithi142.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.1 MB/s | 58 kB 00:00 2024-09-20T18:42:16.882 INFO:teuthology.orchestra.run.smithi142.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 989 kB/s | 74 kB 00:00 2024-09-20T18:42:16.974 INFO:teuthology.orchestra.run.smithi142.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-09-20T18:42:16.976 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-09-20T18:42:16.977 INFO:teuthology.orchestra.run.smithi142.stdout:Total 10 MB/s | 7.0 MB 00:00 2024-09-20T18:42:17.047 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-09-20T18:42:17.058 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-09-20T18:42:17.058 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-09-20T18:42:17.152 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:40 ago on Fri 20 Sep 2024 06:40:37 PM UTC. 2024-09-20T18:42:17.158 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-09-20T18:42:17.159 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-09-20T18:42:17.241 INFO:teuthology.orchestra.run.smithi083.stdout:Package librados2-2:19.3.0-5099.g88deadb8.el9.x86_64 is already installed. 2024-09-20T18:42:17.284 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-20T18:42:17.285 INFO:teuthology.orchestra.run.smithi083.stdout:Nothing to do. 2024-09-20T18:42:17.285 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-20T18:42:17.347 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install librbd1 2024-09-20T18:42:17.467 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-09-20T18:42:17.640 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-20T18:42:17.692 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-20T18:42:17.934 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:40 ago on Fri 20 Sep 2024 06:40:37 PM UTC. 2024-09-20T18:42:18.025 INFO:teuthology.orchestra.run.smithi083.stdout:Package librbd1-2:19.3.0-5099.g88deadb8.el9.x86_64 is already installed. 2024-09-20T18:42:18.069 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-20T18:42:18.070 INFO:teuthology.orchestra.run.smithi083.stdout:Nothing to do. 2024-09-20T18:42:18.070 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-20T18:42:18.136 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install python3-rados 2024-09-20T18:42:18.209 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-20T18:42:18.296 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-20T18:42:18.363 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-20T18:42:18.388 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5099.g88deadb8.el9.noarch 6/6 2024-09-20T18:42:18.717 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:41 ago on Fri 20 Sep 2024 06:40:37 PM UTC. 2024-09-20T18:42:18.808 INFO:teuthology.orchestra.run.smithi083.stdout:Package python3-rados-2:19.3.0-5099.g88deadb8.el9.x86_64 is already installed. 2024-09-20T18:42:18.851 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-20T18:42:18.852 INFO:teuthology.orchestra.run.smithi083.stdout:Nothing to do. 2024-09-20T18:42:18.852 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-20T18:42:18.919 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install python3-rgw 2024-09-20T18:42:19.128 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5099.g88deadb8.el9.noarch 6/6 2024-09-20T18:42:19.129 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5099.g88deadb8.el9.noarch 1/6 2024-09-20T18:42:19.129 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-20T18:42:19.129 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-20T18:42:19.129 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-20T18:42:19.129 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-20T18:42:19.398 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-20T18:42:19.398 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:42:19.398 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-09-20T18:42:19.398 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-cephadm-2:19.3.0-5099.g88deadb8.el9.noarch 2024-09-20T18:42:19.398 INFO:teuthology.orchestra.run.smithi142.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-20T18:42:19.398 INFO:teuthology.orchestra.run.smithi142.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-20T18:42:19.398 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-20T18:42:19.398 INFO:teuthology.orchestra.run.smithi142.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-20T18:42:19.399 INFO:teuthology.orchestra.run.smithi142.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-20T18:42:19.399 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:42:19.399 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-09-20T18:42:19.497 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:42 ago on Fri 20 Sep 2024 06:40:37 PM UTC. 2024-09-20T18:42:19.586 INFO:teuthology.orchestra.run.smithi083.stdout:Package python3-rgw-2:19.3.0-5099.g88deadb8.el9.x86_64 is already installed. 2024-09-20T18:42:19.629 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-20T18:42:19.630 INFO:teuthology.orchestra.run.smithi083.stdout:Nothing to do. 2024-09-20T18:42:19.630 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-20T18:42:19.657 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install ceph-fuse 2024-09-20T18:42:19.694 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install python3-cephfs 2024-09-20T18:42:20.243 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:01:48 ago on Fri 20 Sep 2024 06:40:32 PM UTC. 2024-09-20T18:42:20.273 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:43 ago on Fri 20 Sep 2024 06:40:37 PM UTC. 2024-09-20T18:42:20.362 INFO:teuthology.orchestra.run.smithi083.stdout:Package python3-cephfs-2:19.3.0-5099.g88deadb8.el9.x86_64 is already installed. 2024-09-20T18:42:20.376 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-09-20T18:42:20.377 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:42:20.377 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repository Size 2024-09-20T18:42:20.377 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:42:20.377 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-09-20T18:42:20.377 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-fuse x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 899 k 2024-09-20T18:42:20.377 INFO:teuthology.orchestra.run.smithi142.stdout:Installing dependencies: 2024-09-20T18:42:20.377 INFO:teuthology.orchestra.run.smithi142.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-20T18:42:20.377 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:42:20.377 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-09-20T18:42:20.377 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:42:20.377 INFO:teuthology.orchestra.run.smithi142.stdout:Install 2 Packages 2024-09-20T18:42:20.378 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:42:20.378 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 979 k 2024-09-20T18:42:20.378 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 2.9 M 2024-09-20T18:42:20.378 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-09-20T18:42:20.407 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-20T18:42:20.407 INFO:teuthology.orchestra.run.smithi083.stdout:Nothing to do. 2024-09-20T18:42:20.408 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-20T18:42:20.472 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install python3-rbd 2024-09-20T18:42:20.534 INFO:teuthology.orchestra.run.smithi142.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 595 kB/s | 80 kB 00:00 2024-09-20T18:42:20.651 INFO:teuthology.orchestra.run.smithi142.stdout:(2/2): ceph-fuse-19.3.0-5099.g88deadb8.el9.x86_ 3.5 MB/s | 899 kB 00:00 2024-09-20T18:42:20.651 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-09-20T18:42:20.651 INFO:teuthology.orchestra.run.smithi142.stdout:Total 3.5 MB/s | 979 kB 00:00 2024-09-20T18:42:20.663 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-09-20T18:42:20.686 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-09-20T18:42:20.686 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-09-20T18:42:20.768 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-09-20T18:42:20.768 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-09-20T18:42:20.959 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-09-20T18:42:21.052 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:44 ago on Fri 20 Sep 2024 06:40:37 PM UTC. 2024-09-20T18:42:21.095 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-20T18:42:21.142 INFO:teuthology.orchestra.run.smithi083.stdout:Package python3-rbd-2:19.3.0-5099.g88deadb8.el9.x86_64 is already installed. 2024-09-20T18:42:21.167 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-fuse-2:19.3.0-5099.g88deadb8.el9.x86_64 2/2 2024-09-20T18:42:21.185 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-20T18:42:21.186 INFO:teuthology.orchestra.run.smithi083.stdout:Nothing to do. 2024-09-20T18:42:21.186 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-20T18:42:21.253 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install rbd-fuse 2024-09-20T18:42:21.818 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:44 ago on Fri 20 Sep 2024 06:40:37 PM UTC. 2024-09-20T18:42:21.828 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-fuse-2:19.3.0-5099.g88deadb8.el9.x86_64 2/2 2024-09-20T18:42:21.828 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-fuse-2:19.3.0-5099.g88deadb8.el9.x86_64 1/2 2024-09-20T18:42:21.949 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-20T18:42:21.950 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:42:21.950 INFO:teuthology.orchestra.run.smithi083.stdout: Package Architecture Version Repository Size 2024-09-20T18:42:21.950 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:42:21.950 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2024-09-20T18:42:21.950 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-fuse x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 85 k 2024-09-20T18:42:21.950 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:42:21.951 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-09-20T18:42:21.951 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:42:21.951 INFO:teuthology.orchestra.run.smithi083.stdout:Install 1 Package 2024-09-20T18:42:21.951 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:42:21.951 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 85 k 2024-09-20T18:42:21.951 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 231 k 2024-09-20T18:42:21.951 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-09-20T18:42:22.094 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-20T18:42:22.094 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:42:22.094 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-09-20T18:42:22.095 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-fuse-2:19.3.0-5099.g88deadb8.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-20T18:42:22.095 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:42:22.095 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-09-20T18:42:22.152 INFO:teuthology.orchestra.run.smithi083.stdout:rbd-fuse-19.3.0-5099.g88deadb8.el9.x86_64.rpm 424 kB/s | 85 kB 00:00 2024-09-20T18:42:22.152 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-09-20T18:42:22.153 INFO:teuthology.orchestra.run.smithi083.stdout:Total 421 kB/s | 85 kB 00:00 2024-09-20T18:42:22.153 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-09-20T18:42:22.160 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-09-20T18:42:22.160 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-09-20T18:42:22.223 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-09-20T18:42:22.223 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-09-20T18:42:22.345 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install ceph-volume 2024-09-20T18:42:22.375 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-09-20T18:42:22.474 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-fuse-2:19.3.0-5099.g88deadb8.el9.x86_64 1/1 2024-09-20T18:42:22.842 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: rbd-fuse-2:19.3.0-5099.g88deadb8.el9.x86_64 1/1 2024-09-20T18:42:22.928 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:01:50 ago on Fri 20 Sep 2024 06:40:32 PM UTC. 2024-09-20T18:42:23.065 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-09-20T18:42:23.065 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:42:23.065 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repository Size 2024-09-20T18:42:23.066 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:42:23.066 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-09-20T18:42:23.066 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-volume noarch 2:19.3.0-5099.g88deadb8.el9 ceph-noarch 287 k 2024-09-20T18:42:23.066 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:42:23.066 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-09-20T18:42:23.066 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:42:23.066 INFO:teuthology.orchestra.run.smithi142.stdout:Install 1 Package 2024-09-20T18:42:23.066 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:42:23.066 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 287 k 2024-09-20T18:42:23.066 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 1.4 M 2024-09-20T18:42:23.066 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-09-20T18:42:23.096 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-fuse-2:19.3.0-5099.g88deadb8.el9.x86_64 1/1 2024-09-20T18:42:23.097 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:42:23.097 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2024-09-20T18:42:23.097 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-fuse-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:42:23.097 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:42:23.097 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-20T18:42:23.279 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-volume-19.3.0-5099.g88deadb8.el9.noarch.rp 1.3 MB/s | 287 kB 00:00 2024-09-20T18:42:23.280 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-09-20T18:42:23.280 INFO:teuthology.orchestra.run.smithi142.stdout:Total 1.3 MB/s | 287 kB 00:00 2024-09-20T18:42:23.280 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-09-20T18:42:23.288 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-09-20T18:42:23.289 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-09-20T18:42:23.316 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-09-20T18:42:23.316 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-09-20T18:42:23.352 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install rbd-mirror 2024-09-20T18:42:23.534 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-09-20T18:42:23.598 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-volume-2:19.3.0-5099.g88deadb8.el9.noarch 1/1 2024-09-20T18:42:23.614 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-volume-2:19.3.0-5099.g88deadb8.el9.noarch 1/1 2024-09-20T18:42:23.614 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-20T18:42:23.614 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-20T18:42:23.614 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:42:23.935 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:46 ago on Fri 20 Sep 2024 06:40:37 PM UTC. 2024-09-20T18:42:24.073 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-20T18:42:24.074 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:42:24.074 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repo Size 2024-09-20T18:42:24.075 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:42:24.075 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2024-09-20T18:42:24.075 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-mirror x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 3.2 M 2024-09-20T18:42:24.075 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:42:24.075 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-09-20T18:42:24.075 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:42:24.075 INFO:teuthology.orchestra.run.smithi083.stdout:Install 1 Package 2024-09-20T18:42:24.075 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:42:24.075 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 3.2 M 2024-09-20T18:42:24.075 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 14 M 2024-09-20T18:42:24.075 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-09-20T18:42:24.406 INFO:teuthology.orchestra.run.smithi083.stdout:rbd-mirror-19.3.0-5099.g88deadb8.el9.x86_64.rpm 9.6 MB/s | 3.2 MB 00:00 2024-09-20T18:42:24.407 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-09-20T18:42:24.407 INFO:teuthology.orchestra.run.smithi083.stdout:Total 9.6 MB/s | 3.2 MB 00:00 2024-09-20T18:42:24.407 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-09-20T18:42:24.417 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-09-20T18:42:24.417 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-09-20T18:42:24.503 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-09-20T18:42:24.504 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-09-20T18:42:24.592 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-volume-2:19.3.0-5099.g88deadb8.el9.noarch 1/1 2024-09-20T18:42:24.592 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:42:24.592 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-09-20T18:42:24.592 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-volume-2:19.3.0-5099.g88deadb8.el9.noarch 2024-09-20T18:42:24.593 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:42:24.593 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-09-20T18:42:24.880 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install librados-devel 2024-09-20T18:42:24.999 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-09-20T18:42:25.074 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-mirror-2:19.3.0-5099.g88deadb8.el9.x86_64 1/1 2024-09-20T18:42:25.099 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5099.g88deadb8.el9.x86_64 1/1 2024-09-20T18:42:25.099 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-20T18:42:25.099 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-20T18:42:25.100 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-20T18:42:25.100 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-20T18:42:25.100 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:42:25.474 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:01:53 ago on Fri 20 Sep 2024 06:40:32 PM UTC. 2024-09-20T18:42:25.610 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-09-20T18:42:25.611 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:42:25.611 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repo Size 2024-09-20T18:42:25.611 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:42:25.611 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-09-20T18:42:25.611 INFO:teuthology.orchestra.run.smithi142.stdout: librados-devel x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 126 k 2024-09-20T18:42:25.611 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:42:25.611 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-09-20T18:42:25.611 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:42:25.611 INFO:teuthology.orchestra.run.smithi142.stdout:Install 1 Package 2024-09-20T18:42:25.611 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:42:25.611 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 126 k 2024-09-20T18:42:25.612 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 456 k 2024-09-20T18:42:25.612 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-09-20T18:42:25.817 INFO:teuthology.orchestra.run.smithi142.stdout:librados-devel-19.3.0-5099.g88deadb8.el9.x86_64 615 kB/s | 126 kB 00:00 2024-09-20T18:42:25.818 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-09-20T18:42:25.818 INFO:teuthology.orchestra.run.smithi142.stdout:Total 611 kB/s | 126 kB 00:00 2024-09-20T18:42:25.818 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-09-20T18:42:25.823 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-09-20T18:42:25.824 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-09-20T18:42:25.887 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-09-20T18:42:25.887 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-09-20T18:42:26.041 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-09-20T18:42:26.156 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librados-devel-2:19.3.0-5099.g88deadb8.el9.x86_64 1/1 2024-09-20T18:42:26.235 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-mirror-2:19.3.0-5099.g88deadb8.el9.x86_64 1/1 2024-09-20T18:42:26.235 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:42:26.235 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2024-09-20T18:42:26.235 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-mirror-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:42:26.235 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:42:26.235 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-20T18:42:26.452 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install rbd-nbd 2024-09-20T18:42:26.608 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librados-devel-2:19.3.0-5099.g88deadb8.el9.x86_64 1/1 2024-09-20T18:42:26.865 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados-devel-2:19.3.0-5099.g88deadb8.el9.x86_64 1/1 2024-09-20T18:42:26.866 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:42:26.866 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-09-20T18:42:26.866 INFO:teuthology.orchestra.run.smithi142.stdout: librados-devel-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:42:26.866 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:42:26.866 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-09-20T18:42:27.045 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:50 ago on Fri 20 Sep 2024 06:40:37 PM UTC. 2024-09-20T18:42:27.127 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install libcephfs2 2024-09-20T18:42:27.175 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-20T18:42:27.176 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:42:27.176 INFO:teuthology.orchestra.run.smithi083.stdout: Package Architecture Version Repository Size 2024-09-20T18:42:27.176 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:42:27.176 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2024-09-20T18:42:27.176 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-nbd x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 171 k 2024-09-20T18:42:27.176 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:42:27.176 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-09-20T18:42:27.176 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-20T18:42:27.176 INFO:teuthology.orchestra.run.smithi083.stdout:Install 1 Package 2024-09-20T18:42:27.176 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:42:27.177 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 171 k 2024-09-20T18:42:27.177 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 490 k 2024-09-20T18:42:27.177 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-09-20T18:42:27.385 INFO:teuthology.orchestra.run.smithi083.stdout:rbd-nbd-19.3.0-5099.g88deadb8.el9.x86_64.rpm 821 kB/s | 171 kB 00:00 2024-09-20T18:42:27.386 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-09-20T18:42:27.386 INFO:teuthology.orchestra.run.smithi083.stdout:Total 814 kB/s | 171 kB 00:00 2024-09-20T18:42:27.386 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-09-20T18:42:27.394 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-09-20T18:42:27.394 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-09-20T18:42:27.459 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-09-20T18:42:27.459 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-09-20T18:42:27.616 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-09-20T18:42:27.715 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:01:55 ago on Fri 20 Sep 2024 06:40:32 PM UTC. 2024-09-20T18:42:27.725 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-nbd-2:19.3.0-5099.g88deadb8.el9.x86_64 1/1 2024-09-20T18:42:27.808 INFO:teuthology.orchestra.run.smithi142.stdout:Package libcephfs2-2:19.3.0-5099.g88deadb8.el9.x86_64 is already installed. 2024-09-20T18:42:27.853 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-09-20T18:42:27.854 INFO:teuthology.orchestra.run.smithi142.stdout:Nothing to do. 2024-09-20T18:42:27.855 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-09-20T18:42:27.916 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install libcephfs-devel 2024-09-20T18:42:28.159 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: rbd-nbd-2:19.3.0-5099.g88deadb8.el9.x86_64 1/1 2024-09-20T18:42:28.419 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-nbd-2:19.3.0-5099.g88deadb8.el9.x86_64 1/1 2024-09-20T18:42:28.419 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:42:28.419 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2024-09-20T18:42:28.420 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-nbd-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:42:28.420 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:42:28.420 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-20T18:42:28.509 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:01:56 ago on Fri 20 Sep 2024 06:40:32 PM UTC. 2024-09-20T18:42:28.641 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-09-20T18:42:28.642 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:42:28.643 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repo Size 2024-09-20T18:42:28.643 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:42:28.643 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-09-20T18:42:28.643 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs-devel x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 32 k 2024-09-20T18:42:28.643 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:42:28.644 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-09-20T18:42:28.644 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:42:28.644 INFO:teuthology.orchestra.run.smithi142.stdout:Install 1 Package 2024-09-20T18:42:28.644 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:42:28.644 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 32 k 2024-09-20T18:42:28.644 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 142 k 2024-09-20T18:42:28.644 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-09-20T18:42:28.648 DEBUG:teuthology.parallel:result is None 2024-09-20T18:42:28.796 INFO:teuthology.orchestra.run.smithi142.stdout:libcephfs-devel-19.3.0-5099.g88deadb8.el9.x86_6 207 kB/s | 32 kB 00:00 2024-09-20T18:42:28.796 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-09-20T18:42:28.796 INFO:teuthology.orchestra.run.smithi142.stdout:Total 205 kB/s | 32 kB 00:00 2024-09-20T18:42:28.796 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-09-20T18:42:28.800 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-09-20T18:42:28.800 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-09-20T18:42:28.820 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-09-20T18:42:28.820 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-09-20T18:42:28.913 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-09-20T18:42:29.018 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephfs-devel-2:19.3.0-5099.g88deadb8.el9.x86_64 1/1 2024-09-20T18:42:29.391 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-5099.g88deadb8.el9.x86_64 1/1 2024-09-20T18:42:29.623 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephfs-devel-2:19.3.0-5099.g88deadb8.el9.x86_64 1/1 2024-09-20T18:42:29.623 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:42:29.623 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-09-20T18:42:29.623 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs-devel-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:42:29.624 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:42:29.624 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-09-20T18:42:29.777 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install librados2 2024-09-20T18:42:30.354 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:01:58 ago on Fri 20 Sep 2024 06:40:32 PM UTC. 2024-09-20T18:42:30.444 INFO:teuthology.orchestra.run.smithi142.stdout:Package librados2-2:19.3.0-5099.g88deadb8.el9.x86_64 is already installed. 2024-09-20T18:42:30.488 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-09-20T18:42:30.489 INFO:teuthology.orchestra.run.smithi142.stdout:Nothing to do. 2024-09-20T18:42:30.489 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-09-20T18:42:30.556 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install librbd1 2024-09-20T18:42:31.135 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:01:59 ago on Fri 20 Sep 2024 06:40:32 PM UTC. 2024-09-20T18:42:31.225 INFO:teuthology.orchestra.run.smithi142.stdout:Package librbd1-2:19.3.0-5099.g88deadb8.el9.x86_64 is already installed. 2024-09-20T18:42:31.269 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-09-20T18:42:31.270 INFO:teuthology.orchestra.run.smithi142.stdout:Nothing to do. 2024-09-20T18:42:31.270 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-09-20T18:42:31.340 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install python3-rados 2024-09-20T18:42:31.921 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:01:59 ago on Fri 20 Sep 2024 06:40:32 PM UTC. 2024-09-20T18:42:32.009 INFO:teuthology.orchestra.run.smithi142.stdout:Package python3-rados-2:19.3.0-5099.g88deadb8.el9.x86_64 is already installed. 2024-09-20T18:42:32.053 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-09-20T18:42:32.054 INFO:teuthology.orchestra.run.smithi142.stdout:Nothing to do. 2024-09-20T18:42:32.054 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-09-20T18:42:32.123 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install python3-rgw 2024-09-20T18:42:32.702 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:02:00 ago on Fri 20 Sep 2024 06:40:32 PM UTC. 2024-09-20T18:42:32.792 INFO:teuthology.orchestra.run.smithi142.stdout:Package python3-rgw-2:19.3.0-5099.g88deadb8.el9.x86_64 is already installed. 2024-09-20T18:42:32.835 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-09-20T18:42:32.836 INFO:teuthology.orchestra.run.smithi142.stdout:Nothing to do. 2024-09-20T18:42:32.836 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-09-20T18:42:32.899 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install python3-cephfs 2024-09-20T18:42:33.479 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:02:01 ago on Fri 20 Sep 2024 06:40:32 PM UTC. 2024-09-20T18:42:33.568 INFO:teuthology.orchestra.run.smithi142.stdout:Package python3-cephfs-2:19.3.0-5099.g88deadb8.el9.x86_64 is already installed. 2024-09-20T18:42:33.611 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-09-20T18:42:33.612 INFO:teuthology.orchestra.run.smithi142.stdout:Nothing to do. 2024-09-20T18:42:33.612 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-09-20T18:42:33.677 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install python3-rbd 2024-09-20T18:42:34.256 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:02:02 ago on Fri 20 Sep 2024 06:40:32 PM UTC. 2024-09-20T18:42:34.345 INFO:teuthology.orchestra.run.smithi142.stdout:Package python3-rbd-2:19.3.0-5099.g88deadb8.el9.x86_64 is already installed. 2024-09-20T18:42:34.389 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-09-20T18:42:34.390 INFO:teuthology.orchestra.run.smithi142.stdout:Nothing to do. 2024-09-20T18:42:34.391 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-09-20T18:42:34.458 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install rbd-fuse 2024-09-20T18:42:35.039 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:02:03 ago on Fri 20 Sep 2024 06:40:32 PM UTC. 2024-09-20T18:42:35.171 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-09-20T18:42:35.172 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:42:35.172 INFO:teuthology.orchestra.run.smithi142.stdout: Package Architecture Version Repository Size 2024-09-20T18:42:35.172 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:42:35.173 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-09-20T18:42:35.173 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-fuse x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 85 k 2024-09-20T18:42:35.173 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:42:35.173 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-09-20T18:42:35.173 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:42:35.173 INFO:teuthology.orchestra.run.smithi142.stdout:Install 1 Package 2024-09-20T18:42:35.173 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:42:35.173 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 85 k 2024-09-20T18:42:35.173 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 231 k 2024-09-20T18:42:35.173 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-09-20T18:42:35.374 INFO:teuthology.orchestra.run.smithi142.stdout:rbd-fuse-19.3.0-5099.g88deadb8.el9.x86_64.rpm 425 kB/s | 85 kB 00:00 2024-09-20T18:42:35.374 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-09-20T18:42:35.375 INFO:teuthology.orchestra.run.smithi142.stdout:Total 422 kB/s | 85 kB 00:00 2024-09-20T18:42:35.375 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-09-20T18:42:35.381 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-09-20T18:42:35.382 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-09-20T18:42:35.446 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-09-20T18:42:35.446 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-09-20T18:42:35.599 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-09-20T18:42:35.696 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-fuse-2:19.3.0-5099.g88deadb8.el9.x86_64 1/1 2024-09-20T18:42:36.086 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: rbd-fuse-2:19.3.0-5099.g88deadb8.el9.x86_64 1/1 2024-09-20T18:42:36.338 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-fuse-2:19.3.0-5099.g88deadb8.el9.x86_64 1/1 2024-09-20T18:42:36.338 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:42:36.338 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-09-20T18:42:36.338 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-fuse-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:42:36.339 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:42:36.339 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-09-20T18:42:36.582 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install rbd-mirror 2024-09-20T18:42:37.170 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:02:05 ago on Fri 20 Sep 2024 06:40:32 PM UTC. 2024-09-20T18:42:37.315 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-09-20T18:42:37.316 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:42:37.316 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repo Size 2024-09-20T18:42:37.316 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:42:37.316 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-09-20T18:42:37.316 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-mirror x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 3.2 M 2024-09-20T18:42:37.316 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:42:37.317 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-09-20T18:42:37.317 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:42:37.317 INFO:teuthology.orchestra.run.smithi142.stdout:Install 1 Package 2024-09-20T18:42:37.317 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:42:37.317 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 3.2 M 2024-09-20T18:42:37.317 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 14 M 2024-09-20T18:42:37.317 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-09-20T18:42:37.627 INFO:teuthology.orchestra.run.smithi142.stdout:rbd-mirror-19.3.0-5099.g88deadb8.el9.x86_64.rpm 10 MB/s | 3.2 MB 00:00 2024-09-20T18:42:37.627 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-09-20T18:42:37.628 INFO:teuthology.orchestra.run.smithi142.stdout:Total 10 MB/s | 3.2 MB 00:00 2024-09-20T18:42:37.628 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-09-20T18:42:37.637 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-09-20T18:42:37.637 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-09-20T18:42:37.724 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-09-20T18:42:37.724 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-09-20T18:42:38.218 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-09-20T18:42:38.277 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-mirror-2:19.3.0-5099.g88deadb8.el9.x86_64 1/1 2024-09-20T18:42:38.304 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5099.g88deadb8.el9.x86_64 1/1 2024-09-20T18:42:38.304 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-20T18:42:38.304 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-20T18:42:38.304 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-20T18:42:38.304 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-20T18:42:38.304 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:42:39.365 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-mirror-2:19.3.0-5099.g88deadb8.el9.x86_64 1/1 2024-09-20T18:42:39.366 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:42:39.366 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-09-20T18:42:39.366 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-mirror-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:42:39.366 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:42:39.366 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-09-20T18:42:39.561 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install rbd-nbd 2024-09-20T18:42:40.155 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:02:08 ago on Fri 20 Sep 2024 06:40:32 PM UTC. 2024-09-20T18:42:40.288 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-09-20T18:42:40.289 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:42:40.290 INFO:teuthology.orchestra.run.smithi142.stdout: Package Architecture Version Repository Size 2024-09-20T18:42:40.290 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:42:40.290 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-09-20T18:42:40.290 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-nbd x86_64 2:19.3.0-5099.g88deadb8.el9 ceph 171 k 2024-09-20T18:42:40.290 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:42:40.291 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-09-20T18:42:40.291 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-09-20T18:42:40.291 INFO:teuthology.orchestra.run.smithi142.stdout:Install 1 Package 2024-09-20T18:42:40.291 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:42:40.291 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 171 k 2024-09-20T18:42:40.291 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 490 k 2024-09-20T18:42:40.291 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-09-20T18:42:40.488 INFO:teuthology.orchestra.run.smithi142.stdout:rbd-nbd-19.3.0-5099.g88deadb8.el9.x86_64.rpm 864 kB/s | 171 kB 00:00 2024-09-20T18:42:40.489 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-09-20T18:42:40.489 INFO:teuthology.orchestra.run.smithi142.stdout:Total 857 kB/s | 171 kB 00:00 2024-09-20T18:42:40.489 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-09-20T18:42:40.497 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-09-20T18:42:40.497 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-09-20T18:42:40.562 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-09-20T18:42:40.562 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-09-20T18:42:40.722 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-09-20T18:42:40.828 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-nbd-2:19.3.0-5099.g88deadb8.el9.x86_64 1/1 2024-09-20T18:42:41.296 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: rbd-nbd-2:19.3.0-5099.g88deadb8.el9.x86_64 1/1 2024-09-20T18:42:41.558 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-nbd-2:19.3.0-5099.g88deadb8.el9.x86_64 1/1 2024-09-20T18:42:41.559 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:42:41.559 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-09-20T18:42:41.559 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-nbd-2:19.3.0-5099.g88deadb8.el9.x86_64 2024-09-20T18:42:41.559 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:42:41.559 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-09-20T18:42:41.775 DEBUG:teuthology.parallel:result is None 2024-09-20T18:42:41.776 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=88deadb8082f4a33832e228ccefc59cedba158a5 2024-09-20T18:42:41.926 DEBUG:teuthology.orchestra.run.smithi083:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-20T18:42:41.952 INFO:teuthology.orchestra.run.smithi083.stdout:19.3.0-5099.g88deadb8.el9 2024-09-20T18:42:41.952 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5099.g88deadb8.el9 2024-09-20T18:42:41.953 INFO:teuthology.task.install:The correct ceph version 19.3.0-5099.g88deadb8 is installed. 2024-09-20T18:42:41.955 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=88deadb8082f4a33832e228ccefc59cedba158a5 2024-09-20T18:42:42.089 DEBUG:teuthology.orchestra.run.smithi142:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-20T18:42:42.112 INFO:teuthology.orchestra.run.smithi142.stdout:19.3.0-5099.g88deadb8.el9 2024-09-20T18:42:42.113 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5099.g88deadb8.el9 2024-09-20T18:42:42.113 INFO:teuthology.task.install:The correct ceph version 19.3.0-5099.g88deadb8 is installed. 2024-09-20T18:42:42.116 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-20T18:42:42.116 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-20T18:42:42.116 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-20T18:42:42.147 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-20T18:42:42.147 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-20T18:42:42.179 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-20T18:42:42.179 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-20T18:42:42.179 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/daemon-helper 2024-09-20T18:42:42.216 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-20T18:42:42.283 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-20T18:42:42.283 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/daemon-helper 2024-09-20T18:42:42.308 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-20T18:42:42.374 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-20T18:42:42.375 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-20T18:42:42.375 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-20T18:42:42.402 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-20T18:42:42.469 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-20T18:42:42.469 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-20T18:42:42.493 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-20T18:42:42.562 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-20T18:42:42.562 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-20T18:42:42.562 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/stdin-killer 2024-09-20T18:42:42.590 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-20T18:42:42.656 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-20T18:42:42.656 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/stdin-killer 2024-09-20T18:42:42.681 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-20T18:42:42.746 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-20T18:42:42.860 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'client.0': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'ms bind msgr1': False, 'ms bind msgr2': True, 'ms type': 'async'}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd max backfills': 3, 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'CEPHADM_STRAY_DAEMON', 'CEPHADM_FAILED_DAEMON', 'but it is still running', 'objects unfound and apparently lost', '\\(POOL_APP_NOT_ENABLED\\)'], 'log-only-match': ['CEPHADM_'], 'sha1': '88deadb8082f4a33832e228ccefc59cedba158a5', 'cephadm_mode': 'root'} 2024-09-20T18:42:42.860 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 2024-09-20T18:42:42.861 INFO:tasks.cephadm:Cluster fsid is 1ed5a50a-7780-11ef-baf5-efdc52797490 2024-09-20T18:42:42.861 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-20T18:42:42.861 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.83', 'mon.c': '[v2:172.21.15.83:3301,v1:172.21.15.83:6790]', 'mon.b': '172.21.15.142'} 2024-09-20T18:42:42.861 INFO:tasks.cephadm:First mon is mon.a on smithi083 2024-09-20T18:42:42.861 INFO:tasks.cephadm:First mgr is y 2024-09-20T18:42:42.861 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-20T18:42:42.862 DEBUG:teuthology.orchestra.run.smithi083:> sudo hostname $(hostname -s) 2024-09-20T18:42:42.889 DEBUG:teuthology.orchestra.run.smithi142:> sudo hostname $(hostname -s) 2024-09-20T18:42:42.914 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-20T18:42:42.914 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=88deadb8082f4a33832e228ccefc59cedba158a5 2024-09-20T18:42:43.054 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '88deadb8082f4a33832e228ccefc59cedba158a5', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/83186/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-5099-g88deadb8', 'node_name': '172.21.2.22+braggi22', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-5099.g88deadb8'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-20-1028/88deadb8082f4a33832e228ccefc59cedba158a5/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-20 12:11:57.149019', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-guits-main-2024-09-20-1028', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-guits-main-2024-09-20-1028/88deadb8082f4a33832e228ccefc59cedba158a5/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-20T18:42:43.196 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-guits-main-2024-09-20-1028, sha1 88deadb8082f4a33832e228ccefc59cedba158a5 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=88deadb8082f4a33832e228ccefc59cedba158a5 2024-09-20T18:42:43.198 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-guits-main-2024-09-20-1028/88deadb8082f4a33832e228ccefc59cedba158a5/centos/9/x86_64/flavors/default/cephadm 2024-09-20T18:42:43.198 INFO:tasks.cephadm:Downloading cephadm from url: https://3.chacra.ceph.com/binaries/ceph/wip-guits-main-2024-09-20-1028/88deadb8082f4a33832e228ccefc59cedba158a5/centos/9/x86_64/flavors/default/cephadm 2024-09-20T18:42:43.198 DEBUG:teuthology.orchestra.run.smithi083:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-guits-main-2024-09-20-1028/88deadb8082f4a33832e228ccefc59cedba158a5/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-20T18:42:43.752 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r--. 1 ubuntu ubuntu 804255 Sep 20 18:42 /home/ubuntu/cephtest/cephadm 2024-09-20T18:42:43.753 DEBUG:teuthology.orchestra.run.smithi142:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-guits-main-2024-09-20-1028/88deadb8082f4a33832e228ccefc59cedba158a5/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-20T18:42:44.305 INFO:teuthology.orchestra.run.smithi142.stdout:-rw-r--r--. 1 ubuntu ubuntu 804255 Sep 20 18:42 /home/ubuntu/cephtest/cephadm 2024-09-20T18:42:44.305 DEBUG:teuthology.orchestra.run.smithi083:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-20T18:42:44.325 DEBUG:teuthology.orchestra.run.smithi142:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-20T18:42:44.355 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 on all hosts... 2024-09-20T18:42:44.356 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 pull 2024-09-20T18:42:44.368 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 pull 2024-09-20T18:42:45.935 INFO:teuthology.orchestra.run.smithi142.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5... 2024-09-20T18:42:46.075 INFO:teuthology.orchestra.run.smithi083.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5... 2024-09-20T18:43:16.168 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T18:43:16.168 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_version": "ceph version 19.3.0-5099-g88deadb8 (88deadb8082f4a33832e228ccefc59cedba158a5) squid (dev)", 2024-09-20T18:43:16.169 INFO:teuthology.orchestra.run.smithi142.stdout: "image_id": "441f9a3fe51528ef653b21f42e9d884db6124a0c7d482e865382bdfdb9ec979c", 2024-09-20T18:43:16.169 INFO:teuthology.orchestra.run.smithi142.stdout: "repo_digests": [ 2024-09-20T18:43:16.169 INFO:teuthology.orchestra.run.smithi142.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2" 2024-09-20T18:43:16.169 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:43:16.169 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T18:43:50.028 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T18:43:50.028 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_version": "ceph version 19.3.0-5099-g88deadb8 (88deadb8082f4a33832e228ccefc59cedba158a5) squid (dev)", 2024-09-20T18:43:50.028 INFO:teuthology.orchestra.run.smithi083.stdout: "image_id": "441f9a3fe51528ef653b21f42e9d884db6124a0c7d482e865382bdfdb9ec979c", 2024-09-20T18:43:50.029 INFO:teuthology.orchestra.run.smithi083.stdout: "repo_digests": [ 2024-09-20T18:43:50.029 INFO:teuthology.orchestra.run.smithi083.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2" 2024-09-20T18:43:50.029 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:43:50.029 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T18:43:50.053 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /etc/ceph 2024-09-20T18:43:50.088 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /etc/ceph 2024-09-20T18:43:50.122 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 777 /etc/ceph 2024-09-20T18:43:50.152 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 777 /etc/ceph 2024-09-20T18:43:50.191 INFO:tasks.cephadm:Writing seed config... 2024-09-20T18:43:50.192 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-20T18:43:50.192 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-20T18:43:50.192 INFO:tasks.cephadm: override: [client.0] debug ms = 1 2024-09-20T18:43:50.192 INFO:tasks.cephadm: override: [client.0] debug objecter = 20 2024-09-20T18:43:50.193 INFO:tasks.cephadm: override: [client.0] debug rados = 20 2024-09-20T18:43:50.193 INFO:tasks.cephadm: override: [global] ms bind msgr1 = False 2024-09-20T18:43:50.193 INFO:tasks.cephadm: override: [global] ms bind msgr2 = True 2024-09-20T18:43:50.193 INFO:tasks.cephadm: override: [global] ms type = async 2024-09-20T18:43:50.193 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-20T18:43:50.193 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-20T18:43:50.193 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-20T18:43:50.193 INFO:tasks.cephadm: override: [mon] mon min osdmap epochs = 50 2024-09-20T18:43:50.193 INFO:tasks.cephadm: override: [mon] mon osdmap full prune interval = 2 2024-09-20T18:43:50.193 INFO:tasks.cephadm: override: [mon] mon osdmap full prune min = 15 2024-09-20T18:43:50.193 INFO:tasks.cephadm: override: [mon] mon osdmap full prune txsize = 2 2024-09-20T18:43:50.193 INFO:tasks.cephadm: override: [mon] paxos service trim min = 10 2024-09-20T18:43:50.193 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-20T18:43:50.193 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-20T18:43:50.193 INFO:tasks.cephadm: override: [osd] osd debug reject backfill probability = 0.3 2024-09-20T18:43:50.194 INFO:tasks.cephadm: override: [osd] osd delete sleep = 1 2024-09-20T18:43:50.194 INFO:tasks.cephadm: override: [osd] osd max backfills = 3 2024-09-20T18:43:50.194 INFO:tasks.cephadm: override: [osd] osd scrub max interval = 120 2024-09-20T18:43:50.194 INFO:tasks.cephadm: override: [osd] osd scrub min interval = 60 2024-09-20T18:43:50.194 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-20T18:43:50.194 INFO:tasks.cephadm: override: [osd] osd snap trim sleep = 2 2024-09-20T18:43:50.194 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-20T18:43:50.195 DEBUG:teuthology.orchestra.run.smithi083:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-20T18:43:50.210 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 1ed5a50a-7780-11ef-baf5-efdc52797490 ms bind msgr1 = False ms bind msgr2 = True ms type = async [osd] osd scrub load threshold = 5.0 osd scrub max interval = 120 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 osd debug reject backfill probability = 0.3 osd delete sleep = 1 osd max backfills = 3 osd scrub min interval = 60 osd snap trim sleep = 2 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon min osdmap epochs = 50 mon osdmap full prune interval = 2 mon osdmap full prune min = 15 mon osdmap full prune txsize = 2 paxos service trim min = 10 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client.0] debug ms = 1 debug objecter = 20 debug rados = 20 2024-09-20T18:43:50.211 DEBUG:teuthology.orchestra.run.smithi083:mon.a> sudo journalctl -f -n 0 -u ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@mon.a.service 2024-09-20T18:43:50.253 DEBUG:teuthology.orchestra.run.smithi083:mgr.y> sudo journalctl -f -n 0 -u ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@mgr.y.service 2024-09-20T18:43:50.296 INFO:tasks.cephadm:Bootstrapping... 2024-09-20T18:43:50.296 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 -v bootstrap --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.83 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-20T18:43:50.511 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-09-20T18:43:50.512 INFO:teuthology.orchestra.run.smithi083.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5', '-v', 'bootstrap', '--fsid', '1ed5a50a-7780-11ef-baf5-efdc52797490', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.83', '--skip-admin-label'] 2024-09-20T18:43:50.512 INFO:teuthology.orchestra.run.smithi083.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-20T18:43:50.512 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying podman|docker is present... 2024-09-20T18:43:50.537 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stdout 5.2.2 2024-09-20T18:43:50.537 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying lvm2 is present... 2024-09-20T18:43:50.537 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying time synchronization is in place... 2024-09-20T18:43:50.545 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-20T18:43:50.545 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-20T18:43:50.551 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-20T18:43:50.551 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout inactive 2024-09-20T18:43:50.558 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout enabled 2024-09-20T18:43:50.564 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout active 2024-09-20T18:43:50.564 INFO:teuthology.orchestra.run.smithi083.stdout:Unit chronyd.service is enabled and running 2024-09-20T18:43:50.564 INFO:teuthology.orchestra.run.smithi083.stdout:Repeating the final host check... 2024-09-20T18:43:50.586 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stdout 5.2.2 2024-09-20T18:43:50.586 INFO:teuthology.orchestra.run.smithi083.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-20T18:43:50.586 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl is present 2024-09-20T18:43:50.586 INFO:teuthology.orchestra.run.smithi083.stdout:lvcreate is present 2024-09-20T18:43:50.592 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-20T18:43:50.592 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-20T18:43:50.599 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-20T18:43:50.599 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout inactive 2024-09-20T18:43:50.605 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout enabled 2024-09-20T18:43:50.612 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout active 2024-09-20T18:43:50.612 INFO:teuthology.orchestra.run.smithi083.stdout:Unit chronyd.service is enabled and running 2024-09-20T18:43:50.612 INFO:teuthology.orchestra.run.smithi083.stdout:Host looks OK 2024-09-20T18:43:50.612 INFO:teuthology.orchestra.run.smithi083.stdout:Cluster fsid: 1ed5a50a-7780-11ef-baf5-efdc52797490 2024-09-20T18:43:50.612 INFO:teuthology.orchestra.run.smithi083.stdout:Acquiring lock 140199509719696 on /run/cephadm/1ed5a50a-7780-11ef-baf5-efdc52797490.lock 2024-09-20T18:43:50.612 INFO:teuthology.orchestra.run.smithi083.stdout:Lock 140199509719696 acquired on /run/cephadm/1ed5a50a-7780-11ef-baf5-efdc52797490.lock 2024-09-20T18:43:50.613 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying IP 172.21.15.83 port 3300 ... 2024-09-20T18:43:50.613 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying IP 172.21.15.83 port 6789 ... 2024-09-20T18:43:50.613 INFO:teuthology.orchestra.run.smithi083.stdout:Base mon IP(s) is [172.21.15.83:3300, 172.21.15.83:6789], mon addrv is [v2:172.21.15.83:3300,v1:172.21.15.83:6789] 2024-09-20T18:43:50.616 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.83 metric 100 2024-09-20T18:43:50.616 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.83 metric 100 2024-09-20T18:43:50.619 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-20T18:43:50.619 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-20T18:43:50.619 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-20T18:43:50.621 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-20T18:43:50.621 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-20T18:43:50.621 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-20T18:43:50.622 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-20T18:43:50.622 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link noprefixroute 2024-09-20T18:43:50.622 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-20T18:43:50.622 INFO:teuthology.orchestra.run.smithi083.stdout:Mon IP `172.21.15.83` is in CIDR network `172.21.0.0/20` 2024-09-20T18:43:50.622 INFO:teuthology.orchestra.run.smithi083.stdout:Mon IP `172.21.15.83` is in CIDR network `172.21.0.0/20` 2024-09-20T18:43:50.622 INFO:teuthology.orchestra.run.smithi083.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-20T18:43:50.623 INFO:teuthology.orchestra.run.smithi083.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-20T18:43:50.623 INFO:teuthology.orchestra.run.smithi083.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5... 2024-09-20T18:43:51.114 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stdout 441f9a3fe51528ef653b21f42e9d884db6124a0c7d482e865382bdfdb9ec979c 2024-09-20T18:43:51.114 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5... 2024-09-20T18:43:51.114 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stderr Getting image source signatures 2024-09-20T18:43:51.114 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-20T18:43:51.114 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stderr Copying blob sha256:9d6c646589fc36228019db3565001a566964d275096901584a195e88fe9853c6 2024-09-20T18:43:51.114 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stderr Copying config sha256:441f9a3fe51528ef653b21f42e9d884db6124a0c7d482e865382bdfdb9ec979c 2024-09-20T18:43:51.115 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-20T18:43:57.471 INFO:teuthology.orchestra.run.smithi083.stdout:ceph: stdout ceph version 19.3.0-5099-g88deadb8 (88deadb8082f4a33832e228ccefc59cedba158a5) squid (dev) 2024-09-20T18:43:57.471 INFO:teuthology.orchestra.run.smithi083.stdout:Ceph version: ceph version 19.3.0-5099-g88deadb8 (88deadb8082f4a33832e228ccefc59cedba158a5) squid (dev) 2024-09-20T18:43:57.471 INFO:teuthology.orchestra.run.smithi083.stdout:Extracting ceph user uid/gid from container image... 2024-09-20T18:44:03.951 INFO:teuthology.orchestra.run.smithi083.stdout:stat: stdout 167 167 2024-09-20T18:44:03.952 INFO:teuthology.orchestra.run.smithi083.stdout:Creating initial keys... 2024-09-20T18:44:04.915 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph-authtool: stdout AQB0wu1mNB2OERAAFOesosDE6nJJ9mXQoNkpkQ== 2024-09-20T18:44:05.829 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph-authtool: stdout AQB1wu1mCkhsDRAAduhNotJ0NjTGw5zaStMvHA== 2024-09-20T18:44:06.677 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph-authtool: stdout AQB2wu1m2TJDCBAAaC3mejbIT1LWakH+xpHNBQ== 2024-09-20T18:44:06.677 INFO:teuthology.orchestra.run.smithi083.stdout:Creating initial monmap... 2024-09-20T18:44:07.584 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-20T18:44:07.584 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-20T18:44:07.584 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 1ed5a50a-7780-11ef-baf5-efdc52797490 2024-09-20T18:44:07.584 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-20T18:44:07.584 INFO:teuthology.orchestra.run.smithi083.stdout:monmaptool for a [v2:172.21.15.83:3300,v1:172.21.15.83:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-20T18:44:07.584 INFO:teuthology.orchestra.run.smithi083.stdout:setting min_mon_release = quincy 2024-09-20T18:44:07.585 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/monmaptool: set fsid to 1ed5a50a-7780-11ef-baf5-efdc52797490 2024-09-20T18:44:07.585 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-20T18:44:07.585 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:44:07.585 INFO:teuthology.orchestra.run.smithi083.stdout:Creating mon... 2024-09-20T18:44:08.525 INFO:teuthology.orchestra.run.smithi083.stdout:create mon.a on 2024-09-20T18:44:08.747 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-20T18:44:08.887 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-20T18:44:09.139 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-1ed5a50a-7780-11ef-baf5-efdc52797490.target → /etc/systemd/system/ceph-1ed5a50a-7780-11ef-baf5-efdc52797490.target. 2024-09-20T18:44:09.139 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-1ed5a50a-7780-11ef-baf5-efdc52797490.target → /etc/systemd/system/ceph-1ed5a50a-7780-11ef-baf5-efdc52797490.target. 2024-09-20T18:44:09.752 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@mon.a 2024-09-20T18:44:09.752 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Failed to reset failed state of unit ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@mon.a.service: Unit ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@mon.a.service not loaded. 2024-09-20T18:44:09.910 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-1ed5a50a-7780-11ef-baf5-efdc52797490.target.wants/ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@mon.a.service → /etc/systemd/system/ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@.service. 2024-09-20T18:44:10.417 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:10 smithi083 podman[29034]: 2024-09-20 18:44:10.362473022 +0000 UTC m=+0.357719826 container start c7cd47746729b7f7218a64d0a9205ad2ebec24a005c67c71d227c0349013d923 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mon-a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2) 2024-09-20T18:44:10.473 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-20T18:44:10.473 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout disabled 2024-09-20T18:44:10.480 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-20T18:44:10.480 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout inactive 2024-09-20T18:44:10.480 INFO:teuthology.orchestra.run.smithi083.stdout:firewalld.service is not enabled 2024-09-20T18:44:10.480 INFO:teuthology.orchestra.run.smithi083.stdout:Not possible to enable service . firewalld.service is not available 2024-09-20T18:44:10.481 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for mon to start... 2024-09-20T18:44:10.481 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for mon... 2024-09-20T18:44:10.703 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:10 smithi083 ceph-mon[29053]: mkfs 1ed5a50a-7780-11ef-baf5-efdc52797490 2024-09-20T18:44:10.703 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:10 smithi083 ceph-mon[29053]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-20T18:44:10.703 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:10 smithi083 bash[29034]: c7cd47746729b7f7218a64d0a9205ad2ebec24a005c67c71d227c0349013d923 2024-09-20T18:44:10.703 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:10 smithi083 systemd[1]: Started Ceph mon.a for 1ed5a50a-7780-11ef-baf5-efdc52797490. 2024-09-20T18:44:11.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:11 smithi083 ceph-mon[29053]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-20T18:44:11.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:11 smithi083 ceph-mon[29053]: monmap epoch 1 2024-09-20T18:44:11.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:11 smithi083 ceph-mon[29053]: fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 2024-09-20T18:44:11.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:11 smithi083 ceph-mon[29053]: last_changed 2024-09-20T18:44:06.997446+0000 2024-09-20T18:44:11.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:11 smithi083 ceph-mon[29053]: created 2024-09-20T18:44:06.997446+0000 2024-09-20T18:44:11.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:11 smithi083 ceph-mon[29053]: min_mon_release 19 (squid) 2024-09-20T18:44:11.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:11 smithi083 ceph-mon[29053]: election_strategy: 1 2024-09-20T18:44:11.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:11 smithi083 ceph-mon[29053]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-09-20T18:44:11.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:11 smithi083 ceph-mon[29053]: fsmap 2024-09-20T18:44:11.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:11 smithi083 ceph-mon[29053]: osdmap e1: 0 total, 0 up, 0 in 2024-09-20T18:44:11.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:11 smithi083 ceph-mon[29053]: mgrmap e1: no daemons active 2024-09-20T18:44:11.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:11 smithi083 ceph-mon[29053]: from='client.? 172.21.15.83:0/2632411988' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-09-20T18:44:11.586 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout cluster: 2024-09-20T18:44:11.586 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout id: 1ed5a50a-7780-11ef-baf5-efdc52797490 2024-09-20T18:44:11.586 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-20T18:44:11.586 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-09-20T18:44:11.586 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout services: 2024-09-20T18:44:11.587 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.54161s) 2024-09-20T18:44:11.587 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-20T18:44:11.587 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-20T18:44:11.587 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-09-20T18:44:11.587 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout data: 2024-09-20T18:44:11.587 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-20T18:44:11.587 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-20T18:44:11.587 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-20T18:44:11.587 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout pgs: 2024-09-20T18:44:11.587 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-09-20T18:44:11.587 INFO:teuthology.orchestra.run.smithi083.stdout:mon is available 2024-09-20T18:44:11.587 INFO:teuthology.orchestra.run.smithi083.stdout:Assimilating anything we can from ceph.conf... 2024-09-20T18:44:12.606 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:12 smithi083 ceph-mon[29053]: from='client.? 172.21.15.83:0/2224716183' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-20T18:44:12.606 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:12 smithi083 ceph-mon[29053]: from='client.? 172.21.15.83:0/2224716183' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-20T18:44:12.609 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-09-20T18:44:12.609 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout [global] 2024-09-20T18:44:12.609 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout fsid = 1ed5a50a-7780-11ef-baf5-efdc52797490 2024-09-20T18:44:12.610 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-20T18:44:12.610 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.83:3300,v1:172.21.15.83:6789] 2024-09-20T18:44:12.610 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-20T18:44:12.610 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-20T18:44:12.610 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-20T18:44:12.610 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-20T18:44:12.610 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-09-20T18:44:12.610 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-20T18:44:12.610 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-20T18:44:12.610 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-09-20T18:44:12.610 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout [osd] 2024-09-20T18:44:12.610 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-20T18:44:12.610 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-20T18:44:12.611 INFO:teuthology.orchestra.run.smithi083.stdout:Generating new minimal ceph.conf... 2024-09-20T18:44:13.632 INFO:teuthology.orchestra.run.smithi083.stdout:Restarting the monitor... 2024-09-20T18:44:13.651 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:13 smithi083 ceph-mon[29053]: from='client.? 172.21.15.83:0/4090193457' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:44:14.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:13 smithi083 systemd[1]: Stopping Ceph mon.a for 1ed5a50a-7780-11ef-baf5-efdc52797490... 2024-09-20T18:44:14.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:13 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mon-a[29049]: 2024-09-20T18:44:13.875+0000 7fbf264b2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-20T18:44:14.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:13 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mon-a[29049]: 2024-09-20T18:44:13.875+0000 7fbf264b2640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-20T18:44:14.366 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:14 smithi083 podman[29298]: 2024-09-20 18:44:14.089814372 +0000 UTC m=+0.307184233 container died c7cd47746729b7f7218a64d0a9205ad2ebec24a005c67c71d227c0349013d923 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-20T18:44:14.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:14 smithi083 podman[29298]: 2024-09-20 18:44:14.458480039 +0000 UTC m=+0.675849901 container cleanup c7cd47746729b7f7218a64d0a9205ad2ebec24a005c67c71d227c0349013d923 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mon-a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-20T18:44:14.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:14 smithi083 bash[29298]: ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mon-a 2024-09-20T18:44:15.255 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:14 smithi083 podman[29310]: 2024-09-20 18:44:14.87614569 +0000 UTC m=+0.782688841 container remove c7cd47746729b7f7218a64d0a9205ad2ebec24a005c67c71d227c0349013d923 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-20T18:44:15.255 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 systemd[1]: ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@mon.a.service: Deactivated successfully. 2024-09-20T18:44:15.255 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 systemd[1]: Stopped Ceph mon.a for 1ed5a50a-7780-11ef-baf5-efdc52797490. 2024-09-20T18:44:15.255 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 systemd[1]: Starting Ceph mon.a for 1ed5a50a-7780-11ef-baf5-efdc52797490... 2024-09-20T18:44:15.537 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 podman[29402]: 2024-09-20 18:44:15.141674587 +0000 UTC m=+0.015609333 image pull 441f9a3fe51528ef653b21f42e9d884db6124a0c7d482e865382bdfdb9ec979c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 2024-09-20T18:44:15.537 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 podman[29402]: 2024-09-20 18:44:15.243611968 +0000 UTC m=+0.117546703 container create 16f5b979aa6993ebd1c12c4cfb286a1b6ee3b50a802cdb718cb26d9d680ebcd8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mon-a, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True) 2024-09-20T18:44:15.537 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 podman[29402]: 2024-09-20 18:44:15.446156754 +0000 UTC m=+0.320091494 container init 16f5b979aa6993ebd1c12c4cfb286a1b6ee3b50a802cdb718cb26d9d680ebcd8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True) 2024-09-20T18:44:15.537 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 podman[29402]: 2024-09-20 18:44:15.449341356 +0000 UTC m=+0.323276092 container start 16f5b979aa6993ebd1c12c4cfb286a1b6ee3b50a802cdb718cb26d9d680ebcd8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mon-a, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-20-1028-88deadb, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-20T18:44:15.537 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: set uid:gid to 167:167 (ceph:ceph) 2024-09-20T18:44:15.537 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: ceph version 19.3.0-5099-g88deadb8 (88deadb8082f4a33832e228ccefc59cedba158a5) squid (dev), process ceph-mon, pid 2 2024-09-20T18:44:15.537 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: pidfile_write: ignore empty --pid-file 2024-09-20T18:44:15.537 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: load: jerasure load: lrc 2024-09-20T18:44:15.537 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: RocksDB version: 7.9.2 2024-09-20T18:44:15.537 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Git sha 0 2024-09-20T18:44:15.538 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Compile date 2024-09-20 10:48:39 2024-09-20T18:44:15.538 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: DB SUMMARY 2024-09-20T18:44:15.538 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: DB Session ID: QA9KY682OOI7UASPWCS7 2024-09-20T18:44:15.538 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: CURRENT file: CURRENT 2024-09-20T18:44:15.538 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: IDENTITY file: IDENTITY 2024-09-20T18:44:15.538 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-20T18:44:15.538 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-20T18:44:15.538 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 95892 ; 2024-09-20T18:44:15.538 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.error_if_exists: 0 2024-09-20T18:44:15.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.create_if_missing: 0 2024-09-20T18:44:15.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.paranoid_checks: 1 2024-09-20T18:44:15.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-20T18:44:15.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-20T18:44:15.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-20T18:44:15.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.env: 0x55fe2c9f2120 2024-09-20T18:44:15.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.fs: PosixFileSystem 2024-09-20T18:44:15.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.info_log: 0x55fe2ed3f700 2024-09-20T18:44:15.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.max_file_opening_threads: 16 2024-09-20T18:44:15.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.statistics: (nil) 2024-09-20T18:44:15.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.use_fsync: 0 2024-09-20T18:44:15.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.max_log_file_size: 0 2024-09-20T18:44:15.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-20T18:44:15.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-20T18:44:15.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.keep_log_file_num: 1000 2024-09-20T18:44:15.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.recycle_log_file_num: 0 2024-09-20T18:44:15.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.allow_fallocate: 1 2024-09-20T18:44:15.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.allow_mmap_reads: 0 2024-09-20T18:44:15.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.allow_mmap_writes: 0 2024-09-20T18:44:15.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.use_direct_reads: 0 2024-09-20T18:44:15.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-20T18:44:15.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.create_missing_column_families: 0 2024-09-20T18:44:15.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.db_log_dir: 2024-09-20T18:44:15.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.wal_dir: 2024-09-20T18:44:15.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-20T18:44:15.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-20T18:44:15.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-20T18:44:15.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-20T18:44:15.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-20T18:44:15.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-20T18:44:15.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.advise_random_on_open: 1 2024-09-20T18:44:15.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.db_write_buffer_size: 0 2024-09-20T18:44:15.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.write_buffer_manager: 0x55fe2ed43b80 2024-09-20T18:44:15.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-20T18:44:15.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-20T18:44:15.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-20T18:44:15.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.rate_limiter: (nil) 2024-09-20T18:44:15.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-20T18:44:15.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.wal_recovery_mode: 2 2024-09-20T18:44:15.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.enable_thread_tracking: 0 2024-09-20T18:44:15.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.enable_pipelined_write: 0 2024-09-20T18:44:15.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.unordered_write: 0 2024-09-20T18:44:15.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-20T18:44:15.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-20T18:44:15.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-20T18:44:15.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-20T18:44:15.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.row_cache: None 2024-09-20T18:44:15.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.wal_filter: None 2024-09-20T18:44:15.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-20T18:44:15.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.allow_ingest_behind: 0 2024-09-20T18:44:15.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.two_write_queues: 0 2024-09-20T18:44:15.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.manual_wal_flush: 0 2024-09-20T18:44:15.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.wal_compression: 0 2024-09-20T18:44:15.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.atomic_flush: 0 2024-09-20T18:44:15.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-20T18:44:15.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-20T18:44:15.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-20T18:44:15.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.log_readahead_size: 0 2024-09-20T18:44:15.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-20T18:44:15.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.best_efforts_recovery: 0 2024-09-20T18:44:15.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-20T18:44:15.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-20T18:44:15.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.allow_data_in_errors: 0 2024-09-20T18:44:15.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.db_host_id: __hostname__ 2024-09-20T18:44:15.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-20T18:44:15.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.max_background_jobs: 2 2024-09-20T18:44:15.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.max_background_compactions: -1 2024-09-20T18:44:15.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.max_subcompactions: 1 2024-09-20T18:44:15.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-20T18:44:15.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-20T18:44:15.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-20T18:44:15.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.max_total_wal_size: 0 2024-09-20T18:44:15.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-20T18:44:15.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-20T18:44:15.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-20T18:44:15.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-20T18:44:15.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.max_open_files: -1 2024-09-20T18:44:15.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.bytes_per_sync: 0 2024-09-20T18:44:15.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-20T18:44:15.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-20T18:44:15.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.compaction_readahead_size: 0 2024-09-20T18:44:15.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.max_background_flushes: -1 2024-09-20T18:44:15.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Compression algorithms supported: 2024-09-20T18:44:15.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: kZSTD supported: 0 2024-09-20T18:44:15.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: kXpressCompression supported: 0 2024-09-20T18:44:15.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: kBZip2Compression supported: 0 2024-09-20T18:44:15.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-20T18:44:15.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: kLZ4Compression supported: 1 2024-09-20T18:44:15.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: kZlibCompression supported: 1 2024-09-20T18:44:15.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: kLZ4HCCompression supported: 1 2024-09-20T18:44:15.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: kSnappyCompression supported: 1 2024-09-20T18:44:15.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-20T18:44:15.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-20T18:44:15.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-20T18:44:15.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-20T18:44:15.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-20T18:44:15.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.merge_operator: 2024-09-20T18:44:15.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.compaction_filter: None 2024-09-20T18:44:15.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.compaction_filter_factory: None 2024-09-20T18:44:15.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.sst_partitioner_factory: None 2024-09-20T18:44:15.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-20T18:44:15.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-20T18:44:15.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55fe2ed3f320) 2024-09-20T18:44:15.546 INFO:journalctl@ceph.mon.a.smithi083.stdout: cache_index_and_filter_blocks: 1 2024-09-20T18:44:15.547 INFO:journalctl@ceph.mon.a.smithi083.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-20T18:44:15.547 INFO:journalctl@ceph.mon.a.smithi083.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-20T18:44:15.547 INFO:journalctl@ceph.mon.a.smithi083.stdout: pin_top_level_index_and_filter: 1 2024-09-20T18:44:15.547 INFO:journalctl@ceph.mon.a.smithi083.stdout: index_type: 0 2024-09-20T18:44:15.547 INFO:journalctl@ceph.mon.a.smithi083.stdout: data_block_index_type: 0 2024-09-20T18:44:15.547 INFO:journalctl@ceph.mon.a.smithi083.stdout: index_shortening: 1 2024-09-20T18:44:15.547 INFO:journalctl@ceph.mon.a.smithi083.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-20T18:44:15.547 INFO:journalctl@ceph.mon.a.smithi083.stdout: checksum: 4 2024-09-20T18:44:15.547 INFO:journalctl@ceph.mon.a.smithi083.stdout: no_block_cache: 0 2024-09-20T18:44:15.547 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_cache: 0x55fe2ed33610 2024-09-20T18:44:15.547 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_cache_name: BinnedLRUCache 2024-09-20T18:44:15.547 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_cache_options: 2024-09-20T18:44:15.547 INFO:journalctl@ceph.mon.a.smithi083.stdout: capacity : 536870912 2024-09-20T18:44:15.547 INFO:journalctl@ceph.mon.a.smithi083.stdout: num_shard_bits : 4 2024-09-20T18:44:15.548 INFO:journalctl@ceph.mon.a.smithi083.stdout: strict_capacity_limit : 0 2024-09-20T18:44:15.548 INFO:journalctl@ceph.mon.a.smithi083.stdout: high_pri_pool_ratio: 0.000 2024-09-20T18:44:15.548 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_cache_compressed: (nil) 2024-09-20T18:44:15.548 INFO:journalctl@ceph.mon.a.smithi083.stdout: persistent_cache: (nil) 2024-09-20T18:44:15.548 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_size: 4096 2024-09-20T18:44:15.548 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_size_deviation: 10 2024-09-20T18:44:15.548 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_restart_interval: 16 2024-09-20T18:44:15.549 INFO:journalctl@ceph.mon.a.smithi083.stdout: index_block_restart_interval: 1 2024-09-20T18:44:15.549 INFO:journalctl@ceph.mon.a.smithi083.stdout: metadata_block_size: 4096 2024-09-20T18:44:15.549 INFO:journalctl@ceph.mon.a.smithi083.stdout: partition_filters: 0 2024-09-20T18:44:15.549 INFO:journalctl@ceph.mon.a.smithi083.stdout: use_delta_encoding: 1 2024-09-20T18:44:15.549 INFO:journalctl@ceph.mon.a.smithi083.stdout: filter_policy: bloomfilter 2024-09-20T18:44:15.549 INFO:journalctl@ceph.mon.a.smithi083.stdout: whole_key_filtering: 1 2024-09-20T18:44:15.549 INFO:journalctl@ceph.mon.a.smithi083.stdout: verify_compression: 0 2024-09-20T18:44:15.549 INFO:journalctl@ceph.mon.a.smithi083.stdout: read_amp_bytes_per_bit: 0 2024-09-20T18:44:15.549 INFO:journalctl@ceph.mon.a.smithi083.stdout: format_version: 5 2024-09-20T18:44:15.549 INFO:journalctl@ceph.mon.a.smithi083.stdout: enable_index_compression: 1 2024-09-20T18:44:15.549 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_align: 0 2024-09-20T18:44:15.550 INFO:journalctl@ceph.mon.a.smithi083.stdout: max_auto_readahead_size: 262144 2024-09-20T18:44:15.550 INFO:journalctl@ceph.mon.a.smithi083.stdout: prepopulate_block_cache: 0 2024-09-20T18:44:15.550 INFO:journalctl@ceph.mon.a.smithi083.stdout: initial_auto_readahead_size: 8192 2024-09-20T18:44:15.550 INFO:journalctl@ceph.mon.a.smithi083.stdout: num_file_reads_for_auto_readahead: 2 2024-09-20T18:44:15.550 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.write_buffer_size: 33554432 2024-09-20T18:44:15.550 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.max_write_buffer_number: 2 2024-09-20T18:44:15.550 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.compression: NoCompression 2024-09-20T18:44:15.550 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.bottommost_compression: Disabled 2024-09-20T18:44:15.550 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.prefix_extractor: nullptr 2024-09-20T18:44:15.550 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-20T18:44:15.550 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.num_levels: 7 2024-09-20T18:44:15.550 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-20T18:44:15.550 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-20T18:44:15.550 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-20T18:44:15.550 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-20T18:44:15.551 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-20T18:44:15.551 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-20T18:44:15.551 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-20T18:44:15.551 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-20T18:44:15.551 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-20T18:44:15.551 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-20T18:44:15.551 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-20T18:44:15.551 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-20T18:44:15.551 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-20T18:44:15.551 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.compression_opts.level: 32767 2024-09-20T18:44:15.551 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.compression_opts.strategy: 0 2024-09-20T18:44:15.551 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-20T18:44:15.551 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-20T18:44:15.551 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-20T18:44:15.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-20T18:44:15.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.compression_opts.enabled: false 2024-09-20T18:44:15.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-20T18:44:15.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-20T18:44:15.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-20T18:44:15.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-20T18:44:15.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.target_file_size_base: 67108864 2024-09-20T18:44:15.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-20T18:44:15.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-20T18:44:15.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-20T18:44:15.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-20T18:44:15.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-20T18:44:15.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-20T18:44:15.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-20T18:44:15.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-20T18:44:15.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-20T18:44:15.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-20T18:44:15.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-20T18:44:15.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-20T18:44:15.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-20T18:44:15.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-20T18:44:15.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.arena_block_size: 1048576 2024-09-20T18:44:15.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-20T18:44:15.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-20T18:44:15.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.disable_auto_compactions: 0 2024-09-20T18:44:15.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-20T18:44:15.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-20T18:44:15.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-20T18:44:15.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-20T18:44:15.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-20T18:44:15.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-20T18:44:15.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-20T18:44:15.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-20T18:44:15.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-20T18:44:15.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-20T18:44:15.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-20T18:44:15.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.inplace_update_support: 0 2024-09-20T18:44:15.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-20T18:44:15.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-20T18:44:15.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-20T18:44:15.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-20T18:44:15.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.bloom_locality: 0 2024-09-20T18:44:15.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.max_successive_merges: 0 2024-09-20T18:44:15.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-20T18:44:15.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.paranoid_file_checks: 0 2024-09-20T18:44:15.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.force_consistency_checks: 1 2024-09-20T18:44:15.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.report_bg_io_stats: 0 2024-09-20T18:44:15.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.ttl: 2592000 2024-09-20T18:44:15.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-20T18:44:15.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-20T18:44:15.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-20T18:44:15.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.enable_blob_files: false 2024-09-20T18:44:15.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.min_blob_size: 0 2024-09-20T18:44:15.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.blob_file_size: 268435456 2024-09-20T18:44:15.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-20T18:44:15.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-20T18:44:15.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-20T18:44:15.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-20T18:44:15.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-20T18:44:15.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.blob_file_starting_level: 0 2024-09-20T18:44:15.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-20T18:44:15.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-20T18:44:15.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-20T18:44:15.556 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 50d915dd-c5d3-4a98-affa-2168c3e02234 2024-09-20T18:44:15.556 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726857855475184, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-20T18:44:15.556 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-20T18:44:15.556 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726857855476134, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 91851, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 289, "table_properties": {"data_size": 89899, "index_size": 272, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11920, "raw_average_key_size": 48, "raw_value_size": 83299, "raw_average_value_size": 335, "num_data_blocks": 12, "num_entries": 248, "num_filter_entries": 248, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726857855, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "50d915dd-c5d3-4a98-affa-2168c3e02234", "db_session_id": "QA9KY682OOI7UASPWCS7", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-20T18:44:15.556 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726857855476227, "job": 1, "event": "recovery_finished"} 2024-09-20T18:44:15.556 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-20T18:44:15.556 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-20T18:44:15.556 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55fe2ed62e00 2024-09-20T18:44:15.556 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: DB pointer 0x55fe2ee64000 2024-09-20T18:44:15.556 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-20T18:44:15.556 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-20T18:44:15.556 INFO:journalctl@ceph.mon.a.smithi083.stdout: ** DB Stats ** 2024-09-20T18:44:15.556 INFO:journalctl@ceph.mon.a.smithi083.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-20T18:44:15.556 INFO:journalctl@ceph.mon.a.smithi083.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-20T18:44:15.556 INFO:journalctl@ceph.mon.a.smithi083.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-20T18:44:15.557 INFO:journalctl@ceph.mon.a.smithi083.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-20T18:44:15.557 INFO:journalctl@ceph.mon.a.smithi083.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-20T18:44:15.557 INFO:journalctl@ceph.mon.a.smithi083.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-20T18:44:15.557 INFO:journalctl@ceph.mon.a.smithi083.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-20T18:44:15.557 INFO:journalctl@ceph.mon.a.smithi083.stdout: 2024-09-20T18:44:15.557 INFO:journalctl@ceph.mon.a.smithi083.stdout: ** Compaction Stats [default] ** 2024-09-20T18:44:15.557 INFO:journalctl@ceph.mon.a.smithi083.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-20T18:44:15.557 INFO:journalctl@ceph.mon.a.smithi083.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-20T18:44:15.557 INFO:journalctl@ceph.mon.a.smithi083.stdout: L0 2/0 91.56 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 137.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-20T18:44:15.557 INFO:journalctl@ceph.mon.a.smithi083.stdout: Sum 2/0 91.56 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 137.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-20T18:44:15.557 INFO:journalctl@ceph.mon.a.smithi083.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 137.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-20T18:44:15.557 INFO:journalctl@ceph.mon.a.smithi083.stdout: 2024-09-20T18:44:15.557 INFO:journalctl@ceph.mon.a.smithi083.stdout: ** Compaction Stats [default] ** 2024-09-20T18:44:15.557 INFO:journalctl@ceph.mon.a.smithi083.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-20T18:44:15.557 INFO:journalctl@ceph.mon.a.smithi083.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-20T18:44:15.557 INFO:journalctl@ceph.mon.a.smithi083.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 137.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-20T18:44:15.558 INFO:journalctl@ceph.mon.a.smithi083.stdout: 2024-09-20T18:44:15.558 INFO:journalctl@ceph.mon.a.smithi083.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-20T18:44:15.558 INFO:journalctl@ceph.mon.a.smithi083.stdout: 2024-09-20T18:44:15.558 INFO:journalctl@ceph.mon.a.smithi083.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-20T18:44:15.558 INFO:journalctl@ceph.mon.a.smithi083.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-20T18:44:15.558 INFO:journalctl@ceph.mon.a.smithi083.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-20T18:44:15.558 INFO:journalctl@ceph.mon.a.smithi083.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-20T18:44:15.558 INFO:journalctl@ceph.mon.a.smithi083.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-20T18:44:15.558 INFO:journalctl@ceph.mon.a.smithi083.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-20T18:44:15.558 INFO:journalctl@ceph.mon.a.smithi083.stdout: Cumulative compaction: 0.00 GB write, 28.54 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-20T18:44:15.558 INFO:journalctl@ceph.mon.a.smithi083.stdout: Interval compaction: 0.00 GB write, 28.54 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-20T18:44:15.558 INFO:journalctl@ceph.mon.a.smithi083.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-20T18:44:15.558 INFO:journalctl@ceph.mon.a.smithi083.stdout: Block cache BinnedLRUCache@0x55fe2ed33610#2 capacity: 512.00 MB usage: 1.30 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-20T18:44:15.558 INFO:journalctl@ceph.mon.a.smithi083.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.83 KB,0.000157952%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-09-20T18:44:15.559 INFO:journalctl@ceph.mon.a.smithi083.stdout: 2024-09-20T18:44:15.559 INFO:journalctl@ceph.mon.a.smithi083.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-20T18:44:15.559 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: starting mon.a rank 0 at public addrs [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] at bind addrs [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 2024-09-20T18:44:15.559 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: mon.a@-1(???) e1 preinit fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 2024-09-20T18:44:15.559 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: mon.a@-1(???).mds e1 new map 2024-09-20T18:44:15.559 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: mon.a@-1(???).mds e1 print_map 2024-09-20T18:44:15.559 INFO:journalctl@ceph.mon.a.smithi083.stdout: e1 2024-09-20T18:44:15.559 INFO:journalctl@ceph.mon.a.smithi083.stdout: btime 2024-09-20T18:44:10:399616+0000 2024-09-20T18:44:15.559 INFO:journalctl@ceph.mon.a.smithi083.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-20T18:44:15.559 INFO:journalctl@ceph.mon.a.smithi083.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-20T18:44:15.559 INFO:journalctl@ceph.mon.a.smithi083.stdout: legacy client fscid: -1 2024-09-20T18:44:15.559 INFO:journalctl@ceph.mon.a.smithi083.stdout: 2024-09-20T18:44:15.559 INFO:journalctl@ceph.mon.a.smithi083.stdout: No filesystems configured 2024-09-20T18:44:15.559 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-20T18:44:15.559 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-20T18:44:15.559 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-20T18:44:15.560 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-20T18:44:15.560 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-20T18:44:15.560 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 bash[29402]: 16f5b979aa6993ebd1c12c4cfb286a1b6ee3b50a802cdb718cb26d9d680ebcd8 2024-09-20T18:44:15.560 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 systemd[1]: Started Ceph mon.a for 1ed5a50a-7780-11ef-baf5-efdc52797490. 2024-09-20T18:44:15.560 INFO:teuthology.orchestra.run.smithi083.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-20T18:44:15.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-20T18:44:15.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: monmap epoch 1 2024-09-20T18:44:15.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 2024-09-20T18:44:15.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: last_changed 2024-09-20T18:44:06.997446+0000 2024-09-20T18:44:15.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: created 2024-09-20T18:44:06.997446+0000 2024-09-20T18:44:15.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: min_mon_release 19 (squid) 2024-09-20T18:44:15.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: election_strategy: 1 2024-09-20T18:44:15.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-09-20T18:44:15.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: fsmap 2024-09-20T18:44:15.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: osdmap e1: 0 total, 0 up, 0 in 2024-09-20T18:44:15.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:15 smithi083 ceph-mon[29421]: mgrmap e1: no daemons active 2024-09-20T18:44:16.622 INFO:teuthology.orchestra.run.smithi083.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-20T18:44:16.643 INFO:teuthology.orchestra.run.smithi083.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-20T18:44:16.644 INFO:teuthology.orchestra.run.smithi083.stdout:Creating mgr... 2024-09-20T18:44:16.644 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-20T18:44:16.644 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-20T18:44:16.885 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@mgr.y 2024-09-20T18:44:16.885 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Failed to reset failed state of unit ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@mgr.y.service: Unit ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@mgr.y.service not loaded. 2024-09-20T18:44:17.027 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-1ed5a50a-7780-11ef-baf5-efdc52797490.target.wants/ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@mgr.y.service → /etc/systemd/system/ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@.service. 2024-09-20T18:44:17.246 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:17 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3484295819' entity='client.admin' 2024-09-20T18:44:17.499 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:17 smithi083 podman[29631]: 2024-09-20 18:44:17.411510379 +0000 UTC m=+0.278279284 container start 9a996b91b375bfbd641ea6f030c760cb8c69dfb83105193f6093263cfe478f3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, io.buildah.version=1.37.2) 2024-09-20T18:44:17.499 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:17 smithi083 bash[29631]: 9a996b91b375bfbd641ea6f030c760cb8c69dfb83105193f6093263cfe478f3b 2024-09-20T18:44:17.512 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-20T18:44:17.513 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout disabled 2024-09-20T18:44:17.520 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-20T18:44:17.520 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout inactive 2024-09-20T18:44:17.520 INFO:teuthology.orchestra.run.smithi083.stdout:firewalld.service is not enabled 2024-09-20T18:44:17.520 INFO:teuthology.orchestra.run.smithi083.stdout:Not possible to enable service . firewalld.service is not available 2024-09-20T18:44:17.529 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-20T18:44:17.529 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout disabled 2024-09-20T18:44:17.536 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-20T18:44:17.536 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout inactive 2024-09-20T18:44:17.536 INFO:teuthology.orchestra.run.smithi083.stdout:firewalld.service is not enabled 2024-09-20T18:44:17.537 INFO:teuthology.orchestra.run.smithi083.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-20T18:44:17.537 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for mgr to start... 2024-09-20T18:44:17.537 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for mgr... 2024-09-20T18:44:17.775 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:17 smithi083 systemd[1]: Started Ceph mgr.y for 1ed5a50a-7780-11ef-baf5-efdc52797490. 2024-09-20T18:44:18.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:18 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3075822964' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-20T18:44:18.492 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-09-20T18:44:18.495 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-09-20T18:44:18.495 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "fsid": "1ed5a50a-7780-11ef-baf5-efdc52797490", 2024-09-20T18:44:18.495 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "health": { 2024-09-20T18:44:18.495 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-20T18:44:18.495 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-20T18:44:18.495 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-20T18:44:18.495 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-09-20T18:44:18.495 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-20T18:44:18.496 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-20T18:44:18.496 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 0 2024-09-20T18:44:18.496 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-09-20T18:44:18.496 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-20T18:44:18.496 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "a" 2024-09-20T18:44:18.496 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-09-20T18:44:18.496 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-20T18:44:18.496 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-20T18:44:18.496 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-20T18:44:18.496 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-20T18:44:18.496 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-20T18:44:18.496 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-09-20T18:44:18.496 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-20T18:44:18.496 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-20T18:44:18.497 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-20T18:44:18.497 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-20T18:44:18.497 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-20T18:44:18.497 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-20T18:44:18.497 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-20T18:44:18.497 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-20T18:44:18.500 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-09-20T18:44:18.500 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-20T18:44:18.500 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-20T18:44:18.500 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-20T18:44:18.500 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-20T18:44:18.500 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-20T18:44:18.500 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-20T18:44:18.500 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-20T18:44:18.501 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-20T18:44:18.501 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-20T18:44:18.501 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-09-20T18:44:18.501 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-20T18:44:18.501 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-20T18:44:18.501 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "btime": "2024-09-20T18:44:10:399616+0000", 2024-09-20T18:44:18.501 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-20T18:44:18.501 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-20T18:44:18.501 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-09-20T18:44:18.501 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-20T18:44:18.501 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-20T18:44:18.501 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-20T18:44:18.501 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-20T18:44:18.501 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-20T18:44:18.502 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-20T18:44:18.502 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "restful" 2024-09-20T18:44:18.502 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-09-20T18:44:18.502 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-20T18:44:18.502 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-09-20T18:44:18.502 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-20T18:44:18.502 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-20T18:44:18.502 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "modified": "2024-09-20T18:44:10.400266+0000", 2024-09-20T18:44:18.502 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-20T18:44:18.502 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-09-20T18:44:18.502 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-20T18:44:18.502 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-09-20T18:44:18.502 INFO:teuthology.orchestra.run.smithi083.stdout:mgr not available, waiting (1/15)... 2024-09-20T18:44:21.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:21 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1067287428' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-20T18:44:21.643 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-09-20T18:44:21.643 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-09-20T18:44:21.643 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "fsid": "1ed5a50a-7780-11ef-baf5-efdc52797490", 2024-09-20T18:44:21.643 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "health": { 2024-09-20T18:44:21.643 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-20T18:44:21.643 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-20T18:44:21.643 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-20T18:44:21.643 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-09-20T18:44:21.643 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-20T18:44:21.643 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-20T18:44:21.644 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 0 2024-09-20T18:44:21.644 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-09-20T18:44:21.644 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-20T18:44:21.644 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "a" 2024-09-20T18:44:21.644 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-09-20T18:44:21.644 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-20T18:44:21.644 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-20T18:44:21.644 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-20T18:44:21.644 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-20T18:44:21.644 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-20T18:44:21.644 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-09-20T18:44:21.644 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-20T18:44:21.644 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-20T18:44:21.644 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-20T18:44:21.644 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-20T18:44:21.645 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-20T18:44:21.645 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-20T18:44:21.645 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-20T18:44:21.645 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-20T18:44:21.648 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-09-20T18:44:21.648 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-20T18:44:21.648 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-20T18:44:21.648 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-20T18:44:21.648 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-20T18:44:21.648 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-20T18:44:21.648 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-20T18:44:21.648 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-20T18:44:21.649 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-20T18:44:21.649 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-20T18:44:21.649 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-09-20T18:44:21.649 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-20T18:44:21.649 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-20T18:44:21.649 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "btime": "2024-09-20T18:44:10:399616+0000", 2024-09-20T18:44:21.649 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-20T18:44:21.649 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-20T18:44:21.649 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-09-20T18:44:21.649 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-20T18:44:21.649 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-20T18:44:21.649 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-20T18:44:21.649 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-20T18:44:21.649 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-20T18:44:21.650 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-20T18:44:21.650 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "restful" 2024-09-20T18:44:21.650 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-09-20T18:44:21.650 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-20T18:44:21.650 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-09-20T18:44:21.650 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-20T18:44:21.650 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-20T18:44:21.650 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "modified": "2024-09-20T18:44:10.400266+0000", 2024-09-20T18:44:21.650 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-20T18:44:21.650 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-09-20T18:44:21.650 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-20T18:44:21.650 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-09-20T18:44:21.650 INFO:teuthology.orchestra.run.smithi083.stdout:mgr not available, waiting (2/15)... 2024-09-20T18:44:22.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:22 smithi083 ceph-mon[29421]: Activating manager daemon y 2024-09-20T18:44:22.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:22 smithi083 ceph-mon[29421]: mgrmap e2: y(active, starting, since 0.00347712s) 2024-09-20T18:44:22.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:22 smithi083 ceph-mon[29421]: from='mgr.14100 172.21.15.83:0/2222614546' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-20T18:44:22.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:22 smithi083 ceph-mon[29421]: from='mgr.14100 172.21.15.83:0/2222614546' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-20T18:44:22.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:22 smithi083 ceph-mon[29421]: from='mgr.14100 172.21.15.83:0/2222614546' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-20T18:44:22.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:22 smithi083 ceph-mon[29421]: from='mgr.14100 172.21.15.83:0/2222614546' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-20T18:44:22.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:22 smithi083 ceph-mon[29421]: from='mgr.14100 172.21.15.83:0/2222614546' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-20T18:44:22.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:22 smithi083 ceph-mon[29421]: Manager daemon y is now available 2024-09-20T18:44:22.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:22 smithi083 ceph-mon[29421]: from='mgr.14100 172.21.15.83:0/2222614546' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-20T18:44:22.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:22 smithi083 ceph-mon[29421]: from='mgr.14100 172.21.15.83:0/2222614546' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-20T18:44:22.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:22 smithi083 ceph-mon[29421]: from='mgr.14100 172.21.15.83:0/2222614546' entity='mgr.y' 2024-09-20T18:44:22.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:22 smithi083 ceph-mon[29421]: from='mgr.14100 172.21.15.83:0/2222614546' entity='mgr.y' 2024-09-20T18:44:22.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:22 smithi083 ceph-mon[29421]: from='mgr.14100 172.21.15.83:0/2222614546' entity='mgr.y' 2024-09-20T18:44:24.783 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:24 smithi083 ceph-mon[29421]: mgrmap e3: y(active, since 1.00718s) 2024-09-20T18:44:24.783 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:24 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/466927244' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-20T18:44:24.910 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-09-20T18:44:24.911 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-09-20T18:44:24.911 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "fsid": "1ed5a50a-7780-11ef-baf5-efdc52797490", 2024-09-20T18:44:24.911 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "health": { 2024-09-20T18:44:24.911 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-20T18:44:24.911 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-20T18:44:24.911 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-20T18:44:24.911 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-09-20T18:44:24.911 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-20T18:44:24.911 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-20T18:44:24.911 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 0 2024-09-20T18:44:24.911 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-09-20T18:44:24.912 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-20T18:44:24.912 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "a" 2024-09-20T18:44:24.912 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-09-20T18:44:24.912 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-09-20T18:44:24.912 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-20T18:44:24.912 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-20T18:44:24.912 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-20T18:44:24.912 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-20T18:44:24.912 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-09-20T18:44:24.912 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-20T18:44:24.912 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-20T18:44:24.912 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-20T18:44:24.912 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-20T18:44:24.912 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-20T18:44:24.913 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-20T18:44:24.913 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-20T18:44:24.913 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-20T18:44:24.913 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-09-20T18:44:24.913 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-20T18:44:24.913 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-20T18:44:24.913 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-20T18:44:24.913 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-20T18:44:24.913 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-20T18:44:24.913 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-20T18:44:24.913 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-20T18:44:24.913 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-20T18:44:24.913 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-20T18:44:24.913 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-09-20T18:44:24.913 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-20T18:44:24.914 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-20T18:44:24.914 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "btime": "2024-09-20T18:44:10:399616+0000", 2024-09-20T18:44:24.914 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-20T18:44:24.914 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-20T18:44:24.914 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-09-20T18:44:24.914 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-20T18:44:24.914 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-20T18:44:24.914 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-20T18:44:24.914 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-20T18:44:24.914 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-20T18:44:24.914 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-20T18:44:24.914 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "restful" 2024-09-20T18:44:24.914 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-09-20T18:44:24.914 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-20T18:44:24.915 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-09-20T18:44:24.915 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-20T18:44:24.915 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-20T18:44:24.915 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "modified": "2024-09-20T18:44:10.400266+0000", 2024-09-20T18:44:24.915 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-20T18:44:24.915 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-09-20T18:44:24.915 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-20T18:44:24.915 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-09-20T18:44:24.915 INFO:teuthology.orchestra.run.smithi083.stdout:mgr is available 2024-09-20T18:44:25.784 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:25 smithi083 ceph-mon[29421]: mgrmap e4: y(active, since 2s) 2024-09-20T18:44:26.056 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-09-20T18:44:26.056 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout [global] 2024-09-20T18:44:26.056 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout fsid = 1ed5a50a-7780-11ef-baf5-efdc52797490 2024-09-20T18:44:26.057 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-20T18:44:26.057 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.83:3300,v1:172.21.15.83:6789] 2024-09-20T18:44:26.057 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-20T18:44:26.057 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-20T18:44:26.057 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-20T18:44:26.057 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-20T18:44:26.057 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-09-20T18:44:26.057 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-20T18:44:26.057 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-20T18:44:26.057 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-09-20T18:44:26.057 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout [osd] 2024-09-20T18:44:26.057 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-20T18:44:26.057 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-20T18:44:26.058 INFO:teuthology.orchestra.run.smithi083.stdout:Enabling cephadm module... 2024-09-20T18:44:26.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:26 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2946469223' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-20T18:44:27.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:27 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1008655692' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "cephadm"} : dispatch 2024-09-20T18:44:27.792 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:27 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ignoring --setuser ceph since I am not root 2024-09-20T18:44:27.792 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:27 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ignoring --setgroup ceph since I am not root 2024-09-20T18:44:28.766 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:28 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1008655692' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-20T18:44:28.766 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:28 smithi083 ceph-mon[29421]: mgrmap e5: y(active, since 5s) 2024-09-20T18:44:29.386 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-09-20T18:44:29.386 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-20T18:44:29.386 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-20T18:44:29.386 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-20T18:44:29.386 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-20T18:44:29.386 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-09-20T18:44:29.386 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for the mgr to restart... 2024-09-20T18:44:29.386 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for mgr epoch 5... 2024-09-20T18:44:29.638 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:29 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/662370079' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-20T18:44:32.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:32 smithi083 ceph-mon[29421]: Active manager daemon y restarted 2024-09-20T18:44:32.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:32 smithi083 ceph-mon[29421]: Activating manager daemon y 2024-09-20T18:44:32.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:32 smithi083 ceph-mon[29421]: osdmap e2: 0 total, 0 up, 0 in 2024-09-20T18:44:32.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:32 smithi083 ceph-mon[29421]: mgrmap e6: y(active, starting, since 0.00332765s) 2024-09-20T18:44:32.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:32 smithi083 ceph-mon[29421]: from='mgr.14118 172.21.15.83:0/3019711898' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-20T18:44:32.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:32 smithi083 ceph-mon[29421]: from='mgr.14118 172.21.15.83:0/3019711898' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-20T18:44:32.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:32 smithi083 ceph-mon[29421]: from='mgr.14118 172.21.15.83:0/3019711898' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-20T18:44:32.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:32 smithi083 ceph-mon[29421]: from='mgr.14118 172.21.15.83:0/3019711898' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-20T18:44:32.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:32 smithi083 ceph-mon[29421]: from='mgr.14118 172.21.15.83:0/3019711898' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-20T18:44:32.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:32 smithi083 ceph-mon[29421]: Manager daemon y is now available 2024-09-20T18:44:34.001 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-09-20T18:44:34.001 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-20T18:44:34.001 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-20T18:44:34.001 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-09-20T18:44:34.002 INFO:teuthology.orchestra.run.smithi083.stdout:mgr epoch 5 is available 2024-09-20T18:44:34.002 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying orchestrator module is enabled... 2024-09-20T18:44:34.409 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:34 smithi083 ceph-mon[29421]: from='mgr.14118 172.21.15.83:0/3019711898' entity='mgr.y' 2024-09-20T18:44:34.409 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:34 smithi083 ceph-mon[29421]: from='mgr.14118 172.21.15.83:0/3019711898' entity='mgr.y' 2024-09-20T18:44:34.409 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:34 smithi083 ceph-mon[29421]: Found migration_current of "None". Setting to last migration. 2024-09-20T18:44:34.409 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:34 smithi083 ceph-mon[29421]: from='mgr.14118 172.21.15.83:0/3019711898' entity='mgr.y' 2024-09-20T18:44:34.409 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:34 smithi083 ceph-mon[29421]: from='mgr.14118 172.21.15.83:0/3019711898' entity='mgr.y' 2024-09-20T18:44:34.410 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:34 smithi083 ceph-mon[29421]: from='mgr.14118 172.21.15.83:0/3019711898' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:44:34.410 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:34 smithi083 ceph-mon[29421]: from='mgr.14118 172.21.15.83:0/3019711898' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:44:34.410 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:34 smithi083 ceph-mon[29421]: from='mgr.14118 172.21.15.83:0/3019711898' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-20T18:44:34.410 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:34 smithi083 ceph-mon[29421]: from='mgr.14118 172.21.15.83:0/3019711898' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-20T18:44:34.410 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:34 smithi083 ceph-mon[29421]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-20T18:44:34.410 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:34 smithi083 ceph-mon[29421]: mgrmap e7: y(active, since 1.00536s) 2024-09-20T18:44:34.410 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:34 smithi083 ceph-mon[29421]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-20T18:44:35.054 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:34 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: 2024-09-20T18:44:34.709+0000 7fe5f549f640 -1 log_channel(cephadm) log [ERR] : [20/Sep/2024:18:44:34] ENGINE Error in HTTPServer.serve 2024-09-20T18:44:35.054 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:34 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: Traceback (most recent call last): 2024-09-20T18:44:35.054 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:34 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-20T18:44:35.054 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:34 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: self._connections.run(self.expiration_interval) 2024-09-20T18:44:35.054 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:34 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-20T18:44:35.054 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:34 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: self._run(expiration_interval) 2024-09-20T18:44:35.054 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:34 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-20T18:44:35.054 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:34 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: new_conn = self._from_server_socket(self.server.socket) 2024-09-20T18:44:35.055 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:34 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-20T18:44:35.055 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:34 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-20T18:44:35.055 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:34 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-20T18:44:35.055 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:34 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: s = self.context.wrap_socket( 2024-09-20T18:44:35.055 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:34 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-20T18:44:35.055 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:34 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: return self.sslsocket_class._create( 2024-09-20T18:44:35.055 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:34 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-20T18:44:35.055 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:34 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: self.do_handshake() 2024-09-20T18:44:35.055 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:34 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-20T18:44:35.055 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:34 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: self._sslobj.do_handshake() 2024-09-20T18:44:35.055 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:34 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-20T18:44:35.055 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:34 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: 2024-09-20T18:44:35.490 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:35 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/159809072' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "orchestrator"} : dispatch 2024-09-20T18:44:35.490 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:35 smithi083 ceph-mon[29421]: from='mgr.14118 172.21.15.83:0/3019711898' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:44:35.741 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr module 'orchestrator' is already enabled (always-on) 2024-09-20T18:44:35.742 INFO:teuthology.orchestra.run.smithi083.stdout:Setting orchestrator backend to cephadm... 2024-09-20T18:44:36.389 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:36 smithi083 ceph-mon[29421]: [20/Sep/2024:18:44:34] ENGINE Bus STARTING 2024-09-20T18:44:36.389 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:36 smithi083 ceph-mon[29421]: [20/Sep/2024:18:44:34] ENGINE Serving on https://172.21.15.83:7150 2024-09-20T18:44:36.389 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:36 smithi083 ceph-mon[29421]: [20/Sep/2024:18:44:34] ENGINE Error in HTTPServer.serve 2024-09-20T18:44:36.390 INFO:journalctl@ceph.mon.a.smithi083.stdout: Traceback (most recent call last): 2024-09-20T18:44:36.390 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-20T18:44:36.390 INFO:journalctl@ceph.mon.a.smithi083.stdout: self._connections.run(self.expiration_interval) 2024-09-20T18:44:36.390 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-20T18:44:36.390 INFO:journalctl@ceph.mon.a.smithi083.stdout: self._run(expiration_interval) 2024-09-20T18:44:36.390 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-20T18:44:36.390 INFO:journalctl@ceph.mon.a.smithi083.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-20T18:44:36.390 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-20T18:44:36.390 INFO:journalctl@ceph.mon.a.smithi083.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-20T18:44:36.390 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-20T18:44:36.390 INFO:journalctl@ceph.mon.a.smithi083.stdout: s = self.context.wrap_socket( 2024-09-20T18:44:36.390 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-20T18:44:36.390 INFO:journalctl@ceph.mon.a.smithi083.stdout: return self.sslsocket_class._create( 2024-09-20T18:44:36.391 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-20T18:44:36.391 INFO:journalctl@ceph.mon.a.smithi083.stdout: self.do_handshake() 2024-09-20T18:44:36.391 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-20T18:44:36.391 INFO:journalctl@ceph.mon.a.smithi083.stdout: self._sslobj.do_handshake() 2024-09-20T18:44:36.391 INFO:journalctl@ceph.mon.a.smithi083.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-20T18:44:36.391 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:36 smithi083 ceph-mon[29421]: [20/Sep/2024:18:44:34] ENGINE Serving on http://172.21.15.83:8765 2024-09-20T18:44:36.391 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:36 smithi083 ceph-mon[29421]: [20/Sep/2024:18:44:34] ENGINE Bus STARTED 2024-09-20T18:44:36.391 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:36 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/159809072' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "orchestrator"}]': finished 2024-09-20T18:44:36.391 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:36 smithi083 ceph-mon[29421]: mgrmap e8: y(active, since 2s) 2024-09-20T18:44:37.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:37 smithi083 ceph-mon[29421]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:44:37.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:37 smithi083 ceph-mon[29421]: from='mgr.14118 172.21.15.83:0/3019711898' entity='mgr.y' 2024-09-20T18:44:37.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:37 smithi083 ceph-mon[29421]: from='mgr.14118 172.21.15.83:0/3019711898' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:44:38.171 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-20T18:44:38.171 INFO:teuthology.orchestra.run.smithi083.stdout:Generating ssh key... 2024-09-20T18:44:38.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:38 smithi083 ceph-mon[29421]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:44:39.425 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:39 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: Generating public/private rsa key pair. 2024-09-20T18:44:39.425 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:39 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: Your identification has been saved in /tmp/tmpbe109uaj/key 2024-09-20T18:44:39.425 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:39 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: Your public key has been saved in /tmp/tmpbe109uaj/key.pub 2024-09-20T18:44:39.425 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:39 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: The key fingerprint is: 2024-09-20T18:44:39.425 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:39 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: SHA256:i6k/OSxZAbxKv6O+oaIqIxCzFb5Iaaucg1cqXe5MoyU ceph-1ed5a50a-7780-11ef-baf5-efdc52797490 2024-09-20T18:44:39.425 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:39 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: The key's randomart image is: 2024-09-20T18:44:39.425 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:39 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: +---[RSA 3072]----+ 2024-09-20T18:44:39.425 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:39 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: | . | 2024-09-20T18:44:39.425 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:39 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: | . o | 2024-09-20T18:44:39.425 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:39 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: | ... o | 2024-09-20T18:44:39.425 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:39 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: |o++ . . | 2024-09-20T18:44:39.426 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:39 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: |oB.+ .S | 2024-09-20T18:44:39.426 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:39 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: |+.o + .o . | 2024-09-20T18:44:39.426 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:39 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: |=.E=+=o.. | 2024-09-20T18:44:39.426 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:39 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: |O=+B*o= | 2024-09-20T18:44:39.426 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:39 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: |X=*+++.o | 2024-09-20T18:44:39.426 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:39 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: +----[SHA256]-----+ 2024-09-20T18:44:40.250 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:40 smithi083 ceph-mon[29421]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:44:40.250 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:40 smithi083 ceph-mon[29421]: Generating ssh key... 2024-09-20T18:44:40.250 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:40 smithi083 ceph-mon[29421]: from='mgr.14118 172.21.15.83:0/3019711898' entity='mgr.y' 2024-09-20T18:44:40.250 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:40 smithi083 ceph-mon[29421]: from='mgr.14118 172.21.15.83:0/3019711898' entity='mgr.y' 2024-09-20T18:44:40.869 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCUkF7W/v7Jmqg58IM5vLwZB2iLaJz3bFX3LzNqpuXp4L5QsobSrpDosRJXLigTdNcBSFeOkY+CJZ3i74KPQuUURWzI1AGowuAXb1gSesspevteLl65uiZSFilgZmUGpCd1urCx04pZJHfnabBD0qGAgOwZQiaGJsEqPdOR96dWfFSNLXs6Uh6kTbp252WqPRrYgrqvTHUHKzYOV1oQjGUQNLrGvBcQ+zz1j+a+/IN8QwLfow0KE1Nv1CTGujDZQ5/JVH4P/ELtQXXmHAWuCV8KU8C88ucjMAzef/TuqkwG2hg5hgKPkGgGw3tefY91HvfecPrCf3fifhRwbFN0FDBB9O1+Bm6B05aD/BZ5k1BDlcEoYjJ0w/LwQ5mxKHoYW9Evmw0qBlic+s63Ubokxz7ARG6M7tEJ3Pd9NM6d2EODBs+FI+3laeDA0zWVxYw7ZfGqNk86lAqMjBCEt0iBvgBpewpxSOc50c1q3VmsPT3AlqTUWGh1nU6qs+d80TlXQSE= ceph-1ed5a50a-7780-11ef-baf5-efdc52797490 2024-09-20T18:44:40.869 INFO:teuthology.orchestra.run.smithi083.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-20T18:44:40.869 INFO:teuthology.orchestra.run.smithi083.stdout:Adding key to root@localhost authorized_keys... 2024-09-20T18:44:40.872 INFO:teuthology.orchestra.run.smithi083.stdout:Adding host smithi083... 2024-09-20T18:44:41.185 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:41 smithi083 ceph-mon[29421]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:44:42.213 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:42 smithi083 ceph-mon[29421]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi083", "addr": "172.21.15.83", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:44:43.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:43 smithi083 ceph-mon[29421]: Deploying cephadm binary to smithi083 2024-09-20T18:44:45.153 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout Added host 'smithi083' with addr '172.21.15.83' 2024-09-20T18:44:45.153 INFO:teuthology.orchestra.run.smithi083.stdout:Deploying unmanaged mon service... 2024-09-20T18:44:45.806 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:45 smithi083 ceph-mon[29421]: from='mgr.14118 172.21.15.83:0/3019711898' entity='mgr.y' 2024-09-20T18:44:45.806 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:45 smithi083 ceph-mon[29421]: Added host smithi083 2024-09-20T18:44:45.806 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:45 smithi083 ceph-mon[29421]: from='mgr.14118 172.21.15.83:0/3019711898' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:44:46.547 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-20T18:44:46.547 INFO:teuthology.orchestra.run.smithi083.stdout:Deploying unmanaged mgr service... 2024-09-20T18:44:47.150 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:46 smithi083 ceph-mon[29421]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:44:47.150 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:46 smithi083 ceph-mon[29421]: Saving service mon spec with placement count:5 2024-09-20T18:44:47.150 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:46 smithi083 ceph-mon[29421]: from='mgr.14118 172.21.15.83:0/3019711898' entity='mgr.y' 2024-09-20T18:44:47.873 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-20T18:44:48.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:48 smithi083 ceph-mon[29421]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:44:48.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:48 smithi083 ceph-mon[29421]: Saving service mgr spec with placement count:2 2024-09-20T18:44:48.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:48 smithi083 ceph-mon[29421]: from='mgr.14118 172.21.15.83:0/3019711898' entity='mgr.y' 2024-09-20T18:44:49.666 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:49 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/223826040' entity='client.admin' 2024-09-20T18:44:49.666 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:49 smithi083 ceph-mon[29421]: from='mgr.14118 172.21.15.83:0/3019711898' entity='mgr.y' 2024-09-20T18:44:50.506 INFO:teuthology.orchestra.run.smithi083.stdout:Enabling the dashboard module... 2024-09-20T18:44:51.234 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:50 smithi083 ceph-mon[29421]: from='mgr.14118 172.21.15.83:0/3019711898' entity='mgr.y' 2024-09-20T18:44:51.234 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:50 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1262613251' entity='client.admin' 2024-09-20T18:44:51.947 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:51 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/460840593' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "dashboard"} : dispatch 2024-09-20T18:44:51.947 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:51 smithi083 ceph-mon[29421]: from='mgr.14118 172.21.15.83:0/3019711898' entity='mgr.y' 2024-09-20T18:44:52.247 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:51 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ignoring --setuser ceph since I am not root 2024-09-20T18:44:52.248 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:51 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ignoring --setgroup ceph since I am not root 2024-09-20T18:44:53.223 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:52 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/460840593' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-20T18:44:53.223 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:52 smithi083 ceph-mon[29421]: mgrmap e9: y(active, since 19s) 2024-09-20T18:44:53.767 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-09-20T18:44:53.767 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-20T18:44:53.767 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-20T18:44:53.767 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-20T18:44:53.767 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-20T18:44:53.767 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-09-20T18:44:53.767 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for the mgr to restart... 2024-09-20T18:44:53.768 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for mgr epoch 9... 2024-09-20T18:44:54.200 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:53 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1946216131' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-20T18:44:57.383 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:57 smithi083 ceph-mon[29421]: Active manager daemon y restarted 2024-09-20T18:44:57.383 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:57 smithi083 ceph-mon[29421]: Activating manager daemon y 2024-09-20T18:44:57.383 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:57 smithi083 ceph-mon[29421]: osdmap e3: 0 total, 0 up, 0 in 2024-09-20T18:44:57.383 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:57 smithi083 ceph-mon[29421]: mgrmap e10: y(active, starting, since 0.0038647s) 2024-09-20T18:44:57.383 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:57 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-20T18:44:57.383 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:57 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-20T18:44:57.383 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:57 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-20T18:44:57.383 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:57 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-20T18:44:57.383 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:57 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-20T18:44:57.383 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:57 smithi083 ceph-mon[29421]: Manager daemon y is now available 2024-09-20T18:44:58.283 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:58 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:44:58.283 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:58 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-20T18:44:58.284 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:58 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-20T18:44:58.284 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:58 smithi083 ceph-mon[29421]: mgrmap e11: y(active, since 1.00592s) 2024-09-20T18:44:58.803 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:58 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: 2024-09-20T18:44:58.532+0000 7fbe01ce8640 -1 log_channel(cephadm) log [ERR] : [20/Sep/2024:18:44:58] ENGINE Error in HTTPServer.serve 2024-09-20T18:44:58.803 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:58 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: Traceback (most recent call last): 2024-09-20T18:44:58.803 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:58 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-20T18:44:58.804 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:58 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: self._connections.run(self.expiration_interval) 2024-09-20T18:44:58.804 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:58 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-20T18:44:58.804 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:58 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: self._run(expiration_interval) 2024-09-20T18:44:58.804 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:58 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-20T18:44:58.804 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:58 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: new_conn = self._from_server_socket(self.server.socket) 2024-09-20T18:44:58.804 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:58 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-20T18:44:58.804 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:58 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-20T18:44:58.804 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:58 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-20T18:44:58.804 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:58 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: s = self.context.wrap_socket( 2024-09-20T18:44:58.804 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:58 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-20T18:44:58.804 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:58 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: return self.sslsocket_class._create( 2024-09-20T18:44:58.804 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:58 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-20T18:44:58.804 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:58 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: self.do_handshake() 2024-09-20T18:44:58.804 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:58 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-20T18:44:58.804 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:58 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: self._sslobj.do_handshake() 2024-09-20T18:44:58.805 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:58 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-20T18:44:58.805 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:44:58 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: 2024-09-20T18:44:58.986 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-09-20T18:44:58.986 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-20T18:44:58.986 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-20T18:44:58.986 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-09-20T18:44:58.986 INFO:teuthology.orchestra.run.smithi083.stdout:mgr epoch 9 is available 2024-09-20T18:44:58.986 INFO:teuthology.orchestra.run.smithi083.stdout:Using certmgr to generate dashboard self-signed certificate... 2024-09-20T18:44:59.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:59 smithi083 ceph-mon[29421]: [20/Sep/2024:18:44:58] ENGINE Bus STARTING 2024-09-20T18:44:59.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:59 smithi083 ceph-mon[29421]: [20/Sep/2024:18:44:58] ENGINE Serving on http://172.21.15.83:8765 2024-09-20T18:44:59.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:59 smithi083 ceph-mon[29421]: [20/Sep/2024:18:44:58] ENGINE Serving on https://172.21.15.83:7150 2024-09-20T18:44:59.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:59 smithi083 ceph-mon[29421]: [20/Sep/2024:18:44:58] ENGINE Bus STARTED 2024-09-20T18:44:59.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:59 smithi083 ceph-mon[29421]: [20/Sep/2024:18:44:58] ENGINE Error in HTTPServer.serve 2024-09-20T18:44:59.907 INFO:journalctl@ceph.mon.a.smithi083.stdout: Traceback (most recent call last): 2024-09-20T18:44:59.912 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-20T18:44:59.912 INFO:journalctl@ceph.mon.a.smithi083.stdout: self._connections.run(self.expiration_interval) 2024-09-20T18:44:59.912 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-20T18:44:59.912 INFO:journalctl@ceph.mon.a.smithi083.stdout: self._run(expiration_interval) 2024-09-20T18:44:59.912 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-20T18:44:59.912 INFO:journalctl@ceph.mon.a.smithi083.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-20T18:44:59.912 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-20T18:44:59.912 INFO:journalctl@ceph.mon.a.smithi083.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-20T18:44:59.912 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-20T18:44:59.912 INFO:journalctl@ceph.mon.a.smithi083.stdout: s = self.context.wrap_socket( 2024-09-20T18:44:59.912 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-20T18:44:59.912 INFO:journalctl@ceph.mon.a.smithi083.stdout: return self.sslsocket_class._create( 2024-09-20T18:44:59.913 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-20T18:44:59.913 INFO:journalctl@ceph.mon.a.smithi083.stdout: self.do_handshake() 2024-09-20T18:44:59.913 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-20T18:44:59.913 INFO:journalctl@ceph.mon.a.smithi083.stdout: self._sslobj.do_handshake() 2024-09-20T18:44:59.913 INFO:journalctl@ceph.mon.a.smithi083.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-20T18:44:59.913 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:59 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:44:59.913 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:44:59 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:00.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:00 smithi083 ceph-mon[29421]: mgrmap e12: y(active, since 2s) 2024-09-20T18:45:00.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:00 smithi083 ceph-mon[29421]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "orch certmgr generate-certificates", "module_name": "dashboard", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:45:01.108 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout {"cert": "-----BEGIN CERTIFICATE-----\nMIIFEDCCAvigAwIBAgIUV01tZoyc1M9ocqoIOARVdicy8ZcwDQYJKoZIhvcNAQEL\nBQAwFzEVMBMGA1UEAwwMY2VwaGFkbS1yb290MB4XDTI0MDkyMDE4NDUwMFoXDTM0\nMDkyMTE4NDUwMFowFzEVMBMGA1UEAwwMMTcyLjIxLjE1LjgzMIICIjANBgkqhkiG\n9w0BAQEFAAOCAg8AMIICCgKCAgEAqtfCTDurm3I4zcovOqZhBaZxEcbbKAeV8oNV\nbG+YYWCCAZW5Vjf/zFigHAudesxCn6Yk6VkyVCgmnj/3PKnyUP5iIOfy/9DozoE+\nE4MqsYhfmJcaLxawERCVC/Ij1lN/C+U86T3LET/ZzfLpyppAhw9SSd0I06jXELXh\nAQasgmozhMQ/NBg8MQRJQ5L4d5X1p6hGSAMJit04Ii/SIsCBGLQY5+dYU6lJtxnp\niHb5x0J0XDmNb4rLrtDluVuyvpmt+8xbsftGvZMx05EVMILE5/tEjQh2pDx+24GF\nnffx1mo5w86UEKbjchPFqvEWMBqJyng2Mxo6ljb1ThcJ/jKfItRV2QcQkHPMSEXD\nnFNNNBZWhuqhMoTR4iHKZi3EWH4NPwl1aq7FmP333Q1Z8XIip993xf//6c9Iuely\nVwtqNRkOwaqA3sk2MzrbfTfGKZAIls9vtIIpYnqk4SVHPChhKbajLHVzr3//iNvX\nFoCut3Hherg/V6RMfFejkr8ZDMA23Wma+wfHs/TXQV1m5vy6bvrvJ1PZ9bP5yxcv\noVy+aB+tr79R/ONxGQeHws3CcOlR1+r6hEUAZN7WYK9vbJSrsmSpCjROeE1x3n13\nxcPlPnJa+ZizFti1aK6nTKAxAbV5T3+iJEJSRSZfZs8Yr/IK1+SUGr9TcL6Vk1wE\neJSfQlcCAwEAAaNUMFIwQgYDVR0RBDswOYIec21pdGhpMDgzLmZyb250LnNlcGlh\nLmNlcGguY29tghFkYXNoYm9hcmRfc2VydmVyc4cErBUPUzAMBgNVHRMBAf8EAjAA\nMA0GCSqGSIb3DQEBCwUAA4ICAQCNcI6fyxjFV6G15dHiJo3cox1ca/cbhW7mqsUS\nC6X8dShlh+utNpNxyobdsWlP83CvLG+/N8rgB6iYgrzzI76K7re7CAr4q6/HXiIb\nKc4AvRegWGvmGN/jIDvmtoZxR16Uia4LwOFzjLO4NX4rTcer1gLqz8mmkF7ZSWic\n+fz9IZDcNAcETQ6Utqu5czV+ww1tuUe2glg+9RYYOp625bTBzS4eAwRf+EFs0w8S\nl4+DqZDTT/K+2lm69dDa5el8IA9fvhi876i1EKH4l/8tWdD2TDPT1HekWSZfovlQ\ncIpsj1bCBT70ZQkpWxPO/YLRNHxteeyBK8ALqANSzcyfo1sCHOM+xPsbgm9w0SOx\nfjR6FggdkG+eettzqNPn8GyvOFGVV24DGFpdL5ejysJ2IhcJFeFGGanY1X/1LFRV\n67CqGnBLU8MHvZR1v1rWS/PGrlKNyjp+5zU85XFYWreeYG+01VK1tGiabsK/beV5\n2yD3tIMXaW7A1lblHO3Sj40E6c/jiSgqmaP8sKPwfBqox9Hn/ymrinpIiQZNvsKM\noWh+dlx8SHTCuQ5fGA55jDfF+YSwlgsjRxKh1zMfFMywmyj6cZg1oVAQgctYfPOE\nJ9gZexYl+DB9l0tTi9BjOsAzVKR4WG6cSAM4OaW3Q+b7aOJr0hsIvnghipZuVlLs\ncaWBvQ==\n-----END CERTIFICATE-----\n", "key": "-----BEGIN RSA PRIVATE KEY-----\nMIIJKgIBAAKCAgEAqtfCTDurm3I4zcovOqZhBaZxEcbbKAeV8oNVbG+YYWCCAZW5\nVjf/zFigHAudesxCn6Yk6VkyVCgmnj/3PKnyUP5iIOfy/9DozoE+E4MqsYhfmJca\nLxawERCVC/Ij1lN/C+U86T3LET/ZzfLpyppAhw9SSd0I06jXELXhAQasgmozhMQ/\nNBg8MQRJQ5L4d5X1p6hGSAMJit04Ii/SIsCBGLQY5+dYU6lJtxnpiHb5x0J0XDmN\nb4rLrtDluVuyvpmt+8xbsftGvZMx05EVMILE5/tEjQh2pDx+24GFnffx1mo5w86U\nEKbjchPFqvEWMBqJyng2Mxo6ljb1ThcJ/jKfItRV2QcQkHPMSEXDnFNNNBZWhuqh\nMoTR4iHKZi3EWH4NPwl1aq7FmP333Q1Z8XIip993xf//6c9IuelyVwtqNRkOwaqA\n3sk2MzrbfTfGKZAIls9vtIIpYnqk4SVHPChhKbajLHVzr3//iNvXFoCut3Hherg/\nV6RMfFejkr8ZDMA23Wma+wfHs/TXQV1m5vy6bvrvJ1PZ9bP5yxcvoVy+aB+tr79R\n/ONxGQeHws3CcOlR1+r6hEUAZN7WYK9vbJSrsmSpCjROeE1x3n13xcPlPnJa+Ziz\nFti1aK6nTKAxAbV5T3+iJEJSRSZfZs8Yr/IK1+SUGr9TcL6Vk1wEeJSfQlcCAwEA\nAQKCAgBGr7boq9e5HW17WChqb+S83bzyygBrc5gLG2QA1buKrs+bsENa9RiViCXW\n9MdXw6PdQnULR6iXQXWZDIZTR03HTTl0vAPC7MEmkDsK/75ubip/jCV9TjnTFbVV\nl/5T84Ahf9vP9Y1Luct4dwVcmq4QM7qsbRP132n8tARA1G4UF7TxPfGYsF5XOZrG\npsy/TgCyOeG6cedE2XscSgT79AJ2Ii4b5ZCaLPUfGU2S9Qcr9scGUZm/iQH1duFV\n7CbszdE62bcm0PzX+0smamUGCVVHVqNqZxIPqigi0lhin/Aj8xereJGywhphXOTw\ncYfop1JTv6FNS5DqDhs5EDdGQdOtetlLX3qEsDKMReLTHHZty22MNdti0YKTDcH3\ncocgICrILU9vt381RX8rmY/ilvwJNSelpPKWkVuuYhtkDp/1BIJZXV5BEF6LwFTU\nnGOCHcHD41K/Ya13rFp96PJDOaMy83BcD4vHRwjmYsfaPoSk2XFfPiDiHnG8yDXO\nQy7wg45Un1x1JRdOlevu6RB2+TSGfUKFidEteN55t/JcmSCdl3SWHDJVS/K3Prch\nfG6H0BcrYZc/WmprHqRBzgeET5VoWCysI4PJvPsQfhr/4A660qSLGB+OWFwtK4C/\nFnOWOobgglzdQSWWRKZCclNf1GtWgGmSi58F5yJv6bwNjHxJ4QKCAQEA77TWN3UH\nEeqjFhAVCvk7yviRSTfMrIzhgXYSmG9wP0k5ML5+NzNQjwaUlj2gWAuf/ukQT/4n\nmD6KY9v7ImoqAN55fhp2FdqifvSSty+z+GEzBuIbiZGVBTuPdMDcL17zhF6oJY7z\nsuLPFG6N+XEYFpBMcQQOGdDCYcJnV4+cLJwqSF1xKhr6sXZejXlCHWO7bSMeOlJD\nUoGRYnx0ZF3dPQNnMy+zMqSx7fqs6tLUo0p7ExCSTVbl3R7wyd+K4kKfBJtUhjVr\nbvsbaDtV0KRdc2PL1lV1BFMsWP5mO+cv5OINVBwUoxYzdSeD+UqHMxdiaMQtEXtc\nngoqeBYzI1fBZQKCAQEAtnSeIC1qiqK5/XJDbYxnwVjTuW7nqvpM8Yyola9dKxA6\nCPrtKCofmqsu7hvcuZQ3Lw9hxWEA0+OOSrJvTTmPkKklUfp+xIGN8zhmsg6ZiA7/\nb15BDZ6MrT5Smecfsykj+CfljfDUPtrQja8t8tcGR+K3LSK9EP+3wHURCTrH5oGp\nJO5X49NvPBqjPxxIJZV3rn2mRNts86dBKQLjwFMJA/PBuUHhx/w4G7gPBIH67B9/\np4OFaz1wkiskC7DdcCgfcdSlU53kHV7hqRXv+l29rIiNcjTJSyBv2OiB31W/GhGL\n334PCVyNKIvrNpORPW3vzXTj2WiV+d8QteWq++R3CwKCAQEA5Bjlu3HuH1tygz0c\neQu/nR45Sj4B0rRgpgvQGCP64Q3xkPPEMQzwfJlbbTgJS58MScpHzobf8Jyanfmn\nmREDFT6ex6OweI1LEZnzvsNy80izALrKT3csIgWSNhyLqbwl5/EW+l50geYX/a5S\nBWpb7KS1SNn9/QzM26/dHNI4X67fAWuBkvqHUbV8tg0MH+FWab0AN6kuDOXkjZ42\nN4emTjfdzTR3xho4gmeGghHLa/Mk+HhNb7DgHTEjGkZQ54tTvFN+9BMDP2FNrDrC\nJz8w+eud/T/mkQ4AZlL2v8v3pEqDOmVaMfKfVr0adW85BiIbb+w1oqwL7wtnbExb\nrycDjQKCAQEAjeUz9KJdcWQMcWcjuVawelWyk0wFL5tpq5FQqtivBIaiG8nKFqpr\n2cW3wcY5aroyEI9N4ksGccNPLhiHJu/VDxP8yjydygVXeAgtFdz02jOPw/Txcr4T\necg2RSrSJGuiZaPhivn4lsll2Eqc2XcQk2dmPdv+XfIB3dMtZJgqTsIX4XjbMXuF\nzqCIxNQh59jOMw3U+UgM+5DfxB1TModhQ4mW55StGvK0UBpc04yisG/uv0Axyq0F\nlyel6NGr5UXO7qAJ03W/qnfX18AP2g59LGkmV+NliN+5HF/2YNI2xPdQaIeI+ibY\nQOhQDSnGkpelH4wAh1wnByTLwirTRwLu8QKCAQEAm4INrIwad+XqNEDKCORE11kr\nmztc2ZirNT4e4Enw6bH1e3vLkg38/DlSaTyKYzDsoT/Wm20fdlFziD8eqBBGQwzq\n0DajXeC/jgG2CXSYiV8/AyHBwtTYBS8TASKGPPz7fr/1kAQCUSeXqi98yVkjZf1D\nV+S527N4oTbcSDtOw3/8POtMbiHtBzLM/dElaNLuRaFVhO6+vuJzLZUQ+1vSuKi1\nl+doFMJ5cnVFuaiOp+Klow2MZo8RMqG2Khcw5fyQAiy8zlt4B2cI8wz2LjJcYrud\nnSAJP89ycmYhPcKAoGhf5ZJaTmo+I7dG+tjPZN7UvelWjounYWKpqxQPntCiaA==\n-----END RSA PRIVATE KEY-----\n"} 2024-09-20T18:45:02.340 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout SSL certificate updated 2024-09-20T18:45:02.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:02 smithi083 ceph-mon[29421]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:45:02.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:02 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:03.505 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout SSL certificate key updated 2024-09-20T18:45:03.505 INFO:teuthology.orchestra.run.smithi083.stdout:Creating initial admin user... 2024-09-20T18:45:04.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:03 smithi083 ceph-mon[29421]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:45:04.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:03 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:04.845 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$v.Gm24f9cF0IKL.Z/7gCIOEjJc7Kl4fWHtbx.H.29ERQ1aMxw7d2O", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726857904, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-20T18:45:04.845 INFO:teuthology.orchestra.run.smithi083.stdout:Fetching dashboard port number... 2024-09-20T18:45:05.382 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:05 smithi083 ceph-mon[29421]: from='client.14170 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:45:05.382 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:05 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:06.002 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 8443 2024-09-20T18:45:06.012 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-20T18:45:06.012 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout disabled 2024-09-20T18:45:06.020 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-20T18:45:06.020 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout inactive 2024-09-20T18:45:06.020 INFO:teuthology.orchestra.run.smithi083.stdout:firewalld.service is not enabled 2024-09-20T18:45:06.020 INFO:teuthology.orchestra.run.smithi083.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-20T18:45:06.022 INFO:teuthology.orchestra.run.smithi083.stdout:Ceph Dashboard is now available at: 2024-09-20T18:45:06.022 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:45:06.022 INFO:teuthology.orchestra.run.smithi083.stdout: URL: https://smithi083.front.sepia.ceph.com:8443/ 2024-09-20T18:45:06.023 INFO:teuthology.orchestra.run.smithi083.stdout: User: admin 2024-09-20T18:45:06.023 INFO:teuthology.orchestra.run.smithi083.stdout: Password: 2probo7sys 2024-09-20T18:45:06.023 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:45:06.023 INFO:teuthology.orchestra.run.smithi083.stdout:Saving cluster configuration to /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/config directory 2024-09-20T18:45:06.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:06 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/387561484' entity='client.admin' cmd={"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} : dispatch 2024-09-20T18:45:07.304 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-20T18:45:07.304 INFO:teuthology.orchestra.run.smithi083.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-20T18:45:07.304 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:45:07.304 INFO:teuthology.orchestra.run.smithi083.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-20T18:45:07.304 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:45:07.304 INFO:teuthology.orchestra.run.smithi083.stdout:Or, if you are only running a single cluster on this host: 2024-09-20T18:45:07.304 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:45:07.304 INFO:teuthology.orchestra.run.smithi083.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-20T18:45:07.305 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:45:07.305 INFO:teuthology.orchestra.run.smithi083.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-20T18:45:07.305 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:45:07.305 INFO:teuthology.orchestra.run.smithi083.stdout: ceph telemetry on 2024-09-20T18:45:07.305 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:45:07.305 INFO:teuthology.orchestra.run.smithi083.stdout:For more information see: 2024-09-20T18:45:07.305 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:45:07.305 INFO:teuthology.orchestra.run.smithi083.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-20T18:45:07.305 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:45:07.305 INFO:teuthology.orchestra.run.smithi083.stdout:Bootstrap complete. 2024-09-20T18:45:07.340 INFO:tasks.cephadm:Fetching config... 2024-09-20T18:45:07.340 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-20T18:45:07.341 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-20T18:45:07.356 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-20T18:45:07.356 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-20T18:45:07.356 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-20T18:45:07.411 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-20T18:45:07.411 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-20T18:45:07.411 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/keyring of=/dev/stdout 2024-09-20T18:45:07.475 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-20T18:45:07.476 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-20T18:45:07.476 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-20T18:45:07.531 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-20T18:45:07.531 DEBUG:teuthology.orchestra.run.smithi083:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCUkF7W/v7Jmqg58IM5vLwZB2iLaJz3bFX3LzNqpuXp4L5QsobSrpDosRJXLigTdNcBSFeOkY+CJZ3i74KPQuUURWzI1AGowuAXb1gSesspevteLl65uiZSFilgZmUGpCd1urCx04pZJHfnabBD0qGAgOwZQiaGJsEqPdOR96dWfFSNLXs6Uh6kTbp252WqPRrYgrqvTHUHKzYOV1oQjGUQNLrGvBcQ+zz1j+a+/IN8QwLfow0KE1Nv1CTGujDZQ5/JVH4P/ELtQXXmHAWuCV8KU8C88ucjMAzef/TuqkwG2hg5hgKPkGgGw3tefY91HvfecPrCf3fifhRwbFN0FDBB9O1+Bm6B05aD/BZ5k1BDlcEoYjJ0w/LwQ5mxKHoYW9Evmw0qBlic+s63Ubokxz7ARG6M7tEJ3Pd9NM6d2EODBs+FI+3laeDA0zWVxYw7ZfGqNk86lAqMjBCEt0iBvgBpewpxSOc50c1q3VmsPT3AlqTUWGh1nU6qs+d80TlXQSE= ceph-1ed5a50a-7780-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-20T18:45:07.607 INFO:teuthology.orchestra.run.smithi083.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCUkF7W/v7Jmqg58IM5vLwZB2iLaJz3bFX3LzNqpuXp4L5QsobSrpDosRJXLigTdNcBSFeOkY+CJZ3i74KPQuUURWzI1AGowuAXb1gSesspevteLl65uiZSFilgZmUGpCd1urCx04pZJHfnabBD0qGAgOwZQiaGJsEqPdOR96dWfFSNLXs6Uh6kTbp252WqPRrYgrqvTHUHKzYOV1oQjGUQNLrGvBcQ+zz1j+a+/IN8QwLfow0KE1Nv1CTGujDZQ5/JVH4P/ELtQXXmHAWuCV8KU8C88ucjMAzef/TuqkwG2hg5hgKPkGgGw3tefY91HvfecPrCf3fifhRwbFN0FDBB9O1+Bm6B05aD/BZ5k1BDlcEoYjJ0w/LwQ5mxKHoYW9Evmw0qBlic+s63Ubokxz7ARG6M7tEJ3Pd9NM6d2EODBs+FI+3laeDA0zWVxYw7ZfGqNk86lAqMjBCEt0iBvgBpewpxSOc50c1q3VmsPT3AlqTUWGh1nU6qs+d80TlXQSE= ceph-1ed5a50a-7780-11ef-baf5-efdc52797490 2024-09-20T18:45:07.619 DEBUG:teuthology.orchestra.run.smithi142:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCUkF7W/v7Jmqg58IM5vLwZB2iLaJz3bFX3LzNqpuXp4L5QsobSrpDosRJXLigTdNcBSFeOkY+CJZ3i74KPQuUURWzI1AGowuAXb1gSesspevteLl65uiZSFilgZmUGpCd1urCx04pZJHfnabBD0qGAgOwZQiaGJsEqPdOR96dWfFSNLXs6Uh6kTbp252WqPRrYgrqvTHUHKzYOV1oQjGUQNLrGvBcQ+zz1j+a+/IN8QwLfow0KE1Nv1CTGujDZQ5/JVH4P/ELtQXXmHAWuCV8KU8C88ucjMAzef/TuqkwG2hg5hgKPkGgGw3tefY91HvfecPrCf3fifhRwbFN0FDBB9O1+Bm6B05aD/BZ5k1BDlcEoYjJ0w/LwQ5mxKHoYW9Evmw0qBlic+s63Ubokxz7ARG6M7tEJ3Pd9NM6d2EODBs+FI+3laeDA0zWVxYw7ZfGqNk86lAqMjBCEt0iBvgBpewpxSOc50c1q3VmsPT3AlqTUWGh1nU6qs+d80TlXQSE= ceph-1ed5a50a-7780-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-20T18:45:07.869 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:07 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3382066556' entity='client.admin' 2024-09-20T18:45:07.924 INFO:teuthology.orchestra.run.smithi142.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCUkF7W/v7Jmqg58IM5vLwZB2iLaJz3bFX3LzNqpuXp4L5QsobSrpDosRJXLigTdNcBSFeOkY+CJZ3i74KPQuUURWzI1AGowuAXb1gSesspevteLl65uiZSFilgZmUGpCd1urCx04pZJHfnabBD0qGAgOwZQiaGJsEqPdOR96dWfFSNLXs6Uh6kTbp252WqPRrYgrqvTHUHKzYOV1oQjGUQNLrGvBcQ+zz1j+a+/IN8QwLfow0KE1Nv1CTGujDZQ5/JVH4P/ELtQXXmHAWuCV8KU8C88ucjMAzef/TuqkwG2hg5hgKPkGgGw3tefY91HvfecPrCf3fifhRwbFN0FDBB9O1+Bm6B05aD/BZ5k1BDlcEoYjJ0w/LwQ5mxKHoYW9Evmw0qBlic+s63Ubokxz7ARG6M7tEJ3Pd9NM6d2EODBs+FI+3laeDA0zWVxYw7ZfGqNk86lAqMjBCEt0iBvgBpewpxSOc50c1q3VmsPT3AlqTUWGh1nU6qs+d80TlXQSE= ceph-1ed5a50a-7780-11ef-baf5-efdc52797490 2024-09-20T18:45:07.938 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-20T18:45:08.188 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:45:09.817 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-20T18:45:09.817 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-20T18:45:10.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:09 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1603627324' entity='client.admin' 2024-09-20T18:45:10.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:09 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:10.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:09 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:10.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:09 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd/host:smithi083", "name": "osd_memory_target"} : dispatch 2024-09-20T18:45:10.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:09 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:10.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:09 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:45:10.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:09 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:10.059 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:45:11.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:10 smithi083 ceph-mon[29421]: mgrmap e13: y(active, since 12s) 2024-09-20T18:45:11.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:10 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:11.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:10 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:45:11.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:10 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:45:11.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:10 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:45:11.386 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi142 2024-09-20T18:45:11.387 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-20T18:45:11.387 DEBUG:teuthology.orchestra.run.smithi142:> dd of=/etc/ceph/ceph.conf 2024-09-20T18:45:11.406 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-20T18:45:11.406 DEBUG:teuthology.orchestra.run.smithi142:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-20T18:45:11.464 INFO:tasks.cephadm:Adding host smithi142 to orchestrator... 2024-09-20T18:45:11.464 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph orch host add smithi142 2024-09-20T18:45:11.703 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:45:12.063 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:11 smithi083 ceph-mon[29421]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:45:12.063 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:11 smithi083 ceph-mon[29421]: Updating smithi083:/etc/ceph/ceph.conf 2024-09-20T18:45:12.064 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:11 smithi083 ceph-mon[29421]: Updating smithi083:/var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-20T18:45:12.064 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:11 smithi083 ceph-mon[29421]: Updating smithi083:/etc/ceph/ceph.client.admin.keyring 2024-09-20T18:45:12.064 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:11 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:12.064 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:11 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:12.064 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:11 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:13.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:12 smithi083 ceph-mon[29421]: Updating smithi083:/var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-20T18:45:14.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:13 smithi083 ceph-mon[29421]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi142", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:45:15.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:14 smithi083 ceph-mon[29421]: Deploying cephadm binary to smithi142 2024-09-20T18:45:15.403 INFO:teuthology.orchestra.run.smithi083.stdout:Added host 'smithi142' with addr '172.21.15.142' 2024-09-20T18:45:16.035 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph orch host ls --format=json 2024-09-20T18:45:16.273 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:45:16.695 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:16 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:16.695 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:16 smithi083 ceph-mon[29421]: Added host smithi142 2024-09-20T18:45:16.695 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:16 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:45:16.695 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:16 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:16.695 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:16 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:16.973 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:45:16.974 INFO:teuthology.orchestra.run.smithi083.stdout:[{"addr": "172.21.15.83", "hostname": "smithi083", "labels": [], "status": ""}, {"addr": "172.21.15.142", "hostname": "smithi142", "labels": [], "status": ""}] 2024-09-20T18:45:17.535 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-20T18:45:17.536 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd crush tunables default 2024-09-20T18:45:17.773 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:45:17.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:17 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:17.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:17 smithi083 ceph-mon[29421]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:45:18.722 INFO:teuthology.orchestra.run.smithi083.stderr:adjusted tunables profile to default 2024-09-20T18:45:19.013 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:18 smithi083 ceph-mon[29421]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:19.013 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:18 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1597988666' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-20T18:45:19.293 INFO:tasks.cephadm:Adding mon.a on smithi083 2024-09-20T18:45:19.293 INFO:tasks.cephadm:Adding mon.c on smithi083 2024-09-20T18:45:19.293 INFO:tasks.cephadm:Adding mon.b on smithi142 2024-09-20T18:45:19.293 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph orch apply mon '3;smithi083:172.21.15.83=a;smithi083:[v2:172.21.15.83:3301,v1:172.21.15.83:6790]=c;smithi142:172.21.15.142=b' 2024-09-20T18:45:19.765 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-20T18:45:19.811 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-20T18:45:20.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:19 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1597988666' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-20T18:45:20.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:19 smithi083 ceph-mon[29421]: osdmap e4: 0 total, 0 up, 0 in 2024-09-20T18:45:20.510 INFO:teuthology.orchestra.run.smithi142.stdout:Scheduled mon update... 2024-09-20T18:45:20.751 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:20 smithi083 ceph-mon[29421]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:20.751 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:20 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:20.751 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:20 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:20.751 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:20 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:20.751 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:20 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:20.751 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:20 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd/host:smithi142", "name": "osd_memory_target"} : dispatch 2024-09-20T18:45:20.751 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:20 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:45:20.751 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:20 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:45:20.751 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:20 smithi083 ceph-mon[29421]: Updating smithi142:/etc/ceph/ceph.conf 2024-09-20T18:45:20.752 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:20 smithi083 ceph-mon[29421]: Updating smithi142:/var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-20T18:45:20.752 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:20 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:20.752 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:20 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:20.752 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:20 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:20.752 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:20 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:20.752 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:20 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:45:20.752 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:20 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:45:20.752 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:20 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:45:20.752 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:20 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:20.752 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:20 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-20T18:45:20.752 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:20 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:45:21.120 DEBUG:teuthology.orchestra.run.smithi083:mon.c> sudo journalctl -f -n 0 -u ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@mon.c.service 2024-09-20T18:45:21.123 DEBUG:teuthology.orchestra.run.smithi142:mon.b> sudo journalctl -f -n 0 -u ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@mon.b.service 2024-09-20T18:45:21.125 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-20T18:45:21.126 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-20T18:45:22.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:21 smithi083 ceph-mon[29421]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-09-20T18:45:22.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:21 smithi083 ceph-mon[29421]: Updating smithi142:/var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-20T18:45:22.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:21 smithi083 ceph-mon[29421]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi083:172.21.15.83=a;smithi083:[v2:172.21.15.83:3301,v1:172.21.15.83:6790]=c;smithi142:172.21.15.142=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:45:22.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:21 smithi083 ceph-mon[29421]: Saving service mon spec with placement smithi083:172.21.15.83=a;smithi083:[v2:172.21.15.83:3301,v1:172.21.15.83:6790]=c;smithi142:172.21.15.142=b;count:3 2024-09-20T18:45:22.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:21 smithi083 ceph-mon[29421]: Deploying daemon mon.b on smithi142 2024-09-20T18:45:22.235 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-20T18:45:23.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:22 smithi083 ceph-mon[29421]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:24.084 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:45:24.084 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":1,"fsid":"1ed5a50a-7780-11ef-baf5-efdc52797490","modified":"2024-09-20T18:44:06.997446Z","created":"2024-09-20T18:44:06.997446Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:3300","nonce":0},{"type":"v1","addr":"172.21.15.83:6789","nonce":0}]},"addr":"172.21.15.83:6789/0","public_addr":"172.21.15.83:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-20T18:45:24.085 INFO:teuthology.orchestra.run.smithi142.stderr:dumped monmap epoch 1 2024-09-20T18:45:25.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:24 smithi083 ceph-mon[29421]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:25.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:24 smithi083 ceph-mon[29421]: from='client.? 172.21.15.142:0/3493862986' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-20T18:45:25.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 podman[32488]: 2024-09-20 18:45:25.244873044 +0000 UTC m=+0.357938107 container start 39a1eb744183509762a450d7f8d4dd0244d65c54e86fa5e1e8f0335a4524b998 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mon-b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS) 2024-09-20T18:45:25.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: set uid:gid to 167:167 (ceph:ceph) 2024-09-20T18:45:25.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: ceph version 19.3.0-5099-g88deadb8 (88deadb8082f4a33832e228ccefc59cedba158a5) squid (dev), process ceph-mon, pid 2 2024-09-20T18:45:25.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: pidfile_write: ignore empty --pid-file 2024-09-20T18:45:25.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: load: jerasure load: lrc 2024-09-20T18:45:25.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: RocksDB version: 7.9.2 2024-09-20T18:45:25.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Git sha 0 2024-09-20T18:45:25.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Compile date 2024-09-20 10:48:39 2024-09-20T18:45:25.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: DB SUMMARY 2024-09-20T18:45:25.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: DB Session ID: FKJBDJ776XSY2ISV3R70 2024-09-20T18:45:25.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: CURRENT file: CURRENT 2024-09-20T18:45:25.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: IDENTITY file: IDENTITY 2024-09-20T18:45:25.775 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-20T18:45:25.775 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-20T18:45:25.775 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 511 ; 2024-09-20T18:45:25.775 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.error_if_exists: 0 2024-09-20T18:45:25.775 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.create_if_missing: 0 2024-09-20T18:45:25.775 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.paranoid_checks: 1 2024-09-20T18:45:25.775 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-20T18:45:25.775 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-20T18:45:25.775 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-20T18:45:25.775 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.env: 0x55c223144120 2024-09-20T18:45:25.775 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.fs: PosixFileSystem 2024-09-20T18:45:25.776 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.info_log: 0x55c2256af700 2024-09-20T18:45:25.776 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.max_file_opening_threads: 16 2024-09-20T18:45:25.776 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.statistics: (nil) 2024-09-20T18:45:25.776 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.use_fsync: 0 2024-09-20T18:45:25.776 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.max_log_file_size: 0 2024-09-20T18:45:25.776 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-20T18:45:25.776 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-20T18:45:25.776 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.keep_log_file_num: 1000 2024-09-20T18:45:25.776 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.recycle_log_file_num: 0 2024-09-20T18:45:25.776 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.allow_fallocate: 1 2024-09-20T18:45:25.776 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.allow_mmap_reads: 0 2024-09-20T18:45:25.776 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.allow_mmap_writes: 0 2024-09-20T18:45:25.777 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.use_direct_reads: 0 2024-09-20T18:45:25.777 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-20T18:45:25.777 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.create_missing_column_families: 0 2024-09-20T18:45:25.777 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.db_log_dir: 2024-09-20T18:45:25.777 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.wal_dir: 2024-09-20T18:45:25.777 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-20T18:45:25.777 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-20T18:45:25.777 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-20T18:45:25.777 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-20T18:45:25.777 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-20T18:45:25.777 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-20T18:45:25.777 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.advise_random_on_open: 1 2024-09-20T18:45:25.777 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.db_write_buffer_size: 0 2024-09-20T18:45:25.777 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.write_buffer_manager: 0x55c2256b3b80 2024-09-20T18:45:25.777 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-20T18:45:25.778 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-20T18:45:25.778 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-20T18:45:25.778 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.rate_limiter: (nil) 2024-09-20T18:45:25.778 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-20T18:45:25.778 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.wal_recovery_mode: 2 2024-09-20T18:45:25.778 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.enable_thread_tracking: 0 2024-09-20T18:45:25.778 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.enable_pipelined_write: 0 2024-09-20T18:45:25.778 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.unordered_write: 0 2024-09-20T18:45:25.778 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-20T18:45:25.778 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-20T18:45:25.778 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-20T18:45:25.778 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-20T18:45:25.778 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.row_cache: None 2024-09-20T18:45:25.778 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.wal_filter: None 2024-09-20T18:45:25.778 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-20T18:45:25.779 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.allow_ingest_behind: 0 2024-09-20T18:45:25.779 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.two_write_queues: 0 2024-09-20T18:45:25.779 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.manual_wal_flush: 0 2024-09-20T18:45:25.779 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.wal_compression: 0 2024-09-20T18:45:25.779 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.atomic_flush: 0 2024-09-20T18:45:25.779 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-20T18:45:25.779 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-20T18:45:25.779 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-20T18:45:25.779 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.log_readahead_size: 0 2024-09-20T18:45:25.779 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-20T18:45:25.779 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.best_efforts_recovery: 0 2024-09-20T18:45:25.779 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-20T18:45:25.779 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-20T18:45:25.779 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.allow_data_in_errors: 0 2024-09-20T18:45:25.779 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.db_host_id: __hostname__ 2024-09-20T18:45:25.779 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-20T18:45:25.781 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.max_background_jobs: 2 2024-09-20T18:45:25.781 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.max_background_compactions: -1 2024-09-20T18:45:25.781 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.max_subcompactions: 1 2024-09-20T18:45:25.781 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-20T18:45:25.781 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-20T18:45:25.781 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-20T18:45:25.781 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.max_total_wal_size: 0 2024-09-20T18:45:25.781 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-20T18:45:25.781 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-20T18:45:25.781 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-20T18:45:25.781 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-20T18:45:25.782 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.max_open_files: -1 2024-09-20T18:45:25.782 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.bytes_per_sync: 0 2024-09-20T18:45:25.782 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-20T18:45:25.782 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-20T18:45:25.782 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.compaction_readahead_size: 0 2024-09-20T18:45:25.782 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.max_background_flushes: -1 2024-09-20T18:45:25.782 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Compression algorithms supported: 2024-09-20T18:45:25.782 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: kZSTD supported: 0 2024-09-20T18:45:25.782 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: kXpressCompression supported: 0 2024-09-20T18:45:25.782 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: kBZip2Compression supported: 0 2024-09-20T18:45:25.782 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-20T18:45:25.782 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: kLZ4Compression supported: 1 2024-09-20T18:45:25.782 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: kZlibCompression supported: 1 2024-09-20T18:45:25.783 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: kLZ4HCCompression supported: 1 2024-09-20T18:45:25.783 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: kSnappyCompression supported: 1 2024-09-20T18:45:25.783 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-20T18:45:25.783 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-20T18:45:25.783 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-09-20T18:45:25.783 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-20T18:45:25.783 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-20T18:45:25.783 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.merge_operator: 2024-09-20T18:45:25.783 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.compaction_filter: None 2024-09-20T18:45:25.783 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.compaction_filter_factory: None 2024-09-20T18:45:25.783 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.sst_partitioner_factory: None 2024-09-20T18:45:25.783 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-20T18:45:25.783 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-20T18:45:25.783 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55c2256af320) 2024-09-20T18:45:25.784 INFO:journalctl@ceph.mon.b.smithi142.stdout: cache_index_and_filter_blocks: 1 2024-09-20T18:45:25.784 INFO:journalctl@ceph.mon.b.smithi142.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-20T18:45:25.784 INFO:journalctl@ceph.mon.b.smithi142.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-20T18:45:25.784 INFO:journalctl@ceph.mon.b.smithi142.stdout: pin_top_level_index_and_filter: 1 2024-09-20T18:45:25.784 INFO:journalctl@ceph.mon.b.smithi142.stdout: index_type: 0 2024-09-20T18:45:25.784 INFO:journalctl@ceph.mon.b.smithi142.stdout: data_block_index_type: 0 2024-09-20T18:45:25.784 INFO:journalctl@ceph.mon.b.smithi142.stdout: index_shortening: 1 2024-09-20T18:45:25.784 INFO:journalctl@ceph.mon.b.smithi142.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-20T18:45:25.784 INFO:journalctl@ceph.mon.b.smithi142.stdout: checksum: 4 2024-09-20T18:45:25.784 INFO:journalctl@ceph.mon.b.smithi142.stdout: no_block_cache: 0 2024-09-20T18:45:25.784 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_cache: 0x55c2256a3610 2024-09-20T18:45:25.784 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_cache_name: BinnedLRUCache 2024-09-20T18:45:25.784 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_cache_options: 2024-09-20T18:45:25.784 INFO:journalctl@ceph.mon.b.smithi142.stdout: capacity : 536870912 2024-09-20T18:45:25.784 INFO:journalctl@ceph.mon.b.smithi142.stdout: num_shard_bits : 4 2024-09-20T18:45:25.785 INFO:journalctl@ceph.mon.b.smithi142.stdout: strict_capacity_limit : 0 2024-09-20T18:45:25.785 INFO:journalctl@ceph.mon.b.smithi142.stdout: high_pri_pool_ratio: 0.000 2024-09-20T18:45:25.785 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_cache_compressed: (nil) 2024-09-20T18:45:25.785 INFO:journalctl@ceph.mon.b.smithi142.stdout: persistent_cache: (nil) 2024-09-20T18:45:25.785 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_size: 4096 2024-09-20T18:45:25.785 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_size_deviation: 10 2024-09-20T18:45:25.785 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_restart_interval: 16 2024-09-20T18:45:25.785 INFO:journalctl@ceph.mon.b.smithi142.stdout: index_block_restart_interval: 1 2024-09-20T18:45:25.785 INFO:journalctl@ceph.mon.b.smithi142.stdout: metadata_block_size: 4096 2024-09-20T18:45:25.785 INFO:journalctl@ceph.mon.b.smithi142.stdout: partition_filters: 0 2024-09-20T18:45:25.785 INFO:journalctl@ceph.mon.b.smithi142.stdout: use_delta_encoding: 1 2024-09-20T18:45:25.785 INFO:journalctl@ceph.mon.b.smithi142.stdout: filter_policy: bloomfilter 2024-09-20T18:45:25.785 INFO:journalctl@ceph.mon.b.smithi142.stdout: whole_key_filtering: 1 2024-09-20T18:45:25.786 INFO:journalctl@ceph.mon.b.smithi142.stdout: verify_compression: 0 2024-09-20T18:45:25.786 INFO:journalctl@ceph.mon.b.smithi142.stdout: read_amp_bytes_per_bit: 0 2024-09-20T18:45:25.786 INFO:journalctl@ceph.mon.b.smithi142.stdout: format_version: 5 2024-09-20T18:45:25.786 INFO:journalctl@ceph.mon.b.smithi142.stdout: enable_index_compression: 1 2024-09-20T18:45:25.786 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_align: 0 2024-09-20T18:45:25.786 INFO:journalctl@ceph.mon.b.smithi142.stdout: max_auto_readahead_size: 262144 2024-09-20T18:45:25.786 INFO:journalctl@ceph.mon.b.smithi142.stdout: prepopulate_block_cache: 0 2024-09-20T18:45:25.786 INFO:journalctl@ceph.mon.b.smithi142.stdout: initial_auto_readahead_size: 8192 2024-09-20T18:45:25.786 INFO:journalctl@ceph.mon.b.smithi142.stdout: num_file_reads_for_auto_readahead: 2 2024-09-20T18:45:25.786 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.write_buffer_size: 33554432 2024-09-20T18:45:25.786 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.max_write_buffer_number: 2 2024-09-20T18:45:25.786 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.compression: NoCompression 2024-09-20T18:45:25.786 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.bottommost_compression: Disabled 2024-09-20T18:45:25.786 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.prefix_extractor: nullptr 2024-09-20T18:45:25.787 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-20T18:45:25.787 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.num_levels: 7 2024-09-20T18:45:25.787 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-20T18:45:25.787 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-20T18:45:25.787 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-20T18:45:25.787 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-20T18:45:25.787 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-20T18:45:25.787 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-20T18:45:25.787 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-20T18:45:25.787 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-20T18:45:25.787 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-20T18:45:25.787 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-20T18:45:25.787 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-20T18:45:25.788 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-20T18:45:25.788 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-20T18:45:25.788 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.compression_opts.level: 32767 2024-09-20T18:45:25.788 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.compression_opts.strategy: 0 2024-09-20T18:45:25.788 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-20T18:45:25.788 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-20T18:45:25.788 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-20T18:45:25.788 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-20T18:45:25.788 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.compression_opts.enabled: false 2024-09-20T18:45:25.788 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-20T18:45:25.788 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-20T18:45:25.788 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-20T18:45:25.788 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-20T18:45:25.788 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.target_file_size_base: 67108864 2024-09-20T18:45:25.788 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-20T18:45:25.789 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-20T18:45:25.789 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-20T18:45:25.789 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-20T18:45:25.789 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-20T18:45:25.789 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-20T18:45:25.789 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-20T18:45:25.789 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-20T18:45:25.789 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-20T18:45:25.789 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-20T18:45:25.789 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-20T18:45:25.789 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-20T18:45:25.789 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-20T18:45:25.789 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-20T18:45:25.789 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.arena_block_size: 1048576 2024-09-20T18:45:25.790 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-20T18:45:25.790 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-20T18:45:25.790 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.disable_auto_compactions: 0 2024-09-20T18:45:25.790 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-20T18:45:25.790 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-20T18:45:25.790 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-20T18:45:25.790 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-20T18:45:25.790 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-20T18:45:25.790 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-20T18:45:25.790 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-20T18:45:25.790 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-20T18:45:25.790 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-20T18:45:25.790 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-20T18:45:25.791 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-20T18:45:25.791 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.inplace_update_support: 0 2024-09-20T18:45:25.791 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-20T18:45:25.791 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-20T18:45:25.791 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-20T18:45:25.791 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-20T18:45:25.791 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.bloom_locality: 0 2024-09-20T18:45:25.791 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.max_successive_merges: 0 2024-09-20T18:45:25.791 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-20T18:45:25.791 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.paranoid_file_checks: 0 2024-09-20T18:45:25.791 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.force_consistency_checks: 1 2024-09-20T18:45:25.791 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.report_bg_io_stats: 0 2024-09-20T18:45:25.792 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.ttl: 2592000 2024-09-20T18:45:25.792 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-20T18:45:25.792 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-20T18:45:25.792 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-20T18:45:25.792 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.enable_blob_files: false 2024-09-20T18:45:25.792 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.min_blob_size: 0 2024-09-20T18:45:25.792 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.blob_file_size: 268435456 2024-09-20T18:45:25.792 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-20T18:45:25.792 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-20T18:45:25.792 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-20T18:45:25.792 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-20T18:45:25.792 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-20T18:45:25.792 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.blob_file_starting_level: 0 2024-09-20T18:45:25.792 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-20T18:45:25.793 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-20T18:45:25.793 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-20T18:45:25.793 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: c8db8854-d093-4b4f-a62d-47427bdfbb82 2024-09-20T18:45:25.793 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726857925271840, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-20T18:45:25.793 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-20T18:45:25.793 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726857925272246, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726857925, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "c8db8854-d093-4b4f-a62d-47427bdfbb82", "db_session_id": "FKJBDJ776XSY2ISV3R70", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-20T18:45:25.793 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726857925272339, "job": 1, "event": "recovery_finished"} 2024-09-20T18:45:25.793 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-20T18:45:25.793 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-20T18:45:25.793 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55c2256d2e00 2024-09-20T18:45:25.793 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: DB pointer 0x55c2257d4000 2024-09-20T18:45:25.793 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-20T18:45:25.793 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-20T18:45:25.793 INFO:journalctl@ceph.mon.b.smithi142.stdout: ** DB Stats ** 2024-09-20T18:45:25.794 INFO:journalctl@ceph.mon.b.smithi142.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-20T18:45:25.794 INFO:journalctl@ceph.mon.b.smithi142.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-20T18:45:25.794 INFO:journalctl@ceph.mon.b.smithi142.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-20T18:45:25.794 INFO:journalctl@ceph.mon.b.smithi142.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-20T18:45:25.794 INFO:journalctl@ceph.mon.b.smithi142.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-20T18:45:25.794 INFO:journalctl@ceph.mon.b.smithi142.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-20T18:45:25.794 INFO:journalctl@ceph.mon.b.smithi142.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-20T18:45:25.794 INFO:journalctl@ceph.mon.b.smithi142.stdout: 2024-09-20T18:45:25.794 INFO:journalctl@ceph.mon.b.smithi142.stdout: ** Compaction Stats [default] ** 2024-09-20T18:45:25.794 INFO:journalctl@ceph.mon.b.smithi142.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-20T18:45:25.794 INFO:journalctl@ceph.mon.b.smithi142.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-20T18:45:25.794 INFO:journalctl@ceph.mon.b.smithi142.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-20T18:45:25.794 INFO:journalctl@ceph.mon.b.smithi142.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-20T18:45:25.795 INFO:journalctl@ceph.mon.b.smithi142.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-20T18:45:25.795 INFO:journalctl@ceph.mon.b.smithi142.stdout: 2024-09-20T18:45:25.795 INFO:journalctl@ceph.mon.b.smithi142.stdout: ** Compaction Stats [default] ** 2024-09-20T18:45:25.795 INFO:journalctl@ceph.mon.b.smithi142.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-20T18:45:25.795 INFO:journalctl@ceph.mon.b.smithi142.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-20T18:45:25.795 INFO:journalctl@ceph.mon.b.smithi142.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-20T18:45:25.795 INFO:journalctl@ceph.mon.b.smithi142.stdout: 2024-09-20T18:45:25.795 INFO:journalctl@ceph.mon.b.smithi142.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-20T18:45:25.795 INFO:journalctl@ceph.mon.b.smithi142.stdout: 2024-09-20T18:45:25.795 INFO:journalctl@ceph.mon.b.smithi142.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-20T18:45:25.795 INFO:journalctl@ceph.mon.b.smithi142.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-20T18:45:25.795 INFO:journalctl@ceph.mon.b.smithi142.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-20T18:45:25.795 INFO:journalctl@ceph.mon.b.smithi142.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-20T18:45:25.795 INFO:journalctl@ceph.mon.b.smithi142.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-20T18:45:25.796 INFO:journalctl@ceph.mon.b.smithi142.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-20T18:45:25.796 INFO:journalctl@ceph.mon.b.smithi142.stdout: Cumulative compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-20T18:45:25.796 INFO:journalctl@ceph.mon.b.smithi142.stdout: Interval compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-20T18:45:25.796 INFO:journalctl@ceph.mon.b.smithi142.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-20T18:45:25.796 INFO:journalctl@ceph.mon.b.smithi142.stdout: Block cache BinnedLRUCache@0x55c2256a3610#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-20T18:45:25.796 INFO:journalctl@ceph.mon.b.smithi142.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-20T18:45:25.796 INFO:journalctl@ceph.mon.b.smithi142.stdout: 2024-09-20T18:45:25.796 INFO:journalctl@ceph.mon.b.smithi142.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-20T18:45:25.796 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-20T18:45:25.796 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: using public_addr v2:172.21.15.142:0/0 -> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] 2024-09-20T18:45:25.796 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: starting mon.b rank -1 at public addrs [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] at bind addrs [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 2024-09-20T18:45:25.796 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: mon.b@-1(???) e0 preinit fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 2024-09-20T18:45:25.796 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: mon.b@-1(synchronizing).mds e1 new map 2024-09-20T18:45:25.797 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-20T18:45:25.797 INFO:journalctl@ceph.mon.b.smithi142.stdout: e1 2024-09-20T18:45:25.797 INFO:journalctl@ceph.mon.b.smithi142.stdout: btime 2024-09-20T18:44:10:399616+0000 2024-09-20T18:45:25.797 INFO:journalctl@ceph.mon.b.smithi142.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-20T18:45:25.797 INFO:journalctl@ceph.mon.b.smithi142.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-20T18:45:25.797 INFO:journalctl@ceph.mon.b.smithi142.stdout: legacy client fscid: -1 2024-09-20T18:45:25.797 INFO:journalctl@ceph.mon.b.smithi142.stdout: 2024-09-20T18:45:25.797 INFO:journalctl@ceph.mon.b.smithi142.stdout: No filesystems configured 2024-09-20T18:45:25.797 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-20T18:45:25.797 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-20T18:45:25.797 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-20T18:45:25.797 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-20T18:45:25.797 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-20T18:45:25.797 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-20T18:45:25.797 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-20T18:45:25.798 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-20T18:45:25.798 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-20T18:45:25.798 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-20T18:45:25.798 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:25.798 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1597988666' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-20T18:45:25.798 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1597988666' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-20T18:45:25.798 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: osdmap e4: 0 total, 0 up, 0 in 2024-09-20T18:45:25.798 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:25.798 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:25.798 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:25.798 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:25.798 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:25.798 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd/host:smithi142", "name": "osd_memory_target"} : dispatch 2024-09-20T18:45:25.798 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:45:25.798 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:45:25.799 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: Updating smithi142:/etc/ceph/ceph.conf 2024-09-20T18:45:25.799 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: Updating smithi142:/var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-20T18:45:25.799 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:25.799 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:25.799 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:25.799 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:25.799 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:45:25.799 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:45:25.799 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:45:25.799 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:25.799 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-20T18:45:25.799 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:45:25.799 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-09-20T18:45:25.800 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: Updating smithi142:/var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-20T18:45:25.800 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi083:172.21.15.83=a;smithi083:[v2:172.21.15.83:3301,v1:172.21.15.83:6790]=c;smithi142:172.21.15.142=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:45:25.800 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: Saving service mon spec with placement smithi083:172.21.15.83=a;smithi083:[v2:172.21.15.83:3301,v1:172.21.15.83:6790]=c;smithi142:172.21.15.142=b;count:3 2024-09-20T18:45:25.800 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: Deploying daemon mon.b on smithi142 2024-09-20T18:45:25.800 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:25.800 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:25.800 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: from='client.? 172.21.15.142:0/3493862986' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-20T18:45:25.800 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 ceph-mon[32508]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-20T18:45:25.800 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 bash[32488]: 39a1eb744183509762a450d7f8d4dd0244d65c54e86fa5e1e8f0335a4524b998 2024-09-20T18:45:25.800 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:25 smithi142 systemd[1]: Started Ceph mon.b for 1ed5a50a-7780-11ef-baf5-efdc52797490. 2024-09-20T18:45:25.850 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-20T18:45:25.851 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-20T18:45:26.105 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:45:30.732 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:30 smithi083 ceph-mon[29421]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:30.733 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:30 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-20T18:45:30.733 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:30 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-20T18:45:30.733 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:30 smithi083 ceph-mon[29421]: mon.a calling monitor election 2024-09-20T18:45:30.733 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:30 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-20T18:45:30.733 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:30 smithi083 ceph-mon[29421]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:30.733 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:30 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-20T18:45:30.734 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:30 smithi083 ceph-mon[29421]: mon.b calling monitor election 2024-09-20T18:45:30.734 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:30 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-20T18:45:30.734 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:30 smithi083 ceph-mon[29421]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:30.734 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:30 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-20T18:45:30.734 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:30 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-20T18:45:30.734 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:30 smithi083 ceph-mon[29421]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-20T18:45:30.735 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:30 smithi083 ceph-mon[29421]: monmap epoch 2 2024-09-20T18:45:30.735 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:30 smithi083 ceph-mon[29421]: fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 2024-09-20T18:45:30.735 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:30 smithi083 ceph-mon[29421]: last_changed 2024-09-20T18:45:25.297840+0000 2024-09-20T18:45:30.735 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:30 smithi083 ceph-mon[29421]: created 2024-09-20T18:44:06.997446+0000 2024-09-20T18:45:30.735 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:30 smithi083 ceph-mon[29421]: min_mon_release 19 (squid) 2024-09-20T18:45:30.735 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:30 smithi083 ceph-mon[29421]: election_strategy: 1 2024-09-20T18:45:30.736 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:30 smithi083 ceph-mon[29421]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-09-20T18:45:30.736 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:30 smithi083 ceph-mon[29421]: 1: [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon.b 2024-09-20T18:45:30.736 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:30 smithi083 ceph-mon[29421]: fsmap 2024-09-20T18:45:30.736 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:30 smithi083 ceph-mon[29421]: osdmap e4: 0 total, 0 up, 0 in 2024-09-20T18:45:30.736 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:30 smithi083 ceph-mon[29421]: mgrmap e13: y(active, since 33s) 2024-09-20T18:45:30.736 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:30 smithi083 ceph-mon[29421]: overall HEALTH_OK 2024-09-20T18:45:30.736 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:30 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:30.737 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:30 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:30.737 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:30 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:30.737 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:30 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-20T18:45:30.737 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:30 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:45:30.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:30 smithi142 ceph-mon[32508]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:30.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:30 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-20T18:45:30.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:30 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-20T18:45:30.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:30 smithi142 ceph-mon[32508]: mon.a calling monitor election 2024-09-20T18:45:30.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:30 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-20T18:45:30.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:30 smithi142 ceph-mon[32508]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:30.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:30 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-20T18:45:30.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:30 smithi142 ceph-mon[32508]: mon.b calling monitor election 2024-09-20T18:45:30.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:30 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-20T18:45:30.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:30 smithi142 ceph-mon[32508]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:30.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:30 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-20T18:45:30.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:30 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-20T18:45:30.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:30 smithi142 ceph-mon[32508]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-20T18:45:30.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:30 smithi142 ceph-mon[32508]: monmap epoch 2 2024-09-20T18:45:30.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:30 smithi142 ceph-mon[32508]: fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 2024-09-20T18:45:30.775 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:30 smithi142 ceph-mon[32508]: last_changed 2024-09-20T18:45:25.297840+0000 2024-09-20T18:45:30.775 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:30 smithi142 ceph-mon[32508]: created 2024-09-20T18:44:06.997446+0000 2024-09-20T18:45:30.775 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:30 smithi142 ceph-mon[32508]: min_mon_release 19 (squid) 2024-09-20T18:45:30.775 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:30 smithi142 ceph-mon[32508]: election_strategy: 1 2024-09-20T18:45:30.775 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:30 smithi142 ceph-mon[32508]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-09-20T18:45:30.775 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:30 smithi142 ceph-mon[32508]: 1: [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon.b 2024-09-20T18:45:30.775 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:30 smithi142 ceph-mon[32508]: fsmap 2024-09-20T18:45:30.775 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:30 smithi142 ceph-mon[32508]: osdmap e4: 0 total, 0 up, 0 in 2024-09-20T18:45:30.775 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:30 smithi142 ceph-mon[32508]: mgrmap e13: y(active, since 33s) 2024-09-20T18:45:30.775 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:30 smithi142 ceph-mon[32508]: overall HEALTH_OK 2024-09-20T18:45:30.775 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:30 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:30.775 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:30 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:30.775 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:30 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:30.775 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:30 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-20T18:45:30.776 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:30 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:45:31.220 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:45:31.220 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":2,"fsid":"1ed5a50a-7780-11ef-baf5-efdc52797490","modified":"2024-09-20T18:45:25.297840Z","created":"2024-09-20T18:44:06.997446Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:3300","nonce":0},{"type":"v1","addr":"172.21.15.83:6789","nonce":0}]},"addr":"172.21.15.83:6789/0","public_addr":"172.21.15.83:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:3300","nonce":0},{"type":"v1","addr":"172.21.15.142:6789","nonce":0}]},"addr":"172.21.15.142:6789/0","public_addr":"172.21.15.142:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-20T18:45:31.220 INFO:teuthology.orchestra.run.smithi142.stderr:dumped monmap epoch 2 2024-09-20T18:45:31.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:31 smithi142 ceph-mon[32508]: Deploying daemon mon.c on smithi083 2024-09-20T18:45:31.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:31 smithi142 ceph-mon[32508]: from='client.? 172.21.15.142:0/2529323205' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-20T18:45:31.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:31 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-20T18:45:31.729 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:31 smithi083 ceph-mon[29421]: Deploying daemon mon.c on smithi083 2024-09-20T18:45:31.729 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:31 smithi083 ceph-mon[29421]: from='client.? 172.21.15.142:0/2529323205' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-20T18:45:31.729 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:31 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-20T18:45:32.653 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:32 smithi083 ceph-mon[29421]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:32.772 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:32 smithi142 ceph-mon[32508]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:32.801 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-20T18:45:32.801 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-20T18:45:33.055 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:45:33.297 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 systemd[1]: Starting Ceph mon.c for 1ed5a50a-7780-11ef-baf5-efdc52797490... 2024-09-20T18:45:33.553 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 podman[36242]: 2024-09-20 18:45:33.18357762 +0000 UTC m=+0.015528739 image pull 441f9a3fe51528ef653b21f42e9d884db6124a0c7d482e865382bdfdb9ec979c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2 2024-09-20T18:45:33.554 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 podman[36242]: 2024-09-20 18:45:33.281557567 +0000 UTC m=+0.113508677 container create 2837de37301cefc1c8893b4c3061773c68fd77c27d285be424ef5238bfda0e64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mon-c, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS) 2024-09-20T18:45:34.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 podman[36242]: 2024-09-20 18:45:33.559294928 +0000 UTC m=+0.391246037 container init 2837de37301cefc1c8893b4c3061773c68fd77c27d285be424ef5238bfda0e64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mon-c, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-20T18:45:34.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 podman[36242]: 2024-09-20 18:45:33.562461714 +0000 UTC m=+0.394412824 container start 2837de37301cefc1c8893b4c3061773c68fd77c27d285be424ef5238bfda0e64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mon-c, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-guits-main-2024-09-20-1028-88deadb) 2024-09-20T18:45:34.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: set uid:gid to 167:167 (ceph:ceph) 2024-09-20T18:45:34.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: ceph version 19.3.0-5099-g88deadb8 (88deadb8082f4a33832e228ccefc59cedba158a5) squid (dev), process ceph-mon, pid 2 2024-09-20T18:45:34.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: pidfile_write: ignore empty --pid-file 2024-09-20T18:45:34.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: load: jerasure load: lrc 2024-09-20T18:45:34.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: RocksDB version: 7.9.2 2024-09-20T18:45:34.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Git sha 0 2024-09-20T18:45:34.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Compile date 2024-09-20 10:48:39 2024-09-20T18:45:34.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: DB SUMMARY 2024-09-20T18:45:34.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: DB Session ID: 9OJMUPADUST4YA8SKT0M 2024-09-20T18:45:34.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: CURRENT file: CURRENT 2024-09-20T18:45:34.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: IDENTITY file: IDENTITY 2024-09-20T18:45:34.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-20T18:45:34.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-20T18:45:34.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 636 ; 2024-09-20T18:45:34.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.error_if_exists: 0 2024-09-20T18:45:34.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.create_if_missing: 0 2024-09-20T18:45:34.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.paranoid_checks: 1 2024-09-20T18:45:34.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-20T18:45:34.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-20T18:45:34.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-20T18:45:34.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.env: 0x5603d4f52120 2024-09-20T18:45:34.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.fs: PosixFileSystem 2024-09-20T18:45:34.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.info_log: 0x5603d64f5700 2024-09-20T18:45:34.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.max_file_opening_threads: 16 2024-09-20T18:45:34.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.statistics: (nil) 2024-09-20T18:45:34.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.use_fsync: 0 2024-09-20T18:45:34.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.max_log_file_size: 0 2024-09-20T18:45:34.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-20T18:45:34.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-20T18:45:34.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.keep_log_file_num: 1000 2024-09-20T18:45:34.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.recycle_log_file_num: 0 2024-09-20T18:45:34.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.allow_fallocate: 1 2024-09-20T18:45:34.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.allow_mmap_reads: 0 2024-09-20T18:45:34.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.allow_mmap_writes: 0 2024-09-20T18:45:34.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.use_direct_reads: 0 2024-09-20T18:45:34.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-20T18:45:34.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.create_missing_column_families: 0 2024-09-20T18:45:34.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.db_log_dir: 2024-09-20T18:45:34.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.wal_dir: 2024-09-20T18:45:34.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-20T18:45:34.057 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-20T18:45:34.057 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-20T18:45:34.057 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-20T18:45:34.057 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-20T18:45:34.057 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-20T18:45:34.057 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.advise_random_on_open: 1 2024-09-20T18:45:34.057 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.db_write_buffer_size: 0 2024-09-20T18:45:34.057 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.write_buffer_manager: 0x5603d64f9b80 2024-09-20T18:45:34.057 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-20T18:45:34.057 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-20T18:45:34.057 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-20T18:45:34.057 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.rate_limiter: (nil) 2024-09-20T18:45:34.057 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-20T18:45:34.057 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.wal_recovery_mode: 2 2024-09-20T18:45:34.058 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.enable_thread_tracking: 0 2024-09-20T18:45:34.058 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.enable_pipelined_write: 0 2024-09-20T18:45:34.058 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.unordered_write: 0 2024-09-20T18:45:34.058 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-20T18:45:34.058 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-20T18:45:34.058 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-20T18:45:34.058 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-20T18:45:34.058 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.row_cache: None 2024-09-20T18:45:34.058 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.wal_filter: None 2024-09-20T18:45:34.059 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-20T18:45:34.059 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.allow_ingest_behind: 0 2024-09-20T18:45:34.060 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.two_write_queues: 0 2024-09-20T18:45:34.060 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.manual_wal_flush: 0 2024-09-20T18:45:34.060 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.wal_compression: 0 2024-09-20T18:45:34.060 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.atomic_flush: 0 2024-09-20T18:45:34.060 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-20T18:45:34.060 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-20T18:45:34.060 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-20T18:45:34.060 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.log_readahead_size: 0 2024-09-20T18:45:34.060 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-20T18:45:34.060 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.best_efforts_recovery: 0 2024-09-20T18:45:34.060 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-20T18:45:34.060 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-20T18:45:34.060 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.allow_data_in_errors: 0 2024-09-20T18:45:34.060 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.db_host_id: __hostname__ 2024-09-20T18:45:34.061 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-20T18:45:34.061 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.max_background_jobs: 2 2024-09-20T18:45:34.061 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.max_background_compactions: -1 2024-09-20T18:45:34.061 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.max_subcompactions: 1 2024-09-20T18:45:34.061 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-20T18:45:34.061 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-20T18:45:34.061 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-20T18:45:34.061 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.max_total_wal_size: 0 2024-09-20T18:45:34.061 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-20T18:45:34.061 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-20T18:45:34.061 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-20T18:45:34.061 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-20T18:45:34.061 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.max_open_files: -1 2024-09-20T18:45:34.062 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.bytes_per_sync: 0 2024-09-20T18:45:34.062 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-20T18:45:34.062 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-20T18:45:34.062 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.compaction_readahead_size: 0 2024-09-20T18:45:34.062 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.max_background_flushes: -1 2024-09-20T18:45:34.062 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Compression algorithms supported: 2024-09-20T18:45:34.062 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: kZSTD supported: 0 2024-09-20T18:45:34.062 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: kXpressCompression supported: 0 2024-09-20T18:45:34.062 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: kBZip2Compression supported: 0 2024-09-20T18:45:34.062 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-20T18:45:34.062 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: kLZ4Compression supported: 1 2024-09-20T18:45:34.062 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: kZlibCompression supported: 1 2024-09-20T18:45:34.062 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: kLZ4HCCompression supported: 1 2024-09-20T18:45:34.063 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: kSnappyCompression supported: 1 2024-09-20T18:45:34.063 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-20T18:45:34.063 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-20T18:45:34.063 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-20T18:45:34.063 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-20T18:45:34.063 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-20T18:45:34.063 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.merge_operator: 2024-09-20T18:45:34.063 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.compaction_filter: None 2024-09-20T18:45:34.063 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.compaction_filter_factory: None 2024-09-20T18:45:34.063 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.sst_partitioner_factory: None 2024-09-20T18:45:34.063 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-20T18:45:34.063 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-20T18:45:34.063 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5603d64f5320) 2024-09-20T18:45:34.064 INFO:journalctl@ceph.mon.c.smithi083.stdout: cache_index_and_filter_blocks: 1 2024-09-20T18:45:34.064 INFO:journalctl@ceph.mon.c.smithi083.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-20T18:45:34.064 INFO:journalctl@ceph.mon.c.smithi083.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-20T18:45:34.064 INFO:journalctl@ceph.mon.c.smithi083.stdout: pin_top_level_index_and_filter: 1 2024-09-20T18:45:34.064 INFO:journalctl@ceph.mon.c.smithi083.stdout: index_type: 0 2024-09-20T18:45:34.064 INFO:journalctl@ceph.mon.c.smithi083.stdout: data_block_index_type: 0 2024-09-20T18:45:34.064 INFO:journalctl@ceph.mon.c.smithi083.stdout: index_shortening: 1 2024-09-20T18:45:34.064 INFO:journalctl@ceph.mon.c.smithi083.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-20T18:45:34.064 INFO:journalctl@ceph.mon.c.smithi083.stdout: checksum: 4 2024-09-20T18:45:34.064 INFO:journalctl@ceph.mon.c.smithi083.stdout: no_block_cache: 0 2024-09-20T18:45:34.064 INFO:journalctl@ceph.mon.c.smithi083.stdout: block_cache: 0x5603d64e9610 2024-09-20T18:45:34.064 INFO:journalctl@ceph.mon.c.smithi083.stdout: block_cache_name: BinnedLRUCache 2024-09-20T18:45:34.064 INFO:journalctl@ceph.mon.c.smithi083.stdout: block_cache_options: 2024-09-20T18:45:34.064 INFO:journalctl@ceph.mon.c.smithi083.stdout: capacity : 536870912 2024-09-20T18:45:34.064 INFO:journalctl@ceph.mon.c.smithi083.stdout: num_shard_bits : 4 2024-09-20T18:45:34.065 INFO:journalctl@ceph.mon.c.smithi083.stdout: strict_capacity_limit : 0 2024-09-20T18:45:34.065 INFO:journalctl@ceph.mon.c.smithi083.stdout: high_pri_pool_ratio: 0.000 2024-09-20T18:45:34.065 INFO:journalctl@ceph.mon.c.smithi083.stdout: block_cache_compressed: (nil) 2024-09-20T18:45:34.065 INFO:journalctl@ceph.mon.c.smithi083.stdout: persistent_cache: (nil) 2024-09-20T18:45:34.065 INFO:journalctl@ceph.mon.c.smithi083.stdout: block_size: 4096 2024-09-20T18:45:34.065 INFO:journalctl@ceph.mon.c.smithi083.stdout: block_size_deviation: 10 2024-09-20T18:45:34.065 INFO:journalctl@ceph.mon.c.smithi083.stdout: block_restart_interval: 16 2024-09-20T18:45:34.065 INFO:journalctl@ceph.mon.c.smithi083.stdout: index_block_restart_interval: 1 2024-09-20T18:45:34.065 INFO:journalctl@ceph.mon.c.smithi083.stdout: metadata_block_size: 4096 2024-09-20T18:45:34.065 INFO:journalctl@ceph.mon.c.smithi083.stdout: partition_filters: 0 2024-09-20T18:45:34.065 INFO:journalctl@ceph.mon.c.smithi083.stdout: use_delta_encoding: 1 2024-09-20T18:45:34.065 INFO:journalctl@ceph.mon.c.smithi083.stdout: filter_policy: bloomfilter 2024-09-20T18:45:34.065 INFO:journalctl@ceph.mon.c.smithi083.stdout: whole_key_filtering: 1 2024-09-20T18:45:34.065 INFO:journalctl@ceph.mon.c.smithi083.stdout: verify_compression: 0 2024-09-20T18:45:34.065 INFO:journalctl@ceph.mon.c.smithi083.stdout: read_amp_bytes_per_bit: 0 2024-09-20T18:45:34.065 INFO:journalctl@ceph.mon.c.smithi083.stdout: format_version: 5 2024-09-20T18:45:34.066 INFO:journalctl@ceph.mon.c.smithi083.stdout: enable_index_compression: 1 2024-09-20T18:45:34.066 INFO:journalctl@ceph.mon.c.smithi083.stdout: block_align: 0 2024-09-20T18:45:34.066 INFO:journalctl@ceph.mon.c.smithi083.stdout: max_auto_readahead_size: 262144 2024-09-20T18:45:34.066 INFO:journalctl@ceph.mon.c.smithi083.stdout: prepopulate_block_cache: 0 2024-09-20T18:45:34.066 INFO:journalctl@ceph.mon.c.smithi083.stdout: initial_auto_readahead_size: 8192 2024-09-20T18:45:34.066 INFO:journalctl@ceph.mon.c.smithi083.stdout: num_file_reads_for_auto_readahead: 2 2024-09-20T18:45:34.066 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.write_buffer_size: 33554432 2024-09-20T18:45:34.066 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.max_write_buffer_number: 2 2024-09-20T18:45:34.066 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.compression: NoCompression 2024-09-20T18:45:34.066 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.bottommost_compression: Disabled 2024-09-20T18:45:34.066 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.prefix_extractor: nullptr 2024-09-20T18:45:34.066 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-20T18:45:34.066 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.num_levels: 7 2024-09-20T18:45:34.066 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-20T18:45:34.067 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-20T18:45:34.067 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-20T18:45:34.067 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-20T18:45:34.067 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-20T18:45:34.067 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-20T18:45:34.067 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-20T18:45:34.067 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-20T18:45:34.067 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-20T18:45:34.067 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-20T18:45:34.069 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-20T18:45:34.069 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-20T18:45:34.069 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-20T18:45:34.069 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.compression_opts.level: 32767 2024-09-20T18:45:34.069 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.compression_opts.strategy: 0 2024-09-20T18:45:34.070 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-20T18:45:34.070 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-20T18:45:34.070 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-20T18:45:34.070 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-20T18:45:34.070 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.compression_opts.enabled: false 2024-09-20T18:45:34.070 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-20T18:45:34.070 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-20T18:45:34.070 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-20T18:45:34.070 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-20T18:45:34.070 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.target_file_size_base: 67108864 2024-09-20T18:45:34.070 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-20T18:45:34.070 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-20T18:45:34.070 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-20T18:45:34.070 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-20T18:45:34.071 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-20T18:45:34.071 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-20T18:45:34.071 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-20T18:45:34.071 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-20T18:45:34.071 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-20T18:45:34.071 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-20T18:45:34.071 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-20T18:45:34.071 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-20T18:45:34.071 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-20T18:45:34.071 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-20T18:45:34.071 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.arena_block_size: 1048576 2024-09-20T18:45:34.071 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-20T18:45:34.071 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-20T18:45:34.071 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.disable_auto_compactions: 0 2024-09-20T18:45:34.072 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-20T18:45:34.072 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-20T18:45:34.072 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-20T18:45:34.072 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-20T18:45:34.072 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-20T18:45:34.072 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-20T18:45:34.072 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-20T18:45:34.072 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-20T18:45:34.072 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-20T18:45:34.072 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-20T18:45:34.072 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-20T18:45:34.072 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.inplace_update_support: 0 2024-09-20T18:45:34.072 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-20T18:45:34.072 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-20T18:45:34.073 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-20T18:45:34.073 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-20T18:45:34.073 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.bloom_locality: 0 2024-09-20T18:45:34.073 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.max_successive_merges: 0 2024-09-20T18:45:34.073 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-20T18:45:34.073 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.paranoid_file_checks: 0 2024-09-20T18:45:34.073 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.force_consistency_checks: 1 2024-09-20T18:45:34.073 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.report_bg_io_stats: 0 2024-09-20T18:45:34.073 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.ttl: 2592000 2024-09-20T18:45:34.073 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-20T18:45:34.073 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-20T18:45:34.073 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-20T18:45:34.073 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.enable_blob_files: false 2024-09-20T18:45:34.073 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.min_blob_size: 0 2024-09-20T18:45:34.073 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.blob_file_size: 268435456 2024-09-20T18:45:34.074 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-20T18:45:34.074 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-20T18:45:34.074 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-20T18:45:34.074 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-20T18:45:34.074 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-20T18:45:34.074 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.blob_file_starting_level: 0 2024-09-20T18:45:34.074 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-20T18:45:34.074 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-20T18:45:34.074 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-20T18:45:34.074 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 7265a6a0-ceba-4a55-ad50-b51d7f186f91 2024-09-20T18:45:34.074 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726857933590233, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-20T18:45:34.074 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-20T18:45:34.074 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726857933590734, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726857933, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "7265a6a0-ceba-4a55-ad50-b51d7f186f91", "db_session_id": "9OJMUPADUST4YA8SKT0M", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-20T18:45:34.074 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726857933590867, "job": 1, "event": "recovery_finished"} 2024-09-20T18:45:34.075 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-20T18:45:34.075 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-20T18:45:34.075 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5603d6518e00 2024-09-20T18:45:34.075 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: DB pointer 0x5603d661a000 2024-09-20T18:45:34.075 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-20T18:45:34.075 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-20T18:45:34.075 INFO:journalctl@ceph.mon.c.smithi083.stdout: ** DB Stats ** 2024-09-20T18:45:34.075 INFO:journalctl@ceph.mon.c.smithi083.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-20T18:45:34.075 INFO:journalctl@ceph.mon.c.smithi083.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-20T18:45:34.075 INFO:journalctl@ceph.mon.c.smithi083.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-20T18:45:34.075 INFO:journalctl@ceph.mon.c.smithi083.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-20T18:45:34.075 INFO:journalctl@ceph.mon.c.smithi083.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-20T18:45:34.075 INFO:journalctl@ceph.mon.c.smithi083.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-20T18:45:34.075 INFO:journalctl@ceph.mon.c.smithi083.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-20T18:45:34.076 INFO:journalctl@ceph.mon.c.smithi083.stdout: 2024-09-20T18:45:34.076 INFO:journalctl@ceph.mon.c.smithi083.stdout: ** Compaction Stats [default] ** 2024-09-20T18:45:34.076 INFO:journalctl@ceph.mon.c.smithi083.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-20T18:45:34.076 INFO:journalctl@ceph.mon.c.smithi083.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-20T18:45:34.076 INFO:journalctl@ceph.mon.c.smithi083.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-20T18:45:34.076 INFO:journalctl@ceph.mon.c.smithi083.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-20T18:45:34.076 INFO:journalctl@ceph.mon.c.smithi083.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-20T18:45:34.076 INFO:journalctl@ceph.mon.c.smithi083.stdout: 2024-09-20T18:45:34.076 INFO:journalctl@ceph.mon.c.smithi083.stdout: ** Compaction Stats [default] ** 2024-09-20T18:45:34.076 INFO:journalctl@ceph.mon.c.smithi083.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-20T18:45:34.076 INFO:journalctl@ceph.mon.c.smithi083.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-20T18:45:34.076 INFO:journalctl@ceph.mon.c.smithi083.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-20T18:45:34.076 INFO:journalctl@ceph.mon.c.smithi083.stdout: 2024-09-20T18:45:34.076 INFO:journalctl@ceph.mon.c.smithi083.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-20T18:45:34.076 INFO:journalctl@ceph.mon.c.smithi083.stdout: 2024-09-20T18:45:34.077 INFO:journalctl@ceph.mon.c.smithi083.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-20T18:45:34.077 INFO:journalctl@ceph.mon.c.smithi083.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-20T18:45:34.077 INFO:journalctl@ceph.mon.c.smithi083.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-20T18:45:34.077 INFO:journalctl@ceph.mon.c.smithi083.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-20T18:45:34.077 INFO:journalctl@ceph.mon.c.smithi083.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-20T18:45:34.077 INFO:journalctl@ceph.mon.c.smithi083.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-20T18:45:34.077 INFO:journalctl@ceph.mon.c.smithi083.stdout: Cumulative compaction: 0.00 GB write, 0.49 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-20T18:45:34.077 INFO:journalctl@ceph.mon.c.smithi083.stdout: Interval compaction: 0.00 GB write, 0.49 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-20T18:45:34.077 INFO:journalctl@ceph.mon.c.smithi083.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-20T18:45:34.077 INFO:journalctl@ceph.mon.c.smithi083.stdout: Block cache BinnedLRUCache@0x5603d64e9610#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-09-20T18:45:34.077 INFO:journalctl@ceph.mon.c.smithi083.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-20T18:45:34.077 INFO:journalctl@ceph.mon.c.smithi083.stdout: 2024-09-20T18:45:34.077 INFO:journalctl@ceph.mon.c.smithi083.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-20T18:45:34.077 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-20T18:45:34.078 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: using public_addrv [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] 2024-09-20T18:45:34.078 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: starting mon.c rank -1 at public addrs [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] at bind addrs [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] mon_data /var/lib/ceph/mon/ceph-c fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 2024-09-20T18:45:34.078 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: mon.c@-1(???) e0 preinit fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 2024-09-20T18:45:34.078 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: mon.c@-1(synchronizing).mds e1 new map 2024-09-20T18:45:34.078 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-20T18:45:34.078 INFO:journalctl@ceph.mon.c.smithi083.stdout: e1 2024-09-20T18:45:34.078 INFO:journalctl@ceph.mon.c.smithi083.stdout: btime 2024-09-20T18:44:10:399616+0000 2024-09-20T18:45:34.078 INFO:journalctl@ceph.mon.c.smithi083.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-20T18:45:34.078 INFO:journalctl@ceph.mon.c.smithi083.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-20T18:45:34.078 INFO:journalctl@ceph.mon.c.smithi083.stdout: legacy client fscid: -1 2024-09-20T18:45:34.078 INFO:journalctl@ceph.mon.c.smithi083.stdout: 2024-09-20T18:45:34.078 INFO:journalctl@ceph.mon.c.smithi083.stdout: No filesystems configured 2024-09-20T18:45:34.078 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-20T18:45:34.078 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-20T18:45:34.078 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-20T18:45:34.079 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-20T18:45:34.079 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-20T18:45:34.079 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-20T18:45:34.079 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-20T18:45:34.079 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-20T18:45:34.079 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-20T18:45:34.079 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-20T18:45:34.079 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:34.079 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1597988666' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-20T18:45:34.079 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1597988666' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-20T18:45:34.079 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: osdmap e4: 0 total, 0 up, 0 in 2024-09-20T18:45:34.079 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:34.079 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:34.079 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:34.080 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:34.080 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:34.080 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd/host:smithi142", "name": "osd_memory_target"} : dispatch 2024-09-20T18:45:34.080 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:45:34.080 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:45:34.080 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: Updating smithi142:/etc/ceph/ceph.conf 2024-09-20T18:45:34.080 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: Updating smithi142:/var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-20T18:45:34.080 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:34.080 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:34.080 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:34.080 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:34.080 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:45:34.081 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:45:34.081 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:45:34.081 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:34.081 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-20T18:45:34.081 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:45:34.081 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-09-20T18:45:34.081 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: Updating smithi142:/var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-20T18:45:34.081 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi083:172.21.15.83=a;smithi083:[v2:172.21.15.83:3301,v1:172.21.15.83:6790]=c;smithi142:172.21.15.142=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:45:34.081 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: Saving service mon spec with placement smithi083:172.21.15.83=a;smithi083:[v2:172.21.15.83:3301,v1:172.21.15.83:6790]=c;smithi142:172.21.15.142=b;count:3 2024-09-20T18:45:34.081 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: Deploying daemon mon.b on smithi142 2024-09-20T18:45:34.081 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:34.082 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:34.082 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: from='client.? 172.21.15.142:0/3493862986' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-20T18:45:34.082 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:34.082 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-20T18:45:34.082 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-20T18:45:34.082 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: mon.a calling monitor election 2024-09-20T18:45:34.082 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-20T18:45:34.082 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:34.082 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-20T18:45:34.082 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: mon.b calling monitor election 2024-09-20T18:45:34.082 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-20T18:45:34.082 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:34.082 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-20T18:45:34.083 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-20T18:45:34.083 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-20T18:45:34.083 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: monmap epoch 2 2024-09-20T18:45:34.083 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 2024-09-20T18:45:34.083 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: last_changed 2024-09-20T18:45:25.297840+0000 2024-09-20T18:45:34.083 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: created 2024-09-20T18:44:06.997446+0000 2024-09-20T18:45:34.083 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: min_mon_release 19 (squid) 2024-09-20T18:45:34.083 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: election_strategy: 1 2024-09-20T18:45:34.083 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-09-20T18:45:34.083 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: 1: [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon.b 2024-09-20T18:45:34.083 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: fsmap 2024-09-20T18:45:34.083 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: osdmap e4: 0 total, 0 up, 0 in 2024-09-20T18:45:34.083 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: mgrmap e13: y(active, since 33s) 2024-09-20T18:45:34.084 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: overall HEALTH_OK 2024-09-20T18:45:34.084 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:34.084 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:34.084 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:34.084 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-20T18:45:34.084 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:45:34.084 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: Deploying daemon mon.c on smithi083 2024-09-20T18:45:34.084 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: from='client.? 172.21.15.142:0/2529323205' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-20T18:45:34.084 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-20T18:45:34.084 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:34.084 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 ceph-mon[36260]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-20T18:45:34.084 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 bash[36242]: 2837de37301cefc1c8893b4c3061773c68fd77c27d285be424ef5238bfda0e64 2024-09-20T18:45:34.084 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:33 smithi083 systemd[1]: Started Ceph mon.c for 1ed5a50a-7780-11ef-baf5-efdc52797490. 2024-09-20T18:45:38.645 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:45:38.645 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":3,"fsid":"1ed5a50a-7780-11ef-baf5-efdc52797490","modified":"2024-09-20T18:45:33.616487Z","created":"2024-09-20T18:44:06.997446Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:3300","nonce":0},{"type":"v1","addr":"172.21.15.83:6789","nonce":0}]},"addr":"172.21.15.83:6789/0","public_addr":"172.21.15.83:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:3300","nonce":0},{"type":"v1","addr":"172.21.15.142:6789","nonce":0}]},"addr":"172.21.15.142:6789/0","public_addr":"172.21.15.142:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:3301","nonce":0},{"type":"v1","addr":"172.21.15.83:6790","nonce":0}]},"addr":"172.21.15.83:6790/0","public_addr":"172.21.15.83:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-20T18:45:38.645 INFO:teuthology.orchestra.run.smithi142.stderr:dumped monmap epoch 3 2024-09-20T18:45:39.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:38 smithi142 ceph-mon[32508]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:39.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:38 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-20T18:45:39.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:38 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-20T18:45:39.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:38 smithi142 ceph-mon[32508]: mon.a calling monitor election 2024-09-20T18:45:39.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:38 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-20T18:45:39.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:38 smithi142 ceph-mon[32508]: mon.b calling monitor election 2024-09-20T18:45:39.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:38 smithi142 ceph-mon[32508]: from='client.? 172.21.15.142:0/577190469' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-20T18:45:39.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:38 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-20T18:45:39.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:38 smithi142 ceph-mon[32508]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:39.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:38 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-20T18:45:39.012 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:38 smithi142 ceph-mon[32508]: mon.c calling monitor election 2024-09-20T18:45:39.012 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:38 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-20T18:45:39.012 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:38 smithi142 ceph-mon[32508]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:39.012 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:38 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-20T18:45:39.012 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:38 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-20T18:45:39.012 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:38 smithi142 ceph-mon[32508]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-20T18:45:39.012 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:38 smithi142 ceph-mon[32508]: monmap epoch 3 2024-09-20T18:45:39.012 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:38 smithi142 ceph-mon[32508]: fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 2024-09-20T18:45:39.012 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:38 smithi142 ceph-mon[32508]: last_changed 2024-09-20T18:45:33.616487+0000 2024-09-20T18:45:39.012 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:38 smithi142 ceph-mon[32508]: created 2024-09-20T18:44:06.997446+0000 2024-09-20T18:45:39.012 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:38 smithi142 ceph-mon[32508]: min_mon_release 19 (squid) 2024-09-20T18:45:39.012 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:38 smithi142 ceph-mon[32508]: election_strategy: 1 2024-09-20T18:45:39.012 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:38 smithi142 ceph-mon[32508]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-09-20T18:45:39.012 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:38 smithi142 ceph-mon[32508]: 1: [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon.b 2024-09-20T18:45:39.012 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:38 smithi142 ceph-mon[32508]: 2: [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] mon.c 2024-09-20T18:45:39.013 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:38 smithi142 ceph-mon[32508]: fsmap 2024-09-20T18:45:39.013 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:38 smithi142 ceph-mon[32508]: osdmap e4: 0 total, 0 up, 0 in 2024-09-20T18:45:39.013 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:38 smithi142 ceph-mon[32508]: mgrmap e13: y(active, since 41s) 2024-09-20T18:45:39.013 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:38 smithi142 ceph-mon[32508]: overall HEALTH_OK 2024-09-20T18:45:39.013 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:38 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:39.013 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:38 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:39.013 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:38 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:39.013 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:38 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:39.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[36260]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:39.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-20T18:45:39.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-20T18:45:39.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[36260]: mon.a calling monitor election 2024-09-20T18:45:39.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-20T18:45:39.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[36260]: mon.b calling monitor election 2024-09-20T18:45:39.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[36260]: from='client.? 172.21.15.142:0/577190469' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-20T18:45:39.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-20T18:45:39.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[36260]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:39.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-20T18:45:39.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[36260]: mon.c calling monitor election 2024-09-20T18:45:39.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-20T18:45:39.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[36260]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:39.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-20T18:45:39.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-20T18:45:39.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[36260]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-20T18:45:39.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[36260]: monmap epoch 3 2024-09-20T18:45:39.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[36260]: fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 2024-09-20T18:45:39.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[36260]: last_changed 2024-09-20T18:45:33.616487+0000 2024-09-20T18:45:39.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[36260]: created 2024-09-20T18:44:06.997446+0000 2024-09-20T18:45:39.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[36260]: min_mon_release 19 (squid) 2024-09-20T18:45:39.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[36260]: election_strategy: 1 2024-09-20T18:45:39.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[36260]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-09-20T18:45:39.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[36260]: 1: [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon.b 2024-09-20T18:45:39.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[36260]: 2: [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] mon.c 2024-09-20T18:45:39.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[36260]: fsmap 2024-09-20T18:45:39.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[36260]: osdmap e4: 0 total, 0 up, 0 in 2024-09-20T18:45:39.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[36260]: mgrmap e13: y(active, since 41s) 2024-09-20T18:45:39.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[36260]: overall HEALTH_OK 2024-09-20T18:45:39.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:39.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:39.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:39.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:39.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[29421]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:39.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-20T18:45:39.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-20T18:45:39.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[29421]: mon.a calling monitor election 2024-09-20T18:45:39.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-20T18:45:39.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[29421]: mon.b calling monitor election 2024-09-20T18:45:39.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[29421]: from='client.? 172.21.15.142:0/577190469' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-20T18:45:39.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-20T18:45:39.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[29421]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:39.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-20T18:45:39.057 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[29421]: mon.c calling monitor election 2024-09-20T18:45:39.057 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-20T18:45:39.057 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[29421]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:39.057 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-20T18:45:39.057 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-20T18:45:39.057 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[29421]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-20T18:45:39.057 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[29421]: monmap epoch 3 2024-09-20T18:45:39.057 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[29421]: fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 2024-09-20T18:45:39.057 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[29421]: last_changed 2024-09-20T18:45:33.616487+0000 2024-09-20T18:45:39.057 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[29421]: created 2024-09-20T18:44:06.997446+0000 2024-09-20T18:45:39.057 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[29421]: min_mon_release 19 (squid) 2024-09-20T18:45:39.057 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[29421]: election_strategy: 1 2024-09-20T18:45:39.057 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[29421]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-09-20T18:45:39.057 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[29421]: 1: [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon.b 2024-09-20T18:45:39.058 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[29421]: 2: [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] mon.c 2024-09-20T18:45:39.058 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[29421]: fsmap 2024-09-20T18:45:39.058 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[29421]: osdmap e4: 0 total, 0 up, 0 in 2024-09-20T18:45:39.058 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[29421]: mgrmap e13: y(active, since 41s) 2024-09-20T18:45:39.058 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[29421]: overall HEALTH_OK 2024-09-20T18:45:39.058 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:39.058 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:39.058 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:39.058 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:38 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:39.330 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-20T18:45:39.330 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph config generate-minimal-conf 2024-09-20T18:45:39.682 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:45:39.969 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:39 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:45:39.969 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:39 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-20T18:45:39.969 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:39 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:45:39.969 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:39 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-20T18:45:40.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:39 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:45:40.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:39 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-20T18:45:40.448 INFO:teuthology.orchestra.run.smithi083.stdout:# minimal ceph.conf for 1ed5a50a-7780-11ef-baf5-efdc52797490 2024-09-20T18:45:40.449 INFO:teuthology.orchestra.run.smithi083.stdout:[global] 2024-09-20T18:45:40.449 INFO:teuthology.orchestra.run.smithi083.stdout: fsid = 1ed5a50a-7780-11ef-baf5-efdc52797490 2024-09-20T18:45:40.449 INFO:teuthology.orchestra.run.smithi083.stdout: mon_host = [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] 2024-09-20T18:45:41.003 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:40 smithi083 ceph-mon[36260]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:41.003 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:40 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:41.003 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:40 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:41.003 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:40 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:41.003 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:40 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:41.003 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:40 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:45:41.004 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:40 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:45:41.004 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:40 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:41.004 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:40 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:41.004 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:40 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:41.004 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:40 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:41.004 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:40 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:41.004 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:40 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3530500634' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:45:41.004 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:40 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:41.004 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:40 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:41.004 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:40 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:41.004 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:40 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:41.004 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:40 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-20T18:45:41.004 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:40 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-20T18:45:41.004 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:40 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:45:41.005 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:40 smithi083 ceph-mon[29421]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:41.005 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:40 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:41.005 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:40 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:41.005 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:40 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:41.005 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:40 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:41.005 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:40 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:45:41.005 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:40 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:45:41.005 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:40 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:41.005 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:40 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:41.005 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:40 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:41.005 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:40 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:41.005 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:40 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:41.005 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:40 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3530500634' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:45:41.005 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:40 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:41.005 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:40 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:41.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:40 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:41.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:40 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:41.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:40 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-20T18:45:41.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:40 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-20T18:45:41.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:40 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:45:41.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:40 smithi142 ceph-mon[32508]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:41.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:40 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:41.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:40 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:41.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:40 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:41.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:40 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:41.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:40 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:45:41.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:40 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:45:41.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:40 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:41.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:40 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:41.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:40 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:41.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:40 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:41.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:40 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:41.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:40 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3530500634' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:45:41.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:40 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:41.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:40 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:41.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:40 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:41.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:40 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:41.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:40 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-20T18:45:41.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:40 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-20T18:45:41.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:40 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:45:41.031 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-20T18:45:41.031 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-20T18:45:41.031 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/ceph/ceph.conf 2024-09-20T18:45:41.058 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-20T18:45:41.058 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-20T18:45:41.124 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-20T18:45:41.124 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/ceph/ceph.conf 2024-09-20T18:45:41.151 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-20T18:45:41.151 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-20T18:45:41.216 INFO:tasks.cephadm:Adding mgr.y on smithi083 2024-09-20T18:45:41.216 INFO:tasks.cephadm:Adding mgr.x on smithi142 2024-09-20T18:45:41.216 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph orch apply mgr '2;smithi083=y;smithi142=x' 2024-09-20T18:45:41.495 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:45:41.885 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:41 smithi142 ceph-mon[32508]: Updating smithi083:/etc/ceph/ceph.conf 2024-09-20T18:45:41.885 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:41 smithi142 ceph-mon[32508]: Updating smithi142:/etc/ceph/ceph.conf 2024-09-20T18:45:41.885 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:41 smithi142 ceph-mon[32508]: Updating smithi142:/var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-20T18:45:41.885 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:41 smithi142 ceph-mon[32508]: Updating smithi083:/var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-20T18:45:41.885 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:41 smithi142 ceph-mon[32508]: Reconfiguring mon.a (unknown last config time)... 2024-09-20T18:45:41.885 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:41 smithi142 ceph-mon[32508]: Reconfiguring daemon mon.a on smithi083 2024-09-20T18:45:42.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:41 smithi083 ceph-mon[36260]: Updating smithi083:/etc/ceph/ceph.conf 2024-09-20T18:45:42.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:41 smithi083 ceph-mon[36260]: Updating smithi142:/etc/ceph/ceph.conf 2024-09-20T18:45:42.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:41 smithi083 ceph-mon[36260]: Updating smithi142:/var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-20T18:45:42.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:41 smithi083 ceph-mon[36260]: Updating smithi083:/var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-20T18:45:42.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:41 smithi083 ceph-mon[36260]: Reconfiguring mon.a (unknown last config time)... 2024-09-20T18:45:42.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:41 smithi083 ceph-mon[36260]: Reconfiguring daemon mon.a on smithi083 2024-09-20T18:45:42.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:41 smithi083 ceph-mon[29421]: Updating smithi083:/etc/ceph/ceph.conf 2024-09-20T18:45:42.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:41 smithi083 ceph-mon[29421]: Updating smithi142:/etc/ceph/ceph.conf 2024-09-20T18:45:42.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:41 smithi083 ceph-mon[29421]: Updating smithi142:/var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-20T18:45:42.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:41 smithi083 ceph-mon[29421]: Updating smithi083:/var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-20T18:45:42.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:41 smithi083 ceph-mon[29421]: Reconfiguring mon.a (unknown last config time)... 2024-09-20T18:45:42.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:41 smithi083 ceph-mon[29421]: Reconfiguring daemon mon.a on smithi083 2024-09-20T18:45:42.170 INFO:teuthology.orchestra.run.smithi142.stdout:Scheduled mgr update... 2024-09-20T18:45:42.732 DEBUG:teuthology.orchestra.run.smithi142:mgr.x> sudo journalctl -f -n 0 -u ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@mgr.x.service 2024-09-20T18:45:42.734 DEBUG:tasks.cephadm:set 0 configs 2024-09-20T18:45:42.735 INFO:tasks.cephadm:Deploying OSDs... 2024-09-20T18:45:42.735 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-20T18:45:42.735 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/scratch_devs of=/dev/stdout 2024-09-20T18:45:42.752 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-20T18:45:42.752 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_1 2024-09-20T18:45:42.808 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-20T18:45:42.808 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-20T18:45:42.808 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 984 Links: 1 2024-09-20T18:45:42.808 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-20T18:45:42.808 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-09-20T18:45:42.808 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-09-20 18:45:33.006932444 +0000 2024-09-20T18:45:42.808 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-09-20 18:40:08.831280170 +0000 2024-09-20T18:45:42.808 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-09-20 18:40:08.831280170 +0000 2024-09-20T18:45:42.808 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-09-20 18:40:08.831280170 +0000 2024-09-20T18:45:42.808 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-20T18:45:42.872 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-09-20T18:45:42.872 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-09-20T18:45:42.872 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000213797 s, 2.4 MB/s 2024-09-20T18:45:42.874 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-20T18:45:42.930 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_2 2024-09-20T18:45:42.985 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-20T18:45:42.985 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-20T18:45:42.985 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 974 Links: 1 2024-09-20T18:45:42.985 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-20T18:45:42.985 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-09-20T18:45:42.985 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-09-20 18:45:33.006932444 +0000 2024-09-20T18:45:42.985 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-09-20 18:40:08.830280368 +0000 2024-09-20T18:45:42.985 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-09-20 18:40:08.830280368 +0000 2024-09-20T18:45:42.985 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-09-20 18:40:08.830280368 +0000 2024-09-20T18:45:42.986 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-20T18:45:43.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:42 smithi142 ceph-mon[32508]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:43.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:42 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:43.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:42 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:43.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:42 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-20T18:45:43.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:42 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-20T18:45:43.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:42 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:45:43.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:42 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:43.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:42 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:43.046 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:42 smithi083 ceph-mon[36260]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:43.046 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:42 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:43.046 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:42 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:43.046 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:42 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-20T18:45:43.046 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:42 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-20T18:45:43.046 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:42 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:45:43.046 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:42 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:43.046 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:42 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:43.046 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:42 smithi083 ceph-mon[29421]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:43.046 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:42 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:43.047 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:42 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:43.047 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:42 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-20T18:45:43.047 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:42 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-20T18:45:43.047 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:42 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:45:43.047 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:42 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:43.047 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:42 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:43.048 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-09-20T18:45:43.048 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-09-20T18:45:43.048 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000282388 s, 1.8 MB/s 2024-09-20T18:45:43.050 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-20T18:45:43.106 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_3 2024-09-20T18:45:43.161 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-20T18:45:43.161 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-20T18:45:43.161 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 971 Links: 1 2024-09-20T18:45:43.161 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-20T18:45:43.162 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-09-20T18:45:43.162 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-09-20 18:45:33.007932422 +0000 2024-09-20T18:45:43.162 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-09-20 18:40:08.830280368 +0000 2024-09-20T18:45:43.162 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-09-20 18:40:08.830280368 +0000 2024-09-20T18:45:43.162 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-09-20 18:40:08.830280368 +0000 2024-09-20T18:45:43.162 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-20T18:45:43.224 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-09-20T18:45:43.224 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-09-20T18:45:43.224 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000190926 s, 2.7 MB/s 2024-09-20T18:45:43.225 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-20T18:45:43.292 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_4 2024-09-20T18:45:43.347 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-20T18:45:43.347 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-20T18:45:43.348 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 966 Links: 1 2024-09-20T18:45:43.348 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-20T18:45:43.348 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-09-20T18:45:43.348 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-09-20 18:45:33.007932422 +0000 2024-09-20T18:45:43.348 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-09-20 18:40:08.829280566 +0000 2024-09-20T18:45:43.348 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-09-20 18:40:08.829280566 +0000 2024-09-20T18:45:43.348 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-09-20 18:40:08.829280566 +0000 2024-09-20T18:45:43.348 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-20T18:45:43.411 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-09-20T18:45:43.411 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-09-20T18:45:43.411 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000237526 s, 2.2 MB/s 2024-09-20T18:45:43.413 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-20T18:45:43.469 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-20T18:45:43.469 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/scratch_devs of=/dev/stdout 2024-09-20T18:45:43.488 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-20T18:45:43.489 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_1 2024-09-20T18:45:43.546 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-20T18:45:43.546 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-20T18:45:43.546 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 971 Links: 1 2024-09-20T18:45:43.546 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-20T18:45:43.546 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-09-20T18:45:43.546 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-09-20 18:45:24.565210859 +0000 2024-09-20T18:45:43.546 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-09-20 18:40:09.002819957 +0000 2024-09-20T18:45:43.546 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-09-20 18:40:09.002819957 +0000 2024-09-20T18:45:43.547 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-09-20 18:40:09.002819957 +0000 2024-09-20T18:45:43.547 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-20T18:45:43.610 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-09-20T18:45:43.611 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-09-20T18:45:43.611 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000213197 s, 2.4 MB/s 2024-09-20T18:45:43.612 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-20T18:45:43.668 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_2 2024-09-20T18:45:43.724 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-20T18:45:43.724 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-20T18:45:43.724 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 981 Links: 1 2024-09-20T18:45:43.724 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-20T18:45:43.724 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-09-20T18:45:43.724 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-09-20 18:45:24.565210859 +0000 2024-09-20T18:45:43.724 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-09-20 18:40:09.003819956 +0000 2024-09-20T18:45:43.724 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-09-20 18:40:09.003819956 +0000 2024-09-20T18:45:43.724 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-09-20 18:40:09.003819956 +0000 2024-09-20T18:45:43.725 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-20T18:45:43.789 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-09-20T18:45:43.789 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-09-20T18:45:43.789 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000198487 s, 2.6 MB/s 2024-09-20T18:45:43.791 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-20T18:45:43.847 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_3 2024-09-20T18:45:43.863 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:43 smithi142 ceph-mon[32508]: Reconfiguring mon.c (monmap changed)... 2024-09-20T18:45:43.863 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:43 smithi142 ceph-mon[32508]: Reconfiguring daemon mon.c on smithi083 2024-09-20T18:45:43.863 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:43 smithi142 ceph-mon[32508]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi083=y;smithi142=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:45:43.863 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:43 smithi142 ceph-mon[32508]: Saving service mgr spec with placement smithi083=y;smithi142=x;count:2 2024-09-20T18:45:43.863 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:43 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:43.863 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:43 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:43.863 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:43 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-20T18:45:43.863 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:43 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-20T18:45:43.864 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:43 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:45:43.878 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-20T18:45:43.878 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-20T18:45:43.878 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 970 Links: 1 2024-09-20T18:45:43.878 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-20T18:45:43.878 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-09-20T18:45:43.878 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-09-20 18:45:24.566210858 +0000 2024-09-20T18:45:43.879 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-09-20 18:40:09.002819957 +0000 2024-09-20T18:45:43.879 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-09-20 18:40:09.002819957 +0000 2024-09-20T18:45:43.879 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-09-20 18:40:09.002819957 +0000 2024-09-20T18:45:43.879 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-20T18:45:43.942 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-09-20T18:45:43.942 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-09-20T18:45:43.942 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000189375 s, 2.7 MB/s 2024-09-20T18:45:43.943 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-20T18:45:44.000 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_4 2024-09-20T18:45:44.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:43 smithi083 ceph-mon[29421]: Reconfiguring mon.c (monmap changed)... 2024-09-20T18:45:44.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:43 smithi083 ceph-mon[29421]: Reconfiguring daemon mon.c on smithi083 2024-09-20T18:45:44.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:43 smithi083 ceph-mon[29421]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi083=y;smithi142=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:45:44.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:43 smithi083 ceph-mon[29421]: Saving service mgr spec with placement smithi083=y;smithi142=x;count:2 2024-09-20T18:45:44.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:43 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:44.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:43 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:44.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:43 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-20T18:45:44.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:43 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-20T18:45:44.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:43 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:45:44.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:43 smithi083 ceph-mon[36260]: Reconfiguring mon.c (monmap changed)... 2024-09-20T18:45:44.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:43 smithi083 ceph-mon[36260]: Reconfiguring daemon mon.c on smithi083 2024-09-20T18:45:44.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:43 smithi083 ceph-mon[36260]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi083=y;smithi142=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:45:44.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:43 smithi083 ceph-mon[36260]: Saving service mgr spec with placement smithi083=y;smithi142=x;count:2 2024-09-20T18:45:44.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:43 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:44.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:43 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:44.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:43 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-20T18:45:44.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:43 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-20T18:45:44.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:43 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:45:44.055 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-20T18:45:44.055 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-20T18:45:44.055 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 976 Links: 1 2024-09-20T18:45:44.055 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-20T18:45:44.055 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-09-20T18:45:44.055 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-09-20 18:45:24.566210858 +0000 2024-09-20T18:45:44.056 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-09-20 18:40:09.003819956 +0000 2024-09-20T18:45:44.056 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-09-20 18:40:09.003819956 +0000 2024-09-20T18:45:44.056 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-09-20 18:40:09.003819956 +0000 2024-09-20T18:45:44.056 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-20T18:45:44.118 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-09-20T18:45:44.118 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-09-20T18:45:44.118 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000216326 s, 2.4 MB/s 2024-09-20T18:45:44.120 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-20T18:45:44.176 INFO:tasks.cephadm:Deploying osd.0 on smithi083 with /dev/vg_nvme/lv_4... 2024-09-20T18:45:44.176 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-20T18:45:44.417 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:45:44.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:44 smithi083 ceph-mon[29421]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:44.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:44 smithi083 ceph-mon[29421]: Reconfiguring mon.b (monmap changed)... 2024-09-20T18:45:44.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:44 smithi083 ceph-mon[29421]: Reconfiguring daemon mon.b on smithi142 2024-09-20T18:45:44.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:44 smithi083 ceph-mon[36260]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:44.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:44 smithi083 ceph-mon[36260]: Reconfiguring mon.b (monmap changed)... 2024-09-20T18:45:44.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:44 smithi083 ceph-mon[36260]: Reconfiguring daemon mon.b on smithi142 2024-09-20T18:45:45.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:44 smithi142 ceph-mon[32508]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:45.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:44 smithi142 ceph-mon[32508]: Reconfiguring mon.b (monmap changed)... 2024-09-20T18:45:45.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:44 smithi142 ceph-mon[32508]: Reconfiguring daemon mon.b on smithi142 2024-09-20T18:45:46.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:45 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:46.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:45 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:46.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:45 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:45:46.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:45 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:45:46.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:45 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:45:46.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:45 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:46.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:45 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-20T18:45:46.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:45 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-20T18:45:46.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:45 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-20T18:45:46.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:45 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:45:46.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:45 smithi083 ceph-mon[29421]: Deploying daemon mgr.x on smithi142 2024-09-20T18:45:46.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:45 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:46.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:45 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:46.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:45 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:45:46.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:45 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:45:46.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:45 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:45:46.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:45 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:46.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:45 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-20T18:45:46.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:45 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-20T18:45:46.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:45 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-20T18:45:46.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:45 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:45:46.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:45 smithi083 ceph-mon[36260]: Deploying daemon mgr.x on smithi142 2024-09-20T18:45:46.124 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:45 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:46.124 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:45 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:46.124 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:45 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:45:46.124 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:45 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:45:46.125 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:45 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:45:46.125 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:45 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:46.125 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:45 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-20T18:45:46.125 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:45 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-20T18:45:46.125 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:45 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-20T18:45:46.125 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:45 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:45:46.125 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:45 smithi142 ceph-mon[32508]: Deploying daemon mgr.x on smithi142 2024-09-20T18:45:46.773 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Sep 20 18:45:46 smithi142 systemd[1]: Starting Ceph mgr.x for 1ed5a50a-7780-11ef-baf5-efdc52797490... 2024-09-20T18:45:47.064 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:46 smithi142 ceph-mon[32508]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:47.064 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Sep 20 18:45:46 smithi142 podman[33878]: 2024-09-20 18:45:46.682468578 +0000 UTC m=+0.016077585 image pull 441f9a3fe51528ef653b21f42e9d884db6124a0c7d482e865382bdfdb9ec979c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2 2024-09-20T18:45:47.065 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Sep 20 18:45:46 smithi142 podman[33878]: 2024-09-20 18:45:46.793938428 +0000 UTC m=+0.127547421 container create 43e8111ef62f266b483275ea4d436cb55241141bae96655c111be6e34c313579 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-x, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-20T18:45:47.065 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Sep 20 18:45:47 smithi142 podman[33878]: 2024-09-20 18:45:47.02117203 +0000 UTC m=+0.354781024 container init 43e8111ef62f266b483275ea4d436cb55241141bae96655c111be6e34c313579 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-x, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-20T18:45:47.065 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Sep 20 18:45:47 smithi142 podman[33878]: 2024-09-20 18:45:47.024221864 +0000 UTC m=+0.357830861 container start 43e8111ef62f266b483275ea4d436cb55241141bae96655c111be6e34c313579 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-x, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-20-1028-88deadb, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-20T18:45:47.065 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Sep 20 18:45:47 smithi142 ceph-mgr[33897]: -- 172.21.15.142:0/1703216260 <== mon.1 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a7dee7b0e0 con 0x55a7dee58800 2024-09-20T18:45:47.149 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:46 smithi083 ceph-mon[29421]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:47.149 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:46 smithi083 ceph-mon[36260]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:47.152 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:45:47.180 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi083:vg_nvme/lv_4 2024-09-20T18:45:47.430 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:45:47.523 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Sep 20 18:45:47 smithi142 bash[33878]: 43e8111ef62f266b483275ea4d436cb55241141bae96655c111be6e34c313579 2024-09-20T18:45:47.523 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Sep 20 18:45:47 smithi142 systemd[1]: Started Ceph mgr.x for 1ed5a50a-7780-11ef-baf5-efdc52797490. 2024-09-20T18:45:48.523 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:48 smithi142 ceph-mon[32508]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:48.523 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:48 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:48.523 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:48 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:48.523 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:48 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:48.523 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:48 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:48.523 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:48 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:45:48.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:48 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:48.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:48 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-20T18:45:48.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:48 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-20T18:45:48.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:48 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:45:48.527 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:48 smithi083 ceph-mon[29421]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:48.527 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:48 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:48.527 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:48 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:48.527 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:48 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:48.527 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:48 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:48.528 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:48 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:45:48.528 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:48 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:48.528 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:48 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-20T18:45:48.528 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:48 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-20T18:45:48.528 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:48 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:45:48.528 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:48 smithi083 ceph-mon[36260]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:48.528 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:48 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:48.528 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:48 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:48.528 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:48 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:48.529 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:48 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:48.529 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:48 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:45:48.529 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:48 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:48.529 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:48 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-20T18:45:48.529 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:48 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-20T18:45:48.529 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:48 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:45:49.362 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:49 smithi083 ceph-mon[36260]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:45:49.363 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:49 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:49.363 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:49 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:49.363 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:49 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:45:49.363 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:49 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:45:49.363 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:49 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:49.363 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:49 smithi083 ceph-mon[36260]: Reconfiguring mgr.y (unknown last config time)... 2024-09-20T18:45:49.363 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:49 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-20T18:45:49.363 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:49 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-20T18:45:49.363 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:49 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:45:49.363 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:49 smithi083 ceph-mon[36260]: Reconfiguring daemon mgr.y on smithi083 2024-09-20T18:45:49.363 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:49 smithi083 ceph-mon[29421]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:45:49.364 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:49 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:49.364 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:49 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:49.364 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:49 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:45:49.364 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:49 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:45:49.364 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:49 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:49.364 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:49 smithi083 ceph-mon[29421]: Reconfiguring mgr.y (unknown last config time)... 2024-09-20T18:45:49.364 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:49 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-20T18:45:49.364 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:49 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-20T18:45:49.364 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:49 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:45:49.364 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:49 smithi083 ceph-mon[29421]: Reconfiguring daemon mgr.y on smithi083 2024-09-20T18:45:49.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:49 smithi142 ceph-mon[32508]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:45:49.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:49 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:49.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:49 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:49.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:49 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:45:49.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:49 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:45:49.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:49 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:49.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:49 smithi142 ceph-mon[32508]: Reconfiguring mgr.y (unknown last config time)... 2024-09-20T18:45:49.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:49 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-20T18:45:49.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:49 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-20T18:45:49.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:49 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:45:49.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:49 smithi142 ceph-mon[32508]: Reconfiguring daemon mgr.y on smithi083 2024-09-20T18:45:50.574 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:50 smithi083 ceph-mon[29421]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:50.574 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:50 smithi083 ceph-mon[36260]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:50.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:50 smithi142 ceph-mon[32508]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:51.671 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:51 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:51.671 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:51 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:51.671 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:51 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:45:51.671 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:51 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:45:51.672 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:51 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:45:51.672 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:51 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:51.672 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:51 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:51.672 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:51 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:51.672 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:51 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:45:51.672 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:51 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:45:51.672 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:51 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:45:51.672 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:51 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:51.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:51 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:51.847 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:51 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:51.847 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:51 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:45:51.847 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:51 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:45:51.847 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:51 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:45:51.848 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:51 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:45:52.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:52 smithi142 ceph-mon[32508]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:52.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:52 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2340439452' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c714e1ec-f0c3-4054-9792-61b84e53eaa8"} : dispatch 2024-09-20T18:45:52.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:52 smithi142 ceph-mon[32508]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c714e1ec-f0c3-4054-9792-61b84e53eaa8"} : dispatch 2024-09-20T18:45:52.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:52 smithi142 ceph-mon[32508]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c714e1ec-f0c3-4054-9792-61b84e53eaa8"}]': finished 2024-09-20T18:45:52.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:52 smithi142 ceph-mon[32508]: osdmap e5: 1 total, 0 up, 1 in 2024-09-20T18:45:52.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:52 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-20T18:45:52.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:52 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3969787287' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-20T18:45:52.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:52 smithi142 ceph-mon[32508]: Standby manager daemon x started 2024-09-20T18:45:52.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:52 smithi142 ceph-mon[32508]: from='mgr.? 172.21.15.142:0/2897097744' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-20T18:45:52.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:52 smithi142 ceph-mon[32508]: from='mgr.? 172.21.15.142:0/2897097744' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-20T18:45:52.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:52 smithi142 ceph-mon[32508]: from='mgr.? 172.21.15.142:0/2897097744' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-20T18:45:52.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:52 smithi142 ceph-mon[32508]: from='mgr.? 172.21.15.142:0/2897097744' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-20T18:45:52.803 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:52 smithi083 ceph-mon[36260]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:52.803 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:52 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2340439452' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c714e1ec-f0c3-4054-9792-61b84e53eaa8"} : dispatch 2024-09-20T18:45:52.803 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:52 smithi083 ceph-mon[36260]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c714e1ec-f0c3-4054-9792-61b84e53eaa8"} : dispatch 2024-09-20T18:45:52.803 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:52 smithi083 ceph-mon[36260]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c714e1ec-f0c3-4054-9792-61b84e53eaa8"}]': finished 2024-09-20T18:45:52.803 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:52 smithi083 ceph-mon[36260]: osdmap e5: 1 total, 0 up, 1 in 2024-09-20T18:45:52.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:52 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-20T18:45:52.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:52 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3969787287' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-20T18:45:52.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:52 smithi083 ceph-mon[36260]: Standby manager daemon x started 2024-09-20T18:45:52.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:52 smithi083 ceph-mon[36260]: from='mgr.? 172.21.15.142:0/2897097744' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-20T18:45:52.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:52 smithi083 ceph-mon[36260]: from='mgr.? 172.21.15.142:0/2897097744' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-20T18:45:52.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:52 smithi083 ceph-mon[36260]: from='mgr.? 172.21.15.142:0/2897097744' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-20T18:45:52.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:52 smithi083 ceph-mon[36260]: from='mgr.? 172.21.15.142:0/2897097744' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-20T18:45:52.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:52 smithi083 ceph-mon[29421]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:52.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:52 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2340439452' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c714e1ec-f0c3-4054-9792-61b84e53eaa8"} : dispatch 2024-09-20T18:45:52.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:52 smithi083 ceph-mon[29421]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c714e1ec-f0c3-4054-9792-61b84e53eaa8"} : dispatch 2024-09-20T18:45:52.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:52 smithi083 ceph-mon[29421]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c714e1ec-f0c3-4054-9792-61b84e53eaa8"}]': finished 2024-09-20T18:45:52.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:52 smithi083 ceph-mon[29421]: osdmap e5: 1 total, 0 up, 1 in 2024-09-20T18:45:52.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:52 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-20T18:45:52.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:52 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3969787287' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-20T18:45:52.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:52 smithi083 ceph-mon[29421]: Standby manager daemon x started 2024-09-20T18:45:52.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:52 smithi083 ceph-mon[29421]: from='mgr.? 172.21.15.142:0/2897097744' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-20T18:45:52.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:52 smithi083 ceph-mon[29421]: from='mgr.? 172.21.15.142:0/2897097744' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-20T18:45:52.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:52 smithi083 ceph-mon[29421]: from='mgr.? 172.21.15.142:0/2897097744' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-20T18:45:52.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:52 smithi083 ceph-mon[29421]: from='mgr.? 172.21.15.142:0/2897097744' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-20T18:45:53.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:53 smithi142 ceph-mon[32508]: mgrmap e14: y(active, since 55s), standbys: x 2024-09-20T18:45:53.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:53 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-20T18:45:53.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:53 smithi083 ceph-mon[29421]: mgrmap e14: y(active, since 55s), standbys: x 2024-09-20T18:45:53.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:53 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-20T18:45:53.803 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:53 smithi083 ceph-mon[36260]: mgrmap e14: y(active, since 55s), standbys: x 2024-09-20T18:45:53.803 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:53 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-20T18:45:54.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:54 smithi083 ceph-mon[29421]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:54.803 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:54 smithi083 ceph-mon[36260]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:55.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:54 smithi142 ceph-mon[32508]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:56.630 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:56 smithi083 ceph-mon[29421]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:56.630 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:56 smithi083 ceph-mon[36260]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:57.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:56 smithi142 ceph-mon[32508]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:58.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:58 smithi083 ceph-mon[29421]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:58.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:58 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-20T18:45:58.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:58 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:45:58.803 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:58 smithi083 ceph-mon[36260]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:58.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:58 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-20T18:45:58.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:58 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:45:59.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:58 smithi142 ceph-mon[32508]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:45:59.026 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:58 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-20T18:45:59.026 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:58 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:45:59.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:45:59 smithi083 ceph-mon[29421]: Deploying daemon osd.0 on smithi083 2024-09-20T18:45:59.803 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:45:59 smithi083 ceph-mon[36260]: Deploying daemon osd.0 on smithi083 2024-09-20T18:46:00.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:45:59 smithi142 ceph-mon[32508]: Deploying daemon osd.0 on smithi083 2024-09-20T18:46:00.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:00 smithi083 ceph-mon[29421]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:46:00.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:00 smithi083 ceph-mon[36260]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:46:01.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:00 smithi142 ceph-mon[32508]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:46:02.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:02 smithi083 ceph-mon[29421]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:46:02.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:02 smithi083 ceph-mon[36260]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:46:03.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:02 smithi142 ceph-mon[32508]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:46:04.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:04 smithi083 ceph-mon[29421]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:46:04.803 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:04 smithi083 ceph-mon[36260]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:46:05.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:04 smithi142 ceph-mon[32508]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:46:06.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:06 smithi083 ceph-mon[29421]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:46:06.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:06 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:46:06.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:06 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:06.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:06 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:06.794 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:06 smithi083 ceph-mon[36260]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:46:06.794 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:06 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:46:06.794 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:06 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:06.794 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:06 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:07.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:06 smithi142 ceph-mon[32508]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:46:07.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:06 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:46:07.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:06 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:07.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:06 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:08.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:08 smithi083 ceph-mon[29421]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:46:08.803 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:08 smithi083 ceph-mon[36260]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:46:08.825 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:08 smithi142 ceph-mon[32508]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:46:10.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:09 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:10.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:09 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:10.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:09 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:46:10.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:09 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:46:10.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:09 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:10.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:09 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:10.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:09 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:10.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:09 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:46:10.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:09 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:46:10.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:09 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:10.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:09 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:10.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:09 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:10.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:09 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:46:10.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:09 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:46:10.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:09 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:11.227 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:10 smithi083 ceph-mon[29421]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:46:11.227 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:10 smithi083 ceph-mon[36260]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:46:11.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:10 smithi142 ceph-mon[32508]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:46:11.547 INFO:teuthology.orchestra.run.smithi083.stdout:Created osd(s) 0 on host 'smithi083' 2024-09-20T18:46:12.273 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:11 smithi083 ceph-mon[36260]: from='osd.0 v2:172.21.15.83:6801/2574084049' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-20T18:46:12.273 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:11 smithi083 ceph-mon[36260]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-20T18:46:12.273 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:11 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:46:12.273 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:11 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:12.273 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:11 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:12.273 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:11 smithi083 ceph-mon[29421]: from='osd.0 v2:172.21.15.83:6801/2574084049' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-20T18:46:12.273 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:11 smithi083 ceph-mon[29421]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-20T18:46:12.274 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:11 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:46:12.274 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:11 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:12.274 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:11 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:12.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:11 smithi142 ceph-mon[32508]: from='osd.0 v2:172.21.15.83:6801/2574084049' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-20T18:46:12.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:11 smithi142 ceph-mon[32508]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-20T18:46:12.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:11 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:46:12.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:11 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:12.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:11 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:12.300 DEBUG:teuthology.orchestra.run.smithi083:osd.0> sudo journalctl -f -n 0 -u ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@osd.0.service 2024-09-20T18:46:12.302 INFO:tasks.cephadm:Deploying osd.1 on smithi083 with /dev/vg_nvme/lv_3... 2024-09-20T18:46:12.303 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-20T18:46:12.960 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:46:13.206 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:12 smithi083 ceph-mon[29421]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:46:13.206 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:12 smithi083 ceph-mon[29421]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-20T18:46:13.206 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:12 smithi083 ceph-mon[29421]: osdmap e6: 1 total, 0 up, 1 in 2024-09-20T18:46:13.206 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:12 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-20T18:46:13.206 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:12 smithi083 ceph-mon[29421]: from='osd.0 v2:172.21.15.83:6801/2574084049' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]} : dispatch 2024-09-20T18:46:13.206 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:12 smithi083 ceph-mon[29421]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]} : dispatch 2024-09-20T18:46:13.206 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:12 smithi083 ceph-mon[36260]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:46:13.206 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:12 smithi083 ceph-mon[36260]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-20T18:46:13.206 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:12 smithi083 ceph-mon[36260]: osdmap e6: 1 total, 0 up, 1 in 2024-09-20T18:46:13.207 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:12 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-20T18:46:13.207 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:12 smithi083 ceph-mon[36260]: from='osd.0 v2:172.21.15.83:6801/2574084049' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]} : dispatch 2024-09-20T18:46:13.207 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:12 smithi083 ceph-mon[36260]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]} : dispatch 2024-09-20T18:46:13.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:12 smithi142 ceph-mon[32508]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:46:13.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:12 smithi142 ceph-mon[32508]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-20T18:46:13.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:12 smithi142 ceph-mon[32508]: osdmap e6: 1 total, 0 up, 1 in 2024-09-20T18:46:13.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:12 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-20T18:46:13.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:12 smithi142 ceph-mon[32508]: from='osd.0 v2:172.21.15.83:6801/2574084049' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]} : dispatch 2024-09-20T18:46:13.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:12 smithi142 ceph-mon[32508]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]} : dispatch 2024-09-20T18:46:13.461 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 18:46:13 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-0[39573]: 2024-09-20T18:46:13.323+0000 7f75c2045640 -1 osd.0 0 waiting for initial osdmap 2024-09-20T18:46:13.461 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 18:46:13 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-0[39573]: 2024-09-20T18:46:13.325+0000 7f75b9e64640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-20T18:46:14.153 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:13 smithi083 ceph-mon[29421]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-09-20T18:46:14.153 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:13 smithi083 ceph-mon[29421]: osdmap e7: 1 total, 0 up, 1 in 2024-09-20T18:46:14.153 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:13 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-20T18:46:14.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:13 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-20T18:46:14.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:13 smithi083 ceph-mon[29421]: from='osd.0 ' entity='osd.0' 2024-09-20T18:46:14.154 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:13 smithi083 ceph-mon[36260]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-09-20T18:46:14.154 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:13 smithi083 ceph-mon[36260]: osdmap e7: 1 total, 0 up, 1 in 2024-09-20T18:46:14.154 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:13 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-20T18:46:14.154 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:13 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-20T18:46:14.154 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:13 smithi083 ceph-mon[36260]: from='osd.0 ' entity='osd.0' 2024-09-20T18:46:14.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:13 smithi142 ceph-mon[32508]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-09-20T18:46:14.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:13 smithi142 ceph-mon[32508]: osdmap e7: 1 total, 0 up, 1 in 2024-09-20T18:46:14.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:13 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-20T18:46:14.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:13 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-20T18:46:14.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:13 smithi142 ceph-mon[32508]: from='osd.0 ' entity='osd.0' 2024-09-20T18:46:15.269 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:14 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:46:15.269 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:14 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:46:15.270 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:14 smithi083 ceph-mon[29421]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:46:15.270 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:14 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-20T18:46:15.270 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:14 smithi083 ceph-mon[29421]: osd.0 v2:172.21.15.83:6801/2574084049 boot 2024-09-20T18:46:15.270 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:14 smithi083 ceph-mon[29421]: osdmap e8: 1 total, 1 up, 1 in 2024-09-20T18:46:15.270 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:14 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-20T18:46:15.270 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:14 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:46:15.270 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:14 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:46:15.270 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:14 smithi083 ceph-mon[36260]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:46:15.270 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:14 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-20T18:46:15.270 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:14 smithi083 ceph-mon[36260]: osd.0 v2:172.21.15.83:6801/2574084049 boot 2024-09-20T18:46:15.270 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:14 smithi083 ceph-mon[36260]: osdmap e8: 1 total, 1 up, 1 in 2024-09-20T18:46:15.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:14 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-20T18:46:15.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:14 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:46:15.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:14 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:46:15.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:14 smithi142 ceph-mon[32508]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T18:46:15.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:14 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-20T18:46:15.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:14 smithi142 ceph-mon[32508]: osd.0 v2:172.21.15.83:6801/2574084049 boot 2024-09-20T18:46:15.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:14 smithi142 ceph-mon[32508]: osdmap e8: 1 total, 1 up, 1 in 2024-09-20T18:46:15.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:14 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-20T18:46:16.961 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:16 smithi083 ceph-mon[29421]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-20T18:46:16.961 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:16 smithi083 ceph-mon[29421]: Detected new or changed devices on smithi083 2024-09-20T18:46:16.961 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:16 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:16.961 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:16 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:16.961 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:16 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-20T18:46:16.961 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:16 smithi083 ceph-mon[29421]: Adjusting osd_memory_target on smithi083 to 16064M 2024-09-20T18:46:16.961 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:16 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:16.962 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:16 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:46:16.962 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:16 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:46:16.962 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:16 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:16.962 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:16 smithi083 ceph-mon[29421]: osdmap e9: 1 total, 1 up, 1 in 2024-09-20T18:46:16.962 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:16 smithi083 ceph-mon[36260]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-20T18:46:16.962 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:16 smithi083 ceph-mon[36260]: Detected new or changed devices on smithi083 2024-09-20T18:46:16.962 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:16 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:16.962 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:16 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:16.962 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:16 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-20T18:46:16.962 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:16 smithi083 ceph-mon[36260]: Adjusting osd_memory_target on smithi083 to 16064M 2024-09-20T18:46:16.962 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:16 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:16.963 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:16 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:46:16.963 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:16 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:46:16.963 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:16 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:16.963 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:16 smithi083 ceph-mon[36260]: osdmap e9: 1 total, 1 up, 1 in 2024-09-20T18:46:17.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:16 smithi142 ceph-mon[32508]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-20T18:46:17.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:16 smithi142 ceph-mon[32508]: Detected new or changed devices on smithi083 2024-09-20T18:46:17.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:16 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:17.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:16 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:17.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:16 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-20T18:46:17.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:16 smithi142 ceph-mon[32508]: Adjusting osd_memory_target on smithi083 to 16064M 2024-09-20T18:46:17.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:16 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:17.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:16 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:46:17.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:16 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:46:17.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:16 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:17.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:16 smithi142 ceph-mon[32508]: osdmap e9: 1 total, 1 up, 1 in 2024-09-20T18:46:17.240 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:46:17.274 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi083:vg_nvme/lv_3 2024-09-20T18:46:17.519 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:46:18.918 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:18 smithi083 ceph-mon[36260]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-20T18:46:18.918 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:18 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-20T18:46:18.918 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:18 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-20T18:46:18.918 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:18 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:46:18.919 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:18 smithi083 ceph-mon[29421]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-20T18:46:18.919 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:18 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-20T18:46:18.919 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:18 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-20T18:46:18.919 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:18 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:46:19.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:18 smithi142 ceph-mon[32508]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-20T18:46:19.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:18 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-20T18:46:19.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:18 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-20T18:46:19.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:18 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:46:19.795 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:19 smithi083 ceph-mon[29421]: from='client.14247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:46:19.796 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:19 smithi083 ceph-mon[36260]: from='client.14247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:46:20.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:19 smithi142 ceph-mon[32508]: from='client.14247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:46:21.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:20 smithi142 ceph-mon[32508]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-20T18:46:21.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:20 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/279780678' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9dc0f2a8-bafa-4a42-88f5-0301dd58f78d"} : dispatch 2024-09-20T18:46:21.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:20 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/279780678' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9dc0f2a8-bafa-4a42-88f5-0301dd58f78d"}]': finished 2024-09-20T18:46:21.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:20 smithi142 ceph-mon[32508]: osdmap e10: 2 total, 1 up, 2 in 2024-09-20T18:46:21.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:20 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-20T18:46:21.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:20 smithi083 ceph-mon[29421]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-20T18:46:21.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:20 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/279780678' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9dc0f2a8-bafa-4a42-88f5-0301dd58f78d"} : dispatch 2024-09-20T18:46:21.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:20 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/279780678' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9dc0f2a8-bafa-4a42-88f5-0301dd58f78d"}]': finished 2024-09-20T18:46:21.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:20 smithi083 ceph-mon[29421]: osdmap e10: 2 total, 1 up, 2 in 2024-09-20T18:46:21.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:20 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-20T18:46:21.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:20 smithi083 ceph-mon[36260]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-20T18:46:21.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:20 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/279780678' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9dc0f2a8-bafa-4a42-88f5-0301dd58f78d"} : dispatch 2024-09-20T18:46:21.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:20 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/279780678' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9dc0f2a8-bafa-4a42-88f5-0301dd58f78d"}]': finished 2024-09-20T18:46:21.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:20 smithi083 ceph-mon[36260]: osdmap e10: 2 total, 1 up, 2 in 2024-09-20T18:46:21.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:20 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-20T18:46:22.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:21 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1966068928' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-20T18:46:22.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:21 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1966068928' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-20T18:46:22.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:21 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1966068928' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-20T18:46:23.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:22 smithi142 ceph-mon[32508]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-20T18:46:23.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:22 smithi083 ceph-mon[29421]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-20T18:46:23.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:22 smithi083 ceph-mon[36260]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-20T18:46:24.956 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:24 smithi083 ceph-mon[29421]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-20T18:46:24.957 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:24 smithi083 ceph-mon[36260]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-20T18:46:25.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:24 smithi142 ceph-mon[32508]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-20T18:46:26.852 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:26 smithi083 ceph-mon[29421]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-20T18:46:26.852 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:26 smithi083 ceph-mon[36260]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-20T18:46:27.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:26 smithi142 ceph-mon[32508]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-20T18:46:28.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:28 smithi083 ceph-mon[29421]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-20T18:46:28.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:28 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-20T18:46:28.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:28 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:46:28.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:28 smithi083 ceph-mon[29421]: Deploying daemon osd.1 on smithi083 2024-09-20T18:46:28.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:28 smithi083 ceph-mon[36260]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-20T18:46:28.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:28 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-20T18:46:28.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:28 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:46:28.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:28 smithi083 ceph-mon[36260]: Deploying daemon osd.1 on smithi083 2024-09-20T18:46:29.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:28 smithi142 ceph-mon[32508]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-20T18:46:29.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:28 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-20T18:46:29.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:28 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:46:29.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:28 smithi142 ceph-mon[32508]: Deploying daemon osd.1 on smithi083 2024-09-20T18:46:30.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:30 smithi083 ceph-mon[29421]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-20T18:46:30.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:30 smithi083 ceph-mon[36260]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-20T18:46:31.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:30 smithi142 ceph-mon[32508]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-20T18:46:33.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:32 smithi142 ceph-mon[32508]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-20T18:46:33.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:32 smithi083 ceph-mon[29421]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-20T18:46:33.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:32 smithi083 ceph-mon[36260]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-20T18:46:34.762 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:34 smithi083 ceph-mon[29421]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-20T18:46:34.762 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:34 smithi083 ceph-mon[36260]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-20T18:46:35.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:34 smithi142 ceph-mon[32508]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-20T18:46:36.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:36 smithi142 ceph-mon[32508]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-20T18:46:36.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:36 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:46:36.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:36 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:36.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:36 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:36.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:36 smithi083 ceph-mon[29421]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-20T18:46:36.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:36 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:46:36.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:36 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:36.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:36 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:36.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:36 smithi083 ceph-mon[36260]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-20T18:46:36.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:36 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:46:36.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:36 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:36.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:36 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:38.803 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:38 smithi083 ceph-mon[36260]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-20T18:46:38.803 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:38 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:38.803 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:38 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:38.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:38 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:46:38.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:38 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:46:38.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:38 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:38.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:38 smithi083 ceph-mon[29421]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-20T18:46:38.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:38 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:38.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:38 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:38.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:38 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:46:38.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:38 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:46:38.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:38 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:39.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:38 smithi142 ceph-mon[32508]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-20T18:46:39.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:38 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:39.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:38 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:39.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:38 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:46:39.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:38 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:46:39.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:38 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:40.803 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:40 smithi083 ceph-mon[36260]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-20T18:46:40.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:40 smithi083 ceph-mon[29421]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-20T18:46:41.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:40 smithi142 ceph-mon[32508]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-20T18:46:41.180 INFO:teuthology.orchestra.run.smithi083.stdout:Created osd(s) 1 on host 'smithi083' 2024-09-20T18:46:41.745 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:41 smithi083 ceph-mon[29421]: from='osd.1 v2:172.21.15.83:6805/4213285798' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-20T18:46:41.745 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:41 smithi083 ceph-mon[29421]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-20T18:46:41.746 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:41 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:46:41.746 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:41 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:41.746 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:41 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:41.746 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:41 smithi083 ceph-mon[36260]: from='osd.1 v2:172.21.15.83:6805/4213285798' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-20T18:46:41.746 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:41 smithi083 ceph-mon[36260]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-20T18:46:41.746 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:41 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:46:41.746 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:41 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:41.746 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:41 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:41.771 DEBUG:teuthology.orchestra.run.smithi083:osd.1> sudo journalctl -f -n 0 -u ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@osd.1.service 2024-09-20T18:46:41.773 INFO:tasks.cephadm:Deploying osd.2 on smithi083 with /dev/vg_nvme/lv_2... 2024-09-20T18:46:41.774 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-20T18:46:42.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:41 smithi142 ceph-mon[32508]: from='osd.1 v2:172.21.15.83:6805/4213285798' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-20T18:46:42.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:41 smithi142 ceph-mon[32508]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-20T18:46:42.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:41 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:46:42.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:41 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:42.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:41 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:42.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:42 smithi083 ceph-mon[29421]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-20T18:46:42.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:42 smithi083 ceph-mon[29421]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-20T18:46:42.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:42 smithi083 ceph-mon[29421]: osdmap e11: 2 total, 1 up, 2 in 2024-09-20T18:46:42.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:42 smithi083 ceph-mon[29421]: from='osd.1 v2:172.21.15.83:6805/4213285798' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi083", "root=default"]} : dispatch 2024-09-20T18:46:42.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:42 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-20T18:46:42.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:42 smithi083 ceph-mon[29421]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi083", "root=default"]} : dispatch 2024-09-20T18:46:42.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:42 smithi083 ceph-mon[36260]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-20T18:46:42.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:42 smithi083 ceph-mon[36260]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-20T18:46:42.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:42 smithi083 ceph-mon[36260]: osdmap e11: 2 total, 1 up, 2 in 2024-09-20T18:46:42.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:42 smithi083 ceph-mon[36260]: from='osd.1 v2:172.21.15.83:6805/4213285798' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi083", "root=default"]} : dispatch 2024-09-20T18:46:42.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:42 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-20T18:46:42.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:42 smithi083 ceph-mon[36260]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi083", "root=default"]} : dispatch 2024-09-20T18:46:42.805 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 20 18:46:42 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-1[45160]: 2024-09-20T18:46:42.534+0000 7fa1f9ecf640 -1 osd.1 0 waiting for initial osdmap 2024-09-20T18:46:42.805 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 20 18:46:42 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-1[45160]: 2024-09-20T18:46:42.536+0000 7fa1f5cf6640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-20T18:46:43.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:42 smithi142 ceph-mon[32508]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-20T18:46:43.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:42 smithi142 ceph-mon[32508]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-20T18:46:43.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:42 smithi142 ceph-mon[32508]: osdmap e11: 2 total, 1 up, 2 in 2024-09-20T18:46:43.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:42 smithi142 ceph-mon[32508]: from='osd.1 v2:172.21.15.83:6805/4213285798' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi083", "root=default"]} : dispatch 2024-09-20T18:46:43.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:42 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-20T18:46:43.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:42 smithi142 ceph-mon[32508]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi083", "root=default"]} : dispatch 2024-09-20T18:46:43.487 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:46:43.842 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:43 smithi083 ceph-mon[29421]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-09-20T18:46:43.842 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:43 smithi083 ceph-mon[29421]: osdmap e12: 2 total, 1 up, 2 in 2024-09-20T18:46:43.842 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:43 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-20T18:46:43.843 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:43 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-20T18:46:43.843 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:43 smithi083 ceph-mon[36260]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-09-20T18:46:43.843 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:43 smithi083 ceph-mon[36260]: osdmap e12: 2 total, 1 up, 2 in 2024-09-20T18:46:43.843 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:43 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-20T18:46:43.843 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:43 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-20T18:46:44.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:43 smithi142 ceph-mon[32508]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-09-20T18:46:44.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:43 smithi142 ceph-mon[32508]: osdmap e12: 2 total, 1 up, 2 in 2024-09-20T18:46:44.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:43 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-20T18:46:44.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:43 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-20T18:46:44.773 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:44 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:46:44.773 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:44 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:46:44.773 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:44 smithi083 ceph-mon[29421]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-20T18:46:44.773 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:44 smithi083 ceph-mon[29421]: osd.1 v2:172.21.15.83:6805/4213285798 boot 2024-09-20T18:46:44.773 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:44 smithi083 ceph-mon[29421]: osdmap e13: 2 total, 2 up, 2 in 2024-09-20T18:46:44.773 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:44 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-20T18:46:44.773 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:44 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:46:44.773 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:44 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:46:44.773 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:44 smithi083 ceph-mon[36260]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-20T18:46:44.773 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:44 smithi083 ceph-mon[36260]: osd.1 v2:172.21.15.83:6805/4213285798 boot 2024-09-20T18:46:44.774 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:44 smithi083 ceph-mon[36260]: osdmap e13: 2 total, 2 up, 2 in 2024-09-20T18:46:44.774 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:44 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-20T18:46:45.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:44 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:46:45.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:44 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:46:45.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:44 smithi142 ceph-mon[32508]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-20T18:46:45.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:44 smithi142 ceph-mon[32508]: osd.1 v2:172.21.15.83:6805/4213285798 boot 2024-09-20T18:46:45.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:44 smithi142 ceph-mon[32508]: osdmap e13: 2 total, 2 up, 2 in 2024-09-20T18:46:45.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:44 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-20T18:46:45.680 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:45 smithi083 ceph-mon[29421]: osdmap e14: 2 total, 2 up, 2 in 2024-09-20T18:46:45.680 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:45 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:45.680 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:45 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:45.680 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:45 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-20T18:46:45.680 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:45 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-20T18:46:45.680 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:45 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:45.680 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:45 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:46:45.680 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:45 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:46:45.680 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:45 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:45.681 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:45 smithi083 ceph-mon[36260]: osdmap e14: 2 total, 2 up, 2 in 2024-09-20T18:46:45.681 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:45 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:45.681 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:45 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:45.681 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:45 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-20T18:46:45.681 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:45 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-20T18:46:45.681 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:45 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:45.681 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:45 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:46:45.681 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:45 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:46:45.681 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:45 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:46.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:45 smithi142 ceph-mon[32508]: osdmap e14: 2 total, 2 up, 2 in 2024-09-20T18:46:46.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:45 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:46.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:45 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:46.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:45 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-20T18:46:46.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:45 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-20T18:46:46.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:45 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:46.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:45 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:46:46.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:45 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:46:46.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:45 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:46:46.683 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:46 smithi083 ceph-mon[29421]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-20T18:46:46.683 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:46 smithi083 ceph-mon[29421]: Detected new or changed devices on smithi083 2024-09-20T18:46:46.683 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:46 smithi083 ceph-mon[29421]: Adjusting osd_memory_target on smithi083 to 8032M 2024-09-20T18:46:46.683 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:46 smithi083 ceph-mon[36260]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-20T18:46:46.684 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:46 smithi083 ceph-mon[36260]: Detected new or changed devices on smithi083 2024-09-20T18:46:46.684 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:46 smithi083 ceph-mon[36260]: Adjusting osd_memory_target on smithi083 to 8032M 2024-09-20T18:46:47.012 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:46:47.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:46 smithi142 ceph-mon[32508]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-20T18:46:47.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:46 smithi142 ceph-mon[32508]: Detected new or changed devices on smithi083 2024-09-20T18:46:47.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:46 smithi142 ceph-mon[32508]: Adjusting osd_memory_target on smithi083 to 8032M 2024-09-20T18:46:47.037 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi083:vg_nvme/lv_2 2024-09-20T18:46:47.282 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:46:48.968 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:48 smithi083 ceph-mon[29421]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-20T18:46:48.968 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:48 smithi083 ceph-mon[29421]: from='client.14274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:46:48.968 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:48 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-20T18:46:48.969 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:48 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-20T18:46:48.969 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:48 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:46:48.969 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:48 smithi083 ceph-mon[36260]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-20T18:46:48.969 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:48 smithi083 ceph-mon[36260]: from='client.14274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:46:48.969 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:48 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-20T18:46:48.969 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:48 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-20T18:46:48.969 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:48 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:46:49.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:48 smithi142 ceph-mon[32508]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-20T18:46:49.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:48 smithi142 ceph-mon[32508]: from='client.14274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:46:49.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:48 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-20T18:46:49.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:48 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-20T18:46:49.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:48 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:46:50.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:50 smithi083 ceph-mon[29421]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-20T18:46:50.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:50 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/421688912' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "448ab692-ec41-44dc-b2d5-dbefd255b3fd"} : dispatch 2024-09-20T18:46:50.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:50 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/421688912' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "448ab692-ec41-44dc-b2d5-dbefd255b3fd"}]': finished 2024-09-20T18:46:50.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:50 smithi083 ceph-mon[29421]: osdmap e15: 3 total, 2 up, 3 in 2024-09-20T18:46:50.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:50 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-20T18:46:50.792 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:50 smithi083 ceph-mon[36260]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-20T18:46:50.792 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:50 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/421688912' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "448ab692-ec41-44dc-b2d5-dbefd255b3fd"} : dispatch 2024-09-20T18:46:50.792 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:50 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/421688912' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "448ab692-ec41-44dc-b2d5-dbefd255b3fd"}]': finished 2024-09-20T18:46:50.792 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:50 smithi083 ceph-mon[36260]: osdmap e15: 3 total, 2 up, 3 in 2024-09-20T18:46:50.792 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:50 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-20T18:46:51.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:50 smithi142 ceph-mon[32508]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-20T18:46:51.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:50 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/421688912' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "448ab692-ec41-44dc-b2d5-dbefd255b3fd"} : dispatch 2024-09-20T18:46:51.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:50 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/421688912' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "448ab692-ec41-44dc-b2d5-dbefd255b3fd"}]': finished 2024-09-20T18:46:51.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:50 smithi142 ceph-mon[32508]: osdmap e15: 3 total, 2 up, 3 in 2024-09-20T18:46:51.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:50 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-20T18:46:52.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:51 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2896287878' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-20T18:46:52.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:51 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2896287878' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-20T18:46:52.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:51 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2896287878' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-20T18:46:53.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:52 smithi142 ceph-mon[32508]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-20T18:46:53.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:52 smithi083 ceph-mon[29421]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-20T18:46:53.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:52 smithi083 ceph-mon[36260]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-20T18:46:54.919 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:54 smithi083 ceph-mon[29421]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-20T18:46:54.920 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:54 smithi083 ceph-mon[36260]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-20T18:46:55.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:54 smithi142 ceph-mon[32508]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-20T18:46:56.806 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:56 smithi083 ceph-mon[36260]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-20T18:46:56.807 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:56 smithi083 ceph-mon[29421]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-20T18:46:57.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:56 smithi142 ceph-mon[32508]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-20T18:46:57.803 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:57 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-20T18:46:57.803 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:57 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:46:57.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:57 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-20T18:46:57.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:57 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:46:58.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:57 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-20T18:46:58.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:57 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:46:58.896 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:58 smithi083 ceph-mon[29421]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-20T18:46:58.896 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:46:58 smithi083 ceph-mon[29421]: Deploying daemon osd.2 on smithi083 2024-09-20T18:46:58.896 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:58 smithi083 ceph-mon[36260]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-20T18:46:58.897 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:46:58 smithi083 ceph-mon[36260]: Deploying daemon osd.2 on smithi083 2024-09-20T18:46:59.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:58 smithi142 ceph-mon[32508]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-20T18:46:59.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:46:58 smithi142 ceph-mon[32508]: Deploying daemon osd.2 on smithi083 2024-09-20T18:47:00.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:00 smithi083 ceph-mon[29421]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-20T18:47:00.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:00 smithi083 ceph-mon[36260]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-20T18:47:01.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:00 smithi142 ceph-mon[32508]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-20T18:47:03.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:02 smithi142 ceph-mon[32508]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-20T18:47:03.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:02 smithi083 ceph-mon[29421]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-20T18:47:03.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:02 smithi083 ceph-mon[36260]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-20T18:47:04.998 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:04 smithi083 ceph-mon[29421]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-20T18:47:04.998 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:04 smithi083 ceph-mon[36260]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-20T18:47:05.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:04 smithi142 ceph-mon[32508]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-20T18:47:05.978 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:05 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:47:05.979 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:05 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:05.979 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:05 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:05.979 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:05 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:47:05.979 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:05 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:05.979 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:05 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:06.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:05 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:47:06.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:05 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:06.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:05 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:06.874 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:06 smithi083 ceph-mon[29421]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-20T18:47:06.874 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:06 smithi083 ceph-mon[36260]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-20T18:47:07.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:06 smithi142 ceph-mon[32508]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-20T18:47:09.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:08 smithi142 ceph-mon[32508]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-20T18:47:09.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:08 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:09.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:08 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:09.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:08 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:47:09.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:08 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:47:09.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:08 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:09.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:08 smithi083 ceph-mon[29421]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-20T18:47:09.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:08 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:09.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:08 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:09.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:08 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:47:09.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:08 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:47:09.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:08 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:09.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:08 smithi083 ceph-mon[36260]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-20T18:47:09.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:08 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:09.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:08 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:09.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:08 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:47:09.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:08 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:47:09.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:08 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:10.760 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:10 smithi083 ceph-mon[29421]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-20T18:47:10.760 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:10 smithi083 ceph-mon[36260]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-20T18:47:11.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:10 smithi142 ceph-mon[32508]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-20T18:47:11.102 INFO:teuthology.orchestra.run.smithi083.stdout:Created osd(s) 2 on host 'smithi083' 2024-09-20T18:47:11.706 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:11 smithi083 ceph-mon[29421]: from='osd.2 v2:172.21.15.83:6809/805947571' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-20T18:47:11.707 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:11 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:47:11.707 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:11 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:11.707 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:11 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:11.707 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:11 smithi083 ceph-mon[36260]: from='osd.2 v2:172.21.15.83:6809/805947571' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-20T18:47:11.707 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:11 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:47:11.707 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:11 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:11.707 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:11 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:11.737 DEBUG:teuthology.orchestra.run.smithi083:osd.2> sudo journalctl -f -n 0 -u ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@osd.2.service 2024-09-20T18:47:11.739 INFO:tasks.cephadm:Deploying osd.3 on smithi083 with /dev/vg_nvme/lv_1... 2024-09-20T18:47:11.739 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-20T18:47:12.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:11 smithi142 ceph-mon[32508]: from='osd.2 v2:172.21.15.83:6809/805947571' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-20T18:47:12.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:11 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:47:12.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:11 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:12.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:11 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:12.434 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:47:12.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:12 smithi083 ceph-mon[29421]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-20T18:47:12.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:12 smithi083 ceph-mon[29421]: from='osd.2 v2:172.21.15.83:6809/805947571' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-20T18:47:12.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:12 smithi083 ceph-mon[29421]: osdmap e16: 3 total, 2 up, 3 in 2024-09-20T18:47:12.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:12 smithi083 ceph-mon[29421]: from='osd.2 v2:172.21.15.83:6809/805947571' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi083", "root=default"]} : dispatch 2024-09-20T18:47:12.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:12 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-20T18:47:12.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:12 smithi083 ceph-mon[36260]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-20T18:47:12.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:12 smithi083 ceph-mon[36260]: from='osd.2 v2:172.21.15.83:6809/805947571' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-20T18:47:12.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:12 smithi083 ceph-mon[36260]: osdmap e16: 3 total, 2 up, 3 in 2024-09-20T18:47:12.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:12 smithi083 ceph-mon[36260]: from='osd.2 v2:172.21.15.83:6809/805947571' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi083", "root=default"]} : dispatch 2024-09-20T18:47:12.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:12 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-20T18:47:13.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:12 smithi142 ceph-mon[32508]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-20T18:47:13.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:12 smithi142 ceph-mon[32508]: from='osd.2 v2:172.21.15.83:6809/805947571' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-20T18:47:13.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:12 smithi142 ceph-mon[32508]: osdmap e16: 3 total, 2 up, 3 in 2024-09-20T18:47:13.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:12 smithi142 ceph-mon[32508]: from='osd.2 v2:172.21.15.83:6809/805947571' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi083", "root=default"]} : dispatch 2024-09-20T18:47:13.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:12 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-20T18:47:13.303 INFO:journalctl@ceph.osd.2.smithi083.stdout:Sep 20 18:47:13 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-2[50848]: 2024-09-20T18:47:13.036+0000 7fd689b89640 -1 osd.2 0 waiting for initial osdmap 2024-09-20T18:47:13.303 INFO:journalctl@ceph.osd.2.smithi083.stdout:Sep 20 18:47:13 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-2[50848]: 2024-09-20T18:47:13.039+0000 7fd6819a8640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-20T18:47:14.016 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:13 smithi083 ceph-mon[29421]: from='osd.2 v2:172.21.15.83:6809/805947571' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-09-20T18:47:14.016 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:13 smithi083 ceph-mon[29421]: osdmap e17: 3 total, 2 up, 3 in 2024-09-20T18:47:14.016 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:13 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-20T18:47:14.016 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:13 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-20T18:47:14.017 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:13 smithi083 ceph-mon[29421]: from='osd.2 v2:172.21.15.83:6809/805947571' entity='osd.2' 2024-09-20T18:47:14.017 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:13 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-20T18:47:14.017 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:13 smithi083 ceph-mon[36260]: from='osd.2 v2:172.21.15.83:6809/805947571' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-09-20T18:47:14.017 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:13 smithi083 ceph-mon[36260]: osdmap e17: 3 total, 2 up, 3 in 2024-09-20T18:47:14.017 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:13 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-20T18:47:14.017 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:13 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-20T18:47:14.017 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:13 smithi083 ceph-mon[36260]: from='osd.2 v2:172.21.15.83:6809/805947571' entity='osd.2' 2024-09-20T18:47:14.017 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:13 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-20T18:47:14.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:13 smithi142 ceph-mon[32508]: from='osd.2 v2:172.21.15.83:6809/805947571' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-09-20T18:47:14.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:13 smithi142 ceph-mon[32508]: osdmap e17: 3 total, 2 up, 3 in 2024-09-20T18:47:14.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:13 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-20T18:47:14.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:13 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-20T18:47:14.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:13 smithi142 ceph-mon[32508]: from='osd.2 v2:172.21.15.83:6809/805947571' entity='osd.2' 2024-09-20T18:47:14.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:13 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-20T18:47:15.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:14 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:47:15.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:14 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:47:15.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:14 smithi142 ceph-mon[32508]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-20T18:47:15.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:14 smithi142 ceph-mon[32508]: osd.2 v2:172.21.15.83:6809/805947571 boot 2024-09-20T18:47:15.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:14 smithi142 ceph-mon[32508]: osdmap e18: 3 total, 3 up, 3 in 2024-09-20T18:47:15.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:14 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-20T18:47:15.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:14 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:47:15.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:14 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:47:15.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:14 smithi083 ceph-mon[36260]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-20T18:47:15.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:14 smithi083 ceph-mon[36260]: osd.2 v2:172.21.15.83:6809/805947571 boot 2024-09-20T18:47:15.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:14 smithi083 ceph-mon[36260]: osdmap e18: 3 total, 3 up, 3 in 2024-09-20T18:47:15.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:14 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-20T18:47:15.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:14 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:47:15.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:14 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:47:15.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:14 smithi083 ceph-mon[29421]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-20T18:47:15.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:14 smithi083 ceph-mon[29421]: osd.2 v2:172.21.15.83:6809/805947571 boot 2024-09-20T18:47:15.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:14 smithi083 ceph-mon[29421]: osdmap e18: 3 total, 3 up, 3 in 2024-09-20T18:47:15.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:14 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-20T18:47:15.974 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:15 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-20T18:47:15.974 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:15 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:15.974 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:15 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:15.974 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:15 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-20T18:47:15.974 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:15 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-20T18:47:15.974 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:15 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-20T18:47:15.975 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:15 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:15.975 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:15 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:47:15.975 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:15 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:47:15.975 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:15 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:15.975 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:15 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-20T18:47:15.975 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:15 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:15.975 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:15 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:15.975 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:15 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-20T18:47:15.975 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:15 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-20T18:47:15.975 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:15 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-20T18:47:15.975 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:15 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:15.976 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:15 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:47:15.976 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:15 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:47:15.976 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:15 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:16.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:15 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-20T18:47:16.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:15 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:16.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:15 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:16.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:15 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-20T18:47:16.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:15 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-20T18:47:16.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:15 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-20T18:47:16.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:15 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:16.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:15 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:47:16.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:15 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:47:16.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:15 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:16.803 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:16 smithi083 ceph-mon[36260]: pgmap v77: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-20T18:47:16.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:16 smithi083 ceph-mon[36260]: Detected new or changed devices on smithi083 2024-09-20T18:47:16.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:16 smithi083 ceph-mon[36260]: Adjusting osd_memory_target on smithi083 to 5354M 2024-09-20T18:47:16.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:16 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-20T18:47:16.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:16 smithi083 ceph-mon[36260]: osdmap e19: 3 total, 3 up, 3 in 2024-09-20T18:47:16.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:16 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-20T18:47:16.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:16 smithi083 ceph-mon[29421]: pgmap v77: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-20T18:47:16.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:16 smithi083 ceph-mon[29421]: Detected new or changed devices on smithi083 2024-09-20T18:47:16.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:16 smithi083 ceph-mon[29421]: Adjusting osd_memory_target on smithi083 to 5354M 2024-09-20T18:47:16.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:16 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-20T18:47:16.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:16 smithi083 ceph-mon[29421]: osdmap e19: 3 total, 3 up, 3 in 2024-09-20T18:47:16.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:16 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-20T18:47:17.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:16 smithi142 ceph-mon[32508]: pgmap v77: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-20T18:47:17.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:16 smithi142 ceph-mon[32508]: Detected new or changed devices on smithi083 2024-09-20T18:47:17.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:16 smithi142 ceph-mon[32508]: Adjusting osd_memory_target on smithi083 to 5354M 2024-09-20T18:47:17.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:16 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-20T18:47:17.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:16 smithi142 ceph-mon[32508]: osdmap e19: 3 total, 3 up, 3 in 2024-09-20T18:47:17.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:16 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-20T18:47:17.156 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:47:17.185 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi083:vg_nvme/lv_1 2024-09-20T18:47:17.464 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:47:17.481 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 18:47:17 smithi083 sudo[54902]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-20T18:47:17.481 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 18:47:17 smithi083 sudo[54902]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-20T18:47:17.482 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 18:47:17 smithi083 sudo[54902]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-20T18:47:17.482 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 18:47:17 smithi083 sudo[54902]: pam_unix(sudo:session): session closed for user root 2024-09-20T18:47:17.482 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 18:47:17 smithi083 sudo[54906]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-20T18:47:17.482 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 18:47:17 smithi083 sudo[54906]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-20T18:47:17.482 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 18:47:17 smithi083 sudo[54906]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-20T18:47:17.482 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 18:47:17 smithi083 sudo[54906]: pam_unix(sudo:session): session closed for user root 2024-09-20T18:47:17.482 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 20 18:47:17 smithi083 sudo[54909]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-20T18:47:17.482 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 20 18:47:17 smithi083 sudo[54909]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-20T18:47:17.482 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 20 18:47:17 smithi083 sudo[54909]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-20T18:47:17.482 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 20 18:47:17 smithi083 sudo[54909]: pam_unix(sudo:session): session closed for user root 2024-09-20T18:47:17.482 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 20 18:47:17 smithi083 sudo[54914]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-20T18:47:17.483 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 20 18:47:17 smithi083 sudo[54914]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-20T18:47:17.483 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 20 18:47:17 smithi083 sudo[54914]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-20T18:47:17.483 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 20 18:47:17 smithi083 sudo[54914]: pam_unix(sudo:session): session closed for user root 2024-09-20T18:47:17.483 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:17 smithi083 sudo[54923]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-20T18:47:17.483 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:17 smithi083 sudo[54923]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-20T18:47:17.483 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:17 smithi083 sudo[54923]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-20T18:47:17.483 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:17 smithi083 sudo[54923]: pam_unix(sudo:session): session closed for user root 2024-09-20T18:47:17.483 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:17 smithi083 sudo[54933]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-20T18:47:17.483 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:17 smithi083 sudo[54933]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-20T18:47:17.483 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:17 smithi083 sudo[54933]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-20T18:47:17.484 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:17 smithi083 sudo[54933]: pam_unix(sudo:session): session closed for user root 2024-09-20T18:47:17.484 INFO:journalctl@ceph.osd.2.smithi083.stdout:Sep 20 18:47:17 smithi083 sudo[54917]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-20T18:47:17.484 INFO:journalctl@ceph.osd.2.smithi083.stdout:Sep 20 18:47:17 smithi083 sudo[54917]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-20T18:47:17.484 INFO:journalctl@ceph.osd.2.smithi083.stdout:Sep 20 18:47:17 smithi083 sudo[54917]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-20T18:47:17.484 INFO:journalctl@ceph.osd.2.smithi083.stdout:Sep 20 18:47:17 smithi083 sudo[54917]: pam_unix(sudo:session): session closed for user root 2024-09-20T18:47:17.484 INFO:journalctl@ceph.osd.2.smithi083.stdout:Sep 20 18:47:17 smithi083 sudo[54920]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-20T18:47:17.484 INFO:journalctl@ceph.osd.2.smithi083.stdout:Sep 20 18:47:17 smithi083 sudo[54920]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-20T18:47:17.484 INFO:journalctl@ceph.osd.2.smithi083.stdout:Sep 20 18:47:17 smithi083 sudo[54920]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-20T18:47:17.484 INFO:journalctl@ceph.osd.2.smithi083.stdout:Sep 20 18:47:17 smithi083 sudo[54920]: pam_unix(sudo:session): session closed for user root 2024-09-20T18:47:17.484 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:17 smithi083 sudo[54972]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-20T18:47:17.727 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:17 smithi142 sudo[34327]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-20T18:47:17.728 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:17 smithi142 sudo[34327]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-20T18:47:17.728 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:17 smithi142 sudo[34327]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-20T18:47:17.728 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:17 smithi142 sudo[34327]: pam_unix(sudo:session): session closed for user root 2024-09-20T18:47:17.728 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:17 smithi142 sudo[34330]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-20T18:47:17.728 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:17 smithi142 sudo[34330]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-20T18:47:17.728 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:17 smithi142 sudo[34330]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-20T18:47:17.728 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:17 smithi142 sudo[34330]: pam_unix(sudo:session): session closed for user root 2024-09-20T18:47:17.728 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:17 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-20T18:47:17.728 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:17 smithi142 ceph-mon[32508]: osdmap e20: 3 total, 3 up, 3 in 2024-09-20T18:47:17.728 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:17 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:17.728 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:17 smithi142 ceph-mon[32508]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-20T18:47:17.728 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:17 smithi142 ceph-mon[32508]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-20T18:47:17.728 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:17 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-20T18:47:17.729 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:17 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-20T18:47:17.729 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:17 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-20T18:47:17.729 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:17 smithi142 ceph-mon[32508]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-20T18:47:17.729 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:17 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-20T18:47:17.729 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:17 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-20T18:47:17.729 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:17 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-20T18:47:17.729 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:17 smithi142 ceph-mon[32508]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-20T18:47:17.729 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:17 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:17.729 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:17 smithi142 ceph-mon[32508]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-20T18:47:17.729 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:17 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-20T18:47:17.729 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:17 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-20T18:47:17.729 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:17 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-20T18:47:17.729 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:17 smithi142 ceph-mon[32508]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-20T18:47:17.731 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:17 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-20T18:47:17.732 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:17 smithi083 sudo[54972]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-20T18:47:17.732 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:17 smithi083 sudo[54972]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-20T18:47:17.732 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:17 smithi083 sudo[54972]: pam_unix(sudo:session): session closed for user root 2024-09-20T18:47:17.732 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:17 smithi083 sudo[54982]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-20T18:47:17.732 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:17 smithi083 sudo[54982]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-20T18:47:17.732 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:17 smithi083 sudo[54982]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-20T18:47:17.732 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:17 smithi083 sudo[54982]: pam_unix(sudo:session): session closed for user root 2024-09-20T18:47:17.732 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:17 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-20T18:47:17.732 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:17 smithi083 ceph-mon[36260]: osdmap e20: 3 total, 3 up, 3 in 2024-09-20T18:47:17.732 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:17 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:17.732 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:17 smithi083 ceph-mon[36260]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-20T18:47:17.732 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:17 smithi083 ceph-mon[36260]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-20T18:47:17.732 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:17 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-20T18:47:17.733 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:17 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-20T18:47:17.733 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:17 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-20T18:47:17.733 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:17 smithi083 ceph-mon[36260]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-20T18:47:17.733 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:17 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-20T18:47:18.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:17 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-20T18:47:18.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:17 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-20T18:47:18.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:17 smithi083 ceph-mon[36260]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-20T18:47:18.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:17 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:18.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:17 smithi083 ceph-mon[36260]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-20T18:47:18.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:17 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-20T18:47:18.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:17 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-20T18:47:18.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:17 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-20T18:47:18.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:17 smithi083 ceph-mon[36260]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-20T18:47:18.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:17 smithi083 ceph-mon[29421]: osdmap e20: 3 total, 3 up, 3 in 2024-09-20T18:47:18.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:17 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:18.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:17 smithi083 ceph-mon[29421]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-20T18:47:18.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:17 smithi083 ceph-mon[29421]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-20T18:47:18.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:17 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-20T18:47:18.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:17 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-20T18:47:18.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:17 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-20T18:47:18.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:17 smithi083 ceph-mon[29421]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-20T18:47:18.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:17 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-20T18:47:18.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:17 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-20T18:47:18.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:17 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-20T18:47:18.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:17 smithi083 ceph-mon[29421]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-20T18:47:18.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:17 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:18.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:17 smithi083 ceph-mon[29421]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-20T18:47:18.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:17 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-20T18:47:18.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:17 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-20T18:47:18.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:17 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-20T18:47:18.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:17 smithi083 ceph-mon[29421]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-20T18:47:18.761 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:18 smithi083 ceph-mon[29421]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-20T18:47:18.761 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:18 smithi083 ceph-mon[29421]: osdmap e21: 3 total, 3 up, 3 in 2024-09-20T18:47:18.761 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:18 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-20T18:47:18.761 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:18 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-20T18:47:18.761 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:18 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:47:18.761 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:18 smithi083 ceph-mon[36260]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-20T18:47:18.761 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:18 smithi083 ceph-mon[36260]: osdmap e21: 3 total, 3 up, 3 in 2024-09-20T18:47:18.761 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:18 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-20T18:47:18.762 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:18 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-20T18:47:18.762 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:18 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:47:19.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:18 smithi142 ceph-mon[32508]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-20T18:47:19.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:18 smithi142 ceph-mon[32508]: osdmap e21: 3 total, 3 up, 3 in 2024-09-20T18:47:19.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:18 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-20T18:47:19.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:18 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-20T18:47:19.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:18 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:47:19.852 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:19 smithi083 ceph-mon[29421]: from='client.14310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:47:19.853 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:19 smithi083 ceph-mon[36260]: from='client.14310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:47:20.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:19 smithi142 ceph-mon[32508]: from='client.14310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:47:21.011 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:20 smithi083 ceph-mon[29421]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-20T18:47:21.011 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:20 smithi083 ceph-mon[29421]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-20T18:47:21.011 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:20 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3783407284' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7cda5e50-e22e-4d4a-823b-53c8f5061193"} : dispatch 2024-09-20T18:47:21.011 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:20 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3783407284' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7cda5e50-e22e-4d4a-823b-53c8f5061193"}]': finished 2024-09-20T18:47:21.011 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:20 smithi083 ceph-mon[29421]: osdmap e22: 4 total, 3 up, 4 in 2024-09-20T18:47:21.011 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:20 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-20T18:47:21.012 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:20 smithi083 ceph-mon[36260]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-20T18:47:21.012 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:20 smithi083 ceph-mon[36260]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-20T18:47:21.012 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:20 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3783407284' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7cda5e50-e22e-4d4a-823b-53c8f5061193"} : dispatch 2024-09-20T18:47:21.012 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:20 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3783407284' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7cda5e50-e22e-4d4a-823b-53c8f5061193"}]': finished 2024-09-20T18:47:21.012 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:20 smithi083 ceph-mon[36260]: osdmap e22: 4 total, 3 up, 4 in 2024-09-20T18:47:21.012 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:20 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-20T18:47:21.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:20 smithi142 ceph-mon[32508]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-20T18:47:21.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:20 smithi142 ceph-mon[32508]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-20T18:47:21.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:20 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3783407284' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7cda5e50-e22e-4d4a-823b-53c8f5061193"} : dispatch 2024-09-20T18:47:21.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:20 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3783407284' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7cda5e50-e22e-4d4a-823b-53c8f5061193"}]': finished 2024-09-20T18:47:21.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:20 smithi142 ceph-mon[32508]: osdmap e22: 4 total, 3 up, 4 in 2024-09-20T18:47:21.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:20 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-20T18:47:22.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:21 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2808596036' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-20T18:47:22.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:21 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2808596036' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-20T18:47:22.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:21 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2808596036' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-20T18:47:23.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:22 smithi142 ceph-mon[32508]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-20T18:47:23.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:22 smithi083 ceph-mon[29421]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-20T18:47:23.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:22 smithi083 ceph-mon[36260]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-20T18:47:24.924 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:24 smithi083 ceph-mon[29421]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-20T18:47:24.924 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:24 smithi083 ceph-mon[36260]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-20T18:47:25.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:24 smithi142 ceph-mon[32508]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-20T18:47:27.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:26 smithi142 ceph-mon[32508]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-20T18:47:27.045 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:26 smithi083 ceph-mon[29421]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-20T18:47:27.045 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:26 smithi083 ceph-mon[36260]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-20T18:47:28.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:27 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-20T18:47:28.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:27 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:47:28.026 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:27 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-20T18:47:28.026 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:27 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:47:28.027 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:27 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-20T18:47:28.027 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:27 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:47:29.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:28 smithi142 ceph-mon[32508]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-20T18:47:29.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:28 smithi142 ceph-mon[32508]: Deploying daemon osd.3 on smithi083 2024-09-20T18:47:29.033 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:28 smithi083 ceph-mon[29421]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-20T18:47:29.033 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:28 smithi083 ceph-mon[29421]: Deploying daemon osd.3 on smithi083 2024-09-20T18:47:29.033 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:28 smithi083 ceph-mon[36260]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-20T18:47:29.033 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:28 smithi083 ceph-mon[36260]: Deploying daemon osd.3 on smithi083 2024-09-20T18:47:31.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:30 smithi142 ceph-mon[32508]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-20T18:47:31.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:30 smithi083 ceph-mon[29421]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-20T18:47:31.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:30 smithi083 ceph-mon[36260]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-20T18:47:33.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:32 smithi142 ceph-mon[32508]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-20T18:47:33.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:32 smithi083 ceph-mon[29421]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-20T18:47:33.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:32 smithi083 ceph-mon[36260]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-20T18:47:34.910 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:34 smithi083 ceph-mon[29421]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-20T18:47:34.911 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:34 smithi083 ceph-mon[36260]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-20T18:47:35.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:34 smithi142 ceph-mon[32508]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-20T18:47:37.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:36 smithi142 ceph-mon[32508]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-20T18:47:37.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:36 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:47:37.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:36 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:37.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:36 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:37.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:36 smithi083 ceph-mon[29421]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-20T18:47:37.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:36 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:47:37.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:36 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:37.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:36 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:37.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:36 smithi083 ceph-mon[36260]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-20T18:47:37.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:36 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:47:37.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:36 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:37.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:36 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:39.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:38 smithi142 ceph-mon[32508]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-20T18:47:39.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:38 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:39.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:38 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:39.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:38 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:47:39.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:38 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:47:39.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:38 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:39.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:38 smithi083 ceph-mon[29421]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-20T18:47:39.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:38 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:39.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:38 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:39.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:38 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:47:39.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:38 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:47:39.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:38 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:39.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:38 smithi083 ceph-mon[36260]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-20T18:47:39.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:38 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:39.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:38 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:39.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:38 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:47:39.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:38 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:47:39.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:38 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:41.027 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:40 smithi083 ceph-mon[29421]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-20T18:47:41.027 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:40 smithi083 ceph-mon[29421]: from='osd.3 v2:172.21.15.83:6813/987237318' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-20T18:47:41.028 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:40 smithi083 ceph-mon[36260]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-20T18:47:41.028 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:40 smithi083 ceph-mon[36260]: from='osd.3 v2:172.21.15.83:6813/987237318' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-20T18:47:41.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:40 smithi142 ceph-mon[32508]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-20T18:47:41.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:40 smithi142 ceph-mon[32508]: from='osd.3 v2:172.21.15.83:6813/987237318' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-20T18:47:41.357 INFO:teuthology.orchestra.run.smithi083.stdout:Created osd(s) 3 on host 'smithi083' 2024-09-20T18:47:42.059 DEBUG:teuthology.orchestra.run.smithi083:osd.3> sudo journalctl -f -n 0 -u ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@osd.3.service 2024-09-20T18:47:42.062 INFO:tasks.cephadm:Deploying osd.4 on smithi142 with /dev/vg_nvme/lv_4... 2024-09-20T18:47:42.062 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-20T18:47:42.303 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:42 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:47:42.303 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:42 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:47:42.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:42 smithi083 ceph-mon[36260]: from='osd.3 v2:172.21.15.83:6813/987237318' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-20T18:47:42.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:42 smithi083 ceph-mon[36260]: osdmap e23: 4 total, 3 up, 4 in 2024-09-20T18:47:42.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:42 smithi083 ceph-mon[36260]: from='osd.3 v2:172.21.15.83:6813/987237318' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi083", "root=default"]} : dispatch 2024-09-20T18:47:42.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:42 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-20T18:47:42.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:42 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:47:42.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:42 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:42.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:42 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:42.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:42 smithi083 ceph-mon[36260]: from='osd.3 v2:172.21.15.83:6813/987237318' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-09-20T18:47:42.304 INFO:journalctl@ceph.osd.3.smithi083.stdout:Sep 20 18:47:42 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-3[56873]: 2024-09-20T18:47:42.206+0000 7fd873735640 -1 osd.3 0 waiting for initial osdmap 2024-09-20T18:47:42.305 INFO:journalctl@ceph.osd.3.smithi083.stdout:Sep 20 18:47:42 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-3[56873]: 2024-09-20T18:47:42.210+0000 7fd86b554640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-20T18:47:42.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:42 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:47:42.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:42 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:47:42.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:42 smithi083 ceph-mon[29421]: from='osd.3 v2:172.21.15.83:6813/987237318' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-20T18:47:42.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:42 smithi083 ceph-mon[29421]: osdmap e23: 4 total, 3 up, 4 in 2024-09-20T18:47:42.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:42 smithi083 ceph-mon[29421]: from='osd.3 v2:172.21.15.83:6813/987237318' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi083", "root=default"]} : dispatch 2024-09-20T18:47:42.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:42 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-20T18:47:42.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:42 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:47:42.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:42 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:42.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:42 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:42.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:42 smithi083 ceph-mon[29421]: from='osd.3 v2:172.21.15.83:6813/987237318' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-09-20T18:47:42.306 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:47:42.424 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:42 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:47:42.424 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:42 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:47:42.424 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:42 smithi142 ceph-mon[32508]: from='osd.3 v2:172.21.15.83:6813/987237318' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-20T18:47:42.424 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:42 smithi142 ceph-mon[32508]: osdmap e23: 4 total, 3 up, 4 in 2024-09-20T18:47:42.424 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:42 smithi142 ceph-mon[32508]: from='osd.3 v2:172.21.15.83:6813/987237318' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi083", "root=default"]} : dispatch 2024-09-20T18:47:42.425 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:42 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-20T18:47:42.425 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:42 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:47:42.425 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:42 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:42.425 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:42 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:42.425 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:42 smithi142 ceph-mon[32508]: from='osd.3 v2:172.21.15.83:6813/987237318' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-09-20T18:47:43.303 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:43 smithi083 ceph-mon[36260]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-20T18:47:43.303 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:43 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:47:43.303 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:43 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:47:43.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:43 smithi083 ceph-mon[36260]: osdmap e24: 4 total, 3 up, 4 in 2024-09-20T18:47:43.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:43 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-20T18:47:43.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:43 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-20T18:47:43.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:43 smithi083 ceph-mon[36260]: from='osd.3 v2:172.21.15.83:6813/987237318' entity='osd.3' 2024-09-20T18:47:43.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:43 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-20T18:47:43.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:43 smithi083 ceph-mon[29421]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-20T18:47:43.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:43 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:47:43.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:43 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:47:43.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:43 smithi083 ceph-mon[29421]: osdmap e24: 4 total, 3 up, 4 in 2024-09-20T18:47:43.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:43 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-20T18:47:43.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:43 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-20T18:47:43.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:43 smithi083 ceph-mon[29421]: from='osd.3 v2:172.21.15.83:6813/987237318' entity='osd.3' 2024-09-20T18:47:43.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:43 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-20T18:47:43.441 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:43 smithi142 ceph-mon[32508]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-20T18:47:43.442 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:43 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:47:43.442 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:43 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:47:43.442 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:43 smithi142 ceph-mon[32508]: osdmap e24: 4 total, 3 up, 4 in 2024-09-20T18:47:43.442 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:43 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-20T18:47:43.442 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:43 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-20T18:47:43.442 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:43 smithi142 ceph-mon[32508]: from='osd.3 v2:172.21.15.83:6813/987237318' entity='osd.3' 2024-09-20T18:47:43.442 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:43 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-20T18:47:44.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:44 smithi142 ceph-mon[32508]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-20T18:47:44.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:44 smithi142 ceph-mon[32508]: osd.3 v2:172.21.15.83:6813/987237318 boot 2024-09-20T18:47:44.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:44 smithi142 ceph-mon[32508]: osdmap e25: 4 total, 4 up, 4 in 2024-09-20T18:47:44.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:44 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-20T18:47:44.303 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:44 smithi083 ceph-mon[36260]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-20T18:47:44.303 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:44 smithi083 ceph-mon[36260]: osd.3 v2:172.21.15.83:6813/987237318 boot 2024-09-20T18:47:44.303 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:44 smithi083 ceph-mon[36260]: osdmap e25: 4 total, 4 up, 4 in 2024-09-20T18:47:44.303 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:44 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-20T18:47:44.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:44 smithi083 ceph-mon[29421]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-20T18:47:44.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:44 smithi083 ceph-mon[29421]: osd.3 v2:172.21.15.83:6813/987237318 boot 2024-09-20T18:47:44.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:44 smithi083 ceph-mon[29421]: osdmap e25: 4 total, 4 up, 4 in 2024-09-20T18:47:44.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:44 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-20T18:47:44.896 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:47:44.923 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi142:vg_nvme/lv_4 2024-09-20T18:47:45.168 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:47:45.523 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:45 smithi142 ceph-mon[32508]: osdmap e26: 4 total, 4 up, 4 in 2024-09-20T18:47:45.523 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:45 smithi142 ceph-mon[32508]: Detected new or changed devices on smithi083 2024-09-20T18:47:45.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:45 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:45.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:45 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:45.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:45 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-20T18:47:45.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:45 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-20T18:47:45.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:45 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-20T18:47:45.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:45 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-20T18:47:45.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:45 smithi142 ceph-mon[32508]: Adjusting osd_memory_target on smithi083 to 4016M 2024-09-20T18:47:45.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:45 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:45.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:45 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:47:45.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:45 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:47:45.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:45 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:45.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:45 smithi083 ceph-mon[29421]: osdmap e26: 4 total, 4 up, 4 in 2024-09-20T18:47:45.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:45 smithi083 ceph-mon[29421]: Detected new or changed devices on smithi083 2024-09-20T18:47:45.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:45 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:45.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:45 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:45.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:45 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-20T18:47:45.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:45 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-20T18:47:45.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:45 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-20T18:47:45.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:45 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-20T18:47:45.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:45 smithi083 ceph-mon[29421]: Adjusting osd_memory_target on smithi083 to 4016M 2024-09-20T18:47:45.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:45 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:45.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:45 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:47:45.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:45 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:47:45.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:45 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:45.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:45 smithi083 ceph-mon[36260]: osdmap e26: 4 total, 4 up, 4 in 2024-09-20T18:47:45.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:45 smithi083 ceph-mon[36260]: Detected new or changed devices on smithi083 2024-09-20T18:47:45.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:45 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:45.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:45 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:45.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:45 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-20T18:47:45.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:45 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-20T18:47:45.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:45 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-20T18:47:45.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:45 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-20T18:47:45.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:45 smithi083 ceph-mon[36260]: Adjusting osd_memory_target on smithi083 to 4016M 2024-09-20T18:47:45.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:45 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:45.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:45 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:47:45.556 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:45 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:47:45.556 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:45 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:47:46.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:46 smithi142 ceph-mon[32508]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-20T18:47:46.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:46 smithi142 ceph-mon[32508]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:47:46.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:46 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-20T18:47:46.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:46 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-20T18:47:46.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:46 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:47:47.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:46 smithi083 ceph-mon[29421]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-20T18:47:47.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:46 smithi083 ceph-mon[29421]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:47:47.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:46 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-20T18:47:47.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:46 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-20T18:47:47.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:46 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:47:47.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:46 smithi083 ceph-mon[36260]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-20T18:47:47.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:46 smithi083 ceph-mon[36260]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:47:47.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:46 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-20T18:47:47.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:46 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-20T18:47:47.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:46 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:47:49.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:48 smithi142 ceph-mon[32508]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-20T18:47:49.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:48 smithi142 ceph-mon[32508]: from='client.? 172.21.15.142:0/1658080048' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3d62654d-d31d-4310-9934-49e694dd17d9"} : dispatch 2024-09-20T18:47:49.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:48 smithi142 ceph-mon[32508]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3d62654d-d31d-4310-9934-49e694dd17d9"} : dispatch 2024-09-20T18:47:49.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:48 smithi142 ceph-mon[32508]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3d62654d-d31d-4310-9934-49e694dd17d9"}]': finished 2024-09-20T18:47:49.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:48 smithi142 ceph-mon[32508]: osdmap e27: 5 total, 4 up, 5 in 2024-09-20T18:47:49.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:48 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-20T18:47:49.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:48 smithi142 ceph-mon[32508]: from='client.? 172.21.15.142:0/2265914117' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-20T18:47:49.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:48 smithi083 ceph-mon[29421]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-20T18:47:49.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:48 smithi083 ceph-mon[29421]: from='client.? 172.21.15.142:0/1658080048' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3d62654d-d31d-4310-9934-49e694dd17d9"} : dispatch 2024-09-20T18:47:49.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:48 smithi083 ceph-mon[29421]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3d62654d-d31d-4310-9934-49e694dd17d9"} : dispatch 2024-09-20T18:47:49.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:48 smithi083 ceph-mon[29421]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3d62654d-d31d-4310-9934-49e694dd17d9"}]': finished 2024-09-20T18:47:49.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:48 smithi083 ceph-mon[29421]: osdmap e27: 5 total, 4 up, 5 in 2024-09-20T18:47:49.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:48 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-20T18:47:49.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:48 smithi083 ceph-mon[29421]: from='client.? 172.21.15.142:0/2265914117' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-20T18:47:49.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:48 smithi083 ceph-mon[36260]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-20T18:47:49.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:48 smithi083 ceph-mon[36260]: from='client.? 172.21.15.142:0/1658080048' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3d62654d-d31d-4310-9934-49e694dd17d9"} : dispatch 2024-09-20T18:47:49.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:48 smithi083 ceph-mon[36260]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3d62654d-d31d-4310-9934-49e694dd17d9"} : dispatch 2024-09-20T18:47:49.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:48 smithi083 ceph-mon[36260]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3d62654d-d31d-4310-9934-49e694dd17d9"}]': finished 2024-09-20T18:47:49.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:48 smithi083 ceph-mon[36260]: osdmap e27: 5 total, 4 up, 5 in 2024-09-20T18:47:49.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:48 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-20T18:47:49.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:48 smithi083 ceph-mon[36260]: from='client.? 172.21.15.142:0/2265914117' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-20T18:47:51.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:50 smithi142 ceph-mon[32508]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-20T18:47:51.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:50 smithi083 ceph-mon[29421]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-20T18:47:51.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:50 smithi083 ceph-mon[36260]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-20T18:47:52.734 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:52 smithi142 ceph-mon[32508]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-20T18:47:53.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:52 smithi083 ceph-mon[29421]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-20T18:47:53.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:52 smithi083 ceph-mon[36260]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-20T18:47:55.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:54 smithi142 ceph-mon[32508]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-20T18:47:55.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:54 smithi083 ceph-mon[29421]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-20T18:47:55.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:54 smithi083 ceph-mon[36260]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-20T18:47:55.767 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:55 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-20T18:47:55.767 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:55 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:47:56.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:55 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-20T18:47:56.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:55 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:47:56.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:55 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-20T18:47:56.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:55 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:47:56.889 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:56 smithi142 ceph-mon[32508]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-20T18:47:56.889 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:56 smithi142 ceph-mon[32508]: Deploying daemon osd.4 on smithi142 2024-09-20T18:47:57.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:56 smithi083 ceph-mon[29421]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-20T18:47:57.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:56 smithi083 ceph-mon[29421]: Deploying daemon osd.4 on smithi142 2024-09-20T18:47:57.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:56 smithi083 ceph-mon[36260]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-20T18:47:57.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:56 smithi083 ceph-mon[36260]: Deploying daemon osd.4 on smithi142 2024-09-20T18:47:59.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:47:58 smithi142 ceph-mon[32508]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-20T18:47:59.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:47:58 smithi083 ceph-mon[29421]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-20T18:47:59.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:47:58 smithi083 ceph-mon[36260]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-20T18:48:01.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:00 smithi142 ceph-mon[32508]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-20T18:48:01.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:00 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:48:01.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:00 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:48:01.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:00 smithi083 ceph-mon[29421]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-20T18:48:01.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:00 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:48:01.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:00 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:48:01.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:00 smithi083 ceph-mon[36260]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-20T18:48:01.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:00 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:48:01.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:00 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:48:02.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:01 smithi142 ceph-mon[32508]: osdmap e28: 5 total, 4 up, 5 in 2024-09-20T18:48:02.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:01 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-20T18:48:02.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:01 smithi083 ceph-mon[29421]: osdmap e28: 5 total, 4 up, 5 in 2024-09-20T18:48:02.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:01 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-20T18:48:02.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:01 smithi083 ceph-mon[36260]: osdmap e28: 5 total, 4 up, 5 in 2024-09-20T18:48:02.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:01 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-20T18:48:03.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:02 smithi142 ceph-mon[32508]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-20T18:48:03.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:02 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:48:03.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:02 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:03.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:02 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:03.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:02 smithi083 ceph-mon[29421]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-20T18:48:03.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:02 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:48:03.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:02 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:03.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:02 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:03.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:02 smithi083 ceph-mon[36260]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-20T18:48:03.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:02 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:48:03.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:02 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:03.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:02 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:04.806 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:04 smithi142 ceph-mon[32508]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-20T18:48:05.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:04 smithi083 ceph-mon[29421]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-20T18:48:05.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:04 smithi083 ceph-mon[36260]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-20T18:48:06.523 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:06 smithi142 ceph-mon[32508]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-20T18:48:06.523 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:06 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:06.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:06 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:06.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:06 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:48:06.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:06 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:48:06.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:06 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:06.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:06 smithi083 ceph-mon[29421]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-20T18:48:06.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:06 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:06.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:06 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:06.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:06 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:48:06.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:06 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:48:06.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:06 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:06.554 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:06 smithi083 ceph-mon[36260]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-20T18:48:06.554 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:06 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:06.554 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:06 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:06.554 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:06 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:48:06.554 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:06 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:48:06.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:06 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:07.216 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 4 on host 'smithi142' 2024-09-20T18:48:07.890 DEBUG:teuthology.orchestra.run.smithi142:osd.4> sudo journalctl -f -n 0 -u ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@osd.4.service 2024-09-20T18:48:07.895 INFO:tasks.cephadm:Deploying osd.5 on smithi142 with /dev/vg_nvme/lv_3... 2024-09-20T18:48:07.895 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-20T18:48:08.184 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:08 smithi142 ceph-mon[32508]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-20T18:48:08.184 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:08 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:48:08.185 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:08 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:08.185 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:08 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:08.185 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:08 smithi142 ceph-mon[32508]: from='osd.4 v2:172.21.15.142:6800/2096684065' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-20T18:48:08.185 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:08 smithi142 ceph-mon[32508]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-20T18:48:08.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:08 smithi083 ceph-mon[29421]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-20T18:48:08.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:08 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:48:08.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:08 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:08.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:08 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:08.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:08 smithi083 ceph-mon[29421]: from='osd.4 v2:172.21.15.142:6800/2096684065' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-20T18:48:08.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:08 smithi083 ceph-mon[29421]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-20T18:48:08.554 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:08 smithi083 ceph-mon[36260]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-20T18:48:08.554 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:08 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:48:08.554 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:08 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:08.554 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:08 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:08.554 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:08 smithi083 ceph-mon[36260]: from='osd.4 v2:172.21.15.142:6800/2096684065' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-20T18:48:08.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:08 smithi083 ceph-mon[36260]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-20T18:48:09.371 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:09 smithi142 ceph-mon[32508]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-20T18:48:09.371 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:09 smithi142 ceph-mon[32508]: osdmap e29: 5 total, 4 up, 5 in 2024-09-20T18:48:09.371 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:09 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-20T18:48:09.371 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:09 smithi142 ceph-mon[32508]: from='osd.4 v2:172.21.15.142:6800/2096684065' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-20T18:48:09.371 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:09 smithi142 ceph-mon[32508]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-20T18:48:09.372 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 18:48:09 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4[36598]: 2024-09-20T18:48:09.205+0000 7f936a464640 -1 osd.4 0 waiting for initial osdmap 2024-09-20T18:48:09.372 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 18:48:09 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4[36598]: 2024-09-20T18:48:09.212+0000 7f936628b640 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-20T18:48:09.376 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:48:09.553 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:09 smithi083 ceph-mon[36260]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-20T18:48:09.553 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:09 smithi083 ceph-mon[36260]: osdmap e29: 5 total, 4 up, 5 in 2024-09-20T18:48:09.554 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:09 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-20T18:48:09.554 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:09 smithi083 ceph-mon[36260]: from='osd.4 v2:172.21.15.142:6800/2096684065' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-20T18:48:09.554 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:09 smithi083 ceph-mon[36260]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-20T18:48:09.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:09 smithi083 ceph-mon[29421]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-20T18:48:09.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:09 smithi083 ceph-mon[29421]: osdmap e29: 5 total, 4 up, 5 in 2024-09-20T18:48:09.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:09 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-20T18:48:09.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:09 smithi083 ceph-mon[29421]: from='osd.4 v2:172.21.15.142:6800/2096684065' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-20T18:48:09.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:09 smithi083 ceph-mon[29421]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-20T18:48:10.481 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:10 smithi142 ceph-mon[32508]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-20T18:48:10.482 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:10 smithi142 ceph-mon[32508]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-09-20T18:48:10.482 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:10 smithi142 ceph-mon[32508]: osdmap e30: 5 total, 4 up, 5 in 2024-09-20T18:48:10.482 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:10 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-20T18:48:10.482 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:10 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-20T18:48:10.482 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:10 smithi142 ceph-mon[32508]: osd.4 v2:172.21.15.142:6800/2096684065 boot 2024-09-20T18:48:10.482 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:10 smithi142 ceph-mon[32508]: osdmap e31: 5 total, 5 up, 5 in 2024-09-20T18:48:10.482 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:10 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-20T18:48:10.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:10 smithi083 ceph-mon[29421]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-20T18:48:10.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:10 smithi083 ceph-mon[29421]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-09-20T18:48:10.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:10 smithi083 ceph-mon[29421]: osdmap e30: 5 total, 4 up, 5 in 2024-09-20T18:48:10.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:10 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-20T18:48:10.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:10 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-20T18:48:10.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:10 smithi083 ceph-mon[29421]: osd.4 v2:172.21.15.142:6800/2096684065 boot 2024-09-20T18:48:10.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:10 smithi083 ceph-mon[29421]: osdmap e31: 5 total, 5 up, 5 in 2024-09-20T18:48:10.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:10 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-20T18:48:10.554 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:10 smithi083 ceph-mon[36260]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-20T18:48:10.554 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:10 smithi083 ceph-mon[36260]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-09-20T18:48:10.554 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:10 smithi083 ceph-mon[36260]: osdmap e30: 5 total, 4 up, 5 in 2024-09-20T18:48:10.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:10 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-20T18:48:10.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:10 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-20T18:48:10.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:10 smithi083 ceph-mon[36260]: osd.4 v2:172.21.15.142:6800/2096684065 boot 2024-09-20T18:48:10.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:10 smithi083 ceph-mon[36260]: osdmap e31: 5 total, 5 up, 5 in 2024-09-20T18:48:10.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:10 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-20T18:48:11.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:11 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:48:11.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:11 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:48:11.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:11 smithi083 ceph-mon[29421]: osdmap e32: 5 total, 5 up, 5 in 2024-09-20T18:48:11.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:11 smithi083 ceph-mon[29421]: Detected new or changed devices on smithi142 2024-09-20T18:48:11.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:11 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:11.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:11 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:11.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:11 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-20T18:48:11.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:11 smithi083 ceph-mon[29421]: Adjusting osd_memory_target on smithi142 to 17088M 2024-09-20T18:48:11.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:11 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:11.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:11 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:48:11.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:11 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:48:11.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:11 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:11.805 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:11 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:48:11.805 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:11 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:48:11.805 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:11 smithi083 ceph-mon[36260]: osdmap e32: 5 total, 5 up, 5 in 2024-09-20T18:48:11.805 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:11 smithi083 ceph-mon[36260]: Detected new or changed devices on smithi142 2024-09-20T18:48:11.805 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:11 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:11.805 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:11 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:11.805 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:11 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-20T18:48:11.805 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:11 smithi083 ceph-mon[36260]: Adjusting osd_memory_target on smithi142 to 17088M 2024-09-20T18:48:11.805 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:11 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:11.805 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:11 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:48:11.805 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:11 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:48:11.805 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:11 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:11.945 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:11 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:48:11.946 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:11 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:48:11.946 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:11 smithi142 ceph-mon[32508]: osdmap e32: 5 total, 5 up, 5 in 2024-09-20T18:48:11.946 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:11 smithi142 ceph-mon[32508]: Detected new or changed devices on smithi142 2024-09-20T18:48:11.946 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:11 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:11.946 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:11 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:11.946 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:11 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-20T18:48:11.946 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:11 smithi142 ceph-mon[32508]: Adjusting osd_memory_target on smithi142 to 17088M 2024-09-20T18:48:11.946 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:11 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:11.946 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:11 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:48:11.946 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:11 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:48:11.946 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:11 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:12.539 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:48:12.565 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi142:vg_nvme/lv_3 2024-09-20T18:48:12.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:12 smithi083 ceph-mon[29421]: pgmap v119: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-20T18:48:12.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:12 smithi083 ceph-mon[29421]: osdmap e33: 5 total, 5 up, 5 in 2024-09-20T18:48:12.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:12 smithi083 ceph-mon[36260]: pgmap v119: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-20T18:48:12.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:12 smithi083 ceph-mon[36260]: osdmap e33: 5 total, 5 up, 5 in 2024-09-20T18:48:12.809 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:48:12.929 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:12 smithi142 ceph-mon[32508]: pgmap v119: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-20T18:48:12.929 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:12 smithi142 ceph-mon[32508]: osdmap e33: 5 total, 5 up, 5 in 2024-09-20T18:48:13.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:13 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-20T18:48:13.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:13 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-20T18:48:13.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:13 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:48:13.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:13 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-20T18:48:13.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:13 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-20T18:48:13.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:13 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:48:13.815 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:13 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-20T18:48:13.815 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:13 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-20T18:48:13.815 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:13 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:48:14.671 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:14 smithi142 ceph-mon[32508]: pgmap v121: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-20T18:48:14.671 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:14 smithi142 ceph-mon[32508]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:48:15.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:14 smithi083 ceph-mon[29421]: pgmap v121: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-20T18:48:15.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:14 smithi083 ceph-mon[29421]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:48:15.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:14 smithi083 ceph-mon[36260]: pgmap v121: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-20T18:48:15.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:14 smithi083 ceph-mon[36260]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:48:17.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:16 smithi142 ceph-mon[32508]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-09-20T18:48:17.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:16 smithi142 ceph-mon[32508]: from='client.? 172.21.15.142:0/2887282718' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b3e4e21b-639f-4d25-8bbf-ea8922a9222a"} : dispatch 2024-09-20T18:48:17.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:16 smithi142 ceph-mon[32508]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b3e4e21b-639f-4d25-8bbf-ea8922a9222a"} : dispatch 2024-09-20T18:48:17.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:16 smithi142 ceph-mon[32508]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b3e4e21b-639f-4d25-8bbf-ea8922a9222a"}]': finished 2024-09-20T18:48:17.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:16 smithi142 ceph-mon[32508]: osdmap e34: 6 total, 5 up, 6 in 2024-09-20T18:48:17.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:16 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-20T18:48:17.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:16 smithi142 ceph-mon[32508]: from='client.? 172.21.15.142:0/4159058871' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-20T18:48:17.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:16 smithi083 ceph-mon[29421]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-09-20T18:48:17.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:16 smithi083 ceph-mon[29421]: from='client.? 172.21.15.142:0/2887282718' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b3e4e21b-639f-4d25-8bbf-ea8922a9222a"} : dispatch 2024-09-20T18:48:17.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:16 smithi083 ceph-mon[29421]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b3e4e21b-639f-4d25-8bbf-ea8922a9222a"} : dispatch 2024-09-20T18:48:17.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:16 smithi083 ceph-mon[29421]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b3e4e21b-639f-4d25-8bbf-ea8922a9222a"}]': finished 2024-09-20T18:48:17.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:16 smithi083 ceph-mon[29421]: osdmap e34: 6 total, 5 up, 6 in 2024-09-20T18:48:17.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:16 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-20T18:48:17.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:16 smithi083 ceph-mon[29421]: from='client.? 172.21.15.142:0/4159058871' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-20T18:48:17.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:16 smithi083 ceph-mon[36260]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-09-20T18:48:17.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:16 smithi083 ceph-mon[36260]: from='client.? 172.21.15.142:0/2887282718' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b3e4e21b-639f-4d25-8bbf-ea8922a9222a"} : dispatch 2024-09-20T18:48:17.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:16 smithi083 ceph-mon[36260]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b3e4e21b-639f-4d25-8bbf-ea8922a9222a"} : dispatch 2024-09-20T18:48:17.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:16 smithi083 ceph-mon[36260]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b3e4e21b-639f-4d25-8bbf-ea8922a9222a"}]': finished 2024-09-20T18:48:17.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:16 smithi083 ceph-mon[36260]: osdmap e34: 6 total, 5 up, 6 in 2024-09-20T18:48:17.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:16 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-20T18:48:17.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:16 smithi083 ceph-mon[36260]: from='client.? 172.21.15.142:0/4159058871' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-20T18:48:18.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:17 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:48:18.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:17 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:48:18.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:17 smithi142 ceph-mon[32508]: 1.0 scrub starts 2024-09-20T18:48:18.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:17 smithi142 ceph-mon[32508]: 1.0 scrub ok 2024-09-20T18:48:18.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:17 smithi142 ceph-mon[32508]: osdmap e35: 6 total, 5 up, 6 in 2024-09-20T18:48:18.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:17 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-20T18:48:18.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:17 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:48:18.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:17 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:48:18.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:17 smithi083 ceph-mon[29421]: 1.0 scrub starts 2024-09-20T18:48:18.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:17 smithi083 ceph-mon[29421]: 1.0 scrub ok 2024-09-20T18:48:18.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:17 smithi083 ceph-mon[29421]: osdmap e35: 6 total, 5 up, 6 in 2024-09-20T18:48:18.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:17 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-20T18:48:18.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:17 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:48:18.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:17 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:48:18.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:17 smithi083 ceph-mon[36260]: 1.0 scrub starts 2024-09-20T18:48:18.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:17 smithi083 ceph-mon[36260]: 1.0 scrub ok 2024-09-20T18:48:18.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:17 smithi083 ceph-mon[36260]: osdmap e35: 6 total, 5 up, 6 in 2024-09-20T18:48:18.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:17 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-20T18:48:19.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:18 smithi142 ceph-mon[32508]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-20T18:48:19.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:18 smithi083 ceph-mon[29421]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-20T18:48:19.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:18 smithi083 ceph-mon[36260]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-20T18:48:20.930 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:20 smithi142 ceph-mon[32508]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-20T18:48:21.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:20 smithi083 ceph-mon[29421]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-20T18:48:21.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:20 smithi083 ceph-mon[36260]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-20T18:48:22.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:22 smithi142 ceph-mon[32508]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-20T18:48:23.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:22 smithi083 ceph-mon[29421]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-20T18:48:23.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:22 smithi083 ceph-mon[36260]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-20T18:48:23.920 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:23 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-20T18:48:23.920 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:23 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:48:23.920 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:23 smithi142 ceph-mon[32508]: Deploying daemon osd.5 on smithi142 2024-09-20T18:48:24.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:23 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-20T18:48:24.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:23 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:48:24.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:23 smithi083 ceph-mon[29421]: Deploying daemon osd.5 on smithi142 2024-09-20T18:48:24.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:23 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-20T18:48:24.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:23 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:48:24.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:23 smithi083 ceph-mon[36260]: Deploying daemon osd.5 on smithi142 2024-09-20T18:48:24.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:24 smithi142 ceph-mon[32508]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-20T18:48:25.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:24 smithi083 ceph-mon[29421]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-20T18:48:25.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:24 smithi083 ceph-mon[36260]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-20T18:48:27.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:26 smithi142 ceph-mon[32508]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-20T18:48:27.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:26 smithi083 ceph-mon[29421]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-20T18:48:27.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:26 smithi083 ceph-mon[36260]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-20T18:48:29.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:28 smithi142 ceph-mon[32508]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-20T18:48:29.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:28 smithi083 ceph-mon[29421]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-20T18:48:29.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:28 smithi083 ceph-mon[36260]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-20T18:48:30.872 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:30 smithi142 ceph-mon[32508]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-20T18:48:30.872 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:30 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:48:30.872 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:30 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:30.872 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:30 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:31.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:30 smithi083 ceph-mon[29421]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-20T18:48:31.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:30 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:48:31.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:30 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:31.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:30 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:31.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:30 smithi083 ceph-mon[36260]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-20T18:48:31.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:30 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:48:31.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:30 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:31.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:30 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:32.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:32 smithi142 ceph-mon[32508]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-20T18:48:32.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:32 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:32.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:32 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:32.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:32 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:48:32.775 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:32 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:48:32.775 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:32 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:33.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:32 smithi083 ceph-mon[29421]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-20T18:48:33.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:32 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:33.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:32 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:33.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:32 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:48:33.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:32 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:48:33.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:32 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:33.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:32 smithi083 ceph-mon[36260]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-20T18:48:33.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:32 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:33.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:32 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:33.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:32 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:48:33.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:32 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:48:33.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:32 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:35.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:34 smithi142 ceph-mon[32508]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-20T18:48:35.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:34 smithi083 ceph-mon[29421]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-20T18:48:35.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:34 smithi083 ceph-mon[36260]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-20T18:48:35.588 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 5 on host 'smithi142' 2024-09-20T18:48:35.878 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:35 smithi142 ceph-mon[32508]: from='osd.5 v2:172.21.15.142:6804/2500466067' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-20T18:48:35.878 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:35 smithi142 ceph-mon[32508]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-20T18:48:35.878 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:35 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:48:35.878 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:35 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:35.878 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:35 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:36.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:35 smithi083 ceph-mon[29421]: from='osd.5 v2:172.21.15.142:6804/2500466067' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-20T18:48:36.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:35 smithi083 ceph-mon[29421]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-20T18:48:36.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:35 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:48:36.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:35 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:36.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:35 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:36.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:35 smithi083 ceph-mon[36260]: from='osd.5 v2:172.21.15.142:6804/2500466067' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-20T18:48:36.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:35 smithi083 ceph-mon[36260]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-20T18:48:36.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:35 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:48:36.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:35 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:36.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:35 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:36.140 DEBUG:teuthology.orchestra.run.smithi142:osd.5> sudo journalctl -f -n 0 -u ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@osd.5.service 2024-09-20T18:48:36.143 INFO:tasks.cephadm:Deploying osd.6 on smithi142 with /dev/vg_nvme/lv_2... 2024-09-20T18:48:36.143 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-20T18:48:36.781 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:48:37.003 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:36 smithi142 ceph-mon[32508]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-20T18:48:37.003 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:36 smithi142 ceph-mon[32508]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-20T18:48:37.003 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:36 smithi142 ceph-mon[32508]: from='osd.5 v2:172.21.15.142:6804/2500466067' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-20T18:48:37.003 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:36 smithi142 ceph-mon[32508]: osdmap e36: 6 total, 5 up, 6 in 2024-09-20T18:48:37.003 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:36 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-20T18:48:37.003 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:36 smithi142 ceph-mon[32508]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-20T18:48:37.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:36 smithi083 ceph-mon[29421]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-20T18:48:37.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:36 smithi083 ceph-mon[29421]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-20T18:48:37.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:36 smithi083 ceph-mon[29421]: from='osd.5 v2:172.21.15.142:6804/2500466067' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-20T18:48:37.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:36 smithi083 ceph-mon[29421]: osdmap e36: 6 total, 5 up, 6 in 2024-09-20T18:48:37.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:36 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-20T18:48:37.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:36 smithi083 ceph-mon[29421]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-20T18:48:37.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:36 smithi083 ceph-mon[36260]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-20T18:48:37.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:36 smithi083 ceph-mon[36260]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-20T18:48:37.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:36 smithi083 ceph-mon[36260]: from='osd.5 v2:172.21.15.142:6804/2500466067' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-20T18:48:37.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:36 smithi083 ceph-mon[36260]: osdmap e36: 6 total, 5 up, 6 in 2024-09-20T18:48:37.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:36 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-20T18:48:37.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:36 smithi083 ceph-mon[36260]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-20T18:48:37.273 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 20 18:48:37 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-5[42297]: 2024-09-20T18:48:37.081+0000 7fdd26677640 -1 osd.5 0 waiting for initial osdmap 2024-09-20T18:48:37.274 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 20 18:48:37 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-5[42297]: 2024-09-20T18:48:37.084+0000 7fdd1e496640 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-20T18:48:37.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:37 smithi142 ceph-mon[32508]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-09-20T18:48:37.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:37 smithi142 ceph-mon[32508]: osdmap e37: 6 total, 5 up, 6 in 2024-09-20T18:48:37.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:37 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-20T18:48:37.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:37 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-20T18:48:37.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:37 smithi142 ceph-mon[32508]: from='osd.5 ' entity='osd.5' 2024-09-20T18:48:38.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:37 smithi083 ceph-mon[29421]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-09-20T18:48:38.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:37 smithi083 ceph-mon[29421]: osdmap e37: 6 total, 5 up, 6 in 2024-09-20T18:48:38.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:37 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-20T18:48:38.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:37 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-20T18:48:38.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:37 smithi083 ceph-mon[29421]: from='osd.5 ' entity='osd.5' 2024-09-20T18:48:38.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:37 smithi083 ceph-mon[36260]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-09-20T18:48:38.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:37 smithi083 ceph-mon[36260]: osdmap e37: 6 total, 5 up, 6 in 2024-09-20T18:48:38.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:37 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-20T18:48:38.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:37 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-20T18:48:38.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:37 smithi083 ceph-mon[36260]: from='osd.5 ' entity='osd.5' 2024-09-20T18:48:38.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:38 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:48:38.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:38 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:48:38.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:38 smithi142 ceph-mon[32508]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-20T18:48:38.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:38 smithi142 ceph-mon[32508]: osd.5 v2:172.21.15.142:6804/2500466067 boot 2024-09-20T18:48:38.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:38 smithi142 ceph-mon[32508]: osdmap e38: 6 total, 6 up, 6 in 2024-09-20T18:48:38.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:38 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-20T18:48:39.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:38 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:48:39.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:38 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:48:39.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:38 smithi083 ceph-mon[29421]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-20T18:48:39.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:38 smithi083 ceph-mon[29421]: osd.5 v2:172.21.15.142:6804/2500466067 boot 2024-09-20T18:48:39.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:38 smithi083 ceph-mon[29421]: osdmap e38: 6 total, 6 up, 6 in 2024-09-20T18:48:39.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:38 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-20T18:48:39.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:38 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:48:39.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:38 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:48:39.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:38 smithi083 ceph-mon[36260]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-20T18:48:39.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:38 smithi083 ceph-mon[36260]: osd.5 v2:172.21.15.142:6804/2500466067 boot 2024-09-20T18:48:39.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:38 smithi083 ceph-mon[36260]: osdmap e38: 6 total, 6 up, 6 in 2024-09-20T18:48:39.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:38 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-20T18:48:40.001 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:39 smithi142 ceph-mon[32508]: osdmap e39: 6 total, 6 up, 6 in 2024-09-20T18:48:40.001 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:39 smithi142 ceph-mon[32508]: osdmap e40: 6 total, 6 up, 6 in 2024-09-20T18:48:40.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:39 smithi083 ceph-mon[29421]: osdmap e39: 6 total, 6 up, 6 in 2024-09-20T18:48:40.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:39 smithi083 ceph-mon[29421]: osdmap e40: 6 total, 6 up, 6 in 2024-09-20T18:48:40.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:39 smithi083 ceph-mon[36260]: osdmap e39: 6 total, 6 up, 6 in 2024-09-20T18:48:40.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:39 smithi083 ceph-mon[36260]: osdmap e40: 6 total, 6 up, 6 in 2024-09-20T18:48:41.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:40 smithi142 ceph-mon[32508]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-20T18:48:41.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:40 smithi142 ceph-mon[32508]: Detected new or changed devices on smithi142 2024-09-20T18:48:41.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:40 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:41.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:40 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:41.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:40 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-20T18:48:41.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:40 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-20T18:48:41.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:40 smithi142 ceph-mon[32508]: Adjusting osd_memory_target on smithi142 to 8544M 2024-09-20T18:48:41.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:40 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:41.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:40 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:48:41.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:40 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:48:41.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:40 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:41.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:40 smithi083 ceph-mon[29421]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-20T18:48:41.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:40 smithi083 ceph-mon[29421]: Detected new or changed devices on smithi142 2024-09-20T18:48:41.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:40 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:41.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:40 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:41.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:40 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-20T18:48:41.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:40 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-20T18:48:41.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:40 smithi083 ceph-mon[29421]: Adjusting osd_memory_target on smithi142 to 8544M 2024-09-20T18:48:41.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:40 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:41.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:40 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:48:41.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:40 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:48:41.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:40 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:41.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:40 smithi083 ceph-mon[36260]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-20T18:48:41.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:40 smithi083 ceph-mon[36260]: Detected new or changed devices on smithi142 2024-09-20T18:48:41.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:40 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:41.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:40 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:41.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:40 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-20T18:48:41.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:40 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-20T18:48:41.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:40 smithi083 ceph-mon[36260]: Adjusting osd_memory_target on smithi142 to 8544M 2024-09-20T18:48:41.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:40 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:41.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:40 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:48:41.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:40 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:48:41.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:40 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:48:41.331 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:48:41.360 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi142:vg_nvme/lv_2 2024-09-20T18:48:41.610 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:48:42.916 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:42 smithi142 ceph-mon[32508]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-20T18:48:42.917 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:42 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-20T18:48:42.917 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:42 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-20T18:48:42.917 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:42 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:48:43.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:42 smithi083 ceph-mon[29421]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-20T18:48:43.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:42 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-20T18:48:43.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:42 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-20T18:48:43.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:42 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:48:43.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:42 smithi083 ceph-mon[36260]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-20T18:48:43.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:42 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-20T18:48:43.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:42 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-20T18:48:43.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:42 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:48:43.755 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:43 smithi142 ceph-mon[32508]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:48:44.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:43 smithi083 ceph-mon[29421]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:48:44.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:43 smithi083 ceph-mon[36260]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:48:45.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:44 smithi142 ceph-mon[32508]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-09-20T18:48:45.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:44 smithi142 ceph-mon[32508]: from='client.? 172.21.15.142:0/2968707043' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c5db7107-ccb4-4ae2-9e63-f444897942e3"} : dispatch 2024-09-20T18:48:45.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:44 smithi142 ceph-mon[32508]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c5db7107-ccb4-4ae2-9e63-f444897942e3"} : dispatch 2024-09-20T18:48:45.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:44 smithi142 ceph-mon[32508]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c5db7107-ccb4-4ae2-9e63-f444897942e3"}]': finished 2024-09-20T18:48:45.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:44 smithi142 ceph-mon[32508]: osdmap e41: 7 total, 6 up, 7 in 2024-09-20T18:48:45.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:44 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-20T18:48:45.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:44 smithi083 ceph-mon[29421]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-09-20T18:48:45.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:44 smithi083 ceph-mon[29421]: from='client.? 172.21.15.142:0/2968707043' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c5db7107-ccb4-4ae2-9e63-f444897942e3"} : dispatch 2024-09-20T18:48:45.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:44 smithi083 ceph-mon[29421]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c5db7107-ccb4-4ae2-9e63-f444897942e3"} : dispatch 2024-09-20T18:48:45.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:44 smithi083 ceph-mon[29421]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c5db7107-ccb4-4ae2-9e63-f444897942e3"}]': finished 2024-09-20T18:48:45.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:44 smithi083 ceph-mon[29421]: osdmap e41: 7 total, 6 up, 7 in 2024-09-20T18:48:45.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:44 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-20T18:48:45.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:44 smithi083 ceph-mon[36260]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-09-20T18:48:45.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:44 smithi083 ceph-mon[36260]: from='client.? 172.21.15.142:0/2968707043' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c5db7107-ccb4-4ae2-9e63-f444897942e3"} : dispatch 2024-09-20T18:48:45.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:44 smithi083 ceph-mon[36260]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c5db7107-ccb4-4ae2-9e63-f444897942e3"} : dispatch 2024-09-20T18:48:45.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:44 smithi083 ceph-mon[36260]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c5db7107-ccb4-4ae2-9e63-f444897942e3"}]': finished 2024-09-20T18:48:45.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:44 smithi083 ceph-mon[36260]: osdmap e41: 7 total, 6 up, 7 in 2024-09-20T18:48:45.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:44 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-20T18:48:46.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:45 smithi142 ceph-mon[32508]: from='client.? 172.21.15.142:0/618221592' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-20T18:48:46.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:45 smithi083 ceph-mon[29421]: from='client.? 172.21.15.142:0/618221592' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-20T18:48:46.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:45 smithi083 ceph-mon[36260]: from='client.? 172.21.15.142:0/618221592' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-20T18:48:47.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:46 smithi142 ceph-mon[32508]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-09-20T18:48:47.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:46 smithi083 ceph-mon[29421]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-09-20T18:48:47.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:46 smithi083 ceph-mon[36260]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-09-20T18:48:49.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:48 smithi142 ceph-mon[32508]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-20T18:48:49.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:48 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:48:49.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:48 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:48:49.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:48 smithi142 ceph-mon[32508]: osdmap e42: 7 total, 6 up, 7 in 2024-09-20T18:48:49.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:48 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-20T18:48:49.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:48 smithi083 ceph-mon[29421]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-20T18:48:49.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:48 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:48:49.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:48 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:48:49.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:48 smithi083 ceph-mon[29421]: osdmap e42: 7 total, 6 up, 7 in 2024-09-20T18:48:49.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:48 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-20T18:48:49.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:48 smithi083 ceph-mon[36260]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-20T18:48:49.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:48 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:48:49.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:48 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:48:49.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:48 smithi083 ceph-mon[36260]: osdmap e42: 7 total, 6 up, 7 in 2024-09-20T18:48:49.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:48 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-20T18:48:51.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:50 smithi142 ceph-mon[32508]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-20T18:48:51.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:50 smithi083 ceph-mon[29421]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-20T18:48:51.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:50 smithi083 ceph-mon[36260]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-20T18:48:51.999 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:51 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-20T18:48:51.999 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:51 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:48:52.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:51 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-20T18:48:52.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:51 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:48:52.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:51 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-20T18:48:52.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:51 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:48:52.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:52 smithi142 ceph-mon[32508]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-20T18:48:52.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:52 smithi142 ceph-mon[32508]: Deploying daemon osd.6 on smithi142 2024-09-20T18:48:53.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:52 smithi083 ceph-mon[29421]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-20T18:48:53.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:52 smithi083 ceph-mon[29421]: Deploying daemon osd.6 on smithi142 2024-09-20T18:48:53.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:52 smithi083 ceph-mon[36260]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-20T18:48:53.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:52 smithi083 ceph-mon[36260]: Deploying daemon osd.6 on smithi142 2024-09-20T18:48:55.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:54 smithi142 ceph-mon[32508]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-20T18:48:55.064 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:54 smithi083 ceph-mon[29421]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-20T18:48:55.064 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:54 smithi083 ceph-mon[36260]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-20T18:48:57.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:56 smithi142 ceph-mon[32508]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-20T18:48:57.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:56 smithi083 ceph-mon[29421]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-20T18:48:57.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:56 smithi083 ceph-mon[36260]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-20T18:48:59.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:58 smithi142 ceph-mon[32508]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-20T18:48:59.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:58 smithi083 ceph-mon[29421]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-20T18:48:59.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:58 smithi083 ceph-mon[36260]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-20T18:49:00.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:59 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:49:00.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:59 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:00.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:48:59 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:00.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:59 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:49:00.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:59 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:00.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:48:59 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:00.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:59 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:49:00.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:59 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:00.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:48:59 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:01.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:00 smithi142 ceph-mon[32508]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-20T18:49:01.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:00 smithi083 ceph-mon[29421]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-20T18:49:01.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:00 smithi083 ceph-mon[36260]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-20T18:49:03.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:02 smithi142 ceph-mon[32508]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-20T18:49:03.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:02 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:03.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:02 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:03.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:02 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:49:03.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:02 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:49:03.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:02 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:03.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:02 smithi083 ceph-mon[29421]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-20T18:49:03.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:02 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:03.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:02 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:03.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:02 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:49:03.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:02 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:49:03.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:02 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:03.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:02 smithi083 ceph-mon[36260]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-20T18:49:03.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:02 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:03.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:02 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:03.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:02 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:49:03.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:02 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:49:03.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:02 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:03.766 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 6 on host 'smithi142' 2024-09-20T18:49:04.397 DEBUG:teuthology.orchestra.run.smithi142:osd.6> sudo journalctl -f -n 0 -u ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@osd.6.service 2024-09-20T18:49:04.400 INFO:tasks.cephadm:Deploying osd.7 on smithi142 with /dev/vg_nvme/lv_1... 2024-09-20T18:49:04.400 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-20T18:49:05.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:04 smithi142 ceph-mon[32508]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-20T18:49:05.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:04 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:49:05.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:04 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:05.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:04 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:05.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:04 smithi142 ceph-mon[32508]: from='osd.6 v2:172.21.15.142:6808/1688186226' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-20T18:49:05.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:04 smithi142 ceph-mon[32508]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-20T18:49:05.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:04 smithi083 ceph-mon[29421]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-20T18:49:05.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:04 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:49:05.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:04 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:05.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:04 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:05.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:04 smithi083 ceph-mon[29421]: from='osd.6 v2:172.21.15.142:6808/1688186226' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-20T18:49:05.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:04 smithi083 ceph-mon[29421]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-20T18:49:05.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:04 smithi083 ceph-mon[36260]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-20T18:49:05.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:04 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:49:05.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:04 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:05.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:04 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:05.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:04 smithi083 ceph-mon[36260]: from='osd.6 v2:172.21.15.142:6808/1688186226' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-20T18:49:05.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:04 smithi083 ceph-mon[36260]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-20T18:49:05.877 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:49:05.998 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:05 smithi142 ceph-mon[32508]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-20T18:49:05.999 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:05 smithi142 ceph-mon[32508]: from='osd.6 v2:172.21.15.142:6808/1688186226' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-20T18:49:05.999 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:05 smithi142 ceph-mon[32508]: osdmap e43: 7 total, 6 up, 7 in 2024-09-20T18:49:05.999 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:05 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-20T18:49:05.999 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:05 smithi142 ceph-mon[32508]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-20T18:49:05.999 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 18:49:05 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T18:49:05.765+0000 7fa1d75cc640 -1 osd.6 0 waiting for initial osdmap 2024-09-20T18:49:05.999 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 18:49:05 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T18:49:05.770+0000 7fa1d33f3640 -1 osd.6 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-20T18:49:06.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:05 smithi083 ceph-mon[29421]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-20T18:49:06.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:05 smithi083 ceph-mon[29421]: from='osd.6 v2:172.21.15.142:6808/1688186226' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-20T18:49:06.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:05 smithi083 ceph-mon[29421]: osdmap e43: 7 total, 6 up, 7 in 2024-09-20T18:49:06.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:05 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-20T18:49:06.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:05 smithi083 ceph-mon[29421]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-20T18:49:06.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:05 smithi083 ceph-mon[36260]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-20T18:49:06.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:05 smithi083 ceph-mon[36260]: from='osd.6 v2:172.21.15.142:6808/1688186226' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-20T18:49:06.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:05 smithi083 ceph-mon[36260]: osdmap e43: 7 total, 6 up, 7 in 2024-09-20T18:49:06.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:05 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-20T18:49:06.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:05 smithi083 ceph-mon[36260]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-20T18:49:07.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:06 smithi083 ceph-mon[29421]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-20T18:49:07.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:06 smithi083 ceph-mon[29421]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-09-20T18:49:07.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:06 smithi083 ceph-mon[29421]: osdmap e44: 7 total, 6 up, 7 in 2024-09-20T18:49:07.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:06 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-20T18:49:07.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:06 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-20T18:49:07.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:06 smithi083 ceph-mon[36260]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-20T18:49:07.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:06 smithi083 ceph-mon[36260]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-09-20T18:49:07.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:06 smithi083 ceph-mon[36260]: osdmap e44: 7 total, 6 up, 7 in 2024-09-20T18:49:07.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:06 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-20T18:49:07.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:06 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-20T18:49:07.071 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:06 smithi142 ceph-mon[32508]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-20T18:49:07.071 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:06 smithi142 ceph-mon[32508]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-09-20T18:49:07.071 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:06 smithi142 ceph-mon[32508]: osdmap e44: 7 total, 6 up, 7 in 2024-09-20T18:49:07.071 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:06 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-20T18:49:07.071 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:06 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-20T18:49:08.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:07 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:49:08.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:07 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:49:08.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:07 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-20T18:49:08.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:07 smithi083 ceph-mon[29421]: osd.6 v2:172.21.15.142:6808/1688186226 boot 2024-09-20T18:49:08.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:07 smithi083 ceph-mon[29421]: osdmap e45: 7 total, 7 up, 7 in 2024-09-20T18:49:08.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:07 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-20T18:49:08.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:07 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:08.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:07 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:08.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:07 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-20T18:49:08.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:07 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-20T18:49:08.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:07 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-20T18:49:08.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:07 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:08.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:07 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:49:08.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:07 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:49:08.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:07 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:08.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:07 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:49:08.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:07 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:49:08.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:07 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-20T18:49:08.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:07 smithi083 ceph-mon[36260]: osd.6 v2:172.21.15.142:6808/1688186226 boot 2024-09-20T18:49:08.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:07 smithi083 ceph-mon[36260]: osdmap e45: 7 total, 7 up, 7 in 2024-09-20T18:49:08.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:07 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-20T18:49:08.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:07 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:08.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:07 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:08.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:07 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-20T18:49:08.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:07 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-20T18:49:08.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:07 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-20T18:49:08.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:07 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:08.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:07 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:49:08.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:07 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:49:08.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:07 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:08.060 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:07 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:49:08.060 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:07 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:49:08.060 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:07 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-20T18:49:08.060 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:07 smithi142 ceph-mon[32508]: osd.6 v2:172.21.15.142:6808/1688186226 boot 2024-09-20T18:49:08.060 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:07 smithi142 ceph-mon[32508]: osdmap e45: 7 total, 7 up, 7 in 2024-09-20T18:49:08.060 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:07 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-20T18:49:08.060 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:07 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:08.060 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:07 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:08.061 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:07 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-20T18:49:08.061 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:07 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-20T18:49:08.061 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:07 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-20T18:49:08.061 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:07 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:08.061 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:07 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:49:08.061 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:07 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:49:08.061 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:07 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:08.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:08 smithi142 ceph-mon[32508]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-20T18:49:08.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:08 smithi142 ceph-mon[32508]: Detected new or changed devices on smithi142 2024-09-20T18:49:08.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:08 smithi142 ceph-mon[32508]: Adjusting osd_memory_target on smithi142 to 5696M 2024-09-20T18:49:08.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:08 smithi142 ceph-mon[32508]: osdmap e46: 7 total, 7 up, 7 in 2024-09-20T18:49:09.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:08 smithi083 ceph-mon[29421]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-20T18:49:09.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:08 smithi083 ceph-mon[29421]: Detected new or changed devices on smithi142 2024-09-20T18:49:09.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:08 smithi083 ceph-mon[29421]: Adjusting osd_memory_target on smithi142 to 5696M 2024-09-20T18:49:09.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:08 smithi083 ceph-mon[29421]: osdmap e46: 7 total, 7 up, 7 in 2024-09-20T18:49:09.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:08 smithi083 ceph-mon[36260]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-20T18:49:09.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:08 smithi083 ceph-mon[36260]: Detected new or changed devices on smithi142 2024-09-20T18:49:09.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:08 smithi083 ceph-mon[36260]: Adjusting osd_memory_target on smithi142 to 5696M 2024-09-20T18:49:09.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:08 smithi083 ceph-mon[36260]: osdmap e46: 7 total, 7 up, 7 in 2024-09-20T18:49:09.325 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-20T18:49:09.352 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi142:vg_nvme/lv_1 2024-09-20T18:49:09.601 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:49:09.986 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:09 smithi142 ceph-mon[32508]: osdmap e47: 7 total, 7 up, 7 in 2024-09-20T18:49:10.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:09 smithi083 ceph-mon[29421]: osdmap e47: 7 total, 7 up, 7 in 2024-09-20T18:49:10.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:09 smithi083 ceph-mon[36260]: osdmap e47: 7 total, 7 up, 7 in 2024-09-20T18:49:10.903 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:10 smithi142 ceph-mon[32508]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-20T18:49:10.903 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:10 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-20T18:49:10.903 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:10 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-20T18:49:10.903 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:10 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:49:11.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:10 smithi083 ceph-mon[29421]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-20T18:49:11.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:10 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-20T18:49:11.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:10 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-20T18:49:11.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:10 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:49:11.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:10 smithi083 ceph-mon[36260]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-20T18:49:11.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:10 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-20T18:49:11.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:10 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-20T18:49:11.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:10 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:49:12.084 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:11 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:49:12.084 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:11 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:49:12.084 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:11 smithi142 ceph-mon[32508]: from='client.24253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:49:12.084 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:11 smithi142 ceph-mon[32508]: osdmap e48: 7 total, 7 up, 7 in 2024-09-20T18:49:12.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:11 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:49:12.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:11 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:49:12.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:11 smithi083 ceph-mon[29421]: from='client.24253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:49:12.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:11 smithi083 ceph-mon[29421]: osdmap e48: 7 total, 7 up, 7 in 2024-09-20T18:49:12.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:11 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:49:12.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:11 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:49:12.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:11 smithi083 ceph-mon[36260]: from='client.24253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:49:12.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:11 smithi083 ceph-mon[36260]: osdmap e48: 7 total, 7 up, 7 in 2024-09-20T18:49:12.934 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:12 smithi142 ceph-mon[32508]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-20T18:49:12.934 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:12 smithi142 ceph-mon[32508]: from='client.? 172.21.15.142:0/3750343504' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "07d16806-e2c8-4acb-b284-d75701e61622"} : dispatch 2024-09-20T18:49:12.934 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:12 smithi142 ceph-mon[32508]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "07d16806-e2c8-4acb-b284-d75701e61622"} : dispatch 2024-09-20T18:49:12.934 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:12 smithi142 ceph-mon[32508]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "07d16806-e2c8-4acb-b284-d75701e61622"}]': finished 2024-09-20T18:49:12.934 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:12 smithi142 ceph-mon[32508]: osdmap e49: 8 total, 7 up, 8 in 2024-09-20T18:49:12.934 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:12 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-20T18:49:13.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:12 smithi083 ceph-mon[29421]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-20T18:49:13.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:12 smithi083 ceph-mon[29421]: from='client.? 172.21.15.142:0/3750343504' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "07d16806-e2c8-4acb-b284-d75701e61622"} : dispatch 2024-09-20T18:49:13.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:12 smithi083 ceph-mon[29421]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "07d16806-e2c8-4acb-b284-d75701e61622"} : dispatch 2024-09-20T18:49:13.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:12 smithi083 ceph-mon[29421]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "07d16806-e2c8-4acb-b284-d75701e61622"}]': finished 2024-09-20T18:49:13.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:12 smithi083 ceph-mon[29421]: osdmap e49: 8 total, 7 up, 8 in 2024-09-20T18:49:13.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:12 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-20T18:49:13.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:12 smithi083 ceph-mon[36260]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-20T18:49:13.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:12 smithi083 ceph-mon[36260]: from='client.? 172.21.15.142:0/3750343504' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "07d16806-e2c8-4acb-b284-d75701e61622"} : dispatch 2024-09-20T18:49:13.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:12 smithi083 ceph-mon[36260]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "07d16806-e2c8-4acb-b284-d75701e61622"} : dispatch 2024-09-20T18:49:13.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:12 smithi083 ceph-mon[36260]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "07d16806-e2c8-4acb-b284-d75701e61622"}]': finished 2024-09-20T18:49:13.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:12 smithi083 ceph-mon[36260]: osdmap e49: 8 total, 7 up, 8 in 2024-09-20T18:49:13.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:12 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-20T18:49:14.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:13 smithi142 ceph-mon[32508]: from='client.? 172.21.15.142:0/2097026113' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-20T18:49:14.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:13 smithi083 ceph-mon[29421]: from='client.? 172.21.15.142:0/2097026113' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-20T18:49:14.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:13 smithi083 ceph-mon[36260]: from='client.? 172.21.15.142:0/2097026113' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-20T18:49:15.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:14 smithi142 ceph-mon[32508]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-20T18:49:15.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:14 smithi083 ceph-mon[29421]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-20T18:49:15.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:14 smithi083 ceph-mon[36260]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-20T18:49:17.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:16 smithi142 ceph-mon[32508]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-20T18:49:17.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:16 smithi083 ceph-mon[29421]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-20T18:49:17.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:16 smithi083 ceph-mon[36260]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-20T18:49:19.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:18 smithi142 ceph-mon[32508]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-20T18:49:19.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:18 smithi083 ceph-mon[29421]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-20T18:49:19.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:18 smithi083 ceph-mon[36260]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-20T18:49:20.104 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:19 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-20T18:49:20.105 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:19 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:49:20.205 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:19 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-20T18:49:20.205 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:19 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:49:20.206 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:19 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-20T18:49:20.206 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:19 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:49:20.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:20 smithi142 ceph-mon[32508]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-20T18:49:20.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:20 smithi142 ceph-mon[32508]: Deploying daemon osd.7 on smithi142 2024-09-20T18:49:21.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:20 smithi083 ceph-mon[29421]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-20T18:49:21.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:20 smithi083 ceph-mon[29421]: Deploying daemon osd.7 on smithi142 2024-09-20T18:49:21.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:20 smithi083 ceph-mon[36260]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-20T18:49:21.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:20 smithi083 ceph-mon[36260]: Deploying daemon osd.7 on smithi142 2024-09-20T18:49:22.226 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:21 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:49:22.227 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:21 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:49:22.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:21 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:49:22.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:21 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:49:22.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:21 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:49:22.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:21 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:49:23.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:22 smithi142 ceph-mon[32508]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-20T18:49:23.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:22 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:49:23.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:22 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:49:23.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:22 smithi142 ceph-mon[32508]: osdmap e50: 8 total, 7 up, 8 in 2024-09-20T18:49:23.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:22 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-20T18:49:23.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:22 smithi083 ceph-mon[29421]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-20T18:49:23.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:22 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:49:23.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:22 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:49:23.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:22 smithi083 ceph-mon[29421]: osdmap e50: 8 total, 7 up, 8 in 2024-09-20T18:49:23.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:22 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-20T18:49:23.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:22 smithi083 ceph-mon[36260]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-20T18:49:23.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:22 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:49:23.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:22 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:49:23.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:22 smithi083 ceph-mon[36260]: osdmap e50: 8 total, 7 up, 8 in 2024-09-20T18:49:23.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:22 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-20T18:49:25.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:24 smithi142 ceph-mon[32508]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-20T18:49:25.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:24 smithi083 ceph-mon[29421]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-20T18:49:25.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:24 smithi083 ceph-mon[36260]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-20T18:49:27.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:26 smithi142 ceph-mon[32508]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-20T18:49:27.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:26 smithi083 ceph-mon[29421]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-20T18:49:27.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:26 smithi083 ceph-mon[36260]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-20T18:49:27.964 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:27 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:49:27.964 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:27 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:27.964 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:27 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:28.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:27 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:49:28.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:27 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:28.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:27 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:28.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:27 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:49:28.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:27 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:28.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:27 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:28.945 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:28 smithi142 ceph-mon[32508]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-20T18:49:28.945 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:28 smithi142 ceph-mon[32508]: 1.0 scrub starts 2024-09-20T18:49:28.945 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:28 smithi142 ceph-mon[32508]: 1.0 scrub ok 2024-09-20T18:49:29.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:28 smithi083 ceph-mon[29421]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-20T18:49:29.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:28 smithi083 ceph-mon[29421]: 1.0 scrub starts 2024-09-20T18:49:29.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:28 smithi083 ceph-mon[29421]: 1.0 scrub ok 2024-09-20T18:49:29.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:28 smithi083 ceph-mon[36260]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-20T18:49:29.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:28 smithi083 ceph-mon[36260]: 1.0 scrub starts 2024-09-20T18:49:29.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:28 smithi083 ceph-mon[36260]: 1.0 scrub ok 2024-09-20T18:49:31.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:30 smithi142 ceph-mon[32508]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-20T18:49:31.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:30 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:31.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:30 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:31.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:30 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:49:31.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:30 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:49:31.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:30 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:31.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:30 smithi083 ceph-mon[29421]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-20T18:49:31.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:30 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:31.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:30 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:31.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:30 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:49:31.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:30 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:49:31.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:30 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:31.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:30 smithi083 ceph-mon[36260]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-20T18:49:31.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:30 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:31.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:30 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:31.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:30 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:49:31.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:30 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:49:31.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:30 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:32.760 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 7 on host 'smithi142' 2024-09-20T18:49:33.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:32 smithi142 ceph-mon[32508]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-20T18:49:33.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:32 smithi142 ceph-mon[32508]: from='osd.7 v2:172.21.15.142:6812/3679770984' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-20T18:49:33.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:32 smithi142 ceph-mon[32508]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-20T18:49:33.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:32 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:49:33.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:32 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:33.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:32 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:33.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:32 smithi083 ceph-mon[36260]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-20T18:49:33.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:32 smithi083 ceph-mon[36260]: from='osd.7 v2:172.21.15.142:6812/3679770984' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-20T18:49:33.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:32 smithi083 ceph-mon[36260]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-20T18:49:33.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:32 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:49:33.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:32 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:33.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:32 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:33.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:32 smithi083 ceph-mon[29421]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-20T18:49:33.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:32 smithi083 ceph-mon[29421]: from='osd.7 v2:172.21.15.142:6812/3679770984' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-20T18:49:33.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:32 smithi083 ceph-mon[29421]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-20T18:49:33.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:32 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:49:33.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:32 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:33.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:32 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:33.386 DEBUG:teuthology.orchestra.run.smithi142:osd.7> sudo journalctl -f -n 0 -u ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@osd.7.service 2024-09-20T18:49:33.388 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-09-20T18:49:33.388 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-20T18:49:33.631 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:49:34.014 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:33 smithi083 ceph-mon[29421]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-20T18:49:34.014 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:33 smithi083 ceph-mon[29421]: osdmap e51: 8 total, 7 up, 8 in 2024-09-20T18:49:34.014 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:33 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-20T18:49:34.014 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:33 smithi083 ceph-mon[29421]: from='osd.7 v2:172.21.15.142:6812/3679770984' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-20T18:49:34.014 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:33 smithi083 ceph-mon[29421]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-20T18:49:34.015 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:33 smithi083 ceph-mon[36260]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-20T18:49:34.015 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:33 smithi083 ceph-mon[36260]: osdmap e51: 8 total, 7 up, 8 in 2024-09-20T18:49:34.015 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:33 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-20T18:49:34.015 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:33 smithi083 ceph-mon[36260]: from='osd.7 v2:172.21.15.142:6812/3679770984' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-20T18:49:34.015 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:33 smithi083 ceph-mon[36260]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-20T18:49:34.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:33 smithi142 ceph-mon[32508]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-20T18:49:34.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:33 smithi142 ceph-mon[32508]: osdmap e51: 8 total, 7 up, 8 in 2024-09-20T18:49:34.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:33 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-20T18:49:34.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:33 smithi142 ceph-mon[32508]: from='osd.7 v2:172.21.15.142:6812/3679770984' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-20T18:49:34.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:33 smithi142 ceph-mon[32508]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-20T18:49:34.310 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:49:34.555 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 18:49:34 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T18:49:34.274+0000 7f376524c640 -1 osd.7 0 waiting for initial osdmap 2024-09-20T18:49:34.555 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 18:49:34 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T18:49:34.280+0000 7f375c857640 -1 osd.7 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-20T18:49:35.013 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":52,"num_osds":8,"num_up_osds":7,"osd_up_since":1726858146,"num_in_osds":8,"osd_in_since":1726858152,"num_remapped_pgs":0} 2024-09-20T18:49:35.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:34 smithi083 ceph-mon[29421]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-20T18:49:35.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:34 smithi083 ceph-mon[29421]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-09-20T18:49:35.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:34 smithi083 ceph-mon[29421]: osdmap e52: 8 total, 7 up, 8 in 2024-09-20T18:49:35.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:34 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-20T18:49:35.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:34 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-20T18:49:35.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:34 smithi083 ceph-mon[29421]: from='osd.7 ' entity='osd.7' 2024-09-20T18:49:35.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:34 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1600633758' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-20T18:49:35.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:34 smithi083 ceph-mon[36260]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-20T18:49:35.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:34 smithi083 ceph-mon[36260]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-09-20T18:49:35.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:34 smithi083 ceph-mon[36260]: osdmap e52: 8 total, 7 up, 8 in 2024-09-20T18:49:35.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:34 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-20T18:49:35.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:34 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-20T18:49:35.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:34 smithi083 ceph-mon[36260]: from='osd.7 ' entity='osd.7' 2024-09-20T18:49:35.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:34 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1600633758' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-20T18:49:35.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:34 smithi142 ceph-mon[32508]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-20T18:49:35.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:34 smithi142 ceph-mon[32508]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-09-20T18:49:35.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:34 smithi142 ceph-mon[32508]: osdmap e52: 8 total, 7 up, 8 in 2024-09-20T18:49:35.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:34 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-20T18:49:35.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:34 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-20T18:49:35.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:34 smithi142 ceph-mon[32508]: from='osd.7 ' entity='osd.7' 2024-09-20T18:49:35.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:34 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1600633758' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-20T18:49:36.015 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-20T18:49:36.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:35 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:49:36.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:35 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:49:36.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:35 smithi142 ceph-mon[32508]: osd.7 v2:172.21.15.142:6812/3679770984 boot 2024-09-20T18:49:36.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:35 smithi142 ceph-mon[32508]: osdmap e53: 8 total, 8 up, 8 in 2024-09-20T18:49:36.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:35 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-20T18:49:36.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:35 smithi142 ceph-mon[32508]: osdmap e54: 8 total, 8 up, 8 in 2024-09-20T18:49:36.270 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:49:36.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:35 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:49:36.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:35 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:49:36.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:35 smithi083 ceph-mon[36260]: osd.7 v2:172.21.15.142:6812/3679770984 boot 2024-09-20T18:49:36.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:35 smithi083 ceph-mon[36260]: osdmap e53: 8 total, 8 up, 8 in 2024-09-20T18:49:36.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:35 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-20T18:49:36.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:35 smithi083 ceph-mon[36260]: osdmap e54: 8 total, 8 up, 8 in 2024-09-20T18:49:36.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:35 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:49:36.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:35 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:49:36.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:35 smithi083 ceph-mon[29421]: osd.7 v2:172.21.15.142:6812/3679770984 boot 2024-09-20T18:49:36.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:35 smithi083 ceph-mon[29421]: osdmap e53: 8 total, 8 up, 8 in 2024-09-20T18:49:36.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:35 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-20T18:49:36.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:35 smithi083 ceph-mon[29421]: osdmap e54: 8 total, 8 up, 8 in 2024-09-20T18:49:37.026 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:49:37.026 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:36 smithi083 ceph-mon[29421]: pgmap v182: 1 pgs: 1 remapped; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-09-20T18:49:37.027 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:36 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:37.027 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:36 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:37.027 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:36 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-20T18:49:37.027 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:36 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-20T18:49:37.027 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:36 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-20T18:49:37.028 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:36 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-20T18:49:37.028 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:36 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:37.028 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:36 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:49:37.028 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:36 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:49:37.028 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:36 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:37.029 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:36 smithi083 ceph-mon[29421]: osdmap e55: 8 total, 8 up, 8 in 2024-09-20T18:49:37.029 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:36 smithi083 ceph-mon[36260]: pgmap v182: 1 pgs: 1 remapped; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-09-20T18:49:37.029 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:36 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:37.029 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:36 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:37.029 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:36 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-20T18:49:37.029 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:36 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-20T18:49:37.030 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:36 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-20T18:49:37.030 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:36 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-20T18:49:37.030 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:36 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:37.030 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:36 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:49:37.030 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:36 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:49:37.030 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:36 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:37.030 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:36 smithi083 ceph-mon[36260]: osdmap e55: 8 total, 8 up, 8 in 2024-09-20T18:49:37.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:36 smithi142 ceph-mon[32508]: pgmap v182: 1 pgs: 1 remapped; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-09-20T18:49:37.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:36 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:37.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:36 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:37.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:36 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-20T18:49:37.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:36 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-20T18:49:37.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:36 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-20T18:49:37.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:36 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-20T18:49:37.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:36 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:37.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:36 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:49:37.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:36 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:49:37.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:36 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:37.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:36 smithi142 ceph-mon[32508]: osdmap e55: 8 total, 8 up, 8 in 2024-09-20T18:49:37.686 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":55,"num_osds":8,"num_up_osds":8,"osd_up_since":1726858174,"num_in_osds":8,"osd_in_since":1726858152,"num_remapped_pgs":0} 2024-09-20T18:49:37.687 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-20T18:49:37.933 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:49:38.175 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:37 smithi142 ceph-mon[32508]: Detected new or changed devices on smithi142 2024-09-20T18:49:38.175 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:37 smithi142 ceph-mon[32508]: Adjusting osd_memory_target on smithi142 to 4272M 2024-09-20T18:49:38.175 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:37 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2276617242' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-20T18:49:38.184 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:37 smithi083 ceph-mon[29421]: Detected new or changed devices on smithi142 2024-09-20T18:49:38.184 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:37 smithi083 ceph-mon[29421]: Adjusting osd_memory_target on smithi142 to 4272M 2024-09-20T18:49:38.184 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:37 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2276617242' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-20T18:49:38.184 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:37 smithi083 ceph-mon[36260]: Detected new or changed devices on smithi142 2024-09-20T18:49:38.184 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:37 smithi083 ceph-mon[36260]: Adjusting osd_memory_target on smithi142 to 4272M 2024-09-20T18:49:38.184 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:37 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2276617242' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-20T18:49:38.580 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:49:38.581 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":55,"fsid":"1ed5a50a-7780-11ef-baf5-efdc52797490","created":"2024-09-20T18:44:10.399800+0000","modified":"2024-09-20T18:49:36.538976+0000","last_up_change":"2024-09-20T18:49:34.836377+0000","last_in_change":"2024-09-20T18:49:12.342332+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-20T18:47:15.285795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c714e1ec-f0c3-4054-9792-61b84e53eaa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6801","nonce":2574084049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2574084049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2574084049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6803","nonce":2574084049}]},"public_addr":"172.21.15.83:6801/2574084049","cluster_addr":"172.21.15.83:6802/2574084049","heartbeat_back_addr":"172.21.15.83:6804/2574084049","heartbeat_front_addr":"172.21.15.83:6803/2574084049","state":["exists","up"]},{"osd":1,"uuid":"9dc0f2a8-bafa-4a42-88f5-0301dd58f78d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6805","nonce":4213285798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":4213285798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":4213285798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6807","nonce":4213285798}]},"public_addr":"172.21.15.83:6805/4213285798","cluster_addr":"172.21.15.83:6806/4213285798","heartbeat_back_addr":"172.21.15.83:6808/4213285798","heartbeat_front_addr":"172.21.15.83:6807/4213285798","state":["exists","up"]},{"osd":2,"uuid":"448ab692-ec41-44dc-b2d5-dbefd255b3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":805947571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":805947571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":805947571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6811","nonce":805947571}]},"public_addr":"172.21.15.83:6809/805947571","cluster_addr":"172.21.15.83:6810/805947571","heartbeat_back_addr":"172.21.15.83:6812/805947571","heartbeat_front_addr":"172.21.15.83:6811/805947571","state":["exists","up"]},{"osd":3,"uuid":"7cda5e50-e22e-4d4a-823b-53c8f5061193","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":987237318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":987237318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":987237318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":987237318}]},"public_addr":"172.21.15.83:6813/987237318","cluster_addr":"172.21.15.83:6814/987237318","heartbeat_back_addr":"172.21.15.83:6816/987237318","heartbeat_front_addr":"172.21.15.83:6815/987237318","state":["exists","up"]},{"osd":4,"uuid":"3d62654d-d31d-4310-9934-49e694dd17d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":2096684065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":2096684065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6803","nonce":2096684065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":2096684065}]},"public_addr":"172.21.15.142:6800/2096684065","cluster_addr":"172.21.15.142:6801/2096684065","heartbeat_back_addr":"172.21.15.142:6803/2096684065","heartbeat_front_addr":"172.21.15.142:6802/2096684065","state":["exists","up"]},{"osd":5,"uuid":"b3e4e21b-639f-4d25-8bbf-ea8922a9222a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":2500466067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6805","nonce":2500466067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":2500466067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":2500466067}]},"public_addr":"172.21.15.142:6804/2500466067","cluster_addr":"172.21.15.142:6805/2500466067","heartbeat_back_addr":"172.21.15.142:6807/2500466067","heartbeat_front_addr":"172.21.15.142:6806/2500466067","state":["exists","up"]},{"osd":6,"uuid":"c5db7107-ccb4-4ae2-9e63-f444897942e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1688186226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6809","nonce":1688186226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6811","nonce":1688186226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1688186226}]},"public_addr":"172.21.15.142:6808/1688186226","cluster_addr":"172.21.15.142:6809/1688186226","heartbeat_back_addr":"172.21.15.142:6811/1688186226","heartbeat_front_addr":"172.21.15.142:6810/1688186226","state":["exists","up"]},{"osd":7,"uuid":"07d16806-e2c8-4acb-b284-d75701e61622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3679770984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6813","nonce":3679770984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6815","nonce":3679770984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3679770984}]},"public_addr":"172.21.15.142:6812/3679770984","cluster_addr":"172.21.15.142:6813/3679770984","heartbeat_back_addr":"172.21.15.142:6815/3679770984","heartbeat_front_addr":"172.21.15.142:6814/3679770984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T18:48:47.702826+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T18:49:21.418689+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T18:48:16.168114+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T18:49:10.120530+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T18:49:21.754949+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T18:48:36.227535+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T18:49:05.282644+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T18:49:33.595929+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/4267132131":"2024-09-21T18:44:56.985620+0000","172.21.15.83:0/3009092807":"2024-09-21T18:44:56.985620+0000","172.21.15.83:0/3875445393":"2024-09-21T18:44:56.985620+0000","172.21.15.83:6800/2985596191":"2024-09-21T18:44:56.985620+0000","172.21.15.83:0/220746284":"2024-09-21T18:44:32.434180+0000","172.21.15.83:0/3777331122":"2024-09-21T18:44:32.434180+0000","172.21.15.83:6800/3525856829":"2024-09-21T18:44:32.434180+0000","172.21.15.83:0/3474253603":"2024-09-21T18:44:32.434180+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-20T18:49:39.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:38 smithi142 ceph-mon[32508]: pgmap v185: 1 pgs: 1 remapped; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-09-20T18:49:39.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:38 smithi142 ceph-mon[32508]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-09-20T18:49:39.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:38 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/483111502' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-20T18:49:39.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:38 smithi083 ceph-mon[29421]: pgmap v185: 1 pgs: 1 remapped; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-09-20T18:49:39.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:38 smithi083 ceph-mon[29421]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-09-20T18:49:39.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:38 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/483111502' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-20T18:49:39.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:38 smithi083 ceph-mon[36260]: pgmap v185: 1 pgs: 1 remapped; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-09-20T18:49:39.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:38 smithi083 ceph-mon[36260]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-09-20T18:49:39.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:38 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/483111502' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-20T18:49:39.364 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-20T18:47:15.285795+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-20T18:49:39.364 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd pool get .mgr pg_num 2024-09-20T18:49:39.605 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:49:40.190 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-09-20T18:49:40.983 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi083 2024-09-20T18:49:40.983 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph orch apply rgw foo.a --placement '1;smithi083=foo.a' 2024-09-20T18:49:41.242 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:49:41.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:40 smithi142 ceph-mon[32508]: pgmap v186: 1 pgs: 1 remapped; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-09-20T18:49:41.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:40 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:49:41.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:40 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:49:41.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:40 smithi142 ceph-mon[32508]: osdmap e56: 8 total, 8 up, 8 in 2024-09-20T18:49:41.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:40 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/746813259' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-20T18:49:41.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:40 smithi083 ceph-mon[29421]: pgmap v186: 1 pgs: 1 remapped; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-09-20T18:49:41.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:40 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:49:41.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:40 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:49:41.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:40 smithi083 ceph-mon[29421]: osdmap e56: 8 total, 8 up, 8 in 2024-09-20T18:49:41.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:40 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/746813259' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-20T18:49:41.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:40 smithi083 ceph-mon[36260]: pgmap v186: 1 pgs: 1 remapped; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-09-20T18:49:41.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:40 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:49:41.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:40 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:49:41.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:40 smithi083 ceph-mon[36260]: osdmap e56: 8 total, 8 up, 8 in 2024-09-20T18:49:41.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:40 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/746813259' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-20T18:49:41.918 INFO:teuthology.orchestra.run.smithi142.stdout:Scheduled rgw.foo.a update... 2024-09-20T18:49:42.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:41 smithi142 ceph-mon[32508]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-09-20T18:49:42.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:41 smithi142 ceph-mon[32508]: Cluster is now healthy 2024-09-20T18:49:42.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:41 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:42.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:41 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:49:42.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:41 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:49:42.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:41 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:49:42.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:41 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:42.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:41 smithi083 ceph-mon[29421]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-09-20T18:49:42.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:41 smithi083 ceph-mon[29421]: Cluster is now healthy 2024-09-20T18:49:42.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:41 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:42.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:41 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:49:42.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:41 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:49:42.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:41 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:49:42.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:41 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:42.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:41 smithi083 ceph-mon[36260]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-09-20T18:49:42.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:41 smithi083 ceph-mon[36260]: Cluster is now healthy 2024-09-20T18:49:42.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:41 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:42.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:41 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:49:42.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:41 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:49:42.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:41 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:49:42.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:41 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:42.571 DEBUG:teuthology.orchestra.run.smithi083:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@rgw.foo.a.service 2024-09-20T18:49:42.573 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi142 2024-09-20T18:49:42.573 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd pool create datapool 3 3 replicated 2024-09-20T18:49:42.819 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:49:43.195 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:42 smithi142 ceph-mon[32508]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-20T18:49:43.195 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:42 smithi142 ceph-mon[32508]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi083=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:49:43.195 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:42 smithi142 ceph-mon[32508]: Saving service rgw.foo.a spec with placement smithi083=foo.a;count:1 2024-09-20T18:49:43.195 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:42 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]} : dispatch 2024-09-20T18:49:43.195 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:42 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-20T18:49:43.195 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:42 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:43.196 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:42 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:49:43.196 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:42 smithi142 ceph-mon[32508]: Deploying daemon rgw.foo.a on smithi083 2024-09-20T18:49:43.277 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:42 smithi083 ceph-mon[36260]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-20T18:49:43.277 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:42 smithi083 ceph-mon[36260]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi083=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:49:43.277 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:42 smithi083 ceph-mon[36260]: Saving service rgw.foo.a spec with placement smithi083=foo.a;count:1 2024-09-20T18:49:43.278 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:42 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]} : dispatch 2024-09-20T18:49:43.278 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:42 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-20T18:49:43.278 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:42 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:43.278 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:42 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:49:43.278 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:42 smithi083 ceph-mon[36260]: Deploying daemon rgw.foo.a on smithi083 2024-09-20T18:49:43.278 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:42 smithi083 ceph-mon[29421]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-20T18:49:43.278 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:42 smithi083 ceph-mon[29421]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi083=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:49:43.278 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:42 smithi083 ceph-mon[29421]: Saving service rgw.foo.a spec with placement smithi083=foo.a;count:1 2024-09-20T18:49:43.278 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:42 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]} : dispatch 2024-09-20T18:49:43.278 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:42 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-20T18:49:43.278 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:42 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:43.279 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:42 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:49:43.279 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:42 smithi083 ceph-mon[29421]: Deploying daemon rgw.foo.a on smithi083 2024-09-20T18:49:43.938 INFO:journalctl@ceph.rgw.foo.a.smithi083.stdout:Sep 20 18:49:43 smithi083 systemd[1]: Starting Ceph rgw.foo.a for 1ed5a50a-7780-11ef-baf5-efdc52797490... 2024-09-20T18:49:43.947 INFO:teuthology.orchestra.run.smithi142.stderr:pool 'datapool' created 2024-09-20T18:49:44.191 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:43 smithi083 ceph-mon[29421]: from='client.? 172.21.15.142:0/3814615441' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} : dispatch 2024-09-20T18:49:44.191 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:43 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} : dispatch 2024-09-20T18:49:44.191 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:43 smithi083 ceph-mon[36260]: from='client.? 172.21.15.142:0/3814615441' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} : dispatch 2024-09-20T18:49:44.191 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:43 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} : dispatch 2024-09-20T18:49:44.191 INFO:journalctl@ceph.rgw.foo.a.smithi083.stdout:Sep 20 18:49:43 smithi083 podman[61158]: 2024-09-20 18:49:43.824794495 +0000 UTC m=+0.015653879 image pull 441f9a3fe51528ef653b21f42e9d884db6124a0c7d482e865382bdfdb9ec979c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2 2024-09-20T18:49:44.191 INFO:journalctl@ceph.rgw.foo.a.smithi083.stdout:Sep 20 18:49:43 smithi083 podman[61158]: 2024-09-20 18:49:43.955327968 +0000 UTC m=+0.146187378 container create a80fe9a0c509df35e57ee3fd5ba360c96e33d10cc3deea23c869db363a88a656 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-rgw-foo-a, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-20T18:49:44.256 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:43 smithi142 ceph-mon[32508]: from='client.? 172.21.15.142:0/3814615441' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} : dispatch 2024-09-20T18:49:44.256 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:43 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} : dispatch 2024-09-20T18:49:44.554 INFO:journalctl@ceph.rgw.foo.a.smithi083.stdout:Sep 20 18:49:44 smithi083 podman[61158]: 2024-09-20 18:49:44.174709529 +0000 UTC m=+0.365568906 container init a80fe9a0c509df35e57ee3fd5ba360c96e33d10cc3deea23c869db363a88a656 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-rgw-foo-a, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-20T18:49:44.554 INFO:journalctl@ceph.rgw.foo.a.smithi083.stdout:Sep 20 18:49:44 smithi083 podman[61158]: 2024-09-20 18:49:44.177487104 +0000 UTC m=+0.368346474 container start a80fe9a0c509df35e57ee3fd5ba360c96e33d10cc3deea23c869db363a88a656 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-rgw-foo-a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-20T18:49:44.554 INFO:journalctl@ceph.rgw.foo.a.smithi083.stdout:Sep 20 18:49:44 smithi083 bash[61158]: a80fe9a0c509df35e57ee3fd5ba360c96e33d10cc3deea23c869db363a88a656 2024-09-20T18:49:44.554 INFO:journalctl@ceph.rgw.foo.a.smithi083.stdout:Sep 20 18:49:44 smithi083 systemd[1]: Started Ceph rgw.foo.a for 1ed5a50a-7780-11ef-baf5-efdc52797490. 2024-09-20T18:49:44.567 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- rbd pool init datapool 2024-09-20T18:49:44.816 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:49:45.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:44 smithi083 ceph-mon[29421]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-20T18:49:45.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:44 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-20T18:49:45.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:44 smithi083 ceph-mon[29421]: osdmap e57: 8 total, 8 up, 8 in 2024-09-20T18:49:45.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:44 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:45.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:44 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:45.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:44 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:45.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:44 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:45.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:44 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:45.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:44 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:49:45.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:44 smithi083 ceph-mon[36260]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-20T18:49:45.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:44 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-20T18:49:45.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:44 smithi083 ceph-mon[36260]: osdmap e57: 8 total, 8 up, 8 in 2024-09-20T18:49:45.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:44 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:45.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:44 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:45.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:44 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:45.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:44 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:45.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:44 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:45.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:44 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:49:45.235 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:44 smithi142 ceph-mon[32508]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-20T18:49:45.236 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:44 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-20T18:49:45.236 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:44 smithi142 ceph-mon[32508]: osdmap e57: 8 total, 8 up, 8 in 2024-09-20T18:49:45.236 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:44 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:45.236 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:44 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:45.236 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:44 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:45.236 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:44 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:45.236 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:44 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:45.236 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:44 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:49:46.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:45 smithi142 ceph-mon[32508]: Saving service rgw.foo.a spec with placement smithi083=foo.a;count:1 2024-09-20T18:49:46.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:45 smithi142 ceph-mon[32508]: osdmap e58: 8 total, 8 up, 8 in 2024-09-20T18:49:46.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:45 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2407373972' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"} : dispatch 2024-09-20T18:49:46.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:45 smithi142 ceph-mon[32508]: from='client.? 172.21.15.142:0/3221361152' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "datapool","app": "rbd"} : dispatch 2024-09-20T18:49:46.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:45 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "datapool","app": "rbd"} : dispatch 2024-09-20T18:49:46.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:45 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:46.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:45 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:46.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:45 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:49:46.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:45 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:49:46.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:45 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2407373972' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-20T18:49:46.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:45 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-20T18:49:46.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:45 smithi142 ceph-mon[32508]: osdmap e59: 8 total, 8 up, 8 in 2024-09-20T18:49:46.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:45 smithi083 ceph-mon[29421]: Saving service rgw.foo.a spec with placement smithi083=foo.a;count:1 2024-09-20T18:49:46.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:45 smithi083 ceph-mon[29421]: osdmap e58: 8 total, 8 up, 8 in 2024-09-20T18:49:46.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:45 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2407373972' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"} : dispatch 2024-09-20T18:49:46.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:45 smithi083 ceph-mon[29421]: from='client.? 172.21.15.142:0/3221361152' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "datapool","app": "rbd"} : dispatch 2024-09-20T18:49:46.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:45 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "datapool","app": "rbd"} : dispatch 2024-09-20T18:49:46.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:45 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:46.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:45 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:46.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:45 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:49:46.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:45 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:49:46.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:45 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2407373972' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-20T18:49:46.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:45 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-20T18:49:46.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:45 smithi083 ceph-mon[29421]: osdmap e59: 8 total, 8 up, 8 in 2024-09-20T18:49:46.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:45 smithi083 ceph-mon[36260]: Saving service rgw.foo.a spec with placement smithi083=foo.a;count:1 2024-09-20T18:49:46.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:45 smithi083 ceph-mon[36260]: osdmap e58: 8 total, 8 up, 8 in 2024-09-20T18:49:46.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:45 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2407373972' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"} : dispatch 2024-09-20T18:49:46.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:45 smithi083 ceph-mon[36260]: from='client.? 172.21.15.142:0/3221361152' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "datapool","app": "rbd"} : dispatch 2024-09-20T18:49:46.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:45 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "datapool","app": "rbd"} : dispatch 2024-09-20T18:49:46.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:45 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:46.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:45 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:46.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:45 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:49:46.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:45 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:49:46.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:45 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2407373972' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-20T18:49:46.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:45 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-20T18:49:46.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:45 smithi083 ceph-mon[36260]: osdmap e59: 8 total, 8 up, 8 in 2024-09-20T18:49:47.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:46 smithi142 ceph-mon[32508]: pgmap v192: 36 pgs: 2 creating+peering, 1 active+clean, 33 unknown; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-20T18:49:47.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:46 smithi142 ceph-mon[32508]: Checking dashboard <-> RGW credentials 2024-09-20T18:49:47.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:46 smithi142 ceph-mon[32508]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-20T18:49:47.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:46 smithi142 ceph-mon[32508]: osdmap e60: 8 total, 8 up, 8 in 2024-09-20T18:49:47.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:46 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1555339426' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-20T18:49:47.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:46 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3694956813' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-20T18:49:47.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:46 smithi083 ceph-mon[29421]: pgmap v192: 36 pgs: 2 creating+peering, 1 active+clean, 33 unknown; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-20T18:49:47.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:46 smithi083 ceph-mon[29421]: Checking dashboard <-> RGW credentials 2024-09-20T18:49:47.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:46 smithi083 ceph-mon[29421]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-20T18:49:47.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:46 smithi083 ceph-mon[29421]: osdmap e60: 8 total, 8 up, 8 in 2024-09-20T18:49:47.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:46 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1555339426' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-20T18:49:47.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:46 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3694956813' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-20T18:49:47.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:46 smithi083 ceph-mon[36260]: pgmap v192: 36 pgs: 2 creating+peering, 1 active+clean, 33 unknown; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-20T18:49:47.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:46 smithi083 ceph-mon[36260]: Checking dashboard <-> RGW credentials 2024-09-20T18:49:47.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:46 smithi083 ceph-mon[36260]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-20T18:49:47.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:46 smithi083 ceph-mon[36260]: osdmap e60: 8 total, 8 up, 8 in 2024-09-20T18:49:47.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:46 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1555339426' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-20T18:49:47.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:46 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3694956813' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-20T18:49:48.497 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph orch apply iscsi datapool admin admin --trusted_ip_list 172.21.15.142 --placement '1;smithi142=iscsi.a' 2024-09-20T18:49:48.523 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:48 smithi142 ceph-mon[32508]: pgmap v195: 68 pgs: 6 creating+peering, 1 active+clean, 61 unknown; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-09-20T18:49:48.523 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:48 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:48.523 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:48 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1555339426' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-20T18:49:48.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:48 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3694956813' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-20T18:49:48.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:48 smithi142 ceph-mon[32508]: osdmap e61: 8 total, 8 up, 8 in 2024-09-20T18:49:48.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:48 smithi083 ceph-mon[29421]: pgmap v195: 68 pgs: 6 creating+peering, 1 active+clean, 61 unknown; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-09-20T18:49:48.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:48 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:48.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:48 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1555339426' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-20T18:49:48.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:48 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3694956813' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-20T18:49:48.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:48 smithi083 ceph-mon[29421]: osdmap e61: 8 total, 8 up, 8 in 2024-09-20T18:49:48.554 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:48 smithi083 ceph-mon[36260]: pgmap v195: 68 pgs: 6 creating+peering, 1 active+clean, 61 unknown; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-09-20T18:49:48.554 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:48 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:48.554 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:48 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1555339426' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-20T18:49:48.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:48 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3694956813' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-20T18:49:48.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:48 smithi083 ceph-mon[36260]: osdmap e61: 8 total, 8 up, 8 in 2024-09-20T18:49:48.748 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:49:49.385 INFO:teuthology.orchestra.run.smithi142.stdout:Scheduled iscsi.datapool update... 2024-09-20T18:49:49.928 INFO:tasks.cephadm:Distributing iscsi-gateway.cfg... 2024-09-20T18:49:49.929 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-20T18:49:49.929 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-20T18:49:49.967 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-20T18:49:49.967 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-20T18:49:50.003 DEBUG:teuthology.orchestra.run.smithi142:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@iscsi.iscsi.a.service 2024-09-20T18:49:50.046 INFO:tasks.cephadm:Adding prometheus.a on smithi142 2024-09-20T18:49:50.047 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph orch apply prometheus '1;smithi142=a' 2024-09-20T18:49:50.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:49 smithi142 ceph-mon[32508]: osdmap e62: 8 total, 8 up, 8 in 2024-09-20T18:49:50.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:49 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1555339426' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-20T18:49:50.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:49 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3694956813' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-20T18:49:50.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:49 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:50.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:49 smithi083 ceph-mon[29421]: osdmap e62: 8 total, 8 up, 8 in 2024-09-20T18:49:50.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:49 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1555339426' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-20T18:49:50.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:49 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3694956813' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-20T18:49:50.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:49 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:50.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:49 smithi083 ceph-mon[36260]: osdmap e62: 8 total, 8 up, 8 in 2024-09-20T18:49:50.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:49 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1555339426' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-20T18:49:50.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:49 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3694956813' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-20T18:49:50.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:49 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:50.332 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:49:50.997 INFO:teuthology.orchestra.run.smithi142.stdout:Scheduled prometheus update... 2024-09-20T18:49:51.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:50 smithi142 ceph-mon[32508]: pgmap v198: 100 pgs: 11 creating+peering, 22 active+clean, 67 unknown; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 1 op/s 2024-09-20T18:49:51.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:50 smithi142 ceph-mon[32508]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.142", "placement": "1;smithi142=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:49:51.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:50 smithi142 ceph-mon[32508]: Saving service iscsi.datapool spec with placement smithi142=iscsi.a;count:1 2024-09-20T18:49:51.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:50 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1555339426' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-20T18:49:51.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:50 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3694956813' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-20T18:49:51.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:50 smithi142 ceph-mon[32508]: osdmap e63: 8 total, 8 up, 8 in 2024-09-20T18:49:51.303 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:50 smithi083 ceph-mon[36260]: pgmap v198: 100 pgs: 11 creating+peering, 22 active+clean, 67 unknown; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 1 op/s 2024-09-20T18:49:51.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:50 smithi083 ceph-mon[36260]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.142", "placement": "1;smithi142=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:49:51.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:50 smithi083 ceph-mon[36260]: Saving service iscsi.datapool spec with placement smithi142=iscsi.a;count:1 2024-09-20T18:49:51.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:50 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1555339426' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-20T18:49:51.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:50 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3694956813' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-20T18:49:51.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:50 smithi083 ceph-mon[36260]: osdmap e63: 8 total, 8 up, 8 in 2024-09-20T18:49:51.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:50 smithi083 ceph-mon[29421]: pgmap v198: 100 pgs: 11 creating+peering, 22 active+clean, 67 unknown; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 1 op/s 2024-09-20T18:49:51.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:50 smithi083 ceph-mon[29421]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.142", "placement": "1;smithi142=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:49:51.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:50 smithi083 ceph-mon[29421]: Saving service iscsi.datapool spec with placement smithi142=iscsi.a;count:1 2024-09-20T18:49:51.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:50 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1555339426' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-20T18:49:51.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:50 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3694956813' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-20T18:49:51.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:50 smithi083 ceph-mon[29421]: osdmap e63: 8 total, 8 up, 8 in 2024-09-20T18:49:51.602 DEBUG:teuthology.orchestra.run.smithi142:prometheus.a> sudo journalctl -f -n 0 -u ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@prometheus.a.service 2024-09-20T18:49:51.604 INFO:tasks.cephadm:Adding node-exporter.a on smithi083 2024-09-20T18:49:51.604 INFO:tasks.cephadm:Adding node-exporter.b on smithi142 2024-09-20T18:49:51.604 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph orch apply node-exporter '2;smithi083=a;smithi142=b' 2024-09-20T18:49:51.887 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:49:51.980 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:51 smithi142 ceph-mon[32508]: osdmap e64: 8 total, 8 up, 8 in 2024-09-20T18:49:52.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:51 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1555339426' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-20T18:49:52.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:51 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3694956813' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-20T18:49:52.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:51 smithi142 ceph-mon[32508]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi142=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:49:52.244 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:51 smithi142 ceph-mon[32508]: Saving service prometheus spec with placement smithi142=a;count:1 2024-09-20T18:49:52.244 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:51 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:52.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:51 smithi083 ceph-mon[29421]: osdmap e64: 8 total, 8 up, 8 in 2024-09-20T18:49:52.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:51 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1555339426' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-20T18:49:52.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:51 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3694956813' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-20T18:49:52.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:51 smithi083 ceph-mon[29421]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi142=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:49:52.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:51 smithi083 ceph-mon[29421]: Saving service prometheus spec with placement smithi142=a;count:1 2024-09-20T18:49:52.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:51 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:52.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:51 smithi083 ceph-mon[36260]: osdmap e64: 8 total, 8 up, 8 in 2024-09-20T18:49:52.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:51 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1555339426' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-20T18:49:52.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:51 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3694956813' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-20T18:49:52.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:51 smithi083 ceph-mon[36260]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi142=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:49:52.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:51 smithi083 ceph-mon[36260]: Saving service prometheus spec with placement smithi142=a;count:1 2024-09-20T18:49:52.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:51 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:52.527 INFO:teuthology.orchestra.run.smithi142.stdout:Scheduled node-exporter update... 2024-09-20T18:49:53.180 DEBUG:teuthology.orchestra.run.smithi083:node-exporter.a> sudo journalctl -f -n 0 -u ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@node-exporter.a.service 2024-09-20T18:49:53.183 DEBUG:teuthology.orchestra.run.smithi142:node-exporter.b> sudo journalctl -f -n 0 -u ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@node-exporter.b.service 2024-09-20T18:49:53.185 INFO:tasks.cephadm:Adding alertmanager.a on smithi083 2024-09-20T18:49:53.185 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph orch apply alertmanager '1;smithi083=a' 2024-09-20T18:49:53.250 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:52 smithi142 ceph-mon[32508]: pgmap v201: 132 pgs: 21 creating+peering, 67 active+clean, 44 unknown; 578 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-20T18:49:53.251 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:52 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1555339426' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-20T18:49:53.251 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:52 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3694956813' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-20T18:49:53.251 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:52 smithi142 ceph-mon[32508]: osdmap e65: 8 total, 8 up, 8 in 2024-09-20T18:49:53.251 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:52 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1555339426' entity='client.rgw.foo.a' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-20T18:49:53.251 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:52 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3694956813' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-20T18:49:53.251 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:52 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:53.251 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:52 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1555339426' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-20T18:49:53.251 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:52 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3694956813' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-20T18:49:53.251 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:52 smithi142 ceph-mon[32508]: osdmap e66: 8 total, 8 up, 8 in 2024-09-20T18:49:53.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:52 smithi083 ceph-mon[36260]: pgmap v201: 132 pgs: 21 creating+peering, 67 active+clean, 44 unknown; 578 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-20T18:49:53.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:52 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1555339426' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-20T18:49:53.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:52 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3694956813' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-20T18:49:53.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:52 smithi083 ceph-mon[36260]: osdmap e65: 8 total, 8 up, 8 in 2024-09-20T18:49:53.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:52 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1555339426' entity='client.rgw.foo.a' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-20T18:49:53.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:52 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3694956813' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-20T18:49:53.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:52 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:53.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:52 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1555339426' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-20T18:49:53.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:52 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3694956813' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-20T18:49:53.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:52 smithi083 ceph-mon[36260]: osdmap e66: 8 total, 8 up, 8 in 2024-09-20T18:49:53.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:52 smithi083 ceph-mon[29421]: pgmap v201: 132 pgs: 21 creating+peering, 67 active+clean, 44 unknown; 578 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-20T18:49:53.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:52 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1555339426' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-20T18:49:53.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:52 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3694956813' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-20T18:49:53.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:52 smithi083 ceph-mon[29421]: osdmap e65: 8 total, 8 up, 8 in 2024-09-20T18:49:53.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:52 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1555339426' entity='client.rgw.foo.a' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-20T18:49:53.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:52 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3694956813' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-20T18:49:53.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:52 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:53.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:52 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1555339426' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-20T18:49:53.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:52 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3694956813' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-20T18:49:53.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:52 smithi083 ceph-mon[29421]: osdmap e66: 8 total, 8 up, 8 in 2024-09-20T18:49:53.495 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:49:54.211 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:53 smithi142 ceph-mon[32508]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi083=a;smithi142=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:49:54.211 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:53 smithi142 ceph-mon[32508]: Saving service node-exporter spec with placement smithi083=a;smithi142=b;count:2 2024-09-20T18:49:54.211 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:53 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:54.211 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:53 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:54.211 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:53 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:54.212 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:53 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]} : dispatch 2024-09-20T18:49:54.212 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:53 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-20T18:49:54.212 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:53 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:49:54.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:53 smithi083 ceph-mon[29421]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi083=a;smithi142=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:49:54.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:53 smithi083 ceph-mon[29421]: Saving service node-exporter spec with placement smithi083=a;smithi142=b;count:2 2024-09-20T18:49:54.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:53 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:54.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:53 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:54.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:53 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:54.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:53 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]} : dispatch 2024-09-20T18:49:54.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:53 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-20T18:49:54.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:53 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:49:54.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:53 smithi083 ceph-mon[36260]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi083=a;smithi142=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:49:54.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:53 smithi083 ceph-mon[36260]: Saving service node-exporter spec with placement smithi083=a;smithi142=b;count:2 2024-09-20T18:49:54.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:53 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:54.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:53 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:54.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:53 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:54.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:53 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]} : dispatch 2024-09-20T18:49:54.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:53 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-20T18:49:54.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:53 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:49:54.349 INFO:teuthology.orchestra.run.smithi142.stdout:Scheduled alertmanager update... 2024-09-20T18:49:55.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:54 smithi142 ceph-mon[32508]: pgmap v204: 132 pgs: 16 creating+peering, 84 active+clean, 32 unknown; 578 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-20T18:49:55.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:54 smithi142 ceph-mon[32508]: Deploying daemon iscsi.iscsi.a on smithi142 2024-09-20T18:49:55.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:54 smithi142 ceph-mon[32508]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-20T18:49:55.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:54 smithi142 ceph-mon[32508]: Cluster is now healthy 2024-09-20T18:49:55.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:54 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:55.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:54 smithi083 ceph-mon[29421]: pgmap v204: 132 pgs: 16 creating+peering, 84 active+clean, 32 unknown; 578 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-20T18:49:55.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:54 smithi083 ceph-mon[29421]: Deploying daemon iscsi.iscsi.a on smithi142 2024-09-20T18:49:55.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:54 smithi083 ceph-mon[29421]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-20T18:49:55.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:54 smithi083 ceph-mon[29421]: Cluster is now healthy 2024-09-20T18:49:55.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:54 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:55.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:54 smithi083 ceph-mon[36260]: pgmap v204: 132 pgs: 16 creating+peering, 84 active+clean, 32 unknown; 578 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-20T18:49:55.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:54 smithi083 ceph-mon[36260]: Deploying daemon iscsi.iscsi.a on smithi142 2024-09-20T18:49:55.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:54 smithi083 ceph-mon[36260]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-20T18:49:55.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:54 smithi083 ceph-mon[36260]: Cluster is now healthy 2024-09-20T18:49:55.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:54 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:55.422 DEBUG:teuthology.orchestra.run.smithi083:alertmanager.a> sudo journalctl -f -n 0 -u ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@alertmanager.a.service 2024-09-20T18:49:55.425 INFO:tasks.cephadm:Adding grafana.a on smithi142 2024-09-20T18:49:55.426 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph orch apply grafana '1;smithi142=a' 2024-09-20T18:49:56.146 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:49:56.271 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:55 smithi142 ceph-mon[32508]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi083=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:49:56.271 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:55 smithi142 ceph-mon[32508]: Saving service alertmanager spec with placement smithi083=a;count:1 2024-09-20T18:49:56.271 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:55 smithi142 ceph-mon[32508]: osdmap e67: 8 total, 8 up, 8 in 2024-09-20T18:49:56.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:55 smithi083 ceph-mon[29421]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi083=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:49:56.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:55 smithi083 ceph-mon[29421]: Saving service alertmanager spec with placement smithi083=a;count:1 2024-09-20T18:49:56.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:55 smithi083 ceph-mon[29421]: osdmap e67: 8 total, 8 up, 8 in 2024-09-20T18:49:56.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:55 smithi083 ceph-mon[36260]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi083=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:49:56.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:55 smithi083 ceph-mon[36260]: Saving service alertmanager spec with placement smithi083=a;count:1 2024-09-20T18:49:56.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:55 smithi083 ceph-mon[36260]: osdmap e67: 8 total, 8 up, 8 in 2024-09-20T18:49:57.049 INFO:teuthology.orchestra.run.smithi142.stdout:Scheduled grafana update... 2024-09-20T18:49:57.079 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:56 smithi142 ceph-mon[32508]: pgmap v205: 132 pgs: 127 active+clean, 5 unknown; 581 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 54 KiB/s rd, 5.1 KiB/s wr, 130 op/s 2024-09-20T18:49:57.080 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:56 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:57.080 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:56 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:57.080 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:56 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:57.080 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:56 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:57.080 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:56 smithi142 ceph-mon[32508]: from='client.? 172.21.15.142:0/3051662376' entity='client.iscsi.iscsi.a' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-20T18:49:57.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:56 smithi083 ceph-mon[29421]: pgmap v205: 132 pgs: 127 active+clean, 5 unknown; 581 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 54 KiB/s rd, 5.1 KiB/s wr, 130 op/s 2024-09-20T18:49:57.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:56 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:57.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:56 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:57.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:56 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:57.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:56 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:57.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:56 smithi083 ceph-mon[29421]: from='client.? 172.21.15.142:0/3051662376' entity='client.iscsi.iscsi.a' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-20T18:49:57.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:56 smithi083 ceph-mon[36260]: pgmap v205: 132 pgs: 127 active+clean, 5 unknown; 581 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 54 KiB/s rd, 5.1 KiB/s wr, 130 op/s 2024-09-20T18:49:57.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:56 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:57.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:56 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:57.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:56 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:57.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:56 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:57.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:56 smithi083 ceph-mon[36260]: from='client.? 172.21.15.142:0/3051662376' entity='client.iscsi.iscsi.a' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-20T18:49:57.728 DEBUG:teuthology.orchestra.run.smithi142:grafana.a> sudo journalctl -f -n 0 -u ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@grafana.a.service 2024-09-20T18:49:57.730 INFO:tasks.cephadm:Setting up client nodes... 2024-09-20T18:49:57.731 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-20T18:49:57.989 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:49:58.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:57 smithi083 ceph-mon[36260]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-20T18:49:58.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:57 smithi083 ceph-mon[36260]: Deploying daemon prometheus.a on smithi142 2024-09-20T18:49:58.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:57 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]} : dispatch 2024-09-20T18:49:58.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:57 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 5]} : dispatch 2024-09-20T18:49:58.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:57 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]} : dispatch 2024-09-20T18:49:58.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:57 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:58.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:57 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:58.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:57 smithi083 ceph-mon[29421]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-20T18:49:58.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:57 smithi083 ceph-mon[29421]: Deploying daemon prometheus.a on smithi142 2024-09-20T18:49:58.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:57 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]} : dispatch 2024-09-20T18:49:58.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:57 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 5]} : dispatch 2024-09-20T18:49:58.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:57 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]} : dispatch 2024-09-20T18:49:58.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:57 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:58.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:57 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:58.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:57 smithi142 ceph-mon[32508]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-20T18:49:58.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:57 smithi142 ceph-mon[32508]: Deploying daemon prometheus.a on smithi142 2024-09-20T18:49:58.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:57 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]} : dispatch 2024-09-20T18:49:58.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:57 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 5]} : dispatch 2024-09-20T18:49:58.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:57 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]} : dispatch 2024-09-20T18:49:58.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:57 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:58.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:57 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:49:58.743 INFO:teuthology.orchestra.run.smithi083.stdout:[client.0] 2024-09-20T18:49:58.743 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQDWw+1mp1Y1KxAANihpvU1Ri5quxoW9WiO8KQ== 2024-09-20T18:49:59.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:58 smithi142 ceph-mon[32508]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi142=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:49:59.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:58 smithi142 ceph-mon[32508]: Saving service grafana spec with placement smithi142=a;count:1 2024-09-20T18:49:59.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:58 smithi142 ceph-mon[32508]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 65 KiB/s rd, 5.5 KiB/s wr, 157 op/s 2024-09-20T18:49:59.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:58 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]': finished 2024-09-20T18:49:59.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:58 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 5]}]': finished 2024-09-20T18:49:59.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:58 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]': finished 2024-09-20T18:49:59.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:58 smithi142 ceph-mon[32508]: osdmap e68: 8 total, 8 up, 8 in 2024-09-20T18:49:59.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:58 smithi142 ceph-mon[32508]: mgrmap e16: y(active, since 5m), standbys: x 2024-09-20T18:49:59.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:58 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2619007542' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-20T18:49:59.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:58 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2619007542' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-20T18:49:59.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:58 smithi083 ceph-mon[36260]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi142=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:49:59.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:58 smithi083 ceph-mon[36260]: Saving service grafana spec with placement smithi142=a;count:1 2024-09-20T18:49:59.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:58 smithi083 ceph-mon[36260]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 65 KiB/s rd, 5.5 KiB/s wr, 157 op/s 2024-09-20T18:49:59.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:58 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]': finished 2024-09-20T18:49:59.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:58 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 5]}]': finished 2024-09-20T18:49:59.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:58 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]': finished 2024-09-20T18:49:59.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:58 smithi083 ceph-mon[36260]: osdmap e68: 8 total, 8 up, 8 in 2024-09-20T18:49:59.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:58 smithi083 ceph-mon[36260]: mgrmap e16: y(active, since 5m), standbys: x 2024-09-20T18:49:59.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:58 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2619007542' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-20T18:49:59.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:49:58 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2619007542' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-20T18:49:59.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:58 smithi083 ceph-mon[29421]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi142=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:49:59.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:58 smithi083 ceph-mon[29421]: Saving service grafana spec with placement smithi142=a;count:1 2024-09-20T18:49:59.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:58 smithi083 ceph-mon[29421]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 65 KiB/s rd, 5.5 KiB/s wr, 157 op/s 2024-09-20T18:49:59.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:58 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]': finished 2024-09-20T18:49:59.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:58 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 5]}]': finished 2024-09-20T18:49:59.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:58 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]': finished 2024-09-20T18:49:59.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:58 smithi083 ceph-mon[29421]: osdmap e68: 8 total, 8 up, 8 in 2024-09-20T18:49:59.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:58 smithi083 ceph-mon[29421]: mgrmap e16: y(active, since 5m), standbys: x 2024-09-20T18:49:59.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:58 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2619007542' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-20T18:49:59.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:58 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2619007542' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-20T18:49:59.449 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-20T18:49:59.449 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-20T18:49:59.449 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-20T18:49:59.486 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-20T18:49:59.764 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:50:00.143 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:49:59 smithi142 ceph-mon[32508]: osdmap e69: 8 total, 8 up, 8 in 2024-09-20T18:50:00.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:49:59 smithi083 ceph-mon[29421]: osdmap e69: 8 total, 8 up, 8 in 2024-09-20T18:50:00.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:00 smithi083 ceph-mon[36260]: osdmap e69: 8 total, 8 up, 8 in 2024-09-20T18:50:00.690 INFO:teuthology.orchestra.run.smithi142.stdout:[client.1] 2024-09-20T18:50:00.691 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQDYw+1mwujyJxAABnxPGOzFcLOjOAOPQDZb0A== 2024-09-20T18:50:01.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:01 smithi142 ceph-mon[32508]: pgmap v210: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 77 KiB/s rd, 5.8 KiB/s wr, 186 op/s 2024-09-20T18:50:01.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:01 smithi142 ceph-mon[32508]: overall HEALTH_OK 2024-09-20T18:50:01.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:01 smithi142 ceph-mon[32508]: osdmap e70: 8 total, 8 up, 8 in 2024-09-20T18:50:01.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:01 smithi142 ceph-mon[32508]: from='client.? 172.21.15.142:0/2288917226' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-20T18:50:01.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:01 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-20T18:50:01.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:01 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-20T18:50:01.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:01 smithi083 ceph-mon[29421]: pgmap v210: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 77 KiB/s rd, 5.8 KiB/s wr, 186 op/s 2024-09-20T18:50:01.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:01 smithi083 ceph-mon[29421]: overall HEALTH_OK 2024-09-20T18:50:01.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:01 smithi083 ceph-mon[29421]: osdmap e70: 8 total, 8 up, 8 in 2024-09-20T18:50:01.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:01 smithi083 ceph-mon[29421]: from='client.? 172.21.15.142:0/2288917226' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-20T18:50:01.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:01 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-20T18:50:01.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:01 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-20T18:50:01.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:01 smithi083 ceph-mon[36260]: pgmap v210: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 77 KiB/s rd, 5.8 KiB/s wr, 186 op/s 2024-09-20T18:50:01.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:01 smithi083 ceph-mon[36260]: overall HEALTH_OK 2024-09-20T18:50:01.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:01 smithi083 ceph-mon[36260]: osdmap e70: 8 total, 8 up, 8 in 2024-09-20T18:50:01.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:01 smithi083 ceph-mon[36260]: from='client.? 172.21.15.142:0/2288917226' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-20T18:50:01.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:01 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-20T18:50:01.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:01 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-20T18:50:01.681 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-20T18:50:01.681 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-09-20T18:50:01.681 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-20T18:50:01.718 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph config log 1 --format=json 2024-09-20T18:50:01.971 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:50:02.659 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:50:03.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:03 smithi142 ceph-mon[32508]: pgmap v212: 132 pgs: 1 peering, 131 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 931 B/s wr, 36 op/s; 40 B/s, 1 objects/s recovering 2024-09-20T18:50:03.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:03 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/186934163' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-20T18:50:03.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:03 smithi083 ceph-mon[29421]: pgmap v212: 132 pgs: 1 peering, 131 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 931 B/s wr, 36 op/s; 40 B/s, 1 objects/s recovering 2024-09-20T18:50:03.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:03 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/186934163' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-20T18:50:03.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:03 smithi083 ceph-mon[36260]: pgmap v212: 132 pgs: 1 peering, 131 active+clean; 583 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 931 B/s wr, 36 op/s; 40 B/s, 1 objects/s recovering 2024-09-20T18:50:03.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:03 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/186934163' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-20T18:50:03.418 INFO:teuthology.orchestra.run.smithi083.stdout:[{"version":25,"timestamp":"2024-09-20T18:49:53.180676+0000","name":"","changes":[{"name":"mgr/mgr/dashboard/RGW_API_SECRET_KEY","new_value":"B6yUE8gIjNDJGlTJv1nAqGiwNbzf4gHIl8DehGmZ"}]}] 2024-09-20T18:50:03.419 INFO:tasks.ceph_manager:config epoch is 25 2024-09-20T18:50:03.419 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-20T18:50:03.419 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-20T18:50:03.419 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph mgr dump --format=json 2024-09-20T18:50:03.664 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:50:04.334 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:50:05.166 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":16,"flags":0,"active_gid":14152,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":1007846146}]},"active_addr":"172.21.15.83:6800/1007846146","active_change":"2024-09-20T18:44:56.985723+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":24106,"name":"x","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.83:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"cephadm","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":2410768464}]},{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":3132933178}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":2161728215}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":1868694097}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":3684382972}]}]} 2024-09-20T18:50:05.171 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-20T18:50:05.171 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-20T18:50:05.171 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-20T18:50:05.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:05 smithi142 ceph-mon[32508]: pgmap v213: 132 pgs: 1 peering, 131 active+clean; 583 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 13 KiB/s rd, 853 B/s wr, 33 op/s; 36 B/s, 1 objects/s recovering 2024-09-20T18:50:05.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:05 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3785205422' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-20T18:50:05.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:05 smithi083 ceph-mon[29421]: pgmap v213: 132 pgs: 1 peering, 131 active+clean; 583 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 13 KiB/s rd, 853 B/s wr, 33 op/s; 36 B/s, 1 objects/s recovering 2024-09-20T18:50:05.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:05 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3785205422' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-20T18:50:05.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:05 smithi083 ceph-mon[36260]: pgmap v213: 132 pgs: 1 peering, 131 active+clean; 583 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 13 KiB/s rd, 853 B/s wr, 33 op/s; 36 B/s, 1 objects/s recovering 2024-09-20T18:50:05.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:05 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3785205422' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-20T18:50:05.425 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:50:06.062 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:50:06.062 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":72,"fsid":"1ed5a50a-7780-11ef-baf5-efdc52797490","created":"2024-09-20T18:44:10.399800+0000","modified":"2024-09-20T18:50:05.546140+0000","last_up_change":"2024-09-20T18:49:34.836377+0000","last_in_change":"2024-09-20T18:49:12.342332+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-20T18:47:15.285795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"datapool","create_time":"2024-09-20T18:49:43.452984+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":61,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-09-20T18:49:44.217011+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-20T18:49:46.007275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-20T18:49:47.961258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-20T18:49:50.011276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c714e1ec-f0c3-4054-9792-61b84e53eaa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6801","nonce":2574084049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2574084049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2574084049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6803","nonce":2574084049}]},"public_addr":"172.21.15.83:6801/2574084049","cluster_addr":"172.21.15.83:6802/2574084049","heartbeat_back_addr":"172.21.15.83:6804/2574084049","heartbeat_front_addr":"172.21.15.83:6803/2574084049","state":["exists","up"]},{"osd":1,"uuid":"9dc0f2a8-bafa-4a42-88f5-0301dd58f78d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6805","nonce":4213285798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":4213285798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":4213285798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6807","nonce":4213285798}]},"public_addr":"172.21.15.83:6805/4213285798","cluster_addr":"172.21.15.83:6806/4213285798","heartbeat_back_addr":"172.21.15.83:6808/4213285798","heartbeat_front_addr":"172.21.15.83:6807/4213285798","state":["exists","up"]},{"osd":2,"uuid":"448ab692-ec41-44dc-b2d5-dbefd255b3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":805947571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":805947571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":805947571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6811","nonce":805947571}]},"public_addr":"172.21.15.83:6809/805947571","cluster_addr":"172.21.15.83:6810/805947571","heartbeat_back_addr":"172.21.15.83:6812/805947571","heartbeat_front_addr":"172.21.15.83:6811/805947571","state":["exists","up"]},{"osd":3,"uuid":"7cda5e50-e22e-4d4a-823b-53c8f5061193","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":987237318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":987237318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":987237318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":987237318}]},"public_addr":"172.21.15.83:6813/987237318","cluster_addr":"172.21.15.83:6814/987237318","heartbeat_back_addr":"172.21.15.83:6816/987237318","heartbeat_front_addr":"172.21.15.83:6815/987237318","state":["exists","up"]},{"osd":4,"uuid":"3d62654d-d31d-4310-9934-49e694dd17d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":2096684065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":2096684065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6803","nonce":2096684065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":2096684065}]},"public_addr":"172.21.15.142:6800/2096684065","cluster_addr":"172.21.15.142:6801/2096684065","heartbeat_back_addr":"172.21.15.142:6803/2096684065","heartbeat_front_addr":"172.21.15.142:6802/2096684065","state":["exists","up"]},{"osd":5,"uuid":"b3e4e21b-639f-4d25-8bbf-ea8922a9222a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":2500466067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6805","nonce":2500466067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":2500466067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":2500466067}]},"public_addr":"172.21.15.142:6804/2500466067","cluster_addr":"172.21.15.142:6805/2500466067","heartbeat_back_addr":"172.21.15.142:6807/2500466067","heartbeat_front_addr":"172.21.15.142:6806/2500466067","state":["exists","up"]},{"osd":6,"uuid":"c5db7107-ccb4-4ae2-9e63-f444897942e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1688186226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6809","nonce":1688186226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6811","nonce":1688186226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1688186226}]},"public_addr":"172.21.15.142:6808/1688186226","cluster_addr":"172.21.15.142:6809/1688186226","heartbeat_back_addr":"172.21.15.142:6811/1688186226","heartbeat_front_addr":"172.21.15.142:6810/1688186226","state":["exists","up"]},{"osd":7,"uuid":"07d16806-e2c8-4acb-b284-d75701e61622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3679770984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6813","nonce":3679770984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6815","nonce":3679770984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3679770984}]},"public_addr":"172.21.15.142:6812/3679770984","cluster_addr":"172.21.15.142:6813/3679770984","heartbeat_back_addr":"172.21.15.142:6815/3679770984","heartbeat_front_addr":"172.21.15.142:6814/3679770984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T18:48:47.702826+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T18:49:21.418689+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T18:49:39.137302+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T18:49:10.120530+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T18:49:21.754949+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T18:50:04.573333+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T18:49:05.282644+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T18:49:33.595929+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/4267132131":"2024-09-21T18:44:56.985620+0000","172.21.15.83:0/3009092807":"2024-09-21T18:44:56.985620+0000","172.21.15.83:0/3875445393":"2024-09-21T18:44:56.985620+0000","172.21.15.83:6800/2985596191":"2024-09-21T18:44:56.985620+0000","172.21.15.83:0/220746284":"2024-09-21T18:44:32.434180+0000","172.21.15.83:0/3777331122":"2024-09-21T18:44:32.434180+0000","172.21.15.83:6800/3525856829":"2024-09-21T18:44:32.434180+0000","172.21.15.83:0/3474253603":"2024-09-21T18:44:32.434180+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-20T18:50:06.295 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:06 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:50:06.295 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:06 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:50:06.295 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:06 smithi142 ceph-mon[32508]: osdmap e71: 8 total, 8 up, 8 in 2024-09-20T18:50:06.295 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:06 smithi142 ceph-mon[32508]: osdmap e72: 8 total, 8 up, 8 in 2024-09-20T18:50:06.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:06 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:50:06.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:06 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:50:06.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:06 smithi083 ceph-mon[29421]: osdmap e71: 8 total, 8 up, 8 in 2024-09-20T18:50:06.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:06 smithi083 ceph-mon[29421]: osdmap e72: 8 total, 8 up, 8 in 2024-09-20T18:50:06.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:06 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:50:06.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:06 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:50:06.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:06 smithi083 ceph-mon[36260]: osdmap e71: 8 total, 8 up, 8 in 2024-09-20T18:50:06.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:06 smithi083 ceph-mon[36260]: osdmap e72: 8 total, 8 up, 8 in 2024-09-20T18:50:06.743 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-20T18:50:06.743 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-20T18:50:06.988 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:50:07.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:07 smithi083 ceph-mon[29421]: pgmap v215: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 169 B/s wr, 2 op/s; 36 B/s, 2 objects/s recovering 2024-09-20T18:50:07.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:07 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3732958784' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-20T18:50:07.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:07 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:50:07.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:07 smithi083 ceph-mon[36260]: pgmap v215: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 169 B/s wr, 2 op/s; 36 B/s, 2 objects/s recovering 2024-09-20T18:50:07.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:07 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3732958784' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-20T18:50:07.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:07 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:50:07.345 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:07 smithi142 ceph-mon[32508]: pgmap v215: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 169 B/s wr, 2 op/s; 36 B/s, 2 objects/s recovering 2024-09-20T18:50:07.345 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:07 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3732958784' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-20T18:50:07.345 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:07 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:50:07.632 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:50:07.632 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":72,"fsid":"1ed5a50a-7780-11ef-baf5-efdc52797490","created":"2024-09-20T18:44:10.399800+0000","modified":"2024-09-20T18:50:05.546140+0000","last_up_change":"2024-09-20T18:49:34.836377+0000","last_in_change":"2024-09-20T18:49:12.342332+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-20T18:47:15.285795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"datapool","create_time":"2024-09-20T18:49:43.452984+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":61,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-09-20T18:49:44.217011+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-20T18:49:46.007275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-20T18:49:47.961258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-20T18:49:50.011276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c714e1ec-f0c3-4054-9792-61b84e53eaa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6801","nonce":2574084049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2574084049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2574084049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6803","nonce":2574084049}]},"public_addr":"172.21.15.83:6801/2574084049","cluster_addr":"172.21.15.83:6802/2574084049","heartbeat_back_addr":"172.21.15.83:6804/2574084049","heartbeat_front_addr":"172.21.15.83:6803/2574084049","state":["exists","up"]},{"osd":1,"uuid":"9dc0f2a8-bafa-4a42-88f5-0301dd58f78d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6805","nonce":4213285798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":4213285798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":4213285798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6807","nonce":4213285798}]},"public_addr":"172.21.15.83:6805/4213285798","cluster_addr":"172.21.15.83:6806/4213285798","heartbeat_back_addr":"172.21.15.83:6808/4213285798","heartbeat_front_addr":"172.21.15.83:6807/4213285798","state":["exists","up"]},{"osd":2,"uuid":"448ab692-ec41-44dc-b2d5-dbefd255b3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":805947571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":805947571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":805947571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6811","nonce":805947571}]},"public_addr":"172.21.15.83:6809/805947571","cluster_addr":"172.21.15.83:6810/805947571","heartbeat_back_addr":"172.21.15.83:6812/805947571","heartbeat_front_addr":"172.21.15.83:6811/805947571","state":["exists","up"]},{"osd":3,"uuid":"7cda5e50-e22e-4d4a-823b-53c8f5061193","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":987237318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":987237318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":987237318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":987237318}]},"public_addr":"172.21.15.83:6813/987237318","cluster_addr":"172.21.15.83:6814/987237318","heartbeat_back_addr":"172.21.15.83:6816/987237318","heartbeat_front_addr":"172.21.15.83:6815/987237318","state":["exists","up"]},{"osd":4,"uuid":"3d62654d-d31d-4310-9934-49e694dd17d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":2096684065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":2096684065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6803","nonce":2096684065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":2096684065}]},"public_addr":"172.21.15.142:6800/2096684065","cluster_addr":"172.21.15.142:6801/2096684065","heartbeat_back_addr":"172.21.15.142:6803/2096684065","heartbeat_front_addr":"172.21.15.142:6802/2096684065","state":["exists","up"]},{"osd":5,"uuid":"b3e4e21b-639f-4d25-8bbf-ea8922a9222a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":2500466067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6805","nonce":2500466067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":2500466067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":2500466067}]},"public_addr":"172.21.15.142:6804/2500466067","cluster_addr":"172.21.15.142:6805/2500466067","heartbeat_back_addr":"172.21.15.142:6807/2500466067","heartbeat_front_addr":"172.21.15.142:6806/2500466067","state":["exists","up"]},{"osd":6,"uuid":"c5db7107-ccb4-4ae2-9e63-f444897942e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1688186226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6809","nonce":1688186226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6811","nonce":1688186226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1688186226}]},"public_addr":"172.21.15.142:6808/1688186226","cluster_addr":"172.21.15.142:6809/1688186226","heartbeat_back_addr":"172.21.15.142:6811/1688186226","heartbeat_front_addr":"172.21.15.142:6810/1688186226","state":["exists","up"]},{"osd":7,"uuid":"07d16806-e2c8-4acb-b284-d75701e61622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3679770984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6813","nonce":3679770984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6815","nonce":3679770984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3679770984}]},"public_addr":"172.21.15.142:6812/3679770984","cluster_addr":"172.21.15.142:6813/3679770984","heartbeat_back_addr":"172.21.15.142:6815/3679770984","heartbeat_front_addr":"172.21.15.142:6814/3679770984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T18:48:47.702826+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T18:49:21.418689+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T18:49:39.137302+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T18:49:10.120530+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T18:49:21.754949+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T18:50:04.573333+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T18:49:05.282644+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T18:49:33.595929+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/4267132131":"2024-09-21T18:44:56.985620+0000","172.21.15.83:0/3009092807":"2024-09-21T18:44:56.985620+0000","172.21.15.83:0/3875445393":"2024-09-21T18:44:56.985620+0000","172.21.15.83:6800/2985596191":"2024-09-21T18:44:56.985620+0000","172.21.15.83:0/220746284":"2024-09-21T18:44:32.434180+0000","172.21.15.83:0/3777331122":"2024-09-21T18:44:32.434180+0000","172.21.15.83:6800/3525856829":"2024-09-21T18:44:32.434180+0000","172.21.15.83:0/3474253603":"2024-09-21T18:44:32.434180+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-20T18:50:08.024 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:50:07 smithi142 systemd[1]: Starting Ceph prometheus.a for 1ed5a50a-7780-11ef-baf5-efdc52797490... 2024-09-20T18:50:08.168 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph tell osd.0 flush_pg_stats 2024-09-20T18:50:08.168 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph tell osd.1 flush_pg_stats 2024-09-20T18:50:08.169 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph tell osd.2 flush_pg_stats 2024-09-20T18:50:08.169 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph tell osd.3 flush_pg_stats 2024-09-20T18:50:08.170 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph tell osd.4 flush_pg_stats 2024-09-20T18:50:08.170 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph tell osd.5 flush_pg_stats 2024-09-20T18:50:08.170 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph tell osd.6 flush_pg_stats 2024-09-20T18:50:08.171 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph tell osd.7 flush_pg_stats 2024-09-20T18:50:08.283 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:08 smithi142 ceph-mon[32508]: pgmap v217: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 787 B/s rd, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-20T18:50:08.283 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:08 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/586846783' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-20T18:50:08.283 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:50:08 smithi142 podman[59284]: 2024-09-20 18:50:08.018122322 +0000 UTC m=+0.016923332 image pull 1d3b7f56885b6dd623f1785be963aa9c195f86bc256ea454e8d02a7980b79c53 quay.io/prometheus/prometheus:v2.51.0 2024-09-20T18:50:08.283 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:50:08 smithi142 podman[59284]: 2024-09-20 18:50:08.131241414 +0000 UTC m=+0.130042503 container create 2fa4a328eeaa8b6ccb463b68569468091a044e7370d678722e8f4e9b04aa1602 (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-20T18:50:08.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:08 smithi083 ceph-mon[29421]: pgmap v217: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 787 B/s rd, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-20T18:50:08.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:08 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/586846783' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-20T18:50:08.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:08 smithi083 ceph-mon[36260]: pgmap v217: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 787 B/s rd, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-20T18:50:08.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:08 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/586846783' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-20T18:50:08.768 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:50:08.772 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:50:08.774 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:50:08 smithi142 podman[59284]: 2024-09-20 18:50:08.349738728 +0000 UTC m=+0.348539743 container init 2fa4a328eeaa8b6ccb463b68569468091a044e7370d678722e8f4e9b04aa1602 (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-20T18:50:08.774 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:50:08 smithi142 podman[59284]: 2024-09-20 18:50:08.352616559 +0000 UTC m=+0.351417566 container start 2fa4a328eeaa8b6ccb463b68569468091a044e7370d678722e8f4e9b04aa1602 (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-20T18:50:08.774 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:50:08 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[59300]: ts=2024-09-20T18:50:08.373Z caller=main.go:617 level=info msg="Starting Prometheus Server" mode=server version="(version=2.51.0, branch=HEAD, revision=c05c15512acb675e3f6cd662a6727854e93fc024)" 2024-09-20T18:50:08.774 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:50:08 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[59300]: ts=2024-09-20T18:50:08.373Z caller=main.go:622 level=info build_context="(go=go1.22.1, platform=linux/amd64, user=root@b5723e458358, date=20240319-10:54:45, tags=netgo,builtinassets,stringlabels)" 2024-09-20T18:50:08.774 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:50:08 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[59300]: ts=2024-09-20T18:50:08.373Z caller=main.go:623 level=info host_details="(Linux 5.14.0-508.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 12 15:49:37 UTC 2024 x86_64 smithi142 (none))" 2024-09-20T18:50:08.774 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:50:08 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[59300]: ts=2024-09-20T18:50:08.373Z caller=main.go:624 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-20T18:50:08.775 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:50:08 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[59300]: ts=2024-09-20T18:50:08.373Z caller=main.go:625 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-20T18:50:08.775 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:50:08 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[59300]: ts=2024-09-20T18:50:08.374Z caller=web.go:568 level=info component=web msg="Start listening for connections" address=:9095 2024-09-20T18:50:08.775 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:50:08 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[59300]: ts=2024-09-20T18:50:08.375Z caller=main.go:1129 level=info msg="Starting TSDB ..." 2024-09-20T18:50:08.775 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:50:08 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[59300]: ts=2024-09-20T18:50:08.377Z caller=tls_config.go:313 level=info component=web msg="Listening on" address=[::]:9095 2024-09-20T18:50:08.775 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:50:08 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[59300]: ts=2024-09-20T18:50:08.377Z caller=tls_config.go:316 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-20T18:50:08.775 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:50:08 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[59300]: ts=2024-09-20T18:50:08.378Z caller=head.go:616 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-20T18:50:08.775 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:50:08 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[59300]: ts=2024-09-20T18:50:08.378Z caller=head.go:698 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.924µs 2024-09-20T18:50:08.775 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:50:08 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[59300]: ts=2024-09-20T18:50:08.378Z caller=head.go:706 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-20T18:50:08.775 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:50:08 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[59300]: ts=2024-09-20T18:50:08.378Z caller=head.go:778 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-09-20T18:50:08.775 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:50:08 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[59300]: ts=2024-09-20T18:50:08.378Z caller=head.go:815 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=21.566µs wal_replay_duration=151.107µs wbl_replay_duration=120ns total_replay_duration=190.576µs 2024-09-20T18:50:08.775 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:50:08 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[59300]: ts=2024-09-20T18:50:08.379Z caller=main.go:1150 level=info fs_type=XFS_SUPER_MAGIC 2024-09-20T18:50:08.775 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:50:08 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[59300]: ts=2024-09-20T18:50:08.379Z caller=main.go:1153 level=info msg="TSDB started" 2024-09-20T18:50:08.775 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:50:08 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[59300]: ts=2024-09-20T18:50:08.379Z caller=main.go:1335 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-20T18:50:08.776 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:50:08 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[59300]: ts=2024-09-20T18:50:08.395Z caller=main.go:1372 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=15.692178ms db_storage=693ns remote_storage=1.136µs web_handler=316ns query_engine=554ns scrape=445.239µs scrape_sd=87.028µs notify=677ns notify_sd=657ns rules=14.86587ms tracing=5.79µs 2024-09-20T18:50:08.776 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:50:08 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[59300]: ts=2024-09-20T18:50:08.395Z caller=main.go:1114 level=info msg="Server is ready to receive web requests." 2024-09-20T18:50:08.776 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:50:08 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[59300]: ts=2024-09-20T18:50:08.395Z caller=manager.go:163 level=info component="rule manager" msg="Starting rule manager..." 2024-09-20T18:50:08.776 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:50:08 smithi142 bash[59284]: 2fa4a328eeaa8b6ccb463b68569468091a044e7370d678722e8f4e9b04aa1602 2024-09-20T18:50:08.776 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:50:08 smithi142 systemd[1]: Started Ceph prometheus.a for 1ed5a50a-7780-11ef-baf5-efdc52797490. 2024-09-20T18:50:08.776 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:50:08.776 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:50:08.777 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:50:08.778 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:50:08.779 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:50:08.780 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:50:09.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:09 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:50:09.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:09 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:50:09.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:09 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:50:09.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:09 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mgr module enable", "module": "prometheus"} : dispatch 2024-09-20T18:50:09.774 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Sep 20 18:50:09 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-x[33893]: ignoring --setuser ceph since I am not root 2024-09-20T18:50:09.774 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Sep 20 18:50:09 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-x[33893]: ignoring --setgroup ceph since I am not root 2024-09-20T18:50:09.774 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Sep 20 18:50:09 smithi142 ceph-mgr[33897]: -- 172.21.15.142:0/2941696684 <== mon.1 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55621c4c30e0 con 0x55621c4a1000 2024-09-20T18:50:09.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:09 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:50:09.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:09 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:50:09.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:09 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:50:09.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:09 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mgr module enable", "module": "prometheus"} : dispatch 2024-09-20T18:50:09.804 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:50:09 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ignoring --setuser ceph since I am not root 2024-09-20T18:50:09.804 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:50:09 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ignoring --setgroup ceph since I am not root 2024-09-20T18:50:09.805 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:09 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:50:09.805 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:09 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:50:09.805 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:09 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' 2024-09-20T18:50:09.805 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:09 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd={"prefix": "mgr module enable", "module": "prometheus"} : dispatch 2024-09-20T18:50:10.791 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:10 smithi142 ceph-mon[32508]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-20T18:50:10.791 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:10 smithi142 ceph-mon[32508]: mgrmap e17: y(active, since 5m), standbys: x 2024-09-20T18:50:10.800 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:10 smithi083 ceph-mon[29421]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-20T18:50:10.801 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:10 smithi083 ceph-mon[29421]: mgrmap e17: y(active, since 5m), standbys: x 2024-09-20T18:50:10.801 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:10 smithi083 ceph-mon[36260]: from='mgr.14152 172.21.15.83:0/3442719286' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-20T18:50:10.801 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:10 smithi083 ceph-mon[36260]: mgrmap e17: y(active, since 5m), standbys: x 2024-09-20T18:50:11.286 INFO:teuthology.orchestra.run.smithi083.stdout:34359738418 2024-09-20T18:50:11.286 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.0 2024-09-20T18:50:12.707 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:12 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:50:12.707 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:12 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:50:12.707 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:12 smithi083 ceph-mon[29421]: osdmap e73: 8 total, 8 up, 8 in 2024-09-20T18:50:12.707 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:12 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:50:12.707 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:12 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:50:12.707 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:12 smithi083 ceph-mon[36260]: osdmap e73: 8 total, 8 up, 8 in 2024-09-20T18:50:13.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:12 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:50:13.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:12 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:50:13.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:12 smithi142 ceph-mon[32508]: osdmap e73: 8 total, 8 up, 8 in 2024-09-20T18:50:13.608 INFO:teuthology.orchestra.run.smithi083.stdout:163208757269 2024-09-20T18:50:13.609 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.5 2024-09-20T18:50:14.060 INFO:teuthology.orchestra.run.smithi083.stdout:55834574892 2024-09-20T18:50:14.061 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.1 2024-09-20T18:50:14.687 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:14 smithi083 ceph-mon[29421]: Standby manager daemon x restarted 2024-09-20T18:50:14.688 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:14 smithi083 ceph-mon[29421]: Standby manager daemon x started 2024-09-20T18:50:14.688 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:14 smithi083 ceph-mon[29421]: from='mgr.? 172.21.15.142:0/3632381742' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-20T18:50:14.688 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:14 smithi083 ceph-mon[29421]: from='mgr.? 172.21.15.142:0/3632381742' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-20T18:50:14.688 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:14 smithi083 ceph-mon[29421]: from='mgr.? 172.21.15.142:0/3632381742' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-20T18:50:14.688 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:14 smithi083 ceph-mon[29421]: from='mgr.? 172.21.15.142:0/3632381742' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-20T18:50:14.688 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:14 smithi083 ceph-mon[36260]: Standby manager daemon x restarted 2024-09-20T18:50:14.688 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:14 smithi083 ceph-mon[36260]: Standby manager daemon x started 2024-09-20T18:50:14.688 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:14 smithi083 ceph-mon[36260]: from='mgr.? 172.21.15.142:0/3632381742' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-20T18:50:14.688 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:14 smithi083 ceph-mon[36260]: from='mgr.? 172.21.15.142:0/3632381742' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-20T18:50:14.688 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:14 smithi083 ceph-mon[36260]: from='mgr.? 172.21.15.142:0/3632381742' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-20T18:50:14.689 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:14 smithi083 ceph-mon[36260]: from='mgr.? 172.21.15.142:0/3632381742' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-20T18:50:14.716 INFO:teuthology.orchestra.run.smithi083.stdout:193273528336 2024-09-20T18:50:14.717 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.6 2024-09-20T18:50:15.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:14 smithi142 ceph-mon[32508]: Standby manager daemon x restarted 2024-09-20T18:50:15.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:14 smithi142 ceph-mon[32508]: Standby manager daemon x started 2024-09-20T18:50:15.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:14 smithi142 ceph-mon[32508]: from='mgr.? 172.21.15.142:0/3632381742' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-20T18:50:15.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:14 smithi142 ceph-mon[32508]: from='mgr.? 172.21.15.142:0/3632381742' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-20T18:50:15.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:14 smithi142 ceph-mon[32508]: from='mgr.? 172.21.15.142:0/3632381742' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-20T18:50:15.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:14 smithi142 ceph-mon[32508]: from='mgr.? 172.21.15.142:0/3632381742' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-20T18:50:15.024 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Sep 20 18:50:14 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-x[33893]: [20/Sep/2024:18:50:14] ENGINE Bus STARTING 2024-09-20T18:50:15.024 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Sep 20 18:50:14 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-x[33893]: CherryPy Checker: 2024-09-20T18:50:15.024 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Sep 20 18:50:14 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-x[33893]: The Application mounted at '' has an empty config. 2024-09-20T18:50:15.024 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Sep 20 18:50:14 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-x[33893]: 2024-09-20T18:50:15.024 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Sep 20 18:50:14 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-x[33893]: [20/Sep/2024:18:50:14] ENGINE Serving on http://:::9283 2024-09-20T18:50:15.024 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Sep 20 18:50:14 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-x[33893]: [20/Sep/2024:18:50:14] ENGINE Bus STARTED 2024-09-20T18:50:15.521 INFO:teuthology.orchestra.run.smithi083.stdout:227633266697 2024-09-20T18:50:15.522 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.7 2024-09-20T18:50:15.766 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[29421]: mgrmap e18: y(active, since 5m), standbys: x 2024-09-20T18:50:15.766 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[29421]: Active manager daemon y restarted 2024-09-20T18:50:15.766 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[29421]: Activating manager daemon y 2024-09-20T18:50:15.766 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[29421]: osdmap e74: 8 total, 8 up, 8 in 2024-09-20T18:50:15.766 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[29421]: mgrmap e19: y(active, starting, since 0.0110245s), standbys: x 2024-09-20T18:50:15.767 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-20T18:50:15.767 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-20T18:50:15.767 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-20T18:50:15.767 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-20T18:50:15.767 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-20T18:50:15.767 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-20T18:50:15.767 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-20T18:50:15.767 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-20T18:50:15.767 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-20T18:50:15.767 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-20T18:50:15.767 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-20T18:50:15.767 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-20T18:50:15.767 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-20T18:50:15.767 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-20T18:50:15.768 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-20T18:50:15.768 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-20T18:50:15.768 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[29421]: Manager daemon y is now available 2024-09-20T18:50:15.768 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:50:15.768 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:15.768 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:50:15.768 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-20T18:50:15.768 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-20T18:50:15.768 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-20T18:50:15.768 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[36260]: mgrmap e18: y(active, since 5m), standbys: x 2024-09-20T18:50:15.768 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[36260]: Active manager daemon y restarted 2024-09-20T18:50:15.769 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[36260]: Activating manager daemon y 2024-09-20T18:50:15.769 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[36260]: osdmap e74: 8 total, 8 up, 8 in 2024-09-20T18:50:15.769 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[36260]: mgrmap e19: y(active, starting, since 0.0110245s), standbys: x 2024-09-20T18:50:15.769 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-20T18:50:15.769 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-20T18:50:15.769 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-20T18:50:15.769 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-20T18:50:15.769 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-20T18:50:15.769 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-20T18:50:15.769 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-20T18:50:15.769 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-20T18:50:15.769 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-20T18:50:15.769 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-20T18:50:15.769 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-20T18:50:15.769 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-20T18:50:15.769 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-20T18:50:15.770 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-20T18:50:15.770 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-20T18:50:15.770 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-20T18:50:15.770 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[36260]: Manager daemon y is now available 2024-09-20T18:50:15.770 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:50:15.770 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:15.770 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:50:15.770 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-20T18:50:15.770 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-20T18:50:15.770 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-20T18:50:15.985 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:15 smithi142 ceph-mon[32508]: mgrmap e18: y(active, since 5m), standbys: x 2024-09-20T18:50:15.986 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:15 smithi142 ceph-mon[32508]: Active manager daemon y restarted 2024-09-20T18:50:15.986 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:15 smithi142 ceph-mon[32508]: Activating manager daemon y 2024-09-20T18:50:15.986 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:15 smithi142 ceph-mon[32508]: osdmap e74: 8 total, 8 up, 8 in 2024-09-20T18:50:15.986 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:15 smithi142 ceph-mon[32508]: mgrmap e19: y(active, starting, since 0.0110245s), standbys: x 2024-09-20T18:50:15.986 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:15 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-20T18:50:15.986 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:15 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-20T18:50:15.986 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:15 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-20T18:50:15.986 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:15 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-20T18:50:15.986 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:15 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-20T18:50:15.986 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:15 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-20T18:50:15.986 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:15 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-20T18:50:15.987 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:15 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-20T18:50:15.987 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:15 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-20T18:50:15.987 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:15 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-20T18:50:15.987 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:15 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-20T18:50:15.987 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:15 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-20T18:50:15.987 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:15 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-20T18:50:15.987 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:15 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-20T18:50:15.987 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:15 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-20T18:50:15.987 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:15 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-20T18:50:15.987 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:15 smithi142 ceph-mon[32508]: Manager daemon y is now available 2024-09-20T18:50:15.987 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:15 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:50:15.987 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:15 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:15.987 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:15 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:50:15.987 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:15 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-20T18:50:15.988 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:15 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-20T18:50:15.988 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:15 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-20T18:50:16.054 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: [20/Sep/2024:18:50:15] ENGINE Bus STARTING 2024-09-20T18:50:16.054 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: CherryPy Checker: 2024-09-20T18:50:16.054 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: The Application mounted at '' has an empty config. 2024-09-20T18:50:16.054 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: 2024-09-20T18:50:16.054 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: [20/Sep/2024:18:50:15] ENGINE Serving on http://:::9283 2024-09-20T18:50:16.054 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:50:15 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: [20/Sep/2024:18:50:15] ENGINE Bus STARTED 2024-09-20T18:50:16.200 INFO:teuthology.orchestra.run.smithi083.stdout:107374182433 2024-09-20T18:50:16.200 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.3 2024-09-20T18:50:16.554 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:50:16 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: 2024-09-20T18:50:16.293+0000 7f1d85a42640 -1 log_channel(cephadm) log [ERR] : [20/Sep/2024:18:50:16] ENGINE Error in HTTPServer.serve 2024-09-20T18:50:16.554 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:50:16 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: Traceback (most recent call last): 2024-09-20T18:50:16.555 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:50:16 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-20T18:50:16.555 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:50:16 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: self._connections.run(self.expiration_interval) 2024-09-20T18:50:16.555 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:50:16 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-20T18:50:16.555 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:50:16 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: self._run(expiration_interval) 2024-09-20T18:50:16.555 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:50:16 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-20T18:50:16.555 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:50:16 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: new_conn = self._from_server_socket(self.server.socket) 2024-09-20T18:50:16.555 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:50:16 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-20T18:50:16.555 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:50:16 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-20T18:50:16.555 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:50:16 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-20T18:50:16.555 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:50:16 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: s = self.context.wrap_socket( 2024-09-20T18:50:16.555 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:50:16 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-20T18:50:16.555 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:50:16 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: return self.sslsocket_class._create( 2024-09-20T18:50:16.555 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:50:16 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-20T18:50:16.555 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:50:16 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: self.do_handshake() 2024-09-20T18:50:16.556 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:50:16 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-20T18:50:16.556 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:50:16 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: self._sslobj.do_handshake() 2024-09-20T18:50:16.556 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:50:16 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-20T18:50:16.556 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:50:16 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: 2024-09-20T18:50:16.724 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:50:16 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:50:17.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:16 smithi142 ceph-mon[32508]: mgrmap e20: y(active, since 1.01298s), standbys: x 2024-09-20T18:50:17.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:16 smithi142 ceph-mon[32508]: [20/Sep/2024:18:50:16] ENGINE Bus STARTING 2024-09-20T18:50:17.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:16 smithi142 ceph-mon[32508]: [20/Sep/2024:18:50:16] ENGINE Serving on https://172.21.15.83:7150 2024-09-20T18:50:17.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:16 smithi142 ceph-mon[32508]: [20/Sep/2024:18:50:16] ENGINE Error in HTTPServer.serve 2024-09-20T18:50:17.024 INFO:journalctl@ceph.mon.b.smithi142.stdout: Traceback (most recent call last): 2024-09-20T18:50:17.024 INFO:journalctl@ceph.mon.b.smithi142.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-20T18:50:17.024 INFO:journalctl@ceph.mon.b.smithi142.stdout: self._connections.run(self.expiration_interval) 2024-09-20T18:50:17.024 INFO:journalctl@ceph.mon.b.smithi142.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-20T18:50:17.024 INFO:journalctl@ceph.mon.b.smithi142.stdout: self._run(expiration_interval) 2024-09-20T18:50:17.024 INFO:journalctl@ceph.mon.b.smithi142.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-20T18:50:17.024 INFO:journalctl@ceph.mon.b.smithi142.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-20T18:50:17.024 INFO:journalctl@ceph.mon.b.smithi142.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-20T18:50:17.024 INFO:journalctl@ceph.mon.b.smithi142.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-20T18:50:17.024 INFO:journalctl@ceph.mon.b.smithi142.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-20T18:50:17.024 INFO:journalctl@ceph.mon.b.smithi142.stdout: s = self.context.wrap_socket( 2024-09-20T18:50:17.025 INFO:journalctl@ceph.mon.b.smithi142.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-20T18:50:17.025 INFO:journalctl@ceph.mon.b.smithi142.stdout: return self.sslsocket_class._create( 2024-09-20T18:50:17.025 INFO:journalctl@ceph.mon.b.smithi142.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-20T18:50:17.025 INFO:journalctl@ceph.mon.b.smithi142.stdout: self.do_handshake() 2024-09-20T18:50:17.025 INFO:journalctl@ceph.mon.b.smithi142.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-20T18:50:17.025 INFO:journalctl@ceph.mon.b.smithi142.stdout: self._sslobj.do_handshake() 2024-09-20T18:50:17.025 INFO:journalctl@ceph.mon.b.smithi142.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-20T18:50:17.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:16 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:50:17.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:16 smithi142 ceph-mon[32508]: [20/Sep/2024:18:50:16] ENGINE Serving on http://172.21.15.83:8765 2024-09-20T18:50:17.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:16 smithi142 ceph-mon[32508]: [20/Sep/2024:18:50:16] ENGINE Bus STARTED 2024-09-20T18:50:17.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:16 smithi083 ceph-mon[29421]: mgrmap e20: y(active, since 1.01298s), standbys: x 2024-09-20T18:50:17.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:16 smithi083 ceph-mon[29421]: [20/Sep/2024:18:50:16] ENGINE Bus STARTING 2024-09-20T18:50:17.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:16 smithi083 ceph-mon[29421]: [20/Sep/2024:18:50:16] ENGINE Serving on https://172.21.15.83:7150 2024-09-20T18:50:17.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:16 smithi083 ceph-mon[29421]: [20/Sep/2024:18:50:16] ENGINE Error in HTTPServer.serve 2024-09-20T18:50:17.054 INFO:journalctl@ceph.mon.a.smithi083.stdout: Traceback (most recent call last): 2024-09-20T18:50:17.054 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-20T18:50:17.054 INFO:journalctl@ceph.mon.a.smithi083.stdout: self._connections.run(self.expiration_interval) 2024-09-20T18:50:17.054 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-20T18:50:17.054 INFO:journalctl@ceph.mon.a.smithi083.stdout: self._run(expiration_interval) 2024-09-20T18:50:17.054 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-20T18:50:17.055 INFO:journalctl@ceph.mon.a.smithi083.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-20T18:50:17.055 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-20T18:50:17.055 INFO:journalctl@ceph.mon.a.smithi083.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-20T18:50:17.055 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-20T18:50:17.055 INFO:journalctl@ceph.mon.a.smithi083.stdout: s = self.context.wrap_socket( 2024-09-20T18:50:17.055 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-20T18:50:17.055 INFO:journalctl@ceph.mon.a.smithi083.stdout: return self.sslsocket_class._create( 2024-09-20T18:50:17.055 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-20T18:50:17.055 INFO:journalctl@ceph.mon.a.smithi083.stdout: self.do_handshake() 2024-09-20T18:50:17.055 INFO:journalctl@ceph.mon.a.smithi083.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-20T18:50:17.055 INFO:journalctl@ceph.mon.a.smithi083.stdout: self._sslobj.do_handshake() 2024-09-20T18:50:17.055 INFO:journalctl@ceph.mon.a.smithi083.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-20T18:50:17.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:16 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:50:17.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:16 smithi083 ceph-mon[29421]: [20/Sep/2024:18:50:16] ENGINE Serving on http://172.21.15.83:8765 2024-09-20T18:50:17.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:16 smithi083 ceph-mon[29421]: [20/Sep/2024:18:50:16] ENGINE Bus STARTED 2024-09-20T18:50:17.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:16 smithi083 ceph-mon[36260]: mgrmap e20: y(active, since 1.01298s), standbys: x 2024-09-20T18:50:17.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:16 smithi083 ceph-mon[36260]: [20/Sep/2024:18:50:16] ENGINE Bus STARTING 2024-09-20T18:50:17.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:16 smithi083 ceph-mon[36260]: [20/Sep/2024:18:50:16] ENGINE Serving on https://172.21.15.83:7150 2024-09-20T18:50:17.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:16 smithi083 ceph-mon[36260]: [20/Sep/2024:18:50:16] ENGINE Error in HTTPServer.serve 2024-09-20T18:50:17.056 INFO:journalctl@ceph.mon.c.smithi083.stdout: Traceback (most recent call last): 2024-09-20T18:50:17.056 INFO:journalctl@ceph.mon.c.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-20T18:50:17.056 INFO:journalctl@ceph.mon.c.smithi083.stdout: self._connections.run(self.expiration_interval) 2024-09-20T18:50:17.056 INFO:journalctl@ceph.mon.c.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-20T18:50:17.056 INFO:journalctl@ceph.mon.c.smithi083.stdout: self._run(expiration_interval) 2024-09-20T18:50:17.056 INFO:journalctl@ceph.mon.c.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-20T18:50:17.056 INFO:journalctl@ceph.mon.c.smithi083.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-20T18:50:17.056 INFO:journalctl@ceph.mon.c.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-20T18:50:17.056 INFO:journalctl@ceph.mon.c.smithi083.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-20T18:50:17.056 INFO:journalctl@ceph.mon.c.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-20T18:50:17.056 INFO:journalctl@ceph.mon.c.smithi083.stdout: s = self.context.wrap_socket( 2024-09-20T18:50:17.057 INFO:journalctl@ceph.mon.c.smithi083.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-20T18:50:17.057 INFO:journalctl@ceph.mon.c.smithi083.stdout: return self.sslsocket_class._create( 2024-09-20T18:50:17.057 INFO:journalctl@ceph.mon.c.smithi083.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-20T18:50:17.057 INFO:journalctl@ceph.mon.c.smithi083.stdout: self.do_handshake() 2024-09-20T18:50:17.057 INFO:journalctl@ceph.mon.c.smithi083.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-20T18:50:17.057 INFO:journalctl@ceph.mon.c.smithi083.stdout: self._sslobj.do_handshake() 2024-09-20T18:50:17.057 INFO:journalctl@ceph.mon.c.smithi083.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-20T18:50:17.057 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:16 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:50:17.057 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:16 smithi083 ceph-mon[36260]: [20/Sep/2024:18:50:16] ENGINE Serving on http://172.21.15.83:8765 2024-09-20T18:50:17.057 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:16 smithi083 ceph-mon[36260]: [20/Sep/2024:18:50:16] ENGINE Bus STARTED 2024-09-20T18:50:17.088 INFO:teuthology.orchestra.run.smithi083.stdout:77309411366 2024-09-20T18:50:17.088 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.2 2024-09-20T18:50:17.159 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:50:17.201 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:50:17.213 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:50:17.945 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:17 smithi083 ceph-mon[29421]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-20T18:50:17.945 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:17 smithi083 ceph-mon[29421]: pgmap v4: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-20T18:50:17.946 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:17 smithi083 ceph-mon[36260]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-20T18:50:17.946 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:17 smithi083 ceph-mon[36260]: pgmap v4: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-20T18:50:18.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:17 smithi142 ceph-mon[32508]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-20T18:50:18.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:17 smithi142 ceph-mon[32508]: pgmap v4: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-20T18:50:18.504 INFO:teuthology.orchestra.run.smithi083.stdout:34359738419 2024-09-20T18:50:18.763 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:50:19.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:18 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:19.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:18 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:19.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:18 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3814121284' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-20T18:50:19.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:18 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:19.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:18 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:19.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:18 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-20T18:50:19.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:18 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-20T18:50:19.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:18 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-20T18:50:19.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:18 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-20T18:50:19.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:18 smithi083 ceph-mon[29421]: Adjusting osd_memory_target on smithi142 to 3760M 2024-09-20T18:50:19.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:18 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:19.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:18 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3094726231' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-20T18:50:19.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:18 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:19.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:18 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:19.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:18 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3814121284' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-20T18:50:19.057 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:18 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:19.057 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:18 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:19.057 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:18 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-20T18:50:19.057 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:18 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-20T18:50:19.057 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:18 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-20T18:50:19.057 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:18 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-20T18:50:19.058 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:18 smithi083 ceph-mon[36260]: Adjusting osd_memory_target on smithi142 to 3760M 2024-09-20T18:50:19.058 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:18 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:19.058 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:18 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3094726231' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-20T18:50:19.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:18 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:19.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:18 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:19.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:18 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3814121284' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-20T18:50:19.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:18 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:19.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:18 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:19.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:18 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-20T18:50:19.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:18 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-20T18:50:19.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:18 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-20T18:50:19.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:18 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-20T18:50:19.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:18 smithi142 ceph-mon[32508]: Adjusting osd_memory_target on smithi142 to 3760M 2024-09-20T18:50:19.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:18 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:19.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:18 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3094726231' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-20T18:50:19.289 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738418 got 34359738419 for osd.0 2024-09-20T18:50:19.289 DEBUG:teuthology.parallel:result is None 2024-09-20T18:50:19.378 INFO:teuthology.orchestra.run.smithi083.stdout:163208757270 2024-09-20T18:50:19.932 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757269 got 163208757270 for osd.5 2024-09-20T18:50:19.932 DEBUG:teuthology.parallel:result is None 2024-09-20T18:50:20.328 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:50:20.332 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:50:20.342 INFO:teuthology.orchestra.run.smithi083.stdout:133143986203 2024-09-20T18:50:20.342 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.4 2024-09-20T18:50:20.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:20 smithi083 ceph-mon[29421]: pgmap v5: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-20T18:50:20.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:20 smithi083 ceph-mon[29421]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-20T18:50:20.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:20 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:50:20.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:20 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:50:20.554 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:20 smithi083 ceph-mon[36260]: pgmap v5: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-20T18:50:20.554 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:20 smithi083 ceph-mon[36260]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-20T18:50:20.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:20 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:50:20.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:20 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:50:20.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:20 smithi142 ceph-mon[32508]: pgmap v5: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-20T18:50:20.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:20 smithi142 ceph-mon[32508]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-20T18:50:20.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:20 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:50:20.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:20 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:50:20.829 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:50:20 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:50:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:50:21.537 INFO:teuthology.orchestra.run.smithi083.stdout:193273528337 2024-09-20T18:50:21.639 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:21 smithi083 ceph-mon[29421]: osdmap e75: 8 total, 8 up, 8 in 2024-09-20T18:50:21.639 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:21 smithi083 ceph-mon[29421]: osdmap e76: 8 total, 8 up, 8 in 2024-09-20T18:50:21.639 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:21 smithi083 ceph-mon[36260]: osdmap e75: 8 total, 8 up, 8 in 2024-09-20T18:50:21.639 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:21 smithi083 ceph-mon[36260]: osdmap e76: 8 total, 8 up, 8 in 2024-09-20T18:50:21.748 INFO:teuthology.orchestra.run.smithi083.stdout:55834574894 2024-09-20T18:50:21.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:21 smithi142 ceph-mon[32508]: osdmap e75: 8 total, 8 up, 8 in 2024-09-20T18:50:21.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:21 smithi142 ceph-mon[32508]: osdmap e76: 8 total, 8 up, 8 in 2024-09-20T18:50:21.919 INFO:teuthology.orchestra.run.smithi083.stdout:227633266699 2024-09-20T18:50:22.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:22 smithi083 ceph-mon[29421]: pgmap v8: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 36 KiB/s rd, 0 B/s wr, 15 op/s 2024-09-20T18:50:22.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:22 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1331822237' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-20T18:50:22.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:22 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2766294533' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-20T18:50:22.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:22 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2406178608' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-20T18:50:22.554 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:22 smithi083 ceph-mon[36260]: pgmap v8: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 36 KiB/s rd, 0 B/s wr, 15 op/s 2024-09-20T18:50:22.554 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:22 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1331822237' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-20T18:50:22.554 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:22 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2766294533' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-20T18:50:22.554 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:22 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2406178608' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-20T18:50:22.614 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574892 got 55834574894 for osd.1 2024-09-20T18:50:22.614 DEBUG:teuthology.parallel:result is None 2024-09-20T18:50:22.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:22 smithi142 ceph-mon[32508]: pgmap v8: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 36 KiB/s rd, 0 B/s wr, 15 op/s 2024-09-20T18:50:22.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:22 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1331822237' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-20T18:50:22.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:22 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2766294533' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-20T18:50:22.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:22 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2406178608' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-20T18:50:22.972 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266697 got 227633266699 for osd.7 2024-09-20T18:50:22.972 DEBUG:teuthology.parallel:result is None 2024-09-20T18:50:23.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:23 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:50:23.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:23 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:50:23.507 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:23 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:50:23.507 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:23 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:50:23.533 INFO:tasks.cephadm.ceph_manager.ceph:need seq 193273528336 got 193273528337 for osd.6 2024-09-20T18:50:23.533 DEBUG:teuthology.parallel:result is None 2024-09-20T18:50:23.745 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:50:23.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:23 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:50:23.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:23 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:50:23.990 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:50:24.169 INFO:teuthology.orchestra.run.smithi083.stdout:77309411368 2024-09-20T18:50:24.742 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:24 smithi083 ceph-mon[29421]: pgmap v9: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-20T18:50:24.742 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:24 smithi083 ceph-mon[29421]: osdmap e77: 8 total, 8 up, 8 in 2024-09-20T18:50:24.742 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:24 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3207196432' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-20T18:50:24.743 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:24 smithi083 ceph-mon[36260]: pgmap v9: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-20T18:50:24.743 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:24 smithi083 ceph-mon[36260]: osdmap e77: 8 total, 8 up, 8 in 2024-09-20T18:50:24.743 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:24 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3207196432' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-20T18:50:24.751 INFO:teuthology.orchestra.run.smithi083.stdout:133143986204 2024-09-20T18:50:24.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:24 smithi142 ceph-mon[32508]: pgmap v9: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-20T18:50:24.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:24 smithi142 ceph-mon[32508]: osdmap e77: 8 total, 8 up, 8 in 2024-09-20T18:50:24.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:24 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3207196432' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-20T18:50:25.432 INFO:teuthology.orchestra.run.smithi083.stdout:107374182435 2024-09-20T18:50:25.433 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:25 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/4004811214' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-20T18:50:25.433 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:25 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/4004811214' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-20T18:50:25.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:25 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/4004811214' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-20T18:50:25.819 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411366 got 77309411368 for osd.2 2024-09-20T18:50:25.819 DEBUG:teuthology.parallel:result is None 2024-09-20T18:50:26.162 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182433 got 107374182435 for osd.3 2024-09-20T18:50:26.162 DEBUG:teuthology.parallel:result is None 2024-09-20T18:50:26.722 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:26 smithi083 ceph-mon[29421]: pgmap v11: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-20T18:50:26.722 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:26 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1594256615' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-20T18:50:26.722 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:26 smithi083 ceph-mon[29421]: osdmap e78: 8 total, 8 up, 8 in 2024-09-20T18:50:26.722 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:26 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:26.722 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:26 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:26.723 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:26 smithi083 ceph-mon[36260]: pgmap v11: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-20T18:50:26.723 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:26 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1594256615' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-20T18:50:26.723 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:26 smithi083 ceph-mon[36260]: osdmap e78: 8 total, 8 up, 8 in 2024-09-20T18:50:26.723 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:26 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:26.723 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:26 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:26.748 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986203 got 133143986204 for osd.4 2024-09-20T18:50:26.748 DEBUG:teuthology.parallel:result is None 2024-09-20T18:50:26.748 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-20T18:50:26.748 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T18:50:26.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:26 smithi142 ceph-mon[32508]: pgmap v11: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-20T18:50:26.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:26 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1594256615' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-20T18:50:26.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:26 smithi142 ceph-mon[32508]: osdmap e78: 8 total, 8 up, 8 in 2024-09-20T18:50:26.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:26 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:26.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:26 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:26.775 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:50:26 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:50:27.015 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:50:27.402 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:27 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:50:27.402 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:27 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:27.402 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:27 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:27.402 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:27 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-20T18:50:27.402 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:27 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-20T18:50:27.402 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:27 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-20T18:50:27.403 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:27 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-20T18:50:27.403 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:27 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:27.403 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:27 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:50:27.403 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:27 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:50:27.671 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:50:27.672 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:27 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:50:27.672 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:27 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:27.672 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:27 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:27.672 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:27 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-20T18:50:27.672 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:27 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-20T18:50:27.672 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:27 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-20T18:50:27.672 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:27 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-20T18:50:27.672 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:27 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:27.673 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:27 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:50:27.673 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:27 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:50:27.676 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-20T18:50:27.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:27 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:50:27.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:27 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:27.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:27 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:27.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:27 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-20T18:50:27.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:27 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-20T18:50:27.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:27 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-20T18:50:27.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:27 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-20T18:50:27.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:27 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:27.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:27 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:50:27.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:27 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:50:28.361 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2024-09-20T18:50:26.752257+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":883,"num_read_kb":740,"num_write":549,"num_write_kb":1673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":590588,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":552,"ondisk_log_size":552,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":244968,"kb_used_data":7944,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":749716248,"statfs":{"total":767960285184,"available":767709437952,"internally_reserved":0,"allocated":8134656,"data_stored":4324228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":186,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.198329"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372596+0000","last_change":"2024-09-20T18:49:45.943560+0000","last_active":"2024-09-20T18:50:23.372596+0000","last_peered":"2024-09-20T18:50:23.372596+0000","last_clean":"2024-09-20T18:50:23.372596+0000","last_became_active":"2024-09-20T18:49:45.943201+0000","last_became_peered":"2024-09-20T18:49:45.943201+0000","last_unstale":"2024-09-20T18:50:23.372596+0000","last_undegraded":"2024-09-20T18:50:23.372596+0000","last_fullsized":"2024-09-20T18:50:23.372596+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:11.286122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":59,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372438+0000","last_change":"2024-09-20T18:49:47.947057+0000","last_active":"2024-09-20T18:50:23.372438+0000","last_peered":"2024-09-20T18:50:23.372438+0000","last_clean":"2024-09-20T18:50:23.372438+0000","last_became_active":"2024-09-20T18:49:47.946161+0000","last_became_peered":"2024-09-20T18:49:47.946161+0000","last_unstale":"2024-09-20T18:50:23.372438+0000","last_undegraded":"2024-09-20T18:50:23.372438+0000","last_fullsized":"2024-09-20T18:50:23.372438+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.579926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.554773+0000","last_change":"2024-09-20T18:49:49.959797+0000","last_active":"2024-09-20T18:50:25.554773+0000","last_peered":"2024-09-20T18:50:25.554773+0000","last_clean":"2024-09-20T18:50:25.554773+0000","last_became_active":"2024-09-20T18:49:49.959514+0000","last_became_peered":"2024-09-20T18:49:49.959514+0000","last_unstale":"2024-09-20T18:50:25.554773+0000","last_undegraded":"2024-09-20T18:50:25.554773+0000","last_fullsized":"2024-09-20T18:50:25.554773+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:49.805774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372453+0000","last_change":"2024-09-20T18:49:51.969687+0000","last_active":"2024-09-20T18:50:23.372453+0000","last_peered":"2024-09-20T18:50:23.372453+0000","last_clean":"2024-09-20T18:50:23.372453+0000","last_became_active":"2024-09-20T18:49:51.969421+0000","last_became_peered":"2024-09-20T18:49:51.969421+0000","last_unstale":"2024-09-20T18:50:23.372453+0000","last_undegraded":"2024-09-20T18:50:23.372453+0000","last_fullsized":"2024-09-20T18:50:23.372453+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.534291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629364+0000","last_change":"2024-09-20T18:49:52.142168+0000","last_active":"2024-09-20T18:50:24.629364+0000","last_peered":"2024-09-20T18:50:24.629364+0000","last_clean":"2024-09-20T18:50:24.629364+0000","last_became_active":"2024-09-20T18:49:52.141967+0000","last_became_peered":"2024-09-20T18:49:52.141967+0000","last_unstale":"2024-09-20T18:50:24.629364+0000","last_undegraded":"2024-09-20T18:50:24.629364+0000","last_fullsized":"2024-09-20T18:50:24.629364+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.114636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.630347+0000","last_change":"2024-09-20T18:49:45.943889+0000","last_active":"2024-09-20T18:50:24.630347+0000","last_peered":"2024-09-20T18:50:24.630347+0000","last_clean":"2024-09-20T18:50:24.630347+0000","last_became_active":"2024-09-20T18:49:45.943607+0000","last_became_peered":"2024-09-20T18:49:45.943607+0000","last_unstale":"2024-09-20T18:50:24.630347+0000","last_undegraded":"2024-09-20T18:50:24.630347+0000","last_fullsized":"2024-09-20T18:50:24.630347+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.033360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":68,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629459+0000","last_change":"2024-09-20T18:49:47.952638+0000","last_active":"2024-09-20T18:50:24.629459+0000","last_peered":"2024-09-20T18:50:24.629459+0000","last_clean":"2024-09-20T18:50:24.629459+0000","last_became_active":"2024-09-20T18:49:47.952150+0000","last_became_peered":"2024-09-20T18:49:47.952150+0000","last_unstale":"2024-09-20T18:50:24.629459+0000","last_undegraded":"2024-09-20T18:50:24.629459+0000","last_fullsized":"2024-09-20T18:50:24.629459+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:02.798315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372432+0000","last_change":"2024-09-20T18:49:49.956717+0000","last_active":"2024-09-20T18:50:23.372432+0000","last_peered":"2024-09-20T18:50:23.372432+0000","last_clean":"2024-09-20T18:50:23.372432+0000","last_became_active":"2024-09-20T18:49:49.956445+0000","last_became_peered":"2024-09-20T18:49:49.956445+0000","last_unstale":"2024-09-20T18:50:23.372432+0000","last_undegraded":"2024-09-20T18:50:23.372432+0000","last_fullsized":"2024-09-20T18:50:23.372432+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:53.841144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"59'1","reported_seq":55,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.555194+0000","last_change":"2024-09-20T18:49:45.946450+0000","last_active":"2024-09-20T18:50:25.555194+0000","last_peered":"2024-09-20T18:50:25.555194+0000","last_clean":"2024-09-20T18:50:25.555194+0000","last_became_active":"2024-09-20T18:49:45.946171+0000","last_became_peered":"2024-09-20T18:49:45.946171+0000","last_unstale":"2024-09-20T18:50:25.555194+0000","last_undegraded":"2024-09-20T18:50:25.555194+0000","last_fullsized":"2024-09-20T18:50:25.555194+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:11.299334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":59,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372776+0000","last_change":"2024-09-20T18:49:47.947755+0000","last_active":"2024-09-20T18:50:23.372776+0000","last_peered":"2024-09-20T18:50:23.372776+0000","last_clean":"2024-09-20T18:50:23.372776+0000","last_became_active":"2024-09-20T18:49:47.947448+0000","last_became_peered":"2024-09-20T18:49:47.947448+0000","last_unstale":"2024-09-20T18:50:23.372776+0000","last_undegraded":"2024-09-20T18:50:23.372776+0000","last_fullsized":"2024-09-20T18:50:23.372776+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.588680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.554705+0000","last_change":"2024-09-20T18:49:49.961448+0000","last_active":"2024-09-20T18:50:25.554705+0000","last_peered":"2024-09-20T18:50:25.554705+0000","last_clean":"2024-09-20T18:50:25.554705+0000","last_became_active":"2024-09-20T18:49:49.961188+0000","last_became_peered":"2024-09-20T18:49:49.961188+0000","last_unstale":"2024-09-20T18:50:25.554705+0000","last_undegraded":"2024-09-20T18:50:25.554705+0000","last_fullsized":"2024-09-20T18:50:25.554705+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:12.456737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.371635+0000","last_change":"2024-09-20T18:49:51.966957+0000","last_active":"2024-09-20T18:50:23.371635+0000","last_peered":"2024-09-20T18:50:23.371635+0000","last_clean":"2024-09-20T18:50:23.371635+0000","last_became_active":"2024-09-20T18:49:51.966715+0000","last_became_peered":"2024-09-20T18:49:51.966715+0000","last_unstale":"2024-09-20T18:50:23.371635+0000","last_undegraded":"2024-09-20T18:50:23.371635+0000","last_fullsized":"2024-09-20T18:50:23.371635+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:54.613818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":49,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.555029+0000","last_change":"2024-09-20T18:49:45.945004+0000","last_active":"2024-09-20T18:50:25.555029+0000","last_peered":"2024-09-20T18:50:25.555029+0000","last_clean":"2024-09-20T18:50:25.555029+0000","last_became_active":"2024-09-20T18:49:45.944703+0000","last_became_peered":"2024-09-20T18:49:45.944703+0000","last_unstale":"2024-09-20T18:50:25.555029+0000","last_undegraded":"2024-09-20T18:50:25.555029+0000","last_fullsized":"2024-09-20T18:50:25.555029+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:53.394518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":66,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372594+0000","last_change":"2024-09-20T18:49:58.999374+0000","last_active":"2024-09-20T18:50:23.372594+0000","last_peered":"2024-09-20T18:50:23.372594+0000","last_clean":"2024-09-20T18:50:23.372594+0000","last_became_active":"2024-09-20T18:49:58.995638+0000","last_became_peered":"2024-09-20T18:49:58.995638+0000","last_unstale":"2024-09-20T18:50:23.372594+0000","last_undegraded":"2024-09-20T18:50:23.372594+0000","last_fullsized":"2024-09-20T18:50:23.372594+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:05.131889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629604+0000","last_change":"2024-09-20T18:49:49.960001+0000","last_active":"2024-09-20T18:50:24.629604+0000","last_peered":"2024-09-20T18:50:24.629604+0000","last_clean":"2024-09-20T18:50:24.629604+0000","last_became_active":"2024-09-20T18:49:49.959696+0000","last_became_peered":"2024-09-20T18:49:49.959696+0000","last_unstale":"2024-09-20T18:50:24.629604+0000","last_undegraded":"2024-09-20T18:50:24.629604+0000","last_fullsized":"2024-09-20T18:50:24.629604+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.740255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.554479+0000","last_change":"2024-09-20T18:49:51.970933+0000","last_active":"2024-09-20T18:50:25.554479+0000","last_peered":"2024-09-20T18:50:25.554479+0000","last_clean":"2024-09-20T18:50:25.554479+0000","last_became_active":"2024-09-20T18:49:51.970663+0000","last_became_peered":"2024-09-20T18:49:51.970663+0000","last_unstale":"2024-09-20T18:50:25.554479+0000","last_undegraded":"2024-09-20T18:50:25.554479+0000","last_fullsized":"2024-09-20T18:50:25.554479+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:07.680885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.373280+0000","last_change":"2024-09-20T18:49:52.141482+0000","last_active":"2024-09-20T18:50:23.373280+0000","last_peered":"2024-09-20T18:50:23.373280+0000","last_clean":"2024-09-20T18:50:23.373280+0000","last_became_active":"2024-09-20T18:49:52.141170+0000","last_became_peered":"2024-09-20T18:49:52.141170+0000","last_unstale":"2024-09-20T18:50:23.373280+0000","last_undegraded":"2024-09-20T18:50:23.373280+0000","last_fullsized":"2024-09-20T18:50:23.373280+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:15.351627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372298+0000","last_change":"2024-09-20T18:49:45.950714+0000","last_active":"2024-09-20T18:50:23.372298+0000","last_peered":"2024-09-20T18:50:23.372298+0000","last_clean":"2024-09-20T18:50:23.372298+0000","last_became_active":"2024-09-20T18:49:45.950480+0000","last_became_peered":"2024-09-20T18:49:45.950480+0000","last_unstale":"2024-09-20T18:50:23.372298+0000","last_undegraded":"2024-09-20T18:50:23.372298+0000","last_fullsized":"2024-09-20T18:50:23.372298+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:07.919860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":71,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.556302+0000","last_change":"2024-09-20T18:49:47.951228+0000","last_active":"2024-09-20T18:50:25.556302+0000","last_peered":"2024-09-20T18:50:25.556302+0000","last_clean":"2024-09-20T18:50:25.556302+0000","last_became_active":"2024-09-20T18:49:47.950939+0000","last_became_peered":"2024-09-20T18:49:47.950939+0000","last_unstale":"2024-09-20T18:50:25.556302+0000","last_undegraded":"2024-09-20T18:50:25.556302+0000","last_fullsized":"2024-09-20T18:50:25.556302+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:08.025260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.555139+0000","last_change":"2024-09-20T18:49:49.958867+0000","last_active":"2024-09-20T18:50:25.555139+0000","last_peered":"2024-09-20T18:50:25.555139+0000","last_clean":"2024-09-20T18:50:25.555139+0000","last_became_active":"2024-09-20T18:49:49.958518+0000","last_became_peered":"2024-09-20T18:49:49.958518+0000","last_unstale":"2024-09-20T18:50:25.555139+0000","last_undegraded":"2024-09-20T18:50:25.555139+0000","last_fullsized":"2024-09-20T18:50:25.555139+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:55.540498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.630339+0000","last_change":"2024-09-20T18:49:52.140362+0000","last_active":"2024-09-20T18:50:24.630339+0000","last_peered":"2024-09-20T18:50:24.630339+0000","last_clean":"2024-09-20T18:50:24.630339+0000","last_became_active":"2024-09-20T18:49:52.140040+0000","last_became_peered":"2024-09-20T18:49:52.140040+0000","last_unstale":"2024-09-20T18:50:24.630339+0000","last_undegraded":"2024-09-20T18:50:24.630339+0000","last_fullsized":"2024-09-20T18:50:24.630339+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:52.839541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"59'1","reported_seq":53,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372906+0000","last_change":"2024-09-20T18:49:45.942193+0000","last_active":"2024-09-20T18:50:23.372906+0000","last_peered":"2024-09-20T18:50:23.372906+0000","last_clean":"2024-09-20T18:50:23.372906+0000","last_became_active":"2024-09-20T18:49:45.940100+0000","last_became_peered":"2024-09-20T18:49:45.940100+0000","last_unstale":"2024-09-20T18:50:23.372906+0000","last_undegraded":"2024-09-20T18:50:23.372906+0000","last_fullsized":"2024-09-20T18:50:23.372906+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:52.525377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":66,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.630180+0000","last_change":"2024-09-20T18:49:47.951215+0000","last_active":"2024-09-20T18:50:24.630180+0000","last_peered":"2024-09-20T18:50:24.630180+0000","last_clean":"2024-09-20T18:50:24.630180+0000","last_became_active":"2024-09-20T18:49:47.950909+0000","last_became_peered":"2024-09-20T18:49:47.950909+0000","last_unstale":"2024-09-20T18:50:24.630180+0000","last_undegraded":"2024-09-20T18:50:24.630180+0000","last_fullsized":"2024-09-20T18:50:24.630180+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:52.254250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372880+0000","last_change":"2024-09-20T18:49:49.959365+0000","last_active":"2024-09-20T18:50:23.372880+0000","last_peered":"2024-09-20T18:50:23.372880+0000","last_clean":"2024-09-20T18:50:23.372880+0000","last_became_active":"2024-09-20T18:49:49.959102+0000","last_became_peered":"2024-09-20T18:49:49.959102+0000","last_unstale":"2024-09-20T18:50:23.372880+0000","last_undegraded":"2024-09-20T18:50:23.372880+0000","last_fullsized":"2024-09-20T18:50:23.372880+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:04.872284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629176+0000","last_change":"2024-09-20T18:49:51.970469+0000","last_active":"2024-09-20T18:50:24.629176+0000","last_peered":"2024-09-20T18:50:24.629176+0000","last_clean":"2024-09-20T18:50:24.629176+0000","last_became_active":"2024-09-20T18:49:51.970058+0000","last_became_peered":"2024-09-20T18:49:51.970058+0000","last_unstale":"2024-09-20T18:50:24.629176+0000","last_undegraded":"2024-09-20T18:50:24.629176+0000","last_fullsized":"2024-09-20T18:50:24.629176+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:59.400214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":50,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.554920+0000","last_change":"2024-09-20T18:49:45.933354+0000","last_active":"2024-09-20T18:50:25.554920+0000","last_peered":"2024-09-20T18:50:25.554920+0000","last_clean":"2024-09-20T18:50:25.554920+0000","last_became_active":"2024-09-20T18:49:45.933078+0000","last_became_peered":"2024-09-20T18:49:45.933078+0000","last_unstale":"2024-09-20T18:50:25.554920+0000","last_undegraded":"2024-09-20T18:50:25.554920+0000","last_fullsized":"2024-09-20T18:50:25.554920+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:07.425698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":58,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372450+0000","last_change":"2024-09-20T18:49:48.501151+0000","last_active":"2024-09-20T18:50:23.372450+0000","last_peered":"2024-09-20T18:50:23.372450+0000","last_clean":"2024-09-20T18:50:23.372450+0000","last_became_active":"2024-09-20T18:49:48.500809+0000","last_became_peered":"2024-09-20T18:49:48.500809+0000","last_unstale":"2024-09-20T18:50:23.372450+0000","last_undegraded":"2024-09-20T18:50:23.372450+0000","last_fullsized":"2024-09-20T18:50:23.372450+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:07.803937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":58,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.436880+0000","last_change":"2024-09-20T18:49:49.955074+0000","last_active":"2024-09-20T18:50:24.436880+0000","last_peered":"2024-09-20T18:50:24.436880+0000","last_clean":"2024-09-20T18:50:24.436880+0000","last_became_active":"2024-09-20T18:49:49.954772+0000","last_became_peered":"2024-09-20T18:49:49.954772+0000","last_unstale":"2024-09-20T18:50:24.436880+0000","last_undegraded":"2024-09-20T18:50:24.436880+0000","last_fullsized":"2024-09-20T18:50:24.436880+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:53.352545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":46,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.555765+0000","last_change":"2024-09-20T18:49:59.004173+0000","last_active":"2024-09-20T18:50:25.555765+0000","last_peered":"2024-09-20T18:50:25.555765+0000","last_clean":"2024-09-20T18:50:25.555765+0000","last_became_active":"2024-09-20T18:49:58.992302+0000","last_became_peered":"2024-09-20T18:49:58.992302+0000","last_unstale":"2024-09-20T18:50:25.555765+0000","last_undegraded":"2024-09-20T18:50:25.555765+0000","last_fullsized":"2024-09-20T18:50:25.555765+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:14.502406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629232+0000","last_change":"2024-09-20T18:49:45.936413+0000","last_active":"2024-09-20T18:50:24.629232+0000","last_peered":"2024-09-20T18:50:24.629232+0000","last_clean":"2024-09-20T18:50:24.629232+0000","last_became_active":"2024-09-20T18:49:45.936104+0000","last_became_peered":"2024-09-20T18:49:45.936104+0000","last_unstale":"2024-09-20T18:50:24.629232+0000","last_undegraded":"2024-09-20T18:50:24.629232+0000","last_fullsized":"2024-09-20T18:50:24.629232+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:01.319743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":58,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.373444+0000","last_change":"2024-09-20T18:49:49.960978+0000","last_active":"2024-09-20T18:50:23.373444+0000","last_peered":"2024-09-20T18:50:23.373444+0000","last_clean":"2024-09-20T18:50:23.373444+0000","last_became_active":"2024-09-20T18:49:49.960697+0000","last_became_peered":"2024-09-20T18:49:49.960697+0000","last_unstale":"2024-09-20T18:50:23.373444+0000","last_undegraded":"2024-09-20T18:50:23.373444+0000","last_fullsized":"2024-09-20T18:50:23.373444+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:54.719450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.554863+0000","last_change":"2024-09-20T18:49:51.971713+0000","last_active":"2024-09-20T18:50:25.554863+0000","last_peered":"2024-09-20T18:50:25.554863+0000","last_clean":"2024-09-20T18:50:25.554863+0000","last_became_active":"2024-09-20T18:49:51.971399+0000","last_became_peered":"2024-09-20T18:49:51.971399+0000","last_unstale":"2024-09-20T18:50:25.554863+0000","last_undegraded":"2024-09-20T18:50:25.554863+0000","last_fullsized":"2024-09-20T18:50:25.554863+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:18.352126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":75,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629119+0000","last_change":"2024-09-20T18:49:48.499569+0000","last_active":"2024-09-20T18:50:24.629119+0000","last_peered":"2024-09-20T18:50:24.629119+0000","last_clean":"2024-09-20T18:50:24.629119+0000","last_became_active":"2024-09-20T18:49:48.499189+0000","last_became_peered":"2024-09-20T18:49:48.499189+0000","last_unstale":"2024-09-20T18:50:24.629119+0000","last_undegraded":"2024-09-20T18:50:24.629119+0000","last_fullsized":"2024-09-20T18:50:24.629119+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:16.224188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.630461+0000","last_change":"2024-09-20T18:49:45.940816+0000","last_active":"2024-09-20T18:50:24.630461+0000","last_peered":"2024-09-20T18:50:24.630461+0000","last_clean":"2024-09-20T18:50:24.630461+0000","last_became_active":"2024-09-20T18:49:45.940544+0000","last_became_peered":"2024-09-20T18:49:45.940544+0000","last_unstale":"2024-09-20T18:50:24.630461+0000","last_undegraded":"2024-09-20T18:50:24.630461+0000","last_fullsized":"2024-09-20T18:50:24.630461+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:46.957556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372455+0000","last_change":"2024-09-20T18:49:49.963362+0000","last_active":"2024-09-20T18:50:23.372455+0000","last_peered":"2024-09-20T18:50:23.372455+0000","last_clean":"2024-09-20T18:50:23.372455+0000","last_became_active":"2024-09-20T18:49:49.963073+0000","last_became_peered":"2024-09-20T18:49:49.963073+0000","last_unstale":"2024-09-20T18:50:23.372455+0000","last_undegraded":"2024-09-20T18:50:23.372455+0000","last_fullsized":"2024-09-20T18:50:23.372455+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:01.468417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372690+0000","last_change":"2024-09-20T18:49:51.969120+0000","last_active":"2024-09-20T18:50:23.372690+0000","last_peered":"2024-09-20T18:50:23.372690+0000","last_clean":"2024-09-20T18:50:23.372690+0000","last_became_active":"2024-09-20T18:49:51.968793+0000","last_became_peered":"2024-09-20T18:49:51.968793+0000","last_unstale":"2024-09-20T18:50:23.372690+0000","last_undegraded":"2024-09-20T18:50:23.372690+0000","last_fullsized":"2024-09-20T18:50:23.372690+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:57.615454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":64,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372452+0000","last_change":"2024-09-20T18:49:47.942407+0000","last_active":"2024-09-20T18:50:23.372452+0000","last_peered":"2024-09-20T18:50:23.372452+0000","last_clean":"2024-09-20T18:50:23.372452+0000","last_became_active":"2024-09-20T18:49:47.941895+0000","last_became_peered":"2024-09-20T18:49:47.941895+0000","last_unstale":"2024-09-20T18:50:23.372452+0000","last_undegraded":"2024-09-20T18:50:23.372452+0000","last_fullsized":"2024-09-20T18:50:23.372452+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:08.413407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372573+0000","last_change":"2024-09-20T18:49:45.940728+0000","last_active":"2024-09-20T18:50:23.372573+0000","last_peered":"2024-09-20T18:50:23.372573+0000","last_clean":"2024-09-20T18:50:23.372573+0000","last_became_active":"2024-09-20T18:49:45.940469+0000","last_became_peered":"2024-09-20T18:49:45.940469+0000","last_unstale":"2024-09-20T18:50:23.372573+0000","last_undegraded":"2024-09-20T18:50:23.372573+0000","last_fullsized":"2024-09-20T18:50:23.372573+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:05.840900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"66'8","reported_seq":55,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.628974+0000","last_change":"2024-09-20T18:49:49.956436+0000","last_active":"2024-09-20T18:50:24.628974+0000","last_peered":"2024-09-20T18:50:24.628974+0000","last_clean":"2024-09-20T18:50:24.628974+0000","last_became_active":"2024-09-20T18:49:49.956133+0000","last_became_peered":"2024-09-20T18:49:49.956133+0000","last_unstale":"2024-09-20T18:50:24.628974+0000","last_undegraded":"2024-09-20T18:50:24.628974+0000","last_fullsized":"2024-09-20T18:50:24.628974+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:03.649831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629069+0000","last_change":"2024-09-20T18:49:52.142481+0000","last_active":"2024-09-20T18:50:24.629069+0000","last_peered":"2024-09-20T18:50:24.629069+0000","last_clean":"2024-09-20T18:50:24.629069+0000","last_became_active":"2024-09-20T18:49:52.142113+0000","last_became_peered":"2024-09-20T18:49:52.142113+0000","last_unstale":"2024-09-20T18:50:24.629069+0000","last_undegraded":"2024-09-20T18:50:24.629069+0000","last_fullsized":"2024-09-20T18:50:24.629069+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.140166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":60,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.555344+0000","last_change":"2024-09-20T18:49:47.949952+0000","last_active":"2024-09-20T18:50:25.555344+0000","last_peered":"2024-09-20T18:50:25.555344+0000","last_clean":"2024-09-20T18:50:25.555344+0000","last_became_active":"2024-09-20T18:49:47.949626+0000","last_became_peered":"2024-09-20T18:49:47.949626+0000","last_unstale":"2024-09-20T18:50:25.555344+0000","last_undegraded":"2024-09-20T18:50:25.555344+0000","last_fullsized":"2024-09-20T18:50:25.555344+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:50.929449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":49,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.555297+0000","last_change":"2024-09-20T18:49:45.942852+0000","last_active":"2024-09-20T18:50:25.555297+0000","last_peered":"2024-09-20T18:50:25.555297+0000","last_clean":"2024-09-20T18:50:25.555297+0000","last_became_active":"2024-09-20T18:49:45.942496+0000","last_became_peered":"2024-09-20T18:49:45.942496+0000","last_unstale":"2024-09-20T18:50:25.555297+0000","last_undegraded":"2024-09-20T18:50:25.555297+0000","last_fullsized":"2024-09-20T18:50:25.555297+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:55.917563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":55,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372071+0000","last_change":"2024-09-20T18:49:49.959699+0000","last_active":"2024-09-20T18:50:23.372071+0000","last_peered":"2024-09-20T18:50:23.372071+0000","last_clean":"2024-09-20T18:50:23.372071+0000","last_became_active":"2024-09-20T18:49:49.959365+0000","last_became_peered":"2024-09-20T18:49:49.959365+0000","last_unstale":"2024-09-20T18:50:23.372071+0000","last_undegraded":"2024-09-20T18:50:23.372071+0000","last_fullsized":"2024-09-20T18:50:23.372071+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:01.818798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.371823+0000","last_change":"2024-09-20T18:49:51.965733+0000","last_active":"2024-09-20T18:50:23.371823+0000","last_peered":"2024-09-20T18:50:23.371823+0000","last_clean":"2024-09-20T18:50:23.371823+0000","last_became_active":"2024-09-20T18:49:51.964780+0000","last_became_peered":"2024-09-20T18:49:51.964780+0000","last_unstale":"2024-09-20T18:50:23.371823+0000","last_undegraded":"2024-09-20T18:50:23.371823+0000","last_fullsized":"2024-09-20T18:50:23.371823+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:11.275213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":79,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372181+0000","last_change":"2024-09-20T18:49:48.498141+0000","last_active":"2024-09-20T18:50:23.372181+0000","last_peered":"2024-09-20T18:50:23.372181+0000","last_clean":"2024-09-20T18:50:23.372181+0000","last_became_active":"2024-09-20T18:49:48.497736+0000","last_became_peered":"2024-09-20T18:49:48.497736+0000","last_unstale":"2024-09-20T18:50:23.372181+0000","last_undegraded":"2024-09-20T18:50:23.372181+0000","last_fullsized":"2024-09-20T18:50:23.372181+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.986657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":49,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.555602+0000","last_change":"2024-09-20T18:49:45.942589+0000","last_active":"2024-09-20T18:50:25.555602+0000","last_peered":"2024-09-20T18:50:25.555602+0000","last_clean":"2024-09-20T18:50:25.555602+0000","last_became_active":"2024-09-20T18:49:45.942253+0000","last_became_peered":"2024-09-20T18:49:45.942253+0000","last_unstale":"2024-09-20T18:50:25.555602+0000","last_undegraded":"2024-09-20T18:50:25.555602+0000","last_fullsized":"2024-09-20T18:50:25.555602+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.673189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.436485+0000","last_change":"2024-09-20T18:49:49.954594+0000","last_active":"2024-09-20T18:50:24.436485+0000","last_peered":"2024-09-20T18:50:24.436485+0000","last_clean":"2024-09-20T18:50:24.436485+0000","last_became_active":"2024-09-20T18:49:49.954299+0000","last_became_peered":"2024-09-20T18:49:49.954299+0000","last_unstale":"2024-09-20T18:50:24.436485+0000","last_undegraded":"2024-09-20T18:50:24.436485+0000","last_fullsized":"2024-09-20T18:50:24.436485+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:52.123030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.554684+0000","last_change":"2024-09-20T18:49:52.140276+0000","last_active":"2024-09-20T18:50:25.554684+0000","last_peered":"2024-09-20T18:50:25.554684+0000","last_clean":"2024-09-20T18:50:25.554684+0000","last_became_active":"2024-09-20T18:49:52.139981+0000","last_became_peered":"2024-09-20T18:49:52.139981+0000","last_unstale":"2024-09-20T18:50:25.554684+0000","last_undegraded":"2024-09-20T18:50:25.554684+0000","last_fullsized":"2024-09-20T18:50:25.554684+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:52.117965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"66'28","reported_seq":101,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.555578+0000","last_change":"2024-09-20T18:49:47.955615+0000","last_active":"2024-09-20T18:50:25.555578+0000","last_peered":"2024-09-20T18:50:25.555578+0000","last_clean":"2024-09-20T18:50:25.555578+0000","last_became_active":"2024-09-20T18:49:47.955335+0000","last_became_peered":"2024-09-20T18:49:47.955335+0000","last_unstale":"2024-09-20T18:50:25.555578+0000","last_undegraded":"2024-09-20T18:50:25.555578+0000","last_fullsized":"2024-09-20T18:50:25.555578+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:47.506576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.373045+0000","last_change":"2024-09-20T18:49:45.945275+0000","last_active":"2024-09-20T18:50:23.373045+0000","last_peered":"2024-09-20T18:50:23.373045+0000","last_clean":"2024-09-20T18:50:23.373045+0000","last_became_active":"2024-09-20T18:49:45.944992+0000","last_became_peered":"2024-09-20T18:49:45.944992+0000","last_unstale":"2024-09-20T18:50:23.373045+0000","last_undegraded":"2024-09-20T18:50:23.373045+0000","last_fullsized":"2024-09-20T18:50:23.373045+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:47.768689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":59,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.554638+0000","last_change":"2024-09-20T18:49:47.940836+0000","last_active":"2024-09-20T18:50:25.554638+0000","last_peered":"2024-09-20T18:50:25.554638+0000","last_clean":"2024-09-20T18:50:25.554638+0000","last_became_active":"2024-09-20T18:49:44.936027+0000","last_became_peered":"2024-09-20T18:49:44.936027+0000","last_unstale":"2024-09-20T18:50:25.554638+0000","last_undegraded":"2024-09-20T18:50:25.554638+0000","last_fullsized":"2024-09-20T18:50:25.554638+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:49:43.921002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:47.157665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012781159999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372293+0000","last_change":"2024-09-20T18:49:49.954281+0000","last_active":"2024-09-20T18:50:23.372293+0000","last_peered":"2024-09-20T18:50:23.372293+0000","last_clean":"2024-09-20T18:50:23.372293+0000","last_became_active":"2024-09-20T18:49:49.953971+0000","last_became_peered":"2024-09-20T18:49:49.953971+0000","last_unstale":"2024-09-20T18:50:23.372293+0000","last_undegraded":"2024-09-20T18:50:23.372293+0000","last_fullsized":"2024-09-20T18:50:23.372293+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:12.341509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.628803+0000","last_change":"2024-09-20T18:49:51.969702+0000","last_active":"2024-09-20T18:50:24.628803+0000","last_peered":"2024-09-20T18:50:24.628803+0000","last_clean":"2024-09-20T18:50:24.628803+0000","last_became_active":"2024-09-20T18:49:51.969400+0000","last_became_peered":"2024-09-20T18:49:51.969400+0000","last_unstale":"2024-09-20T18:50:24.628803+0000","last_undegraded":"2024-09-20T18:50:24.628803+0000","last_fullsized":"2024-09-20T18:50:24.628803+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.486248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":72,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.555137+0000","last_change":"2024-09-20T18:49:47.951311+0000","last_active":"2024-09-20T18:50:25.555137+0000","last_peered":"2024-09-20T18:50:25.555137+0000","last_clean":"2024-09-20T18:50:25.555137+0000","last_became_active":"2024-09-20T18:49:47.950904+0000","last_became_peered":"2024-09-20T18:49:47.950904+0000","last_unstale":"2024-09-20T18:50:25.555137+0000","last_undegraded":"2024-09-20T18:50:25.555137+0000","last_fullsized":"2024-09-20T18:50:25.555137+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:06.445493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":49,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.554810+0000","last_change":"2024-09-20T18:49:45.940667+0000","last_active":"2024-09-20T18:50:25.554810+0000","last_peered":"2024-09-20T18:50:25.554810+0000","last_clean":"2024-09-20T18:50:25.554810+0000","last_became_active":"2024-09-20T18:49:45.939884+0000","last_became_peered":"2024-09-20T18:49:45.939884+0000","last_unstale":"2024-09-20T18:50:25.554810+0000","last_undegraded":"2024-09-20T18:50:25.554810+0000","last_fullsized":"2024-09-20T18:50:25.554810+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:49.675839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":59,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.554624+0000","last_change":"2024-09-20T18:49:47.939858+0000","last_active":"2024-09-20T18:50:25.554624+0000","last_peered":"2024-09-20T18:50:25.554624+0000","last_clean":"2024-09-20T18:50:25.554624+0000","last_became_active":"2024-09-20T18:49:44.937135+0000","last_became_peered":"2024-09-20T18:49:44.937135+0000","last_unstale":"2024-09-20T18:50:25.554624+0000","last_undegraded":"2024-09-20T18:50:25.554624+0000","last_fullsized":"2024-09-20T18:50:25.554624+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:49:43.921002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:56.442658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095558800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.555620+0000","last_change":"2024-09-20T18:49:49.954880+0000","last_active":"2024-09-20T18:50:25.555620+0000","last_peered":"2024-09-20T18:50:25.555620+0000","last_clean":"2024-09-20T18:50:25.555620+0000","last_became_active":"2024-09-20T18:49:49.954424+0000","last_became_peered":"2024-09-20T18:49:49.954424+0000","last_unstale":"2024-09-20T18:50:25.555620+0000","last_undegraded":"2024-09-20T18:50:25.555620+0000","last_fullsized":"2024-09-20T18:50:25.555620+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:54.964168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629997+0000","last_change":"2024-09-20T18:49:52.140366+0000","last_active":"2024-09-20T18:50:24.629997+0000","last_peered":"2024-09-20T18:50:24.629997+0000","last_clean":"2024-09-20T18:50:24.629997+0000","last_became_active":"2024-09-20T18:49:52.139789+0000","last_became_peered":"2024-09-20T18:49:52.139789+0000","last_unstale":"2024-09-20T18:50:24.629997+0000","last_undegraded":"2024-09-20T18:50:24.629997+0000","last_fullsized":"2024-09-20T18:50:24.629997+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:16.224499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":61,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372491+0000","last_change":"2024-09-20T18:49:47.949547+0000","last_active":"2024-09-20T18:50:23.372491+0000","last_peered":"2024-09-20T18:50:23.372491+0000","last_clean":"2024-09-20T18:50:23.372491+0000","last_became_active":"2024-09-20T18:49:47.949230+0000","last_became_peered":"2024-09-20T18:49:47.949230+0000","last_unstale":"2024-09-20T18:50:23.372491+0000","last_undegraded":"2024-09-20T18:50:23.372491+0000","last_fullsized":"2024-09-20T18:50:23.372491+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:07.802093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.371747+0000","last_change":"2024-09-20T18:49:45.939454+0000","last_active":"2024-09-20T18:50:23.371747+0000","last_peered":"2024-09-20T18:50:23.371747+0000","last_clean":"2024-09-20T18:50:23.371747+0000","last_became_active":"2024-09-20T18:49:45.939197+0000","last_became_peered":"2024-09-20T18:49:45.939197+0000","last_unstale":"2024-09-20T18:50:23.371747+0000","last_undegraded":"2024-09-20T18:50:23.371747+0000","last_fullsized":"2024-09-20T18:50:23.371747+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:12.763563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":93,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629751+0000","last_change":"2024-09-20T18:49:48.498983+0000","last_active":"2024-09-20T18:50:24.629751+0000","last_peered":"2024-09-20T18:50:24.629751+0000","last_clean":"2024-09-20T18:50:24.629751+0000","last_became_active":"2024-09-20T18:49:44.936819+0000","last_became_peered":"2024-09-20T18:49:44.936819+0000","last_unstale":"2024-09-20T18:50:24.629751+0000","last_undegraded":"2024-09-20T18:50:24.629751+0000","last_fullsized":"2024-09-20T18:50:24.629751+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:49:43.921002+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:58.186148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010996199999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":35,"num_read_kb":30,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.555463+0000","last_change":"2024-09-20T18:49:49.961346+0000","last_active":"2024-09-20T18:50:25.555463+0000","last_peered":"2024-09-20T18:50:25.555463+0000","last_clean":"2024-09-20T18:50:25.555463+0000","last_became_active":"2024-09-20T18:49:49.961066+0000","last_became_peered":"2024-09-20T18:49:49.961066+0000","last_unstale":"2024-09-20T18:50:25.555463+0000","last_undegraded":"2024-09-20T18:50:25.555463+0000","last_fullsized":"2024-09-20T18:50:25.555463+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:53.706599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.555486+0000","last_change":"2024-09-20T18:49:51.970609+0000","last_active":"2024-09-20T18:50:25.555486+0000","last_peered":"2024-09-20T18:50:25.555486+0000","last_clean":"2024-09-20T18:50:25.555486+0000","last_became_active":"2024-09-20T18:49:51.970343+0000","last_became_peered":"2024-09-20T18:49:51.970343+0000","last_unstale":"2024-09-20T18:50:25.555486+0000","last_undegraded":"2024-09-20T18:50:25.555486+0000","last_fullsized":"2024-09-20T18:50:25.555486+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:06.660017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":73,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.436831+0000","last_change":"2024-09-20T18:49:48.498345+0000","last_active":"2024-09-20T18:50:24.436831+0000","last_peered":"2024-09-20T18:50:24.436831+0000","last_clean":"2024-09-20T18:50:24.436831+0000","last_became_active":"2024-09-20T18:49:48.497936+0000","last_became_peered":"2024-09-20T18:49:48.497936+0000","last_unstale":"2024-09-20T18:50:24.436831+0000","last_undegraded":"2024-09-20T18:50:24.436831+0000","last_fullsized":"2024-09-20T18:50:24.436831+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:51.535171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629427+0000","last_change":"2024-09-20T18:49:45.941387+0000","last_active":"2024-09-20T18:50:24.629427+0000","last_peered":"2024-09-20T18:50:24.629427+0000","last_clean":"2024-09-20T18:50:24.629427+0000","last_became_active":"2024-09-20T18:49:45.941136+0000","last_became_peered":"2024-09-20T18:49:45.941136+0000","last_unstale":"2024-09-20T18:50:24.629427+0000","last_undegraded":"2024-09-20T18:50:24.629427+0000","last_fullsized":"2024-09-20T18:50:24.629427+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:09.074025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":86,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629043+0000","last_change":"2024-09-20T18:49:36.549748+0000","last_active":"2024-09-20T18:50:24.629043+0000","last_peered":"2024-09-20T18:50:24.629043+0000","last_clean":"2024-09-20T18:50:24.629043+0000","last_became_active":"2024-09-20T18:49:36.545306+0000","last_became_peered":"2024-09-20T18:49:36.545306+0000","last_unstale":"2024-09-20T18:50:24.629043+0000","last_undegraded":"2024-09-20T18:50:24.629043+0000","last_fullsized":"2024-09-20T18:50:24.629043+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"20'72","last_scrub_stamp":"2024-09-20T18:49:27.311157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T18:49:27.311157+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:30.469238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629912+0000","last_change":"2024-09-20T18:49:49.956441+0000","last_active":"2024-09-20T18:50:24.629912+0000","last_peered":"2024-09-20T18:50:24.629912+0000","last_clean":"2024-09-20T18:50:24.629912+0000","last_became_active":"2024-09-20T18:49:49.956128+0000","last_became_peered":"2024-09-20T18:49:49.956128+0000","last_unstale":"2024-09-20T18:50:24.629912+0000","last_undegraded":"2024-09-20T18:50:24.629912+0000","last_fullsized":"2024-09-20T18:50:24.629912+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:04.622346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.555344+0000","last_change":"2024-09-20T18:49:51.971980+0000","last_active":"2024-09-20T18:50:25.555344+0000","last_peered":"2024-09-20T18:50:25.555344+0000","last_clean":"2024-09-20T18:50:25.555344+0000","last_became_active":"2024-09-20T18:49:51.971332+0000","last_became_peered":"2024-09-20T18:49:51.971332+0000","last_unstale":"2024-09-20T18:50:25.555344+0000","last_undegraded":"2024-09-20T18:50:25.555344+0000","last_fullsized":"2024-09-20T18:50:25.555344+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:13.174503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":62,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.373437+0000","last_change":"2024-09-20T18:49:47.950762+0000","last_active":"2024-09-20T18:50:23.373437+0000","last_peered":"2024-09-20T18:50:23.373437+0000","last_clean":"2024-09-20T18:50:23.373437+0000","last_became_active":"2024-09-20T18:49:47.950073+0000","last_became_peered":"2024-09-20T18:49:47.950073+0000","last_unstale":"2024-09-20T18:50:23.373437+0000","last_undegraded":"2024-09-20T18:50:23.373437+0000","last_fullsized":"2024-09-20T18:50:23.373437+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:57.527730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372852+0000","last_change":"2024-09-20T18:49:45.944925+0000","last_active":"2024-09-20T18:50:23.372852+0000","last_peered":"2024-09-20T18:50:23.372852+0000","last_clean":"2024-09-20T18:50:23.372852+0000","last_became_active":"2024-09-20T18:49:45.944579+0000","last_became_peered":"2024-09-20T18:49:45.944579+0000","last_unstale":"2024-09-20T18:50:23.372852+0000","last_undegraded":"2024-09-20T18:50:23.372852+0000","last_fullsized":"2024-09-20T18:50:23.372852+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:03.333030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.555244+0000","last_change":"2024-09-20T18:49:49.954673+0000","last_active":"2024-09-20T18:50:25.555244+0000","last_peered":"2024-09-20T18:50:25.555244+0000","last_clean":"2024-09-20T18:50:25.555244+0000","last_became_active":"2024-09-20T18:49:49.954391+0000","last_became_peered":"2024-09-20T18:49:49.954391+0000","last_unstale":"2024-09-20T18:50:25.555244+0000","last_undegraded":"2024-09-20T18:50:25.555244+0000","last_fullsized":"2024-09-20T18:50:25.555244+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:12.999716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629582+0000","last_change":"2024-09-20T18:49:52.140252+0000","last_active":"2024-09-20T18:50:24.629582+0000","last_peered":"2024-09-20T18:50:24.629582+0000","last_clean":"2024-09-20T18:50:24.629582+0000","last_became_active":"2024-09-20T18:49:52.139959+0000","last_became_peered":"2024-09-20T18:49:52.139959+0000","last_unstale":"2024-09-20T18:50:24.629582+0000","last_undegraded":"2024-09-20T18:50:24.629582+0000","last_fullsized":"2024-09-20T18:50:24.629582+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:54.070093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":71,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.373360+0000","last_change":"2024-09-20T18:49:47.947920+0000","last_active":"2024-09-20T18:50:23.373360+0000","last_peered":"2024-09-20T18:50:23.373360+0000","last_clean":"2024-09-20T18:50:23.373360+0000","last_became_active":"2024-09-20T18:49:47.947636+0000","last_became_peered":"2024-09-20T18:49:47.947636+0000","last_unstale":"2024-09-20T18:50:23.373360+0000","last_undegraded":"2024-09-20T18:50:23.373360+0000","last_fullsized":"2024-09-20T18:50:23.373360+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:47.335335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.436881+0000","last_change":"2024-09-20T18:49:45.946517+0000","last_active":"2024-09-20T18:50:24.436881+0000","last_peered":"2024-09-20T18:50:24.436881+0000","last_clean":"2024-09-20T18:50:24.436881+0000","last_became_active":"2024-09-20T18:49:45.946153+0000","last_became_peered":"2024-09-20T18:49:45.946153+0000","last_unstale":"2024-09-20T18:50:24.436881+0000","last_undegraded":"2024-09-20T18:50:24.436881+0000","last_fullsized":"2024-09-20T18:50:24.436881+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:05.889152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.555493+0000","last_change":"2024-09-20T18:49:49.959440+0000","last_active":"2024-09-20T18:50:25.555493+0000","last_peered":"2024-09-20T18:50:25.555493+0000","last_clean":"2024-09-20T18:50:25.555493+0000","last_became_active":"2024-09-20T18:49:49.959160+0000","last_became_peered":"2024-09-20T18:49:49.959160+0000","last_unstale":"2024-09-20T18:50:25.555493+0000","last_undegraded":"2024-09-20T18:50:25.555493+0000","last_fullsized":"2024-09-20T18:50:25.555493+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:08.849486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.555908+0000","last_change":"2024-09-20T18:49:51.970386+0000","last_active":"2024-09-20T18:50:25.555908+0000","last_peered":"2024-09-20T18:50:25.555908+0000","last_clean":"2024-09-20T18:50:25.555908+0000","last_became_active":"2024-09-20T18:49:51.970108+0000","last_became_peered":"2024-09-20T18:49:51.970108+0000","last_unstale":"2024-09-20T18:50:25.555908+0000","last_undegraded":"2024-09-20T18:50:25.555908+0000","last_fullsized":"2024-09-20T18:50:25.555908+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:04.919785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":58,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.373003+0000","last_change":"2024-09-20T18:49:47.948046+0000","last_active":"2024-09-20T18:50:23.373003+0000","last_peered":"2024-09-20T18:50:23.373003+0000","last_clean":"2024-09-20T18:50:23.373003+0000","last_became_active":"2024-09-20T18:49:47.947126+0000","last_became_peered":"2024-09-20T18:49:47.947126+0000","last_unstale":"2024-09-20T18:50:23.373003+0000","last_undegraded":"2024-09-20T18:50:23.373003+0000","last_fullsized":"2024-09-20T18:50:23.373003+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:16.466592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629607+0000","last_change":"2024-09-20T18:49:45.937511+0000","last_active":"2024-09-20T18:50:24.629607+0000","last_peered":"2024-09-20T18:50:24.629607+0000","last_clean":"2024-09-20T18:50:24.629607+0000","last_became_active":"2024-09-20T18:49:45.937264+0000","last_became_peered":"2024-09-20T18:49:45.937264+0000","last_unstale":"2024-09-20T18:50:24.629607+0000","last_undegraded":"2024-09-20T18:50:24.629607+0000","last_fullsized":"2024-09-20T18:50:24.629607+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:06.234318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":58,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.555249+0000","last_change":"2024-09-20T18:49:49.957286+0000","last_active":"2024-09-20T18:50:25.555249+0000","last_peered":"2024-09-20T18:50:25.555249+0000","last_clean":"2024-09-20T18:50:25.555249+0000","last_became_active":"2024-09-20T18:49:49.957005+0000","last_became_peered":"2024-09-20T18:49:49.957005+0000","last_unstale":"2024-09-20T18:50:25.555249+0000","last_undegraded":"2024-09-20T18:50:25.555249+0000","last_fullsized":"2024-09-20T18:50:25.555249+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:11.425942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.373273+0000","last_change":"2024-09-20T18:49:51.969633+0000","last_active":"2024-09-20T18:50:23.373273+0000","last_peered":"2024-09-20T18:50:23.373273+0000","last_clean":"2024-09-20T18:50:23.373273+0000","last_became_active":"2024-09-20T18:49:51.969355+0000","last_became_peered":"2024-09-20T18:49:51.969355+0000","last_unstale":"2024-09-20T18:50:23.373273+0000","last_undegraded":"2024-09-20T18:50:23.373273+0000","last_fullsized":"2024-09-20T18:50:23.373273+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:53.655593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":59,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372126+0000","last_change":"2024-09-20T18:49:47.946309+0000","last_active":"2024-09-20T18:50:23.372126+0000","last_peered":"2024-09-20T18:50:23.372126+0000","last_clean":"2024-09-20T18:50:23.372126+0000","last_became_active":"2024-09-20T18:49:47.946008+0000","last_became_peered":"2024-09-20T18:49:47.946008+0000","last_unstale":"2024-09-20T18:50:23.372126+0000","last_undegraded":"2024-09-20T18:50:23.372126+0000","last_fullsized":"2024-09-20T18:50:23.372126+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:55.957874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":49,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.555134+0000","last_change":"2024-09-20T18:49:45.941766+0000","last_active":"2024-09-20T18:50:25.555134+0000","last_peered":"2024-09-20T18:50:25.555134+0000","last_clean":"2024-09-20T18:50:25.555134+0000","last_became_active":"2024-09-20T18:49:45.941406+0000","last_became_peered":"2024-09-20T18:49:45.941406+0000","last_unstale":"2024-09-20T18:50:25.555134+0000","last_undegraded":"2024-09-20T18:50:25.555134+0000","last_fullsized":"2024-09-20T18:50:25.555134+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:46.650041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.554800+0000","last_change":"2024-09-20T18:49:49.957141+0000","last_active":"2024-09-20T18:50:25.554800+0000","last_peered":"2024-09-20T18:50:25.554800+0000","last_clean":"2024-09-20T18:50:25.554800+0000","last_became_active":"2024-09-20T18:49:49.956767+0000","last_became_peered":"2024-09-20T18:49:49.956767+0000","last_unstale":"2024-09-20T18:50:25.554800+0000","last_undegraded":"2024-09-20T18:50:25.554800+0000","last_fullsized":"2024-09-20T18:50:25.554800+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:55.731396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.371640+0000","last_change":"2024-09-20T18:49:51.964375+0000","last_active":"2024-09-20T18:50:23.371640+0000","last_peered":"2024-09-20T18:50:23.371640+0000","last_clean":"2024-09-20T18:50:23.371640+0000","last_became_active":"2024-09-20T18:49:51.964098+0000","last_became_peered":"2024-09-20T18:49:51.964098+0000","last_unstale":"2024-09-20T18:50:23.371640+0000","last_undegraded":"2024-09-20T18:50:23.371640+0000","last_fullsized":"2024-09-20T18:50:23.371640+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.566377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":74,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.436613+0000","last_change":"2024-09-20T18:49:48.498846+0000","last_active":"2024-09-20T18:50:24.436613+0000","last_peered":"2024-09-20T18:50:24.436613+0000","last_clean":"2024-09-20T18:50:24.436613+0000","last_became_active":"2024-09-20T18:49:48.498625+0000","last_became_peered":"2024-09-20T18:49:48.498625+0000","last_unstale":"2024-09-20T18:50:24.436613+0000","last_undegraded":"2024-09-20T18:50:24.436613+0000","last_fullsized":"2024-09-20T18:50:24.436613+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:03.186719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.630180+0000","last_change":"2024-09-20T18:49:45.941764+0000","last_active":"2024-09-20T18:50:24.630180+0000","last_peered":"2024-09-20T18:50:24.630180+0000","last_clean":"2024-09-20T18:50:24.630180+0000","last_became_active":"2024-09-20T18:49:45.941448+0000","last_became_peered":"2024-09-20T18:49:45.941448+0000","last_unstale":"2024-09-20T18:50:24.630180+0000","last_undegraded":"2024-09-20T18:50:24.630180+0000","last_fullsized":"2024-09-20T18:50:24.630180+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:45.120244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.555111+0000","last_change":"2024-09-20T18:49:49.957845+0000","last_active":"2024-09-20T18:50:25.555111+0000","last_peered":"2024-09-20T18:50:25.555111+0000","last_clean":"2024-09-20T18:50:25.555111+0000","last_became_active":"2024-09-20T18:49:49.957558+0000","last_became_peered":"2024-09-20T18:49:49.957558+0000","last_unstale":"2024-09-20T18:50:25.555111+0000","last_undegraded":"2024-09-20T18:50:25.555111+0000","last_fullsized":"2024-09-20T18:50:25.555111+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:14.726319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629280+0000","last_change":"2024-09-20T18:49:51.971132+0000","last_active":"2024-09-20T18:50:24.629280+0000","last_peered":"2024-09-20T18:50:24.629280+0000","last_clean":"2024-09-20T18:50:24.629280+0000","last_became_active":"2024-09-20T18:49:51.970798+0000","last_became_peered":"2024-09-20T18:49:51.970798+0000","last_unstale":"2024-09-20T18:50:24.629280+0000","last_undegraded":"2024-09-20T18:50:24.629280+0000","last_fullsized":"2024-09-20T18:50:24.629280+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:09.540547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":66,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372699+0000","last_change":"2024-09-20T18:49:47.949428+0000","last_active":"2024-09-20T18:50:23.372699+0000","last_peered":"2024-09-20T18:50:23.372699+0000","last_clean":"2024-09-20T18:50:23.372699+0000","last_became_active":"2024-09-20T18:49:47.949116+0000","last_became_peered":"2024-09-20T18:49:47.949116+0000","last_unstale":"2024-09-20T18:50:23.372699+0000","last_undegraded":"2024-09-20T18:50:23.372699+0000","last_fullsized":"2024-09-20T18:50:23.372699+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:11.352253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.630326+0000","last_change":"2024-09-20T18:49:45.945147+0000","last_active":"2024-09-20T18:50:24.630326+0000","last_peered":"2024-09-20T18:50:24.630326+0000","last_clean":"2024-09-20T18:50:24.630326+0000","last_became_active":"2024-09-20T18:49:45.944858+0000","last_became_peered":"2024-09-20T18:49:45.944858+0000","last_unstale":"2024-09-20T18:50:24.630326+0000","last_undegraded":"2024-09-20T18:50:24.630326+0000","last_fullsized":"2024-09-20T18:50:24.630326+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:53.147843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.554463+0000","last_change":"2024-09-20T18:49:49.955243+0000","last_active":"2024-09-20T18:50:25.554463+0000","last_peered":"2024-09-20T18:50:25.554463+0000","last_clean":"2024-09-20T18:50:25.554463+0000","last_became_active":"2024-09-20T18:49:49.954944+0000","last_became_peered":"2024-09-20T18:49:49.954944+0000","last_unstale":"2024-09-20T18:50:25.554463+0000","last_undegraded":"2024-09-20T18:50:25.554463+0000","last_fullsized":"2024-09-20T18:50:25.554463+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:00.542377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.628764+0000","last_change":"2024-09-20T18:49:51.973284+0000","last_active":"2024-09-20T18:50:24.628764+0000","last_peered":"2024-09-20T18:50:24.628764+0000","last_clean":"2024-09-20T18:50:24.628764+0000","last_became_active":"2024-09-20T18:49:51.972970+0000","last_became_peered":"2024-09-20T18:49:51.972970+0000","last_unstale":"2024-09-20T18:50:24.628764+0000","last_undegraded":"2024-09-20T18:50:24.628764+0000","last_fullsized":"2024-09-20T18:50:24.628764+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:55.977221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":70,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.554310+0000","last_change":"2024-09-20T18:49:48.500174+0000","last_active":"2024-09-20T18:50:25.554310+0000","last_peered":"2024-09-20T18:50:25.554310+0000","last_clean":"2024-09-20T18:50:25.554310+0000","last_became_active":"2024-09-20T18:49:48.499847+0000","last_became_peered":"2024-09-20T18:49:48.499847+0000","last_unstale":"2024-09-20T18:50:25.554310+0000","last_undegraded":"2024-09-20T18:50:25.554310+0000","last_fullsized":"2024-09-20T18:50:25.554310+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:15.572682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":59,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.630364+0000","last_change":"2024-09-20T18:49:45.945391+0000","last_active":"2024-09-20T18:50:24.630364+0000","last_peered":"2024-09-20T18:50:24.630364+0000","last_clean":"2024-09-20T18:50:24.630364+0000","last_became_active":"2024-09-20T18:49:45.944256+0000","last_became_peered":"2024-09-20T18:49:45.944256+0000","last_unstale":"2024-09-20T18:50:24.630364+0000","last_undegraded":"2024-09-20T18:50:24.630364+0000","last_fullsized":"2024-09-20T18:50:24.630364+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:47.823570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":55,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629594+0000","last_change":"2024-09-20T18:49:49.958686+0000","last_active":"2024-09-20T18:50:24.629594+0000","last_peered":"2024-09-20T18:50:24.629594+0000","last_clean":"2024-09-20T18:50:24.629594+0000","last_became_active":"2024-09-20T18:49:49.958411+0000","last_became_peered":"2024-09-20T18:49:49.958411+0000","last_unstale":"2024-09-20T18:50:24.629594+0000","last_undegraded":"2024-09-20T18:50:24.629594+0000","last_fullsized":"2024-09-20T18:50:24.629594+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:57.283019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.554442+0000","last_change":"2024-09-20T18:49:52.139994+0000","last_active":"2024-09-20T18:50:25.554442+0000","last_peered":"2024-09-20T18:50:25.554442+0000","last_clean":"2024-09-20T18:50:25.554442+0000","last_became_active":"2024-09-20T18:49:52.139685+0000","last_became_peered":"2024-09-20T18:49:52.139685+0000","last_unstale":"2024-09-20T18:50:25.554442+0000","last_undegraded":"2024-09-20T18:50:25.554442+0000","last_fullsized":"2024-09-20T18:50:25.554442+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:53.034093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.436316+0000","last_change":"2024-09-20T18:49:45.940360+0000","last_active":"2024-09-20T18:50:24.436316+0000","last_peered":"2024-09-20T18:50:24.436316+0000","last_clean":"2024-09-20T18:50:24.436316+0000","last_became_active":"2024-09-20T18:49:45.940060+0000","last_became_peered":"2024-09-20T18:49:45.940060+0000","last_unstale":"2024-09-20T18:50:24.436316+0000","last_undegraded":"2024-09-20T18:50:24.436316+0000","last_fullsized":"2024-09-20T18:50:24.436316+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:59.071733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":52,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.630441+0000","last_change":"2024-09-20T18:49:48.501010+0000","last_active":"2024-09-20T18:50:24.630441+0000","last_peered":"2024-09-20T18:50:24.630441+0000","last_clean":"2024-09-20T18:50:24.630441+0000","last_became_active":"2024-09-20T18:49:48.500662+0000","last_became_peered":"2024-09-20T18:49:48.500662+0000","last_unstale":"2024-09-20T18:50:24.630441+0000","last_undegraded":"2024-09-20T18:50:24.630441+0000","last_fullsized":"2024-09-20T18:50:24.630441+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:01.787506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.555213+0000","last_change":"2024-09-20T18:49:49.957092+0000","last_active":"2024-09-20T18:50:25.555213+0000","last_peered":"2024-09-20T18:50:25.555213+0000","last_clean":"2024-09-20T18:50:25.555213+0000","last_became_active":"2024-09-20T18:49:49.956749+0000","last_became_peered":"2024-09-20T18:49:49.956749+0000","last_unstale":"2024-09-20T18:50:25.555213+0000","last_undegraded":"2024-09-20T18:50:25.555213+0000","last_fullsized":"2024-09-20T18:50:25.555213+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:53.840760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629239+0000","last_change":"2024-09-20T18:49:52.142789+0000","last_active":"2024-09-20T18:50:24.629239+0000","last_peered":"2024-09-20T18:50:24.629239+0000","last_clean":"2024-09-20T18:50:24.629239+0000","last_became_active":"2024-09-20T18:49:52.142291+0000","last_became_peered":"2024-09-20T18:49:52.142291+0000","last_unstale":"2024-09-20T18:50:24.629239+0000","last_undegraded":"2024-09-20T18:50:24.629239+0000","last_fullsized":"2024-09-20T18:50:24.629239+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:58.299262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":59,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372537+0000","last_change":"2024-09-20T18:49:48.501376+0000","last_active":"2024-09-20T18:50:23.372537+0000","last_peered":"2024-09-20T18:50:23.372537+0000","last_clean":"2024-09-20T18:50:23.372537+0000","last_became_active":"2024-09-20T18:49:48.500803+0000","last_became_peered":"2024-09-20T18:49:48.500803+0000","last_unstale":"2024-09-20T18:50:23.372537+0000","last_undegraded":"2024-09-20T18:50:23.372537+0000","last_fullsized":"2024-09-20T18:50:23.372537+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:02.881066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629764+0000","last_change":"2024-09-20T18:49:45.947791+0000","last_active":"2024-09-20T18:50:24.629764+0000","last_peered":"2024-09-20T18:50:24.629764+0000","last_clean":"2024-09-20T18:50:24.629764+0000","last_became_active":"2024-09-20T18:49:45.946600+0000","last_became_peered":"2024-09-20T18:49:45.946600+0000","last_unstale":"2024-09-20T18:50:24.629764+0000","last_undegraded":"2024-09-20T18:50:24.629764+0000","last_fullsized":"2024-09-20T18:50:24.629764+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:13.160428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.630358+0000","last_change":"2024-09-20T18:49:49.962290+0000","last_active":"2024-09-20T18:50:24.630358+0000","last_peered":"2024-09-20T18:50:24.630358+0000","last_clean":"2024-09-20T18:50:24.630358+0000","last_became_active":"2024-09-20T18:49:49.961947+0000","last_became_peered":"2024-09-20T18:49:49.961947+0000","last_unstale":"2024-09-20T18:50:24.630358+0000","last_undegraded":"2024-09-20T18:50:24.630358+0000","last_fullsized":"2024-09-20T18:50:24.630358+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:54.834258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"66'1","reported_seq":39,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.556081+0000","last_change":"2024-09-20T18:49:51.965397+0000","last_active":"2024-09-20T18:50:25.556081+0000","last_peered":"2024-09-20T18:50:25.556081+0000","last_clean":"2024-09-20T18:50:25.556081+0000","last_became_active":"2024-09-20T18:49:51.965121+0000","last_became_peered":"2024-09-20T18:49:51.965121+0000","last_unstale":"2024-09-20T18:50:25.556081+0000","last_undegraded":"2024-09-20T18:50:25.556081+0000","last_fullsized":"2024-09-20T18:50:25.556081+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:57.419566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.555112+0000","last_change":"2024-09-20T18:49:48.501359+0000","last_active":"2024-09-20T18:50:25.555112+0000","last_peered":"2024-09-20T18:50:25.555112+0000","last_clean":"2024-09-20T18:50:25.555112+0000","last_became_active":"2024-09-20T18:49:48.501011+0000","last_became_peered":"2024-09-20T18:49:48.501011+0000","last_unstale":"2024-09-20T18:50:25.555112+0000","last_undegraded":"2024-09-20T18:50:25.555112+0000","last_fullsized":"2024-09-20T18:50:25.555112+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:50.431705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.371926+0000","last_change":"2024-09-20T18:49:45.938820+0000","last_active":"2024-09-20T18:50:23.371926+0000","last_peered":"2024-09-20T18:50:23.371926+0000","last_clean":"2024-09-20T18:50:23.371926+0000","last_became_active":"2024-09-20T18:49:45.938511+0000","last_became_peered":"2024-09-20T18:49:45.938511+0000","last_unstale":"2024-09-20T18:50:23.371926+0000","last_undegraded":"2024-09-20T18:50:23.371926+0000","last_fullsized":"2024-09-20T18:50:23.371926+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:02.383015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":55,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629715+0000","last_change":"2024-09-20T18:49:49.960889+0000","last_active":"2024-09-20T18:50:24.629715+0000","last_peered":"2024-09-20T18:50:24.629715+0000","last_clean":"2024-09-20T18:50:24.629715+0000","last_became_active":"2024-09-20T18:49:49.960550+0000","last_became_peered":"2024-09-20T18:49:49.960550+0000","last_unstale":"2024-09-20T18:50:24.629715+0000","last_undegraded":"2024-09-20T18:50:24.629715+0000","last_fullsized":"2024-09-20T18:50:24.629715+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:58.739350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.373458+0000","last_change":"2024-09-20T18:49:51.966530+0000","last_active":"2024-09-20T18:50:23.373458+0000","last_peered":"2024-09-20T18:50:23.373458+0000","last_clean":"2024-09-20T18:50:23.373458+0000","last_became_active":"2024-09-20T18:49:51.966254+0000","last_became_peered":"2024-09-20T18:49:51.966254+0000","last_unstale":"2024-09-20T18:50:23.373458+0000","last_undegraded":"2024-09-20T18:50:23.373458+0000","last_fullsized":"2024-09-20T18:50:23.373458+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:14.193792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":77,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629923+0000","last_change":"2024-09-20T18:49:59.003234+0000","last_active":"2024-09-20T18:50:24.629923+0000","last_peered":"2024-09-20T18:50:24.629923+0000","last_clean":"2024-09-20T18:50:24.629923+0000","last_became_active":"2024-09-20T18:49:58.997494+0000","last_became_peered":"2024-09-20T18:49:58.997494+0000","last_unstale":"2024-09-20T18:50:24.629923+0000","last_undegraded":"2024-09-20T18:50:24.629923+0000","last_fullsized":"2024-09-20T18:50:24.629923+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:05.235529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629902+0000","last_change":"2024-09-20T18:49:45.949895+0000","last_active":"2024-09-20T18:50:24.629902+0000","last_peered":"2024-09-20T18:50:24.629902+0000","last_clean":"2024-09-20T18:50:24.629902+0000","last_became_active":"2024-09-20T18:49:45.949492+0000","last_became_peered":"2024-09-20T18:49:45.949492+0000","last_unstale":"2024-09-20T18:50:24.629902+0000","last_undegraded":"2024-09-20T18:50:24.629902+0000","last_fullsized":"2024-09-20T18:50:24.629902+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:02.958108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":57,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.554954+0000","last_change":"2024-09-20T18:49:49.962519+0000","last_active":"2024-09-20T18:50:25.554954+0000","last_peered":"2024-09-20T18:50:25.554954+0000","last_clean":"2024-09-20T18:50:25.554954+0000","last_became_active":"2024-09-20T18:49:49.962243+0000","last_became_peered":"2024-09-20T18:49:49.962243+0000","last_unstale":"2024-09-20T18:50:25.554954+0000","last_undegraded":"2024-09-20T18:50:25.554954+0000","last_fullsized":"2024-09-20T18:50:25.554954+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:16.990588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372337+0000","last_change":"2024-09-20T18:49:51.969358+0000","last_active":"2024-09-20T18:50:23.372337+0000","last_peered":"2024-09-20T18:50:23.372337+0000","last_clean":"2024-09-20T18:50:23.372337+0000","last_became_active":"2024-09-20T18:49:51.969093+0000","last_became_peered":"2024-09-20T18:49:51.969093+0000","last_unstale":"2024-09-20T18:50:23.372337+0000","last_undegraded":"2024-09-20T18:50:23.372337+0000","last_fullsized":"2024-09-20T18:50:23.372337+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:17.459087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":62,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372736+0000","last_change":"2024-09-20T18:49:47.950488+0000","last_active":"2024-09-20T18:50:23.372736+0000","last_peered":"2024-09-20T18:50:23.372736+0000","last_clean":"2024-09-20T18:50:23.372736+0000","last_became_active":"2024-09-20T18:49:47.949703+0000","last_became_peered":"2024-09-20T18:49:47.949703+0000","last_unstale":"2024-09-20T18:50:23.372736+0000","last_undegraded":"2024-09-20T18:50:23.372736+0000","last_fullsized":"2024-09-20T18:50:23.372736+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:53.129485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.373124+0000","last_change":"2024-09-20T18:49:45.941578+0000","last_active":"2024-09-20T18:50:23.373124+0000","last_peered":"2024-09-20T18:50:23.373124+0000","last_clean":"2024-09-20T18:50:23.373124+0000","last_became_active":"2024-09-20T18:49:45.939949+0000","last_became_peered":"2024-09-20T18:49:45.939949+0000","last_unstale":"2024-09-20T18:50:23.373124+0000","last_undegraded":"2024-09-20T18:50:23.373124+0000","last_fullsized":"2024-09-20T18:50:23.373124+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:53.450364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.555272+0000","last_change":"2024-09-20T18:49:49.957358+0000","last_active":"2024-09-20T18:50:25.555272+0000","last_peered":"2024-09-20T18:50:25.555272+0000","last_clean":"2024-09-20T18:50:25.555272+0000","last_became_active":"2024-09-20T18:49:49.957067+0000","last_became_peered":"2024-09-20T18:49:49.957067+0000","last_unstale":"2024-09-20T18:50:25.555272+0000","last_undegraded":"2024-09-20T18:50:25.555272+0000","last_fullsized":"2024-09-20T18:50:25.555272+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:54.100322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.628698+0000","last_change":"2024-09-20T18:49:51.967168+0000","last_active":"2024-09-20T18:50:24.628698+0000","last_peered":"2024-09-20T18:50:24.628698+0000","last_clean":"2024-09-20T18:50:24.628698+0000","last_became_active":"2024-09-20T18:49:51.966915+0000","last_became_peered":"2024-09-20T18:49:51.966915+0000","last_unstale":"2024-09-20T18:50:24.628698+0000","last_undegraded":"2024-09-20T18:50:24.628698+0000","last_fullsized":"2024-09-20T18:50:24.628698+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:04.536536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.555361+0000","last_change":"2024-09-20T18:49:47.953603+0000","last_active":"2024-09-20T18:50:25.555361+0000","last_peered":"2024-09-20T18:50:25.555361+0000","last_clean":"2024-09-20T18:50:25.555361+0000","last_became_active":"2024-09-20T18:49:47.953310+0000","last_became_peered":"2024-09-20T18:49:47.953310+0000","last_unstale":"2024-09-20T18:50:25.555361+0000","last_undegraded":"2024-09-20T18:50:25.555361+0000","last_fullsized":"2024-09-20T18:50:25.555361+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:06.158344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.630136+0000","last_change":"2024-09-20T18:49:45.944827+0000","last_active":"2024-09-20T18:50:24.630136+0000","last_peered":"2024-09-20T18:50:24.630136+0000","last_clean":"2024-09-20T18:50:24.630136+0000","last_became_active":"2024-09-20T18:49:45.944357+0000","last_became_peered":"2024-09-20T18:49:45.944357+0000","last_unstale":"2024-09-20T18:50:24.630136+0000","last_undegraded":"2024-09-20T18:50:24.630136+0000","last_fullsized":"2024-09-20T18:50:24.630136+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:14.013129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629464+0000","last_change":"2024-09-20T18:49:49.962794+0000","last_active":"2024-09-20T18:50:24.629464+0000","last_peered":"2024-09-20T18:50:24.629464+0000","last_clean":"2024-09-20T18:50:24.629464+0000","last_became_active":"2024-09-20T18:49:49.962523+0000","last_became_peered":"2024-09-20T18:49:49.962523+0000","last_unstale":"2024-09-20T18:50:24.629464+0000","last_undegraded":"2024-09-20T18:50:24.629464+0000","last_fullsized":"2024-09-20T18:50:24.629464+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:06.567112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.371568+0000","last_change":"2024-09-20T18:49:51.967581+0000","last_active":"2024-09-20T18:50:23.371568+0000","last_peered":"2024-09-20T18:50:23.371568+0000","last_clean":"2024-09-20T18:50:23.371568+0000","last_became_active":"2024-09-20T18:49:51.967313+0000","last_became_peered":"2024-09-20T18:49:51.967313+0000","last_unstale":"2024-09-20T18:50:23.371568+0000","last_undegraded":"2024-09-20T18:50:23.371568+0000","last_fullsized":"2024-09-20T18:50:23.371568+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:05.774315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":62,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629291+0000","last_change":"2024-09-20T18:49:48.499421+0000","last_active":"2024-09-20T18:50:24.629291+0000","last_peered":"2024-09-20T18:50:24.629291+0000","last_clean":"2024-09-20T18:50:24.629291+0000","last_became_active":"2024-09-20T18:49:48.499111+0000","last_became_peered":"2024-09-20T18:49:48.499111+0000","last_unstale":"2024-09-20T18:50:24.629291+0000","last_undegraded":"2024-09-20T18:50:24.629291+0000","last_fullsized":"2024-09-20T18:50:24.629291+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:57.119627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":49,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.555318+0000","last_change":"2024-09-20T18:49:45.942406+0000","last_active":"2024-09-20T18:50:25.555318+0000","last_peered":"2024-09-20T18:50:25.555318+0000","last_clean":"2024-09-20T18:50:25.555318+0000","last_became_active":"2024-09-20T18:49:45.942105+0000","last_became_peered":"2024-09-20T18:49:45.942105+0000","last_unstale":"2024-09-20T18:50:25.555318+0000","last_undegraded":"2024-09-20T18:50:25.555318+0000","last_fullsized":"2024-09-20T18:50:25.555318+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:56.309962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629418+0000","last_change":"2024-09-20T18:49:49.959361+0000","last_active":"2024-09-20T18:50:24.629418+0000","last_peered":"2024-09-20T18:50:24.629418+0000","last_clean":"2024-09-20T18:50:24.629418+0000","last_became_active":"2024-09-20T18:49:49.958159+0000","last_became_peered":"2024-09-20T18:49:49.958159+0000","last_unstale":"2024-09-20T18:50:24.629418+0000","last_undegraded":"2024-09-20T18:50:24.629418+0000","last_fullsized":"2024-09-20T18:50:24.629418+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:12.385752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.628729+0000","last_change":"2024-09-20T18:49:52.140525+0000","last_active":"2024-09-20T18:50:24.628729+0000","last_peered":"2024-09-20T18:50:24.628729+0000","last_clean":"2024-09-20T18:50:24.628729+0000","last_became_active":"2024-09-20T18:49:52.139978+0000","last_became_peered":"2024-09-20T18:49:52.139978+0000","last_unstale":"2024-09-20T18:50:24.628729+0000","last_undegraded":"2024-09-20T18:50:24.628729+0000","last_fullsized":"2024-09-20T18:50:24.628729+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:19.358011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":49,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.628968+0000","last_change":"2024-09-20T18:49:47.955120+0000","last_active":"2024-09-20T18:50:24.628968+0000","last_peered":"2024-09-20T18:50:24.628968+0000","last_clean":"2024-09-20T18:50:24.628968+0000","last_became_active":"2024-09-20T18:49:47.954777+0000","last_became_peered":"2024-09-20T18:49:47.954777+0000","last_unstale":"2024-09-20T18:50:24.628968+0000","last_undegraded":"2024-09-20T18:50:24.628968+0000","last_fullsized":"2024-09-20T18:50:24.628968+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:04.481222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372637+0000","last_change":"2024-09-20T18:49:45.941248+0000","last_active":"2024-09-20T18:50:23.372637+0000","last_peered":"2024-09-20T18:50:23.372637+0000","last_clean":"2024-09-20T18:50:23.372637+0000","last_became_active":"2024-09-20T18:49:45.940988+0000","last_became_peered":"2024-09-20T18:49:45.940988+0000","last_unstale":"2024-09-20T18:50:23.372637+0000","last_undegraded":"2024-09-20T18:50:23.372637+0000","last_fullsized":"2024-09-20T18:50:23.372637+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:58.598619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.436254+0000","last_change":"2024-09-20T18:49:49.954779+0000","last_active":"2024-09-20T18:50:24.436254+0000","last_peered":"2024-09-20T18:50:24.436254+0000","last_clean":"2024-09-20T18:50:24.436254+0000","last_became_active":"2024-09-20T18:49:49.954502+0000","last_became_peered":"2024-09-20T18:49:49.954502+0000","last_unstale":"2024-09-20T18:50:24.436254+0000","last_undegraded":"2024-09-20T18:50:24.436254+0000","last_fullsized":"2024-09-20T18:50:24.436254+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:15.141742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629065+0000","last_change":"2024-09-20T18:49:51.968968+0000","last_active":"2024-09-20T18:50:24.629065+0000","last_peered":"2024-09-20T18:50:24.629065+0000","last_clean":"2024-09-20T18:50:24.629065+0000","last_became_active":"2024-09-20T18:49:51.968662+0000","last_became_peered":"2024-09-20T18:49:51.968662+0000","last_unstale":"2024-09-20T18:50:24.629065+0000","last_undegraded":"2024-09-20T18:50:24.629065+0000","last_fullsized":"2024-09-20T18:50:24.629065+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:02.092669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.554921+0000","last_change":"2024-09-20T18:49:51.968476+0000","last_active":"2024-09-20T18:50:25.554921+0000","last_peered":"2024-09-20T18:50:25.554921+0000","last_clean":"2024-09-20T18:50:25.554921+0000","last_became_active":"2024-09-20T18:49:51.968216+0000","last_became_peered":"2024-09-20T18:49:51.968216+0000","last_unstale":"2024-09-20T18:50:25.554921+0000","last_undegraded":"2024-09-20T18:50:25.554921+0000","last_fullsized":"2024-09-20T18:50:25.554921+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:11.930622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629136+0000","last_change":"2024-09-20T18:49:45.938413+0000","last_active":"2024-09-20T18:50:24.629136+0000","last_peered":"2024-09-20T18:50:24.629136+0000","last_clean":"2024-09-20T18:50:24.629136+0000","last_became_active":"2024-09-20T18:49:45.937992+0000","last_became_peered":"2024-09-20T18:49:45.937992+0000","last_unstale":"2024-09-20T18:50:24.629136+0000","last_undegraded":"2024-09-20T18:50:24.629136+0000","last_fullsized":"2024-09-20T18:50:24.629136+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:12.596336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":62,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.436878+0000","last_change":"2024-09-20T18:49:47.950306+0000","last_active":"2024-09-20T18:50:24.436878+0000","last_peered":"2024-09-20T18:50:24.436878+0000","last_clean":"2024-09-20T18:50:24.436878+0000","last_became_active":"2024-09-20T18:49:47.949582+0000","last_became_peered":"2024-09-20T18:49:47.949582+0000","last_unstale":"2024-09-20T18:50:24.436878+0000","last_undegraded":"2024-09-20T18:50:24.436878+0000","last_fullsized":"2024-09-20T18:50:24.436878+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:03.753158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372414+0000","last_change":"2024-09-20T18:49:49.957874+0000","last_active":"2024-09-20T18:50:23.372414+0000","last_peered":"2024-09-20T18:50:23.372414+0000","last_clean":"2024-09-20T18:50:23.372414+0000","last_became_active":"2024-09-20T18:49:49.957582+0000","last_became_peered":"2024-09-20T18:49:49.957582+0000","last_unstale":"2024-09-20T18:50:23.372414+0000","last_undegraded":"2024-09-20T18:50:23.372414+0000","last_fullsized":"2024-09-20T18:50:23.372414+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:07.454286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":35,"num_read_kb":30,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266700,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32540,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712612,"statfs":{"total":95995035648,"available":95961714688,"internally_reserved":0,"allocated":1384448,"data_stored":908550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528339,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28360,"kb_used_data":1332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716792,"statfs":{"total":95995035648,"available":95965995008,"internally_reserved":0,"allocated":1363968,"data_stored":907888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757272,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31944,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713208,"statfs":{"total":95995035648,"available":95962324992,"internally_reserved":0,"allocated":774144,"data_stored":317879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986205,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":806912,"data_stored":319131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182436,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31972,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713180,"statfs":{"total":95995035648,"available":95962296320,"internally_reserved":0,"allocated":802816,"data_stored":317975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411369,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31960,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713192,"statfs":{"total":95995035648,"available":95962308608,"internally_reserved":0,"allocated":790528,"data_stored":324917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574895,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":827392,"data_stored":319642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738421,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32540,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712612,"statfs":{"total":95995035648,"available":95961714688,"internally_reserved":0,"allocated":1384448,"data_stored":908246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T18:50:28.368 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T18:50:28.626 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:50:28.734 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:28 smithi083 ceph-mon[29421]: pgmap v13: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 13 op/s 2024-09-20T18:50:28.735 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:28 smithi083 ceph-mon[29421]: Adjusting osd_memory_target on smithi083 to 3760M 2024-09-20T18:50:28.735 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:28 smithi083 ceph-mon[29421]: Updating smithi083:/etc/ceph/ceph.conf 2024-09-20T18:50:28.735 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:28 smithi083 ceph-mon[29421]: Updating smithi142:/etc/ceph/ceph.conf 2024-09-20T18:50:28.735 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:28 smithi083 ceph-mon[29421]: Updating smithi083:/var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-20T18:50:28.735 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:28 smithi083 ceph-mon[29421]: Updating smithi142:/var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-20T18:50:28.735 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:28 smithi083 ceph-mon[29421]: Updating smithi083:/etc/ceph/ceph.client.admin.keyring 2024-09-20T18:50:28.735 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:28 smithi083 ceph-mon[29421]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-09-20T18:50:28.735 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:28 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:28.735 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:28 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:28.735 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:28 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:28.735 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:28 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:28.735 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:28 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:28.736 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:28 smithi083 ceph-mon[36260]: pgmap v13: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 13 op/s 2024-09-20T18:50:28.736 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:28 smithi083 ceph-mon[36260]: Adjusting osd_memory_target on smithi083 to 3760M 2024-09-20T18:50:28.736 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:28 smithi083 ceph-mon[36260]: Updating smithi083:/etc/ceph/ceph.conf 2024-09-20T18:50:28.736 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:28 smithi083 ceph-mon[36260]: Updating smithi142:/etc/ceph/ceph.conf 2024-09-20T18:50:28.736 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:28 smithi083 ceph-mon[36260]: Updating smithi083:/var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-20T18:50:28.736 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:28 smithi083 ceph-mon[36260]: Updating smithi142:/var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-20T18:50:28.736 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:28 smithi083 ceph-mon[36260]: Updating smithi083:/etc/ceph/ceph.client.admin.keyring 2024-09-20T18:50:28.736 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:28 smithi083 ceph-mon[36260]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-09-20T18:50:28.736 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:28 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:28.736 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:28 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:28.736 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:28 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:28.736 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:28 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:28.736 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:28 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:28.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:28 smithi142 ceph-mon[32508]: pgmap v13: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 13 op/s 2024-09-20T18:50:28.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:28 smithi142 ceph-mon[32508]: Adjusting osd_memory_target on smithi083 to 3760M 2024-09-20T18:50:28.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:28 smithi142 ceph-mon[32508]: Updating smithi083:/etc/ceph/ceph.conf 2024-09-20T18:50:28.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:28 smithi142 ceph-mon[32508]: Updating smithi142:/etc/ceph/ceph.conf 2024-09-20T18:50:28.775 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:28 smithi142 ceph-mon[32508]: Updating smithi083:/var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-20T18:50:28.775 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:28 smithi142 ceph-mon[32508]: Updating smithi142:/var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-20T18:50:28.775 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:28 smithi142 ceph-mon[32508]: Updating smithi083:/etc/ceph/ceph.client.admin.keyring 2024-09-20T18:50:28.775 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:28 smithi142 ceph-mon[32508]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-09-20T18:50:28.775 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:28 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:28.775 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:28 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:28.775 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:28 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:28.776 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:28 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:28.776 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:28 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:29.361 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:50:29.363 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-20T18:50:29.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:29 smithi142 ceph-mon[32508]: from='client.24581 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:50:29.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:29 smithi142 ceph-mon[32508]: Updating smithi083:/var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-20T18:50:29.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:29 smithi142 ceph-mon[32508]: Updating smithi142:/var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-20T18:50:29.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:29 smithi142 ceph-mon[32508]: Deploying daemon alertmanager.a on smithi083 2024-09-20T18:50:29.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:29 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:50:29.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:29 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:50:29.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:29 smithi142 ceph-mon[32508]: osdmap e79: 8 total, 8 up, 8 in 2024-09-20T18:50:29.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:29 smithi083 ceph-mon[36260]: from='client.24581 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:50:29.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:29 smithi083 ceph-mon[36260]: Updating smithi083:/var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-20T18:50:29.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:29 smithi083 ceph-mon[36260]: Updating smithi142:/var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-20T18:50:29.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:29 smithi083 ceph-mon[36260]: Deploying daemon alertmanager.a on smithi083 2024-09-20T18:50:29.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:29 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:50:29.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:29 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:50:29.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:29 smithi083 ceph-mon[36260]: osdmap e79: 8 total, 8 up, 8 in 2024-09-20T18:50:29.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:29 smithi083 ceph-mon[29421]: from='client.24581 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:50:29.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:29 smithi083 ceph-mon[29421]: Updating smithi083:/var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-20T18:50:29.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:29 smithi083 ceph-mon[29421]: Updating smithi142:/var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-20T18:50:29.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:29 smithi083 ceph-mon[29421]: Deploying daemon alertmanager.a on smithi083 2024-09-20T18:50:29.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:29 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:50:29.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:29 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:50:29.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:29 smithi083 ceph-mon[29421]: osdmap e79: 8 total, 8 up, 8 in 2024-09-20T18:50:30.350 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2024-09-20T18:50:28.809323+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":883,"num_read_kb":740,"num_write":549,"num_write_kb":1673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":590588,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":552,"ondisk_log_size":552,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":244968,"kb_used_data":7944,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":749716248,"statfs":{"total":767960285184,"available":767709437952,"internally_reserved":0,"allocated":8134656,"data_stored":4324228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.057668"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372596+0000","last_change":"2024-09-20T18:49:45.943560+0000","last_active":"2024-09-20T18:50:23.372596+0000","last_peered":"2024-09-20T18:50:23.372596+0000","last_clean":"2024-09-20T18:50:23.372596+0000","last_became_active":"2024-09-20T18:49:45.943201+0000","last_became_peered":"2024-09-20T18:49:45.943201+0000","last_unstale":"2024-09-20T18:50:23.372596+0000","last_undegraded":"2024-09-20T18:50:23.372596+0000","last_fullsized":"2024-09-20T18:50:23.372596+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:11.286122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":59,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372438+0000","last_change":"2024-09-20T18:49:47.947057+0000","last_active":"2024-09-20T18:50:23.372438+0000","last_peered":"2024-09-20T18:50:23.372438+0000","last_clean":"2024-09-20T18:50:23.372438+0000","last_became_active":"2024-09-20T18:49:47.946161+0000","last_became_peered":"2024-09-20T18:49:47.946161+0000","last_unstale":"2024-09-20T18:50:23.372438+0000","last_undegraded":"2024-09-20T18:50:23.372438+0000","last_fullsized":"2024-09-20T18:50:23.372438+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.579926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.554773+0000","last_change":"2024-09-20T18:49:49.959797+0000","last_active":"2024-09-20T18:50:25.554773+0000","last_peered":"2024-09-20T18:50:25.554773+0000","last_clean":"2024-09-20T18:50:25.554773+0000","last_became_active":"2024-09-20T18:49:49.959514+0000","last_became_peered":"2024-09-20T18:49:49.959514+0000","last_unstale":"2024-09-20T18:50:25.554773+0000","last_undegraded":"2024-09-20T18:50:25.554773+0000","last_fullsized":"2024-09-20T18:50:25.554773+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:49.805774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372453+0000","last_change":"2024-09-20T18:49:51.969687+0000","last_active":"2024-09-20T18:50:23.372453+0000","last_peered":"2024-09-20T18:50:23.372453+0000","last_clean":"2024-09-20T18:50:23.372453+0000","last_became_active":"2024-09-20T18:49:51.969421+0000","last_became_peered":"2024-09-20T18:49:51.969421+0000","last_unstale":"2024-09-20T18:50:23.372453+0000","last_undegraded":"2024-09-20T18:50:23.372453+0000","last_fullsized":"2024-09-20T18:50:23.372453+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.534291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629364+0000","last_change":"2024-09-20T18:49:52.142168+0000","last_active":"2024-09-20T18:50:24.629364+0000","last_peered":"2024-09-20T18:50:24.629364+0000","last_clean":"2024-09-20T18:50:24.629364+0000","last_became_active":"2024-09-20T18:49:52.141967+0000","last_became_peered":"2024-09-20T18:49:52.141967+0000","last_unstale":"2024-09-20T18:50:24.629364+0000","last_undegraded":"2024-09-20T18:50:24.629364+0000","last_fullsized":"2024-09-20T18:50:24.629364+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.114636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.630347+0000","last_change":"2024-09-20T18:49:45.943889+0000","last_active":"2024-09-20T18:50:24.630347+0000","last_peered":"2024-09-20T18:50:24.630347+0000","last_clean":"2024-09-20T18:50:24.630347+0000","last_became_active":"2024-09-20T18:49:45.943607+0000","last_became_peered":"2024-09-20T18:49:45.943607+0000","last_unstale":"2024-09-20T18:50:24.630347+0000","last_undegraded":"2024-09-20T18:50:24.630347+0000","last_fullsized":"2024-09-20T18:50:24.630347+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.033360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":68,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629459+0000","last_change":"2024-09-20T18:49:47.952638+0000","last_active":"2024-09-20T18:50:24.629459+0000","last_peered":"2024-09-20T18:50:24.629459+0000","last_clean":"2024-09-20T18:50:24.629459+0000","last_became_active":"2024-09-20T18:49:47.952150+0000","last_became_peered":"2024-09-20T18:49:47.952150+0000","last_unstale":"2024-09-20T18:50:24.629459+0000","last_undegraded":"2024-09-20T18:50:24.629459+0000","last_fullsized":"2024-09-20T18:50:24.629459+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:02.798315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372432+0000","last_change":"2024-09-20T18:49:49.956717+0000","last_active":"2024-09-20T18:50:23.372432+0000","last_peered":"2024-09-20T18:50:23.372432+0000","last_clean":"2024-09-20T18:50:23.372432+0000","last_became_active":"2024-09-20T18:49:49.956445+0000","last_became_peered":"2024-09-20T18:49:49.956445+0000","last_unstale":"2024-09-20T18:50:23.372432+0000","last_undegraded":"2024-09-20T18:50:23.372432+0000","last_fullsized":"2024-09-20T18:50:23.372432+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:53.841144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"59'1","reported_seq":55,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.555194+0000","last_change":"2024-09-20T18:49:45.946450+0000","last_active":"2024-09-20T18:50:25.555194+0000","last_peered":"2024-09-20T18:50:25.555194+0000","last_clean":"2024-09-20T18:50:25.555194+0000","last_became_active":"2024-09-20T18:49:45.946171+0000","last_became_peered":"2024-09-20T18:49:45.946171+0000","last_unstale":"2024-09-20T18:50:25.555194+0000","last_undegraded":"2024-09-20T18:50:25.555194+0000","last_fullsized":"2024-09-20T18:50:25.555194+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:11.299334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":59,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372776+0000","last_change":"2024-09-20T18:49:47.947755+0000","last_active":"2024-09-20T18:50:23.372776+0000","last_peered":"2024-09-20T18:50:23.372776+0000","last_clean":"2024-09-20T18:50:23.372776+0000","last_became_active":"2024-09-20T18:49:47.947448+0000","last_became_peered":"2024-09-20T18:49:47.947448+0000","last_unstale":"2024-09-20T18:50:23.372776+0000","last_undegraded":"2024-09-20T18:50:23.372776+0000","last_fullsized":"2024-09-20T18:50:23.372776+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.588680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.554705+0000","last_change":"2024-09-20T18:49:49.961448+0000","last_active":"2024-09-20T18:50:25.554705+0000","last_peered":"2024-09-20T18:50:25.554705+0000","last_clean":"2024-09-20T18:50:25.554705+0000","last_became_active":"2024-09-20T18:49:49.961188+0000","last_became_peered":"2024-09-20T18:49:49.961188+0000","last_unstale":"2024-09-20T18:50:25.554705+0000","last_undegraded":"2024-09-20T18:50:25.554705+0000","last_fullsized":"2024-09-20T18:50:25.554705+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:12.456737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.371635+0000","last_change":"2024-09-20T18:49:51.966957+0000","last_active":"2024-09-20T18:50:23.371635+0000","last_peered":"2024-09-20T18:50:23.371635+0000","last_clean":"2024-09-20T18:50:23.371635+0000","last_became_active":"2024-09-20T18:49:51.966715+0000","last_became_peered":"2024-09-20T18:49:51.966715+0000","last_unstale":"2024-09-20T18:50:23.371635+0000","last_undegraded":"2024-09-20T18:50:23.371635+0000","last_fullsized":"2024-09-20T18:50:23.371635+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:54.613818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":49,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.555029+0000","last_change":"2024-09-20T18:49:45.945004+0000","last_active":"2024-09-20T18:50:25.555029+0000","last_peered":"2024-09-20T18:50:25.555029+0000","last_clean":"2024-09-20T18:50:25.555029+0000","last_became_active":"2024-09-20T18:49:45.944703+0000","last_became_peered":"2024-09-20T18:49:45.944703+0000","last_unstale":"2024-09-20T18:50:25.555029+0000","last_undegraded":"2024-09-20T18:50:25.555029+0000","last_fullsized":"2024-09-20T18:50:25.555029+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:53.394518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":66,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372594+0000","last_change":"2024-09-20T18:49:58.999374+0000","last_active":"2024-09-20T18:50:23.372594+0000","last_peered":"2024-09-20T18:50:23.372594+0000","last_clean":"2024-09-20T18:50:23.372594+0000","last_became_active":"2024-09-20T18:49:58.995638+0000","last_became_peered":"2024-09-20T18:49:58.995638+0000","last_unstale":"2024-09-20T18:50:23.372594+0000","last_undegraded":"2024-09-20T18:50:23.372594+0000","last_fullsized":"2024-09-20T18:50:23.372594+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:05.131889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629604+0000","last_change":"2024-09-20T18:49:49.960001+0000","last_active":"2024-09-20T18:50:24.629604+0000","last_peered":"2024-09-20T18:50:24.629604+0000","last_clean":"2024-09-20T18:50:24.629604+0000","last_became_active":"2024-09-20T18:49:49.959696+0000","last_became_peered":"2024-09-20T18:49:49.959696+0000","last_unstale":"2024-09-20T18:50:24.629604+0000","last_undegraded":"2024-09-20T18:50:24.629604+0000","last_fullsized":"2024-09-20T18:50:24.629604+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.740255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.554479+0000","last_change":"2024-09-20T18:49:51.970933+0000","last_active":"2024-09-20T18:50:25.554479+0000","last_peered":"2024-09-20T18:50:25.554479+0000","last_clean":"2024-09-20T18:50:25.554479+0000","last_became_active":"2024-09-20T18:49:51.970663+0000","last_became_peered":"2024-09-20T18:49:51.970663+0000","last_unstale":"2024-09-20T18:50:25.554479+0000","last_undegraded":"2024-09-20T18:50:25.554479+0000","last_fullsized":"2024-09-20T18:50:25.554479+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:07.680885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.373280+0000","last_change":"2024-09-20T18:49:52.141482+0000","last_active":"2024-09-20T18:50:23.373280+0000","last_peered":"2024-09-20T18:50:23.373280+0000","last_clean":"2024-09-20T18:50:23.373280+0000","last_became_active":"2024-09-20T18:49:52.141170+0000","last_became_peered":"2024-09-20T18:49:52.141170+0000","last_unstale":"2024-09-20T18:50:23.373280+0000","last_undegraded":"2024-09-20T18:50:23.373280+0000","last_fullsized":"2024-09-20T18:50:23.373280+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:15.351627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372298+0000","last_change":"2024-09-20T18:49:45.950714+0000","last_active":"2024-09-20T18:50:23.372298+0000","last_peered":"2024-09-20T18:50:23.372298+0000","last_clean":"2024-09-20T18:50:23.372298+0000","last_became_active":"2024-09-20T18:49:45.950480+0000","last_became_peered":"2024-09-20T18:49:45.950480+0000","last_unstale":"2024-09-20T18:50:23.372298+0000","last_undegraded":"2024-09-20T18:50:23.372298+0000","last_fullsized":"2024-09-20T18:50:23.372298+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:07.919860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":71,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.556302+0000","last_change":"2024-09-20T18:49:47.951228+0000","last_active":"2024-09-20T18:50:25.556302+0000","last_peered":"2024-09-20T18:50:25.556302+0000","last_clean":"2024-09-20T18:50:25.556302+0000","last_became_active":"2024-09-20T18:49:47.950939+0000","last_became_peered":"2024-09-20T18:49:47.950939+0000","last_unstale":"2024-09-20T18:50:25.556302+0000","last_undegraded":"2024-09-20T18:50:25.556302+0000","last_fullsized":"2024-09-20T18:50:25.556302+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:08.025260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.555139+0000","last_change":"2024-09-20T18:49:49.958867+0000","last_active":"2024-09-20T18:50:25.555139+0000","last_peered":"2024-09-20T18:50:25.555139+0000","last_clean":"2024-09-20T18:50:25.555139+0000","last_became_active":"2024-09-20T18:49:49.958518+0000","last_became_peered":"2024-09-20T18:49:49.958518+0000","last_unstale":"2024-09-20T18:50:25.555139+0000","last_undegraded":"2024-09-20T18:50:25.555139+0000","last_fullsized":"2024-09-20T18:50:25.555139+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:55.540498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.630339+0000","last_change":"2024-09-20T18:49:52.140362+0000","last_active":"2024-09-20T18:50:24.630339+0000","last_peered":"2024-09-20T18:50:24.630339+0000","last_clean":"2024-09-20T18:50:24.630339+0000","last_became_active":"2024-09-20T18:49:52.140040+0000","last_became_peered":"2024-09-20T18:49:52.140040+0000","last_unstale":"2024-09-20T18:50:24.630339+0000","last_undegraded":"2024-09-20T18:50:24.630339+0000","last_fullsized":"2024-09-20T18:50:24.630339+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:52.839541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"59'1","reported_seq":53,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372906+0000","last_change":"2024-09-20T18:49:45.942193+0000","last_active":"2024-09-20T18:50:23.372906+0000","last_peered":"2024-09-20T18:50:23.372906+0000","last_clean":"2024-09-20T18:50:23.372906+0000","last_became_active":"2024-09-20T18:49:45.940100+0000","last_became_peered":"2024-09-20T18:49:45.940100+0000","last_unstale":"2024-09-20T18:50:23.372906+0000","last_undegraded":"2024-09-20T18:50:23.372906+0000","last_fullsized":"2024-09-20T18:50:23.372906+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:52.525377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":66,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.630180+0000","last_change":"2024-09-20T18:49:47.951215+0000","last_active":"2024-09-20T18:50:24.630180+0000","last_peered":"2024-09-20T18:50:24.630180+0000","last_clean":"2024-09-20T18:50:24.630180+0000","last_became_active":"2024-09-20T18:49:47.950909+0000","last_became_peered":"2024-09-20T18:49:47.950909+0000","last_unstale":"2024-09-20T18:50:24.630180+0000","last_undegraded":"2024-09-20T18:50:24.630180+0000","last_fullsized":"2024-09-20T18:50:24.630180+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:52.254250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372880+0000","last_change":"2024-09-20T18:49:49.959365+0000","last_active":"2024-09-20T18:50:23.372880+0000","last_peered":"2024-09-20T18:50:23.372880+0000","last_clean":"2024-09-20T18:50:23.372880+0000","last_became_active":"2024-09-20T18:49:49.959102+0000","last_became_peered":"2024-09-20T18:49:49.959102+0000","last_unstale":"2024-09-20T18:50:23.372880+0000","last_undegraded":"2024-09-20T18:50:23.372880+0000","last_fullsized":"2024-09-20T18:50:23.372880+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:04.872284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629176+0000","last_change":"2024-09-20T18:49:51.970469+0000","last_active":"2024-09-20T18:50:24.629176+0000","last_peered":"2024-09-20T18:50:24.629176+0000","last_clean":"2024-09-20T18:50:24.629176+0000","last_became_active":"2024-09-20T18:49:51.970058+0000","last_became_peered":"2024-09-20T18:49:51.970058+0000","last_unstale":"2024-09-20T18:50:24.629176+0000","last_undegraded":"2024-09-20T18:50:24.629176+0000","last_fullsized":"2024-09-20T18:50:24.629176+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:59.400214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":50,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.554920+0000","last_change":"2024-09-20T18:49:45.933354+0000","last_active":"2024-09-20T18:50:25.554920+0000","last_peered":"2024-09-20T18:50:25.554920+0000","last_clean":"2024-09-20T18:50:25.554920+0000","last_became_active":"2024-09-20T18:49:45.933078+0000","last_became_peered":"2024-09-20T18:49:45.933078+0000","last_unstale":"2024-09-20T18:50:25.554920+0000","last_undegraded":"2024-09-20T18:50:25.554920+0000","last_fullsized":"2024-09-20T18:50:25.554920+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:07.425698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":58,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372450+0000","last_change":"2024-09-20T18:49:48.501151+0000","last_active":"2024-09-20T18:50:23.372450+0000","last_peered":"2024-09-20T18:50:23.372450+0000","last_clean":"2024-09-20T18:50:23.372450+0000","last_became_active":"2024-09-20T18:49:48.500809+0000","last_became_peered":"2024-09-20T18:49:48.500809+0000","last_unstale":"2024-09-20T18:50:23.372450+0000","last_undegraded":"2024-09-20T18:50:23.372450+0000","last_fullsized":"2024-09-20T18:50:23.372450+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:07.803937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":58,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.436880+0000","last_change":"2024-09-20T18:49:49.955074+0000","last_active":"2024-09-20T18:50:24.436880+0000","last_peered":"2024-09-20T18:50:24.436880+0000","last_clean":"2024-09-20T18:50:24.436880+0000","last_became_active":"2024-09-20T18:49:49.954772+0000","last_became_peered":"2024-09-20T18:49:49.954772+0000","last_unstale":"2024-09-20T18:50:24.436880+0000","last_undegraded":"2024-09-20T18:50:24.436880+0000","last_fullsized":"2024-09-20T18:50:24.436880+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:53.352545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":46,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.555765+0000","last_change":"2024-09-20T18:49:59.004173+0000","last_active":"2024-09-20T18:50:25.555765+0000","last_peered":"2024-09-20T18:50:25.555765+0000","last_clean":"2024-09-20T18:50:25.555765+0000","last_became_active":"2024-09-20T18:49:58.992302+0000","last_became_peered":"2024-09-20T18:49:58.992302+0000","last_unstale":"2024-09-20T18:50:25.555765+0000","last_undegraded":"2024-09-20T18:50:25.555765+0000","last_fullsized":"2024-09-20T18:50:25.555765+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:14.502406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629232+0000","last_change":"2024-09-20T18:49:45.936413+0000","last_active":"2024-09-20T18:50:24.629232+0000","last_peered":"2024-09-20T18:50:24.629232+0000","last_clean":"2024-09-20T18:50:24.629232+0000","last_became_active":"2024-09-20T18:49:45.936104+0000","last_became_peered":"2024-09-20T18:49:45.936104+0000","last_unstale":"2024-09-20T18:50:24.629232+0000","last_undegraded":"2024-09-20T18:50:24.629232+0000","last_fullsized":"2024-09-20T18:50:24.629232+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:01.319743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":58,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.373444+0000","last_change":"2024-09-20T18:49:49.960978+0000","last_active":"2024-09-20T18:50:23.373444+0000","last_peered":"2024-09-20T18:50:23.373444+0000","last_clean":"2024-09-20T18:50:23.373444+0000","last_became_active":"2024-09-20T18:49:49.960697+0000","last_became_peered":"2024-09-20T18:49:49.960697+0000","last_unstale":"2024-09-20T18:50:23.373444+0000","last_undegraded":"2024-09-20T18:50:23.373444+0000","last_fullsized":"2024-09-20T18:50:23.373444+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:54.719450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.554863+0000","last_change":"2024-09-20T18:49:51.971713+0000","last_active":"2024-09-20T18:50:25.554863+0000","last_peered":"2024-09-20T18:50:25.554863+0000","last_clean":"2024-09-20T18:50:25.554863+0000","last_became_active":"2024-09-20T18:49:51.971399+0000","last_became_peered":"2024-09-20T18:49:51.971399+0000","last_unstale":"2024-09-20T18:50:25.554863+0000","last_undegraded":"2024-09-20T18:50:25.554863+0000","last_fullsized":"2024-09-20T18:50:25.554863+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:18.352126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":75,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629119+0000","last_change":"2024-09-20T18:49:48.499569+0000","last_active":"2024-09-20T18:50:24.629119+0000","last_peered":"2024-09-20T18:50:24.629119+0000","last_clean":"2024-09-20T18:50:24.629119+0000","last_became_active":"2024-09-20T18:49:48.499189+0000","last_became_peered":"2024-09-20T18:49:48.499189+0000","last_unstale":"2024-09-20T18:50:24.629119+0000","last_undegraded":"2024-09-20T18:50:24.629119+0000","last_fullsized":"2024-09-20T18:50:24.629119+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:16.224188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.630461+0000","last_change":"2024-09-20T18:49:45.940816+0000","last_active":"2024-09-20T18:50:24.630461+0000","last_peered":"2024-09-20T18:50:24.630461+0000","last_clean":"2024-09-20T18:50:24.630461+0000","last_became_active":"2024-09-20T18:49:45.940544+0000","last_became_peered":"2024-09-20T18:49:45.940544+0000","last_unstale":"2024-09-20T18:50:24.630461+0000","last_undegraded":"2024-09-20T18:50:24.630461+0000","last_fullsized":"2024-09-20T18:50:24.630461+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:46.957556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372455+0000","last_change":"2024-09-20T18:49:49.963362+0000","last_active":"2024-09-20T18:50:23.372455+0000","last_peered":"2024-09-20T18:50:23.372455+0000","last_clean":"2024-09-20T18:50:23.372455+0000","last_became_active":"2024-09-20T18:49:49.963073+0000","last_became_peered":"2024-09-20T18:49:49.963073+0000","last_unstale":"2024-09-20T18:50:23.372455+0000","last_undegraded":"2024-09-20T18:50:23.372455+0000","last_fullsized":"2024-09-20T18:50:23.372455+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:01.468417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372690+0000","last_change":"2024-09-20T18:49:51.969120+0000","last_active":"2024-09-20T18:50:23.372690+0000","last_peered":"2024-09-20T18:50:23.372690+0000","last_clean":"2024-09-20T18:50:23.372690+0000","last_became_active":"2024-09-20T18:49:51.968793+0000","last_became_peered":"2024-09-20T18:49:51.968793+0000","last_unstale":"2024-09-20T18:50:23.372690+0000","last_undegraded":"2024-09-20T18:50:23.372690+0000","last_fullsized":"2024-09-20T18:50:23.372690+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:57.615454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":64,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372452+0000","last_change":"2024-09-20T18:49:47.942407+0000","last_active":"2024-09-20T18:50:23.372452+0000","last_peered":"2024-09-20T18:50:23.372452+0000","last_clean":"2024-09-20T18:50:23.372452+0000","last_became_active":"2024-09-20T18:49:47.941895+0000","last_became_peered":"2024-09-20T18:49:47.941895+0000","last_unstale":"2024-09-20T18:50:23.372452+0000","last_undegraded":"2024-09-20T18:50:23.372452+0000","last_fullsized":"2024-09-20T18:50:23.372452+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:08.413407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372573+0000","last_change":"2024-09-20T18:49:45.940728+0000","last_active":"2024-09-20T18:50:23.372573+0000","last_peered":"2024-09-20T18:50:23.372573+0000","last_clean":"2024-09-20T18:50:23.372573+0000","last_became_active":"2024-09-20T18:49:45.940469+0000","last_became_peered":"2024-09-20T18:49:45.940469+0000","last_unstale":"2024-09-20T18:50:23.372573+0000","last_undegraded":"2024-09-20T18:50:23.372573+0000","last_fullsized":"2024-09-20T18:50:23.372573+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:05.840900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"66'8","reported_seq":55,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.628974+0000","last_change":"2024-09-20T18:49:49.956436+0000","last_active":"2024-09-20T18:50:24.628974+0000","last_peered":"2024-09-20T18:50:24.628974+0000","last_clean":"2024-09-20T18:50:24.628974+0000","last_became_active":"2024-09-20T18:49:49.956133+0000","last_became_peered":"2024-09-20T18:49:49.956133+0000","last_unstale":"2024-09-20T18:50:24.628974+0000","last_undegraded":"2024-09-20T18:50:24.628974+0000","last_fullsized":"2024-09-20T18:50:24.628974+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:03.649831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629069+0000","last_change":"2024-09-20T18:49:52.142481+0000","last_active":"2024-09-20T18:50:24.629069+0000","last_peered":"2024-09-20T18:50:24.629069+0000","last_clean":"2024-09-20T18:50:24.629069+0000","last_became_active":"2024-09-20T18:49:52.142113+0000","last_became_peered":"2024-09-20T18:49:52.142113+0000","last_unstale":"2024-09-20T18:50:24.629069+0000","last_undegraded":"2024-09-20T18:50:24.629069+0000","last_fullsized":"2024-09-20T18:50:24.629069+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.140166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":60,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.555344+0000","last_change":"2024-09-20T18:49:47.949952+0000","last_active":"2024-09-20T18:50:25.555344+0000","last_peered":"2024-09-20T18:50:25.555344+0000","last_clean":"2024-09-20T18:50:25.555344+0000","last_became_active":"2024-09-20T18:49:47.949626+0000","last_became_peered":"2024-09-20T18:49:47.949626+0000","last_unstale":"2024-09-20T18:50:25.555344+0000","last_undegraded":"2024-09-20T18:50:25.555344+0000","last_fullsized":"2024-09-20T18:50:25.555344+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:50.929449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":49,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.555297+0000","last_change":"2024-09-20T18:49:45.942852+0000","last_active":"2024-09-20T18:50:25.555297+0000","last_peered":"2024-09-20T18:50:25.555297+0000","last_clean":"2024-09-20T18:50:25.555297+0000","last_became_active":"2024-09-20T18:49:45.942496+0000","last_became_peered":"2024-09-20T18:49:45.942496+0000","last_unstale":"2024-09-20T18:50:25.555297+0000","last_undegraded":"2024-09-20T18:50:25.555297+0000","last_fullsized":"2024-09-20T18:50:25.555297+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:55.917563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":55,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372071+0000","last_change":"2024-09-20T18:49:49.959699+0000","last_active":"2024-09-20T18:50:23.372071+0000","last_peered":"2024-09-20T18:50:23.372071+0000","last_clean":"2024-09-20T18:50:23.372071+0000","last_became_active":"2024-09-20T18:49:49.959365+0000","last_became_peered":"2024-09-20T18:49:49.959365+0000","last_unstale":"2024-09-20T18:50:23.372071+0000","last_undegraded":"2024-09-20T18:50:23.372071+0000","last_fullsized":"2024-09-20T18:50:23.372071+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:01.818798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.371823+0000","last_change":"2024-09-20T18:49:51.965733+0000","last_active":"2024-09-20T18:50:23.371823+0000","last_peered":"2024-09-20T18:50:23.371823+0000","last_clean":"2024-09-20T18:50:23.371823+0000","last_became_active":"2024-09-20T18:49:51.964780+0000","last_became_peered":"2024-09-20T18:49:51.964780+0000","last_unstale":"2024-09-20T18:50:23.371823+0000","last_undegraded":"2024-09-20T18:50:23.371823+0000","last_fullsized":"2024-09-20T18:50:23.371823+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:11.275213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":79,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372181+0000","last_change":"2024-09-20T18:49:48.498141+0000","last_active":"2024-09-20T18:50:23.372181+0000","last_peered":"2024-09-20T18:50:23.372181+0000","last_clean":"2024-09-20T18:50:23.372181+0000","last_became_active":"2024-09-20T18:49:48.497736+0000","last_became_peered":"2024-09-20T18:49:48.497736+0000","last_unstale":"2024-09-20T18:50:23.372181+0000","last_undegraded":"2024-09-20T18:50:23.372181+0000","last_fullsized":"2024-09-20T18:50:23.372181+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.986657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":49,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.555602+0000","last_change":"2024-09-20T18:49:45.942589+0000","last_active":"2024-09-20T18:50:25.555602+0000","last_peered":"2024-09-20T18:50:25.555602+0000","last_clean":"2024-09-20T18:50:25.555602+0000","last_became_active":"2024-09-20T18:49:45.942253+0000","last_became_peered":"2024-09-20T18:49:45.942253+0000","last_unstale":"2024-09-20T18:50:25.555602+0000","last_undegraded":"2024-09-20T18:50:25.555602+0000","last_fullsized":"2024-09-20T18:50:25.555602+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.673189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.436485+0000","last_change":"2024-09-20T18:49:49.954594+0000","last_active":"2024-09-20T18:50:24.436485+0000","last_peered":"2024-09-20T18:50:24.436485+0000","last_clean":"2024-09-20T18:50:24.436485+0000","last_became_active":"2024-09-20T18:49:49.954299+0000","last_became_peered":"2024-09-20T18:49:49.954299+0000","last_unstale":"2024-09-20T18:50:24.436485+0000","last_undegraded":"2024-09-20T18:50:24.436485+0000","last_fullsized":"2024-09-20T18:50:24.436485+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:52.123030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.554684+0000","last_change":"2024-09-20T18:49:52.140276+0000","last_active":"2024-09-20T18:50:25.554684+0000","last_peered":"2024-09-20T18:50:25.554684+0000","last_clean":"2024-09-20T18:50:25.554684+0000","last_became_active":"2024-09-20T18:49:52.139981+0000","last_became_peered":"2024-09-20T18:49:52.139981+0000","last_unstale":"2024-09-20T18:50:25.554684+0000","last_undegraded":"2024-09-20T18:50:25.554684+0000","last_fullsized":"2024-09-20T18:50:25.554684+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:52.117965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"66'28","reported_seq":101,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.555578+0000","last_change":"2024-09-20T18:49:47.955615+0000","last_active":"2024-09-20T18:50:25.555578+0000","last_peered":"2024-09-20T18:50:25.555578+0000","last_clean":"2024-09-20T18:50:25.555578+0000","last_became_active":"2024-09-20T18:49:47.955335+0000","last_became_peered":"2024-09-20T18:49:47.955335+0000","last_unstale":"2024-09-20T18:50:25.555578+0000","last_undegraded":"2024-09-20T18:50:25.555578+0000","last_fullsized":"2024-09-20T18:50:25.555578+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:47.506576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.373045+0000","last_change":"2024-09-20T18:49:45.945275+0000","last_active":"2024-09-20T18:50:23.373045+0000","last_peered":"2024-09-20T18:50:23.373045+0000","last_clean":"2024-09-20T18:50:23.373045+0000","last_became_active":"2024-09-20T18:49:45.944992+0000","last_became_peered":"2024-09-20T18:49:45.944992+0000","last_unstale":"2024-09-20T18:50:23.373045+0000","last_undegraded":"2024-09-20T18:50:23.373045+0000","last_fullsized":"2024-09-20T18:50:23.373045+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:47.768689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":59,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.554638+0000","last_change":"2024-09-20T18:49:47.940836+0000","last_active":"2024-09-20T18:50:25.554638+0000","last_peered":"2024-09-20T18:50:25.554638+0000","last_clean":"2024-09-20T18:50:25.554638+0000","last_became_active":"2024-09-20T18:49:44.936027+0000","last_became_peered":"2024-09-20T18:49:44.936027+0000","last_unstale":"2024-09-20T18:50:25.554638+0000","last_undegraded":"2024-09-20T18:50:25.554638+0000","last_fullsized":"2024-09-20T18:50:25.554638+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:49:43.921002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:47.157665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012781159999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372293+0000","last_change":"2024-09-20T18:49:49.954281+0000","last_active":"2024-09-20T18:50:23.372293+0000","last_peered":"2024-09-20T18:50:23.372293+0000","last_clean":"2024-09-20T18:50:23.372293+0000","last_became_active":"2024-09-20T18:49:49.953971+0000","last_became_peered":"2024-09-20T18:49:49.953971+0000","last_unstale":"2024-09-20T18:50:23.372293+0000","last_undegraded":"2024-09-20T18:50:23.372293+0000","last_fullsized":"2024-09-20T18:50:23.372293+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:12.341509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.628803+0000","last_change":"2024-09-20T18:49:51.969702+0000","last_active":"2024-09-20T18:50:24.628803+0000","last_peered":"2024-09-20T18:50:24.628803+0000","last_clean":"2024-09-20T18:50:24.628803+0000","last_became_active":"2024-09-20T18:49:51.969400+0000","last_became_peered":"2024-09-20T18:49:51.969400+0000","last_unstale":"2024-09-20T18:50:24.628803+0000","last_undegraded":"2024-09-20T18:50:24.628803+0000","last_fullsized":"2024-09-20T18:50:24.628803+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.486248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":72,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.555137+0000","last_change":"2024-09-20T18:49:47.951311+0000","last_active":"2024-09-20T18:50:25.555137+0000","last_peered":"2024-09-20T18:50:25.555137+0000","last_clean":"2024-09-20T18:50:25.555137+0000","last_became_active":"2024-09-20T18:49:47.950904+0000","last_became_peered":"2024-09-20T18:49:47.950904+0000","last_unstale":"2024-09-20T18:50:25.555137+0000","last_undegraded":"2024-09-20T18:50:25.555137+0000","last_fullsized":"2024-09-20T18:50:25.555137+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:06.445493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":49,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.554810+0000","last_change":"2024-09-20T18:49:45.940667+0000","last_active":"2024-09-20T18:50:25.554810+0000","last_peered":"2024-09-20T18:50:25.554810+0000","last_clean":"2024-09-20T18:50:25.554810+0000","last_became_active":"2024-09-20T18:49:45.939884+0000","last_became_peered":"2024-09-20T18:49:45.939884+0000","last_unstale":"2024-09-20T18:50:25.554810+0000","last_undegraded":"2024-09-20T18:50:25.554810+0000","last_fullsized":"2024-09-20T18:50:25.554810+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:49.675839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":59,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.554624+0000","last_change":"2024-09-20T18:49:47.939858+0000","last_active":"2024-09-20T18:50:25.554624+0000","last_peered":"2024-09-20T18:50:25.554624+0000","last_clean":"2024-09-20T18:50:25.554624+0000","last_became_active":"2024-09-20T18:49:44.937135+0000","last_became_peered":"2024-09-20T18:49:44.937135+0000","last_unstale":"2024-09-20T18:50:25.554624+0000","last_undegraded":"2024-09-20T18:50:25.554624+0000","last_fullsized":"2024-09-20T18:50:25.554624+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:49:43.921002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:56.442658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095558800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.555620+0000","last_change":"2024-09-20T18:49:49.954880+0000","last_active":"2024-09-20T18:50:25.555620+0000","last_peered":"2024-09-20T18:50:25.555620+0000","last_clean":"2024-09-20T18:50:25.555620+0000","last_became_active":"2024-09-20T18:49:49.954424+0000","last_became_peered":"2024-09-20T18:49:49.954424+0000","last_unstale":"2024-09-20T18:50:25.555620+0000","last_undegraded":"2024-09-20T18:50:25.555620+0000","last_fullsized":"2024-09-20T18:50:25.555620+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:54.964168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629997+0000","last_change":"2024-09-20T18:49:52.140366+0000","last_active":"2024-09-20T18:50:24.629997+0000","last_peered":"2024-09-20T18:50:24.629997+0000","last_clean":"2024-09-20T18:50:24.629997+0000","last_became_active":"2024-09-20T18:49:52.139789+0000","last_became_peered":"2024-09-20T18:49:52.139789+0000","last_unstale":"2024-09-20T18:50:24.629997+0000","last_undegraded":"2024-09-20T18:50:24.629997+0000","last_fullsized":"2024-09-20T18:50:24.629997+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:16.224499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":61,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372491+0000","last_change":"2024-09-20T18:49:47.949547+0000","last_active":"2024-09-20T18:50:23.372491+0000","last_peered":"2024-09-20T18:50:23.372491+0000","last_clean":"2024-09-20T18:50:23.372491+0000","last_became_active":"2024-09-20T18:49:47.949230+0000","last_became_peered":"2024-09-20T18:49:47.949230+0000","last_unstale":"2024-09-20T18:50:23.372491+0000","last_undegraded":"2024-09-20T18:50:23.372491+0000","last_fullsized":"2024-09-20T18:50:23.372491+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:07.802093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.371747+0000","last_change":"2024-09-20T18:49:45.939454+0000","last_active":"2024-09-20T18:50:23.371747+0000","last_peered":"2024-09-20T18:50:23.371747+0000","last_clean":"2024-09-20T18:50:23.371747+0000","last_became_active":"2024-09-20T18:49:45.939197+0000","last_became_peered":"2024-09-20T18:49:45.939197+0000","last_unstale":"2024-09-20T18:50:23.371747+0000","last_undegraded":"2024-09-20T18:50:23.371747+0000","last_fullsized":"2024-09-20T18:50:23.371747+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:12.763563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":93,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629751+0000","last_change":"2024-09-20T18:49:48.498983+0000","last_active":"2024-09-20T18:50:24.629751+0000","last_peered":"2024-09-20T18:50:24.629751+0000","last_clean":"2024-09-20T18:50:24.629751+0000","last_became_active":"2024-09-20T18:49:44.936819+0000","last_became_peered":"2024-09-20T18:49:44.936819+0000","last_unstale":"2024-09-20T18:50:24.629751+0000","last_undegraded":"2024-09-20T18:50:24.629751+0000","last_fullsized":"2024-09-20T18:50:24.629751+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:49:43.921002+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:58.186148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010996199999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":35,"num_read_kb":30,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.555463+0000","last_change":"2024-09-20T18:49:49.961346+0000","last_active":"2024-09-20T18:50:25.555463+0000","last_peered":"2024-09-20T18:50:25.555463+0000","last_clean":"2024-09-20T18:50:25.555463+0000","last_became_active":"2024-09-20T18:49:49.961066+0000","last_became_peered":"2024-09-20T18:49:49.961066+0000","last_unstale":"2024-09-20T18:50:25.555463+0000","last_undegraded":"2024-09-20T18:50:25.555463+0000","last_fullsized":"2024-09-20T18:50:25.555463+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:53.706599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.555486+0000","last_change":"2024-09-20T18:49:51.970609+0000","last_active":"2024-09-20T18:50:25.555486+0000","last_peered":"2024-09-20T18:50:25.555486+0000","last_clean":"2024-09-20T18:50:25.555486+0000","last_became_active":"2024-09-20T18:49:51.970343+0000","last_became_peered":"2024-09-20T18:49:51.970343+0000","last_unstale":"2024-09-20T18:50:25.555486+0000","last_undegraded":"2024-09-20T18:50:25.555486+0000","last_fullsized":"2024-09-20T18:50:25.555486+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:06.660017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":73,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.436831+0000","last_change":"2024-09-20T18:49:48.498345+0000","last_active":"2024-09-20T18:50:24.436831+0000","last_peered":"2024-09-20T18:50:24.436831+0000","last_clean":"2024-09-20T18:50:24.436831+0000","last_became_active":"2024-09-20T18:49:48.497936+0000","last_became_peered":"2024-09-20T18:49:48.497936+0000","last_unstale":"2024-09-20T18:50:24.436831+0000","last_undegraded":"2024-09-20T18:50:24.436831+0000","last_fullsized":"2024-09-20T18:50:24.436831+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:51.535171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629427+0000","last_change":"2024-09-20T18:49:45.941387+0000","last_active":"2024-09-20T18:50:24.629427+0000","last_peered":"2024-09-20T18:50:24.629427+0000","last_clean":"2024-09-20T18:50:24.629427+0000","last_became_active":"2024-09-20T18:49:45.941136+0000","last_became_peered":"2024-09-20T18:49:45.941136+0000","last_unstale":"2024-09-20T18:50:24.629427+0000","last_undegraded":"2024-09-20T18:50:24.629427+0000","last_fullsized":"2024-09-20T18:50:24.629427+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:09.074025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":86,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629043+0000","last_change":"2024-09-20T18:49:36.549748+0000","last_active":"2024-09-20T18:50:24.629043+0000","last_peered":"2024-09-20T18:50:24.629043+0000","last_clean":"2024-09-20T18:50:24.629043+0000","last_became_active":"2024-09-20T18:49:36.545306+0000","last_became_peered":"2024-09-20T18:49:36.545306+0000","last_unstale":"2024-09-20T18:50:24.629043+0000","last_undegraded":"2024-09-20T18:50:24.629043+0000","last_fullsized":"2024-09-20T18:50:24.629043+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"20'72","last_scrub_stamp":"2024-09-20T18:49:27.311157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T18:49:27.311157+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:30.469238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629912+0000","last_change":"2024-09-20T18:49:49.956441+0000","last_active":"2024-09-20T18:50:24.629912+0000","last_peered":"2024-09-20T18:50:24.629912+0000","last_clean":"2024-09-20T18:50:24.629912+0000","last_became_active":"2024-09-20T18:49:49.956128+0000","last_became_peered":"2024-09-20T18:49:49.956128+0000","last_unstale":"2024-09-20T18:50:24.629912+0000","last_undegraded":"2024-09-20T18:50:24.629912+0000","last_fullsized":"2024-09-20T18:50:24.629912+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:04.622346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.555344+0000","last_change":"2024-09-20T18:49:51.971980+0000","last_active":"2024-09-20T18:50:25.555344+0000","last_peered":"2024-09-20T18:50:25.555344+0000","last_clean":"2024-09-20T18:50:25.555344+0000","last_became_active":"2024-09-20T18:49:51.971332+0000","last_became_peered":"2024-09-20T18:49:51.971332+0000","last_unstale":"2024-09-20T18:50:25.555344+0000","last_undegraded":"2024-09-20T18:50:25.555344+0000","last_fullsized":"2024-09-20T18:50:25.555344+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:13.174503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":62,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.373437+0000","last_change":"2024-09-20T18:49:47.950762+0000","last_active":"2024-09-20T18:50:23.373437+0000","last_peered":"2024-09-20T18:50:23.373437+0000","last_clean":"2024-09-20T18:50:23.373437+0000","last_became_active":"2024-09-20T18:49:47.950073+0000","last_became_peered":"2024-09-20T18:49:47.950073+0000","last_unstale":"2024-09-20T18:50:23.373437+0000","last_undegraded":"2024-09-20T18:50:23.373437+0000","last_fullsized":"2024-09-20T18:50:23.373437+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:57.527730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372852+0000","last_change":"2024-09-20T18:49:45.944925+0000","last_active":"2024-09-20T18:50:23.372852+0000","last_peered":"2024-09-20T18:50:23.372852+0000","last_clean":"2024-09-20T18:50:23.372852+0000","last_became_active":"2024-09-20T18:49:45.944579+0000","last_became_peered":"2024-09-20T18:49:45.944579+0000","last_unstale":"2024-09-20T18:50:23.372852+0000","last_undegraded":"2024-09-20T18:50:23.372852+0000","last_fullsized":"2024-09-20T18:50:23.372852+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:03.333030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.555244+0000","last_change":"2024-09-20T18:49:49.954673+0000","last_active":"2024-09-20T18:50:25.555244+0000","last_peered":"2024-09-20T18:50:25.555244+0000","last_clean":"2024-09-20T18:50:25.555244+0000","last_became_active":"2024-09-20T18:49:49.954391+0000","last_became_peered":"2024-09-20T18:49:49.954391+0000","last_unstale":"2024-09-20T18:50:25.555244+0000","last_undegraded":"2024-09-20T18:50:25.555244+0000","last_fullsized":"2024-09-20T18:50:25.555244+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:12.999716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629582+0000","last_change":"2024-09-20T18:49:52.140252+0000","last_active":"2024-09-20T18:50:24.629582+0000","last_peered":"2024-09-20T18:50:24.629582+0000","last_clean":"2024-09-20T18:50:24.629582+0000","last_became_active":"2024-09-20T18:49:52.139959+0000","last_became_peered":"2024-09-20T18:49:52.139959+0000","last_unstale":"2024-09-20T18:50:24.629582+0000","last_undegraded":"2024-09-20T18:50:24.629582+0000","last_fullsized":"2024-09-20T18:50:24.629582+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:54.070093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":71,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.373360+0000","last_change":"2024-09-20T18:49:47.947920+0000","last_active":"2024-09-20T18:50:23.373360+0000","last_peered":"2024-09-20T18:50:23.373360+0000","last_clean":"2024-09-20T18:50:23.373360+0000","last_became_active":"2024-09-20T18:49:47.947636+0000","last_became_peered":"2024-09-20T18:49:47.947636+0000","last_unstale":"2024-09-20T18:50:23.373360+0000","last_undegraded":"2024-09-20T18:50:23.373360+0000","last_fullsized":"2024-09-20T18:50:23.373360+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:47.335335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.436881+0000","last_change":"2024-09-20T18:49:45.946517+0000","last_active":"2024-09-20T18:50:24.436881+0000","last_peered":"2024-09-20T18:50:24.436881+0000","last_clean":"2024-09-20T18:50:24.436881+0000","last_became_active":"2024-09-20T18:49:45.946153+0000","last_became_peered":"2024-09-20T18:49:45.946153+0000","last_unstale":"2024-09-20T18:50:24.436881+0000","last_undegraded":"2024-09-20T18:50:24.436881+0000","last_fullsized":"2024-09-20T18:50:24.436881+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:05.889152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.555493+0000","last_change":"2024-09-20T18:49:49.959440+0000","last_active":"2024-09-20T18:50:25.555493+0000","last_peered":"2024-09-20T18:50:25.555493+0000","last_clean":"2024-09-20T18:50:25.555493+0000","last_became_active":"2024-09-20T18:49:49.959160+0000","last_became_peered":"2024-09-20T18:49:49.959160+0000","last_unstale":"2024-09-20T18:50:25.555493+0000","last_undegraded":"2024-09-20T18:50:25.555493+0000","last_fullsized":"2024-09-20T18:50:25.555493+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:08.849486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.555908+0000","last_change":"2024-09-20T18:49:51.970386+0000","last_active":"2024-09-20T18:50:25.555908+0000","last_peered":"2024-09-20T18:50:25.555908+0000","last_clean":"2024-09-20T18:50:25.555908+0000","last_became_active":"2024-09-20T18:49:51.970108+0000","last_became_peered":"2024-09-20T18:49:51.970108+0000","last_unstale":"2024-09-20T18:50:25.555908+0000","last_undegraded":"2024-09-20T18:50:25.555908+0000","last_fullsized":"2024-09-20T18:50:25.555908+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:04.919785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":58,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.373003+0000","last_change":"2024-09-20T18:49:47.948046+0000","last_active":"2024-09-20T18:50:23.373003+0000","last_peered":"2024-09-20T18:50:23.373003+0000","last_clean":"2024-09-20T18:50:23.373003+0000","last_became_active":"2024-09-20T18:49:47.947126+0000","last_became_peered":"2024-09-20T18:49:47.947126+0000","last_unstale":"2024-09-20T18:50:23.373003+0000","last_undegraded":"2024-09-20T18:50:23.373003+0000","last_fullsized":"2024-09-20T18:50:23.373003+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:16.466592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629607+0000","last_change":"2024-09-20T18:49:45.937511+0000","last_active":"2024-09-20T18:50:24.629607+0000","last_peered":"2024-09-20T18:50:24.629607+0000","last_clean":"2024-09-20T18:50:24.629607+0000","last_became_active":"2024-09-20T18:49:45.937264+0000","last_became_peered":"2024-09-20T18:49:45.937264+0000","last_unstale":"2024-09-20T18:50:24.629607+0000","last_undegraded":"2024-09-20T18:50:24.629607+0000","last_fullsized":"2024-09-20T18:50:24.629607+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:06.234318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":58,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.555249+0000","last_change":"2024-09-20T18:49:49.957286+0000","last_active":"2024-09-20T18:50:25.555249+0000","last_peered":"2024-09-20T18:50:25.555249+0000","last_clean":"2024-09-20T18:50:25.555249+0000","last_became_active":"2024-09-20T18:49:49.957005+0000","last_became_peered":"2024-09-20T18:49:49.957005+0000","last_unstale":"2024-09-20T18:50:25.555249+0000","last_undegraded":"2024-09-20T18:50:25.555249+0000","last_fullsized":"2024-09-20T18:50:25.555249+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:11.425942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.373273+0000","last_change":"2024-09-20T18:49:51.969633+0000","last_active":"2024-09-20T18:50:23.373273+0000","last_peered":"2024-09-20T18:50:23.373273+0000","last_clean":"2024-09-20T18:50:23.373273+0000","last_became_active":"2024-09-20T18:49:51.969355+0000","last_became_peered":"2024-09-20T18:49:51.969355+0000","last_unstale":"2024-09-20T18:50:23.373273+0000","last_undegraded":"2024-09-20T18:50:23.373273+0000","last_fullsized":"2024-09-20T18:50:23.373273+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:53.655593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":59,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372126+0000","last_change":"2024-09-20T18:49:47.946309+0000","last_active":"2024-09-20T18:50:23.372126+0000","last_peered":"2024-09-20T18:50:23.372126+0000","last_clean":"2024-09-20T18:50:23.372126+0000","last_became_active":"2024-09-20T18:49:47.946008+0000","last_became_peered":"2024-09-20T18:49:47.946008+0000","last_unstale":"2024-09-20T18:50:23.372126+0000","last_undegraded":"2024-09-20T18:50:23.372126+0000","last_fullsized":"2024-09-20T18:50:23.372126+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:55.957874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":49,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.555134+0000","last_change":"2024-09-20T18:49:45.941766+0000","last_active":"2024-09-20T18:50:25.555134+0000","last_peered":"2024-09-20T18:50:25.555134+0000","last_clean":"2024-09-20T18:50:25.555134+0000","last_became_active":"2024-09-20T18:49:45.941406+0000","last_became_peered":"2024-09-20T18:49:45.941406+0000","last_unstale":"2024-09-20T18:50:25.555134+0000","last_undegraded":"2024-09-20T18:50:25.555134+0000","last_fullsized":"2024-09-20T18:50:25.555134+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:46.650041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.554800+0000","last_change":"2024-09-20T18:49:49.957141+0000","last_active":"2024-09-20T18:50:25.554800+0000","last_peered":"2024-09-20T18:50:25.554800+0000","last_clean":"2024-09-20T18:50:25.554800+0000","last_became_active":"2024-09-20T18:49:49.956767+0000","last_became_peered":"2024-09-20T18:49:49.956767+0000","last_unstale":"2024-09-20T18:50:25.554800+0000","last_undegraded":"2024-09-20T18:50:25.554800+0000","last_fullsized":"2024-09-20T18:50:25.554800+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:55.731396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.371640+0000","last_change":"2024-09-20T18:49:51.964375+0000","last_active":"2024-09-20T18:50:23.371640+0000","last_peered":"2024-09-20T18:50:23.371640+0000","last_clean":"2024-09-20T18:50:23.371640+0000","last_became_active":"2024-09-20T18:49:51.964098+0000","last_became_peered":"2024-09-20T18:49:51.964098+0000","last_unstale":"2024-09-20T18:50:23.371640+0000","last_undegraded":"2024-09-20T18:50:23.371640+0000","last_fullsized":"2024-09-20T18:50:23.371640+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.566377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":74,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.436613+0000","last_change":"2024-09-20T18:49:48.498846+0000","last_active":"2024-09-20T18:50:24.436613+0000","last_peered":"2024-09-20T18:50:24.436613+0000","last_clean":"2024-09-20T18:50:24.436613+0000","last_became_active":"2024-09-20T18:49:48.498625+0000","last_became_peered":"2024-09-20T18:49:48.498625+0000","last_unstale":"2024-09-20T18:50:24.436613+0000","last_undegraded":"2024-09-20T18:50:24.436613+0000","last_fullsized":"2024-09-20T18:50:24.436613+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:03.186719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.630180+0000","last_change":"2024-09-20T18:49:45.941764+0000","last_active":"2024-09-20T18:50:24.630180+0000","last_peered":"2024-09-20T18:50:24.630180+0000","last_clean":"2024-09-20T18:50:24.630180+0000","last_became_active":"2024-09-20T18:49:45.941448+0000","last_became_peered":"2024-09-20T18:49:45.941448+0000","last_unstale":"2024-09-20T18:50:24.630180+0000","last_undegraded":"2024-09-20T18:50:24.630180+0000","last_fullsized":"2024-09-20T18:50:24.630180+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:45.120244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.555111+0000","last_change":"2024-09-20T18:49:49.957845+0000","last_active":"2024-09-20T18:50:25.555111+0000","last_peered":"2024-09-20T18:50:25.555111+0000","last_clean":"2024-09-20T18:50:25.555111+0000","last_became_active":"2024-09-20T18:49:49.957558+0000","last_became_peered":"2024-09-20T18:49:49.957558+0000","last_unstale":"2024-09-20T18:50:25.555111+0000","last_undegraded":"2024-09-20T18:50:25.555111+0000","last_fullsized":"2024-09-20T18:50:25.555111+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:14.726319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629280+0000","last_change":"2024-09-20T18:49:51.971132+0000","last_active":"2024-09-20T18:50:24.629280+0000","last_peered":"2024-09-20T18:50:24.629280+0000","last_clean":"2024-09-20T18:50:24.629280+0000","last_became_active":"2024-09-20T18:49:51.970798+0000","last_became_peered":"2024-09-20T18:49:51.970798+0000","last_unstale":"2024-09-20T18:50:24.629280+0000","last_undegraded":"2024-09-20T18:50:24.629280+0000","last_fullsized":"2024-09-20T18:50:24.629280+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:09.540547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":66,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372699+0000","last_change":"2024-09-20T18:49:47.949428+0000","last_active":"2024-09-20T18:50:23.372699+0000","last_peered":"2024-09-20T18:50:23.372699+0000","last_clean":"2024-09-20T18:50:23.372699+0000","last_became_active":"2024-09-20T18:49:47.949116+0000","last_became_peered":"2024-09-20T18:49:47.949116+0000","last_unstale":"2024-09-20T18:50:23.372699+0000","last_undegraded":"2024-09-20T18:50:23.372699+0000","last_fullsized":"2024-09-20T18:50:23.372699+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:11.352253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.630326+0000","last_change":"2024-09-20T18:49:45.945147+0000","last_active":"2024-09-20T18:50:24.630326+0000","last_peered":"2024-09-20T18:50:24.630326+0000","last_clean":"2024-09-20T18:50:24.630326+0000","last_became_active":"2024-09-20T18:49:45.944858+0000","last_became_peered":"2024-09-20T18:49:45.944858+0000","last_unstale":"2024-09-20T18:50:24.630326+0000","last_undegraded":"2024-09-20T18:50:24.630326+0000","last_fullsized":"2024-09-20T18:50:24.630326+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:53.147843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.554463+0000","last_change":"2024-09-20T18:49:49.955243+0000","last_active":"2024-09-20T18:50:25.554463+0000","last_peered":"2024-09-20T18:50:25.554463+0000","last_clean":"2024-09-20T18:50:25.554463+0000","last_became_active":"2024-09-20T18:49:49.954944+0000","last_became_peered":"2024-09-20T18:49:49.954944+0000","last_unstale":"2024-09-20T18:50:25.554463+0000","last_undegraded":"2024-09-20T18:50:25.554463+0000","last_fullsized":"2024-09-20T18:50:25.554463+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:00.542377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.628764+0000","last_change":"2024-09-20T18:49:51.973284+0000","last_active":"2024-09-20T18:50:24.628764+0000","last_peered":"2024-09-20T18:50:24.628764+0000","last_clean":"2024-09-20T18:50:24.628764+0000","last_became_active":"2024-09-20T18:49:51.972970+0000","last_became_peered":"2024-09-20T18:49:51.972970+0000","last_unstale":"2024-09-20T18:50:24.628764+0000","last_undegraded":"2024-09-20T18:50:24.628764+0000","last_fullsized":"2024-09-20T18:50:24.628764+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:55.977221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":70,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.554310+0000","last_change":"2024-09-20T18:49:48.500174+0000","last_active":"2024-09-20T18:50:25.554310+0000","last_peered":"2024-09-20T18:50:25.554310+0000","last_clean":"2024-09-20T18:50:25.554310+0000","last_became_active":"2024-09-20T18:49:48.499847+0000","last_became_peered":"2024-09-20T18:49:48.499847+0000","last_unstale":"2024-09-20T18:50:25.554310+0000","last_undegraded":"2024-09-20T18:50:25.554310+0000","last_fullsized":"2024-09-20T18:50:25.554310+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:15.572682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":59,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.630364+0000","last_change":"2024-09-20T18:49:45.945391+0000","last_active":"2024-09-20T18:50:24.630364+0000","last_peered":"2024-09-20T18:50:24.630364+0000","last_clean":"2024-09-20T18:50:24.630364+0000","last_became_active":"2024-09-20T18:49:45.944256+0000","last_became_peered":"2024-09-20T18:49:45.944256+0000","last_unstale":"2024-09-20T18:50:24.630364+0000","last_undegraded":"2024-09-20T18:50:24.630364+0000","last_fullsized":"2024-09-20T18:50:24.630364+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:47.823570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":55,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629594+0000","last_change":"2024-09-20T18:49:49.958686+0000","last_active":"2024-09-20T18:50:24.629594+0000","last_peered":"2024-09-20T18:50:24.629594+0000","last_clean":"2024-09-20T18:50:24.629594+0000","last_became_active":"2024-09-20T18:49:49.958411+0000","last_became_peered":"2024-09-20T18:49:49.958411+0000","last_unstale":"2024-09-20T18:50:24.629594+0000","last_undegraded":"2024-09-20T18:50:24.629594+0000","last_fullsized":"2024-09-20T18:50:24.629594+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:57.283019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.554442+0000","last_change":"2024-09-20T18:49:52.139994+0000","last_active":"2024-09-20T18:50:25.554442+0000","last_peered":"2024-09-20T18:50:25.554442+0000","last_clean":"2024-09-20T18:50:25.554442+0000","last_became_active":"2024-09-20T18:49:52.139685+0000","last_became_peered":"2024-09-20T18:49:52.139685+0000","last_unstale":"2024-09-20T18:50:25.554442+0000","last_undegraded":"2024-09-20T18:50:25.554442+0000","last_fullsized":"2024-09-20T18:50:25.554442+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:53.034093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.436316+0000","last_change":"2024-09-20T18:49:45.940360+0000","last_active":"2024-09-20T18:50:24.436316+0000","last_peered":"2024-09-20T18:50:24.436316+0000","last_clean":"2024-09-20T18:50:24.436316+0000","last_became_active":"2024-09-20T18:49:45.940060+0000","last_became_peered":"2024-09-20T18:49:45.940060+0000","last_unstale":"2024-09-20T18:50:24.436316+0000","last_undegraded":"2024-09-20T18:50:24.436316+0000","last_fullsized":"2024-09-20T18:50:24.436316+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:59.071733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":52,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.630441+0000","last_change":"2024-09-20T18:49:48.501010+0000","last_active":"2024-09-20T18:50:24.630441+0000","last_peered":"2024-09-20T18:50:24.630441+0000","last_clean":"2024-09-20T18:50:24.630441+0000","last_became_active":"2024-09-20T18:49:48.500662+0000","last_became_peered":"2024-09-20T18:49:48.500662+0000","last_unstale":"2024-09-20T18:50:24.630441+0000","last_undegraded":"2024-09-20T18:50:24.630441+0000","last_fullsized":"2024-09-20T18:50:24.630441+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:01.787506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.555213+0000","last_change":"2024-09-20T18:49:49.957092+0000","last_active":"2024-09-20T18:50:25.555213+0000","last_peered":"2024-09-20T18:50:25.555213+0000","last_clean":"2024-09-20T18:50:25.555213+0000","last_became_active":"2024-09-20T18:49:49.956749+0000","last_became_peered":"2024-09-20T18:49:49.956749+0000","last_unstale":"2024-09-20T18:50:25.555213+0000","last_undegraded":"2024-09-20T18:50:25.555213+0000","last_fullsized":"2024-09-20T18:50:25.555213+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:53.840760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629239+0000","last_change":"2024-09-20T18:49:52.142789+0000","last_active":"2024-09-20T18:50:24.629239+0000","last_peered":"2024-09-20T18:50:24.629239+0000","last_clean":"2024-09-20T18:50:24.629239+0000","last_became_active":"2024-09-20T18:49:52.142291+0000","last_became_peered":"2024-09-20T18:49:52.142291+0000","last_unstale":"2024-09-20T18:50:24.629239+0000","last_undegraded":"2024-09-20T18:50:24.629239+0000","last_fullsized":"2024-09-20T18:50:24.629239+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:58.299262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":59,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372537+0000","last_change":"2024-09-20T18:49:48.501376+0000","last_active":"2024-09-20T18:50:23.372537+0000","last_peered":"2024-09-20T18:50:23.372537+0000","last_clean":"2024-09-20T18:50:23.372537+0000","last_became_active":"2024-09-20T18:49:48.500803+0000","last_became_peered":"2024-09-20T18:49:48.500803+0000","last_unstale":"2024-09-20T18:50:23.372537+0000","last_undegraded":"2024-09-20T18:50:23.372537+0000","last_fullsized":"2024-09-20T18:50:23.372537+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:02.881066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629764+0000","last_change":"2024-09-20T18:49:45.947791+0000","last_active":"2024-09-20T18:50:24.629764+0000","last_peered":"2024-09-20T18:50:24.629764+0000","last_clean":"2024-09-20T18:50:24.629764+0000","last_became_active":"2024-09-20T18:49:45.946600+0000","last_became_peered":"2024-09-20T18:49:45.946600+0000","last_unstale":"2024-09-20T18:50:24.629764+0000","last_undegraded":"2024-09-20T18:50:24.629764+0000","last_fullsized":"2024-09-20T18:50:24.629764+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:13.160428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.630358+0000","last_change":"2024-09-20T18:49:49.962290+0000","last_active":"2024-09-20T18:50:24.630358+0000","last_peered":"2024-09-20T18:50:24.630358+0000","last_clean":"2024-09-20T18:50:24.630358+0000","last_became_active":"2024-09-20T18:49:49.961947+0000","last_became_peered":"2024-09-20T18:49:49.961947+0000","last_unstale":"2024-09-20T18:50:24.630358+0000","last_undegraded":"2024-09-20T18:50:24.630358+0000","last_fullsized":"2024-09-20T18:50:24.630358+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:54.834258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"66'1","reported_seq":39,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.556081+0000","last_change":"2024-09-20T18:49:51.965397+0000","last_active":"2024-09-20T18:50:25.556081+0000","last_peered":"2024-09-20T18:50:25.556081+0000","last_clean":"2024-09-20T18:50:25.556081+0000","last_became_active":"2024-09-20T18:49:51.965121+0000","last_became_peered":"2024-09-20T18:49:51.965121+0000","last_unstale":"2024-09-20T18:50:25.556081+0000","last_undegraded":"2024-09-20T18:50:25.556081+0000","last_fullsized":"2024-09-20T18:50:25.556081+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:57.419566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.555112+0000","last_change":"2024-09-20T18:49:48.501359+0000","last_active":"2024-09-20T18:50:25.555112+0000","last_peered":"2024-09-20T18:50:25.555112+0000","last_clean":"2024-09-20T18:50:25.555112+0000","last_became_active":"2024-09-20T18:49:48.501011+0000","last_became_peered":"2024-09-20T18:49:48.501011+0000","last_unstale":"2024-09-20T18:50:25.555112+0000","last_undegraded":"2024-09-20T18:50:25.555112+0000","last_fullsized":"2024-09-20T18:50:25.555112+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:50.431705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.371926+0000","last_change":"2024-09-20T18:49:45.938820+0000","last_active":"2024-09-20T18:50:23.371926+0000","last_peered":"2024-09-20T18:50:23.371926+0000","last_clean":"2024-09-20T18:50:23.371926+0000","last_became_active":"2024-09-20T18:49:45.938511+0000","last_became_peered":"2024-09-20T18:49:45.938511+0000","last_unstale":"2024-09-20T18:50:23.371926+0000","last_undegraded":"2024-09-20T18:50:23.371926+0000","last_fullsized":"2024-09-20T18:50:23.371926+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:02.383015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":55,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629715+0000","last_change":"2024-09-20T18:49:49.960889+0000","last_active":"2024-09-20T18:50:24.629715+0000","last_peered":"2024-09-20T18:50:24.629715+0000","last_clean":"2024-09-20T18:50:24.629715+0000","last_became_active":"2024-09-20T18:49:49.960550+0000","last_became_peered":"2024-09-20T18:49:49.960550+0000","last_unstale":"2024-09-20T18:50:24.629715+0000","last_undegraded":"2024-09-20T18:50:24.629715+0000","last_fullsized":"2024-09-20T18:50:24.629715+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:58.739350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.373458+0000","last_change":"2024-09-20T18:49:51.966530+0000","last_active":"2024-09-20T18:50:23.373458+0000","last_peered":"2024-09-20T18:50:23.373458+0000","last_clean":"2024-09-20T18:50:23.373458+0000","last_became_active":"2024-09-20T18:49:51.966254+0000","last_became_peered":"2024-09-20T18:49:51.966254+0000","last_unstale":"2024-09-20T18:50:23.373458+0000","last_undegraded":"2024-09-20T18:50:23.373458+0000","last_fullsized":"2024-09-20T18:50:23.373458+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:14.193792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":77,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629923+0000","last_change":"2024-09-20T18:49:59.003234+0000","last_active":"2024-09-20T18:50:24.629923+0000","last_peered":"2024-09-20T18:50:24.629923+0000","last_clean":"2024-09-20T18:50:24.629923+0000","last_became_active":"2024-09-20T18:49:58.997494+0000","last_became_peered":"2024-09-20T18:49:58.997494+0000","last_unstale":"2024-09-20T18:50:24.629923+0000","last_undegraded":"2024-09-20T18:50:24.629923+0000","last_fullsized":"2024-09-20T18:50:24.629923+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:05.235529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629902+0000","last_change":"2024-09-20T18:49:45.949895+0000","last_active":"2024-09-20T18:50:24.629902+0000","last_peered":"2024-09-20T18:50:24.629902+0000","last_clean":"2024-09-20T18:50:24.629902+0000","last_became_active":"2024-09-20T18:49:45.949492+0000","last_became_peered":"2024-09-20T18:49:45.949492+0000","last_unstale":"2024-09-20T18:50:24.629902+0000","last_undegraded":"2024-09-20T18:50:24.629902+0000","last_fullsized":"2024-09-20T18:50:24.629902+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:02.958108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":57,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.554954+0000","last_change":"2024-09-20T18:49:49.962519+0000","last_active":"2024-09-20T18:50:25.554954+0000","last_peered":"2024-09-20T18:50:25.554954+0000","last_clean":"2024-09-20T18:50:25.554954+0000","last_became_active":"2024-09-20T18:49:49.962243+0000","last_became_peered":"2024-09-20T18:49:49.962243+0000","last_unstale":"2024-09-20T18:50:25.554954+0000","last_undegraded":"2024-09-20T18:50:25.554954+0000","last_fullsized":"2024-09-20T18:50:25.554954+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:16.990588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372337+0000","last_change":"2024-09-20T18:49:51.969358+0000","last_active":"2024-09-20T18:50:23.372337+0000","last_peered":"2024-09-20T18:50:23.372337+0000","last_clean":"2024-09-20T18:50:23.372337+0000","last_became_active":"2024-09-20T18:49:51.969093+0000","last_became_peered":"2024-09-20T18:49:51.969093+0000","last_unstale":"2024-09-20T18:50:23.372337+0000","last_undegraded":"2024-09-20T18:50:23.372337+0000","last_fullsized":"2024-09-20T18:50:23.372337+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:17.459087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":62,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372736+0000","last_change":"2024-09-20T18:49:47.950488+0000","last_active":"2024-09-20T18:50:23.372736+0000","last_peered":"2024-09-20T18:50:23.372736+0000","last_clean":"2024-09-20T18:50:23.372736+0000","last_became_active":"2024-09-20T18:49:47.949703+0000","last_became_peered":"2024-09-20T18:49:47.949703+0000","last_unstale":"2024-09-20T18:50:23.372736+0000","last_undegraded":"2024-09-20T18:50:23.372736+0000","last_fullsized":"2024-09-20T18:50:23.372736+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:53.129485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.373124+0000","last_change":"2024-09-20T18:49:45.941578+0000","last_active":"2024-09-20T18:50:23.373124+0000","last_peered":"2024-09-20T18:50:23.373124+0000","last_clean":"2024-09-20T18:50:23.373124+0000","last_became_active":"2024-09-20T18:49:45.939949+0000","last_became_peered":"2024-09-20T18:49:45.939949+0000","last_unstale":"2024-09-20T18:50:23.373124+0000","last_undegraded":"2024-09-20T18:50:23.373124+0000","last_fullsized":"2024-09-20T18:50:23.373124+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:53.450364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.555272+0000","last_change":"2024-09-20T18:49:49.957358+0000","last_active":"2024-09-20T18:50:25.555272+0000","last_peered":"2024-09-20T18:50:25.555272+0000","last_clean":"2024-09-20T18:50:25.555272+0000","last_became_active":"2024-09-20T18:49:49.957067+0000","last_became_peered":"2024-09-20T18:49:49.957067+0000","last_unstale":"2024-09-20T18:50:25.555272+0000","last_undegraded":"2024-09-20T18:50:25.555272+0000","last_fullsized":"2024-09-20T18:50:25.555272+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:54.100322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.628698+0000","last_change":"2024-09-20T18:49:51.967168+0000","last_active":"2024-09-20T18:50:24.628698+0000","last_peered":"2024-09-20T18:50:24.628698+0000","last_clean":"2024-09-20T18:50:24.628698+0000","last_became_active":"2024-09-20T18:49:51.966915+0000","last_became_peered":"2024-09-20T18:49:51.966915+0000","last_unstale":"2024-09-20T18:50:24.628698+0000","last_undegraded":"2024-09-20T18:50:24.628698+0000","last_fullsized":"2024-09-20T18:50:24.628698+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:04.536536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.555361+0000","last_change":"2024-09-20T18:49:47.953603+0000","last_active":"2024-09-20T18:50:25.555361+0000","last_peered":"2024-09-20T18:50:25.555361+0000","last_clean":"2024-09-20T18:50:25.555361+0000","last_became_active":"2024-09-20T18:49:47.953310+0000","last_became_peered":"2024-09-20T18:49:47.953310+0000","last_unstale":"2024-09-20T18:50:25.555361+0000","last_undegraded":"2024-09-20T18:50:25.555361+0000","last_fullsized":"2024-09-20T18:50:25.555361+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:06.158344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.630136+0000","last_change":"2024-09-20T18:49:45.944827+0000","last_active":"2024-09-20T18:50:24.630136+0000","last_peered":"2024-09-20T18:50:24.630136+0000","last_clean":"2024-09-20T18:50:24.630136+0000","last_became_active":"2024-09-20T18:49:45.944357+0000","last_became_peered":"2024-09-20T18:49:45.944357+0000","last_unstale":"2024-09-20T18:50:24.630136+0000","last_undegraded":"2024-09-20T18:50:24.630136+0000","last_fullsized":"2024-09-20T18:50:24.630136+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:14.013129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629464+0000","last_change":"2024-09-20T18:49:49.962794+0000","last_active":"2024-09-20T18:50:24.629464+0000","last_peered":"2024-09-20T18:50:24.629464+0000","last_clean":"2024-09-20T18:50:24.629464+0000","last_became_active":"2024-09-20T18:49:49.962523+0000","last_became_peered":"2024-09-20T18:49:49.962523+0000","last_unstale":"2024-09-20T18:50:24.629464+0000","last_undegraded":"2024-09-20T18:50:24.629464+0000","last_fullsized":"2024-09-20T18:50:24.629464+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:06.567112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.371568+0000","last_change":"2024-09-20T18:49:51.967581+0000","last_active":"2024-09-20T18:50:23.371568+0000","last_peered":"2024-09-20T18:50:23.371568+0000","last_clean":"2024-09-20T18:50:23.371568+0000","last_became_active":"2024-09-20T18:49:51.967313+0000","last_became_peered":"2024-09-20T18:49:51.967313+0000","last_unstale":"2024-09-20T18:50:23.371568+0000","last_undegraded":"2024-09-20T18:50:23.371568+0000","last_fullsized":"2024-09-20T18:50:23.371568+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:05.774315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":62,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629291+0000","last_change":"2024-09-20T18:49:48.499421+0000","last_active":"2024-09-20T18:50:24.629291+0000","last_peered":"2024-09-20T18:50:24.629291+0000","last_clean":"2024-09-20T18:50:24.629291+0000","last_became_active":"2024-09-20T18:49:48.499111+0000","last_became_peered":"2024-09-20T18:49:48.499111+0000","last_unstale":"2024-09-20T18:50:24.629291+0000","last_undegraded":"2024-09-20T18:50:24.629291+0000","last_fullsized":"2024-09-20T18:50:24.629291+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:57.119627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":49,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.555318+0000","last_change":"2024-09-20T18:49:45.942406+0000","last_active":"2024-09-20T18:50:25.555318+0000","last_peered":"2024-09-20T18:50:25.555318+0000","last_clean":"2024-09-20T18:50:25.555318+0000","last_became_active":"2024-09-20T18:49:45.942105+0000","last_became_peered":"2024-09-20T18:49:45.942105+0000","last_unstale":"2024-09-20T18:50:25.555318+0000","last_undegraded":"2024-09-20T18:50:25.555318+0000","last_fullsized":"2024-09-20T18:50:25.555318+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:56.309962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629418+0000","last_change":"2024-09-20T18:49:49.959361+0000","last_active":"2024-09-20T18:50:24.629418+0000","last_peered":"2024-09-20T18:50:24.629418+0000","last_clean":"2024-09-20T18:50:24.629418+0000","last_became_active":"2024-09-20T18:49:49.958159+0000","last_became_peered":"2024-09-20T18:49:49.958159+0000","last_unstale":"2024-09-20T18:50:24.629418+0000","last_undegraded":"2024-09-20T18:50:24.629418+0000","last_fullsized":"2024-09-20T18:50:24.629418+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:12.385752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.628729+0000","last_change":"2024-09-20T18:49:52.140525+0000","last_active":"2024-09-20T18:50:24.628729+0000","last_peered":"2024-09-20T18:50:24.628729+0000","last_clean":"2024-09-20T18:50:24.628729+0000","last_became_active":"2024-09-20T18:49:52.139978+0000","last_became_peered":"2024-09-20T18:49:52.139978+0000","last_unstale":"2024-09-20T18:50:24.628729+0000","last_undegraded":"2024-09-20T18:50:24.628729+0000","last_fullsized":"2024-09-20T18:50:24.628729+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:19.358011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":49,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.628968+0000","last_change":"2024-09-20T18:49:47.955120+0000","last_active":"2024-09-20T18:50:24.628968+0000","last_peered":"2024-09-20T18:50:24.628968+0000","last_clean":"2024-09-20T18:50:24.628968+0000","last_became_active":"2024-09-20T18:49:47.954777+0000","last_became_peered":"2024-09-20T18:49:47.954777+0000","last_unstale":"2024-09-20T18:50:24.628968+0000","last_undegraded":"2024-09-20T18:50:24.628968+0000","last_fullsized":"2024-09-20T18:50:24.628968+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:04.481222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372637+0000","last_change":"2024-09-20T18:49:45.941248+0000","last_active":"2024-09-20T18:50:23.372637+0000","last_peered":"2024-09-20T18:50:23.372637+0000","last_clean":"2024-09-20T18:50:23.372637+0000","last_became_active":"2024-09-20T18:49:45.940988+0000","last_became_peered":"2024-09-20T18:49:45.940988+0000","last_unstale":"2024-09-20T18:50:23.372637+0000","last_undegraded":"2024-09-20T18:50:23.372637+0000","last_fullsized":"2024-09-20T18:50:23.372637+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:58.598619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.436254+0000","last_change":"2024-09-20T18:49:49.954779+0000","last_active":"2024-09-20T18:50:24.436254+0000","last_peered":"2024-09-20T18:50:24.436254+0000","last_clean":"2024-09-20T18:50:24.436254+0000","last_became_active":"2024-09-20T18:49:49.954502+0000","last_became_peered":"2024-09-20T18:49:49.954502+0000","last_unstale":"2024-09-20T18:50:24.436254+0000","last_undegraded":"2024-09-20T18:50:24.436254+0000","last_fullsized":"2024-09-20T18:50:24.436254+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:15.141742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629065+0000","last_change":"2024-09-20T18:49:51.968968+0000","last_active":"2024-09-20T18:50:24.629065+0000","last_peered":"2024-09-20T18:50:24.629065+0000","last_clean":"2024-09-20T18:50:24.629065+0000","last_became_active":"2024-09-20T18:49:51.968662+0000","last_became_peered":"2024-09-20T18:49:51.968662+0000","last_unstale":"2024-09-20T18:50:24.629065+0000","last_undegraded":"2024-09-20T18:50:24.629065+0000","last_fullsized":"2024-09-20T18:50:24.629065+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:02.092669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-20T18:50:25.554921+0000","last_change":"2024-09-20T18:49:51.968476+0000","last_active":"2024-09-20T18:50:25.554921+0000","last_peered":"2024-09-20T18:50:25.554921+0000","last_clean":"2024-09-20T18:50:25.554921+0000","last_became_active":"2024-09-20T18:49:51.968216+0000","last_became_peered":"2024-09-20T18:49:51.968216+0000","last_unstale":"2024-09-20T18:50:25.554921+0000","last_undegraded":"2024-09-20T18:50:25.554921+0000","last_fullsized":"2024-09-20T18:50:25.554921+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:11.930622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.629136+0000","last_change":"2024-09-20T18:49:45.938413+0000","last_active":"2024-09-20T18:50:24.629136+0000","last_peered":"2024-09-20T18:50:24.629136+0000","last_clean":"2024-09-20T18:50:24.629136+0000","last_became_active":"2024-09-20T18:49:45.937992+0000","last_became_peered":"2024-09-20T18:49:45.937992+0000","last_unstale":"2024-09-20T18:50:24.629136+0000","last_undegraded":"2024-09-20T18:50:24.629136+0000","last_fullsized":"2024-09-20T18:50:24.629136+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:12.596336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":62,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:24.436878+0000","last_change":"2024-09-20T18:49:47.950306+0000","last_active":"2024-09-20T18:50:24.436878+0000","last_peered":"2024-09-20T18:50:24.436878+0000","last_clean":"2024-09-20T18:50:24.436878+0000","last_became_active":"2024-09-20T18:49:47.949582+0000","last_became_peered":"2024-09-20T18:49:47.949582+0000","last_unstale":"2024-09-20T18:50:24.436878+0000","last_undegraded":"2024-09-20T18:50:24.436878+0000","last_fullsized":"2024-09-20T18:50:24.436878+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:03.753158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T18:50:23.372414+0000","last_change":"2024-09-20T18:49:49.957874+0000","last_active":"2024-09-20T18:50:23.372414+0000","last_peered":"2024-09-20T18:50:23.372414+0000","last_clean":"2024-09-20T18:50:23.372414+0000","last_became_active":"2024-09-20T18:49:49.957582+0000","last_became_peered":"2024-09-20T18:49:49.957582+0000","last_unstale":"2024-09-20T18:50:23.372414+0000","last_undegraded":"2024-09-20T18:50:23.372414+0000","last_fullsized":"2024-09-20T18:50:23.372414+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:07.454286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":35,"num_read_kb":30,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266700,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32540,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712612,"statfs":{"total":95995035648,"available":95961714688,"internally_reserved":0,"allocated":1384448,"data_stored":908550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528339,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28360,"kb_used_data":1332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716792,"statfs":{"total":95995035648,"available":95965995008,"internally_reserved":0,"allocated":1363968,"data_stored":907888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757272,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31944,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713208,"statfs":{"total":95995035648,"available":95962324992,"internally_reserved":0,"allocated":774144,"data_stored":317879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986205,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":806912,"data_stored":319131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182436,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31972,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713180,"statfs":{"total":95995035648,"available":95962296320,"internally_reserved":0,"allocated":802816,"data_stored":317975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411369,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31960,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713192,"statfs":{"total":95995035648,"available":95962308608,"internally_reserved":0,"allocated":790528,"data_stored":324917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574895,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":827392,"data_stored":319642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738421,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32540,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712612,"statfs":{"total":95995035648,"available":95961714688,"internally_reserved":0,"allocated":1384448,"data_stored":908246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T18:50:30.357 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-20T18:50:30.357 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-20T18:50:30.357 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-20T18:50:30.357 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph health --format=json 2024-09-20T18:50:30.737 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:30 smithi083 ceph-mon[29421]: pgmap v14: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-20T18:50:30.737 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:30 smithi083 ceph-mon[29421]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:50:30.737 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:30 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:50:30.737 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:30 smithi083 ceph-mon[36260]: pgmap v14: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-20T18:50:30.737 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:30 smithi083 ceph-mon[36260]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:50:30.737 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:30 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:50:30.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:30 smithi142 ceph-mon[32508]: pgmap v14: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-20T18:50:30.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:30 smithi142 ceph-mon[32508]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:50:30.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:30 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:50:30.811 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:50:31.005 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:50:30 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:50:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:50:31.676 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:50:31.676 INFO:teuthology.orchestra.run.smithi083.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-20T18:50:31.676 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:31 smithi083 ceph-mon[29421]: osdmap e80: 8 total, 8 up, 8 in 2024-09-20T18:50:31.676 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:31 smithi083 ceph-mon[36260]: osdmap e80: 8 total, 8 up, 8 in 2024-09-20T18:50:32.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:31 smithi142 ceph-mon[32508]: osdmap e80: 8 total, 8 up, 8 in 2024-09-20T18:50:33.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:32 smithi142 ceph-mon[32508]: pgmap v17: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-20T18:50:33.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:32 smithi142 ceph-mon[32508]: 1.0 scrub starts 2024-09-20T18:50:33.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:32 smithi142 ceph-mon[32508]: 1.0 scrub ok 2024-09-20T18:50:33.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:32 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1285202179' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-20T18:50:33.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:32 smithi083 ceph-mon[29421]: pgmap v17: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-20T18:50:33.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:32 smithi083 ceph-mon[29421]: 1.0 scrub starts 2024-09-20T18:50:33.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:32 smithi083 ceph-mon[29421]: 1.0 scrub ok 2024-09-20T18:50:33.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:32 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1285202179' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-20T18:50:33.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:32 smithi083 ceph-mon[36260]: pgmap v17: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-20T18:50:33.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:32 smithi083 ceph-mon[36260]: 1.0 scrub starts 2024-09-20T18:50:33.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:32 smithi083 ceph-mon[36260]: 1.0 scrub ok 2024-09-20T18:50:33.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:32 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1285202179' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-20T18:50:33.340 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-20T18:50:33.340 INFO:tasks.cephadm:Setup complete, yielding 2024-09-20T18:50:33.341 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-20T18:50:33.351 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-20T18:50:33.351 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'wip-guits-main-2024-09-20-1028'}, 'ceph': {'conf': {'client.0': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'ms bind msgr1': False, 'ms bind msgr2': True, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd max backfills': 3, 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'CEPHADM_STRAY_DAEMON', 'CEPHADM_FAILED_DAEMON', 'but it is still running', 'objects unfound and apparently lost', '\\(POOL_APP_NOT_ENABLED\\)'], 'log-only-match': ['CEPHADM_'], 'sha1': '88deadb8082f4a33832e228ccefc59cedba158a5'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'cephadm': {'cephadm_mode': 'root'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '88deadb8082f4a33832e228ccefc59cedba158a5'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']}, 'workunit': {'branch': 'wip-guits-main-2024-09-20-1028', 'sha1': '88deadb8082f4a33832e228ccefc59cedba158a5'}} 2024-09-20T18:50:33.352 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-20T18:50:33.352 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-20T18:50:34.565 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:50:34.913 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:34 smithi083 ceph-mon[29421]: pgmap v18: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-20T18:50:34.913 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:34 smithi083 ceph-mon[36260]: pgmap v18: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-20T18:50:35.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:34 smithi142 ceph-mon[32508]: pgmap v18: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-20T18:50:36.774 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:50:36 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:50:36.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:36 smithi142 ceph-mon[32508]: pgmap v19: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-20T18:50:36.910 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:36 smithi083 ceph-mon[29421]: pgmap v19: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-20T18:50:36.910 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:36 smithi083 ceph-mon[36260]: pgmap v19: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-20T18:50:37.237 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:50:37 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: [20/Sep/2024:18:50:37] ENGINE Bus STOPPING 2024-09-20T18:50:37.237 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 18:50:37 smithi083 podman[68164]: 2024-09-20 18:50:37.041879256 +0000 UTC m=+0.015792896 image pull 11f11916f8cdfb17c1da74b20deb2075d778357b4d60474d82453303f8d461f5 quay.io/prometheus/alertmanager:v0.27.0 2024-09-20T18:50:37.237 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 18:50:37 smithi083 podman[68164]: 2024-09-20 18:50:37.22072566 +0000 UTC m=+0.194639340 container create 61f225cfe95ae817aa7eed6e0e9be3324260782480cd8e437276895d0e3e2e5a (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-20T18:50:37.524 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:50:37 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: [20/Sep/2024:18:50:37] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-20T18:50:37.525 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:50:37 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: [20/Sep/2024:18:50:37] ENGINE Bus STOPPED 2024-09-20T18:50:37.525 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:50:37 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: [20/Sep/2024:18:50:37] ENGINE Bus STARTING 2024-09-20T18:50:37.525 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:50:37 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: [20/Sep/2024:18:50:37] ENGINE Serving on http://:::9283 2024-09-20T18:50:37.525 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:50:37 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: [20/Sep/2024:18:50:37] ENGINE Bus STARTED 2024-09-20T18:50:37.801 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:37 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:50:37.801 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:37 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' 2024-09-20T18:50:37.801 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:37 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-20T18:50:37.801 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:37 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:50:37.801 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:37 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' 2024-09-20T18:50:37.802 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:37 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-20T18:50:37.829 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-20T18:50:37.829 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-20T18:50:37.829 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T18:50:38.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:37 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:50:38.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:37 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' 2024-09-20T18:50:38.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:37 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-20T18:50:38.874 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:38 smithi083 ceph-mon[29421]: pgmap v20: 132 pgs: 132 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T18:50:38.874 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:38 smithi083 ceph-mon[29421]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-20T18:50:38.875 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:38 smithi083 ceph-mon[36260]: pgmap v20: 132 pgs: 132 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T18:50:38.875 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:38 smithi083 ceph-mon[36260]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-20T18:50:38.875 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 18:50:38 smithi083 podman[68164]: 2024-09-20 18:50:38.766109584 +0000 UTC m=+1.740023236 container init 61f225cfe95ae817aa7eed6e0e9be3324260782480cd8e437276895d0e3e2e5a (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-20T18:50:38.875 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 18:50:38 smithi083 podman[68164]: 2024-09-20 18:50:38.769633866 +0000 UTC m=+1.743547504 container start 61f225cfe95ae817aa7eed6e0e9be3324260782480cd8e437276895d0e3e2e5a (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-20T18:50:38.875 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 18:50:38 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-alertmanager-a[68238]: ts=2024-09-20T18:50:38.779Z caller=main.go:181 level=info msg="Starting Alertmanager" version="(version=0.27.0, branch=HEAD, revision=0aa3c2aad14cff039931923ab16b26b7481783b5)" 2024-09-20T18:50:38.875 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 18:50:38 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-alertmanager-a[68238]: ts=2024-09-20T18:50:38.779Z caller=main.go:182 level=info build_context="(go=go1.21.7, platform=linux/amd64, user=root@22cd11f671e9, date=20240228-11:51:20, tags=netgo)" 2024-09-20T18:50:38.876 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 18:50:38 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-alertmanager-a[68238]: ts=2024-09-20T18:50:38.781Z caller=cluster.go:186 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.83 port=9094 2024-09-20T18:50:38.876 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 18:50:38 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-alertmanager-a[68238]: ts=2024-09-20T18:50:38.781Z caller=cluster.go:683 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-20T18:50:38.876 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 18:50:38 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-alertmanager-a[68238]: ts=2024-09-20T18:50:38.804Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-20T18:50:38.876 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 18:50:38 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-alertmanager-a[68238]: ts=2024-09-20T18:50:38.805Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-20T18:50:38.876 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 18:50:38 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-alertmanager-a[68238]: ts=2024-09-20T18:50:38.807Z caller=tls_config.go:313 level=info msg="Listening on" address=[::]:9093 2024-09-20T18:50:38.876 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 18:50:38 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-alertmanager-a[68238]: ts=2024-09-20T18:50:38.807Z caller=tls_config.go:316 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-20T18:50:38.876 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 18:50:38 smithi083 bash[68164]: 61f225cfe95ae817aa7eed6e0e9be3324260782480cd8e437276895d0e3e2e5a 2024-09-20T18:50:38.925 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:38 smithi142 ceph-mon[32508]: pgmap v20: 132 pgs: 132 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T18:50:38.925 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:38 smithi142 ceph-mon[32508]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-20T18:50:39.013 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:50:39.126 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 18:50:38 smithi083 systemd[1]: Started Ceph alertmanager.a for 1ed5a50a-7780-11ef-baf5-efdc52797490. 2024-09-20T18:50:39.679 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:50:39.681 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-20T18:50:39.804 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:50:39 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: [20/Sep/2024:18:50:39] ENGINE Bus STOPPING 2024-09-20T18:50:39.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:39 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:50:39.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:39 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:50:39.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:39 smithi083 ceph-mon[29421]: osdmap e81: 8 total, 8 up, 8 in 2024-09-20T18:50:39.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:39 smithi083 ceph-mon[29421]: pgmap v22: 132 pgs: 132 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T18:50:39.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:39 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:39.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:39 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:39.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:39 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:39.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:39 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:39.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:39 smithi083 ceph-mon[29421]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-20T18:50:39.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:39 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:39.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:39 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:39.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:39 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"} : dispatch 2024-09-20T18:50:39.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:39 smithi083 ceph-mon[29421]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-20T18:50:39.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:39 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:39.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:39 smithi083 ceph-mon[29421]: Deploying daemon grafana.a on smithi142 2024-09-20T18:50:39.805 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:39 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:50:39.805 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:39 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:50:39.805 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:39 smithi083 ceph-mon[36260]: osdmap e81: 8 total, 8 up, 8 in 2024-09-20T18:50:39.805 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:39 smithi083 ceph-mon[36260]: pgmap v22: 132 pgs: 132 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T18:50:39.805 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:39 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:39.806 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:39 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:39.806 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:39 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:39.806 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:39 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:39.806 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:39 smithi083 ceph-mon[36260]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-20T18:50:39.806 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:39 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:39.806 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:39 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:39.806 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:39 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"} : dispatch 2024-09-20T18:50:39.806 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:39 smithi083 ceph-mon[36260]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-20T18:50:39.806 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:39 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:39.806 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:39 smithi083 ceph-mon[36260]: Deploying daemon grafana.a on smithi142 2024-09-20T18:50:40.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:39 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:50:40.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:39 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:50:40.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:39 smithi142 ceph-mon[32508]: osdmap e81: 8 total, 8 up, 8 in 2024-09-20T18:50:40.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:39 smithi142 ceph-mon[32508]: pgmap v22: 132 pgs: 132 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T18:50:40.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:39 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:40.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:39 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:40.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:39 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:40.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:39 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:40.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:39 smithi142 ceph-mon[32508]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-20T18:50:40.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:39 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:40.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:39 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:40.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:39 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"} : dispatch 2024-09-20T18:50:40.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:39 smithi142 ceph-mon[32508]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-20T18:50:40.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:39 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:40.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:39 smithi142 ceph-mon[32508]: Deploying daemon grafana.a on smithi142 2024-09-20T18:50:40.305 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:50:39 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: [20/Sep/2024:18:50:39] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-20T18:50:40.306 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:50:39 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: [20/Sep/2024:18:50:39] ENGINE Bus STOPPED 2024-09-20T18:50:40.306 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:50:39 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: [20/Sep/2024:18:50:39] ENGINE Bus STARTING 2024-09-20T18:50:40.306 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:50:39 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: [20/Sep/2024:18:50:39] ENGINE Serving on http://:::9283 2024-09-20T18:50:40.306 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:50:39 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: [20/Sep/2024:18:50:39] ENGINE Bus STARTED 2024-09-20T18:50:40.430 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-20T18:50:38.755108+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":893,"num_read_kb":750,"num_write":549,"num_write_kb":1673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":590588,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":552,"ondisk_log_size":552,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":245260,"kb_used_data":8220,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":749715956,"statfs":{"total":767960285184,"available":767709138944,"internally_reserved":0,"allocated":8417280,"data_stored":4473185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.200613"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":53,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.777541+0000","last_change":"2024-09-20T18:49:45.943560+0000","last_active":"2024-09-20T18:50:30.777541+0000","last_peered":"2024-09-20T18:50:30.777541+0000","last_clean":"2024-09-20T18:50:30.777541+0000","last_became_active":"2024-09-20T18:49:45.943201+0000","last_became_peered":"2024-09-20T18:49:45.943201+0000","last_unstale":"2024-09-20T18:50:30.777541+0000","last_undegraded":"2024-09-20T18:50:30.777541+0000","last_fullsized":"2024-09-20T18:50:30.777541+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:11.286122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":65,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.556166+0000","last_change":"2024-09-20T18:49:47.947057+0000","last_active":"2024-09-20T18:50:30.556166+0000","last_peered":"2024-09-20T18:50:30.556166+0000","last_clean":"2024-09-20T18:50:30.556166+0000","last_became_active":"2024-09-20T18:49:47.946161+0000","last_became_peered":"2024-09-20T18:49:47.946161+0000","last_unstale":"2024-09-20T18:50:30.556166+0000","last_undegraded":"2024-09-20T18:50:30.556166+0000","last_fullsized":"2024-09-20T18:50:30.556166+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.579926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":45,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.556202+0000","last_change":"2024-09-20T18:49:49.959797+0000","last_active":"2024-09-20T18:50:30.556202+0000","last_peered":"2024-09-20T18:50:30.556202+0000","last_clean":"2024-09-20T18:50:30.556202+0000","last_became_active":"2024-09-20T18:49:49.959514+0000","last_became_peered":"2024-09-20T18:49:49.959514+0000","last_unstale":"2024-09-20T18:50:30.556202+0000","last_undegraded":"2024-09-20T18:50:30.556202+0000","last_fullsized":"2024-09-20T18:50:30.556202+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:49.805774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":41,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.556281+0000","last_change":"2024-09-20T18:49:51.969687+0000","last_active":"2024-09-20T18:50:30.556281+0000","last_peered":"2024-09-20T18:50:30.556281+0000","last_clean":"2024-09-20T18:50:30.556281+0000","last_became_active":"2024-09-20T18:49:51.969421+0000","last_became_peered":"2024-09-20T18:49:51.969421+0000","last_unstale":"2024-09-20T18:50:30.556281+0000","last_undegraded":"2024-09-20T18:50:30.556281+0000","last_fullsized":"2024-09-20T18:50:30.556281+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.534291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":41,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:31.026537+0000","last_change":"2024-09-20T18:49:52.142168+0000","last_active":"2024-09-20T18:50:31.026537+0000","last_peered":"2024-09-20T18:50:31.026537+0000","last_clean":"2024-09-20T18:50:31.026537+0000","last_became_active":"2024-09-20T18:49:52.141967+0000","last_became_peered":"2024-09-20T18:49:52.141967+0000","last_unstale":"2024-09-20T18:50:31.026537+0000","last_undegraded":"2024-09-20T18:50:31.026537+0000","last_fullsized":"2024-09-20T18:50:31.026537+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.114636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":53,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:31.027023+0000","last_change":"2024-09-20T18:49:45.943889+0000","last_active":"2024-09-20T18:50:31.027023+0000","last_peered":"2024-09-20T18:50:31.027023+0000","last_clean":"2024-09-20T18:50:31.027023+0000","last_became_active":"2024-09-20T18:49:45.943607+0000","last_became_peered":"2024-09-20T18:49:45.943607+0000","last_unstale":"2024-09-20T18:50:31.027023+0000","last_undegraded":"2024-09-20T18:50:31.027023+0000","last_fullsized":"2024-09-20T18:50:31.027023+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.033360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":74,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:31.026594+0000","last_change":"2024-09-20T18:49:47.952638+0000","last_active":"2024-09-20T18:50:31.026594+0000","last_peered":"2024-09-20T18:50:31.026594+0000","last_clean":"2024-09-20T18:50:31.026594+0000","last_became_active":"2024-09-20T18:49:47.952150+0000","last_became_peered":"2024-09-20T18:49:47.952150+0000","last_unstale":"2024-09-20T18:50:31.026594+0000","last_undegraded":"2024-09-20T18:50:31.026594+0000","last_fullsized":"2024-09-20T18:50:31.026594+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:02.798315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":45,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.556161+0000","last_change":"2024-09-20T18:49:49.956717+0000","last_active":"2024-09-20T18:50:30.556161+0000","last_peered":"2024-09-20T18:50:30.556161+0000","last_clean":"2024-09-20T18:50:30.556161+0000","last_became_active":"2024-09-20T18:49:49.956445+0000","last_became_peered":"2024-09-20T18:49:49.956445+0000","last_unstale":"2024-09-20T18:50:30.556161+0000","last_undegraded":"2024-09-20T18:50:30.556161+0000","last_fullsized":"2024-09-20T18:50:30.556161+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:53.841144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"59'1","reported_seq":59,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.897985+0000","last_change":"2024-09-20T18:49:45.946450+0000","last_active":"2024-09-20T18:50:30.897985+0000","last_peered":"2024-09-20T18:50:30.897985+0000","last_clean":"2024-09-20T18:50:30.897985+0000","last_became_active":"2024-09-20T18:49:45.946171+0000","last_became_peered":"2024-09-20T18:49:45.946171+0000","last_unstale":"2024-09-20T18:50:30.897985+0000","last_undegraded":"2024-09-20T18:50:30.897985+0000","last_fullsized":"2024-09-20T18:50:30.897985+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:11.299334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":65,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.556586+0000","last_change":"2024-09-20T18:49:47.947755+0000","last_active":"2024-09-20T18:50:30.556586+0000","last_peered":"2024-09-20T18:50:30.556586+0000","last_clean":"2024-09-20T18:50:30.556586+0000","last_became_active":"2024-09-20T18:49:47.947448+0000","last_became_peered":"2024-09-20T18:49:47.947448+0000","last_unstale":"2024-09-20T18:50:30.556586+0000","last_undegraded":"2024-09-20T18:50:30.556586+0000","last_fullsized":"2024-09-20T18:50:30.556586+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.588680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":45,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.897468+0000","last_change":"2024-09-20T18:49:49.961448+0000","last_active":"2024-09-20T18:50:30.897468+0000","last_peered":"2024-09-20T18:50:30.897468+0000","last_clean":"2024-09-20T18:50:30.897468+0000","last_became_active":"2024-09-20T18:49:49.961188+0000","last_became_peered":"2024-09-20T18:49:49.961188+0000","last_unstale":"2024-09-20T18:50:30.897468+0000","last_undegraded":"2024-09-20T18:50:30.897468+0000","last_fullsized":"2024-09-20T18:50:30.897468+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:12.456737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":41,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.776574+0000","last_change":"2024-09-20T18:49:51.966957+0000","last_active":"2024-09-20T18:50:30.776574+0000","last_peered":"2024-09-20T18:50:30.776574+0000","last_clean":"2024-09-20T18:50:30.776574+0000","last_became_active":"2024-09-20T18:49:51.966715+0000","last_became_peered":"2024-09-20T18:49:51.966715+0000","last_unstale":"2024-09-20T18:50:30.776574+0000","last_undegraded":"2024-09-20T18:50:30.776574+0000","last_fullsized":"2024-09-20T18:50:30.776574+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:54.613818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":53,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.897734+0000","last_change":"2024-09-20T18:49:45.945004+0000","last_active":"2024-09-20T18:50:30.897734+0000","last_peered":"2024-09-20T18:50:30.897734+0000","last_clean":"2024-09-20T18:50:30.897734+0000","last_became_active":"2024-09-20T18:49:45.944703+0000","last_became_peered":"2024-09-20T18:49:45.944703+0000","last_unstale":"2024-09-20T18:50:30.897734+0000","last_undegraded":"2024-09-20T18:50:30.897734+0000","last_fullsized":"2024-09-20T18:50:30.897734+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:53.394518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":72,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.557121+0000","last_change":"2024-09-20T18:49:58.999374+0000","last_active":"2024-09-20T18:50:30.557121+0000","last_peered":"2024-09-20T18:50:30.557121+0000","last_clean":"2024-09-20T18:50:30.557121+0000","last_became_active":"2024-09-20T18:49:58.995638+0000","last_became_peered":"2024-09-20T18:49:58.995638+0000","last_unstale":"2024-09-20T18:50:30.557121+0000","last_undegraded":"2024-09-20T18:50:30.557121+0000","last_fullsized":"2024-09-20T18:50:30.557121+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:05.131889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":45,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:31.301044+0000","last_change":"2024-09-20T18:49:49.960001+0000","last_active":"2024-09-20T18:50:31.301044+0000","last_peered":"2024-09-20T18:50:31.301044+0000","last_clean":"2024-09-20T18:50:31.301044+0000","last_became_active":"2024-09-20T18:49:49.959696+0000","last_became_peered":"2024-09-20T18:49:49.959696+0000","last_unstale":"2024-09-20T18:50:31.301044+0000","last_undegraded":"2024-09-20T18:50:31.301044+0000","last_fullsized":"2024-09-20T18:50:31.301044+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.740255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":41,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.897322+0000","last_change":"2024-09-20T18:49:51.970933+0000","last_active":"2024-09-20T18:50:30.897322+0000","last_peered":"2024-09-20T18:50:30.897322+0000","last_clean":"2024-09-20T18:50:30.897322+0000","last_became_active":"2024-09-20T18:49:51.970663+0000","last_became_peered":"2024-09-20T18:49:51.970663+0000","last_unstale":"2024-09-20T18:50:30.897322+0000","last_undegraded":"2024-09-20T18:50:30.897322+0000","last_fullsized":"2024-09-20T18:50:30.897322+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:07.680885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":41,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.556662+0000","last_change":"2024-09-20T18:49:52.141482+0000","last_active":"2024-09-20T18:50:30.556662+0000","last_peered":"2024-09-20T18:50:30.556662+0000","last_clean":"2024-09-20T18:50:30.556662+0000","last_became_active":"2024-09-20T18:49:52.141170+0000","last_became_peered":"2024-09-20T18:49:52.141170+0000","last_unstale":"2024-09-20T18:50:30.556662+0000","last_undegraded":"2024-09-20T18:50:30.556662+0000","last_fullsized":"2024-09-20T18:50:30.556662+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:15.351627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":53,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.777179+0000","last_change":"2024-09-20T18:49:45.950714+0000","last_active":"2024-09-20T18:50:30.777179+0000","last_peered":"2024-09-20T18:50:30.777179+0000","last_clean":"2024-09-20T18:50:30.777179+0000","last_became_active":"2024-09-20T18:49:45.950480+0000","last_became_peered":"2024-09-20T18:49:45.950480+0000","last_unstale":"2024-09-20T18:50:30.777179+0000","last_undegraded":"2024-09-20T18:50:30.777179+0000","last_fullsized":"2024-09-20T18:50:30.777179+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:07.919860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":75,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.557140+0000","last_change":"2024-09-20T18:49:47.951228+0000","last_active":"2024-09-20T18:50:30.557140+0000","last_peered":"2024-09-20T18:50:30.557140+0000","last_clean":"2024-09-20T18:50:30.557140+0000","last_became_active":"2024-09-20T18:49:47.950939+0000","last_became_peered":"2024-09-20T18:49:47.950939+0000","last_unstale":"2024-09-20T18:50:30.557140+0000","last_undegraded":"2024-09-20T18:50:30.557140+0000","last_fullsized":"2024-09-20T18:50:30.557140+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:08.025260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":45,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.557558+0000","last_change":"2024-09-20T18:49:49.958867+0000","last_active":"2024-09-20T18:50:30.557558+0000","last_peered":"2024-09-20T18:50:30.557558+0000","last_clean":"2024-09-20T18:50:30.557558+0000","last_became_active":"2024-09-20T18:49:49.958518+0000","last_became_peered":"2024-09-20T18:49:49.958518+0000","last_unstale":"2024-09-20T18:50:30.557558+0000","last_undegraded":"2024-09-20T18:50:30.557558+0000","last_fullsized":"2024-09-20T18:50:30.557558+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:55.540498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":41,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:31.026967+0000","last_change":"2024-09-20T18:49:52.140362+0000","last_active":"2024-09-20T18:50:31.026967+0000","last_peered":"2024-09-20T18:50:31.026967+0000","last_clean":"2024-09-20T18:50:31.026967+0000","last_became_active":"2024-09-20T18:49:52.140040+0000","last_became_peered":"2024-09-20T18:49:52.140040+0000","last_unstale":"2024-09-20T18:50:31.026967+0000","last_undegraded":"2024-09-20T18:50:31.026967+0000","last_fullsized":"2024-09-20T18:50:31.026967+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:52.839541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"59'1","reported_seq":59,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.556720+0000","last_change":"2024-09-20T18:49:45.942193+0000","last_active":"2024-09-20T18:50:30.556720+0000","last_peered":"2024-09-20T18:50:30.556720+0000","last_clean":"2024-09-20T18:50:30.556720+0000","last_became_active":"2024-09-20T18:49:45.940100+0000","last_became_peered":"2024-09-20T18:49:45.940100+0000","last_unstale":"2024-09-20T18:50:30.556720+0000","last_undegraded":"2024-09-20T18:50:30.556720+0000","last_fullsized":"2024-09-20T18:50:30.556720+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:52.525377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":72,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:31.026820+0000","last_change":"2024-09-20T18:49:47.951215+0000","last_active":"2024-09-20T18:50:31.026820+0000","last_peered":"2024-09-20T18:50:31.026820+0000","last_clean":"2024-09-20T18:50:31.026820+0000","last_became_active":"2024-09-20T18:49:47.950909+0000","last_became_peered":"2024-09-20T18:49:47.950909+0000","last_unstale":"2024-09-20T18:50:31.026820+0000","last_undegraded":"2024-09-20T18:50:31.026820+0000","last_fullsized":"2024-09-20T18:50:31.026820+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:52.254250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":45,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.556440+0000","last_change":"2024-09-20T18:49:49.959365+0000","last_active":"2024-09-20T18:50:30.556440+0000","last_peered":"2024-09-20T18:50:30.556440+0000","last_clean":"2024-09-20T18:50:30.556440+0000","last_became_active":"2024-09-20T18:49:49.959102+0000","last_became_peered":"2024-09-20T18:49:49.959102+0000","last_unstale":"2024-09-20T18:50:30.556440+0000","last_undegraded":"2024-09-20T18:50:30.556440+0000","last_fullsized":"2024-09-20T18:50:30.556440+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:04.872284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":42,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:31.301535+0000","last_change":"2024-09-20T18:49:51.970469+0000","last_active":"2024-09-20T18:50:31.301535+0000","last_peered":"2024-09-20T18:50:31.301535+0000","last_clean":"2024-09-20T18:50:31.301535+0000","last_became_active":"2024-09-20T18:49:51.970058+0000","last_became_peered":"2024-09-20T18:49:51.970058+0000","last_unstale":"2024-09-20T18:50:31.301535+0000","last_undegraded":"2024-09-20T18:50:31.301535+0000","last_fullsized":"2024-09-20T18:50:31.301535+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:59.400214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":54,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.556344+0000","last_change":"2024-09-20T18:49:45.933354+0000","last_active":"2024-09-20T18:50:30.556344+0000","last_peered":"2024-09-20T18:50:30.556344+0000","last_clean":"2024-09-20T18:50:30.556344+0000","last_became_active":"2024-09-20T18:49:45.933078+0000","last_became_peered":"2024-09-20T18:49:45.933078+0000","last_unstale":"2024-09-20T18:50:30.556344+0000","last_undegraded":"2024-09-20T18:50:30.556344+0000","last_fullsized":"2024-09-20T18:50:30.556344+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:07.425698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":64,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.777451+0000","last_change":"2024-09-20T18:49:48.501151+0000","last_active":"2024-09-20T18:50:30.777451+0000","last_peered":"2024-09-20T18:50:30.777451+0000","last_clean":"2024-09-20T18:50:30.777451+0000","last_became_active":"2024-09-20T18:49:48.500809+0000","last_became_peered":"2024-09-20T18:49:48.500809+0000","last_unstale":"2024-09-20T18:50:30.777451+0000","last_undegraded":"2024-09-20T18:50:30.777451+0000","last_fullsized":"2024-09-20T18:50:30.777451+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:07.803937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":66,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:31.301690+0000","last_change":"2024-09-20T18:49:49.955074+0000","last_active":"2024-09-20T18:50:31.301690+0000","last_peered":"2024-09-20T18:50:31.301690+0000","last_clean":"2024-09-20T18:50:31.301690+0000","last_became_active":"2024-09-20T18:49:49.954772+0000","last_became_peered":"2024-09-20T18:49:49.954772+0000","last_unstale":"2024-09-20T18:50:31.301690+0000","last_undegraded":"2024-09-20T18:50:31.301690+0000","last_fullsized":"2024-09-20T18:50:31.301690+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:53.352545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":50,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.556952+0000","last_change":"2024-09-20T18:49:59.004173+0000","last_active":"2024-09-20T18:50:30.556952+0000","last_peered":"2024-09-20T18:50:30.556952+0000","last_clean":"2024-09-20T18:50:30.556952+0000","last_became_active":"2024-09-20T18:49:58.992302+0000","last_became_peered":"2024-09-20T18:49:58.992302+0000","last_unstale":"2024-09-20T18:50:30.556952+0000","last_undegraded":"2024-09-20T18:50:30.556952+0000","last_fullsized":"2024-09-20T18:50:30.556952+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:14.502406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":53,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:31.026479+0000","last_change":"2024-09-20T18:49:45.936413+0000","last_active":"2024-09-20T18:50:31.026479+0000","last_peered":"2024-09-20T18:50:31.026479+0000","last_clean":"2024-09-20T18:50:31.026479+0000","last_became_active":"2024-09-20T18:49:45.936104+0000","last_became_peered":"2024-09-20T18:49:45.936104+0000","last_unstale":"2024-09-20T18:50:31.026479+0000","last_undegraded":"2024-09-20T18:50:31.026479+0000","last_fullsized":"2024-09-20T18:50:31.026479+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:01.319743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":66,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.556892+0000","last_change":"2024-09-20T18:49:49.960978+0000","last_active":"2024-09-20T18:50:30.556892+0000","last_peered":"2024-09-20T18:50:30.556892+0000","last_clean":"2024-09-20T18:50:30.556892+0000","last_became_active":"2024-09-20T18:49:49.960697+0000","last_became_peered":"2024-09-20T18:49:49.960697+0000","last_unstale":"2024-09-20T18:50:30.556892+0000","last_undegraded":"2024-09-20T18:50:30.556892+0000","last_fullsized":"2024-09-20T18:50:30.556892+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:54.719450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":41,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.897635+0000","last_change":"2024-09-20T18:49:51.971713+0000","last_active":"2024-09-20T18:50:30.897635+0000","last_peered":"2024-09-20T18:50:30.897635+0000","last_clean":"2024-09-20T18:50:30.897635+0000","last_became_active":"2024-09-20T18:49:51.971399+0000","last_became_peered":"2024-09-20T18:49:51.971399+0000","last_unstale":"2024-09-20T18:50:30.897635+0000","last_undegraded":"2024-09-20T18:50:30.897635+0000","last_fullsized":"2024-09-20T18:50:30.897635+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:18.352126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":81,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:31.026322+0000","last_change":"2024-09-20T18:49:48.499569+0000","last_active":"2024-09-20T18:50:31.026322+0000","last_peered":"2024-09-20T18:50:31.026322+0000","last_clean":"2024-09-20T18:50:31.026322+0000","last_became_active":"2024-09-20T18:49:48.499189+0000","last_became_peered":"2024-09-20T18:49:48.499189+0000","last_unstale":"2024-09-20T18:50:31.026322+0000","last_undegraded":"2024-09-20T18:50:31.026322+0000","last_fullsized":"2024-09-20T18:50:31.026322+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:16.224188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":53,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:31.027114+0000","last_change":"2024-09-20T18:49:45.940816+0000","last_active":"2024-09-20T18:50:31.027114+0000","last_peered":"2024-09-20T18:50:31.027114+0000","last_clean":"2024-09-20T18:50:31.027114+0000","last_became_active":"2024-09-20T18:49:45.940544+0000","last_became_peered":"2024-09-20T18:49:45.940544+0000","last_unstale":"2024-09-20T18:50:31.027114+0000","last_undegraded":"2024-09-20T18:50:31.027114+0000","last_fullsized":"2024-09-20T18:50:31.027114+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:46.957556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":45,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.556166+0000","last_change":"2024-09-20T18:49:49.963362+0000","last_active":"2024-09-20T18:50:30.556166+0000","last_peered":"2024-09-20T18:50:30.556166+0000","last_clean":"2024-09-20T18:50:30.556166+0000","last_became_active":"2024-09-20T18:49:49.963073+0000","last_became_peered":"2024-09-20T18:49:49.963073+0000","last_unstale":"2024-09-20T18:50:30.556166+0000","last_undegraded":"2024-09-20T18:50:30.556166+0000","last_fullsized":"2024-09-20T18:50:30.556166+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:01.468417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":41,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.556450+0000","last_change":"2024-09-20T18:49:51.969120+0000","last_active":"2024-09-20T18:50:30.556450+0000","last_peered":"2024-09-20T18:50:30.556450+0000","last_clean":"2024-09-20T18:50:30.556450+0000","last_became_active":"2024-09-20T18:49:51.968793+0000","last_became_peered":"2024-09-20T18:49:51.968793+0000","last_unstale":"2024-09-20T18:50:30.556450+0000","last_undegraded":"2024-09-20T18:50:30.556450+0000","last_fullsized":"2024-09-20T18:50:30.556450+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:57.615454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":70,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.556205+0000","last_change":"2024-09-20T18:49:47.942407+0000","last_active":"2024-09-20T18:50:30.556205+0000","last_peered":"2024-09-20T18:50:30.556205+0000","last_clean":"2024-09-20T18:50:30.556205+0000","last_became_active":"2024-09-20T18:49:47.941895+0000","last_became_peered":"2024-09-20T18:49:47.941895+0000","last_unstale":"2024-09-20T18:50:30.556205+0000","last_undegraded":"2024-09-20T18:50:30.556205+0000","last_fullsized":"2024-09-20T18:50:30.556205+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:08.413407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":54,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.777719+0000","last_change":"2024-09-20T18:49:45.940728+0000","last_active":"2024-09-20T18:50:30.777719+0000","last_peered":"2024-09-20T18:50:30.777719+0000","last_clean":"2024-09-20T18:50:30.777719+0000","last_became_active":"2024-09-20T18:49:45.940469+0000","last_became_peered":"2024-09-20T18:49:45.940469+0000","last_unstale":"2024-09-20T18:50:30.777719+0000","last_undegraded":"2024-09-20T18:50:30.777719+0000","last_fullsized":"2024-09-20T18:50:30.777719+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:05.840900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"66'8","reported_seq":63,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:31.026232+0000","last_change":"2024-09-20T18:49:49.956436+0000","last_active":"2024-09-20T18:50:31.026232+0000","last_peered":"2024-09-20T18:50:31.026232+0000","last_clean":"2024-09-20T18:50:31.026232+0000","last_became_active":"2024-09-20T18:49:49.956133+0000","last_became_peered":"2024-09-20T18:49:49.956133+0000","last_unstale":"2024-09-20T18:50:31.026232+0000","last_undegraded":"2024-09-20T18:50:31.026232+0000","last_fullsized":"2024-09-20T18:50:31.026232+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:03.649831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":41,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:31.301200+0000","last_change":"2024-09-20T18:49:52.142481+0000","last_active":"2024-09-20T18:50:31.301200+0000","last_peered":"2024-09-20T18:50:31.301200+0000","last_clean":"2024-09-20T18:50:31.301200+0000","last_became_active":"2024-09-20T18:49:52.142113+0000","last_became_peered":"2024-09-20T18:49:52.142113+0000","last_unstale":"2024-09-20T18:50:31.301200+0000","last_undegraded":"2024-09-20T18:50:31.301200+0000","last_fullsized":"2024-09-20T18:50:31.301200+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.140166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":64,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.556698+0000","last_change":"2024-09-20T18:49:47.949952+0000","last_active":"2024-09-20T18:50:30.556698+0000","last_peered":"2024-09-20T18:50:30.556698+0000","last_clean":"2024-09-20T18:50:30.556698+0000","last_became_active":"2024-09-20T18:49:47.949626+0000","last_became_peered":"2024-09-20T18:49:47.949626+0000","last_unstale":"2024-09-20T18:50:30.556698+0000","last_undegraded":"2024-09-20T18:50:30.556698+0000","last_fullsized":"2024-09-20T18:50:30.556698+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:50.929449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":53,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.898064+0000","last_change":"2024-09-20T18:49:45.942852+0000","last_active":"2024-09-20T18:50:30.898064+0000","last_peered":"2024-09-20T18:50:30.898064+0000","last_clean":"2024-09-20T18:50:30.898064+0000","last_became_active":"2024-09-20T18:49:45.942496+0000","last_became_peered":"2024-09-20T18:49:45.942496+0000","last_unstale":"2024-09-20T18:50:30.898064+0000","last_undegraded":"2024-09-20T18:50:30.898064+0000","last_fullsized":"2024-09-20T18:50:30.898064+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:55.917563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":63,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.776970+0000","last_change":"2024-09-20T18:49:49.959699+0000","last_active":"2024-09-20T18:50:30.776970+0000","last_peered":"2024-09-20T18:50:30.776970+0000","last_clean":"2024-09-20T18:50:30.776970+0000","last_became_active":"2024-09-20T18:49:49.959365+0000","last_became_peered":"2024-09-20T18:49:49.959365+0000","last_unstale":"2024-09-20T18:50:30.776970+0000","last_undegraded":"2024-09-20T18:50:30.776970+0000","last_fullsized":"2024-09-20T18:50:30.776970+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:01.818798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":41,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.776650+0000","last_change":"2024-09-20T18:49:51.965733+0000","last_active":"2024-09-20T18:50:30.776650+0000","last_peered":"2024-09-20T18:50:30.776650+0000","last_clean":"2024-09-20T18:50:30.776650+0000","last_became_active":"2024-09-20T18:49:51.964780+0000","last_became_peered":"2024-09-20T18:49:51.964780+0000","last_unstale":"2024-09-20T18:50:30.776650+0000","last_undegraded":"2024-09-20T18:50:30.776650+0000","last_fullsized":"2024-09-20T18:50:30.776650+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:11.275213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":85,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.777028+0000","last_change":"2024-09-20T18:49:48.498141+0000","last_active":"2024-09-20T18:50:30.777028+0000","last_peered":"2024-09-20T18:50:30.777028+0000","last_clean":"2024-09-20T18:50:30.777028+0000","last_became_active":"2024-09-20T18:49:48.497736+0000","last_became_peered":"2024-09-20T18:49:48.497736+0000","last_unstale":"2024-09-20T18:50:30.777028+0000","last_undegraded":"2024-09-20T18:50:30.777028+0000","last_fullsized":"2024-09-20T18:50:30.777028+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.986657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":53,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.557245+0000","last_change":"2024-09-20T18:49:45.942589+0000","last_active":"2024-09-20T18:50:30.557245+0000","last_peered":"2024-09-20T18:50:30.557245+0000","last_clean":"2024-09-20T18:50:30.557245+0000","last_became_active":"2024-09-20T18:49:45.942253+0000","last_became_peered":"2024-09-20T18:49:45.942253+0000","last_unstale":"2024-09-20T18:50:30.557245+0000","last_undegraded":"2024-09-20T18:50:30.557245+0000","last_fullsized":"2024-09-20T18:50:30.557245+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.673189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":45,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:31.301053+0000","last_change":"2024-09-20T18:49:49.954594+0000","last_active":"2024-09-20T18:50:31.301053+0000","last_peered":"2024-09-20T18:50:31.301053+0000","last_clean":"2024-09-20T18:50:31.301053+0000","last_became_active":"2024-09-20T18:49:49.954299+0000","last_became_peered":"2024-09-20T18:49:49.954299+0000","last_unstale":"2024-09-20T18:50:31.301053+0000","last_undegraded":"2024-09-20T18:50:31.301053+0000","last_fullsized":"2024-09-20T18:50:31.301053+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:52.123030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":41,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.556116+0000","last_change":"2024-09-20T18:49:52.140276+0000","last_active":"2024-09-20T18:50:30.556116+0000","last_peered":"2024-09-20T18:50:30.556116+0000","last_clean":"2024-09-20T18:50:30.556116+0000","last_became_active":"2024-09-20T18:49:52.139981+0000","last_became_peered":"2024-09-20T18:49:52.139981+0000","last_unstale":"2024-09-20T18:50:30.556116+0000","last_undegraded":"2024-09-20T18:50:30.556116+0000","last_fullsized":"2024-09-20T18:50:30.556116+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:52.117965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"66'28","reported_seq":107,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.557172+0000","last_change":"2024-09-20T18:49:47.955615+0000","last_active":"2024-09-20T18:50:30.557172+0000","last_peered":"2024-09-20T18:50:30.557172+0000","last_clean":"2024-09-20T18:50:30.557172+0000","last_became_active":"2024-09-20T18:49:47.955335+0000","last_became_peered":"2024-09-20T18:49:47.955335+0000","last_unstale":"2024-09-20T18:50:30.557172+0000","last_undegraded":"2024-09-20T18:50:30.557172+0000","last_fullsized":"2024-09-20T18:50:30.557172+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:47.506576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":53,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.557086+0000","last_change":"2024-09-20T18:49:45.945275+0000","last_active":"2024-09-20T18:50:30.557086+0000","last_peered":"2024-09-20T18:50:30.557086+0000","last_clean":"2024-09-20T18:50:30.557086+0000","last_became_active":"2024-09-20T18:49:45.944992+0000","last_became_peered":"2024-09-20T18:49:45.944992+0000","last_unstale":"2024-09-20T18:50:30.557086+0000","last_undegraded":"2024-09-20T18:50:30.557086+0000","last_fullsized":"2024-09-20T18:50:30.557086+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:47.768689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":63,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.897581+0000","last_change":"2024-09-20T18:49:47.940836+0000","last_active":"2024-09-20T18:50:30.897581+0000","last_peered":"2024-09-20T18:50:30.897581+0000","last_clean":"2024-09-20T18:50:30.897581+0000","last_became_active":"2024-09-20T18:49:44.936027+0000","last_became_peered":"2024-09-20T18:49:44.936027+0000","last_unstale":"2024-09-20T18:50:30.897581+0000","last_undegraded":"2024-09-20T18:50:30.897581+0000","last_fullsized":"2024-09-20T18:50:30.897581+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:49:43.921002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:47.157665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012781159999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":45,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.777216+0000","last_change":"2024-09-20T18:49:49.954281+0000","last_active":"2024-09-20T18:50:30.777216+0000","last_peered":"2024-09-20T18:50:30.777216+0000","last_clean":"2024-09-20T18:50:30.777216+0000","last_became_active":"2024-09-20T18:49:49.953971+0000","last_became_peered":"2024-09-20T18:49:49.953971+0000","last_unstale":"2024-09-20T18:50:30.777216+0000","last_undegraded":"2024-09-20T18:50:30.777216+0000","last_fullsized":"2024-09-20T18:50:30.777216+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:12.341509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":41,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:31.026834+0000","last_change":"2024-09-20T18:49:51.969702+0000","last_active":"2024-09-20T18:50:31.026834+0000","last_peered":"2024-09-20T18:50:31.026834+0000","last_clean":"2024-09-20T18:50:31.026834+0000","last_became_active":"2024-09-20T18:49:51.969400+0000","last_became_peered":"2024-09-20T18:49:51.969400+0000","last_unstale":"2024-09-20T18:50:31.026834+0000","last_undegraded":"2024-09-20T18:50:31.026834+0000","last_fullsized":"2024-09-20T18:50:31.026834+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.486248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":76,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.558280+0000","last_change":"2024-09-20T18:49:47.951311+0000","last_active":"2024-09-20T18:50:30.558280+0000","last_peered":"2024-09-20T18:50:30.558280+0000","last_clean":"2024-09-20T18:50:30.558280+0000","last_became_active":"2024-09-20T18:49:47.950904+0000","last_became_peered":"2024-09-20T18:49:47.950904+0000","last_unstale":"2024-09-20T18:50:30.558280+0000","last_undegraded":"2024-09-20T18:50:30.558280+0000","last_fullsized":"2024-09-20T18:50:30.558280+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:06.445493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":53,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.555993+0000","last_change":"2024-09-20T18:49:45.940667+0000","last_active":"2024-09-20T18:50:30.555993+0000","last_peered":"2024-09-20T18:50:30.555993+0000","last_clean":"2024-09-20T18:50:30.555993+0000","last_became_active":"2024-09-20T18:49:45.939884+0000","last_became_peered":"2024-09-20T18:49:45.939884+0000","last_unstale":"2024-09-20T18:50:30.555993+0000","last_undegraded":"2024-09-20T18:50:30.555993+0000","last_fullsized":"2024-09-20T18:50:30.555993+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:49.675839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":63,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.556309+0000","last_change":"2024-09-20T18:49:47.939858+0000","last_active":"2024-09-20T18:50:30.556309+0000","last_peered":"2024-09-20T18:50:30.556309+0000","last_clean":"2024-09-20T18:50:30.556309+0000","last_became_active":"2024-09-20T18:49:44.937135+0000","last_became_peered":"2024-09-20T18:49:44.937135+0000","last_unstale":"2024-09-20T18:50:30.556309+0000","last_undegraded":"2024-09-20T18:50:30.556309+0000","last_fullsized":"2024-09-20T18:50:30.556309+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:49:43.921002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:56.442658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095558800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.557445+0000","last_change":"2024-09-20T18:49:49.954880+0000","last_active":"2024-09-20T18:50:30.557445+0000","last_peered":"2024-09-20T18:50:30.557445+0000","last_clean":"2024-09-20T18:50:30.557445+0000","last_became_active":"2024-09-20T18:49:49.954424+0000","last_became_peered":"2024-09-20T18:49:49.954424+0000","last_unstale":"2024-09-20T18:50:30.557445+0000","last_undegraded":"2024-09-20T18:50:30.557445+0000","last_fullsized":"2024-09-20T18:50:30.557445+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:54.964168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":41,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:31.301544+0000","last_change":"2024-09-20T18:49:52.140366+0000","last_active":"2024-09-20T18:50:31.301544+0000","last_peered":"2024-09-20T18:50:31.301544+0000","last_clean":"2024-09-20T18:50:31.301544+0000","last_became_active":"2024-09-20T18:49:52.139789+0000","last_became_peered":"2024-09-20T18:49:52.139789+0000","last_unstale":"2024-09-20T18:50:31.301544+0000","last_undegraded":"2024-09-20T18:50:31.301544+0000","last_fullsized":"2024-09-20T18:50:31.301544+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:16.224499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":67,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.777547+0000","last_change":"2024-09-20T18:49:47.949547+0000","last_active":"2024-09-20T18:50:30.777547+0000","last_peered":"2024-09-20T18:50:30.777547+0000","last_clean":"2024-09-20T18:50:30.777547+0000","last_became_active":"2024-09-20T18:49:47.949230+0000","last_became_peered":"2024-09-20T18:49:47.949230+0000","last_unstale":"2024-09-20T18:50:30.777547+0000","last_undegraded":"2024-09-20T18:50:30.777547+0000","last_fullsized":"2024-09-20T18:50:30.777547+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:07.802093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":53,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.776606+0000","last_change":"2024-09-20T18:49:45.939454+0000","last_active":"2024-09-20T18:50:30.776606+0000","last_peered":"2024-09-20T18:50:30.776606+0000","last_clean":"2024-09-20T18:50:30.776606+0000","last_became_active":"2024-09-20T18:49:45.939197+0000","last_became_peered":"2024-09-20T18:49:45.939197+0000","last_unstale":"2024-09-20T18:50:30.776606+0000","last_undegraded":"2024-09-20T18:50:30.776606+0000","last_fullsized":"2024-09-20T18:50:30.776606+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:12.763563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":109,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:34.355606+0000","last_change":"2024-09-20T18:49:48.498983+0000","last_active":"2024-09-20T18:50:34.355606+0000","last_peered":"2024-09-20T18:50:34.355606+0000","last_clean":"2024-09-20T18:50:34.355606+0000","last_became_active":"2024-09-20T18:49:44.936819+0000","last_became_peered":"2024-09-20T18:49:44.936819+0000","last_unstale":"2024-09-20T18:50:34.355606+0000","last_undegraded":"2024-09-20T18:50:34.355606+0000","last_fullsized":"2024-09-20T18:50:34.355606+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:49:43.921002+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:58.186148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010996199999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":45,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.898178+0000","last_change":"2024-09-20T18:49:49.961346+0000","last_active":"2024-09-20T18:50:30.898178+0000","last_peered":"2024-09-20T18:50:30.898178+0000","last_clean":"2024-09-20T18:50:30.898178+0000","last_became_active":"2024-09-20T18:49:49.961066+0000","last_became_peered":"2024-09-20T18:49:49.961066+0000","last_unstale":"2024-09-20T18:50:30.898178+0000","last_undegraded":"2024-09-20T18:50:30.898178+0000","last_fullsized":"2024-09-20T18:50:30.898178+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:53.706599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":41,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.555884+0000","last_change":"2024-09-20T18:49:51.970609+0000","last_active":"2024-09-20T18:50:30.555884+0000","last_peered":"2024-09-20T18:50:30.555884+0000","last_clean":"2024-09-20T18:50:30.555884+0000","last_became_active":"2024-09-20T18:49:51.970343+0000","last_became_peered":"2024-09-20T18:49:51.970343+0000","last_unstale":"2024-09-20T18:50:30.555884+0000","last_undegraded":"2024-09-20T18:50:30.555884+0000","last_fullsized":"2024-09-20T18:50:30.555884+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:06.660017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":81,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:31.301397+0000","last_change":"2024-09-20T18:49:48.498345+0000","last_active":"2024-09-20T18:50:31.301397+0000","last_peered":"2024-09-20T18:50:31.301397+0000","last_clean":"2024-09-20T18:50:31.301397+0000","last_became_active":"2024-09-20T18:49:48.497936+0000","last_became_peered":"2024-09-20T18:49:48.497936+0000","last_unstale":"2024-09-20T18:50:31.301397+0000","last_undegraded":"2024-09-20T18:50:31.301397+0000","last_fullsized":"2024-09-20T18:50:31.301397+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:51.535171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":53,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:31.026587+0000","last_change":"2024-09-20T18:49:45.941387+0000","last_active":"2024-09-20T18:50:31.026587+0000","last_peered":"2024-09-20T18:50:31.026587+0000","last_clean":"2024-09-20T18:50:31.026587+0000","last_became_active":"2024-09-20T18:49:45.941136+0000","last_became_peered":"2024-09-20T18:49:45.941136+0000","last_unstale":"2024-09-20T18:50:31.026587+0000","last_undegraded":"2024-09-20T18:50:31.026587+0000","last_fullsized":"2024-09-20T18:50:31.026587+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:09.074025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":100,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:31.593683+0000","last_change":"2024-09-20T18:50:31.593683+0000","last_active":"2024-09-20T18:50:31.593683+0000","last_peered":"2024-09-20T18:50:31.593683+0000","last_clean":"2024-09-20T18:50:31.593683+0000","last_became_active":"2024-09-20T18:49:36.545306+0000","last_became_peered":"2024-09-20T18:49:36.545306+0000","last_unstale":"2024-09-20T18:50:31.593683+0000","last_undegraded":"2024-09-20T18:50:31.593683+0000","last_fullsized":"2024-09-20T18:50:31.593683+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T18:50:31.593610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T18:50:31.593610+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:00.939267+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":45,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:31.301936+0000","last_change":"2024-09-20T18:49:49.956441+0000","last_active":"2024-09-20T18:50:31.301936+0000","last_peered":"2024-09-20T18:50:31.301936+0000","last_clean":"2024-09-20T18:50:31.301936+0000","last_became_active":"2024-09-20T18:49:49.956128+0000","last_became_peered":"2024-09-20T18:49:49.956128+0000","last_unstale":"2024-09-20T18:50:31.301936+0000","last_undegraded":"2024-09-20T18:50:31.301936+0000","last_fullsized":"2024-09-20T18:50:31.301936+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:04.622346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":41,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.898087+0000","last_change":"2024-09-20T18:49:51.971980+0000","last_active":"2024-09-20T18:50:30.898087+0000","last_peered":"2024-09-20T18:50:30.898087+0000","last_clean":"2024-09-20T18:50:30.898087+0000","last_became_active":"2024-09-20T18:49:51.971332+0000","last_became_peered":"2024-09-20T18:49:51.971332+0000","last_unstale":"2024-09-20T18:50:30.898087+0000","last_undegraded":"2024-09-20T18:50:30.898087+0000","last_fullsized":"2024-09-20T18:50:30.898087+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:13.174503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":68,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.556815+0000","last_change":"2024-09-20T18:49:47.950762+0000","last_active":"2024-09-20T18:50:30.556815+0000","last_peered":"2024-09-20T18:50:30.556815+0000","last_clean":"2024-09-20T18:50:30.556815+0000","last_became_active":"2024-09-20T18:49:47.950073+0000","last_became_peered":"2024-09-20T18:49:47.950073+0000","last_unstale":"2024-09-20T18:50:30.556815+0000","last_undegraded":"2024-09-20T18:50:30.556815+0000","last_fullsized":"2024-09-20T18:50:30.556815+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:57.527730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":53,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.556895+0000","last_change":"2024-09-20T18:49:45.944925+0000","last_active":"2024-09-20T18:50:30.556895+0000","last_peered":"2024-09-20T18:50:30.556895+0000","last_clean":"2024-09-20T18:50:30.556895+0000","last_became_active":"2024-09-20T18:49:45.944579+0000","last_became_peered":"2024-09-20T18:49:45.944579+0000","last_unstale":"2024-09-20T18:50:30.556895+0000","last_undegraded":"2024-09-20T18:50:30.556895+0000","last_fullsized":"2024-09-20T18:50:30.556895+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:03.333030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":45,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.897966+0000","last_change":"2024-09-20T18:49:49.954673+0000","last_active":"2024-09-20T18:50:30.897966+0000","last_peered":"2024-09-20T18:50:30.897966+0000","last_clean":"2024-09-20T18:50:30.897966+0000","last_became_active":"2024-09-20T18:49:49.954391+0000","last_became_peered":"2024-09-20T18:49:49.954391+0000","last_unstale":"2024-09-20T18:50:30.897966+0000","last_undegraded":"2024-09-20T18:50:30.897966+0000","last_fullsized":"2024-09-20T18:50:30.897966+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:12.999716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":41,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:31.026585+0000","last_change":"2024-09-20T18:49:52.140252+0000","last_active":"2024-09-20T18:50:31.026585+0000","last_peered":"2024-09-20T18:50:31.026585+0000","last_clean":"2024-09-20T18:50:31.026585+0000","last_became_active":"2024-09-20T18:49:52.139959+0000","last_became_peered":"2024-09-20T18:49:52.139959+0000","last_unstale":"2024-09-20T18:50:31.026585+0000","last_undegraded":"2024-09-20T18:50:31.026585+0000","last_fullsized":"2024-09-20T18:50:31.026585+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:54.070093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":77,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.557178+0000","last_change":"2024-09-20T18:49:47.947920+0000","last_active":"2024-09-20T18:50:30.557178+0000","last_peered":"2024-09-20T18:50:30.557178+0000","last_clean":"2024-09-20T18:50:30.557178+0000","last_became_active":"2024-09-20T18:49:47.947636+0000","last_became_peered":"2024-09-20T18:49:47.947636+0000","last_unstale":"2024-09-20T18:50:30.557178+0000","last_undegraded":"2024-09-20T18:50:30.557178+0000","last_fullsized":"2024-09-20T18:50:30.557178+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:47.335335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":53,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:31.301268+0000","last_change":"2024-09-20T18:49:45.946517+0000","last_active":"2024-09-20T18:50:31.301268+0000","last_peered":"2024-09-20T18:50:31.301268+0000","last_clean":"2024-09-20T18:50:31.301268+0000","last_became_active":"2024-09-20T18:49:45.946153+0000","last_became_peered":"2024-09-20T18:49:45.946153+0000","last_unstale":"2024-09-20T18:50:31.301268+0000","last_undegraded":"2024-09-20T18:50:31.301268+0000","last_fullsized":"2024-09-20T18:50:31.301268+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:05.889152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":45,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.556000+0000","last_change":"2024-09-20T18:49:49.959440+0000","last_active":"2024-09-20T18:50:30.556000+0000","last_peered":"2024-09-20T18:50:30.556000+0000","last_clean":"2024-09-20T18:50:30.556000+0000","last_became_active":"2024-09-20T18:49:49.959160+0000","last_became_peered":"2024-09-20T18:49:49.959160+0000","last_unstale":"2024-09-20T18:50:30.556000+0000","last_undegraded":"2024-09-20T18:50:30.556000+0000","last_fullsized":"2024-09-20T18:50:30.556000+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:08.849486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":42,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.557087+0000","last_change":"2024-09-20T18:49:51.970386+0000","last_active":"2024-09-20T18:50:30.557087+0000","last_peered":"2024-09-20T18:50:30.557087+0000","last_clean":"2024-09-20T18:50:30.557087+0000","last_became_active":"2024-09-20T18:49:51.970108+0000","last_became_peered":"2024-09-20T18:49:51.970108+0000","last_unstale":"2024-09-20T18:50:30.557087+0000","last_undegraded":"2024-09-20T18:50:30.557087+0000","last_fullsized":"2024-09-20T18:50:30.557087+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:04.919785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":64,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.556491+0000","last_change":"2024-09-20T18:49:47.948046+0000","last_active":"2024-09-20T18:50:30.556491+0000","last_peered":"2024-09-20T18:50:30.556491+0000","last_clean":"2024-09-20T18:50:30.556491+0000","last_became_active":"2024-09-20T18:49:47.947126+0000","last_became_peered":"2024-09-20T18:49:47.947126+0000","last_unstale":"2024-09-20T18:50:30.556491+0000","last_undegraded":"2024-09-20T18:50:30.556491+0000","last_fullsized":"2024-09-20T18:50:30.556491+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:16.466592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":53,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:31.026712+0000","last_change":"2024-09-20T18:49:45.937511+0000","last_active":"2024-09-20T18:50:31.026712+0000","last_peered":"2024-09-20T18:50:31.026712+0000","last_clean":"2024-09-20T18:50:31.026712+0000","last_became_active":"2024-09-20T18:49:45.937264+0000","last_became_peered":"2024-09-20T18:49:45.937264+0000","last_unstale":"2024-09-20T18:50:31.026712+0000","last_undegraded":"2024-09-20T18:50:31.026712+0000","last_fullsized":"2024-09-20T18:50:31.026712+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:06.234318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":64,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.556793+0000","last_change":"2024-09-20T18:49:49.957286+0000","last_active":"2024-09-20T18:50:30.556793+0000","last_peered":"2024-09-20T18:50:30.556793+0000","last_clean":"2024-09-20T18:50:30.556793+0000","last_became_active":"2024-09-20T18:49:49.957005+0000","last_became_peered":"2024-09-20T18:49:49.957005+0000","last_unstale":"2024-09-20T18:50:30.556793+0000","last_undegraded":"2024-09-20T18:50:30.556793+0000","last_fullsized":"2024-09-20T18:50:30.556793+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:11.425942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":41,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.556502+0000","last_change":"2024-09-20T18:49:51.969633+0000","last_active":"2024-09-20T18:50:30.556502+0000","last_peered":"2024-09-20T18:50:30.556502+0000","last_clean":"2024-09-20T18:50:30.556502+0000","last_became_active":"2024-09-20T18:49:51.969355+0000","last_became_peered":"2024-09-20T18:49:51.969355+0000","last_unstale":"2024-09-20T18:50:30.556502+0000","last_undegraded":"2024-09-20T18:50:30.556502+0000","last_fullsized":"2024-09-20T18:50:30.556502+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:53.655593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":65,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.777020+0000","last_change":"2024-09-20T18:49:47.946309+0000","last_active":"2024-09-20T18:50:30.777020+0000","last_peered":"2024-09-20T18:50:30.777020+0000","last_clean":"2024-09-20T18:50:30.777020+0000","last_became_active":"2024-09-20T18:49:47.946008+0000","last_became_peered":"2024-09-20T18:49:47.946008+0000","last_unstale":"2024-09-20T18:50:30.777020+0000","last_undegraded":"2024-09-20T18:50:30.777020+0000","last_fullsized":"2024-09-20T18:50:30.777020+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:55.957874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":53,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.897847+0000","last_change":"2024-09-20T18:49:45.941766+0000","last_active":"2024-09-20T18:50:30.897847+0000","last_peered":"2024-09-20T18:50:30.897847+0000","last_clean":"2024-09-20T18:50:30.897847+0000","last_became_active":"2024-09-20T18:49:45.941406+0000","last_became_peered":"2024-09-20T18:49:45.941406+0000","last_unstale":"2024-09-20T18:50:30.897847+0000","last_undegraded":"2024-09-20T18:50:30.897847+0000","last_fullsized":"2024-09-20T18:50:30.897847+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:46.650041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.556315+0000","last_change":"2024-09-20T18:49:49.957141+0000","last_active":"2024-09-20T18:50:30.556315+0000","last_peered":"2024-09-20T18:50:30.556315+0000","last_clean":"2024-09-20T18:50:30.556315+0000","last_became_active":"2024-09-20T18:49:49.956767+0000","last_became_peered":"2024-09-20T18:49:49.956767+0000","last_unstale":"2024-09-20T18:50:30.556315+0000","last_undegraded":"2024-09-20T18:50:30.556315+0000","last_fullsized":"2024-09-20T18:50:30.556315+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:55.731396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":41,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.776604+0000","last_change":"2024-09-20T18:49:51.964375+0000","last_active":"2024-09-20T18:50:30.776604+0000","last_peered":"2024-09-20T18:50:30.776604+0000","last_clean":"2024-09-20T18:50:30.776604+0000","last_became_active":"2024-09-20T18:49:51.964098+0000","last_became_peered":"2024-09-20T18:49:51.964098+0000","last_unstale":"2024-09-20T18:50:30.776604+0000","last_undegraded":"2024-09-20T18:50:30.776604+0000","last_fullsized":"2024-09-20T18:50:30.776604+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.566377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":80,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:31.300602+0000","last_change":"2024-09-20T18:49:48.498846+0000","last_active":"2024-09-20T18:50:31.300602+0000","last_peered":"2024-09-20T18:50:31.300602+0000","last_clean":"2024-09-20T18:50:31.300602+0000","last_became_active":"2024-09-20T18:49:48.498625+0000","last_became_peered":"2024-09-20T18:49:48.498625+0000","last_unstale":"2024-09-20T18:50:31.300602+0000","last_undegraded":"2024-09-20T18:50:31.300602+0000","last_fullsized":"2024-09-20T18:50:31.300602+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:03.186719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":53,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:31.301876+0000","last_change":"2024-09-20T18:49:45.941764+0000","last_active":"2024-09-20T18:50:31.301876+0000","last_peered":"2024-09-20T18:50:31.301876+0000","last_clean":"2024-09-20T18:50:31.301876+0000","last_became_active":"2024-09-20T18:49:45.941448+0000","last_became_peered":"2024-09-20T18:49:45.941448+0000","last_unstale":"2024-09-20T18:50:31.301876+0000","last_undegraded":"2024-09-20T18:50:31.301876+0000","last_fullsized":"2024-09-20T18:50:31.301876+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:45.120244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.556668+0000","last_change":"2024-09-20T18:49:49.957845+0000","last_active":"2024-09-20T18:50:30.556668+0000","last_peered":"2024-09-20T18:50:30.556668+0000","last_clean":"2024-09-20T18:50:30.556668+0000","last_became_active":"2024-09-20T18:49:49.957558+0000","last_became_peered":"2024-09-20T18:49:49.957558+0000","last_unstale":"2024-09-20T18:50:30.556668+0000","last_undegraded":"2024-09-20T18:50:30.556668+0000","last_fullsized":"2024-09-20T18:50:30.556668+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:14.726319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":41,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:31.301305+0000","last_change":"2024-09-20T18:49:51.971132+0000","last_active":"2024-09-20T18:50:31.301305+0000","last_peered":"2024-09-20T18:50:31.301305+0000","last_clean":"2024-09-20T18:50:31.301305+0000","last_became_active":"2024-09-20T18:49:51.970798+0000","last_became_peered":"2024-09-20T18:49:51.970798+0000","last_unstale":"2024-09-20T18:50:31.301305+0000","last_undegraded":"2024-09-20T18:50:31.301305+0000","last_fullsized":"2024-09-20T18:50:31.301305+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:09.540547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":72,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.556787+0000","last_change":"2024-09-20T18:49:47.949428+0000","last_active":"2024-09-20T18:50:30.556787+0000","last_peered":"2024-09-20T18:50:30.556787+0000","last_clean":"2024-09-20T18:50:30.556787+0000","last_became_active":"2024-09-20T18:49:47.949116+0000","last_became_peered":"2024-09-20T18:49:47.949116+0000","last_unstale":"2024-09-20T18:50:30.556787+0000","last_undegraded":"2024-09-20T18:50:30.556787+0000","last_fullsized":"2024-09-20T18:50:30.556787+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:11.352253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":53,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:31.301924+0000","last_change":"2024-09-20T18:49:45.945147+0000","last_active":"2024-09-20T18:50:31.301924+0000","last_peered":"2024-09-20T18:50:31.301924+0000","last_clean":"2024-09-20T18:50:31.301924+0000","last_became_active":"2024-09-20T18:49:45.944858+0000","last_became_peered":"2024-09-20T18:49:45.944858+0000","last_unstale":"2024-09-20T18:50:31.301924+0000","last_undegraded":"2024-09-20T18:50:31.301924+0000","last_fullsized":"2024-09-20T18:50:31.301924+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:53.147843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.555512+0000","last_change":"2024-09-20T18:49:49.955243+0000","last_active":"2024-09-20T18:50:30.555512+0000","last_peered":"2024-09-20T18:50:30.555512+0000","last_clean":"2024-09-20T18:50:30.555512+0000","last_became_active":"2024-09-20T18:49:49.954944+0000","last_became_peered":"2024-09-20T18:49:49.954944+0000","last_unstale":"2024-09-20T18:50:30.555512+0000","last_undegraded":"2024-09-20T18:50:30.555512+0000","last_fullsized":"2024-09-20T18:50:30.555512+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:00.542377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":41,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:31.026565+0000","last_change":"2024-09-20T18:49:51.973284+0000","last_active":"2024-09-20T18:50:31.026565+0000","last_peered":"2024-09-20T18:50:31.026565+0000","last_clean":"2024-09-20T18:50:31.026565+0000","last_became_active":"2024-09-20T18:49:51.972970+0000","last_became_peered":"2024-09-20T18:49:51.972970+0000","last_unstale":"2024-09-20T18:50:31.026565+0000","last_undegraded":"2024-09-20T18:50:31.026565+0000","last_fullsized":"2024-09-20T18:50:31.026565+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:55.977221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":74,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.897216+0000","last_change":"2024-09-20T18:49:48.500174+0000","last_active":"2024-09-20T18:50:30.897216+0000","last_peered":"2024-09-20T18:50:30.897216+0000","last_clean":"2024-09-20T18:50:30.897216+0000","last_became_active":"2024-09-20T18:49:48.499847+0000","last_became_peered":"2024-09-20T18:49:48.499847+0000","last_unstale":"2024-09-20T18:50:30.897216+0000","last_undegraded":"2024-09-20T18:50:30.897216+0000","last_fullsized":"2024-09-20T18:50:30.897216+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:15.572682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":65,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:31.301953+0000","last_change":"2024-09-20T18:49:45.945391+0000","last_active":"2024-09-20T18:50:31.301953+0000","last_peered":"2024-09-20T18:50:31.301953+0000","last_clean":"2024-09-20T18:50:31.301953+0000","last_became_active":"2024-09-20T18:49:45.944256+0000","last_became_peered":"2024-09-20T18:49:45.944256+0000","last_unstale":"2024-09-20T18:50:31.301953+0000","last_undegraded":"2024-09-20T18:50:31.301953+0000","last_fullsized":"2024-09-20T18:50:31.301953+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:47.823570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":63,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:31.300912+0000","last_change":"2024-09-20T18:49:49.958686+0000","last_active":"2024-09-20T18:50:31.300912+0000","last_peered":"2024-09-20T18:50:31.300912+0000","last_clean":"2024-09-20T18:50:31.300912+0000","last_became_active":"2024-09-20T18:49:49.958411+0000","last_became_peered":"2024-09-20T18:49:49.958411+0000","last_unstale":"2024-09-20T18:50:31.300912+0000","last_undegraded":"2024-09-20T18:50:31.300912+0000","last_fullsized":"2024-09-20T18:50:31.300912+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:57.283019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":41,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.897322+0000","last_change":"2024-09-20T18:49:52.139994+0000","last_active":"2024-09-20T18:50:30.897322+0000","last_peered":"2024-09-20T18:50:30.897322+0000","last_clean":"2024-09-20T18:50:30.897322+0000","last_became_active":"2024-09-20T18:49:52.139685+0000","last_became_peered":"2024-09-20T18:49:52.139685+0000","last_unstale":"2024-09-20T18:50:30.897322+0000","last_undegraded":"2024-09-20T18:50:30.897322+0000","last_fullsized":"2024-09-20T18:50:30.897322+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:53.034093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":53,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:31.300561+0000","last_change":"2024-09-20T18:49:45.940360+0000","last_active":"2024-09-20T18:50:31.300561+0000","last_peered":"2024-09-20T18:50:31.300561+0000","last_clean":"2024-09-20T18:50:31.300561+0000","last_became_active":"2024-09-20T18:49:45.940060+0000","last_became_peered":"2024-09-20T18:49:45.940060+0000","last_unstale":"2024-09-20T18:50:31.300561+0000","last_undegraded":"2024-09-20T18:50:31.300561+0000","last_fullsized":"2024-09-20T18:50:31.300561+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:59.071733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":58,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:31.027063+0000","last_change":"2024-09-20T18:49:48.501010+0000","last_active":"2024-09-20T18:50:31.027063+0000","last_peered":"2024-09-20T18:50:31.027063+0000","last_clean":"2024-09-20T18:50:31.027063+0000","last_became_active":"2024-09-20T18:49:48.500662+0000","last_became_peered":"2024-09-20T18:49:48.500662+0000","last_unstale":"2024-09-20T18:50:31.027063+0000","last_undegraded":"2024-09-20T18:50:31.027063+0000","last_fullsized":"2024-09-20T18:50:31.027063+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:01.787506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":45,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.897878+0000","last_change":"2024-09-20T18:49:49.957092+0000","last_active":"2024-09-20T18:50:30.897878+0000","last_peered":"2024-09-20T18:50:30.897878+0000","last_clean":"2024-09-20T18:50:30.897878+0000","last_became_active":"2024-09-20T18:49:49.956749+0000","last_became_peered":"2024-09-20T18:49:49.956749+0000","last_unstale":"2024-09-20T18:50:30.897878+0000","last_undegraded":"2024-09-20T18:50:30.897878+0000","last_fullsized":"2024-09-20T18:50:30.897878+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:53.840760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":41,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:31.301582+0000","last_change":"2024-09-20T18:49:52.142789+0000","last_active":"2024-09-20T18:50:31.301582+0000","last_peered":"2024-09-20T18:50:31.301582+0000","last_clean":"2024-09-20T18:50:31.301582+0000","last_became_active":"2024-09-20T18:49:52.142291+0000","last_became_peered":"2024-09-20T18:49:52.142291+0000","last_unstale":"2024-09-20T18:50:31.301582+0000","last_undegraded":"2024-09-20T18:50:31.301582+0000","last_fullsized":"2024-09-20T18:50:31.301582+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:58.299262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":65,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.777634+0000","last_change":"2024-09-20T18:49:48.501376+0000","last_active":"2024-09-20T18:50:30.777634+0000","last_peered":"2024-09-20T18:50:30.777634+0000","last_clean":"2024-09-20T18:50:30.777634+0000","last_became_active":"2024-09-20T18:49:48.500803+0000","last_became_peered":"2024-09-20T18:49:48.500803+0000","last_unstale":"2024-09-20T18:50:30.777634+0000","last_undegraded":"2024-09-20T18:50:30.777634+0000","last_fullsized":"2024-09-20T18:50:30.777634+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:02.881066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":53,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:31.301125+0000","last_change":"2024-09-20T18:49:45.947791+0000","last_active":"2024-09-20T18:50:31.301125+0000","last_peered":"2024-09-20T18:50:31.301125+0000","last_clean":"2024-09-20T18:50:31.301125+0000","last_became_active":"2024-09-20T18:49:45.946600+0000","last_became_peered":"2024-09-20T18:49:45.946600+0000","last_unstale":"2024-09-20T18:50:31.301125+0000","last_undegraded":"2024-09-20T18:50:31.301125+0000","last_fullsized":"2024-09-20T18:50:31.301125+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:13.160428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":45,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:31.026998+0000","last_change":"2024-09-20T18:49:49.962290+0000","last_active":"2024-09-20T18:50:31.026998+0000","last_peered":"2024-09-20T18:50:31.026998+0000","last_clean":"2024-09-20T18:50:31.026998+0000","last_became_active":"2024-09-20T18:49:49.961947+0000","last_became_peered":"2024-09-20T18:49:49.961947+0000","last_unstale":"2024-09-20T18:50:31.026998+0000","last_undegraded":"2024-09-20T18:50:31.026998+0000","last_fullsized":"2024-09-20T18:50:31.026998+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:54.834258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"66'1","reported_seq":43,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.556752+0000","last_change":"2024-09-20T18:49:51.965397+0000","last_active":"2024-09-20T18:50:30.556752+0000","last_peered":"2024-09-20T18:50:30.556752+0000","last_clean":"2024-09-20T18:50:30.556752+0000","last_became_active":"2024-09-20T18:49:51.965121+0000","last_became_peered":"2024-09-20T18:49:51.965121+0000","last_unstale":"2024-09-20T18:50:30.556752+0000","last_undegraded":"2024-09-20T18:50:30.556752+0000","last_fullsized":"2024-09-20T18:50:30.556752+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:57.419566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":65,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.897896+0000","last_change":"2024-09-20T18:49:48.501359+0000","last_active":"2024-09-20T18:50:30.897896+0000","last_peered":"2024-09-20T18:50:30.897896+0000","last_clean":"2024-09-20T18:50:30.897896+0000","last_became_active":"2024-09-20T18:49:48.501011+0000","last_became_peered":"2024-09-20T18:49:48.501011+0000","last_unstale":"2024-09-20T18:50:30.897896+0000","last_undegraded":"2024-09-20T18:50:30.897896+0000","last_fullsized":"2024-09-20T18:50:30.897896+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:50.431705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":53,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.776728+0000","last_change":"2024-09-20T18:49:45.938820+0000","last_active":"2024-09-20T18:50:30.776728+0000","last_peered":"2024-09-20T18:50:30.776728+0000","last_clean":"2024-09-20T18:50:30.776728+0000","last_became_active":"2024-09-20T18:49:45.938511+0000","last_became_peered":"2024-09-20T18:49:45.938511+0000","last_unstale":"2024-09-20T18:50:30.776728+0000","last_undegraded":"2024-09-20T18:50:30.776728+0000","last_fullsized":"2024-09-20T18:50:30.776728+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:02.383015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":63,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:31.026705+0000","last_change":"2024-09-20T18:49:49.960889+0000","last_active":"2024-09-20T18:50:31.026705+0000","last_peered":"2024-09-20T18:50:31.026705+0000","last_clean":"2024-09-20T18:50:31.026705+0000","last_became_active":"2024-09-20T18:49:49.960550+0000","last_became_peered":"2024-09-20T18:49:49.960550+0000","last_unstale":"2024-09-20T18:50:31.026705+0000","last_undegraded":"2024-09-20T18:50:31.026705+0000","last_fullsized":"2024-09-20T18:50:31.026705+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:58.739350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":41,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.557030+0000","last_change":"2024-09-20T18:49:51.966530+0000","last_active":"2024-09-20T18:50:30.557030+0000","last_peered":"2024-09-20T18:50:30.557030+0000","last_clean":"2024-09-20T18:50:30.557030+0000","last_became_active":"2024-09-20T18:49:51.966254+0000","last_became_peered":"2024-09-20T18:49:51.966254+0000","last_unstale":"2024-09-20T18:50:30.557030+0000","last_undegraded":"2024-09-20T18:50:30.557030+0000","last_fullsized":"2024-09-20T18:50:30.557030+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:14.193792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":83,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:31.026151+0000","last_change":"2024-09-20T18:49:59.003234+0000","last_active":"2024-09-20T18:50:31.026151+0000","last_peered":"2024-09-20T18:50:31.026151+0000","last_clean":"2024-09-20T18:50:31.026151+0000","last_became_active":"2024-09-20T18:49:58.997494+0000","last_became_peered":"2024-09-20T18:49:58.997494+0000","last_unstale":"2024-09-20T18:50:31.026151+0000","last_undegraded":"2024-09-20T18:50:31.026151+0000","last_fullsized":"2024-09-20T18:50:31.026151+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:05.235529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":53,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:31.301518+0000","last_change":"2024-09-20T18:49:45.949895+0000","last_active":"2024-09-20T18:50:31.301518+0000","last_peered":"2024-09-20T18:50:31.301518+0000","last_clean":"2024-09-20T18:50:31.301518+0000","last_became_active":"2024-09-20T18:49:45.949492+0000","last_became_peered":"2024-09-20T18:49:45.949492+0000","last_unstale":"2024-09-20T18:50:31.301518+0000","last_undegraded":"2024-09-20T18:50:31.301518+0000","last_fullsized":"2024-09-20T18:50:31.301518+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:02.958108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":63,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.897777+0000","last_change":"2024-09-20T18:49:49.962519+0000","last_active":"2024-09-20T18:50:30.897777+0000","last_peered":"2024-09-20T18:50:30.897777+0000","last_clean":"2024-09-20T18:50:30.897777+0000","last_became_active":"2024-09-20T18:49:49.962243+0000","last_became_peered":"2024-09-20T18:49:49.962243+0000","last_unstale":"2024-09-20T18:50:30.897777+0000","last_undegraded":"2024-09-20T18:50:30.897777+0000","last_fullsized":"2024-09-20T18:50:30.897777+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:16.990588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":41,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.777111+0000","last_change":"2024-09-20T18:49:51.969358+0000","last_active":"2024-09-20T18:50:30.777111+0000","last_peered":"2024-09-20T18:50:30.777111+0000","last_clean":"2024-09-20T18:50:30.777111+0000","last_became_active":"2024-09-20T18:49:51.969093+0000","last_became_peered":"2024-09-20T18:49:51.969093+0000","last_unstale":"2024-09-20T18:50:30.777111+0000","last_undegraded":"2024-09-20T18:50:30.777111+0000","last_fullsized":"2024-09-20T18:50:30.777111+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:17.459087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":68,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.556448+0000","last_change":"2024-09-20T18:49:47.950488+0000","last_active":"2024-09-20T18:50:30.556448+0000","last_peered":"2024-09-20T18:50:30.556448+0000","last_clean":"2024-09-20T18:50:30.556448+0000","last_became_active":"2024-09-20T18:49:47.949703+0000","last_became_peered":"2024-09-20T18:49:47.949703+0000","last_unstale":"2024-09-20T18:50:30.556448+0000","last_undegraded":"2024-09-20T18:50:30.556448+0000","last_fullsized":"2024-09-20T18:50:30.556448+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:53.129485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":53,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.556657+0000","last_change":"2024-09-20T18:49:45.941578+0000","last_active":"2024-09-20T18:50:30.556657+0000","last_peered":"2024-09-20T18:50:30.556657+0000","last_clean":"2024-09-20T18:50:30.556657+0000","last_became_active":"2024-09-20T18:49:45.939949+0000","last_became_peered":"2024-09-20T18:49:45.939949+0000","last_unstale":"2024-09-20T18:50:30.556657+0000","last_undegraded":"2024-09-20T18:50:30.556657+0000","last_fullsized":"2024-09-20T18:50:30.556657+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:53.450364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":45,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.556551+0000","last_change":"2024-09-20T18:49:49.957358+0000","last_active":"2024-09-20T18:50:30.556551+0000","last_peered":"2024-09-20T18:50:30.556551+0000","last_clean":"2024-09-20T18:50:30.556551+0000","last_became_active":"2024-09-20T18:49:49.957067+0000","last_became_peered":"2024-09-20T18:49:49.957067+0000","last_unstale":"2024-09-20T18:50:30.556551+0000","last_undegraded":"2024-09-20T18:50:30.556551+0000","last_fullsized":"2024-09-20T18:50:30.556551+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:54.100322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":41,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:31.026311+0000","last_change":"2024-09-20T18:49:51.967168+0000","last_active":"2024-09-20T18:50:31.026311+0000","last_peered":"2024-09-20T18:50:31.026311+0000","last_clean":"2024-09-20T18:50:31.026311+0000","last_became_active":"2024-09-20T18:49:51.966915+0000","last_became_peered":"2024-09-20T18:49:51.966915+0000","last_unstale":"2024-09-20T18:50:31.026311+0000","last_undegraded":"2024-09-20T18:50:31.026311+0000","last_fullsized":"2024-09-20T18:50:31.026311+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:04.536536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":65,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.556799+0000","last_change":"2024-09-20T18:49:47.953603+0000","last_active":"2024-09-20T18:50:30.556799+0000","last_peered":"2024-09-20T18:50:30.556799+0000","last_clean":"2024-09-20T18:50:30.556799+0000","last_became_active":"2024-09-20T18:49:47.953310+0000","last_became_peered":"2024-09-20T18:49:47.953310+0000","last_unstale":"2024-09-20T18:50:30.556799+0000","last_undegraded":"2024-09-20T18:50:30.556799+0000","last_fullsized":"2024-09-20T18:50:30.556799+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:06.158344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":53,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:31.301771+0000","last_change":"2024-09-20T18:49:45.944827+0000","last_active":"2024-09-20T18:50:31.301771+0000","last_peered":"2024-09-20T18:50:31.301771+0000","last_clean":"2024-09-20T18:50:31.301771+0000","last_became_active":"2024-09-20T18:49:45.944357+0000","last_became_peered":"2024-09-20T18:49:45.944357+0000","last_unstale":"2024-09-20T18:50:31.301771+0000","last_undegraded":"2024-09-20T18:50:31.301771+0000","last_fullsized":"2024-09-20T18:50:31.301771+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:14.013129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":45,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:31.026639+0000","last_change":"2024-09-20T18:49:49.962794+0000","last_active":"2024-09-20T18:50:31.026639+0000","last_peered":"2024-09-20T18:50:31.026639+0000","last_clean":"2024-09-20T18:50:31.026639+0000","last_became_active":"2024-09-20T18:49:49.962523+0000","last_became_peered":"2024-09-20T18:49:49.962523+0000","last_unstale":"2024-09-20T18:50:31.026639+0000","last_undegraded":"2024-09-20T18:50:31.026639+0000","last_fullsized":"2024-09-20T18:50:31.026639+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:06.567112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":41,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.776465+0000","last_change":"2024-09-20T18:49:51.967581+0000","last_active":"2024-09-20T18:50:30.776465+0000","last_peered":"2024-09-20T18:50:30.776465+0000","last_clean":"2024-09-20T18:50:30.776465+0000","last_became_active":"2024-09-20T18:49:51.967313+0000","last_became_peered":"2024-09-20T18:49:51.967313+0000","last_unstale":"2024-09-20T18:50:30.776465+0000","last_undegraded":"2024-09-20T18:50:30.776465+0000","last_fullsized":"2024-09-20T18:50:30.776465+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:05.774315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":68,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:31.026428+0000","last_change":"2024-09-20T18:49:48.499421+0000","last_active":"2024-09-20T18:50:31.026428+0000","last_peered":"2024-09-20T18:50:31.026428+0000","last_clean":"2024-09-20T18:50:31.026428+0000","last_became_active":"2024-09-20T18:49:48.499111+0000","last_became_peered":"2024-09-20T18:49:48.499111+0000","last_unstale":"2024-09-20T18:50:31.026428+0000","last_undegraded":"2024-09-20T18:50:31.026428+0000","last_fullsized":"2024-09-20T18:50:31.026428+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:57.119627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":53,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.897952+0000","last_change":"2024-09-20T18:49:45.942406+0000","last_active":"2024-09-20T18:50:30.897952+0000","last_peered":"2024-09-20T18:50:30.897952+0000","last_clean":"2024-09-20T18:50:30.897952+0000","last_became_active":"2024-09-20T18:49:45.942105+0000","last_became_peered":"2024-09-20T18:49:45.942105+0000","last_unstale":"2024-09-20T18:50:30.897952+0000","last_undegraded":"2024-09-20T18:50:30.897952+0000","last_fullsized":"2024-09-20T18:50:30.897952+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:56.309962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":45,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:31.300356+0000","last_change":"2024-09-20T18:49:49.959361+0000","last_active":"2024-09-20T18:50:31.300356+0000","last_peered":"2024-09-20T18:50:31.300356+0000","last_clean":"2024-09-20T18:50:31.300356+0000","last_became_active":"2024-09-20T18:49:49.958159+0000","last_became_peered":"2024-09-20T18:49:49.958159+0000","last_unstale":"2024-09-20T18:50:31.300356+0000","last_undegraded":"2024-09-20T18:50:31.300356+0000","last_fullsized":"2024-09-20T18:50:31.300356+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:12.385752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":41,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:31.026678+0000","last_change":"2024-09-20T18:49:52.140525+0000","last_active":"2024-09-20T18:50:31.026678+0000","last_peered":"2024-09-20T18:50:31.026678+0000","last_clean":"2024-09-20T18:50:31.026678+0000","last_became_active":"2024-09-20T18:49:52.139978+0000","last_became_peered":"2024-09-20T18:49:52.139978+0000","last_unstale":"2024-09-20T18:50:31.026678+0000","last_undegraded":"2024-09-20T18:50:31.026678+0000","last_fullsized":"2024-09-20T18:50:31.026678+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:19.358011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":55,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:31.026285+0000","last_change":"2024-09-20T18:49:47.955120+0000","last_active":"2024-09-20T18:50:31.026285+0000","last_peered":"2024-09-20T18:50:31.026285+0000","last_clean":"2024-09-20T18:50:31.026285+0000","last_became_active":"2024-09-20T18:49:47.954777+0000","last_became_peered":"2024-09-20T18:49:47.954777+0000","last_unstale":"2024-09-20T18:50:31.026285+0000","last_undegraded":"2024-09-20T18:50:31.026285+0000","last_fullsized":"2024-09-20T18:50:31.026285+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:04.481222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":53,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.777616+0000","last_change":"2024-09-20T18:49:45.941248+0000","last_active":"2024-09-20T18:50:30.777616+0000","last_peered":"2024-09-20T18:50:30.777616+0000","last_clean":"2024-09-20T18:50:30.777616+0000","last_became_active":"2024-09-20T18:49:45.940988+0000","last_became_peered":"2024-09-20T18:49:45.940988+0000","last_unstale":"2024-09-20T18:50:30.777616+0000","last_undegraded":"2024-09-20T18:50:30.777616+0000","last_fullsized":"2024-09-20T18:50:30.777616+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:58.598619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":45,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:31.300506+0000","last_change":"2024-09-20T18:49:49.954779+0000","last_active":"2024-09-20T18:50:31.300506+0000","last_peered":"2024-09-20T18:50:31.300506+0000","last_clean":"2024-09-20T18:50:31.300506+0000","last_became_active":"2024-09-20T18:49:49.954502+0000","last_became_peered":"2024-09-20T18:49:49.954502+0000","last_unstale":"2024-09-20T18:50:31.300506+0000","last_undegraded":"2024-09-20T18:50:31.300506+0000","last_fullsized":"2024-09-20T18:50:31.300506+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:15.141742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":41,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:31.300991+0000","last_change":"2024-09-20T18:49:51.968968+0000","last_active":"2024-09-20T18:50:31.300991+0000","last_peered":"2024-09-20T18:50:31.300991+0000","last_clean":"2024-09-20T18:50:31.300991+0000","last_became_active":"2024-09-20T18:49:51.968662+0000","last_became_peered":"2024-09-20T18:49:51.968662+0000","last_unstale":"2024-09-20T18:50:31.300991+0000","last_undegraded":"2024-09-20T18:50:31.300991+0000","last_fullsized":"2024-09-20T18:50:31.300991+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:02.092669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":41,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.557310+0000","last_change":"2024-09-20T18:49:51.968476+0000","last_active":"2024-09-20T18:50:30.557310+0000","last_peered":"2024-09-20T18:50:30.557310+0000","last_clean":"2024-09-20T18:50:30.557310+0000","last_became_active":"2024-09-20T18:49:51.968216+0000","last_became_peered":"2024-09-20T18:49:51.968216+0000","last_unstale":"2024-09-20T18:50:30.557310+0000","last_undegraded":"2024-09-20T18:50:30.557310+0000","last_fullsized":"2024-09-20T18:50:30.557310+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:11.930622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":53,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:31.026422+0000","last_change":"2024-09-20T18:49:45.938413+0000","last_active":"2024-09-20T18:50:31.026422+0000","last_peered":"2024-09-20T18:50:31.026422+0000","last_clean":"2024-09-20T18:50:31.026422+0000","last_became_active":"2024-09-20T18:49:45.937992+0000","last_became_peered":"2024-09-20T18:49:45.937992+0000","last_unstale":"2024-09-20T18:50:31.026422+0000","last_undegraded":"2024-09-20T18:50:31.026422+0000","last_fullsized":"2024-09-20T18:50:31.026422+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:12.596336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":68,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:31.301784+0000","last_change":"2024-09-20T18:49:47.950306+0000","last_active":"2024-09-20T18:50:31.301784+0000","last_peered":"2024-09-20T18:50:31.301784+0000","last_clean":"2024-09-20T18:50:31.301784+0000","last_became_active":"2024-09-20T18:49:47.949582+0000","last_became_peered":"2024-09-20T18:49:47.949582+0000","last_unstale":"2024-09-20T18:50:31.301784+0000","last_undegraded":"2024-09-20T18:50:31.301784+0000","last_fullsized":"2024-09-20T18:50:31.301784+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:03.753158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":45,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-20T18:50:30.777329+0000","last_change":"2024-09-20T18:49:49.957874+0000","last_active":"2024-09-20T18:50:30.777329+0000","last_peered":"2024-09-20T18:50:30.777329+0000","last_clean":"2024-09-20T18:50:30.777329+0000","last_became_active":"2024-09-20T18:49:49.957582+0000","last_became_peered":"2024-09-20T18:49:49.957582+0000","last_unstale":"2024-09-20T18:50:30.777329+0000","last_undegraded":"2024-09-20T18:50:30.777329+0000","last_fullsized":"2024-09-20T18:50:30.777329+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:07.454286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266702,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32580,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712572,"statfs":{"total":95995035648,"available":95961673728,"internally_reserved":0,"allocated":1421312,"data_stored":927977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528341,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28400,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716752,"statfs":{"total":95995035648,"available":95965954048,"internally_reserved":0,"allocated":1400832,"data_stored":927315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757274,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31984,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713168,"statfs":{"total":95995035648,"available":95962284032,"internally_reserved":0,"allocated":811008,"data_stored":337306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986207,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":843776,"data_stored":338558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182438,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32008,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713144,"statfs":{"total":95995035648,"available":95962259456,"internally_reserved":0,"allocated":839680,"data_stored":337402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411371,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31984,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713168,"statfs":{"total":95995035648,"available":95962284032,"internally_reserved":0,"allocated":815104,"data_stored":337885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":864256,"data_stored":339069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738423,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32576,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712576,"statfs":{"total":95995035648,"available":95961677824,"internally_reserved":0,"allocated":1421312,"data_stored":927673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T18:50:40.436 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T18:50:40.690 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:50:40.737 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:40 smithi083 ceph-mon[29421]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:50:40.737 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:40 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-20T18:50:40.737 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:40 smithi083 ceph-mon[29421]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-20T18:50:40.737 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:40 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:40.737 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:40 smithi083 ceph-mon[29421]: osdmap e82: 8 total, 8 up, 8 in 2024-09-20T18:50:40.738 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:40 smithi083 ceph-mon[36260]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:50:40.738 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:40 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-20T18:50:40.738 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:40 smithi083 ceph-mon[36260]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-20T18:50:40.738 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:40 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:40.738 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:40 smithi083 ceph-mon[36260]: osdmap e82: 8 total, 8 up, 8 in 2024-09-20T18:50:41.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:40 smithi142 ceph-mon[32508]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:50:41.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:40 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-20T18:50:41.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:40 smithi142 ceph-mon[32508]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-20T18:50:41.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:40 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:41.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:40 smithi142 ceph-mon[32508]: osdmap e82: 8 total, 8 up, 8 in 2024-09-20T18:50:41.038 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:50:40 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:50:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:50:41.039 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 18:50:40 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-alertmanager-a[68238]: ts=2024-09-20T18:50:40.782Z caller=cluster.go:708 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000170249s 2024-09-20T18:50:41.311 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:50:41.312 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-20T18:50:42.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:41 smithi142 ceph-mon[32508]: pgmap v24: 132 pgs: 132 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T18:50:42.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:41 smithi142 ceph-mon[32508]: from='client.24611 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:50:42.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:41 smithi083 ceph-mon[29421]: pgmap v24: 132 pgs: 132 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T18:50:42.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:41 smithi083 ceph-mon[29421]: from='client.24611 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:50:42.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:41 smithi083 ceph-mon[36260]: pgmap v24: 132 pgs: 132 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T18:50:42.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:41 smithi083 ceph-mon[36260]: from='client.24611 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:50:42.096 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-20T18:50:40.755410+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":898,"num_read_kb":755,"num_write":549,"num_write_kb":1673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":590588,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":552,"ondisk_log_size":552,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":245352,"kb_used_data":8328,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":749715864,"statfs":{"total":767960285184,"available":767709044736,"internally_reserved":0,"allocated":8527872,"data_stored":4531716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001665"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":55,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.583615+0000","last_change":"2024-09-20T18:49:45.943560+0000","last_active":"2024-09-20T18:50:38.583615+0000","last_peered":"2024-09-20T18:50:38.583615+0000","last_clean":"2024-09-20T18:50:38.583615+0000","last_became_active":"2024-09-20T18:49:45.943201+0000","last_became_peered":"2024-09-20T18:49:45.943201+0000","last_unstale":"2024-09-20T18:50:38.583615+0000","last_undegraded":"2024-09-20T18:50:38.583615+0000","last_fullsized":"2024-09-20T18:50:38.583615+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:11.286122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":69,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-20T18:50:40.562774+0000","last_change":"2024-09-20T18:49:47.947057+0000","last_active":"2024-09-20T18:50:40.562774+0000","last_peered":"2024-09-20T18:50:40.562774+0000","last_clean":"2024-09-20T18:50:40.562774+0000","last_became_active":"2024-09-20T18:49:47.946161+0000","last_became_peered":"2024-09-20T18:49:47.946161+0000","last_unstale":"2024-09-20T18:50:40.562774+0000","last_undegraded":"2024-09-20T18:50:40.562774+0000","last_fullsized":"2024-09-20T18:50:40.562774+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.579926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":49,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-20T18:50:40.564081+0000","last_change":"2024-09-20T18:49:49.959797+0000","last_active":"2024-09-20T18:50:40.564081+0000","last_peered":"2024-09-20T18:50:40.564081+0000","last_clean":"2024-09-20T18:50:40.564081+0000","last_became_active":"2024-09-20T18:49:49.959514+0000","last_became_peered":"2024-09-20T18:49:49.959514+0000","last_unstale":"2024-09-20T18:50:40.564081+0000","last_undegraded":"2024-09-20T18:50:40.564081+0000","last_fullsized":"2024-09-20T18:50:40.564081+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:49.805774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-20T18:50:40.562902+0000","last_change":"2024-09-20T18:49:51.969687+0000","last_active":"2024-09-20T18:50:40.562902+0000","last_peered":"2024-09-20T18:50:40.562902+0000","last_clean":"2024-09-20T18:50:40.562902+0000","last_became_active":"2024-09-20T18:49:51.969421+0000","last_became_peered":"2024-09-20T18:49:51.969421+0000","last_unstale":"2024-09-20T18:50:40.562902+0000","last_undegraded":"2024-09-20T18:50:40.562902+0000","last_fullsized":"2024-09-20T18:50:40.562902+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.534291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":43,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.802189+0000","last_change":"2024-09-20T18:49:52.142168+0000","last_active":"2024-09-20T18:50:38.802189+0000","last_peered":"2024-09-20T18:50:38.802189+0000","last_clean":"2024-09-20T18:50:38.802189+0000","last_became_active":"2024-09-20T18:49:52.141967+0000","last_became_peered":"2024-09-20T18:49:52.141967+0000","last_unstale":"2024-09-20T18:50:38.802189+0000","last_undegraded":"2024-09-20T18:50:38.802189+0000","last_fullsized":"2024-09-20T18:50:38.802189+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.114636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":55,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.802896+0000","last_change":"2024-09-20T18:49:45.943889+0000","last_active":"2024-09-20T18:50:38.802896+0000","last_peered":"2024-09-20T18:50:38.802896+0000","last_clean":"2024-09-20T18:50:38.802896+0000","last_became_active":"2024-09-20T18:49:45.943607+0000","last_became_peered":"2024-09-20T18:49:45.943607+0000","last_unstale":"2024-09-20T18:50:38.802896+0000","last_undegraded":"2024-09-20T18:50:38.802896+0000","last_fullsized":"2024-09-20T18:50:38.802896+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.033360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":76,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.802140+0000","last_change":"2024-09-20T18:49:47.952638+0000","last_active":"2024-09-20T18:50:38.802140+0000","last_peered":"2024-09-20T18:50:38.802140+0000","last_clean":"2024-09-20T18:50:38.802140+0000","last_became_active":"2024-09-20T18:49:47.952150+0000","last_became_peered":"2024-09-20T18:49:47.952150+0000","last_unstale":"2024-09-20T18:50:38.802140+0000","last_undegraded":"2024-09-20T18:50:38.802140+0000","last_fullsized":"2024-09-20T18:50:38.802140+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:02.798315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":49,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-20T18:50:40.562748+0000","last_change":"2024-09-20T18:49:49.956717+0000","last_active":"2024-09-20T18:50:40.562748+0000","last_peered":"2024-09-20T18:50:40.562748+0000","last_clean":"2024-09-20T18:50:40.562748+0000","last_became_active":"2024-09-20T18:49:49.956445+0000","last_became_peered":"2024-09-20T18:49:49.956445+0000","last_unstale":"2024-09-20T18:50:40.562748+0000","last_undegraded":"2024-09-20T18:50:40.562748+0000","last_fullsized":"2024-09-20T18:50:40.562748+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:53.841144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"59'1","reported_seq":61,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.804106+0000","last_change":"2024-09-20T18:49:45.946450+0000","last_active":"2024-09-20T18:50:38.804106+0000","last_peered":"2024-09-20T18:50:38.804106+0000","last_clean":"2024-09-20T18:50:38.804106+0000","last_became_active":"2024-09-20T18:49:45.946171+0000","last_became_peered":"2024-09-20T18:49:45.946171+0000","last_unstale":"2024-09-20T18:50:38.804106+0000","last_undegraded":"2024-09-20T18:50:38.804106+0000","last_fullsized":"2024-09-20T18:50:38.804106+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:11.299334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":69,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-20T18:50:40.563167+0000","last_change":"2024-09-20T18:49:47.947755+0000","last_active":"2024-09-20T18:50:40.563167+0000","last_peered":"2024-09-20T18:50:40.563167+0000","last_clean":"2024-09-20T18:50:40.563167+0000","last_became_active":"2024-09-20T18:49:47.947448+0000","last_became_peered":"2024-09-20T18:49:47.947448+0000","last_unstale":"2024-09-20T18:50:40.563167+0000","last_undegraded":"2024-09-20T18:50:40.563167+0000","last_fullsized":"2024-09-20T18:50:40.563167+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.588680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":47,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.803548+0000","last_change":"2024-09-20T18:49:49.961448+0000","last_active":"2024-09-20T18:50:38.803548+0000","last_peered":"2024-09-20T18:50:38.803548+0000","last_clean":"2024-09-20T18:50:38.803548+0000","last_became_active":"2024-09-20T18:49:49.961188+0000","last_became_peered":"2024-09-20T18:49:49.961188+0000","last_unstale":"2024-09-20T18:50:38.803548+0000","last_undegraded":"2024-09-20T18:50:38.803548+0000","last_fullsized":"2024-09-20T18:50:38.803548+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:12.456737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":43,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.582389+0000","last_change":"2024-09-20T18:49:51.966957+0000","last_active":"2024-09-20T18:50:38.582389+0000","last_peered":"2024-09-20T18:50:38.582389+0000","last_clean":"2024-09-20T18:50:38.582389+0000","last_became_active":"2024-09-20T18:49:51.966715+0000","last_became_peered":"2024-09-20T18:49:51.966715+0000","last_unstale":"2024-09-20T18:50:38.582389+0000","last_undegraded":"2024-09-20T18:50:38.582389+0000","last_fullsized":"2024-09-20T18:50:38.582389+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:54.613818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":55,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.803973+0000","last_change":"2024-09-20T18:49:45.945004+0000","last_active":"2024-09-20T18:50:38.803973+0000","last_peered":"2024-09-20T18:50:38.803973+0000","last_clean":"2024-09-20T18:50:38.803973+0000","last_became_active":"2024-09-20T18:49:45.944703+0000","last_became_peered":"2024-09-20T18:49:45.944703+0000","last_unstale":"2024-09-20T18:50:38.803973+0000","last_undegraded":"2024-09-20T18:50:38.803973+0000","last_fullsized":"2024-09-20T18:50:38.803973+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:53.394518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":76,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-20T18:50:40.562928+0000","last_change":"2024-09-20T18:49:58.999374+0000","last_active":"2024-09-20T18:50:40.562928+0000","last_peered":"2024-09-20T18:50:40.562928+0000","last_clean":"2024-09-20T18:50:40.562928+0000","last_became_active":"2024-09-20T18:49:58.995638+0000","last_became_peered":"2024-09-20T18:49:58.995638+0000","last_unstale":"2024-09-20T18:50:40.562928+0000","last_undegraded":"2024-09-20T18:50:40.562928+0000","last_fullsized":"2024-09-20T18:50:40.562928+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:05.131889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":47,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.802766+0000","last_change":"2024-09-20T18:49:49.960001+0000","last_active":"2024-09-20T18:50:38.802766+0000","last_peered":"2024-09-20T18:50:38.802766+0000","last_clean":"2024-09-20T18:50:38.802766+0000","last_became_active":"2024-09-20T18:49:49.959696+0000","last_became_peered":"2024-09-20T18:49:49.959696+0000","last_unstale":"2024-09-20T18:50:38.802766+0000","last_undegraded":"2024-09-20T18:50:38.802766+0000","last_fullsized":"2024-09-20T18:50:38.802766+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.740255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":43,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.802934+0000","last_change":"2024-09-20T18:49:51.970933+0000","last_active":"2024-09-20T18:50:38.802934+0000","last_peered":"2024-09-20T18:50:38.802934+0000","last_clean":"2024-09-20T18:50:38.802934+0000","last_became_active":"2024-09-20T18:49:51.970663+0000","last_became_peered":"2024-09-20T18:49:51.970663+0000","last_unstale":"2024-09-20T18:50:38.802934+0000","last_undegraded":"2024-09-20T18:50:38.802934+0000","last_fullsized":"2024-09-20T18:50:38.802934+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:07.680885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-20T18:50:40.563382+0000","last_change":"2024-09-20T18:49:52.141482+0000","last_active":"2024-09-20T18:50:40.563382+0000","last_peered":"2024-09-20T18:50:40.563382+0000","last_clean":"2024-09-20T18:50:40.563382+0000","last_became_active":"2024-09-20T18:49:52.141170+0000","last_became_peered":"2024-09-20T18:49:52.141170+0000","last_unstale":"2024-09-20T18:50:40.563382+0000","last_undegraded":"2024-09-20T18:50:40.563382+0000","last_fullsized":"2024-09-20T18:50:40.563382+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:15.351627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":55,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.583229+0000","last_change":"2024-09-20T18:49:45.950714+0000","last_active":"2024-09-20T18:50:38.583229+0000","last_peered":"2024-09-20T18:50:38.583229+0000","last_clean":"2024-09-20T18:50:38.583229+0000","last_became_active":"2024-09-20T18:49:45.950480+0000","last_became_peered":"2024-09-20T18:49:45.950480+0000","last_unstale":"2024-09-20T18:50:38.583229+0000","last_undegraded":"2024-09-20T18:50:38.583229+0000","last_fullsized":"2024-09-20T18:50:38.583229+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:07.919860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":79,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-20T18:50:40.563441+0000","last_change":"2024-09-20T18:49:47.951228+0000","last_active":"2024-09-20T18:50:40.563441+0000","last_peered":"2024-09-20T18:50:40.563441+0000","last_clean":"2024-09-20T18:50:40.563441+0000","last_became_active":"2024-09-20T18:49:47.950939+0000","last_became_peered":"2024-09-20T18:49:47.950939+0000","last_unstale":"2024-09-20T18:50:40.563441+0000","last_undegraded":"2024-09-20T18:50:40.563441+0000","last_fullsized":"2024-09-20T18:50:40.563441+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:08.025260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":49,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-20T18:50:40.564517+0000","last_change":"2024-09-20T18:49:49.958867+0000","last_active":"2024-09-20T18:50:40.564517+0000","last_peered":"2024-09-20T18:50:40.564517+0000","last_clean":"2024-09-20T18:50:40.564517+0000","last_became_active":"2024-09-20T18:49:49.958518+0000","last_became_peered":"2024-09-20T18:49:49.958518+0000","last_unstale":"2024-09-20T18:50:40.564517+0000","last_undegraded":"2024-09-20T18:50:40.564517+0000","last_fullsized":"2024-09-20T18:50:40.564517+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:55.540498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":43,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.802790+0000","last_change":"2024-09-20T18:49:52.140362+0000","last_active":"2024-09-20T18:50:38.802790+0000","last_peered":"2024-09-20T18:50:38.802790+0000","last_clean":"2024-09-20T18:50:38.802790+0000","last_became_active":"2024-09-20T18:49:52.140040+0000","last_became_peered":"2024-09-20T18:49:52.140040+0000","last_unstale":"2024-09-20T18:50:38.802790+0000","last_undegraded":"2024-09-20T18:50:38.802790+0000","last_fullsized":"2024-09-20T18:50:38.802790+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:52.839541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"59'1","reported_seq":63,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-20T18:50:40.563249+0000","last_change":"2024-09-20T18:49:45.942193+0000","last_active":"2024-09-20T18:50:40.563249+0000","last_peered":"2024-09-20T18:50:40.563249+0000","last_clean":"2024-09-20T18:50:40.563249+0000","last_became_active":"2024-09-20T18:49:45.940100+0000","last_became_peered":"2024-09-20T18:49:45.940100+0000","last_unstale":"2024-09-20T18:50:40.563249+0000","last_undegraded":"2024-09-20T18:50:40.563249+0000","last_fullsized":"2024-09-20T18:50:40.563249+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:52.525377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":74,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.802562+0000","last_change":"2024-09-20T18:49:47.951215+0000","last_active":"2024-09-20T18:50:38.802562+0000","last_peered":"2024-09-20T18:50:38.802562+0000","last_clean":"2024-09-20T18:50:38.802562+0000","last_became_active":"2024-09-20T18:49:47.950909+0000","last_became_peered":"2024-09-20T18:49:47.950909+0000","last_unstale":"2024-09-20T18:50:38.802562+0000","last_undegraded":"2024-09-20T18:50:38.802562+0000","last_fullsized":"2024-09-20T18:50:38.802562+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:52.254250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":49,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-20T18:50:40.563144+0000","last_change":"2024-09-20T18:49:49.959365+0000","last_active":"2024-09-20T18:50:40.563144+0000","last_peered":"2024-09-20T18:50:40.563144+0000","last_clean":"2024-09-20T18:50:40.563144+0000","last_became_active":"2024-09-20T18:49:49.959102+0000","last_became_peered":"2024-09-20T18:49:49.959102+0000","last_unstale":"2024-09-20T18:50:40.563144+0000","last_undegraded":"2024-09-20T18:50:40.563144+0000","last_fullsized":"2024-09-20T18:50:40.563144+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:04.872284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":44,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.803416+0000","last_change":"2024-09-20T18:49:51.970469+0000","last_active":"2024-09-20T18:50:38.803416+0000","last_peered":"2024-09-20T18:50:38.803416+0000","last_clean":"2024-09-20T18:50:38.803416+0000","last_became_active":"2024-09-20T18:49:51.970058+0000","last_became_peered":"2024-09-20T18:49:51.970058+0000","last_unstale":"2024-09-20T18:50:38.803416+0000","last_undegraded":"2024-09-20T18:50:38.803416+0000","last_fullsized":"2024-09-20T18:50:38.803416+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:59.400214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":58,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-20T18:50:40.564171+0000","last_change":"2024-09-20T18:49:45.933354+0000","last_active":"2024-09-20T18:50:40.564171+0000","last_peered":"2024-09-20T18:50:40.564171+0000","last_clean":"2024-09-20T18:50:40.564171+0000","last_became_active":"2024-09-20T18:49:45.933078+0000","last_became_peered":"2024-09-20T18:49:45.933078+0000","last_unstale":"2024-09-20T18:50:40.564171+0000","last_undegraded":"2024-09-20T18:50:40.564171+0000","last_fullsized":"2024-09-20T18:50:40.564171+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:07.425698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":66,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.583649+0000","last_change":"2024-09-20T18:49:48.501151+0000","last_active":"2024-09-20T18:50:38.583649+0000","last_peered":"2024-09-20T18:50:38.583649+0000","last_clean":"2024-09-20T18:50:38.583649+0000","last_became_active":"2024-09-20T18:49:48.500809+0000","last_became_peered":"2024-09-20T18:49:48.500809+0000","last_unstale":"2024-09-20T18:50:38.583649+0000","last_undegraded":"2024-09-20T18:50:38.583649+0000","last_fullsized":"2024-09-20T18:50:38.583649+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:07.803937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":69,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.803372+0000","last_change":"2024-09-20T18:49:49.955074+0000","last_active":"2024-09-20T18:50:38.803372+0000","last_peered":"2024-09-20T18:50:38.803372+0000","last_clean":"2024-09-20T18:50:38.803372+0000","last_became_active":"2024-09-20T18:49:49.954772+0000","last_became_peered":"2024-09-20T18:49:49.954772+0000","last_unstale":"2024-09-20T18:50:38.803372+0000","last_undegraded":"2024-09-20T18:50:38.803372+0000","last_fullsized":"2024-09-20T18:50:38.803372+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:53.352545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":54,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-20T18:50:40.562951+0000","last_change":"2024-09-20T18:49:59.004173+0000","last_active":"2024-09-20T18:50:40.562951+0000","last_peered":"2024-09-20T18:50:40.562951+0000","last_clean":"2024-09-20T18:50:40.562951+0000","last_became_active":"2024-09-20T18:49:58.992302+0000","last_became_peered":"2024-09-20T18:49:58.992302+0000","last_unstale":"2024-09-20T18:50:40.562951+0000","last_undegraded":"2024-09-20T18:50:40.562951+0000","last_fullsized":"2024-09-20T18:50:40.562951+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:14.502406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":55,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.801685+0000","last_change":"2024-09-20T18:49:45.936413+0000","last_active":"2024-09-20T18:50:38.801685+0000","last_peered":"2024-09-20T18:50:38.801685+0000","last_clean":"2024-09-20T18:50:38.801685+0000","last_became_active":"2024-09-20T18:49:45.936104+0000","last_became_peered":"2024-09-20T18:49:45.936104+0000","last_unstale":"2024-09-20T18:50:38.801685+0000","last_undegraded":"2024-09-20T18:50:38.801685+0000","last_fullsized":"2024-09-20T18:50:38.801685+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:01.319743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":71,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-20T18:50:40.563466+0000","last_change":"2024-09-20T18:49:49.960978+0000","last_active":"2024-09-20T18:50:40.563466+0000","last_peered":"2024-09-20T18:50:40.563466+0000","last_clean":"2024-09-20T18:50:40.563466+0000","last_became_active":"2024-09-20T18:49:49.960697+0000","last_became_peered":"2024-09-20T18:49:49.960697+0000","last_unstale":"2024-09-20T18:50:40.563466+0000","last_undegraded":"2024-09-20T18:50:40.563466+0000","last_fullsized":"2024-09-20T18:50:40.563466+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:54.719450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":43,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.803676+0000","last_change":"2024-09-20T18:49:51.971713+0000","last_active":"2024-09-20T18:50:38.803676+0000","last_peered":"2024-09-20T18:50:38.803676+0000","last_clean":"2024-09-20T18:50:38.803676+0000","last_became_active":"2024-09-20T18:49:51.971399+0000","last_became_peered":"2024-09-20T18:49:51.971399+0000","last_unstale":"2024-09-20T18:50:38.803676+0000","last_undegraded":"2024-09-20T18:50:38.803676+0000","last_fullsized":"2024-09-20T18:50:38.803676+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:18.352126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":83,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.801472+0000","last_change":"2024-09-20T18:49:48.499569+0000","last_active":"2024-09-20T18:50:38.801472+0000","last_peered":"2024-09-20T18:50:38.801472+0000","last_clean":"2024-09-20T18:50:38.801472+0000","last_became_active":"2024-09-20T18:49:48.499189+0000","last_became_peered":"2024-09-20T18:49:48.499189+0000","last_unstale":"2024-09-20T18:50:38.801472+0000","last_undegraded":"2024-09-20T18:50:38.801472+0000","last_fullsized":"2024-09-20T18:50:38.801472+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:16.224188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":55,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.803109+0000","last_change":"2024-09-20T18:49:45.940816+0000","last_active":"2024-09-20T18:50:38.803109+0000","last_peered":"2024-09-20T18:50:38.803109+0000","last_clean":"2024-09-20T18:50:38.803109+0000","last_became_active":"2024-09-20T18:49:45.940544+0000","last_became_peered":"2024-09-20T18:49:45.940544+0000","last_unstale":"2024-09-20T18:50:38.803109+0000","last_undegraded":"2024-09-20T18:50:38.803109+0000","last_fullsized":"2024-09-20T18:50:38.803109+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:46.957556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":49,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-20T18:50:40.562780+0000","last_change":"2024-09-20T18:49:49.963362+0000","last_active":"2024-09-20T18:50:40.562780+0000","last_peered":"2024-09-20T18:50:40.562780+0000","last_clean":"2024-09-20T18:50:40.562780+0000","last_became_active":"2024-09-20T18:49:49.963073+0000","last_became_peered":"2024-09-20T18:49:49.963073+0000","last_unstale":"2024-09-20T18:50:40.562780+0000","last_undegraded":"2024-09-20T18:50:40.562780+0000","last_fullsized":"2024-09-20T18:50:40.562780+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:01.468417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-20T18:50:40.563086+0000","last_change":"2024-09-20T18:49:51.969120+0000","last_active":"2024-09-20T18:50:40.563086+0000","last_peered":"2024-09-20T18:50:40.563086+0000","last_clean":"2024-09-20T18:50:40.563086+0000","last_became_active":"2024-09-20T18:49:51.968793+0000","last_became_peered":"2024-09-20T18:49:51.968793+0000","last_unstale":"2024-09-20T18:50:40.563086+0000","last_undegraded":"2024-09-20T18:50:40.563086+0000","last_fullsized":"2024-09-20T18:50:40.563086+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:57.615454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":74,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-20T18:50:40.562830+0000","last_change":"2024-09-20T18:49:47.942407+0000","last_active":"2024-09-20T18:50:40.562830+0000","last_peered":"2024-09-20T18:50:40.562830+0000","last_clean":"2024-09-20T18:50:40.562830+0000","last_became_active":"2024-09-20T18:49:47.941895+0000","last_became_peered":"2024-09-20T18:49:47.941895+0000","last_unstale":"2024-09-20T18:50:40.562830+0000","last_undegraded":"2024-09-20T18:50:40.562830+0000","last_fullsized":"2024-09-20T18:50:40.562830+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:08.413407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":56,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.583853+0000","last_change":"2024-09-20T18:49:45.940728+0000","last_active":"2024-09-20T18:50:38.583853+0000","last_peered":"2024-09-20T18:50:38.583853+0000","last_clean":"2024-09-20T18:50:38.583853+0000","last_became_active":"2024-09-20T18:49:45.940469+0000","last_became_peered":"2024-09-20T18:49:45.940469+0000","last_unstale":"2024-09-20T18:50:38.583853+0000","last_undegraded":"2024-09-20T18:50:38.583853+0000","last_fullsized":"2024-09-20T18:50:38.583853+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:05.840900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"66'8","reported_seq":66,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.801278+0000","last_change":"2024-09-20T18:49:49.956436+0000","last_active":"2024-09-20T18:50:38.801278+0000","last_peered":"2024-09-20T18:50:38.801278+0000","last_clean":"2024-09-20T18:50:38.801278+0000","last_became_active":"2024-09-20T18:49:49.956133+0000","last_became_peered":"2024-09-20T18:49:49.956133+0000","last_unstale":"2024-09-20T18:50:38.801278+0000","last_undegraded":"2024-09-20T18:50:38.801278+0000","last_fullsized":"2024-09-20T18:50:38.801278+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:03.649831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":43,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.803065+0000","last_change":"2024-09-20T18:49:52.142481+0000","last_active":"2024-09-20T18:50:38.803065+0000","last_peered":"2024-09-20T18:50:38.803065+0000","last_clean":"2024-09-20T18:50:38.803065+0000","last_became_active":"2024-09-20T18:49:52.142113+0000","last_became_peered":"2024-09-20T18:49:52.142113+0000","last_unstale":"2024-09-20T18:50:38.803065+0000","last_undegraded":"2024-09-20T18:50:38.803065+0000","last_fullsized":"2024-09-20T18:50:38.803065+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.140166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":68,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-20T18:50:40.564526+0000","last_change":"2024-09-20T18:49:47.949952+0000","last_active":"2024-09-20T18:50:40.564526+0000","last_peered":"2024-09-20T18:50:40.564526+0000","last_clean":"2024-09-20T18:50:40.564526+0000","last_became_active":"2024-09-20T18:49:47.949626+0000","last_became_peered":"2024-09-20T18:49:47.949626+0000","last_unstale":"2024-09-20T18:50:40.564526+0000","last_undegraded":"2024-09-20T18:50:40.564526+0000","last_fullsized":"2024-09-20T18:50:40.564526+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:50.929449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":55,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.804238+0000","last_change":"2024-09-20T18:49:45.942852+0000","last_active":"2024-09-20T18:50:38.804238+0000","last_peered":"2024-09-20T18:50:38.804238+0000","last_clean":"2024-09-20T18:50:38.804238+0000","last_became_active":"2024-09-20T18:49:45.942496+0000","last_became_peered":"2024-09-20T18:49:45.942496+0000","last_unstale":"2024-09-20T18:50:38.804238+0000","last_undegraded":"2024-09-20T18:50:38.804238+0000","last_fullsized":"2024-09-20T18:50:38.804238+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:55.917563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":66,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.583038+0000","last_change":"2024-09-20T18:49:49.959699+0000","last_active":"2024-09-20T18:50:38.583038+0000","last_peered":"2024-09-20T18:50:38.583038+0000","last_clean":"2024-09-20T18:50:38.583038+0000","last_became_active":"2024-09-20T18:49:49.959365+0000","last_became_peered":"2024-09-20T18:49:49.959365+0000","last_unstale":"2024-09-20T18:50:38.583038+0000","last_undegraded":"2024-09-20T18:50:38.583038+0000","last_fullsized":"2024-09-20T18:50:38.583038+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:01.818798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":43,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.582595+0000","last_change":"2024-09-20T18:49:51.965733+0000","last_active":"2024-09-20T18:50:38.582595+0000","last_peered":"2024-09-20T18:50:38.582595+0000","last_clean":"2024-09-20T18:50:38.582595+0000","last_became_active":"2024-09-20T18:49:51.964780+0000","last_became_peered":"2024-09-20T18:49:51.964780+0000","last_unstale":"2024-09-20T18:50:38.582595+0000","last_undegraded":"2024-09-20T18:50:38.582595+0000","last_fullsized":"2024-09-20T18:50:38.582595+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:11.275213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":87,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.583128+0000","last_change":"2024-09-20T18:49:48.498141+0000","last_active":"2024-09-20T18:50:38.583128+0000","last_peered":"2024-09-20T18:50:38.583128+0000","last_clean":"2024-09-20T18:50:38.583128+0000","last_became_active":"2024-09-20T18:49:48.497736+0000","last_became_peered":"2024-09-20T18:49:48.497736+0000","last_unstale":"2024-09-20T18:50:38.583128+0000","last_undegraded":"2024-09-20T18:50:38.583128+0000","last_fullsized":"2024-09-20T18:50:38.583128+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.986657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":57,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-20T18:50:40.565043+0000","last_change":"2024-09-20T18:49:45.942589+0000","last_active":"2024-09-20T18:50:40.565043+0000","last_peered":"2024-09-20T18:50:40.565043+0000","last_clean":"2024-09-20T18:50:40.565043+0000","last_became_active":"2024-09-20T18:49:45.942253+0000","last_became_peered":"2024-09-20T18:49:45.942253+0000","last_unstale":"2024-09-20T18:50:40.565043+0000","last_undegraded":"2024-09-20T18:50:40.565043+0000","last_fullsized":"2024-09-20T18:50:40.565043+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.673189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":47,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.801722+0000","last_change":"2024-09-20T18:49:49.954594+0000","last_active":"2024-09-20T18:50:38.801722+0000","last_peered":"2024-09-20T18:50:38.801722+0000","last_clean":"2024-09-20T18:50:38.801722+0000","last_became_active":"2024-09-20T18:49:49.954299+0000","last_became_peered":"2024-09-20T18:49:49.954299+0000","last_unstale":"2024-09-20T18:50:38.801722+0000","last_undegraded":"2024-09-20T18:50:38.801722+0000","last_fullsized":"2024-09-20T18:50:38.801722+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:52.123030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-20T18:50:40.564076+0000","last_change":"2024-09-20T18:49:52.140276+0000","last_active":"2024-09-20T18:50:40.564076+0000","last_peered":"2024-09-20T18:50:40.564076+0000","last_clean":"2024-09-20T18:50:40.564076+0000","last_became_active":"2024-09-20T18:49:52.139981+0000","last_became_peered":"2024-09-20T18:49:52.139981+0000","last_unstale":"2024-09-20T18:50:40.564076+0000","last_undegraded":"2024-09-20T18:50:40.564076+0000","last_fullsized":"2024-09-20T18:50:40.564076+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:52.117965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"66'28","reported_seq":112,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-20T18:50:40.564998+0000","last_change":"2024-09-20T18:49:47.955615+0000","last_active":"2024-09-20T18:50:40.564998+0000","last_peered":"2024-09-20T18:50:40.564998+0000","last_clean":"2024-09-20T18:50:40.564998+0000","last_became_active":"2024-09-20T18:49:47.955335+0000","last_became_peered":"2024-09-20T18:49:47.955335+0000","last_unstale":"2024-09-20T18:50:40.564998+0000","last_undegraded":"2024-09-20T18:50:40.564998+0000","last_fullsized":"2024-09-20T18:50:40.564998+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:47.506576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":57,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-20T18:50:40.563173+0000","last_change":"2024-09-20T18:49:45.945275+0000","last_active":"2024-09-20T18:50:40.563173+0000","last_peered":"2024-09-20T18:50:40.563173+0000","last_clean":"2024-09-20T18:50:40.563173+0000","last_became_active":"2024-09-20T18:49:45.944992+0000","last_became_peered":"2024-09-20T18:49:45.944992+0000","last_unstale":"2024-09-20T18:50:40.563173+0000","last_undegraded":"2024-09-20T18:50:40.563173+0000","last_fullsized":"2024-09-20T18:50:40.563173+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:47.768689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":65,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.803511+0000","last_change":"2024-09-20T18:49:47.940836+0000","last_active":"2024-09-20T18:50:38.803511+0000","last_peered":"2024-09-20T18:50:38.803511+0000","last_clean":"2024-09-20T18:50:38.803511+0000","last_became_active":"2024-09-20T18:49:44.936027+0000","last_became_peered":"2024-09-20T18:49:44.936027+0000","last_unstale":"2024-09-20T18:50:38.803511+0000","last_undegraded":"2024-09-20T18:50:38.803511+0000","last_fullsized":"2024-09-20T18:50:38.803511+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:49:43.921002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:47.157665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012781159999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":47,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.583230+0000","last_change":"2024-09-20T18:49:49.954281+0000","last_active":"2024-09-20T18:50:38.583230+0000","last_peered":"2024-09-20T18:50:38.583230+0000","last_clean":"2024-09-20T18:50:38.583230+0000","last_became_active":"2024-09-20T18:49:49.953971+0000","last_became_peered":"2024-09-20T18:49:49.953971+0000","last_unstale":"2024-09-20T18:50:38.583230+0000","last_undegraded":"2024-09-20T18:50:38.583230+0000","last_fullsized":"2024-09-20T18:50:38.583230+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:12.341509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":43,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.802728+0000","last_change":"2024-09-20T18:49:51.969702+0000","last_active":"2024-09-20T18:50:38.802728+0000","last_peered":"2024-09-20T18:50:38.802728+0000","last_clean":"2024-09-20T18:50:38.802728+0000","last_became_active":"2024-09-20T18:49:51.969400+0000","last_became_peered":"2024-09-20T18:49:51.969400+0000","last_unstale":"2024-09-20T18:50:38.802728+0000","last_undegraded":"2024-09-20T18:50:38.802728+0000","last_fullsized":"2024-09-20T18:50:38.802728+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.486248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":80,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-20T18:50:40.563957+0000","last_change":"2024-09-20T18:49:47.951311+0000","last_active":"2024-09-20T18:50:40.563957+0000","last_peered":"2024-09-20T18:50:40.563957+0000","last_clean":"2024-09-20T18:50:40.563957+0000","last_became_active":"2024-09-20T18:49:47.950904+0000","last_became_peered":"2024-09-20T18:49:47.950904+0000","last_unstale":"2024-09-20T18:50:40.563957+0000","last_undegraded":"2024-09-20T18:50:40.563957+0000","last_fullsized":"2024-09-20T18:50:40.563957+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:06.445493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":57,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-20T18:50:40.563639+0000","last_change":"2024-09-20T18:49:45.940667+0000","last_active":"2024-09-20T18:50:40.563639+0000","last_peered":"2024-09-20T18:50:40.563639+0000","last_clean":"2024-09-20T18:50:40.563639+0000","last_became_active":"2024-09-20T18:49:45.939884+0000","last_became_peered":"2024-09-20T18:49:45.939884+0000","last_unstale":"2024-09-20T18:50:40.563639+0000","last_undegraded":"2024-09-20T18:50:40.563639+0000","last_fullsized":"2024-09-20T18:50:40.563639+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:49.675839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":67,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-20T18:50:40.562803+0000","last_change":"2024-09-20T18:49:47.939858+0000","last_active":"2024-09-20T18:50:40.562803+0000","last_peered":"2024-09-20T18:50:40.562803+0000","last_clean":"2024-09-20T18:50:40.562803+0000","last_became_active":"2024-09-20T18:49:44.937135+0000","last_became_peered":"2024-09-20T18:49:44.937135+0000","last_unstale":"2024-09-20T18:50:40.562803+0000","last_undegraded":"2024-09-20T18:50:40.562803+0000","last_fullsized":"2024-09-20T18:50:40.562803+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:49:43.921002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:56.442658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095558800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":50,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-20T18:50:40.562914+0000","last_change":"2024-09-20T18:49:49.954880+0000","last_active":"2024-09-20T18:50:40.562914+0000","last_peered":"2024-09-20T18:50:40.562914+0000","last_clean":"2024-09-20T18:50:40.562914+0000","last_became_active":"2024-09-20T18:49:49.954424+0000","last_became_peered":"2024-09-20T18:49:49.954424+0000","last_unstale":"2024-09-20T18:50:40.562914+0000","last_undegraded":"2024-09-20T18:50:40.562914+0000","last_fullsized":"2024-09-20T18:50:40.562914+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:54.964168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":43,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.802496+0000","last_change":"2024-09-20T18:49:52.140366+0000","last_active":"2024-09-20T18:50:38.802496+0000","last_peered":"2024-09-20T18:50:38.802496+0000","last_clean":"2024-09-20T18:50:38.802496+0000","last_became_active":"2024-09-20T18:49:52.139789+0000","last_became_peered":"2024-09-20T18:49:52.139789+0000","last_unstale":"2024-09-20T18:50:38.802496+0000","last_undegraded":"2024-09-20T18:50:38.802496+0000","last_fullsized":"2024-09-20T18:50:38.802496+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:16.224499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":69,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.583735+0000","last_change":"2024-09-20T18:49:47.949547+0000","last_active":"2024-09-20T18:50:38.583735+0000","last_peered":"2024-09-20T18:50:38.583735+0000","last_clean":"2024-09-20T18:50:38.583735+0000","last_became_active":"2024-09-20T18:49:47.949230+0000","last_became_peered":"2024-09-20T18:49:47.949230+0000","last_unstale":"2024-09-20T18:50:38.583735+0000","last_undegraded":"2024-09-20T18:50:38.583735+0000","last_fullsized":"2024-09-20T18:50:38.583735+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:07.802093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":55,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.583102+0000","last_change":"2024-09-20T18:49:45.939454+0000","last_active":"2024-09-20T18:50:38.583102+0000","last_peered":"2024-09-20T18:50:38.583102+0000","last_clean":"2024-09-20T18:50:38.583102+0000","last_became_active":"2024-09-20T18:49:45.939197+0000","last_became_peered":"2024-09-20T18:49:45.939197+0000","last_unstale":"2024-09-20T18:50:38.583102+0000","last_undegraded":"2024-09-20T18:50:38.583102+0000","last_fullsized":"2024-09-20T18:50:38.583102+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:12.763563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":116,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:39.360490+0000","last_change":"2024-09-20T18:49:48.498983+0000","last_active":"2024-09-20T18:50:39.360490+0000","last_peered":"2024-09-20T18:50:39.360490+0000","last_clean":"2024-09-20T18:50:39.360490+0000","last_became_active":"2024-09-20T18:49:44.936819+0000","last_became_peered":"2024-09-20T18:49:44.936819+0000","last_unstale":"2024-09-20T18:50:39.360490+0000","last_undegraded":"2024-09-20T18:50:39.360490+0000","last_fullsized":"2024-09-20T18:50:39.360490+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:49:43.921002+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:58.186148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010996199999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":50,"num_read_kb":45,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":47,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.804273+0000","last_change":"2024-09-20T18:49:49.961346+0000","last_active":"2024-09-20T18:50:38.804273+0000","last_peered":"2024-09-20T18:50:38.804273+0000","last_clean":"2024-09-20T18:50:38.804273+0000","last_became_active":"2024-09-20T18:49:49.961066+0000","last_became_peered":"2024-09-20T18:49:49.961066+0000","last_unstale":"2024-09-20T18:50:38.804273+0000","last_undegraded":"2024-09-20T18:50:38.804273+0000","last_fullsized":"2024-09-20T18:50:38.804273+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:53.706599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-20T18:50:40.564785+0000","last_change":"2024-09-20T18:49:51.970609+0000","last_active":"2024-09-20T18:50:40.564785+0000","last_peered":"2024-09-20T18:50:40.564785+0000","last_clean":"2024-09-20T18:50:40.564785+0000","last_became_active":"2024-09-20T18:49:51.970343+0000","last_became_peered":"2024-09-20T18:49:51.970343+0000","last_unstale":"2024-09-20T18:50:40.564785+0000","last_undegraded":"2024-09-20T18:50:40.564785+0000","last_fullsized":"2024-09-20T18:50:40.564785+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:06.660017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":84,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.802093+0000","last_change":"2024-09-20T18:49:48.498345+0000","last_active":"2024-09-20T18:50:38.802093+0000","last_peered":"2024-09-20T18:50:38.802093+0000","last_clean":"2024-09-20T18:50:38.802093+0000","last_became_active":"2024-09-20T18:49:48.497936+0000","last_became_peered":"2024-09-20T18:49:48.497936+0000","last_unstale":"2024-09-20T18:50:38.802093+0000","last_undegraded":"2024-09-20T18:50:38.802093+0000","last_fullsized":"2024-09-20T18:50:38.802093+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:51.535171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":55,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.802136+0000","last_change":"2024-09-20T18:49:45.941387+0000","last_active":"2024-09-20T18:50:38.802136+0000","last_peered":"2024-09-20T18:50:38.802136+0000","last_clean":"2024-09-20T18:50:38.802136+0000","last_became_active":"2024-09-20T18:49:45.941136+0000","last_became_peered":"2024-09-20T18:49:45.941136+0000","last_unstale":"2024-09-20T18:50:38.802136+0000","last_undegraded":"2024-09-20T18:50:38.802136+0000","last_fullsized":"2024-09-20T18:50:38.802136+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:09.074025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":102,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.802158+0000","last_change":"2024-09-20T18:50:31.593683+0000","last_active":"2024-09-20T18:50:38.802158+0000","last_peered":"2024-09-20T18:50:38.802158+0000","last_clean":"2024-09-20T18:50:38.802158+0000","last_became_active":"2024-09-20T18:49:36.545306+0000","last_became_peered":"2024-09-20T18:49:36.545306+0000","last_unstale":"2024-09-20T18:50:38.802158+0000","last_undegraded":"2024-09-20T18:50:38.802158+0000","last_fullsized":"2024-09-20T18:50:38.802158+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T18:50:31.593610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T18:50:31.593610+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:00.939267+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":47,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.803836+0000","last_change":"2024-09-20T18:49:49.956441+0000","last_active":"2024-09-20T18:50:38.803836+0000","last_peered":"2024-09-20T18:50:38.803836+0000","last_clean":"2024-09-20T18:50:38.803836+0000","last_became_active":"2024-09-20T18:49:49.956128+0000","last_became_peered":"2024-09-20T18:49:49.956128+0000","last_unstale":"2024-09-20T18:50:38.803836+0000","last_undegraded":"2024-09-20T18:50:38.803836+0000","last_fullsized":"2024-09-20T18:50:38.803836+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:04.622346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":43,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.804125+0000","last_change":"2024-09-20T18:49:51.971980+0000","last_active":"2024-09-20T18:50:38.804125+0000","last_peered":"2024-09-20T18:50:38.804125+0000","last_clean":"2024-09-20T18:50:38.804125+0000","last_became_active":"2024-09-20T18:49:51.971332+0000","last_became_peered":"2024-09-20T18:49:51.971332+0000","last_unstale":"2024-09-20T18:50:38.804125+0000","last_undegraded":"2024-09-20T18:50:38.804125+0000","last_fullsized":"2024-09-20T18:50:38.804125+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:13.174503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":72,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-20T18:50:40.563552+0000","last_change":"2024-09-20T18:49:47.950762+0000","last_active":"2024-09-20T18:50:40.563552+0000","last_peered":"2024-09-20T18:50:40.563552+0000","last_clean":"2024-09-20T18:50:40.563552+0000","last_became_active":"2024-09-20T18:49:47.950073+0000","last_became_peered":"2024-09-20T18:49:47.950073+0000","last_unstale":"2024-09-20T18:50:40.563552+0000","last_undegraded":"2024-09-20T18:50:40.563552+0000","last_fullsized":"2024-09-20T18:50:40.563552+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:57.527730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":57,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-20T18:50:40.563092+0000","last_change":"2024-09-20T18:49:45.944925+0000","last_active":"2024-09-20T18:50:40.563092+0000","last_peered":"2024-09-20T18:50:40.563092+0000","last_clean":"2024-09-20T18:50:40.563092+0000","last_became_active":"2024-09-20T18:49:45.944579+0000","last_became_peered":"2024-09-20T18:49:45.944579+0000","last_unstale":"2024-09-20T18:50:40.563092+0000","last_undegraded":"2024-09-20T18:50:40.563092+0000","last_fullsized":"2024-09-20T18:50:40.563092+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:03.333030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":47,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.803909+0000","last_change":"2024-09-20T18:49:49.954673+0000","last_active":"2024-09-20T18:50:38.803909+0000","last_peered":"2024-09-20T18:50:38.803909+0000","last_clean":"2024-09-20T18:50:38.803909+0000","last_became_active":"2024-09-20T18:49:49.954391+0000","last_became_peered":"2024-09-20T18:49:49.954391+0000","last_unstale":"2024-09-20T18:50:38.803909+0000","last_undegraded":"2024-09-20T18:50:38.803909+0000","last_fullsized":"2024-09-20T18:50:38.803909+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:12.999716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":43,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.802293+0000","last_change":"2024-09-20T18:49:52.140252+0000","last_active":"2024-09-20T18:50:38.802293+0000","last_peered":"2024-09-20T18:50:38.802293+0000","last_clean":"2024-09-20T18:50:38.802293+0000","last_became_active":"2024-09-20T18:49:52.139959+0000","last_became_peered":"2024-09-20T18:49:52.139959+0000","last_unstale":"2024-09-20T18:50:38.802293+0000","last_undegraded":"2024-09-20T18:50:38.802293+0000","last_fullsized":"2024-09-20T18:50:38.802293+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:54.070093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":81,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-20T18:50:40.563624+0000","last_change":"2024-09-20T18:49:47.947920+0000","last_active":"2024-09-20T18:50:40.563624+0000","last_peered":"2024-09-20T18:50:40.563624+0000","last_clean":"2024-09-20T18:50:40.563624+0000","last_became_active":"2024-09-20T18:49:47.947636+0000","last_became_peered":"2024-09-20T18:49:47.947636+0000","last_unstale":"2024-09-20T18:50:40.563624+0000","last_undegraded":"2024-09-20T18:50:40.563624+0000","last_fullsized":"2024-09-20T18:50:40.563624+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:47.335335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":55,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.802299+0000","last_change":"2024-09-20T18:49:45.946517+0000","last_active":"2024-09-20T18:50:38.802299+0000","last_peered":"2024-09-20T18:50:38.802299+0000","last_clean":"2024-09-20T18:50:38.802299+0000","last_became_active":"2024-09-20T18:49:45.946153+0000","last_became_peered":"2024-09-20T18:49:45.946153+0000","last_unstale":"2024-09-20T18:50:38.802299+0000","last_undegraded":"2024-09-20T18:50:38.802299+0000","last_fullsized":"2024-09-20T18:50:38.802299+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:05.889152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":49,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-20T18:50:40.564870+0000","last_change":"2024-09-20T18:49:49.959440+0000","last_active":"2024-09-20T18:50:40.564870+0000","last_peered":"2024-09-20T18:50:40.564870+0000","last_clean":"2024-09-20T18:50:40.564870+0000","last_became_active":"2024-09-20T18:49:49.959160+0000","last_became_peered":"2024-09-20T18:49:49.959160+0000","last_unstale":"2024-09-20T18:50:40.564870+0000","last_undegraded":"2024-09-20T18:50:40.564870+0000","last_fullsized":"2024-09-20T18:50:40.564870+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:08.849486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-20T18:50:40.563131+0000","last_change":"2024-09-20T18:49:51.970386+0000","last_active":"2024-09-20T18:50:40.563131+0000","last_peered":"2024-09-20T18:50:40.563131+0000","last_clean":"2024-09-20T18:50:40.563131+0000","last_became_active":"2024-09-20T18:49:51.970108+0000","last_became_peered":"2024-09-20T18:49:51.970108+0000","last_unstale":"2024-09-20T18:50:40.563131+0000","last_undegraded":"2024-09-20T18:50:40.563131+0000","last_fullsized":"2024-09-20T18:50:40.563131+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:04.919785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":68,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-20T18:50:40.563276+0000","last_change":"2024-09-20T18:49:47.948046+0000","last_active":"2024-09-20T18:50:40.563276+0000","last_peered":"2024-09-20T18:50:40.563276+0000","last_clean":"2024-09-20T18:50:40.563276+0000","last_became_active":"2024-09-20T18:49:47.947126+0000","last_became_peered":"2024-09-20T18:49:47.947126+0000","last_unstale":"2024-09-20T18:50:40.563276+0000","last_undegraded":"2024-09-20T18:50:40.563276+0000","last_fullsized":"2024-09-20T18:50:40.563276+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:16.466592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":55,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.802402+0000","last_change":"2024-09-20T18:49:45.937511+0000","last_active":"2024-09-20T18:50:38.802402+0000","last_peered":"2024-09-20T18:50:38.802402+0000","last_clean":"2024-09-20T18:50:38.802402+0000","last_became_active":"2024-09-20T18:49:45.937264+0000","last_became_peered":"2024-09-20T18:49:45.937264+0000","last_unstale":"2024-09-20T18:50:38.802402+0000","last_undegraded":"2024-09-20T18:50:38.802402+0000","last_fullsized":"2024-09-20T18:50:38.802402+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:06.234318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":69,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-20T18:50:40.562724+0000","last_change":"2024-09-20T18:49:49.957286+0000","last_active":"2024-09-20T18:50:40.562724+0000","last_peered":"2024-09-20T18:50:40.562724+0000","last_clean":"2024-09-20T18:50:40.562724+0000","last_became_active":"2024-09-20T18:49:49.957005+0000","last_became_peered":"2024-09-20T18:49:49.957005+0000","last_unstale":"2024-09-20T18:50:40.562724+0000","last_undegraded":"2024-09-20T18:50:40.562724+0000","last_fullsized":"2024-09-20T18:50:40.562724+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:11.425942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-20T18:50:40.563294+0000","last_change":"2024-09-20T18:49:51.969633+0000","last_active":"2024-09-20T18:50:40.563294+0000","last_peered":"2024-09-20T18:50:40.563294+0000","last_clean":"2024-09-20T18:50:40.563294+0000","last_became_active":"2024-09-20T18:49:51.969355+0000","last_became_peered":"2024-09-20T18:49:51.969355+0000","last_unstale":"2024-09-20T18:50:40.563294+0000","last_undegraded":"2024-09-20T18:50:40.563294+0000","last_fullsized":"2024-09-20T18:50:40.563294+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:53.655593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":67,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.583056+0000","last_change":"2024-09-20T18:49:47.946309+0000","last_active":"2024-09-20T18:50:38.583056+0000","last_peered":"2024-09-20T18:50:38.583056+0000","last_clean":"2024-09-20T18:50:38.583056+0000","last_became_active":"2024-09-20T18:49:47.946008+0000","last_became_peered":"2024-09-20T18:49:47.946008+0000","last_unstale":"2024-09-20T18:50:38.583056+0000","last_undegraded":"2024-09-20T18:50:38.583056+0000","last_fullsized":"2024-09-20T18:50:38.583056+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:55.957874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":55,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.804069+0000","last_change":"2024-09-20T18:49:45.941766+0000","last_active":"2024-09-20T18:50:38.804069+0000","last_peered":"2024-09-20T18:50:38.804069+0000","last_clean":"2024-09-20T18:50:38.804069+0000","last_became_active":"2024-09-20T18:49:45.941406+0000","last_became_peered":"2024-09-20T18:49:45.941406+0000","last_unstale":"2024-09-20T18:50:38.804069+0000","last_undegraded":"2024-09-20T18:50:38.804069+0000","last_fullsized":"2024-09-20T18:50:38.804069+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:46.650041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":50,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-20T18:50:40.562874+0000","last_change":"2024-09-20T18:49:49.957141+0000","last_active":"2024-09-20T18:50:40.562874+0000","last_peered":"2024-09-20T18:50:40.562874+0000","last_clean":"2024-09-20T18:50:40.562874+0000","last_became_active":"2024-09-20T18:49:49.956767+0000","last_became_peered":"2024-09-20T18:49:49.956767+0000","last_unstale":"2024-09-20T18:50:40.562874+0000","last_undegraded":"2024-09-20T18:50:40.562874+0000","last_fullsized":"2024-09-20T18:50:40.562874+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:55.731396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":43,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.583015+0000","last_change":"2024-09-20T18:49:51.964375+0000","last_active":"2024-09-20T18:50:38.583015+0000","last_peered":"2024-09-20T18:50:38.583015+0000","last_clean":"2024-09-20T18:50:38.583015+0000","last_became_active":"2024-09-20T18:49:51.964098+0000","last_became_peered":"2024-09-20T18:49:51.964098+0000","last_unstale":"2024-09-20T18:50:38.583015+0000","last_undegraded":"2024-09-20T18:50:38.583015+0000","last_fullsized":"2024-09-20T18:50:38.583015+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.566377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":82,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.802015+0000","last_change":"2024-09-20T18:49:48.498846+0000","last_active":"2024-09-20T18:50:38.802015+0000","last_peered":"2024-09-20T18:50:38.802015+0000","last_clean":"2024-09-20T18:50:38.802015+0000","last_became_active":"2024-09-20T18:49:48.498625+0000","last_became_peered":"2024-09-20T18:49:48.498625+0000","last_unstale":"2024-09-20T18:50:38.802015+0000","last_undegraded":"2024-09-20T18:50:38.802015+0000","last_fullsized":"2024-09-20T18:50:38.802015+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:03.186719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":55,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.802835+0000","last_change":"2024-09-20T18:49:45.941764+0000","last_active":"2024-09-20T18:50:38.802835+0000","last_peered":"2024-09-20T18:50:38.802835+0000","last_clean":"2024-09-20T18:50:38.802835+0000","last_became_active":"2024-09-20T18:49:45.941448+0000","last_became_peered":"2024-09-20T18:49:45.941448+0000","last_unstale":"2024-09-20T18:50:38.802835+0000","last_undegraded":"2024-09-20T18:50:38.802835+0000","last_fullsized":"2024-09-20T18:50:38.802835+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:45.120244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":50,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-20T18:50:40.563722+0000","last_change":"2024-09-20T18:49:49.957845+0000","last_active":"2024-09-20T18:50:40.563722+0000","last_peered":"2024-09-20T18:50:40.563722+0000","last_clean":"2024-09-20T18:50:40.563722+0000","last_became_active":"2024-09-20T18:49:49.957558+0000","last_became_peered":"2024-09-20T18:49:49.957558+0000","last_unstale":"2024-09-20T18:50:40.563722+0000","last_undegraded":"2024-09-20T18:50:40.563722+0000","last_fullsized":"2024-09-20T18:50:40.563722+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:14.726319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":43,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.802044+0000","last_change":"2024-09-20T18:49:51.971132+0000","last_active":"2024-09-20T18:50:38.802044+0000","last_peered":"2024-09-20T18:50:38.802044+0000","last_clean":"2024-09-20T18:50:38.802044+0000","last_became_active":"2024-09-20T18:49:51.970798+0000","last_became_peered":"2024-09-20T18:49:51.970798+0000","last_unstale":"2024-09-20T18:50:38.802044+0000","last_undegraded":"2024-09-20T18:50:38.802044+0000","last_fullsized":"2024-09-20T18:50:38.802044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:09.540547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":76,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-20T18:50:40.562947+0000","last_change":"2024-09-20T18:49:47.949428+0000","last_active":"2024-09-20T18:50:40.562947+0000","last_peered":"2024-09-20T18:50:40.562947+0000","last_clean":"2024-09-20T18:50:40.562947+0000","last_became_active":"2024-09-20T18:49:47.949116+0000","last_became_peered":"2024-09-20T18:49:47.949116+0000","last_unstale":"2024-09-20T18:50:40.562947+0000","last_undegraded":"2024-09-20T18:50:40.562947+0000","last_fullsized":"2024-09-20T18:50:40.562947+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:11.352253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":55,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.803910+0000","last_change":"2024-09-20T18:49:45.945147+0000","last_active":"2024-09-20T18:50:38.803910+0000","last_peered":"2024-09-20T18:50:38.803910+0000","last_clean":"2024-09-20T18:50:38.803910+0000","last_became_active":"2024-09-20T18:49:45.944858+0000","last_became_peered":"2024-09-20T18:49:45.944858+0000","last_unstale":"2024-09-20T18:50:38.803910+0000","last_undegraded":"2024-09-20T18:50:38.803910+0000","last_fullsized":"2024-09-20T18:50:38.803910+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:53.147843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":50,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-20T18:50:40.562565+0000","last_change":"2024-09-20T18:49:49.955243+0000","last_active":"2024-09-20T18:50:40.562565+0000","last_peered":"2024-09-20T18:50:40.562565+0000","last_clean":"2024-09-20T18:50:40.562565+0000","last_became_active":"2024-09-20T18:49:49.954944+0000","last_became_peered":"2024-09-20T18:49:49.954944+0000","last_unstale":"2024-09-20T18:50:40.562565+0000","last_undegraded":"2024-09-20T18:50:40.562565+0000","last_fullsized":"2024-09-20T18:50:40.562565+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:00.542377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":43,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.802267+0000","last_change":"2024-09-20T18:49:51.973284+0000","last_active":"2024-09-20T18:50:38.802267+0000","last_peered":"2024-09-20T18:50:38.802267+0000","last_clean":"2024-09-20T18:50:38.802267+0000","last_became_active":"2024-09-20T18:49:51.972970+0000","last_became_peered":"2024-09-20T18:49:51.972970+0000","last_unstale":"2024-09-20T18:50:38.802267+0000","last_undegraded":"2024-09-20T18:50:38.802267+0000","last_fullsized":"2024-09-20T18:50:38.802267+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:55.977221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":76,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.802763+0000","last_change":"2024-09-20T18:49:48.500174+0000","last_active":"2024-09-20T18:50:38.802763+0000","last_peered":"2024-09-20T18:50:38.802763+0000","last_clean":"2024-09-20T18:50:38.802763+0000","last_became_active":"2024-09-20T18:49:48.499847+0000","last_became_peered":"2024-09-20T18:49:48.499847+0000","last_unstale":"2024-09-20T18:50:38.802763+0000","last_undegraded":"2024-09-20T18:50:38.802763+0000","last_fullsized":"2024-09-20T18:50:38.802763+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:15.572682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":67,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.802322+0000","last_change":"2024-09-20T18:49:45.945391+0000","last_active":"2024-09-20T18:50:38.802322+0000","last_peered":"2024-09-20T18:50:38.802322+0000","last_clean":"2024-09-20T18:50:38.802322+0000","last_became_active":"2024-09-20T18:49:45.944256+0000","last_became_peered":"2024-09-20T18:49:45.944256+0000","last_unstale":"2024-09-20T18:50:38.802322+0000","last_undegraded":"2024-09-20T18:50:38.802322+0000","last_fullsized":"2024-09-20T18:50:38.802322+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:47.823570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":66,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.802307+0000","last_change":"2024-09-20T18:49:49.958686+0000","last_active":"2024-09-20T18:50:38.802307+0000","last_peered":"2024-09-20T18:50:38.802307+0000","last_clean":"2024-09-20T18:50:38.802307+0000","last_became_active":"2024-09-20T18:49:49.958411+0000","last_became_peered":"2024-09-20T18:49:49.958411+0000","last_unstale":"2024-09-20T18:50:38.802307+0000","last_undegraded":"2024-09-20T18:50:38.802307+0000","last_fullsized":"2024-09-20T18:50:38.802307+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:57.283019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":43,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.803198+0000","last_change":"2024-09-20T18:49:52.139994+0000","last_active":"2024-09-20T18:50:38.803198+0000","last_peered":"2024-09-20T18:50:38.803198+0000","last_clean":"2024-09-20T18:50:38.803198+0000","last_became_active":"2024-09-20T18:49:52.139685+0000","last_became_peered":"2024-09-20T18:49:52.139685+0000","last_unstale":"2024-09-20T18:50:38.803198+0000","last_undegraded":"2024-09-20T18:50:38.803198+0000","last_fullsized":"2024-09-20T18:50:38.803198+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:53.034093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":55,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.801845+0000","last_change":"2024-09-20T18:49:45.940360+0000","last_active":"2024-09-20T18:50:38.801845+0000","last_peered":"2024-09-20T18:50:38.801845+0000","last_clean":"2024-09-20T18:50:38.801845+0000","last_became_active":"2024-09-20T18:49:45.940060+0000","last_became_peered":"2024-09-20T18:49:45.940060+0000","last_unstale":"2024-09-20T18:50:38.801845+0000","last_undegraded":"2024-09-20T18:50:38.801845+0000","last_fullsized":"2024-09-20T18:50:38.801845+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:59.071733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":60,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.803019+0000","last_change":"2024-09-20T18:49:48.501010+0000","last_active":"2024-09-20T18:50:38.803019+0000","last_peered":"2024-09-20T18:50:38.803019+0000","last_clean":"2024-09-20T18:50:38.803019+0000","last_became_active":"2024-09-20T18:49:48.500662+0000","last_became_peered":"2024-09-20T18:49:48.500662+0000","last_unstale":"2024-09-20T18:50:38.803019+0000","last_undegraded":"2024-09-20T18:50:38.803019+0000","last_fullsized":"2024-09-20T18:50:38.803019+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:01.787506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":47,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.804238+0000","last_change":"2024-09-20T18:49:49.957092+0000","last_active":"2024-09-20T18:50:38.804238+0000","last_peered":"2024-09-20T18:50:38.804238+0000","last_clean":"2024-09-20T18:50:38.804238+0000","last_became_active":"2024-09-20T18:49:49.956749+0000","last_became_peered":"2024-09-20T18:49:49.956749+0000","last_unstale":"2024-09-20T18:50:38.804238+0000","last_undegraded":"2024-09-20T18:50:38.804238+0000","last_fullsized":"2024-09-20T18:50:38.804238+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:53.840760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":43,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.802389+0000","last_change":"2024-09-20T18:49:52.142789+0000","last_active":"2024-09-20T18:50:38.802389+0000","last_peered":"2024-09-20T18:50:38.802389+0000","last_clean":"2024-09-20T18:50:38.802389+0000","last_became_active":"2024-09-20T18:49:52.142291+0000","last_became_peered":"2024-09-20T18:49:52.142291+0000","last_unstale":"2024-09-20T18:50:38.802389+0000","last_undegraded":"2024-09-20T18:50:38.802389+0000","last_fullsized":"2024-09-20T18:50:38.802389+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:58.299262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":67,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.583819+0000","last_change":"2024-09-20T18:49:48.501376+0000","last_active":"2024-09-20T18:50:38.583819+0000","last_peered":"2024-09-20T18:50:38.583819+0000","last_clean":"2024-09-20T18:50:38.583819+0000","last_became_active":"2024-09-20T18:49:48.500803+0000","last_became_peered":"2024-09-20T18:49:48.500803+0000","last_unstale":"2024-09-20T18:50:38.583819+0000","last_undegraded":"2024-09-20T18:50:38.583819+0000","last_fullsized":"2024-09-20T18:50:38.583819+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:02.881066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":55,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.802556+0000","last_change":"2024-09-20T18:49:45.947791+0000","last_active":"2024-09-20T18:50:38.802556+0000","last_peered":"2024-09-20T18:50:38.802556+0000","last_clean":"2024-09-20T18:50:38.802556+0000","last_became_active":"2024-09-20T18:49:45.946600+0000","last_became_peered":"2024-09-20T18:49:45.946600+0000","last_unstale":"2024-09-20T18:50:38.802556+0000","last_undegraded":"2024-09-20T18:50:38.802556+0000","last_fullsized":"2024-09-20T18:50:38.802556+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:13.160428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":47,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.802827+0000","last_change":"2024-09-20T18:49:49.962290+0000","last_active":"2024-09-20T18:50:38.802827+0000","last_peered":"2024-09-20T18:50:38.802827+0000","last_clean":"2024-09-20T18:50:38.802827+0000","last_became_active":"2024-09-20T18:49:49.961947+0000","last_became_peered":"2024-09-20T18:49:49.961947+0000","last_unstale":"2024-09-20T18:50:38.802827+0000","last_undegraded":"2024-09-20T18:50:38.802827+0000","last_fullsized":"2024-09-20T18:50:38.802827+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:54.834258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"66'1","reported_seq":47,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-20T18:50:40.563144+0000","last_change":"2024-09-20T18:49:51.965397+0000","last_active":"2024-09-20T18:50:40.563144+0000","last_peered":"2024-09-20T18:50:40.563144+0000","last_clean":"2024-09-20T18:50:40.563144+0000","last_became_active":"2024-09-20T18:49:51.965121+0000","last_became_peered":"2024-09-20T18:49:51.965121+0000","last_unstale":"2024-09-20T18:50:40.563144+0000","last_undegraded":"2024-09-20T18:50:40.563144+0000","last_fullsized":"2024-09-20T18:50:40.563144+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:57.419566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":67,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.804095+0000","last_change":"2024-09-20T18:49:48.501359+0000","last_active":"2024-09-20T18:50:38.804095+0000","last_peered":"2024-09-20T18:50:38.804095+0000","last_clean":"2024-09-20T18:50:38.804095+0000","last_became_active":"2024-09-20T18:49:48.501011+0000","last_became_peered":"2024-09-20T18:49:48.501011+0000","last_unstale":"2024-09-20T18:50:38.804095+0000","last_undegraded":"2024-09-20T18:50:38.804095+0000","last_fullsized":"2024-09-20T18:50:38.804095+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:50.431705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":55,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.583508+0000","last_change":"2024-09-20T18:49:45.938820+0000","last_active":"2024-09-20T18:50:38.583508+0000","last_peered":"2024-09-20T18:50:38.583508+0000","last_clean":"2024-09-20T18:50:38.583508+0000","last_became_active":"2024-09-20T18:49:45.938511+0000","last_became_peered":"2024-09-20T18:49:45.938511+0000","last_unstale":"2024-09-20T18:50:38.583508+0000","last_undegraded":"2024-09-20T18:50:38.583508+0000","last_fullsized":"2024-09-20T18:50:38.583508+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:02.383015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":66,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.802419+0000","last_change":"2024-09-20T18:49:49.960889+0000","last_active":"2024-09-20T18:50:38.802419+0000","last_peered":"2024-09-20T18:50:38.802419+0000","last_clean":"2024-09-20T18:50:38.802419+0000","last_became_active":"2024-09-20T18:49:49.960550+0000","last_became_peered":"2024-09-20T18:49:49.960550+0000","last_unstale":"2024-09-20T18:50:38.802419+0000","last_undegraded":"2024-09-20T18:50:38.802419+0000","last_fullsized":"2024-09-20T18:50:38.802419+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:58.739350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-20T18:50:40.563825+0000","last_change":"2024-09-20T18:49:51.966530+0000","last_active":"2024-09-20T18:50:40.563825+0000","last_peered":"2024-09-20T18:50:40.563825+0000","last_clean":"2024-09-20T18:50:40.563825+0000","last_became_active":"2024-09-20T18:49:51.966254+0000","last_became_peered":"2024-09-20T18:49:51.966254+0000","last_unstale":"2024-09-20T18:50:40.563825+0000","last_undegraded":"2024-09-20T18:50:40.563825+0000","last_fullsized":"2024-09-20T18:50:40.563825+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:14.193792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":85,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.802561+0000","last_change":"2024-09-20T18:49:59.003234+0000","last_active":"2024-09-20T18:50:38.802561+0000","last_peered":"2024-09-20T18:50:38.802561+0000","last_clean":"2024-09-20T18:50:38.802561+0000","last_became_active":"2024-09-20T18:49:58.997494+0000","last_became_peered":"2024-09-20T18:49:58.997494+0000","last_unstale":"2024-09-20T18:50:38.802561+0000","last_undegraded":"2024-09-20T18:50:38.802561+0000","last_fullsized":"2024-09-20T18:50:38.802561+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:05.235529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":55,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.803450+0000","last_change":"2024-09-20T18:49:45.949895+0000","last_active":"2024-09-20T18:50:38.803450+0000","last_peered":"2024-09-20T18:50:38.803450+0000","last_clean":"2024-09-20T18:50:38.803450+0000","last_became_active":"2024-09-20T18:49:45.949492+0000","last_became_peered":"2024-09-20T18:49:45.949492+0000","last_unstale":"2024-09-20T18:50:38.803450+0000","last_undegraded":"2024-09-20T18:50:38.803450+0000","last_fullsized":"2024-09-20T18:50:38.803450+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:02.958108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":66,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.803732+0000","last_change":"2024-09-20T18:49:49.962519+0000","last_active":"2024-09-20T18:50:38.803732+0000","last_peered":"2024-09-20T18:50:38.803732+0000","last_clean":"2024-09-20T18:50:38.803732+0000","last_became_active":"2024-09-20T18:49:49.962243+0000","last_became_peered":"2024-09-20T18:49:49.962243+0000","last_unstale":"2024-09-20T18:50:38.803732+0000","last_undegraded":"2024-09-20T18:50:38.803732+0000","last_fullsized":"2024-09-20T18:50:38.803732+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:16.990588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":43,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.583401+0000","last_change":"2024-09-20T18:49:51.969358+0000","last_active":"2024-09-20T18:50:38.583401+0000","last_peered":"2024-09-20T18:50:38.583401+0000","last_clean":"2024-09-20T18:50:38.583401+0000","last_became_active":"2024-09-20T18:49:51.969093+0000","last_became_peered":"2024-09-20T18:49:51.969093+0000","last_unstale":"2024-09-20T18:50:38.583401+0000","last_undegraded":"2024-09-20T18:50:38.583401+0000","last_fullsized":"2024-09-20T18:50:38.583401+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:17.459087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":72,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-20T18:50:40.562968+0000","last_change":"2024-09-20T18:49:47.950488+0000","last_active":"2024-09-20T18:50:40.562968+0000","last_peered":"2024-09-20T18:50:40.562968+0000","last_clean":"2024-09-20T18:50:40.562968+0000","last_became_active":"2024-09-20T18:49:47.949703+0000","last_became_peered":"2024-09-20T18:49:47.949703+0000","last_unstale":"2024-09-20T18:50:40.562968+0000","last_undegraded":"2024-09-20T18:50:40.562968+0000","last_fullsized":"2024-09-20T18:50:40.562968+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:53.129485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":57,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-20T18:50:40.563474+0000","last_change":"2024-09-20T18:49:45.941578+0000","last_active":"2024-09-20T18:50:40.563474+0000","last_peered":"2024-09-20T18:50:40.563474+0000","last_clean":"2024-09-20T18:50:40.563474+0000","last_became_active":"2024-09-20T18:49:45.939949+0000","last_became_peered":"2024-09-20T18:49:45.939949+0000","last_unstale":"2024-09-20T18:50:40.563474+0000","last_undegraded":"2024-09-20T18:50:40.563474+0000","last_fullsized":"2024-09-20T18:50:40.563474+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:53.450364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":49,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-20T18:50:40.564352+0000","last_change":"2024-09-20T18:49:49.957358+0000","last_active":"2024-09-20T18:50:40.564352+0000","last_peered":"2024-09-20T18:50:40.564352+0000","last_clean":"2024-09-20T18:50:40.564352+0000","last_became_active":"2024-09-20T18:49:49.957067+0000","last_became_peered":"2024-09-20T18:49:49.957067+0000","last_unstale":"2024-09-20T18:50:40.564352+0000","last_undegraded":"2024-09-20T18:50:40.564352+0000","last_fullsized":"2024-09-20T18:50:40.564352+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:54.100322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":43,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.801802+0000","last_change":"2024-09-20T18:49:51.967168+0000","last_active":"2024-09-20T18:50:38.801802+0000","last_peered":"2024-09-20T18:50:38.801802+0000","last_clean":"2024-09-20T18:50:38.801802+0000","last_became_active":"2024-09-20T18:49:51.966915+0000","last_became_peered":"2024-09-20T18:49:51.966915+0000","last_unstale":"2024-09-20T18:50:38.801802+0000","last_undegraded":"2024-09-20T18:50:38.801802+0000","last_fullsized":"2024-09-20T18:50:38.801802+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:04.536536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":69,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-20T18:50:40.564621+0000","last_change":"2024-09-20T18:49:47.953603+0000","last_active":"2024-09-20T18:50:40.564621+0000","last_peered":"2024-09-20T18:50:40.564621+0000","last_clean":"2024-09-20T18:50:40.564621+0000","last_became_active":"2024-09-20T18:49:47.953310+0000","last_became_peered":"2024-09-20T18:49:47.953310+0000","last_unstale":"2024-09-20T18:50:40.564621+0000","last_undegraded":"2024-09-20T18:50:40.564621+0000","last_fullsized":"2024-09-20T18:50:40.564621+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:06.158344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":55,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.802651+0000","last_change":"2024-09-20T18:49:45.944827+0000","last_active":"2024-09-20T18:50:38.802651+0000","last_peered":"2024-09-20T18:50:38.802651+0000","last_clean":"2024-09-20T18:50:38.802651+0000","last_became_active":"2024-09-20T18:49:45.944357+0000","last_became_peered":"2024-09-20T18:49:45.944357+0000","last_unstale":"2024-09-20T18:50:38.802651+0000","last_undegraded":"2024-09-20T18:50:38.802651+0000","last_fullsized":"2024-09-20T18:50:38.802651+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:14.013129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":47,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.802290+0000","last_change":"2024-09-20T18:49:49.962794+0000","last_active":"2024-09-20T18:50:38.802290+0000","last_peered":"2024-09-20T18:50:38.802290+0000","last_clean":"2024-09-20T18:50:38.802290+0000","last_became_active":"2024-09-20T18:49:49.962523+0000","last_became_peered":"2024-09-20T18:49:49.962523+0000","last_unstale":"2024-09-20T18:50:38.802290+0000","last_undegraded":"2024-09-20T18:50:38.802290+0000","last_fullsized":"2024-09-20T18:50:38.802290+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:06.567112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":43,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.582299+0000","last_change":"2024-09-20T18:49:51.967581+0000","last_active":"2024-09-20T18:50:38.582299+0000","last_peered":"2024-09-20T18:50:38.582299+0000","last_clean":"2024-09-20T18:50:38.582299+0000","last_became_active":"2024-09-20T18:49:51.967313+0000","last_became_peered":"2024-09-20T18:49:51.967313+0000","last_unstale":"2024-09-20T18:50:38.582299+0000","last_undegraded":"2024-09-20T18:50:38.582299+0000","last_fullsized":"2024-09-20T18:50:38.582299+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:05.774315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":70,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.801903+0000","last_change":"2024-09-20T18:49:48.499421+0000","last_active":"2024-09-20T18:50:38.801903+0000","last_peered":"2024-09-20T18:50:38.801903+0000","last_clean":"2024-09-20T18:50:38.801903+0000","last_became_active":"2024-09-20T18:49:48.499111+0000","last_became_peered":"2024-09-20T18:49:48.499111+0000","last_unstale":"2024-09-20T18:50:38.801903+0000","last_undegraded":"2024-09-20T18:50:38.801903+0000","last_fullsized":"2024-09-20T18:50:38.801903+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:57.119627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":55,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.804284+0000","last_change":"2024-09-20T18:49:45.942406+0000","last_active":"2024-09-20T18:50:38.804284+0000","last_peered":"2024-09-20T18:50:38.804284+0000","last_clean":"2024-09-20T18:50:38.804284+0000","last_became_active":"2024-09-20T18:49:45.942105+0000","last_became_peered":"2024-09-20T18:49:45.942105+0000","last_unstale":"2024-09-20T18:50:38.804284+0000","last_undegraded":"2024-09-20T18:50:38.804284+0000","last_fullsized":"2024-09-20T18:50:38.804284+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:56.309962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":47,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.802963+0000","last_change":"2024-09-20T18:49:49.959361+0000","last_active":"2024-09-20T18:50:38.802963+0000","last_peered":"2024-09-20T18:50:38.802963+0000","last_clean":"2024-09-20T18:50:38.802963+0000","last_became_active":"2024-09-20T18:49:49.958159+0000","last_became_peered":"2024-09-20T18:49:49.958159+0000","last_unstale":"2024-09-20T18:50:38.802963+0000","last_undegraded":"2024-09-20T18:50:38.802963+0000","last_fullsized":"2024-09-20T18:50:38.802963+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:12.385752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":43,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.802379+0000","last_change":"2024-09-20T18:49:52.140525+0000","last_active":"2024-09-20T18:50:38.802379+0000","last_peered":"2024-09-20T18:50:38.802379+0000","last_clean":"2024-09-20T18:50:38.802379+0000","last_became_active":"2024-09-20T18:49:52.139978+0000","last_became_peered":"2024-09-20T18:49:52.139978+0000","last_unstale":"2024-09-20T18:50:38.802379+0000","last_undegraded":"2024-09-20T18:50:38.802379+0000","last_fullsized":"2024-09-20T18:50:38.802379+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:19.358011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":57,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.801383+0000","last_change":"2024-09-20T18:49:47.955120+0000","last_active":"2024-09-20T18:50:38.801383+0000","last_peered":"2024-09-20T18:50:38.801383+0000","last_clean":"2024-09-20T18:50:38.801383+0000","last_became_active":"2024-09-20T18:49:47.954777+0000","last_became_peered":"2024-09-20T18:49:47.954777+0000","last_unstale":"2024-09-20T18:50:38.801383+0000","last_undegraded":"2024-09-20T18:50:38.801383+0000","last_fullsized":"2024-09-20T18:50:38.801383+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:04.481222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":55,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.583697+0000","last_change":"2024-09-20T18:49:45.941248+0000","last_active":"2024-09-20T18:50:38.583697+0000","last_peered":"2024-09-20T18:50:38.583697+0000","last_clean":"2024-09-20T18:50:38.583697+0000","last_became_active":"2024-09-20T18:49:45.940988+0000","last_became_peered":"2024-09-20T18:49:45.940988+0000","last_unstale":"2024-09-20T18:50:38.583697+0000","last_undegraded":"2024-09-20T18:50:38.583697+0000","last_fullsized":"2024-09-20T18:50:38.583697+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:58.598619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":47,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.802296+0000","last_change":"2024-09-20T18:49:49.954779+0000","last_active":"2024-09-20T18:50:38.802296+0000","last_peered":"2024-09-20T18:50:38.802296+0000","last_clean":"2024-09-20T18:50:38.802296+0000","last_became_active":"2024-09-20T18:49:49.954502+0000","last_became_peered":"2024-09-20T18:49:49.954502+0000","last_unstale":"2024-09-20T18:50:38.802296+0000","last_undegraded":"2024-09-20T18:50:38.802296+0000","last_fullsized":"2024-09-20T18:50:38.802296+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:15.141742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":43,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.802668+0000","last_change":"2024-09-20T18:49:51.968968+0000","last_active":"2024-09-20T18:50:38.802668+0000","last_peered":"2024-09-20T18:50:38.802668+0000","last_clean":"2024-09-20T18:50:38.802668+0000","last_became_active":"2024-09-20T18:49:51.968662+0000","last_became_peered":"2024-09-20T18:49:51.968662+0000","last_unstale":"2024-09-20T18:50:38.802668+0000","last_undegraded":"2024-09-20T18:50:38.802668+0000","last_fullsized":"2024-09-20T18:50:38.802668+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:02.092669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-20T18:50:40.564281+0000","last_change":"2024-09-20T18:49:51.968476+0000","last_active":"2024-09-20T18:50:40.564281+0000","last_peered":"2024-09-20T18:50:40.564281+0000","last_clean":"2024-09-20T18:50:40.564281+0000","last_became_active":"2024-09-20T18:49:51.968216+0000","last_became_peered":"2024-09-20T18:49:51.968216+0000","last_unstale":"2024-09-20T18:50:40.564281+0000","last_undegraded":"2024-09-20T18:50:40.564281+0000","last_fullsized":"2024-09-20T18:50:40.564281+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:11.930622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":55,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.801608+0000","last_change":"2024-09-20T18:49:45.938413+0000","last_active":"2024-09-20T18:50:38.801608+0000","last_peered":"2024-09-20T18:50:38.801608+0000","last_clean":"2024-09-20T18:50:38.801608+0000","last_became_active":"2024-09-20T18:49:45.937992+0000","last_became_peered":"2024-09-20T18:49:45.937992+0000","last_unstale":"2024-09-20T18:50:38.801608+0000","last_undegraded":"2024-09-20T18:50:38.801608+0000","last_fullsized":"2024-09-20T18:50:38.801608+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:12.596336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":70,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.803491+0000","last_change":"2024-09-20T18:49:47.950306+0000","last_active":"2024-09-20T18:50:38.803491+0000","last_peered":"2024-09-20T18:50:38.803491+0000","last_clean":"2024-09-20T18:50:38.803491+0000","last_became_active":"2024-09-20T18:49:47.949582+0000","last_became_peered":"2024-09-20T18:49:47.949582+0000","last_unstale":"2024-09-20T18:50:38.803491+0000","last_undegraded":"2024-09-20T18:50:38.803491+0000","last_fullsized":"2024-09-20T18:50:38.803491+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:03.753158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":47,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-20T18:50:38.583569+0000","last_change":"2024-09-20T18:49:49.957874+0000","last_active":"2024-09-20T18:50:38.583569+0000","last_peered":"2024-09-20T18:50:38.583569+0000","last_clean":"2024-09-20T18:50:38.583569+0000","last_became_active":"2024-09-20T18:49:49.957582+0000","last_became_peered":"2024-09-20T18:49:49.957582+0000","last_unstale":"2024-09-20T18:50:38.583569+0000","last_undegraded":"2024-09-20T18:50:38.583569+0000","last_fullsized":"2024-09-20T18:50:38.583569+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:07.454286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":50,"num_read_kb":45,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266703,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32588,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712564,"statfs":{"total":95995035648,"available":95961665536,"internally_reserved":0,"allocated":1433600,"data_stored":934486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528342,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28408,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716744,"statfs":{"total":95995035648,"available":95965945856,"internally_reserved":0,"allocated":1413120,"data_stored":933824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757275,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31992,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713160,"statfs":{"total":95995035648,"available":95962275840,"internally_reserved":0,"allocated":823296,"data_stored":343815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986208,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":856064,"data_stored":345067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182439,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32020,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713132,"statfs":{"total":95995035648,"available":95962247168,"internally_reserved":0,"allocated":851968,"data_stored":343911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411372,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32008,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713144,"statfs":{"total":95995035648,"available":95962259456,"internally_reserved":0,"allocated":839680,"data_stored":350853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574898,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":876544,"data_stored":345578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738424,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32588,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712564,"statfs":{"total":95995035648,"available":95961665536,"internally_reserved":0,"allocated":1433600,"data_stored":934182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T18:50:42.102 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-20T18:50:42.102 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd dump 2024-09-20T18:50:42.349 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:50:43.018 INFO:teuthology.orchestra.run.smithi083.stdout:epoch 82 2024-09-20T18:50:43.018 INFO:teuthology.orchestra.run.smithi083.stdout:fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 2024-09-20T18:50:43.018 INFO:teuthology.orchestra.run.smithi083.stdout:created 2024-09-20T18:44:10.399800+0000 2024-09-20T18:50:43.018 INFO:teuthology.orchestra.run.smithi083.stdout:modified 2024-09-20T18:50:40.555466+0000 2024-09-20T18:50:43.018 INFO:teuthology.orchestra.run.smithi083.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-20T18:50:43.018 INFO:teuthology.orchestra.run.smithi083.stdout:crush_version 18 2024-09-20T18:50:43.018 INFO:teuthology.orchestra.run.smithi083.stdout:full_ratio 0.95 2024-09-20T18:50:43.018 INFO:teuthology.orchestra.run.smithi083.stdout:backfillfull_ratio 0.9 2024-09-20T18:50:43.019 INFO:teuthology.orchestra.run.smithi083.stdout:nearfull_ratio 0.85 2024-09-20T18:50:43.019 INFO:teuthology.orchestra.run.smithi083.stdout:require_min_compat_client luminous 2024-09-20T18:50:43.019 INFO:teuthology.orchestra.run.smithi083.stdout:min_compat_client luminous 2024-09-20T18:50:43.019 INFO:teuthology.orchestra.run.smithi083.stdout:require_osd_release squid 2024-09-20T18:50:43.019 INFO:teuthology.orchestra.run.smithi083.stdout:stretch_mode_enabled false 2024-09-20T18:50:43.019 INFO:teuthology.orchestra.run.smithi083.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 7.89 2024-09-20T18:50:43.019 INFO:teuthology.orchestra.run.smithi083.stdout:pool 2 'datapool' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 3 pgp_num 3 autoscale_mode off last_change 61 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 2.65 2024-09-20T18:50:43.019 INFO:teuthology.orchestra.run.smithi083.stdout:pool 3 '.rgw.root' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 60 flags hashpspool stripe_width 0 application rgw read_balance_score 1.50 2024-09-20T18:50:43.019 INFO:teuthology.orchestra.run.smithi083.stdout:pool 4 'default.rgw.log' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 62 flags hashpspool stripe_width 0 application rgw read_balance_score 2.25 2024-09-20T18:50:43.019 INFO:teuthology.orchestra.run.smithi083.stdout:pool 5 'default.rgw.control' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 64 flags hashpspool stripe_width 0 application rgw read_balance_score 1.25 2024-09-20T18:50:43.019 INFO:teuthology.orchestra.run.smithi083.stdout:pool 6 'default.rgw.meta' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 66 flags hashpspool stripe_width 0 pg_autoscale_bias 4 application rgw read_balance_score 1.75 2024-09-20T18:50:43.019 INFO:teuthology.orchestra.run.smithi083.stdout:max_osd 8 2024-09-20T18:50:43.019 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0 up in weight 1 up_from 8 up_thru 64 down_at 0 last_clean_interval [0,0) v2:172.21.15.83:6801/2574084049 v2:172.21.15.83:6802/2574084049 exists,up c714e1ec-f0c3-4054-9792-61b84e53eaa8 2024-09-20T18:50:43.020 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1 up in weight 1 up_from 13 up_thru 64 down_at 0 last_clean_interval [0,0) v2:172.21.15.83:6805/4213285798 v2:172.21.15.83:6806/4213285798 exists,up 9dc0f2a8-bafa-4a42-88f5-0301dd58f78d 2024-09-20T18:50:43.020 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2 up in weight 1 up_from 18 up_thru 68 down_at 0 last_clean_interval [0,0) v2:172.21.15.83:6809/805947571 v2:172.21.15.83:6810/805947571 exists,up 448ab692-ec41-44dc-b2d5-dbefd255b3fd 2024-09-20T18:50:43.020 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3 up in weight 1 up_from 25 up_thru 68 down_at 0 last_clean_interval [0,0) v2:172.21.15.83:6813/987237318 v2:172.21.15.83:6814/987237318 exists,up 7cda5e50-e22e-4d4a-823b-53c8f5061193 2024-09-20T18:50:43.020 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4 up in weight 1 up_from 31 up_thru 68 down_at 0 last_clean_interval [0,0) v2:172.21.15.142:6800/2096684065 v2:172.21.15.142:6801/2096684065 exists,up 3d62654d-d31d-4310-9934-49e694dd17d9 2024-09-20T18:50:43.020 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5 up in weight 1 up_from 38 up_thru 64 down_at 0 last_clean_interval [0,0) v2:172.21.15.142:6804/2500466067 v2:172.21.15.142:6805/2500466067 exists,up b3e4e21b-639f-4d25-8bbf-ea8922a9222a 2024-09-20T18:50:43.020 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6 up in weight 1 up_from 45 up_thru 62 down_at 0 last_clean_interval [0,0) v2:172.21.15.142:6808/1688186226 v2:172.21.15.142:6809/1688186226 exists,up c5db7107-ccb4-4ae2-9e63-f444897942e3 2024-09-20T18:50:43.020 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7 up in weight 1 up_from 53 up_thru 64 down_at 0 last_clean_interval [0,0) v2:172.21.15.142:6812/3679770984 v2:172.21.15.142:6813/3679770984 exists,up 07d16806-e2c8-4acb-b284-d75701e61622 2024-09-20T18:50:43.020 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 4.f [1,2] 2024-09-20T18:50:43.020 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 4.14 [1,5] 2024-09-20T18:50:43.020 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 4.1b [1,2] 2024-09-20T18:50:43.020 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:0/3474253603 expires 2024-09-21T18:44:32.434180+0000 2024-09-20T18:50:43.020 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:6800/3525856829 expires 2024-09-21T18:44:32.434180+0000 2024-09-20T18:50:43.020 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:0/220746284 expires 2024-09-21T18:44:32.434180+0000 2024-09-20T18:50:43.021 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:0/3875445393 expires 2024-09-21T18:44:56.985620+0000 2024-09-20T18:50:43.021 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:0/3009092807 expires 2024-09-21T18:44:56.985620+0000 2024-09-20T18:50:43.021 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:0/4267132131 expires 2024-09-21T18:44:56.985620+0000 2024-09-20T18:50:43.021 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:0/3684382972 expires 2024-09-21T18:50:14.729526+0000 2024-09-20T18:50:43.021 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:0/3132933178 expires 2024-09-21T18:50:14.729526+0000 2024-09-20T18:50:43.021 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:6800/1007846146 expires 2024-09-21T18:50:14.729526+0000 2024-09-20T18:50:43.021 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:6800/2985596191 expires 2024-09-21T18:44:56.985620+0000 2024-09-20T18:50:43.021 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:0/2410768464 expires 2024-09-21T18:50:14.729526+0000 2024-09-20T18:50:43.021 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:0/1868694097 expires 2024-09-21T18:50:14.729526+0000 2024-09-20T18:50:43.021 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:0/3777331122 expires 2024-09-21T18:44:32.434180+0000 2024-09-20T18:50:43.021 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:0/2161728215 expires 2024-09-21T18:50:14.729526+0000 2024-09-20T18:50:43.699 INFO:tasks.cephadm.ceph_manager.ceph:['osd.0 up in weight 1 up_from 8 up_thru 64 down_at 0 last_clean_interval [0,0) v2:172.21.15.83:6801/2574084049 v2:172.21.15.83:6802/2574084049 exists,up c714e1ec-f0c3-4054-9792-61b84e53eaa8', 'osd.1 up in weight 1 up_from 13 up_thru 64 down_at 0 last_clean_interval [0,0) v2:172.21.15.83:6805/4213285798 v2:172.21.15.83:6806/4213285798 exists,up 9dc0f2a8-bafa-4a42-88f5-0301dd58f78d', 'osd.2 up in weight 1 up_from 18 up_thru 68 down_at 0 last_clean_interval [0,0) v2:172.21.15.83:6809/805947571 v2:172.21.15.83:6810/805947571 exists,up 448ab692-ec41-44dc-b2d5-dbefd255b3fd', 'osd.3 up in weight 1 up_from 25 up_thru 68 down_at 0 last_clean_interval [0,0) v2:172.21.15.83:6813/987237318 v2:172.21.15.83:6814/987237318 exists,up 7cda5e50-e22e-4d4a-823b-53c8f5061193', 'osd.4 up in weight 1 up_from 31 up_thru 68 down_at 0 last_clean_interval [0,0) v2:172.21.15.142:6800/2096684065 v2:172.21.15.142:6801/2096684065 exists,up 3d62654d-d31d-4310-9934-49e694dd17d9', 'osd.5 up in weight 1 up_from 38 up_thru 64 down_at 0 last_clean_interval [0,0) v2:172.21.15.142:6804/2500466067 v2:172.21.15.142:6805/2500466067 exists,up b3e4e21b-639f-4d25-8bbf-ea8922a9222a', 'osd.6 up in weight 1 up_from 45 up_thru 62 down_at 0 last_clean_interval [0,0) v2:172.21.15.142:6808/1688186226 v2:172.21.15.142:6809/1688186226 exists,up c5db7107-ccb4-4ae2-9e63-f444897942e3', 'osd.7 up in weight 1 up_from 53 up_thru 64 down_at 0 last_clean_interval [0,0) v2:172.21.15.142:6812/3679770984 v2:172.21.15.142:6813/3679770984 exists,up 07d16806-e2c8-4acb-b284-d75701e61622'] 2024-09-20T18:50:43.700 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon mon.a config show 2024-09-20T18:50:43.947 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:50:44.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:43 smithi083 ceph-mon[29421]: pgmap v25: 132 pgs: 132 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T18:50:44.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:43 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1789378706' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-20T18:50:44.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:43 smithi083 ceph-mon[36260]: pgmap v25: 132 pgs: 132 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T18:50:44.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:43 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1789378706' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-20T18:50:44.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:43 smithi142 ceph-mon[32508]: pgmap v25: 132 pgs: 132 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T18:50:44.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:43 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1789378706' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-20T18:50:44.399 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T18:50:44.399 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mon.a", 2024-09-20T18:50:44.399 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster": "ceph", 2024-09-20T18:50:44.399 INFO:teuthology.orchestra.run.smithi083.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-20T18:50:44.399 INFO:teuthology.orchestra.run.smithi083.stdout: "admin_socket_mode": "", 2024-09-20T18:50:44.399 INFO:teuthology.orchestra.run.smithi083.stdout: "auth_allow_insecure_global_id_reclaim": "false", 2024-09-20T18:50:44.399 INFO:teuthology.orchestra.run.smithi083.stdout: "auth_client_required": "cephx, none", 2024-09-20T18:50:44.399 INFO:teuthology.orchestra.run.smithi083.stdout: "auth_cluster_required": "cephx", 2024-09-20T18:50:44.399 INFO:teuthology.orchestra.run.smithi083.stdout: "auth_debug": "true", 2024-09-20T18:50:44.400 INFO:teuthology.orchestra.run.smithi083.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-20T18:50:44.400 INFO:teuthology.orchestra.run.smithi083.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-20T18:50:44.400 INFO:teuthology.orchestra.run.smithi083.stdout: "auth_service_required": "cephx", 2024-09-20T18:50:44.400 INFO:teuthology.orchestra.run.smithi083.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-20T18:50:44.400 INFO:teuthology.orchestra.run.smithi083.stdout: "auth_supported": "", 2024-09-20T18:50:44.400 INFO:teuthology.orchestra.run.smithi083.stdout: "bdev_aio": "true", 2024-09-20T18:50:44.400 INFO:teuthology.orchestra.run.smithi083.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-20T18:50:44.400 INFO:teuthology.orchestra.run.smithi083.stdout: "bdev_aio_poll_ms": "250", 2024-09-20T18:50:44.400 INFO:teuthology.orchestra.run.smithi083.stdout: "bdev_aio_reap_max": "16", 2024-09-20T18:50:44.400 INFO:teuthology.orchestra.run.smithi083.stdout: "bdev_async_discard_threads": "0", 2024-09-20T18:50:44.400 INFO:teuthology.orchestra.run.smithi083.stdout: "bdev_block_size": "4096", 2024-09-20T18:50:44.400 INFO:teuthology.orchestra.run.smithi083.stdout: "bdev_debug_aio": "false", 2024-09-20T18:50:44.400 INFO:teuthology.orchestra.run.smithi083.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-20T18:50:44.401 INFO:teuthology.orchestra.run.smithi083.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-20T18:50:44.401 INFO:teuthology.orchestra.run.smithi083.stdout: "bdev_debug_inflight_ios": "false", 2024-09-20T18:50:44.401 INFO:teuthology.orchestra.run.smithi083.stdout: "bdev_enable_discard": "false", 2024-09-20T18:50:44.401 INFO:teuthology.orchestra.run.smithi083.stdout: "bdev_flock_retry": "3", 2024-09-20T18:50:44.401 INFO:teuthology.orchestra.run.smithi083.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-20T18:50:44.401 INFO:teuthology.orchestra.run.smithi083.stdout: "bdev_inject_crash": "0", 2024-09-20T18:50:44.401 INFO:teuthology.orchestra.run.smithi083.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-20T18:50:44.401 INFO:teuthology.orchestra.run.smithi083.stdout: "bdev_ioring": "false", 2024-09-20T18:50:44.401 INFO:teuthology.orchestra.run.smithi083.stdout: "bdev_ioring_hipri": "false", 2024-09-20T18:50:44.401 INFO:teuthology.orchestra.run.smithi083.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-20T18:50:44.401 INFO:teuthology.orchestra.run.smithi083.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-20T18:50:44.401 INFO:teuthology.orchestra.run.smithi083.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-20T18:50:44.401 INFO:teuthology.orchestra.run.smithi083.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-20T18:50:44.401 INFO:teuthology.orchestra.run.smithi083.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-20T18:50:44.401 INFO:teuthology.orchestra.run.smithi083.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-20T18:50:44.402 INFO:teuthology.orchestra.run.smithi083.stdout: "bdev_type": "", 2024-09-20T18:50:44.402 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_alloc_size": "1048576", 2024-09-20T18:50:44.402 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_allocator": "hybrid", 2024-09-20T18:50:44.402 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_buffered_io": "true", 2024-09-20T18:50:44.402 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_check_for_zeros": "false", 2024-09-20T18:50:44.402 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-20T18:50:44.402 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-20T18:50:44.402 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_compact_log_sync": "false", 2024-09-20T18:50:44.402 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-20T18:50:44.402 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-20T18:50:44.402 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-20T18:50:44.402 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-20T18:50:44.402 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_max_log_runway": "4194304", 2024-09-20T18:50:44.403 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_max_prefetch": "1048576", 2024-09-20T18:50:44.403 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_min_flush_size": "524288", 2024-09-20T18:50:44.403 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_min_log_runway": "1048576", 2024-09-20T18:50:44.403 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_replay_recovery": "false", 2024-09-20T18:50:44.403 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-20T18:50:44.403 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-20T18:50:44.403 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_sync_write": "false", 2024-09-20T18:50:44.403 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-20T18:50:44.403 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-20T18:50:44.403 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-20T18:50:44.403 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_allocation_from_file": "true", 2024-09-20T18:50:44.403 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_allocator": "hybrid", 2024-09-20T18:50:44.403 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_async_db_compaction": "true", 2024-09-20T18:50:44.403 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-20T18:50:44.404 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-20T18:50:44.404 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-20T18:50:44.404 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-20T18:50:44.404 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_bdev_label_multi": "true", 2024-09-20T18:50:44.404 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-20T18:50:44.404 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-20T18:50:44.404 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-20T18:50:44.404 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-20T18:50:44.404 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-20T18:50:44.404 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_block_create": "true", 2024-09-20T18:50:44.404 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_block_db_create": "false", 2024-09-20T18:50:44.404 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_block_db_path": "", 2024-09-20T18:50:44.404 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_block_db_size": "0", 2024-09-20T18:50:44.404 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_block_path": "", 2024-09-20T18:50:44.405 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_block_preallocate_file": "false", 2024-09-20T18:50:44.405 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_block_size": "107374182400", 2024-09-20T18:50:44.405 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_block_wal_create": "false", 2024-09-20T18:50:44.405 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_block_wal_path": "", 2024-09-20T18:50:44.405 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_block_wal_size": "100663296", 2024-09-20T18:50:44.405 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_bluefs": "true", 2024-09-20T18:50:44.405 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-20T18:50:44.405 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-20T18:50:44.405 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-20T18:50:44.405 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_btree2_alloc_weight_factor": "2.000000", 2024-09-20T18:50:44.405 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-20T18:50:44.405 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-20T18:50:44.405 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-20T18:50:44.405 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-20T18:50:44.406 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-20T18:50:44.406 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_autotune": "true", 2024-09-20T18:50:44.406 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-20T18:50:44.406 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-20T18:50:44.406 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-20T18:50:44.406 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-20T18:50:44.406 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_size": "0", 2024-09-20T18:50:44.406 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-20T18:50:44.406 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-20T18:50:44.406 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-20T18:50:44.406 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-20T18:50:44.406 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_type": "2q", 2024-09-20T18:50:44.406 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-20T18:50:44.406 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_clone_cow": "true", 2024-09-20T18:50:44.407 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-20T18:50:44.407 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-20T18:50:44.407 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-20T18:50:44.407 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-20T18:50:44.407 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-20T18:50:44.407 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-20T18:50:44.407 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-20T18:50:44.407 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_compression_mode": "none", 2024-09-20T18:50:44.407 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-20T18:50:44.407 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_csum_type": "crc32c", 2024-09-20T18:50:44.407 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_debug_enforce_min_alloc_size": "0", 2024-09-20T18:50:44.407 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-20T18:50:44.407 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_debug_freelist": "false", 2024-09-20T18:50:44.408 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-20T18:50:44.408 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-20T18:50:44.408 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-20T18:50:44.408 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-20T18:50:44.408 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-20T18:50:44.408 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-20T18:50:44.408 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-20T18:50:44.408 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-20T18:50:44.408 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-20T18:50:44.408 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-20T18:50:44.408 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-20T18:50:44.408 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-20T18:50:44.408 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_debug_small_allocations": "0", 2024-09-20T18:50:44.409 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-20T18:50:44.409 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_default_buffered_read": "true", 2024-09-20T18:50:44.409 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_default_buffered_write": "false", 2024-09-20T18:50:44.409 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-20T18:50:44.409 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-20T18:50:44.409 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-20T18:50:44.409 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-20T18:50:44.409 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-20T18:50:44.409 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-20T18:50:44.409 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-20T18:50:44.409 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-20T18:50:44.409 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-20T18:50:44.409 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fail_eio": "false", 2024-09-20T18:50:44.409 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-20T18:50:44.409 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-20T18:50:44.410 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-20T18:50:44.410 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-20T18:50:44.410 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-20T18:50:44.410 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-20T18:50:44.410 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_on_mount": "false", 2024-09-20T18:50:44.410 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-20T18:50:44.410 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_on_umount": "false", 2024-09-20T18:50:44.410 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-20T18:50:44.410 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-20T18:50:44.410 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-20T18:50:44.410 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-20T18:50:44.410 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-20T18:50:44.410 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-20T18:50:44.410 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-20T18:50:44.411 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-20T18:50:44.411 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_ignore_data_csum": "false", 2024-09-20T18:50:44.411 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-20T18:50:44.411 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-20T18:50:44.411 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-20T18:50:44.411 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-20T18:50:44.411 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_log_op_age": "5.000000", 2024-09-20T18:50:44.411 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_max_alloc_size": "0", 2024-09-20T18:50:44.411 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_max_blob_size": "0", 2024-09-20T18:50:44.411 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-20T18:50:44.411 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-20T18:50:44.411 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-20T18:50:44.411 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_max_deferred_txc": "32", 2024-09-20T18:50:44.412 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_min_alloc_size": "0", 2024-09-20T18:50:44.412 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-20T18:50:44.412 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-20T18:50:44.412 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_nid_prealloc": "1024", 2024-09-20T18:50:44.412 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-20T18:50:44.412 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-20T18:50:44.412 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-20T18:50:44.412 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-20T18:50:44.412 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_retry_disk_reads": "3", 2024-09-20T18:50:44.412 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_rocksdb_cf": "true", 2024-09-20T18:50:44.412 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-20T18:50:44.412 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-20T18:50:44.412 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-20T18:50:44.412 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-20T18:50:44.413 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-20T18:50:44.413 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-20T18:50:44.413 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-20T18:50:44.413 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-20T18:50:44.413 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_spdk_mem": "512", 2024-09-20T18:50:44.413 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-20T18:50:44.413 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-20T18:50:44.413 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-20T18:50:44.413 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-20T18:50:44.413 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-20T18:50:44.413 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-20T18:50:44.413 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-20T18:50:44.413 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_tracing": "false", 2024-09-20T18:50:44.413 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-20T18:50:44.414 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-20T18:50:44.414 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-20T18:50:44.414 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-20T18:50:44.414 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-20T18:50:44.414 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-20T18:50:44.414 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-20T18:50:44.414 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-20T18:50:44.414 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-20T18:50:44.414 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_write_v2": "false", 2024-09-20T18:50:44.414 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_write_v2_random": "false", 2024-09-20T18:50:44.414 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_zero_block_detection": "false", 2024-09-20T18:50:44.414 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-20T18:50:44.414 INFO:teuthology.orchestra.run.smithi083.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-20T18:50:44.415 INFO:teuthology.orchestra.run.smithi083.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-20T18:50:44.415 INFO:teuthology.orchestra.run.smithi083.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-20T18:50:44.415 INFO:teuthology.orchestra.run.smithi083.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-20T18:50:44.415 INFO:teuthology.orchestra.run.smithi083.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-20T18:50:44.415 INFO:teuthology.orchestra.run.smithi083.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-20T18:50:44.415 INFO:teuthology.orchestra.run.smithi083.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-20T18:50:44.415 INFO:teuthology.orchestra.run.smithi083.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-20T18:50:44.415 INFO:teuthology.orchestra.run.smithi083.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-20T18:50:44.415 INFO:teuthology.orchestra.run.smithi083.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-20T18:50:44.415 INFO:teuthology.orchestra.run.smithi083.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-20T18:50:44.415 INFO:teuthology.orchestra.run.smithi083.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-20T18:50:44.415 INFO:teuthology.orchestra.run.smithi083.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-20T18:50:44.415 INFO:teuthology.orchestra.run.smithi083.stdout: "cephx_cluster_require_signatures": "false", 2024-09-20T18:50:44.415 INFO:teuthology.orchestra.run.smithi083.stdout: "cephx_cluster_require_version": "2", 2024-09-20T18:50:44.416 INFO:teuthology.orchestra.run.smithi083.stdout: "cephx_require_signatures": "false", 2024-09-20T18:50:44.416 INFO:teuthology.orchestra.run.smithi083.stdout: "cephx_require_version": "2", 2024-09-20T18:50:44.416 INFO:teuthology.orchestra.run.smithi083.stdout: "cephx_service_require_signatures": "false", 2024-09-20T18:50:44.416 INFO:teuthology.orchestra.run.smithi083.stdout: "cephx_service_require_version": "2", 2024-09-20T18:50:44.416 INFO:teuthology.orchestra.run.smithi083.stdout: "cephx_sign_messages": "true", 2024-09-20T18:50:44.416 INFO:teuthology.orchestra.run.smithi083.stdout: "chdir": "", 2024-09-20T18:50:44.416 INFO:teuthology.orchestra.run.smithi083.stdout: "client_acl_type": "", 2024-09-20T18:50:44.416 INFO:teuthology.orchestra.run.smithi083.stdout: "client_asio_thread_count": "2", 2024-09-20T18:50:44.416 INFO:teuthology.orchestra.run.smithi083.stdout: "client_cache_mid": "0.750000", 2024-09-20T18:50:44.416 INFO:teuthology.orchestra.run.smithi083.stdout: "client_cache_size": "16384", 2024-09-20T18:50:44.416 INFO:teuthology.orchestra.run.smithi083.stdout: "client_caps_release_delay": "5", 2024-09-20T18:50:44.416 INFO:teuthology.orchestra.run.smithi083.stdout: "client_check_pool_perm": "true", 2024-09-20T18:50:44.416 INFO:teuthology.orchestra.run.smithi083.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-20T18:50:44.417 INFO:teuthology.orchestra.run.smithi083.stdout: "client_debug_force_sync_read": "false", 2024-09-20T18:50:44.417 INFO:teuthology.orchestra.run.smithi083.stdout: "client_debug_getattr_caps": "false", 2024-09-20T18:50:44.417 INFO:teuthology.orchestra.run.smithi083.stdout: "client_debug_inject_features": "", 2024-09-20T18:50:44.417 INFO:teuthology.orchestra.run.smithi083.stdout: "client_debug_inject_tick_delay": "0", 2024-09-20T18:50:44.417 INFO:teuthology.orchestra.run.smithi083.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-20T18:50:44.417 INFO:teuthology.orchestra.run.smithi083.stdout: "client_die_on_failed_remount": "false", 2024-09-20T18:50:44.417 INFO:teuthology.orchestra.run.smithi083.stdout: "client_dirsize_rbytes": "true", 2024-09-20T18:50:44.417 INFO:teuthology.orchestra.run.smithi083.stdout: "client_force_lazyio": "false", 2024-09-20T18:50:44.417 INFO:teuthology.orchestra.run.smithi083.stdout: "client_fs": "", 2024-09-20T18:50:44.417 INFO:teuthology.orchestra.run.smithi083.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-20T18:50:44.417 INFO:teuthology.orchestra.run.smithi083.stdout: "client_inject_release_failure": "false", 2024-09-20T18:50:44.417 INFO:teuthology.orchestra.run.smithi083.stdout: "client_max_inline_size": "4096", 2024-09-20T18:50:44.417 INFO:teuthology.orchestra.run.smithi083.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-20T18:50:44.418 INFO:teuthology.orchestra.run.smithi083.stdout: "client_mds_namespace": "", 2024-09-20T18:50:44.418 INFO:teuthology.orchestra.run.smithi083.stdout: "client_metadata": "", 2024-09-20T18:50:44.418 INFO:teuthology.orchestra.run.smithi083.stdout: "client_mount_gid": "-1", 2024-09-20T18:50:44.418 INFO:teuthology.orchestra.run.smithi083.stdout: "client_mount_timeout": "300", 2024-09-20T18:50:44.418 INFO:teuthology.orchestra.run.smithi083.stdout: "client_mount_uid": "-1", 2024-09-20T18:50:44.418 INFO:teuthology.orchestra.run.smithi083.stdout: "client_mountpoint": "/", 2024-09-20T18:50:44.418 INFO:teuthology.orchestra.run.smithi083.stdout: "client_notify_timeout": "10", 2024-09-20T18:50:44.418 INFO:teuthology.orchestra.run.smithi083.stdout: "client_oc": "true", 2024-09-20T18:50:44.418 INFO:teuthology.orchestra.run.smithi083.stdout: "client_oc_max_dirty": "104857600", 2024-09-20T18:50:44.418 INFO:teuthology.orchestra.run.smithi083.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-20T18:50:44.418 INFO:teuthology.orchestra.run.smithi083.stdout: "client_oc_max_objects": "1000", 2024-09-20T18:50:44.418 INFO:teuthology.orchestra.run.smithi083.stdout: "client_oc_size": "209715200", 2024-09-20T18:50:44.418 INFO:teuthology.orchestra.run.smithi083.stdout: "client_oc_target_dirty": "8388608", 2024-09-20T18:50:44.418 INFO:teuthology.orchestra.run.smithi083.stdout: "client_permissions": "true", 2024-09-20T18:50:44.418 INFO:teuthology.orchestra.run.smithi083.stdout: "client_quota": "true", 2024-09-20T18:50:44.419 INFO:teuthology.orchestra.run.smithi083.stdout: "client_quota_df": "true", 2024-09-20T18:50:44.419 INFO:teuthology.orchestra.run.smithi083.stdout: "client_readahead_max_bytes": "0", 2024-09-20T18:50:44.419 INFO:teuthology.orchestra.run.smithi083.stdout: "client_readahead_max_periods": "4", 2024-09-20T18:50:44.419 INFO:teuthology.orchestra.run.smithi083.stdout: "client_readahead_min": "131072", 2024-09-20T18:50:44.419 INFO:teuthology.orchestra.run.smithi083.stdout: "client_reconnect_stale": "false", 2024-09-20T18:50:44.419 INFO:teuthology.orchestra.run.smithi083.stdout: "client_shutdown_timeout": "30", 2024-09-20T18:50:44.419 INFO:teuthology.orchestra.run.smithi083.stdout: "client_snapdir": ".snap", 2024-09-20T18:50:44.419 INFO:teuthology.orchestra.run.smithi083.stdout: "client_tick_interval": "1", 2024-09-20T18:50:44.419 INFO:teuthology.orchestra.run.smithi083.stdout: "client_trace": "", 2024-09-20T18:50:44.419 INFO:teuthology.orchestra.run.smithi083.stdout: "client_try_dentry_invalidate": "false", 2024-09-20T18:50:44.419 INFO:teuthology.orchestra.run.smithi083.stdout: "client_use_faked_inos": "false", 2024-09-20T18:50:44.419 INFO:teuthology.orchestra.run.smithi083.stdout: "client_use_random_mds": "false", 2024-09-20T18:50:44.419 INFO:teuthology.orchestra.run.smithi083.stdout: "clog_to_graylog": "false", 2024-09-20T18:50:44.420 INFO:teuthology.orchestra.run.smithi083.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-20T18:50:44.420 INFO:teuthology.orchestra.run.smithi083.stdout: "clog_to_graylog_port": "12201", 2024-09-20T18:50:44.420 INFO:teuthology.orchestra.run.smithi083.stdout: "clog_to_monitors": "default=true", 2024-09-20T18:50:44.420 INFO:teuthology.orchestra.run.smithi083.stdout: "clog_to_syslog": "false", 2024-09-20T18:50:44.420 INFO:teuthology.orchestra.run.smithi083.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-20T18:50:44.420 INFO:teuthology.orchestra.run.smithi083.stdout: "clog_to_syslog_level": "info", 2024-09-20T18:50:44.420 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "-", 2024-09-20T18:50:44.420 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_network": "", 2024-09-20T18:50:44.420 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_network_interface": "", 2024-09-20T18:50:44.420 INFO:teuthology.orchestra.run.smithi083.stdout: "compressor_zlib_isal": "false", 2024-09-20T18:50:44.420 INFO:teuthology.orchestra.run.smithi083.stdout: "compressor_zlib_level": "5", 2024-09-20T18:50:44.420 INFO:teuthology.orchestra.run.smithi083.stdout: "compressor_zlib_winsize": "-15", 2024-09-20T18:50:44.420 INFO:teuthology.orchestra.run.smithi083.stdout: "compressor_zstd_level": "1", 2024-09-20T18:50:44.420 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image": "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2", 2024-09-20T18:50:44.421 INFO:teuthology.orchestra.run.smithi083.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-20T18:50:44.421 INFO:teuthology.orchestra.run.smithi083.stdout: "crimson_alien_op_num_threads": "6", 2024-09-20T18:50:44.421 INFO:teuthology.orchestra.run.smithi083.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-20T18:50:44.421 INFO:teuthology.orchestra.run.smithi083.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-20T18:50:44.421 INFO:teuthology.orchestra.run.smithi083.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-20T18:50:44.421 INFO:teuthology.orchestra.run.smithi083.stdout: "crimson_osd_stat_interval": "0", 2024-09-20T18:50:44.421 INFO:teuthology.orchestra.run.smithi083.stdout: "crimson_seastar_cpu_cores": "", 2024-09-20T18:50:44.421 INFO:teuthology.orchestra.run.smithi083.stdout: "crimson_seastar_num_threads": "0", 2024-09-20T18:50:44.421 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_location": "", 2024-09-20T18:50:44.421 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_location_hook": "", 2024-09-20T18:50:44.421 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_location_hook_timeout": "10", 2024-09-20T18:50:44.421 INFO:teuthology.orchestra.run.smithi083.stdout: "daemonize": "false", 2024-09-20T18:50:44.421 INFO:teuthology.orchestra.run.smithi083.stdout: "daos_pool": "tank", 2024-09-20T18:50:44.421 INFO:teuthology.orchestra.run.smithi083.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-20T18:50:44.421 INFO:teuthology.orchestra.run.smithi083.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-20T18:50:44.422 INFO:teuthology.orchestra.run.smithi083.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-20T18:50:44.422 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_alienstore": "0/5", 2024-09-20T18:50:44.422 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_allow_any_pool_priority": "false", 2024-09-20T18:50:44.422 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_asok": "1/5", 2024-09-20T18:50:44.422 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_asok_assert_abort": "false", 2024-09-20T18:50:44.422 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_asserts_on_shutdown": "true", 2024-09-20T18:50:44.422 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_auth": "1/5", 2024-09-20T18:50:44.422 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_bdev": "1/3", 2024-09-20T18:50:44.422 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_bluefs": "1/5", 2024-09-20T18:50:44.422 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_bluestore": "1/5", 2024-09-20T18:50:44.422 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_buffer": "0/1", 2024-09-20T18:50:44.422 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_ceph_dedup": "0/5", 2024-09-20T18:50:44.422 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_ceph_exporter": "1/5", 2024-09-20T18:50:44.422 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_cephfs_mirror": "0/5", 2024-09-20T18:50:44.423 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_cephsqlite": "0/5", 2024-09-20T18:50:44.423 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_client": "0/5", 2024-09-20T18:50:44.423 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_compressor": "1/5", 2024-09-20T18:50:44.423 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_context": "0/1", 2024-09-20T18:50:44.423 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_crimson_interrupt": "0/5", 2024-09-20T18:50:44.423 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_crush": "1/1", 2024-09-20T18:50:44.423 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_crypto": "1/5", 2024-09-20T18:50:44.423 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_cyanstore": "0/5", 2024-09-20T18:50:44.423 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_deliberately_leak_memory": "false", 2024-09-20T18:50:44.423 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_disable_randomized_ping": "false", 2024-09-20T18:50:44.423 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_dpdk": "1/5", 2024-09-20T18:50:44.423 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_eventtrace": "1/5", 2024-09-20T18:50:44.423 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_filer": "0/1", 2024-09-20T18:50:44.424 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_filestore": "1/3", 2024-09-20T18:50:44.424 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_finisher": "1/1", 2024-09-20T18:50:44.424 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_fuse": "1/5", 2024-09-20T18:50:44.424 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_heartbeat_testing_span": "0", 2024-09-20T18:50:44.424 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_heartbeatmap": "1/5", 2024-09-20T18:50:44.424 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-20T18:50:44.424 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_javaclient": "1/5", 2024-09-20T18:50:44.424 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_journal": "1/3", 2024-09-20T18:50:44.424 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_journaler": "0/5", 2024-09-20T18:50:44.424 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_kstore": "1/5", 2024-09-20T18:50:44.424 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_lockdep": "0/1", 2024-09-20T18:50:44.424 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_mclock": "1/5", 2024-09-20T18:50:44.424 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_mds": "1/5", 2024-09-20T18:50:44.425 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_mds_balancer": "1/5", 2024-09-20T18:50:44.425 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_mds_locker": "1/5", 2024-09-20T18:50:44.425 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_mds_log": "1/5", 2024-09-20T18:50:44.425 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_mds_log_expire": "1/5", 2024-09-20T18:50:44.425 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_mds_migrator": "1/5", 2024-09-20T18:50:44.425 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_mds_quiesce": "3/5", 2024-09-20T18:50:44.425 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_memstore": "1/5", 2024-09-20T18:50:44.425 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_mgr": "2/5", 2024-09-20T18:50:44.425 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_mgrc": "1/5", 2024-09-20T18:50:44.425 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_mon": "20/20", 2024-09-20T18:50:44.425 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_monc": "0/10", 2024-09-20T18:50:44.425 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_ms": "1/1", 2024-09-20T18:50:44.425 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_none": "0/5", 2024-09-20T18:50:44.425 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_objclass": "0/5", 2024-09-20T18:50:44.425 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_objectcacher": "0/5", 2024-09-20T18:50:44.426 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_objecter": "0/1", 2024-09-20T18:50:44.426 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_optracker": "0/5", 2024-09-20T18:50:44.426 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_osd": "1/5", 2024-09-20T18:50:44.426 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_paxos": "20/20", 2024-09-20T18:50:44.426 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_perfcounter": "1/5", 2024-09-20T18:50:44.426 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_prioritycache": "1/5", 2024-09-20T18:50:44.426 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_rados": "0/5", 2024-09-20T18:50:44.426 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_rbd": "0/5", 2024-09-20T18:50:44.426 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_rbd_mirror": "0/5", 2024-09-20T18:50:44.426 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_rbd_pwl": "0/5", 2024-09-20T18:50:44.426 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_rbd_replay": "0/5", 2024-09-20T18:50:44.426 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_refs": "0/0", 2024-09-20T18:50:44.426 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_reserver": "1/1", 2024-09-20T18:50:44.426 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_rgw": "1/5", 2024-09-20T18:50:44.427 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_rgw_access": "1/5", 2024-09-20T18:50:44.427 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_rgw_datacache": "1/5", 2024-09-20T18:50:44.427 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_rgw_dbstore": "1/5", 2024-09-20T18:50:44.427 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_rgw_flight": "1/5", 2024-09-20T18:50:44.428 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-20T18:50:44.428 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_rgw_notification": "1/5", 2024-09-20T18:50:44.428 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_rgw_sync": "1/5", 2024-09-20T18:50:44.428 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_rocksdb": "4/5", 2024-09-20T18:50:44.428 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_seastore": "0/5", 2024-09-20T18:50:44.428 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_seastore_backref": "0/5", 2024-09-20T18:50:44.429 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_seastore_cache": "0/5", 2024-09-20T18:50:44.429 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_seastore_cleaner": "0/5", 2024-09-20T18:50:44.429 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_seastore_device": "0/5", 2024-09-20T18:50:44.429 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_seastore_epm": "0/5", 2024-09-20T18:50:44.429 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-20T18:50:44.429 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_seastore_journal": "0/5", 2024-09-20T18:50:44.429 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_seastore_lba": "0/5", 2024-09-20T18:50:44.429 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_seastore_odata": "0/5", 2024-09-20T18:50:44.429 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_seastore_omap": "0/5", 2024-09-20T18:50:44.429 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_seastore_onode": "0/5", 2024-09-20T18:50:44.429 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_seastore_t": "0/5", 2024-09-20T18:50:44.429 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_seastore_tm": "0/5", 2024-09-20T18:50:44.429 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_striper": "0/1", 2024-09-20T18:50:44.430 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_test": "0/5", 2024-09-20T18:50:44.430 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_throttle": "1/1", 2024-09-20T18:50:44.430 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_timer": "0/1", 2024-09-20T18:50:44.430 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_tp": "0/5", 2024-09-20T18:50:44.430 INFO:teuthology.orchestra.run.smithi083.stdout: "debug_trace": "1/5", 2024-09-20T18:50:44.430 INFO:teuthology.orchestra.run.smithi083.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-20T18:50:44.430 INFO:teuthology.orchestra.run.smithi083.stdout: "device_failure_prediction_mode": "none", 2024-09-20T18:50:44.430 INFO:teuthology.orchestra.run.smithi083.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-20T18:50:44.430 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-09-20T18:50:44.430 INFO:teuthology.orchestra.run.smithi083.stdout: "err_to_graylog": "false", 2024-09-20T18:50:44.430 INFO:teuthology.orchestra.run.smithi083.stdout: "err_to_journald": "false", 2024-09-20T18:50:44.430 INFO:teuthology.orchestra.run.smithi083.stdout: "err_to_stderr": "true", 2024-09-20T18:50:44.430 INFO:teuthology.orchestra.run.smithi083.stdout: "err_to_syslog": "false", 2024-09-20T18:50:44.431 INFO:teuthology.orchestra.run.smithi083.stdout: "event_tracing": "false", 2024-09-20T18:50:44.431 INFO:teuthology.orchestra.run.smithi083.stdout: "exporter_addr": "0.0.0.0", 2024-09-20T18:50:44.431 INFO:teuthology.orchestra.run.smithi083.stdout: "exporter_cert_file": "", 2024-09-20T18:50:44.431 INFO:teuthology.orchestra.run.smithi083.stdout: "exporter_http_port": "9926", 2024-09-20T18:50:44.431 INFO:teuthology.orchestra.run.smithi083.stdout: "exporter_key_file": "", 2024-09-20T18:50:44.431 INFO:teuthology.orchestra.run.smithi083.stdout: "exporter_prio_limit": "5", 2024-09-20T18:50:44.431 INFO:teuthology.orchestra.run.smithi083.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-20T18:50:44.431 INFO:teuthology.orchestra.run.smithi083.stdout: "exporter_sort_metrics": "true", 2024-09-20T18:50:44.431 INFO:teuthology.orchestra.run.smithi083.stdout: "exporter_stats_period": "5", 2024-09-20T18:50:44.431 INFO:teuthology.orchestra.run.smithi083.stdout: "fake_statfs_for_testing": "0", 2024-09-20T18:50:44.431 INFO:teuthology.orchestra.run.smithi083.stdout: "fatal_signal_handlers": "true", 2024-09-20T18:50:44.431 INFO:teuthology.orchestra.run.smithi083.stdout: "filer_max_purge_ops": "10", 2024-09-20T18:50:44.431 INFO:teuthology.orchestra.run.smithi083.stdout: "filer_max_truncate_ops": "128", 2024-09-20T18:50:44.432 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_apply_finisher_threads": "1", 2024-09-20T18:50:44.432 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_blackhole": "false", 2024-09-20T18:50:44.432 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_btrfs_clone_range": "true", 2024-09-20T18:50:44.432 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_btrfs_snap": "true", 2024-09-20T18:50:44.432 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_caller_concurrency": "10", 2024-09-20T18:50:44.432 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_collect_device_partition_information": "true", 2024-09-20T18:50:44.432 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_commit_timeout": "600.000000", 2024-09-20T18:50:44.432 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_debug_inject_read_err": "false", 2024-09-20T18:50:44.432 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_debug_omap_check": "false", 2024-09-20T18:50:44.432 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_debug_verify_split": "false", 2024-09-20T18:50:44.432 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_dump_file": "", 2024-09-20T18:50:44.432 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-20T18:50:44.432 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-20T18:50:44.432 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_fadvise": "true", 2024-09-20T18:50:44.433 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_fail_eio": "true", 2024-09-20T18:50:44.433 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_fd_cache_shards": "16", 2024-09-20T18:50:44.433 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_fd_cache_size": "128", 2024-09-20T18:50:44.433 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_fiemap": "false", 2024-09-20T18:50:44.433 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_fiemap_threshold": "4096", 2024-09-20T18:50:44.433 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-20T18:50:44.433 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-20T18:50:44.433 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_inject_stall": "0", 2024-09-20T18:50:44.433 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_journal_parallel": "false", 2024-09-20T18:50:44.433 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_journal_trailing": "false", 2024-09-20T18:50:44.433 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_journal_writeahead": "false", 2024-09-20T18:50:44.433 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_kill_at": "0", 2024-09-20T18:50:44.433 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-20T18:50:44.433 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-20T18:50:44.434 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-20T18:50:44.434 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-20T18:50:44.434 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-20T18:50:44.434 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_max_inline_xattrs": "0", 2024-09-20T18:50:44.434 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-20T18:50:44.434 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-20T18:50:44.434 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-20T18:50:44.434 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-20T18:50:44.434 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_max_xattr_value_size": "0", 2024-09-20T18:50:44.434 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-20T18:50:44.434 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-20T18:50:44.434 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-20T18:50:44.434 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_merge_threshold": "-10", 2024-09-20T18:50:44.434 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-20T18:50:44.435 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_odsync_write": "false", 2024-09-20T18:50:44.435 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_omap_backend": "rocksdb", 2024-09-20T18:50:44.435 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_omap_backend_path": "", 2024-09-20T18:50:44.435 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-20T18:50:44.435 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-20T18:50:44.435 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-20T18:50:44.435 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_op_thread_timeout": "60", 2024-09-20T18:50:44.435 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_op_threads": "2", 2024-09-20T18:50:44.435 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_punch_hole": "false", 2024-09-20T18:50:44.435 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-20T18:50:44.435 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-20T18:50:44.435 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-20T18:50:44.435 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-20T18:50:44.435 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-20T18:50:44.436 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-20T18:50:44.436 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-20T18:50:44.436 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-20T18:50:44.436 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-20T18:50:44.436 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_queue_max_ops": "50", 2024-09-20T18:50:44.436 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-20T18:50:44.436 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_seek_data_hole": "false", 2024-09-20T18:50:44.436 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_sloppy_crc": "false", 2024-09-20T18:50:44.436 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-20T18:50:44.436 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_splice": "false", 2024-09-20T18:50:44.436 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_split_multiple": "2", 2024-09-20T18:50:44.436 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_split_rand_factor": "20", 2024-09-20T18:50:44.436 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_update_to": "1000", 2024-09-20T18:50:44.436 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-20T18:50:44.437 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-20T18:50:44.437 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-20T18:50:44.437 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-20T18:50:44.437 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-20T18:50:44.437 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-20T18:50:44.437 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_wbthrottle_enable": "true", 2024-09-20T18:50:44.437 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-20T18:50:44.437 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-20T18:50:44.437 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-20T18:50:44.437 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-20T18:50:44.437 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-20T18:50:44.437 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-20T18:50:44.437 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_xfs_extsize": "false", 2024-09-20T18:50:44.437 INFO:teuthology.orchestra.run.smithi083.stdout: "filestore_zfs_snap": "false", 2024-09-20T18:50:44.438 INFO:teuthology.orchestra.run.smithi083.stdout: "fio_dir": "/tmp/fio", 2024-09-20T18:50:44.438 INFO:teuthology.orchestra.run.smithi083.stdout: "fsid": "1ed5a50a-7780-11ef-baf5-efdc52797490", 2024-09-20T18:50:44.438 INFO:teuthology.orchestra.run.smithi083.stdout: "fuse_allow_other": "true", 2024-09-20T18:50:44.438 INFO:teuthology.orchestra.run.smithi083.stdout: "fuse_atomic_o_trunc": "true", 2024-09-20T18:50:44.438 INFO:teuthology.orchestra.run.smithi083.stdout: "fuse_big_writes": "true", 2024-09-20T18:50:44.438 INFO:teuthology.orchestra.run.smithi083.stdout: "fuse_debug": "false", 2024-09-20T18:50:44.438 INFO:teuthology.orchestra.run.smithi083.stdout: "fuse_default_permissions": "false", 2024-09-20T18:50:44.438 INFO:teuthology.orchestra.run.smithi083.stdout: "fuse_disable_pagecache": "false", 2024-09-20T18:50:44.438 INFO:teuthology.orchestra.run.smithi083.stdout: "fuse_max_write": "0", 2024-09-20T18:50:44.438 INFO:teuthology.orchestra.run.smithi083.stdout: "fuse_multithreaded": "true", 2024-09-20T18:50:44.438 INFO:teuthology.orchestra.run.smithi083.stdout: "fuse_require_active_mds": "true", 2024-09-20T18:50:44.438 INFO:teuthology.orchestra.run.smithi083.stdout: "fuse_set_user_groups": "true", 2024-09-20T18:50:44.438 INFO:teuthology.orchestra.run.smithi083.stdout: "fuse_splice_move": "true", 2024-09-20T18:50:44.438 INFO:teuthology.orchestra.run.smithi083.stdout: "fuse_splice_read": "true", 2024-09-20T18:50:44.439 INFO:teuthology.orchestra.run.smithi083.stdout: "fuse_splice_write": "true", 2024-09-20T18:50:44.439 INFO:teuthology.orchestra.run.smithi083.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-20T18:50:44.439 INFO:teuthology.orchestra.run.smithi083.stdout: "fuse_use_invalidate_cb": "true", 2024-09-20T18:50:44.439 INFO:teuthology.orchestra.run.smithi083.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-20T18:50:44.439 INFO:teuthology.orchestra.run.smithi083.stdout: "gss_target_name": "ceph", 2024-09-20T18:50:44.439 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_file": "", 2024-09-20T18:50:44.439 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_inject_failure": "0", 2024-09-20T18:50:44.439 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_interval": "5", 2024-09-20T18:50:44.439 INFO:teuthology.orchestra.run.smithi083.stdout: "host": "smithi083", 2024-09-20T18:50:44.439 INFO:teuthology.orchestra.run.smithi083.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-20T18:50:44.439 INFO:teuthology.orchestra.run.smithi083.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-20T18:50:44.439 INFO:teuthology.orchestra.run.smithi083.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-20T18:50:44.439 INFO:teuthology.orchestra.run.smithi083.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-20T18:50:44.440 INFO:teuthology.orchestra.run.smithi083.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-20T18:50:44.440 INFO:teuthology.orchestra.run.smithi083.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-20T18:50:44.440 INFO:teuthology.orchestra.run.smithi083.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-20T18:50:44.440 INFO:teuthology.orchestra.run.smithi083.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-20T18:50:44.440 INFO:teuthology.orchestra.run.smithi083.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-20T18:50:44.440 INFO:teuthology.orchestra.run.smithi083.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-20T18:50:44.440 INFO:teuthology.orchestra.run.smithi083.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-20T18:50:44.440 INFO:teuthology.orchestra.run.smithi083.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-20T18:50:44.440 INFO:teuthology.orchestra.run.smithi083.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-20T18:50:44.440 INFO:teuthology.orchestra.run.smithi083.stdout: "inject_early_sigterm": "false", 2024-09-20T18:50:44.440 INFO:teuthology.orchestra.run.smithi083.stdout: "jaeger_agent_port": "6799", 2024-09-20T18:50:44.440 INFO:teuthology.orchestra.run.smithi083.stdout: "jaeger_tracing_enable": "false", 2024-09-20T18:50:44.440 INFO:teuthology.orchestra.run.smithi083.stdout: "journal_aio": "true", 2024-09-20T18:50:44.440 INFO:teuthology.orchestra.run.smithi083.stdout: "journal_align_min_size": "65536", 2024-09-20T18:50:44.441 INFO:teuthology.orchestra.run.smithi083.stdout: "journal_block_align": "true", 2024-09-20T18:50:44.441 INFO:teuthology.orchestra.run.smithi083.stdout: "journal_block_size": "4096", 2024-09-20T18:50:44.441 INFO:teuthology.orchestra.run.smithi083.stdout: "journal_dio": "true", 2024-09-20T18:50:44.441 INFO:teuthology.orchestra.run.smithi083.stdout: "journal_discard": "false", 2024-09-20T18:50:44.441 INFO:teuthology.orchestra.run.smithi083.stdout: "journal_force_aio": "false", 2024-09-20T18:50:44.441 INFO:teuthology.orchestra.run.smithi083.stdout: "journal_ignore_corruption": "false", 2024-09-20T18:50:44.441 INFO:teuthology.orchestra.run.smithi083.stdout: "journal_max_write_bytes": "10485760", 2024-09-20T18:50:44.441 INFO:teuthology.orchestra.run.smithi083.stdout: "journal_max_write_entries": "100", 2024-09-20T18:50:44.441 INFO:teuthology.orchestra.run.smithi083.stdout: "journal_replay_from": "0", 2024-09-20T18:50:44.441 INFO:teuthology.orchestra.run.smithi083.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-20T18:50:44.441 INFO:teuthology.orchestra.run.smithi083.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-20T18:50:44.441 INFO:teuthology.orchestra.run.smithi083.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-20T18:50:44.441 INFO:teuthology.orchestra.run.smithi083.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-20T18:50:44.441 INFO:teuthology.orchestra.run.smithi083.stdout: "journal_write_header_frequency": "0", 2024-09-20T18:50:44.441 INFO:teuthology.orchestra.run.smithi083.stdout: "journal_zero_on_create": "false", 2024-09-20T18:50:44.442 INFO:teuthology.orchestra.run.smithi083.stdout: "journaler_prefetch_periods": "10", 2024-09-20T18:50:44.442 INFO:teuthology.orchestra.run.smithi083.stdout: "journaler_prezero_periods": "5", 2024-09-20T18:50:44.442 INFO:teuthology.orchestra.run.smithi083.stdout: "journaler_write_head_interval": "15", 2024-09-20T18:50:44.442 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "", 2024-09-20T18:50:44.442 INFO:teuthology.orchestra.run.smithi083.stdout: "keyfile": "", 2024-09-20T18:50:44.442 INFO:teuthology.orchestra.run.smithi083.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-20T18:50:44.442 INFO:teuthology.orchestra.run.smithi083.stdout: "kstore_backend": "rocksdb", 2024-09-20T18:50:44.442 INFO:teuthology.orchestra.run.smithi083.stdout: "kstore_default_stripe_size": "65536", 2024-09-20T18:50:44.442 INFO:teuthology.orchestra.run.smithi083.stdout: "kstore_fsck_on_mount": "false", 2024-09-20T18:50:44.442 INFO:teuthology.orchestra.run.smithi083.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-20T18:50:44.442 INFO:teuthology.orchestra.run.smithi083.stdout: "kstore_max_bytes": "67108864", 2024-09-20T18:50:44.442 INFO:teuthology.orchestra.run.smithi083.stdout: "kstore_max_ops": "512", 2024-09-20T18:50:44.442 INFO:teuthology.orchestra.run.smithi083.stdout: "kstore_nid_prealloc": "1024", 2024-09-20T18:50:44.443 INFO:teuthology.orchestra.run.smithi083.stdout: "kstore_onode_map_size": "1024", 2024-09-20T18:50:44.443 INFO:teuthology.orchestra.run.smithi083.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-20T18:50:44.443 INFO:teuthology.orchestra.run.smithi083.stdout: "kstore_sync_submit_transaction": "false", 2024-09-20T18:50:44.443 INFO:teuthology.orchestra.run.smithi083.stdout: "kstore_sync_transaction": "false", 2024-09-20T18:50:44.443 INFO:teuthology.orchestra.run.smithi083.stdout: "librados_thread_count": "2", 2024-09-20T18:50:44.443 INFO:teuthology.orchestra.run.smithi083.stdout: "lockdep": "false", 2024-09-20T18:50:44.443 INFO:teuthology.orchestra.run.smithi083.stdout: "lockdep_force_backtrace": "false", 2024-09-20T18:50:44.443 INFO:teuthology.orchestra.run.smithi083.stdout: "log_coarse_timestamps": "true", 2024-09-20T18:50:44.443 INFO:teuthology.orchestra.run.smithi083.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-20T18:50:44.443 INFO:teuthology.orchestra.run.smithi083.stdout: "log_flush_on_exit": "false", 2024-09-20T18:50:44.443 INFO:teuthology.orchestra.run.smithi083.stdout: "log_graylog_host": "127.0.0.1", 2024-09-20T18:50:44.443 INFO:teuthology.orchestra.run.smithi083.stdout: "log_graylog_port": "12201", 2024-09-20T18:50:44.443 INFO:teuthology.orchestra.run.smithi083.stdout: "log_max_new": "1000", 2024-09-20T18:50:44.444 INFO:teuthology.orchestra.run.smithi083.stdout: "log_max_recent": "10000", 2024-09-20T18:50:44.444 INFO:teuthology.orchestra.run.smithi083.stdout: "log_stderr_prefix": "", 2024-09-20T18:50:44.444 INFO:teuthology.orchestra.run.smithi083.stdout: "log_stop_at_utilization": "0.970000", 2024-09-20T18:50:44.444 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": "true", 2024-09-20T18:50:44.444 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_graylog": "false", 2024-09-20T18:50:44.444 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_journald": "false", 2024-09-20T18:50:44.444 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_stderr": "false", 2024-09-20T18:50:44.444 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_syslog": "false", 2024-09-20T18:50:44.444 INFO:teuthology.orchestra.run.smithi083.stdout: "max_rotating_auth_attempts": "10", 2024-09-20T18:50:44.444 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-20T18:50:44.444 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_action_on_write_error": "1", 2024-09-20T18:50:44.444 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_alternate_name_max": "8192", 2024-09-20T18:50:44.445 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_asio_thread_count": "2", 2024-09-20T18:50:44.445 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_export_pin": "true", 2024-09-20T18:50:44.445 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_fragment_dirs": "true", 2024-09-20T18:50:44.445 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-20T18:50:44.445 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_fragment_interval": "5", 2024-09-20T18:50:44.445 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-20T18:50:44.445 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-20T18:50:44.445 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_interval": "10", 2024-09-20T18:50:44.445 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_max": "-1", 2024-09-20T18:50:44.445 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_max_until": "-1", 2024-09-20T18:50:44.445 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_merge_size": "50", 2024-09-20T18:50:44.445 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_midchunk": "0.300000", 2024-09-20T18:50:44.445 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-20T18:50:44.445 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_min_start": "0.200000", 2024-09-20T18:50:44.446 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_minchunk": "0.001000", 2024-09-20T18:50:44.446 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_mode": "0", 2024-09-20T18:50:44.446 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_need_max": "1.200000", 2024-09-20T18:50:44.446 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_need_min": "0.800000", 2024-09-20T18:50:44.446 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_overload_epochs": "2", 2024-09-20T18:50:44.446 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-20T18:50:44.446 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-20T18:50:44.446 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_split_bits": "3", 2024-09-20T18:50:44.446 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-20T18:50:44.446 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_split_size": "10000", 2024-09-20T18:50:44.446 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-20T18:50:44.446 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_target_decay": "10.000000", 2024-09-20T18:50:44.446 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-20T18:50:44.447 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_beacon_grace": "15.000000", 2024-09-20T18:50:44.447 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_beacon_interval": "4.000000", 2024-09-20T18:50:44.447 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-20T18:50:44.447 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-20T18:50:44.447 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cache_mid": "0.700000", 2024-09-20T18:50:44.447 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-20T18:50:44.447 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cache_quiesce_delay": "0", 2024-09-20T18:50:44.447 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-20T18:50:44.447 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-20T18:50:44.447 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-20T18:50:44.447 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cache_release_free_interval": "10", 2024-09-20T18:50:44.447 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cache_reservation": "0.050000", 2024-09-20T18:50:44.447 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-20T18:50:44.448 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cache_trim_interval": "1", 2024-09-20T18:50:44.448 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cache_trim_threshold": "262144", 2024-09-20T18:50:44.448 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-20T18:50:44.448 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-20T18:50:44.448 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-20T18:50:44.448 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_client_prealloc_inos": "1000", 2024-09-20T18:50:44.448 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-20T18:50:44.448 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_connect_bootstrapping": "false", 2024-09-20T18:50:44.448 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_damage_table_max_entries": "10000", 2024-09-20T18:50:44.448 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-20T18:50:44.448 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_debug_auth_pins": "false", 2024-09-20T18:50:44.448 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_debug_frag": "false", 2024-09-20T18:50:44.448 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_debug_scatterstat": "false", 2024-09-20T18:50:44.449 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_debug_subtrees": "false", 2024-09-20T18:50:44.449 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_decay_halflife": "5.000000", 2024-09-20T18:50:44.449 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_default_dir_hash": "2", 2024-09-20T18:50:44.449 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_defer_session_stale": "true", 2024-09-20T18:50:44.449 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_deny_all_reconnect": "false", 2024-09-20T18:50:44.449 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_dir_keys_per_op": "16384", 2024-09-20T18:50:44.449 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_dir_max_commit_size": "10", 2024-09-20T18:50:44.449 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_dir_max_entries": "0", 2024-09-20T18:50:44.449 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_dir_prefetch": "true", 2024-09-20T18:50:44.449 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-20T18:50:44.449 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-20T18:50:44.449 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_dump_cache_on_map": "false", 2024-09-20T18:50:44.449 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-20T18:50:44.450 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-20T18:50:44.450 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_early_reply": "true", 2024-09-20T18:50:44.450 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_enable_op_tracker": "true", 2024-09-20T18:50:44.450 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_enforce_unique_name": "true", 2024-09-20T18:50:44.450 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-20T18:50:44.450 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-20T18:50:44.450 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_export_ephemeral_random": "true", 2024-09-20T18:50:44.450 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-20T18:50:44.450 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-20T18:50:44.450 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-20T18:50:44.450 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-20T18:50:44.450 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-20T18:50:44.451 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-20T18:50:44.451 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-20T18:50:44.451 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-20T18:50:44.451 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_health_summarize_threshold": "10", 2024-09-20T18:50:44.451 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-20T18:50:44.451 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-20T18:50:44.451 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_inject_health_dummy": "false", 2024-09-20T18:50:44.451 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-20T18:50:44.451 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_inject_migrator_session_race": "false", 2024-09-20T18:50:44.451 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-20T18:50:44.451 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-20T18:50:44.451 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-20T18:50:44.451 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_join_fs": "", 2024-09-20T18:50:44.452 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_journal_format": "1", 2024-09-20T18:50:44.452 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-20T18:50:44.452 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_kill_create_at": "0", 2024-09-20T18:50:44.452 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_kill_dirfrag_at": "0", 2024-09-20T18:50:44.452 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_kill_export_at": "0", 2024-09-20T18:50:44.452 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_kill_import_at": "0", 2024-09-20T18:50:44.452 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_kill_journal_at": "0", 2024-09-20T18:50:44.452 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_kill_journal_expire_at": "0", 2024-09-20T18:50:44.452 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_kill_journal_replay_at": "0", 2024-09-20T18:50:44.452 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_kill_link_at": "0", 2024-09-20T18:50:44.452 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_kill_mdstable_at": "0", 2024-09-20T18:50:44.452 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_kill_openc_at": "0", 2024-09-20T18:50:44.452 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_kill_rename_at": "0", 2024-09-20T18:50:44.452 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_kill_shutdown_at": "0", 2024-09-20T18:50:44.453 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log_event_large_threshold": "524288", 2024-09-20T18:50:44.453 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log_events_per_segment": "1024", 2024-09-20T18:50:44.453 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-20T18:50:44.453 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log_max_events": "-1", 2024-09-20T18:50:44.453 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log_max_segments": "128", 2024-09-20T18:50:44.453 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log_pause": "false", 2024-09-20T18:50:44.453 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log_segment_size": "0", 2024-09-20T18:50:44.453 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-20T18:50:44.453 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-20T18:50:44.453 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-20T18:50:44.453 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log_trim_threshold": "128", 2024-09-20T18:50:44.453 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-20T18:50:44.453 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log_warn_factor": "2.000000", 2024-09-20T18:50:44.453 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_max_caps_per_client": "1048576", 2024-09-20T18:50:44.454 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_max_completed_flushes": "100000", 2024-09-20T18:50:44.454 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_max_completed_requests": "100000", 2024-09-20T18:50:44.454 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_max_export_size": "20971520", 2024-09-20T18:50:44.454 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_max_file_recover": "32", 2024-09-20T18:50:44.454 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_max_purge_files": "64", 2024-09-20T18:50:44.454 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_max_purge_ops": "8192", 2024-09-20T18:50:44.454 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-20T18:50:44.454 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-20T18:50:44.454 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_max_snaps_per_dir": "100", 2024-09-20T18:50:44.454 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_metrics_update_interval": "2", 2024-09-20T18:50:44.454 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_min_caps_per_client": "100", 2024-09-20T18:50:44.454 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_min_caps_working_set": "10000", 2024-09-20T18:50:44.454 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-20T18:50:44.454 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_numa_node": "-1", 2024-09-20T18:50:44.454 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-20T18:50:44.455 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_op_complaint_time": "30.000000", 2024-09-20T18:50:44.455 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_op_history_duration": "600", 2024-09-20T18:50:44.455 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_op_history_size": "20", 2024-09-20T18:50:44.455 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_op_history_slow_op_size": "20", 2024-09-20T18:50:44.455 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-20T18:50:44.455 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_op_log_threshold": "5", 2024-09-20T18:50:44.455 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_ping_grace": "15", 2024-09-20T18:50:44.455 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_ping_interval": "5", 2024-09-20T18:50:44.455 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-20T18:50:44.455 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-20T18:50:44.455 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_recall_max_caps": "30000", 2024-09-20T18:50:44.455 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-20T18:50:44.455 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-20T18:50:44.455 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-20T18:50:44.456 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_recall_warning_threshold": "262144", 2024-09-20T18:50:44.456 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-20T18:50:44.456 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_replay_interval": "1.000000", 2024-09-20T18:50:44.456 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-20T18:50:44.456 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-20T18:50:44.456 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_root_ino_gid": "0", 2024-09-20T18:50:44.456 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_root_ino_uid": "0", 2024-09-20T18:50:44.456 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-20T18:50:44.456 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_scrub_stats_review_period": "1", 2024-09-20T18:50:44.456 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_server_dispatch_client_request_delay": "0", 2024-09-20T18:50:44.456 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_server_dispatch_killpoint_random": "0.000000", 2024-09-20T18:50:44.456 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-20T18:50:44.456 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-20T18:50:44.457 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-20T18:50:44.457 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-20T18:50:44.457 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-20T18:50:44.457 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-20T18:50:44.457 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-20T18:50:44.457 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-20T18:50:44.457 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-20T18:50:44.457 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_shutdown_check": "0", 2024-09-20T18:50:44.457 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_skip_ino": "0", 2024-09-20T18:50:44.457 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-20T18:50:44.457 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_snap_max_uid": "4294967294", 2024-09-20T18:50:44.457 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_snap_min_uid": "0", 2024-09-20T18:50:44.457 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_snap_rstat": "false", 2024-09-20T18:50:44.457 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_standby_replay_damaged": "false", 2024-09-20T18:50:44.457 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_symlink_recovery": "true", 2024-09-20T18:50:44.458 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-20T18:50:44.458 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_thrash_exports": "0", 2024-09-20T18:50:44.458 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_thrash_fragments": "0", 2024-09-20T18:50:44.458 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_tick_interval": "5.000000", 2024-09-20T18:50:44.458 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_valgrind_exit": "false", 2024-09-20T18:50:44.458 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_verify_backtrace": "1", 2024-09-20T18:50:44.458 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_verify_scatter": "false", 2024-09-20T18:50:44.458 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-20T18:50:44.458 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_wipe_sessions": "false", 2024-09-20T18:50:44.458 INFO:teuthology.orchestra.run.smithi083.stdout: "mempool_debug": "false", 2024-09-20T18:50:44.458 INFO:teuthology.orchestra.run.smithi083.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-20T18:50:44.458 INFO:teuthology.orchestra.run.smithi083.stdout: "memstore_device_bytes": "1073741824", 2024-09-20T18:50:44.459 INFO:teuthology.orchestra.run.smithi083.stdout: "memstore_page_set": "false", 2024-09-20T18:50:44.459 INFO:teuthology.orchestra.run.smithi083.stdout: "memstore_page_size": "65536", 2024-09-20T18:50:44.459 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_client_bytes": "134217728", 2024-09-20T18:50:44.459 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_client_messages": "512", 2024-09-20T18:50:44.459 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-20T18:50:44.459 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-20T18:50:44.459 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-20T18:50:44.459 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-20T18:50:44.459 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_disabled_modules": "", 2024-09-20T18:50:44.459 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_enable_op_tracker": "true", 2024-09-20T18:50:44.459 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-20T18:50:44.459 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_max_pg_creating": "1024", 2024-09-20T18:50:44.459 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_max_pg_num_change": "128", 2024-09-20T18:50:44.460 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_mds_bytes": "134217728", 2024-09-20T18:50:44.460 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_mds_messages": "128", 2024-09-20T18:50:44.460 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-20T18:50:44.460 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_mon_bytes": "134217728", 2024-09-20T18:50:44.460 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_mon_messages": "128", 2024-09-20T18:50:44.460 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-20T18:50:44.460 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-20T18:50:44.460 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_op_history_duration": "600", 2024-09-20T18:50:44.460 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_op_history_size": "20", 2024-09-20T18:50:44.460 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-20T18:50:44.460 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-20T18:50:44.460 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_op_log_threshold": "5", 2024-09-20T18:50:44.460 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_osd_bytes": "536870912", 2024-09-20T18:50:44.460 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_osd_messages": "8192", 2024-09-20T18:50:44.460 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_pool": "true", 2024-09-20T18:50:44.461 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-20T18:50:44.461 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_standby_modules": "true", 2024-09-20T18:50:44.461 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_stats_period": "5", 2024-09-20T18:50:44.461 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_stats_threshold": "5", 2024-09-20T18:50:44.461 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_tick_period": "2", 2024-09-20T18:50:44.461 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-20T18:50:44.461 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-20T18:50:44.461 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_allow_pool_delete": "true", 2024-09-20T18:50:44.461 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_allow_pool_size_one": "false", 2024-09-20T18:50:44.461 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_auth_validate_all_caps": "true", 2024-09-20T18:50:44.461 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-20T18:50:44.461 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-20T18:50:44.461 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_client_bytes": "104857600", 2024-09-20T18:50:44.461 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_client_directed_command_retry": "2", 2024-09-20T18:50:44.462 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-20T18:50:44.462 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-20T18:50:44.462 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-20T18:50:44.462 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-20T18:50:44.462 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_client_hunt_parallel": "3", 2024-09-20T18:50:44.462 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_client_log_interval": "1.000000", 2024-09-20T18:50:44.462 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-20T18:50:44.462 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_client_ping_interval": "10.000000", 2024-09-20T18:50:44.462 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-20T18:50:44.462 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_client_target_rank": "-1", 2024-09-20T18:50:44.462 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-20T18:50:44.462 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-20T18:50:44.462 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-20T18:50:44.463 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_cluster_log_level": "debug", 2024-09-20T18:50:44.463 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_cluster_log_to_file": "true", 2024-09-20T18:50:44.463 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-20T18:50:44.463 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-20T18:50:44.463 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-20T18:50:44.463 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_cluster_log_to_journald": "true", 2024-09-20T18:50:44.463 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-20T18:50:44.463 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-20T18:50:44.463 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-20T18:50:44.463 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_compact_on_bootstrap": "false", 2024-09-20T18:50:44.463 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_compact_on_start": "false", 2024-09-20T18:50:44.463 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_compact_on_trim": "true", 2024-09-20T18:50:44.463 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-20T18:50:44.463 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-20T18:50:44.464 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-20T18:50:44.464 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_cpu_threads": "4", 2024-09-20T18:50:44.464 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_crush_min_required_version": "hammer", 2024-09-20T18:50:44.464 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_daemon_bytes": "419430400", 2024-09-20T18:50:44.464 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-20T18:50:44.464 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_data_avail_crit": "5", 2024-09-20T18:50:44.464 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_data_avail_warn": "5", 2024-09-20T18:50:44.464 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_data_size_warn": "16106127360", 2024-09-20T18:50:44.464 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-20T18:50:44.464 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-20T18:50:44.464 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_debug_dump_json": "false", 2024-09-20T18:50:44.464 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-20T18:50:44.464 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_debug_dump_transactions": "false", 2024-09-20T18:50:44.464 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_debug_extra_checks": "false", 2024-09-20T18:50:44.465 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-20T18:50:44.465 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-20T18:50:44.465 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_debug_no_require_reef": "false", 2024-09-20T18:50:44.465 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_debug_no_require_squid": "false", 2024-09-20T18:50:44.465 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-20T18:50:44.465 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-20T18:50:44.465 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-20T18:50:44.465 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_down_mkfs_grace": "60", 2024-09-20T18:50:44.465 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_down_uptime_grace": "60", 2024-09-20T18:50:44.465 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_election_default_strategy": "1", 2024-09-20T18:50:44.465 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_election_timeout": "5.000000", 2024-09-20T18:50:44.465 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-20T18:50:44.465 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_elector_ping_divisor": "2", 2024-09-20T18:50:44.465 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-20T18:50:44.465 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_enable_op_tracker": "true", 2024-09-20T18:50:44.466 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_fake_pool_delete": "false", 2024-09-20T18:50:44.466 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_force_quorum_join": "false", 2024-09-20T18:50:44.466 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-20T18:50:44.466 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_globalid_prealloc": "10000", 2024-09-20T18:50:44.466 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_health_detail_to_clog": "true", 2024-09-20T18:50:44.466 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_health_log_update_period": "5", 2024-09-20T18:50:44.466 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_health_max_detail": "50", 2024-09-20T18:50:44.466 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_health_to_clog": "true", 2024-09-20T18:50:44.466 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_health_to_clog_interval": "600", 2024-09-20T18:50:44.466 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-20T18:50:44.466 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_host": "[v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0]", 2024-09-20T18:50:44.466 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_host_override": "", 2024-09-20T18:50:44.466 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_initial_members": "", 2024-09-20T18:50:44.467 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-20T18:50:44.467 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-20T18:50:44.467 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-20T18:50:44.467 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-20T18:50:44.467 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-20T18:50:44.467 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_lease": "5.000000", 2024-09-20T18:50:44.467 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-20T18:50:44.467 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-20T18:50:44.467 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_log_full_interval": "50", 2024-09-20T18:50:44.467 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_log_max": "10000", 2024-09-20T18:50:44.467 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_log_max_summary": "50", 2024-09-20T18:50:44.467 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-20T18:50:44.467 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_max_log_epochs": "500", 2024-09-20T18:50:44.467 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-20T18:50:44.468 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-20T18:50:44.468 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_max_nvmeof_epochs": "500", 2024-09-20T18:50:44.468 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_max_osd": "10000", 2024-09-20T18:50:44.468 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_max_pg_per_osd": "10000", 2024-09-20T18:50:44.468 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_max_pool_pg_num": "65536", 2024-09-20T18:50:44.468 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-20T18:50:44.468 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-20T18:50:44.468 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_mds_force_trim_to": "0", 2024-09-20T18:50:44.468 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_mds_skip_sanity": "false", 2024-09-20T18:50:44.468 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_memory_autotune": "true", 2024-09-20T18:50:44.468 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_memory_target": "2147483648", 2024-09-20T18:50:44.468 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_mgr_beacon_grace": "30", 2024-09-20T18:50:44.468 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-20T18:50:44.468 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_mgr_digest_period": "5", 2024-09-20T18:50:44.468 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_mgr_inactive_grace": "60", 2024-09-20T18:50:44.469 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-20T18:50:44.469 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-20T18:50:44.469 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_min_osdmap_epochs": "50", 2024-09-20T18:50:44.469 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_nvmeofgw_beacon_grace": "10", 2024-09-20T18:50:44.469 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_nvmeofgw_set_group_id_retry": "1000", 2024-09-20T18:50:44.469 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_op_complaint_time": "30", 2024-09-20T18:50:44.469 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_op_history_duration": "600", 2024-09-20T18:50:44.469 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_op_history_size": "20", 2024-09-20T18:50:44.469 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_op_history_slow_op_size": "20", 2024-09-20T18:50:44.469 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-20T18:50:44.469 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_op_log_threshold": "5", 2024-09-20T18:50:44.469 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-20T18:50:44.469 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-20T18:50:44.469 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-20T18:50:44.470 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_auto_mark_in": "false", 2024-09-20T18:50:44.470 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-20T18:50:44.470 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-20T18:50:44.470 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-20T18:50:44.470 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_cache_size": "500", 2024-09-20T18:50:44.470 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-20T18:50:44.470 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-20T18:50:44.470 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-20T18:50:44.470 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_down_out_interval": "600", 2024-09-20T18:50:44.470 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-20T18:50:44.470 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-20T18:50:44.470 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_force_trim_to": "0", 2024-09-20T18:50:44.470 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-20T18:50:44.470 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-20T18:50:44.471 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-20T18:50:44.471 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-20T18:50:44.471 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-20T18:50:44.471 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-20T18:50:44.471 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-20T18:50:44.471 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_min_down_reporters": "2", 2024-09-20T18:50:44.471 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-20T18:50:44.471 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-20T18:50:44.471 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-20T18:50:44.471 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-20T18:50:44.471 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-20T18:50:44.471 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-20T18:50:44.471 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_report_timeout": "900", 2024-09-20T18:50:44.471 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-20T18:50:44.472 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-20T18:50:44.472 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-20T18:50:44.472 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-20T18:50:44.472 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-20T18:50:44.472 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osdmap_full_prune_interval": "2", 2024-09-20T18:50:44.472 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osdmap_full_prune_min": "15", 2024-09-20T18:50:44.472 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_osdmap_full_prune_txsize": "2", 2024-09-20T18:50:44.472 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-20T18:50:44.472 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_pg_stuck_threshold": "60", 2024-09-20T18:50:44.472 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-20T18:50:44.472 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-20T18:50:44.472 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-20T18:50:44.473 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-20T18:50:44.473 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-20T18:50:44.473 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-20T18:50:44.473 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_probe_timeout": "2.000000", 2024-09-20T18:50:44.473 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_reweight_max_change": "0.050000", 2024-09-20T18:50:44.473 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_reweight_max_osds": "4", 2024-09-20T18:50:44.473 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-20T18:50:44.473 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-20T18:50:44.473 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-20T18:50:44.473 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-20T18:50:44.473 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-20T18:50:44.473 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_scrub_interval": "86400", 2024-09-20T18:50:44.473 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_scrub_max_keys": "100", 2024-09-20T18:50:44.473 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_scrub_timeout": "300", 2024-09-20T18:50:44.474 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_session_timeout": "300", 2024-09-20T18:50:44.474 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_smart_report_timeout": "5", 2024-09-20T18:50:44.474 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_stat_smooth_intervals": "6", 2024-09-20T18:50:44.474 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-20T18:50:44.474 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_stretch_pool_min_size": "2", 2024-09-20T18:50:44.474 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_stretch_pool_size": "4", 2024-09-20T18:50:44.474 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-20T18:50:44.474 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-20T18:50:44.474 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_sync_debug": "false", 2024-09-20T18:50:44.474 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-20T18:50:44.474 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-20T18:50:44.474 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_sync_provider_kill_at": "0", 2024-09-20T18:50:44.474 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_sync_requester_kill_at": "0", 2024-09-20T18:50:44.474 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_sync_timeout": "60.000000", 2024-09-20T18:50:44.475 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_target_pg_per_osd": "100", 2024-09-20T18:50:44.475 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_tick_interval": "5", 2024-09-20T18:50:44.475 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_timecheck_interval": "300.000000", 2024-09-20T18:50:44.475 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-20T18:50:44.475 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_use_min_delay_socket": "false", 2024-09-20T18:50:44.475 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_warn_older_version_delay": "604800", 2024-09-20T18:50:44.475 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-20T18:50:44.475 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-20T18:50:44.475 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-20T18:50:44.475 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_warn_on_filestore_osds": "true", 2024-09-20T18:50:44.475 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-20T18:50:44.475 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-20T18:50:44.475 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-20T18:50:44.475 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_warn_on_misplaced": "false", 2024-09-20T18:50:44.476 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-20T18:50:44.476 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_warn_on_older_version": "true", 2024-09-20T18:50:44.476 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-20T18:50:44.476 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_warn_on_pool_no_app": "true", 2024-09-20T18:50:44.476 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_warn_on_pool_no_redundancy": "true", 2024-09-20T18:50:44.476 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-20T18:50:44.476 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-20T18:50:44.476 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-20T18:50:44.476 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-20T18:50:44.476 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-20T18:50:44.476 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-20T18:50:44.476 INFO:teuthology.orchestra.run.smithi083.stdout: "monmap": "", 2024-09-20T18:50:44.476 INFO:teuthology.orchestra.run.smithi083.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-20T18:50:44.476 INFO:teuthology.orchestra.run.smithi083.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-20T18:50:44.477 INFO:teuthology.orchestra.run.smithi083.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-20T18:50:44.477 INFO:teuthology.orchestra.run.smithi083.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-20T18:50:44.477 INFO:teuthology.orchestra.run.smithi083.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-20T18:50:44.477 INFO:teuthology.orchestra.run.smithi083.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-20T18:50:44.477 INFO:teuthology.orchestra.run.smithi083.stdout: "motr_tracing_enabled": "false", 2024-09-20T18:50:44.477 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_async_op_threads": "3", 2024-09-20T18:50:44.477 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-20T18:50:44.477 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_async_rdma_cm": "false", 2024-09-20T18:50:44.477 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_async_rdma_device_name": "", 2024-09-20T18:50:44.477 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_async_rdma_dscp": "96", 2024-09-20T18:50:44.477 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-20T18:50:44.477 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-20T18:50:44.477 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_async_rdma_local_gid": "", 2024-09-20T18:50:44.477 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-20T18:50:44.478 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_async_rdma_port_num": "1", 2024-09-20T18:50:44.478 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-20T18:50:44.478 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-20T18:50:44.478 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-20T18:50:44.478 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-20T18:50:44.478 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_async_rdma_sl": "3", 2024-09-20T18:50:44.478 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_async_rdma_support_srq": "true", 2024-09-20T18:50:44.478 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_async_rdma_type": "ib", 2024-09-20T18:50:44.478 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_async_reap_threshold": "5", 2024-09-20T18:50:44.478 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_bind_before_connect": "false", 2024-09-20T18:50:44.478 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_bind_ipv4": "true", 2024-09-20T18:50:44.478 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_bind_ipv6": "false", 2024-09-20T18:50:44.478 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_bind_msgr1": "false", 2024-09-20T18:50:44.478 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_bind_msgr2": "true", 2024-09-20T18:50:44.479 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_bind_port_max": "7568", 2024-09-20T18:50:44.479 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_bind_port_min": "6800", 2024-09-20T18:50:44.479 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-20T18:50:44.479 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_bind_retry_count": "3", 2024-09-20T18:50:44.479 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_bind_retry_delay": "5", 2024-09-20T18:50:44.479 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_blackhole_client": "false", 2024-09-20T18:50:44.479 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_blackhole_mds": "false", 2024-09-20T18:50:44.479 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_blackhole_mgr": "false", 2024-09-20T18:50:44.479 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_blackhole_mon": "false", 2024-09-20T18:50:44.479 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_blackhole_osd": "false", 2024-09-20T18:50:44.479 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_client_mode": "crc secure", 2024-09-20T18:50:44.479 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_client_throttle_retry_time_interval": "5000", 2024-09-20T18:50:44.479 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_cluster_mode": "crc secure", 2024-09-20T18:50:44.479 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_cluster_type": "", 2024-09-20T18:50:44.480 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_compress_secure": "false", 2024-09-20T18:50:44.480 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_connection_idle_timeout": "900", 2024-09-20T18:50:44.480 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_connection_ready_timeout": "10", 2024-09-20T18:50:44.480 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_crc_data": "true", 2024-09-20T18:50:44.480 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_crc_header": "true", 2024-09-20T18:50:44.480 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_die_on_bad_msg": "false", 2024-09-20T18:50:44.480 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_die_on_bug": "true", 2024-09-20T18:50:44.480 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_die_on_old_message": "true", 2024-09-20T18:50:44.480 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_die_on_skipped_message": "false", 2024-09-20T18:50:44.480 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-20T18:50:44.480 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-20T18:50:44.480 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_dpdk_coremask": "0xF", 2024-09-20T18:50:44.480 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-20T18:50:44.480 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-20T18:50:44.481 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_dpdk_enable_tso": "true", 2024-09-20T18:50:44.481 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-20T18:50:44.481 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-20T18:50:44.481 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_dpdk_hugepages": "", 2024-09-20T18:50:44.481 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-20T18:50:44.481 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-20T18:50:44.481 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_dpdk_lro": "true", 2024-09-20T18:50:44.481 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_dpdk_memory_channel": "4", 2024-09-20T18:50:44.481 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-20T18:50:44.481 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_dpdk_pmd": "", 2024-09-20T18:50:44.481 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_dpdk_port_id": "0", 2024-09-20T18:50:44.481 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-20T18:50:44.481 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-20T18:50:44.481 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_dump_on_send": "false", 2024-09-20T18:50:44.482 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_initial_backoff": "0.200000", 2024-09-20T18:50:44.482 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_inject_delay_max": "1.000000", 2024-09-20T18:50:44.482 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-20T18:50:44.482 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_inject_delay_type": "", 2024-09-20T18:50:44.482 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-20T18:50:44.482 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_inject_network_congestion": "0", 2024-09-20T18:50:44.482 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_inject_socket_failures": "0", 2024-09-20T18:50:44.482 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_learn_addr_from_peer": "true", 2024-09-20T18:50:44.482 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_max_accept_failures": "4", 2024-09-20T18:50:44.482 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_max_backoff": "15.000000", 2024-09-20T18:50:44.482 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_mon_client_mode": "secure crc", 2024-09-20T18:50:44.482 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-20T18:50:44.482 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_mon_service_mode": "secure crc", 2024-09-20T18:50:44.482 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_osd_compress_min_size": "1024", 2024-09-20T18:50:44.483 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_osd_compress_mode": "none", 2024-09-20T18:50:44.483 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-20T18:50:44.483 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-20T18:50:44.483 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_pq_min_cost": "65536", 2024-09-20T18:50:44.483 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_public_type": "", 2024-09-20T18:50:44.483 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_service_mode": "crc secure", 2024-09-20T18:50:44.483 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_tcp_listen_backlog": "512", 2024-09-20T18:50:44.483 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_tcp_nodelay": "true", 2024-09-20T18:50:44.483 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-20T18:50:44.483 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_tcp_rcvbuf": "0", 2024-09-20T18:50:44.483 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_time_events_min_wait_interval": "1000", 2024-09-20T18:50:44.483 INFO:teuthology.orchestra.run.smithi083.stdout: "ms_type": "async+posix", 2024-09-20T18:50:44.483 INFO:teuthology.orchestra.run.smithi083.stdout: "no_config_file": "false", 2024-09-20T18:50:44.483 INFO:teuthology.orchestra.run.smithi083.stdout: "nvmeof_mon_client_disconnect_panic": "100", 2024-09-20T18:50:44.483 INFO:teuthology.orchestra.run.smithi083.stdout: "nvmeof_mon_client_tick_period": "2", 2024-09-20T18:50:44.484 INFO:teuthology.orchestra.run.smithi083.stdout: "objecter_completion_locks_per_session": "32", 2024-09-20T18:50:44.484 INFO:teuthology.orchestra.run.smithi083.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-20T18:50:44.484 INFO:teuthology.orchestra.run.smithi083.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-20T18:50:44.484 INFO:teuthology.orchestra.run.smithi083.stdout: "objecter_inflight_ops": "1024", 2024-09-20T18:50:44.484 INFO:teuthology.orchestra.run.smithi083.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-20T18:50:44.484 INFO:teuthology.orchestra.run.smithi083.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-20T18:50:44.484 INFO:teuthology.orchestra.run.smithi083.stdout: "objecter_tick_interval": "5.000000", 2024-09-20T18:50:44.484 INFO:teuthology.orchestra.run.smithi083.stdout: "objecter_timeout": "10.000000", 2024-09-20T18:50:44.484 INFO:teuthology.orchestra.run.smithi083.stdout: "objectstore_blackhole": "false", 2024-09-20T18:50:44.484 INFO:teuthology.orchestra.run.smithi083.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-20T18:50:44.484 INFO:teuthology.orchestra.run.smithi083.stdout: "openssl_engine_opts": "", 2024-09-20T18:50:44.484 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_agent_delay_time": "5.000000", 2024-09-20T18:50:44.484 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_agent_hist_halflife": "1000", 2024-09-20T18:50:44.484 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_agent_max_low_ops": "2", 2024-09-20T18:50:44.484 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_agent_max_ops": "4", 2024-09-20T18:50:44.485 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-20T18:50:44.485 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-20T18:50:44.485 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_agent_slop": "0.020000", 2024-09-20T18:50:44.485 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-20T18:50:44.485 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-20T18:50:44.485 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_asio_thread_count": "2", 2024-09-20T18:50:44.485 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_async_recovery_min_cost": "100", 2024-09-20T18:50:44.485 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-20T18:50:44.485 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-20T18:50:44.485 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_backfill_scan_max": "512", 2024-09-20T18:50:44.485 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_backfill_scan_min": "64", 2024-09-20T18:50:44.485 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_backoff_on_degraded": "false", 2024-09-20T18:50:44.485 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_backoff_on_peering": "false", 2024-09-20T18:50:44.485 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_backoff_on_unfound": "true", 2024-09-20T18:50:44.486 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_beacon_report_interval": "300", 2024-09-20T18:50:44.486 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bench_duration": "30", 2024-09-20T18:50:44.486 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-20T18:50:44.486 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bench_max_block_size": "67108864", 2024-09-20T18:50:44.486 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-20T18:50:44.486 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_blkin_trace_all": "false", 2024-09-20T18:50:44.486 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-20T18:50:44.486 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-20T18:50:44.486 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-20T18:50:44.486 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-20T18:50:44.486 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_check_for_log_corruption": "false", 2024-09-20T18:50:44.486 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-20T18:50:44.486 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-20T18:50:44.487 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-09-20T18:50:44.487 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-20T18:50:44.487 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_class_update_on_start": "true", 2024-09-20T18:50:44.487 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_client_message_cap": "256", 2024-09-20T18:50:44.487 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_client_message_size_cap": "524288000", 2024-09-20T18:50:44.487 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_client_op_priority": "63", 2024-09-20T18:50:44.487 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_client_watch_timeout": "30", 2024-09-20T18:50:44.487 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_command_max_records": "256", 2024-09-20T18:50:44.487 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_compact_on_start": "false", 2024-09-20T18:50:44.487 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-20T18:50:44.487 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_crush_chooseleaf_type": "1", 2024-09-20T18:50:44.487 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-20T18:50:44.487 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_crush_update_on_start": "true", 2024-09-20T18:50:44.487 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_crush_update_weight_set": "true", 2024-09-20T18:50:44.487 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-20T18:50:44.488 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-20T18:50:44.488 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-20T18:50:44.488 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-20T18:50:44.488 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-20T18:50:44.488 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_debug_feed_pullee": "-1", 2024-09-20T18:50:44.488 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-20T18:50:44.488 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-20T18:50:44.488 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-20T18:50:44.488 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_debug_misdirected_ops": "false", 2024-09-20T18:50:44.488 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_debug_no_acting_change": "false", 2024-09-20T18:50:44.488 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_debug_no_purge_strays": "false", 2024-09-20T18:50:44.488 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_debug_op_order": "false", 2024-09-20T18:50:44.488 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-20T18:50:44.488 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-20T18:50:44.488 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-20T18:50:44.489 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-20T18:50:44.489 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_debug_shutdown": "false", 2024-09-20T18:50:44.489 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-20T18:50:44.489 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-20T18:50:44.489 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_debug_trim_objects": "false", 2024-09-20T18:50:44.489 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-20T18:50:44.489 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-20T18:50:44.489 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_debug_verify_snaps": "false", 2024-09-20T18:50:44.489 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-20T18:50:44.489 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-20T18:50:44.489 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_deep_scrub_keys": "1024", 2024-09-20T18:50:44.489 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-20T18:50:44.489 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-20T18:50:44.489 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-20T18:50:44.490 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_deep_scrub_stride": "524288", 2024-09-20T18:50:44.490 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-20T18:50:44.490 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_default_data_pool_replay_window": "45", 2024-09-20T18:50:44.490 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_default_notify_timeout": "30", 2024-09-20T18:50:44.490 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_delete_sleep": "0.000000", 2024-09-20T18:50:44.490 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-20T18:50:44.490 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-20T18:50:44.490 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-20T18:50:44.490 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_discard_disconnected_ops": "true", 2024-09-20T18:50:44.490 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_ec_partial_reads": "true", 2024-09-20T18:50:44.490 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_enable_op_tracker": "true", 2024-09-20T18:50:44.490 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-20T18:50:44.490 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-20T18:50:44.490 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-20T18:50:44.491 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-20T18:50:44.491 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_fast_info": "true", 2024-09-20T18:50:44.491 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_fast_shutdown": "true", 2024-09-20T18:50:44.491 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-20T18:50:44.491 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-20T18:50:44.491 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-20T18:50:44.491 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-20T18:50:44.491 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-20T18:50:44.491 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_function_tracing": "false", 2024-09-20T18:50:44.491 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_heartbeat_grace": "20", 2024-09-20T18:50:44.491 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_heartbeat_interval": "6", 2024-09-20T18:50:44.491 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-20T18:50:44.491 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_heartbeat_min_peers": "10", 2024-09-20T18:50:44.491 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_heartbeat_min_size": "2000", 2024-09-20T18:50:44.491 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_heartbeat_stale": "600", 2024-09-20T18:50:44.492 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-20T18:50:44.492 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_hit_set_max_size": "100000", 2024-09-20T18:50:44.492 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_hit_set_min_size": "1000", 2024-09-20T18:50:44.492 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-20T18:50:44.492 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-20T18:50:44.492 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-20T18:50:44.492 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-20T18:50:44.492 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-20T18:50:44.492 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-20T18:50:44.492 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_journal_size": "5120", 2024-09-20T18:50:44.492 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_kill_backfill_at": "0", 2024-09-20T18:50:44.492 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-20T18:50:44.492 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_map_cache_size": "50", 2024-09-20T18:50:44.492 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_map_dedup": "true", 2024-09-20T18:50:44.493 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_map_message_max": "40", 2024-09-20T18:50:44.493 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-20T18:50:44.493 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_map_share_max_epochs": "40", 2024-09-20T18:50:44.493 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_max_attr_name_len": "100", 2024-09-20T18:50:44.493 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_max_attr_size": "0", 2024-09-20T18:50:44.493 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_max_backfills": "1", 2024-09-20T18:50:44.493 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_max_markdown_count": "5", 2024-09-20T18:50:44.493 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_max_markdown_period": "600", 2024-09-20T18:50:44.493 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_max_object_name_len": "2048", 2024-09-20T18:50:44.493 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_max_object_namespace_len": "256", 2024-09-20T18:50:44.493 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_max_object_size": "134217728", 2024-09-20T18:50:44.493 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-20T18:50:44.493 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-20T18:50:44.494 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_max_pg_blocked_by": "16", 2024-09-20T18:50:44.494 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_max_pg_log_entries": "10000", 2024-09-20T18:50:44.494 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-20T18:50:44.494 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_max_pgls": "1024", 2024-09-20T18:50:44.494 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_max_push_cost": "8388608", 2024-09-20T18:50:44.494 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_max_push_objects": "10", 2024-09-20T18:50:44.494 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_max_scrubs": "3", 2024-09-20T18:50:44.494 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-20T18:50:44.494 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_max_trimming_pgs": "2", 2024-09-20T18:50:44.494 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_max_write_op_reply_len": "64", 2024-09-20T18:50:44.494 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_max_write_size": "90", 2024-09-20T18:50:44.494 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-20T18:50:44.494 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-20T18:50:44.494 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-20T18:50:44.495 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-20T18:50:44.495 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-20T18:50:44.495 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-20T18:50:44.495 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-20T18:50:44.495 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-20T18:50:44.495 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mclock_profile": "balanced", 2024-09-20T18:50:44.495 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-20T18:50:44.495 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-20T18:50:44.495 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-20T18:50:44.495 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-20T18:50:44.495 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-20T18:50:44.495 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-20T18:50:44.496 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-20T18:50:44.496 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-20T18:50:44.496 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-20T18:50:44.496 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-20T18:50:44.496 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-20T18:50:44.496 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_memory_base": "805306368", 2024-09-20T18:50:44.496 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_memory_cache_min": "134217728", 2024-09-20T18:50:44.496 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-20T18:50:44.496 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-20T18:50:44.496 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_memory_target": "4294967296", 2024-09-20T18:50:44.496 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_memory_target_autotune": "false", 2024-09-20T18:50:44.496 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-20T18:50:44.496 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_min_pg_log_entries": "250", 2024-09-20T18:50:44.496 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_min_recovery_priority": "0", 2024-09-20T18:50:44.497 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-20T18:50:44.497 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-20T18:50:44.497 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mon_report_interval": "5", 2024-09-20T18:50:44.497 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-20T18:50:44.497 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-20T18:50:44.497 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_num_cache_shards": "32", 2024-09-20T18:50:44.497 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_num_op_tracker_shard": "32", 2024-09-20T18:50:44.497 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_numa_auto_affinity": "true", 2024-09-20T18:50:44.497 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_numa_node": "-1", 2024-09-20T18:50:44.497 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_numa_prefer_iface": "true", 2024-09-20T18:50:44.497 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-20T18:50:44.497 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_objecter_finishers": "1", 2024-09-20T18:50:44.497 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_objectstore": "bluestore", 2024-09-20T18:50:44.497 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_objectstore_fuse": "false", 2024-09-20T18:50:44.498 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_objectstore_ideal_list_max": "64", 2024-09-20T18:50:44.498 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_objectstore_tracing": "false", 2024-09-20T18:50:44.498 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_op_complaint_time": "30.000000", 2024-09-20T18:50:44.498 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_op_history_duration": "600", 2024-09-20T18:50:44.498 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_op_history_size": "20", 2024-09-20T18:50:44.498 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_op_history_slow_op_size": "20", 2024-09-20T18:50:44.498 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-20T18:50:44.498 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_op_log_threshold": "5", 2024-09-20T18:50:44.498 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_op_num_shards": "0", 2024-09-20T18:50:44.498 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_op_num_shards_hdd": "5", 2024-09-20T18:50:44.498 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_op_num_shards_ssd": "8", 2024-09-20T18:50:44.498 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-20T18:50:44.498 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-20T18:50:44.498 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-20T18:50:44.498 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-20T18:50:44.499 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_op_pq_min_cost": "65536", 2024-09-20T18:50:44.499 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-20T18:50:44.499 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_op_queue_cut_off": "high", 2024-09-20T18:50:44.499 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-20T18:50:44.499 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_op_thread_timeout": "15", 2024-09-20T18:50:44.499 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_open_classes_on_start": "true", 2024-09-20T18:50:44.499 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_os_flags": "0", 2024-09-20T18:50:44.499 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_peering_op_priority": "255", 2024-09-20T18:50:44.499 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pg_delete_cost": "1048576", 2024-09-20T18:50:44.499 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pg_delete_priority": "5", 2024-09-20T18:50:44.499 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-20T18:50:44.499 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-20T18:50:44.499 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-20T18:50:44.499 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pg_log_trim_max": "10000", 2024-09-20T18:50:44.500 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pg_log_trim_min": "100", 2024-09-20T18:50:44.500 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-20T18:50:44.500 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-20T18:50:44.500 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-20T18:50:44.500 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-20T18:50:44.500 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-20T18:50:44.500 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-20T18:50:44.500 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-20T18:50:44.500 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-20T18:50:44.500 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-20T18:50:44.500 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-20T18:50:44.500 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_default_crimson": "false", 2024-09-20T18:50:44.500 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-20T18:50:44.500 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-20T18:50:44.501 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-20T18:50:44.501 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-20T18:50:44.501 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-20T18:50:44.501 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-20T18:50:44.501 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-20T18:50:44.501 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-20T18:50:44.501 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_default_flags": "0", 2024-09-20T18:50:44.501 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-20T18:50:44.501 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_default_min_size": "0", 2024-09-20T18:50:44.501 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-20T18:50:44.501 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_default_pg_num": "32", 2024-09-20T18:50:44.501 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_default_pgp_num": "0", 2024-09-20T18:50:44.501 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-20T18:50:44.502 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_default_read_ratio": "70", 2024-09-20T18:50:44.502 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_default_size": "3", 2024-09-20T18:50:44.502 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_default_type": "replicated", 2024-09-20T18:50:44.502 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-20T18:50:44.502 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-20T18:50:44.502 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_push_per_object_cost": "1000", 2024-09-20T18:50:44.502 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-20T18:50:44.502 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_recover_clone_overlap": "true", 2024-09-20T18:50:44.502 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-20T18:50:44.502 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_recovery_cost": "20971520", 2024-09-20T18:50:44.502 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-20T18:50:44.502 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_recovery_max_active": "0", 2024-09-20T18:50:44.502 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-20T18:50:44.502 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-20T18:50:44.503 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-20T18:50:44.503 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-20T18:50:44.503 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_recovery_max_single_start": "1", 2024-09-20T18:50:44.503 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_recovery_op_priority": "3", 2024-09-20T18:50:44.503 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-20T18:50:44.503 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_recovery_priority": "5", 2024-09-20T18:50:44.503 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-20T18:50:44.503 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_recovery_sleep": "0.000000", 2024-09-20T18:50:44.503 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-20T18:50:44.503 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-20T18:50:44.503 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-20T18:50:44.503 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_repair_during_recovery": "false", 2024-09-20T18:50:44.503 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_requested_scrub_priority": "5", 2024-09-20T18:50:44.503 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-20T18:50:44.504 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-20T18:50:44.504 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_auto_repair": "false", 2024-09-20T18:50:44.504 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-20T18:50:44.504 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-20T18:50:44.504 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_begin_hour": "0", 2024-09-20T18:50:44.504 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_begin_week_day": "0", 2024-09-20T18:50:44.504 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_chunk_max": "15", 2024-09-20T18:50:44.504 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_chunk_min": "5", 2024-09-20T18:50:44.504 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_cost": "52428800", 2024-09-20T18:50:44.504 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-20T18:50:44.504 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_during_recovery": "false", 2024-09-20T18:50:44.504 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_end_hour": "0", 2024-09-20T18:50:44.504 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_end_week_day": "0", 2024-09-20T18:50:44.505 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_event_cost": "4096", 2024-09-20T18:50:44.505 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-20T18:50:44.505 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-20T18:50:44.505 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_invalid_stats": "true", 2024-09-20T18:50:44.505 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-20T18:50:44.505 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-20T18:50:44.505 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_max_preemptions": "5", 2024-09-20T18:50:44.505 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-20T18:50:44.505 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_priority": "5", 2024-09-20T18:50:44.505 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_retry_after_noscrub": "60", 2024-09-20T18:50:44.505 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_retry_delay": "30", 2024-09-20T18:50:44.505 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_retry_new_interval": "10", 2024-09-20T18:50:44.505 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_retry_pg_state": "60", 2024-09-20T18:50:44.505 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_retry_trimming": "10", 2024-09-20T18:50:44.505 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_scrub_sleep": "0.000000", 2024-09-20T18:50:44.506 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-20T18:50:44.506 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-20T18:50:44.506 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-20T18:50:44.506 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_skip_check_past_interval_bounds": "false", 2024-09-20T18:50:44.506 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_skip_data_digest": "false", 2024-09-20T18:50:44.506 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_smart_report_timeout": "5", 2024-09-20T18:50:44.506 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_snap_trim_cost": "1048576", 2024-09-20T18:50:44.506 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_snap_trim_priority": "5", 2024-09-20T18:50:44.506 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-20T18:50:44.506 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-20T18:50:44.506 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-20T18:50:44.506 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-20T18:50:44.506 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-20T18:50:44.506 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-20T18:50:44.507 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-20T18:50:44.507 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_target_transaction_size": "30", 2024-09-20T18:50:44.507 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-20T18:50:44.507 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-20T18:50:44.507 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-20T18:50:44.507 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-20T18:50:44.507 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-20T18:50:44.507 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-20T18:50:44.507 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-20T18:50:44.507 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-20T18:50:44.507 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-20T18:50:44.507 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-20T18:50:44.507 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_tracing": "false", 2024-09-20T18:50:44.508 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_use_stale_snap": "false", 2024-09-20T18:50:44.508 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-20T18:50:44.508 INFO:teuthology.orchestra.run.smithi083.stdout: "osdc_blkin_trace_all": "false", 2024-09-20T18:50:44.508 INFO:teuthology.orchestra.run.smithi083.stdout: "paxos_kill_at": "0", 2024-09-20T18:50:44.508 INFO:teuthology.orchestra.run.smithi083.stdout: "paxos_max_join_drift": "10", 2024-09-20T18:50:44.508 INFO:teuthology.orchestra.run.smithi083.stdout: "paxos_min": "500", 2024-09-20T18:50:44.508 INFO:teuthology.orchestra.run.smithi083.stdout: "paxos_min_wait": "0.050000", 2024-09-20T18:50:44.508 INFO:teuthology.orchestra.run.smithi083.stdout: "paxos_propose_interval": "1.000000", 2024-09-20T18:50:44.508 INFO:teuthology.orchestra.run.smithi083.stdout: "paxos_service_trim_max": "500", 2024-09-20T18:50:44.508 INFO:teuthology.orchestra.run.smithi083.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-20T18:50:44.508 INFO:teuthology.orchestra.run.smithi083.stdout: "paxos_service_trim_min": "10", 2024-09-20T18:50:44.508 INFO:teuthology.orchestra.run.smithi083.stdout: "paxos_stash_full_interval": "25", 2024-09-20T18:50:44.508 INFO:teuthology.orchestra.run.smithi083.stdout: "paxos_trim_max": "500", 2024-09-20T18:50:44.508 INFO:teuthology.orchestra.run.smithi083.stdout: "paxos_trim_min": "250", 2024-09-20T18:50:44.509 INFO:teuthology.orchestra.run.smithi083.stdout: "perf": "true", 2024-09-20T18:50:44.509 INFO:teuthology.orchestra.run.smithi083.stdout: "pid_file": "", 2024-09-20T18:50:44.509 INFO:teuthology.orchestra.run.smithi083.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-20T18:50:44.509 INFO:teuthology.orchestra.run.smithi083.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-09-20T18:50:44.509 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "-", 2024-09-20T18:50:44.509 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrv": "", 2024-09-20T18:50:44.509 INFO:teuthology.orchestra.run.smithi083.stdout: "public_bind_addr": "-", 2024-09-20T18:50:44.509 INFO:teuthology.orchestra.run.smithi083.stdout: "public_network": "", 2024-09-20T18:50:44.509 INFO:teuthology.orchestra.run.smithi083.stdout: "public_network_interface": "", 2024-09-20T18:50:44.509 INFO:teuthology.orchestra.run.smithi083.stdout: "qat_compressor_busy_polling": "false", 2024-09-20T18:50:44.509 INFO:teuthology.orchestra.run.smithi083.stdout: "qat_compressor_enabled": "false", 2024-09-20T18:50:44.509 INFO:teuthology.orchestra.run.smithi083.stdout: "qat_compressor_session_max_number": "256", 2024-09-20T18:50:44.509 INFO:teuthology.orchestra.run.smithi083.stdout: "rados_mon_op_timeout": "0", 2024-09-20T18:50:44.509 INFO:teuthology.orchestra.run.smithi083.stdout: "rados_osd_op_timeout": "0", 2024-09-20T18:50:44.509 INFO:teuthology.orchestra.run.smithi083.stdout: "rados_tracing": "false", 2024-09-20T18:50:44.510 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_atime_update_interval": "60", 2024-09-20T18:50:44.510 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-20T18:50:44.510 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_balance_parent_reads": "false", 2024-09-20T18:50:44.510 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_balance_snap_reads": "false", 2024-09-20T18:50:44.510 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_blkin_trace_all": "false", 2024-09-20T18:50:44.510 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-20T18:50:44.510 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-20T18:50:44.510 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_cache": "true", 2024-09-20T18:50:44.510 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-20T18:50:44.510 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-20T18:50:44.510 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-20T18:50:44.510 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-20T18:50:44.510 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_cache_policy": "writearound", 2024-09-20T18:50:44.510 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_cache_size": "33554432", 2024-09-20T18:50:44.511 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-20T18:50:44.511 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-20T18:50:44.511 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_clone_copy_on_read": "false", 2024-09-20T18:50:44.511 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_compression_hint": "none", 2024-09-20T18:50:44.511 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_concurrent_management_ops": "10", 2024-09-20T18:50:44.511 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-20T18:50:44.511 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_default_clone_format": "auto", 2024-09-20T18:50:44.511 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_default_data_pool": "", 2024-09-20T18:50:44.511 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_default_features": "61", 2024-09-20T18:50:44.511 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_default_format": "2", 2024-09-20T18:50:44.511 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_default_map_options": "", 2024-09-20T18:50:44.511 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_default_order": "22", 2024-09-20T18:50:44.511 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_default_pool": "rbd", 2024-09-20T18:50:44.511 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-20T18:50:44.512 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_default_stripe_count": "0", 2024-09-20T18:50:44.512 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_default_stripe_unit": "0", 2024-09-20T18:50:44.512 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-20T18:50:44.512 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-20T18:50:44.512 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-20T18:50:44.512 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_enable_alloc_hint": "true", 2024-09-20T18:50:44.512 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-20T18:50:44.512 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_io_scheduler": "simple", 2024-09-20T18:50:44.512 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-20T18:50:44.512 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-20T18:50:44.512 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-20T18:50:44.512 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-20T18:50:44.512 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-20T18:50:44.512 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-20T18:50:44.512 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-20T18:50:44.513 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-20T18:50:44.513 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-20T18:50:44.513 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_journal_order": "24", 2024-09-20T18:50:44.513 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_journal_pool": "", 2024-09-20T18:50:44.513 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_journal_splay_width": "4", 2024-09-20T18:50:44.513 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_localize_parent_reads": "false", 2024-09-20T18:50:44.513 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_localize_snap_reads": "false", 2024-09-20T18:50:44.514 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-20T18:50:44.514 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-20T18:50:44.514 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-20T18:50:44.514 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-20T18:50:44.514 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-20T18:50:44.514 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-20T18:50:44.514 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-20T18:50:44.514 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-20T18:50:44.515 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-20T18:50:44.515 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-20T18:50:44.515 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-20T18:50:44.515 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-20T18:50:44.515 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-20T18:50:44.515 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-20T18:50:44.515 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-20T18:50:44.515 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-20T18:50:44.515 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-20T18:50:44.515 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-20T18:50:44.515 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-20T18:50:44.515 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-20T18:50:44.515 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-20T18:50:44.515 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-20T18:50:44.516 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-20T18:50:44.516 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-20T18:50:44.516 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-20T18:50:44.516 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-20T18:50:44.516 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-20T18:50:44.516 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-20T18:50:44.516 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-20T18:50:44.516 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-20T18:50:44.516 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-20T18:50:44.516 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_mtime_update_interval": "60", 2024-09-20T18:50:44.516 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_non_blocking_aio": "true", 2024-09-20T18:50:44.516 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_op_thread_timeout": "60", 2024-09-20T18:50:44.516 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_op_threads": "1", 2024-09-20T18:50:44.517 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_parent_cache_enabled": "false", 2024-09-20T18:50:44.517 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-20T18:50:44.517 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-20T18:50:44.517 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-20T18:50:44.517 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_plugins": "", 2024-09-20T18:50:44.517 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_qos_bps_burst": "0", 2024-09-20T18:50:44.517 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-20T18:50:44.517 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_qos_bps_limit": "0", 2024-09-20T18:50:44.517 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_qos_exclude_ops": "0", 2024-09-20T18:50:44.517 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_qos_iops_burst": "0", 2024-09-20T18:50:44.517 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-20T18:50:44.517 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_qos_iops_limit": "0", 2024-09-20T18:50:44.517 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-20T18:50:44.517 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-20T18:50:44.517 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-20T18:50:44.518 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-20T18:50:44.518 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-20T18:50:44.518 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-20T18:50:44.518 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-20T18:50:44.518 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-20T18:50:44.518 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-20T18:50:44.518 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-20T18:50:44.518 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-20T18:50:44.518 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-20T18:50:44.518 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-20T18:50:44.518 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-20T18:50:44.518 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_read_from_replica_policy": "default", 2024-09-20T18:50:44.518 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-20T18:50:44.519 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-20T18:50:44.519 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-20T18:50:44.519 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-20T18:50:44.519 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_skip_partial_discard": "true", 2024-09-20T18:50:44.519 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-20T18:50:44.519 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_tracing": "false", 2024-09-20T18:50:44.519 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_validate_names": "true", 2024-09-20T18:50:44.519 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_validate_pool": "true", 2024-09-20T18:50:44.519 INFO:teuthology.orchestra.run.smithi083.stdout: "restapi_base_url": "", 2024-09-20T18:50:44.519 INFO:teuthology.orchestra.run.smithi083.stdout: "restapi_log_level": "", 2024-09-20T18:50:44.519 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-20T18:50:44.519 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-20T18:50:44.519 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_acl_grants_max_num": "100", 2024-09-20T18:50:44.520 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_admin_entry": "admin", 2024-09-20T18:50:44.520 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-20T18:50:44.520 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_asio_assert_yielding": "false", 2024-09-20T18:50:44.520 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_backend_store": "rados", 2024-09-20T18:50:44.520 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_barbican_url": "", 2024-09-20T18:50:44.520 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_beast_enable_async": "true", 2024-09-20T18:50:44.520 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_bucket_counters_cache": "false", 2024-09-20T18:50:44.520 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-20T18:50:44.520 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-20T18:50:44.520 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-20T18:50:44.520 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-20T18:50:44.520 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-20T18:50:44.520 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-20T18:50:44.520 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-20T18:50:44.521 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-20T18:50:44.521 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_cache_enabled": "true", 2024-09-20T18:50:44.521 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_cache_expiry_interval": "900", 2024-09-20T18:50:44.521 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_cache_lru_size": "25000", 2024-09-20T18:50:44.521 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_config_store": "rados", 2024-09-20T18:50:44.521 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_content_length_compat": "false", 2024-09-20T18:50:44.521 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_copy_obj_progress": "true", 2024-09-20T18:50:44.521 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-20T18:50:44.521 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_cors_rules_max_num": "100", 2024-09-20T18:50:44.521 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_cross_domain_policy": "", 2024-09-20T18:50:44.521 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-20T18:50:44.521 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_kmip_addr": "", 2024-09-20T18:50:44.521 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-20T18:50:44.522 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-20T18:50:44.522 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-20T18:50:44.522 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-20T18:50:44.522 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_kmip_password": "", 2024-09-20T18:50:44.522 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-20T18:50:44.522 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_kmip_username": "", 2024-09-20T18:50:44.522 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_require_ssl": "true", 2024-09-20T18:50:44.522 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-20T18:50:44.522 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-20T18:50:44.522 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-20T18:50:44.522 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-20T18:50:44.522 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-20T18:50:44.522 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-20T18:50:44.522 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-20T18:50:44.523 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-20T18:50:44.523 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-20T18:50:44.523 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-20T18:50:44.523 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-20T18:50:44.523 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-20T18:50:44.523 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-20T18:50:44.523 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-20T18:50:44.523 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-20T18:50:44.523 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_vault_addr": "", 2024-09-20T18:50:44.523 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_vault_auth": "token", 2024-09-20T18:50:44.523 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_vault_namespace": "", 2024-09-20T18:50:44.523 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_vault_prefix": "", 2024-09-20T18:50:44.523 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-20T18:50:44.523 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-20T18:50:44.523 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-20T18:50:44.524 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-20T18:50:44.524 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_vault_token_file": "", 2024-09-20T18:50:44.524 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-20T18:50:44.524 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_curl_buffersize": "524288", 2024-09-20T18:50:44.524 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-20T18:50:44.524 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_curl_low_speed_time": "300", 2024-09-20T18:50:44.524 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-20T18:50:44.524 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-20T18:50:44.524 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-20T18:50:44.524 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-20T18:50:44.524 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-20T18:50:44.524 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-20T18:50:44.524 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-20T18:50:44.525 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-20T18:50:44.525 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-20T18:50:44.525 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-20T18:50:44.525 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_d4n_address": "127.0.0.1:6379", 2024-09-20T18:50:44.525 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_d4n_l1_datacache_address": "127.0.0.1:6379", 2024-09-20T18:50:44.525 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_d4n_l1_datacache_persistent_path": "/tmp/rgw_d4n_datacache/", 2024-09-20T18:50:44.525 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_d4n_l1_datacache_size": "1073741824", 2024-09-20T18:50:44.525 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_d4n_l1_evict_cache_on_start": "true", 2024-09-20T18:50:44.525 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_d4n_l1_fadvise": "4", 2024-09-20T18:50:44.525 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_d4n_libaio_aio_num": "64", 2024-09-20T18:50:44.525 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_d4n_libaio_aio_threads": "20", 2024-09-20T18:50:44.525 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-20T18:50:44.526 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_data_log_changes_size": "1000", 2024-09-20T18:50:44.526 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_data_log_num_shards": "128", 2024-09-20T18:50:44.526 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-20T18:50:44.526 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_data_log_window": "30", 2024-09-20T18:50:44.526 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-20T18:50:44.526 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-20T18:50:44.526 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-20T18:50:44.526 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-20T18:50:44.526 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-20T18:50:44.526 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-20T18:50:44.526 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-20T18:50:44.526 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-20T18:50:44.526 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-20T18:50:44.526 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-20T18:50:44.527 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-20T18:50:44.527 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-20T18:50:44.527 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-20T18:50:44.527 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_disable_s3select": "false", 2024-09-20T18:50:44.527 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-20T18:50:44.527 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-20T18:50:44.527 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-20T18:50:44.527 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-20T18:50:44.527 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-20T18:50:44.527 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-20T18:50:44.527 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-20T18:50:44.527 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-20T18:50:44.527 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-20T18:50:44.528 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-20T18:50:44.528 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-20T18:50:44.528 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-20T18:50:44.528 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_dns_name": "", 2024-09-20T18:50:44.528 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_dns_s3website_name": "", 2024-09-20T18:50:44.528 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_dynamic_resharding": "true", 2024-09-20T18:50:44.528 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_dynamic_resharding_may_reduce": "true", 2024-09-20T18:50:44.528 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_dynamic_resharding_reduction_wait": "120", 2024-09-20T18:50:44.528 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-20T18:50:44.528 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_enable_gc_threads": "true", 2024-09-20T18:50:44.528 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_enable_lc_threads": "true", 2024-09-20T18:50:44.528 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_enable_mdsearch": "true", 2024-09-20T18:50:44.528 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_enable_ops_log": "false", 2024-09-20T18:50:44.528 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_enable_quota_threads": "true", 2024-09-20T18:50:44.529 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_enable_static_website": "false", 2024-09-20T18:50:44.529 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_enable_usage_log": "false", 2024-09-20T18:50:44.529 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_enforce_swift_acls": "true", 2024-09-20T18:50:44.529 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_exit_timeout_secs": "120", 2024-09-20T18:50:44.529 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_expose_bucket": "false", 2024-09-20T18:50:44.529 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_extended_http_attrs": "", 2024-09-20T18:50:44.529 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_filter": "none", 2024-09-20T18:50:44.529 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-20T18:50:44.529 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_frontends": "beast port=7480", 2024-09-20T18:50:44.529 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-20T18:50:44.529 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_gc_max_deferred": "50", 2024-09-20T18:50:44.529 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-20T18:50:44.529 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_gc_max_objs": "32", 2024-09-20T18:50:44.529 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-20T18:50:44.529 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-20T18:50:44.530 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-20T18:50:44.530 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-20T18:50:44.530 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_gc_processor_period": "3600", 2024-09-20T18:50:44.530 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-20T18:50:44.530 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-20T18:50:44.530 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-20T18:50:44.530 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-20T18:50:44.530 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_init_timeout": "300", 2024-09-20T18:50:44.530 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-20T18:50:44.530 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-20T18:50:44.530 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_kafka_connection_idle": "300", 2024-09-20T18:50:44.530 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-20T18:50:44.530 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-20T18:50:44.530 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-20T18:50:44.531 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-20T18:50:44.531 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-20T18:50:44.531 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_keystone_admin_domain": "", 2024-09-20T18:50:44.531 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_keystone_admin_password": "", 2024-09-20T18:50:44.531 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_keystone_admin_password_path": "", 2024-09-20T18:50:44.531 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_keystone_admin_project": "", 2024-09-20T18:50:44.531 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_keystone_admin_tenant": "", 2024-09-20T18:50:44.531 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_keystone_admin_token": "", 2024-09-20T18:50:44.531 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_keystone_admin_token_path": "", 2024-09-20T18:50:44.531 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_keystone_admin_user": "", 2024-09-20T18:50:44.531 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_keystone_barbican_domain": "", 2024-09-20T18:50:44.531 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_keystone_barbican_password": "", 2024-09-20T18:50:44.531 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_keystone_barbican_project": "", 2024-09-20T18:50:44.532 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-20T18:50:44.532 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_keystone_barbican_user": "", 2024-09-20T18:50:44.532 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-20T18:50:44.532 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-20T18:50:44.532 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-20T18:50:44.532 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-20T18:50:44.532 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-20T18:50:44.532 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_keystone_url": "", 2024-09-20T18:50:44.532 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-20T18:50:44.532 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_lc_debug_interval": "-1", 2024-09-20T18:50:44.532 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_lc_lock_max_time": "90", 2024-09-20T18:50:44.532 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_lc_max_objs": "32", 2024-09-20T18:50:44.532 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_lc_max_rules": "1000", 2024-09-20T18:50:44.533 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_lc_max_worker": "3", 2024-09-20T18:50:44.533 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-20T18:50:44.533 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_lc_thread_delay": "0", 2024-09-20T18:50:44.533 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-20T18:50:44.533 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_ldap_dnattr": "uid", 2024-09-20T18:50:44.533 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-20T18:50:44.533 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_ldap_searchfilter": "", 2024-09-20T18:50:44.533 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-20T18:50:44.533 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_ldap_uri": "", 2024-09-20T18:50:44.533 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_lfuda_sync_frequency": "60", 2024-09-20T18:50:44.533 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-20T18:50:44.533 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-20T18:50:44.533 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-20T18:50:44.533 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_log_http_headers": "", 2024-09-20T18:50:44.534 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-20T18:50:44.534 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-20T18:50:44.534 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_log_object_name_utc": "false", 2024-09-20T18:50:44.534 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-20T18:50:44.534 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-20T18:50:44.534 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_max_attr_name_len": "0", 2024-09-20T18:50:44.534 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_max_attr_size": "0", 2024-09-20T18:50:44.534 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-20T18:50:44.534 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_max_chunk_size": "4194304", 2024-09-20T18:50:44.534 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-20T18:50:44.534 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-20T18:50:44.534 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-20T18:50:44.534 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_max_listing_results": "5000", 2024-09-20T18:50:44.535 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_max_notify_retries": "10", 2024-09-20T18:50:44.535 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-20T18:50:44.535 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_max_put_param_size": "1048576", 2024-09-20T18:50:44.535 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_max_put_size": "5368709120", 2024-09-20T18:50:44.535 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_max_slo_entries": "1000", 2024-09-20T18:50:44.535 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_md_log_max_shards": "64", 2024-09-20T18:50:44.535 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-20T18:50:44.535 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-20T18:50:44.535 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-20T18:50:44.535 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-20T18:50:44.535 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_mp_lock_max_time": "600", 2024-09-20T18:50:44.535 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-20T18:50:44.535 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-20T18:50:44.536 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-20T18:50:44.536 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-20T18:50:44.536 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-20T18:50:44.536 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-20T18:50:44.536 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-20T18:50:44.536 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-20T18:50:44.536 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_nfs_max_gc": "300", 2024-09-20T18:50:44.536 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-20T18:50:44.536 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-20T18:50:44.536 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-20T18:50:44.536 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-20T18:50:44.536 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-20T18:50:44.536 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-20T18:50:44.537 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-20T18:50:44.537 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_num_async_rados_threads": "32", 2024-09-20T18:50:44.537 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_num_control_oids": "8", 2024-09-20T18:50:44.537 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_numa_node": "-1", 2024-09-20T18:50:44.537 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-20T18:50:44.537 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-20T18:50:44.537 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_objexp_chunk_size": "100", 2024-09-20T18:50:44.537 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_objexp_gc_interval": "600", 2024-09-20T18:50:44.537 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-20T18:50:44.537 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-20T18:50:44.537 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-20T18:50:44.537 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_op_thread_timeout": "600", 2024-09-20T18:50:44.537 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_op_tracing": "false", 2024-09-20T18:50:44.537 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_opa_token": "", 2024-09-20T18:50:44.538 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_opa_url": "", 2024-09-20T18:50:44.538 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_opa_verify_ssl": "true", 2024-09-20T18:50:44.538 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-20T18:50:44.538 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-20T18:50:44.538 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_ops_log_rados": "false", 2024-09-20T18:50:44.538 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_ops_log_socket_path": "", 2024-09-20T18:50:44.538 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-20T18:50:44.538 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-20T18:50:44.538 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-20T18:50:44.538 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_period_push_interval": "2.000000", 2024-09-20T18:50:44.538 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-20T18:50:44.538 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-20T18:50:44.538 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-20T18:50:44.539 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-20T18:50:44.539 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_posix_cache_lanes": "3", 2024-09-20T18:50:44.539 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-20T18:50:44.539 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-20T18:50:44.539 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_posix_cache_partitions": "3", 2024-09-20T18:50:44.539 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-20T18:50:44.539 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_print_continue": "true", 2024-09-20T18:50:44.539 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-20T18:50:44.539 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-20T18:50:44.539 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-20T18:50:44.539 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-20T18:50:44.539 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-20T18:50:44.539 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_rados_tracing": "false", 2024-09-20T18:50:44.540 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_realm": "", 2024-09-20T18:50:44.540 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_realm_id": "", 2024-09-20T18:50:44.540 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-20T18:50:44.540 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_region": "", 2024-09-20T18:50:44.540 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-20T18:50:44.540 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-20T18:50:44.540 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-20T18:50:44.540 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-20T18:50:44.540 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_request_uri": "", 2024-09-20T18:50:44.540 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_reshard_batch_size": "64", 2024-09-20T18:50:44.540 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-20T18:50:44.540 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_reshard_debug_interval": "-1", 2024-09-20T18:50:44.540 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_reshard_max_aio": "128", 2024-09-20T18:50:44.540 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_reshard_num_logs": "16", 2024-09-20T18:50:44.541 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_reshard_progress_judge_interval": "120", 2024-09-20T18:50:44.541 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_reshard_progress_judge_ratio": "0.500000", 2024-09-20T18:50:44.541 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_reshard_thread_interval": "600", 2024-09-20T18:50:44.541 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_reshardlog_threshold": "30000", 2024-09-20T18:50:44.541 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_resolve_cname": "false", 2024-09-20T18:50:44.541 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-20T18:50:44.541 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_run_sync_thread": "true", 2024-09-20T18:50:44.541 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-20T18:50:44.541 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-20T18:50:44.541 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-20T18:50:44.541 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-20T18:50:44.541 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-20T18:50:44.541 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-20T18:50:44.541 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-20T18:50:44.542 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-20T18:50:44.542 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-20T18:50:44.542 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_scheduler_type": "throttler", 2024-09-20T18:50:44.542 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_script_uri": "", 2024-09-20T18:50:44.542 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_service_provider_name": "", 2024-09-20T18:50:44.542 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-20T18:50:44.542 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_sts_client_id": "", 2024-09-20T18:50:44.542 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_sts_client_secret": "", 2024-09-20T18:50:44.542 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_sts_entry": "sts", 2024-09-20T18:50:44.542 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_sts_key": "sts", 2024-09-20T18:50:44.542 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-20T18:50:44.542 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_sts_min_session_duration": "900", 2024-09-20T18:50:44.542 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_sts_token_introspection_url": "", 2024-09-20T18:50:44.542 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_swift_account_in_url": "false", 2024-09-20T18:50:44.543 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_swift_auth_entry": "auth", 2024-09-20T18:50:44.543 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_swift_auth_url": "", 2024-09-20T18:50:44.543 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_swift_custom_header": "", 2024-09-20T18:50:44.543 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-20T18:50:44.543 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_swift_need_stats": "true", 2024-09-20T18:50:44.543 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_swift_tenant_name": "", 2024-09-20T18:50:44.543 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_swift_token_expiration": "86400", 2024-09-20T18:50:44.543 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_swift_url": "", 2024-09-20T18:50:44.543 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_swift_url_prefix": "swift", 2024-09-20T18:50:44.543 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-20T18:50:44.543 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-20T18:50:44.543 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-20T18:50:44.543 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_sync_lease_period": "120", 2024-09-20T18:50:44.543 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-20T18:50:44.544 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-20T18:50:44.544 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-20T18:50:44.544 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-20T18:50:44.544 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-20T18:50:44.544 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-20T18:50:44.544 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-20T18:50:44.544 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-20T18:50:44.544 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-20T18:50:44.544 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_thread_pool_size": "512", 2024-09-20T18:50:44.544 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-20T18:50:44.544 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-20T18:50:44.544 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-20T18:50:44.544 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-20T18:50:44.544 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_torrent_comment": "", 2024-09-20T18:50:44.545 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_torrent_createby": "", 2024-09-20T18:50:44.545 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_torrent_encoding": "", 2024-09-20T18:50:44.545 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_torrent_flag": "false", 2024-09-20T18:50:44.545 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-20T18:50:44.545 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_torrent_origin": "", 2024-09-20T18:50:44.545 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-20T18:50:44.545 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_torrent_tracker": "", 2024-09-20T18:50:44.545 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_trust_forwarded_https": "false", 2024-09-20T18:50:44.545 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-20T18:50:44.545 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-20T18:50:44.545 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_usage_max_shards": "32", 2024-09-20T18:50:44.545 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_usage_max_user_shards": "1", 2024-09-20T18:50:44.545 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_use_opa_authz": "false", 2024-09-20T18:50:44.545 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_user_counters_cache": "false", 2024-09-20T18:50:44.546 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-20T18:50:44.546 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-20T18:50:44.546 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-20T18:50:44.546 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_user_max_buckets": "1000", 2024-09-20T18:50:44.546 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_user_policies_max_num": "100", 2024-09-20T18:50:44.546 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-20T18:50:44.546 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-20T18:50:44.546 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-20T18:50:44.546 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-20T18:50:44.546 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_user_unique_email": "true", 2024-09-20T18:50:44.546 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_verify_ssl": "true", 2024-09-20T18:50:44.546 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-20T18:50:44.546 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_zone": "", 2024-09-20T18:50:44.546 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_zone_id": "", 2024-09-20T18:50:44.547 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-20T18:50:44.547 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_zonegroup": "", 2024-09-20T18:50:44.547 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_zonegroup_id": "", 2024-09-20T18:50:44.547 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-20T18:50:44.547 INFO:teuthology.orchestra.run.smithi083.stdout: "rgwlc_auto_session_clear": "true", 2024-09-20T18:50:44.547 INFO:teuthology.orchestra.run.smithi083.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-20T18:50:44.547 INFO:teuthology.orchestra.run.smithi083.stdout: "rocksdb_block_size": "4096", 2024-09-20T18:50:44.547 INFO:teuthology.orchestra.run.smithi083.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-20T18:50:44.547 INFO:teuthology.orchestra.run.smithi083.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-20T18:50:44.547 INFO:teuthology.orchestra.run.smithi083.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-20T18:50:44.547 INFO:teuthology.orchestra.run.smithi083.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-20T18:50:44.547 INFO:teuthology.orchestra.run.smithi083.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-20T18:50:44.547 INFO:teuthology.orchestra.run.smithi083.stdout: "rocksdb_cache_size": "536870912", 2024-09-20T18:50:44.547 INFO:teuthology.orchestra.run.smithi083.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-20T18:50:44.548 INFO:teuthology.orchestra.run.smithi083.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-20T18:50:44.548 INFO:teuthology.orchestra.run.smithi083.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-20T18:50:44.548 INFO:teuthology.orchestra.run.smithi083.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-20T18:50:44.548 INFO:teuthology.orchestra.run.smithi083.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-20T18:50:44.548 INFO:teuthology.orchestra.run.smithi083.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-20T18:50:44.548 INFO:teuthology.orchestra.run.smithi083.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-20T18:50:44.548 INFO:teuthology.orchestra.run.smithi083.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-20T18:50:44.548 INFO:teuthology.orchestra.run.smithi083.stdout: "rocksdb_index_type": "binary_search", 2024-09-20T18:50:44.548 INFO:teuthology.orchestra.run.smithi083.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-20T18:50:44.548 INFO:teuthology.orchestra.run.smithi083.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-20T18:50:44.548 INFO:teuthology.orchestra.run.smithi083.stdout: "rocksdb_partition_filters": "false", 2024-09-20T18:50:44.548 INFO:teuthology.orchestra.run.smithi083.stdout: "rocksdb_perf": "false", 2024-09-20T18:50:44.548 INFO:teuthology.orchestra.run.smithi083.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-20T18:50:44.548 INFO:teuthology.orchestra.run.smithi083.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-20T18:50:44.549 INFO:teuthology.orchestra.run.smithi083.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-20T18:50:44.549 INFO:teuthology.orchestra.run.smithi083.stdout: "run_dir": "/var/run/ceph", 2024-09-20T18:50:44.549 INFO:teuthology.orchestra.run.smithi083.stdout: "seastore_block_create": "true", 2024-09-20T18:50:44.549 INFO:teuthology.orchestra.run.smithi083.stdout: "seastore_cache_lru_size": "67108864", 2024-09-20T18:50:44.549 INFO:teuthology.orchestra.run.smithi083.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-20T18:50:44.549 INFO:teuthology.orchestra.run.smithi083.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-20T18:50:44.549 INFO:teuthology.orchestra.run.smithi083.stdout: "seastore_default_max_object_size": "16777216", 2024-09-20T18:50:44.549 INFO:teuthology.orchestra.run.smithi083.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-20T18:50:44.549 INFO:teuthology.orchestra.run.smithi083.stdout: "seastore_device_size": "53687091200", 2024-09-20T18:50:44.549 INFO:teuthology.orchestra.run.smithi083.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-20T18:50:44.549 INFO:teuthology.orchestra.run.smithi083.stdout: "seastore_full_integrity_check": "false", 2024-09-20T18:50:44.549 INFO:teuthology.orchestra.run.smithi083.stdout: "seastore_journal_batch_capacity": "16", 2024-09-20T18:50:44.549 INFO:teuthology.orchestra.run.smithi083.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-20T18:50:44.549 INFO:teuthology.orchestra.run.smithi083.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-20T18:50:44.549 INFO:teuthology.orchestra.run.smithi083.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-20T18:50:44.550 INFO:teuthology.orchestra.run.smithi083.stdout: "seastore_main_device_type": "SSD", 2024-09-20T18:50:44.550 INFO:teuthology.orchestra.run.smithi083.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-20T18:50:44.550 INFO:teuthology.orchestra.run.smithi083.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-20T18:50:44.550 INFO:teuthology.orchestra.run.smithi083.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-20T18:50:44.550 INFO:teuthology.orchestra.run.smithi083.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-20T18:50:44.550 INFO:teuthology.orchestra.run.smithi083.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-20T18:50:44.550 INFO:teuthology.orchestra.run.smithi083.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-20T18:50:44.550 INFO:teuthology.orchestra.run.smithi083.stdout: "seastore_segment_size": "67108864", 2024-09-20T18:50:44.550 INFO:teuthology.orchestra.run.smithi083.stdout: "set_keepcaps": "false", 2024-09-20T18:50:44.550 INFO:teuthology.orchestra.run.smithi083.stdout: "setgroup": "ceph", 2024-09-20T18:50:44.550 INFO:teuthology.orchestra.run.smithi083.stdout: "setuser": "ceph", 2024-09-20T18:50:44.550 INFO:teuthology.orchestra.run.smithi083.stdout: "setuser_match_path": "", 2024-09-20T18:50:44.550 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-20T18:50:44.551 INFO:teuthology.orchestra.run.smithi083.stdout: "thp": "false", 2024-09-20T18:50:44.551 INFO:teuthology.orchestra.run.smithi083.stdout: "threadpool_default_timeout": "60", 2024-09-20T18:50:44.551 INFO:teuthology.orchestra.run.smithi083.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-20T18:50:44.551 INFO:teuthology.orchestra.run.smithi083.stdout: "throttler_perf_counter": "true", 2024-09-20T18:50:44.551 INFO:teuthology.orchestra.run.smithi083.stdout: "tmp_dir": "/tmp", 2024-09-20T18:50:44.551 INFO:teuthology.orchestra.run.smithi083.stdout: "tmp_file_template": "/tmp/ceph-mon.a.XXXXXX", 2024-09-20T18:50:44.551 INFO:teuthology.orchestra.run.smithi083.stdout: "uadk_compressor_enabled": "false", 2024-09-20T18:50:44.551 INFO:teuthology.orchestra.run.smithi083.stdout: "uadk_wd_sync_ctx_num": "2" 2024-09-20T18:50:44.551 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T18:50:45.452 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-20T18:50:45.699 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:50:46.104 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:45 smithi083 ceph-mon[29421]: pgmap v26: 132 pgs: 132 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-20T18:50:46.104 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:45 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:50:46.104 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:45 smithi083 ceph-mon[36260]: pgmap v26: 132 pgs: 132 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-20T18:50:46.104 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:45 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:50:46.118 INFO:teuthology.orchestra.run.smithi083.stdout:mon.a: {} 2024-09-20T18:50:46.118 INFO:teuthology.orchestra.run.smithi083.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-20T18:50:46.127 INFO:teuthology.orchestra.run.smithi083.stdout:mon.b: {} 2024-09-20T18:50:46.127 INFO:teuthology.orchestra.run.smithi083.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-20T18:50:46.135 INFO:teuthology.orchestra.run.smithi083.stdout:mon.c: {} 2024-09-20T18:50:46.136 INFO:teuthology.orchestra.run.smithi083.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-20T18:50:46.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:45 smithi142 ceph-mon[32508]: pgmap v26: 132 pgs: 132 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-20T18:50:46.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:45 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:50:46.727 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:50:46 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:50:46.792 INFO:teuthology.run_tasks:Running task full_sequential... 2024-09-20T18:50:46.804 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-20T18:50:46.806 INFO:tasks.radosbench:Beginning radosbench... 2024-09-20T18:50:46.806 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_0 2024-09-20T18:50:46.806 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd pool create unique_pool_0 16 2024-09-20T18:50:46.806 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-20T18:50:46.807 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-20T18:50:46.807 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T18:50:46.807 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-20T18:50:46.807 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd dump -f json-pretty 2024-09-20T18:50:46.807 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-20T18:50:46.807 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:50:46.808 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-20T18:50:46.808 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T18:50:46.808 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-20T18:50:46.808 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-20T18:50:46.809 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-20T18:50:46.809 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T18:50:46.839 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:50:46.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:46.840 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:50:46.941 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:50:46.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:46.966 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:46.966 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:50:46.988 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:46 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1775187159' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-20T18:50:46.988 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:46 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:50:47.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:46 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1775187159' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-20T18:50:47.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:46 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:50:47.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:46 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1775187159' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-20T18:50:47.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:46 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:50:47.067 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:50:47.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:47.089 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:47.089 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:50:47.121 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:50:47.128 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:50:47.130 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:50:47.190 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:50:47.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:47.213 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:50:47.213 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:50:47.314 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:50:47.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:47.335 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:50:47.336 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:50:47.436 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:50:47.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:47.459 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:50:47.459 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:50:47.560 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:50:47.581 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:50:47.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:47.582 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:50:47.682 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:50:47.703 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:50:47.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:47.705 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:50:47.805 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:50:47.829 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:47.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:47.830 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:50:47.931 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:50:47.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:47.966 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:50:47.966 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:50:48.067 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:50:48.089 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:48.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:48.090 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:50:48.153 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:47 smithi142 ceph-mon[32508]: 3.c scrub starts 2024-09-20T18:50:48.153 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:47 smithi142 ceph-mon[32508]: 3.c scrub ok 2024-09-20T18:50:48.153 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:47 smithi142 ceph-mon[32508]: pgmap v27: 132 pgs: 132 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T18:50:48.163 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:47 smithi083 ceph-mon[29421]: 3.c scrub starts 2024-09-20T18:50:48.163 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:47 smithi083 ceph-mon[29421]: 3.c scrub ok 2024-09-20T18:50:48.164 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:47 smithi083 ceph-mon[29421]: pgmap v27: 132 pgs: 132 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T18:50:48.164 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:47 smithi083 ceph-mon[36260]: 3.c scrub starts 2024-09-20T18:50:48.164 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:47 smithi083 ceph-mon[36260]: 3.c scrub ok 2024-09-20T18:50:48.164 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:47 smithi083 ceph-mon[36260]: pgmap v27: 132 pgs: 132 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T18:50:48.191 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:50:48.213 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:48.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:48.214 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:50:48.245 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:50:48.245 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T18:50:48.245 INFO:teuthology.orchestra.run.smithi083.stdout: "epoch": 82, 2024-09-20T18:50:48.246 INFO:teuthology.orchestra.run.smithi083.stdout: "fsid": "1ed5a50a-7780-11ef-baf5-efdc52797490", 2024-09-20T18:50:48.246 INFO:teuthology.orchestra.run.smithi083.stdout: "created": "2024-09-20T18:44:10.399800+0000", 2024-09-20T18:50:48.246 INFO:teuthology.orchestra.run.smithi083.stdout: "modified": "2024-09-20T18:50:40.555466+0000", 2024-09-20T18:50:48.246 INFO:teuthology.orchestra.run.smithi083.stdout: "last_up_change": "2024-09-20T18:49:34.836377+0000", 2024-09-20T18:50:48.246 INFO:teuthology.orchestra.run.smithi083.stdout: "last_in_change": "2024-09-20T18:49:12.342332+0000", 2024-09-20T18:50:48.246 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-20T18:50:48.246 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_num": 5799936, 2024-09-20T18:50:48.246 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_set": [ 2024-09-20T18:50:48.246 INFO:teuthology.orchestra.run.smithi083.stdout: "pglog_hardlimit", 2024-09-20T18:50:48.246 INFO:teuthology.orchestra.run.smithi083.stdout: "purged_snapdirs", 2024-09-20T18:50:48.246 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_deletes", 2024-09-20T18:50:48.246 INFO:teuthology.orchestra.run.smithi083.stdout: "sortbitwise" 2024-09-20T18:50:48.246 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T18:50:48.247 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_version": 18, 2024-09-20T18:50:48.247 INFO:teuthology.orchestra.run.smithi083.stdout: "full_ratio": 0.94999998807907104, 2024-09-20T18:50:48.247 INFO:teuthology.orchestra.run.smithi083.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-20T18:50:48.247 INFO:teuthology.orchestra.run.smithi083.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-20T18:50:48.247 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_snapshot": "", 2024-09-20T18:50:48.247 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_max": 6, 2024-09-20T18:50:48.247 INFO:teuthology.orchestra.run.smithi083.stdout: "max_osd": 8, 2024-09-20T18:50:48.247 INFO:teuthology.orchestra.run.smithi083.stdout: "require_min_compat_client": "luminous", 2024-09-20T18:50:48.247 INFO:teuthology.orchestra.run.smithi083.stdout: "min_compat_client": "luminous", 2024-09-20T18:50:48.247 INFO:teuthology.orchestra.run.smithi083.stdout: "require_osd_release": "squid", 2024-09-20T18:50:48.247 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_crimson": false, 2024-09-20T18:50:48.247 INFO:teuthology.orchestra.run.smithi083.stdout: "pools": [ 2024-09-20T18:50:48.247 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.247 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 1, 2024-09-20T18:50:48.248 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": ".mgr", 2024-09-20T18:50:48.248 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-20T18:47:15.285795+0000", 2024-09-20T18:50:48.248 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-20T18:50:48.248 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool", 2024-09-20T18:50:48.248 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-20T18:50:48.248 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 3, 2024-09-20T18:50:48.248 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 2, 2024-09-20T18:50:48.248 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-20T18:50:48.248 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-20T18:50:48.248 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-20T18:50:48.248 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T18:50:48.248 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T18:50:48.248 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-20T18:50:48.248 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-20T18:50:48.249 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-20T18:50:48.249 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 1, 2024-09-20T18:50:48.249 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 1, 2024-09-20T18:50:48.249 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 1, 2024-09-20T18:50:48.249 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 1, 2024-09-20T18:50:48.249 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 1, 2024-09-20T18:50:48.249 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-20T18:50:48.249 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-20T18:50:48.249 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-20T18:50:48.249 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-20T18:50:48.249 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-20T18:50:48.249 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-20T18:50:48.249 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-20T18:50:48.250 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.250 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "21", 2024-09-20T18:50:48.250 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-20T18:50:48.250 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T18:50:48.250 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T18:50:48.250 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-20T18:50:48.250 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-20T18:50:48.250 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 0, 2024-09-20T18:50:48.250 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 0, 2024-09-20T18:50:48.250 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-20T18:50:48.250 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-20T18:50:48.250 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-20T18:50:48.250 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-20T18:50:48.250 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-20T18:50:48.251 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-20T18:50:48.251 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-20T18:50:48.251 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-20T18:50:48.251 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-20T18:50:48.251 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-20T18:50:48.251 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-20T18:50:48.251 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T18:50:48.251 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T18:50:48.251 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T18:50:48.251 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-20T18:50:48.251 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-20T18:50:48.251 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-20T18:50:48.251 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-20T18:50:48.252 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-20T18:50:48.252 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.252 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-20T18:50:48.252 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-20T18:50:48.252 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-20T18:50:48.252 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-20T18:50:48.252 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-20T18:50:48.252 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T18:50:48.252 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-20T18:50:48.252 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-20T18:50:48.252 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-20T18:50:48.252 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-20T18:50:48.252 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-20T18:50:48.252 INFO:teuthology.orchestra.run.smithi083.stdout: "options": { 2024-09-20T18:50:48.253 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_max": 32, 2024-09-20T18:50:48.253 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_min": 1 2024-09-20T18:50:48.253 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.253 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-20T18:50:48.253 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": {} 2024-09-20T18:50:48.253 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.253 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-20T18:50:48.253 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-20T18:50:48.253 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 7.8899998664855957, 2024-09-20T18:50:48.253 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 7.8899998664855957, 2024-09-20T18:50:48.253 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 0.37999999523162842, 2024-09-20T18:50:48.253 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 3, 2024-09-20T18:50:48.253 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 3, 2024-09-20T18:50:48.253 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 1, 2024-09-20T18:50:48.254 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 1, 2024-09-20T18:50:48.254 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 1 2024-09-20T18:50:48.254 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:50:48.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.254 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.254 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 2, 2024-09-20T18:50:48.254 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": "datapool", 2024-09-20T18:50:48.254 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-20T18:49:43.452984+0000", 2024-09-20T18:50:48.254 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 8193, 2024-09-20T18:50:48.254 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-20T18:50:48.254 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-20T18:50:48.254 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 3, 2024-09-20T18:50:48.254 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 2, 2024-09-20T18:50:48.254 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-20T18:50:48.255 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-20T18:50:48.255 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-20T18:50:48.255 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T18:50:48.255 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T18:50:48.255 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-20T18:50:48.255 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-20T18:50:48.255 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-20T18:50:48.255 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 3, 2024-09-20T18:50:48.255 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 3, 2024-09-20T18:50:48.255 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 3, 2024-09-20T18:50:48.255 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 3, 2024-09-20T18:50:48.255 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 3, 2024-09-20T18:50:48.255 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-20T18:50:48.256 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-20T18:50:48.256 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-20T18:50:48.256 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-20T18:50:48.256 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-20T18:50:48.256 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-20T18:50:48.256 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-20T18:50:48.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.256 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "61", 2024-09-20T18:50:48.256 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-20T18:50:48.256 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T18:50:48.256 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T18:50:48.256 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-20T18:50:48.256 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-20T18:50:48.256 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 2, 2024-09-20T18:50:48.257 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 61, 2024-09-20T18:50:48.257 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-20T18:50:48.257 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-20T18:50:48.257 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-20T18:50:48.257 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-20T18:50:48.257 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-20T18:50:48.257 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-20T18:50:48.257 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-20T18:50:48.257 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-20T18:50:48.257 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-20T18:50:48.257 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-20T18:50:48.257 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-20T18:50:48.257 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T18:50:48.257 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T18:50:48.258 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T18:50:48.258 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-20T18:50:48.258 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-20T18:50:48.258 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-20T18:50:48.258 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-20T18:50:48.258 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-20T18:50:48.258 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.258 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-20T18:50:48.258 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-20T18:50:48.258 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-20T18:50:48.258 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-20T18:50:48.258 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-20T18:50:48.258 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T18:50:48.259 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-20T18:50:48.259 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-20T18:50:48.259 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-20T18:50:48.259 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-20T18:50:48.259 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-20T18:50:48.259 INFO:teuthology.orchestra.run.smithi083.stdout: "options": {}, 2024-09-20T18:50:48.259 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-20T18:50:48.259 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd": {} 2024-09-20T18:50:48.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.259 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-20T18:50:48.259 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-20T18:50:48.259 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 2.6500000953674316, 2024-09-20T18:50:48.259 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 2.6500000953674316, 2024-09-20T18:50:48.259 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 0.87999999523162842, 2024-09-20T18:50:48.260 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 2.3299999237060547, 2024-09-20T18:50:48.260 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 2.3299999237060547, 2024-09-20T18:50:48.260 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 1, 2024-09-20T18:50:48.260 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 1, 2024-09-20T18:50:48.260 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 1 2024-09-20T18:50:48.260 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:50:48.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.260 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.260 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 3, 2024-09-20T18:50:48.260 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": ".rgw.root", 2024-09-20T18:50:48.260 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-20T18:49:44.217011+0000", 2024-09-20T18:50:48.260 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-20T18:50:48.260 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool", 2024-09-20T18:50:48.261 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-20T18:50:48.261 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 3, 2024-09-20T18:50:48.261 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 2, 2024-09-20T18:50:48.261 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-20T18:50:48.261 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-20T18:50:48.261 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-20T18:50:48.261 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T18:50:48.261 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T18:50:48.261 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-20T18:50:48.261 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-20T18:50:48.261 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-20T18:50:48.261 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 32, 2024-09-20T18:50:48.261 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 32, 2024-09-20T18:50:48.261 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 32, 2024-09-20T18:50:48.262 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 32, 2024-09-20T18:50:48.262 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 32, 2024-09-20T18:50:48.262 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-20T18:50:48.262 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-20T18:50:48.262 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-20T18:50:48.262 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-20T18:50:48.262 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-20T18:50:48.263 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-20T18:50:48.263 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-20T18:50:48.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.263 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "60", 2024-09-20T18:50:48.263 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-20T18:50:48.263 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T18:50:48.263 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T18:50:48.263 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-20T18:50:48.263 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-20T18:50:48.263 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 0, 2024-09-20T18:50:48.263 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 0, 2024-09-20T18:50:48.264 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-20T18:50:48.264 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-20T18:50:48.264 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-20T18:50:48.264 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-20T18:50:48.264 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-20T18:50:48.264 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-20T18:50:48.264 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-20T18:50:48.264 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-20T18:50:48.264 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-20T18:50:48.264 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-20T18:50:48.264 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-20T18:50:48.264 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T18:50:48.264 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T18:50:48.264 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T18:50:48.265 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-20T18:50:48.265 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-20T18:50:48.265 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-20T18:50:48.265 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-20T18:50:48.265 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-20T18:50:48.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.265 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-20T18:50:48.265 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-20T18:50:48.265 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-20T18:50:48.265 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-20T18:50:48.265 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-20T18:50:48.265 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T18:50:48.265 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-20T18:50:48.265 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-20T18:50:48.266 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-20T18:50:48.266 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-20T18:50:48.266 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-20T18:50:48.266 INFO:teuthology.orchestra.run.smithi083.stdout: "options": {}, 2024-09-20T18:50:48.266 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-20T18:50:48.266 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw": {} 2024-09-20T18:50:48.266 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.266 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-20T18:50:48.266 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-20T18:50:48.266 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 1.5, 2024-09-20T18:50:48.266 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 1.5, 2024-09-20T18:50:48.266 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 1, 2024-09-20T18:50:48.266 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 1.5, 2024-09-20T18:50:48.267 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 1.5, 2024-09-20T18:50:48.267 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 1, 2024-09-20T18:50:48.267 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 1, 2024-09-20T18:50:48.267 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 1 2024-09-20T18:50:48.267 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:50:48.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.267 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.267 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 4, 2024-09-20T18:50:48.267 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": "default.rgw.log", 2024-09-20T18:50:48.267 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-20T18:49:46.007275+0000", 2024-09-20T18:50:48.267 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-20T18:50:48.267 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool", 2024-09-20T18:50:48.267 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-20T18:50:48.267 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 3, 2024-09-20T18:50:48.267 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 2, 2024-09-20T18:50:48.268 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-20T18:50:48.268 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-20T18:50:48.268 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-20T18:50:48.268 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T18:50:48.268 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T18:50:48.268 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-20T18:50:48.268 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-20T18:50:48.268 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-20T18:50:48.268 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 32, 2024-09-20T18:50:48.268 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 32, 2024-09-20T18:50:48.268 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 32, 2024-09-20T18:50:48.268 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 32, 2024-09-20T18:50:48.268 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 32, 2024-09-20T18:50:48.269 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-20T18:50:48.269 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-20T18:50:48.269 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-20T18:50:48.269 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-20T18:50:48.269 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-20T18:50:48.269 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-20T18:50:48.269 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-20T18:50:48.269 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.269 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "62", 2024-09-20T18:50:48.269 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-20T18:50:48.269 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T18:50:48.269 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T18:50:48.269 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-20T18:50:48.270 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-20T18:50:48.270 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 0, 2024-09-20T18:50:48.270 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 0, 2024-09-20T18:50:48.270 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-20T18:50:48.270 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-20T18:50:48.270 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-20T18:50:48.270 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-20T18:50:48.270 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-20T18:50:48.270 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-20T18:50:48.270 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-20T18:50:48.270 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-20T18:50:48.270 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-20T18:50:48.270 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-20T18:50:48.270 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-20T18:50:48.271 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T18:50:48.271 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T18:50:48.271 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T18:50:48.271 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-20T18:50:48.271 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-20T18:50:48.271 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-20T18:50:48.271 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-20T18:50:48.271 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-20T18:50:48.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.271 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-20T18:50:48.271 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-20T18:50:48.271 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-20T18:50:48.271 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-20T18:50:48.271 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-20T18:50:48.272 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T18:50:48.272 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-20T18:50:48.272 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-20T18:50:48.272 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-20T18:50:48.272 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-20T18:50:48.272 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-20T18:50:48.272 INFO:teuthology.orchestra.run.smithi083.stdout: "options": {}, 2024-09-20T18:50:48.272 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-20T18:50:48.272 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw": {} 2024-09-20T18:50:48.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.272 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-20T18:50:48.272 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-20T18:50:48.272 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 2.25, 2024-09-20T18:50:48.273 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 2.25, 2024-09-20T18:50:48.273 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 1, 2024-09-20T18:50:48.273 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 2.25, 2024-09-20T18:50:48.273 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 2.25, 2024-09-20T18:50:48.273 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 1, 2024-09-20T18:50:48.273 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 1, 2024-09-20T18:50:48.273 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 1 2024-09-20T18:50:48.273 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:50:48.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.273 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.273 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 5, 2024-09-20T18:50:48.273 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": "default.rgw.control", 2024-09-20T18:50:48.273 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-20T18:49:47.961258+0000", 2024-09-20T18:50:48.274 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-20T18:50:48.274 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool", 2024-09-20T18:50:48.274 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-20T18:50:48.274 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 3, 2024-09-20T18:50:48.274 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 2, 2024-09-20T18:50:48.274 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-20T18:50:48.274 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-20T18:50:48.274 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-20T18:50:48.274 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T18:50:48.274 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T18:50:48.274 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-20T18:50:48.274 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-20T18:50:48.274 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-20T18:50:48.274 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 32, 2024-09-20T18:50:48.275 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 32, 2024-09-20T18:50:48.275 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 32, 2024-09-20T18:50:48.275 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 32, 2024-09-20T18:50:48.275 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 32, 2024-09-20T18:50:48.275 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-20T18:50:48.275 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-20T18:50:48.275 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-20T18:50:48.275 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-20T18:50:48.275 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-20T18:50:48.275 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-20T18:50:48.275 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-20T18:50:48.275 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.275 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "64", 2024-09-20T18:50:48.276 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-20T18:50:48.276 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T18:50:48.276 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T18:50:48.276 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-20T18:50:48.276 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-20T18:50:48.276 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 0, 2024-09-20T18:50:48.276 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 0, 2024-09-20T18:50:48.276 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-20T18:50:48.276 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-20T18:50:48.276 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-20T18:50:48.276 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-20T18:50:48.276 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-20T18:50:48.276 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-20T18:50:48.276 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-20T18:50:48.276 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-20T18:50:48.277 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-20T18:50:48.277 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-20T18:50:48.277 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-20T18:50:48.277 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T18:50:48.277 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T18:50:48.277 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T18:50:48.277 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-20T18:50:48.277 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-20T18:50:48.277 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-20T18:50:48.277 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-20T18:50:48.277 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-20T18:50:48.277 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.277 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-20T18:50:48.277 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-20T18:50:48.277 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-20T18:50:48.278 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-20T18:50:48.278 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-20T18:50:48.278 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T18:50:48.278 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-20T18:50:48.278 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-20T18:50:48.278 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-20T18:50:48.278 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-20T18:50:48.278 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-20T18:50:48.278 INFO:teuthology.orchestra.run.smithi083.stdout: "options": {}, 2024-09-20T18:50:48.278 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-20T18:50:48.278 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw": {} 2024-09-20T18:50:48.278 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.278 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-20T18:50:48.278 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-20T18:50:48.279 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 1.25, 2024-09-20T18:50:48.279 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 1.25, 2024-09-20T18:50:48.279 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 1, 2024-09-20T18:50:48.279 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 1.25, 2024-09-20T18:50:48.279 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 1.25, 2024-09-20T18:50:48.279 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 1, 2024-09-20T18:50:48.279 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 1, 2024-09-20T18:50:48.279 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 1 2024-09-20T18:50:48.279 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:50:48.279 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.279 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.279 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 6, 2024-09-20T18:50:48.279 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": "default.rgw.meta", 2024-09-20T18:50:48.279 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-20T18:49:50.011276+0000", 2024-09-20T18:50:48.280 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-20T18:50:48.280 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool", 2024-09-20T18:50:48.280 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-20T18:50:48.280 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 3, 2024-09-20T18:50:48.280 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 2, 2024-09-20T18:50:48.280 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-20T18:50:48.280 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-20T18:50:48.280 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-20T18:50:48.280 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T18:50:48.280 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T18:50:48.280 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-20T18:50:48.280 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-20T18:50:48.280 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-20T18:50:48.281 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 32, 2024-09-20T18:50:48.281 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 32, 2024-09-20T18:50:48.281 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 32, 2024-09-20T18:50:48.281 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 32, 2024-09-20T18:50:48.281 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 32, 2024-09-20T18:50:48.281 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-20T18:50:48.281 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-20T18:50:48.281 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-20T18:50:48.281 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-20T18:50:48.281 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-20T18:50:48.281 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-20T18:50:48.281 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-20T18:50:48.282 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.282 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "66", 2024-09-20T18:50:48.282 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-20T18:50:48.282 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T18:50:48.282 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T18:50:48.282 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-20T18:50:48.282 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-20T18:50:48.282 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 0, 2024-09-20T18:50:48.282 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 0, 2024-09-20T18:50:48.282 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-20T18:50:48.282 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-20T18:50:48.282 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-20T18:50:48.282 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-20T18:50:48.283 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-20T18:50:48.283 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-20T18:50:48.283 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-20T18:50:48.283 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-20T18:50:48.283 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-20T18:50:48.283 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-20T18:50:48.283 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-20T18:50:48.283 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T18:50:48.283 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T18:50:48.283 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T18:50:48.283 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-20T18:50:48.283 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-20T18:50:48.283 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-20T18:50:48.284 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-20T18:50:48.284 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-20T18:50:48.284 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.284 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-20T18:50:48.284 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-20T18:50:48.284 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-20T18:50:48.284 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-20T18:50:48.284 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-20T18:50:48.284 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T18:50:48.284 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-20T18:50:48.284 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-20T18:50:48.284 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-20T18:50:48.284 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-20T18:50:48.284 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-20T18:50:48.284 INFO:teuthology.orchestra.run.smithi083.stdout: "options": { 2024-09-20T18:50:48.285 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_bias": 4 2024-09-20T18:50:48.285 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.285 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-20T18:50:48.285 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw": {} 2024-09-20T18:50:48.285 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.285 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-20T18:50:48.285 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-20T18:50:48.285 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 1.75, 2024-09-20T18:50:48.285 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 1.75, 2024-09-20T18:50:48.285 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 1, 2024-09-20T18:50:48.285 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 1.75, 2024-09-20T18:50:48.285 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 1.75, 2024-09-20T18:50:48.285 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 1, 2024-09-20T18:50:48.285 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 1, 2024-09-20T18:50:48.286 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 1 2024-09-20T18:50:48.286 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:50:48.286 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:50:48.286 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T18:50:48.286 INFO:teuthology.orchestra.run.smithi083.stdout: "osds": [ 2024-09-20T18:50:48.286 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.286 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 0, 2024-09-20T18:50:48.286 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "c714e1ec-f0c3-4054-9792-61b84e53eaa8", 2024-09-20T18:50:48.286 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T18:50:48.286 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T18:50:48.286 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T18:50:48.286 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T18:50:48.286 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T18:50:48.287 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T18:50:48.287 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 8, 2024-09-20T18:50:48.287 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 64, 2024-09-20T18:50:48.287 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T18:50:48.287 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T18:50:48.287 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T18:50:48.287 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:50:48.287 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.287 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:50:48.287 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6801", 2024-09-20T18:50:48.287 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2574084049 2024-09-20T18:50:48.287 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:50:48.287 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:50:48.287 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.288 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T18:50:48.288 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:50:48.288 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.288 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:50:48.288 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6802", 2024-09-20T18:50:48.288 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2574084049 2024-09-20T18:50:48.288 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:50:48.288 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:50:48.288 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.288 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T18:50:48.288 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:50:48.288 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.288 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:50:48.288 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6804", 2024-09-20T18:50:48.288 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2574084049 2024-09-20T18:50:48.289 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:50:48.289 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:50:48.289 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.289 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T18:50:48.289 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:50:48.289 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.289 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:50:48.289 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6803", 2024-09-20T18:50:48.289 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2574084049 2024-09-20T18:50:48.289 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:50:48.289 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:50:48.289 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.289 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.83:6801/2574084049", 2024-09-20T18:50:48.290 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.83:6802/2574084049", 2024-09-20T18:50:48.290 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.83:6804/2574084049", 2024-09-20T18:50:48.290 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.83:6803/2574084049", 2024-09-20T18:50:48.290 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T18:50:48.290 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T18:50:48.290 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T18:50:48.290 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:50:48.290 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.290 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.290 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 1, 2024-09-20T18:50:48.290 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "9dc0f2a8-bafa-4a42-88f5-0301dd58f78d", 2024-09-20T18:50:48.290 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T18:50:48.290 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T18:50:48.290 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T18:50:48.291 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T18:50:48.291 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T18:50:48.291 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T18:50:48.291 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 13, 2024-09-20T18:50:48.291 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 64, 2024-09-20T18:50:48.291 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T18:50:48.291 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T18:50:48.291 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T18:50:48.291 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:50:48.291 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.291 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:50:48.291 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6805", 2024-09-20T18:50:48.291 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 4213285798 2024-09-20T18:50:48.291 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:50:48.292 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:50:48.292 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.292 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T18:50:48.292 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:50:48.292 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.292 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:50:48.292 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6806", 2024-09-20T18:50:48.292 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 4213285798 2024-09-20T18:50:48.292 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:50:48.292 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:50:48.292 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.292 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T18:50:48.292 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:50:48.293 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.293 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:50:48.293 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6808", 2024-09-20T18:50:48.293 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 4213285798 2024-09-20T18:50:48.293 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:50:48.293 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:50:48.293 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.293 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T18:50:48.293 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:50:48.293 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.293 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:50:48.293 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6807", 2024-09-20T18:50:48.293 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 4213285798 2024-09-20T18:50:48.293 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:50:48.294 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:50:48.294 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.294 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.83:6805/4213285798", 2024-09-20T18:50:48.294 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.83:6806/4213285798", 2024-09-20T18:50:48.294 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.83:6808/4213285798", 2024-09-20T18:50:48.294 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.83:6807/4213285798", 2024-09-20T18:50:48.294 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T18:50:48.294 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T18:50:48.294 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T18:50:48.294 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:50:48.294 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.294 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.294 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 2, 2024-09-20T18:50:48.294 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "448ab692-ec41-44dc-b2d5-dbefd255b3fd", 2024-09-20T18:50:48.295 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T18:50:48.295 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T18:50:48.295 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T18:50:48.295 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T18:50:48.295 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T18:50:48.295 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T18:50:48.295 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 18, 2024-09-20T18:50:48.295 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 68, 2024-09-20T18:50:48.295 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T18:50:48.295 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T18:50:48.295 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T18:50:48.295 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:50:48.295 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.295 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:50:48.296 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6809", 2024-09-20T18:50:48.296 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 805947571 2024-09-20T18:50:48.296 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:50:48.296 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:50:48.296 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.296 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T18:50:48.296 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:50:48.296 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.296 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:50:48.296 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6810", 2024-09-20T18:50:48.296 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 805947571 2024-09-20T18:50:48.296 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:50:48.296 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:50:48.296 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.297 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T18:50:48.297 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:50:48.297 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.297 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:50:48.297 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6812", 2024-09-20T18:50:48.297 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 805947571 2024-09-20T18:50:48.297 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:50:48.297 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:50:48.297 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.297 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T18:50:48.297 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:50:48.297 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.297 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:50:48.297 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6811", 2024-09-20T18:50:48.298 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 805947571 2024-09-20T18:50:48.298 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:50:48.298 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:50:48.298 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.298 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.83:6809/805947571", 2024-09-20T18:50:48.299 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.83:6810/805947571", 2024-09-20T18:50:48.299 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.83:6812/805947571", 2024-09-20T18:50:48.299 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.83:6811/805947571", 2024-09-20T18:50:48.299 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T18:50:48.299 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T18:50:48.299 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T18:50:48.299 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:50:48.299 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.299 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.299 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 3, 2024-09-20T18:50:48.299 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "7cda5e50-e22e-4d4a-823b-53c8f5061193", 2024-09-20T18:50:48.299 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T18:50:48.299 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T18:50:48.299 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T18:50:48.300 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T18:50:48.300 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T18:50:48.300 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T18:50:48.300 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 25, 2024-09-20T18:50:48.300 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 68, 2024-09-20T18:50:48.300 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T18:50:48.300 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T18:50:48.300 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T18:50:48.300 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:50:48.300 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.300 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:50:48.300 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6813", 2024-09-20T18:50:48.300 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 987237318 2024-09-20T18:50:48.301 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:50:48.301 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:50:48.301 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.301 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T18:50:48.301 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:50:48.301 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.301 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:50:48.301 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6814", 2024-09-20T18:50:48.301 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 987237318 2024-09-20T18:50:48.301 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:50:48.301 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:50:48.301 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.301 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T18:50:48.301 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:50:48.302 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.302 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:50:48.302 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6816", 2024-09-20T18:50:48.302 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 987237318 2024-09-20T18:50:48.302 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:50:48.302 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:50:48.302 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.302 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T18:50:48.302 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:50:48.302 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.302 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:50:48.302 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6815", 2024-09-20T18:50:48.302 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 987237318 2024-09-20T18:50:48.303 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:50:48.303 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:50:48.303 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.303 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.83:6813/987237318", 2024-09-20T18:50:48.303 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.83:6814/987237318", 2024-09-20T18:50:48.303 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.83:6816/987237318", 2024-09-20T18:50:48.303 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.83:6815/987237318", 2024-09-20T18:50:48.303 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T18:50:48.303 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T18:50:48.303 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T18:50:48.303 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:50:48.303 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.303 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.303 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 4, 2024-09-20T18:50:48.303 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "3d62654d-d31d-4310-9934-49e694dd17d9", 2024-09-20T18:50:48.304 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T18:50:48.304 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T18:50:48.304 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T18:50:48.304 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T18:50:48.304 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T18:50:48.304 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T18:50:48.304 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 31, 2024-09-20T18:50:48.304 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 68, 2024-09-20T18:50:48.304 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T18:50:48.304 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T18:50:48.304 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T18:50:48.304 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:50:48.304 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.304 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:50:48.305 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6800", 2024-09-20T18:50:48.305 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2096684065 2024-09-20T18:50:48.305 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:50:48.305 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:50:48.305 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.305 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T18:50:48.305 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:50:48.305 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.305 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:50:48.305 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6801", 2024-09-20T18:50:48.305 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2096684065 2024-09-20T18:50:48.305 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:50:48.305 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:50:48.306 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.306 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T18:50:48.306 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:50:48.306 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.306 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:50:48.306 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6803", 2024-09-20T18:50:48.306 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2096684065 2024-09-20T18:50:48.306 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:50:48.306 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:50:48.306 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.306 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T18:50:48.306 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:50:48.306 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.306 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:50:48.307 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6802", 2024-09-20T18:50:48.307 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2096684065 2024-09-20T18:50:48.307 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:50:48.307 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:50:48.307 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.307 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.142:6800/2096684065", 2024-09-20T18:50:48.307 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.142:6801/2096684065", 2024-09-20T18:50:48.307 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.142:6803/2096684065", 2024-09-20T18:50:48.307 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.142:6802/2096684065", 2024-09-20T18:50:48.307 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T18:50:48.307 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T18:50:48.307 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T18:50:48.307 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:50:48.308 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.308 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.308 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 5, 2024-09-20T18:50:48.308 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "b3e4e21b-639f-4d25-8bbf-ea8922a9222a", 2024-09-20T18:50:48.308 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T18:50:48.308 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T18:50:48.308 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T18:50:48.308 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T18:50:48.308 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T18:50:48.308 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T18:50:48.308 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 38, 2024-09-20T18:50:48.308 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 64, 2024-09-20T18:50:48.308 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T18:50:48.308 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T18:50:48.309 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T18:50:48.309 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:50:48.309 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.309 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:50:48.309 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6804", 2024-09-20T18:50:48.309 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2500466067 2024-09-20T18:50:48.309 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:50:48.309 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:50:48.309 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.309 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T18:50:48.309 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:50:48.309 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.309 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:50:48.309 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6805", 2024-09-20T18:50:48.310 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2500466067 2024-09-20T18:50:48.310 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:50:48.310 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:50:48.310 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.310 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T18:50:48.310 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:50:48.310 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.310 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:50:48.310 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6807", 2024-09-20T18:50:48.310 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2500466067 2024-09-20T18:50:48.310 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:50:48.310 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:50:48.311 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.311 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T18:50:48.311 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:50:48.311 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.311 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:50:48.311 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6806", 2024-09-20T18:50:48.311 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2500466067 2024-09-20T18:50:48.311 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:50:48.311 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:50:48.311 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.311 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.142:6804/2500466067", 2024-09-20T18:50:48.311 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.142:6805/2500466067", 2024-09-20T18:50:48.311 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.142:6807/2500466067", 2024-09-20T18:50:48.311 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.142:6806/2500466067", 2024-09-20T18:50:48.312 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T18:50:48.312 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T18:50:48.312 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T18:50:48.312 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:50:48.312 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.312 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.312 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 6, 2024-09-20T18:50:48.312 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "c5db7107-ccb4-4ae2-9e63-f444897942e3", 2024-09-20T18:50:48.312 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T18:50:48.312 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T18:50:48.312 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T18:50:48.312 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T18:50:48.312 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T18:50:48.312 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T18:50:48.313 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 45, 2024-09-20T18:50:48.313 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 62, 2024-09-20T18:50:48.313 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T18:50:48.313 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T18:50:48.313 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T18:50:48.313 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:50:48.313 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.313 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:50:48.313 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6808", 2024-09-20T18:50:48.313 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1688186226 2024-09-20T18:50:48.313 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:50:48.313 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:50:48.313 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.313 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T18:50:48.313 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:50:48.314 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.314 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:50:48.314 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6809", 2024-09-20T18:50:48.314 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1688186226 2024-09-20T18:50:48.314 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:50:48.314 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:50:48.314 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.314 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T18:50:48.314 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:50:48.314 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.314 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:50:48.314 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6811", 2024-09-20T18:50:48.314 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1688186226 2024-09-20T18:50:48.315 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:50:48.315 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:50:48.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.315 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T18:50:48.315 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:50:48.315 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.315 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:50:48.315 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6810", 2024-09-20T18:50:48.315 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1688186226 2024-09-20T18:50:48.315 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:50:48.315 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:50:48.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.315 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.142:6808/1688186226", 2024-09-20T18:50:48.315 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.142:6809/1688186226", 2024-09-20T18:50:48.316 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.142:6811/1688186226", 2024-09-20T18:50:48.316 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.142:6810/1688186226", 2024-09-20T18:50:48.316 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T18:50:48.316 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T18:50:48.316 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T18:50:48.316 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:50:48.316 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.316 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.316 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 7, 2024-09-20T18:50:48.316 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "07d16806-e2c8-4acb-b284-d75701e61622", 2024-09-20T18:50:48.316 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T18:50:48.316 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T18:50:48.316 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T18:50:48.316 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T18:50:48.316 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T18:50:48.317 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T18:50:48.317 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 53, 2024-09-20T18:50:48.317 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 64, 2024-09-20T18:50:48.317 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T18:50:48.317 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T18:50:48.317 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T18:50:48.317 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:50:48.317 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.317 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:50:48.317 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6812", 2024-09-20T18:50:48.317 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3679770984 2024-09-20T18:50:48.317 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:50:48.317 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:50:48.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.318 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T18:50:48.318 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:50:48.318 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.318 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:50:48.318 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6813", 2024-09-20T18:50:48.318 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3679770984 2024-09-20T18:50:48.318 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:50:48.318 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:50:48.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.318 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T18:50:48.318 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:50:48.318 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.318 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:50:48.319 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6815", 2024-09-20T18:50:48.319 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3679770984 2024-09-20T18:50:48.319 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:50:48.319 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:50:48.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.319 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T18:50:48.319 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:50:48.319 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.319 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:50:48.319 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6814", 2024-09-20T18:50:48.319 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3679770984 2024-09-20T18:50:48.319 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:50:48.319 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:50:48.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.320 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.142:6812/3679770984", 2024-09-20T18:50:48.320 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.142:6813/3679770984", 2024-09-20T18:50:48.320 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.142:6815/3679770984", 2024-09-20T18:50:48.320 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.142:6814/3679770984", 2024-09-20T18:50:48.320 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T18:50:48.320 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T18:50:48.320 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T18:50:48.320 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:50:48.320 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:50:48.320 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T18:50:48.320 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_xinfo": [ 2024-09-20T18:50:48.320 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.320 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 0, 2024-09-20T18:50:48.321 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T18:50:48.321 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T18:50:48.321 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T18:50:48.321 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T18:50:48.321 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T18:50:48.321 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T18:50:11.165148+0000", 2024-09-20T18:50:48.321 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T18:50:48.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.321 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.321 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 1, 2024-09-20T18:50:48.321 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T18:50:48.321 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T18:50:48.321 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T18:50:48.321 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T18:50:48.322 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T18:50:48.322 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T18:50:22.526229+0000", 2024-09-20T18:50:48.322 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T18:50:48.322 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.322 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.322 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 2, 2024-09-20T18:50:48.322 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T18:50:48.322 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T18:50:48.322 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T18:50:48.322 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T18:50:48.322 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T18:50:48.322 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T18:49:39.137302+0000", 2024-09-20T18:50:48.322 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T18:50:48.322 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.323 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.323 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 3, 2024-09-20T18:50:48.323 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T18:50:48.323 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T18:50:48.323 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T18:50:48.323 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T18:50:48.323 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T18:50:48.323 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T18:50:38.304985+0000", 2024-09-20T18:50:48.323 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T18:50:48.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.323 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.323 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 4, 2024-09-20T18:50:48.323 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T18:50:48.323 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T18:50:48.324 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T18:50:48.324 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T18:50:48.324 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T18:50:48.324 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T18:50:27.991715+0000", 2024-09-20T18:50:48.324 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T18:50:48.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.324 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.324 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 5, 2024-09-20T18:50:48.324 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T18:50:48.324 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T18:50:48.324 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T18:50:48.324 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T18:50:48.324 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T18:50:48.324 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T18:50:04.573333+0000", 2024-09-20T18:50:48.325 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T18:50:48.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.325 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.325 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 6, 2024-09-20T18:50:48.325 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T18:50:48.325 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T18:50:48.325 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T18:50:48.325 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T18:50:48.325 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T18:50:48.325 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T18:50:20.101102+0000", 2024-09-20T18:50:48.325 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T18:50:48.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.325 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.325 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 7, 2024-09-20T18:50:48.326 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T18:50:48.326 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T18:50:48.326 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T18:50:48.326 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T18:50:48.326 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T18:50:48.326 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T18:49:33.595929+0000", 2024-09-20T18:50:48.326 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T18:50:48.326 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:50:48.326 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T18:50:48.326 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_upmap": [], 2024-09-20T18:50:48.326 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_upmap_items": [ 2024-09-20T18:50:48.326 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.326 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "4.f", 2024-09-20T18:50:48.326 INFO:teuthology.orchestra.run.smithi083.stdout: "mappings": [ 2024-09-20T18:50:48.327 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.327 INFO:teuthology.orchestra.run.smithi083.stdout: "from": 1, 2024-09-20T18:50:48.327 INFO:teuthology.orchestra.run.smithi083.stdout: "to": 2 2024-09-20T18:50:48.327 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:50:48.327 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:50:48.327 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.327 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.327 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "4.14", 2024-09-20T18:50:48.327 INFO:teuthology.orchestra.run.smithi083.stdout: "mappings": [ 2024-09-20T18:50:48.327 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.327 INFO:teuthology.orchestra.run.smithi083.stdout: "from": 1, 2024-09-20T18:50:48.327 INFO:teuthology.orchestra.run.smithi083.stdout: "to": 5 2024-09-20T18:50:48.328 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:50:48.328 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:50:48.328 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.328 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.328 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "4.1b", 2024-09-20T18:50:48.328 INFO:teuthology.orchestra.run.smithi083.stdout: "mappings": [ 2024-09-20T18:50:48.328 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:50:48.328 INFO:teuthology.orchestra.run.smithi083.stdout: "from": 1, 2024-09-20T18:50:48.328 INFO:teuthology.orchestra.run.smithi083.stdout: "to": 2 2024-09-20T18:50:48.328 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:50:48.328 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:50:48.328 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:50:48.328 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T18:50:48.328 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_upmap_primaries": [], 2024-09-20T18:50:48.329 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_temp": [], 2024-09-20T18:50:48.329 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_temp": [], 2024-09-20T18:50:48.329 INFO:teuthology.orchestra.run.smithi083.stdout: "blocklist": { 2024-09-20T18:50:48.329 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/3474253603": "2024-09-21T18:44:32.434180+0000", 2024-09-20T18:50:48.329 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:6800/3525856829": "2024-09-21T18:44:32.434180+0000", 2024-09-20T18:50:48.329 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/220746284": "2024-09-21T18:44:32.434180+0000", 2024-09-20T18:50:48.329 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/3875445393": "2024-09-21T18:44:56.985620+0000", 2024-09-20T18:50:48.329 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/3009092807": "2024-09-21T18:44:56.985620+0000", 2024-09-20T18:50:48.329 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/4267132131": "2024-09-21T18:44:56.985620+0000", 2024-09-20T18:50:48.329 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/3684382972": "2024-09-21T18:50:14.729526+0000", 2024-09-20T18:50:48.329 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/3132933178": "2024-09-21T18:50:14.729526+0000", 2024-09-20T18:50:48.329 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:6800/1007846146": "2024-09-21T18:50:14.729526+0000", 2024-09-20T18:50:48.329 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:6800/2985596191": "2024-09-21T18:44:56.985620+0000", 2024-09-20T18:50:48.329 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/2410768464": "2024-09-21T18:50:14.729526+0000", 2024-09-20T18:50:48.330 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/1868694097": "2024-09-21T18:50:14.729526+0000", 2024-09-20T18:50:48.330 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/3777331122": "2024-09-21T18:44:32.434180+0000", 2024-09-20T18:50:48.330 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/2161728215": "2024-09-21T18:50:14.729526+0000" 2024-09-20T18:50:48.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.330 INFO:teuthology.orchestra.run.smithi083.stdout: "range_blocklist": {}, 2024-09-20T18:50:48.330 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profiles": { 2024-09-20T18:50:48.330 INFO:teuthology.orchestra.run.smithi083.stdout: "default": { 2024-09-20T18:50:48.330 INFO:teuthology.orchestra.run.smithi083.stdout: "crush-failure-domain": "osd", 2024-09-20T18:50:48.330 INFO:teuthology.orchestra.run.smithi083.stdout: "k": "2", 2024-09-20T18:50:48.330 INFO:teuthology.orchestra.run.smithi083.stdout: "m": "1", 2024-09-20T18:50:48.330 INFO:teuthology.orchestra.run.smithi083.stdout: "plugin": "jerasure", 2024-09-20T18:50:48.330 INFO:teuthology.orchestra.run.smithi083.stdout: "technique": "reed_sol_van" 2024-09-20T18:50:48.330 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:50:48.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:50:48.331 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps_queue": [], 2024-09-20T18:50:48.331 INFO:teuthology.orchestra.run.smithi083.stdout: "new_removed_snaps": [], 2024-09-20T18:50:48.331 INFO:teuthology.orchestra.run.smithi083.stdout: "new_purged_snaps": [], 2024-09-20T18:50:48.331 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_node_flags": {}, 2024-09-20T18:50:48.331 INFO:teuthology.orchestra.run.smithi083.stdout: "device_class_flags": {}, 2024-09-20T18:50:48.331 INFO:teuthology.orchestra.run.smithi083.stdout: "stretch_mode": { 2024-09-20T18:50:48.331 INFO:teuthology.orchestra.run.smithi083.stdout: "stretch_mode_enabled": false, 2024-09-20T18:50:48.331 INFO:teuthology.orchestra.run.smithi083.stdout: "stretch_bucket_count": 0, 2024-09-20T18:50:48.331 INFO:teuthology.orchestra.run.smithi083.stdout: "degraded_stretch_mode": 0, 2024-09-20T18:50:48.331 INFO:teuthology.orchestra.run.smithi083.stdout: "recovering_stretch_mode": 0, 2024-09-20T18:50:48.331 INFO:teuthology.orchestra.run.smithi083.stdout: "stretch_mode_bucket": 0 2024-09-20T18:50:48.331 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:50:48.331 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T18:50:48.332 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:50:48.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:48.354 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:50:48.354 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:50:48.455 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:50:48.477 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:48.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:48.478 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:50:48.578 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:50:48.600 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:48.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:48.601 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:50:48.702 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:50:48.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:48.724 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:48.724 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:50:48.825 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:50:48.840 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'unique_pool_0' created 2024-09-20T18:50:48.848 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-20T18:50:48.849 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:48.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:48.850 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:50:48.951 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:50:48.973 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:50:48.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:48.974 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:50:49.054 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 18:50:48 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-alertmanager-a[68238]: ts=2024-09-20T18:50:48.783Z caller=cluster.go:700 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.001824692s 2024-09-20T18:50:49.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:48 smithi083 ceph-mon[29421]: 4.d scrub starts 2024-09-20T18:50:49.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:48 smithi083 ceph-mon[29421]: 4.d scrub ok 2024-09-20T18:50:49.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:48 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2466751430' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-20T18:50:49.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:48 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3942319158' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:50:49.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:48 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:50:49.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:48 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1411588373' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16} : dispatch 2024-09-20T18:50:49.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:48 smithi083 ceph-mon[36260]: 4.d scrub starts 2024-09-20T18:50:49.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:48 smithi083 ceph-mon[36260]: 4.d scrub ok 2024-09-20T18:50:49.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:48 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2466751430' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-20T18:50:49.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:48 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3942319158' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:50:49.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:48 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:50:49.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:48 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1411588373' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16} : dispatch 2024-09-20T18:50:49.070 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:48 smithi142 ceph-mon[32508]: 4.d scrub starts 2024-09-20T18:50:49.070 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:48 smithi142 ceph-mon[32508]: 4.d scrub ok 2024-09-20T18:50:49.070 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:48 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2466751430' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-20T18:50:49.070 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:48 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3942319158' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:50:49.070 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:48 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:50:49.070 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:48 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1411588373' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16} : dispatch 2024-09-20T18:50:49.075 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:50:49.096 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:49.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:49.097 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:50:49.198 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:50:49.219 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:49.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:49.220 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:50:49.321 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:50:49.344 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:49.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:49.345 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:50:49.370 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:50:49.370 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:50:49.446 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:50:49.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:49.469 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:49.469 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:50:49.485 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 systemd[1]: Starting Ceph grafana.a for 1ed5a50a-7780-11ef-baf5-efdc52797490... 2024-09-20T18:50:49.569 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:50:49.591 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:49.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:49.592 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:50:49.692 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:50:49.714 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:50:49.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:49.715 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:50:49.774 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 podman[60991]: 2024-09-20 18:50:49.371639847 +0000 UTC m=+0.016544615 image pull 5aad1d7cf004c80a4acb170e1c80fb85781b022f00f08c6bcd22608882b4c748 quay.io/ceph/grafana:10.4.8 2024-09-20T18:50:49.774 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 podman[60991]: 2024-09-20 18:50:49.493422347 +0000 UTC m=+0.138327117 container create d6156fc1566a6dd842aa6ea09bdc43c5b2782008b443904ba3fd7694a4c7b76d (image=quay.io/ceph/grafana:10.4.8, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a, maintainer=Grafana Labs ) 2024-09-20T18:50:49.774 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 podman[60991]: 2024-09-20 18:50:49.70432115 +0000 UTC m=+0.349225917 container init d6156fc1566a6dd842aa6ea09bdc43c5b2782008b443904ba3fd7694a4c7b76d (image=quay.io/ceph/grafana:10.4.8, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a, maintainer=Grafana Labs ) 2024-09-20T18:50:49.774 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 podman[60991]: 2024-09-20 18:50:49.707278005 +0000 UTC m=+0.352182773 container start d6156fc1566a6dd842aa6ea09bdc43c5b2782008b443904ba3fd7694a4c7b76d (image=quay.io/ceph/grafana:10.4.8, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a, maintainer=Grafana Labs ) 2024-09-20T18:50:49.816 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:50:49.839 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:49.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:49.840 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:50:49.941 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:50:49.965 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:50:49.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:49.966 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:50:50.027 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-mon[32508]: 2.2 scrub starts 2024-09-20T18:50:50.027 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-mon[32508]: 2.2 scrub ok 2024-09-20T18:50:50.027 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-mon[32508]: pgmap v28: 132 pgs: 132 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-20T18:50:50.027 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-mon[32508]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:50:50.027 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T18:50:50.027 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1411588373' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]': finished 2024-09-20T18:50:50.027 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-mon[32508]: osdmap e83: 8 total, 8 up, 8 in 2024-09-20T18:50:50.028 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 bash[60991]: d6156fc1566a6dd842aa6ea09bdc43c5b2782008b443904ba3fd7694a4c7b76d 2024-09-20T18:50:50.028 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 systemd[1]: Started Ceph grafana.a for 1ed5a50a-7780-11ef-baf5-efdc52797490. 2024-09-20T18:50:50.028 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=settings t=2024-09-20T18:50:49.82691152Z level=info msg="Starting Grafana" version=10.4.8 commit=8c384b71bde142a25ddc01ed5a183ca572b2b7f7 branch=v10.4.x compiled=2024-09-20T18:50:49Z 2024-09-20T18:50:50.028 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=settings t=2024-09-20T18:50:49.827206377Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-09-20T18:50:50.028 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=settings t=2024-09-20T18:50:49.8272158Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-09-20T18:50:50.029 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=settings t=2024-09-20T18:50:49.827220346Z level=info msg="Config overridden from command line" arg="default.paths.data=/var/lib/grafana" 2024-09-20T18:50:50.029 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=settings t=2024-09-20T18:50:49.827224389Z level=info msg="Config overridden from command line" arg="default.paths.logs=/var/log/grafana" 2024-09-20T18:50:50.029 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=settings t=2024-09-20T18:50:49.827227901Z level=info msg="Config overridden from command line" arg="default.paths.plugins=/var/lib/grafana/plugins" 2024-09-20T18:50:50.029 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=settings t=2024-09-20T18:50:49.827232533Z level=info msg="Config overridden from command line" arg="default.paths.provisioning=/etc/grafana/provisioning" 2024-09-20T18:50:50.029 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=settings t=2024-09-20T18:50:49.827236015Z level=info msg="Config overridden from command line" arg="default.log.mode=console" 2024-09-20T18:50:50.029 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=settings t=2024-09-20T18:50:49.827240244Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-09-20T18:50:50.029 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=settings t=2024-09-20T18:50:49.82724417Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-09-20T18:50:50.029 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=settings t=2024-09-20T18:50:49.827247663Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-09-20T18:50:50.029 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=settings t=2024-09-20T18:50:49.827251159Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-09-20T18:50:50.029 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=settings t=2024-09-20T18:50:49.82725474Z level=info msg=Target target=[all] 2024-09-20T18:50:50.029 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=settings t=2024-09-20T18:50:49.827263389Z level=info msg="Path Home" path=/usr/share/grafana 2024-09-20T18:50:50.029 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=settings t=2024-09-20T18:50:49.827266975Z level=info msg="Path Data" path=/var/lib/grafana 2024-09-20T18:50:50.029 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=settings t=2024-09-20T18:50:49.827270681Z level=info msg="Path Logs" path=/var/log/grafana 2024-09-20T18:50:50.029 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=settings t=2024-09-20T18:50:49.827274002Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-09-20T18:50:50.030 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=settings t=2024-09-20T18:50:49.827277503Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-09-20T18:50:50.030 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=settings t=2024-09-20T18:50:49.827280962Z level=info msg="App mode production" 2024-09-20T18:50:50.030 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=sqlstore t=2024-09-20T18:50:49.827610168Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-09-20T18:50:50.030 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=sqlstore t=2024-09-20T18:50:49.827627032Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-09-20T18:50:50.030 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.828895321Z level=info msg="Starting DB migrations" 2024-09-20T18:50:50.030 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.830350635Z level=info msg="Executing migration" id="create migration_log table" 2024-09-20T18:50:50.030 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.83149407Z level=info msg="Migration successfully executed" id="create migration_log table" duration=1.139452ms 2024-09-20T18:50:50.030 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.832236287Z level=info msg="Executing migration" id="create user table" 2024-09-20T18:50:50.030 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.833230852Z level=info msg="Migration successfully executed" id="create user table" duration=978.293µs 2024-09-20T18:50:50.030 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.834682362Z level=info msg="Executing migration" id="add unique index user.login" 2024-09-20T18:50:50.030 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.835676425Z level=info msg="Migration successfully executed" id="add unique index user.login" duration=996.854µs 2024-09-20T18:50:50.030 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.836230204Z level=info msg="Executing migration" id="add unique index user.email" 2024-09-20T18:50:50.030 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.83739143Z level=info msg="Migration successfully executed" id="add unique index user.email" duration=1.159864ms 2024-09-20T18:50:50.030 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.839783542Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-09-20T18:50:50.031 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.840801996Z level=info msg="Migration successfully executed" id="drop index UQE_user_login - v1" duration=1.019645ms 2024-09-20T18:50:50.031 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.841647012Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-09-20T18:50:50.031 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.842602483Z level=info msg="Migration successfully executed" id="drop index UQE_user_email - v1" duration=952.074µs 2024-09-20T18:50:50.031 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.843301382Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-09-20T18:50:50.031 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.845978862Z level=info msg="Migration successfully executed" id="Rename table user to user_v1 - v1" duration=2.675325ms 2024-09-20T18:50:50.031 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.846512352Z level=info msg="Executing migration" id="create user table v2" 2024-09-20T18:50:50.031 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.8471674Z level=info msg="Migration successfully executed" id="create user table v2" duration=655.809µs 2024-09-20T18:50:50.031 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.847553232Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-09-20T18:50:50.031 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.848048912Z level=info msg="Migration successfully executed" id="create index UQE_user_login - v2" duration=495.991µs 2024-09-20T18:50:50.031 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.848395715Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-09-20T18:50:50.031 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.848859321Z level=info msg="Migration successfully executed" id="create index UQE_user_email - v2" duration=463.981µs 2024-09-20T18:50:50.031 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.849256842Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-09-20T18:50:50.031 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.849519714Z level=info msg="Migration successfully executed" id="copy data_source v1 to v2" duration=262.812µs 2024-09-20T18:50:50.032 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.850093138Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-09-20T18:50:50.032 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.850471918Z level=info msg="Migration successfully executed" id="Drop old table user_v1" duration=378.553µs 2024-09-20T18:50:50.032 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.85096179Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-09-20T18:50:50.032 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.851680439Z level=info msg="Migration successfully executed" id="Add column help_flags1 to user table" duration=719.087µs 2024-09-20T18:50:50.032 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.852043857Z level=info msg="Executing migration" id="Update user table charset" 2024-09-20T18:50:50.032 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.852059943Z level=info msg="Migration successfully executed" id="Update user table charset" duration=16.741µs 2024-09-20T18:50:50.032 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.852874603Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-09-20T18:50:50.032 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.853575424Z level=info msg="Migration successfully executed" id="Add last_seen_at column to user" duration=700.519µs 2024-09-20T18:50:50.032 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.853913674Z level=info msg="Executing migration" id="Add missing user data" 2024-09-20T18:50:50.032 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.854094417Z level=info msg="Migration successfully executed" id="Add missing user data" duration=181.045µs 2024-09-20T18:50:50.032 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.8547788Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-09-20T18:50:50.032 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.855474411Z level=info msg="Migration successfully executed" id="Add is_disabled column to user" duration=695.588µs 2024-09-20T18:50:50.032 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.855884621Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-09-20T18:50:50.032 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.856490509Z level=info msg="Migration successfully executed" id="Add index user.login/user.email" duration=605.421µs 2024-09-20T18:50:50.033 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.856921087Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-09-20T18:50:50.033 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.857920402Z level=info msg="Migration successfully executed" id="Add is_service_account column to user" duration=999.18µs 2024-09-20T18:50:50.033 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.858350156Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-09-20T18:50:50.033 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.864723172Z level=info msg="Migration successfully executed" id="Update is_service_account column to nullable" duration=6.372695ms 2024-09-20T18:50:50.033 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.865831673Z level=info msg="Executing migration" id="Add uid column to user" 2024-09-20T18:50:50.033 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.866886117Z level=info msg="Migration successfully executed" id="Add uid column to user" duration=1.054875ms 2024-09-20T18:50:50.033 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.867273169Z level=info msg="Executing migration" id="Update uid column values for users" 2024-09-20T18:50:50.033 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.867433658Z level=info msg="Migration successfully executed" id="Update uid column values for users" duration=162.365µs 2024-09-20T18:50:50.033 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.868345922Z level=info msg="Executing migration" id="Add unique index user_uid" 2024-09-20T18:50:50.033 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.868977497Z level=info msg="Migration successfully executed" id="Add unique index user_uid" duration=633.312µs 2024-09-20T18:50:50.033 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.869390996Z level=info msg="Executing migration" id="update login field with orgid to allow for multiple service accounts with same name across orgs" 2024-09-20T18:50:50.033 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.86960242Z level=info msg="Migration successfully executed" id="update login field with orgid to allow for multiple service accounts with same name across orgs" duration=211.637µs 2024-09-20T18:50:50.033 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.870053652Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-09-20T18:50:50.033 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.870572566Z level=info msg="Migration successfully executed" id="create temp user table v1-7" duration=519.008µs 2024-09-20T18:50:50.035 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.871356636Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-09-20T18:50:50.035 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.871855809Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_email - v1-7" duration=498.908µs 2024-09-20T18:50:50.035 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.872236308Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-09-20T18:50:50.035 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.872688709Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_org_id - v1-7" duration=452.143µs 2024-09-20T18:50:50.035 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.873039817Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-09-20T18:50:50.035 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.873487701Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_code - v1-7" duration=455.046µs 2024-09-20T18:50:50.036 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.873837854Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-09-20T18:50:50.036 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.874290909Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_status - v1-7" duration=452.843µs 2024-09-20T18:50:50.036 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.874627255Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-09-20T18:50:50.036 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.874647686Z level=info msg="Migration successfully executed" id="Update temp_user table charset" duration=20.384µs 2024-09-20T18:50:50.036 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.875066653Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-09-20T18:50:50.036 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.875525309Z level=info msg="Migration successfully executed" id="drop index IDX_temp_user_email - v1" duration=458.484µs 2024-09-20T18:50:50.036 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.875863264Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-09-20T18:50:50.036 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.876325433Z level=info msg="Migration successfully executed" id="drop index IDX_temp_user_org_id - v1" duration=463.267µs 2024-09-20T18:50:50.036 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.876663086Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-09-20T18:50:50.036 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.877117387Z level=info msg="Migration successfully executed" id="drop index IDX_temp_user_code - v1" duration=454.084µs 2024-09-20T18:50:50.036 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.877454696Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-09-20T18:50:50.036 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.877888547Z level=info msg="Migration successfully executed" id="drop index IDX_temp_user_status - v1" duration=433.689µs 2024-09-20T18:50:50.036 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.878250094Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-09-20T18:50:50.037 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.880226903Z level=info msg="Migration successfully executed" id="Rename table temp_user to temp_user_tmp_qwerty - v1" duration=1.976416ms 2024-09-20T18:50:50.037 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.88057679Z level=info msg="Executing migration" id="create temp_user v2" 2024-09-20T18:50:50.037 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.881124598Z level=info msg="Migration successfully executed" id="create temp_user v2" duration=547.546µs 2024-09-20T18:50:50.037 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.881464924Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-09-20T18:50:50.037 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.881919937Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_email - v2" duration=453.704µs 2024-09-20T18:50:50.037 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.88226087Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-09-20T18:50:50.037 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.882716403Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_org_id - v2" duration=455.383µs 2024-09-20T18:50:50.037 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.883057271Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-09-20T18:50:50.037 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.883524768Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_code - v2" duration=466.504µs 2024-09-20T18:50:50.037 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.883869704Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-09-20T18:50:50.037 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.884352652Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_status - v2" duration=482.714µs 2024-09-20T18:50:50.037 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.884700757Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-09-20T18:50:50.037 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.884966147Z level=info msg="Migration successfully executed" id="copy temp_user v1 to v2" duration=265.137µs 2024-09-20T18:50:50.038 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.885333147Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-09-20T18:50:50.038 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.885687699Z level=info msg="Migration successfully executed" id="drop temp_user_tmp_qwerty" duration=354.498µs 2024-09-20T18:50:50.038 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.886018998Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-09-20T18:50:50.038 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.886279283Z level=info msg="Migration successfully executed" id="Set created for temp users that will otherwise prematurely expire" duration=260.196µs 2024-09-20T18:50:50.038 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.886694238Z level=info msg="Executing migration" id="create star table" 2024-09-20T18:50:50.038 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.887123124Z level=info msg="Migration successfully executed" id="create star table" duration=428.605µs 2024-09-20T18:50:50.038 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.8874607Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-09-20T18:50:50.038 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.887926866Z level=info msg="Migration successfully executed" id="add unique index star.user_id_dashboard_id" duration=465.827µs 2024-09-20T18:50:50.038 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.88830343Z level=info msg="Executing migration" id="create org table v1" 2024-09-20T18:50:50.038 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.888774572Z level=info msg="Migration successfully executed" id="create org table v1" duration=470.795µs 2024-09-20T18:50:50.038 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.889112144Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-09-20T18:50:50.038 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.889568858Z level=info msg="Migration successfully executed" id="create index UQE_org_name - v1" duration=456.482µs 2024-09-20T18:50:50.038 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.889911748Z level=info msg="Executing migration" id="create org_user table v1" 2024-09-20T18:50:50.039 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.890380562Z level=info msg="Migration successfully executed" id="create org_user table v1" duration=468.872µs 2024-09-20T18:50:50.039 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.890758855Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-09-20T18:50:50.039 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.891264782Z level=info msg="Migration successfully executed" id="create index IDX_org_user_org_id - v1" duration=505.863µs 2024-09-20T18:50:50.039 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.893644045Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-09-20T18:50:50.039 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.894290962Z level=info msg="Migration successfully executed" id="create index UQE_org_user_org_id_user_id - v1" duration=648.731µs 2024-09-20T18:50:50.039 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.896156142Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-09-20T18:50:50.039 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.89663652Z level=info msg="Migration successfully executed" id="create index IDX_org_user_user_id - v1" duration=480.209µs 2024-09-20T18:50:50.039 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.899085308Z level=info msg="Executing migration" id="Update org table charset" 2024-09-20T18:50:50.039 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.899123977Z level=info msg="Migration successfully executed" id="Update org table charset" duration=40.202µs 2024-09-20T18:50:50.039 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.901796811Z level=info msg="Executing migration" id="Update org_user table charset" 2024-09-20T18:50:50.039 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.90183649Z level=info msg="Migration successfully executed" id="Update org_user table charset" duration=41.129µs 2024-09-20T18:50:50.039 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.904689206Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-09-20T18:50:50.039 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.904793428Z level=info msg="Migration successfully executed" id="Migrate all Read Only Viewers to Viewers" duration=104.613µs 2024-09-20T18:50:50.039 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.907410254Z level=info msg="Executing migration" id="create dashboard table" 2024-09-20T18:50:50.040 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.907937476Z level=info msg="Migration successfully executed" id="create dashboard table" duration=528.758µs 2024-09-20T18:50:50.040 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.91002372Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-09-20T18:50:50.040 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.910572011Z level=info msg="Migration successfully executed" id="add index dashboard.account_id" duration=547.947µs 2024-09-20T18:50:50.040 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.912897417Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-09-20T18:50:50.040 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.913452425Z level=info msg="Migration successfully executed" id="add unique index dashboard_account_id_slug" duration=555.732µs 2024-09-20T18:50:50.040 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.915847921Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-09-20T18:50:50.040 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.916367452Z level=info msg="Migration successfully executed" id="create dashboard_tag table" duration=521.897µs 2024-09-20T18:50:50.040 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.91839099Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-09-20T18:50:50.040 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.918871641Z level=info msg="Migration successfully executed" id="add unique index dashboard_tag.dasboard_id_term" duration=480.297µs 2024-09-20T18:50:50.040 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.921159091Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-09-20T18:50:50.040 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.92161999Z level=info msg="Migration successfully executed" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" duration=443.17µs 2024-09-20T18:50:50.040 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.924126555Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-09-20T18:50:50.040 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.927396092Z level=info msg="Migration successfully executed" id="Rename table dashboard to dashboard_v1 - v1" duration=3.269107ms 2024-09-20T18:50:50.041 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.927771287Z level=info msg="Executing migration" id="create dashboard v2" 2024-09-20T18:50:50.041 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.928266636Z level=info msg="Migration successfully executed" id="create dashboard v2" duration=495.019µs 2024-09-20T18:50:50.041 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.930705294Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-09-20T18:50:50.041 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.931162233Z level=info msg="Migration successfully executed" id="create index IDX_dashboard_org_id - v2" duration=456.694µs 2024-09-20T18:50:50.041 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.933446035Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-09-20T18:50:50.041 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.933952959Z level=info msg="Migration successfully executed" id="create index UQE_dashboard_org_id_slug - v2" duration=506.693µs 2024-09-20T18:50:50.041 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.936243011Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-09-20T18:50:50.041 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.936493364Z level=info msg="Migration successfully executed" id="copy dashboard v1 to v2" duration=250.476µs 2024-09-20T18:50:50.041 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.93922498Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-09-20T18:50:50.041 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.939869173Z level=info msg="Migration successfully executed" id="drop table dashboard_v1" duration=646.181µs 2024-09-20T18:50:50.041 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.94187513Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-09-20T18:50:50.041 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.941933333Z level=info msg="Migration successfully executed" id="alter dashboard.data to mediumtext v1" duration=59.205µs 2024-09-20T18:50:50.041 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.94440402Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-09-20T18:50:50.041 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.945729284Z level=info msg="Migration successfully executed" id="Add column updated_by in dashboard - v2" duration=1.324429ms 2024-09-20T18:50:50.042 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.947122489Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-09-20T18:50:50.042 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.948389176Z level=info msg="Migration successfully executed" id="Add column created_by in dashboard - v2" duration=1.266229ms 2024-09-20T18:50:50.042 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.949762143Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-09-20T18:50:50.042 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.950939877Z level=info msg="Migration successfully executed" id="Add column gnetId in dashboard" duration=1.177436ms 2024-09-20T18:50:50.042 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.952268338Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-09-20T18:50:50.042 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.952810723Z level=info msg="Migration successfully executed" id="Add index for gnetId in dashboard" duration=542.055µs 2024-09-20T18:50:50.042 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.954923212Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-09-20T18:50:50.042 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.956021205Z level=info msg="Migration successfully executed" id="Add column plugin_id in dashboard" duration=1.09788ms 2024-09-20T18:50:50.042 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.957459425Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-09-20T18:50:50.042 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.958012968Z level=info msg="Migration successfully executed" id="Add index for plugin_id in dashboard" duration=553.786µs 2024-09-20T18:50:50.042 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.960139575Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-09-20T18:50:50.042 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.960631416Z level=info msg="Migration successfully executed" id="Add index for dashboard_id in dashboard_tag" duration=492.587µs 2024-09-20T18:50:50.042 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.96280726Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-09-20T18:50:50.042 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.962839676Z level=info msg="Migration successfully executed" id="Update dashboard table charset" duration=32.917µs 2024-09-20T18:50:50.043 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.963335567Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-09-20T18:50:50.043 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.963351736Z level=info msg="Migration successfully executed" id="Update dashboard_tag table charset" duration=16.956µs 2024-09-20T18:50:50.043 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.965329918Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-09-20T18:50:50.043 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.966561192Z level=info msg="Migration successfully executed" id="Add column folder_id in dashboard" duration=1.231079ms 2024-09-20T18:50:50.043 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.967912312Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-09-20T18:50:50.043 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.969105567Z level=info msg="Migration successfully executed" id="Add column isFolder in dashboard" duration=1.192805ms 2024-09-20T18:50:50.043 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.970612684Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-09-20T18:50:50.043 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.971817947Z level=info msg="Migration successfully executed" id="Add column has_acl in dashboard" duration=1.207694ms 2024-09-20T18:50:50.043 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.973089779Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-09-20T18:50:50.043 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.974253753Z level=info msg="Migration successfully executed" id="Add column uid in dashboard" duration=1.163478ms 2024-09-20T18:50:50.043 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.975889033Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-09-20T18:50:50.043 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.976120468Z level=info msg="Migration successfully executed" id="Update uid column values in dashboard" duration=233.71µs 2024-09-20T18:50:50.043 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.978411203Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-09-20T18:50:50.044 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.978902957Z level=info msg="Migration successfully executed" id="Add unique index dashboard_org_id_uid" duration=532.532µs 2024-09-20T18:50:50.044 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.980725831Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-09-20T18:50:50.044 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.981218219Z level=info msg="Migration successfully executed" id="Remove unique index org_id_slug" duration=492.983µs 2024-09-20T18:50:50.044 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.983237079Z level=info msg="Executing migration" id="Update dashboard title length" 2024-09-20T18:50:50.044 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.983252814Z level=info msg="Migration successfully executed" id="Update dashboard title length" duration=16.322µs 2024-09-20T18:50:50.044 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.98599516Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-09-20T18:50:50.044 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.986539217Z level=info msg="Migration successfully executed" id="Add unique index for dashboard_org_id_title_folder_id" duration=543.73µs 2024-09-20T18:50:50.044 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.986945308Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-09-20T18:50:50.044 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.987433429Z level=info msg="Migration successfully executed" id="create dashboard_provisioning" duration=487.859µs 2024-09-20T18:50:50.044 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.987841175Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-09-20T18:50:50.044 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.990979427Z level=info msg="Migration successfully executed" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" duration=3.137981ms 2024-09-20T18:50:50.044 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.991385122Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-09-20T18:50:50.044 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.99186503Z level=info msg="Migration successfully executed" id="create dashboard_provisioning v2" duration=479.71µs 2024-09-20T18:50:50.044 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.992257391Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-09-20T18:50:50.045 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.992770144Z level=info msg="Migration successfully executed" id="create index IDX_dashboard_provisioning_dashboard_id - v2" duration=512.688µs 2024-09-20T18:50:50.045 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.993155531Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-09-20T18:50:50.045 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.993670543Z level=info msg="Migration successfully executed" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" duration=514.742µs 2024-09-20T18:50:50.045 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.994079477Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-09-20T18:50:50.045 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.994311826Z level=info msg="Migration successfully executed" id="copy dashboard_provisioning v1 to v2" duration=231.473µs 2024-09-20T18:50:50.045 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.994962461Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-09-20T18:50:50.045 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.995371379Z level=info msg="Migration successfully executed" id="drop dashboard_provisioning_tmp_qwerty" duration=409.497µs 2024-09-20T18:50:50.045 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.995766408Z level=info msg="Executing migration" id="Add check_sum column" 2024-09-20T18:50:50.045 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.996954468Z level=info msg="Migration successfully executed" id="Add check_sum column" duration=1.187961ms 2024-09-20T18:50:50.045 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.997348508Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-09-20T18:50:50.045 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.997829424Z level=info msg="Migration successfully executed" id="Add index for dashboard_title" duration=480.621µs 2024-09-20T18:50:50.045 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.998187195Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-09-20T18:50:50.046 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.998314663Z level=info msg="Migration successfully executed" id="delete tags for deleted dashboards" duration=127.313µs 2024-09-20T18:50:50.046 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.999028672Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-09-20T18:50:50.046 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.999181903Z level=info msg="Migration successfully executed" id="delete stars for deleted dashboards" duration=151.85µs 2024-09-20T18:50:50.046 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:49 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:49.999912332Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-09-20T18:50:50.046 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.00042514Z level=info msg="Migration successfully executed" id="Add index for dashboard_is_folder" duration=512.602µs 2024-09-20T18:50:50.046 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.000839308Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-09-20T18:50:50.046 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.002106774Z level=info msg="Migration successfully executed" id="Add isPublic for dashboard" duration=1.267168ms 2024-09-20T18:50:50.046 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.002529689Z level=info msg="Executing migration" id="create data_source table" 2024-09-20T18:50:50.046 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.003185714Z level=info msg="Migration successfully executed" id="create data_source table" duration=655.784µs 2024-09-20T18:50:50.046 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.003636094Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-09-20T18:50:50.046 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.004152431Z level=info msg="Migration successfully executed" id="add index data_source.account_id" duration=516.768µs 2024-09-20T18:50:50.046 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.004541364Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-09-20T18:50:50.046 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.005050839Z level=info msg="Migration successfully executed" id="add unique index data_source.account_id_name" duration=508.217µs 2024-09-20T18:50:50.047 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.005455281Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-09-20T18:50:50.047 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.00593063Z level=info msg="Migration successfully executed" id="drop index IDX_data_source_account_id - v1" duration=475.227µs 2024-09-20T18:50:50.047 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.006302027Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-09-20T18:50:50.047 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.00677858Z level=info msg="Migration successfully executed" id="drop index UQE_data_source_account_id_name - v1" duration=476.35µs 2024-09-20T18:50:50.047 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.007143712Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-09-20T18:50:50.047 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.01056347Z level=info msg="Migration successfully executed" id="Rename table data_source to data_source_v1 - v1" duration=3.4184ms 2024-09-20T18:50:50.047 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.010991394Z level=info msg="Executing migration" id="create data_source table v2" 2024-09-20T18:50:50.047 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.011604586Z level=info msg="Migration successfully executed" id="create data_source table v2" duration=614.332µs 2024-09-20T18:50:50.047 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.011997296Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-09-20T18:50:50.047 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.012534388Z level=info msg="Migration successfully executed" id="create index IDX_data_source_org_id - v2" duration=535.698µs 2024-09-20T18:50:50.047 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.012916572Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-09-20T18:50:50.047 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.013469011Z level=info msg="Migration successfully executed" id="create index UQE_data_source_org_id_name - v2" duration=553.736µs 2024-09-20T18:50:50.047 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.013841436Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-09-20T18:50:50.047 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.01425128Z level=info msg="Migration successfully executed" id="Drop old table data_source_v1 #2" duration=409.682µs 2024-09-20T18:50:50.048 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.014722771Z level=info msg="Executing migration" id="Add column with_credentials" 2024-09-20T18:50:50.067 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:50:50.089 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:50:50.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:50.090 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:50:50.128 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:49 smithi083 ceph-mon[29421]: 2.2 scrub starts 2024-09-20T18:50:50.128 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:49 smithi083 ceph-mon[29421]: 2.2 scrub ok 2024-09-20T18:50:50.128 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:49 smithi083 ceph-mon[29421]: pgmap v28: 132 pgs: 132 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-20T18:50:50.128 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:49 smithi083 ceph-mon[29421]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:50:50.128 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:49 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T18:50:50.128 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:49 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1411588373' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]': finished 2024-09-20T18:50:50.129 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:49 smithi083 ceph-mon[29421]: osdmap e83: 8 total, 8 up, 8 in 2024-09-20T18:50:50.129 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:49 smithi083 ceph-mon[36260]: 2.2 scrub starts 2024-09-20T18:50:50.129 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:49 smithi083 ceph-mon[36260]: 2.2 scrub ok 2024-09-20T18:50:50.129 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:49 smithi083 ceph-mon[36260]: pgmap v28: 132 pgs: 132 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-20T18:50:50.129 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:49 smithi083 ceph-mon[36260]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:50:50.129 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:49 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T18:50:50.129 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:49 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1411588373' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]': finished 2024-09-20T18:50:50.129 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:49 smithi083 ceph-mon[36260]: osdmap e83: 8 total, 8 up, 8 in 2024-09-20T18:50:50.191 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:50:50.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:50.213 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:50.213 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:50:50.279 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.016143219Z level=info msg="Migration successfully executed" id="Add column with_credentials" duration=1.420088ms 2024-09-20T18:50:50.280 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.01651477Z level=info msg="Executing migration" id="Add secure json data column" 2024-09-20T18:50:50.280 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.017926659Z level=info msg="Migration successfully executed" id="Add secure json data column" duration=1.411724ms 2024-09-20T18:50:50.280 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.018300769Z level=info msg="Executing migration" id="Update data_source table charset" 2024-09-20T18:50:50.280 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.018317308Z level=info msg="Migration successfully executed" id="Update data_source table charset" duration=17.168µs 2024-09-20T18:50:50.280 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.019149061Z level=info msg="Executing migration" id="Update initial version to 1" 2024-09-20T18:50:50.280 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.019296171Z level=info msg="Migration successfully executed" id="Update initial version to 1" duration=147.086µs 2024-09-20T18:50:50.280 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.019995023Z level=info msg="Executing migration" id="Add read_only data column" 2024-09-20T18:50:50.280 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.021432828Z level=info msg="Migration successfully executed" id="Add read_only data column" duration=1.436756ms 2024-09-20T18:50:50.281 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.021804107Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-09-20T18:50:50.281 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.021948807Z level=info msg="Migration successfully executed" id="Migrate logging ds to loki ds" duration=144.773µs 2024-09-20T18:50:50.281 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.022683766Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-09-20T18:50:50.281 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.02282095Z level=info msg="Migration successfully executed" id="Update json_data with nulls" duration=137.174µs 2024-09-20T18:50:50.281 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.023508249Z level=info msg="Executing migration" id="Add uid column" 2024-09-20T18:50:50.281 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.024912854Z level=info msg="Migration successfully executed" id="Add uid column" duration=1.405259ms 2024-09-20T18:50:50.281 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.025291475Z level=info msg="Executing migration" id="Update uid value" 2024-09-20T18:50:50.281 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.025436996Z level=info msg="Migration successfully executed" id="Update uid value" duration=145.575µs 2024-09-20T18:50:50.281 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.026148101Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-09-20T18:50:50.281 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.026676246Z level=info msg="Migration successfully executed" id="Add unique index datasource_org_id_uid" duration=527.921µs 2024-09-20T18:50:50.281 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.027061177Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-09-20T18:50:50.281 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.027570256Z level=info msg="Migration successfully executed" id="add unique index datasource_org_id_is_default" duration=508.803µs 2024-09-20T18:50:50.281 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.027949624Z level=info msg="Executing migration" id="create api_key table" 2024-09-20T18:50:50.282 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.028510037Z level=info msg="Migration successfully executed" id="create api_key table" duration=559.401µs 2024-09-20T18:50:50.282 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.030813243Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-09-20T18:50:50.282 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.031500299Z level=info msg="Migration successfully executed" id="add index api_key.account_id" duration=689.271µs 2024-09-20T18:50:50.282 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.033593469Z level=info msg="Executing migration" id="add index api_key.key" 2024-09-20T18:50:50.282 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.034192117Z level=info msg="Migration successfully executed" id="add index api_key.key" duration=600.404µs 2024-09-20T18:50:50.282 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.036307209Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-09-20T18:50:50.282 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.036872526Z level=info msg="Migration successfully executed" id="add index api_key.account_id_name" duration=565.257µs 2024-09-20T18:50:50.282 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.037276149Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-09-20T18:50:50.282 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.037771706Z level=info msg="Migration successfully executed" id="drop index IDX_api_key_account_id - v1" duration=496.638µs 2024-09-20T18:50:50.282 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.038171993Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-09-20T18:50:50.282 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.038666977Z level=info msg="Migration successfully executed" id="drop index UQE_api_key_key - v1" duration=494.874µs 2024-09-20T18:50:50.282 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.039087161Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-09-20T18:50:50.282 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.03957846Z level=info msg="Migration successfully executed" id="drop index UQE_api_key_account_id_name - v1" duration=491.836µs 2024-09-20T18:50:50.283 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.039989225Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-09-20T18:50:50.285 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.043620856Z level=info msg="Migration successfully executed" id="Rename table api_key to api_key_v1 - v1" duration=3.629721ms 2024-09-20T18:50:50.285 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.044041386Z level=info msg="Executing migration" id="create api_key table v2" 2024-09-20T18:50:50.285 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.044568837Z level=info msg="Migration successfully executed" id="create api_key table v2" duration=536.465µs 2024-09-20T18:50:50.285 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.044968778Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-09-20T18:50:50.285 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.045496363Z level=info msg="Migration successfully executed" id="create index IDX_api_key_org_id - v2" duration=527.374µs 2024-09-20T18:50:50.285 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.045889409Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-09-20T18:50:50.285 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.046393434Z level=info msg="Migration successfully executed" id="create index UQE_api_key_key - v2" duration=503.981µs 2024-09-20T18:50:50.285 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.046745304Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-09-20T18:50:50.285 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.047257556Z level=info msg="Migration successfully executed" id="create index UQE_api_key_org_id_name - v2" duration=511.943µs 2024-09-20T18:50:50.285 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.047625283Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-09-20T18:50:50.285 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.047884721Z level=info msg="Migration successfully executed" id="copy api_key v1 to v2" duration=259.165µs 2024-09-20T18:50:50.286 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.048448566Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-09-20T18:50:50.286 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.048869409Z level=info msg="Migration successfully executed" id="Drop old table api_key_v1" duration=420.81µs 2024-09-20T18:50:50.286 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.049259735Z level=info msg="Executing migration" id="Update api_key table charset" 2024-09-20T18:50:50.286 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.049274317Z level=info msg="Migration successfully executed" id="Update api_key table charset" duration=15.373µs 2024-09-20T18:50:50.286 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.050095922Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-09-20T18:50:50.286 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.051582225Z level=info msg="Migration successfully executed" id="Add expires to api_key table" duration=1.486ms 2024-09-20T18:50:50.286 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.05194965Z level=info msg="Executing migration" id="Add service account foreign key" 2024-09-20T18:50:50.286 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.053483502Z level=info msg="Migration successfully executed" id="Add service account foreign key" duration=1.532815ms 2024-09-20T18:50:50.286 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.053878341Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-09-20T18:50:50.286 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.053999512Z level=info msg="Migration successfully executed" id="set service account foreign key to nil if 0" duration=122.195µs 2024-09-20T18:50:50.287 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.05466094Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-09-20T18:50:50.287 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.056225254Z level=info msg="Migration successfully executed" id="Add last_used_at to api_key table" duration=1.56409ms 2024-09-20T18:50:50.287 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.056624495Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-09-20T18:50:50.287 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.058096513Z level=info msg="Migration successfully executed" id="Add is_revoked column to api_key table" duration=1.471939ms 2024-09-20T18:50:50.287 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.058475272Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-09-20T18:50:50.287 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.05899911Z level=info msg="Migration successfully executed" id="create dashboard_snapshot table v4" duration=523.525µs 2024-09-20T18:50:50.287 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.059384368Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-09-20T18:50:50.287 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.059787254Z level=info msg="Migration successfully executed" id="drop table dashboard_snapshot_v4 #1" duration=402.861µs 2024-09-20T18:50:50.287 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.060174626Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-09-20T18:50:50.287 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.060755928Z level=info msg="Migration successfully executed" id="create dashboard_snapshot table v5 #2" duration=581.061µs 2024-09-20T18:50:50.287 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.06117606Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-09-20T18:50:50.287 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.061736199Z level=info msg="Migration successfully executed" id="create index UQE_dashboard_snapshot_key - v5" duration=560.153µs 2024-09-20T18:50:50.287 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.062098052Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-09-20T18:50:50.288 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.062644982Z level=info msg="Migration successfully executed" id="create index UQE_dashboard_snapshot_delete_key - v5" duration=546.82µs 2024-09-20T18:50:50.288 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.062986537Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-09-20T18:50:50.288 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.063484878Z level=info msg="Migration successfully executed" id="create index IDX_dashboard_snapshot_user_id - v5" duration=498.115µs 2024-09-20T18:50:50.288 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.063817491Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-09-20T18:50:50.288 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.063854764Z level=info msg="Migration successfully executed" id="alter dashboard_snapshot to mediumtext v2" duration=37.487µs 2024-09-20T18:50:50.288 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.06460114Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-09-20T18:50:50.288 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.064615717Z level=info msg="Migration successfully executed" id="Update dashboard_snapshot table charset" duration=15.145µs 2024-09-20T18:50:50.288 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.065008631Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-09-20T18:50:50.288 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.066628555Z level=info msg="Migration successfully executed" id="Add column external_delete_url to dashboard_snapshots table" duration=1.621112ms 2024-09-20T18:50:50.288 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.066987278Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-09-20T18:50:50.288 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.068644807Z level=info msg="Migration successfully executed" id="Add encrypted dashboard json column" duration=1.657259ms 2024-09-20T18:50:50.288 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.069014812Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-09-20T18:50:50.289 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.069062965Z level=info msg="Migration successfully executed" id="Change dashboard_encrypted column to MEDIUMBLOB" duration=48.039µs 2024-09-20T18:50:50.289 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.069876865Z level=info msg="Executing migration" id="create quota table v1" 2024-09-20T18:50:50.289 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.070351323Z level=info msg="Migration successfully executed" id="create quota table v1" duration=474.193µs 2024-09-20T18:50:50.289 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.070698661Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-09-20T18:50:50.289 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.071202853Z level=info msg="Migration successfully executed" id="create index UQE_quota_org_id_user_id_target - v1" duration=503.893µs 2024-09-20T18:50:50.289 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.071567939Z level=info msg="Executing migration" id="Update quota table charset" 2024-09-20T18:50:50.289 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.071581825Z level=info msg="Migration successfully executed" id="Update quota table charset" duration=14.439µs 2024-09-20T18:50:50.289 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.072442124Z level=info msg="Executing migration" id="create plugin_setting table" 2024-09-20T18:50:50.289 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.072930336Z level=info msg="Migration successfully executed" id="create plugin_setting table" duration=487.947µs 2024-09-20T18:50:50.289 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.07327425Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-09-20T18:50:50.289 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.073775869Z level=info msg="Migration successfully executed" id="create index UQE_plugin_setting_org_id_plugin_id - v1" duration=501.919µs 2024-09-20T18:50:50.289 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.074106292Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-09-20T18:50:50.289 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.075896773Z level=info msg="Migration successfully executed" id="Add column plugin_version to plugin_settings" duration=1.789853ms 2024-09-20T18:50:50.290 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.076240896Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-09-20T18:50:50.290 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.076256095Z level=info msg="Migration successfully executed" id="Update plugin_setting table charset" duration=15.839µs 2024-09-20T18:50:50.290 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.077284907Z level=info msg="Executing migration" id="create session table" 2024-09-20T18:50:50.290 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.077880769Z level=info msg="Migration successfully executed" id="create session table" duration=597.47µs 2024-09-20T18:50:50.290 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.078260534Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-09-20T18:50:50.290 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.078318702Z level=info msg="Migration successfully executed" id="Drop old table playlist table" duration=57.649µs 2024-09-20T18:50:50.290 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.079098528Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-09-20T18:50:50.290 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.079155011Z level=info msg="Migration successfully executed" id="Drop old table playlist_item table" duration=56.698µs 2024-09-20T18:50:50.290 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.079961914Z level=info msg="Executing migration" id="create playlist table v2" 2024-09-20T18:50:50.290 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.080456263Z level=info msg="Migration successfully executed" id="create playlist table v2" duration=494.091µs 2024-09-20T18:50:50.290 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.080825579Z level=info msg="Executing migration" id="create playlist item table v2" 2024-09-20T18:50:50.290 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.081327463Z level=info msg="Migration successfully executed" id="create playlist item table v2" duration=501.631µs 2024-09-20T18:50:50.290 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.081669591Z level=info msg="Executing migration" id="Update playlist table charset" 2024-09-20T18:50:50.291 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.081684341Z level=info msg="Migration successfully executed" id="Update playlist table charset" duration=15.276µs 2024-09-20T18:50:50.291 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.082531081Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-09-20T18:50:50.291 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.082545307Z level=info msg="Migration successfully executed" id="Update playlist_item table charset" duration=14.774µs 2024-09-20T18:50:50.291 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.083352012Z level=info msg="Executing migration" id="Add playlist column created_at" 2024-09-20T18:50:50.291 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.085157003Z level=info msg="Migration successfully executed" id="Add playlist column created_at" duration=1.804583ms 2024-09-20T18:50:50.291 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.08549836Z level=info msg="Executing migration" id="Add playlist column updated_at" 2024-09-20T18:50:50.291 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.087322647Z level=info msg="Migration successfully executed" id="Add playlist column updated_at" duration=1.824008ms 2024-09-20T18:50:50.291 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.087676162Z level=info msg="Executing migration" id="drop preferences table v2" 2024-09-20T18:50:50.291 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.087735367Z level=info msg="Migration successfully executed" id="drop preferences table v2" duration=60.68µs 2024-09-20T18:50:50.291 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.090474924Z level=info msg="Executing migration" id="drop preferences table v3" 2024-09-20T18:50:50.291 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.090535759Z level=info msg="Migration successfully executed" id="drop preferences table v3" duration=60.894µs 2024-09-20T18:50:50.291 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.09329988Z level=info msg="Executing migration" id="create preferences table v3" 2024-09-20T18:50:50.291 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.093809267Z level=info msg="Migration successfully executed" id="create preferences table v3" duration=525.698µs 2024-09-20T18:50:50.292 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.094168271Z level=info msg="Executing migration" id="Update preferences table charset" 2024-09-20T18:50:50.292 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.09418612Z level=info msg="Migration successfully executed" id="Update preferences table charset" duration=18.749µs 2024-09-20T18:50:50.292 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.096892863Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-09-20T18:50:50.292 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.09872647Z level=info msg="Migration successfully executed" id="Add column team_id in preferences" duration=1.833483ms 2024-09-20T18:50:50.292 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.099695453Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-09-20T18:50:50.292 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.099834529Z level=info msg="Migration successfully executed" id="Update team_id column values in preferences" duration=141.295µs 2024-09-20T18:50:50.292 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.102271288Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-09-20T18:50:50.292 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.104102437Z level=info msg="Migration successfully executed" id="Add column week_start in preferences" duration=1.830894ms 2024-09-20T18:50:50.292 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.104442941Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-09-20T18:50:50.292 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.106269619Z level=info msg="Migration successfully executed" id="Add column preferences.json_data" duration=1.826462ms 2024-09-20T18:50:50.292 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.107325275Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-09-20T18:50:50.292 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.10738321Z level=info msg="Migration successfully executed" id="alter preferences.json_data to mediumtext v1" duration=58.063µs 2024-09-20T18:50:50.292 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.110272559Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-09-20T18:50:50.292 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.110900419Z level=info msg="Migration successfully executed" id="Add preferences index org_id" duration=627.81µs 2024-09-20T18:50:50.293 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.112987819Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-09-20T18:50:50.293 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.11351072Z level=info msg="Migration successfully executed" id="Add preferences index user_id" duration=522.656µs 2024-09-20T18:50:50.293 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.113854629Z level=info msg="Executing migration" id="create alert table v1" 2024-09-20T18:50:50.293 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.114614334Z level=info msg="Migration successfully executed" id="create alert table v1" duration=759.612µs 2024-09-20T18:50:50.293 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.11674171Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-09-20T18:50:50.293 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.11735862Z level=info msg="Migration successfully executed" id="add index alert org_id & id " duration=617.367µs 2024-09-20T18:50:50.293 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.11953519Z level=info msg="Executing migration" id="add index alert state" 2024-09-20T18:50:50.293 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.120144756Z level=info msg="Migration successfully executed" id="add index alert state" duration=611.592µs 2024-09-20T18:50:50.293 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.121986545Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-09-20T18:50:50.293 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.122552705Z level=info msg="Migration successfully executed" id="add index alert dashboard_id" duration=565.835µs 2024-09-20T18:50:50.293 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.122896531Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-09-20T18:50:50.294 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.123353626Z level=info msg="Migration successfully executed" id="Create alert_rule_tag table v1" duration=456.943µs 2024-09-20T18:50:50.294 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.12586885Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-09-20T18:50:50.294 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.126571166Z level=info msg="Migration successfully executed" id="Add unique index alert_rule_tag.alert_id_tag_id" duration=706.454µs 2024-09-20T18:50:50.294 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.128404674Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-09-20T18:50:50.294 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.129043757Z level=info msg="Migration successfully executed" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" duration=638.927µs 2024-09-20T18:50:50.294 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.131223905Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-09-20T18:50:50.294 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.136756381Z level=info msg="Migration successfully executed" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" duration=5.532268ms 2024-09-20T18:50:50.294 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.137117326Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-09-20T18:50:50.294 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.13757177Z level=info msg="Migration successfully executed" id="Create alert_rule_tag table v2" duration=454.351µs 2024-09-20T18:50:50.294 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.140030602Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-09-20T18:50:50.294 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.140647802Z level=info msg="Migration successfully executed" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" duration=618.849µs 2024-09-20T18:50:50.294 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.142749948Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-09-20T18:50:50.294 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.142986135Z level=info msg="Migration successfully executed" id="copy alert_rule_tag v1 to v2" duration=238.377µs 2024-09-20T18:50:50.295 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.145294891Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-09-20T18:50:50.295 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.145686453Z level=info msg="Migration successfully executed" id="drop table alert_rule_tag_v1" duration=392.745µs 2024-09-20T18:50:50.295 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.146066253Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-09-20T18:50:50.295 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.146572055Z level=info msg="Migration successfully executed" id="create alert_notification table v1" duration=505.635µs 2024-09-20T18:50:50.295 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.148975918Z level=info msg="Executing migration" id="Add column is_default" 2024-09-20T18:50:50.295 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.151177845Z level=info msg="Migration successfully executed" id="Add column is_default" duration=2.204447ms 2024-09-20T18:50:50.295 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.151604762Z level=info msg="Executing migration" id="Add column frequency" 2024-09-20T18:50:50.295 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.153798805Z level=info msg="Migration successfully executed" id="Add column frequency" duration=2.193596ms 2024-09-20T18:50:50.295 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.154342608Z level=info msg="Executing migration" id="Add column send_reminder" 2024-09-20T18:50:50.295 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.156567819Z level=info msg="Migration successfully executed" id="Add column send_reminder" duration=2.225139ms 2024-09-20T18:50:50.295 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.156934442Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-09-20T18:50:50.295 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.158966397Z level=info msg="Migration successfully executed" id="Add column disable_resolve_message" duration=2.031784ms 2024-09-20T18:50:50.295 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.159663916Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-09-20T18:50:50.295 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.160227163Z level=info msg="Migration successfully executed" id="add index alert_notification org_id & name" duration=563.067µs 2024-09-20T18:50:50.296 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.162513754Z level=info msg="Executing migration" id="Update alert table charset" 2024-09-20T18:50:50.296 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.162552046Z level=info msg="Migration successfully executed" id="Update alert table charset" duration=39.976µs 2024-09-20T18:50:50.296 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.16524356Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-09-20T18:50:50.296 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.165282012Z level=info msg="Migration successfully executed" id="Update alert_notification table charset" duration=40.116µs 2024-09-20T18:50:50.296 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.165831392Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-09-20T18:50:50.296 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.166351696Z level=info msg="Migration successfully executed" id="create notification_journal table v1" duration=520.154µs 2024-09-20T18:50:50.296 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.168531456Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-09-20T18:50:50.296 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.169100273Z level=info msg="Migration successfully executed" id="add index notification_journal org_id & alert_id & notifier_id" duration=568.627µs 2024-09-20T18:50:50.296 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.171239078Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-09-20T18:50:50.296 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.171775209Z level=info msg="Migration successfully executed" id="drop alert_notification_journal" duration=535.902µs 2024-09-20T18:50:50.296 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.173886437Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-09-20T18:50:50.296 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.17443636Z level=info msg="Migration successfully executed" id="create alert_notification_state table v1" duration=549.909µs 2024-09-20T18:50:50.296 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.174816331Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-09-20T18:50:50.296 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.175394177Z level=info msg="Migration successfully executed" id="add index alert_notification_state org_id & alert_id & notifier_id" duration=577.638µs 2024-09-20T18:50:50.297 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.175756831Z level=info msg="Executing migration" id="Add for to alert table" 2024-09-20T18:50:50.297 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.177988038Z level=info msg="Migration successfully executed" id="Add for to alert table" duration=2.230797ms 2024-09-20T18:50:50.297 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.1783635Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-09-20T18:50:50.297 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.180667922Z level=info msg="Migration successfully executed" id="Add column uid in alert_notification" duration=2.303711ms 2024-09-20T18:50:50.297 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.181031329Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-09-20T18:50:50.297 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.181178602Z level=info msg="Migration successfully executed" id="Update uid column values in alert_notification" duration=147.044µs 2024-09-20T18:50:50.297 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.181611598Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-09-20T18:50:50.297 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.182153034Z level=info msg="Migration successfully executed" id="Add unique index alert_notification_org_id_uid" duration=541.348µs 2024-09-20T18:50:50.297 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.182547332Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-09-20T18:50:50.297 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.183089324Z level=info msg="Migration successfully executed" id="Remove unique index org_id_name" duration=541.96µs 2024-09-20T18:50:50.297 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.183471712Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-09-20T18:50:50.297 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.185892401Z level=info msg="Migration successfully executed" id="Add column secure_settings in alert_notification" duration=2.420165ms 2024-09-20T18:50:50.297 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.186279096Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-09-20T18:50:50.298 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.186323917Z level=info msg="Migration successfully executed" id="alter alert.settings to mediumtext" duration=44.481µs 2024-09-20T18:50:50.298 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.187010256Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-09-20T18:50:50.298 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.187553647Z level=info msg="Migration successfully executed" id="Add non-unique index alert_notification_state_alert_id" duration=543.211µs 2024-09-20T18:50:50.298 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.187913349Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-09-20T18:50:50.298 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.188530778Z level=info msg="Migration successfully executed" id="Add non-unique index alert_rule_tag_alert_id" duration=617.165µs 2024-09-20T18:50:50.298 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.188907344Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-09-20T18:50:50.298 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.188979062Z level=info msg="Migration successfully executed" id="Drop old annotation table v4" duration=71.209µs 2024-09-20T18:50:50.298 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.189670715Z level=info msg="Executing migration" id="create annotation table v5" 2024-09-20T18:50:50.298 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.190320014Z level=info msg="Migration successfully executed" id="create annotation table v5" duration=649.955µs 2024-09-20T18:50:50.298 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.190703006Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-09-20T18:50:50.298 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.191248327Z level=info msg="Migration successfully executed" id="add index annotation 0 v3" duration=545.012µs 2024-09-20T18:50:50.298 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.191612257Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-09-20T18:50:50.298 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.192172494Z level=info msg="Migration successfully executed" id="add index annotation 1 v3" duration=559.976µs 2024-09-20T18:50:50.298 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.192538845Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-09-20T18:50:50.299 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.193086289Z level=info msg="Migration successfully executed" id="add index annotation 2 v3" duration=547.162µs 2024-09-20T18:50:50.299 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.193457269Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-09-20T18:50:50.299 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.194064701Z level=info msg="Migration successfully executed" id="add index annotation 3 v3" duration=605.89µs 2024-09-20T18:50:50.299 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.194453446Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-09-20T18:50:50.299 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.195071124Z level=info msg="Migration successfully executed" id="add index annotation 4 v3" duration=617.513µs 2024-09-20T18:50:50.299 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.195444189Z level=info msg="Executing migration" id="Update annotation table charset" 2024-09-20T18:50:50.299 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.195459986Z level=info msg="Migration successfully executed" id="Update annotation table charset" duration=15.939µs 2024-09-20T18:50:50.299 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.196285791Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-09-20T18:50:50.299 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.198843637Z level=info msg="Migration successfully executed" id="Add column region_id to annotation table" duration=2.557255ms 2024-09-20T18:50:50.299 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.199216841Z level=info msg="Executing migration" id="Drop category_id index" 2024-09-20T18:50:50.299 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.19973971Z level=info msg="Migration successfully executed" id="Drop category_id index" duration=522.613µs 2024-09-20T18:50:50.299 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.20007652Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-09-20T18:50:50.299 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.202364618Z level=info msg="Migration successfully executed" id="Add column tags to annotation table" duration=2.287992ms 2024-09-20T18:50:50.300 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.20273231Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-09-20T18:50:50.300 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.203176025Z level=info msg="Migration successfully executed" id="Create annotation_tag table v2" duration=443.438µs 2024-09-20T18:50:50.300 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.203557945Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-09-20T18:50:50.300 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.204117285Z level=info msg="Migration successfully executed" id="Add unique index annotation_tag.annotation_id_tag_id" duration=558.991µs 2024-09-20T18:50:50.300 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.204525522Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-09-20T18:50:50.300 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.205045979Z level=info msg="Migration successfully executed" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" duration=520.299µs 2024-09-20T18:50:50.300 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.205397557Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-09-20T18:50:50.300 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.211973303Z level=info msg="Migration successfully executed" id="Rename table annotation_tag to annotation_tag_v2 - v2" duration=6.5723ms 2024-09-20T18:50:50.300 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.212511216Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-09-20T18:50:50.300 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.213007704Z level=info msg="Migration successfully executed" id="Create annotation_tag table v3" duration=497.763µs 2024-09-20T18:50:50.300 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.213377155Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-09-20T18:50:50.300 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.213930636Z level=info msg="Migration successfully executed" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" duration=553.248µs 2024-09-20T18:50:50.300 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.214292282Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-09-20T18:50:50.300 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.214483736Z level=info msg="Migration successfully executed" id="copy annotation_tag v2 to v3" duration=191.292µs 2024-09-20T18:50:50.301 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.214795361Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-09-20T18:50:50.301 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.215180266Z level=info msg="Migration successfully executed" id="drop table annotation_tag_v2" duration=384.555µs 2024-09-20T18:50:50.301 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.215643849Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-09-20T18:50:50.301 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.215753848Z level=info msg="Migration successfully executed" id="Update alert annotations and set TEXT to empty" duration=110.045µs 2024-09-20T18:50:50.301 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.216518225Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-09-20T18:50:50.301 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.218868475Z level=info msg="Migration successfully executed" id="Add created time to annotation table" duration=2.34826ms 2024-09-20T18:50:50.301 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.219341697Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-09-20T18:50:50.301 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.221839499Z level=info msg="Migration successfully executed" id="Add updated time to annotation table" duration=2.487942ms 2024-09-20T18:50:50.301 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.222351035Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-09-20T18:50:50.301 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.222972639Z level=info msg="Migration successfully executed" id="Add index for created in annotation table" duration=622.092µs 2024-09-20T18:50:50.301 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.223369816Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-09-20T18:50:50.301 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.223945173Z level=info msg="Migration successfully executed" id="Add index for updated in annotation table" duration=575.221µs 2024-09-20T18:50:50.301 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.224342009Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-09-20T18:50:50.301 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.224509914Z level=info msg="Migration successfully executed" id="Convert existing annotations from seconds to milliseconds" duration=168.197µs 2024-09-20T18:50:50.302 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.225166132Z level=info msg="Executing migration" id="Add epoch_end column" 2024-09-20T18:50:50.302 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.227651509Z level=info msg="Migration successfully executed" id="Add epoch_end column" duration=2.481957ms 2024-09-20T18:50:50.302 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.228101613Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-09-20T18:50:50.302 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.228705616Z level=info msg="Migration successfully executed" id="Add index for epoch_end" duration=604.095µs 2024-09-20T18:50:50.302 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.230783698Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-09-20T18:50:50.302 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.230942266Z level=info msg="Migration successfully executed" id="Make epoch_end the same as epoch" duration=159.966µs 2024-09-20T18:50:50.302 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.233535586Z level=info msg="Executing migration" id="Move region to single row" 2024-09-20T18:50:50.302 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.23382731Z level=info msg="Migration successfully executed" id="Move region to single row" duration=292.917µs 2024-09-20T18:50:50.302 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.236267945Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-09-20T18:50:50.302 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.23693609Z level=info msg="Migration successfully executed" id="Remove index org_id_epoch from annotation table" duration=668.559µs 2024-09-20T18:50:50.302 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.237324831Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-09-20T18:50:50.302 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.237868055Z level=info msg="Migration successfully executed" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" duration=543.139µs 2024-09-20T18:50:50.303 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.239998105Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-09-20T18:50:50.303 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.240629133Z level=info msg="Migration successfully executed" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" duration=630.828µs 2024-09-20T18:50:50.303 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.242782065Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-09-20T18:50:50.303 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.243407708Z level=info msg="Migration successfully executed" id="Add index for org_id_epoch_end_epoch on annotation table" duration=625.607µs 2024-09-20T18:50:50.303 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.245471601Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-09-20T18:50:50.303 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.246158524Z level=info msg="Migration successfully executed" id="Remove index org_id_epoch_epoch_end from annotation table" duration=687.611µs 2024-09-20T18:50:50.304 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.248213508Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-09-20T18:50:50.304 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.248813954Z level=info msg="Migration successfully executed" id="Add index for alert_id on annotation table" duration=600.996µs 2024-09-20T18:50:50.304 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.250810042Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-09-20T18:50:50.304 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.250852758Z level=info msg="Migration successfully executed" id="Increase tags column to length 4096" duration=43.388µs 2024-09-20T18:50:50.304 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.253664102Z level=info msg="Executing migration" id="create test_data table" 2024-09-20T18:50:50.304 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.254334195Z level=info msg="Migration successfully executed" id="create test_data table" duration=671.638µs 2024-09-20T18:50:50.304 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.256334917Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-09-20T18:50:50.304 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.256942964Z level=info msg="Migration successfully executed" id="create dashboard_version table v1" duration=610.812µs 2024-09-20T18:50:50.304 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.257325294Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-09-20T18:50:50.304 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.257859915Z level=info msg="Migration successfully executed" id="add index dashboard_version.dashboard_id" duration=534.488µs 2024-09-20T18:50:50.304 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.260104222Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-09-20T18:50:50.304 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.26070792Z level=info msg="Migration successfully executed" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" duration=603.33µs 2024-09-20T18:50:50.304 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.262848373Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-09-20T18:50:50.304 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.263003526Z level=info msg="Migration successfully executed" id="Set dashboard version to 1 where 0" duration=154.525µs 2024-09-20T18:50:50.305 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.265482813Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-09-20T18:50:50.305 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.265816104Z level=info msg="Migration successfully executed" id="save existing dashboard data in dashboard_version table v1" duration=333.154µs 2024-09-20T18:50:50.305 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.266279739Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-09-20T18:50:50.305 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.266340389Z level=info msg="Migration successfully executed" id="alter dashboard_version.data to mediumtext v1" duration=61.425µs 2024-09-20T18:50:50.305 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.268904437Z level=info msg="Executing migration" id="create team table" 2024-09-20T18:50:50.305 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.269654233Z level=info msg="Migration successfully executed" id="create team table" duration=749.382µs 2024-09-20T18:50:50.314 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:50:50.335 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:50.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:50.336 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:50:50.407 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd pool application enable unique_pool_0 rados --yes-i-really-mean-it || true 2024-09-20T18:50:50.437 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:50:50.458 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:50.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:50.459 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:50:50.530 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.271575193Z level=info msg="Executing migration" id="add index team.org_id" 2024-09-20T18:50:50.530 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.272495098Z level=info msg="Migration successfully executed" id="add index team.org_id" duration=919.494µs 2024-09-20T18:50:50.530 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.274355161Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-09-20T18:50:50.530 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.275021029Z level=info msg="Migration successfully executed" id="add unique index team_org_id_name" duration=666.174µs 2024-09-20T18:50:50.531 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.275379213Z level=info msg="Executing migration" id="Add column uid in team" 2024-09-20T18:50:50.531 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.277907551Z level=info msg="Migration successfully executed" id="Add column uid in team" duration=2.528331ms 2024-09-20T18:50:50.531 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.278283605Z level=info msg="Executing migration" id="Update uid column values in team" 2024-09-20T18:50:50.531 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.278392523Z level=info msg="Migration successfully executed" id="Update uid column values in team" duration=108.944µs 2024-09-20T18:50:50.531 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.279133461Z level=info msg="Executing migration" id="Add unique index team_org_id_uid" 2024-09-20T18:50:50.531 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.279679062Z level=info msg="Migration successfully executed" id="Add unique index team_org_id_uid" duration=543.841µs 2024-09-20T18:50:50.531 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.280021556Z level=info msg="Executing migration" id="create team member table" 2024-09-20T18:50:50.531 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.280514057Z level=info msg="Migration successfully executed" id="create team member table" duration=492.137µs 2024-09-20T18:50:50.531 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.280851121Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-09-20T18:50:50.531 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.281386806Z level=info msg="Migration successfully executed" id="add index team_member.org_id" duration=536.702µs 2024-09-20T18:50:50.531 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.281743962Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-09-20T18:50:50.531 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.282284923Z level=info msg="Migration successfully executed" id="add unique index team_member_org_id_team_id_user_id" duration=540.762µs 2024-09-20T18:50:50.531 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.282625368Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-09-20T18:50:50.531 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.283155333Z level=info msg="Migration successfully executed" id="add index team_member.team_id" duration=529.611µs 2024-09-20T18:50:50.533 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.285416931Z level=info msg="Executing migration" id="Add column email to team table" 2024-09-20T18:50:50.533 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.288176294Z level=info msg="Migration successfully executed" id="Add column email to team table" duration=2.759092ms 2024-09-20T18:50:50.533 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.28851234Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-09-20T18:50:50.533 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.291321257Z level=info msg="Migration successfully executed" id="Add column external to team_member table" duration=2.808689ms 2024-09-20T18:50:50.533 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.291694092Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-09-20T18:50:50.533 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.295709841Z level=info msg="Migration successfully executed" id="Add column permission to team_member table" duration=4.014959ms 2024-09-20T18:50:50.533 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.296162603Z level=info msg="Executing migration" id="create dashboard acl table" 2024-09-20T18:50:50.533 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.29706636Z level=info msg="Migration successfully executed" id="create dashboard acl table" duration=903.474µs 2024-09-20T18:50:50.533 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.297520859Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-09-20T18:50:50.533 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.29835995Z level=info msg="Migration successfully executed" id="add index dashboard_acl_dashboard_id" duration=838.509µs 2024-09-20T18:50:50.533 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.299028608Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-09-20T18:50:50.534 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.299937894Z level=info msg="Migration successfully executed" id="add unique index dashboard_acl_dashboard_id_user_id" duration=908.923µs 2024-09-20T18:50:50.534 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.301788542Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-09-20T18:50:50.534 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.302648603Z level=info msg="Migration successfully executed" id="add unique index dashboard_acl_dashboard_id_team_id" duration=859.672µs 2024-09-20T18:50:50.534 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.304509188Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-09-20T18:50:50.534 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.305085258Z level=info msg="Migration successfully executed" id="add index dashboard_acl_user_id" duration=575.832µs 2024-09-20T18:50:50.534 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.307477722Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-09-20T18:50:50.534 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.308018262Z level=info msg="Migration successfully executed" id="add index dashboard_acl_team_id" duration=540.35µs 2024-09-20T18:50:50.534 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.310191963Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-09-20T18:50:50.534 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.310771273Z level=info msg="Migration successfully executed" id="add index dashboard_acl_org_id_role" duration=579.147µs 2024-09-20T18:50:50.534 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.312962804Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-09-20T18:50:50.534 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.313567496Z level=info msg="Migration successfully executed" id="add index dashboard_permission" duration=604.492µs 2024-09-20T18:50:50.534 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.31574558Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-09-20T18:50:50.534 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.316177733Z level=info msg="Migration successfully executed" id="save default acl rules in dashboard_acl table" duration=432.049µs 2024-09-20T18:50:50.535 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.31846792Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-09-20T18:50:50.535 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.318623279Z level=info msg="Migration successfully executed" id="delete acl rules for deleted dashboards and folders" duration=155.419µs 2024-09-20T18:50:50.535 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.321290314Z level=info msg="Executing migration" id="create tag table" 2024-09-20T18:50:50.535 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.321858425Z level=info msg="Migration successfully executed" id="create tag table" duration=566.241µs 2024-09-20T18:50:50.535 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.324161014Z level=info msg="Executing migration" id="add index tag.key_value" 2024-09-20T18:50:50.535 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.324746152Z level=info msg="Migration successfully executed" id="add index tag.key_value" duration=584.904µs 2024-09-20T18:50:50.535 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.326877725Z level=info msg="Executing migration" id="create login attempt table" 2024-09-20T18:50:50.535 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.327377828Z level=info msg="Migration successfully executed" id="create login attempt table" duration=501.366µs 2024-09-20T18:50:50.535 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.329783822Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-09-20T18:50:50.535 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.330498536Z level=info msg="Migration successfully executed" id="add index login_attempt.username" duration=716.623µs 2024-09-20T18:50:50.535 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.332528721Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-09-20T18:50:50.535 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.333120405Z level=info msg="Migration successfully executed" id="drop index IDX_login_attempt_username - v1" duration=592.501µs 2024-09-20T18:50:50.535 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.335383206Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-09-20T18:50:50.535 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.343448967Z level=info msg="Migration successfully executed" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" duration=8.065366ms 2024-09-20T18:50:50.536 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.343811445Z level=info msg="Executing migration" id="create login_attempt v2" 2024-09-20T18:50:50.536 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.344285208Z level=info msg="Migration successfully executed" id="create login_attempt v2" duration=473.245µs 2024-09-20T18:50:50.536 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.346732439Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-09-20T18:50:50.536 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.347441112Z level=info msg="Migration successfully executed" id="create index IDX_login_attempt_username - v2" duration=710.461µs 2024-09-20T18:50:50.536 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.349237156Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-09-20T18:50:50.536 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.349438622Z level=info msg="Migration successfully executed" id="copy login_attempt v1 to v2" duration=201.429µs 2024-09-20T18:50:50.536 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.351702913Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-09-20T18:50:50.536 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.35213083Z level=info msg="Migration successfully executed" id="drop login_attempt_tmp_qwerty" duration=427.682µs 2024-09-20T18:50:50.536 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.354415776Z level=info msg="Executing migration" id="create user auth table" 2024-09-20T18:50:50.536 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.354918434Z level=info msg="Migration successfully executed" id="create user auth table" duration=484.654µs 2024-09-20T18:50:50.536 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.356902533Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-09-20T18:50:50.536 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.357482739Z level=info msg="Migration successfully executed" id="create index IDX_user_auth_auth_module_auth_id - v1" duration=579.983µs 2024-09-20T18:50:50.536 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.359360855Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-09-20T18:50:50.537 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.359403239Z level=info msg="Migration successfully executed" id="alter user_auth.auth_id to length 190" duration=42.84µs 2024-09-20T18:50:50.537 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.361892841Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-09-20T18:50:50.537 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.364945458Z level=info msg="Migration successfully executed" id="Add OAuth access token to user_auth" duration=3.052768ms 2024-09-20T18:50:50.537 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.365347673Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-09-20T18:50:50.537 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.368320776Z level=info msg="Migration successfully executed" id="Add OAuth refresh token to user_auth" duration=2.974238ms 2024-09-20T18:50:50.537 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.36868146Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-09-20T18:50:50.537 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.371526383Z level=info msg="Migration successfully executed" id="Add OAuth token type to user_auth" duration=2.8446ms 2024-09-20T18:50:50.537 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.371892094Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-09-20T18:50:50.537 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.374867974Z level=info msg="Migration successfully executed" id="Add OAuth expiry to user_auth" duration=2.97391ms 2024-09-20T18:50:50.537 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.375247912Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-09-20T18:50:50.537 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.375818791Z level=info msg="Migration successfully executed" id="Add index to user_id column in user_auth" duration=570.715µs 2024-09-20T18:50:50.537 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.376179504Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-09-20T18:50:50.537 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.379126375Z level=info msg="Migration successfully executed" id="Add OAuth ID token to user_auth" duration=2.946913ms 2024-09-20T18:50:50.537 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.379488517Z level=info msg="Executing migration" id="create server_lock table" 2024-09-20T18:50:50.537 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.380135267Z level=info msg="Migration successfully executed" id="create server_lock table" duration=646.551µs 2024-09-20T18:50:50.538 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.38050484Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-09-20T18:50:50.538 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.381086341Z level=info msg="Migration successfully executed" id="add index server_lock.operation_uid" duration=581.47µs 2024-09-20T18:50:50.538 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.381452881Z level=info msg="Executing migration" id="create user auth token table" 2024-09-20T18:50:50.538 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.382026221Z level=info msg="Migration successfully executed" id="create user auth token table" duration=572.287µs 2024-09-20T18:50:50.538 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.382399694Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-09-20T18:50:50.538 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.382977015Z level=info msg="Migration successfully executed" id="add unique index user_auth_token.auth_token" duration=577.254µs 2024-09-20T18:50:50.538 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.383362249Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-09-20T18:50:50.538 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.383946787Z level=info msg="Migration successfully executed" id="add unique index user_auth_token.prev_auth_token" duration=584.481µs 2024-09-20T18:50:50.538 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.384338473Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-09-20T18:50:50.538 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.384978841Z level=info msg="Migration successfully executed" id="add index user_auth_token.user_id" duration=639.045µs 2024-09-20T18:50:50.538 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.385375237Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-09-20T18:50:50.538 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.388543955Z level=info msg="Migration successfully executed" id="Add revoked_at to the user auth token" duration=3.168387ms 2024-09-20T18:50:50.538 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.388944844Z level=info msg="Executing migration" id="add index user_auth_token.revoked_at" 2024-09-20T18:50:50.539 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.389578231Z level=info msg="Migration successfully executed" id="add index user_auth_token.revoked_at" duration=633.164µs 2024-09-20T18:50:50.539 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.389940116Z level=info msg="Executing migration" id="create cache_data table" 2024-09-20T18:50:50.539 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.390516417Z level=info msg="Migration successfully executed" id="create cache_data table" duration=576.193µs 2024-09-20T18:50:50.539 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.390889757Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-09-20T18:50:50.539 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.391488048Z level=info msg="Migration successfully executed" id="add unique index cache_data.cache_key" duration=598.142µs 2024-09-20T18:50:50.539 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.391860353Z level=info msg="Executing migration" id="create short_url table v1" 2024-09-20T18:50:50.539 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.392472528Z level=info msg="Migration successfully executed" id="create short_url table v1" duration=611.906µs 2024-09-20T18:50:50.539 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.392885136Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-09-20T18:50:50.539 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.393548865Z level=info msg="Migration successfully executed" id="add index short_url.org_id-uid" duration=662.985µs 2024-09-20T18:50:50.539 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.393903727Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-09-20T18:50:50.539 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.393942937Z level=info msg="Migration successfully executed" id="alter table short_url alter column created_by type to bigint" duration=39.53µs 2024-09-20T18:50:50.539 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.394356607Z level=info msg="Executing migration" id="delete alert_definition table" 2024-09-20T18:50:50.539 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.394440674Z level=info msg="Migration successfully executed" id="delete alert_definition table" duration=84.631µs 2024-09-20T18:50:50.539 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.394812329Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-09-20T18:50:50.540 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.395390879Z level=info msg="Migration successfully executed" id="recreate alert_definition table" duration=578.764µs 2024-09-20T18:50:50.540 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.395752261Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-09-20T18:50:50.540 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.396379723Z level=info msg="Migration successfully executed" id="add index in alert_definition on org_id and title columns" duration=627.337µs 2024-09-20T18:50:50.540 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.39674831Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-09-20T18:50:50.540 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.397389189Z level=info msg="Migration successfully executed" id="add index in alert_definition on org_id and uid columns" duration=640.644µs 2024-09-20T18:50:50.540 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.397722593Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-09-20T18:50:50.540 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.397760736Z level=info msg="Migration successfully executed" id="alter alert_definition table data column to mediumtext in mysql" duration=37.421µs 2024-09-20T18:50:50.540 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.398148262Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-09-20T18:50:50.540 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.398737653Z level=info msg="Migration successfully executed" id="drop index in alert_definition on org_id and title columns" duration=589.65µs 2024-09-20T18:50:50.540 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.399088553Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-09-20T18:50:50.540 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.399666726Z level=info msg="Migration successfully executed" id="drop index in alert_definition on org_id and uid columns" duration=577.993µs 2024-09-20T18:50:50.540 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.400002231Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-09-20T18:50:50.540 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.400624811Z level=info msg="Migration successfully executed" id="add unique index in alert_definition on org_id and title columns" duration=622.745µs 2024-09-20T18:50:50.540 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.402506931Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-09-20T18:50:50.541 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.403133526Z level=info msg="Migration successfully executed" id="add unique index in alert_definition on org_id and uid columns" duration=626.743µs 2024-09-20T18:50:50.541 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.405167672Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-09-20T18:50:50.541 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.408586647Z level=info msg="Migration successfully executed" id="Add column paused in alert_definition" duration=3.422752ms 2024-09-20T18:50:50.541 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.408942829Z level=info msg="Executing migration" id="drop alert_definition table" 2024-09-20T18:50:50.541 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.409617384Z level=info msg="Migration successfully executed" id="drop alert_definition table" duration=674.099µs 2024-09-20T18:50:50.541 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.40997687Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-09-20T18:50:50.541 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.410030695Z level=info msg="Migration successfully executed" id="delete alert_definition_version table" duration=53.908µs 2024-09-20T18:50:50.541 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.412560812Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-09-20T18:50:50.541 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.413179306Z level=info msg="Migration successfully executed" id="recreate alert_definition_version table" duration=618.539µs 2024-09-20T18:50:50.541 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.413549379Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-09-20T18:50:50.541 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.414178298Z level=info msg="Migration successfully executed" id="add index in alert_definition_version table on alert_definition_id and version columns" duration=629.15µs 2024-09-20T18:50:50.541 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.416321456Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-09-20T18:50:50.541 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.416998042Z level=info msg="Migration successfully executed" id="add index in alert_definition_version table on alert_definition_uid and version columns" duration=678.22µs 2024-09-20T18:50:50.541 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.418766166Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-09-20T18:50:50.542 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.418826326Z level=info msg="Migration successfully executed" id="alter alert_definition_version table data column to mediumtext in mysql" duration=60.37µs 2024-09-20T18:50:50.542 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.421268591Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-09-20T18:50:50.542 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.421883881Z level=info msg="Migration successfully executed" id="drop alert_definition_version table" duration=614.894µs 2024-09-20T18:50:50.542 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.422290875Z level=info msg="Executing migration" id="create alert_instance table" 2024-09-20T18:50:50.543 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.422986935Z level=info msg="Migration successfully executed" id="create alert_instance table" duration=696.059µs 2024-09-20T18:50:50.543 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.424855865Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-09-20T18:50:50.543 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.425490564Z level=info msg="Migration successfully executed" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" duration=640.133µs 2024-09-20T18:50:50.543 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.425826621Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-09-20T18:50:50.543 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.426446245Z level=info msg="Migration successfully executed" id="add index in alert_instance table on def_org_id, current_state columns" duration=619.618µs 2024-09-20T18:50:50.543 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.426774286Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-09-20T18:50:50.543 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.430203258Z level=info msg="Migration successfully executed" id="add column current_state_end to alert_instance" duration=3.428356ms 2024-09-20T18:50:50.543 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.430547345Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-09-20T18:50:50.543 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.431186473Z level=info msg="Migration successfully executed" id="remove index def_org_id, def_uid, current_state on alert_instance" duration=638.869µs 2024-09-20T18:50:50.543 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.431618435Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-09-20T18:50:50.543 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.432218917Z level=info msg="Migration successfully executed" id="remove index def_org_id, current_state on alert_instance" duration=600.438µs 2024-09-20T18:50:50.543 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.432626783Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-09-20T18:50:50.543 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.451259747Z level=info msg="Migration successfully executed" id="rename def_org_id to rule_org_id in alert_instance" duration=18.629948ms 2024-09-20T18:50:50.543 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.451734257Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-09-20T18:50:50.544 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.468100587Z level=info msg="Migration successfully executed" id="rename def_uid to rule_uid in alert_instance" duration=16.358412ms 2024-09-20T18:50:50.544 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.468454046Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-09-20T18:50:50.544 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.469081937Z level=info msg="Migration successfully executed" id="add index rule_org_id, rule_uid, current_state on alert_instance" duration=627.686µs 2024-09-20T18:50:50.544 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.469425939Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-09-20T18:50:50.544 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.47002376Z level=info msg="Migration successfully executed" id="add index rule_org_id, current_state on alert_instance" duration=597.526µs 2024-09-20T18:50:50.544 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.470382975Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-09-20T18:50:50.544 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.473613649Z level=info msg="Migration successfully executed" id="add current_reason column related to current_state" duration=3.230871ms 2024-09-20T18:50:50.544 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.473992641Z level=info msg="Executing migration" id="add result_fingerprint column to alert_instance" 2024-09-20T18:50:50.544 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.477212994Z level=info msg="Migration successfully executed" id="add result_fingerprint column to alert_instance" duration=3.222173ms 2024-09-20T18:50:50.544 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.477612842Z level=info msg="Executing migration" id="create alert_rule table" 2024-09-20T18:50:50.544 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.47823927Z level=info msg="Migration successfully executed" id="create alert_rule table" duration=626.169µs 2024-09-20T18:50:50.544 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.478591321Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-09-20T18:50:50.544 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.479225302Z level=info msg="Migration successfully executed" id="add index in alert_rule on org_id and title columns" duration=633.847µs 2024-09-20T18:50:50.544 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.479570546Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-09-20T18:50:50.545 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.480199756Z level=info msg="Migration successfully executed" id="add index in alert_rule on org_id and uid columns" duration=629.314µs 2024-09-20T18:50:50.545 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.480546109Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-09-20T18:50:50.545 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.481235636Z level=info msg="Migration successfully executed" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" duration=689.16µs 2024-09-20T18:50:50.545 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.48158656Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-09-20T18:50:50.545 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.481626357Z level=info msg="Migration successfully executed" id="alter alert_rule table data column to mediumtext in mysql" duration=40.21µs 2024-09-20T18:50:50.545 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.482477675Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-09-20T18:50:50.545 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.485923189Z level=info msg="Migration successfully executed" id="add column for to alert_rule" duration=3.445554ms 2024-09-20T18:50:50.545 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.486309259Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-09-20T18:50:50.545 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.489645525Z level=info msg="Migration successfully executed" id="add column annotations to alert_rule" duration=3.336537ms 2024-09-20T18:50:50.545 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.490005593Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-09-20T18:50:50.545 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.493357944Z level=info msg="Migration successfully executed" id="add column labels to alert_rule" duration=3.352186ms 2024-09-20T18:50:50.545 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.493739243Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-09-20T18:50:50.545 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.494346687Z level=info msg="Migration successfully executed" id="remove unique index from alert_rule on org_id, title columns" duration=607.343µs 2024-09-20T18:50:50.546 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.494709042Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-09-20T18:50:50.546 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.495338705Z level=info msg="Migration successfully executed" id="add index in alert_rule on org_id, namespase_uid and title columns" duration=628.491µs 2024-09-20T18:50:50.546 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.49567156Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-09-20T18:50:50.546 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.499059899Z level=info msg="Migration successfully executed" id="add dashboard_uid column to alert_rule" duration=3.387612ms 2024-09-20T18:50:50.546 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.499503967Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-09-20T18:50:50.546 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.502875398Z level=info msg="Migration successfully executed" id="add panel_id column to alert_rule" duration=3.369535ms 2024-09-20T18:50:50.546 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.503256997Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-09-20T18:50:50.546 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.503867856Z level=info msg="Migration successfully executed" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" duration=610.552µs 2024-09-20T18:50:50.546 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.504229291Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-09-20T18:50:50.546 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.50760712Z level=info msg="Migration successfully executed" id="add rule_group_idx column to alert_rule" duration=3.377553ms 2024-09-20T18:50:50.546 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.507970558Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-09-20T18:50:50.546 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.511374774Z level=info msg="Migration successfully executed" id="add is_paused column to alert_rule table" duration=3.403794ms 2024-09-20T18:50:50.546 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.511738732Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-09-20T18:50:50.546 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.511785501Z level=info msg="Migration successfully executed" id="fix is_paused column for alert_rule table" duration=46.802µs 2024-09-20T18:50:50.546 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.512651643Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-09-20T18:50:50.547 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.513421458Z level=info msg="Migration successfully executed" id="create alert_rule_version table" duration=769.479µs 2024-09-20T18:50:50.547 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.513828246Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-09-20T18:50:50.547 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.514515915Z level=info msg="Migration successfully executed" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" duration=687.382µs 2024-09-20T18:50:50.547 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.514878646Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-09-20T18:50:50.547 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.515617835Z level=info msg="Migration successfully executed" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" duration=738.844µs 2024-09-20T18:50:50.547 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.515982615Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-09-20T18:50:50.547 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.516041941Z level=info msg="Migration successfully executed" id="alter alert_rule_version table data column to mediumtext in mysql" duration=58.904µs 2024-09-20T18:50:50.547 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.516830209Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-09-20T18:50:50.561 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:50:50.583 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:50.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:50.584 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:50:50.685 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:50:50.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:50.707 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:50:50.707 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:50:50.782 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.520615383Z level=info msg="Migration successfully executed" id="add column for to alert_rule_version" duration=3.782383ms 2024-09-20T18:50:50.783 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.520994967Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-09-20T18:50:50.783 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.524671699Z level=info msg="Migration successfully executed" id="add column annotations to alert_rule_version" duration=3.676471ms 2024-09-20T18:50:50.783 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.525051108Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-09-20T18:50:50.783 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.528607958Z level=info msg="Migration successfully executed" id="add column labels to alert_rule_version" duration=3.556655ms 2024-09-20T18:50:50.783 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.528980784Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-09-20T18:50:50.783 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.532505424Z level=info msg="Migration successfully executed" id="add rule_group_idx column to alert_rule_version" duration=3.524406ms 2024-09-20T18:50:50.783 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.532881344Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-09-20T18:50:50.783 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.536536592Z level=info msg="Migration successfully executed" id="add is_paused column to alert_rule_versions table" duration=3.634727ms 2024-09-20T18:50:50.783 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.536972025Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-09-20T18:50:50.783 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.537025069Z level=info msg="Migration successfully executed" id="fix is_paused column for alert_rule_version table" duration=58.617µs 2024-09-20T18:50:50.783 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.53783724Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-09-20T18:50:50.784 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.538373031Z level=info msg="Migration successfully executed" id=create_alert_configuration_table duration=535.635µs 2024-09-20T18:50:50.784 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.538757643Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-09-20T18:50:50.784 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.542379241Z level=info msg="Migration successfully executed" id="Add column default in alert_configuration" duration=3.619841ms 2024-09-20T18:50:50.784 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.542752866Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-09-20T18:50:50.784 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.54279763Z level=info msg="Migration successfully executed" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" duration=44.509µs 2024-09-20T18:50:50.784 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.545521417Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-09-20T18:50:50.784 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.549445074Z level=info msg="Migration successfully executed" id="add column org_id in alert_configuration" duration=3.920782ms 2024-09-20T18:50:50.784 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.549920263Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-09-20T18:50:50.784 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.550555916Z level=info msg="Migration successfully executed" id="add index in alert_configuration table on org_id column" duration=635.58µs 2024-09-20T18:50:50.784 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.55263162Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-09-20T18:50:50.784 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.556491848Z level=info msg="Migration successfully executed" id="add configuration_hash column to alert_configuration" duration=3.859479ms 2024-09-20T18:50:50.784 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.556896907Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-09-20T18:50:50.784 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.557460328Z level=info msg="Migration successfully executed" id=create_ngalert_configuration_table duration=563.183µs 2024-09-20T18:50:50.785 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.559633378Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-09-20T18:50:50.785 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.560292583Z level=info msg="Migration successfully executed" id="add index in ngalert_configuration on org_id column" duration=659.194µs 2024-09-20T18:50:50.785 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.562423106Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-09-20T18:50:50.785 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.56648741Z level=info msg="Migration successfully executed" id="add column send_alerts_to in ngalert_configuration" duration=4.061876ms 2024-09-20T18:50:50.785 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.568017356Z level=info msg="Executing migration" id="create provenance_type table" 2024-09-20T18:50:50.785 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.568695181Z level=info msg="Migration successfully executed" id="create provenance_type table" duration=677.183µs 2024-09-20T18:50:50.785 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.569120689Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-09-20T18:50:50.785 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.569804235Z level=info msg="Migration successfully executed" id="add index to uniquify (record_key, record_type, org_id) columns" duration=683.448µs 2024-09-20T18:50:50.785 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.571896037Z level=info msg="Executing migration" id="create alert_image table" 2024-09-20T18:50:50.786 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.572581009Z level=info msg="Migration successfully executed" id="create alert_image table" duration=685.117µs 2024-09-20T18:50:50.786 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.573000364Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-09-20T18:50:50.786 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.573632847Z level=info msg="Migration successfully executed" id="add unique index on token to alert_image table" duration=632.106µs 2024-09-20T18:50:50.786 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.575786052Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-09-20T18:50:50.787 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.575849751Z level=info msg="Migration successfully executed" id="support longer URLs in alert_image table" duration=64.343µs 2024-09-20T18:50:50.787 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.581959902Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-09-20T18:50:50.787 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.582814001Z level=info msg="Migration successfully executed" id=create_alert_configuration_history_table duration=856.264µs 2024-09-20T18:50:50.787 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.583196387Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-09-20T18:50:50.787 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.583863212Z level=info msg="Migration successfully executed" id="drop non-unique orgID index on alert_configuration" duration=667.059µs 2024-09-20T18:50:50.787 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.584233608Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-09-20T18:50:50.787 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.584480179Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-09-20T18:50:50.787 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.584881501Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-09-20T18:50:50.787 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.585192099Z level=info msg="Migration successfully executed" id="extract alertmanager configuration history to separate table" duration=310.375µs 2024-09-20T18:50:50.787 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.585675822Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-09-20T18:50:50.787 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.586329758Z level=info msg="Migration successfully executed" id="add unique index on orgID to alert_configuration" duration=653.638µs 2024-09-20T18:50:50.787 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.586707752Z level=info msg="Executing migration" id="add last_applied column to alert_configuration_history" 2024-09-20T18:50:50.788 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.590527911Z level=info msg="Migration successfully executed" id="add last_applied column to alert_configuration_history" duration=3.819912ms 2024-09-20T18:50:50.788 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.590922212Z level=info msg="Executing migration" id="create library_element table v1" 2024-09-20T18:50:50.788 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.5916398Z level=info msg="Migration successfully executed" id="create library_element table v1" duration=717.304µs 2024-09-20T18:50:50.788 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.592048299Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-09-20T18:50:50.788 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.592729471Z level=info msg="Migration successfully executed" id="add index library_element org_id-folder_id-name-kind" duration=680.773µs 2024-09-20T18:50:50.788 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.593141942Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-09-20T18:50:50.788 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.593716299Z level=info msg="Migration successfully executed" id="create library_element_connection table v1" duration=573.082µs 2024-09-20T18:50:50.788 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.594093093Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-09-20T18:50:50.788 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.594814693Z level=info msg="Migration successfully executed" id="add index library_element_connection element_id-kind-connection_id" duration=721.213µs 2024-09-20T18:50:50.788 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.59517176Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-09-20T18:50:50.788 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.595800575Z level=info msg="Migration successfully executed" id="add unique index library_element org_id_uid" duration=627.109µs 2024-09-20T18:50:50.789 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.596166069Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-09-20T18:50:50.789 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.596184619Z level=info msg="Migration successfully executed" id="increase max description length to 2048" duration=18.594µs 2024-09-20T18:50:50.789 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.597054026Z level=info msg="Executing migration" id="alter library_element model to mediumtext" 2024-09-20T18:50:50.789 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.597102762Z level=info msg="Migration successfully executed" id="alter library_element model to mediumtext" duration=48.837µs 2024-09-20T18:50:50.789 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.597894817Z level=info msg="Executing migration" id="add library_element folder uid" 2024-09-20T18:50:50.789 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.602011724Z level=info msg="Migration successfully executed" id="add library_element folder uid" duration=4.118658ms 2024-09-20T18:50:50.789 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.602392704Z level=info msg="Executing migration" id="populate library_element folder_uid" 2024-09-20T18:50:50.789 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.602643286Z level=info msg="Migration successfully executed" id="populate library_element folder_uid" duration=250.42µs 2024-09-20T18:50:50.789 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.603234038Z level=info msg="Executing migration" id="add index library_element org_id-folder_uid-name-kind" 2024-09-20T18:50:50.789 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.603906002Z level=info msg="Migration successfully executed" id="add index library_element org_id-folder_uid-name-kind" duration=671.627µs 2024-09-20T18:50:50.790 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.604265041Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-09-20T18:50:50.790 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.604443547Z level=info msg="Migration successfully executed" id="clone move dashboard alerts to unified alerting" duration=178.326µs 2024-09-20T18:50:50.790 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.604834582Z level=info msg="Executing migration" id="create data_keys table" 2024-09-20T18:50:50.790 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.605509504Z level=info msg="Migration successfully executed" id="create data_keys table" duration=674.497µs 2024-09-20T18:50:50.790 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.605887441Z level=info msg="Executing migration" id="create secrets table" 2024-09-20T18:50:50.790 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.606448251Z level=info msg="Migration successfully executed" id="create secrets table" duration=560.615µs 2024-09-20T18:50:50.790 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.606792543Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-09-20T18:50:50.790 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.629154106Z level=info msg="Migration successfully executed" id="rename data_keys name column to id" duration=22.361255ms 2024-09-20T18:50:50.790 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.629515891Z level=info msg="Executing migration" id="add name column into data_keys" 2024-09-20T18:50:50.790 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.63364158Z level=info msg="Migration successfully executed" id="add name column into data_keys" duration=4.125219ms 2024-09-20T18:50:50.791 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.634031274Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-09-20T18:50:50.791 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.634142091Z level=info msg="Migration successfully executed" id="copy data_keys id column values into name" duration=110.872µs 2024-09-20T18:50:50.791 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.636846527Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-09-20T18:50:50.791 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.65760755Z level=info msg="Migration successfully executed" id="rename data_keys name column to label" duration=20.760831ms 2024-09-20T18:50:50.791 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.65800645Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-09-20T18:50:50.791 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.679427481Z level=info msg="Migration successfully executed" id="rename data_keys id column back to name" duration=21.420448ms 2024-09-20T18:50:50.791 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.679840101Z level=info msg="Executing migration" id="create kv_store table v1" 2024-09-20T18:50:50.791 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.680412367Z level=info msg="Migration successfully executed" id="create kv_store table v1" duration=572.226µs 2024-09-20T18:50:50.791 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.68080122Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-09-20T18:50:50.791 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.681503714Z level=info msg="Migration successfully executed" id="add index kv_store.org_id-namespace-key" duration=702.142µs 2024-09-20T18:50:50.791 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.683575069Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-09-20T18:50:50.791 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.683708227Z level=info msg="Migration successfully executed" id="update dashboard_uid and panel_id from existing annotations" duration=132.477µs 2024-09-20T18:50:50.791 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.68617993Z level=info msg="Executing migration" id="create permission table" 2024-09-20T18:50:50.791 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.686732785Z level=info msg="Migration successfully executed" id="create permission table" duration=552.587µs 2024-09-20T18:50:50.792 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.689079286Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-09-20T18:50:50.792 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.689707144Z level=info msg="Migration successfully executed" id="add unique index permission.role_id" duration=628.24µs 2024-09-20T18:50:50.792 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.690150364Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-09-20T18:50:50.792 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.69080495Z level=info msg="Migration successfully executed" id="add unique index role_id_action_scope" duration=654.359µs 2024-09-20T18:50:50.792 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.692944298Z level=info msg="Executing migration" id="create role table" 2024-09-20T18:50:50.792 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.69353868Z level=info msg="Migration successfully executed" id="create role table" duration=594.142µs 2024-09-20T18:50:50.792 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.695826837Z level=info msg="Executing migration" id="add column display_name" 2024-09-20T18:50:50.792 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.700358843Z level=info msg="Migration successfully executed" id="add column display_name" duration=4.534048ms 2024-09-20T18:50:50.792 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.700742404Z level=info msg="Executing migration" id="add column group_name" 2024-09-20T18:50:50.792 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.705184672Z level=info msg="Migration successfully executed" id="add column group_name" duration=4.442111ms 2024-09-20T18:50:50.792 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.705570891Z level=info msg="Executing migration" id="add index role.org_id" 2024-09-20T18:50:50.793 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.706212039Z level=info msg="Migration successfully executed" id="add index role.org_id" duration=639.792µs 2024-09-20T18:50:50.793 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.708345421Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-09-20T18:50:50.793 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.709072407Z level=info msg="Migration successfully executed" id="add unique index role_org_id_name" duration=726.967µs 2024-09-20T18:50:50.793 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.711199988Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-09-20T18:50:50.793 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.711893826Z level=info msg="Migration successfully executed" id="add index role_org_id_uid" duration=693.682µs 2024-09-20T18:50:50.793 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.713843418Z level=info msg="Executing migration" id="create team role table" 2024-09-20T18:50:50.793 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.714554107Z level=info msg="Migration successfully executed" id="create team role table" duration=710.353µs 2024-09-20T18:50:50.793 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.715053072Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-09-20T18:50:50.793 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.715736943Z level=info msg="Migration successfully executed" id="add index team_role.org_id" duration=683.75µs 2024-09-20T18:50:50.793 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.717802116Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-09-20T18:50:50.793 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.718534233Z level=info msg="Migration successfully executed" id="add unique index team_role_org_id_team_id_role_id" duration=732.738µs 2024-09-20T18:50:50.793 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.720618992Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-09-20T18:50:50.793 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.721419697Z level=info msg="Migration successfully executed" id="add index team_role.team_id" duration=802.529µs 2024-09-20T18:50:50.793 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.723143994Z level=info msg="Executing migration" id="create user role table" 2024-09-20T18:50:50.794 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.723701106Z level=info msg="Migration successfully executed" id="create user role table" duration=557.151µs 2024-09-20T18:50:50.794 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.724061631Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-09-20T18:50:50.794 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.724711985Z level=info msg="Migration successfully executed" id="add index user_role.org_id" duration=650.146µs 2024-09-20T18:50:50.794 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.726964606Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-09-20T18:50:50.794 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.727695347Z level=info msg="Migration successfully executed" id="add unique index user_role_org_id_user_id_role_id" duration=732.585µs 2024-09-20T18:50:50.794 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.729644229Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-09-20T18:50:50.794 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.730423543Z level=info msg="Migration successfully executed" id="add index user_role.user_id" duration=787.268µs 2024-09-20T18:50:50.794 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.732151989Z level=info msg="Executing migration" id="create builtin role table" 2024-09-20T18:50:50.794 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.732733722Z level=info msg="Migration successfully executed" id="create builtin role table" duration=581.54µs 2024-09-20T18:50:50.794 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.734913929Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-09-20T18:50:50.794 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.735637301Z level=info msg="Migration successfully executed" id="add index builtin_role.role_id" duration=723.153µs 2024-09-20T18:50:50.794 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.737753912Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-09-20T18:50:50.794 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.738536508Z level=info msg="Migration successfully executed" id="add index builtin_role.name" duration=788.629µs 2024-09-20T18:50:50.794 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.7403661Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-09-20T18:50:50.795 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.744900738Z level=info msg="Migration successfully executed" id="Add column org_id to builtin_role table" duration=4.5327ms 2024-09-20T18:50:50.795 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.745276562Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-09-20T18:50:50.795 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.745934652Z level=info msg="Migration successfully executed" id="add index builtin_role.org_id" duration=658.985µs 2024-09-20T18:50:50.795 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.748013658Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-09-20T18:50:50.795 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.748699253Z level=info msg="Migration successfully executed" id="add unique index builtin_role_org_id_role_id_role" duration=685.369µs 2024-09-20T18:50:50.795 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.750685945Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-09-20T18:50:50.795 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.751379538Z level=info msg="Migration successfully executed" id="Remove unique index role_org_id_uid" duration=693.617µs 2024-09-20T18:50:50.795 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.753475501Z level=info msg="Executing migration" id="add unique index role.uid" 2024-09-20T18:50:50.795 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.754146642Z level=info msg="Migration successfully executed" id="add unique index role.uid" duration=670.908µs 2024-09-20T18:50:50.795 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.756352814Z level=info msg="Executing migration" id="create seed assignment table" 2024-09-20T18:50:50.795 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.756920219Z level=info msg="Migration successfully executed" id="create seed assignment table" duration=569.319µs 2024-09-20T18:50:50.795 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.758748519Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-09-20T18:50:50.795 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.759471128Z level=info msg="Migration successfully executed" id="add unique index builtin_role_role_name" duration=722.671µs 2024-09-20T18:50:50.795 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.76145038Z level=info msg="Executing migration" id="add column hidden to role table" 2024-09-20T18:50:50.796 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.766092725Z level=info msg="Migration successfully executed" id="add column hidden to role table" duration=4.642052ms 2024-09-20T18:50:50.796 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.766460113Z level=info msg="Executing migration" id="permission kind migration" 2024-09-20T18:50:50.808 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:50:50.829 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:50.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:50.830 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:50:50.860 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:50:50.860 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:50:50.867 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:50:50.867 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:50:50.884 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:50:50.885 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:50:50.902 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:50:50.902 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:50:50.920 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:50:50.920 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:50:50.931 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:50:50.937 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:50:50.937 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:50:50.956 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:50:50.956 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:50:50.957 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:50:50 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:50:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:50:50.958 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:50 smithi083 ceph-mon[29421]: osdmap e84: 8 total, 8 up, 8 in 2024-09-20T18:50:50.958 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:50 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:50.958 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:50 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:50.958 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:50 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:50.958 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:50 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:50.958 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:50 smithi083 ceph-mon[29421]: Deploying daemon node-exporter.a on smithi083 2024-09-20T18:50:50.958 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:50 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:50.958 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:50 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/85751978' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:50:50.959 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:50 smithi083 ceph-mon[36260]: osdmap e84: 8 total, 8 up, 8 in 2024-09-20T18:50:50.959 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:50 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:50.959 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:50 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:50.959 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:50 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:50.959 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:50 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:50.959 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:50 smithi083 ceph-mon[36260]: Deploying daemon node-exporter.a on smithi083 2024-09-20T18:50:50.959 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:50 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:50.959 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:50 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/85751978' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:50:50.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:50.960 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:50:50.961 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:50:50.973 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:50:50.973 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:50:51.033 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-mon[32508]: osdmap e84: 8 total, 8 up, 8 in 2024-09-20T18:50:51.034 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:51.034 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:51.034 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:51.034 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:51.034 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-mon[32508]: Deploying daemon node-exporter.a on smithi083 2024-09-20T18:50:51.034 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:51.034 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/85751978' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:50:51.035 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.773007791Z level=info msg="Migration successfully executed" id="permission kind migration" duration=6.547482ms 2024-09-20T18:50:51.035 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.773400224Z level=info msg="Executing migration" id="permission attribute migration" 2024-09-20T18:50:51.035 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.778257873Z level=info msg="Migration successfully executed" id="permission attribute migration" duration=4.857981ms 2024-09-20T18:50:51.035 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.778589847Z level=info msg="Executing migration" id="permission identifier migration" 2024-09-20T18:50:51.035 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.78310425Z level=info msg="Migration successfully executed" id="permission identifier migration" duration=4.514045ms 2024-09-20T18:50:51.035 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.783451919Z level=info msg="Executing migration" id="add permission identifier index" 2024-09-20T18:50:51.035 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.784116762Z level=info msg="Migration successfully executed" id="add permission identifier index" duration=664.622µs 2024-09-20T18:50:51.036 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.784454105Z level=info msg="Executing migration" id="add permission action scope role_id index" 2024-09-20T18:50:51.036 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.785102396Z level=info msg="Migration successfully executed" id="add permission action scope role_id index" duration=647.888µs 2024-09-20T18:50:51.036 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.785433561Z level=info msg="Executing migration" id="remove permission role_id action scope index" 2024-09-20T18:50:51.036 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.786069747Z level=info msg="Migration successfully executed" id="remove permission role_id action scope index" duration=636.78µs 2024-09-20T18:50:51.036 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.786398684Z level=info msg="Executing migration" id="create query_history table v1" 2024-09-20T18:50:51.036 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.786955361Z level=info msg="Migration successfully executed" id="create query_history table v1" duration=554.717µs 2024-09-20T18:50:51.036 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.787314273Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-09-20T18:50:51.036 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.787955225Z level=info msg="Migration successfully executed" id="add index query_history.org_id-created_by-datasource_uid" duration=640.576µs 2024-09-20T18:50:51.036 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.788287139Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-09-20T18:50:51.036 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.788324437Z level=info msg="Migration successfully executed" id="alter table query_history alter column created_by type to bigint" duration=37.6µs 2024-09-20T18:50:51.036 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.789238679Z level=info msg="Executing migration" id="rbac disabled migrator" 2024-09-20T18:50:51.036 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.789282595Z level=info msg="Migration successfully executed" id="rbac disabled migrator" duration=45.27µs 2024-09-20T18:50:51.037 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.789872458Z level=info msg="Executing migration" id="teams permissions migration" 2024-09-20T18:50:51.037 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.790127815Z level=info msg="Migration successfully executed" id="teams permissions migration" duration=255.544µs 2024-09-20T18:50:51.037 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.790627927Z level=info msg="Executing migration" id="dashboard permissions" 2024-09-20T18:50:51.037 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.790942211Z level=info msg="Migration successfully executed" id="dashboard permissions" duration=314.883µs 2024-09-20T18:50:51.037 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.791445826Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-09-20T18:50:51.037 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.791866247Z level=info msg="Migration successfully executed" id="dashboard permissions uid scopes" duration=420.436µs 2024-09-20T18:50:51.037 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.792247383Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-09-20T18:50:51.037 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.792373649Z level=info msg="Migration successfully executed" id="drop managed folder create actions" duration=126.237µs 2024-09-20T18:50:51.037 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.79304055Z level=info msg="Executing migration" id="alerting notification permissions" 2024-09-20T18:50:51.037 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.79331576Z level=info msg="Migration successfully executed" id="alerting notification permissions" duration=283.448µs 2024-09-20T18:50:51.037 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.793895348Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-09-20T18:50:51.037 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.794432213Z level=info msg="Migration successfully executed" id="create query_history_star table v1" duration=536.621µs 2024-09-20T18:50:51.037 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.794764375Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-09-20T18:50:51.037 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.795479114Z level=info msg="Migration successfully executed" id="add index query_history.user_id-query_uid" duration=714.492µs 2024-09-20T18:50:51.038 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.795824467Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-09-20T18:50:51.038 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.800602389Z level=info msg="Migration successfully executed" id="add column org_id in query_history_star" duration=4.777543ms 2024-09-20T18:50:51.038 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.800944687Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-09-20T18:50:51.038 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.800982467Z level=info msg="Migration successfully executed" id="alter table query_history_star_mig column user_id type to bigint" duration=38.194µs 2024-09-20T18:50:51.038 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.801799026Z level=info msg="Executing migration" id="create correlation table v1" 2024-09-20T18:50:51.038 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.802451765Z level=info msg="Migration successfully executed" id="create correlation table v1" duration=652.464µs 2024-09-20T18:50:51.038 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.802785905Z level=info msg="Executing migration" id="add index correlations.uid" 2024-09-20T18:50:51.038 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.803488017Z level=info msg="Migration successfully executed" id="add index correlations.uid" duration=701.75µs 2024-09-20T18:50:51.038 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.803826956Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-09-20T18:50:51.038 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.8044749Z level=info msg="Migration successfully executed" id="add index correlations.source_uid" duration=647.627µs 2024-09-20T18:50:51.038 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.804798151Z level=info msg="Executing migration" id="add correlation config column" 2024-09-20T18:50:51.038 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.809766974Z level=info msg="Migration successfully executed" id="add correlation config column" duration=4.967424ms 2024-09-20T18:50:51.039 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.810178379Z level=info msg="Executing migration" id="drop index IDX_correlation_uid - v1" 2024-09-20T18:50:51.039 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.810848028Z level=info msg="Migration successfully executed" id="drop index IDX_correlation_uid - v1" duration=669.615µs 2024-09-20T18:50:51.039 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.811203184Z level=info msg="Executing migration" id="drop index IDX_correlation_source_uid - v1" 2024-09-20T18:50:51.039 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.811835534Z level=info msg="Migration successfully executed" id="drop index IDX_correlation_source_uid - v1" duration=632.048µs 2024-09-20T18:50:51.039 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.812170318Z level=info msg="Executing migration" id="Rename table correlation to correlation_tmp_qwerty - v1" 2024-09-20T18:50:51.039 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.82927939Z level=info msg="Migration successfully executed" id="Rename table correlation to correlation_tmp_qwerty - v1" duration=17.108265ms 2024-09-20T18:50:51.039 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.82964851Z level=info msg="Executing migration" id="create correlation v2" 2024-09-20T18:50:51.039 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.830412804Z level=info msg="Migration successfully executed" id="create correlation v2" duration=763.947µs 2024-09-20T18:50:51.039 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.830791834Z level=info msg="Executing migration" id="create index IDX_correlation_uid - v2" 2024-09-20T18:50:51.039 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.831472448Z level=info msg="Migration successfully executed" id="create index IDX_correlation_uid - v2" duration=680.237µs 2024-09-20T18:50:51.039 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.831805026Z level=info msg="Executing migration" id="create index IDX_correlation_source_uid - v2" 2024-09-20T18:50:51.039 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.832472375Z level=info msg="Migration successfully executed" id="create index IDX_correlation_source_uid - v2" duration=667.128µs 2024-09-20T18:50:51.039 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.832804517Z level=info msg="Executing migration" id="create index IDX_correlation_org_id - v2" 2024-09-20T18:50:51.039 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.833578126Z level=info msg="Migration successfully executed" id="create index IDX_correlation_org_id - v2" duration=774.285µs 2024-09-20T18:50:51.040 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.834159714Z level=info msg="Executing migration" id="copy correlation v1 to v2" 2024-09-20T18:50:51.040 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.834434949Z level=info msg="Migration successfully executed" id="copy correlation v1 to v2" duration=275.368µs 2024-09-20T18:50:51.040 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.834998249Z level=info msg="Executing migration" id="drop correlation_tmp_qwerty" 2024-09-20T18:50:51.040 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.835932823Z level=info msg="Migration successfully executed" id="drop correlation_tmp_qwerty" duration=934.258µs 2024-09-20T18:50:51.040 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.836320851Z level=info msg="Executing migration" id="add provisioning column" 2024-09-20T18:50:51.040 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.841140332Z level=info msg="Migration successfully executed" id="add provisioning column" duration=4.81779ms 2024-09-20T18:50:51.040 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.841481011Z level=info msg="Executing migration" id="create entity_events table" 2024-09-20T18:50:51.040 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.842002696Z level=info msg="Migration successfully executed" id="create entity_events table" duration=521.525µs 2024-09-20T18:50:51.041 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.842375966Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-09-20T18:50:51.041 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.843013764Z level=info msg="Migration successfully executed" id="create dashboard public config v1" duration=637.697µs 2024-09-20T18:50:51.041 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.843419844Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-20T18:50:51.041 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.843680387Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-20T18:50:51.041 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.844061605Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-20T18:50:51.041 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.844316208Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-20T18:50:51.041 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.846736838Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-09-20T18:50:51.041 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.84732461Z level=info msg="Migration successfully executed" id="Drop old dashboard public config table" duration=587.663µs 2024-09-20T18:50:51.041 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.849528227Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-09-20T18:50:51.041 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.85028209Z level=info msg="Migration successfully executed" id="recreate dashboard public config v1" duration=756.2µs 2024-09-20T18:50:51.041 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.850653652Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-09-20T18:50:51.042 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.85134198Z level=info msg="Migration successfully executed" id="create index UQE_dashboard_public_config_uid - v1" duration=688.26µs 2024-09-20T18:50:51.042 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.853298965Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-20T18:50:51.042 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.853959742Z level=info msg="Migration successfully executed" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" duration=660.796µs 2024-09-20T18:50:51.042 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.856056733Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-09-20T18:50:51.042 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.856704665Z level=info msg="Migration successfully executed" id="drop index UQE_dashboard_public_config_uid - v2" duration=645.87µs 2024-09-20T18:50:51.042 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.858670164Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-20T18:50:51.042 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.859349494Z level=info msg="Migration successfully executed" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" duration=679.179µs 2024-09-20T18:50:51.042 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.859702166Z level=info msg="Executing migration" id="Drop public config table" 2024-09-20T18:50:51.042 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.86028203Z level=info msg="Migration successfully executed" id="Drop public config table" duration=579.56µs 2024-09-20T18:50:51.042 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.862557984Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-09-20T18:50:51.043 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.863313564Z level=info msg="Migration successfully executed" id="Recreate dashboard public config v2" duration=756.388µs 2024-09-20T18:50:51.043 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.865324394Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-09-20T18:50:51.043 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.866004459Z level=info msg="Migration successfully executed" id="create index UQE_dashboard_public_config_uid - v2" duration=679.851µs 2024-09-20T18:50:51.043 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.867807597Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-20T18:50:51.043 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.86851682Z level=info msg="Migration successfully executed" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" duration=709.025µs 2024-09-20T18:50:51.044 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.868857112Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-09-20T18:50:51.044 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.869522438Z level=info msg="Migration successfully executed" id="create index UQE_dashboard_public_config_access_token - v2" duration=665.103µs 2024-09-20T18:50:51.044 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.869872818Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-09-20T18:50:51.044 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.884458208Z level=info msg="Migration successfully executed" id="Rename table dashboard_public_config to dashboard_public - v2" duration=14.585211ms 2024-09-20T18:50:51.044 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.88482742Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-09-20T18:50:51.044 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.890140007Z level=info msg="Migration successfully executed" id="add annotations_enabled column" duration=5.312226ms 2024-09-20T18:50:51.044 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.89048439Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-09-20T18:50:51.044 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.895439877Z level=info msg="Migration successfully executed" id="add time_selection_enabled column" duration=4.952638ms 2024-09-20T18:50:51.044 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.895793248Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-09-20T18:50:51.044 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.895933745Z level=info msg="Migration successfully executed" id="delete orphaned public dashboards" duration=140.63µs 2024-09-20T18:50:51.044 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.896654698Z level=info msg="Executing migration" id="add share column" 2024-09-20T18:50:51.044 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.901768341Z level=info msg="Migration successfully executed" id="add share column" duration=5.113421ms 2024-09-20T18:50:51.044 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.902130977Z level=info msg="Executing migration" id="backfill empty share column fields with default of public" 2024-09-20T18:50:51.044 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.90225808Z level=info msg="Migration successfully executed" id="backfill empty share column fields with default of public" duration=127.11µs 2024-09-20T18:50:51.045 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.902993075Z level=info msg="Executing migration" id="create file table" 2024-09-20T18:50:51.045 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.903627393Z level=info msg="Migration successfully executed" id="create file table" duration=635.93µs 2024-09-20T18:50:51.045 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.903998996Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-09-20T18:50:51.045 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.904678984Z level=info msg="Migration successfully executed" id="file table idx: path natural pk" duration=679.729µs 2024-09-20T18:50:51.045 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.905014809Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-09-20T18:50:51.045 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.905732915Z level=info msg="Migration successfully executed" id="file table idx: parent_folder_path_hash fast folder retrieval" duration=717.999µs 2024-09-20T18:50:51.045 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.906077815Z level=info msg="Executing migration" id="create file_meta table" 2024-09-20T18:50:51.045 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.906637644Z level=info msg="Migration successfully executed" id="create file_meta table" duration=559.528µs 2024-09-20T18:50:51.045 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.906981137Z level=info msg="Executing migration" id="file table idx: path key" 2024-09-20T18:50:51.045 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.907677485Z level=info msg="Migration successfully executed" id="file table idx: path key" duration=696.033µs 2024-09-20T18:50:51.045 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.908010582Z level=info msg="Executing migration" id="set path collation in file table" 2024-09-20T18:50:51.045 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.90805704Z level=info msg="Migration successfully executed" id="set path collation in file table" duration=46.639µs 2024-09-20T18:50:51.045 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.908833865Z level=info msg="Executing migration" id="migrate contents column to mediumblob for MySQL" 2024-09-20T18:50:51.046 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.908876875Z level=info msg="Migration successfully executed" id="migrate contents column to mediumblob for MySQL" duration=43.079µs 2024-09-20T18:50:51.046 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.909826258Z level=info msg="Executing migration" id="managed permissions migration" 2024-09-20T18:50:51.046 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.91021852Z level=info msg="Migration successfully executed" id="managed permissions migration" duration=394.769µs 2024-09-20T18:50:51.046 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.910596513Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-09-20T18:50:51.046 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.910715685Z level=info msg="Migration successfully executed" id="managed folder permissions alert actions migration" duration=119.206µs 2024-09-20T18:50:51.046 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.911080578Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-09-20T18:50:51.046 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.911872425Z level=info msg="Migration successfully executed" id="RBAC action name migrator" duration=791.451µs 2024-09-20T18:50:51.046 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.912261643Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-09-20T18:50:51.046 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.917492458Z level=info msg="Migration successfully executed" id="Add UID column to playlist" duration=5.23055ms 2024-09-20T18:50:51.046 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.917844289Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-09-20T18:50:51.046 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.917945308Z level=info msg="Migration successfully executed" id="Update uid column values in playlist" duration=101.333µs 2024-09-20T18:50:51.046 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.918741821Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-09-20T18:50:51.046 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.919446338Z level=info msg="Migration successfully executed" id="Add index for uid in playlist" duration=705.036µs 2024-09-20T18:50:51.046 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.919797078Z level=info msg="Executing migration" id="update group index for alert rules" 2024-09-20T18:50:51.047 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.920017352Z level=info msg="Migration successfully executed" id="update group index for alert rules" duration=220.764µs 2024-09-20T18:50:51.047 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.920435147Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-09-20T18:50:51.047 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.920555485Z level=info msg="Migration successfully executed" id="managed folder permissions alert actions repeated migration" duration=120.348µs 2024-09-20T18:50:51.047 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.923146394Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-09-20T18:50:51.047 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.923430769Z level=info msg="Migration successfully executed" id="admin only folder/dashboard permission" duration=284.508µs 2024-09-20T18:50:51.047 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.926031207Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-09-20T18:50:51.047 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.931009334Z level=info msg="Migration successfully executed" id="add action column to seed_assignment" duration=4.949547ms 2024-09-20T18:50:51.047 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.931375656Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-09-20T18:50:51.047 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.936839818Z level=info msg="Migration successfully executed" id="add scope column to seed_assignment" duration=5.464011ms 2024-09-20T18:50:51.047 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.937192563Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-09-20T18:50:51.047 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.937854295Z level=info msg="Migration successfully executed" id="remove unique index builtin_role_role_name before nullable update" duration=661.809µs 2024-09-20T18:50:51.047 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.940158596Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-09-20T18:50:51.047 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.996346666Z level=info msg="Migration successfully executed" id="update seed_assignment role_name column to nullable" duration=56.189494ms 2024-09-20T18:50:51.047 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.996743063Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-09-20T18:50:51.048 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.997444501Z level=info msg="Migration successfully executed" id="add unique index builtin_role_name back" duration=701.303µs 2024-09-20T18:50:51.048 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:50 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:50.99950521Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-09-20T18:50:51.048 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.000201563Z level=info msg="Migration successfully executed" id="add unique index builtin_role_action_scope" duration=696.041µs 2024-09-20T18:50:51.048 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.000770309Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-09-20T18:50:51.048 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.017413481Z level=info msg="Migration successfully executed" id="add primary key to seed_assigment" duration=16.642847ms 2024-09-20T18:50:51.048 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.017860089Z level=info msg="Executing migration" id="add origin column to seed_assignment" 2024-09-20T18:50:51.061 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:50:51.082 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:50:51.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:51.083 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:50:51.183 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:50:51.206 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:50:51.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:51.207 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:50:51.290 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.023814129Z level=info msg="Migration successfully executed" id="add origin column to seed_assignment" duration=5.955904ms 2024-09-20T18:50:51.290 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.024212959Z level=info msg="Executing migration" id="add origin to plugin seed_assignment" 2024-09-20T18:50:51.290 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.024435183Z level=info msg="Migration successfully executed" id="add origin to plugin seed_assignment" duration=222.234µs 2024-09-20T18:50:51.290 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.026889315Z level=info msg="Executing migration" id="prevent seeding OnCall access" 2024-09-20T18:50:51.291 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.027008355Z level=info msg="Migration successfully executed" id="prevent seeding OnCall access" duration=118.54µs 2024-09-20T18:50:51.291 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.027573929Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-09-20T18:50:51.291 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.027720989Z level=info msg="Migration successfully executed" id="managed folder permissions alert actions repeated fixed migration" duration=146.995µs 2024-09-20T18:50:51.291 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.03041068Z level=info msg="Executing migration" id="managed folder permissions library panel actions migration" 2024-09-20T18:50:51.291 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.030551032Z level=info msg="Migration successfully executed" id="managed folder permissions library panel actions migration" duration=138.345µs 2024-09-20T18:50:51.291 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.033333404Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-09-20T18:50:51.291 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.033475251Z level=info msg="Migration successfully executed" id="migrate external alertmanagers to datsourcse" duration=141.667µs 2024-09-20T18:50:51.291 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.036166331Z level=info msg="Executing migration" id="create folder table" 2024-09-20T18:50:51.291 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.036797591Z level=info msg="Migration successfully executed" id="create folder table" duration=631.006µs 2024-09-20T18:50:51.291 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.037174575Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-09-20T18:50:51.291 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.037869335Z level=info msg="Migration successfully executed" id="Add index for parent_uid" duration=694.589µs 2024-09-20T18:50:51.291 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.038227034Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-09-20T18:50:51.292 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.038924576Z level=info msg="Migration successfully executed" id="Add unique index for folder.uid and folder.org_id" duration=697.169µs 2024-09-20T18:50:51.292 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.041303801Z level=info msg="Executing migration" id="Update folder title length" 2024-09-20T18:50:51.292 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.041341734Z level=info msg="Migration successfully executed" id="Update folder title length" duration=39.416µs 2024-09-20T18:50:51.292 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.043885043Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-09-20T18:50:51.292 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.044636119Z level=info msg="Migration successfully executed" id="Add unique index for folder.title and folder.parent_uid" duration=749.194µs 2024-09-20T18:50:51.292 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.045000559Z level=info msg="Executing migration" id="Remove unique index for folder.title and folder.parent_uid" 2024-09-20T18:50:51.292 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.045690736Z level=info msg="Migration successfully executed" id="Remove unique index for folder.title and folder.parent_uid" duration=690.037µs 2024-09-20T18:50:51.292 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.047841594Z level=info msg="Executing migration" id="Add unique index for title, parent_uid, and org_id" 2024-09-20T18:50:51.292 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.048565198Z level=info msg="Migration successfully executed" id="Add unique index for title, parent_uid, and org_id" duration=723.369µs 2024-09-20T18:50:51.292 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.05031086Z level=info msg="Executing migration" id="Sync dashboard and folder table" 2024-09-20T18:50:51.292 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.050632591Z level=info msg="Migration successfully executed" id="Sync dashboard and folder table" duration=321.561µs 2024-09-20T18:50:51.292 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.05310136Z level=info msg="Executing migration" id="Remove ghost folders from the folder table" 2024-09-20T18:50:51.292 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.053270408Z level=info msg="Migration successfully executed" id="Remove ghost folders from the folder table" duration=169.015µs 2024-09-20T18:50:51.293 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.053644563Z level=info msg="Executing migration" id="Remove unique index UQE_folder_uid_org_id" 2024-09-20T18:50:51.293 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.054289312Z level=info msg="Migration successfully executed" id="Remove unique index UQE_folder_uid_org_id" duration=644.514µs 2024-09-20T18:50:51.293 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.056173346Z level=info msg="Executing migration" id="Add unique index UQE_folder_org_id_uid" 2024-09-20T18:50:51.293 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.056848999Z level=info msg="Migration successfully executed" id="Add unique index UQE_folder_org_id_uid" duration=675.607µs 2024-09-20T18:50:51.293 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.059058225Z level=info msg="Executing migration" id="Remove unique index UQE_folder_title_parent_uid_org_id" 2024-09-20T18:50:51.293 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.059742995Z level=info msg="Migration successfully executed" id="Remove unique index UQE_folder_title_parent_uid_org_id" duration=686.503µs 2024-09-20T18:50:51.293 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.061558265Z level=info msg="Executing migration" id="Add unique index UQE_folder_org_id_parent_uid_title" 2024-09-20T18:50:51.293 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.062400893Z level=info msg="Migration successfully executed" id="Add unique index UQE_folder_org_id_parent_uid_title" duration=844.193µs 2024-09-20T18:50:51.293 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.06274134Z level=info msg="Executing migration" id="Remove index IDX_folder_parent_uid_org_id" 2024-09-20T18:50:51.293 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.063399192Z level=info msg="Migration successfully executed" id="Remove index IDX_folder_parent_uid_org_id" duration=657.449µs 2024-09-20T18:50:51.293 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.065472857Z level=info msg="Executing migration" id="create anon_device table" 2024-09-20T18:50:51.293 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.066051856Z level=info msg="Migration successfully executed" id="create anon_device table" duration=556.723µs 2024-09-20T18:50:51.293 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.068230549Z level=info msg="Executing migration" id="add unique index anon_device.device_id" 2024-09-20T18:50:51.294 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.069045118Z level=info msg="Migration successfully executed" id="add unique index anon_device.device_id" duration=803.99µs 2024-09-20T18:50:51.294 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.070829347Z level=info msg="Executing migration" id="add index anon_device.updated_at" 2024-09-20T18:50:51.294 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.071647941Z level=info msg="Migration successfully executed" id="add index anon_device.updated_at" duration=820.489µs 2024-09-20T18:50:51.294 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.072001343Z level=info msg="Executing migration" id="create signing_key table" 2024-09-20T18:50:51.294 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.072547557Z level=info msg="Migration successfully executed" id="create signing_key table" duration=545.974µs 2024-09-20T18:50:51.294 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.074646415Z level=info msg="Executing migration" id="add unique index signing_key.key_id" 2024-09-20T18:50:51.294 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.075467316Z level=info msg="Migration successfully executed" id="add unique index signing_key.key_id" duration=827.185µs 2024-09-20T18:50:51.294 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.077169888Z level=info msg="Executing migration" id="set legacy alert migration status in kvstore" 2024-09-20T18:50:51.294 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.077932278Z level=info msg="Migration successfully executed" id="set legacy alert migration status in kvstore" duration=766.29µs 2024-09-20T18:50:51.294 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.079514586Z level=info msg="Executing migration" id="migrate record of created folders during legacy migration to kvstore" 2024-09-20T18:50:51.294 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.079697212Z level=info msg="Migration successfully executed" id="migrate record of created folders during legacy migration to kvstore" duration=183.192µs 2024-09-20T18:50:51.294 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.080093823Z level=info msg="Executing migration" id="Add folder_uid for dashboard" 2024-09-20T18:50:51.294 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.0854854Z level=info msg="Migration successfully executed" id="Add folder_uid for dashboard" duration=5.391273ms 2024-09-20T18:50:51.294 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.085841893Z level=info msg="Executing migration" id="Populate dashboard folder_uid column" 2024-09-20T18:50:51.295 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.086331188Z level=info msg="Migration successfully executed" id="Populate dashboard folder_uid column" duration=489.885µs 2024-09-20T18:50:51.295 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.086739292Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_folder_uid_title" 2024-09-20T18:50:51.295 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.086749787Z level=info msg="Migration successfully executed" id="Add unique index for dashboard_org_id_folder_uid_title" duration=12.322µs 2024-09-20T18:50:51.295 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.087751486Z level=info msg="Executing migration" id="Delete unique index for dashboard_org_id_folder_id_title" 2024-09-20T18:50:51.295 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.088500887Z level=info msg="Migration successfully executed" id="Delete unique index for dashboard_org_id_folder_id_title" duration=749.818µs 2024-09-20T18:50:51.295 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.088849323Z level=info msg="Executing migration" id="Delete unique index for dashboard_org_id_folder_uid_title" 2024-09-20T18:50:51.295 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.088861451Z level=info msg="Migration successfully executed" id="Delete unique index for dashboard_org_id_folder_uid_title" duration=12.715µs 2024-09-20T18:50:51.295 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.089664173Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_folder_uid_title_is_folder" 2024-09-20T18:50:51.295 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.090430357Z level=info msg="Migration successfully executed" id="Add unique index for dashboard_org_id_folder_uid_title_is_folder" duration=766.249µs 2024-09-20T18:50:51.295 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.090787971Z level=info msg="Executing migration" id="Restore index for dashboard_org_id_folder_id_title" 2024-09-20T18:50:51.295 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.091508559Z level=info msg="Migration successfully executed" id="Restore index for dashboard_org_id_folder_id_title" duration=720.077µs 2024-09-20T18:50:51.295 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.091871414Z level=info msg="Executing migration" id="create sso_setting table" 2024-09-20T18:50:51.295 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.092538237Z level=info msg="Migration successfully executed" id="create sso_setting table" duration=666.679µs 2024-09-20T18:50:51.296 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.092898684Z level=info msg="Executing migration" id="copy kvstore migration status to each org" 2024-09-20T18:50:51.296 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.093419142Z level=info msg="Migration successfully executed" id="copy kvstore migration status to each org" duration=522.407µs 2024-09-20T18:50:51.296 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.093776125Z level=info msg="Executing migration" id="add back entry for orgid=0 migrated status" 2024-09-20T18:50:51.296 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.093939805Z level=info msg="Migration successfully executed" id="add back entry for orgid=0 migrated status" duration=164.694µs 2024-09-20T18:50:51.296 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.094621901Z level=info msg="Executing migration" id="alter kv_store.value to longtext" 2024-09-20T18:50:51.296 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.094683584Z level=info msg="Migration successfully executed" id="alter kv_store.value to longtext" duration=61.904µs 2024-09-20T18:50:51.296 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.09543526Z level=info msg="Executing migration" id="add notification_settings column to alert_rule table" 2024-09-20T18:50:51.296 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.100527981Z level=info msg="Migration successfully executed" id="add notification_settings column to alert_rule table" duration=5.093299ms 2024-09-20T18:50:51.296 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.100902107Z level=info msg="Executing migration" id="add notification_settings column to alert_rule_version table" 2024-09-20T18:50:51.296 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.106118045Z level=info msg="Migration successfully executed" id="add notification_settings column to alert_rule_version table" duration=5.215678ms 2024-09-20T18:50:51.296 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.10647317Z level=info msg="Executing migration" id="removing scope from alert.instances:read action migration" 2024-09-20T18:50:51.296 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.106683018Z level=info msg="Migration successfully executed" id="removing scope from alert.instances:read action migration" duration=209.831µs 2024-09-20T18:50:51.296 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=migrator t=2024-09-20T18:50:51.107373816Z level=info msg="migrations completed" performed=551 skipped=0 duration=1.277093451s 2024-09-20T18:50:51.296 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=sqlstore t=2024-09-20T18:50:51.108439385Z level=info msg="Created default organization" 2024-09-20T18:50:51.297 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=secrets t=2024-09-20T18:50:51.108996752Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-09-20T18:50:51.297 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=plugin.store t=2024-09-20T18:50:51.123358499Z level=info msg="Loading plugins..." 2024-09-20T18:50:51.297 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=local.finder t=2024-09-20T18:50:51.165359305Z level=warn msg="Skipping finding plugins as directory does not exist" path=/usr/share/grafana/plugins-bundled 2024-09-20T18:50:51.297 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=plugin.store t=2024-09-20T18:50:51.165378418Z level=info msg="Plugins loaded" count=55 duration=42.02049ms 2024-09-20T18:50:51.297 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=query_data t=2024-09-20T18:50:51.167365987Z level=info msg="Query Service initialization" 2024-09-20T18:50:51.298 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=live.push_http t=2024-09-20T18:50:51.169635247Z level=info msg="Live Push Gateway initialization" 2024-09-20T18:50:51.298 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=ngalert.migration t=2024-09-20T18:50:51.17068006Z level=info msg=Starting 2024-09-20T18:50:51.298 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=ngalert.migration t=2024-09-20T18:50:51.170941005Z level=info msg="Applying transition" currentType=Legacy desiredType=UnifiedAlerting cleanOnDowngrade=false cleanOnUpgrade=false 2024-09-20T18:50:51.298 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=ngalert.migration orgID=1 t=2024-09-20T18:50:51.171211082Z level=info msg="Migrating alerts for organisation" 2024-09-20T18:50:51.298 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=ngalert.migration orgID=1 t=2024-09-20T18:50:51.171630685Z level=info msg="Alerts found to migrate" alerts=0 2024-09-20T18:50:51.298 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=ngalert.migration t=2024-09-20T18:50:51.172723145Z level=info msg="Completed alerting migration" 2024-09-20T18:50:51.298 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=ngalert.state.manager t=2024-09-20T18:50:51.183171091Z level=info msg="Running in alternative execution of Error/NoData mode" 2024-09-20T18:50:51.298 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=infra.usagestats.collector t=2024-09-20T18:50:51.184401541Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-09-20T18:50:51.298 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=provisioning.datasources t=2024-09-20T18:50:51.185934551Z level=info msg="inserting datasource from configuration" name=Dashboard1 uid=P43CA22E17D0F9596 2024-09-20T18:50:51.298 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=provisioning.datasources t=2024-09-20T18:50:51.197812509Z level=info msg="inserting datasource from configuration" name=Loki uid=P8E80F9AEF21F6940 2024-09-20T18:50:51.298 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=provisioning.alerting t=2024-09-20T18:50:51.210115723Z level=info msg="starting to provision alerting" 2024-09-20T18:50:51.298 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=provisioning.alerting t=2024-09-20T18:50:51.210126848Z level=info msg="finished to provision alerting" 2024-09-20T18:50:51.299 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=grafanaStorageLogger t=2024-09-20T18:50:51.210220932Z level=info msg="Storage starting" 2024-09-20T18:50:51.299 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=ngalert.state.manager t=2024-09-20T18:50:51.210416938Z level=info msg="Warming state cache for startup" 2024-09-20T18:50:51.299 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=http.server t=2024-09-20T18:50:51.21282832Z level=info msg="HTTP Server TLS settings" MinTLSVersion=TLS1.2 configuredciphers=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,TLS_RSA_WITH_AES_128_GCM_SHA256,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA 2024-09-20T18:50:51.299 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=http.server t=2024-09-20T18:50:51.213162573Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-09-20T18:50:51.299 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=ngalert.multiorg.alertmanager t=2024-09-20T18:50:51.225426469Z level=info msg="Starting MultiOrg Alertmanager" 2024-09-20T18:50:51.299 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=ngalert.state.manager t=2024-09-20T18:50:51.23885503Z level=info msg="State cache has been initialized" states=0 duration=28.437259ms 2024-09-20T18:50:51.299 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=ngalert.scheduler t=2024-09-20T18:50:51.238899807Z level=info msg="Starting scheduler" tickInterval=10s maxAttempts=1 2024-09-20T18:50:51.299 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=ticker t=2024-09-20T18:50:51.238941279Z level=info msg=starting first_tick=2024-09-20T18:51:00Z 2024-09-20T18:50:51.299 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=provisioning.dashboard t=2024-09-20T18:50:51.270665929Z level=info msg="starting to provision dashboards" 2024-09-20T18:50:51.307 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:50:51.331 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:51.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:51.332 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:50:51.433 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:50:51.473 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:50:51.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:51.474 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:50:51.578 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:50:51.579 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=sqlstore.transactions t=2024-09-20T18:50:51.288153316Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-20T18:50:51.579 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=sqlstore.transactions t=2024-09-20T18:50:51.298811922Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=1 code="database is locked" 2024-09-20T18:50:51.579 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=grafana.update.checker t=2024-09-20T18:50:51.303238383Z level=info msg="Update check succeeded" duration=89.855244ms 2024-09-20T18:50:51.579 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=plugins.update.checker t=2024-09-20T18:50:51.305268454Z level=info msg="Update check succeeded" duration=94.982798ms 2024-09-20T18:50:51.579 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=sqlstore.transactions t=2024-09-20T18:50:51.309622604Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=2 code="database is locked" 2024-09-20T18:50:51.579 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=sqlstore.transactions t=2024-09-20T18:50:51.322576078Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-20T18:50:51.579 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=sqlstore.transactions t=2024-09-20T18:50:51.344265364Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-20T18:50:51.580 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=provisioning.dashboard t=2024-09-20T18:50:51.4200723Z level=info msg="finished to provision dashboards" 2024-09-20T18:50:51.580 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=grafana-apiserver t=2024-09-20T18:50:51.517121439Z level=info msg="Adding GroupVersion playlist.grafana.app v0alpha1 to ResourceManager" 2024-09-20T18:50:51.580 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=grafana-apiserver t=2024-09-20T18:50:51.517488422Z level=info msg="Adding GroupVersion featuretoggle.grafana.app v0alpha1 to ResourceManager" 2024-09-20T18:50:51.600 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:51.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:51.601 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:50:51.701 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:50:51.724 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:50:51.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:51.725 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:50:51.826 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:50:51.846 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:50:51.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:51.847 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:50:51.859 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T18:50:51.949 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:50:51.971 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:50:51.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:51.972 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:50:52.072 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:50:52.095 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:50:52.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:52.096 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:50:52.167 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:51 smithi083 ceph-mon[36260]: pgmap v31: 148 pgs: 148 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T18:50:52.167 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:51 smithi083 ceph-mon[36260]: osdmap e85: 8 total, 8 up, 8 in 2024-09-20T18:50:52.168 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:51 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:50:52.168 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:51 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:50:52.168 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:51 smithi083 ceph-mon[36260]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-20T18:50:52.168 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:51 smithi083 ceph-mon[29421]: pgmap v31: 148 pgs: 148 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T18:50:52.168 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:51 smithi083 ceph-mon[29421]: osdmap e85: 8 total, 8 up, 8 in 2024-09-20T18:50:52.168 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:51 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:50:52.168 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:51 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:50:52.168 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:51 smithi083 ceph-mon[29421]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-20T18:50:52.185 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T18:50:52.185 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:50:52.198 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:50:52.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:52.220 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:52.220 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:50:52.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-mon[32508]: pgmap v31: 148 pgs: 148 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T18:50:52.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-mon[32508]: osdmap e85: 8 total, 8 up, 8 in 2024-09-20T18:50:52.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:50:52.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:50:52.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:51 smithi142 ceph-mon[32508]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-20T18:50:52.321 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:50:52.342 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:52.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:52.343 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:50:52.444 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:50:52.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:52.466 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:52.466 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:50:52.568 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:50:52.589 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:52.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:52.590 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:50:52.691 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:50:52.712 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:52.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:52.713 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:50:52.753 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_blocked_ops 2024-09-20T18:50:52.814 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:50:52.835 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:50:52.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:52.836 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:50:52.937 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:50:52.958 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:50:52.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:52.959 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:50:53.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:52 smithi083 ceph-mon[36260]: osdmap e86: 8 total, 8 up, 8 in 2024-09-20T18:50:53.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:52 smithi083 ceph-mon[29421]: osdmap e86: 8 total, 8 up, 8 in 2024-09-20T18:50:53.060 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:50:53.081 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:50:53.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:53.082 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:50:53.183 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:50:53.187 INFO:tasks.thrashosds.thrasher:j is {'epoch': 82, 'fsid': '1ed5a50a-7780-11ef-baf5-efdc52797490', 'created': '2024-09-20T18:44:10.399800+0000', 'modified': '2024-09-20T18:50:40.555466+0000', 'last_up_change': '2024-09-20T18:49:34.836377+0000', 'last_in_change': '2024-09-20T18:49:12.342332+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 18, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 6, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-20T18:47:15.285795+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'datapool', 'create_time': '2024-09-20T18:49:43.452984+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 61, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.6500000953674316, 'score_stable': 2.6500000953674316, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 2.3299999237060547, 'raw_score_stable': 2.3299999237060547, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': '.rgw.root', 'create_time': '2024-09-20T18:49:44.217011+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-20T18:49:46.007275+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-20T18:49:47.961258+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-20T18:49:50.011276+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': 'c714e1ec-f0c3-4054-9792-61b84e53eaa8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 64, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6801', 'nonce': 2574084049}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6802', 'nonce': 2574084049}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6804', 'nonce': 2574084049}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6803', 'nonce': 2574084049}]}, 'public_addr': '172.21.15.83:6801/2574084049', 'cluster_addr': '172.21.15.83:6802/2574084049', 'heartbeat_back_addr': '172.21.15.83:6804/2574084049', 'heartbeat_front_addr': '172.21.15.83:6803/2574084049', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '9dc0f2a8-bafa-4a42-88f5-0301dd58f78d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 64, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6805', 'nonce': 4213285798}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6806', 'nonce': 4213285798}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6808', 'nonce': 4213285798}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6807', 'nonce': 4213285798}]}, 'public_addr': '172.21.15.83:6805/4213285798', 'cluster_addr': '172.21.15.83:6806/4213285798', 'heartbeat_back_addr': '172.21.15.83:6808/4213285798', 'heartbeat_front_addr': '172.21.15.83:6807/4213285798', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '448ab692-ec41-44dc-b2d5-dbefd255b3fd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 68, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6809', 'nonce': 805947571}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6810', 'nonce': 805947571}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6812', 'nonce': 805947571}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6811', 'nonce': 805947571}]}, 'public_addr': '172.21.15.83:6809/805947571', 'cluster_addr': '172.21.15.83:6810/805947571', 'heartbeat_back_addr': '172.21.15.83:6812/805947571', 'heartbeat_front_addr': '172.21.15.83:6811/805947571', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '7cda5e50-e22e-4d4a-823b-53c8f5061193', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 25, 'up_thru': 68, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6813', 'nonce': 987237318}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6814', 'nonce': 987237318}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6816', 'nonce': 987237318}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6815', 'nonce': 987237318}]}, 'public_addr': '172.21.15.83:6813/987237318', 'cluster_addr': '172.21.15.83:6814/987237318', 'heartbeat_back_addr': '172.21.15.83:6816/987237318', 'heartbeat_front_addr': '172.21.15.83:6815/987237318', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '3d62654d-d31d-4310-9934-49e694dd17d9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 31, 'up_thru': 68, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6800', 'nonce': 2096684065}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6801', 'nonce': 2096684065}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6803', 'nonce': 2096684065}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6802', 'nonce': 2096684065}]}, 'public_addr': '172.21.15.142:6800/2096684065', 'cluster_addr': '172.21.15.142:6801/2096684065', 'heartbeat_back_addr': '172.21.15.142:6803/2096684065', 'heartbeat_front_addr': '172.21.15.142:6802/2096684065', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'b3e4e21b-639f-4d25-8bbf-ea8922a9222a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 38, 'up_thru': 64, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6804', 'nonce': 2500466067}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6805', 'nonce': 2500466067}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6807', 'nonce': 2500466067}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6806', 'nonce': 2500466067}]}, 'public_addr': '172.21.15.142:6804/2500466067', 'cluster_addr': '172.21.15.142:6805/2500466067', 'heartbeat_back_addr': '172.21.15.142:6807/2500466067', 'heartbeat_front_addr': '172.21.15.142:6806/2500466067', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'c5db7107-ccb4-4ae2-9e63-f444897942e3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 45, 'up_thru': 62, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6808', 'nonce': 1688186226}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6809', 'nonce': 1688186226}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6811', 'nonce': 1688186226}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6810', 'nonce': 1688186226}]}, 'public_addr': '172.21.15.142:6808/1688186226', 'cluster_addr': '172.21.15.142:6809/1688186226', 'heartbeat_back_addr': '172.21.15.142:6811/1688186226', 'heartbeat_front_addr': '172.21.15.142:6810/1688186226', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '07d16806-e2c8-4acb-b284-d75701e61622', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 53, 'up_thru': 64, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6812', 'nonce': 3679770984}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6813', 'nonce': 3679770984}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6815', 'nonce': 3679770984}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6814', 'nonce': 3679770984}]}, 'public_addr': '172.21.15.142:6812/3679770984', 'cluster_addr': '172.21.15.142:6813/3679770984', 'heartbeat_back_addr': '172.21.15.142:6815/3679770984', 'heartbeat_front_addr': '172.21.15.142:6814/3679770984', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T18:50:11.165148+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T18:50:22.526229+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T18:49:39.137302+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T18:50:38.304985+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T18:50:27.991715+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T18:50:04.573333+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T18:50:20.101102+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T18:49:33.595929+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '4.f', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.14', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.1b', 'mappings': [{'from': 1, 'to': 2}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {'172.21.15.83:0/3474253603': '2024-09-21T18:44:32.434180+0000', '172.21.15.83:6800/3525856829': '2024-09-21T18:44:32.434180+0000', '172.21.15.83:0/220746284': '2024-09-21T18:44:32.434180+0000', '172.21.15.83:0/3875445393': '2024-09-21T18:44:56.985620+0000', '172.21.15.83:0/3009092807': '2024-09-21T18:44:56.985620+0000', '172.21.15.83:0/4267132131': '2024-09-21T18:44:56.985620+0000', '172.21.15.83:0/3684382972': '2024-09-21T18:50:14.729526+0000', '172.21.15.83:0/3132933178': '2024-09-21T18:50:14.729526+0000', '172.21.15.83:6800/1007846146': '2024-09-21T18:50:14.729526+0000', '172.21.15.83:6800/2985596191': '2024-09-21T18:44:56.985620+0000', '172.21.15.83:0/2410768464': '2024-09-21T18:50:14.729526+0000', '172.21.15.83:0/1868694097': '2024-09-21T18:50:14.729526+0000', '172.21.15.83:0/3777331122': '2024-09-21T18:44:32.434180+0000', '172.21.15.83:0/2161728215': '2024-09-21T18:50:14.729526+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-20T18:50:53.187 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T18:50:53.206 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:50:53.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:53.207 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:50:53.208 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:50:53.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:52 smithi142 ceph-mon[32508]: osdmap e86: 8 total, 8 up, 8 in 2024-09-20T18:50:53.308 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:50:53.329 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:50:53.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:53.330 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:50:53.431 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:50:53.453 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:53.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:53.454 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:50:53.554 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:50:53.576 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:53.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:53.577 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:50:53.588 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:50:53.678 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:50:53.699 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:53.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:53.701 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:50:53.801 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:50:53.825 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:50:53.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:53.872 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:50:53.899 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T18:50:53.973 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:50:53.995 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:53.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:53.996 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:50:54.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:53 smithi083 ceph-mon[29421]: pgmap v34: 148 pgs: 148 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T18:50:54.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:53 smithi083 ceph-mon[36260]: pgmap v34: 148 pgs: 148 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T18:50:54.072 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:50:54.096 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:50:54.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:54.118 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:54.118 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:50:54.219 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:50:54.241 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:54.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:54.242 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:50:54.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:53 smithi142 ceph-mon[32508]: pgmap v34: 148 pgs: 148 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T18:50:54.343 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:50:54.344 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:50:54.365 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:54.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:54.366 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:50:54.467 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:50:54.488 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:54.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:54.489 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:50:54.590 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:50:54.612 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:54.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:54.613 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:50:54.713 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:50:54.735 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:50:54.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:54.736 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:50:54.837 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:50:54.859 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:54.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:54.862 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:50:54.890 INFO:teuthology.orchestra.run.smithi083.stderr:enabled application 'rados' on pool 'unique_pool_0' 2024-09-20T18:50:54.962 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:50:54.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:54.985 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:54.985 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:50:55.039 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:54 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/768407393' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-20T18:50:55.039 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:54 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-20T18:50:55.040 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:54 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/768407393' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-20T18:50:55.040 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:54 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-20T18:50:55.086 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:50:55.107 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:55.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:55.108 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:50:55.209 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:50:55.230 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:55.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:55.232 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:50:55.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:54 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/768407393' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-20T18:50:55.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:54 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-20T18:50:55.332 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:50:55.354 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:50:55.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:55.355 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:50:55.455 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:50:55.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:55.478 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:55.478 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:50:55.579 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:50:55.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:55.602 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:55.602 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:50:55.703 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:50:55.725 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:55.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:55.726 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:50:55.826 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:50:55.848 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:55.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:55.849 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:50:55.887 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:55 smithi083 podman[69983]: 2024-09-20 18:50:55.789289406 +0000 UTC m=+3.422309598 container create 7f83f79ccb492f2ab13d53dac42b62bae921065d3f46d47bf4d42aaa44eee1ee (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-20T18:50:55.950 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:50:55.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:55.973 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:50:55.973 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:50:56.074 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:50:56.095 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:56.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:56.096 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:50:56.197 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:50:56.218 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:50:56.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:56.219 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:50:56.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:55 smithi142 ceph-mon[32508]: pgmap v35: 148 pgs: 148 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 862 B/s rd, 0 op/s 2024-09-20T18:50:56.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:55 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-20T18:50:56.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:55 smithi142 ceph-mon[32508]: osdmap e87: 8 total, 8 up, 8 in 2024-09-20T18:50:56.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:55 smithi142 ceph-mon[32508]: osdmap e88: 8 total, 8 up, 8 in 2024-09-20T18:50:56.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:55 smithi083 ceph-mon[29421]: pgmap v35: 148 pgs: 148 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 862 B/s rd, 0 op/s 2024-09-20T18:50:56.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:55 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-20T18:50:56.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:55 smithi083 ceph-mon[29421]: osdmap e87: 8 total, 8 up, 8 in 2024-09-20T18:50:56.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:55 smithi083 ceph-mon[29421]: osdmap e88: 8 total, 8 up, 8 in 2024-09-20T18:50:56.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:55 smithi083 ceph-mon[36260]: pgmap v35: 148 pgs: 148 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 862 B/s rd, 0 op/s 2024-09-20T18:50:56.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:55 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-20T18:50:56.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:55 smithi083 ceph-mon[36260]: osdmap e87: 8 total, 8 up, 8 in 2024-09-20T18:50:56.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:55 smithi083 ceph-mon[36260]: osdmap e88: 8 total, 8 up, 8 in 2024-09-20T18:50:56.320 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:50:56.342 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:56.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:56.343 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:50:56.443 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:50:56.465 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:50:56.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:56.466 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:50:56.567 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:50:56.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:56.590 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:56.590 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:50:56.691 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:50:56.712 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:56.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:56.713 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:50:56.774 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:50:56 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:50:56.814 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:50:56.835 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:56.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:56.836 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:50:56.894 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-20T18:50:56.937 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:50:56.958 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:56.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:56.960 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:50:57.003 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1888785152' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:50:57.003 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:50:57.004 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1888785152' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:50:57.004 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:50:57.004 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 podman[69983]: 2024-09-20 18:50:56.790238295 +0000 UTC m=+4.423258496 container init 7f83f79ccb492f2ab13d53dac42b62bae921065d3f46d47bf4d42aaa44eee1ee (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-20T18:50:57.004 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 podman[69983]: 2024-09-20 18:50:56.793528618 +0000 UTC m=+4.426548813 container start 7f83f79ccb492f2ab13d53dac42b62bae921065d3f46d47bf4d42aaa44eee1ee (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-20T18:50:57.004 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.796Z caller=node_exporter.go:192 level=info msg="Starting node_exporter" version="(version=1.7.0, branch=HEAD, revision=7333465abf9efba81876303bb57e6fadb946041b)" 2024-09-20T18:50:57.004 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.796Z caller=node_exporter.go:193 level=info msg="Build context" build_context="(go=go1.21.4, platform=linux/amd64, user=root@35918982f6d8, date=20231112-23:53:35, tags=netgo osusergo static_build)" 2024-09-20T18:50:57.004 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.797Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-20T18:50:57.005 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.797Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-20T18:50:57.005 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.797Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-20T18:50:57.005 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.797Z caller=diskstats_linux.go:265 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-20T18:50:57.005 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.798Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-20T18:50:57.005 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.798Z caller=node_exporter.go:117 level=info collector=arp 2024-09-20T18:50:57.005 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.798Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-20T18:50:57.005 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.798Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-20T18:50:57.005 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.798Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-20T18:50:57.005 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.798Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-20T18:50:57.005 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.798Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-20T18:50:57.005 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.798Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-20T18:50:57.005 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.798Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-20T18:50:57.005 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.798Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-20T18:50:57.005 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.798Z caller=node_exporter.go:117 level=info collector=edac 2024-09-20T18:50:57.006 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.798Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-20T18:50:57.006 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.798Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-20T18:50:57.006 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.798Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-20T18:50:57.006 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.798Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-20T18:50:57.006 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.798Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-20T18:50:57.006 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.798Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-20T18:50:57.006 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.798Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-20T18:50:57.006 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.798Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-20T18:50:57.006 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.798Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-20T18:50:57.006 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.798Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-20T18:50:57.007 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.798Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-20T18:50:57.007 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.798Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-20T18:50:57.007 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.798Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-20T18:50:57.007 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.798Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-20T18:50:57.007 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.798Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-20T18:50:57.007 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.798Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-20T18:50:57.007 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.798Z caller=node_exporter.go:117 level=info collector=os 2024-09-20T18:50:57.007 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.798Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-20T18:50:57.007 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.798Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-20T18:50:57.007 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.798Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-20T18:50:57.007 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.798Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-20T18:50:57.008 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.798Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-20T18:50:57.008 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.798Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-20T18:50:57.008 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.798Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-20T18:50:57.008 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.798Z caller=node_exporter.go:117 level=info collector=stat 2024-09-20T18:50:57.008 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.798Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-20T18:50:57.008 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.798Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-20T18:50:57.008 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.798Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-20T18:50:57.008 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.798Z caller=node_exporter.go:117 level=info collector=time 2024-09-20T18:50:57.008 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.798Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-20T18:50:57.008 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.798Z caller=node_exporter.go:117 level=info collector=uname 2024-09-20T18:50:57.008 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.798Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-20T18:50:57.008 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.798Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-20T18:50:57.008 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.798Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-20T18:50:57.008 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.799Z caller=tls_config.go:274 level=info msg="Listening on" address=[::]:9100 2024-09-20T18:50:57.009 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a[70523]: ts=2024-09-20T18:50:56.799Z caller=tls_config.go:277 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-20T18:50:57.009 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 bash[69983]: 7f83f79ccb492f2ab13d53dac42b62bae921065d3f46d47bf4d42aaa44eee1ee 2024-09-20T18:50:57.009 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 18:50:56 smithi083 systemd[1]: Started Ceph node-exporter.a for 1ed5a50a-7780-11ef-baf5-efdc52797490. 2024-09-20T18:50:57.060 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:50:57.081 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:57.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:57.082 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:50:57.183 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:50:57.205 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:57.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:57.206 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:50:57.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:56 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1888785152' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:50:57.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:56 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:50:57.280 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:50:57.282 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-20T18:50:57.307 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:50:57.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:57.329 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:57.330 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:50:57.431 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:50:57.431 DEBUG:teuthology.orchestra.run.smithi083:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_0 bench 90 write' 2024-09-20T18:50:57.435 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-20T18:50:57.435 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-20T18:50:57.453 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:57.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:57.454 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:50:57.512 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:hints = 1 2024-09-20T18:50:57.512 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-20T18:50:57.513 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Object prefix: benchmark_data_smithi083_70643 2024-09-20T18:50:57.513 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-20T18:50:57.513 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 0 0 0 0 0 0 - 0 2024-09-20T18:50:57.555 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:50:57.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:57.592 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:50:57.592 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:50:57.693 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:50:57.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:57.734 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:50:57.734 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:50:57.836 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:50:57.870 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:57.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:57.871 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:50:57.972 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:50:58.010 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:50:58.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:58.011 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:50:58.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:57 smithi083 ceph-mon[29421]: pgmap v38: 148 pgs: 148 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 864 B/s rd, 0 op/s 2024-09-20T18:50:58.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:57 smithi083 ceph-mon[29421]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-20T18:50:58.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:57 smithi083 ceph-mon[29421]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:50:58.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:57 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1888785152' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T18:50:58.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:57 smithi083 ceph-mon[29421]: osdmap e89: 8 total, 8 up, 8 in 2024-09-20T18:50:58.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:57 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:58.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:57 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:58.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:57 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:58.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:57 smithi083 ceph-mon[29421]: Deploying daemon node-exporter.b on smithi142 2024-09-20T18:50:58.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:57 smithi083 ceph-mon[29421]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:50:58.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:57 smithi083 ceph-mon[36260]: pgmap v38: 148 pgs: 148 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 864 B/s rd, 0 op/s 2024-09-20T18:50:58.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:57 smithi083 ceph-mon[36260]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-20T18:50:58.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:57 smithi083 ceph-mon[36260]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:50:58.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:57 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1888785152' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T18:50:58.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:57 smithi083 ceph-mon[36260]: osdmap e89: 8 total, 8 up, 8 in 2024-09-20T18:50:58.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:57 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:58.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:57 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:58.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:57 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:58.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:57 smithi083 ceph-mon[36260]: Deploying daemon node-exporter.b on smithi142 2024-09-20T18:50:58.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:57 smithi083 ceph-mon[36260]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:50:58.067 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T18:50:58.067 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:50:58.080 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:57 smithi142 ceph-mon[32508]: pgmap v38: 148 pgs: 148 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 864 B/s rd, 0 op/s 2024-09-20T18:50:58.080 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:57 smithi142 ceph-mon[32508]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-20T18:50:58.080 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:57 smithi142 ceph-mon[32508]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:50:58.080 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:57 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1888785152' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T18:50:58.080 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:57 smithi142 ceph-mon[32508]: osdmap e89: 8 total, 8 up, 8 in 2024-09-20T18:50:58.080 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:57 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:58.080 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:57 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:58.080 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:57 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:50:58.081 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:57 smithi142 ceph-mon[32508]: Deploying daemon node-exporter.b on smithi142 2024-09-20T18:50:58.081 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:57 smithi142 ceph-mon[32508]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:50:58.112 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:50:58.151 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:50:58.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:58.152 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:50:58.253 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:50:58.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:58.290 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:50:58.291 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:50:58.391 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:50:58.428 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:50:58.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:58.429 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:50:58.469 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:50:58 smithi142 bash[61969]: Getting image source signatures 2024-09-20T18:50:58.469 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:50:58 smithi142 bash[61969]: Copying blob sha256:324153f2810a9927fcce320af9e4e291e0b6e805cbdd1f338386c756b9defa24 2024-09-20T18:50:58.469 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:50:58 smithi142 bash[61969]: Copying blob sha256:2abcce694348cd2c949c0e98a7400ebdfd8341021bcf6b541bc72033ce982510 2024-09-20T18:50:58.469 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:50:58 smithi142 bash[61969]: Copying blob sha256:455fd88e5221bc1e278ef2d059cd70e4df99a24e5af050ede621534276f6cf9a 2024-09-20T18:50:58.513 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 1 15 3481 3466 216.613 216.625 0.00347087 0.00459902 2024-09-20T18:50:58.530 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:50:58.569 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:50:58.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:58.570 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:50:58.670 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:50:58.707 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:50:58.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:58.708 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:50:58.809 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:50:58.845 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:58.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:58.846 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:50:58.900 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:50:58.904 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-09-20T18:50:56.883255+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":913,"num_read_kb":770,"num_write":549,"num_write_kb":1673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":590588,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":552,"ondisk_log_size":552,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":245864,"kb_used_data":8832,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":749715352,"statfs":{"total":767960285184,"available":767708520448,"internally_reserved":0,"allocated":9043968,"data_stored":4822877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.028234"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":68,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.565115+0000","last_change":"2024-09-20T18:49:45.943560+0000","last_active":"2024-09-20T18:50:55.565115+0000","last_peered":"2024-09-20T18:50:55.565115+0000","last_clean":"2024-09-20T18:50:55.565115+0000","last_became_active":"2024-09-20T18:49:45.943201+0000","last_became_peered":"2024-09-20T18:49:45.943201+0000","last_unstale":"2024-09-20T18:50:55.565115+0000","last_undegraded":"2024-09-20T18:50:55.565115+0000","last_fullsized":"2024-09-20T18:50:55.565115+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:11.286122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":81,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.565767+0000","last_change":"2024-09-20T18:49:47.947057+0000","last_active":"2024-09-20T18:50:55.565767+0000","last_peered":"2024-09-20T18:50:55.565767+0000","last_clean":"2024-09-20T18:50:55.565767+0000","last_became_active":"2024-09-20T18:49:47.946161+0000","last_became_peered":"2024-09-20T18:49:47.946161+0000","last_unstale":"2024-09-20T18:50:55.565767+0000","last_undegraded":"2024-09-20T18:50:55.565767+0000","last_fullsized":"2024-09-20T18:50:55.565767+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.579926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":59,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:54.871910+0000","last_change":"2024-09-20T18:49:49.959797+0000","last_active":"2024-09-20T18:50:54.871910+0000","last_peered":"2024-09-20T18:50:54.871910+0000","last_clean":"2024-09-20T18:50:54.871910+0000","last_became_active":"2024-09-20T18:49:49.959514+0000","last_became_peered":"2024-09-20T18:49:49.959514+0000","last_unstale":"2024-09-20T18:50:54.871910+0000","last_undegraded":"2024-09-20T18:50:54.871910+0000","last_fullsized":"2024-09-20T18:50:54.871910+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":57,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.565831+0000","last_change":"2024-09-20T18:49:51.969687+0000","last_active":"2024-09-20T18:50:55.565831+0000","last_peered":"2024-09-20T18:50:55.565831+0000","last_clean":"2024-09-20T18:50:55.565831+0000","last_became_active":"2024-09-20T18:49:51.969421+0000","last_became_peered":"2024-09-20T18:49:51.969421+0000","last_unstale":"2024-09-20T18:50:55.565831+0000","last_undegraded":"2024-09-20T18:50:55.565831+0000","last_fullsized":"2024-09-20T18:50:55.565831+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.534291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":55,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:54.871943+0000","last_change":"2024-09-20T18:49:52.142168+0000","last_active":"2024-09-20T18:50:54.871943+0000","last_peered":"2024-09-20T18:50:54.871943+0000","last_clean":"2024-09-20T18:50:54.871943+0000","last_became_active":"2024-09-20T18:49:52.141967+0000","last_became_peered":"2024-09-20T18:49:52.141967+0000","last_unstale":"2024-09-20T18:50:54.871943+0000","last_undegraded":"2024-09-20T18:50:54.871943+0000","last_fullsized":"2024-09-20T18:50:54.871943+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.114636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":67,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:54.872760+0000","last_change":"2024-09-20T18:49:45.943889+0000","last_active":"2024-09-20T18:50:54.872760+0000","last_peered":"2024-09-20T18:50:54.872760+0000","last_clean":"2024-09-20T18:50:54.872760+0000","last_became_active":"2024-09-20T18:49:45.943607+0000","last_became_peered":"2024-09-20T18:49:45.943607+0000","last_unstale":"2024-09-20T18:50:54.872760+0000","last_undegraded":"2024-09-20T18:50:54.872760+0000","last_fullsized":"2024-09-20T18:50:54.872760+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.033360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":88,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:54.871766+0000","last_change":"2024-09-20T18:49:47.952638+0000","last_active":"2024-09-20T18:50:54.871766+0000","last_peered":"2024-09-20T18:50:54.871766+0000","last_clean":"2024-09-20T18:50:54.871766+0000","last_became_active":"2024-09-20T18:49:47.952150+0000","last_became_peered":"2024-09-20T18:49:47.952150+0000","last_unstale":"2024-09-20T18:50:54.871766+0000","last_undegraded":"2024-09-20T18:50:54.871766+0000","last_fullsized":"2024-09-20T18:50:54.871766+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:02.798315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":61,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.565687+0000","last_change":"2024-09-20T18:49:49.956717+0000","last_active":"2024-09-20T18:50:55.565687+0000","last_peered":"2024-09-20T18:50:55.565687+0000","last_clean":"2024-09-20T18:50:55.565687+0000","last_became_active":"2024-09-20T18:49:49.956445+0000","last_became_peered":"2024-09-20T18:49:49.956445+0000","last_unstale":"2024-09-20T18:50:55.565687+0000","last_undegraded":"2024-09-20T18:50:55.565687+0000","last_fullsized":"2024-09-20T18:50:55.565687+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"59'1","reported_seq":72,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.534626+0000","last_change":"2024-09-20T18:49:45.946450+0000","last_active":"2024-09-20T18:50:55.534626+0000","last_peered":"2024-09-20T18:50:55.534626+0000","last_clean":"2024-09-20T18:50:55.534626+0000","last_became_active":"2024-09-20T18:49:45.946171+0000","last_became_peered":"2024-09-20T18:49:45.946171+0000","last_unstale":"2024-09-20T18:50:55.534626+0000","last_undegraded":"2024-09-20T18:50:55.534626+0000","last_fullsized":"2024-09-20T18:50:55.534626+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:11.299334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":81,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.565972+0000","last_change":"2024-09-20T18:49:47.947755+0000","last_active":"2024-09-20T18:50:55.565972+0000","last_peered":"2024-09-20T18:50:55.565972+0000","last_clean":"2024-09-20T18:50:55.565972+0000","last_became_active":"2024-09-20T18:49:47.947448+0000","last_became_peered":"2024-09-20T18:49:47.947448+0000","last_unstale":"2024-09-20T18:50:55.565972+0000","last_undegraded":"2024-09-20T18:50:55.565972+0000","last_fullsized":"2024-09-20T18:50:55.565972+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.588680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":58,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.533325+0000","last_change":"2024-09-20T18:49:49.961448+0000","last_active":"2024-09-20T18:50:55.533325+0000","last_peered":"2024-09-20T18:50:55.533325+0000","last_clean":"2024-09-20T18:50:55.533325+0000","last_became_active":"2024-09-20T18:49:49.961188+0000","last_became_peered":"2024-09-20T18:49:49.961188+0000","last_unstale":"2024-09-20T18:50:55.533325+0000","last_undegraded":"2024-09-20T18:50:55.533325+0000","last_fullsized":"2024-09-20T18:50:55.533325+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:12.456737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":56,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.564135+0000","last_change":"2024-09-20T18:49:51.966957+0000","last_active":"2024-09-20T18:50:55.564135+0000","last_peered":"2024-09-20T18:50:55.564135+0000","last_clean":"2024-09-20T18:50:55.564135+0000","last_became_active":"2024-09-20T18:49:51.966715+0000","last_became_peered":"2024-09-20T18:49:51.966715+0000","last_unstale":"2024-09-20T18:50:55.564135+0000","last_undegraded":"2024-09-20T18:50:55.564135+0000","last_fullsized":"2024-09-20T18:50:55.564135+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:54.991522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":66,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.532542+0000","last_change":"2024-09-20T18:49:45.945004+0000","last_active":"2024-09-20T18:50:55.532542+0000","last_peered":"2024-09-20T18:50:55.532542+0000","last_clean":"2024-09-20T18:50:55.532542+0000","last_became_active":"2024-09-20T18:49:45.944703+0000","last_became_peered":"2024-09-20T18:49:45.944703+0000","last_unstale":"2024-09-20T18:50:55.532542+0000","last_undegraded":"2024-09-20T18:50:55.532542+0000","last_fullsized":"2024-09-20T18:50:55.532542+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":88,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.566082+0000","last_change":"2024-09-20T18:49:58.999374+0000","last_active":"2024-09-20T18:50:55.566082+0000","last_peered":"2024-09-20T18:50:55.566082+0000","last_clean":"2024-09-20T18:50:55.566082+0000","last_became_active":"2024-09-20T18:49:58.995638+0000","last_became_peered":"2024-09-20T18:49:58.995638+0000","last_unstale":"2024-09-20T18:50:55.566082+0000","last_undegraded":"2024-09-20T18:50:55.566082+0000","last_fullsized":"2024-09-20T18:50:55.566082+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:05.131889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":58,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.532826+0000","last_change":"2024-09-20T18:49:49.960001+0000","last_active":"2024-09-20T18:50:55.532826+0000","last_peered":"2024-09-20T18:50:55.532826+0000","last_clean":"2024-09-20T18:50:55.532826+0000","last_became_active":"2024-09-20T18:49:49.959696+0000","last_became_peered":"2024-09-20T18:49:49.959696+0000","last_unstale":"2024-09-20T18:50:55.532826+0000","last_undegraded":"2024-09-20T18:50:55.532826+0000","last_fullsized":"2024-09-20T18:50:55.532826+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.740255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":54,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.532428+0000","last_change":"2024-09-20T18:49:51.970933+0000","last_active":"2024-09-20T18:50:55.532428+0000","last_peered":"2024-09-20T18:50:55.532428+0000","last_clean":"2024-09-20T18:50:55.532428+0000","last_became_active":"2024-09-20T18:49:51.970663+0000","last_became_peered":"2024-09-20T18:49:51.970663+0000","last_unstale":"2024-09-20T18:50:55.532428+0000","last_undegraded":"2024-09-20T18:50:55.532428+0000","last_fullsized":"2024-09-20T18:50:55.532428+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:07.680885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":57,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.566271+0000","last_change":"2024-09-20T18:49:52.141482+0000","last_active":"2024-09-20T18:50:55.566271+0000","last_peered":"2024-09-20T18:50:55.566271+0000","last_clean":"2024-09-20T18:50:55.566271+0000","last_became_active":"2024-09-20T18:49:52.141170+0000","last_became_peered":"2024-09-20T18:49:52.141170+0000","last_unstale":"2024-09-20T18:50:55.566271+0000","last_undegraded":"2024-09-20T18:50:55.566271+0000","last_fullsized":"2024-09-20T18:50:55.566271+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:15.351627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":68,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.564991+0000","last_change":"2024-09-20T18:49:45.950714+0000","last_active":"2024-09-20T18:50:55.564991+0000","last_peered":"2024-09-20T18:50:55.564991+0000","last_clean":"2024-09-20T18:50:55.564991+0000","last_became_active":"2024-09-20T18:49:45.950480+0000","last_became_peered":"2024-09-20T18:49:45.950480+0000","last_unstale":"2024-09-20T18:50:55.564991+0000","last_undegraded":"2024-09-20T18:50:55.564991+0000","last_fullsized":"2024-09-20T18:50:55.564991+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:07.919860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":91,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.563633+0000","last_change":"2024-09-20T18:49:47.951228+0000","last_active":"2024-09-20T18:50:55.563633+0000","last_peered":"2024-09-20T18:50:55.563633+0000","last_clean":"2024-09-20T18:50:55.563633+0000","last_became_active":"2024-09-20T18:49:47.950939+0000","last_became_peered":"2024-09-20T18:49:47.950939+0000","last_unstale":"2024-09-20T18:50:55.563633+0000","last_undegraded":"2024-09-20T18:50:55.563633+0000","last_fullsized":"2024-09-20T18:50:55.563633+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:08.025260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":59,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:54.873110+0000","last_change":"2024-09-20T18:49:49.958867+0000","last_active":"2024-09-20T18:50:54.873110+0000","last_peered":"2024-09-20T18:50:54.873110+0000","last_clean":"2024-09-20T18:50:54.873110+0000","last_became_active":"2024-09-20T18:49:49.958518+0000","last_became_peered":"2024-09-20T18:49:49.958518+0000","last_unstale":"2024-09-20T18:50:54.873110+0000","last_undegraded":"2024-09-20T18:50:54.873110+0000","last_fullsized":"2024-09-20T18:50:54.873110+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:55.540498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":55,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:54.873064+0000","last_change":"2024-09-20T18:49:52.140362+0000","last_active":"2024-09-20T18:50:54.873064+0000","last_peered":"2024-09-20T18:50:54.873064+0000","last_clean":"2024-09-20T18:50:54.873064+0000","last_became_active":"2024-09-20T18:49:52.140040+0000","last_became_peered":"2024-09-20T18:49:52.140040+0000","last_unstale":"2024-09-20T18:50:54.873064+0000","last_undegraded":"2024-09-20T18:50:54.873064+0000","last_fullsized":"2024-09-20T18:50:54.873064+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:53.831495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"59'1","reported_seq":75,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.566088+0000","last_change":"2024-09-20T18:49:45.942193+0000","last_active":"2024-09-20T18:50:55.566088+0000","last_peered":"2024-09-20T18:50:55.566088+0000","last_clean":"2024-09-20T18:50:55.566088+0000","last_became_active":"2024-09-20T18:49:45.940100+0000","last_became_peered":"2024-09-20T18:49:45.940100+0000","last_unstale":"2024-09-20T18:50:55.566088+0000","last_undegraded":"2024-09-20T18:50:55.566088+0000","last_fullsized":"2024-09-20T18:50:55.566088+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:53.263392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":86,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:54.871992+0000","last_change":"2024-09-20T18:49:47.951215+0000","last_active":"2024-09-20T18:50:54.871992+0000","last_peered":"2024-09-20T18:50:54.871992+0000","last_clean":"2024-09-20T18:50:54.871992+0000","last_became_active":"2024-09-20T18:49:47.950909+0000","last_became_peered":"2024-09-20T18:49:47.950909+0000","last_unstale":"2024-09-20T18:50:54.871992+0000","last_undegraded":"2024-09-20T18:50:54.871992+0000","last_fullsized":"2024-09-20T18:50:54.871992+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:52.850095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":61,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.566100+0000","last_change":"2024-09-20T18:49:49.959365+0000","last_active":"2024-09-20T18:50:55.566100+0000","last_peered":"2024-09-20T18:50:55.566100+0000","last_clean":"2024-09-20T18:50:55.566100+0000","last_became_active":"2024-09-20T18:49:49.959102+0000","last_became_peered":"2024-09-20T18:49:49.959102+0000","last_unstale":"2024-09-20T18:50:55.566100+0000","last_undegraded":"2024-09-20T18:50:55.566100+0000","last_fullsized":"2024-09-20T18:50:55.566100+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:04.872284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":55,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.533680+0000","last_change":"2024-09-20T18:49:51.970469+0000","last_active":"2024-09-20T18:50:55.533680+0000","last_peered":"2024-09-20T18:50:55.533680+0000","last_clean":"2024-09-20T18:50:55.533680+0000","last_became_active":"2024-09-20T18:49:51.970058+0000","last_became_peered":"2024-09-20T18:49:51.970058+0000","last_unstale":"2024-09-20T18:50:55.533680+0000","last_undegraded":"2024-09-20T18:50:55.533680+0000","last_fullsized":"2024-09-20T18:50:55.533680+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:59.400214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":68,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:54.872025+0000","last_change":"2024-09-20T18:49:45.933354+0000","last_active":"2024-09-20T18:50:54.872025+0000","last_peered":"2024-09-20T18:50:54.872025+0000","last_clean":"2024-09-20T18:50:54.872025+0000","last_became_active":"2024-09-20T18:49:45.933078+0000","last_became_peered":"2024-09-20T18:49:45.933078+0000","last_unstale":"2024-09-20T18:50:54.872025+0000","last_undegraded":"2024-09-20T18:50:54.872025+0000","last_fullsized":"2024-09-20T18:50:54.872025+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:07.425698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":79,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.564994+0000","last_change":"2024-09-20T18:49:48.501151+0000","last_active":"2024-09-20T18:50:55.564994+0000","last_peered":"2024-09-20T18:50:55.564994+0000","last_clean":"2024-09-20T18:50:55.564994+0000","last_became_active":"2024-09-20T18:49:48.500809+0000","last_became_peered":"2024-09-20T18:49:48.500809+0000","last_unstale":"2024-09-20T18:50:55.564994+0000","last_undegraded":"2024-09-20T18:50:55.564994+0000","last_fullsized":"2024-09-20T18:50:55.564994+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:07.803937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":83,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.534240+0000","last_change":"2024-09-20T18:49:49.955074+0000","last_active":"2024-09-20T18:50:55.534240+0000","last_peered":"2024-09-20T18:50:55.534240+0000","last_clean":"2024-09-20T18:50:55.534240+0000","last_became_active":"2024-09-20T18:49:49.954772+0000","last_became_peered":"2024-09-20T18:49:49.954772+0000","last_unstale":"2024-09-20T18:50:55.534240+0000","last_undegraded":"2024-09-20T18:50:55.534240+0000","last_fullsized":"2024-09-20T18:50:55.534240+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.533287+0000","last_change":"2024-09-20T18:50:49.844070+0000","last_active":"2024-09-20T18:50:55.533287+0000","last_peered":"2024-09-20T18:50:55.533287+0000","last_clean":"2024-09-20T18:50:55.533287+0000","last_became_active":"2024-09-20T18:50:49.843744+0000","last_became_peered":"2024-09-20T18:50:49.843744+0000","last_unstale":"2024-09-20T18:50:55.533287+0000","last_undegraded":"2024-09-20T18:50:55.533287+0000","last_fullsized":"2024-09-20T18:50:55.533287+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.819660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:57.802543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":66,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.563607+0000","last_change":"2024-09-20T18:49:59.004173+0000","last_active":"2024-09-20T18:50:55.563607+0000","last_peered":"2024-09-20T18:50:55.563607+0000","last_clean":"2024-09-20T18:50:55.563607+0000","last_became_active":"2024-09-20T18:49:58.992302+0000","last_became_peered":"2024-09-20T18:49:58.992302+0000","last_unstale":"2024-09-20T18:50:55.563607+0000","last_undegraded":"2024-09-20T18:50:55.563607+0000","last_fullsized":"2024-09-20T18:50:55.563607+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:14.502406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":67,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:54.871333+0000","last_change":"2024-09-20T18:49:45.936413+0000","last_active":"2024-09-20T18:50:54.871333+0000","last_peered":"2024-09-20T18:50:54.871333+0000","last_clean":"2024-09-20T18:50:54.871333+0000","last_became_active":"2024-09-20T18:49:45.936104+0000","last_became_peered":"2024-09-20T18:49:45.936104+0000","last_unstale":"2024-09-20T18:50:54.871333+0000","last_undegraded":"2024-09-20T18:50:54.871333+0000","last_fullsized":"2024-09-20T18:50:54.871333+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:01.319743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":86,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.566365+0000","last_change":"2024-09-20T18:49:49.960978+0000","last_active":"2024-09-20T18:50:55.566365+0000","last_peered":"2024-09-20T18:50:55.566365+0000","last_clean":"2024-09-20T18:50:55.566365+0000","last_became_active":"2024-09-20T18:49:49.960697+0000","last_became_peered":"2024-09-20T18:49:49.960697+0000","last_unstale":"2024-09-20T18:50:55.566365+0000","last_undegraded":"2024-09-20T18:50:55.566365+0000","last_fullsized":"2024-09-20T18:50:55.566365+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":54,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.533659+0000","last_change":"2024-09-20T18:49:51.971713+0000","last_active":"2024-09-20T18:50:55.533659+0000","last_peered":"2024-09-20T18:50:55.533659+0000","last_clean":"2024-09-20T18:50:55.533659+0000","last_became_active":"2024-09-20T18:49:51.971399+0000","last_became_peered":"2024-09-20T18:49:51.971399+0000","last_unstale":"2024-09-20T18:50:55.533659+0000","last_undegraded":"2024-09-20T18:50:55.533659+0000","last_fullsized":"2024-09-20T18:50:55.533659+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:18.352126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:54.871798+0000","last_change":"2024-09-20T18:50:49.841111+0000","last_active":"2024-09-20T18:50:54.871798+0000","last_peered":"2024-09-20T18:50:54.871798+0000","last_clean":"2024-09-20T18:50:54.871798+0000","last_became_active":"2024-09-20T18:50:49.840819+0000","last_became_peered":"2024-09-20T18:50:49.840819+0000","last_unstale":"2024-09-20T18:50:54.871798+0000","last_undegraded":"2024-09-20T18:50:54.871798+0000","last_fullsized":"2024-09-20T18:50:54.871798+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.819660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:18.306766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":95,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:54.871218+0000","last_change":"2024-09-20T18:49:48.499569+0000","last_active":"2024-09-20T18:50:54.871218+0000","last_peered":"2024-09-20T18:50:54.871218+0000","last_clean":"2024-09-20T18:50:54.871218+0000","last_became_active":"2024-09-20T18:49:48.499189+0000","last_became_peered":"2024-09-20T18:49:48.499189+0000","last_unstale":"2024-09-20T18:50:54.871218+0000","last_undegraded":"2024-09-20T18:50:54.871218+0000","last_fullsized":"2024-09-20T18:50:54.871218+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:16.224188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":67,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:54.873417+0000","last_change":"2024-09-20T18:49:45.940816+0000","last_active":"2024-09-20T18:50:54.873417+0000","last_peered":"2024-09-20T18:50:54.873417+0000","last_clean":"2024-09-20T18:50:54.873417+0000","last_became_active":"2024-09-20T18:49:45.940544+0000","last_became_peered":"2024-09-20T18:49:45.940544+0000","last_unstale":"2024-09-20T18:50:54.873417+0000","last_undegraded":"2024-09-20T18:50:54.873417+0000","last_fullsized":"2024-09-20T18:50:54.873417+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:50.841381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":61,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.565759+0000","last_change":"2024-09-20T18:49:49.963362+0000","last_active":"2024-09-20T18:50:55.565759+0000","last_peered":"2024-09-20T18:50:55.565759+0000","last_clean":"2024-09-20T18:50:55.565759+0000","last_became_active":"2024-09-20T18:49:49.963073+0000","last_became_peered":"2024-09-20T18:49:49.963073+0000","last_unstale":"2024-09-20T18:50:55.565759+0000","last_undegraded":"2024-09-20T18:50:55.565759+0000","last_fullsized":"2024-09-20T18:50:55.565759+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:01.468417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":57,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.565924+0000","last_change":"2024-09-20T18:49:51.969120+0000","last_active":"2024-09-20T18:50:55.565924+0000","last_peered":"2024-09-20T18:50:55.565924+0000","last_clean":"2024-09-20T18:50:55.565924+0000","last_became_active":"2024-09-20T18:49:51.968793+0000","last_became_peered":"2024-09-20T18:49:51.968793+0000","last_unstale":"2024-09-20T18:50:55.565924+0000","last_undegraded":"2024-09-20T18:50:55.565924+0000","last_fullsized":"2024-09-20T18:50:55.565924+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:57.615454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:54.871311+0000","last_change":"2024-09-20T18:50:49.842736+0000","last_active":"2024-09-20T18:50:54.871311+0000","last_peered":"2024-09-20T18:50:54.871311+0000","last_clean":"2024-09-20T18:50:54.871311+0000","last_became_active":"2024-09-20T18:50:49.842465+0000","last_became_peered":"2024-09-20T18:50:49.842465+0000","last_unstale":"2024-09-20T18:50:54.871311+0000","last_undegraded":"2024-09-20T18:50:54.871311+0000","last_fullsized":"2024-09-20T18:50:54.871311+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.819660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:10.093828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":86,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.565825+0000","last_change":"2024-09-20T18:49:47.942407+0000","last_active":"2024-09-20T18:50:55.565825+0000","last_peered":"2024-09-20T18:50:55.565825+0000","last_clean":"2024-09-20T18:50:55.565825+0000","last_became_active":"2024-09-20T18:49:47.941895+0000","last_became_peered":"2024-09-20T18:49:47.941895+0000","last_unstale":"2024-09-20T18:50:55.565825+0000","last_undegraded":"2024-09-20T18:50:55.565825+0000","last_fullsized":"2024-09-20T18:50:55.565825+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:08.413407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":69,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.565108+0000","last_change":"2024-09-20T18:49:45.940728+0000","last_active":"2024-09-20T18:50:55.565108+0000","last_peered":"2024-09-20T18:50:55.565108+0000","last_clean":"2024-09-20T18:50:55.565108+0000","last_became_active":"2024-09-20T18:49:45.940469+0000","last_became_peered":"2024-09-20T18:49:45.940469+0000","last_unstale":"2024-09-20T18:50:55.565108+0000","last_undegraded":"2024-09-20T18:50:55.565108+0000","last_fullsized":"2024-09-20T18:50:55.565108+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:05.840900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"66'8","reported_seq":81,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:54.871092+0000","last_change":"2024-09-20T18:49:49.956436+0000","last_active":"2024-09-20T18:50:54.871092+0000","last_peered":"2024-09-20T18:50:54.871092+0000","last_clean":"2024-09-20T18:50:54.871092+0000","last_became_active":"2024-09-20T18:49:49.956133+0000","last_became_peered":"2024-09-20T18:49:49.956133+0000","last_unstale":"2024-09-20T18:50:54.871092+0000","last_undegraded":"2024-09-20T18:50:54.871092+0000","last_fullsized":"2024-09-20T18:50:54.871092+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:03.649831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":54,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.533073+0000","last_change":"2024-09-20T18:49:52.142481+0000","last_active":"2024-09-20T18:50:55.533073+0000","last_peered":"2024-09-20T18:50:55.533073+0000","last_clean":"2024-09-20T18:50:55.533073+0000","last_became_active":"2024-09-20T18:49:52.142113+0000","last_became_peered":"2024-09-20T18:49:52.142113+0000","last_unstale":"2024-09-20T18:50:55.533073+0000","last_undegraded":"2024-09-20T18:50:55.533073+0000","last_fullsized":"2024-09-20T18:50:55.533073+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.140166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.565619+0000","last_change":"2024-09-20T18:50:49.845003+0000","last_active":"2024-09-20T18:50:55.565619+0000","last_peered":"2024-09-20T18:50:55.565619+0000","last_clean":"2024-09-20T18:50:55.565619+0000","last_became_active":"2024-09-20T18:50:49.844624+0000","last_became_peered":"2024-09-20T18:50:49.844624+0000","last_unstale":"2024-09-20T18:50:55.565619+0000","last_undegraded":"2024-09-20T18:50:55.565619+0000","last_fullsized":"2024-09-20T18:50:55.565619+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.819660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:50.608277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":78,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:54.872795+0000","last_change":"2024-09-20T18:49:47.949952+0000","last_active":"2024-09-20T18:50:54.872795+0000","last_peered":"2024-09-20T18:50:54.872795+0000","last_clean":"2024-09-20T18:50:54.872795+0000","last_became_active":"2024-09-20T18:49:47.949626+0000","last_became_peered":"2024-09-20T18:49:47.949626+0000","last_unstale":"2024-09-20T18:50:54.872795+0000","last_undegraded":"2024-09-20T18:50:54.872795+0000","last_fullsized":"2024-09-20T18:50:54.872795+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":66,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.534713+0000","last_change":"2024-09-20T18:49:45.942852+0000","last_active":"2024-09-20T18:50:55.534713+0000","last_peered":"2024-09-20T18:50:55.534713+0000","last_clean":"2024-09-20T18:50:55.534713+0000","last_became_active":"2024-09-20T18:49:45.942496+0000","last_became_peered":"2024-09-20T18:49:45.942496+0000","last_unstale":"2024-09-20T18:50:55.534713+0000","last_undegraded":"2024-09-20T18:50:55.534713+0000","last_fullsized":"2024-09-20T18:50:55.534713+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:55.917563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":82,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.564636+0000","last_change":"2024-09-20T18:49:49.959699+0000","last_active":"2024-09-20T18:50:55.564636+0000","last_peered":"2024-09-20T18:50:55.564636+0000","last_clean":"2024-09-20T18:50:55.564636+0000","last_became_active":"2024-09-20T18:49:49.959365+0000","last_became_peered":"2024-09-20T18:49:49.959365+0000","last_unstale":"2024-09-20T18:50:55.564636+0000","last_undegraded":"2024-09-20T18:50:55.564636+0000","last_fullsized":"2024-09-20T18:50:55.564636+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:01.818798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":56,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.564248+0000","last_change":"2024-09-20T18:49:51.965733+0000","last_active":"2024-09-20T18:50:55.564248+0000","last_peered":"2024-09-20T18:50:55.564248+0000","last_clean":"2024-09-20T18:50:55.564248+0000","last_became_active":"2024-09-20T18:49:51.964780+0000","last_became_peered":"2024-09-20T18:49:51.964780+0000","last_unstale":"2024-09-20T18:50:55.564248+0000","last_undegraded":"2024-09-20T18:50:55.564248+0000","last_fullsized":"2024-09-20T18:50:55.564248+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:11.275213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.533519+0000","last_change":"2024-09-20T18:50:49.841428+0000","last_active":"2024-09-20T18:50:55.533519+0000","last_peered":"2024-09-20T18:50:55.533519+0000","last_clean":"2024-09-20T18:50:55.533519+0000","last_became_active":"2024-09-20T18:50:49.841019+0000","last_became_peered":"2024-09-20T18:50:49.841019+0000","last_unstale":"2024-09-20T18:50:55.533519+0000","last_undegraded":"2024-09-20T18:50:55.533519+0000","last_fullsized":"2024-09-20T18:50:55.533519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.819660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:05.804622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":100,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.564720+0000","last_change":"2024-09-20T18:49:48.498141+0000","last_active":"2024-09-20T18:50:55.564720+0000","last_peered":"2024-09-20T18:50:55.564720+0000","last_clean":"2024-09-20T18:50:55.564720+0000","last_became_active":"2024-09-20T18:49:48.497736+0000","last_became_peered":"2024-09-20T18:49:48.497736+0000","last_unstale":"2024-09-20T18:50:55.564720+0000","last_undegraded":"2024-09-20T18:50:55.564720+0000","last_fullsized":"2024-09-20T18:50:55.564720+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.986657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":67,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:54.873444+0000","last_change":"2024-09-20T18:49:45.942589+0000","last_active":"2024-09-20T18:50:54.873444+0000","last_peered":"2024-09-20T18:50:54.873444+0000","last_clean":"2024-09-20T18:50:54.873444+0000","last_became_active":"2024-09-20T18:49:45.942253+0000","last_became_peered":"2024-09-20T18:49:45.942253+0000","last_unstale":"2024-09-20T18:50:54.873444+0000","last_undegraded":"2024-09-20T18:50:54.873444+0000","last_fullsized":"2024-09-20T18:50:54.873444+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.673189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":58,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.532776+0000","last_change":"2024-09-20T18:49:49.954594+0000","last_active":"2024-09-20T18:50:55.532776+0000","last_peered":"2024-09-20T18:50:55.532776+0000","last_clean":"2024-09-20T18:50:55.532776+0000","last_became_active":"2024-09-20T18:49:49.954299+0000","last_became_peered":"2024-09-20T18:49:49.954299+0000","last_unstale":"2024-09-20T18:50:55.532776+0000","last_undegraded":"2024-09-20T18:50:55.532776+0000","last_fullsized":"2024-09-20T18:50:55.532776+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":55,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:54.871818+0000","last_change":"2024-09-20T18:49:52.140276+0000","last_active":"2024-09-20T18:50:54.871818+0000","last_peered":"2024-09-20T18:50:54.871818+0000","last_clean":"2024-09-20T18:50:54.871818+0000","last_became_active":"2024-09-20T18:49:52.139981+0000","last_became_peered":"2024-09-20T18:49:52.139981+0000","last_unstale":"2024-09-20T18:50:54.871818+0000","last_undegraded":"2024-09-20T18:50:54.871818+0000","last_fullsized":"2024-09-20T18:50:54.871818+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.534114+0000","last_change":"2024-09-20T18:50:49.842749+0000","last_active":"2024-09-20T18:50:55.534114+0000","last_peered":"2024-09-20T18:50:55.534114+0000","last_clean":"2024-09-20T18:50:55.534114+0000","last_became_active":"2024-09-20T18:50:49.842473+0000","last_became_peered":"2024-09-20T18:50:49.842473+0000","last_unstale":"2024-09-20T18:50:55.534114+0000","last_undegraded":"2024-09-20T18:50:55.534114+0000","last_fullsized":"2024-09-20T18:50:55.534114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.819660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:57.122239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"66'28","reported_seq":125,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:54.873361+0000","last_change":"2024-09-20T18:49:47.955615+0000","last_active":"2024-09-20T18:50:54.873361+0000","last_peered":"2024-09-20T18:50:54.873361+0000","last_clean":"2024-09-20T18:50:54.873361+0000","last_became_active":"2024-09-20T18:49:47.955335+0000","last_became_peered":"2024-09-20T18:49:47.955335+0000","last_unstale":"2024-09-20T18:50:54.873361+0000","last_undegraded":"2024-09-20T18:50:54.873361+0000","last_fullsized":"2024-09-20T18:50:54.873361+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":69,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.566673+0000","last_change":"2024-09-20T18:49:45.945275+0000","last_active":"2024-09-20T18:50:55.566673+0000","last_peered":"2024-09-20T18:50:55.566673+0000","last_clean":"2024-09-20T18:50:55.566673+0000","last_became_active":"2024-09-20T18:49:45.944992+0000","last_became_peered":"2024-09-20T18:49:45.944992+0000","last_unstale":"2024-09-20T18:50:55.566673+0000","last_undegraded":"2024-09-20T18:50:55.566673+0000","last_fullsized":"2024-09-20T18:50:55.566673+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:50.184705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":84,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.533027+0000","last_change":"2024-09-20T18:50:48.711466+0000","last_active":"2024-09-20T18:50:55.533027+0000","last_peered":"2024-09-20T18:50:55.533027+0000","last_clean":"2024-09-20T18:50:55.533027+0000","last_became_active":"2024-09-20T18:49:44.936027+0000","last_became_peered":"2024-09-20T18:49:44.936027+0000","last_unstale":"2024-09-20T18:50:55.533027+0000","last_undegraded":"2024-09-20T18:50:55.533027+0000","last_fullsized":"2024-09-20T18:50:55.533027+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-20T18:50:48.711416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.711416+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:07.241951+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.0012781159999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":60,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.564867+0000","last_change":"2024-09-20T18:49:49.954281+0000","last_active":"2024-09-20T18:50:55.564867+0000","last_peered":"2024-09-20T18:50:55.564867+0000","last_clean":"2024-09-20T18:50:55.564867+0000","last_became_active":"2024-09-20T18:49:49.953971+0000","last_became_peered":"2024-09-20T18:49:49.953971+0000","last_unstale":"2024-09-20T18:50:55.564867+0000","last_undegraded":"2024-09-20T18:50:55.564867+0000","last_fullsized":"2024-09-20T18:50:55.564867+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:12.341509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":55,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:54.872535+0000","last_change":"2024-09-20T18:49:51.969702+0000","last_active":"2024-09-20T18:50:54.872535+0000","last_peered":"2024-09-20T18:50:54.872535+0000","last_clean":"2024-09-20T18:50:54.872535+0000","last_became_active":"2024-09-20T18:49:51.969400+0000","last_became_peered":"2024-09-20T18:49:51.969400+0000","last_unstale":"2024-09-20T18:50:54.872535+0000","last_undegraded":"2024-09-20T18:50:54.872535+0000","last_fullsized":"2024-09-20T18:50:54.872535+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.486248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.564642+0000","last_change":"2024-09-20T18:50:49.843571+0000","last_active":"2024-09-20T18:50:55.564642+0000","last_peered":"2024-09-20T18:50:55.564642+0000","last_clean":"2024-09-20T18:50:55.564642+0000","last_became_active":"2024-09-20T18:50:49.843321+0000","last_became_peered":"2024-09-20T18:50:49.843321+0000","last_unstale":"2024-09-20T18:50:55.564642+0000","last_undegraded":"2024-09-20T18:50:55.564642+0000","last_fullsized":"2024-09-20T18:50:55.564642+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.819660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:13.806482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":90,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:54.872680+0000","last_change":"2024-09-20T18:49:47.951311+0000","last_active":"2024-09-20T18:50:54.872680+0000","last_peered":"2024-09-20T18:50:54.872680+0000","last_clean":"2024-09-20T18:50:54.872680+0000","last_became_active":"2024-09-20T18:49:47.950904+0000","last_became_peered":"2024-09-20T18:49:47.950904+0000","last_unstale":"2024-09-20T18:50:54.872680+0000","last_undegraded":"2024-09-20T18:50:54.872680+0000","last_fullsized":"2024-09-20T18:50:54.872680+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:06.445493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":67,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:54.872537+0000","last_change":"2024-09-20T18:49:45.940667+0000","last_active":"2024-09-20T18:50:54.872537+0000","last_peered":"2024-09-20T18:50:54.872537+0000","last_clean":"2024-09-20T18:50:54.872537+0000","last_became_active":"2024-09-20T18:49:45.939884+0000","last_became_peered":"2024-09-20T18:49:45.939884+0000","last_unstale":"2024-09-20T18:50:54.872537+0000","last_undegraded":"2024-09-20T18:50:54.872537+0000","last_fullsized":"2024-09-20T18:50:54.872537+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":79,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.563117+0000","last_change":"2024-09-20T18:49:47.939858+0000","last_active":"2024-09-20T18:50:55.563117+0000","last_peered":"2024-09-20T18:50:55.563117+0000","last_clean":"2024-09-20T18:50:55.563117+0000","last_became_active":"2024-09-20T18:49:44.937135+0000","last_became_peered":"2024-09-20T18:49:44.937135+0000","last_unstale":"2024-09-20T18:50:55.563117+0000","last_undegraded":"2024-09-20T18:50:55.563117+0000","last_fullsized":"2024-09-20T18:50:55.563117+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:49:43.921002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:56.442658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095558800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":62,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.563781+0000","last_change":"2024-09-20T18:49:49.954880+0000","last_active":"2024-09-20T18:50:55.563781+0000","last_peered":"2024-09-20T18:50:55.563781+0000","last_clean":"2024-09-20T18:50:55.563781+0000","last_became_active":"2024-09-20T18:49:49.954424+0000","last_became_peered":"2024-09-20T18:49:49.954424+0000","last_unstale":"2024-09-20T18:50:55.563781+0000","last_undegraded":"2024-09-20T18:50:55.563781+0000","last_fullsized":"2024-09-20T18:50:55.563781+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":54,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.533865+0000","last_change":"2024-09-20T18:49:52.140366+0000","last_active":"2024-09-20T18:50:55.533865+0000","last_peered":"2024-09-20T18:50:55.533865+0000","last_clean":"2024-09-20T18:50:55.533865+0000","last_became_active":"2024-09-20T18:49:52.139789+0000","last_became_peered":"2024-09-20T18:49:52.139789+0000","last_unstale":"2024-09-20T18:50:55.533865+0000","last_undegraded":"2024-09-20T18:50:55.533865+0000","last_fullsized":"2024-09-20T18:50:55.533865+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:16.224499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.533758+0000","last_change":"2024-09-20T18:50:49.846470+0000","last_active":"2024-09-20T18:50:55.533758+0000","last_peered":"2024-09-20T18:50:55.533758+0000","last_clean":"2024-09-20T18:50:55.533758+0000","last_became_active":"2024-09-20T18:50:49.846124+0000","last_became_peered":"2024-09-20T18:50:49.846124+0000","last_unstale":"2024-09-20T18:50:55.533758+0000","last_undegraded":"2024-09-20T18:50:55.533758+0000","last_fullsized":"2024-09-20T18:50:55.533758+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.819660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:58.163086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":82,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.565017+0000","last_change":"2024-09-20T18:49:47.949547+0000","last_active":"2024-09-20T18:50:55.565017+0000","last_peered":"2024-09-20T18:50:55.565017+0000","last_clean":"2024-09-20T18:50:55.565017+0000","last_became_active":"2024-09-20T18:49:47.949230+0000","last_became_peered":"2024-09-20T18:49:47.949230+0000","last_unstale":"2024-09-20T18:50:55.565017+0000","last_undegraded":"2024-09-20T18:50:55.565017+0000","last_fullsized":"2024-09-20T18:50:55.565017+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:07.802093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":68,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.564467+0000","last_change":"2024-09-20T18:49:45.939454+0000","last_active":"2024-09-20T18:50:55.564467+0000","last_peered":"2024-09-20T18:50:55.564467+0000","last_clean":"2024-09-20T18:50:55.564467+0000","last_became_active":"2024-09-20T18:49:45.939197+0000","last_became_peered":"2024-09-20T18:49:45.939197+0000","last_unstale":"2024-09-20T18:50:55.564467+0000","last_undegraded":"2024-09-20T18:50:55.564467+0000","last_fullsized":"2024-09-20T18:50:55.564467+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:12.763563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":142,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.533245+0000","last_change":"2024-09-20T18:49:48.498983+0000","last_active":"2024-09-20T18:50:55.533245+0000","last_peered":"2024-09-20T18:50:55.533245+0000","last_clean":"2024-09-20T18:50:55.533245+0000","last_became_active":"2024-09-20T18:49:44.936819+0000","last_became_peered":"2024-09-20T18:49:44.936819+0000","last_unstale":"2024-09-20T18:50:55.533245+0000","last_undegraded":"2024-09-20T18:50:55.533245+0000","last_fullsized":"2024-09-20T18:50:55.533245+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:49:43.921002+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:58.186148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010996199999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":65,"num_read_kb":60,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":58,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.534356+0000","last_change":"2024-09-20T18:49:49.961346+0000","last_active":"2024-09-20T18:50:55.534356+0000","last_peered":"2024-09-20T18:50:55.534356+0000","last_clean":"2024-09-20T18:50:55.534356+0000","last_became_active":"2024-09-20T18:49:49.961066+0000","last_became_peered":"2024-09-20T18:49:49.961066+0000","last_unstale":"2024-09-20T18:50:55.534356+0000","last_undegraded":"2024-09-20T18:50:55.534356+0000","last_fullsized":"2024-09-20T18:50:55.534356+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":55,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:54.873209+0000","last_change":"2024-09-20T18:49:51.970609+0000","last_active":"2024-09-20T18:50:54.873209+0000","last_peered":"2024-09-20T18:50:54.873209+0000","last_clean":"2024-09-20T18:50:54.873209+0000","last_became_active":"2024-09-20T18:49:51.970343+0000","last_became_peered":"2024-09-20T18:49:51.970343+0000","last_unstale":"2024-09-20T18:50:54.873209+0000","last_undegraded":"2024-09-20T18:50:54.873209+0000","last_fullsized":"2024-09-20T18:50:54.873209+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:06.660017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.564898+0000","last_change":"2024-09-20T18:50:49.842714+0000","last_active":"2024-09-20T18:50:55.564898+0000","last_peered":"2024-09-20T18:50:55.564898+0000","last_clean":"2024-09-20T18:50:55.564898+0000","last_became_active":"2024-09-20T18:50:49.842310+0000","last_became_peered":"2024-09-20T18:50:49.842310+0000","last_unstale":"2024-09-20T18:50:55.564898+0000","last_undegraded":"2024-09-20T18:50:55.564898+0000","last_fullsized":"2024-09-20T18:50:55.564898+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.819660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:55.921403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":98,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.533764+0000","last_change":"2024-09-20T18:49:48.498345+0000","last_active":"2024-09-20T18:50:55.533764+0000","last_peered":"2024-09-20T18:50:55.533764+0000","last_clean":"2024-09-20T18:50:55.533764+0000","last_became_active":"2024-09-20T18:49:48.497936+0000","last_became_peered":"2024-09-20T18:49:48.497936+0000","last_unstale":"2024-09-20T18:50:55.533764+0000","last_undegraded":"2024-09-20T18:50:55.533764+0000","last_fullsized":"2024-09-20T18:50:55.533764+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":67,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:54.872634+0000","last_change":"2024-09-20T18:49:45.941387+0000","last_active":"2024-09-20T18:50:54.872634+0000","last_peered":"2024-09-20T18:50:54.872634+0000","last_clean":"2024-09-20T18:50:54.872634+0000","last_became_active":"2024-09-20T18:49:45.941136+0000","last_became_peered":"2024-09-20T18:49:45.941136+0000","last_unstale":"2024-09-20T18:50:54.872634+0000","last_undegraded":"2024-09-20T18:50:54.872634+0000","last_fullsized":"2024-09-20T18:50:54.872634+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:09.074025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":113,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.533422+0000","last_change":"2024-09-20T18:50:31.593683+0000","last_active":"2024-09-20T18:50:55.533422+0000","last_peered":"2024-09-20T18:50:55.533422+0000","last_clean":"2024-09-20T18:50:55.533422+0000","last_became_active":"2024-09-20T18:49:36.545306+0000","last_became_peered":"2024-09-20T18:49:36.545306+0000","last_unstale":"2024-09-20T18:50:55.533422+0000","last_undegraded":"2024-09-20T18:50:55.533422+0000","last_fullsized":"2024-09-20T18:50:55.533422+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T18:50:31.593610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T18:50:31.593610+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:00.939267+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":58,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.533741+0000","last_change":"2024-09-20T18:49:49.956441+0000","last_active":"2024-09-20T18:50:55.533741+0000","last_peered":"2024-09-20T18:50:55.533741+0000","last_clean":"2024-09-20T18:50:55.533741+0000","last_became_active":"2024-09-20T18:49:49.956128+0000","last_became_peered":"2024-09-20T18:49:49.956128+0000","last_unstale":"2024-09-20T18:50:55.533741+0000","last_undegraded":"2024-09-20T18:50:55.533741+0000","last_fullsized":"2024-09-20T18:50:55.533741+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:04.622346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":54,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.534171+0000","last_change":"2024-09-20T18:49:51.971980+0000","last_active":"2024-09-20T18:50:55.534171+0000","last_peered":"2024-09-20T18:50:55.534171+0000","last_clean":"2024-09-20T18:50:55.534171+0000","last_became_active":"2024-09-20T18:49:51.971332+0000","last_became_peered":"2024-09-20T18:49:51.971332+0000","last_unstale":"2024-09-20T18:50:55.534171+0000","last_undegraded":"2024-09-20T18:50:55.534171+0000","last_fullsized":"2024-09-20T18:50:55.534171+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:13.174503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.533345+0000","last_change":"2024-09-20T18:50:49.842768+0000","last_active":"2024-09-20T18:50:55.533345+0000","last_peered":"2024-09-20T18:50:55.533345+0000","last_clean":"2024-09-20T18:50:55.533345+0000","last_became_active":"2024-09-20T18:50:49.842483+0000","last_became_peered":"2024-09-20T18:50:49.842483+0000","last_unstale":"2024-09-20T18:50:55.533345+0000","last_undegraded":"2024-09-20T18:50:55.533345+0000","last_fullsized":"2024-09-20T18:50:55.533345+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.819660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:01.275928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":84,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.566681+0000","last_change":"2024-09-20T18:49:47.950762+0000","last_active":"2024-09-20T18:50:55.566681+0000","last_peered":"2024-09-20T18:50:55.566681+0000","last_clean":"2024-09-20T18:50:55.566681+0000","last_became_active":"2024-09-20T18:49:47.950073+0000","last_became_peered":"2024-09-20T18:49:47.950073+0000","last_unstale":"2024-09-20T18:50:55.566681+0000","last_undegraded":"2024-09-20T18:50:55.566681+0000","last_fullsized":"2024-09-20T18:50:55.566681+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:57.527730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":69,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.566168+0000","last_change":"2024-09-20T18:49:45.944925+0000","last_active":"2024-09-20T18:50:55.566168+0000","last_peered":"2024-09-20T18:50:55.566168+0000","last_clean":"2024-09-20T18:50:55.566168+0000","last_became_active":"2024-09-20T18:49:45.944579+0000","last_became_peered":"2024-09-20T18:49:45.944579+0000","last_unstale":"2024-09-20T18:50:55.566168+0000","last_undegraded":"2024-09-20T18:50:55.566168+0000","last_fullsized":"2024-09-20T18:50:55.566168+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:03.333030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":58,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.534020+0000","last_change":"2024-09-20T18:49:49.954673+0000","last_active":"2024-09-20T18:50:55.534020+0000","last_peered":"2024-09-20T18:50:55.534020+0000","last_clean":"2024-09-20T18:50:55.534020+0000","last_became_active":"2024-09-20T18:49:49.954391+0000","last_became_peered":"2024-09-20T18:49:49.954391+0000","last_unstale":"2024-09-20T18:50:55.534020+0000","last_undegraded":"2024-09-20T18:50:55.534020+0000","last_fullsized":"2024-09-20T18:50:55.534020+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:12.999716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":55,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:54.873043+0000","last_change":"2024-09-20T18:49:52.140252+0000","last_active":"2024-09-20T18:50:54.873043+0000","last_peered":"2024-09-20T18:50:54.873043+0000","last_clean":"2024-09-20T18:50:54.873043+0000","last_became_active":"2024-09-20T18:49:52.139959+0000","last_became_peered":"2024-09-20T18:49:52.139959+0000","last_unstale":"2024-09-20T18:50:54.873043+0000","last_undegraded":"2024-09-20T18:50:54.873043+0000","last_fullsized":"2024-09-20T18:50:54.873043+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:54.812391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.563572+0000","last_change":"2024-09-20T18:50:49.842002+0000","last_active":"2024-09-20T18:50:55.563572+0000","last_peered":"2024-09-20T18:50:55.563572+0000","last_clean":"2024-09-20T18:50:55.563572+0000","last_became_active":"2024-09-20T18:50:49.841683+0000","last_became_peered":"2024-09-20T18:50:49.841683+0000","last_unstale":"2024-09-20T18:50:55.563572+0000","last_undegraded":"2024-09-20T18:50:55.563572+0000","last_fullsized":"2024-09-20T18:50:55.563572+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.819660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:13.650405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":101,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.566441+0000","last_change":"2024-09-20T18:50:48.145398+0000","last_active":"2024-09-20T18:50:55.566441+0000","last_peered":"2024-09-20T18:50:55.566441+0000","last_clean":"2024-09-20T18:50:55.566441+0000","last_became_active":"2024-09-20T18:49:47.947636+0000","last_became_peered":"2024-09-20T18:49:47.947636+0000","last_unstale":"2024-09-20T18:50:55.566441+0000","last_undegraded":"2024-09-20T18:50:55.566441+0000","last_fullsized":"2024-09-20T18:50:55.566441+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-20T18:50:48.145313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.145313+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:52.345089+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":66,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.533099+0000","last_change":"2024-09-20T18:49:45.946517+0000","last_active":"2024-09-20T18:50:55.533099+0000","last_peered":"2024-09-20T18:50:55.533099+0000","last_clean":"2024-09-20T18:50:55.533099+0000","last_became_active":"2024-09-20T18:49:45.946153+0000","last_became_peered":"2024-09-20T18:49:45.946153+0000","last_unstale":"2024-09-20T18:50:55.533099+0000","last_undegraded":"2024-09-20T18:50:55.533099+0000","last_fullsized":"2024-09-20T18:50:55.533099+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:05.889152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":59,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:54.873288+0000","last_change":"2024-09-20T18:49:49.959440+0000","last_active":"2024-09-20T18:50:54.873288+0000","last_peered":"2024-09-20T18:50:54.873288+0000","last_clean":"2024-09-20T18:50:54.873288+0000","last_became_active":"2024-09-20T18:49:49.959160+0000","last_became_peered":"2024-09-20T18:49:49.959160+0000","last_unstale":"2024-09-20T18:50:54.873288+0000","last_undegraded":"2024-09-20T18:50:54.873288+0000","last_fullsized":"2024-09-20T18:50:54.873288+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:08.849486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":58,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.563806+0000","last_change":"2024-09-20T18:49:51.970386+0000","last_active":"2024-09-20T18:50:55.563806+0000","last_peered":"2024-09-20T18:50:55.563806+0000","last_clean":"2024-09-20T18:50:55.563806+0000","last_became_active":"2024-09-20T18:49:51.970108+0000","last_became_peered":"2024-09-20T18:49:51.970108+0000","last_unstale":"2024-09-20T18:50:55.563806+0000","last_undegraded":"2024-09-20T18:50:55.563806+0000","last_fullsized":"2024-09-20T18:50:55.563806+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:04.919785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.564818+0000","last_change":"2024-09-20T18:50:49.841900+0000","last_active":"2024-09-20T18:50:55.564818+0000","last_peered":"2024-09-20T18:50:55.564818+0000","last_clean":"2024-09-20T18:50:55.564818+0000","last_became_active":"2024-09-20T18:50:49.841628+0000","last_became_peered":"2024-09-20T18:50:49.841628+0000","last_unstale":"2024-09-20T18:50:55.564818+0000","last_undegraded":"2024-09-20T18:50:55.564818+0000","last_fullsized":"2024-09-20T18:50:55.564818+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.819660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:04.408905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":80,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.566361+0000","last_change":"2024-09-20T18:49:47.948046+0000","last_active":"2024-09-20T18:50:55.566361+0000","last_peered":"2024-09-20T18:50:55.566361+0000","last_clean":"2024-09-20T18:50:55.566361+0000","last_became_active":"2024-09-20T18:49:47.947126+0000","last_became_peered":"2024-09-20T18:49:47.947126+0000","last_unstale":"2024-09-20T18:50:55.566361+0000","last_undegraded":"2024-09-20T18:50:55.566361+0000","last_fullsized":"2024-09-20T18:50:55.566361+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:16.466592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":67,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:54.872326+0000","last_change":"2024-09-20T18:49:45.937511+0000","last_active":"2024-09-20T18:50:54.872326+0000","last_peered":"2024-09-20T18:50:54.872326+0000","last_clean":"2024-09-20T18:50:54.872326+0000","last_became_active":"2024-09-20T18:49:45.937264+0000","last_became_peered":"2024-09-20T18:49:45.937264+0000","last_unstale":"2024-09-20T18:50:54.872326+0000","last_undegraded":"2024-09-20T18:50:54.872326+0000","last_fullsized":"2024-09-20T18:50:54.872326+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:06.234318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":84,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.563465+0000","last_change":"2024-09-20T18:49:49.957286+0000","last_active":"2024-09-20T18:50:55.563465+0000","last_peered":"2024-09-20T18:50:55.563465+0000","last_clean":"2024-09-20T18:50:55.563465+0000","last_became_active":"2024-09-20T18:49:49.957005+0000","last_became_peered":"2024-09-20T18:49:49.957005+0000","last_unstale":"2024-09-20T18:50:55.563465+0000","last_undegraded":"2024-09-20T18:50:55.563465+0000","last_fullsized":"2024-09-20T18:50:55.563465+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:11.425942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":57,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.566294+0000","last_change":"2024-09-20T18:49:51.969633+0000","last_active":"2024-09-20T18:50:55.566294+0000","last_peered":"2024-09-20T18:50:55.566294+0000","last_clean":"2024-09-20T18:50:55.566294+0000","last_became_active":"2024-09-20T18:49:51.969355+0000","last_became_peered":"2024-09-20T18:49:51.969355+0000","last_unstale":"2024-09-20T18:50:55.566294+0000","last_undegraded":"2024-09-20T18:50:55.566294+0000","last_fullsized":"2024-09-20T18:50:55.566294+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.562872+0000","last_change":"2024-09-20T18:50:49.844296+0000","last_active":"2024-09-20T18:50:55.562872+0000","last_peered":"2024-09-20T18:50:55.562872+0000","last_clean":"2024-09-20T18:50:55.562872+0000","last_became_active":"2024-09-20T18:50:49.844028+0000","last_became_peered":"2024-09-20T18:50:49.844028+0000","last_unstale":"2024-09-20T18:50:55.562872+0000","last_undegraded":"2024-09-20T18:50:55.562872+0000","last_fullsized":"2024-09-20T18:50:55.562872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.819660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:07.943453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":80,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.564691+0000","last_change":"2024-09-20T18:49:47.946309+0000","last_active":"2024-09-20T18:50:55.564691+0000","last_peered":"2024-09-20T18:50:55.564691+0000","last_clean":"2024-09-20T18:50:55.564691+0000","last_became_active":"2024-09-20T18:49:47.946008+0000","last_became_peered":"2024-09-20T18:49:47.946008+0000","last_unstale":"2024-09-20T18:50:55.564691+0000","last_undegraded":"2024-09-20T18:50:55.564691+0000","last_fullsized":"2024-09-20T18:50:55.564691+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:55.957874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":74,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.533648+0000","last_change":"2024-09-20T18:50:46.750354+0000","last_active":"2024-09-20T18:50:55.533648+0000","last_peered":"2024-09-20T18:50:55.533648+0000","last_clean":"2024-09-20T18:50:55.533648+0000","last_became_active":"2024-09-20T18:49:45.941406+0000","last_became_peered":"2024-09-20T18:49:45.941406+0000","last_unstale":"2024-09-20T18:50:55.533648+0000","last_undegraded":"2024-09-20T18:50:55.533648+0000","last_fullsized":"2024-09-20T18:50:55.533648+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:50:46.750281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:50:46.750281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:07.951779+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":62,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.563294+0000","last_change":"2024-09-20T18:49:49.957141+0000","last_active":"2024-09-20T18:50:55.563294+0000","last_peered":"2024-09-20T18:50:55.563294+0000","last_clean":"2024-09-20T18:50:55.563294+0000","last_became_active":"2024-09-20T18:49:49.956767+0000","last_became_peered":"2024-09-20T18:49:49.956767+0000","last_unstale":"2024-09-20T18:50:55.563294+0000","last_undegraded":"2024-09-20T18:50:55.563294+0000","last_fullsized":"2024-09-20T18:50:55.563294+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:55.731396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":56,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.564461+0000","last_change":"2024-09-20T18:49:51.964375+0000","last_active":"2024-09-20T18:50:55.564461+0000","last_peered":"2024-09-20T18:50:55.564461+0000","last_clean":"2024-09-20T18:50:55.564461+0000","last_became_active":"2024-09-20T18:49:51.964098+0000","last_became_peered":"2024-09-20T18:49:51.964098+0000","last_unstale":"2024-09-20T18:50:55.564461+0000","last_undegraded":"2024-09-20T18:50:55.564461+0000","last_fullsized":"2024-09-20T18:50:55.564461+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:10.566377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.564216+0000","last_change":"2024-09-20T18:50:49.842558+0000","last_active":"2024-09-20T18:50:55.564216+0000","last_peered":"2024-09-20T18:50:55.564216+0000","last_clean":"2024-09-20T18:50:55.564216+0000","last_became_active":"2024-09-20T18:50:49.840023+0000","last_became_peered":"2024-09-20T18:50:49.840023+0000","last_unstale":"2024-09-20T18:50:55.564216+0000","last_undegraded":"2024-09-20T18:50:55.564216+0000","last_fullsized":"2024-09-20T18:50:55.564216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.819660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:55.634729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":93,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.532865+0000","last_change":"2024-09-20T18:49:48.498846+0000","last_active":"2024-09-20T18:50:55.532865+0000","last_peered":"2024-09-20T18:50:55.532865+0000","last_clean":"2024-09-20T18:50:55.532865+0000","last_became_active":"2024-09-20T18:49:48.498625+0000","last_became_peered":"2024-09-20T18:49:48.498625+0000","last_unstale":"2024-09-20T18:50:55.532865+0000","last_undegraded":"2024-09-20T18:50:55.532865+0000","last_fullsized":"2024-09-20T18:50:55.532865+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:03.186719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":66,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.534054+0000","last_change":"2024-09-20T18:49:45.941764+0000","last_active":"2024-09-20T18:50:55.534054+0000","last_peered":"2024-09-20T18:50:55.534054+0000","last_clean":"2024-09-20T18:50:55.534054+0000","last_became_active":"2024-09-20T18:49:45.941448+0000","last_became_peered":"2024-09-20T18:49:45.941448+0000","last_unstale":"2024-09-20T18:50:55.534054+0000","last_undegraded":"2024-09-20T18:50:55.534054+0000","last_fullsized":"2024-09-20T18:50:55.534054+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:53.774098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":62,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.563388+0000","last_change":"2024-09-20T18:49:49.957845+0000","last_active":"2024-09-20T18:50:55.563388+0000","last_peered":"2024-09-20T18:50:55.563388+0000","last_clean":"2024-09-20T18:50:55.563388+0000","last_became_active":"2024-09-20T18:49:49.957558+0000","last_became_peered":"2024-09-20T18:49:49.957558+0000","last_unstale":"2024-09-20T18:50:55.563388+0000","last_undegraded":"2024-09-20T18:50:55.563388+0000","last_fullsized":"2024-09-20T18:50:55.563388+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:14.726319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":54,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.533337+0000","last_change":"2024-09-20T18:49:51.971132+0000","last_active":"2024-09-20T18:50:55.533337+0000","last_peered":"2024-09-20T18:50:55.533337+0000","last_clean":"2024-09-20T18:50:55.533337+0000","last_became_active":"2024-09-20T18:49:51.970798+0000","last_became_peered":"2024-09-20T18:49:51.970798+0000","last_unstale":"2024-09-20T18:50:55.533337+0000","last_undegraded":"2024-09-20T18:50:55.533337+0000","last_fullsized":"2024-09-20T18:50:55.533337+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:09.540547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:54.871511+0000","last_change":"2024-09-20T18:50:49.842445+0000","last_active":"2024-09-20T18:50:54.871511+0000","last_peered":"2024-09-20T18:50:54.871511+0000","last_clean":"2024-09-20T18:50:54.871511+0000","last_became_active":"2024-09-20T18:50:49.842179+0000","last_became_peered":"2024-09-20T18:50:49.842179+0000","last_unstale":"2024-09-20T18:50:54.871511+0000","last_undegraded":"2024-09-20T18:50:54.871511+0000","last_fullsized":"2024-09-20T18:50:54.871511+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.819660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:52.881363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":88,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.565932+0000","last_change":"2024-09-20T18:49:47.949428+0000","last_active":"2024-09-20T18:50:55.565932+0000","last_peered":"2024-09-20T18:50:55.565932+0000","last_clean":"2024-09-20T18:50:55.565932+0000","last_became_active":"2024-09-20T18:49:47.949116+0000","last_became_peered":"2024-09-20T18:49:47.949116+0000","last_unstale":"2024-09-20T18:50:55.565932+0000","last_undegraded":"2024-09-20T18:50:55.565932+0000","last_fullsized":"2024-09-20T18:50:55.565932+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:11.352253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":66,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.534329+0000","last_change":"2024-09-20T18:49:45.945147+0000","last_active":"2024-09-20T18:50:55.534329+0000","last_peered":"2024-09-20T18:50:55.534329+0000","last_clean":"2024-09-20T18:50:55.534329+0000","last_became_active":"2024-09-20T18:49:45.944858+0000","last_became_peered":"2024-09-20T18:49:45.944858+0000","last_unstale":"2024-09-20T18:50:55.534329+0000","last_undegraded":"2024-09-20T18:50:55.534329+0000","last_fullsized":"2024-09-20T18:50:55.534329+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":62,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.562958+0000","last_change":"2024-09-20T18:49:49.955243+0000","last_active":"2024-09-20T18:50:55.562958+0000","last_peered":"2024-09-20T18:50:55.562958+0000","last_clean":"2024-09-20T18:50:55.562958+0000","last_became_active":"2024-09-20T18:49:49.954944+0000","last_became_peered":"2024-09-20T18:49:49.954944+0000","last_unstale":"2024-09-20T18:50:55.562958+0000","last_undegraded":"2024-09-20T18:50:55.562958+0000","last_fullsized":"2024-09-20T18:50:55.562958+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:00.542377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":55,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:54.872027+0000","last_change":"2024-09-20T18:49:51.973284+0000","last_active":"2024-09-20T18:50:54.872027+0000","last_peered":"2024-09-20T18:50:54.872027+0000","last_clean":"2024-09-20T18:50:54.872027+0000","last_became_active":"2024-09-20T18:49:51.972970+0000","last_became_peered":"2024-09-20T18:49:51.972970+0000","last_unstale":"2024-09-20T18:50:54.872027+0000","last_undegraded":"2024-09-20T18:50:54.872027+0000","last_fullsized":"2024-09-20T18:50:54.872027+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:55.977221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:54.873390+0000","last_change":"2024-09-20T18:50:49.843796+0000","last_active":"2024-09-20T18:50:54.873390+0000","last_peered":"2024-09-20T18:50:54.873390+0000","last_clean":"2024-09-20T18:50:54.873390+0000","last_became_active":"2024-09-20T18:50:49.843516+0000","last_became_peered":"2024-09-20T18:50:49.843516+0000","last_unstale":"2024-09-20T18:50:54.873390+0000","last_undegraded":"2024-09-20T18:50:54.873390+0000","last_fullsized":"2024-09-20T18:50:54.873390+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.819660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:55.068715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":87,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.532378+0000","last_change":"2024-09-20T18:49:48.500174+0000","last_active":"2024-09-20T18:50:55.532378+0000","last_peered":"2024-09-20T18:50:55.532378+0000","last_clean":"2024-09-20T18:50:55.532378+0000","last_became_active":"2024-09-20T18:49:48.499847+0000","last_became_peered":"2024-09-20T18:49:48.499847+0000","last_unstale":"2024-09-20T18:50:55.532378+0000","last_undegraded":"2024-09-20T18:50:55.532378+0000","last_fullsized":"2024-09-20T18:50:55.532378+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:15.572682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":78,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.534287+0000","last_change":"2024-09-20T18:49:45.945391+0000","last_active":"2024-09-20T18:50:55.534287+0000","last_peered":"2024-09-20T18:50:55.534287+0000","last_clean":"2024-09-20T18:50:55.534287+0000","last_became_active":"2024-09-20T18:49:45.944256+0000","last_became_peered":"2024-09-20T18:49:45.944256+0000","last_unstale":"2024-09-20T18:50:55.534287+0000","last_undegraded":"2024-09-20T18:50:55.534287+0000","last_fullsized":"2024-09-20T18:50:55.534287+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:54.747555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":80,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.532455+0000","last_change":"2024-09-20T18:49:49.958686+0000","last_active":"2024-09-20T18:50:55.532455+0000","last_peered":"2024-09-20T18:50:55.532455+0000","last_clean":"2024-09-20T18:50:55.532455+0000","last_became_active":"2024-09-20T18:49:49.958411+0000","last_became_peered":"2024-09-20T18:49:49.958411+0000","last_unstale":"2024-09-20T18:50:55.532455+0000","last_undegraded":"2024-09-20T18:50:55.532455+0000","last_fullsized":"2024-09-20T18:50:55.532455+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:57.283019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":54,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.532649+0000","last_change":"2024-09-20T18:49:52.139994+0000","last_active":"2024-09-20T18:50:55.532649+0000","last_peered":"2024-09-20T18:50:55.532649+0000","last_clean":"2024-09-20T18:50:55.532649+0000","last_became_active":"2024-09-20T18:49:52.139685+0000","last_became_peered":"2024-09-20T18:49:52.139685+0000","last_unstale":"2024-09-20T18:50:55.532649+0000","last_undegraded":"2024-09-20T18:50:55.532649+0000","last_fullsized":"2024-09-20T18:50:55.532649+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":66,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.532464+0000","last_change":"2024-09-20T18:49:45.940360+0000","last_active":"2024-09-20T18:50:55.532464+0000","last_peered":"2024-09-20T18:50:55.532464+0000","last_clean":"2024-09-20T18:50:55.532464+0000","last_became_active":"2024-09-20T18:49:45.940060+0000","last_became_peered":"2024-09-20T18:49:45.940060+0000","last_unstale":"2024-09-20T18:50:55.532464+0000","last_undegraded":"2024-09-20T18:50:55.532464+0000","last_fullsized":"2024-09-20T18:50:55.532464+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:59.071733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":72,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:54.873277+0000","last_change":"2024-09-20T18:49:48.501010+0000","last_active":"2024-09-20T18:50:54.873277+0000","last_peered":"2024-09-20T18:50:54.873277+0000","last_clean":"2024-09-20T18:50:54.873277+0000","last_became_active":"2024-09-20T18:49:48.500662+0000","last_became_peered":"2024-09-20T18:49:48.500662+0000","last_unstale":"2024-09-20T18:50:54.873277+0000","last_undegraded":"2024-09-20T18:50:54.873277+0000","last_fullsized":"2024-09-20T18:50:54.873277+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:01.787506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":58,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.534121+0000","last_change":"2024-09-20T18:49:49.957092+0000","last_active":"2024-09-20T18:50:55.534121+0000","last_peered":"2024-09-20T18:50:55.534121+0000","last_clean":"2024-09-20T18:50:55.534121+0000","last_became_active":"2024-09-20T18:49:49.956749+0000","last_became_peered":"2024-09-20T18:49:49.956749+0000","last_unstale":"2024-09-20T18:50:55.534121+0000","last_undegraded":"2024-09-20T18:50:55.534121+0000","last_fullsized":"2024-09-20T18:50:55.534121+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":54,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.533770+0000","last_change":"2024-09-20T18:49:52.142789+0000","last_active":"2024-09-20T18:50:55.533770+0000","last_peered":"2024-09-20T18:50:55.533770+0000","last_clean":"2024-09-20T18:50:55.533770+0000","last_became_active":"2024-09-20T18:49:52.142291+0000","last_became_peered":"2024-09-20T18:49:52.142291+0000","last_unstale":"2024-09-20T18:50:55.533770+0000","last_undegraded":"2024-09-20T18:50:55.533770+0000","last_fullsized":"2024-09-20T18:50:55.533770+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:58.299262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":80,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.565066+0000","last_change":"2024-09-20T18:49:48.501376+0000","last_active":"2024-09-20T18:50:55.565066+0000","last_peered":"2024-09-20T18:50:55.565066+0000","last_clean":"2024-09-20T18:50:55.565066+0000","last_became_active":"2024-09-20T18:49:48.500803+0000","last_became_peered":"2024-09-20T18:49:48.500803+0000","last_unstale":"2024-09-20T18:50:55.565066+0000","last_undegraded":"2024-09-20T18:50:55.565066+0000","last_fullsized":"2024-09-20T18:50:55.565066+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:02.881066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":66,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.532656+0000","last_change":"2024-09-20T18:49:45.947791+0000","last_active":"2024-09-20T18:50:55.532656+0000","last_peered":"2024-09-20T18:50:55.532656+0000","last_clean":"2024-09-20T18:50:55.532656+0000","last_became_active":"2024-09-20T18:49:45.946600+0000","last_became_peered":"2024-09-20T18:49:45.946600+0000","last_unstale":"2024-09-20T18:50:55.532656+0000","last_undegraded":"2024-09-20T18:50:55.532656+0000","last_fullsized":"2024-09-20T18:50:55.532656+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:13.160428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":59,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:54.873144+0000","last_change":"2024-09-20T18:49:49.962290+0000","last_active":"2024-09-20T18:50:54.873144+0000","last_peered":"2024-09-20T18:50:54.873144+0000","last_clean":"2024-09-20T18:50:54.873144+0000","last_became_active":"2024-09-20T18:49:49.961947+0000","last_became_peered":"2024-09-20T18:49:49.961947+0000","last_unstale":"2024-09-20T18:50:54.873144+0000","last_undegraded":"2024-09-20T18:50:54.873144+0000","last_fullsized":"2024-09-20T18:50:54.873144+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"66'1","reported_seq":59,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.563428+0000","last_change":"2024-09-20T18:49:51.965397+0000","last_active":"2024-09-20T18:50:55.563428+0000","last_peered":"2024-09-20T18:50:55.563428+0000","last_clean":"2024-09-20T18:50:55.563428+0000","last_became_active":"2024-09-20T18:49:51.965121+0000","last_became_peered":"2024-09-20T18:49:51.965121+0000","last_unstale":"2024-09-20T18:50:55.563428+0000","last_undegraded":"2024-09-20T18:50:55.563428+0000","last_fullsized":"2024-09-20T18:50:55.563428+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:57.419566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":78,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.534621+0000","last_change":"2024-09-20T18:49:48.501359+0000","last_active":"2024-09-20T18:50:55.534621+0000","last_peered":"2024-09-20T18:50:55.534621+0000","last_clean":"2024-09-20T18:50:55.534621+0000","last_became_active":"2024-09-20T18:49:48.501011+0000","last_became_peered":"2024-09-20T18:49:48.501011+0000","last_unstale":"2024-09-20T18:50:55.534621+0000","last_undegraded":"2024-09-20T18:50:55.534621+0000","last_fullsized":"2024-09-20T18:50:55.534621+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:54.722340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":68,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.564518+0000","last_change":"2024-09-20T18:49:45.938820+0000","last_active":"2024-09-20T18:50:55.564518+0000","last_peered":"2024-09-20T18:50:55.564518+0000","last_clean":"2024-09-20T18:50:55.564518+0000","last_became_active":"2024-09-20T18:49:45.938511+0000","last_became_peered":"2024-09-20T18:49:45.938511+0000","last_unstale":"2024-09-20T18:50:55.564518+0000","last_undegraded":"2024-09-20T18:50:55.564518+0000","last_fullsized":"2024-09-20T18:50:55.564518+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:02.383015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":81,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:54.873203+0000","last_change":"2024-09-20T18:49:49.960889+0000","last_active":"2024-09-20T18:50:54.873203+0000","last_peered":"2024-09-20T18:50:54.873203+0000","last_clean":"2024-09-20T18:50:54.873203+0000","last_became_active":"2024-09-20T18:49:49.960550+0000","last_became_peered":"2024-09-20T18:49:49.960550+0000","last_unstale":"2024-09-20T18:50:54.873203+0000","last_undegraded":"2024-09-20T18:50:54.873203+0000","last_fullsized":"2024-09-20T18:50:54.873203+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:58.739350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":57,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.566552+0000","last_change":"2024-09-20T18:49:51.966530+0000","last_active":"2024-09-20T18:50:55.566552+0000","last_peered":"2024-09-20T18:50:55.566552+0000","last_clean":"2024-09-20T18:50:55.566552+0000","last_became_active":"2024-09-20T18:49:51.966254+0000","last_became_peered":"2024-09-20T18:49:51.966254+0000","last_unstale":"2024-09-20T18:50:55.566552+0000","last_undegraded":"2024-09-20T18:50:55.566552+0000","last_fullsized":"2024-09-20T18:50:55.566552+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:14.193792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":97,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:54.873472+0000","last_change":"2024-09-20T18:49:59.003234+0000","last_active":"2024-09-20T18:50:54.873472+0000","last_peered":"2024-09-20T18:50:54.873472+0000","last_clean":"2024-09-20T18:50:54.873472+0000","last_became_active":"2024-09-20T18:49:58.997494+0000","last_became_peered":"2024-09-20T18:49:58.997494+0000","last_unstale":"2024-09-20T18:50:54.873472+0000","last_undegraded":"2024-09-20T18:50:54.873472+0000","last_fullsized":"2024-09-20T18:50:54.873472+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:05.235529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":66,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.533358+0000","last_change":"2024-09-20T18:49:45.949895+0000","last_active":"2024-09-20T18:50:55.533358+0000","last_peered":"2024-09-20T18:50:55.533358+0000","last_clean":"2024-09-20T18:50:55.533358+0000","last_became_active":"2024-09-20T18:49:45.949492+0000","last_became_peered":"2024-09-20T18:49:45.949492+0000","last_unstale":"2024-09-20T18:50:55.533358+0000","last_undegraded":"2024-09-20T18:50:55.533358+0000","last_fullsized":"2024-09-20T18:50:55.533358+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:02.958108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":80,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.534521+0000","last_change":"2024-09-20T18:49:49.962519+0000","last_active":"2024-09-20T18:50:55.534521+0000","last_peered":"2024-09-20T18:50:55.534521+0000","last_clean":"2024-09-20T18:50:55.534521+0000","last_became_active":"2024-09-20T18:49:49.962243+0000","last_became_peered":"2024-09-20T18:49:49.962243+0000","last_unstale":"2024-09-20T18:50:55.534521+0000","last_undegraded":"2024-09-20T18:50:55.534521+0000","last_fullsized":"2024-09-20T18:50:55.534521+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:16.990588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":56,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.564823+0000","last_change":"2024-09-20T18:49:51.969358+0000","last_active":"2024-09-20T18:50:55.564823+0000","last_peered":"2024-09-20T18:50:55.564823+0000","last_clean":"2024-09-20T18:50:55.564823+0000","last_became_active":"2024-09-20T18:49:51.969093+0000","last_became_peered":"2024-09-20T18:49:51.969093+0000","last_unstale":"2024-09-20T18:50:55.564823+0000","last_undegraded":"2024-09-20T18:50:55.564823+0000","last_fullsized":"2024-09-20T18:50:55.564823+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:17.459087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":84,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.566118+0000","last_change":"2024-09-20T18:49:47.950488+0000","last_active":"2024-09-20T18:50:55.566118+0000","last_peered":"2024-09-20T18:50:55.566118+0000","last_clean":"2024-09-20T18:50:55.566118+0000","last_became_active":"2024-09-20T18:49:47.949703+0000","last_became_peered":"2024-09-20T18:49:47.949703+0000","last_unstale":"2024-09-20T18:50:55.566118+0000","last_undegraded":"2024-09-20T18:50:55.566118+0000","last_fullsized":"2024-09-20T18:50:55.566118+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":69,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.566562+0000","last_change":"2024-09-20T18:49:45.941578+0000","last_active":"2024-09-20T18:50:55.566562+0000","last_peered":"2024-09-20T18:50:55.566562+0000","last_clean":"2024-09-20T18:50:55.566562+0000","last_became_active":"2024-09-20T18:49:45.939949+0000","last_became_peered":"2024-09-20T18:49:45.939949+0000","last_unstale":"2024-09-20T18:50:55.566562+0000","last_undegraded":"2024-09-20T18:50:55.566562+0000","last_fullsized":"2024-09-20T18:50:55.566562+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":59,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:54.872705+0000","last_change":"2024-09-20T18:49:49.957358+0000","last_active":"2024-09-20T18:50:54.872705+0000","last_peered":"2024-09-20T18:50:54.872705+0000","last_clean":"2024-09-20T18:50:54.872705+0000","last_became_active":"2024-09-20T18:49:49.957067+0000","last_became_peered":"2024-09-20T18:49:49.957067+0000","last_unstale":"2024-09-20T18:50:54.872705+0000","last_undegraded":"2024-09-20T18:50:54.872705+0000","last_fullsized":"2024-09-20T18:50:54.872705+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":55,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:54.871393+0000","last_change":"2024-09-20T18:49:51.967168+0000","last_active":"2024-09-20T18:50:54.871393+0000","last_peered":"2024-09-20T18:50:54.871393+0000","last_clean":"2024-09-20T18:50:54.871393+0000","last_became_active":"2024-09-20T18:49:51.966915+0000","last_became_peered":"2024-09-20T18:49:51.966915+0000","last_unstale":"2024-09-20T18:50:54.871393+0000","last_undegraded":"2024-09-20T18:50:54.871393+0000","last_fullsized":"2024-09-20T18:50:54.871393+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:04.536536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":79,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:54.872793+0000","last_change":"2024-09-20T18:49:47.953603+0000","last_active":"2024-09-20T18:50:54.872793+0000","last_peered":"2024-09-20T18:50:54.872793+0000","last_clean":"2024-09-20T18:50:54.872793+0000","last_became_active":"2024-09-20T18:49:47.953310+0000","last_became_peered":"2024-09-20T18:49:47.953310+0000","last_unstale":"2024-09-20T18:50:54.872793+0000","last_undegraded":"2024-09-20T18:50:54.872793+0000","last_fullsized":"2024-09-20T18:50:54.872793+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:06.158344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":66,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.533969+0000","last_change":"2024-09-20T18:49:45.944827+0000","last_active":"2024-09-20T18:50:55.533969+0000","last_peered":"2024-09-20T18:50:55.533969+0000","last_clean":"2024-09-20T18:50:55.533969+0000","last_became_active":"2024-09-20T18:49:45.944357+0000","last_became_peered":"2024-09-20T18:49:45.944357+0000","last_unstale":"2024-09-20T18:50:55.533969+0000","last_undegraded":"2024-09-20T18:50:55.533969+0000","last_fullsized":"2024-09-20T18:50:55.533969+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:14.013129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":59,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:54.872112+0000","last_change":"2024-09-20T18:49:49.962794+0000","last_active":"2024-09-20T18:50:54.872112+0000","last_peered":"2024-09-20T18:50:54.872112+0000","last_clean":"2024-09-20T18:50:54.872112+0000","last_became_active":"2024-09-20T18:49:49.962523+0000","last_became_peered":"2024-09-20T18:49:49.962523+0000","last_unstale":"2024-09-20T18:50:54.872112+0000","last_undegraded":"2024-09-20T18:50:54.872112+0000","last_fullsized":"2024-09-20T18:50:54.872112+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:06.567112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":56,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.564059+0000","last_change":"2024-09-20T18:49:51.967581+0000","last_active":"2024-09-20T18:50:55.564059+0000","last_peered":"2024-09-20T18:50:55.564059+0000","last_clean":"2024-09-20T18:50:55.564059+0000","last_became_active":"2024-09-20T18:49:51.967313+0000","last_became_peered":"2024-09-20T18:49:51.967313+0000","last_unstale":"2024-09-20T18:50:55.564059+0000","last_undegraded":"2024-09-20T18:50:55.564059+0000","last_fullsized":"2024-09-20T18:50:55.564059+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:05.774315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":82,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:54.871524+0000","last_change":"2024-09-20T18:49:48.499421+0000","last_active":"2024-09-20T18:50:54.871524+0000","last_peered":"2024-09-20T18:50:54.871524+0000","last_clean":"2024-09-20T18:50:54.871524+0000","last_became_active":"2024-09-20T18:49:48.499111+0000","last_became_peered":"2024-09-20T18:49:48.499111+0000","last_unstale":"2024-09-20T18:50:54.871524+0000","last_undegraded":"2024-09-20T18:50:54.871524+0000","last_fullsized":"2024-09-20T18:50:54.871524+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:57.119627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":66,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.534208+0000","last_change":"2024-09-20T18:49:45.942406+0000","last_active":"2024-09-20T18:50:55.534208+0000","last_peered":"2024-09-20T18:50:55.534208+0000","last_clean":"2024-09-20T18:50:55.534208+0000","last_became_active":"2024-09-20T18:49:45.942105+0000","last_became_peered":"2024-09-20T18:49:45.942105+0000","last_unstale":"2024-09-20T18:50:55.534208+0000","last_undegraded":"2024-09-20T18:50:55.534208+0000","last_fullsized":"2024-09-20T18:50:55.534208+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:56.309962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":58,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.532073+0000","last_change":"2024-09-20T18:49:49.959361+0000","last_active":"2024-09-20T18:50:55.532073+0000","last_peered":"2024-09-20T18:50:55.532073+0000","last_clean":"2024-09-20T18:50:55.532073+0000","last_became_active":"2024-09-20T18:49:49.958159+0000","last_became_peered":"2024-09-20T18:49:49.958159+0000","last_unstale":"2024-09-20T18:50:55.532073+0000","last_undegraded":"2024-09-20T18:50:55.532073+0000","last_fullsized":"2024-09-20T18:50:55.532073+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:12.385752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":55,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:54.872248+0000","last_change":"2024-09-20T18:49:52.140525+0000","last_active":"2024-09-20T18:50:54.872248+0000","last_peered":"2024-09-20T18:50:54.872248+0000","last_clean":"2024-09-20T18:50:54.872248+0000","last_became_active":"2024-09-20T18:49:52.139978+0000","last_became_peered":"2024-09-20T18:49:52.139978+0000","last_unstale":"2024-09-20T18:50:54.872248+0000","last_undegraded":"2024-09-20T18:50:54.872248+0000","last_fullsized":"2024-09-20T18:50:54.872248+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:19.358011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":69,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:54.871194+0000","last_change":"2024-09-20T18:49:47.955120+0000","last_active":"2024-09-20T18:50:54.871194+0000","last_peered":"2024-09-20T18:50:54.871194+0000","last_clean":"2024-09-20T18:50:54.871194+0000","last_became_active":"2024-09-20T18:49:47.954777+0000","last_became_peered":"2024-09-20T18:49:47.954777+0000","last_unstale":"2024-09-20T18:50:54.871194+0000","last_undegraded":"2024-09-20T18:50:54.871194+0000","last_fullsized":"2024-09-20T18:50:54.871194+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:04.481222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":68,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.565121+0000","last_change":"2024-09-20T18:49:45.941248+0000","last_active":"2024-09-20T18:50:55.565121+0000","last_peered":"2024-09-20T18:50:55.565121+0000","last_clean":"2024-09-20T18:50:55.565121+0000","last_became_active":"2024-09-20T18:49:45.940988+0000","last_became_peered":"2024-09-20T18:49:45.940988+0000","last_unstale":"2024-09-20T18:50:55.565121+0000","last_undegraded":"2024-09-20T18:50:55.565121+0000","last_fullsized":"2024-09-20T18:50:55.565121+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:58.598619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":58,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.532675+0000","last_change":"2024-09-20T18:49:49.954779+0000","last_active":"2024-09-20T18:50:55.532675+0000","last_peered":"2024-09-20T18:50:55.532675+0000","last_clean":"2024-09-20T18:50:55.532675+0000","last_became_active":"2024-09-20T18:49:49.954502+0000","last_became_peered":"2024-09-20T18:49:49.954502+0000","last_unstale":"2024-09-20T18:50:55.532675+0000","last_undegraded":"2024-09-20T18:50:55.532675+0000","last_fullsized":"2024-09-20T18:50:55.532675+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:15.141742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":54,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.532744+0000","last_change":"2024-09-20T18:49:51.968968+0000","last_active":"2024-09-20T18:50:55.532744+0000","last_peered":"2024-09-20T18:50:55.532744+0000","last_clean":"2024-09-20T18:50:55.532744+0000","last_became_active":"2024-09-20T18:49:51.968662+0000","last_became_peered":"2024-09-20T18:49:51.968662+0000","last_unstale":"2024-09-20T18:50:55.532744+0000","last_undegraded":"2024-09-20T18:50:55.532744+0000","last_fullsized":"2024-09-20T18:50:55.532744+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:02.092669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":55,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:54.872862+0000","last_change":"2024-09-20T18:49:51.968476+0000","last_active":"2024-09-20T18:50:54.872862+0000","last_peered":"2024-09-20T18:50:54.872862+0000","last_clean":"2024-09-20T18:50:54.872862+0000","last_became_active":"2024-09-20T18:49:51.968216+0000","last_became_peered":"2024-09-20T18:49:51.968216+0000","last_unstale":"2024-09-20T18:50:54.872862+0000","last_undegraded":"2024-09-20T18:50:54.872862+0000","last_fullsized":"2024-09-20T18:50:54.872862+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:11.930622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":67,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:54.871318+0000","last_change":"2024-09-20T18:49:45.938413+0000","last_active":"2024-09-20T18:50:54.871318+0000","last_peered":"2024-09-20T18:50:54.871318+0000","last_clean":"2024-09-20T18:50:54.871318+0000","last_became_active":"2024-09-20T18:49:45.937992+0000","last_became_peered":"2024-09-20T18:49:45.937992+0000","last_unstale":"2024-09-20T18:50:54.871318+0000","last_undegraded":"2024-09-20T18:50:54.871318+0000","last_fullsized":"2024-09-20T18:50:54.871318+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:12.596336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":81,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.534328+0000","last_change":"2024-09-20T18:49:47.950306+0000","last_active":"2024-09-20T18:50:55.534328+0000","last_peered":"2024-09-20T18:50:55.534328+0000","last_clean":"2024-09-20T18:50:55.534328+0000","last_became_active":"2024-09-20T18:49:47.949582+0000","last_became_peered":"2024-09-20T18:49:47.949582+0000","last_unstale":"2024-09-20T18:50:55.534328+0000","last_undegraded":"2024-09-20T18:50:55.534328+0000","last_fullsized":"2024-09-20T18:50:55.534328+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:03.753158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":60,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-20T18:50:55.564929+0000","last_change":"2024-09-20T18:49:49.957874+0000","last_active":"2024-09-20T18:50:55.564929+0000","last_peered":"2024-09-20T18:50:55.564929+0000","last_clean":"2024-09-20T18:50:55.564929+0000","last_became_active":"2024-09-20T18:49:49.957582+0000","last_became_peered":"2024-09-20T18:49:49.957582+0000","last_unstale":"2024-09-20T18:50:55.564929+0000","last_undegraded":"2024-09-20T18:50:55.564929+0000","last_fullsized":"2024-09-20T18:50:55.564929+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:07.454286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":65,"num_read_kb":60,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266706,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32652,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712500,"statfs":{"total":95995035648,"available":95961600000,"internally_reserved":0,"allocated":1495040,"data_stored":969002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528345,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28472,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716680,"statfs":{"total":95995035648,"available":95965880320,"internally_reserved":0,"allocated":1474560,"data_stored":968340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757278,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":897024,"data_stored":385495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986211,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":929792,"data_stored":386747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182442,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":913408,"data_stored":378427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411375,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":888832,"data_stored":378910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":950272,"data_stored":387258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738427,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32648,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712504,"statfs":{"total":95995035648,"available":95961604096,"internally_reserved":0,"allocated":1495040,"data_stored":968698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T18:50:58.910 INFO:tasks.thrashosds.thrasher:Setting 3.a to [5, 7, 0, 2, 4, 3] 2024-09-20T18:50:58.910 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.a', '5', '7', '0', '2', '4', '3'] 2024-09-20T18:50:58.910 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd pg-upmap-items 3.a 5 7 0 2 4 3 2024-09-20T18:50:58.947 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:50:59.002 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:50:59.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:59.003 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:50:59.104 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:50:59.133 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:59.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:59.134 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:50:59.234 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:50:59.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:59.281 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:50:59.281 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:50:59.382 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:50:59.416 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:59.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:59.417 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:50:59.513 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 2 16 6762 6746 210.797 205 0.00284607 0.00473331 2024-09-20T18:50:59.517 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:50:59.554 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:50:59.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:59.556 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:50:59.656 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:50:59.664 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_historic_ops 2024-09-20T18:50:59.688 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:59.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:59.689 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:50:59.790 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:50:59.819 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:59.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:59.820 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:50:59.828 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T18:50:59.921 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:50:59.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:50:59.949 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:50:59.949 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:00.050 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:00.076 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:00.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:00.077 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:00.133 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:50:59 smithi083 ceph-mon[29421]: pgmap v40: 148 pgs: 148 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T18:51:00.134 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:50:59 smithi083 ceph-mon[36260]: pgmap v40: 148 pgs: 148 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T18:51:00.156 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:50:59 smithi142 ceph-mon[32508]: pgmap v40: 148 pgs: 148 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T18:51:00.178 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:00.215 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:51:00.215 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:51:00.229 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:51:00.239 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:00.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:00.240 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:00.251 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:51:00.252 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:51:00.257 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:51:00.289 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:51:00.289 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:51:00.322 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:51:00.322 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:51:00.342 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:00.352 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:51:00.353 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:51:00.370 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:00.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:00.371 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:00.386 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:51:00.387 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:51:00.419 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:51:00.419 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:51:00.452 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:51:00.452 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:51:00.472 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:00.513 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 3 15 9274 9259 192.878 157.062 0.00597451 0.00517482 2024-09-20T18:51:00.513 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:00.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:00.514 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:00.523 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:00 smithi142 bash[61969]: Copying config sha256:72c9c208898624938c9e4183d6686ea4a5fd3f912bc29bc3f00147924c521a3e 2024-09-20T18:51:00.615 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:00.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:00.643 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:00.643 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:00.744 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:00.785 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:00.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:00.787 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:00.863 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:00 smithi142 bash[61969]: Writing manifest to image destination 2024-09-20T18:51:00.888 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:51:00.925 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:00.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:00.926 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:51:01.027 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:01.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:00 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:51:01.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:00 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:51:01.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:00 smithi083 ceph-mon[29421]: osdmap e90: 8 total, 8 up, 8 in 2024-09-20T18:51:01.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:00 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:01.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:00 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:01.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:00 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:51:01.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:00 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1498763017' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:51:01.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:00 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:51:01.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:00 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:51:01.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:00 smithi083 ceph-mon[36260]: osdmap e90: 8 total, 8 up, 8 in 2024-09-20T18:51:01.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:00 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:01.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:00 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:01.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:00 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:51:01.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:00 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1498763017' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:51:01.055 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:51:00 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:51:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:51:01.057 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:01.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:01.058 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:01.159 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:01.199 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:00 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:51:01.199 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:00 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:51:01.199 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:00 smithi142 ceph-mon[32508]: osdmap e90: 8 total, 8 up, 8 in 2024-09-20T18:51:01.199 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:00 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:01.200 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:00 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:01.200 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:00 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:51:01.200 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:00 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1498763017' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:51:01.200 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:00 smithi142 podman[61969]: 2024-09-20 18:51:00.840500837 +0000 UTC m=+2.924050398 image pull 72c9c208898624938c9e4183d6686ea4a5fd3f912bc29bc3f00147924c521a3e quay.io/prometheus/node-exporter:v1.7.0 2024-09-20T18:51:01.200 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:00 smithi142 podman[61969]: 2024-09-20 18:51:00.955254582 +0000 UTC m=+3.038804132 container create 04b8bd889b9399d276f21196e1d787b41e4e3ed213a8ffa2cd8adeedf42ab377 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-20T18:51:01.200 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 podman[61969]: 2024-09-20 18:51:01.182964037 +0000 UTC m=+3.266513588 container init 04b8bd889b9399d276f21196e1d787b41e4e3ed213a8ffa2cd8adeedf42ab377 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-20T18:51:01.201 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:01.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:01.202 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:01.303 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:01.342 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:01.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:01.343 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:01.444 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:01.476 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:01.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:01.477 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:01.513 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 4 16 12135 12119 189.343 178.75 0.0015367 0.00526672 2024-09-20T18:51:01.524 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 podman[61969]: 2024-09-20 18:51:01.186756235 +0000 UTC m=+3.270305786 container start 04b8bd889b9399d276f21196e1d787b41e4e3ed213a8ffa2cd8adeedf42ab377 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-20T18:51:01.524 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.191Z caller=node_exporter.go:192 level=info msg="Starting node_exporter" version="(version=1.7.0, branch=HEAD, revision=7333465abf9efba81876303bb57e6fadb946041b)" 2024-09-20T18:51:01.524 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.191Z caller=node_exporter.go:193 level=info msg="Build context" build_context="(go=go1.21.4, platform=linux/amd64, user=root@35918982f6d8, date=20231112-23:53:35, tags=netgo osusergo static_build)" 2024-09-20T18:51:01.524 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.192Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-20T18:51:01.524 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.192Z caller=diskstats_linux.go:265 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-20T18:51:01.524 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.192Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-20T18:51:01.524 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.192Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-20T18:51:01.524 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.193Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-20T18:51:01.524 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.193Z caller=node_exporter.go:117 level=info collector=arp 2024-09-20T18:51:01.525 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.193Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-20T18:51:01.525 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.193Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-20T18:51:01.525 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.193Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-20T18:51:01.525 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.193Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-20T18:51:01.525 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.193Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-20T18:51:01.525 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.193Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-20T18:51:01.525 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.193Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-20T18:51:01.525 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.193Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-20T18:51:01.526 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.193Z caller=node_exporter.go:117 level=info collector=edac 2024-09-20T18:51:01.526 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.193Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-20T18:51:01.526 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.193Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-20T18:51:01.526 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.193Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-20T18:51:01.526 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.193Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-20T18:51:01.526 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.193Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-20T18:51:01.526 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.193Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-20T18:51:01.526 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.193Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-20T18:51:01.526 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.193Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-20T18:51:01.526 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.193Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-20T18:51:01.526 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.193Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-20T18:51:01.526 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.193Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-20T18:51:01.526 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.193Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-20T18:51:01.527 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.193Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-20T18:51:01.527 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.193Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-20T18:51:01.527 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.193Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-20T18:51:01.527 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.193Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-20T18:51:01.527 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.193Z caller=node_exporter.go:117 level=info collector=os 2024-09-20T18:51:01.527 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.193Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-20T18:51:01.527 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.193Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-20T18:51:01.527 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.193Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-20T18:51:01.527 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.193Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-20T18:51:01.527 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.193Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-20T18:51:01.527 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.193Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-20T18:51:01.527 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.193Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-20T18:51:01.527 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.193Z caller=node_exporter.go:117 level=info collector=stat 2024-09-20T18:51:01.527 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.193Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-20T18:51:01.528 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.193Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-20T18:51:01.528 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.193Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-20T18:51:01.528 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.193Z caller=node_exporter.go:117 level=info collector=time 2024-09-20T18:51:01.528 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.193Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-20T18:51:01.528 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.193Z caller=node_exporter.go:117 level=info collector=uname 2024-09-20T18:51:01.528 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.193Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-20T18:51:01.528 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.193Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-20T18:51:01.528 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.193Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-20T18:51:01.528 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.193Z caller=tls_config.go:274 level=info msg="Listening on" address=[::]:9100 2024-09-20T18:51:01.528 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b[62211]: ts=2024-09-20T18:51:01.193Z caller=tls_config.go:277 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-20T18:51:01.528 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 bash[61969]: 04b8bd889b9399d276f21196e1d787b41e4e3ed213a8ffa2cd8adeedf42ab377 2024-09-20T18:51:01.528 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 18:51:01 smithi142 systemd[1]: Started Ceph node-exporter.b for 1ed5a50a-7780-11ef-baf5-efdc52797490. 2024-09-20T18:51:01.578 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:01.623 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:01.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:01.624 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:01.725 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:51:01.789 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:01.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:01.790 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:51:01.891 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:51:01.932 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:01.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:01.933 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:51:02.034 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:02.068 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:02.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:02.069 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:02.169 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:02.205 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:02.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:02.206 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:02.307 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:02.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:02.355 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:02.355 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:02.456 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:02.484 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:02.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:02.485 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:02.513 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 5 16 14718 14702 183.759 161.438 0.00383385 0.00543354 2024-09-20T18:51:02.586 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:51:02.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:02.626 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:02.626 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:51:02.683 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:02 smithi142 ceph-mon[32508]: pgmap v42: 148 pgs: 148 active+clean; 623 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 106 MiB/s wr, 1.69k op/s 2024-09-20T18:51:02.683 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:02 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:02.683 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:02 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:02.683 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:02 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:02.683 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:02 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:02.683 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:02 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:02.683 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:02 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:51:02.714 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:02 smithi083 ceph-mon[29421]: pgmap v42: 148 pgs: 148 active+clean; 623 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 106 MiB/s wr, 1.69k op/s 2024-09-20T18:51:02.714 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:02 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:02.714 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:02 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:02.714 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:02 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:02.714 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:02 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:02.714 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:02 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:02.714 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:02 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:51:02.714 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:02 smithi083 ceph-mon[36260]: pgmap v42: 148 pgs: 148 active+clean; 623 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 106 MiB/s wr, 1.69k op/s 2024-09-20T18:51:02.714 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:02 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:02.715 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:02 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:02.715 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:02 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:02.715 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:02 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:02.715 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:02 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:02.715 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:02 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:51:02.727 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:02.756 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:02.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:02.757 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:02.858 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:02.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:02.887 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:02.888 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:02.988 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:03.023 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:03.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:03.024 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:03.125 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:03.154 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:03.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:03.155 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:03.209 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T18:51:03.209 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-20T18:51:03.209 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-20T18:51:03.210 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-20T18:51:03.210 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.210 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14442.0:1 3.12 3:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-20T18:51:03.210 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:49:45.769138+0000", 2024-09-20T18:51:03.210 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 77.413799976999996, 2024-09-20T18:51:03.210 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.16996755799999999, 2024-09-20T18:51:03.210 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:03.210 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:03.210 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-20T18:51:03.210 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:51:03.210 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14442", 2024-09-20T18:51:03.210 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/3586269208", 2024-09-20T18:51:03.210 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 1 2024-09-20T18:51:03.210 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.210 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:03.211 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.211 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:03.211 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:49:45.769138+0000", 2024-09-20T18:51:03.211 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:03.211 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.211 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.211 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:03.211 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:49:45.769138+0000", 2024-09-20T18:51:03.211 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:03.211 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.211 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.211 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:03.211 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:49:45.769145+0000", 2024-09-20T18:51:03.211 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8450000000000003e-06 2024-09-20T18:51:03.212 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.212 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.212 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:03.212 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:49:45.769151+0000", 2024-09-20T18:51:03.212 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2989999999999996e-06 2024-09-20T18:51:03.212 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.212 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.212 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:03.212 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:49:45.769160+0000", 2024-09-20T18:51:03.212 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0089999999999996e-06 2024-09-20T18:51:03.212 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.212 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.212 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:03.212 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:49:45.769168+0000", 2024-09-20T18:51:03.212 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5490000000000006e-06 2024-09-20T18:51:03.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.213 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.213 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:03.213 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:49:45.769240+0000", 2024-09-20T18:51:03.213 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2081000000000003e-05 2024-09-20T18:51:03.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.213 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.213 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:51:03.213 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:49:45.769247+0000", 2024-09-20T18:51:03.213 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5309999999999998e-06 2024-09-20T18:51:03.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.213 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.213 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:03.213 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:49:45.939030+0000", 2024-09-20T18:51:03.214 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.169782715 2024-09-20T18:51:03.214 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.214 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.214 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:03.214 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:49:45.939105+0000", 2024-09-20T18:51:03.214 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5529000000000001e-05 2024-09-20T18:51:03.214 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:03.214 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:03.214 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:03.214 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.214 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.214 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:9600 7.f 7:fbac77ea:::benchmark_data_smithi083_70643_object9599:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-20T18:51:03.214 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:00.617413+0000", 2024-09-20T18:51:03.214 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 2.5655247550000002, 2024-09-20T18:51:03.215 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.029307698, 2024-09-20T18:51:03.215 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:03.215 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:03.215 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:51:03.215 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:51:03.215 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:51:03.215 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:51:03.215 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 9600 2024-09-20T18:51:03.215 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.215 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:03.215 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.215 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:03.215 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:00.617413+0000", 2024-09-20T18:51:03.215 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:03.215 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.215 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.216 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:03.216 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:00.617413+0000", 2024-09-20T18:51:03.216 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:03.216 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.216 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.216 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:03.216 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:00.617414+0000", 2024-09-20T18:51:03.216 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.063e-06 2024-09-20T18:51:03.216 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.216 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.216 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:03.216 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:00.617435+0000", 2024-09-20T18:51:03.216 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1243e-05 2024-09-20T18:51:03.216 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.216 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.217 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:03.217 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:00.617445+0000", 2024-09-20T18:51:03.217 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4939999999999992e-06 2024-09-20T18:51:03.217 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.217 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.217 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:03.217 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:00.617452+0000", 2024-09-20T18:51:03.217 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0010000000000004e-06 2024-09-20T18:51:03.217 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.217 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.217 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:03.217 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:00.617953+0000", 2024-09-20T18:51:03.217 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00050149199999999998 2024-09-20T18:51:03.217 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.217 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.217 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:03.218 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:00.631795+0000", 2024-09-20T18:51:03.218 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.013842254 2024-09-20T18:51:03.218 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.218 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.218 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:51:03.218 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:00.631994+0000", 2024-09-20T18:51:03.218 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019885700000000001 2024-09-20T18:51:03.218 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.218 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.218 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:03.218 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:00.636296+0000", 2024-09-20T18:51:03.218 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0043021939999999996 2024-09-20T18:51:03.218 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.218 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.218 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:51:03.219 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:00.636346+0000", 2024-09-20T18:51:03.219 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0000000000000002e-05 2024-09-20T18:51:03.219 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.219 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.219 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:03.219 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:00.646489+0000", 2024-09-20T18:51:03.219 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010142853 2024-09-20T18:51:03.219 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.219 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.219 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:51:03.219 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:00.646666+0000", 2024-09-20T18:51:03.219 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017694600000000001 2024-09-20T18:51:03.219 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.220 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.220 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:03.220 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:00.646720+0000", 2024-09-20T18:51:03.220 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4301000000000001e-05 2024-09-20T18:51:03.220 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:03.220 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:03.220 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:03.220 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.220 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.220 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:9591 7.7 e90/83 7:e0c6b621:::benchmark_data_smithi083_70643_object9590:head v 90'585, mlcod=90'583)", 2024-09-20T18:51:03.220 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:00.618070+0000", 2024-09-20T18:51:03.220 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 2.564867488, 2024-09-20T18:51:03.220 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.018312967999999999, 2024-09-20T18:51:03.220 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:03.221 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:03.221 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:51:03.221 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:03.221 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.221 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:03.221 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:00.618070+0000", 2024-09-20T18:51:03.221 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:03.221 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.221 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.221 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:03.221 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:00.618070+0000", 2024-09-20T18:51:03.221 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:03.221 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.221 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.221 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:03.222 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:00.618071+0000", 2024-09-20T18:51:03.222 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3200000000000005e-07 2024-09-20T18:51:03.222 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.222 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.222 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:03.222 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:00.618160+0000", 2024-09-20T18:51:03.222 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8911000000000003e-05 2024-09-20T18:51:03.222 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.222 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.222 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:03.222 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:00.631630+0000", 2024-09-20T18:51:03.222 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.013470436000000001 2024-09-20T18:51:03.222 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.222 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.223 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:03.223 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:00.631638+0000", 2024-09-20T18:51:03.223 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.6550000000000004e-06 2024-09-20T18:51:03.223 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.223 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.223 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:03.223 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:00.632536+0000", 2024-09-20T18:51:03.223 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00089827500000000003 2024-09-20T18:51:03.223 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.223 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.223 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:03.223 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:00.632553+0000", 2024-09-20T18:51:03.224 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7056999999999999e-05 2024-09-20T18:51:03.224 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.224 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.224 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:51:03.224 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:00.636347+0000", 2024-09-20T18:51:03.224 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0037943540000000002 2024-09-20T18:51:03.224 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.224 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.224 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:03.224 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:00.636383+0000", 2024-09-20T18:51:03.224 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5648000000000002e-05 2024-09-20T18:51:03.224 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:03.224 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:03.224 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:03.224 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.225 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.225 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:10961 7.6 7:65a56ae6:::benchmark_data_smithi083_70643_object10960:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-20T18:51:03.225 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:01.071626+0000", 2024-09-20T18:51:03.225 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 2.1113111670000002, 2024-09-20T18:51:03.225 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.018244902, 2024-09-20T18:51:03.225 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:03.225 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:03.225 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:51:03.225 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:51:03.225 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:51:03.225 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:51:03.225 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 10961 2024-09-20T18:51:03.225 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.225 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:03.225 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.225 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:03.226 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.071626+0000", 2024-09-20T18:51:03.226 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:03.226 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.226 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.226 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:03.226 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.071626+0000", 2024-09-20T18:51:03.226 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:03.226 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.226 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.226 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:03.226 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.071627+0000", 2024-09-20T18:51:03.226 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4399999999999998e-07 2024-09-20T18:51:03.226 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.227 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.227 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:03.227 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.071674+0000", 2024-09-20T18:51:03.227 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6669000000000003e-05 2024-09-20T18:51:03.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.227 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.227 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:03.227 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.071702+0000", 2024-09-20T18:51:03.227 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8157000000000002e-05 2024-09-20T18:51:03.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.227 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.227 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:03.227 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.071720+0000", 2024-09-20T18:51:03.227 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7915999999999999e-05 2024-09-20T18:51:03.228 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.228 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.228 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:03.228 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.071832+0000", 2024-09-20T18:51:03.228 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000112014 2024-09-20T18:51:03.228 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.228 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.228 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:03.228 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.071956+0000", 2024-09-20T18:51:03.228 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012392699999999999 2024-09-20T18:51:03.228 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.228 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.228 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:51:03.228 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.072081+0000", 2024-09-20T18:51:03.229 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012519399999999999 2024-09-20T18:51:03.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.229 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.229 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:51:03.229 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.072918+0000", 2024-09-20T18:51:03.229 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00083644199999999996 2024-09-20T18:51:03.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.229 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.229 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:03.229 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.074433+0000", 2024-09-20T18:51:03.229 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001515689 2024-09-20T18:51:03.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.229 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.229 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:03.230 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.089740+0000", 2024-09-20T18:51:03.230 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.015306594 2024-09-20T18:51:03.230 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.230 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.230 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:51:03.230 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.089813+0000", 2024-09-20T18:51:03.230 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2873000000000005e-05 2024-09-20T18:51:03.230 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.230 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.230 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:03.230 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.089871+0000", 2024-09-20T18:51:03.230 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8483000000000002e-05 2024-09-20T18:51:03.230 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:03.231 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:03.231 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:03.231 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.231 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.231 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:10967 7.6 7:689d62cb:::benchmark_data_smithi083_70643_object10966:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-20T18:51:03.231 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:01.073980+0000", 2024-09-20T18:51:03.231 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 2.1089572780000001, 2024-09-20T18:51:03.231 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.017492473000000001, 2024-09-20T18:51:03.231 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:03.231 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:03.231 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:51:03.231 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:51:03.232 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:51:03.232 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:51:03.232 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 10967 2024-09-20T18:51:03.232 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.232 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:03.232 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.232 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:03.232 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.073980+0000", 2024-09-20T18:51:03.232 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:03.232 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.232 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.232 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:03.232 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.073980+0000", 2024-09-20T18:51:03.233 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:03.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.233 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.233 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:03.233 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.073981+0000", 2024-09-20T18:51:03.233 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7000000000000005e-07 2024-09-20T18:51:03.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.233 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.233 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:03.233 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.074000+0000", 2024-09-20T18:51:03.233 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9453000000000002e-05 2024-09-20T18:51:03.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.233 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.234 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:03.234 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.074007+0000", 2024-09-20T18:51:03.234 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6989999999999997e-06 2024-09-20T18:51:03.234 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.234 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.234 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:03.234 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.074012+0000", 2024-09-20T18:51:03.234 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1669999999999998e-06 2024-09-20T18:51:03.234 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.234 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.234 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:03.234 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.074464+0000", 2024-09-20T18:51:03.235 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00045173199999999999 2024-09-20T18:51:03.235 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.235 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.235 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:03.235 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.074562+0000", 2024-09-20T18:51:03.235 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8363000000000007e-05 2024-09-20T18:51:03.235 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.235 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.235 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:51:03.235 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.074687+0000", 2024-09-20T18:51:03.235 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012478399999999999 2024-09-20T18:51:03.235 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.235 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.236 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:51:03.236 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.075529+0000", 2024-09-20T18:51:03.236 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084232899999999995 2024-09-20T18:51:03.236 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.236 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.236 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:03.236 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.076716+0000", 2024-09-20T18:51:03.236 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011869510000000001 2024-09-20T18:51:03.236 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.236 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.236 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:03.236 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.091389+0000", 2024-09-20T18:51:03.236 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.014673169999999999 2024-09-20T18:51:03.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.237 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.237 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:51:03.237 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.091435+0000", 2024-09-20T18:51:03.237 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6016999999999997e-05 2024-09-20T18:51:03.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.237 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.237 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:03.237 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.091473+0000", 2024-09-20T18:51:03.237 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7237999999999998e-05 2024-09-20T18:51:03.237 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:03.237 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:03.237 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:03.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.238 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.238 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:12107 7.6 7:6515ca77:::benchmark_data_smithi083_70643_object12106:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-20T18:51:03.238 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:01.484007+0000", 2024-09-20T18:51:03.238 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 1.6989306200000001, 2024-09-20T18:51:03.238 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.058406495000000003, 2024-09-20T18:51:03.238 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:03.238 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:03.238 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:51:03.238 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:51:03.238 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:51:03.238 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:51:03.238 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12107 2024-09-20T18:51:03.238 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.239 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:03.239 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.239 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:03.239 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.484007+0000", 2024-09-20T18:51:03.239 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:03.239 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.239 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.239 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:03.239 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.484007+0000", 2024-09-20T18:51:03.239 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:03.239 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.239 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.239 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:03.239 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.484008+0000", 2024-09-20T18:51:03.240 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.051e-06 2024-09-20T18:51:03.240 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.240 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.240 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:03.240 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.484028+0000", 2024-09-20T18:51:03.240 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0191999999999999e-05 2024-09-20T18:51:03.240 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.240 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.240 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:03.240 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.484041+0000", 2024-09-20T18:51:03.240 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2465e-05 2024-09-20T18:51:03.240 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.240 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.240 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:03.240 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.484048+0000", 2024-09-20T18:51:03.241 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7740000000000004e-06 2024-09-20T18:51:03.241 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.241 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.241 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:03.241 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.484605+0000", 2024-09-20T18:51:03.241 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00055631699999999997 2024-09-20T18:51:03.241 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.241 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.241 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:03.241 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.484706+0000", 2024-09-20T18:51:03.241 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010145900000000001 2024-09-20T18:51:03.241 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.241 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.241 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:51:03.241 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.485401+0000", 2024-09-20T18:51:03.242 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00069481199999999995 2024-09-20T18:51:03.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.242 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.242 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:51:03.242 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.486293+0000", 2024-09-20T18:51:03.242 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00089157199999999998 2024-09-20T18:51:03.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.242 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.242 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:03.242 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.486883+0000", 2024-09-20T18:51:03.242 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00059063799999999999 2024-09-20T18:51:03.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.242 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.242 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:03.243 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.542315+0000", 2024-09-20T18:51:03.243 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.055432197000000002 2024-09-20T18:51:03.243 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.243 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.243 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:51:03.243 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.542364+0000", 2024-09-20T18:51:03.243 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8241000000000001e-05 2024-09-20T18:51:03.243 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.243 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.243 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:03.243 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.542413+0000", 2024-09-20T18:51:03.243 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9777000000000001e-05 2024-09-20T18:51:03.243 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:03.243 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:03.243 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:03.244 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.244 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.244 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:12114 7.9 7:9b4d7c26:::benchmark_data_smithi083_70643_object12113:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-20T18:51:03.244 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:01.487468+0000", 2024-09-20T18:51:03.244 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 1.6954698349999999, 2024-09-20T18:51:03.244 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.055632143000000002, 2024-09-20T18:51:03.244 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:03.244 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:03.244 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:51:03.244 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:51:03.244 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:51:03.244 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:51:03.244 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12114 2024-09-20T18:51:03.244 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.244 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:03.245 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.245 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:03.245 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.487468+0000", 2024-09-20T18:51:03.245 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:03.245 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.245 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.245 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:03.245 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.487468+0000", 2024-09-20T18:51:03.245 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:03.245 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.245 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.245 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:03.245 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.487469+0000", 2024-09-20T18:51:03.245 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4300000000000001e-06 2024-09-20T18:51:03.246 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.246 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.246 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:03.246 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.487492+0000", 2024-09-20T18:51:03.246 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3198e-05 2024-09-20T18:51:03.246 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.246 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.246 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:03.246 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.487503+0000", 2024-09-20T18:51:03.246 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1100999999999999e-05 2024-09-20T18:51:03.246 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.246 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.246 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:03.247 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.487511+0000", 2024-09-20T18:51:03.247 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.204e-06 2024-09-20T18:51:03.247 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.247 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.247 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:03.247 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.487578+0000", 2024-09-20T18:51:03.247 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7251999999999999e-05 2024-09-20T18:51:03.247 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.247 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.247 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:03.247 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.487709+0000", 2024-09-20T18:51:03.247 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013133400000000001 2024-09-20T18:51:03.250 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.250 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.250 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,4", 2024-09-20T18:51:03.250 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.487939+0000", 2024-09-20T18:51:03.250 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022938099999999999 2024-09-20T18:51:03.250 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.251 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.251 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:03.251 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.542015+0000", 2024-09-20T18:51:03.251 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.054076732000000002 2024-09-20T18:51:03.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.251 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.251 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:51:03.251 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.542039+0000", 2024-09-20T18:51:03.251 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4017000000000001e-05 2024-09-20T18:51:03.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.251 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.251 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:03.251 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.543010+0000", 2024-09-20T18:51:03.251 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00097026800000000004 2024-09-20T18:51:03.252 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.252 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.252 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:51:03.252 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.543061+0000", 2024-09-20T18:51:03.252 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1870000000000003e-05 2024-09-20T18:51:03.252 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.252 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.252 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:03.252 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.543100+0000", 2024-09-20T18:51:03.252 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8355999999999997e-05 2024-09-20T18:51:03.252 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:03.252 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:03.252 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:03.253 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.253 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.253 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:12113 7.e e90/83 7:7e894313:::benchmark_data_smithi083_70643_object12112:head v 90'785, mlcod=90'783)", 2024-09-20T18:51:03.253 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:01.488130+0000", 2024-09-20T18:51:03.253 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 1.6948072349999999, 2024-09-20T18:51:03.254 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.054168239999999999, 2024-09-20T18:51:03.254 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:03.254 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:03.254 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:51:03.254 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:03.254 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.254 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:03.254 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.488130+0000", 2024-09-20T18:51:03.254 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:03.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.254 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.254 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:03.254 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.488130+0000", 2024-09-20T18:51:03.254 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:03.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.255 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.255 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:03.255 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.488131+0000", 2024-09-20T18:51:03.255 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9500000000000001e-07 2024-09-20T18:51:03.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.255 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.255 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:03.255 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.488153+0000", 2024-09-20T18:51:03.255 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2138999999999999e-05 2024-09-20T18:51:03.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.255 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.255 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:03.255 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.488162+0000", 2024-09-20T18:51:03.255 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2189999999999993e-06 2024-09-20T18:51:03.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.256 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.256 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:03.256 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.488169+0000", 2024-09-20T18:51:03.256 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3049999999999998e-06 2024-09-20T18:51:03.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.256 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.256 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:03.256 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.488312+0000", 2024-09-20T18:51:03.256 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014339100000000001 2024-09-20T18:51:03.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.256 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.256 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:03.256 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.488327+0000", 2024-09-20T18:51:03.256 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4585e-05 2024-09-20T18:51:03.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.257 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.257 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:51:03.257 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.542263+0000", 2024-09-20T18:51:03.257 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.053936034000000001 2024-09-20T18:51:03.257 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.257 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.257 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:03.257 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.542298+0000", 2024-09-20T18:51:03.257 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5772000000000002e-05 2024-09-20T18:51:03.257 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:03.257 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:03.257 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:03.257 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.257 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.258 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:12121 7.4 7:232caefc:::benchmark_data_smithi083_70643_object12120:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-20T18:51:03.258 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:01.488198+0000", 2024-09-20T18:51:03.258 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 1.6947399619999999, 2024-09-20T18:51:03.258 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.054392383000000002, 2024-09-20T18:51:03.258 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:03.258 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:03.258 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:51:03.258 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:51:03.258 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:51:03.258 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:51:03.258 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12121 2024-09-20T18:51:03.258 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.258 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:03.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.259 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:03.259 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.488198+0000", 2024-09-20T18:51:03.259 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:03.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.259 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:03.259 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.488198+0000", 2024-09-20T18:51:03.259 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:03.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.259 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:03.259 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.488198+0000", 2024-09-20T18:51:03.259 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1999999999999999e-07 2024-09-20T18:51:03.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.260 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.260 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:03.260 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.488217+0000", 2024-09-20T18:51:03.260 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9303000000000001e-05 2024-09-20T18:51:03.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.260 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.260 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:03.260 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.488225+0000", 2024-09-20T18:51:03.260 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.6020000000000002e-06 2024-09-20T18:51:03.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.260 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.260 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:03.260 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.488231+0000", 2024-09-20T18:51:03.260 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.48e-06 2024-09-20T18:51:03.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.261 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.261 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:03.261 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.489040+0000", 2024-09-20T18:51:03.261 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00080898099999999996 2024-09-20T18:51:03.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.261 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.261 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:03.261 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.489152+0000", 2024-09-20T18:51:03.261 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000112162 2024-09-20T18:51:03.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.261 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.261 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,4", 2024-09-20T18:51:03.261 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.489264+0000", 2024-09-20T18:51:03.261 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011273999999999999 2024-09-20T18:51:03.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.262 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.262 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:51:03.262 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.542321+0000", 2024-09-20T18:51:03.262 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.053057029999999998 2024-09-20T18:51:03.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.262 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.262 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:03.262 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.542358+0000", 2024-09-20T18:51:03.262 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6968999999999998e-05 2024-09-20T18:51:03.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.262 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.262 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:03.262 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.542516+0000", 2024-09-20T18:51:03.263 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000157202 2024-09-20T18:51:03.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.263 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.263 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:51:03.263 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.542558+0000", 2024-09-20T18:51:03.263 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2673999999999998e-05 2024-09-20T18:51:03.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.263 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.263 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:03.263 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.542590+0000", 2024-09-20T18:51:03.263 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1619999999999999e-05 2024-09-20T18:51:03.263 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:03.263 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:03.263 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:03.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.264 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:12122 7.6 7:6ed35bb6:::benchmark_data_smithi083_70643_object12121:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-20T18:51:03.264 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:01.488245+0000", 2024-09-20T18:51:03.264 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 1.6946920570000001, 2024-09-20T18:51:03.264 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.055287818000000002, 2024-09-20T18:51:03.264 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:03.264 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:03.264 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:51:03.264 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:51:03.264 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:51:03.264 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:51:03.264 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12122 2024-09-20T18:51:03.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.264 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:03.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.264 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:03.264 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.488245+0000", 2024-09-20T18:51:03.265 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:03.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.265 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:03.265 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.488245+0000", 2024-09-20T18:51:03.265 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:03.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.265 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:03.265 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.488246+0000", 2024-09-20T18:51:03.265 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2399999999999999e-07 2024-09-20T18:51:03.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.265 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:03.265 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.488264+0000", 2024-09-20T18:51:03.266 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7955e-05 2024-09-20T18:51:03.266 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.266 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.266 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:03.266 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.488270+0000", 2024-09-20T18:51:03.266 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7200000000000003e-06 2024-09-20T18:51:03.266 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.266 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.266 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:51:03.267 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:03.267 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.488274+0000", 2024-09-20T18:51:03.267 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7639999999999996e-06 2024-09-20T18:51:03.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.267 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.267 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:03.267 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.488394+0000", 2024-09-20T18:51:03.268 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000119219 2024-09-20T18:51:03.268 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.268 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.268 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:03.268 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.488514+0000", 2024-09-20T18:51:03.268 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000120097 2024-09-20T18:51:03.268 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.268 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.268 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:51:03.268 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.488640+0000", 2024-09-20T18:51:03.268 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000126406 2024-09-20T18:51:03.268 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.268 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.268 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:51:03.268 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.543367+0000", 2024-09-20T18:51:03.269 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.054727046000000001 2024-09-20T18:51:03.269 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.269 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.269 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:03.269 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.543416+0000", 2024-09-20T18:51:03.269 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8940000000000002e-05 2024-09-20T18:51:03.269 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.269 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.269 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:03.269 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.543450+0000", 2024-09-20T18:51:03.269 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3503000000000001e-05 2024-09-20T18:51:03.269 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.269 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.269 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:51:03.269 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.543498+0000", 2024-09-20T18:51:03.270 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8513999999999997e-05 2024-09-20T18:51:03.270 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.270 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.270 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:03.270 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.543533+0000", 2024-09-20T18:51:03.270 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.523e-05 2024-09-20T18:51:03.270 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:03.270 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:03.270 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:03.270 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.270 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.270 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:12125 7.6 7:6187ecb2:::benchmark_data_smithi083_70643_object12124:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-20T18:51:03.270 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:01.488281+0000", 2024-09-20T18:51:03.270 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 1.694656275, 2024-09-20T18:51:03.270 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.055725859000000003, 2024-09-20T18:51:03.271 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:03.271 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:03.271 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:51:03.271 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:51:03.271 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:51:03.271 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:51:03.271 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 12125 2024-09-20T18:51:03.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.271 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:03.271 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.271 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:03.271 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.488281+0000", 2024-09-20T18:51:03.271 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:03.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.272 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.272 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:03.272 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.488281+0000", 2024-09-20T18:51:03.272 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:03.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.272 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.272 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:03.272 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.488282+0000", 2024-09-20T18:51:03.272 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9299999999999999e-07 2024-09-20T18:51:03.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.272 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.272 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:03.272 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.488301+0000", 2024-09-20T18:51:03.272 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9531999999999999e-05 2024-09-20T18:51:03.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.273 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.273 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:03.273 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.488309+0000", 2024-09-20T18:51:03.273 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.486e-06 2024-09-20T18:51:03.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.273 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.273 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:03.273 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.488315+0000", 2024-09-20T18:51:03.273 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1750000000000002e-06 2024-09-20T18:51:03.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.273 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.273 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:03.273 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.541348+0000", 2024-09-20T18:51:03.273 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.053033410000000003 2024-09-20T18:51:03.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.274 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.274 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:03.274 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.541501+0000", 2024-09-20T18:51:03.274 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015299199999999999 2024-09-20T18:51:03.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.274 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.274 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:51:03.274 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.541660+0000", 2024-09-20T18:51:03.274 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015910100000000001 2024-09-20T18:51:03.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.274 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.274 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:03.274 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.543568+0000", 2024-09-20T18:51:03.275 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001907433 2024-09-20T18:51:03.275 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.275 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.275 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:51:03.275 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.543829+0000", 2024-09-20T18:51:03.275 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00026111100000000001 2024-09-20T18:51:03.275 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.275 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.275 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:03.275 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.543927+0000", 2024-09-20T18:51:03.275 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8638e-05 2024-09-20T18:51:03.275 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.275 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.275 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:51:03.275 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.543973+0000", 2024-09-20T18:51:03.275 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5073e-05 2024-09-20T18:51:03.276 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.276 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.276 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:03.276 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.544007+0000", 2024-09-20T18:51:03.276 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4514999999999997e-05 2024-09-20T18:51:03.276 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:03.276 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:03.276 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:03.276 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.276 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.276 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:12107 7.6 e90/83 ondisk, result = 0)", 2024-09-20T18:51:03.276 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:01.488980+0000", 2024-09-20T18:51:03.276 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 1.6939577969999999, 2024-09-20T18:51:03.276 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.053443826, 2024-09-20T18:51:03.276 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:03.277 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:03.277 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:51:03.277 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:03.277 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.277 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:03.277 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.488980+0000", 2024-09-20T18:51:03.277 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:03.277 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.277 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.277 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:03.277 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.488980+0000", 2024-09-20T18:51:03.277 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:03.277 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.277 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.277 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:03.277 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.488980+0000", 2024-09-20T18:51:03.278 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3900000000000004e-07 2024-09-20T18:51:03.278 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.278 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.278 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:03.278 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.488984+0000", 2024-09-20T18:51:03.278 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0029999999999996e-06 2024-09-20T18:51:03.278 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.278 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.278 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:03.278 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.488993+0000", 2024-09-20T18:51:03.278 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8040000000000007e-06 2024-09-20T18:51:03.278 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.278 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.278 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:03.279 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.488999+0000", 2024-09-20T18:51:03.279 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9699999999999996e-06 2024-09-20T18:51:03.279 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.279 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.279 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:03.279 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.542303+0000", 2024-09-20T18:51:03.279 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.053304091999999997 2024-09-20T18:51:03.279 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.279 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.279 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:03.279 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.542310+0000", 2024-09-20T18:51:03.279 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.737e-06 2024-09-20T18:51:03.279 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.279 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.279 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:03.279 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.542424+0000", 2024-09-20T18:51:03.280 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000113581 2024-09-20T18:51:03.280 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:03.280 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:03.280 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:03.280 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.280 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.280 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:12119 7.e e90/83 7:71cdccf8:::benchmark_data_smithi083_70643_object12118:head v 90'786, mlcod=90'783)", 2024-09-20T18:51:03.280 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:01.489024+0000", 2024-09-20T18:51:03.280 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 1.693913775, 2024-09-20T18:51:03.280 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.054844301999999998, 2024-09-20T18:51:03.280 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:03.280 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:03.280 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:51:03.280 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:03.280 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.280 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:03.281 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.489024+0000", 2024-09-20T18:51:03.281 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:03.281 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.281 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.281 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:03.281 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.489024+0000", 2024-09-20T18:51:03.281 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:03.281 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.281 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.281 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:03.281 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.489024+0000", 2024-09-20T18:51:03.281 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4000000000000002e-07 2024-09-20T18:51:03.281 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.281 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.281 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:03.282 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.489043+0000", 2024-09-20T18:51:03.282 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8487000000000001e-05 2024-09-20T18:51:03.282 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.282 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.282 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:03.282 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.489049+0000", 2024-09-20T18:51:03.282 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0229999999999998e-06 2024-09-20T18:51:03.282 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.282 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.282 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:03.282 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.489053+0000", 2024-09-20T18:51:03.282 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.51e-06 2024-09-20T18:51:03.282 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.282 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.283 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:03.283 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.542453+0000", 2024-09-20T18:51:03.283 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.053400239000000002 2024-09-20T18:51:03.283 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.283 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.283 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:03.283 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.542471+0000", 2024-09-20T18:51:03.283 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7218e-05 2024-09-20T18:51:03.283 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.283 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.283 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:51:03.283 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.543830+0000", 2024-09-20T18:51:03.284 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001359603 2024-09-20T18:51:03.284 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.284 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.284 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:03.284 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.543868+0000", 2024-09-20T18:51:03.284 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7781999999999998e-05 2024-09-20T18:51:03.284 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:03.284 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:03.284 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:03.284 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.284 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.284 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:12124 7.5 e90/83 7:aef2b662:::benchmark_data_smithi083_70643_object12123:head v 90'782, mlcod=90'780)", 2024-09-20T18:51:03.284 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:01.489661+0000", 2024-09-20T18:51:03.284 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 1.6932766429999999, 2024-09-20T18:51:03.285 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.053340977999999997, 2024-09-20T18:51:03.285 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:03.285 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:03.285 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:51:03.285 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:03.285 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.285 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:03.285 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.489661+0000", 2024-09-20T18:51:03.285 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:03.285 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.285 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.285 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:03.285 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.489661+0000", 2024-09-20T18:51:03.285 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:03.286 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.286 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.286 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:03.286 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.489661+0000", 2024-09-20T18:51:03.286 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3099999999999997e-07 2024-09-20T18:51:03.286 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.286 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.286 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:03.286 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.489692+0000", 2024-09-20T18:51:03.286 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0800999999999999e-05 2024-09-20T18:51:03.286 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.286 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.286 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:03.287 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.489702+0000", 2024-09-20T18:51:03.287 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3409999999999992e-06 2024-09-20T18:51:03.287 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.287 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.287 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:03.287 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.489708+0000", 2024-09-20T18:51:03.287 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1240000000000002e-06 2024-09-20T18:51:03.287 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.287 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.287 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:03.287 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.489819+0000", 2024-09-20T18:51:03.287 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000111081 2024-09-20T18:51:03.287 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.287 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.288 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:03.288 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.489836+0000", 2024-09-20T18:51:03.288 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7377e-05 2024-09-20T18:51:03.288 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.288 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.288 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:51:03.288 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.542965+0000", 2024-09-20T18:51:03.288 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.053128715 2024-09-20T18:51:03.288 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.288 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.288 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:03.288 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.543002+0000", 2024-09-20T18:51:03.288 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6908e-05 2024-09-20T18:51:03.288 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:03.288 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:03.288 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:03.289 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.289 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.289 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:12115 7.d e90/83 7:be4c8eac:::benchmark_data_smithi083_70643_object12114:head v 90'765, mlcod=90'763)", 2024-09-20T18:51:03.289 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:01.489816+0000", 2024-09-20T18:51:03.289 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 1.6931215929999999, 2024-09-20T18:51:03.289 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.054203746999999997, 2024-09-20T18:51:03.289 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:03.289 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:03.289 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:51:03.289 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:03.289 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.289 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:03.289 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.489816+0000", 2024-09-20T18:51:03.289 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:03.289 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.289 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.290 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:03.290 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.489816+0000", 2024-09-20T18:51:03.290 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:03.290 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.290 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.290 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:03.290 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.489817+0000", 2024-09-20T18:51:03.290 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3799999999999997e-07 2024-09-20T18:51:03.290 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.290 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.290 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:03.290 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.489859+0000", 2024-09-20T18:51:03.290 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.282e-05 2024-09-20T18:51:03.290 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.291 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.291 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:03.291 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.489869+0000", 2024-09-20T18:51:03.291 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2159999999999995e-06 2024-09-20T18:51:03.291 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.291 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.291 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:03.291 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.489875+0000", 2024-09-20T18:51:03.291 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9649999999999996e-06 2024-09-20T18:51:03.291 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.291 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.291 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:03.291 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.541741+0000", 2024-09-20T18:51:03.292 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.051866284999999998 2024-09-20T18:51:03.292 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.292 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.292 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:03.292 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.541761+0000", 2024-09-20T18:51:03.292 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0197e-05 2024-09-20T18:51:03.292 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.292 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.292 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:51:03.292 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.543900+0000", 2024-09-20T18:51:03.292 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0021385879999999999 2024-09-20T18:51:03.292 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.292 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.292 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:03.292 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.544020+0000", 2024-09-20T18:51:03.293 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012003800000000001 2024-09-20T18:51:03.293 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:03.293 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:03.293 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:03.293 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.293 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.293 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:12118 7.7 e90/83 7:e1f45e5b:::benchmark_data_smithi083_70643_object12117:head v 90'746, mlcod=90'744)", 2024-09-20T18:51:03.293 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:01.489889+0000", 2024-09-20T18:51:03.293 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 1.693048589, 2024-09-20T18:51:03.293 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.054057960000000002, 2024-09-20T18:51:03.293 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:03.293 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:03.293 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:51:03.293 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:03.293 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.293 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:03.294 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.489889+0000", 2024-09-20T18:51:03.294 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:03.294 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.294 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.294 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:03.294 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.489889+0000", 2024-09-20T18:51:03.294 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:03.294 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.294 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.294 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:03.294 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.489889+0000", 2024-09-20T18:51:03.294 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8000000000000001e-07 2024-09-20T18:51:03.294 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.294 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.295 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:03.295 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.489918+0000", 2024-09-20T18:51:03.295 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8821000000000001e-05 2024-09-20T18:51:03.295 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.295 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.295 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:03.295 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.489925+0000", 2024-09-20T18:51:03.295 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.63e-06 2024-09-20T18:51:03.295 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.295 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.295 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:03.295 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.489930+0000", 2024-09-20T18:51:03.295 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8099999999999997e-06 2024-09-20T18:51:03.295 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.295 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.295 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:03.296 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.541736+0000", 2024-09-20T18:51:03.296 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.051806898999999997 2024-09-20T18:51:03.296 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.296 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.296 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:03.296 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.541761+0000", 2024-09-20T18:51:03.296 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4989000000000001e-05 2024-09-20T18:51:03.296 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.296 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.296 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:51:03.296 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.543866+0000", 2024-09-20T18:51:03.296 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0021048849999999999 2024-09-20T18:51:03.296 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.296 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.297 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:03.297 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.543947+0000", 2024-09-20T18:51:03.297 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0545999999999998e-05 2024-09-20T18:51:03.297 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:03.297 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:03.297 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:03.297 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.297 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.297 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:12114 7.9 e90/83 ondisk, result = 0)", 2024-09-20T18:51:03.297 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:01.489953+0000", 2024-09-20T18:51:03.297 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 1.692984477, 2024-09-20T18:51:03.297 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.052069838, 2024-09-20T18:51:03.297 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:03.298 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:03.298 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:51:03.298 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:03.298 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.298 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:03.298 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.489953+0000", 2024-09-20T18:51:03.298 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:03.298 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.298 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.298 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:03.298 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.489953+0000", 2024-09-20T18:51:03.298 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:03.298 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.299 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.299 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:03.299 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.489953+0000", 2024-09-20T18:51:03.299 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2e-07 2024-09-20T18:51:03.299 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.299 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.299 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:03.299 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.489956+0000", 2024-09-20T18:51:03.299 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6089999999999999e-06 2024-09-20T18:51:03.299 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.299 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.299 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:03.299 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.489964+0000", 2024-09-20T18:51:03.299 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.1179999999999992e-06 2024-09-20T18:51:03.299 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.299 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.300 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:03.300 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.489969+0000", 2024-09-20T18:51:03.300 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6240000000000001e-06 2024-09-20T18:51:03.300 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.300 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.300 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:03.300 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.542000+0000", 2024-09-20T18:51:03.300 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.052031713 2024-09-20T18:51:03.300 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.300 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.300 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:03.300 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.542009+0000", 2024-09-20T18:51:03.300 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.1289999999999996e-06 2024-09-20T18:51:03.300 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.301 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.301 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:03.301 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.542023+0000", 2024-09-20T18:51:03.301 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4225e-05 2024-09-20T18:51:03.301 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:03.301 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:03.301 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:03.301 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.301 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.301 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:12116 7.d e90/83 7:b5863670:::benchmark_data_smithi083_70643_object12115:head v 90'766, mlcod=90'763)", 2024-09-20T18:51:03.301 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:01.490327+0000", 2024-09-20T18:51:03.301 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 1.6926100989999999, 2024-09-20T18:51:03.301 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.053716370999999999, 2024-09-20T18:51:03.301 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:03.301 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:03.302 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:51:03.302 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:03.302 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.302 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:03.302 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.490327+0000", 2024-09-20T18:51:03.302 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:03.302 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.302 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.302 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:03.302 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.490327+0000", 2024-09-20T18:51:03.302 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:03.302 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.302 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.302 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:03.302 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.490328+0000", 2024-09-20T18:51:03.303 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4499999999999997e-07 2024-09-20T18:51:03.303 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.303 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.303 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:03.303 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.490362+0000", 2024-09-20T18:51:03.303 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3664999999999997e-05 2024-09-20T18:51:03.303 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.303 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.303 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:03.303 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.541793+0000", 2024-09-20T18:51:03.303 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.051431078999999998 2024-09-20T18:51:03.303 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.303 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.303 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:03.303 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.541799+0000", 2024-09-20T18:51:03.303 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4300000000000003e-06 2024-09-20T18:51:03.304 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.304 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.304 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:03.304 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.542637+0000", 2024-09-20T18:51:03.304 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00083786099999999997 2024-09-20T18:51:03.304 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.304 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.304 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:03.304 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.542652+0000", 2024-09-20T18:51:03.304 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4642e-05 2024-09-20T18:51:03.304 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.304 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.304 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:51:03.304 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.544022+0000", 2024-09-20T18:51:03.304 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0013698370000000001 2024-09-20T18:51:03.304 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.305 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.305 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:03.305 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.544044+0000", 2024-09-20T18:51:03.305 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2212e-05 2024-09-20T18:51:03.305 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:03.305 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:03.305 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:03.305 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.305 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.305 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:12122 7.6 e90/83 ondisk, result = 0)", 2024-09-20T18:51:03.305 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:01.490394+0000", 2024-09-20T18:51:03.305 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 1.692543286, 2024-09-20T18:51:03.305 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.053144021, 2024-09-20T18:51:03.305 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:03.306 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:03.306 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:51:03.306 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:03.306 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.306 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:03.306 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.490394+0000", 2024-09-20T18:51:03.306 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:03.306 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.306 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.306 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:03.306 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.490394+0000", 2024-09-20T18:51:03.306 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:03.306 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.306 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.306 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:03.307 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.490395+0000", 2024-09-20T18:51:03.307 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5700000000000002e-07 2024-09-20T18:51:03.307 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.307 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.307 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:03.307 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.490399+0000", 2024-09-20T18:51:03.307 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7919999999999999e-06 2024-09-20T18:51:03.307 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.307 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.307 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:03.307 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.542341+0000", 2024-09-20T18:51:03.307 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.051942482999999998 2024-09-20T18:51:03.307 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.308 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.308 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:03.308 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.542349+0000", 2024-09-20T18:51:03.308 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7919999999999993e-06 2024-09-20T18:51:03.308 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.308 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.308 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:03.308 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.543440+0000", 2024-09-20T18:51:03.308 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0010913940000000001 2024-09-20T18:51:03.308 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.308 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.308 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:03.308 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.543445+0000", 2024-09-20T18:51:03.308 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0950000000000003e-06 2024-09-20T18:51:03.309 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.309 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.309 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:03.309 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.543538+0000", 2024-09-20T18:51:03.309 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2907999999999999e-05 2024-09-20T18:51:03.309 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:03.309 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:03.309 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:03.309 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.309 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.309 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:12123 7.e e90/83 7:745b1895:::benchmark_data_smithi083_70643_object12122:head v 90'787, mlcod=90'783)", 2024-09-20T18:51:03.309 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:01.490491+0000", 2024-09-20T18:51:03.309 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 1.692446543, 2024-09-20T18:51:03.309 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.057751351999999999, 2024-09-20T18:51:03.309 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:03.310 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:03.310 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:51:03.310 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:03.310 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.310 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:03.310 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.490491+0000", 2024-09-20T18:51:03.310 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:03.310 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.310 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.310 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:03.310 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.490491+0000", 2024-09-20T18:51:03.310 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:03.310 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.310 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.311 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:03.311 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.490492+0000", 2024-09-20T18:51:03.311 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9400000000000004e-07 2024-09-20T18:51:03.311 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.311 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.311 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:03.311 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.490517+0000", 2024-09-20T18:51:03.311 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4836000000000001e-05 2024-09-20T18:51:03.311 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.311 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.311 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:03.311 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.541501+0000", 2024-09-20T18:51:03.311 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050984759999999997 2024-09-20T18:51:03.312 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.312 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.312 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:03.312 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.541511+0000", 2024-09-20T18:51:03.312 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6700000000000006e-06 2024-09-20T18:51:03.312 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.312 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.312 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:03.312 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.542996+0000", 2024-09-20T18:51:03.312 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001484773 2024-09-20T18:51:03.312 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.312 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.312 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:03.312 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.543027+0000", 2024-09-20T18:51:03.312 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0803000000000003e-05 2024-09-20T18:51:03.313 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.313 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.313 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:51:03.313 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.548195+0000", 2024-09-20T18:51:03.313 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0051683170000000004 2024-09-20T18:51:03.313 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:03.313 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:03.313 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:03.313 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:01.548242+0000", 2024-09-20T18:51:03.313 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7398999999999998e-05 2024-09-20T18:51:03.313 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:03.313 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:03.313 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:03.313 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:03.314 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:03.314 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T18:51:03.314 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:03.343 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:03.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:03.344 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:03.389 INFO:teuthology.orchestra.run.smithi083.stderr:set 3.a pg_upmap_items mapping to [5->7,0->2,4->3] 2024-09-20T18:51:03.446 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:51:03.485 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:03.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:03.486 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:51:03.513 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 6 15 17113 17098 178.089 149.75 0.00420795 0.00560745 2024-09-20T18:51:03.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:03 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2079903827' entity='client.admin' cmd={"prefix": "osd pg-upmap-items", "pgid": "3.a", "id": [5, 7, 0, 2, 4, 3]} : dispatch 2024-09-20T18:51:03.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:03 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pg-upmap-items", "pgid": "3.a", "id": [5, 7, 0, 2, 4, 3]} : dispatch 2024-09-20T18:51:03.554 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:03 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2079903827' entity='client.admin' cmd={"prefix": "osd pg-upmap-items", "pgid": "3.a", "id": [5, 7, 0, 2, 4, 3]} : dispatch 2024-09-20T18:51:03.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:03 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pg-upmap-items", "pgid": "3.a", "id": [5, 7, 0, 2, 4, 3]} : dispatch 2024-09-20T18:51:03.587 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:51:03.623 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:03.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:03.624 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:51:03.725 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:03.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:03.770 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:03.771 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:03.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:03 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2079903827' entity='client.admin' cmd={"prefix": "osd pg-upmap-items", "pgid": "3.a", "id": [5, 7, 0, 2, 4, 3]} : dispatch 2024-09-20T18:51:03.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:03 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pg-upmap-items", "pgid": "3.a", "id": [5, 7, 0, 2, 4, 3]} : dispatch 2024-09-20T18:51:03.871 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:03.900 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:03.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:03.901 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:04.002 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:04.031 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:04.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:04.032 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:04.133 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:04.163 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:04.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:04.164 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:04.264 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:51:04.300 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:04.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:04.301 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:51:04.370 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T18:51:04.402 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:51:04.462 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:04.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:04.463 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:51:04.514 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 7 16 19599 19583 174.81 155.312 0.00572302 0.00571412 2024-09-20T18:51:04.564 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:51:04.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:04.604 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:04.605 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:51:04.705 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:51:04.757 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:04.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:04.758 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:51:04.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:04 smithi142 ceph-mon[32508]: pgmap v43: 148 pgs: 148 active+clean; 623 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 86 MiB/s wr, 1.38k op/s 2024-09-20T18:51:04.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:04 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "3.a", "id": [5, 7, 0, 2, 4, 3]}]': finished 2024-09-20T18:51:04.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:04 smithi142 ceph-mon[32508]: osdmap e91: 8 total, 8 up, 8 in 2024-09-20T18:51:04.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:04 smithi083 ceph-mon[29421]: pgmap v43: 148 pgs: 148 active+clean; 623 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 86 MiB/s wr, 1.38k op/s 2024-09-20T18:51:04.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:04 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "3.a", "id": [5, 7, 0, 2, 4, 3]}]': finished 2024-09-20T18:51:04.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:04 smithi083 ceph-mon[29421]: osdmap e91: 8 total, 8 up, 8 in 2024-09-20T18:51:04.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:04 smithi083 ceph-mon[36260]: pgmap v43: 148 pgs: 148 active+clean; 623 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 86 MiB/s wr, 1.38k op/s 2024-09-20T18:51:04.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:04 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "3.a", "id": [5, 7, 0, 2, 4, 3]}]': finished 2024-09-20T18:51:04.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:04 smithi083 ceph-mon[36260]: osdmap e91: 8 total, 8 up, 8 in 2024-09-20T18:51:04.860 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:04.892 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:04.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:04.893 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:04.993 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:05.023 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:05.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:05.024 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:05.125 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:05.154 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:05.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:05.155 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:05.256 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:51:05.294 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:05.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:05.295 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:51:05.394 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-20T18:51:05.395 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:05.421 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:05.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:05.422 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:05.515 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 8 16 22029 22013 171.934 151.875 0.0025323 0.00580912 2024-09-20T18:51:05.523 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:05.549 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:05.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:05.550 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:05.650 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:05 smithi083 ceph-mon[29421]: osdmap e92: 8 total, 8 up, 8 in 2024-09-20T18:51:05.650 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:05 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:05.650 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:05 smithi083 ceph-mon[36260]: osdmap e92: 8 total, 8 up, 8 in 2024-09-20T18:51:05.651 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:05 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:05.651 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:05.676 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:05.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:05.677 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:05.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:05 smithi142 ceph-mon[32508]: osdmap e92: 8 total, 8 up, 8 in 2024-09-20T18:51:05.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:05 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:05.778 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:05.806 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:05.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:05.807 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:05.836 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:51:05.908 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:05.947 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:05.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:05.948 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:06.049 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:51:06.088 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:06.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:06.089 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:51:06.190 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:06.228 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:06.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:06.229 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:06.330 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:06.367 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:06.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:06.368 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:06.468 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:51:06.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:06.508 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:06.509 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:51:06.514 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 9 16 23858 23842 165.531 114.312 0.00703574 0.0060346 2024-09-20T18:51:06.594 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-20T18:51:06.609 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:06.641 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:06.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:06.642 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:06.675 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:51:06 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:51:06.676 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:06 smithi142 ceph-mon[32508]: pgmap v46: 148 pgs: 148 active+clean; 623 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 104 MiB/s wr, 1.66k op/s 2024-09-20T18:51:06.676 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:06 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/496484424' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:51:06.676 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:06 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:51:06.743 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:06.768 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:06.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:06.769 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:06.870 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:06.905 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:06.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:06.906 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:07.007 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:07.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:07.035 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:07.035 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:07.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:06 smithi083 ceph-mon[29421]: pgmap v46: 148 pgs: 148 active+clean; 623 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 104 MiB/s wr, 1.66k op/s 2024-09-20T18:51:07.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:06 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/496484424' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:51:07.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:06 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:51:07.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:06 smithi083 ceph-mon[36260]: pgmap v46: 148 pgs: 148 active+clean; 623 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 104 MiB/s wr, 1.66k op/s 2024-09-20T18:51:07.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:06 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/496484424' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:51:07.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:06 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:51:07.136 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:07.164 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:07.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:07.166 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:07.266 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:07.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:07.298 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:07.298 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:07.399 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:51:07.420 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:07.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:07.421 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:51:07.515 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 10 16 25486 25470 159.153 101.75 0.00899698 0.00627717 2024-09-20T18:51:07.522 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:07.552 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:07.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:07.553 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:07.654 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:07.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:07.676 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:07.676 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:07.777 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:51:07.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:07.814 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:07.814 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:51:07.915 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:07.951 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:07.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:07.952 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:08.019 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:07 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:51:08.020 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:07 smithi142 ceph-mon[32508]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:51:08.020 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:07 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T18:51:08.020 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:07 smithi142 ceph-mon[32508]: osdmap e93: 8 total, 8 up, 8 in 2024-09-20T18:51:08.052 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:08.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:07 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:51:08.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:07 smithi083 ceph-mon[29421]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:51:08.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:07 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T18:51:08.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:07 smithi083 ceph-mon[29421]: osdmap e93: 8 total, 8 up, 8 in 2024-09-20T18:51:08.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:07 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:51:08.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:07 smithi083 ceph-mon[36260]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:51:08.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:07 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T18:51:08.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:07 smithi083 ceph-mon[36260]: osdmap e93: 8 total, 8 up, 8 in 2024-09-20T18:51:08.073 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:08.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:08.074 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:08.175 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:51:08.210 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:08.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:08.211 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:51:08.312 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:08.358 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:08.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:08.359 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:08.460 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:08.481 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:08.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:08.482 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:08.515 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 11 16 27046 27030 153.548 97.5 0.102708 0.00647149 2024-09-20T18:51:08.583 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:08.620 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:08.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:08.621 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:08.722 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:08.762 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:08.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:08.763 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:08.863 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:08.897 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:08.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:08.898 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:08.959 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:08 smithi142 ceph-mon[32508]: pgmap v48: 148 pgs: 148 active+clean; 1.4 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 853 B/s rd, 130 MiB/s wr, 2.08k op/s 2024-09-20T18:51:08.959 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:08 smithi142 ceph-mon[32508]: 4.19 scrub starts 2024-09-20T18:51:08.959 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:08 smithi142 ceph-mon[32508]: 4.19 scrub ok 2024-09-20T18:51:08.959 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:08 smithi142 ceph-mon[32508]: 6.f scrub starts 2024-09-20T18:51:08.959 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:08 smithi142 ceph-mon[32508]: 6.f scrub ok 2024-09-20T18:51:08.999 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:51:09.035 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:09.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:09.036 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:51:09.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:08 smithi083 ceph-mon[29421]: pgmap v48: 148 pgs: 148 active+clean; 1.4 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 853 B/s rd, 130 MiB/s wr, 2.08k op/s 2024-09-20T18:51:09.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:08 smithi083 ceph-mon[29421]: 4.19 scrub starts 2024-09-20T18:51:09.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:08 smithi083 ceph-mon[29421]: 4.19 scrub ok 2024-09-20T18:51:09.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:08 smithi083 ceph-mon[29421]: 6.f scrub starts 2024-09-20T18:51:09.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:08 smithi083 ceph-mon[29421]: 6.f scrub ok 2024-09-20T18:51:09.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:08 smithi083 ceph-mon[36260]: pgmap v48: 148 pgs: 148 active+clean; 1.4 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 853 B/s rd, 130 MiB/s wr, 2.08k op/s 2024-09-20T18:51:09.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:08 smithi083 ceph-mon[36260]: 4.19 scrub starts 2024-09-20T18:51:09.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:08 smithi083 ceph-mon[36260]: 4.19 scrub ok 2024-09-20T18:51:09.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:08 smithi083 ceph-mon[36260]: 6.f scrub starts 2024-09-20T18:51:09.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:08 smithi083 ceph-mon[36260]: 6.f scrub ok 2024-09-20T18:51:09.137 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:09.170 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:09.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:09.171 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:09.272 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:51:09.310 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:09.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:09.311 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:51:09.412 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:09.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:09.434 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:09.434 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:09.515 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 12 16 28989 28973 150.872 121.438 0.00667552 0.00662188 2024-09-20T18:51:09.536 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:09.571 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:09.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:09.572 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:09.673 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:09.701 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:09.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:09.702 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:09.803 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:51:09.844 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:09.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:09.845 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:51:09.946 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:09.957 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-20T18:51:09.957 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T18:51:09.958 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-20T18:51:09.958 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd dump -f json-pretty 2024-09-20T18:51:09.973 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:09.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:09.974 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:09.996 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:09 smithi083 ceph-mon[29421]: 6.15 scrub starts 2024-09-20T18:51:09.997 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:09 smithi083 ceph-mon[29421]: 6.15 scrub ok 2024-09-20T18:51:09.997 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:09 smithi083 ceph-mon[29421]: 4.1c scrub starts 2024-09-20T18:51:09.997 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:09 smithi083 ceph-mon[29421]: 4.1c scrub ok 2024-09-20T18:51:09.997 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:09 smithi083 ceph-mon[29421]: pgmap v49: 148 pgs: 148 active+clean; 1.4 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 853 B/s rd, 130 MiB/s wr, 2.08k op/s 2024-09-20T18:51:09.997 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:09 smithi083 ceph-mon[29421]: 5.18 scrub starts 2024-09-20T18:51:09.997 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:09 smithi083 ceph-mon[29421]: 5.18 scrub ok 2024-09-20T18:51:09.997 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:09 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:09.997 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:09 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:09.997 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:09 smithi083 ceph-mon[36260]: 6.15 scrub starts 2024-09-20T18:51:09.997 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:09 smithi083 ceph-mon[36260]: 6.15 scrub ok 2024-09-20T18:51:09.997 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:09 smithi083 ceph-mon[36260]: 4.1c scrub starts 2024-09-20T18:51:09.997 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:09 smithi083 ceph-mon[36260]: 4.1c scrub ok 2024-09-20T18:51:09.997 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:09 smithi083 ceph-mon[36260]: pgmap v49: 148 pgs: 148 active+clean; 1.4 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 853 B/s rd, 130 MiB/s wr, 2.08k op/s 2024-09-20T18:51:09.998 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:09 smithi083 ceph-mon[36260]: 5.18 scrub starts 2024-09-20T18:51:09.998 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:09 smithi083 ceph-mon[36260]: 5.18 scrub ok 2024-09-20T18:51:09.998 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:09 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:09.998 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:09 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:10.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:09 smithi142 ceph-mon[32508]: 6.15 scrub starts 2024-09-20T18:51:10.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:09 smithi142 ceph-mon[32508]: 6.15 scrub ok 2024-09-20T18:51:10.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:09 smithi142 ceph-mon[32508]: 4.1c scrub starts 2024-09-20T18:51:10.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:09 smithi142 ceph-mon[32508]: 4.1c scrub ok 2024-09-20T18:51:10.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:09 smithi142 ceph-mon[32508]: pgmap v49: 148 pgs: 148 active+clean; 1.4 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 853 B/s rd, 130 MiB/s wr, 2.08k op/s 2024-09-20T18:51:10.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:09 smithi142 ceph-mon[32508]: 5.18 scrub starts 2024-09-20T18:51:10.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:09 smithi142 ceph-mon[32508]: 5.18 scrub ok 2024-09-20T18:51:10.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:09 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:10.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:09 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:10.075 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:10.102 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:10.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:10.103 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:10.204 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:51:10.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:10.225 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:10.226 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:51:10.326 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:10.356 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:10.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:10.357 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:10.458 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:10.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:10.487 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:10.487 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:10.515 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 13 16 30778 30762 147.867 111.812 0.00646408 0.00675757 2024-09-20T18:51:10.589 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:10.624 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:10.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:10.625 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:10.726 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:10.747 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:10.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:10.748 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:10.848 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:10.889 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:10.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:10.890 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:10.991 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:11.019 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:11.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:11.020 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:11.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:10 smithi142 ceph-mon[32508]: 6.1c scrub starts 2024-09-20T18:51:11.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:10 smithi142 ceph-mon[32508]: 6.1c scrub ok 2024-09-20T18:51:11.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:10 smithi142 ceph-mon[32508]: 4.2 scrub starts 2024-09-20T18:51:11.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:10 smithi142 ceph-mon[32508]: 4.2 scrub ok 2024-09-20T18:51:11.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:10 smithi142 ceph-mon[32508]: 3.a scrub starts 2024-09-20T18:51:11.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:10 smithi142 ceph-mon[32508]: 3.a scrub ok 2024-09-20T18:51:11.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:10 smithi142 ceph-mon[32508]: osdmap e94: 8 total, 8 up, 8 in 2024-09-20T18:51:11.054 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:51:10 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:51:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:51:11.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:10 smithi083 ceph-mon[29421]: 6.1c scrub starts 2024-09-20T18:51:11.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:10 smithi083 ceph-mon[29421]: 6.1c scrub ok 2024-09-20T18:51:11.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:10 smithi083 ceph-mon[29421]: 4.2 scrub starts 2024-09-20T18:51:11.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:10 smithi083 ceph-mon[29421]: 4.2 scrub ok 2024-09-20T18:51:11.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:10 smithi083 ceph-mon[29421]: 3.a scrub starts 2024-09-20T18:51:11.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:10 smithi083 ceph-mon[29421]: 3.a scrub ok 2024-09-20T18:51:11.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:10 smithi083 ceph-mon[29421]: osdmap e94: 8 total, 8 up, 8 in 2024-09-20T18:51:11.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:10 smithi083 ceph-mon[36260]: 6.1c scrub starts 2024-09-20T18:51:11.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:10 smithi083 ceph-mon[36260]: 6.1c scrub ok 2024-09-20T18:51:11.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:10 smithi083 ceph-mon[36260]: 4.2 scrub starts 2024-09-20T18:51:11.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:10 smithi083 ceph-mon[36260]: 4.2 scrub ok 2024-09-20T18:51:11.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:10 smithi083 ceph-mon[36260]: 3.a scrub starts 2024-09-20T18:51:11.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:10 smithi083 ceph-mon[36260]: 3.a scrub ok 2024-09-20T18:51:11.057 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:10 smithi083 ceph-mon[36260]: osdmap e94: 8 total, 8 up, 8 in 2024-09-20T18:51:11.121 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:51:11.157 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:11.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:11.158 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:51:11.259 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:11.284 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:11.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:11.285 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:11.386 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:11.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:11.417 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:11.417 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:11.515 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 14 16 32416 32400 144.617 102.375 0.0133395 0.00690608 2024-09-20T18:51:11.518 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:11.545 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:11.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:11.546 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:11.647 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:11.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:11.678 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:11.678 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:11.779 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:51:11.799 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:11.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:11.800 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:51:11.901 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:11.931 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:11.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:11.932 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:12.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:11 smithi142 ceph-mon[32508]: 6.12 scrub starts 2024-09-20T18:51:12.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:11 smithi142 ceph-mon[32508]: 6.12 scrub ok 2024-09-20T18:51:12.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:11 smithi142 ceph-mon[32508]: 5.0 scrub starts 2024-09-20T18:51:12.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:11 smithi142 ceph-mon[32508]: 5.0 scrub ok 2024-09-20T18:51:12.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:11 smithi142 ceph-mon[32508]: pgmap v51: 148 pgs: 148 active+clean; 1.8 GiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 1.6 KiB/s rd, 186 MiB/s wr, 2.98k op/s 2024-09-20T18:51:12.033 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:12.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:11 smithi083 ceph-mon[29421]: 6.12 scrub starts 2024-09-20T18:51:12.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:11 smithi083 ceph-mon[29421]: 6.12 scrub ok 2024-09-20T18:51:12.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:11 smithi083 ceph-mon[29421]: 5.0 scrub starts 2024-09-20T18:51:12.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:11 smithi083 ceph-mon[29421]: 5.0 scrub ok 2024-09-20T18:51:12.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:11 smithi083 ceph-mon[29421]: pgmap v51: 148 pgs: 148 active+clean; 1.8 GiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 1.6 KiB/s rd, 186 MiB/s wr, 2.98k op/s 2024-09-20T18:51:12.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:11 smithi083 ceph-mon[36260]: 6.12 scrub starts 2024-09-20T18:51:12.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:11 smithi083 ceph-mon[36260]: 6.12 scrub ok 2024-09-20T18:51:12.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:11 smithi083 ceph-mon[36260]: 5.0 scrub starts 2024-09-20T18:51:12.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:11 smithi083 ceph-mon[36260]: 5.0 scrub ok 2024-09-20T18:51:12.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:11 smithi083 ceph-mon[36260]: pgmap v51: 148 pgs: 148 active+clean; 1.8 GiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 1.6 KiB/s rd, 186 MiB/s wr, 2.98k op/s 2024-09-20T18:51:12.062 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:12.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:12.063 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:12.164 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:12.194 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:12.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:12.195 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:12.295 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:12.322 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:12.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:12.323 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:12.424 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:12.452 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:12.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:12.453 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:12.484 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:51:12.515 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 15 16 34368 34352 143.109 122 0.0168603 0.00697955 2024-09-20T18:51:12.554 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:51:12.591 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:12.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:12.592 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:51:12.692 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:51:12.731 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:12.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:12.732 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:51:12.833 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:12.872 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:12.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:12.873 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:12.973 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:13.000 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:13.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:13.001 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:13.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:12 smithi142 ceph-mon[32508]: 3.13 scrub starts 2024-09-20T18:51:13.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:12 smithi142 ceph-mon[32508]: 3.13 scrub ok 2024-09-20T18:51:13.029 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:12 smithi083 ceph-mon[29421]: 3.13 scrub starts 2024-09-20T18:51:13.029 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:12 smithi083 ceph-mon[29421]: 3.13 scrub ok 2024-09-20T18:51:13.029 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:12 smithi083 ceph-mon[36260]: 3.13 scrub starts 2024-09-20T18:51:13.029 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:12 smithi083 ceph-mon[36260]: 3.13 scrub ok 2024-09-20T18:51:13.102 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:13.132 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:13.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:13.133 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:13.234 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:13.259 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:13.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:13.260 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:13.341 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:51:13.341 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:51:13.361 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:13.381 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:51:13.381 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:51:13.386 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:13.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:13.387 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:13.417 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:51:13.418 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:51:13.457 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:51:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:51:13.488 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:13.493 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:51:13.493 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:51:13.518 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 16 16 36082 36066 140.841 107.125 0.00712166 0.00709126 2024-09-20T18:51:13.533 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:51:13.533 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:51:13.535 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:13.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:13.536 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:13.586 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:51:13.586 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:51:13.619 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:51:13.620 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:51:13.637 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:13.664 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:13.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:13.665 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:13.766 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:51:13.806 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:13.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:13.807 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:51:13.845 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T18:51:13.845 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:51:13.908 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:13.935 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:13.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:13.936 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:13.967 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:13 smithi083 ceph-mon[29421]: 4.a scrub starts 2024-09-20T18:51:13.967 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:13 smithi083 ceph-mon[29421]: 4.a scrub ok 2024-09-20T18:51:13.967 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:13 smithi083 ceph-mon[29421]: pgmap v52: 148 pgs: 148 active+clean; 1.9 GiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 165 MiB/s wr, 2.64k op/s 2024-09-20T18:51:13.967 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:13 smithi083 ceph-mon[29421]: 6.19 scrub starts 2024-09-20T18:51:13.967 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:13 smithi083 ceph-mon[29421]: 6.19 scrub ok 2024-09-20T18:51:13.967 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:13 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/885605516' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:51:13.967 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:13 smithi083 ceph-mon[29421]: 6.1 scrub starts 2024-09-20T18:51:13.967 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:13 smithi083 ceph-mon[29421]: 6.1 scrub ok 2024-09-20T18:51:13.967 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:13 smithi083 ceph-mon[29421]: 5.e scrub starts 2024-09-20T18:51:13.967 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:13 smithi083 ceph-mon[29421]: 5.e scrub ok 2024-09-20T18:51:13.968 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:13 smithi083 ceph-mon[36260]: 4.a scrub starts 2024-09-20T18:51:13.968 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:13 smithi083 ceph-mon[36260]: 4.a scrub ok 2024-09-20T18:51:13.968 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:13 smithi083 ceph-mon[36260]: pgmap v52: 148 pgs: 148 active+clean; 1.9 GiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 165 MiB/s wr, 2.64k op/s 2024-09-20T18:51:13.968 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:13 smithi083 ceph-mon[36260]: 6.19 scrub starts 2024-09-20T18:51:13.968 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:13 smithi083 ceph-mon[36260]: 6.19 scrub ok 2024-09-20T18:51:13.968 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:13 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/885605516' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:51:13.968 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:13 smithi083 ceph-mon[36260]: 6.1 scrub starts 2024-09-20T18:51:13.968 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:13 smithi083 ceph-mon[36260]: 6.1 scrub ok 2024-09-20T18:51:13.968 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:13 smithi083 ceph-mon[36260]: 5.e scrub starts 2024-09-20T18:51:13.968 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:13 smithi083 ceph-mon[36260]: 5.e scrub ok 2024-09-20T18:51:14.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:13 smithi142 ceph-mon[32508]: 4.a scrub starts 2024-09-20T18:51:14.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:13 smithi142 ceph-mon[32508]: 4.a scrub ok 2024-09-20T18:51:14.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:13 smithi142 ceph-mon[32508]: pgmap v52: 148 pgs: 148 active+clean; 1.9 GiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 165 MiB/s wr, 2.64k op/s 2024-09-20T18:51:14.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:13 smithi142 ceph-mon[32508]: 6.19 scrub starts 2024-09-20T18:51:14.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:13 smithi142 ceph-mon[32508]: 6.19 scrub ok 2024-09-20T18:51:14.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:13 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/885605516' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:51:14.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:13 smithi142 ceph-mon[32508]: 6.1 scrub starts 2024-09-20T18:51:14.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:13 smithi142 ceph-mon[32508]: 6.1 scrub ok 2024-09-20T18:51:14.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:13 smithi142 ceph-mon[32508]: 5.e scrub starts 2024-09-20T18:51:14.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:13 smithi142 ceph-mon[32508]: 5.e scrub ok 2024-09-20T18:51:14.037 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:14.062 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:14.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:14.063 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:14.164 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:14.188 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:51:14.190 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:14.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:14.191 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:14.292 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:14.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:14.320 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:14.321 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:14.421 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:14.475 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:14.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:14.476 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:14.517 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 17 16 37664 37648 138.373 98.875 0.00836971 0.00722154 2024-09-20T18:51:14.577 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:14.605 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:14.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:14.606 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:14.707 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:14.733 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:14.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:14.735 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:14.835 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:14.862 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:14.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:14.863 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:14.964 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:15.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:15.003 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:15.003 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:15.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:14 smithi142 ceph-mon[32508]: 4.10 scrub starts 2024-09-20T18:51:15.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:14 smithi142 ceph-mon[32508]: 4.10 scrub ok 2024-09-20T18:51:15.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:14 smithi142 ceph-mon[32508]: 3.1d scrub starts 2024-09-20T18:51:15.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:14 smithi142 ceph-mon[32508]: 3.1d scrub ok 2024-09-20T18:51:15.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:14 smithi142 ceph-mon[32508]: 5.12 scrub starts 2024-09-20T18:51:15.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:14 smithi142 ceph-mon[32508]: 5.12 scrub ok 2024-09-20T18:51:15.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:14 smithi142 ceph-mon[32508]: 4.e scrub starts 2024-09-20T18:51:15.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:14 smithi142 ceph-mon[32508]: 4.e scrub ok 2024-09-20T18:51:15.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:14 smithi083 ceph-mon[29421]: 4.10 scrub starts 2024-09-20T18:51:15.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:14 smithi083 ceph-mon[29421]: 4.10 scrub ok 2024-09-20T18:51:15.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:14 smithi083 ceph-mon[29421]: 3.1d scrub starts 2024-09-20T18:51:15.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:14 smithi083 ceph-mon[29421]: 3.1d scrub ok 2024-09-20T18:51:15.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:14 smithi083 ceph-mon[29421]: 5.12 scrub starts 2024-09-20T18:51:15.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:14 smithi083 ceph-mon[29421]: 5.12 scrub ok 2024-09-20T18:51:15.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:14 smithi083 ceph-mon[29421]: 4.e scrub starts 2024-09-20T18:51:15.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:14 smithi083 ceph-mon[29421]: 4.e scrub ok 2024-09-20T18:51:15.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:14 smithi083 ceph-mon[36260]: 4.10 scrub starts 2024-09-20T18:51:15.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:14 smithi083 ceph-mon[36260]: 4.10 scrub ok 2024-09-20T18:51:15.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:14 smithi083 ceph-mon[36260]: 3.1d scrub starts 2024-09-20T18:51:15.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:14 smithi083 ceph-mon[36260]: 3.1d scrub ok 2024-09-20T18:51:15.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:14 smithi083 ceph-mon[36260]: 5.12 scrub starts 2024-09-20T18:51:15.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:14 smithi083 ceph-mon[36260]: 5.12 scrub ok 2024-09-20T18:51:15.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:14 smithi083 ceph-mon[36260]: 4.e scrub starts 2024-09-20T18:51:15.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:14 smithi083 ceph-mon[36260]: 4.e scrub ok 2024-09-20T18:51:15.088 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T18:51:15.104 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:15.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:15.174 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:15.174 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:15.275 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:15.297 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:15.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:15.298 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:15.399 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:15.426 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:15.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:15.427 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:15.526 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 18 16 39272 39256 136.269 100.5 0.00732984 0.00733079 2024-09-20T18:51:15.528 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:15.558 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:15.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:15.559 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:15.660 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:51:15.695 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:15.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:15.696 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:51:15.798 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:51:15.836 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:15.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:15.837 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:51:15.938 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:51:15.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:15.979 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:15.979 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:51:16.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:15 smithi142 ceph-mon[32508]: 6.11 scrub starts 2024-09-20T18:51:16.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:15 smithi142 ceph-mon[32508]: 6.11 scrub ok 2024-09-20T18:51:16.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:15 smithi142 ceph-mon[32508]: 4.1f scrub starts 2024-09-20T18:51:16.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:15 smithi142 ceph-mon[32508]: 4.1f scrub ok 2024-09-20T18:51:16.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:15 smithi142 ceph-mon[32508]: pgmap v53: 148 pgs: 148 active+clean; 1.9 GiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 624 B/s rd, 66 MiB/s wr, 1.05k op/s 2024-09-20T18:51:16.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:15 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:51:16.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:15 smithi142 ceph-mon[32508]: 6.2 scrub starts 2024-09-20T18:51:16.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:15 smithi142 ceph-mon[32508]: 6.2 scrub ok 2024-09-20T18:51:16.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:15 smithi083 ceph-mon[29421]: 6.11 scrub starts 2024-09-20T18:51:16.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:15 smithi083 ceph-mon[29421]: 6.11 scrub ok 2024-09-20T18:51:16.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:15 smithi083 ceph-mon[29421]: 4.1f scrub starts 2024-09-20T18:51:16.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:15 smithi083 ceph-mon[29421]: 4.1f scrub ok 2024-09-20T18:51:16.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:15 smithi083 ceph-mon[29421]: pgmap v53: 148 pgs: 148 active+clean; 1.9 GiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 624 B/s rd, 66 MiB/s wr, 1.05k op/s 2024-09-20T18:51:16.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:15 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:51:16.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:15 smithi083 ceph-mon[29421]: 6.2 scrub starts 2024-09-20T18:51:16.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:15 smithi083 ceph-mon[29421]: 6.2 scrub ok 2024-09-20T18:51:16.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:15 smithi083 ceph-mon[36260]: 6.11 scrub starts 2024-09-20T18:51:16.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:15 smithi083 ceph-mon[36260]: 6.11 scrub ok 2024-09-20T18:51:16.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:15 smithi083 ceph-mon[36260]: 4.1f scrub starts 2024-09-20T18:51:16.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:15 smithi083 ceph-mon[36260]: 4.1f scrub ok 2024-09-20T18:51:16.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:15 smithi083 ceph-mon[36260]: pgmap v53: 148 pgs: 148 active+clean; 1.9 GiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 624 B/s rd, 66 MiB/s wr, 1.05k op/s 2024-09-20T18:51:16.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:15 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:51:16.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:15 smithi083 ceph-mon[36260]: 6.2 scrub starts 2024-09-20T18:51:16.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:15 smithi083 ceph-mon[36260]: 6.2 scrub ok 2024-09-20T18:51:16.080 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:16.122 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:16.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:16.124 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:16.224 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:51:16.253 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:16.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:16.255 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:51:16.355 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:16.384 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:16.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:16.385 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:16.486 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:51:16.518 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 19 15 40849 40834 134.287 98.625 0.0119627 0.00744137 2024-09-20T18:51:16.523 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:16.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:16.525 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:51:16.625 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:16.664 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:16.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:16.665 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:16.714 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:51:16.715 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T18:51:16.715 INFO:teuthology.orchestra.run.smithi083.stdout: "epoch": 95, 2024-09-20T18:51:16.715 INFO:teuthology.orchestra.run.smithi083.stdout: "fsid": "1ed5a50a-7780-11ef-baf5-efdc52797490", 2024-09-20T18:51:16.715 INFO:teuthology.orchestra.run.smithi083.stdout: "created": "2024-09-20T18:44:10.399800+0000", 2024-09-20T18:51:16.715 INFO:teuthology.orchestra.run.smithi083.stdout: "modified": "2024-09-20T18:51:15.713595+0000", 2024-09-20T18:51:16.715 INFO:teuthology.orchestra.run.smithi083.stdout: "last_up_change": "2024-09-20T18:49:34.836377+0000", 2024-09-20T18:51:16.715 INFO:teuthology.orchestra.run.smithi083.stdout: "last_in_change": "2024-09-20T18:49:12.342332+0000", 2024-09-20T18:51:16.715 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-20T18:51:16.715 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_num": 5804032, 2024-09-20T18:51:16.715 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_set": [ 2024-09-20T18:51:16.716 INFO:teuthology.orchestra.run.smithi083.stdout: "nodeep-scrub", 2024-09-20T18:51:16.716 INFO:teuthology.orchestra.run.smithi083.stdout: "pglog_hardlimit", 2024-09-20T18:51:16.716 INFO:teuthology.orchestra.run.smithi083.stdout: "purged_snapdirs", 2024-09-20T18:51:16.716 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_deletes", 2024-09-20T18:51:16.716 INFO:teuthology.orchestra.run.smithi083.stdout: "sortbitwise" 2024-09-20T18:51:16.716 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T18:51:16.716 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_version": 18, 2024-09-20T18:51:16.716 INFO:teuthology.orchestra.run.smithi083.stdout: "full_ratio": 0.94999998807907104, 2024-09-20T18:51:16.716 INFO:teuthology.orchestra.run.smithi083.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-20T18:51:16.716 INFO:teuthology.orchestra.run.smithi083.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-20T18:51:16.716 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_snapshot": "", 2024-09-20T18:51:16.716 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_max": 7, 2024-09-20T18:51:16.717 INFO:teuthology.orchestra.run.smithi083.stdout: "max_osd": 8, 2024-09-20T18:51:16.717 INFO:teuthology.orchestra.run.smithi083.stdout: "require_min_compat_client": "luminous", 2024-09-20T18:51:16.717 INFO:teuthology.orchestra.run.smithi083.stdout: "min_compat_client": "luminous", 2024-09-20T18:51:16.717 INFO:teuthology.orchestra.run.smithi083.stdout: "require_osd_release": "squid", 2024-09-20T18:51:16.717 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_crimson": false, 2024-09-20T18:51:16.717 INFO:teuthology.orchestra.run.smithi083.stdout: "pools": [ 2024-09-20T18:51:16.717 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.717 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 1, 2024-09-20T18:51:16.717 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": ".mgr", 2024-09-20T18:51:16.717 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-20T18:47:15.285795+0000", 2024-09-20T18:51:16.717 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-20T18:51:16.717 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool", 2024-09-20T18:51:16.717 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-20T18:51:16.718 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 3, 2024-09-20T18:51:16.718 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 2, 2024-09-20T18:51:16.718 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-20T18:51:16.718 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-20T18:51:16.718 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-20T18:51:16.718 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T18:51:16.718 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T18:51:16.718 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-20T18:51:16.718 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-20T18:51:16.718 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-20T18:51:16.718 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 1, 2024-09-20T18:51:16.719 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 1, 2024-09-20T18:51:16.719 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 1, 2024-09-20T18:51:16.719 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 1, 2024-09-20T18:51:16.719 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 1, 2024-09-20T18:51:16.719 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-20T18:51:16.719 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-20T18:51:16.719 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-20T18:51:16.719 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-20T18:51:16.719 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-20T18:51:16.719 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-20T18:51:16.719 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-20T18:51:16.720 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.720 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "21", 2024-09-20T18:51:16.720 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-20T18:51:16.720 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T18:51:16.720 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T18:51:16.720 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-20T18:51:16.720 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-20T18:51:16.721 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 0, 2024-09-20T18:51:16.721 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 0, 2024-09-20T18:51:16.721 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-20T18:51:16.721 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-20T18:51:16.721 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-20T18:51:16.721 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-20T18:51:16.721 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-20T18:51:16.722 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-20T18:51:16.722 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-20T18:51:16.722 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-20T18:51:16.722 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-20T18:51:16.722 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-20T18:51:16.722 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-20T18:51:16.722 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T18:51:16.722 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T18:51:16.723 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T18:51:16.723 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-20T18:51:16.723 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-20T18:51:16.723 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-20T18:51:16.723 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-20T18:51:16.723 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-20T18:51:16.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.723 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-20T18:51:16.723 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-20T18:51:16.723 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-20T18:51:16.723 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-20T18:51:16.724 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-20T18:51:16.724 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T18:51:16.724 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-20T18:51:16.724 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-20T18:51:16.724 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-20T18:51:16.724 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-20T18:51:16.724 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-20T18:51:16.724 INFO:teuthology.orchestra.run.smithi083.stdout: "options": { 2024-09-20T18:51:16.724 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_max": 32, 2024-09-20T18:51:16.724 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_min": 1 2024-09-20T18:51:16.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.725 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-20T18:51:16.725 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": {} 2024-09-20T18:51:16.725 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.725 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-20T18:51:16.725 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-20T18:51:16.725 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 7.8899998664855957, 2024-09-20T18:51:16.725 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 7.8899998664855957, 2024-09-20T18:51:16.725 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 0.37999999523162842, 2024-09-20T18:51:16.725 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 3, 2024-09-20T18:51:16.725 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 3, 2024-09-20T18:51:16.726 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 1, 2024-09-20T18:51:16.726 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 1, 2024-09-20T18:51:16.726 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 1 2024-09-20T18:51:16.726 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:16.726 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.726 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.726 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 2, 2024-09-20T18:51:16.726 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": "datapool", 2024-09-20T18:51:16.726 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-20T18:49:43.452984+0000", 2024-09-20T18:51:16.726 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 8193, 2024-09-20T18:51:16.726 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-20T18:51:16.727 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-20T18:51:16.727 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 3, 2024-09-20T18:51:16.727 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 2, 2024-09-20T18:51:16.727 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-20T18:51:16.727 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-20T18:51:16.727 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-20T18:51:16.727 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T18:51:16.727 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T18:51:16.727 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-20T18:51:16.728 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-20T18:51:16.728 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-20T18:51:16.728 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 3, 2024-09-20T18:51:16.728 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 3, 2024-09-20T18:51:16.728 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 3, 2024-09-20T18:51:16.728 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 3, 2024-09-20T18:51:16.728 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 3, 2024-09-20T18:51:16.728 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-20T18:51:16.728 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-20T18:51:16.728 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-20T18:51:16.728 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-20T18:51:16.728 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-20T18:51:16.729 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-20T18:51:16.729 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-20T18:51:16.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.729 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "61", 2024-09-20T18:51:16.729 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-20T18:51:16.729 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T18:51:16.729 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T18:51:16.729 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-20T18:51:16.729 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-20T18:51:16.729 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 2, 2024-09-20T18:51:16.729 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 61, 2024-09-20T18:51:16.730 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-20T18:51:16.730 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-20T18:51:16.730 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-20T18:51:16.730 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-20T18:51:16.730 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-20T18:51:16.730 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-20T18:51:16.730 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-20T18:51:16.730 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-20T18:51:16.730 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-20T18:51:16.730 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-20T18:51:16.730 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-20T18:51:16.731 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T18:51:16.731 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T18:51:16.731 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T18:51:16.731 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-20T18:51:16.731 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-20T18:51:16.731 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-20T18:51:16.731 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-20T18:51:16.731 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-20T18:51:16.731 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.731 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-20T18:51:16.731 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-20T18:51:16.732 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-20T18:51:16.732 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-20T18:51:16.732 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-20T18:51:16.732 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T18:51:16.732 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-20T18:51:16.732 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-20T18:51:16.732 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-20T18:51:16.733 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-20T18:51:16.733 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-20T18:51:16.733 INFO:teuthology.orchestra.run.smithi083.stdout: "options": {}, 2024-09-20T18:51:16.733 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-20T18:51:16.733 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd": {} 2024-09-20T18:51:16.733 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.733 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-20T18:51:16.733 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-20T18:51:16.733 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 2.6500000953674316, 2024-09-20T18:51:16.733 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 2.6500000953674316, 2024-09-20T18:51:16.733 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 0.87999999523162842, 2024-09-20T18:51:16.733 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 2.3299999237060547, 2024-09-20T18:51:16.733 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 2.3299999237060547, 2024-09-20T18:51:16.734 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 1, 2024-09-20T18:51:16.734 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 1, 2024-09-20T18:51:16.734 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 1 2024-09-20T18:51:16.734 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:16.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.734 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.734 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 3, 2024-09-20T18:51:16.734 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": ".rgw.root", 2024-09-20T18:51:16.734 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-20T18:49:44.217011+0000", 2024-09-20T18:51:16.734 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-20T18:51:16.734 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool", 2024-09-20T18:51:16.734 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-20T18:51:16.735 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 3, 2024-09-20T18:51:16.735 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 2, 2024-09-20T18:51:16.735 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-20T18:51:16.735 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-20T18:51:16.735 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-20T18:51:16.735 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T18:51:16.735 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T18:51:16.735 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-20T18:51:16.735 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-20T18:51:16.735 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-20T18:51:16.736 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 32, 2024-09-20T18:51:16.736 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 32, 2024-09-20T18:51:16.736 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 32, 2024-09-20T18:51:16.736 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 32, 2024-09-20T18:51:16.736 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 32, 2024-09-20T18:51:16.736 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-20T18:51:16.736 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-20T18:51:16.736 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-20T18:51:16.736 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-20T18:51:16.736 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-20T18:51:16.736 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-20T18:51:16.736 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-20T18:51:16.736 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.737 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "60", 2024-09-20T18:51:16.737 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-20T18:51:16.737 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T18:51:16.737 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T18:51:16.737 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-20T18:51:16.737 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-20T18:51:16.737 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 0, 2024-09-20T18:51:16.737 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 0, 2024-09-20T18:51:16.737 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-20T18:51:16.737 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-20T18:51:16.737 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-20T18:51:16.738 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-20T18:51:16.738 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-20T18:51:16.738 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-20T18:51:16.738 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-20T18:51:16.738 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-20T18:51:16.738 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-20T18:51:16.738 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-20T18:51:16.738 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-20T18:51:16.738 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T18:51:16.738 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T18:51:16.739 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T18:51:16.739 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-20T18:51:16.739 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-20T18:51:16.739 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-20T18:51:16.739 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-20T18:51:16.739 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-20T18:51:16.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.739 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-20T18:51:16.739 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-20T18:51:16.739 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-20T18:51:16.739 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-20T18:51:16.739 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-20T18:51:16.740 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T18:51:16.740 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-20T18:51:16.740 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-20T18:51:16.740 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-20T18:51:16.740 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-20T18:51:16.740 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-20T18:51:16.740 INFO:teuthology.orchestra.run.smithi083.stdout: "options": {}, 2024-09-20T18:51:16.740 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-20T18:51:16.740 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw": {} 2024-09-20T18:51:16.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.740 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-20T18:51:16.740 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-20T18:51:16.741 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 1.5, 2024-09-20T18:51:16.741 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 1.5, 2024-09-20T18:51:16.741 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 1, 2024-09-20T18:51:16.741 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 1.5, 2024-09-20T18:51:16.741 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 1.5, 2024-09-20T18:51:16.741 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 1, 2024-09-20T18:51:16.741 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 1, 2024-09-20T18:51:16.741 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 1 2024-09-20T18:51:16.741 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:16.741 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.741 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.741 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 4, 2024-09-20T18:51:16.742 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": "default.rgw.log", 2024-09-20T18:51:16.742 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-20T18:49:46.007275+0000", 2024-09-20T18:51:16.742 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-20T18:51:16.742 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool", 2024-09-20T18:51:16.742 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-20T18:51:16.742 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 3, 2024-09-20T18:51:16.743 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 2, 2024-09-20T18:51:16.743 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-20T18:51:16.743 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-20T18:51:16.743 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-20T18:51:16.743 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T18:51:16.743 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T18:51:16.743 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-20T18:51:16.743 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-20T18:51:16.743 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-20T18:51:16.743 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 32, 2024-09-20T18:51:16.743 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 32, 2024-09-20T18:51:16.744 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 32, 2024-09-20T18:51:16.744 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 32, 2024-09-20T18:51:16.744 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 32, 2024-09-20T18:51:16.744 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-20T18:51:16.744 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-20T18:51:16.744 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-20T18:51:16.744 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-20T18:51:16.744 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-20T18:51:16.744 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-20T18:51:16.744 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-20T18:51:16.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.744 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "62", 2024-09-20T18:51:16.745 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-20T18:51:16.745 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T18:51:16.745 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T18:51:16.745 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-20T18:51:16.745 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-20T18:51:16.745 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 0, 2024-09-20T18:51:16.745 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 0, 2024-09-20T18:51:16.745 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-20T18:51:16.745 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-20T18:51:16.745 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-20T18:51:16.745 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-20T18:51:16.745 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-20T18:51:16.746 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-20T18:51:16.746 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-20T18:51:16.746 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-20T18:51:16.746 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-20T18:51:16.746 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-20T18:51:16.746 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-20T18:51:16.746 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T18:51:16.746 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T18:51:16.746 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T18:51:16.746 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-20T18:51:16.746 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-20T18:51:16.747 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-20T18:51:16.747 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-20T18:51:16.747 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-20T18:51:16.747 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.747 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-20T18:51:16.747 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-20T18:51:16.747 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-20T18:51:16.747 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-20T18:51:16.747 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-20T18:51:16.747 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T18:51:16.747 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-20T18:51:16.747 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-20T18:51:16.748 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-20T18:51:16.748 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-20T18:51:16.748 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-20T18:51:16.748 INFO:teuthology.orchestra.run.smithi083.stdout: "options": {}, 2024-09-20T18:51:16.748 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-20T18:51:16.748 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw": {} 2024-09-20T18:51:16.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.749 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-20T18:51:16.749 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-20T18:51:16.749 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 2.25, 2024-09-20T18:51:16.750 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 2.25, 2024-09-20T18:51:16.750 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 1, 2024-09-20T18:51:16.750 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 2.25, 2024-09-20T18:51:16.750 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 2.25, 2024-09-20T18:51:16.750 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 1, 2024-09-20T18:51:16.750 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 1, 2024-09-20T18:51:16.750 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 1 2024-09-20T18:51:16.750 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:16.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.750 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.750 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 5, 2024-09-20T18:51:16.751 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": "default.rgw.control", 2024-09-20T18:51:16.751 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-20T18:49:47.961258+0000", 2024-09-20T18:51:16.751 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-20T18:51:16.751 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool", 2024-09-20T18:51:16.751 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-20T18:51:16.751 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 3, 2024-09-20T18:51:16.751 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 2, 2024-09-20T18:51:16.751 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-20T18:51:16.751 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-20T18:51:16.751 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-20T18:51:16.751 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T18:51:16.751 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T18:51:16.751 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-20T18:51:16.751 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-20T18:51:16.752 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-20T18:51:16.752 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 32, 2024-09-20T18:51:16.752 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 32, 2024-09-20T18:51:16.752 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 32, 2024-09-20T18:51:16.752 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 32, 2024-09-20T18:51:16.752 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 32, 2024-09-20T18:51:16.752 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-20T18:51:16.752 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-20T18:51:16.752 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-20T18:51:16.752 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-20T18:51:16.752 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-20T18:51:16.752 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-20T18:51:16.752 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-20T18:51:16.752 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.752 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "64", 2024-09-20T18:51:16.753 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-20T18:51:16.753 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T18:51:16.753 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T18:51:16.753 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-20T18:51:16.753 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-20T18:51:16.753 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 0, 2024-09-20T18:51:16.753 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 0, 2024-09-20T18:51:16.753 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-20T18:51:16.753 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-20T18:51:16.753 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-20T18:51:16.753 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-20T18:51:16.753 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-20T18:51:16.753 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-20T18:51:16.753 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-20T18:51:16.753 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-20T18:51:16.754 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-20T18:51:16.754 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-20T18:51:16.754 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-20T18:51:16.754 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T18:51:16.754 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T18:51:16.754 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T18:51:16.754 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-20T18:51:16.754 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-20T18:51:16.754 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-20T18:51:16.754 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-20T18:51:16.754 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-20T18:51:16.754 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.754 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-20T18:51:16.755 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-20T18:51:16.755 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-20T18:51:16.755 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-20T18:51:16.755 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-20T18:51:16.755 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T18:51:16.755 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-20T18:51:16.755 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-20T18:51:16.755 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-20T18:51:16.755 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-20T18:51:16.755 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-20T18:51:16.755 INFO:teuthology.orchestra.run.smithi083.stdout: "options": {}, 2024-09-20T18:51:16.755 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-20T18:51:16.755 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw": {} 2024-09-20T18:51:16.755 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.755 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-20T18:51:16.756 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-20T18:51:16.756 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 1.25, 2024-09-20T18:51:16.756 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 1.25, 2024-09-20T18:51:16.756 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 1, 2024-09-20T18:51:16.756 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 1.25, 2024-09-20T18:51:16.756 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 1.25, 2024-09-20T18:51:16.756 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 1, 2024-09-20T18:51:16.756 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 1, 2024-09-20T18:51:16.756 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 1 2024-09-20T18:51:16.756 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:16.756 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.756 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.756 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 6, 2024-09-20T18:51:16.756 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": "default.rgw.meta", 2024-09-20T18:51:16.757 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-20T18:49:50.011276+0000", 2024-09-20T18:51:16.757 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-20T18:51:16.757 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool", 2024-09-20T18:51:16.757 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-20T18:51:16.757 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 3, 2024-09-20T18:51:16.757 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 2, 2024-09-20T18:51:16.757 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-20T18:51:16.757 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-20T18:51:16.757 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-20T18:51:16.757 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T18:51:16.757 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T18:51:16.757 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-20T18:51:16.757 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-20T18:51:16.757 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-20T18:51:16.757 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 32, 2024-09-20T18:51:16.758 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 32, 2024-09-20T18:51:16.758 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 32, 2024-09-20T18:51:16.758 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 32, 2024-09-20T18:51:16.758 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 32, 2024-09-20T18:51:16.758 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-20T18:51:16.758 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-20T18:51:16.758 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-20T18:51:16.758 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-20T18:51:16.758 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-20T18:51:16.758 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-20T18:51:16.758 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-20T18:51:16.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.758 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "66", 2024-09-20T18:51:16.759 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-20T18:51:16.759 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T18:51:16.759 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T18:51:16.759 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-20T18:51:16.759 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-20T18:51:16.759 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 0, 2024-09-20T18:51:16.759 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 0, 2024-09-20T18:51:16.759 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-20T18:51:16.759 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-20T18:51:16.759 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-20T18:51:16.759 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-20T18:51:16.759 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-20T18:51:16.759 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-20T18:51:16.759 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-20T18:51:16.760 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-20T18:51:16.760 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-20T18:51:16.760 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-20T18:51:16.760 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-20T18:51:16.760 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T18:51:16.760 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T18:51:16.760 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T18:51:16.760 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-20T18:51:16.760 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-20T18:51:16.760 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-20T18:51:16.760 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-20T18:51:16.760 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-20T18:51:16.760 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.760 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-20T18:51:16.761 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-20T18:51:16.761 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-20T18:51:16.761 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-20T18:51:16.761 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-20T18:51:16.761 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T18:51:16.761 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-20T18:51:16.761 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-20T18:51:16.761 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-20T18:51:16.761 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-20T18:51:16.761 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-20T18:51:16.761 INFO:teuthology.orchestra.run.smithi083.stdout: "options": { 2024-09-20T18:51:16.761 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_bias": 4 2024-09-20T18:51:16.761 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.761 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-20T18:51:16.762 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw": {} 2024-09-20T18:51:16.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.762 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-20T18:51:16.762 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-20T18:51:16.762 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 1.75, 2024-09-20T18:51:16.762 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 1.75, 2024-09-20T18:51:16.762 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 1, 2024-09-20T18:51:16.762 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 1.75, 2024-09-20T18:51:16.762 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 1.75, 2024-09-20T18:51:16.762 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 1, 2024-09-20T18:51:16.762 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 1, 2024-09-20T18:51:16.762 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 1 2024-09-20T18:51:16.762 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:16.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.763 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.763 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 7, 2024-09-20T18:51:16.763 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": "unique_pool_0", 2024-09-20T18:51:16.763 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-20T18:50:48.697483+0000", 2024-09-20T18:51:16.763 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-20T18:51:16.763 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool", 2024-09-20T18:51:16.763 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-20T18:51:16.763 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 3, 2024-09-20T18:51:16.763 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 2, 2024-09-20T18:51:16.763 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-20T18:51:16.763 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-20T18:51:16.763 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-20T18:51:16.763 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T18:51:16.763 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T18:51:16.764 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-20T18:51:16.764 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-20T18:51:16.764 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-20T18:51:16.764 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 16, 2024-09-20T18:51:16.764 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 16, 2024-09-20T18:51:16.764 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 16, 2024-09-20T18:51:16.764 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 16, 2024-09-20T18:51:16.764 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 16, 2024-09-20T18:51:16.764 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-20T18:51:16.764 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-20T18:51:16.764 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-20T18:51:16.765 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-20T18:51:16.766 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-20T18:51:16.766 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-20T18:51:16.766 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-20T18:51:16.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.766 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "87", 2024-09-20T18:51:16.766 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-20T18:51:16.766 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T18:51:16.766 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T18:51:16.766 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-20T18:51:16.766 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-20T18:51:16.766 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 0, 2024-09-20T18:51:16.766 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 0, 2024-09-20T18:51:16.766 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-20T18:51:16.767 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-20T18:51:16.767 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-20T18:51:16.767 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-20T18:51:16.767 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-20T18:51:16.767 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-20T18:51:16.767 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-20T18:51:16.767 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-20T18:51:16.767 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-20T18:51:16.767 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-20T18:51:16.767 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-20T18:51:16.767 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T18:51:16.767 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T18:51:16.767 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T18:51:16.767 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-20T18:51:16.768 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-20T18:51:16.768 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-20T18:51:16.768 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-20T18:51:16.768 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-20T18:51:16.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.768 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-20T18:51:16.768 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-20T18:51:16.768 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-20T18:51:16.768 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-20T18:51:16.768 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-20T18:51:16.768 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T18:51:16.768 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-20T18:51:16.768 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-20T18:51:16.768 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-20T18:51:16.768 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-20T18:51:16.769 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-20T18:51:16.769 INFO:teuthology.orchestra.run.smithi083.stdout: "options": {}, 2024-09-20T18:51:16.769 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-20T18:51:16.769 INFO:teuthology.orchestra.run.smithi083.stdout: "rados": {} 2024-09-20T18:51:16.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.769 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-20T18:51:16.769 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-20T18:51:16.769 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 2, 2024-09-20T18:51:16.769 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 2, 2024-09-20T18:51:16.769 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 1, 2024-09-20T18:51:16.769 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 2, 2024-09-20T18:51:16.769 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 2, 2024-09-20T18:51:16.769 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 1, 2024-09-20T18:51:16.769 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 1, 2024-09-20T18:51:16.770 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 1 2024-09-20T18:51:16.770 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:16.770 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:16.770 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T18:51:16.770 INFO:teuthology.orchestra.run.smithi083.stdout: "osds": [ 2024-09-20T18:51:16.770 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.770 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 0, 2024-09-20T18:51:16.770 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "c714e1ec-f0c3-4054-9792-61b84e53eaa8", 2024-09-20T18:51:16.770 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T18:51:16.770 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T18:51:16.770 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T18:51:16.770 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T18:51:16.770 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T18:51:16.770 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T18:51:16.771 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 8, 2024-09-20T18:51:16.771 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 83, 2024-09-20T18:51:16.771 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T18:51:16.771 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T18:51:16.771 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T18:51:16.771 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:16.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.771 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:16.771 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6801", 2024-09-20T18:51:16.771 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2574084049 2024-09-20T18:51:16.771 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:16.771 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:16.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.771 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T18:51:16.772 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:16.772 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.772 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:16.772 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6802", 2024-09-20T18:51:16.772 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2574084049 2024-09-20T18:51:16.772 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:16.772 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:16.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.772 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T18:51:16.772 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:16.772 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.772 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:16.772 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6804", 2024-09-20T18:51:16.772 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2574084049 2024-09-20T18:51:16.773 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:16.773 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:16.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.773 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T18:51:16.773 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:16.773 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.773 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:16.773 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6803", 2024-09-20T18:51:16.773 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2574084049 2024-09-20T18:51:16.773 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:16.773 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:16.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.773 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.83:6801/2574084049", 2024-09-20T18:51:16.773 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.83:6802/2574084049", 2024-09-20T18:51:16.773 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.83:6804/2574084049", 2024-09-20T18:51:16.774 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.83:6803/2574084049", 2024-09-20T18:51:16.774 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T18:51:16.774 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T18:51:16.774 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T18:51:16.774 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:16.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.774 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 1, 2024-09-20T18:51:16.774 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "9dc0f2a8-bafa-4a42-88f5-0301dd58f78d", 2024-09-20T18:51:16.774 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T18:51:16.774 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T18:51:16.774 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T18:51:16.774 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T18:51:16.774 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T18:51:16.775 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T18:51:16.775 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 13, 2024-09-20T18:51:16.775 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 83, 2024-09-20T18:51:16.775 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T18:51:16.775 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T18:51:16.775 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T18:51:16.775 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:16.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.775 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:16.775 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6805", 2024-09-20T18:51:16.775 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 4213285798 2024-09-20T18:51:16.775 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:16.775 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:16.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.776 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T18:51:16.776 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:16.776 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.776 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:16.776 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6806", 2024-09-20T18:51:16.776 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 4213285798 2024-09-20T18:51:16.776 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:16.776 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:16.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.776 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T18:51:16.776 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:16.776 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.776 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:16.776 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6808", 2024-09-20T18:51:16.777 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 4213285798 2024-09-20T18:51:16.777 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:16.777 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:16.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.777 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T18:51:16.777 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:16.777 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.777 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:16.777 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6807", 2024-09-20T18:51:16.777 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 4213285798 2024-09-20T18:51:16.777 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:16.777 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:16.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.777 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.83:6805/4213285798", 2024-09-20T18:51:16.778 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.83:6806/4213285798", 2024-09-20T18:51:16.778 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.83:6808/4213285798", 2024-09-20T18:51:16.778 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.83:6807/4213285798", 2024-09-20T18:51:16.778 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T18:51:16.778 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T18:51:16.778 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T18:51:16.778 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:16.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.778 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.778 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 2, 2024-09-20T18:51:16.778 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "448ab692-ec41-44dc-b2d5-dbefd255b3fd", 2024-09-20T18:51:16.778 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T18:51:16.778 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T18:51:16.778 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T18:51:16.779 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T18:51:16.779 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T18:51:16.779 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T18:51:16.779 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 18, 2024-09-20T18:51:16.779 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 83, 2024-09-20T18:51:16.779 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T18:51:16.779 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T18:51:16.779 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T18:51:16.779 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:16.779 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.779 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:16.779 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6809", 2024-09-20T18:51:16.779 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 805947571 2024-09-20T18:51:16.779 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:16.780 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:16.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.780 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T18:51:16.780 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:16.780 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.780 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:16.780 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6810", 2024-09-20T18:51:16.780 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 805947571 2024-09-20T18:51:16.780 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:16.780 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:16.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.780 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T18:51:16.780 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:16.780 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.780 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:16.781 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6812", 2024-09-20T18:51:16.781 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 805947571 2024-09-20T18:51:16.781 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:16.781 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:16.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.781 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T18:51:16.781 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:16.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.781 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:16.781 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6811", 2024-09-20T18:51:16.781 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 805947571 2024-09-20T18:51:16.781 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:16.781 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:16.782 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.782 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.83:6809/805947571", 2024-09-20T18:51:16.782 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.83:6810/805947571", 2024-09-20T18:51:16.782 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.83:6812/805947571", 2024-09-20T18:51:16.782 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.83:6811/805947571", 2024-09-20T18:51:16.782 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T18:51:16.782 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T18:51:16.782 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T18:51:16.782 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:16.782 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.782 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.782 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 3, 2024-09-20T18:51:16.782 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "7cda5e50-e22e-4d4a-823b-53c8f5061193", 2024-09-20T18:51:16.782 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T18:51:16.783 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T18:51:16.783 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T18:51:16.783 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T18:51:16.783 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T18:51:16.783 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T18:51:16.783 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 25, 2024-09-20T18:51:16.783 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 83, 2024-09-20T18:51:16.783 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T18:51:16.783 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T18:51:16.783 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T18:51:16.783 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:16.783 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.783 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:16.783 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6813", 2024-09-20T18:51:16.784 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 987237318 2024-09-20T18:51:16.784 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:16.784 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:16.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.784 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T18:51:16.784 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:16.784 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.784 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:16.784 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6814", 2024-09-20T18:51:16.784 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 987237318 2024-09-20T18:51:16.784 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:16.784 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:16.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.784 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T18:51:16.785 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:16.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.785 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:16.785 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6816", 2024-09-20T18:51:16.785 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 987237318 2024-09-20T18:51:16.785 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:16.785 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:16.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.785 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T18:51:16.785 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:16.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.785 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:16.785 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6815", 2024-09-20T18:51:16.785 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 987237318 2024-09-20T18:51:16.786 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:16.786 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:16.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.786 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.83:6813/987237318", 2024-09-20T18:51:16.786 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.83:6814/987237318", 2024-09-20T18:51:16.786 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.83:6816/987237318", 2024-09-20T18:51:16.786 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.83:6815/987237318", 2024-09-20T18:51:16.786 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T18:51:16.786 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T18:51:16.786 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T18:51:16.786 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:16.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.787 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.787 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 4, 2024-09-20T18:51:16.787 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "3d62654d-d31d-4310-9934-49e694dd17d9", 2024-09-20T18:51:16.787 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T18:51:16.787 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T18:51:16.787 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T18:51:16.787 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T18:51:16.787 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T18:51:16.787 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T18:51:16.787 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 31, 2024-09-20T18:51:16.787 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 83, 2024-09-20T18:51:16.787 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T18:51:16.787 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T18:51:16.787 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T18:51:16.788 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:16.788 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.788 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:16.788 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6800", 2024-09-20T18:51:16.788 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2096684065 2024-09-20T18:51:16.788 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:16.788 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:16.788 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.788 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T18:51:16.788 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:16.788 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.788 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:16.788 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6801", 2024-09-20T18:51:16.788 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2096684065 2024-09-20T18:51:16.788 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:16.789 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:16.789 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.789 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T18:51:16.789 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:16.789 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.789 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:16.789 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6803", 2024-09-20T18:51:16.789 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2096684065 2024-09-20T18:51:16.789 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:16.789 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:16.789 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.790 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T18:51:16.790 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:16.790 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.790 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:16.790 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6802", 2024-09-20T18:51:16.790 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2096684065 2024-09-20T18:51:16.790 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:16.790 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:16.790 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.790 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.142:6800/2096684065", 2024-09-20T18:51:16.790 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.142:6801/2096684065", 2024-09-20T18:51:16.790 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.142:6803/2096684065", 2024-09-20T18:51:16.790 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.142:6802/2096684065", 2024-09-20T18:51:16.791 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T18:51:16.791 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T18:51:16.791 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T18:51:16.791 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:16.791 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.791 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.791 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 5, 2024-09-20T18:51:16.791 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "b3e4e21b-639f-4d25-8bbf-ea8922a9222a", 2024-09-20T18:51:16.791 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T18:51:16.791 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T18:51:16.791 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T18:51:16.791 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T18:51:16.791 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T18:51:16.791 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T18:51:16.792 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 38, 2024-09-20T18:51:16.792 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 83, 2024-09-20T18:51:16.792 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T18:51:16.792 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T18:51:16.792 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T18:51:16.792 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:16.792 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.792 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:16.792 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6804", 2024-09-20T18:51:16.792 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2500466067 2024-09-20T18:51:16.792 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:16.792 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:16.792 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.792 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T18:51:16.793 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:16.793 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.793 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:16.793 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6805", 2024-09-20T18:51:16.793 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2500466067 2024-09-20T18:51:16.793 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:16.793 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:16.793 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.793 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T18:51:16.793 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:16.793 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.793 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:16.793 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6807", 2024-09-20T18:51:16.794 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2500466067 2024-09-20T18:51:16.794 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:16.794 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:16.794 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.794 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T18:51:16.794 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:16.794 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.794 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:16.794 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6806", 2024-09-20T18:51:16.794 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2500466067 2024-09-20T18:51:16.794 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:16.794 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:16.794 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.794 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.142:6804/2500466067", 2024-09-20T18:51:16.795 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.142:6805/2500466067", 2024-09-20T18:51:16.795 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.142:6807/2500466067", 2024-09-20T18:51:16.795 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.142:6806/2500466067", 2024-09-20T18:51:16.795 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T18:51:16.795 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T18:51:16.795 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T18:51:16.795 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:16.795 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.795 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.795 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 6, 2024-09-20T18:51:16.795 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "c5db7107-ccb4-4ae2-9e63-f444897942e3", 2024-09-20T18:51:16.795 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T18:51:16.795 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T18:51:16.795 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T18:51:16.796 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T18:51:16.796 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T18:51:16.796 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T18:51:16.796 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 45, 2024-09-20T18:51:16.796 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 91, 2024-09-20T18:51:16.796 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T18:51:16.796 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T18:51:16.796 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T18:51:16.796 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:16.796 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.796 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:16.796 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6808", 2024-09-20T18:51:16.796 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1688186226 2024-09-20T18:51:16.796 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:16.796 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:16.797 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.797 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T18:51:16.797 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:16.797 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.797 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:16.797 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6809", 2024-09-20T18:51:16.797 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1688186226 2024-09-20T18:51:16.797 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:16.797 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:16.797 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.797 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T18:51:16.797 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:16.797 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.798 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:16.798 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6811", 2024-09-20T18:51:16.798 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1688186226 2024-09-20T18:51:16.798 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:16.798 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:16.798 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.798 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T18:51:16.798 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:16.798 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.798 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:16.798 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6810", 2024-09-20T18:51:16.798 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1688186226 2024-09-20T18:51:16.798 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:16.799 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:16.799 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.799 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.142:6808/1688186226", 2024-09-20T18:51:16.799 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.142:6809/1688186226", 2024-09-20T18:51:16.799 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.142:6811/1688186226", 2024-09-20T18:51:16.799 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.142:6810/1688186226", 2024-09-20T18:51:16.799 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T18:51:16.799 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T18:51:16.799 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T18:51:16.799 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:16.799 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.799 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.799 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 7, 2024-09-20T18:51:16.799 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "07d16806-e2c8-4acb-b284-d75701e61622", 2024-09-20T18:51:16.800 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T18:51:16.800 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T18:51:16.800 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T18:51:16.800 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T18:51:16.800 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T18:51:16.800 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T18:51:16.800 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 53, 2024-09-20T18:51:16.800 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 83, 2024-09-20T18:51:16.800 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T18:51:16.800 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T18:51:16.800 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T18:51:16.800 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:16.800 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.800 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:16.800 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6812", 2024-09-20T18:51:16.801 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3679770984 2024-09-20T18:51:16.801 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:16.801 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:16.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.801 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T18:51:16.801 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:16.801 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.801 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:16.801 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6813", 2024-09-20T18:51:16.801 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3679770984 2024-09-20T18:51:16.801 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:16.801 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:16.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.801 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T18:51:16.802 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:16.802 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.802 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:16.802 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6815", 2024-09-20T18:51:16.802 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3679770984 2024-09-20T18:51:16.802 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:16.802 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:16.802 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.802 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T18:51:16.802 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:16.802 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.802 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:16.802 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6814", 2024-09-20T18:51:16.802 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3679770984 2024-09-20T18:51:16.803 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:16.803 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:16.803 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.803 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.142:6812/3679770984", 2024-09-20T18:51:16.803 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.142:6813/3679770984", 2024-09-20T18:51:16.803 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.142:6815/3679770984", 2024-09-20T18:51:16.803 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.142:6814/3679770984", 2024-09-20T18:51:16.803 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T18:51:16.803 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T18:51:16.803 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T18:51:16.803 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:16.803 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:16.803 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T18:51:16.803 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_xinfo": [ 2024-09-20T18:51:16.803 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.804 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 0, 2024-09-20T18:51:16.804 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T18:51:16.804 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T18:51:16.804 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T18:51:16.804 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T18:51:16.804 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T18:51:16.804 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T18:50:11.165148+0000", 2024-09-20T18:51:16.804 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T18:51:16.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.804 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.804 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 1, 2024-09-20T18:51:16.804 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T18:51:16.804 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T18:51:16.804 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T18:51:16.805 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T18:51:16.805 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T18:51:16.805 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T18:50:22.526229+0000", 2024-09-20T18:51:16.805 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T18:51:16.805 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.805 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.805 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 2, 2024-09-20T18:51:16.805 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T18:51:16.805 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T18:51:16.805 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T18:51:16.805 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T18:51:16.805 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T18:51:16.805 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T18:50:58.991919+0000", 2024-09-20T18:51:16.805 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T18:51:16.806 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.806 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.806 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 3, 2024-09-20T18:51:16.806 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T18:51:16.806 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T18:51:16.806 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T18:51:16.806 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T18:51:16.806 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T18:51:16.806 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T18:50:38.304985+0000", 2024-09-20T18:51:16.806 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T18:51:16.806 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.806 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.806 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 4, 2024-09-20T18:51:16.806 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T18:51:16.807 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T18:51:16.807 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T18:51:16.807 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T18:51:16.807 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T18:51:16.807 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T18:50:27.991715+0000", 2024-09-20T18:51:16.807 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T18:51:16.807 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.807 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.807 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 5, 2024-09-20T18:51:16.807 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T18:51:16.807 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T18:51:16.807 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T18:51:16.807 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T18:51:16.807 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T18:51:16.808 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T18:51:15.329061+0000", 2024-09-20T18:51:16.808 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T18:51:16.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.808 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.808 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 6, 2024-09-20T18:51:16.808 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T18:51:16.808 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T18:51:16.808 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T18:51:16.808 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T18:51:16.808 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T18:51:16.808 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T18:50:20.101102+0000", 2024-09-20T18:51:16.808 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T18:51:16.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.808 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.808 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 7, 2024-09-20T18:51:16.809 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T18:51:16.809 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T18:51:16.809 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T18:51:16.809 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T18:51:16.809 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T18:51:16.809 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T18:50:51.478598+0000", 2024-09-20T18:51:16.809 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T18:51:16.809 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:16.809 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T18:51:16.809 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_upmap": [], 2024-09-20T18:51:16.809 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_upmap_items": [ 2024-09-20T18:51:16.809 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.809 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "3.a", 2024-09-20T18:51:16.809 INFO:teuthology.orchestra.run.smithi083.stdout: "mappings": [ 2024-09-20T18:51:16.809 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.810 INFO:teuthology.orchestra.run.smithi083.stdout: "from": 4, 2024-09-20T18:51:16.810 INFO:teuthology.orchestra.run.smithi083.stdout: "to": 3 2024-09-20T18:51:16.810 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:16.810 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:16.810 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.810 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.810 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "4.f", 2024-09-20T18:51:16.810 INFO:teuthology.orchestra.run.smithi083.stdout: "mappings": [ 2024-09-20T18:51:16.810 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.810 INFO:teuthology.orchestra.run.smithi083.stdout: "from": 1, 2024-09-20T18:51:16.810 INFO:teuthology.orchestra.run.smithi083.stdout: "to": 2 2024-09-20T18:51:16.810 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:16.810 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:16.811 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.811 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.811 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "4.14", 2024-09-20T18:51:16.811 INFO:teuthology.orchestra.run.smithi083.stdout: "mappings": [ 2024-09-20T18:51:16.811 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.811 INFO:teuthology.orchestra.run.smithi083.stdout: "from": 1, 2024-09-20T18:51:16.811 INFO:teuthology.orchestra.run.smithi083.stdout: "to": 5 2024-09-20T18:51:16.811 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:16.811 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:16.811 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.811 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.811 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "4.1b", 2024-09-20T18:51:16.811 INFO:teuthology.orchestra.run.smithi083.stdout: "mappings": [ 2024-09-20T18:51:16.811 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:16.811 INFO:teuthology.orchestra.run.smithi083.stdout: "from": 1, 2024-09-20T18:51:16.812 INFO:teuthology.orchestra.run.smithi083.stdout: "to": 2 2024-09-20T18:51:16.812 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:16.812 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:16.812 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:16.812 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T18:51:16.812 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_upmap_primaries": [], 2024-09-20T18:51:16.812 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_temp": [], 2024-09-20T18:51:16.812 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_temp": [], 2024-09-20T18:51:16.812 INFO:teuthology.orchestra.run.smithi083.stdout: "blocklist": { 2024-09-20T18:51:16.812 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/3474253603": "2024-09-21T18:44:32.434180+0000", 2024-09-20T18:51:16.812 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:6800/3525856829": "2024-09-21T18:44:32.434180+0000", 2024-09-20T18:51:16.812 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/220746284": "2024-09-21T18:44:32.434180+0000", 2024-09-20T18:51:16.812 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/3875445393": "2024-09-21T18:44:56.985620+0000", 2024-09-20T18:51:16.812 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/3009092807": "2024-09-21T18:44:56.985620+0000", 2024-09-20T18:51:16.813 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/4267132131": "2024-09-21T18:44:56.985620+0000", 2024-09-20T18:51:16.813 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/3684382972": "2024-09-21T18:50:14.729526+0000", 2024-09-20T18:51:16.813 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/3132933178": "2024-09-21T18:50:14.729526+0000", 2024-09-20T18:51:16.813 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:6800/1007846146": "2024-09-21T18:50:14.729526+0000", 2024-09-20T18:51:16.813 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:6800/2985596191": "2024-09-21T18:44:56.985620+0000", 2024-09-20T18:51:16.813 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/2410768464": "2024-09-21T18:50:14.729526+0000", 2024-09-20T18:51:16.813 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/1868694097": "2024-09-21T18:50:14.729526+0000", 2024-09-20T18:51:16.813 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/3777331122": "2024-09-21T18:44:32.434180+0000", 2024-09-20T18:51:16.813 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/2161728215": "2024-09-21T18:50:14.729526+0000" 2024-09-20T18:51:16.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.813 INFO:teuthology.orchestra.run.smithi083.stdout: "range_blocklist": {}, 2024-09-20T18:51:16.813 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profiles": { 2024-09-20T18:51:16.813 INFO:teuthology.orchestra.run.smithi083.stdout: "default": { 2024-09-20T18:51:16.813 INFO:teuthology.orchestra.run.smithi083.stdout: "crush-failure-domain": "osd", 2024-09-20T18:51:16.814 INFO:teuthology.orchestra.run.smithi083.stdout: "k": "2", 2024-09-20T18:51:16.814 INFO:teuthology.orchestra.run.smithi083.stdout: "m": "1", 2024-09-20T18:51:16.814 INFO:teuthology.orchestra.run.smithi083.stdout: "plugin": "jerasure", 2024-09-20T18:51:16.814 INFO:teuthology.orchestra.run.smithi083.stdout: "technique": "reed_sol_van" 2024-09-20T18:51:16.814 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:16.814 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:16.814 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps_queue": [], 2024-09-20T18:51:16.814 INFO:teuthology.orchestra.run.smithi083.stdout: "new_removed_snaps": [], 2024-09-20T18:51:16.814 INFO:teuthology.orchestra.run.smithi083.stdout: "new_purged_snaps": [], 2024-09-20T18:51:16.814 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_node_flags": {}, 2024-09-20T18:51:16.814 INFO:teuthology.orchestra.run.smithi083.stdout: "device_class_flags": {}, 2024-09-20T18:51:16.814 INFO:teuthology.orchestra.run.smithi083.stdout: "stretch_mode": { 2024-09-20T18:51:16.814 INFO:teuthology.orchestra.run.smithi083.stdout: "stretch_mode_enabled": false, 2024-09-20T18:51:16.814 INFO:teuthology.orchestra.run.smithi083.stdout: "stretch_bucket_count": 0, 2024-09-20T18:51:16.815 INFO:teuthology.orchestra.run.smithi083.stdout: "degraded_stretch_mode": 0, 2024-09-20T18:51:16.815 INFO:teuthology.orchestra.run.smithi083.stdout: "recovering_stretch_mode": 0, 2024-09-20T18:51:16.815 INFO:teuthology.orchestra.run.smithi083.stdout: "stretch_mode_bucket": 0 2024-09-20T18:51:16.815 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:16.815 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T18:51:16.815 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:16.816 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:51:16 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:51:16.843 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:16.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:16.844 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:16.945 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:16.973 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:16.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:16.974 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:17.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:16 smithi142 ceph-mon[32508]: 4.14 scrub starts 2024-09-20T18:51:17.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:16 smithi142 ceph-mon[32508]: 4.14 scrub ok 2024-09-20T18:51:17.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:16 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:51:17.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:16 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:51:17.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:16 smithi142 ceph-mon[32508]: osdmap e95: 8 total, 8 up, 8 in 2024-09-20T18:51:17.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:16 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1141503739' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-20T18:51:17.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:16 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:51:17.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:16 smithi083 ceph-mon[29421]: 4.14 scrub starts 2024-09-20T18:51:17.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:16 smithi083 ceph-mon[29421]: 4.14 scrub ok 2024-09-20T18:51:17.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:16 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:51:17.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:16 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:51:17.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:16 smithi083 ceph-mon[29421]: osdmap e95: 8 total, 8 up, 8 in 2024-09-20T18:51:17.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:16 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1141503739' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-20T18:51:17.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:16 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:51:17.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:16 smithi083 ceph-mon[36260]: 4.14 scrub starts 2024-09-20T18:51:17.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:16 smithi083 ceph-mon[36260]: 4.14 scrub ok 2024-09-20T18:51:17.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:16 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:51:17.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:16 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:51:17.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:16 smithi083 ceph-mon[36260]: osdmap e95: 8 total, 8 up, 8 in 2024-09-20T18:51:17.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:16 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1141503739' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-20T18:51:17.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:16 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:51:17.076 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:17.105 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:17.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:17.106 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:17.207 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:17.235 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:17.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:17.236 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:17.337 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:17.358 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:17.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:17.359 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:17.460 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:17.486 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:17.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:17.487 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:17.518 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:2024-09-20T18:51:17.502021+0000 min lat: 0.00137732 max lat: 0.186958 avg lat: 0.00750973 2024-09-20T18:51:17.518 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-20T18:51:17.518 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 20 15 42609 42594 133.072 110 0.0118628 0.00750973 2024-09-20T18:51:17.588 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:17.618 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:17.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:17.619 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:17.719 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:17.749 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:17.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:17.750 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:17.851 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:17.888 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:17.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:17.889 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:17.990 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:51:18.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:17 smithi142 ceph-mon[32508]: 5.3 scrub starts 2024-09-20T18:51:18.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:17 smithi142 ceph-mon[32508]: 5.3 scrub ok 2024-09-20T18:51:18.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:17 smithi142 ceph-mon[32508]: 3.b scrub starts 2024-09-20T18:51:18.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:17 smithi142 ceph-mon[32508]: 3.b scrub ok 2024-09-20T18:51:18.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:17 smithi142 ceph-mon[32508]: 5.d scrub starts 2024-09-20T18:51:18.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:17 smithi142 ceph-mon[32508]: 5.d scrub ok 2024-09-20T18:51:18.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:17 smithi142 ceph-mon[32508]: pgmap v55: 148 pgs: 148 active+clean; 2.4 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 135 MiB/s wr, 2.16k op/s 2024-09-20T18:51:18.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:17 smithi142 ceph-mon[32508]: 5.1d scrub starts 2024-09-20T18:51:18.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:17 smithi142 ceph-mon[32508]: 5.1d scrub ok 2024-09-20T18:51:18.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:17 smithi142 ceph-mon[32508]: 5.1 scrub starts 2024-09-20T18:51:18.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:17 smithi142 ceph-mon[32508]: 5.1 scrub ok 2024-09-20T18:51:18.028 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:18.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:18.030 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:51:18.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:17 smithi083 ceph-mon[29421]: 5.3 scrub starts 2024-09-20T18:51:18.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:17 smithi083 ceph-mon[29421]: 5.3 scrub ok 2024-09-20T18:51:18.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:17 smithi083 ceph-mon[29421]: 3.b scrub starts 2024-09-20T18:51:18.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:17 smithi083 ceph-mon[29421]: 3.b scrub ok 2024-09-20T18:51:18.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:17 smithi083 ceph-mon[29421]: 5.d scrub starts 2024-09-20T18:51:18.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:17 smithi083 ceph-mon[29421]: 5.d scrub ok 2024-09-20T18:51:18.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:17 smithi083 ceph-mon[29421]: pgmap v55: 148 pgs: 148 active+clean; 2.4 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 135 MiB/s wr, 2.16k op/s 2024-09-20T18:51:18.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:17 smithi083 ceph-mon[29421]: 5.1d scrub starts 2024-09-20T18:51:18.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:17 smithi083 ceph-mon[29421]: 5.1d scrub ok 2024-09-20T18:51:18.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:17 smithi083 ceph-mon[29421]: 5.1 scrub starts 2024-09-20T18:51:18.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:17 smithi083 ceph-mon[29421]: 5.1 scrub ok 2024-09-20T18:51:18.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:17 smithi083 ceph-mon[36260]: 5.3 scrub starts 2024-09-20T18:51:18.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:17 smithi083 ceph-mon[36260]: 5.3 scrub ok 2024-09-20T18:51:18.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:17 smithi083 ceph-mon[36260]: 3.b scrub starts 2024-09-20T18:51:18.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:17 smithi083 ceph-mon[36260]: 3.b scrub ok 2024-09-20T18:51:18.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:17 smithi083 ceph-mon[36260]: 5.d scrub starts 2024-09-20T18:51:18.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:17 smithi083 ceph-mon[36260]: 5.d scrub ok 2024-09-20T18:51:18.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:17 smithi083 ceph-mon[36260]: pgmap v55: 148 pgs: 148 active+clean; 2.4 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 135 MiB/s wr, 2.16k op/s 2024-09-20T18:51:18.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:17 smithi083 ceph-mon[36260]: 5.1d scrub starts 2024-09-20T18:51:18.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:17 smithi083 ceph-mon[36260]: 5.1d scrub ok 2024-09-20T18:51:18.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:17 smithi083 ceph-mon[36260]: 5.1 scrub starts 2024-09-20T18:51:18.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:17 smithi083 ceph-mon[36260]: 5.1 scrub ok 2024-09-20T18:51:18.130 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:18.159 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:18.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:18.160 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:18.260 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:18.299 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:18.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:18.300 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:18.401 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:18.430 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:18.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:18.431 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:18.506 INFO:tasks.thrashosds.thrasher:j is {'epoch': 95, 'fsid': '1ed5a50a-7780-11ef-baf5-efdc52797490', 'created': '2024-09-20T18:44:10.399800+0000', 'modified': '2024-09-20T18:51:15.713595+0000', 'last_up_change': '2024-09-20T18:49:34.836377+0000', 'last_in_change': '2024-09-20T18:49:12.342332+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 18, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 7, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-20T18:47:15.285795+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'datapool', 'create_time': '2024-09-20T18:49:43.452984+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 61, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.6500000953674316, 'score_stable': 2.6500000953674316, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 2.3299999237060547, 'raw_score_stable': 2.3299999237060547, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': '.rgw.root', 'create_time': '2024-09-20T18:49:44.217011+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-20T18:49:46.007275+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-20T18:49:47.961258+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-20T18:49:50.011276+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'unique_pool_0', 'create_time': '2024-09-20T18:50:48.697483+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': 'c714e1ec-f0c3-4054-9792-61b84e53eaa8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 83, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6801', 'nonce': 2574084049}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6802', 'nonce': 2574084049}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6804', 'nonce': 2574084049}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6803', 'nonce': 2574084049}]}, 'public_addr': '172.21.15.83:6801/2574084049', 'cluster_addr': '172.21.15.83:6802/2574084049', 'heartbeat_back_addr': '172.21.15.83:6804/2574084049', 'heartbeat_front_addr': '172.21.15.83:6803/2574084049', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '9dc0f2a8-bafa-4a42-88f5-0301dd58f78d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 83, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6805', 'nonce': 4213285798}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6806', 'nonce': 4213285798}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6808', 'nonce': 4213285798}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6807', 'nonce': 4213285798}]}, 'public_addr': '172.21.15.83:6805/4213285798', 'cluster_addr': '172.21.15.83:6806/4213285798', 'heartbeat_back_addr': '172.21.15.83:6808/4213285798', 'heartbeat_front_addr': '172.21.15.83:6807/4213285798', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '448ab692-ec41-44dc-b2d5-dbefd255b3fd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 83, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6809', 'nonce': 805947571}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6810', 'nonce': 805947571}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6812', 'nonce': 805947571}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6811', 'nonce': 805947571}]}, 'public_addr': '172.21.15.83:6809/805947571', 'cluster_addr': '172.21.15.83:6810/805947571', 'heartbeat_back_addr': '172.21.15.83:6812/805947571', 'heartbeat_front_addr': '172.21.15.83:6811/805947571', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '7cda5e50-e22e-4d4a-823b-53c8f5061193', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 25, 'up_thru': 83, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6813', 'nonce': 987237318}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6814', 'nonce': 987237318}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6816', 'nonce': 987237318}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6815', 'nonce': 987237318}]}, 'public_addr': '172.21.15.83:6813/987237318', 'cluster_addr': '172.21.15.83:6814/987237318', 'heartbeat_back_addr': '172.21.15.83:6816/987237318', 'heartbeat_front_addr': '172.21.15.83:6815/987237318', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '3d62654d-d31d-4310-9934-49e694dd17d9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 31, 'up_thru': 83, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6800', 'nonce': 2096684065}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6801', 'nonce': 2096684065}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6803', 'nonce': 2096684065}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6802', 'nonce': 2096684065}]}, 'public_addr': '172.21.15.142:6800/2096684065', 'cluster_addr': '172.21.15.142:6801/2096684065', 'heartbeat_back_addr': '172.21.15.142:6803/2096684065', 'heartbeat_front_addr': '172.21.15.142:6802/2096684065', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'b3e4e21b-639f-4d25-8bbf-ea8922a9222a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 38, 'up_thru': 83, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6804', 'nonce': 2500466067}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6805', 'nonce': 2500466067}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6807', 'nonce': 2500466067}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6806', 'nonce': 2500466067}]}, 'public_addr': '172.21.15.142:6804/2500466067', 'cluster_addr': '172.21.15.142:6805/2500466067', 'heartbeat_back_addr': '172.21.15.142:6807/2500466067', 'heartbeat_front_addr': '172.21.15.142:6806/2500466067', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'c5db7107-ccb4-4ae2-9e63-f444897942e3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 45, 'up_thru': 91, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6808', 'nonce': 1688186226}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6809', 'nonce': 1688186226}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6811', 'nonce': 1688186226}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6810', 'nonce': 1688186226}]}, 'public_addr': '172.21.15.142:6808/1688186226', 'cluster_addr': '172.21.15.142:6809/1688186226', 'heartbeat_back_addr': '172.21.15.142:6811/1688186226', 'heartbeat_front_addr': '172.21.15.142:6810/1688186226', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '07d16806-e2c8-4acb-b284-d75701e61622', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 53, 'up_thru': 83, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6812', 'nonce': 3679770984}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6813', 'nonce': 3679770984}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6815', 'nonce': 3679770984}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6814', 'nonce': 3679770984}]}, 'public_addr': '172.21.15.142:6812/3679770984', 'cluster_addr': '172.21.15.142:6813/3679770984', 'heartbeat_back_addr': '172.21.15.142:6815/3679770984', 'heartbeat_front_addr': '172.21.15.142:6814/3679770984', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T18:50:11.165148+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T18:50:22.526229+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T18:50:58.991919+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T18:50:38.304985+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T18:50:27.991715+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T18:51:15.329061+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T18:50:20.101102+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T18:50:51.478598+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.a', 'mappings': [{'from': 4, 'to': 3}]}, {'pgid': '4.f', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.14', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.1b', 'mappings': [{'from': 1, 'to': 2}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {'172.21.15.83:0/3474253603': '2024-09-21T18:44:32.434180+0000', '172.21.15.83:6800/3525856829': '2024-09-21T18:44:32.434180+0000', '172.21.15.83:0/220746284': '2024-09-21T18:44:32.434180+0000', '172.21.15.83:0/3875445393': '2024-09-21T18:44:56.985620+0000', '172.21.15.83:0/3009092807': '2024-09-21T18:44:56.985620+0000', '172.21.15.83:0/4267132131': '2024-09-21T18:44:56.985620+0000', '172.21.15.83:0/3684382972': '2024-09-21T18:50:14.729526+0000', '172.21.15.83:0/3132933178': '2024-09-21T18:50:14.729526+0000', '172.21.15.83:6800/1007846146': '2024-09-21T18:50:14.729526+0000', '172.21.15.83:6800/2985596191': '2024-09-21T18:44:56.985620+0000', '172.21.15.83:0/2410768464': '2024-09-21T18:50:14.729526+0000', '172.21.15.83:0/1868694097': '2024-09-21T18:50:14.729526+0000', '172.21.15.83:0/3777331122': '2024-09-21T18:44:32.434180+0000', '172.21.15.83:0/2161728215': '2024-09-21T18:50:14.729526+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-20T18:51:18.506 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 4.1b 2024-09-20T18:51:18.506 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd rm-pg-upmap-items 4.1b 2024-09-20T18:51:18.518 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 21 16 44405 44389 132.077 112.188 0.0028293 0.00755631 2024-09-20T18:51:18.532 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:51:18.574 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:18.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:18.576 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:51:18.662 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_blocked_ops 2024-09-20T18:51:18.671 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:51:18.677 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:18.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:18.731 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:18.731 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:18.832 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:51:18.914 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:18.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:18.915 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:51:19.001 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:18 smithi083 ceph-mon[29421]: 5.11 scrub starts 2024-09-20T18:51:19.001 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:18 smithi083 ceph-mon[29421]: 5.11 scrub ok 2024-09-20T18:51:19.001 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:18 smithi083 ceph-mon[29421]: 3.9 scrub starts 2024-09-20T18:51:19.001 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:18 smithi083 ceph-mon[29421]: 3.9 scrub ok 2024-09-20T18:51:19.001 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:18 smithi083 ceph-mon[36260]: 5.11 scrub starts 2024-09-20T18:51:19.001 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:18 smithi083 ceph-mon[36260]: 5.11 scrub ok 2024-09-20T18:51:19.002 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:18 smithi083 ceph-mon[36260]: 3.9 scrub starts 2024-09-20T18:51:19.002 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:18 smithi083 ceph-mon[36260]: 3.9 scrub ok 2024-09-20T18:51:19.016 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:51:19.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:18 smithi142 ceph-mon[32508]: 5.11 scrub starts 2024-09-20T18:51:19.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:18 smithi142 ceph-mon[32508]: 5.11 scrub ok 2024-09-20T18:51:19.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:18 smithi142 ceph-mon[32508]: 3.9 scrub starts 2024-09-20T18:51:19.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:18 smithi142 ceph-mon[32508]: 3.9 scrub ok 2024-09-20T18:51:19.069 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:19.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:19.070 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:51:19.170 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:51:19.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:19.220 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:19.220 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:51:19.290 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T18:51:19.321 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:19.345 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:19.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:19.346 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:19.422 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:51:19.447 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:19.450 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:51:19.470 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:19.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:19.471 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:19.524 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 22 16 45742 45726 129.868 83.5625 0.00754774 0.00769479 2024-09-20T18:51:19.572 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:51:19.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:19.631 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:19.632 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:51:19.732 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:19.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:19.760 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:19.760 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:19.827 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-20T18:51:19.862 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:19.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:19.908 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:19.908 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:20.009 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:51:20.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:19 smithi142 ceph-mon[32508]: pgmap v56: 148 pgs: 148 active+clean; 2.4 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 132 MiB/s wr, 2.10k op/s 2024-09-20T18:51:20.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:20.052 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:20.053 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:51:20.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:19 smithi083 ceph-mon[36260]: pgmap v56: 148 pgs: 148 active+clean; 2.4 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 132 MiB/s wr, 2.10k op/s 2024-09-20T18:51:20.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:19 smithi083 ceph-mon[29421]: pgmap v56: 148 pgs: 148 active+clean; 2.4 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 132 MiB/s wr, 2.10k op/s 2024-09-20T18:51:20.154 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:20.182 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:20.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:20.183 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:20.284 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:20.310 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:20.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:20.311 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:20.411 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:20.439 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:20.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:20.441 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:20.519 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 23 16 47401 47385 128.73 103.688 0.0083624 0.00776323 2024-09-20T18:51:20.541 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:51:20.573 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:20.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:20.574 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:51:20.675 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:20.703 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:20.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:20.704 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:20.741 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:51:20 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:51:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:51:20.804 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:20.831 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:20.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:20.832 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:20.932 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T18:51:20.932 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:51:20.933 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:20.991 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:20.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:20.992 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:21.011 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:20 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1110483866' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:51:21.012 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:20 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:51:21.012 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:20 smithi083 ceph-mon[29421]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T18:51:21.012 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:20 smithi083 ceph-mon[29421]: Cluster is now healthy 2024-09-20T18:51:21.012 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:20 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T18:51:21.012 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:20 smithi083 ceph-mon[29421]: osdmap e96: 8 total, 8 up, 8 in 2024-09-20T18:51:21.012 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:20 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1110483866' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:51:21.012 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:20 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:51:21.012 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:20 smithi083 ceph-mon[36260]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T18:51:21.012 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:20 smithi083 ceph-mon[36260]: Cluster is now healthy 2024-09-20T18:51:21.012 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:20 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T18:51:21.012 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:20 smithi083 ceph-mon[36260]: osdmap e96: 8 total, 8 up, 8 in 2024-09-20T18:51:21.092 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:21.117 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:21.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:21.118 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:21.218 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:21.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:21.258 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:21.258 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:21.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:20 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1110483866' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:51:21.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:20 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:51:21.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:20 smithi142 ceph-mon[32508]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T18:51:21.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:20 smithi142 ceph-mon[32508]: Cluster is now healthy 2024-09-20T18:51:21.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:20 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T18:51:21.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:20 smithi142 ceph-mon[32508]: osdmap e96: 8 total, 8 up, 8 in 2024-09-20T18:51:21.358 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:21.385 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:21.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:21.387 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:21.487 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:51:21.519 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 24 16 49009 48993 127.553 100.5 0.00745638 0.00783457 2024-09-20T18:51:21.527 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:21.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:21.528 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:51:21.629 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:51:21.667 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:21.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:21.668 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:51:21.769 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:51:21.810 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:21.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:21.811 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:51:21.912 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:21.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:21.940 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:21.940 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:22.041 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:22.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:21 smithi083 ceph-mon[29421]: 3.12 scrub starts 2024-09-20T18:51:22.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:21 smithi083 ceph-mon[29421]: 3.12 scrub ok 2024-09-20T18:51:22.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:21 smithi083 ceph-mon[29421]: 5.13 scrub starts 2024-09-20T18:51:22.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:21 smithi083 ceph-mon[29421]: 5.13 scrub ok 2024-09-20T18:51:22.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:21 smithi083 ceph-mon[29421]: pgmap v58: 148 pgs: 148 active+clean; 2.8 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 115 MiB/s wr, 1.84k op/s 2024-09-20T18:51:22.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:21 smithi083 ceph-mon[36260]: 3.12 scrub starts 2024-09-20T18:51:22.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:21 smithi083 ceph-mon[36260]: 3.12 scrub ok 2024-09-20T18:51:22.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:21 smithi083 ceph-mon[36260]: 5.13 scrub starts 2024-09-20T18:51:22.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:21 smithi083 ceph-mon[36260]: 5.13 scrub ok 2024-09-20T18:51:22.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:21 smithi083 ceph-mon[36260]: pgmap v58: 148 pgs: 148 active+clean; 2.8 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 115 MiB/s wr, 1.84k op/s 2024-09-20T18:51:22.065 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:22.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:22.066 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:22.082 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_historic_ops 2024-09-20T18:51:22.167 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:22.219 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:22.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:22.220 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:22.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:21 smithi142 ceph-mon[32508]: 3.12 scrub starts 2024-09-20T18:51:22.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:21 smithi142 ceph-mon[32508]: 3.12 scrub ok 2024-09-20T18:51:22.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:21 smithi142 ceph-mon[32508]: 5.13 scrub starts 2024-09-20T18:51:22.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:21 smithi142 ceph-mon[32508]: 5.13 scrub ok 2024-09-20T18:51:22.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:21 smithi142 ceph-mon[32508]: pgmap v58: 148 pgs: 148 active+clean; 2.8 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 115 MiB/s wr, 1.84k op/s 2024-09-20T18:51:22.321 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:22.345 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:22.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:22.346 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:22.447 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:22.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:22.501 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:22.501 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:22.519 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 25 16 50527 50511 126.245 94.875 0.00824791 0.00791528 2024-09-20T18:51:22.602 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:51:22.655 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:51:22.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:22.672 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:22.672 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:51:22.773 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:22.817 INFO:teuthology.orchestra.run.smithi083.stderr:clear 4.1b pg_upmap_items mapping 2024-09-20T18:51:22.828 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:22.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:22.829 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:22.836 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:22 smithi083 ceph-mon[29421]: 4.f scrub starts 2024-09-20T18:51:22.836 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:22 smithi083 ceph-mon[29421]: 4.f scrub ok 2024-09-20T18:51:22.836 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:22 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2779967636' entity='client.admin' cmd={"prefix": "osd rm-pg-upmap-items", "pgid": "4.1b"} : dispatch 2024-09-20T18:51:22.836 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:22 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd rm-pg-upmap-items", "pgid": "4.1b"} : dispatch 2024-09-20T18:51:22.836 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:22 smithi083 ceph-mon[36260]: 4.f scrub starts 2024-09-20T18:51:22.836 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:22 smithi083 ceph-mon[36260]: 4.f scrub ok 2024-09-20T18:51:22.836 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:22 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2779967636' entity='client.admin' cmd={"prefix": "osd rm-pg-upmap-items", "pgid": "4.1b"} : dispatch 2024-09-20T18:51:22.836 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:22 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd rm-pg-upmap-items", "pgid": "4.1b"} : dispatch 2024-09-20T18:51:22.931 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:51:22.972 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:22.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:22.974 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:51:23.074 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:23.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:23.116 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:23.116 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:23.217 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:23.251 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:23.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:23.252 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:23.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:22 smithi142 ceph-mon[32508]: 4.f scrub starts 2024-09-20T18:51:23.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:22 smithi142 ceph-mon[32508]: 4.f scrub ok 2024-09-20T18:51:23.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:22 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2779967636' entity='client.admin' cmd={"prefix": "osd rm-pg-upmap-items", "pgid": "4.1b"} : dispatch 2024-09-20T18:51:23.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:22 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd rm-pg-upmap-items", "pgid": "4.1b"} : dispatch 2024-09-20T18:51:23.353 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:23.382 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:23.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:23.383 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:23.484 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:23.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:23.508 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:23.509 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:23.519 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 26 16 52017 52001 124.971 93.125 0.0736088 0.00799652 2024-09-20T18:51:23.610 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:51:23.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:23.648 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:23.648 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:51:23.749 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:23.788 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:23.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:23.789 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:23.890 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:23.921 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:23.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:23.922 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:24.023 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:24.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:24.056 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:24.056 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:24.156 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:24.188 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:24.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:24.189 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:24.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:23 smithi142 ceph-mon[32508]: pgmap v59: 148 pgs: 148 active+clean; 2.9 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 131 MiB/s wr, 2.09k op/s 2024-09-20T18:51:24.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:23 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd rm-pg-upmap-items", "pgid": "4.1b"}]': finished 2024-09-20T18:51:24.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:23 smithi142 ceph-mon[32508]: osdmap e97: 8 total, 8 up, 8 in 2024-09-20T18:51:24.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:23 smithi142 ceph-mon[32508]: 5.4 scrub starts 2024-09-20T18:51:24.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:23 smithi142 ceph-mon[32508]: 5.4 scrub ok 2024-09-20T18:51:24.290 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:51:24.307 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:23 smithi083 ceph-mon[29421]: pgmap v59: 148 pgs: 148 active+clean; 2.9 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 131 MiB/s wr, 2.09k op/s 2024-09-20T18:51:24.307 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:23 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd rm-pg-upmap-items", "pgid": "4.1b"}]': finished 2024-09-20T18:51:24.307 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:23 smithi083 ceph-mon[29421]: osdmap e97: 8 total, 8 up, 8 in 2024-09-20T18:51:24.307 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:23 smithi083 ceph-mon[29421]: 5.4 scrub starts 2024-09-20T18:51:24.307 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:23 smithi083 ceph-mon[29421]: 5.4 scrub ok 2024-09-20T18:51:24.307 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:23 smithi083 ceph-mon[36260]: pgmap v59: 148 pgs: 148 active+clean; 2.9 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 131 MiB/s wr, 2.09k op/s 2024-09-20T18:51:24.307 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:23 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd rm-pg-upmap-items", "pgid": "4.1b"}]': finished 2024-09-20T18:51:24.308 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:23 smithi083 ceph-mon[36260]: osdmap e97: 8 total, 8 up, 8 in 2024-09-20T18:51:24.308 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:23 smithi083 ceph-mon[36260]: 5.4 scrub starts 2024-09-20T18:51:24.308 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:23 smithi083 ceph-mon[36260]: 5.4 scrub ok 2024-09-20T18:51:24.330 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:24.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:24.331 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:51:24.374 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:51:24.432 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:24.475 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:24.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:24.476 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:24.519 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 27 16 53654 53638 124.131 102.312 0.00679396 0.00805028 2024-09-20T18:51:24.577 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:24.640 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T18:51:24.644 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:24.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:24.645 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:24.679 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:51:24.680 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:51:24.717 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:51:24.717 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:51:24.746 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:51:24.755 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:51:24.755 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:51:24.793 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:51:24.794 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:51:24.804 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:24.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:24.805 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:51:24.839 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:51:24.839 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:51:24.877 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:51:24.878 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:51:24.906 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:51:24.916 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:51:24.917 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:51:24.956 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:51:24.957 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:51:24.957 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:24 smithi083 ceph-mon[36260]: 4.16 scrub starts 2024-09-20T18:51:24.957 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:24 smithi083 ceph-mon[36260]: 4.16 scrub ok 2024-09-20T18:51:24.957 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:24 smithi083 ceph-mon[36260]: 3.2 scrub starts 2024-09-20T18:51:24.957 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:24 smithi083 ceph-mon[36260]: 3.2 scrub ok 2024-09-20T18:51:24.958 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:24 smithi083 ceph-mon[36260]: 5.b scrub starts 2024-09-20T18:51:24.958 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:24 smithi083 ceph-mon[36260]: 5.b scrub ok 2024-09-20T18:51:24.958 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:24 smithi083 ceph-mon[36260]: 5.1b scrub starts 2024-09-20T18:51:24.958 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:24 smithi083 ceph-mon[36260]: 5.1b scrub ok 2024-09-20T18:51:24.958 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:24 smithi083 ceph-mon[36260]: osdmap e98: 8 total, 8 up, 8 in 2024-09-20T18:51:24.958 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:24 smithi083 ceph-mon[36260]: 6.8 scrub starts 2024-09-20T18:51:24.958 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:24 smithi083 ceph-mon[36260]: 6.8 scrub ok 2024-09-20T18:51:24.958 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:24 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/393942942' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:51:24.958 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:24 smithi083 ceph-mon[29421]: 4.16 scrub starts 2024-09-20T18:51:24.958 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:24 smithi083 ceph-mon[29421]: 4.16 scrub ok 2024-09-20T18:51:24.958 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:24 smithi083 ceph-mon[29421]: 3.2 scrub starts 2024-09-20T18:51:24.958 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:24 smithi083 ceph-mon[29421]: 3.2 scrub ok 2024-09-20T18:51:24.959 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:24 smithi083 ceph-mon[29421]: 5.b scrub starts 2024-09-20T18:51:24.959 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:24 smithi083 ceph-mon[29421]: 5.b scrub ok 2024-09-20T18:51:24.959 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:24 smithi083 ceph-mon[29421]: 5.1b scrub starts 2024-09-20T18:51:24.959 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:24 smithi083 ceph-mon[29421]: 5.1b scrub ok 2024-09-20T18:51:24.959 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:24 smithi083 ceph-mon[29421]: osdmap e98: 8 total, 8 up, 8 in 2024-09-20T18:51:24.959 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:24 smithi083 ceph-mon[29421]: 6.8 scrub starts 2024-09-20T18:51:24.959 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:24 smithi083 ceph-mon[29421]: 6.8 scrub ok 2024-09-20T18:51:24.959 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:24 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/393942942' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:51:24.974 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:24.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:24.975 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:51:25.076 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:25.108 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:25.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:25.110 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:25.210 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:25.239 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:25.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:25.240 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:25.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:24 smithi142 ceph-mon[32508]: 4.16 scrub starts 2024-09-20T18:51:25.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:24 smithi142 ceph-mon[32508]: 4.16 scrub ok 2024-09-20T18:51:25.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:24 smithi142 ceph-mon[32508]: 3.2 scrub starts 2024-09-20T18:51:25.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:24 smithi142 ceph-mon[32508]: 3.2 scrub ok 2024-09-20T18:51:25.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:24 smithi142 ceph-mon[32508]: 5.b scrub starts 2024-09-20T18:51:25.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:24 smithi142 ceph-mon[32508]: 5.b scrub ok 2024-09-20T18:51:25.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:24 smithi142 ceph-mon[32508]: 5.1b scrub starts 2024-09-20T18:51:25.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:24 smithi142 ceph-mon[32508]: 5.1b scrub ok 2024-09-20T18:51:25.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:24 smithi142 ceph-mon[32508]: osdmap e98: 8 total, 8 up, 8 in 2024-09-20T18:51:25.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:24 smithi142 ceph-mon[32508]: 6.8 scrub starts 2024-09-20T18:51:25.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:24 smithi142 ceph-mon[32508]: 6.8 scrub ok 2024-09-20T18:51:25.276 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:24 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/393942942' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:51:25.341 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:51:25.382 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:25.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:25.383 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:51:25.484 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:51:25.520 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 28 16 55108 55092 122.94 90.875 0.0136217 0.00812769 2024-09-20T18:51:25.521 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:25.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:25.522 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:51:25.623 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:25.676 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:25.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:25.677 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:25.778 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:25.821 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:25.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:25.822 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:25.923 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:25.949 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:25 smithi083 ceph-mon[29421]: 5.f scrub starts 2024-09-20T18:51:25.950 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:25 smithi083 ceph-mon[29421]: 5.f scrub ok 2024-09-20T18:51:25.950 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:25 smithi083 ceph-mon[29421]: pgmap v62: 148 pgs: 148 active+clean; 2.9 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 853 B/s rd, 84 MiB/s wr, 1.35k op/s 2024-09-20T18:51:25.950 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:25 smithi083 ceph-mon[29421]: 6.3 scrub starts 2024-09-20T18:51:25.950 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:25 smithi083 ceph-mon[29421]: 6.3 scrub ok 2024-09-20T18:51:25.950 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:25 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:51:25.950 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:25 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:51:25.950 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:25 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:25.951 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:25 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:25.951 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:25 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:51:25.951 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:25 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:51:25.951 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:25 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:25.951 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:25 smithi083 ceph-mon[36260]: 5.f scrub starts 2024-09-20T18:51:25.951 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:25 smithi083 ceph-mon[36260]: 5.f scrub ok 2024-09-20T18:51:25.951 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:25 smithi083 ceph-mon[36260]: pgmap v62: 148 pgs: 148 active+clean; 2.9 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 853 B/s rd, 84 MiB/s wr, 1.35k op/s 2024-09-20T18:51:25.951 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:25 smithi083 ceph-mon[36260]: 6.3 scrub starts 2024-09-20T18:51:25.951 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:25 smithi083 ceph-mon[36260]: 6.3 scrub ok 2024-09-20T18:51:25.952 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:25 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:51:25.952 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:25 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:51:25.952 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:25 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:25.952 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:25 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:25.952 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:25 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:51:25.952 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:25 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:51:25.952 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:25 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:25.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:25.953 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:25.953 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:26.054 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:26.079 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:26.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:26.081 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:26.122 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T18:51:26.122 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-20T18:51:26.122 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-20T18:51:26.122 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-20T18:51:26.122 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.122 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14158.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-20T18:51:26.122 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:47:16.702121+0000", 2024-09-20T18:51:26.123 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 249.39784505599999, 2024-09-20T18:51:26.123 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.46027232899999998, 2024-09-20T18:51:26.123 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:26.123 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:26.123 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:51:26.123 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:51:26.123 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14158", 2024-09-20T18:51:26.123 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2161728215", 2024-09-20T18:51:26.123 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 1 2024-09-20T18:51:26.123 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.123 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:26.124 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.124 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:26.124 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:47:16.702121+0000", 2024-09-20T18:51:26.124 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:26.124 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.124 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.124 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:26.124 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:47:16.702121+0000", 2024-09-20T18:51:26.124 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:26.124 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.124 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.124 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:26.124 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:47:16.702122+0000", 2024-09-20T18:51:26.125 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9900000000000009e-07 2024-09-20T18:51:26.125 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.125 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.125 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:26.125 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:47:16.702135+0000", 2024-09-20T18:51:26.125 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2087e-05 2024-09-20T18:51:26.125 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.125 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.125 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:26.125 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:47:16.702143+0000", 2024-09-20T18:51:26.125 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5420000000000005e-06 2024-09-20T18:51:26.125 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.125 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.126 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:26.126 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:47:16.702150+0000", 2024-09-20T18:51:26.126 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1910000000000003e-06 2024-09-20T18:51:26.126 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.126 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.126 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:26.126 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:47:16.702193+0000", 2024-09-20T18:51:26.126 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2398999999999998e-05 2024-09-20T18:51:26.126 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.126 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.126 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:51:26.126 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:47:16.702198+0000", 2024-09-20T18:51:26.127 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4009999999999996e-06 2024-09-20T18:51:26.127 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.127 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.127 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:26.127 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:47:17.161275+0000", 2024-09-20T18:51:26.127 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.45907645200000002 2024-09-20T18:51:26.127 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.127 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.127 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:26.127 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:47:17.161417+0000", 2024-09-20T18:51:26.127 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014258900000000001 2024-09-20T18:51:26.127 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.127 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.128 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,2", 2024-09-20T18:51:26.128 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:47:17.161566+0000", 2024-09-20T18:51:26.128 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014886599999999999 2024-09-20T18:51:26.128 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.128 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.128 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:51:26.128 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:47:17.162060+0000", 2024-09-20T18:51:26.128 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00049417099999999995 2024-09-20T18:51:26.128 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.128 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.128 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:26.128 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:47:17.162256+0000", 2024-09-20T18:51:26.128 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019619200000000001 2024-09-20T18:51:26.129 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.129 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.129 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:26.129 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:47:17.162303+0000", 2024-09-20T18:51:26.129 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6159999999999999e-05 2024-09-20T18:51:26.129 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.129 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.129 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:51:26.129 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:47:17.162357+0000", 2024-09-20T18:51:26.129 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4236e-05 2024-09-20T18:51:26.129 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.129 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.130 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:26.130 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:47:17.162394+0000", 2024-09-20T18:51:26.130 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7044e-05 2024-09-20T18:51:26.130 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:26.130 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:26.130 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:26.130 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.130 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.130 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:21865 7.b 7:dc2922ad:::benchmark_data_smithi083_70643_object21864:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-20T18:51:26.130 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:05.417932+0000", 2024-09-20T18:51:26.131 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 20.68203471, 2024-09-20T18:51:26.131 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.018201715, 2024-09-20T18:51:26.131 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:26.131 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:26.131 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:51:26.131 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:51:26.131 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:51:26.131 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:51:26.131 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 21865 2024-09-20T18:51:26.131 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.131 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:26.131 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.131 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:26.131 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.417932+0000", 2024-09-20T18:51:26.132 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:26.132 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.132 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.132 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:26.132 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.417932+0000", 2024-09-20T18:51:26.132 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:26.132 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.132 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.132 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:26.132 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.417934+0000", 2024-09-20T18:51:26.132 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8169999999999999e-06 2024-09-20T18:51:26.132 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.132 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.133 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:26.133 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.417957+0000", 2024-09-20T18:51:26.133 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2861e-05 2024-09-20T18:51:26.133 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.133 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.133 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:26.133 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.417971+0000", 2024-09-20T18:51:26.133 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4175000000000001e-05 2024-09-20T18:51:26.133 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.133 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.133 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:26.133 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.417979+0000", 2024-09-20T18:51:26.133 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.1440000000000003e-06 2024-09-20T18:51:26.133 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.133 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.134 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:26.134 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.418039+0000", 2024-09-20T18:51:26.134 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0028000000000002e-05 2024-09-20T18:51:26.134 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.134 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.134 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:26.134 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.418132+0000", 2024-09-20T18:51:26.134 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3580999999999997e-05 2024-09-20T18:51:26.134 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.134 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.134 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,6", 2024-09-20T18:51:26.134 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.418239+0000", 2024-09-20T18:51:26.134 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106543 2024-09-20T18:51:26.134 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.134 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.135 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:51:26.135 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.419195+0000", 2024-09-20T18:51:26.135 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000956101 2024-09-20T18:51:26.135 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.135 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.135 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:26.135 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.419323+0000", 2024-09-20T18:51:26.135 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012773999999999999 2024-09-20T18:51:26.135 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.135 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.135 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:26.135 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.436028+0000", 2024-09-20T18:51:26.135 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.016704871999999999 2024-09-20T18:51:26.135 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.135 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.136 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:51:26.136 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.436084+0000", 2024-09-20T18:51:26.136 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6654999999999999e-05 2024-09-20T18:51:26.136 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.136 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.136 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:26.136 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.436134+0000", 2024-09-20T18:51:26.136 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9197999999999999e-05 2024-09-20T18:51:26.137 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:26.137 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:26.137 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:26.137 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.137 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.137 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:22321 7.b 7:d8a46785:::benchmark_data_smithi083_70643_object22320:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-20T18:51:26.137 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:05.657680+0000", 2024-09-20T18:51:26.137 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 20.442286633999998, 2024-09-20T18:51:26.137 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.031306155000000002, 2024-09-20T18:51:26.137 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:26.137 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:26.137 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:51:26.137 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:51:26.137 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:51:26.138 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:51:26.138 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 22321 2024-09-20T18:51:26.138 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.138 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:26.138 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.138 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:26.138 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.657680+0000", 2024-09-20T18:51:26.138 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:26.138 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.138 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.138 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:26.138 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.657680+0000", 2024-09-20T18:51:26.138 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:26.139 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.139 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.139 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:26.139 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.657681+0000", 2024-09-20T18:51:26.139 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5290000000000001e-06 2024-09-20T18:51:26.139 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.139 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.139 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:26.139 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.657705+0000", 2024-09-20T18:51:26.139 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3142e-05 2024-09-20T18:51:26.139 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.139 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.139 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:26.140 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.657720+0000", 2024-09-20T18:51:26.140 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5282999999999999e-05 2024-09-20T18:51:26.140 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.140 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.140 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:26.140 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.657728+0000", 2024-09-20T18:51:26.140 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8760000000000005e-06 2024-09-20T18:51:26.140 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.140 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.140 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:26.140 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.657781+0000", 2024-09-20T18:51:26.140 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3236000000000003e-05 2024-09-20T18:51:26.140 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.141 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.141 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:26.141 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.657922+0000", 2024-09-20T18:51:26.141 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014139299999999999 2024-09-20T18:51:26.141 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.141 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.141 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,6", 2024-09-20T18:51:26.141 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.658037+0000", 2024-09-20T18:51:26.141 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000114947 2024-09-20T18:51:26.141 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.141 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.141 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:51:26.141 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.659192+0000", 2024-09-20T18:51:26.142 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011548330000000001 2024-09-20T18:51:26.142 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.142 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.142 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:26.142 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.661272+0000", 2024-09-20T18:51:26.142 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0020796030000000002 2024-09-20T18:51:26.142 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.142 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.142 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:26.142 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.688823+0000", 2024-09-20T18:51:26.142 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.027551737 2024-09-20T18:51:26.142 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.142 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.142 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:51:26.143 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.688933+0000", 2024-09-20T18:51:26.143 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010918599999999999 2024-09-20T18:51:26.143 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.143 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.143 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:26.143 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.688986+0000", 2024-09-20T18:51:26.143 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3390000000000002e-05 2024-09-20T18:51:26.143 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:26.143 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:26.143 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:26.143 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.143 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.143 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:22336 7.b 7:d8a6fafa:::benchmark_data_smithi083_70643_object22335:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-20T18:51:26.143 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:05.669033+0000", 2024-09-20T18:51:26.144 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 20.430933964000001, 2024-09-20T18:51:26.144 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.023357587999999999, 2024-09-20T18:51:26.144 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:26.144 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:26.144 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:51:26.144 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:51:26.144 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:51:26.144 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:51:26.144 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 22336 2024-09-20T18:51:26.144 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.144 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:26.144 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.144 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:26.144 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.669033+0000", 2024-09-20T18:51:26.144 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:26.145 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.145 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.145 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:26.145 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.669033+0000", 2024-09-20T18:51:26.145 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:26.145 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.145 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.145 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:26.145 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.669034+0000", 2024-09-20T18:51:26.145 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.77e-06 2024-09-20T18:51:26.145 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.145 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.145 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:26.145 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.669058+0000", 2024-09-20T18:51:26.146 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4077e-05 2024-09-20T18:51:26.146 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.146 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.146 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:26.146 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.669076+0000", 2024-09-20T18:51:26.146 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7901999999999999e-05 2024-09-20T18:51:26.146 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.146 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.146 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:26.146 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.669085+0000", 2024-09-20T18:51:26.146 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8249999999999994e-06 2024-09-20T18:51:26.146 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.146 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.146 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:26.147 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.669148+0000", 2024-09-20T18:51:26.147 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3072999999999997e-05 2024-09-20T18:51:26.147 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.147 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.147 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:26.147 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.669301+0000", 2024-09-20T18:51:26.147 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015296799999999999 2024-09-20T18:51:26.147 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.147 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.147 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,6", 2024-09-20T18:51:26.147 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.669445+0000", 2024-09-20T18:51:26.147 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014425700000000001 2024-09-20T18:51:26.147 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.148 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.148 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:26.148 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.671564+0000", 2024-09-20T18:51:26.148 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0021183420000000001 2024-09-20T18:51:26.148 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.148 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.148 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:51:26.148 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.671575+0000", 2024-09-20T18:51:26.148 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1294000000000001e-05 2024-09-20T18:51:26.148 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.148 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.148 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:26.148 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.692249+0000", 2024-09-20T18:51:26.148 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.020673984999999999 2024-09-20T18:51:26.149 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.149 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.149 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:51:26.149 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.692340+0000", 2024-09-20T18:51:26.149 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0676999999999999e-05 2024-09-20T18:51:26.149 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.149 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.149 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:26.149 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.692390+0000", 2024-09-20T18:51:26.149 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0417999999999997e-05 2024-09-20T18:51:26.149 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:26.149 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:26.149 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:26.149 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.150 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.150 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:22344 7.b 7:da7dd668:::benchmark_data_smithi083_70643_object22343:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-20T18:51:26.150 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:05.672139+0000", 2024-09-20T18:51:26.150 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 20.427827454999999, 2024-09-20T18:51:26.150 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.020595638999999999, 2024-09-20T18:51:26.150 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:26.150 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:26.150 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:51:26.150 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:51:26.150 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:51:26.150 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:51:26.150 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 22344 2024-09-20T18:51:26.150 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.151 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:26.151 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.151 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:26.151 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.672139+0000", 2024-09-20T18:51:26.151 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:26.151 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.151 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.151 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:26.151 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.672139+0000", 2024-09-20T18:51:26.151 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:26.151 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.152 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.152 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:26.152 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.672141+0000", 2024-09-20T18:51:26.152 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5e-06 2024-09-20T18:51:26.152 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.152 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.152 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:26.152 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.672159+0000", 2024-09-20T18:51:26.152 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8493999999999999e-05 2024-09-20T18:51:26.152 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.152 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.152 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:26.152 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.672171+0000", 2024-09-20T18:51:26.153 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1907e-05 2024-09-20T18:51:26.153 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.153 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.153 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:26.153 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.672178+0000", 2024-09-20T18:51:26.153 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0149999999999998e-06 2024-09-20T18:51:26.153 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.153 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.153 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:26.153 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.672225+0000", 2024-09-20T18:51:26.153 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7182999999999999e-05 2024-09-20T18:51:26.153 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.153 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.153 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:26.154 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.672340+0000", 2024-09-20T18:51:26.154 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115241 2024-09-20T18:51:26.154 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.154 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.154 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,6", 2024-09-20T18:51:26.154 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.672478+0000", 2024-09-20T18:51:26.154 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000137286 2024-09-20T18:51:26.154 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.154 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.154 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:51:26.154 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.674624+0000", 2024-09-20T18:51:26.154 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0021462740000000001 2024-09-20T18:51:26.154 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.154 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.155 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:26.155 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.689016+0000", 2024-09-20T18:51:26.155 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0143921 2024-09-20T18:51:26.155 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.155 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.155 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:26.155 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.692428+0000", 2024-09-20T18:51:26.155 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00341201 2024-09-20T18:51:26.155 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.155 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.156 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:51:26.156 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.692696+0000", 2024-09-20T18:51:26.156 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00026757499999999999 2024-09-20T18:51:26.156 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.156 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.156 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:26.156 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.692735+0000", 2024-09-20T18:51:26.156 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9054000000000003e-05 2024-09-20T18:51:26.156 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:26.156 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:26.156 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:26.156 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.156 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.157 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:22350 7.b 7:d89037b8:::benchmark_data_smithi083_70643_object22349:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-20T18:51:26.157 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:05.676242+0000", 2024-09-20T18:51:26.157 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 20.423724982, 2024-09-20T18:51:26.157 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.016721440000000001, 2024-09-20T18:51:26.157 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:26.157 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:26.157 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:51:26.157 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:51:26.157 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:51:26.157 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:51:26.157 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 22350 2024-09-20T18:51:26.157 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.157 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:26.158 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.158 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:26.158 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.676242+0000", 2024-09-20T18:51:26.158 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:26.158 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.158 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.158 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:26.158 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.676242+0000", 2024-09-20T18:51:26.158 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:26.158 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.158 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.158 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:26.158 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.676243+0000", 2024-09-20T18:51:26.158 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8500000000000001e-06 2024-09-20T18:51:26.158 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.159 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.159 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:26.159 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.676263+0000", 2024-09-20T18:51:26.159 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9391000000000001e-05 2024-09-20T18:51:26.159 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.159 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.159 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:26.159 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.676276+0000", 2024-09-20T18:51:26.159 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3047000000000001e-05 2024-09-20T18:51:26.159 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.159 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.159 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:26.159 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.676284+0000", 2024-09-20T18:51:26.159 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7810000000000005e-06 2024-09-20T18:51:26.160 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.160 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.160 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:26.160 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.676363+0000", 2024-09-20T18:51:26.160 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9369000000000005e-05 2024-09-20T18:51:26.160 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.160 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.160 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:26.160 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.676516+0000", 2024-09-20T18:51:26.160 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000153246 2024-09-20T18:51:26.160 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.160 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.160 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,6", 2024-09-20T18:51:26.160 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.676649+0000", 2024-09-20T18:51:26.161 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000133075 2024-09-20T18:51:26.161 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.161 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.161 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:51:26.161 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.677643+0000", 2024-09-20T18:51:26.161 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00099359100000000005 2024-09-20T18:51:26.161 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.161 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.161 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:26.161 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.689108+0000", 2024-09-20T18:51:26.161 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011465377000000001 2024-09-20T18:51:26.161 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.161 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.161 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:26.161 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.692772+0000", 2024-09-20T18:51:26.162 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0036637520000000002 2024-09-20T18:51:26.162 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.162 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.162 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:51:26.162 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.692922+0000", 2024-09-20T18:51:26.162 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000150428 2024-09-20T18:51:26.162 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.162 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.162 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:26.162 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:05.692963+0000", 2024-09-20T18:51:26.162 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0533e-05 2024-09-20T18:51:26.162 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:26.163 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:26.163 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:26.163 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.163 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.163 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:23591 7.b 7:dd07bdd9:::benchmark_data_smithi083_70643_object23590:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-20T18:51:26.163 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:06.367987+0000", 2024-09-20T18:51:26.163 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 19.731979484, 2024-09-20T18:51:26.163 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.030545533999999999, 2024-09-20T18:51:26.163 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:26.163 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:26.163 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:51:26.163 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:51:26.163 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:51:26.163 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:51:26.163 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 23591 2024-09-20T18:51:26.164 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.164 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:26.164 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.164 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:26.164 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.367987+0000", 2024-09-20T18:51:26.164 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:26.164 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.164 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.164 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:26.164 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.367987+0000", 2024-09-20T18:51:26.164 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:26.164 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.165 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.165 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:26.165 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.367988+0000", 2024-09-20T18:51:26.165 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2500000000000001e-06 2024-09-20T18:51:26.165 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.165 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.166 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:26.166 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.368013+0000", 2024-09-20T18:51:26.166 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4593999999999999e-05 2024-09-20T18:51:26.166 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.166 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.166 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:26.166 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.368024+0000", 2024-09-20T18:51:26.166 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.079e-05 2024-09-20T18:51:26.166 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.166 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.166 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:26.166 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.368031+0000", 2024-09-20T18:51:26.166 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.6410000000000002e-06 2024-09-20T18:51:26.166 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.167 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.167 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:26.167 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.370159+0000", 2024-09-20T18:51:26.167 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0021280660000000001 2024-09-20T18:51:26.167 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.167 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.167 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:26.167 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.370277+0000", 2024-09-20T18:51:26.167 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000117198 2024-09-20T18:51:26.167 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.167 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.167 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,6", 2024-09-20T18:51:26.167 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.370401+0000", 2024-09-20T18:51:26.167 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012465600000000001 2024-09-20T18:51:26.168 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.168 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.168 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:51:26.168 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.371613+0000", 2024-09-20T18:51:26.168 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001211953 2024-09-20T18:51:26.168 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.168 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.168 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:26.168 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.398392+0000", 2024-09-20T18:51:26.168 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.026778689000000001 2024-09-20T18:51:26.168 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.168 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.168 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:26.168 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.398437+0000", 2024-09-20T18:51:26.168 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4739e-05 2024-09-20T18:51:26.169 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.169 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.169 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:51:26.169 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.398491+0000", 2024-09-20T18:51:26.169 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4722000000000002e-05 2024-09-20T18:51:26.169 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.169 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.169 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:26.169 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.398533+0000", 2024-09-20T18:51:26.169 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1236000000000003e-05 2024-09-20T18:51:26.169 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:26.169 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:26.169 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:26.169 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.169 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.170 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:23602 7.b 7:d51fe338:::benchmark_data_smithi083_70643_object23601:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-20T18:51:26.170 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:06.371799+0000", 2024-09-20T18:51:26.170 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 19.728167592999998, 2024-09-20T18:51:26.170 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.026913688000000002, 2024-09-20T18:51:26.170 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:26.170 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:26.170 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:51:26.170 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:51:26.170 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:51:26.170 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:51:26.170 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 23602 2024-09-20T18:51:26.170 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.170 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:26.171 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.171 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:26.171 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.371799+0000", 2024-09-20T18:51:26.171 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:26.171 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.171 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.171 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:26.171 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.371799+0000", 2024-09-20T18:51:26.171 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:26.171 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.171 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.171 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:26.171 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.371800+0000", 2024-09-20T18:51:26.171 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.212e-06 2024-09-20T18:51:26.172 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.172 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.172 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:26.172 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.371819+0000", 2024-09-20T18:51:26.172 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9232000000000002e-05 2024-09-20T18:51:26.172 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.172 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.172 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:26.172 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.371828+0000", 2024-09-20T18:51:26.172 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7790000000000001e-06 2024-09-20T18:51:26.172 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.172 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.172 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:26.173 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.371837+0000", 2024-09-20T18:51:26.173 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7469999999999994e-06 2024-09-20T18:51:26.173 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.173 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.173 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:26.173 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.371895+0000", 2024-09-20T18:51:26.173 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7917000000000001e-05 2024-09-20T18:51:26.173 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.173 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.173 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:26.173 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.372013+0000", 2024-09-20T18:51:26.173 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011793 2024-09-20T18:51:26.173 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.173 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.174 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,6", 2024-09-20T18:51:26.174 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.372150+0000", 2024-09-20T18:51:26.174 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000136911 2024-09-20T18:51:26.174 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.174 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.174 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:26.174 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.398574+0000", 2024-09-20T18:51:26.174 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.026423945000000001 2024-09-20T18:51:26.174 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.174 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.174 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:26.174 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.398624+0000", 2024-09-20T18:51:26.174 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0249e-05 2024-09-20T18:51:26.174 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.175 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.175 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:51:26.175 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.398638+0000", 2024-09-20T18:51:26.175 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3692e-05 2024-09-20T18:51:26.175 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.175 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.175 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:51:26.175 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.398676+0000", 2024-09-20T18:51:26.175 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8245999999999999e-05 2024-09-20T18:51:26.175 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.175 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.175 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:26.175 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.398713+0000", 2024-09-20T18:51:26.175 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6828e-05 2024-09-20T18:51:26.175 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:26.176 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:26.176 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:26.176 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.176 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.176 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:23605 7.b 7:d9c5658a:::benchmark_data_smithi083_70643_object23604:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-20T18:51:26.176 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:06.371935+0000", 2024-09-20T18:51:26.176 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 19.72803184, 2024-09-20T18:51:26.176 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.027832982999999999, 2024-09-20T18:51:26.176 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:26.176 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:26.176 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:51:26.176 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:51:26.176 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:51:26.176 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:51:26.176 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 23605 2024-09-20T18:51:26.177 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.177 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:26.177 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.177 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:26.177 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.371935+0000", 2024-09-20T18:51:26.177 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:26.177 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.177 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.177 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:26.177 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.371935+0000", 2024-09-20T18:51:26.177 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:26.177 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.177 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.177 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:26.178 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.371935+0000", 2024-09-20T18:51:26.178 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9999999999999997e-07 2024-09-20T18:51:26.178 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.178 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.178 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:26.178 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.371956+0000", 2024-09-20T18:51:26.178 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0945999999999999e-05 2024-09-20T18:51:26.178 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.178 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.178 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:26.178 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.371966+0000", 2024-09-20T18:51:26.178 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9199999999999999e-06 2024-09-20T18:51:26.178 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.178 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.178 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:26.179 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.371972+0000", 2024-09-20T18:51:26.179 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6980000000000003e-06 2024-09-20T18:51:26.179 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.179 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.179 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:26.179 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.397297+0000", 2024-09-20T18:51:26.179 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.025324625 2024-09-20T18:51:26.179 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.179 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.179 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:26.179 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.397400+0000", 2024-09-20T18:51:26.179 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000103849 2024-09-20T18:51:26.179 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.180 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.180 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,6", 2024-09-20T18:51:26.180 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.397522+0000", 2024-09-20T18:51:26.180 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121904 2024-09-20T18:51:26.180 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.180 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.180 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:51:26.180 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.398719+0000", 2024-09-20T18:51:26.180 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011972020000000001 2024-09-20T18:51:26.180 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.180 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.180 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:26.180 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.399631+0000", 2024-09-20T18:51:26.180 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00091135599999999999 2024-09-20T18:51:26.180 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.181 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.181 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:26.181 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.399674+0000", 2024-09-20T18:51:26.181 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3217000000000003e-05 2024-09-20T18:51:26.181 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.181 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.181 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:51:26.181 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.399729+0000", 2024-09-20T18:51:26.181 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.465e-05 2024-09-20T18:51:26.181 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.181 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.181 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:26.181 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.399768+0000", 2024-09-20T18:51:26.181 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9016e-05 2024-09-20T18:51:26.182 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:26.182 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:26.182 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:26.182 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.182 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.182 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:23591 7.b e92/83 ondisk, result = 0)", 2024-09-20T18:51:26.182 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:06.372093+0000", 2024-09-20T18:51:26.182 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 19.727873577, 2024-09-20T18:51:26.182 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.026308642, 2024-09-20T18:51:26.182 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:26.183 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:26.183 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:51:26.183 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:26.183 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.183 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:26.183 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.372093+0000", 2024-09-20T18:51:26.183 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:26.183 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.183 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.183 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:26.183 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.372093+0000", 2024-09-20T18:51:26.183 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:26.183 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.184 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.184 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:26.184 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.372094+0000", 2024-09-20T18:51:26.184 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0079999999999999e-06 2024-09-20T18:51:26.184 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.184 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.184 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:26.184 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.372099+0000", 2024-09-20T18:51:26.184 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7690000000000004e-06 2024-09-20T18:51:26.184 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.184 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.184 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:26.184 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.372108+0000", 2024-09-20T18:51:26.184 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0759999999999992e-06 2024-09-20T18:51:26.185 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.185 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.185 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:26.185 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.372113+0000", 2024-09-20T18:51:26.185 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.66e-06 2024-09-20T18:51:26.185 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.185 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.185 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:26.185 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.398378+0000", 2024-09-20T18:51:26.185 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.026264781000000001 2024-09-20T18:51:26.185 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.185 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.185 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:26.185 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.398386+0000", 2024-09-20T18:51:26.186 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2649999999999999e-06 2024-09-20T18:51:26.186 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.186 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.186 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:26.186 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.398402+0000", 2024-09-20T18:51:26.186 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6082999999999998e-05 2024-09-20T18:51:26.186 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:26.186 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:26.186 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:26.186 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.186 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.186 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:23591 7.b e92/83 ondisk, result = 0)", 2024-09-20T18:51:26.186 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:06.372174+0000", 2024-09-20T18:51:26.187 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 19.727792198, 2024-09-20T18:51:26.187 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.02636616, 2024-09-20T18:51:26.187 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:26.187 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:26.187 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:51:26.187 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:26.187 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.187 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:26.187 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.372174+0000", 2024-09-20T18:51:26.187 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:26.187 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.187 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.187 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:26.188 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.372174+0000", 2024-09-20T18:51:26.188 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:26.188 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.188 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.188 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:26.188 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.372175+0000", 2024-09-20T18:51:26.188 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8299999999999998e-07 2024-09-20T18:51:26.188 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.188 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.188 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:26.188 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.372177+0000", 2024-09-20T18:51:26.188 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5170000000000002e-06 2024-09-20T18:51:26.188 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.188 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.189 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:26.189 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.396275+0000", 2024-09-20T18:51:26.189 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.024097741999999998 2024-09-20T18:51:26.189 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.189 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.189 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:26.189 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.396301+0000", 2024-09-20T18:51:26.189 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6492999999999999e-05 2024-09-20T18:51:26.189 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.189 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.189 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:26.189 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.398426+0000", 2024-09-20T18:51:26.189 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0021247869999999999 2024-09-20T18:51:26.189 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.190 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.190 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:26.190 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.398432+0000", 2024-09-20T18:51:26.190 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9410000000000004e-06 2024-09-20T18:51:26.190 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.190 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.190 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:26.190 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.398541+0000", 2024-09-20T18:51:26.190 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000108297 2024-09-20T18:51:26.190 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:26.190 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:26.190 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:26.190 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.190 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.191 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:24397 7.b 7:d6836ea8:::benchmark_data_smithi083_70643_object24396:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:51:26.191 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:06.732401+0000", 2024-09-20T18:51:26.191 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 19.367565844000001, 2024-09-20T18:51:26.191 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044853705000000001, 2024-09-20T18:51:26.191 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:26.191 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:26.191 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:51:26.191 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:51:26.191 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:51:26.191 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:51:26.191 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 24397 2024-09-20T18:51:26.192 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.192 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:26.192 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.192 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:26.192 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.732401+0000", 2024-09-20T18:51:26.192 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:26.192 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.192 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.192 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:26.192 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.732401+0000", 2024-09-20T18:51:26.192 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:26.192 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.192 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.192 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:26.192 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.732402+0000", 2024-09-20T18:51:26.193 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.302e-06 2024-09-20T18:51:26.193 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.193 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.193 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:26.193 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.732421+0000", 2024-09-20T18:51:26.193 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8586000000000001e-05 2024-09-20T18:51:26.193 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.193 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.193 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:26.193 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.732428+0000", 2024-09-20T18:51:26.193 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.6140000000000003e-06 2024-09-20T18:51:26.193 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.193 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.193 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:26.194 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.732433+0000", 2024-09-20T18:51:26.194 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1080000000000001e-06 2024-09-20T18:51:26.194 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.194 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.194 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:26.194 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.732907+0000", 2024-09-20T18:51:26.194 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000473451 2024-09-20T18:51:26.194 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.194 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.194 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:26.194 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.733002+0000", 2024-09-20T18:51:26.194 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5525000000000004e-05 2024-09-20T18:51:26.194 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.194 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.195 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,6", 2024-09-20T18:51:26.195 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.733140+0000", 2024-09-20T18:51:26.195 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013752999999999999 2024-09-20T18:51:26.195 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.195 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.195 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:51:26.195 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.733992+0000", 2024-09-20T18:51:26.195 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00085219799999999997 2024-09-20T18:51:26.195 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.195 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.195 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:26.195 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.734616+0000", 2024-09-20T18:51:26.195 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00062414899999999995 2024-09-20T18:51:26.195 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.195 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.196 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:26.196 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.777162+0000", 2024-09-20T18:51:26.196 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042545432000000001 2024-09-20T18:51:26.196 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.196 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.196 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:51:26.196 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.777213+0000", 2024-09-20T18:51:26.196 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1499000000000003e-05 2024-09-20T18:51:26.196 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.196 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.196 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:26.196 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.777254+0000", 2024-09-20T18:51:26.196 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1310999999999998e-05 2024-09-20T18:51:26.196 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:26.196 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:26.197 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:26.197 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.197 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.197 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:24402 7.d e93/83 7:ba8a96ba:::benchmark_data_smithi083_70643_object24401:head v 93'1559, mlcod=93'1557)", 2024-09-20T18:51:26.197 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:06.734946+0000", 2024-09-20T18:51:26.197 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 19.365020218000002, 2024-09-20T18:51:26.197 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044916588, 2024-09-20T18:51:26.197 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:26.197 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:26.197 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:51:26.197 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:26.197 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.197 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:26.197 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.734946+0000", 2024-09-20T18:51:26.198 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:26.198 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.198 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.198 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:26.198 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.734946+0000", 2024-09-20T18:51:26.198 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:26.198 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.198 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.198 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:26.198 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.734947+0000", 2024-09-20T18:51:26.198 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1800000000000001e-07 2024-09-20T18:51:26.198 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.198 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.198 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:26.199 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.735008+0000", 2024-09-20T18:51:26.199 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1414000000000005e-05 2024-09-20T18:51:26.199 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.199 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.199 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:26.199 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.735013+0000", 2024-09-20T18:51:26.199 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8659999999999997e-06 2024-09-20T18:51:26.199 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.199 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.199 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:26.199 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.735017+0000", 2024-09-20T18:51:26.199 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5269999999999999e-06 2024-09-20T18:51:26.199 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.200 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.200 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:26.200 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.777629+0000", 2024-09-20T18:51:26.200 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042612075999999999 2024-09-20T18:51:26.200 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.200 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.200 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:26.200 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.777651+0000", 2024-09-20T18:51:26.200 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1936000000000001e-05 2024-09-20T18:51:26.200 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.200 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.200 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:51:26.200 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.779794+0000", 2024-09-20T18:51:26.200 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0021430529999999998 2024-09-20T18:51:26.201 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.201 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.201 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:26.201 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.779863+0000", 2024-09-20T18:51:26.201 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9298e-05 2024-09-20T18:51:26.201 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:26.201 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:26.201 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:26.201 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.201 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.201 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:24397 7.b e93/83 ondisk, result = 0)", 2024-09-20T18:51:26.201 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:06.735299+0000", 2024-09-20T18:51:26.201 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 19.364667282999999, 2024-09-20T18:51:26.201 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041962289999999999, 2024-09-20T18:51:26.202 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:26.202 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:26.202 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:51:26.202 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:26.202 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.202 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:26.202 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.735299+0000", 2024-09-20T18:51:26.202 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:26.202 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.202 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.202 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:26.202 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.735299+0000", 2024-09-20T18:51:26.202 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:26.202 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.203 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.203 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:26.203 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.735300+0000", 2024-09-20T18:51:26.203 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2899999999999999e-07 2024-09-20T18:51:26.203 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.203 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.203 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:26.203 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.735301+0000", 2024-09-20T18:51:26.203 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7570000000000001e-06 2024-09-20T18:51:26.203 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.203 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.203 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:26.203 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.776997+0000", 2024-09-20T18:51:26.204 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041696080000000003 2024-09-20T18:51:26.204 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.204 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.204 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:26.204 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.777014+0000", 2024-09-20T18:51:26.204 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6084999999999999e-05 2024-09-20T18:51:26.204 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.204 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.204 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:26.204 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.777148+0000", 2024-09-20T18:51:26.204 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000134356 2024-09-20T18:51:26.204 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.204 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.204 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:26.205 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.777156+0000", 2024-09-20T18:51:26.205 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.1000000000000004e-06 2024-09-20T18:51:26.205 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.205 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.205 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:26.205 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.777262+0000", 2024-09-20T18:51:26.205 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000105583 2024-09-20T18:51:26.205 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:26.205 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:26.205 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:26.205 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.205 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.205 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:24395 7.4 e93/83 7:2a1f9aae:::benchmark_data_smithi083_70643_object24394:head v 93'1498, mlcod=93'1495)", 2024-09-20T18:51:26.205 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:06.742919+0000", 2024-09-20T18:51:26.205 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 19.357047942000001, 2024-09-20T18:51:26.206 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.038764387999999997, 2024-09-20T18:51:26.206 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:26.206 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:26.206 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:51:26.206 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:26.206 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.206 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:26.206 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.742919+0000", 2024-09-20T18:51:26.206 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:26.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.206 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.206 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:26.206 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.742919+0000", 2024-09-20T18:51:26.206 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:26.207 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.207 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.207 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:26.207 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.742919+0000", 2024-09-20T18:51:26.207 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7000000000000004e-07 2024-09-20T18:51:26.207 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:26.208 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.208 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.208 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:26.209 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.742947+0000", 2024-09-20T18:51:26.209 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7368999999999998e-05 2024-09-20T18:51:26.209 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.209 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.209 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:26.209 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.777468+0000", 2024-09-20T18:51:26.209 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.034521005 2024-09-20T18:51:26.209 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.209 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.209 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:26.209 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.777479+0000", 2024-09-20T18:51:26.209 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0951000000000001e-05 2024-09-20T18:51:26.209 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.209 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.210 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:26.210 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.777532+0000", 2024-09-20T18:51:26.210 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3461999999999997e-05 2024-09-20T18:51:26.210 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.210 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.210 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:26.210 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.777550+0000", 2024-09-20T18:51:26.210 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8006e-05 2024-09-20T18:51:26.210 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.210 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.210 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:51:26.210 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.781650+0000", 2024-09-20T18:51:26.210 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0040995040000000003 2024-09-20T18:51:26.210 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.211 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.211 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:26.211 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.781683+0000", 2024-09-20T18:51:26.211 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3321000000000002e-05 2024-09-20T18:51:26.211 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:26.211 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:26.211 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:26.211 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.211 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.211 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:32898 7.b 7:d9277d93:::benchmark_data_smithi083_70643_object32897:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T18:51:26.211 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:11.727738+0000", 2024-09-20T18:51:26.211 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 14.372228842, 2024-09-20T18:51:26.211 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.063835517999999994, 2024-09-20T18:51:26.211 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:26.212 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:26.212 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:51:26.212 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:51:26.212 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:51:26.212 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:51:26.212 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 32898 2024-09-20T18:51:26.212 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.212 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:26.212 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.212 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:26.212 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:11.727738+0000", 2024-09-20T18:51:26.212 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:26.212 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.212 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.213 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:26.213 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:11.727738+0000", 2024-09-20T18:51:26.213 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:26.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.213 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.213 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:26.213 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:11.727739+0000", 2024-09-20T18:51:26.213 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7179999999999999e-06 2024-09-20T18:51:26.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.213 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.213 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:26.213 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:11.727765+0000", 2024-09-20T18:51:26.213 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5757000000000001e-05 2024-09-20T18:51:26.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.214 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.214 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:26.214 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:11.727783+0000", 2024-09-20T18:51:26.214 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8201000000000001e-05 2024-09-20T18:51:26.214 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.214 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.214 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:26.214 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:11.727793+0000", 2024-09-20T18:51:26.214 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2129999999999997e-06 2024-09-20T18:51:26.214 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.214 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.214 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:26.214 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:11.727901+0000", 2024-09-20T18:51:26.214 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000108597 2024-09-20T18:51:26.215 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.215 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.215 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:26.215 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:11.728021+0000", 2024-09-20T18:51:26.215 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000119924 2024-09-20T18:51:26.215 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.215 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.215 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,6", 2024-09-20T18:51:26.215 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:11.728156+0000", 2024-09-20T18:51:26.215 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000135088 2024-09-20T18:51:26.215 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.215 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.215 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:51:26.215 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:11.732685+0000", 2024-09-20T18:51:26.215 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0045288430000000003 2024-09-20T18:51:26.216 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.216 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.216 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:26.216 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:11.787480+0000", 2024-09-20T18:51:26.216 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.054795409000000003 2024-09-20T18:51:26.216 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.216 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.216 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:26.216 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:11.791490+0000", 2024-09-20T18:51:26.216 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0040093719999999998 2024-09-20T18:51:26.216 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.216 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.216 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:51:26.216 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:11.791531+0000", 2024-09-20T18:51:26.216 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1421000000000002e-05 2024-09-20T18:51:26.217 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.217 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.217 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:26.217 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:11.791573+0000", 2024-09-20T18:51:26.217 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1974999999999997e-05 2024-09-20T18:51:26.217 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:26.217 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:26.217 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:26.217 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.217 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.217 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:35683 7.b 7:db6e997a:::benchmark_data_smithi083_70643_object35682:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T18:51:26.217 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:13.220311+0000", 2024-09-20T18:51:26.217 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 12.879655949, 2024-09-20T18:51:26.217 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019046355000000001, 2024-09-20T18:51:26.217 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:26.217 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:26.218 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:51:26.218 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:51:26.218 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:51:26.218 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:51:26.218 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 35683 2024-09-20T18:51:26.218 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.218 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:26.218 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.218 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:26.218 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:13.220311+0000", 2024-09-20T18:51:26.218 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:26.218 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.218 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.218 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:26.219 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:13.220311+0000", 2024-09-20T18:51:26.219 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:26.219 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.219 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.219 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:26.219 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:13.220312+0000", 2024-09-20T18:51:26.219 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.435e-06 2024-09-20T18:51:26.219 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.219 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.219 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:26.219 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:13.220334+0000", 2024-09-20T18:51:26.219 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1480000000000001e-05 2024-09-20T18:51:26.219 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.219 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.219 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:26.219 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:13.220342+0000", 2024-09-20T18:51:26.220 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6989999999999992e-06 2024-09-20T18:51:26.220 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.220 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.220 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:26.220 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:13.220348+0000", 2024-09-20T18:51:26.220 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1650000000000003e-06 2024-09-20T18:51:26.220 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.220 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.220 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:26.220 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:13.223719+0000", 2024-09-20T18:51:26.220 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0033710469999999998 2024-09-20T18:51:26.220 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.220 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.220 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:26.220 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:13.223871+0000", 2024-09-20T18:51:26.221 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000151104 2024-09-20T18:51:26.221 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.221 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.221 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,6", 2024-09-20T18:51:26.221 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:13.223983+0000", 2024-09-20T18:51:26.222 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000112321 2024-09-20T18:51:26.222 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.222 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.222 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:26.222 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:13.232644+0000", 2024-09-20T18:51:26.223 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0086607799999999999 2024-09-20T18:51:26.223 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.223 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.223 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:26.223 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:13.232680+0000", 2024-09-20T18:51:26.223 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6757000000000001e-05 2024-09-20T18:51:26.223 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.223 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.223 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:51:26.223 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:13.239266+0000", 2024-09-20T18:51:26.223 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065857779999999996 2024-09-20T18:51:26.223 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.223 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.223 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:51:26.224 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:13.239312+0000", 2024-09-20T18:51:26.224 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5701000000000002e-05 2024-09-20T18:51:26.224 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.224 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.224 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:26.224 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:13.239357+0000", 2024-09-20T18:51:26.224 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5087999999999999e-05 2024-09-20T18:51:26.224 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:26.224 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:26.224 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:26.225 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.225 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.225 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:35687 7.b 7:db4e71b8:::benchmark_data_smithi083_70643_object35686:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T18:51:26.225 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:13.223301+0000", 2024-09-20T18:51:26.225 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 12.87666533, 2024-09-20T18:51:26.225 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.016187709000000002, 2024-09-20T18:51:26.225 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:26.225 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:26.225 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:51:26.225 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:51:26.225 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:51:26.225 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:51:26.225 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 35687 2024-09-20T18:51:26.226 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.226 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:26.226 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.226 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:26.226 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:13.223301+0000", 2024-09-20T18:51:26.226 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:26.226 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.226 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.226 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:26.226 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:13.223301+0000", 2024-09-20T18:51:26.226 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:26.226 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.226 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.226 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:26.227 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:13.223303+0000", 2024-09-20T18:51:26.227 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.705e-06 2024-09-20T18:51:26.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.227 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.227 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:26.227 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:13.223332+0000", 2024-09-20T18:51:26.227 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.868e-05 2024-09-20T18:51:26.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.227 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.227 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:26.227 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:13.223340+0000", 2024-09-20T18:51:26.227 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9549999999999992e-06 2024-09-20T18:51:26.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.227 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.228 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:26.228 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:13.223346+0000", 2024-09-20T18:51:26.228 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2450000000000003e-06 2024-09-20T18:51:26.228 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.228 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.228 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:26.228 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:13.227754+0000", 2024-09-20T18:51:26.228 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.004408454 2024-09-20T18:51:26.228 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.228 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.228 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:26.228 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:13.230459+0000", 2024-09-20T18:51:26.228 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0027050210000000002 2024-09-20T18:51:26.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.229 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.229 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,6", 2024-09-20T18:51:26.229 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:13.230596+0000", 2024-09-20T18:51:26.229 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013645700000000001 2024-09-20T18:51:26.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.229 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.229 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:26.229 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:13.232712+0000", 2024-09-20T18:51:26.229 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0021162860000000002 2024-09-20T18:51:26.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.229 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.229 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:26.229 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:13.232745+0000", 2024-09-20T18:51:26.230 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2833999999999998e-05 2024-09-20T18:51:26.230 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.230 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.230 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:51:26.230 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:13.239411+0000", 2024-09-20T18:51:26.230 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0066666249999999998 2024-09-20T18:51:26.230 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.230 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.230 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:51:26.230 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:13.239455+0000", 2024-09-20T18:51:26.230 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3238999999999997e-05 2024-09-20T18:51:26.230 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.230 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.231 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:26.231 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:13.239489+0000", 2024-09-20T18:51:26.231 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4208000000000002e-05 2024-09-20T18:51:26.231 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:26.231 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:26.231 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:26.231 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.231 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.231 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:54530 7.d e98/83 7:b23c48bc:::benchmark_data_smithi083_70643_object54529:head v 98'3473, mlcod=98'3470)", 2024-09-20T18:51:26.231 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:25.155335+0000", 2024-09-20T18:51:26.231 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.94463164899999996, 2024-09-20T18:51:26.231 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.016108796000000002, 2024-09-20T18:51:26.231 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:26.231 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:26.232 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:51:26.232 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:26.232 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.232 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:26.232 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:25.155335+0000", 2024-09-20T18:51:26.232 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:26.232 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.232 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.232 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:26.232 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:25.155335+0000", 2024-09-20T18:51:26.232 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:26.232 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.233 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.233 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:26.233 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:25.155336+0000", 2024-09-20T18:51:26.233 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9500000000000002e-07 2024-09-20T18:51:26.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.233 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.233 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:26.233 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:25.155377+0000", 2024-09-20T18:51:26.233 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1199000000000003e-05 2024-09-20T18:51:26.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.233 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.233 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:26.233 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:25.155387+0000", 2024-09-20T18:51:26.233 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8039999999999996e-06 2024-09-20T18:51:26.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.234 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.234 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:26.234 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:25.155393+0000", 2024-09-20T18:51:26.234 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0610000000000001e-06 2024-09-20T18:51:26.234 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.234 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.234 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:26.234 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:25.155532+0000", 2024-09-20T18:51:26.234 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013931600000000001 2024-09-20T18:51:26.234 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.234 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.234 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:26.234 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:25.155548+0000", 2024-09-20T18:51:26.234 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6436e-05 2024-09-20T18:51:26.235 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.235 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.235 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:51:26.235 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:25.171363+0000", 2024-09-20T18:51:26.235 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.015814992 2024-09-20T18:51:26.235 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.235 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.235 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:26.235 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:25.171444+0000", 2024-09-20T18:51:26.235 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0292999999999998e-05 2024-09-20T18:51:26.235 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:26.235 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:26.235 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:26.235 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.236 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.236 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:55807 7.b 7:d480c117:::benchmark_data_smithi083_70643_object55806:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-20T18:51:26.236 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:26.003505+0000", 2024-09-20T18:51:26.236 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.096461794000000003, 2024-09-20T18:51:26.236 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.016223411, 2024-09-20T18:51:26.236 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:26.236 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:26.236 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:51:26.236 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:51:26.236 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:51:26.236 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:51:26.236 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 55807 2024-09-20T18:51:26.236 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.236 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:26.237 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.237 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:26.237 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:26.003505+0000", 2024-09-20T18:51:26.237 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:26.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.237 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.237 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:26.237 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:26.003505+0000", 2024-09-20T18:51:26.237 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:26.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.237 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.237 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:26.237 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:26.003506+0000", 2024-09-20T18:51:26.237 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.53e-06 2024-09-20T18:51:26.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.238 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.238 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:26.238 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:26.003543+0000", 2024-09-20T18:51:26.238 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7027999999999999e-05 2024-09-20T18:51:26.238 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.238 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.238 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:26.238 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:26.003558+0000", 2024-09-20T18:51:26.238 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5109999999999999e-05 2024-09-20T18:51:26.238 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.238 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.238 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:26.238 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:26.003568+0000", 2024-09-20T18:51:26.239 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0899999999999994e-06 2024-09-20T18:51:26.239 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.239 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.239 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:26.239 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:26.003624+0000", 2024-09-20T18:51:26.239 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6808000000000002e-05 2024-09-20T18:51:26.239 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.239 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.239 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:26.239 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:26.003747+0000", 2024-09-20T18:51:26.239 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000123103 2024-09-20T18:51:26.239 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.239 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.239 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,6", 2024-09-20T18:51:26.240 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:26.003887+0000", 2024-09-20T18:51:26.240 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000139147 2024-09-20T18:51:26.240 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.240 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.240 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:26.240 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:26.011312+0000", 2024-09-20T18:51:26.240 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074254339999999999 2024-09-20T18:51:26.240 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.240 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.240 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:26.240 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:26.011479+0000", 2024-09-20T18:51:26.240 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016672599999999999 2024-09-20T18:51:26.240 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.241 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.241 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:51:26.241 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:26.019639+0000", 2024-09-20T18:51:26.241 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0081602659999999994 2024-09-20T18:51:26.241 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.241 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.241 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:51:26.241 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:26.019684+0000", 2024-09-20T18:51:26.241 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5386999999999998e-05 2024-09-20T18:51:26.241 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:26.241 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:26.241 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:26.241 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:26.019728+0000", 2024-09-20T18:51:26.241 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3782000000000002e-05 2024-09-20T18:51:26.241 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:26.242 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:26.242 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:26.242 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:26.242 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:26.242 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T18:51:26.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:26.243 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:26.244 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:26.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:25 smithi142 ceph-mon[32508]: 5.f scrub starts 2024-09-20T18:51:26.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:25 smithi142 ceph-mon[32508]: 5.f scrub ok 2024-09-20T18:51:26.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:25 smithi142 ceph-mon[32508]: pgmap v62: 148 pgs: 148 active+clean; 2.9 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 853 B/s rd, 84 MiB/s wr, 1.35k op/s 2024-09-20T18:51:26.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:25 smithi142 ceph-mon[32508]: 6.3 scrub starts 2024-09-20T18:51:26.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:25 smithi142 ceph-mon[32508]: 6.3 scrub ok 2024-09-20T18:51:26.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:25 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:51:26.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:25 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:51:26.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:25 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:26.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:25 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:26.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:25 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:51:26.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:25 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:51:26.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:25 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:26.344 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:26.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:26.386 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:26.386 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:26.487 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:26.519 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:26.520 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 29 15 56521 56506 121.748 88.375 0.022469 0.00820843 2024-09-20T18:51:26.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:26.525 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:26.626 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:26.652 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:26.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:26.653 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:26.754 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:26.773 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:51:26 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:51:26.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:26.781 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:26.782 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:26.883 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:26.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:26.924 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:26.924 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:27.025 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:27.055 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:27.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:27.056 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:27.156 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:27.181 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:26 smithi083 ceph-mon[29421]: 6.10 scrub starts 2024-09-20T18:51:27.181 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:26 smithi083 ceph-mon[29421]: 6.10 scrub ok 2024-09-20T18:51:27.181 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:26 smithi083 ceph-mon[29421]: 3.1e scrub starts 2024-09-20T18:51:27.181 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:26 smithi083 ceph-mon[29421]: 3.1e scrub ok 2024-09-20T18:51:27.181 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:26 smithi083 ceph-mon[29421]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-20T18:51:27.181 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:26 smithi083 ceph-mon[29421]: Reconfiguring daemon alertmanager.a on smithi083 2024-09-20T18:51:27.181 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:26 smithi083 ceph-mon[29421]: 6.7 scrub starts 2024-09-20T18:51:27.181 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:26 smithi083 ceph-mon[29421]: 6.7 scrub ok 2024-09-20T18:51:27.182 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:26 smithi083 ceph-mon[29421]: osdmap e99: 8 total, 8 up, 8 in 2024-09-20T18:51:27.182 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:26 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:51:27.182 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:26 smithi083 ceph-mon[29421]: 4.1b scrub starts 2024-09-20T18:51:27.182 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:26 smithi083 ceph-mon[29421]: 4.1b scrub ok 2024-09-20T18:51:27.182 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:26 smithi083 ceph-mon[36260]: 6.10 scrub starts 2024-09-20T18:51:27.182 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:26 smithi083 ceph-mon[36260]: 6.10 scrub ok 2024-09-20T18:51:27.182 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:26 smithi083 ceph-mon[36260]: 3.1e scrub starts 2024-09-20T18:51:27.182 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:26 smithi083 ceph-mon[36260]: 3.1e scrub ok 2024-09-20T18:51:27.182 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:26 smithi083 ceph-mon[36260]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-20T18:51:27.182 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:26 smithi083 ceph-mon[36260]: Reconfiguring daemon alertmanager.a on smithi083 2024-09-20T18:51:27.182 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:26 smithi083 ceph-mon[36260]: 6.7 scrub starts 2024-09-20T18:51:27.182 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:26 smithi083 ceph-mon[36260]: 6.7 scrub ok 2024-09-20T18:51:27.183 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:26 smithi083 ceph-mon[36260]: osdmap e99: 8 total, 8 up, 8 in 2024-09-20T18:51:27.183 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:26 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:51:27.183 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:26 smithi083 ceph-mon[36260]: 4.1b scrub starts 2024-09-20T18:51:27.183 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:26 smithi083 ceph-mon[36260]: 4.1b scrub ok 2024-09-20T18:51:27.199 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:27.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:27.200 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:27.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:26 smithi142 ceph-mon[32508]: 6.10 scrub starts 2024-09-20T18:51:27.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:26 smithi142 ceph-mon[32508]: 6.10 scrub ok 2024-09-20T18:51:27.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:26 smithi142 ceph-mon[32508]: 3.1e scrub starts 2024-09-20T18:51:27.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:26 smithi142 ceph-mon[32508]: 3.1e scrub ok 2024-09-20T18:51:27.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:26 smithi142 ceph-mon[32508]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-20T18:51:27.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:26 smithi142 ceph-mon[32508]: Reconfiguring daemon alertmanager.a on smithi083 2024-09-20T18:51:27.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:26 smithi142 ceph-mon[32508]: 6.7 scrub starts 2024-09-20T18:51:27.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:26 smithi142 ceph-mon[32508]: 6.7 scrub ok 2024-09-20T18:51:27.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:26 smithi142 ceph-mon[32508]: osdmap e99: 8 total, 8 up, 8 in 2024-09-20T18:51:27.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:26 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:51:27.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:26 smithi142 ceph-mon[32508]: 4.1b scrub starts 2024-09-20T18:51:27.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:26 smithi142 ceph-mon[32508]: 4.1b scrub ok 2024-09-20T18:51:27.300 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:27.330 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:27.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:27.331 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:27.432 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:27.468 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:27.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:27.469 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:27.521 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 30 16 58072 58056 120.919 96.875 0.0044876 0.00826518 2024-09-20T18:51:27.569 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:27.599 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:27.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:27.600 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:27.701 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:27.729 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:27.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:27.730 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:27.830 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:27.855 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:27.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:27.856 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:27.863 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-20T18:51:27.956 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:28.008 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:28.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:28.009 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:28.078 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:27 smithi083 ceph-mon[29421]: pgmap v64: 148 pgs: 148 active+clean; 3.4 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 853 B/s rd, 101 MiB/s wr, 1.62k op/s; 18 B/s, 0 objects/s recovering 2024-09-20T18:51:28.078 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:27 smithi083 ceph-mon[29421]: 4.12 scrub starts 2024-09-20T18:51:28.078 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:27 smithi083 ceph-mon[29421]: 4.12 scrub ok 2024-09-20T18:51:28.079 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:27 smithi083 ceph-mon[29421]: 5.1c scrub starts 2024-09-20T18:51:28.079 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:27 smithi083 ceph-mon[29421]: 5.1c scrub ok 2024-09-20T18:51:28.080 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:27 smithi083 ceph-mon[36260]: pgmap v64: 148 pgs: 148 active+clean; 3.4 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 853 B/s rd, 101 MiB/s wr, 1.62k op/s; 18 B/s, 0 objects/s recovering 2024-09-20T18:51:28.080 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:27 smithi083 ceph-mon[36260]: 4.12 scrub starts 2024-09-20T18:51:28.080 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:27 smithi083 ceph-mon[36260]: 4.12 scrub ok 2024-09-20T18:51:28.080 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:27 smithi083 ceph-mon[36260]: 5.1c scrub starts 2024-09-20T18:51:28.080 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:27 smithi083 ceph-mon[36260]: 5.1c scrub ok 2024-09-20T18:51:28.109 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:51:28.170 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:28.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:28.171 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:51:28.272 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:51:28.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:27 smithi142 ceph-mon[32508]: pgmap v64: 148 pgs: 148 active+clean; 3.4 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 853 B/s rd, 101 MiB/s wr, 1.62k op/s; 18 B/s, 0 objects/s recovering 2024-09-20T18:51:28.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:27 smithi142 ceph-mon[32508]: 4.12 scrub starts 2024-09-20T18:51:28.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:27 smithi142 ceph-mon[32508]: 4.12 scrub ok 2024-09-20T18:51:28.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:27 smithi142 ceph-mon[32508]: 5.1c scrub starts 2024-09-20T18:51:28.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:27 smithi142 ceph-mon[32508]: 5.1c scrub ok 2024-09-20T18:51:28.318 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:28.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:28.319 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:51:28.419 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:28.445 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:28.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:28.447 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:28.520 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 31 16 59521 59505 119.94 90.5625 0.00663983 0.00833224 2024-09-20T18:51:28.548 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:28.576 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:28.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:28.578 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:28.678 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:28.707 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:28.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:28.708 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:28.809 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:28.838 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:28.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:28.839 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:28.879 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T18:51:28.940 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:28.968 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:28.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:28.969 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:29.070 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:29.096 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:29.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:29.097 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:29.141 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:51:29.153 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:51:29.198 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:29.224 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:29.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:29.225 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:29.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:28 smithi142 ceph-mon[32508]: 3.6 scrub starts 2024-09-20T18:51:29.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:28 smithi142 ceph-mon[32508]: 3.6 scrub ok 2024-09-20T18:51:29.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:28 smithi142 ceph-mon[32508]: 5.1a scrub starts 2024-09-20T18:51:29.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:28 smithi142 ceph-mon[32508]: 5.1a scrub ok 2024-09-20T18:51:29.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:28 smithi142 ceph-mon[32508]: 4.1 scrub starts 2024-09-20T18:51:29.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:28 smithi142 ceph-mon[32508]: 4.1 scrub ok 2024-09-20T18:51:29.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:28 smithi083 ceph-mon[29421]: 3.6 scrub starts 2024-09-20T18:51:29.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:28 smithi083 ceph-mon[29421]: 3.6 scrub ok 2024-09-20T18:51:29.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:28 smithi083 ceph-mon[29421]: 5.1a scrub starts 2024-09-20T18:51:29.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:28 smithi083 ceph-mon[29421]: 5.1a scrub ok 2024-09-20T18:51:29.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:28 smithi083 ceph-mon[29421]: 4.1 scrub starts 2024-09-20T18:51:29.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:28 smithi083 ceph-mon[29421]: 4.1 scrub ok 2024-09-20T18:51:29.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:28 smithi083 ceph-mon[36260]: 3.6 scrub starts 2024-09-20T18:51:29.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:28 smithi083 ceph-mon[36260]: 3.6 scrub ok 2024-09-20T18:51:29.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:28 smithi083 ceph-mon[36260]: 5.1a scrub starts 2024-09-20T18:51:29.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:28 smithi083 ceph-mon[36260]: 5.1a scrub ok 2024-09-20T18:51:29.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:28 smithi083 ceph-mon[36260]: 4.1 scrub starts 2024-09-20T18:51:29.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:28 smithi083 ceph-mon[36260]: 4.1 scrub ok 2024-09-20T18:51:29.326 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:51:29.370 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:29.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:29.371 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:51:29.429 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-20T18:51:29.429 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T18:51:29.429 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-20T18:51:29.429 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd dump -f json-pretty 2024-09-20T18:51:29.472 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:51:29.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:29.513 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:29.514 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:51:29.520 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 32 16 60953 60937 118.988 89.5 0.0118434 0.00839911 2024-09-20T18:51:29.614 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:29.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:29.656 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:29.656 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:29.757 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:29.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:29.803 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:29.804 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:29.904 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:29.932 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:29.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:29.934 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:30.034 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:51:30.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:29 smithi083 ceph-mon[29421]: 5.1e scrub starts 2024-09-20T18:51:30.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:29 smithi083 ceph-mon[29421]: 5.1e scrub ok 2024-09-20T18:51:30.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:29 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:51:30.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:29 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:51:30.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:29 smithi083 ceph-mon[29421]: pgmap v65: 148 pgs: 148 active+clean; 3.4 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 853 B/s rd, 80 MiB/s wr, 1.28k op/s; 18 B/s, 0 objects/s recovering 2024-09-20T18:51:30.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:29 smithi083 ceph-mon[29421]: osdmap e100: 8 total, 8 up, 8 in 2024-09-20T18:51:30.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:29 smithi083 ceph-mon[29421]: 6.1a scrub starts 2024-09-20T18:51:30.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:29 smithi083 ceph-mon[29421]: 6.1a scrub ok 2024-09-20T18:51:30.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:29 smithi083 ceph-mon[36260]: 5.1e scrub starts 2024-09-20T18:51:30.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:29 smithi083 ceph-mon[36260]: 5.1e scrub ok 2024-09-20T18:51:30.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:29 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:51:30.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:29 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:51:30.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:29 smithi083 ceph-mon[36260]: pgmap v65: 148 pgs: 148 active+clean; 3.4 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 853 B/s rd, 80 MiB/s wr, 1.28k op/s; 18 B/s, 0 objects/s recovering 2024-09-20T18:51:30.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:29 smithi083 ceph-mon[36260]: osdmap e100: 8 total, 8 up, 8 in 2024-09-20T18:51:30.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:29 smithi083 ceph-mon[36260]: 6.1a scrub starts 2024-09-20T18:51:30.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:29 smithi083 ceph-mon[36260]: 6.1a scrub ok 2024-09-20T18:51:30.077 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:30.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:30.078 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:51:30.179 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:30.205 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:30.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:30.206 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:30.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:29 smithi142 ceph-mon[32508]: 5.1e scrub starts 2024-09-20T18:51:30.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:29 smithi142 ceph-mon[32508]: 5.1e scrub ok 2024-09-20T18:51:30.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:29 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:51:30.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:29 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:51:30.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:29 smithi142 ceph-mon[32508]: pgmap v65: 148 pgs: 148 active+clean; 3.4 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 853 B/s rd, 80 MiB/s wr, 1.28k op/s; 18 B/s, 0 objects/s recovering 2024-09-20T18:51:30.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:29 smithi142 ceph-mon[32508]: osdmap e100: 8 total, 8 up, 8 in 2024-09-20T18:51:30.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:29 smithi142 ceph-mon[32508]: 6.1a scrub starts 2024-09-20T18:51:30.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:29 smithi142 ceph-mon[32508]: 6.1a scrub ok 2024-09-20T18:51:30.310 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:51:30.346 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:30.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:30.347 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:51:30.448 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:51:30.499 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:30.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:30.500 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:51:30.521 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 33 15 62361 62346 118.051 88.0625 0.00795386 0.0084658 2024-09-20T18:51:30.601 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:30.627 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:30.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:30.629 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:30.729 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:30.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:30.757 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:30.757 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:30.858 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:30.887 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:30.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:30.888 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:30.989 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:31.018 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:30 smithi083 ceph-mon[29421]: 6.1b scrub starts 2024-09-20T18:51:31.018 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:30 smithi083 ceph-mon[29421]: 6.1b scrub ok 2024-09-20T18:51:31.018 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:30 smithi083 ceph-mon[29421]: 5.10 scrub starts 2024-09-20T18:51:31.018 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:30 smithi083 ceph-mon[29421]: 5.10 scrub ok 2024-09-20T18:51:31.018 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:30 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:31.018 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:30 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:51:31.018 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:30 smithi083 ceph-mon[29421]: 4.18 scrub starts 2024-09-20T18:51:31.018 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:30 smithi083 ceph-mon[29421]: 4.18 scrub ok 2024-09-20T18:51:31.019 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:51:30 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:51:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:51:31.021 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:31.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:31.022 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:31.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:30 smithi083 ceph-mon[36260]: 6.1b scrub starts 2024-09-20T18:51:31.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:30 smithi083 ceph-mon[36260]: 6.1b scrub ok 2024-09-20T18:51:31.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:30 smithi083 ceph-mon[36260]: 5.10 scrub starts 2024-09-20T18:51:31.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:30 smithi083 ceph-mon[36260]: 5.10 scrub ok 2024-09-20T18:51:31.023 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:30 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:31.023 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:30 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:51:31.023 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:30 smithi083 ceph-mon[36260]: 4.18 scrub starts 2024-09-20T18:51:31.023 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:30 smithi083 ceph-mon[36260]: 4.18 scrub ok 2024-09-20T18:51:31.122 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:31.150 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:31.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:31.151 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:31.252 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:31.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:30 smithi142 ceph-mon[32508]: 6.1b scrub starts 2024-09-20T18:51:31.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:30 smithi142 ceph-mon[32508]: 6.1b scrub ok 2024-09-20T18:51:31.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:30 smithi142 ceph-mon[32508]: 5.10 scrub starts 2024-09-20T18:51:31.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:30 smithi142 ceph-mon[32508]: 5.10 scrub ok 2024-09-20T18:51:31.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:30 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:31.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:30 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:51:31.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:30 smithi142 ceph-mon[32508]: 4.18 scrub starts 2024-09-20T18:51:31.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:30 smithi142 ceph-mon[32508]: 4.18 scrub ok 2024-09-20T18:51:31.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:31.279 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:31.279 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:31.380 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:51:31.419 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:31.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:31.420 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:51:31.520 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:51:31.521 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 34 16 63757 63741 117.143 87.1875 0.0181483 0.00853117 2024-09-20T18:51:31.561 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:31.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:31.563 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:51:31.649 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T18:51:31.649 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-20T18:51:31.649 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:31.649 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:63841 7.a e100/83 7:5fd9ccd3:::benchmark_data_smithi083_70643_object63840:head v 100'3946, mlcod=100'3943)", 2024-09-20T18:51:31.649 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:31.587483+0000", 2024-09-20T18:51:31.649 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0047483969999999997, 2024-09-20T18:51:31.649 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0047806999999999997, 2024-09-20T18:51:31.649 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:31.650 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:31.650 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:51:31.650 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:31.650 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:31.650 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:31.650 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:31.587483+0000", 2024-09-20T18:51:31.650 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:31.650 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:31.650 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:31.650 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:31.650 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:31.587483+0000", 2024-09-20T18:51:31.650 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:31.651 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:31.651 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:31.651 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:31.651 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:31.587484+0000", 2024-09-20T18:51:31.651 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2399999999999996e-07 2024-09-20T18:51:31.651 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:31.651 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:31.651 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:31.651 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:31.587516+0000", 2024-09-20T18:51:31.651 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2719000000000003e-05 2024-09-20T18:51:31.651 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:31.651 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:31.651 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:31.652 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:31.587526+0000", 2024-09-20T18:51:31.652 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.679e-06 2024-09-20T18:51:31.652 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:31.652 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:31.652 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:31.652 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:31.587533+0000", 2024-09-20T18:51:31.652 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8360000000000003e-06 2024-09-20T18:51:31.652 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:31.652 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:31.652 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:31.652 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:31.587925+0000", 2024-09-20T18:51:31.652 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00039246499999999999 2024-09-20T18:51:31.652 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:31.653 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:31.653 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:31.653 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:31.587943+0000", 2024-09-20T18:51:31.653 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7258999999999999e-05 2024-09-20T18:51:31.653 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:31.653 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:31.653 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:31.653 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:31.653 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:31.653 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:63842 7.3 e100/83 7:cf54dcf0:::benchmark_data_smithi083_70643_object63841:head v 100'3900, mlcod=100'3898)", 2024-09-20T18:51:31.653 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:31.587555+0000", 2024-09-20T18:51:31.653 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0046765169999999998, 2024-09-20T18:51:31.653 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0047726080000000002, 2024-09-20T18:51:31.653 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:31.654 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:31.654 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:51:31.654 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:31.654 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:31.654 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:31.654 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:31.587555+0000", 2024-09-20T18:51:31.654 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:31.654 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:31.654 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:31.654 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:31.654 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:31.587555+0000", 2024-09-20T18:51:31.655 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:31.655 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:31.655 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:31.655 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:31.655 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:31.587555+0000", 2024-09-20T18:51:31.655 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6199999999999999e-07 2024-09-20T18:51:31.655 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:31.655 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:31.655 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:31.655 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:31.587576+0000", 2024-09-20T18:51:31.655 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1109000000000001e-05 2024-09-20T18:51:31.655 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:31.655 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:31.655 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:31.656 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:31.587583+0000", 2024-09-20T18:51:31.656 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6200000000000001e-06 2024-09-20T18:51:31.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:31.656 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:31.656 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:31.656 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:31.587588+0000", 2024-09-20T18:51:31.656 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0989999999999998e-06 2024-09-20T18:51:31.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:31.656 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:31.656 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:31.656 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:31.587705+0000", 2024-09-20T18:51:31.656 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011751099999999999 2024-09-20T18:51:31.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:31.656 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:31.656 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:31.656 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:31.587726+0000", 2024-09-20T18:51:31.657 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0610000000000001e-05 2024-09-20T18:51:31.657 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:31.657 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:31.657 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:31.657 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:31.657 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:31.657 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:63843 7.3 e100/83)", 2024-09-20T18:51:31.657 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:31.590500+0000", 2024-09-20T18:51:31.657 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.001730917, 2024-09-20T18:51:31.657 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0018785659999999999, 2024-09-20T18:51:31.657 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:31.657 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:31.657 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "queued for pg", 2024-09-20T18:51:31.657 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:31.657 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:31.658 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:31.658 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:31.590500+0000", 2024-09-20T18:51:31.658 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:31.658 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:31.658 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:31.658 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:31.658 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:31.590500+0000", 2024-09-20T18:51:31.658 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:31.658 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:31.658 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:31.658 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:31.658 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:31.590501+0000", 2024-09-20T18:51:31.658 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9400000000000005e-07 2024-09-20T18:51:31.658 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:31.659 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:31.659 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:31.659 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:31.590522+0000", 2024-09-20T18:51:31.659 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0948999999999999e-05 2024-09-20T18:51:31.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:31.659 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:31.659 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:31.659 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:31.590531+0000", 2024-09-20T18:51:31.659 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4150000000000005e-06 2024-09-20T18:51:31.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:31.659 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:31.659 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:31.659 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:31.590537+0000", 2024-09-20T18:51:31.660 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.964e-06 2024-09-20T18:51:31.660 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:31.660 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:31.660 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:31.660 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:31.660 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T18:51:31.660 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 3 2024-09-20T18:51:31.660 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T18:51:31.663 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:31.706 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:31.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:31.707 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:31.808 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:31.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:31.854 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:31.855 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:31.955 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:51:31.996 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:31.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:31.997 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:51:32.098 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:32.126 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:32.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:32.127 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:32.177 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-20T18:51:32.228 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:32.263 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:32.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:32.264 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:32.364 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:51:32.403 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:32.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:32.405 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:51:32.431 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:32 smithi083 ceph-mon[29421]: pgmap v67: 148 pgs: 148 active+clean; 3.6 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.4 KiB/s rd, 106 MiB/s wr, 1.69k op/s; 15 B/s, 0 objects/s recovering 2024-09-20T18:51:32.431 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:32 smithi083 ceph-mon[29421]: 3.11 scrub starts 2024-09-20T18:51:32.432 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:32 smithi083 ceph-mon[29421]: 3.11 scrub ok 2024-09-20T18:51:32.432 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:32 smithi083 ceph-mon[29421]: 4.6 scrub starts 2024-09-20T18:51:32.432 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:32 smithi083 ceph-mon[29421]: 4.6 scrub ok 2024-09-20T18:51:32.432 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:32 smithi083 ceph-mon[29421]: 4.1a scrub starts 2024-09-20T18:51:32.432 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:32 smithi083 ceph-mon[29421]: 4.1a scrub ok 2024-09-20T18:51:32.432 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:32 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1056371709' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:51:32.432 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:32 smithi083 ceph-mon[36260]: pgmap v67: 148 pgs: 148 active+clean; 3.6 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.4 KiB/s rd, 106 MiB/s wr, 1.69k op/s; 15 B/s, 0 objects/s recovering 2024-09-20T18:51:32.432 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:32 smithi083 ceph-mon[36260]: 3.11 scrub starts 2024-09-20T18:51:32.432 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:32 smithi083 ceph-mon[36260]: 3.11 scrub ok 2024-09-20T18:51:32.432 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:32 smithi083 ceph-mon[36260]: 4.6 scrub starts 2024-09-20T18:51:32.433 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:32 smithi083 ceph-mon[36260]: 4.6 scrub ok 2024-09-20T18:51:32.433 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:32 smithi083 ceph-mon[36260]: 4.1a scrub starts 2024-09-20T18:51:32.433 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:32 smithi083 ceph-mon[36260]: 4.1a scrub ok 2024-09-20T18:51:32.433 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:32 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1056371709' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:51:32.505 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:32.521 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 35 16 65030 65014 116.069 79.5625 0.00934386 0.00860275 2024-09-20T18:51:32.523 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:32 smithi142 ceph-mon[32508]: pgmap v67: 148 pgs: 148 active+clean; 3.6 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.4 KiB/s rd, 106 MiB/s wr, 1.69k op/s; 15 B/s, 0 objects/s recovering 2024-09-20T18:51:32.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:32 smithi142 ceph-mon[32508]: 3.11 scrub starts 2024-09-20T18:51:32.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:32 smithi142 ceph-mon[32508]: 3.11 scrub ok 2024-09-20T18:51:32.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:32 smithi142 ceph-mon[32508]: 4.6 scrub starts 2024-09-20T18:51:32.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:32 smithi142 ceph-mon[32508]: 4.6 scrub ok 2024-09-20T18:51:32.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:32 smithi142 ceph-mon[32508]: 4.1a scrub starts 2024-09-20T18:51:32.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:32 smithi142 ceph-mon[32508]: 4.1a scrub ok 2024-09-20T18:51:32.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:32 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1056371709' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:51:32.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:32.528 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:32.528 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:32.629 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:32.654 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:32.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:32.655 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:32.756 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:32.782 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:32.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:32.783 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:32.884 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:32.909 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:32.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:32.911 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:33.012 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:33.039 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:33.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:33.040 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:33.141 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:33.167 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:33.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:33.168 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:33.269 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:33.308 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:33.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:33.309 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:33.410 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:33.444 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:33.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:33.445 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:33.521 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 36 16 66306 66290 115.06 79.75 0.0071533 0.00868669 2024-09-20T18:51:33.523 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:33 smithi142 ceph-mon[32508]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:51:33.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:33 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1056371709' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T18:51:33.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:33 smithi142 ceph-mon[32508]: osdmap e101: 8 total, 8 up, 8 in 2024-09-20T18:51:33.546 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:33.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:33 smithi083 ceph-mon[29421]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:51:33.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:33 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1056371709' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T18:51:33.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:33 smithi083 ceph-mon[29421]: osdmap e101: 8 total, 8 up, 8 in 2024-09-20T18:51:33.554 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:33 smithi083 ceph-mon[36260]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:51:33.554 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:33 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1056371709' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T18:51:33.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:33 smithi083 ceph-mon[36260]: osdmap e101: 8 total, 8 up, 8 in 2024-09-20T18:51:33.590 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:33.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:33.591 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:33.691 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:33.722 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:33.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:33.722 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:33.823 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:33.857 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:33.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:33.858 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:33.959 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:33.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:33.996 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:33.997 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:34.097 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:34.125 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:34.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:34.127 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:34.228 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:51:34.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:34.263 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:34.264 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:51:34.364 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:34.394 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:34.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:34.395 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:34.468 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:34 smithi083 ceph-mon[29421]: pgmap v69: 148 pgs: 148 active+clean; 3.8 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 736 B/s rd, 65 MiB/s wr, 1.04k op/s 2024-09-20T18:51:34.468 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:34 smithi083 ceph-mon[36260]: pgmap v69: 148 pgs: 148 active+clean; 3.8 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 736 B/s rd, 65 MiB/s wr, 1.04k op/s 2024-09-20T18:51:34.469 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_blocked_ops 2024-09-20T18:51:34.496 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:34.521 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 37 16 67673 67657 114.258 85.4375 0.0119057 0.00874716 2024-09-20T18:51:34.522 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:34.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:34.523 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:34.523 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:34 smithi142 ceph-mon[32508]: pgmap v69: 148 pgs: 148 active+clean; 3.8 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 736 B/s rd, 65 MiB/s wr, 1.04k op/s 2024-09-20T18:51:34.624 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:51:34.671 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:34.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:34.672 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:51:34.772 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:34.833 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:34.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:34.835 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:34.935 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:51:34.972 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:34.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:34.973 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:51:35.073 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:35.099 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:35.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:35.100 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:35.200 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:35.227 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:35.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:35.228 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:35.304 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 18:51:35 smithi083 systemd[1]: Stopping Ceph alertmanager.a for 1ed5a50a-7780-11ef-baf5-efdc52797490... 2024-09-20T18:51:35.329 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:35.358 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:35.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:35.359 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:35.460 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:51:35.461 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:51:35.491 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:51:35.503 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:35.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:35.505 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:51:35.521 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 38 16 68947 68931 113.347 79.625 0.0218581 0.00881724 2024-09-20T18:51:35.605 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:35.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:35.655 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:35.656 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:35.756 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:35.785 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:35.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:35.786 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:35.887 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:35.912 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:35.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:35.913 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:36.014 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:36.039 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:36.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:36.040 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:36.141 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:36.170 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:36.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:36.171 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:36.271 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:36.299 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:36.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:36.300 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:36.401 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:36.427 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:36.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:36.428 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:36.522 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 39 16 70183 70167 112.421 77.25 0.0100501 0.0088842 2024-09-20T18:51:36.528 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:36.549 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:36.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:36.550 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:36.651 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:51:36.673 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:51:36.673 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T18:51:36.673 INFO:teuthology.orchestra.run.smithi083.stdout: "epoch": 102, 2024-09-20T18:51:36.673 INFO:teuthology.orchestra.run.smithi083.stdout: "fsid": "1ed5a50a-7780-11ef-baf5-efdc52797490", 2024-09-20T18:51:36.673 INFO:teuthology.orchestra.run.smithi083.stdout: "created": "2024-09-20T18:44:10.399800+0000", 2024-09-20T18:51:36.673 INFO:teuthology.orchestra.run.smithi083.stdout: "modified": "2024-09-20T18:51:35.571589+0000", 2024-09-20T18:51:36.673 INFO:teuthology.orchestra.run.smithi083.stdout: "last_up_change": "2024-09-20T18:49:34.836377+0000", 2024-09-20T18:51:36.673 INFO:teuthology.orchestra.run.smithi083.stdout: "last_in_change": "2024-09-20T18:49:12.342332+0000", 2024-09-20T18:51:36.674 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-20T18:51:36.674 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_num": 5801984, 2024-09-20T18:51:36.674 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_set": [ 2024-09-20T18:51:36.674 INFO:teuthology.orchestra.run.smithi083.stdout: "noscrub", 2024-09-20T18:51:36.674 INFO:teuthology.orchestra.run.smithi083.stdout: "pglog_hardlimit", 2024-09-20T18:51:36.674 INFO:teuthology.orchestra.run.smithi083.stdout: "purged_snapdirs", 2024-09-20T18:51:36.674 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_deletes", 2024-09-20T18:51:36.674 INFO:teuthology.orchestra.run.smithi083.stdout: "sortbitwise" 2024-09-20T18:51:36.674 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T18:51:36.674 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_version": 18, 2024-09-20T18:51:36.674 INFO:teuthology.orchestra.run.smithi083.stdout: "full_ratio": 0.94999998807907104, 2024-09-20T18:51:36.674 INFO:teuthology.orchestra.run.smithi083.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-20T18:51:36.674 INFO:teuthology.orchestra.run.smithi083.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-20T18:51:36.674 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_snapshot": "", 2024-09-20T18:51:36.675 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_max": 7, 2024-09-20T18:51:36.675 INFO:teuthology.orchestra.run.smithi083.stdout: "max_osd": 8, 2024-09-20T18:51:36.675 INFO:teuthology.orchestra.run.smithi083.stdout: "require_min_compat_client": "luminous", 2024-09-20T18:51:36.675 INFO:teuthology.orchestra.run.smithi083.stdout: "min_compat_client": "luminous", 2024-09-20T18:51:36.675 INFO:teuthology.orchestra.run.smithi083.stdout: "require_osd_release": "squid", 2024-09-20T18:51:36.675 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_crimson": false, 2024-09-20T18:51:36.675 INFO:teuthology.orchestra.run.smithi083.stdout: "pools": [ 2024-09-20T18:51:36.675 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.675 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 1, 2024-09-20T18:51:36.675 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": ".mgr", 2024-09-20T18:51:36.675 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-20T18:47:15.285795+0000", 2024-09-20T18:51:36.675 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-20T18:51:36.675 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool", 2024-09-20T18:51:36.675 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-20T18:51:36.675 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 3, 2024-09-20T18:51:36.676 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 2, 2024-09-20T18:51:36.676 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-20T18:51:36.676 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-20T18:51:36.676 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-20T18:51:36.676 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T18:51:36.676 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T18:51:36.676 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-20T18:51:36.676 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-20T18:51:36.676 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-20T18:51:36.676 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 1, 2024-09-20T18:51:36.676 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 1, 2024-09-20T18:51:36.676 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 1, 2024-09-20T18:51:36.676 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 1, 2024-09-20T18:51:36.676 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 1, 2024-09-20T18:51:36.677 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-20T18:51:36.677 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-20T18:51:36.677 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-20T18:51:36.677 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-20T18:51:36.677 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-20T18:51:36.677 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-20T18:51:36.677 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-20T18:51:36.677 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.677 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "21", 2024-09-20T18:51:36.677 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-20T18:51:36.677 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T18:51:36.677 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T18:51:36.677 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-20T18:51:36.677 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-20T18:51:36.677 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 0, 2024-09-20T18:51:36.678 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 0, 2024-09-20T18:51:36.678 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-20T18:51:36.678 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-20T18:51:36.678 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-20T18:51:36.678 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-20T18:51:36.678 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-20T18:51:36.678 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-20T18:51:36.678 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-20T18:51:36.678 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-20T18:51:36.678 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-20T18:51:36.678 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-20T18:51:36.678 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-20T18:51:36.678 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T18:51:36.678 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T18:51:36.679 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T18:51:36.679 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-20T18:51:36.679 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-20T18:51:36.679 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-20T18:51:36.679 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-20T18:51:36.679 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-20T18:51:36.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.679 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-20T18:51:36.679 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-20T18:51:36.679 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-20T18:51:36.679 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-20T18:51:36.679 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-20T18:51:36.679 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T18:51:36.679 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-20T18:51:36.680 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-20T18:51:36.680 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-20T18:51:36.680 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-20T18:51:36.680 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-20T18:51:36.680 INFO:teuthology.orchestra.run.smithi083.stdout: "options": { 2024-09-20T18:51:36.680 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_max": 32, 2024-09-20T18:51:36.680 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_min": 1 2024-09-20T18:51:36.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.680 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-20T18:51:36.680 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": {} 2024-09-20T18:51:36.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.680 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-20T18:51:36.680 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-20T18:51:36.680 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 7.8899998664855957, 2024-09-20T18:51:36.680 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 7.8899998664855957, 2024-09-20T18:51:36.681 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 0.37999999523162842, 2024-09-20T18:51:36.681 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 3, 2024-09-20T18:51:36.681 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 3, 2024-09-20T18:51:36.681 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 1, 2024-09-20T18:51:36.681 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 1, 2024-09-20T18:51:36.681 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 1 2024-09-20T18:51:36.681 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:36.681 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.681 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.681 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 2, 2024-09-20T18:51:36.681 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": "datapool", 2024-09-20T18:51:36.681 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-20T18:49:43.452984+0000", 2024-09-20T18:51:36.681 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 8193, 2024-09-20T18:51:36.682 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-20T18:51:36.682 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-20T18:51:36.682 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 3, 2024-09-20T18:51:36.682 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 2, 2024-09-20T18:51:36.682 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-20T18:51:36.682 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-20T18:51:36.682 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-20T18:51:36.682 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T18:51:36.682 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T18:51:36.682 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-20T18:51:36.682 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-20T18:51:36.682 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-20T18:51:36.683 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 3, 2024-09-20T18:51:36.683 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 3, 2024-09-20T18:51:36.683 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 3, 2024-09-20T18:51:36.683 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 3, 2024-09-20T18:51:36.683 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 3, 2024-09-20T18:51:36.683 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-20T18:51:36.683 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-20T18:51:36.683 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-20T18:51:36.683 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-20T18:51:36.683 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-20T18:51:36.683 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-20T18:51:36.683 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-20T18:51:36.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.683 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "61", 2024-09-20T18:51:36.684 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-20T18:51:36.684 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T18:51:36.684 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T18:51:36.684 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-20T18:51:36.684 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-20T18:51:36.684 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 2, 2024-09-20T18:51:36.684 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 61, 2024-09-20T18:51:36.684 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-20T18:51:36.684 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-20T18:51:36.684 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-20T18:51:36.684 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-20T18:51:36.684 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-20T18:51:36.684 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-20T18:51:36.684 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-20T18:51:36.684 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-20T18:51:36.685 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-20T18:51:36.685 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-20T18:51:36.685 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-20T18:51:36.685 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T18:51:36.685 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T18:51:36.685 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T18:51:36.685 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-20T18:51:36.685 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-20T18:51:36.685 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-20T18:51:36.685 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-20T18:51:36.685 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-20T18:51:36.685 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.685 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-20T18:51:36.685 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-20T18:51:36.685 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-20T18:51:36.685 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-20T18:51:36.686 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-20T18:51:36.686 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T18:51:36.686 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-20T18:51:36.686 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-20T18:51:36.686 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-20T18:51:36.686 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-20T18:51:36.686 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-20T18:51:36.686 INFO:teuthology.orchestra.run.smithi083.stdout: "options": {}, 2024-09-20T18:51:36.686 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-20T18:51:36.686 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd": {} 2024-09-20T18:51:36.686 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.686 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-20T18:51:36.687 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-20T18:51:36.687 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 2.6500000953674316, 2024-09-20T18:51:36.687 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 2.6500000953674316, 2024-09-20T18:51:36.687 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 0.87999999523162842, 2024-09-20T18:51:36.687 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 2.3299999237060547, 2024-09-20T18:51:36.687 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 2.3299999237060547, 2024-09-20T18:51:36.687 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 1, 2024-09-20T18:51:36.687 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 1, 2024-09-20T18:51:36.687 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 1 2024-09-20T18:51:36.687 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:36.687 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.687 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.687 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 3, 2024-09-20T18:51:36.687 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": ".rgw.root", 2024-09-20T18:51:36.688 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-20T18:49:44.217011+0000", 2024-09-20T18:51:36.688 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-20T18:51:36.688 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool", 2024-09-20T18:51:36.688 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-20T18:51:36.688 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 3, 2024-09-20T18:51:36.688 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 2, 2024-09-20T18:51:36.688 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-20T18:51:36.688 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-20T18:51:36.688 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-20T18:51:36.688 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T18:51:36.688 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T18:51:36.688 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-20T18:51:36.688 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-20T18:51:36.689 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-20T18:51:36.689 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 32, 2024-09-20T18:51:36.689 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 32, 2024-09-20T18:51:36.689 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 32, 2024-09-20T18:51:36.689 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 32, 2024-09-20T18:51:36.689 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 32, 2024-09-20T18:51:36.689 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-20T18:51:36.689 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-20T18:51:36.689 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-20T18:51:36.689 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-20T18:51:36.690 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-20T18:51:36.690 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-20T18:51:36.690 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-20T18:51:36.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.690 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "60", 2024-09-20T18:51:36.690 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-20T18:51:36.690 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T18:51:36.690 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T18:51:36.690 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-20T18:51:36.691 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-20T18:51:36.691 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 0, 2024-09-20T18:51:36.691 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 0, 2024-09-20T18:51:36.691 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-20T18:51:36.691 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-20T18:51:36.691 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-20T18:51:36.691 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-20T18:51:36.691 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-20T18:51:36.691 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-20T18:51:36.691 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-20T18:51:36.691 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-20T18:51:36.691 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-20T18:51:36.691 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-20T18:51:36.691 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-20T18:51:36.692 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T18:51:36.692 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T18:51:36.692 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T18:51:36.692 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-20T18:51:36.692 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-20T18:51:36.692 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-20T18:51:36.692 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-20T18:51:36.692 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-20T18:51:36.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.692 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-20T18:51:36.692 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-20T18:51:36.692 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-20T18:51:36.692 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-20T18:51:36.692 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-20T18:51:36.693 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T18:51:36.693 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-20T18:51:36.693 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-20T18:51:36.693 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-20T18:51:36.693 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-20T18:51:36.693 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-20T18:51:36.693 INFO:teuthology.orchestra.run.smithi083.stdout: "options": {}, 2024-09-20T18:51:36.693 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-20T18:51:36.693 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw": {} 2024-09-20T18:51:36.693 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.693 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-20T18:51:36.693 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-20T18:51:36.693 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 1.5, 2024-09-20T18:51:36.694 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 1.5, 2024-09-20T18:51:36.694 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 1, 2024-09-20T18:51:36.694 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 1.5, 2024-09-20T18:51:36.694 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 1.5, 2024-09-20T18:51:36.694 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 1, 2024-09-20T18:51:36.694 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 1, 2024-09-20T18:51:36.694 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 1 2024-09-20T18:51:36.694 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:36.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.694 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.694 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 4, 2024-09-20T18:51:36.694 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": "default.rgw.log", 2024-09-20T18:51:36.694 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-20T18:49:46.007275+0000", 2024-09-20T18:51:36.694 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-20T18:51:36.695 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool", 2024-09-20T18:51:36.695 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-20T18:51:36.695 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 3, 2024-09-20T18:51:36.695 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 2, 2024-09-20T18:51:36.695 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-20T18:51:36.695 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-20T18:51:36.695 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-20T18:51:36.695 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T18:51:36.695 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T18:51:36.695 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-20T18:51:36.695 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-20T18:51:36.695 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-20T18:51:36.695 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 32, 2024-09-20T18:51:36.696 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 32, 2024-09-20T18:51:36.696 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 32, 2024-09-20T18:51:36.696 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 32, 2024-09-20T18:51:36.696 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 32, 2024-09-20T18:51:36.696 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-20T18:51:36.696 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-20T18:51:36.696 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-20T18:51:36.696 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-20T18:51:36.696 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-20T18:51:36.696 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-20T18:51:36.696 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-20T18:51:36.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.696 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "62", 2024-09-20T18:51:36.697 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-20T18:51:36.697 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T18:51:36.697 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T18:51:36.697 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-20T18:51:36.697 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-20T18:51:36.697 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 0, 2024-09-20T18:51:36.697 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 0, 2024-09-20T18:51:36.697 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-20T18:51:36.697 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-20T18:51:36.697 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-20T18:51:36.697 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-20T18:51:36.697 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-20T18:51:36.697 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-20T18:51:36.697 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-20T18:51:36.698 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-20T18:51:36.698 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-20T18:51:36.698 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-20T18:51:36.698 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-20T18:51:36.698 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T18:51:36.698 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T18:51:36.698 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T18:51:36.698 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-20T18:51:36.698 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-20T18:51:36.698 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-20T18:51:36.698 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-20T18:51:36.698 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-20T18:51:36.698 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.698 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-20T18:51:36.699 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-20T18:51:36.699 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-20T18:51:36.699 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-20T18:51:36.699 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-20T18:51:36.699 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T18:51:36.699 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-20T18:51:36.699 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-20T18:51:36.699 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-20T18:51:36.699 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-20T18:51:36.699 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-20T18:51:36.699 INFO:teuthology.orchestra.run.smithi083.stdout: "options": {}, 2024-09-20T18:51:36.699 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-20T18:51:36.699 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw": {} 2024-09-20T18:51:36.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.700 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-20T18:51:36.700 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-20T18:51:36.700 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 2.25, 2024-09-20T18:51:36.700 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 2.25, 2024-09-20T18:51:36.700 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 1, 2024-09-20T18:51:36.700 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 2.25, 2024-09-20T18:51:36.700 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 2.25, 2024-09-20T18:51:36.700 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 1, 2024-09-20T18:51:36.700 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 1, 2024-09-20T18:51:36.700 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 1 2024-09-20T18:51:36.700 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:36.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.701 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.701 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 5, 2024-09-20T18:51:36.701 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": "default.rgw.control", 2024-09-20T18:51:36.701 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-20T18:49:47.961258+0000", 2024-09-20T18:51:36.701 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-20T18:51:36.701 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool", 2024-09-20T18:51:36.701 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-20T18:51:36.701 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 3, 2024-09-20T18:51:36.701 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 2, 2024-09-20T18:51:36.701 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-20T18:51:36.701 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-20T18:51:36.701 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-20T18:51:36.701 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T18:51:36.702 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T18:51:36.702 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-20T18:51:36.702 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-20T18:51:36.702 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-20T18:51:36.702 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 32, 2024-09-20T18:51:36.702 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 32, 2024-09-20T18:51:36.702 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 32, 2024-09-20T18:51:36.702 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 32, 2024-09-20T18:51:36.702 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 32, 2024-09-20T18:51:36.702 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-20T18:51:36.702 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-20T18:51:36.702 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-20T18:51:36.702 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-20T18:51:36.702 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-20T18:51:36.703 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-20T18:51:36.703 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-20T18:51:36.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.703 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "64", 2024-09-20T18:51:36.703 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-20T18:51:36.703 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T18:51:36.703 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T18:51:36.703 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-20T18:51:36.703 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-20T18:51:36.703 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 0, 2024-09-20T18:51:36.703 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 0, 2024-09-20T18:51:36.703 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-20T18:51:36.703 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-20T18:51:36.704 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-20T18:51:36.704 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-20T18:51:36.704 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-20T18:51:36.704 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-20T18:51:36.704 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-20T18:51:36.704 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-20T18:51:36.704 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-20T18:51:36.704 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-20T18:51:36.704 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-20T18:51:36.704 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T18:51:36.704 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T18:51:36.704 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T18:51:36.704 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-20T18:51:36.704 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-20T18:51:36.705 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-20T18:51:36.705 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-20T18:51:36.705 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-20T18:51:36.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.705 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-20T18:51:36.705 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-20T18:51:36.705 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-20T18:51:36.705 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-20T18:51:36.705 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-20T18:51:36.705 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T18:51:36.705 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-20T18:51:36.705 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-20T18:51:36.705 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-20T18:51:36.706 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-20T18:51:36.706 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-20T18:51:36.706 INFO:teuthology.orchestra.run.smithi083.stdout: "options": {}, 2024-09-20T18:51:36.706 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-20T18:51:36.706 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw": {} 2024-09-20T18:51:36.706 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.706 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-20T18:51:36.706 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-20T18:51:36.706 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 1.25, 2024-09-20T18:51:36.706 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 1.25, 2024-09-20T18:51:36.706 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 1, 2024-09-20T18:51:36.706 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 1.25, 2024-09-20T18:51:36.706 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 1.25, 2024-09-20T18:51:36.706 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 1, 2024-09-20T18:51:36.707 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 1, 2024-09-20T18:51:36.707 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 1 2024-09-20T18:51:36.707 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:36.707 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.707 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.707 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 6, 2024-09-20T18:51:36.707 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": "default.rgw.meta", 2024-09-20T18:51:36.707 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-20T18:49:50.011276+0000", 2024-09-20T18:51:36.707 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-20T18:51:36.707 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool", 2024-09-20T18:51:36.707 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-20T18:51:36.707 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 3, 2024-09-20T18:51:36.707 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 2, 2024-09-20T18:51:36.707 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-20T18:51:36.708 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-20T18:51:36.708 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-20T18:51:36.708 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T18:51:36.708 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T18:51:36.708 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-20T18:51:36.708 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-20T18:51:36.708 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-20T18:51:36.708 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 32, 2024-09-20T18:51:36.708 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 32, 2024-09-20T18:51:36.708 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 32, 2024-09-20T18:51:36.708 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 32, 2024-09-20T18:51:36.708 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 32, 2024-09-20T18:51:36.708 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-20T18:51:36.708 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-20T18:51:36.709 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-20T18:51:36.709 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-20T18:51:36.709 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-20T18:51:36.709 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-20T18:51:36.709 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-20T18:51:36.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.709 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "66", 2024-09-20T18:51:36.709 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-20T18:51:36.709 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T18:51:36.709 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T18:51:36.709 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-20T18:51:36.709 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-20T18:51:36.709 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 0, 2024-09-20T18:51:36.710 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 0, 2024-09-20T18:51:36.710 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-20T18:51:36.710 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-20T18:51:36.710 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-20T18:51:36.710 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-20T18:51:36.710 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-20T18:51:36.710 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-20T18:51:36.710 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-20T18:51:36.710 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-20T18:51:36.710 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-20T18:51:36.710 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-20T18:51:36.710 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-20T18:51:36.710 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T18:51:36.710 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T18:51:36.711 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T18:51:36.711 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-20T18:51:36.711 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-20T18:51:36.711 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-20T18:51:36.711 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-20T18:51:36.711 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-20T18:51:36.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.711 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-20T18:51:36.711 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-20T18:51:36.711 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-20T18:51:36.711 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-20T18:51:36.711 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-20T18:51:36.711 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T18:51:36.712 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-20T18:51:36.712 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-20T18:51:36.712 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-20T18:51:36.712 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-20T18:51:36.712 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-20T18:51:36.712 INFO:teuthology.orchestra.run.smithi083.stdout: "options": { 2024-09-20T18:51:36.712 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_bias": 4 2024-09-20T18:51:36.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.712 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-20T18:51:36.712 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw": {} 2024-09-20T18:51:36.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.712 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-20T18:51:36.712 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-20T18:51:36.712 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 1.75, 2024-09-20T18:51:36.713 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 1.75, 2024-09-20T18:51:36.713 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 1, 2024-09-20T18:51:36.713 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 1.75, 2024-09-20T18:51:36.713 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 1.75, 2024-09-20T18:51:36.713 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 1, 2024-09-20T18:51:36.713 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 1, 2024-09-20T18:51:36.713 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 1 2024-09-20T18:51:36.713 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:36.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.713 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.713 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 7, 2024-09-20T18:51:36.713 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": "unique_pool_0", 2024-09-20T18:51:36.713 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-20T18:50:48.697483+0000", 2024-09-20T18:51:36.713 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-20T18:51:36.714 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool", 2024-09-20T18:51:36.714 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-20T18:51:36.714 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 3, 2024-09-20T18:51:36.714 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 2, 2024-09-20T18:51:36.714 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-20T18:51:36.714 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-20T18:51:36.714 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-20T18:51:36.714 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T18:51:36.714 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T18:51:36.714 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-20T18:51:36.714 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-20T18:51:36.714 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-20T18:51:36.714 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 16, 2024-09-20T18:51:36.715 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 16, 2024-09-20T18:51:36.715 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 16, 2024-09-20T18:51:36.715 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 16, 2024-09-20T18:51:36.715 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 16, 2024-09-20T18:51:36.715 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-20T18:51:36.715 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-20T18:51:36.715 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-20T18:51:36.715 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-20T18:51:36.715 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-20T18:51:36.715 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-20T18:51:36.715 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-20T18:51:36.715 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.715 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "87", 2024-09-20T18:51:36.715 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-20T18:51:36.716 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T18:51:36.716 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T18:51:36.716 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-20T18:51:36.716 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-20T18:51:36.716 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 0, 2024-09-20T18:51:36.716 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 0, 2024-09-20T18:51:36.716 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-20T18:51:36.716 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-20T18:51:36.716 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-20T18:51:36.716 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-20T18:51:36.716 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-20T18:51:36.716 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-20T18:51:36.716 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-20T18:51:36.716 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-20T18:51:36.716 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-20T18:51:36.717 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-20T18:51:36.717 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-20T18:51:36.717 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T18:51:36.717 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T18:51:36.717 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T18:51:36.717 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-20T18:51:36.717 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-20T18:51:36.717 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-20T18:51:36.717 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-20T18:51:36.717 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-20T18:51:36.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.717 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-20T18:51:36.717 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-20T18:51:36.718 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-20T18:51:36.718 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-20T18:51:36.718 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-20T18:51:36.718 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T18:51:36.718 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-20T18:51:36.718 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-20T18:51:36.718 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-20T18:51:36.718 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-20T18:51:36.718 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-20T18:51:36.718 INFO:teuthology.orchestra.run.smithi083.stdout: "options": {}, 2024-09-20T18:51:36.718 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-20T18:51:36.718 INFO:teuthology.orchestra.run.smithi083.stdout: "rados": {} 2024-09-20T18:51:36.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.719 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-20T18:51:36.719 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-20T18:51:36.719 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 2, 2024-09-20T18:51:36.719 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 2, 2024-09-20T18:51:36.719 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 1, 2024-09-20T18:51:36.719 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 2, 2024-09-20T18:51:36.719 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 2, 2024-09-20T18:51:36.719 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 1, 2024-09-20T18:51:36.719 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 1, 2024-09-20T18:51:36.719 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 1 2024-09-20T18:51:36.719 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:36.719 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:36.719 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T18:51:36.720 INFO:teuthology.orchestra.run.smithi083.stdout: "osds": [ 2024-09-20T18:51:36.720 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.720 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 0, 2024-09-20T18:51:36.720 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "c714e1ec-f0c3-4054-9792-61b84e53eaa8", 2024-09-20T18:51:36.720 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T18:51:36.720 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T18:51:36.720 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T18:51:36.720 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T18:51:36.720 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T18:51:36.720 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T18:51:36.720 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 8, 2024-09-20T18:51:36.720 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 83, 2024-09-20T18:51:36.720 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T18:51:36.720 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T18:51:36.720 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T18:51:36.721 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:36.721 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.721 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:36.721 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6801", 2024-09-20T18:51:36.721 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2574084049 2024-09-20T18:51:36.721 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:36.721 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:36.721 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.721 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T18:51:36.721 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:36.721 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.721 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:36.721 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6802", 2024-09-20T18:51:36.721 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2574084049 2024-09-20T18:51:36.721 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:36.722 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:36.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.722 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T18:51:36.722 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:36.722 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.722 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:36.722 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6804", 2024-09-20T18:51:36.722 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2574084049 2024-09-20T18:51:36.722 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:36.722 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:36.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.722 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T18:51:36.722 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:36.723 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.723 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:36.723 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6803", 2024-09-20T18:51:36.723 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2574084049 2024-09-20T18:51:36.723 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:36.723 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:36.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.723 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.83:6801/2574084049", 2024-09-20T18:51:36.723 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.83:6802/2574084049", 2024-09-20T18:51:36.723 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.83:6804/2574084049", 2024-09-20T18:51:36.723 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.83:6803/2574084049", 2024-09-20T18:51:36.723 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T18:51:36.723 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T18:51:36.723 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T18:51:36.723 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:36.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.724 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.724 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 1, 2024-09-20T18:51:36.724 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "9dc0f2a8-bafa-4a42-88f5-0301dd58f78d", 2024-09-20T18:51:36.724 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T18:51:36.724 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T18:51:36.724 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T18:51:36.724 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T18:51:36.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:36.725 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:36.726 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:51:36.726 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T18:51:36.726 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T18:51:36.726 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 13, 2024-09-20T18:51:36.726 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 83, 2024-09-20T18:51:36.726 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T18:51:36.726 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T18:51:36.727 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T18:51:36.727 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:36.727 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.727 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:36.727 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6805", 2024-09-20T18:51:36.727 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 4213285798 2024-09-20T18:51:36.727 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:36.727 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:36.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.727 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T18:51:36.727 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:36.727 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.728 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:36.728 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6806", 2024-09-20T18:51:36.728 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 4213285798 2024-09-20T18:51:36.728 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:36.728 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:36.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.728 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T18:51:36.728 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:36.728 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.728 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:36.728 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6808", 2024-09-20T18:51:36.728 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 4213285798 2024-09-20T18:51:36.728 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:36.728 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:36.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.729 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T18:51:36.729 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:36.729 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.729 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:36.729 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6807", 2024-09-20T18:51:36.729 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 4213285798 2024-09-20T18:51:36.729 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:36.729 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:36.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.729 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.83:6805/4213285798", 2024-09-20T18:51:36.729 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.83:6806/4213285798", 2024-09-20T18:51:36.730 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.83:6808/4213285798", 2024-09-20T18:51:36.730 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.83:6807/4213285798", 2024-09-20T18:51:36.730 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T18:51:36.730 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T18:51:36.730 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T18:51:36.730 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:36.730 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.730 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.730 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 2, 2024-09-20T18:51:36.730 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "448ab692-ec41-44dc-b2d5-dbefd255b3fd", 2024-09-20T18:51:36.730 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T18:51:36.730 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T18:51:36.731 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T18:51:36.731 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T18:51:36.731 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T18:51:36.731 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T18:51:36.731 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 18, 2024-09-20T18:51:36.731 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 83, 2024-09-20T18:51:36.731 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T18:51:36.731 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T18:51:36.731 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T18:51:36.731 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:36.731 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.731 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:36.731 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6809", 2024-09-20T18:51:36.732 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 805947571 2024-09-20T18:51:36.732 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:36.732 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:36.732 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.732 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T18:51:36.732 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:36.732 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.732 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:36.732 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6810", 2024-09-20T18:51:36.732 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 805947571 2024-09-20T18:51:36.732 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:36.732 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:36.732 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.733 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T18:51:36.733 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:36.733 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.733 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:36.733 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6812", 2024-09-20T18:51:36.733 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 805947571 2024-09-20T18:51:36.733 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:36.733 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:36.733 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.733 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T18:51:36.733 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:36.733 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.733 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:36.733 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6811", 2024-09-20T18:51:36.734 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 805947571 2024-09-20T18:51:36.734 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:36.734 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:36.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.734 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.83:6809/805947571", 2024-09-20T18:51:36.734 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.83:6810/805947571", 2024-09-20T18:51:36.734 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.83:6812/805947571", 2024-09-20T18:51:36.734 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.83:6811/805947571", 2024-09-20T18:51:36.734 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T18:51:36.734 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T18:51:36.734 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T18:51:36.734 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:36.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.735 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.735 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 3, 2024-09-20T18:51:36.735 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "7cda5e50-e22e-4d4a-823b-53c8f5061193", 2024-09-20T18:51:36.735 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T18:51:36.735 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T18:51:36.735 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T18:51:36.735 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T18:51:36.735 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T18:51:36.735 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T18:51:36.735 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 25, 2024-09-20T18:51:36.735 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 83, 2024-09-20T18:51:36.735 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T18:51:36.735 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T18:51:36.735 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T18:51:36.736 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:36.736 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.736 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:36.736 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6813", 2024-09-20T18:51:36.736 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 987237318 2024-09-20T18:51:36.736 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:36.736 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:36.736 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.736 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T18:51:36.736 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:36.736 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.736 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:36.736 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6814", 2024-09-20T18:51:36.736 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 987237318 2024-09-20T18:51:36.737 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:36.737 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:36.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.737 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T18:51:36.737 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:36.737 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.737 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:36.737 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6816", 2024-09-20T18:51:36.737 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 987237318 2024-09-20T18:51:36.737 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:36.737 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:36.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.737 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T18:51:36.738 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:36.738 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.738 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:36.738 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6815", 2024-09-20T18:51:36.738 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 987237318 2024-09-20T18:51:36.738 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:36.738 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:36.738 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.738 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.83:6813/987237318", 2024-09-20T18:51:36.738 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.83:6814/987237318", 2024-09-20T18:51:36.738 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.83:6816/987237318", 2024-09-20T18:51:36.738 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.83:6815/987237318", 2024-09-20T18:51:36.738 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T18:51:36.739 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T18:51:36.739 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T18:51:36.739 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:36.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.739 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.739 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 4, 2024-09-20T18:51:36.739 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "3d62654d-d31d-4310-9934-49e694dd17d9", 2024-09-20T18:51:36.739 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T18:51:36.739 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T18:51:36.739 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T18:51:36.739 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T18:51:36.739 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T18:51:36.739 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T18:51:36.739 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 31, 2024-09-20T18:51:36.740 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 97, 2024-09-20T18:51:36.740 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T18:51:36.740 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T18:51:36.740 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T18:51:36.740 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:36.740 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.740 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:36.740 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6800", 2024-09-20T18:51:36.740 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2096684065 2024-09-20T18:51:36.740 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:36.740 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:36.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.740 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T18:51:36.740 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:36.741 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.741 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:36.741 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6801", 2024-09-20T18:51:36.741 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2096684065 2024-09-20T18:51:36.741 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:36.741 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:36.741 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.741 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T18:51:36.741 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:36.741 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.741 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:36.741 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6803", 2024-09-20T18:51:36.741 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2096684065 2024-09-20T18:51:36.741 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:36.741 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:36.742 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.742 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T18:51:36.742 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:36.742 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.742 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:36.742 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6802", 2024-09-20T18:51:36.742 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2096684065 2024-09-20T18:51:36.742 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:36.742 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:36.742 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.742 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.142:6800/2096684065", 2024-09-20T18:51:36.742 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.142:6801/2096684065", 2024-09-20T18:51:36.742 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.142:6803/2096684065", 2024-09-20T18:51:36.742 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.142:6802/2096684065", 2024-09-20T18:51:36.743 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T18:51:36.743 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T18:51:36.743 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T18:51:36.743 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:36.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.743 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.743 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 5, 2024-09-20T18:51:36.743 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "b3e4e21b-639f-4d25-8bbf-ea8922a9222a", 2024-09-20T18:51:36.743 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T18:51:36.743 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T18:51:36.743 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T18:51:36.743 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T18:51:36.743 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T18:51:36.743 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T18:51:36.743 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 38, 2024-09-20T18:51:36.744 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 83, 2024-09-20T18:51:36.744 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T18:51:36.744 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T18:51:36.744 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T18:51:36.744 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:36.744 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.744 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:36.744 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6804", 2024-09-20T18:51:36.744 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2500466067 2024-09-20T18:51:36.744 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:36.744 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:36.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.744 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T18:51:36.744 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:36.745 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.745 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:36.745 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6805", 2024-09-20T18:51:36.745 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2500466067 2024-09-20T18:51:36.745 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:36.745 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:36.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.745 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T18:51:36.745 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:36.745 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.745 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:36.745 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6807", 2024-09-20T18:51:36.745 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2500466067 2024-09-20T18:51:36.745 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:36.746 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:36.746 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.746 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T18:51:36.746 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:36.746 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.746 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:36.746 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6806", 2024-09-20T18:51:36.746 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2500466067 2024-09-20T18:51:36.746 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:36.746 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:36.746 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.746 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.142:6804/2500466067", 2024-09-20T18:51:36.746 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.142:6805/2500466067", 2024-09-20T18:51:36.746 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.142:6807/2500466067", 2024-09-20T18:51:36.747 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.142:6806/2500466067", 2024-09-20T18:51:36.747 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T18:51:36.747 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T18:51:36.747 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T18:51:36.747 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:36.747 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.747 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.747 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 6, 2024-09-20T18:51:36.747 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "c5db7107-ccb4-4ae2-9e63-f444897942e3", 2024-09-20T18:51:36.747 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T18:51:36.747 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T18:51:36.747 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T18:51:36.747 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T18:51:36.747 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T18:51:36.748 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T18:51:36.748 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 45, 2024-09-20T18:51:36.748 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 91, 2024-09-20T18:51:36.748 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T18:51:36.748 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T18:51:36.748 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T18:51:36.748 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:36.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.748 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:36.748 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6808", 2024-09-20T18:51:36.748 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1688186226 2024-09-20T18:51:36.748 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:36.748 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:36.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.749 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T18:51:36.749 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:36.749 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.749 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:36.749 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6809", 2024-09-20T18:51:36.749 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1688186226 2024-09-20T18:51:36.749 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:36.749 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:36.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.749 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T18:51:36.749 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:36.749 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.749 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:36.749 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6811", 2024-09-20T18:51:36.750 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1688186226 2024-09-20T18:51:36.750 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:36.750 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:36.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.750 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T18:51:36.750 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:36.750 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.750 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:36.750 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6810", 2024-09-20T18:51:36.750 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1688186226 2024-09-20T18:51:36.750 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:36.750 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:36.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.750 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.142:6808/1688186226", 2024-09-20T18:51:36.750 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.142:6809/1688186226", 2024-09-20T18:51:36.751 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.142:6811/1688186226", 2024-09-20T18:51:36.751 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.142:6810/1688186226", 2024-09-20T18:51:36.751 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T18:51:36.751 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T18:51:36.751 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T18:51:36.751 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:36.751 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.751 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.751 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 7, 2024-09-20T18:51:36.751 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "07d16806-e2c8-4acb-b284-d75701e61622", 2024-09-20T18:51:36.751 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T18:51:36.751 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T18:51:36.751 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T18:51:36.751 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T18:51:36.752 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T18:51:36.752 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T18:51:36.752 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 53, 2024-09-20T18:51:36.752 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 83, 2024-09-20T18:51:36.752 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T18:51:36.752 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T18:51:36.752 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T18:51:36.752 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:36.752 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.752 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:36.752 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6812", 2024-09-20T18:51:36.752 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3679770984 2024-09-20T18:51:36.753 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:36.753 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:36.753 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.753 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T18:51:36.753 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:36.753 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.753 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:36.753 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6813", 2024-09-20T18:51:36.753 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3679770984 2024-09-20T18:51:36.753 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:36.753 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:36.753 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.753 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T18:51:36.753 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:36.754 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.754 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:36.754 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6815", 2024-09-20T18:51:36.754 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3679770984 2024-09-20T18:51:36.754 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:36.754 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:36.754 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.754 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T18:51:36.754 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:51:36.754 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.754 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:51:36.754 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6814", 2024-09-20T18:51:36.754 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3679770984 2024-09-20T18:51:36.755 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:36.755 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:36.755 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.755 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.142:6812/3679770984", 2024-09-20T18:51:36.755 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.142:6813/3679770984", 2024-09-20T18:51:36.755 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.142:6815/3679770984", 2024-09-20T18:51:36.755 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.142:6814/3679770984", 2024-09-20T18:51:36.755 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T18:51:36.755 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T18:51:36.755 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T18:51:36.755 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:36.755 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:36.755 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T18:51:36.756 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_xinfo": [ 2024-09-20T18:51:36.756 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.756 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 0, 2024-09-20T18:51:36.756 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T18:51:36.756 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T18:51:36.756 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T18:51:36.756 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T18:51:36.756 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T18:51:36.756 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T18:51:28.720503+0000", 2024-09-20T18:51:36.756 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T18:51:36.756 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.756 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.756 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 1, 2024-09-20T18:51:36.756 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T18:51:36.757 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T18:51:36.757 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T18:51:36.757 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T18:51:36.757 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T18:51:36.757 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T18:51:25.600927+0000", 2024-09-20T18:51:36.757 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T18:51:36.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.757 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.757 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 2, 2024-09-20T18:51:36.757 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T18:51:36.757 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T18:51:36.757 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T18:51:36.757 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T18:51:36.758 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T18:51:36.758 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T18:50:58.991919+0000", 2024-09-20T18:51:36.758 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T18:51:36.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.758 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.758 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 3, 2024-09-20T18:51:36.758 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T18:51:36.758 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T18:51:36.758 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T18:51:36.758 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T18:51:36.758 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T18:51:36.758 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T18:50:38.304985+0000", 2024-09-20T18:51:36.758 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T18:51:36.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.759 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.759 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 4, 2024-09-20T18:51:36.759 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T18:51:36.759 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T18:51:36.759 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T18:51:36.759 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T18:51:36.759 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T18:51:36.759 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T18:50:27.991715+0000", 2024-09-20T18:51:36.759 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T18:51:36.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.759 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.759 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 5, 2024-09-20T18:51:36.759 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T18:51:36.759 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T18:51:36.760 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T18:51:36.760 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T18:51:36.760 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T18:51:36.760 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T18:51:15.329061+0000", 2024-09-20T18:51:36.760 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T18:51:36.760 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.760 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.760 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 6, 2024-09-20T18:51:36.760 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T18:51:36.760 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T18:51:36.760 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T18:51:36.760 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T18:51:36.760 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T18:51:36.760 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T18:50:20.101102+0000", 2024-09-20T18:51:36.761 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T18:51:36.761 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.761 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.761 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 7, 2024-09-20T18:51:36.761 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T18:51:36.761 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T18:51:36.761 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T18:51:36.761 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T18:51:36.761 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T18:51:36.761 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T18:50:51.478598+0000", 2024-09-20T18:51:36.761 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T18:51:36.761 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:36.761 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T18:51:36.761 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_upmap": [], 2024-09-20T18:51:36.762 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_upmap_items": [ 2024-09-20T18:51:36.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.762 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "3.a", 2024-09-20T18:51:36.762 INFO:teuthology.orchestra.run.smithi083.stdout: "mappings": [ 2024-09-20T18:51:36.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.762 INFO:teuthology.orchestra.run.smithi083.stdout: "from": 4, 2024-09-20T18:51:36.762 INFO:teuthology.orchestra.run.smithi083.stdout: "to": 3 2024-09-20T18:51:36.762 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:36.762 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:36.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.762 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "4.f", 2024-09-20T18:51:36.762 INFO:teuthology.orchestra.run.smithi083.stdout: "mappings": [ 2024-09-20T18:51:36.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.763 INFO:teuthology.orchestra.run.smithi083.stdout: "from": 1, 2024-09-20T18:51:36.763 INFO:teuthology.orchestra.run.smithi083.stdout: "to": 2 2024-09-20T18:51:36.763 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:36.763 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:36.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.763 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.763 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "4.14", 2024-09-20T18:51:36.763 INFO:teuthology.orchestra.run.smithi083.stdout: "mappings": [ 2024-09-20T18:51:36.763 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:36.763 INFO:teuthology.orchestra.run.smithi083.stdout: "from": 1, 2024-09-20T18:51:36.763 INFO:teuthology.orchestra.run.smithi083.stdout: "to": 5 2024-09-20T18:51:36.763 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:36.763 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:36.763 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:36.764 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T18:51:36.764 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_upmap_primaries": [], 2024-09-20T18:51:36.764 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_temp": [], 2024-09-20T18:51:36.764 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_temp": [], 2024-09-20T18:51:36.764 INFO:teuthology.orchestra.run.smithi083.stdout: "blocklist": { 2024-09-20T18:51:36.764 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/3474253603": "2024-09-21T18:44:32.434180+0000", 2024-09-20T18:51:36.764 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:6800/3525856829": "2024-09-21T18:44:32.434180+0000", 2024-09-20T18:51:36.764 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/220746284": "2024-09-21T18:44:32.434180+0000", 2024-09-20T18:51:36.764 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/3875445393": "2024-09-21T18:44:56.985620+0000", 2024-09-20T18:51:36.764 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/3009092807": "2024-09-21T18:44:56.985620+0000", 2024-09-20T18:51:36.764 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/4267132131": "2024-09-21T18:44:56.985620+0000", 2024-09-20T18:51:36.764 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/3684382972": "2024-09-21T18:50:14.729526+0000", 2024-09-20T18:51:36.764 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/3132933178": "2024-09-21T18:50:14.729526+0000", 2024-09-20T18:51:36.765 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:6800/1007846146": "2024-09-21T18:50:14.729526+0000", 2024-09-20T18:51:36.765 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:6800/2985596191": "2024-09-21T18:44:56.985620+0000", 2024-09-20T18:51:36.765 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/2410768464": "2024-09-21T18:50:14.729526+0000", 2024-09-20T18:51:36.765 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/1868694097": "2024-09-21T18:50:14.729526+0000", 2024-09-20T18:51:36.765 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/3777331122": "2024-09-21T18:44:32.434180+0000", 2024-09-20T18:51:36.765 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/2161728215": "2024-09-21T18:50:14.729526+0000" 2024-09-20T18:51:36.765 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.765 INFO:teuthology.orchestra.run.smithi083.stdout: "range_blocklist": {}, 2024-09-20T18:51:36.765 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profiles": { 2024-09-20T18:51:36.765 INFO:teuthology.orchestra.run.smithi083.stdout: "default": { 2024-09-20T18:51:36.765 INFO:teuthology.orchestra.run.smithi083.stdout: "crush-failure-domain": "osd", 2024-09-20T18:51:36.765 INFO:teuthology.orchestra.run.smithi083.stdout: "k": "2", 2024-09-20T18:51:36.765 INFO:teuthology.orchestra.run.smithi083.stdout: "m": "1", 2024-09-20T18:51:36.765 INFO:teuthology.orchestra.run.smithi083.stdout: "plugin": "jerasure", 2024-09-20T18:51:36.766 INFO:teuthology.orchestra.run.smithi083.stdout: "technique": "reed_sol_van" 2024-09-20T18:51:36.766 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:36.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:36.766 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps_queue": [], 2024-09-20T18:51:36.766 INFO:teuthology.orchestra.run.smithi083.stdout: "new_removed_snaps": [], 2024-09-20T18:51:36.766 INFO:teuthology.orchestra.run.smithi083.stdout: "new_purged_snaps": [], 2024-09-20T18:51:36.766 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_node_flags": {}, 2024-09-20T18:51:36.766 INFO:teuthology.orchestra.run.smithi083.stdout: "device_class_flags": {}, 2024-09-20T18:51:36.766 INFO:teuthology.orchestra.run.smithi083.stdout: "stretch_mode": { 2024-09-20T18:51:36.766 INFO:teuthology.orchestra.run.smithi083.stdout: "stretch_mode_enabled": false, 2024-09-20T18:51:36.766 INFO:teuthology.orchestra.run.smithi083.stdout: "stretch_bucket_count": 0, 2024-09-20T18:51:36.766 INFO:teuthology.orchestra.run.smithi083.stdout: "degraded_stretch_mode": 0, 2024-09-20T18:51:36.766 INFO:teuthology.orchestra.run.smithi083.stdout: "recovering_stretch_mode": 0, 2024-09-20T18:51:36.767 INFO:teuthology.orchestra.run.smithi083.stdout: "stretch_mode_bucket": 0 2024-09-20T18:51:36.767 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:36.767 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T18:51:36.773 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:51:36 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:51:36.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:36 smithi142 ceph-mon[32508]: pgmap v70: 148 pgs: 148 active+clean; 3.8 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 639 B/s rd, 57 MiB/s wr, 907 op/s 2024-09-20T18:51:36.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:36 smithi142 ceph-mon[32508]: osdmap e102: 8 total, 8 up, 8 in 2024-09-20T18:51:36.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:36 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3170935820' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:51:36.826 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:36.848 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:36 smithi083 ceph-mon[29421]: pgmap v70: 148 pgs: 148 active+clean; 3.8 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 639 B/s rd, 57 MiB/s wr, 907 op/s 2024-09-20T18:51:36.848 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:36 smithi083 ceph-mon[29421]: osdmap e102: 8 total, 8 up, 8 in 2024-09-20T18:51:36.849 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:36 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3170935820' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:51:36.849 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:36 smithi083 ceph-mon[36260]: pgmap v70: 148 pgs: 148 active+clean; 3.8 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 639 B/s rd, 57 MiB/s wr, 907 op/s 2024-09-20T18:51:36.849 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:36 smithi083 ceph-mon[36260]: osdmap e102: 8 total, 8 up, 8 in 2024-09-20T18:51:36.849 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:36 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3170935820' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:51:36.853 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:36.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:36.854 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:36.954 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:36.982 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:36.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:36.983 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:37.084 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:51:37.121 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:37.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:37.122 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:51:37.223 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:37.252 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:37.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:37.253 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:37.354 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:37.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:37.391 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:37.391 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:37.489 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T18:51:37.492 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:37.501 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:51:37.502 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:51:37.502 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:51:37.502 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:51:37.502 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:51:37.502 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:51:37.502 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:51:37.502 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:51:37.502 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:51:37.502 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:51:37.503 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:51:37.503 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:51:37.503 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:51:37.503 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:51:37.503 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:51:37.503 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:51:37.515 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:37.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:37.516 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:37.522 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:2024-09-20T18:51:37.506604+0000 min lat: 0.00137732 max lat: 0.186958 avg lat: 0.0089631 2024-09-20T18:51:37.522 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-20T18:51:37.522 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 40 16 71389 71373 111.493 75.375 0.0170616 0.0089631 2024-09-20T18:51:37.617 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:37.689 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:37.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:37.689 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:37.790 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:37.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:37 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:51:37.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:37 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2751331917' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-20T18:51:37.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:37 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:51:37.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:37 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2751331917' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-20T18:51:37.817 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:37.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:37.818 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:37.919 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:51:37.964 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:37.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:37.965 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:51:38.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:37 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:51:38.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:37 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2751331917' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-20T18:51:38.066 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:38.090 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:38.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:38.091 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:38.192 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:51:38.232 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:38.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:38.233 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:51:38.334 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:38.361 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:38.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:38.362 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:38.463 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:38.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:38.490 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:38.491 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:38.522 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 41 16 72581 72565 110.591 74.5 0.0119775 0.00903761 2024-09-20T18:51:38.591 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:38.619 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:38.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:38.620 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:38.721 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:38.749 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:38.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:38.750 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:38.850 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:38.879 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:38.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:38.880 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:38.924 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:38 smithi142 ceph-mon[32508]: pgmap v72: 148 pgs: 148 active+clean; 4.2 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.3 KiB/s rd, 109 MiB/s wr, 1.74k op/s 2024-09-20T18:51:38.924 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:38 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:51:38.925 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:38 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:51:38.981 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:39.021 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:39.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:39.022 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:39.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:38 smithi083 ceph-mon[29421]: pgmap v72: 148 pgs: 148 active+clean; 4.2 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.3 KiB/s rd, 109 MiB/s wr, 1.74k op/s 2024-09-20T18:51:39.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:38 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:51:39.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:38 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:51:39.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:38 smithi083 ceph-mon[36260]: pgmap v72: 148 pgs: 148 active+clean; 4.2 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.3 KiB/s rd, 109 MiB/s wr, 1.74k op/s 2024-09-20T18:51:39.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:38 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:51:39.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:38 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:51:39.123 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:39.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:39.157 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:39.157 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:39.258 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:39.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:39.295 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:39.296 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:39.396 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:39.435 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:39.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:39.436 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:39.523 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 42 16 73876 73860 109.885 80.9375 0.00317812 0.00909136 2024-09-20T18:51:39.537 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:39.563 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:39.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:39.564 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:39.665 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:51:39.700 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:39.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:39.701 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:51:39.802 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:39.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:39.824 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:39.824 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:39.925 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:39.956 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:39.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:39.957 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:40.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:39 smithi142 ceph-mon[32508]: osdmap e103: 8 total, 8 up, 8 in 2024-09-20T18:51:40.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:39 smithi083 ceph-mon[29421]: osdmap e103: 8 total, 8 up, 8 in 2024-09-20T18:51:40.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:39 smithi083 ceph-mon[36260]: osdmap e103: 8 total, 8 up, 8 in 2024-09-20T18:51:40.058 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:40.093 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:40.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:40.094 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:40.194 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:40.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:40.234 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:40.235 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:40.335 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:40.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:40.375 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:40.375 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:40.476 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:40.509 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:40.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:40.511 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:40.523 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 43 16 75038 75022 109.018 72.625 0.0289146 0.0091672 2024-09-20T18:51:40.611 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:40.638 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:40.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:40.639 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:40.740 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:40.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:40.779 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:40.779 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:40.880 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:40.894 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:40 smithi083 ceph-mon[29421]: pgmap v74: 148 pgs: 148 active+clean; 4.2 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 774 B/s rd, 62 MiB/s wr, 987 op/s 2024-09-20T18:51:40.894 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:40 smithi083 ceph-mon[29421]: osdmap e104: 8 total, 8 up, 8 in 2024-09-20T18:51:40.894 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:40 smithi083 ceph-mon[36260]: pgmap v74: 148 pgs: 148 active+clean; 4.2 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 774 B/s rd, 62 MiB/s wr, 987 op/s 2024-09-20T18:51:40.895 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:40 smithi083 ceph-mon[36260]: osdmap e104: 8 total, 8 up, 8 in 2024-09-20T18:51:40.895 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:51:40 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:51:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:51:40.904 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:40.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:40.905 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:41.006 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:51:41.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:40 smithi142 ceph-mon[32508]: pgmap v74: 148 pgs: 148 active+clean; 4.2 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 774 B/s rd, 62 MiB/s wr, 987 op/s 2024-09-20T18:51:41.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:40 smithi142 ceph-mon[32508]: osdmap e104: 8 total, 8 up, 8 in 2024-09-20T18:51:41.042 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:41.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:41.043 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:51:41.144 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:41.191 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:41.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:41.192 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:41.293 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:41.326 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:41.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:41.327 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:41.428 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:41.457 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:41.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:41.458 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:41.523 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 44 16 76303 76287 108.337 79.0625 0.00905156 0.00922522 2024-09-20T18:51:41.559 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:41.588 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:41.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:41.589 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:41.690 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:41.707 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:51:41.717 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:41.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:41.718 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:41.761 INFO:tasks.thrashosds.thrasher:j is {'epoch': 102, 'fsid': '1ed5a50a-7780-11ef-baf5-efdc52797490', 'created': '2024-09-20T18:44:10.399800+0000', 'modified': '2024-09-20T18:51:35.571589+0000', 'last_up_change': '2024-09-20T18:49:34.836377+0000', 'last_in_change': '2024-09-20T18:49:12.342332+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 18, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 7, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-20T18:47:15.285795+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'datapool', 'create_time': '2024-09-20T18:49:43.452984+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 61, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.6500000953674316, 'score_stable': 2.6500000953674316, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 2.3299999237060547, 'raw_score_stable': 2.3299999237060547, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': '.rgw.root', 'create_time': '2024-09-20T18:49:44.217011+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-20T18:49:46.007275+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-20T18:49:47.961258+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-20T18:49:50.011276+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'unique_pool_0', 'create_time': '2024-09-20T18:50:48.697483+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': 'c714e1ec-f0c3-4054-9792-61b84e53eaa8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 83, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6801', 'nonce': 2574084049}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6802', 'nonce': 2574084049}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6804', 'nonce': 2574084049}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6803', 'nonce': 2574084049}]}, 'public_addr': '172.21.15.83:6801/2574084049', 'cluster_addr': '172.21.15.83:6802/2574084049', 'heartbeat_back_addr': '172.21.15.83:6804/2574084049', 'heartbeat_front_addr': '172.21.15.83:6803/2574084049', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '9dc0f2a8-bafa-4a42-88f5-0301dd58f78d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 83, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6805', 'nonce': 4213285798}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6806', 'nonce': 4213285798}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6808', 'nonce': 4213285798}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6807', 'nonce': 4213285798}]}, 'public_addr': '172.21.15.83:6805/4213285798', 'cluster_addr': '172.21.15.83:6806/4213285798', 'heartbeat_back_addr': '172.21.15.83:6808/4213285798', 'heartbeat_front_addr': '172.21.15.83:6807/4213285798', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '448ab692-ec41-44dc-b2d5-dbefd255b3fd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 83, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6809', 'nonce': 805947571}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6810', 'nonce': 805947571}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6812', 'nonce': 805947571}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6811', 'nonce': 805947571}]}, 'public_addr': '172.21.15.83:6809/805947571', 'cluster_addr': '172.21.15.83:6810/805947571', 'heartbeat_back_addr': '172.21.15.83:6812/805947571', 'heartbeat_front_addr': '172.21.15.83:6811/805947571', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '7cda5e50-e22e-4d4a-823b-53c8f5061193', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 25, 'up_thru': 83, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6813', 'nonce': 987237318}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6814', 'nonce': 987237318}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6816', 'nonce': 987237318}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6815', 'nonce': 987237318}]}, 'public_addr': '172.21.15.83:6813/987237318', 'cluster_addr': '172.21.15.83:6814/987237318', 'heartbeat_back_addr': '172.21.15.83:6816/987237318', 'heartbeat_front_addr': '172.21.15.83:6815/987237318', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '3d62654d-d31d-4310-9934-49e694dd17d9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 31, 'up_thru': 97, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6800', 'nonce': 2096684065}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6801', 'nonce': 2096684065}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6803', 'nonce': 2096684065}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6802', 'nonce': 2096684065}]}, 'public_addr': '172.21.15.142:6800/2096684065', 'cluster_addr': '172.21.15.142:6801/2096684065', 'heartbeat_back_addr': '172.21.15.142:6803/2096684065', 'heartbeat_front_addr': '172.21.15.142:6802/2096684065', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'b3e4e21b-639f-4d25-8bbf-ea8922a9222a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 38, 'up_thru': 83, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6804', 'nonce': 2500466067}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6805', 'nonce': 2500466067}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6807', 'nonce': 2500466067}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6806', 'nonce': 2500466067}]}, 'public_addr': '172.21.15.142:6804/2500466067', 'cluster_addr': '172.21.15.142:6805/2500466067', 'heartbeat_back_addr': '172.21.15.142:6807/2500466067', 'heartbeat_front_addr': '172.21.15.142:6806/2500466067', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'c5db7107-ccb4-4ae2-9e63-f444897942e3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 45, 'up_thru': 91, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6808', 'nonce': 1688186226}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6809', 'nonce': 1688186226}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6811', 'nonce': 1688186226}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6810', 'nonce': 1688186226}]}, 'public_addr': '172.21.15.142:6808/1688186226', 'cluster_addr': '172.21.15.142:6809/1688186226', 'heartbeat_back_addr': '172.21.15.142:6811/1688186226', 'heartbeat_front_addr': '172.21.15.142:6810/1688186226', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '07d16806-e2c8-4acb-b284-d75701e61622', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 53, 'up_thru': 83, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6812', 'nonce': 3679770984}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6813', 'nonce': 3679770984}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6815', 'nonce': 3679770984}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6814', 'nonce': 3679770984}]}, 'public_addr': '172.21.15.142:6812/3679770984', 'cluster_addr': '172.21.15.142:6813/3679770984', 'heartbeat_back_addr': '172.21.15.142:6815/3679770984', 'heartbeat_front_addr': '172.21.15.142:6814/3679770984', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T18:51:28.720503+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T18:51:25.600927+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T18:50:58.991919+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T18:50:38.304985+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T18:50:27.991715+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T18:51:15.329061+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T18:50:20.101102+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T18:50:51.478598+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.a', 'mappings': [{'from': 4, 'to': 3}]}, {'pgid': '4.f', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.14', 'mappings': [{'from': 1, 'to': 5}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {'172.21.15.83:0/3474253603': '2024-09-21T18:44:32.434180+0000', '172.21.15.83:6800/3525856829': '2024-09-21T18:44:32.434180+0000', '172.21.15.83:0/220746284': '2024-09-21T18:44:32.434180+0000', '172.21.15.83:0/3875445393': '2024-09-21T18:44:56.985620+0000', '172.21.15.83:0/3009092807': '2024-09-21T18:44:56.985620+0000', '172.21.15.83:0/4267132131': '2024-09-21T18:44:56.985620+0000', '172.21.15.83:0/3684382972': '2024-09-21T18:50:14.729526+0000', '172.21.15.83:0/3132933178': '2024-09-21T18:50:14.729526+0000', '172.21.15.83:6800/1007846146': '2024-09-21T18:50:14.729526+0000', '172.21.15.83:6800/2985596191': '2024-09-21T18:44:56.985620+0000', '172.21.15.83:0/2410768464': '2024-09-21T18:50:14.729526+0000', '172.21.15.83:0/1868694097': '2024-09-21T18:50:14.729526+0000', '172.21.15.83:0/3777331122': '2024-09-21T18:44:32.434180+0000', '172.21.15.83:0/2161728215': '2024-09-21T18:50:14.729526+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-20T18:51:41.762 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T18:51:41.819 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:41.845 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:41.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:41.846 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:41.947 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:51:41.991 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 18:51:41 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-alertmanager-a[68238]: ts=2024-09-20T18:51:41.853Z caller=main.go:542 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-20T18:51:41.991 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:41 smithi083 ceph-mon[36260]: pgmap v76: 148 pgs: 148 active+clean; 4.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.7 KiB/s rd, 92 MiB/s wr, 1.47k op/s 2024-09-20T18:51:41.991 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:41 smithi083 ceph-mon[29421]: pgmap v76: 148 pgs: 148 active+clean; 4.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.7 KiB/s rd, 92 MiB/s wr, 1.47k op/s 2024-09-20T18:51:42.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:42.010 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:42.011 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:51:42.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:41 smithi142 ceph-mon[32508]: pgmap v76: 148 pgs: 148 active+clean; 4.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.7 KiB/s rd, 92 MiB/s wr, 1.47k op/s 2024-09-20T18:51:42.111 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:51:42.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:42.164 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:42.164 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:51:42.265 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:51:42.304 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 18:51:42 smithi083 podman[75451]: 2024-09-20 18:51:42.084549659 +0000 UTC m=+0.913484753 container died 61f225cfe95ae817aa7eed6e0e9be3324260782480cd8e437276895d0e3e2e5a (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-20T18:51:42.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:42.309 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:42.310 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:51:42.410 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:51:42.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:42.447 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:42.447 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:51:42.523 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 45 16 77448 77432 107.52 71.5625 0.0113444 0.00929516 2024-09-20T18:51:42.548 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:42.574 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:42.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:42.575 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:42.676 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:42.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:42.712 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:42.712 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:42.813 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:51:42.848 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:42.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:42.849 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:51:42.866 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T18:51:42.866 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:51:42.941 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T18:51:42.950 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:42.986 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 18:51:42 smithi083 podman[75451]: 2024-09-20 18:51:42.731867486 +0000 UTC m=+1.560802585 container cleanup 61f225cfe95ae817aa7eed6e0e9be3324260782480cd8e437276895d0e3e2e5a (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-20T18:51:42.986 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 18:51:42 smithi083 bash[75451]: ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-alertmanager-a 2024-09-20T18:51:43.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:43.033 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:43.033 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:43.093 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:51:43.134 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:43.184 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:43.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:43.185 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:43.285 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:51:43.337 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:43.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:43.339 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:51:43.439 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:43.479 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:43.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:43.480 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:43.523 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 46 16 78459 78443 106.556 63.1875 0.00612967 0.00938047 2024-09-20T18:51:43.581 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:43.608 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:43.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:43.609 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:43.662 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_historic_ops 2024-09-20T18:51:43.710 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:43.733 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:43.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:43.734 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:43.835 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:43.860 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:43.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:43.861 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:43.961 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:43.985 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:43 smithi083 ceph-mon[36260]: pgmap v77: 148 pgs: 148 active+clean; 4.6 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.4 KiB/s rd, 109 MiB/s wr, 1.75k op/s 2024-09-20T18:51:43.987 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:43 smithi083 ceph-mon[29421]: pgmap v77: 148 pgs: 148 active+clean; 4.6 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.4 KiB/s rd, 109 MiB/s wr, 1.75k op/s 2024-09-20T18:51:43.987 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 18:51:43 smithi083 podman[75516]: 2024-09-20 18:51:43.967966463 +0000 UTC m=+2.055838938 container remove 61f225cfe95ae817aa7eed6e0e9be3324260782480cd8e437276895d0e3e2e5a (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-20T18:51:44.012 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:44.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:44.013 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:44.114 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:51:44.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:44.153 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:44.154 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:51:44.254 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:44.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:43 smithi142 ceph-mon[32508]: pgmap v77: 148 pgs: 148 active+clean; 4.6 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.4 KiB/s rd, 109 MiB/s wr, 1.75k op/s 2024-09-20T18:51:44.280 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:44.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:44.281 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:44.293 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 18:51:44 smithi083 podman[75516]: 2024-09-20 18:51:44.089051357 +0000 UTC m=+2.176923838 volume remove d71a8516ecdfea61b679898626879b7e0a1cdfa4ea624376820a704eaf8806c4 2024-09-20T18:51:44.382 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:51:44.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:44.431 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:44.431 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:51:44.524 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 47 16 79411 79395 105.555 59.5 0.0101054 0.00946778 2024-09-20T18:51:44.532 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:51:44.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:44.576 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:44.576 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:51:44.677 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:44.702 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:44.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:44.703 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:44.749 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:51:44.804 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:44.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:44.858 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:44.859 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:44.959 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:45.001 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:45.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:45.002 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:45.102 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:45.125 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:45.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:45.126 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:45.227 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:45.259 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:45.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:45.261 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:45.361 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:51:45.383 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:45.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:45.384 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:51:45.485 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:51:45.522 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:45.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:45.523 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:51:45.523 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 48 16 80430 80414 104.683 63.6875 0.0137365 0.00954756 2024-09-20T18:51:45.624 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:45.648 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:45.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:45.649 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:45.750 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:45.786 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:45.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:45.787 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:45.873 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-20T18:51:45.888 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:45.908 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:45.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:45.909 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:46.010 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:46.047 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:46.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:46.048 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:46.149 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:46.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:46.190 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:46.190 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:46.242 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:45 smithi083 ceph-mon[29421]: pgmap v78: 148 pgs: 148 active+clean; 4.6 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 639 B/s rd, 47 MiB/s wr, 760 op/s 2024-09-20T18:51:46.242 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:45 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3633679352' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:51:46.242 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:45 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:51:46.242 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:45 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:46.242 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:45 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:51:46.242 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:45 smithi083 ceph-mon[36260]: pgmap v78: 148 pgs: 148 active+clean; 4.6 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 639 B/s rd, 47 MiB/s wr, 760 op/s 2024-09-20T18:51:46.242 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:45 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3633679352' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:51:46.243 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:45 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:51:46.243 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:45 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:46.243 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:45 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:51:46.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:45 smithi142 ceph-mon[32508]: pgmap v78: 148 pgs: 148 active+clean; 4.6 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 639 B/s rd, 47 MiB/s wr, 760 op/s 2024-09-20T18:51:46.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:45 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3633679352' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:51:46.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:45 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:51:46.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:45 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:46.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:45 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:51:46.291 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:46.312 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:46.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:46.313 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:46.414 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:46.421 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:51:46.421 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:51:46.421 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:51:46.421 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:51:46.421 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:51:46.421 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:51:46.422 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:51:46.422 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:51:46.422 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:51:46.422 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:51:46.422 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:51:46.422 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:51:46.445 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:51:46.445 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:51:46.452 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:46.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:46.453 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:46.476 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:51:46.476 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:51:46.523 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 49 16 81510 81494 103.924 67.5 0.0123919 0.00961726 2024-09-20T18:51:46.554 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:46.581 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:46.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:46.582 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:46.685 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:46.712 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:46.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:46.713 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:46.774 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:51:46 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:51:46.814 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:51:46.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:46.853 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:46.853 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:51:46.954 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:46.981 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:46.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:46.982 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:47.083 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:47.111 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:47.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:47.112 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:47.212 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:47.238 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:47.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:47.240 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:47.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:46 smithi142 ceph-mon[32508]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:51:47.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:46 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T18:51:47.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:46 smithi142 ceph-mon[32508]: osdmap e105: 8 total, 8 up, 8 in 2024-09-20T18:51:47.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:46 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3695863891' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:51:47.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:46 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:51:47.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:46 smithi083 ceph-mon[36260]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:51:47.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:46 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T18:51:47.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:46 smithi083 ceph-mon[36260]: osdmap e105: 8 total, 8 up, 8 in 2024-09-20T18:51:47.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:46 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3695863891' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:51:47.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:46 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:51:47.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:46 smithi083 ceph-mon[29421]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:51:47.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:46 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T18:51:47.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:46 smithi083 ceph-mon[29421]: osdmap e105: 8 total, 8 up, 8 in 2024-09-20T18:51:47.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:46 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3695863891' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:51:47.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:46 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:51:47.340 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:47.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:47.372 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:47.372 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:47.473 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:47.499 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:47.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:47.502 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:47.523 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 50 16 82769 82753 103.419 78.6875 0.00624577 0.00966459 2024-09-20T18:51:47.603 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:51:47.640 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:47.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:47.641 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:51:47.742 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:47.767 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:47.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:47.768 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:47.869 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:47.893 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:51:47.895 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:47.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:47.896 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:47.900 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:51:47.997 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:48.022 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:48.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:48.023 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:48.124 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:48.160 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:48.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:48.161 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:48.237 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:47 smithi083 ceph-mon[29421]: pgmap v80: 148 pgs: 148 active+clean; 4.9 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 90 MiB/s wr, 1.44k op/s 2024-09-20T18:51:48.237 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:47 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:51:48.237 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:47 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:51:48.238 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:47 smithi083 ceph-mon[36260]: pgmap v80: 148 pgs: 148 active+clean; 4.9 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 90 MiB/s wr, 1.44k op/s 2024-09-20T18:51:48.238 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:47 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:51:48.238 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:47 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:51:48.238 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 18:51:47 smithi083 systemd[1]: ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@alertmanager.a.service: Deactivated successfully. 2024-09-20T18:51:48.238 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 18:51:47 smithi083 systemd[1]: Stopped Ceph alertmanager.a for 1ed5a50a-7780-11ef-baf5-efdc52797490. 2024-09-20T18:51:48.239 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 18:51:47 smithi083 systemd[1]: ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@alertmanager.a.service: Consumed 1.158s CPU time. 2024-09-20T18:51:48.239 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 18:51:47 smithi083 systemd[1]: Starting Ceph alertmanager.a for 1ed5a50a-7780-11ef-baf5-efdc52797490... 2024-09-20T18:51:48.262 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:48.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:47 smithi142 ceph-mon[32508]: pgmap v80: 148 pgs: 148 active+clean; 4.9 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 90 MiB/s wr, 1.44k op/s 2024-09-20T18:51:48.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:47 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:51:48.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:47 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:51:48.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:48.292 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:48.292 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:48.392 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:48.418 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:48.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:48.420 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:48.520 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:48.524 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 51 16 83944 83928 102.829 73.4375 0.00964395 0.00971945 2024-09-20T18:51:48.549 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:48.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:48.550 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:48.650 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:48.680 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:48.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:48.681 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:48.782 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:48.807 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:48.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:48.808 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:48.909 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:48.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:48.936 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:48.936 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:49.037 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:49.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:48 smithi083 ceph-mon[29421]: osdmap e106: 8 total, 8 up, 8 in 2024-09-20T18:51:49.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:48 smithi083 ceph-mon[36260]: osdmap e106: 8 total, 8 up, 8 in 2024-09-20T18:51:49.061 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:49.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:49.063 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:49.158 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T18:51:49.158 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-20T18:51:49.158 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-20T18:51:49.158 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-20T18:51:49.159 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.159 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:23101 7.e 7:75708ce7:::benchmark_data_smithi083_70643_object23100:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-20T18:51:49.159 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:06.086510+0000", 2024-09-20T18:51:49.159 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 42.744401308999997, 2024-09-20T18:51:49.159 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.067645045000000001, 2024-09-20T18:51:49.159 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:49.159 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:49.159 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:51:49.160 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:51:49.160 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:51:49.160 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:51:49.160 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 23101 2024-09-20T18:51:49.160 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.160 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:49.160 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.161 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:49.161 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.086510+0000", 2024-09-20T18:51:49.161 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:49.161 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.161 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.162 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:49.162 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.086510+0000", 2024-09-20T18:51:49.162 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:49.162 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.162 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.162 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:49.163 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.086511+0000", 2024-09-20T18:51:49.163 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3599999999999999e-06 2024-09-20T18:51:49.163 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.163 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.163 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:49.163 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.086541+0000", 2024-09-20T18:51:49.163 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0222e-05 2024-09-20T18:51:49.164 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.164 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.164 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:49.164 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.086552+0000", 2024-09-20T18:51:49.164 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0614e-05 2024-09-20T18:51:49.164 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.164 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.164 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:49.164 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.086559+0000", 2024-09-20T18:51:49.164 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2470000000000002e-06 2024-09-20T18:51:49.165 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.165 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.165 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:49.165 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.086612+0000", 2024-09-20T18:51:49.165 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2448999999999999e-05 2024-09-20T18:51:49.165 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.165 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.165 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:49.165 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.087015+0000", 2024-09-20T18:51:49.165 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00040302699999999998 2024-09-20T18:51:49.166 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.166 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.166 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:51:49.166 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.087132+0000", 2024-09-20T18:51:49.166 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011758 2024-09-20T18:51:49.166 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.166 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.167 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:49.167 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.088691+0000", 2024-09-20T18:51:49.167 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0015584279999999999 2024-09-20T18:51:49.167 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.167 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.167 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:51:49.168 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.088709+0000", 2024-09-20T18:51:49.168 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8346999999999999e-05 2024-09-20T18:51:49.168 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.168 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.168 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:49.168 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.154060+0000", 2024-09-20T18:51:49.169 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.065350983000000001 2024-09-20T18:51:49.169 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.169 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.169 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:51:49.169 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.154115+0000", 2024-09-20T18:51:49.169 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4589000000000002e-05 2024-09-20T18:51:49.169 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.170 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.170 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:49.170 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:06.154155+0000", 2024-09-20T18:51:49.170 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0198999999999999e-05 2024-09-20T18:51:49.170 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:49.170 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:49.170 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:49.170 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.170 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.170 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:27549 7.e 7:77436293:::benchmark_data_smithi083_70643_object27548:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:51:49.170 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:08.731657+0000", 2024-09-20T18:51:49.170 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 40.099253949000001, 2024-09-20T18:51:49.170 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.113586293, 2024-09-20T18:51:49.171 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:49.171 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:49.171 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:51:49.171 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:51:49.171 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:51:49.171 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:51:49.171 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 27549 2024-09-20T18:51:49.171 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.171 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:49.171 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.171 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:49.171 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.731657+0000", 2024-09-20T18:51:49.171 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:49.172 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.172 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.172 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:49.172 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.731657+0000", 2024-09-20T18:51:49.172 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:49.172 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.172 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.172 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:49.172 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.731658+0000", 2024-09-20T18:51:49.172 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1400000000000001e-06 2024-09-20T18:51:49.172 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.172 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.172 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:49.172 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.731686+0000", 2024-09-20T18:51:49.173 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7840000000000001e-05 2024-09-20T18:51:49.173 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.173 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.173 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:49.173 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.731697+0000", 2024-09-20T18:51:49.173 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0774999999999999e-05 2024-09-20T18:51:49.173 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.173 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.173 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:49.173 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.731704+0000", 2024-09-20T18:51:49.173 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1740000000000003e-06 2024-09-20T18:51:49.173 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.173 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.174 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:49.174 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.731757+0000", 2024-09-20T18:51:49.174 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3214000000000002e-05 2024-09-20T18:51:49.174 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.174 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.174 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:49.174 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.731876+0000", 2024-09-20T18:51:49.174 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000118829 2024-09-20T18:51:49.174 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.174 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.174 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:51:49.174 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.732005+0000", 2024-09-20T18:51:49.175 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000128937 2024-09-20T18:51:49.175 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.175 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.175 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:51:49.175 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733315+0000", 2024-09-20T18:51:49.175 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001309943 2024-09-20T18:51:49.175 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.175 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.175 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:49.175 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843149+0000", 2024-09-20T18:51:49.175 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.109834472 2024-09-20T18:51:49.175 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.175 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.175 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:49.175 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843225+0000", 2024-09-20T18:51:49.176 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5304999999999998e-05 2024-09-20T18:51:49.176 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.176 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.176 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:51:49.176 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.844447+0000", 2024-09-20T18:51:49.176 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0012227239999999999 2024-09-20T18:51:49.176 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.176 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.176 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:49.176 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.845243+0000", 2024-09-20T18:51:49.176 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00079593999999999997 2024-09-20T18:51:49.176 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:49.176 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:49.176 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:49.177 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.177 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.177 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:27551 7.8 7:1bc4112c:::benchmark_data_smithi083_70643_object27550:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:51:49.177 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:08.733037+0000", 2024-09-20T18:51:49.177 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 40.097874160000003, 2024-09-20T18:51:49.177 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.110337717, 2024-09-20T18:51:49.177 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:49.177 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:49.177 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:51:49.177 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:51:49.177 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:51:49.177 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:51:49.177 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 27551 2024-09-20T18:51:49.177 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.178 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:49.178 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.178 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:49.178 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733037+0000", 2024-09-20T18:51:49.178 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:49.178 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.178 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.178 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:49.178 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733037+0000", 2024-09-20T18:51:49.178 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:49.178 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.178 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.178 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:49.179 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733038+0000", 2024-09-20T18:51:49.179 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.497e-06 2024-09-20T18:51:49.179 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.179 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.179 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:49.179 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733062+0000", 2024-09-20T18:51:49.179 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3498000000000001e-05 2024-09-20T18:51:49.179 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.179 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.179 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:49.179 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733072+0000", 2024-09-20T18:51:49.179 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0173000000000001e-05 2024-09-20T18:51:49.179 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.179 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.180 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:49.180 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733079+0000", 2024-09-20T18:51:49.180 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5370000000000003e-06 2024-09-20T18:51:49.180 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.180 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.180 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:49.180 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733187+0000", 2024-09-20T18:51:49.180 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010888 2024-09-20T18:51:49.180 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.180 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.181 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:49.181 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733330+0000", 2024-09-20T18:51:49.181 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000142206 2024-09-20T18:51:49.181 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.181 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.181 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4,7", 2024-09-20T18:51:49.181 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733549+0000", 2024-09-20T18:51:49.181 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0002198 2024-09-20T18:51:49.181 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.181 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.181 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:51:49.181 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.735541+0000", 2024-09-20T18:51:49.181 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0019915129999999999 2024-09-20T18:51:49.181 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.181 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.182 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:49.182 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.842311+0000", 2024-09-20T18:51:49.182 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.106769612 2024-09-20T18:51:49.182 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.182 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.182 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:49.182 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.842372+0000", 2024-09-20T18:51:49.182 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0968000000000003e-05 2024-09-20T18:51:49.182 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.182 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.182 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:51:49.182 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843315+0000", 2024-09-20T18:51:49.182 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00094376300000000002 2024-09-20T18:51:49.183 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.183 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.183 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:49.183 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843375+0000", 2024-09-20T18:51:49.183 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.927e-05 2024-09-20T18:51:49.183 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:49.183 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:49.183 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:49.183 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.183 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.183 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:27555 7.e 7:73709da2:::benchmark_data_smithi083_70643_object27554:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:51:49.183 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:08.733695+0000", 2024-09-20T18:51:49.183 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 40.097216037000003, 2024-09-20T18:51:49.183 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.11168107300000001, 2024-09-20T18:51:49.183 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:49.184 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:49.184 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:51:49.184 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:51:49.184 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:51:49.184 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:51:49.184 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 27555 2024-09-20T18:51:49.184 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.184 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:49.184 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.184 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:49.184 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733695+0000", 2024-09-20T18:51:49.184 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:49.184 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.184 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.185 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:49.185 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733695+0000", 2024-09-20T18:51:49.185 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:49.185 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.185 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.185 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:49.185 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733696+0000", 2024-09-20T18:51:49.185 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0200000000000001e-07 2024-09-20T18:51:49.185 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.185 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.185 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:49.185 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733718+0000", 2024-09-20T18:51:49.185 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2226999999999999e-05 2024-09-20T18:51:49.185 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.185 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.186 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:49.186 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733727+0000", 2024-09-20T18:51:49.186 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6049999999999998e-06 2024-09-20T18:51:49.186 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.186 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.186 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:49.186 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733734+0000", 2024-09-20T18:51:49.186 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0639999999999996e-06 2024-09-20T18:51:49.186 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.186 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.186 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:49.186 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733789+0000", 2024-09-20T18:51:49.186 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5087999999999998e-05 2024-09-20T18:51:49.187 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.187 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.187 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:49.187 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733920+0000", 2024-09-20T18:51:49.187 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013146800000000001 2024-09-20T18:51:49.187 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.187 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.187 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:51:49.187 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733995+0000", 2024-09-20T18:51:49.187 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4664000000000002e-05 2024-09-20T18:51:49.187 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.187 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.187 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:51:49.187 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.735538+0000", 2024-09-20T18:51:49.188 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001542671 2024-09-20T18:51:49.188 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.188 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.188 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:49.188 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843187+0000", 2024-09-20T18:51:49.188 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.107649921 2024-09-20T18:51:49.188 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.188 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.188 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:49.188 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.845296+0000", 2024-09-20T18:51:49.188 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0021081920000000001 2024-09-20T18:51:49.188 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.188 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.188 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:51:49.188 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.845341+0000", 2024-09-20T18:51:49.189 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5141e-05 2024-09-20T18:51:49.189 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.189 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.189 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:49.189 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.845376+0000", 2024-09-20T18:51:49.189 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.523e-05 2024-09-20T18:51:49.189 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:49.189 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:49.189 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:49.189 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.189 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.189 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:27560 7.e 7:7d7d0b94:::benchmark_data_smithi083_70643_object27559:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:51:49.189 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:08.736128+0000", 2024-09-20T18:51:49.189 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 40.094783438, 2024-09-20T18:51:49.190 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.112670376, 2024-09-20T18:51:49.190 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:49.190 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:49.190 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:51:49.190 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:51:49.190 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:51:49.190 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:51:49.190 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 27560 2024-09-20T18:51:49.190 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.190 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:49.190 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.190 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:49.191 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736128+0000", 2024-09-20T18:51:49.191 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:49.191 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.191 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.191 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:49.191 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736128+0000", 2024-09-20T18:51:49.191 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:49.191 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.191 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.191 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:49.191 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736129+0000", 2024-09-20T18:51:49.191 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.251e-06 2024-09-20T18:51:49.191 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.192 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.192 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:49.192 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736150+0000", 2024-09-20T18:51:49.192 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1450999999999998e-05 2024-09-20T18:51:49.192 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.192 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.192 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:49.192 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736161+0000", 2024-09-20T18:51:49.192 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0682e-05 2024-09-20T18:51:49.192 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.192 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.192 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:49.192 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736168+0000", 2024-09-20T18:51:49.192 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8549999999999996e-06 2024-09-20T18:51:49.193 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.193 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.193 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:49.193 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736218+0000", 2024-09-20T18:51:49.193 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0497000000000001e-05 2024-09-20T18:51:49.193 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.193 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.193 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:49.193 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736327+0000", 2024-09-20T18:51:49.193 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010837599999999999 2024-09-20T18:51:49.193 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.193 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.193 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:51:49.193 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.841982+0000", 2024-09-20T18:51:49.193 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.105654846 2024-09-20T18:51:49.194 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.194 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.194 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:51:49.194 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.845386+0000", 2024-09-20T18:51:49.194 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0034042719999999999 2024-09-20T18:51:49.194 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.194 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.194 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:49.194 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.845423+0000", 2024-09-20T18:51:49.194 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7160000000000003e-05 2024-09-20T18:51:49.194 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.194 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.194 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:49.195 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.848702+0000", 2024-09-20T18:51:49.195 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0032792099999999999 2024-09-20T18:51:49.195 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.195 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.195 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:51:49.195 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.848758+0000", 2024-09-20T18:51:49.195 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.537e-05 2024-09-20T18:51:49.195 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.195 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.195 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:49.195 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.848798+0000", 2024-09-20T18:51:49.195 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0405999999999999e-05 2024-09-20T18:51:49.195 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:49.195 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:49.196 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:49.196 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.196 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.196 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:27559 7.a e93/83 7:5f89880a:::benchmark_data_smithi083_70643_object27558:head v 93'1704, mlcod=93'1702)", 2024-09-20T18:51:49.196 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:08.736205+0000", 2024-09-20T18:51:49.196 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 40.094705908999998, 2024-09-20T18:51:49.196 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.108390217, 2024-09-20T18:51:49.196 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:49.196 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:49.196 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:51:49.196 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:49.196 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.196 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:49.196 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736205+0000", 2024-09-20T18:51:49.197 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:49.197 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.197 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.197 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:49.197 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736205+0000", 2024-09-20T18:51:49.197 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:49.197 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.197 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.197 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:49.197 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736206+0000", 2024-09-20T18:51:49.197 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4000000000000001e-07 2024-09-20T18:51:49.197 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.197 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.197 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:49.198 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736226+0000", 2024-09-20T18:51:49.198 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0426999999999999e-05 2024-09-20T18:51:49.198 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.198 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.198 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:49.198 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736236+0000", 2024-09-20T18:51:49.198 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.781e-06 2024-09-20T18:51:49.198 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.198 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.198 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:49.198 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736242+0000", 2024-09-20T18:51:49.198 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0179999999999998e-06 2024-09-20T18:51:49.198 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.198 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.199 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:49.199 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736307+0000", 2024-09-20T18:51:49.199 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5042000000000005e-05 2024-09-20T18:51:49.199 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.199 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.199 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:49.199 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736325+0000", 2024-09-20T18:51:49.199 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7895999999999999e-05 2024-09-20T18:51:49.199 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.199 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.199 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:51:49.199 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.844517+0000", 2024-09-20T18:51:49.199 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.10819235100000001 2024-09-20T18:51:49.200 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.200 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.200 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:49.200 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.844595+0000", 2024-09-20T18:51:49.200 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7961999999999997e-05 2024-09-20T18:51:49.200 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:49.200 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:49.200 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:49.200 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.200 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.200 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:27549 7.e e93/83 ondisk, result = 0)", 2024-09-20T18:51:49.200 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:08.736243+0000", 2024-09-20T18:51:49.200 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 40.094668315, 2024-09-20T18:51:49.200 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.106913722, 2024-09-20T18:51:49.200 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:49.200 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:49.201 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:51:49.201 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:49.201 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.201 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:49.201 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736243+0000", 2024-09-20T18:51:49.201 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:49.201 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.201 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.201 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:49.201 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736243+0000", 2024-09-20T18:51:49.201 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:49.202 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.202 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.202 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:49.202 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736243+0000", 2024-09-20T18:51:49.202 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3200000000000004e-07 2024-09-20T18:51:49.202 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.202 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.202 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:49.202 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736248+0000", 2024-09-20T18:51:49.202 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2780000000000004e-06 2024-09-20T18:51:49.202 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.203 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.203 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:49.203 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736257+0000", 2024-09-20T18:51:49.203 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2790000000000004e-06 2024-09-20T18:51:49.203 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.203 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.203 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:49.203 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736263+0000", 2024-09-20T18:51:49.203 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3570000000000003e-06 2024-09-20T18:51:49.203 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.203 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.203 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:49.203 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843136+0000", 2024-09-20T18:51:49.203 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.10687292499999999 2024-09-20T18:51:49.203 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.204 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.204 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:49.204 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843143+0000", 2024-09-20T18:51:49.204 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2300000000000002e-06 2024-09-20T18:51:49.204 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.204 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.204 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:49.204 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843156+0000", 2024-09-20T18:51:49.204 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2921e-05 2024-09-20T18:51:49.204 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:49.204 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:49.204 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:49.204 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.204 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.204 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:27555 7.e e93/83 ondisk, result = 0)", 2024-09-20T18:51:49.205 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:08.736486+0000", 2024-09-20T18:51:49.205 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 40.094425227000002, 2024-09-20T18:51:49.205 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.106708079, 2024-09-20T18:51:49.205 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:49.205 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:49.205 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:51:49.205 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:49.205 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.205 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:49.206 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736486+0000", 2024-09-20T18:51:49.206 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:49.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.206 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.206 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:49.206 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736486+0000", 2024-09-20T18:51:49.206 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:49.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.206 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.207 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:49.207 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736486+0000", 2024-09-20T18:51:49.207 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8100000000000002e-07 2024-09-20T18:51:49.207 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.207 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.207 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:49.207 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736490+0000", 2024-09-20T18:51:49.207 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.546e-06 2024-09-20T18:51:49.207 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.207 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.207 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:49.207 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.841491+0000", 2024-09-20T18:51:49.208 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.105001423 2024-09-20T18:51:49.208 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.208 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.208 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:49.208 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.841503+0000", 2024-09-20T18:51:49.208 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1399e-05 2024-09-20T18:51:49.208 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.208 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.209 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:49.209 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843177+0000", 2024-09-20T18:51:49.209 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0016741900000000001 2024-09-20T18:51:49.209 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.209 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.209 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:49.209 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843183+0000", 2024-09-20T18:51:49.209 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7209999999999999e-06 2024-09-20T18:51:49.209 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.209 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.209 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:49.209 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843194+0000", 2024-09-20T18:51:49.210 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1119e-05 2024-09-20T18:51:49.210 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:49.210 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:49.210 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:49.210 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.210 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.210 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:27551 7.8 e93/83 ondisk, result = 0)", 2024-09-20T18:51:49.210 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:08.736884+0000", 2024-09-20T18:51:49.210 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 40.094027429, 2024-09-20T18:51:49.210 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.105442044, 2024-09-20T18:51:49.210 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:49.210 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:49.210 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:51:49.211 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:49.211 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.211 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:49.211 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736884+0000", 2024-09-20T18:51:49.211 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:49.211 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.211 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.211 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:49.211 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736884+0000", 2024-09-20T18:51:49.211 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:49.211 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.211 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.211 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:49.211 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736884+0000", 2024-09-20T18:51:49.212 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6499999999999999e-07 2024-09-20T18:51:49.212 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.212 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.212 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:49.212 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736888+0000", 2024-09-20T18:51:49.212 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7170000000000002e-06 2024-09-20T18:51:49.212 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.212 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.212 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:49.212 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.841517+0000", 2024-09-20T18:51:49.212 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.104628836 2024-09-20T18:51:49.212 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.212 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.213 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:49.213 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.841525+0000", 2024-09-20T18:51:49.213 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7110000000000001e-06 2024-09-20T18:51:49.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.213 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.213 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:49.213 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.842286+0000", 2024-09-20T18:51:49.213 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00076098500000000005 2024-09-20T18:51:49.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.213 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.213 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:49.213 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.842304+0000", 2024-09-20T18:51:49.213 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7898e-05 2024-09-20T18:51:49.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.214 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.214 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:49.214 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.842326+0000", 2024-09-20T18:51:49.214 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1331999999999998e-05 2024-09-20T18:51:49.214 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:49.214 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:49.214 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:49.214 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.214 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.214 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:27571 7.8 7:1e0af83a:::benchmark_data_smithi083_70643_object27570:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:51:49.214 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:08.741453+0000", 2024-09-20T18:51:49.214 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 40.089458225000001, 2024-09-20T18:51:49.215 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.103905603, 2024-09-20T18:51:49.215 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:49.215 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:49.215 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:51:49.215 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:51:49.215 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:51:49.215 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:51:49.215 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 27571 2024-09-20T18:51:49.215 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.215 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:49.215 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.215 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:49.215 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.741453+0000", 2024-09-20T18:51:49.215 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:49.215 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.216 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.216 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:49.216 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.741453+0000", 2024-09-20T18:51:49.216 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:49.216 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.216 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.216 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:49.216 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.741455+0000", 2024-09-20T18:51:49.216 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4669999999999998e-06 2024-09-20T18:51:49.216 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.216 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.216 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:49.216 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.741483+0000", 2024-09-20T18:51:49.216 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7472999999999999e-05 2024-09-20T18:51:49.217 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.217 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.217 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:49.217 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.841438+0000", 2024-09-20T18:51:49.217 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.099955024000000003 2024-09-20T18:51:49.217 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.217 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.217 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:49.217 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.841457+0000", 2024-09-20T18:51:49.217 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9264999999999999e-05 2024-09-20T18:51:49.217 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.217 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.217 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:49.217 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843412+0000", 2024-09-20T18:51:49.217 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0019553869999999998 2024-09-20T18:51:49.218 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.218 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.218 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:49.218 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843559+0000", 2024-09-20T18:51:49.218 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014645399999999999 2024-09-20T18:51:49.218 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.218 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.218 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4,7", 2024-09-20T18:51:49.218 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843706+0000", 2024-09-20T18:51:49.218 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014695600000000001 2024-09-20T18:51:49.218 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.218 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.218 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:51:49.218 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.844855+0000", 2024-09-20T18:51:49.219 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001149424 2024-09-20T18:51:49.219 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.219 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.219 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:49.219 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.845215+0000", 2024-09-20T18:51:49.219 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00035930100000000002 2024-09-20T18:51:49.219 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.219 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.219 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:49.219 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.845270+0000", 2024-09-20T18:51:49.219 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5461999999999999e-05 2024-09-20T18:51:49.219 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.219 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.219 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:51:49.220 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.845321+0000", 2024-09-20T18:51:49.220 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1360000000000003e-05 2024-09-20T18:51:49.220 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.220 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.220 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:49.220 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.845358+0000", 2024-09-20T18:51:49.220 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7030000000000003e-05 2024-09-20T18:51:49.220 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:49.220 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:49.220 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:49.220 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.220 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.220 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:31046 7.e 7:727c2417:::benchmark_data_smithi083_70643_object31045:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T18:51:49.220 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:10.676401+0000", 2024-09-20T18:51:49.221 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 38.154510344000002, 2024-09-20T18:51:49.221 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090219776000000002, 2024-09-20T18:51:49.221 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:49.221 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:49.221 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:51:49.221 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:51:49.221 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:51:49.221 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:51:49.221 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 31046 2024-09-20T18:51:49.221 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.221 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:49.221 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.221 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:49.221 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:10.676401+0000", 2024-09-20T18:51:49.221 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:49.222 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.222 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.222 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:49.222 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:10.676401+0000", 2024-09-20T18:51:49.222 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:49.222 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.222 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.222 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:49.222 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:10.676402+0000", 2024-09-20T18:51:49.222 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4920000000000001e-06 2024-09-20T18:51:49.222 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.222 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.222 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:49.222 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:10.676424+0000", 2024-09-20T18:51:49.223 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2031999999999998e-05 2024-09-20T18:51:49.223 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.223 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.223 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:49.223 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:10.676439+0000", 2024-09-20T18:51:49.223 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4405e-05 2024-09-20T18:51:49.223 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.223 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.223 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:49.223 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:10.676446+0000", 2024-09-20T18:51:49.223 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2869999999999999e-06 2024-09-20T18:51:49.223 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.223 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.223 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:49.224 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:10.676506+0000", 2024-09-20T18:51:49.224 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0334999999999997e-05 2024-09-20T18:51:49.224 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.224 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.224 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:49.224 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:10.676629+0000", 2024-09-20T18:51:49.224 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000123237 2024-09-20T18:51:49.224 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.224 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.224 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:51:49.224 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:10.676758+0000", 2024-09-20T18:51:49.224 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000128718 2024-09-20T18:51:49.224 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.224 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.224 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:49.225 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:10.678317+0000", 2024-09-20T18:51:49.225 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001559326 2024-09-20T18:51:49.225 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.225 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.225 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:51:49.225 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:10.678816+0000", 2024-09-20T18:51:49.225 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00049899099999999995 2024-09-20T18:51:49.225 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.225 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.225 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:49.225 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:10.766483+0000", 2024-09-20T18:51:49.225 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087666633999999993 2024-09-20T18:51:49.225 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.225 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.226 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:51:49.226 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:10.766576+0000", 2024-09-20T18:51:49.226 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2731000000000004e-05 2024-09-20T18:51:49.226 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.226 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.226 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:49.226 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:10.766620+0000", 2024-09-20T18:51:49.226 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4588000000000001e-05 2024-09-20T18:51:49.226 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:49.226 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:49.226 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:49.226 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.227 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.227 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:31056 7.e 7:7b0c61ea:::benchmark_data_smithi083_70643_object31055:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T18:51:49.227 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:10.682329+0000", 2024-09-20T18:51:49.227 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 38.148581682, 2024-09-20T18:51:49.227 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.084433085000000005, 2024-09-20T18:51:49.227 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:49.227 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:49.227 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:51:49.227 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:51:49.227 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:51:49.227 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:51:49.227 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 31056 2024-09-20T18:51:49.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.227 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:49.228 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.228 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:49.228 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:10.682329+0000", 2024-09-20T18:51:49.228 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:49.228 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.228 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.228 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:49.228 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:10.682329+0000", 2024-09-20T18:51:49.228 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:49.228 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.228 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.228 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:49.228 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:10.682330+0000", 2024-09-20T18:51:49.228 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4200000000000004e-07 2024-09-20T18:51:49.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.229 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.229 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:49.229 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:10.682351+0000", 2024-09-20T18:51:49.229 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0775e-05 2024-09-20T18:51:49.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.229 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.229 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:49.229 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:10.682364+0000", 2024-09-20T18:51:49.229 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2661e-05 2024-09-20T18:51:49.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.229 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.229 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:49.229 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:10.682373+0000", 2024-09-20T18:51:49.230 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8550000000000008e-06 2024-09-20T18:51:49.230 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.230 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.230 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:49.230 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:10.682464+0000", 2024-09-20T18:51:49.230 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1780999999999994e-05 2024-09-20T18:51:49.230 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.230 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.230 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:49.230 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:10.682566+0000", 2024-09-20T18:51:49.230 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000101882 2024-09-20T18:51:49.230 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.230 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.230 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:51:49.230 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:10.682701+0000", 2024-09-20T18:51:49.231 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000134439 2024-09-20T18:51:49.231 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.231 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.231 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:51:49.231 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:10.684350+0000", 2024-09-20T18:51:49.231 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0016488989999999999 2024-09-20T18:51:49.231 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.231 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.231 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:49.231 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:10.684389+0000", 2024-09-20T18:51:49.231 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9651999999999999e-05 2024-09-20T18:51:49.231 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.231 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.231 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:49.231 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:10.766661+0000", 2024-09-20T18:51:49.232 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.082271644000000005 2024-09-20T18:51:49.232 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.232 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.232 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:51:49.232 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:10.766730+0000", 2024-09-20T18:51:49.232 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.923e-05 2024-09-20T18:51:49.232 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.232 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.232 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:49.232 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:10.766762+0000", 2024-09-20T18:51:49.232 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2325e-05 2024-09-20T18:51:49.232 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:49.232 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:49.232 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:49.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.233 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.233 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:31057 7.e 7:79e8999d:::benchmark_data_smithi083_70643_object31056:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T18:51:49.233 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:10.682447+0000", 2024-09-20T18:51:49.233 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 38.148463632999999, 2024-09-20T18:51:49.233 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.085146274999999993, 2024-09-20T18:51:49.233 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:49.233 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:49.233 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:51:49.233 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:51:49.233 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:51:49.233 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:51:49.233 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 31057 2024-09-20T18:51:49.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.234 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:49.234 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.234 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:49.234 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:10.682447+0000", 2024-09-20T18:51:49.234 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:49.234 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.234 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.234 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:49.234 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:10.682447+0000", 2024-09-20T18:51:49.234 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:49.234 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.234 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.234 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:49.234 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:10.682448+0000", 2024-09-20T18:51:49.235 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7000000000000003e-07 2024-09-20T18:51:49.235 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.235 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.235 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:49.235 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:10.682467+0000", 2024-09-20T18:51:49.235 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.838e-05 2024-09-20T18:51:49.235 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.235 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.235 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:49.235 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:10.682476+0000", 2024-09-20T18:51:49.235 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8670000000000006e-06 2024-09-20T18:51:49.235 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.235 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.235 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:49.235 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:10.682483+0000", 2024-09-20T18:51:49.236 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8220000000000001e-06 2024-09-20T18:51:49.236 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.236 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.236 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:49.236 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:10.683666+0000", 2024-09-20T18:51:49.236 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011830650000000001 2024-09-20T18:51:49.236 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.236 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.236 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:49.236 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:10.683735+0000", 2024-09-20T18:51:49.236 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.881e-05 2024-09-20T18:51:49.236 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.236 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.236 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:51:49.236 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:10.683837+0000", 2024-09-20T18:51:49.237 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000101474 2024-09-20T18:51:49.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.237 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.237 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:51:49.237 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:10.684475+0000", 2024-09-20T18:51:49.237 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00063832600000000004 2024-09-20T18:51:49.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.237 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.237 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:49.237 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:10.684845+0000", 2024-09-20T18:51:49.237 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00037019499999999998 2024-09-20T18:51:49.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.237 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.237 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:49.238 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:10.767010+0000", 2024-09-20T18:51:49.238 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.082164514999999994 2024-09-20T18:51:49.238 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.238 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.238 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:51:49.238 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:10.767420+0000", 2024-09-20T18:51:49.238 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00041014099999999999 2024-09-20T18:51:49.238 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.238 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.238 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:49.238 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:10.767594+0000", 2024-09-20T18:51:49.238 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017380999999999999 2024-09-20T18:51:49.238 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:49.238 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:49.239 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:49.239 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.239 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.239 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:31542 7.e 7:712fc081:::benchmark_data_smithi083_70643_object31541:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T18:51:49.239 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:10.999948+0000", 2024-09-20T18:51:49.239 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 37.830962528000001, 2024-09-20T18:51:49.239 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.065962138000000003, 2024-09-20T18:51:49.239 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:49.239 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:49.239 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:51:49.239 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:51:49.239 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:51:49.239 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:51:49.239 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 31542 2024-09-20T18:51:49.240 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.240 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:49.240 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.240 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:49.240 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:10.999948+0000", 2024-09-20T18:51:49.240 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:49.240 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.240 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.240 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:49.240 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:10.999948+0000", 2024-09-20T18:51:49.240 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:49.240 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.240 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.240 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:49.241 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:10.999949+0000", 2024-09-20T18:51:49.241 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.1200000000000002e-07 2024-09-20T18:51:49.241 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.241 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.241 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:49.241 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:10.999969+0000", 2024-09-20T18:51:49.241 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.959e-05 2024-09-20T18:51:49.241 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.241 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.241 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:49.241 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:10.999979+0000", 2024-09-20T18:51:49.241 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9199999999999999e-06 2024-09-20T18:51:49.241 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.241 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.241 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:49.242 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:10.999985+0000", 2024-09-20T18:51:49.242 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6950000000000003e-06 2024-09-20T18:51:49.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.242 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.242 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:49.242 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:11.000929+0000", 2024-09-20T18:51:49.242 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00094328000000000001 2024-09-20T18:51:49.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.242 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.242 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:49.242 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:11.001034+0000", 2024-09-20T18:51:49.242 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000104984 2024-09-20T18:51:49.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.242 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.243 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:51:49.243 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:11.061659+0000", 2024-09-20T18:51:49.243 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.060625701999999997 2024-09-20T18:51:49.243 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.243 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.243 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:49.243 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:11.063523+0000", 2024-09-20T18:51:49.243 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0018637790000000001 2024-09-20T18:51:49.243 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.243 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.243 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:51:49.243 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:11.063627+0000", 2024-09-20T18:51:49.243 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010357999999999999 2024-09-20T18:51:49.243 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.244 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.244 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:49.244 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:11.065809+0000", 2024-09-20T18:51:49.244 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002181807 2024-09-20T18:51:49.244 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.244 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.244 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:51:49.244 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:11.065869+0000", 2024-09-20T18:51:49.244 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0797000000000001e-05 2024-09-20T18:51:49.244 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.244 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.244 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:49.244 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:11.065911+0000", 2024-09-20T18:51:49.244 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1192000000000001e-05 2024-09-20T18:51:49.244 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:49.245 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:49.245 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:49.245 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.245 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.245 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:65458 7.e 7:749eeaa0:::benchmark_data_smithi083_70643_object65457:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-20T18:51:49.245 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:32.821549+0000", 2024-09-20T18:51:49.245 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 16.009362337999999, 2024-09-20T18:51:49.245 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.070008487999999994, 2024-09-20T18:51:49.245 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:49.245 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:49.245 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:51:49.245 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:51:49.245 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:51:49.245 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:51:49.246 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 65458 2024-09-20T18:51:49.246 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.246 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:49.246 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.246 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:49.246 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:32.821549+0000", 2024-09-20T18:51:49.246 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:49.246 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.246 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.246 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:49.246 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:32.821549+0000", 2024-09-20T18:51:49.246 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:49.246 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.246 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.246 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:49.247 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:32.821550+0000", 2024-09-20T18:51:49.247 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5e-06 2024-09-20T18:51:49.247 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.247 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.247 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:49.247 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:32.821579+0000", 2024-09-20T18:51:49.247 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8603e-05 2024-09-20T18:51:49.247 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.247 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.247 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:49.247 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:32.821590+0000", 2024-09-20T18:51:49.247 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1094000000000001e-05 2024-09-20T18:51:49.247 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.247 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.248 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:49.248 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:32.821597+0000", 2024-09-20T18:51:49.248 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2099999999999996e-06 2024-09-20T18:51:49.248 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.248 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.248 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:49.248 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:32.821682+0000", 2024-09-20T18:51:49.248 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4561000000000006e-05 2024-09-20T18:51:49.248 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.248 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.248 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:49.248 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:32.821795+0000", 2024-09-20T18:51:49.248 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011341300000000001 2024-09-20T18:51:49.248 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.248 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.249 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:51:49.249 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:32.821922+0000", 2024-09-20T18:51:49.249 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012677899999999999 2024-09-20T18:51:49.249 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.249 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.249 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:49.249 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:32.826135+0000", 2024-09-20T18:51:49.249 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0042128859999999999 2024-09-20T18:51:49.249 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.249 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.249 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:51:49.249 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:32.828568+0000", 2024-09-20T18:51:49.249 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0024332189999999999 2024-09-20T18:51:49.249 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.250 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.250 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:49.250 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:32.891073+0000", 2024-09-20T18:51:49.250 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.062504590999999998 2024-09-20T18:51:49.250 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.250 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.250 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:51:49.250 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:32.891180+0000", 2024-09-20T18:51:49.250 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010772600000000001 2024-09-20T18:51:49.250 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.250 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.250 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:49.250 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:32.891557+0000", 2024-09-20T18:51:49.250 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00037690599999999999 2024-09-20T18:51:49.251 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:49.251 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:49.251 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:49.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.251 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.251 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:65464 7.e 7:706f6e3e:::benchmark_data_smithi083_70643_object65463:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-20T18:51:49.251 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:32.823703+0000", 2024-09-20T18:51:49.251 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 16.007208044999999, 2024-09-20T18:51:49.251 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.070951142999999994, 2024-09-20T18:51:49.251 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:49.251 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:49.251 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:51:49.251 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:51:49.251 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:51:49.251 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:51:49.252 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 65464 2024-09-20T18:51:49.252 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.252 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:49.252 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.252 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:49.252 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:32.823703+0000", 2024-09-20T18:51:49.252 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:49.252 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.252 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.252 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:49.252 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:32.823703+0000", 2024-09-20T18:51:49.252 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:49.252 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.252 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.253 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:49.253 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:32.823704+0000", 2024-09-20T18:51:49.253 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.429e-06 2024-09-20T18:51:49.253 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.253 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.253 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:49.253 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:32.823727+0000", 2024-09-20T18:51:49.253 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2354999999999999e-05 2024-09-20T18:51:49.253 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.253 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.253 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:49.253 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:32.823739+0000", 2024-09-20T18:51:49.253 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1861e-05 2024-09-20T18:51:49.253 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.254 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.254 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:49.254 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:32.823747+0000", 2024-09-20T18:51:49.254 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0530000000000007e-06 2024-09-20T18:51:49.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.254 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.254 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:49.254 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:32.823860+0000", 2024-09-20T18:51:49.254 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000113763 2024-09-20T18:51:49.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.254 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.254 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:49.254 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:32.823971+0000", 2024-09-20T18:51:49.254 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011014 2024-09-20T18:51:49.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.255 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.255 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:51:49.255 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:32.824145+0000", 2024-09-20T18:51:49.255 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017480300000000001 2024-09-20T18:51:49.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.255 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.255 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:49.255 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:32.828556+0000", 2024-09-20T18:51:49.255 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.004410448 2024-09-20T18:51:49.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.255 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.255 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:51:49.255 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:32.828576+0000", 2024-09-20T18:51:49.255 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0415e-05 2024-09-20T18:51:49.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.256 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.256 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:49.256 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:32.894541+0000", 2024-09-20T18:51:49.256 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.065965135999999994 2024-09-20T18:51:49.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.256 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.256 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:51:49.256 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:32.894616+0000", 2024-09-20T18:51:49.256 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4194000000000001e-05 2024-09-20T18:51:49.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.256 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.256 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:49.256 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:32.894654+0000", 2024-09-20T18:51:49.256 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8546e-05 2024-09-20T18:51:49.257 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:49.257 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:49.257 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:49.257 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.257 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.257 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:65468 7.e 7:7d6dd239:::benchmark_data_smithi083_70643_object65467:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-20T18:51:49.257 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:32.824918+0000", 2024-09-20T18:51:49.257 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 16.005993478000001, 2024-09-20T18:51:49.257 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.069861882, 2024-09-20T18:51:49.257 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:49.257 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:49.257 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:51:49.257 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:51:49.257 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:51:49.257 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:51:49.258 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 65468 2024-09-20T18:51:49.258 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.258 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:49.258 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.258 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:49.258 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:32.824918+0000", 2024-09-20T18:51:49.258 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:49.258 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.258 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.258 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:49.258 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:32.824918+0000", 2024-09-20T18:51:49.258 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:49.258 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.258 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.259 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:49.259 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:32.824919+0000", 2024-09-20T18:51:49.259 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.392e-06 2024-09-20T18:51:49.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.259 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:49.259 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:32.824941+0000", 2024-09-20T18:51:49.259 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2481999999999999e-05 2024-09-20T18:51:49.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.259 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:49.259 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:32.824952+0000", 2024-09-20T18:51:49.259 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0141e-05 2024-09-20T18:51:49.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.260 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:49.260 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:32.824959+0000", 2024-09-20T18:51:49.260 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2069999999999998e-06 2024-09-20T18:51:49.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.260 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.260 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:49.260 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:32.826224+0000", 2024-09-20T18:51:49.260 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0012657409999999999 2024-09-20T18:51:49.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.260 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.260 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:49.260 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:32.826348+0000", 2024-09-20T18:51:49.260 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012305000000000001 2024-09-20T18:51:49.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.260 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.261 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:51:49.261 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:32.826488+0000", 2024-09-20T18:51:49.261 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014057699999999999 2024-09-20T18:51:49.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.261 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.261 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:49.261 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:32.828604+0000", 2024-09-20T18:51:49.261 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0021163929999999998 2024-09-20T18:51:49.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.261 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.261 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:51:49.261 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:32.828643+0000", 2024-09-20T18:51:49.261 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8473999999999998e-05 2024-09-20T18:51:49.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.261 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.262 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:49.262 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:32.894695+0000", 2024-09-20T18:51:49.262 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.066051622000000004 2024-09-20T18:51:49.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.262 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.262 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:51:49.262 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:32.894745+0000", 2024-09-20T18:51:49.262 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9968e-05 2024-09-20T18:51:49.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.262 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.262 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:49.262 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:32.894779+0000", 2024-09-20T18:51:49.262 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4835000000000001e-05 2024-09-20T18:51:49.262 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:49.263 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:49.263 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:49.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.263 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.263 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:70159 7.e 7:717ad21a:::benchmark_data_smithi083_70643_object70158:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-20T18:51:49.263 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:36.462374+0000", 2024-09-20T18:51:49.263 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 12.368537178, 2024-09-20T18:51:49.263 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.074706155999999996, 2024-09-20T18:51:49.263 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:49.263 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:49.263 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:51:49.263 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:51:49.263 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:51:49.263 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:51:49.263 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 70159 2024-09-20T18:51:49.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.264 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:49.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.264 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:49.264 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:36.462374+0000", 2024-09-20T18:51:49.264 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:49.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.264 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:49.264 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:36.462374+0000", 2024-09-20T18:51:49.264 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:49.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.264 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:49.265 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:36.462374+0000", 2024-09-20T18:51:49.265 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.37e-07 2024-09-20T18:51:49.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.265 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:49.265 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:36.462405+0000", 2024-09-20T18:51:49.265 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0196e-05 2024-09-20T18:51:49.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.265 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:49.265 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:36.462414+0000", 2024-09-20T18:51:49.265 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0790000000000007e-06 2024-09-20T18:51:49.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.265 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:49.266 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:36.462420+0000", 2024-09-20T18:51:49.266 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8119999999999996e-06 2024-09-20T18:51:49.266 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.266 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.266 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:49.266 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:36.464804+0000", 2024-09-20T18:51:49.266 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0023840739999999999 2024-09-20T18:51:49.266 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.266 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.266 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:49.266 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:36.464948+0000", 2024-09-20T18:51:49.266 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014421700000000001 2024-09-20T18:51:49.266 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.266 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.267 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:51:49.267 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:36.465088+0000", 2024-09-20T18:51:49.267 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014051800000000001 2024-09-20T18:51:49.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.267 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.267 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:51:49.267 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:36.477584+0000", 2024-09-20T18:51:49.268 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.012495852 2024-09-20T18:51:49.268 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.268 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.268 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:49.268 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:36.478093+0000", 2024-09-20T18:51:49.268 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00050868099999999998 2024-09-20T18:51:49.268 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.268 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.269 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:49.269 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:36.536182+0000", 2024-09-20T18:51:49.269 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.058088965999999999 2024-09-20T18:51:49.269 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.269 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.269 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:51:49.269 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:36.537029+0000", 2024-09-20T18:51:49.269 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00084695699999999996 2024-09-20T18:51:49.269 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.269 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.269 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:49.269 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:36.537080+0000", 2024-09-20T18:51:49.269 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1167e-05 2024-09-20T18:51:49.270 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:49.270 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:49.270 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:49.270 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.270 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.270 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:70170 7.e 7:7bd0b565:::benchmark_data_smithi083_70643_object70169:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-20T18:51:49.270 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:36.470857+0000", 2024-09-20T18:51:49.270 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 12.36005394, 2024-09-20T18:51:49.270 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.077080552999999996, 2024-09-20T18:51:49.270 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:49.270 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:49.270 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:51:49.270 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:51:49.270 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:51:49.271 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:51:49.271 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 70170 2024-09-20T18:51:49.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.271 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:49.271 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.271 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:49.271 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:36.470857+0000", 2024-09-20T18:51:49.271 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:49.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.271 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.271 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:49.271 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:36.470857+0000", 2024-09-20T18:51:49.271 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:49.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.272 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.272 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:49.272 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:36.470859+0000", 2024-09-20T18:51:49.272 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5719999999999999e-06 2024-09-20T18:51:49.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.272 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.272 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:49.272 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:36.470885+0000", 2024-09-20T18:51:49.272 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6387e-05 2024-09-20T18:51:49.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.272 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.272 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:49.272 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:36.470897+0000", 2024-09-20T18:51:49.273 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2459999999999999e-05 2024-09-20T18:51:49.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.273 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.273 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:49.273 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:36.470905+0000", 2024-09-20T18:51:49.273 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.6520000000000006e-06 2024-09-20T18:51:49.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.273 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.273 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:49.273 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:36.475067+0000", 2024-09-20T18:51:49.273 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0041617929999999996 2024-09-20T18:51:49.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.273 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.273 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:49.274 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:36.475226+0000", 2024-09-20T18:51:49.274 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015885899999999999 2024-09-20T18:51:49.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.274 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.274 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:51:49.274 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:36.475367+0000", 2024-09-20T18:51:49.274 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000141435 2024-09-20T18:51:49.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.274 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.274 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:51:49.274 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:36.481022+0000", 2024-09-20T18:51:49.274 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0056544999999999998 2024-09-20T18:51:49.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.274 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.275 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:49.275 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:36.483003+0000", 2024-09-20T18:51:49.275 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0019811020000000002 2024-09-20T18:51:49.275 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.275 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.275 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:49.275 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:36.547841+0000", 2024-09-20T18:51:49.275 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.064837832999999997 2024-09-20T18:51:49.275 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.275 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.275 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:51:49.275 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:36.547897+0000", 2024-09-20T18:51:49.275 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5979000000000001e-05 2024-09-20T18:51:49.276 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.276 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.276 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:49.276 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:36.547938+0000", 2024-09-20T18:51:49.276 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0980999999999999e-05 2024-09-20T18:51:49.276 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:49.276 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:49.276 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:49.276 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.276 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.276 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:70178 7.e 7:779eac47:::benchmark_data_smithi083_70643_object70177:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-20T18:51:49.276 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:36.478222+0000", 2024-09-20T18:51:49.276 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 12.352688712999999, 2024-09-20T18:51:49.276 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.069840810000000003, 2024-09-20T18:51:49.277 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:49.277 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:49.277 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:51:49.277 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:51:49.277 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:51:49.277 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:51:49.277 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 70178 2024-09-20T18:51:49.277 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.277 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:49.277 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.277 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:49.277 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:36.478222+0000", 2024-09-20T18:51:49.277 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:49.277 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.277 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.278 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:49.278 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:36.478222+0000", 2024-09-20T18:51:49.278 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:49.278 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.278 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.278 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:49.278 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:36.478224+0000", 2024-09-20T18:51:49.278 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.437e-06 2024-09-20T18:51:49.278 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.278 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.278 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:49.278 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:36.478245+0000", 2024-09-20T18:51:49.278 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1121999999999999e-05 2024-09-20T18:51:49.278 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.279 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.279 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:49.279 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:36.478255+0000", 2024-09-20T18:51:49.279 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9029999999999999e-06 2024-09-20T18:51:49.279 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.279 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.279 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:49.279 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:36.478262+0000", 2024-09-20T18:51:49.279 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0090000000000001e-06 2024-09-20T18:51:49.279 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.279 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.279 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:49.279 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:36.478313+0000", 2024-09-20T18:51:49.279 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1508000000000002e-05 2024-09-20T18:51:49.279 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.280 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.280 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:49.280 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:36.478435+0000", 2024-09-20T18:51:49.280 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121539 2024-09-20T18:51:49.280 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.280 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.280 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:51:49.280 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:36.478596+0000", 2024-09-20T18:51:49.280 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000161101 2024-09-20T18:51:49.280 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.280 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.280 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:51:49.280 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:36.482067+0000", 2024-09-20T18:51:49.280 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003470767 2024-09-20T18:51:49.280 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.281 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.281 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:49.281 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:36.535947+0000", 2024-09-20T18:51:49.281 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.053880439000000002 2024-09-20T18:51:49.281 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.281 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.281 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:49.281 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:36.547989+0000", 2024-09-20T18:51:49.281 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.012041938 2024-09-20T18:51:49.281 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.281 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.281 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:51:49.281 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:36.548030+0000", 2024-09-20T18:51:49.281 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0497999999999997e-05 2024-09-20T18:51:49.281 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:49.282 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:49.282 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:51:49.282 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:36.548063+0000", 2024-09-20T18:51:49.282 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3549e-05 2024-09-20T18:51:49.282 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:49.282 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:49.282 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:49.282 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:49.282 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:49.282 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T18:51:49.282 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:49.283 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:48 smithi142 ceph-mon[32508]: osdmap e106: 8 total, 8 up, 8 in 2024-09-20T18:51:49.310 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:49.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:49.311 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:49.349 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:51:49.351 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-20T18:51:49.412 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:49.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:49.451 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:49.452 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:49.517 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T18:51:49.524 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 52 16 85032 85016 102.159 68 0.0121963 0.00978377 2024-09-20T18:51:49.553 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:49.579 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:49.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:49.580 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:49.681 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:51:49.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:49.738 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:49.738 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:51:49.839 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:51:49.876 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:49.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:49.877 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:51:49.978 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T18:51:49.979 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:50.007 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:50.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:50.008 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:50.109 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:50.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:50.152 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:50.153 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:50.185 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:49 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:51:50.186 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:49 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:51:50.186 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:49 smithi083 ceph-mon[29421]: pgmap v82: 148 pgs: 148 active+clean; 4.9 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 72 MiB/s wr, 1.15k op/s 2024-09-20T18:51:50.186 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:49 smithi083 ceph-mon[29421]: osdmap e107: 8 total, 8 up, 8 in 2024-09-20T18:51:50.186 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:49 smithi083 ceph-mon[29421]: from='client.14868 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:51:50.186 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:49 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:51:50.187 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:49 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:51:50.187 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:49 smithi083 ceph-mon[36260]: pgmap v82: 148 pgs: 148 active+clean; 4.9 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 72 MiB/s wr, 1.15k op/s 2024-09-20T18:51:50.187 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:49 smithi083 ceph-mon[36260]: osdmap e107: 8 total, 8 up, 8 in 2024-09-20T18:51:50.187 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:49 smithi083 ceph-mon[36260]: from='client.14868 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:51:50.254 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:50.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:49 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:51:50.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:49 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:51:50.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:49 smithi142 ceph-mon[32508]: pgmap v82: 148 pgs: 148 active+clean; 4.9 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 72 MiB/s wr, 1.15k op/s 2024-09-20T18:51:50.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:49 smithi142 ceph-mon[32508]: osdmap e107: 8 total, 8 up, 8 in 2024-09-20T18:51:50.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:49 smithi142 ceph-mon[32508]: from='client.14868 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:51:50.280 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:50.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:50.281 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:50.382 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:50.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:50.413 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:50.413 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:50.514 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:50.526 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 53 16 86125 86109 101.521 68.3125 0.00868461 0.00984488 2024-09-20T18:51:50.539 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:50.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:50.541 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:50.640 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2024-09-20T18:51:48.898900+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5294791195,"num_objects":80982,"num_object_clones":0,"num_object_copies":242946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80982,"num_whiteouts":0,"num_read":963,"num_read_kb":820,"num_write":81332,"num_write_kb":5171785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":590698,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79635,"ondisk_log_size":79635,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":15840092,"kb_used_data":15405980,"kb_used_omap":12,"kb_used_meta":433971,"kb_avail":734121124,"statfs":{"total":767960285184,"available":751740030976,"internally_reserved":0,"allocated":15775723520,"data_stored":15770737593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":444386864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":355270656,"num_objects":5421,"num_object_clones":0,"num_object_copies":16263,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5421,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5421,"num_write_kb":346944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.122096"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":100,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.596911+0000","last_change":"2024-09-20T18:49:45.943560+0000","last_active":"2024-09-20T18:51:40.596911+0000","last_peered":"2024-09-20T18:51:40.596911+0000","last_clean":"2024-09-20T18:51:40.596911+0000","last_became_active":"2024-09-20T18:49:45.943201+0000","last_became_peered":"2024-09-20T18:49:45.943201+0000","last_unstale":"2024-09-20T18:51:40.596911+0000","last_undegraded":"2024-09-20T18:51:40.596911+0000","last_fullsized":"2024-09-20T18:51:40.596911+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":121,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.581955+0000","last_change":"2024-09-20T18:51:30.551475+0000","last_active":"2024-09-20T18:51:40.581955+0000","last_peered":"2024-09-20T18:51:40.581955+0000","last_clean":"2024-09-20T18:51:40.581955+0000","last_became_active":"2024-09-20T18:49:47.946161+0000","last_became_peered":"2024-09-20T18:49:47.946161+0000","last_unstale":"2024-09-20T18:51:40.581955+0000","last_undegraded":"2024-09-20T18:51:40.581955+0000","last_fullsized":"2024-09-20T18:51:40.581955+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:51:30.551395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:30.551395+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:53.379329+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":93,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.589945+0000","last_change":"2024-09-20T18:49:49.959797+0000","last_active":"2024-09-20T18:51:40.589945+0000","last_peered":"2024-09-20T18:51:40.589945+0000","last_clean":"2024-09-20T18:51:40.589945+0000","last_became_active":"2024-09-20T18:49:49.959514+0000","last_became_peered":"2024-09-20T18:49:49.959514+0000","last_unstale":"2024-09-20T18:51:40.589945+0000","last_undegraded":"2024-09-20T18:51:40.589945+0000","last_fullsized":"2024-09-20T18:51:40.589945+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:06.201644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":97,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.582157+0000","last_change":"2024-09-20T18:51:29.514758+0000","last_active":"2024-09-20T18:51:40.582157+0000","last_peered":"2024-09-20T18:51:40.582157+0000","last_clean":"2024-09-20T18:51:40.582157+0000","last_became_active":"2024-09-20T18:49:51.969421+0000","last_became_peered":"2024-09-20T18:49:51.969421+0000","last_unstale":"2024-09-20T18:51:40.582157+0000","last_undegraded":"2024-09-20T18:51:40.582157+0000","last_fullsized":"2024-09-20T18:51:40.582157+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:29.514693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:29.514693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:46.221285+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":97,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.587127+0000","last_change":"2024-09-20T18:51:29.257299+0000","last_active":"2024-09-20T18:51:40.587127+0000","last_peered":"2024-09-20T18:51:40.587127+0000","last_clean":"2024-09-20T18:51:40.587127+0000","last_became_active":"2024-09-20T18:49:52.141967+0000","last_became_peered":"2024-09-20T18:49:52.141967+0000","last_unstale":"2024-09-20T18:51:40.587127+0000","last_undegraded":"2024-09-20T18:51:40.587127+0000","last_fullsized":"2024-09-20T18:51:40.587127+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:29.257220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:29.257220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:48.433324+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":109,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.586440+0000","last_change":"2024-09-20T18:51:25.208165+0000","last_active":"2024-09-20T18:51:40.586440+0000","last_peered":"2024-09-20T18:51:40.586440+0000","last_clean":"2024-09-20T18:51:40.586440+0000","last_became_active":"2024-09-20T18:49:45.943607+0000","last_became_peered":"2024-09-20T18:49:45.943607+0000","last_unstale":"2024-09-20T18:51:40.586440+0000","last_undegraded":"2024-09-20T18:51:40.586440+0000","last_fullsized":"2024-09-20T18:51:40.586440+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:25.208073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:25.208073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:40.322474+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":130,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.585207+0000","last_change":"2024-09-20T18:51:07.130137+0000","last_active":"2024-09-20T18:51:40.585207+0000","last_peered":"2024-09-20T18:51:40.585207+0000","last_clean":"2024-09-20T18:51:40.585207+0000","last_became_active":"2024-09-20T18:49:47.952150+0000","last_became_peered":"2024-09-20T18:49:47.952150+0000","last_unstale":"2024-09-20T18:51:40.585207+0000","last_undegraded":"2024-09-20T18:51:40.585207+0000","last_fullsized":"2024-09-20T18:51:40.585207+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:51:07.130062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:07.130062+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:12.295095+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":101,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.581867+0000","last_change":"2024-09-20T18:51:09.436429+0000","last_active":"2024-09-20T18:51:40.581867+0000","last_peered":"2024-09-20T18:51:40.581867+0000","last_clean":"2024-09-20T18:51:40.581867+0000","last_became_active":"2024-09-20T18:49:49.956445+0000","last_became_peered":"2024-09-20T18:49:49.956445+0000","last_unstale":"2024-09-20T18:51:40.581867+0000","last_undegraded":"2024-09-20T18:51:40.581867+0000","last_fullsized":"2024-09-20T18:51:40.581867+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:09.436302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:09.436302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:35.779478+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"59'1","reported_seq":114,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.588258+0000","last_change":"2024-09-20T18:51:13.868434+0000","last_active":"2024-09-20T18:51:40.588258+0000","last_peered":"2024-09-20T18:51:40.588258+0000","last_clean":"2024-09-20T18:51:40.588258+0000","last_became_active":"2024-09-20T18:49:45.946171+0000","last_became_peered":"2024-09-20T18:49:45.946171+0000","last_unstale":"2024-09-20T18:51:40.588258+0000","last_undegraded":"2024-09-20T18:51:40.588258+0000","last_fullsized":"2024-09-20T18:51:40.588258+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:51:13.868378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:13.868378+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:36.299231+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":121,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.583234+0000","last_change":"2024-09-20T18:51:31.541589+0000","last_active":"2024-09-20T18:51:40.583234+0000","last_peered":"2024-09-20T18:51:40.583234+0000","last_clean":"2024-09-20T18:51:40.583234+0000","last_became_active":"2024-09-20T18:49:47.947448+0000","last_became_peered":"2024-09-20T18:49:47.947448+0000","last_unstale":"2024-09-20T18:51:40.583234+0000","last_undegraded":"2024-09-20T18:51:40.583234+0000","last_fullsized":"2024-09-20T18:51:40.583234+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:51:31.541518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:31.541518+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:00.525209+0000","scrub_duration":38,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":100,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.588293+0000","last_change":"2024-09-20T18:51:23.780930+0000","last_active":"2024-09-20T18:51:40.588293+0000","last_peered":"2024-09-20T18:51:40.588293+0000","last_clean":"2024-09-20T18:51:40.588293+0000","last_became_active":"2024-09-20T18:49:49.961188+0000","last_became_peered":"2024-09-20T18:49:49.961188+0000","last_unstale":"2024-09-20T18:51:40.588293+0000","last_undegraded":"2024-09-20T18:51:40.588293+0000","last_fullsized":"2024-09-20T18:51:40.588293+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:23.780885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:23.780885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:29.953296+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":88,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.594679+0000","last_change":"2024-09-20T18:49:51.966957+0000","last_active":"2024-09-20T18:51:40.594679+0000","last_peered":"2024-09-20T18:51:40.594679+0000","last_clean":"2024-09-20T18:51:40.594679+0000","last_became_active":"2024-09-20T18:49:51.966715+0000","last_became_peered":"2024-09-20T18:49:51.966715+0000","last_unstale":"2024-09-20T18:51:40.594679+0000","last_undegraded":"2024-09-20T18:51:40.594679+0000","last_fullsized":"2024-09-20T18:51:40.594679+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:54.991522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":100,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.588288+0000","last_change":"2024-09-20T18:49:45.945004+0000","last_active":"2024-09-20T18:51:40.588288+0000","last_peered":"2024-09-20T18:51:40.588288+0000","last_clean":"2024-09-20T18:51:40.588288+0000","last_became_active":"2024-09-20T18:49:45.944703+0000","last_became_peered":"2024-09-20T18:49:45.944703+0000","last_unstale":"2024-09-20T18:51:40.588288+0000","last_undegraded":"2024-09-20T18:51:40.588288+0000","last_fullsized":"2024-09-20T18:51:40.588288+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:57.775249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":141,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.582150+0000","last_change":"2024-09-20T18:51:26.494194+0000","last_active":"2024-09-20T18:51:40.582150+0000","last_peered":"2024-09-20T18:51:40.582150+0000","last_clean":"2024-09-20T18:51:40.582150+0000","last_became_active":"2024-09-20T18:51:23.813399+0000","last_became_peered":"2024-09-20T18:51:23.813399+0000","last_unstale":"2024-09-20T18:51:40.582150+0000","last_undegraded":"2024-09-20T18:51:40.582150+0000","last_fullsized":"2024-09-20T18:51:40.582150+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-20T18:51:26.494118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:26.494118+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:33.623927+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":100,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.587305+0000","last_change":"2024-09-20T18:51:27.981304+0000","last_active":"2024-09-20T18:51:40.587305+0000","last_peered":"2024-09-20T18:51:40.587305+0000","last_clean":"2024-09-20T18:51:40.587305+0000","last_became_active":"2024-09-20T18:49:49.959696+0000","last_became_peered":"2024-09-20T18:49:49.959696+0000","last_unstale":"2024-09-20T18:51:40.587305+0000","last_undegraded":"2024-09-20T18:51:40.587305+0000","last_fullsized":"2024-09-20T18:51:40.587305+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:27.981238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:27.981238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:28.100332+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":96,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.586511+0000","last_change":"2024-09-20T18:51:12.918301+0000","last_active":"2024-09-20T18:51:40.586511+0000","last_peered":"2024-09-20T18:51:40.586511+0000","last_clean":"2024-09-20T18:51:40.586511+0000","last_became_active":"2024-09-20T18:49:51.970663+0000","last_became_peered":"2024-09-20T18:49:51.970663+0000","last_unstale":"2024-09-20T18:51:40.586511+0000","last_undegraded":"2024-09-20T18:51:40.586511+0000","last_fullsized":"2024-09-20T18:51:40.586511+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:12.918228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:12.918228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:14.801225+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":89,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.582166+0000","last_change":"2024-09-20T18:49:52.141482+0000","last_active":"2024-09-20T18:51:40.582166+0000","last_peered":"2024-09-20T18:51:40.582166+0000","last_clean":"2024-09-20T18:51:40.582166+0000","last_became_active":"2024-09-20T18:49:52.141170+0000","last_became_peered":"2024-09-20T18:49:52.141170+0000","last_unstale":"2024-09-20T18:51:40.582166+0000","last_undegraded":"2024-09-20T18:51:40.582166+0000","last_fullsized":"2024-09-20T18:51:40.582166+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":100,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.598389+0000","last_change":"2024-09-20T18:49:45.950714+0000","last_active":"2024-09-20T18:51:40.598389+0000","last_peered":"2024-09-20T18:51:40.598389+0000","last_clean":"2024-09-20T18:51:40.598389+0000","last_became_active":"2024-09-20T18:49:45.950480+0000","last_became_peered":"2024-09-20T18:49:45.950480+0000","last_unstale":"2024-09-20T18:51:40.598389+0000","last_undegraded":"2024-09-20T18:51:40.598389+0000","last_fullsized":"2024-09-20T18:51:40.598389+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:40.252027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":131,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.578715+0000","last_change":"2024-09-20T18:51:08.502411+0000","last_active":"2024-09-20T18:51:40.578715+0000","last_peered":"2024-09-20T18:51:40.578715+0000","last_clean":"2024-09-20T18:51:40.578715+0000","last_became_active":"2024-09-20T18:49:47.950939+0000","last_became_peered":"2024-09-20T18:49:47.950939+0000","last_unstale":"2024-09-20T18:51:40.578715+0000","last_undegraded":"2024-09-20T18:51:40.578715+0000","last_fullsized":"2024-09-20T18:51:40.578715+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:51:08.502315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:08.502315+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:22.323106+0000","scrub_duration":47,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":101,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.591103+0000","last_change":"2024-09-20T18:51:17.248705+0000","last_active":"2024-09-20T18:51:40.591103+0000","last_peered":"2024-09-20T18:51:40.591103+0000","last_clean":"2024-09-20T18:51:40.591103+0000","last_became_active":"2024-09-20T18:49:49.958518+0000","last_became_peered":"2024-09-20T18:49:49.958518+0000","last_unstale":"2024-09-20T18:51:40.591103+0000","last_undegraded":"2024-09-20T18:51:40.591103+0000","last_fullsized":"2024-09-20T18:51:40.591103+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:17.248634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:17.248634+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:32.434240+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":89,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.585190+0000","last_change":"2024-09-20T18:49:52.140362+0000","last_active":"2024-09-20T18:51:40.585190+0000","last_peered":"2024-09-20T18:51:40.585190+0000","last_clean":"2024-09-20T18:51:40.585190+0000","last_became_active":"2024-09-20T18:49:52.140040+0000","last_became_peered":"2024-09-20T18:49:52.140040+0000","last_unstale":"2024-09-20T18:51:40.585190+0000","last_undegraded":"2024-09-20T18:51:40.585190+0000","last_fullsized":"2024-09-20T18:51:40.585190+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:53.831495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"59'1","reported_seq":107,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.583242+0000","last_change":"2024-09-20T18:49:45.942193+0000","last_active":"2024-09-20T18:51:40.583242+0000","last_peered":"2024-09-20T18:51:40.583242+0000","last_clean":"2024-09-20T18:51:40.583242+0000","last_became_active":"2024-09-20T18:49:45.940100+0000","last_became_peered":"2024-09-20T18:49:45.940100+0000","last_unstale":"2024-09-20T18:51:40.583242+0000","last_undegraded":"2024-09-20T18:51:40.583242+0000","last_fullsized":"2024-09-20T18:51:40.583242+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:53.263392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":120,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.584552+0000","last_change":"2024-09-20T18:49:47.951215+0000","last_active":"2024-09-20T18:51:40.584552+0000","last_peered":"2024-09-20T18:51:40.584552+0000","last_clean":"2024-09-20T18:51:40.584552+0000","last_became_active":"2024-09-20T18:49:47.950909+0000","last_became_peered":"2024-09-20T18:49:47.950909+0000","last_unstale":"2024-09-20T18:51:40.584552+0000","last_undegraded":"2024-09-20T18:51:40.584552+0000","last_fullsized":"2024-09-20T18:51:40.584552+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:52.850095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":101,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.582232+0000","last_change":"2024-09-20T18:51:27.480617+0000","last_active":"2024-09-20T18:51:40.582232+0000","last_peered":"2024-09-20T18:51:40.582232+0000","last_clean":"2024-09-20T18:51:40.582232+0000","last_became_active":"2024-09-20T18:49:49.959102+0000","last_became_peered":"2024-09-20T18:49:49.959102+0000","last_unstale":"2024-09-20T18:51:40.582232+0000","last_undegraded":"2024-09-20T18:51:40.582232+0000","last_fullsized":"2024-09-20T18:51:40.582232+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:27.480405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:27.480405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:51.716129+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":97,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.586829+0000","last_change":"2024-09-20T18:51:08.725493+0000","last_active":"2024-09-20T18:51:40.586829+0000","last_peered":"2024-09-20T18:51:40.586829+0000","last_clean":"2024-09-20T18:51:40.586829+0000","last_became_active":"2024-09-20T18:49:51.970058+0000","last_became_peered":"2024-09-20T18:49:51.970058+0000","last_unstale":"2024-09-20T18:51:40.586829+0000","last_undegraded":"2024-09-20T18:51:40.586829+0000","last_fullsized":"2024-09-20T18:51:40.586829+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T18:51:08.725377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:08.725377+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:08.903104+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":102,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.590107+0000","last_change":"2024-09-20T18:49:45.933354+0000","last_active":"2024-09-20T18:51:40.590107+0000","last_peered":"2024-09-20T18:51:40.590107+0000","last_clean":"2024-09-20T18:51:40.590107+0000","last_became_active":"2024-09-20T18:49:45.933078+0000","last_became_peered":"2024-09-20T18:49:45.933078+0000","last_unstale":"2024-09-20T18:51:40.590107+0000","last_undegraded":"2024-09-20T18:51:40.590107+0000","last_fullsized":"2024-09-20T18:51:40.590107+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":111,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.597505+0000","last_change":"2024-09-20T18:49:48.501151+0000","last_active":"2024-09-20T18:51:40.597505+0000","last_peered":"2024-09-20T18:51:40.597505+0000","last_clean":"2024-09-20T18:51:40.597505+0000","last_became_active":"2024-09-20T18:49:48.500809+0000","last_became_peered":"2024-09-20T18:49:48.500809+0000","last_unstale":"2024-09-20T18:51:40.597505+0000","last_undegraded":"2024-09-20T18:51:40.597505+0000","last_fullsized":"2024-09-20T18:51:40.597505+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:35.210820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":127,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.585432+0000","last_change":"2024-09-20T18:49:49.955074+0000","last_active":"2024-09-20T18:51:40.585432+0000","last_peered":"2024-09-20T18:51:40.585432+0000","last_clean":"2024-09-20T18:51:40.585432+0000","last_became_active":"2024-09-20T18:49:49.954772+0000","last_became_peered":"2024-09-20T18:49:49.954772+0000","last_unstale":"2024-09-20T18:51:40.585432+0000","last_undegraded":"2024-09-20T18:51:40.585432+0000","last_fullsized":"2024-09-20T18:51:40.585432+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:02.439161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.c","version":"104'5077","reported_seq":5129,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:45.722307+0000","last_change":"2024-09-20T18:50:49.844070+0000","last_active":"2024-09-20T18:51:45.722307+0000","last_peered":"2024-09-20T18:51:45.722307+0000","last_clean":"2024-09-20T18:51:45.722307+0000","last_became_active":"2024-09-20T18:50:49.843744+0000","last_became_peered":"2024-09-20T18:50:49.843744+0000","last_unstale":"2024-09-20T18:51:45.722307+0000","last_undegraded":"2024-09-20T18:51:45.722307+0000","last_fullsized":"2024-09-20T18:51:45.722307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.819660+0000","objects_scrubbed":0,"log_size":5077,"log_dups_size":0,"ondisk_log_size":5077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:57.802543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":332726272,"num_objects":5077,"num_object_clones":0,"num_object_copies":15231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5077,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5077,"num_write_kb":324928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":106,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.578949+0000","last_change":"2024-09-20T18:51:21.612067+0000","last_active":"2024-09-20T18:51:40.578949+0000","last_peered":"2024-09-20T18:51:40.578949+0000","last_clean":"2024-09-20T18:51:40.578949+0000","last_became_active":"2024-09-20T18:49:58.992302+0000","last_became_peered":"2024-09-20T18:49:58.992302+0000","last_unstale":"2024-09-20T18:51:40.578949+0000","last_undegraded":"2024-09-20T18:51:40.578949+0000","last_fullsized":"2024-09-20T18:51:40.578949+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:51:21.611976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:21.611976+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:28.034177+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":101,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.584563+0000","last_change":"2024-09-20T18:49:45.936413+0000","last_active":"2024-09-20T18:51:40.584563+0000","last_peered":"2024-09-20T18:51:40.584563+0000","last_clean":"2024-09-20T18:51:40.584563+0000","last_became_active":"2024-09-20T18:49:45.936104+0000","last_became_peered":"2024-09-20T18:49:45.936104+0000","last_unstale":"2024-09-20T18:51:40.584563+0000","last_undegraded":"2024-09-20T18:51:40.584563+0000","last_fullsized":"2024-09-20T18:51:40.584563+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:03.039679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":136,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.582304+0000","last_change":"2024-09-20T18:51:13.395980+0000","last_active":"2024-09-20T18:51:40.582304+0000","last_peered":"2024-09-20T18:51:40.582304+0000","last_clean":"2024-09-20T18:51:40.582304+0000","last_became_active":"2024-09-20T18:49:49.960697+0000","last_became_peered":"2024-09-20T18:49:49.960697+0000","last_unstale":"2024-09-20T18:51:40.582304+0000","last_undegraded":"2024-09-20T18:51:40.582304+0000","last_fullsized":"2024-09-20T18:51:40.582304+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:51:13.395908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:13.395908+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:41.039582+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":88,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.587815+0000","last_change":"2024-09-20T18:49:51.971713+0000","last_active":"2024-09-20T18:51:40.587815+0000","last_peered":"2024-09-20T18:51:40.587815+0000","last_clean":"2024-09-20T18:51:40.587815+0000","last_became_active":"2024-09-20T18:49:51.971399+0000","last_became_peered":"2024-09-20T18:49:51.971399+0000","last_unstale":"2024-09-20T18:51:40.587815+0000","last_undegraded":"2024-09-20T18:51:40.587815+0000","last_fullsized":"2024-09-20T18:51:40.587815+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"104'4966","reported_seq":5019,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:45.814668+0000","last_change":"2024-09-20T18:50:49.841111+0000","last_active":"2024-09-20T18:51:45.814668+0000","last_peered":"2024-09-20T18:51:45.814668+0000","last_clean":"2024-09-20T18:51:45.814668+0000","last_became_active":"2024-09-20T18:50:49.840819+0000","last_became_peered":"2024-09-20T18:50:49.840819+0000","last_unstale":"2024-09-20T18:51:45.814668+0000","last_undegraded":"2024-09-20T18:51:45.814668+0000","last_fullsized":"2024-09-20T18:51:45.814668+0000","mapping_epoch":83,"log_start":"89'400","ondisk_log_start":"89'400","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.819660+0000","objects_scrubbed":0,"log_size":4566,"log_dups_size":0,"ondisk_log_size":4566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:18.306766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":325451776,"num_objects":4966,"num_object_clones":0,"num_object_copies":14898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4966,"num_write_kb":317824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":129,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.584343+0000","last_change":"2024-09-20T18:49:48.499569+0000","last_active":"2024-09-20T18:51:40.584343+0000","last_peered":"2024-09-20T18:51:40.584343+0000","last_clean":"2024-09-20T18:51:40.584343+0000","last_became_active":"2024-09-20T18:49:48.499189+0000","last_became_peered":"2024-09-20T18:49:48.499189+0000","last_unstale":"2024-09-20T18:51:40.584343+0000","last_undegraded":"2024-09-20T18:51:40.584343+0000","last_fullsized":"2024-09-20T18:51:40.584343+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:37.118333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":101,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.585592+0000","last_change":"2024-09-20T18:49:45.940816+0000","last_active":"2024-09-20T18:51:40.585592+0000","last_peered":"2024-09-20T18:51:40.585592+0000","last_clean":"2024-09-20T18:51:40.585592+0000","last_became_active":"2024-09-20T18:49:45.940544+0000","last_became_peered":"2024-09-20T18:49:45.940544+0000","last_unstale":"2024-09-20T18:51:40.585592+0000","last_undegraded":"2024-09-20T18:51:40.585592+0000","last_fullsized":"2024-09-20T18:51:40.585592+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:50.841381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":101,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.582957+0000","last_change":"2024-09-20T18:51:17.391115+0000","last_active":"2024-09-20T18:51:40.582957+0000","last_peered":"2024-09-20T18:51:40.582957+0000","last_clean":"2024-09-20T18:51:40.582957+0000","last_became_active":"2024-09-20T18:49:49.963073+0000","last_became_peered":"2024-09-20T18:49:49.963073+0000","last_unstale":"2024-09-20T18:51:40.582957+0000","last_undegraded":"2024-09-20T18:51:40.582957+0000","last_fullsized":"2024-09-20T18:51:40.582957+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:17.391069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:17.391069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:47.248134+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":97,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.583041+0000","last_change":"2024-09-20T18:51:15.372816+0000","last_active":"2024-09-20T18:51:40.583041+0000","last_peered":"2024-09-20T18:51:40.583041+0000","last_clean":"2024-09-20T18:51:40.583041+0000","last_became_active":"2024-09-20T18:49:51.968793+0000","last_became_peered":"2024-09-20T18:49:51.968793+0000","last_unstale":"2024-09-20T18:51:40.583041+0000","last_undegraded":"2024-09-20T18:51:40.583041+0000","last_fullsized":"2024-09-20T18:51:40.583041+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:15.372764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:15.372764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:18.362037+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2","version":"104'5072","reported_seq":5124,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:45.799332+0000","last_change":"2024-09-20T18:50:49.842736+0000","last_active":"2024-09-20T18:51:45.799332+0000","last_peered":"2024-09-20T18:51:45.799332+0000","last_clean":"2024-09-20T18:51:45.799332+0000","last_became_active":"2024-09-20T18:50:49.842465+0000","last_became_peered":"2024-09-20T18:50:49.842465+0000","last_unstale":"2024-09-20T18:51:45.799332+0000","last_undegraded":"2024-09-20T18:51:45.799332+0000","last_fullsized":"2024-09-20T18:51:45.799332+0000","mapping_epoch":83,"log_start":"89'500","ondisk_log_start":"89'500","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.819660+0000","objects_scrubbed":0,"log_size":4572,"log_dups_size":0,"ondisk_log_size":4572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:10.093828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":332398592,"num_objects":5072,"num_object_clones":0,"num_object_copies":15216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5072,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5072,"num_write_kb":324608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":126,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.582976+0000","last_change":"2024-09-20T18:51:28.496707+0000","last_active":"2024-09-20T18:51:40.582976+0000","last_peered":"2024-09-20T18:51:40.582976+0000","last_clean":"2024-09-20T18:51:40.582976+0000","last_became_active":"2024-09-20T18:49:47.941895+0000","last_became_peered":"2024-09-20T18:49:47.941895+0000","last_unstale":"2024-09-20T18:51:40.582976+0000","last_undegraded":"2024-09-20T18:51:40.582976+0000","last_fullsized":"2024-09-20T18:51:40.582976+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-20T18:51:28.496632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:28.496632+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:34.748131+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":109,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.598032+0000","last_change":"2024-09-20T18:51:27.187060+0000","last_active":"2024-09-20T18:51:40.598032+0000","last_peered":"2024-09-20T18:51:40.598032+0000","last_clean":"2024-09-20T18:51:40.598032+0000","last_became_active":"2024-09-20T18:49:45.940469+0000","last_became_peered":"2024-09-20T18:49:45.940469+0000","last_unstale":"2024-09-20T18:51:40.598032+0000","last_undegraded":"2024-09-20T18:51:40.598032+0000","last_fullsized":"2024-09-20T18:51:40.598032+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:51:27.186991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:27.186991+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:46.267710+0000","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"66'8","reported_seq":133,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.584165+0000","last_change":"2024-09-20T18:51:10.158452+0000","last_active":"2024-09-20T18:51:40.584165+0000","last_peered":"2024-09-20T18:51:40.584165+0000","last_clean":"2024-09-20T18:51:40.584165+0000","last_became_active":"2024-09-20T18:49:49.956133+0000","last_became_peered":"2024-09-20T18:49:49.956133+0000","last_unstale":"2024-09-20T18:51:40.584165+0000","last_undegraded":"2024-09-20T18:51:40.584165+0000","last_fullsized":"2024-09-20T18:51:40.584165+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:51:10.158379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:10.158379+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:26.531819+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":96,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.587562+0000","last_change":"2024-09-20T18:51:24.901023+0000","last_active":"2024-09-20T18:51:40.587562+0000","last_peered":"2024-09-20T18:51:40.587562+0000","last_clean":"2024-09-20T18:51:40.587562+0000","last_became_active":"2024-09-20T18:49:52.142113+0000","last_became_peered":"2024-09-20T18:49:52.142113+0000","last_unstale":"2024-09-20T18:51:40.587562+0000","last_undegraded":"2024-09-20T18:51:40.587562+0000","last_fullsized":"2024-09-20T18:51:40.587562+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:24.900973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:24.900973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:32.258760+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"104'5059","reported_seq":5112,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:45.757536+0000","last_change":"2024-09-20T18:50:49.845003+0000","last_active":"2024-09-20T18:51:45.757536+0000","last_peered":"2024-09-20T18:51:45.757536+0000","last_clean":"2024-09-20T18:51:45.757536+0000","last_became_active":"2024-09-20T18:50:49.844624+0000","last_became_peered":"2024-09-20T18:50:49.844624+0000","last_unstale":"2024-09-20T18:51:45.757536+0000","last_undegraded":"2024-09-20T18:51:45.757536+0000","last_fullsized":"2024-09-20T18:51:45.757536+0000","mapping_epoch":83,"log_start":"89'200","ondisk_log_start":"89'200","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.819660+0000","objects_scrubbed":0,"log_size":4859,"log_dups_size":0,"ondisk_log_size":4859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:50.608277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":331546624,"num_objects":5059,"num_object_clones":0,"num_object_copies":15177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5059,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5059,"num_write_kb":323776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":120,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.588383+0000","last_change":"2024-09-20T18:51:09.292083+0000","last_active":"2024-09-20T18:51:40.588383+0000","last_peered":"2024-09-20T18:51:40.588383+0000","last_clean":"2024-09-20T18:51:40.588383+0000","last_became_active":"2024-09-20T18:49:47.949626+0000","last_became_peered":"2024-09-20T18:49:47.949626+0000","last_unstale":"2024-09-20T18:51:40.588383+0000","last_undegraded":"2024-09-20T18:51:40.588383+0000","last_fullsized":"2024-09-20T18:51:40.588383+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:51:09.291872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:09.291872+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:38.737937+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":100,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.588344+0000","last_change":"2024-09-20T18:49:45.942852+0000","last_active":"2024-09-20T18:51:40.588344+0000","last_peered":"2024-09-20T18:51:40.588344+0000","last_clean":"2024-09-20T18:51:40.588344+0000","last_became_active":"2024-09-20T18:49:45.942496+0000","last_became_peered":"2024-09-20T18:49:45.942496+0000","last_unstale":"2024-09-20T18:51:40.588344+0000","last_undegraded":"2024-09-20T18:51:40.588344+0000","last_fullsized":"2024-09-20T18:51:40.588344+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:04.785738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":132,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.598088+0000","last_change":"2024-09-20T18:51:16.088407+0000","last_active":"2024-09-20T18:51:40.598088+0000","last_peered":"2024-09-20T18:51:40.598088+0000","last_clean":"2024-09-20T18:51:40.598088+0000","last_became_active":"2024-09-20T18:49:49.959365+0000","last_became_peered":"2024-09-20T18:49:49.959365+0000","last_unstale":"2024-09-20T18:51:40.598088+0000","last_undegraded":"2024-09-20T18:51:40.598088+0000","last_fullsized":"2024-09-20T18:51:40.598088+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:51:16.088343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:16.088343+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:32.644283+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":88,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.594819+0000","last_change":"2024-09-20T18:49:51.965733+0000","last_active":"2024-09-20T18:51:40.594819+0000","last_peered":"2024-09-20T18:51:40.594819+0000","last_clean":"2024-09-20T18:51:40.594819+0000","last_became_active":"2024-09-20T18:49:51.964780+0000","last_became_peered":"2024-09-20T18:49:51.964780+0000","last_unstale":"2024-09-20T18:51:40.594819+0000","last_undegraded":"2024-09-20T18:51:40.594819+0000","last_fullsized":"2024-09-20T18:51:40.594819+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.0","version":"104'5128","reported_seq":5180,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:45.723512+0000","last_change":"2024-09-20T18:50:49.841428+0000","last_active":"2024-09-20T18:51:45.723512+0000","last_peered":"2024-09-20T18:51:45.723512+0000","last_clean":"2024-09-20T18:51:45.723512+0000","last_became_active":"2024-09-20T18:50:49.841019+0000","last_became_peered":"2024-09-20T18:50:49.841019+0000","last_unstale":"2024-09-20T18:51:45.723512+0000","last_undegraded":"2024-09-20T18:51:45.723512+0000","last_fullsized":"2024-09-20T18:51:45.723512+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.819660+0000","objects_scrubbed":0,"log_size":5128,"log_dups_size":0,"ondisk_log_size":5128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:05.804622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":336068608,"num_objects":5128,"num_object_clones":0,"num_object_copies":15384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5128,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5128,"num_write_kb":328192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":132,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.598227+0000","last_change":"2024-09-20T18:49:48.498141+0000","last_active":"2024-09-20T18:51:40.598227+0000","last_peered":"2024-09-20T18:51:40.598227+0000","last_clean":"2024-09-20T18:51:40.598227+0000","last_became_active":"2024-09-20T18:49:48.497736+0000","last_became_peered":"2024-09-20T18:49:48.497736+0000","last_unstale":"2024-09-20T18:51:40.598227+0000","last_undegraded":"2024-09-20T18:51:40.598227+0000","last_fullsized":"2024-09-20T18:51:40.598227+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":101,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.589869+0000","last_change":"2024-09-20T18:49:45.942589+0000","last_active":"2024-09-20T18:51:40.589869+0000","last_peered":"2024-09-20T18:51:40.589869+0000","last_clean":"2024-09-20T18:51:40.589869+0000","last_became_active":"2024-09-20T18:49:45.942253+0000","last_became_peered":"2024-09-20T18:49:45.942253+0000","last_unstale":"2024-09-20T18:51:40.589869+0000","last_undegraded":"2024-09-20T18:51:40.589869+0000","last_fullsized":"2024-09-20T18:51:40.589869+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":92,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.584403+0000","last_change":"2024-09-20T18:49:49.954594+0000","last_active":"2024-09-20T18:51:40.584403+0000","last_peered":"2024-09-20T18:51:40.584403+0000","last_clean":"2024-09-20T18:51:40.584403+0000","last_became_active":"2024-09-20T18:49:49.954299+0000","last_became_peered":"2024-09-20T18:49:49.954299+0000","last_unstale":"2024-09-20T18:51:40.584403+0000","last_undegraded":"2024-09-20T18:51:40.584403+0000","last_fullsized":"2024-09-20T18:51:40.584403+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:01.406572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":97,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.589844+0000","last_change":"2024-09-20T18:51:13.293682+0000","last_active":"2024-09-20T18:51:40.589844+0000","last_peered":"2024-09-20T18:51:40.589844+0000","last_clean":"2024-09-20T18:51:40.589844+0000","last_became_active":"2024-09-20T18:49:52.139981+0000","last_became_peered":"2024-09-20T18:49:52.139981+0000","last_unstale":"2024-09-20T18:51:40.589844+0000","last_undegraded":"2024-09-20T18:51:40.589844+0000","last_fullsized":"2024-09-20T18:51:40.589844+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:13.293619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:13.293619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:13.954706+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"104'4990","reported_seq":5042,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:45.745840+0000","last_change":"2024-09-20T18:50:49.842749+0000","last_active":"2024-09-20T18:51:45.745840+0000","last_peered":"2024-09-20T18:51:45.745840+0000","last_clean":"2024-09-20T18:51:45.745840+0000","last_became_active":"2024-09-20T18:50:49.842473+0000","last_became_peered":"2024-09-20T18:50:49.842473+0000","last_unstale":"2024-09-20T18:51:45.745840+0000","last_undegraded":"2024-09-20T18:51:45.745840+0000","last_fullsized":"2024-09-20T18:51:45.745840+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.819660+0000","objects_scrubbed":0,"log_size":4990,"log_dups_size":0,"ondisk_log_size":4990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:57.122239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327024640,"num_objects":4990,"num_object_clones":0,"num_object_copies":14970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4990,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4990,"num_write_kb":319360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"66'28","reported_seq":169,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.589771+0000","last_change":"2024-09-20T18:49:47.955615+0000","last_active":"2024-09-20T18:51:40.589771+0000","last_peered":"2024-09-20T18:51:40.589771+0000","last_clean":"2024-09-20T18:51:40.589771+0000","last_became_active":"2024-09-20T18:49:47.955335+0000","last_became_peered":"2024-09-20T18:49:47.955335+0000","last_unstale":"2024-09-20T18:51:40.589771+0000","last_undegraded":"2024-09-20T18:51:40.589771+0000","last_fullsized":"2024-09-20T18:51:40.589771+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:56.998307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":101,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.582611+0000","last_change":"2024-09-20T18:49:45.945275+0000","last_active":"2024-09-20T18:51:40.582611+0000","last_peered":"2024-09-20T18:51:40.582611+0000","last_clean":"2024-09-20T18:51:40.582611+0000","last_became_active":"2024-09-20T18:49:45.944992+0000","last_became_peered":"2024-09-20T18:49:45.944992+0000","last_unstale":"2024-09-20T18:51:40.582611+0000","last_undegraded":"2024-09-20T18:51:40.582611+0000","last_fullsized":"2024-09-20T18:51:40.582611+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:50.184705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":118,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.587514+0000","last_change":"2024-09-20T18:50:48.711466+0000","last_active":"2024-09-20T18:51:40.587514+0000","last_peered":"2024-09-20T18:51:40.587514+0000","last_clean":"2024-09-20T18:51:40.587514+0000","last_became_active":"2024-09-20T18:49:44.936027+0000","last_became_peered":"2024-09-20T18:49:44.936027+0000","last_unstale":"2024-09-20T18:51:40.587514+0000","last_undegraded":"2024-09-20T18:51:40.587514+0000","last_fullsized":"2024-09-20T18:51:40.587514+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-20T18:50:48.711416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.711416+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:07.241951+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.0012781159999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":92,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.594867+0000","last_change":"2024-09-20T18:49:49.954281+0000","last_active":"2024-09-20T18:51:40.594867+0000","last_peered":"2024-09-20T18:51:40.594867+0000","last_clean":"2024-09-20T18:51:40.594867+0000","last_became_active":"2024-09-20T18:49:49.953971+0000","last_became_peered":"2024-09-20T18:49:49.953971+0000","last_unstale":"2024-09-20T18:51:40.594867+0000","last_undegraded":"2024-09-20T18:51:40.594867+0000","last_fullsized":"2024-09-20T18:51:40.594867+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":89,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.585942+0000","last_change":"2024-09-20T18:49:51.969702+0000","last_active":"2024-09-20T18:51:40.585942+0000","last_peered":"2024-09-20T18:51:40.585942+0000","last_clean":"2024-09-20T18:51:40.585942+0000","last_became_active":"2024-09-20T18:49:51.969400+0000","last_became_peered":"2024-09-20T18:49:51.969400+0000","last_unstale":"2024-09-20T18:51:40.585942+0000","last_undegraded":"2024-09-20T18:51:40.585942+0000","last_fullsized":"2024-09-20T18:51:40.585942+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:33.201421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"104'4996","reported_seq":5048,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:45.820142+0000","last_change":"2024-09-20T18:50:49.843571+0000","last_active":"2024-09-20T18:51:45.820142+0000","last_peered":"2024-09-20T18:51:45.820142+0000","last_clean":"2024-09-20T18:51:45.820142+0000","last_became_active":"2024-09-20T18:50:49.843321+0000","last_became_peered":"2024-09-20T18:50:49.843321+0000","last_unstale":"2024-09-20T18:51:45.820142+0000","last_undegraded":"2024-09-20T18:51:45.820142+0000","last_fullsized":"2024-09-20T18:51:45.820142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.819660+0000","objects_scrubbed":0,"log_size":4996,"log_dups_size":0,"ondisk_log_size":4996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:13.806482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327417856,"num_objects":4996,"num_object_clones":0,"num_object_copies":14988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4996,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4996,"num_write_kb":319744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":124,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.588888+0000","last_change":"2024-09-20T18:49:47.951311+0000","last_active":"2024-09-20T18:51:40.588888+0000","last_peered":"2024-09-20T18:51:40.588888+0000","last_clean":"2024-09-20T18:51:40.588888+0000","last_became_active":"2024-09-20T18:49:47.950904+0000","last_became_peered":"2024-09-20T18:49:47.950904+0000","last_unstale":"2024-09-20T18:51:40.588888+0000","last_undegraded":"2024-09-20T18:51:40.588888+0000","last_fullsized":"2024-09-20T18:51:40.588888+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:35.206639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":101,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.588138+0000","last_change":"2024-09-20T18:49:45.940667+0000","last_active":"2024-09-20T18:51:40.588138+0000","last_peered":"2024-09-20T18:51:40.588138+0000","last_clean":"2024-09-20T18:51:40.588138+0000","last_became_active":"2024-09-20T18:49:45.939884+0000","last_became_peered":"2024-09-20T18:49:45.939884+0000","last_unstale":"2024-09-20T18:51:40.588138+0000","last_undegraded":"2024-09-20T18:51:40.588138+0000","last_fullsized":"2024-09-20T18:51:40.588138+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:00.025281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":112,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.578342+0000","last_change":"2024-09-20T18:49:47.939858+0000","last_active":"2024-09-20T18:51:40.578342+0000","last_peered":"2024-09-20T18:51:40.578342+0000","last_clean":"2024-09-20T18:51:40.578342+0000","last_became_active":"2024-09-20T18:49:44.937135+0000","last_became_peered":"2024-09-20T18:49:44.937135+0000","last_unstale":"2024-09-20T18:51:40.578342+0000","last_undegraded":"2024-09-20T18:51:40.578342+0000","last_fullsized":"2024-09-20T18:51:40.578342+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:49:43.921002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:00.480665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095558800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":94,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.581206+0000","last_change":"2024-09-20T18:49:49.954880+0000","last_active":"2024-09-20T18:51:40.581206+0000","last_peered":"2024-09-20T18:51:40.581206+0000","last_clean":"2024-09-20T18:51:40.581206+0000","last_became_active":"2024-09-20T18:49:49.954424+0000","last_became_peered":"2024-09-20T18:49:49.954424+0000","last_unstale":"2024-09-20T18:51:40.581206+0000","last_undegraded":"2024-09-20T18:51:40.581206+0000","last_fullsized":"2024-09-20T18:51:40.581206+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:57.475800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":88,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.587274+0000","last_change":"2024-09-20T18:49:52.140366+0000","last_active":"2024-09-20T18:51:40.587274+0000","last_peered":"2024-09-20T18:51:40.587274+0000","last_clean":"2024-09-20T18:51:40.587274+0000","last_became_active":"2024-09-20T18:49:52.139789+0000","last_became_peered":"2024-09-20T18:49:52.139789+0000","last_unstale":"2024-09-20T18:51:40.587274+0000","last_undegraded":"2024-09-20T18:51:40.587274+0000","last_fullsized":"2024-09-20T18:51:40.587274+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:38.801091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"104'5080","reported_seq":5132,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:45.751381+0000","last_change":"2024-09-20T18:50:49.846470+0000","last_active":"2024-09-20T18:51:45.751381+0000","last_peered":"2024-09-20T18:51:45.751381+0000","last_clean":"2024-09-20T18:51:45.751381+0000","last_became_active":"2024-09-20T18:50:49.846124+0000","last_became_peered":"2024-09-20T18:50:49.846124+0000","last_unstale":"2024-09-20T18:51:45.751381+0000","last_undegraded":"2024-09-20T18:51:45.751381+0000","last_fullsized":"2024-09-20T18:51:45.751381+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.819660+0000","objects_scrubbed":0,"log_size":5080,"log_dups_size":0,"ondisk_log_size":5080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:58.163086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":332922880,"num_objects":5080,"num_object_clones":0,"num_object_copies":15240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5080,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5080,"num_write_kb":325120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":122,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.597834+0000","last_change":"2024-09-20T18:51:31.136569+0000","last_active":"2024-09-20T18:51:40.597834+0000","last_peered":"2024-09-20T18:51:40.597834+0000","last_clean":"2024-09-20T18:51:40.597834+0000","last_became_active":"2024-09-20T18:49:47.949230+0000","last_became_peered":"2024-09-20T18:49:47.949230+0000","last_unstale":"2024-09-20T18:51:40.597834+0000","last_undegraded":"2024-09-20T18:51:40.597834+0000","last_fullsized":"2024-09-20T18:51:40.597834+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:51:31.136463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:31.136463+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:41.956368+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":100,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.595654+0000","last_change":"2024-09-20T18:49:45.939454+0000","last_active":"2024-09-20T18:51:40.595654+0000","last_peered":"2024-09-20T18:51:40.595654+0000","last_clean":"2024-09-20T18:51:40.595654+0000","last_became_active":"2024-09-20T18:49:45.939197+0000","last_became_peered":"2024-09-20T18:49:45.939197+0000","last_unstale":"2024-09-20T18:51:40.595654+0000","last_undegraded":"2024-09-20T18:51:40.595654+0000","last_fullsized":"2024-09-20T18:51:40.595654+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":227,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:45.438317+0000","last_change":"2024-09-20T18:49:48.498983+0000","last_active":"2024-09-20T18:51:45.438317+0000","last_peered":"2024-09-20T18:51:45.438317+0000","last_clean":"2024-09-20T18:51:45.438317+0000","last_became_active":"2024-09-20T18:49:44.936819+0000","last_became_peered":"2024-09-20T18:49:44.936819+0000","last_unstale":"2024-09-20T18:51:45.438317+0000","last_undegraded":"2024-09-20T18:51:45.438317+0000","last_fullsized":"2024-09-20T18:51:45.438317+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:49:43.921002+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:05.780783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010996199999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":115,"num_read_kb":110,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":92,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.588900+0000","last_change":"2024-09-20T18:49:49.961346+0000","last_active":"2024-09-20T18:51:40.588900+0000","last_peered":"2024-09-20T18:51:40.588900+0000","last_clean":"2024-09-20T18:51:40.588900+0000","last_became_active":"2024-09-20T18:49:49.961066+0000","last_became_peered":"2024-09-20T18:49:49.961066+0000","last_unstale":"2024-09-20T18:51:40.588900+0000","last_undegraded":"2024-09-20T18:51:40.588900+0000","last_fullsized":"2024-09-20T18:51:40.588900+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:02.804079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":89,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.589453+0000","last_change":"2024-09-20T18:49:51.970609+0000","last_active":"2024-09-20T18:51:40.589453+0000","last_peered":"2024-09-20T18:51:40.589453+0000","last_clean":"2024-09-20T18:51:40.589453+0000","last_became_active":"2024-09-20T18:49:51.970343+0000","last_became_peered":"2024-09-20T18:49:51.970343+0000","last_unstale":"2024-09-20T18:51:40.589453+0000","last_undegraded":"2024-09-20T18:51:40.589453+0000","last_fullsized":"2024-09-20T18:51:40.589453+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:40.226114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"104'5118","reported_seq":5167,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:45.807934+0000","last_change":"2024-09-20T18:50:49.842714+0000","last_active":"2024-09-20T18:51:45.807934+0000","last_peered":"2024-09-20T18:51:45.807934+0000","last_clean":"2024-09-20T18:51:45.807934+0000","last_became_active":"2024-09-20T18:50:49.842310+0000","last_became_peered":"2024-09-20T18:50:49.842310+0000","last_unstale":"2024-09-20T18:51:45.807934+0000","last_undegraded":"2024-09-20T18:51:45.807934+0000","last_fullsized":"2024-09-20T18:51:45.807934+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.819660+0000","objects_scrubbed":0,"log_size":5118,"log_dups_size":0,"ondisk_log_size":5118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:55.921403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":335413248,"num_objects":5118,"num_object_clones":0,"num_object_copies":15354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5118,"num_write_kb":327552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":142,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.584782+0000","last_change":"2024-09-20T18:49:48.498345+0000","last_active":"2024-09-20T18:51:40.584782+0000","last_peered":"2024-09-20T18:51:40.584782+0000","last_clean":"2024-09-20T18:51:40.584782+0000","last_became_active":"2024-09-20T18:49:48.497936+0000","last_became_peered":"2024-09-20T18:49:48.497936+0000","last_unstale":"2024-09-20T18:51:40.584782+0000","last_undegraded":"2024-09-20T18:51:40.584782+0000","last_fullsized":"2024-09-20T18:51:40.584782+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:56.361041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":109,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.584549+0000","last_change":"2024-09-20T18:51:23.188218+0000","last_active":"2024-09-20T18:51:40.584549+0000","last_peered":"2024-09-20T18:51:40.584549+0000","last_clean":"2024-09-20T18:51:40.584549+0000","last_became_active":"2024-09-20T18:49:45.941136+0000","last_became_peered":"2024-09-20T18:49:45.941136+0000","last_unstale":"2024-09-20T18:51:40.584549+0000","last_undegraded":"2024-09-20T18:51:40.584549+0000","last_fullsized":"2024-09-20T18:51:40.584549+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:23.188155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:23.188155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:47.504912+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":148,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.588588+0000","last_change":"2024-09-20T18:50:31.593683+0000","last_active":"2024-09-20T18:51:40.588588+0000","last_peered":"2024-09-20T18:51:40.588588+0000","last_clean":"2024-09-20T18:51:40.588588+0000","last_became_active":"2024-09-20T18:49:36.545306+0000","last_became_peered":"2024-09-20T18:49:36.545306+0000","last_unstale":"2024-09-20T18:51:40.588588+0000","last_undegraded":"2024-09-20T18:51:40.588588+0000","last_fullsized":"2024-09-20T18:51:40.588588+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T18:50:31.593610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T18:50:31.593610+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:00.939267+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":100,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.586984+0000","last_change":"2024-09-20T18:51:22.852942+0000","last_active":"2024-09-20T18:51:40.586984+0000","last_peered":"2024-09-20T18:51:40.586984+0000","last_clean":"2024-09-20T18:51:40.586984+0000","last_became_active":"2024-09-20T18:49:49.956128+0000","last_became_peered":"2024-09-20T18:49:49.956128+0000","last_unstale":"2024-09-20T18:51:40.586984+0000","last_undegraded":"2024-09-20T18:51:40.586984+0000","last_fullsized":"2024-09-20T18:51:40.586984+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:22.852869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:22.852869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:27.510671+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":96,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.588720+0000","last_change":"2024-09-20T18:51:25.779876+0000","last_active":"2024-09-20T18:51:40.588720+0000","last_peered":"2024-09-20T18:51:40.588720+0000","last_clean":"2024-09-20T18:51:40.588720+0000","last_became_active":"2024-09-20T18:49:51.971332+0000","last_became_peered":"2024-09-20T18:49:51.971332+0000","last_unstale":"2024-09-20T18:51:40.588720+0000","last_undegraded":"2024-09-20T18:51:40.588720+0000","last_fullsized":"2024-09-20T18:51:40.588720+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:25.779791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:25.779791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:28.787780+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"104'5069","reported_seq":5121,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:45.743001+0000","last_change":"2024-09-20T18:50:49.842768+0000","last_active":"2024-09-20T18:51:45.743001+0000","last_peered":"2024-09-20T18:51:45.743001+0000","last_clean":"2024-09-20T18:51:45.743001+0000","last_became_active":"2024-09-20T18:50:49.842483+0000","last_became_peered":"2024-09-20T18:50:49.842483+0000","last_unstale":"2024-09-20T18:51:45.743001+0000","last_undegraded":"2024-09-20T18:51:45.743001+0000","last_fullsized":"2024-09-20T18:51:45.743001+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.819660+0000","objects_scrubbed":0,"log_size":5069,"log_dups_size":0,"ondisk_log_size":5069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:01.275928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":332201984,"num_objects":5069,"num_object_clones":0,"num_object_copies":15207,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5069,"num_write_kb":324416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":124,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.582358+0000","last_change":"2024-09-20T18:51:14.397996+0000","last_active":"2024-09-20T18:51:40.582358+0000","last_peered":"2024-09-20T18:51:40.582358+0000","last_clean":"2024-09-20T18:51:40.582358+0000","last_became_active":"2024-09-20T18:49:47.950073+0000","last_became_peered":"2024-09-20T18:49:47.950073+0000","last_unstale":"2024-09-20T18:51:40.582358+0000","last_undegraded":"2024-09-20T18:51:40.582358+0000","last_fullsized":"2024-09-20T18:51:40.582358+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:51:14.397930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.397930+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:24.627773+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":109,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.583218+0000","last_change":"2024-09-20T18:51:18.410601+0000","last_active":"2024-09-20T18:51:40.583218+0000","last_peered":"2024-09-20T18:51:40.583218+0000","last_clean":"2024-09-20T18:51:40.583218+0000","last_became_active":"2024-09-20T18:49:45.944579+0000","last_became_peered":"2024-09-20T18:49:45.944579+0000","last_unstale":"2024-09-20T18:51:40.583218+0000","last_undegraded":"2024-09-20T18:51:40.583218+0000","last_fullsized":"2024-09-20T18:51:40.583218+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:18.410447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:18.410447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:21.409424+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":100,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.588588+0000","last_change":"2024-09-20T18:51:24.752755+0000","last_active":"2024-09-20T18:51:40.588588+0000","last_peered":"2024-09-20T18:51:40.588588+0000","last_clean":"2024-09-20T18:51:40.588588+0000","last_became_active":"2024-09-20T18:49:49.954391+0000","last_became_peered":"2024-09-20T18:49:49.954391+0000","last_unstale":"2024-09-20T18:51:40.588588+0000","last_undegraded":"2024-09-20T18:51:40.588588+0000","last_fullsized":"2024-09-20T18:51:40.588588+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:24.752697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:24.752697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:31.329723+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":89,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.584621+0000","last_change":"2024-09-20T18:49:52.140252+0000","last_active":"2024-09-20T18:51:40.584621+0000","last_peered":"2024-09-20T18:51:40.584621+0000","last_clean":"2024-09-20T18:51:40.584621+0000","last_became_active":"2024-09-20T18:49:52.139959+0000","last_became_peered":"2024-09-20T18:49:52.139959+0000","last_unstale":"2024-09-20T18:51:40.584621+0000","last_undegraded":"2024-09-20T18:51:40.584621+0000","last_fullsized":"2024-09-20T18:51:40.584621+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:54.812391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"104'4996","reported_seq":5049,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:45.719032+0000","last_change":"2024-09-20T18:50:49.842002+0000","last_active":"2024-09-20T18:51:45.719032+0000","last_peered":"2024-09-20T18:51:45.719032+0000","last_clean":"2024-09-20T18:51:45.719032+0000","last_became_active":"2024-09-20T18:50:49.841683+0000","last_became_peered":"2024-09-20T18:50:49.841683+0000","last_unstale":"2024-09-20T18:51:45.719032+0000","last_undegraded":"2024-09-20T18:51:45.719032+0000","last_fullsized":"2024-09-20T18:51:45.719032+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.819660+0000","objects_scrubbed":0,"log_size":4996,"log_dups_size":0,"ondisk_log_size":4996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:13.650405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327417856,"num_objects":4996,"num_object_clones":0,"num_object_copies":14988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4996,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4996,"num_write_kb":319744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":133,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.582734+0000","last_change":"2024-09-20T18:50:48.145398+0000","last_active":"2024-09-20T18:51:40.582734+0000","last_peered":"2024-09-20T18:51:40.582734+0000","last_clean":"2024-09-20T18:51:40.582734+0000","last_became_active":"2024-09-20T18:49:47.947636+0000","last_became_peered":"2024-09-20T18:49:47.947636+0000","last_unstale":"2024-09-20T18:51:40.582734+0000","last_undegraded":"2024-09-20T18:51:40.582734+0000","last_fullsized":"2024-09-20T18:51:40.582734+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-20T18:50:48.145313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.145313+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:52.345089+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":115,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.584774+0000","last_change":"2024-09-20T18:51:09.507506+0000","last_active":"2024-09-20T18:51:40.584774+0000","last_peered":"2024-09-20T18:51:40.584774+0000","last_clean":"2024-09-20T18:51:40.584774+0000","last_became_active":"2024-09-20T18:51:04.397927+0000","last_became_peered":"2024-09-20T18:51:04.397927+0000","last_unstale":"2024-09-20T18:51:40.584774+0000","last_undegraded":"2024-09-20T18:51:40.584774+0000","last_fullsized":"2024-09-20T18:51:40.584774+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:09.507432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:09.507432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:36.618983+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":93,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.589540+0000","last_change":"2024-09-20T18:49:49.959440+0000","last_active":"2024-09-20T18:51:40.589540+0000","last_peered":"2024-09-20T18:51:40.589540+0000","last_clean":"2024-09-20T18:51:40.589540+0000","last_became_active":"2024-09-20T18:49:49.959160+0000","last_became_peered":"2024-09-20T18:49:49.959160+0000","last_unstale":"2024-09-20T18:51:40.589540+0000","last_undegraded":"2024-09-20T18:51:40.589540+0000","last_fullsized":"2024-09-20T18:51:40.589540+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":98,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.579083+0000","last_change":"2024-09-20T18:51:07.489133+0000","last_active":"2024-09-20T18:51:40.579083+0000","last_peered":"2024-09-20T18:51:40.579083+0000","last_clean":"2024-09-20T18:51:40.579083+0000","last_became_active":"2024-09-20T18:49:51.970108+0000","last_became_peered":"2024-09-20T18:49:51.970108+0000","last_unstale":"2024-09-20T18:51:40.579083+0000","last_undegraded":"2024-09-20T18:51:40.579083+0000","last_fullsized":"2024-09-20T18:51:40.579083+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:07.489061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:07.489061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:31.199401+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"104'5115","reported_seq":5167,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:45.832540+0000","last_change":"2024-09-20T18:50:49.841900+0000","last_active":"2024-09-20T18:51:45.832540+0000","last_peered":"2024-09-20T18:51:45.832540+0000","last_clean":"2024-09-20T18:51:45.832540+0000","last_became_active":"2024-09-20T18:50:49.841628+0000","last_became_peered":"2024-09-20T18:50:49.841628+0000","last_unstale":"2024-09-20T18:51:45.832540+0000","last_undegraded":"2024-09-20T18:51:45.832540+0000","last_fullsized":"2024-09-20T18:51:45.832540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.819660+0000","objects_scrubbed":0,"log_size":5115,"log_dups_size":0,"ondisk_log_size":5115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:04.408905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":335216640,"num_objects":5115,"num_object_clones":0,"num_object_copies":15345,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5115,"num_write_kb":327360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":112,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.582325+0000","last_change":"2024-09-20T18:49:47.948046+0000","last_active":"2024-09-20T18:51:40.582325+0000","last_peered":"2024-09-20T18:51:40.582325+0000","last_clean":"2024-09-20T18:51:40.582325+0000","last_became_active":"2024-09-20T18:49:47.947126+0000","last_became_peered":"2024-09-20T18:49:47.947126+0000","last_unstale":"2024-09-20T18:51:40.582325+0000","last_undegraded":"2024-09-20T18:51:40.582325+0000","last_fullsized":"2024-09-20T18:51:40.582325+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":109,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.587455+0000","last_change":"2024-09-20T18:51:16.216939+0000","last_active":"2024-09-20T18:51:40.587455+0000","last_peered":"2024-09-20T18:51:40.587455+0000","last_clean":"2024-09-20T18:51:40.587455+0000","last_became_active":"2024-09-20T18:49:45.937264+0000","last_became_peered":"2024-09-20T18:49:45.937264+0000","last_unstale":"2024-09-20T18:51:40.587455+0000","last_undegraded":"2024-09-20T18:51:40.587455+0000","last_fullsized":"2024-09-20T18:51:40.587455+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:16.216867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:16.216867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:33.874355+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":134,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.578584+0000","last_change":"2024-09-20T18:51:16.501407+0000","last_active":"2024-09-20T18:51:40.578584+0000","last_peered":"2024-09-20T18:51:40.578584+0000","last_clean":"2024-09-20T18:51:40.578584+0000","last_became_active":"2024-09-20T18:49:49.957005+0000","last_became_peered":"2024-09-20T18:49:49.957005+0000","last_unstale":"2024-09-20T18:51:40.578584+0000","last_undegraded":"2024-09-20T18:51:40.578584+0000","last_fullsized":"2024-09-20T18:51:40.578584+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:51:16.501211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:16.501211+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:39.765628+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":89,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.582061+0000","last_change":"2024-09-20T18:49:51.969633+0000","last_active":"2024-09-20T18:51:40.582061+0000","last_peered":"2024-09-20T18:51:40.582061+0000","last_clean":"2024-09-20T18:51:40.582061+0000","last_became_active":"2024-09-20T18:49:51.969355+0000","last_became_peered":"2024-09-20T18:49:51.969355+0000","last_unstale":"2024-09-20T18:51:40.582061+0000","last_undegraded":"2024-09-20T18:51:40.582061+0000","last_fullsized":"2024-09-20T18:51:40.582061+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:05.411584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.8","version":"104'5049","reported_seq":5102,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:45.780763+0000","last_change":"2024-09-20T18:50:49.844296+0000","last_active":"2024-09-20T18:51:45.780763+0000","last_peered":"2024-09-20T18:51:45.780763+0000","last_clean":"2024-09-20T18:51:45.780763+0000","last_became_active":"2024-09-20T18:50:49.844028+0000","last_became_peered":"2024-09-20T18:50:49.844028+0000","last_unstale":"2024-09-20T18:51:45.780763+0000","last_undegraded":"2024-09-20T18:51:45.780763+0000","last_fullsized":"2024-09-20T18:51:45.780763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.819660+0000","objects_scrubbed":0,"log_size":5049,"log_dups_size":0,"ondisk_log_size":5049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:07.943453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330891264,"num_objects":5049,"num_object_clones":0,"num_object_copies":15147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5049,"num_write_kb":323136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":112,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.598137+0000","last_change":"2024-09-20T18:49:47.946309+0000","last_active":"2024-09-20T18:51:40.598137+0000","last_peered":"2024-09-20T18:51:40.598137+0000","last_clean":"2024-09-20T18:51:40.598137+0000","last_became_active":"2024-09-20T18:49:47.946008+0000","last_became_peered":"2024-09-20T18:49:47.946008+0000","last_unstale":"2024-09-20T18:51:40.598137+0000","last_undegraded":"2024-09-20T18:51:40.598137+0000","last_fullsized":"2024-09-20T18:51:40.598137+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:03.927702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":108,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.588422+0000","last_change":"2024-09-20T18:50:46.750354+0000","last_active":"2024-09-20T18:51:40.588422+0000","last_peered":"2024-09-20T18:51:40.588422+0000","last_clean":"2024-09-20T18:51:40.588422+0000","last_became_active":"2024-09-20T18:49:45.941406+0000","last_became_peered":"2024-09-20T18:49:45.941406+0000","last_unstale":"2024-09-20T18:51:40.588422+0000","last_undegraded":"2024-09-20T18:51:40.588422+0000","last_fullsized":"2024-09-20T18:51:40.588422+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:50:46.750281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:50:46.750281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:07.951779+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":94,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.581673+0000","last_change":"2024-09-20T18:49:49.957141+0000","last_active":"2024-09-20T18:51:40.581673+0000","last_peered":"2024-09-20T18:51:40.581673+0000","last_clean":"2024-09-20T18:51:40.581673+0000","last_became_active":"2024-09-20T18:49:49.956767+0000","last_became_peered":"2024-09-20T18:49:49.956767+0000","last_unstale":"2024-09-20T18:51:40.581673+0000","last_undegraded":"2024-09-20T18:51:40.581673+0000","last_fullsized":"2024-09-20T18:51:40.581673+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:58.460680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":88,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.595568+0000","last_change":"2024-09-20T18:49:51.964375+0000","last_active":"2024-09-20T18:51:40.595568+0000","last_peered":"2024-09-20T18:51:40.595568+0000","last_clean":"2024-09-20T18:51:40.595568+0000","last_became_active":"2024-09-20T18:49:51.964098+0000","last_became_peered":"2024-09-20T18:49:51.964098+0000","last_unstale":"2024-09-20T18:51:40.595568+0000","last_undegraded":"2024-09-20T18:51:40.595568+0000","last_fullsized":"2024-09-20T18:51:40.595568+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"104'4969","reported_seq":5021,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:45.828519+0000","last_change":"2024-09-20T18:50:49.842558+0000","last_active":"2024-09-20T18:51:45.828519+0000","last_peered":"2024-09-20T18:51:45.828519+0000","last_clean":"2024-09-20T18:51:45.828519+0000","last_became_active":"2024-09-20T18:50:49.840023+0000","last_became_peered":"2024-09-20T18:50:49.840023+0000","last_unstale":"2024-09-20T18:51:45.828519+0000","last_undegraded":"2024-09-20T18:51:45.828519+0000","last_fullsized":"2024-09-20T18:51:45.828519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.819660+0000","objects_scrubbed":0,"log_size":4969,"log_dups_size":0,"ondisk_log_size":4969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:55.634729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":325648384,"num_objects":4969,"num_object_clones":0,"num_object_copies":14907,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4969,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4969,"num_write_kb":318016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":135,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.584531+0000","last_change":"2024-09-20T18:51:12.551897+0000","last_active":"2024-09-20T18:51:40.584531+0000","last_peered":"2024-09-20T18:51:40.584531+0000","last_clean":"2024-09-20T18:51:40.584531+0000","last_became_active":"2024-09-20T18:49:48.498625+0000","last_became_peered":"2024-09-20T18:49:48.498625+0000","last_unstale":"2024-09-20T18:51:40.584531+0000","last_undegraded":"2024-09-20T18:51:40.584531+0000","last_fullsized":"2024-09-20T18:51:40.584531+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-20T18:51:12.551804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:12.551804+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:17.559045+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":100,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.587605+0000","last_change":"2024-09-20T18:49:45.941764+0000","last_active":"2024-09-20T18:51:40.587605+0000","last_peered":"2024-09-20T18:51:40.587605+0000","last_clean":"2024-09-20T18:51:40.587605+0000","last_became_active":"2024-09-20T18:49:45.941448+0000","last_became_peered":"2024-09-20T18:49:45.941448+0000","last_unstale":"2024-09-20T18:51:40.587605+0000","last_undegraded":"2024-09-20T18:51:40.587605+0000","last_fullsized":"2024-09-20T18:51:40.587605+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:53.774098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":102,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.581293+0000","last_change":"2024-09-20T18:51:23.564776+0000","last_active":"2024-09-20T18:51:40.581293+0000","last_peered":"2024-09-20T18:51:40.581293+0000","last_clean":"2024-09-20T18:51:40.581293+0000","last_became_active":"2024-09-20T18:49:49.957558+0000","last_became_peered":"2024-09-20T18:49:49.957558+0000","last_unstale":"2024-09-20T18:51:40.581293+0000","last_undegraded":"2024-09-20T18:51:40.581293+0000","last_fullsized":"2024-09-20T18:51:40.581293+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:23.564416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:23.564416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:34.630995+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":96,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.588465+0000","last_change":"2024-09-20T18:51:23.858963+0000","last_active":"2024-09-20T18:51:40.588465+0000","last_peered":"2024-09-20T18:51:40.588465+0000","last_clean":"2024-09-20T18:51:40.588465+0000","last_became_active":"2024-09-20T18:49:51.970798+0000","last_became_peered":"2024-09-20T18:49:51.970798+0000","last_unstale":"2024-09-20T18:51:40.588465+0000","last_undegraded":"2024-09-20T18:51:40.588465+0000","last_fullsized":"2024-09-20T18:51:40.588465+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:23.858872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:23.858872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:27.732293+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.a","version":"104'5050","reported_seq":5103,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:45.799150+0000","last_change":"2024-09-20T18:50:49.842445+0000","last_active":"2024-09-20T18:51:45.799150+0000","last_peered":"2024-09-20T18:51:45.799150+0000","last_clean":"2024-09-20T18:51:45.799150+0000","last_became_active":"2024-09-20T18:50:49.842179+0000","last_became_peered":"2024-09-20T18:50:49.842179+0000","last_unstale":"2024-09-20T18:51:45.799150+0000","last_undegraded":"2024-09-20T18:51:45.799150+0000","last_fullsized":"2024-09-20T18:51:45.799150+0000","mapping_epoch":83,"log_start":"90'500","ondisk_log_start":"90'500","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.819660+0000","objects_scrubbed":0,"log_size":4550,"log_dups_size":0,"ondisk_log_size":4550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:52.881363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330956800,"num_objects":5050,"num_object_clones":0,"num_object_copies":15150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5050,"num_write_kb":323200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":120,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.583108+0000","last_change":"2024-09-20T18:49:47.949428+0000","last_active":"2024-09-20T18:51:40.583108+0000","last_peered":"2024-09-20T18:51:40.583108+0000","last_clean":"2024-09-20T18:51:40.583108+0000","last_became_active":"2024-09-20T18:49:47.949116+0000","last_became_peered":"2024-09-20T18:49:47.949116+0000","last_unstale":"2024-09-20T18:51:40.583108+0000","last_undegraded":"2024-09-20T18:51:40.583108+0000","last_fullsized":"2024-09-20T18:51:40.583108+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:36.577089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":100,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.587905+0000","last_change":"2024-09-20T18:49:45.945147+0000","last_active":"2024-09-20T18:51:40.587905+0000","last_peered":"2024-09-20T18:51:40.587905+0000","last_clean":"2024-09-20T18:51:40.587905+0000","last_became_active":"2024-09-20T18:49:45.944858+0000","last_became_peered":"2024-09-20T18:49:45.944858+0000","last_unstale":"2024-09-20T18:51:40.587905+0000","last_undegraded":"2024-09-20T18:51:40.587905+0000","last_fullsized":"2024-09-20T18:51:40.587905+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:55.785984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":94,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.579390+0000","last_change":"2024-09-20T18:49:49.955243+0000","last_active":"2024-09-20T18:51:40.579390+0000","last_peered":"2024-09-20T18:51:40.579390+0000","last_clean":"2024-09-20T18:51:40.579390+0000","last_became_active":"2024-09-20T18:49:49.954944+0000","last_became_peered":"2024-09-20T18:49:49.954944+0000","last_unstale":"2024-09-20T18:51:40.579390+0000","last_undegraded":"2024-09-20T18:51:40.579390+0000","last_fullsized":"2024-09-20T18:51:40.579390+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:06.459420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":89,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.587201+0000","last_change":"2024-09-20T18:49:51.973284+0000","last_active":"2024-09-20T18:51:40.587201+0000","last_peered":"2024-09-20T18:51:40.587201+0000","last_clean":"2024-09-20T18:51:40.587201+0000","last_became_active":"2024-09-20T18:49:51.972970+0000","last_became_peered":"2024-09-20T18:49:51.972970+0000","last_unstale":"2024-09-20T18:51:40.587201+0000","last_undegraded":"2024-09-20T18:51:40.587201+0000","last_fullsized":"2024-09-20T18:51:40.587201+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:57.894840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"104'5049","reported_seq":5102,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:45.756098+0000","last_change":"2024-09-20T18:50:49.843796+0000","last_active":"2024-09-20T18:51:45.756098+0000","last_peered":"2024-09-20T18:51:45.756098+0000","last_clean":"2024-09-20T18:51:45.756098+0000","last_became_active":"2024-09-20T18:50:49.843516+0000","last_became_peered":"2024-09-20T18:50:49.843516+0000","last_unstale":"2024-09-20T18:51:45.756098+0000","last_undegraded":"2024-09-20T18:51:45.756098+0000","last_fullsized":"2024-09-20T18:51:45.756098+0000","mapping_epoch":83,"log_start":"89'100","ondisk_log_start":"89'100","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.819660+0000","objects_scrubbed":0,"log_size":4949,"log_dups_size":0,"ondisk_log_size":4949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:55.068715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330891264,"num_objects":5049,"num_object_clones":0,"num_object_copies":15147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5049,"num_write_kb":323136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":121,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.586558+0000","last_change":"2024-09-20T18:49:48.500174+0000","last_active":"2024-09-20T18:51:40.586558+0000","last_peered":"2024-09-20T18:51:40.586558+0000","last_clean":"2024-09-20T18:51:40.586558+0000","last_became_active":"2024-09-20T18:49:48.499847+0000","last_became_peered":"2024-09-20T18:49:48.499847+0000","last_unstale":"2024-09-20T18:51:40.586558+0000","last_undegraded":"2024-09-20T18:51:40.586558+0000","last_fullsized":"2024-09-20T18:51:40.586558+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:34.702829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":112,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.588707+0000","last_change":"2024-09-20T18:49:45.945391+0000","last_active":"2024-09-20T18:51:40.588707+0000","last_peered":"2024-09-20T18:51:40.588707+0000","last_clean":"2024-09-20T18:51:40.588707+0000","last_became_active":"2024-09-20T18:49:45.944256+0000","last_became_peered":"2024-09-20T18:49:45.944256+0000","last_unstale":"2024-09-20T18:51:40.588707+0000","last_undegraded":"2024-09-20T18:51:40.588707+0000","last_fullsized":"2024-09-20T18:51:40.588707+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:54.747555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":124,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.586993+0000","last_change":"2024-09-20T18:49:49.958686+0000","last_active":"2024-09-20T18:51:40.586993+0000","last_peered":"2024-09-20T18:51:40.586993+0000","last_clean":"2024-09-20T18:51:40.586993+0000","last_became_active":"2024-09-20T18:49:49.958411+0000","last_became_peered":"2024-09-20T18:49:49.958411+0000","last_unstale":"2024-09-20T18:51:40.586993+0000","last_undegraded":"2024-09-20T18:51:40.586993+0000","last_fullsized":"2024-09-20T18:51:40.586993+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:58.832583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":88,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.586802+0000","last_change":"2024-09-20T18:49:52.139994+0000","last_active":"2024-09-20T18:51:40.586802+0000","last_peered":"2024-09-20T18:51:40.586802+0000","last_clean":"2024-09-20T18:51:40.586802+0000","last_became_active":"2024-09-20T18:49:52.139685+0000","last_became_peered":"2024-09-20T18:49:52.139685+0000","last_unstale":"2024-09-20T18:51:40.586802+0000","last_undegraded":"2024-09-20T18:51:40.586802+0000","last_fullsized":"2024-09-20T18:51:40.586802+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:55.694146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":100,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.584331+0000","last_change":"2024-09-20T18:49:45.940360+0000","last_active":"2024-09-20T18:51:40.584331+0000","last_peered":"2024-09-20T18:51:40.584331+0000","last_clean":"2024-09-20T18:51:40.584331+0000","last_became_active":"2024-09-20T18:49:45.940060+0000","last_became_peered":"2024-09-20T18:49:45.940060+0000","last_unstale":"2024-09-20T18:51:40.584331+0000","last_undegraded":"2024-09-20T18:51:40.584331+0000","last_fullsized":"2024-09-20T18:51:40.584331+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:03.464395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":106,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.585455+0000","last_change":"2024-09-20T18:49:48.501010+0000","last_active":"2024-09-20T18:51:40.585455+0000","last_peered":"2024-09-20T18:51:40.585455+0000","last_clean":"2024-09-20T18:51:40.585455+0000","last_became_active":"2024-09-20T18:49:48.500662+0000","last_became_peered":"2024-09-20T18:49:48.500662+0000","last_unstale":"2024-09-20T18:51:40.585455+0000","last_undegraded":"2024-09-20T18:51:40.585455+0000","last_fullsized":"2024-09-20T18:51:40.585455+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:04.011061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":92,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.587223+0000","last_change":"2024-09-20T18:49:49.957092+0000","last_active":"2024-09-20T18:51:40.587223+0000","last_peered":"2024-09-20T18:51:40.587223+0000","last_clean":"2024-09-20T18:51:40.587223+0000","last_became_active":"2024-09-20T18:49:49.956749+0000","last_became_peered":"2024-09-20T18:49:49.956749+0000","last_unstale":"2024-09-20T18:51:40.587223+0000","last_undegraded":"2024-09-20T18:51:40.587223+0000","last_fullsized":"2024-09-20T18:51:40.587223+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:03.829260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":96,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.587186+0000","last_change":"2024-09-20T18:51:07.757046+0000","last_active":"2024-09-20T18:51:40.587186+0000","last_peered":"2024-09-20T18:51:40.587186+0000","last_clean":"2024-09-20T18:51:40.587186+0000","last_became_active":"2024-09-20T18:49:52.142291+0000","last_became_peered":"2024-09-20T18:49:52.142291+0000","last_unstale":"2024-09-20T18:51:40.587186+0000","last_undegraded":"2024-09-20T18:51:40.587186+0000","last_fullsized":"2024-09-20T18:51:40.587186+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:07.756968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:07.756968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:10.004330+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":120,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.597934+0000","last_change":"2024-09-20T18:51:23.065382+0000","last_active":"2024-09-20T18:51:40.597934+0000","last_peered":"2024-09-20T18:51:40.597934+0000","last_clean":"2024-09-20T18:51:40.597934+0000","last_became_active":"2024-09-20T18:49:48.500803+0000","last_became_peered":"2024-09-20T18:49:48.500803+0000","last_unstale":"2024-09-20T18:51:40.597934+0000","last_undegraded":"2024-09-20T18:51:40.597934+0000","last_fullsized":"2024-09-20T18:51:40.597934+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:51:23.064536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:23.064536+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:33.720318+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":108,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.587151+0000","last_change":"2024-09-20T18:51:30.982535+0000","last_active":"2024-09-20T18:51:40.587151+0000","last_peered":"2024-09-20T18:51:40.587151+0000","last_clean":"2024-09-20T18:51:40.587151+0000","last_became_active":"2024-09-20T18:49:45.946600+0000","last_became_peered":"2024-09-20T18:49:45.946600+0000","last_unstale":"2024-09-20T18:51:40.587151+0000","last_undegraded":"2024-09-20T18:51:40.587151+0000","last_fullsized":"2024-09-20T18:51:40.587151+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:30.982454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:30.982454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:54.797436+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":93,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.585274+0000","last_change":"2024-09-20T18:49:49.962290+0000","last_active":"2024-09-20T18:51:40.585274+0000","last_peered":"2024-09-20T18:51:40.585274+0000","last_clean":"2024-09-20T18:51:40.585274+0000","last_became_active":"2024-09-20T18:49:49.961947+0000","last_became_peered":"2024-09-20T18:49:49.961947+0000","last_unstale":"2024-09-20T18:51:40.585274+0000","last_undegraded":"2024-09-20T18:51:40.585274+0000","last_fullsized":"2024-09-20T18:51:40.585274+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:56.880733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"66'1","reported_seq":91,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.578197+0000","last_change":"2024-09-20T18:49:51.965397+0000","last_active":"2024-09-20T18:51:40.578197+0000","last_peered":"2024-09-20T18:51:40.578197+0000","last_clean":"2024-09-20T18:51:40.578197+0000","last_became_active":"2024-09-20T18:49:51.965121+0000","last_became_peered":"2024-09-20T18:49:51.965121+0000","last_unstale":"2024-09-20T18:51:40.578197+0000","last_undegraded":"2024-09-20T18:51:40.578197+0000","last_fullsized":"2024-09-20T18:51:40.578197+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:03.495063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":112,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.588167+0000","last_change":"2024-09-20T18:49:48.501359+0000","last_active":"2024-09-20T18:51:40.588167+0000","last_peered":"2024-09-20T18:51:40.588167+0000","last_clean":"2024-09-20T18:51:40.588167+0000","last_became_active":"2024-09-20T18:49:48.501011+0000","last_became_peered":"2024-09-20T18:49:48.501011+0000","last_unstale":"2024-09-20T18:51:40.588167+0000","last_undegraded":"2024-09-20T18:51:40.588167+0000","last_fullsized":"2024-09-20T18:51:40.588167+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:54.722340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":108,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.595290+0000","last_change":"2024-09-20T18:51:20.067303+0000","last_active":"2024-09-20T18:51:40.595290+0000","last_peered":"2024-09-20T18:51:40.595290+0000","last_clean":"2024-09-20T18:51:40.595290+0000","last_became_active":"2024-09-20T18:49:45.938511+0000","last_became_peered":"2024-09-20T18:49:45.938511+0000","last_unstale":"2024-09-20T18:51:40.595290+0000","last_undegraded":"2024-09-20T18:51:40.595290+0000","last_fullsized":"2024-09-20T18:51:40.595290+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:20.067233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:20.067233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:48.987502+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":125,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.584748+0000","last_change":"2024-09-20T18:49:49.960889+0000","last_active":"2024-09-20T18:51:40.584748+0000","last_peered":"2024-09-20T18:51:40.584748+0000","last_clean":"2024-09-20T18:51:40.584748+0000","last_became_active":"2024-09-20T18:49:49.960550+0000","last_became_peered":"2024-09-20T18:49:49.960550+0000","last_unstale":"2024-09-20T18:51:40.584748+0000","last_undegraded":"2024-09-20T18:51:40.584748+0000","last_fullsized":"2024-09-20T18:51:40.584748+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:59.949758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":89,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.582790+0000","last_change":"2024-09-20T18:49:51.966530+0000","last_active":"2024-09-20T18:51:40.582790+0000","last_peered":"2024-09-20T18:51:40.582790+0000","last_clean":"2024-09-20T18:51:40.582790+0000","last_became_active":"2024-09-20T18:49:51.966254+0000","last_became_peered":"2024-09-20T18:49:51.966254+0000","last_unstale":"2024-09-20T18:51:40.582790+0000","last_undegraded":"2024-09-20T18:51:40.582790+0000","last_fullsized":"2024-09-20T18:51:40.582790+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:37.605695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":139,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.584821+0000","last_change":"2024-09-20T18:51:15.223913+0000","last_active":"2024-09-20T18:51:40.584821+0000","last_peered":"2024-09-20T18:51:40.584821+0000","last_clean":"2024-09-20T18:51:40.584821+0000","last_became_active":"2024-09-20T18:49:58.997494+0000","last_became_peered":"2024-09-20T18:49:58.997494+0000","last_unstale":"2024-09-20T18:51:40.584821+0000","last_undegraded":"2024-09-20T18:51:40.584821+0000","last_fullsized":"2024-09-20T18:51:40.584821+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:51:15.223826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:15.223826+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:39.964162+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":108,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.587861+0000","last_change":"2024-09-20T18:51:11.708427+0000","last_active":"2024-09-20T18:51:40.587861+0000","last_peered":"2024-09-20T18:51:40.587861+0000","last_clean":"2024-09-20T18:51:40.587861+0000","last_became_active":"2024-09-20T18:49:45.949492+0000","last_became_peered":"2024-09-20T18:49:45.949492+0000","last_unstale":"2024-09-20T18:51:40.587861+0000","last_undegraded":"2024-09-20T18:51:40.587861+0000","last_fullsized":"2024-09-20T18:51:40.587861+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:11.708358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:11.708358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:25.590418+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":124,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.588011+0000","last_change":"2024-09-20T18:49:49.962519+0000","last_active":"2024-09-20T18:51:40.588011+0000","last_peered":"2024-09-20T18:51:40.588011+0000","last_clean":"2024-09-20T18:51:40.588011+0000","last_became_active":"2024-09-20T18:49:49.962243+0000","last_became_peered":"2024-09-20T18:49:49.962243+0000","last_unstale":"2024-09-20T18:51:40.588011+0000","last_undegraded":"2024-09-20T18:51:40.588011+0000","last_fullsized":"2024-09-20T18:51:40.588011+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:49:48.932694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:38.717965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":88,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.597152+0000","last_change":"2024-09-20T18:49:51.969358+0000","last_active":"2024-09-20T18:51:40.597152+0000","last_peered":"2024-09-20T18:51:40.597152+0000","last_clean":"2024-09-20T18:51:40.597152+0000","last_became_active":"2024-09-20T18:49:51.969093+0000","last_became_peered":"2024-09-20T18:49:51.969093+0000","last_unstale":"2024-09-20T18:51:40.597152+0000","last_undegraded":"2024-09-20T18:51:40.597152+0000","last_fullsized":"2024-09-20T18:51:40.597152+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":116,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.582268+0000","last_change":"2024-09-20T18:49:47.950488+0000","last_active":"2024-09-20T18:51:40.582268+0000","last_peered":"2024-09-20T18:51:40.582268+0000","last_clean":"2024-09-20T18:51:40.582268+0000","last_became_active":"2024-09-20T18:49:47.949703+0000","last_became_peered":"2024-09-20T18:49:47.949703+0000","last_unstale":"2024-09-20T18:51:40.582268+0000","last_undegraded":"2024-09-20T18:51:40.582268+0000","last_fullsized":"2024-09-20T18:51:40.582268+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:00.418527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":101,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.582443+0000","last_change":"2024-09-20T18:49:45.941578+0000","last_active":"2024-09-20T18:51:40.582443+0000","last_peered":"2024-09-20T18:51:40.582443+0000","last_clean":"2024-09-20T18:51:40.582443+0000","last_became_active":"2024-09-20T18:49:45.939949+0000","last_became_peered":"2024-09-20T18:49:45.939949+0000","last_unstale":"2024-09-20T18:51:40.582443+0000","last_undegraded":"2024-09-20T18:51:40.582443+0000","last_fullsized":"2024-09-20T18:51:40.582443+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:03.466983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":101,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.588313+0000","last_change":"2024-09-20T18:51:14.271954+0000","last_active":"2024-09-20T18:51:40.588313+0000","last_peered":"2024-09-20T18:51:40.588313+0000","last_clean":"2024-09-20T18:51:40.588313+0000","last_became_active":"2024-09-20T18:49:49.957067+0000","last_became_peered":"2024-09-20T18:49:49.957067+0000","last_unstale":"2024-09-20T18:51:40.588313+0000","last_undegraded":"2024-09-20T18:51:40.588313+0000","last_fullsized":"2024-09-20T18:51:40.588313+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:14.271719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.271719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:29.386666+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":97,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.584836+0000","last_change":"2024-09-20T18:51:14.210438+0000","last_active":"2024-09-20T18:51:40.584836+0000","last_peered":"2024-09-20T18:51:40.584836+0000","last_clean":"2024-09-20T18:51:40.584836+0000","last_became_active":"2024-09-20T18:49:51.966915+0000","last_became_peered":"2024-09-20T18:49:51.966915+0000","last_unstale":"2024-09-20T18:51:40.584836+0000","last_undegraded":"2024-09-20T18:51:40.584836+0000","last_fullsized":"2024-09-20T18:51:40.584836+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:14.210358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.210358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:36.112364+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":121,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.588417+0000","last_change":"2024-09-20T18:51:27.215834+0000","last_active":"2024-09-20T18:51:40.588417+0000","last_peered":"2024-09-20T18:51:40.588417+0000","last_clean":"2024-09-20T18:51:40.588417+0000","last_became_active":"2024-09-20T18:49:47.953310+0000","last_became_peered":"2024-09-20T18:49:47.953310+0000","last_unstale":"2024-09-20T18:51:40.588417+0000","last_undegraded":"2024-09-20T18:51:40.588417+0000","last_fullsized":"2024-09-20T18:51:40.588417+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:51:27.215761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:27.215761+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:38.030277+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":100,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.587473+0000","last_change":"2024-09-20T18:49:45.944827+0000","last_active":"2024-09-20T18:51:40.587473+0000","last_peered":"2024-09-20T18:51:40.587473+0000","last_clean":"2024-09-20T18:51:40.587473+0000","last_became_active":"2024-09-20T18:49:45.944357+0000","last_became_peered":"2024-09-20T18:49:45.944357+0000","last_unstale":"2024-09-20T18:51:40.587473+0000","last_undegraded":"2024-09-20T18:51:40.587473+0000","last_fullsized":"2024-09-20T18:51:40.587473+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:33.881107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":101,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.587270+0000","last_change":"2024-09-20T18:51:20.208925+0000","last_active":"2024-09-20T18:51:40.587270+0000","last_peered":"2024-09-20T18:51:40.587270+0000","last_clean":"2024-09-20T18:51:40.587270+0000","last_became_active":"2024-09-20T18:49:49.962523+0000","last_became_peered":"2024-09-20T18:49:49.962523+0000","last_unstale":"2024-09-20T18:51:40.587270+0000","last_undegraded":"2024-09-20T18:51:40.587270+0000","last_fullsized":"2024-09-20T18:51:40.587270+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:20.208786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:20.208786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:41.121665+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":96,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.594500+0000","last_change":"2024-09-20T18:51:25.075071+0000","last_active":"2024-09-20T18:51:40.594500+0000","last_peered":"2024-09-20T18:51:40.594500+0000","last_clean":"2024-09-20T18:51:40.594500+0000","last_became_active":"2024-09-20T18:49:51.967313+0000","last_became_peered":"2024-09-20T18:49:51.967313+0000","last_unstale":"2024-09-20T18:51:40.594500+0000","last_undegraded":"2024-09-20T18:51:40.594500+0000","last_fullsized":"2024-09-20T18:51:40.594500+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:25.075012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:25.075012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:35.438098+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":116,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.584981+0000","last_change":"2024-09-20T18:49:48.499421+0000","last_active":"2024-09-20T18:51:40.584981+0000","last_peered":"2024-09-20T18:51:40.584981+0000","last_clean":"2024-09-20T18:51:40.584981+0000","last_became_active":"2024-09-20T18:49:48.499111+0000","last_became_peered":"2024-09-20T18:49:48.499111+0000","last_unstale":"2024-09-20T18:51:40.584981+0000","last_undegraded":"2024-09-20T18:51:40.584981+0000","last_fullsized":"2024-09-20T18:51:40.584981+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:58.923856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":100,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.587335+0000","last_change":"2024-09-20T18:49:45.942406+0000","last_active":"2024-09-20T18:51:40.587335+0000","last_peered":"2024-09-20T18:51:40.587335+0000","last_clean":"2024-09-20T18:51:40.587335+0000","last_became_active":"2024-09-20T18:49:45.942105+0000","last_became_peered":"2024-09-20T18:49:45.942105+0000","last_unstale":"2024-09-20T18:51:40.587335+0000","last_undegraded":"2024-09-20T18:51:40.587335+0000","last_fullsized":"2024-09-20T18:51:40.587335+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:05.749853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":100,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.588050+0000","last_change":"2024-09-20T18:51:29.979130+0000","last_active":"2024-09-20T18:51:40.588050+0000","last_peered":"2024-09-20T18:51:40.588050+0000","last_clean":"2024-09-20T18:51:40.588050+0000","last_became_active":"2024-09-20T18:49:49.958159+0000","last_became_peered":"2024-09-20T18:49:49.958159+0000","last_unstale":"2024-09-20T18:51:40.588050+0000","last_undegraded":"2024-09-20T18:51:40.588050+0000","last_fullsized":"2024-09-20T18:51:40.588050+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:29.979058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:29.979058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:30.983783+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":89,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.587386+0000","last_change":"2024-09-20T18:49:52.140525+0000","last_active":"2024-09-20T18:51:40.587386+0000","last_peered":"2024-09-20T18:51:40.587386+0000","last_clean":"2024-09-20T18:51:40.587386+0000","last_became_active":"2024-09-20T18:49:52.139978+0000","last_became_peered":"2024-09-20T18:49:52.139978+0000","last_unstale":"2024-09-20T18:51:40.587386+0000","last_undegraded":"2024-09-20T18:51:40.587386+0000","last_fullsized":"2024-09-20T18:51:40.587386+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:39.158132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":111,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.584260+0000","last_change":"2024-09-20T18:51:13.218721+0000","last_active":"2024-09-20T18:51:40.584260+0000","last_peered":"2024-09-20T18:51:40.584260+0000","last_clean":"2024-09-20T18:51:40.584260+0000","last_became_active":"2024-09-20T18:49:47.954777+0000","last_became_peered":"2024-09-20T18:49:47.954777+0000","last_unstale":"2024-09-20T18:51:40.584260+0000","last_undegraded":"2024-09-20T18:51:40.584260+0000","last_fullsized":"2024-09-20T18:51:40.584260+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-20T18:51:13.218648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:13.218648+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:21.958438+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":100,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.596985+0000","last_change":"2024-09-20T18:49:45.941248+0000","last_active":"2024-09-20T18:51:40.596985+0000","last_peered":"2024-09-20T18:51:40.596985+0000","last_clean":"2024-09-20T18:51:40.596985+0000","last_became_active":"2024-09-20T18:49:45.940988+0000","last_became_peered":"2024-09-20T18:49:45.940988+0000","last_unstale":"2024-09-20T18:51:40.596985+0000","last_undegraded":"2024-09-20T18:51:40.596985+0000","last_fullsized":"2024-09-20T18:51:40.596985+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:05.962842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":100,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.584409+0000","last_change":"2024-09-20T18:51:17.573996+0000","last_active":"2024-09-20T18:51:40.584409+0000","last_peered":"2024-09-20T18:51:40.584409+0000","last_clean":"2024-09-20T18:51:40.584409+0000","last_became_active":"2024-09-20T18:49:49.954502+0000","last_became_peered":"2024-09-20T18:49:49.954502+0000","last_unstale":"2024-09-20T18:51:40.584409+0000","last_undegraded":"2024-09-20T18:51:40.584409+0000","last_fullsized":"2024-09-20T18:51:40.584409+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:17.573927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:17.573927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:41.951863+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":96,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.587225+0000","last_change":"2024-09-20T18:51:09.761331+0000","last_active":"2024-09-20T18:51:40.587225+0000","last_peered":"2024-09-20T18:51:40.587225+0000","last_clean":"2024-09-20T18:51:40.587225+0000","last_became_active":"2024-09-20T18:49:51.968662+0000","last_became_peered":"2024-09-20T18:49:51.968662+0000","last_unstale":"2024-09-20T18:51:40.587225+0000","last_undegraded":"2024-09-20T18:51:40.587225+0000","last_fullsized":"2024-09-20T18:51:40.587225+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:09.761283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:09.761283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:26.235481+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":89,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.590607+0000","last_change":"2024-09-20T18:49:51.968476+0000","last_active":"2024-09-20T18:51:40.590607+0000","last_peered":"2024-09-20T18:51:40.590607+0000","last_clean":"2024-09-20T18:51:40.590607+0000","last_became_active":"2024-09-20T18:49:51.968216+0000","last_became_peered":"2024-09-20T18:49:51.968216+0000","last_unstale":"2024-09-20T18:51:40.590607+0000","last_undegraded":"2024-09-20T18:51:40.590607+0000","last_fullsized":"2024-09-20T18:51:40.590607+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":101,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.584479+0000","last_change":"2024-09-20T18:49:45.938413+0000","last_active":"2024-09-20T18:51:40.584479+0000","last_peered":"2024-09-20T18:51:40.584479+0000","last_clean":"2024-09-20T18:51:40.584479+0000","last_became_active":"2024-09-20T18:49:45.937992+0000","last_became_peered":"2024-09-20T18:49:45.937992+0000","last_unstale":"2024-09-20T18:51:40.584479+0000","last_undegraded":"2024-09-20T18:51:40.584479+0000","last_fullsized":"2024-09-20T18:51:40.584479+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:35.183568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":123,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.585465+0000","last_change":"2024-09-20T18:51:14.545122+0000","last_active":"2024-09-20T18:51:40.585465+0000","last_peered":"2024-09-20T18:51:40.585465+0000","last_clean":"2024-09-20T18:51:40.585465+0000","last_became_active":"2024-09-20T18:49:47.949582+0000","last_became_peered":"2024-09-20T18:49:47.949582+0000","last_unstale":"2024-09-20T18:51:40.585465+0000","last_undegraded":"2024-09-20T18:51:40.585465+0000","last_fullsized":"2024-09-20T18:51:40.585465+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:51:14.545043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.545043+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:26.009249+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":100,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-20T18:51:40.597404+0000","last_change":"2024-09-20T18:51:28.120100+0000","last_active":"2024-09-20T18:51:40.597404+0000","last_peered":"2024-09-20T18:51:40.597404+0000","last_clean":"2024-09-20T18:51:40.597404+0000","last_became_active":"2024-09-20T18:49:49.957582+0000","last_became_peered":"2024-09-20T18:49:49.957582+0000","last_unstale":"2024-09-20T18:51:40.597404+0000","last_undegraded":"2024-09-20T18:51:40.597404+0000","last_fullsized":"2024-09-20T18:51:40.597404+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:28.119694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:28.119694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:32.124530+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":5294194688,"num_objects":80783,"num_object_clones":0,"num_object_copies":242349,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80783,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80783,"num_write_kb":5170112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":15882059776,"data_stored":15882059776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79083,"ondisk_log_size":79083,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":115,"num_read_kb":110,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266716,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1656028,"kb_used_data":1606796,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":92089124,"statfs":{"total":95995035648,"available":94299262976,"internally_reserved":0,"allocated":1645359104,"data_stored":1644735540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528355,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1654856,"kb_used_data":1609784,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":92090296,"statfs":{"total":95995035648,"available":94300463104,"internally_reserved":0,"allocated":1648418816,"data_stored":1647815070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757288,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1322568,"kb_used_data":1277496,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":92422584,"statfs":{"total":95995035648,"available":94640726016,"internally_reserved":0,"allocated":1308155904,"data_stored":1307551973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986221,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1645800,"kb_used_data":1600728,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":92099352,"statfs":{"total":95995035648,"available":94309736448,"internally_reserved":0,"allocated":1639145472,"data_stored":1638510025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182452,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2958500,"kb_used_data":2891220,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":90786652,"statfs":{"total":95995035648,"available":92965531648,"internally_reserved":0,"allocated":2960609280,"data_stored":2959976773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411385,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1984712,"kb_used_data":1931320,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":91760440,"statfs":{"total":95995035648,"available":93962690560,"internally_reserved":0,"allocated":1977671680,"data_stored":1977068218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1977984,"kb_used_data":1924592,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":91767168,"statfs":{"total":95995035648,"available":93969580032,"internally_reserved":0,"allocated":1970782208,"data_stored":1970122806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2639644,"kb_used_data":2564044,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":91105508,"statfs":{"total":95995035648,"available":93292040192,"internally_reserved":0,"allocated":2625581056,"data_stored":2624957188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2644770816,"data_stored":2644770816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1984036864,"data_stored":1984036864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1979842560,"data_stored":1979842560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2988244992,"data_stored":2988244992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1639514112,"data_stored":1639514112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1325137920,"data_stored":1325137920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1654587392,"data_stored":1654587392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1665925120,"data_stored":1665925120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T18:51:50.649 INFO:tasks.thrashosds.thrasher:Setting 3.a to [3, 7, 5, 1, 0, 2] 2024-09-20T18:51:50.649 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.a', '3', '7', '5', '1', '0', '2'] 2024-09-20T18:51:50.650 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd pg-upmap-items 3.a 3 7 5 1 0 2 2024-09-20T18:51:50.650 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:51:50.727 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:50.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:50.728 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:51:50.829 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:50.859 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:50.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:50.860 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:50.960 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:50.989 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:50.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:50.990 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:51.055 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:51:50 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:51:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:51:51.055 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 18:51:50 smithi083 podman[76434]: 2024-09-20 18:51:50.64649754 +0000 UTC m=+0.993859986 image pull 11f11916f8cdfb17c1da74b20deb2075d778357b4d60474d82453303f8d461f5 quay.io/prometheus/alertmanager:v0.27.0 2024-09-20T18:51:51.091 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:51.123 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:51.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:51.124 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:51.225 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:51.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:51.247 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:51.248 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:51.348 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:51.383 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:51.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:51.384 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:51.485 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:51.513 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:51.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:51.514 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:51.524 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 54 16 87178 87162 100.86 65.8125 0.0166483 0.00991004 2024-09-20T18:51:51.615 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:51.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:51.643 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:51.644 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:51.744 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:51.779 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:51.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:51.780 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:51.880 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:51.919 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:51.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:51.920 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:51.931 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:51 smithi083 ceph-mon[29421]: osdmap e108: 8 total, 8 up, 8 in 2024-09-20T18:51:51.931 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:51 smithi083 ceph-mon[36260]: osdmap e108: 8 total, 8 up, 8 in 2024-09-20T18:51:51.994 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-20T18:51:52.021 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:52.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:51 smithi142 ceph-mon[32508]: osdmap e108: 8 total, 8 up, 8 in 2024-09-20T18:51:52.049 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:52.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:52.050 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:52.151 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:52.179 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:52.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:52.180 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:52.218 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 18:51:52 smithi083 podman[76434]: 2024-09-20 18:51:52.091022818 +0000 UTC m=+2.438385255 volume create c060cec2a873706de75405ae24ec4a1c6125b8682cbf27b596fb17f273a8b608 2024-09-20T18:51:52.219 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 18:51:52 smithi083 podman[76434]: 2024-09-20 18:51:52.2000992 +0000 UTC m=+2.547461634 container create b430eb5089f642f0ecfd2acd70578c45654c89ff3fb2b3a944d95b5ed008c92d (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-20T18:51:52.281 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:52.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:52.308 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:52.308 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:52.410 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:52.439 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:52.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:52.440 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:52.525 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 55 16 88303 88287 100.304 70.3125 0.0188312 0.00996397 2024-09-20T18:51:52.541 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:51:52.584 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:52.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:52.585 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:51:52.686 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:52.712 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:52.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:52.713 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:52.758 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:51:52.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:52 smithi083 ceph-mon[29421]: pgmap v85: 148 pgs: 148 active+clean; 5.1 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1.0 KiB/s rd, 45 MiB/s wr, 717 op/s 2024-09-20T18:51:52.805 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 18:51:52 smithi083 podman[76434]: 2024-09-20 18:51:52.596617847 +0000 UTC m=+2.943980305 container init b430eb5089f642f0ecfd2acd70578c45654c89ff3fb2b3a944d95b5ed008c92d (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-20T18:51:52.805 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 18:51:52 smithi083 podman[76434]: 2024-09-20 18:51:52.604469125 +0000 UTC m=+2.951831563 container start b430eb5089f642f0ecfd2acd70578c45654c89ff3fb2b3a944d95b5ed008c92d (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-20T18:51:52.805 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 18:51:52 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-alertmanager-a[76723]: ts=2024-09-20T18:51:52.635Z caller=main.go:181 level=info msg="Starting Alertmanager" version="(version=0.27.0, branch=HEAD, revision=0aa3c2aad14cff039931923ab16b26b7481783b5)" 2024-09-20T18:51:52.805 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 18:51:52 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-alertmanager-a[76723]: ts=2024-09-20T18:51:52.635Z caller=main.go:182 level=info build_context="(go=go1.21.7, platform=linux/amd64, user=root@22cd11f671e9, date=20240228-11:51:20, tags=netgo)" 2024-09-20T18:51:52.805 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 18:51:52 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-alertmanager-a[76723]: ts=2024-09-20T18:51:52.636Z caller=cluster.go:186 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.83 port=9094 2024-09-20T18:51:52.805 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 18:51:52 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-alertmanager-a[76723]: ts=2024-09-20T18:51:52.643Z caller=cluster.go:683 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-20T18:51:52.805 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 18:51:52 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-alertmanager-a[76723]: ts=2024-09-20T18:51:52.684Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-20T18:51:52.805 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 18:51:52 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-alertmanager-a[76723]: ts=2024-09-20T18:51:52.685Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-20T18:51:52.806 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 18:51:52 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-alertmanager-a[76723]: ts=2024-09-20T18:51:52.687Z caller=tls_config.go:313 level=info msg="Listening on" address=[::]:9093 2024-09-20T18:51:52.806 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 18:51:52 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-alertmanager-a[76723]: ts=2024-09-20T18:51:52.687Z caller=tls_config.go:316 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-20T18:51:52.806 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 18:51:52 smithi083 bash[76434]: b430eb5089f642f0ecfd2acd70578c45654c89ff3fb2b3a944d95b5ed008c92d 2024-09-20T18:51:52.806 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 18:51:52 smithi083 systemd[1]: Started Ceph alertmanager.a for 1ed5a50a-7780-11ef-baf5-efdc52797490. 2024-09-20T18:51:52.806 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:52 smithi083 ceph-mon[36260]: pgmap v85: 148 pgs: 148 active+clean; 5.1 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1.0 KiB/s rd, 45 MiB/s wr, 717 op/s 2024-09-20T18:51:52.813 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:52.837 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:52.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:52.839 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:52.851 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:51:52.852 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:51:52.939 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:51:52.979 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:52.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:52.984 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:51:53.026 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:52 smithi142 ceph-mon[32508]: pgmap v85: 148 pgs: 148 active+clean; 5.1 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1.0 KiB/s rd, 45 MiB/s wr, 717 op/s 2024-09-20T18:51:53.085 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:51:53.102 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:51:53.135 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:53.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:53.136 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:51:53.236 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:53.278 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:53.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:53.279 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:53.380 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:53.418 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:53.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:53.419 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:53.520 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:51:53.525 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 56 16 89342 89326 99.6725 64.9375 0.0100948 0.0100281 2024-09-20T18:51:53.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:53.569 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:53.569 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:51:53.669 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:53.694 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:53.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:53.695 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:53.796 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:51:53.837 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:53.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:53.838 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:51:53.938 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:53.967 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:53.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:53.968 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:54.069 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:54.083 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:53 smithi142 ceph-mon[32508]: pgmap v86: 148 pgs: 148 active+clean; 5.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 852 B/s rd, 59 MiB/s wr, 941 op/s 2024-09-20T18:51:54.083 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:53 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:54.083 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:53 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:54.083 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:53 smithi142 ceph-mon[32508]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-20T18:51:54.083 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:53 smithi142 ceph-mon[32508]: Reconfiguring daemon prometheus.a on smithi142 2024-09-20T18:51:54.109 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:54.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:54.110 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:54.146 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:53 smithi083 ceph-mon[29421]: pgmap v86: 148 pgs: 148 active+clean; 5.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 852 B/s rd, 59 MiB/s wr, 941 op/s 2024-09-20T18:51:54.146 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:53 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:54.146 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:53 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:54.147 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:53 smithi083 ceph-mon[29421]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-20T18:51:54.147 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:53 smithi083 ceph-mon[29421]: Reconfiguring daemon prometheus.a on smithi142 2024-09-20T18:51:54.147 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:53 smithi083 ceph-mon[36260]: pgmap v86: 148 pgs: 148 active+clean; 5.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 852 B/s rd, 59 MiB/s wr, 941 op/s 2024-09-20T18:51:54.148 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:53 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:54.148 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:53 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:51:54.148 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:53 smithi083 ceph-mon[36260]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-20T18:51:54.148 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:53 smithi083 ceph-mon[36260]: Reconfiguring daemon prometheus.a on smithi142 2024-09-20T18:51:54.211 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:54.238 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:54.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:54.239 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:54.339 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:54.365 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:54.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:54.366 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:54.467 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:54.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:54.494 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:54.494 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:54.525 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 57 16 90398 90382 99.0818 66 0.0268835 0.0100874 2024-09-20T18:51:54.595 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:54.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:54.622 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:54.622 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:54.723 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:54.753 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:54.754 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T18:51:54.754 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-20T18:51:54.754 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.754 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:90134 7.a 7:564cbe79:::benchmark_data_smithi083_70643_object90133:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-20T18:51:54.754 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:54.274455+0000", 2024-09-20T18:51:54.754 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.023414633000000001, 2024-09-20T18:51:54.754 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.023446068, 2024-09-20T18:51:54.754 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:54.754 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:54.754 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "waiting for sub ops", 2024-09-20T18:51:54.754 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:51:54.754 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:51:54.754 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:51:54.755 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 90134 2024-09-20T18:51:54.755 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.755 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:54.755 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.755 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:54.755 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:54.274455+0000", 2024-09-20T18:51:54.755 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:54.755 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.755 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.755 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:54.755 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:54.274455+0000", 2024-09-20T18:51:54.755 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:54.755 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.755 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.756 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:54.756 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:54.274457+0000", 2024-09-20T18:51:54.756 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7489999999999999e-06 2024-09-20T18:51:54.756 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.756 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.756 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:54.756 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:54.274482+0000", 2024-09-20T18:51:54.756 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5335000000000001e-05 2024-09-20T18:51:54.756 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.756 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.756 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:54.756 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:54.274495+0000", 2024-09-20T18:51:54.756 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2513e-05 2024-09-20T18:51:54.756 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.757 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.757 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:54.757 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:54.274503+0000", 2024-09-20T18:51:54.757 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.79e-06 2024-09-20T18:51:54.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.757 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.757 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:54.757 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:54.274618+0000", 2024-09-20T18:51:54.757 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115868 2024-09-20T18:51:54.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.757 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.757 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:54.757 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:54.275032+0000", 2024-09-20T18:51:54.757 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00041351700000000002 2024-09-20T18:51:54.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.758 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.758 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,5", 2024-09-20T18:51:54.758 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:54.278913+0000", 2024-09-20T18:51:54.758 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0038812260000000002 2024-09-20T18:51:54.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.758 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.758 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:54.758 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:54.294263+0000", 2024-09-20T18:51:54.758 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.015350109000000001 2024-09-20T18:51:54.758 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:54.758 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:54.758 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:54.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.759 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.759 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:90137 7.a 7:553781d2:::benchmark_data_smithi083_70643_object90136:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-20T18:51:54.759 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:54.278160+0000", 2024-09-20T18:51:54.759 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.019710203999999999, 2024-09-20T18:51:54.759 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019816329000000001, 2024-09-20T18:51:54.759 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:54.759 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:54.759 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "waiting for sub ops", 2024-09-20T18:51:54.759 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:51:54.759 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:51:54.759 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:51:54.759 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 90137 2024-09-20T18:51:54.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.759 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:54.760 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.760 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:54.760 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:54.278160+0000", 2024-09-20T18:51:54.760 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:54.760 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.760 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.760 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:54.760 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:54.278160+0000", 2024-09-20T18:51:54.760 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:54.760 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.760 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.760 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:54.760 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:54.278161+0000", 2024-09-20T18:51:54.760 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3349999999999999e-06 2024-09-20T18:51:54.761 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.761 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.761 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:54.761 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:54.278185+0000", 2024-09-20T18:51:54.761 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3893e-05 2024-09-20T18:51:54.761 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.761 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.761 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:54.761 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:54.278194+0000", 2024-09-20T18:51:54.761 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2280000000000004e-06 2024-09-20T18:51:54.761 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.761 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.761 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:54.761 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:54.278200+0000", 2024-09-20T18:51:54.761 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2609999999999999e-06 2024-09-20T18:51:54.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.762 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:54.762 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:54.287914+0000", 2024-09-20T18:51:54.762 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0097135569999999994 2024-09-20T18:51:54.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.762 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:54.762 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:54.288302+0000", 2024-09-20T18:51:54.762 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00038797700000000002 2024-09-20T18:51:54.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.762 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,5", 2024-09-20T18:51:54.762 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:54.288799+0000", 2024-09-20T18:51:54.763 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00049690300000000004 2024-09-20T18:51:54.763 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:54.763 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:54.763 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:54.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.763 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.763 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:90140 7.2 7:41c42ac0:::benchmark_data_smithi083_70643_object90139:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-20T18:51:54.763 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:54.281946+0000", 2024-09-20T18:51:54.763 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.015924029999999999, 2024-09-20T18:51:54.763 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.016090477999999998, 2024-09-20T18:51:54.763 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:54.763 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:54.763 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "waiting for sub ops", 2024-09-20T18:51:54.763 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:51:54.764 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:51:54.764 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:51:54.764 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 90140 2024-09-20T18:51:54.764 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.764 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:54.764 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.764 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:54.764 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:54.281946+0000", 2024-09-20T18:51:54.764 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:54.764 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.764 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.764 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:54.764 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:54.281946+0000", 2024-09-20T18:51:54.764 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:54.764 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.765 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.765 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:54.765 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:54.281948+0000", 2024-09-20T18:51:54.765 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8560000000000001e-06 2024-09-20T18:51:54.765 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.765 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.765 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:54.765 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:54.281974+0000", 2024-09-20T18:51:54.765 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6332000000000001e-05 2024-09-20T18:51:54.765 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.765 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.765 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:54.765 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:54.281988+0000", 2024-09-20T18:51:54.765 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3794e-05 2024-09-20T18:51:54.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.766 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.766 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:54.766 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:54.281995+0000", 2024-09-20T18:51:54.766 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5429999999999996e-06 2024-09-20T18:51:54.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.766 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.766 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:54.766 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:54.294304+0000", 2024-09-20T18:51:54.766 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.012309188 2024-09-20T18:51:54.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.766 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.766 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:54.766 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:54.294410+0000", 2024-09-20T18:51:54.767 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000105039 2024-09-20T18:51:54.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.767 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.767 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T18:51:54.767 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:54.294537+0000", 2024-09-20T18:51:54.767 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012714000000000001 2024-09-20T18:51:54.767 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:54.767 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:54.767 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:54.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.767 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.767 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:90143 7.3 7:c6f5495c:::benchmark_data_smithi083_70643_object90142:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-20T18:51:54.767 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:54.282011+0000", 2024-09-20T18:51:54.767 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.015858712000000001, 2024-09-20T18:51:54.768 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.016085565, 2024-09-20T18:51:54.768 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:54.768 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:54.768 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "waiting for sub ops", 2024-09-20T18:51:54.768 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:51:54.768 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:51:54.768 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:51:54.768 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 90143 2024-09-20T18:51:54.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.768 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:54.768 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.768 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:54.768 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:54.282011+0000", 2024-09-20T18:51:54.768 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:54.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.769 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.769 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:54.769 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:54.282011+0000", 2024-09-20T18:51:54.769 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:54.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.769 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.769 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:54.769 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:54.282011+0000", 2024-09-20T18:51:54.769 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1300000000000001e-07 2024-09-20T18:51:54.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.769 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.769 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:54.769 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:54.282034+0000", 2024-09-20T18:51:54.769 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2180000000000001e-05 2024-09-20T18:51:54.770 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.770 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.770 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:54.770 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:54.282040+0000", 2024-09-20T18:51:54.770 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4150000000000004e-06 2024-09-20T18:51:54.770 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.770 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.770 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:54.770 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:54.282045+0000", 2024-09-20T18:51:54.770 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0769999999999998e-06 2024-09-20T18:51:54.770 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.770 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.771 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:54.771 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:54.282174+0000", 2024-09-20T18:51:54.771 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012893 2024-09-20T18:51:54.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.771 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:54.771 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:54.282284+0000", 2024-09-20T18:51:54.771 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000110395 2024-09-20T18:51:54.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.771 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,4", 2024-09-20T18:51:54.771 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:54.282417+0000", 2024-09-20T18:51:54.771 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000132039 2024-09-20T18:51:54.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.772 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.772 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:54.772 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:54.285326+0000", 2024-09-20T18:51:54.772 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0029091880000000001 2024-09-20T18:51:54.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.772 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.772 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:51:54.772 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:54.292355+0000", 2024-09-20T18:51:54.772 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0070293159999999999 2024-09-20T18:51:54.772 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:54.772 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:54.772 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:54.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.772 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.772 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:90144 7.3 7:c360881d:::benchmark_data_smithi083_70643_object90143:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-20T18:51:54.773 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:54.285959+0000", 2024-09-20T18:51:54.773 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.011911148999999999, 2024-09-20T18:51:54.773 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.012207832, 2024-09-20T18:51:54.773 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:51:54.773 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:51:54.773 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "waiting for sub ops", 2024-09-20T18:51:54.773 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:51:54.773 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:51:54.773 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:51:54.773 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 90144 2024-09-20T18:51:54.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.773 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:51:54.773 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.773 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:51:54.774 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:54.285959+0000", 2024-09-20T18:51:54.774 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:54.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.774 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:51:54.774 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:54.285959+0000", 2024-09-20T18:51:54.774 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:51:54.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.774 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:51:54.774 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:54.285960+0000", 2024-09-20T18:51:54.774 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.629e-06 2024-09-20T18:51:54.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.775 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:51:54.775 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:54.285983+0000", 2024-09-20T18:51:54.775 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2399e-05 2024-09-20T18:51:54.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.775 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:51:54.775 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:54.285999+0000", 2024-09-20T18:51:54.775 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6147e-05 2024-09-20T18:51:54.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.775 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:51:54.775 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:54.286007+0000", 2024-09-20T18:51:54.775 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.473e-06 2024-09-20T18:51:54.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.776 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.776 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:51:54.776 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:54.286867+0000", 2024-09-20T18:51:54.776 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00085941000000000001 2024-09-20T18:51:54.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.776 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.776 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:51:54.776 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:54.287137+0000", 2024-09-20T18:51:54.776 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00027006600000000001 2024-09-20T18:51:54.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.776 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.776 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,4", 2024-09-20T18:51:54.776 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:54.287530+0000", 2024-09-20T18:51:54.776 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00039338399999999999 2024-09-20T18:51:54.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.777 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.777 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:51:54.777 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:54.292336+0000", 2024-09-20T18:51:54.777 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0048054600000000001 2024-09-20T18:51:54.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:51:54.777 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:51:54.777 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:51:54.777 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:54.292361+0000", 2024-09-20T18:51:54.777 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5378e-05 2024-09-20T18:51:54.777 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:54.777 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:51:54.777 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:54.777 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:51:54.777 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T18:51:54.777 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 5 2024-09-20T18:51:54.778 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T18:51:54.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:54.778 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:54.837 INFO:teuthology.orchestra.run.smithi083.stderr:set 3.a pg_upmap_items mapping to [3->7,5->1,0->2] 2024-09-20T18:51:54.879 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:51:54.903 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:54.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:54.904 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:51:54.956 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:54 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3573034244' entity='client.admin' cmd={"prefix": "osd pg-upmap-items", "pgid": "3.a", "id": [3, 7, 5, 1, 0, 2]} : dispatch 2024-09-20T18:51:54.956 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:54 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pg-upmap-items", "pgid": "3.a", "id": [3, 7, 5, 1, 0, 2]} : dispatch 2024-09-20T18:51:55.005 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:55.031 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:55.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:55.032 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:55.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:54 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3573034244' entity='client.admin' cmd={"prefix": "osd pg-upmap-items", "pgid": "3.a", "id": [3, 7, 5, 1, 0, 2]} : dispatch 2024-09-20T18:51:55.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:54 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pg-upmap-items", "pgid": "3.a", "id": [3, 7, 5, 1, 0, 2]} : dispatch 2024-09-20T18:51:55.054 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 18:51:54 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-alertmanager-a[76723]: ts=2024-09-20T18:51:54.644Z caller=cluster.go:708 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000215802s 2024-09-20T18:51:55.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:54 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3573034244' entity='client.admin' cmd={"prefix": "osd pg-upmap-items", "pgid": "3.a", "id": [3, 7, 5, 1, 0, 2]} : dispatch 2024-09-20T18:51:55.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:54 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pg-upmap-items", "pgid": "3.a", "id": [3, 7, 5, 1, 0, 2]} : dispatch 2024-09-20T18:51:55.133 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:51:55.172 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:55.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:55.173 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:51:55.274 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:55.390 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:55.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:55.392 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:55.493 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:55.521 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:55.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:55.522 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:55.525 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 58 16 91562 91546 98.6277 72.75 0.00872574 0.0101345 2024-09-20T18:51:55.623 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:55.667 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:55.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:55.668 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:55.769 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:51:55.811 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:55.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:55.813 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:51:55.914 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:55.940 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:55.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:55.941 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:56.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:55 smithi142 ceph-mon[32508]: pgmap v87: 148 pgs: 148 active+clean; 5.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 741 B/s rd, 51 MiB/s wr, 818 op/s 2024-09-20T18:51:56.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:55 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "3.a", "id": [3, 7, 5, 1, 0, 2]}]': finished 2024-09-20T18:51:56.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:55 smithi142 ceph-mon[32508]: osdmap e109: 8 total, 8 up, 8 in 2024-09-20T18:51:56.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:55 smithi142 ceph-mon[32508]: osdmap e110: 8 total, 8 up, 8 in 2024-09-20T18:51:56.042 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:56.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:56.072 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:56.072 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:56.173 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:51:56.210 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:56.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:56.211 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:51:56.320 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:56.321 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:55 smithi083 ceph-mon[29421]: pgmap v87: 148 pgs: 148 active+clean; 5.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 741 B/s rd, 51 MiB/s wr, 818 op/s 2024-09-20T18:51:56.321 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:55 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "3.a", "id": [3, 7, 5, 1, 0, 2]}]': finished 2024-09-20T18:51:56.321 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:55 smithi083 ceph-mon[29421]: osdmap e109: 8 total, 8 up, 8 in 2024-09-20T18:51:56.321 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:55 smithi083 ceph-mon[29421]: osdmap e110: 8 total, 8 up, 8 in 2024-09-20T18:51:56.322 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:55 smithi083 ceph-mon[36260]: pgmap v87: 148 pgs: 148 active+clean; 5.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 741 B/s rd, 51 MiB/s wr, 818 op/s 2024-09-20T18:51:56.322 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:55 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "3.a", "id": [3, 7, 5, 1, 0, 2]}]': finished 2024-09-20T18:51:56.322 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:55 smithi083 ceph-mon[36260]: osdmap e109: 8 total, 8 up, 8 in 2024-09-20T18:51:56.322 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:55 smithi083 ceph-mon[36260]: osdmap e110: 8 total, 8 up, 8 in 2024-09-20T18:51:56.371 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:56.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:56.372 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:56.473 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:56.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:56.500 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:56.500 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:56.525 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 59 16 92677 92661 98.1372 69.6875 0.00666725 0.0101836 2024-09-20T18:51:56.601 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:56.628 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:56.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:56.629 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:56.729 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:51:56.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:56.774 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:56.774 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:51:56.775 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:51:56 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:51:56.799 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:51:56.799 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:51:56.836 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:51:56.836 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:51:56.840 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-20T18:51:56.861 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:56 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/4160390314' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:51:56.861 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:56 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:51:56.861 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:56 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:51:56.861 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:56 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2241409197' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:51:56.861 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:56 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/4160390314' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:51:56.861 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:56 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:51:56.862 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:56 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:51:56.862 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:56 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2241409197' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:51:56.875 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:56.876 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:51:56.877 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:51:56.901 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:56.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:56.902 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:56.918 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:51:56.918 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:51:56.952 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:51:56.953 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:51:56.986 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:51:56.986 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:51:57.003 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:57.021 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:51:57.022 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:51:57.030 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:57.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:57.031 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:57.059 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:51:57.059 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:51:57.132 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:57.163 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:57.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:57.164 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:57.265 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:57.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:56 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/4160390314' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:51:57.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:56 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:51:57.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:56 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:51:57.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:56 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2241409197' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:51:57.313 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:57.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:57.314 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:57.337 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_blocked_ops 2024-09-20T18:51:57.416 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:57.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:57.463 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:57.463 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:57.525 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:2024-09-20T18:51:57.509363+0000 min lat: 0.00137732 max lat: 0.186958 avg lat: 0.0102427 2024-09-20T18:51:57.525 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-20T18:51:57.526 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 60 16 93710 93694 97.5776 64.5625 0.0119833 0.0102427 2024-09-20T18:51:57.564 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:57.592 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:57.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:57.593 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:57.693 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:57.720 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:57.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:57.722 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:57.822 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:57.853 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:57.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:57.854 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:57.955 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:57.966 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:57 smithi083 ceph-mon[29421]: pgmap v90: 148 pgs: 148 active+clean; 5.6 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.6 KiB/s rd, 112 MiB/s wr, 1.79k op/s 2024-09-20T18:51:57.966 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:57 smithi083 ceph-mon[29421]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:51:57.966 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:57 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T18:51:57.967 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:57 smithi083 ceph-mon[29421]: osdmap e111: 8 total, 8 up, 8 in 2024-09-20T18:51:57.967 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:57 smithi083 ceph-mon[29421]: 3.1a scrub starts 2024-09-20T18:51:57.967 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:57 smithi083 ceph-mon[29421]: 3.1a scrub ok 2024-09-20T18:51:57.967 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:57 smithi083 ceph-mon[29421]: 6.a scrub starts 2024-09-20T18:51:57.967 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:57 smithi083 ceph-mon[29421]: 6.a scrub ok 2024-09-20T18:51:57.967 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:57 smithi083 ceph-mon[29421]: 4.4 scrub starts 2024-09-20T18:51:57.967 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:57 smithi083 ceph-mon[29421]: 4.4 scrub ok 2024-09-20T18:51:57.967 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:57 smithi083 ceph-mon[36260]: pgmap v90: 148 pgs: 148 active+clean; 5.6 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.6 KiB/s rd, 112 MiB/s wr, 1.79k op/s 2024-09-20T18:51:57.967 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:57 smithi083 ceph-mon[36260]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:51:57.968 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:57 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T18:51:57.968 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:57 smithi083 ceph-mon[36260]: osdmap e111: 8 total, 8 up, 8 in 2024-09-20T18:51:57.968 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:57 smithi083 ceph-mon[36260]: 3.1a scrub starts 2024-09-20T18:51:57.968 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:57 smithi083 ceph-mon[36260]: 3.1a scrub ok 2024-09-20T18:51:57.968 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:57 smithi083 ceph-mon[36260]: 6.a scrub starts 2024-09-20T18:51:57.968 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:57 smithi083 ceph-mon[36260]: 6.a scrub ok 2024-09-20T18:51:57.968 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:57 smithi083 ceph-mon[36260]: 4.4 scrub starts 2024-09-20T18:51:57.968 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:57 smithi083 ceph-mon[36260]: 4.4 scrub ok 2024-09-20T18:51:57.977 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:57.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:57.978 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:58.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:57 smithi142 ceph-mon[32508]: pgmap v90: 148 pgs: 148 active+clean; 5.6 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.6 KiB/s rd, 112 MiB/s wr, 1.79k op/s 2024-09-20T18:51:58.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:57 smithi142 ceph-mon[32508]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:51:58.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:57 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T18:51:58.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:57 smithi142 ceph-mon[32508]: osdmap e111: 8 total, 8 up, 8 in 2024-09-20T18:51:58.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:57 smithi142 ceph-mon[32508]: 3.1a scrub starts 2024-09-20T18:51:58.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:57 smithi142 ceph-mon[32508]: 3.1a scrub ok 2024-09-20T18:51:58.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:57 smithi142 ceph-mon[32508]: 6.a scrub starts 2024-09-20T18:51:58.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:57 smithi142 ceph-mon[32508]: 6.a scrub ok 2024-09-20T18:51:58.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:57 smithi142 ceph-mon[32508]: 4.4 scrub starts 2024-09-20T18:51:58.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:57 smithi142 ceph-mon[32508]: 4.4 scrub ok 2024-09-20T18:51:58.079 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:58.105 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:58.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:58.106 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:58.207 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:58.231 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:58.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:58.232 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:58.333 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:58.370 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:58.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:58.371 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:58.472 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:58.496 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:58.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:58.498 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:58.527 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 61 16 94823 94807 97.116 69.5625 0.00936442 0.0102923 2024-09-20T18:51:58.598 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:51:58.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:58.639 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:58.639 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:51:58.740 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:58.789 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:58.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:58.790 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:58.890 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:51:58.919 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:58.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:58.920 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:51:59.021 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:51:59.023 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:51:58 smithi142 systemd[1]: Stopping Ceph prometheus.a for 1ed5a50a-7780-11ef-baf5-efdc52797490... 2024-09-20T18:51:59.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:58 smithi142 ceph-mon[32508]: 3.1f scrub starts 2024-09-20T18:51:59.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:58 smithi142 ceph-mon[32508]: 3.1f scrub ok 2024-09-20T18:51:59.060 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:59.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:59.061 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:51:59.162 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:59.192 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:59.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:59.193 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:59.287 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:58 smithi083 ceph-mon[29421]: 3.1f scrub starts 2024-09-20T18:51:59.288 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:58 smithi083 ceph-mon[29421]: 3.1f scrub ok 2024-09-20T18:51:59.288 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:58 smithi083 ceph-mon[36260]: 3.1f scrub starts 2024-09-20T18:51:59.288 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:58 smithi083 ceph-mon[36260]: 3.1f scrub ok 2024-09-20T18:51:59.293 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:51:59.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:59.330 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:59.331 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:51:59.431 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:51:59.466 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:59.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:59.467 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:51:59.527 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 62 16 95804 95788 96.5386 61.3125 0.0142972 0.0103541 2024-09-20T18:51:59.568 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:51:59.607 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:51:59.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:59.608 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:51:59.709 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:51:59.736 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:59.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:59.737 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:51:59.799 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:51:59.837 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:51:59.864 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:51:59 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[59300]: ts=2024-09-20T18:51:59.557Z caller=main.go:964 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-20T18:51:59.864 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:51:59 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[59300]: ts=2024-09-20T18:51:59.557Z caller=main.go:988 level=info msg="Stopping scrape discovery manager..." 2024-09-20T18:51:59.864 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:51:59 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[59300]: ts=2024-09-20T18:51:59.557Z caller=main.go:1002 level=info msg="Stopping notify discovery manager..." 2024-09-20T18:51:59.864 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:51:59 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[59300]: ts=2024-09-20T18:51:59.557Z caller=manager.go:177 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-20T18:51:59.864 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:51:59 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[59300]: ts=2024-09-20T18:51:59.557Z caller=main.go:984 level=info msg="Scrape discovery manager stopped" 2024-09-20T18:51:59.864 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:51:59 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[59300]: ts=2024-09-20T18:51:59.557Z caller=manager.go:187 level=info component="rule manager" msg="Rule manager stopped" 2024-09-20T18:51:59.864 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:51:59 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[59300]: ts=2024-09-20T18:51:59.557Z caller=main.go:1039 level=info msg="Stopping scrape manager..." 2024-09-20T18:51:59.864 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:51:59 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[59300]: ts=2024-09-20T18:51:59.557Z caller=main.go:998 level=info msg="Notify discovery manager stopped" 2024-09-20T18:51:59.864 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:51:59 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[59300]: ts=2024-09-20T18:51:59.558Z caller=main.go:1031 level=info msg="Scrape manager stopped" 2024-09-20T18:51:59.864 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:51:59 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[59300]: ts=2024-09-20T18:51:59.558Z caller=notifier.go:618 level=info component=notifier msg="Stopping notification manager..." 2024-09-20T18:51:59.864 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:51:59 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[59300]: ts=2024-09-20T18:51:59.558Z caller=main.go:1261 level=info msg="Notifier manager stopped" 2024-09-20T18:51:59.865 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:51:59 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[59300]: ts=2024-09-20T18:51:59.558Z caller=main.go:1273 level=info msg="See you next time!" 2024-09-20T18:51:59.865 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:51:59 smithi142 podman[66755]: 2024-09-20 18:51:59.569430592 +0000 UTC m=+0.671539863 container died 2fa4a328eeaa8b6ccb463b68569468091a044e7370d678722e8f4e9b04aa1602 (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-20T18:51:59.865 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:51:59.865 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:59 smithi142 ceph-mon[32508]: 5.5 scrub starts 2024-09-20T18:51:59.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:51:59.866 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:51:59.931 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:59 smithi083 ceph-mon[29421]: 5.5 scrub starts 2024-09-20T18:51:59.931 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:59 smithi083 ceph-mon[29421]: 5.5 scrub ok 2024-09-20T18:51:59.931 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:59 smithi083 ceph-mon[29421]: pgmap v92: 148 pgs: 148 active+clean; 5.6 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 853 B/s rd, 57 MiB/s wr, 905 op/s 2024-09-20T18:51:59.931 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:51:59 smithi083 ceph-mon[29421]: 7.1 scrub starts 2024-09-20T18:51:59.931 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:59 smithi083 ceph-mon[36260]: 5.5 scrub starts 2024-09-20T18:51:59.931 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:59 smithi083 ceph-mon[36260]: 5.5 scrub ok 2024-09-20T18:51:59.932 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:59 smithi083 ceph-mon[36260]: pgmap v92: 148 pgs: 148 active+clean; 5.6 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 853 B/s rd, 57 MiB/s wr, 905 op/s 2024-09-20T18:51:59.932 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:51:59 smithi083 ceph-mon[36260]: 7.1 scrub starts 2024-09-20T18:51:59.967 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:00.007 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:00.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:00.008 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:00.109 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:00.140 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:59 smithi142 ceph-mon[32508]: 5.5 scrub ok 2024-09-20T18:52:00.140 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:59 smithi142 ceph-mon[32508]: pgmap v92: 148 pgs: 148 active+clean; 5.6 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 853 B/s rd, 57 MiB/s wr, 905 op/s 2024-09-20T18:52:00.140 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:51:59 smithi142 ceph-mon[32508]: 7.1 scrub starts 2024-09-20T18:52:00.153 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:00.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:00.154 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:00.254 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:00.291 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:00.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:00.292 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:00.392 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:52:00 smithi142 podman[66755]: 2024-09-20 18:52:00.124112929 +0000 UTC m=+1.226222200 container cleanup 2fa4a328eeaa8b6ccb463b68569468091a044e7370d678722e8f4e9b04aa1602 (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-20T18:52:00.393 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:52:00 smithi142 bash[66755]: ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a 2024-09-20T18:52:00.393 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:52:00.420 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:00.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:00.421 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:52:00.441 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T18:52:00.480 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T18:52:00.480 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:52:00.522 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:00.527 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 63 16 96876 96860 96.0697 67 0.00594583 0.0104036 2024-09-20T18:52:00.551 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:00.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:00.552 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:52:00.653 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:00.694 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:52:00 smithi142 podman[66790]: 2024-09-20 18:52:00.376063036 +0000 UTC m=+0.801676524 container remove 2fa4a328eeaa8b6ccb463b68569468091a044e7370d678722e8f4e9b04aa1602 (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-20T18:52:00.696 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:00.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:00.697 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:00.798 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:00.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:00.846 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:00.846 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:00.871 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:00 smithi083 ceph-mon[29421]: 3.1 scrub starts 2024-09-20T18:52:00.871 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:00 smithi083 ceph-mon[29421]: 3.1 scrub ok 2024-09-20T18:52:00.871 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:00 smithi083 ceph-mon[29421]: 3.d scrub starts 2024-09-20T18:52:00.871 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:00 smithi083 ceph-mon[29421]: 3.d scrub ok 2024-09-20T18:52:00.871 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:00 smithi083 ceph-mon[29421]: 5.6 scrub starts 2024-09-20T18:52:00.871 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:00 smithi083 ceph-mon[29421]: 5.6 scrub ok 2024-09-20T18:52:00.871 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:00 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:52:00.871 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:00 smithi083 ceph-mon[29421]: 7.1 scrub ok 2024-09-20T18:52:00.871 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:00 smithi083 ceph-mon[29421]: osdmap e112: 8 total, 8 up, 8 in 2024-09-20T18:52:00.871 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:00 smithi083 ceph-mon[36260]: 3.1 scrub starts 2024-09-20T18:52:00.872 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:00 smithi083 ceph-mon[36260]: 3.1 scrub ok 2024-09-20T18:52:00.872 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:00 smithi083 ceph-mon[36260]: 3.d scrub starts 2024-09-20T18:52:00.872 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:00 smithi083 ceph-mon[36260]: 3.d scrub ok 2024-09-20T18:52:00.947 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:00.977 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:00.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:00.978 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:52:01.024 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:52:00 smithi142 systemd[1]: ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@prometheus.a.service: Deactivated successfully. 2024-09-20T18:52:01.024 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:52:00 smithi142 systemd[1]: Stopped Ceph prometheus.a for 1ed5a50a-7780-11ef-baf5-efdc52797490. 2024-09-20T18:52:01.024 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:52:00 smithi142 systemd[1]: ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@prometheus.a.service: Consumed 1.185s CPU time. 2024-09-20T18:52:01.024 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:52:00 smithi142 systemd[1]: Starting Ceph prometheus.a for 1ed5a50a-7780-11ef-baf5-efdc52797490... 2024-09-20T18:52:01.024 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:52:00 smithi142 podman[66942]: 2024-09-20 18:52:00.842327623 +0000 UTC m=+0.025118050 image pull 1d3b7f56885b6dd623f1785be963aa9c195f86bc256ea454e8d02a7980b79c53 quay.io/prometheus/prometheus:v2.51.0 2024-09-20T18:52:01.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:00 smithi142 ceph-mon[32508]: 3.1 scrub starts 2024-09-20T18:52:01.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:00 smithi142 ceph-mon[32508]: 3.1 scrub ok 2024-09-20T18:52:01.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:00 smithi142 ceph-mon[32508]: 3.d scrub starts 2024-09-20T18:52:01.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:00 smithi142 ceph-mon[32508]: 3.d scrub ok 2024-09-20T18:52:01.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:00 smithi142 ceph-mon[32508]: 5.6 scrub starts 2024-09-20T18:52:01.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:00 smithi142 ceph-mon[32508]: 5.6 scrub ok 2024-09-20T18:52:01.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:00 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:52:01.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:00 smithi142 ceph-mon[32508]: 7.1 scrub ok 2024-09-20T18:52:01.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:00 smithi142 ceph-mon[32508]: osdmap e112: 8 total, 8 up, 8 in 2024-09-20T18:52:01.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:00 smithi083 ceph-mon[36260]: 5.6 scrub starts 2024-09-20T18:52:01.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:00 smithi083 ceph-mon[36260]: 5.6 scrub ok 2024-09-20T18:52:01.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:00 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:52:01.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:00 smithi083 ceph-mon[36260]: 7.1 scrub ok 2024-09-20T18:52:01.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:00 smithi083 ceph-mon[36260]: osdmap e112: 8 total, 8 up, 8 in 2024-09-20T18:52:01.079 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:01.115 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:01.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:01.116 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:01.208 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_historic_ops 2024-09-20T18:52:01.218 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:01.304 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:01.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:01.305 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:01.326 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:52:01.386 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-20T18:52:01.386 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T18:52:01.387 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-20T18:52:01.387 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph tell osd.0 'trim stale osdmaps' 2024-09-20T18:52:01.406 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:01.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:01.508 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:01.508 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:01.531 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 64 16 97873 97857 95.54 62.3125 0.0188884 0.0104616 2024-09-20T18:52:01.610 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:01.636 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:01.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:01.637 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:01.728 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T18:52:01.738 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:01.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:01.819 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:01.819 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:01.824 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:52:01 smithi142 podman[66942]: 2024-09-20 18:52:01.536286778 +0000 UTC m=+0.719077175 container create 790d73aeb9d9c2099db6e1fa2bf1b460ea298eb02a70b14f37a146af9298b73a (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-20T18:52:01.824 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:52:01 smithi142 podman[66942]: 2024-09-20 18:52:01.77258762 +0000 UTC m=+0.955378033 container init 790d73aeb9d9c2099db6e1fa2bf1b460ea298eb02a70b14f37a146af9298b73a (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-20T18:52:01.824 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:52:01 smithi142 podman[66942]: 2024-09-20 18:52:01.777240427 +0000 UTC m=+0.960030841 container start 790d73aeb9d9c2099db6e1fa2bf1b460ea298eb02a70b14f37a146af9298b73a (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-20T18:52:01.920 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:02.009 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:02.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:02.010 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:02.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:01 smithi083 ceph-mon[29421]: pgmap v94: 148 pgs: 148 active+clean; 5.7 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.7 KiB/s rd, 77 MiB/s wr, 1.24k op/s 2024-09-20T18:52:02.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:01 smithi083 ceph-mon[29421]: 3.1c scrub starts 2024-09-20T18:52:02.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:01 smithi083 ceph-mon[29421]: 3.1c scrub ok 2024-09-20T18:52:02.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:01 smithi083 ceph-mon[36260]: pgmap v94: 148 pgs: 148 active+clean; 5.7 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.7 KiB/s rd, 77 MiB/s wr, 1.24k op/s 2024-09-20T18:52:02.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:01 smithi083 ceph-mon[36260]: 3.1c scrub starts 2024-09-20T18:52:02.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:01 smithi083 ceph-mon[36260]: 3.1c scrub ok 2024-09-20T18:52:02.057 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:52:01 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: [20/Sep/2024:18:52:01] ENGINE Bus STOPPING 2024-09-20T18:52:02.111 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:02.135 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:02.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:02.136 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:52:02.182 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:01 smithi142 ceph-mon[32508]: pgmap v94: 148 pgs: 148 active+clean; 5.7 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.7 KiB/s rd, 77 MiB/s wr, 1.24k op/s 2024-09-20T18:52:02.182 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:01 smithi142 ceph-mon[32508]: 3.1c scrub starts 2024-09-20T18:52:02.183 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:01 smithi142 ceph-mon[32508]: 3.1c scrub ok 2024-09-20T18:52:02.183 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:52:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[66989]: ts=2024-09-20T18:52:01.807Z caller=main.go:617 level=info msg="Starting Prometheus Server" mode=server version="(version=2.51.0, branch=HEAD, revision=c05c15512acb675e3f6cd662a6727854e93fc024)" 2024-09-20T18:52:02.183 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:52:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[66989]: ts=2024-09-20T18:52:01.807Z caller=main.go:622 level=info build_context="(go=go1.22.1, platform=linux/amd64, user=root@b5723e458358, date=20240319-10:54:45, tags=netgo,builtinassets,stringlabels)" 2024-09-20T18:52:02.183 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:52:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[66989]: ts=2024-09-20T18:52:01.807Z caller=main.go:623 level=info host_details="(Linux 5.14.0-508.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 12 15:49:37 UTC 2024 x86_64 smithi142 (none))" 2024-09-20T18:52:02.183 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:52:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[66989]: ts=2024-09-20T18:52:01.807Z caller=main.go:624 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-20T18:52:02.183 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:52:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[66989]: ts=2024-09-20T18:52:01.807Z caller=main.go:625 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-20T18:52:02.183 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:52:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[66989]: ts=2024-09-20T18:52:01.811Z caller=web.go:568 level=info component=web msg="Start listening for connections" address=:9095 2024-09-20T18:52:02.183 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:52:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[66989]: ts=2024-09-20T18:52:01.811Z caller=main.go:1129 level=info msg="Starting TSDB ..." 2024-09-20T18:52:02.184 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:52:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[66989]: ts=2024-09-20T18:52:01.812Z caller=tls_config.go:313 level=info component=web msg="Listening on" address=[::]:9095 2024-09-20T18:52:02.184 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:52:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[66989]: ts=2024-09-20T18:52:01.812Z caller=tls_config.go:316 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-20T18:52:02.184 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:52:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[66989]: ts=2024-09-20T18:52:01.818Z caller=head.go:616 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-20T18:52:02.184 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:52:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[66989]: ts=2024-09-20T18:52:01.818Z caller=head.go:698 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=2.733µs 2024-09-20T18:52:02.184 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:52:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[66989]: ts=2024-09-20T18:52:01.818Z caller=head.go:706 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-20T18:52:02.184 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:52:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[66989]: ts=2024-09-20T18:52:01.819Z caller=head.go:778 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-09-20T18:52:02.184 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:52:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[66989]: ts=2024-09-20T18:52:01.819Z caller=head.go:778 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-09-20T18:52:02.184 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:52:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[66989]: ts=2024-09-20T18:52:01.820Z caller=head.go:815 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=34.002µs wal_replay_duration=1.33849ms wbl_replay_duration=153ns total_replay_duration=1.42394ms 2024-09-20T18:52:02.184 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:52:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[66989]: ts=2024-09-20T18:52:01.823Z caller=main.go:1150 level=info fs_type=XFS_SUPER_MAGIC 2024-09-20T18:52:02.184 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:52:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[66989]: ts=2024-09-20T18:52:01.823Z caller=main.go:1153 level=info msg="TSDB started" 2024-09-20T18:52:02.184 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:52:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[66989]: ts=2024-09-20T18:52:01.823Z caller=main.go:1335 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-20T18:52:02.184 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:52:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[66989]: ts=2024-09-20T18:52:01.851Z caller=main.go:1372 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=28.155047ms db_storage=949ns remote_storage=1.527µs web_handler=418ns query_engine=728ns scrape=709.368µs scrape_sd=155.06µs notify=10.639µs notify_sd=7.73µs rules=26.670867ms tracing=8.557µs 2024-09-20T18:52:02.184 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:52:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[66989]: ts=2024-09-20T18:52:01.851Z caller=main.go:1114 level=info msg="Server is ready to receive web requests." 2024-09-20T18:52:02.184 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:52:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[66989]: ts=2024-09-20T18:52:01.851Z caller=manager.go:163 level=info component="rule manager" msg="Starting rule manager..." 2024-09-20T18:52:02.185 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:52:01 smithi142 bash[66942]: 790d73aeb9d9c2099db6e1fa2bf1b460ea298eb02a70b14f37a146af9298b73a 2024-09-20T18:52:02.185 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 18:52:01 smithi142 systemd[1]: Started Ceph prometheus.a for 1ed5a50a-7780-11ef-baf5-efdc52797490. 2024-09-20T18:52:02.237 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:02.299 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:52:02.299 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:52:02.339 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:52:02.340 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:52:02.341 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: [20/Sep/2024:18:52:02] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-20T18:52:02.341 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: [20/Sep/2024:18:52:02] ENGINE Bus STOPPED 2024-09-20T18:52:02.341 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: [20/Sep/2024:18:52:02] ENGINE Bus STARTING 2024-09-20T18:52:02.341 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: [20/Sep/2024:18:52:02] ENGINE Serving on http://:::9283 2024-09-20T18:52:02.341 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: [20/Sep/2024:18:52:02] ENGINE Bus STARTED 2024-09-20T18:52:02.341 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: [20/Sep/2024:18:52:02] ENGINE Bus STOPPING 2024-09-20T18:52:02.343 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:02.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:02.344 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:02.349 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:52:02.384 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:52:02.390 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:52:02.390 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:52:02.394 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:52:02.441 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:52:02.441 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:52:02.445 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:02.479 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:52:02.480 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:52:02.488 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:02.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:02.489 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:02.515 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:52:02.516 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:52:02.530 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 65 16 98665 98649 94.8291 49.5 0.0164424 0.0105399 2024-09-20T18:52:02.548 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:52:02.548 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:52:02.589 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:52:02.589 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:52:02.590 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:02.627 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:02.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:02.628 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:02.728 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:02.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:02.766 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:02.766 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:02.866 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:02.912 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:02.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:02.913 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:02.932 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-alertmanager-a[76723]: ts=2024-09-20T18:52:02.649Z caller=cluster.go:700 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.005239955s 2024-09-20T18:52:02.932 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: [20/Sep/2024:18:52:02] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-20T18:52:02.932 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: [20/Sep/2024:18:52:02] ENGINE Bus STOPPED 2024-09-20T18:52:02.932 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: [20/Sep/2024:18:52:02] ENGINE Bus STARTING 2024-09-20T18:52:02.932 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: [20/Sep/2024:18:52:02] ENGINE Serving on http://:::9283 2024-09-20T18:52:02.933 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: [20/Sep/2024:18:52:02] ENGINE Bus STARTED 2024-09-20T18:52:02.933 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: [20/Sep/2024:18:52:02] ENGINE Bus STOPPING 2024-09-20T18:52:02.933 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: [20/Sep/2024:18:52:02] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-20T18:52:02.933 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: [20/Sep/2024:18:52:02] ENGINE Bus STOPPED 2024-09-20T18:52:02.933 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: [20/Sep/2024:18:52:02] ENGINE Bus STARTING 2024-09-20T18:52:03.014 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:03.052 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:03.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:03.054 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:03.154 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:03.198 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:03.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:03.199 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:03.276 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:02 smithi142 ceph-mon[32508]: 7.7 scrub starts 2024-09-20T18:52:03.276 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:02 smithi142 ceph-mon[32508]: 4.13 scrub starts 2024-09-20T18:52:03.276 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:02 smithi142 ceph-mon[32508]: 4.13 scrub ok 2024-09-20T18:52:03.276 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:02 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:52:03.276 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:02 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:52:03.276 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:02 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "dashboard get-alertmanager-api-host"} : dispatch 2024-09-20T18:52:03.276 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:02 smithi142 ceph-mon[32508]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-20T18:52:03.276 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:02 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi083.front.sepia.ceph.com:9093"} : dispatch 2024-09-20T18:52:03.277 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:02 smithi142 ceph-mon[32508]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi083.front.sepia.ceph.com:9093"}]: dispatch 2024-09-20T18:52:03.277 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:02 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:52:03.277 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:02 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "dashboard get-prometheus-api-host"} : dispatch 2024-09-20T18:52:03.277 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:02 smithi142 ceph-mon[32508]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-20T18:52:03.277 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:02 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi142.front.sepia.ceph.com:9095"} : dispatch 2024-09-20T18:52:03.277 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:02 smithi142 ceph-mon[32508]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi142.front.sepia.ceph.com:9095"}]: dispatch 2024-09-20T18:52:03.277 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:02 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:52:03.277 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:02 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "dashboard get-grafana-api-url"} : dispatch 2024-09-20T18:52:03.277 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:02 smithi142 ceph-mon[32508]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-20T18:52:03.277 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:02 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "dashboard set-grafana-api-url", "value": "https://smithi142.front.sepia.ceph.com:3000"} : dispatch 2024-09-20T18:52:03.277 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:02 smithi142 ceph-mon[32508]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi142.front.sepia.ceph.com:3000"}]: dispatch 2024-09-20T18:52:03.277 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:02 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:52:03.277 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:02 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-20T18:52:03.277 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:02 smithi142 ceph-mon[32508]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-20T18:52:03.278 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:02 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3274933139' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:52:03.278 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:02 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:52:03.278 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:02 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-20T18:52:03.278 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:02 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-20T18:52:03.299 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:03.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[29421]: 7.7 scrub starts 2024-09-20T18:52:03.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[29421]: 4.13 scrub starts 2024-09-20T18:52:03.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[29421]: 4.13 scrub ok 2024-09-20T18:52:03.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:52:03.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:52:03.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "dashboard get-alertmanager-api-host"} : dispatch 2024-09-20T18:52:03.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[29421]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-20T18:52:03.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi083.front.sepia.ceph.com:9093"} : dispatch 2024-09-20T18:52:03.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[29421]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi083.front.sepia.ceph.com:9093"}]: dispatch 2024-09-20T18:52:03.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:52:03.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "dashboard get-prometheus-api-host"} : dispatch 2024-09-20T18:52:03.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[29421]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-20T18:52:03.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi142.front.sepia.ceph.com:9095"} : dispatch 2024-09-20T18:52:03.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[29421]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi142.front.sepia.ceph.com:9095"}]: dispatch 2024-09-20T18:52:03.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:52:03.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "dashboard get-grafana-api-url"} : dispatch 2024-09-20T18:52:03.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[29421]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-20T18:52:03.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "dashboard set-grafana-api-url", "value": "https://smithi142.front.sepia.ceph.com:3000"} : dispatch 2024-09-20T18:52:03.306 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[29421]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi142.front.sepia.ceph.com:3000"}]: dispatch 2024-09-20T18:52:03.306 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:52:03.306 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-20T18:52:03.306 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[29421]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-20T18:52:03.306 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3274933139' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:52:03.306 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:52:03.306 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-20T18:52:03.306 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-20T18:52:03.307 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[36260]: 7.7 scrub starts 2024-09-20T18:52:03.307 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[36260]: 4.13 scrub starts 2024-09-20T18:52:03.307 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[36260]: 4.13 scrub ok 2024-09-20T18:52:03.307 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:52:03.307 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:52:03.307 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "dashboard get-alertmanager-api-host"} : dispatch 2024-09-20T18:52:03.307 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[36260]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-20T18:52:03.307 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi083.front.sepia.ceph.com:9093"} : dispatch 2024-09-20T18:52:03.307 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[36260]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi083.front.sepia.ceph.com:9093"}]: dispatch 2024-09-20T18:52:03.307 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:52:03.307 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "dashboard get-prometheus-api-host"} : dispatch 2024-09-20T18:52:03.308 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[36260]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-20T18:52:03.308 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi142.front.sepia.ceph.com:9095"} : dispatch 2024-09-20T18:52:03.308 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[36260]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi142.front.sepia.ceph.com:9095"}]: dispatch 2024-09-20T18:52:03.308 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:52:03.308 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "dashboard get-grafana-api-url"} : dispatch 2024-09-20T18:52:03.308 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[36260]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-20T18:52:03.308 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "dashboard set-grafana-api-url", "value": "https://smithi142.front.sepia.ceph.com:3000"} : dispatch 2024-09-20T18:52:03.308 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[36260]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi142.front.sepia.ceph.com:3000"}]: dispatch 2024-09-20T18:52:03.308 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:52:03.308 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-20T18:52:03.308 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[36260]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-20T18:52:03.308 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3274933139' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:52:03.309 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:52:03.309 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-20T18:52:03.309 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-20T18:52:03.309 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: [20/Sep/2024:18:52:02] ENGINE Serving on http://:::9283 2024-09-20T18:52:03.309 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:52:02 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: [20/Sep/2024:18:52:02] ENGINE Bus STARTED 2024-09-20T18:52:03.339 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:03.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:03.340 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:03.441 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:03.471 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:03.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:03.472 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:03.531 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 66 16 99689 99673 94.362 64 0.0198821 0.0105931 2024-09-20T18:52:03.572 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:03.614 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:03.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:03.615 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:03.716 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:03.746 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:03.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:03.747 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:52:03.848 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:03.875 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:03.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:03.876 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:03.976 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:04.022 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:04.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:04.023 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:04.124 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:04.151 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:04 smithi083 ceph-mon[29421]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-20T18:52:04.151 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:04 smithi083 ceph-mon[29421]: pgmap v95: 148 pgs: 148 active+clean; 5.9 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.4 KiB/s rd, 93 MiB/s wr, 1.49k op/s 2024-09-20T18:52:04.152 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:04 smithi083 ceph-mon[29421]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-20T18:52:04.152 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:04 smithi083 ceph-mon[29421]: 4.d scrub starts 2024-09-20T18:52:04.152 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:04 smithi083 ceph-mon[29421]: 4.d scrub ok 2024-09-20T18:52:04.152 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:04 smithi083 ceph-mon[29421]: 5.7 scrub starts 2024-09-20T18:52:04.152 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:04 smithi083 ceph-mon[29421]: 5.7 scrub ok 2024-09-20T18:52:04.152 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:04 smithi083 ceph-mon[29421]: 6.1f scrub starts 2024-09-20T18:52:04.152 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:04 smithi083 ceph-mon[29421]: 6.1f scrub ok 2024-09-20T18:52:04.152 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:04 smithi083 ceph-mon[36260]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-20T18:52:04.152 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:04 smithi083 ceph-mon[36260]: pgmap v95: 148 pgs: 148 active+clean; 5.9 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.4 KiB/s rd, 93 MiB/s wr, 1.49k op/s 2024-09-20T18:52:04.152 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:04 smithi083 ceph-mon[36260]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-20T18:52:04.152 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:04 smithi083 ceph-mon[36260]: 4.d scrub starts 2024-09-20T18:52:04.153 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:04 smithi083 ceph-mon[36260]: 4.d scrub ok 2024-09-20T18:52:04.153 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:04 smithi083 ceph-mon[36260]: 5.7 scrub starts 2024-09-20T18:52:04.153 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:04 smithi083 ceph-mon[36260]: 5.7 scrub ok 2024-09-20T18:52:04.153 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:04 smithi083 ceph-mon[36260]: 6.1f scrub starts 2024-09-20T18:52:04.153 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:04 smithi083 ceph-mon[36260]: 6.1f scrub ok 2024-09-20T18:52:04.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:04.161 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:04.161 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:04.262 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:04.304 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:04.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:04.305 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:04.406 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:52:04.433 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:04.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:04.434 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:52:04.448 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T18:52:04.448 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-20T18:52:04.448 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-20T18:52:04.448 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-20T18:52:04.448 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.448 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:25327 7.3 7:cbe7b718:::benchmark_data_smithi083_70643_object25326:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:52:04.448 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.263028+0000", 2024-09-20T18:52:04.448 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 57.163475476000002, 2024-09-20T18:52:04.448 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.1801133, 2024-09-20T18:52:04.449 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:52:04.449 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:52:04.449 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:52:04.449 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:52:04.449 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:52:04.449 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:52:04.449 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25327 2024-09-20T18:52:04.449 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.449 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:52:04.449 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.449 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:52:04.449 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263028+0000", 2024-09-20T18:52:04.449 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:52:04.450 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.450 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.450 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:52:04.450 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263028+0000", 2024-09-20T18:52:04.450 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:52:04.450 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.450 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.450 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:52:04.450 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263029+0000", 2024-09-20T18:52:04.450 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1200000000000001e-06 2024-09-20T18:52:04.450 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.450 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.450 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:52:04.450 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263050+0000", 2024-09-20T18:52:04.451 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1019999999999999e-05 2024-09-20T18:52:04.451 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.451 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.451 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:52:04.451 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263060+0000", 2024-09-20T18:52:04.451 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0049e-05 2024-09-20T18:52:04.451 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.451 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.451 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:52:04.451 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263069+0000", 2024-09-20T18:52:04.451 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8999999999999995e-06 2024-09-20T18:52:04.451 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.451 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.452 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:52:04.452 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263770+0000", 2024-09-20T18:52:04.452 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00070160100000000005 2024-09-20T18:52:04.452 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.452 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.452 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:52:04.452 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263876+0000", 2024-09-20T18:52:04.452 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010538400000000001 2024-09-20T18:52:04.452 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.452 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.452 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,4", 2024-09-20T18:52:04.452 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263991+0000", 2024-09-20T18:52:04.452 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115151 2024-09-20T18:52:04.453 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.453 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.453 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:52:04.453 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.265911+0000", 2024-09-20T18:52:04.453 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0019197319999999999 2024-09-20T18:52:04.453 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.453 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.453 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:04.453 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.267644+0000", 2024-09-20T18:52:04.453 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0017335600000000001 2024-09-20T18:52:04.453 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.453 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.453 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:04.453 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443038+0000", 2024-09-20T18:52:04.454 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17539418100000001 2024-09-20T18:52:04.454 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.454 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.454 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:52:04.454 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443095+0000", 2024-09-20T18:52:04.454 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6972999999999998e-05 2024-09-20T18:52:04.454 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.454 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.454 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:52:04.454 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443141+0000", 2024-09-20T18:52:04.454 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5629e-05 2024-09-20T18:52:04.454 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:52:04.454 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:52:04.455 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:52:04.455 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.455 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.455 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:25329 7.3 7:c0dcc395:::benchmark_data_smithi083_70643_object25328:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:52:04.455 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.263576+0000", 2024-09-20T18:52:04.455 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 57.162927285999999, 2024-09-20T18:52:04.455 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17968018699999999, 2024-09-20T18:52:04.455 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:52:04.455 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:52:04.455 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:52:04.455 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:52:04.455 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:52:04.455 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:52:04.455 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25329 2024-09-20T18:52:04.456 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.456 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:52:04.456 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.456 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:52:04.456 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263576+0000", 2024-09-20T18:52:04.456 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:52:04.456 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.456 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.456 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:52:04.456 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263576+0000", 2024-09-20T18:52:04.456 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:52:04.456 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.456 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.456 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:52:04.457 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263577+0000", 2024-09-20T18:52:04.457 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.3799999999999996e-07 2024-09-20T18:52:04.457 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.457 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.457 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:52:04.457 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263599+0000", 2024-09-20T18:52:04.457 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2333000000000001e-05 2024-09-20T18:52:04.457 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.457 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.457 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:52:04.457 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263608+0000", 2024-09-20T18:52:04.457 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5690000000000005e-06 2024-09-20T18:52:04.457 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.458 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.458 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:52:04.458 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263614+0000", 2024-09-20T18:52:04.458 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6449999999999999e-06 2024-09-20T18:52:04.458 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.458 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.458 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:52:04.458 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.264856+0000", 2024-09-20T18:52:04.458 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001241486 2024-09-20T18:52:04.458 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.458 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.458 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:52:04.458 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.264959+0000", 2024-09-20T18:52:04.458 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000103037 2024-09-20T18:52:04.459 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.459 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.459 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,4", 2024-09-20T18:52:04.459 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.265077+0000", 2024-09-20T18:52:04.459 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000118163 2024-09-20T18:52:04.459 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.459 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.459 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:52:04.459 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266326+0000", 2024-09-20T18:52:04.459 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001248929 2024-09-20T18:52:04.459 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.459 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.459 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:04.459 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.267696+0000", 2024-09-20T18:52:04.460 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001369924 2024-09-20T18:52:04.460 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.460 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.460 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:04.460 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443180+0000", 2024-09-20T18:52:04.460 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17548422 2024-09-20T18:52:04.460 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.460 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.460 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:52:04.460 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443222+0000", 2024-09-20T18:52:04.460 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2426999999999999e-05 2024-09-20T18:52:04.460 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.460 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.460 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:52:04.461 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443256+0000", 2024-09-20T18:52:04.461 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3615999999999998e-05 2024-09-20T18:52:04.461 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:52:04.461 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:52:04.461 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:52:04.461 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.461 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.461 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:25332 7.2 7:4a99327a:::benchmark_data_smithi083_70643_object25331:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:52:04.461 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.266138+0000", 2024-09-20T18:52:04.461 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 57.160364821999998, 2024-09-20T18:52:04.461 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17949907200000001, 2024-09-20T18:52:04.461 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:52:04.461 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:52:04.461 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:52:04.462 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:52:04.462 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:52:04.462 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:52:04.462 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25332 2024-09-20T18:52:04.462 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.462 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:52:04.462 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.462 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:52:04.462 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266138+0000", 2024-09-20T18:52:04.462 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:52:04.462 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.462 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.463 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:52:04.463 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266138+0000", 2024-09-20T18:52:04.463 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:52:04.463 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.463 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.463 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:52:04.463 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266140+0000", 2024-09-20T18:52:04.463 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1480000000000001e-06 2024-09-20T18:52:04.463 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.463 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.463 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:52:04.463 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266160+0000", 2024-09-20T18:52:04.463 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0908999999999999e-05 2024-09-20T18:52:04.463 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.464 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.464 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:52:04.464 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266170+0000", 2024-09-20T18:52:04.464 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1390000000000001e-06 2024-09-20T18:52:04.464 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.464 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.464 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:52:04.464 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266176+0000", 2024-09-20T18:52:04.464 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3740000000000003e-06 2024-09-20T18:52:04.464 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.464 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.464 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:52:04.464 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266234+0000", 2024-09-20T18:52:04.464 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8084999999999997e-05 2024-09-20T18:52:04.465 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.465 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.465 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:52:04.465 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266342+0000", 2024-09-20T18:52:04.465 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000108448 2024-09-20T18:52:04.465 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.465 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.465 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T18:52:04.465 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266470+0000", 2024-09-20T18:52:04.465 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012756700000000001 2024-09-20T18:52:04.465 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.465 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.465 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:04.466 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444956+0000", 2024-09-20T18:52:04.466 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.178485641 2024-09-20T18:52:04.466 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.466 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.466 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:04.466 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445026+0000", 2024-09-20T18:52:04.466 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0778999999999994e-05 2024-09-20T18:52:04.466 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.466 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.466 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:52:04.466 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445522+0000", 2024-09-20T18:52:04.466 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00049521500000000002 2024-09-20T18:52:04.466 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.467 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.467 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:52:04.467 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445598+0000", 2024-09-20T18:52:04.467 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5993000000000002e-05 2024-09-20T18:52:04.467 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.467 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.467 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:52:04.467 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445637+0000", 2024-09-20T18:52:04.467 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9774000000000002e-05 2024-09-20T18:52:04.467 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:52:04.467 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:52:04.467 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:52:04.467 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.467 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.468 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:25333 7.2 7:4c00f64e:::benchmark_data_smithi083_70643_object25332:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:52:04.468 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.266192+0000", 2024-09-20T18:52:04.468 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 57.160310957, 2024-09-20T18:52:04.468 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17952103699999999, 2024-09-20T18:52:04.468 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:52:04.468 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:52:04.468 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:52:04.468 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:52:04.468 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:52:04.468 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:52:04.468 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25333 2024-09-20T18:52:04.468 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.468 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:52:04.469 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.469 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:52:04.469 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266192+0000", 2024-09-20T18:52:04.469 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:52:04.469 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.469 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.469 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:52:04.469 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266192+0000", 2024-09-20T18:52:04.469 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:52:04.469 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.469 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.469 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:52:04.469 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266193+0000", 2024-09-20T18:52:04.469 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0999999999999998e-07 2024-09-20T18:52:04.470 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.470 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.470 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:52:04.470 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266211+0000", 2024-09-20T18:52:04.470 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8587e-05 2024-09-20T18:52:04.470 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.470 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.470 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:52:04.470 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266218+0000", 2024-09-20T18:52:04.470 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7340000000000002e-06 2024-09-20T18:52:04.470 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.470 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.470 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:52:04.470 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266224+0000", 2024-09-20T18:52:04.471 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6590000000000004e-06 2024-09-20T18:52:04.471 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.471 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.471 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:52:04.471 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268185+0000", 2024-09-20T18:52:04.471 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0019615240000000001 2024-09-20T18:52:04.471 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.471 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.471 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:52:04.471 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268326+0000", 2024-09-20T18:52:04.471 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014032399999999999 2024-09-20T18:52:04.471 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.471 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.471 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T18:52:04.472 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268450+0000", 2024-09-20T18:52:04.472 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012426599999999999 2024-09-20T18:52:04.472 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.472 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.472 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:04.472 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444991+0000", 2024-09-20T18:52:04.472 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17654108199999999 2024-09-20T18:52:04.472 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.472 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.472 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:04.472 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445507+0000", 2024-09-20T18:52:04.472 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00051583599999999999 2024-09-20T18:52:04.472 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.472 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.473 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:52:04.473 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445645+0000", 2024-09-20T18:52:04.473 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000137959 2024-09-20T18:52:04.473 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.473 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.473 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:52:04.473 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445682+0000", 2024-09-20T18:52:04.473 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7455e-05 2024-09-20T18:52:04.473 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.473 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.473 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:52:04.473 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445713+0000", 2024-09-20T18:52:04.473 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1000999999999997e-05 2024-09-20T18:52:04.474 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:52:04.474 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:52:04.474 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:52:04.474 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.474 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.474 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:25336 7.2 7:4be582b4:::benchmark_data_smithi083_70643_object25335:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:52:04.474 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.267773+0000", 2024-09-20T18:52:04.474 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 57.158730103000003, 2024-09-20T18:52:04.474 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.18517899500000001, 2024-09-20T18:52:04.474 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:52:04.474 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:52:04.474 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:52:04.474 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:52:04.474 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:52:04.475 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:52:04.475 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25336 2024-09-20T18:52:04.475 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.475 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:52:04.475 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.475 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:52:04.475 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.267773+0000", 2024-09-20T18:52:04.475 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:52:04.475 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.475 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.475 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:52:04.475 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.267773+0000", 2024-09-20T18:52:04.475 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:52:04.475 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.475 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.476 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:52:04.476 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.267774+0000", 2024-09-20T18:52:04.476 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.13e-06 2024-09-20T18:52:04.477 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.477 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.477 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:52:04.477 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.267797+0000", 2024-09-20T18:52:04.477 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2852000000000001e-05 2024-09-20T18:52:04.477 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.477 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.477 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:52:04.477 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.267806+0000", 2024-09-20T18:52:04.477 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4519999999999997e-06 2024-09-20T18:52:04.477 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.477 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.477 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:52:04.477 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.267812+0000", 2024-09-20T18:52:04.478 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3489999999999997e-06 2024-09-20T18:52:04.478 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.478 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.478 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:52:04.478 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270828+0000", 2024-09-20T18:52:04.478 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0030158020000000002 2024-09-20T18:52:04.478 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.478 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.478 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:52:04.478 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443376+0000", 2024-09-20T18:52:04.478 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17254844 2024-09-20T18:52:04.478 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.478 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.478 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T18:52:04.479 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443495+0000", 2024-09-20T18:52:04.479 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000119338 2024-09-20T18:52:04.479 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.479 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.479 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:52:04.479 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.446411+0000", 2024-09-20T18:52:04.479 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0029155869999999999 2024-09-20T18:52:04.479 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.479 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.479 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:04.479 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.452802+0000", 2024-09-20T18:52:04.479 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063913169999999997 2024-09-20T18:52:04.479 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.479 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.480 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:04.480 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.452870+0000", 2024-09-20T18:52:04.480 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7824999999999995e-05 2024-09-20T18:52:04.480 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.480 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.480 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:52:04.480 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.452915+0000", 2024-09-20T18:52:04.480 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4815999999999999e-05 2024-09-20T18:52:04.480 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.480 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.480 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:52:04.480 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.452952+0000", 2024-09-20T18:52:04.480 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7086999999999999e-05 2024-09-20T18:52:04.480 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:52:04.481 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:52:04.481 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:52:04.481 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.481 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.481 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:25335 7.6 e93/83 7:63a06433:::benchmark_data_smithi083_70643_object25334:head v 93'1581, mlcod=93'1579)", 2024-09-20T18:52:04.481 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.268963+0000", 2024-09-20T18:52:04.481 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 57.157540314000002, 2024-09-20T18:52:04.481 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.173429424, 2024-09-20T18:52:04.481 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:52:04.481 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:52:04.481 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:52:04.481 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:52:04.481 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.482 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:52:04.482 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268963+0000", 2024-09-20T18:52:04.482 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:52:04.482 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.482 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.482 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:52:04.482 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268963+0000", 2024-09-20T18:52:04.482 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:52:04.482 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.482 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.482 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:52:04.482 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268964+0000", 2024-09-20T18:52:04.482 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8800000000000001e-07 2024-09-20T18:52:04.483 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.483 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.483 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:52:04.483 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268994+0000", 2024-09-20T18:52:04.483 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0167000000000001e-05 2024-09-20T18:52:04.483 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.483 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.483 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:52:04.483 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269003+0000", 2024-09-20T18:52:04.483 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4709999999999996e-06 2024-09-20T18:52:04.483 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.483 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.483 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:52:04.484 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269109+0000", 2024-09-20T18:52:04.484 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000105831 2024-09-20T18:52:04.484 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.484 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.484 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:52:04.484 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269160+0000", 2024-09-20T18:52:04.484 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0466000000000001e-05 2024-09-20T18:52:04.484 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.484 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.484 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:52:04.484 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269178+0000", 2024-09-20T18:52:04.484 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8258e-05 2024-09-20T18:52:04.484 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.485 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.485 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:52:04.485 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270796+0000", 2024-09-20T18:52:04.485 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001618445 2024-09-20T18:52:04.485 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.485 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.485 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:52:04.485 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.442392+0000", 2024-09-20T18:52:04.485 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.171595898 2024-09-20T18:52:04.485 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:52:04.485 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:52:04.485 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:52:04.485 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.486 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.486 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:25337 7.f e93/83 7:fd2b3e49:::benchmark_data_smithi083_70643_object25336:head v 93'1586, mlcod=93'1584)", 2024-09-20T18:52:04.486 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.269129+0000", 2024-09-20T18:52:04.486 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 57.157373790999998, 2024-09-20T18:52:04.486 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.174986899, 2024-09-20T18:52:04.486 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:52:04.486 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:52:04.486 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:52:04.486 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:52:04.486 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.486 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:52:04.486 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269129+0000", 2024-09-20T18:52:04.486 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:52:04.486 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.487 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.487 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:52:04.487 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269129+0000", 2024-09-20T18:52:04.487 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:52:04.487 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.487 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.487 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:52:04.487 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269130+0000", 2024-09-20T18:52:04.487 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8000000000000001e-07 2024-09-20T18:52:04.487 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.490 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.490 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:52:04.490 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269151+0000", 2024-09-20T18:52:04.490 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1593000000000001e-05 2024-09-20T18:52:04.490 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.490 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.490 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:52:04.490 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269158+0000", 2024-09-20T18:52:04.490 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1069999999999999e-06 2024-09-20T18:52:04.490 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.490 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.490 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:52:04.490 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269163+0000", 2024-09-20T18:52:04.491 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7029999999999997e-06 2024-09-20T18:52:04.491 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.491 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.491 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:52:04.491 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269349+0000", 2024-09-20T18:52:04.491 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000185331 2024-09-20T18:52:04.491 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.491 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.491 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:52:04.491 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269365+0000", 2024-09-20T18:52:04.491 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6424000000000001e-05 2024-09-20T18:52:04.491 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.491 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.491 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:52:04.492 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271621+0000", 2024-09-20T18:52:04.492 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0022557200000000001 2024-09-20T18:52:04.492 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.492 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.492 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:52:04.492 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444116+0000", 2024-09-20T18:52:04.492 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17249564100000001 2024-09-20T18:52:04.492 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:52:04.492 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:52:04.492 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:52:04.492 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.492 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.493 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:25340 7.6 e93/83 7:6dd1b43c:::benchmark_data_smithi083_70643_object25339:head v 93'1582, mlcod=93'1579)", 2024-09-20T18:52:04.493 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.269179+0000", 2024-09-20T18:52:04.493 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 57.157323949999999, 2024-09-20T18:52:04.493 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.173948506, 2024-09-20T18:52:04.493 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:52:04.493 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:52:04.493 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:52:04.493 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:52:04.493 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.493 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:52:04.493 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269179+0000", 2024-09-20T18:52:04.493 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:52:04.493 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.493 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.494 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:52:04.494 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269179+0000", 2024-09-20T18:52:04.494 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:52:04.494 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.494 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.494 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:52:04.494 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269179+0000", 2024-09-20T18:52:04.494 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.48e-07 2024-09-20T18:52:04.494 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.494 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.494 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:52:04.494 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269198+0000", 2024-09-20T18:52:04.494 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8972999999999999e-05 2024-09-20T18:52:04.494 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.495 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.495 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:52:04.495 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269204+0000", 2024-09-20T18:52:04.495 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0440000000000003e-06 2024-09-20T18:52:04.495 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.495 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.495 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:52:04.495 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269207+0000", 2024-09-20T18:52:04.495 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6789999999999998e-06 2024-09-20T18:52:04.495 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.495 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.495 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:52:04.496 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270103+0000", 2024-09-20T18:52:04.496 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00089533500000000003 2024-09-20T18:52:04.496 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.496 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.496 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:52:04.496 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270120+0000", 2024-09-20T18:52:04.496 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7455000000000001e-05 2024-09-20T18:52:04.496 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.496 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.496 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:52:04.496 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443096+0000", 2024-09-20T18:52:04.496 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17297637699999999 2024-09-20T18:52:04.496 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.496 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.497 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:52:04.497 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443128+0000", 2024-09-20T18:52:04.497 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1395e-05 2024-09-20T18:52:04.497 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:52:04.497 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:52:04.497 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:52:04.497 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.497 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.497 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:25341 7.a 7:5bb50a52:::benchmark_data_smithi083_70643_object25340:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:52:04.497 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.269273+0000", 2024-09-20T18:52:04.497 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 57.157229719999997, 2024-09-20T18:52:04.497 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.18597625000000001, 2024-09-20T18:52:04.497 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:52:04.498 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:52:04.498 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:52:04.498 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:52:04.498 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:52:04.498 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:52:04.498 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25341 2024-09-20T18:52:04.498 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.498 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:52:04.498 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.498 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:52:04.498 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269273+0000", 2024-09-20T18:52:04.498 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:52:04.498 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.499 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.499 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:52:04.499 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269273+0000", 2024-09-20T18:52:04.499 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:52:04.499 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.499 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.499 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:52:04.499 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269274+0000", 2024-09-20T18:52:04.499 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0139999999999999e-06 2024-09-20T18:52:04.499 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.499 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.499 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:52:04.499 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269293+0000", 2024-09-20T18:52:04.499 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8499e-05 2024-09-20T18:52:04.500 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.500 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.500 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:52:04.500 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269301+0000", 2024-09-20T18:52:04.500 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8429999999999993e-06 2024-09-20T18:52:04.500 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.500 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.500 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:52:04.500 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269306+0000", 2024-09-20T18:52:04.500 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2809999999999999e-06 2024-09-20T18:52:04.500 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.500 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.500 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:52:04.500 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445057+0000", 2024-09-20T18:52:04.501 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17575099 2024-09-20T18:52:04.501 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.501 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.501 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:52:04.501 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445166+0000", 2024-09-20T18:52:04.501 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000109335 2024-09-20T18:52:04.501 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.501 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.501 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,5", 2024-09-20T18:52:04.501 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445293+0000", 2024-09-20T18:52:04.501 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000126995 2024-09-20T18:52:04.501 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.501 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.501 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:04.502 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.452767+0000", 2024-09-20T18:52:04.502 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074736009999999999 2024-09-20T18:52:04.502 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.502 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.502 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:04.502 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.452832+0000", 2024-09-20T18:52:04.502 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4817e-05 2024-09-20T18:52:04.502 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.502 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.502 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:52:04.502 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.455135+0000", 2024-09-20T18:52:04.502 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0023034459999999998 2024-09-20T18:52:04.502 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.502 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.503 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:52:04.503 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.455211+0000", 2024-09-20T18:52:04.503 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5896999999999995e-05 2024-09-20T18:52:04.503 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.503 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.503 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:52:04.503 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.455250+0000", 2024-09-20T18:52:04.503 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8532000000000003e-05 2024-09-20T18:52:04.503 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:52:04.503 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:52:04.503 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:52:04.503 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.503 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.503 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:25327 7.3 e93/83 ondisk, result = 0)", 2024-09-20T18:52:04.504 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.270007+0000", 2024-09-20T18:52:04.504 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 57.156495816000003, 2024-09-20T18:52:04.504 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17314204, 2024-09-20T18:52:04.504 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:52:04.504 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:52:04.504 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:52:04.504 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:52:04.504 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.504 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:52:04.504 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270007+0000", 2024-09-20T18:52:04.504 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:52:04.504 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.504 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.504 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:52:04.505 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270007+0000", 2024-09-20T18:52:04.505 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:52:04.505 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.505 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.505 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:52:04.505 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270008+0000", 2024-09-20T18:52:04.505 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4300000000000002e-07 2024-09-20T18:52:04.505 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.505 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.505 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:52:04.505 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270013+0000", 2024-09-20T18:52:04.505 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7230000000000004e-06 2024-09-20T18:52:04.505 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.505 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.505 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:52:04.506 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270023+0000", 2024-09-20T18:52:04.506 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9520000000000006e-06 2024-09-20T18:52:04.506 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.506 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.506 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:52:04.506 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270030+0000", 2024-09-20T18:52:04.506 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1799999999999999e-06 2024-09-20T18:52:04.506 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.506 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.506 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:52:04.506 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443024+0000", 2024-09-20T18:52:04.506 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17299360799999999 2024-09-20T18:52:04.506 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.506 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.507 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:52:04.507 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443032+0000", 2024-09-20T18:52:04.507 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6009999999999995e-06 2024-09-20T18:52:04.507 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.507 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.507 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:52:04.507 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443149+0000", 2024-09-20T18:52:04.507 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000117133 2024-09-20T18:52:04.507 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:52:04.507 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:52:04.507 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:52:04.507 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.507 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.508 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:25329 7.3 e93/83 ondisk, result = 0)", 2024-09-20T18:52:04.508 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.270049+0000", 2024-09-20T18:52:04.508 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 57.156453718999998, 2024-09-20T18:52:04.508 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17321238999999999, 2024-09-20T18:52:04.508 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:52:04.508 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:52:04.508 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:52:04.508 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:52:04.508 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.508 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:52:04.508 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270049+0000", 2024-09-20T18:52:04.508 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:52:04.508 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.509 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.509 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:52:04.509 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270049+0000", 2024-09-20T18:52:04.509 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:52:04.509 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.509 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.509 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:52:04.509 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270050+0000", 2024-09-20T18:52:04.509 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1699999999999999e-07 2024-09-20T18:52:04.509 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.509 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.509 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:52:04.509 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270052+0000", 2024-09-20T18:52:04.509 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1880000000000001e-06 2024-09-20T18:52:04.510 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.510 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.510 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:52:04.510 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270058+0000", 2024-09-20T18:52:04.510 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6910000000000002e-06 2024-09-20T18:52:04.510 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.510 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.510 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:52:04.510 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270062+0000", 2024-09-20T18:52:04.510 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6840000000000004e-06 2024-09-20T18:52:04.510 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.510 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.510 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:52:04.510 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443169+0000", 2024-09-20T18:52:04.511 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.173106484 2024-09-20T18:52:04.511 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.511 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.511 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:52:04.511 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443175+0000", 2024-09-20T18:52:04.511 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5509999999999996e-06 2024-09-20T18:52:04.511 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.511 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.511 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:52:04.511 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443262+0000", 2024-09-20T18:52:04.511 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6475000000000001e-05 2024-09-20T18:52:04.511 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:52:04.511 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:52:04.511 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:52:04.512 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.512 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.512 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:25332 7.2 e93/83 ondisk, result = 0)", 2024-09-20T18:52:04.512 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.270341+0000", 2024-09-20T18:52:04.512 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 57.156162658, 2024-09-20T18:52:04.512 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.174621416, 2024-09-20T18:52:04.512 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:52:04.512 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:52:04.512 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:52:04.512 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:52:04.512 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.512 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:52:04.512 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270341+0000", 2024-09-20T18:52:04.512 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:52:04.513 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.513 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.513 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:52:04.513 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270341+0000", 2024-09-20T18:52:04.513 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:52:04.513 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.513 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.513 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:52:04.513 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270341+0000", 2024-09-20T18:52:04.513 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6000000000000004e-07 2024-09-20T18:52:04.513 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.513 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.513 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:52:04.514 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270345+0000", 2024-09-20T18:52:04.514 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.444e-06 2024-09-20T18:52:04.514 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.514 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.514 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:52:04.514 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270353+0000", 2024-09-20T18:52:04.514 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5580000000000001e-06 2024-09-20T18:52:04.514 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.514 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.514 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:52:04.514 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270359+0000", 2024-09-20T18:52:04.514 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7459999999999997e-06 2024-09-20T18:52:04.514 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.514 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.515 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:52:04.515 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444944+0000", 2024-09-20T18:52:04.515 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.174585351 2024-09-20T18:52:04.515 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.515 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.515 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:52:04.515 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444950+0000", 2024-09-20T18:52:04.515 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0530000000000003e-06 2024-09-20T18:52:04.515 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.515 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.515 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:52:04.515 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444962+0000", 2024-09-20T18:52:04.515 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1704e-05 2024-09-20T18:52:04.516 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:52:04.516 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:52:04.516 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:52:04.516 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.516 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.516 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:25333 7.2 e93/83 ondisk, result = 0)", 2024-09-20T18:52:04.516 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.270458+0000", 2024-09-20T18:52:04.516 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 57.156045266, 2024-09-20T18:52:04.516 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17453908100000001, 2024-09-20T18:52:04.516 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:52:04.516 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:52:04.516 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:52:04.516 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:52:04.517 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.517 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:52:04.517 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270458+0000", 2024-09-20T18:52:04.517 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:52:04.517 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.517 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.517 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:52:04.517 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270458+0000", 2024-09-20T18:52:04.517 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:52:04.517 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.517 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.517 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:52:04.517 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270458+0000", 2024-09-20T18:52:04.517 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3499999999999996e-07 2024-09-20T18:52:04.517 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.518 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.518 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:52:04.518 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270462+0000", 2024-09-20T18:52:04.518 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1499999999999999e-06 2024-09-20T18:52:04.518 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.518 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.518 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:52:04.518 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270469+0000", 2024-09-20T18:52:04.518 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.695e-06 2024-09-20T18:52:04.518 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.518 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.518 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:52:04.518 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270475+0000", 2024-09-20T18:52:04.519 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5470000000000004e-06 2024-09-20T18:52:04.519 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.519 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.519 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:52:04.519 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444981+0000", 2024-09-20T18:52:04.519 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17450622299999999 2024-09-20T18:52:04.519 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.519 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.519 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:52:04.519 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444987+0000", 2024-09-20T18:52:04.519 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5219999999999998e-06 2024-09-20T18:52:04.519 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.519 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.519 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:52:04.520 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444997+0000", 2024-09-20T18:52:04.520 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0409e-05 2024-09-20T18:52:04.520 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:52:04.520 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:52:04.520 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:52:04.520 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.520 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.520 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:25332 7.2 e93/83 ondisk, result = 0)", 2024-09-20T18:52:04.520 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.270547+0000", 2024-09-20T18:52:04.520 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 57.155956584000002, 2024-09-20T18:52:04.520 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.174485849, 2024-09-20T18:52:04.520 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:52:04.520 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:52:04.521 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:52:04.521 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:52:04.521 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.521 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:52:04.521 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270547+0000", 2024-09-20T18:52:04.521 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:52:04.521 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.521 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.521 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:52:04.521 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270547+0000", 2024-09-20T18:52:04.521 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:52:04.521 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.521 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.522 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:52:04.522 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270547+0000", 2024-09-20T18:52:04.522 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7599999999999997e-07 2024-09-20T18:52:04.522 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.522 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.522 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:52:04.522 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270550+0000", 2024-09-20T18:52:04.523 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7470000000000001e-06 2024-09-20T18:52:04.523 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.523 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.523 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:52:04.523 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270557+0000", 2024-09-20T18:52:04.523 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2620000000000001e-06 2024-09-20T18:52:04.523 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.523 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.523 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:52:04.524 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270562+0000", 2024-09-20T18:52:04.524 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1839999999999998e-06 2024-09-20T18:52:04.524 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.524 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.524 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:52:04.524 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445016+0000", 2024-09-20T18:52:04.524 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17445396699999999 2024-09-20T18:52:04.524 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.524 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.524 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:52:04.524 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445022+0000", 2024-09-20T18:52:04.524 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7559999999999996e-06 2024-09-20T18:52:04.524 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.524 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.525 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:52:04.525 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445032+0000", 2024-09-20T18:52:04.525 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0457e-05 2024-09-20T18:52:04.525 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:52:04.525 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:52:04.525 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:52:04.525 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.525 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.525 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:25343 7.6 e93/83 7:61f2dd3b:::benchmark_data_smithi083_70643_object25342:head v 93'1583, mlcod=93'1579)", 2024-09-20T18:52:04.525 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.271493+0000", 2024-09-20T18:52:04.525 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 57.155010515999997, 2024-09-20T18:52:04.525 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17491087999999999, 2024-09-20T18:52:04.525 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:52:04.526 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:52:04.526 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:52:04.526 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:52:04.526 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.526 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:52:04.526 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271493+0000", 2024-09-20T18:52:04.526 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:52:04.526 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.526 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.526 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:52:04.526 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271493+0000", 2024-09-20T18:52:04.526 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:52:04.527 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.527 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.527 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:52:04.527 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271493+0000", 2024-09-20T18:52:04.527 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8400000000000004e-07 2024-09-20T18:52:04.527 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.527 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.527 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:52:04.527 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271513+0000", 2024-09-20T18:52:04.527 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0126999999999999e-05 2024-09-20T18:52:04.527 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.527 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.527 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:52:04.528 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443960+0000", 2024-09-20T18:52:04.528 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.172446983 2024-09-20T18:52:04.528 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.528 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.528 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:52:04.528 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443970+0000", 2024-09-20T18:52:04.528 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5119999999999997e-06 2024-09-20T18:52:04.528 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.528 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.528 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:52:04.528 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444547+0000", 2024-09-20T18:52:04.528 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00057698700000000005 2024-09-20T18:52:04.528 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.528 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.529 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:52:04.529 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444578+0000", 2024-09-20T18:52:04.529 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0956e-05 2024-09-20T18:52:04.529 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.529 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.529 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:52:04.529 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.446361+0000", 2024-09-20T18:52:04.529 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001783473 2024-09-20T18:52:04.529 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.529 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.529 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:52:04.529 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.446404+0000", 2024-09-20T18:52:04.529 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2157999999999999e-05 2024-09-20T18:52:04.530 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:52:04.530 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:52:04.530 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:52:04.530 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.530 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.530 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:25344 7.7 e93/83 7:ead30b22:::benchmark_data_smithi083_70643_object25343:head v 93'1577, mlcod=93'1575)", 2024-09-20T18:52:04.530 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.271541+0000", 2024-09-20T18:52:04.530 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 57.154962144999999, 2024-09-20T18:52:04.530 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17264948199999999, 2024-09-20T18:52:04.530 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:52:04.530 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:52:04.530 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:52:04.530 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:52:04.530 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.531 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:52:04.531 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271541+0000", 2024-09-20T18:52:04.531 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:52:04.531 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.531 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.531 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:52:04.531 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271541+0000", 2024-09-20T18:52:04.531 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:52:04.531 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.531 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.531 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:52:04.531 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271541+0000", 2024-09-20T18:52:04.531 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1300000000000001e-07 2024-09-20T18:52:04.531 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.532 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.532 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:52:04.532 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271571+0000", 2024-09-20T18:52:04.532 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9949000000000001e-05 2024-09-20T18:52:04.532 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.532 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.532 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:52:04.532 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.442465+0000", 2024-09-20T18:52:04.532 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.170894087 2024-09-20T18:52:04.532 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.532 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.533 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:52:04.533 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.442479+0000", 2024-09-20T18:52:04.533 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.309e-05 2024-09-20T18:52:04.533 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.533 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.533 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:52:04.533 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.442626+0000", 2024-09-20T18:52:04.533 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014740399999999999 2024-09-20T18:52:04.533 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.533 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.533 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:52:04.533 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.442649+0000", 2024-09-20T18:52:04.533 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3107000000000001e-05 2024-09-20T18:52:04.533 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.534 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.534 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:52:04.534 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444120+0000", 2024-09-20T18:52:04.534 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001471354 2024-09-20T18:52:04.534 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.534 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.534 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:52:04.534 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444191+0000", 2024-09-20T18:52:04.534 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0078000000000002e-05 2024-09-20T18:52:04.534 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:52:04.534 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:52:04.534 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:52:04.534 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.535 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.535 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:26458 7.2 7:4defa689:::benchmark_data_smithi083_70643_object26457:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:52:04.535 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.993077+0000", 2024-09-20T18:52:04.535 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 56.433426007999998, 2024-09-20T18:52:04.535 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.139540674, 2024-09-20T18:52:04.535 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:52:04.535 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:52:04.535 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:52:04.535 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:52:04.535 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:52:04.535 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:52:04.535 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 26458 2024-09-20T18:52:04.535 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.535 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:52:04.536 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.536 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:52:04.536 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.993077+0000", 2024-09-20T18:52:04.536 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:52:04.536 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.536 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.536 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:52:04.536 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.993077+0000", 2024-09-20T18:52:04.536 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:52:04.536 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.536 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.536 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:52:04.536 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.993078+0000", 2024-09-20T18:52:04.536 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.167e-06 2024-09-20T18:52:04.537 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.537 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.537 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:52:04.537 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.993097+0000", 2024-09-20T18:52:04.537 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8777999999999999e-05 2024-09-20T18:52:04.537 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.537 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.537 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:52:04.537 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.993107+0000", 2024-09-20T18:52:04.537 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9189999999999994e-06 2024-09-20T18:52:04.537 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.537 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.537 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:52:04.537 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.993114+0000", 2024-09-20T18:52:04.538 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1180000000000003e-06 2024-09-20T18:52:04.538 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.538 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.538 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:52:04.538 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.995113+0000", 2024-09-20T18:52:04.538 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001999055 2024-09-20T18:52:04.538 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.538 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.538 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:52:04.538 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.995230+0000", 2024-09-20T18:52:04.538 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011663 2024-09-20T18:52:04.538 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.538 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.539 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T18:52:04.539 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.995352+0000", 2024-09-20T18:52:04.539 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012200499999999999 2024-09-20T18:52:04.539 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.539 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.539 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:04.539 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.998898+0000", 2024-09-20T18:52:04.539 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0035456929999999999 2024-09-20T18:52:04.539 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.539 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.539 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:04.539 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.000083+0000", 2024-09-20T18:52:04.539 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001185834 2024-09-20T18:52:04.539 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.540 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.540 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:52:04.540 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.132527+0000", 2024-09-20T18:52:04.540 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.132443637 2024-09-20T18:52:04.540 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.540 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.540 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:52:04.540 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.132576+0000", 2024-09-20T18:52:04.540 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8885999999999999e-05 2024-09-20T18:52:04.540 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.540 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.540 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:52:04.540 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.132618+0000", 2024-09-20T18:52:04.540 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1952000000000001e-05 2024-09-20T18:52:04.541 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:52:04.541 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:52:04.541 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:52:04.541 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.541 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.541 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:26459 7.a 7:585d31b8:::benchmark_data_smithi083_70643_object26458:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:52:04.541 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.993130+0000", 2024-09-20T18:52:04.541 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 56.433372976999998, 2024-09-20T18:52:04.541 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.13957813499999999, 2024-09-20T18:52:04.541 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:52:04.541 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:52:04.541 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:52:04.541 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:52:04.542 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:52:04.542 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:52:04.542 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 26459 2024-09-20T18:52:04.542 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.542 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:52:04.542 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.542 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:52:04.542 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.993130+0000", 2024-09-20T18:52:04.542 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:52:04.542 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.542 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.542 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:52:04.542 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.993130+0000", 2024-09-20T18:52:04.542 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:52:04.543 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.543 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.543 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:52:04.543 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.993131+0000", 2024-09-20T18:52:04.543 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2900000000000003e-07 2024-09-20T18:52:04.543 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.543 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.543 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:52:04.543 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.993152+0000", 2024-09-20T18:52:04.543 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1019e-05 2024-09-20T18:52:04.543 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.543 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.543 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:52:04.544 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.993159+0000", 2024-09-20T18:52:04.544 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0999999999999998e-06 2024-09-20T18:52:04.544 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.544 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.544 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:52:04.544 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.993165+0000", 2024-09-20T18:52:04.544 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6500000000000001e-06 2024-09-20T18:52:04.544 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.544 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.544 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:52:04.544 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.998927+0000", 2024-09-20T18:52:04.544 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0057625829999999999 2024-09-20T18:52:04.544 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.544 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.545 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:52:04.545 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.999033+0000", 2024-09-20T18:52:04.545 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010542100000000001 2024-09-20T18:52:04.545 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.545 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.545 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,5", 2024-09-20T18:52:04.545 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.999150+0000", 2024-09-20T18:52:04.545 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000117345 2024-09-20T18:52:04.545 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.545 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.545 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:04.545 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.131620+0000", 2024-09-20T18:52:04.545 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.13246966499999999 2024-09-20T18:52:04.545 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.545 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.546 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:04.546 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.132508+0000", 2024-09-20T18:52:04.546 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00088801799999999997 2024-09-20T18:52:04.546 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.546 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.546 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:52:04.546 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.132644+0000", 2024-09-20T18:52:04.546 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000136422 2024-09-20T18:52:04.546 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.546 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.546 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:52:04.546 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.132680+0000", 2024-09-20T18:52:04.546 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6183999999999998e-05 2024-09-20T18:52:04.546 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.547 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.547 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:52:04.547 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.132708+0000", 2024-09-20T18:52:04.547 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8099e-05 2024-09-20T18:52:04.547 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:52:04.547 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:52:04.547 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:52:04.547 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.547 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.547 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:26460 7.3 7:cd1ed93e:::benchmark_data_smithi083_70643_object26459:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:52:04.547 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.994908+0000", 2024-09-20T18:52:04.547 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 56.431595647000002, 2024-09-20T18:52:04.547 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.137972241, 2024-09-20T18:52:04.547 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:52:04.548 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:52:04.548 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:52:04.548 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:52:04.548 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:52:04.548 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:52:04.548 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 26460 2024-09-20T18:52:04.548 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.548 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:52:04.548 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.548 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:52:04.548 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.994908+0000", 2024-09-20T18:52:04.548 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:52:04.548 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.548 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.549 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:52:04.549 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.994908+0000", 2024-09-20T18:52:04.549 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:52:04.549 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.549 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.549 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:52:04.549 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.994909+0000", 2024-09-20T18:52:04.549 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.246e-06 2024-09-20T18:52:04.549 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.549 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.549 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:52:04.549 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.994935+0000", 2024-09-20T18:52:04.549 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.603e-05 2024-09-20T18:52:04.549 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.550 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.550 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:52:04.550 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.994943+0000", 2024-09-20T18:52:04.550 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2679999999999994e-06 2024-09-20T18:52:04.550 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.550 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.550 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:52:04.550 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.994948+0000", 2024-09-20T18:52:04.550 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6800000000000001e-06 2024-09-20T18:52:04.550 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.550 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.550 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:52:04.550 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.994996+0000", 2024-09-20T18:52:04.550 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8452999999999998e-05 2024-09-20T18:52:04.551 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.551 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.551 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:52:04.551 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.995099+0000", 2024-09-20T18:52:04.551 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000102921 2024-09-20T18:52:04.551 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.551 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.551 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,4", 2024-09-20T18:52:04.551 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.995384+0000", 2024-09-20T18:52:04.551 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00028438699999999999 2024-09-20T18:52:04.551 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.551 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.551 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:04.551 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.998671+0000", 2024-09-20T18:52:04.552 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0032877560000000002 2024-09-20T18:52:04.552 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.552 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.552 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:04.552 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.998703+0000", 2024-09-20T18:52:04.552 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2107000000000003e-05 2024-09-20T18:52:04.552 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.552 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.552 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:52:04.552 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.132742+0000", 2024-09-20T18:52:04.552 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.13403827200000001 2024-09-20T18:52:04.552 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.552 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.552 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:52:04.553 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.132782+0000", 2024-09-20T18:52:04.553 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9888999999999997e-05 2024-09-20T18:52:04.553 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.553 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.553 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:52:04.553 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.132880+0000", 2024-09-20T18:52:04.553 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8232000000000004e-05 2024-09-20T18:52:04.553 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:52:04.553 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:52:04.553 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:52:04.553 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.553 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.553 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:26461 7.2 7:469ac956:::benchmark_data_smithi083_70643_object26460:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:52:04.553 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.994961+0000", 2024-09-20T18:52:04.553 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 56.431541863, 2024-09-20T18:52:04.554 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.13992532699999999, 2024-09-20T18:52:04.554 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:52:04.554 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:52:04.554 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:52:04.554 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:52:04.554 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:52:04.554 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:52:04.554 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 26461 2024-09-20T18:52:04.554 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.554 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:52:04.554 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.554 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:52:04.554 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.994961+0000", 2024-09-20T18:52:04.554 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:52:04.555 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.555 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.555 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:52:04.555 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.994961+0000", 2024-09-20T18:52:04.555 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:52:04.555 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.555 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.555 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:52:04.555 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.994962+0000", 2024-09-20T18:52:04.555 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7399999999999999e-07 2024-09-20T18:52:04.555 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.555 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.555 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:52:04.556 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.994978+0000", 2024-09-20T18:52:04.556 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6024000000000001e-05 2024-09-20T18:52:04.556 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.556 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.556 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:52:04.556 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.994982+0000", 2024-09-20T18:52:04.556 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1080000000000003e-06 2024-09-20T18:52:04.556 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.556 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.556 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:52:04.556 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.994985+0000", 2024-09-20T18:52:04.556 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.084e-06 2024-09-20T18:52:04.556 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.556 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.556 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:52:04.556 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.131132+0000", 2024-09-20T18:52:04.557 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.13614691900000001 2024-09-20T18:52:04.557 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.557 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.557 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:52:04.557 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.131257+0000", 2024-09-20T18:52:04.557 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012540699999999999 2024-09-20T18:52:04.557 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.557 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.557 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T18:52:04.557 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.131387+0000", 2024-09-20T18:52:04.557 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012971700000000001 2024-09-20T18:52:04.557 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.557 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.557 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:52:04.558 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.132716+0000", 2024-09-20T18:52:04.558 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001329507 2024-09-20T18:52:04.558 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.558 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.558 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:04.558 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.133133+0000", 2024-09-20T18:52:04.558 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00041698400000000002 2024-09-20T18:52:04.558 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.558 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.558 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:04.558 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.134801+0000", 2024-09-20T18:52:04.558 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001667511 2024-09-20T18:52:04.558 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.558 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.558 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:52:04.559 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.134845+0000", 2024-09-20T18:52:04.559 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3884000000000002e-05 2024-09-20T18:52:04.559 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:52:04.559 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:52:04.559 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:52:04.559 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.134887+0000", 2024-09-20T18:52:04.559 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1807999999999997e-05 2024-09-20T18:52:04.559 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:52:04.559 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:52:04.559 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:52:04.559 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:52:04.559 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:52:04.559 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T18:52:04.560 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:04.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:04 smithi142 ceph-mon[32508]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-20T18:52:04.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:04 smithi142 ceph-mon[32508]: pgmap v95: 148 pgs: 148 active+clean; 5.9 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.4 KiB/s rd, 93 MiB/s wr, 1.49k op/s 2024-09-20T18:52:04.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:04 smithi142 ceph-mon[32508]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-20T18:52:04.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:04 smithi142 ceph-mon[32508]: 4.d scrub starts 2024-09-20T18:52:04.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:04 smithi142 ceph-mon[32508]: 4.d scrub ok 2024-09-20T18:52:04.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:04 smithi142 ceph-mon[32508]: 5.7 scrub starts 2024-09-20T18:52:04.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:04 smithi142 ceph-mon[32508]: 5.7 scrub ok 2024-09-20T18:52:04.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:04 smithi142 ceph-mon[32508]: 6.1f scrub starts 2024-09-20T18:52:04.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:04 smithi142 ceph-mon[32508]: 6.1f scrub ok 2024-09-20T18:52:04.561 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 67 16 100689 100673 93.8865 62.5 0.0216408 0.0106455 2024-09-20T18:52:04.587 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:04.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:04.588 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:04.689 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:04.722 INFO:teuthology.orchestra.run.smithi083.stderr:no valid command found; 10 closest matches: 2024-09-20T18:52:04.722 INFO:teuthology.orchestra.run.smithi083.stderr:0 2024-09-20T18:52:04.722 INFO:teuthology.orchestra.run.smithi083.stderr:1 2024-09-20T18:52:04.723 INFO:teuthology.orchestra.run.smithi083.stderr:2 2024-09-20T18:52:04.723 INFO:teuthology.orchestra.run.smithi083.stderr:abort 2024-09-20T18:52:04.723 INFO:teuthology.orchestra.run.smithi083.stderr:assert 2024-09-20T18:52:04.723 INFO:teuthology.orchestra.run.smithi083.stderr:bench [] [] [] [] 2024-09-20T18:52:04.723 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs debug_inject_read_zeros 2024-09-20T18:52:04.723 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs files list 2024-09-20T18:52:04.723 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs stats 2024-09-20T18:52:04.723 INFO:teuthology.orchestra.run.smithi083.stderr:bluestore allocator dump block 2024-09-20T18:52:04.723 INFO:teuthology.orchestra.run.smithi083.stderr:Error EINVAL: invalid command 2024-09-20T18:52:04.726 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:04.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:04.727 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:04.829 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:04.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:04.872 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:04.872 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:04.973 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:05.013 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:05.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:05.015 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:05.069 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-20T18:52:05.115 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:05.151 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:05.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:05.152 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:05.253 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:05.283 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:05.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:05.284 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:05.306 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:05 smithi083 ceph-mon[29421]: 7.7 scrub ok 2024-09-20T18:52:05.306 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:05 smithi083 ceph-mon[29421]: 3.f scrub starts 2024-09-20T18:52:05.306 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:05 smithi083 ceph-mon[29421]: 3.f scrub ok 2024-09-20T18:52:05.306 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:05 smithi083 ceph-mon[29421]: 5.16 scrub starts 2024-09-20T18:52:05.306 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:05 smithi083 ceph-mon[29421]: 5.16 scrub ok 2024-09-20T18:52:05.306 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:05 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1739558999' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:52:05.307 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:05 smithi083 ceph-mon[36260]: 7.7 scrub ok 2024-09-20T18:52:05.307 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:05 smithi083 ceph-mon[36260]: 3.f scrub starts 2024-09-20T18:52:05.307 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:05 smithi083 ceph-mon[36260]: 3.f scrub ok 2024-09-20T18:52:05.307 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:05 smithi083 ceph-mon[36260]: 5.16 scrub starts 2024-09-20T18:52:05.307 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:05 smithi083 ceph-mon[36260]: 5.16 scrub ok 2024-09-20T18:52:05.307 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:05 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1739558999' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:52:05.384 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:05.388 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T18:52:05.411 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:05.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:05.412 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:05.513 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:05.523 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:05 smithi142 ceph-mon[32508]: 7.7 scrub ok 2024-09-20T18:52:05.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:05 smithi142 ceph-mon[32508]: 3.f scrub starts 2024-09-20T18:52:05.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:05 smithi142 ceph-mon[32508]: 3.f scrub ok 2024-09-20T18:52:05.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:05 smithi142 ceph-mon[32508]: 5.16 scrub starts 2024-09-20T18:52:05.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:05 smithi142 ceph-mon[32508]: 5.16 scrub ok 2024-09-20T18:52:05.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:05 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1739558999' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:52:05.531 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 68 16 101749 101733 93.4792 66.25 0.0163356 0.0106926 2024-09-20T18:52:05.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:05.567 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:05.568 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:05.668 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:05.694 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:05.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:05.695 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:05.796 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:05.843 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:05.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:05.844 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:05.945 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:05.973 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:05.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:05.974 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:52:06.075 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:06.109 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:06.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:06.110 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:06.211 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:06.238 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:06.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:06.239 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:06.340 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:06.370 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:06.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:06.371 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:06.453 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:06 smithi142 ceph-mon[32508]: 5.2 scrub starts 2024-09-20T18:52:06.453 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:06 smithi142 ceph-mon[32508]: 5.2 scrub ok 2024-09-20T18:52:06.453 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:06 smithi142 ceph-mon[32508]: 3.e scrub starts 2024-09-20T18:52:06.453 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:06 smithi142 ceph-mon[32508]: 3.e scrub ok 2024-09-20T18:52:06.453 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:06 smithi142 ceph-mon[32508]: pgmap v96: 148 pgs: 148 active+clean; 5.9 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 42 MiB/s wr, 665 op/s 2024-09-20T18:52:06.453 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:06 smithi142 ceph-mon[32508]: 6.c scrub starts 2024-09-20T18:52:06.453 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:06 smithi142 ceph-mon[32508]: 6.c scrub ok 2024-09-20T18:52:06.453 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:06 smithi142 ceph-mon[32508]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T18:52:06.453 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:06 smithi142 ceph-mon[32508]: Cluster is now healthy 2024-09-20T18:52:06.453 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:06 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1739558999' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T18:52:06.453 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:06 smithi142 ceph-mon[32508]: osdmap e113: 8 total, 8 up, 8 in 2024-09-20T18:52:06.454 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:06 smithi142 ceph-mon[32508]: osdmap e114: 8 total, 8 up, 8 in 2024-09-20T18:52:06.454 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:06 smithi142 ceph-mon[32508]: 5.a scrub starts 2024-09-20T18:52:06.454 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:06 smithi142 ceph-mon[32508]: 5.a scrub ok 2024-09-20T18:52:06.472 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:06.502 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:06.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:06.503 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:06.519 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:06 smithi083 ceph-mon[29421]: 5.2 scrub starts 2024-09-20T18:52:06.519 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:06 smithi083 ceph-mon[29421]: 5.2 scrub ok 2024-09-20T18:52:06.519 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:06 smithi083 ceph-mon[29421]: 3.e scrub starts 2024-09-20T18:52:06.519 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:06 smithi083 ceph-mon[29421]: 3.e scrub ok 2024-09-20T18:52:06.520 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:06 smithi083 ceph-mon[29421]: pgmap v96: 148 pgs: 148 active+clean; 5.9 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 42 MiB/s wr, 665 op/s 2024-09-20T18:52:06.520 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:06 smithi083 ceph-mon[29421]: 6.c scrub starts 2024-09-20T18:52:06.520 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:06 smithi083 ceph-mon[29421]: 6.c scrub ok 2024-09-20T18:52:06.520 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:06 smithi083 ceph-mon[29421]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T18:52:06.520 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:06 smithi083 ceph-mon[29421]: Cluster is now healthy 2024-09-20T18:52:06.520 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:06 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1739558999' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T18:52:06.520 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:06 smithi083 ceph-mon[29421]: osdmap e113: 8 total, 8 up, 8 in 2024-09-20T18:52:06.520 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:06 smithi083 ceph-mon[29421]: osdmap e114: 8 total, 8 up, 8 in 2024-09-20T18:52:06.520 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:06 smithi083 ceph-mon[29421]: 5.a scrub starts 2024-09-20T18:52:06.520 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:06 smithi083 ceph-mon[29421]: 5.a scrub ok 2024-09-20T18:52:06.520 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:06 smithi083 ceph-mon[36260]: 5.2 scrub starts 2024-09-20T18:52:06.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:06 smithi083 ceph-mon[36260]: 5.2 scrub ok 2024-09-20T18:52:06.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:06 smithi083 ceph-mon[36260]: 3.e scrub starts 2024-09-20T18:52:06.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:06 smithi083 ceph-mon[36260]: 3.e scrub ok 2024-09-20T18:52:06.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:06 smithi083 ceph-mon[36260]: pgmap v96: 148 pgs: 148 active+clean; 5.9 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 42 MiB/s wr, 665 op/s 2024-09-20T18:52:06.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:06 smithi083 ceph-mon[36260]: 6.c scrub starts 2024-09-20T18:52:06.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:06 smithi083 ceph-mon[36260]: 6.c scrub ok 2024-09-20T18:52:06.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:06 smithi083 ceph-mon[36260]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T18:52:06.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:06 smithi083 ceph-mon[36260]: Cluster is now healthy 2024-09-20T18:52:06.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:06 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1739558999' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T18:52:06.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:06 smithi083 ceph-mon[36260]: osdmap e113: 8 total, 8 up, 8 in 2024-09-20T18:52:06.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:06 smithi083 ceph-mon[36260]: osdmap e114: 8 total, 8 up, 8 in 2024-09-20T18:52:06.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:06 smithi083 ceph-mon[36260]: 5.a scrub starts 2024-09-20T18:52:06.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:06 smithi083 ceph-mon[36260]: 5.a scrub ok 2024-09-20T18:52:06.532 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 69 16 102841 102825 93.1136 68.25 0.010543 0.0107343 2024-09-20T18:52:06.580 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-20T18:52:06.581 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-20T18:52:06.581 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph tell osd.1 'trim stale osdmaps' 2024-09-20T18:52:06.604 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:06.634 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:06.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:06.636 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:06.736 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:06.749 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:52:06 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:52:06.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:06.787 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:06.787 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:06.888 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:06.933 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:06.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:06.934 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:07.020 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-20T18:52:07.035 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:07.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:07.121 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:07.122 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:07.222 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:07.255 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:07.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:07.256 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:07.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:07 smithi083 ceph-mon[29421]: 5.1f scrub starts 2024-09-20T18:52:07.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:07 smithi083 ceph-mon[29421]: 5.1f scrub ok 2024-09-20T18:52:07.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:07 smithi083 ceph-mon[29421]: 4.b scrub starts 2024-09-20T18:52:07.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:07 smithi083 ceph-mon[29421]: 4.b scrub ok 2024-09-20T18:52:07.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:07 smithi083 ceph-mon[29421]: 5.9 scrub starts 2024-09-20T18:52:07.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:07 smithi083 ceph-mon[29421]: 5.9 scrub ok 2024-09-20T18:52:07.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:07 smithi083 ceph-mon[29421]: 3.14 scrub starts 2024-09-20T18:52:07.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:07 smithi083 ceph-mon[29421]: 3.14 scrub ok 2024-09-20T18:52:07.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:07 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:52:07.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:07 smithi083 ceph-mon[36260]: 5.1f scrub starts 2024-09-20T18:52:07.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:07 smithi083 ceph-mon[36260]: 5.1f scrub ok 2024-09-20T18:52:07.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:07 smithi083 ceph-mon[36260]: 4.b scrub starts 2024-09-20T18:52:07.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:07 smithi083 ceph-mon[36260]: 4.b scrub ok 2024-09-20T18:52:07.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:07 smithi083 ceph-mon[36260]: 5.9 scrub starts 2024-09-20T18:52:07.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:07 smithi083 ceph-mon[36260]: 5.9 scrub ok 2024-09-20T18:52:07.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:07 smithi083 ceph-mon[36260]: 3.14 scrub starts 2024-09-20T18:52:07.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:07 smithi083 ceph-mon[36260]: 3.14 scrub ok 2024-09-20T18:52:07.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:07 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:52:07.357 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:07.396 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:07.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:07.397 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:07.430 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:07 smithi142 ceph-mon[32508]: 5.1f scrub starts 2024-09-20T18:52:07.431 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:07 smithi142 ceph-mon[32508]: 5.1f scrub ok 2024-09-20T18:52:07.431 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:07 smithi142 ceph-mon[32508]: 4.b scrub starts 2024-09-20T18:52:07.431 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:07 smithi142 ceph-mon[32508]: 4.b scrub ok 2024-09-20T18:52:07.431 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:07 smithi142 ceph-mon[32508]: 5.9 scrub starts 2024-09-20T18:52:07.431 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:07 smithi142 ceph-mon[32508]: 5.9 scrub ok 2024-09-20T18:52:07.431 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:07 smithi142 ceph-mon[32508]: 3.14 scrub starts 2024-09-20T18:52:07.431 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:07 smithi142 ceph-mon[32508]: 3.14 scrub ok 2024-09-20T18:52:07.431 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:07 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:52:07.498 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:07.531 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 70 16 103845 103829 92.6798 62.75 0.019987 0.0107849 2024-09-20T18:52:07.539 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:07.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:07.540 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:07.641 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:52:07.670 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:07.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:07.671 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:52:07.772 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:07.807 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:07.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:07.808 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:07.909 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:07.955 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:07.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:07.956 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:08.045 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:52:08.056 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:08.098 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:08.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:08.099 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:08.200 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:08.233 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:08.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:08.234 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:08.334 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:08.339 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:08 smithi142 ceph-mon[32508]: pgmap v99: 148 pgs: 148 active+clean; 6.2 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.6 KiB/s rd, 103 MiB/s wr, 1.65k op/s 2024-09-20T18:52:08.339 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:08 smithi142 ceph-mon[32508]: 2.0 scrub starts 2024-09-20T18:52:08.339 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:08 smithi142 ceph-mon[32508]: 2.0 scrub ok 2024-09-20T18:52:08.339 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:08 smithi142 ceph-mon[32508]: 3.5 scrub starts 2024-09-20T18:52:08.339 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:08 smithi142 ceph-mon[32508]: 3.5 scrub ok 2024-09-20T18:52:08.339 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:08 smithi142 ceph-mon[32508]: 2.1 scrub starts 2024-09-20T18:52:08.339 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:08 smithi142 ceph-mon[32508]: 2.1 scrub ok 2024-09-20T18:52:08.339 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:08 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:52:08.339 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:08 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:52:08.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:08.397 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:08.397 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:52:08.498 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:08.532 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 71 16 104953 104937 92.3497 69.25 0.0104723 0.0108225 2024-09-20T18:52:08.549 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:08.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:08.550 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:08.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:08 smithi083 ceph-mon[29421]: pgmap v99: 148 pgs: 148 active+clean; 6.2 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.6 KiB/s rd, 103 MiB/s wr, 1.65k op/s 2024-09-20T18:52:08.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:08 smithi083 ceph-mon[29421]: 2.0 scrub starts 2024-09-20T18:52:08.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:08 smithi083 ceph-mon[29421]: 2.0 scrub ok 2024-09-20T18:52:08.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:08 smithi083 ceph-mon[29421]: 3.5 scrub starts 2024-09-20T18:52:08.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:08 smithi083 ceph-mon[29421]: 3.5 scrub ok 2024-09-20T18:52:08.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:08 smithi083 ceph-mon[29421]: 2.1 scrub starts 2024-09-20T18:52:08.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:08 smithi083 ceph-mon[29421]: 2.1 scrub ok 2024-09-20T18:52:08.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:08 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:52:08.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:08 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:52:08.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:08 smithi083 ceph-mon[36260]: pgmap v99: 148 pgs: 148 active+clean; 6.2 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.6 KiB/s rd, 103 MiB/s wr, 1.65k op/s 2024-09-20T18:52:08.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:08 smithi083 ceph-mon[36260]: 2.0 scrub starts 2024-09-20T18:52:08.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:08 smithi083 ceph-mon[36260]: 2.0 scrub ok 2024-09-20T18:52:08.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:08 smithi083 ceph-mon[36260]: 3.5 scrub starts 2024-09-20T18:52:08.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:08 smithi083 ceph-mon[36260]: 3.5 scrub ok 2024-09-20T18:52:08.556 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:08 smithi083 ceph-mon[36260]: 2.1 scrub starts 2024-09-20T18:52:08.556 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:08 smithi083 ceph-mon[36260]: 2.1 scrub ok 2024-09-20T18:52:08.556 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:08 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:52:08.556 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:08 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:52:08.650 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:08.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:08.682 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:08.682 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:08.783 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:08.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:08.814 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:08.815 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:08.918 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:08.953 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:08.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:08.954 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:08.955 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T18:52:08.955 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-09-20T18:52:08.955 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:08.955 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:105379 7.1 7:83184746:::benchmark_data_smithi083_70643_object105378:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-20T18:52:08.955 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:08.934259+0000", 2024-09-20T18:52:08.955 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.003471404, 2024-09-20T18:52:08.955 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.003502649, 2024-09-20T18:52:08.956 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:52:08.956 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:52:08.956 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "waiting for sub ops", 2024-09-20T18:52:08.956 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:52:08.956 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:52:08.956 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:52:08.956 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 105379 2024-09-20T18:52:08.956 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:08.956 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:52:08.956 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:08.956 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:52:08.956 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:08.934259+0000", 2024-09-20T18:52:08.956 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:08.956 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:08.957 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:08.957 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:52:08.957 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:08.934259+0000", 2024-09-20T18:52:08.957 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:08.957 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:08.957 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:08.957 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:52:08.957 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:08.934271+0000", 2024-09-20T18:52:08.957 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.133e-05 2024-09-20T18:52:08.957 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:08.957 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:08.957 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:52:08.957 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:08.934315+0000", 2024-09-20T18:52:08.957 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3841000000000003e-05 2024-09-20T18:52:08.957 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:08.958 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:08.958 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:52:08.958 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:08.934326+0000", 2024-09-20T18:52:08.958 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1168e-05 2024-09-20T18:52:08.958 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:08.958 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:08.958 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:52:08.958 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:08.934333+0000", 2024-09-20T18:52:08.958 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.9650000000000002e-06 2024-09-20T18:52:08.958 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:08.958 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:08.959 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:52:08.959 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:08.934383+0000", 2024-09-20T18:52:08.959 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0089000000000001e-05 2024-09-20T18:52:08.959 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:08.959 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:08.959 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:52:08.959 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:08.934460+0000", 2024-09-20T18:52:08.959 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.7588000000000003e-05 2024-09-20T18:52:08.959 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:08.959 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:08.959 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,6", 2024-09-20T18:52:08.959 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:08.934548+0000", 2024-09-20T18:52:08.959 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.7686e-05 2024-09-20T18:52:08.959 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:08.960 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:08.960 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:52:08.960 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:08.937319+0000", 2024-09-20T18:52:08.960 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0027710510000000001 2024-09-20T18:52:08.960 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:08.960 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:52:08.960 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:08.960 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:08.960 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:08.960 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.14808.0:105380 7.8 e114/83 7:1e46927b:::benchmark_data_smithi083_70643_object105379:head v 114'6625, mlcod=114'6623)", 2024-09-20T18:52:08.960 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:08.936598+0000", 2024-09-20T18:52:08.960 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.001132854, 2024-09-20T18:52:08.960 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0012340000000000001, 2024-09-20T18:52:08.960 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:52:08.960 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:52:08.961 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-09-20T18:52:08.961 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:52:08.961 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:08.961 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:52:08.961 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:08.936598+0000", 2024-09-20T18:52:08.961 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:08.961 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:08.961 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:08.961 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:52:08.961 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:08.936598+0000", 2024-09-20T18:52:08.961 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:08.961 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:08.961 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:08.961 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:52:08.962 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:08.936599+0000", 2024-09-20T18:52:08.962 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5799999999999998e-07 2024-09-20T18:52:08.962 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:08.962 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:08.962 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:52:08.962 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:08.936702+0000", 2024-09-20T18:52:08.962 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000103822 2024-09-20T18:52:08.962 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:08.962 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:08.962 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:52:08.962 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:08.936719+0000", 2024-09-20T18:52:08.962 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6487e-05 2024-09-20T18:52:08.962 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:08.962 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:08.962 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:52:08.963 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:08.936726+0000", 2024-09-20T18:52:08.963 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.9430000000000003e-06 2024-09-20T18:52:08.963 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:08.963 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:08.963 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:52:08.963 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:08.936777+0000", 2024-09-20T18:52:08.963 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.1226e-05 2024-09-20T18:52:08.963 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:08.963 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:08.963 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:52:08.963 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:08.936795+0000", 2024-09-20T18:52:08.963 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7541e-05 2024-09-20T18:52:08.963 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:08.963 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:52:08.964 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:08.964 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:08.964 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-09-20T18:52:08.964 INFO:teuthology.orchestra.run.smithi142.stdout: "num_ops": 2 2024-09-20T18:52:08.964 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T18:52:09.055 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:09.083 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:09.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:09.084 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:09.185 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:09.227 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:09.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:09.228 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:09.328 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:09.354 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:09 smithi142 ceph-mon[32508]: 3.17 scrub starts 2024-09-20T18:52:09.354 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:09 smithi142 ceph-mon[32508]: 3.17 scrub ok 2024-09-20T18:52:09.354 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:09 smithi142 ceph-mon[32508]: 3.10 scrub starts 2024-09-20T18:52:09.354 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:09 smithi142 ceph-mon[32508]: 3.10 scrub ok 2024-09-20T18:52:09.354 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:09 smithi142 ceph-mon[32508]: 5.17 scrub starts 2024-09-20T18:52:09.354 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:09 smithi142 ceph-mon[32508]: 5.17 scrub ok 2024-09-20T18:52:09.354 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:09 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:52:09.354 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:09 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:52:09.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:09.360 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:09.360 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:52:09.399 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:09 smithi083 ceph-mon[29421]: 3.17 scrub starts 2024-09-20T18:52:09.399 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:09 smithi083 ceph-mon[29421]: 3.17 scrub ok 2024-09-20T18:52:09.399 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:09 smithi083 ceph-mon[29421]: 3.10 scrub starts 2024-09-20T18:52:09.399 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:09 smithi083 ceph-mon[29421]: 3.10 scrub ok 2024-09-20T18:52:09.399 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:09 smithi083 ceph-mon[29421]: 5.17 scrub starts 2024-09-20T18:52:09.399 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:09 smithi083 ceph-mon[29421]: 5.17 scrub ok 2024-09-20T18:52:09.399 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:09 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:52:09.399 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:09 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:52:09.400 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:09 smithi083 ceph-mon[36260]: 3.17 scrub starts 2024-09-20T18:52:09.400 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:09 smithi083 ceph-mon[36260]: 3.17 scrub ok 2024-09-20T18:52:09.400 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:09 smithi083 ceph-mon[36260]: 3.10 scrub starts 2024-09-20T18:52:09.400 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:09 smithi083 ceph-mon[36260]: 3.10 scrub ok 2024-09-20T18:52:09.400 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:09 smithi083 ceph-mon[36260]: 5.17 scrub starts 2024-09-20T18:52:09.400 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:09 smithi083 ceph-mon[36260]: 5.17 scrub ok 2024-09-20T18:52:09.400 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:09 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:52:09.400 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:09 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:52:09.462 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:09.486 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:52:09.487 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:52:09.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:09.488 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:09.488 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:09.532 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 72 16 105962 105946 91.9428 63.0625 0.0199832 0.01087 2024-09-20T18:52:09.588 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:09.616 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:09.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:09.617 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:09.696 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_blocked_ops 2024-09-20T18:52:09.718 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:09.772 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:09.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:09.774 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:09.874 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:09.905 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:09.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:09.907 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:10.007 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:10.026 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:52:10.042 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:10.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:10.043 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:10.143 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:10.187 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:10.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:10.188 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:10.289 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:10.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:10.319 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:10.319 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:10.344 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:10 smithi142 ceph-mon[32508]: 6.16 scrub starts 2024-09-20T18:52:10.344 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:10 smithi142 ceph-mon[32508]: 6.16 scrub ok 2024-09-20T18:52:10.344 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:10 smithi142 ceph-mon[32508]: pgmap v100: 148 pgs: 148 active+clean; 6.2 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 639 B/s rd, 64 MiB/s wr, 1.03k op/s 2024-09-20T18:52:10.344 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:10 smithi142 ceph-mon[32508]: 7.5 scrub starts 2024-09-20T18:52:10.344 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:10 smithi142 ceph-mon[32508]: 6.b scrub starts 2024-09-20T18:52:10.344 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:10 smithi142 ceph-mon[32508]: 6.b scrub ok 2024-09-20T18:52:10.344 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:10 smithi142 ceph-mon[32508]: osdmap e115: 8 total, 8 up, 8 in 2024-09-20T18:52:10.345 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:10 smithi142 ceph-mon[32508]: 7.9 scrub starts 2024-09-20T18:52:10.345 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:10 smithi142 ceph-mon[32508]: 6.14 scrub starts 2024-09-20T18:52:10.345 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:10 smithi142 ceph-mon[32508]: 6.14 scrub ok 2024-09-20T18:52:10.420 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:10.456 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:10.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:10.457 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:10.534 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 73 16 106820 106804 91.4179 53.625 0.0175541 0.0109336 2024-09-20T18:52:10.538 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:10 smithi083 ceph-mon[29421]: 6.16 scrub starts 2024-09-20T18:52:10.538 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:10 smithi083 ceph-mon[29421]: 6.16 scrub ok 2024-09-20T18:52:10.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:10 smithi083 ceph-mon[29421]: pgmap v100: 148 pgs: 148 active+clean; 6.2 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 639 B/s rd, 64 MiB/s wr, 1.03k op/s 2024-09-20T18:52:10.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:10 smithi083 ceph-mon[29421]: 7.5 scrub starts 2024-09-20T18:52:10.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:10 smithi083 ceph-mon[29421]: 6.b scrub starts 2024-09-20T18:52:10.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:10 smithi083 ceph-mon[29421]: 6.b scrub ok 2024-09-20T18:52:10.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:10 smithi083 ceph-mon[29421]: osdmap e115: 8 total, 8 up, 8 in 2024-09-20T18:52:10.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:10 smithi083 ceph-mon[29421]: 7.9 scrub starts 2024-09-20T18:52:10.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:10 smithi083 ceph-mon[29421]: 6.14 scrub starts 2024-09-20T18:52:10.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:10 smithi083 ceph-mon[29421]: 6.14 scrub ok 2024-09-20T18:52:10.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:10 smithi083 ceph-mon[36260]: 6.16 scrub starts 2024-09-20T18:52:10.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:10 smithi083 ceph-mon[36260]: 6.16 scrub ok 2024-09-20T18:52:10.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:10 smithi083 ceph-mon[36260]: pgmap v100: 148 pgs: 148 active+clean; 6.2 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 639 B/s rd, 64 MiB/s wr, 1.03k op/s 2024-09-20T18:52:10.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:10 smithi083 ceph-mon[36260]: 7.5 scrub starts 2024-09-20T18:52:10.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:10 smithi083 ceph-mon[36260]: 6.b scrub starts 2024-09-20T18:52:10.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:10 smithi083 ceph-mon[36260]: 6.b scrub ok 2024-09-20T18:52:10.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:10 smithi083 ceph-mon[36260]: osdmap e115: 8 total, 8 up, 8 in 2024-09-20T18:52:10.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:10 smithi083 ceph-mon[36260]: 7.9 scrub starts 2024-09-20T18:52:10.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:10 smithi083 ceph-mon[36260]: 6.14 scrub starts 2024-09-20T18:52:10.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:10 smithi083 ceph-mon[36260]: 6.14 scrub ok 2024-09-20T18:52:10.558 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:52:10.583 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:10.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:10.584 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:52:10.685 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:10.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:10.830 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:10.830 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:10.831 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:52:10 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:52:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:52:10.831 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T18:52:10.831 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-09-20T18:52:10.831 INFO:teuthology.orchestra.run.smithi142.stdout: "complaint_time": 30, 2024-09-20T18:52:10.831 INFO:teuthology.orchestra.run.smithi142.stdout: "num_blocked_ops": 0 2024-09-20T18:52:10.831 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T18:52:10.931 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:10.956 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:10.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:10.957 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:52:11.058 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:11.114 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:11.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:11.115 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:11.216 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:11.241 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:11.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:11.242 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:11.342 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:11.371 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:11.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:11.372 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:11.388 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:11 smithi083 ceph-mon[29421]: 6.e scrub starts 2024-09-20T18:52:11.388 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:11 smithi083 ceph-mon[29421]: 6.e scrub ok 2024-09-20T18:52:11.388 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:11 smithi083 ceph-mon[29421]: 3.16 scrub starts 2024-09-20T18:52:11.388 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:11 smithi083 ceph-mon[29421]: 3.16 scrub ok 2024-09-20T18:52:11.388 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:11 smithi083 ceph-mon[29421]: osdmap e116: 8 total, 8 up, 8 in 2024-09-20T18:52:11.388 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:11 smithi083 ceph-mon[36260]: 6.e scrub starts 2024-09-20T18:52:11.388 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:11 smithi083 ceph-mon[36260]: 6.e scrub ok 2024-09-20T18:52:11.388 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:11 smithi083 ceph-mon[36260]: 3.16 scrub starts 2024-09-20T18:52:11.388 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:11 smithi083 ceph-mon[36260]: 3.16 scrub ok 2024-09-20T18:52:11.388 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:11 smithi083 ceph-mon[36260]: osdmap e116: 8 total, 8 up, 8 in 2024-09-20T18:52:11.469 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T18:52:11.472 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:11 smithi142 ceph-mon[32508]: 6.e scrub starts 2024-09-20T18:52:11.472 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:11 smithi142 ceph-mon[32508]: 6.e scrub ok 2024-09-20T18:52:11.472 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:11 smithi142 ceph-mon[32508]: 3.16 scrub starts 2024-09-20T18:52:11.472 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:11 smithi142 ceph-mon[32508]: 3.16 scrub ok 2024-09-20T18:52:11.472 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:11 smithi142 ceph-mon[32508]: osdmap e116: 8 total, 8 up, 8 in 2024-09-20T18:52:11.472 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:11.502 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:11.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:11.503 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:11.516 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_historic_ops 2024-09-20T18:52:11.532 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 74 16 107688 107672 90.9156 54.25 0.0258645 0.0109926 2024-09-20T18:52:11.604 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:11.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:11.658 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:11.658 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:11.723 INFO:teuthology.orchestra.run.smithi083.stderr:no valid command found; 10 closest matches: 2024-09-20T18:52:11.723 INFO:teuthology.orchestra.run.smithi083.stderr:0 2024-09-20T18:52:11.723 INFO:teuthology.orchestra.run.smithi083.stderr:1 2024-09-20T18:52:11.723 INFO:teuthology.orchestra.run.smithi083.stderr:2 2024-09-20T18:52:11.724 INFO:teuthology.orchestra.run.smithi083.stderr:abort 2024-09-20T18:52:11.724 INFO:teuthology.orchestra.run.smithi083.stderr:assert 2024-09-20T18:52:11.724 INFO:teuthology.orchestra.run.smithi083.stderr:bench [] [] [] [] 2024-09-20T18:52:11.724 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs debug_inject_read_zeros 2024-09-20T18:52:11.724 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs files list 2024-09-20T18:52:11.724 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs stats 2024-09-20T18:52:11.724 INFO:teuthology.orchestra.run.smithi083.stderr:bluestore allocator dump block 2024-09-20T18:52:11.724 INFO:teuthology.orchestra.run.smithi083.stderr:Error EINVAL: invalid command 2024-09-20T18:52:11.759 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:11.766 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:52:11.766 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:52:11.766 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:52:11.766 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:52:11.766 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:52:11.766 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:52:11.766 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:52:11.766 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:52:11.799 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:11.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:11.801 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:11.806 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:52:11.806 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:52:11.858 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:52:11.858 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:52:11.882 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:52:11.892 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:52:11.892 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:52:11.902 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:11.928 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:52:11.929 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:52:11.950 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:11.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:11.952 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:12.052 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:12.090 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:12.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:12.092 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:12.193 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:12.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:12.229 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:12.229 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:12.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:12 smithi142 ceph-mon[32508]: 7.6 scrub starts 2024-09-20T18:52:12.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:12 smithi142 ceph-mon[32508]: 7.5 scrub ok 2024-09-20T18:52:12.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:12 smithi142 ceph-mon[32508]: pgmap v103: 148 pgs: 148 active+clean; 6.3 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1.7 KiB/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-20T18:52:12.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:12 smithi142 ceph-mon[32508]: 7.0 scrub starts 2024-09-20T18:52:12.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:12 smithi142 ceph-mon[32508]: 3.0 scrub starts 2024-09-20T18:52:12.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:12 smithi142 ceph-mon[32508]: 3.0 scrub ok 2024-09-20T18:52:12.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:12 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1483841108' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:52:12.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:12 smithi083 ceph-mon[29421]: 7.6 scrub starts 2024-09-20T18:52:12.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:12 smithi083 ceph-mon[29421]: 7.5 scrub ok 2024-09-20T18:52:12.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:12 smithi083 ceph-mon[29421]: pgmap v103: 148 pgs: 148 active+clean; 6.3 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1.7 KiB/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-20T18:52:12.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:12 smithi083 ceph-mon[29421]: 7.0 scrub starts 2024-09-20T18:52:12.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:12 smithi083 ceph-mon[29421]: 3.0 scrub starts 2024-09-20T18:52:12.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:12 smithi083 ceph-mon[29421]: 3.0 scrub ok 2024-09-20T18:52:12.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:12 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1483841108' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:52:12.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:12 smithi083 ceph-mon[36260]: 7.6 scrub starts 2024-09-20T18:52:12.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:12 smithi083 ceph-mon[36260]: 7.5 scrub ok 2024-09-20T18:52:12.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:12 smithi083 ceph-mon[36260]: pgmap v103: 148 pgs: 148 active+clean; 6.3 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1.7 KiB/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-20T18:52:12.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:12 smithi083 ceph-mon[36260]: 7.0 scrub starts 2024-09-20T18:52:12.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:12 smithi083 ceph-mon[36260]: 3.0 scrub starts 2024-09-20T18:52:12.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:12 smithi083 ceph-mon[36260]: 3.0 scrub ok 2024-09-20T18:52:12.306 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:12 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1483841108' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:52:12.330 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:12.371 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:12.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:12.372 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:12.465 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T18:52:12.465 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:52:12.474 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:12.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:12.514 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:12.514 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:12.532 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 75 16 108559 108543 90.4293 54.4375 0.0374636 0.0110533 2024-09-20T18:52:12.543 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-20T18:52:12.544 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-20T18:52:12.544 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph tell osd.2 'trim stale osdmaps' 2024-09-20T18:52:12.615 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:12.643 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:12.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:12.644 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:12.744 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:12.770 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:12.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:12.771 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:12.872 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:12.917 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:12.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:12.918 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:13.019 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:13.046 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:13.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:13.047 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:13.131 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-20T18:52:13.149 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:13.199 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:13.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:13.200 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:13.301 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:13.334 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:13.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:13.335 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:13.435 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:13.469 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:52:13.477 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:13.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:13.478 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:13.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:13 smithi142 ceph-mon[32508]: 7.9 scrub ok 2024-09-20T18:52:13.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:13 smithi142 ceph-mon[32508]: 1.0 scrub starts 2024-09-20T18:52:13.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:13 smithi142 ceph-mon[32508]: 1.0 scrub ok 2024-09-20T18:52:13.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:13 smithi142 ceph-mon[32508]: 5.19 scrub starts 2024-09-20T18:52:13.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:13 smithi142 ceph-mon[32508]: 5.19 scrub ok 2024-09-20T18:52:13.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:13 smithi142 ceph-mon[32508]: 5.8 scrub starts 2024-09-20T18:52:13.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:13 smithi142 ceph-mon[32508]: 5.8 scrub ok 2024-09-20T18:52:13.532 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 76 16 109477 109461 89.9943 57.375 0.0138272 0.0111067 2024-09-20T18:52:13.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:13 smithi083 ceph-mon[29421]: 7.9 scrub ok 2024-09-20T18:52:13.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:13 smithi083 ceph-mon[29421]: 1.0 scrub starts 2024-09-20T18:52:13.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:13 smithi083 ceph-mon[29421]: 1.0 scrub ok 2024-09-20T18:52:13.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:13 smithi083 ceph-mon[29421]: 5.19 scrub starts 2024-09-20T18:52:13.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:13 smithi083 ceph-mon[29421]: 5.19 scrub ok 2024-09-20T18:52:13.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:13 smithi083 ceph-mon[29421]: 5.8 scrub starts 2024-09-20T18:52:13.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:13 smithi083 ceph-mon[29421]: 5.8 scrub ok 2024-09-20T18:52:13.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:13 smithi083 ceph-mon[36260]: 7.9 scrub ok 2024-09-20T18:52:13.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:13 smithi083 ceph-mon[36260]: 1.0 scrub starts 2024-09-20T18:52:13.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:13 smithi083 ceph-mon[36260]: 1.0 scrub ok 2024-09-20T18:52:13.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:13 smithi083 ceph-mon[36260]: 5.19 scrub starts 2024-09-20T18:52:13.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:13 smithi083 ceph-mon[36260]: 5.19 scrub ok 2024-09-20T18:52:13.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:13 smithi083 ceph-mon[36260]: 5.8 scrub starts 2024-09-20T18:52:13.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:13 smithi083 ceph-mon[36260]: 5.8 scrub ok 2024-09-20T18:52:13.579 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:13.608 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:13.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:13.609 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:13.710 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:13.737 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:13.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:13.738 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:52:13.839 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:13.883 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:13.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:13.884 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:13.985 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:14.021 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:14.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:14.022 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:14.103 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T18:52:14.103 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:52:14.122 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:14.158 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:14.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:14.159 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:14.260 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:14.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:14 smithi142 ceph-mon[32508]: pgmap v104: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.5 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.4 KiB/s rd, 86 MiB/s wr, 1.38k op/s 2024-09-20T18:52:14.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:14 smithi142 ceph-mon[32508]: 7.0 scrub ok 2024-09-20T18:52:14.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:14 smithi142 ceph-mon[32508]: 4.11 scrub starts 2024-09-20T18:52:14.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:14 smithi142 ceph-mon[32508]: 4.11 scrub ok 2024-09-20T18:52:14.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:14 smithi142 ceph-mon[32508]: 7.6 scrub ok 2024-09-20T18:52:14.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:14 smithi142 ceph-mon[32508]: 7.b scrub starts 2024-09-20T18:52:14.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:14.303 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:14.303 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:14.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:14 smithi083 ceph-mon[29421]: pgmap v104: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.5 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.4 KiB/s rd, 86 MiB/s wr, 1.38k op/s 2024-09-20T18:52:14.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:14 smithi083 ceph-mon[29421]: 7.0 scrub ok 2024-09-20T18:52:14.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:14 smithi083 ceph-mon[29421]: 4.11 scrub starts 2024-09-20T18:52:14.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:14 smithi083 ceph-mon[29421]: 4.11 scrub ok 2024-09-20T18:52:14.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:14 smithi083 ceph-mon[29421]: 7.6 scrub ok 2024-09-20T18:52:14.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:14 smithi083 ceph-mon[29421]: 7.b scrub starts 2024-09-20T18:52:14.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:14 smithi083 ceph-mon[36260]: pgmap v104: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.5 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.4 KiB/s rd, 86 MiB/s wr, 1.38k op/s 2024-09-20T18:52:14.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:14 smithi083 ceph-mon[36260]: 7.0 scrub ok 2024-09-20T18:52:14.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:14 smithi083 ceph-mon[36260]: 4.11 scrub starts 2024-09-20T18:52:14.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:14 smithi083 ceph-mon[36260]: 4.11 scrub ok 2024-09-20T18:52:14.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:14 smithi083 ceph-mon[36260]: 7.6 scrub ok 2024-09-20T18:52:14.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:14 smithi083 ceph-mon[36260]: 7.b scrub starts 2024-09-20T18:52:14.404 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:14.430 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:14.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:14.431 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:14.531 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:14.532 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 77 15 110508 110493 89.6632 64.5 0.00701005 0.0111479 2024-09-20T18:52:14.570 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:14.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:14.571 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:14.672 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:14.692 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:14.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:14.694 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:52:14.794 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:14.821 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:14.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:14.822 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:52:14.923 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:14.949 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:14.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:14.951 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:15.052 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:15.084 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_blocked_ops 2024-09-20T18:52:15.089 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:15.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:15.090 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:15.190 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:52:15.219 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:15.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:15.220 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:52:15.321 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:15.360 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:15.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:15.361 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:15.398 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:15 smithi083 ceph-mon[29421]: 5.14 scrub starts 2024-09-20T18:52:15.398 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:15 smithi083 ceph-mon[29421]: 5.14 scrub ok 2024-09-20T18:52:15.398 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:15 smithi083 ceph-mon[29421]: 7.8 scrub starts 2024-09-20T18:52:15.398 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:15 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]} : dispatch 2024-09-20T18:52:15.398 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:15 smithi083 ceph-mon[36260]: 5.14 scrub starts 2024-09-20T18:52:15.398 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:15 smithi083 ceph-mon[36260]: 5.14 scrub ok 2024-09-20T18:52:15.399 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:15 smithi083 ceph-mon[36260]: 7.8 scrub starts 2024-09-20T18:52:15.399 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:15 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]} : dispatch 2024-09-20T18:52:15.433 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:52:15.462 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:15.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:15 smithi142 ceph-mon[32508]: 5.14 scrub starts 2024-09-20T18:52:15.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:15 smithi142 ceph-mon[32508]: 5.14 scrub ok 2024-09-20T18:52:15.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:15 smithi142 ceph-mon[32508]: 7.8 scrub starts 2024-09-20T18:52:15.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:15 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]} : dispatch 2024-09-20T18:52:15.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:15.534 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:15.534 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:15.534 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 78 16 111308 111292 89.1533 49.9375 0.0319386 0.0112106 2024-09-20T18:52:15.536 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:52:15.539 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:52:15.635 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:15.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:15.686 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:15.686 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:15.787 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:15.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:15.820 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:15.820 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:15.921 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:15.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:15.986 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:15.987 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:16.088 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:16.121 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T18:52:16.121 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:52:16.134 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:16.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:16.135 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:16.235 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:52:16.262 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:16.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:16.263 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:52:16.364 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:16.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:16.405 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:16.405 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:16.464 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:16 smithi142 ceph-mon[32508]: pgmap v105: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.5 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 39 MiB/s wr, 632 op/s 2024-09-20T18:52:16.465 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:16 smithi142 ceph-mon[32508]: 2.2 scrub starts 2024-09-20T18:52:16.465 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:16 smithi142 ceph-mon[32508]: 2.2 scrub ok 2024-09-20T18:52:16.465 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:16 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]': finished 2024-09-20T18:52:16.465 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:16 smithi142 ceph-mon[32508]: osdmap e117: 8 total, 8 up, 8 in 2024-09-20T18:52:16.465 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:16 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:52:16.465 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:16 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:52:16.465 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:16 smithi142 ceph-mon[32508]: 7.b scrub ok 2024-09-20T18:52:16.465 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:16 smithi142 ceph-mon[32508]: osdmap e118: 8 total, 8 up, 8 in 2024-09-20T18:52:16.490 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:16 smithi083 ceph-mon[29421]: pgmap v105: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.5 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 39 MiB/s wr, 632 op/s 2024-09-20T18:52:16.490 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:16 smithi083 ceph-mon[29421]: 2.2 scrub starts 2024-09-20T18:52:16.490 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:16 smithi083 ceph-mon[29421]: 2.2 scrub ok 2024-09-20T18:52:16.490 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:16 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]': finished 2024-09-20T18:52:16.490 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:16 smithi083 ceph-mon[29421]: osdmap e117: 8 total, 8 up, 8 in 2024-09-20T18:52:16.490 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:16 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:52:16.490 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:16 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:52:16.490 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:16 smithi083 ceph-mon[29421]: 7.b scrub ok 2024-09-20T18:52:16.490 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:16 smithi083 ceph-mon[29421]: osdmap e118: 8 total, 8 up, 8 in 2024-09-20T18:52:16.491 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:16 smithi083 ceph-mon[36260]: pgmap v105: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.5 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 39 MiB/s wr, 632 op/s 2024-09-20T18:52:16.491 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:16 smithi083 ceph-mon[36260]: 2.2 scrub starts 2024-09-20T18:52:16.491 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:16 smithi083 ceph-mon[36260]: 2.2 scrub ok 2024-09-20T18:52:16.491 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:16 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]': finished 2024-09-20T18:52:16.491 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:16 smithi083 ceph-mon[36260]: osdmap e117: 8 total, 8 up, 8 in 2024-09-20T18:52:16.491 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:16 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:52:16.491 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:16 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:52:16.491 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:16 smithi083 ceph-mon[36260]: 7.b scrub ok 2024-09-20T18:52:16.491 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:16 smithi083 ceph-mon[36260]: osdmap e118: 8 total, 8 up, 8 in 2024-09-20T18:52:16.507 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:16.533 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 79 16 112155 112139 88.6948 52.9375 0.0267103 0.0112698 2024-09-20T18:52:16.546 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:16.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:16.547 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:16.648 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:52:16.675 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:16.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:16.676 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:52:16.769 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:52:16 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:52:16.776 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:16.810 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_historic_ops 2024-09-20T18:52:16.819 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:16.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:16.820 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:16.921 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:52:16.948 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:16.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:16.949 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:52:17.050 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:17.068 INFO:teuthology.orchestra.run.smithi083.stderr:no valid command found; 10 closest matches: 2024-09-20T18:52:17.068 INFO:teuthology.orchestra.run.smithi083.stderr:0 2024-09-20T18:52:17.068 INFO:teuthology.orchestra.run.smithi083.stderr:1 2024-09-20T18:52:17.068 INFO:teuthology.orchestra.run.smithi083.stderr:2 2024-09-20T18:52:17.069 INFO:teuthology.orchestra.run.smithi083.stderr:abort 2024-09-20T18:52:17.069 INFO:teuthology.orchestra.run.smithi083.stderr:assert 2024-09-20T18:52:17.069 INFO:teuthology.orchestra.run.smithi083.stderr:bench [] [] [] [] 2024-09-20T18:52:17.069 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs debug_inject_read_zeros 2024-09-20T18:52:17.069 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs files list 2024-09-20T18:52:17.069 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs stats 2024-09-20T18:52:17.069 INFO:teuthology.orchestra.run.smithi083.stderr:bluestore allocator dump block 2024-09-20T18:52:17.069 INFO:teuthology.orchestra.run.smithi083.stderr:Error EINVAL: invalid command 2024-09-20T18:52:17.116 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:17.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:17.118 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:17.137 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:52:17.218 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:17.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:17.255 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:17.255 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:17.356 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:17.400 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:17.400 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T18:52:17.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:17.402 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:17.424 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:17 smithi142 ceph-mon[32508]: 7.c scrub starts 2024-09-20T18:52:17.424 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:17 smithi142 ceph-mon[32508]: 6.1c scrub starts 2024-09-20T18:52:17.424 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:17 smithi142 ceph-mon[32508]: 6.1c scrub ok 2024-09-20T18:52:17.424 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:17 smithi142 ceph-mon[32508]: 7.f scrub starts 2024-09-20T18:52:17.424 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:17 smithi142 ceph-mon[32508]: 7.d scrub starts 2024-09-20T18:52:17.424 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:17 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:52:17.424 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:17 smithi142 ceph-mon[32508]: 7.8 scrub ok 2024-09-20T18:52:17.448 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:17 smithi083 ceph-mon[29421]: 7.c scrub starts 2024-09-20T18:52:17.448 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:17 smithi083 ceph-mon[29421]: 6.1c scrub starts 2024-09-20T18:52:17.448 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:17 smithi083 ceph-mon[29421]: 6.1c scrub ok 2024-09-20T18:52:17.448 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:17 smithi083 ceph-mon[29421]: 7.f scrub starts 2024-09-20T18:52:17.448 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:17 smithi083 ceph-mon[29421]: 7.d scrub starts 2024-09-20T18:52:17.448 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:17 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:52:17.448 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:17 smithi083 ceph-mon[29421]: 7.8 scrub ok 2024-09-20T18:52:17.449 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:17 smithi083 ceph-mon[36260]: 7.c scrub starts 2024-09-20T18:52:17.449 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:17 smithi083 ceph-mon[36260]: 6.1c scrub starts 2024-09-20T18:52:17.449 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:17 smithi083 ceph-mon[36260]: 6.1c scrub ok 2024-09-20T18:52:17.449 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:17 smithi083 ceph-mon[36260]: 7.f scrub starts 2024-09-20T18:52:17.449 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:17 smithi083 ceph-mon[36260]: 7.d scrub starts 2024-09-20T18:52:17.449 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:17 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:52:17.449 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:17 smithi083 ceph-mon[36260]: 7.8 scrub ok 2024-09-20T18:52:17.502 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:17.536 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:2024-09-20T18:52:17.519860+0000 min lat: 0.00137732 max lat: 0.186958 avg lat: 0.0113404 2024-09-20T18:52:17.536 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-20T18:52:17.536 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 80 16 112851 112835 88.127 43.5 0.0409169 0.0113404 2024-09-20T18:52:17.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:17.554 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:17.554 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:17.655 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:17.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:17.686 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:17.686 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:17.696 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T18:52:17.696 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:52:17.788 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:52:17.815 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:17.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:17.816 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:52:17.918 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:17.922 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-20T18:52:17.923 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-20T18:52:17.923 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph tell osd.3 'trim stale osdmaps' 2024-09-20T18:52:17.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:17.998 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:17.999 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:18.025 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:52:18.099 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:18.157 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-20T18:52:18.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:18.182 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:18.182 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:18.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:18 smithi142 ceph-mon[32508]: pgmap v108: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.8 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.6 KiB/s rd, 97 MiB/s wr, 1.55k op/s; 17 B/s, 0 objects/s recovering 2024-09-20T18:52:18.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:18 smithi142 ceph-mon[32508]: 7.c scrub ok 2024-09-20T18:52:18.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:18 smithi142 ceph-mon[32508]: 3.c scrub starts 2024-09-20T18:52:18.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:18 smithi142 ceph-mon[32508]: 3.c scrub ok 2024-09-20T18:52:18.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:18 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3205379000' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:52:18.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:18 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:52:18.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:18 smithi142 ceph-mon[32508]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:52:18.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:18 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T18:52:18.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:18 smithi142 ceph-mon[32508]: osdmap e119: 8 total, 8 up, 8 in 2024-09-20T18:52:18.283 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:18.312 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:18.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:18.313 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:18.369 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-20T18:52:18.414 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:18.455 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:18.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:18.456 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:18.536 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 81 16 113674 113658 87.6741 51.4375 0.00736574 0.0114011 2024-09-20T18:52:18.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:18 smithi083 ceph-mon[29421]: pgmap v108: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.8 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.6 KiB/s rd, 97 MiB/s wr, 1.55k op/s; 17 B/s, 0 objects/s recovering 2024-09-20T18:52:18.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:18 smithi083 ceph-mon[29421]: 7.c scrub ok 2024-09-20T18:52:18.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:18 smithi083 ceph-mon[29421]: 3.c scrub starts 2024-09-20T18:52:18.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:18 smithi083 ceph-mon[29421]: 3.c scrub ok 2024-09-20T18:52:18.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:18 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3205379000' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:52:18.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:18 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:52:18.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:18 smithi083 ceph-mon[29421]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:52:18.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:18 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T18:52:18.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:18 smithi083 ceph-mon[29421]: osdmap e119: 8 total, 8 up, 8 in 2024-09-20T18:52:18.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:18 smithi083 ceph-mon[36260]: pgmap v108: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.8 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.6 KiB/s rd, 97 MiB/s wr, 1.55k op/s; 17 B/s, 0 objects/s recovering 2024-09-20T18:52:18.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:18 smithi083 ceph-mon[36260]: 7.c scrub ok 2024-09-20T18:52:18.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:18 smithi083 ceph-mon[36260]: 3.c scrub starts 2024-09-20T18:52:18.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:18 smithi083 ceph-mon[36260]: 3.c scrub ok 2024-09-20T18:52:18.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:18 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3205379000' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:52:18.556 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:18 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:52:18.556 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:18 smithi083 ceph-mon[36260]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:52:18.556 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:18 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T18:52:18.556 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:18 smithi083 ceph-mon[36260]: osdmap e119: 8 total, 8 up, 8 in 2024-09-20T18:52:18.557 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:18.593 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:18.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:18.594 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:18.695 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:18.730 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:18.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:18.731 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:52:18.744 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:52:18.832 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:18.877 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:18.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:18.878 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:18.978 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:19.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:19.027 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:19.027 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:19.128 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:19.153 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:19.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:19.155 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:19.256 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:19.291 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:19.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:19.292 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:19.307 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T18:52:19.307 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:52:19.393 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:19.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:19.424 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:19.424 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:52:19.525 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:19.536 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 82 16 114783 114767 87.45 69.3125 0.0206424 0.01143 2024-09-20T18:52:19.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:19.563 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:19.563 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:19.664 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:19.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:19.709 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:19.709 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:19.810 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:52:19.837 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:19.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:19.839 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:52:19.939 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:19.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:19.975 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:19.975 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:52:20.076 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:20.110 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_blocked_ops 2024-09-20T18:52:20.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:20.127 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:20.128 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:20.190 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:52:20.190 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:52:20.207 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:52:20.229 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:20.239 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:52:20.239 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:52:20.239 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:20 smithi083 ceph-mon[29421]: pgmap v110: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.8 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 853 B/s rd, 47 MiB/s wr, 752 op/s; 18 B/s, 0 objects/s recovering 2024-09-20T18:52:20.240 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:20 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/735986561' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:52:20.240 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:20 smithi083 ceph-mon[36260]: pgmap v110: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.8 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 853 B/s rd, 47 MiB/s wr, 752 op/s; 18 B/s, 0 objects/s recovering 2024-09-20T18:52:20.240 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:20 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/735986561' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:52:20.275 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:52:20.275 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:52:20.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:20.299 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:20.300 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:20.314 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:52:20.314 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:52:20.346 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:52:20.346 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:52:20.381 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:52:20.381 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:52:20.401 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:20.424 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:52:20.424 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:52:20.438 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:52:20.441 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:20.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:20.441 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:20.456 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:52:20.456 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:52:20.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:20 smithi142 ceph-mon[32508]: pgmap v110: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.8 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 853 B/s rd, 47 MiB/s wr, 752 op/s; 18 B/s, 0 objects/s recovering 2024-09-20T18:52:20.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:20 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/735986561' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:52:20.539 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 83 16 115805 115789 87.1626 63.875 0.00415064 0.011468 2024-09-20T18:52:20.543 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:52:20.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:20.572 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:20.572 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:52:20.673 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:20.708 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:20.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:20.709 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:20.810 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:20.843 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:20.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:20.844 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:20.945 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:20.983 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:20.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:20.984 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:21.054 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:52:20 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:52:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:52:21.085 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:21.112 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:21.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:21.113 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:52:21.114 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T18:52:21.114 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-09-20T18:52:21.114 INFO:teuthology.orchestra.run.smithi142.stdout: "complaint_time": 30, 2024-09-20T18:52:21.114 INFO:teuthology.orchestra.run.smithi142.stdout: "num_blocked_ops": 0 2024-09-20T18:52:21.114 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T18:52:21.214 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:21.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:21.264 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:21.264 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:21.365 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:21.425 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:21.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:21.426 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:21.526 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:21.540 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T18:52:21.541 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 84 16 116823 116807 86.881 63.625 0.00744958 0.0115049 2024-09-20T18:52:21.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:21.577 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:21.578 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:21.678 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:21.721 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:21.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:21.722 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:21.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:21 smithi142 ceph-mon[32508]: osdmap e120: 8 total, 8 up, 8 in 2024-09-20T18:52:21.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:21 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:52:21.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:21 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:52:21.823 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:21.844 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_historic_ops 2024-09-20T18:52:21.848 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:21.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:21.849 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:21.949 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:52:21.981 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:21.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:21.982 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:52:22.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:21 smithi083 ceph-mon[29421]: osdmap e120: 8 total, 8 up, 8 in 2024-09-20T18:52:22.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:21 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:52:22.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:21 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:52:22.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:21 smithi083 ceph-mon[36260]: osdmap e120: 8 total, 8 up, 8 in 2024-09-20T18:52:22.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:21 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:52:22.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:21 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:52:22.083 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:22.127 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:22.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:22.128 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:22.229 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:22.232 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:52:22.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:22.258 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:22.258 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:22.359 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:22.388 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:22.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:22.389 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:22.490 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:22.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:22.530 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:22.530 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:22.541 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 85 16 117818 117802 86.5905 62.1875 0.0123237 0.0115437 2024-09-20T18:52:22.631 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:22.657 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:22.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:22.658 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:22.759 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:22.783 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:22.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:22.784 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:22.880 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:22 smithi083 ceph-mon[29421]: pgmap v112: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.9 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.8 KiB/s rd, 69 MiB/s wr, 1.10k op/s; 19 B/s, 0 objects/s recovering 2024-09-20T18:52:22.881 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:22 smithi083 ceph-mon[36260]: pgmap v112: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.9 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.8 KiB/s rd, 69 MiB/s wr, 1.10k op/s; 19 B/s, 0 objects/s recovering 2024-09-20T18:52:22.884 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:22.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:22.936 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:22.936 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:22.992 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:52:22.994 INFO:teuthology.orchestra.run.smithi083.stderr:no valid command found; 10 closest matches: 2024-09-20T18:52:22.994 INFO:teuthology.orchestra.run.smithi083.stderr:0 2024-09-20T18:52:22.994 INFO:teuthology.orchestra.run.smithi083.stderr:1 2024-09-20T18:52:22.995 INFO:teuthology.orchestra.run.smithi083.stderr:2 2024-09-20T18:52:22.995 INFO:teuthology.orchestra.run.smithi083.stderr:abort 2024-09-20T18:52:22.995 INFO:teuthology.orchestra.run.smithi083.stderr:assert 2024-09-20T18:52:22.995 INFO:teuthology.orchestra.run.smithi083.stderr:bench [] [] [] [] 2024-09-20T18:52:22.995 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs debug_inject_read_zeros 2024-09-20T18:52:22.995 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs files list 2024-09-20T18:52:22.995 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs stats 2024-09-20T18:52:22.995 INFO:teuthology.orchestra.run.smithi083.stderr:bluestore allocator dump block 2024-09-20T18:52:22.995 INFO:teuthology.orchestra.run.smithi083.stderr:Error EINVAL: invalid command 2024-09-20T18:52:23.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:22 smithi142 ceph-mon[32508]: pgmap v112: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.9 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.8 KiB/s rd, 69 MiB/s wr, 1.10k op/s; 19 B/s, 0 objects/s recovering 2024-09-20T18:52:23.037 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:23.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:23.099 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:23.099 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:23.200 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:23.224 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:23.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:23.225 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:52:23.326 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:23.356 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:23.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:23.357 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:23.458 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:23.491 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:23.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:23.492 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:23.541 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 86 16 118761 118745 86.269 58.9375 0.0179015 0.0115864 2024-09-20T18:52:23.593 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:23.631 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:23.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:23.632 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:23.654 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:23 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:52:23.654 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:23 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:52:23.654 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:23 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:52:23.655 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T18:52:23.655 INFO:teuthology.orchestra.run.smithi142.stdout: "size": 20, 2024-09-20T18:52:23.655 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 600, 2024-09-20T18:52:23.655 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-09-20T18:52:23.655 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.655 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:23105 7.c 7:37789ba3:::benchmark_data_smithi083_70643_object23104:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-20T18:52:23.655 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:06.088529+0000", 2024-09-20T18:52:23.655 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 77.545442554000005, 2024-09-20T18:52:23.655 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.088489872999999997, 2024-09-20T18:52:23.655 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:52:23.656 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:52:23.656 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:52:23.656 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:52:23.656 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:52:23.656 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:52:23.656 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 23105 2024-09-20T18:52:23.656 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.656 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:52:23.656 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.656 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:52:23.656 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.088529+0000", 2024-09-20T18:52:23.656 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:23.656 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.657 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.657 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:52:23.657 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.088529+0000", 2024-09-20T18:52:23.657 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:23.657 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.657 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.657 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:52:23.657 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.088531+0000", 2024-09-20T18:52:23.657 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.776e-06 2024-09-20T18:52:23.657 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.657 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.657 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:52:23.657 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.088585+0000", 2024-09-20T18:52:23.657 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4432999999999999e-05 2024-09-20T18:52:23.658 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.658 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.658 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:52:23.658 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.088600+0000", 2024-09-20T18:52:23.658 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4365999999999999e-05 2024-09-20T18:52:23.658 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.658 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.658 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:52:23.658 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.088607+0000", 2024-09-20T18:52:23.658 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.3030000000000002e-06 2024-09-20T18:52:23.658 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.658 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.658 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:52:23.659 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.088667+0000", 2024-09-20T18:52:23.659 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0219000000000001e-05 2024-09-20T18:52:23.659 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.659 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.659 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:52:23.659 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.088800+0000", 2024-09-20T18:52:23.659 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013312799999999999 2024-09-20T18:52:23.659 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.659 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.659 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:52:23.659 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.088928+0000", 2024-09-20T18:52:23.659 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012773799999999999 2024-09-20T18:52:23.659 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.659 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.660 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:52:23.660 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.090073+0000", 2024-09-20T18:52:23.660 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001145423 2024-09-20T18:52:23.660 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.660 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.660 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:23.660 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.092135+0000", 2024-09-20T18:52:23.660 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0020611079999999999 2024-09-20T18:52:23.660 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.660 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.660 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:23.660 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.176931+0000", 2024-09-20T18:52:23.660 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.084796175000000001 2024-09-20T18:52:23.661 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.661 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.661 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:52:23.661 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.176973+0000", 2024-09-20T18:52:23.661 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.1919000000000003e-05 2024-09-20T18:52:23.661 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.661 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.661 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:52:23.661 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.177019+0000", 2024-09-20T18:52:23.661 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.6285000000000002e-05 2024-09-20T18:52:23.661 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:23.661 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:52:23.661 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:23.662 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.662 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.662 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:23115 7.7 7:e8538c26:::benchmark_data_smithi083_70643_object23114:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-20T18:52:23.662 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:06.091673+0000", 2024-09-20T18:52:23.662 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 77.542298328000001, 2024-09-20T18:52:23.662 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.088653336999999999, 2024-09-20T18:52:23.662 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:52:23.662 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:52:23.662 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:52:23.662 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:52:23.662 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:52:23.662 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:52:23.662 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 23115 2024-09-20T18:52:23.663 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.663 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:52:23.663 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.663 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:52:23.663 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.091673+0000", 2024-09-20T18:52:23.663 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:23.663 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.663 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.663 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:52:23.663 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.091673+0000", 2024-09-20T18:52:23.663 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:23.663 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.664 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.664 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:52:23.664 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.091675+0000", 2024-09-20T18:52:23.664 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5859999999999999e-06 2024-09-20T18:52:23.664 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.664 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.664 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:52:23.664 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.091739+0000", 2024-09-20T18:52:23.664 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.3928000000000002e-05 2024-09-20T18:52:23.664 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.664 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.664 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:52:23.664 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.091753+0000", 2024-09-20T18:52:23.664 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4436e-05 2024-09-20T18:52:23.665 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.665 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.665 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:52:23.665 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.091762+0000", 2024-09-20T18:52:23.665 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.6500000000000002e-06 2024-09-20T18:52:23.665 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.665 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.665 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:52:23.665 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.091818+0000", 2024-09-20T18:52:23.666 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5846e-05 2024-09-20T18:52:23.666 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.666 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.666 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:52:23.666 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.091953+0000", 2024-09-20T18:52:23.666 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013510900000000001 2024-09-20T18:52:23.666 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.667 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.667 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,3", 2024-09-20T18:52:23.667 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.092094+0000", 2024-09-20T18:52:23.667 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00014164000000000001 2024-09-20T18:52:23.667 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.667 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.668 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:52:23.668 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.092998+0000", 2024-09-20T18:52:23.668 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00090327400000000003 2024-09-20T18:52:23.668 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.668 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.668 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:23.668 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.154085+0000", 2024-09-20T18:52:23.668 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.061087229999999999 2024-09-20T18:52:23.668 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.669 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.669 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:23.669 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.180215+0000", 2024-09-20T18:52:23.669 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.026130434000000001 2024-09-20T18:52:23.669 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.669 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.669 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:52:23.669 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.180272+0000", 2024-09-20T18:52:23.669 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6338000000000002e-05 2024-09-20T18:52:23.669 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.669 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.669 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:52:23.669 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.180327+0000", 2024-09-20T18:52:23.669 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4866e-05 2024-09-20T18:52:23.670 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:23.670 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:52:23.670 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:23.670 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.670 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.670 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:24945 7.7 7:e97e81cf:::benchmark_data_smithi083_70643_object24944:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:52:23.670 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:06.996434+0000", 2024-09-20T18:52:23.670 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 76.637537936000001, 2024-09-20T18:52:23.670 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.117472831, 2024-09-20T18:52:23.670 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:52:23.670 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:52:23.670 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:52:23.670 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:52:23.671 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:52:23.671 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:52:23.671 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 24945 2024-09-20T18:52:23.671 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.671 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:52:23.671 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.671 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:52:23.671 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.996434+0000", 2024-09-20T18:52:23.671 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:23.671 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.671 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.671 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:52:23.672 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.996434+0000", 2024-09-20T18:52:23.672 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:23.672 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.672 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.672 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:52:23.672 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.996435+0000", 2024-09-20T18:52:23.672 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4869999999999999e-06 2024-09-20T18:52:23.672 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.672 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.672 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:52:23.672 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.996477+0000", 2024-09-20T18:52:23.673 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.1601999999999999e-05 2024-09-20T18:52:23.673 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.673 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.673 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:52:23.673 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.996488+0000", 2024-09-20T18:52:23.673 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0893999999999999e-05 2024-09-20T18:52:23.673 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.673 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.673 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:52:23.673 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.996495+0000", 2024-09-20T18:52:23.674 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.1720000000000001e-06 2024-09-20T18:52:23.674 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.674 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.674 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:52:23.674 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.996709+0000", 2024-09-20T18:52:23.674 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00021389099999999999 2024-09-20T18:52:23.674 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.674 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.674 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:52:23.674 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.996918+0000", 2024-09-20T18:52:23.674 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00020888100000000001 2024-09-20T18:52:23.674 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.674 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.675 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,3", 2024-09-20T18:52:23.675 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.997122+0000", 2024-09-20T18:52:23.675 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00020468 2024-09-20T18:52:23.675 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.675 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.675 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:52:23.675 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.004028+0000", 2024-09-20T18:52:23.675 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0069058180000000002 2024-09-20T18:52:23.675 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.675 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.675 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:23.675 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.004092+0000", 2024-09-20T18:52:23.675 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4323000000000001e-05 2024-09-20T18:52:23.675 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.675 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.676 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:23.676 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.113828+0000", 2024-09-20T18:52:23.676 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.109735151 2024-09-20T18:52:23.676 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.676 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.676 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:52:23.676 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.113870+0000", 2024-09-20T18:52:23.676 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.2258000000000001e-05 2024-09-20T18:52:23.676 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.676 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.676 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:52:23.676 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.113906+0000", 2024-09-20T18:52:23.676 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6674000000000002e-05 2024-09-20T18:52:23.676 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:23.676 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:52:23.676 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:23.677 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.677 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.677 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:24951 7.7 7:ebe82687:::benchmark_data_smithi083_70643_object24950:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:52:23.677 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:06.997097+0000", 2024-09-20T18:52:23.677 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 76.636874969000004, 2024-09-20T18:52:23.677 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.12447738799999999, 2024-09-20T18:52:23.677 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:52:23.677 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:52:23.677 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:52:23.677 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:52:23.677 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:52:23.677 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:52:23.677 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 24951 2024-09-20T18:52:23.678 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.678 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:52:23.678 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.678 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:52:23.678 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.997097+0000", 2024-09-20T18:52:23.678 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:23.678 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.678 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.678 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:52:23.678 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.997097+0000", 2024-09-20T18:52:23.678 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:23.678 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.678 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.678 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:52:23.679 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.997098+0000", 2024-09-20T18:52:23.679 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.082e-06 2024-09-20T18:52:23.679 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.679 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.679 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:52:23.679 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.997210+0000", 2024-09-20T18:52:23.679 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011240500000000001 2024-09-20T18:52:23.679 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.679 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.679 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:52:23.679 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.997223+0000", 2024-09-20T18:52:23.679 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2609000000000001e-05 2024-09-20T18:52:23.679 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.679 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.679 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:52:23.680 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.997230+0000", 2024-09-20T18:52:23.680 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5700000000000004e-06 2024-09-20T18:52:23.680 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.680 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.680 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:52:23.680 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.000085+0000", 2024-09-20T18:52:23.680 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.002855013 2024-09-20T18:52:23.680 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.680 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.680 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:52:23.680 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.000200+0000", 2024-09-20T18:52:23.680 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011476600000000001 2024-09-20T18:52:23.680 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.680 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.681 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,3", 2024-09-20T18:52:23.681 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.001210+0000", 2024-09-20T18:52:23.681 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0010098850000000001 2024-09-20T18:52:23.681 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.681 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.681 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:52:23.681 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.005107+0000", 2024-09-20T18:52:23.681 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0038972730000000001 2024-09-20T18:52:23.681 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.681 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.681 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:23.681 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.103702+0000", 2024-09-20T18:52:23.681 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.098595123000000007 2024-09-20T18:52:23.681 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.681 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.682 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:23.682 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.121471+0000", 2024-09-20T18:52:23.682 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.017768227000000001 2024-09-20T18:52:23.682 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.682 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.682 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:52:23.682 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.121533+0000", 2024-09-20T18:52:23.682 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2489000000000004e-05 2024-09-20T18:52:23.682 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.682 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.682 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:52:23.682 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.121574+0000", 2024-09-20T18:52:23.682 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0945999999999997e-05 2024-09-20T18:52:23.682 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:23.682 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:52:23.683 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:23.683 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.683 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.683 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:24956 7.7 7:e91ba513:::benchmark_data_smithi083_70643_object24955:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:52:23.683 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:06.999473+0000", 2024-09-20T18:52:23.683 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 76.634498858000001, 2024-09-20T18:52:23.683 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.122248832, 2024-09-20T18:52:23.683 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:52:23.683 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:52:23.683 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:52:23.683 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:52:23.683 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:52:23.683 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:52:23.683 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 24956 2024-09-20T18:52:23.683 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.684 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:52:23.684 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.684 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:52:23.684 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.999473+0000", 2024-09-20T18:52:23.684 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:23.684 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.684 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.684 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:52:23.684 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.999473+0000", 2024-09-20T18:52:23.684 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:23.685 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.685 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.685 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:52:23.685 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.999475+0000", 2024-09-20T18:52:23.685 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9750000000000001e-06 2024-09-20T18:52:23.685 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.685 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.685 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:52:23.685 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.999545+0000", 2024-09-20T18:52:23.685 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.0104000000000005e-05 2024-09-20T18:52:23.685 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.685 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.686 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:52:23.686 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.999580+0000", 2024-09-20T18:52:23.686 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.5284000000000003e-05 2024-09-20T18:52:23.686 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.686 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.686 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:52:23.686 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.999590+0000", 2024-09-20T18:52:23.686 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.6739999999999992e-06 2024-09-20T18:52:23.686 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.686 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.686 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:52:23.686 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002183+0000", 2024-09-20T18:52:23.686 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0025934899999999999 2024-09-20T18:52:23.686 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.687 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.687 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:52:23.687 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002933+0000", 2024-09-20T18:52:23.687 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00074923700000000004 2024-09-20T18:52:23.687 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.687 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.687 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,3", 2024-09-20T18:52:23.687 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.003100+0000", 2024-09-20T18:52:23.687 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000167651 2024-09-20T18:52:23.687 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.687 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.687 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:52:23.688 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.005113+0000", 2024-09-20T18:52:23.688 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0020131749999999999 2024-09-20T18:52:23.688 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.688 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.688 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:23.688 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.104767+0000", 2024-09-20T18:52:23.688 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.099653505000000003 2024-09-20T18:52:23.688 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.688 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.688 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:23.688 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.121622+0000", 2024-09-20T18:52:23.688 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.016855098999999998 2024-09-20T18:52:23.688 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.688 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.689 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:52:23.689 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.121681+0000", 2024-09-20T18:52:23.689 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9199e-05 2024-09-20T18:52:23.689 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.689 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.689 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:52:23.689 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.121722+0000", 2024-09-20T18:52:23.689 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0438999999999997e-05 2024-09-20T18:52:23.689 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:23.689 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:52:23.689 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:23.689 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.689 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.690 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:24960 7.c 7:3cbb268e:::benchmark_data_smithi083_70643_object24959:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:52:23.690 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:07.001793+0000", 2024-09-20T18:52:23.690 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 76.632178109999998, 2024-09-20T18:52:23.690 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.117162266, 2024-09-20T18:52:23.690 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:52:23.690 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:52:23.690 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:52:23.690 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:52:23.690 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:52:23.690 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:52:23.690 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 24960 2024-09-20T18:52:23.690 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.690 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:52:23.691 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.691 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:52:23.691 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.001793+0000", 2024-09-20T18:52:23.691 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:23.691 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.691 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.691 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:52:23.692 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.001793+0000", 2024-09-20T18:52:23.692 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:23.692 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.692 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.692 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:52:23.692 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.001795+0000", 2024-09-20T18:52:23.693 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3069999999999999e-06 2024-09-20T18:52:23.693 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.693 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.693 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:52:23.693 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.001853+0000", 2024-09-20T18:52:23.693 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8372999999999998e-05 2024-09-20T18:52:23.693 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.693 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.693 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:52:23.693 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.001865+0000", 2024-09-20T18:52:23.693 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2004999999999999e-05 2024-09-20T18:52:23.693 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.694 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.694 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:52:23.694 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.001873+0000", 2024-09-20T18:52:23.694 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.362e-06 2024-09-20T18:52:23.694 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.694 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.694 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:52:23.694 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.001933+0000", 2024-09-20T18:52:23.694 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9998999999999999e-05 2024-09-20T18:52:23.694 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.694 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.694 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:52:23.694 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002069+0000", 2024-09-20T18:52:23.694 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013620799999999999 2024-09-20T18:52:23.695 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.695 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.695 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:52:23.695 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002203+0000", 2024-09-20T18:52:23.695 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000134048 2024-09-20T18:52:23.695 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.695 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.695 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:23.695 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.004679+0000", 2024-09-20T18:52:23.695 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0024765479999999999 2024-09-20T18:52:23.695 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.695 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.695 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:52:23.695 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.005474+0000", 2024-09-20T18:52:23.696 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00079475399999999995 2024-09-20T18:52:23.696 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.696 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.696 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:23.696 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.118886+0000", 2024-09-20T18:52:23.696 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.11341159200000001 2024-09-20T18:52:23.696 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.696 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.696 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:52:23.696 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.118928+0000", 2024-09-20T18:52:23.696 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.2460999999999999e-05 2024-09-20T18:52:23.696 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.697 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.697 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:52:23.697 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.118956+0000", 2024-09-20T18:52:23.697 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.7608999999999999e-05 2024-09-20T18:52:23.697 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:23.697 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:52:23.697 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:23.697 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.697 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.697 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:24961 7.c 7:35ca9a87:::benchmark_data_smithi083_70643_object24960:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:52:23.697 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:07.001951+0000", 2024-09-20T18:52:23.697 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 76.632020901000004, 2024-09-20T18:52:23.697 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.11716043299999999, 2024-09-20T18:52:23.698 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:52:23.698 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:52:23.698 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:52:23.698 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:52:23.698 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:52:23.698 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:52:23.698 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 24961 2024-09-20T18:52:23.698 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.698 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:52:23.698 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.698 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:52:23.698 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.001951+0000", 2024-09-20T18:52:23.699 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:23.699 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.699 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.699 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:52:23.699 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.001951+0000", 2024-09-20T18:52:23.699 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:23.699 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.699 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.699 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:52:23.699 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.001951+0000", 2024-09-20T18:52:23.699 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.7499999999999999e-07 2024-09-20T18:52:23.699 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.699 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.700 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:52:23.700 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002093+0000", 2024-09-20T18:52:23.700 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000141433 2024-09-20T18:52:23.700 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.700 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.700 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:52:23.700 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002103+0000", 2024-09-20T18:52:23.700 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.9979999999999998e-06 2024-09-20T18:52:23.700 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.700 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.700 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:52:23.700 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002110+0000", 2024-09-20T18:52:23.700 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.6170000000000003e-06 2024-09-20T18:52:23.700 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.701 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.701 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:52:23.701 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.003810+0000", 2024-09-20T18:52:23.701 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001700221 2024-09-20T18:52:23.701 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.701 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.701 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:52:23.701 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.003895+0000", 2024-09-20T18:52:23.701 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.4886e-05 2024-09-20T18:52:23.701 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.701 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.701 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:52:23.701 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.003978+0000", 2024-09-20T18:52:23.701 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.3812000000000003e-05 2024-09-20T18:52:23.702 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.702 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.702 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:52:23.702 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.005480+0000", 2024-09-20T18:52:23.702 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001501245 2024-09-20T18:52:23.702 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.702 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.702 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:23.702 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.005508+0000", 2024-09-20T18:52:23.702 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.8668000000000001e-05 2024-09-20T18:52:23.702 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.702 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.702 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:23.703 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.119009+0000", 2024-09-20T18:52:23.703 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.113500961 2024-09-20T18:52:23.703 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.703 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.703 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:52:23.703 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.119073+0000", 2024-09-20T18:52:23.703 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.3378000000000002e-05 2024-09-20T18:52:23.703 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.703 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.703 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:52:23.703 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.119111+0000", 2024-09-20T18:52:23.703 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.8439000000000002e-05 2024-09-20T18:52:23.704 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:23.704 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:52:23.704 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:23.704 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.704 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.704 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:24962 7.7 7:efd119ca:::benchmark_data_smithi083_70643_object24961:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:52:23.704 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:07.002480+0000", 2024-09-20T18:52:23.704 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 76.631491209999993, 2024-09-20T18:52:23.704 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.11963515, 2024-09-20T18:52:23.704 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:52:23.704 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:52:23.705 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:52:23.705 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:52:23.705 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:52:23.705 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:52:23.705 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 24962 2024-09-20T18:52:23.705 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.705 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:52:23.705 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.705 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:52:23.705 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002480+0000", 2024-09-20T18:52:23.705 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:23.705 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.705 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.706 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:52:23.706 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002480+0000", 2024-09-20T18:52:23.706 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:23.706 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.706 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.706 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:52:23.706 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002481+0000", 2024-09-20T18:52:23.706 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0860000000000001e-06 2024-09-20T18:52:23.706 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.706 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.706 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:52:23.706 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002511+0000", 2024-09-20T18:52:23.706 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9499999999999999e-05 2024-09-20T18:52:23.707 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.707 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.707 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:52:23.707 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002522+0000", 2024-09-20T18:52:23.707 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0652000000000001e-05 2024-09-20T18:52:23.707 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.707 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.707 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:52:23.707 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002529+0000", 2024-09-20T18:52:23.707 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.1319999999999996e-06 2024-09-20T18:52:23.707 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.707 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.707 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:52:23.707 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.004125+0000", 2024-09-20T18:52:23.708 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0015959570000000001 2024-09-20T18:52:23.708 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.708 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.708 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:52:23.708 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.004232+0000", 2024-09-20T18:52:23.708 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000107767 2024-09-20T18:52:23.708 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.708 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.708 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,3", 2024-09-20T18:52:23.708 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.004366+0000", 2024-09-20T18:52:23.708 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013360299999999999 2024-09-20T18:52:23.708 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.708 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.709 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:52:23.709 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.005279+0000", 2024-09-20T18:52:23.709 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00091312100000000003 2024-09-20T18:52:23.709 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.709 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.709 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:23.709 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.104849+0000", 2024-09-20T18:52:23.709 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.099569569999999996 2024-09-20T18:52:23.709 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.709 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.709 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:23.709 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.122006+0000", 2024-09-20T18:52:23.709 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.017157531 2024-09-20T18:52:23.710 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.710 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.710 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:52:23.710 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.122071+0000", 2024-09-20T18:52:23.710 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4372e-05 2024-09-20T18:52:23.710 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.710 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.710 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:52:23.710 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.122116+0000", 2024-09-20T18:52:23.710 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.4858999999999999e-05 2024-09-20T18:52:23.710 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:23.710 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:52:23.710 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:23.710 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.711 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.711 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:24963 7.c 7:38caa154:::benchmark_data_smithi083_70643_object24962:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:52:23.711 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:07.002541+0000", 2024-09-20T18:52:23.711 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 76.631430730999995, 2024-09-20T18:52:23.711 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.11664609500000001, 2024-09-20T18:52:23.711 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:52:23.711 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:52:23.711 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:52:23.711 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:52:23.711 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:52:23.711 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:52:23.711 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 24963 2024-09-20T18:52:23.711 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.711 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:52:23.712 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.712 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:52:23.712 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002541+0000", 2024-09-20T18:52:23.712 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:23.712 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.712 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.712 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:52:23.712 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002541+0000", 2024-09-20T18:52:23.712 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:23.712 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.712 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.712 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:52:23.712 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002541+0000", 2024-09-20T18:52:23.713 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.2900000000000004e-07 2024-09-20T18:52:23.713 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.713 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.713 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:52:23.713 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002569+0000", 2024-09-20T18:52:23.713 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.7866999999999999e-05 2024-09-20T18:52:23.713 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.713 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.713 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:52:23.713 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002576+0000", 2024-09-20T18:52:23.713 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.0929999999999997e-06 2024-09-20T18:52:23.713 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.713 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.714 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:52:23.714 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002582+0000", 2024-09-20T18:52:23.714 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.575e-06 2024-09-20T18:52:23.714 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.714 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.714 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:52:23.714 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.004714+0000", 2024-09-20T18:52:23.714 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0021320430000000001 2024-09-20T18:52:23.714 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.715 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.715 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:52:23.715 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.004828+0000", 2024-09-20T18:52:23.715 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000113632 2024-09-20T18:52:23.715 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.715 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.715 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:52:23.715 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.004947+0000", 2024-09-20T18:52:23.715 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000119367 2024-09-20T18:52:23.715 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.715 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.715 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:52:23.715 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.005749+0000", 2024-09-20T18:52:23.716 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000801898 2024-09-20T18:52:23.716 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.716 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.716 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:23.716 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.005930+0000", 2024-09-20T18:52:23.716 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000180686 2024-09-20T18:52:23.716 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.716 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.716 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:23.716 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.119138+0000", 2024-09-20T18:52:23.716 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.11320831200000001 2024-09-20T18:52:23.716 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.716 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.716 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:52:23.716 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.119166+0000", 2024-09-20T18:52:23.717 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.8121000000000001e-05 2024-09-20T18:52:23.717 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.717 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.717 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:52:23.717 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.119187+0000", 2024-09-20T18:52:23.717 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0971999999999998e-05 2024-09-20T18:52:23.717 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:23.717 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:52:23.717 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:23.717 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.717 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.717 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:25344 7.7 7:ead30b22:::benchmark_data_smithi083_70643_object25343:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:52:23.717 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:07.270133+0000", 2024-09-20T18:52:23.717 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 76.363838586, 2024-09-20T18:52:23.718 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.17431155100000001, 2024-09-20T18:52:23.718 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:52:23.718 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:52:23.718 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:52:23.718 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:52:23.718 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:52:23.718 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:52:23.718 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 25344 2024-09-20T18:52:23.718 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.718 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:52:23.718 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.718 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:52:23.718 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.270133+0000", 2024-09-20T18:52:23.719 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:23.719 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.719 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.719 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:52:23.719 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.270133+0000", 2024-09-20T18:52:23.719 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:23.719 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.719 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.719 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:52:23.719 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.270134+0000", 2024-09-20T18:52:23.719 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4619999999999999e-06 2024-09-20T18:52:23.719 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.719 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.719 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:52:23.720 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.270200+0000", 2024-09-20T18:52:23.720 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5998000000000001e-05 2024-09-20T18:52:23.720 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.720 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.720 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:52:23.720 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.270214+0000", 2024-09-20T18:52:23.720 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3631e-05 2024-09-20T18:52:23.720 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.720 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.720 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:52:23.720 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.270222+0000", 2024-09-20T18:52:23.720 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.6149999999999999e-06 2024-09-20T18:52:23.721 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.721 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.721 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:52:23.721 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.270277+0000", 2024-09-20T18:52:23.721 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4981000000000001e-05 2024-09-20T18:52:23.721 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.721 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.722 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:52:23.722 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.270400+0000", 2024-09-20T18:52:23.722 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012379000000000001 2024-09-20T18:52:23.722 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.722 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.722 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,3", 2024-09-20T18:52:23.723 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.270534+0000", 2024-09-20T18:52:23.723 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013307600000000001 2024-09-20T18:52:23.723 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.723 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.723 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:52:23.723 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.271704+0000", 2024-09-20T18:52:23.724 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001170152 2024-09-20T18:52:23.724 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.724 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.724 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:23.724 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.442076+0000", 2024-09-20T18:52:23.724 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.170371932 2024-09-20T18:52:23.724 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.725 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.725 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:23.725 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.444327+0000", 2024-09-20T18:52:23.725 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.002251454 2024-09-20T18:52:23.725 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.725 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.726 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:52:23.726 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.444386+0000", 2024-09-20T18:52:23.726 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8579999999999998e-05 2024-09-20T18:52:23.726 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.726 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.726 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:52:23.727 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.444445+0000", 2024-09-20T18:52:23.727 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8879999999999999e-05 2024-09-20T18:52:23.727 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:23.727 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:52:23.727 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:23.727 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.727 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.728 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:26469 7.7 7:edf96f0c:::benchmark_data_smithi083_70643_object26468:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:52:23.728 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:07.996717+0000", 2024-09-20T18:52:23.728 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 75.637255060000001, 2024-09-20T18:52:23.728 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13515669799999999, 2024-09-20T18:52:23.728 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:52:23.728 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:52:23.729 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:52:23.729 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:52:23.729 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:52:23.729 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:52:23.729 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 26469 2024-09-20T18:52:23.729 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.730 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:52:23.730 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.730 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:52:23.730 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.996717+0000", 2024-09-20T18:52:23.730 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:23.730 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.731 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.731 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:52:23.731 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.996717+0000", 2024-09-20T18:52:23.731 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:23.731 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.731 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.732 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:52:23.732 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.996718+0000", 2024-09-20T18:52:23.732 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6020000000000001e-06 2024-09-20T18:52:23.732 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.732 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.732 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:52:23.732 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.996814+0000", 2024-09-20T18:52:23.733 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.5480000000000001e-05 2024-09-20T18:52:23.733 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.733 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.733 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:52:23.733 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.996827+0000", 2024-09-20T18:52:23.733 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3749e-05 2024-09-20T18:52:23.734 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.734 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.734 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:52:23.734 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.996835+0000", 2024-09-20T18:52:23.734 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.2309999999999999e-06 2024-09-20T18:52:23.734 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.734 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.734 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:52:23.735 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.996897+0000", 2024-09-20T18:52:23.735 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2515999999999996e-05 2024-09-20T18:52:23.735 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.735 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.735 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:52:23.735 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.996992+0000", 2024-09-20T18:52:23.735 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.4870000000000005e-05 2024-09-20T18:52:23.735 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.735 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.736 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,3", 2024-09-20T18:52:23.736 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.997083+0000", 2024-09-20T18:52:23.736 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.1144999999999996e-05 2024-09-20T18:52:23.736 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.736 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.736 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:52:23.736 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.998108+0000", 2024-09-20T18:52:23.736 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0010251679999999999 2024-09-20T18:52:23.736 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.736 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.736 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:23.736 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:23.738 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.998717+0000", 2024-09-20T18:52:23.738 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00060899999999999995 2024-09-20T18:52:23.738 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.739 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.739 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:23.739 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.131755+0000", 2024-09-20T18:52:23.739 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13303793799999999 2024-09-20T18:52:23.739 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.739 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.739 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:52:23.739 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.131820+0000", 2024-09-20T18:52:23.739 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4678999999999995e-05 2024-09-20T18:52:23.739 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.739 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.739 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:52:23.739 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.131873+0000", 2024-09-20T18:52:23.740 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3319999999999998e-05 2024-09-20T18:52:23.740 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:23.740 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:52:23.740 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:23.740 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.740 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.740 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:26472 7.c 7:326250cc:::benchmark_data_smithi083_70643_object26471:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:52:23.740 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:07.999252+0000", 2024-09-20T18:52:23.740 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 75.634719517999997, 2024-09-20T18:52:23.740 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.133437312, 2024-09-20T18:52:23.740 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:52:23.740 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:52:23.740 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:52:23.740 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:52:23.741 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:52:23.741 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:52:23.741 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 26472 2024-09-20T18:52:23.741 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.741 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:52:23.741 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.741 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:52:23.741 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.999252+0000", 2024-09-20T18:52:23.741 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:23.741 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.741 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.741 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:52:23.741 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.999252+0000", 2024-09-20T18:52:23.742 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:23.742 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.742 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.742 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:52:23.742 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.999253+0000", 2024-09-20T18:52:23.742 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.44e-07 2024-09-20T18:52:23.742 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.742 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.742 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:52:23.742 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.999278+0000", 2024-09-20T18:52:23.742 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.5661e-05 2024-09-20T18:52:23.742 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.742 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.743 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:52:23.743 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.999287+0000", 2024-09-20T18:52:23.743 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.2600000000000005e-06 2024-09-20T18:52:23.743 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.743 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.743 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:52:23.743 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.999293+0000", 2024-09-20T18:52:23.743 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4609999999999996e-06 2024-09-20T18:52:23.743 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.743 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.743 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:52:23.744 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.999366+0000", 2024-09-20T18:52:23.744 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.2695999999999997e-05 2024-09-20T18:52:23.744 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.744 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.744 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:52:23.744 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.999475+0000", 2024-09-20T18:52:23.744 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000109472 2024-09-20T18:52:23.744 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.744 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.744 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:52:23.744 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.000511+0000", 2024-09-20T18:52:23.744 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0010355 2024-09-20T18:52:23.744 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.744 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.745 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:52:23.745 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.002097+0000", 2024-09-20T18:52:23.745 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0015861429999999999 2024-09-20T18:52:23.745 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.745 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.745 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:23.745 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.002781+0000", 2024-09-20T18:52:23.745 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000684039 2024-09-20T18:52:23.745 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.745 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.745 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:23.746 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.132602+0000", 2024-09-20T18:52:23.746 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.129820825 2024-09-20T18:52:23.746 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.746 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.746 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:52:23.746 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.132652+0000", 2024-09-20T18:52:23.746 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0139999999999998e-05 2024-09-20T18:52:23.746 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.746 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.746 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:52:23.746 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.132689+0000", 2024-09-20T18:52:23.747 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.7471000000000001e-05 2024-09-20T18:52:23.747 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:23.747 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:52:23.747 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:23.747 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.747 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.747 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:26478 7.d 7:b12f12bc:::benchmark_data_smithi083_70643_object26477:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:52:23.747 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:08.003662+0000", 2024-09-20T18:52:23.747 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 75.630309654000001, 2024-09-20T18:52:23.747 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.15004061199999999, 2024-09-20T18:52:23.747 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:52:23.747 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:52:23.747 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:52:23.748 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:52:23.748 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:52:23.748 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:52:23.748 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 26478 2024-09-20T18:52:23.748 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.748 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:52:23.748 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.748 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:52:23.748 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003662+0000", 2024-09-20T18:52:23.748 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:23.748 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.748 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.748 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:52:23.749 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003662+0000", 2024-09-20T18:52:23.749 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:23.749 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.749 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.749 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:52:23.749 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003664+0000", 2024-09-20T18:52:23.749 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8959999999999999e-06 2024-09-20T18:52:23.749 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.749 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.749 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:52:23.749 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003707+0000", 2024-09-20T18:52:23.749 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3288999999999998e-05 2024-09-20T18:52:23.749 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.749 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.750 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:52:23.750 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003724+0000", 2024-09-20T18:52:23.750 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6529e-05 2024-09-20T18:52:23.750 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.750 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.750 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:52:23.750 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003732+0000", 2024-09-20T18:52:23.750 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.7679999999999997e-06 2024-09-20T18:52:23.750 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.750 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.750 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:52:23.750 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003800+0000", 2024-09-20T18:52:23.751 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.7904999999999994e-05 2024-09-20T18:52:23.751 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.751 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.751 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:52:23.751 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003941+0000", 2024-09-20T18:52:23.751 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000140731 2024-09-20T18:52:23.751 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.751 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.751 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,1", 2024-09-20T18:52:23.751 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.004096+0000", 2024-09-20T18:52:23.751 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015452700000000001 2024-09-20T18:52:23.751 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.751 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.751 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:52:23.752 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.005860+0000", 2024-09-20T18:52:23.752 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001764383 2024-09-20T18:52:23.752 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.752 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.752 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:23.752 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.005912+0000", 2024-09-20T18:52:23.752 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.2519000000000003e-05 2024-09-20T18:52:23.752 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.752 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.752 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:23.752 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.153622+0000", 2024-09-20T18:52:23.752 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.147709641 2024-09-20T18:52:23.752 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.752 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.753 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:52:23.753 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.153662+0000", 2024-09-20T18:52:23.753 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9966000000000003e-05 2024-09-20T18:52:23.753 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.753 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.753 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:52:23.753 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.153703+0000", 2024-09-20T18:52:23.753 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0457999999999998e-05 2024-09-20T18:52:23.753 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:23.753 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:52:23.753 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:23.753 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.753 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.753 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:26479 7.d 7:bdad06c1:::benchmark_data_smithi083_70643_object26478:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:52:23.754 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:08.003826+0000", 2024-09-20T18:52:23.754 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 75.630145491999997, 2024-09-20T18:52:23.754 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.15140265899999999, 2024-09-20T18:52:23.754 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:52:23.754 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:52:23.754 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:52:23.754 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:52:23.754 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:52:23.754 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:52:23.754 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 26479 2024-09-20T18:52:23.754 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.754 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:52:23.754 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.755 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:52:23.755 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003826+0000", 2024-09-20T18:52:23.755 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:23.755 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.755 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.755 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:52:23.755 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003826+0000", 2024-09-20T18:52:23.755 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:23.755 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.755 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.755 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:52:23.755 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003827+0000", 2024-09-20T18:52:23.755 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.8100000000000002e-07 2024-09-20T18:52:23.755 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.756 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.756 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:52:23.756 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003876+0000", 2024-09-20T18:52:23.756 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8888999999999999e-05 2024-09-20T18:52:23.756 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.756 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.756 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:52:23.756 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003883+0000", 2024-09-20T18:52:23.756 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.8909999999999998e-06 2024-09-20T18:52:23.757 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.757 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.757 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:52:23.757 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003887+0000", 2024-09-20T18:52:23.757 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.5909999999999998e-06 2024-09-20T18:52:23.757 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.757 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.757 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:52:23.757 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.005030+0000", 2024-09-20T18:52:23.757 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0011424289999999999 2024-09-20T18:52:23.757 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.757 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.757 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:52:23.758 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.005164+0000", 2024-09-20T18:52:23.758 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000134537 2024-09-20T18:52:23.758 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.758 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.758 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,1", 2024-09-20T18:52:23.758 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.005242+0000", 2024-09-20T18:52:23.758 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.7956999999999999e-05 2024-09-20T18:52:23.758 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.758 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.758 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:23.758 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.006119+0000", 2024-09-20T18:52:23.758 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00087690700000000001 2024-09-20T18:52:23.759 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.759 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.759 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:52:23.759 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.006138+0000", 2024-09-20T18:52:23.759 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8450000000000001e-05 2024-09-20T18:52:23.759 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.759 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.759 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:23.759 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.155098+0000", 2024-09-20T18:52:23.759 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.14896046900000001 2024-09-20T18:52:23.759 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.760 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.760 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:52:23.760 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.155153+0000", 2024-09-20T18:52:23.760 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5058e-05 2024-09-20T18:52:23.760 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.760 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.760 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:52:23.760 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.155229+0000", 2024-09-20T18:52:23.760 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5699999999999997e-05 2024-09-20T18:52:23.760 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:23.761 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:52:23.761 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:23.761 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.761 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.761 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:26481 7.7 7:eecb08c1:::benchmark_data_smithi083_70643_object26480:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:52:23.761 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:08.006431+0000", 2024-09-20T18:52:23.761 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 75.627540835000005, 2024-09-20T18:52:23.761 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.14993074200000001, 2024-09-20T18:52:23.761 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:52:23.761 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:52:23.761 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:52:23.761 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:52:23.761 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:52:23.762 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:52:23.762 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 26481 2024-09-20T18:52:23.762 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.762 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:52:23.762 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.762 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:52:23.762 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.006431+0000", 2024-09-20T18:52:23.762 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:23.762 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.762 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.762 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:52:23.763 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.006431+0000", 2024-09-20T18:52:23.763 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:23.763 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.763 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.763 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:52:23.763 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.006432+0000", 2024-09-20T18:52:23.763 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.291e-06 2024-09-20T18:52:23.763 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.763 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.763 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:52:23.763 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.006487+0000", 2024-09-20T18:52:23.764 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5058e-05 2024-09-20T18:52:23.764 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.764 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.764 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:52:23.764 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.006497+0000", 2024-09-20T18:52:23.764 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.8579999999999995e-06 2024-09-20T18:52:23.764 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.764 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.764 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:52:23.764 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.006502+0000", 2024-09-20T18:52:23.764 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7140000000000001e-06 2024-09-20T18:52:23.764 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.764 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.765 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:52:23.765 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.006544+0000", 2024-09-20T18:52:23.765 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.2518000000000002e-05 2024-09-20T18:52:23.765 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.765 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.765 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:52:23.765 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.006636+0000", 2024-09-20T18:52:23.765 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.1865000000000002e-05 2024-09-20T18:52:23.765 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.765 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.765 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,3", 2024-09-20T18:52:23.765 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.006753+0000", 2024-09-20T18:52:23.765 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011691100000000001 2024-09-20T18:52:23.765 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.766 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.766 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:52:23.766 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.007675+0000", 2024-09-20T18:52:23.766 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00092171199999999999 2024-09-20T18:52:23.766 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.766 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.766 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:23.766 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.132841+0000", 2024-09-20T18:52:23.766 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.125165949 2024-09-20T18:52:23.766 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.766 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.767 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:23.767 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.156244+0000", 2024-09-20T18:52:23.767 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.023402896999999999 2024-09-20T18:52:23.767 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.767 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.767 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:52:23.767 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.156321+0000", 2024-09-20T18:52:23.767 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.7451000000000001e-05 2024-09-20T18:52:23.767 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.767 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.767 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:52:23.767 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.156362+0000", 2024-09-20T18:52:23.767 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0518000000000001e-05 2024-09-20T18:52:23.767 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:23.768 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:52:23.768 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:23.768 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.768 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.768 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:27562 7.7 7:edf78f76:::benchmark_data_smithi083_70643_object27561:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:52:23.768 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:08.736852+0000", 2024-09-20T18:52:23.768 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 74.897120061999999, 2024-09-20T18:52:23.768 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.105453935, 2024-09-20T18:52:23.768 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:52:23.768 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:52:23.768 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:52:23.768 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:52:23.769 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:52:23.769 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:52:23.769 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 27562 2024-09-20T18:52:23.769 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.769 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:52:23.769 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.769 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:52:23.769 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.736852+0000", 2024-09-20T18:52:23.769 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:23.769 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.770 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.770 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:52:23.770 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.736852+0000", 2024-09-20T18:52:23.770 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:23.770 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.770 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.770 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:52:23.770 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.736854+0000", 2024-09-20T18:52:23.770 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9700000000000002e-06 2024-09-20T18:52:23.770 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.770 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.770 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:52:23.771 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.736905+0000", 2024-09-20T18:52:23.771 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.1069999999999997e-05 2024-09-20T18:52:23.771 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.771 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.771 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:52:23.771 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.736915+0000", 2024-09-20T18:52:23.771 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0624e-05 2024-09-20T18:52:23.771 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.771 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.771 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:52:23.771 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.736922+0000", 2024-09-20T18:52:23.772 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.635e-06 2024-09-20T18:52:23.772 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.772 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.772 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:52:23.772 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.736978+0000", 2024-09-20T18:52:23.772 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6138999999999999e-05 2024-09-20T18:52:23.772 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.772 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.772 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:52:23.772 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.737106+0000", 2024-09-20T18:52:23.772 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012784499999999999 2024-09-20T18:52:23.773 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.773 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.773 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,3", 2024-09-20T18:52:23.773 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.737235+0000", 2024-09-20T18:52:23.773 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012885699999999999 2024-09-20T18:52:23.773 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.773 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.773 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:52:23.773 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.738726+0000", 2024-09-20T18:52:23.773 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001491497 2024-09-20T18:52:23.773 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.773 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.774 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:23.774 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.739908+0000", 2024-09-20T18:52:23.774 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001181522 2024-09-20T18:52:23.774 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.774 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.774 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:23.774 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.841929+0000", 2024-09-20T18:52:23.774 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.102021444 2024-09-20T18:52:23.774 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.774 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.774 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:52:23.774 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.842232+0000", 2024-09-20T18:52:23.775 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00030261200000000001 2024-09-20T18:52:23.775 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.775 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.775 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:52:23.775 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.842305+0000", 2024-09-20T18:52:23.775 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.3720000000000006e-05 2024-09-20T18:52:23.775 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:23.775 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:52:23.775 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:23.775 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.776 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.776 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:27572 7.c 7:330a73dc:::benchmark_data_smithi083_70643_object27571:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:52:23.776 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:08.741631+0000", 2024-09-20T18:52:23.776 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 74.892341074000001, 2024-09-20T18:52:23.776 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100702955, 2024-09-20T18:52:23.776 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:52:23.776 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:52:23.776 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:52:23.776 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:52:23.776 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:52:23.776 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:52:23.776 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 27572 2024-09-20T18:52:23.777 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.777 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:52:23.777 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.777 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:52:23.777 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.741631+0000", 2024-09-20T18:52:23.777 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:23.777 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.777 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.777 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:52:23.777 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.741631+0000", 2024-09-20T18:52:23.777 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:23.777 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.778 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.778 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:52:23.778 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.741632+0000", 2024-09-20T18:52:23.778 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8950000000000001e-06 2024-09-20T18:52:23.778 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.778 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.778 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:52:23.778 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.741675+0000", 2024-09-20T18:52:23.778 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.2731999999999997e-05 2024-09-20T18:52:23.778 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.778 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.778 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:52:23.779 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.741686+0000", 2024-09-20T18:52:23.779 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0550000000000001e-05 2024-09-20T18:52:23.779 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.779 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.779 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:52:23.779 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.741692+0000", 2024-09-20T18:52:23.779 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8780000000000003e-06 2024-09-20T18:52:23.779 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.779 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.779 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:52:23.779 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.741744+0000", 2024-09-20T18:52:23.779 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.2457999999999998e-05 2024-09-20T18:52:23.779 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.780 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.780 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:52:23.780 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.741842+0000", 2024-09-20T18:52:23.780 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.8195000000000004e-05 2024-09-20T18:52:23.780 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.780 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.780 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:52:23.780 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.741934+0000", 2024-09-20T18:52:23.780 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.1986999999999999e-05 2024-09-20T18:52:23.780 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.780 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.780 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:52:23.781 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.742906+0000", 2024-09-20T18:52:23.781 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00097175899999999999 2024-09-20T18:52:23.781 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.781 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.781 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:23.781 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.743310+0000", 2024-09-20T18:52:23.781 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00040369899999999998 2024-09-20T18:52:23.781 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.781 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.781 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:23.781 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.842239+0000", 2024-09-20T18:52:23.781 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.098929589999999998 2024-09-20T18:52:23.782 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.782 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.782 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:52:23.782 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.842297+0000", 2024-09-20T18:52:23.782 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.7725e-05 2024-09-20T18:52:23.782 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.782 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.782 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:52:23.782 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.842333+0000", 2024-09-20T18:52:23.782 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6486999999999998e-05 2024-09-20T18:52:23.782 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:23.782 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:52:23.783 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:23.783 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.783 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.783 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:117106 7.7 7:e9541342:::benchmark_data_smithi083_70643_object117105:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-20T18:52:23.783 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:21.812375+0000", 2024-09-20T18:52:23.783 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 1.821597012, 2024-09-20T18:52:23.783 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.095326804000000001, 2024-09-20T18:52:23.783 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:52:23.783 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:52:23.783 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:52:23.783 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:52:23.784 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:52:23.784 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:52:23.784 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 117106 2024-09-20T18:52:23.784 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.784 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:52:23.784 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.784 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:52:23.784 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:21.812375+0000", 2024-09-20T18:52:23.784 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:23.784 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.784 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.785 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:52:23.785 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:21.812375+0000", 2024-09-20T18:52:23.785 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:23.785 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.785 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.785 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:52:23.785 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:21.812377+0000", 2024-09-20T18:52:23.785 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9939999999999999e-06 2024-09-20T18:52:23.785 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.785 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.785 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:52:23.786 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:21.812417+0000", 2024-09-20T18:52:23.786 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.083e-05 2024-09-20T18:52:23.786 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.786 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.786 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:52:23.786 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:21.812433+0000", 2024-09-20T18:52:23.786 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5753999999999999e-05 2024-09-20T18:52:23.786 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.786 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.786 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:52:23.786 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:21.812441+0000", 2024-09-20T18:52:23.787 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.4669999999999998e-06 2024-09-20T18:52:23.787 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.787 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.787 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:52:23.787 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:21.812493+0000", 2024-09-20T18:52:23.787 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.2172000000000001e-05 2024-09-20T18:52:23.787 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.787 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.787 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:52:23.787 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:21.812615+0000", 2024-09-20T18:52:23.787 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000122414 2024-09-20T18:52:23.787 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.788 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.788 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,3", 2024-09-20T18:52:23.788 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:21.812784+0000", 2024-09-20T18:52:23.788 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00016854199999999999 2024-09-20T18:52:23.788 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.788 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.788 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:52:23.788 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:21.818994+0000", 2024-09-20T18:52:23.788 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0062097569999999998 2024-09-20T18:52:23.788 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.788 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.789 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:23.789 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:21.900400+0000", 2024-09-20T18:52:23.789 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.081406016999999997 2024-09-20T18:52:23.789 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.789 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.789 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:23.789 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:21.907584+0000", 2024-09-20T18:52:23.789 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0071845880000000004 2024-09-20T18:52:23.789 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.789 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.789 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:52:23.789 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:21.907652+0000", 2024-09-20T18:52:23.790 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.7749000000000005e-05 2024-09-20T18:52:23.790 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.790 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.790 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:52:23.790 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:21.907701+0000", 2024-09-20T18:52:23.790 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.952e-05 2024-09-20T18:52:23.790 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:23.790 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:52:23.790 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:23.790 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.790 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.791 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:117108 7.7 7:e9087e0e:::benchmark_data_smithi083_70643_object117107:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-20T18:52:23.791 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:21.813325+0000", 2024-09-20T18:52:23.791 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 1.820646899, 2024-09-20T18:52:23.791 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100200289, 2024-09-20T18:52:23.791 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:52:23.791 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:52:23.791 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:52:23.791 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:52:23.791 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:52:23.791 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:52:23.791 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 117108 2024-09-20T18:52:23.792 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.792 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:52:23.792 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.792 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:52:23.792 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:21.813325+0000", 2024-09-20T18:52:23.792 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:23.792 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.792 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.792 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:52:23.792 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:21.813325+0000", 2024-09-20T18:52:23.792 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:23.793 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.793 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.793 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:52:23.793 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:21.813326+0000", 2024-09-20T18:52:23.793 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.375e-06 2024-09-20T18:52:23.793 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.793 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.793 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:52:23.793 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:21.813381+0000", 2024-09-20T18:52:23.793 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4929000000000002e-05 2024-09-20T18:52:23.793 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.793 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.793 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:52:23.794 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:21.813392+0000", 2024-09-20T18:52:23.794 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1102e-05 2024-09-20T18:52:23.794 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.794 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.794 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:52:23.794 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:21.813400+0000", 2024-09-20T18:52:23.794 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.9619999999999993e-06 2024-09-20T18:52:23.794 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.794 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.794 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:52:23.794 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:21.815544+0000", 2024-09-20T18:52:23.795 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0021435019999999998 2024-09-20T18:52:23.795 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.795 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.795 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:52:23.795 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:21.815696+0000", 2024-09-20T18:52:23.795 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015226700000000001 2024-09-20T18:52:23.795 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.795 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.795 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,3", 2024-09-20T18:52:23.795 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:21.815851+0000", 2024-09-20T18:52:23.795 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015548 2024-09-20T18:52:23.795 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.796 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.796 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:52:23.796 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:21.822408+0000", 2024-09-20T18:52:23.796 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0065566189999999996 2024-09-20T18:52:23.796 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.796 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.796 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:23.796 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:21.900443+0000", 2024-09-20T18:52:23.796 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.078034808999999997 2024-09-20T18:52:23.796 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.796 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.796 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:23.796 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:21.913398+0000", 2024-09-20T18:52:23.797 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.012955183 2024-09-20T18:52:23.797 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.797 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.797 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:52:23.797 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:21.913469+0000", 2024-09-20T18:52:23.797 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.1198000000000005e-05 2024-09-20T18:52:23.797 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.797 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.797 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:52:23.797 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:21.913525+0000", 2024-09-20T18:52:23.797 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5862999999999997e-05 2024-09-20T18:52:23.797 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:23.798 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:52:23.798 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:23.798 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.798 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.798 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:117111 7.7 7:ed65c94a:::benchmark_data_smithi083_70643_object117110:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-20T18:52:23.798 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:21.813504+0000", 2024-09-20T18:52:23.798 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 1.820467152, 2024-09-20T18:52:23.798 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100161164, 2024-09-20T18:52:23.798 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:52:23.798 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:52:23.798 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:52:23.798 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:52:23.799 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:52:23.799 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:52:23.799 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 117111 2024-09-20T18:52:23.799 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.799 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:52:23.799 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.799 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:52:23.799 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:21.813504+0000", 2024-09-20T18:52:23.799 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:23.799 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.799 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.799 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:52:23.800 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:21.813504+0000", 2024-09-20T18:52:23.800 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:23.800 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.800 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.800 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:52:23.800 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:21.813505+0000", 2024-09-20T18:52:23.800 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.9700000000000005e-07 2024-09-20T18:52:23.800 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.800 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.800 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:52:23.800 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:21.813565+0000", 2024-09-20T18:52:23.800 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9806999999999998e-05 2024-09-20T18:52:23.800 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.801 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.801 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:52:23.801 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:21.813575+0000", 2024-09-20T18:52:23.801 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.7259999999999997e-06 2024-09-20T18:52:23.801 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.801 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.801 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:52:23.801 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:21.813582+0000", 2024-09-20T18:52:23.801 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.1239999999999999e-06 2024-09-20T18:52:23.801 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.801 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.801 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:52:23.802 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:21.819031+0000", 2024-09-20T18:52:23.802 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0054486630000000003 2024-09-20T18:52:23.802 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.802 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.802 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:52:23.802 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:21.819167+0000", 2024-09-20T18:52:23.802 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000136394 2024-09-20T18:52:23.802 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.802 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.802 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,3", 2024-09-20T18:52:23.802 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:21.819326+0000", 2024-09-20T18:52:23.803 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015905199999999999 2024-09-20T18:52:23.803 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.803 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.803 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:52:23.803 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:21.822508+0000", 2024-09-20T18:52:23.803 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0031819499999999998 2024-09-20T18:52:23.803 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.803 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.803 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:23.803 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:21.901789+0000", 2024-09-20T18:52:23.803 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.079281237000000004 2024-09-20T18:52:23.803 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.803 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.804 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:23.804 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:21.913569+0000", 2024-09-20T18:52:23.804 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.011780017 2024-09-20T18:52:23.804 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.804 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.804 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:52:23.804 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:21.913628+0000", 2024-09-20T18:52:23.804 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9166000000000002e-05 2024-09-20T18:52:23.804 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:23.804 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:23.804 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:52:23.804 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:21.913666+0000", 2024-09-20T18:52:23.805 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.7131000000000001e-05 2024-09-20T18:52:23.805 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:23.805 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:52:23.805 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:23.805 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:23.805 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:52:23.805 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T18:52:23.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:23.807 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:23.807 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:23.909 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:52:23.935 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:23.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:23.936 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:52:23.969 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:23 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:52:23.969 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:23 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:52:23.969 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:23 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:52:23.969 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:23 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:52:23.969 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:23 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:52:23.969 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:23 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:52:24.017 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-20T18:52:24.018 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-20T18:52:24.018 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph tell osd.4 'trim stale osdmaps' 2024-09-20T18:52:24.037 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:52:24.064 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:24.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:24.065 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:52:24.165 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:24.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:24.223 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:24.223 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:24.324 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:24.372 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:24.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:24.373 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:24.474 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:24.479 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:52:24.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:24.500 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:24.500 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:24.541 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 87 16 119755 119739 85.9915 62.125 0.015292 0.0116242 2024-09-20T18:52:24.601 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:24.651 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:24.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:24.652 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:24.655 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-20T18:52:24.661 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T18:52:24.686 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-20T18:52:24.753 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:24.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:24.806 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:24.806 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:24.907 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:24.940 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:24.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:24.941 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:52:25.021 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:52:25.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:24 smithi142 ceph-mon[32508]: pgmap v113: 148 pgs: 148 active+clean; 7.1 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.4 KiB/s rd, 79 MiB/s wr, 1.26k op/s; 15 B/s, 0 objects/s recovering 2024-09-20T18:52:25.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:24 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2730605530' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:52:25.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:24 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:52:25.042 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:25.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:24 smithi083 ceph-mon[29421]: pgmap v113: 148 pgs: 148 active+clean; 7.1 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.4 KiB/s rd, 79 MiB/s wr, 1.26k op/s; 15 B/s, 0 objects/s recovering 2024-09-20T18:52:25.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:24 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2730605530' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:52:25.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:24 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:52:25.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:24 smithi083 ceph-mon[36260]: pgmap v113: 148 pgs: 148 active+clean; 7.1 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.4 KiB/s rd, 79 MiB/s wr, 1.26k op/s; 15 B/s, 0 objects/s recovering 2024-09-20T18:52:25.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:24 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2730605530' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:52:25.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:24 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:52:25.064 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:25.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:25.065 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:25.166 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:25.195 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:25.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:25.196 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:25.297 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:25.336 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:25.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:25.337 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:25.438 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:25.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:25.467 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:25.467 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:25.541 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 88 16 120725 120709 85.7032 60.625 0.0186287 0.0116636 2024-09-20T18:52:25.568 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:25.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:25.596 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:25.596 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:25.697 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:52:25.726 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:25.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:25.727 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:52:25.828 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:25.850 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:25.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:25.851 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:25.952 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:25.984 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:25.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:25.985 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:52:26.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:25 smithi142 ceph-mon[32508]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:52:26.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:25 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T18:52:26.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:25 smithi142 ceph-mon[32508]: osdmap e121: 8 total, 8 up, 8 in 2024-09-20T18:52:26.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:25 smithi142 ceph-mon[32508]: pgmap v115: 148 pgs: 148 active+clean; 7.1 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 768 B/s rd, 43 MiB/s wr, 683 op/s 2024-09-20T18:52:26.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:25 smithi142 ceph-mon[32508]: osdmap e122: 8 total, 8 up, 8 in 2024-09-20T18:52:26.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:25 smithi083 ceph-mon[29421]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:52:26.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:25 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T18:52:26.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:25 smithi083 ceph-mon[29421]: osdmap e121: 8 total, 8 up, 8 in 2024-09-20T18:52:26.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:25 smithi083 ceph-mon[29421]: pgmap v115: 148 pgs: 148 active+clean; 7.1 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 768 B/s rd, 43 MiB/s wr, 683 op/s 2024-09-20T18:52:26.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:25 smithi083 ceph-mon[29421]: osdmap e122: 8 total, 8 up, 8 in 2024-09-20T18:52:26.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:25 smithi083 ceph-mon[36260]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:52:26.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:25 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T18:52:26.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:25 smithi083 ceph-mon[36260]: osdmap e121: 8 total, 8 up, 8 in 2024-09-20T18:52:26.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:25 smithi083 ceph-mon[36260]: pgmap v115: 148 pgs: 148 active+clean; 7.1 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 768 B/s rd, 43 MiB/s wr, 683 op/s 2024-09-20T18:52:26.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:25 smithi083 ceph-mon[36260]: osdmap e122: 8 total, 8 up, 8 in 2024-09-20T18:52:26.086 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:26.117 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:26.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:26.118 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:26.219 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:26.245 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:26.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:26.246 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:26.347 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:52:26.375 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:26.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:26.376 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:52:26.477 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:26.522 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:26.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:26.523 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:26.541 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 89 16 121754 121738 85.4629 64.3125 0.0148825 0.0116962 2024-09-20T18:52:26.566 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T18:52:26.566 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-09-20T18:52:26.566 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:26.566 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:121764 7.5 7:abac3d1c:::benchmark_data_smithi083_70643_object121763:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-20T18:52:26.566 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:26.531111+0000", 2024-09-20T18:52:26.566 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.016309857000000001, 2024-09-20T18:52:26.566 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.016341844000000001, 2024-09-20T18:52:26.566 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:52:26.566 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:52:26.566 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "waiting for sub ops", 2024-09-20T18:52:26.566 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:52:26.567 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:52:26.567 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:52:26.567 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 121764 2024-09-20T18:52:26.567 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:26.567 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:52:26.567 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:26.567 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:52:26.567 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:26.531111+0000", 2024-09-20T18:52:26.567 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:26.567 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:26.567 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:26.567 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:52:26.567 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:26.531111+0000", 2024-09-20T18:52:26.567 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:26.567 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:26.568 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:26.568 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:52:26.568 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:26.531113+0000", 2024-09-20T18:52:26.568 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.646e-06 2024-09-20T18:52:26.568 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:26.568 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:26.568 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:52:26.568 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:26.531282+0000", 2024-09-20T18:52:26.568 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000169395 2024-09-20T18:52:26.568 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:26.568 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:26.568 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:52:26.568 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:26.531299+0000", 2024-09-20T18:52:26.568 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6651999999999999e-05 2024-09-20T18:52:26.569 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:26.569 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:26.569 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:52:26.569 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:26.531307+0000", 2024-09-20T18:52:26.569 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.9240000000000007e-06 2024-09-20T18:52:26.569 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:26.569 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:26.569 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:52:26.569 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:26.531362+0000", 2024-09-20T18:52:26.569 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5225000000000001e-05 2024-09-20T18:52:26.569 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:26.569 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:26.569 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:52:26.569 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:26.531485+0000", 2024-09-20T18:52:26.569 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000122991 2024-09-20T18:52:26.570 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:26.570 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:26.570 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,2", 2024-09-20T18:52:26.570 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:26.531616+0000", 2024-09-20T18:52:26.570 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013128900000000001 2024-09-20T18:52:26.570 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:26.570 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:26.570 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:52:26.570 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:26.535221+0000", 2024-09-20T18:52:26.570 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0036046120000000001 2024-09-20T18:52:26.570 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:26.570 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:26.570 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:26.570 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:26.540047+0000", 2024-09-20T18:52:26.571 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0048267559999999998 2024-09-20T18:52:26.571 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:26.571 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:52:26.571 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:26.571 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:26.571 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:26.571 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:121769 7.5 7:a673974b:::benchmark_data_smithi083_70643_object121768:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-20T18:52:26.571 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:26.535306+0000", 2024-09-20T18:52:26.571 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.01211506, 2024-09-20T18:52:26.571 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.012229817, 2024-09-20T18:52:26.571 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:52:26.571 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:52:26.571 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "waiting for sub ops", 2024-09-20T18:52:26.571 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:52:26.571 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:52:26.572 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:52:26.572 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 121769 2024-09-20T18:52:26.572 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:26.572 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:52:26.572 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:26.572 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:52:26.572 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:26.535306+0000", 2024-09-20T18:52:26.572 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:26.572 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:26.572 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:26.572 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:52:26.572 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:26.535306+0000", 2024-09-20T18:52:26.572 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:26.572 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:26.573 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:26.573 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:52:26.573 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:26.535307+0000", 2024-09-20T18:52:26.573 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.536e-06 2024-09-20T18:52:26.573 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:26.573 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:26.573 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:52:26.573 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:26.535361+0000", 2024-09-20T18:52:26.573 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3913999999999999e-05 2024-09-20T18:52:26.573 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:26.573 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:26.573 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:52:26.573 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:26.535375+0000", 2024-09-20T18:52:26.573 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4072e-05 2024-09-20T18:52:26.574 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:26.574 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:26.574 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:52:26.574 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:26.535384+0000", 2024-09-20T18:52:26.574 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.6580000000000008e-06 2024-09-20T18:52:26.574 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:26.574 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:26.574 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:52:26.574 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:26.535456+0000", 2024-09-20T18:52:26.574 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.2127999999999998e-05 2024-09-20T18:52:26.574 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:26.574 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:26.574 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:52:26.574 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:26.535573+0000", 2024-09-20T18:52:26.575 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000117373 2024-09-20T18:52:26.575 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:26.575 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:26.575 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,2", 2024-09-20T18:52:26.575 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:26.535713+0000", 2024-09-20T18:52:26.575 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013907399999999999 2024-09-20T18:52:26.575 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:26.575 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:26.575 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:52:26.575 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:26.538691+0000", 2024-09-20T18:52:26.575 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0029788520000000001 2024-09-20T18:52:26.576 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:26.576 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:26.576 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:52:26.576 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:26.540101+0000", 2024-09-20T18:52:26.576 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001409506 2024-09-20T18:52:26.576 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:26.576 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:52:26.576 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:26.576 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:26.576 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:26.576 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.14808.0:121778 7.f e122/83 7:f6a9a77b:::benchmark_data_smithi083_70643_object121777:head v 122'7681, mlcod=122'7679)", 2024-09-20T18:52:26.576 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:26.546410+0000", 2024-09-20T18:52:26.576 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.0010111409999999999, 2024-09-20T18:52:26.577 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0011910779999999999, 2024-09-20T18:52:26.577 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:52:26.577 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:52:26.577 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-09-20T18:52:26.577 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:52:26.577 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:26.577 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:52:26.577 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:26.546410+0000", 2024-09-20T18:52:26.577 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:26.577 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:26.577 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:26.577 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:52:26.577 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:26.546410+0000", 2024-09-20T18:52:26.577 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:26.578 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:26.578 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:26.578 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:52:26.578 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:26.546411+0000", 2024-09-20T18:52:26.578 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.353e-06 2024-09-20T18:52:26.578 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:26.578 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:26.578 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:52:26.578 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:26.546480+0000", 2024-09-20T18:52:26.578 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.9068999999999999e-05 2024-09-20T18:52:26.578 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:26.578 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:26.578 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:52:26.578 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:26.546494+0000", 2024-09-20T18:52:26.579 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4096e-05 2024-09-20T18:52:26.579 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:26.579 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:26.579 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:52:26.579 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:26.546502+0000", 2024-09-20T18:52:26.579 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.4240000000000004e-06 2024-09-20T18:52:26.579 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:26.579 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:26.579 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:52:26.579 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:26.546578+0000", 2024-09-20T18:52:26.579 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.6365000000000005e-05 2024-09-20T18:52:26.579 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:26.579 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:26.579 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:52:26.580 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:26.546610+0000", 2024-09-20T18:52:26.580 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.1538999999999997e-05 2024-09-20T18:52:26.580 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:26.580 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:52:26.580 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:26.580 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:26.580 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:26.580 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop_reply(client.14808.0:121769 7.5 e122/83)", 2024-09-20T18:52:26.580 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:26.547444+0000", 2024-09-20T18:52:26.580 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 4294967295.9999766, 2024-09-20T18:52:26.580 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00020050800000000001, 2024-09-20T18:52:26.580 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:52:26.580 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:52:26.580 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "queued for pg", 2024-09-20T18:52:26.581 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:52:26.581 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:26.581 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:52:26.581 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:26.547444+0000", 2024-09-20T18:52:26.581 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:26.581 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:26.581 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:26.581 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:52:26.581 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:26.547444+0000", 2024-09-20T18:52:26.581 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:52:26.581 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:26.581 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:26.581 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:52:26.581 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:26.547445+0000", 2024-09-20T18:52:26.582 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.7399999999999999e-07 2024-09-20T18:52:26.582 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:26.582 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:26.582 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:52:26.582 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:26.547447+0000", 2024-09-20T18:52:26.582 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2900000000000001e-06 2024-09-20T18:52:26.582 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:26.582 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:26.582 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:52:26.582 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:26.547452+0000", 2024-09-20T18:52:26.582 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.865e-06 2024-09-20T18:52:26.582 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:52:26.582 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:52:26.582 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:52:26.582 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:26.547456+0000", 2024-09-20T18:52:26.583 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9910000000000004e-06 2024-09-20T18:52:26.583 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:26.583 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:52:26.583 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:26.583 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:52:26.583 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-09-20T18:52:26.583 INFO:teuthology.orchestra.run.smithi142.stdout: "num_ops": 4 2024-09-20T18:52:26.583 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T18:52:26.624 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:26.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:26.663 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:26.664 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:26.764 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:52:26.774 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:52:26 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:52:26.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:26 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:52:26.800 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:26.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:26.800 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:52:26.901 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:26.941 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:26.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:26.942 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:27.014 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:26 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:52:27.015 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:26 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:52:27.043 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:27.068 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:27.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:27.069 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:27.170 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:27.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:27.217 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:27.217 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:27.264 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_blocked_ops 2024-09-20T18:52:27.301 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:52:27.318 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:27.327 INFO:teuthology.orchestra.run.smithi083.stderr:no valid command found; 10 closest matches: 2024-09-20T18:52:27.327 INFO:teuthology.orchestra.run.smithi083.stderr:0 2024-09-20T18:52:27.327 INFO:teuthology.orchestra.run.smithi083.stderr:1 2024-09-20T18:52:27.327 INFO:teuthology.orchestra.run.smithi083.stderr:2 2024-09-20T18:52:27.327 INFO:teuthology.orchestra.run.smithi083.stderr:abort 2024-09-20T18:52:27.327 INFO:teuthology.orchestra.run.smithi083.stderr:assert 2024-09-20T18:52:27.328 INFO:teuthology.orchestra.run.smithi083.stderr:bench [] [] [] [] 2024-09-20T18:52:27.328 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs debug_inject_read_zeros 2024-09-20T18:52:27.328 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs files list 2024-09-20T18:52:27.328 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs stats 2024-09-20T18:52:27.328 INFO:teuthology.orchestra.run.smithi083.stderr:bluestore allocator dump block 2024-09-20T18:52:27.328 INFO:teuthology.orchestra.run.smithi083.stderr:Error EINVAL: invalid command 2024-09-20T18:52:27.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:27.386 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:27.386 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:27.487 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:27.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:27.526 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:27.526 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:27.541 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Total time run: 90.0108 2024-09-20T18:52:27.541 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Total writes made: 122799 2024-09-20T18:52:27.542 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Write size: 65536 2024-09-20T18:52:27.542 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Object size: 65536 2024-09-20T18:52:27.542 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Bandwidth (MB/sec): 85.2668 2024-09-20T18:52:27.542 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Stddev Bandwidth: 33.4038 2024-09-20T18:52:27.542 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Max bandwidth (MB/sec): 216.625 2024-09-20T18:52:27.542 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Min bandwidth (MB/sec): 43.5 2024-09-20T18:52:27.542 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Average IOPS: 1364 2024-09-20T18:52:27.542 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Stddev IOPS: 534.461 2024-09-20T18:52:27.542 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Max IOPS: 3466 2024-09-20T18:52:27.542 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Min IOPS: 696 2024-09-20T18:52:27.542 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Average Latency(s): 0.0117238 2024-09-20T18:52:27.542 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Stddev Latency(s): 0.00898345 2024-09-20T18:52:27.542 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Max latency(s): 0.186958 2024-09-20T18:52:27.542 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Min latency(s): 0.00137732 2024-09-20T18:52:27.546 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Cleaning up (deleting benchmark objects) 2024-09-20T18:52:27.601 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:52:27.626 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:27.670 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:27.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:27.671 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:27.771 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:27.802 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:27.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:27.803 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:27.904 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:27.924 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:27 smithi083 ceph-mon[29421]: pgmap v117: 148 pgs: 148 active+clean; 7.4 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1.6 KiB/s rd, 96 MiB/s wr, 1.53k op/s 2024-09-20T18:52:27.924 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:27 smithi083 ceph-mon[36260]: pgmap v117: 148 pgs: 148 active+clean; 7.4 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1.6 KiB/s rd, 96 MiB/s wr, 1.53k op/s 2024-09-20T18:52:27.946 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:27.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:27.947 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:28.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:27 smithi142 ceph-mon[32508]: pgmap v117: 148 pgs: 148 active+clean; 7.4 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1.6 KiB/s rd, 96 MiB/s wr, 1.53k op/s 2024-09-20T18:52:28.047 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:28.081 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:28.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:28.082 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:28.114 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:52:28.114 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:52:28.151 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:52:28.152 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:52:28.162 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T18:52:28.162 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-09-20T18:52:28.162 INFO:teuthology.orchestra.run.smithi142.stdout: "complaint_time": 30, 2024-09-20T18:52:28.162 INFO:teuthology.orchestra.run.smithi142.stdout: "num_blocked_ops": 0 2024-09-20T18:52:28.162 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T18:52:28.184 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:28.189 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:52:28.190 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:52:28.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:28.217 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:28.218 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:28.223 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:52:28.224 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:52:28.254 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:52:28.255 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:52:28.285 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:52:28.286 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:52:28.316 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:52:28.317 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:52:28.318 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:28.355 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:52:28.355 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:52:28.358 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:28.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:28.359 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:28.460 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:28.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:28.502 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:28.502 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:28.603 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:28.643 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:28.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:28.644 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:28.700 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-20T18:52:28.701 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-20T18:52:28.701 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph tell osd.5 'trim stale osdmaps' 2024-09-20T18:52:28.745 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:28.772 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:28.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:28.773 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:28.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:28 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:52:28.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:28 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:52:28.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:28 smithi142 ceph-mon[32508]: osdmap e123: 8 total, 8 up, 8 in 2024-09-20T18:52:28.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:28 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1209463748' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:52:28.815 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_historic_ops 2024-09-20T18:52:28.873 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:52:28.925 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:28.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:28.926 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:52:29.027 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:29.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:28 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:52:29.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:28 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:52:29.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:28 smithi083 ceph-mon[29421]: osdmap e123: 8 total, 8 up, 8 in 2024-09-20T18:52:29.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:28 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1209463748' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:52:29.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:28 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:52:29.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:28 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:52:29.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:28 smithi083 ceph-mon[36260]: osdmap e123: 8 total, 8 up, 8 in 2024-09-20T18:52:29.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:28 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1209463748' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:52:29.059 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:29.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:29.060 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:52:29.067 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T18:52:29.160 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:29.191 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:29.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:29.192 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:29.200 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:52:29.293 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:29.322 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:29.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:29.323 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:29.423 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:29.450 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:29.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:29.451 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:29.552 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:29.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:29.597 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:29.598 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:29.618 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:52:29.618 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:52:29.699 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:29.755 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:29.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:29.756 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:29.770 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T18:52:29.770 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:52:29.857 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:29.901 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:29.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:29.902 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:30.003 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:30.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:29 smithi142 ceph-mon[32508]: pgmap v119: 148 pgs: 148 active+clean; 7.4 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 853 B/s rd, 51 MiB/s wr, 821 op/s 2024-09-20T18:52:30.030 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:30.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:30.031 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:30.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:29 smithi083 ceph-mon[29421]: pgmap v119: 148 pgs: 148 active+clean; 7.4 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 853 B/s rd, 51 MiB/s wr, 821 op/s 2024-09-20T18:52:30.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:29 smithi083 ceph-mon[36260]: pgmap v119: 148 pgs: 148 active+clean; 7.4 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 853 B/s rd, 51 MiB/s wr, 821 op/s 2024-09-20T18:52:30.132 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:30.165 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:30.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:30.166 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:30.267 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:30.291 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:30.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:30.292 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:30.392 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:30.417 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:30.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:30.418 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:30.518 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-20T18:52:30.519 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:52:30.544 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:30.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:30.545 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:52:30.646 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:30.667 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:30.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:30.668 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:30.769 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:30.816 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:30.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:30.817 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:30.918 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:30.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:30.960 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:30.960 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:31.054 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:52:30 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:52:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:52:31.061 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:31.090 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:31.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:31.092 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:52:31.192 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:31.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:31.223 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:31.223 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:31.301 INFO:teuthology.orchestra.run.smithi083.stderr:no valid command found; 10 closest matches: 2024-09-20T18:52:31.301 INFO:teuthology.orchestra.run.smithi083.stderr:0 2024-09-20T18:52:31.301 INFO:teuthology.orchestra.run.smithi083.stderr:1 2024-09-20T18:52:31.301 INFO:teuthology.orchestra.run.smithi083.stderr:2 2024-09-20T18:52:31.301 INFO:teuthology.orchestra.run.smithi083.stderr:abort 2024-09-20T18:52:31.301 INFO:teuthology.orchestra.run.smithi083.stderr:assert 2024-09-20T18:52:31.301 INFO:teuthology.orchestra.run.smithi083.stderr:bench [] [] [] [] 2024-09-20T18:52:31.301 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs debug_inject_read_zeros 2024-09-20T18:52:31.301 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs files list 2024-09-20T18:52:31.301 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs stats 2024-09-20T18:52:31.301 INFO:teuthology.orchestra.run.smithi083.stderr:bluestore allocator dump block 2024-09-20T18:52:31.302 INFO:teuthology.orchestra.run.smithi083.stderr:Error EINVAL: invalid command 2024-09-20T18:52:31.324 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:31.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:31.365 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:31.365 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:31.466 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:31.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:31.488 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:31.488 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:52:31.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:31 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:52:31.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:31 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:52:31.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:31 smithi142 ceph-mon[32508]: osdmap e124: 8 total, 8 up, 8 in 2024-09-20T18:52:31.538 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T18:52:31.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:31 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:52:31.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:31 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:52:31.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:31 smithi083 ceph-mon[29421]: osdmap e124: 8 total, 8 up, 8 in 2024-09-20T18:52:31.557 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:31 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:52:31.557 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:31 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:52:31.557 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:31 smithi083 ceph-mon[36260]: osdmap e124: 8 total, 8 up, 8 in 2024-09-20T18:52:31.589 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:31.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:31.615 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:31.615 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:31.716 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:31.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:31.738 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:31.739 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:31.840 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:31.885 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:31.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:31.886 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:31.987 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:32.015 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:32.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:32.016 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:52:32.117 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:32.143 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:32.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:32.144 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:52:32.245 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:52:32.275 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:32.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:32.276 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:52:32.377 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:32.403 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:32.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:32.404 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:32.504 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:52:32.530 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:32.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:32.531 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:52:32.631 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:32.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:32.671 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:32.671 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:32.772 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:32.798 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:32.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:32.799 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:32.900 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:32.927 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:32.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:32.928 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:33.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:32 smithi142 ceph-mon[32508]: pgmap v121: 148 pgs: 148 active+clean; 7.4 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1.7 KiB/s rd, 55 MiB/s wr, 996 op/s 2024-09-20T18:52:33.028 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:33.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:32 smithi083 ceph-mon[29421]: pgmap v121: 148 pgs: 148 active+clean; 7.4 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1.7 KiB/s rd, 55 MiB/s wr, 996 op/s 2024-09-20T18:52:33.057 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:32 smithi083 ceph-mon[36260]: pgmap v121: 148 pgs: 148 active+clean; 7.4 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1.7 KiB/s rd, 55 MiB/s wr, 996 op/s 2024-09-20T18:52:33.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:33.069 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:33.069 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:33.170 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:33.196 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:33.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:33.197 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:33.298 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:33.338 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:33.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:33.339 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:33.440 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:33.478 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:33.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:33.479 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:33.523 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-20T18:52:33.523 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-20T18:52:33.524 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph tell osd.6 'trim stale osdmaps' 2024-09-20T18:52:33.579 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:33.604 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:33.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:33.605 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:33.707 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:33.752 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:33.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:33.753 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:33.854 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:33.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:33.902 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:33.902 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:34.003 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:34.026 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:34.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:34.027 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:52:34.038 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:52:34.045 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:52:34.128 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:34.184 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:34.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:34.185 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:34.286 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:34.312 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:34.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:34.313 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:34.358 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:52:34.413 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:34.454 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:34.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:34.455 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:34.556 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:52:34.581 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:34.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:34.582 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:52:34.646 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-20T18:52:34.683 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:34.721 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:34.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:34.722 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:34.786 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:34 smithi083 ceph-mon[29421]: pgmap v122: 148 pgs: 148 active+clean; 7.3 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1.5 KiB/s rd, 58 MiB/s wr, 1.43k op/s 2024-09-20T18:52:34.787 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:34 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/352888404' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:52:34.787 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:34 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:52:34.787 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:34 smithi083 ceph-mon[36260]: pgmap v122: 148 pgs: 148 active+clean; 7.3 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1.5 KiB/s rd, 58 MiB/s wr, 1.43k op/s 2024-09-20T18:52:34.787 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:34 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/352888404' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:52:34.787 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:34 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:52:34.823 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:34.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:34.869 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:34.869 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:34.948 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T18:52:34.948 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:52:34.969 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:34.996 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:34.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:34.997 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:52:35.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:34 smithi142 ceph-mon[32508]: pgmap v122: 148 pgs: 148 active+clean; 7.3 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1.5 KiB/s rd, 58 MiB/s wr, 1.43k op/s 2024-09-20T18:52:35.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:34 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/352888404' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:52:35.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:34 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:52:35.098 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:35.133 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:35.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:35.134 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:35.235 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:35.261 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:35.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:35.262 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:52:35.363 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:35.400 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:35.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:35.401 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:35.502 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:35.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:35.531 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:35.531 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:35.632 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:35.668 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:35.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:35.669 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:35.770 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:35.797 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:35.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:35.798 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:52:35.895 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:35 smithi083 ceph-mon[29421]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:52:35.895 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:35 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T18:52:35.895 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:35 smithi083 ceph-mon[29421]: osdmap e125: 8 total, 8 up, 8 in 2024-09-20T18:52:35.895 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:35 smithi083 ceph-mon[29421]: 4.f scrub starts 2024-09-20T18:52:35.895 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:35 smithi083 ceph-mon[29421]: 4.f scrub ok 2024-09-20T18:52:35.895 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:35 smithi083 ceph-mon[29421]: 4.1 scrub starts 2024-09-20T18:52:35.895 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:35 smithi083 ceph-mon[29421]: 4.1 scrub ok 2024-09-20T18:52:35.895 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:35 smithi083 ceph-mon[29421]: 4.8 scrub starts 2024-09-20T18:52:35.895 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:35 smithi083 ceph-mon[29421]: 4.8 scrub ok 2024-09-20T18:52:35.896 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:35 smithi083 ceph-mon[29421]: osdmap e126: 8 total, 8 up, 8 in 2024-09-20T18:52:35.896 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:35 smithi083 ceph-mon[36260]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:52:35.896 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:35 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T18:52:35.896 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:35 smithi083 ceph-mon[36260]: osdmap e125: 8 total, 8 up, 8 in 2024-09-20T18:52:35.896 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:35 smithi083 ceph-mon[36260]: 4.f scrub starts 2024-09-20T18:52:35.896 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:35 smithi083 ceph-mon[36260]: 4.f scrub ok 2024-09-20T18:52:35.896 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:35 smithi083 ceph-mon[36260]: 4.1 scrub starts 2024-09-20T18:52:35.896 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:35 smithi083 ceph-mon[36260]: 4.1 scrub ok 2024-09-20T18:52:35.896 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:35 smithi083 ceph-mon[36260]: 4.8 scrub starts 2024-09-20T18:52:35.896 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:35 smithi083 ceph-mon[36260]: 4.8 scrub ok 2024-09-20T18:52:35.896 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:35 smithi083 ceph-mon[36260]: osdmap e126: 8 total, 8 up, 8 in 2024-09-20T18:52:35.899 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:35.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:35.929 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:35.930 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:36.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:35 smithi142 ceph-mon[32508]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:52:36.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:35 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T18:52:36.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:35 smithi142 ceph-mon[32508]: osdmap e125: 8 total, 8 up, 8 in 2024-09-20T18:52:36.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:35 smithi142 ceph-mon[32508]: 4.f scrub starts 2024-09-20T18:52:36.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:35 smithi142 ceph-mon[32508]: 4.f scrub ok 2024-09-20T18:52:36.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:35 smithi142 ceph-mon[32508]: 4.1 scrub starts 2024-09-20T18:52:36.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:35 smithi142 ceph-mon[32508]: 4.1 scrub ok 2024-09-20T18:52:36.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:35 smithi142 ceph-mon[32508]: 4.8 scrub starts 2024-09-20T18:52:36.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:35 smithi142 ceph-mon[32508]: 4.8 scrub ok 2024-09-20T18:52:36.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:35 smithi142 ceph-mon[32508]: osdmap e126: 8 total, 8 up, 8 in 2024-09-20T18:52:36.030 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:52:36.056 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:36.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:36.057 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:52:36.158 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:36.183 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:36.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:36.184 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:52:36.286 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:36.306 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:52:36.307 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:52:36.322 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:36.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:36.323 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:36.344 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:52:36.344 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:52:36.399 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:52:36.400 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:52:36.424 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:36.449 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:36.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:36.450 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:36.450 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:52:36.450 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:52:36.492 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:52:36.492 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:52:36.537 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:52:36.537 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:52:36.550 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:36.560 INFO:teuthology.orchestra.run.smithi083.stderr:no valid command found; 10 closest matches: 2024-09-20T18:52:36.560 INFO:teuthology.orchestra.run.smithi083.stderr:0 2024-09-20T18:52:36.560 INFO:teuthology.orchestra.run.smithi083.stderr:1 2024-09-20T18:52:36.560 INFO:teuthology.orchestra.run.smithi083.stderr:2 2024-09-20T18:52:36.560 INFO:teuthology.orchestra.run.smithi083.stderr:abort 2024-09-20T18:52:36.560 INFO:teuthology.orchestra.run.smithi083.stderr:assert 2024-09-20T18:52:36.560 INFO:teuthology.orchestra.run.smithi083.stderr:bench [] [] [] [] 2024-09-20T18:52:36.560 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs debug_inject_read_zeros 2024-09-20T18:52:36.560 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs files list 2024-09-20T18:52:36.560 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs stats 2024-09-20T18:52:36.560 INFO:teuthology.orchestra.run.smithi083.stderr:bluestore allocator dump block 2024-09-20T18:52:36.561 INFO:teuthology.orchestra.run.smithi083.stderr:Error EINVAL: invalid command 2024-09-20T18:52:36.579 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:52:36.580 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:52:36.588 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:36.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:36.589 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:36.612 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:52:36.613 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:52:36.690 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:36.729 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:36.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:36.730 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:36.774 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:52:36 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:52:36.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:36 smithi142 ceph-mon[32508]: pgmap v124: 148 pgs: 148 active+clean; 7.3 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 863 B/s rd, 15 MiB/s wr, 750 op/s 2024-09-20T18:52:36.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:36 smithi142 ceph-mon[32508]: 3.15 scrub starts 2024-09-20T18:52:36.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:36 smithi142 ceph-mon[32508]: 3.15 scrub ok 2024-09-20T18:52:36.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:36 smithi142 ceph-mon[32508]: 6.f scrub starts 2024-09-20T18:52:36.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:36 smithi142 ceph-mon[32508]: 6.f scrub ok 2024-09-20T18:52:36.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:36 smithi142 ceph-mon[32508]: 5.1b scrub starts 2024-09-20T18:52:36.775 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:36 smithi142 ceph-mon[32508]: 5.1b scrub ok 2024-09-20T18:52:36.775 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:36 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3182866403' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:52:36.806 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:36 smithi083 ceph-mon[29421]: pgmap v124: 148 pgs: 148 active+clean; 7.3 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 863 B/s rd, 15 MiB/s wr, 750 op/s 2024-09-20T18:52:36.806 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:36 smithi083 ceph-mon[29421]: 3.15 scrub starts 2024-09-20T18:52:36.806 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:36 smithi083 ceph-mon[29421]: 3.15 scrub ok 2024-09-20T18:52:36.806 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:36 smithi083 ceph-mon[29421]: 6.f scrub starts 2024-09-20T18:52:36.806 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:36 smithi083 ceph-mon[29421]: 6.f scrub ok 2024-09-20T18:52:36.806 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:36 smithi083 ceph-mon[29421]: 5.1b scrub starts 2024-09-20T18:52:36.806 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:36 smithi083 ceph-mon[29421]: 5.1b scrub ok 2024-09-20T18:52:36.806 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:36 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3182866403' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:52:36.807 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:36 smithi083 ceph-mon[36260]: pgmap v124: 148 pgs: 148 active+clean; 7.3 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 863 B/s rd, 15 MiB/s wr, 750 op/s 2024-09-20T18:52:36.807 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:36 smithi083 ceph-mon[36260]: 3.15 scrub starts 2024-09-20T18:52:36.807 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:36 smithi083 ceph-mon[36260]: 3.15 scrub ok 2024-09-20T18:52:36.807 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:36 smithi083 ceph-mon[36260]: 6.f scrub starts 2024-09-20T18:52:36.807 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:36 smithi083 ceph-mon[36260]: 6.f scrub ok 2024-09-20T18:52:36.807 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:36 smithi083 ceph-mon[36260]: 5.1b scrub starts 2024-09-20T18:52:36.807 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:36 smithi083 ceph-mon[36260]: 5.1b scrub ok 2024-09-20T18:52:36.807 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:36 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3182866403' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:52:36.832 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:36.869 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:36.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:36.870 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:36.971 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:37.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:37.010 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:37.011 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:37.111 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:37.136 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:37.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:37.137 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:37.237 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:37.240 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_blocked_ops 2024-09-20T18:52:37.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:37.316 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:37.317 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:37.417 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:37.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:37.467 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:37.467 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:37.568 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:37.615 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:37.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:37.616 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:37.717 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:37.744 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:37.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:37.745 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:37.783 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T18:52:37.846 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:37.852 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-20T18:52:37.852 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-20T18:52:37.852 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph tell osd.7 'trim stale osdmaps' 2024-09-20T18:52:37.871 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:37.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:37.872 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:37.973 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:38.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:37 smithi142 ceph-mon[32508]: 4.1e scrub starts 2024-09-20T18:52:38.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:37 smithi142 ceph-mon[32508]: 4.1e scrub ok 2024-09-20T18:52:38.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:37 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:52:38.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:37 smithi142 ceph-mon[32508]: 5.b scrub starts 2024-09-20T18:52:38.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:37 smithi142 ceph-mon[32508]: 5.b scrub ok 2024-09-20T18:52:38.040 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:38.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:38.041 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:38.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:37 smithi083 ceph-mon[29421]: 4.1e scrub starts 2024-09-20T18:52:38.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:37 smithi083 ceph-mon[29421]: 4.1e scrub ok 2024-09-20T18:52:38.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:37 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:52:38.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:37 smithi083 ceph-mon[29421]: 5.b scrub starts 2024-09-20T18:52:38.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:37 smithi083 ceph-mon[29421]: 5.b scrub ok 2024-09-20T18:52:38.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:37 smithi083 ceph-mon[36260]: 4.1e scrub starts 2024-09-20T18:52:38.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:37 smithi083 ceph-mon[36260]: 4.1e scrub ok 2024-09-20T18:52:38.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:37 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:52:38.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:37 smithi083 ceph-mon[36260]: 5.b scrub starts 2024-09-20T18:52:38.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:37 smithi083 ceph-mon[36260]: 5.b scrub ok 2024-09-20T18:52:38.142 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:52:38.168 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:38.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:38.169 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:52:38.270 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:38.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:38.316 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:38.316 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:38.417 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:38.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:38.456 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:38.456 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:38.557 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:38.582 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:38.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:38.583 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:38.684 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:38.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:38.721 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:38.721 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:38.822 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:38.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:38.861 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:38.861 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:38.926 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:38 smithi142 ceph-mon[32508]: pgmap v126: 148 pgs: 148 active+clean; 6.9 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1.8 KiB/s rd, 17 MiB/s wr, 1.73k op/s 2024-09-20T18:52:38.926 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:38 smithi142 ceph-mon[32508]: 4.7 scrub starts 2024-09-20T18:52:38.926 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:38 smithi142 ceph-mon[32508]: 4.7 scrub ok 2024-09-20T18:52:38.926 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:38 smithi142 ceph-mon[32508]: 4.9 scrub starts 2024-09-20T18:52:38.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:38 smithi142 ceph-mon[32508]: 4.9 scrub ok 2024-09-20T18:52:38.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:38 smithi142 ceph-mon[32508]: 5.f scrub starts 2024-09-20T18:52:38.927 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:38 smithi142 ceph-mon[32508]: 5.f scrub ok 2024-09-20T18:52:38.962 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:39.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:39.006 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:39.006 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:39.023 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:38 smithi083 ceph-mon[29421]: pgmap v126: 148 pgs: 148 active+clean; 6.9 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1.8 KiB/s rd, 17 MiB/s wr, 1.73k op/s 2024-09-20T18:52:39.023 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:38 smithi083 ceph-mon[29421]: 4.7 scrub starts 2024-09-20T18:52:39.023 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:38 smithi083 ceph-mon[29421]: 4.7 scrub ok 2024-09-20T18:52:39.023 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:38 smithi083 ceph-mon[29421]: 4.9 scrub starts 2024-09-20T18:52:39.023 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:38 smithi083 ceph-mon[29421]: 4.9 scrub ok 2024-09-20T18:52:39.023 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:38 smithi083 ceph-mon[29421]: 5.f scrub starts 2024-09-20T18:52:39.023 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:38 smithi083 ceph-mon[29421]: 5.f scrub ok 2024-09-20T18:52:39.024 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:38 smithi083 ceph-mon[36260]: pgmap v126: 148 pgs: 148 active+clean; 6.9 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1.8 KiB/s rd, 17 MiB/s wr, 1.73k op/s 2024-09-20T18:52:39.024 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:38 smithi083 ceph-mon[36260]: 4.7 scrub starts 2024-09-20T18:52:39.024 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:38 smithi083 ceph-mon[36260]: 4.7 scrub ok 2024-09-20T18:52:39.024 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:38 smithi083 ceph-mon[36260]: 4.9 scrub starts 2024-09-20T18:52:39.024 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:38 smithi083 ceph-mon[36260]: 4.9 scrub ok 2024-09-20T18:52:39.024 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:38 smithi083 ceph-mon[36260]: 5.f scrub starts 2024-09-20T18:52:39.024 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:38 smithi083 ceph-mon[36260]: 5.f scrub ok 2024-09-20T18:52:39.108 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:39.141 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:52:39.171 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:39.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:39.173 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:39.233 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:52:39.274 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:39.322 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:39.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:39.323 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:39.358 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:52:39.424 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:39.456 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:39.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:39.457 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:39.558 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:39.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:39.601 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:39.602 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:39.685 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:39 smithi142 ceph-mon[32508]: 6.8 scrub starts 2024-09-20T18:52:39.685 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:39 smithi142 ceph-mon[32508]: 6.8 scrub ok 2024-09-20T18:52:39.686 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:39 smithi142 ceph-mon[32508]: 6.10 scrub starts 2024-09-20T18:52:39.686 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:39 smithi142 ceph-mon[32508]: 6.10 scrub ok 2024-09-20T18:52:39.686 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:39 smithi142 ceph-mon[32508]: 4.12 scrub starts 2024-09-20T18:52:39.686 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:39 smithi142 ceph-mon[32508]: 4.12 scrub ok 2024-09-20T18:52:39.686 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:39 smithi142 ceph-mon[32508]: pgmap v127: 148 pgs: 148 active+clean; 6.9 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 767 B/s rd, 11 MiB/s wr, 1.21k op/s 2024-09-20T18:52:39.686 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:39 smithi142 ceph-mon[32508]: 5.15 scrub starts 2024-09-20T18:52:39.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:39 smithi142 ceph-mon[32508]: 5.15 scrub ok 2024-09-20T18:52:39.702 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:39.735 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:39.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:39.737 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:39.838 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:39.868 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:39.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:39.869 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:39.930 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:39 smithi083 ceph-mon[29421]: 6.8 scrub starts 2024-09-20T18:52:39.930 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:39 smithi083 ceph-mon[29421]: 6.8 scrub ok 2024-09-20T18:52:39.930 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:39 smithi083 ceph-mon[29421]: 6.10 scrub starts 2024-09-20T18:52:39.930 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:39 smithi083 ceph-mon[29421]: 6.10 scrub ok 2024-09-20T18:52:39.930 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:39 smithi083 ceph-mon[29421]: 4.12 scrub starts 2024-09-20T18:52:39.930 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:39 smithi083 ceph-mon[29421]: 4.12 scrub ok 2024-09-20T18:52:39.930 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:39 smithi083 ceph-mon[29421]: pgmap v127: 148 pgs: 148 active+clean; 6.9 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 767 B/s rd, 11 MiB/s wr, 1.21k op/s 2024-09-20T18:52:39.930 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:39 smithi083 ceph-mon[29421]: 5.15 scrub starts 2024-09-20T18:52:39.930 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:39 smithi083 ceph-mon[29421]: 5.15 scrub ok 2024-09-20T18:52:39.930 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:39 smithi083 ceph-mon[36260]: 6.8 scrub starts 2024-09-20T18:52:39.931 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:39 smithi083 ceph-mon[36260]: 6.8 scrub ok 2024-09-20T18:52:39.931 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:39 smithi083 ceph-mon[36260]: 6.10 scrub starts 2024-09-20T18:52:39.931 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:39 smithi083 ceph-mon[36260]: 6.10 scrub ok 2024-09-20T18:52:39.931 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:39 smithi083 ceph-mon[36260]: 4.12 scrub starts 2024-09-20T18:52:39.931 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:39 smithi083 ceph-mon[36260]: 4.12 scrub ok 2024-09-20T18:52:39.931 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:39 smithi083 ceph-mon[36260]: pgmap v127: 148 pgs: 148 active+clean; 6.9 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 767 B/s rd, 11 MiB/s wr, 1.21k op/s 2024-09-20T18:52:39.931 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:39 smithi083 ceph-mon[36260]: 5.15 scrub starts 2024-09-20T18:52:39.931 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:39 smithi083 ceph-mon[36260]: 5.15 scrub ok 2024-09-20T18:52:39.969 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:40.014 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:40.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:40.015 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:40.100 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T18:52:40.100 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-20T18:52:40.100 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-20T18:52:40.100 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-20T18:52:40.100 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T18:52:40.116 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:40.146 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:40.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:40.147 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:40.248 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:40.293 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:40.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:40.294 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:40.344 INFO:teuthology.orchestra.run.smithi083.stderr:no valid command found; 10 closest matches: 2024-09-20T18:52:40.344 INFO:teuthology.orchestra.run.smithi083.stderr:0 2024-09-20T18:52:40.344 INFO:teuthology.orchestra.run.smithi083.stderr:1 2024-09-20T18:52:40.344 INFO:teuthology.orchestra.run.smithi083.stderr:2 2024-09-20T18:52:40.345 INFO:teuthology.orchestra.run.smithi083.stderr:abort 2024-09-20T18:52:40.345 INFO:teuthology.orchestra.run.smithi083.stderr:assert 2024-09-20T18:52:40.345 INFO:teuthology.orchestra.run.smithi083.stderr:bench [] [] [] [] 2024-09-20T18:52:40.345 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs debug_inject_read_zeros 2024-09-20T18:52:40.345 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs files list 2024-09-20T18:52:40.345 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs stats 2024-09-20T18:52:40.345 INFO:teuthology.orchestra.run.smithi083.stderr:bluestore allocator dump block 2024-09-20T18:52:40.345 INFO:teuthology.orchestra.run.smithi083.stderr:Error EINVAL: invalid command 2024-09-20T18:52:40.395 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:40.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:40.426 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:40.426 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:52:40.527 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:40.555 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:40.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:40.558 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:40.659 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:40.688 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:40.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:40.689 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:52:40.789 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:40.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:40 smithi083 ceph-mon[29421]: 5.1a scrub starts 2024-09-20T18:52:40.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:40 smithi083 ceph-mon[29421]: 5.1a scrub ok 2024-09-20T18:52:40.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:40 smithi083 ceph-mon[29421]: 4.2 scrub starts 2024-09-20T18:52:40.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:40 smithi083 ceph-mon[29421]: 4.2 scrub ok 2024-09-20T18:52:40.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:40 smithi083 ceph-mon[29421]: 6.5 scrub starts 2024-09-20T18:52:40.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:40 smithi083 ceph-mon[29421]: 6.5 scrub ok 2024-09-20T18:52:40.805 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:40 smithi083 ceph-mon[36260]: 5.1a scrub starts 2024-09-20T18:52:40.805 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:40 smithi083 ceph-mon[36260]: 5.1a scrub ok 2024-09-20T18:52:40.805 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:40 smithi083 ceph-mon[36260]: 4.2 scrub starts 2024-09-20T18:52:40.805 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:40 smithi083 ceph-mon[36260]: 4.2 scrub ok 2024-09-20T18:52:40.805 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:40 smithi083 ceph-mon[36260]: 6.5 scrub starts 2024-09-20T18:52:40.805 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:40 smithi083 ceph-mon[36260]: 6.5 scrub ok 2024-09-20T18:52:40.806 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:52:40 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:52:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:52:40.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:40.829 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:40.830 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:40.930 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:40.970 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:40.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:40.971 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:41.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:40 smithi142 ceph-mon[32508]: 5.1a scrub starts 2024-09-20T18:52:41.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:40 smithi142 ceph-mon[32508]: 5.1a scrub ok 2024-09-20T18:52:41.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:40 smithi142 ceph-mon[32508]: 4.2 scrub starts 2024-09-20T18:52:41.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:40 smithi142 ceph-mon[32508]: 4.2 scrub ok 2024-09-20T18:52:41.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:40 smithi142 ceph-mon[32508]: 6.5 scrub starts 2024-09-20T18:52:41.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:40 smithi142 ceph-mon[32508]: 6.5 scrub ok 2024-09-20T18:52:41.071 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T18:52:41.071 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:41.104 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:41.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:41.105 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:41.206 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:41.248 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:41.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:41.249 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:41.335 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-20T18:52:41.424 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-20T18:52:41.424 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:41.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:41.469 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:41.469 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:41.570 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:41.608 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:41.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:41.609 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:41.710 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:41.746 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:41.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:41.747 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:41.848 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:41.885 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:41.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:41.886 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:41.987 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:42.013 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:42.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:42.015 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:42.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:41 smithi142 ceph-mon[32508]: 5.d scrub starts 2024-09-20T18:52:42.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:41 smithi142 ceph-mon[32508]: 5.d scrub ok 2024-09-20T18:52:42.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:41 smithi142 ceph-mon[32508]: pgmap v128: 148 pgs: 148 active+clean; 6.9 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 795 op/s 2024-09-20T18:52:42.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:41 smithi142 ceph-mon[32508]: 3.18 scrub starts 2024-09-20T18:52:42.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:41 smithi142 ceph-mon[32508]: 3.18 scrub ok 2024-09-20T18:52:42.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:41 smithi083 ceph-mon[36260]: 5.d scrub starts 2024-09-20T18:52:42.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:41 smithi083 ceph-mon[36260]: 5.d scrub ok 2024-09-20T18:52:42.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:41 smithi083 ceph-mon[36260]: pgmap v128: 148 pgs: 148 active+clean; 6.9 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 795 op/s 2024-09-20T18:52:42.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:41 smithi083 ceph-mon[36260]: 3.18 scrub starts 2024-09-20T18:52:42.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:41 smithi083 ceph-mon[36260]: 3.18 scrub ok 2024-09-20T18:52:42.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:41 smithi083 ceph-mon[29421]: 5.d scrub starts 2024-09-20T18:52:42.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:41 smithi083 ceph-mon[29421]: 5.d scrub ok 2024-09-20T18:52:42.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:41 smithi083 ceph-mon[29421]: pgmap v128: 148 pgs: 148 active+clean; 6.9 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 795 op/s 2024-09-20T18:52:42.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:41 smithi083 ceph-mon[29421]: 3.18 scrub starts 2024-09-20T18:52:42.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:41 smithi083 ceph-mon[29421]: 3.18 scrub ok 2024-09-20T18:52:42.115 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:42.145 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:42.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:42.146 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:52:42.247 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:42.296 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:42.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:42.297 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:42.393 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_historic_ops 2024-09-20T18:52:42.397 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:42.495 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:42.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:42.496 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:42.584 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:52:42.597 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:42.666 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:42.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:42.668 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:42.698 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-20T18:52:42.769 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:42.794 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:42.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:42.795 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:42.896 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:42.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:42.919 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:42.919 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:43.020 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:43.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:42 smithi142 ceph-mon[32508]: 6.17 scrub starts 2024-09-20T18:52:43.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:42 smithi142 ceph-mon[32508]: 6.17 scrub ok 2024-09-20T18:52:43.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:42 smithi142 ceph-mon[32508]: 5.10 scrub starts 2024-09-20T18:52:43.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:42 smithi142 ceph-mon[32508]: 5.10 scrub ok 2024-09-20T18:52:43.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:42 smithi142 ceph-mon[32508]: 3.1b scrub starts 2024-09-20T18:52:43.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:42 smithi142 ceph-mon[32508]: 3.1b scrub ok 2024-09-20T18:52:43.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:42 smithi142 ceph-mon[32508]: 4.0 scrub starts 2024-09-20T18:52:43.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:42 smithi142 ceph-mon[32508]: 4.0 scrub ok 2024-09-20T18:52:43.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:42 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1605148258' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:52:43.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:42 smithi083 ceph-mon[29421]: 6.17 scrub starts 2024-09-20T18:52:43.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:42 smithi083 ceph-mon[29421]: 6.17 scrub ok 2024-09-20T18:52:43.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:42 smithi083 ceph-mon[29421]: 5.10 scrub starts 2024-09-20T18:52:43.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:42 smithi083 ceph-mon[29421]: 5.10 scrub ok 2024-09-20T18:52:43.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:42 smithi083 ceph-mon[29421]: 3.1b scrub starts 2024-09-20T18:52:43.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:42 smithi083 ceph-mon[29421]: 3.1b scrub ok 2024-09-20T18:52:43.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:42 smithi083 ceph-mon[29421]: 4.0 scrub starts 2024-09-20T18:52:43.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:42 smithi083 ceph-mon[29421]: 4.0 scrub ok 2024-09-20T18:52:43.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:42 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1605148258' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:52:43.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:42 smithi083 ceph-mon[36260]: 6.17 scrub starts 2024-09-20T18:52:43.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:42 smithi083 ceph-mon[36260]: 6.17 scrub ok 2024-09-20T18:52:43.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:42 smithi083 ceph-mon[36260]: 5.10 scrub starts 2024-09-20T18:52:43.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:42 smithi083 ceph-mon[36260]: 5.10 scrub ok 2024-09-20T18:52:43.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:42 smithi083 ceph-mon[36260]: 3.1b scrub starts 2024-09-20T18:52:43.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:42 smithi083 ceph-mon[36260]: 3.1b scrub ok 2024-09-20T18:52:43.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:42 smithi083 ceph-mon[36260]: 4.0 scrub starts 2024-09-20T18:52:43.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:42 smithi083 ceph-mon[36260]: 4.0 scrub ok 2024-09-20T18:52:43.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:42 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1605148258' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:52:43.057 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:43.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:43.058 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:43.159 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:43.181 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:43.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:43.182 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:43.283 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:43.317 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:43.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:43.319 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:43.419 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:43.456 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:43.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:43.457 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:43.557 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:43.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:43.597 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:43.597 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:43.698 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:52:43.726 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:43.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:43.727 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:52:43.828 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:43.856 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:43.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:43.857 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:52:43.958 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:43.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:43.986 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:43.987 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:44.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:43 smithi142 ceph-mon[32508]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T18:52:44.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:43 smithi142 ceph-mon[32508]: Cluster is now healthy 2024-09-20T18:52:44.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:43 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1605148258' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T18:52:44.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:43 smithi142 ceph-mon[32508]: osdmap e127: 8 total, 8 up, 8 in 2024-09-20T18:52:44.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:43 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:52:44.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:43 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:52:44.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:43 smithi142 ceph-mon[32508]: pgmap v130: 148 pgs: 148 active+clean; 6.6 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.34k op/s 2024-09-20T18:52:44.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:43 smithi142 ceph-mon[32508]: 6.3 scrub starts 2024-09-20T18:52:44.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:43 smithi142 ceph-mon[32508]: 6.3 scrub ok 2024-09-20T18:52:44.026 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:43 smithi142 ceph-mon[32508]: 4.6 scrub starts 2024-09-20T18:52:44.026 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:43 smithi142 ceph-mon[32508]: 4.6 scrub ok 2024-09-20T18:52:44.026 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:43 smithi142 ceph-mon[32508]: 6.13 scrub starts 2024-09-20T18:52:44.026 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:43 smithi142 ceph-mon[32508]: 6.d scrub starts 2024-09-20T18:52:44.026 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:43 smithi142 ceph-mon[32508]: 6.13 scrub ok 2024-09-20T18:52:44.027 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:43 smithi142 ceph-mon[32508]: 6.d scrub ok 2024-09-20T18:52:44.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:43 smithi083 ceph-mon[29421]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T18:52:44.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:43 smithi083 ceph-mon[29421]: Cluster is now healthy 2024-09-20T18:52:44.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:43 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1605148258' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T18:52:44.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:43 smithi083 ceph-mon[29421]: osdmap e127: 8 total, 8 up, 8 in 2024-09-20T18:52:44.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:43 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:52:44.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:43 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:52:44.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:43 smithi083 ceph-mon[29421]: pgmap v130: 148 pgs: 148 active+clean; 6.6 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.34k op/s 2024-09-20T18:52:44.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:43 smithi083 ceph-mon[29421]: 6.3 scrub starts 2024-09-20T18:52:44.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:43 smithi083 ceph-mon[29421]: 6.3 scrub ok 2024-09-20T18:52:44.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:43 smithi083 ceph-mon[29421]: 4.6 scrub starts 2024-09-20T18:52:44.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:43 smithi083 ceph-mon[29421]: 4.6 scrub ok 2024-09-20T18:52:44.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:43 smithi083 ceph-mon[29421]: 6.13 scrub starts 2024-09-20T18:52:44.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:43 smithi083 ceph-mon[29421]: 6.d scrub starts 2024-09-20T18:52:44.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:43 smithi083 ceph-mon[29421]: 6.13 scrub ok 2024-09-20T18:52:44.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:43 smithi083 ceph-mon[29421]: 6.d scrub ok 2024-09-20T18:52:44.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:43 smithi083 ceph-mon[36260]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T18:52:44.057 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:43 smithi083 ceph-mon[36260]: Cluster is now healthy 2024-09-20T18:52:44.057 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:43 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1605148258' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T18:52:44.057 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:43 smithi083 ceph-mon[36260]: osdmap e127: 8 total, 8 up, 8 in 2024-09-20T18:52:44.057 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:43 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:52:44.057 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:43 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:52:44.057 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:43 smithi083 ceph-mon[36260]: pgmap v130: 148 pgs: 148 active+clean; 6.6 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.34k op/s 2024-09-20T18:52:44.057 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:43 smithi083 ceph-mon[36260]: 6.3 scrub starts 2024-09-20T18:52:44.057 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:43 smithi083 ceph-mon[36260]: 6.3 scrub ok 2024-09-20T18:52:44.057 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:43 smithi083 ceph-mon[36260]: 4.6 scrub starts 2024-09-20T18:52:44.057 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:43 smithi083 ceph-mon[36260]: 4.6 scrub ok 2024-09-20T18:52:44.057 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:43 smithi083 ceph-mon[36260]: 6.13 scrub starts 2024-09-20T18:52:44.057 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:43 smithi083 ceph-mon[36260]: 6.d scrub starts 2024-09-20T18:52:44.058 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:43 smithi083 ceph-mon[36260]: 6.13 scrub ok 2024-09-20T18:52:44.058 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:43 smithi083 ceph-mon[36260]: 6.d scrub ok 2024-09-20T18:52:44.087 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:44.114 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:44.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:44.115 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:52:44.216 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:44.244 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:44.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:44.245 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:44.346 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:52:44.371 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:44.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:44.372 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:52:44.473 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:44.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:44.510 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:44.510 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:44.612 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:44.649 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:44.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:44.651 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:44.751 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:44.776 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:44.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:44.777 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:44.846 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:52:44.847 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:52:44.847 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:44 smithi083 ceph-mon[29421]: osdmap e128: 8 total, 8 up, 8 in 2024-09-20T18:52:44.847 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:44 smithi083 ceph-mon[36260]: osdmap e128: 8 total, 8 up, 8 in 2024-09-20T18:52:44.878 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:44.884 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:52:44.885 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:52:44.914 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:44.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:44.915 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:44.917 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:52:44.918 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:52:44.954 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:52:44.955 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:52:44.991 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:52:44.991 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:52:45.017 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:45.021 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:52:45.022 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:52:45.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:44 smithi142 ceph-mon[32508]: osdmap e128: 8 total, 8 up, 8 in 2024-09-20T18:52:45.043 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:45.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:45.044 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:45.057 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:52:45.057 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:52:45.086 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:52:45.086 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:52:45.145 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:45.173 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:45.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:45.174 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:45.275 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:45.298 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:45.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:45.299 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:45.399 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:45.426 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:45.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:45.428 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:45.528 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:45.555 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:45.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:45.556 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:52:45.657 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:45.696 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:45.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:45.697 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:45.798 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:45.834 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:45.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:45.835 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:45.936 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:52:45.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:45.964 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:45.964 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:52:46.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:45 smithi083 ceph-mon[29421]: 5.11 scrub starts 2024-09-20T18:52:46.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:45 smithi083 ceph-mon[29421]: 5.11 scrub ok 2024-09-20T18:52:46.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:45 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2476058483' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:52:46.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:45 smithi083 ceph-mon[29421]: pgmap v132: 148 pgs: 148 active+clean; 6.6 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 673 op/s 2024-09-20T18:52:46.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:45 smithi083 ceph-mon[29421]: 6.4 scrub starts 2024-09-20T18:52:46.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:45 smithi083 ceph-mon[29421]: 6.4 scrub ok 2024-09-20T18:52:46.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:45 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:52:46.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:45 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:52:46.007 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:45 smithi083 ceph-mon[36260]: 5.11 scrub starts 2024-09-20T18:52:46.007 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:45 smithi083 ceph-mon[36260]: 5.11 scrub ok 2024-09-20T18:52:46.007 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:45 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2476058483' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:52:46.007 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:45 smithi083 ceph-mon[36260]: pgmap v132: 148 pgs: 148 active+clean; 6.6 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 673 op/s 2024-09-20T18:52:46.007 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:45 smithi083 ceph-mon[36260]: 6.4 scrub starts 2024-09-20T18:52:46.007 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:45 smithi083 ceph-mon[36260]: 6.4 scrub ok 2024-09-20T18:52:46.007 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:45 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:52:46.007 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:45 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:52:46.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:45 smithi142 ceph-mon[32508]: 5.11 scrub starts 2024-09-20T18:52:46.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:45 smithi142 ceph-mon[32508]: 5.11 scrub ok 2024-09-20T18:52:46.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:45 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2476058483' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:52:46.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:45 smithi142 ceph-mon[32508]: pgmap v132: 148 pgs: 148 active+clean; 6.6 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 673 op/s 2024-09-20T18:52:46.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:45 smithi142 ceph-mon[32508]: 6.4 scrub starts 2024-09-20T18:52:46.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:45 smithi142 ceph-mon[32508]: 6.4 scrub ok 2024-09-20T18:52:46.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:45 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:52:46.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:45 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:52:46.066 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:46.106 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:46.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:46.107 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:46.134 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:52:46.209 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:46.235 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:46.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:46.237 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:52:46.337 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:46.365 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:46.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:46.366 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:46.425 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-20T18:52:46.425 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T18:52:46.425 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-20T18:52:46.425 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-20T18:52:46.425 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-20T18:52:46.425 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.0 config set filestore_inject_stall 3 2024-09-20T18:52:46.467 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:46.494 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:46.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:46.495 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:52:46.495 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 18:52:46 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=infra.usagestats t=2024-09-20T18:52:46.222951662Z level=info msg="Usage stats are ready to report" 2024-09-20T18:52:46.596 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:46.619 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:46.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:46.620 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:52:46.721 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:46.747 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:46.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:46.748 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:52:46.774 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:52:46 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:52:46.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:46 smithi142 ceph-mon[32508]: 3.b scrub starts 2024-09-20T18:52:46.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:46 smithi142 ceph-mon[32508]: 6.9 scrub starts 2024-09-20T18:52:46.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:46 smithi142 ceph-mon[32508]: 6.9 scrub ok 2024-09-20T18:52:46.775 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:46 smithi142 ceph-mon[32508]: 3.b scrub ok 2024-09-20T18:52:46.775 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:46 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:52:46.849 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:52:46.875 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:46.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:46.876 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:52:46.957 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T18:52:46.957 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:52:46.977 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:47.000 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:47.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:47.001 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:47.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:46 smithi083 ceph-mon[29421]: 3.b scrub starts 2024-09-20T18:52:47.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:46 smithi083 ceph-mon[29421]: 6.9 scrub starts 2024-09-20T18:52:47.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:46 smithi083 ceph-mon[29421]: 6.9 scrub ok 2024-09-20T18:52:47.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:46 smithi083 ceph-mon[29421]: 3.b scrub ok 2024-09-20T18:52:47.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:46 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:52:47.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:46 smithi083 ceph-mon[36260]: 3.b scrub starts 2024-09-20T18:52:47.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:46 smithi083 ceph-mon[36260]: 6.9 scrub starts 2024-09-20T18:52:47.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:46 smithi083 ceph-mon[36260]: 6.9 scrub ok 2024-09-20T18:52:47.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:46 smithi083 ceph-mon[36260]: 3.b scrub ok 2024-09-20T18:52:47.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:46 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:52:47.103 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:47.156 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:47.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:47.158 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:47.259 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:47.284 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:47.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:47.285 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:47.354 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T18:52:47.386 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:47.411 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:47.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:47.412 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:52:47.512 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:52:47.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:47.538 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:47.538 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:52:47.624 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:52:47.639 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:47.662 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:47.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:47.663 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:47.671 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-20T18:52:47.764 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:47.788 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:47.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:47.789 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:47.890 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:47.932 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:47.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:47.933 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:47.976 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:47 smithi083 ceph-mon[29421]: pgmap v133: 148 pgs: 148 active+clean; 6.3 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.34k op/s 2024-09-20T18:52:47.976 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:47 smithi083 ceph-mon[29421]: 6.1e scrub starts 2024-09-20T18:52:47.976 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:47 smithi083 ceph-mon[29421]: 6.1e scrub ok 2024-09-20T18:52:47.976 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:47 smithi083 ceph-mon[29421]: 3.19 scrub starts 2024-09-20T18:52:47.976 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:47 smithi083 ceph-mon[29421]: 3.19 scrub ok 2024-09-20T18:52:47.976 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:47 smithi083 ceph-mon[36260]: pgmap v133: 148 pgs: 148 active+clean; 6.3 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.34k op/s 2024-09-20T18:52:47.976 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:47 smithi083 ceph-mon[36260]: 6.1e scrub starts 2024-09-20T18:52:47.977 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:47 smithi083 ceph-mon[36260]: 6.1e scrub ok 2024-09-20T18:52:47.977 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:47 smithi083 ceph-mon[36260]: 3.19 scrub starts 2024-09-20T18:52:47.977 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:47 smithi083 ceph-mon[36260]: 3.19 scrub ok 2024-09-20T18:52:48.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:47 smithi142 ceph-mon[32508]: pgmap v133: 148 pgs: 148 active+clean; 6.3 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.34k op/s 2024-09-20T18:52:48.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:47 smithi142 ceph-mon[32508]: 6.1e scrub starts 2024-09-20T18:52:48.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:47 smithi142 ceph-mon[32508]: 6.1e scrub ok 2024-09-20T18:52:48.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:47 smithi142 ceph-mon[32508]: 3.19 scrub starts 2024-09-20T18:52:48.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:47 smithi142 ceph-mon[32508]: 3.19 scrub ok 2024-09-20T18:52:48.034 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:48.059 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T18:52:48.067 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:48.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:48.069 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:48.170 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:48.193 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:48.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:48.194 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:48.295 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:48.319 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:48.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:48.320 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:48.334 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T18:52:48.334 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-20T18:52:48.334 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T18:52:48.421 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:48.467 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:48.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:48.468 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:48.569 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:48.618 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:48.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:48.619 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:48.719 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:52:48.741 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:48.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:48.742 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:52:48.842 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:48.868 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:48.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:48.869 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:48.970 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:48.998 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:48.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:48.999 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:49.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:48 smithi142 ceph-mon[32508]: 3.1d scrub starts 2024-09-20T18:52:49.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:48 smithi142 ceph-mon[32508]: 3.1d scrub ok 2024-09-20T18:52:49.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:48 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:52:49.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:48 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:52:49.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:48 smithi083 ceph-mon[29421]: 3.1d scrub starts 2024-09-20T18:52:49.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:48 smithi083 ceph-mon[29421]: 3.1d scrub ok 2024-09-20T18:52:49.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:48 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:52:49.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:48 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:52:49.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:48 smithi083 ceph-mon[36260]: 3.1d scrub starts 2024-09-20T18:52:49.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:48 smithi083 ceph-mon[36260]: 3.1d scrub ok 2024-09-20T18:52:49.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:48 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:52:49.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:48 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:52:49.100 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:52:49.127 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:49.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:49.128 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:52:49.229 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:49.265 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:49.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:49.267 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:49.367 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:49.396 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:49.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:49.397 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:52:49.498 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:49.531 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:49.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:49.532 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:49.633 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:49.663 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:49.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:49.665 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:49.766 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:49.793 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:49.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:49.795 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:49.896 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:49.926 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:49.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:49.927 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:52:50.027 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:50.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:49 smithi083 ceph-mon[29421]: osdmap e129: 8 total, 8 up, 8 in 2024-09-20T18:52:50.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:49 smithi083 ceph-mon[29421]: pgmap v135: 148 pgs: 148 active+clean; 6.3 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 836 B/s rd, 0 B/s wr, 872 op/s 2024-09-20T18:52:50.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:49 smithi083 ceph-mon[29421]: 4.3 scrub starts 2024-09-20T18:52:50.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:49 smithi083 ceph-mon[29421]: 4.3 scrub ok 2024-09-20T18:52:50.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:49 smithi083 ceph-mon[29421]: 5.c scrub starts 2024-09-20T18:52:50.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:49 smithi083 ceph-mon[29421]: 5.c scrub ok 2024-09-20T18:52:50.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:49 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:52:50.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:49 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:52:50.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:49 smithi083 ceph-mon[36260]: osdmap e129: 8 total, 8 up, 8 in 2024-09-20T18:52:50.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:49 smithi083 ceph-mon[36260]: pgmap v135: 148 pgs: 148 active+clean; 6.3 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 836 B/s rd, 0 B/s wr, 872 op/s 2024-09-20T18:52:50.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:49 smithi083 ceph-mon[36260]: 4.3 scrub starts 2024-09-20T18:52:50.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:49 smithi083 ceph-mon[36260]: 4.3 scrub ok 2024-09-20T18:52:50.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:49 smithi083 ceph-mon[36260]: 5.c scrub starts 2024-09-20T18:52:50.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:49 smithi083 ceph-mon[36260]: 5.c scrub ok 2024-09-20T18:52:50.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:49 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:52:50.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:49 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:52:50.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:50.059 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:50.059 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:52:50.159 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:50.202 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:50.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:50.203 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:50.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:49 smithi142 ceph-mon[32508]: osdmap e129: 8 total, 8 up, 8 in 2024-09-20T18:52:50.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:49 smithi142 ceph-mon[32508]: pgmap v135: 148 pgs: 148 active+clean; 6.3 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 836 B/s rd, 0 B/s wr, 872 op/s 2024-09-20T18:52:50.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:49 smithi142 ceph-mon[32508]: 4.3 scrub starts 2024-09-20T18:52:50.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:49 smithi142 ceph-mon[32508]: 4.3 scrub ok 2024-09-20T18:52:50.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:49 smithi142 ceph-mon[32508]: 5.c scrub starts 2024-09-20T18:52:50.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:49 smithi142 ceph-mon[32508]: 5.c scrub ok 2024-09-20T18:52:50.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:49 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:52:50.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:49 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:52:50.304 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:52:50.330 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:50.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:50.331 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:52:50.432 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:50.463 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:50.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:50.464 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:50.564 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:50.592 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:50.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:50.593 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:52:50.694 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:50.722 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:50.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:50.723 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:50.824 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:50.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:50.849 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:50.849 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:50.950 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:52:50.979 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:50.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:50.980 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:52:51.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:50 smithi083 ceph-mon[29421]: 7.d scrub starts 2024-09-20T18:52:51.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:50 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:52:51.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:50 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:52:51.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:50 smithi083 ceph-mon[29421]: osdmap e130: 8 total, 8 up, 8 in 2024-09-20T18:52:51.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:50 smithi083 ceph-mon[29421]: 7.f scrub starts 2024-09-20T18:52:51.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:50 smithi083 ceph-mon[36260]: 7.d scrub starts 2024-09-20T18:52:51.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:50 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:52:51.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:50 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:52:51.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:50 smithi083 ceph-mon[36260]: osdmap e130: 8 total, 8 up, 8 in 2024-09-20T18:52:51.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:50 smithi083 ceph-mon[36260]: 7.f scrub starts 2024-09-20T18:52:51.055 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:52:50 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:52:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:52:51.080 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:51.119 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:51.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:51.120 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:51.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:50 smithi142 ceph-mon[32508]: 7.d scrub starts 2024-09-20T18:52:51.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:50 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:52:51.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:50 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:52:51.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:50 smithi142 ceph-mon[32508]: osdmap e130: 8 total, 8 up, 8 in 2024-09-20T18:52:51.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:50 smithi142 ceph-mon[32508]: 7.f scrub starts 2024-09-20T18:52:51.201 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:52:51.201 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:52:51.221 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:52:51.244 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:51.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:51.245 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:52:51.287 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:52:51.346 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:51.397 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:51.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:51.398 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:51.499 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:51.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:51.532 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:51.532 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:51.633 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:51.661 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:51.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:51.662 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:51.766 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:51.790 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:51.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:51.791 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:51.892 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:51.919 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:51.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:51.920 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:52.021 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:52.047 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:51 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:52:52.047 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:51 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:52:52.047 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:51 smithi083 ceph-mon[29421]: osdmap e131: 8 total, 8 up, 8 in 2024-09-20T18:52:52.047 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:51 smithi083 ceph-mon[29421]: pgmap v138: 148 pgs: 148 active+clean; 6.2 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.05k op/s 2024-09-20T18:52:52.047 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:51 smithi083 ceph-mon[29421]: 7.d scrub ok 2024-09-20T18:52:52.047 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:51 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:52:52.047 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:51 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:52:52.047 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:51 smithi083 ceph-mon[36260]: osdmap e131: 8 total, 8 up, 8 in 2024-09-20T18:52:52.047 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:51 smithi083 ceph-mon[36260]: pgmap v138: 148 pgs: 148 active+clean; 6.2 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.05k op/s 2024-09-20T18:52:52.047 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:51 smithi083 ceph-mon[36260]: 7.d scrub ok 2024-09-20T18:52:52.048 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T18:52:52.048 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:52:52.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:52.058 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:52.059 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:52.159 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:52.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:52.184 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:52.184 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:52:52.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:51 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:52:52.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:51 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:52:52.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:51 smithi142 ceph-mon[32508]: osdmap e131: 8 total, 8 up, 8 in 2024-09-20T18:52:52.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:51 smithi142 ceph-mon[32508]: pgmap v138: 148 pgs: 148 active+clean; 6.2 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.05k op/s 2024-09-20T18:52:52.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:51 smithi142 ceph-mon[32508]: 7.d scrub ok 2024-09-20T18:52:52.285 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:52.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:52.331 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:52.331 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:52.431 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:52:52.456 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:52.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:52.457 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:52:52.557 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:52.591 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:52.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:52.592 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:52.693 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:52.720 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:52.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:52.721 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:52.821 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-20T18:52:52.821 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:52.863 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:52.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:52.864 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:52.916 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_blocked_ops 2024-09-20T18:52:52.965 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:53.009 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:53.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:53.010 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:53.111 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:53.135 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:53.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:53.136 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:53.237 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:52:53.264 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:53.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:53.265 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:52:53.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:52 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/4007345745' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:52:53.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:52 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/4007345745' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:52:53.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:52 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/4007345745' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:52:53.366 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:53.411 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:53.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:53.412 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:53.513 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:53.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:53.558 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:53.559 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:53.660 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:53.689 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:53.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:53.690 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:53.791 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:53.819 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:53.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:53.820 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:52:53.921 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:53.967 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:53.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:53.969 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:53.989 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:53 smithi083 ceph-mon[29421]: 7.f scrub ok 2024-09-20T18:52:53.989 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:53 smithi083 ceph-mon[29421]: 4.c scrub starts 2024-09-20T18:52:53.989 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:53 smithi083 ceph-mon[29421]: 4.c scrub ok 2024-09-20T18:52:53.989 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:53 smithi083 ceph-mon[29421]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:52:53.989 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:53 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/4007345745' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T18:52:53.990 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:53 smithi083 ceph-mon[29421]: osdmap e132: 8 total, 8 up, 8 in 2024-09-20T18:52:53.990 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:53 smithi083 ceph-mon[29421]: pgmap v140: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.0 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.29k op/s 2024-09-20T18:52:53.990 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:53 smithi083 ceph-mon[36260]: 7.f scrub ok 2024-09-20T18:52:53.990 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:53 smithi083 ceph-mon[36260]: 4.c scrub starts 2024-09-20T18:52:53.990 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:53 smithi083 ceph-mon[36260]: 4.c scrub ok 2024-09-20T18:52:53.990 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:53 smithi083 ceph-mon[36260]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:52:53.990 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:53 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/4007345745' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T18:52:53.990 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:53 smithi083 ceph-mon[36260]: osdmap e132: 8 total, 8 up, 8 in 2024-09-20T18:52:53.990 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:53 smithi083 ceph-mon[36260]: pgmap v140: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.0 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.29k op/s 2024-09-20T18:52:54.069 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:54.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:54.108 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:54.109 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:54.157 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:52:54.185 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:52:54.185 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:52:54.210 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:54.221 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:52:54.221 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:52:54.237 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:54.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:54.238 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:54.256 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:52:54.256 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:52:54.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:53 smithi142 ceph-mon[32508]: 7.f scrub ok 2024-09-20T18:52:54.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:53 smithi142 ceph-mon[32508]: 4.c scrub starts 2024-09-20T18:52:54.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:53 smithi142 ceph-mon[32508]: 4.c scrub ok 2024-09-20T18:52:54.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:53 smithi142 ceph-mon[32508]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:52:54.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:53 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/4007345745' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T18:52:54.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:53 smithi142 ceph-mon[32508]: osdmap e132: 8 total, 8 up, 8 in 2024-09-20T18:52:54.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:53 smithi142 ceph-mon[32508]: pgmap v140: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.0 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.29k op/s 2024-09-20T18:52:54.288 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:52:54.289 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:52:54.322 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:52:54.323 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:52:54.339 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:54.356 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:52:54.356 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:52:54.366 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:54.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:54.367 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:54.381 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:52:54.381 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:52:54.406 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:52:54.406 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:52:54.468 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:54.501 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:54.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:54.502 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:54.603 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:54.633 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:54.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:54.634 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:54.735 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:54.771 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:54.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:54.772 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:54.872 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:52:54.897 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:54.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:54.898 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:52:54.999 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:55.025 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:55.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:55.027 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:52:55.124 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:54 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2182167221' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:52:55.124 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:54 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2182167221' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:52:55.128 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:55.154 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:55.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:55.155 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:55.255 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:55.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:54 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2182167221' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:52:55.281 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:55.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:55.282 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:55.383 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:55.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:55.422 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:55.422 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:55.522 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:55.562 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:55.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:55.564 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:55.580 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T18:52:55.665 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:55.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:55.737 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:55.737 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:55.838 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:52:55.862 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:55.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:55.863 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:52:55.963 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:52:55.990 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:55.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:55.991 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:52:56.045 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T18:52:56.045 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-20T18:52:56.045 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-20T18:52:56.045 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-20T18:52:56.045 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T18:52:56.091 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:52:56.115 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:56.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:56.116 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:52:56.174 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-20T18:52:56.175 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd reweight 0 1 2024-09-20T18:52:56.213 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:55 smithi083 ceph-mon[29421]: pgmap v141: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.0 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 871 op/s 2024-09-20T18:52:56.213 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:55 smithi083 ceph-mon[36260]: pgmap v141: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.0 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 871 op/s 2024-09-20T18:52:56.217 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:56.267 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:56.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:56.268 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:56.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:55 smithi142 ceph-mon[32508]: pgmap v141: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.0 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 871 op/s 2024-09-20T18:52:56.369 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:56.435 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:56.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:56.437 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:56.538 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:56.563 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:56.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:56.564 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:56.665 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:56.692 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:56.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:56.693 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:52:56.774 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:52:56 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:52:56.794 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:56.822 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:56.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:56.823 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:56.923 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:56.951 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:56.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:56.952 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:52:57.053 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:52:57.080 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:57.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:57.081 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:52:57.141 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:56 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:52:57.141 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:56 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:52:57.182 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:57.194 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_historic_ops 2024-09-20T18:52:57.205 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:52:57.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:57.243 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:57.243 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:57.252 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:52:57.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:56 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:52:57.344 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:57.404 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:57.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:57.405 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:57.505 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:57.531 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:57.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:57.532 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:52:57.634 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:57.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:57.686 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:57.686 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:57.787 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:52:57.831 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:57.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:57.832 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:52:57.933 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:57.959 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:57.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:57.960 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:58.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:57 smithi083 ceph-mon[29421]: pgmap v142: 148 pgs: 148 active+clean; 5.6 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.50k op/s 2024-09-20T18:52:58.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:57 smithi083 ceph-mon[36260]: pgmap v142: 148 pgs: 148 active+clean; 5.6 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.50k op/s 2024-09-20T18:52:58.061 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:58.103 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:58.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:58.104 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:58.205 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:52:58.232 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:58.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:58.233 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:52:58.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:57 smithi142 ceph-mon[32508]: pgmap v142: 148 pgs: 148 active+clean; 5.6 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.50k op/s 2024-09-20T18:52:58.334 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:58.360 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:58.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:58.361 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:58.406 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:52:58.462 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:58.486 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:58.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:58.487 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:58.521 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T18:52:58.587 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:52:58.614 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:58.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:58.615 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:52:58.716 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:58.739 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:58.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:58.740 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:58.841 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:58.865 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-20T18:52:58.865 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.0 to 1 (10000) 2024-09-20T18:52:58.905 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:58.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:58.906 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:59.007 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:59.032 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:59.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:59.033 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:52:59.118 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:58 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1602898802' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:52:59.119 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:58 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:52:59.119 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:58 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3676277443' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-20T18:52:59.119 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:58 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-20T18:52:59.119 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:58 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1602898802' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:52:59.119 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:58 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:52:59.119 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:58 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3676277443' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-20T18:52:59.119 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:58 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-20T18:52:59.134 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:59.181 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:59.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:59.182 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:59.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:58 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1602898802' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:52:59.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:58 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:52:59.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:58 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3676277443' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-20T18:52:59.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:58 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-20T18:52:59.284 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:52:59.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:59.322 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:59.322 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:52:59.423 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:52:59.453 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:59.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:59.455 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:52:59.555 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:52:59.580 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:59.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:59.582 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:52:59.683 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:52:59.710 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:59.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:59.712 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:52:59.814 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:52:59.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:59.847 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:52:59.847 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:52:59.948 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:52:59.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:52:59.977 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:52:59.977 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:00.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:59 smithi083 ceph-mon[29421]: pgmap v143: 148 pgs: 148 active+clean; 5.6 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.32k op/s 2024-09-20T18:53:00.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:59 smithi083 ceph-mon[29421]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:53:00.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:59 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T18:53:00.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:59 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-20T18:53:00.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:59 smithi083 ceph-mon[29421]: osdmap e133: 8 total, 8 up, 8 in 2024-09-20T18:53:00.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:59 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:53:00.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:52:59 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:53:00.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:59 smithi083 ceph-mon[36260]: pgmap v143: 148 pgs: 148 active+clean; 5.6 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.32k op/s 2024-09-20T18:53:00.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:59 smithi083 ceph-mon[36260]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:53:00.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:59 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T18:53:00.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:59 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-20T18:53:00.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:59 smithi083 ceph-mon[36260]: osdmap e133: 8 total, 8 up, 8 in 2024-09-20T18:53:00.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:59 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:53:00.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:52:59 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:53:00.073 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T18:53:00.073 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-20T18:53:00.073 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-20T18:53:00.073 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-20T18:53:00.074 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.074 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14158.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-20T18:53:00.074 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:47:16.702121+0000", 2024-09-20T18:53:00.074 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 342.34330842999998, 2024-09-20T18:53:00.074 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.46027232899999998, 2024-09-20T18:53:00.074 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:53:00.074 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:53:00.074 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:00.074 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:53:00.074 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14158", 2024-09-20T18:53:00.074 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2161728215", 2024-09-20T18:53:00.074 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 1 2024-09-20T18:53:00.074 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.074 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:53:00.075 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.075 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:53:00.075 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:47:16.702121+0000", 2024-09-20T18:53:00.075 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:00.075 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.075 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.075 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:53:00.075 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:47:16.702121+0000", 2024-09-20T18:53:00.075 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:00.075 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.075 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.076 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:53:00.076 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:47:16.702122+0000", 2024-09-20T18:53:00.076 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9900000000000009e-07 2024-09-20T18:53:00.076 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.076 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.076 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:53:00.076 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:47:16.702135+0000", 2024-09-20T18:53:00.076 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2087e-05 2024-09-20T18:53:00.076 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.076 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.076 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:53:00.076 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:47:16.702143+0000", 2024-09-20T18:53:00.076 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5420000000000005e-06 2024-09-20T18:53:00.076 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.077 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.077 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:53:00.077 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:47:16.702150+0000", 2024-09-20T18:53:00.077 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1910000000000003e-06 2024-09-20T18:53:00.077 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.077 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.077 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:53:00.077 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:47:16.702193+0000", 2024-09-20T18:53:00.077 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2398999999999998e-05 2024-09-20T18:53:00.077 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.077 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.077 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:53:00.077 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:47:16.702198+0000", 2024-09-20T18:53:00.077 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4009999999999996e-06 2024-09-20T18:53:00.078 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.078 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.078 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:53:00.078 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:47:17.161275+0000", 2024-09-20T18:53:00.078 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.45907645200000002 2024-09-20T18:53:00.078 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.078 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.078 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:53:00.078 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:47:17.161417+0000", 2024-09-20T18:53:00.078 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014258900000000001 2024-09-20T18:53:00.078 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.079 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.079 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,2", 2024-09-20T18:53:00.079 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:47:17.161566+0000", 2024-09-20T18:53:00.079 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014886599999999999 2024-09-20T18:53:00.079 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.079 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.079 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:53:00.079 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:47:17.162060+0000", 2024-09-20T18:53:00.079 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00049417099999999995 2024-09-20T18:53:00.079 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.079 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.079 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:00.079 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:47:17.162256+0000", 2024-09-20T18:53:00.079 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019619200000000001 2024-09-20T18:53:00.079 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.080 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.080 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:00.080 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:47:17.162303+0000", 2024-09-20T18:53:00.080 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6159999999999999e-05 2024-09-20T18:53:00.080 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.080 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.080 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:53:00.080 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:47:17.162357+0000", 2024-09-20T18:53:00.080 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4236e-05 2024-09-20T18:53:00.080 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.080 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.080 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:53:00.080 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:47:17.162394+0000", 2024-09-20T18:53:00.080 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7044e-05 2024-09-20T18:53:00.080 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:00.081 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:53:00.081 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:00.081 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.081 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.081 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:32898 7.b 7:d9277d93:::benchmark_data_smithi083_70643_object32897:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T18:53:00.081 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:11.727738+0000", 2024-09-20T18:53:00.081 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 107.317692216, 2024-09-20T18:53:00.081 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.063835517999999994, 2024-09-20T18:53:00.081 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:53:00.081 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:53:00.081 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:00.081 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:53:00.081 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:53:00.081 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:00.082 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 32898 2024-09-20T18:53:00.082 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.082 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:53:00.082 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.082 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:53:00.082 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:11.727738+0000", 2024-09-20T18:53:00.082 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:00.082 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.082 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.082 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:53:00.082 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:11.727738+0000", 2024-09-20T18:53:00.082 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:00.082 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.082 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.082 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:53:00.083 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:11.727739+0000", 2024-09-20T18:53:00.083 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7179999999999999e-06 2024-09-20T18:53:00.083 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.083 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.083 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:53:00.083 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:11.727765+0000", 2024-09-20T18:53:00.083 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5757000000000001e-05 2024-09-20T18:53:00.083 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.083 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.083 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:53:00.083 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:11.727783+0000", 2024-09-20T18:53:00.083 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8201000000000001e-05 2024-09-20T18:53:00.083 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.083 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.084 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:53:00.084 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:11.727793+0000", 2024-09-20T18:53:00.084 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2129999999999997e-06 2024-09-20T18:53:00.084 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.084 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.084 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:53:00.084 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:11.727901+0000", 2024-09-20T18:53:00.084 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000108597 2024-09-20T18:53:00.084 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.084 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.084 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:53:00.084 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:11.728021+0000", 2024-09-20T18:53:00.084 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000119924 2024-09-20T18:53:00.084 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.084 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.085 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,6", 2024-09-20T18:53:00.085 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:11.728156+0000", 2024-09-20T18:53:00.085 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000135088 2024-09-20T18:53:00.085 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.085 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.085 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:53:00.085 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:11.732685+0000", 2024-09-20T18:53:00.085 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0045288430000000003 2024-09-20T18:53:00.085 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.085 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.085 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:00.085 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:11.787480+0000", 2024-09-20T18:53:00.085 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.054795409000000003 2024-09-20T18:53:00.085 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.085 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.086 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:00.086 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:11.791490+0000", 2024-09-20T18:53:00.086 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0040093719999999998 2024-09-20T18:53:00.086 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.086 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.086 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:53:00.086 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:11.791531+0000", 2024-09-20T18:53:00.086 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1421000000000002e-05 2024-09-20T18:53:00.086 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.086 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.086 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:53:00.086 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:11.791573+0000", 2024-09-20T18:53:00.086 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1974999999999997e-05 2024-09-20T18:53:00.086 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:00.087 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:53:00.087 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:00.087 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.087 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.087 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:92646 7.b 7:de35d51d:::benchmark_data_smithi083_70643_object92645:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-20T18:53:00.087 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:56.479115+0000", 2024-09-20T18:53:00.087 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 62.566315375000002, 2024-09-20T18:53:00.087 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.054343221999999997, 2024-09-20T18:53:00.087 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:53:00.087 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:53:00.087 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:00.087 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:53:00.087 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:53:00.087 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:00.088 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 92646 2024-09-20T18:53:00.088 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.088 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:53:00.088 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.088 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:53:00.088 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:56.479115+0000", 2024-09-20T18:53:00.088 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:00.088 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.088 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.088 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:53:00.088 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:56.479115+0000", 2024-09-20T18:53:00.088 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:00.088 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.088 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.088 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:53:00.089 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:56.479116+0000", 2024-09-20T18:53:00.089 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.438e-06 2024-09-20T18:53:00.089 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.089 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.089 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:53:00.089 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:56.479136+0000", 2024-09-20T18:53:00.089 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9692999999999999e-05 2024-09-20T18:53:00.089 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.089 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.089 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:53:00.089 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:56.479143+0000", 2024-09-20T18:53:00.089 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7090000000000002e-06 2024-09-20T18:53:00.089 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.089 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.089 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:53:00.090 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:56.479149+0000", 2024-09-20T18:53:00.090 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4380000000000003e-06 2024-09-20T18:53:00.090 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.090 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.090 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:53:00.090 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:56.482052+0000", 2024-09-20T18:53:00.090 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0029035850000000002 2024-09-20T18:53:00.090 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.090 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.090 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:53:00.090 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:56.482338+0000", 2024-09-20T18:53:00.090 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00028554299999999997 2024-09-20T18:53:00.090 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.090 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.091 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,6", 2024-09-20T18:53:00.091 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:56.482723+0000", 2024-09-20T18:53:00.091 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00038478399999999999 2024-09-20T18:53:00.091 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.091 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.091 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:53:00.091 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:56.487461+0000", 2024-09-20T18:53:00.091 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0047381539999999996 2024-09-20T18:53:00.091 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.091 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.091 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:00.091 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:56.497226+0000", 2024-09-20T18:53:00.091 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0097655250000000006 2024-09-20T18:53:00.091 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.091 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.092 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:00.092 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:56.533363+0000", 2024-09-20T18:53:00.092 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.036136405000000003 2024-09-20T18:53:00.092 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.092 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.092 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:53:00.092 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:56.533417+0000", 2024-09-20T18:53:00.092 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3949000000000001e-05 2024-09-20T18:53:00.092 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.092 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.092 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:53:00.092 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:56.533458+0000", 2024-09-20T18:53:00.092 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0998999999999998e-05 2024-09-20T18:53:00.092 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:00.093 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:53:00.093 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:00.093 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.093 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.093 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:105080 7.b 7:d6e0dc28:::benchmark_data_smithi083_70643_object105079:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-20T18:53:00.093 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:52:08.621413+0000", 2024-09-20T18:53:00.093 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 50.424017175000003, 2024-09-20T18:53:00.093 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047483605999999998, 2024-09-20T18:53:00.093 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:53:00.093 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:53:00.093 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:00.093 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:53:00.093 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:53:00.093 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:00.093 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 105080 2024-09-20T18:53:00.094 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.094 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:53:00.094 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.094 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:53:00.094 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:08.621413+0000", 2024-09-20T18:53:00.094 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:00.094 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.094 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.094 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:53:00.094 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:08.621413+0000", 2024-09-20T18:53:00.094 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:00.094 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.094 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.094 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:53:00.095 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:08.621414+0000", 2024-09-20T18:53:00.095 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.719e-06 2024-09-20T18:53:00.095 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.095 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.095 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:53:00.095 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:08.621438+0000", 2024-09-20T18:53:00.095 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3796999999999999e-05 2024-09-20T18:53:00.095 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.095 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.095 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:53:00.095 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:08.621454+0000", 2024-09-20T18:53:00.096 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5543e-05 2024-09-20T18:53:00.096 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.096 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.096 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:53:00.096 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:08.621461+0000", 2024-09-20T18:53:00.096 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5460000000000003e-06 2024-09-20T18:53:00.096 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.096 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.096 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:53:00.096 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:08.621518+0000", 2024-09-20T18:53:00.096 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6320999999999998e-05 2024-09-20T18:53:00.096 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.096 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.097 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:53:00.097 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:08.621632+0000", 2024-09-20T18:53:00.097 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000114114 2024-09-20T18:53:00.097 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.097 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.097 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,6", 2024-09-20T18:53:00.097 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:08.622102+0000", 2024-09-20T18:53:00.097 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00046989200000000002 2024-09-20T18:53:00.097 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.097 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.097 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:53:00.097 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:08.625796+0000", 2024-09-20T18:53:00.097 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0036941249999999999 2024-09-20T18:53:00.098 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.098 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.098 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:00.098 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:08.639514+0000", 2024-09-20T18:53:00.098 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.013717796000000001 2024-09-20T18:53:00.098 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.098 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.098 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:00.098 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:08.668504+0000", 2024-09-20T18:53:00.098 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.028990295999999999 2024-09-20T18:53:00.098 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.098 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.098 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:53:00.098 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:08.668767+0000", 2024-09-20T18:53:00.099 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00026280299999999998 2024-09-20T18:53:00.099 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.099 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.099 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:53:00.099 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:08.668896+0000", 2024-09-20T18:53:00.099 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012965399999999999 2024-09-20T18:53:00.099 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:00.099 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:53:00.099 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:00.099 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.099 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.100 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:117478 7.b 7:dec98362:::benchmark_data_smithi083_70643_object117477:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-20T18:53:00.100 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:52:22.210131+0000", 2024-09-20T18:53:00.100 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 36.835298532000003, 2024-09-20T18:53:00.100 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047599704, 2024-09-20T18:53:00.100 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:53:00.100 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:53:00.100 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:00.100 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:53:00.100 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:53:00.100 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:00.100 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 117478 2024-09-20T18:53:00.101 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.101 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:53:00.101 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.101 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:53:00.101 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:22.210131+0000", 2024-09-20T18:53:00.101 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:00.101 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.101 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.101 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:53:00.101 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:22.210131+0000", 2024-09-20T18:53:00.101 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:00.101 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.101 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.101 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:53:00.102 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:22.210133+0000", 2024-09-20T18:53:00.102 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.46e-06 2024-09-20T18:53:00.102 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.102 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.102 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:53:00.102 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:22.210154+0000", 2024-09-20T18:53:00.102 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1254e-05 2024-09-20T18:53:00.102 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.102 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.102 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:53:00.102 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:22.210165+0000", 2024-09-20T18:53:00.102 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1202e-05 2024-09-20T18:53:00.102 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.102 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.103 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:53:00.103 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:22.210172+0000", 2024-09-20T18:53:00.103 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9480000000000002e-06 2024-09-20T18:53:00.103 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.103 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.103 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:53:00.103 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:22.211174+0000", 2024-09-20T18:53:00.103 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0010019919999999999 2024-09-20T18:53:00.103 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.103 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.103 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:53:00.103 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:22.211359+0000", 2024-09-20T18:53:00.103 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000184537 2024-09-20T18:53:00.103 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.103 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.104 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,6", 2024-09-20T18:53:00.104 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:22.211509+0000", 2024-09-20T18:53:00.104 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015046300000000001 2024-09-20T18:53:00.104 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.104 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.104 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:53:00.104 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:22.214310+0000", 2024-09-20T18:53:00.104 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002801095 2024-09-20T18:53:00.104 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.104 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.104 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:00.104 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:22.257193+0000", 2024-09-20T18:53:00.104 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.042882548999999999 2024-09-20T18:53:00.104 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.105 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.105 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:00.105 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:22.257635+0000", 2024-09-20T18:53:00.105 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00044211700000000001 2024-09-20T18:53:00.105 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.105 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.105 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:53:00.105 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:22.257688+0000", 2024-09-20T18:53:00.105 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2880000000000002e-05 2024-09-20T18:53:00.105 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.105 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.105 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:53:00.105 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:22.257731+0000", 2024-09-20T18:53:00.105 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3207000000000002e-05 2024-09-20T18:53:00.105 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:00.106 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:53:00.106 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:00.106 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.106 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.106 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:117484 7.9 e120/83 7:938deca3:::benchmark_data_smithi083_70643_object117483:head v 120'7301, mlcod=120'7299)", 2024-09-20T18:53:00.106 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:52:22.215979+0000", 2024-09-20T18:53:00.106 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 36.829450967, 2024-09-20T18:53:00.106 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047190779000000002, 2024-09-20T18:53:00.106 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:53:00.106 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:53:00.106 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:00.106 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:53:00.106 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.106 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:53:00.107 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:22.215979+0000", 2024-09-20T18:53:00.107 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:00.107 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.107 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.107 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:53:00.107 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:22.215979+0000", 2024-09-20T18:53:00.107 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:00.107 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.107 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.107 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:53:00.107 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:22.215980+0000", 2024-09-20T18:53:00.107 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.215e-06 2024-09-20T18:53:00.107 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.107 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.107 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:53:00.108 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:22.216004+0000", 2024-09-20T18:53:00.108 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3799e-05 2024-09-20T18:53:00.108 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.108 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.108 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:53:00.108 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:22.216014+0000", 2024-09-20T18:53:00.108 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0438999999999999e-05 2024-09-20T18:53:00.108 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.108 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.108 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:53:00.108 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:22.216021+0000", 2024-09-20T18:53:00.108 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0260000000000002e-06 2024-09-20T18:53:00.108 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.108 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.109 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:53:00.109 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:22.216094+0000", 2024-09-20T18:53:00.109 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2719000000000006e-05 2024-09-20T18:53:00.109 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.109 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.109 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:53:00.109 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:22.216112+0000", 2024-09-20T18:53:00.109 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7898999999999999e-05 2024-09-20T18:53:00.109 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.109 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.109 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:53:00.109 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:22.262668+0000", 2024-09-20T18:53:00.109 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046555992999999997 2024-09-20T18:53:00.109 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.109 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.110 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:53:00.110 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:22.263170+0000", 2024-09-20T18:53:00.110 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00050168900000000004 2024-09-20T18:53:00.110 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:00.110 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:53:00.110 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:00.110 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.110 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.110 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:117482 7.0 e120/83 7:0f650018:::benchmark_data_smithi083_70643_object117481:head v 120'7433, mlcod=120'7430)", 2024-09-20T18:53:00.110 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:52:22.217036+0000", 2024-09-20T18:53:00.110 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 36.828394406000001, 2024-09-20T18:53:00.110 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.050800141, 2024-09-20T18:53:00.110 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:53:00.110 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:53:00.111 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:00.111 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:53:00.111 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.111 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:53:00.111 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:22.217036+0000", 2024-09-20T18:53:00.111 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:00.111 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.111 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.111 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:53:00.111 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:22.217036+0000", 2024-09-20T18:53:00.111 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:00.111 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.111 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.111 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:53:00.111 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:22.217037+0000", 2024-09-20T18:53:00.112 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1480000000000001e-06 2024-09-20T18:53:00.112 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.112 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.112 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:53:00.112 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:22.217077+0000", 2024-09-20T18:53:00.112 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0808e-05 2024-09-20T18:53:00.112 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.112 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.112 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:53:00.112 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:22.257067+0000", 2024-09-20T18:53:00.112 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.039989477000000002 2024-09-20T18:53:00.112 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.112 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.113 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:53:00.113 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:22.257081+0000", 2024-09-20T18:53:00.113 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3611000000000001e-05 2024-09-20T18:53:00.113 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.114 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.114 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:53:00.114 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:22.257136+0000", 2024-09-20T18:53:00.114 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5658999999999997e-05 2024-09-20T18:53:00.114 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.114 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.114 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:53:00.114 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:22.257155+0000", 2024-09-20T18:53:00.114 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9086999999999999e-05 2024-09-20T18:53:00.114 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.114 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.114 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:53:00.114 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:22.267638+0000", 2024-09-20T18:53:00.115 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010483164 2024-09-20T18:53:00.115 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.115 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.115 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:53:00.115 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:22.267836+0000", 2024-09-20T18:53:00.115 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019718700000000001 2024-09-20T18:53:00.115 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:00.115 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:53:00.115 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:00.115 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.115 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.115 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:120191 7.b 7:d45eee6a:::benchmark_data_smithi083_70643_object120190:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T18:53:00.115 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:52:24.970674+0000", 2024-09-20T18:53:00.115 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 34.074755768000003, 2024-09-20T18:53:00.116 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.115151315, 2024-09-20T18:53:00.116 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:53:00.116 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:53:00.116 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:00.116 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:53:00.116 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:53:00.116 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:00.116 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 120191 2024-09-20T18:53:00.116 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.116 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:53:00.116 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.116 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:53:00.116 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.970674+0000", 2024-09-20T18:53:00.116 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:00.117 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.117 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.117 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:53:00.117 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.970674+0000", 2024-09-20T18:53:00.117 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:00.117 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.117 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.117 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:53:00.117 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.970676+0000", 2024-09-20T18:53:00.117 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.796e-06 2024-09-20T18:53:00.117 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.117 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.117 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:53:00.117 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.970706+0000", 2024-09-20T18:53:00.118 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9938e-05 2024-09-20T18:53:00.118 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.118 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.118 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:53:00.118 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.970721+0000", 2024-09-20T18:53:00.118 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4657999999999999e-05 2024-09-20T18:53:00.118 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.118 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.118 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:53:00.118 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.970729+0000", 2024-09-20T18:53:00.118 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.994e-06 2024-09-20T18:53:00.118 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.118 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.118 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:53:00.119 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.970786+0000", 2024-09-20T18:53:00.119 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7113e-05 2024-09-20T18:53:00.119 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.119 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.119 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:53:00.119 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.971178+0000", 2024-09-20T18:53:00.119 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00039247099999999998 2024-09-20T18:53:00.119 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.119 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.119 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,6", 2024-09-20T18:53:00.119 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.971619+0000", 2024-09-20T18:53:00.119 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000440956 2024-09-20T18:53:00.119 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.119 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.120 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:00.120 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.975170+0000", 2024-09-20T18:53:00.120 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0035509000000000001 2024-09-20T18:53:00.120 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.120 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.120 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:53:00.120 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.975661+0000", 2024-09-20T18:53:00.120 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00049051900000000002 2024-09-20T18:53:00.120 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.120 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.120 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:00.120 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.085675+0000", 2024-09-20T18:53:00.120 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.110014127 2024-09-20T18:53:00.120 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.121 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.121 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:53:00.121 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.085748+0000", 2024-09-20T18:53:00.121 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3512999999999999e-05 2024-09-20T18:53:00.121 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.121 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.121 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:53:00.121 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.085825+0000", 2024-09-20T18:53:00.121 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7330000000000007e-05 2024-09-20T18:53:00.121 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:00.121 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:53:00.121 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:00.121 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.121 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.121 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:120201 7.b 7:d1c07770:::benchmark_data_smithi083_70643_object120200:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T18:53:00.122 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:52:24.982656+0000", 2024-09-20T18:53:00.122 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 34.062773854, 2024-09-20T18:53:00.122 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.103339863, 2024-09-20T18:53:00.122 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:53:00.122 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:53:00.122 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:00.122 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:53:00.122 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:53:00.122 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:00.122 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 120201 2024-09-20T18:53:00.122 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.122 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:53:00.122 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.123 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:53:00.123 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.982656+0000", 2024-09-20T18:53:00.123 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:00.123 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.123 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.123 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:53:00.123 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.982656+0000", 2024-09-20T18:53:00.123 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:00.123 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.123 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.123 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:53:00.123 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.982658+0000", 2024-09-20T18:53:00.123 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.77e-06 2024-09-20T18:53:00.123 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.123 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.124 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:53:00.124 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.982682+0000", 2024-09-20T18:53:00.124 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4309999999999999e-05 2024-09-20T18:53:00.124 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.124 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.124 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:53:00.124 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.982698+0000", 2024-09-20T18:53:00.124 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5389000000000001e-05 2024-09-20T18:53:00.124 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.124 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.124 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:53:00.124 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.982705+0000", 2024-09-20T18:53:00.124 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9019999999999999e-06 2024-09-20T18:53:00.124 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.125 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.125 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:53:00.125 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.982858+0000", 2024-09-20T18:53:00.125 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015291400000000001 2024-09-20T18:53:00.125 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.125 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.125 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:53:00.125 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.983315+0000", 2024-09-20T18:53:00.125 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000456563 2024-09-20T18:53:00.125 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.125 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.125 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,6", 2024-09-20T18:53:00.126 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.983727+0000", 2024-09-20T18:53:00.126 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00041162000000000002 2024-09-20T18:53:00.126 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.126 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.126 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:00.126 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.993189+0000", 2024-09-20T18:53:00.126 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0094625750000000008 2024-09-20T18:53:00.126 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.126 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.126 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:53:00.126 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.993200+0000", 2024-09-20T18:53:00.126 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0866000000000001e-05 2024-09-20T18:53:00.126 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.126 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.127 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:00.127 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.085878+0000", 2024-09-20T18:53:00.127 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.092678153999999999 2024-09-20T18:53:00.127 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.127 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.127 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:53:00.127 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.085938+0000", 2024-09-20T18:53:00.127 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9991999999999998e-05 2024-09-20T18:53:00.127 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.127 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.127 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:53:00.127 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.085996+0000", 2024-09-20T18:53:00.127 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7807999999999999e-05 2024-09-20T18:53:00.127 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:00.128 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:53:00.128 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:00.128 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.128 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.128 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:120205 7.b 7:d694ab9d:::benchmark_data_smithi083_70643_object120204:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T18:53:00.128 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:52:24.983125+0000", 2024-09-20T18:53:00.128 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 34.062305283999997, 2024-09-20T18:53:00.128 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.10302963699999999, 2024-09-20T18:53:00.128 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:53:00.128 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:53:00.128 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:00.128 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:53:00.128 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:53:00.128 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:00.129 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 120205 2024-09-20T18:53:00.129 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.129 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:53:00.129 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.129 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:53:00.129 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.983125+0000", 2024-09-20T18:53:00.129 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:00.129 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.129 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.129 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:53:00.129 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.983125+0000", 2024-09-20T18:53:00.129 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:00.129 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.129 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.130 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:53:00.130 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.983126+0000", 2024-09-20T18:53:00.130 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6899999999999996e-07 2024-09-20T18:53:00.130 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.130 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.130 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:53:00.130 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.983153+0000", 2024-09-20T18:53:00.130 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7821e-05 2024-09-20T18:53:00.130 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.130 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.130 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:53:00.130 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.983165+0000", 2024-09-20T18:53:00.130 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1443999999999999e-05 2024-09-20T18:53:00.130 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.131 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.131 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:53:00.131 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.983172+0000", 2024-09-20T18:53:00.131 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4070000000000004e-06 2024-09-20T18:53:00.131 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.131 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.131 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:53:00.131 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.989732+0000", 2024-09-20T18:53:00.131 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00656012 2024-09-20T18:53:00.131 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.131 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.131 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:53:00.131 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.989873+0000", 2024-09-20T18:53:00.132 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000141011 2024-09-20T18:53:00.132 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.132 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.132 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,6", 2024-09-20T18:53:00.132 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.990287+0000", 2024-09-20T18:53:00.132 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00041316299999999997 2024-09-20T18:53:00.132 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.132 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.132 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:53:00.132 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.993386+0000", 2024-09-20T18:53:00.132 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0030997239999999999 2024-09-20T18:53:00.133 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.133 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.133 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:00.133 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.995564+0000", 2024-09-20T18:53:00.133 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0021777810000000002 2024-09-20T18:53:00.133 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.133 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.133 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:00.133 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.086036+0000", 2024-09-20T18:53:00.133 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090471996999999998 2024-09-20T18:53:00.133 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.133 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.133 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:53:00.133 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.086120+0000", 2024-09-20T18:53:00.134 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4123999999999996e-05 2024-09-20T18:53:00.134 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.134 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.134 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:53:00.134 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.086154+0000", 2024-09-20T18:53:00.134 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4075999999999997e-05 2024-09-20T18:53:00.134 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:00.134 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:53:00.134 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:00.134 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.134 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.134 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:120233 7.b 7:d0b1697a:::benchmark_data_smithi083_70643_object120232:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T18:53:00.134 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:52:25.037816+0000", 2024-09-20T18:53:00.135 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 34.007613814999999, 2024-09-20T18:53:00.135 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048489325999999999, 2024-09-20T18:53:00.135 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:53:00.135 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:53:00.135 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:00.135 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:53:00.135 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:53:00.135 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:00.135 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 120233 2024-09-20T18:53:00.135 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.135 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:53:00.135 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.135 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:53:00.135 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.037816+0000", 2024-09-20T18:53:00.136 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:00.136 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.136 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.136 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:53:00.136 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.037816+0000", 2024-09-20T18:53:00.136 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:00.136 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.136 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.136 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:53:00.136 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.037818+0000", 2024-09-20T18:53:00.136 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0679999999999999e-06 2024-09-20T18:53:00.136 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.136 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.136 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:53:00.137 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.037845+0000", 2024-09-20T18:53:00.137 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6690999999999999e-05 2024-09-20T18:53:00.137 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.137 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.137 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:53:00.137 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.037872+0000", 2024-09-20T18:53:00.137 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7430999999999999e-05 2024-09-20T18:53:00.137 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.137 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.137 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:53:00.137 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.037883+0000", 2024-09-20T18:53:00.137 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0886e-05 2024-09-20T18:53:00.137 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.137 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.137 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:53:00.138 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.038034+0000", 2024-09-20T18:53:00.138 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000150421 2024-09-20T18:53:00.138 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.138 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.138 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:53:00.138 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.038175+0000", 2024-09-20T18:53:00.138 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014172800000000001 2024-09-20T18:53:00.138 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.138 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.138 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,6", 2024-09-20T18:53:00.138 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.038331+0000", 2024-09-20T18:53:00.138 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015552299999999999 2024-09-20T18:53:00.138 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.138 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.139 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:53:00.139 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.040408+0000", 2024-09-20T18:53:00.139 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0020766690000000002 2024-09-20T18:53:00.139 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.139 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.139 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:00.139 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.040684+0000", 2024-09-20T18:53:00.139 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00027634699999999998 2024-09-20T18:53:00.139 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.139 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.139 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:00.139 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.086196+0000", 2024-09-20T18:53:00.139 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.045511919999999997 2024-09-20T18:53:00.139 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.140 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.140 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:53:00.140 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.086251+0000", 2024-09-20T18:53:00.140 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5519000000000002e-05 2024-09-20T18:53:00.140 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.140 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.140 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:53:00.140 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.086305+0000", 2024-09-20T18:53:00.140 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4122999999999997e-05 2024-09-20T18:53:00.140 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:00.140 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:53:00.140 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:00.140 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.140 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.141 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:120774 7.b 7:d6f6bab8:::benchmark_data_smithi083_70643_object120773:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T18:53:00.141 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:52:25.569877+0000", 2024-09-20T18:53:00.141 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 33.475552725, 2024-09-20T18:53:00.141 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.052170454999999998, 2024-09-20T18:53:00.141 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:53:00.141 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:53:00.141 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:00.141 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:53:00.141 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:53:00.141 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:00.141 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 120774 2024-09-20T18:53:00.141 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.141 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:53:00.141 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.141 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:53:00.142 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.569877+0000", 2024-09-20T18:53:00.142 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:00.142 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.142 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.142 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:53:00.142 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.569877+0000", 2024-09-20T18:53:00.142 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:00.142 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.142 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.142 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:53:00.142 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.569879+0000", 2024-09-20T18:53:00.142 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6109999999999999e-06 2024-09-20T18:53:00.142 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.143 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.143 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:53:00.143 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.569908+0000", 2024-09-20T18:53:00.143 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9535000000000001e-05 2024-09-20T18:53:00.143 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.143 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.143 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:53:00.143 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.569920+0000", 2024-09-20T18:53:00.143 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1819e-05 2024-09-20T18:53:00.143 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.143 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.143 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:53:00.143 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.569928+0000", 2024-09-20T18:53:00.143 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4399999999999999e-06 2024-09-20T18:53:00.144 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.144 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.144 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:53:00.144 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.576819+0000", 2024-09-20T18:53:00.144 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0068915579999999999 2024-09-20T18:53:00.144 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.144 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.144 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:53:00.144 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.577295+0000", 2024-09-20T18:53:00.144 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00047626199999999997 2024-09-20T18:53:00.144 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.144 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.144 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,6", 2024-09-20T18:53:00.145 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.577754+0000", 2024-09-20T18:53:00.145 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00045848999999999998 2024-09-20T18:53:00.145 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.145 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.145 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:00.145 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.582076+0000", 2024-09-20T18:53:00.145 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0043223180000000003 2024-09-20T18:53:00.145 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.145 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.145 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:53:00.145 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.582835+0000", 2024-09-20T18:53:00.145 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00075891299999999995 2024-09-20T18:53:00.145 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.145 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.145 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:00.146 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.621947+0000", 2024-09-20T18:53:00.146 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.039111817 2024-09-20T18:53:00.146 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.146 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.146 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:53:00.146 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.622001+0000", 2024-09-20T18:53:00.146 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4358999999999999e-05 2024-09-20T18:53:00.146 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.146 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.146 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:53:00.146 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.622048+0000", 2024-09-20T18:53:00.146 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6332999999999998e-05 2024-09-20T18:53:00.146 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:00.147 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:53:00.147 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:00.147 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.147 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.147 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:121026 7.b 7:dc01920f:::benchmark_data_smithi083_70643_object121025:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-20T18:53:00.147 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:52:25.797178+0000", 2024-09-20T18:53:00.147 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 33.248252032000003, 2024-09-20T18:53:00.147 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046932834999999999, 2024-09-20T18:53:00.147 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:53:00.147 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:53:00.147 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:00.147 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:53:00.147 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:53:00.147 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:00.148 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 121026 2024-09-20T18:53:00.148 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.148 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:53:00.148 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.148 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:53:00.148 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.797178+0000", 2024-09-20T18:53:00.148 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:00.148 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.148 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.148 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:53:00.148 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.797178+0000", 2024-09-20T18:53:00.148 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:00.148 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.148 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.149 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:53:00.149 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.797179+0000", 2024-09-20T18:53:00.149 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3289999999999999e-06 2024-09-20T18:53:00.149 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.149 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.149 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:53:00.149 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.797220+0000", 2024-09-20T18:53:00.149 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0911000000000002e-05 2024-09-20T18:53:00.149 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.149 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.149 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:53:00.149 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.797232+0000", 2024-09-20T18:53:00.149 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1389999999999999e-05 2024-09-20T18:53:00.149 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.149 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.150 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:53:00.150 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.797239+0000", 2024-09-20T18:53:00.150 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5580000000000003e-06 2024-09-20T18:53:00.150 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.150 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.150 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:53:00.150 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.797350+0000", 2024-09-20T18:53:00.150 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011114 2024-09-20T18:53:00.150 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.150 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.150 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:53:00.150 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.797586+0000", 2024-09-20T18:53:00.150 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000235496 2024-09-20T18:53:00.150 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.151 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.151 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,6", 2024-09-20T18:53:00.151 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.797741+0000", 2024-09-20T18:53:00.151 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015518500000000001 2024-09-20T18:53:00.151 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.151 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.151 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:00.151 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.800530+0000", 2024-09-20T18:53:00.151 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002789494 2024-09-20T18:53:00.151 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.151 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.151 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:53:00.151 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.800544+0000", 2024-09-20T18:53:00.151 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.403e-05 2024-09-20T18:53:00.152 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.152 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.152 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:00.152 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.843832+0000", 2024-09-20T18:53:00.152 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043287436999999998 2024-09-20T18:53:00.152 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.152 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.152 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:53:00.152 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.844065+0000", 2024-09-20T18:53:00.152 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00023329399999999999 2024-09-20T18:53:00.152 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.152 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.152 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:53:00.152 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.844111+0000", 2024-09-20T18:53:00.152 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5571000000000001e-05 2024-09-20T18:53:00.153 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:00.153 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:53:00.153 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:00.153 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.153 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.153 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:134371 7.9 e126/83 7:950d90e8:::benchmark_data_smithi083_70643_object11569:head v 126'8357, mlcod=126'8353)", 2024-09-20T18:53:00.153 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:52:38.343657+0000", 2024-09-20T18:53:00.153 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 20.701772731999998, 2024-09-20T18:53:00.153 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.054387652000000002, 2024-09-20T18:53:00.153 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:53:00.153 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:53:00.153 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:00.153 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:53:00.153 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.154 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:53:00.154 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:38.343657+0000", 2024-09-20T18:53:00.154 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:00.154 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.154 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.154 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:53:00.154 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:38.343657+0000", 2024-09-20T18:53:00.154 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:00.154 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.154 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.154 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:53:00.154 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:38.343658+0000", 2024-09-20T18:53:00.154 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.009e-06 2024-09-20T18:53:00.154 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.154 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.155 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:53:00.155 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:38.343664+0000", 2024-09-20T18:53:00.155 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9510000000000003e-06 2024-09-20T18:53:00.155 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.155 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.155 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:53:00.155 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:38.343673+0000", 2024-09-20T18:53:00.155 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2890000000000003e-06 2024-09-20T18:53:00.155 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.155 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.155 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:53:00.155 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:38.343681+0000", 2024-09-20T18:53:00.155 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3470000000000001e-06 2024-09-20T18:53:00.155 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.156 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.156 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:53:00.156 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:38.343744+0000", 2024-09-20T18:53:00.156 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3058000000000005e-05 2024-09-20T18:53:00.156 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.156 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.156 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:53:00.156 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:38.343763+0000", 2024-09-20T18:53:00.156 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8692999999999999e-05 2024-09-20T18:53:00.156 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.156 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.156 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:53:00.156 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:38.398007+0000", 2024-09-20T18:53:00.156 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.054244011000000002 2024-09-20T18:53:00.157 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.157 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.157 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:53:00.157 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:38.398045+0000", 2024-09-20T18:53:00.157 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8294000000000003e-05 2024-09-20T18:53:00.157 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:00.157 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:53:00.157 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:00.157 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.157 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.157 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:134384 7.9 e126/83 7:9c4bca2c:::benchmark_data_smithi083_70643_object11582:head v 126'8358, mlcod=126'8353)", 2024-09-20T18:53:00.157 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:52:38.348874+0000", 2024-09-20T18:53:00.157 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 20.696555741000001, 2024-09-20T18:53:00.157 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049201582000000001, 2024-09-20T18:53:00.158 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:53:00.158 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:53:00.158 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:00.158 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:53:00.158 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.158 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:53:00.158 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:38.348874+0000", 2024-09-20T18:53:00.158 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:00.158 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.158 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.158 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:53:00.158 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:38.348874+0000", 2024-09-20T18:53:00.158 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:00.158 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.158 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.159 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:53:00.159 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:38.348875+0000", 2024-09-20T18:53:00.159 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3099999999999997e-07 2024-09-20T18:53:00.159 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.159 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.159 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:53:00.159 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:38.348879+0000", 2024-09-20T18:53:00.159 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5070000000000002e-06 2024-09-20T18:53:00.159 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.159 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.159 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:53:00.159 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:38.348885+0000", 2024-09-20T18:53:00.159 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.519e-06 2024-09-20T18:53:00.159 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.160 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.160 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:53:00.160 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:38.348891+0000", 2024-09-20T18:53:00.160 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0519999999999998e-06 2024-09-20T18:53:00.160 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.160 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.160 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:53:00.160 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:38.349590+0000", 2024-09-20T18:53:00.160 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00069920900000000001 2024-09-20T18:53:00.160 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.160 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.160 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:53:00.160 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:38.349608+0000", 2024-09-20T18:53:00.160 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8031999999999999e-05 2024-09-20T18:53:00.160 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.161 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.161 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:53:00.161 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:38.398052+0000", 2024-09-20T18:53:00.161 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048443704999999997 2024-09-20T18:53:00.161 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.161 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.161 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:53:00.161 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:38.398076+0000", 2024-09-20T18:53:00.161 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3827000000000001e-05 2024-09-20T18:53:00.161 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:00.161 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:53:00.161 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:00.161 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.161 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.162 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:134387 7.9 e126/83 7:952e5dc0:::benchmark_data_smithi083_70643_object11585:head v 126'8359, mlcod=126'8353)", 2024-09-20T18:53:00.162 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:52:38.349274+0000", 2024-09-20T18:53:00.162 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 20.696156049999999, 2024-09-20T18:53:00.162 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.048826321999999998, 2024-09-20T18:53:00.162 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:53:00.162 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:53:00.162 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:00.162 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:53:00.162 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.162 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:53:00.162 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:38.349274+0000", 2024-09-20T18:53:00.162 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:00.162 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.162 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.163 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:53:00.163 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:38.349274+0000", 2024-09-20T18:53:00.163 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:00.163 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.163 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.163 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:53:00.163 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:38.349275+0000", 2024-09-20T18:53:00.163 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0999999999999998e-07 2024-09-20T18:53:00.163 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.163 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.163 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:53:00.163 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:38.349280+0000", 2024-09-20T18:53:00.163 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.254e-06 2024-09-20T18:53:00.163 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.164 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.164 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:53:00.164 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:38.349289+0000", 2024-09-20T18:53:00.164 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4229999999999994e-06 2024-09-20T18:53:00.164 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.164 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.164 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:53:00.164 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:38.349296+0000", 2024-09-20T18:53:00.164 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5899999999999996e-06 2024-09-20T18:53:00.164 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.164 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.164 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:53:00.164 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:38.353105+0000", 2024-09-20T18:53:00.164 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0038087759999999998 2024-09-20T18:53:00.165 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.165 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.165 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:53:00.165 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:38.353126+0000", 2024-09-20T18:53:00.165 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1131000000000001e-05 2024-09-20T18:53:00.165 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.165 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.165 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:53:00.165 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:38.398077+0000", 2024-09-20T18:53:00.165 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044951725999999997 2024-09-20T18:53:00.165 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.165 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.165 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:53:00.165 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:38.398100+0000", 2024-09-20T18:53:00.165 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2711999999999999e-05 2024-09-20T18:53:00.166 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:00.166 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:53:00.166 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:00.166 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.166 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.166 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:134395 7.9 e126/83 7:984bdcbc:::benchmark_data_smithi083_70643_object11593:head v 126'8360, mlcod=126'8353)", 2024-09-20T18:53:00.166 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:52:38.354411+0000", 2024-09-20T18:53:00.166 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 20.691018661000001, 2024-09-20T18:53:00.166 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.049069491, 2024-09-20T18:53:00.166 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:53:00.166 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:53:00.166 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:00.166 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:53:00.166 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.167 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:53:00.167 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:38.354411+0000", 2024-09-20T18:53:00.167 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:00.167 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.167 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.167 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:53:00.167 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:38.354411+0000", 2024-09-20T18:53:00.167 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:00.167 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.167 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.167 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:53:00.167 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:38.354412+0000", 2024-09-20T18:53:00.167 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.47e-07 2024-09-20T18:53:00.167 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.167 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.168 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:53:00.168 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:38.354419+0000", 2024-09-20T18:53:00.168 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.37e-06 2024-09-20T18:53:00.168 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.168 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.168 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:53:00.168 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:38.354428+0000", 2024-09-20T18:53:00.168 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.628e-06 2024-09-20T18:53:00.168 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.168 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.168 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:53:00.168 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:38.354435+0000", 2024-09-20T18:53:00.168 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9990000000000002e-06 2024-09-20T18:53:00.168 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.168 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.169 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:53:00.169 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:38.356276+0000", 2024-09-20T18:53:00.169 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0018410690000000001 2024-09-20T18:53:00.169 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.169 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.169 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:53:00.169 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:38.356292+0000", 2024-09-20T18:53:00.169 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6113999999999999e-05 2024-09-20T18:53:00.169 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.169 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.169 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:53:00.169 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:38.403448+0000", 2024-09-20T18:53:00.169 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047155322999999999 2024-09-20T18:53:00.169 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.170 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.170 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:53:00.170 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:38.403481+0000", 2024-09-20T18:53:00.170 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3040999999999998e-05 2024-09-20T18:53:00.170 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:00.170 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:53:00.170 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:00.170 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.170 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.170 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:134401 7.9 e126/83 7:9243da2e:::benchmark_data_smithi083_70643_object11599:head v 126'8361, mlcod=126'8355)", 2024-09-20T18:53:00.170 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:52:38.357412+0000", 2024-09-20T18:53:00.170 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 20.688017765000001, 2024-09-20T18:53:00.170 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.046092116000000002, 2024-09-20T18:53:00.170 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:53:00.171 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:53:00.171 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:00.171 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:53:00.171 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.171 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:53:00.171 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:38.357412+0000", 2024-09-20T18:53:00.171 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:00.171 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.171 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.171 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:53:00.171 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:38.357412+0000", 2024-09-20T18:53:00.171 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:00.171 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.171 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.172 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:53:00.172 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:38.357413+0000", 2024-09-20T18:53:00.172 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8400000000000004e-07 2024-09-20T18:53:00.172 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.172 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.172 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:53:00.172 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:38.357419+0000", 2024-09-20T18:53:00.172 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0959999999999997e-06 2024-09-20T18:53:00.172 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.172 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.172 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:53:00.172 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:38.357427+0000", 2024-09-20T18:53:00.172 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5019999999999992e-06 2024-09-20T18:53:00.172 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.172 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.173 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:53:00.173 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:38.357433+0000", 2024-09-20T18:53:00.173 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7350000000000001e-06 2024-09-20T18:53:00.173 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.173 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.173 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:53:00.173 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:38.359236+0000", 2024-09-20T18:53:00.173 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0018029439999999999 2024-09-20T18:53:00.173 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.173 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.173 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:53:00.173 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:38.359257+0000", 2024-09-20T18:53:00.173 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1262000000000001e-05 2024-09-20T18:53:00.173 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.174 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.174 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:53:00.174 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:38.403485+0000", 2024-09-20T18:53:00.174 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044227305000000001 2024-09-20T18:53:00.174 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.174 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.174 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:53:00.174 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:38.403504+0000", 2024-09-20T18:53:00.174 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9587999999999999e-05 2024-09-20T18:53:00.174 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:00.174 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:53:00.174 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:00.174 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.174 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.175 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:140134 7.b 7:d3ef5ce1:::benchmark_data_smithi083_70643_object17332:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-20T18:53:00.175 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:52:43.833953+0000", 2024-09-20T18:53:00.175 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.211477205, 2024-09-20T18:53:00.175 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047263837000000003, 2024-09-20T18:53:00.175 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:53:00.175 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:53:00.175 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:00.175 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:53:00.176 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:53:00.176 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:00.176 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 140134 2024-09-20T18:53:00.176 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.176 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:53:00.176 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.176 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:53:00.176 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:43.833953+0000", 2024-09-20T18:53:00.176 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:00.176 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.176 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.176 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:53:00.176 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:43.833953+0000", 2024-09-20T18:53:00.176 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:00.177 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.177 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.177 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:53:00.177 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:43.833954+0000", 2024-09-20T18:53:00.177 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2160000000000001e-06 2024-09-20T18:53:00.177 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.177 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.177 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:53:00.177 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:43.833959+0000", 2024-09-20T18:53:00.177 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9239999999999998e-06 2024-09-20T18:53:00.177 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.177 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.177 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:53:00.177 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:43.833970+0000", 2024-09-20T18:53:00.177 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0886999999999999e-05 2024-09-20T18:53:00.178 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.178 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.178 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:53:00.178 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:43.833977+0000", 2024-09-20T18:53:00.178 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4719999999999998e-06 2024-09-20T18:53:00.178 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.178 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.178 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:53:00.178 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:43.835345+0000", 2024-09-20T18:53:00.178 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0013679580000000001 2024-09-20T18:53:00.178 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.178 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.178 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:53:00.178 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:43.835445+0000", 2024-09-20T18:53:00.179 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9331000000000002e-05 2024-09-20T18:53:00.179 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.179 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.179 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,6", 2024-09-20T18:53:00.179 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:43.835549+0000", 2024-09-20T18:53:00.179 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000104174 2024-09-20T18:53:00.179 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.179 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.179 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:53:00.179 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:43.840595+0000", 2024-09-20T18:53:00.179 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0050465689999999999 2024-09-20T18:53:00.179 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.179 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.180 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:00.180 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:43.878517+0000", 2024-09-20T18:53:00.180 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.037921786999999998 2024-09-20T18:53:00.180 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.180 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.180 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:00.180 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:43.878582+0000", 2024-09-20T18:53:00.180 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5117999999999996e-05 2024-09-20T18:53:00.180 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.180 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.180 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:53:00.180 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:43.881166+0000", 2024-09-20T18:53:00.180 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0025842560000000001 2024-09-20T18:53:00.180 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.180 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.181 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:53:00.181 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:43.881217+0000", 2024-09-20T18:53:00.181 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0145000000000002e-05 2024-09-20T18:53:00.181 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:00.181 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:53:00.181 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:00.181 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.181 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.181 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:140137 7.b 7:d4106c75:::benchmark_data_smithi083_70643_object17335:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-20T18:53:00.181 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:52:43.834463+0000", 2024-09-20T18:53:00.181 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.210967408, 2024-09-20T18:53:00.181 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.047145975999999999, 2024-09-20T18:53:00.181 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:53:00.181 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:53:00.181 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:00.182 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:53:00.182 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:53:00.182 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:00.182 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 140137 2024-09-20T18:53:00.182 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.182 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:53:00.182 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.182 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:53:00.182 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:43.834463+0000", 2024-09-20T18:53:00.182 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:00.182 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.182 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.182 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:53:00.183 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:43.834463+0000", 2024-09-20T18:53:00.183 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:00.183 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.183 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.183 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:53:00.183 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:43.834463+0000", 2024-09-20T18:53:00.183 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.71e-07 2024-09-20T18:53:00.183 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.183 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.183 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:53:00.183 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:43.834468+0000", 2024-09-20T18:53:00.183 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7230000000000004e-06 2024-09-20T18:53:00.183 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.183 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.183 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:53:00.184 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:43.834478+0000", 2024-09-20T18:53:00.184 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5750000000000007e-06 2024-09-20T18:53:00.184 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.184 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.184 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:53:00.184 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:43.834485+0000", 2024-09-20T18:53:00.184 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3799999999999996e-06 2024-09-20T18:53:00.184 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.184 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.184 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:53:00.184 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:43.838021+0000", 2024-09-20T18:53:00.184 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003536094 2024-09-20T18:53:00.184 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.184 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.185 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:53:00.185 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:43.838129+0000", 2024-09-20T18:53:00.185 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010787500000000001 2024-09-20T18:53:00.185 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.185 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.185 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,6", 2024-09-20T18:53:00.185 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:43.838237+0000", 2024-09-20T18:53:00.185 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000108391 2024-09-20T18:53:00.185 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.185 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.185 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:53:00.185 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:43.878530+0000", 2024-09-20T18:53:00.185 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.040292399 2024-09-20T18:53:00.185 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.185 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.186 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:00.186 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:43.881263+0000", 2024-09-20T18:53:00.186 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0027332989999999998 2024-09-20T18:53:00.186 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.186 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.186 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:00.186 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:43.881305+0000", 2024-09-20T18:53:00.186 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1774999999999999e-05 2024-09-20T18:53:00.186 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.186 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.186 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:53:00.186 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:43.881572+0000", 2024-09-20T18:53:00.186 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000267125 2024-09-20T18:53:00.187 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:00.187 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:00.187 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:53:00.187 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:43.881608+0000", 2024-09-20T18:53:00.187 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6468999999999999e-05 2024-09-20T18:53:00.187 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:00.187 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:53:00.187 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:00.187 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:00.187 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:53:00.187 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T18:53:00.187 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:00.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:00.240 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:00.240 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:00.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:59 smithi142 ceph-mon[32508]: pgmap v143: 148 pgs: 148 active+clean; 5.6 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.32k op/s 2024-09-20T18:53:00.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:59 smithi142 ceph-mon[32508]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:53:00.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:59 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T18:53:00.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:59 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-20T18:53:00.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:59 smithi142 ceph-mon[32508]: osdmap e133: 8 total, 8 up, 8 in 2024-09-20T18:53:00.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:59 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:53:00.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:52:59 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:53:00.342 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:00.368 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:00.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:00.369 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:00.470 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:53:00.510 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:00.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:00.512 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:53:00.613 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:00.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:00.648 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:00.648 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:00.749 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:00.774 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:00.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:00.775 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:00.876 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:00.903 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:00.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:00.904 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:01.004 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:53:01.046 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:01.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:01.047 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:53:01.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:00 smithi083 ceph-mon[29421]: osdmap e134: 8 total, 8 up, 8 in 2024-09-20T18:53:01.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:00 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:53:01.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:00 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:53:01.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:00 smithi083 ceph-mon[36260]: osdmap e134: 8 total, 8 up, 8 in 2024-09-20T18:53:01.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:00 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:53:01.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:00 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:53:01.055 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:53:00 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:53:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:53:01.147 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:01.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:01.174 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:01.174 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:01.256 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-20T18:53:01.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:00 smithi142 ceph-mon[32508]: osdmap e134: 8 total, 8 up, 8 in 2024-09-20T18:53:01.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:00 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:53:01.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:00 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:53:01.274 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:01.301 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:01.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:01.302 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:01.403 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:53:01.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:01.441 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:01.441 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:53:01.542 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:01.566 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:01.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:01.567 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:01.668 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:01.693 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:01.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:01.694 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:01.795 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:01.827 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:01.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:01.828 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:01.929 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:01.948 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd reweight 1 1 2024-09-20T18:53:01.959 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:53:01.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:01.975 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:01.975 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:02.077 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:02.101 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:02.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:02.102 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:02.203 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:02.229 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:02.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:02.230 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:02.331 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:02.356 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:02.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:02.357 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:02.458 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:02.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:02.498 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:02.498 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:02.544 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:53:02.597 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:53:02.597 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:53:02.598 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:02.622 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:02.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:02.623 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:02.643 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:53:02.643 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:53:02.683 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:53:02.683 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:53:02.719 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:53:02.719 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:53:02.724 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:02.749 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:53:02.749 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:53:02.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:02.776 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:02.777 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:02.784 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:53:02.784 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:53:02.792 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T18:53:02.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:02 smithi083 ceph-mon[29421]: pgmap v146: 148 pgs: 148 active+clean; 5.6 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 795 op/s 2024-09-20T18:53:02.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:02 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2654183365' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:53:02.805 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:02 smithi083 ceph-mon[36260]: pgmap v146: 148 pgs: 148 active+clean; 5.6 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 795 op/s 2024-09-20T18:53:02.805 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:02 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2654183365' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:53:02.820 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:53:02.821 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:53:02.854 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:53:02.854 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:53:02.878 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:02.904 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:02.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:02.904 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:03.005 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:03.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:02 smithi142 ceph-mon[32508]: pgmap v146: 148 pgs: 148 active+clean; 5.6 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 795 op/s 2024-09-20T18:53:03.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:02 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2654183365' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:53:03.046 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:03.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:03.047 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:03.148 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:03.176 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:03.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:03.177 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:03.278 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:03.306 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:03.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:03.307 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:03.352 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:53:03.383 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T18:53:03.383 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:53:03.407 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:53:03.433 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:03.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:03.434 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:53:03.534 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:53:03.561 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:03.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:03.562 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:53:03.663 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:03.702 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:03.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:03.703 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:03.804 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:03.847 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:03.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:03.848 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:03.948 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:03.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:03.991 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:03.991 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:04.092 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:04.115 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:04.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:04.116 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:04.216 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:53:04.244 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:04.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:04.245 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:53:04.346 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:04.371 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:04.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:04.372 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:04.473 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:04.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:04.526 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:04.526 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:04.536 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_blocked_ops 2024-09-20T18:53:04.627 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:04.655 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:04.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:04.656 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:04.756 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:53:04.783 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:04.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:04.784 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:53:04.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:04 smithi083 ceph-mon[29421]: pgmap v147: 148 pgs: 148 active+clean; 5.3 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.34k op/s 2024-09-20T18:53:04.807 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:04 smithi083 ceph-mon[36260]: pgmap v147: 148 pgs: 148 active+clean; 5.3 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.34k op/s 2024-09-20T18:53:04.885 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:04.908 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:04.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:04.909 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:05.009 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:05.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:04 smithi142 ceph-mon[32508]: pgmap v147: 148 pgs: 148 active+clean; 5.3 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.34k op/s 2024-09-20T18:53:05.035 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:05.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:05.036 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:05.136 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:05.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:05.163 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:05.163 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:05.264 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:05.290 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:05.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:05.291 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:05.392 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:53:05.430 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:05.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:05.432 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:53:05.533 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:53:05.560 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:05.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:05.561 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:53:05.662 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:53:05.689 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:05.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:05.690 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:53:05.791 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:05.818 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T18:53:05.829 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:05.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:05.830 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:05.931 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:05.943 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.1 to 1 (10000) 2024-09-20T18:53:05.943 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:05 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2290880995' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-20T18:53:05.943 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:05 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-20T18:53:05.944 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:05 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2290880995' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-20T18:53:05.944 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:05 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-20T18:53:05.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:05.981 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:05.981 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:06.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:05 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2290880995' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-20T18:53:06.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:05 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-20T18:53:06.081 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:06.105 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:06.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:06.107 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:06.207 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:06.233 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:06.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:06.234 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:06.335 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:06.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:06.370 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:06.371 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:06.471 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:06.494 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:06.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:06.495 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:06.595 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:53:06.619 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:06.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:06.620 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:53:06.721 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:06.760 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:06.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:06.761 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:06.774 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:53:06 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:53:06.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:06 smithi142 ceph-mon[32508]: pgmap v148: 148 pgs: 148 active+clean; 5.3 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 666 op/s 2024-09-20T18:53:06.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:06 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-20T18:53:06.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:06 smithi142 ceph-mon[32508]: osdmap e135: 8 total, 8 up, 8 in 2024-09-20T18:53:06.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:06 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:53:06.862 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:06.888 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:06.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:06.889 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:06.989 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:07.015 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:07.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:07.016 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:07.046 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:53:07.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:06 smithi083 ceph-mon[29421]: pgmap v148: 148 pgs: 148 active+clean; 5.3 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 666 op/s 2024-09-20T18:53:07.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:06 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-20T18:53:07.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:06 smithi083 ceph-mon[29421]: osdmap e135: 8 total, 8 up, 8 in 2024-09-20T18:53:07.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:06 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:53:07.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:06 smithi083 ceph-mon[36260]: pgmap v148: 148 pgs: 148 active+clean; 5.3 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 666 op/s 2024-09-20T18:53:07.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:06 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-20T18:53:07.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:06 smithi083 ceph-mon[36260]: osdmap e135: 8 total, 8 up, 8 in 2024-09-20T18:53:07.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:06 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:53:07.117 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:07.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:07.171 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:07.171 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:07.272 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:07.301 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:07.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:07.302 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:07.403 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:53:07.430 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:07.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:07.431 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:53:07.532 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:53:07.558 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:07.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:07.559 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:53:07.659 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd reweight 2 1 2024-09-20T18:53:07.659 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:53:07.701 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:07.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:07.703 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:53:07.803 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:53:07.829 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:07.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:07.830 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:53:07.903 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:07 smithi083 ceph-mon[29421]: pgmap v150: 148 pgs: 148 active+clean; 5.0 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.34k op/s 2024-09-20T18:53:07.904 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:07 smithi083 ceph-mon[36260]: pgmap v150: 148 pgs: 148 active+clean; 5.0 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.34k op/s 2024-09-20T18:53:07.930 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:07.988 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:07.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:07.989 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:08.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:07 smithi142 ceph-mon[32508]: pgmap v150: 148 pgs: 148 active+clean; 5.0 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.34k op/s 2024-09-20T18:53:08.090 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:08.115 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:08.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:08.116 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:08.204 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:53:08.216 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:08.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:08.291 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:08.291 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:08.392 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:53:08.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:08.431 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:08.431 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:53:08.532 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:08.535 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:53:08.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:08.592 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:08.592 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:08.679 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-20T18:53:08.693 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:08.732 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:08.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:08.733 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:08.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:08 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2715018270' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:53:08.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:08 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:53:08.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:08 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2715018270' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:53:08.804 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:08 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:53:08.834 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:08.878 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:08.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:08.879 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:08.979 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:09.007 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:09.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:09.008 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:09.026 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:08 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2715018270' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:53:09.027 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:08 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:53:09.071 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T18:53:09.071 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:53:09.109 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:53:09.146 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:09.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:09.147 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:53:09.248 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:53:09.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:09.288 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:09.288 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:53:09.389 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:53:09.415 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:09.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:09.416 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:53:09.517 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:09.557 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:09.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:09.561 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:09.662 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:09.685 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:09.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:09.686 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:09.787 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:09.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:09 smithi083 ceph-mon[29421]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:53:09.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:09 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T18:53:09.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:09 smithi083 ceph-mon[29421]: osdmap e136: 8 total, 8 up, 8 in 2024-09-20T18:53:09.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:09 smithi083 ceph-mon[29421]: 6.1a scrub starts 2024-09-20T18:53:09.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:09 smithi083 ceph-mon[29421]: 6.1a scrub ok 2024-09-20T18:53:09.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:09 smithi083 ceph-mon[29421]: pgmap v152: 148 pgs: 148 active+clean; 5.0 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.21k op/s 2024-09-20T18:53:09.805 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:09 smithi083 ceph-mon[36260]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:53:09.805 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:09 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T18:53:09.805 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:09 smithi083 ceph-mon[36260]: osdmap e136: 8 total, 8 up, 8 in 2024-09-20T18:53:09.805 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:09 smithi083 ceph-mon[36260]: 6.1a scrub starts 2024-09-20T18:53:09.805 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:09 smithi083 ceph-mon[36260]: 6.1a scrub ok 2024-09-20T18:53:09.805 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:09 smithi083 ceph-mon[36260]: pgmap v152: 148 pgs: 148 active+clean; 5.0 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.21k op/s 2024-09-20T18:53:09.822 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:09.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:09.823 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:09.923 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:09.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:09.959 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:09.959 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:10.006 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_historic_ops 2024-09-20T18:53:10.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:09 smithi142 ceph-mon[32508]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:53:10.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:09 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T18:53:10.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:09 smithi142 ceph-mon[32508]: osdmap e136: 8 total, 8 up, 8 in 2024-09-20T18:53:10.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:09 smithi142 ceph-mon[32508]: 6.1a scrub starts 2024-09-20T18:53:10.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:09 smithi142 ceph-mon[32508]: 6.1a scrub ok 2024-09-20T18:53:10.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:09 smithi142 ceph-mon[32508]: pgmap v152: 148 pgs: 148 active+clean; 5.0 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.21k op/s 2024-09-20T18:53:10.061 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:10.089 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:10.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:10.090 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:10.190 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:53:10.246 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:10.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:10.247 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:53:10.280 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:53:10.348 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:53:10.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:10.395 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:10.395 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:53:10.496 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:53:10.521 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:10.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:10.522 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:53:10.623 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:10.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:10.663 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:10.663 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:10.721 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.2 to 1 (10000) 2024-09-20T18:53:10.764 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:10.806 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:10.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:10.807 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:10.908 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:10.937 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:10.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:10.938 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:11.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:10 smithi142 ceph-mon[32508]: 3.10 scrub starts 2024-09-20T18:53:11.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:10 smithi142 ceph-mon[32508]: 3.10 scrub ok 2024-09-20T18:53:11.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:10 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2635237557' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-20T18:53:11.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:10 smithi142 ceph-mon[32508]: 5.1 scrub starts 2024-09-20T18:53:11.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:10 smithi142 ceph-mon[32508]: 5.1 scrub ok 2024-09-20T18:53:11.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:10 smithi142 ceph-mon[32508]: 3.1 scrub starts 2024-09-20T18:53:11.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:10 smithi142 ceph-mon[32508]: 3.1 scrub ok 2024-09-20T18:53:11.040 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:11.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:10 smithi083 ceph-mon[29421]: 3.10 scrub starts 2024-09-20T18:53:11.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:10 smithi083 ceph-mon[29421]: 3.10 scrub ok 2024-09-20T18:53:11.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:10 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2635237557' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-20T18:53:11.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:10 smithi083 ceph-mon[29421]: 5.1 scrub starts 2024-09-20T18:53:11.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:10 smithi083 ceph-mon[29421]: 5.1 scrub ok 2024-09-20T18:53:11.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:10 smithi083 ceph-mon[29421]: 3.1 scrub starts 2024-09-20T18:53:11.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:10 smithi083 ceph-mon[29421]: 3.1 scrub ok 2024-09-20T18:53:11.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:10 smithi083 ceph-mon[36260]: 3.10 scrub starts 2024-09-20T18:53:11.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:10 smithi083 ceph-mon[36260]: 3.10 scrub ok 2024-09-20T18:53:11.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:10 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2635237557' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-20T18:53:11.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:10 smithi083 ceph-mon[36260]: 5.1 scrub starts 2024-09-20T18:53:11.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:10 smithi083 ceph-mon[36260]: 5.1 scrub ok 2024-09-20T18:53:11.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:10 smithi083 ceph-mon[36260]: 3.1 scrub starts 2024-09-20T18:53:11.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:10 smithi083 ceph-mon[36260]: 3.1 scrub ok 2024-09-20T18:53:11.056 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:53:10 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:53:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:53:11.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:11.071 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:11.071 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:11.172 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:11.208 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:11.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:11.209 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:11.309 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:11.341 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:11.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:11.342 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:11.443 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:11.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:11.481 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:11.481 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:11.581 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:11.584 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd reweight 3 1 2024-09-20T18:53:11.639 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:11.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:11.640 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:11.742 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:11.764 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:11.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:11.765 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:11.770 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:53:11.808 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:11 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2635237557' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-20T18:53:11.808 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:11 smithi083 ceph-mon[36260]: osdmap e137: 8 total, 8 up, 8 in 2024-09-20T18:53:11.808 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:11 smithi083 ceph-mon[36260]: 3.f scrub starts 2024-09-20T18:53:11.808 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:11 smithi083 ceph-mon[36260]: 3.f scrub ok 2024-09-20T18:53:11.808 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:11 smithi083 ceph-mon[36260]: pgmap v154: 148 pgs: 148 active+clean; 4.9 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.05k op/s 2024-09-20T18:53:11.808 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:11 smithi083 ceph-mon[36260]: 3.5 scrub starts 2024-09-20T18:53:11.808 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:11 smithi083 ceph-mon[36260]: 3.5 scrub ok 2024-09-20T18:53:11.808 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:11 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2635237557' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-20T18:53:11.809 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:11 smithi083 ceph-mon[29421]: osdmap e137: 8 total, 8 up, 8 in 2024-09-20T18:53:11.809 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:11 smithi083 ceph-mon[29421]: 3.f scrub starts 2024-09-20T18:53:11.809 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:11 smithi083 ceph-mon[29421]: 3.f scrub ok 2024-09-20T18:53:11.809 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:11 smithi083 ceph-mon[29421]: pgmap v154: 148 pgs: 148 active+clean; 4.9 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.05k op/s 2024-09-20T18:53:11.809 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:11 smithi083 ceph-mon[29421]: 3.5 scrub starts 2024-09-20T18:53:11.809 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:11 smithi083 ceph-mon[29421]: 3.5 scrub ok 2024-09-20T18:53:11.866 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:11.891 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:11.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:11.892 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:11.993 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:12.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:11 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2635237557' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-20T18:53:12.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:11 smithi142 ceph-mon[32508]: osdmap e137: 8 total, 8 up, 8 in 2024-09-20T18:53:12.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:11 smithi142 ceph-mon[32508]: 3.f scrub starts 2024-09-20T18:53:12.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:11 smithi142 ceph-mon[32508]: 3.f scrub ok 2024-09-20T18:53:12.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:11 smithi142 ceph-mon[32508]: pgmap v154: 148 pgs: 148 active+clean; 4.9 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.05k op/s 2024-09-20T18:53:12.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:11 smithi142 ceph-mon[32508]: 3.5 scrub starts 2024-09-20T18:53:12.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:11 smithi142 ceph-mon[32508]: 3.5 scrub ok 2024-09-20T18:53:12.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:12.046 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:12.046 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:12.147 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:53:12.210 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:12.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:12.211 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:53:12.311 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:12.338 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:12.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:12.339 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:12.439 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:12.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:12.461 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:12.461 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:12.562 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:53:12.603 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:12.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:12.604 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:53:12.706 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:12.747 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:12.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:12.748 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:12.849 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:12.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:12.886 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:12.886 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:12.987 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:13.027 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:13.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:13.028 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:13.129 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:53:13.168 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:13.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:13.169 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:53:13.270 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:13.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:13.303 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:13.303 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:13.404 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:53:13.429 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:13.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:13.430 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:53:13.530 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:13.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:13.558 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:13.558 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:13.659 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:13.685 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:13.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:13.686 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:13.732 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T18:53:13.786 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:13.811 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:13.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:13.812 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:13.912 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:13.936 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:13.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:13.937 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:14.031 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:13 smithi083 ceph-mon[29421]: 5.2 scrub starts 2024-09-20T18:53:14.031 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:13 smithi083 ceph-mon[29421]: 5.2 scrub ok 2024-09-20T18:53:14.031 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:13 smithi083 ceph-mon[29421]: 5.1c scrub starts 2024-09-20T18:53:14.031 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:13 smithi083 ceph-mon[29421]: pgmap v155: 148 pgs: 148 active+clean; 4.7 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.47k op/s 2024-09-20T18:53:14.031 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:13 smithi083 ceph-mon[29421]: 5.1c scrub ok 2024-09-20T18:53:14.031 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:13 smithi083 ceph-mon[29421]: 5.5 scrub starts 2024-09-20T18:53:14.032 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:13 smithi083 ceph-mon[29421]: 5.5 scrub ok 2024-09-20T18:53:14.032 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:13 smithi083 ceph-mon[29421]: 3.2 scrub starts 2024-09-20T18:53:14.032 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:13 smithi083 ceph-mon[29421]: 3.2 scrub ok 2024-09-20T18:53:14.032 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:13 smithi083 ceph-mon[36260]: 5.2 scrub starts 2024-09-20T18:53:14.032 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:13 smithi083 ceph-mon[36260]: 5.2 scrub ok 2024-09-20T18:53:14.032 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:13 smithi083 ceph-mon[36260]: 5.1c scrub starts 2024-09-20T18:53:14.032 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:13 smithi083 ceph-mon[36260]: pgmap v155: 148 pgs: 148 active+clean; 4.7 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.47k op/s 2024-09-20T18:53:14.032 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:13 smithi083 ceph-mon[36260]: 5.1c scrub ok 2024-09-20T18:53:14.032 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:13 smithi083 ceph-mon[36260]: 5.5 scrub starts 2024-09-20T18:53:14.032 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:13 smithi083 ceph-mon[36260]: 5.5 scrub ok 2024-09-20T18:53:14.032 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:13 smithi083 ceph-mon[36260]: 3.2 scrub starts 2024-09-20T18:53:14.033 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:13 smithi083 ceph-mon[36260]: 3.2 scrub ok 2024-09-20T18:53:14.033 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T18:53:14.033 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:53:14.038 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:14.094 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:14.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:14.095 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:14.196 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:14.222 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:14.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:14.223 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:14.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:13 smithi142 ceph-mon[32508]: 5.2 scrub starts 2024-09-20T18:53:14.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:13 smithi142 ceph-mon[32508]: 5.2 scrub ok 2024-09-20T18:53:14.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:13 smithi142 ceph-mon[32508]: 5.1c scrub starts 2024-09-20T18:53:14.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:13 smithi142 ceph-mon[32508]: pgmap v155: 148 pgs: 148 active+clean; 4.7 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.47k op/s 2024-09-20T18:53:14.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:13 smithi142 ceph-mon[32508]: 5.1c scrub ok 2024-09-20T18:53:14.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:13 smithi142 ceph-mon[32508]: 5.5 scrub starts 2024-09-20T18:53:14.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:13 smithi142 ceph-mon[32508]: 5.5 scrub ok 2024-09-20T18:53:14.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:13 smithi142 ceph-mon[32508]: 3.2 scrub starts 2024-09-20T18:53:14.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:13 smithi142 ceph-mon[32508]: 3.2 scrub ok 2024-09-20T18:53:14.323 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:53:14.349 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:14.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:14.350 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:53:14.447 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:53:14.447 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:53:14.450 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:14.476 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:14.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:14.477 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:14.480 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:53:14.481 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:53:14.511 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:53:14.512 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:53:14.548 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:53:14.549 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:53:14.578 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:14.579 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:53:14.580 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:53:14.604 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:14.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:14.605 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:14.618 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:53:14.618 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:53:14.654 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:53:14.654 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:53:14.695 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:53:14.696 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:53:14.706 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:53:14.733 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:14.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:14.734 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:53:14.835 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:14.870 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:14.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:14.871 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:14.972 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:53:15.004 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:15.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:15.005 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:53:15.106 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:15.140 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:15.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:15.141 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:15.194 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:14 smithi083 ceph-mon[36260]: 6.16 scrub starts 2024-09-20T18:53:15.194 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:14 smithi083 ceph-mon[36260]: 6.16 scrub ok 2024-09-20T18:53:15.194 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:14 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2941153149' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:53:15.196 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:14 smithi083 ceph-mon[29421]: 6.16 scrub starts 2024-09-20T18:53:15.196 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:14 smithi083 ceph-mon[29421]: 6.16 scrub ok 2024-09-20T18:53:15.196 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:14 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2941153149' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:53:15.242 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:15.243 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-20T18:53:15.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:14 smithi142 ceph-mon[32508]: 6.16 scrub starts 2024-09-20T18:53:15.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:14 smithi142 ceph-mon[32508]: 6.16 scrub ok 2024-09-20T18:53:15.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:14 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2941153149' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:53:15.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:15.329 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:15.329 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:15.430 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:15.477 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:15.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:15.477 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:15.578 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:15.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:15.627 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:15.627 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:15.728 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:15.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:15.753 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:15.753 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:15.776 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:53:15.823 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:53:15.854 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:53:15.881 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:15.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:15.882 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:53:15.983 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:16.011 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:16.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:16.012 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:16.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:15 smithi083 ceph-mon[29421]: 6.14 scrub starts 2024-09-20T18:53:16.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:15 smithi083 ceph-mon[29421]: 6.14 scrub ok 2024-09-20T18:53:16.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:15 smithi083 ceph-mon[29421]: pgmap v156: 148 pgs: 148 active+clean; 4.7 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 648 op/s 2024-09-20T18:53:16.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:15 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:53:16.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:15 smithi083 ceph-mon[36260]: 6.14 scrub starts 2024-09-20T18:53:16.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:15 smithi083 ceph-mon[36260]: 6.14 scrub ok 2024-09-20T18:53:16.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:15 smithi083 ceph-mon[36260]: pgmap v156: 148 pgs: 148 active+clean; 4.7 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 648 op/s 2024-09-20T18:53:16.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:15 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:53:16.113 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:16.148 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:16.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:16.149 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:16.250 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:16.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:15 smithi142 ceph-mon[32508]: 6.14 scrub starts 2024-09-20T18:53:16.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:15 smithi142 ceph-mon[32508]: 6.14 scrub ok 2024-09-20T18:53:16.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:15 smithi142 ceph-mon[32508]: pgmap v156: 148 pgs: 148 active+clean; 4.7 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 648 op/s 2024-09-20T18:53:16.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:15 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:53:16.284 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:16.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:16.285 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:16.386 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:16.410 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:16.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:16.411 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:16.512 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:16.549 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:16.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:16.550 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:16.650 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:53:16.674 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:16.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:16.675 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:53:16.776 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:16.807 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:16.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:16.808 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:16.909 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:16.923 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:53:16 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:53:16.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:16.954 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:16.955 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:17.055 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:17.076 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:17.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:17.077 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:17.178 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:53:17.215 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:17.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:17.217 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:53:17.249 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:16 smithi083 ceph-mon[29421]: 4.18 scrub starts 2024-09-20T18:53:17.249 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:16 smithi083 ceph-mon[29421]: 4.18 scrub ok 2024-09-20T18:53:17.249 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:16 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:53:17.249 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:16 smithi083 ceph-mon[36260]: 4.18 scrub starts 2024-09-20T18:53:17.249 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:16 smithi083 ceph-mon[36260]: 4.18 scrub ok 2024-09-20T18:53:17.249 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:16 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:53:17.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:16 smithi142 ceph-mon[32508]: 4.18 scrub starts 2024-09-20T18:53:17.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:16 smithi142 ceph-mon[32508]: 4.18 scrub ok 2024-09-20T18:53:17.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:16 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:53:17.317 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:17.355 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:17.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:17.356 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:17.456 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:17.481 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:17.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:17.482 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:17.583 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:17.623 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:17.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:17.624 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:17.724 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:17.747 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:17.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:17.748 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:17.769 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T18:53:17.835 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:53:17.849 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:17.911 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:17.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:17.912 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:17.951 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-20T18:53:18.013 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:18.036 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:18.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:18.037 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:18.138 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:18.145 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:17 smithi083 ceph-mon[29421]: 5.16 scrub starts 2024-09-20T18:53:18.146 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:17 smithi083 ceph-mon[29421]: 5.16 scrub ok 2024-09-20T18:53:18.146 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:17 smithi083 ceph-mon[29421]: pgmap v157: 148 pgs: 148 active+clean; 4.3 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.29k op/s 2024-09-20T18:53:18.146 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:17 smithi083 ceph-mon[29421]: 3.17 scrub starts 2024-09-20T18:53:18.146 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:17 smithi083 ceph-mon[29421]: 3.17 scrub ok 2024-09-20T18:53:18.146 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:17 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2718752463' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:53:18.146 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:17 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:53:18.146 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:17 smithi083 ceph-mon[36260]: 5.16 scrub starts 2024-09-20T18:53:18.146 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:17 smithi083 ceph-mon[36260]: 5.16 scrub ok 2024-09-20T18:53:18.146 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:17 smithi083 ceph-mon[36260]: pgmap v157: 148 pgs: 148 active+clean; 4.3 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.29k op/s 2024-09-20T18:53:18.147 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:17 smithi083 ceph-mon[36260]: 3.17 scrub starts 2024-09-20T18:53:18.147 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:17 smithi083 ceph-mon[36260]: 3.17 scrub ok 2024-09-20T18:53:18.147 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:17 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2718752463' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:53:18.147 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:17 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:53:18.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:18.188 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:18.188 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:18.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:17 smithi142 ceph-mon[32508]: 5.16 scrub starts 2024-09-20T18:53:18.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:17 smithi142 ceph-mon[32508]: 5.16 scrub ok 2024-09-20T18:53:18.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:17 smithi142 ceph-mon[32508]: pgmap v157: 148 pgs: 148 active+clean; 4.3 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.29k op/s 2024-09-20T18:53:18.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:17 smithi142 ceph-mon[32508]: 3.17 scrub starts 2024-09-20T18:53:18.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:17 smithi142 ceph-mon[32508]: 3.17 scrub ok 2024-09-20T18:53:18.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:17 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2718752463' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:53:18.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:17 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:53:18.289 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:18.329 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:18.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:18.330 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:18.431 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:18.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:18.475 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:18.476 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:18.577 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:53:18.605 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:18.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:18.606 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:53:18.707 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:53:18.748 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:18.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:18.749 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:53:18.849 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:18.876 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:18.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:18.877 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:18.960 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.3 to 1 (10000) 2024-09-20T18:53:18.978 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:53:19.025 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:19.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:19.026 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:53:19.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:18 smithi083 ceph-mon[29421]: 4.b scrub starts 2024-09-20T18:53:19.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:18 smithi083 ceph-mon[29421]: 4.b scrub ok 2024-09-20T18:53:19.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:18 smithi083 ceph-mon[29421]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T18:53:19.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:18 smithi083 ceph-mon[29421]: Cluster is now healthy 2024-09-20T18:53:19.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:18 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T18:53:19.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:18 smithi083 ceph-mon[29421]: osdmap e138: 8 total, 8 up, 8 in 2024-09-20T18:53:19.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:18 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3796811813' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-20T18:53:19.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:18 smithi083 ceph-mon[36260]: 4.b scrub starts 2024-09-20T18:53:19.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:18 smithi083 ceph-mon[36260]: 4.b scrub ok 2024-09-20T18:53:19.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:18 smithi083 ceph-mon[36260]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T18:53:19.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:18 smithi083 ceph-mon[36260]: Cluster is now healthy 2024-09-20T18:53:19.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:18 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T18:53:19.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:18 smithi083 ceph-mon[36260]: osdmap e138: 8 total, 8 up, 8 in 2024-09-20T18:53:19.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:18 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3796811813' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-20T18:53:19.093 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T18:53:19.094 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-20T18:53:19.094 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:19.094 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:177528 7.f e139/83 7:f35cb1fb:::benchmark_data_smithi083_70643_object54726:head v 139'11208, mlcod=139'11205)", 2024-09-20T18:53:19.094 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:53:19.072145+0000", 2024-09-20T18:53:19.094 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0033766500000000001, 2024-09-20T18:53:19.094 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0034019419999999998, 2024-09-20T18:53:19.094 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:53:19.094 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:53:19.094 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:53:19.094 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:53:19.094 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:19.095 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:53:19.095 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:19.072145+0000", 2024-09-20T18:53:19.095 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:19.095 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:19.095 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:19.095 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:53:19.095 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:19.072145+0000", 2024-09-20T18:53:19.095 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:19.095 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:19.095 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:19.095 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:53:19.095 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:19.072145+0000", 2024-09-20T18:53:19.095 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1800000000000001e-07 2024-09-20T18:53:19.095 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:19.096 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:19.096 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:53:19.096 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:19.072149+0000", 2024-09-20T18:53:19.096 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6490000000000001e-06 2024-09-20T18:53:19.096 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:19.096 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:19.096 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:53:19.097 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:19.072155+0000", 2024-09-20T18:53:19.097 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.711e-06 2024-09-20T18:53:19.097 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:19.097 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:19.097 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:53:19.097 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:19.072159+0000", 2024-09-20T18:53:19.097 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5680000000000001e-06 2024-09-20T18:53:19.097 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:19.097 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:19.097 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:53:19.097 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:19.075030+0000", 2024-09-20T18:53:19.097 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0028701769999999998 2024-09-20T18:53:19.097 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:19.097 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:19.098 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:53:19.098 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:19.075047+0000", 2024-09-20T18:53:19.098 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7388999999999999e-05 2024-09-20T18:53:19.098 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:19.098 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:53:19.098 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:19.098 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:19.098 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:19.098 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:177533 7.1 e139/83 7:83c919ea:::benchmark_data_smithi083_70643_object54731:head v 139'11097, mlcod=139'11095)", 2024-09-20T18:53:19.098 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:53:19.071220+0000", 2024-09-20T18:53:19.098 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0043013310000000003, 2024-09-20T18:53:19.098 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0043807840000000004, 2024-09-20T18:53:19.098 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:53:19.098 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:53:19.099 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:53:19.099 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:53:19.099 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:19.099 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:53:19.099 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:19.071220+0000", 2024-09-20T18:53:19.099 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:19.099 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:19.099 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:19.099 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:53:19.099 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:19.071220+0000", 2024-09-20T18:53:19.099 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:19.099 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:19.099 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:19.099 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:53:19.099 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:19.071221+0000", 2024-09-20T18:53:19.100 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9800000000000002e-07 2024-09-20T18:53:19.100 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:19.100 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:19.100 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:53:19.100 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:19.071231+0000", 2024-09-20T18:53:19.100 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1589999999999999e-06 2024-09-20T18:53:19.100 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:19.100 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:19.100 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:53:19.100 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:19.071245+0000", 2024-09-20T18:53:19.100 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4861999999999999e-05 2024-09-20T18:53:19.100 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:19.100 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:19.100 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:53:19.100 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:19.071253+0000", 2024-09-20T18:53:19.101 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8800000000000008e-06 2024-09-20T18:53:19.101 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:19.101 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:19.101 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:53:19.101 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:19.071308+0000", 2024-09-20T18:53:19.101 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4469000000000003e-05 2024-09-20T18:53:19.101 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:19.101 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:19.101 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:53:19.101 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:19.071369+0000", 2024-09-20T18:53:19.101 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0779999999999997e-05 2024-09-20T18:53:19.101 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:19.101 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:53:19.101 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:19.102 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:19.102 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:19.102 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:177538 7.1 e139/83)", 2024-09-20T18:53:19.102 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:53:19.071274+0000", 2024-09-20T18:53:19.102 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.00424801, 2024-09-20T18:53:19.102 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0043714160000000004, 2024-09-20T18:53:19.102 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:53:19.102 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:53:19.102 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "queued for pg", 2024-09-20T18:53:19.103 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:53:19.103 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:19.103 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:53:19.103 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:19.071274+0000", 2024-09-20T18:53:19.103 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:19.103 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:19.103 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:19.103 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:53:19.103 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:19.071274+0000", 2024-09-20T18:53:19.103 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:19.103 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:19.103 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:19.103 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:53:19.103 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:19.071274+0000", 2024-09-20T18:53:19.104 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.58e-07 2024-09-20T18:53:19.104 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:19.104 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:19.104 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:53:19.104 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:19.071278+0000", 2024-09-20T18:53:19.104 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7000000000000002e-06 2024-09-20T18:53:19.104 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:19.104 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:19.104 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:53:19.104 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:19.071284+0000", 2024-09-20T18:53:19.104 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.55e-06 2024-09-20T18:53:19.104 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:19.104 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:19.104 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:53:19.105 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:19.071289+0000", 2024-09-20T18:53:19.105 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9300000000000002e-06 2024-09-20T18:53:19.105 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:19.105 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:53:19.105 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:19.105 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:19.105 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:19.105 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:177534 7.0 e139/83 7:0bc6d6b6:::benchmark_data_smithi083_70643_object54732:head v 139'11263, mlcod=139'11261)", 2024-09-20T18:53:19.105 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:53:19.072052+0000", 2024-09-20T18:53:19.105 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0034692989999999999, 2024-09-20T18:53:19.105 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0036403, 2024-09-20T18:53:19.105 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:53:19.105 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:53:19.105 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:53:19.106 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:53:19.106 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:19.106 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:53:19.106 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:19.072052+0000", 2024-09-20T18:53:19.106 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:19.106 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:19.106 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:19.106 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:53:19.106 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:19.072052+0000", 2024-09-20T18:53:19.106 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:19.106 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:19.106 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:19.106 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:53:19.106 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:19.072053+0000", 2024-09-20T18:53:19.107 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1999999999999999e-07 2024-09-20T18:53:19.107 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:19.107 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:19.107 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:53:19.108 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:19.072059+0000", 2024-09-20T18:53:19.108 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4689999999999996e-06 2024-09-20T18:53:19.108 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:19.108 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:19.108 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:53:19.109 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:19.072068+0000", 2024-09-20T18:53:19.109 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7750000000000004e-06 2024-09-20T18:53:19.109 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:19.109 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:19.109 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:53:19.109 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:19.072075+0000", 2024-09-20T18:53:19.109 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2219999999999999e-06 2024-09-20T18:53:19.109 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:19.109 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:19.109 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:53:19.109 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:19.072119+0000", 2024-09-20T18:53:19.109 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4641999999999997e-05 2024-09-20T18:53:19.109 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:19.109 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:19.110 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:53:19.110 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:19.072136+0000", 2024-09-20T18:53:19.110 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6497000000000001e-05 2024-09-20T18:53:19.110 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:19.110 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:53:19.110 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:19.110 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:19.110 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:19.110 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:177537 7.2 7:452ffecd:::benchmark_data_smithi083_70643_object54735:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139)", 2024-09-20T18:53:19.110 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:53:19.072099+0000", 2024-09-20T18:53:19.110 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0034227900000000002, 2024-09-20T18:53:19.110 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0036489399999999998, 2024-09-20T18:53:19.111 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:53:19.111 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:53:19.111 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "waiting for sub ops", 2024-09-20T18:53:19.111 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:53:19.111 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:53:19.111 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:19.111 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 177537 2024-09-20T18:53:19.111 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:19.111 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:53:19.111 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:19.111 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:53:19.111 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:19.072099+0000", 2024-09-20T18:53:19.111 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:19.112 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:19.112 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:19.112 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:53:19.112 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:19.072099+0000", 2024-09-20T18:53:19.112 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:19.112 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:19.112 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:19.112 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:53:19.112 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:19.072100+0000", 2024-09-20T18:53:19.112 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2589999999999999e-06 2024-09-20T18:53:19.112 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:19.112 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:19.112 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:53:19.112 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:19.072104+0000", 2024-09-20T18:53:19.112 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.32e-06 2024-09-20T18:53:19.113 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:19.113 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:19.113 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:53:19.113 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:19.072111+0000", 2024-09-20T18:53:19.113 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5190000000000003e-06 2024-09-20T18:53:19.113 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:19.113 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:19.113 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:53:19.113 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:19.072116+0000", 2024-09-20T18:53:19.113 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2569999999999998e-06 2024-09-20T18:53:19.114 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:19.114 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:19.114 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:53:19.114 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:19.072298+0000", 2024-09-20T18:53:19.114 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000181748 2024-09-20T18:53:19.114 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:19.114 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:19.114 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:53:19.114 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:19.072398+0000", 2024-09-20T18:53:19.114 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010011899999999999 2024-09-20T18:53:19.115 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:19.115 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:19.115 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T18:53:19.115 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:19.072497+0000", 2024-09-20T18:53:19.115 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8974999999999993e-05 2024-09-20T18:53:19.115 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:19.115 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:53:19.115 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:19.115 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:19.115 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T18:53:19.115 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 5 2024-09-20T18:53:19.115 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T18:53:19.127 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:19.153 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:19.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:19.154 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:19.254 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:19.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:18 smithi142 ceph-mon[32508]: 4.b scrub starts 2024-09-20T18:53:19.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:18 smithi142 ceph-mon[32508]: 4.b scrub ok 2024-09-20T18:53:19.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:18 smithi142 ceph-mon[32508]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T18:53:19.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:18 smithi142 ceph-mon[32508]: Cluster is now healthy 2024-09-20T18:53:19.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:18 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T18:53:19.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:18 smithi142 ceph-mon[32508]: osdmap e138: 8 total, 8 up, 8 in 2024-09-20T18:53:19.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:18 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3796811813' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-20T18:53:19.283 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:19.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:19.284 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:19.385 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:19.410 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:19.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:19.411 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:19.512 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:53:19.549 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:19.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:19.550 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:53:19.650 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:19.678 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:19.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:19.679 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:19.779 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd reweight 4 1 2024-09-20T18:53:19.779 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:19.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:19.826 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:19.826 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:19.927 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:19.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:19.970 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:19.971 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:20.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:19 smithi083 ceph-mon[29421]: pgmap v159: 148 pgs: 148 active+clean; 4.3 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.30k op/s 2024-09-20T18:53:20.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:19 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3796811813' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-20T18:53:20.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:19 smithi083 ceph-mon[29421]: osdmap e139: 8 total, 8 up, 8 in 2024-09-20T18:53:20.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:19 smithi083 ceph-mon[29421]: 3.0 scrub starts 2024-09-20T18:53:20.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:19 smithi083 ceph-mon[29421]: 3.0 scrub ok 2024-09-20T18:53:20.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:19 smithi083 ceph-mon[29421]: 6.15 scrub starts 2024-09-20T18:53:20.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:19 smithi083 ceph-mon[29421]: 6.15 scrub ok 2024-09-20T18:53:20.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:19 smithi083 ceph-mon[36260]: pgmap v159: 148 pgs: 148 active+clean; 4.3 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.30k op/s 2024-09-20T18:53:20.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:19 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3796811813' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-20T18:53:20.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:19 smithi083 ceph-mon[36260]: osdmap e139: 8 total, 8 up, 8 in 2024-09-20T18:53:20.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:19 smithi083 ceph-mon[36260]: 3.0 scrub starts 2024-09-20T18:53:20.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:19 smithi083 ceph-mon[36260]: 3.0 scrub ok 2024-09-20T18:53:20.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:19 smithi083 ceph-mon[36260]: 6.15 scrub starts 2024-09-20T18:53:20.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:19 smithi083 ceph-mon[36260]: 6.15 scrub ok 2024-09-20T18:53:20.072 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:20.098 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:20.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:20.099 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:20.199 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:20.226 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:20.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:20.227 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:20.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:19 smithi142 ceph-mon[32508]: pgmap v159: 148 pgs: 148 active+clean; 4.3 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.30k op/s 2024-09-20T18:53:20.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:19 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3796811813' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-20T18:53:20.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:19 smithi142 ceph-mon[32508]: osdmap e139: 8 total, 8 up, 8 in 2024-09-20T18:53:20.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:19 smithi142 ceph-mon[32508]: 3.0 scrub starts 2024-09-20T18:53:20.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:19 smithi142 ceph-mon[32508]: 3.0 scrub ok 2024-09-20T18:53:20.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:19 smithi142 ceph-mon[32508]: 6.15 scrub starts 2024-09-20T18:53:20.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:19 smithi142 ceph-mon[32508]: 6.15 scrub ok 2024-09-20T18:53:20.328 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:20.364 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:20.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:20.365 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:20.445 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_blocked_ops 2024-09-20T18:53:20.465 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:20.471 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:53:20.489 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:20.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:20.490 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:20.591 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:20.673 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:20.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:20.674 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:20.775 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:20.793 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:53:20 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:53:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:53:20.798 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:20.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:20.799 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:20.837 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T18:53:20.900 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:20.925 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:20.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:20.926 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:21.027 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:21.051 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:21.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:21.052 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:21.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:20 smithi083 ceph-mon[29421]: 4.1a scrub starts 2024-09-20T18:53:21.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:20 smithi083 ceph-mon[29421]: 4.1a scrub ok 2024-09-20T18:53:21.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:20 smithi083 ceph-mon[29421]: 3.d scrub starts 2024-09-20T18:53:21.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:20 smithi083 ceph-mon[29421]: 3.d scrub ok 2024-09-20T18:53:21.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:20 smithi083 ceph-mon[36260]: 4.1a scrub starts 2024-09-20T18:53:21.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:20 smithi083 ceph-mon[36260]: 4.1a scrub ok 2024-09-20T18:53:21.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:20 smithi083 ceph-mon[36260]: 3.d scrub starts 2024-09-20T18:53:21.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:20 smithi083 ceph-mon[36260]: 3.d scrub ok 2024-09-20T18:53:21.114 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:53:21.152 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:21.155 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:53:21.155 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:53:21.175 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:21.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:21.176 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:21.183 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:53:21.193 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:53:21.194 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:53:21.231 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:53:21.232 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:53:21.267 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:53:21.267 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:53:21.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:20 smithi142 ceph-mon[32508]: 4.1a scrub starts 2024-09-20T18:53:21.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:20 smithi142 ceph-mon[32508]: 4.1a scrub ok 2024-09-20T18:53:21.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:20 smithi142 ceph-mon[32508]: 3.d scrub starts 2024-09-20T18:53:21.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:20 smithi142 ceph-mon[32508]: 3.d scrub ok 2024-09-20T18:53:21.277 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:21.300 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:21.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:21.301 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:21.302 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:53:21.302 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:53:21.334 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:53:21.334 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:53:21.369 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:53:21.369 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:53:21.401 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:53:21.402 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:53:21.402 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:21.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:21.438 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:21.438 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:21.539 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:53:21.585 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:21.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:21.586 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:53:21.687 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:21.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:21.724 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:21.724 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:21.825 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:21.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:21.871 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:21.871 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:21.972 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:22.012 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:22.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:22.013 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:22.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:21 smithi083 ceph-mon[29421]: 7.e scrub starts 2024-09-20T18:53:22.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:21 smithi083 ceph-mon[29421]: pgmap v161: 148 pgs: 148 active+clean; 4.3 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 796 op/s 2024-09-20T18:53:22.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:21 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1301769714' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:53:22.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:21 smithi083 ceph-mon[29421]: 5.19 scrub starts 2024-09-20T18:53:22.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:21 smithi083 ceph-mon[29421]: 5.19 scrub ok 2024-09-20T18:53:22.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:21 smithi083 ceph-mon[29421]: 3.e scrub starts 2024-09-20T18:53:22.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:21 smithi083 ceph-mon[29421]: 3.e scrub ok 2024-09-20T18:53:22.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:21 smithi083 ceph-mon[36260]: 7.e scrub starts 2024-09-20T18:53:22.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:21 smithi083 ceph-mon[36260]: pgmap v161: 148 pgs: 148 active+clean; 4.3 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 796 op/s 2024-09-20T18:53:22.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:21 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1301769714' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:53:22.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:21 smithi083 ceph-mon[36260]: 5.19 scrub starts 2024-09-20T18:53:22.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:21 smithi083 ceph-mon[36260]: 5.19 scrub ok 2024-09-20T18:53:22.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:21 smithi083 ceph-mon[36260]: 3.e scrub starts 2024-09-20T18:53:22.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:21 smithi083 ceph-mon[36260]: 3.e scrub ok 2024-09-20T18:53:22.114 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:22.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:22.166 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:22.166 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:22.267 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:53:22.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:21 smithi142 ceph-mon[32508]: 7.e scrub starts 2024-09-20T18:53:22.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:21 smithi142 ceph-mon[32508]: pgmap v161: 148 pgs: 148 active+clean; 4.3 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 796 op/s 2024-09-20T18:53:22.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:21 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1301769714' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:53:22.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:21 smithi142 ceph-mon[32508]: 5.19 scrub starts 2024-09-20T18:53:22.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:21 smithi142 ceph-mon[32508]: 5.19 scrub ok 2024-09-20T18:53:22.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:21 smithi142 ceph-mon[32508]: 3.e scrub starts 2024-09-20T18:53:22.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:21 smithi142 ceph-mon[32508]: 3.e scrub ok 2024-09-20T18:53:22.292 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:22.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:22.293 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:53:22.394 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:22.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:22.422 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:22.422 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:22.522 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:22.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:22.565 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:22.565 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:22.666 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:53:22.698 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:22.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:22.699 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:53:22.800 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:22.826 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:22.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:22.827 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:22.928 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:22.952 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:22.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:22.953 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:22.982 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.4 to 1 (10000) 2024-09-20T18:53:23.054 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:23.091 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:23.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:23.092 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:23.163 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:22 smithi083 ceph-mon[29421]: 6.19 scrub starts 2024-09-20T18:53:23.163 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:22 smithi083 ceph-mon[29421]: 6.19 scrub ok 2024-09-20T18:53:23.163 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:22 smithi083 ceph-mon[29421]: 7.4 scrub starts 2024-09-20T18:53:23.163 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:22 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1999060461' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-20T18:53:23.163 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:22 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-20T18:53:23.163 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:22 smithi083 ceph-mon[29421]: 2.0 scrub starts 2024-09-20T18:53:23.164 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:22 smithi083 ceph-mon[29421]: 2.0 scrub ok 2024-09-20T18:53:23.164 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:22 smithi083 ceph-mon[36260]: 6.19 scrub starts 2024-09-20T18:53:23.164 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:22 smithi083 ceph-mon[36260]: 6.19 scrub ok 2024-09-20T18:53:23.164 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:22 smithi083 ceph-mon[36260]: 7.4 scrub starts 2024-09-20T18:53:23.164 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:22 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1999060461' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-20T18:53:23.164 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:22 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-20T18:53:23.164 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:22 smithi083 ceph-mon[36260]: 2.0 scrub starts 2024-09-20T18:53:23.164 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:22 smithi083 ceph-mon[36260]: 2.0 scrub ok 2024-09-20T18:53:23.193 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:23.218 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:23.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:23.219 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:23.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:22 smithi142 ceph-mon[32508]: 6.19 scrub starts 2024-09-20T18:53:23.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:22 smithi142 ceph-mon[32508]: 6.19 scrub ok 2024-09-20T18:53:23.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:22 smithi142 ceph-mon[32508]: 7.4 scrub starts 2024-09-20T18:53:23.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:22 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1999060461' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-20T18:53:23.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:22 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-20T18:53:23.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:22 smithi142 ceph-mon[32508]: 2.0 scrub starts 2024-09-20T18:53:23.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:22 smithi142 ceph-mon[32508]: 2.0 scrub ok 2024-09-20T18:53:23.320 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:23.352 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:23.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:23.353 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:23.435 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T18:53:23.435 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:53:23.454 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:23.494 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:23.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:23.495 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:23.596 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:23.627 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:23.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:23.628 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:23.729 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:23.769 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:23.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:23.771 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:23.871 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:23.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:23.917 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:23.917 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:24.018 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:53:24.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:23 smithi083 ceph-mon[29421]: 7.e scrub ok 2024-09-20T18:53:24.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:23 smithi083 ceph-mon[29421]: pgmap v162: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 4.0 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.32k op/s 2024-09-20T18:53:24.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:23 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-20T18:53:24.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:23 smithi083 ceph-mon[29421]: osdmap e140: 8 total, 8 up, 8 in 2024-09-20T18:53:24.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:23 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:53:24.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:23 smithi083 ceph-mon[29421]: 7.5 scrub starts 2024-09-20T18:53:24.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:23 smithi083 ceph-mon[36260]: 7.e scrub ok 2024-09-20T18:53:24.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:23 smithi083 ceph-mon[36260]: pgmap v162: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 4.0 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.32k op/s 2024-09-20T18:53:24.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:23 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-20T18:53:24.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:23 smithi083 ceph-mon[36260]: osdmap e140: 8 total, 8 up, 8 in 2024-09-20T18:53:24.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:23 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:53:24.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:23 smithi083 ceph-mon[36260]: 7.5 scrub starts 2024-09-20T18:53:24.057 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:24.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:24.058 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:53:24.159 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:24.186 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:24.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:24.187 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:24.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:23 smithi142 ceph-mon[32508]: 7.e scrub ok 2024-09-20T18:53:24.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:23 smithi142 ceph-mon[32508]: pgmap v162: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 4.0 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.32k op/s 2024-09-20T18:53:24.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:23 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-20T18:53:24.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:23 smithi142 ceph-mon[32508]: osdmap e140: 8 total, 8 up, 8 in 2024-09-20T18:53:24.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:23 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:53:24.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:23 smithi142 ceph-mon[32508]: 7.5 scrub starts 2024-09-20T18:53:24.288 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:24.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:24.321 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:24.321 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:24.422 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:24.455 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:24.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:24.456 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:24.557 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:53:24.581 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:24.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:24.583 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:53:24.683 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:24.713 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:24.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:24.714 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:24.815 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:24.856 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:24.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:24.857 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:24.957 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:24.987 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:24.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:24.988 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:25.089 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:25.124 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:25.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:25.125 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:25.213 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T18:53:25.226 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:25.248 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:24 smithi083 ceph-mon[29421]: 7.4 scrub ok 2024-09-20T18:53:25.249 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:24 smithi083 ceph-mon[29421]: 3.a scrub starts 2024-09-20T18:53:25.249 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:24 smithi083 ceph-mon[29421]: 3.a scrub ok 2024-09-20T18:53:25.249 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:24 smithi083 ceph-mon[36260]: 7.4 scrub ok 2024-09-20T18:53:25.249 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:24 smithi083 ceph-mon[36260]: 3.a scrub starts 2024-09-20T18:53:25.249 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:24 smithi083 ceph-mon[36260]: 3.a scrub ok 2024-09-20T18:53:25.258 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:25.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:25.260 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:25.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:24 smithi142 ceph-mon[32508]: 7.4 scrub ok 2024-09-20T18:53:25.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:24 smithi142 ceph-mon[32508]: 3.a scrub starts 2024-09-20T18:53:25.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:24 smithi142 ceph-mon[32508]: 3.a scrub ok 2024-09-20T18:53:25.361 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:25.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:25.391 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:25.391 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:25.492 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:25.538 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:25.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:25.539 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:25.640 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:25.666 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:25.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:25.667 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:25.768 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:25.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:25.801 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:25.802 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:25.872 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_historic_ops 2024-09-20T18:53:25.902 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:25.927 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:25.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:25.928 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:25.936 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd reweight 5 1 2024-09-20T18:53:25.964 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:53:26.030 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:26.033 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:53:26.052 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:26.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:26.053 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:26.107 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:25 smithi083 ceph-mon[29421]: 4.a scrub starts 2024-09-20T18:53:26.107 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:25 smithi083 ceph-mon[29421]: 4.a scrub ok 2024-09-20T18:53:26.107 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:25 smithi083 ceph-mon[29421]: 7.5 scrub ok 2024-09-20T18:53:26.107 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:25 smithi083 ceph-mon[29421]: pgmap v164: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 4.0 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 742 B/s rd, 0 B/s wr, 754 op/s 2024-09-20T18:53:26.107 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:25 smithi083 ceph-mon[29421]: 6.1 scrub starts 2024-09-20T18:53:26.107 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:25 smithi083 ceph-mon[29421]: 6.1 scrub ok 2024-09-20T18:53:26.108 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:25 smithi083 ceph-mon[36260]: 4.a scrub starts 2024-09-20T18:53:26.108 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:25 smithi083 ceph-mon[36260]: 4.a scrub ok 2024-09-20T18:53:26.108 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:25 smithi083 ceph-mon[36260]: 7.5 scrub ok 2024-09-20T18:53:26.108 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:25 smithi083 ceph-mon[36260]: pgmap v164: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 4.0 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 742 B/s rd, 0 B/s wr, 754 op/s 2024-09-20T18:53:26.108 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:25 smithi083 ceph-mon[36260]: 6.1 scrub starts 2024-09-20T18:53:26.108 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:25 smithi083 ceph-mon[36260]: 6.1 scrub ok 2024-09-20T18:53:26.153 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:26.178 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:26.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:26.179 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:26.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:25 smithi142 ceph-mon[32508]: 4.a scrub starts 2024-09-20T18:53:26.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:25 smithi142 ceph-mon[32508]: 4.a scrub ok 2024-09-20T18:53:26.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:25 smithi142 ceph-mon[32508]: 7.5 scrub ok 2024-09-20T18:53:26.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:25 smithi142 ceph-mon[32508]: pgmap v164: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 4.0 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 742 B/s rd, 0 B/s wr, 754 op/s 2024-09-20T18:53:26.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:25 smithi142 ceph-mon[32508]: 6.1 scrub starts 2024-09-20T18:53:26.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:25 smithi142 ceph-mon[32508]: 6.1 scrub ok 2024-09-20T18:53:26.280 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:26.346 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:26.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:26.352 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:26.452 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:26.480 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:26.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:26.481 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:26.581 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:53:26.622 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:26.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:26.623 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:53:26.723 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:53:26.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:26.779 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:26.779 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:53:26.880 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:26.918 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:26.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:26.919 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:26.919 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:53:26 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:53:27.021 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:53:27.045 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:27.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:27.046 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:53:27.147 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:27.166 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:53:27.166 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:53:27.189 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:27.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:27.190 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:27.200 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:53:27.201 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:53:27.236 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:53:27.236 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:53:27.237 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:27 smithi083 ceph-mon[29421]: 3.8 scrub starts 2024-09-20T18:53:27.237 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:27 smithi083 ceph-mon[29421]: 3.8 scrub ok 2024-09-20T18:53:27.237 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:27 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:53:27.237 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:27 smithi083 ceph-mon[36260]: 3.8 scrub starts 2024-09-20T18:53:27.237 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:27 smithi083 ceph-mon[36260]: 3.8 scrub ok 2024-09-20T18:53:27.237 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:27 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:53:27.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:27 smithi142 ceph-mon[32508]: 3.8 scrub starts 2024-09-20T18:53:27.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:27 smithi142 ceph-mon[32508]: 3.8 scrub ok 2024-09-20T18:53:27.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:27 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:53:27.277 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:53:27.277 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:53:27.292 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:27.309 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:53:27.309 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:53:27.332 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:27.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:27.333 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:27.339 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:53:27.340 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:53:27.377 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:53:27.378 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:53:27.407 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:53:27.407 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:53:27.434 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:27.460 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:27.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:27.461 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:27.561 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:27.585 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:27.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:27.586 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:27.686 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:27.709 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:27.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:27.710 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:27.810 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:27.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:27.839 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:27.839 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:27.940 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:27.964 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:27.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:27.965 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:28.030 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-20T18:53:28.065 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:28.090 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:28.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:28.091 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:28.192 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:28.217 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:28.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:28.218 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:28.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:28 smithi142 ceph-mon[32508]: 4.1c scrub starts 2024-09-20T18:53:28.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:28 smithi142 ceph-mon[32508]: 4.1c scrub ok 2024-09-20T18:53:28.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:28 smithi142 ceph-mon[32508]: pgmap v165: 148 pgs: 148 active+clean; 3.7 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.29k op/s 2024-09-20T18:53:28.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:28 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1198156394' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:53:28.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:28 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:53:28.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:28 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2147947191' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:53:28.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:28 smithi142 ceph-mon[32508]: 4.17 scrub starts 2024-09-20T18:53:28.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:28 smithi142 ceph-mon[32508]: 4.17 scrub ok 2024-09-20T18:53:28.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:28 smithi142 ceph-mon[32508]: 3.13 scrub starts 2024-09-20T18:53:28.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:28 smithi142 ceph-mon[32508]: 3.13 scrub ok 2024-09-20T18:53:28.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:28 smithi083 ceph-mon[29421]: 4.1c scrub starts 2024-09-20T18:53:28.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:28 smithi083 ceph-mon[29421]: 4.1c scrub ok 2024-09-20T18:53:28.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:28 smithi083 ceph-mon[29421]: pgmap v165: 148 pgs: 148 active+clean; 3.7 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.29k op/s 2024-09-20T18:53:28.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:28 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1198156394' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:53:28.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:28 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:53:28.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:28 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2147947191' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:53:28.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:28 smithi083 ceph-mon[29421]: 4.17 scrub starts 2024-09-20T18:53:28.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:28 smithi083 ceph-mon[29421]: 4.17 scrub ok 2024-09-20T18:53:28.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:28 smithi083 ceph-mon[29421]: 3.13 scrub starts 2024-09-20T18:53:28.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:28 smithi083 ceph-mon[29421]: 3.13 scrub ok 2024-09-20T18:53:28.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:28 smithi083 ceph-mon[36260]: 4.1c scrub starts 2024-09-20T18:53:28.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:28 smithi083 ceph-mon[36260]: 4.1c scrub ok 2024-09-20T18:53:28.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:28 smithi083 ceph-mon[36260]: pgmap v165: 148 pgs: 148 active+clean; 3.7 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.29k op/s 2024-09-20T18:53:28.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:28 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1198156394' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:53:28.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:28 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:53:28.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:28 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2147947191' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:53:28.306 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:28 smithi083 ceph-mon[36260]: 4.17 scrub starts 2024-09-20T18:53:28.306 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:28 smithi083 ceph-mon[36260]: 4.17 scrub ok 2024-09-20T18:53:28.306 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:28 smithi083 ceph-mon[36260]: 3.13 scrub starts 2024-09-20T18:53:28.306 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:28 smithi083 ceph-mon[36260]: 3.13 scrub ok 2024-09-20T18:53:28.319 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:53:28.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:28.358 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:28.358 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:53:28.459 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:28.486 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:28.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:28.487 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:28.588 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:28.626 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:28.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:28.627 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:28.728 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:28.765 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:28.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:28.766 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:28.867 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:28.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:28.894 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:28.894 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:28.995 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:29.023 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:29.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:29.024 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:29.125 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:29.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:29.147 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:29.147 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:29.248 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:53:29.285 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:29.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:29.286 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:53:29.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:29 smithi083 ceph-mon[36260]: 5.a scrub starts 2024-09-20T18:53:29.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:29 smithi083 ceph-mon[36260]: 5.a scrub ok 2024-09-20T18:53:29.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:29 smithi083 ceph-mon[36260]: 6.2 scrub starts 2024-09-20T18:53:29.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:29 smithi083 ceph-mon[36260]: 6.2 scrub ok 2024-09-20T18:53:29.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:29 smithi083 ceph-mon[36260]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:53:29.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:29 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T18:53:29.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:29 smithi083 ceph-mon[36260]: osdmap e141: 8 total, 8 up, 8 in 2024-09-20T18:53:29.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:29 smithi083 ceph-mon[29421]: 5.a scrub starts 2024-09-20T18:53:29.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:29 smithi083 ceph-mon[29421]: 5.a scrub ok 2024-09-20T18:53:29.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:29 smithi083 ceph-mon[29421]: 6.2 scrub starts 2024-09-20T18:53:29.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:29 smithi083 ceph-mon[29421]: 6.2 scrub ok 2024-09-20T18:53:29.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:29 smithi083 ceph-mon[29421]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:53:29.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:29 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T18:53:29.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:29 smithi083 ceph-mon[29421]: osdmap e141: 8 total, 8 up, 8 in 2024-09-20T18:53:29.387 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:53:29.388 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:29 smithi142 ceph-mon[32508]: 5.a scrub starts 2024-09-20T18:53:29.388 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:29 smithi142 ceph-mon[32508]: 5.a scrub ok 2024-09-20T18:53:29.388 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:29 smithi142 ceph-mon[32508]: 6.2 scrub starts 2024-09-20T18:53:29.388 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:29 smithi142 ceph-mon[32508]: 6.2 scrub ok 2024-09-20T18:53:29.388 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:29 smithi142 ceph-mon[32508]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:53:29.388 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:29 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T18:53:29.388 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:29 smithi142 ceph-mon[32508]: osdmap e141: 8 total, 8 up, 8 in 2024-09-20T18:53:29.424 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:29.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:29.425 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:53:29.526 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:29.553 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:29.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:29.554 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:29.655 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:29.692 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:29.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:29.693 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:29.793 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:29.830 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:29.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:29.831 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:29.932 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:53:29.977 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:29.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:29.979 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:53:30.079 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:30.115 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:30.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:30.116 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:30.217 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:53:30.238 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:30.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:30.239 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:53:30.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:30 smithi142 ceph-mon[32508]: pgmap v167: 148 pgs: 148 active+clean; 3.7 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.16k op/s 2024-09-20T18:53:30.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:30 smithi083 ceph-mon[29421]: pgmap v167: 148 pgs: 148 active+clean; 3.7 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.16k op/s 2024-09-20T18:53:30.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:30 smithi083 ceph-mon[36260]: pgmap v167: 148 pgs: 148 active+clean; 3.7 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.16k op/s 2024-09-20T18:53:30.340 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:30.373 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:30.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:30.375 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:30.476 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:53:30.512 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:30.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:30.513 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:53:30.614 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:30.641 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:30.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:30.642 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:30.743 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:53:30.778 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:30.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:30.779 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:53:30.880 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:30.908 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:30.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:30.909 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:30.997 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:53:30 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:53:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:53:31.010 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:31.039 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:31.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:31.040 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:31.141 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:31.168 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:31.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:31.169 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:31.270 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:31.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:31 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:53:31.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:31 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:53:31.307 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:31.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:31.308 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:31.409 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:31.433 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:53:31.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:31.464 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:31.465 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:31.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:31 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:53:31.566 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:53:31.603 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:31.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:31.604 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:53:31.705 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:31.746 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:31.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:31.747 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:31.848 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:31.873 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:31.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:31.874 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:31.927 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:53:31.974 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:53:32.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:32.021 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:32.021 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:53:32.122 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:53:32.148 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:32.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:32.149 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:53:32.250 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:53:32.274 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:32.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:32.276 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:53:32.376 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:32.402 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:32.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:32.402 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:32.473 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:32 smithi083 ceph-mon[29421]: pgmap v168: 148 pgs: 148 active+clean; 3.6 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 873 op/s 2024-09-20T18:53:32.473 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:32 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:53:32.474 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:32 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:53:32.474 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:32 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:53:32.474 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:32 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:53:32.474 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:32 smithi083 ceph-mon[36260]: pgmap v168: 148 pgs: 148 active+clean; 3.6 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 873 op/s 2024-09-20T18:53:32.474 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:32 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:53:32.474 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:32 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:53:32.474 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:32 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:53:32.474 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:32 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:53:32.504 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:32.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:32 smithi142 ceph-mon[32508]: pgmap v168: 148 pgs: 148 active+clean; 3.6 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 873 op/s 2024-09-20T18:53:32.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:32 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:53:32.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:32 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:53:32.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:32 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:53:32.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:32 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:53:32.527 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:32.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:32.527 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:32.529 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T18:53:32.529 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-20T18:53:32.529 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-20T18:53:32.529 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-20T18:53:32.529 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.529 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:25327 7.3 7:cbe7b718:::benchmark_data_smithi083_70643_object25326:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:53:32.529 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.263028+0000", 2024-09-20T18:53:32.529 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 145.24616400100001, 2024-09-20T18:53:32.529 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.1801133, 2024-09-20T18:53:32.529 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:53:32.529 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:53:32.530 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:32.530 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:53:32.530 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:53:32.530 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:32.530 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25327 2024-09-20T18:53:32.530 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.530 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:53:32.530 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.530 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:53:32.530 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263028+0000", 2024-09-20T18:53:32.530 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:32.530 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.530 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.531 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:53:32.531 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263028+0000", 2024-09-20T18:53:32.531 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:32.531 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.531 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.531 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:53:32.531 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263029+0000", 2024-09-20T18:53:32.531 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1200000000000001e-06 2024-09-20T18:53:32.531 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.531 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.531 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:53:32.531 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263050+0000", 2024-09-20T18:53:32.531 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1019999999999999e-05 2024-09-20T18:53:32.531 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.532 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.532 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:53:32.532 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263060+0000", 2024-09-20T18:53:32.532 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0049e-05 2024-09-20T18:53:32.532 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.532 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.532 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:53:32.532 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263069+0000", 2024-09-20T18:53:32.532 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8999999999999995e-06 2024-09-20T18:53:32.532 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.532 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.532 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:53:32.532 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263770+0000", 2024-09-20T18:53:32.532 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00070160100000000005 2024-09-20T18:53:32.533 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.533 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.533 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:53:32.533 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263876+0000", 2024-09-20T18:53:32.533 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010538400000000001 2024-09-20T18:53:32.533 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.533 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.533 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,4", 2024-09-20T18:53:32.533 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263991+0000", 2024-09-20T18:53:32.533 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115151 2024-09-20T18:53:32.533 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.533 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.533 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:53:32.533 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.265911+0000", 2024-09-20T18:53:32.533 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0019197319999999999 2024-09-20T18:53:32.534 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.534 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.534 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:32.534 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.267644+0000", 2024-09-20T18:53:32.534 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0017335600000000001 2024-09-20T18:53:32.534 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.534 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.534 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:32.534 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443038+0000", 2024-09-20T18:53:32.534 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17539418100000001 2024-09-20T18:53:32.534 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.534 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.534 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:53:32.534 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443095+0000", 2024-09-20T18:53:32.535 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6972999999999998e-05 2024-09-20T18:53:32.535 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.535 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.535 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:53:32.535 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443141+0000", 2024-09-20T18:53:32.535 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5629e-05 2024-09-20T18:53:32.535 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:32.535 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:53:32.535 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:32.535 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.535 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.535 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:25329 7.3 7:c0dcc395:::benchmark_data_smithi083_70643_object25328:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:53:32.536 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.263576+0000", 2024-09-20T18:53:32.536 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 145.24561581099999, 2024-09-20T18:53:32.536 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17968018699999999, 2024-09-20T18:53:32.536 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:53:32.536 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:53:32.536 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:32.536 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:53:32.536 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:53:32.536 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:32.536 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25329 2024-09-20T18:53:32.536 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.536 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:53:32.536 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.537 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:53:32.537 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263576+0000", 2024-09-20T18:53:32.537 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:32.537 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.537 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.537 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:53:32.537 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263576+0000", 2024-09-20T18:53:32.537 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:32.537 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.537 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.537 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:53:32.537 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263577+0000", 2024-09-20T18:53:32.538 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.3799999999999996e-07 2024-09-20T18:53:32.538 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.538 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.538 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:53:32.538 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263599+0000", 2024-09-20T18:53:32.538 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2333000000000001e-05 2024-09-20T18:53:32.538 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.538 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.539 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:53:32.539 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263608+0000", 2024-09-20T18:53:32.539 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5690000000000005e-06 2024-09-20T18:53:32.539 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.539 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.539 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:53:32.539 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263614+0000", 2024-09-20T18:53:32.539 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6449999999999999e-06 2024-09-20T18:53:32.539 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.539 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.539 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:53:32.539 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.264856+0000", 2024-09-20T18:53:32.539 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001241486 2024-09-20T18:53:32.540 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.540 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.540 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:53:32.540 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.264959+0000", 2024-09-20T18:53:32.540 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000103037 2024-09-20T18:53:32.540 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.540 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.540 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,4", 2024-09-20T18:53:32.540 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.265077+0000", 2024-09-20T18:53:32.540 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000118163 2024-09-20T18:53:32.540 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.540 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.540 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:53:32.540 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266326+0000", 2024-09-20T18:53:32.541 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001248929 2024-09-20T18:53:32.541 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.541 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.541 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:32.541 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.267696+0000", 2024-09-20T18:53:32.541 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001369924 2024-09-20T18:53:32.541 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.541 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.541 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:32.541 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443180+0000", 2024-09-20T18:53:32.541 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17548422 2024-09-20T18:53:32.541 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.541 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.542 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:53:32.542 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443222+0000", 2024-09-20T18:53:32.542 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2426999999999999e-05 2024-09-20T18:53:32.542 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.542 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.542 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:53:32.542 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443256+0000", 2024-09-20T18:53:32.542 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3615999999999998e-05 2024-09-20T18:53:32.542 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:32.542 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:53:32.542 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:32.542 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.543 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.543 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:25332 7.2 7:4a99327a:::benchmark_data_smithi083_70643_object25331:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:53:32.543 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.266138+0000", 2024-09-20T18:53:32.543 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 145.243053347, 2024-09-20T18:53:32.543 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17949907200000001, 2024-09-20T18:53:32.543 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:53:32.543 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:53:32.543 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:32.543 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:53:32.544 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:53:32.544 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:32.544 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25332 2024-09-20T18:53:32.544 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.544 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:53:32.544 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.544 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:53:32.544 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266138+0000", 2024-09-20T18:53:32.544 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:32.544 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.545 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.545 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:53:32.545 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266138+0000", 2024-09-20T18:53:32.545 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:32.545 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.545 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.545 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:53:32.545 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266140+0000", 2024-09-20T18:53:32.545 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1480000000000001e-06 2024-09-20T18:53:32.545 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.545 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.546 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:53:32.546 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266160+0000", 2024-09-20T18:53:32.546 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0908999999999999e-05 2024-09-20T18:53:32.546 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.546 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.546 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:53:32.546 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266170+0000", 2024-09-20T18:53:32.546 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1390000000000001e-06 2024-09-20T18:53:32.546 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.546 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.547 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:53:32.547 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266176+0000", 2024-09-20T18:53:32.547 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3740000000000003e-06 2024-09-20T18:53:32.547 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.547 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.547 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:53:32.547 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266234+0000", 2024-09-20T18:53:32.547 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8084999999999997e-05 2024-09-20T18:53:32.547 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.547 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.547 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:53:32.547 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266342+0000", 2024-09-20T18:53:32.548 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000108448 2024-09-20T18:53:32.548 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.548 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.548 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T18:53:32.548 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266470+0000", 2024-09-20T18:53:32.548 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012756700000000001 2024-09-20T18:53:32.548 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.548 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.548 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:32.548 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444956+0000", 2024-09-20T18:53:32.548 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.178485641 2024-09-20T18:53:32.548 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.549 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.549 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:32.549 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445026+0000", 2024-09-20T18:53:32.549 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0778999999999994e-05 2024-09-20T18:53:32.549 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.549 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.549 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:53:32.549 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445522+0000", 2024-09-20T18:53:32.549 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00049521500000000002 2024-09-20T18:53:32.549 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.550 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.550 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:53:32.550 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445598+0000", 2024-09-20T18:53:32.550 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5993000000000002e-05 2024-09-20T18:53:32.550 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.550 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.550 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:53:32.550 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445637+0000", 2024-09-20T18:53:32.550 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9774000000000002e-05 2024-09-20T18:53:32.550 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:32.550 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:53:32.550 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:32.550 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.551 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.551 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:25333 7.2 7:4c00f64e:::benchmark_data_smithi083_70643_object25332:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:53:32.551 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.266192+0000", 2024-09-20T18:53:32.551 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 145.24299948199999, 2024-09-20T18:53:32.551 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17952103699999999, 2024-09-20T18:53:32.551 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:53:32.551 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:53:32.551 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:32.551 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:53:32.551 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:53:32.551 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:32.551 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25333 2024-09-20T18:53:32.551 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.551 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:53:32.552 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.552 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:53:32.552 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266192+0000", 2024-09-20T18:53:32.552 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:32.552 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.552 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.552 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:53:32.552 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266192+0000", 2024-09-20T18:53:32.552 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:32.552 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.552 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.552 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:53:32.553 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266193+0000", 2024-09-20T18:53:32.553 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0999999999999998e-07 2024-09-20T18:53:32.553 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.553 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.553 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:53:32.553 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266211+0000", 2024-09-20T18:53:32.553 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8587e-05 2024-09-20T18:53:32.553 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.553 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.553 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:53:32.553 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266218+0000", 2024-09-20T18:53:32.553 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7340000000000002e-06 2024-09-20T18:53:32.553 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.554 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.554 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:53:32.554 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266224+0000", 2024-09-20T18:53:32.554 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6590000000000004e-06 2024-09-20T18:53:32.554 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.554 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.554 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:53:32.554 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268185+0000", 2024-09-20T18:53:32.554 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0019615240000000001 2024-09-20T18:53:32.554 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.554 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.554 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:53:32.554 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268326+0000", 2024-09-20T18:53:32.554 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014032399999999999 2024-09-20T18:53:32.555 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.555 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.555 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T18:53:32.555 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268450+0000", 2024-09-20T18:53:32.555 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012426599999999999 2024-09-20T18:53:32.555 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.555 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.555 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:32.555 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444991+0000", 2024-09-20T18:53:32.555 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17654108199999999 2024-09-20T18:53:32.555 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.555 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.556 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:32.556 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445507+0000", 2024-09-20T18:53:32.556 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00051583599999999999 2024-09-20T18:53:32.556 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.556 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.556 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:53:32.556 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445645+0000", 2024-09-20T18:53:32.556 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000137959 2024-09-20T18:53:32.556 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.556 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.556 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:53:32.556 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445682+0000", 2024-09-20T18:53:32.557 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7455e-05 2024-09-20T18:53:32.557 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.557 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.557 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:53:32.557 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445713+0000", 2024-09-20T18:53:32.557 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1000999999999997e-05 2024-09-20T18:53:32.557 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:32.557 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:53:32.557 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:32.557 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.557 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.557 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:25336 7.2 7:4be582b4:::benchmark_data_smithi083_70643_object25335:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:53:32.557 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.267773+0000", 2024-09-20T18:53:32.557 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 145.24141862799999, 2024-09-20T18:53:32.558 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.18517899500000001, 2024-09-20T18:53:32.558 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:53:32.558 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:53:32.558 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:32.558 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:53:32.558 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:53:32.558 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:32.558 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25336 2024-09-20T18:53:32.558 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.558 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:53:32.558 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.559 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:53:32.559 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.267773+0000", 2024-09-20T18:53:32.559 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:32.559 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.559 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.559 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:53:32.559 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.267773+0000", 2024-09-20T18:53:32.559 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:32.559 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.559 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.559 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:53:32.560 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.267774+0000", 2024-09-20T18:53:32.560 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.13e-06 2024-09-20T18:53:32.560 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.560 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.560 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:53:32.560 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.267797+0000", 2024-09-20T18:53:32.560 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2852000000000001e-05 2024-09-20T18:53:32.560 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.560 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.560 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:53:32.560 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.267806+0000", 2024-09-20T18:53:32.560 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4519999999999997e-06 2024-09-20T18:53:32.561 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.561 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.561 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:53:32.561 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.267812+0000", 2024-09-20T18:53:32.561 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3489999999999997e-06 2024-09-20T18:53:32.561 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.561 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.561 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:53:32.561 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270828+0000", 2024-09-20T18:53:32.561 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0030158020000000002 2024-09-20T18:53:32.561 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.561 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.561 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:53:32.562 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443376+0000", 2024-09-20T18:53:32.562 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17254844 2024-09-20T18:53:32.562 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.562 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.562 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T18:53:32.562 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443495+0000", 2024-09-20T18:53:32.562 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000119338 2024-09-20T18:53:32.562 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.562 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.562 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:53:32.562 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.446411+0000", 2024-09-20T18:53:32.563 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0029155869999999999 2024-09-20T18:53:32.563 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.563 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.563 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:32.563 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.452802+0000", 2024-09-20T18:53:32.563 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063913169999999997 2024-09-20T18:53:32.563 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.563 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.563 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:32.563 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.452870+0000", 2024-09-20T18:53:32.564 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7824999999999995e-05 2024-09-20T18:53:32.564 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.564 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.564 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:53:32.564 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.452915+0000", 2024-09-20T18:53:32.564 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4815999999999999e-05 2024-09-20T18:53:32.564 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.564 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.564 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:53:32.564 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.452952+0000", 2024-09-20T18:53:32.565 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7086999999999999e-05 2024-09-20T18:53:32.565 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:32.565 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:53:32.565 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:32.565 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.565 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.565 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:25335 7.6 e93/83 7:63a06433:::benchmark_data_smithi083_70643_object25334:head v 93'1581, mlcod=93'1579)", 2024-09-20T18:53:32.565 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.268963+0000", 2024-09-20T18:53:32.565 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 145.240228839, 2024-09-20T18:53:32.565 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.173429424, 2024-09-20T18:53:32.566 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:53:32.566 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:53:32.566 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:32.566 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:53:32.566 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.566 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:53:32.566 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268963+0000", 2024-09-20T18:53:32.566 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:32.566 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.566 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.566 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:53:32.567 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268963+0000", 2024-09-20T18:53:32.567 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:32.567 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.567 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.567 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:53:32.567 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268964+0000", 2024-09-20T18:53:32.567 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8800000000000001e-07 2024-09-20T18:53:32.567 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.567 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.567 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:53:32.568 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268994+0000", 2024-09-20T18:53:32.568 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0167000000000001e-05 2024-09-20T18:53:32.568 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.568 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.568 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:53:32.568 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269003+0000", 2024-09-20T18:53:32.568 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4709999999999996e-06 2024-09-20T18:53:32.568 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.568 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.568 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:53:32.568 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269109+0000", 2024-09-20T18:53:32.569 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000105831 2024-09-20T18:53:32.569 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.569 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.569 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:53:32.569 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269160+0000", 2024-09-20T18:53:32.569 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0466000000000001e-05 2024-09-20T18:53:32.569 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.569 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.569 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:53:32.569 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269178+0000", 2024-09-20T18:53:32.570 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8258e-05 2024-09-20T18:53:32.570 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.570 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.570 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:53:32.570 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270796+0000", 2024-09-20T18:53:32.570 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001618445 2024-09-20T18:53:32.570 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.570 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.570 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:53:32.570 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.442392+0000", 2024-09-20T18:53:32.571 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.171595898 2024-09-20T18:53:32.571 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:32.571 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:53:32.571 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:32.571 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.571 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.571 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:25337 7.f e93/83 7:fd2b3e49:::benchmark_data_smithi083_70643_object25336:head v 93'1586, mlcod=93'1584)", 2024-09-20T18:53:32.571 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.269129+0000", 2024-09-20T18:53:32.571 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 145.24006231600001, 2024-09-20T18:53:32.571 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.174986899, 2024-09-20T18:53:32.572 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:53:32.572 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:53:32.572 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:32.572 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:53:32.572 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.572 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:53:32.572 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269129+0000", 2024-09-20T18:53:32.572 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:32.572 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.572 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.572 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:53:32.573 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269129+0000", 2024-09-20T18:53:32.573 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:32.573 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.573 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.573 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:53:32.573 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269130+0000", 2024-09-20T18:53:32.573 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8000000000000001e-07 2024-09-20T18:53:32.573 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.573 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.573 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:53:32.573 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269151+0000", 2024-09-20T18:53:32.574 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1593000000000001e-05 2024-09-20T18:53:32.574 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.574 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.574 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:53:32.574 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269158+0000", 2024-09-20T18:53:32.574 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1069999999999999e-06 2024-09-20T18:53:32.574 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.574 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.574 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:53:32.574 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269163+0000", 2024-09-20T18:53:32.575 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7029999999999997e-06 2024-09-20T18:53:32.575 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.575 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.575 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:53:32.575 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269349+0000", 2024-09-20T18:53:32.575 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000185331 2024-09-20T18:53:32.575 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.575 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.575 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:53:32.575 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269365+0000", 2024-09-20T18:53:32.576 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6424000000000001e-05 2024-09-20T18:53:32.576 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.576 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.576 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:53:32.576 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271621+0000", 2024-09-20T18:53:32.576 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0022557200000000001 2024-09-20T18:53:32.576 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.576 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.576 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:53:32.576 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444116+0000", 2024-09-20T18:53:32.576 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17249564100000001 2024-09-20T18:53:32.577 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:32.577 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:53:32.577 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:32.577 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.577 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.577 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:25340 7.6 e93/83 7:6dd1b43c:::benchmark_data_smithi083_70643_object25339:head v 93'1582, mlcod=93'1579)", 2024-09-20T18:53:32.577 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.269179+0000", 2024-09-20T18:53:32.577 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 145.24001247499999, 2024-09-20T18:53:32.577 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.173948506, 2024-09-20T18:53:32.577 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:53:32.577 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:53:32.579 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:32.579 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:53:32.579 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.579 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:53:32.579 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269179+0000", 2024-09-20T18:53:32.579 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:32.579 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.579 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.579 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:53:32.579 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269179+0000", 2024-09-20T18:53:32.579 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:32.580 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.580 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.580 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:53:32.580 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269179+0000", 2024-09-20T18:53:32.580 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.48e-07 2024-09-20T18:53:32.580 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.580 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.580 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:53:32.580 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269198+0000", 2024-09-20T18:53:32.580 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8972999999999999e-05 2024-09-20T18:53:32.580 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.580 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.580 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:53:32.580 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269204+0000", 2024-09-20T18:53:32.581 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0440000000000003e-06 2024-09-20T18:53:32.581 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.581 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.581 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:53:32.581 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269207+0000", 2024-09-20T18:53:32.581 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6789999999999998e-06 2024-09-20T18:53:32.581 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.599 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.599 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:53:32.599 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270103+0000", 2024-09-20T18:53:32.599 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00089533500000000003 2024-09-20T18:53:32.599 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.600 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.600 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:53:32.600 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270120+0000", 2024-09-20T18:53:32.600 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7455000000000001e-05 2024-09-20T18:53:32.600 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.600 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.600 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:53:32.600 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443096+0000", 2024-09-20T18:53:32.600 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17297637699999999 2024-09-20T18:53:32.600 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.600 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.600 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:53:32.601 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443128+0000", 2024-09-20T18:53:32.601 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1395e-05 2024-09-20T18:53:32.601 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:32.601 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:53:32.601 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:32.601 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.601 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.601 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:25341 7.a 7:5bb50a52:::benchmark_data_smithi083_70643_object25340:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:53:32.601 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.269273+0000", 2024-09-20T18:53:32.602 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 145.23991824500001, 2024-09-20T18:53:32.602 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.18597625000000001, 2024-09-20T18:53:32.602 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:53:32.602 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:53:32.602 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:32.602 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:53:32.602 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:53:32.602 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:32.602 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25341 2024-09-20T18:53:32.602 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.602 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:53:32.602 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.603 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:53:32.603 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269273+0000", 2024-09-20T18:53:32.603 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:32.603 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.603 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.603 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:53:32.603 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269273+0000", 2024-09-20T18:53:32.603 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:32.603 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.603 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.603 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:53:32.603 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269274+0000", 2024-09-20T18:53:32.604 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0139999999999999e-06 2024-09-20T18:53:32.604 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.604 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.604 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:53:32.604 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269293+0000", 2024-09-20T18:53:32.604 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8499e-05 2024-09-20T18:53:32.604 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.604 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.604 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:53:32.604 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269301+0000", 2024-09-20T18:53:32.604 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8429999999999993e-06 2024-09-20T18:53:32.604 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.605 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.605 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:53:32.605 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269306+0000", 2024-09-20T18:53:32.605 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2809999999999999e-06 2024-09-20T18:53:32.605 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.605 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.605 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:53:32.605 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445057+0000", 2024-09-20T18:53:32.605 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17575099 2024-09-20T18:53:32.605 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.605 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.606 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:53:32.606 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445166+0000", 2024-09-20T18:53:32.606 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000109335 2024-09-20T18:53:32.606 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.606 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.606 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,5", 2024-09-20T18:53:32.606 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445293+0000", 2024-09-20T18:53:32.606 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000126995 2024-09-20T18:53:32.606 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.606 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.606 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:32.607 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.452767+0000", 2024-09-20T18:53:32.607 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074736009999999999 2024-09-20T18:53:32.607 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.607 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.607 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:32.607 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.452832+0000", 2024-09-20T18:53:32.607 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4817e-05 2024-09-20T18:53:32.607 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.607 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.607 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:53:32.607 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.455135+0000", 2024-09-20T18:53:32.607 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0023034459999999998 2024-09-20T18:53:32.608 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.608 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.608 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:53:32.608 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.455211+0000", 2024-09-20T18:53:32.608 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5896999999999995e-05 2024-09-20T18:53:32.608 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.608 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.608 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:53:32.608 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.455250+0000", 2024-09-20T18:53:32.608 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8532000000000003e-05 2024-09-20T18:53:32.608 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:32.608 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:53:32.609 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:32.609 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.609 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.609 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:25327 7.3 e93/83 ondisk, result = 0)", 2024-09-20T18:53:32.609 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.270007+0000", 2024-09-20T18:53:32.609 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 145.239184341, 2024-09-20T18:53:32.609 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17314204, 2024-09-20T18:53:32.609 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:53:32.609 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:53:32.609 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:53:32.609 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:53:32.609 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.609 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:53:32.610 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270007+0000", 2024-09-20T18:53:32.610 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:32.610 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.610 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.610 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:53:32.610 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270007+0000", 2024-09-20T18:53:32.610 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:32.610 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.610 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.610 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:53:32.610 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270008+0000", 2024-09-20T18:53:32.611 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4300000000000002e-07 2024-09-20T18:53:32.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.611 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.611 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:53:32.611 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270013+0000", 2024-09-20T18:53:32.611 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7230000000000004e-06 2024-09-20T18:53:32.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.611 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.611 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:53:32.611 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270023+0000", 2024-09-20T18:53:32.611 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9520000000000006e-06 2024-09-20T18:53:32.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.612 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.612 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:53:32.612 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270030+0000", 2024-09-20T18:53:32.612 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1799999999999999e-06 2024-09-20T18:53:32.612 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.612 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.612 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:53:32.612 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443024+0000", 2024-09-20T18:53:32.612 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17299360799999999 2024-09-20T18:53:32.612 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.612 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.613 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:53:32.613 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443032+0000", 2024-09-20T18:53:32.613 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6009999999999995e-06 2024-09-20T18:53:32.613 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.613 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.613 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:53:32.613 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443149+0000", 2024-09-20T18:53:32.613 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000117133 2024-09-20T18:53:32.613 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:32.613 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:53:32.613 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:32.613 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.614 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.614 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:25329 7.3 e93/83 ondisk, result = 0)", 2024-09-20T18:53:32.614 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.270049+0000", 2024-09-20T18:53:32.614 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 145.23914224399999, 2024-09-20T18:53:32.614 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17321238999999999, 2024-09-20T18:53:32.614 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:53:32.614 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:53:32.614 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:53:32.614 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:53:32.614 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.614 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:53:32.615 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270049+0000", 2024-09-20T18:53:32.615 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:32.615 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.615 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.615 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:53:32.615 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270049+0000", 2024-09-20T18:53:32.615 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:32.615 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.615 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.615 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:53:32.615 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270050+0000", 2024-09-20T18:53:32.615 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1699999999999999e-07 2024-09-20T18:53:32.616 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.616 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.616 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:53:32.616 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270052+0000", 2024-09-20T18:53:32.616 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1880000000000001e-06 2024-09-20T18:53:32.616 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.616 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.616 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:53:32.616 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270058+0000", 2024-09-20T18:53:32.616 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6910000000000002e-06 2024-09-20T18:53:32.616 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.616 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.616 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:53:32.617 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270062+0000", 2024-09-20T18:53:32.617 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6840000000000004e-06 2024-09-20T18:53:32.617 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.617 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.617 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:53:32.617 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443169+0000", 2024-09-20T18:53:32.617 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.173106484 2024-09-20T18:53:32.617 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.617 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.617 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:53:32.617 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443175+0000", 2024-09-20T18:53:32.617 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5509999999999996e-06 2024-09-20T18:53:32.618 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.618 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.618 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:53:32.618 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443262+0000", 2024-09-20T18:53:32.618 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6475000000000001e-05 2024-09-20T18:53:32.618 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:32.618 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:53:32.618 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:32.618 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.618 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.618 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:25332 7.2 e93/83 ondisk, result = 0)", 2024-09-20T18:53:32.618 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.270341+0000", 2024-09-20T18:53:32.618 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 145.23885118300001, 2024-09-20T18:53:32.619 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.174621416, 2024-09-20T18:53:32.619 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:53:32.619 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:53:32.619 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:53:32.619 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:53:32.619 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.619 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:53:32.619 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270341+0000", 2024-09-20T18:53:32.619 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:32.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.619 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.619 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:53:32.620 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270341+0000", 2024-09-20T18:53:32.620 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:32.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.620 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.620 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:53:32.620 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270341+0000", 2024-09-20T18:53:32.620 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6000000000000004e-07 2024-09-20T18:53:32.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.620 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.620 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:53:32.620 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270345+0000", 2024-09-20T18:53:32.621 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.444e-06 2024-09-20T18:53:32.621 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.621 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.621 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:53:32.621 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270353+0000", 2024-09-20T18:53:32.621 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5580000000000001e-06 2024-09-20T18:53:32.621 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.621 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.621 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:53:32.621 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270359+0000", 2024-09-20T18:53:32.621 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7459999999999997e-06 2024-09-20T18:53:32.621 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.621 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.622 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:53:32.622 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444944+0000", 2024-09-20T18:53:32.622 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.174585351 2024-09-20T18:53:32.622 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.622 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.622 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:53:32.622 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444950+0000", 2024-09-20T18:53:32.622 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0530000000000003e-06 2024-09-20T18:53:32.622 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.622 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.622 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:53:32.622 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444962+0000", 2024-09-20T18:53:32.623 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1704e-05 2024-09-20T18:53:32.623 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:32.623 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:53:32.623 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:32.623 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.623 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.623 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:25333 7.2 e93/83 ondisk, result = 0)", 2024-09-20T18:53:32.623 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.270458+0000", 2024-09-20T18:53:32.623 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 145.23873379099999, 2024-09-20T18:53:32.623 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17453908100000001, 2024-09-20T18:53:32.623 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:53:32.623 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:53:32.623 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:53:32.624 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:53:32.624 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.624 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:53:32.624 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270458+0000", 2024-09-20T18:53:32.624 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:32.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.624 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.624 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:53:32.624 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270458+0000", 2024-09-20T18:53:32.624 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:32.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.624 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.624 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:53:32.625 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270458+0000", 2024-09-20T18:53:32.625 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3499999999999996e-07 2024-09-20T18:53:32.625 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.625 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.625 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:53:32.625 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270462+0000", 2024-09-20T18:53:32.625 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1499999999999999e-06 2024-09-20T18:53:32.625 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.625 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.625 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:53:32.625 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270469+0000", 2024-09-20T18:53:32.625 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.695e-06 2024-09-20T18:53:32.626 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.626 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.626 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:53:32.626 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270475+0000", 2024-09-20T18:53:32.626 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5470000000000004e-06 2024-09-20T18:53:32.626 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.626 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.626 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:53:32.626 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444981+0000", 2024-09-20T18:53:32.626 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17450622299999999 2024-09-20T18:53:32.626 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.626 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.626 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:53:32.627 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444987+0000", 2024-09-20T18:53:32.627 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5219999999999998e-06 2024-09-20T18:53:32.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.627 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.627 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:53:32.627 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444997+0000", 2024-09-20T18:53:32.627 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0409e-05 2024-09-20T18:53:32.627 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:32.627 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:53:32.627 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:32.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.627 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.628 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:25332 7.2 e93/83 ondisk, result = 0)", 2024-09-20T18:53:32.628 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.270547+0000", 2024-09-20T18:53:32.628 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 145.238645109, 2024-09-20T18:53:32.628 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.174485849, 2024-09-20T18:53:32.628 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:53:32.628 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:53:32.628 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:53:32.628 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:53:32.628 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.628 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:53:32.628 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270547+0000", 2024-09-20T18:53:32.628 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:32.629 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.629 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.629 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:53:32.629 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270547+0000", 2024-09-20T18:53:32.629 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:32.629 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.629 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.629 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:53:32.629 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270547+0000", 2024-09-20T18:53:32.629 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7599999999999997e-07 2024-09-20T18:53:32.629 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.629 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.629 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:53:32.630 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270550+0000", 2024-09-20T18:53:32.630 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7470000000000001e-06 2024-09-20T18:53:32.630 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.630 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.630 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:53:32.630 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270557+0000", 2024-09-20T18:53:32.630 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2620000000000001e-06 2024-09-20T18:53:32.630 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.630 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.630 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:53:32.631 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270562+0000", 2024-09-20T18:53:32.631 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1839999999999998e-06 2024-09-20T18:53:32.631 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.631 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.631 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:53:32.631 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445016+0000", 2024-09-20T18:53:32.631 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17445396699999999 2024-09-20T18:53:32.631 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.631 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.631 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:53:32.631 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445022+0000", 2024-09-20T18:53:32.631 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7559999999999996e-06 2024-09-20T18:53:32.632 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.632 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.632 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:53:32.632 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445032+0000", 2024-09-20T18:53:32.632 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0457e-05 2024-09-20T18:53:32.632 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:32.632 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:53:32.632 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:32.632 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.632 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.632 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:25343 7.6 e93/83 7:61f2dd3b:::benchmark_data_smithi083_70643_object25342:head v 93'1583, mlcod=93'1579)", 2024-09-20T18:53:32.632 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.271493+0000", 2024-09-20T18:53:32.632 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 145.23769904100001, 2024-09-20T18:53:32.633 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17491087999999999, 2024-09-20T18:53:32.633 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:53:32.633 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:53:32.633 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:32.633 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:53:32.633 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.633 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:53:32.633 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271493+0000", 2024-09-20T18:53:32.633 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:32.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.633 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.633 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:53:32.634 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271493+0000", 2024-09-20T18:53:32.634 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:32.634 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.634 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.634 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:53:32.634 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271493+0000", 2024-09-20T18:53:32.634 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8400000000000004e-07 2024-09-20T18:53:32.634 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.634 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.634 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:53:32.634 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271513+0000", 2024-09-20T18:53:32.634 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0126999999999999e-05 2024-09-20T18:53:32.634 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.635 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.635 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:53:32.635 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443960+0000", 2024-09-20T18:53:32.635 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.172446983 2024-09-20T18:53:32.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.635 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.635 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:53:32.635 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443970+0000", 2024-09-20T18:53:32.635 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5119999999999997e-06 2024-09-20T18:53:32.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.635 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.635 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:53:32.635 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444547+0000", 2024-09-20T18:53:32.636 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00057698700000000005 2024-09-20T18:53:32.636 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.636 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.636 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:53:32.636 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444578+0000", 2024-09-20T18:53:32.636 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0956e-05 2024-09-20T18:53:32.636 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.636 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.636 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:53:32.636 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.446361+0000", 2024-09-20T18:53:32.636 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001783473 2024-09-20T18:53:32.637 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.637 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.637 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:53:32.637 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.446404+0000", 2024-09-20T18:53:32.637 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2157999999999999e-05 2024-09-20T18:53:32.637 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:32.637 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:53:32.637 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:32.637 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.637 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.637 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:25344 7.7 e93/83 7:ead30b22:::benchmark_data_smithi083_70643_object25343:head v 93'1577, mlcod=93'1575)", 2024-09-20T18:53:32.637 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.271541+0000", 2024-09-20T18:53:32.638 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 145.23765066999999, 2024-09-20T18:53:32.638 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17264948199999999, 2024-09-20T18:53:32.638 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:53:32.638 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:53:32.638 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:32.638 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:53:32.638 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.638 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:53:32.638 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271541+0000", 2024-09-20T18:53:32.638 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:32.638 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.638 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.639 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:53:32.639 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271541+0000", 2024-09-20T18:53:32.639 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:32.639 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.639 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.639 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:53:32.639 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271541+0000", 2024-09-20T18:53:32.639 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1300000000000001e-07 2024-09-20T18:53:32.639 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.639 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.639 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:53:32.640 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271571+0000", 2024-09-20T18:53:32.640 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9949000000000001e-05 2024-09-20T18:53:32.640 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.640 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.640 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:53:32.640 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.442465+0000", 2024-09-20T18:53:32.640 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.170894087 2024-09-20T18:53:32.640 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.640 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.640 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:53:32.640 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.442479+0000", 2024-09-20T18:53:32.640 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.309e-05 2024-09-20T18:53:32.641 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.641 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.641 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:53:32.641 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.442626+0000", 2024-09-20T18:53:32.641 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014740399999999999 2024-09-20T18:53:32.641 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.641 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.641 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:53:32.641 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.442649+0000", 2024-09-20T18:53:32.641 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3107000000000001e-05 2024-09-20T18:53:32.641 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.641 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.641 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:53:32.642 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444120+0000", 2024-09-20T18:53:32.642 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001471354 2024-09-20T18:53:32.642 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.642 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.642 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:53:32.642 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444191+0000", 2024-09-20T18:53:32.642 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0078000000000002e-05 2024-09-20T18:53:32.642 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:32.642 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:53:32.642 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:32.642 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.642 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.643 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:26458 7.2 7:4defa689:::benchmark_data_smithi083_70643_object26457:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:53:32.643 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.993077+0000", 2024-09-20T18:53:32.643 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 144.51611453300001, 2024-09-20T18:53:32.643 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.139540674, 2024-09-20T18:53:32.643 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:53:32.643 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:53:32.643 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:32.643 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:53:32.643 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:53:32.643 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:32.643 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 26458 2024-09-20T18:53:32.643 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.644 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:53:32.644 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.644 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:53:32.644 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.993077+0000", 2024-09-20T18:53:32.644 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:32.644 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.644 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.644 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:53:32.644 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.993077+0000", 2024-09-20T18:53:32.644 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:32.644 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.644 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.644 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:53:32.644 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.993078+0000", 2024-09-20T18:53:32.645 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.167e-06 2024-09-20T18:53:32.645 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.645 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.645 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:53:32.645 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.993097+0000", 2024-09-20T18:53:32.645 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8777999999999999e-05 2024-09-20T18:53:32.645 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.645 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.645 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:53:32.645 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.993107+0000", 2024-09-20T18:53:32.645 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9189999999999994e-06 2024-09-20T18:53:32.645 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.646 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.646 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:32.647 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:53:32.647 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.993114+0000", 2024-09-20T18:53:32.647 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1180000000000003e-06 2024-09-20T18:53:32.647 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.647 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.648 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:53:32.648 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.995113+0000", 2024-09-20T18:53:32.648 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001999055 2024-09-20T18:53:32.648 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.648 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.648 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:53:32.648 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.995230+0000", 2024-09-20T18:53:32.648 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011663 2024-09-20T18:53:32.648 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.648 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.648 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T18:53:32.648 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.995352+0000", 2024-09-20T18:53:32.649 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012200499999999999 2024-09-20T18:53:32.649 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.649 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.649 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:32.649 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.998898+0000", 2024-09-20T18:53:32.649 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0035456929999999999 2024-09-20T18:53:32.649 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.649 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.649 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:32.649 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.000083+0000", 2024-09-20T18:53:32.649 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001185834 2024-09-20T18:53:32.649 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.650 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.650 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:53:32.650 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.132527+0000", 2024-09-20T18:53:32.650 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.132443637 2024-09-20T18:53:32.650 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.650 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.650 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:53:32.650 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.132576+0000", 2024-09-20T18:53:32.650 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8885999999999999e-05 2024-09-20T18:53:32.650 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.650 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.651 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:53:32.651 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.132618+0000", 2024-09-20T18:53:32.651 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1952000000000001e-05 2024-09-20T18:53:32.651 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:32.651 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:53:32.651 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:32.651 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.651 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.651 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:26459 7.a 7:585d31b8:::benchmark_data_smithi083_70643_object26458:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:53:32.651 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.993130+0000", 2024-09-20T18:53:32.651 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 144.51606150200001, 2024-09-20T18:53:32.651 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.13957813499999999, 2024-09-20T18:53:32.651 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:53:32.652 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:53:32.652 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:32.652 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:53:32.652 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:53:32.652 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:32.652 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 26459 2024-09-20T18:53:32.652 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.652 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:53:32.652 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.652 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:53:32.652 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.993130+0000", 2024-09-20T18:53:32.652 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:32.652 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.653 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.653 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:53:32.653 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.993130+0000", 2024-09-20T18:53:32.653 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:32.653 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.653 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.653 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:53:32.653 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.993131+0000", 2024-09-20T18:53:32.653 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2900000000000003e-07 2024-09-20T18:53:32.653 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.653 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.653 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:53:32.653 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.993152+0000", 2024-09-20T18:53:32.654 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1019e-05 2024-09-20T18:53:32.654 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.654 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.654 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:53:32.654 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.993159+0000", 2024-09-20T18:53:32.654 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0999999999999998e-06 2024-09-20T18:53:32.654 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.654 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.654 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:53:32.654 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.993165+0000", 2024-09-20T18:53:32.654 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6500000000000001e-06 2024-09-20T18:53:32.654 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.654 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.655 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:53:32.655 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.998927+0000", 2024-09-20T18:53:32.655 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0057625829999999999 2024-09-20T18:53:32.655 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.655 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.655 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:53:32.655 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.999033+0000", 2024-09-20T18:53:32.655 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010542100000000001 2024-09-20T18:53:32.655 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.655 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.655 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,5", 2024-09-20T18:53:32.655 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.999150+0000", 2024-09-20T18:53:32.656 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000117345 2024-09-20T18:53:32.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.656 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.656 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:32.656 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.131620+0000", 2024-09-20T18:53:32.656 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.13246966499999999 2024-09-20T18:53:32.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.656 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.656 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:32.656 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.132508+0000", 2024-09-20T18:53:32.656 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00088801799999999997 2024-09-20T18:53:32.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.656 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.656 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:53:32.657 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.132644+0000", 2024-09-20T18:53:32.657 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000136422 2024-09-20T18:53:32.657 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.657 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.657 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:53:32.657 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.132680+0000", 2024-09-20T18:53:32.657 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6183999999999998e-05 2024-09-20T18:53:32.657 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.657 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.657 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:53:32.657 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.132708+0000", 2024-09-20T18:53:32.657 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8099e-05 2024-09-20T18:53:32.658 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:32.658 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:53:32.658 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:32.658 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.658 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.658 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:26460 7.3 7:cd1ed93e:::benchmark_data_smithi083_70643_object26459:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:53:32.658 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.994908+0000", 2024-09-20T18:53:32.658 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 144.514284172, 2024-09-20T18:53:32.658 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.137972241, 2024-09-20T18:53:32.658 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:53:32.658 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:53:32.658 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:32.659 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:53:32.659 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:53:32.659 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:32.659 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 26460 2024-09-20T18:53:32.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.659 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:53:32.659 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.659 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:53:32.659 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.994908+0000", 2024-09-20T18:53:32.659 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:32.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.659 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.660 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:53:32.660 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.994908+0000", 2024-09-20T18:53:32.660 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:32.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.660 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.660 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:53:32.660 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.994909+0000", 2024-09-20T18:53:32.660 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.246e-06 2024-09-20T18:53:32.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.660 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.660 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:53:32.660 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.994935+0000", 2024-09-20T18:53:32.660 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.603e-05 2024-09-20T18:53:32.661 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.661 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.661 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:53:32.661 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.994943+0000", 2024-09-20T18:53:32.661 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2679999999999994e-06 2024-09-20T18:53:32.661 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.661 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.661 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:53:32.661 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.994948+0000", 2024-09-20T18:53:32.661 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6800000000000001e-06 2024-09-20T18:53:32.661 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.662 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.662 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:53:32.662 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.994996+0000", 2024-09-20T18:53:32.662 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8452999999999998e-05 2024-09-20T18:53:32.662 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.662 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.662 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:53:32.662 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.995099+0000", 2024-09-20T18:53:32.662 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000102921 2024-09-20T18:53:32.662 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.662 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.662 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,4", 2024-09-20T18:53:32.662 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.995384+0000", 2024-09-20T18:53:32.663 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00028438699999999999 2024-09-20T18:53:32.663 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.663 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.663 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:32.663 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.998671+0000", 2024-09-20T18:53:32.663 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0032877560000000002 2024-09-20T18:53:32.663 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.663 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.663 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:32.663 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.998703+0000", 2024-09-20T18:53:32.663 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2107000000000003e-05 2024-09-20T18:53:32.663 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.663 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.664 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:53:32.664 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.132742+0000", 2024-09-20T18:53:32.664 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.13403827200000001 2024-09-20T18:53:32.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.664 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.664 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:53:32.664 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.132782+0000", 2024-09-20T18:53:32.664 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9888999999999997e-05 2024-09-20T18:53:32.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.664 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.664 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:53:32.664 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.132880+0000", 2024-09-20T18:53:32.665 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8232000000000004e-05 2024-09-20T18:53:32.665 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:32.665 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:53:32.665 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:32.665 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.665 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.665 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:26461 7.2 7:469ac956:::benchmark_data_smithi083_70643_object26460:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:53:32.665 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.994961+0000", 2024-09-20T18:53:32.665 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 144.51423038799999, 2024-09-20T18:53:32.665 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.13992532699999999, 2024-09-20T18:53:32.665 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:53:32.665 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:53:32.666 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:32.666 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:53:32.666 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:53:32.666 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:32.666 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 26461 2024-09-20T18:53:32.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.666 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:53:32.666 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.666 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:53:32.666 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.994961+0000", 2024-09-20T18:53:32.666 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:32.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.666 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.667 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:53:32.667 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.994961+0000", 2024-09-20T18:53:32.667 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:53:32.667 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.667 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.667 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:53:32.667 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.994962+0000", 2024-09-20T18:53:32.667 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7399999999999999e-07 2024-09-20T18:53:32.667 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.667 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.667 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:53:32.667 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.994978+0000", 2024-09-20T18:53:32.668 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6024000000000001e-05 2024-09-20T18:53:32.668 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.668 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.668 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:53:32.668 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.994982+0000", 2024-09-20T18:53:32.668 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1080000000000003e-06 2024-09-20T18:53:32.668 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.668 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.668 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:53:32.668 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.994985+0000", 2024-09-20T18:53:32.668 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.084e-06 2024-09-20T18:53:32.668 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.668 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.669 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:53:32.669 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.131132+0000", 2024-09-20T18:53:32.669 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.13614691900000001 2024-09-20T18:53:32.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.669 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.669 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:53:32.669 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.131257+0000", 2024-09-20T18:53:32.669 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012540699999999999 2024-09-20T18:53:32.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.669 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.669 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T18:53:32.669 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.131387+0000", 2024-09-20T18:53:32.669 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012971700000000001 2024-09-20T18:53:32.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.670 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.670 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:53:32.670 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.132716+0000", 2024-09-20T18:53:32.670 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001329507 2024-09-20T18:53:32.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.670 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.670 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:32.670 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.133133+0000", 2024-09-20T18:53:32.670 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00041698400000000002 2024-09-20T18:53:32.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.671 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.671 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:32.671 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.134801+0000", 2024-09-20T18:53:32.671 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001667511 2024-09-20T18:53:32.671 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.671 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.671 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:53:32.671 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.134845+0000", 2024-09-20T18:53:32.671 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3884000000000002e-05 2024-09-20T18:53:32.671 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:53:32.671 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:53:32.671 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:53:32.671 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.134887+0000", 2024-09-20T18:53:32.671 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1807999999999997e-05 2024-09-20T18:53:32.672 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:32.672 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:53:32.672 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:32.672 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:53:32.672 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:53:32.672 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T18:53:32.709 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:32.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:32.710 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:32.811 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:32.837 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:32.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:32.838 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:32.939 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:32.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:32.978 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:32.978 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:33.079 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:33.105 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:33.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:33.106 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:33.206 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:33.232 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:33.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:33.233 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:33.334 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:53:33.361 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:33.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:33.362 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:53:33.372 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T18:53:33.464 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:53:33.489 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:33.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:33.490 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:53:33.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:33 smithi083 ceph-mon[29421]: osdmap e142: 8 total, 8 up, 8 in 2024-09-20T18:53:33.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:33 smithi083 ceph-mon[36260]: osdmap e142: 8 total, 8 up, 8 in 2024-09-20T18:53:33.591 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:33.631 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-20T18:53:33.649 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:33.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:33.650 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:33.656 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:33 smithi142 ceph-mon[32508]: osdmap e142: 8 total, 8 up, 8 in 2024-09-20T18:53:33.750 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:33.785 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:33.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:33.788 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:33.889 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:33.906 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T18:53:33.920 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:33.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:33.921 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:33.963 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:53:34.021 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:34.046 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:34.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:34.047 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:34.070 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:53:34.148 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:53:34.172 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:34.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:34.173 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:53:34.274 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:53:34.300 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.5 to 1 (10000) 2024-09-20T18:53:34.335 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:34.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:34.336 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:53:34.341 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:34 smithi142 ceph-mon[32508]: pgmap v170: 148 pgs: 148 active+clean; 3.4 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.27k op/s 2024-09-20T18:53:34.341 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:34 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2084365632' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-20T18:53:34.437 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:34.479 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:34.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:34.480 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:34.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:34 smithi083 ceph-mon[29421]: pgmap v170: 148 pgs: 148 active+clean; 3.4 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.27k op/s 2024-09-20T18:53:34.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:34 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2084365632' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-20T18:53:34.554 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:34 smithi083 ceph-mon[36260]: pgmap v170: 148 pgs: 148 active+clean; 3.4 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.27k op/s 2024-09-20T18:53:34.554 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:34 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2084365632' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-20T18:53:34.581 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:34.592 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T18:53:34.592 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-09-20T18:53:34.592 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:34.592 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.14808.0:193527 7.9 e143/83 7:9536fb53:::benchmark_data_smithi083_70643_object70725:head v 143'11990, mlcod=143'11988)", 2024-09-20T18:53:34.592 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:53:34.571676+0000", 2024-09-20T18:53:34.592 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.0028625809999999999, 2024-09-20T18:53:34.593 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0028885289999999999, 2024-09-20T18:53:34.593 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:53:34.593 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:53:34.593 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-09-20T18:53:34.593 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:53:34.593 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:34.593 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:53:34.593 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:34.571676+0000", 2024-09-20T18:53:34.593 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:34.593 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:34.593 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:34.593 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:53:34.593 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:34.571676+0000", 2024-09-20T18:53:34.594 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:34.594 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:34.594 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:34.594 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:53:34.594 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:34.571676+0000", 2024-09-20T18:53:34.594 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3099999999999998e-07 2024-09-20T18:53:34.594 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:34.594 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:34.594 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:53:34.594 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:34.571680+0000", 2024-09-20T18:53:34.594 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9400000000000004e-06 2024-09-20T18:53:34.594 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:34.594 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:34.594 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:53:34.595 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:34.571688+0000", 2024-09-20T18:53:34.595 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.0139999999999998e-06 2024-09-20T18:53:34.595 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:34.595 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:34.595 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:53:34.595 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:34.571694+0000", 2024-09-20T18:53:34.595 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8440000000000003e-06 2024-09-20T18:53:34.595 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:34.595 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:34.595 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:53:34.595 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:34.571749+0000", 2024-09-20T18:53:34.595 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4440000000000001e-05 2024-09-20T18:53:34.595 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:34.595 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:34.596 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:53:34.596 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:34.571779+0000", 2024-09-20T18:53:34.596 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.0185e-05 2024-09-20T18:53:34.596 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:34.596 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:53:34.596 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:34.596 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:34.596 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-09-20T18:53:34.596 INFO:teuthology.orchestra.run.smithi142.stdout: "num_ops": 1 2024-09-20T18:53:34.596 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T18:53:34.615 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:34.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:34.616 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:34.717 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:34.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:34.750 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:34.750 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:34.851 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:34.879 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:34.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:34.880 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:34.981 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:35.031 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:35.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:35.035 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:35.136 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:35.164 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:35.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:35.165 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:35.265 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:35.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:35.298 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:35.298 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:35.317 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-20T18:53:35.378 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_blocked_ops 2024-09-20T18:53:35.390 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd reweight 6 1 2024-09-20T18:53:35.399 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:35.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:35.465 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:35.466 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:35.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:35 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2084365632' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-20T18:53:35.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:35 smithi142 ceph-mon[32508]: osdmap e143: 8 total, 8 up, 8 in 2024-09-20T18:53:35.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:35 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3201908620' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:53:35.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:35 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2084365632' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-20T18:53:35.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:35 smithi083 ceph-mon[29421]: osdmap e143: 8 total, 8 up, 8 in 2024-09-20T18:53:35.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:35 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3201908620' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:53:35.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:35 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2084365632' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-20T18:53:35.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:35 smithi083 ceph-mon[36260]: osdmap e143: 8 total, 8 up, 8 in 2024-09-20T18:53:35.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:35 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3201908620' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:53:35.566 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:53:35.624 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:35.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:35.625 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:53:35.726 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:35.740 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:53:35.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:35.748 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:35.748 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:35.849 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:35.887 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:35.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:35.890 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:35.991 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:36.030 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:36.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:36.032 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:36.132 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:36.160 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:36.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:36.161 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:36.262 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:36.294 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:36.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:36.295 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:36.328 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T18:53:36.328 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-09-20T18:53:36.328 INFO:teuthology.orchestra.run.smithi142.stdout: "complaint_time": 30, 2024-09-20T18:53:36.328 INFO:teuthology.orchestra.run.smithi142.stdout: "num_blocked_ops": 0 2024-09-20T18:53:36.329 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T18:53:36.396 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:36.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:36.462 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:36.462 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:36.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:36 smithi142 ceph-mon[32508]: pgmap v172: 148 pgs: 148 active+clean; 3.4 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 750 B/s rd, 0 B/s wr, 749 op/s 2024-09-20T18:53:36.525 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:36 smithi142 ceph-mon[32508]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:53:36.525 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:36 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3201908620' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T18:53:36.525 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:36 smithi142 ceph-mon[32508]: osdmap e144: 8 total, 8 up, 8 in 2024-09-20T18:53:36.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:36 smithi083 ceph-mon[29421]: pgmap v172: 148 pgs: 148 active+clean; 3.4 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 750 B/s rd, 0 B/s wr, 749 op/s 2024-09-20T18:53:36.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:36 smithi083 ceph-mon[29421]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:53:36.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:36 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3201908620' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T18:53:36.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:36 smithi083 ceph-mon[29421]: osdmap e144: 8 total, 8 up, 8 in 2024-09-20T18:53:36.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:36 smithi083 ceph-mon[36260]: pgmap v172: 148 pgs: 148 active+clean; 3.4 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 750 B/s rd, 0 B/s wr, 749 op/s 2024-09-20T18:53:36.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:36 smithi083 ceph-mon[36260]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:53:36.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:36 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3201908620' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T18:53:36.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:36 smithi083 ceph-mon[36260]: osdmap e144: 8 total, 8 up, 8 in 2024-09-20T18:53:36.562 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:36.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:36.601 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:36.601 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:36.702 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:53:36.734 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:36.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:36.735 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:53:36.836 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:53:36.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:36.884 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:36.884 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:53:36.985 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:37.011 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:37.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:37.012 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:37.024 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:53:36 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:53:37.113 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:37.145 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_historic_ops 2024-09-20T18:53:37.147 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:37.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:37.149 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:37.249 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:37.274 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:37.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:37.275 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:37.349 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:53:37.355 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:53:37.376 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:37.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:37.426 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:37.427 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:37.516 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:53:37.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:37 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:53:37.527 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:53:37.559 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:37.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:37.560 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:53:37.602 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:37 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:53:37.605 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:37 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:53:37.661 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:37.708 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:37.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:37.709 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:37.810 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:37.840 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:37.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:37.841 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:37.941 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:37.976 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:37.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:37.978 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:38.078 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:38.086 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T18:53:38.086 INFO:teuthology.orchestra.run.smithi142.stdout: "size": 20, 2024-09-20T18:53:38.087 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 600, 2024-09-20T18:53:38.087 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-09-20T18:53:38.087 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.087 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:23109 7.1 7:8a12d0b9:::benchmark_data_smithi083_70643_object23108:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-20T18:53:38.087 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:06.089353+0000", 2024-09-20T18:53:38.087 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 151.97784168999999, 2024-09-20T18:53:38.087 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.087697153, 2024-09-20T18:53:38.087 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:53:38.087 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:53:38.087 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:38.087 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:53:38.087 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:53:38.087 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:38.087 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 23109 2024-09-20T18:53:38.088 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.088 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:53:38.088 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.088 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:53:38.088 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.089353+0000", 2024-09-20T18:53:38.088 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:38.088 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.088 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.088 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:53:38.088 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.089353+0000", 2024-09-20T18:53:38.088 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:38.088 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.088 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.088 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:53:38.089 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.089354+0000", 2024-09-20T18:53:38.089 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.358e-06 2024-09-20T18:53:38.089 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.089 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.089 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:53:38.089 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.089526+0000", 2024-09-20T18:53:38.089 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000171077 2024-09-20T18:53:38.089 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.089 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.089 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:53:38.089 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.089541+0000", 2024-09-20T18:53:38.089 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5634999999999999e-05 2024-09-20T18:53:38.089 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.089 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.090 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:53:38.090 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.089549+0000", 2024-09-20T18:53:38.090 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.8189999999999992e-06 2024-09-20T18:53:38.090 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.090 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.090 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:53:38.090 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.089602+0000", 2024-09-20T18:53:38.090 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.2988000000000002e-05 2024-09-20T18:53:38.090 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.090 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.090 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:53:38.090 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.089719+0000", 2024-09-20T18:53:38.090 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000117374 2024-09-20T18:53:38.091 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.091 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.091 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,6", 2024-09-20T18:53:38.091 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.089889+0000", 2024-09-20T18:53:38.091 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00016921300000000001 2024-09-20T18:53:38.091 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.091 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.091 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:53:38.091 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.091577+0000", 2024-09-20T18:53:38.091 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001688316 2024-09-20T18:53:38.091 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.091 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.091 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:38.091 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.092076+0000", 2024-09-20T18:53:38.092 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000498705 2024-09-20T18:53:38.092 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.092 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.092 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:38.092 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.176935+0000", 2024-09-20T18:53:38.092 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.084859749999999998 2024-09-20T18:53:38.092 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.092 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.092 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:53:38.092 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.176999+0000", 2024-09-20T18:53:38.092 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.3960000000000004e-05 2024-09-20T18:53:38.092 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.092 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.092 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:53:38.093 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.177050+0000", 2024-09-20T18:53:38.093 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0958000000000002e-05 2024-09-20T18:53:38.093 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:38.093 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:53:38.093 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:38.093 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.093 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.093 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:25818 7.1 7:8b1d7f7e:::benchmark_data_smithi083_70643_object25817:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:53:38.093 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:07.637397+0000", 2024-09-20T18:53:38.093 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 150.42979803399999, 2024-09-20T18:53:38.093 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.069413953, 2024-09-20T18:53:38.093 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:53:38.093 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:53:38.093 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:38.094 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:53:38.094 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:53:38.094 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:38.094 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 25818 2024-09-20T18:53:38.094 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.094 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:53:38.094 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.094 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:53:38.094 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.637397+0000", 2024-09-20T18:53:38.094 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:38.094 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.094 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.094 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:53:38.095 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.637397+0000", 2024-09-20T18:53:38.095 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:38.095 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.095 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.095 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:53:38.095 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.637398+0000", 2024-09-20T18:53:38.095 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.254e-06 2024-09-20T18:53:38.095 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.095 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.095 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:53:38.095 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.637441+0000", 2024-09-20T18:53:38.095 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.2985000000000003e-05 2024-09-20T18:53:38.095 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.095 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.095 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:53:38.096 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.637450+0000", 2024-09-20T18:53:38.096 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.7409999999999998e-06 2024-09-20T18:53:38.096 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.096 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.096 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:53:38.096 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.637454+0000", 2024-09-20T18:53:38.096 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.5229999999999997e-06 2024-09-20T18:53:38.096 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.096 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.096 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:53:38.096 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.637494+0000", 2024-09-20T18:53:38.096 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9595999999999998e-05 2024-09-20T18:53:38.096 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.096 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.097 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:53:38.097 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.637606+0000", 2024-09-20T18:53:38.097 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000111779 2024-09-20T18:53:38.097 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.097 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.097 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,6", 2024-09-20T18:53:38.097 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.637776+0000", 2024-09-20T18:53:38.097 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00017017900000000001 2024-09-20T18:53:38.097 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.097 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.097 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:53:38.097 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.638868+0000", 2024-09-20T18:53:38.097 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0010925830000000001 2024-09-20T18:53:38.097 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.097 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.098 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:38.098 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.639923+0000", 2024-09-20T18:53:38.098 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00105495 2024-09-20T18:53:38.098 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.098 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.098 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:38.098 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.706721+0000", 2024-09-20T18:53:38.098 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.066797422999999995 2024-09-20T18:53:38.098 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.098 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.098 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:53:38.098 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.706772+0000", 2024-09-20T18:53:38.101 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0788999999999998e-05 2024-09-20T18:53:38.102 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.102 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.102 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:53:38.102 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.706811+0000", 2024-09-20T18:53:38.102 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9150999999999998e-05 2024-09-20T18:53:38.102 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:38.102 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:53:38.102 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:38.102 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.102 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.102 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:26463 7.1 7:878a0722:::benchmark_data_smithi083_70643_object26462:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:53:38.102 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:07.994856+0000", 2024-09-20T18:53:38.102 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 150.07233853299999, 2024-09-20T18:53:38.103 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13761967, 2024-09-20T18:53:38.103 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:53:38.103 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:53:38.103 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:38.103 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:53:38.104 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:53:38.104 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:38.104 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 26463 2024-09-20T18:53:38.104 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.104 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:53:38.104 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.104 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:53:38.104 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.994856+0000", 2024-09-20T18:53:38.104 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:38.104 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.104 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.104 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:53:38.105 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.994856+0000", 2024-09-20T18:53:38.105 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:38.105 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.105 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.105 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:53:38.105 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.994858+0000", 2024-09-20T18:53:38.105 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.649e-06 2024-09-20T18:53:38.105 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.105 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.105 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:53:38.105 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.994924+0000", 2024-09-20T18:53:38.105 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.6484999999999998e-05 2024-09-20T18:53:38.105 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.106 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.106 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:53:38.106 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.994937+0000", 2024-09-20T18:53:38.106 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3084e-05 2024-09-20T18:53:38.106 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.106 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.106 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:53:38.106 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.994945+0000", 2024-09-20T18:53:38.106 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.4889999999999998e-06 2024-09-20T18:53:38.106 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.106 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.106 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:53:38.106 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.994984+0000", 2024-09-20T18:53:38.106 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9270999999999997e-05 2024-09-20T18:53:38.107 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.107 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.107 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:53:38.107 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.995073+0000", 2024-09-20T18:53:38.107 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.8747999999999998e-05 2024-09-20T18:53:38.107 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.107 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.107 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,6", 2024-09-20T18:53:38.107 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.995161+0000", 2024-09-20T18:53:38.107 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.7538000000000001e-05 2024-09-20T18:53:38.107 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.107 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.107 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:53:38.107 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.996461+0000", 2024-09-20T18:53:38.108 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0013008060000000001 2024-09-20T18:53:38.108 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.108 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.108 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:38.108 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.996788+0000", 2024-09-20T18:53:38.108 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00032643100000000002 2024-09-20T18:53:38.108 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.108 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.108 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:38.108 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.132406+0000", 2024-09-20T18:53:38.108 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13561836799999999 2024-09-20T18:53:38.108 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.108 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.109 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:53:38.109 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.132445+0000", 2024-09-20T18:53:38.109 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.8640999999999998e-05 2024-09-20T18:53:38.109 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.109 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.109 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:53:38.109 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.132476+0000", 2024-09-20T18:53:38.109 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.116e-05 2024-09-20T18:53:38.109 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:38.109 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:53:38.109 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:38.109 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.109 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.109 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:26470 7.1 7:85a6a71e:::benchmark_data_smithi083_70643_object26469:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:53:38.110 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:07.997258+0000", 2024-09-20T18:53:38.110 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 150.06993718999999, 2024-09-20T18:53:38.110 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13539621399999999, 2024-09-20T18:53:38.110 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:53:38.110 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:53:38.110 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:38.110 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:53:38.110 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:53:38.110 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:38.110 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 26470 2024-09-20T18:53:38.110 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.110 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:53:38.110 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.111 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:53:38.111 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.997258+0000", 2024-09-20T18:53:38.111 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:38.111 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.111 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.111 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:53:38.111 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.997258+0000", 2024-09-20T18:53:38.111 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:38.111 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.111 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.111 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:53:38.111 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.997259+0000", 2024-09-20T18:53:38.111 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6810000000000001e-06 2024-09-20T18:53:38.111 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.112 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.112 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:53:38.112 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.997317+0000", 2024-09-20T18:53:38.112 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8174000000000002e-05 2024-09-20T18:53:38.112 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.112 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.112 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:53:38.112 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.997332+0000", 2024-09-20T18:53:38.112 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4687e-05 2024-09-20T18:53:38.112 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.112 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.112 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:53:38.112 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.997340+0000", 2024-09-20T18:53:38.112 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.4089999999999997e-06 2024-09-20T18:53:38.112 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.113 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.113 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:53:38.113 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.997395+0000", 2024-09-20T18:53:38.113 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5318000000000001e-05 2024-09-20T18:53:38.113 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.113 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.113 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:53:38.113 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.997488+0000", 2024-09-20T18:53:38.113 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.2930000000000006e-05 2024-09-20T18:53:38.113 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.113 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.113 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,6", 2024-09-20T18:53:38.113 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.997587+0000", 2024-09-20T18:53:38.113 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.9167999999999996e-05 2024-09-20T18:53:38.113 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.114 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.114 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:53:38.114 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.998680+0000", 2024-09-20T18:53:38.114 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001092759 2024-09-20T18:53:38.114 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.114 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.114 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:38.114 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.001637+0000", 2024-09-20T18:53:38.114 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.002956991 2024-09-20T18:53:38.114 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.114 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.114 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:38.114 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.132549+0000", 2024-09-20T18:53:38.114 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13091224700000001 2024-09-20T18:53:38.115 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.115 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.115 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:53:38.115 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.132609+0000", 2024-09-20T18:53:38.115 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0123e-05 2024-09-20T18:53:38.115 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.115 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.115 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:53:38.115 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.132654+0000", 2024-09-20T18:53:38.115 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.4727000000000001e-05 2024-09-20T18:53:38.115 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:38.115 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:53:38.115 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:38.115 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.116 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.116 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:27565 7.1 7:8e95a048:::benchmark_data_smithi083_70643_object27564:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:53:38.116 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:08.738835+0000", 2024-09-20T18:53:38.116 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 149.328360148, 2024-09-20T18:53:38.116 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.103678123, 2024-09-20T18:53:38.116 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:53:38.116 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:53:38.116 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:38.116 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:53:38.116 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:53:38.116 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:38.116 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 27565 2024-09-20T18:53:38.116 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.116 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:53:38.117 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.117 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:53:38.117 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.738835+0000", 2024-09-20T18:53:38.117 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:38.117 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.117 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.117 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:53:38.117 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.738835+0000", 2024-09-20T18:53:38.117 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:38.117 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.117 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.117 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:53:38.117 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.738836+0000", 2024-09-20T18:53:38.117 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.451e-06 2024-09-20T18:53:38.117 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.118 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.118 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:53:38.118 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.738941+0000", 2024-09-20T18:53:38.118 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00010476000000000001 2024-09-20T18:53:38.118 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.118 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.118 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:53:38.118 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.738949+0000", 2024-09-20T18:53:38.118 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.1869999999999997e-06 2024-09-20T18:53:38.118 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.118 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.118 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:53:38.118 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.738955+0000", 2024-09-20T18:53:38.118 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.908e-06 2024-09-20T18:53:38.119 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.119 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.119 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:53:38.119 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.739267+0000", 2024-09-20T18:53:38.119 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00031249899999999999 2024-09-20T18:53:38.119 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.119 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.119 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:53:38.119 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.739392+0000", 2024-09-20T18:53:38.119 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012450699999999999 2024-09-20T18:53:38.119 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.119 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.119 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,6", 2024-09-20T18:53:38.120 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.739517+0000", 2024-09-20T18:53:38.120 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012527400000000001 2024-09-20T18:53:38.120 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.120 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.120 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:53:38.120 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.740572+0000", 2024-09-20T18:53:38.120 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001054452 2024-09-20T18:53:38.120 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.120 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.120 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:38.120 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.741601+0000", 2024-09-20T18:53:38.120 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0010288160000000001 2024-09-20T18:53:38.120 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.120 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.121 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:38.121 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.842412+0000", 2024-09-20T18:53:38.121 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100811759 2024-09-20T18:53:38.121 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.121 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.121 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:53:38.121 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.842469+0000", 2024-09-20T18:53:38.121 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.7169000000000001e-05 2024-09-20T18:53:38.121 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.121 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.121 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:53:38.121 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.842513+0000", 2024-09-20T18:53:38.121 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3340999999999997e-05 2024-09-20T18:53:38.121 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:38.122 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:53:38.122 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:38.122 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.122 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.122 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:31526 7.1 7:8e4a4ffd:::benchmark_data_smithi083_70643_object31525:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T18:53:38.122 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:10.995167+0000", 2024-09-20T18:53:38.122 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 147.07202787200001, 2024-09-20T18:53:38.122 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.064702838999999998, 2024-09-20T18:53:38.122 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:53:38.122 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:53:38.122 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:38.122 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:53:38.122 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:53:38.122 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:38.123 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 31526 2024-09-20T18:53:38.123 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.123 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:53:38.123 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.123 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:53:38.123 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:10.995167+0000", 2024-09-20T18:53:38.123 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:38.123 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.123 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.123 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:53:38.123 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:10.995167+0000", 2024-09-20T18:53:38.123 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:38.123 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.123 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.124 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:53:38.124 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:10.995169+0000", 2024-09-20T18:53:38.124 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.784e-06 2024-09-20T18:53:38.124 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.124 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.124 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:53:38.124 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:10.995212+0000", 2024-09-20T18:53:38.124 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.2911000000000003e-05 2024-09-20T18:53:38.124 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.124 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.124 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:53:38.124 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:10.995227+0000", 2024-09-20T18:53:38.124 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5078e-05 2024-09-20T18:53:38.124 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.124 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.125 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:53:38.125 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:10.995235+0000", 2024-09-20T18:53:38.125 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.9929999999999995e-06 2024-09-20T18:53:38.125 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.125 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.125 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:53:38.125 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:10.995282+0000", 2024-09-20T18:53:38.125 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7355999999999999e-05 2024-09-20T18:53:38.125 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.125 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.125 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:53:38.125 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:10.995400+0000", 2024-09-20T18:53:38.125 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000118262 2024-09-20T18:53:38.125 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.125 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.126 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,6", 2024-09-20T18:53:38.126 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:10.995613+0000", 2024-09-20T18:53:38.126 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000213048 2024-09-20T18:53:38.126 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.126 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.126 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:53:38.126 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:10.996704+0000", 2024-09-20T18:53:38.126 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0010906290000000001 2024-09-20T18:53:38.126 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.126 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.126 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:38.126 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:10.997397+0000", 2024-09-20T18:53:38.126 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00069284900000000005 2024-09-20T18:53:38.126 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.127 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.127 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:38.127 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:11.059767+0000", 2024-09-20T18:53:38.127 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.062369804000000001 2024-09-20T18:53:38.127 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.127 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.127 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:53:38.127 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:11.059820+0000", 2024-09-20T18:53:38.127 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.342e-05 2024-09-20T18:53:38.127 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.127 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.127 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:53:38.127 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:11.059870+0000", 2024-09-20T18:53:38.127 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.9704999999999999e-05 2024-09-20T18:53:38.128 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:38.128 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:53:38.128 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:38.128 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.128 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.128 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:32908 7.1 7:8a214de6:::benchmark_data_smithi083_70643_object32907:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T18:53:38.128 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:11.728778+0000", 2024-09-20T18:53:38.128 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 146.338417085, 2024-09-20T18:53:38.128 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.062731522999999997, 2024-09-20T18:53:38.128 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:53:38.128 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:53:38.128 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:38.128 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:53:38.128 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:53:38.129 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:38.129 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 32908 2024-09-20T18:53:38.129 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.129 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:53:38.129 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.129 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:53:38.129 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:11.728778+0000", 2024-09-20T18:53:38.129 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:38.129 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.129 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.129 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:53:38.129 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:11.728778+0000", 2024-09-20T18:53:38.129 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:38.129 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.129 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.130 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:53:38.130 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:11.728780+0000", 2024-09-20T18:53:38.130 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8160000000000001e-06 2024-09-20T18:53:38.130 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.130 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.130 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:53:38.130 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:11.728921+0000", 2024-09-20T18:53:38.130 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00014168199999999999 2024-09-20T18:53:38.130 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.130 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.130 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:53:38.130 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:11.728935+0000", 2024-09-20T18:53:38.130 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3423e-05 2024-09-20T18:53:38.130 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.131 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.131 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:53:38.131 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:11.728942+0000", 2024-09-20T18:53:38.131 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.1459999999999999e-06 2024-09-20T18:53:38.131 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.131 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.131 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:53:38.131 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:11.729001+0000", 2024-09-20T18:53:38.131 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9005000000000002e-05 2024-09-20T18:53:38.131 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.131 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.131 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:53:38.131 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:11.729100+0000", 2024-09-20T18:53:38.131 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.8869999999999994e-05 2024-09-20T18:53:38.132 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.132 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.132 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,6", 2024-09-20T18:53:38.132 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:11.729218+0000", 2024-09-20T18:53:38.132 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000118084 2024-09-20T18:53:38.132 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.132 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.132 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:53:38.132 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:11.730418+0000", 2024-09-20T18:53:38.132 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001199961 2024-09-20T18:53:38.132 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.132 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.132 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:38.133 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:11.732927+0000", 2024-09-20T18:53:38.133 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.002509627 2024-09-20T18:53:38.133 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.133 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.133 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:38.133 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:11.791406+0000", 2024-09-20T18:53:38.133 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.058478324999999998 2024-09-20T18:53:38.133 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.133 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.133 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:53:38.133 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:11.791462+0000", 2024-09-20T18:53:38.133 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6659000000000001e-05 2024-09-20T18:53:38.133 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.133 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.134 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:53:38.134 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:11.791509+0000", 2024-09-20T18:53:38.134 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.6925000000000002e-05 2024-09-20T18:53:38.134 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:38.134 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:53:38.134 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:38.134 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.134 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.134 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:32920 7.1 7:804c24fd:::benchmark_data_smithi083_70643_object32919:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T18:53:38.134 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:11.736287+0000", 2024-09-20T18:53:38.134 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 146.330907419, 2024-09-20T18:53:38.134 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.055350518000000001, 2024-09-20T18:53:38.134 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:53:38.134 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:53:38.135 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:38.135 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:53:38.135 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:53:38.135 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:38.135 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 32920 2024-09-20T18:53:38.135 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.135 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:53:38.135 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.135 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:53:38.135 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:11.736287+0000", 2024-09-20T18:53:38.135 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:38.135 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.135 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.135 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:53:38.135 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:11.736287+0000", 2024-09-20T18:53:38.136 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:38.136 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.136 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.136 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:53:38.136 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:11.736289+0000", 2024-09-20T18:53:38.136 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.35e-06 2024-09-20T18:53:38.136 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.136 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.136 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:53:38.136 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:11.736347+0000", 2024-09-20T18:53:38.136 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8029999999999998e-05 2024-09-20T18:53:38.136 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.136 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.136 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:53:38.137 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:11.736354+0000", 2024-09-20T18:53:38.137 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.267e-06 2024-09-20T18:53:38.137 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.137 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.137 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:53:38.137 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:11.736359+0000", 2024-09-20T18:53:38.137 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.6550000000000003e-06 2024-09-20T18:53:38.137 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.137 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.137 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:53:38.137 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:11.736430+0000", 2024-09-20T18:53:38.137 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.1397999999999996e-05 2024-09-20T18:53:38.137 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.137 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.137 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:53:38.138 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:11.736505+0000", 2024-09-20T18:53:38.138 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.4989999999999999e-05 2024-09-20T18:53:38.138 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.138 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.138 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,6", 2024-09-20T18:53:38.138 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:11.736611+0000", 2024-09-20T18:53:38.138 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000105973 2024-09-20T18:53:38.138 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.138 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.138 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:53:38.138 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:11.737654+0000", 2024-09-20T18:53:38.138 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0010430089999999999 2024-09-20T18:53:38.138 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.138 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.138 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:38.139 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:11.738195+0000", 2024-09-20T18:53:38.139 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00054048900000000001 2024-09-20T18:53:38.139 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.139 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.139 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:38.139 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:11.791556+0000", 2024-09-20T18:53:38.139 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.053361839000000001 2024-09-20T18:53:38.139 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.139 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.139 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:53:38.139 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:11.791604+0000", 2024-09-20T18:53:38.139 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7766999999999999e-05 2024-09-20T18:53:38.139 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.139 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.140 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:53:38.140 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:11.791638+0000", 2024-09-20T18:53:38.140 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.3751000000000003e-05 2024-09-20T18:53:38.140 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:38.140 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:53:38.140 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:38.140 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.140 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.140 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:65664 7.1 7:8df7e116:::benchmark_data_smithi083_70643_object65663:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-20T18:53:38.140 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:33.007523+0000", 2024-09-20T18:53:38.140 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 125.05967192, 2024-09-20T18:53:38.140 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.057929815000000003, 2024-09-20T18:53:38.140 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:53:38.140 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:53:38.140 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:38.141 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:53:38.141 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:53:38.141 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:38.141 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 65664 2024-09-20T18:53:38.141 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.141 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:53:38.141 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.141 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:53:38.141 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:33.007523+0000", 2024-09-20T18:53:38.141 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:38.141 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.141 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.141 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:53:38.141 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:33.007523+0000", 2024-09-20T18:53:38.141 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:38.142 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.142 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.142 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:53:38.142 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:33.007524+0000", 2024-09-20T18:53:38.142 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1909999999999999e-06 2024-09-20T18:53:38.142 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.142 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.142 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:53:38.142 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:33.007570+0000", 2024-09-20T18:53:38.142 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.6269e-05 2024-09-20T18:53:38.142 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.142 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.142 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:53:38.142 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:33.007579+0000", 2024-09-20T18:53:38.143 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.0410000000000003e-06 2024-09-20T18:53:38.143 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.143 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.143 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:53:38.143 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:33.007585+0000", 2024-09-20T18:53:38.143 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8730000000000004e-06 2024-09-20T18:53:38.143 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.143 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.143 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:53:38.143 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:33.007631+0000", 2024-09-20T18:53:38.143 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.5274000000000001e-05 2024-09-20T18:53:38.143 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.143 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.143 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:53:38.143 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:33.007750+0000", 2024-09-20T18:53:38.144 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000119877 2024-09-20T18:53:38.144 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.144 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:38.146 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:38.146 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,6", 2024-09-20T18:53:38.146 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:33.007901+0000", 2024-09-20T18:53:38.146 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015013799999999999 2024-09-20T18:53:38.146 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.146 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.146 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:38.146 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:33.011091+0000", 2024-09-20T18:53:38.146 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0031903809999999999 2024-09-20T18:53:38.146 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.146 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.146 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:53:38.147 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:33.012083+0000", 2024-09-20T18:53:38.147 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00099244799999999994 2024-09-20T18:53:38.147 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.147 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.147 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:38.147 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:33.065380+0000", 2024-09-20T18:53:38.147 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.053296171000000003 2024-09-20T18:53:38.147 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.147 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.147 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:53:38.147 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:33.065419+0000", 2024-09-20T18:53:38.147 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.8986999999999998e-05 2024-09-20T18:53:38.147 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.147 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.147 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:53:38.148 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:33.065453+0000", 2024-09-20T18:53:38.148 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4165000000000002e-05 2024-09-20T18:53:38.148 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:38.148 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:53:38.148 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:38.148 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.148 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.148 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:86923 7.1 7:863eafec:::benchmark_data_smithi083_70643_object86922:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-20T18:53:38.148 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:51.203211+0000", 2024-09-20T18:53:38.148 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 106.863984156, 2024-09-20T18:53:38.148 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.062398980999999999, 2024-09-20T18:53:38.148 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:53:38.148 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:53:38.148 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:38.149 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:53:38.149 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:53:38.149 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:38.149 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 86923 2024-09-20T18:53:38.149 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.149 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:53:38.149 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.149 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:53:38.149 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:51.203211+0000", 2024-09-20T18:53:38.149 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:38.149 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.149 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.149 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:53:38.149 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:51.203211+0000", 2024-09-20T18:53:38.150 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:38.150 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.150 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.150 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:53:38.150 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:51.203212+0000", 2024-09-20T18:53:38.150 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5200000000000001e-06 2024-09-20T18:53:38.150 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.150 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.150 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:53:38.150 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:51.203256+0000", 2024-09-20T18:53:38.150 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3556000000000001e-05 2024-09-20T18:53:38.150 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.150 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.150 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:53:38.151 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:51.203271+0000", 2024-09-20T18:53:38.151 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4951e-05 2024-09-20T18:53:38.151 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.151 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.151 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:53:38.151 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:51.203279+0000", 2024-09-20T18:53:38.151 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.7520000000000002e-06 2024-09-20T18:53:38.151 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.151 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.151 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:53:38.151 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:51.203366+0000", 2024-09-20T18:53:38.151 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.7069000000000003e-05 2024-09-20T18:53:38.151 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.151 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.151 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:53:38.152 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:51.203486+0000", 2024-09-20T18:53:38.152 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011953 2024-09-20T18:53:38.152 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.152 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.152 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,6", 2024-09-20T18:53:38.152 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:51.203579+0000", 2024-09-20T18:53:38.152 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.3245000000000006e-05 2024-09-20T18:53:38.152 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.152 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.152 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:53:38.152 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:51.205657+0000", 2024-09-20T18:53:38.152 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0020782180000000002 2024-09-20T18:53:38.152 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.152 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.152 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:38.153 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:51.205770+0000", 2024-09-20T18:53:38.153 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011293099999999999 2024-09-20T18:53:38.153 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.153 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.153 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:38.153 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:51.265545+0000", 2024-09-20T18:53:38.153 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.059774233000000003 2024-09-20T18:53:38.153 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.153 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.153 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:53:38.153 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:51.265581+0000", 2024-09-20T18:53:38.153 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6473999999999997e-05 2024-09-20T18:53:38.153 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.153 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.153 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:53:38.154 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:51.265610+0000", 2024-09-20T18:53:38.154 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.8501999999999999e-05 2024-09-20T18:53:38.154 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:38.154 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:53:38.154 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:38.154 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.154 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.154 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:101255 7.1 7:80ab34f1:::benchmark_data_smithi083_70643_object101254:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-20T18:53:38.154 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:05.038883+0000", 2024-09-20T18:53:38.154 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 93.028312111999995, 2024-09-20T18:53:38.154 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.064111823999999998, 2024-09-20T18:53:38.154 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:53:38.154 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:53:38.154 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:38.154 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:53:38.155 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:53:38.155 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:38.155 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 101255 2024-09-20T18:53:38.155 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.155 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:53:38.155 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.155 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:53:38.155 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:05.038883+0000", 2024-09-20T18:53:38.155 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:38.155 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.155 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.156 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:53:38.156 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:05.038883+0000", 2024-09-20T18:53:38.156 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:38.156 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.156 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.156 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:53:38.156 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:05.038883+0000", 2024-09-20T18:53:38.156 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.6899999999999997e-07 2024-09-20T18:53:38.156 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.156 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.156 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:53:38.156 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:05.038920+0000", 2024-09-20T18:53:38.156 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6785000000000001e-05 2024-09-20T18:53:38.156 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.156 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.157 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:53:38.157 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:05.038928+0000", 2024-09-20T18:53:38.157 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.2069999999999995e-06 2024-09-20T18:53:38.157 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.157 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.157 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:53:38.157 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:05.038934+0000", 2024-09-20T18:53:38.157 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0299999999999999e-06 2024-09-20T18:53:38.157 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.157 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.157 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:53:38.157 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:05.041619+0000", 2024-09-20T18:53:38.157 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0026841819999999998 2024-09-20T18:53:38.157 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.157 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.158 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:53:38.158 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:05.041716+0000", 2024-09-20T18:53:38.158 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.7922000000000002e-05 2024-09-20T18:53:38.158 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.158 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.158 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,6", 2024-09-20T18:53:38.158 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:05.041844+0000", 2024-09-20T18:53:38.158 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000127033 2024-09-20T18:53:38.158 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.158 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.158 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:53:38.158 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:05.044356+0000", 2024-09-20T18:53:38.158 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0025127579999999999 2024-09-20T18:53:38.158 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.158 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.159 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:38.159 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:05.044617+0000", 2024-09-20T18:53:38.159 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000261165 2024-09-20T18:53:38.159 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.159 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.159 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:38.159 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:05.102905+0000", 2024-09-20T18:53:38.159 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.058287525 2024-09-20T18:53:38.159 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.159 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.159 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:53:38.159 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:05.102959+0000", 2024-09-20T18:53:38.159 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4398000000000003e-05 2024-09-20T18:53:38.159 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.159 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.160 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:53:38.160 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:05.102995+0000", 2024-09-20T18:53:38.160 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.5150000000000001e-05 2024-09-20T18:53:38.160 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:38.160 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:53:38.160 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:38.160 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.160 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.160 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:120202 7.1 7:8a289ff9:::benchmark_data_smithi083_70643_object120201:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T18:53:38.160 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:24.982702+0000", 2024-09-20T18:53:38.160 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 73.084493300999995, 2024-09-20T18:53:38.160 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.096958845000000002, 2024-09-20T18:53:38.160 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:53:38.160 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:53:38.160 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:38.160 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:53:38.161 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:53:38.161 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:38.161 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 120202 2024-09-20T18:53:38.161 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.161 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:53:38.161 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.161 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:53:38.161 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.982702+0000", 2024-09-20T18:53:38.161 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:38.161 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.161 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.161 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:53:38.161 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.982702+0000", 2024-09-20T18:53:38.161 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:38.161 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.162 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.162 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:53:38.162 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.982703+0000", 2024-09-20T18:53:38.162 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9199999999999998e-06 2024-09-20T18:53:38.162 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.162 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.162 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:53:38.162 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.982746+0000", 2024-09-20T18:53:38.162 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.2809000000000003e-05 2024-09-20T18:53:38.162 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.162 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.162 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:53:38.162 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.982757+0000", 2024-09-20T18:53:38.162 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0412e-05 2024-09-20T18:53:38.162 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.163 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.163 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:53:38.163 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.982762+0000", 2024-09-20T18:53:38.163 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6080000000000003e-06 2024-09-20T18:53:38.163 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.163 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.163 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:53:38.163 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.982802+0000", 2024-09-20T18:53:38.163 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9857000000000001e-05 2024-09-20T18:53:38.163 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.163 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.163 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:53:38.163 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.982899+0000", 2024-09-20T18:53:38.163 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.7187000000000003e-05 2024-09-20T18:53:38.164 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.164 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.164 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,6", 2024-09-20T18:53:38.164 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.982999+0000", 2024-09-20T18:53:38.164 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.9246999999999994e-05 2024-09-20T18:53:38.164 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.164 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.164 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:53:38.164 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.985215+0000", 2024-09-20T18:53:38.164 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0022165829999999998 2024-09-20T18:53:38.164 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.164 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.164 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:38.164 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.010015+0000", 2024-09-20T18:53:38.164 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.024799796999999998 2024-09-20T18:53:38.164 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.165 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.165 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:38.165 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.079514+0000", 2024-09-20T18:53:38.165 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.069499218000000001 2024-09-20T18:53:38.165 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.165 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.165 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:53:38.165 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.079616+0000", 2024-09-20T18:53:38.165 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000102236 2024-09-20T18:53:38.165 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.165 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.165 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:53:38.165 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.079660+0000", 2024-09-20T18:53:38.165 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3971000000000003e-05 2024-09-20T18:53:38.165 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:38.165 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:53:38.166 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:38.166 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.166 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.166 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:120207 7.1 7:8d667e1f:::benchmark_data_smithi083_70643_object120206:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T18:53:38.166 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:24.990072+0000", 2024-09-20T18:53:38.166 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 73.077123033999996, 2024-09-20T18:53:38.166 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.089731845000000005, 2024-09-20T18:53:38.166 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:53:38.166 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:53:38.166 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:38.166 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:53:38.166 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:53:38.166 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:38.166 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 120207 2024-09-20T18:53:38.167 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.167 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:53:38.167 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.167 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:53:38.167 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.990072+0000", 2024-09-20T18:53:38.167 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:38.167 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.167 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.167 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:53:38.167 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.990072+0000", 2024-09-20T18:53:38.167 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:38.167 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.167 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.167 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:53:38.168 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.990073+0000", 2024-09-20T18:53:38.168 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6950000000000001e-06 2024-09-20T18:53:38.168 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.168 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.168 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:53:38.168 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.990122+0000", 2024-09-20T18:53:38.168 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8696999999999998e-05 2024-09-20T18:53:38.168 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.168 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.168 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:53:38.168 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.009979+0000", 2024-09-20T18:53:38.168 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.019857099999999999 2024-09-20T18:53:38.168 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.168 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.168 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:53:38.169 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.009989+0000", 2024-09-20T18:53:38.169 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.3000000000000007e-06 2024-09-20T18:53:38.169 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.169 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.169 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:53:38.169 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.010053+0000", 2024-09-20T18:53:38.169 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4024999999999998e-05 2024-09-20T18:53:38.169 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.169 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.169 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:53:38.169 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.010170+0000", 2024-09-20T18:53:38.169 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000117617 2024-09-20T18:53:38.169 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.169 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.170 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,6", 2024-09-20T18:53:38.170 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.010268+0000", 2024-09-20T18:53:38.170 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.7713000000000004e-05 2024-09-20T18:53:38.170 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.170 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.170 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:53:38.170 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.012425+0000", 2024-09-20T18:53:38.170 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0021567829999999998 2024-09-20T18:53:38.170 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.170 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.170 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:38.170 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.012576+0000", 2024-09-20T18:53:38.170 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015128 2024-09-20T18:53:38.170 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.170 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.171 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:38.171 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.079714+0000", 2024-09-20T18:53:38.171 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.067138113999999999 2024-09-20T18:53:38.171 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.171 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.171 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:53:38.171 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.079774+0000", 2024-09-20T18:53:38.171 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0402000000000002e-05 2024-09-20T18:53:38.171 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.171 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.171 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:53:38.171 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.079804+0000", 2024-09-20T18:53:38.171 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9119e-05 2024-09-20T18:53:38.171 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:38.171 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:53:38.172 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:38.172 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.172 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.172 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:139464 7.1 7:8d273f11:::benchmark_data_smithi083_70643_object16662:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-20T18:53:38.172 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:43.118758+0000", 2024-09-20T18:53:38.172 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 54.948437259999999, 2024-09-20T18:53:38.172 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.094121622000000002, 2024-09-20T18:53:38.172 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:53:38.172 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:53:38.172 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:38.172 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:53:38.172 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:53:38.172 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:38.172 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 139464 2024-09-20T18:53:38.173 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.173 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:53:38.173 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.173 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:53:38.173 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.118758+0000", 2024-09-20T18:53:38.173 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:38.173 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.173 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.173 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:53:38.173 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.118758+0000", 2024-09-20T18:53:38.173 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:38.173 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.173 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.173 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:53:38.174 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.118759+0000", 2024-09-20T18:53:38.174 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5659999999999999e-06 2024-09-20T18:53:38.174 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.174 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.174 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:53:38.174 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.118769+0000", 2024-09-20T18:53:38.174 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0091e-05 2024-09-20T18:53:38.174 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.174 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.174 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:53:38.174 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.118782+0000", 2024-09-20T18:53:38.174 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3293e-05 2024-09-20T18:53:38.174 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.174 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.174 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:53:38.175 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.118790+0000", 2024-09-20T18:53:38.175 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.0380000000000002e-06 2024-09-20T18:53:38.175 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.175 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.175 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:53:38.175 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.118845+0000", 2024-09-20T18:53:38.175 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5640999999999998e-05 2024-09-20T18:53:38.175 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.175 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.175 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:53:38.175 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.118969+0000", 2024-09-20T18:53:38.175 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012337599999999999 2024-09-20T18:53:38.175 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.175 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.175 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,6", 2024-09-20T18:53:38.176 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.119100+0000", 2024-09-20T18:53:38.176 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000130994 2024-09-20T18:53:38.176 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.176 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.176 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:38.176 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.123341+0000", 2024-09-20T18:53:38.176 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.004240995 2024-09-20T18:53:38.176 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.176 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.176 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:53:38.176 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.123423+0000", 2024-09-20T18:53:38.176 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.2300999999999997e-05 2024-09-20T18:53:38.176 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.176 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.176 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:38.177 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.212798+0000", 2024-09-20T18:53:38.177 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.089375316999999996 2024-09-20T18:53:38.177 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.177 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.177 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:53:38.177 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.212835+0000", 2024-09-20T18:53:38.177 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6758000000000003e-05 2024-09-20T18:53:38.177 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.177 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.177 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:53:38.177 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.212879+0000", 2024-09-20T18:53:38.177 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.4252000000000003e-05 2024-09-20T18:53:38.177 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:38.177 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:53:38.178 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:38.178 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.178 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.178 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:139471 7.1 7:8b47cc00:::benchmark_data_smithi083_70643_object16669:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-20T18:53:38.178 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:43.123464+0000", 2024-09-20T18:53:38.178 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 54.943730873, 2024-09-20T18:53:38.178 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.089504035999999995, 2024-09-20T18:53:38.178 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:53:38.178 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:53:38.178 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:38.178 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:53:38.178 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:53:38.178 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:38.178 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 139471 2024-09-20T18:53:38.178 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.179 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:53:38.179 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.179 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:53:38.179 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.123464+0000", 2024-09-20T18:53:38.179 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:38.179 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.179 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.179 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:53:38.179 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.123464+0000", 2024-09-20T18:53:38.179 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:38.179 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.179 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.179 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:53:38.179 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.123465+0000", 2024-09-20T18:53:38.180 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.471e-06 2024-09-20T18:53:38.180 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.180 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.180 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:53:38.180 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.123474+0000", 2024-09-20T18:53:38.180 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.67e-06 2024-09-20T18:53:38.180 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.180 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.180 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:53:38.180 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.123487+0000", 2024-09-20T18:53:38.180 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3134e-05 2024-09-20T18:53:38.180 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.180 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.180 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:53:38.181 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.123494+0000", 2024-09-20T18:53:38.181 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5950000000000004e-06 2024-09-20T18:53:38.181 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.181 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.181 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:53:38.181 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.123544+0000", 2024-09-20T18:53:38.181 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0099999999999998e-05 2024-09-20T18:53:38.181 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.181 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.181 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:53:38.181 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.123661+0000", 2024-09-20T18:53:38.181 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000116925 2024-09-20T18:53:38.181 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.181 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.182 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,6", 2024-09-20T18:53:38.182 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.123772+0000", 2024-09-20T18:53:38.182 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011149700000000001 2024-09-20T18:53:38.182 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.182 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.182 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:53:38.182 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.126783+0000", 2024-09-20T18:53:38.182 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00301031 2024-09-20T18:53:38.182 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.182 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.182 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:38.182 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.127290+0000", 2024-09-20T18:53:38.182 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00050741800000000004 2024-09-20T18:53:38.182 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.183 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.183 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:38.183 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.212914+0000", 2024-09-20T18:53:38.183 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.085624152999999995 2024-09-20T18:53:38.183 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.183 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.183 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:53:38.183 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.212947+0000", 2024-09-20T18:53:38.183 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.2781999999999999e-05 2024-09-20T18:53:38.183 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.183 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.183 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:53:38.183 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.212968+0000", 2024-09-20T18:53:38.183 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0981000000000001e-05 2024-09-20T18:53:38.183 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:38.184 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:53:38.184 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:38.184 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.184 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.184 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:156706 7.1 7:8e235435:::benchmark_data_smithi083_70643_object33904:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133)", 2024-09-20T18:53:38.184 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:59.417396+0000", 2024-09-20T18:53:38.184 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 38.649799299999998, 2024-09-20T18:53:38.184 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.065579346999999996, 2024-09-20T18:53:38.184 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:53:38.184 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:53:38.184 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:38.184 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:53:38.184 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:53:38.184 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:38.185 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 156706 2024-09-20T18:53:38.185 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.185 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:53:38.185 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.185 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:53:38.185 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:59.417396+0000", 2024-09-20T18:53:38.185 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:38.185 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.185 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.185 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:53:38.185 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:59.417396+0000", 2024-09-20T18:53:38.185 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:38.185 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.185 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.185 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:53:38.185 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:59.417397+0000", 2024-09-20T18:53:38.186 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5379999999999999e-06 2024-09-20T18:53:38.186 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.186 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.186 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:53:38.186 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:59.417402+0000", 2024-09-20T18:53:38.186 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.5369999999999999e-06 2024-09-20T18:53:38.186 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.186 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.186 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:53:38.186 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:59.417412+0000", 2024-09-20T18:53:38.186 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0573e-05 2024-09-20T18:53:38.186 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.186 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.186 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:53:38.186 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:59.417418+0000", 2024-09-20T18:53:38.187 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3510000000000001e-06 2024-09-20T18:53:38.187 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.187 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.187 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:53:38.187 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:59.417454+0000", 2024-09-20T18:53:38.187 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6559e-05 2024-09-20T18:53:38.187 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.187 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.187 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:53:38.187 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:59.417542+0000", 2024-09-20T18:53:38.187 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.7867000000000004e-05 2024-09-20T18:53:38.187 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.187 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.187 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,6", 2024-09-20T18:53:38.187 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:59.417632+0000", 2024-09-20T18:53:38.188 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.9689000000000001e-05 2024-09-20T18:53:38.188 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.188 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.188 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:53:38.188 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:59.419875+0000", 2024-09-20T18:53:38.188 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0022428919999999998 2024-09-20T18:53:38.188 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.188 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.188 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:38.188 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:59.419955+0000", 2024-09-20T18:53:38.188 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.0074999999999995e-05 2024-09-20T18:53:38.188 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.188 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.188 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:38.189 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:59.482908+0000", 2024-09-20T18:53:38.189 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.062953141000000004 2024-09-20T18:53:38.189 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.189 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.189 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:53:38.189 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:59.482943+0000", 2024-09-20T18:53:38.189 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4876999999999998e-05 2024-09-20T18:53:38.189 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.189 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.189 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:53:38.189 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:59.482975+0000", 2024-09-20T18:53:38.189 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.2248e-05 2024-09-20T18:53:38.189 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:38.189 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:53:38.189 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:38.190 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.190 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.190 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:156715 7.1 7:87dc47fe:::benchmark_data_smithi083_70643_object33913:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133)", 2024-09-20T18:53:38.190 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:59.426244+0000", 2024-09-20T18:53:38.190 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 38.640950754999999, 2024-09-20T18:53:38.190 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.059665502000000002, 2024-09-20T18:53:38.190 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:53:38.190 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:53:38.190 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:38.190 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:53:38.190 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:53:38.190 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:38.190 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 156715 2024-09-20T18:53:38.190 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.190 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:53:38.191 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.191 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:53:38.191 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:59.426244+0000", 2024-09-20T18:53:38.191 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:38.191 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.191 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.191 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:53:38.191 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:59.426244+0000", 2024-09-20T18:53:38.191 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:38.191 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.191 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.191 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:53:38.191 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:59.426245+0000", 2024-09-20T18:53:38.192 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3790000000000001e-06 2024-09-20T18:53:38.192 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.192 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.192 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:53:38.192 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:59.426249+0000", 2024-09-20T18:53:38.192 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.2650000000000001e-06 2024-09-20T18:53:38.192 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.192 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.192 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:53:38.192 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:59.426258+0000", 2024-09-20T18:53:38.192 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.3030000000000005e-06 2024-09-20T18:53:38.192 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.192 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.192 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:53:38.192 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:59.426263+0000", 2024-09-20T18:53:38.193 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.2009999999999998e-06 2024-09-20T18:53:38.193 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.193 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.193 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:53:38.193 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:59.426305+0000", 2024-09-20T18:53:38.193 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.1866999999999997e-05 2024-09-20T18:53:38.193 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.193 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.193 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:53:38.193 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:59.426387+0000", 2024-09-20T18:53:38.193 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.1630000000000003e-05 2024-09-20T18:53:38.193 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.193 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.193 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,6", 2024-09-20T18:53:38.194 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:59.426466+0000", 2024-09-20T18:53:38.194 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.8857999999999996e-05 2024-09-20T18:53:38.194 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.194 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.194 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:53:38.194 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:59.428407+0000", 2024-09-20T18:53:38.194 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0019413130000000001 2024-09-20T18:53:38.194 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.194 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.194 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:38.194 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:59.428685+0000", 2024-09-20T18:53:38.194 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000278452 2024-09-20T18:53:38.194 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.194 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.194 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:38.195 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:59.485830+0000", 2024-09-20T18:53:38.195 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.057144844 2024-09-20T18:53:38.195 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.195 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.195 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:53:38.195 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:59.485875+0000", 2024-09-20T18:53:38.195 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.4774000000000002e-05 2024-09-20T18:53:38.195 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.195 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.195 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:53:38.195 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:59.485910+0000", 2024-09-20T18:53:38.195 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4616000000000002e-05 2024-09-20T18:53:38.195 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:38.195 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:53:38.195 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:38.196 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.196 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.196 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:186552 7.1 7:842285f5:::benchmark_data_smithi083_70643_object63750:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e140)", 2024-09-20T18:53:38.196 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:53:27.920438+0000", 2024-09-20T18:53:38.196 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 10.146756468, 2024-09-20T18:53:38.196 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.055870298999999998, 2024-09-20T18:53:38.196 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:53:38.196 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:53:38.196 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:38.196 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:53:38.196 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:53:38.196 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:38.196 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 186552 2024-09-20T18:53:38.196 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.196 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:53:38.197 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.197 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:53:38.197 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:27.920438+0000", 2024-09-20T18:53:38.197 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:38.197 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.197 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.197 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:53:38.197 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:27.920438+0000", 2024-09-20T18:53:38.197 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:38.197 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.197 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.197 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:53:38.197 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:27.920440+0000", 2024-09-20T18:53:38.197 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1939999999999999e-06 2024-09-20T18:53:38.198 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.198 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.198 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:53:38.198 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:27.920443+0000", 2024-09-20T18:53:38.198 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9280000000000003e-06 2024-09-20T18:53:38.198 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.198 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.198 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:53:38.198 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:27.920454+0000", 2024-09-20T18:53:38.198 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0251000000000001e-05 2024-09-20T18:53:38.198 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.198 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.198 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:53:38.198 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:27.920461+0000", 2024-09-20T18:53:38.199 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.1590000000000004e-06 2024-09-20T18:53:38.199 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.199 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.199 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:53:38.199 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:27.920495+0000", 2024-09-20T18:53:38.199 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4291000000000001e-05 2024-09-20T18:53:38.199 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.199 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.199 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:53:38.199 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:27.920562+0000", 2024-09-20T18:53:38.199 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.6736999999999995e-05 2024-09-20T18:53:38.199 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.199 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.199 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,6", 2024-09-20T18:53:38.199 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:27.920628+0000", 2024-09-20T18:53:38.200 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.6119999999999997e-05 2024-09-20T18:53:38.200 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.200 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.200 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:53:38.200 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:27.922220+0000", 2024-09-20T18:53:38.200 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001592277 2024-09-20T18:53:38.200 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.200 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.200 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:38.200 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:27.922595+0000", 2024-09-20T18:53:38.200 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00037461500000000002 2024-09-20T18:53:38.200 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.200 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.200 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:38.200 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:27.976181+0000", 2024-09-20T18:53:38.201 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.053585989000000001 2024-09-20T18:53:38.201 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.201 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.201 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:53:38.201 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:27.976264+0000", 2024-09-20T18:53:38.201 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.3436000000000005e-05 2024-09-20T18:53:38.201 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.201 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.201 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:53:38.201 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:27.976309+0000", 2024-09-20T18:53:38.201 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.4301999999999997e-05 2024-09-20T18:53:38.201 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:38.201 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:53:38.201 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:38.202 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.202 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.202 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:188393 7.1 7:83b2a712:::benchmark_data_smithi083_70643_object65591:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-20T18:53:38.202 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:53:29.674052+0000", 2024-09-20T18:53:38.202 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 8.3931432529999999, 2024-09-20T18:53:38.202 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.087096657999999993, 2024-09-20T18:53:38.202 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:53:38.202 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:53:38.202 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:38.202 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:53:38.202 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:53:38.202 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:38.202 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 188393 2024-09-20T18:53:38.202 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.202 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:53:38.203 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.203 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:53:38.203 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.674052+0000", 2024-09-20T18:53:38.203 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:38.203 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.203 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.203 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:53:38.203 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.674052+0000", 2024-09-20T18:53:38.203 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:38.203 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.203 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.203 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:53:38.203 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.674053+0000", 2024-09-20T18:53:38.203 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.722e-06 2024-09-20T18:53:38.204 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.204 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.204 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:53:38.204 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.674058+0000", 2024-09-20T18:53:38.204 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8860000000000003e-06 2024-09-20T18:53:38.204 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.204 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.204 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:53:38.204 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.674071+0000", 2024-09-20T18:53:38.204 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3071000000000001e-05 2024-09-20T18:53:38.204 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.204 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.204 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:53:38.204 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.674079+0000", 2024-09-20T18:53:38.205 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.3390000000000004e-06 2024-09-20T18:53:38.205 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.205 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.205 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:53:38.205 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.674133+0000", 2024-09-20T18:53:38.205 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4320000000000002e-05 2024-09-20T18:53:38.205 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.205 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.205 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:53:38.205 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.674257+0000", 2024-09-20T18:53:38.205 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000124282 2024-09-20T18:53:38.205 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.205 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.205 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,6", 2024-09-20T18:53:38.205 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.674377+0000", 2024-09-20T18:53:38.206 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000120233 2024-09-20T18:53:38.206 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.206 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.206 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:38.206 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.677167+0000", 2024-09-20T18:53:38.206 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0027899330000000001 2024-09-20T18:53:38.206 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.206 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.206 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:53:38.206 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.682503+0000", 2024-09-20T18:53:38.206 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0053358820000000001 2024-09-20T18:53:38.206 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.206 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.206 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:38.207 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.761074+0000", 2024-09-20T18:53:38.207 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.078571252999999994 2024-09-20T18:53:38.207 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.207 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.207 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:53:38.207 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.761110+0000", 2024-09-20T18:53:38.207 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.5768e-05 2024-09-20T18:53:38.207 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.207 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.207 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:53:38.207 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.761148+0000", 2024-09-20T18:53:38.207 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.7969000000000002e-05 2024-09-20T18:53:38.207 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:38.207 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:53:38.208 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:38.208 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.208 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.208 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:188400 7.1 7:8dbba07a:::benchmark_data_smithi083_70643_object65598:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-20T18:53:38.208 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:53:29.679529+0000", 2024-09-20T18:53:38.208 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 8.3876659260000004, 2024-09-20T18:53:38.208 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.081768355000000001, 2024-09-20T18:53:38.208 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:53:38.208 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:53:38.208 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:38.208 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:53:38.208 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:53:38.208 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:38.208 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 188400 2024-09-20T18:53:38.208 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.209 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:53:38.209 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.209 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:53:38.209 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.679529+0000", 2024-09-20T18:53:38.209 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:38.209 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.209 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.209 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:53:38.209 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.679529+0000", 2024-09-20T18:53:38.209 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:38.209 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.209 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.209 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:53:38.209 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.679530+0000", 2024-09-20T18:53:38.210 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5379999999999999e-06 2024-09-20T18:53:38.210 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.210 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.210 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:53:38.210 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.679539+0000", 2024-09-20T18:53:38.210 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.9400000000000008e-06 2024-09-20T18:53:38.210 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.210 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.210 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:53:38.210 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.679555+0000", 2024-09-20T18:53:38.210 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5308999999999999e-05 2024-09-20T18:53:38.210 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.210 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.210 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:53:38.211 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.679562+0000", 2024-09-20T18:53:38.211 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.9009999999999997e-06 2024-09-20T18:53:38.211 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.211 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.211 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:53:38.211 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.679677+0000", 2024-09-20T18:53:38.211 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000115882 2024-09-20T18:53:38.211 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.211 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.211 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:53:38.211 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.679809+0000", 2024-09-20T18:53:38.211 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013178699999999999 2024-09-20T18:53:38.211 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.211 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.212 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,6", 2024-09-20T18:53:38.212 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.679935+0000", 2024-09-20T18:53:38.212 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012615799999999999 2024-09-20T18:53:38.212 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.212 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.212 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:53:38.212 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.682690+0000", 2024-09-20T18:53:38.212 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.002754322 2024-09-20T18:53:38.212 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.212 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.212 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:38.212 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.683012+0000", 2024-09-20T18:53:38.212 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00032239600000000002 2024-09-20T18:53:38.212 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.213 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.213 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:38.213 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.761198+0000", 2024-09-20T18:53:38.213 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.078186126999999994 2024-09-20T18:53:38.213 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.213 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.213 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:53:38.213 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.761247+0000", 2024-09-20T18:53:38.213 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8720000000000001e-05 2024-09-20T18:53:38.213 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:38.213 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:38.213 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:53:38.213 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.761297+0000", 2024-09-20T18:53:38.213 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0275000000000002e-05 2024-09-20T18:53:38.213 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:38.214 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:53:38.214 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:38.214 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:38.214 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:53:38.214 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T18:53:38.214 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:38.315 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:38.352 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:38.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:38.353 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:38.432 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T18:53:38.454 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:38.471 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:38 smithi083 ceph-mon[29421]: pgmap v174: 148 pgs: 148 active+clean; 3.1 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.40k op/s 2024-09-20T18:53:38.471 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:38 smithi083 ceph-mon[36260]: pgmap v174: 148 pgs: 148 active+clean; 3.1 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.40k op/s 2024-09-20T18:53:38.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:38.515 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:38.516 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:38.616 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:38.645 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:38.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:38.646 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:38.686 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:38 smithi142 ceph-mon[32508]: pgmap v174: 148 pgs: 148 active+clean; 3.1 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.40k op/s 2024-09-20T18:53:38.747 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:38.773 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:38.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:38.774 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:38.875 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:53:38.914 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:38.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:38.915 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:53:39.016 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:53:39.049 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:39.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:39.050 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:53:39.150 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:39.176 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:39.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:39.177 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:39.230 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-20T18:53:39.278 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:53:39.303 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:39.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:39.304 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:53:39.339 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.6 to 1 (10000) 2024-09-20T18:53:39.340 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:53:39.405 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:39.442 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:53:39.442 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:53:39.446 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:39.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:39.447 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:39.487 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:53:39.487 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:53:39.487 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:39 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2883921801' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-20T18:53:39.487 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:39 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-20T18:53:39.487 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:39 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:53:39.487 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:39 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:53:39.488 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:39 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2883921801' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-20T18:53:39.488 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:39 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-20T18:53:39.488 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:39 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:53:39.488 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:39 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:53:39.520 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:53:39.520 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:53:39.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:39 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2883921801' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-20T18:53:39.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:39 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-20T18:53:39.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:39 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:53:39.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:39 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:53:39.545 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:53:39.548 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:39.560 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:53:39.560 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:53:39.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:39.591 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:39.591 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:39.593 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:53:39.593 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:53:39.626 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:53:39.626 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:53:39.659 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:53:39.659 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:53:39.692 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:39.693 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:53:39.693 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:53:39.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:39.742 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:39.742 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:39.843 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:53:39.870 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:39.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:39.871 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:53:39.972 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:40.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:40.007 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:40.008 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:40.083 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T18:53:40.083 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:53:40.108 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:53:40.145 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:40.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:40.146 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:53:40.247 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:40.275 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:40.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:40.276 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:40.377 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:40.406 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:40.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:40.408 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:40.508 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:53:40.537 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:40.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:40.539 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:53:40.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:40 smithi083 ceph-mon[29421]: pgmap v175: 148 pgs: 148 active+clean; 3.1 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 779 B/s rd, 0 B/s wr, 793 op/s 2024-09-20T18:53:40.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:40 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-20T18:53:40.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:40 smithi083 ceph-mon[29421]: osdmap e145: 8 total, 8 up, 8 in 2024-09-20T18:53:40.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:40 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3669620236' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:53:40.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:40 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:53:40.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:40 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:53:40.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:40 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:53:40.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:40 smithi083 ceph-mon[36260]: pgmap v175: 148 pgs: 148 active+clean; 3.1 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 779 B/s rd, 0 B/s wr, 793 op/s 2024-09-20T18:53:40.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:40 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-20T18:53:40.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:40 smithi083 ceph-mon[36260]: osdmap e145: 8 total, 8 up, 8 in 2024-09-20T18:53:40.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:40 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3669620236' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:53:40.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:40 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:53:40.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:40 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:53:40.555 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:40 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:53:40.640 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:40.664 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:40.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:40.665 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:40.699 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:40 smithi142 ceph-mon[32508]: pgmap v175: 148 pgs: 148 active+clean; 3.1 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 779 B/s rd, 0 B/s wr, 793 op/s 2024-09-20T18:53:40.699 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:40 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-20T18:53:40.699 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:40 smithi142 ceph-mon[32508]: osdmap e145: 8 total, 8 up, 8 in 2024-09-20T18:53:40.700 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:40 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3669620236' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:53:40.700 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:40 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:53:40.700 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:40 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:53:40.700 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:40 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:53:40.766 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:53:40.799 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:40.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:40.800 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:53:40.868 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_blocked_ops 2024-09-20T18:53:40.901 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:40.938 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:40.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:40.939 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:41.040 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:41.054 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:53:40 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:53:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:53:41.063 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:41.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:41.063 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:41.164 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:41.200 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:41.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:41.201 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:41.202 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:53:41.302 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:53:41.327 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:41.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:41.328 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:53:41.428 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:41.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:41.454 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:41.454 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:41.548 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd reweight 7 1 2024-09-20T18:53:41.556 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:53:41.639 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:41.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:41.640 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:53:41.741 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:41.767 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:41.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:41.768 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:41.790 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T18:53:41.791 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:53:41.868 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:41.909 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:41.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:41.910 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:42.011 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:53:42.068 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:42.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:42.069 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:53:42.128 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:53:42.170 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:42.199 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:42.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:42.201 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:42.301 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:42.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:42.336 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:42.337 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:42.437 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:42.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:42.475 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:42.475 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:42.576 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:42.577 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_historic_ops 2024-09-20T18:53:42.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:42.617 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:42.617 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:42.664 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-20T18:53:42.718 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:42.758 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:42.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:42.760 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:42.860 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:42.917 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:53:42.944 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:42.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:42.945 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:43.017 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:42 smithi142 ceph-mon[32508]: pgmap v177: 148 pgs: 148 active+clean; 3.0 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.09k op/s 2024-09-20T18:53:43.017 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:42 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3299187299' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:53:43.046 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:53:43.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:42 smithi083 ceph-mon[29421]: pgmap v177: 148 pgs: 148 active+clean; 3.0 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.09k op/s 2024-09-20T18:53:43.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:42 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3299187299' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:53:43.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:42 smithi083 ceph-mon[36260]: pgmap v177: 148 pgs: 148 active+clean; 3.0 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.09k op/s 2024-09-20T18:53:43.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:42 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3299187299' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:53:43.072 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:43.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:43.073 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:53:43.174 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:43.204 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:43.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:43.204 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:43.305 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:53:43.370 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:43.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:43.371 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:53:43.466 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T18:53:43.466 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:53:43.471 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:43.513 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:43.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:43.514 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:43.614 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:43.650 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:43.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:43.650 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:43.679 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.7 to 1 (10000) 2024-09-20T18:53:43.752 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:53:43.788 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:43.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:43.789 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:53:43.889 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:53:43.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:43.925 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:43.926 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:53:44.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:43 smithi142 ceph-mon[32508]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:53:44.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:43 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3299187299' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T18:53:44.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:43 smithi142 ceph-mon[32508]: osdmap e146: 8 total, 8 up, 8 in 2024-09-20T18:53:44.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:43 smithi142 ceph-mon[32508]: 3.1a scrub starts 2024-09-20T18:53:44.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:43 smithi142 ceph-mon[32508]: 3.1a scrub ok 2024-09-20T18:53:44.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:43 smithi142 ceph-mon[32508]: 7.9 scrub starts 2024-09-20T18:53:44.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:43 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1804587700' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-20T18:53:44.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:43 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-20T18:53:44.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:43 smithi142 ceph-mon[32508]: 4.10 scrub starts 2024-09-20T18:53:44.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:43 smithi142 ceph-mon[32508]: 4.10 scrub ok 2024-09-20T18:53:44.026 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:44.041 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T18:53:44.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:43 smithi083 ceph-mon[29421]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:53:44.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:43 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3299187299' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T18:53:44.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:43 smithi083 ceph-mon[29421]: osdmap e146: 8 total, 8 up, 8 in 2024-09-20T18:53:44.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:43 smithi083 ceph-mon[29421]: 3.1a scrub starts 2024-09-20T18:53:44.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:43 smithi083 ceph-mon[29421]: 3.1a scrub ok 2024-09-20T18:53:44.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:43 smithi083 ceph-mon[29421]: 7.9 scrub starts 2024-09-20T18:53:44.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:43 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1804587700' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-20T18:53:44.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:43 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-20T18:53:44.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:43 smithi083 ceph-mon[29421]: 4.10 scrub starts 2024-09-20T18:53:44.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:43 smithi083 ceph-mon[29421]: 4.10 scrub ok 2024-09-20T18:53:44.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:43 smithi083 ceph-mon[36260]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:53:44.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:43 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3299187299' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T18:53:44.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:43 smithi083 ceph-mon[36260]: osdmap e146: 8 total, 8 up, 8 in 2024-09-20T18:53:44.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:43 smithi083 ceph-mon[36260]: 3.1a scrub starts 2024-09-20T18:53:44.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:43 smithi083 ceph-mon[36260]: 3.1a scrub ok 2024-09-20T18:53:44.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:43 smithi083 ceph-mon[36260]: 7.9 scrub starts 2024-09-20T18:53:44.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:43 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1804587700' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-20T18:53:44.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:43 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-20T18:53:44.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:43 smithi083 ceph-mon[36260]: 4.10 scrub starts 2024-09-20T18:53:44.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:43 smithi083 ceph-mon[36260]: 4.10 scrub ok 2024-09-20T18:53:44.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:44.063 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:44.063 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:44.122 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-20T18:53:44.164 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:53:44.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:44.195 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:44.195 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:53:44.296 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:44.323 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:44.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:44.324 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:44.425 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:53:44.457 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:44.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:44.458 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:53:44.474 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:53:44.560 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:44.588 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:44.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:44.589 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:44.690 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:44.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:44.718 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:44.718 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:44.819 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:44.848 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:44.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:44.849 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:44.900 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:44 smithi142 ceph-mon[32508]: pgmap v179: 148 pgs: 148 active+clean; 2.8 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.37k op/s 2024-09-20T18:53:44.900 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:44 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-20T18:53:44.900 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:44 smithi142 ceph-mon[32508]: osdmap e147: 8 total, 8 up, 8 in 2024-09-20T18:53:44.900 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:44 smithi142 ceph-mon[32508]: 7.9 scrub ok 2024-09-20T18:53:44.900 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:44 smithi142 ceph-mon[32508]: 4.13 scrub starts 2024-09-20T18:53:44.900 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:44 smithi142 ceph-mon[32508]: 4.13 scrub ok 2024-09-20T18:53:44.900 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:44 smithi142 ceph-mon[32508]: 5.0 scrub starts 2024-09-20T18:53:44.900 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:44 smithi142 ceph-mon[32508]: 5.0 scrub ok 2024-09-20T18:53:44.950 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:53:44.981 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:44.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:44.981 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:53:45.027 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:44 smithi083 ceph-mon[29421]: pgmap v179: 148 pgs: 148 active+clean; 2.8 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.37k op/s 2024-09-20T18:53:45.028 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:44 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-20T18:53:45.028 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:44 smithi083 ceph-mon[29421]: osdmap e147: 8 total, 8 up, 8 in 2024-09-20T18:53:45.028 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:44 smithi083 ceph-mon[29421]: 7.9 scrub ok 2024-09-20T18:53:45.028 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:44 smithi083 ceph-mon[29421]: 4.13 scrub starts 2024-09-20T18:53:45.028 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:44 smithi083 ceph-mon[29421]: 4.13 scrub ok 2024-09-20T18:53:45.028 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:44 smithi083 ceph-mon[29421]: 5.0 scrub starts 2024-09-20T18:53:45.029 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:44 smithi083 ceph-mon[29421]: 5.0 scrub ok 2024-09-20T18:53:45.029 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:44 smithi083 ceph-mon[36260]: pgmap v179: 148 pgs: 148 active+clean; 2.8 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.37k op/s 2024-09-20T18:53:45.029 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:44 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-20T18:53:45.029 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:44 smithi083 ceph-mon[36260]: osdmap e147: 8 total, 8 up, 8 in 2024-09-20T18:53:45.029 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:44 smithi083 ceph-mon[36260]: 7.9 scrub ok 2024-09-20T18:53:45.029 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:44 smithi083 ceph-mon[36260]: 4.13 scrub starts 2024-09-20T18:53:45.030 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:44 smithi083 ceph-mon[36260]: 4.13 scrub ok 2024-09-20T18:53:45.030 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:44 smithi083 ceph-mon[36260]: 5.0 scrub starts 2024-09-20T18:53:45.030 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:44 smithi083 ceph-mon[36260]: 5.0 scrub ok 2024-09-20T18:53:45.043 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T18:53:45.043 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:53:45.074 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-09-20T18:53:45.074 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T18:53:45.082 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:53:45.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:45.179 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:45.179 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:53:45.248 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:53:45.280 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:45.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:45.328 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:45.329 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:45.365 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T18:53:45.429 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:45.486 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:45.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:45.487 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:45.588 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:45.636 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:45.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:45.648 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:45.682 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:45 smithi083 ceph-mon[29421]: 5.1f scrub starts 2024-09-20T18:53:45.682 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:45 smithi083 ceph-mon[29421]: 5.1f scrub ok 2024-09-20T18:53:45.682 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:45 smithi083 ceph-mon[29421]: 4.1 scrub starts 2024-09-20T18:53:45.682 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:45 smithi083 ceph-mon[29421]: 4.1 scrub ok 2024-09-20T18:53:45.683 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:45 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:53:45.749 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:45.773 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:45.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:45.774 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:45.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:45 smithi142 ceph-mon[32508]: 5.1f scrub starts 2024-09-20T18:53:45.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:45 smithi142 ceph-mon[32508]: 5.1f scrub ok 2024-09-20T18:53:45.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:45 smithi142 ceph-mon[32508]: 4.1 scrub starts 2024-09-20T18:53:45.775 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:45 smithi142 ceph-mon[32508]: 4.1 scrub ok 2024-09-20T18:53:45.775 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:45 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:53:45.875 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:45.901 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:45.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:45.902 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:45.958 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:45 smithi083 ceph-mon[36260]: 5.1f scrub starts 2024-09-20T18:53:45.959 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:45 smithi083 ceph-mon[36260]: 5.1f scrub ok 2024-09-20T18:53:45.959 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:45 smithi083 ceph-mon[36260]: 4.1 scrub starts 2024-09-20T18:53:45.959 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:45 smithi083 ceph-mon[36260]: 4.1 scrub ok 2024-09-20T18:53:45.959 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:45 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:53:45.959 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:53:45.959 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:53:45.963 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:53:45.994 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:53:45.994 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:53:46.000 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_blocked_ops 2024-09-20T18:53:46.001 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:53:46.003 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:46.027 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:46.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:46.028 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:46.029 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:53:46.029 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:53:46.071 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:53:46.072 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:53:46.103 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:53:46.103 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:53:46.129 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:46.141 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:53:46.141 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:53:46.157 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:46.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:46.158 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:46.173 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:53:46.173 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:53:46.206 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:53:46.206 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:53:46.259 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:46.307 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:46.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:46.308 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:46.331 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:53:46.409 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:46.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:46.438 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:46.438 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:46.539 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:46.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:46.577 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:46.577 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:46.678 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:46.723 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:46.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:46.724 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:46.774 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:53:46 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:53:46.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:46 smithi142 ceph-mon[32508]: pgmap v181: 148 pgs: 148 active+clean; 2.8 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 853 op/s 2024-09-20T18:53:46.775 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:46 smithi142 ceph-mon[32508]: 6.7 scrub starts 2024-09-20T18:53:46.775 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:46 smithi142 ceph-mon[32508]: 6.7 scrub ok 2024-09-20T18:53:46.775 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:46 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2111946820' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:53:46.775 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:46 smithi142 ceph-mon[32508]: 6.1b scrub starts 2024-09-20T18:53:46.775 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:46 smithi142 ceph-mon[32508]: 6.1b scrub ok 2024-09-20T18:53:46.775 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:46 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:53:46.824 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:46.863 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:46.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:46.864 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:46.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:46 smithi083 ceph-mon[29421]: pgmap v181: 148 pgs: 148 active+clean; 2.8 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 853 op/s 2024-09-20T18:53:46.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:46 smithi083 ceph-mon[29421]: 6.7 scrub starts 2024-09-20T18:53:46.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:46 smithi083 ceph-mon[29421]: 6.7 scrub ok 2024-09-20T18:53:46.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:46 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2111946820' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:53:46.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:46 smithi083 ceph-mon[29421]: 6.1b scrub starts 2024-09-20T18:53:46.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:46 smithi083 ceph-mon[29421]: 6.1b scrub ok 2024-09-20T18:53:46.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:46 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:53:46.906 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:46 smithi083 ceph-mon[36260]: pgmap v181: 148 pgs: 148 active+clean; 2.8 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 853 op/s 2024-09-20T18:53:46.907 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:46 smithi083 ceph-mon[36260]: 6.7 scrub starts 2024-09-20T18:53:46.907 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:46 smithi083 ceph-mon[36260]: 6.7 scrub ok 2024-09-20T18:53:46.907 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:46 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2111946820' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:53:46.907 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:46 smithi083 ceph-mon[36260]: 6.1b scrub starts 2024-09-20T18:53:46.907 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:46 smithi083 ceph-mon[36260]: 6.1b scrub ok 2024-09-20T18:53:46.907 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:46 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:53:46.929 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T18:53:46.930 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-09-20T18:53:46.930 INFO:teuthology.orchestra.run.smithi142.stdout: "complaint_time": 30, 2024-09-20T18:53:46.930 INFO:teuthology.orchestra.run.smithi142.stdout: "num_blocked_ops": 0 2024-09-20T18:53:46.930 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T18:53:46.965 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:47.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:47.007 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:47.007 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:47.108 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:47.137 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:47.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:47.138 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:47.239 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:47.268 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:47.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:47.269 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:47.370 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:47.413 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:47.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:47.414 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:47.428 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:53:47.430 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-20T18:53:47.514 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:47.549 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:47.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:47.550 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:47.609 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_historic_ops 2024-09-20T18:53:47.651 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:47.681 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:47.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:47.682 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:47.783 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:47.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:47 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:53:47.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:47 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:53:47.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:47 smithi083 ceph-mon[29421]: 5.10 scrub starts 2024-09-20T18:53:47.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:47 smithi083 ceph-mon[29421]: 5.10 scrub ok 2024-09-20T18:53:47.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:47 smithi083 ceph-mon[29421]: 7.c scrub starts 2024-09-20T18:53:47.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:47 smithi083 ceph-mon[29421]: osdmap e148: 8 total, 8 up, 8 in 2024-09-20T18:53:47.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:47 smithi083 ceph-mon[29421]: pgmap v183: 148 pgs: 148 active+clean; 2.5 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.37k op/s 2024-09-20T18:53:47.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:47 smithi083 ceph-mon[29421]: 7.c scrub ok 2024-09-20T18:53:47.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:47 smithi083 ceph-mon[29421]: 5.14 scrub starts 2024-09-20T18:53:47.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:47 smithi083 ceph-mon[29421]: 5.14 scrub ok 2024-09-20T18:53:47.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:47 smithi083 ceph-mon[29421]: 6.8 scrub starts 2024-09-20T18:53:47.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:47 smithi083 ceph-mon[29421]: 6.8 scrub ok 2024-09-20T18:53:47.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:47 smithi083 ceph-mon[29421]: from='client.24935 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:53:47.806 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:47 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:53:47.806 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:47 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:53:47.806 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:47 smithi083 ceph-mon[36260]: 5.10 scrub starts 2024-09-20T18:53:47.806 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:47 smithi083 ceph-mon[36260]: 5.10 scrub ok 2024-09-20T18:53:47.806 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:47 smithi083 ceph-mon[36260]: 7.c scrub starts 2024-09-20T18:53:47.806 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:47 smithi083 ceph-mon[36260]: osdmap e148: 8 total, 8 up, 8 in 2024-09-20T18:53:47.806 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:47 smithi083 ceph-mon[36260]: pgmap v183: 148 pgs: 148 active+clean; 2.5 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.37k op/s 2024-09-20T18:53:47.806 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:47 smithi083 ceph-mon[36260]: 7.c scrub ok 2024-09-20T18:53:47.806 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:47 smithi083 ceph-mon[36260]: 5.14 scrub starts 2024-09-20T18:53:47.806 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:47 smithi083 ceph-mon[36260]: 5.14 scrub ok 2024-09-20T18:53:47.806 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:47 smithi083 ceph-mon[36260]: 6.8 scrub starts 2024-09-20T18:53:47.806 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:47 smithi083 ceph-mon[36260]: 6.8 scrub ok 2024-09-20T18:53:47.807 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:47 smithi083 ceph-mon[36260]: from='client.24935 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:53:47.827 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:47.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:47.828 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:47.929 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:47.945 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:53:47.963 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:47.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:47.964 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:48.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:47 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:53:48.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:47 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:53:48.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:47 smithi142 ceph-mon[32508]: 5.10 scrub starts 2024-09-20T18:53:48.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:47 smithi142 ceph-mon[32508]: 5.10 scrub ok 2024-09-20T18:53:48.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:47 smithi142 ceph-mon[32508]: 7.c scrub starts 2024-09-20T18:53:48.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:47 smithi142 ceph-mon[32508]: osdmap e148: 8 total, 8 up, 8 in 2024-09-20T18:53:48.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:47 smithi142 ceph-mon[32508]: pgmap v183: 148 pgs: 148 active+clean; 2.5 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.37k op/s 2024-09-20T18:53:48.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:47 smithi142 ceph-mon[32508]: 7.c scrub ok 2024-09-20T18:53:48.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:47 smithi142 ceph-mon[32508]: 5.14 scrub starts 2024-09-20T18:53:48.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:47 smithi142 ceph-mon[32508]: 5.14 scrub ok 2024-09-20T18:53:48.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:47 smithi142 ceph-mon[32508]: 6.8 scrub starts 2024-09-20T18:53:48.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:47 smithi142 ceph-mon[32508]: 6.8 scrub ok 2024-09-20T18:53:48.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:47 smithi142 ceph-mon[32508]: from='client.24935 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:53:48.065 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:48.097 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:48.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:48.098 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:48.199 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:48.231 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:48.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:48.232 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:48.333 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:48.357 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":183,"stamp":"2024-09-20T18:53:46.833216+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2652838451,"num_objects":40670,"num_object_clones":0,"num_object_copies":122010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40670,"num_whiteouts":0,"num_read":1084,"num_read_kb":941,"num_write":205678,"num_write_kb":7860810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":590808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":90875,"ondisk_log_size":90875,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":8790644,"kb_used_data":7880232,"kb_used_omap":12,"kb_used_meta":910131,"kb_avail":741170572,"statfs":{"total":767960285184,"available":758958665728,"internally_reserved":0,"allocated":8069357568,"data_stored":8066053329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":931974704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-538312704,"num_objects":-8214,"num_object_clones":0,"num_object_copies":-24642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-8214,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":8214,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001393"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":195,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.673887+0000","last_change":"2024-09-20T18:51:57.238254+0000","last_active":"2024-09-20T18:53:43.673887+0000","last_peered":"2024-09-20T18:53:43.673887+0000","last_clean":"2024-09-20T18:53:43.673887+0000","last_became_active":"2024-09-20T18:49:45.943201+0000","last_became_peered":"2024-09-20T18:49:45.943201+0000","last_unstale":"2024-09-20T18:53:43.673887+0000","last_undegraded":"2024-09-20T18:53:43.673887+0000","last_fullsized":"2024-09-20T18:53:43.673887+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:57.238177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:57.238177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:06.040036+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":233,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.655474+0000","last_change":"2024-09-20T18:53:15.772300+0000","last_active":"2024-09-20T18:53:43.655474+0000","last_peered":"2024-09-20T18:53:43.655474+0000","last_clean":"2024-09-20T18:53:43.655474+0000","last_became_active":"2024-09-20T18:52:15.609648+0000","last_became_peered":"2024-09-20T18:52:15.609648+0000","last_unstale":"2024-09-20T18:53:43.655474+0000","last_undegraded":"2024-09-20T18:53:43.655474+0000","last_fullsized":"2024-09-20T18:53:43.655474+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:53:15.772228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:15.772228+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:33.912905+0000","scrub_duration":37,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":196,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.668345+0000","last_change":"2024-09-20T18:53:21.291759+0000","last_active":"2024-09-20T18:53:43.668345+0000","last_peered":"2024-09-20T18:53:43.668345+0000","last_clean":"2024-09-20T18:53:43.668345+0000","last_became_active":"2024-09-20T18:49:49.959514+0000","last_became_peered":"2024-09-20T18:49:49.959514+0000","last_unstale":"2024-09-20T18:53:43.668345+0000","last_undegraded":"2024-09-20T18:53:43.668345+0000","last_fullsized":"2024-09-20T18:53:43.668345+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:21.291695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:21.291695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:25.316294+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":192,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.655726+0000","last_change":"2024-09-20T18:53:08.740138+0000","last_active":"2024-09-20T18:53:43.655726+0000","last_peered":"2024-09-20T18:53:43.655726+0000","last_clean":"2024-09-20T18:53:43.655726+0000","last_became_active":"2024-09-20T18:49:51.969421+0000","last_became_peered":"2024-09-20T18:49:51.969421+0000","last_unstale":"2024-09-20T18:53:43.655726+0000","last_undegraded":"2024-09-20T18:53:43.655726+0000","last_fullsized":"2024-09-20T18:53:43.655726+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:08.740093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:08.740093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:11.987287+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":184,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.675479+0000","last_change":"2024-09-20T18:51:29.257299+0000","last_active":"2024-09-20T18:53:43.675479+0000","last_peered":"2024-09-20T18:53:43.675479+0000","last_clean":"2024-09-20T18:53:43.675479+0000","last_became_active":"2024-09-20T18:49:52.141967+0000","last_became_peered":"2024-09-20T18:49:52.141967+0000","last_unstale":"2024-09-20T18:53:43.675479+0000","last_undegraded":"2024-09-20T18:53:43.675479+0000","last_fullsized":"2024-09-20T18:53:43.675479+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:29.257220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:29.257220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":196,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.676143+0000","last_change":"2024-09-20T18:51:25.208165+0000","last_active":"2024-09-20T18:53:43.676143+0000","last_peered":"2024-09-20T18:53:43.676143+0000","last_clean":"2024-09-20T18:53:43.676143+0000","last_became_active":"2024-09-20T18:49:45.943607+0000","last_became_peered":"2024-09-20T18:49:45.943607+0000","last_unstale":"2024-09-20T18:53:43.676143+0000","last_undegraded":"2024-09-20T18:53:43.676143+0000","last_fullsized":"2024-09-20T18:53:43.676143+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:25.208073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:25.208073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:07.185746+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":217,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.675428+0000","last_change":"2024-09-20T18:51:07.130137+0000","last_active":"2024-09-20T18:53:43.675428+0000","last_peered":"2024-09-20T18:53:43.675428+0000","last_clean":"2024-09-20T18:53:43.675428+0000","last_became_active":"2024-09-20T18:49:47.952150+0000","last_became_peered":"2024-09-20T18:49:47.952150+0000","last_unstale":"2024-09-20T18:53:43.675428+0000","last_undegraded":"2024-09-20T18:53:43.675428+0000","last_fullsized":"2024-09-20T18:53:43.675428+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:51:07.130062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:07.130062+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:33.208818+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":188,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.655403+0000","last_change":"2024-09-20T18:51:09.436429+0000","last_active":"2024-09-20T18:53:43.655403+0000","last_peered":"2024-09-20T18:53:43.655403+0000","last_clean":"2024-09-20T18:53:43.655403+0000","last_became_active":"2024-09-20T18:49:49.956445+0000","last_became_peered":"2024-09-20T18:49:49.956445+0000","last_unstale":"2024-09-20T18:53:43.655403+0000","last_undegraded":"2024-09-20T18:53:43.655403+0000","last_fullsized":"2024-09-20T18:53:43.655403+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:09.436302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:09.436302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:04.774926+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"59'1","reported_seq":209,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.682185+0000","last_change":"2024-09-20T18:52:48.035578+0000","last_active":"2024-09-20T18:53:43.682185+0000","last_peered":"2024-09-20T18:53:43.682185+0000","last_clean":"2024-09-20T18:53:43.682185+0000","last_became_active":"2024-09-20T18:49:45.946171+0000","last_became_peered":"2024-09-20T18:49:45.946171+0000","last_unstale":"2024-09-20T18:53:43.682185+0000","last_undegraded":"2024-09-20T18:53:43.682185+0000","last_fullsized":"2024-09-20T18:53:43.682185+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:52:48.035532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:48.035532+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:01.343903+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":216,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.656053+0000","last_change":"2024-09-20T18:53:19.836082+0000","last_active":"2024-09-20T18:53:43.656053+0000","last_peered":"2024-09-20T18:53:43.656053+0000","last_clean":"2024-09-20T18:53:43.656053+0000","last_became_active":"2024-09-20T18:49:47.947448+0000","last_became_peered":"2024-09-20T18:49:47.947448+0000","last_unstale":"2024-09-20T18:53:43.656053+0000","last_undegraded":"2024-09-20T18:53:43.656053+0000","last_fullsized":"2024-09-20T18:53:43.656053+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:53:19.835996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:19.835996+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:39.939119+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":195,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.681798+0000","last_change":"2024-09-20T18:52:36.183932+0000","last_active":"2024-09-20T18:53:43.681798+0000","last_peered":"2024-09-20T18:53:43.681798+0000","last_clean":"2024-09-20T18:53:43.681798+0000","last_became_active":"2024-09-20T18:49:49.961188+0000","last_became_peered":"2024-09-20T18:49:49.961188+0000","last_unstale":"2024-09-20T18:53:43.681798+0000","last_undegraded":"2024-09-20T18:53:43.681798+0000","last_fullsized":"2024-09-20T18:53:43.681798+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:36.183875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:36.183875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:49.223570+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":175,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.670137+0000","last_change":"2024-09-20T18:49:51.966957+0000","last_active":"2024-09-20T18:53:43.670137+0000","last_peered":"2024-09-20T18:53:43.670137+0000","last_clean":"2024-09-20T18:53:43.670137+0000","last_became_active":"2024-09-20T18:49:51.966715+0000","last_became_peered":"2024-09-20T18:49:51.966715+0000","last_unstale":"2024-09-20T18:53:43.670137+0000","last_undegraded":"2024-09-20T18:53:43.670137+0000","last_fullsized":"2024-09-20T18:53:43.670137+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:59.013361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":195,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.681982+0000","last_change":"2024-09-20T18:52:00.970719+0000","last_active":"2024-09-20T18:53:43.681982+0000","last_peered":"2024-09-20T18:53:43.681982+0000","last_clean":"2024-09-20T18:53:43.681982+0000","last_became_active":"2024-09-20T18:49:45.944703+0000","last_became_peered":"2024-09-20T18:49:45.944703+0000","last_unstale":"2024-09-20T18:53:43.681982+0000","last_undegraded":"2024-09-20T18:53:43.681982+0000","last_fullsized":"2024-09-20T18:53:43.681982+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:00.970650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:00.970650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:07.049473+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":228,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.655947+0000","last_change":"2024-09-20T18:51:26.494194+0000","last_active":"2024-09-20T18:53:43.655947+0000","last_peered":"2024-09-20T18:53:43.655947+0000","last_clean":"2024-09-20T18:53:43.655947+0000","last_became_active":"2024-09-20T18:51:23.813399+0000","last_became_peered":"2024-09-20T18:51:23.813399+0000","last_unstale":"2024-09-20T18:53:43.655947+0000","last_undegraded":"2024-09-20T18:53:43.655947+0000","last_fullsized":"2024-09-20T18:53:43.655947+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-20T18:51:26.494118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:26.494118+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:34.803592+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":195,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.662238+0000","last_change":"2024-09-20T18:52:38.799542+0000","last_active":"2024-09-20T18:53:43.662238+0000","last_peered":"2024-09-20T18:53:43.662238+0000","last_clean":"2024-09-20T18:53:43.662238+0000","last_became_active":"2024-09-20T18:49:49.959696+0000","last_became_peered":"2024-09-20T18:49:49.959696+0000","last_unstale":"2024-09-20T18:53:43.662238+0000","last_undegraded":"2024-09-20T18:53:43.662238+0000","last_fullsized":"2024-09-20T18:53:43.662238+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:38.799487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:38.799487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:04.479508+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":191,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.681365+0000","last_change":"2024-09-20T18:53:21.273835+0000","last_active":"2024-09-20T18:53:43.681365+0000","last_peered":"2024-09-20T18:53:43.681365+0000","last_clean":"2024-09-20T18:53:43.681365+0000","last_became_active":"2024-09-20T18:49:51.970663+0000","last_became_peered":"2024-09-20T18:49:51.970663+0000","last_unstale":"2024-09-20T18:53:43.681365+0000","last_undegraded":"2024-09-20T18:53:43.681365+0000","last_fullsized":"2024-09-20T18:53:43.681365+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:21.273788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:21.273788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:31.269805+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":184,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.656777+0000","last_change":"2024-09-20T18:52:46.828811+0000","last_active":"2024-09-20T18:53:43.656777+0000","last_peered":"2024-09-20T18:53:43.656777+0000","last_clean":"2024-09-20T18:53:43.656777+0000","last_became_active":"2024-09-20T18:49:52.141170+0000","last_became_peered":"2024-09-20T18:49:52.141170+0000","last_unstale":"2024-09-20T18:53:43.656777+0000","last_undegraded":"2024-09-20T18:53:43.656777+0000","last_fullsized":"2024-09-20T18:53:43.656777+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:46.828749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:46.828749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:50.780880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":195,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.669819+0000","last_change":"2024-09-20T18:52:42.099489+0000","last_active":"2024-09-20T18:53:43.669819+0000","last_peered":"2024-09-20T18:53:43.669819+0000","last_clean":"2024-09-20T18:53:43.669819+0000","last_became_active":"2024-09-20T18:49:45.950480+0000","last_became_peered":"2024-09-20T18:49:45.950480+0000","last_unstale":"2024-09-20T18:53:43.669819+0000","last_undegraded":"2024-09-20T18:53:43.669819+0000","last_fullsized":"2024-09-20T18:53:43.669819+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:42.099418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:42.099418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:03.758400+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":226,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.667299+0000","last_change":"2024-09-20T18:53:26.762207+0000","last_active":"2024-09-20T18:53:43.667299+0000","last_peered":"2024-09-20T18:53:43.667299+0000","last_clean":"2024-09-20T18:53:43.667299+0000","last_became_active":"2024-09-20T18:49:47.950939+0000","last_became_peered":"2024-09-20T18:49:47.950939+0000","last_unstale":"2024-09-20T18:53:43.667299+0000","last_undegraded":"2024-09-20T18:53:43.667299+0000","last_fullsized":"2024-09-20T18:53:43.667299+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:53:26.762131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:26.762131+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:38.227744+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":188,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.671111+0000","last_change":"2024-09-20T18:51:17.248705+0000","last_active":"2024-09-20T18:53:43.671111+0000","last_peered":"2024-09-20T18:53:43.671111+0000","last_clean":"2024-09-20T18:53:43.671111+0000","last_became_active":"2024-09-20T18:49:49.958518+0000","last_became_peered":"2024-09-20T18:49:49.958518+0000","last_unstale":"2024-09-20T18:53:43.671111+0000","last_undegraded":"2024-09-20T18:53:43.671111+0000","last_fullsized":"2024-09-20T18:53:43.671111+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:17.248634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:17.248634+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:35.297969+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":184,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.676064+0000","last_change":"2024-09-20T18:52:03.023669+0000","last_active":"2024-09-20T18:53:43.676064+0000","last_peered":"2024-09-20T18:53:43.676064+0000","last_clean":"2024-09-20T18:53:43.676064+0000","last_became_active":"2024-09-20T18:49:52.140040+0000","last_became_peered":"2024-09-20T18:49:52.140040+0000","last_unstale":"2024-09-20T18:53:43.676064+0000","last_undegraded":"2024-09-20T18:53:43.676064+0000","last_fullsized":"2024-09-20T18:53:43.676064+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:03.023603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:03.023603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"59'1","reported_seq":210,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.656142+0000","last_change":"2024-09-20T18:53:42.840884+0000","last_active":"2024-09-20T18:53:43.656142+0000","last_peered":"2024-09-20T18:53:43.656142+0000","last_clean":"2024-09-20T18:53:43.656142+0000","last_became_active":"2024-09-20T18:49:45.940100+0000","last_became_peered":"2024-09-20T18:49:45.940100+0000","last_unstale":"2024-09-20T18:53:43.656142+0000","last_undegraded":"2024-09-20T18:53:43.656142+0000","last_fullsized":"2024-09-20T18:53:43.656142+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:53:42.840800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:42.840800+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:56.760029+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":207,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.675564+0000","last_change":"2024-09-20T18:49:47.951215+0000","last_active":"2024-09-20T18:53:43.675564+0000","last_peered":"2024-09-20T18:53:43.675564+0000","last_clean":"2024-09-20T18:53:43.675564+0000","last_became_active":"2024-09-20T18:49:47.950909+0000","last_became_peered":"2024-09-20T18:49:47.950909+0000","last_unstale":"2024-09-20T18:53:43.675564+0000","last_undegraded":"2024-09-20T18:53:43.675564+0000","last_fullsized":"2024-09-20T18:53:43.675564+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:00.096165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":196,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.656449+0000","last_change":"2024-09-20T18:53:12.820752+0000","last_active":"2024-09-20T18:53:43.656449+0000","last_peered":"2024-09-20T18:53:43.656449+0000","last_clean":"2024-09-20T18:53:43.656449+0000","last_became_active":"2024-09-20T18:49:49.959102+0000","last_became_peered":"2024-09-20T18:49:49.959102+0000","last_unstale":"2024-09-20T18:53:43.656449+0000","last_undegraded":"2024-09-20T18:53:43.656449+0000","last_fullsized":"2024-09-20T18:53:43.656449+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:12.820697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:12.820697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:29.865011+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":192,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.662234+0000","last_change":"2024-09-20T18:52:15.826819+0000","last_active":"2024-09-20T18:53:43.662234+0000","last_peered":"2024-09-20T18:53:43.662234+0000","last_clean":"2024-09-20T18:53:43.662234+0000","last_became_active":"2024-09-20T18:49:51.970058+0000","last_became_peered":"2024-09-20T18:49:51.970058+0000","last_unstale":"2024-09-20T18:53:43.662234+0000","last_undegraded":"2024-09-20T18:53:43.662234+0000","last_fullsized":"2024-09-20T18:53:43.662234+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T18:52:15.826638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:15.826638+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:28.538285+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":197,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.668474+0000","last_change":"2024-09-20T18:52:47.158384+0000","last_active":"2024-09-20T18:53:43.668474+0000","last_peered":"2024-09-20T18:53:43.668474+0000","last_clean":"2024-09-20T18:53:43.668474+0000","last_became_active":"2024-09-20T18:49:45.933078+0000","last_became_peered":"2024-09-20T18:49:45.933078+0000","last_unstale":"2024-09-20T18:53:43.668474+0000","last_undegraded":"2024-09-20T18:53:43.668474+0000","last_fullsized":"2024-09-20T18:53:43.668474+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:52:47.158260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:47.158260+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:57.719080+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":206,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.663919+0000","last_change":"2024-09-20T18:52:36.175063+0000","last_active":"2024-09-20T18:53:43.663919+0000","last_peered":"2024-09-20T18:53:43.663919+0000","last_clean":"2024-09-20T18:53:43.663919+0000","last_became_active":"2024-09-20T18:49:48.500809+0000","last_became_peered":"2024-09-20T18:49:48.500809+0000","last_unstale":"2024-09-20T18:53:43.663919+0000","last_undegraded":"2024-09-20T18:53:43.663919+0000","last_fullsized":"2024-09-20T18:53:43.663919+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:52:36.174993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:36.174993+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:57.398808+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":254,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:44.524093+0000","last_change":"2024-09-20T18:53:44.524093+0000","last_active":"2024-09-20T18:53:44.524093+0000","last_peered":"2024-09-20T18:53:44.524093+0000","last_clean":"2024-09-20T18:53:44.524093+0000","last_became_active":"2024-09-20T18:49:49.954772+0000","last_became_peered":"2024-09-20T18:49:49.954772+0000","last_unstale":"2024-09-20T18:53:44.524093+0000","last_undegraded":"2024-09-20T18:53:44.524093+0000","last_fullsized":"2024-09-20T18:53:44.524093+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:53:44.524040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:44.524040+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:03.085945+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.c","version":"147'12888","reported_seq":13107,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:45.816352+0000","last_change":"2024-09-20T18:52:16.997356+0000","last_active":"2024-09-20T18:53:45.816352+0000","last_peered":"2024-09-20T18:53:45.816352+0000","last_clean":"2024-09-20T18:53:45.816352+0000","last_became_active":"2024-09-20T18:50:49.843744+0000","last_became_peered":"2024-09-20T18:50:49.843744+0000","last_unstale":"2024-09-20T18:53:45.816352+0000","last_undegraded":"2024-09-20T18:53:45.816352+0000","last_fullsized":"2024-09-20T18:53:45.816352+0000","mapping_epoch":83,"log_start":"114'6600","ondisk_log_start":"114'6600","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"118'7088","last_scrub_stamp":"2024-09-20T18:52:16.997280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:52:16.997280+0000","objects_scrubbed":7043,"log_size":6288,"log_dups_size":0,"ondisk_log_size":6288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"queued for scrub","scrub_duration":1683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":167116800,"num_objects":2550,"num_object_clones":0,"num_object_copies":7650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2550,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12888,"num_write_kb":494016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":201,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.667785+0000","last_change":"2024-09-20T18:52:34.649470+0000","last_active":"2024-09-20T18:53:43.667785+0000","last_peered":"2024-09-20T18:53:43.667785+0000","last_clean":"2024-09-20T18:53:43.667785+0000","last_became_active":"2024-09-20T18:49:58.992302+0000","last_became_peered":"2024-09-20T18:49:58.992302+0000","last_unstale":"2024-09-20T18:53:43.667785+0000","last_undegraded":"2024-09-20T18:53:43.667785+0000","last_fullsized":"2024-09-20T18:53:43.667785+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:52:34.649398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:34.649398+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:47.951675+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":196,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.675382+0000","last_change":"2024-09-20T18:53:26.175148+0000","last_active":"2024-09-20T18:53:43.675382+0000","last_peered":"2024-09-20T18:53:43.675382+0000","last_clean":"2024-09-20T18:53:43.675382+0000","last_became_active":"2024-09-20T18:49:45.936104+0000","last_became_peered":"2024-09-20T18:49:45.936104+0000","last_unstale":"2024-09-20T18:53:43.675382+0000","last_undegraded":"2024-09-20T18:53:43.675382+0000","last_fullsized":"2024-09-20T18:53:43.675382+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:26.175083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:26.175083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:49.575901+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":247,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.656864+0000","last_change":"2024-09-20T18:51:13.395980+0000","last_active":"2024-09-20T18:53:43.656864+0000","last_peered":"2024-09-20T18:53:43.656864+0000","last_clean":"2024-09-20T18:53:43.656864+0000","last_became_active":"2024-09-20T18:49:49.960697+0000","last_became_peered":"2024-09-20T18:49:49.960697+0000","last_unstale":"2024-09-20T18:53:43.656864+0000","last_undegraded":"2024-09-20T18:53:43.656864+0000","last_fullsized":"2024-09-20T18:53:43.656864+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:51:13.395908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:13.395908+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:07.764237+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":183,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.681704+0000","last_change":"2024-09-20T18:52:43.126389+0000","last_active":"2024-09-20T18:53:43.681704+0000","last_peered":"2024-09-20T18:53:43.681704+0000","last_clean":"2024-09-20T18:53:43.681704+0000","last_became_active":"2024-09-20T18:49:51.971399+0000","last_became_peered":"2024-09-20T18:49:51.971399+0000","last_unstale":"2024-09-20T18:53:43.681704+0000","last_undegraded":"2024-09-20T18:53:43.681704+0000","last_fullsized":"2024-09-20T18:53:43.681704+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:43.126320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:43.126320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:52.243850+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"147'12684","reported_seq":12824,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:46.127044+0000","last_change":"2024-09-20T18:50:49.841111+0000","last_active":"2024-09-20T18:53:46.127044+0000","last_peered":"2024-09-20T18:53:46.127044+0000","last_clean":"2024-09-20T18:53:46.127044+0000","last_became_active":"2024-09-20T18:50:49.840819+0000","last_became_peered":"2024-09-20T18:50:49.840819+0000","last_unstale":"2024-09-20T18:53:46.127044+0000","last_undegraded":"2024-09-20T18:53:46.127044+0000","last_fullsized":"2024-09-20T18:53:46.127044+0000","mapping_epoch":83,"log_start":"124'8000","ondisk_log_start":"124'8000","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.819660+0000","objects_scrubbed":0,"log_size":4684,"log_dups_size":0,"ondisk_log_size":4684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:38.259712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":166985728,"num_objects":2548,"num_object_clones":0,"num_object_copies":7644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2548,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12684,"num_write_kb":487424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":224,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.675143+0000","last_change":"2024-09-20T18:52:42.130129+0000","last_active":"2024-09-20T18:53:43.675143+0000","last_peered":"2024-09-20T18:53:43.675143+0000","last_clean":"2024-09-20T18:53:43.675143+0000","last_became_active":"2024-09-20T18:49:48.499189+0000","last_became_peered":"2024-09-20T18:49:48.499189+0000","last_unstale":"2024-09-20T18:53:43.675143+0000","last_undegraded":"2024-09-20T18:53:43.675143+0000","last_fullsized":"2024-09-20T18:53:43.675143+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-20T18:52:42.130045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:42.130045+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:44.974510+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":188,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.676252+0000","last_change":"2024-09-20T18:49:45.940816+0000","last_active":"2024-09-20T18:53:43.676252+0000","last_peered":"2024-09-20T18:53:43.676252+0000","last_clean":"2024-09-20T18:53:43.676252+0000","last_became_active":"2024-09-20T18:49:45.940544+0000","last_became_peered":"2024-09-20T18:49:45.940544+0000","last_unstale":"2024-09-20T18:53:43.676252+0000","last_undegraded":"2024-09-20T18:53:43.676252+0000","last_fullsized":"2024-09-20T18:53:43.676252+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:56.100894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":196,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.655523+0000","last_change":"2024-09-20T18:53:09.779280+0000","last_active":"2024-09-20T18:53:43.655523+0000","last_peered":"2024-09-20T18:53:43.655523+0000","last_clean":"2024-09-20T18:53:43.655523+0000","last_became_active":"2024-09-20T18:49:49.963073+0000","last_became_peered":"2024-09-20T18:49:49.963073+0000","last_unstale":"2024-09-20T18:53:43.655523+0000","last_undegraded":"2024-09-20T18:53:43.655523+0000","last_fullsized":"2024-09-20T18:53:43.655523+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:09.779228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:09.779228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:39.433897+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":192,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.655959+0000","last_change":"2024-09-20T18:53:27.796357+0000","last_active":"2024-09-20T18:53:43.655959+0000","last_peered":"2024-09-20T18:53:43.655959+0000","last_clean":"2024-09-20T18:53:43.655959+0000","last_became_active":"2024-09-20T18:49:51.968793+0000","last_became_peered":"2024-09-20T18:49:51.968793+0000","last_unstale":"2024-09-20T18:53:43.655959+0000","last_undegraded":"2024-09-20T18:53:43.655959+0000","last_fullsized":"2024-09-20T18:53:43.655959+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:27.796289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:27.796289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:48.888717+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2","version":"147'12893","reported_seq":13033,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:46.127230+0000","last_change":"2024-09-20T18:50:49.842736+0000","last_active":"2024-09-20T18:53:46.127230+0000","last_peered":"2024-09-20T18:53:46.127230+0000","last_clean":"2024-09-20T18:53:46.127230+0000","last_became_active":"2024-09-20T18:50:49.842465+0000","last_became_peered":"2024-09-20T18:50:49.842465+0000","last_unstale":"2024-09-20T18:53:46.127230+0000","last_undegraded":"2024-09-20T18:53:46.127230+0000","last_fullsized":"2024-09-20T18:53:46.127230+0000","mapping_epoch":83,"log_start":"125'8200","ondisk_log_start":"125'8200","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.819660+0000","objects_scrubbed":0,"log_size":4693,"log_dups_size":0,"ondisk_log_size":4693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:31.286096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163512320,"num_objects":2495,"num_object_clones":0,"num_object_copies":7485,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12893,"num_write_kb":492416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":229,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:44.879133+0000","last_change":"2024-09-20T18:53:44.879133+0000","last_active":"2024-09-20T18:53:44.879133+0000","last_peered":"2024-09-20T18:53:44.879133+0000","last_clean":"2024-09-20T18:53:44.879133+0000","last_became_active":"2024-09-20T18:49:47.941895+0000","last_became_peered":"2024-09-20T18:49:47.941895+0000","last_unstale":"2024-09-20T18:53:44.879133+0000","last_undegraded":"2024-09-20T18:53:44.879133+0000","last_fullsized":"2024-09-20T18:53:44.879133+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-20T18:53:44.879048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:44.879048+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:52.038099+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":196,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.664158+0000","last_change":"2024-09-20T18:51:27.187060+0000","last_active":"2024-09-20T18:53:43.664158+0000","last_peered":"2024-09-20T18:53:43.664158+0000","last_clean":"2024-09-20T18:53:43.664158+0000","last_became_active":"2024-09-20T18:49:45.940469+0000","last_became_peered":"2024-09-20T18:49:45.940469+0000","last_unstale":"2024-09-20T18:53:43.664158+0000","last_undegraded":"2024-09-20T18:53:43.664158+0000","last_fullsized":"2024-09-20T18:53:43.664158+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:51:27.186991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:27.186991+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:53.996237+0000","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"66'8","reported_seq":253,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:44.293228+0000","last_change":"2024-09-20T18:53:44.293165+0000","last_active":"2024-09-20T18:53:44.293228+0000","last_peered":"2024-09-20T18:53:44.293228+0000","last_clean":"2024-09-20T18:53:44.293228+0000","last_became_active":"2024-09-20T18:49:49.956133+0000","last_became_peered":"2024-09-20T18:49:49.956133+0000","last_unstale":"2024-09-20T18:53:44.293228+0000","last_undegraded":"2024-09-20T18:53:44.293228+0000","last_fullsized":"2024-09-20T18:53:44.293228+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:53:44.293094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:44.293094+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:47.176037+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":191,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.662724+0000","last_change":"2024-09-20T18:52:42.880767+0000","last_active":"2024-09-20T18:53:43.662724+0000","last_peered":"2024-09-20T18:53:43.662724+0000","last_clean":"2024-09-20T18:53:43.662724+0000","last_became_active":"2024-09-20T18:49:52.142113+0000","last_became_peered":"2024-09-20T18:49:52.142113+0000","last_unstale":"2024-09-20T18:53:43.662724+0000","last_undegraded":"2024-09-20T18:53:43.662724+0000","last_fullsized":"2024-09-20T18:53:43.662724+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:42.880724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:42.880724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:04.690799+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"147'12879","reported_seq":13087,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:45.772849+0000","last_change":"2024-09-20T18:52:00.368447+0000","last_active":"2024-09-20T18:53:45.772849+0000","last_peered":"2024-09-20T18:53:45.772849+0000","last_clean":"2024-09-20T18:53:45.772849+0000","last_became_active":"2024-09-20T18:50:49.844624+0000","last_became_peered":"2024-09-20T18:50:49.844624+0000","last_unstale":"2024-09-20T18:53:45.772849+0000","last_undegraded":"2024-09-20T18:53:45.772849+0000","last_fullsized":"2024-09-20T18:53:45.772849+0000","mapping_epoch":83,"log_start":"124'8026","ondisk_log_start":"124'8026","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"111'6092","last_scrub_stamp":"2024-09-20T18:52:00.368276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:52:00.368276+0000","objects_scrubbed":6040,"log_size":4853,"log_dups_size":0,"ondisk_log_size":4853,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:40.875204+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":168099840,"num_objects":2565,"num_object_clones":0,"num_object_copies":7695,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12879,"num_write_kb":494208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":215,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.668322+0000","last_change":"2024-09-20T18:52:39.246125+0000","last_active":"2024-09-20T18:53:43.668322+0000","last_peered":"2024-09-20T18:53:43.668322+0000","last_clean":"2024-09-20T18:53:43.668322+0000","last_became_active":"2024-09-20T18:49:47.949626+0000","last_became_peered":"2024-09-20T18:49:47.949626+0000","last_unstale":"2024-09-20T18:53:43.668322+0000","last_undegraded":"2024-09-20T18:53:43.668322+0000","last_fullsized":"2024-09-20T18:53:43.668322+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:52:39.246050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:39.246050+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:07.125503+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":203,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.682267+0000","last_change":"2024-09-20T18:53:11.130341+0000","last_active":"2024-09-20T18:53:43.682267+0000","last_peered":"2024-09-20T18:53:43.682267+0000","last_clean":"2024-09-20T18:53:43.682267+0000","last_became_active":"2024-09-20T18:49:45.942496+0000","last_became_peered":"2024-09-20T18:49:45.942496+0000","last_unstale":"2024-09-20T18:53:43.682267+0000","last_undegraded":"2024-09-20T18:53:43.682267+0000","last_fullsized":"2024-09-20T18:53:43.682267+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:11.130274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:11.130274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:17.935310+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":244,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.669533+0000","last_change":"2024-09-20T18:51:16.088407+0000","last_active":"2024-09-20T18:53:43.669533+0000","last_peered":"2024-09-20T18:53:43.669533+0000","last_clean":"2024-09-20T18:53:43.669533+0000","last_became_active":"2024-09-20T18:49:49.959365+0000","last_became_peered":"2024-09-20T18:49:49.959365+0000","last_unstale":"2024-09-20T18:53:43.669533+0000","last_undegraded":"2024-09-20T18:53:43.669533+0000","last_fullsized":"2024-09-20T18:53:43.669533+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:51:16.088343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:16.088343+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:35.866980+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":175,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.670292+0000","last_change":"2024-09-20T18:49:51.965733+0000","last_active":"2024-09-20T18:53:43.670292+0000","last_peered":"2024-09-20T18:53:43.670292+0000","last_clean":"2024-09-20T18:53:43.670292+0000","last_became_active":"2024-09-20T18:49:51.964780+0000","last_became_peered":"2024-09-20T18:49:51.964780+0000","last_unstale":"2024-09-20T18:53:43.670292+0000","last_undegraded":"2024-09-20T18:53:43.670292+0000","last_fullsized":"2024-09-20T18:53:43.670292+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:54.992353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.0","version":"147'12969","reported_seq":13184,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:45.790480+0000","last_change":"2024-09-20T18:52:12.840296+0000","last_active":"2024-09-20T18:53:45.790480+0000","last_peered":"2024-09-20T18:53:45.790480+0000","last_clean":"2024-09-20T18:53:45.790480+0000","last_became_active":"2024-09-20T18:50:49.841019+0000","last_became_peered":"2024-09-20T18:50:49.841019+0000","last_unstale":"2024-09-20T18:53:45.790480+0000","last_undegraded":"2024-09-20T18:53:45.790480+0000","last_fullsized":"2024-09-20T18:53:45.790480+0000","mapping_epoch":83,"log_start":"118'7100","ondisk_log_start":"118'7100","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"116'6885","last_scrub_stamp":"2024-09-20T18:52:12.840191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:52:12.840191+0000","objects_scrubbed":6837,"log_size":5869,"log_dups_size":0,"ondisk_log_size":5869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"queued for scrub","scrub_duration":1721,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":167706624,"num_objects":2559,"num_object_clones":0,"num_object_copies":7677,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2559,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12969,"num_write_kb":496896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":227,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.669659+0000","last_change":"2024-09-20T18:52:49.043282+0000","last_active":"2024-09-20T18:53:43.669659+0000","last_peered":"2024-09-20T18:53:43.669659+0000","last_clean":"2024-09-20T18:53:43.669659+0000","last_became_active":"2024-09-20T18:49:48.497736+0000","last_became_peered":"2024-09-20T18:49:48.497736+0000","last_unstale":"2024-09-20T18:53:43.669659+0000","last_undegraded":"2024-09-20T18:53:43.669659+0000","last_fullsized":"2024-09-20T18:53:43.669659+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-20T18:52:49.043189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:49.043189+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:54.819771+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":188,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.671637+0000","last_change":"2024-09-20T18:49:45.942589+0000","last_active":"2024-09-20T18:53:43.671637+0000","last_peered":"2024-09-20T18:53:43.671637+0000","last_clean":"2024-09-20T18:53:43.671637+0000","last_became_active":"2024-09-20T18:49:45.942253+0000","last_became_peered":"2024-09-20T18:49:45.942253+0000","last_unstale":"2024-09-20T18:53:43.671637+0000","last_undegraded":"2024-09-20T18:53:43.671637+0000","last_fullsized":"2024-09-20T18:53:43.671637+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:56.093800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":195,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.672666+0000","last_change":"2024-09-20T18:53:12.347123+0000","last_active":"2024-09-20T18:53:43.672666+0000","last_peered":"2024-09-20T18:53:43.672666+0000","last_clean":"2024-09-20T18:53:43.672666+0000","last_became_active":"2024-09-20T18:49:49.954299+0000","last_became_peered":"2024-09-20T18:49:49.954299+0000","last_unstale":"2024-09-20T18:53:43.672666+0000","last_undegraded":"2024-09-20T18:53:43.672666+0000","last_fullsized":"2024-09-20T18:53:43.672666+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:12.347044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:12.347044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:41.912430+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":192,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.668260+0000","last_change":"2024-09-20T18:53:25.362907+0000","last_active":"2024-09-20T18:53:43.668260+0000","last_peered":"2024-09-20T18:53:43.668260+0000","last_clean":"2024-09-20T18:53:43.668260+0000","last_became_active":"2024-09-20T18:49:52.139981+0000","last_became_peered":"2024-09-20T18:49:52.139981+0000","last_unstale":"2024-09-20T18:53:43.668260+0000","last_undegraded":"2024-09-20T18:53:43.668260+0000","last_fullsized":"2024-09-20T18:53:43.668260+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:25.362828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:25.362828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:55.107515+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"147'12647","reported_seq":12855,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:45.816511+0000","last_change":"2024-09-20T18:52:03.401113+0000","last_active":"2024-09-20T18:53:45.816511+0000","last_peered":"2024-09-20T18:53:45.816511+0000","last_clean":"2024-09-20T18:53:45.816511+0000","last_became_active":"2024-09-20T18:50:49.842473+0000","last_became_peered":"2024-09-20T18:50:49.842473+0000","last_unstale":"2024-09-20T18:53:45.816511+0000","last_undegraded":"2024-09-20T18:53:45.816511+0000","last_fullsized":"2024-09-20T18:53:45.816511+0000","mapping_epoch":83,"log_start":"114'6300","ondisk_log_start":"114'6300","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"112'6142","last_scrub_stamp":"2024-09-20T18:52:03.400886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:52:03.400886+0000","objects_scrubbed":6079,"log_size":6347,"log_dups_size":0,"ondisk_log_size":6347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:41.532845+0000","scrub_duration":2059,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":162594816,"num_objects":2481,"num_object_clones":0,"num_object_copies":7443,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2481,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12647,"num_write_kb":484096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"66'28","reported_seq":288,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.671477+0000","last_change":"2024-09-20T18:51:57.293659+0000","last_active":"2024-09-20T18:53:43.671477+0000","last_peered":"2024-09-20T18:53:43.671477+0000","last_clean":"2024-09-20T18:53:43.671477+0000","last_became_active":"2024-09-20T18:49:47.955335+0000","last_became_peered":"2024-09-20T18:49:47.955335+0000","last_unstale":"2024-09-20T18:53:43.671477+0000","last_undegraded":"2024-09-20T18:53:43.671477+0000","last_fullsized":"2024-09-20T18:53:43.671477+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'28","last_scrub_stamp":"2024-09-20T18:51:57.293574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:57.293574+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:06.036152+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":188,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.656246+0000","last_change":"2024-09-20T18:49:45.945275+0000","last_active":"2024-09-20T18:53:43.656246+0000","last_peered":"2024-09-20T18:53:43.656246+0000","last_clean":"2024-09-20T18:53:43.656246+0000","last_became_active":"2024-09-20T18:49:45.944992+0000","last_became_peered":"2024-09-20T18:49:45.944992+0000","last_unstale":"2024-09-20T18:53:43.656246+0000","last_undegraded":"2024-09-20T18:53:43.656246+0000","last_fullsized":"2024-09-20T18:53:43.656246+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:54.777986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":213,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.681829+0000","last_change":"2024-09-20T18:52:15.112959+0000","last_active":"2024-09-20T18:53:43.681829+0000","last_peered":"2024-09-20T18:53:43.681829+0000","last_clean":"2024-09-20T18:53:43.681829+0000","last_became_active":"2024-09-20T18:49:44.936027+0000","last_became_peered":"2024-09-20T18:49:44.936027+0000","last_unstale":"2024-09-20T18:53:43.681829+0000","last_undegraded":"2024-09-20T18:53:43.681829+0000","last_fullsized":"2024-09-20T18:53:43.681829+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-20T18:52:15.112895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:52:15.112895+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0012781159999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":195,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.665349+0000","last_change":"2024-09-20T18:53:12.897519+0000","last_active":"2024-09-20T18:53:43.665349+0000","last_peered":"2024-09-20T18:53:43.665349+0000","last_clean":"2024-09-20T18:53:43.665349+0000","last_became_active":"2024-09-20T18:49:49.953971+0000","last_became_peered":"2024-09-20T18:49:49.953971+0000","last_unstale":"2024-09-20T18:53:43.665349+0000","last_undegraded":"2024-09-20T18:53:43.665349+0000","last_fullsized":"2024-09-20T18:53:43.665349+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:12.897051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:12.897051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:39.168470+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":176,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.675745+0000","last_change":"2024-09-20T18:49:51.969702+0000","last_active":"2024-09-20T18:53:43.675745+0000","last_peered":"2024-09-20T18:53:43.675745+0000","last_clean":"2024-09-20T18:53:43.675745+0000","last_became_active":"2024-09-20T18:49:51.969400+0000","last_became_peered":"2024-09-20T18:49:51.969400+0000","last_unstale":"2024-09-20T18:53:43.675745+0000","last_undegraded":"2024-09-20T18:53:43.675745+0000","last_fullsized":"2024-09-20T18:53:43.675745+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:36.285843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"147'12729","reported_seq":12916,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:46.169976+0000","last_change":"2024-09-20T18:53:23.208574+0000","last_active":"2024-09-20T18:53:46.169976+0000","last_peered":"2024-09-20T18:53:46.169976+0000","last_clean":"2024-09-20T18:53:46.169976+0000","last_became_active":"2024-09-20T18:50:49.843321+0000","last_became_peered":"2024-09-20T18:50:49.843321+0000","last_unstale":"2024-09-20T18:53:46.169976+0000","last_undegraded":"2024-09-20T18:53:46.169976+0000","last_fullsized":"2024-09-20T18:53:46.169976+0000","mapping_epoch":83,"log_start":"119'7100","ondisk_log_start":"119'7100","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"140'11334","last_scrub_stamp":"2024-09-20T18:53:23.208497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:53:23.208497+0000","objects_scrubbed":3959,"log_size":5629,"log_dups_size":0,"ondisk_log_size":5629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:48.313048+0000","scrub_duration":1357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":165216256,"num_objects":2521,"num_object_clones":0,"num_object_copies":7563,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12729,"num_write_kb":488000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":219,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.668577+0000","last_change":"2024-09-20T18:52:37.246051+0000","last_active":"2024-09-20T18:53:43.668577+0000","last_peered":"2024-09-20T18:53:43.668577+0000","last_clean":"2024-09-20T18:53:43.668577+0000","last_became_active":"2024-09-20T18:49:47.950904+0000","last_became_peered":"2024-09-20T18:49:47.950904+0000","last_unstale":"2024-09-20T18:53:43.668577+0000","last_undegraded":"2024-09-20T18:53:43.668577+0000","last_fullsized":"2024-09-20T18:53:43.668577+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-20T18:52:37.245970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:37.245970+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:46.997532+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":204,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.670010+0000","last_change":"2024-09-20T18:53:19.278005+0000","last_active":"2024-09-20T18:53:43.670010+0000","last_peered":"2024-09-20T18:53:43.670010+0000","last_clean":"2024-09-20T18:53:43.670010+0000","last_became_active":"2024-09-20T18:49:45.939884+0000","last_became_peered":"2024-09-20T18:49:45.939884+0000","last_unstale":"2024-09-20T18:53:43.670010+0000","last_undegraded":"2024-09-20T18:53:43.670010+0000","last_fullsized":"2024-09-20T18:53:43.670010+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:19.277650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:19.277650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:22.181893+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":206,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.667584+0000","last_change":"2024-09-20T18:52:07.718766+0000","last_active":"2024-09-20T18:53:43.667584+0000","last_peered":"2024-09-20T18:53:43.667584+0000","last_clean":"2024-09-20T18:53:43.667584+0000","last_became_active":"2024-09-20T18:49:44.937135+0000","last_became_peered":"2024-09-20T18:49:44.937135+0000","last_unstale":"2024-09-20T18:53:43.667584+0000","last_undegraded":"2024-09-20T18:53:43.667584+0000","last_fullsized":"2024-09-20T18:53:43.667584+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:52:07.718646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:52:07.718646+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:37.805116+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.00095558800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":189,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.670958+0000","last_change":"2024-09-20T18:51:59.796450+0000","last_active":"2024-09-20T18:53:43.670958+0000","last_peered":"2024-09-20T18:53:43.670958+0000","last_clean":"2024-09-20T18:53:43.670958+0000","last_became_active":"2024-09-20T18:49:49.954424+0000","last_became_peered":"2024-09-20T18:49:49.954424+0000","last_unstale":"2024-09-20T18:53:43.670958+0000","last_undegraded":"2024-09-20T18:53:43.670958+0000","last_fullsized":"2024-09-20T18:53:43.670958+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:59.796383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:59.796383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:03.718427+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":183,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.663096+0000","last_change":"2024-09-20T18:52:39.814324+0000","last_active":"2024-09-20T18:53:43.663096+0000","last_peered":"2024-09-20T18:53:43.663096+0000","last_clean":"2024-09-20T18:53:43.663096+0000","last_became_active":"2024-09-20T18:49:52.139789+0000","last_became_peered":"2024-09-20T18:49:52.139789+0000","last_unstale":"2024-09-20T18:53:43.663096+0000","last_undegraded":"2024-09-20T18:53:43.663096+0000","last_fullsized":"2024-09-20T18:53:43.663096+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:39.814259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:39.814259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:55.444061+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"147'12942","reported_seq":13204,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:45.788654+0000","last_change":"2024-09-20T18:53:24.693140+0000","last_active":"2024-09-20T18:53:45.788654+0000","last_peered":"2024-09-20T18:53:45.788654+0000","last_clean":"2024-09-20T18:53:45.788654+0000","last_became_active":"2024-09-20T18:50:49.846124+0000","last_became_peered":"2024-09-20T18:50:49.846124+0000","last_unstale":"2024-09-20T18:53:45.788654+0000","last_undegraded":"2024-09-20T18:53:45.788654+0000","last_fullsized":"2024-09-20T18:53:45.788654+0000","mapping_epoch":83,"log_start":"122'7700","ondisk_log_start":"122'7700","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"140'11593","last_scrub_stamp":"2024-09-20T18:53:24.692917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:53:24.692917+0000","objects_scrubbed":4004,"log_size":5242,"log_dups_size":0,"ondisk_log_size":5242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:35.585228+0000","scrub_duration":1122,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":171048960,"num_objects":2610,"num_object_clones":0,"num_object_copies":7830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2610,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12942,"num_write_kb":497664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":217,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.664002+0000","last_change":"2024-09-20T18:52:43.089145+0000","last_active":"2024-09-20T18:53:43.664002+0000","last_peered":"2024-09-20T18:53:43.664002+0000","last_clean":"2024-09-20T18:53:43.664002+0000","last_became_active":"2024-09-20T18:49:47.949230+0000","last_became_peered":"2024-09-20T18:49:47.949230+0000","last_unstale":"2024-09-20T18:53:43.664002+0000","last_undegraded":"2024-09-20T18:53:43.664002+0000","last_fullsized":"2024-09-20T18:53:43.664002+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:52:43.089065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:43.089065+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:47.134487+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":203,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.665567+0000","last_change":"2024-09-20T18:53:09.891235+0000","last_active":"2024-09-20T18:53:43.665567+0000","last_peered":"2024-09-20T18:53:43.665567+0000","last_clean":"2024-09-20T18:53:43.665567+0000","last_became_active":"2024-09-20T18:49:45.939197+0000","last_became_peered":"2024-09-20T18:49:45.939197+0000","last_unstale":"2024-09-20T18:53:43.665567+0000","last_undegraded":"2024-09-20T18:53:43.665567+0000","last_fullsized":"2024-09-20T18:53:43.665567+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:09.890987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:09.890987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:17.075156+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":450,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:45.579294+0000","last_change":"2024-09-20T18:53:22.616236+0000","last_active":"2024-09-20T18:53:45.579294+0000","last_peered":"2024-09-20T18:53:45.579294+0000","last_clean":"2024-09-20T18:53:45.579294+0000","last_became_active":"2024-09-20T18:49:44.936819+0000","last_became_peered":"2024-09-20T18:49:44.936819+0000","last_unstale":"2024-09-20T18:53:45.579294+0000","last_undegraded":"2024-09-20T18:53:45.579294+0000","last_fullsized":"2024-09-20T18:53:45.579294+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-20T18:53:22.616166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:53:22.616166+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:25.872696+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0010996199999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":235,"num_read_kb":230,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":187,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.682482+0000","last_change":"2024-09-20T18:52:02.981845+0000","last_active":"2024-09-20T18:53:43.682482+0000","last_peered":"2024-09-20T18:53:43.682482+0000","last_clean":"2024-09-20T18:53:43.682482+0000","last_became_active":"2024-09-20T18:49:49.961066+0000","last_became_peered":"2024-09-20T18:49:49.961066+0000","last_unstale":"2024-09-20T18:53:43.682482+0000","last_undegraded":"2024-09-20T18:53:43.682482+0000","last_fullsized":"2024-09-20T18:53:43.682482+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:02.981762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:02.981762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":184,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.671211+0000","last_change":"2024-09-20T18:52:45.158726+0000","last_active":"2024-09-20T18:53:43.671211+0000","last_peered":"2024-09-20T18:53:43.671211+0000","last_clean":"2024-09-20T18:53:43.671211+0000","last_became_active":"2024-09-20T18:49:51.970343+0000","last_became_peered":"2024-09-20T18:49:51.970343+0000","last_unstale":"2024-09-20T18:53:43.671211+0000","last_undegraded":"2024-09-20T18:53:43.671211+0000","last_fullsized":"2024-09-20T18:53:43.671211+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:45.158601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:45.158601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:47.806684+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"147'12924","reported_seq":13140,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:46.162899+0000","last_change":"2024-09-20T18:52:13.298242+0000","last_active":"2024-09-20T18:53:46.162899+0000","last_peered":"2024-09-20T18:53:46.162899+0000","last_clean":"2024-09-20T18:53:46.162899+0000","last_became_active":"2024-09-20T18:50:49.842310+0000","last_became_peered":"2024-09-20T18:50:49.842310+0000","last_unstale":"2024-09-20T18:53:46.162899+0000","last_undegraded":"2024-09-20T18:53:46.162899+0000","last_fullsized":"2024-09-20T18:53:46.162899+0000","mapping_epoch":83,"log_start":"120'7300","ondisk_log_start":"120'7300","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"116'6867","last_scrub_stamp":"2024-09-20T18:52:13.297990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:52:13.297990+0000","objects_scrubbed":6775,"log_size":5624,"log_dups_size":0,"ondisk_log_size":5624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:40.850770+0000","scrub_duration":3018,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":161873920,"num_objects":2470,"num_object_clones":0,"num_object_copies":7410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2470,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12924,"num_write_kb":492608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":253,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.673225+0000","last_change":"2024-09-20T18:49:48.498345+0000","last_active":"2024-09-20T18:53:43.673225+0000","last_peered":"2024-09-20T18:53:43.673225+0000","last_clean":"2024-09-20T18:53:43.673225+0000","last_became_active":"2024-09-20T18:49:48.497936+0000","last_became_peered":"2024-09-20T18:49:48.497936+0000","last_unstale":"2024-09-20T18:53:43.673225+0000","last_undegraded":"2024-09-20T18:53:43.673225+0000","last_fullsized":"2024-09-20T18:53:43.673225+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:57.367087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":204,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.676084+0000","last_change":"2024-09-20T18:53:13.145127+0000","last_active":"2024-09-20T18:53:43.676084+0000","last_peered":"2024-09-20T18:53:43.676084+0000","last_clean":"2024-09-20T18:53:43.676084+0000","last_became_active":"2024-09-20T18:49:45.941136+0000","last_became_peered":"2024-09-20T18:49:45.941136+0000","last_unstale":"2024-09-20T18:53:43.676084+0000","last_undegraded":"2024-09-20T18:53:43.676084+0000","last_fullsized":"2024-09-20T18:53:43.676084+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:13.144896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:13.144896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:18.711819+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":243,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.662461+0000","last_change":"2024-09-20T18:52:11.887878+0000","last_active":"2024-09-20T18:53:43.662461+0000","last_peered":"2024-09-20T18:53:43.662461+0000","last_clean":"2024-09-20T18:53:43.662461+0000","last_became_active":"2024-09-20T18:49:36.545306+0000","last_became_peered":"2024-09-20T18:49:36.545306+0000","last_unstale":"2024-09-20T18:53:43.662461+0000","last_undegraded":"2024-09-20T18:53:43.662461+0000","last_fullsized":"2024-09-20T18:53:43.662461+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T18:52:11.887733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T18:52:11.887733+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:32.438028+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":187,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.662692+0000","last_change":"2024-09-20T18:51:22.852942+0000","last_active":"2024-09-20T18:53:43.662692+0000","last_peered":"2024-09-20T18:53:43.662692+0000","last_clean":"2024-09-20T18:53:43.662692+0000","last_became_active":"2024-09-20T18:49:49.956128+0000","last_became_peered":"2024-09-20T18:49:49.956128+0000","last_unstale":"2024-09-20T18:53:43.662692+0000","last_undegraded":"2024-09-20T18:53:43.662692+0000","last_fullsized":"2024-09-20T18:53:43.662692+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:22.852869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:22.852869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:37.516435+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":191,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:45.306565+0000","last_change":"2024-09-20T18:53:45.306565+0000","last_active":"2024-09-20T18:53:45.306565+0000","last_peered":"2024-09-20T18:53:45.306565+0000","last_clean":"2024-09-20T18:53:45.306565+0000","last_became_active":"2024-09-20T18:49:51.971332+0000","last_became_peered":"2024-09-20T18:49:51.971332+0000","last_unstale":"2024-09-20T18:53:45.306565+0000","last_undegraded":"2024-09-20T18:53:45.306565+0000","last_fullsized":"2024-09-20T18:53:45.306565+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:45.306514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:45.306514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:10.093423+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"147'12788","reported_seq":13063,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:45.820965+0000","last_change":"2024-09-20T18:52:51.095285+0000","last_active":"2024-09-20T18:53:45.820965+0000","last_peered":"2024-09-20T18:53:45.820965+0000","last_clean":"2024-09-20T18:53:45.820965+0000","last_became_active":"2024-09-20T18:50:49.842483+0000","last_became_peered":"2024-09-20T18:50:49.842483+0000","last_unstale":"2024-09-20T18:53:45.820965+0000","last_undegraded":"2024-09-20T18:53:45.820965+0000","last_fullsized":"2024-09-20T18:53:45.820965+0000","mapping_epoch":83,"log_start":"114'6500","ondisk_log_start":"114'6500","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"131'9234","last_scrub_stamp":"2024-09-20T18:52:51.095167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:52:51.095167+0000","objects_scrubbed":6083,"log_size":6288,"log_dups_size":0,"ondisk_log_size":6288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:09.470192+0000","scrub_duration":1756,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":161415192,"num_objects":2464,"num_object_clones":0,"num_object_copies":7392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2464,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12788,"num_write_kb":488001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":211,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.656950+0000","last_change":"2024-09-20T18:51:14.397996+0000","last_active":"2024-09-20T18:53:43.656950+0000","last_peered":"2024-09-20T18:53:43.656950+0000","last_clean":"2024-09-20T18:53:43.656950+0000","last_became_active":"2024-09-20T18:49:47.950073+0000","last_became_peered":"2024-09-20T18:49:47.950073+0000","last_unstale":"2024-09-20T18:53:43.656950+0000","last_undegraded":"2024-09-20T18:53:43.656950+0000","last_fullsized":"2024-09-20T18:53:43.656950+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:51:14.397930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.397930+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:32.825060+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":196,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.655744+0000","last_change":"2024-09-20T18:51:18.410601+0000","last_active":"2024-09-20T18:53:43.655744+0000","last_peered":"2024-09-20T18:53:43.655744+0000","last_clean":"2024-09-20T18:53:43.655744+0000","last_became_active":"2024-09-20T18:49:45.944579+0000","last_became_peered":"2024-09-20T18:49:45.944579+0000","last_unstale":"2024-09-20T18:53:43.655744+0000","last_undegraded":"2024-09-20T18:53:43.655744+0000","last_fullsized":"2024-09-20T18:53:43.655744+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:18.410447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:18.410447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:29.855147+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":195,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.682017+0000","last_change":"2024-09-20T18:52:38.128487+0000","last_active":"2024-09-20T18:53:43.682017+0000","last_peered":"2024-09-20T18:53:43.682017+0000","last_clean":"2024-09-20T18:53:43.682017+0000","last_became_active":"2024-09-20T18:49:49.954391+0000","last_became_peered":"2024-09-20T18:49:49.954391+0000","last_unstale":"2024-09-20T18:53:43.682017+0000","last_undegraded":"2024-09-20T18:53:43.682017+0000","last_fullsized":"2024-09-20T18:53:43.682017+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:38.128438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:38.128438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:55.529118+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":184,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.675383+0000","last_change":"2024-09-20T18:52:05.021228+0000","last_active":"2024-09-20T18:53:43.675383+0000","last_peered":"2024-09-20T18:53:43.675383+0000","last_clean":"2024-09-20T18:53:43.675383+0000","last_became_active":"2024-09-20T18:49:52.139959+0000","last_became_peered":"2024-09-20T18:49:52.139959+0000","last_unstale":"2024-09-20T18:53:43.675383+0000","last_undegraded":"2024-09-20T18:53:43.675383+0000","last_fullsized":"2024-09-20T18:53:43.675383+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:05.021071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.021071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"147'12706","reported_seq":12897,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:45.978729+0000","last_change":"2024-09-20T18:53:22.451111+0000","last_active":"2024-09-20T18:53:45.978729+0000","last_peered":"2024-09-20T18:53:45.978729+0000","last_clean":"2024-09-20T18:53:45.978729+0000","last_became_active":"2024-09-20T18:50:49.841683+0000","last_became_peered":"2024-09-20T18:50:49.841683+0000","last_unstale":"2024-09-20T18:53:45.978729+0000","last_undegraded":"2024-09-20T18:53:45.978729+0000","last_fullsized":"2024-09-20T18:53:45.978729+0000","mapping_epoch":83,"log_start":"111'5800","ondisk_log_start":"111'5800","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'11259","last_scrub_stamp":"2024-09-20T18:53:22.450961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:53:22.450961+0000","objects_scrubbed":4058,"log_size":6906,"log_dups_size":0,"ondisk_log_size":6906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:26.478768+0000","scrub_duration":1718,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":167247872,"num_objects":2552,"num_object_clones":0,"num_object_copies":7656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12706,"num_write_kb":488256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":228,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.656951+0000","last_change":"2024-09-20T18:52:02.944536+0000","last_active":"2024-09-20T18:53:43.656951+0000","last_peered":"2024-09-20T18:53:43.656951+0000","last_clean":"2024-09-20T18:53:43.656951+0000","last_became_active":"2024-09-20T18:49:47.947636+0000","last_became_peered":"2024-09-20T18:49:47.947636+0000","last_unstale":"2024-09-20T18:53:43.656951+0000","last_undegraded":"2024-09-20T18:53:43.656951+0000","last_fullsized":"2024-09-20T18:53:43.656951+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-20T18:52:02.944434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:02.944434+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:30.835162+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":217,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.672805+0000","last_change":"2024-09-20T18:53:23.472809+0000","last_active":"2024-09-20T18:53:43.672805+0000","last_peered":"2024-09-20T18:53:43.672805+0000","last_clean":"2024-09-20T18:53:43.672805+0000","last_became_active":"2024-09-20T18:51:55.596689+0000","last_became_peered":"2024-09-20T18:51:55.596689+0000","last_unstale":"2024-09-20T18:53:43.672805+0000","last_undegraded":"2024-09-20T18:53:43.672805+0000","last_fullsized":"2024-09-20T18:53:43.672805+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:23.472757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:23.472757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:39.082278+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":188,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.671303+0000","last_change":"2024-09-20T18:52:49.137080+0000","last_active":"2024-09-20T18:53:43.671303+0000","last_peered":"2024-09-20T18:53:43.671303+0000","last_clean":"2024-09-20T18:53:43.671303+0000","last_became_active":"2024-09-20T18:49:49.959160+0000","last_became_peered":"2024-09-20T18:49:49.959160+0000","last_unstale":"2024-09-20T18:53:43.671303+0000","last_undegraded":"2024-09-20T18:53:43.671303+0000","last_fullsized":"2024-09-20T18:53:43.671303+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:49.137023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:49.137023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:07.149566+0000","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":193,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.667920+0000","last_change":"2024-09-20T18:52:35.635897+0000","last_active":"2024-09-20T18:53:43.667920+0000","last_peered":"2024-09-20T18:53:43.667920+0000","last_clean":"2024-09-20T18:53:43.667920+0000","last_became_active":"2024-09-20T18:49:51.970108+0000","last_became_peered":"2024-09-20T18:49:51.970108+0000","last_unstale":"2024-09-20T18:53:43.667920+0000","last_undegraded":"2024-09-20T18:53:43.667920+0000","last_fullsized":"2024-09-20T18:53:43.667920+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:35.635818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:35.635818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:56.643452+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"147'12945","reported_seq":13194,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:46.071858+0000","last_change":"2024-09-20T18:52:52.583857+0000","last_active":"2024-09-20T18:53:46.071858+0000","last_peered":"2024-09-20T18:53:46.071858+0000","last_clean":"2024-09-20T18:53:46.071858+0000","last_became_active":"2024-09-20T18:50:49.841628+0000","last_became_peered":"2024-09-20T18:50:49.841628+0000","last_unstale":"2024-09-20T18:53:46.071858+0000","last_undegraded":"2024-09-20T18:53:46.071858+0000","last_fullsized":"2024-09-20T18:53:46.071858+0000","mapping_epoch":83,"log_start":"119'7300","ondisk_log_start":"119'7300","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"131'9412","last_scrub_stamp":"2024-09-20T18:52:52.583713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:52:52.583713+0000","objects_scrubbed":6144,"log_size":5645,"log_dups_size":0,"ondisk_log_size":5645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:55.820131+0000","scrub_duration":2542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":165871616,"num_objects":2531,"num_object_clones":0,"num_object_copies":7593,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2531,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12945,"num_write_kb":495232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":207,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.656523+0000","last_change":"2024-09-20T18:52:52.775966+0000","last_active":"2024-09-20T18:53:43.656523+0000","last_peered":"2024-09-20T18:53:43.656523+0000","last_clean":"2024-09-20T18:53:43.656523+0000","last_became_active":"2024-09-20T18:49:47.947126+0000","last_became_peered":"2024-09-20T18:49:47.947126+0000","last_unstale":"2024-09-20T18:53:43.656523+0000","last_undegraded":"2024-09-20T18:53:43.656523+0000","last_fullsized":"2024-09-20T18:53:43.656523+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:52:52.775889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:52.775889+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:14.126093+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":204,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.675759+0000","last_change":"2024-09-20T18:52:46.005172+0000","last_active":"2024-09-20T18:53:43.675759+0000","last_peered":"2024-09-20T18:53:43.675759+0000","last_clean":"2024-09-20T18:53:43.675759+0000","last_became_active":"2024-09-20T18:49:45.937264+0000","last_became_peered":"2024-09-20T18:49:45.937264+0000","last_unstale":"2024-09-20T18:53:43.675759+0000","last_undegraded":"2024-09-20T18:53:43.675759+0000","last_fullsized":"2024-09-20T18:53:43.675759+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:46.005109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:46.005109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:04.505427+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":253,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.661908+0000","last_change":"2024-09-20T18:52:40.647833+0000","last_active":"2024-09-20T18:53:43.661908+0000","last_peered":"2024-09-20T18:53:43.661908+0000","last_clean":"2024-09-20T18:53:43.661908+0000","last_became_active":"2024-09-20T18:49:49.957005+0000","last_became_peered":"2024-09-20T18:49:49.957005+0000","last_unstale":"2024-09-20T18:53:43.661908+0000","last_undegraded":"2024-09-20T18:53:43.661908+0000","last_fullsized":"2024-09-20T18:53:43.661908+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:52:40.647761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:40.647761+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:03.992292+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":184,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.656691+0000","last_change":"2024-09-20T18:52:10.024610+0000","last_active":"2024-09-20T18:53:43.656691+0000","last_peered":"2024-09-20T18:53:43.656691+0000","last_clean":"2024-09-20T18:53:43.656691+0000","last_became_active":"2024-09-20T18:49:51.969355+0000","last_became_peered":"2024-09-20T18:49:51.969355+0000","last_unstale":"2024-09-20T18:53:43.656691+0000","last_undegraded":"2024-09-20T18:53:43.656691+0000","last_fullsized":"2024-09-20T18:53:43.656691+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:10.024557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:10.024557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:38.850775+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.8","version":"147'12875","reported_seq":13093,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:45.917273+0000","last_change":"2024-09-20T18:52:16.931599+0000","last_active":"2024-09-20T18:53:45.917273+0000","last_peered":"2024-09-20T18:53:45.917273+0000","last_clean":"2024-09-20T18:53:45.917273+0000","last_became_active":"2024-09-20T18:50:49.844028+0000","last_became_peered":"2024-09-20T18:50:49.844028+0000","last_unstale":"2024-09-20T18:53:45.917273+0000","last_undegraded":"2024-09-20T18:53:45.917273+0000","last_fullsized":"2024-09-20T18:53:45.917273+0000","mapping_epoch":83,"log_start":"111'5980","ondisk_log_start":"111'5980","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"118'7065","last_scrub_stamp":"2024-09-20T18:52:16.931418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:52:16.931418+0000","objects_scrubbed":7008,"log_size":6895,"log_dups_size":0,"ondisk_log_size":6895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:31.738778+0000","scrub_duration":2274,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":167444480,"num_objects":2555,"num_object_clones":0,"num_object_copies":7665,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2555,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12875,"num_write_kb":493760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":215,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.669579+0000","last_change":"2024-09-20T18:53:17.863709+0000","last_active":"2024-09-20T18:53:43.669579+0000","last_peered":"2024-09-20T18:53:43.669579+0000","last_clean":"2024-09-20T18:53:43.669579+0000","last_became_active":"2024-09-20T18:49:47.946008+0000","last_became_peered":"2024-09-20T18:49:47.946008+0000","last_unstale":"2024-09-20T18:53:43.669579+0000","last_undegraded":"2024-09-20T18:53:43.669579+0000","last_fullsized":"2024-09-20T18:53:43.669579+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:53:17.863623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:17.863623+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:35.672196+0000","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":203,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.682089+0000","last_change":"2024-09-20T18:52:17.120051+0000","last_active":"2024-09-20T18:53:43.682089+0000","last_peered":"2024-09-20T18:53:43.682089+0000","last_clean":"2024-09-20T18:53:43.682089+0000","last_became_active":"2024-09-20T18:49:45.941406+0000","last_became_peered":"2024-09-20T18:49:45.941406+0000","last_unstale":"2024-09-20T18:53:43.682089+0000","last_undegraded":"2024-09-20T18:53:43.682089+0000","last_fullsized":"2024-09-20T18:53:43.682089+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:17.119986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:17.119986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":197,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.672654+0000","last_change":"2024-09-20T18:53:27.722068+0000","last_active":"2024-09-20T18:53:43.672654+0000","last_peered":"2024-09-20T18:53:43.672654+0000","last_clean":"2024-09-20T18:53:43.672654+0000","last_became_active":"2024-09-20T18:49:49.956767+0000","last_became_peered":"2024-09-20T18:49:49.956767+0000","last_unstale":"2024-09-20T18:53:43.672654+0000","last_undegraded":"2024-09-20T18:53:43.672654+0000","last_fullsized":"2024-09-20T18:53:43.672654+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:27.721940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:27.721940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:42.447352+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":183,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.665494+0000","last_change":"2024-09-20T18:52:46.004013+0000","last_active":"2024-09-20T18:53:43.665494+0000","last_peered":"2024-09-20T18:53:43.665494+0000","last_clean":"2024-09-20T18:53:43.665494+0000","last_became_active":"2024-09-20T18:49:51.964098+0000","last_became_peered":"2024-09-20T18:49:51.964098+0000","last_unstale":"2024-09-20T18:53:43.665494+0000","last_undegraded":"2024-09-20T18:53:43.665494+0000","last_fullsized":"2024-09-20T18:53:43.665494+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:46.003836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:46.003836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:04.821993+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"147'12701","reported_seq":12950,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:46.155263+0000","last_change":"2024-09-20T18:53:43.753759+0000","last_active":"2024-09-20T18:53:46.155263+0000","last_peered":"2024-09-20T18:53:46.155263+0000","last_clean":"2024-09-20T18:53:46.155263+0000","last_became_active":"2024-09-20T18:50:49.840023+0000","last_became_peered":"2024-09-20T18:50:49.840023+0000","last_unstale":"2024-09-20T18:53:46.155263+0000","last_undegraded":"2024-09-20T18:53:46.155263+0000","last_fullsized":"2024-09-20T18:53:46.155263+0000","mapping_epoch":83,"log_start":"119'7100","ondisk_log_start":"119'7100","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'12545","last_scrub_stamp":"2024-09-20T18:53:43.753508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:53:43.753508+0000","objects_scrubbed":2735,"log_size":5601,"log_dups_size":0,"ondisk_log_size":5601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:44.964709+0000","scrub_duration":892,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":167706624,"num_objects":2559,"num_object_clones":0,"num_object_copies":7677,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2559,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12701,"num_write_kb":488320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":230,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.672689+0000","last_change":"2024-09-20T18:53:24.458093+0000","last_active":"2024-09-20T18:53:43.672689+0000","last_peered":"2024-09-20T18:53:43.672689+0000","last_clean":"2024-09-20T18:53:43.672689+0000","last_became_active":"2024-09-20T18:49:48.498625+0000","last_became_peered":"2024-09-20T18:49:48.498625+0000","last_unstale":"2024-09-20T18:53:43.672689+0000","last_undegraded":"2024-09-20T18:53:43.672689+0000","last_fullsized":"2024-09-20T18:53:43.672689+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-20T18:53:24.458005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:24.458005+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:31.316232+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":203,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.663336+0000","last_change":"2024-09-20T18:53:20.564464+0000","last_active":"2024-09-20T18:53:43.663336+0000","last_peered":"2024-09-20T18:53:43.663336+0000","last_clean":"2024-09-20T18:53:43.663336+0000","last_became_active":"2024-09-20T18:49:45.941448+0000","last_became_peered":"2024-09-20T18:49:45.941448+0000","last_unstale":"2024-09-20T18:53:43.663336+0000","last_undegraded":"2024-09-20T18:53:43.663336+0000","last_fullsized":"2024-09-20T18:53:43.663336+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:20.564413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:20.564413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:38.260460+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":197,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.661913+0000","last_change":"2024-09-20T18:52:36.662703+0000","last_active":"2024-09-20T18:53:43.661913+0000","last_peered":"2024-09-20T18:53:43.661913+0000","last_clean":"2024-09-20T18:53:43.661913+0000","last_became_active":"2024-09-20T18:49:49.957558+0000","last_became_peered":"2024-09-20T18:49:49.957558+0000","last_unstale":"2024-09-20T18:53:43.661913+0000","last_undegraded":"2024-09-20T18:53:43.661913+0000","last_fullsized":"2024-09-20T18:53:43.661913+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:36.662524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:36.662524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:58.211161+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":191,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.662379+0000","last_change":"2024-09-20T18:52:37.819215+0000","last_active":"2024-09-20T18:53:43.662379+0000","last_peered":"2024-09-20T18:53:43.662379+0000","last_clean":"2024-09-20T18:53:43.662379+0000","last_became_active":"2024-09-20T18:49:51.970798+0000","last_became_peered":"2024-09-20T18:49:51.970798+0000","last_unstale":"2024-09-20T18:53:43.662379+0000","last_undegraded":"2024-09-20T18:53:43.662379+0000","last_fullsized":"2024-09-20T18:53:43.662379+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:37.819164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:37.819164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:45.240111+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.a","version":"147'12793","reported_seq":12933,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:46.143398+0000","last_change":"2024-09-20T18:50:49.842445+0000","last_active":"2024-09-20T18:53:46.143398+0000","last_peered":"2024-09-20T18:53:46.143398+0000","last_clean":"2024-09-20T18:53:46.143398+0000","last_became_active":"2024-09-20T18:50:49.842179+0000","last_became_peered":"2024-09-20T18:50:49.842179+0000","last_unstale":"2024-09-20T18:53:46.143398+0000","last_undegraded":"2024-09-20T18:53:46.143398+0000","last_fullsized":"2024-09-20T18:53:46.143398+0000","mapping_epoch":83,"log_start":"124'8100","ondisk_log_start":"124'8100","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.819660+0000","objects_scrubbed":0,"log_size":4693,"log_dups_size":0,"ondisk_log_size":4693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:29.169615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":164823040,"num_objects":2515,"num_object_clones":0,"num_object_copies":7545,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2515,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12793,"num_write_kb":489856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":215,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.655674+0000","last_change":"2024-09-20T18:52:37.881436+0000","last_active":"2024-09-20T18:53:43.655674+0000","last_peered":"2024-09-20T18:53:43.655674+0000","last_clean":"2024-09-20T18:53:43.655674+0000","last_became_active":"2024-09-20T18:49:47.949116+0000","last_became_peered":"2024-09-20T18:49:47.949116+0000","last_unstale":"2024-09-20T18:53:43.655674+0000","last_undegraded":"2024-09-20T18:53:43.655674+0000","last_fullsized":"2024-09-20T18:53:43.655674+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:52:37.881385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:37.881385+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:49.557204+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":203,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.663154+0000","last_change":"2024-09-20T18:53:21.578905+0000","last_active":"2024-09-20T18:53:43.663154+0000","last_peered":"2024-09-20T18:53:43.663154+0000","last_clean":"2024-09-20T18:53:43.663154+0000","last_became_active":"2024-09-20T18:49:45.944858+0000","last_became_peered":"2024-09-20T18:49:45.944858+0000","last_unstale":"2024-09-20T18:53:43.663154+0000","last_undegraded":"2024-09-20T18:53:43.663154+0000","last_fullsized":"2024-09-20T18:53:43.663154+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:21.578845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:21.578845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:47.899680+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":189,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.661915+0000","last_change":"2024-09-20T18:52:12.674502+0000","last_active":"2024-09-20T18:53:43.661915+0000","last_peered":"2024-09-20T18:53:43.661915+0000","last_clean":"2024-09-20T18:53:43.661915+0000","last_became_active":"2024-09-20T18:49:49.954944+0000","last_became_peered":"2024-09-20T18:49:49.954944+0000","last_unstale":"2024-09-20T18:53:43.661915+0000","last_undegraded":"2024-09-20T18:53:43.661915+0000","last_fullsized":"2024-09-20T18:53:43.661915+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:12.674203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:12.674203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:41.868905+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":184,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.675565+0000","last_change":"2024-09-20T18:52:09.017828+0000","last_active":"2024-09-20T18:53:43.675565+0000","last_peered":"2024-09-20T18:53:43.675565+0000","last_clean":"2024-09-20T18:53:43.675565+0000","last_became_active":"2024-09-20T18:49:51.972970+0000","last_became_peered":"2024-09-20T18:49:51.972970+0000","last_unstale":"2024-09-20T18:53:43.675565+0000","last_undegraded":"2024-09-20T18:53:43.675565+0000","last_fullsized":"2024-09-20T18:53:43.675565+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:09.017759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:09.017759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"147'12766","reported_seq":12984,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:45.910459+0000","last_change":"2024-09-20T18:52:15.505711+0000","last_active":"2024-09-20T18:53:45.910459+0000","last_peered":"2024-09-20T18:53:45.910459+0000","last_clean":"2024-09-20T18:53:45.910459+0000","last_became_active":"2024-09-20T18:50:49.843516+0000","last_became_peered":"2024-09-20T18:50:49.843516+0000","last_unstale":"2024-09-20T18:53:45.910459+0000","last_undegraded":"2024-09-20T18:53:45.910459+0000","last_fullsized":"2024-09-20T18:53:45.910459+0000","mapping_epoch":83,"log_start":"123'7700","ondisk_log_start":"123'7700","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'6958","last_scrub_stamp":"2024-09-20T18:52:15.505628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:52:15.505628+0000","objects_scrubbed":6900,"log_size":5066,"log_dups_size":0,"ondisk_log_size":5066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:29.307707+0000","scrub_duration":2196,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163577856,"num_objects":2496,"num_object_clones":0,"num_object_copies":7488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12766,"num_write_kb":488384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":216,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.682442+0000","last_change":"2024-09-20T18:52:35.182422+0000","last_active":"2024-09-20T18:53:43.682442+0000","last_peered":"2024-09-20T18:53:43.682442+0000","last_clean":"2024-09-20T18:53:43.682442+0000","last_became_active":"2024-09-20T18:49:48.499847+0000","last_became_peered":"2024-09-20T18:49:48.499847+0000","last_unstale":"2024-09-20T18:53:43.682442+0000","last_undegraded":"2024-09-20T18:53:43.682442+0000","last_fullsized":"2024-09-20T18:53:43.682442+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:52:35.182330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:35.182330+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:57.232688+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":215,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.662961+0000","last_change":"2024-09-20T18:53:10.784695+0000","last_active":"2024-09-20T18:53:43.662961+0000","last_peered":"2024-09-20T18:53:43.662961+0000","last_clean":"2024-09-20T18:53:43.662961+0000","last_became_active":"2024-09-20T18:49:45.944256+0000","last_became_peered":"2024-09-20T18:49:45.944256+0000","last_unstale":"2024-09-20T18:53:43.662961+0000","last_undegraded":"2024-09-20T18:53:43.662961+0000","last_fullsized":"2024-09-20T18:53:43.662961+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-20T18:53:10.784625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:10.784625+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:16.037732+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":243,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.662100+0000","last_change":"2024-09-20T18:52:05.788988+0000","last_active":"2024-09-20T18:53:43.662100+0000","last_peered":"2024-09-20T18:53:43.662100+0000","last_clean":"2024-09-20T18:53:43.662100+0000","last_became_active":"2024-09-20T18:49:49.958411+0000","last_became_peered":"2024-09-20T18:49:49.958411+0000","last_unstale":"2024-09-20T18:53:43.662100+0000","last_undegraded":"2024-09-20T18:53:43.662100+0000","last_fullsized":"2024-09-20T18:53:43.662100+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:52:05.788924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.788924+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:39.468023+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":183,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.681550+0000","last_change":"2024-09-20T18:51:56.970415+0000","last_active":"2024-09-20T18:53:43.681550+0000","last_peered":"2024-09-20T18:53:43.681550+0000","last_clean":"2024-09-20T18:53:43.681550+0000","last_became_active":"2024-09-20T18:49:52.139685+0000","last_became_peered":"2024-09-20T18:49:52.139685+0000","last_unstale":"2024-09-20T18:53:43.681550+0000","last_undegraded":"2024-09-20T18:53:43.681550+0000","last_fullsized":"2024-09-20T18:53:43.681550+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:56.970349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:56.970349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:05.081675+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":203,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.672468+0000","last_change":"2024-09-20T18:53:09.388116+0000","last_active":"2024-09-20T18:53:43.672468+0000","last_peered":"2024-09-20T18:53:43.672468+0000","last_clean":"2024-09-20T18:53:43.672468+0000","last_became_active":"2024-09-20T18:49:45.940060+0000","last_became_peered":"2024-09-20T18:49:45.940060+0000","last_unstale":"2024-09-20T18:53:43.672468+0000","last_undegraded":"2024-09-20T18:53:43.672468+0000","last_fullsized":"2024-09-20T18:53:43.672468+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:09.388050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:09.388050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:17.378691+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":201,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.676208+0000","last_change":"2024-09-20T18:53:27.186982+0000","last_active":"2024-09-20T18:53:43.676208+0000","last_peered":"2024-09-20T18:53:43.676208+0000","last_clean":"2024-09-20T18:53:43.676208+0000","last_became_active":"2024-09-20T18:49:48.500662+0000","last_became_peered":"2024-09-20T18:49:48.500662+0000","last_unstale":"2024-09-20T18:53:43.676208+0000","last_undegraded":"2024-09-20T18:53:43.676208+0000","last_fullsized":"2024-09-20T18:53:43.676208+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-20T18:53:27.186631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:27.186631+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:42.463558+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":195,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.684338+0000","last_change":"2024-09-20T18:53:16.220554+0000","last_active":"2024-09-20T18:53:43.684338+0000","last_peered":"2024-09-20T18:53:43.684338+0000","last_clean":"2024-09-20T18:53:43.684338+0000","last_became_active":"2024-09-20T18:49:49.956749+0000","last_became_peered":"2024-09-20T18:49:49.956749+0000","last_unstale":"2024-09-20T18:53:43.684338+0000","last_undegraded":"2024-09-20T18:53:43.684338+0000","last_fullsized":"2024-09-20T18:53:43.684338+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:16.220490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:16.220490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:36.759843+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":191,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.663186+0000","last_change":"2024-09-20T18:53:19.591395+0000","last_active":"2024-09-20T18:53:43.663186+0000","last_peered":"2024-09-20T18:53:43.663186+0000","last_clean":"2024-09-20T18:53:43.663186+0000","last_became_active":"2024-09-20T18:49:52.142291+0000","last_became_peered":"2024-09-20T18:49:52.142291+0000","last_unstale":"2024-09-20T18:53:43.663186+0000","last_undegraded":"2024-09-20T18:53:43.663186+0000","last_fullsized":"2024-09-20T18:53:43.663186+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:19.591346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:19.591346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:40.474867+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":207,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.664080+0000","last_change":"2024-09-20T18:51:23.065382+0000","last_active":"2024-09-20T18:53:43.664080+0000","last_peered":"2024-09-20T18:53:43.664080+0000","last_clean":"2024-09-20T18:53:43.664080+0000","last_became_active":"2024-09-20T18:49:48.500803+0000","last_became_peered":"2024-09-20T18:49:48.500803+0000","last_unstale":"2024-09-20T18:53:43.664080+0000","last_undegraded":"2024-09-20T18:53:43.664080+0000","last_fullsized":"2024-09-20T18:53:43.664080+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:51:23.064536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:23.064536+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:39.863201+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":195,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.662218+0000","last_change":"2024-09-20T18:51:30.982535+0000","last_active":"2024-09-20T18:53:43.662218+0000","last_peered":"2024-09-20T18:53:43.662218+0000","last_clean":"2024-09-20T18:53:43.662218+0000","last_became_active":"2024-09-20T18:49:45.946600+0000","last_became_peered":"2024-09-20T18:49:45.946600+0000","last_unstale":"2024-09-20T18:53:43.662218+0000","last_undegraded":"2024-09-20T18:53:43.662218+0000","last_fullsized":"2024-09-20T18:53:43.662218+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:30.982454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:30.982454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:57.922735+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":188,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.676111+0000","last_change":"2024-09-20T18:52:08.010340+0000","last_active":"2024-09-20T18:53:43.676111+0000","last_peered":"2024-09-20T18:53:43.676111+0000","last_clean":"2024-09-20T18:53:43.676111+0000","last_became_active":"2024-09-20T18:49:49.961947+0000","last_became_peered":"2024-09-20T18:49:49.961947+0000","last_unstale":"2024-09-20T18:53:43.676111+0000","last_undegraded":"2024-09-20T18:53:43.676111+0000","last_fullsized":"2024-09-20T18:53:43.676111+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:08.010222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:08.010222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"66'1","reported_seq":194,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.666919+0000","last_change":"2024-09-20T18:53:14.689510+0000","last_active":"2024-09-20T18:53:43.666919+0000","last_peered":"2024-09-20T18:53:43.666919+0000","last_clean":"2024-09-20T18:53:43.666919+0000","last_became_active":"2024-09-20T18:49:51.965121+0000","last_became_peered":"2024-09-20T18:49:51.965121+0000","last_unstale":"2024-09-20T18:53:43.666919+0000","last_undegraded":"2024-09-20T18:53:43.666919+0000","last_fullsized":"2024-09-20T18:53:43.666919+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T18:53:14.689422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:14.689422+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:19.716830+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":199,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.682628+0000","last_change":"2024-09-20T18:49:48.501359+0000","last_active":"2024-09-20T18:53:43.682628+0000","last_peered":"2024-09-20T18:53:43.682628+0000","last_clean":"2024-09-20T18:53:43.682628+0000","last_became_active":"2024-09-20T18:49:48.501011+0000","last_became_peered":"2024-09-20T18:49:48.501011+0000","last_unstale":"2024-09-20T18:53:43.682628+0000","last_undegraded":"2024-09-20T18:53:43.682628+0000","last_fullsized":"2024-09-20T18:53:43.682628+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:02.117814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":195,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.669176+0000","last_change":"2024-09-20T18:51:20.067303+0000","last_active":"2024-09-20T18:53:43.669176+0000","last_peered":"2024-09-20T18:53:43.669176+0000","last_clean":"2024-09-20T18:53:43.669176+0000","last_became_active":"2024-09-20T18:49:45.938511+0000","last_became_peered":"2024-09-20T18:49:45.938511+0000","last_unstale":"2024-09-20T18:53:43.669176+0000","last_undegraded":"2024-09-20T18:53:43.669176+0000","last_fullsized":"2024-09-20T18:53:43.669176+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:20.067233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:20.067233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:55.959831+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":245,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.675569+0000","last_change":"2024-09-20T18:52:14.049155+0000","last_active":"2024-09-20T18:53:43.675569+0000","last_peered":"2024-09-20T18:53:43.675569+0000","last_clean":"2024-09-20T18:53:43.675569+0000","last_became_active":"2024-09-20T18:49:49.960550+0000","last_became_peered":"2024-09-20T18:49:49.960550+0000","last_unstale":"2024-09-20T18:53:43.675569+0000","last_undegraded":"2024-09-20T18:53:43.675569+0000","last_fullsized":"2024-09-20T18:53:43.675569+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:52:14.048921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:14.048921+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":184,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.656775+0000","last_change":"2024-09-20T18:52:41.825306+0000","last_active":"2024-09-20T18:53:43.656775+0000","last_peered":"2024-09-20T18:53:43.656775+0000","last_clean":"2024-09-20T18:53:43.656775+0000","last_became_active":"2024-09-20T18:49:51.966254+0000","last_became_peered":"2024-09-20T18:49:51.966254+0000","last_unstale":"2024-09-20T18:53:43.656775+0000","last_undegraded":"2024-09-20T18:53:43.656775+0000","last_fullsized":"2024-09-20T18:53:43.656775+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:41.825248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:41.825248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:47.685753+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":226,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.675927+0000","last_change":"2024-09-20T18:51:15.223913+0000","last_active":"2024-09-20T18:53:43.675927+0000","last_peered":"2024-09-20T18:53:43.675927+0000","last_clean":"2024-09-20T18:53:43.675927+0000","last_became_active":"2024-09-20T18:49:58.997494+0000","last_became_peered":"2024-09-20T18:49:58.997494+0000","last_unstale":"2024-09-20T18:53:43.675927+0000","last_undegraded":"2024-09-20T18:53:43.675927+0000","last_fullsized":"2024-09-20T18:53:43.675927+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:51:15.223826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:15.223826+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:06.150447+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":203,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.662641+0000","last_change":"2024-09-20T18:53:27.589481+0000","last_active":"2024-09-20T18:53:43.662641+0000","last_peered":"2024-09-20T18:53:43.662641+0000","last_clean":"2024-09-20T18:53:43.662641+0000","last_became_active":"2024-09-20T18:49:45.949492+0000","last_became_peered":"2024-09-20T18:49:45.949492+0000","last_unstale":"2024-09-20T18:53:43.662641+0000","last_undegraded":"2024-09-20T18:53:43.662641+0000","last_fullsized":"2024-09-20T18:53:43.662641+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:27.589413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:27.589413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:48.562654+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":243,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.681942+0000","last_change":"2024-09-20T18:52:39.168501+0000","last_active":"2024-09-20T18:53:43.681942+0000","last_peered":"2024-09-20T18:53:43.681942+0000","last_clean":"2024-09-20T18:53:43.681942+0000","last_became_active":"2024-09-20T18:49:49.962243+0000","last_became_peered":"2024-09-20T18:49:49.962243+0000","last_unstale":"2024-09-20T18:53:43.681942+0000","last_undegraded":"2024-09-20T18:53:43.681942+0000","last_fullsized":"2024-09-20T18:53:43.681942+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:52:39.168442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:39.168442+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:02.311754+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":191,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.663076+0000","last_change":"2024-09-20T18:53:13.863129+0000","last_active":"2024-09-20T18:53:43.663076+0000","last_peered":"2024-09-20T18:53:43.663076+0000","last_clean":"2024-09-20T18:53:43.663076+0000","last_became_active":"2024-09-20T18:49:51.969093+0000","last_became_peered":"2024-09-20T18:49:51.969093+0000","last_unstale":"2024-09-20T18:53:43.663076+0000","last_undegraded":"2024-09-20T18:53:43.663076+0000","last_fullsized":"2024-09-20T18:53:43.663076+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:13.863061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:13.863061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:37.505050+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":219,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.879001+0000","last_change":"2024-09-20T18:53:43.879001+0000","last_active":"2024-09-20T18:53:43.879001+0000","last_peered":"2024-09-20T18:53:43.879001+0000","last_clean":"2024-09-20T18:53:43.879001+0000","last_became_active":"2024-09-20T18:49:47.949703+0000","last_became_peered":"2024-09-20T18:49:47.949703+0000","last_unstale":"2024-09-20T18:53:43.879001+0000","last_undegraded":"2024-09-20T18:53:43.879001+0000","last_fullsized":"2024-09-20T18:53:43.879001+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:53:43.878921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:43.878921+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:02.026666+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":196,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.657061+0000","last_change":"2024-09-20T18:52:05.985622+0000","last_active":"2024-09-20T18:53:43.657061+0000","last_peered":"2024-09-20T18:53:43.657061+0000","last_clean":"2024-09-20T18:53:43.657061+0000","last_became_active":"2024-09-20T18:49:45.939949+0000","last_became_peered":"2024-09-20T18:49:45.939949+0000","last_unstale":"2024-09-20T18:53:43.657061+0000","last_undegraded":"2024-09-20T18:53:43.657061+0000","last_fullsized":"2024-09-20T18:53:43.657061+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:05.985551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.985551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:36.823731+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":188,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.668182+0000","last_change":"2024-09-20T18:51:14.271954+0000","last_active":"2024-09-20T18:53:43.668182+0000","last_peered":"2024-09-20T18:53:43.668182+0000","last_clean":"2024-09-20T18:53:43.668182+0000","last_became_active":"2024-09-20T18:49:49.957067+0000","last_became_peered":"2024-09-20T18:49:49.957067+0000","last_unstale":"2024-09-20T18:53:43.668182+0000","last_undegraded":"2024-09-20T18:53:43.668182+0000","last_fullsized":"2024-09-20T18:53:43.668182+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:14.271719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.271719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:31.305762+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":184,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.675162+0000","last_change":"2024-09-20T18:51:14.210438+0000","last_active":"2024-09-20T18:53:43.675162+0000","last_peered":"2024-09-20T18:53:43.675162+0000","last_clean":"2024-09-20T18:53:43.675162+0000","last_became_active":"2024-09-20T18:49:51.966915+0000","last_became_peered":"2024-09-20T18:49:51.966915+0000","last_unstale":"2024-09-20T18:53:43.675162+0000","last_undegraded":"2024-09-20T18:53:43.675162+0000","last_fullsized":"2024-09-20T18:53:43.675162+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:14.210358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.210358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:04.079575+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":216,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.668406+0000","last_change":"2024-09-20T18:52:38.240463+0000","last_active":"2024-09-20T18:53:43.668406+0000","last_peered":"2024-09-20T18:53:43.668406+0000","last_clean":"2024-09-20T18:53:43.668406+0000","last_became_active":"2024-09-20T18:49:47.953310+0000","last_became_peered":"2024-09-20T18:49:47.953310+0000","last_unstale":"2024-09-20T18:53:43.668406+0000","last_undegraded":"2024-09-20T18:53:43.668406+0000","last_fullsized":"2024-09-20T18:53:43.668406+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:52:38.240386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:38.240386+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:52.778987+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":195,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.663274+0000","last_change":"2024-09-20T18:52:34.834333+0000","last_active":"2024-09-20T18:53:43.663274+0000","last_peered":"2024-09-20T18:53:43.663274+0000","last_clean":"2024-09-20T18:53:43.663274+0000","last_became_active":"2024-09-20T18:49:45.944357+0000","last_became_peered":"2024-09-20T18:49:45.944357+0000","last_unstale":"2024-09-20T18:53:43.663274+0000","last_undegraded":"2024-09-20T18:53:43.663274+0000","last_fullsized":"2024-09-20T18:53:43.663274+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:34.834273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:34.834273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:42.456584+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":188,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.675570+0000","last_change":"2024-09-20T18:51:20.208925+0000","last_active":"2024-09-20T18:53:43.675570+0000","last_peered":"2024-09-20T18:53:43.675570+0000","last_clean":"2024-09-20T18:53:43.675570+0000","last_became_active":"2024-09-20T18:49:49.962523+0000","last_became_peered":"2024-09-20T18:49:49.962523+0000","last_unstale":"2024-09-20T18:53:43.675570+0000","last_undegraded":"2024-09-20T18:53:43.675570+0000","last_fullsized":"2024-09-20T18:53:43.675570+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:20.208786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:20.208786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:08.165781+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":191,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.670016+0000","last_change":"2024-09-20T18:52:38.137485+0000","last_active":"2024-09-20T18:53:43.670016+0000","last_peered":"2024-09-20T18:53:43.670016+0000","last_clean":"2024-09-20T18:53:43.670016+0000","last_became_active":"2024-09-20T18:49:51.967313+0000","last_became_peered":"2024-09-20T18:49:51.967313+0000","last_unstale":"2024-09-20T18:53:43.670016+0000","last_undegraded":"2024-09-20T18:53:43.670016+0000","last_fullsized":"2024-09-20T18:53:43.670016+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:38.137431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:38.137431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:07.091791+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":211,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.675241+0000","last_change":"2024-09-20T18:52:13.020995+0000","last_active":"2024-09-20T18:53:43.675241+0000","last_peered":"2024-09-20T18:53:43.675241+0000","last_clean":"2024-09-20T18:53:43.675241+0000","last_became_active":"2024-09-20T18:49:48.499111+0000","last_became_peered":"2024-09-20T18:49:48.499111+0000","last_unstale":"2024-09-20T18:53:43.675241+0000","last_undegraded":"2024-09-20T18:53:43.675241+0000","last_fullsized":"2024-09-20T18:53:43.675241+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:52:13.020920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:13.020920+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":195,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.684459+0000","last_change":"2024-09-20T18:52:10.090860+0000","last_active":"2024-09-20T18:53:43.684459+0000","last_peered":"2024-09-20T18:53:43.684459+0000","last_clean":"2024-09-20T18:53:43.684459+0000","last_became_active":"2024-09-20T18:49:45.942105+0000","last_became_peered":"2024-09-20T18:49:45.942105+0000","last_unstale":"2024-09-20T18:53:43.684459+0000","last_undegraded":"2024-09-20T18:53:43.684459+0000","last_fullsized":"2024-09-20T18:53:43.684459+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:10.090802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:10.090802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":195,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.661886+0000","last_change":"2024-09-20T18:52:41.836356+0000","last_active":"2024-09-20T18:53:43.661886+0000","last_peered":"2024-09-20T18:53:43.661886+0000","last_clean":"2024-09-20T18:53:43.661886+0000","last_became_active":"2024-09-20T18:49:49.958159+0000","last_became_peered":"2024-09-20T18:49:49.958159+0000","last_unstale":"2024-09-20T18:53:43.661886+0000","last_undegraded":"2024-09-20T18:53:43.661886+0000","last_fullsized":"2024-09-20T18:53:43.661886+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:41.836294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:41.836294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:44.305979+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":184,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.675671+0000","last_change":"2024-09-20T18:52:43.118886+0000","last_active":"2024-09-20T18:53:43.675671+0000","last_peered":"2024-09-20T18:53:43.675671+0000","last_clean":"2024-09-20T18:53:43.675671+0000","last_became_active":"2024-09-20T18:49:52.139978+0000","last_became_peered":"2024-09-20T18:49:52.139978+0000","last_unstale":"2024-09-20T18:53:43.675671+0000","last_undegraded":"2024-09-20T18:53:43.675671+0000","last_fullsized":"2024-09-20T18:53:43.675671+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:43.118805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:43.118805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:11.215766+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":206,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.675056+0000","last_change":"2024-09-20T18:53:43.253204+0000","last_active":"2024-09-20T18:53:43.675056+0000","last_peered":"2024-09-20T18:53:43.675056+0000","last_clean":"2024-09-20T18:53:43.675056+0000","last_became_active":"2024-09-20T18:49:47.954777+0000","last_became_peered":"2024-09-20T18:49:47.954777+0000","last_unstale":"2024-09-20T18:53:43.675056+0000","last_undegraded":"2024-09-20T18:53:43.675056+0000","last_fullsized":"2024-09-20T18:53:43.675056+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-20T18:53:43.253132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:43.253132+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:00.654449+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":203,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.673978+0000","last_change":"2024-09-20T18:53:16.921320+0000","last_active":"2024-09-20T18:53:43.673978+0000","last_peered":"2024-09-20T18:53:43.673978+0000","last_clean":"2024-09-20T18:53:43.673978+0000","last_became_active":"2024-09-20T18:49:45.940988+0000","last_became_peered":"2024-09-20T18:49:45.940988+0000","last_unstale":"2024-09-20T18:53:43.673978+0000","last_undegraded":"2024-09-20T18:53:43.673978+0000","last_fullsized":"2024-09-20T18:53:43.673978+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:16.921263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:16.921263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:35.775476+0000","scrub_duration":137,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":195,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.672468+0000","last_change":"2024-09-20T18:52:44.344764+0000","last_active":"2024-09-20T18:53:43.672468+0000","last_peered":"2024-09-20T18:53:43.672468+0000","last_clean":"2024-09-20T18:53:43.672468+0000","last_became_active":"2024-09-20T18:49:49.954502+0000","last_became_peered":"2024-09-20T18:49:49.954502+0000","last_unstale":"2024-09-20T18:53:43.672468+0000","last_undegraded":"2024-09-20T18:53:43.672468+0000","last_fullsized":"2024-09-20T18:53:43.672468+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:44.344700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:44.344700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:52.432651+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":183,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.662108+0000","last_change":"2024-09-20T18:51:09.761331+0000","last_active":"2024-09-20T18:53:43.662108+0000","last_peered":"2024-09-20T18:53:43.662108+0000","last_clean":"2024-09-20T18:53:43.662108+0000","last_became_active":"2024-09-20T18:49:51.968662+0000","last_became_peered":"2024-09-20T18:49:51.968662+0000","last_unstale":"2024-09-20T18:53:43.662108+0000","last_undegraded":"2024-09-20T18:53:43.662108+0000","last_fullsized":"2024-09-20T18:53:43.662108+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:09.761283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:09.761283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:30.451288+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":176,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.670837+0000","last_change":"2024-09-20T18:49:51.968476+0000","last_active":"2024-09-20T18:53:43.670837+0000","last_peered":"2024-09-20T18:53:43.670837+0000","last_clean":"2024-09-20T18:53:43.670837+0000","last_became_active":"2024-09-20T18:49:51.968216+0000","last_became_peered":"2024-09-20T18:49:51.968216+0000","last_unstale":"2024-09-20T18:53:43.670837+0000","last_undegraded":"2024-09-20T18:53:43.670837+0000","last_fullsized":"2024-09-20T18:53:43.670837+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:58.058065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":196,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.675282+0000","last_change":"2024-09-20T18:52:41.106558+0000","last_active":"2024-09-20T18:53:43.675282+0000","last_peered":"2024-09-20T18:53:43.675282+0000","last_clean":"2024-09-20T18:53:43.675282+0000","last_became_active":"2024-09-20T18:49:45.937992+0000","last_became_peered":"2024-09-20T18:49:45.937992+0000","last_unstale":"2024-09-20T18:53:43.675282+0000","last_undegraded":"2024-09-20T18:53:43.675282+0000","last_fullsized":"2024-09-20T18:53:43.675282+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:41.106332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:41.106332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:50.275314+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":210,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.673473+0000","last_change":"2024-09-20T18:51:14.545122+0000","last_active":"2024-09-20T18:53:43.673473+0000","last_peered":"2024-09-20T18:53:43.673473+0000","last_clean":"2024-09-20T18:53:43.673473+0000","last_became_active":"2024-09-20T18:49:47.949582+0000","last_became_peered":"2024-09-20T18:49:47.949582+0000","last_unstale":"2024-09-20T18:53:43.673473+0000","last_undegraded":"2024-09-20T18:53:43.673473+0000","last_fullsized":"2024-09-20T18:53:43.673473+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:51:14.545043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.545043+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:40.491358+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":187,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.663308+0000","last_change":"2024-09-20T18:51:28.120100+0000","last_active":"2024-09-20T18:53:43.663308+0000","last_peered":"2024-09-20T18:53:43.663308+0000","last_clean":"2024-09-20T18:53:43.663308+0000","last_became_active":"2024-09-20T18:49:49.957582+0000","last_became_peered":"2024-09-20T18:49:49.957582+0000","last_unstale":"2024-09-20T18:53:43.663308+0000","last_undegraded":"2024-09-20T18:53:43.663308+0000","last_fullsized":"2024-09-20T18:53:43.663308+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:28.119694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:28.119694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:32.813511+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":2652241944,"num_objects":40471,"num_object_clones":0,"num_object_copies":121413,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40471,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":205129,"num_write_kb":7859137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":7960670208,"data_stored":7960657992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":90323,"ondisk_log_size":90323,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":235,"num_read_kb":230,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266740,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":940548,"kb_used_data":828852,"kb_used_omap":1,"kb_used_meta":111678,"kb_avail":92804604,"statfs":{"total":95995035648,"available":95031914496,"internally_reserved":0,"allocated":848744448,"data_stored":848332622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":114358730},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528379,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":913908,"kb_used_data":810532,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":92831244,"statfs":{"total":95995035648,"available":95059193856,"internally_reserved":0,"allocated":829984768,"data_stored":829588688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":105839046},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757312,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":728880,"kb_used_data":657440,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":93016272,"statfs":{"total":95995035648,"available":95248662528,"internally_reserved":0,"allocated":673218560,"data_stored":672826367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986245,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":942800,"kb_used_data":839424,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":92802352,"statfs":{"total":95995035648,"available":95029608448,"internally_reserved":0,"allocated":859570176,"data_stored":859146467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105839045},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1635220,"kb_used_data":1477628,"kb_used_omap":1,"kb_used_meta":157502,"kb_avail":92109932,"statfs":{"total":95995035648,"available":94320570368,"internally_reserved":0,"allocated":1513091072,"data_stored":1512670303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":161282503},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1095540,"kb_used_data":983844,"kb_used_omap":1,"kb_used_meta":111678,"kb_avail":92649612,"statfs":{"total":95995035648,"available":94873202688,"internally_reserved":0,"allocated":1007456256,"data_stored":1007060546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":114358725},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1095400,"kb_used_data":983640,"kb_used_omap":1,"kb_used_meta":111678,"kb_avail":92649752,"statfs":{"total":95995035648,"available":94873346048,"internally_reserved":0,"allocated":1007247360,"data_stored":1006799610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":114358725},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1438348,"kb_used_data":1298872,"kb_used_omap":1,"kb_used_meta":139454,"kb_avail":92306804,"statfs":{"total":95995035648,"available":94522167296,"internally_reserved":0,"allocated":1330044928,"data_stored":1329628726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142801349},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1317670912,"data_stored":1317666840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":991563776,"data_stored":991559704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1002307584,"data_stored":1002307584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1483276288,"data_stored":1483276288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":840302592,"data_stored":840302592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":665911296,"data_stored":665911296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":822480896,"data_stored":822476824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":837156864,"data_stored":837156864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T18:53:48.364 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T18:53:48.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:48.365 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:48.365 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:48.466 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:48.511 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T18:53:48.511 INFO:teuthology.orchestra.run.smithi142.stdout: "size": 20, 2024-09-20T18:53:48.511 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 600, 2024-09-20T18:53:48.512 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-09-20T18:53:48.512 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.512 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:24945 7.7 7:e97e81cf:::benchmark_data_smithi083_70643_object24944:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:53:48.512 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:06.996434+0000", 2024-09-20T18:53:48.512 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 161.49546973899999, 2024-09-20T18:53:48.512 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.117472831, 2024-09-20T18:53:48.512 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:53:48.513 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:53:48.513 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:48.513 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:53:48.513 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:53:48.513 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:48.513 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 24945 2024-09-20T18:53:48.513 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.514 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:53:48.514 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.514 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:53:48.514 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.996434+0000", 2024-09-20T18:53:48.514 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:48.514 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.515 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.515 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:53:48.515 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.996434+0000", 2024-09-20T18:53:48.515 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:48.515 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.515 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.515 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:53:48.516 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.996435+0000", 2024-09-20T18:53:48.516 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4869999999999999e-06 2024-09-20T18:53:48.516 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.516 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.516 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:53:48.516 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.996477+0000", 2024-09-20T18:53:48.517 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.1601999999999999e-05 2024-09-20T18:53:48.517 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.517 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.517 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:53:48.517 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.996488+0000", 2024-09-20T18:53:48.517 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0893999999999999e-05 2024-09-20T18:53:48.517 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.518 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.518 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:53:48.518 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.996495+0000", 2024-09-20T18:53:48.518 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.1720000000000001e-06 2024-09-20T18:53:48.518 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.518 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.519 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:53:48.519 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.996709+0000", 2024-09-20T18:53:48.519 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00021389099999999999 2024-09-20T18:53:48.519 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.519 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.519 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:53:48.519 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.996918+0000", 2024-09-20T18:53:48.520 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00020888100000000001 2024-09-20T18:53:48.520 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.520 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.520 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,3", 2024-09-20T18:53:48.520 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.997122+0000", 2024-09-20T18:53:48.520 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00020468 2024-09-20T18:53:48.520 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.521 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.521 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:53:48.521 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.004028+0000", 2024-09-20T18:53:48.521 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0069058180000000002 2024-09-20T18:53:48.521 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.521 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.522 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:48.522 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.004092+0000", 2024-09-20T18:53:48.522 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4323000000000001e-05 2024-09-20T18:53:48.522 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.522 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.522 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:48.522 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.113828+0000", 2024-09-20T18:53:48.523 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.109735151 2024-09-20T18:53:48.523 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.523 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.523 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:53:48.523 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.113870+0000", 2024-09-20T18:53:48.523 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.2258000000000001e-05 2024-09-20T18:53:48.524 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.524 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.524 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:53:48.524 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.113906+0000", 2024-09-20T18:53:48.524 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6674000000000002e-05 2024-09-20T18:53:48.524 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:48.525 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:53:48.525 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:48.525 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.525 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.525 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:24951 7.7 7:ebe82687:::benchmark_data_smithi083_70643_object24950:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:53:48.525 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:06.997097+0000", 2024-09-20T18:53:48.525 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 161.494806772, 2024-09-20T18:53:48.526 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.12447738799999999, 2024-09-20T18:53:48.526 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:53:48.526 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:53:48.526 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:48.526 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:53:48.527 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:53:48.527 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:48.527 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 24951 2024-09-20T18:53:48.527 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.527 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:53:48.527 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.528 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:53:48.528 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.997097+0000", 2024-09-20T18:53:48.528 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:48.528 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.528 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.528 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:53:48.528 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.997097+0000", 2024-09-20T18:53:48.529 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:48.529 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.529 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.529 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:53:48.529 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.997098+0000", 2024-09-20T18:53:48.529 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.082e-06 2024-09-20T18:53:48.529 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.530 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.530 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:53:48.530 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.997210+0000", 2024-09-20T18:53:48.530 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011240500000000001 2024-09-20T18:53:48.530 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.530 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.531 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:53:48.531 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.997223+0000", 2024-09-20T18:53:48.531 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2609000000000001e-05 2024-09-20T18:53:48.531 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.531 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.531 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:53:48.532 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.997230+0000", 2024-09-20T18:53:48.532 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5700000000000004e-06 2024-09-20T18:53:48.532 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.532 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.532 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:53:48.532 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.000085+0000", 2024-09-20T18:53:48.533 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.002855013 2024-09-20T18:53:48.533 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.533 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.533 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:53:48.533 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.000200+0000", 2024-09-20T18:53:48.533 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011476600000000001 2024-09-20T18:53:48.534 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.534 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.534 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,3", 2024-09-20T18:53:48.534 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.001210+0000", 2024-09-20T18:53:48.534 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0010098850000000001 2024-09-20T18:53:48.534 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.535 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.535 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:53:48.535 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.005107+0000", 2024-09-20T18:53:48.535 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0038972730000000001 2024-09-20T18:53:48.535 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.535 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.535 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:48.536 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.103702+0000", 2024-09-20T18:53:48.536 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.098595123000000007 2024-09-20T18:53:48.536 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.536 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.536 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:48.536 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.121471+0000", 2024-09-20T18:53:48.537 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.017768227000000001 2024-09-20T18:53:48.537 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.537 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.537 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:53:48.537 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.121533+0000", 2024-09-20T18:53:48.537 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2489000000000004e-05 2024-09-20T18:53:48.537 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.538 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.538 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:53:48.538 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.121574+0000", 2024-09-20T18:53:48.538 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0945999999999997e-05 2024-09-20T18:53:48.538 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:48.538 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:53:48.539 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:48.539 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.539 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.539 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:24956 7.7 7:e91ba513:::benchmark_data_smithi083_70643_object24955:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:53:48.539 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:06.999473+0000", 2024-09-20T18:53:48.539 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 161.49243066099999, 2024-09-20T18:53:48.539 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.122248832, 2024-09-20T18:53:48.540 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:53:48.540 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:53:48.540 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:48.540 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:53:48.540 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:53:48.540 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:48.541 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 24956 2024-09-20T18:53:48.541 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.541 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:53:48.541 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.541 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:53:48.541 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.999473+0000", 2024-09-20T18:53:48.542 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:48.542 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.542 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.542 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:53:48.542 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.999473+0000", 2024-09-20T18:53:48.542 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:48.542 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.542 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.543 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:53:48.543 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.999475+0000", 2024-09-20T18:53:48.543 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9750000000000001e-06 2024-09-20T18:53:48.543 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.543 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.543 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:53:48.543 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.999545+0000", 2024-09-20T18:53:48.543 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.0104000000000005e-05 2024-09-20T18:53:48.543 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.543 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.543 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:53:48.544 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.999580+0000", 2024-09-20T18:53:48.544 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.5284000000000003e-05 2024-09-20T18:53:48.544 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.544 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.544 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:53:48.544 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.999590+0000", 2024-09-20T18:53:48.544 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.6739999999999992e-06 2024-09-20T18:53:48.544 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.544 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.544 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:53:48.544 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002183+0000", 2024-09-20T18:53:48.544 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0025934899999999999 2024-09-20T18:53:48.544 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.544 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.545 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:53:48.545 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002933+0000", 2024-09-20T18:53:48.545 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00074923700000000004 2024-09-20T18:53:48.545 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.545 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.545 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,3", 2024-09-20T18:53:48.545 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.003100+0000", 2024-09-20T18:53:48.545 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000167651 2024-09-20T18:53:48.545 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.545 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.545 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:53:48.545 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.005113+0000", 2024-09-20T18:53:48.545 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0020131749999999999 2024-09-20T18:53:48.545 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.546 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.546 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:48.546 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.104767+0000", 2024-09-20T18:53:48.546 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.099653505000000003 2024-09-20T18:53:48.546 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.546 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.546 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:48.546 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.121622+0000", 2024-09-20T18:53:48.546 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.016855098999999998 2024-09-20T18:53:48.546 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.546 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.546 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:53:48.546 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.121681+0000", 2024-09-20T18:53:48.547 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9199e-05 2024-09-20T18:53:48.547 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.547 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.547 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:53:48.547 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.121722+0000", 2024-09-20T18:53:48.547 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0438999999999997e-05 2024-09-20T18:53:48.547 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:48.547 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:53:48.547 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:48.547 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.547 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.547 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:24960 7.c 7:3cbb268e:::benchmark_data_smithi083_70643_object24959:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:53:48.547 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:07.001793+0000", 2024-09-20T18:53:48.548 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 161.490109913, 2024-09-20T18:53:48.548 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.117162266, 2024-09-20T18:53:48.548 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:53:48.548 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:53:48.548 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:48.548 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:53:48.548 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:53:48.548 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:48.548 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 24960 2024-09-20T18:53:48.548 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.548 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:53:48.548 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.548 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:53:48.548 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.001793+0000", 2024-09-20T18:53:48.548 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:48.549 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.549 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.549 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:53:48.549 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.001793+0000", 2024-09-20T18:53:48.549 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:48.549 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.549 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.549 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:53:48.549 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.001795+0000", 2024-09-20T18:53:48.549 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3069999999999999e-06 2024-09-20T18:53:48.549 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.549 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.549 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:53:48.549 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.001853+0000", 2024-09-20T18:53:48.550 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8372999999999998e-05 2024-09-20T18:53:48.550 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.550 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.550 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:53:48.550 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.001865+0000", 2024-09-20T18:53:48.550 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2004999999999999e-05 2024-09-20T18:53:48.550 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.550 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.550 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:53:48.550 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.001873+0000", 2024-09-20T18:53:48.550 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.362e-06 2024-09-20T18:53:48.550 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.550 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.550 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:53:48.551 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.001933+0000", 2024-09-20T18:53:48.551 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9998999999999999e-05 2024-09-20T18:53:48.551 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.551 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.551 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:53:48.551 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002069+0000", 2024-09-20T18:53:48.551 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013620799999999999 2024-09-20T18:53:48.551 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.551 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.551 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:53:48.551 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002203+0000", 2024-09-20T18:53:48.551 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000134048 2024-09-20T18:53:48.551 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.551 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.551 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:48.552 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.004679+0000", 2024-09-20T18:53:48.552 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0024765479999999999 2024-09-20T18:53:48.552 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.552 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.552 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:53:48.552 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.005474+0000", 2024-09-20T18:53:48.552 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00079475399999999995 2024-09-20T18:53:48.552 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.552 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.552 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:48.552 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.118886+0000", 2024-09-20T18:53:48.552 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.11341159200000001 2024-09-20T18:53:48.552 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.552 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.552 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:53:48.553 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.118928+0000", 2024-09-20T18:53:48.553 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.2460999999999999e-05 2024-09-20T18:53:48.553 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.553 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.553 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:53:48.553 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.118956+0000", 2024-09-20T18:53:48.553 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.7608999999999999e-05 2024-09-20T18:53:48.553 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:48.553 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:53:48.553 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:48.553 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.553 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.553 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:24961 7.c 7:35ca9a87:::benchmark_data_smithi083_70643_object24960:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:53:48.553 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:07.001951+0000", 2024-09-20T18:53:48.554 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 161.48995270399999, 2024-09-20T18:53:48.554 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.11716043299999999, 2024-09-20T18:53:48.554 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:53:48.554 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:53:48.554 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:48.554 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:53:48.554 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:53:48.554 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:48.554 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 24961 2024-09-20T18:53:48.554 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.554 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:53:48.554 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.554 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:53:48.555 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.001951+0000", 2024-09-20T18:53:48.555 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:48.555 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.555 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.555 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:53:48.555 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.001951+0000", 2024-09-20T18:53:48.555 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:48.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:48.557 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:48.557 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.557 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.557 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:53:48.558 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.001951+0000", 2024-09-20T18:53:48.558 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.7499999999999999e-07 2024-09-20T18:53:48.558 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.558 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.558 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:53:48.558 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002093+0000", 2024-09-20T18:53:48.558 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000141433 2024-09-20T18:53:48.558 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.558 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.558 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:53:48.558 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002103+0000", 2024-09-20T18:53:48.559 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.9979999999999998e-06 2024-09-20T18:53:48.559 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.559 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.559 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:53:48.559 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002110+0000", 2024-09-20T18:53:48.559 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.6170000000000003e-06 2024-09-20T18:53:48.559 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.559 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.559 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:53:48.560 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.003810+0000", 2024-09-20T18:53:48.560 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001700221 2024-09-20T18:53:48.560 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.560 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.560 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:53:48.560 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.003895+0000", 2024-09-20T18:53:48.560 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.4886e-05 2024-09-20T18:53:48.560 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.561 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.561 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:53:48.561 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.003978+0000", 2024-09-20T18:53:48.561 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.3812000000000003e-05 2024-09-20T18:53:48.561 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.561 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.561 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:53:48.561 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.005480+0000", 2024-09-20T18:53:48.561 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001501245 2024-09-20T18:53:48.561 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.561 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.562 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:48.562 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.005508+0000", 2024-09-20T18:53:48.562 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.8668000000000001e-05 2024-09-20T18:53:48.562 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.562 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.562 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:48.562 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.119009+0000", 2024-09-20T18:53:48.562 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.113500961 2024-09-20T18:53:48.562 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.562 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.562 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:53:48.563 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.119073+0000", 2024-09-20T18:53:48.563 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.3378000000000002e-05 2024-09-20T18:53:48.563 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.563 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.563 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:53:48.563 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.119111+0000", 2024-09-20T18:53:48.563 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.8439000000000002e-05 2024-09-20T18:53:48.563 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:48.563 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:53:48.563 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:48.563 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.563 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.563 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:24962 7.7 7:efd119ca:::benchmark_data_smithi083_70643_object24961:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:53:48.563 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:07.002480+0000", 2024-09-20T18:53:48.564 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 161.48942301299999, 2024-09-20T18:53:48.564 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.11963515, 2024-09-20T18:53:48.564 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:53:48.564 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:53:48.564 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:48.564 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:53:48.564 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:53:48.564 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:48.564 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 24962 2024-09-20T18:53:48.564 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.564 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:53:48.564 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.564 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:53:48.564 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002480+0000", 2024-09-20T18:53:48.564 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:48.565 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.565 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.565 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:53:48.565 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002480+0000", 2024-09-20T18:53:48.565 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:48.565 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.565 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.565 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:53:48.565 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002481+0000", 2024-09-20T18:53:48.565 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0860000000000001e-06 2024-09-20T18:53:48.565 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.565 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.565 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:53:48.565 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002511+0000", 2024-09-20T18:53:48.566 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9499999999999999e-05 2024-09-20T18:53:48.566 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.566 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.566 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:53:48.566 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002522+0000", 2024-09-20T18:53:48.566 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0652000000000001e-05 2024-09-20T18:53:48.566 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.566 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.566 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:53:48.566 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002529+0000", 2024-09-20T18:53:48.566 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.1319999999999996e-06 2024-09-20T18:53:48.566 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.566 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.566 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:53:48.567 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.004125+0000", 2024-09-20T18:53:48.567 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0015959570000000001 2024-09-20T18:53:48.567 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.567 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.567 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:53:48.567 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.004232+0000", 2024-09-20T18:53:48.567 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000107767 2024-09-20T18:53:48.567 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.567 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.567 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,3", 2024-09-20T18:53:48.567 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.004366+0000", 2024-09-20T18:53:48.567 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013360299999999999 2024-09-20T18:53:48.567 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.567 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.568 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:53:48.568 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.005279+0000", 2024-09-20T18:53:48.568 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00091312100000000003 2024-09-20T18:53:48.568 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.568 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.568 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:48.568 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.104849+0000", 2024-09-20T18:53:48.568 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.099569569999999996 2024-09-20T18:53:48.568 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.568 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.568 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:48.568 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.122006+0000", 2024-09-20T18:53:48.568 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.017157531 2024-09-20T18:53:48.568 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.569 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.569 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:53:48.569 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.122071+0000", 2024-09-20T18:53:48.569 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4372e-05 2024-09-20T18:53:48.569 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.569 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.569 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:53:48.569 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.122116+0000", 2024-09-20T18:53:48.569 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.4858999999999999e-05 2024-09-20T18:53:48.569 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:48.569 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:53:48.569 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:48.569 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.569 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.569 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:24963 7.c 7:38caa154:::benchmark_data_smithi083_70643_object24962:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:53:48.570 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:07.002541+0000", 2024-09-20T18:53:48.570 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 161.48936253400001, 2024-09-20T18:53:48.570 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.11664609500000001, 2024-09-20T18:53:48.570 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:53:48.570 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:53:48.570 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:48.570 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:53:48.570 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:53:48.570 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:48.570 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 24963 2024-09-20T18:53:48.570 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.570 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:53:48.570 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.570 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:53:48.571 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002541+0000", 2024-09-20T18:53:48.571 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:48.571 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.571 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.571 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:53:48.571 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002541+0000", 2024-09-20T18:53:48.571 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:48.571 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.571 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.571 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:53:48.571 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002541+0000", 2024-09-20T18:53:48.571 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.2900000000000004e-07 2024-09-20T18:53:48.571 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.571 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.572 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:53:48.572 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002569+0000", 2024-09-20T18:53:48.572 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.7866999999999999e-05 2024-09-20T18:53:48.572 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.572 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.572 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:53:48.572 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002576+0000", 2024-09-20T18:53:48.572 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.0929999999999997e-06 2024-09-20T18:53:48.572 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.572 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.572 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:53:48.572 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002582+0000", 2024-09-20T18:53:48.572 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.575e-06 2024-09-20T18:53:48.572 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.573 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.573 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:53:48.573 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.004714+0000", 2024-09-20T18:53:48.573 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0021320430000000001 2024-09-20T18:53:48.573 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.573 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.573 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:53:48.573 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.004828+0000", 2024-09-20T18:53:48.573 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000113632 2024-09-20T18:53:48.573 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.573 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.573 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:53:48.573 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.004947+0000", 2024-09-20T18:53:48.573 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000119367 2024-09-20T18:53:48.574 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.574 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.574 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:53:48.574 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.005749+0000", 2024-09-20T18:53:48.574 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000801898 2024-09-20T18:53:48.574 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.574 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.574 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:48.574 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.005930+0000", 2024-09-20T18:53:48.574 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000180686 2024-09-20T18:53:48.574 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.574 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.574 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:48.574 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.119138+0000", 2024-09-20T18:53:48.575 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.11320831200000001 2024-09-20T18:53:48.575 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.575 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.575 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:53:48.575 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.119166+0000", 2024-09-20T18:53:48.575 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.8121000000000001e-05 2024-09-20T18:53:48.575 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.575 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.575 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:53:48.575 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.119187+0000", 2024-09-20T18:53:48.575 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0971999999999998e-05 2024-09-20T18:53:48.575 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:48.575 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:53:48.575 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:48.575 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.576 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.576 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:25344 7.7 7:ead30b22:::benchmark_data_smithi083_70643_object25343:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:53:48.576 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:07.270133+0000", 2024-09-20T18:53:48.576 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 161.221770389, 2024-09-20T18:53:48.576 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.17431155100000001, 2024-09-20T18:53:48.576 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:53:48.576 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:53:48.576 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:48.576 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:53:48.576 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:53:48.576 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:48.576 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 25344 2024-09-20T18:53:48.576 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.576 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:53:48.576 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.577 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:53:48.577 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.270133+0000", 2024-09-20T18:53:48.577 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:48.577 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.577 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.577 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:53:48.577 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.270133+0000", 2024-09-20T18:53:48.577 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:48.577 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.577 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.577 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:53:48.577 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.270134+0000", 2024-09-20T18:53:48.577 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4619999999999999e-06 2024-09-20T18:53:48.577 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.578 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.578 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:53:48.578 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.270200+0000", 2024-09-20T18:53:48.578 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5998000000000001e-05 2024-09-20T18:53:48.578 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.578 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.578 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:53:48.578 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.270214+0000", 2024-09-20T18:53:48.578 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3631e-05 2024-09-20T18:53:48.578 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.578 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.578 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:53:48.579 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.270222+0000", 2024-09-20T18:53:48.579 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.6149999999999999e-06 2024-09-20T18:53:48.579 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.579 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.579 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:53:48.579 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.270277+0000", 2024-09-20T18:53:48.580 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4981000000000001e-05 2024-09-20T18:53:48.580 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.580 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.580 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:53:48.580 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.270400+0000", 2024-09-20T18:53:48.580 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012379000000000001 2024-09-20T18:53:48.580 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.580 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.580 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,3", 2024-09-20T18:53:48.581 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.270534+0000", 2024-09-20T18:53:48.581 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013307600000000001 2024-09-20T18:53:48.581 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.581 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.581 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:53:48.581 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.271704+0000", 2024-09-20T18:53:48.582 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001170152 2024-09-20T18:53:48.582 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.582 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.582 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:48.582 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.442076+0000", 2024-09-20T18:53:48.582 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.170371932 2024-09-20T18:53:48.582 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.582 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.582 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:48.582 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.444327+0000", 2024-09-20T18:53:48.582 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.002251454 2024-09-20T18:53:48.582 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.582 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.583 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:53:48.583 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.444386+0000", 2024-09-20T18:53:48.583 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8579999999999998e-05 2024-09-20T18:53:48.583 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.583 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.583 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:53:48.583 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.444445+0000", 2024-09-20T18:53:48.583 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8879999999999999e-05 2024-09-20T18:53:48.583 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:48.583 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:53:48.583 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:48.583 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.583 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.583 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:26469 7.7 7:edf96f0c:::benchmark_data_smithi083_70643_object26468:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:53:48.584 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:07.996717+0000", 2024-09-20T18:53:48.584 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 160.49518686299999, 2024-09-20T18:53:48.584 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13515669799999999, 2024-09-20T18:53:48.584 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:53:48.584 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:53:48.584 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:48.584 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:53:48.584 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:53:48.584 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:48.584 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 26469 2024-09-20T18:53:48.584 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.585 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:53:48.585 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.585 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:53:48.585 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.996717+0000", 2024-09-20T18:53:48.585 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:48.585 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.585 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.585 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:53:48.585 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.996717+0000", 2024-09-20T18:53:48.585 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:48.585 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.585 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.586 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:53:48.586 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.996718+0000", 2024-09-20T18:53:48.586 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6020000000000001e-06 2024-09-20T18:53:48.586 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.586 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.586 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:53:48.586 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.996814+0000", 2024-09-20T18:53:48.586 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.5480000000000001e-05 2024-09-20T18:53:48.586 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.586 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.586 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:53:48.586 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.996827+0000", 2024-09-20T18:53:48.586 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3749e-05 2024-09-20T18:53:48.586 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.586 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.587 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:53:48.587 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.996835+0000", 2024-09-20T18:53:48.587 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.2309999999999999e-06 2024-09-20T18:53:48.587 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.587 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.587 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:53:48.587 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.996897+0000", 2024-09-20T18:53:48.587 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2515999999999996e-05 2024-09-20T18:53:48.587 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.587 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.587 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:53:48.587 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.996992+0000", 2024-09-20T18:53:48.587 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.4870000000000005e-05 2024-09-20T18:53:48.587 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.588 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.588 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,3", 2024-09-20T18:53:48.588 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.997083+0000", 2024-09-20T18:53:48.588 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.1144999999999996e-05 2024-09-20T18:53:48.588 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.588 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.588 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:53:48.588 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.998108+0000", 2024-09-20T18:53:48.588 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0010251679999999999 2024-09-20T18:53:48.588 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.588 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.588 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:48.588 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.998717+0000", 2024-09-20T18:53:48.588 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00060899999999999995 2024-09-20T18:53:48.588 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.589 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.589 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:48.589 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.131755+0000", 2024-09-20T18:53:48.589 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13303793799999999 2024-09-20T18:53:48.589 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.589 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.589 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:53:48.589 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.131820+0000", 2024-09-20T18:53:48.589 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4678999999999995e-05 2024-09-20T18:53:48.589 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.589 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.589 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:53:48.589 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.131873+0000", 2024-09-20T18:53:48.589 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3319999999999998e-05 2024-09-20T18:53:48.590 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:48.590 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:53:48.590 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:48.590 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.590 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.590 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:26472 7.c 7:326250cc:::benchmark_data_smithi083_70643_object26471:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:53:48.590 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:07.999252+0000", 2024-09-20T18:53:48.590 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 160.49265132100001, 2024-09-20T18:53:48.590 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.133437312, 2024-09-20T18:53:48.590 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:53:48.590 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:53:48.590 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:48.591 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:53:48.591 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:53:48.591 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:48.591 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 26472 2024-09-20T18:53:48.591 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.591 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:53:48.591 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.591 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:53:48.591 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.999252+0000", 2024-09-20T18:53:48.591 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:48.591 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.591 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.591 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:53:48.591 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.999252+0000", 2024-09-20T18:53:48.592 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:48.592 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.592 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.592 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:53:48.592 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.999253+0000", 2024-09-20T18:53:48.592 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.44e-07 2024-09-20T18:53:48.592 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.592 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.592 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:53:48.592 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.999278+0000", 2024-09-20T18:53:48.592 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.5661e-05 2024-09-20T18:53:48.592 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.592 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.592 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:53:48.592 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.999287+0000", 2024-09-20T18:53:48.593 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.2600000000000005e-06 2024-09-20T18:53:48.593 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.593 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.593 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:53:48.593 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.999293+0000", 2024-09-20T18:53:48.593 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4609999999999996e-06 2024-09-20T18:53:48.593 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.593 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.593 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:53:48.593 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.999366+0000", 2024-09-20T18:53:48.593 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.2695999999999997e-05 2024-09-20T18:53:48.593 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.593 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.593 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:53:48.593 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.999475+0000", 2024-09-20T18:53:48.594 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000109472 2024-09-20T18:53:48.594 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.594 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.594 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:53:48.594 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.000511+0000", 2024-09-20T18:53:48.594 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0010355 2024-09-20T18:53:48.594 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.594 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.594 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:53:48.594 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.002097+0000", 2024-09-20T18:53:48.594 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0015861429999999999 2024-09-20T18:53:48.594 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.594 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.594 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:48.595 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.002781+0000", 2024-09-20T18:53:48.595 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000684039 2024-09-20T18:53:48.595 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.595 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.595 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:48.595 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.132602+0000", 2024-09-20T18:53:48.595 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.129820825 2024-09-20T18:53:48.595 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.595 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.595 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:53:48.595 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.132652+0000", 2024-09-20T18:53:48.595 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0139999999999998e-05 2024-09-20T18:53:48.595 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.595 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.596 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:53:48.596 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.132689+0000", 2024-09-20T18:53:48.596 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.7471000000000001e-05 2024-09-20T18:53:48.596 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:48.596 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:53:48.596 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:48.596 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.596 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.596 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:26478 7.d 7:b12f12bc:::benchmark_data_smithi083_70643_object26477:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:53:48.596 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:08.003662+0000", 2024-09-20T18:53:48.596 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 160.48824145699999, 2024-09-20T18:53:48.596 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.15004061199999999, 2024-09-20T18:53:48.596 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:53:48.596 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:53:48.596 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:48.597 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:53:48.597 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:53:48.597 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:48.597 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 26478 2024-09-20T18:53:48.597 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.597 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:53:48.597 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.597 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:53:48.597 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003662+0000", 2024-09-20T18:53:48.597 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:48.597 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.597 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.597 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:53:48.597 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003662+0000", 2024-09-20T18:53:48.597 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:48.598 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.598 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.598 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:53:48.598 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003664+0000", 2024-09-20T18:53:48.598 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8959999999999999e-06 2024-09-20T18:53:48.598 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.598 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.598 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:53:48.598 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003707+0000", 2024-09-20T18:53:48.598 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3288999999999998e-05 2024-09-20T18:53:48.598 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.598 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.598 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:53:48.598 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003724+0000", 2024-09-20T18:53:48.599 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6529e-05 2024-09-20T18:53:48.599 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.599 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.599 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:53:48.599 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003732+0000", 2024-09-20T18:53:48.599 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.7679999999999997e-06 2024-09-20T18:53:48.599 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.599 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.599 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:53:48.599 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003800+0000", 2024-09-20T18:53:48.599 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.7904999999999994e-05 2024-09-20T18:53:48.599 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.599 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.599 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:53:48.599 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003941+0000", 2024-09-20T18:53:48.600 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000140731 2024-09-20T18:53:48.600 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.600 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.600 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,1", 2024-09-20T18:53:48.600 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.004096+0000", 2024-09-20T18:53:48.600 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015452700000000001 2024-09-20T18:53:48.600 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.600 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.600 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:53:48.600 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.005860+0000", 2024-09-20T18:53:48.600 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001764383 2024-09-20T18:53:48.600 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.600 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.600 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:48.600 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.005912+0000", 2024-09-20T18:53:48.601 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.2519000000000003e-05 2024-09-20T18:53:48.601 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.601 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.601 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:48.601 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.153622+0000", 2024-09-20T18:53:48.601 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.147709641 2024-09-20T18:53:48.601 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.601 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.601 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:53:48.601 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.153662+0000", 2024-09-20T18:53:48.601 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9966000000000003e-05 2024-09-20T18:53:48.601 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.601 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.601 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:53:48.602 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.153703+0000", 2024-09-20T18:53:48.602 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0457999999999998e-05 2024-09-20T18:53:48.602 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:48.602 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:53:48.602 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:48.602 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.602 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.602 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:26479 7.d 7:bdad06c1:::benchmark_data_smithi083_70643_object26478:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:53:48.602 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:08.003826+0000", 2024-09-20T18:53:48.602 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 160.48807729500001, 2024-09-20T18:53:48.602 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.15140265899999999, 2024-09-20T18:53:48.602 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:53:48.602 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:53:48.602 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:48.603 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:53:48.603 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:53:48.603 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:48.603 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 26479 2024-09-20T18:53:48.603 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.603 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:53:48.603 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.603 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:53:48.603 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003826+0000", 2024-09-20T18:53:48.603 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:48.603 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.603 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.603 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:53:48.603 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003826+0000", 2024-09-20T18:53:48.603 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:48.604 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.604 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.604 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:53:48.604 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003827+0000", 2024-09-20T18:53:48.604 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.8100000000000002e-07 2024-09-20T18:53:48.604 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.604 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.604 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:53:48.604 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003876+0000", 2024-09-20T18:53:48.604 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8888999999999999e-05 2024-09-20T18:53:48.604 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.604 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.604 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:53:48.604 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003883+0000", 2024-09-20T18:53:48.604 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.8909999999999998e-06 2024-09-20T18:53:48.605 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.605 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.605 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:53:48.605 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003887+0000", 2024-09-20T18:53:48.605 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.5909999999999998e-06 2024-09-20T18:53:48.605 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.605 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.605 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:53:48.605 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.005030+0000", 2024-09-20T18:53:48.605 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0011424289999999999 2024-09-20T18:53:48.605 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.605 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.605 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:53:48.605 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.005164+0000", 2024-09-20T18:53:48.606 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000134537 2024-09-20T18:53:48.606 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.606 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.606 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,1", 2024-09-20T18:53:48.606 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.005242+0000", 2024-09-20T18:53:48.606 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.7956999999999999e-05 2024-09-20T18:53:48.606 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.606 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.606 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:48.606 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.006119+0000", 2024-09-20T18:53:48.606 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00087690700000000001 2024-09-20T18:53:48.606 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.606 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.606 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:53:48.606 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.006138+0000", 2024-09-20T18:53:48.607 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8450000000000001e-05 2024-09-20T18:53:48.607 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.607 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.607 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:48.607 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.155098+0000", 2024-09-20T18:53:48.607 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.14896046900000001 2024-09-20T18:53:48.607 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.607 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.607 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:53:48.607 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.155153+0000", 2024-09-20T18:53:48.607 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5058e-05 2024-09-20T18:53:48.607 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.607 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.607 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:53:48.607 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.155229+0000", 2024-09-20T18:53:48.608 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5699999999999997e-05 2024-09-20T18:53:48.608 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:48.608 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:53:48.608 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:48.608 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.608 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.608 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:26481 7.7 7:eecb08c1:::benchmark_data_smithi083_70643_object26480:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:53:48.608 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:08.006431+0000", 2024-09-20T18:53:48.608 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 160.485472638, 2024-09-20T18:53:48.608 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.14993074200000001, 2024-09-20T18:53:48.608 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:53:48.608 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:53:48.608 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:48.608 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:53:48.609 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:53:48.609 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:48.609 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 26481 2024-09-20T18:53:48.609 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.609 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:53:48.609 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.609 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:53:48.609 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.006431+0000", 2024-09-20T18:53:48.609 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:48.609 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.609 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.609 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:53:48.609 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.006431+0000", 2024-09-20T18:53:48.609 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:48.609 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.610 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.610 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:53:48.610 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.006432+0000", 2024-09-20T18:53:48.610 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.291e-06 2024-09-20T18:53:48.610 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.610 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.610 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:53:48.610 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.006487+0000", 2024-09-20T18:53:48.610 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5058e-05 2024-09-20T18:53:48.610 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.610 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.610 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:53:48.610 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.006497+0000", 2024-09-20T18:53:48.610 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.8579999999999995e-06 2024-09-20T18:53:48.610 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.611 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.611 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:53:48.611 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.006502+0000", 2024-09-20T18:53:48.611 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7140000000000001e-06 2024-09-20T18:53:48.611 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.611 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.611 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:53:48.611 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.006544+0000", 2024-09-20T18:53:48.611 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.2518000000000002e-05 2024-09-20T18:53:48.611 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.611 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.611 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:53:48.611 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.006636+0000", 2024-09-20T18:53:48.611 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.1865000000000002e-05 2024-09-20T18:53:48.612 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.612 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.612 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,3", 2024-09-20T18:53:48.612 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.006753+0000", 2024-09-20T18:53:48.612 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011691100000000001 2024-09-20T18:53:48.612 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.612 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.612 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:53:48.612 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.007675+0000", 2024-09-20T18:53:48.612 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00092171199999999999 2024-09-20T18:53:48.612 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.612 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.612 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:48.612 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.132841+0000", 2024-09-20T18:53:48.612 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.125165949 2024-09-20T18:53:48.613 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.613 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.613 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:48.613 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.156244+0000", 2024-09-20T18:53:48.613 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.023402896999999999 2024-09-20T18:53:48.613 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.613 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.613 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:53:48.613 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.156321+0000", 2024-09-20T18:53:48.613 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.7451000000000001e-05 2024-09-20T18:53:48.613 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.613 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.613 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:53:48.613 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.156362+0000", 2024-09-20T18:53:48.614 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0518000000000001e-05 2024-09-20T18:53:48.614 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:48.614 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:53:48.614 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:48.614 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.614 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.614 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:120182 7.d 7:b9824c9e:::benchmark_data_smithi083_70643_object120181:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T18:53:48.614 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:24.962568+0000", 2024-09-20T18:53:48.614 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 83.529335365999998, 2024-09-20T18:53:48.614 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.116782992, 2024-09-20T18:53:48.614 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:53:48.614 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:53:48.614 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:48.614 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:53:48.614 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:53:48.615 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:48.615 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 120182 2024-09-20T18:53:48.615 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.615 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:53:48.615 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.615 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:53:48.615 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962568+0000", 2024-09-20T18:53:48.615 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:48.615 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.615 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.615 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:53:48.615 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962568+0000", 2024-09-20T18:53:48.615 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:48.615 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.615 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.616 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:53:48.616 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962569+0000", 2024-09-20T18:53:48.616 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4160000000000001e-06 2024-09-20T18:53:48.616 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.616 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.616 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:53:48.616 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962623+0000", 2024-09-20T18:53:48.616 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4024000000000003e-05 2024-09-20T18:53:48.616 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.616 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.616 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:53:48.616 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962636+0000", 2024-09-20T18:53:48.616 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2149e-05 2024-09-20T18:53:48.616 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.617 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.617 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:53:48.617 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962643+0000", 2024-09-20T18:53:48.617 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.554e-06 2024-09-20T18:53:48.617 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.617 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.617 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:53:48.617 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962684+0000", 2024-09-20T18:53:48.617 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.1139e-05 2024-09-20T18:53:48.617 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.617 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.617 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:53:48.617 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962772+0000", 2024-09-20T18:53:48.617 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.7750999999999994e-05 2024-09-20T18:53:48.617 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.618 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.618 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,1", 2024-09-20T18:53:48.618 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962859+0000", 2024-09-20T18:53:48.618 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.6434999999999995e-05 2024-09-20T18:53:48.618 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.618 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.618 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:53:48.618 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.966204+0000", 2024-09-20T18:53:48.618 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0033459029999999999 2024-09-20T18:53:48.618 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.618 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.618 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:48.618 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.970405+0000", 2024-09-20T18:53:48.618 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0042008729999999999 2024-09-20T18:53:48.619 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.619 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.619 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:48.619 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.079209+0000", 2024-09-20T18:53:48.619 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10880400899999999 2024-09-20T18:53:48.619 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.619 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.619 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:53:48.619 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.079303+0000", 2024-09-20T18:53:48.619 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.4022000000000002e-05 2024-09-20T18:53:48.619 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.619 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.619 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:53:48.619 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.079351+0000", 2024-09-20T18:53:48.619 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7716999999999997e-05 2024-09-20T18:53:48.620 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:48.620 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:53:48.620 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:48.620 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.620 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.620 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:120184 7.c 7:3c34b9f0:::benchmark_data_smithi083_70643_object120183:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T18:53:48.620 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:24.962667+0000", 2024-09-20T18:53:48.620 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 83.529236542000007, 2024-09-20T18:53:48.620 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.113312679, 2024-09-20T18:53:48.620 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:53:48.620 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:53:48.620 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:48.620 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:53:48.620 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:53:48.621 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:48.621 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 120184 2024-09-20T18:53:48.621 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.621 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:53:48.621 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.621 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:53:48.621 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962667+0000", 2024-09-20T18:53:48.621 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:48.621 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.621 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.621 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:53:48.621 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962667+0000", 2024-09-20T18:53:48.621 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:48.621 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.622 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.622 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:53:48.622 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962667+0000", 2024-09-20T18:53:48.622 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8400000000000004e-07 2024-09-20T18:53:48.622 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.622 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.622 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:53:48.622 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962702+0000", 2024-09-20T18:53:48.622 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4734000000000003e-05 2024-09-20T18:53:48.622 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.622 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.622 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:53:48.622 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962710+0000", 2024-09-20T18:53:48.622 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.0900000000000005e-06 2024-09-20T18:53:48.623 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.623 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.623 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:53:48.623 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962716+0000", 2024-09-20T18:53:48.623 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.1820000000000003e-06 2024-09-20T18:53:48.623 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.623 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.623 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:53:48.623 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962771+0000", 2024-09-20T18:53:48.623 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4447000000000002e-05 2024-09-20T18:53:48.623 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.623 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.624 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:53:48.624 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962883+0000", 2024-09-20T18:53:48.624 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000111996 2024-09-20T18:53:48.624 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.624 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.624 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:53:48.624 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.963021+0000", 2024-09-20T18:53:48.624 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013835799999999999 2024-09-20T18:53:48.624 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.624 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.624 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:48.624 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.966241+0000", 2024-09-20T18:53:48.624 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0032201299999999999 2024-09-20T18:53:48.624 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.625 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.625 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:53:48.625 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.966257+0000", 2024-09-20T18:53:48.625 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5325e-05 2024-09-20T18:53:48.625 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.625 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.625 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:48.625 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.075873+0000", 2024-09-20T18:53:48.625 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.109616093 2024-09-20T18:53:48.625 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.625 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.625 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:53:48.625 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.075932+0000", 2024-09-20T18:53:48.625 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9487999999999997e-05 2024-09-20T18:53:48.625 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.625 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.626 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:53:48.626 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.075980+0000", 2024-09-20T18:53:48.626 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7252000000000001e-05 2024-09-20T18:53:48.626 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:48.626 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:53:48.626 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:48.626 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.626 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.626 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.14808.0:120191 7.b e121/83 7:d45eee6a:::benchmark_data_smithi083_70643_object120190:head v 121'7491, mlcod=121'7489)", 2024-09-20T18:53:48.626 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:24.971948+0000", 2024-09-20T18:53:48.626 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 83.519955573999994, 2024-09-20T18:53:48.626 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.111765014, 2024-09-20T18:53:48.626 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:53:48.626 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:53:48.626 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:48.627 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:53:48.627 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.627 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:53:48.627 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.971948+0000", 2024-09-20T18:53:48.627 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:48.627 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.627 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.627 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:53:48.627 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.971948+0000", 2024-09-20T18:53:48.627 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:48.627 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.627 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.627 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:53:48.627 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.971949+0000", 2024-09-20T18:53:48.628 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.7400000000000002e-07 2024-09-20T18:53:48.628 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.628 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.628 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:53:48.628 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.971990+0000", 2024-09-20T18:53:48.628 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.1825e-05 2024-09-20T18:53:48.628 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.628 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.628 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:53:48.628 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.073923+0000", 2024-09-20T18:53:48.628 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.101932732 2024-09-20T18:53:48.628 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.628 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.628 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:53:48.628 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.073931+0000", 2024-09-20T18:53:48.629 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.1510000000000004e-06 2024-09-20T18:53:48.629 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.629 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.629 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:53:48.629 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.073984+0000", 2024-09-20T18:53:48.629 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.2679000000000002e-05 2024-09-20T18:53:48.629 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.629 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.629 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:53:48.629 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.074004+0000", 2024-09-20T18:53:48.629 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9882e-05 2024-09-20T18:53:48.629 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.629 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.629 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:53:48.629 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.083676+0000", 2024-09-20T18:53:48.630 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0096721900000000006 2024-09-20T18:53:48.630 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.630 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.630 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:53:48.630 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.083713+0000", 2024-09-20T18:53:48.630 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6780999999999999e-05 2024-09-20T18:53:48.630 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:48.630 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:53:48.630 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:48.630 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.630 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.630 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:120194 7.d 7:b6298d4c:::benchmark_data_smithi083_70643_object120193:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T18:53:48.630 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:24.977580+0000", 2024-09-20T18:53:48.630 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 83.514323755000007, 2024-09-20T18:53:48.631 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.111515778, 2024-09-20T18:53:48.631 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:53:48.631 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:53:48.631 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:48.631 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:53:48.631 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:53:48.631 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:48.631 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 120194 2024-09-20T18:53:48.632 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.632 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:53:48.632 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.632 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:53:48.632 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.977580+0000", 2024-09-20T18:53:48.632 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:48.632 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.632 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.632 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:53:48.632 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.977580+0000", 2024-09-20T18:53:48.632 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:48.632 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.632 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.633 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:53:48.633 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.977582+0000", 2024-09-20T18:53:48.633 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9520000000000001e-06 2024-09-20T18:53:48.633 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.633 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.633 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:53:48.633 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.977642+0000", 2024-09-20T18:53:48.633 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0204000000000002e-05 2024-09-20T18:53:48.633 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.633 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.633 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:53:48.633 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.072252+0000", 2024-09-20T18:53:48.633 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.094610194999999994 2024-09-20T18:53:48.633 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.633 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.634 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:53:48.634 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.072287+0000", 2024-09-20T18:53:48.634 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4542999999999998e-05 2024-09-20T18:53:48.634 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.634 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.634 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:53:48.634 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.072751+0000", 2024-09-20T18:53:48.634 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000464289 2024-09-20T18:53:48.634 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.634 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.634 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:53:48.634 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.072883+0000", 2024-09-20T18:53:48.634 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013254099999999999 2024-09-20T18:53:48.634 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.635 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.635 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,1", 2024-09-20T18:53:48.635 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.073544+0000", 2024-09-20T18:53:48.635 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00066074700000000005 2024-09-20T18:53:48.635 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.635 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.635 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:48.635 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.088934+0000", 2024-09-20T18:53:48.635 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.015389551 2024-09-20T18:53:48.635 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.635 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.635 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:48.635 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.088971+0000", 2024-09-20T18:53:48.635 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.7249000000000002e-05 2024-09-20T18:53:48.636 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.636 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.636 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:53:48.636 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.088990+0000", 2024-09-20T18:53:48.636 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8770000000000002e-05 2024-09-20T18:53:48.636 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.636 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.636 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:53:48.636 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.089045+0000", 2024-09-20T18:53:48.636 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5386999999999997e-05 2024-09-20T18:53:48.636 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.637 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.637 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:53:48.637 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.089095+0000", 2024-09-20T18:53:48.637 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0349999999999997e-05 2024-09-20T18:53:48.637 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:48.637 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:53:48.637 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:48.637 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.638 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.638 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:139218 7.d 7:bef6f051:::benchmark_data_smithi083_70643_object16416:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-20T18:53:48.638 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:42.802191+0000", 2024-09-20T18:53:48.638 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 65.689712559, 2024-09-20T18:53:48.638 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.12718874399999999, 2024-09-20T18:53:48.638 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:53:48.639 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:53:48.639 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:48.639 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:53:48.639 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:53:48.639 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:48.639 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 139218 2024-09-20T18:53:48.639 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.640 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:53:48.640 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.640 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:53:48.640 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.802191+0000", 2024-09-20T18:53:48.640 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:48.640 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.640 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.641 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:53:48.641 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.802191+0000", 2024-09-20T18:53:48.641 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:48.641 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.641 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.641 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:53:48.642 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.802192+0000", 2024-09-20T18:53:48.642 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0759999999999999e-06 2024-09-20T18:53:48.642 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.642 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.642 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:53:48.642 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.802196+0000", 2024-09-20T18:53:48.642 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.7309999999999999e-06 2024-09-20T18:53:48.643 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.643 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.643 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:53:48.643 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.802203+0000", 2024-09-20T18:53:48.643 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.4599999999999997e-06 2024-09-20T18:53:48.643 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.644 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.644 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:53:48.644 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.802209+0000", 2024-09-20T18:53:48.644 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5550000000000002e-06 2024-09-20T18:53:48.644 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.644 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.645 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:53:48.645 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.802250+0000", 2024-09-20T18:53:48.645 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.1502999999999999e-05 2024-09-20T18:53:48.645 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.645 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.645 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:48.646 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:53:48.647 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.802322+0000", 2024-09-20T18:53:48.647 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.1723000000000004e-05 2024-09-20T18:53:48.647 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.647 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.647 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,1", 2024-09-20T18:53:48.647 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.802390+0000", 2024-09-20T18:53:48.648 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.8120000000000005e-05 2024-09-20T18:53:48.648 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.648 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.648 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:48.648 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.807367+0000", 2024-09-20T18:53:48.648 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0049773769999999998 2024-09-20T18:53:48.649 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.649 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.649 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:53:48.649 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.807382+0000", 2024-09-20T18:53:48.649 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4667e-05 2024-09-20T18:53:48.649 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.650 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.650 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:48.650 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.929326+0000", 2024-09-20T18:53:48.650 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.121944356 2024-09-20T18:53:48.650 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.650 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.651 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:53:48.651 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.929356+0000", 2024-09-20T18:53:48.651 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9995e-05 2024-09-20T18:53:48.651 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.651 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.651 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:53:48.652 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.929380+0000", 2024-09-20T18:53:48.652 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3181e-05 2024-09-20T18:53:48.652 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:48.652 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:53:48.652 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:48.652 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.652 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.653 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:139220 7.d 7:b6a17747:::benchmark_data_smithi083_70643_object16418:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-20T18:53:48.653 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:42.803402+0000", 2024-09-20T18:53:48.653 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 65.688501368000004, 2024-09-20T18:53:48.653 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.129377032, 2024-09-20T18:53:48.653 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:53:48.653 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:53:48.653 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:48.654 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:53:48.654 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:53:48.654 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:48.654 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 139220 2024-09-20T18:53:48.654 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.654 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:53:48.655 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.655 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:53:48.655 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.803402+0000", 2024-09-20T18:53:48.655 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:48.655 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.655 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.656 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:53:48.656 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.803402+0000", 2024-09-20T18:53:48.656 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:48.656 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.656 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.656 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:53:48.657 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.803403+0000", 2024-09-20T18:53:48.657 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.097e-06 2024-09-20T18:53:48.657 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.657 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.657 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:53:48.657 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.803408+0000", 2024-09-20T18:53:48.657 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.561e-06 2024-09-20T18:53:48.658 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.658 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.658 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:53:48.658 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.803419+0000", 2024-09-20T18:53:48.658 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1591e-05 2024-09-20T18:53:48.658 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.659 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.659 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:53:48.659 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.803426+0000", 2024-09-20T18:53:48.659 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.9160000000000004e-06 2024-09-20T18:53:48.659 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.659 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.659 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:53:48.660 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.805298+0000", 2024-09-20T18:53:48.660 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0018719780000000001 2024-09-20T18:53:48.660 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.660 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.660 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:53:48.660 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.805418+0000", 2024-09-20T18:53:48.660 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000119739 2024-09-20T18:53:48.660 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.661 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.661 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,1", 2024-09-20T18:53:48.661 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.805543+0000", 2024-09-20T18:53:48.661 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012481699999999999 2024-09-20T18:53:48.661 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.661 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.662 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:53:48.662 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.807413+0000", 2024-09-20T18:53:48.662 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001870678 2024-09-20T18:53:48.662 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.662 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.662 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:48.662 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.808024+0000", 2024-09-20T18:53:48.662 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00061090700000000005 2024-09-20T18:53:48.662 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.663 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.663 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:48.663 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.932673+0000", 2024-09-20T18:53:48.663 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.12464900299999999 2024-09-20T18:53:48.663 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.663 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.663 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:53:48.663 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.932739+0000", 2024-09-20T18:53:48.663 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5979e-05 2024-09-20T18:53:48.663 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.663 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.663 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:53:48.663 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.932779+0000", 2024-09-20T18:53:48.663 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9765999999999998e-05 2024-09-20T18:53:48.664 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:48.664 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:53:48.664 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:48.664 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.664 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.664 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:139224 7.7 7:e3ad72f8:::benchmark_data_smithi083_70643_object16422:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-20T18:53:48.664 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:42.806758+0000", 2024-09-20T18:53:48.664 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 65.685145362, 2024-09-20T18:53:48.664 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.121962579, 2024-09-20T18:53:48.664 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:53:48.664 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:53:48.664 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:53:48.664 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:53:48.664 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:53:48.664 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:48.665 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 139224 2024-09-20T18:53:48.665 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.665 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:53:48.665 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.665 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:53:48.665 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806758+0000", 2024-09-20T18:53:48.665 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:48.665 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.665 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.665 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:53:48.665 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806758+0000", 2024-09-20T18:53:48.665 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:48.665 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.665 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.665 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:53:48.666 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806759+0000", 2024-09-20T18:53:48.666 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.065e-06 2024-09-20T18:53:48.666 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.666 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.666 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:53:48.666 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806764+0000", 2024-09-20T18:53:48.666 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8010000000000003e-06 2024-09-20T18:53:48.666 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.666 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.666 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:53:48.666 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806777+0000", 2024-09-20T18:53:48.666 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3292000000000001e-05 2024-09-20T18:53:48.666 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.667 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.667 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:53:48.667 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806785+0000", 2024-09-20T18:53:48.667 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.9959999999999993e-06 2024-09-20T18:53:48.667 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.667 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.667 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:53:48.667 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806840+0000", 2024-09-20T18:53:48.667 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4308000000000002e-05 2024-09-20T18:53:48.667 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.667 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.667 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:53:48.667 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806948+0000", 2024-09-20T18:53:48.667 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00010843799999999999 2024-09-20T18:53:48.668 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.668 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.668 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,3", 2024-09-20T18:53:48.668 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.807066+0000", 2024-09-20T18:53:48.668 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000117766 2024-09-20T18:53:48.668 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.668 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.668 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:53:48.668 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.809368+0000", 2024-09-20T18:53:48.668 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0023018130000000002 2024-09-20T18:53:48.668 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.668 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.668 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:48.668 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.809785+0000", 2024-09-20T18:53:48.669 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00041736699999999998 2024-09-20T18:53:48.669 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.669 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.669 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:48.669 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.928629+0000", 2024-09-20T18:53:48.669 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.118843746 2024-09-20T18:53:48.669 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.669 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.669 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:53:48.669 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.928687+0000", 2024-09-20T18:53:48.669 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8418999999999998e-05 2024-09-20T18:53:48.669 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:48.669 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:48.669 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:53:48.669 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.928721+0000", 2024-09-20T18:53:48.670 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.3568000000000001e-05 2024-09-20T18:53:48.670 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:48.670 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:53:48.670 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:48.670 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:48.670 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:53:48.670 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T18:53:48.717 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-20T18:53:48.746 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:48.792 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:48.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:48.793 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:48.893 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:48.943 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:48.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:48.944 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:49.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:48 smithi142 ceph-mon[32508]: 4.6 scrub starts 2024-09-20T18:53:49.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:48 smithi142 ceph-mon[32508]: 4.6 scrub ok 2024-09-20T18:53:49.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:48 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3897636758' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:53:49.045 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:49.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:48 smithi083 ceph-mon[29421]: 4.6 scrub starts 2024-09-20T18:53:49.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:48 smithi083 ceph-mon[29421]: 4.6 scrub ok 2024-09-20T18:53:49.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:48 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3897636758' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:53:49.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:48 smithi083 ceph-mon[36260]: 4.6 scrub starts 2024-09-20T18:53:49.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:48 smithi083 ceph-mon[36260]: 4.6 scrub ok 2024-09-20T18:53:49.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:48 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3897636758' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:53:49.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:49.092 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:49.092 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:49.193 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:49.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:49.221 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:49.222 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:49.322 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:53:49.357 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:49.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:49.358 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:53:49.458 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:49.482 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:49.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:49.483 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:49.584 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:49.628 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:49.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:49.629 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:49.687 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:53:49.729 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:49.779 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:49.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:49.780 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:49.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:49 smithi083 ceph-mon[29421]: 4.7 scrub starts 2024-09-20T18:53:49.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:49 smithi083 ceph-mon[29421]: 4.7 scrub ok 2024-09-20T18:53:49.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:49 smithi083 ceph-mon[29421]: 3.16 scrub starts 2024-09-20T18:53:49.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:49 smithi083 ceph-mon[29421]: 3.16 scrub ok 2024-09-20T18:53:49.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:49 smithi083 ceph-mon[29421]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T18:53:49.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:49 smithi083 ceph-mon[29421]: Cluster is now healthy 2024-09-20T18:53:49.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:49 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3897636758' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T18:53:49.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:49 smithi083 ceph-mon[29421]: osdmap e149: 8 total, 8 up, 8 in 2024-09-20T18:53:49.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:49 smithi083 ceph-mon[29421]: 4.f scrub starts 2024-09-20T18:53:49.806 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:49 smithi083 ceph-mon[29421]: 4.f scrub ok 2024-09-20T18:53:49.806 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:49 smithi083 ceph-mon[29421]: pgmap v185: 148 pgs: 148 active+clean; 2.5 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 842 op/s 2024-09-20T18:53:49.806 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:49 smithi083 ceph-mon[29421]: 6.17 scrub starts 2024-09-20T18:53:49.806 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:49 smithi083 ceph-mon[29421]: 6.17 scrub ok 2024-09-20T18:53:49.806 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:49 smithi083 ceph-mon[36260]: 4.7 scrub starts 2024-09-20T18:53:49.806 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:49 smithi083 ceph-mon[36260]: 4.7 scrub ok 2024-09-20T18:53:49.806 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:49 smithi083 ceph-mon[36260]: 3.16 scrub starts 2024-09-20T18:53:49.806 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:49 smithi083 ceph-mon[36260]: 3.16 scrub ok 2024-09-20T18:53:49.806 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:49 smithi083 ceph-mon[36260]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T18:53:49.806 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:49 smithi083 ceph-mon[36260]: Cluster is now healthy 2024-09-20T18:53:49.806 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:49 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3897636758' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T18:53:49.807 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:49 smithi083 ceph-mon[36260]: osdmap e149: 8 total, 8 up, 8 in 2024-09-20T18:53:49.807 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:49 smithi083 ceph-mon[36260]: 4.f scrub starts 2024-09-20T18:53:49.807 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:49 smithi083 ceph-mon[36260]: 4.f scrub ok 2024-09-20T18:53:49.807 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:49 smithi083 ceph-mon[36260]: pgmap v185: 148 pgs: 148 active+clean; 2.5 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 842 op/s 2024-09-20T18:53:49.807 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:49 smithi083 ceph-mon[36260]: 6.17 scrub starts 2024-09-20T18:53:49.807 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:49 smithi083 ceph-mon[36260]: 6.17 scrub ok 2024-09-20T18:53:49.881 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:49.918 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:49.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:49.920 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:50.020 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:50.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:49 smithi142 ceph-mon[32508]: 4.7 scrub starts 2024-09-20T18:53:50.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:49 smithi142 ceph-mon[32508]: 4.7 scrub ok 2024-09-20T18:53:50.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:49 smithi142 ceph-mon[32508]: 3.16 scrub starts 2024-09-20T18:53:50.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:49 smithi142 ceph-mon[32508]: 3.16 scrub ok 2024-09-20T18:53:50.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:49 smithi142 ceph-mon[32508]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T18:53:50.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:49 smithi142 ceph-mon[32508]: Cluster is now healthy 2024-09-20T18:53:50.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:49 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3897636758' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T18:53:50.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:49 smithi142 ceph-mon[32508]: osdmap e149: 8 total, 8 up, 8 in 2024-09-20T18:53:50.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:49 smithi142 ceph-mon[32508]: 4.f scrub starts 2024-09-20T18:53:50.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:49 smithi142 ceph-mon[32508]: 4.f scrub ok 2024-09-20T18:53:50.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:49 smithi142 ceph-mon[32508]: pgmap v185: 148 pgs: 148 active+clean; 2.5 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 842 op/s 2024-09-20T18:53:50.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:49 smithi142 ceph-mon[32508]: 6.17 scrub starts 2024-09-20T18:53:50.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:49 smithi142 ceph-mon[32508]: 6.17 scrub ok 2024-09-20T18:53:50.074 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:50.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:50.075 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:50.099 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-20T18:53:50.176 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:50.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:50.209 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:50.210 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:50.310 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:50.341 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:50.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:50.342 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:50.443 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:50.474 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:53:50.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:50.481 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:50.481 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:50.582 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:50.613 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:50.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:50.614 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:50.693 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:53:50.694 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-20T18:53:50.714 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:53:50.744 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:50.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:50.745 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:53:50.846 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:50.868 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:50.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:50.869 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:50.940 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T18:53:50.970 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:50.977 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:50 smithi083 ceph-mon[29421]: 6.4 scrub starts 2024-09-20T18:53:50.977 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:50 smithi083 ceph-mon[29421]: 6.4 scrub ok 2024-09-20T18:53:50.978 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:50 smithi083 ceph-mon[29421]: 2.2 scrub starts 2024-09-20T18:53:50.978 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:50 smithi083 ceph-mon[29421]: 2.2 scrub ok 2024-09-20T18:53:50.978 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:50 smithi083 ceph-mon[29421]: 5.17 scrub starts 2024-09-20T18:53:50.978 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:50 smithi083 ceph-mon[29421]: 5.17 scrub ok 2024-09-20T18:53:50.978 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:50 smithi083 ceph-mon[36260]: 6.4 scrub starts 2024-09-20T18:53:50.978 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:50 smithi083 ceph-mon[36260]: 6.4 scrub ok 2024-09-20T18:53:50.979 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:50 smithi083 ceph-mon[36260]: 2.2 scrub starts 2024-09-20T18:53:50.979 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:50 smithi083 ceph-mon[36260]: 2.2 scrub ok 2024-09-20T18:53:50.979 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:50 smithi083 ceph-mon[36260]: 5.17 scrub starts 2024-09-20T18:53:50.979 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:50 smithi083 ceph-mon[36260]: 5.17 scrub ok 2024-09-20T18:53:50.982 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:53:50 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:53:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:53:51.004 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:51.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:51.005 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:51.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:50 smithi142 ceph-mon[32508]: 6.4 scrub starts 2024-09-20T18:53:51.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:50 smithi142 ceph-mon[32508]: 6.4 scrub ok 2024-09-20T18:53:51.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:50 smithi142 ceph-mon[32508]: 2.2 scrub starts 2024-09-20T18:53:51.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:50 smithi142 ceph-mon[32508]: 2.2 scrub ok 2024-09-20T18:53:51.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:50 smithi142 ceph-mon[32508]: 5.17 scrub starts 2024-09-20T18:53:51.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:50 smithi142 ceph-mon[32508]: 5.17 scrub ok 2024-09-20T18:53:51.105 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:51.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:51.155 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:51.156 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:51.256 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:51.297 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:51.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:51.299 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:51.399 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:51.442 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:51.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:51.443 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:51.544 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:51.575 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T18:53:51.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:51.589 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:51.589 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:51.634 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T18:53:51.634 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-09-20T18:53:51.634 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:51.634 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.14808.0:210928 7.c e149/83 7:385d5319:::benchmark_data_smithi083_70643_object88126:head v 149'13266, mlcod=149'13261)", 2024-09-20T18:53:51.635 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:53:51.613825+0000", 2024-09-20T18:53:51.635 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.002513918, 2024-09-20T18:53:51.635 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0025376769999999999, 2024-09-20T18:53:51.635 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:53:51.635 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:53:51.635 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-09-20T18:53:51.635 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:53:51.635 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:51.635 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:53:51.635 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:51.613825+0000", 2024-09-20T18:53:51.635 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:51.635 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:51.635 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:51.636 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:53:51.636 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:51.613825+0000", 2024-09-20T18:53:51.636 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:51.636 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:51.636 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:51.636 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:53:51.636 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:51.613826+0000", 2024-09-20T18:53:51.636 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.8599999999999996e-07 2024-09-20T18:53:51.636 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:51.636 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:51.636 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:53:51.636 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:51.613831+0000", 2024-09-20T18:53:51.636 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.4020000000000004e-06 2024-09-20T18:53:51.636 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:51.637 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:51.637 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:53:51.637 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:51.613838+0000", 2024-09-20T18:53:51.637 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.7579999999999992e-06 2024-09-20T18:53:51.637 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:51.637 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:51.637 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:53:51.637 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:51.613845+0000", 2024-09-20T18:53:51.637 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4389999999999997e-06 2024-09-20T18:53:51.637 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:51.637 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:51.637 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:53:51.637 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:51.613899+0000", 2024-09-20T18:53:51.637 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4543000000000003e-05 2024-09-20T18:53:51.638 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:51.638 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:51.638 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:53:51.638 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:51.613924+0000", 2024-09-20T18:53:51.638 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.4807000000000002e-05 2024-09-20T18:53:51.638 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:51.638 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:53:51.638 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:51.638 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:51.638 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:51.638 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.14808.0:210926 7.8 e149/83 7:1409ed29:::benchmark_data_smithi083_70643_object88124:head v 149'13224, mlcod=149'13221)", 2024-09-20T18:53:51.638 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:53:51.614164+0000", 2024-09-20T18:53:51.638 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.002174685, 2024-09-20T18:53:51.639 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0022518490000000002, 2024-09-20T18:53:51.639 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:53:51.639 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:53:51.639 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-09-20T18:53:51.639 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:53:51.639 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:51.639 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:53:51.639 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:51.614164+0000", 2024-09-20T18:53:51.639 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:51.639 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:51.639 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:51.639 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:53:51.639 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:51.614164+0000", 2024-09-20T18:53:51.639 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:51.639 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:51.639 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:51.640 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:53:51.640 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:51.614165+0000", 2024-09-20T18:53:51.640 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.9800000000000003e-07 2024-09-20T18:53:51.640 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:51.640 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:51.640 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:53:51.640 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:51.614173+0000", 2024-09-20T18:53:51.640 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.7610000000000007e-06 2024-09-20T18:53:51.640 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:51.640 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:51.640 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:53:51.640 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:51.614185+0000", 2024-09-20T18:53:51.640 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2133000000000001e-05 2024-09-20T18:53:51.640 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:51.641 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:51.641 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:53:51.641 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:51.614192+0000", 2024-09-20T18:53:51.641 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.9369999999999998e-06 2024-09-20T18:53:51.641 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:51.641 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:51.641 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:53:51.641 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:51.614263+0000", 2024-09-20T18:53:51.641 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.0630999999999995e-05 2024-09-20T18:53:51.641 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:51.641 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:51.641 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:53:51.641 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:51.614281+0000", 2024-09-20T18:53:51.641 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8725000000000001e-05 2024-09-20T18:53:51.641 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:51.641 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:53:51.641 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:51.642 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:51.642 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:51.642 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.14808.0:210932 7.f e149/83 7:fbc7f37e:::benchmark_data_smithi083_70643_object88130:head v 149'13307, mlcod=149'13305)", 2024-09-20T18:53:51.642 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:53:51.616173+0000", 2024-09-20T18:53:51.642 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.00016628899999999999, 2024-09-20T18:53:51.642 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00028883700000000001, 2024-09-20T18:53:51.642 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:53:51.642 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:53:51.642 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-09-20T18:53:51.642 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:53:51.642 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:51.642 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:53:51.642 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:51.616173+0000", 2024-09-20T18:53:51.642 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:51.643 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:51.643 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:51.643 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:53:51.643 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:51.616173+0000", 2024-09-20T18:53:51.643 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:51.643 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:51.643 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:51.643 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:53:51.643 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:51.616174+0000", 2024-09-20T18:53:51.643 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.6499999999999998e-07 2024-09-20T18:53:51.643 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:51.643 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:51.643 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:53:51.643 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:51.616179+0000", 2024-09-20T18:53:51.644 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0329999999999999e-06 2024-09-20T18:53:51.644 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:51.644 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:51.644 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:53:51.644 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:51.616190+0000", 2024-09-20T18:53:51.644 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1463e-05 2024-09-20T18:53:51.644 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:51.644 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:51.644 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:53:51.644 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:51.616196+0000", 2024-09-20T18:53:51.644 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.1260000000000003e-06 2024-09-20T18:53:51.644 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:51.644 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:51.644 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:53:51.644 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:51.616245+0000", 2024-09-20T18:53:51.644 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8251000000000003e-05 2024-09-20T18:53:51.645 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:51.645 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:51.645 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:53:51.645 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:51.616265+0000", 2024-09-20T18:53:51.645 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0426999999999999e-05 2024-09-20T18:53:51.645 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:51.645 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:53:51.645 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:51.645 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:51.645 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:51.645 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:210923 7.5 7:ae47c722:::benchmark_data_smithi083_70643_object88121:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-20T18:53:51.645 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:53:51.608600+0000", 2024-09-20T18:53:51.645 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.007739035, 2024-09-20T18:53:51.645 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0079108549999999996, 2024-09-20T18:53:51.645 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:53:51.645 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:53:51.646 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "waiting for sub ops", 2024-09-20T18:53:51.646 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:53:51.646 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:53:51.646 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:53:51.646 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 210923 2024-09-20T18:53:51.646 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:51.646 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:53:51.646 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:51.646 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:53:51.646 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:51.608600+0000", 2024-09-20T18:53:51.646 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:51.646 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:51.646 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:51.646 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:53:51.646 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:51.608600+0000", 2024-09-20T18:53:51.646 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:53:51.647 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:51.647 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:51.647 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:53:51.647 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:51.608602+0000", 2024-09-20T18:53:51.647 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.922e-06 2024-09-20T18:53:51.647 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:51.647 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:51.647 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:53:51.647 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:51.608611+0000", 2024-09-20T18:53:51.647 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.6379999999999993e-06 2024-09-20T18:53:51.647 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:51.647 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:51.647 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:53:51.647 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:51.608624+0000", 2024-09-20T18:53:51.647 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3686000000000001e-05 2024-09-20T18:53:51.648 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:51.648 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:51.648 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:53:51.648 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:51.608633+0000", 2024-09-20T18:53:51.648 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.3969999999999994e-06 2024-09-20T18:53:51.648 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:51.648 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:51.648 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:53:51.648 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:51.608780+0000", 2024-09-20T18:53:51.648 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00014766900000000001 2024-09-20T18:53:51.648 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:51.648 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:51.648 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:53:51.648 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:51.608879+0000", 2024-09-20T18:53:51.648 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.9075999999999998e-05 2024-09-20T18:53:51.648 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:51.649 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:51.649 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,2", 2024-09-20T18:53:51.649 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:51.608979+0000", 2024-09-20T18:53:51.649 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.9191999999999995e-05 2024-09-20T18:53:51.649 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:51.649 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:51.649 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:53:51.649 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:51.611257+0000", 2024-09-20T18:53:51.649 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0022787160000000001 2024-09-20T18:53:51.649 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:53:51.649 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:53:51.649 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:53:51.649 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:51.613973+0000", 2024-09-20T18:53:51.649 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0027157829999999998 2024-09-20T18:53:51.649 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:51.649 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:53:51.649 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:51.650 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:53:51.650 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-09-20T18:53:51.650 INFO:teuthology.orchestra.run.smithi142.stdout: "num_ops": 4 2024-09-20T18:53:51.650 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T18:53:51.690 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:51.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:51.717 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:51.717 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:51.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:51 smithi142 ceph-mon[32508]: 7.0 scrub starts 2024-09-20T18:53:51.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:51 smithi142 ceph-mon[32508]: from='client.24947 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:53:51.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:51 smithi142 ceph-mon[32508]: pgmap v186: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 2.3 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 983 op/s 2024-09-20T18:53:51.774 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:51 smithi142 ceph-mon[32508]: 7.0 scrub ok 2024-09-20T18:53:51.818 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:53:51.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:51.869 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:51.869 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:53:51.970 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:53:51.999 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:52.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:52.000 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:53:52.018 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:51 smithi083 ceph-mon[29421]: 7.0 scrub starts 2024-09-20T18:53:52.018 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:51 smithi083 ceph-mon[29421]: from='client.24947 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:53:52.018 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:51 smithi083 ceph-mon[29421]: pgmap v186: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 2.3 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 983 op/s 2024-09-20T18:53:52.018 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:51 smithi083 ceph-mon[29421]: 7.0 scrub ok 2024-09-20T18:53:52.018 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:51 smithi083 ceph-mon[36260]: 7.0 scrub starts 2024-09-20T18:53:52.019 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:51 smithi083 ceph-mon[36260]: from='client.24947 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:53:52.019 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:51 smithi083 ceph-mon[36260]: pgmap v186: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 2.3 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 983 op/s 2024-09-20T18:53:52.019 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:51 smithi083 ceph-mon[36260]: 7.0 scrub ok 2024-09-20T18:53:52.070 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":185,"stamp":"2024-09-20T18:53:48.833612+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2652838451,"num_objects":40670,"num_object_clones":0,"num_object_copies":122010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40670,"num_whiteouts":0,"num_read":1084,"num_read_kb":941,"num_write":205678,"num_write_kb":7860810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":590808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":90875,"ondisk_log_size":90875,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":8790644,"kb_used_data":7880232,"kb_used_omap":12,"kb_used_meta":910131,"kb_avail":741170572,"statfs":{"total":767960285184,"available":758958665728,"internally_reserved":0,"allocated":8069357568,"data_stored":8066053329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":931974704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-331087872,"num_objects":-5052,"num_object_clones":0,"num_object_copies":-15156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5052,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5052,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001186"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":195,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.673887+0000","last_change":"2024-09-20T18:51:57.238254+0000","last_active":"2024-09-20T18:53:43.673887+0000","last_peered":"2024-09-20T18:53:43.673887+0000","last_clean":"2024-09-20T18:53:43.673887+0000","last_became_active":"2024-09-20T18:49:45.943201+0000","last_became_peered":"2024-09-20T18:49:45.943201+0000","last_unstale":"2024-09-20T18:53:43.673887+0000","last_undegraded":"2024-09-20T18:53:43.673887+0000","last_fullsized":"2024-09-20T18:53:43.673887+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:57.238177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:57.238177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:06.040036+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":233,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.655474+0000","last_change":"2024-09-20T18:53:15.772300+0000","last_active":"2024-09-20T18:53:43.655474+0000","last_peered":"2024-09-20T18:53:43.655474+0000","last_clean":"2024-09-20T18:53:43.655474+0000","last_became_active":"2024-09-20T18:52:15.609648+0000","last_became_peered":"2024-09-20T18:52:15.609648+0000","last_unstale":"2024-09-20T18:53:43.655474+0000","last_undegraded":"2024-09-20T18:53:43.655474+0000","last_fullsized":"2024-09-20T18:53:43.655474+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:53:15.772228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:15.772228+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:33.912905+0000","scrub_duration":37,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":196,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.668345+0000","last_change":"2024-09-20T18:53:21.291759+0000","last_active":"2024-09-20T18:53:43.668345+0000","last_peered":"2024-09-20T18:53:43.668345+0000","last_clean":"2024-09-20T18:53:43.668345+0000","last_became_active":"2024-09-20T18:49:49.959514+0000","last_became_peered":"2024-09-20T18:49:49.959514+0000","last_unstale":"2024-09-20T18:53:43.668345+0000","last_undegraded":"2024-09-20T18:53:43.668345+0000","last_fullsized":"2024-09-20T18:53:43.668345+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:21.291695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:21.291695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:25.316294+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":192,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.655726+0000","last_change":"2024-09-20T18:53:08.740138+0000","last_active":"2024-09-20T18:53:43.655726+0000","last_peered":"2024-09-20T18:53:43.655726+0000","last_clean":"2024-09-20T18:53:43.655726+0000","last_became_active":"2024-09-20T18:49:51.969421+0000","last_became_peered":"2024-09-20T18:49:51.969421+0000","last_unstale":"2024-09-20T18:53:43.655726+0000","last_undegraded":"2024-09-20T18:53:43.655726+0000","last_fullsized":"2024-09-20T18:53:43.655726+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:08.740093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:08.740093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:11.987287+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":184,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.675479+0000","last_change":"2024-09-20T18:51:29.257299+0000","last_active":"2024-09-20T18:53:43.675479+0000","last_peered":"2024-09-20T18:53:43.675479+0000","last_clean":"2024-09-20T18:53:43.675479+0000","last_became_active":"2024-09-20T18:49:52.141967+0000","last_became_peered":"2024-09-20T18:49:52.141967+0000","last_unstale":"2024-09-20T18:53:43.675479+0000","last_undegraded":"2024-09-20T18:53:43.675479+0000","last_fullsized":"2024-09-20T18:53:43.675479+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:29.257220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:29.257220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":196,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.676143+0000","last_change":"2024-09-20T18:51:25.208165+0000","last_active":"2024-09-20T18:53:43.676143+0000","last_peered":"2024-09-20T18:53:43.676143+0000","last_clean":"2024-09-20T18:53:43.676143+0000","last_became_active":"2024-09-20T18:49:45.943607+0000","last_became_peered":"2024-09-20T18:49:45.943607+0000","last_unstale":"2024-09-20T18:53:43.676143+0000","last_undegraded":"2024-09-20T18:53:43.676143+0000","last_fullsized":"2024-09-20T18:53:43.676143+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:25.208073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:25.208073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:07.185746+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":217,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.675428+0000","last_change":"2024-09-20T18:51:07.130137+0000","last_active":"2024-09-20T18:53:43.675428+0000","last_peered":"2024-09-20T18:53:43.675428+0000","last_clean":"2024-09-20T18:53:43.675428+0000","last_became_active":"2024-09-20T18:49:47.952150+0000","last_became_peered":"2024-09-20T18:49:47.952150+0000","last_unstale":"2024-09-20T18:53:43.675428+0000","last_undegraded":"2024-09-20T18:53:43.675428+0000","last_fullsized":"2024-09-20T18:53:43.675428+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:51:07.130062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:07.130062+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:33.208818+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":188,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.655403+0000","last_change":"2024-09-20T18:51:09.436429+0000","last_active":"2024-09-20T18:53:43.655403+0000","last_peered":"2024-09-20T18:53:43.655403+0000","last_clean":"2024-09-20T18:53:43.655403+0000","last_became_active":"2024-09-20T18:49:49.956445+0000","last_became_peered":"2024-09-20T18:49:49.956445+0000","last_unstale":"2024-09-20T18:53:43.655403+0000","last_undegraded":"2024-09-20T18:53:43.655403+0000","last_fullsized":"2024-09-20T18:53:43.655403+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:09.436302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:09.436302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:04.774926+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"59'1","reported_seq":209,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.682185+0000","last_change":"2024-09-20T18:52:48.035578+0000","last_active":"2024-09-20T18:53:43.682185+0000","last_peered":"2024-09-20T18:53:43.682185+0000","last_clean":"2024-09-20T18:53:43.682185+0000","last_became_active":"2024-09-20T18:49:45.946171+0000","last_became_peered":"2024-09-20T18:49:45.946171+0000","last_unstale":"2024-09-20T18:53:43.682185+0000","last_undegraded":"2024-09-20T18:53:43.682185+0000","last_fullsized":"2024-09-20T18:53:43.682185+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:52:48.035532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:48.035532+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:01.343903+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":216,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.656053+0000","last_change":"2024-09-20T18:53:19.836082+0000","last_active":"2024-09-20T18:53:43.656053+0000","last_peered":"2024-09-20T18:53:43.656053+0000","last_clean":"2024-09-20T18:53:43.656053+0000","last_became_active":"2024-09-20T18:49:47.947448+0000","last_became_peered":"2024-09-20T18:49:47.947448+0000","last_unstale":"2024-09-20T18:53:43.656053+0000","last_undegraded":"2024-09-20T18:53:43.656053+0000","last_fullsized":"2024-09-20T18:53:43.656053+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:53:19.835996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:19.835996+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:39.939119+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":195,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.681798+0000","last_change":"2024-09-20T18:52:36.183932+0000","last_active":"2024-09-20T18:53:43.681798+0000","last_peered":"2024-09-20T18:53:43.681798+0000","last_clean":"2024-09-20T18:53:43.681798+0000","last_became_active":"2024-09-20T18:49:49.961188+0000","last_became_peered":"2024-09-20T18:49:49.961188+0000","last_unstale":"2024-09-20T18:53:43.681798+0000","last_undegraded":"2024-09-20T18:53:43.681798+0000","last_fullsized":"2024-09-20T18:53:43.681798+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:36.183875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:36.183875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:49.223570+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":175,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.670137+0000","last_change":"2024-09-20T18:49:51.966957+0000","last_active":"2024-09-20T18:53:43.670137+0000","last_peered":"2024-09-20T18:53:43.670137+0000","last_clean":"2024-09-20T18:53:43.670137+0000","last_became_active":"2024-09-20T18:49:51.966715+0000","last_became_peered":"2024-09-20T18:49:51.966715+0000","last_unstale":"2024-09-20T18:53:43.670137+0000","last_undegraded":"2024-09-20T18:53:43.670137+0000","last_fullsized":"2024-09-20T18:53:43.670137+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:59.013361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":195,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.681982+0000","last_change":"2024-09-20T18:52:00.970719+0000","last_active":"2024-09-20T18:53:43.681982+0000","last_peered":"2024-09-20T18:53:43.681982+0000","last_clean":"2024-09-20T18:53:43.681982+0000","last_became_active":"2024-09-20T18:49:45.944703+0000","last_became_peered":"2024-09-20T18:49:45.944703+0000","last_unstale":"2024-09-20T18:53:43.681982+0000","last_undegraded":"2024-09-20T18:53:43.681982+0000","last_fullsized":"2024-09-20T18:53:43.681982+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:00.970650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:00.970650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:07.049473+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":228,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.655947+0000","last_change":"2024-09-20T18:51:26.494194+0000","last_active":"2024-09-20T18:53:43.655947+0000","last_peered":"2024-09-20T18:53:43.655947+0000","last_clean":"2024-09-20T18:53:43.655947+0000","last_became_active":"2024-09-20T18:51:23.813399+0000","last_became_peered":"2024-09-20T18:51:23.813399+0000","last_unstale":"2024-09-20T18:53:43.655947+0000","last_undegraded":"2024-09-20T18:53:43.655947+0000","last_fullsized":"2024-09-20T18:53:43.655947+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-20T18:51:26.494118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:26.494118+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:34.803592+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":195,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.662238+0000","last_change":"2024-09-20T18:52:38.799542+0000","last_active":"2024-09-20T18:53:43.662238+0000","last_peered":"2024-09-20T18:53:43.662238+0000","last_clean":"2024-09-20T18:53:43.662238+0000","last_became_active":"2024-09-20T18:49:49.959696+0000","last_became_peered":"2024-09-20T18:49:49.959696+0000","last_unstale":"2024-09-20T18:53:43.662238+0000","last_undegraded":"2024-09-20T18:53:43.662238+0000","last_fullsized":"2024-09-20T18:53:43.662238+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:38.799487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:38.799487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:04.479508+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":191,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.681365+0000","last_change":"2024-09-20T18:53:21.273835+0000","last_active":"2024-09-20T18:53:43.681365+0000","last_peered":"2024-09-20T18:53:43.681365+0000","last_clean":"2024-09-20T18:53:43.681365+0000","last_became_active":"2024-09-20T18:49:51.970663+0000","last_became_peered":"2024-09-20T18:49:51.970663+0000","last_unstale":"2024-09-20T18:53:43.681365+0000","last_undegraded":"2024-09-20T18:53:43.681365+0000","last_fullsized":"2024-09-20T18:53:43.681365+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:21.273788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:21.273788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:31.269805+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":184,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.656777+0000","last_change":"2024-09-20T18:52:46.828811+0000","last_active":"2024-09-20T18:53:43.656777+0000","last_peered":"2024-09-20T18:53:43.656777+0000","last_clean":"2024-09-20T18:53:43.656777+0000","last_became_active":"2024-09-20T18:49:52.141170+0000","last_became_peered":"2024-09-20T18:49:52.141170+0000","last_unstale":"2024-09-20T18:53:43.656777+0000","last_undegraded":"2024-09-20T18:53:43.656777+0000","last_fullsized":"2024-09-20T18:53:43.656777+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:46.828749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:46.828749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:50.780880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":195,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.669819+0000","last_change":"2024-09-20T18:52:42.099489+0000","last_active":"2024-09-20T18:53:43.669819+0000","last_peered":"2024-09-20T18:53:43.669819+0000","last_clean":"2024-09-20T18:53:43.669819+0000","last_became_active":"2024-09-20T18:49:45.950480+0000","last_became_peered":"2024-09-20T18:49:45.950480+0000","last_unstale":"2024-09-20T18:53:43.669819+0000","last_undegraded":"2024-09-20T18:53:43.669819+0000","last_fullsized":"2024-09-20T18:53:43.669819+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:42.099418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:42.099418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:03.758400+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":226,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.667299+0000","last_change":"2024-09-20T18:53:26.762207+0000","last_active":"2024-09-20T18:53:43.667299+0000","last_peered":"2024-09-20T18:53:43.667299+0000","last_clean":"2024-09-20T18:53:43.667299+0000","last_became_active":"2024-09-20T18:49:47.950939+0000","last_became_peered":"2024-09-20T18:49:47.950939+0000","last_unstale":"2024-09-20T18:53:43.667299+0000","last_undegraded":"2024-09-20T18:53:43.667299+0000","last_fullsized":"2024-09-20T18:53:43.667299+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:53:26.762131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:26.762131+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:38.227744+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":188,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.671111+0000","last_change":"2024-09-20T18:51:17.248705+0000","last_active":"2024-09-20T18:53:43.671111+0000","last_peered":"2024-09-20T18:53:43.671111+0000","last_clean":"2024-09-20T18:53:43.671111+0000","last_became_active":"2024-09-20T18:49:49.958518+0000","last_became_peered":"2024-09-20T18:49:49.958518+0000","last_unstale":"2024-09-20T18:53:43.671111+0000","last_undegraded":"2024-09-20T18:53:43.671111+0000","last_fullsized":"2024-09-20T18:53:43.671111+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:17.248634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:17.248634+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:35.297969+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":184,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.676064+0000","last_change":"2024-09-20T18:52:03.023669+0000","last_active":"2024-09-20T18:53:43.676064+0000","last_peered":"2024-09-20T18:53:43.676064+0000","last_clean":"2024-09-20T18:53:43.676064+0000","last_became_active":"2024-09-20T18:49:52.140040+0000","last_became_peered":"2024-09-20T18:49:52.140040+0000","last_unstale":"2024-09-20T18:53:43.676064+0000","last_undegraded":"2024-09-20T18:53:43.676064+0000","last_fullsized":"2024-09-20T18:53:43.676064+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:03.023603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:03.023603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"59'1","reported_seq":210,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.656142+0000","last_change":"2024-09-20T18:53:42.840884+0000","last_active":"2024-09-20T18:53:43.656142+0000","last_peered":"2024-09-20T18:53:43.656142+0000","last_clean":"2024-09-20T18:53:43.656142+0000","last_became_active":"2024-09-20T18:49:45.940100+0000","last_became_peered":"2024-09-20T18:49:45.940100+0000","last_unstale":"2024-09-20T18:53:43.656142+0000","last_undegraded":"2024-09-20T18:53:43.656142+0000","last_fullsized":"2024-09-20T18:53:43.656142+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:53:42.840800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:42.840800+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:56.760029+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":207,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.675564+0000","last_change":"2024-09-20T18:49:47.951215+0000","last_active":"2024-09-20T18:53:43.675564+0000","last_peered":"2024-09-20T18:53:43.675564+0000","last_clean":"2024-09-20T18:53:43.675564+0000","last_became_active":"2024-09-20T18:49:47.950909+0000","last_became_peered":"2024-09-20T18:49:47.950909+0000","last_unstale":"2024-09-20T18:53:43.675564+0000","last_undegraded":"2024-09-20T18:53:43.675564+0000","last_fullsized":"2024-09-20T18:53:43.675564+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:00.096165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":196,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.656449+0000","last_change":"2024-09-20T18:53:12.820752+0000","last_active":"2024-09-20T18:53:43.656449+0000","last_peered":"2024-09-20T18:53:43.656449+0000","last_clean":"2024-09-20T18:53:43.656449+0000","last_became_active":"2024-09-20T18:49:49.959102+0000","last_became_peered":"2024-09-20T18:49:49.959102+0000","last_unstale":"2024-09-20T18:53:43.656449+0000","last_undegraded":"2024-09-20T18:53:43.656449+0000","last_fullsized":"2024-09-20T18:53:43.656449+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:12.820697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:12.820697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:29.865011+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":192,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.662234+0000","last_change":"2024-09-20T18:52:15.826819+0000","last_active":"2024-09-20T18:53:43.662234+0000","last_peered":"2024-09-20T18:53:43.662234+0000","last_clean":"2024-09-20T18:53:43.662234+0000","last_became_active":"2024-09-20T18:49:51.970058+0000","last_became_peered":"2024-09-20T18:49:51.970058+0000","last_unstale":"2024-09-20T18:53:43.662234+0000","last_undegraded":"2024-09-20T18:53:43.662234+0000","last_fullsized":"2024-09-20T18:53:43.662234+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T18:52:15.826638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:15.826638+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:28.538285+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":197,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.668474+0000","last_change":"2024-09-20T18:52:47.158384+0000","last_active":"2024-09-20T18:53:43.668474+0000","last_peered":"2024-09-20T18:53:43.668474+0000","last_clean":"2024-09-20T18:53:43.668474+0000","last_became_active":"2024-09-20T18:49:45.933078+0000","last_became_peered":"2024-09-20T18:49:45.933078+0000","last_unstale":"2024-09-20T18:53:43.668474+0000","last_undegraded":"2024-09-20T18:53:43.668474+0000","last_fullsized":"2024-09-20T18:53:43.668474+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:52:47.158260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:47.158260+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:57.719080+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":206,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.663919+0000","last_change":"2024-09-20T18:52:36.175063+0000","last_active":"2024-09-20T18:53:43.663919+0000","last_peered":"2024-09-20T18:53:43.663919+0000","last_clean":"2024-09-20T18:53:43.663919+0000","last_became_active":"2024-09-20T18:49:48.500809+0000","last_became_peered":"2024-09-20T18:49:48.500809+0000","last_unstale":"2024-09-20T18:53:43.663919+0000","last_undegraded":"2024-09-20T18:53:43.663919+0000","last_fullsized":"2024-09-20T18:53:43.663919+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:52:36.174993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:36.174993+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:57.398808+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":254,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:44.524093+0000","last_change":"2024-09-20T18:53:44.524093+0000","last_active":"2024-09-20T18:53:44.524093+0000","last_peered":"2024-09-20T18:53:44.524093+0000","last_clean":"2024-09-20T18:53:44.524093+0000","last_became_active":"2024-09-20T18:49:49.954772+0000","last_became_peered":"2024-09-20T18:49:49.954772+0000","last_unstale":"2024-09-20T18:53:44.524093+0000","last_undegraded":"2024-09-20T18:53:44.524093+0000","last_fullsized":"2024-09-20T18:53:44.524093+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:53:44.524040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:44.524040+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:03.085945+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.c","version":"147'12888","reported_seq":13107,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:45.816352+0000","last_change":"2024-09-20T18:52:16.997356+0000","last_active":"2024-09-20T18:53:45.816352+0000","last_peered":"2024-09-20T18:53:45.816352+0000","last_clean":"2024-09-20T18:53:45.816352+0000","last_became_active":"2024-09-20T18:50:49.843744+0000","last_became_peered":"2024-09-20T18:50:49.843744+0000","last_unstale":"2024-09-20T18:53:45.816352+0000","last_undegraded":"2024-09-20T18:53:45.816352+0000","last_fullsized":"2024-09-20T18:53:45.816352+0000","mapping_epoch":83,"log_start":"114'6600","ondisk_log_start":"114'6600","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"118'7088","last_scrub_stamp":"2024-09-20T18:52:16.997280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:52:16.997280+0000","objects_scrubbed":7043,"log_size":6288,"log_dups_size":0,"ondisk_log_size":6288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"queued for scrub","scrub_duration":1683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":167116800,"num_objects":2550,"num_object_clones":0,"num_object_copies":7650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2550,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12888,"num_write_kb":494016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":201,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.667785+0000","last_change":"2024-09-20T18:52:34.649470+0000","last_active":"2024-09-20T18:53:43.667785+0000","last_peered":"2024-09-20T18:53:43.667785+0000","last_clean":"2024-09-20T18:53:43.667785+0000","last_became_active":"2024-09-20T18:49:58.992302+0000","last_became_peered":"2024-09-20T18:49:58.992302+0000","last_unstale":"2024-09-20T18:53:43.667785+0000","last_undegraded":"2024-09-20T18:53:43.667785+0000","last_fullsized":"2024-09-20T18:53:43.667785+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:52:34.649398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:34.649398+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:47.951675+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":196,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.675382+0000","last_change":"2024-09-20T18:53:26.175148+0000","last_active":"2024-09-20T18:53:43.675382+0000","last_peered":"2024-09-20T18:53:43.675382+0000","last_clean":"2024-09-20T18:53:43.675382+0000","last_became_active":"2024-09-20T18:49:45.936104+0000","last_became_peered":"2024-09-20T18:49:45.936104+0000","last_unstale":"2024-09-20T18:53:43.675382+0000","last_undegraded":"2024-09-20T18:53:43.675382+0000","last_fullsized":"2024-09-20T18:53:43.675382+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:26.175083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:26.175083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:49.575901+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":247,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.656864+0000","last_change":"2024-09-20T18:51:13.395980+0000","last_active":"2024-09-20T18:53:43.656864+0000","last_peered":"2024-09-20T18:53:43.656864+0000","last_clean":"2024-09-20T18:53:43.656864+0000","last_became_active":"2024-09-20T18:49:49.960697+0000","last_became_peered":"2024-09-20T18:49:49.960697+0000","last_unstale":"2024-09-20T18:53:43.656864+0000","last_undegraded":"2024-09-20T18:53:43.656864+0000","last_fullsized":"2024-09-20T18:53:43.656864+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:51:13.395908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:13.395908+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:07.764237+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":183,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.681704+0000","last_change":"2024-09-20T18:52:43.126389+0000","last_active":"2024-09-20T18:53:43.681704+0000","last_peered":"2024-09-20T18:53:43.681704+0000","last_clean":"2024-09-20T18:53:43.681704+0000","last_became_active":"2024-09-20T18:49:51.971399+0000","last_became_peered":"2024-09-20T18:49:51.971399+0000","last_unstale":"2024-09-20T18:53:43.681704+0000","last_undegraded":"2024-09-20T18:53:43.681704+0000","last_fullsized":"2024-09-20T18:53:43.681704+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:43.126320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:43.126320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:52.243850+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"147'12684","reported_seq":12824,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:46.127044+0000","last_change":"2024-09-20T18:50:49.841111+0000","last_active":"2024-09-20T18:53:46.127044+0000","last_peered":"2024-09-20T18:53:46.127044+0000","last_clean":"2024-09-20T18:53:46.127044+0000","last_became_active":"2024-09-20T18:50:49.840819+0000","last_became_peered":"2024-09-20T18:50:49.840819+0000","last_unstale":"2024-09-20T18:53:46.127044+0000","last_undegraded":"2024-09-20T18:53:46.127044+0000","last_fullsized":"2024-09-20T18:53:46.127044+0000","mapping_epoch":83,"log_start":"124'8000","ondisk_log_start":"124'8000","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.819660+0000","objects_scrubbed":0,"log_size":4684,"log_dups_size":0,"ondisk_log_size":4684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:38.259712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":166985728,"num_objects":2548,"num_object_clones":0,"num_object_copies":7644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2548,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12684,"num_write_kb":487424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":224,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.675143+0000","last_change":"2024-09-20T18:52:42.130129+0000","last_active":"2024-09-20T18:53:43.675143+0000","last_peered":"2024-09-20T18:53:43.675143+0000","last_clean":"2024-09-20T18:53:43.675143+0000","last_became_active":"2024-09-20T18:49:48.499189+0000","last_became_peered":"2024-09-20T18:49:48.499189+0000","last_unstale":"2024-09-20T18:53:43.675143+0000","last_undegraded":"2024-09-20T18:53:43.675143+0000","last_fullsized":"2024-09-20T18:53:43.675143+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-20T18:52:42.130045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:42.130045+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:44.974510+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":188,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.676252+0000","last_change":"2024-09-20T18:49:45.940816+0000","last_active":"2024-09-20T18:53:43.676252+0000","last_peered":"2024-09-20T18:53:43.676252+0000","last_clean":"2024-09-20T18:53:43.676252+0000","last_became_active":"2024-09-20T18:49:45.940544+0000","last_became_peered":"2024-09-20T18:49:45.940544+0000","last_unstale":"2024-09-20T18:53:43.676252+0000","last_undegraded":"2024-09-20T18:53:43.676252+0000","last_fullsized":"2024-09-20T18:53:43.676252+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:56.100894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":196,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.655523+0000","last_change":"2024-09-20T18:53:09.779280+0000","last_active":"2024-09-20T18:53:43.655523+0000","last_peered":"2024-09-20T18:53:43.655523+0000","last_clean":"2024-09-20T18:53:43.655523+0000","last_became_active":"2024-09-20T18:49:49.963073+0000","last_became_peered":"2024-09-20T18:49:49.963073+0000","last_unstale":"2024-09-20T18:53:43.655523+0000","last_undegraded":"2024-09-20T18:53:43.655523+0000","last_fullsized":"2024-09-20T18:53:43.655523+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:09.779228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:09.779228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:39.433897+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":192,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.655959+0000","last_change":"2024-09-20T18:53:27.796357+0000","last_active":"2024-09-20T18:53:43.655959+0000","last_peered":"2024-09-20T18:53:43.655959+0000","last_clean":"2024-09-20T18:53:43.655959+0000","last_became_active":"2024-09-20T18:49:51.968793+0000","last_became_peered":"2024-09-20T18:49:51.968793+0000","last_unstale":"2024-09-20T18:53:43.655959+0000","last_undegraded":"2024-09-20T18:53:43.655959+0000","last_fullsized":"2024-09-20T18:53:43.655959+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:27.796289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:27.796289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:48.888717+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2","version":"147'12893","reported_seq":13033,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:46.127230+0000","last_change":"2024-09-20T18:50:49.842736+0000","last_active":"2024-09-20T18:53:46.127230+0000","last_peered":"2024-09-20T18:53:46.127230+0000","last_clean":"2024-09-20T18:53:46.127230+0000","last_became_active":"2024-09-20T18:50:49.842465+0000","last_became_peered":"2024-09-20T18:50:49.842465+0000","last_unstale":"2024-09-20T18:53:46.127230+0000","last_undegraded":"2024-09-20T18:53:46.127230+0000","last_fullsized":"2024-09-20T18:53:46.127230+0000","mapping_epoch":83,"log_start":"125'8200","ondisk_log_start":"125'8200","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.819660+0000","objects_scrubbed":0,"log_size":4693,"log_dups_size":0,"ondisk_log_size":4693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:31.286096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163512320,"num_objects":2495,"num_object_clones":0,"num_object_copies":7485,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12893,"num_write_kb":492416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":229,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:44.879133+0000","last_change":"2024-09-20T18:53:44.879133+0000","last_active":"2024-09-20T18:53:44.879133+0000","last_peered":"2024-09-20T18:53:44.879133+0000","last_clean":"2024-09-20T18:53:44.879133+0000","last_became_active":"2024-09-20T18:49:47.941895+0000","last_became_peered":"2024-09-20T18:49:47.941895+0000","last_unstale":"2024-09-20T18:53:44.879133+0000","last_undegraded":"2024-09-20T18:53:44.879133+0000","last_fullsized":"2024-09-20T18:53:44.879133+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-20T18:53:44.879048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:44.879048+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:52.038099+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":196,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.664158+0000","last_change":"2024-09-20T18:51:27.187060+0000","last_active":"2024-09-20T18:53:43.664158+0000","last_peered":"2024-09-20T18:53:43.664158+0000","last_clean":"2024-09-20T18:53:43.664158+0000","last_became_active":"2024-09-20T18:49:45.940469+0000","last_became_peered":"2024-09-20T18:49:45.940469+0000","last_unstale":"2024-09-20T18:53:43.664158+0000","last_undegraded":"2024-09-20T18:53:43.664158+0000","last_fullsized":"2024-09-20T18:53:43.664158+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:51:27.186991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:27.186991+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:53.996237+0000","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"66'8","reported_seq":253,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:44.293228+0000","last_change":"2024-09-20T18:53:44.293165+0000","last_active":"2024-09-20T18:53:44.293228+0000","last_peered":"2024-09-20T18:53:44.293228+0000","last_clean":"2024-09-20T18:53:44.293228+0000","last_became_active":"2024-09-20T18:49:49.956133+0000","last_became_peered":"2024-09-20T18:49:49.956133+0000","last_unstale":"2024-09-20T18:53:44.293228+0000","last_undegraded":"2024-09-20T18:53:44.293228+0000","last_fullsized":"2024-09-20T18:53:44.293228+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:53:44.293094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:44.293094+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:47.176037+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":191,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.662724+0000","last_change":"2024-09-20T18:52:42.880767+0000","last_active":"2024-09-20T18:53:43.662724+0000","last_peered":"2024-09-20T18:53:43.662724+0000","last_clean":"2024-09-20T18:53:43.662724+0000","last_became_active":"2024-09-20T18:49:52.142113+0000","last_became_peered":"2024-09-20T18:49:52.142113+0000","last_unstale":"2024-09-20T18:53:43.662724+0000","last_undegraded":"2024-09-20T18:53:43.662724+0000","last_fullsized":"2024-09-20T18:53:43.662724+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:42.880724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:42.880724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:04.690799+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"147'12879","reported_seq":13087,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:45.772849+0000","last_change":"2024-09-20T18:52:00.368447+0000","last_active":"2024-09-20T18:53:45.772849+0000","last_peered":"2024-09-20T18:53:45.772849+0000","last_clean":"2024-09-20T18:53:45.772849+0000","last_became_active":"2024-09-20T18:50:49.844624+0000","last_became_peered":"2024-09-20T18:50:49.844624+0000","last_unstale":"2024-09-20T18:53:45.772849+0000","last_undegraded":"2024-09-20T18:53:45.772849+0000","last_fullsized":"2024-09-20T18:53:45.772849+0000","mapping_epoch":83,"log_start":"124'8026","ondisk_log_start":"124'8026","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"111'6092","last_scrub_stamp":"2024-09-20T18:52:00.368276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:52:00.368276+0000","objects_scrubbed":6040,"log_size":4853,"log_dups_size":0,"ondisk_log_size":4853,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:40.875204+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":168099840,"num_objects":2565,"num_object_clones":0,"num_object_copies":7695,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12879,"num_write_kb":494208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":215,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.668322+0000","last_change":"2024-09-20T18:52:39.246125+0000","last_active":"2024-09-20T18:53:43.668322+0000","last_peered":"2024-09-20T18:53:43.668322+0000","last_clean":"2024-09-20T18:53:43.668322+0000","last_became_active":"2024-09-20T18:49:47.949626+0000","last_became_peered":"2024-09-20T18:49:47.949626+0000","last_unstale":"2024-09-20T18:53:43.668322+0000","last_undegraded":"2024-09-20T18:53:43.668322+0000","last_fullsized":"2024-09-20T18:53:43.668322+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:52:39.246050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:39.246050+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:07.125503+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":203,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.682267+0000","last_change":"2024-09-20T18:53:11.130341+0000","last_active":"2024-09-20T18:53:43.682267+0000","last_peered":"2024-09-20T18:53:43.682267+0000","last_clean":"2024-09-20T18:53:43.682267+0000","last_became_active":"2024-09-20T18:49:45.942496+0000","last_became_peered":"2024-09-20T18:49:45.942496+0000","last_unstale":"2024-09-20T18:53:43.682267+0000","last_undegraded":"2024-09-20T18:53:43.682267+0000","last_fullsized":"2024-09-20T18:53:43.682267+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:11.130274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:11.130274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:17.935310+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":244,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.669533+0000","last_change":"2024-09-20T18:51:16.088407+0000","last_active":"2024-09-20T18:53:43.669533+0000","last_peered":"2024-09-20T18:53:43.669533+0000","last_clean":"2024-09-20T18:53:43.669533+0000","last_became_active":"2024-09-20T18:49:49.959365+0000","last_became_peered":"2024-09-20T18:49:49.959365+0000","last_unstale":"2024-09-20T18:53:43.669533+0000","last_undegraded":"2024-09-20T18:53:43.669533+0000","last_fullsized":"2024-09-20T18:53:43.669533+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:51:16.088343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:16.088343+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:35.866980+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":175,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.670292+0000","last_change":"2024-09-20T18:49:51.965733+0000","last_active":"2024-09-20T18:53:43.670292+0000","last_peered":"2024-09-20T18:53:43.670292+0000","last_clean":"2024-09-20T18:53:43.670292+0000","last_became_active":"2024-09-20T18:49:51.964780+0000","last_became_peered":"2024-09-20T18:49:51.964780+0000","last_unstale":"2024-09-20T18:53:43.670292+0000","last_undegraded":"2024-09-20T18:53:43.670292+0000","last_fullsized":"2024-09-20T18:53:43.670292+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:54.992353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.0","version":"147'12969","reported_seq":13184,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:45.790480+0000","last_change":"2024-09-20T18:52:12.840296+0000","last_active":"2024-09-20T18:53:45.790480+0000","last_peered":"2024-09-20T18:53:45.790480+0000","last_clean":"2024-09-20T18:53:45.790480+0000","last_became_active":"2024-09-20T18:50:49.841019+0000","last_became_peered":"2024-09-20T18:50:49.841019+0000","last_unstale":"2024-09-20T18:53:45.790480+0000","last_undegraded":"2024-09-20T18:53:45.790480+0000","last_fullsized":"2024-09-20T18:53:45.790480+0000","mapping_epoch":83,"log_start":"118'7100","ondisk_log_start":"118'7100","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"116'6885","last_scrub_stamp":"2024-09-20T18:52:12.840191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:52:12.840191+0000","objects_scrubbed":6837,"log_size":5869,"log_dups_size":0,"ondisk_log_size":5869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"queued for scrub","scrub_duration":1721,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":167706624,"num_objects":2559,"num_object_clones":0,"num_object_copies":7677,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2559,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12969,"num_write_kb":496896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":227,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.669659+0000","last_change":"2024-09-20T18:52:49.043282+0000","last_active":"2024-09-20T18:53:43.669659+0000","last_peered":"2024-09-20T18:53:43.669659+0000","last_clean":"2024-09-20T18:53:43.669659+0000","last_became_active":"2024-09-20T18:49:48.497736+0000","last_became_peered":"2024-09-20T18:49:48.497736+0000","last_unstale":"2024-09-20T18:53:43.669659+0000","last_undegraded":"2024-09-20T18:53:43.669659+0000","last_fullsized":"2024-09-20T18:53:43.669659+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-20T18:52:49.043189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:49.043189+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:54.819771+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":188,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.671637+0000","last_change":"2024-09-20T18:49:45.942589+0000","last_active":"2024-09-20T18:53:43.671637+0000","last_peered":"2024-09-20T18:53:43.671637+0000","last_clean":"2024-09-20T18:53:43.671637+0000","last_became_active":"2024-09-20T18:49:45.942253+0000","last_became_peered":"2024-09-20T18:49:45.942253+0000","last_unstale":"2024-09-20T18:53:43.671637+0000","last_undegraded":"2024-09-20T18:53:43.671637+0000","last_fullsized":"2024-09-20T18:53:43.671637+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:56.093800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":195,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.672666+0000","last_change":"2024-09-20T18:53:12.347123+0000","last_active":"2024-09-20T18:53:43.672666+0000","last_peered":"2024-09-20T18:53:43.672666+0000","last_clean":"2024-09-20T18:53:43.672666+0000","last_became_active":"2024-09-20T18:49:49.954299+0000","last_became_peered":"2024-09-20T18:49:49.954299+0000","last_unstale":"2024-09-20T18:53:43.672666+0000","last_undegraded":"2024-09-20T18:53:43.672666+0000","last_fullsized":"2024-09-20T18:53:43.672666+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:12.347044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:12.347044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:41.912430+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":192,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.668260+0000","last_change":"2024-09-20T18:53:25.362907+0000","last_active":"2024-09-20T18:53:43.668260+0000","last_peered":"2024-09-20T18:53:43.668260+0000","last_clean":"2024-09-20T18:53:43.668260+0000","last_became_active":"2024-09-20T18:49:52.139981+0000","last_became_peered":"2024-09-20T18:49:52.139981+0000","last_unstale":"2024-09-20T18:53:43.668260+0000","last_undegraded":"2024-09-20T18:53:43.668260+0000","last_fullsized":"2024-09-20T18:53:43.668260+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:25.362828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:25.362828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:55.107515+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"147'12647","reported_seq":12855,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:45.816511+0000","last_change":"2024-09-20T18:52:03.401113+0000","last_active":"2024-09-20T18:53:45.816511+0000","last_peered":"2024-09-20T18:53:45.816511+0000","last_clean":"2024-09-20T18:53:45.816511+0000","last_became_active":"2024-09-20T18:50:49.842473+0000","last_became_peered":"2024-09-20T18:50:49.842473+0000","last_unstale":"2024-09-20T18:53:45.816511+0000","last_undegraded":"2024-09-20T18:53:45.816511+0000","last_fullsized":"2024-09-20T18:53:45.816511+0000","mapping_epoch":83,"log_start":"114'6300","ondisk_log_start":"114'6300","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"112'6142","last_scrub_stamp":"2024-09-20T18:52:03.400886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:52:03.400886+0000","objects_scrubbed":6079,"log_size":6347,"log_dups_size":0,"ondisk_log_size":6347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:41.532845+0000","scrub_duration":2059,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":162594816,"num_objects":2481,"num_object_clones":0,"num_object_copies":7443,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2481,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12647,"num_write_kb":484096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"66'28","reported_seq":288,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.671477+0000","last_change":"2024-09-20T18:51:57.293659+0000","last_active":"2024-09-20T18:53:43.671477+0000","last_peered":"2024-09-20T18:53:43.671477+0000","last_clean":"2024-09-20T18:53:43.671477+0000","last_became_active":"2024-09-20T18:49:47.955335+0000","last_became_peered":"2024-09-20T18:49:47.955335+0000","last_unstale":"2024-09-20T18:53:43.671477+0000","last_undegraded":"2024-09-20T18:53:43.671477+0000","last_fullsized":"2024-09-20T18:53:43.671477+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'28","last_scrub_stamp":"2024-09-20T18:51:57.293574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:57.293574+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:06.036152+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":188,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.656246+0000","last_change":"2024-09-20T18:49:45.945275+0000","last_active":"2024-09-20T18:53:43.656246+0000","last_peered":"2024-09-20T18:53:43.656246+0000","last_clean":"2024-09-20T18:53:43.656246+0000","last_became_active":"2024-09-20T18:49:45.944992+0000","last_became_peered":"2024-09-20T18:49:45.944992+0000","last_unstale":"2024-09-20T18:53:43.656246+0000","last_undegraded":"2024-09-20T18:53:43.656246+0000","last_fullsized":"2024-09-20T18:53:43.656246+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:54.777986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":213,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.681829+0000","last_change":"2024-09-20T18:52:15.112959+0000","last_active":"2024-09-20T18:53:43.681829+0000","last_peered":"2024-09-20T18:53:43.681829+0000","last_clean":"2024-09-20T18:53:43.681829+0000","last_became_active":"2024-09-20T18:49:44.936027+0000","last_became_peered":"2024-09-20T18:49:44.936027+0000","last_unstale":"2024-09-20T18:53:43.681829+0000","last_undegraded":"2024-09-20T18:53:43.681829+0000","last_fullsized":"2024-09-20T18:53:43.681829+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-20T18:52:15.112895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:52:15.112895+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0012781159999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":195,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.665349+0000","last_change":"2024-09-20T18:53:12.897519+0000","last_active":"2024-09-20T18:53:43.665349+0000","last_peered":"2024-09-20T18:53:43.665349+0000","last_clean":"2024-09-20T18:53:43.665349+0000","last_became_active":"2024-09-20T18:49:49.953971+0000","last_became_peered":"2024-09-20T18:49:49.953971+0000","last_unstale":"2024-09-20T18:53:43.665349+0000","last_undegraded":"2024-09-20T18:53:43.665349+0000","last_fullsized":"2024-09-20T18:53:43.665349+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:12.897051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:12.897051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:39.168470+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":176,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.675745+0000","last_change":"2024-09-20T18:49:51.969702+0000","last_active":"2024-09-20T18:53:43.675745+0000","last_peered":"2024-09-20T18:53:43.675745+0000","last_clean":"2024-09-20T18:53:43.675745+0000","last_became_active":"2024-09-20T18:49:51.969400+0000","last_became_peered":"2024-09-20T18:49:51.969400+0000","last_unstale":"2024-09-20T18:53:43.675745+0000","last_undegraded":"2024-09-20T18:53:43.675745+0000","last_fullsized":"2024-09-20T18:53:43.675745+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:36.285843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"147'12729","reported_seq":12916,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:46.169976+0000","last_change":"2024-09-20T18:53:23.208574+0000","last_active":"2024-09-20T18:53:46.169976+0000","last_peered":"2024-09-20T18:53:46.169976+0000","last_clean":"2024-09-20T18:53:46.169976+0000","last_became_active":"2024-09-20T18:50:49.843321+0000","last_became_peered":"2024-09-20T18:50:49.843321+0000","last_unstale":"2024-09-20T18:53:46.169976+0000","last_undegraded":"2024-09-20T18:53:46.169976+0000","last_fullsized":"2024-09-20T18:53:46.169976+0000","mapping_epoch":83,"log_start":"119'7100","ondisk_log_start":"119'7100","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"140'11334","last_scrub_stamp":"2024-09-20T18:53:23.208497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:53:23.208497+0000","objects_scrubbed":3959,"log_size":5629,"log_dups_size":0,"ondisk_log_size":5629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:48.313048+0000","scrub_duration":1357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":165216256,"num_objects":2521,"num_object_clones":0,"num_object_copies":7563,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12729,"num_write_kb":488000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":219,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.668577+0000","last_change":"2024-09-20T18:52:37.246051+0000","last_active":"2024-09-20T18:53:43.668577+0000","last_peered":"2024-09-20T18:53:43.668577+0000","last_clean":"2024-09-20T18:53:43.668577+0000","last_became_active":"2024-09-20T18:49:47.950904+0000","last_became_peered":"2024-09-20T18:49:47.950904+0000","last_unstale":"2024-09-20T18:53:43.668577+0000","last_undegraded":"2024-09-20T18:53:43.668577+0000","last_fullsized":"2024-09-20T18:53:43.668577+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-20T18:52:37.245970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:37.245970+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:46.997532+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":204,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.670010+0000","last_change":"2024-09-20T18:53:19.278005+0000","last_active":"2024-09-20T18:53:43.670010+0000","last_peered":"2024-09-20T18:53:43.670010+0000","last_clean":"2024-09-20T18:53:43.670010+0000","last_became_active":"2024-09-20T18:49:45.939884+0000","last_became_peered":"2024-09-20T18:49:45.939884+0000","last_unstale":"2024-09-20T18:53:43.670010+0000","last_undegraded":"2024-09-20T18:53:43.670010+0000","last_fullsized":"2024-09-20T18:53:43.670010+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:19.277650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:19.277650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:22.181893+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":206,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.667584+0000","last_change":"2024-09-20T18:52:07.718766+0000","last_active":"2024-09-20T18:53:43.667584+0000","last_peered":"2024-09-20T18:53:43.667584+0000","last_clean":"2024-09-20T18:53:43.667584+0000","last_became_active":"2024-09-20T18:49:44.937135+0000","last_became_peered":"2024-09-20T18:49:44.937135+0000","last_unstale":"2024-09-20T18:53:43.667584+0000","last_undegraded":"2024-09-20T18:53:43.667584+0000","last_fullsized":"2024-09-20T18:53:43.667584+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:52:07.718646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:52:07.718646+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:37.805116+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.00095558800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":189,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.670958+0000","last_change":"2024-09-20T18:51:59.796450+0000","last_active":"2024-09-20T18:53:43.670958+0000","last_peered":"2024-09-20T18:53:43.670958+0000","last_clean":"2024-09-20T18:53:43.670958+0000","last_became_active":"2024-09-20T18:49:49.954424+0000","last_became_peered":"2024-09-20T18:49:49.954424+0000","last_unstale":"2024-09-20T18:53:43.670958+0000","last_undegraded":"2024-09-20T18:53:43.670958+0000","last_fullsized":"2024-09-20T18:53:43.670958+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:59.796383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:59.796383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:03.718427+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":183,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.663096+0000","last_change":"2024-09-20T18:52:39.814324+0000","last_active":"2024-09-20T18:53:43.663096+0000","last_peered":"2024-09-20T18:53:43.663096+0000","last_clean":"2024-09-20T18:53:43.663096+0000","last_became_active":"2024-09-20T18:49:52.139789+0000","last_became_peered":"2024-09-20T18:49:52.139789+0000","last_unstale":"2024-09-20T18:53:43.663096+0000","last_undegraded":"2024-09-20T18:53:43.663096+0000","last_fullsized":"2024-09-20T18:53:43.663096+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:39.814259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:39.814259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:55.444061+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"147'12942","reported_seq":13204,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:45.788654+0000","last_change":"2024-09-20T18:53:24.693140+0000","last_active":"2024-09-20T18:53:45.788654+0000","last_peered":"2024-09-20T18:53:45.788654+0000","last_clean":"2024-09-20T18:53:45.788654+0000","last_became_active":"2024-09-20T18:50:49.846124+0000","last_became_peered":"2024-09-20T18:50:49.846124+0000","last_unstale":"2024-09-20T18:53:45.788654+0000","last_undegraded":"2024-09-20T18:53:45.788654+0000","last_fullsized":"2024-09-20T18:53:45.788654+0000","mapping_epoch":83,"log_start":"122'7700","ondisk_log_start":"122'7700","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"140'11593","last_scrub_stamp":"2024-09-20T18:53:24.692917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:53:24.692917+0000","objects_scrubbed":4004,"log_size":5242,"log_dups_size":0,"ondisk_log_size":5242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:35.585228+0000","scrub_duration":1122,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":171048960,"num_objects":2610,"num_object_clones":0,"num_object_copies":7830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2610,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12942,"num_write_kb":497664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":217,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.664002+0000","last_change":"2024-09-20T18:52:43.089145+0000","last_active":"2024-09-20T18:53:43.664002+0000","last_peered":"2024-09-20T18:53:43.664002+0000","last_clean":"2024-09-20T18:53:43.664002+0000","last_became_active":"2024-09-20T18:49:47.949230+0000","last_became_peered":"2024-09-20T18:49:47.949230+0000","last_unstale":"2024-09-20T18:53:43.664002+0000","last_undegraded":"2024-09-20T18:53:43.664002+0000","last_fullsized":"2024-09-20T18:53:43.664002+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:52:43.089065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:43.089065+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:47.134487+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":203,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.665567+0000","last_change":"2024-09-20T18:53:09.891235+0000","last_active":"2024-09-20T18:53:43.665567+0000","last_peered":"2024-09-20T18:53:43.665567+0000","last_clean":"2024-09-20T18:53:43.665567+0000","last_became_active":"2024-09-20T18:49:45.939197+0000","last_became_peered":"2024-09-20T18:49:45.939197+0000","last_unstale":"2024-09-20T18:53:43.665567+0000","last_undegraded":"2024-09-20T18:53:43.665567+0000","last_fullsized":"2024-09-20T18:53:43.665567+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:09.890987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:09.890987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:17.075156+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":450,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:45.579294+0000","last_change":"2024-09-20T18:53:22.616236+0000","last_active":"2024-09-20T18:53:45.579294+0000","last_peered":"2024-09-20T18:53:45.579294+0000","last_clean":"2024-09-20T18:53:45.579294+0000","last_became_active":"2024-09-20T18:49:44.936819+0000","last_became_peered":"2024-09-20T18:49:44.936819+0000","last_unstale":"2024-09-20T18:53:45.579294+0000","last_undegraded":"2024-09-20T18:53:45.579294+0000","last_fullsized":"2024-09-20T18:53:45.579294+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-20T18:53:22.616166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:53:22.616166+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:25.872696+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0010996199999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":235,"num_read_kb":230,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":187,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.682482+0000","last_change":"2024-09-20T18:52:02.981845+0000","last_active":"2024-09-20T18:53:43.682482+0000","last_peered":"2024-09-20T18:53:43.682482+0000","last_clean":"2024-09-20T18:53:43.682482+0000","last_became_active":"2024-09-20T18:49:49.961066+0000","last_became_peered":"2024-09-20T18:49:49.961066+0000","last_unstale":"2024-09-20T18:53:43.682482+0000","last_undegraded":"2024-09-20T18:53:43.682482+0000","last_fullsized":"2024-09-20T18:53:43.682482+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:02.981762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:02.981762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":184,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.671211+0000","last_change":"2024-09-20T18:52:45.158726+0000","last_active":"2024-09-20T18:53:43.671211+0000","last_peered":"2024-09-20T18:53:43.671211+0000","last_clean":"2024-09-20T18:53:43.671211+0000","last_became_active":"2024-09-20T18:49:51.970343+0000","last_became_peered":"2024-09-20T18:49:51.970343+0000","last_unstale":"2024-09-20T18:53:43.671211+0000","last_undegraded":"2024-09-20T18:53:43.671211+0000","last_fullsized":"2024-09-20T18:53:43.671211+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:45.158601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:45.158601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:47.806684+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"147'12924","reported_seq":13140,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:46.162899+0000","last_change":"2024-09-20T18:52:13.298242+0000","last_active":"2024-09-20T18:53:46.162899+0000","last_peered":"2024-09-20T18:53:46.162899+0000","last_clean":"2024-09-20T18:53:46.162899+0000","last_became_active":"2024-09-20T18:50:49.842310+0000","last_became_peered":"2024-09-20T18:50:49.842310+0000","last_unstale":"2024-09-20T18:53:46.162899+0000","last_undegraded":"2024-09-20T18:53:46.162899+0000","last_fullsized":"2024-09-20T18:53:46.162899+0000","mapping_epoch":83,"log_start":"120'7300","ondisk_log_start":"120'7300","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"116'6867","last_scrub_stamp":"2024-09-20T18:52:13.297990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:52:13.297990+0000","objects_scrubbed":6775,"log_size":5624,"log_dups_size":0,"ondisk_log_size":5624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:40.850770+0000","scrub_duration":3018,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":161873920,"num_objects":2470,"num_object_clones":0,"num_object_copies":7410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2470,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12924,"num_write_kb":492608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":253,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.673225+0000","last_change":"2024-09-20T18:49:48.498345+0000","last_active":"2024-09-20T18:53:43.673225+0000","last_peered":"2024-09-20T18:53:43.673225+0000","last_clean":"2024-09-20T18:53:43.673225+0000","last_became_active":"2024-09-20T18:49:48.497936+0000","last_became_peered":"2024-09-20T18:49:48.497936+0000","last_unstale":"2024-09-20T18:53:43.673225+0000","last_undegraded":"2024-09-20T18:53:43.673225+0000","last_fullsized":"2024-09-20T18:53:43.673225+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:57.367087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":204,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.676084+0000","last_change":"2024-09-20T18:53:13.145127+0000","last_active":"2024-09-20T18:53:43.676084+0000","last_peered":"2024-09-20T18:53:43.676084+0000","last_clean":"2024-09-20T18:53:43.676084+0000","last_became_active":"2024-09-20T18:49:45.941136+0000","last_became_peered":"2024-09-20T18:49:45.941136+0000","last_unstale":"2024-09-20T18:53:43.676084+0000","last_undegraded":"2024-09-20T18:53:43.676084+0000","last_fullsized":"2024-09-20T18:53:43.676084+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:13.144896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:13.144896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:18.711819+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":243,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.662461+0000","last_change":"2024-09-20T18:52:11.887878+0000","last_active":"2024-09-20T18:53:43.662461+0000","last_peered":"2024-09-20T18:53:43.662461+0000","last_clean":"2024-09-20T18:53:43.662461+0000","last_became_active":"2024-09-20T18:49:36.545306+0000","last_became_peered":"2024-09-20T18:49:36.545306+0000","last_unstale":"2024-09-20T18:53:43.662461+0000","last_undegraded":"2024-09-20T18:53:43.662461+0000","last_fullsized":"2024-09-20T18:53:43.662461+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T18:52:11.887733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T18:52:11.887733+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:32.438028+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":187,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.662692+0000","last_change":"2024-09-20T18:51:22.852942+0000","last_active":"2024-09-20T18:53:43.662692+0000","last_peered":"2024-09-20T18:53:43.662692+0000","last_clean":"2024-09-20T18:53:43.662692+0000","last_became_active":"2024-09-20T18:49:49.956128+0000","last_became_peered":"2024-09-20T18:49:49.956128+0000","last_unstale":"2024-09-20T18:53:43.662692+0000","last_undegraded":"2024-09-20T18:53:43.662692+0000","last_fullsized":"2024-09-20T18:53:43.662692+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:22.852869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:22.852869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:37.516435+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":191,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:45.306565+0000","last_change":"2024-09-20T18:53:45.306565+0000","last_active":"2024-09-20T18:53:45.306565+0000","last_peered":"2024-09-20T18:53:45.306565+0000","last_clean":"2024-09-20T18:53:45.306565+0000","last_became_active":"2024-09-20T18:49:51.971332+0000","last_became_peered":"2024-09-20T18:49:51.971332+0000","last_unstale":"2024-09-20T18:53:45.306565+0000","last_undegraded":"2024-09-20T18:53:45.306565+0000","last_fullsized":"2024-09-20T18:53:45.306565+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:45.306514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:45.306514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:10.093423+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"147'12788","reported_seq":13063,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:45.820965+0000","last_change":"2024-09-20T18:52:51.095285+0000","last_active":"2024-09-20T18:53:45.820965+0000","last_peered":"2024-09-20T18:53:45.820965+0000","last_clean":"2024-09-20T18:53:45.820965+0000","last_became_active":"2024-09-20T18:50:49.842483+0000","last_became_peered":"2024-09-20T18:50:49.842483+0000","last_unstale":"2024-09-20T18:53:45.820965+0000","last_undegraded":"2024-09-20T18:53:45.820965+0000","last_fullsized":"2024-09-20T18:53:45.820965+0000","mapping_epoch":83,"log_start":"114'6500","ondisk_log_start":"114'6500","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"131'9234","last_scrub_stamp":"2024-09-20T18:52:51.095167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:52:51.095167+0000","objects_scrubbed":6083,"log_size":6288,"log_dups_size":0,"ondisk_log_size":6288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:09.470192+0000","scrub_duration":1756,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":161415192,"num_objects":2464,"num_object_clones":0,"num_object_copies":7392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2464,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12788,"num_write_kb":488001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":211,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.656950+0000","last_change":"2024-09-20T18:51:14.397996+0000","last_active":"2024-09-20T18:53:43.656950+0000","last_peered":"2024-09-20T18:53:43.656950+0000","last_clean":"2024-09-20T18:53:43.656950+0000","last_became_active":"2024-09-20T18:49:47.950073+0000","last_became_peered":"2024-09-20T18:49:47.950073+0000","last_unstale":"2024-09-20T18:53:43.656950+0000","last_undegraded":"2024-09-20T18:53:43.656950+0000","last_fullsized":"2024-09-20T18:53:43.656950+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:51:14.397930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.397930+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:32.825060+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":196,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.655744+0000","last_change":"2024-09-20T18:51:18.410601+0000","last_active":"2024-09-20T18:53:43.655744+0000","last_peered":"2024-09-20T18:53:43.655744+0000","last_clean":"2024-09-20T18:53:43.655744+0000","last_became_active":"2024-09-20T18:49:45.944579+0000","last_became_peered":"2024-09-20T18:49:45.944579+0000","last_unstale":"2024-09-20T18:53:43.655744+0000","last_undegraded":"2024-09-20T18:53:43.655744+0000","last_fullsized":"2024-09-20T18:53:43.655744+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:18.410447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:18.410447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:29.855147+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":195,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.682017+0000","last_change":"2024-09-20T18:52:38.128487+0000","last_active":"2024-09-20T18:53:43.682017+0000","last_peered":"2024-09-20T18:53:43.682017+0000","last_clean":"2024-09-20T18:53:43.682017+0000","last_became_active":"2024-09-20T18:49:49.954391+0000","last_became_peered":"2024-09-20T18:49:49.954391+0000","last_unstale":"2024-09-20T18:53:43.682017+0000","last_undegraded":"2024-09-20T18:53:43.682017+0000","last_fullsized":"2024-09-20T18:53:43.682017+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:38.128438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:38.128438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:55.529118+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":184,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.675383+0000","last_change":"2024-09-20T18:52:05.021228+0000","last_active":"2024-09-20T18:53:43.675383+0000","last_peered":"2024-09-20T18:53:43.675383+0000","last_clean":"2024-09-20T18:53:43.675383+0000","last_became_active":"2024-09-20T18:49:52.139959+0000","last_became_peered":"2024-09-20T18:49:52.139959+0000","last_unstale":"2024-09-20T18:53:43.675383+0000","last_undegraded":"2024-09-20T18:53:43.675383+0000","last_fullsized":"2024-09-20T18:53:43.675383+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:05.021071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.021071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"147'12706","reported_seq":12897,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:45.978729+0000","last_change":"2024-09-20T18:53:22.451111+0000","last_active":"2024-09-20T18:53:45.978729+0000","last_peered":"2024-09-20T18:53:45.978729+0000","last_clean":"2024-09-20T18:53:45.978729+0000","last_became_active":"2024-09-20T18:50:49.841683+0000","last_became_peered":"2024-09-20T18:50:49.841683+0000","last_unstale":"2024-09-20T18:53:45.978729+0000","last_undegraded":"2024-09-20T18:53:45.978729+0000","last_fullsized":"2024-09-20T18:53:45.978729+0000","mapping_epoch":83,"log_start":"111'5800","ondisk_log_start":"111'5800","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'11259","last_scrub_stamp":"2024-09-20T18:53:22.450961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:53:22.450961+0000","objects_scrubbed":4058,"log_size":6906,"log_dups_size":0,"ondisk_log_size":6906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:26.478768+0000","scrub_duration":1718,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":167247872,"num_objects":2552,"num_object_clones":0,"num_object_copies":7656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12706,"num_write_kb":488256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":228,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.656951+0000","last_change":"2024-09-20T18:52:02.944536+0000","last_active":"2024-09-20T18:53:43.656951+0000","last_peered":"2024-09-20T18:53:43.656951+0000","last_clean":"2024-09-20T18:53:43.656951+0000","last_became_active":"2024-09-20T18:49:47.947636+0000","last_became_peered":"2024-09-20T18:49:47.947636+0000","last_unstale":"2024-09-20T18:53:43.656951+0000","last_undegraded":"2024-09-20T18:53:43.656951+0000","last_fullsized":"2024-09-20T18:53:43.656951+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-20T18:52:02.944434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:02.944434+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:30.835162+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":217,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.672805+0000","last_change":"2024-09-20T18:53:23.472809+0000","last_active":"2024-09-20T18:53:43.672805+0000","last_peered":"2024-09-20T18:53:43.672805+0000","last_clean":"2024-09-20T18:53:43.672805+0000","last_became_active":"2024-09-20T18:51:55.596689+0000","last_became_peered":"2024-09-20T18:51:55.596689+0000","last_unstale":"2024-09-20T18:53:43.672805+0000","last_undegraded":"2024-09-20T18:53:43.672805+0000","last_fullsized":"2024-09-20T18:53:43.672805+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:23.472757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:23.472757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:39.082278+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":188,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.671303+0000","last_change":"2024-09-20T18:52:49.137080+0000","last_active":"2024-09-20T18:53:43.671303+0000","last_peered":"2024-09-20T18:53:43.671303+0000","last_clean":"2024-09-20T18:53:43.671303+0000","last_became_active":"2024-09-20T18:49:49.959160+0000","last_became_peered":"2024-09-20T18:49:49.959160+0000","last_unstale":"2024-09-20T18:53:43.671303+0000","last_undegraded":"2024-09-20T18:53:43.671303+0000","last_fullsized":"2024-09-20T18:53:43.671303+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:49.137023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:49.137023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:07.149566+0000","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":193,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.667920+0000","last_change":"2024-09-20T18:52:35.635897+0000","last_active":"2024-09-20T18:53:43.667920+0000","last_peered":"2024-09-20T18:53:43.667920+0000","last_clean":"2024-09-20T18:53:43.667920+0000","last_became_active":"2024-09-20T18:49:51.970108+0000","last_became_peered":"2024-09-20T18:49:51.970108+0000","last_unstale":"2024-09-20T18:53:43.667920+0000","last_undegraded":"2024-09-20T18:53:43.667920+0000","last_fullsized":"2024-09-20T18:53:43.667920+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:35.635818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:35.635818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:56.643452+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"147'12945","reported_seq":13194,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:46.071858+0000","last_change":"2024-09-20T18:52:52.583857+0000","last_active":"2024-09-20T18:53:46.071858+0000","last_peered":"2024-09-20T18:53:46.071858+0000","last_clean":"2024-09-20T18:53:46.071858+0000","last_became_active":"2024-09-20T18:50:49.841628+0000","last_became_peered":"2024-09-20T18:50:49.841628+0000","last_unstale":"2024-09-20T18:53:46.071858+0000","last_undegraded":"2024-09-20T18:53:46.071858+0000","last_fullsized":"2024-09-20T18:53:46.071858+0000","mapping_epoch":83,"log_start":"119'7300","ondisk_log_start":"119'7300","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"131'9412","last_scrub_stamp":"2024-09-20T18:52:52.583713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:52:52.583713+0000","objects_scrubbed":6144,"log_size":5645,"log_dups_size":0,"ondisk_log_size":5645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:55.820131+0000","scrub_duration":2542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":165871616,"num_objects":2531,"num_object_clones":0,"num_object_copies":7593,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2531,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12945,"num_write_kb":495232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":207,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.656523+0000","last_change":"2024-09-20T18:52:52.775966+0000","last_active":"2024-09-20T18:53:43.656523+0000","last_peered":"2024-09-20T18:53:43.656523+0000","last_clean":"2024-09-20T18:53:43.656523+0000","last_became_active":"2024-09-20T18:49:47.947126+0000","last_became_peered":"2024-09-20T18:49:47.947126+0000","last_unstale":"2024-09-20T18:53:43.656523+0000","last_undegraded":"2024-09-20T18:53:43.656523+0000","last_fullsized":"2024-09-20T18:53:43.656523+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:52:52.775889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:52.775889+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:14.126093+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":204,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.675759+0000","last_change":"2024-09-20T18:52:46.005172+0000","last_active":"2024-09-20T18:53:43.675759+0000","last_peered":"2024-09-20T18:53:43.675759+0000","last_clean":"2024-09-20T18:53:43.675759+0000","last_became_active":"2024-09-20T18:49:45.937264+0000","last_became_peered":"2024-09-20T18:49:45.937264+0000","last_unstale":"2024-09-20T18:53:43.675759+0000","last_undegraded":"2024-09-20T18:53:43.675759+0000","last_fullsized":"2024-09-20T18:53:43.675759+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:46.005109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:46.005109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:04.505427+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":253,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.661908+0000","last_change":"2024-09-20T18:52:40.647833+0000","last_active":"2024-09-20T18:53:43.661908+0000","last_peered":"2024-09-20T18:53:43.661908+0000","last_clean":"2024-09-20T18:53:43.661908+0000","last_became_active":"2024-09-20T18:49:49.957005+0000","last_became_peered":"2024-09-20T18:49:49.957005+0000","last_unstale":"2024-09-20T18:53:43.661908+0000","last_undegraded":"2024-09-20T18:53:43.661908+0000","last_fullsized":"2024-09-20T18:53:43.661908+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:52:40.647761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:40.647761+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:03.992292+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":184,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.656691+0000","last_change":"2024-09-20T18:52:10.024610+0000","last_active":"2024-09-20T18:53:43.656691+0000","last_peered":"2024-09-20T18:53:43.656691+0000","last_clean":"2024-09-20T18:53:43.656691+0000","last_became_active":"2024-09-20T18:49:51.969355+0000","last_became_peered":"2024-09-20T18:49:51.969355+0000","last_unstale":"2024-09-20T18:53:43.656691+0000","last_undegraded":"2024-09-20T18:53:43.656691+0000","last_fullsized":"2024-09-20T18:53:43.656691+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:10.024557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:10.024557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:38.850775+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.8","version":"147'12875","reported_seq":13093,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:45.917273+0000","last_change":"2024-09-20T18:52:16.931599+0000","last_active":"2024-09-20T18:53:45.917273+0000","last_peered":"2024-09-20T18:53:45.917273+0000","last_clean":"2024-09-20T18:53:45.917273+0000","last_became_active":"2024-09-20T18:50:49.844028+0000","last_became_peered":"2024-09-20T18:50:49.844028+0000","last_unstale":"2024-09-20T18:53:45.917273+0000","last_undegraded":"2024-09-20T18:53:45.917273+0000","last_fullsized":"2024-09-20T18:53:45.917273+0000","mapping_epoch":83,"log_start":"111'5980","ondisk_log_start":"111'5980","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"118'7065","last_scrub_stamp":"2024-09-20T18:52:16.931418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:52:16.931418+0000","objects_scrubbed":7008,"log_size":6895,"log_dups_size":0,"ondisk_log_size":6895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:31.738778+0000","scrub_duration":2274,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":167444480,"num_objects":2555,"num_object_clones":0,"num_object_copies":7665,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2555,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12875,"num_write_kb":493760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":215,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.669579+0000","last_change":"2024-09-20T18:53:17.863709+0000","last_active":"2024-09-20T18:53:43.669579+0000","last_peered":"2024-09-20T18:53:43.669579+0000","last_clean":"2024-09-20T18:53:43.669579+0000","last_became_active":"2024-09-20T18:49:47.946008+0000","last_became_peered":"2024-09-20T18:49:47.946008+0000","last_unstale":"2024-09-20T18:53:43.669579+0000","last_undegraded":"2024-09-20T18:53:43.669579+0000","last_fullsized":"2024-09-20T18:53:43.669579+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:53:17.863623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:17.863623+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:35.672196+0000","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":203,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.682089+0000","last_change":"2024-09-20T18:52:17.120051+0000","last_active":"2024-09-20T18:53:43.682089+0000","last_peered":"2024-09-20T18:53:43.682089+0000","last_clean":"2024-09-20T18:53:43.682089+0000","last_became_active":"2024-09-20T18:49:45.941406+0000","last_became_peered":"2024-09-20T18:49:45.941406+0000","last_unstale":"2024-09-20T18:53:43.682089+0000","last_undegraded":"2024-09-20T18:53:43.682089+0000","last_fullsized":"2024-09-20T18:53:43.682089+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:17.119986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:17.119986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":197,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.672654+0000","last_change":"2024-09-20T18:53:27.722068+0000","last_active":"2024-09-20T18:53:43.672654+0000","last_peered":"2024-09-20T18:53:43.672654+0000","last_clean":"2024-09-20T18:53:43.672654+0000","last_became_active":"2024-09-20T18:49:49.956767+0000","last_became_peered":"2024-09-20T18:49:49.956767+0000","last_unstale":"2024-09-20T18:53:43.672654+0000","last_undegraded":"2024-09-20T18:53:43.672654+0000","last_fullsized":"2024-09-20T18:53:43.672654+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:27.721940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:27.721940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:42.447352+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":183,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.665494+0000","last_change":"2024-09-20T18:52:46.004013+0000","last_active":"2024-09-20T18:53:43.665494+0000","last_peered":"2024-09-20T18:53:43.665494+0000","last_clean":"2024-09-20T18:53:43.665494+0000","last_became_active":"2024-09-20T18:49:51.964098+0000","last_became_peered":"2024-09-20T18:49:51.964098+0000","last_unstale":"2024-09-20T18:53:43.665494+0000","last_undegraded":"2024-09-20T18:53:43.665494+0000","last_fullsized":"2024-09-20T18:53:43.665494+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:46.003836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:46.003836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:04.821993+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"147'12701","reported_seq":12950,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:46.155263+0000","last_change":"2024-09-20T18:53:43.753759+0000","last_active":"2024-09-20T18:53:46.155263+0000","last_peered":"2024-09-20T18:53:46.155263+0000","last_clean":"2024-09-20T18:53:46.155263+0000","last_became_active":"2024-09-20T18:50:49.840023+0000","last_became_peered":"2024-09-20T18:50:49.840023+0000","last_unstale":"2024-09-20T18:53:46.155263+0000","last_undegraded":"2024-09-20T18:53:46.155263+0000","last_fullsized":"2024-09-20T18:53:46.155263+0000","mapping_epoch":83,"log_start":"119'7100","ondisk_log_start":"119'7100","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'12545","last_scrub_stamp":"2024-09-20T18:53:43.753508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:53:43.753508+0000","objects_scrubbed":2735,"log_size":5601,"log_dups_size":0,"ondisk_log_size":5601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:44.964709+0000","scrub_duration":892,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":167706624,"num_objects":2559,"num_object_clones":0,"num_object_copies":7677,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2559,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12701,"num_write_kb":488320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":230,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.672689+0000","last_change":"2024-09-20T18:53:24.458093+0000","last_active":"2024-09-20T18:53:43.672689+0000","last_peered":"2024-09-20T18:53:43.672689+0000","last_clean":"2024-09-20T18:53:43.672689+0000","last_became_active":"2024-09-20T18:49:48.498625+0000","last_became_peered":"2024-09-20T18:49:48.498625+0000","last_unstale":"2024-09-20T18:53:43.672689+0000","last_undegraded":"2024-09-20T18:53:43.672689+0000","last_fullsized":"2024-09-20T18:53:43.672689+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-20T18:53:24.458005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:24.458005+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:31.316232+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":203,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.663336+0000","last_change":"2024-09-20T18:53:20.564464+0000","last_active":"2024-09-20T18:53:43.663336+0000","last_peered":"2024-09-20T18:53:43.663336+0000","last_clean":"2024-09-20T18:53:43.663336+0000","last_became_active":"2024-09-20T18:49:45.941448+0000","last_became_peered":"2024-09-20T18:49:45.941448+0000","last_unstale":"2024-09-20T18:53:43.663336+0000","last_undegraded":"2024-09-20T18:53:43.663336+0000","last_fullsized":"2024-09-20T18:53:43.663336+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:20.564413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:20.564413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:38.260460+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":197,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.661913+0000","last_change":"2024-09-20T18:52:36.662703+0000","last_active":"2024-09-20T18:53:43.661913+0000","last_peered":"2024-09-20T18:53:43.661913+0000","last_clean":"2024-09-20T18:53:43.661913+0000","last_became_active":"2024-09-20T18:49:49.957558+0000","last_became_peered":"2024-09-20T18:49:49.957558+0000","last_unstale":"2024-09-20T18:53:43.661913+0000","last_undegraded":"2024-09-20T18:53:43.661913+0000","last_fullsized":"2024-09-20T18:53:43.661913+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:36.662524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:36.662524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:58.211161+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":191,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.662379+0000","last_change":"2024-09-20T18:52:37.819215+0000","last_active":"2024-09-20T18:53:43.662379+0000","last_peered":"2024-09-20T18:53:43.662379+0000","last_clean":"2024-09-20T18:53:43.662379+0000","last_became_active":"2024-09-20T18:49:51.970798+0000","last_became_peered":"2024-09-20T18:49:51.970798+0000","last_unstale":"2024-09-20T18:53:43.662379+0000","last_undegraded":"2024-09-20T18:53:43.662379+0000","last_fullsized":"2024-09-20T18:53:43.662379+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:37.819164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:37.819164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:45.240111+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.a","version":"147'12793","reported_seq":12933,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:46.143398+0000","last_change":"2024-09-20T18:50:49.842445+0000","last_active":"2024-09-20T18:53:46.143398+0000","last_peered":"2024-09-20T18:53:46.143398+0000","last_clean":"2024-09-20T18:53:46.143398+0000","last_became_active":"2024-09-20T18:50:49.842179+0000","last_became_peered":"2024-09-20T18:50:49.842179+0000","last_unstale":"2024-09-20T18:53:46.143398+0000","last_undegraded":"2024-09-20T18:53:46.143398+0000","last_fullsized":"2024-09-20T18:53:46.143398+0000","mapping_epoch":83,"log_start":"124'8100","ondisk_log_start":"124'8100","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.819660+0000","objects_scrubbed":0,"log_size":4693,"log_dups_size":0,"ondisk_log_size":4693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:29.169615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":164823040,"num_objects":2515,"num_object_clones":0,"num_object_copies":7545,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2515,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12793,"num_write_kb":489856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":215,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.655674+0000","last_change":"2024-09-20T18:52:37.881436+0000","last_active":"2024-09-20T18:53:43.655674+0000","last_peered":"2024-09-20T18:53:43.655674+0000","last_clean":"2024-09-20T18:53:43.655674+0000","last_became_active":"2024-09-20T18:49:47.949116+0000","last_became_peered":"2024-09-20T18:49:47.949116+0000","last_unstale":"2024-09-20T18:53:43.655674+0000","last_undegraded":"2024-09-20T18:53:43.655674+0000","last_fullsized":"2024-09-20T18:53:43.655674+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:52:37.881385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:37.881385+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:49.557204+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":203,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.663154+0000","last_change":"2024-09-20T18:53:21.578905+0000","last_active":"2024-09-20T18:53:43.663154+0000","last_peered":"2024-09-20T18:53:43.663154+0000","last_clean":"2024-09-20T18:53:43.663154+0000","last_became_active":"2024-09-20T18:49:45.944858+0000","last_became_peered":"2024-09-20T18:49:45.944858+0000","last_unstale":"2024-09-20T18:53:43.663154+0000","last_undegraded":"2024-09-20T18:53:43.663154+0000","last_fullsized":"2024-09-20T18:53:43.663154+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:21.578845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:21.578845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:47.899680+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":189,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.661915+0000","last_change":"2024-09-20T18:52:12.674502+0000","last_active":"2024-09-20T18:53:43.661915+0000","last_peered":"2024-09-20T18:53:43.661915+0000","last_clean":"2024-09-20T18:53:43.661915+0000","last_became_active":"2024-09-20T18:49:49.954944+0000","last_became_peered":"2024-09-20T18:49:49.954944+0000","last_unstale":"2024-09-20T18:53:43.661915+0000","last_undegraded":"2024-09-20T18:53:43.661915+0000","last_fullsized":"2024-09-20T18:53:43.661915+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:12.674203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:12.674203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:41.868905+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":184,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.675565+0000","last_change":"2024-09-20T18:52:09.017828+0000","last_active":"2024-09-20T18:53:43.675565+0000","last_peered":"2024-09-20T18:53:43.675565+0000","last_clean":"2024-09-20T18:53:43.675565+0000","last_became_active":"2024-09-20T18:49:51.972970+0000","last_became_peered":"2024-09-20T18:49:51.972970+0000","last_unstale":"2024-09-20T18:53:43.675565+0000","last_undegraded":"2024-09-20T18:53:43.675565+0000","last_fullsized":"2024-09-20T18:53:43.675565+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:09.017759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:09.017759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"147'12766","reported_seq":12984,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:45.910459+0000","last_change":"2024-09-20T18:52:15.505711+0000","last_active":"2024-09-20T18:53:45.910459+0000","last_peered":"2024-09-20T18:53:45.910459+0000","last_clean":"2024-09-20T18:53:45.910459+0000","last_became_active":"2024-09-20T18:50:49.843516+0000","last_became_peered":"2024-09-20T18:50:49.843516+0000","last_unstale":"2024-09-20T18:53:45.910459+0000","last_undegraded":"2024-09-20T18:53:45.910459+0000","last_fullsized":"2024-09-20T18:53:45.910459+0000","mapping_epoch":83,"log_start":"123'7700","ondisk_log_start":"123'7700","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'6958","last_scrub_stamp":"2024-09-20T18:52:15.505628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:52:15.505628+0000","objects_scrubbed":6900,"log_size":5066,"log_dups_size":0,"ondisk_log_size":5066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:29.307707+0000","scrub_duration":2196,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163577856,"num_objects":2496,"num_object_clones":0,"num_object_copies":7488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12766,"num_write_kb":488384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":216,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.682442+0000","last_change":"2024-09-20T18:52:35.182422+0000","last_active":"2024-09-20T18:53:43.682442+0000","last_peered":"2024-09-20T18:53:43.682442+0000","last_clean":"2024-09-20T18:53:43.682442+0000","last_became_active":"2024-09-20T18:49:48.499847+0000","last_became_peered":"2024-09-20T18:49:48.499847+0000","last_unstale":"2024-09-20T18:53:43.682442+0000","last_undegraded":"2024-09-20T18:53:43.682442+0000","last_fullsized":"2024-09-20T18:53:43.682442+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:52:35.182330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:35.182330+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:57.232688+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":215,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.662961+0000","last_change":"2024-09-20T18:53:10.784695+0000","last_active":"2024-09-20T18:53:43.662961+0000","last_peered":"2024-09-20T18:53:43.662961+0000","last_clean":"2024-09-20T18:53:43.662961+0000","last_became_active":"2024-09-20T18:49:45.944256+0000","last_became_peered":"2024-09-20T18:49:45.944256+0000","last_unstale":"2024-09-20T18:53:43.662961+0000","last_undegraded":"2024-09-20T18:53:43.662961+0000","last_fullsized":"2024-09-20T18:53:43.662961+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-20T18:53:10.784625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:10.784625+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:16.037732+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":243,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.662100+0000","last_change":"2024-09-20T18:52:05.788988+0000","last_active":"2024-09-20T18:53:43.662100+0000","last_peered":"2024-09-20T18:53:43.662100+0000","last_clean":"2024-09-20T18:53:43.662100+0000","last_became_active":"2024-09-20T18:49:49.958411+0000","last_became_peered":"2024-09-20T18:49:49.958411+0000","last_unstale":"2024-09-20T18:53:43.662100+0000","last_undegraded":"2024-09-20T18:53:43.662100+0000","last_fullsized":"2024-09-20T18:53:43.662100+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:52:05.788924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.788924+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:39.468023+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":183,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.681550+0000","last_change":"2024-09-20T18:51:56.970415+0000","last_active":"2024-09-20T18:53:43.681550+0000","last_peered":"2024-09-20T18:53:43.681550+0000","last_clean":"2024-09-20T18:53:43.681550+0000","last_became_active":"2024-09-20T18:49:52.139685+0000","last_became_peered":"2024-09-20T18:49:52.139685+0000","last_unstale":"2024-09-20T18:53:43.681550+0000","last_undegraded":"2024-09-20T18:53:43.681550+0000","last_fullsized":"2024-09-20T18:53:43.681550+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:56.970349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:56.970349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:05.081675+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":203,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.672468+0000","last_change":"2024-09-20T18:53:09.388116+0000","last_active":"2024-09-20T18:53:43.672468+0000","last_peered":"2024-09-20T18:53:43.672468+0000","last_clean":"2024-09-20T18:53:43.672468+0000","last_became_active":"2024-09-20T18:49:45.940060+0000","last_became_peered":"2024-09-20T18:49:45.940060+0000","last_unstale":"2024-09-20T18:53:43.672468+0000","last_undegraded":"2024-09-20T18:53:43.672468+0000","last_fullsized":"2024-09-20T18:53:43.672468+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:09.388050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:09.388050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:17.378691+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":201,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.676208+0000","last_change":"2024-09-20T18:53:27.186982+0000","last_active":"2024-09-20T18:53:43.676208+0000","last_peered":"2024-09-20T18:53:43.676208+0000","last_clean":"2024-09-20T18:53:43.676208+0000","last_became_active":"2024-09-20T18:49:48.500662+0000","last_became_peered":"2024-09-20T18:49:48.500662+0000","last_unstale":"2024-09-20T18:53:43.676208+0000","last_undegraded":"2024-09-20T18:53:43.676208+0000","last_fullsized":"2024-09-20T18:53:43.676208+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-20T18:53:27.186631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:27.186631+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:42.463558+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":195,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.684338+0000","last_change":"2024-09-20T18:53:16.220554+0000","last_active":"2024-09-20T18:53:43.684338+0000","last_peered":"2024-09-20T18:53:43.684338+0000","last_clean":"2024-09-20T18:53:43.684338+0000","last_became_active":"2024-09-20T18:49:49.956749+0000","last_became_peered":"2024-09-20T18:49:49.956749+0000","last_unstale":"2024-09-20T18:53:43.684338+0000","last_undegraded":"2024-09-20T18:53:43.684338+0000","last_fullsized":"2024-09-20T18:53:43.684338+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:16.220490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:16.220490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:36.759843+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":191,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.663186+0000","last_change":"2024-09-20T18:53:19.591395+0000","last_active":"2024-09-20T18:53:43.663186+0000","last_peered":"2024-09-20T18:53:43.663186+0000","last_clean":"2024-09-20T18:53:43.663186+0000","last_became_active":"2024-09-20T18:49:52.142291+0000","last_became_peered":"2024-09-20T18:49:52.142291+0000","last_unstale":"2024-09-20T18:53:43.663186+0000","last_undegraded":"2024-09-20T18:53:43.663186+0000","last_fullsized":"2024-09-20T18:53:43.663186+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:19.591346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:19.591346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:40.474867+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":207,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.664080+0000","last_change":"2024-09-20T18:51:23.065382+0000","last_active":"2024-09-20T18:53:43.664080+0000","last_peered":"2024-09-20T18:53:43.664080+0000","last_clean":"2024-09-20T18:53:43.664080+0000","last_became_active":"2024-09-20T18:49:48.500803+0000","last_became_peered":"2024-09-20T18:49:48.500803+0000","last_unstale":"2024-09-20T18:53:43.664080+0000","last_undegraded":"2024-09-20T18:53:43.664080+0000","last_fullsized":"2024-09-20T18:53:43.664080+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:51:23.064536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:23.064536+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:39.863201+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":195,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.662218+0000","last_change":"2024-09-20T18:51:30.982535+0000","last_active":"2024-09-20T18:53:43.662218+0000","last_peered":"2024-09-20T18:53:43.662218+0000","last_clean":"2024-09-20T18:53:43.662218+0000","last_became_active":"2024-09-20T18:49:45.946600+0000","last_became_peered":"2024-09-20T18:49:45.946600+0000","last_unstale":"2024-09-20T18:53:43.662218+0000","last_undegraded":"2024-09-20T18:53:43.662218+0000","last_fullsized":"2024-09-20T18:53:43.662218+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:30.982454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:30.982454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:57.922735+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":188,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.676111+0000","last_change":"2024-09-20T18:52:08.010340+0000","last_active":"2024-09-20T18:53:43.676111+0000","last_peered":"2024-09-20T18:53:43.676111+0000","last_clean":"2024-09-20T18:53:43.676111+0000","last_became_active":"2024-09-20T18:49:49.961947+0000","last_became_peered":"2024-09-20T18:49:49.961947+0000","last_unstale":"2024-09-20T18:53:43.676111+0000","last_undegraded":"2024-09-20T18:53:43.676111+0000","last_fullsized":"2024-09-20T18:53:43.676111+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:08.010222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:08.010222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"66'1","reported_seq":194,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.666919+0000","last_change":"2024-09-20T18:53:14.689510+0000","last_active":"2024-09-20T18:53:43.666919+0000","last_peered":"2024-09-20T18:53:43.666919+0000","last_clean":"2024-09-20T18:53:43.666919+0000","last_became_active":"2024-09-20T18:49:51.965121+0000","last_became_peered":"2024-09-20T18:49:51.965121+0000","last_unstale":"2024-09-20T18:53:43.666919+0000","last_undegraded":"2024-09-20T18:53:43.666919+0000","last_fullsized":"2024-09-20T18:53:43.666919+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T18:53:14.689422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:14.689422+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:19.716830+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":199,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.682628+0000","last_change":"2024-09-20T18:49:48.501359+0000","last_active":"2024-09-20T18:53:43.682628+0000","last_peered":"2024-09-20T18:53:43.682628+0000","last_clean":"2024-09-20T18:53:43.682628+0000","last_became_active":"2024-09-20T18:49:48.501011+0000","last_became_peered":"2024-09-20T18:49:48.501011+0000","last_unstale":"2024-09-20T18:53:43.682628+0000","last_undegraded":"2024-09-20T18:53:43.682628+0000","last_fullsized":"2024-09-20T18:53:43.682628+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:02.117814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":195,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.669176+0000","last_change":"2024-09-20T18:51:20.067303+0000","last_active":"2024-09-20T18:53:43.669176+0000","last_peered":"2024-09-20T18:53:43.669176+0000","last_clean":"2024-09-20T18:53:43.669176+0000","last_became_active":"2024-09-20T18:49:45.938511+0000","last_became_peered":"2024-09-20T18:49:45.938511+0000","last_unstale":"2024-09-20T18:53:43.669176+0000","last_undegraded":"2024-09-20T18:53:43.669176+0000","last_fullsized":"2024-09-20T18:53:43.669176+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:20.067233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:20.067233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:55.959831+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":245,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.675569+0000","last_change":"2024-09-20T18:52:14.049155+0000","last_active":"2024-09-20T18:53:43.675569+0000","last_peered":"2024-09-20T18:53:43.675569+0000","last_clean":"2024-09-20T18:53:43.675569+0000","last_became_active":"2024-09-20T18:49:49.960550+0000","last_became_peered":"2024-09-20T18:49:49.960550+0000","last_unstale":"2024-09-20T18:53:43.675569+0000","last_undegraded":"2024-09-20T18:53:43.675569+0000","last_fullsized":"2024-09-20T18:53:43.675569+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:52:14.048921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:14.048921+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":184,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.656775+0000","last_change":"2024-09-20T18:52:41.825306+0000","last_active":"2024-09-20T18:53:43.656775+0000","last_peered":"2024-09-20T18:53:43.656775+0000","last_clean":"2024-09-20T18:53:43.656775+0000","last_became_active":"2024-09-20T18:49:51.966254+0000","last_became_peered":"2024-09-20T18:49:51.966254+0000","last_unstale":"2024-09-20T18:53:43.656775+0000","last_undegraded":"2024-09-20T18:53:43.656775+0000","last_fullsized":"2024-09-20T18:53:43.656775+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:41.825248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:41.825248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:47.685753+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":226,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.675927+0000","last_change":"2024-09-20T18:51:15.223913+0000","last_active":"2024-09-20T18:53:43.675927+0000","last_peered":"2024-09-20T18:53:43.675927+0000","last_clean":"2024-09-20T18:53:43.675927+0000","last_became_active":"2024-09-20T18:49:58.997494+0000","last_became_peered":"2024-09-20T18:49:58.997494+0000","last_unstale":"2024-09-20T18:53:43.675927+0000","last_undegraded":"2024-09-20T18:53:43.675927+0000","last_fullsized":"2024-09-20T18:53:43.675927+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:51:15.223826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:15.223826+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:06.150447+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":203,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.662641+0000","last_change":"2024-09-20T18:53:27.589481+0000","last_active":"2024-09-20T18:53:43.662641+0000","last_peered":"2024-09-20T18:53:43.662641+0000","last_clean":"2024-09-20T18:53:43.662641+0000","last_became_active":"2024-09-20T18:49:45.949492+0000","last_became_peered":"2024-09-20T18:49:45.949492+0000","last_unstale":"2024-09-20T18:53:43.662641+0000","last_undegraded":"2024-09-20T18:53:43.662641+0000","last_fullsized":"2024-09-20T18:53:43.662641+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:27.589413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:27.589413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:48.562654+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":243,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.681942+0000","last_change":"2024-09-20T18:52:39.168501+0000","last_active":"2024-09-20T18:53:43.681942+0000","last_peered":"2024-09-20T18:53:43.681942+0000","last_clean":"2024-09-20T18:53:43.681942+0000","last_became_active":"2024-09-20T18:49:49.962243+0000","last_became_peered":"2024-09-20T18:49:49.962243+0000","last_unstale":"2024-09-20T18:53:43.681942+0000","last_undegraded":"2024-09-20T18:53:43.681942+0000","last_fullsized":"2024-09-20T18:53:43.681942+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:52:39.168442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:39.168442+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:02.311754+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":191,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.663076+0000","last_change":"2024-09-20T18:53:13.863129+0000","last_active":"2024-09-20T18:53:43.663076+0000","last_peered":"2024-09-20T18:53:43.663076+0000","last_clean":"2024-09-20T18:53:43.663076+0000","last_became_active":"2024-09-20T18:49:51.969093+0000","last_became_peered":"2024-09-20T18:49:51.969093+0000","last_unstale":"2024-09-20T18:53:43.663076+0000","last_undegraded":"2024-09-20T18:53:43.663076+0000","last_fullsized":"2024-09-20T18:53:43.663076+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:13.863061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:13.863061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:37.505050+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":219,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.879001+0000","last_change":"2024-09-20T18:53:43.879001+0000","last_active":"2024-09-20T18:53:43.879001+0000","last_peered":"2024-09-20T18:53:43.879001+0000","last_clean":"2024-09-20T18:53:43.879001+0000","last_became_active":"2024-09-20T18:49:47.949703+0000","last_became_peered":"2024-09-20T18:49:47.949703+0000","last_unstale":"2024-09-20T18:53:43.879001+0000","last_undegraded":"2024-09-20T18:53:43.879001+0000","last_fullsized":"2024-09-20T18:53:43.879001+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:53:43.878921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:43.878921+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:02.026666+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":196,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.657061+0000","last_change":"2024-09-20T18:52:05.985622+0000","last_active":"2024-09-20T18:53:43.657061+0000","last_peered":"2024-09-20T18:53:43.657061+0000","last_clean":"2024-09-20T18:53:43.657061+0000","last_became_active":"2024-09-20T18:49:45.939949+0000","last_became_peered":"2024-09-20T18:49:45.939949+0000","last_unstale":"2024-09-20T18:53:43.657061+0000","last_undegraded":"2024-09-20T18:53:43.657061+0000","last_fullsized":"2024-09-20T18:53:43.657061+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:05.985551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.985551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:36.823731+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":188,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.668182+0000","last_change":"2024-09-20T18:51:14.271954+0000","last_active":"2024-09-20T18:53:43.668182+0000","last_peered":"2024-09-20T18:53:43.668182+0000","last_clean":"2024-09-20T18:53:43.668182+0000","last_became_active":"2024-09-20T18:49:49.957067+0000","last_became_peered":"2024-09-20T18:49:49.957067+0000","last_unstale":"2024-09-20T18:53:43.668182+0000","last_undegraded":"2024-09-20T18:53:43.668182+0000","last_fullsized":"2024-09-20T18:53:43.668182+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:14.271719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.271719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:31.305762+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":184,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.675162+0000","last_change":"2024-09-20T18:51:14.210438+0000","last_active":"2024-09-20T18:53:43.675162+0000","last_peered":"2024-09-20T18:53:43.675162+0000","last_clean":"2024-09-20T18:53:43.675162+0000","last_became_active":"2024-09-20T18:49:51.966915+0000","last_became_peered":"2024-09-20T18:49:51.966915+0000","last_unstale":"2024-09-20T18:53:43.675162+0000","last_undegraded":"2024-09-20T18:53:43.675162+0000","last_fullsized":"2024-09-20T18:53:43.675162+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:14.210358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.210358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:04.079575+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":216,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.668406+0000","last_change":"2024-09-20T18:52:38.240463+0000","last_active":"2024-09-20T18:53:43.668406+0000","last_peered":"2024-09-20T18:53:43.668406+0000","last_clean":"2024-09-20T18:53:43.668406+0000","last_became_active":"2024-09-20T18:49:47.953310+0000","last_became_peered":"2024-09-20T18:49:47.953310+0000","last_unstale":"2024-09-20T18:53:43.668406+0000","last_undegraded":"2024-09-20T18:53:43.668406+0000","last_fullsized":"2024-09-20T18:53:43.668406+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:52:38.240386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:38.240386+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:52.778987+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":195,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.663274+0000","last_change":"2024-09-20T18:52:34.834333+0000","last_active":"2024-09-20T18:53:43.663274+0000","last_peered":"2024-09-20T18:53:43.663274+0000","last_clean":"2024-09-20T18:53:43.663274+0000","last_became_active":"2024-09-20T18:49:45.944357+0000","last_became_peered":"2024-09-20T18:49:45.944357+0000","last_unstale":"2024-09-20T18:53:43.663274+0000","last_undegraded":"2024-09-20T18:53:43.663274+0000","last_fullsized":"2024-09-20T18:53:43.663274+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:34.834273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:34.834273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:42.456584+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":188,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.675570+0000","last_change":"2024-09-20T18:51:20.208925+0000","last_active":"2024-09-20T18:53:43.675570+0000","last_peered":"2024-09-20T18:53:43.675570+0000","last_clean":"2024-09-20T18:53:43.675570+0000","last_became_active":"2024-09-20T18:49:49.962523+0000","last_became_peered":"2024-09-20T18:49:49.962523+0000","last_unstale":"2024-09-20T18:53:43.675570+0000","last_undegraded":"2024-09-20T18:53:43.675570+0000","last_fullsized":"2024-09-20T18:53:43.675570+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:20.208786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:20.208786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:08.165781+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":191,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.670016+0000","last_change":"2024-09-20T18:52:38.137485+0000","last_active":"2024-09-20T18:53:43.670016+0000","last_peered":"2024-09-20T18:53:43.670016+0000","last_clean":"2024-09-20T18:53:43.670016+0000","last_became_active":"2024-09-20T18:49:51.967313+0000","last_became_peered":"2024-09-20T18:49:51.967313+0000","last_unstale":"2024-09-20T18:53:43.670016+0000","last_undegraded":"2024-09-20T18:53:43.670016+0000","last_fullsized":"2024-09-20T18:53:43.670016+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:38.137431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:38.137431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:07.091791+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":211,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.675241+0000","last_change":"2024-09-20T18:52:13.020995+0000","last_active":"2024-09-20T18:53:43.675241+0000","last_peered":"2024-09-20T18:53:43.675241+0000","last_clean":"2024-09-20T18:53:43.675241+0000","last_became_active":"2024-09-20T18:49:48.499111+0000","last_became_peered":"2024-09-20T18:49:48.499111+0000","last_unstale":"2024-09-20T18:53:43.675241+0000","last_undegraded":"2024-09-20T18:53:43.675241+0000","last_fullsized":"2024-09-20T18:53:43.675241+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:52:13.020920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:13.020920+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":195,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.684459+0000","last_change":"2024-09-20T18:52:10.090860+0000","last_active":"2024-09-20T18:53:43.684459+0000","last_peered":"2024-09-20T18:53:43.684459+0000","last_clean":"2024-09-20T18:53:43.684459+0000","last_became_active":"2024-09-20T18:49:45.942105+0000","last_became_peered":"2024-09-20T18:49:45.942105+0000","last_unstale":"2024-09-20T18:53:43.684459+0000","last_undegraded":"2024-09-20T18:53:43.684459+0000","last_fullsized":"2024-09-20T18:53:43.684459+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:10.090802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:10.090802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":195,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.661886+0000","last_change":"2024-09-20T18:52:41.836356+0000","last_active":"2024-09-20T18:53:43.661886+0000","last_peered":"2024-09-20T18:53:43.661886+0000","last_clean":"2024-09-20T18:53:43.661886+0000","last_became_active":"2024-09-20T18:49:49.958159+0000","last_became_peered":"2024-09-20T18:49:49.958159+0000","last_unstale":"2024-09-20T18:53:43.661886+0000","last_undegraded":"2024-09-20T18:53:43.661886+0000","last_fullsized":"2024-09-20T18:53:43.661886+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:41.836294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:41.836294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:44.305979+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":184,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.675671+0000","last_change":"2024-09-20T18:52:43.118886+0000","last_active":"2024-09-20T18:53:43.675671+0000","last_peered":"2024-09-20T18:53:43.675671+0000","last_clean":"2024-09-20T18:53:43.675671+0000","last_became_active":"2024-09-20T18:49:52.139978+0000","last_became_peered":"2024-09-20T18:49:52.139978+0000","last_unstale":"2024-09-20T18:53:43.675671+0000","last_undegraded":"2024-09-20T18:53:43.675671+0000","last_fullsized":"2024-09-20T18:53:43.675671+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:43.118805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:43.118805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:11.215766+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":206,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.675056+0000","last_change":"2024-09-20T18:53:43.253204+0000","last_active":"2024-09-20T18:53:43.675056+0000","last_peered":"2024-09-20T18:53:43.675056+0000","last_clean":"2024-09-20T18:53:43.675056+0000","last_became_active":"2024-09-20T18:49:47.954777+0000","last_became_peered":"2024-09-20T18:49:47.954777+0000","last_unstale":"2024-09-20T18:53:43.675056+0000","last_undegraded":"2024-09-20T18:53:43.675056+0000","last_fullsized":"2024-09-20T18:53:43.675056+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-20T18:53:43.253132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:43.253132+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:00.654449+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":203,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.673978+0000","last_change":"2024-09-20T18:53:16.921320+0000","last_active":"2024-09-20T18:53:43.673978+0000","last_peered":"2024-09-20T18:53:43.673978+0000","last_clean":"2024-09-20T18:53:43.673978+0000","last_became_active":"2024-09-20T18:49:45.940988+0000","last_became_peered":"2024-09-20T18:49:45.940988+0000","last_unstale":"2024-09-20T18:53:43.673978+0000","last_undegraded":"2024-09-20T18:53:43.673978+0000","last_fullsized":"2024-09-20T18:53:43.673978+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:16.921263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:16.921263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:35.775476+0000","scrub_duration":137,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":195,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.672468+0000","last_change":"2024-09-20T18:52:44.344764+0000","last_active":"2024-09-20T18:53:43.672468+0000","last_peered":"2024-09-20T18:53:43.672468+0000","last_clean":"2024-09-20T18:53:43.672468+0000","last_became_active":"2024-09-20T18:49:49.954502+0000","last_became_peered":"2024-09-20T18:49:49.954502+0000","last_unstale":"2024-09-20T18:53:43.672468+0000","last_undegraded":"2024-09-20T18:53:43.672468+0000","last_fullsized":"2024-09-20T18:53:43.672468+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:44.344700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:44.344700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:52.432651+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":183,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.662108+0000","last_change":"2024-09-20T18:51:09.761331+0000","last_active":"2024-09-20T18:53:43.662108+0000","last_peered":"2024-09-20T18:53:43.662108+0000","last_clean":"2024-09-20T18:53:43.662108+0000","last_became_active":"2024-09-20T18:49:51.968662+0000","last_became_peered":"2024-09-20T18:49:51.968662+0000","last_unstale":"2024-09-20T18:53:43.662108+0000","last_undegraded":"2024-09-20T18:53:43.662108+0000","last_fullsized":"2024-09-20T18:53:43.662108+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:09.761283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:09.761283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:30.451288+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":176,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.670837+0000","last_change":"2024-09-20T18:49:51.968476+0000","last_active":"2024-09-20T18:53:43.670837+0000","last_peered":"2024-09-20T18:53:43.670837+0000","last_clean":"2024-09-20T18:53:43.670837+0000","last_became_active":"2024-09-20T18:49:51.968216+0000","last_became_peered":"2024-09-20T18:49:51.968216+0000","last_unstale":"2024-09-20T18:53:43.670837+0000","last_undegraded":"2024-09-20T18:53:43.670837+0000","last_fullsized":"2024-09-20T18:53:43.670837+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:58.058065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":196,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.675282+0000","last_change":"2024-09-20T18:52:41.106558+0000","last_active":"2024-09-20T18:53:43.675282+0000","last_peered":"2024-09-20T18:53:43.675282+0000","last_clean":"2024-09-20T18:53:43.675282+0000","last_became_active":"2024-09-20T18:49:45.937992+0000","last_became_peered":"2024-09-20T18:49:45.937992+0000","last_unstale":"2024-09-20T18:53:43.675282+0000","last_undegraded":"2024-09-20T18:53:43.675282+0000","last_fullsized":"2024-09-20T18:53:43.675282+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:41.106332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:41.106332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:50.275314+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":210,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.673473+0000","last_change":"2024-09-20T18:51:14.545122+0000","last_active":"2024-09-20T18:53:43.673473+0000","last_peered":"2024-09-20T18:53:43.673473+0000","last_clean":"2024-09-20T18:53:43.673473+0000","last_became_active":"2024-09-20T18:49:47.949582+0000","last_became_peered":"2024-09-20T18:49:47.949582+0000","last_unstale":"2024-09-20T18:53:43.673473+0000","last_undegraded":"2024-09-20T18:53:43.673473+0000","last_fullsized":"2024-09-20T18:53:43.673473+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:51:14.545043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.545043+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:40.491358+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":187,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-20T18:53:43.663308+0000","last_change":"2024-09-20T18:51:28.120100+0000","last_active":"2024-09-20T18:53:43.663308+0000","last_peered":"2024-09-20T18:53:43.663308+0000","last_clean":"2024-09-20T18:53:43.663308+0000","last_became_active":"2024-09-20T18:49:49.957582+0000","last_became_peered":"2024-09-20T18:49:49.957582+0000","last_unstale":"2024-09-20T18:53:43.663308+0000","last_undegraded":"2024-09-20T18:53:43.663308+0000","last_fullsized":"2024-09-20T18:53:43.663308+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:28.119694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:28.119694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:32.813511+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":2652241944,"num_objects":40471,"num_object_clones":0,"num_object_copies":121413,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40471,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":205129,"num_write_kb":7859137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":7960670208,"data_stored":7960657992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":90323,"ondisk_log_size":90323,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":235,"num_read_kb":230,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266740,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":940548,"kb_used_data":828852,"kb_used_omap":1,"kb_used_meta":111678,"kb_avail":92804604,"statfs":{"total":95995035648,"available":95031914496,"internally_reserved":0,"allocated":848744448,"data_stored":848332622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":114358730},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528379,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":913908,"kb_used_data":810532,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":92831244,"statfs":{"total":95995035648,"available":95059193856,"internally_reserved":0,"allocated":829984768,"data_stored":829588688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":105839046},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757312,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":728880,"kb_used_data":657440,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":93016272,"statfs":{"total":95995035648,"available":95248662528,"internally_reserved":0,"allocated":673218560,"data_stored":672826367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986245,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":942800,"kb_used_data":839424,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":92802352,"statfs":{"total":95995035648,"available":95029608448,"internally_reserved":0,"allocated":859570176,"data_stored":859146467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105839045},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1635220,"kb_used_data":1477628,"kb_used_omap":1,"kb_used_meta":157502,"kb_avail":92109932,"statfs":{"total":95995035648,"available":94320570368,"internally_reserved":0,"allocated":1513091072,"data_stored":1512670303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":161282503},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1095540,"kb_used_data":983844,"kb_used_omap":1,"kb_used_meta":111678,"kb_avail":92649612,"statfs":{"total":95995035648,"available":94873202688,"internally_reserved":0,"allocated":1007456256,"data_stored":1007060546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":114358725},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1095400,"kb_used_data":983640,"kb_used_omap":1,"kb_used_meta":111678,"kb_avail":92649752,"statfs":{"total":95995035648,"available":94873346048,"internally_reserved":0,"allocated":1007247360,"data_stored":1006799610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":114358725},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1438348,"kb_used_data":1298872,"kb_used_omap":1,"kb_used_meta":139454,"kb_avail":92306804,"statfs":{"total":95995035648,"available":94522167296,"internally_reserved":0,"allocated":1330044928,"data_stored":1329628726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142801349},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1317670912,"data_stored":1317666840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":991563776,"data_stored":991559704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1002307584,"data_stored":1002307584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1483276288,"data_stored":1483276288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":840302592,"data_stored":840302592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":665911296,"data_stored":665911296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":822480896,"data_stored":822476824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":837156864,"data_stored":837156864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T18:53:52.077 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-09-20T18:53:52.077 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T18:53:52.077 INFO:tasks.thrashosds.thrasher:Shrinking pool .mgr 2024-09-20T18:53:52.077 INFO:tasks.cephadm.ceph_manager.ceph:contract_pool .mgr by 10 min 8 2024-09-20T18:53:52.078 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T18:53:52.101 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:52.124 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:52.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:52.126 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:52.156 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:53:52.166 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:53:52.226 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:52.277 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:52.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:52.279 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:52.380 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:52.405 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:52.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:52.406 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:52.443 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_blocked_ops 2024-09-20T18:53:52.507 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:52.536 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:52.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:52.537 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:52.637 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:52.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:52.672 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:52.672 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:52.773 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:52.800 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:52.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:52.801 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:52.817 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:53:52.901 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:52.927 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:52.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:52.928 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:53.029 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:53.066 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:53.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:53.067 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:53.168 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:53:53.209 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:53.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:53.210 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:53:53.311 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:53.339 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:53.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:53.340 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:53.408 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T18:53:53.408 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-09-20T18:53:53.408 INFO:teuthology.orchestra.run.smithi142.stdout: "complaint_time": 30, 2024-09-20T18:53:53.408 INFO:teuthology.orchestra.run.smithi142.stdout: "num_blocked_ops": 0 2024-09-20T18:53:53.408 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T18:53:53.441 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:53.471 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:53.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:53.472 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:53.573 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:53.602 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:53.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:53.603 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:53.704 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:53.732 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:53.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:53.733 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:53.834 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:53.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:53.878 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:53.879 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:53.979 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:54.006 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:54.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:54.007 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:54.108 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:53:54.135 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:53 smithi083 ceph-mon[29421]: 5.11 scrub starts 2024-09-20T18:53:54.136 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:53 smithi083 ceph-mon[29421]: 5.11 scrub ok 2024-09-20T18:53:54.136 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:53 smithi083 ceph-mon[29421]: pgmap v187: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 2.2 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.28k op/s 2024-09-20T18:53:54.136 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:53 smithi083 ceph-mon[29421]: 4.9 scrub starts 2024-09-20T18:53:54.136 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:53 smithi083 ceph-mon[29421]: 4.9 scrub ok 2024-09-20T18:53:54.136 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:53 smithi083 ceph-mon[29421]: 4.12 scrub starts 2024-09-20T18:53:54.136 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:53 smithi083 ceph-mon[29421]: 4.12 scrub ok 2024-09-20T18:53:54.136 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:53 smithi083 ceph-mon[36260]: 5.11 scrub starts 2024-09-20T18:53:54.136 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:53 smithi083 ceph-mon[36260]: 5.11 scrub ok 2024-09-20T18:53:54.136 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:53 smithi083 ceph-mon[36260]: pgmap v187: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 2.2 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.28k op/s 2024-09-20T18:53:54.136 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:53 smithi083 ceph-mon[36260]: 4.9 scrub starts 2024-09-20T18:53:54.137 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:53 smithi083 ceph-mon[36260]: 4.9 scrub ok 2024-09-20T18:53:54.137 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:53 smithi083 ceph-mon[36260]: 4.12 scrub starts 2024-09-20T18:53:54.137 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:53 smithi083 ceph-mon[36260]: 4.12 scrub ok 2024-09-20T18:53:54.149 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:54.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:54.151 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:53:54.204 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:53:54.219 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:53:54.219 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:53:54.252 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:54.259 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:53:54.259 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:53:54.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:53 smithi142 ceph-mon[32508]: 5.11 scrub starts 2024-09-20T18:53:54.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:53 smithi142 ceph-mon[32508]: 5.11 scrub ok 2024-09-20T18:53:54.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:53 smithi142 ceph-mon[32508]: pgmap v187: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 2.2 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.28k op/s 2024-09-20T18:53:54.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:53 smithi142 ceph-mon[32508]: 4.9 scrub starts 2024-09-20T18:53:54.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:53 smithi142 ceph-mon[32508]: 4.9 scrub ok 2024-09-20T18:53:54.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:53 smithi142 ceph-mon[32508]: 4.12 scrub starts 2024-09-20T18:53:54.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:53 smithi142 ceph-mon[32508]: 4.12 scrub ok 2024-09-20T18:53:54.276 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:54.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:54.277 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:54.293 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:53:54.293 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:53:54.329 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:53:54.329 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:53:54.359 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:53:54.360 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:53:54.378 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:53:54.399 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:53:54.399 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:53:54.406 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:54.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:54.407 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:53:54.433 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:53:54.434 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:53:54.466 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:53:54.467 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:53:54.508 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:54.557 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:54.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:54.558 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:54.659 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:54.687 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:54.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:54.688 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:54.719 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_historic_ops 2024-09-20T18:53:54.789 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:53:54.833 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:54.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:54.834 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:53:54.928 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-20T18:53:54.935 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:54.971 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:54.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:54.972 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:55.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:54 smithi142 ceph-mon[32508]: 6.1e scrub starts 2024-09-20T18:53:55.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:54 smithi142 ceph-mon[32508]: 6.1e scrub ok 2024-09-20T18:53:55.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:54 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/4143756495' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:53:55.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:54 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3333360807' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:53:55.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:54 smithi142 ceph-mon[32508]: 6.1f scrub starts 2024-09-20T18:53:55.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:54 smithi142 ceph-mon[32508]: 6.1f scrub ok 2024-09-20T18:53:55.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:54 smithi083 ceph-mon[29421]: 6.1e scrub starts 2024-09-20T18:53:55.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:54 smithi083 ceph-mon[29421]: 6.1e scrub ok 2024-09-20T18:53:55.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:54 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/4143756495' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:53:55.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:54 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3333360807' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:53:55.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:54 smithi083 ceph-mon[29421]: 6.1f scrub starts 2024-09-20T18:53:55.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:54 smithi083 ceph-mon[29421]: 6.1f scrub ok 2024-09-20T18:53:55.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:54 smithi083 ceph-mon[36260]: 6.1e scrub starts 2024-09-20T18:53:55.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:54 smithi083 ceph-mon[36260]: 6.1e scrub ok 2024-09-20T18:53:55.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:54 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/4143756495' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:53:55.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:54 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3333360807' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:53:55.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:54 smithi083 ceph-mon[36260]: 6.1f scrub starts 2024-09-20T18:53:55.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:54 smithi083 ceph-mon[36260]: 6.1f scrub ok 2024-09-20T18:53:55.073 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:55.112 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:55.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:55.114 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:55.214 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:53:55.239 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:55.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:55.240 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:53:55.333 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:53:55.335 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-20T18:53:55.341 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:55.360 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:53:55.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:55.371 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:55.371 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:55.472 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:55.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:55.508 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:55.508 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:55.609 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:55.646 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:55.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:55.647 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:55.749 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:55.777 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:55.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:55.778 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:55.879 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:55.915 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:55.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:55.916 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:56.017 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:56.045 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:56.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:56.046 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:56.062 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:55 smithi142 ceph-mon[32508]: pgmap v188: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 2.2 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 626 B/s rd, 0 B/s wr, 637 op/s 2024-09-20T18:53:56.062 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:55 smithi142 ceph-mon[32508]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:53:56.063 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:55 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/4143756495' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T18:53:56.063 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:55 smithi142 ceph-mon[32508]: osdmap e150: 8 total, 8 up, 8 in 2024-09-20T18:53:56.063 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:55 smithi142 ceph-mon[32508]: from='client.24883 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:53:56.146 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:56.183 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:56.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:56.184 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:56.199 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T18:53:56.199 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:53:56.278 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:55 smithi083 ceph-mon[36260]: pgmap v188: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 2.2 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 626 B/s rd, 0 B/s wr, 637 op/s 2024-09-20T18:53:56.278 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:55 smithi083 ceph-mon[36260]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:53:56.278 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:55 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/4143756495' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T18:53:56.278 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:55 smithi083 ceph-mon[36260]: osdmap e150: 8 total, 8 up, 8 in 2024-09-20T18:53:56.278 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:55 smithi083 ceph-mon[36260]: from='client.24883 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:53:56.279 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:55 smithi083 ceph-mon[29421]: pgmap v188: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 2.2 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 626 B/s rd, 0 B/s wr, 637 op/s 2024-09-20T18:53:56.279 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:55 smithi083 ceph-mon[29421]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:53:56.279 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:55 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/4143756495' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T18:53:56.279 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:55 smithi083 ceph-mon[29421]: osdmap e150: 8 total, 8 up, 8 in 2024-09-20T18:53:56.279 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:55 smithi083 ceph-mon[29421]: from='client.24883 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:53:56.285 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:56.310 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:56.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:56.311 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:56.326 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":189,"stamp":"2024-09-20T18:53:54.911627+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2311658035,"num_objects":35464,"num_object_clones":0,"num_object_copies":106392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35464,"num_whiteouts":0,"num_read":1089,"num_read_kb":946,"num_write":210884,"num_write_kb":7860810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":590808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":90581,"ondisk_log_size":90581,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":7803924,"kb_used_data":6889288,"kb_used_omap":12,"kb_used_meta":914291,"kb_avail":742157292,"statfs":{"total":767960285184,"available":759969067008,"internally_reserved":0,"allocated":7054630912,"data_stored":7051714977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":936234544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,10,4,2,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-341180416,"num_objects":-5206,"num_object_clones":0,"num_object_copies":-15618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5206,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5206,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.078410"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":199,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.711466+0000","last_change":"2024-09-20T18:51:57.238254+0000","last_active":"2024-09-20T18:53:48.711466+0000","last_peered":"2024-09-20T18:53:48.711466+0000","last_clean":"2024-09-20T18:53:48.711466+0000","last_became_active":"2024-09-20T18:49:45.943201+0000","last_became_peered":"2024-09-20T18:49:45.943201+0000","last_unstale":"2024-09-20T18:53:48.711466+0000","last_undegraded":"2024-09-20T18:53:48.711466+0000","last_fullsized":"2024-09-20T18:53:48.711466+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:57.238177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:57.238177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:06.040036+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":237,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.701384+0000","last_change":"2024-09-20T18:53:15.772300+0000","last_active":"2024-09-20T18:53:48.701384+0000","last_peered":"2024-09-20T18:53:48.701384+0000","last_clean":"2024-09-20T18:53:48.701384+0000","last_became_active":"2024-09-20T18:52:15.609648+0000","last_became_peered":"2024-09-20T18:52:15.609648+0000","last_unstale":"2024-09-20T18:53:48.701384+0000","last_undegraded":"2024-09-20T18:53:48.701384+0000","last_fullsized":"2024-09-20T18:53:48.701384+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:53:15.772228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:15.772228+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:33.912905+0000","scrub_duration":37,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":200,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.719313+0000","last_change":"2024-09-20T18:53:21.291759+0000","last_active":"2024-09-20T18:53:48.719313+0000","last_peered":"2024-09-20T18:53:48.719313+0000","last_clean":"2024-09-20T18:53:48.719313+0000","last_became_active":"2024-09-20T18:49:49.959514+0000","last_became_peered":"2024-09-20T18:49:49.959514+0000","last_unstale":"2024-09-20T18:53:48.719313+0000","last_undegraded":"2024-09-20T18:53:48.719313+0000","last_fullsized":"2024-09-20T18:53:48.719313+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:21.291695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:21.291695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:25.316294+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":196,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.701551+0000","last_change":"2024-09-20T18:53:08.740138+0000","last_active":"2024-09-20T18:53:48.701551+0000","last_peered":"2024-09-20T18:53:48.701551+0000","last_clean":"2024-09-20T18:53:48.701551+0000","last_became_active":"2024-09-20T18:49:51.969421+0000","last_became_peered":"2024-09-20T18:49:51.969421+0000","last_unstale":"2024-09-20T18:53:48.701551+0000","last_undegraded":"2024-09-20T18:53:48.701551+0000","last_fullsized":"2024-09-20T18:53:48.701551+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:08.740093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:08.740093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:11.987287+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":196,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.714574+0000","last_change":"2024-09-20T18:53:46.283952+0000","last_active":"2024-09-20T18:53:48.714574+0000","last_peered":"2024-09-20T18:53:48.714574+0000","last_clean":"2024-09-20T18:53:48.714574+0000","last_became_active":"2024-09-20T18:49:52.141967+0000","last_became_peered":"2024-09-20T18:49:52.141967+0000","last_unstale":"2024-09-20T18:53:48.714574+0000","last_undegraded":"2024-09-20T18:53:48.714574+0000","last_fullsized":"2024-09-20T18:53:48.714574+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:46.283881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:46.283881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:49.076672+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":201,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.714001+0000","last_change":"2024-09-20T18:51:25.208165+0000","last_active":"2024-09-20T18:53:48.714001+0000","last_peered":"2024-09-20T18:53:48.714001+0000","last_clean":"2024-09-20T18:53:48.714001+0000","last_became_active":"2024-09-20T18:49:45.943607+0000","last_became_peered":"2024-09-20T18:49:45.943607+0000","last_unstale":"2024-09-20T18:53:48.714001+0000","last_undegraded":"2024-09-20T18:53:48.714001+0000","last_fullsized":"2024-09-20T18:53:48.714001+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:25.208073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:25.208073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:07.185746+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":221,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.717108+0000","last_change":"2024-09-20T18:51:07.130137+0000","last_active":"2024-09-20T18:53:48.717108+0000","last_peered":"2024-09-20T18:53:48.717108+0000","last_clean":"2024-09-20T18:53:48.717108+0000","last_became_active":"2024-09-20T18:49:47.952150+0000","last_became_peered":"2024-09-20T18:49:47.952150+0000","last_unstale":"2024-09-20T18:53:48.717108+0000","last_undegraded":"2024-09-20T18:53:48.717108+0000","last_fullsized":"2024-09-20T18:53:48.717108+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:51:07.130062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:07.130062+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:33.208818+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":192,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.701315+0000","last_change":"2024-09-20T18:51:09.436429+0000","last_active":"2024-09-20T18:53:48.701315+0000","last_peered":"2024-09-20T18:53:48.701315+0000","last_clean":"2024-09-20T18:53:48.701315+0000","last_became_active":"2024-09-20T18:49:49.956445+0000","last_became_peered":"2024-09-20T18:49:49.956445+0000","last_unstale":"2024-09-20T18:53:48.701315+0000","last_undegraded":"2024-09-20T18:53:48.701315+0000","last_fullsized":"2024-09-20T18:53:48.701315+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:09.436302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:09.436302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:04.774926+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"59'1","reported_seq":213,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.716002+0000","last_change":"2024-09-20T18:52:48.035578+0000","last_active":"2024-09-20T18:53:48.716002+0000","last_peered":"2024-09-20T18:53:48.716002+0000","last_clean":"2024-09-20T18:53:48.716002+0000","last_became_active":"2024-09-20T18:49:45.946171+0000","last_became_peered":"2024-09-20T18:49:45.946171+0000","last_unstale":"2024-09-20T18:53:48.716002+0000","last_undegraded":"2024-09-20T18:53:48.716002+0000","last_fullsized":"2024-09-20T18:53:48.716002+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:52:48.035532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:48.035532+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:01.343903+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":220,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.701703+0000","last_change":"2024-09-20T18:53:19.836082+0000","last_active":"2024-09-20T18:53:48.701703+0000","last_peered":"2024-09-20T18:53:48.701703+0000","last_clean":"2024-09-20T18:53:48.701703+0000","last_became_active":"2024-09-20T18:49:47.947448+0000","last_became_peered":"2024-09-20T18:49:47.947448+0000","last_unstale":"2024-09-20T18:53:48.701703+0000","last_undegraded":"2024-09-20T18:53:48.701703+0000","last_fullsized":"2024-09-20T18:53:48.701703+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:53:19.835996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:19.835996+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:39.939119+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":199,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.715208+0000","last_change":"2024-09-20T18:52:36.183932+0000","last_active":"2024-09-20T18:53:48.715208+0000","last_peered":"2024-09-20T18:53:48.715208+0000","last_clean":"2024-09-20T18:53:48.715208+0000","last_became_active":"2024-09-20T18:49:49.961188+0000","last_became_peered":"2024-09-20T18:49:49.961188+0000","last_unstale":"2024-09-20T18:53:48.715208+0000","last_undegraded":"2024-09-20T18:53:48.715208+0000","last_fullsized":"2024-09-20T18:53:48.715208+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:36.183875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:36.183875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:49.223570+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":179,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.710143+0000","last_change":"2024-09-20T18:49:51.966957+0000","last_active":"2024-09-20T18:53:48.710143+0000","last_peered":"2024-09-20T18:53:48.710143+0000","last_clean":"2024-09-20T18:53:48.710143+0000","last_became_active":"2024-09-20T18:49:51.966715+0000","last_became_peered":"2024-09-20T18:49:51.966715+0000","last_unstale":"2024-09-20T18:53:48.710143+0000","last_undegraded":"2024-09-20T18:53:48.710143+0000","last_fullsized":"2024-09-20T18:53:48.710143+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:59.013361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":199,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.715567+0000","last_change":"2024-09-20T18:52:00.970719+0000","last_active":"2024-09-20T18:53:48.715567+0000","last_peered":"2024-09-20T18:53:48.715567+0000","last_clean":"2024-09-20T18:53:48.715567+0000","last_became_active":"2024-09-20T18:49:45.944703+0000","last_became_peered":"2024-09-20T18:49:45.944703+0000","last_unstale":"2024-09-20T18:53:48.715567+0000","last_undegraded":"2024-09-20T18:53:48.715567+0000","last_fullsized":"2024-09-20T18:53:48.715567+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:00.970650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:00.970650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:07.049473+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":232,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.701723+0000","last_change":"2024-09-20T18:51:26.494194+0000","last_active":"2024-09-20T18:53:48.701723+0000","last_peered":"2024-09-20T18:53:48.701723+0000","last_clean":"2024-09-20T18:53:48.701723+0000","last_became_active":"2024-09-20T18:51:23.813399+0000","last_became_peered":"2024-09-20T18:51:23.813399+0000","last_unstale":"2024-09-20T18:53:48.701723+0000","last_undegraded":"2024-09-20T18:53:48.701723+0000","last_fullsized":"2024-09-20T18:53:48.701723+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-20T18:51:26.494118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:26.494118+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:34.803592+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":199,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.715280+0000","last_change":"2024-09-20T18:52:38.799542+0000","last_active":"2024-09-20T18:53:48.715280+0000","last_peered":"2024-09-20T18:53:48.715280+0000","last_clean":"2024-09-20T18:53:48.715280+0000","last_became_active":"2024-09-20T18:49:49.959696+0000","last_became_peered":"2024-09-20T18:49:49.959696+0000","last_unstale":"2024-09-20T18:53:48.715280+0000","last_undegraded":"2024-09-20T18:53:48.715280+0000","last_fullsized":"2024-09-20T18:53:48.715280+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:38.799487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:38.799487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:04.479508+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":195,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.714838+0000","last_change":"2024-09-20T18:53:21.273835+0000","last_active":"2024-09-20T18:53:48.714838+0000","last_peered":"2024-09-20T18:53:48.714838+0000","last_clean":"2024-09-20T18:53:48.714838+0000","last_became_active":"2024-09-20T18:49:51.970663+0000","last_became_peered":"2024-09-20T18:49:51.970663+0000","last_unstale":"2024-09-20T18:53:48.714838+0000","last_undegraded":"2024-09-20T18:53:48.714838+0000","last_fullsized":"2024-09-20T18:53:48.714838+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:21.273788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:21.273788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:31.269805+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":188,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.702274+0000","last_change":"2024-09-20T18:52:46.828811+0000","last_active":"2024-09-20T18:53:48.702274+0000","last_peered":"2024-09-20T18:53:48.702274+0000","last_clean":"2024-09-20T18:53:48.702274+0000","last_became_active":"2024-09-20T18:49:52.141170+0000","last_became_peered":"2024-09-20T18:49:52.141170+0000","last_unstale":"2024-09-20T18:53:48.702274+0000","last_undegraded":"2024-09-20T18:53:48.702274+0000","last_fullsized":"2024-09-20T18:53:48.702274+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:46.828749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:46.828749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:50.780880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":199,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.711752+0000","last_change":"2024-09-20T18:52:42.099489+0000","last_active":"2024-09-20T18:53:48.711752+0000","last_peered":"2024-09-20T18:53:48.711752+0000","last_clean":"2024-09-20T18:53:48.711752+0000","last_became_active":"2024-09-20T18:49:45.950480+0000","last_became_peered":"2024-09-20T18:49:45.950480+0000","last_unstale":"2024-09-20T18:53:48.711752+0000","last_undegraded":"2024-09-20T18:53:48.711752+0000","last_fullsized":"2024-09-20T18:53:48.711752+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:42.099418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:42.099418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:03.758400+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":230,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.709331+0000","last_change":"2024-09-20T18:53:26.762207+0000","last_active":"2024-09-20T18:53:48.709331+0000","last_peered":"2024-09-20T18:53:48.709331+0000","last_clean":"2024-09-20T18:53:48.709331+0000","last_became_active":"2024-09-20T18:49:47.950939+0000","last_became_peered":"2024-09-20T18:49:47.950939+0000","last_unstale":"2024-09-20T18:53:48.709331+0000","last_undegraded":"2024-09-20T18:53:48.709331+0000","last_fullsized":"2024-09-20T18:53:48.709331+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:53:26.762131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:26.762131+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:38.227744+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":192,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.721448+0000","last_change":"2024-09-20T18:51:17.248705+0000","last_active":"2024-09-20T18:53:48.721448+0000","last_peered":"2024-09-20T18:53:48.721448+0000","last_clean":"2024-09-20T18:53:48.721448+0000","last_became_active":"2024-09-20T18:49:49.958518+0000","last_became_peered":"2024-09-20T18:49:49.958518+0000","last_unstale":"2024-09-20T18:53:48.721448+0000","last_undegraded":"2024-09-20T18:53:48.721448+0000","last_fullsized":"2024-09-20T18:53:48.721448+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:17.248634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:17.248634+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:35.297969+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":188,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.719738+0000","last_change":"2024-09-20T18:52:03.023669+0000","last_active":"2024-09-20T18:53:48.719738+0000","last_peered":"2024-09-20T18:53:48.719738+0000","last_clean":"2024-09-20T18:53:48.719738+0000","last_became_active":"2024-09-20T18:49:52.140040+0000","last_became_peered":"2024-09-20T18:49:52.140040+0000","last_unstale":"2024-09-20T18:53:48.719738+0000","last_undegraded":"2024-09-20T18:53:48.719738+0000","last_fullsized":"2024-09-20T18:53:48.719738+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:03.023603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:03.023603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"59'1","reported_seq":214,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.701792+0000","last_change":"2024-09-20T18:53:42.840884+0000","last_active":"2024-09-20T18:53:48.701792+0000","last_peered":"2024-09-20T18:53:48.701792+0000","last_clean":"2024-09-20T18:53:48.701792+0000","last_became_active":"2024-09-20T18:49:45.940100+0000","last_became_peered":"2024-09-20T18:49:45.940100+0000","last_unstale":"2024-09-20T18:53:48.701792+0000","last_undegraded":"2024-09-20T18:53:48.701792+0000","last_fullsized":"2024-09-20T18:53:48.701792+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:53:42.840800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:42.840800+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:56.760029+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":211,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.725536+0000","last_change":"2024-09-20T18:49:47.951215+0000","last_active":"2024-09-20T18:53:48.725536+0000","last_peered":"2024-09-20T18:53:48.725536+0000","last_clean":"2024-09-20T18:53:48.725536+0000","last_became_active":"2024-09-20T18:49:47.950909+0000","last_became_peered":"2024-09-20T18:49:47.950909+0000","last_unstale":"2024-09-20T18:53:48.725536+0000","last_undegraded":"2024-09-20T18:53:48.725536+0000","last_fullsized":"2024-09-20T18:53:48.725536+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:00.096165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":200,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.701956+0000","last_change":"2024-09-20T18:53:12.820752+0000","last_active":"2024-09-20T18:53:48.701956+0000","last_peered":"2024-09-20T18:53:48.701956+0000","last_clean":"2024-09-20T18:53:48.701956+0000","last_became_active":"2024-09-20T18:49:49.959102+0000","last_became_peered":"2024-09-20T18:49:49.959102+0000","last_unstale":"2024-09-20T18:53:48.701956+0000","last_undegraded":"2024-09-20T18:53:48.701956+0000","last_fullsized":"2024-09-20T18:53:48.701956+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:12.820697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:12.820697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:29.865011+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":196,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.714754+0000","last_change":"2024-09-20T18:52:15.826819+0000","last_active":"2024-09-20T18:53:48.714754+0000","last_peered":"2024-09-20T18:53:48.714754+0000","last_clean":"2024-09-20T18:53:48.714754+0000","last_became_active":"2024-09-20T18:49:51.970058+0000","last_became_peered":"2024-09-20T18:49:51.970058+0000","last_unstale":"2024-09-20T18:53:48.714754+0000","last_undegraded":"2024-09-20T18:53:48.714754+0000","last_fullsized":"2024-09-20T18:53:48.714754+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T18:52:15.826638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:15.826638+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:28.538285+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":201,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.719427+0000","last_change":"2024-09-20T18:52:47.158384+0000","last_active":"2024-09-20T18:53:48.719427+0000","last_peered":"2024-09-20T18:53:48.719427+0000","last_clean":"2024-09-20T18:53:48.719427+0000","last_became_active":"2024-09-20T18:49:45.933078+0000","last_became_peered":"2024-09-20T18:49:45.933078+0000","last_unstale":"2024-09-20T18:53:48.719427+0000","last_undegraded":"2024-09-20T18:53:48.719427+0000","last_fullsized":"2024-09-20T18:53:48.719427+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:52:47.158260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:47.158260+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:57.719080+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":211,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.715787+0000","last_change":"2024-09-20T18:52:36.175063+0000","last_active":"2024-09-20T18:53:48.715787+0000","last_peered":"2024-09-20T18:53:48.715787+0000","last_clean":"2024-09-20T18:53:48.715787+0000","last_became_active":"2024-09-20T18:49:48.500809+0000","last_became_peered":"2024-09-20T18:49:48.500809+0000","last_unstale":"2024-09-20T18:53:48.715787+0000","last_undegraded":"2024-09-20T18:53:48.715787+0000","last_fullsized":"2024-09-20T18:53:48.715787+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:52:36.174993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:36.174993+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:57.398808+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":259,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.714207+0000","last_change":"2024-09-20T18:53:44.524093+0000","last_active":"2024-09-20T18:53:48.714207+0000","last_peered":"2024-09-20T18:53:48.714207+0000","last_clean":"2024-09-20T18:53:48.714207+0000","last_became_active":"2024-09-20T18:49:49.954772+0000","last_became_peered":"2024-09-20T18:49:49.954772+0000","last_unstale":"2024-09-20T18:53:48.714207+0000","last_undegraded":"2024-09-20T18:53:48.714207+0000","last_fullsized":"2024-09-20T18:53:48.714207+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:53:44.524040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:44.524040+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:03.085945+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.c","version":"149'13220","reported_seq":13475,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:50.750578+0000","last_change":"2024-09-20T18:53:47.244353+0000","last_active":"2024-09-20T18:53:50.750578+0000","last_peered":"2024-09-20T18:53:50.750578+0000","last_clean":"2024-09-20T18:53:50.750578+0000","last_became_active":"2024-09-20T18:50:49.843744+0000","last_became_peered":"2024-09-20T18:50:49.843744+0000","last_unstale":"2024-09-20T18:53:50.750578+0000","last_undegraded":"2024-09-20T18:53:50.750578+0000","last_fullsized":"2024-09-20T18:53:50.750578+0000","mapping_epoch":83,"log_start":"116'6900","ondisk_log_start":"116'6900","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'12973","last_scrub_stamp":"2024-09-20T18:53:47.243983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:53:47.243983+0000","objects_scrubbed":2483,"log_size":6320,"log_dups_size":0,"ondisk_log_size":6320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:08.150940+0000","scrub_duration":693,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":145358848,"num_objects":2218,"num_object_clones":0,"num_object_copies":6654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13220,"num_write_kb":494016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":213,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.782289+0000","last_change":"2024-09-20T18:53:48.782289+0000","last_active":"2024-09-20T18:53:48.782289+0000","last_peered":"2024-09-20T18:53:48.782289+0000","last_clean":"2024-09-20T18:53:48.782289+0000","last_became_active":"2024-09-20T18:49:58.992302+0000","last_became_peered":"2024-09-20T18:49:58.992302+0000","last_unstale":"2024-09-20T18:53:48.782289+0000","last_undegraded":"2024-09-20T18:53:48.782289+0000","last_fullsized":"2024-09-20T18:53:48.782289+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:53:48.782192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:48.782192+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:04.815274+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":200,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.714496+0000","last_change":"2024-09-20T18:53:26.175148+0000","last_active":"2024-09-20T18:53:48.714496+0000","last_peered":"2024-09-20T18:53:48.714496+0000","last_clean":"2024-09-20T18:53:48.714496+0000","last_became_active":"2024-09-20T18:49:45.936104+0000","last_became_peered":"2024-09-20T18:49:45.936104+0000","last_unstale":"2024-09-20T18:53:48.714496+0000","last_undegraded":"2024-09-20T18:53:48.714496+0000","last_fullsized":"2024-09-20T18:53:48.714496+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:26.175083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:26.175083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:49.575901+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":252,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.702297+0000","last_change":"2024-09-20T18:51:13.395980+0000","last_active":"2024-09-20T18:53:48.702297+0000","last_peered":"2024-09-20T18:53:48.702297+0000","last_clean":"2024-09-20T18:53:48.702297+0000","last_became_active":"2024-09-20T18:49:49.960697+0000","last_became_peered":"2024-09-20T18:49:49.960697+0000","last_unstale":"2024-09-20T18:53:48.702297+0000","last_undegraded":"2024-09-20T18:53:48.702297+0000","last_fullsized":"2024-09-20T18:53:48.702297+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:51:13.395908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:13.395908+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:07.764237+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":187,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.715442+0000","last_change":"2024-09-20T18:52:43.126389+0000","last_active":"2024-09-20T18:53:48.715442+0000","last_peered":"2024-09-20T18:53:48.715442+0000","last_clean":"2024-09-20T18:53:48.715442+0000","last_became_active":"2024-09-20T18:49:51.971399+0000","last_became_peered":"2024-09-20T18:49:51.971399+0000","last_unstale":"2024-09-20T18:53:48.715442+0000","last_undegraded":"2024-09-20T18:53:48.715442+0000","last_fullsized":"2024-09-20T18:53:48.715442+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:43.126320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:43.126320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:52.243850+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"149'12996","reported_seq":13140,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:51.125989+0000","last_change":"2024-09-20T18:50:49.841111+0000","last_active":"2024-09-20T18:53:51.125989+0000","last_peered":"2024-09-20T18:53:51.125989+0000","last_clean":"2024-09-20T18:53:51.125989+0000","last_became_active":"2024-09-20T18:50:49.840819+0000","last_became_peered":"2024-09-20T18:50:49.840819+0000","last_unstale":"2024-09-20T18:53:51.125989+0000","last_undegraded":"2024-09-20T18:53:51.125989+0000","last_fullsized":"2024-09-20T18:53:51.125989+0000","mapping_epoch":83,"log_start":"126'8300","ondisk_log_start":"126'8300","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.819660+0000","objects_scrubbed":0,"log_size":4696,"log_dups_size":0,"ondisk_log_size":4696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:38.259712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":146538496,"num_objects":2236,"num_object_clones":0,"num_object_copies":6708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12996,"num_write_kb":487424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":229,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.714333+0000","last_change":"2024-09-20T18:52:42.130129+0000","last_active":"2024-09-20T18:53:48.714333+0000","last_peered":"2024-09-20T18:53:48.714333+0000","last_clean":"2024-09-20T18:53:48.714333+0000","last_became_active":"2024-09-20T18:49:48.499189+0000","last_became_peered":"2024-09-20T18:49:48.499189+0000","last_unstale":"2024-09-20T18:53:48.714333+0000","last_undegraded":"2024-09-20T18:53:48.714333+0000","last_fullsized":"2024-09-20T18:53:48.714333+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-20T18:52:42.130045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:42.130045+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":192,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.720067+0000","last_change":"2024-09-20T18:49:45.940816+0000","last_active":"2024-09-20T18:53:48.720067+0000","last_peered":"2024-09-20T18:53:48.720067+0000","last_clean":"2024-09-20T18:53:48.720067+0000","last_became_active":"2024-09-20T18:49:45.940544+0000","last_became_peered":"2024-09-20T18:49:45.940544+0000","last_unstale":"2024-09-20T18:53:48.720067+0000","last_undegraded":"2024-09-20T18:53:48.720067+0000","last_fullsized":"2024-09-20T18:53:48.720067+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:56.100894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":200,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.701452+0000","last_change":"2024-09-20T18:53:09.779280+0000","last_active":"2024-09-20T18:53:48.701452+0000","last_peered":"2024-09-20T18:53:48.701452+0000","last_clean":"2024-09-20T18:53:48.701452+0000","last_became_active":"2024-09-20T18:49:49.963073+0000","last_became_peered":"2024-09-20T18:49:49.963073+0000","last_unstale":"2024-09-20T18:53:48.701452+0000","last_undegraded":"2024-09-20T18:53:48.701452+0000","last_fullsized":"2024-09-20T18:53:48.701452+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:09.779228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:09.779228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:39.433897+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":196,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.701712+0000","last_change":"2024-09-20T18:53:27.796357+0000","last_active":"2024-09-20T18:53:48.701712+0000","last_peered":"2024-09-20T18:53:48.701712+0000","last_clean":"2024-09-20T18:53:48.701712+0000","last_became_active":"2024-09-20T18:49:51.968793+0000","last_became_peered":"2024-09-20T18:49:51.968793+0000","last_unstale":"2024-09-20T18:53:48.701712+0000","last_undegraded":"2024-09-20T18:53:48.701712+0000","last_fullsized":"2024-09-20T18:53:48.701712+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:27.796289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:27.796289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:48.888717+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2","version":"149'13212","reported_seq":13353,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:51.132789+0000","last_change":"2024-09-20T18:50:49.842736+0000","last_active":"2024-09-20T18:53:51.132789+0000","last_peered":"2024-09-20T18:53:51.132789+0000","last_clean":"2024-09-20T18:53:51.132789+0000","last_became_active":"2024-09-20T18:50:49.842465+0000","last_became_peered":"2024-09-20T18:50:49.842465+0000","last_unstale":"2024-09-20T18:53:51.132789+0000","last_undegraded":"2024-09-20T18:53:51.132789+0000","last_fullsized":"2024-09-20T18:53:51.132789+0000","mapping_epoch":83,"log_start":"126'8500","ondisk_log_start":"126'8500","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.819660+0000","objects_scrubbed":0,"log_size":4712,"log_dups_size":0,"ondisk_log_size":4712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:31.286096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":142606336,"num_objects":2176,"num_object_clones":0,"num_object_copies":6528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13212,"num_write_kb":492416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":233,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.701480+0000","last_change":"2024-09-20T18:53:44.879133+0000","last_active":"2024-09-20T18:53:48.701480+0000","last_peered":"2024-09-20T18:53:48.701480+0000","last_clean":"2024-09-20T18:53:48.701480+0000","last_became_active":"2024-09-20T18:49:47.941895+0000","last_became_peered":"2024-09-20T18:49:47.941895+0000","last_unstale":"2024-09-20T18:53:48.701480+0000","last_undegraded":"2024-09-20T18:53:48.701480+0000","last_fullsized":"2024-09-20T18:53:48.701480+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-20T18:53:44.879048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:44.879048+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:52.038099+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":200,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.716252+0000","last_change":"2024-09-20T18:51:27.187060+0000","last_active":"2024-09-20T18:53:48.716252+0000","last_peered":"2024-09-20T18:53:48.716252+0000","last_clean":"2024-09-20T18:53:48.716252+0000","last_became_active":"2024-09-20T18:49:45.940469+0000","last_became_peered":"2024-09-20T18:49:45.940469+0000","last_unstale":"2024-09-20T18:53:48.716252+0000","last_undegraded":"2024-09-20T18:53:48.716252+0000","last_fullsized":"2024-09-20T18:53:48.716252+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:51:27.186991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:27.186991+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:53.996237+0000","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"66'8","reported_seq":258,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.714131+0000","last_change":"2024-09-20T18:53:44.293165+0000","last_active":"2024-09-20T18:53:48.714131+0000","last_peered":"2024-09-20T18:53:48.714131+0000","last_clean":"2024-09-20T18:53:48.714131+0000","last_became_active":"2024-09-20T18:49:49.956133+0000","last_became_peered":"2024-09-20T18:49:49.956133+0000","last_unstale":"2024-09-20T18:53:48.714131+0000","last_undegraded":"2024-09-20T18:53:48.714131+0000","last_fullsized":"2024-09-20T18:53:48.714131+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:53:44.293094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:44.293094+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:47.176037+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":195,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.715540+0000","last_change":"2024-09-20T18:52:42.880767+0000","last_active":"2024-09-20T18:53:48.715540+0000","last_peered":"2024-09-20T18:53:48.715540+0000","last_clean":"2024-09-20T18:53:48.715540+0000","last_became_active":"2024-09-20T18:49:52.142113+0000","last_became_peered":"2024-09-20T18:49:52.142113+0000","last_unstale":"2024-09-20T18:53:48.715540+0000","last_undegraded":"2024-09-20T18:53:48.715540+0000","last_fullsized":"2024-09-20T18:53:48.715540+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:42.880724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:42.880724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:04.690799+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"149'13237","reported_seq":13449,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:50.772855+0000","last_change":"2024-09-20T18:52:00.368447+0000","last_active":"2024-09-20T18:53:50.772855+0000","last_peered":"2024-09-20T18:53:50.772855+0000","last_clean":"2024-09-20T18:53:50.772855+0000","last_became_active":"2024-09-20T18:50:49.844624+0000","last_became_peered":"2024-09-20T18:50:49.844624+0000","last_unstale":"2024-09-20T18:53:50.772855+0000","last_undegraded":"2024-09-20T18:53:50.772855+0000","last_fullsized":"2024-09-20T18:53:50.772855+0000","mapping_epoch":83,"log_start":"126'8426","ondisk_log_start":"126'8426","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"111'6092","last_scrub_stamp":"2024-09-20T18:52:00.368276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:52:00.368276+0000","objects_scrubbed":6040,"log_size":4811,"log_dups_size":0,"ondisk_log_size":4811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:40.875204+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":144637952,"num_objects":2207,"num_object_clones":0,"num_object_copies":6621,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13237,"num_write_kb":494208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":219,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.718284+0000","last_change":"2024-09-20T18:52:39.246125+0000","last_active":"2024-09-20T18:53:48.718284+0000","last_peered":"2024-09-20T18:53:48.718284+0000","last_clean":"2024-09-20T18:53:48.718284+0000","last_became_active":"2024-09-20T18:49:47.949626+0000","last_became_peered":"2024-09-20T18:49:47.949626+0000","last_unstale":"2024-09-20T18:53:48.718284+0000","last_undegraded":"2024-09-20T18:53:48.718284+0000","last_fullsized":"2024-09-20T18:53:48.718284+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:52:39.246050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:39.246050+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:07.125503+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":207,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.715878+0000","last_change":"2024-09-20T18:53:11.130341+0000","last_active":"2024-09-20T18:53:48.715878+0000","last_peered":"2024-09-20T18:53:48.715878+0000","last_clean":"2024-09-20T18:53:48.715878+0000","last_became_active":"2024-09-20T18:49:45.942496+0000","last_became_peered":"2024-09-20T18:49:45.942496+0000","last_unstale":"2024-09-20T18:53:48.715878+0000","last_undegraded":"2024-09-20T18:53:48.715878+0000","last_fullsized":"2024-09-20T18:53:48.715878+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:11.130274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:11.130274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:17.935310+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":249,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.711447+0000","last_change":"2024-09-20T18:51:16.088407+0000","last_active":"2024-09-20T18:53:48.711447+0000","last_peered":"2024-09-20T18:53:48.711447+0000","last_clean":"2024-09-20T18:53:48.711447+0000","last_became_active":"2024-09-20T18:49:49.959365+0000","last_became_peered":"2024-09-20T18:49:49.959365+0000","last_unstale":"2024-09-20T18:53:48.711447+0000","last_undegraded":"2024-09-20T18:53:48.711447+0000","last_fullsized":"2024-09-20T18:53:48.711447+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:51:16.088343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:16.088343+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:35.866980+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":179,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.710286+0000","last_change":"2024-09-20T18:49:51.965733+0000","last_active":"2024-09-20T18:53:48.710286+0000","last_peered":"2024-09-20T18:53:48.710286+0000","last_clean":"2024-09-20T18:53:48.710286+0000","last_became_active":"2024-09-20T18:49:51.964780+0000","last_became_peered":"2024-09-20T18:49:51.964780+0000","last_unstale":"2024-09-20T18:53:48.710286+0000","last_undegraded":"2024-09-20T18:53:48.710286+0000","last_fullsized":"2024-09-20T18:53:48.710286+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:54.992353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.0","version":"149'13297","reported_seq":13538,"reported_epoch":149,"state":"active+clean+scrubbing","last_fresh":"2024-09-20T18:53:50.799163+0000","last_change":"2024-09-20T18:53:50.309253+0000","last_active":"2024-09-20T18:53:50.799163+0000","last_peered":"2024-09-20T18:53:50.799163+0000","last_clean":"2024-09-20T18:53:50.799163+0000","last_became_active":"2024-09-20T18:50:49.841019+0000","last_became_peered":"2024-09-20T18:50:49.841019+0000","last_unstale":"2024-09-20T18:53:50.799163+0000","last_undegraded":"2024-09-20T18:53:50.799163+0000","last_fullsized":"2024-09-20T18:53:50.799163+0000","mapping_epoch":83,"log_start":"120'7500","ondisk_log_start":"120'7500","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"116'6885","last_scrub_stamp":"2024-09-20T18:52:12.840191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:52:12.840191+0000","objects_scrubbed":1949,"log_size":5797,"log_dups_size":0,"ondisk_log_size":5797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"scrubbing for 0s","scrub_duration":1721,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":146210816,"num_objects":2231,"num_object_clones":0,"num_object_copies":6693,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13297,"num_write_kb":496896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":231,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.711586+0000","last_change":"2024-09-20T18:52:49.043282+0000","last_active":"2024-09-20T18:53:48.711586+0000","last_peered":"2024-09-20T18:53:48.711586+0000","last_clean":"2024-09-20T18:53:48.711586+0000","last_became_active":"2024-09-20T18:49:48.497736+0000","last_became_peered":"2024-09-20T18:49:48.497736+0000","last_unstale":"2024-09-20T18:53:48.711586+0000","last_undegraded":"2024-09-20T18:53:48.711586+0000","last_fullsized":"2024-09-20T18:53:48.711586+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-20T18:52:49.043189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:49.043189+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:54.819771+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":192,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.718931+0000","last_change":"2024-09-20T18:49:45.942589+0000","last_active":"2024-09-20T18:53:48.718931+0000","last_peered":"2024-09-20T18:53:48.718931+0000","last_clean":"2024-09-20T18:53:48.718931+0000","last_became_active":"2024-09-20T18:49:45.942253+0000","last_became_peered":"2024-09-20T18:49:45.942253+0000","last_unstale":"2024-09-20T18:53:48.718931+0000","last_undegraded":"2024-09-20T18:53:48.718931+0000","last_fullsized":"2024-09-20T18:53:48.718931+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:56.093800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":199,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.713647+0000","last_change":"2024-09-20T18:53:12.347123+0000","last_active":"2024-09-20T18:53:48.713647+0000","last_peered":"2024-09-20T18:53:48.713647+0000","last_clean":"2024-09-20T18:53:48.713647+0000","last_became_active":"2024-09-20T18:49:49.954299+0000","last_became_peered":"2024-09-20T18:49:49.954299+0000","last_unstale":"2024-09-20T18:53:48.713647+0000","last_undegraded":"2024-09-20T18:53:48.713647+0000","last_fullsized":"2024-09-20T18:53:48.713647+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:12.347044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:12.347044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:41.912430+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":196,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.719243+0000","last_change":"2024-09-20T18:53:25.362907+0000","last_active":"2024-09-20T18:53:48.719243+0000","last_peered":"2024-09-20T18:53:48.719243+0000","last_clean":"2024-09-20T18:53:48.719243+0000","last_became_active":"2024-09-20T18:49:52.139981+0000","last_became_peered":"2024-09-20T18:49:52.139981+0000","last_unstale":"2024-09-20T18:53:48.719243+0000","last_undegraded":"2024-09-20T18:53:48.719243+0000","last_fullsized":"2024-09-20T18:53:48.719243+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:25.362828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:25.362828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:55.107515+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"149'12967","reported_seq":13179,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:50.802200+0000","last_change":"2024-09-20T18:52:03.401113+0000","last_active":"2024-09-20T18:53:50.802200+0000","last_peered":"2024-09-20T18:53:50.802200+0000","last_clean":"2024-09-20T18:53:50.802200+0000","last_became_active":"2024-09-20T18:50:49.842473+0000","last_became_peered":"2024-09-20T18:50:49.842473+0000","last_unstale":"2024-09-20T18:53:50.802200+0000","last_undegraded":"2024-09-20T18:53:50.802200+0000","last_fullsized":"2024-09-20T18:53:50.802200+0000","mapping_epoch":83,"log_start":"116'6700","ondisk_log_start":"116'6700","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"112'6142","last_scrub_stamp":"2024-09-20T18:52:03.400886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:52:03.400886+0000","objects_scrubbed":6079,"log_size":6267,"log_dups_size":0,"ondisk_log_size":6267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:41.532845+0000","scrub_duration":2059,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":141623296,"num_objects":2161,"num_object_clones":0,"num_object_copies":6483,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2161,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12967,"num_write_kb":484096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"66'28","reported_seq":293,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.718774+0000","last_change":"2024-09-20T18:51:57.293659+0000","last_active":"2024-09-20T18:53:48.718774+0000","last_peered":"2024-09-20T18:53:48.718774+0000","last_clean":"2024-09-20T18:53:48.718774+0000","last_became_active":"2024-09-20T18:49:47.955335+0000","last_became_peered":"2024-09-20T18:49:47.955335+0000","last_unstale":"2024-09-20T18:53:48.718774+0000","last_undegraded":"2024-09-20T18:53:48.718774+0000","last_fullsized":"2024-09-20T18:53:48.718774+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'28","last_scrub_stamp":"2024-09-20T18:51:57.293574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:57.293574+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:06.036152+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":192,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.701986+0000","last_change":"2024-09-20T18:49:45.945275+0000","last_active":"2024-09-20T18:53:48.701986+0000","last_peered":"2024-09-20T18:53:48.701986+0000","last_clean":"2024-09-20T18:53:48.701986+0000","last_became_active":"2024-09-20T18:49:45.944992+0000","last_became_peered":"2024-09-20T18:49:45.944992+0000","last_unstale":"2024-09-20T18:53:48.701986+0000","last_undegraded":"2024-09-20T18:53:48.701986+0000","last_fullsized":"2024-09-20T18:53:48.701986+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:54.777986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":225,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:49.323451+0000","last_change":"2024-09-20T18:53:49.323451+0000","last_active":"2024-09-20T18:53:49.323451+0000","last_peered":"2024-09-20T18:53:49.323451+0000","last_clean":"2024-09-20T18:53:49.323451+0000","last_became_active":"2024-09-20T18:49:44.936027+0000","last_became_peered":"2024-09-20T18:49:44.936027+0000","last_unstale":"2024-09-20T18:53:49.323451+0000","last_undegraded":"2024-09-20T18:53:49.323451+0000","last_fullsized":"2024-09-20T18:53:49.323451+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-20T18:53:49.323385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:53:49.323385+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:50.958195+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.0012781159999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":199,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.711405+0000","last_change":"2024-09-20T18:53:12.897519+0000","last_active":"2024-09-20T18:53:48.711405+0000","last_peered":"2024-09-20T18:53:48.711405+0000","last_clean":"2024-09-20T18:53:48.711405+0000","last_became_active":"2024-09-20T18:49:49.953971+0000","last_became_peered":"2024-09-20T18:49:49.953971+0000","last_unstale":"2024-09-20T18:53:48.711405+0000","last_undegraded":"2024-09-20T18:53:48.711405+0000","last_fullsized":"2024-09-20T18:53:48.711405+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:12.897051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:12.897051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:39.168470+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":180,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.713723+0000","last_change":"2024-09-20T18:49:51.969702+0000","last_active":"2024-09-20T18:53:48.713723+0000","last_peered":"2024-09-20T18:53:48.713723+0000","last_clean":"2024-09-20T18:53:48.713723+0000","last_became_active":"2024-09-20T18:49:51.969400+0000","last_became_peered":"2024-09-20T18:49:51.969400+0000","last_unstale":"2024-09-20T18:53:48.713723+0000","last_undegraded":"2024-09-20T18:53:48.713723+0000","last_fullsized":"2024-09-20T18:53:48.713723+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:36.285843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"149'13026","reported_seq":13217,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:51.130559+0000","last_change":"2024-09-20T18:53:23.208574+0000","last_active":"2024-09-20T18:53:51.130559+0000","last_peered":"2024-09-20T18:53:51.130559+0000","last_clean":"2024-09-20T18:53:51.130559+0000","last_became_active":"2024-09-20T18:50:49.843321+0000","last_became_peered":"2024-09-20T18:50:49.843321+0000","last_unstale":"2024-09-20T18:53:51.130559+0000","last_undegraded":"2024-09-20T18:53:51.130559+0000","last_fullsized":"2024-09-20T18:53:51.130559+0000","mapping_epoch":83,"log_start":"120'7400","ondisk_log_start":"120'7400","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"140'11334","last_scrub_stamp":"2024-09-20T18:53:23.208497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:53:23.208497+0000","objects_scrubbed":3959,"log_size":5626,"log_dups_size":0,"ondisk_log_size":5626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:48.313048+0000","scrub_duration":1357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":145752064,"num_objects":2224,"num_object_clones":0,"num_object_copies":6672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13026,"num_write_kb":488000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":231,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.718546+0000","last_change":"2024-09-20T18:53:48.214566+0000","last_active":"2024-09-20T18:53:48.718546+0000","last_peered":"2024-09-20T18:53:48.718546+0000","last_clean":"2024-09-20T18:53:48.718546+0000","last_became_active":"2024-09-20T18:49:47.950904+0000","last_became_peered":"2024-09-20T18:49:47.950904+0000","last_unstale":"2024-09-20T18:53:48.718546+0000","last_undegraded":"2024-09-20T18:53:48.718546+0000","last_fullsized":"2024-09-20T18:53:48.718546+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-20T18:53:48.214491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:48.214491+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:15.858989+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":209,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.719043+0000","last_change":"2024-09-20T18:53:19.278005+0000","last_active":"2024-09-20T18:53:48.719043+0000","last_peered":"2024-09-20T18:53:48.719043+0000","last_clean":"2024-09-20T18:53:48.719043+0000","last_became_active":"2024-09-20T18:49:45.939884+0000","last_became_peered":"2024-09-20T18:49:45.939884+0000","last_unstale":"2024-09-20T18:53:48.719043+0000","last_undegraded":"2024-09-20T18:53:48.719043+0000","last_fullsized":"2024-09-20T18:53:48.719043+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:19.277650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:19.277650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:22.181893+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":211,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.707235+0000","last_change":"2024-09-20T18:52:07.718766+0000","last_active":"2024-09-20T18:53:48.707235+0000","last_peered":"2024-09-20T18:53:48.707235+0000","last_clean":"2024-09-20T18:53:48.707235+0000","last_became_active":"2024-09-20T18:49:44.937135+0000","last_became_peered":"2024-09-20T18:49:44.937135+0000","last_unstale":"2024-09-20T18:53:48.707235+0000","last_undegraded":"2024-09-20T18:53:48.707235+0000","last_fullsized":"2024-09-20T18:53:48.707235+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:52:07.718646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:52:07.718646+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:37.805116+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.00095558800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":193,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.709145+0000","last_change":"2024-09-20T18:51:59.796450+0000","last_active":"2024-09-20T18:53:48.709145+0000","last_peered":"2024-09-20T18:53:48.709145+0000","last_clean":"2024-09-20T18:53:48.709145+0000","last_became_active":"2024-09-20T18:49:49.954424+0000","last_became_peered":"2024-09-20T18:49:49.954424+0000","last_unstale":"2024-09-20T18:53:48.709145+0000","last_undegraded":"2024-09-20T18:53:48.709145+0000","last_fullsized":"2024-09-20T18:53:48.709145+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:59.796383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:59.796383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:03.718427+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":187,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.715205+0000","last_change":"2024-09-20T18:52:39.814324+0000","last_active":"2024-09-20T18:53:48.715205+0000","last_peered":"2024-09-20T18:53:48.715205+0000","last_clean":"2024-09-20T18:53:48.715205+0000","last_became_active":"2024-09-20T18:49:52.139789+0000","last_became_peered":"2024-09-20T18:49:52.139789+0000","last_unstale":"2024-09-20T18:53:48.715205+0000","last_undegraded":"2024-09-20T18:53:48.715205+0000","last_fullsized":"2024-09-20T18:53:48.715205+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:39.814259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:39.814259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:55.444061+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"149'13273","reported_seq":13539,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:50.785975+0000","last_change":"2024-09-20T18:53:24.693140+0000","last_active":"2024-09-20T18:53:50.785975+0000","last_peered":"2024-09-20T18:53:50.785975+0000","last_clean":"2024-09-20T18:53:50.785975+0000","last_became_active":"2024-09-20T18:50:49.846124+0000","last_became_peered":"2024-09-20T18:50:49.846124+0000","last_unstale":"2024-09-20T18:53:50.785975+0000","last_undegraded":"2024-09-20T18:53:50.785975+0000","last_fullsized":"2024-09-20T18:53:50.785975+0000","mapping_epoch":83,"log_start":"124'8100","ondisk_log_start":"124'8100","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"140'11593","last_scrub_stamp":"2024-09-20T18:53:24.692917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:53:24.692917+0000","objects_scrubbed":4004,"log_size":5173,"log_dups_size":0,"ondisk_log_size":5173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:35.585228+0000","scrub_duration":1122,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":149356544,"num_objects":2279,"num_object_clones":0,"num_object_copies":6837,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13273,"num_write_kb":497664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":229,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.715928+0000","last_change":"2024-09-20T18:53:47.786753+0000","last_active":"2024-09-20T18:53:48.715928+0000","last_peered":"2024-09-20T18:53:48.715928+0000","last_clean":"2024-09-20T18:53:48.715928+0000","last_became_active":"2024-09-20T18:49:47.949230+0000","last_became_peered":"2024-09-20T18:49:47.949230+0000","last_unstale":"2024-09-20T18:53:48.715928+0000","last_undegraded":"2024-09-20T18:53:48.715928+0000","last_fullsized":"2024-09-20T18:53:48.715928+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:53:47.786681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:47.786681+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:54.774464+0000","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":207,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.710813+0000","last_change":"2024-09-20T18:53:09.891235+0000","last_active":"2024-09-20T18:53:48.710813+0000","last_peered":"2024-09-20T18:53:48.710813+0000","last_clean":"2024-09-20T18:53:48.710813+0000","last_became_active":"2024-09-20T18:49:45.939197+0000","last_became_peered":"2024-09-20T18:49:45.939197+0000","last_unstale":"2024-09-20T18:53:48.710813+0000","last_undegraded":"2024-09-20T18:53:48.710813+0000","last_fullsized":"2024-09-20T18:53:48.710813+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:09.890987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:09.890987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:17.075156+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":459,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:49.584693+0000","last_change":"2024-09-20T18:53:22.616236+0000","last_active":"2024-09-20T18:53:49.584693+0000","last_peered":"2024-09-20T18:53:49.584693+0000","last_clean":"2024-09-20T18:53:49.584693+0000","last_became_active":"2024-09-20T18:49:44.936819+0000","last_became_peered":"2024-09-20T18:49:44.936819+0000","last_unstale":"2024-09-20T18:53:49.584693+0000","last_undegraded":"2024-09-20T18:53:49.584693+0000","last_fullsized":"2024-09-20T18:53:49.584693+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-20T18:53:22.616166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:53:22.616166+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:25.872696+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0010996199999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":240,"num_read_kb":235,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":191,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.716139+0000","last_change":"2024-09-20T18:52:02.981845+0000","last_active":"2024-09-20T18:53:48.716139+0000","last_peered":"2024-09-20T18:53:48.716139+0000","last_clean":"2024-09-20T18:53:48.716139+0000","last_became_active":"2024-09-20T18:49:49.961066+0000","last_became_peered":"2024-09-20T18:49:49.961066+0000","last_unstale":"2024-09-20T18:53:48.716139+0000","last_undegraded":"2024-09-20T18:53:48.716139+0000","last_fullsized":"2024-09-20T18:53:48.716139+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:02.981762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:02.981762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":196,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:49.227504+0000","last_change":"2024-09-20T18:53:49.227504+0000","last_active":"2024-09-20T18:53:49.227504+0000","last_peered":"2024-09-20T18:53:49.227504+0000","last_clean":"2024-09-20T18:53:49.227504+0000","last_became_active":"2024-09-20T18:49:51.970343+0000","last_became_peered":"2024-09-20T18:49:51.970343+0000","last_unstale":"2024-09-20T18:53:49.227504+0000","last_undegraded":"2024-09-20T18:53:49.227504+0000","last_fullsized":"2024-09-20T18:53:49.227504+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:49.227436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:49.227436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:07.422442+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"149'13241","reported_seq":13461,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:51.130089+0000","last_change":"2024-09-20T18:52:13.298242+0000","last_active":"2024-09-20T18:53:51.130089+0000","last_peered":"2024-09-20T18:53:51.130089+0000","last_clean":"2024-09-20T18:53:51.130089+0000","last_became_active":"2024-09-20T18:50:49.842310+0000","last_became_peered":"2024-09-20T18:50:49.842310+0000","last_unstale":"2024-09-20T18:53:51.130089+0000","last_undegraded":"2024-09-20T18:53:51.130089+0000","last_fullsized":"2024-09-20T18:53:51.130089+0000","mapping_epoch":83,"log_start":"122'7600","ondisk_log_start":"122'7600","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"116'6867","last_scrub_stamp":"2024-09-20T18:52:13.297990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:52:13.297990+0000","objects_scrubbed":6775,"log_size":5641,"log_dups_size":0,"ondisk_log_size":5641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:40.850770+0000","scrub_duration":3018,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":141099008,"num_objects":2153,"num_object_clones":0,"num_object_copies":6459,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2153,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13241,"num_write_kb":492608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":258,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.713779+0000","last_change":"2024-09-20T18:49:48.498345+0000","last_active":"2024-09-20T18:53:48.713779+0000","last_peered":"2024-09-20T18:53:48.713779+0000","last_clean":"2024-09-20T18:53:48.713779+0000","last_became_active":"2024-09-20T18:49:48.497936+0000","last_became_peered":"2024-09-20T18:49:48.497936+0000","last_unstale":"2024-09-20T18:53:48.713779+0000","last_undegraded":"2024-09-20T18:53:48.713779+0000","last_fullsized":"2024-09-20T18:53:48.713779+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:57.367087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":208,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.715773+0000","last_change":"2024-09-20T18:53:13.145127+0000","last_active":"2024-09-20T18:53:48.715773+0000","last_peered":"2024-09-20T18:53:48.715773+0000","last_clean":"2024-09-20T18:53:48.715773+0000","last_became_active":"2024-09-20T18:49:45.941136+0000","last_became_peered":"2024-09-20T18:49:45.941136+0000","last_unstale":"2024-09-20T18:53:48.715773+0000","last_undegraded":"2024-09-20T18:53:48.715773+0000","last_fullsized":"2024-09-20T18:53:48.715773+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:13.144896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:13.144896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:18.711819+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":247,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.715098+0000","last_change":"2024-09-20T18:52:11.887878+0000","last_active":"2024-09-20T18:53:48.715098+0000","last_peered":"2024-09-20T18:53:48.715098+0000","last_clean":"2024-09-20T18:53:48.715098+0000","last_became_active":"2024-09-20T18:49:36.545306+0000","last_became_peered":"2024-09-20T18:49:36.545306+0000","last_unstale":"2024-09-20T18:53:48.715098+0000","last_undegraded":"2024-09-20T18:53:48.715098+0000","last_fullsized":"2024-09-20T18:53:48.715098+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T18:52:11.887733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T18:52:11.887733+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:32.438028+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":191,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.714966+0000","last_change":"2024-09-20T18:51:22.852942+0000","last_active":"2024-09-20T18:53:48.714966+0000","last_peered":"2024-09-20T18:53:48.714966+0000","last_clean":"2024-09-20T18:53:48.714966+0000","last_became_active":"2024-09-20T18:49:49.956128+0000","last_became_peered":"2024-09-20T18:49:49.956128+0000","last_unstale":"2024-09-20T18:53:48.714966+0000","last_undegraded":"2024-09-20T18:53:48.714966+0000","last_fullsized":"2024-09-20T18:53:48.714966+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:22.852869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:22.852869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:37.516435+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":195,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.715930+0000","last_change":"2024-09-20T18:53:45.306565+0000","last_active":"2024-09-20T18:53:48.715930+0000","last_peered":"2024-09-20T18:53:48.715930+0000","last_clean":"2024-09-20T18:53:48.715930+0000","last_became_active":"2024-09-20T18:49:51.971332+0000","last_became_peered":"2024-09-20T18:49:51.971332+0000","last_unstale":"2024-09-20T18:53:48.715930+0000","last_undegraded":"2024-09-20T18:53:48.715930+0000","last_fullsized":"2024-09-20T18:53:48.715930+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:45.306514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:45.306514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:10.093423+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"149'13116","reported_seq":13395,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:50.786322+0000","last_change":"2024-09-20T18:52:51.095285+0000","last_active":"2024-09-20T18:53:50.786322+0000","last_peered":"2024-09-20T18:53:50.786322+0000","last_clean":"2024-09-20T18:53:50.786322+0000","last_became_active":"2024-09-20T18:50:49.842483+0000","last_became_peered":"2024-09-20T18:50:49.842483+0000","last_unstale":"2024-09-20T18:53:50.786322+0000","last_undegraded":"2024-09-20T18:53:50.786322+0000","last_fullsized":"2024-09-20T18:53:50.786322+0000","mapping_epoch":83,"log_start":"116'6800","ondisk_log_start":"116'6800","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"131'9234","last_scrub_stamp":"2024-09-20T18:52:51.095167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:52:51.095167+0000","objects_scrubbed":6083,"log_size":6316,"log_dups_size":0,"ondisk_log_size":6316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:09.470192+0000","scrub_duration":1756,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":139919384,"num_objects":2136,"num_object_clones":0,"num_object_copies":6408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2136,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13116,"num_write_kb":488001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":215,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.702351+0000","last_change":"2024-09-20T18:51:14.397996+0000","last_active":"2024-09-20T18:53:48.702351+0000","last_peered":"2024-09-20T18:53:48.702351+0000","last_clean":"2024-09-20T18:53:48.702351+0000","last_became_active":"2024-09-20T18:49:47.950073+0000","last_became_peered":"2024-09-20T18:49:47.950073+0000","last_unstale":"2024-09-20T18:53:48.702351+0000","last_undegraded":"2024-09-20T18:53:48.702351+0000","last_fullsized":"2024-09-20T18:53:48.702351+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:51:14.397930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.397930+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:32.825060+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":200,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.701692+0000","last_change":"2024-09-20T18:51:18.410601+0000","last_active":"2024-09-20T18:53:48.701692+0000","last_peered":"2024-09-20T18:53:48.701692+0000","last_clean":"2024-09-20T18:53:48.701692+0000","last_became_active":"2024-09-20T18:49:45.944579+0000","last_became_peered":"2024-09-20T18:49:45.944579+0000","last_unstale":"2024-09-20T18:53:48.701692+0000","last_undegraded":"2024-09-20T18:53:48.701692+0000","last_fullsized":"2024-09-20T18:53:48.701692+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:18.410447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:18.410447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:29.855147+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":199,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.715756+0000","last_change":"2024-09-20T18:52:38.128487+0000","last_active":"2024-09-20T18:53:48.715756+0000","last_peered":"2024-09-20T18:53:48.715756+0000","last_clean":"2024-09-20T18:53:48.715756+0000","last_became_active":"2024-09-20T18:49:49.954391+0000","last_became_peered":"2024-09-20T18:49:49.954391+0000","last_unstale":"2024-09-20T18:53:48.715756+0000","last_undegraded":"2024-09-20T18:53:48.715756+0000","last_fullsized":"2024-09-20T18:53:48.715756+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:38.128438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:38.128438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:55.529118+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":188,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.714577+0000","last_change":"2024-09-20T18:52:05.021228+0000","last_active":"2024-09-20T18:53:48.714577+0000","last_peered":"2024-09-20T18:53:48.714577+0000","last_clean":"2024-09-20T18:53:48.714577+0000","last_became_active":"2024-09-20T18:49:52.139959+0000","last_became_peered":"2024-09-20T18:49:52.139959+0000","last_unstale":"2024-09-20T18:53:48.714577+0000","last_undegraded":"2024-09-20T18:53:48.714577+0000","last_fullsized":"2024-09-20T18:53:48.714577+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:05.021071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.021071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"149'13074","reported_seq":13268,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:50.966497+0000","last_change":"2024-09-20T18:53:22.451111+0000","last_active":"2024-09-20T18:53:50.966497+0000","last_peered":"2024-09-20T18:53:50.966497+0000","last_clean":"2024-09-20T18:53:50.966497+0000","last_became_active":"2024-09-20T18:50:49.841683+0000","last_became_peered":"2024-09-20T18:50:49.841683+0000","last_unstale":"2024-09-20T18:53:50.966497+0000","last_undegraded":"2024-09-20T18:53:50.966497+0000","last_fullsized":"2024-09-20T18:53:50.966497+0000","mapping_epoch":83,"log_start":"112'6200","ondisk_log_start":"112'6200","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'11259","last_scrub_stamp":"2024-09-20T18:53:22.450961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:53:22.450961+0000","objects_scrubbed":4058,"log_size":6874,"log_dups_size":0,"ondisk_log_size":6874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:26.478768+0000","scrub_duration":1718,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":143130624,"num_objects":2184,"num_object_clones":0,"num_object_copies":6552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13074,"num_write_kb":488256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":232,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.702198+0000","last_change":"2024-09-20T18:52:02.944536+0000","last_active":"2024-09-20T18:53:48.702198+0000","last_peered":"2024-09-20T18:53:48.702198+0000","last_clean":"2024-09-20T18:53:48.702198+0000","last_became_active":"2024-09-20T18:49:47.947636+0000","last_became_peered":"2024-09-20T18:49:47.947636+0000","last_unstale":"2024-09-20T18:53:48.702198+0000","last_undegraded":"2024-09-20T18:53:48.702198+0000","last_fullsized":"2024-09-20T18:53:48.702198+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-20T18:52:02.944434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:02.944434+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:30.835162+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":221,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.715764+0000","last_change":"2024-09-20T18:53:23.472809+0000","last_active":"2024-09-20T18:53:48.715764+0000","last_peered":"2024-09-20T18:53:48.715764+0000","last_clean":"2024-09-20T18:53:48.715764+0000","last_became_active":"2024-09-20T18:51:55.596689+0000","last_became_peered":"2024-09-20T18:51:55.596689+0000","last_unstale":"2024-09-20T18:53:48.715764+0000","last_undegraded":"2024-09-20T18:53:48.715764+0000","last_fullsized":"2024-09-20T18:53:48.715764+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:23.472757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:23.472757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:39.082278+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":192,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.718649+0000","last_change":"2024-09-20T18:52:49.137080+0000","last_active":"2024-09-20T18:53:48.718649+0000","last_peered":"2024-09-20T18:53:48.718649+0000","last_clean":"2024-09-20T18:53:48.718649+0000","last_became_active":"2024-09-20T18:49:49.959160+0000","last_became_peered":"2024-09-20T18:49:49.959160+0000","last_unstale":"2024-09-20T18:53:48.718649+0000","last_undegraded":"2024-09-20T18:53:48.718649+0000","last_fullsized":"2024-09-20T18:53:48.718649+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:49.137023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:49.137023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:07.149566+0000","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":197,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.707343+0000","last_change":"2024-09-20T18:52:35.635897+0000","last_active":"2024-09-20T18:53:48.707343+0000","last_peered":"2024-09-20T18:53:48.707343+0000","last_clean":"2024-09-20T18:53:48.707343+0000","last_became_active":"2024-09-20T18:49:51.970108+0000","last_became_peered":"2024-09-20T18:49:51.970108+0000","last_unstale":"2024-09-20T18:53:48.707343+0000","last_undegraded":"2024-09-20T18:53:48.707343+0000","last_fullsized":"2024-09-20T18:53:48.707343+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:35.635818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:35.635818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:56.643452+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"149'13272","reported_seq":13525,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:51.122884+0000","last_change":"2024-09-20T18:52:52.583857+0000","last_active":"2024-09-20T18:53:51.122884+0000","last_peered":"2024-09-20T18:53:51.122884+0000","last_clean":"2024-09-20T18:53:51.122884+0000","last_became_active":"2024-09-20T18:50:49.841628+0000","last_became_peered":"2024-09-20T18:50:49.841628+0000","last_unstale":"2024-09-20T18:53:51.122884+0000","last_undegraded":"2024-09-20T18:53:51.122884+0000","last_fullsized":"2024-09-20T18:53:51.122884+0000","mapping_epoch":83,"log_start":"122'7700","ondisk_log_start":"122'7700","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"131'9412","last_scrub_stamp":"2024-09-20T18:52:52.583713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:52:52.583713+0000","objects_scrubbed":6144,"log_size":5572,"log_dups_size":0,"ondisk_log_size":5572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:55.820131+0000","scrub_duration":2542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":144441344,"num_objects":2204,"num_object_clones":0,"num_object_copies":6612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2204,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13272,"num_write_kb":495232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":211,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.701981+0000","last_change":"2024-09-20T18:52:52.775966+0000","last_active":"2024-09-20T18:53:48.701981+0000","last_peered":"2024-09-20T18:53:48.701981+0000","last_clean":"2024-09-20T18:53:48.701981+0000","last_became_active":"2024-09-20T18:49:47.947126+0000","last_became_peered":"2024-09-20T18:49:47.947126+0000","last_unstale":"2024-09-20T18:53:48.701981+0000","last_undegraded":"2024-09-20T18:53:48.701981+0000","last_fullsized":"2024-09-20T18:53:48.701981+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:52:52.775889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:52.775889+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:14.126093+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":208,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.714846+0000","last_change":"2024-09-20T18:52:46.005172+0000","last_active":"2024-09-20T18:53:48.714846+0000","last_peered":"2024-09-20T18:53:48.714846+0000","last_clean":"2024-09-20T18:53:48.714846+0000","last_became_active":"2024-09-20T18:49:45.937264+0000","last_became_peered":"2024-09-20T18:49:45.937264+0000","last_unstale":"2024-09-20T18:53:48.714846+0000","last_undegraded":"2024-09-20T18:53:48.714846+0000","last_fullsized":"2024-09-20T18:53:48.714846+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:46.005109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:46.005109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:04.505427+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":258,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.707485+0000","last_change":"2024-09-20T18:52:40.647833+0000","last_active":"2024-09-20T18:53:48.707485+0000","last_peered":"2024-09-20T18:53:48.707485+0000","last_clean":"2024-09-20T18:53:48.707485+0000","last_became_active":"2024-09-20T18:49:49.957005+0000","last_became_peered":"2024-09-20T18:49:49.957005+0000","last_unstale":"2024-09-20T18:53:48.707485+0000","last_undegraded":"2024-09-20T18:53:48.707485+0000","last_fullsized":"2024-09-20T18:53:48.707485+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:52:40.647761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:40.647761+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:03.992292+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":188,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.702208+0000","last_change":"2024-09-20T18:52:10.024610+0000","last_active":"2024-09-20T18:53:48.702208+0000","last_peered":"2024-09-20T18:53:48.702208+0000","last_clean":"2024-09-20T18:53:48.702208+0000","last_became_active":"2024-09-20T18:49:51.969355+0000","last_became_peered":"2024-09-20T18:49:51.969355+0000","last_unstale":"2024-09-20T18:53:48.702208+0000","last_undegraded":"2024-09-20T18:53:48.702208+0000","last_fullsized":"2024-09-20T18:53:48.702208+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:10.024557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:10.024557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:38.850775+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.8","version":"149'13182","reported_seq":13404,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:50.960195+0000","last_change":"2024-09-20T18:52:16.931599+0000","last_active":"2024-09-20T18:53:50.960195+0000","last_peered":"2024-09-20T18:53:50.960195+0000","last_clean":"2024-09-20T18:53:50.960195+0000","last_became_active":"2024-09-20T18:50:49.844028+0000","last_became_peered":"2024-09-20T18:50:49.844028+0000","last_unstale":"2024-09-20T18:53:50.960195+0000","last_undegraded":"2024-09-20T18:53:50.960195+0000","last_fullsized":"2024-09-20T18:53:50.960195+0000","mapping_epoch":83,"log_start":"112'6280","ondisk_log_start":"112'6280","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"118'7065","last_scrub_stamp":"2024-09-20T18:52:16.931418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:52:16.931418+0000","objects_scrubbed":7008,"log_size":6902,"log_dups_size":0,"ondisk_log_size":6902,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:31.738778+0000","scrub_duration":2274,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":147324928,"num_objects":2248,"num_object_clones":0,"num_object_copies":6744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2248,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13182,"num_write_kb":493760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":219,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.711499+0000","last_change":"2024-09-20T18:53:17.863709+0000","last_active":"2024-09-20T18:53:48.711499+0000","last_peered":"2024-09-20T18:53:48.711499+0000","last_clean":"2024-09-20T18:53:48.711499+0000","last_became_active":"2024-09-20T18:49:47.946008+0000","last_became_peered":"2024-09-20T18:49:47.946008+0000","last_unstale":"2024-09-20T18:53:48.711499+0000","last_undegraded":"2024-09-20T18:53:48.711499+0000","last_fullsized":"2024-09-20T18:53:48.711499+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:53:17.863623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:17.863623+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:35.672196+0000","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":207,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.715713+0000","last_change":"2024-09-20T18:52:17.120051+0000","last_active":"2024-09-20T18:53:48.715713+0000","last_peered":"2024-09-20T18:53:48.715713+0000","last_clean":"2024-09-20T18:53:48.715713+0000","last_became_active":"2024-09-20T18:49:45.941406+0000","last_became_peered":"2024-09-20T18:49:45.941406+0000","last_unstale":"2024-09-20T18:53:48.715713+0000","last_undegraded":"2024-09-20T18:53:48.715713+0000","last_fullsized":"2024-09-20T18:53:48.715713+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:17.119986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:17.119986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":201,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.710517+0000","last_change":"2024-09-20T18:53:27.722068+0000","last_active":"2024-09-20T18:53:48.710517+0000","last_peered":"2024-09-20T18:53:48.710517+0000","last_clean":"2024-09-20T18:53:48.710517+0000","last_became_active":"2024-09-20T18:49:49.956767+0000","last_became_peered":"2024-09-20T18:49:49.956767+0000","last_unstale":"2024-09-20T18:53:48.710517+0000","last_undegraded":"2024-09-20T18:53:48.710517+0000","last_fullsized":"2024-09-20T18:53:48.710517+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:27.721940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:27.721940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:42.447352+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":187,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.710723+0000","last_change":"2024-09-20T18:52:46.004013+0000","last_active":"2024-09-20T18:53:48.710723+0000","last_peered":"2024-09-20T18:53:48.710723+0000","last_clean":"2024-09-20T18:53:48.710723+0000","last_became_active":"2024-09-20T18:49:51.964098+0000","last_became_peered":"2024-09-20T18:49:51.964098+0000","last_unstale":"2024-09-20T18:53:48.710723+0000","last_undegraded":"2024-09-20T18:53:48.710723+0000","last_fullsized":"2024-09-20T18:53:48.710723+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:46.003836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:46.003836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:04.821993+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"149'13018","reported_seq":13271,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:51.140895+0000","last_change":"2024-09-20T18:53:43.753759+0000","last_active":"2024-09-20T18:53:51.140895+0000","last_peered":"2024-09-20T18:53:51.140895+0000","last_clean":"2024-09-20T18:53:51.140895+0000","last_became_active":"2024-09-20T18:50:49.840023+0000","last_became_peered":"2024-09-20T18:50:49.840023+0000","last_unstale":"2024-09-20T18:53:51.140895+0000","last_undegraded":"2024-09-20T18:53:51.140895+0000","last_fullsized":"2024-09-20T18:53:51.140895+0000","mapping_epoch":83,"log_start":"120'7400","ondisk_log_start":"120'7400","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'12545","last_scrub_stamp":"2024-09-20T18:53:43.753508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:53:43.753508+0000","objects_scrubbed":2735,"log_size":5618,"log_dups_size":0,"ondisk_log_size":5618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:44.964709+0000","scrub_duration":892,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":146931712,"num_objects":2242,"num_object_clones":0,"num_object_copies":6726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2242,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13018,"num_write_kb":488320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":234,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.713736+0000","last_change":"2024-09-20T18:53:24.458093+0000","last_active":"2024-09-20T18:53:48.713736+0000","last_peered":"2024-09-20T18:53:48.713736+0000","last_clean":"2024-09-20T18:53:48.713736+0000","last_became_active":"2024-09-20T18:49:48.498625+0000","last_became_peered":"2024-09-20T18:49:48.498625+0000","last_unstale":"2024-09-20T18:53:48.713736+0000","last_undegraded":"2024-09-20T18:53:48.713736+0000","last_fullsized":"2024-09-20T18:53:48.713736+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-20T18:53:24.458005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:24.458005+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:31.316232+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":207,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.715534+0000","last_change":"2024-09-20T18:53:20.564464+0000","last_active":"2024-09-20T18:53:48.715534+0000","last_peered":"2024-09-20T18:53:48.715534+0000","last_clean":"2024-09-20T18:53:48.715534+0000","last_became_active":"2024-09-20T18:49:45.941448+0000","last_became_peered":"2024-09-20T18:49:45.941448+0000","last_unstale":"2024-09-20T18:53:48.715534+0000","last_undegraded":"2024-09-20T18:53:48.715534+0000","last_fullsized":"2024-09-20T18:53:48.715534+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:20.564413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:20.564413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:38.260460+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":201,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.709196+0000","last_change":"2024-09-20T18:52:36.662703+0000","last_active":"2024-09-20T18:53:48.709196+0000","last_peered":"2024-09-20T18:53:48.709196+0000","last_clean":"2024-09-20T18:53:48.709196+0000","last_became_active":"2024-09-20T18:49:49.957558+0000","last_became_peered":"2024-09-20T18:49:49.957558+0000","last_unstale":"2024-09-20T18:53:48.709196+0000","last_undegraded":"2024-09-20T18:53:48.709196+0000","last_fullsized":"2024-09-20T18:53:48.709196+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:36.662524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:36.662524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:58.211161+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":203,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.714998+0000","last_change":"2024-09-20T18:53:47.370152+0000","last_active":"2024-09-20T18:53:48.714998+0000","last_peered":"2024-09-20T18:53:48.714998+0000","last_clean":"2024-09-20T18:53:48.714998+0000","last_became_active":"2024-09-20T18:49:51.970798+0000","last_became_peered":"2024-09-20T18:49:51.970798+0000","last_unstale":"2024-09-20T18:53:48.714998+0000","last_undegraded":"2024-09-20T18:53:48.714998+0000","last_fullsized":"2024-09-20T18:53:48.714998+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:47.370108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:47.370108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:53.917763+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.a","version":"149'13132","reported_seq":13276,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:51.164408+0000","last_change":"2024-09-20T18:50:49.842445+0000","last_active":"2024-09-20T18:53:51.164408+0000","last_peered":"2024-09-20T18:53:51.164408+0000","last_clean":"2024-09-20T18:53:51.164408+0000","last_became_active":"2024-09-20T18:50:49.842179+0000","last_became_peered":"2024-09-20T18:50:49.842179+0000","last_unstale":"2024-09-20T18:53:51.164408+0000","last_undegraded":"2024-09-20T18:53:51.164408+0000","last_fullsized":"2024-09-20T18:53:51.164408+0000","mapping_epoch":83,"log_start":"126'8500","ondisk_log_start":"126'8500","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.819660+0000","objects_scrubbed":0,"log_size":4632,"log_dups_size":0,"ondisk_log_size":4632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:29.169615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":142606336,"num_objects":2176,"num_object_clones":0,"num_object_copies":6528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13132,"num_write_kb":489856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":219,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.701603+0000","last_change":"2024-09-20T18:52:37.881436+0000","last_active":"2024-09-20T18:53:48.701603+0000","last_peered":"2024-09-20T18:53:48.701603+0000","last_clean":"2024-09-20T18:53:48.701603+0000","last_became_active":"2024-09-20T18:49:47.949116+0000","last_became_peered":"2024-09-20T18:49:47.949116+0000","last_unstale":"2024-09-20T18:53:48.701603+0000","last_undegraded":"2024-09-20T18:53:48.701603+0000","last_fullsized":"2024-09-20T18:53:48.701603+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:52:37.881385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:37.881385+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:49.557204+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":207,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.715714+0000","last_change":"2024-09-20T18:53:21.578905+0000","last_active":"2024-09-20T18:53:48.715714+0000","last_peered":"2024-09-20T18:53:48.715714+0000","last_clean":"2024-09-20T18:53:48.715714+0000","last_became_active":"2024-09-20T18:49:45.944858+0000","last_became_peered":"2024-09-20T18:49:45.944858+0000","last_unstale":"2024-09-20T18:53:48.715714+0000","last_undegraded":"2024-09-20T18:53:48.715714+0000","last_fullsized":"2024-09-20T18:53:48.715714+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:21.578845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:21.578845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:47.899680+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":193,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.707225+0000","last_change":"2024-09-20T18:52:12.674502+0000","last_active":"2024-09-20T18:53:48.707225+0000","last_peered":"2024-09-20T18:53:48.707225+0000","last_clean":"2024-09-20T18:53:48.707225+0000","last_became_active":"2024-09-20T18:49:49.954944+0000","last_became_peered":"2024-09-20T18:49:49.954944+0000","last_unstale":"2024-09-20T18:53:48.707225+0000","last_undegraded":"2024-09-20T18:53:48.707225+0000","last_fullsized":"2024-09-20T18:53:48.707225+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:12.674203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:12.674203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:41.868905+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":188,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.714674+0000","last_change":"2024-09-20T18:52:09.017828+0000","last_active":"2024-09-20T18:53:48.714674+0000","last_peered":"2024-09-20T18:53:48.714674+0000","last_clean":"2024-09-20T18:53:48.714674+0000","last_became_active":"2024-09-20T18:49:51.972970+0000","last_became_peered":"2024-09-20T18:49:51.972970+0000","last_unstale":"2024-09-20T18:53:48.714674+0000","last_undegraded":"2024-09-20T18:53:48.714674+0000","last_fullsized":"2024-09-20T18:53:48.714674+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:09.017759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:09.017759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"149'13072","reported_seq":13294,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:50.893912+0000","last_change":"2024-09-20T18:52:15.505711+0000","last_active":"2024-09-20T18:53:50.893912+0000","last_peered":"2024-09-20T18:53:50.893912+0000","last_clean":"2024-09-20T18:53:50.893912+0000","last_became_active":"2024-09-20T18:50:49.843516+0000","last_became_peered":"2024-09-20T18:50:49.843516+0000","last_unstale":"2024-09-20T18:53:50.893912+0000","last_undegraded":"2024-09-20T18:53:50.893912+0000","last_fullsized":"2024-09-20T18:53:50.893912+0000","mapping_epoch":83,"log_start":"124'8000","ondisk_log_start":"124'8000","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'6958","last_scrub_stamp":"2024-09-20T18:52:15.505628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:52:15.505628+0000","objects_scrubbed":6900,"log_size":5072,"log_dups_size":0,"ondisk_log_size":5072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:29.307707+0000","scrub_duration":2196,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":143523840,"num_objects":2190,"num_object_clones":0,"num_object_copies":6570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2190,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13072,"num_write_kb":488384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":220,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.714886+0000","last_change":"2024-09-20T18:52:35.182422+0000","last_active":"2024-09-20T18:53:48.714886+0000","last_peered":"2024-09-20T18:53:48.714886+0000","last_clean":"2024-09-20T18:53:48.714886+0000","last_became_active":"2024-09-20T18:49:48.499847+0000","last_became_peered":"2024-09-20T18:49:48.499847+0000","last_unstale":"2024-09-20T18:53:48.714886+0000","last_undegraded":"2024-09-20T18:53:48.714886+0000","last_fullsized":"2024-09-20T18:53:48.714886+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:52:35.182330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:35.182330+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:57.232688+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":219,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.716101+0000","last_change":"2024-09-20T18:53:10.784695+0000","last_active":"2024-09-20T18:53:48.716101+0000","last_peered":"2024-09-20T18:53:48.716101+0000","last_clean":"2024-09-20T18:53:48.716101+0000","last_became_active":"2024-09-20T18:49:45.944256+0000","last_became_peered":"2024-09-20T18:49:45.944256+0000","last_unstale":"2024-09-20T18:53:48.716101+0000","last_undegraded":"2024-09-20T18:53:48.716101+0000","last_fullsized":"2024-09-20T18:53:48.716101+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-20T18:53:10.784625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:10.784625+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:16.037732+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":248,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.714933+0000","last_change":"2024-09-20T18:52:05.788988+0000","last_active":"2024-09-20T18:53:48.714933+0000","last_peered":"2024-09-20T18:53:48.714933+0000","last_clean":"2024-09-20T18:53:48.714933+0000","last_became_active":"2024-09-20T18:49:49.958411+0000","last_became_peered":"2024-09-20T18:49:49.958411+0000","last_unstale":"2024-09-20T18:53:48.714933+0000","last_undegraded":"2024-09-20T18:53:48.714933+0000","last_fullsized":"2024-09-20T18:53:48.714933+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:52:05.788924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.788924+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:39.468023+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":187,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.715021+0000","last_change":"2024-09-20T18:51:56.970415+0000","last_active":"2024-09-20T18:53:48.715021+0000","last_peered":"2024-09-20T18:53:48.715021+0000","last_clean":"2024-09-20T18:53:48.715021+0000","last_became_active":"2024-09-20T18:49:52.139685+0000","last_became_peered":"2024-09-20T18:49:52.139685+0000","last_unstale":"2024-09-20T18:53:48.715021+0000","last_undegraded":"2024-09-20T18:53:48.715021+0000","last_fullsized":"2024-09-20T18:53:48.715021+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:56.970349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:56.970349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:05.081675+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":207,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.713387+0000","last_change":"2024-09-20T18:53:09.388116+0000","last_active":"2024-09-20T18:53:48.713387+0000","last_peered":"2024-09-20T18:53:48.713387+0000","last_clean":"2024-09-20T18:53:48.713387+0000","last_became_active":"2024-09-20T18:49:45.940060+0000","last_became_peered":"2024-09-20T18:49:45.940060+0000","last_unstale":"2024-09-20T18:53:48.713387+0000","last_undegraded":"2024-09-20T18:53:48.713387+0000","last_fullsized":"2024-09-20T18:53:48.713387+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:09.388050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:09.388050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:17.378691+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":205,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.719948+0000","last_change":"2024-09-20T18:53:27.186982+0000","last_active":"2024-09-20T18:53:48.719948+0000","last_peered":"2024-09-20T18:53:48.719948+0000","last_clean":"2024-09-20T18:53:48.719948+0000","last_became_active":"2024-09-20T18:49:48.500662+0000","last_became_peered":"2024-09-20T18:49:48.500662+0000","last_unstale":"2024-09-20T18:53:48.719948+0000","last_undegraded":"2024-09-20T18:53:48.719948+0000","last_fullsized":"2024-09-20T18:53:48.719948+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-20T18:53:27.186631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:27.186631+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:42.463558+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":199,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.718568+0000","last_change":"2024-09-20T18:53:16.220554+0000","last_active":"2024-09-20T18:53:48.718568+0000","last_peered":"2024-09-20T18:53:48.718568+0000","last_clean":"2024-09-20T18:53:48.718568+0000","last_became_active":"2024-09-20T18:49:49.956749+0000","last_became_peered":"2024-09-20T18:49:49.956749+0000","last_unstale":"2024-09-20T18:53:48.718568+0000","last_undegraded":"2024-09-20T18:53:48.718568+0000","last_fullsized":"2024-09-20T18:53:48.718568+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:16.220490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:16.220490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:36.759843+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":195,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.715116+0000","last_change":"2024-09-20T18:53:19.591395+0000","last_active":"2024-09-20T18:53:48.715116+0000","last_peered":"2024-09-20T18:53:48.715116+0000","last_clean":"2024-09-20T18:53:48.715116+0000","last_became_active":"2024-09-20T18:49:52.142291+0000","last_became_peered":"2024-09-20T18:49:52.142291+0000","last_unstale":"2024-09-20T18:53:48.715116+0000","last_undegraded":"2024-09-20T18:53:48.715116+0000","last_fullsized":"2024-09-20T18:53:48.715116+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:19.591346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:19.591346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:40.474867+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":211,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.720186+0000","last_change":"2024-09-20T18:51:23.065382+0000","last_active":"2024-09-20T18:53:48.720186+0000","last_peered":"2024-09-20T18:53:48.720186+0000","last_clean":"2024-09-20T18:53:48.720186+0000","last_became_active":"2024-09-20T18:49:48.500803+0000","last_became_peered":"2024-09-20T18:49:48.500803+0000","last_unstale":"2024-09-20T18:53:48.720186+0000","last_undegraded":"2024-09-20T18:53:48.720186+0000","last_fullsized":"2024-09-20T18:53:48.720186+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:51:23.064536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:23.064536+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:39.863201+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":199,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.715104+0000","last_change":"2024-09-20T18:51:30.982535+0000","last_active":"2024-09-20T18:53:48.715104+0000","last_peered":"2024-09-20T18:53:48.715104+0000","last_clean":"2024-09-20T18:53:48.715104+0000","last_became_active":"2024-09-20T18:49:45.946600+0000","last_became_peered":"2024-09-20T18:49:45.946600+0000","last_unstale":"2024-09-20T18:53:48.715104+0000","last_undegraded":"2024-09-20T18:53:48.715104+0000","last_fullsized":"2024-09-20T18:53:48.715104+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:30.982454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:30.982454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:57.922735+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":200,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:50.253721+0000","last_change":"2024-09-20T18:53:50.253721+0000","last_active":"2024-09-20T18:53:50.253721+0000","last_peered":"2024-09-20T18:53:50.253721+0000","last_clean":"2024-09-20T18:53:50.253721+0000","last_became_active":"2024-09-20T18:49:49.961947+0000","last_became_peered":"2024-09-20T18:49:49.961947+0000","last_unstale":"2024-09-20T18:53:50.253721+0000","last_undegraded":"2024-09-20T18:53:50.253721+0000","last_fullsized":"2024-09-20T18:53:50.253721+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:50.253660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:50.253660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:15.284040+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"66'1","reported_seq":198,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.708933+0000","last_change":"2024-09-20T18:53:14.689510+0000","last_active":"2024-09-20T18:53:48.708933+0000","last_peered":"2024-09-20T18:53:48.708933+0000","last_clean":"2024-09-20T18:53:48.708933+0000","last_became_active":"2024-09-20T18:49:51.965121+0000","last_became_peered":"2024-09-20T18:49:51.965121+0000","last_unstale":"2024-09-20T18:53:48.708933+0000","last_undegraded":"2024-09-20T18:53:48.708933+0000","last_fullsized":"2024-09-20T18:53:48.708933+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T18:53:14.689422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:14.689422+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:19.716830+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":203,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.715763+0000","last_change":"2024-09-20T18:49:48.501359+0000","last_active":"2024-09-20T18:53:48.715763+0000","last_peered":"2024-09-20T18:53:48.715763+0000","last_clean":"2024-09-20T18:53:48.715763+0000","last_became_active":"2024-09-20T18:49:48.501011+0000","last_became_peered":"2024-09-20T18:49:48.501011+0000","last_unstale":"2024-09-20T18:53:48.715763+0000","last_undegraded":"2024-09-20T18:53:48.715763+0000","last_fullsized":"2024-09-20T18:53:48.715763+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:02.117814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":199,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.711164+0000","last_change":"2024-09-20T18:51:20.067303+0000","last_active":"2024-09-20T18:53:48.711164+0000","last_peered":"2024-09-20T18:53:48.711164+0000","last_clean":"2024-09-20T18:53:48.711164+0000","last_became_active":"2024-09-20T18:49:45.938511+0000","last_became_peered":"2024-09-20T18:49:45.938511+0000","last_unstale":"2024-09-20T18:53:48.711164+0000","last_undegraded":"2024-09-20T18:53:48.711164+0000","last_fullsized":"2024-09-20T18:53:48.711164+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:20.067233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:20.067233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:55.959831+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":258,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.714980+0000","last_change":"2024-09-20T18:53:47.337912+0000","last_active":"2024-09-20T18:53:48.714980+0000","last_peered":"2024-09-20T18:53:48.714980+0000","last_clean":"2024-09-20T18:53:48.714980+0000","last_became_active":"2024-09-20T18:49:49.960550+0000","last_became_peered":"2024-09-20T18:49:49.960550+0000","last_unstale":"2024-09-20T18:53:48.714980+0000","last_undegraded":"2024-09-20T18:53:48.714980+0000","last_fullsized":"2024-09-20T18:53:48.714980+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:53:47.337658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:47.337658+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:01.412695+0000","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":196,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.839130+0000","last_change":"2024-09-20T18:53:48.839130+0000","last_active":"2024-09-20T18:53:48.839130+0000","last_peered":"2024-09-20T18:53:48.839130+0000","last_clean":"2024-09-20T18:53:48.839130+0000","last_became_active":"2024-09-20T18:49:51.966254+0000","last_became_peered":"2024-09-20T18:49:51.966254+0000","last_unstale":"2024-09-20T18:53:48.839130+0000","last_undegraded":"2024-09-20T18:53:48.839130+0000","last_fullsized":"2024-09-20T18:53:48.839130+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:48.839015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:48.839015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:07.481453+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":230,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.715187+0000","last_change":"2024-09-20T18:51:15.223913+0000","last_active":"2024-09-20T18:53:48.715187+0000","last_peered":"2024-09-20T18:53:48.715187+0000","last_clean":"2024-09-20T18:53:48.715187+0000","last_became_active":"2024-09-20T18:49:58.997494+0000","last_became_peered":"2024-09-20T18:49:58.997494+0000","last_unstale":"2024-09-20T18:53:48.715187+0000","last_undegraded":"2024-09-20T18:53:48.715187+0000","last_fullsized":"2024-09-20T18:53:48.715187+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:51:15.223826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:15.223826+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:06.150447+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":207,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.715838+0000","last_change":"2024-09-20T18:53:27.589481+0000","last_active":"2024-09-20T18:53:48.715838+0000","last_peered":"2024-09-20T18:53:48.715838+0000","last_clean":"2024-09-20T18:53:48.715838+0000","last_became_active":"2024-09-20T18:49:45.949492+0000","last_became_peered":"2024-09-20T18:49:45.949492+0000","last_unstale":"2024-09-20T18:53:48.715838+0000","last_undegraded":"2024-09-20T18:53:48.715838+0000","last_fullsized":"2024-09-20T18:53:48.715838+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:27.589413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:27.589413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:48.562654+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":248,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.715585+0000","last_change":"2024-09-20T18:52:39.168501+0000","last_active":"2024-09-20T18:53:48.715585+0000","last_peered":"2024-09-20T18:53:48.715585+0000","last_clean":"2024-09-20T18:53:48.715585+0000","last_became_active":"2024-09-20T18:49:49.962243+0000","last_became_peered":"2024-09-20T18:49:49.962243+0000","last_unstale":"2024-09-20T18:53:48.715585+0000","last_undegraded":"2024-09-20T18:53:48.715585+0000","last_fullsized":"2024-09-20T18:53:48.715585+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:52:39.168442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:39.168442+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:02.311754+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":195,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.713914+0000","last_change":"2024-09-20T18:53:13.863129+0000","last_active":"2024-09-20T18:53:48.713914+0000","last_peered":"2024-09-20T18:53:48.713914+0000","last_clean":"2024-09-20T18:53:48.713914+0000","last_became_active":"2024-09-20T18:49:51.969093+0000","last_became_peered":"2024-09-20T18:49:51.969093+0000","last_unstale":"2024-09-20T18:53:48.713914+0000","last_undegraded":"2024-09-20T18:53:48.713914+0000","last_fullsized":"2024-09-20T18:53:48.713914+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:13.863061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:13.863061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:37.505050+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":223,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.701834+0000","last_change":"2024-09-20T18:53:43.879001+0000","last_active":"2024-09-20T18:53:48.701834+0000","last_peered":"2024-09-20T18:53:48.701834+0000","last_clean":"2024-09-20T18:53:48.701834+0000","last_became_active":"2024-09-20T18:49:47.949703+0000","last_became_peered":"2024-09-20T18:49:47.949703+0000","last_unstale":"2024-09-20T18:53:48.701834+0000","last_undegraded":"2024-09-20T18:53:48.701834+0000","last_fullsized":"2024-09-20T18:53:48.701834+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:53:43.878921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:43.878921+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:02.026666+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":200,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.702122+0000","last_change":"2024-09-20T18:52:05.985622+0000","last_active":"2024-09-20T18:53:48.702122+0000","last_peered":"2024-09-20T18:53:48.702122+0000","last_clean":"2024-09-20T18:53:48.702122+0000","last_became_active":"2024-09-20T18:49:45.939949+0000","last_became_peered":"2024-09-20T18:49:45.939949+0000","last_unstale":"2024-09-20T18:53:48.702122+0000","last_undegraded":"2024-09-20T18:53:48.702122+0000","last_fullsized":"2024-09-20T18:53:48.702122+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:05.985551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.985551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:36.823731+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":192,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.718146+0000","last_change":"2024-09-20T18:51:14.271954+0000","last_active":"2024-09-20T18:53:48.718146+0000","last_peered":"2024-09-20T18:53:48.718146+0000","last_clean":"2024-09-20T18:53:48.718146+0000","last_became_active":"2024-09-20T18:49:49.957067+0000","last_became_peered":"2024-09-20T18:49:49.957067+0000","last_unstale":"2024-09-20T18:53:48.718146+0000","last_undegraded":"2024-09-20T18:53:48.718146+0000","last_fullsized":"2024-09-20T18:53:48.718146+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:14.271719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.271719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:31.305762+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":188,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.716694+0000","last_change":"2024-09-20T18:51:14.210438+0000","last_active":"2024-09-20T18:53:48.716694+0000","last_peered":"2024-09-20T18:53:48.716694+0000","last_clean":"2024-09-20T18:53:48.716694+0000","last_became_active":"2024-09-20T18:49:51.966915+0000","last_became_peered":"2024-09-20T18:49:51.966915+0000","last_unstale":"2024-09-20T18:53:48.716694+0000","last_undegraded":"2024-09-20T18:53:48.716694+0000","last_fullsized":"2024-09-20T18:53:48.716694+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:14.210358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.210358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:04.079575+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":220,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.718369+0000","last_change":"2024-09-20T18:52:38.240463+0000","last_active":"2024-09-20T18:53:48.718369+0000","last_peered":"2024-09-20T18:53:48.718369+0000","last_clean":"2024-09-20T18:53:48.718369+0000","last_became_active":"2024-09-20T18:49:47.953310+0000","last_became_peered":"2024-09-20T18:49:47.953310+0000","last_unstale":"2024-09-20T18:53:48.718369+0000","last_undegraded":"2024-09-20T18:53:48.718369+0000","last_fullsized":"2024-09-20T18:53:48.718369+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:52:38.240386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:38.240386+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:52.778987+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":199,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.715404+0000","last_change":"2024-09-20T18:52:34.834333+0000","last_active":"2024-09-20T18:53:48.715404+0000","last_peered":"2024-09-20T18:53:48.715404+0000","last_clean":"2024-09-20T18:53:48.715404+0000","last_became_active":"2024-09-20T18:49:45.944357+0000","last_became_peered":"2024-09-20T18:49:45.944357+0000","last_unstale":"2024-09-20T18:53:48.715404+0000","last_undegraded":"2024-09-20T18:53:48.715404+0000","last_fullsized":"2024-09-20T18:53:48.715404+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:34.834273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:34.834273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:42.456584+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":192,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.714746+0000","last_change":"2024-09-20T18:51:20.208925+0000","last_active":"2024-09-20T18:53:48.714746+0000","last_peered":"2024-09-20T18:53:48.714746+0000","last_clean":"2024-09-20T18:53:48.714746+0000","last_became_active":"2024-09-20T18:49:49.962523+0000","last_became_peered":"2024-09-20T18:49:49.962523+0000","last_unstale":"2024-09-20T18:53:48.714746+0000","last_undegraded":"2024-09-20T18:53:48.714746+0000","last_fullsized":"2024-09-20T18:53:48.714746+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:20.208786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:20.208786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:08.165781+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":195,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.709994+0000","last_change":"2024-09-20T18:52:38.137485+0000","last_active":"2024-09-20T18:53:48.709994+0000","last_peered":"2024-09-20T18:53:48.709994+0000","last_clean":"2024-09-20T18:53:48.709994+0000","last_became_active":"2024-09-20T18:49:51.967313+0000","last_became_peered":"2024-09-20T18:49:51.967313+0000","last_unstale":"2024-09-20T18:53:48.709994+0000","last_undegraded":"2024-09-20T18:53:48.709994+0000","last_fullsized":"2024-09-20T18:53:48.709994+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:38.137431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:38.137431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:07.091791+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":215,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.716828+0000","last_change":"2024-09-20T18:52:13.020995+0000","last_active":"2024-09-20T18:53:48.716828+0000","last_peered":"2024-09-20T18:53:48.716828+0000","last_clean":"2024-09-20T18:53:48.716828+0000","last_became_active":"2024-09-20T18:49:48.499111+0000","last_became_peered":"2024-09-20T18:49:48.499111+0000","last_unstale":"2024-09-20T18:53:48.716828+0000","last_undegraded":"2024-09-20T18:53:48.716828+0000","last_fullsized":"2024-09-20T18:53:48.716828+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:52:13.020920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:13.020920+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":207,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.718577+0000","last_change":"2024-09-20T18:53:48.301845+0000","last_active":"2024-09-20T18:53:48.718577+0000","last_peered":"2024-09-20T18:53:48.718577+0000","last_clean":"2024-09-20T18:53:48.718577+0000","last_became_active":"2024-09-20T18:49:45.942105+0000","last_became_peered":"2024-09-20T18:49:45.942105+0000","last_unstale":"2024-09-20T18:53:48.718577+0000","last_undegraded":"2024-09-20T18:53:48.718577+0000","last_fullsized":"2024-09-20T18:53:48.718577+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:48.301801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:48.301801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:06.232022+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":207,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.713583+0000","last_change":"2024-09-20T18:53:46.329643+0000","last_active":"2024-09-20T18:53:48.713583+0000","last_peered":"2024-09-20T18:53:48.713583+0000","last_clean":"2024-09-20T18:53:48.713583+0000","last_became_active":"2024-09-20T18:49:49.958159+0000","last_became_peered":"2024-09-20T18:49:49.958159+0000","last_unstale":"2024-09-20T18:53:48.713583+0000","last_undegraded":"2024-09-20T18:53:48.713583+0000","last_fullsized":"2024-09-20T18:53:48.713583+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:46.329547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:46.329547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:09.181263+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":188,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.714815+0000","last_change":"2024-09-20T18:52:43.118886+0000","last_active":"2024-09-20T18:53:48.714815+0000","last_peered":"2024-09-20T18:53:48.714815+0000","last_clean":"2024-09-20T18:53:48.714815+0000","last_became_active":"2024-09-20T18:49:52.139978+0000","last_became_peered":"2024-09-20T18:49:52.139978+0000","last_unstale":"2024-09-20T18:53:48.714815+0000","last_undegraded":"2024-09-20T18:53:48.714815+0000","last_fullsized":"2024-09-20T18:53:48.714815+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:43.118805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:43.118805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:11.215766+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":210,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.714220+0000","last_change":"2024-09-20T18:53:43.253204+0000","last_active":"2024-09-20T18:53:48.714220+0000","last_peered":"2024-09-20T18:53:48.714220+0000","last_clean":"2024-09-20T18:53:48.714220+0000","last_became_active":"2024-09-20T18:49:47.954777+0000","last_became_peered":"2024-09-20T18:49:47.954777+0000","last_unstale":"2024-09-20T18:53:48.714220+0000","last_undegraded":"2024-09-20T18:53:48.714220+0000","last_fullsized":"2024-09-20T18:53:48.714220+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-20T18:53:43.253132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:43.253132+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:00.654449+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":207,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.711537+0000","last_change":"2024-09-20T18:53:16.921320+0000","last_active":"2024-09-20T18:53:48.711537+0000","last_peered":"2024-09-20T18:53:48.711537+0000","last_clean":"2024-09-20T18:53:48.711537+0000","last_became_active":"2024-09-20T18:49:45.940988+0000","last_became_peered":"2024-09-20T18:49:45.940988+0000","last_unstale":"2024-09-20T18:53:48.711537+0000","last_undegraded":"2024-09-20T18:53:48.711537+0000","last_fullsized":"2024-09-20T18:53:48.711537+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:16.921263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:16.921263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:35.775476+0000","scrub_duration":137,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":199,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.714849+0000","last_change":"2024-09-20T18:52:44.344764+0000","last_active":"2024-09-20T18:53:48.714849+0000","last_peered":"2024-09-20T18:53:48.714849+0000","last_clean":"2024-09-20T18:53:48.714849+0000","last_became_active":"2024-09-20T18:49:49.954502+0000","last_became_peered":"2024-09-20T18:49:49.954502+0000","last_unstale":"2024-09-20T18:53:48.714849+0000","last_undegraded":"2024-09-20T18:53:48.714849+0000","last_fullsized":"2024-09-20T18:53:48.714849+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:44.344700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:44.344700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:52.432651+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":187,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.715195+0000","last_change":"2024-09-20T18:51:09.761331+0000","last_active":"2024-09-20T18:53:48.715195+0000","last_peered":"2024-09-20T18:53:48.715195+0000","last_clean":"2024-09-20T18:53:48.715195+0000","last_became_active":"2024-09-20T18:49:51.968662+0000","last_became_peered":"2024-09-20T18:49:51.968662+0000","last_unstale":"2024-09-20T18:53:48.715195+0000","last_undegraded":"2024-09-20T18:53:48.715195+0000","last_fullsized":"2024-09-20T18:53:48.715195+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:09.761283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:09.761283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:30.451288+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":180,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.721200+0000","last_change":"2024-09-20T18:49:51.968476+0000","last_active":"2024-09-20T18:53:48.721200+0000","last_peered":"2024-09-20T18:53:48.721200+0000","last_clean":"2024-09-20T18:53:48.721200+0000","last_became_active":"2024-09-20T18:49:51.968216+0000","last_became_peered":"2024-09-20T18:49:51.968216+0000","last_unstale":"2024-09-20T18:53:48.721200+0000","last_undegraded":"2024-09-20T18:53:48.721200+0000","last_fullsized":"2024-09-20T18:53:48.721200+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:58.058065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":200,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.714426+0000","last_change":"2024-09-20T18:52:41.106558+0000","last_active":"2024-09-20T18:53:48.714426+0000","last_peered":"2024-09-20T18:53:48.714426+0000","last_clean":"2024-09-20T18:53:48.714426+0000","last_became_active":"2024-09-20T18:49:45.937992+0000","last_became_peered":"2024-09-20T18:49:45.937992+0000","last_unstale":"2024-09-20T18:53:48.714426+0000","last_undegraded":"2024-09-20T18:53:48.714426+0000","last_fullsized":"2024-09-20T18:53:48.714426+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:41.106332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:41.106332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:50.275314+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":214,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.714298+0000","last_change":"2024-09-20T18:51:14.545122+0000","last_active":"2024-09-20T18:53:48.714298+0000","last_peered":"2024-09-20T18:53:48.714298+0000","last_clean":"2024-09-20T18:53:48.714298+0000","last_became_active":"2024-09-20T18:49:47.949582+0000","last_became_peered":"2024-09-20T18:49:47.949582+0000","last_unstale":"2024-09-20T18:53:48.714298+0000","last_undegraded":"2024-09-20T18:53:48.714298+0000","last_fullsized":"2024-09-20T18:53:48.714298+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:51:14.545043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.545043+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:40.491358+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":192,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-20T18:53:48.715412+0000","last_change":"2024-09-20T18:51:28.120100+0000","last_active":"2024-09-20T18:53:48.715412+0000","last_peered":"2024-09-20T18:53:48.715412+0000","last_clean":"2024-09-20T18:53:48.715412+0000","last_became_active":"2024-09-20T18:49:49.957582+0000","last_became_peered":"2024-09-20T18:49:49.957582+0000","last_unstale":"2024-09-20T18:53:48.715412+0000","last_undegraded":"2024-09-20T18:53:48.715412+0000","last_fullsized":"2024-09-20T18:53:48.715412+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:28.119694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:28.119694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:32.813511+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":2311061528,"num_objects":35265,"num_object_clones":0,"num_object_copies":105795,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35265,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":210335,"num_write_kb":7859137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6936997888,"data_stored":6936985672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":90029,"ondisk_log_size":90029,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":240,"num_read_kb":235,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266741,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":823768,"kb_used_data":724552,"kb_used_omap":1,"kb_used_meta":99198,"kb_avail":92921384,"statfs":{"total":95995035648,"available":95151497216,"internally_reserved":0,"allocated":741941248,"data_stored":741577960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":101579210},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528380,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":815880,"kb_used_data":704184,"kb_used_omap":1,"kb_used_meta":111678,"kb_avail":92929272,"statfs":{"total":95995035648,"available":95159574528,"internally_reserved":0,"allocated":721084416,"data_stored":720736874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":114358726},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757313,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":654852,"kb_used_data":575092,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":93090300,"statfs":{"total":95995035648,"available":95324467200,"internally_reserved":0,"allocated":588894208,"data_stored":588550553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":81656261},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986246,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":848616,"kb_used_data":736916,"kb_used_omap":1,"kb_used_meta":111678,"kb_avail":92896536,"statfs":{"total":95995035648,"available":95126052864,"internally_reserved":0,"allocated":754601984,"data_stored":754226813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":114358725},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1459104,"kb_used_data":1293072,"kb_used_omap":1,"kb_used_meta":165822,"kb_avail":92286048,"statfs":{"total":95995035648,"available":94500913152,"internally_reserved":0,"allocated":1324105728,"data_stored":1323733497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":169802183},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":956620,"kb_used_data":857400,"kb_used_omap":1,"kb_used_meta":99198,"kb_avail":92788532,"statfs":{"total":95995035648,"available":95015456768,"internally_reserved":0,"allocated":877977600,"data_stored":877630428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":101579205},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,1,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":963072,"kb_used_data":863852,"kb_used_omap":1,"kb_used_meta":99198,"kb_avail":92782080,"statfs":{"total":95995035648,"available":95008849920,"internally_reserved":0,"allocated":884584448,"data_stored":884185236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":101579205},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1282012,"kb_used_data":1134220,"kb_used_omap":1,"kb_used_meta":147774,"kb_avail":92463140,"statfs":{"total":95995035648,"available":94682255360,"internally_reserved":0,"allocated":1161441280,"data_stored":1161073616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151321029},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1148456960,"data_stored":1148452888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":865865728,"data_stored":865861656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":872546304,"data_stored":872546304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1290207232,"data_stored":1290207232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":735707136,"data_stored":735707136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":578027520,"data_stored":578027520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":715132928,"data_stored":715128856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":731054080,"data_stored":731054080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T18:53:56.333 INFO:tasks.cephadm.ceph_manager.ceph:would drop below min_pgs, proj -9, currently 1 2024-09-20T18:53:56.412 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:56.448 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:56.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:56.449 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:56.549 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:53:56.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:56.580 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:56.581 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:53:56.681 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:56.709 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:56.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:56.710 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:56.810 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:56.844 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:56.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:56.845 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:56.845 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:53:56 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:53:56.886 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-20T18:53:56.946 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:53:56.991 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:56.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:56.992 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:53:57.093 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:57.134 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:57.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:57.135 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:57.236 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:57.262 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:57.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:57.263 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:57.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:56 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:53:57.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:56 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:53:57.306 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:56 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:53:57.338 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:53:57.364 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:57.388 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:57.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:57.389 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:57.490 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:57.513 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:57.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:57.514 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:57.615 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:57.640 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:57.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:57.641 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:57.677 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T18:53:57.742 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:53:57.780 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:57.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:57.781 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:53:57.882 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:53:57.906 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:57.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:57.907 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:53:57.952 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T18:53:57.952 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:53:58.007 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:58.051 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:58.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:58.051 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:58.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:57 smithi083 ceph-mon[29421]: pgmap v190: 148 pgs: 148 active+clean; 1.8 GiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.27k op/s 2024-09-20T18:53:58.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:57 smithi083 ceph-mon[36260]: pgmap v190: 148 pgs: 148 active+clean; 1.8 GiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.27k op/s 2024-09-20T18:53:58.152 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:58.189 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:58.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:58.190 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:58.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:57 smithi142 ceph-mon[32508]: pgmap v190: 148 pgs: 148 active+clean; 1.8 GiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.27k op/s 2024-09-20T18:53:58.290 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:53:58.327 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:58.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:58.328 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:53:58.428 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:58.456 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:58.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:58.457 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:58.557 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:58.582 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:58.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:58.583 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:58.684 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:58.746 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_blocked_ops 2024-09-20T18:53:58.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:58.754 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:58.754 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:58.855 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:53:58.855 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:53:58.904 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:58.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:58.905 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:53:58.919 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T18:53:59.006 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:53:59.029 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:59.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:59.030 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:53:59.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:58 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:53:59.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:58 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:53:59.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:58 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:53:59.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:58 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:53:59.131 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:59.212 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:59.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:59.213 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:59.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:58 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:53:59.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:58 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:53:59.314 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:53:59.363 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:59.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:59.364 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:53:59.465 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:53:59.490 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:59.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:59.491 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:53:59.500 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:53:59.592 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:53:59.619 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:59.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:59.621 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:53:59.721 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:53:59.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:59.773 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:53:59.773 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:53:59.873 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:53:59.899 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:53:59.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:53:59.900 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:53:59.966 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-20T18:54:00.001 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:54:00.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:00.034 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:00.034 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:54:00.135 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:00.160 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:00.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:00.161 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:00.253 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:59 smithi083 ceph-mon[29421]: pgmap v191: 148 pgs: 148 active+clean; 1.8 GiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.03k op/s 2024-09-20T18:54:00.253 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:59 smithi083 ceph-mon[29421]: osdmap e151: 8 total, 8 up, 8 in 2024-09-20T18:54:00.253 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:59 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/650896038' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:54:00.253 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:53:59 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:54:00.254 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:59 smithi083 ceph-mon[36260]: pgmap v191: 148 pgs: 148 active+clean; 1.8 GiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.03k op/s 2024-09-20T18:54:00.254 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:59 smithi083 ceph-mon[36260]: osdmap e151: 8 total, 8 up, 8 in 2024-09-20T18:54:00.254 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:59 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/650896038' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:54:00.254 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:53:59 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:54:00.261 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:00.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:59 smithi142 ceph-mon[32508]: pgmap v191: 148 pgs: 148 active+clean; 1.8 GiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.03k op/s 2024-09-20T18:54:00.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:59 smithi142 ceph-mon[32508]: osdmap e151: 8 total, 8 up, 8 in 2024-09-20T18:54:00.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:59 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/650896038' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:54:00.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:53:59 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:54:00.285 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:00.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:00.286 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:00.387 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:54:00.414 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:00.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:00.415 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:54:00.430 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T18:54:00.430 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:54:00.517 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:00.544 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:00.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:00.545 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:00.645 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:00.671 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:00.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:00.672 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:00.773 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:00.775 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:54:00.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:00.827 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:00.828 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:00.928 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:00.952 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:00.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:00.953 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:01.054 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:01.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:00 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:54:01.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:00 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:54:01.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:00 smithi083 ceph-mon[29421]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:54:01.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:00 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T18:54:01.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:00 smithi083 ceph-mon[29421]: osdmap e152: 8 total, 8 up, 8 in 2024-09-20T18:54:01.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:00 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:54:01.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:00 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:54:01.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:00 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:54:01.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:00 smithi083 ceph-mon[36260]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:54:01.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:00 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T18:54:01.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:00 smithi083 ceph-mon[36260]: osdmap e152: 8 total, 8 up, 8 in 2024-09-20T18:54:01.056 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:00 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:54:01.057 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:54:00 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:54:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:54:01.093 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:01.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:01.094 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:01.130 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_historic_ops 2024-09-20T18:54:01.195 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:54:01.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:01.220 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:01.221 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:54:01.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:00 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:54:01.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:00 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:54:01.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:00 smithi142 ceph-mon[32508]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:54:01.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:00 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T18:54:01.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:00 smithi142 ceph-mon[32508]: osdmap e152: 8 total, 8 up, 8 in 2024-09-20T18:54:01.275 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:00 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:54:01.322 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:54:01.334 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-20T18:54:01.334 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T18:54:01.334 INFO:tasks.thrashosds.thrasher:Shrinking pool unique_pool_0 2024-09-20T18:54:01.335 INFO:tasks.cephadm.ceph_manager.ceph:contract_pool unique_pool_0 by 10 min 8 2024-09-20T18:54:01.335 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T18:54:01.349 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:01.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:01.350 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:54:01.451 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:01.498 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:01.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:01.499 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:01.599 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:01.639 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:01.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:01.640 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:01.741 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:01.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:01.786 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:01.786 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:01.887 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:01.932 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:01.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:01.934 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:02.034 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:02.059 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:02.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:02.060 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:02.154 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:54:02.155 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:54:02.161 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:02.177 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:54:02.189 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:54:02.199 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:54:02.199 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:54:02.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:02.226 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:02.227 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:02.243 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:54:02.243 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:54:02.280 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:54:02.280 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:54:02.313 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:54:02.314 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:54:02.328 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:02.348 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:54:02.348 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:54:02.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:02.373 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:02.373 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:02.380 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:54:02.380 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:54:02.413 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:54:02.413 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:54:02.474 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:02.501 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:02.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:02.502 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:02.603 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:02.631 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:02.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:02.632 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:02.733 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:02.758 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:02.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:02.759 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:02.860 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:54:02.888 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:02.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:02.889 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:54:02.990 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:03.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:03.016 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:03.016 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:03.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:02 smithi142 ceph-mon[32508]: pgmap v194: 148 pgs: 148 active+clean; 1.7 GiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.18k op/s 2024-09-20T18:54:03.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:02 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/4287003908' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:54:03.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:02 smithi083 ceph-mon[29421]: pgmap v194: 148 pgs: 148 active+clean; 1.7 GiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.18k op/s 2024-09-20T18:54:03.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:02 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/4287003908' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:54:03.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:02 smithi083 ceph-mon[36260]: pgmap v194: 148 pgs: 148 active+clean; 1.7 GiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.18k op/s 2024-09-20T18:54:03.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:02 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/4287003908' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:54:03.117 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:03.154 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:03.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:03.155 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:03.255 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:03.280 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:03.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:03.281 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:03.382 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:03.403 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:03.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:03.404 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:03.505 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:03.542 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:03.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:03.543 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:03.587 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T18:54:03.644 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:03.687 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:03.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:03.688 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:03.789 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:03.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:03.823 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:03.824 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:03.924 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:03.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:03.965 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:03.966 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:04.066 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:04.112 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:04.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:04.114 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:04.214 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:54:04.240 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:04.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:04.241 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:54:04.342 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:04.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:04.402 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:04.402 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:04.503 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:04.539 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:04.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:04.540 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:04.617 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T18:54:04.617 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-20T18:54:04.617 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-20T18:54:04.617 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-20T18:54:04.617 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.617 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14442.0:1 3.12 3:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-20T18:54:04.618 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:49:45.769138+0000", 2024-09-20T18:54:04.618 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 258.828155041, 2024-09-20T18:54:04.618 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.16996755799999999, 2024-09-20T18:54:04.618 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:54:04.618 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:54:04.618 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-20T18:54:04.618 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:54:04.618 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14442", 2024-09-20T18:54:04.618 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/3586269208", 2024-09-20T18:54:04.618 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 1 2024-09-20T18:54:04.618 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.618 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:54:04.618 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.618 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:54:04.619 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:49:45.769138+0000", 2024-09-20T18:54:04.619 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:04.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.619 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.619 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:54:04.619 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:49:45.769138+0000", 2024-09-20T18:54:04.619 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:04.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.619 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.619 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:54:04.619 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:49:45.769145+0000", 2024-09-20T18:54:04.619 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8450000000000003e-06 2024-09-20T18:54:04.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.619 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.620 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:54:04.620 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:49:45.769151+0000", 2024-09-20T18:54:04.620 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2989999999999996e-06 2024-09-20T18:54:04.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.620 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.620 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:54:04.620 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:49:45.769160+0000", 2024-09-20T18:54:04.620 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0089999999999996e-06 2024-09-20T18:54:04.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.620 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.620 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:54:04.620 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:49:45.769168+0000", 2024-09-20T18:54:04.620 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5490000000000006e-06 2024-09-20T18:54:04.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.620 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.621 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:54:04.621 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:49:45.769240+0000", 2024-09-20T18:54:04.621 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2081000000000003e-05 2024-09-20T18:54:04.621 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.621 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.621 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:54:04.621 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:49:45.769247+0000", 2024-09-20T18:54:04.621 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5309999999999998e-06 2024-09-20T18:54:04.621 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.621 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.621 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:54:04.621 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:49:45.939030+0000", 2024-09-20T18:54:04.621 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.169782715 2024-09-20T18:54:04.621 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.622 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.622 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:54:04.622 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:49:45.939105+0000", 2024-09-20T18:54:04.622 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5529000000000001e-05 2024-09-20T18:54:04.622 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:04.622 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:54:04.622 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:04.622 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.622 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.622 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:25335 7.6 7:63a06433:::benchmark_data_smithi083_70643_object25334:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:54:04.622 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.266613+0000", 2024-09-20T18:54:04.622 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 177.33067918500001, 2024-09-20T18:54:04.622 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17508610599999999, 2024-09-20T18:54:04.622 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:54:04.623 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:54:04.623 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:54:04.623 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:54:04.623 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:54:04.623 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:54:04.623 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25335 2024-09-20T18:54:04.623 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.623 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:54:04.623 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.623 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:54:04.623 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266613+0000", 2024-09-20T18:54:04.623 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:04.623 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.623 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.624 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:54:04.624 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266613+0000", 2024-09-20T18:54:04.624 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:04.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.624 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.624 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:54:04.624 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266615+0000", 2024-09-20T18:54:04.624 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2240000000000001e-06 2024-09-20T18:54:04.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.624 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.624 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:54:04.624 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266641+0000", 2024-09-20T18:54:04.624 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6848000000000001e-05 2024-09-20T18:54:04.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.625 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.625 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:54:04.625 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266650+0000", 2024-09-20T18:54:04.625 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.3569999999999998e-06 2024-09-20T18:54:04.625 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.625 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.625 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:54:04.625 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266656+0000", 2024-09-20T18:54:04.625 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0839999999999997e-06 2024-09-20T18:54:04.625 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.625 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.625 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:54:04.625 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266923+0000", 2024-09-20T18:54:04.625 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00026740999999999999 2024-09-20T18:54:04.625 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.626 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.626 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:54:04.626 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.267055+0000", 2024-09-20T18:54:04.626 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013152800000000001 2024-09-20T18:54:04.626 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.626 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.626 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:54:04.626 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.267176+0000", 2024-09-20T18:54:04.626 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012149799999999999 2024-09-20T18:54:04.626 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.626 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.626 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:54:04.626 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269576+0000", 2024-09-20T18:54:04.626 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002400157 2024-09-20T18:54:04.626 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.627 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.627 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:04.627 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441551+0000", 2024-09-20T18:54:04.627 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17197433400000001 2024-09-20T18:54:04.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.627 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.627 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:04.627 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441597+0000", 2024-09-20T18:54:04.627 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5986999999999999e-05 2024-09-20T18:54:04.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.627 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.627 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:54:04.627 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441650+0000", 2024-09-20T18:54:04.627 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.325e-05 2024-09-20T18:54:04.628 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.628 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.628 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:54:04.628 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441699+0000", 2024-09-20T18:54:04.628 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9428999999999997e-05 2024-09-20T18:54:04.628 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:04.628 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:54:04.628 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:04.628 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.628 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.628 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:25337 7.f 7:fd2b3e49:::benchmark_data_smithi083_70643_object25336:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:54:04.628 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.268131+0000", 2024-09-20T18:54:04.628 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 177.32916195600001, 2024-09-20T18:54:04.628 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17632631300000001, 2024-09-20T18:54:04.628 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:54:04.629 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:54:04.629 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:54:04.629 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:54:04.629 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:54:04.629 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:54:04.629 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25337 2024-09-20T18:54:04.629 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.629 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:54:04.629 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.629 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:54:04.629 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268131+0000", 2024-09-20T18:54:04.629 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:04.629 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.629 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.629 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:54:04.630 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268131+0000", 2024-09-20T18:54:04.630 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:04.630 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.630 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.630 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:54:04.630 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268132+0000", 2024-09-20T18:54:04.630 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3659999999999999e-06 2024-09-20T18:54:04.630 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.630 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.630 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:54:04.630 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268162+0000", 2024-09-20T18:54:04.630 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9572000000000001e-05 2024-09-20T18:54:04.630 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.630 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.631 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:54:04.631 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268173+0000", 2024-09-20T18:54:04.631 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.132e-05 2024-09-20T18:54:04.631 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.631 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.631 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:54:04.631 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268200+0000", 2024-09-20T18:54:04.631 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7081999999999999e-05 2024-09-20T18:54:04.631 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.631 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.631 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:54:04.631 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268365+0000", 2024-09-20T18:54:04.631 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016553299999999999 2024-09-20T18:54:04.631 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.631 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.632 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:54:04.632 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268480+0000", 2024-09-20T18:54:04.632 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000114595 2024-09-20T18:54:04.632 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.632 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.632 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:54:04.632 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268598+0000", 2024-09-20T18:54:04.632 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000117909 2024-09-20T18:54:04.632 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.632 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.632 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:54:04.632 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270238+0000", 2024-09-20T18:54:04.632 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001639604 2024-09-20T18:54:04.632 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.633 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.633 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:04.633 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441916+0000", 2024-09-20T18:54:04.633 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.171678468 2024-09-20T18:54:04.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.633 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.633 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:04.633 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444363+0000", 2024-09-20T18:54:04.633 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0024469090000000001 2024-09-20T18:54:04.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.633 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.633 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:54:04.633 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444423+0000", 2024-09-20T18:54:04.633 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9645999999999998e-05 2024-09-20T18:54:04.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.634 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.634 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:54:04.634 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444457+0000", 2024-09-20T18:54:04.634 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4309e-05 2024-09-20T18:54:04.634 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:04.634 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:54:04.634 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:04.634 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.634 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.634 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:25338 7.9 7:9270090d:::benchmark_data_smithi083_70643_object25337:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:54:04.634 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.268223+0000", 2024-09-20T18:54:04.634 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 177.32906999799999, 2024-09-20T18:54:04.634 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.173517488, 2024-09-20T18:54:04.635 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:54:04.635 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:54:04.635 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:54:04.635 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:54:04.635 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:54:04.635 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:54:04.635 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25338 2024-09-20T18:54:04.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.635 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:54:04.635 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.635 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:54:04.635 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268223+0000", 2024-09-20T18:54:04.635 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:04.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.635 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.636 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:54:04.636 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268223+0000", 2024-09-20T18:54:04.636 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:04.636 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.636 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.636 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:54:04.636 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268223+0000", 2024-09-20T18:54:04.636 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2799999999999996e-07 2024-09-20T18:54:04.636 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.636 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.636 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:54:04.636 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268277+0000", 2024-09-20T18:54:04.636 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4076000000000002e-05 2024-09-20T18:54:04.636 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.637 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.637 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:54:04.637 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268284+0000", 2024-09-20T18:54:04.637 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1849999999999998e-06 2024-09-20T18:54:04.637 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.637 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.637 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:54:04.637 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268290+0000", 2024-09-20T18:54:04.637 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0390000000000001e-06 2024-09-20T18:54:04.637 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.637 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.637 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:54:04.638 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268372+0000", 2024-09-20T18:54:04.638 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.1027999999999997e-05 2024-09-20T18:54:04.638 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.638 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.638 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:54:04.638 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268482+0000", 2024-09-20T18:54:04.638 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000110979 2024-09-20T18:54:04.638 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.638 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.638 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,4", 2024-09-20T18:54:04.638 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268615+0000", 2024-09-20T18:54:04.638 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013288199999999999 2024-09-20T18:54:04.638 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.638 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.639 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:04.639 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271414+0000", 2024-09-20T18:54:04.639 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0027988100000000001 2024-09-20T18:54:04.639 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.639 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.639 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:54:04.639 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271646+0000", 2024-09-20T18:54:04.639 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000231449 2024-09-20T18:54:04.639 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.639 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.639 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:04.639 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271757+0000", 2024-09-20T18:54:04.639 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011127400000000001 2024-09-20T18:54:04.639 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.640 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.640 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:54:04.640 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441704+0000", 2024-09-20T18:54:04.640 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.16994677499999999 2024-09-20T18:54:04.640 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.640 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.640 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:54:04.640 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441740+0000", 2024-09-20T18:54:04.640 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6362999999999997e-05 2024-09-20T18:54:04.640 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:04.640 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:54:04.640 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:04.640 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.640 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.640 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:25340 7.6 7:6dd1b43c:::benchmark_data_smithi083_70643_object25339:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:54:04.641 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.268308+0000", 2024-09-20T18:54:04.641 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 177.328984074, 2024-09-20T18:54:04.641 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17672299299999999, 2024-09-20T18:54:04.641 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:54:04.641 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:54:04.641 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:54:04.641 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:54:04.641 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:54:04.641 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:54:04.641 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25340 2024-09-20T18:54:04.641 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.641 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:54:04.641 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.641 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:54:04.641 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268308+0000", 2024-09-20T18:54:04.642 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:04.642 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.642 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.642 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:54:04.642 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268308+0000", 2024-09-20T18:54:04.642 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:04.642 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.642 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.642 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:54:04.642 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268309+0000", 2024-09-20T18:54:04.642 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3499999999999996e-07 2024-09-20T18:54:04.642 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.642 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.642 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:54:04.643 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268332+0000", 2024-09-20T18:54:04.643 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2741000000000001e-05 2024-09-20T18:54:04.643 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.643 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.643 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:54:04.643 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268341+0000", 2024-09-20T18:54:04.643 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7630000000000006e-06 2024-09-20T18:54:04.643 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.643 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.643 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:54:04.643 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268346+0000", 2024-09-20T18:54:04.643 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7790000000000001e-06 2024-09-20T18:54:04.643 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.643 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.643 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:54:04.644 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:04.644 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268450+0000", 2024-09-20T18:54:04.645 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000103745 2024-09-20T18:54:04.645 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.645 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.645 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:54:04.645 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268553+0000", 2024-09-20T18:54:04.645 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000102575 2024-09-20T18:54:04.645 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.645 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.645 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:54:04.645 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268667+0000", 2024-09-20T18:54:04.645 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000114449 2024-09-20T18:54:04.645 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.645 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.646 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:54:04.646 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270201+0000", 2024-09-20T18:54:04.646 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001533834 2024-09-20T18:54:04.646 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.646 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.646 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:04.646 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441748+0000", 2024-09-20T18:54:04.646 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.171546802 2024-09-20T18:54:04.646 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.646 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.646 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:04.646 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444881+0000", 2024-09-20T18:54:04.646 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0031329589999999998 2024-09-20T18:54:04.646 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.647 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.647 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:54:04.647 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444940+0000", 2024-09-20T18:54:04.647 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9416999999999997e-05 2024-09-20T18:54:04.647 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.647 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.647 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:54:04.647 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445031+0000", 2024-09-20T18:54:04.647 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1393999999999999e-05 2024-09-20T18:54:04.647 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:04.647 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:54:04.647 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:04.647 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.647 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.648 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:25343 7.6 7:61f2dd3b:::benchmark_data_smithi083_70643_object25342:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:54:04.648 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.270643+0000", 2024-09-20T18:54:04.648 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 177.32664941499999, 2024-09-20T18:54:04.648 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17748241200000001, 2024-09-20T18:54:04.648 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:54:04.648 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:54:04.648 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:54:04.648 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:54:04.648 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:54:04.648 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:54:04.648 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25343 2024-09-20T18:54:04.648 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.648 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:54:04.648 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.648 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:54:04.649 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270643+0000", 2024-09-20T18:54:04.649 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:04.649 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.649 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.649 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:54:04.649 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270643+0000", 2024-09-20T18:54:04.649 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:04.649 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.649 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.649 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:54:04.649 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270645+0000", 2024-09-20T18:54:04.649 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4810000000000001e-06 2024-09-20T18:54:04.649 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.649 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.650 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:54:04.650 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270664+0000", 2024-09-20T18:54:04.650 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9709999999999999e-05 2024-09-20T18:54:04.650 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.650 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.650 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:54:04.650 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270674+0000", 2024-09-20T18:54:04.650 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6120000000000005e-06 2024-09-20T18:54:04.650 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.650 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.650 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:54:04.650 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270680+0000", 2024-09-20T18:54:04.650 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7089999999999999e-06 2024-09-20T18:54:04.650 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.651 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.651 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:54:04.651 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270841+0000", 2024-09-20T18:54:04.651 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016147500000000001 2024-09-20T18:54:04.651 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.651 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.651 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:54:04.651 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271023+0000", 2024-09-20T18:54:04.651 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000181836 2024-09-20T18:54:04.651 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.651 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.651 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:54:04.651 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271282+0000", 2024-09-20T18:54:04.651 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00025898300000000001 2024-09-20T18:54:04.651 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.652 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.652 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:54:04.652 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441716+0000", 2024-09-20T18:54:04.652 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17043377800000001 2024-09-20T18:54:04.652 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.652 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.652 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:04.652 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441787+0000", 2024-09-20T18:54:04.652 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1465999999999996e-05 2024-09-20T18:54:04.652 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.652 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.652 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:04.652 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.448030+0000", 2024-09-20T18:54:04.652 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0062427089999999999 2024-09-20T18:54:04.652 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.653 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.653 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:54:04.653 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.448081+0000", 2024-09-20T18:54:04.653 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1255000000000003e-05 2024-09-20T18:54:04.653 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.653 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.653 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:54:04.653 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.448126+0000", 2024-09-20T18:54:04.653 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4397999999999997e-05 2024-09-20T18:54:04.653 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:04.653 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:54:04.653 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:04.653 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.653 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.653 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:25344 7.7 e93/83 7:ead30b22:::benchmark_data_smithi083_70643_object25343:head v 93'1577, mlcod=93'1575)", 2024-09-20T18:54:04.654 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.270793+0000", 2024-09-20T18:54:04.654 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 177.32649940799999, 2024-09-20T18:54:04.654 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17078492100000001, 2024-09-20T18:54:04.654 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:54:04.654 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:54:04.654 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:54:04.654 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:54:04.654 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.654 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:54:04.654 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270793+0000", 2024-09-20T18:54:04.654 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:04.654 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.654 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.654 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:54:04.655 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270793+0000", 2024-09-20T18:54:04.655 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:04.655 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.655 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.655 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:54:04.655 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270794+0000", 2024-09-20T18:54:04.655 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9100000000000004e-07 2024-09-20T18:54:04.655 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.655 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.655 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:54:04.655 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270929+0000", 2024-09-20T18:54:04.655 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000134795 2024-09-20T18:54:04.655 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.655 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.656 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:54:04.656 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270941+0000", 2024-09-20T18:54:04.656 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2431e-05 2024-09-20T18:54:04.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.656 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.656 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:54:04.656 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270948+0000", 2024-09-20T18:54:04.656 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8109999999999997e-06 2024-09-20T18:54:04.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.656 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.656 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:54:04.656 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271000+0000", 2024-09-20T18:54:04.656 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2122e-05 2024-09-20T18:54:04.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.656 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.657 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:54:04.657 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271019+0000", 2024-09-20T18:54:04.657 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9490000000000001e-05 2024-09-20T18:54:04.657 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.657 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.657 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:54:04.657 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441531+0000", 2024-09-20T18:54:04.657 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.170511842 2024-09-20T18:54:04.657 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.657 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.657 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:54:04.657 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441578+0000", 2024-09-20T18:54:04.657 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6839000000000003e-05 2024-09-20T18:54:04.657 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:04.657 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:54:04.658 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:04.658 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.658 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.658 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:25335 7.6 e93/83 ondisk, result = 0)", 2024-09-20T18:54:04.658 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.270924+0000", 2024-09-20T18:54:04.658 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 177.326368621, 2024-09-20T18:54:04.658 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.170638343, 2024-09-20T18:54:04.658 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:54:04.658 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:54:04.658 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:54:04.658 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:54:04.658 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.658 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:54:04.658 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270924+0000", 2024-09-20T18:54:04.659 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:04.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.659 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.659 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:54:04.659 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270924+0000", 2024-09-20T18:54:04.659 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:04.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.659 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.659 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:54:04.659 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270925+0000", 2024-09-20T18:54:04.659 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8899999999999999e-07 2024-09-20T18:54:04.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.659 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.659 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:54:04.659 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270929+0000", 2024-09-20T18:54:04.660 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9829999999999998e-06 2024-09-20T18:54:04.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.660 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.660 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:54:04.660 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270936+0000", 2024-09-20T18:54:04.660 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5900000000000002e-06 2024-09-20T18:54:04.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.660 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.660 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:54:04.660 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270942+0000", 2024-09-20T18:54:04.660 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8420000000000001e-06 2024-09-20T18:54:04.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.660 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.660 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:54:04.661 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441532+0000", 2024-09-20T18:54:04.661 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17059043900000001 2024-09-20T18:54:04.661 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.661 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.661 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:54:04.661 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441543+0000", 2024-09-20T18:54:04.661 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0549e-05 2024-09-20T18:54:04.661 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.661 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.661 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:54:04.661 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441562+0000", 2024-09-20T18:54:04.661 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9250999999999999e-05 2024-09-20T18:54:04.661 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:04.661 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:54:04.661 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:04.662 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.662 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.662 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:25345 7.4 7:2557809c:::benchmark_data_smithi083_70643_object25344:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:54:04.662 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.270978+0000", 2024-09-20T18:54:04.662 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 177.326314341, 2024-09-20T18:54:04.662 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.170924456, 2024-09-20T18:54:04.662 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:54:04.662 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:54:04.662 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:54:04.662 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:54:04.662 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:54:04.662 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:54:04.662 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25345 2024-09-20T18:54:04.663 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.663 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:54:04.663 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.663 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:54:04.663 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270978+0000", 2024-09-20T18:54:04.663 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:04.663 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.663 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.663 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:54:04.663 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270978+0000", 2024-09-20T18:54:04.663 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:04.663 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.663 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.664 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:54:04.664 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270979+0000", 2024-09-20T18:54:04.664 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9400000000000004e-07 2024-09-20T18:54:04.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.664 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.664 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:54:04.664 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271001+0000", 2024-09-20T18:54:04.664 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1520000000000001e-05 2024-09-20T18:54:04.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.664 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.664 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:54:04.664 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271008+0000", 2024-09-20T18:54:04.664 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4699999999999996e-06 2024-09-20T18:54:04.665 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.665 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.665 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:54:04.665 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271015+0000", 2024-09-20T18:54:04.665 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3529999999999997e-06 2024-09-20T18:54:04.665 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.665 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.665 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:54:04.665 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271108+0000", 2024-09-20T18:54:04.665 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2194999999999994e-05 2024-09-20T18:54:04.665 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.665 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.665 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:54:04.665 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271218+0000", 2024-09-20T18:54:04.666 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000110797 2024-09-20T18:54:04.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.666 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.666 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,4", 2024-09-20T18:54:04.666 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271334+0000", 2024-09-20T18:54:04.666 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115202 2024-09-20T18:54:04.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.666 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.666 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:04.666 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441629+0000", 2024-09-20T18:54:04.666 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.170295053 2024-09-20T18:54:04.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.666 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.666 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:54:04.667 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441647+0000", 2024-09-20T18:54:04.667 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8465999999999999e-05 2024-09-20T18:54:04.667 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.667 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.667 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:04.667 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441814+0000", 2024-09-20T18:54:04.667 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016683799999999999 2024-09-20T18:54:04.667 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.667 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.667 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:54:04.667 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441869+0000", 2024-09-20T18:54:04.667 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4693e-05 2024-09-20T18:54:04.667 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.667 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.667 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:54:04.668 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441903+0000", 2024-09-20T18:54:04.668 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3974999999999999e-05 2024-09-20T18:54:04.668 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:04.668 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:54:04.668 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:04.668 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.668 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.668 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:25335 7.6 e93/83 ondisk, result = 0)", 2024-09-20T18:54:04.668 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.270989+0000", 2024-09-20T18:54:04.668 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 177.32630379400001, 2024-09-20T18:54:04.668 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17072014499999999, 2024-09-20T18:54:04.668 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:54:04.668 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:54:04.668 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:54:04.669 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:54:04.669 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.669 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:54:04.669 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270989+0000", 2024-09-20T18:54:04.669 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:04.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.669 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.669 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:54:04.669 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270989+0000", 2024-09-20T18:54:04.669 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:04.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.669 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.669 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:54:04.669 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270990+0000", 2024-09-20T18:54:04.669 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.1200000000000002e-07 2024-09-20T18:54:04.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.670 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.670 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:54:04.670 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270994+0000", 2024-09-20T18:54:04.670 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2119999999999997e-06 2024-09-20T18:54:04.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.670 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.670 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:54:04.670 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271010+0000", 2024-09-20T18:54:04.670 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6070999999999999e-05 2024-09-20T18:54:04.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.670 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.670 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:54:04.670 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271016+0000", 2024-09-20T18:54:04.671 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2859999999999996e-06 2024-09-20T18:54:04.671 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.671 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.671 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:54:04.671 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441586+0000", 2024-09-20T18:54:04.671 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17057012499999999 2024-09-20T18:54:04.671 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.671 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.671 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:54:04.671 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441592+0000", 2024-09-20T18:54:04.671 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8690000000000001e-06 2024-09-20T18:54:04.671 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.671 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.671 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:54:04.671 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441709+0000", 2024-09-20T18:54:04.672 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011677 2024-09-20T18:54:04.673 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:04.673 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:54:04.673 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:04.673 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.673 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.673 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:25342 7.5 e93/83 7:ae6f0d57:::benchmark_data_smithi083_70643_object25341:head v 93'1574, mlcod=93'1572)", 2024-09-20T18:54:04.673 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.271037+0000", 2024-09-20T18:54:04.673 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 177.32625562999999, 2024-09-20T18:54:04.673 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.170762196, 2024-09-20T18:54:04.673 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:54:04.673 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:54:04.673 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:54:04.674 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:54:04.674 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.674 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:54:04.674 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271037+0000", 2024-09-20T18:54:04.674 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:04.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.674 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.674 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:54:04.674 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271037+0000", 2024-09-20T18:54:04.674 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:04.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.674 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.674 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:54:04.675 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271037+0000", 2024-09-20T18:54:04.675 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9700000000000002e-07 2024-09-20T18:54:04.675 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.675 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.675 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:54:04.675 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271069+0000", 2024-09-20T18:54:04.675 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1647999999999999e-05 2024-09-20T18:54:04.675 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.675 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.675 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:54:04.675 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271075+0000", 2024-09-20T18:54:04.675 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0889999999999996e-06 2024-09-20T18:54:04.675 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.675 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.676 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:54:04.676 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271080+0000", 2024-09-20T18:54:04.676 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5129999999999998e-06 2024-09-20T18:54:04.676 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.676 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.676 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:54:04.676 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271147+0000", 2024-09-20T18:54:04.676 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7470000000000003e-05 2024-09-20T18:54:04.676 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.676 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.676 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:54:04.676 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271165+0000", 2024-09-20T18:54:04.676 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8292e-05 2024-09-20T18:54:04.676 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.676 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.677 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:54:04.677 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441764+0000", 2024-09-20T18:54:04.677 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.170598789 2024-09-20T18:54:04.677 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.677 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.677 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:54:04.677 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441799+0000", 2024-09-20T18:54:04.677 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4997999999999999e-05 2024-09-20T18:54:04.677 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:04.677 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:54:04.677 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:04.677 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.677 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.677 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:25338 7.9 e93/83 ondisk, result = 0)", 2024-09-20T18:54:04.678 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.271688+0000", 2024-09-20T18:54:04.678 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 177.32560498800001, 2024-09-20T18:54:04.678 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.170058869, 2024-09-20T18:54:04.678 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:54:04.678 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:54:04.678 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:54:04.678 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:54:04.678 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.678 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:54:04.678 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271688+0000", 2024-09-20T18:54:04.678 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:04.678 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.678 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.678 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:54:04.679 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271688+0000", 2024-09-20T18:54:04.679 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:04.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.679 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.679 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:54:04.679 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271688+0000", 2024-09-20T18:54:04.679 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5600000000000005e-07 2024-09-20T18:54:04.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.679 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.679 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:54:04.679 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271692+0000", 2024-09-20T18:54:04.679 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.844e-06 2024-09-20T18:54:04.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.679 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.680 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:54:04.680 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271700+0000", 2024-09-20T18:54:04.680 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2619999999999998e-06 2024-09-20T18:54:04.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.680 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.680 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:54:04.680 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271706+0000", 2024-09-20T18:54:04.680 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8019999999999997e-06 2024-09-20T18:54:04.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.680 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.680 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:54:04.680 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271746+0000", 2024-09-20T18:54:04.680 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0163999999999997e-05 2024-09-20T18:54:04.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.680 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.681 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:54:04.681 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271752+0000", 2024-09-20T18:54:04.681 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.896e-06 2024-09-20T18:54:04.681 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.681 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.681 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:54:04.681 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441746+0000", 2024-09-20T18:54:04.681 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.16999414500000001 2024-09-20T18:54:04.681 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:04.681 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:54:04.681 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:04.681 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.681 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.681 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:25340 7.6 e93/83 ondisk, result = 0)", 2024-09-20T18:54:04.682 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.272049+0000", 2024-09-20T18:54:04.682 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 177.325243613, 2024-09-20T18:54:04.682 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.16970538299999999, 2024-09-20T18:54:04.682 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:54:04.682 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:54:04.682 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:54:04.682 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:54:04.682 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.682 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:54:04.682 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.272049+0000", 2024-09-20T18:54:04.682 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:04.682 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.682 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.682 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:54:04.682 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.272049+0000", 2024-09-20T18:54:04.683 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:04.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.683 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.683 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:54:04.683 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.272049+0000", 2024-09-20T18:54:04.683 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7999999999999996e-07 2024-09-20T18:54:04.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.683 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.683 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:54:04.683 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.272053+0000", 2024-09-20T18:54:04.683 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4970000000000001e-06 2024-09-20T18:54:04.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.683 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.683 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:54:04.684 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441627+0000", 2024-09-20T18:54:04.684 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.169574321 2024-09-20T18:54:04.684 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.684 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.684 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:54:04.684 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441637+0000", 2024-09-20T18:54:04.684 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9750000000000002e-06 2024-09-20T18:54:04.684 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.684 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.684 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:54:04.684 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441738+0000", 2024-09-20T18:54:04.684 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000100342 2024-09-20T18:54:04.684 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.684 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.684 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:54:04.685 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441743+0000", 2024-09-20T18:54:04.685 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4280000000000004e-06 2024-09-20T18:54:04.685 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.685 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.685 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:54:04.685 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441754+0000", 2024-09-20T18:54:04.685 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.134e-05 2024-09-20T18:54:04.685 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:04.685 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:54:04.685 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:04.685 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.685 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.685 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:25345 7.4 e93/83 ondisk, result = 0)", 2024-09-20T18:54:04.685 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.272463+0000", 2024-09-20T18:54:04.686 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 177.32482914299999, 2024-09-20T18:54:04.686 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.169185207, 2024-09-20T18:54:04.686 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:54:04.686 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:54:04.686 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:54:04.686 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:54:04.686 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.686 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:54:04.686 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.272463+0000", 2024-09-20T18:54:04.686 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:04.686 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.686 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.686 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:54:04.686 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.272463+0000", 2024-09-20T18:54:04.687 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:04.687 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.687 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.687 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:54:04.687 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.272464+0000", 2024-09-20T18:54:04.687 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9999999999999997e-07 2024-09-20T18:54:04.687 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.687 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.687 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:54:04.687 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.272468+0000", 2024-09-20T18:54:04.687 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9779999999999999e-06 2024-09-20T18:54:04.687 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.687 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.687 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:54:04.688 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441420+0000", 2024-09-20T18:54:04.688 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.16895175900000001 2024-09-20T18:54:04.688 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.688 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.688 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:54:04.688 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441438+0000", 2024-09-20T18:54:04.688 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8244999999999999e-05 2024-09-20T18:54:04.688 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.688 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.688 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:54:04.688 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441615+0000", 2024-09-20T18:54:04.688 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017724900000000001 2024-09-20T18:54:04.688 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.688 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.688 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:54:04.689 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441623+0000", 2024-09-20T18:54:04.689 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.6399999999999997e-06 2024-09-20T18:54:04.689 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.689 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.689 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:54:04.689 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441649+0000", 2024-09-20T18:54:04.689 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5636e-05 2024-09-20T18:54:04.689 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:04.689 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:54:04.689 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:04.689 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.689 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.689 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:26465 7.6 7:63edb752:::benchmark_data_smithi083_70643_object26464:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:54:04.689 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.995659+0000", 2024-09-20T18:54:04.690 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 176.60163364300001, 2024-09-20T18:54:04.690 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.159355475, 2024-09-20T18:54:04.690 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:54:04.690 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:54:04.690 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:54:04.690 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:54:04.690 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:54:04.690 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:54:04.690 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 26465 2024-09-20T18:54:04.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.690 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:54:04.690 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.690 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:54:04.690 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.995659+0000", 2024-09-20T18:54:04.691 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:04.691 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.691 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.691 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:54:04.691 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.995659+0000", 2024-09-20T18:54:04.691 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:04.691 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.691 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.691 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:54:04.691 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.995660+0000", 2024-09-20T18:54:04.691 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2410000000000001e-06 2024-09-20T18:54:04.691 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.691 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.691 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:54:04.691 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.995684+0000", 2024-09-20T18:54:04.692 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3417999999999998e-05 2024-09-20T18:54:04.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.692 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.692 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:54:04.692 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.995693+0000", 2024-09-20T18:54:04.692 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8519999999999999e-06 2024-09-20T18:54:04.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.692 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.692 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:54:04.692 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.995700+0000", 2024-09-20T18:54:04.692 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.849e-06 2024-09-20T18:54:04.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.692 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.692 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:54:04.693 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.996430+0000", 2024-09-20T18:54:04.693 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00072999800000000004 2024-09-20T18:54:04.693 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.693 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.693 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:54:04.693 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.996548+0000", 2024-09-20T18:54:04.693 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000117537 2024-09-20T18:54:04.693 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.693 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.693 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:54:04.693 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.996682+0000", 2024-09-20T18:54:04.693 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000134609 2024-09-20T18:54:04.693 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.693 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.694 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:54:04.694 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.998822+0000", 2024-09-20T18:54:04.694 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0021392630000000002 2024-09-20T18:54:04.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.694 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.694 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:04.694 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.003162+0000", 2024-09-20T18:54:04.694 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0043400319999999997 2024-09-20T18:54:04.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.694 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.694 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:04.694 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.154547+0000", 2024-09-20T18:54:04.694 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.151385617 2024-09-20T18:54:04.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.695 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.695 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:54:04.695 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.154601+0000", 2024-09-20T18:54:04.695 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4146e-05 2024-09-20T18:54:04.695 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.695 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.695 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:54:04.695 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.155014+0000", 2024-09-20T18:54:04.695 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000412913 2024-09-20T18:54:04.695 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:04.695 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:54:04.695 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:04.695 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.695 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.695 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:26466 7.6 7:641ccaab:::benchmark_data_smithi083_70643_object26465:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:54:04.696 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.995720+0000", 2024-09-20T18:54:04.696 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 176.60157239500001, 2024-09-20T18:54:04.696 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.15942645999999999, 2024-09-20T18:54:04.696 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:54:04.696 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:54:04.696 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:54:04.696 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:54:04.696 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:54:04.696 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:54:04.696 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 26466 2024-09-20T18:54:04.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.696 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:54:04.696 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.696 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:54:04.697 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.995720+0000", 2024-09-20T18:54:04.697 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:04.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.697 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.697 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:54:04.697 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.995720+0000", 2024-09-20T18:54:04.697 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:04.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.697 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.697 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:54:04.697 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.995721+0000", 2024-09-20T18:54:04.697 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5199999999999997e-07 2024-09-20T18:54:04.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.697 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.698 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:54:04.698 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.995773+0000", 2024-09-20T18:54:04.698 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2679999999999997e-05 2024-09-20T18:54:04.698 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.698 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.698 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:54:04.698 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.995780+0000", 2024-09-20T18:54:04.698 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6880000000000002e-06 2024-09-20T18:54:04.698 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.698 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.698 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:54:04.698 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.995785+0000", 2024-09-20T18:54:04.698 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3129999999999998e-06 2024-09-20T18:54:04.698 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.698 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.699 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:54:04.699 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.997812+0000", 2024-09-20T18:54:04.699 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0020264710000000002 2024-09-20T18:54:04.699 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.699 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.699 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:54:04.699 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.997923+0000", 2024-09-20T18:54:04.699 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000110788 2024-09-20T18:54:04.699 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.699 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.699 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:54:04.699 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.998039+0000", 2024-09-20T18:54:04.699 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115961 2024-09-20T18:54:04.699 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.700 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.700 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:54:04.700 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.999810+0000", 2024-09-20T18:54:04.700 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001771051 2024-09-20T18:54:04.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.700 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.700 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:04.700 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.003194+0000", 2024-09-20T18:54:04.700 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003383952 2024-09-20T18:54:04.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.700 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.700 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:04.700 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.155059+0000", 2024-09-20T18:54:04.700 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.151865219 2024-09-20T18:54:04.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.701 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.701 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:54:04.701 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.155110+0000", 2024-09-20T18:54:04.701 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1004999999999997e-05 2024-09-20T18:54:04.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.701 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.701 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:54:04.701 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.155147+0000", 2024-09-20T18:54:04.701 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6779999999999997e-05 2024-09-20T18:54:04.701 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:04.701 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:54:04.701 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:04.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.702 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.702 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:26467 7.6 7:6fa2bfec:::benchmark_data_smithi083_70643_object26466:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:54:04.702 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.995794+0000", 2024-09-20T18:54:04.702 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 176.60149898200001, 2024-09-20T18:54:04.702 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.15946484, 2024-09-20T18:54:04.702 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:54:04.702 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:54:04.702 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:54:04.702 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:54:04.702 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:54:04.702 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:54:04.702 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 26467 2024-09-20T18:54:04.702 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.702 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:54:04.703 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.703 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:54:04.703 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.995794+0000", 2024-09-20T18:54:04.703 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:04.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.703 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.703 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:54:04.703 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.995794+0000", 2024-09-20T18:54:04.703 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:04.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.703 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.703 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:54:04.703 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.995794+0000", 2024-09-20T18:54:04.703 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4000000000000002e-07 2024-09-20T18:54:04.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.704 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.704 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:54:04.704 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.995813+0000", 2024-09-20T18:54:04.704 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9134999999999999e-05 2024-09-20T18:54:04.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.704 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.704 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:54:04.704 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.995819+0000", 2024-09-20T18:54:04.704 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.524e-06 2024-09-20T18:54:04.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.704 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.704 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:54:04.704 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.995823+0000", 2024-09-20T18:54:04.704 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6789999999999996e-06 2024-09-20T18:54:04.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.705 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.705 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:54:04.705 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.998856+0000", 2024-09-20T18:54:04.705 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0030330299999999999 2024-09-20T18:54:04.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.705 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.705 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:54:04.705 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.998973+0000", 2024-09-20T18:54:04.705 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000116214 2024-09-20T18:54:04.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.705 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.705 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:54:04.705 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.999103+0000", 2024-09-20T18:54:04.705 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013050200000000001 2024-09-20T18:54:04.706 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.706 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.706 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:54:04.706 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.000382+0000", 2024-09-20T18:54:04.706 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001278532 2024-09-20T18:54:04.706 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.706 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.706 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:04.706 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.003226+0000", 2024-09-20T18:54:04.706 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0028442160000000001 2024-09-20T18:54:04.706 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.706 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.706 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:04.706 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.155183+0000", 2024-09-20T18:54:04.707 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.15195683700000001 2024-09-20T18:54:04.707 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.707 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.707 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:54:04.707 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.155225+0000", 2024-09-20T18:54:04.707 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2528999999999999e-05 2024-09-20T18:54:04.707 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.707 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.707 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:54:04.707 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.155258+0000", 2024-09-20T18:54:04.707 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3201999999999998e-05 2024-09-20T18:54:04.707 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:04.707 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:54:04.707 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:04.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.708 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.708 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:26474 7.f 7:f7988c88:::benchmark_data_smithi083_70643_object26473:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:54:04.708 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.998994+0000", 2024-09-20T18:54:04.708 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 176.59829819199999, 2024-09-20T18:54:04.708 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.157028792, 2024-09-20T18:54:04.708 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:54:04.708 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:54:04.708 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:54:04.708 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:54:04.708 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:54:04.708 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:54:04.708 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 26474 2024-09-20T18:54:04.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.708 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:54:04.709 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.709 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:54:04.709 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.998994+0000", 2024-09-20T18:54:04.709 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:04.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.709 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.709 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:54:04.709 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.998994+0000", 2024-09-20T18:54:04.709 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:04.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.709 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.709 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:54:04.709 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.998995+0000", 2024-09-20T18:54:04.709 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0380000000000001e-06 2024-09-20T18:54:04.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.710 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.710 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:54:04.710 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.999012+0000", 2024-09-20T18:54:04.710 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6949e-05 2024-09-20T18:54:04.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.710 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.710 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:54:04.710 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.999021+0000", 2024-09-20T18:54:04.710 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7239999999999998e-06 2024-09-20T18:54:04.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.710 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.710 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:54:04.710 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.999026+0000", 2024-09-20T18:54:04.710 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7539999999999997e-06 2024-09-20T18:54:04.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.711 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.711 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:54:04.711 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.999072+0000", 2024-09-20T18:54:04.711 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5920000000000001e-05 2024-09-20T18:54:04.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.711 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.711 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:54:04.711 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.999192+0000", 2024-09-20T18:54:04.711 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000119943 2024-09-20T18:54:04.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.711 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.711 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:54:04.711 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.999269+0000", 2024-09-20T18:54:04.711 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.6956e-05 2024-09-20T18:54:04.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.712 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.712 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:54:04.712 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.000374+0000", 2024-09-20T18:54:04.712 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011057829999999999 2024-09-20T18:54:04.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.712 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.712 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:04.712 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.001757+0000", 2024-09-20T18:54:04.712 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0013829529999999999 2024-09-20T18:54:04.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.712 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.712 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:04.712 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.155951+0000", 2024-09-20T18:54:04.713 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.15419313300000001 2024-09-20T18:54:04.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.713 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.713 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:54:04.713 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.155990+0000", 2024-09-20T18:54:04.713 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9714e-05 2024-09-20T18:54:04.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.713 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.713 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:54:04.713 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.156023+0000", 2024-09-20T18:54:04.714 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2925000000000001e-05 2024-09-20T18:54:04.714 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:04.714 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:54:04.714 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:04.714 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.715 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.715 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:26476 7.6 7:696c1e7a:::benchmark_data_smithi083_70643_object26475:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:54:04.715 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:08.002180+0000", 2024-09-20T18:54:04.715 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 176.5951129, 2024-09-20T18:54:04.715 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.15334672699999999, 2024-09-20T18:54:04.715 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:54:04.715 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:54:04.715 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:54:04.715 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:54:04.715 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:54:04.715 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:54:04.715 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 26476 2024-09-20T18:54:04.715 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.715 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:54:04.716 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.716 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:54:04.716 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.002180+0000", 2024-09-20T18:54:04.716 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:04.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.716 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.716 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:54:04.716 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.002180+0000", 2024-09-20T18:54:04.716 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:04.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.716 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.716 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:54:04.716 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.002181+0000", 2024-09-20T18:54:04.716 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.173e-06 2024-09-20T18:54:04.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.717 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.717 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:54:04.717 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.002198+0000", 2024-09-20T18:54:04.717 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6895999999999999e-05 2024-09-20T18:54:04.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.717 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.717 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:54:04.717 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.002205+0000", 2024-09-20T18:54:04.717 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.6590000000000007e-06 2024-09-20T18:54:04.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.717 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.717 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:54:04.717 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.002210+0000", 2024-09-20T18:54:04.717 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6099999999999999e-06 2024-09-20T18:54:04.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.718 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.718 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:54:04.718 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.002251+0000", 2024-09-20T18:54:04.718 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0828000000000003e-05 2024-09-20T18:54:04.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.718 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.718 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:54:04.718 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.002333+0000", 2024-09-20T18:54:04.718 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2410000000000005e-05 2024-09-20T18:54:04.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.718 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.718 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:54:04.718 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.002416+0000", 2024-09-20T18:54:04.719 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2363999999999999e-05 2024-09-20T18:54:04.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.719 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.719 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:54:04.719 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.003273+0000", 2024-09-20T18:54:04.719 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00085788100000000001 2024-09-20T18:54:04.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.719 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.719 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:04.719 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.004070+0000", 2024-09-20T18:54:04.719 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000796644 2024-09-20T18:54:04.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.719 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.719 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:04.719 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.155292+0000", 2024-09-20T18:54:04.720 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.15122216099999999 2024-09-20T18:54:04.720 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.720 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.720 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:54:04.720 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.155494+0000", 2024-09-20T18:54:04.720 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020213300000000001 2024-09-20T18:54:04.720 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.720 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.720 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:54:04.720 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.155526+0000", 2024-09-20T18:54:04.720 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1968000000000003e-05 2024-09-20T18:54:04.720 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:04.720 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:54:04.720 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:04.721 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.721 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.721 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:26479 7.d e93/83 7:bdad06c1:::benchmark_data_smithi083_70643_object26478:head v 93'1700, mlcod=93'1697)", 2024-09-20T18:54:04.721 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:08.005291+0000", 2024-09-20T18:54:04.721 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 176.592001909, 2024-09-20T18:54:04.721 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.14962600100000001, 2024-09-20T18:54:04.721 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:54:04.721 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:54:04.721 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:54:04.721 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:54:04.721 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.721 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:54:04.721 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.005291+0000", 2024-09-20T18:54:04.721 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:04.721 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.722 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.722 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:54:04.722 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.005291+0000", 2024-09-20T18:54:04.722 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:04.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.722 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.722 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:54:04.722 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.005291+0000", 2024-09-20T18:54:04.722 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1999999999999999e-07 2024-09-20T18:54:04.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.722 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.722 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:54:04.722 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.005402+0000", 2024-09-20T18:54:04.722 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000110463 2024-09-20T18:54:04.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.723 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.723 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:54:04.723 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.153269+0000", 2024-09-20T18:54:04.723 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.14786682100000001 2024-09-20T18:54:04.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.723 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.723 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:54:04.723 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.153279+0000", 2024-09-20T18:54:04.723 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0431999999999999e-05 2024-09-20T18:54:04.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.723 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.723 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:54:04.723 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.153448+0000", 2024-09-20T18:54:04.723 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016901899999999999 2024-09-20T18:54:04.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.724 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.724 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:54:04.724 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.153470+0000", 2024-09-20T18:54:04.724 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1747999999999999e-05 2024-09-20T18:54:04.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.724 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.724 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:54:04.724 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.154874+0000", 2024-09-20T18:54:04.724 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001403767 2024-09-20T18:54:04.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:04.724 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:04.724 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:54:04.724 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.154917+0000", 2024-09-20T18:54:04.724 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3130999999999998e-05 2024-09-20T18:54:04.725 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:04.725 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:54:04.725 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:04.725 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:04.725 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:54:04.725 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T18:54:04.725 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:04 smithi083 ceph-mon[29421]: pgmap v195: 148 pgs: 148 active+clean; 1.5 GiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.31k op/s 2024-09-20T18:54:04.726 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:04 smithi083 ceph-mon[36260]: pgmap v195: 148 pgs: 148 active+clean; 1.5 GiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.31k op/s 2024-09-20T18:54:04.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:04.726 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:04.726 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:04.827 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:04.853 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:04.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:04.854 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:04.954 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:54:04.981 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:04.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:04.982 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:54:05.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:04 smithi142 ceph-mon[32508]: pgmap v195: 148 pgs: 148 active+clean; 1.5 GiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.31k op/s 2024-09-20T18:54:05.083 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:54:05.106 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:05.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:05.107 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:54:05.208 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:05.233 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:05.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:05.234 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:05.244 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:54:05.251 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-20T18:54:05.334 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:05.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:05.375 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:05.375 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:05.476 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:05.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:05.508 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:05.508 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:05.609 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:54:05.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:05.640 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:05.640 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:54:05.741 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:05.786 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:05.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:05.787 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:05.888 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:05.921 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:05.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:05.922 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:06.023 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:06.060 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:06.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:06.062 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:06.162 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:54:06.192 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:06.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:06.193 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:54:06.294 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:06.319 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:06.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:06.321 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:06.421 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:06.459 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:06.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:06.460 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:06.561 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:54:06.587 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:06.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:06.588 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:54:06.689 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:54:06.716 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:06.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:06.717 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:54:06.818 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:54:06.844 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:06.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:06.845 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:54:06.946 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:54:06.973 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:06.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:06.974 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:54:07.024 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:54:06 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:54:07.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:06 smithi142 ceph-mon[32508]: pgmap v196: 148 pgs: 148 active+clean; 1.5 GiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 660 op/s 2024-09-20T18:54:07.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:06 smithi142 ceph-mon[32508]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:54:07.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:06 smithi083 ceph-mon[29421]: pgmap v196: 148 pgs: 148 active+clean; 1.5 GiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 660 op/s 2024-09-20T18:54:07.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:06 smithi083 ceph-mon[29421]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:54:07.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:06 smithi083 ceph-mon[36260]: pgmap v196: 148 pgs: 148 active+clean; 1.5 GiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 660 op/s 2024-09-20T18:54:07.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:06 smithi083 ceph-mon[36260]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:54:07.075 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:07.104 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:07.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:07.105 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:07.206 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:07.244 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:07.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:07.245 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:07.345 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:07.391 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":196,"stamp":"2024-09-20T18:54:04.839923+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1630280243,"num_objects":25067,"num_object_clones":0,"num_object_copies":75201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25067,"num_whiteouts":0,"num_read":1099,"num_read_kb":956,"num_write":221281,"num_write_kb":7860810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":590808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":90478,"ondisk_log_size":90478,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":5816464,"kb_used_data":4901544,"kb_used_omap":12,"kb_used_meta":914291,"kb_avail":744144752,"statfs":{"total":767960285184,"available":762004226048,"internally_reserved":0,"allocated":5019181056,"data_stored":5016134785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":936234544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-345964544,"num_objects":-5279,"num_object_clones":0,"num_object_copies":-15837,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5279,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5279,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003893"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":206,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.957631+0000","last_change":"2024-09-20T18:51:57.238254+0000","last_active":"2024-09-20T18:53:59.957631+0000","last_peered":"2024-09-20T18:53:59.957631+0000","last_clean":"2024-09-20T18:53:59.957631+0000","last_became_active":"2024-09-20T18:49:45.943201+0000","last_became_peered":"2024-09-20T18:49:45.943201+0000","last_unstale":"2024-09-20T18:53:59.957631+0000","last_undegraded":"2024-09-20T18:53:59.957631+0000","last_fullsized":"2024-09-20T18:53:59.957631+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:57.238177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:57.238177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:06.040036+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":243,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.944345+0000","last_change":"2024-09-20T18:53:15.772300+0000","last_active":"2024-09-20T18:53:59.944345+0000","last_peered":"2024-09-20T18:53:59.944345+0000","last_clean":"2024-09-20T18:53:59.944345+0000","last_became_active":"2024-09-20T18:52:15.609648+0000","last_became_peered":"2024-09-20T18:52:15.609648+0000","last_unstale":"2024-09-20T18:53:59.944345+0000","last_undegraded":"2024-09-20T18:53:59.944345+0000","last_fullsized":"2024-09-20T18:53:59.944345+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:53:15.772228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:15.772228+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:33.912905+0000","scrub_duration":37,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":206,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.961071+0000","last_change":"2024-09-20T18:53:21.291759+0000","last_active":"2024-09-20T18:53:59.961071+0000","last_peered":"2024-09-20T18:53:59.961071+0000","last_clean":"2024-09-20T18:53:59.961071+0000","last_became_active":"2024-09-20T18:49:49.959514+0000","last_became_peered":"2024-09-20T18:49:49.959514+0000","last_unstale":"2024-09-20T18:53:59.961071+0000","last_undegraded":"2024-09-20T18:53:59.961071+0000","last_fullsized":"2024-09-20T18:53:59.961071+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:21.291695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:21.291695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:25.316294+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":202,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.944472+0000","last_change":"2024-09-20T18:53:08.740138+0000","last_active":"2024-09-20T18:53:59.944472+0000","last_peered":"2024-09-20T18:53:59.944472+0000","last_clean":"2024-09-20T18:53:59.944472+0000","last_became_active":"2024-09-20T18:49:51.969421+0000","last_became_peered":"2024-09-20T18:49:51.969421+0000","last_unstale":"2024-09-20T18:53:59.944472+0000","last_undegraded":"2024-09-20T18:53:59.944472+0000","last_fullsized":"2024-09-20T18:53:59.944472+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:08.740093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:08.740093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:11.987287+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":202,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.960760+0000","last_change":"2024-09-20T18:53:46.283952+0000","last_active":"2024-09-20T18:53:59.960760+0000","last_peered":"2024-09-20T18:53:59.960760+0000","last_clean":"2024-09-20T18:53:59.960760+0000","last_became_active":"2024-09-20T18:49:52.141967+0000","last_became_peered":"2024-09-20T18:49:52.141967+0000","last_unstale":"2024-09-20T18:53:59.960760+0000","last_undegraded":"2024-09-20T18:53:59.960760+0000","last_fullsized":"2024-09-20T18:53:59.960760+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:46.283881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:46.283881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:49.076672+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":207,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.965251+0000","last_change":"2024-09-20T18:51:25.208165+0000","last_active":"2024-09-20T18:53:59.965251+0000","last_peered":"2024-09-20T18:53:59.965251+0000","last_clean":"2024-09-20T18:53:59.965251+0000","last_became_active":"2024-09-20T18:49:45.943607+0000","last_became_peered":"2024-09-20T18:49:45.943607+0000","last_unstale":"2024-09-20T18:53:59.965251+0000","last_undegraded":"2024-09-20T18:53:59.965251+0000","last_fullsized":"2024-09-20T18:53:59.965251+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:25.208073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:25.208073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:07.185746+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":227,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.962409+0000","last_change":"2024-09-20T18:51:07.130137+0000","last_active":"2024-09-20T18:53:59.962409+0000","last_peered":"2024-09-20T18:53:59.962409+0000","last_clean":"2024-09-20T18:53:59.962409+0000","last_became_active":"2024-09-20T18:49:47.952150+0000","last_became_peered":"2024-09-20T18:49:47.952150+0000","last_unstale":"2024-09-20T18:53:59.962409+0000","last_undegraded":"2024-09-20T18:53:59.962409+0000","last_fullsized":"2024-09-20T18:53:59.962409+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:51:07.130062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:07.130062+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:33.208818+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":198,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.944276+0000","last_change":"2024-09-20T18:51:09.436429+0000","last_active":"2024-09-20T18:53:59.944276+0000","last_peered":"2024-09-20T18:53:59.944276+0000","last_clean":"2024-09-20T18:53:59.944276+0000","last_became_active":"2024-09-20T18:49:49.956445+0000","last_became_peered":"2024-09-20T18:49:49.956445+0000","last_unstale":"2024-09-20T18:53:59.944276+0000","last_undegraded":"2024-09-20T18:53:59.944276+0000","last_fullsized":"2024-09-20T18:53:59.944276+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:09.436302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:09.436302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:04.774926+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"59'1","reported_seq":219,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.963481+0000","last_change":"2024-09-20T18:52:48.035578+0000","last_active":"2024-09-20T18:53:59.963481+0000","last_peered":"2024-09-20T18:53:59.963481+0000","last_clean":"2024-09-20T18:53:59.963481+0000","last_became_active":"2024-09-20T18:49:45.946171+0000","last_became_peered":"2024-09-20T18:49:45.946171+0000","last_unstale":"2024-09-20T18:53:59.963481+0000","last_undegraded":"2024-09-20T18:53:59.963481+0000","last_fullsized":"2024-09-20T18:53:59.963481+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:52:48.035532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:48.035532+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:01.343903+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":226,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.944631+0000","last_change":"2024-09-20T18:53:19.836082+0000","last_active":"2024-09-20T18:53:59.944631+0000","last_peered":"2024-09-20T18:53:59.944631+0000","last_clean":"2024-09-20T18:53:59.944631+0000","last_became_active":"2024-09-20T18:49:47.947448+0000","last_became_peered":"2024-09-20T18:49:47.947448+0000","last_unstale":"2024-09-20T18:53:59.944631+0000","last_undegraded":"2024-09-20T18:53:59.944631+0000","last_fullsized":"2024-09-20T18:53:59.944631+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:53:19.835996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:19.835996+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:39.939119+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":205,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.963054+0000","last_change":"2024-09-20T18:52:36.183932+0000","last_active":"2024-09-20T18:53:59.963054+0000","last_peered":"2024-09-20T18:53:59.963054+0000","last_clean":"2024-09-20T18:53:59.963054+0000","last_became_active":"2024-09-20T18:49:49.961188+0000","last_became_peered":"2024-09-20T18:49:49.961188+0000","last_unstale":"2024-09-20T18:53:59.963054+0000","last_undegraded":"2024-09-20T18:53:59.963054+0000","last_fullsized":"2024-09-20T18:53:59.963054+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:36.183875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:36.183875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":185,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.951051+0000","last_change":"2024-09-20T18:49:51.966957+0000","last_active":"2024-09-20T18:53:59.951051+0000","last_peered":"2024-09-20T18:53:59.951051+0000","last_clean":"2024-09-20T18:53:59.951051+0000","last_became_active":"2024-09-20T18:49:51.966715+0000","last_became_peered":"2024-09-20T18:49:51.966715+0000","last_unstale":"2024-09-20T18:53:59.951051+0000","last_undegraded":"2024-09-20T18:53:59.951051+0000","last_fullsized":"2024-09-20T18:53:59.951051+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":205,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.963229+0000","last_change":"2024-09-20T18:52:00.970719+0000","last_active":"2024-09-20T18:53:59.963229+0000","last_peered":"2024-09-20T18:53:59.963229+0000","last_clean":"2024-09-20T18:53:59.963229+0000","last_became_active":"2024-09-20T18:49:45.944703+0000","last_became_peered":"2024-09-20T18:49:45.944703+0000","last_unstale":"2024-09-20T18:53:59.963229+0000","last_undegraded":"2024-09-20T18:53:59.963229+0000","last_fullsized":"2024-09-20T18:53:59.963229+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:00.970650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:00.970650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:07.049473+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":238,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.944660+0000","last_change":"2024-09-20T18:51:26.494194+0000","last_active":"2024-09-20T18:53:59.944660+0000","last_peered":"2024-09-20T18:53:59.944660+0000","last_clean":"2024-09-20T18:53:59.944660+0000","last_became_active":"2024-09-20T18:51:23.813399+0000","last_became_peered":"2024-09-20T18:51:23.813399+0000","last_unstale":"2024-09-20T18:53:59.944660+0000","last_undegraded":"2024-09-20T18:53:59.944660+0000","last_fullsized":"2024-09-20T18:53:59.944660+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-20T18:51:26.494118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:26.494118+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:34.803592+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":205,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.958042+0000","last_change":"2024-09-20T18:52:38.799542+0000","last_active":"2024-09-20T18:53:59.958042+0000","last_peered":"2024-09-20T18:53:59.958042+0000","last_clean":"2024-09-20T18:53:59.958042+0000","last_became_active":"2024-09-20T18:49:49.959696+0000","last_became_peered":"2024-09-20T18:49:49.959696+0000","last_unstale":"2024-09-20T18:53:59.958042+0000","last_undegraded":"2024-09-20T18:53:59.958042+0000","last_fullsized":"2024-09-20T18:53:59.958042+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:38.799487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:38.799487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:04.479508+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":201,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.963058+0000","last_change":"2024-09-20T18:53:21.273835+0000","last_active":"2024-09-20T18:53:59.963058+0000","last_peered":"2024-09-20T18:53:59.963058+0000","last_clean":"2024-09-20T18:53:59.963058+0000","last_became_active":"2024-09-20T18:49:51.970663+0000","last_became_peered":"2024-09-20T18:49:51.970663+0000","last_unstale":"2024-09-20T18:53:59.963058+0000","last_undegraded":"2024-09-20T18:53:59.963058+0000","last_fullsized":"2024-09-20T18:53:59.963058+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:21.273788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:21.273788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:31.269805+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":203,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.945130+0000","last_change":"2024-09-20T18:53:53.940250+0000","last_active":"2024-09-20T18:53:59.945130+0000","last_peered":"2024-09-20T18:53:59.945130+0000","last_clean":"2024-09-20T18:53:59.945130+0000","last_became_active":"2024-09-20T18:49:52.141170+0000","last_became_peered":"2024-09-20T18:49:52.141170+0000","last_unstale":"2024-09-20T18:53:59.945130+0000","last_undegraded":"2024-09-20T18:53:59.945130+0000","last_fullsized":"2024-09-20T18:53:59.945130+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:53.940191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:53.940191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:59.342447+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":205,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.954789+0000","last_change":"2024-09-20T18:52:42.099489+0000","last_active":"2024-09-20T18:53:59.954789+0000","last_peered":"2024-09-20T18:53:59.954789+0000","last_clean":"2024-09-20T18:53:59.954789+0000","last_became_active":"2024-09-20T18:49:45.950480+0000","last_became_peered":"2024-09-20T18:49:45.950480+0000","last_unstale":"2024-09-20T18:53:59.954789+0000","last_undegraded":"2024-09-20T18:53:59.954789+0000","last_fullsized":"2024-09-20T18:53:59.954789+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:42.099418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:42.099418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:03.758400+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":236,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.956010+0000","last_change":"2024-09-20T18:53:26.762207+0000","last_active":"2024-09-20T18:53:59.956010+0000","last_peered":"2024-09-20T18:53:59.956010+0000","last_clean":"2024-09-20T18:53:59.956010+0000","last_became_active":"2024-09-20T18:49:47.950939+0000","last_became_peered":"2024-09-20T18:49:47.950939+0000","last_unstale":"2024-09-20T18:53:59.956010+0000","last_undegraded":"2024-09-20T18:53:59.956010+0000","last_fullsized":"2024-09-20T18:53:59.956010+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:53:26.762131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:26.762131+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:38.227744+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":198,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.961674+0000","last_change":"2024-09-20T18:51:17.248705+0000","last_active":"2024-09-20T18:53:59.961674+0000","last_peered":"2024-09-20T18:53:59.961674+0000","last_clean":"2024-09-20T18:53:59.961674+0000","last_became_active":"2024-09-20T18:49:49.958518+0000","last_became_peered":"2024-09-20T18:49:49.958518+0000","last_unstale":"2024-09-20T18:53:59.961674+0000","last_undegraded":"2024-09-20T18:53:59.961674+0000","last_fullsized":"2024-09-20T18:53:59.961674+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:17.248634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:17.248634+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:35.297969+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":203,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.965268+0000","last_change":"2024-09-20T18:53:54.259243+0000","last_active":"2024-09-20T18:53:59.965268+0000","last_peered":"2024-09-20T18:53:59.965268+0000","last_clean":"2024-09-20T18:53:59.965268+0000","last_became_active":"2024-09-20T18:49:52.140040+0000","last_became_peered":"2024-09-20T18:49:52.140040+0000","last_unstale":"2024-09-20T18:53:59.965268+0000","last_undegraded":"2024-09-20T18:53:59.965268+0000","last_fullsized":"2024-09-20T18:53:59.965268+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:54.259174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:54.259174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:06.632880+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"59'1","reported_seq":220,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.944731+0000","last_change":"2024-09-20T18:53:42.840884+0000","last_active":"2024-09-20T18:53:59.944731+0000","last_peered":"2024-09-20T18:53:59.944731+0000","last_clean":"2024-09-20T18:53:59.944731+0000","last_became_active":"2024-09-20T18:49:45.940100+0000","last_became_peered":"2024-09-20T18:49:45.940100+0000","last_unstale":"2024-09-20T18:53:59.944731+0000","last_undegraded":"2024-09-20T18:53:59.944731+0000","last_fullsized":"2024-09-20T18:53:59.944731+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:53:42.840800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:42.840800+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:56.760029+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":217,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.960360+0000","last_change":"2024-09-20T18:49:47.951215+0000","last_active":"2024-09-20T18:53:59.960360+0000","last_peered":"2024-09-20T18:53:59.960360+0000","last_clean":"2024-09-20T18:53:59.960360+0000","last_became_active":"2024-09-20T18:49:47.950909+0000","last_became_peered":"2024-09-20T18:49:47.950909+0000","last_unstale":"2024-09-20T18:53:59.960360+0000","last_undegraded":"2024-09-20T18:53:59.960360+0000","last_fullsized":"2024-09-20T18:53:59.960360+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:00.096165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":206,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.944745+0000","last_change":"2024-09-20T18:53:12.820752+0000","last_active":"2024-09-20T18:53:59.944745+0000","last_peered":"2024-09-20T18:53:59.944745+0000","last_clean":"2024-09-20T18:53:59.944745+0000","last_became_active":"2024-09-20T18:49:49.959102+0000","last_became_peered":"2024-09-20T18:49:49.959102+0000","last_unstale":"2024-09-20T18:53:59.944745+0000","last_undegraded":"2024-09-20T18:53:59.944745+0000","last_fullsized":"2024-09-20T18:53:59.944745+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:12.820697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:12.820697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:29.865011+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":202,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.958293+0000","last_change":"2024-09-20T18:52:15.826819+0000","last_active":"2024-09-20T18:53:59.958293+0000","last_peered":"2024-09-20T18:53:59.958293+0000","last_clean":"2024-09-20T18:53:59.958293+0000","last_became_active":"2024-09-20T18:49:51.970058+0000","last_became_peered":"2024-09-20T18:49:51.970058+0000","last_unstale":"2024-09-20T18:53:59.958293+0000","last_undegraded":"2024-09-20T18:53:59.958293+0000","last_fullsized":"2024-09-20T18:53:59.958293+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T18:52:15.826638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:15.826638+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:28.538285+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":207,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.961215+0000","last_change":"2024-09-20T18:52:47.158384+0000","last_active":"2024-09-20T18:53:59.961215+0000","last_peered":"2024-09-20T18:53:59.961215+0000","last_clean":"2024-09-20T18:53:59.961215+0000","last_became_active":"2024-09-20T18:49:45.933078+0000","last_became_peered":"2024-09-20T18:49:45.933078+0000","last_unstale":"2024-09-20T18:53:59.961215+0000","last_undegraded":"2024-09-20T18:53:59.961215+0000","last_fullsized":"2024-09-20T18:53:59.961215+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:52:47.158260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:47.158260+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":217,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.957084+0000","last_change":"2024-09-20T18:52:36.175063+0000","last_active":"2024-09-20T18:53:59.957084+0000","last_peered":"2024-09-20T18:53:59.957084+0000","last_clean":"2024-09-20T18:53:59.957084+0000","last_became_active":"2024-09-20T18:49:48.500809+0000","last_became_peered":"2024-09-20T18:49:48.500809+0000","last_unstale":"2024-09-20T18:53:59.957084+0000","last_undegraded":"2024-09-20T18:53:59.957084+0000","last_fullsized":"2024-09-20T18:53:59.957084+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:52:36.174993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:36.174993+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":268,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.962184+0000","last_change":"2024-09-20T18:53:44.524093+0000","last_active":"2024-09-20T18:53:59.962184+0000","last_peered":"2024-09-20T18:53:59.962184+0000","last_clean":"2024-09-20T18:53:59.962184+0000","last_became_active":"2024-09-20T18:49:49.954772+0000","last_became_peered":"2024-09-20T18:49:49.954772+0000","last_unstale":"2024-09-20T18:53:59.962184+0000","last_undegraded":"2024-09-20T18:53:59.962184+0000","last_fullsized":"2024-09-20T18:53:59.962184+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:53:44.524040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:44.524040+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:03.085945+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.c","version":"152'13892","reported_seq":14153,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:54:00.820731+0000","last_change":"2024-09-20T18:53:47.244353+0000","last_active":"2024-09-20T18:54:00.820731+0000","last_peered":"2024-09-20T18:54:00.820731+0000","last_clean":"2024-09-20T18:54:00.820731+0000","last_became_active":"2024-09-20T18:50:49.843744+0000","last_became_peered":"2024-09-20T18:50:49.843744+0000","last_unstale":"2024-09-20T18:54:00.820731+0000","last_undegraded":"2024-09-20T18:54:00.820731+0000","last_fullsized":"2024-09-20T18:54:00.820731+0000","mapping_epoch":83,"log_start":"122'7600","ondisk_log_start":"122'7600","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'12973","last_scrub_stamp":"2024-09-20T18:53:47.243983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:53:47.243983+0000","objects_scrubbed":2483,"log_size":6292,"log_dups_size":0,"ondisk_log_size":6292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:08.150940+0000","scrub_duration":693,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":101318656,"num_objects":1546,"num_object_clones":0,"num_object_copies":4638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13892,"num_write_kb":494016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":219,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.952384+0000","last_change":"2024-09-20T18:53:48.782289+0000","last_active":"2024-09-20T18:53:59.952384+0000","last_peered":"2024-09-20T18:53:59.952384+0000","last_clean":"2024-09-20T18:53:59.952384+0000","last_became_active":"2024-09-20T18:49:58.992302+0000","last_became_peered":"2024-09-20T18:49:58.992302+0000","last_unstale":"2024-09-20T18:53:59.952384+0000","last_undegraded":"2024-09-20T18:53:59.952384+0000","last_fullsized":"2024-09-20T18:53:59.952384+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:53:48.782192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:48.782192+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:04.815274+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":206,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.960555+0000","last_change":"2024-09-20T18:53:26.175148+0000","last_active":"2024-09-20T18:53:59.960555+0000","last_peered":"2024-09-20T18:53:59.960555+0000","last_clean":"2024-09-20T18:53:59.960555+0000","last_became_active":"2024-09-20T18:49:45.936104+0000","last_became_peered":"2024-09-20T18:49:45.936104+0000","last_unstale":"2024-09-20T18:53:59.960555+0000","last_undegraded":"2024-09-20T18:53:59.960555+0000","last_fullsized":"2024-09-20T18:53:59.960555+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:26.175083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:26.175083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:49.575901+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":260,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.945213+0000","last_change":"2024-09-20T18:51:13.395980+0000","last_active":"2024-09-20T18:53:59.945213+0000","last_peered":"2024-09-20T18:53:59.945213+0000","last_clean":"2024-09-20T18:53:59.945213+0000","last_became_active":"2024-09-20T18:49:49.960697+0000","last_became_peered":"2024-09-20T18:49:49.960697+0000","last_unstale":"2024-09-20T18:53:59.945213+0000","last_undegraded":"2024-09-20T18:53:59.945213+0000","last_fullsized":"2024-09-20T18:53:59.945213+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:51:13.395908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:13.395908+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:07.764237+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":193,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.963102+0000","last_change":"2024-09-20T18:52:43.126389+0000","last_active":"2024-09-20T18:53:59.963102+0000","last_peered":"2024-09-20T18:53:59.963102+0000","last_clean":"2024-09-20T18:53:59.963102+0000","last_became_active":"2024-09-20T18:49:51.971399+0000","last_became_peered":"2024-09-20T18:49:51.971399+0000","last_unstale":"2024-09-20T18:53:59.963102+0000","last_undegraded":"2024-09-20T18:53:59.963102+0000","last_fullsized":"2024-09-20T18:53:59.963102+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:43.126320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:43.126320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"152'13625","reported_seq":13775,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:54:01.145794+0000","last_change":"2024-09-20T18:50:49.841111+0000","last_active":"2024-09-20T18:54:01.145794+0000","last_peered":"2024-09-20T18:54:01.145794+0000","last_clean":"2024-09-20T18:54:01.145794+0000","last_became_active":"2024-09-20T18:50:49.840819+0000","last_became_peered":"2024-09-20T18:50:49.840819+0000","last_unstale":"2024-09-20T18:54:01.145794+0000","last_undegraded":"2024-09-20T18:54:01.145794+0000","last_fullsized":"2024-09-20T18:54:01.145794+0000","mapping_epoch":83,"log_start":"129'9000","ondisk_log_start":"129'9000","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.819660+0000","objects_scrubbed":0,"log_size":4625,"log_dups_size":0,"ondisk_log_size":4625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:38.259712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":105316352,"num_objects":1607,"num_object_clones":0,"num_object_copies":4821,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1607,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13625,"num_write_kb":487424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":235,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.960302+0000","last_change":"2024-09-20T18:52:42.130129+0000","last_active":"2024-09-20T18:53:59.960302+0000","last_peered":"2024-09-20T18:53:59.960302+0000","last_clean":"2024-09-20T18:53:59.960302+0000","last_became_active":"2024-09-20T18:49:48.499189+0000","last_became_peered":"2024-09-20T18:49:48.499189+0000","last_unstale":"2024-09-20T18:53:59.960302+0000","last_undegraded":"2024-09-20T18:53:59.960302+0000","last_fullsized":"2024-09-20T18:53:59.960302+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-20T18:52:42.130045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:42.130045+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":198,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.965581+0000","last_change":"2024-09-20T18:49:45.940816+0000","last_active":"2024-09-20T18:53:59.965581+0000","last_peered":"2024-09-20T18:53:59.965581+0000","last_clean":"2024-09-20T18:53:59.965581+0000","last_became_active":"2024-09-20T18:49:45.940544+0000","last_became_peered":"2024-09-20T18:49:45.940544+0000","last_unstale":"2024-09-20T18:53:59.965581+0000","last_undegraded":"2024-09-20T18:53:59.965581+0000","last_fullsized":"2024-09-20T18:53:59.965581+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:56.293250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":207,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.946667+0000","last_change":"2024-09-20T18:53:09.779280+0000","last_active":"2024-09-20T18:53:59.946667+0000","last_peered":"2024-09-20T18:53:59.946667+0000","last_clean":"2024-09-20T18:53:59.946667+0000","last_became_active":"2024-09-20T18:49:49.963073+0000","last_became_peered":"2024-09-20T18:49:49.963073+0000","last_unstale":"2024-09-20T18:53:59.946667+0000","last_undegraded":"2024-09-20T18:53:59.946667+0000","last_fullsized":"2024-09-20T18:53:59.946667+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:09.779228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:09.779228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:39.433897+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":202,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.944635+0000","last_change":"2024-09-20T18:53:27.796357+0000","last_active":"2024-09-20T18:53:59.944635+0000","last_peered":"2024-09-20T18:53:59.944635+0000","last_clean":"2024-09-20T18:53:59.944635+0000","last_became_active":"2024-09-20T18:49:51.968793+0000","last_became_peered":"2024-09-20T18:49:51.968793+0000","last_unstale":"2024-09-20T18:53:59.944635+0000","last_undegraded":"2024-09-20T18:53:59.944635+0000","last_fullsized":"2024-09-20T18:53:59.944635+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:27.796289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:27.796289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:48.888717+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2","version":"152'13843","reported_seq":13993,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:54:01.207241+0000","last_change":"2024-09-20T18:50:49.842736+0000","last_active":"2024-09-20T18:54:01.207241+0000","last_peered":"2024-09-20T18:54:01.207241+0000","last_clean":"2024-09-20T18:54:01.207241+0000","last_became_active":"2024-09-20T18:50:49.842465+0000","last_became_peered":"2024-09-20T18:50:49.842465+0000","last_unstale":"2024-09-20T18:54:01.207241+0000","last_undegraded":"2024-09-20T18:54:01.207241+0000","last_fullsized":"2024-09-20T18:54:01.207241+0000","mapping_epoch":83,"log_start":"129'9200","ondisk_log_start":"129'9200","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.819660+0000","objects_scrubbed":0,"log_size":4643,"log_dups_size":0,"ondisk_log_size":4643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:31.286096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":101253120,"num_objects":1545,"num_object_clones":0,"num_object_copies":4635,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13843,"num_write_kb":492416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":240,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.946729+0000","last_change":"2024-09-20T18:53:44.879133+0000","last_active":"2024-09-20T18:53:59.946729+0000","last_peered":"2024-09-20T18:53:59.946729+0000","last_clean":"2024-09-20T18:53:59.946729+0000","last_became_active":"2024-09-20T18:49:47.941895+0000","last_became_peered":"2024-09-20T18:49:47.941895+0000","last_unstale":"2024-09-20T18:53:59.946729+0000","last_undegraded":"2024-09-20T18:53:59.946729+0000","last_fullsized":"2024-09-20T18:53:59.946729+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-20T18:53:44.879048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:44.879048+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:52.038099+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":206,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.957334+0000","last_change":"2024-09-20T18:51:27.187060+0000","last_active":"2024-09-20T18:53:59.957334+0000","last_peered":"2024-09-20T18:53:59.957334+0000","last_clean":"2024-09-20T18:53:59.957334+0000","last_became_active":"2024-09-20T18:49:45.940469+0000","last_became_peered":"2024-09-20T18:49:45.940469+0000","last_unstale":"2024-09-20T18:53:59.957334+0000","last_undegraded":"2024-09-20T18:53:59.957334+0000","last_fullsized":"2024-09-20T18:53:59.957334+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:51:27.186991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:27.186991+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:58.816707+0000","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"66'8","reported_seq":267,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.960114+0000","last_change":"2024-09-20T18:53:44.293165+0000","last_active":"2024-09-20T18:53:59.960114+0000","last_peered":"2024-09-20T18:53:59.960114+0000","last_clean":"2024-09-20T18:53:59.960114+0000","last_became_active":"2024-09-20T18:49:49.956133+0000","last_became_peered":"2024-09-20T18:49:49.956133+0000","last_unstale":"2024-09-20T18:53:59.960114+0000","last_undegraded":"2024-09-20T18:53:59.960114+0000","last_fullsized":"2024-09-20T18:53:59.960114+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:53:44.293094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:44.293094+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:47.176037+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":201,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.958193+0000","last_change":"2024-09-20T18:52:42.880767+0000","last_active":"2024-09-20T18:53:59.958193+0000","last_peered":"2024-09-20T18:53:59.958193+0000","last_clean":"2024-09-20T18:53:59.958193+0000","last_became_active":"2024-09-20T18:49:52.142113+0000","last_became_peered":"2024-09-20T18:49:52.142113+0000","last_unstale":"2024-09-20T18:53:59.958193+0000","last_undegraded":"2024-09-20T18:53:59.958193+0000","last_fullsized":"2024-09-20T18:53:59.958193+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:42.880724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:42.880724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:04.690799+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"152'13880","reported_seq":14097,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:54:00.791780+0000","last_change":"2024-09-20T18:52:00.368447+0000","last_active":"2024-09-20T18:54:00.791780+0000","last_peered":"2024-09-20T18:54:00.791780+0000","last_clean":"2024-09-20T18:54:00.791780+0000","last_became_active":"2024-09-20T18:50:49.844624+0000","last_became_peered":"2024-09-20T18:50:49.844624+0000","last_unstale":"2024-09-20T18:54:00.791780+0000","last_undegraded":"2024-09-20T18:54:00.791780+0000","last_fullsized":"2024-09-20T18:54:00.791780+0000","mapping_epoch":83,"log_start":"128'9026","ondisk_log_start":"128'9026","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"111'6092","last_scrub_stamp":"2024-09-20T18:52:00.368276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:52:00.368276+0000","objects_scrubbed":6040,"log_size":4854,"log_dups_size":0,"ondisk_log_size":4854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:40.875204+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":102498304,"num_objects":1564,"num_object_clones":0,"num_object_copies":4692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1564,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13880,"num_write_kb":494208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":226,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.961421+0000","last_change":"2024-09-20T18:52:39.246125+0000","last_active":"2024-09-20T18:53:59.961421+0000","last_peered":"2024-09-20T18:53:59.961421+0000","last_clean":"2024-09-20T18:53:59.961421+0000","last_became_active":"2024-09-20T18:49:47.949626+0000","last_became_peered":"2024-09-20T18:49:47.949626+0000","last_unstale":"2024-09-20T18:53:59.961421+0000","last_undegraded":"2024-09-20T18:53:59.961421+0000","last_fullsized":"2024-09-20T18:53:59.961421+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:52:39.246050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:39.246050+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:07.125503+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":214,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.963617+0000","last_change":"2024-09-20T18:53:11.130341+0000","last_active":"2024-09-20T18:53:59.963617+0000","last_peered":"2024-09-20T18:53:59.963617+0000","last_clean":"2024-09-20T18:53:59.963617+0000","last_became_active":"2024-09-20T18:49:45.942496+0000","last_became_peered":"2024-09-20T18:49:45.942496+0000","last_unstale":"2024-09-20T18:53:59.963617+0000","last_undegraded":"2024-09-20T18:53:59.963617+0000","last_fullsized":"2024-09-20T18:53:59.963617+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:11.130274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:11.130274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:17.935310+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":258,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:54:01.038646+0000","last_change":"2024-09-20T18:51:16.088407+0000","last_active":"2024-09-20T18:54:01.038646+0000","last_peered":"2024-09-20T18:54:01.038646+0000","last_clean":"2024-09-20T18:54:01.038646+0000","last_became_active":"2024-09-20T18:49:49.959365+0000","last_became_peered":"2024-09-20T18:49:49.959365+0000","last_unstale":"2024-09-20T18:54:01.038646+0000","last_undegraded":"2024-09-20T18:54:01.038646+0000","last_fullsized":"2024-09-20T18:54:01.038646+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:51:16.088343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:16.088343+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:35.866980+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":186,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.951173+0000","last_change":"2024-09-20T18:49:51.965733+0000","last_active":"2024-09-20T18:53:59.951173+0000","last_peered":"2024-09-20T18:53:59.951173+0000","last_clean":"2024-09-20T18:53:59.951173+0000","last_became_active":"2024-09-20T18:49:51.964780+0000","last_became_peered":"2024-09-20T18:49:51.964780+0000","last_unstale":"2024-09-20T18:53:59.951173+0000","last_undegraded":"2024-09-20T18:53:59.951173+0000","last_fullsized":"2024-09-20T18:53:59.951173+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:56.818438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.0","version":"152'13956","reported_seq":14211,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:54:00.814479+0000","last_change":"2024-09-20T18:53:50.922274+0000","last_active":"2024-09-20T18:54:00.814479+0000","last_peered":"2024-09-20T18:54:00.814479+0000","last_clean":"2024-09-20T18:54:00.814479+0000","last_became_active":"2024-09-20T18:50:49.841019+0000","last_became_peered":"2024-09-20T18:50:49.841019+0000","last_unstale":"2024-09-20T18:54:00.814479+0000","last_undegraded":"2024-09-20T18:54:00.814479+0000","last_fullsized":"2024-09-20T18:54:00.814479+0000","mapping_epoch":83,"log_start":"124'8100","ondisk_log_start":"124'8100","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'13304","last_scrub_stamp":"2024-09-20T18:53:50.921985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:53:50.921985+0000","objects_scrubbed":2243,"log_size":5856,"log_dups_size":0,"ondisk_log_size":5856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:54.364286+0000","scrub_duration":608,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":103022592,"num_objects":1572,"num_object_clones":0,"num_object_copies":4716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13956,"num_write_kb":496896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":238,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.952463+0000","last_change":"2024-09-20T18:52:49.043282+0000","last_active":"2024-09-20T18:53:59.952463+0000","last_peered":"2024-09-20T18:53:59.952463+0000","last_clean":"2024-09-20T18:53:59.952463+0000","last_became_active":"2024-09-20T18:49:48.497736+0000","last_became_peered":"2024-09-20T18:49:48.497736+0000","last_unstale":"2024-09-20T18:53:59.952463+0000","last_undegraded":"2024-09-20T18:53:59.952463+0000","last_fullsized":"2024-09-20T18:53:59.952463+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-20T18:52:49.043189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:49.043189+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":199,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.959173+0000","last_change":"2024-09-20T18:49:45.942589+0000","last_active":"2024-09-20T18:53:59.959173+0000","last_peered":"2024-09-20T18:53:59.959173+0000","last_clean":"2024-09-20T18:53:59.959173+0000","last_became_active":"2024-09-20T18:49:45.942253+0000","last_became_peered":"2024-09-20T18:49:45.942253+0000","last_unstale":"2024-09-20T18:53:59.959173+0000","last_undegraded":"2024-09-20T18:53:59.959173+0000","last_fullsized":"2024-09-20T18:53:59.959173+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:59.256981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":206,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.961606+0000","last_change":"2024-09-20T18:53:12.347123+0000","last_active":"2024-09-20T18:53:59.961606+0000","last_peered":"2024-09-20T18:53:59.961606+0000","last_clean":"2024-09-20T18:53:59.961606+0000","last_became_active":"2024-09-20T18:49:49.954299+0000","last_became_peered":"2024-09-20T18:49:49.954299+0000","last_unstale":"2024-09-20T18:53:59.961606+0000","last_undegraded":"2024-09-20T18:53:59.961606+0000","last_fullsized":"2024-09-20T18:53:59.961606+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:12.347044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:12.347044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:41.912430+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":203,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.961023+0000","last_change":"2024-09-20T18:53:25.362907+0000","last_active":"2024-09-20T18:53:59.961023+0000","last_peered":"2024-09-20T18:53:59.961023+0000","last_clean":"2024-09-20T18:53:59.961023+0000","last_became_active":"2024-09-20T18:49:52.139981+0000","last_became_peered":"2024-09-20T18:49:52.139981+0000","last_unstale":"2024-09-20T18:53:59.961023+0000","last_undegraded":"2024-09-20T18:53:59.961023+0000","last_fullsized":"2024-09-20T18:53:59.961023+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:25.362828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:25.362828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:55.107515+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"152'13588","reported_seq":13806,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:54:00.795177+0000","last_change":"2024-09-20T18:52:03.401113+0000","last_active":"2024-09-20T18:54:00.795177+0000","last_peered":"2024-09-20T18:54:00.795177+0000","last_clean":"2024-09-20T18:54:00.795177+0000","last_became_active":"2024-09-20T18:50:49.842473+0000","last_became_peered":"2024-09-20T18:50:49.842473+0000","last_unstale":"2024-09-20T18:54:00.795177+0000","last_undegraded":"2024-09-20T18:54:00.795177+0000","last_fullsized":"2024-09-20T18:54:00.795177+0000","mapping_epoch":83,"log_start":"120'7300","ondisk_log_start":"120'7300","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"112'6142","last_scrub_stamp":"2024-09-20T18:52:03.400886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:52:03.400886+0000","objects_scrubbed":6079,"log_size":6288,"log_dups_size":0,"ondisk_log_size":6288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:41.532845+0000","scrub_duration":2059,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100925440,"num_objects":1540,"num_object_clones":0,"num_object_copies":4620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1540,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13588,"num_write_kb":484096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"66'28","reported_seq":302,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.959008+0000","last_change":"2024-09-20T18:51:57.293659+0000","last_active":"2024-09-20T18:53:59.959008+0000","last_peered":"2024-09-20T18:53:59.959008+0000","last_clean":"2024-09-20T18:53:59.959008+0000","last_became_active":"2024-09-20T18:49:47.955335+0000","last_became_peered":"2024-09-20T18:49:47.955335+0000","last_unstale":"2024-09-20T18:53:59.959008+0000","last_undegraded":"2024-09-20T18:53:59.959008+0000","last_fullsized":"2024-09-20T18:53:59.959008+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'28","last_scrub_stamp":"2024-09-20T18:51:57.293574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:57.293574+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:06.036152+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":199,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.944992+0000","last_change":"2024-09-20T18:49:45.945275+0000","last_active":"2024-09-20T18:53:59.944992+0000","last_peered":"2024-09-20T18:53:59.944992+0000","last_clean":"2024-09-20T18:53:59.944992+0000","last_became_active":"2024-09-20T18:49:45.944992+0000","last_became_peered":"2024-09-20T18:49:45.944992+0000","last_unstale":"2024-09-20T18:53:59.944992+0000","last_undegraded":"2024-09-20T18:53:59.944992+0000","last_fullsized":"2024-09-20T18:53:59.944992+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":232,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.963877+0000","last_change":"2024-09-20T18:53:49.323451+0000","last_active":"2024-09-20T18:53:59.963877+0000","last_peered":"2024-09-20T18:53:59.963877+0000","last_clean":"2024-09-20T18:53:59.963877+0000","last_became_active":"2024-09-20T18:49:44.936027+0000","last_became_peered":"2024-09-20T18:49:44.936027+0000","last_unstale":"2024-09-20T18:53:59.963877+0000","last_undegraded":"2024-09-20T18:53:59.963877+0000","last_fullsized":"2024-09-20T18:53:59.963877+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-20T18:53:49.323385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:53:49.323385+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:50.958195+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.0012781159999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":206,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.967629+0000","last_change":"2024-09-20T18:53:12.897519+0000","last_active":"2024-09-20T18:53:59.967629+0000","last_peered":"2024-09-20T18:53:59.967629+0000","last_clean":"2024-09-20T18:53:59.967629+0000","last_became_active":"2024-09-20T18:49:49.953971+0000","last_became_peered":"2024-09-20T18:49:49.953971+0000","last_unstale":"2024-09-20T18:53:59.967629+0000","last_undegraded":"2024-09-20T18:53:59.967629+0000","last_fullsized":"2024-09-20T18:53:59.967629+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:12.897051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:12.897051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:39.168470+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":186,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.965014+0000","last_change":"2024-09-20T18:49:51.969702+0000","last_active":"2024-09-20T18:53:59.965014+0000","last_peered":"2024-09-20T18:53:59.965014+0000","last_clean":"2024-09-20T18:53:59.965014+0000","last_became_active":"2024-09-20T18:49:51.969400+0000","last_became_peered":"2024-09-20T18:49:51.969400+0000","last_unstale":"2024-09-20T18:53:59.965014+0000","last_undegraded":"2024-09-20T18:53:59.965014+0000","last_fullsized":"2024-09-20T18:53:59.965014+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:36.285843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"152'13671","reported_seq":13868,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:54:01.173316+0000","last_change":"2024-09-20T18:53:23.208574+0000","last_active":"2024-09-20T18:54:01.173316+0000","last_peered":"2024-09-20T18:54:01.173316+0000","last_clean":"2024-09-20T18:54:01.173316+0000","last_became_active":"2024-09-20T18:50:49.843321+0000","last_became_peered":"2024-09-20T18:50:49.843321+0000","last_unstale":"2024-09-20T18:54:01.173316+0000","last_undegraded":"2024-09-20T18:54:01.173316+0000","last_fullsized":"2024-09-20T18:54:01.173316+0000","mapping_epoch":83,"log_start":"125'8100","ondisk_log_start":"125'8100","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"140'11334","last_scrub_stamp":"2024-09-20T18:53:23.208497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:53:23.208497+0000","objects_scrubbed":3959,"log_size":5571,"log_dups_size":0,"ondisk_log_size":5571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:48.313048+0000","scrub_duration":1357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":103481344,"num_objects":1579,"num_object_clones":0,"num_object_copies":4737,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1579,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13671,"num_write_kb":488000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":237,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.959083+0000","last_change":"2024-09-20T18:53:48.214566+0000","last_active":"2024-09-20T18:53:59.959083+0000","last_peered":"2024-09-20T18:53:59.959083+0000","last_clean":"2024-09-20T18:53:59.959083+0000","last_became_active":"2024-09-20T18:49:47.950904+0000","last_became_peered":"2024-09-20T18:49:47.950904+0000","last_unstale":"2024-09-20T18:53:59.959083+0000","last_undegraded":"2024-09-20T18:53:59.959083+0000","last_fullsized":"2024-09-20T18:53:59.959083+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-20T18:53:48.214491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:48.214491+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:15.858989+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":215,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.959668+0000","last_change":"2024-09-20T18:53:19.278005+0000","last_active":"2024-09-20T18:53:59.959668+0000","last_peered":"2024-09-20T18:53:59.959668+0000","last_clean":"2024-09-20T18:53:59.959668+0000","last_became_active":"2024-09-20T18:49:45.939884+0000","last_became_peered":"2024-09-20T18:49:45.939884+0000","last_unstale":"2024-09-20T18:53:59.959668+0000","last_undegraded":"2024-09-20T18:53:59.959668+0000","last_fullsized":"2024-09-20T18:53:59.959668+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:19.277650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:19.277650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:22.181893+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":217,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.952148+0000","last_change":"2024-09-20T18:52:07.718766+0000","last_active":"2024-09-20T18:53:59.952148+0000","last_peered":"2024-09-20T18:53:59.952148+0000","last_clean":"2024-09-20T18:53:59.952148+0000","last_became_active":"2024-09-20T18:49:44.937135+0000","last_became_peered":"2024-09-20T18:49:44.937135+0000","last_unstale":"2024-09-20T18:53:59.952148+0000","last_undegraded":"2024-09-20T18:53:59.952148+0000","last_fullsized":"2024-09-20T18:53:59.952148+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:52:07.718646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:52:07.718646+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:37.805116+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.00095558800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":199,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.966128+0000","last_change":"2024-09-20T18:51:59.796450+0000","last_active":"2024-09-20T18:53:59.966128+0000","last_peered":"2024-09-20T18:53:59.966128+0000","last_clean":"2024-09-20T18:53:59.966128+0000","last_became_active":"2024-09-20T18:49:49.954424+0000","last_became_peered":"2024-09-20T18:49:49.954424+0000","last_unstale":"2024-09-20T18:53:59.966128+0000","last_undegraded":"2024-09-20T18:53:59.966128+0000","last_fullsized":"2024-09-20T18:53:59.966128+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:59.796383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:59.796383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:03.718427+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":193,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.958512+0000","last_change":"2024-09-20T18:52:39.814324+0000","last_active":"2024-09-20T18:53:59.958512+0000","last_peered":"2024-09-20T18:53:59.958512+0000","last_clean":"2024-09-20T18:53:59.958512+0000","last_became_active":"2024-09-20T18:49:52.139789+0000","last_became_peered":"2024-09-20T18:49:52.139789+0000","last_unstale":"2024-09-20T18:53:59.958512+0000","last_undegraded":"2024-09-20T18:53:59.958512+0000","last_fullsized":"2024-09-20T18:53:59.958512+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:39.814259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:39.814259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"152'13942","reported_seq":14214,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:54:00.786812+0000","last_change":"2024-09-20T18:53:24.693140+0000","last_active":"2024-09-20T18:54:00.786812+0000","last_peered":"2024-09-20T18:54:00.786812+0000","last_clean":"2024-09-20T18:54:00.786812+0000","last_became_active":"2024-09-20T18:50:49.846124+0000","last_became_peered":"2024-09-20T18:50:49.846124+0000","last_unstale":"2024-09-20T18:54:00.786812+0000","last_undegraded":"2024-09-20T18:54:00.786812+0000","last_fullsized":"2024-09-20T18:54:00.786812+0000","mapping_epoch":83,"log_start":"126'8700","ondisk_log_start":"126'8700","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"140'11593","last_scrub_stamp":"2024-09-20T18:53:24.692917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:53:24.692917+0000","objects_scrubbed":4004,"log_size":5242,"log_dups_size":0,"ondisk_log_size":5242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:35.585228+0000","scrub_duration":1122,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":105512960,"num_objects":1610,"num_object_clones":0,"num_object_copies":4830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1610,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13942,"num_write_kb":497664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":235,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.957180+0000","last_change":"2024-09-20T18:53:47.786753+0000","last_active":"2024-09-20T18:53:59.957180+0000","last_peered":"2024-09-20T18:53:59.957180+0000","last_clean":"2024-09-20T18:53:59.957180+0000","last_became_active":"2024-09-20T18:49:47.949230+0000","last_became_peered":"2024-09-20T18:49:47.949230+0000","last_unstale":"2024-09-20T18:53:59.957180+0000","last_undegraded":"2024-09-20T18:53:59.957180+0000","last_fullsized":"2024-09-20T18:53:59.957180+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:53:47.786681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:47.786681+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:54.774464+0000","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":214,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.952264+0000","last_change":"2024-09-20T18:53:09.891235+0000","last_active":"2024-09-20T18:53:59.952264+0000","last_peered":"2024-09-20T18:53:59.952264+0000","last_clean":"2024-09-20T18:53:59.952264+0000","last_became_active":"2024-09-20T18:49:45.939197+0000","last_became_peered":"2024-09-20T18:49:45.939197+0000","last_unstale":"2024-09-20T18:53:59.952264+0000","last_undegraded":"2024-09-20T18:53:59.952264+0000","last_fullsized":"2024-09-20T18:53:59.952264+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:09.890987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:09.890987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:17.075156+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":475,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.958119+0000","last_change":"2024-09-20T18:53:22.616236+0000","last_active":"2024-09-20T18:53:59.958119+0000","last_peered":"2024-09-20T18:53:59.958119+0000","last_clean":"2024-09-20T18:53:59.958119+0000","last_became_active":"2024-09-20T18:49:44.936819+0000","last_became_peered":"2024-09-20T18:49:44.936819+0000","last_unstale":"2024-09-20T18:53:59.958119+0000","last_undegraded":"2024-09-20T18:53:59.958119+0000","last_fullsized":"2024-09-20T18:53:59.958119+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-20T18:53:22.616166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:53:22.616166+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:25.872696+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0010996199999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":250,"num_read_kb":245,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":197,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.963974+0000","last_change":"2024-09-20T18:52:02.981845+0000","last_active":"2024-09-20T18:53:59.963974+0000","last_peered":"2024-09-20T18:53:59.963974+0000","last_clean":"2024-09-20T18:53:59.963974+0000","last_became_active":"2024-09-20T18:49:49.961066+0000","last_became_peered":"2024-09-20T18:49:49.961066+0000","last_unstale":"2024-09-20T18:53:59.963974+0000","last_undegraded":"2024-09-20T18:53:59.963974+0000","last_fullsized":"2024-09-20T18:53:59.963974+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:02.981762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:02.981762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:58.477818+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":202,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.958751+0000","last_change":"2024-09-20T18:53:49.227504+0000","last_active":"2024-09-20T18:53:59.958751+0000","last_peered":"2024-09-20T18:53:59.958751+0000","last_clean":"2024-09-20T18:53:59.958751+0000","last_became_active":"2024-09-20T18:49:51.970343+0000","last_became_peered":"2024-09-20T18:49:51.970343+0000","last_unstale":"2024-09-20T18:53:59.958751+0000","last_undegraded":"2024-09-20T18:53:59.958751+0000","last_fullsized":"2024-09-20T18:53:59.958751+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:49.227436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:49.227436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:07.422442+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"152'13886","reported_seq":14112,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:54:01.167123+0000","last_change":"2024-09-20T18:52:13.298242+0000","last_active":"2024-09-20T18:54:01.167123+0000","last_peered":"2024-09-20T18:54:01.167123+0000","last_clean":"2024-09-20T18:54:01.167123+0000","last_became_active":"2024-09-20T18:50:49.842310+0000","last_became_peered":"2024-09-20T18:50:49.842310+0000","last_unstale":"2024-09-20T18:54:01.167123+0000","last_undegraded":"2024-09-20T18:54:01.167123+0000","last_fullsized":"2024-09-20T18:54:01.167123+0000","mapping_epoch":83,"log_start":"126'8300","ondisk_log_start":"126'8300","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"116'6867","last_scrub_stamp":"2024-09-20T18:52:13.297990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:52:13.297990+0000","objects_scrubbed":6775,"log_size":5586,"log_dups_size":0,"ondisk_log_size":5586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:40.850770+0000","scrub_duration":3018,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":98828288,"num_objects":1508,"num_object_clones":0,"num_object_copies":4524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13886,"num_write_kb":492608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":267,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.961979+0000","last_change":"2024-09-20T18:49:48.498345+0000","last_active":"2024-09-20T18:53:59.961979+0000","last_peered":"2024-09-20T18:53:59.961979+0000","last_clean":"2024-09-20T18:53:59.961979+0000","last_became_active":"2024-09-20T18:49:48.497936+0000","last_became_peered":"2024-09-20T18:49:48.497936+0000","last_unstale":"2024-09-20T18:53:59.961979+0000","last_undegraded":"2024-09-20T18:53:59.961979+0000","last_fullsized":"2024-09-20T18:53:59.961979+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:59.631793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":215,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.964146+0000","last_change":"2024-09-20T18:53:13.145127+0000","last_active":"2024-09-20T18:53:59.964146+0000","last_peered":"2024-09-20T18:53:59.964146+0000","last_clean":"2024-09-20T18:53:59.964146+0000","last_became_active":"2024-09-20T18:49:45.941136+0000","last_became_peered":"2024-09-20T18:49:45.941136+0000","last_unstale":"2024-09-20T18:53:59.964146+0000","last_undegraded":"2024-09-20T18:53:59.964146+0000","last_fullsized":"2024-09-20T18:53:59.964146+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:13.144896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:13.144896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:18.711819+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":253,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.958176+0000","last_change":"2024-09-20T18:52:11.887878+0000","last_active":"2024-09-20T18:53:59.958176+0000","last_peered":"2024-09-20T18:53:59.958176+0000","last_clean":"2024-09-20T18:53:59.958176+0000","last_became_active":"2024-09-20T18:49:36.545306+0000","last_became_peered":"2024-09-20T18:49:36.545306+0000","last_unstale":"2024-09-20T18:53:59.958176+0000","last_undegraded":"2024-09-20T18:53:59.958176+0000","last_fullsized":"2024-09-20T18:53:59.958176+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T18:52:11.887733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T18:52:11.887733+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:32.438028+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":198,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.958457+0000","last_change":"2024-09-20T18:51:22.852942+0000","last_active":"2024-09-20T18:53:59.958457+0000","last_peered":"2024-09-20T18:53:59.958457+0000","last_clean":"2024-09-20T18:53:59.958457+0000","last_became_active":"2024-09-20T18:49:49.956128+0000","last_became_peered":"2024-09-20T18:49:49.956128+0000","last_unstale":"2024-09-20T18:53:59.958457+0000","last_undegraded":"2024-09-20T18:53:59.958457+0000","last_fullsized":"2024-09-20T18:53:59.958457+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:22.852869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:22.852869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:37.516435+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":201,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.963869+0000","last_change":"2024-09-20T18:53:45.306565+0000","last_active":"2024-09-20T18:53:59.963869+0000","last_peered":"2024-09-20T18:53:59.963869+0000","last_clean":"2024-09-20T18:53:59.963869+0000","last_became_active":"2024-09-20T18:49:51.971332+0000","last_became_peered":"2024-09-20T18:49:51.971332+0000","last_unstale":"2024-09-20T18:53:59.963869+0000","last_undegraded":"2024-09-20T18:53:59.963869+0000","last_fullsized":"2024-09-20T18:53:59.963869+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:45.306514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:45.306514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:10.093423+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"152'13712","reported_seq":13997,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:54:00.821345+0000","last_change":"2024-09-20T18:52:51.095285+0000","last_active":"2024-09-20T18:54:00.821345+0000","last_peered":"2024-09-20T18:54:00.821345+0000","last_clean":"2024-09-20T18:54:00.821345+0000","last_became_active":"2024-09-20T18:50:49.842483+0000","last_became_peered":"2024-09-20T18:50:49.842483+0000","last_unstale":"2024-09-20T18:54:00.821345+0000","last_undegraded":"2024-09-20T18:54:00.821345+0000","last_fullsized":"2024-09-20T18:54:00.821345+0000","mapping_epoch":83,"log_start":"120'7400","ondisk_log_start":"120'7400","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"131'9234","last_scrub_stamp":"2024-09-20T18:52:51.095167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:52:51.095167+0000","objects_scrubbed":6083,"log_size":6312,"log_dups_size":0,"ondisk_log_size":6312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:09.470192+0000","scrub_duration":1756,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100859928,"num_objects":1540,"num_object_clones":0,"num_object_copies":4620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1540,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13712,"num_write_kb":488001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":221,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.945294+0000","last_change":"2024-09-20T18:51:14.397996+0000","last_active":"2024-09-20T18:53:59.945294+0000","last_peered":"2024-09-20T18:53:59.945294+0000","last_clean":"2024-09-20T18:53:59.945294+0000","last_became_active":"2024-09-20T18:49:47.950073+0000","last_became_peered":"2024-09-20T18:49:47.950073+0000","last_unstale":"2024-09-20T18:53:59.945294+0000","last_undegraded":"2024-09-20T18:53:59.945294+0000","last_fullsized":"2024-09-20T18:53:59.945294+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:51:14.397930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.397930+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:32.825060+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":206,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.946848+0000","last_change":"2024-09-20T18:51:18.410601+0000","last_active":"2024-09-20T18:53:59.946848+0000","last_peered":"2024-09-20T18:53:59.946848+0000","last_clean":"2024-09-20T18:53:59.946848+0000","last_became_active":"2024-09-20T18:49:45.944579+0000","last_became_peered":"2024-09-20T18:49:45.944579+0000","last_unstale":"2024-09-20T18:53:59.946848+0000","last_undegraded":"2024-09-20T18:53:59.946848+0000","last_fullsized":"2024-09-20T18:53:59.946848+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:18.410447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:18.410447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:29.855147+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":205,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.963818+0000","last_change":"2024-09-20T18:52:38.128487+0000","last_active":"2024-09-20T18:53:59.963818+0000","last_peered":"2024-09-20T18:53:59.963818+0000","last_clean":"2024-09-20T18:53:59.963818+0000","last_became_active":"2024-09-20T18:49:49.954391+0000","last_became_peered":"2024-09-20T18:49:49.954391+0000","last_unstale":"2024-09-20T18:53:59.963818+0000","last_undegraded":"2024-09-20T18:53:59.963818+0000","last_fullsized":"2024-09-20T18:53:59.963818+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:38.128438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:38.128438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":194,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.960635+0000","last_change":"2024-09-20T18:52:05.021228+0000","last_active":"2024-09-20T18:53:59.960635+0000","last_peered":"2024-09-20T18:53:59.960635+0000","last_clean":"2024-09-20T18:53:59.960635+0000","last_became_active":"2024-09-20T18:49:52.139959+0000","last_became_peered":"2024-09-20T18:49:52.139959+0000","last_unstale":"2024-09-20T18:53:59.960635+0000","last_undegraded":"2024-09-20T18:53:59.960635+0000","last_fullsized":"2024-09-20T18:53:59.960635+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:05.021071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.021071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"152'13722","reported_seq":13923,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:54:00.993757+0000","last_change":"2024-09-20T18:53:22.451111+0000","last_active":"2024-09-20T18:54:00.993757+0000","last_peered":"2024-09-20T18:54:00.993757+0000","last_clean":"2024-09-20T18:54:00.993757+0000","last_became_active":"2024-09-20T18:50:49.841683+0000","last_became_peered":"2024-09-20T18:50:49.841683+0000","last_unstale":"2024-09-20T18:54:00.993757+0000","last_undegraded":"2024-09-20T18:54:00.993757+0000","last_fullsized":"2024-09-20T18:54:00.993757+0000","mapping_epoch":83,"log_start":"117'6900","ondisk_log_start":"117'6900","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'11259","last_scrub_stamp":"2024-09-20T18:53:22.450961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:53:22.450961+0000","objects_scrubbed":4058,"log_size":6822,"log_dups_size":0,"ondisk_log_size":6822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:26.478768+0000","scrub_duration":1718,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":1536,"num_object_clones":0,"num_object_copies":4608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13722,"num_write_kb":488256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":238,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.945178+0000","last_change":"2024-09-20T18:52:02.944536+0000","last_active":"2024-09-20T18:53:59.945178+0000","last_peered":"2024-09-20T18:53:59.945178+0000","last_clean":"2024-09-20T18:53:59.945178+0000","last_became_active":"2024-09-20T18:49:47.947636+0000","last_became_peered":"2024-09-20T18:49:47.947636+0000","last_unstale":"2024-09-20T18:53:59.945178+0000","last_undegraded":"2024-09-20T18:53:59.945178+0000","last_fullsized":"2024-09-20T18:53:59.945178+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-20T18:52:02.944434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:02.944434+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:30.835162+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":227,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.961872+0000","last_change":"2024-09-20T18:53:23.472809+0000","last_active":"2024-09-20T18:53:59.961872+0000","last_peered":"2024-09-20T18:53:59.961872+0000","last_clean":"2024-09-20T18:53:59.961872+0000","last_became_active":"2024-09-20T18:51:55.596689+0000","last_became_peered":"2024-09-20T18:51:55.596689+0000","last_unstale":"2024-09-20T18:53:59.961872+0000","last_undegraded":"2024-09-20T18:53:59.961872+0000","last_fullsized":"2024-09-20T18:53:59.961872+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:23.472757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:23.472757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:39.082278+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":198,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.958842+0000","last_change":"2024-09-20T18:52:49.137080+0000","last_active":"2024-09-20T18:53:59.958842+0000","last_peered":"2024-09-20T18:53:59.958842+0000","last_clean":"2024-09-20T18:53:59.958842+0000","last_became_active":"2024-09-20T18:49:49.959160+0000","last_became_peered":"2024-09-20T18:49:49.959160+0000","last_unstale":"2024-09-20T18:53:59.958842+0000","last_undegraded":"2024-09-20T18:53:59.958842+0000","last_fullsized":"2024-09-20T18:53:59.958842+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:49.137023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:49.137023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:07.149566+0000","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":203,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.952525+0000","last_change":"2024-09-20T18:52:35.635897+0000","last_active":"2024-09-20T18:53:59.952525+0000","last_peered":"2024-09-20T18:53:59.952525+0000","last_clean":"2024-09-20T18:53:59.952525+0000","last_became_active":"2024-09-20T18:49:51.970108+0000","last_became_peered":"2024-09-20T18:49:51.970108+0000","last_unstale":"2024-09-20T18:53:59.952525+0000","last_undegraded":"2024-09-20T18:53:59.952525+0000","last_fullsized":"2024-09-20T18:53:59.952525+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:35.635818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:35.635818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:56.872031+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"152'13928","reported_seq":14187,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:54:01.189325+0000","last_change":"2024-09-20T18:52:52.583857+0000","last_active":"2024-09-20T18:54:01.189325+0000","last_peered":"2024-09-20T18:54:01.189325+0000","last_clean":"2024-09-20T18:54:01.189325+0000","last_became_active":"2024-09-20T18:50:49.841628+0000","last_became_peered":"2024-09-20T18:50:49.841628+0000","last_unstale":"2024-09-20T18:54:01.189325+0000","last_undegraded":"2024-09-20T18:54:01.189325+0000","last_fullsized":"2024-09-20T18:54:01.189325+0000","mapping_epoch":83,"log_start":"126'8300","ondisk_log_start":"126'8300","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"131'9412","last_scrub_stamp":"2024-09-20T18:52:52.583713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:52:52.583713+0000","objects_scrubbed":6144,"log_size":5628,"log_dups_size":0,"ondisk_log_size":5628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"queued for scrub","scrub_duration":2542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":101449728,"num_objects":1548,"num_object_clones":0,"num_object_copies":4644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1548,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13928,"num_write_kb":495232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":217,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.944842+0000","last_change":"2024-09-20T18:52:52.775966+0000","last_active":"2024-09-20T18:53:59.944842+0000","last_peered":"2024-09-20T18:53:59.944842+0000","last_clean":"2024-09-20T18:53:59.944842+0000","last_became_active":"2024-09-20T18:49:47.947126+0000","last_became_peered":"2024-09-20T18:49:47.947126+0000","last_unstale":"2024-09-20T18:53:59.944842+0000","last_undegraded":"2024-09-20T18:53:59.944842+0000","last_fullsized":"2024-09-20T18:53:59.944842+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:52:52.775889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:52.775889+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:14.126093+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":214,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.961219+0000","last_change":"2024-09-20T18:52:46.005172+0000","last_active":"2024-09-20T18:53:59.961219+0000","last_peered":"2024-09-20T18:53:59.961219+0000","last_clean":"2024-09-20T18:53:59.961219+0000","last_became_active":"2024-09-20T18:49:45.937264+0000","last_became_peered":"2024-09-20T18:49:45.937264+0000","last_unstale":"2024-09-20T18:53:59.961219+0000","last_undegraded":"2024-09-20T18:53:59.961219+0000","last_fullsized":"2024-09-20T18:53:59.961219+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:46.005109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:46.005109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:04.505427+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":266,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.956983+0000","last_change":"2024-09-20T18:52:40.647833+0000","last_active":"2024-09-20T18:53:59.956983+0000","last_peered":"2024-09-20T18:53:59.956983+0000","last_clean":"2024-09-20T18:53:59.956983+0000","last_became_active":"2024-09-20T18:49:49.957005+0000","last_became_peered":"2024-09-20T18:49:49.957005+0000","last_unstale":"2024-09-20T18:53:59.956983+0000","last_undegraded":"2024-09-20T18:53:59.956983+0000","last_fullsized":"2024-09-20T18:53:59.956983+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:52:40.647761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:40.647761+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:03.992292+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":194,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.945050+0000","last_change":"2024-09-20T18:52:10.024610+0000","last_active":"2024-09-20T18:53:59.945050+0000","last_peered":"2024-09-20T18:53:59.945050+0000","last_clean":"2024-09-20T18:53:59.945050+0000","last_became_active":"2024-09-20T18:49:51.969355+0000","last_became_peered":"2024-09-20T18:49:51.969355+0000","last_unstale":"2024-09-20T18:53:59.945050+0000","last_undegraded":"2024-09-20T18:53:59.945050+0000","last_fullsized":"2024-09-20T18:53:59.945050+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:10.024557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:10.024557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:38.850775+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.8","version":"152'13886","reported_seq":14114,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:54:00.986671+0000","last_change":"2024-09-20T18:52:16.931599+0000","last_active":"2024-09-20T18:54:00.986671+0000","last_peered":"2024-09-20T18:54:00.986671+0000","last_clean":"2024-09-20T18:54:00.986671+0000","last_became_active":"2024-09-20T18:50:49.844028+0000","last_became_peered":"2024-09-20T18:50:49.844028+0000","last_unstale":"2024-09-20T18:54:00.986671+0000","last_undegraded":"2024-09-20T18:54:00.986671+0000","last_fullsized":"2024-09-20T18:54:00.986671+0000","mapping_epoch":83,"log_start":"117'6980","ondisk_log_start":"117'6980","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"118'7065","last_scrub_stamp":"2024-09-20T18:52:16.931418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:52:16.931418+0000","objects_scrubbed":7008,"log_size":6906,"log_dups_size":0,"ondisk_log_size":6906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:31.738778+0000","scrub_duration":2274,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":101187584,"num_objects":1544,"num_object_clones":0,"num_object_copies":4632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13886,"num_write_kb":493760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":225,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.952349+0000","last_change":"2024-09-20T18:53:17.863709+0000","last_active":"2024-09-20T18:53:59.952349+0000","last_peered":"2024-09-20T18:53:59.952349+0000","last_clean":"2024-09-20T18:53:59.952349+0000","last_became_active":"2024-09-20T18:49:47.946008+0000","last_became_peered":"2024-09-20T18:49:47.946008+0000","last_unstale":"2024-09-20T18:53:59.952349+0000","last_undegraded":"2024-09-20T18:53:59.952349+0000","last_fullsized":"2024-09-20T18:53:59.952349+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:53:17.863623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:17.863623+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:35.672196+0000","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":213,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.963322+0000","last_change":"2024-09-20T18:52:17.120051+0000","last_active":"2024-09-20T18:53:59.963322+0000","last_peered":"2024-09-20T18:53:59.963322+0000","last_clean":"2024-09-20T18:53:59.963322+0000","last_became_active":"2024-09-20T18:49:45.941406+0000","last_became_peered":"2024-09-20T18:49:45.941406+0000","last_unstale":"2024-09-20T18:53:59.963322+0000","last_undegraded":"2024-09-20T18:53:59.963322+0000","last_fullsized":"2024-09-20T18:53:59.963322+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:17.119986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:17.119986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:55.385828+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":207,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.953052+0000","last_change":"2024-09-20T18:53:27.722068+0000","last_active":"2024-09-20T18:53:59.953052+0000","last_peered":"2024-09-20T18:53:59.953052+0000","last_clean":"2024-09-20T18:53:59.953052+0000","last_became_active":"2024-09-20T18:49:49.956767+0000","last_became_peered":"2024-09-20T18:49:49.956767+0000","last_unstale":"2024-09-20T18:53:59.953052+0000","last_undegraded":"2024-09-20T18:53:59.953052+0000","last_fullsized":"2024-09-20T18:53:59.953052+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:27.721940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:27.721940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:42.447352+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":193,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.952155+0000","last_change":"2024-09-20T18:52:46.004013+0000","last_active":"2024-09-20T18:53:59.952155+0000","last_peered":"2024-09-20T18:53:59.952155+0000","last_clean":"2024-09-20T18:53:59.952155+0000","last_became_active":"2024-09-20T18:49:51.964098+0000","last_became_peered":"2024-09-20T18:49:51.964098+0000","last_unstale":"2024-09-20T18:53:59.952155+0000","last_undegraded":"2024-09-20T18:53:59.952155+0000","last_fullsized":"2024-09-20T18:53:59.952155+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:46.003836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:46.003836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:04.821993+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"152'13678","reported_seq":13937,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:54:01.191957+0000","last_change":"2024-09-20T18:53:43.753759+0000","last_active":"2024-09-20T18:54:01.191957+0000","last_peered":"2024-09-20T18:54:01.191957+0000","last_clean":"2024-09-20T18:54:01.191957+0000","last_became_active":"2024-09-20T18:50:49.840023+0000","last_became_peered":"2024-09-20T18:50:49.840023+0000","last_unstale":"2024-09-20T18:54:01.191957+0000","last_undegraded":"2024-09-20T18:54:01.191957+0000","last_fullsized":"2024-09-20T18:54:01.191957+0000","mapping_epoch":83,"log_start":"125'8100","ondisk_log_start":"125'8100","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'12545","last_scrub_stamp":"2024-09-20T18:53:43.753508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:53:43.753508+0000","objects_scrubbed":2735,"log_size":5578,"log_dups_size":0,"ondisk_log_size":5578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:44.964709+0000","scrub_duration":892,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":103677952,"num_objects":1582,"num_object_clones":0,"num_object_copies":4746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13678,"num_write_kb":488320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":240,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.961645+0000","last_change":"2024-09-20T18:53:24.458093+0000","last_active":"2024-09-20T18:53:59.961645+0000","last_peered":"2024-09-20T18:53:59.961645+0000","last_clean":"2024-09-20T18:53:59.961645+0000","last_became_active":"2024-09-20T18:49:48.498625+0000","last_became_peered":"2024-09-20T18:49:48.498625+0000","last_unstale":"2024-09-20T18:53:59.961645+0000","last_undegraded":"2024-09-20T18:53:59.961645+0000","last_fullsized":"2024-09-20T18:53:59.961645+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-20T18:53:24.458005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:24.458005+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:31.316232+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":213,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.958782+0000","last_change":"2024-09-20T18:53:20.564464+0000","last_active":"2024-09-20T18:53:59.958782+0000","last_peered":"2024-09-20T18:53:59.958782+0000","last_clean":"2024-09-20T18:53:59.958782+0000","last_became_active":"2024-09-20T18:49:45.941448+0000","last_became_peered":"2024-09-20T18:49:45.941448+0000","last_unstale":"2024-09-20T18:53:59.958782+0000","last_undegraded":"2024-09-20T18:53:59.958782+0000","last_fullsized":"2024-09-20T18:53:59.958782+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:20.564413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:20.564413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:38.260460+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":207,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.955429+0000","last_change":"2024-09-20T18:52:36.662703+0000","last_active":"2024-09-20T18:53:59.955429+0000","last_peered":"2024-09-20T18:53:59.955429+0000","last_clean":"2024-09-20T18:53:59.955429+0000","last_became_active":"2024-09-20T18:49:49.957558+0000","last_became_peered":"2024-09-20T18:49:49.957558+0000","last_unstale":"2024-09-20T18:53:59.955429+0000","last_undegraded":"2024-09-20T18:53:59.955429+0000","last_fullsized":"2024-09-20T18:53:59.955429+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:36.662524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:36.662524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:58.801684+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":209,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.958088+0000","last_change":"2024-09-20T18:53:47.370152+0000","last_active":"2024-09-20T18:53:59.958088+0000","last_peered":"2024-09-20T18:53:59.958088+0000","last_clean":"2024-09-20T18:53:59.958088+0000","last_became_active":"2024-09-20T18:49:51.970798+0000","last_became_peered":"2024-09-20T18:49:51.970798+0000","last_unstale":"2024-09-20T18:53:59.958088+0000","last_undegraded":"2024-09-20T18:53:59.958088+0000","last_fullsized":"2024-09-20T18:53:59.958088+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:47.370108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:47.370108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:53.917763+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.a","version":"152'13764","reported_seq":13914,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:54:01.164802+0000","last_change":"2024-09-20T18:50:49.842445+0000","last_active":"2024-09-20T18:54:01.164802+0000","last_peered":"2024-09-20T18:54:01.164802+0000","last_clean":"2024-09-20T18:54:01.164802+0000","last_became_active":"2024-09-20T18:50:49.842179+0000","last_became_peered":"2024-09-20T18:50:49.842179+0000","last_unstale":"2024-09-20T18:54:01.164802+0000","last_undegraded":"2024-09-20T18:54:01.164802+0000","last_fullsized":"2024-09-20T18:54:01.164802+0000","mapping_epoch":83,"log_start":"129'9100","ondisk_log_start":"129'9100","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.819660+0000","objects_scrubbed":0,"log_size":4664,"log_dups_size":0,"ondisk_log_size":4664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:29.169615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":101187584,"num_objects":1544,"num_object_clones":0,"num_object_copies":4632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13764,"num_write_kb":489856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":233,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.946772+0000","last_change":"2024-09-20T18:53:52.898818+0000","last_active":"2024-09-20T18:53:59.946772+0000","last_peered":"2024-09-20T18:53:59.946772+0000","last_clean":"2024-09-20T18:53:59.946772+0000","last_became_active":"2024-09-20T18:49:47.949116+0000","last_became_peered":"2024-09-20T18:49:47.949116+0000","last_unstale":"2024-09-20T18:53:59.946772+0000","last_undegraded":"2024-09-20T18:53:59.946772+0000","last_fullsized":"2024-09-20T18:53:59.946772+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:53:52.898760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:52.898760+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:53.100006+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":213,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.958695+0000","last_change":"2024-09-20T18:53:21.578905+0000","last_active":"2024-09-20T18:53:59.958695+0000","last_peered":"2024-09-20T18:53:59.958695+0000","last_clean":"2024-09-20T18:53:59.958695+0000","last_became_active":"2024-09-20T18:49:45.944858+0000","last_became_peered":"2024-09-20T18:49:45.944858+0000","last_unstale":"2024-09-20T18:53:59.958695+0000","last_undegraded":"2024-09-20T18:53:59.958695+0000","last_fullsized":"2024-09-20T18:53:59.958695+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:21.578845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:21.578845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:47.899680+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":199,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.958820+0000","last_change":"2024-09-20T18:52:12.674502+0000","last_active":"2024-09-20T18:53:59.958820+0000","last_peered":"2024-09-20T18:53:59.958820+0000","last_clean":"2024-09-20T18:53:59.958820+0000","last_became_active":"2024-09-20T18:49:49.954944+0000","last_became_peered":"2024-09-20T18:49:49.954944+0000","last_unstale":"2024-09-20T18:53:59.958820+0000","last_undegraded":"2024-09-20T18:53:59.958820+0000","last_fullsized":"2024-09-20T18:53:59.958820+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:12.674203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:12.674203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:41.868905+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":194,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.960910+0000","last_change":"2024-09-20T18:52:09.017828+0000","last_active":"2024-09-20T18:53:59.960910+0000","last_peered":"2024-09-20T18:53:59.960910+0000","last_clean":"2024-09-20T18:53:59.960910+0000","last_became_active":"2024-09-20T18:49:51.972970+0000","last_became_peered":"2024-09-20T18:49:51.972970+0000","last_unstale":"2024-09-20T18:53:59.960910+0000","last_undegraded":"2024-09-20T18:53:59.960910+0000","last_fullsized":"2024-09-20T18:53:59.960910+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:09.017759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:09.017759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"152'13759","reported_seq":13987,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:54:00.898065+0000","last_change":"2024-09-20T18:52:15.505711+0000","last_active":"2024-09-20T18:54:00.898065+0000","last_peered":"2024-09-20T18:54:00.898065+0000","last_clean":"2024-09-20T18:54:00.898065+0000","last_became_active":"2024-09-20T18:50:49.843516+0000","last_became_peered":"2024-09-20T18:50:49.843516+0000","last_unstale":"2024-09-20T18:54:00.898065+0000","last_undegraded":"2024-09-20T18:54:00.898065+0000","last_fullsized":"2024-09-20T18:54:00.898065+0000","mapping_epoch":83,"log_start":"127'8700","ondisk_log_start":"127'8700","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'6958","last_scrub_stamp":"2024-09-20T18:52:15.505628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:52:15.505628+0000","objects_scrubbed":6900,"log_size":5059,"log_dups_size":0,"ondisk_log_size":5059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:29.307707+0000","scrub_duration":2196,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":98500608,"num_objects":1503,"num_object_clones":0,"num_object_copies":4509,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1503,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13759,"num_write_kb":488384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":226,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.962906+0000","last_change":"2024-09-20T18:52:35.182422+0000","last_active":"2024-09-20T18:53:59.962906+0000","last_peered":"2024-09-20T18:53:59.962906+0000","last_clean":"2024-09-20T18:53:59.962906+0000","last_became_active":"2024-09-20T18:49:48.499847+0000","last_became_peered":"2024-09-20T18:49:48.499847+0000","last_unstale":"2024-09-20T18:53:59.962906+0000","last_undegraded":"2024-09-20T18:53:59.962906+0000","last_fullsized":"2024-09-20T18:53:59.962906+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:52:35.182330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:35.182330+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":225,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.958615+0000","last_change":"2024-09-20T18:53:10.784695+0000","last_active":"2024-09-20T18:53:59.958615+0000","last_peered":"2024-09-20T18:53:59.958615+0000","last_clean":"2024-09-20T18:53:59.958615+0000","last_became_active":"2024-09-20T18:49:45.944256+0000","last_became_peered":"2024-09-20T18:49:45.944256+0000","last_unstale":"2024-09-20T18:53:59.958615+0000","last_undegraded":"2024-09-20T18:53:59.958615+0000","last_fullsized":"2024-09-20T18:53:59.958615+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-20T18:53:10.784625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:10.784625+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:16.037732+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":256,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.958076+0000","last_change":"2024-09-20T18:52:05.788988+0000","last_active":"2024-09-20T18:53:59.958076+0000","last_peered":"2024-09-20T18:53:59.958076+0000","last_clean":"2024-09-20T18:53:59.958076+0000","last_became_active":"2024-09-20T18:49:49.958411+0000","last_became_peered":"2024-09-20T18:49:49.958411+0000","last_unstale":"2024-09-20T18:53:59.958076+0000","last_undegraded":"2024-09-20T18:53:59.958076+0000","last_fullsized":"2024-09-20T18:53:59.958076+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:52:05.788924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.788924+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:39.468023+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":193,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.963657+0000","last_change":"2024-09-20T18:51:56.970415+0000","last_active":"2024-09-20T18:53:59.963657+0000","last_peered":"2024-09-20T18:53:59.963657+0000","last_clean":"2024-09-20T18:53:59.963657+0000","last_became_active":"2024-09-20T18:49:52.139685+0000","last_became_peered":"2024-09-20T18:49:52.139685+0000","last_unstale":"2024-09-20T18:53:59.963657+0000","last_undegraded":"2024-09-20T18:53:59.963657+0000","last_fullsized":"2024-09-20T18:53:59.963657+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:56.970349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:56.970349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:05.081675+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":213,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.961465+0000","last_change":"2024-09-20T18:53:09.388116+0000","last_active":"2024-09-20T18:53:59.961465+0000","last_peered":"2024-09-20T18:53:59.961465+0000","last_clean":"2024-09-20T18:53:59.961465+0000","last_became_active":"2024-09-20T18:49:45.940060+0000","last_became_peered":"2024-09-20T18:49:45.940060+0000","last_unstale":"2024-09-20T18:53:59.961465+0000","last_undegraded":"2024-09-20T18:53:59.961465+0000","last_fullsized":"2024-09-20T18:53:59.961465+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:09.388050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:09.388050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:17.378691+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":211,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.965447+0000","last_change":"2024-09-20T18:53:27.186982+0000","last_active":"2024-09-20T18:53:59.965447+0000","last_peered":"2024-09-20T18:53:59.965447+0000","last_clean":"2024-09-20T18:53:59.965447+0000","last_became_active":"2024-09-20T18:49:48.500662+0000","last_became_peered":"2024-09-20T18:49:48.500662+0000","last_unstale":"2024-09-20T18:53:59.965447+0000","last_undegraded":"2024-09-20T18:53:59.965447+0000","last_fullsized":"2024-09-20T18:53:59.965447+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-20T18:53:27.186631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:27.186631+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:42.463558+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":205,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.965163+0000","last_change":"2024-09-20T18:53:16.220554+0000","last_active":"2024-09-20T18:53:59.965163+0000","last_peered":"2024-09-20T18:53:59.965163+0000","last_clean":"2024-09-20T18:53:59.965163+0000","last_became_active":"2024-09-20T18:49:49.956749+0000","last_became_peered":"2024-09-20T18:49:49.956749+0000","last_unstale":"2024-09-20T18:53:59.965163+0000","last_undegraded":"2024-09-20T18:53:59.965163+0000","last_fullsized":"2024-09-20T18:53:59.965163+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:16.220490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:16.220490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:36.759843+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":201,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.958359+0000","last_change":"2024-09-20T18:53:19.591395+0000","last_active":"2024-09-20T18:53:59.958359+0000","last_peered":"2024-09-20T18:53:59.958359+0000","last_clean":"2024-09-20T18:53:59.958359+0000","last_became_active":"2024-09-20T18:49:52.142291+0000","last_became_peered":"2024-09-20T18:49:52.142291+0000","last_unstale":"2024-09-20T18:53:59.958359+0000","last_undegraded":"2024-09-20T18:53:59.958359+0000","last_fullsized":"2024-09-20T18:53:59.958359+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:19.591346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:19.591346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:40.474867+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":217,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.957262+0000","last_change":"2024-09-20T18:51:23.065382+0000","last_active":"2024-09-20T18:53:59.957262+0000","last_peered":"2024-09-20T18:53:59.957262+0000","last_clean":"2024-09-20T18:53:59.957262+0000","last_became_active":"2024-09-20T18:49:48.500803+0000","last_became_peered":"2024-09-20T18:49:48.500803+0000","last_unstale":"2024-09-20T18:53:59.957262+0000","last_undegraded":"2024-09-20T18:53:59.957262+0000","last_fullsized":"2024-09-20T18:53:59.957262+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:51:23.064536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:23.064536+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:39.863201+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":205,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.958219+0000","last_change":"2024-09-20T18:51:30.982535+0000","last_active":"2024-09-20T18:53:59.958219+0000","last_peered":"2024-09-20T18:53:59.958219+0000","last_clean":"2024-09-20T18:53:59.958219+0000","last_became_active":"2024-09-20T18:49:45.946600+0000","last_became_peered":"2024-09-20T18:49:45.946600+0000","last_unstale":"2024-09-20T18:53:59.958219+0000","last_undegraded":"2024-09-20T18:53:59.958219+0000","last_fullsized":"2024-09-20T18:53:59.958219+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:30.982454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:30.982454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":206,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.965314+0000","last_change":"2024-09-20T18:53:50.253721+0000","last_active":"2024-09-20T18:53:59.965314+0000","last_peered":"2024-09-20T18:53:59.965314+0000","last_clean":"2024-09-20T18:53:59.965314+0000","last_became_active":"2024-09-20T18:49:49.961947+0000","last_became_peered":"2024-09-20T18:49:49.961947+0000","last_unstale":"2024-09-20T18:53:59.965314+0000","last_undegraded":"2024-09-20T18:53:59.965314+0000","last_fullsized":"2024-09-20T18:53:59.965314+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:50.253660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:50.253660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:15.284040+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"66'1","reported_seq":204,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.952696+0000","last_change":"2024-09-20T18:53:14.689510+0000","last_active":"2024-09-20T18:53:59.952696+0000","last_peered":"2024-09-20T18:53:59.952696+0000","last_clean":"2024-09-20T18:53:59.952696+0000","last_became_active":"2024-09-20T18:49:51.965121+0000","last_became_peered":"2024-09-20T18:49:51.965121+0000","last_unstale":"2024-09-20T18:53:59.952696+0000","last_undegraded":"2024-09-20T18:53:59.952696+0000","last_fullsized":"2024-09-20T18:53:59.952696+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T18:53:14.689422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:14.689422+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:19.716830+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":209,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.963403+0000","last_change":"2024-09-20T18:49:48.501359+0000","last_active":"2024-09-20T18:53:59.963403+0000","last_peered":"2024-09-20T18:53:59.963403+0000","last_clean":"2024-09-20T18:53:59.963403+0000","last_became_active":"2024-09-20T18:49:48.501011+0000","last_became_peered":"2024-09-20T18:49:48.501011+0000","last_unstale":"2024-09-20T18:53:59.963403+0000","last_undegraded":"2024-09-20T18:53:59.963403+0000","last_fullsized":"2024-09-20T18:53:59.963403+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:02.117814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":205,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.951843+0000","last_change":"2024-09-20T18:51:20.067303+0000","last_active":"2024-09-20T18:53:59.951843+0000","last_peered":"2024-09-20T18:53:59.951843+0000","last_clean":"2024-09-20T18:53:59.951843+0000","last_became_active":"2024-09-20T18:49:45.938511+0000","last_became_peered":"2024-09-20T18:49:45.938511+0000","last_unstale":"2024-09-20T18:53:59.951843+0000","last_undegraded":"2024-09-20T18:53:59.951843+0000","last_fullsized":"2024-09-20T18:53:59.951843+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:20.067233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:20.067233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":266,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.960807+0000","last_change":"2024-09-20T18:53:47.337912+0000","last_active":"2024-09-20T18:53:59.960807+0000","last_peered":"2024-09-20T18:53:59.960807+0000","last_clean":"2024-09-20T18:53:59.960807+0000","last_became_active":"2024-09-20T18:49:49.960550+0000","last_became_peered":"2024-09-20T18:49:49.960550+0000","last_unstale":"2024-09-20T18:53:59.960807+0000","last_undegraded":"2024-09-20T18:53:59.960807+0000","last_fullsized":"2024-09-20T18:53:59.960807+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:53:47.337658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:47.337658+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:01.412695+0000","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":202,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.945603+0000","last_change":"2024-09-20T18:53:48.839130+0000","last_active":"2024-09-20T18:53:59.945603+0000","last_peered":"2024-09-20T18:53:59.945603+0000","last_clean":"2024-09-20T18:53:59.945603+0000","last_became_active":"2024-09-20T18:49:51.966254+0000","last_became_peered":"2024-09-20T18:49:51.966254+0000","last_unstale":"2024-09-20T18:53:59.945603+0000","last_undegraded":"2024-09-20T18:53:59.945603+0000","last_fullsized":"2024-09-20T18:53:59.945603+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:48.839015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:48.839015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:07.481453+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":236,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.961203+0000","last_change":"2024-09-20T18:51:15.223913+0000","last_active":"2024-09-20T18:53:59.961203+0000","last_peered":"2024-09-20T18:53:59.961203+0000","last_clean":"2024-09-20T18:53:59.961203+0000","last_became_active":"2024-09-20T18:49:58.997494+0000","last_became_peered":"2024-09-20T18:49:58.997494+0000","last_unstale":"2024-09-20T18:53:59.961203+0000","last_undegraded":"2024-09-20T18:53:59.961203+0000","last_fullsized":"2024-09-20T18:53:59.961203+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:51:15.223826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:15.223826+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:06.150447+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":213,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.958412+0000","last_change":"2024-09-20T18:53:27.589481+0000","last_active":"2024-09-20T18:53:59.958412+0000","last_peered":"2024-09-20T18:53:59.958412+0000","last_clean":"2024-09-20T18:53:59.958412+0000","last_became_active":"2024-09-20T18:49:45.949492+0000","last_became_peered":"2024-09-20T18:49:45.949492+0000","last_unstale":"2024-09-20T18:53:59.958412+0000","last_undegraded":"2024-09-20T18:53:59.958412+0000","last_fullsized":"2024-09-20T18:53:59.958412+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:27.589413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:27.589413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:48.562654+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":256,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.963212+0000","last_change":"2024-09-20T18:52:39.168501+0000","last_active":"2024-09-20T18:53:59.963212+0000","last_peered":"2024-09-20T18:53:59.963212+0000","last_clean":"2024-09-20T18:53:59.963212+0000","last_became_active":"2024-09-20T18:49:49.962243+0000","last_became_peered":"2024-09-20T18:49:49.962243+0000","last_unstale":"2024-09-20T18:53:59.963212+0000","last_undegraded":"2024-09-20T18:53:59.963212+0000","last_fullsized":"2024-09-20T18:53:59.963212+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:52:39.168442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:39.168442+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:02.311754+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":201,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.952482+0000","last_change":"2024-09-20T18:53:13.863129+0000","last_active":"2024-09-20T18:53:59.952482+0000","last_peered":"2024-09-20T18:53:59.952482+0000","last_clean":"2024-09-20T18:53:59.952482+0000","last_became_active":"2024-09-20T18:49:51.969093+0000","last_became_peered":"2024-09-20T18:49:51.969093+0000","last_unstale":"2024-09-20T18:53:59.952482+0000","last_undegraded":"2024-09-20T18:53:59.952482+0000","last_fullsized":"2024-09-20T18:53:59.952482+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:13.863061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:13.863061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:37.505050+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":229,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.944760+0000","last_change":"2024-09-20T18:53:43.879001+0000","last_active":"2024-09-20T18:53:59.944760+0000","last_peered":"2024-09-20T18:53:59.944760+0000","last_clean":"2024-09-20T18:53:59.944760+0000","last_became_active":"2024-09-20T18:49:47.949703+0000","last_became_peered":"2024-09-20T18:49:47.949703+0000","last_unstale":"2024-09-20T18:53:59.944760+0000","last_undegraded":"2024-09-20T18:53:59.944760+0000","last_fullsized":"2024-09-20T18:53:59.944760+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:53:43.878921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:43.878921+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:02.026666+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":206,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.944927+0000","last_change":"2024-09-20T18:52:05.985622+0000","last_active":"2024-09-20T18:53:59.944927+0000","last_peered":"2024-09-20T18:53:59.944927+0000","last_clean":"2024-09-20T18:53:59.944927+0000","last_became_active":"2024-09-20T18:49:45.939949+0000","last_became_peered":"2024-09-20T18:49:45.939949+0000","last_unstale":"2024-09-20T18:53:59.944927+0000","last_undegraded":"2024-09-20T18:53:59.944927+0000","last_fullsized":"2024-09-20T18:53:59.944927+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:05.985551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.985551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:36.823731+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":198,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.959596+0000","last_change":"2024-09-20T18:51:14.271954+0000","last_active":"2024-09-20T18:53:59.959596+0000","last_peered":"2024-09-20T18:53:59.959596+0000","last_clean":"2024-09-20T18:53:59.959596+0000","last_became_active":"2024-09-20T18:49:49.957067+0000","last_became_peered":"2024-09-20T18:49:49.957067+0000","last_unstale":"2024-09-20T18:53:59.959596+0000","last_undegraded":"2024-09-20T18:53:59.959596+0000","last_fullsized":"2024-09-20T18:53:59.959596+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:14.271719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.271719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:31.305762+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":194,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.962049+0000","last_change":"2024-09-20T18:51:14.210438+0000","last_active":"2024-09-20T18:53:59.962049+0000","last_peered":"2024-09-20T18:53:59.962049+0000","last_clean":"2024-09-20T18:53:59.962049+0000","last_became_active":"2024-09-20T18:49:51.966915+0000","last_became_peered":"2024-09-20T18:49:51.966915+0000","last_unstale":"2024-09-20T18:53:59.962049+0000","last_undegraded":"2024-09-20T18:53:59.962049+0000","last_fullsized":"2024-09-20T18:53:59.962049+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:14.210358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.210358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:04.079575+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":234,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.959688+0000","last_change":"2024-09-20T18:53:53.264294+0000","last_active":"2024-09-20T18:53:59.959688+0000","last_peered":"2024-09-20T18:53:59.959688+0000","last_clean":"2024-09-20T18:53:59.959688+0000","last_became_active":"2024-09-20T18:49:47.953310+0000","last_became_peered":"2024-09-20T18:49:47.953310+0000","last_unstale":"2024-09-20T18:53:59.959688+0000","last_undegraded":"2024-09-20T18:53:59.959688+0000","last_fullsized":"2024-09-20T18:53:59.959688+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:53:53.263976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:53.263976+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:04.916734+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":205,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.958735+0000","last_change":"2024-09-20T18:52:34.834333+0000","last_active":"2024-09-20T18:53:59.958735+0000","last_peered":"2024-09-20T18:53:59.958735+0000","last_clean":"2024-09-20T18:53:59.958735+0000","last_became_active":"2024-09-20T18:49:45.944357+0000","last_became_peered":"2024-09-20T18:49:45.944357+0000","last_unstale":"2024-09-20T18:53:59.958735+0000","last_undegraded":"2024-09-20T18:53:59.958735+0000","last_fullsized":"2024-09-20T18:53:59.958735+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:34.834273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:34.834273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:42.456584+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":198,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.960993+0000","last_change":"2024-09-20T18:51:20.208925+0000","last_active":"2024-09-20T18:53:59.960993+0000","last_peered":"2024-09-20T18:53:59.960993+0000","last_clean":"2024-09-20T18:53:59.960993+0000","last_became_active":"2024-09-20T18:49:49.962523+0000","last_became_peered":"2024-09-20T18:49:49.962523+0000","last_unstale":"2024-09-20T18:53:59.960993+0000","last_undegraded":"2024-09-20T18:53:59.960993+0000","last_fullsized":"2024-09-20T18:53:59.960993+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:20.208786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:20.208786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:08.165781+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":201,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.950904+0000","last_change":"2024-09-20T18:52:38.137485+0000","last_active":"2024-09-20T18:53:59.950904+0000","last_peered":"2024-09-20T18:53:59.950904+0000","last_clean":"2024-09-20T18:53:59.950904+0000","last_became_active":"2024-09-20T18:49:51.967313+0000","last_became_peered":"2024-09-20T18:49:51.967313+0000","last_unstale":"2024-09-20T18:53:59.950904+0000","last_undegraded":"2024-09-20T18:53:59.950904+0000","last_fullsized":"2024-09-20T18:53:59.950904+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:38.137431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:38.137431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:07.091791+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":221,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.962178+0000","last_change":"2024-09-20T18:52:13.020995+0000","last_active":"2024-09-20T18:53:59.962178+0000","last_peered":"2024-09-20T18:53:59.962178+0000","last_clean":"2024-09-20T18:53:59.962178+0000","last_became_active":"2024-09-20T18:49:48.499111+0000","last_became_peered":"2024-09-20T18:49:48.499111+0000","last_unstale":"2024-09-20T18:53:59.962178+0000","last_undegraded":"2024-09-20T18:53:59.962178+0000","last_fullsized":"2024-09-20T18:53:59.962178+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:52:13.020920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:13.020920+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":213,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.965225+0000","last_change":"2024-09-20T18:53:48.301845+0000","last_active":"2024-09-20T18:53:59.965225+0000","last_peered":"2024-09-20T18:53:59.965225+0000","last_clean":"2024-09-20T18:53:59.965225+0000","last_became_active":"2024-09-20T18:49:45.942105+0000","last_became_peered":"2024-09-20T18:49:45.942105+0000","last_unstale":"2024-09-20T18:53:59.965225+0000","last_undegraded":"2024-09-20T18:53:59.965225+0000","last_fullsized":"2024-09-20T18:53:59.965225+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:48.301801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:48.301801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:06.232022+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":213,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.957836+0000","last_change":"2024-09-20T18:53:46.329643+0000","last_active":"2024-09-20T18:53:59.957836+0000","last_peered":"2024-09-20T18:53:59.957836+0000","last_clean":"2024-09-20T18:53:59.957836+0000","last_became_active":"2024-09-20T18:49:49.958159+0000","last_became_peered":"2024-09-20T18:49:49.958159+0000","last_unstale":"2024-09-20T18:53:59.957836+0000","last_undegraded":"2024-09-20T18:53:59.957836+0000","last_fullsized":"2024-09-20T18:53:59.957836+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:46.329547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:46.329547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:09.181263+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":194,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.961130+0000","last_change":"2024-09-20T18:52:43.118886+0000","last_active":"2024-09-20T18:53:59.961130+0000","last_peered":"2024-09-20T18:53:59.961130+0000","last_clean":"2024-09-20T18:53:59.961130+0000","last_became_active":"2024-09-20T18:49:52.139978+0000","last_became_peered":"2024-09-20T18:49:52.139978+0000","last_unstale":"2024-09-20T18:53:59.961130+0000","last_undegraded":"2024-09-20T18:53:59.961130+0000","last_fullsized":"2024-09-20T18:53:59.961130+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:43.118805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:43.118805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:11.215766+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":216,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.960217+0000","last_change":"2024-09-20T18:53:43.253204+0000","last_active":"2024-09-20T18:53:59.960217+0000","last_peered":"2024-09-20T18:53:59.960217+0000","last_clean":"2024-09-20T18:53:59.960217+0000","last_became_active":"2024-09-20T18:49:47.954777+0000","last_became_peered":"2024-09-20T18:49:47.954777+0000","last_unstale":"2024-09-20T18:53:59.960217+0000","last_undegraded":"2024-09-20T18:53:59.960217+0000","last_fullsized":"2024-09-20T18:53:59.960217+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-20T18:53:43.253132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:43.253132+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:00.654449+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":213,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.957623+0000","last_change":"2024-09-20T18:53:16.921320+0000","last_active":"2024-09-20T18:53:59.957623+0000","last_peered":"2024-09-20T18:53:59.957623+0000","last_clean":"2024-09-20T18:53:59.957623+0000","last_became_active":"2024-09-20T18:49:45.940988+0000","last_became_peered":"2024-09-20T18:49:45.940988+0000","last_unstale":"2024-09-20T18:53:59.957623+0000","last_undegraded":"2024-09-20T18:53:59.957623+0000","last_fullsized":"2024-09-20T18:53:59.957623+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:16.921263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:16.921263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:35.775476+0000","scrub_duration":137,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":213,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.961443+0000","last_change":"2024-09-20T18:53:52.635213+0000","last_active":"2024-09-20T18:53:59.961443+0000","last_peered":"2024-09-20T18:53:59.961443+0000","last_clean":"2024-09-20T18:53:59.961443+0000","last_became_active":"2024-09-20T18:49:49.954502+0000","last_became_peered":"2024-09-20T18:49:49.954502+0000","last_unstale":"2024-09-20T18:53:59.961443+0000","last_undegraded":"2024-09-20T18:53:59.961443+0000","last_fullsized":"2024-09-20T18:53:59.961443+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:52.635138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:52.635138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:08.278673+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":193,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.958008+0000","last_change":"2024-09-20T18:51:09.761331+0000","last_active":"2024-09-20T18:53:59.958008+0000","last_peered":"2024-09-20T18:53:59.958008+0000","last_clean":"2024-09-20T18:53:59.958008+0000","last_became_active":"2024-09-20T18:49:51.968662+0000","last_became_peered":"2024-09-20T18:49:51.968662+0000","last_unstale":"2024-09-20T18:53:59.958008+0000","last_undegraded":"2024-09-20T18:53:59.958008+0000","last_fullsized":"2024-09-20T18:53:59.958008+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:09.761283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:09.761283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:30.451288+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":186,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.961376+0000","last_change":"2024-09-20T18:49:51.968476+0000","last_active":"2024-09-20T18:53:59.961376+0000","last_peered":"2024-09-20T18:53:59.961376+0000","last_clean":"2024-09-20T18:53:59.961376+0000","last_became_active":"2024-09-20T18:49:51.968216+0000","last_became_peered":"2024-09-20T18:49:51.968216+0000","last_unstale":"2024-09-20T18:53:59.961376+0000","last_undegraded":"2024-09-20T18:53:59.961376+0000","last_fullsized":"2024-09-20T18:53:59.961376+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":206,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.960478+0000","last_change":"2024-09-20T18:52:41.106558+0000","last_active":"2024-09-20T18:53:59.960478+0000","last_peered":"2024-09-20T18:53:59.960478+0000","last_clean":"2024-09-20T18:53:59.960478+0000","last_became_active":"2024-09-20T18:49:45.937992+0000","last_became_peered":"2024-09-20T18:49:45.937992+0000","last_unstale":"2024-09-20T18:53:59.960478+0000","last_undegraded":"2024-09-20T18:53:59.960478+0000","last_fullsized":"2024-09-20T18:53:59.960478+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:41.106332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:41.106332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":221,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.962228+0000","last_change":"2024-09-20T18:51:14.545122+0000","last_active":"2024-09-20T18:53:59.962228+0000","last_peered":"2024-09-20T18:53:59.962228+0000","last_clean":"2024-09-20T18:53:59.962228+0000","last_became_active":"2024-09-20T18:49:47.949582+0000","last_became_peered":"2024-09-20T18:49:47.949582+0000","last_unstale":"2024-09-20T18:53:59.962228+0000","last_undegraded":"2024-09-20T18:53:59.962228+0000","last_fullsized":"2024-09-20T18:53:59.962228+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:51:14.545043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.545043+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:40.491358+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":198,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-20T18:53:59.957003+0000","last_change":"2024-09-20T18:51:28.120100+0000","last_active":"2024-09-20T18:53:59.957003+0000","last_peered":"2024-09-20T18:53:59.957003+0000","last_clean":"2024-09-20T18:53:59.957003+0000","last_became_active":"2024-09-20T18:49:49.957582+0000","last_became_peered":"2024-09-20T18:49:49.957582+0000","last_unstale":"2024-09-20T18:53:59.957003+0000","last_undegraded":"2024-09-20T18:53:59.957003+0000","last_fullsized":"2024-09-20T18:53:59.957003+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:28.119694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:28.119694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:32.813511+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":1629683736,"num_objects":24868,"num_object_clones":0,"num_object_copies":74604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24868,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":220732,"num_write_kb":7859137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4890963968,"data_stored":4890951752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":89926,"ondisk_log_size":89926,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":250,"num_read_kb":245,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266743,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":629696,"kb_used_data":513836,"kb_used_omap":1,"kb_used_meta":115838,"kb_avail":93115456,"statfs":{"total":95995035648,"available":95350226944,"internally_reserved":0,"allocated":526168064,"data_stored":525788484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":118618570},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528382,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":606192,"kb_used_data":498652,"kb_used_omap":1,"kb_used_meta":107518,"kb_avail":93138960,"statfs":{"total":95995035648,"available":95374295040,"internally_reserved":0,"allocated":510619648,"data_stored":510255814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":110098886},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757315,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":474988,"kb_used_data":403544,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":93270164,"statfs":{"total":95995035648,"available":95508647936,"internally_reserved":0,"allocated":413229056,"data_stored":412869109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986248,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":635788,"kb_used_data":528248,"kb_used_omap":1,"kb_used_meta":107518,"kb_avail":93109364,"statfs":{"total":95995035648,"available":95343988736,"internally_reserved":0,"allocated":540925952,"data_stored":540534489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":110098885},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182479,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1092748,"kb_used_data":926580,"kb_used_omap":1,"kb_used_meta":165822,"kb_avail":92652404,"statfs":{"total":95995035648,"available":94876061696,"internally_reserved":0,"allocated":948817920,"data_stored":948429397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":169802183},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":707436,"kb_used_data":608220,"kb_used_omap":1,"kb_used_meta":99198,"kb_avail":93037716,"statfs":{"total":95995035648,"available":95270621184,"internally_reserved":0,"allocated":622817280,"data_stored":622453816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":101579205},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":717864,"kb_used_data":618512,"kb_used_omap":1,"kb_used_meta":99198,"kb_avail":93027288,"statfs":{"total":95995035648,"available":95259942912,"internally_reserved":0,"allocated":633356288,"data_stored":632940784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":101579205},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":951752,"kb_used_data":803952,"kb_used_omap":1,"kb_used_meta":147774,"kb_avail":92793400,"statfs":{"total":95995035648,"available":95020441600,"internally_reserved":0,"allocated":823246848,"data_stored":822862892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151321029},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":808062976,"data_stored":808058904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":612962304,"data_stored":612958232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":615841792,"data_stored":615841792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":908001280,"data_stored":908001280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":521732096,"data_stored":521732096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":406126592,"data_stored":406126592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":504434688,"data_stored":504430616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":513802240,"data_stored":513802240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T18:54:07.397 INFO:tasks.cephadm.ceph_manager.ceph:would drop below min_pgs, proj 6, currently 16 2024-09-20T18:54:07.409 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:54:07.418 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:07.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:07.419 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:07.520 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:07.563 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:07.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:07.564 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:07.665 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:07.700 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:07.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:07.701 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:07.802 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:07.833 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:07.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:07.835 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:07.887 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-20T18:54:07.926 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:07 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:54:07.927 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:07 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:54:07.935 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:07.986 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:07.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:07.987 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:08.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:07 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:54:08.088 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:08.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:08.127 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:08.128 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:08.228 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:54:08.255 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:08.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:08.256 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:54:08.356 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:08.397 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:08.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:08.398 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:08.446 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T18:54:08.499 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:54:08.522 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:08.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:08.523 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:54:08.607 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:54:08.624 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:08.705 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:08.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:08.707 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:08.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:08 smithi083 ceph-mon[29421]: pgmap v197: 148 pgs: 148 active+clean; 1.2 GiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.32k op/s 2024-09-20T18:54:08.759 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:08 smithi083 ceph-mon[36260]: pgmap v197: 148 pgs: 148 active+clean; 1.2 GiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.32k op/s 2024-09-20T18:54:08.808 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:08.849 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:08.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:08.850 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:08.925 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:08 smithi142 ceph-mon[32508]: pgmap v197: 148 pgs: 148 active+clean; 1.2 GiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.32k op/s 2024-09-20T18:54:08.950 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:08.996 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:08.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:08.997 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:09.097 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:54:09.126 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:09.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:09.127 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:54:09.228 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:54:09.255 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:09.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:09.256 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:54:09.347 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T18:54:09.347 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-20T18:54:09.348 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:09.348 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:229523 7.2 e153/83 7:4261aaf0:::benchmark_data_smithi083_70643_object106721:head v 153'14355, mlcod=153'14353)", 2024-09-20T18:54:09.348 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:54:09.323510+0000", 2024-09-20T18:54:09.348 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0037791779999999998, 2024-09-20T18:54:09.348 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003805938, 2024-09-20T18:54:09.348 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:54:09.348 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:54:09.348 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:54:09.348 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:54:09.348 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:09.348 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:54:09.348 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:09.323510+0000", 2024-09-20T18:54:09.348 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:09.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:09.349 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:09.349 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:54:09.349 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:09.323510+0000", 2024-09-20T18:54:09.349 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:09.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:09.349 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:09.349 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:54:09.349 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:09.323511+0000", 2024-09-20T18:54:09.349 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4499999999999997e-07 2024-09-20T18:54:09.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:09.349 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:09.349 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:54:09.349 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:09.323515+0000", 2024-09-20T18:54:09.350 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4039999999999997e-06 2024-09-20T18:54:09.350 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:09.350 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:09.350 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:54:09.350 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:09.323525+0000", 2024-09-20T18:54:09.350 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5640000000000002e-06 2024-09-20T18:54:09.350 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:09.350 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:09.350 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:54:09.350 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:09.323531+0000", 2024-09-20T18:54:09.350 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4169999999999997e-06 2024-09-20T18:54:09.350 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:09.350 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:09.351 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:54:09.351 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:09.324280+0000", 2024-09-20T18:54:09.351 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00074900599999999998 2024-09-20T18:54:09.351 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:09.351 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:09.351 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:54:09.351 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:09.324374+0000", 2024-09-20T18:54:09.351 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3943000000000005e-05 2024-09-20T18:54:09.351 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:09.351 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:54:09.351 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:09.351 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:09.351 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:09.351 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:229522 7.0 e153/83 7:0cf38503:::benchmark_data_smithi083_70643_object106720:head v 153'14511, mlcod=153'14508)", 2024-09-20T18:54:09.352 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:54:09.323561+0000", 2024-09-20T18:54:09.352 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0037282890000000001, 2024-09-20T18:54:09.352 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0038077839999999998, 2024-09-20T18:54:09.352 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:54:09.352 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:54:09.352 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:54:09.352 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:54:09.352 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:09.352 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:54:09.352 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:09.323561+0000", 2024-09-20T18:54:09.352 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:09.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:09.352 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:09.352 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:54:09.353 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:09.323561+0000", 2024-09-20T18:54:09.353 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:09.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:09.353 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:09.353 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:54:09.353 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:09.323561+0000", 2024-09-20T18:54:09.353 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3000000000000001e-07 2024-09-20T18:54:09.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:09.353 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:09.353 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:54:09.353 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:09.323565+0000", 2024-09-20T18:54:09.353 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3780000000000001e-06 2024-09-20T18:54:09.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:09.353 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:09.354 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:54:09.354 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:09.323570+0000", 2024-09-20T18:54:09.354 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6629999999999998e-06 2024-09-20T18:54:09.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:09.354 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:09.354 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:54:09.354 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:09.323575+0000", 2024-09-20T18:54:09.354 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3139999999999997e-06 2024-09-20T18:54:09.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:09.354 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:09.354 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:54:09.354 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:09.323703+0000", 2024-09-20T18:54:09.354 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012788599999999999 2024-09-20T18:54:09.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:09.355 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:09.355 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:54:09.355 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:09.323720+0000", 2024-09-20T18:54:09.355 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7187e-05 2024-09-20T18:54:09.355 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:09.355 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:54:09.355 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:09.355 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:09.355 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:09.355 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:229524 7.0 e153/83)", 2024-09-20T18:54:09.355 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:54:09.323589+0000", 2024-09-20T18:54:09.355 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0037004799999999999, 2024-09-20T18:54:09.355 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0038249500000000001, 2024-09-20T18:54:09.355 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:54:09.356 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:54:09.356 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "queued for pg", 2024-09-20T18:54:09.356 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:54:09.356 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:09.356 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:54:09.356 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:09.323589+0000", 2024-09-20T18:54:09.356 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:09.356 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:09.356 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:09.356 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:54:09.356 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:09.323589+0000", 2024-09-20T18:54:09.358 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:09.358 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:09.358 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:09.358 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:54:09.358 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:09.323589+0000", 2024-09-20T18:54:09.358 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.92e-07 2024-09-20T18:54:09.358 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:09.358 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:09.358 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:54:09.358 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:09.323592+0000", 2024-09-20T18:54:09.358 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7089999999999998e-06 2024-09-20T18:54:09.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:09.359 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:09.359 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:54:09.359 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:09.323596+0000", 2024-09-20T18:54:09.359 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2880000000000003e-06 2024-09-20T18:54:09.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:09.359 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:09.359 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:54:09.359 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:09.323599+0000", 2024-09-20T18:54:09.359 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4819999999999999e-06 2024-09-20T18:54:09.359 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:09.359 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:54:09.359 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:09.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:09.359 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:09.360 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:229525 7.d e153/83 7:b99df1e7:::benchmark_data_smithi083_70643_object106723:head v 153'14265, mlcod=153'14260)", 2024-09-20T18:54:09.360 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:54:09.323613+0000", 2024-09-20T18:54:09.360 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0036761509999999999, 2024-09-20T18:54:09.360 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0038440639999999999, 2024-09-20T18:54:09.360 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:54:09.360 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:54:09.360 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:54:09.360 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:54:09.360 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:09.360 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:54:09.360 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:09.323613+0000", 2024-09-20T18:54:09.360 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:09.360 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:09.360 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:09.361 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:54:09.361 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:09.323613+0000", 2024-09-20T18:54:09.361 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:09.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:09.361 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:09.361 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:54:09.361 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:09.323613+0000", 2024-09-20T18:54:09.361 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6300000000000001e-07 2024-09-20T18:54:09.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:09.361 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:09.361 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:54:09.361 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:09.323616+0000", 2024-09-20T18:54:09.361 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0989999999999999e-06 2024-09-20T18:54:09.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:09.361 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:09.362 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:54:09.362 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:09.323621+0000", 2024-09-20T18:54:09.362 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6709999999999998e-06 2024-09-20T18:54:09.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:09.362 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:09.362 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:54:09.362 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:09.323624+0000", 2024-09-20T18:54:09.362 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4410000000000002e-06 2024-09-20T18:54:09.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:09.362 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:09.362 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:54:09.362 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:09.323871+0000", 2024-09-20T18:54:09.362 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000247011 2024-09-20T18:54:09.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:09.363 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:09.363 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:54:09.363 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:09.323891+0000", 2024-09-20T18:54:09.363 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9661e-05 2024-09-20T18:54:09.363 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:09.363 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:54:09.363 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:09.363 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:09.363 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T18:54:09.363 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 4 2024-09-20T18:54:09.363 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T18:54:09.364 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:09.364 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:54:09.392 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:09.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:09.393 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:09.494 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:09.523 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:09.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:09.524 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:09.625 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:09.656 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:09.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:09.657 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:09.706 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-20T18:54:09.758 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:09.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:09.792 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:09.792 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:09.892 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:09.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:09.922 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:09.922 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:10.004 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:09 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:54:10.004 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:09 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:54:10.005 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:09 smithi083 ceph-mon[29421]: osdmap e153: 8 total, 8 up, 8 in 2024-09-20T18:54:10.005 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:09 smithi083 ceph-mon[29421]: pgmap v199: 148 pgs: 148 active+clean; 1.2 GiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.19k op/s 2024-09-20T18:54:10.005 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:09 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2088094555' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:54:10.005 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:09 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:54:10.005 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:09 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:54:10.005 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:09 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:54:10.005 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:09 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:54:10.005 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:09 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:54:10.005 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:09 smithi083 ceph-mon[36260]: osdmap e153: 8 total, 8 up, 8 in 2024-09-20T18:54:10.005 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:09 smithi083 ceph-mon[36260]: pgmap v199: 148 pgs: 148 active+clean; 1.2 GiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.19k op/s 2024-09-20T18:54:10.005 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:09 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2088094555' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:54:10.006 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:09 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:54:10.006 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:09 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:54:10.006 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:09 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:54:10.022 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:10.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:09 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:54:10.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:09 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:54:10.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:09 smithi142 ceph-mon[32508]: osdmap e153: 8 total, 8 up, 8 in 2024-09-20T18:54:10.025 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:09 smithi142 ceph-mon[32508]: pgmap v199: 148 pgs: 148 active+clean; 1.2 GiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.19k op/s 2024-09-20T18:54:10.026 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:09 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2088094555' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:54:10.026 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:09 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:54:10.026 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:09 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:54:10.026 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:09 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:54:10.049 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:10.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:10.050 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:10.059 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_blocked_ops 2024-09-20T18:54:10.151 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:10.201 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:10.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:10.202 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:10.303 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:10.328 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:10.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:10.329 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:10.430 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:10.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:10.480 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:10.481 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:10.581 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:10.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:10.636 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:10.636 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:10.649 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:54:10.649 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:54:10.689 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:54:10.689 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:54:10.724 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:54:10.724 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:54:10.737 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:54:10.761 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:10.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:10.762 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:54:10.764 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:54:10.764 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:54:10.798 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:54:10.798 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:54:10.830 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:54:10.830 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:54:10.831 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:10 smithi083 ceph-mon[29421]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:54:10.831 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:10 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T18:54:10.831 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:10 smithi083 ceph-mon[29421]: osdmap e154: 8 total, 8 up, 8 in 2024-09-20T18:54:10.831 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:10 smithi083 ceph-mon[29421]: 7.d scrub starts 2024-09-20T18:54:10.831 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:10 smithi083 ceph-mon[29421]: 7.d scrub ok 2024-09-20T18:54:10.831 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:10 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3935580375' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:54:10.831 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:54:10 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:54:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:54:10.832 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:10 smithi083 ceph-mon[36260]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:54:10.832 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:10 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T18:54:10.832 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:10 smithi083 ceph-mon[36260]: osdmap e154: 8 total, 8 up, 8 in 2024-09-20T18:54:10.832 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:10 smithi083 ceph-mon[36260]: 7.d scrub starts 2024-09-20T18:54:10.832 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:10 smithi083 ceph-mon[36260]: 7.d scrub ok 2024-09-20T18:54:10.832 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:10 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3935580375' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:54:10.863 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:54:10.863 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:54:10.863 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:10.887 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:10.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:10.888 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:10.897 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:54:10.898 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:54:10.989 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:11.014 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:11.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:11.015 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:11.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:10 smithi142 ceph-mon[32508]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:54:11.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:10 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T18:54:11.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:10 smithi142 ceph-mon[32508]: osdmap e154: 8 total, 8 up, 8 in 2024-09-20T18:54:11.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:10 smithi142 ceph-mon[32508]: 7.d scrub starts 2024-09-20T18:54:11.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:10 smithi142 ceph-mon[32508]: 7.d scrub ok 2024-09-20T18:54:11.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:10 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3935580375' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:54:11.118 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:11.156 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:11.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:11.157 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:11.258 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:11.302 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:11.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:11.303 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:11.405 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:54:11.433 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:11.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:11.435 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:54:11.535 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:54:11.561 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:11.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:11.562 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:54:11.580 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:54:11.663 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:11.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:11.702 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:11.702 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:11.803 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:11.838 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:11.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:11.839 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:11.940 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:54:11.967 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:11.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:11.968 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:54:11.996 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:11 smithi083 ceph-mon[29421]: 6.d scrub starts 2024-09-20T18:54:11.997 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:11 smithi083 ceph-mon[29421]: 6.d scrub ok 2024-09-20T18:54:11.997 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:11 smithi083 ceph-mon[29421]: pgmap v201: 148 pgs: 148 active+clean; 1.1 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 907 op/s 2024-09-20T18:54:11.997 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:11 smithi083 ceph-mon[29421]: 5.18 scrub starts 2024-09-20T18:54:11.997 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:11 smithi083 ceph-mon[29421]: 5.18 scrub ok 2024-09-20T18:54:11.997 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:11 smithi083 ceph-mon[36260]: 6.d scrub starts 2024-09-20T18:54:11.997 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:11 smithi083 ceph-mon[36260]: 6.d scrub ok 2024-09-20T18:54:11.997 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:11 smithi083 ceph-mon[36260]: pgmap v201: 148 pgs: 148 active+clean; 1.1 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 907 op/s 2024-09-20T18:54:11.998 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:11 smithi083 ceph-mon[36260]: 5.18 scrub starts 2024-09-20T18:54:11.998 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:11 smithi083 ceph-mon[36260]: 5.18 scrub ok 2024-09-20T18:54:12.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:11 smithi142 ceph-mon[32508]: 6.d scrub starts 2024-09-20T18:54:12.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:11 smithi142 ceph-mon[32508]: 6.d scrub ok 2024-09-20T18:54:12.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:11 smithi142 ceph-mon[32508]: pgmap v201: 148 pgs: 148 active+clean; 1.1 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 907 op/s 2024-09-20T18:54:12.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:11 smithi142 ceph-mon[32508]: 5.18 scrub starts 2024-09-20T18:54:12.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:11 smithi142 ceph-mon[32508]: 5.18 scrub ok 2024-09-20T18:54:12.068 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:12.111 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:12.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:12.112 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:12.213 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:12.251 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:12.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:12.252 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:12.353 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:54:12.378 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:12.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:12.379 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:54:12.398 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-20T18:54:12.398 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T18:54:12.399 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-20T18:54:12.399 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-20T18:54:12.399 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-20T18:54:12.399 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.1 config set filestore_inject_stall 3 2024-09-20T18:54:12.479 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:12.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:12.541 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:12.541 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:12.643 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:54:12.651 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T18:54:12.651 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:54:12.668 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:12.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:12.669 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:54:12.769 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:12.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:12.810 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:12.811 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:12.912 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:54:12.938 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:12.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:12.939 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:54:13.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:12 smithi142 ceph-mon[32508]: 5.c scrub starts 2024-09-20T18:54:13.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:12 smithi142 ceph-mon[32508]: 5.c scrub ok 2024-09-20T18:54:13.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:12 smithi142 ceph-mon[32508]: 5.f scrub starts 2024-09-20T18:54:13.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:12 smithi142 ceph-mon[32508]: 5.f scrub ok 2024-09-20T18:54:13.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:12 smithi142 ceph-mon[32508]: 3.12 scrub starts 2024-09-20T18:54:13.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:12 smithi142 ceph-mon[32508]: 3.12 scrub ok 2024-09-20T18:54:13.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:12 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:54:13.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:12 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:54:13.041 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:54:13.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:12 smithi083 ceph-mon[29421]: 5.c scrub starts 2024-09-20T18:54:13.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:12 smithi083 ceph-mon[29421]: 5.c scrub ok 2024-09-20T18:54:13.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:12 smithi083 ceph-mon[29421]: 5.f scrub starts 2024-09-20T18:54:13.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:12 smithi083 ceph-mon[29421]: 5.f scrub ok 2024-09-20T18:54:13.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:12 smithi083 ceph-mon[29421]: 3.12 scrub starts 2024-09-20T18:54:13.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:12 smithi083 ceph-mon[29421]: 3.12 scrub ok 2024-09-20T18:54:13.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:12 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:54:13.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:12 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:54:13.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:12 smithi083 ceph-mon[36260]: 5.c scrub starts 2024-09-20T18:54:13.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:12 smithi083 ceph-mon[36260]: 5.c scrub ok 2024-09-20T18:54:13.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:12 smithi083 ceph-mon[36260]: 5.f scrub starts 2024-09-20T18:54:13.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:12 smithi083 ceph-mon[36260]: 5.f scrub ok 2024-09-20T18:54:13.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:12 smithi083 ceph-mon[36260]: 3.12 scrub starts 2024-09-20T18:54:13.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:12 smithi083 ceph-mon[36260]: 3.12 scrub ok 2024-09-20T18:54:13.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:12 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:54:13.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:12 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:54:13.067 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:13.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:13.068 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:54:13.169 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:54:13.195 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:13.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:13.196 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:54:13.297 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:13.336 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:13.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:13.337 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:13.346 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T18:54:13.438 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:13.464 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:13.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:13.465 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:13.566 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:13.706 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:13.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:13.707 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:13.808 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:13.848 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:13.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:13.850 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:13.950 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:13.958 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:13 smithi083 ceph-mon[29421]: 4.8 scrub starts 2024-09-20T18:54:13.958 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:13 smithi083 ceph-mon[29421]: 4.8 scrub ok 2024-09-20T18:54:13.958 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:13 smithi083 ceph-mon[29421]: osdmap e155: 8 total, 8 up, 8 in 2024-09-20T18:54:13.958 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:13 smithi083 ceph-mon[29421]: pgmap v203: 148 pgs: 148 active+clean; 893 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 884 op/s 2024-09-20T18:54:13.959 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:13 smithi083 ceph-mon[36260]: 4.8 scrub starts 2024-09-20T18:54:13.959 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:13 smithi083 ceph-mon[36260]: 4.8 scrub ok 2024-09-20T18:54:13.959 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:13 smithi083 ceph-mon[36260]: osdmap e155: 8 total, 8 up, 8 in 2024-09-20T18:54:13.959 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:13 smithi083 ceph-mon[36260]: pgmap v203: 148 pgs: 148 active+clean; 893 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 884 op/s 2024-09-20T18:54:13.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:13.996 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:13.997 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:14.015 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_historic_ops 2024-09-20T18:54:14.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:13 smithi142 ceph-mon[32508]: 4.8 scrub starts 2024-09-20T18:54:14.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:13 smithi142 ceph-mon[32508]: 4.8 scrub ok 2024-09-20T18:54:14.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:13 smithi142 ceph-mon[32508]: osdmap e155: 8 total, 8 up, 8 in 2024-09-20T18:54:14.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:13 smithi142 ceph-mon[32508]: pgmap v203: 148 pgs: 148 active+clean; 893 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 884 op/s 2024-09-20T18:54:14.047 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T18:54:14.097 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:14.150 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:54:14.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:14.167 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:14.167 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:14.190 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:54:14.268 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:14.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:14.333 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:14.333 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:14.433 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:14.476 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:14.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:14.477 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:14.578 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:14.623 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:14.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:14.624 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:14.725 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:14.756 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:14.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:14.757 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:14.858 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:14.901 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:14.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:14.902 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:15.002 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:15.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:14 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:54:15.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:14 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:54:15.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:14 smithi142 ceph-mon[32508]: 3.1f scrub starts 2024-09-20T18:54:15.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:14 smithi142 ceph-mon[32508]: 3.1f scrub ok 2024-09-20T18:54:15.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:14 smithi142 ceph-mon[32508]: 5.e scrub starts 2024-09-20T18:54:15.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:14 smithi142 ceph-mon[32508]: 5.e scrub ok 2024-09-20T18:54:15.040 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:15.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:15.041 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:15.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:14 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:54:15.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:14 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:54:15.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:14 smithi083 ceph-mon[29421]: 3.1f scrub starts 2024-09-20T18:54:15.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:14 smithi083 ceph-mon[29421]: 3.1f scrub ok 2024-09-20T18:54:15.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:14 smithi083 ceph-mon[29421]: 5.e scrub starts 2024-09-20T18:54:15.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:14 smithi083 ceph-mon[29421]: 5.e scrub ok 2024-09-20T18:54:15.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:14 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:54:15.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:14 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:54:15.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:14 smithi083 ceph-mon[36260]: 3.1f scrub starts 2024-09-20T18:54:15.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:14 smithi083 ceph-mon[36260]: 3.1f scrub ok 2024-09-20T18:54:15.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:14 smithi083 ceph-mon[36260]: 5.e scrub starts 2024-09-20T18:54:15.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:14 smithi083 ceph-mon[36260]: 5.e scrub ok 2024-09-20T18:54:15.142 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:15.180 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:15.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:15.182 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:15.282 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:15.321 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:15.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:15.322 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:15.423 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:15.463 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:15.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:15.464 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:15.565 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:54:15.594 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:15.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:15.595 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:54:15.696 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:15.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:15.747 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:15.747 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:15.848 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:15.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:15.896 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:15.897 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:15.902 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:15 smithi083 ceph-mon[29421]: osdmap e156: 8 total, 8 up, 8 in 2024-09-20T18:54:15.902 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:15 smithi083 ceph-mon[29421]: pgmap v205: 148 pgs: 148 active+clean; 893 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 884 op/s 2024-09-20T18:54:15.902 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:15 smithi083 ceph-mon[29421]: 4.3 scrub starts 2024-09-20T18:54:15.902 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:15 smithi083 ceph-mon[29421]: 4.3 scrub ok 2024-09-20T18:54:15.902 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:15 smithi083 ceph-mon[29421]: 6.1a scrub starts 2024-09-20T18:54:15.902 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:15 smithi083 ceph-mon[29421]: 6.1a scrub ok 2024-09-20T18:54:15.902 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:15 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:54:15.902 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:15 smithi083 ceph-mon[36260]: osdmap e156: 8 total, 8 up, 8 in 2024-09-20T18:54:15.902 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:15 smithi083 ceph-mon[36260]: pgmap v205: 148 pgs: 148 active+clean; 893 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 884 op/s 2024-09-20T18:54:15.902 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:15 smithi083 ceph-mon[36260]: 4.3 scrub starts 2024-09-20T18:54:15.903 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:15 smithi083 ceph-mon[36260]: 4.3 scrub ok 2024-09-20T18:54:15.903 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:15 smithi083 ceph-mon[36260]: 6.1a scrub starts 2024-09-20T18:54:15.903 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:15 smithi083 ceph-mon[36260]: 6.1a scrub ok 2024-09-20T18:54:15.903 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:15 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:54:15.969 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T18:54:15.969 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-20T18:54:15.969 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T18:54:15.998 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:16.023 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:16.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:16.024 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:16.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:15 smithi142 ceph-mon[32508]: osdmap e156: 8 total, 8 up, 8 in 2024-09-20T18:54:16.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:15 smithi142 ceph-mon[32508]: pgmap v205: 148 pgs: 148 active+clean; 893 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 884 op/s 2024-09-20T18:54:16.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:15 smithi142 ceph-mon[32508]: 4.3 scrub starts 2024-09-20T18:54:16.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:15 smithi142 ceph-mon[32508]: 4.3 scrub ok 2024-09-20T18:54:16.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:15 smithi142 ceph-mon[32508]: 6.1a scrub starts 2024-09-20T18:54:16.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:15 smithi142 ceph-mon[32508]: 6.1a scrub ok 2024-09-20T18:54:16.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:15 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:54:16.124 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:16.154 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:16.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:16.155 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:16.231 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:54:16.242 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:54:16.255 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:16.306 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:16.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:16.307 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:16.408 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:54:16.434 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:16.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:16.435 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:54:16.536 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:16.564 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:16.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:16.565 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:16.666 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:16.692 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:16.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:16.693 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:16.793 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:16.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:16.837 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:16.837 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:16.938 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:16.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:16.976 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:16.976 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:17.023 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:54:16 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:54:17.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:16 smithi142 ceph-mon[32508]: 3.1d scrub starts 2024-09-20T18:54:17.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:16 smithi142 ceph-mon[32508]: 3.1d scrub ok 2024-09-20T18:54:17.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:16 smithi142 ceph-mon[32508]: 7.f scrub starts 2024-09-20T18:54:17.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:16 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:54:17.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:16 smithi083 ceph-mon[29421]: 3.1d scrub starts 2024-09-20T18:54:17.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:16 smithi083 ceph-mon[29421]: 3.1d scrub ok 2024-09-20T18:54:17.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:16 smithi083 ceph-mon[29421]: 7.f scrub starts 2024-09-20T18:54:17.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:16 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:54:17.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:16 smithi083 ceph-mon[36260]: 3.1d scrub starts 2024-09-20T18:54:17.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:16 smithi083 ceph-mon[36260]: 3.1d scrub ok 2024-09-20T18:54:17.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:16 smithi083 ceph-mon[36260]: 7.f scrub starts 2024-09-20T18:54:17.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:16 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:54:17.077 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:54:17.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:17.104 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:17.104 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:54:17.205 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:17.230 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:17.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:17.231 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:17.331 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:17.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:17.376 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:17.376 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:17.477 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:17.521 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:17.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:17.522 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:17.622 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:17.653 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:17.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:17.654 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:17.755 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:17.780 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:17.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:17.781 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:17.882 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:17.926 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:17.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:17.927 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:18.027 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T18:54:18.027 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:54:18.029 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:18.056 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:18.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:18.057 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:18.058 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:17 smithi083 ceph-mon[29421]: 7.f scrub ok 2024-09-20T18:54:18.058 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:17 smithi083 ceph-mon[29421]: pgmap v206: 148 pgs: 148 active+clean; 571 MiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.46k op/s 2024-09-20T18:54:18.058 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:17 smithi083 ceph-mon[29421]: 4.c scrub starts 2024-09-20T18:54:18.058 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:17 smithi083 ceph-mon[29421]: 4.c scrub ok 2024-09-20T18:54:18.058 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:17 smithi083 ceph-mon[36260]: 7.f scrub ok 2024-09-20T18:54:18.058 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:17 smithi083 ceph-mon[36260]: pgmap v206: 148 pgs: 148 active+clean; 571 MiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.46k op/s 2024-09-20T18:54:18.059 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:17 smithi083 ceph-mon[36260]: 4.c scrub starts 2024-09-20T18:54:18.059 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:17 smithi083 ceph-mon[36260]: 4.c scrub ok 2024-09-20T18:54:18.158 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:18.193 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:18.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:18.194 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:18.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:17 smithi142 ceph-mon[32508]: 7.f scrub ok 2024-09-20T18:54:18.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:17 smithi142 ceph-mon[32508]: pgmap v206: 148 pgs: 148 active+clean; 571 MiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.46k op/s 2024-09-20T18:54:18.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:17 smithi142 ceph-mon[32508]: 4.c scrub starts 2024-09-20T18:54:18.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:17 smithi142 ceph-mon[32508]: 4.c scrub ok 2024-09-20T18:54:18.294 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:54:18.320 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:18.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:18.321 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:54:18.422 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:18.450 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:18.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:18.451 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:18.552 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:18.578 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:18.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:18.579 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:18.680 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:54:18.706 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:18.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:18.707 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:54:18.808 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:18.809 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-20T18:54:18.853 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:18.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:18.854 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:18.955 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:18.991 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:18.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:18.992 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:19.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:18 smithi083 ceph-mon[29421]: 5.15 scrub starts 2024-09-20T18:54:19.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:18 smithi083 ceph-mon[29421]: 5.15 scrub ok 2024-09-20T18:54:19.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:18 smithi083 ceph-mon[29421]: 4.1e scrub starts 2024-09-20T18:54:19.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:18 smithi083 ceph-mon[29421]: 4.1e scrub ok 2024-09-20T18:54:19.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:18 smithi083 ceph-mon[29421]: 5.13 scrub starts 2024-09-20T18:54:19.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:18 smithi083 ceph-mon[29421]: 5.13 scrub ok 2024-09-20T18:54:19.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:18 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/826706707' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:54:19.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:18 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:54:19.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:18 smithi083 ceph-mon[36260]: 5.15 scrub starts 2024-09-20T18:54:19.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:18 smithi083 ceph-mon[36260]: 5.15 scrub ok 2024-09-20T18:54:19.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:18 smithi083 ceph-mon[36260]: 4.1e scrub starts 2024-09-20T18:54:19.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:18 smithi083 ceph-mon[36260]: 4.1e scrub ok 2024-09-20T18:54:19.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:18 smithi083 ceph-mon[36260]: 5.13 scrub starts 2024-09-20T18:54:19.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:18 smithi083 ceph-mon[36260]: 5.13 scrub ok 2024-09-20T18:54:19.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:18 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/826706707' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:54:19.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:18 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:54:19.093 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:19.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:19.120 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:19.120 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:19.149 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-20T18:54:19.221 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:19.248 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:19.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:19.249 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:19.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:18 smithi142 ceph-mon[32508]: 5.15 scrub starts 2024-09-20T18:54:19.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:18 smithi142 ceph-mon[32508]: 5.15 scrub ok 2024-09-20T18:54:19.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:18 smithi142 ceph-mon[32508]: 4.1e scrub starts 2024-09-20T18:54:19.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:18 smithi142 ceph-mon[32508]: 4.1e scrub ok 2024-09-20T18:54:19.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:18 smithi142 ceph-mon[32508]: 5.13 scrub starts 2024-09-20T18:54:19.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:18 smithi142 ceph-mon[32508]: 5.13 scrub ok 2024-09-20T18:54:19.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:18 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/826706707' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:54:19.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:18 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:54:19.350 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:19.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:19.403 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:19.404 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:19.504 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:19.529 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:19.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:19.530 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:19.630 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:54:19.658 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:19.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:19.659 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:54:19.667 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:54:19.667 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:54:19.667 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:54:19.667 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:54:19.667 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:54:19.667 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:54:19.667 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:54:19.668 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:54:19.668 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:54:19.668 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:54:19.668 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:54:19.668 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:54:19.668 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:54:19.668 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:54:19.668 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:54:19.668 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:54:19.760 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:19.796 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:19.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:19.796 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:19.897 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:19.934 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:19.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:19.935 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:20.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:19 smithi083 ceph-mon[29421]: 3.5 scrub starts 2024-09-20T18:54:20.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:19 smithi083 ceph-mon[29421]: 3.5 scrub ok 2024-09-20T18:54:20.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:19 smithi083 ceph-mon[29421]: 3.10 scrub starts 2024-09-20T18:54:20.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:19 smithi083 ceph-mon[29421]: 3.10 scrub ok 2024-09-20T18:54:20.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:19 smithi083 ceph-mon[29421]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T18:54:20.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:19 smithi083 ceph-mon[29421]: Cluster is now healthy 2024-09-20T18:54:20.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:19 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T18:54:20.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:19 smithi083 ceph-mon[29421]: osdmap e157: 8 total, 8 up, 8 in 2024-09-20T18:54:20.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:19 smithi083 ceph-mon[29421]: pgmap v208: 148 pgs: 148 active+clean; 571 MiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 833 B/s rd, 0 B/s wr, 837 op/s 2024-09-20T18:54:20.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:19 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1805185374' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:54:20.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:19 smithi083 ceph-mon[29421]: 3.f scrub starts 2024-09-20T18:54:20.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:19 smithi083 ceph-mon[29421]: 3.f scrub ok 2024-09-20T18:54:20.008 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:19 smithi083 ceph-mon[36260]: 3.5 scrub starts 2024-09-20T18:54:20.008 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:19 smithi083 ceph-mon[36260]: 3.5 scrub ok 2024-09-20T18:54:20.008 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:19 smithi083 ceph-mon[36260]: 3.10 scrub starts 2024-09-20T18:54:20.008 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:19 smithi083 ceph-mon[36260]: 3.10 scrub ok 2024-09-20T18:54:20.009 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:19 smithi083 ceph-mon[36260]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T18:54:20.009 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:19 smithi083 ceph-mon[36260]: Cluster is now healthy 2024-09-20T18:54:20.009 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:19 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T18:54:20.009 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:19 smithi083 ceph-mon[36260]: osdmap e157: 8 total, 8 up, 8 in 2024-09-20T18:54:20.009 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:19 smithi083 ceph-mon[36260]: pgmap v208: 148 pgs: 148 active+clean; 571 MiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 833 B/s rd, 0 B/s wr, 837 op/s 2024-09-20T18:54:20.009 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:19 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1805185374' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:54:20.009 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:19 smithi083 ceph-mon[36260]: 3.f scrub starts 2024-09-20T18:54:20.009 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:19 smithi083 ceph-mon[36260]: 3.f scrub ok 2024-09-20T18:54:20.036 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:20.063 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:20.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:20.064 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:20.165 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:20.199 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:20.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:20.200 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:20.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:19 smithi142 ceph-mon[32508]: 3.5 scrub starts 2024-09-20T18:54:20.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:19 smithi142 ceph-mon[32508]: 3.5 scrub ok 2024-09-20T18:54:20.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:19 smithi142 ceph-mon[32508]: 3.10 scrub starts 2024-09-20T18:54:20.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:19 smithi142 ceph-mon[32508]: 3.10 scrub ok 2024-09-20T18:54:20.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:19 smithi142 ceph-mon[32508]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T18:54:20.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:19 smithi142 ceph-mon[32508]: Cluster is now healthy 2024-09-20T18:54:20.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:19 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T18:54:20.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:19 smithi142 ceph-mon[32508]: osdmap e157: 8 total, 8 up, 8 in 2024-09-20T18:54:20.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:19 smithi142 ceph-mon[32508]: pgmap v208: 148 pgs: 148 active+clean; 571 MiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 833 B/s rd, 0 B/s wr, 837 op/s 2024-09-20T18:54:20.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:19 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1805185374' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:54:20.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:19 smithi142 ceph-mon[32508]: 3.f scrub starts 2024-09-20T18:54:20.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:19 smithi142 ceph-mon[32508]: 3.f scrub ok 2024-09-20T18:54:20.301 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:20.340 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:20.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:20.341 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:20.442 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:20.466 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:20.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:20.467 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:20.568 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:20.605 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:20.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:20.606 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:20.707 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:20.741 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:20.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:20.742 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:20.843 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:20.885 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:20.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:20.887 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:20.987 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:20.996 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:54:20 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:54:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:54:21.012 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:21.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:21.013 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:21.114 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:54:21.145 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:21.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:21.146 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:54:21.147 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:54:21.246 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:21.296 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:21.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:21.297 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:21.398 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:21.437 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:21.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:21.438 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:21.538 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:54:21.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:21.566 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:21.566 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:54:21.667 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:54:21.692 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:21.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:21.693 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:54:21.766 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T18:54:21.766 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-20T18:54:21.766 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.766 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:242561 7.e 7:739451b0:::benchmark_data_smithi083_70643_object119759:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e157)", 2024-09-20T18:54:21.766 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:54:21.736589+0000", 2024-09-20T18:54:21.766 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.011069869, 2024-09-20T18:54:21.766 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011099876999999999, 2024-09-20T18:54:21.767 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:54:21.767 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:54:21.767 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "waiting for sub ops", 2024-09-20T18:54:21.767 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:54:21.767 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:54:21.767 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:54:21.767 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 242561 2024-09-20T18:54:21.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:21.767 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:54:21.767 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.767 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:54:21.767 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:21.736589+0000", 2024-09-20T18:54:21.767 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:21.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:21.768 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.768 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:54:21.768 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:21.736589+0000", 2024-09-20T18:54:21.768 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:21.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:21.768 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.768 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:54:21.768 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:21.736590+0000", 2024-09-20T18:54:21.768 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.409e-06 2024-09-20T18:54:21.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:21.768 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.768 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:54:21.768 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:21.736596+0000", 2024-09-20T18:54:21.768 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7960000000000001e-06 2024-09-20T18:54:21.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:21.769 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.769 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:54:21.769 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:21.736607+0000", 2024-09-20T18:54:21.769 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0672e-05 2024-09-20T18:54:21.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:21.769 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.769 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:54:21.769 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:21.736615+0000", 2024-09-20T18:54:21.769 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9500000000000001e-06 2024-09-20T18:54:21.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:21.769 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.769 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:54:21.769 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:21.738861+0000", 2024-09-20T18:54:21.770 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002246456 2024-09-20T18:54:21.770 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:21.770 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.770 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:54:21.770 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:21.738963+0000", 2024-09-20T18:54:21.770 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000101759 2024-09-20T18:54:21.770 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:21.770 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.770 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:54:21.770 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:21.739062+0000", 2024-09-20T18:54:21.770 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9477999999999999e-05 2024-09-20T18:54:21.770 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:21.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.771 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:54:21.771 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:21.742838+0000", 2024-09-20T18:54:21.771 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00377547 2024-09-20T18:54:21.771 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:21.771 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:54:21.771 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:21.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:21.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.771 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:242566 7.e 7:7569ec47:::benchmark_data_smithi083_70643_object119764:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e157)", 2024-09-20T18:54:21.771 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:54:21.739491+0000", 2024-09-20T18:54:21.771 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0081679790000000006, 2024-09-20T18:54:21.771 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0082707600000000003, 2024-09-20T18:54:21.771 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:54:21.772 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:54:21.772 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "waiting for sub ops", 2024-09-20T18:54:21.772 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:54:21.772 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:54:21.772 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:54:21.772 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 242566 2024-09-20T18:54:21.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:21.772 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:54:21.772 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.772 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:54:21.772 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:21.739491+0000", 2024-09-20T18:54:21.772 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:21.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:21.772 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.773 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:54:21.773 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:21.739491+0000", 2024-09-20T18:54:21.773 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:21.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:21.773 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.773 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:54:21.773 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:21.739492+0000", 2024-09-20T18:54:21.773 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4619999999999999e-06 2024-09-20T18:54:21.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:21.773 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.773 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:54:21.773 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:21.739497+0000", 2024-09-20T18:54:21.773 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.356e-06 2024-09-20T18:54:21.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:21.773 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.773 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:54:21.774 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:21.739507+0000", 2024-09-20T18:54:21.774 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8640000000000008e-06 2024-09-20T18:54:21.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:21.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.774 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:54:21.774 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:21.739515+0000", 2024-09-20T18:54:21.774 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5979999999999999e-06 2024-09-20T18:54:21.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:21.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.774 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:54:21.774 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:21.743054+0000", 2024-09-20T18:54:21.774 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0035387700000000001 2024-09-20T18:54:21.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:21.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.774 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:54:21.775 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:21.743181+0000", 2024-09-20T18:54:21.775 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012704799999999999 2024-09-20T18:54:21.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:21.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.775 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:54:21.775 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:21.743394+0000", 2024-09-20T18:54:21.775 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00021298899999999999 2024-09-20T18:54:21.775 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:21.775 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:54:21.775 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:21.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:21.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.775 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:242568 7.a e157/83 7:543eb1d5:::benchmark_data_smithi083_70643_object119766:head v 157'15128, mlcod=157'15126)", 2024-09-20T18:54:21.775 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:54:21.745596+0000", 2024-09-20T18:54:21.776 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0020624800000000002, 2024-09-20T18:54:21.776 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0022267020000000001, 2024-09-20T18:54:21.776 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:54:21.776 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:54:21.776 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:54:21.776 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:54:21.776 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.776 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:54:21.777 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:21.745596+0000", 2024-09-20T18:54:21.777 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:21.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:21.777 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.777 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:54:21.777 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:21.745596+0000", 2024-09-20T18:54:21.777 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:21.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:21.777 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.777 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:54:21.777 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:21.745597+0000", 2024-09-20T18:54:21.777 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.068e-06 2024-09-20T18:54:21.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:21.777 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.777 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:54:21.778 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:21.745604+0000", 2024-09-20T18:54:21.778 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0230000000000004e-06 2024-09-20T18:54:21.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:21.778 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.778 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:54:21.778 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:21.745620+0000", 2024-09-20T18:54:21.778 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5523999999999999e-05 2024-09-20T18:54:21.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:21.778 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.778 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:54:21.778 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:21.745628+0000", 2024-09-20T18:54:21.778 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0050000000000004e-06 2024-09-20T18:54:21.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:21.778 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.779 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:54:21.779 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:21.745787+0000", 2024-09-20T18:54:21.779 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000159796 2024-09-20T18:54:21.779 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:21.779 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.779 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:54:21.779 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:21.745806+0000", 2024-09-20T18:54:21.779 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8467000000000001e-05 2024-09-20T18:54:21.779 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:21.779 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:54:21.779 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:21.779 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:21.779 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.779 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:242570 7.5 e157/83)", 2024-09-20T18:54:21.780 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:54:21.745662+0000", 2024-09-20T18:54:21.780 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0019963160000000001, 2024-09-20T18:54:21.780 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0022125669999999999, 2024-09-20T18:54:21.780 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:54:21.780 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:54:21.780 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "queued for pg", 2024-09-20T18:54:21.780 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:54:21.780 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.780 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:54:21.780 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:21.745662+0000", 2024-09-20T18:54:21.780 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:21.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:21.780 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.780 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:54:21.780 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:21.745662+0000", 2024-09-20T18:54:21.781 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:21.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:21.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.781 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:54:21.781 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:21.745663+0000", 2024-09-20T18:54:21.781 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0200000000000002e-07 2024-09-20T18:54:21.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:21.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.781 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:54:21.781 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:21.745667+0000", 2024-09-20T18:54:21.781 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9890000000000003e-06 2024-09-20T18:54:21.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:21.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.781 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:54:21.781 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:21.745674+0000", 2024-09-20T18:54:21.781 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0840000000000003e-06 2024-09-20T18:54:21.782 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:21.782 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.782 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:54:21.782 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:21.745683+0000", 2024-09-20T18:54:21.782 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8740000000000001e-06 2024-09-20T18:54:21.782 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:21.782 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:54:21.782 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:21.782 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:21.782 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.782 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:242573 7.8 7:1e029f08:::benchmark_data_smithi083_70643_object119771:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e157)", 2024-09-20T18:54:21.783 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:54:21.745710+0000", 2024-09-20T18:54:21.783 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.001948272, 2024-09-20T18:54:21.783 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0022107979999999999, 2024-09-20T18:54:21.783 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:54:21.783 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:54:21.783 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:54:21.783 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:54:21.783 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:54:21.783 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:54:21.783 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 242573 2024-09-20T18:54:21.783 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:21.783 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:54:21.783 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.783 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:54:21.783 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:21.745710+0000", 2024-09-20T18:54:21.784 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:21.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:21.784 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.784 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:54:21.784 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:21.745710+0000", 2024-09-20T18:54:21.784 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:21.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:21.784 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.784 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:54:21.784 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:21.745712+0000", 2024-09-20T18:54:21.784 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5549999999999999e-06 2024-09-20T18:54:21.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:21.784 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.784 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:54:21.784 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:21.745717+0000", 2024-09-20T18:54:21.784 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0610000000000003e-06 2024-09-20T18:54:21.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:21.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.785 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:54:21.785 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:21.745728+0000", 2024-09-20T18:54:21.785 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1331e-05 2024-09-20T18:54:21.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:21.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.785 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:54:21.785 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:21.745735+0000", 2024-09-20T18:54:21.785 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3790000000000003e-06 2024-09-20T18:54:21.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:21.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.785 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:54:21.786 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:21.745788+0000", 2024-09-20T18:54:21.786 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3325999999999997e-05 2024-09-20T18:54:21.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:21.786 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.786 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:54:21.786 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:21.745917+0000", 2024-09-20T18:54:21.786 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012923200000000001 2024-09-20T18:54:21.786 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:21.786 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:54:21.786 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:21.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:21.787 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.787 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:242567 7.2 e157/83)", 2024-09-20T18:54:21.787 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:54:21.745721+0000", 2024-09-20T18:54:21.787 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0019376459999999999, 2024-09-20T18:54:21.787 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0022550489999999999, 2024-09-20T18:54:21.788 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:54:21.788 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:54:21.788 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "queued for pg", 2024-09-20T18:54:21.788 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:54:21.788 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.788 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:54:21.788 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:21.745721+0000", 2024-09-20T18:54:21.788 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:21.788 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:21.788 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.788 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:54:21.788 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:21.745721+0000", 2024-09-20T18:54:21.788 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:54:21.789 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:21.789 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.789 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:54:21.789 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:21.745721+0000", 2024-09-20T18:54:21.789 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9799999999999999e-07 2024-09-20T18:54:21.789 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:21.789 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.789 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:54:21.789 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:21.745724+0000", 2024-09-20T18:54:21.789 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2459999999999999e-06 2024-09-20T18:54:21.789 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:21.790 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.790 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:54:21.790 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:21.745729+0000", 2024-09-20T18:54:21.790 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6689999999999997e-06 2024-09-20T18:54:21.790 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:54:21.790 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:54:21.790 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:54:21.790 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:21.745734+0000", 2024-09-20T18:54:21.790 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5059999999999997e-06 2024-09-20T18:54:21.790 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:21.790 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:54:21.790 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:21.790 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:54:21.791 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T18:54:21.791 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 6 2024-09-20T18:54:21.791 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T18:54:21.793 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:21.807 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-20T18:54:21.807 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T18:54:21.808 INFO:tasks.thrashosds.thrasher:Shrinking pool .mgr 2024-09-20T18:54:21.808 INFO:tasks.cephadm.ceph_manager.ceph:contract_pool .mgr by 10 min 8 2024-09-20T18:54:21.808 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T18:54:21.830 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:21.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:21.831 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:21.867 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:21 smithi083 ceph-mon[29421]: 6.14 scrub starts 2024-09-20T18:54:21.867 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:21 smithi083 ceph-mon[29421]: 6.14 scrub ok 2024-09-20T18:54:21.867 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:21 smithi083 ceph-mon[29421]: 4.11 scrub starts 2024-09-20T18:54:21.867 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:21 smithi083 ceph-mon[29421]: 4.11 scrub ok 2024-09-20T18:54:21.867 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:21 smithi083 ceph-mon[36260]: 6.14 scrub starts 2024-09-20T18:54:21.867 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:21 smithi083 ceph-mon[36260]: 6.14 scrub ok 2024-09-20T18:54:21.867 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:21 smithi083 ceph-mon[36260]: 4.11 scrub starts 2024-09-20T18:54:21.867 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:21 smithi083 ceph-mon[36260]: 4.11 scrub ok 2024-09-20T18:54:21.932 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:21.959 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:21.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:21.960 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:22.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:21 smithi142 ceph-mon[32508]: 6.14 scrub starts 2024-09-20T18:54:22.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:21 smithi142 ceph-mon[32508]: 6.14 scrub ok 2024-09-20T18:54:22.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:21 smithi142 ceph-mon[32508]: 4.11 scrub starts 2024-09-20T18:54:22.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:21 smithi142 ceph-mon[32508]: 4.11 scrub ok 2024-09-20T18:54:22.060 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:22.101 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:22.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:22.102 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:22.203 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:22.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:22.249 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:22.249 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:22.350 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:22.374 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:22.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:22.375 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:22.408 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T18:54:22.475 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:22.499 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:22.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:22.500 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:22.601 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:22.661 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_blocked_ops 2024-09-20T18:54:22.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:22.678 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:22.678 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:22.740 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:54:22.779 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:22.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:22.861 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:22.861 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:22.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:22 smithi083 ceph-mon[29421]: pgmap v209: 148 pgs: 148 active+clean; 448 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 890 op/s 2024-09-20T18:54:22.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:22 smithi083 ceph-mon[29421]: 4.0 scrub starts 2024-09-20T18:54:22.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:22 smithi083 ceph-mon[29421]: 4.0 scrub ok 2024-09-20T18:54:22.904 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:22 smithi083 ceph-mon[36260]: pgmap v209: 148 pgs: 148 active+clean; 448 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 890 op/s 2024-09-20T18:54:22.904 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:22 smithi083 ceph-mon[36260]: 4.0 scrub starts 2024-09-20T18:54:22.905 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:22 smithi083 ceph-mon[36260]: 4.0 scrub ok 2024-09-20T18:54:22.962 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:23.005 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:23.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:23.006 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:23.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:22 smithi142 ceph-mon[32508]: pgmap v209: 148 pgs: 148 active+clean; 448 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 890 op/s 2024-09-20T18:54:23.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:22 smithi142 ceph-mon[32508]: 4.0 scrub starts 2024-09-20T18:54:23.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:22 smithi142 ceph-mon[32508]: 4.0 scrub ok 2024-09-20T18:54:23.052 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T18:54:23.107 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:54:23.132 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:23.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:23.133 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:54:23.234 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:54:23.261 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:23.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:23.262 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:54:23.362 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:23.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:23.414 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:23.414 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:23.441 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:54:23.505 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:54:23.515 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:23.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:23.583 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:23.583 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:23.684 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:23.709 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:23.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:23.710 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:23.810 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:23.843 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:54:23.847 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-20T18:54:23.853 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:23.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:23.854 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:23.955 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:54:23.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:23.982 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:23.982 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:54:24.083 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:24.107 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:24.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:24.109 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:24.209 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:24.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:24.239 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:24.239 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:24.340 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:24.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:24.377 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:24.377 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:24.478 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:24.523 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:24.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:24.524 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:24.625 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:24.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:24.666 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:24.666 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:24.704 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":210,"stamp":"2024-09-20T18:54:22.849665+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":248191539,"num_objects":3978,"num_object_clones":0,"num_object_copies":11934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3978,"num_whiteouts":0,"num_read":1119,"num_read_kb":976,"num_write":242370,"num_write_kb":7860810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":590808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":90567,"ondisk_log_size":90567,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":1831316,"kb_used_data":822088,"kb_used_omap":12,"kb_used_meta":1008691,"kb_avail":748129900,"statfs":{"total":767960285184,"available":766085017600,"internally_reserved":0,"allocated":841818112,"data_stored":838554881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1032900144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,6,2,7,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-687931392,"num_objects":-10497,"num_object_clones":0,"num_object_copies":-31491,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-10497,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":10497,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.110170"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":224,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.800340+0000","last_change":"2024-09-20T18:54:13.958541+0000","last_active":"2024-09-20T18:54:18.800340+0000","last_peered":"2024-09-20T18:54:18.800340+0000","last_clean":"2024-09-20T18:54:18.800340+0000","last_became_active":"2024-09-20T18:49:45.943201+0000","last_became_peered":"2024-09-20T18:49:45.943201+0000","last_unstale":"2024-09-20T18:54:18.800340+0000","last_undegraded":"2024-09-20T18:54:18.800340+0000","last_fullsized":"2024-09-20T18:54:18.800340+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:13.958355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:13.958355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:27.449343+0000","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":253,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.797065+0000","last_change":"2024-09-20T18:53:15.772300+0000","last_active":"2024-09-20T18:54:18.797065+0000","last_peered":"2024-09-20T18:54:18.797065+0000","last_clean":"2024-09-20T18:54:18.797065+0000","last_became_active":"2024-09-20T18:52:15.609648+0000","last_became_peered":"2024-09-20T18:52:15.609648+0000","last_unstale":"2024-09-20T18:54:18.797065+0000","last_undegraded":"2024-09-20T18:54:18.797065+0000","last_fullsized":"2024-09-20T18:54:18.797065+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:53:15.772228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:15.772228+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:33.912905+0000","scrub_duration":37,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":216,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.807970+0000","last_change":"2024-09-20T18:53:21.291759+0000","last_active":"2024-09-20T18:54:18.807970+0000","last_peered":"2024-09-20T18:54:18.807970+0000","last_clean":"2024-09-20T18:54:18.807970+0000","last_became_active":"2024-09-20T18:49:49.959514+0000","last_became_peered":"2024-09-20T18:49:49.959514+0000","last_unstale":"2024-09-20T18:54:18.807970+0000","last_undegraded":"2024-09-20T18:54:18.807970+0000","last_fullsized":"2024-09-20T18:54:18.807970+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:21.291695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:21.291695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:25.316294+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":220,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.797157+0000","last_change":"2024-09-20T18:54:15.221404+0000","last_active":"2024-09-20T18:54:18.797157+0000","last_peered":"2024-09-20T18:54:18.797157+0000","last_clean":"2024-09-20T18:54:18.797157+0000","last_became_active":"2024-09-20T18:49:51.969421+0000","last_became_peered":"2024-09-20T18:49:51.969421+0000","last_unstale":"2024-09-20T18:54:18.797157+0000","last_undegraded":"2024-09-20T18:54:18.797157+0000","last_fullsized":"2024-09-20T18:54:18.797157+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:15.221342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:15.221342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:21.616726+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":212,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.800040+0000","last_change":"2024-09-20T18:53:46.283952+0000","last_active":"2024-09-20T18:54:18.800040+0000","last_peered":"2024-09-20T18:54:18.800040+0000","last_clean":"2024-09-20T18:54:18.800040+0000","last_became_active":"2024-09-20T18:49:52.141967+0000","last_became_peered":"2024-09-20T18:49:52.141967+0000","last_unstale":"2024-09-20T18:54:18.800040+0000","last_undegraded":"2024-09-20T18:54:18.800040+0000","last_fullsized":"2024-09-20T18:54:18.800040+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:46.283881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:46.283881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:49.076672+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":217,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.809752+0000","last_change":"2024-09-20T18:51:25.208165+0000","last_active":"2024-09-20T18:54:18.809752+0000","last_peered":"2024-09-20T18:54:18.809752+0000","last_clean":"2024-09-20T18:54:18.809752+0000","last_became_active":"2024-09-20T18:49:45.943607+0000","last_became_peered":"2024-09-20T18:49:45.943607+0000","last_unstale":"2024-09-20T18:54:18.809752+0000","last_undegraded":"2024-09-20T18:54:18.809752+0000","last_fullsized":"2024-09-20T18:54:18.809752+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:25.208073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:25.208073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:09.117437+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":237,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.803443+0000","last_change":"2024-09-20T18:51:07.130137+0000","last_active":"2024-09-20T18:54:18.803443+0000","last_peered":"2024-09-20T18:54:18.803443+0000","last_clean":"2024-09-20T18:54:18.803443+0000","last_became_active":"2024-09-20T18:49:47.952150+0000","last_became_peered":"2024-09-20T18:49:47.952150+0000","last_unstale":"2024-09-20T18:54:18.803443+0000","last_undegraded":"2024-09-20T18:54:18.803443+0000","last_fullsized":"2024-09-20T18:54:18.803443+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:51:07.130062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:07.130062+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:33.208818+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":216,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.796984+0000","last_change":"2024-09-20T18:54:11.246973+0000","last_active":"2024-09-20T18:54:18.796984+0000","last_peered":"2024-09-20T18:54:18.796984+0000","last_clean":"2024-09-20T18:54:18.796984+0000","last_became_active":"2024-09-20T18:49:49.956445+0000","last_became_peered":"2024-09-20T18:49:49.956445+0000","last_unstale":"2024-09-20T18:54:18.796984+0000","last_undegraded":"2024-09-20T18:54:18.796984+0000","last_fullsized":"2024-09-20T18:54:18.796984+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:11.246922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:11.246922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:18.822524+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"59'1","reported_seq":237,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.805847+0000","last_change":"2024-09-20T18:54:15.326527+0000","last_active":"2024-09-20T18:54:18.805847+0000","last_peered":"2024-09-20T18:54:18.805847+0000","last_clean":"2024-09-20T18:54:18.805847+0000","last_became_active":"2024-09-20T18:49:45.946171+0000","last_became_peered":"2024-09-20T18:49:45.946171+0000","last_unstale":"2024-09-20T18:54:18.805847+0000","last_undegraded":"2024-09-20T18:54:18.805847+0000","last_fullsized":"2024-09-20T18:54:18.805847+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:54:15.326476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:15.326476+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:27.762998+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":236,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.797324+0000","last_change":"2024-09-20T18:53:19.836082+0000","last_active":"2024-09-20T18:54:18.797324+0000","last_peered":"2024-09-20T18:54:18.797324+0000","last_clean":"2024-09-20T18:54:18.797324+0000","last_became_active":"2024-09-20T18:49:47.947448+0000","last_became_peered":"2024-09-20T18:49:47.947448+0000","last_unstale":"2024-09-20T18:54:18.797324+0000","last_undegraded":"2024-09-20T18:54:18.797324+0000","last_fullsized":"2024-09-20T18:54:18.797324+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:53:19.835996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:19.835996+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:39.939119+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":215,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.805495+0000","last_change":"2024-09-20T18:52:36.183932+0000","last_active":"2024-09-20T18:54:18.805495+0000","last_peered":"2024-09-20T18:54:18.805495+0000","last_clean":"2024-09-20T18:54:18.805495+0000","last_became_active":"2024-09-20T18:49:49.961188+0000","last_became_peered":"2024-09-20T18:49:49.961188+0000","last_unstale":"2024-09-20T18:54:18.805495+0000","last_undegraded":"2024-09-20T18:54:18.805495+0000","last_fullsized":"2024-09-20T18:54:18.805495+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:36.183875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:36.183875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:01.443173+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":195,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.794321+0000","last_change":"2024-09-20T18:49:51.966957+0000","last_active":"2024-09-20T18:54:18.794321+0000","last_peered":"2024-09-20T18:54:18.794321+0000","last_clean":"2024-09-20T18:54:18.794321+0000","last_became_active":"2024-09-20T18:49:51.966715+0000","last_became_peered":"2024-09-20T18:49:51.966715+0000","last_unstale":"2024-09-20T18:54:18.794321+0000","last_undegraded":"2024-09-20T18:54:18.794321+0000","last_fullsized":"2024-09-20T18:54:18.794321+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:01.897842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":215,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.805489+0000","last_change":"2024-09-20T18:52:00.970719+0000","last_active":"2024-09-20T18:54:18.805489+0000","last_peered":"2024-09-20T18:54:18.805489+0000","last_clean":"2024-09-20T18:54:18.805489+0000","last_became_active":"2024-09-20T18:49:45.944703+0000","last_became_peered":"2024-09-20T18:49:45.944703+0000","last_unstale":"2024-09-20T18:54:18.805489+0000","last_undegraded":"2024-09-20T18:54:18.805489+0000","last_fullsized":"2024-09-20T18:54:18.805489+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:00.970650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:00.970650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:09.393254+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":248,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.797255+0000","last_change":"2024-09-20T18:51:26.494194+0000","last_active":"2024-09-20T18:54:18.797255+0000","last_peered":"2024-09-20T18:54:18.797255+0000","last_clean":"2024-09-20T18:54:18.797255+0000","last_became_active":"2024-09-20T18:51:23.813399+0000","last_became_peered":"2024-09-20T18:51:23.813399+0000","last_unstale":"2024-09-20T18:54:18.797255+0000","last_undegraded":"2024-09-20T18:54:18.797255+0000","last_fullsized":"2024-09-20T18:54:18.797255+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-20T18:51:26.494118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:26.494118+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:34.803592+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":215,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.790941+0000","last_change":"2024-09-20T18:52:38.799542+0000","last_active":"2024-09-20T18:54:18.790941+0000","last_peered":"2024-09-20T18:54:18.790941+0000","last_clean":"2024-09-20T18:54:18.790941+0000","last_became_active":"2024-09-20T18:49:49.959696+0000","last_became_peered":"2024-09-20T18:49:49.959696+0000","last_unstale":"2024-09-20T18:54:18.790941+0000","last_undegraded":"2024-09-20T18:54:18.790941+0000","last_fullsized":"2024-09-20T18:54:18.790941+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:38.799487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:38.799487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:06.357517+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":211,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.805253+0000","last_change":"2024-09-20T18:53:21.273835+0000","last_active":"2024-09-20T18:54:18.805253+0000","last_peered":"2024-09-20T18:54:18.805253+0000","last_clean":"2024-09-20T18:54:18.805253+0000","last_became_active":"2024-09-20T18:49:51.970663+0000","last_became_peered":"2024-09-20T18:49:51.970663+0000","last_unstale":"2024-09-20T18:54:18.805253+0000","last_undegraded":"2024-09-20T18:54:18.805253+0000","last_fullsized":"2024-09-20T18:54:18.805253+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:21.273788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:21.273788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:31.269805+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":213,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.797462+0000","last_change":"2024-09-20T18:53:53.940250+0000","last_active":"2024-09-20T18:54:18.797462+0000","last_peered":"2024-09-20T18:54:18.797462+0000","last_clean":"2024-09-20T18:54:18.797462+0000","last_became_active":"2024-09-20T18:49:52.141170+0000","last_became_peered":"2024-09-20T18:49:52.141170+0000","last_unstale":"2024-09-20T18:54:18.797462+0000","last_undegraded":"2024-09-20T18:54:18.797462+0000","last_fullsized":"2024-09-20T18:54:18.797462+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:53.940191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:53.940191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:59.342447+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":215,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.796165+0000","last_change":"2024-09-20T18:52:42.099489+0000","last_active":"2024-09-20T18:54:18.796165+0000","last_peered":"2024-09-20T18:54:18.796165+0000","last_clean":"2024-09-20T18:54:18.796165+0000","last_became_active":"2024-09-20T18:49:45.950480+0000","last_became_peered":"2024-09-20T18:49:45.950480+0000","last_unstale":"2024-09-20T18:54:18.796165+0000","last_undegraded":"2024-09-20T18:54:18.796165+0000","last_fullsized":"2024-09-20T18:54:18.796165+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:42.099418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:42.099418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":246,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.802047+0000","last_change":"2024-09-20T18:53:26.762207+0000","last_active":"2024-09-20T18:54:18.802047+0000","last_peered":"2024-09-20T18:54:18.802047+0000","last_clean":"2024-09-20T18:54:18.802047+0000","last_became_active":"2024-09-20T18:49:47.950939+0000","last_became_peered":"2024-09-20T18:49:47.950939+0000","last_unstale":"2024-09-20T18:54:18.802047+0000","last_undegraded":"2024-09-20T18:54:18.802047+0000","last_fullsized":"2024-09-20T18:54:18.802047+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:53:26.762131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:26.762131+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:38.227744+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":208,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.808501+0000","last_change":"2024-09-20T18:51:17.248705+0000","last_active":"2024-09-20T18:54:18.808501+0000","last_peered":"2024-09-20T18:54:18.808501+0000","last_clean":"2024-09-20T18:54:18.808501+0000","last_became_active":"2024-09-20T18:49:49.958518+0000","last_became_peered":"2024-09-20T18:49:49.958518+0000","last_unstale":"2024-09-20T18:54:18.808501+0000","last_undegraded":"2024-09-20T18:54:18.808501+0000","last_fullsized":"2024-09-20T18:54:18.808501+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:17.248634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:17.248634+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:35.297969+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":213,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.802193+0000","last_change":"2024-09-20T18:53:54.259243+0000","last_active":"2024-09-20T18:54:18.802193+0000","last_peered":"2024-09-20T18:54:18.802193+0000","last_clean":"2024-09-20T18:54:18.802193+0000","last_became_active":"2024-09-20T18:49:52.140040+0000","last_became_peered":"2024-09-20T18:49:52.140040+0000","last_unstale":"2024-09-20T18:54:18.802193+0000","last_undegraded":"2024-09-20T18:54:18.802193+0000","last_fullsized":"2024-09-20T18:54:18.802193+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:54.259174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:54.259174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:06.632880+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"59'1","reported_seq":230,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.797376+0000","last_change":"2024-09-20T18:53:42.840884+0000","last_active":"2024-09-20T18:54:18.797376+0000","last_peered":"2024-09-20T18:54:18.797376+0000","last_clean":"2024-09-20T18:54:18.797376+0000","last_became_active":"2024-09-20T18:49:45.940100+0000","last_became_peered":"2024-09-20T18:49:45.940100+0000","last_unstale":"2024-09-20T18:54:18.797376+0000","last_undegraded":"2024-09-20T18:54:18.797376+0000","last_fullsized":"2024-09-20T18:54:18.797376+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:53:42.840800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:42.840800+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:56.760029+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":227,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.799729+0000","last_change":"2024-09-20T18:49:47.951215+0000","last_active":"2024-09-20T18:54:18.799729+0000","last_peered":"2024-09-20T18:54:18.799729+0000","last_clean":"2024-09-20T18:54:18.799729+0000","last_became_active":"2024-09-20T18:49:47.950909+0000","last_became_peered":"2024-09-20T18:49:47.950909+0000","last_unstale":"2024-09-20T18:54:18.799729+0000","last_undegraded":"2024-09-20T18:54:18.799729+0000","last_fullsized":"2024-09-20T18:54:18.799729+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:00.257976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":216,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.797513+0000","last_change":"2024-09-20T18:53:12.820752+0000","last_active":"2024-09-20T18:54:18.797513+0000","last_peered":"2024-09-20T18:54:18.797513+0000","last_clean":"2024-09-20T18:54:18.797513+0000","last_became_active":"2024-09-20T18:49:49.959102+0000","last_became_peered":"2024-09-20T18:49:49.959102+0000","last_unstale":"2024-09-20T18:54:18.797513+0000","last_undegraded":"2024-09-20T18:54:18.797513+0000","last_fullsized":"2024-09-20T18:54:18.797513+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:12.820697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:12.820697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:29.865011+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":212,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.791165+0000","last_change":"2024-09-20T18:52:15.826819+0000","last_active":"2024-09-20T18:54:18.791165+0000","last_peered":"2024-09-20T18:54:18.791165+0000","last_clean":"2024-09-20T18:54:18.791165+0000","last_became_active":"2024-09-20T18:49:51.970058+0000","last_became_peered":"2024-09-20T18:49:51.970058+0000","last_unstale":"2024-09-20T18:54:18.791165+0000","last_undegraded":"2024-09-20T18:54:18.791165+0000","last_fullsized":"2024-09-20T18:54:18.791165+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T18:52:15.826638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:15.826638+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:28.538285+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":217,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.808105+0000","last_change":"2024-09-20T18:52:47.158384+0000","last_active":"2024-09-20T18:54:18.808105+0000","last_peered":"2024-09-20T18:54:18.808105+0000","last_clean":"2024-09-20T18:54:18.808105+0000","last_became_active":"2024-09-20T18:49:45.933078+0000","last_became_peered":"2024-09-20T18:49:45.933078+0000","last_unstale":"2024-09-20T18:54:18.808105+0000","last_undegraded":"2024-09-20T18:54:18.808105+0000","last_fullsized":"2024-09-20T18:54:18.808105+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:52:47.158260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:47.158260+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:05.321731+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":235,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.800034+0000","last_change":"2024-09-20T18:54:17.919123+0000","last_active":"2024-09-20T18:54:18.800034+0000","last_peered":"2024-09-20T18:54:18.800034+0000","last_clean":"2024-09-20T18:54:18.800034+0000","last_became_active":"2024-09-20T18:49:48.500809+0000","last_became_peered":"2024-09-20T18:49:48.500809+0000","last_unstale":"2024-09-20T18:54:18.800034+0000","last_undegraded":"2024-09-20T18:54:18.800034+0000","last_fullsized":"2024-09-20T18:54:18.800034+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:54:17.919055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:17.919055+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:19.776366+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":282,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.803569+0000","last_change":"2024-09-20T18:53:44.524093+0000","last_active":"2024-09-20T18:54:18.803569+0000","last_peered":"2024-09-20T18:54:18.803569+0000","last_clean":"2024-09-20T18:54:18.803569+0000","last_became_active":"2024-09-20T18:49:49.954772+0000","last_became_peered":"2024-09-20T18:49:49.954772+0000","last_unstale":"2024-09-20T18:54:18.803569+0000","last_undegraded":"2024-09-20T18:54:18.803569+0000","last_fullsized":"2024-09-20T18:54:18.803569+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:53:44.524040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:44.524040+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:03.085945+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.c","version":"157'15193","reported_seq":15463,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:20.822951+0000","last_change":"2024-09-20T18:53:47.244353+0000","last_active":"2024-09-20T18:54:20.822951+0000","last_peered":"2024-09-20T18:54:20.822951+0000","last_clean":"2024-09-20T18:54:20.822951+0000","last_became_active":"2024-09-20T18:50:49.843744+0000","last_became_peered":"2024-09-20T18:50:49.843744+0000","last_unstale":"2024-09-20T18:54:20.822951+0000","last_undegraded":"2024-09-20T18:54:20.822951+0000","last_fullsized":"2024-09-20T18:54:20.822951+0000","mapping_epoch":83,"log_start":"128'8900","ondisk_log_start":"128'8900","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'12973","last_scrub_stamp":"2024-09-20T18:53:47.243983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:53:47.243983+0000","objects_scrubbed":2483,"log_size":6293,"log_dups_size":0,"ondisk_log_size":6293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:08.150940+0000","scrub_duration":693,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16056320,"num_objects":245,"num_object_clones":0,"num_object_copies":735,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15193,"num_write_kb":494016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":229,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.801362+0000","last_change":"2024-09-20T18:53:48.782289+0000","last_active":"2024-09-20T18:54:18.801362+0000","last_peered":"2024-09-20T18:54:18.801362+0000","last_clean":"2024-09-20T18:54:18.801362+0000","last_became_active":"2024-09-20T18:49:58.992302+0000","last_became_peered":"2024-09-20T18:49:58.992302+0000","last_unstale":"2024-09-20T18:54:18.801362+0000","last_undegraded":"2024-09-20T18:54:18.801362+0000","last_fullsized":"2024-09-20T18:54:18.801362+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:53:48.782192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:48.782192+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:04.815274+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":217,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.813656+0000","last_change":"2024-09-20T18:53:26.175148+0000","last_active":"2024-09-20T18:54:18.813656+0000","last_peered":"2024-09-20T18:54:18.813656+0000","last_clean":"2024-09-20T18:54:18.813656+0000","last_became_active":"2024-09-20T18:49:45.936104+0000","last_became_peered":"2024-09-20T18:49:45.936104+0000","last_unstale":"2024-09-20T18:54:18.813656+0000","last_undegraded":"2024-09-20T18:54:18.813656+0000","last_fullsized":"2024-09-20T18:54:18.813656+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:26.175083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:26.175083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:49.575901+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":283,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.797542+0000","last_change":"2024-09-20T18:54:14.292660+0000","last_active":"2024-09-20T18:54:18.797542+0000","last_peered":"2024-09-20T18:54:18.797542+0000","last_clean":"2024-09-20T18:54:18.797542+0000","last_became_active":"2024-09-20T18:49:49.960697+0000","last_became_peered":"2024-09-20T18:49:49.960697+0000","last_unstale":"2024-09-20T18:54:18.797542+0000","last_undegraded":"2024-09-20T18:54:18.797542+0000","last_fullsized":"2024-09-20T18:54:18.797542+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:54:14.292610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:14.292610+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:35.787441+0000","scrub_duration":34,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":212,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.805393+0000","last_change":"2024-09-20T18:54:10.381171+0000","last_active":"2024-09-20T18:54:18.805393+0000","last_peered":"2024-09-20T18:54:18.805393+0000","last_clean":"2024-09-20T18:54:18.805393+0000","last_became_active":"2024-09-20T18:49:51.971399+0000","last_became_peered":"2024-09-20T18:49:51.971399+0000","last_unstale":"2024-09-20T18:54:18.805393+0000","last_undegraded":"2024-09-20T18:54:18.805393+0000","last_fullsized":"2024-09-20T18:54:18.805393+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:10.381108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:10.381108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:32.368649+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"157'14999","reported_seq":15159,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:21.219937+0000","last_change":"2024-09-20T18:50:49.841111+0000","last_active":"2024-09-20T18:54:21.219937+0000","last_peered":"2024-09-20T18:54:21.219937+0000","last_clean":"2024-09-20T18:54:21.219937+0000","last_became_active":"2024-09-20T18:50:49.840819+0000","last_became_peered":"2024-09-20T18:50:49.840819+0000","last_unstale":"2024-09-20T18:54:21.219937+0000","last_undegraded":"2024-09-20T18:54:21.219937+0000","last_fullsized":"2024-09-20T18:54:21.219937+0000","mapping_epoch":83,"log_start":"136'10300","ondisk_log_start":"136'10300","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.819660+0000","objects_scrubbed":0,"log_size":4699,"log_dups_size":0,"ondisk_log_size":4699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:38.259712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15269888,"num_objects":233,"num_object_clones":0,"num_object_copies":699,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14999,"num_write_kb":487424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":245,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.813419+0000","last_change":"2024-09-20T18:52:42.130129+0000","last_active":"2024-09-20T18:54:18.813419+0000","last_peered":"2024-09-20T18:54:18.813419+0000","last_clean":"2024-09-20T18:54:18.813419+0000","last_became_active":"2024-09-20T18:49:48.499189+0000","last_became_peered":"2024-09-20T18:49:48.499189+0000","last_unstale":"2024-09-20T18:54:18.813419+0000","last_undegraded":"2024-09-20T18:54:18.813419+0000","last_fullsized":"2024-09-20T18:54:18.813419+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-20T18:52:42.130045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:42.130045+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":209,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.802809+0000","last_change":"2024-09-20T18:49:45.940816+0000","last_active":"2024-09-20T18:54:18.802809+0000","last_peered":"2024-09-20T18:54:18.802809+0000","last_clean":"2024-09-20T18:54:18.802809+0000","last_became_active":"2024-09-20T18:49:45.940544+0000","last_became_peered":"2024-09-20T18:49:45.940544+0000","last_unstale":"2024-09-20T18:54:18.802809+0000","last_undegraded":"2024-09-20T18:54:18.802809+0000","last_fullsized":"2024-09-20T18:54:18.802809+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:56.293250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":217,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.797159+0000","last_change":"2024-09-20T18:53:09.779280+0000","last_active":"2024-09-20T18:54:18.797159+0000","last_peered":"2024-09-20T18:54:18.797159+0000","last_clean":"2024-09-20T18:54:18.797159+0000","last_became_active":"2024-09-20T18:49:49.963073+0000","last_became_peered":"2024-09-20T18:49:49.963073+0000","last_unstale":"2024-09-20T18:54:18.797159+0000","last_undegraded":"2024-09-20T18:54:18.797159+0000","last_fullsized":"2024-09-20T18:54:18.797159+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:09.779228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:09.779228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:39.433897+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":213,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.797270+0000","last_change":"2024-09-20T18:53:27.796357+0000","last_active":"2024-09-20T18:54:18.797270+0000","last_peered":"2024-09-20T18:54:18.797270+0000","last_clean":"2024-09-20T18:54:18.797270+0000","last_became_active":"2024-09-20T18:49:51.968793+0000","last_became_peered":"2024-09-20T18:49:51.968793+0000","last_unstale":"2024-09-20T18:54:18.797270+0000","last_undegraded":"2024-09-20T18:54:18.797270+0000","last_fullsized":"2024-09-20T18:54:18.797270+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:27.796289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:27.796289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:48.888717+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2","version":"157'15162","reported_seq":15322,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:21.248513+0000","last_change":"2024-09-20T18:50:49.842736+0000","last_active":"2024-09-20T18:54:21.248513+0000","last_peered":"2024-09-20T18:54:21.248513+0000","last_clean":"2024-09-20T18:54:21.248513+0000","last_became_active":"2024-09-20T18:50:49.842465+0000","last_became_peered":"2024-09-20T18:50:49.842465+0000","last_unstale":"2024-09-20T18:54:21.248513+0000","last_undegraded":"2024-09-20T18:54:21.248513+0000","last_fullsized":"2024-09-20T18:54:21.248513+0000","mapping_epoch":83,"log_start":"135'10500","ondisk_log_start":"135'10500","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.819660+0000","objects_scrubbed":0,"log_size":4662,"log_dups_size":0,"ondisk_log_size":4662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:31.286096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14811136,"num_objects":226,"num_object_clones":0,"num_object_copies":678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15162,"num_write_kb":492416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":250,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.797138+0000","last_change":"2024-09-20T18:53:44.879133+0000","last_active":"2024-09-20T18:54:18.797138+0000","last_peered":"2024-09-20T18:54:18.797138+0000","last_clean":"2024-09-20T18:54:18.797138+0000","last_became_active":"2024-09-20T18:49:47.941895+0000","last_became_peered":"2024-09-20T18:49:47.941895+0000","last_unstale":"2024-09-20T18:54:18.797138+0000","last_undegraded":"2024-09-20T18:54:18.797138+0000","last_fullsized":"2024-09-20T18:54:18.797138+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-20T18:53:44.879048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:44.879048+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:52.038099+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":217,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.800322+0000","last_change":"2024-09-20T18:51:27.187060+0000","last_active":"2024-09-20T18:54:18.800322+0000","last_peered":"2024-09-20T18:54:18.800322+0000","last_clean":"2024-09-20T18:54:18.800322+0000","last_became_active":"2024-09-20T18:49:45.940469+0000","last_became_peered":"2024-09-20T18:49:45.940469+0000","last_unstale":"2024-09-20T18:54:18.800322+0000","last_undegraded":"2024-09-20T18:54:18.800322+0000","last_fullsized":"2024-09-20T18:54:18.800322+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:51:27.186991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:27.186991+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:58.816707+0000","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"66'8","reported_seq":281,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.813209+0000","last_change":"2024-09-20T18:53:44.293165+0000","last_active":"2024-09-20T18:54:18.813209+0000","last_peered":"2024-09-20T18:54:18.813209+0000","last_clean":"2024-09-20T18:54:18.813209+0000","last_became_active":"2024-09-20T18:49:49.956133+0000","last_became_peered":"2024-09-20T18:49:49.956133+0000","last_unstale":"2024-09-20T18:54:18.813209+0000","last_undegraded":"2024-09-20T18:54:18.813209+0000","last_fullsized":"2024-09-20T18:54:18.813209+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:53:44.293094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:44.293094+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:47.176037+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":212,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.791128+0000","last_change":"2024-09-20T18:52:42.880767+0000","last_active":"2024-09-20T18:54:18.791128+0000","last_peered":"2024-09-20T18:54:18.791128+0000","last_clean":"2024-09-20T18:54:18.791128+0000","last_became_active":"2024-09-20T18:49:52.142113+0000","last_became_peered":"2024-09-20T18:49:52.142113+0000","last_unstale":"2024-09-20T18:54:18.791128+0000","last_undegraded":"2024-09-20T18:54:18.791128+0000","last_fullsized":"2024-09-20T18:54:18.791128+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:42.880724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:42.880724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:08.373192+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"157'15172","reported_seq":15400,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:20.762264+0000","last_change":"2024-09-20T18:52:00.368447+0000","last_active":"2024-09-20T18:54:20.762264+0000","last_peered":"2024-09-20T18:54:20.762264+0000","last_clean":"2024-09-20T18:54:20.762264+0000","last_became_active":"2024-09-20T18:50:49.844624+0000","last_became_peered":"2024-09-20T18:50:49.844624+0000","last_unstale":"2024-09-20T18:54:20.762264+0000","last_undegraded":"2024-09-20T18:54:20.762264+0000","last_fullsized":"2024-09-20T18:54:20.762264+0000","mapping_epoch":83,"log_start":"135'10326","ondisk_log_start":"135'10326","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"111'6092","last_scrub_stamp":"2024-09-20T18:52:00.368276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:52:00.368276+0000","objects_scrubbed":6040,"log_size":4846,"log_dups_size":0,"ondisk_log_size":4846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:40.875204+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17825792,"num_objects":272,"num_object_clones":0,"num_object_copies":816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15172,"num_write_kb":494208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":236,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.812697+0000","last_change":"2024-09-20T18:52:39.246125+0000","last_active":"2024-09-20T18:54:18.812697+0000","last_peered":"2024-09-20T18:54:18.812697+0000","last_clean":"2024-09-20T18:54:18.812697+0000","last_became_active":"2024-09-20T18:49:47.949626+0000","last_became_peered":"2024-09-20T18:49:47.949626+0000","last_unstale":"2024-09-20T18:54:18.812697+0000","last_undegraded":"2024-09-20T18:54:18.812697+0000","last_fullsized":"2024-09-20T18:54:18.812697+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:52:39.246050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:39.246050+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:09.285670+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":232,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.805889+0000","last_change":"2024-09-20T18:54:18.326947+0000","last_active":"2024-09-20T18:54:18.805889+0000","last_peered":"2024-09-20T18:54:18.805889+0000","last_clean":"2024-09-20T18:54:18.805889+0000","last_became_active":"2024-09-20T18:49:45.942496+0000","last_became_peered":"2024-09-20T18:49:45.942496+0000","last_unstale":"2024-09-20T18:54:18.805889+0000","last_undegraded":"2024-09-20T18:54:18.805889+0000","last_fullsized":"2024-09-20T18:54:18.805889+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:18.326875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:18.326875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:33.692074+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":272,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.795834+0000","last_change":"2024-09-20T18:51:16.088407+0000","last_active":"2024-09-20T18:54:18.795834+0000","last_peered":"2024-09-20T18:54:18.795834+0000","last_clean":"2024-09-20T18:54:18.795834+0000","last_became_active":"2024-09-20T18:49:49.959365+0000","last_became_peered":"2024-09-20T18:49:49.959365+0000","last_unstale":"2024-09-20T18:54:18.795834+0000","last_undegraded":"2024-09-20T18:54:18.795834+0000","last_fullsized":"2024-09-20T18:54:18.795834+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:51:16.088343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:16.088343+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:35.866980+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":196,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.794453+0000","last_change":"2024-09-20T18:49:51.965733+0000","last_active":"2024-09-20T18:54:18.794453+0000","last_peered":"2024-09-20T18:54:18.794453+0000","last_clean":"2024-09-20T18:54:18.794453+0000","last_became_active":"2024-09-20T18:49:51.964780+0000","last_became_peered":"2024-09-20T18:49:51.964780+0000","last_unstale":"2024-09-20T18:54:18.794453+0000","last_undegraded":"2024-09-20T18:54:18.794453+0000","last_fullsized":"2024-09-20T18:54:18.794453+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:56.818438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.0","version":"157'15283","reported_seq":15549,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:20.815415+0000","last_change":"2024-09-20T18:53:50.922274+0000","last_active":"2024-09-20T18:54:20.815415+0000","last_peered":"2024-09-20T18:54:20.815415+0000","last_clean":"2024-09-20T18:54:20.815415+0000","last_became_active":"2024-09-20T18:50:49.841019+0000","last_became_peered":"2024-09-20T18:50:49.841019+0000","last_unstale":"2024-09-20T18:54:20.815415+0000","last_undegraded":"2024-09-20T18:54:20.815415+0000","last_fullsized":"2024-09-20T18:54:20.815415+0000","mapping_epoch":83,"log_start":"132'9500","ondisk_log_start":"132'9500","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'13304","last_scrub_stamp":"2024-09-20T18:53:50.921985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:53:50.921985+0000","objects_scrubbed":2243,"log_size":5783,"log_dups_size":0,"ondisk_log_size":5783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:54.364286+0000","scrub_duration":608,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16056320,"num_objects":245,"num_object_clones":0,"num_object_copies":735,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15283,"num_write_kb":496896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":256,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.795982+0000","last_change":"2024-09-20T18:54:14.923895+0000","last_active":"2024-09-20T18:54:18.795982+0000","last_peered":"2024-09-20T18:54:18.795982+0000","last_clean":"2024-09-20T18:54:18.795982+0000","last_became_active":"2024-09-20T18:49:48.497736+0000","last_became_peered":"2024-09-20T18:49:48.497736+0000","last_unstale":"2024-09-20T18:54:18.795982+0000","last_undegraded":"2024-09-20T18:54:18.795982+0000","last_fullsized":"2024-09-20T18:54:18.795982+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-20T18:54:14.923682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:14.923682+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:38.986258+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":209,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.807868+0000","last_change":"2024-09-20T18:49:45.942589+0000","last_active":"2024-09-20T18:54:18.807868+0000","last_peered":"2024-09-20T18:54:18.807868+0000","last_clean":"2024-09-20T18:54:18.807868+0000","last_became_active":"2024-09-20T18:49:45.942253+0000","last_became_peered":"2024-09-20T18:49:45.942253+0000","last_unstale":"2024-09-20T18:54:18.807868+0000","last_undegraded":"2024-09-20T18:54:18.807868+0000","last_fullsized":"2024-09-20T18:54:18.807868+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:59.256981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":216,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.803051+0000","last_change":"2024-09-20T18:53:12.347123+0000","last_active":"2024-09-20T18:54:18.803051+0000","last_peered":"2024-09-20T18:54:18.803051+0000","last_clean":"2024-09-20T18:54:18.803051+0000","last_became_active":"2024-09-20T18:49:49.954299+0000","last_became_peered":"2024-09-20T18:49:49.954299+0000","last_unstale":"2024-09-20T18:54:18.803051+0000","last_undegraded":"2024-09-20T18:54:18.803051+0000","last_fullsized":"2024-09-20T18:54:18.803051+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:12.347044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:12.347044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:41.912430+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":213,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.807256+0000","last_change":"2024-09-20T18:53:25.362907+0000","last_active":"2024-09-20T18:54:18.807256+0000","last_peered":"2024-09-20T18:54:18.807256+0000","last_clean":"2024-09-20T18:54:18.807256+0000","last_became_active":"2024-09-20T18:49:52.139981+0000","last_became_peered":"2024-09-20T18:49:52.139981+0000","last_unstale":"2024-09-20T18:54:18.807256+0000","last_undegraded":"2024-09-20T18:54:18.807256+0000","last_fullsized":"2024-09-20T18:54:18.807256+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:25.362828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:25.362828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:55.107515+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"157'14889","reported_seq":15117,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:20.811746+0000","last_change":"2024-09-20T18:52:03.401113+0000","last_active":"2024-09-20T18:54:20.811746+0000","last_peered":"2024-09-20T18:54:20.811746+0000","last_clean":"2024-09-20T18:54:20.811746+0000","last_became_active":"2024-09-20T18:50:49.842473+0000","last_became_peered":"2024-09-20T18:50:49.842473+0000","last_unstale":"2024-09-20T18:54:20.811746+0000","last_undegraded":"2024-09-20T18:54:20.811746+0000","last_fullsized":"2024-09-20T18:54:20.811746+0000","mapping_epoch":83,"log_start":"127'8600","ondisk_log_start":"127'8600","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"112'6142","last_scrub_stamp":"2024-09-20T18:52:03.400886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:52:03.400886+0000","objects_scrubbed":6079,"log_size":6289,"log_dups_size":0,"ondisk_log_size":6289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:41.532845+0000","scrub_duration":2059,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15663104,"num_objects":239,"num_object_clones":0,"num_object_copies":717,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14889,"num_write_kb":484096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"66'28","reported_seq":316,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.807720+0000","last_change":"2024-09-20T18:51:57.293659+0000","last_active":"2024-09-20T18:54:18.807720+0000","last_peered":"2024-09-20T18:54:18.807720+0000","last_clean":"2024-09-20T18:54:18.807720+0000","last_became_active":"2024-09-20T18:49:47.955335+0000","last_became_peered":"2024-09-20T18:49:47.955335+0000","last_unstale":"2024-09-20T18:54:18.807720+0000","last_undegraded":"2024-09-20T18:54:18.807720+0000","last_fullsized":"2024-09-20T18:54:18.807720+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'28","last_scrub_stamp":"2024-09-20T18:51:57.293574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:57.293574+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:08.276804+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":209,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.797458+0000","last_change":"2024-09-20T18:49:45.945275+0000","last_active":"2024-09-20T18:54:18.797458+0000","last_peered":"2024-09-20T18:54:18.797458+0000","last_clean":"2024-09-20T18:54:18.797458+0000","last_became_active":"2024-09-20T18:49:45.944992+0000","last_became_peered":"2024-09-20T18:49:45.944992+0000","last_unstale":"2024-09-20T18:54:18.797458+0000","last_undegraded":"2024-09-20T18:54:18.797458+0000","last_fullsized":"2024-09-20T18:54:18.797458+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:01.052027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":242,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.805632+0000","last_change":"2024-09-20T18:53:49.323451+0000","last_active":"2024-09-20T18:54:18.805632+0000","last_peered":"2024-09-20T18:54:18.805632+0000","last_clean":"2024-09-20T18:54:18.805632+0000","last_became_active":"2024-09-20T18:49:44.936027+0000","last_became_peered":"2024-09-20T18:49:44.936027+0000","last_unstale":"2024-09-20T18:54:18.805632+0000","last_undegraded":"2024-09-20T18:54:18.805632+0000","last_fullsized":"2024-09-20T18:54:18.805632+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-20T18:53:49.323385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:53:49.323385+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:50.958195+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.0012781159999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":216,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.802836+0000","last_change":"2024-09-20T18:53:12.897519+0000","last_active":"2024-09-20T18:54:18.802836+0000","last_peered":"2024-09-20T18:54:18.802836+0000","last_clean":"2024-09-20T18:54:18.802836+0000","last_became_active":"2024-09-20T18:49:49.953971+0000","last_became_peered":"2024-09-20T18:49:49.953971+0000","last_unstale":"2024-09-20T18:54:18.802836+0000","last_undegraded":"2024-09-20T18:54:18.802836+0000","last_fullsized":"2024-09-20T18:54:18.802836+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:12.897051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:12.897051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:39.168470+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":197,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.809569+0000","last_change":"2024-09-20T18:49:51.969702+0000","last_active":"2024-09-20T18:54:18.809569+0000","last_peered":"2024-09-20T18:54:18.809569+0000","last_clean":"2024-09-20T18:54:18.809569+0000","last_became_active":"2024-09-20T18:49:51.969400+0000","last_became_peered":"2024-09-20T18:49:51.969400+0000","last_unstale":"2024-09-20T18:54:18.809569+0000","last_undegraded":"2024-09-20T18:54:18.809569+0000","last_fullsized":"2024-09-20T18:54:18.809569+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:36.285843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"157'15022","reported_seq":15229,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:21.207937+0000","last_change":"2024-09-20T18:53:23.208574+0000","last_active":"2024-09-20T18:54:21.207937+0000","last_peered":"2024-09-20T18:54:21.207937+0000","last_clean":"2024-09-20T18:54:21.207937+0000","last_became_active":"2024-09-20T18:50:49.843321+0000","last_became_peered":"2024-09-20T18:50:49.843321+0000","last_unstale":"2024-09-20T18:54:21.207937+0000","last_undegraded":"2024-09-20T18:54:21.207937+0000","last_fullsized":"2024-09-20T18:54:21.207937+0000","mapping_epoch":83,"log_start":"132'9400","ondisk_log_start":"132'9400","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"140'11334","last_scrub_stamp":"2024-09-20T18:53:23.208497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:53:23.208497+0000","objects_scrubbed":3959,"log_size":5622,"log_dups_size":0,"ondisk_log_size":5622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:48.313048+0000","scrub_duration":1357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14942208,"num_objects":228,"num_object_clones":0,"num_object_copies":684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15022,"num_write_kb":488000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":248,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.807623+0000","last_change":"2024-09-20T18:53:48.214566+0000","last_active":"2024-09-20T18:54:18.807623+0000","last_peered":"2024-09-20T18:54:18.807623+0000","last_clean":"2024-09-20T18:54:18.807623+0000","last_became_active":"2024-09-20T18:49:47.950904+0000","last_became_peered":"2024-09-20T18:49:47.950904+0000","last_unstale":"2024-09-20T18:54:18.807623+0000","last_undegraded":"2024-09-20T18:54:18.807623+0000","last_fullsized":"2024-09-20T18:54:18.807623+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-20T18:53:48.214491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:48.214491+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:15.858989+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":225,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.808374+0000","last_change":"2024-09-20T18:53:19.278005+0000","last_active":"2024-09-20T18:54:18.808374+0000","last_peered":"2024-09-20T18:54:18.808374+0000","last_clean":"2024-09-20T18:54:18.808374+0000","last_became_active":"2024-09-20T18:49:45.939884+0000","last_became_peered":"2024-09-20T18:49:45.939884+0000","last_unstale":"2024-09-20T18:54:18.808374+0000","last_undegraded":"2024-09-20T18:54:18.808374+0000","last_fullsized":"2024-09-20T18:54:18.808374+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:19.277650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:19.277650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:22.181893+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":227,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.801588+0000","last_change":"2024-09-20T18:52:07.718766+0000","last_active":"2024-09-20T18:54:18.801588+0000","last_peered":"2024-09-20T18:54:18.801588+0000","last_clean":"2024-09-20T18:54:18.801588+0000","last_became_active":"2024-09-20T18:49:44.937135+0000","last_became_peered":"2024-09-20T18:49:44.937135+0000","last_unstale":"2024-09-20T18:54:18.801588+0000","last_undegraded":"2024-09-20T18:54:18.801588+0000","last_fullsized":"2024-09-20T18:54:18.801588+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:52:07.718646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:52:07.718646+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:37.805116+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.00095558800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":210,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.802984+0000","last_change":"2024-09-20T18:51:59.796450+0000","last_active":"2024-09-20T18:54:18.802984+0000","last_peered":"2024-09-20T18:54:18.802984+0000","last_clean":"2024-09-20T18:54:18.802984+0000","last_became_active":"2024-09-20T18:49:49.954424+0000","last_became_peered":"2024-09-20T18:49:49.954424+0000","last_unstale":"2024-09-20T18:54:18.802984+0000","last_undegraded":"2024-09-20T18:54:18.802984+0000","last_fullsized":"2024-09-20T18:54:18.802984+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:59.796383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:59.796383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:06.819934+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":204,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.791367+0000","last_change":"2024-09-20T18:52:39.814324+0000","last_active":"2024-09-20T18:54:18.791367+0000","last_peered":"2024-09-20T18:54:18.791367+0000","last_clean":"2024-09-20T18:54:18.791367+0000","last_became_active":"2024-09-20T18:49:52.139789+0000","last_became_peered":"2024-09-20T18:49:52.139789+0000","last_unstale":"2024-09-20T18:54:18.791367+0000","last_undegraded":"2024-09-20T18:54:18.791367+0000","last_fullsized":"2024-09-20T18:54:18.791367+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:39.814259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:39.814259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:05.336625+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"157'15298","reported_seq":15579,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:20.760412+0000","last_change":"2024-09-20T18:53:24.693140+0000","last_active":"2024-09-20T18:54:20.760412+0000","last_peered":"2024-09-20T18:54:20.760412+0000","last_clean":"2024-09-20T18:54:20.760412+0000","last_became_active":"2024-09-20T18:50:49.846124+0000","last_became_peered":"2024-09-20T18:50:49.846124+0000","last_unstale":"2024-09-20T18:54:20.760412+0000","last_undegraded":"2024-09-20T18:54:20.760412+0000","last_fullsized":"2024-09-20T18:54:20.760412+0000","mapping_epoch":83,"log_start":"134'10100","ondisk_log_start":"134'10100","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"140'11593","last_scrub_stamp":"2024-09-20T18:53:24.692917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:53:24.692917+0000","objects_scrubbed":4004,"log_size":5198,"log_dups_size":0,"ondisk_log_size":5198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:35.585228+0000","scrub_duration":1122,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16646144,"num_objects":254,"num_object_clones":0,"num_object_copies":762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15298,"num_write_kb":497664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":246,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.800134+0000","last_change":"2024-09-20T18:53:47.786753+0000","last_active":"2024-09-20T18:54:18.800134+0000","last_peered":"2024-09-20T18:54:18.800134+0000","last_clean":"2024-09-20T18:54:18.800134+0000","last_became_active":"2024-09-20T18:49:47.949230+0000","last_became_peered":"2024-09-20T18:49:47.949230+0000","last_unstale":"2024-09-20T18:54:18.800134+0000","last_undegraded":"2024-09-20T18:54:18.800134+0000","last_fullsized":"2024-09-20T18:54:18.800134+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:53:47.786681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:47.786681+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:54.774464+0000","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":224,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.801892+0000","last_change":"2024-09-20T18:53:09.891235+0000","last_active":"2024-09-20T18:54:18.801892+0000","last_peered":"2024-09-20T18:54:18.801892+0000","last_clean":"2024-09-20T18:54:18.801892+0000","last_became_active":"2024-09-20T18:49:45.939197+0000","last_became_peered":"2024-09-20T18:49:45.939197+0000","last_unstale":"2024-09-20T18:54:18.801892+0000","last_undegraded":"2024-09-20T18:54:18.801892+0000","last_fullsized":"2024-09-20T18:54:18.801892+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:09.890987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:09.890987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":505,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:19.619397+0000","last_change":"2024-09-20T18:53:22.616236+0000","last_active":"2024-09-20T18:54:19.619397+0000","last_peered":"2024-09-20T18:54:19.619397+0000","last_clean":"2024-09-20T18:54:19.619397+0000","last_became_active":"2024-09-20T18:49:44.936819+0000","last_became_peered":"2024-09-20T18:49:44.936819+0000","last_unstale":"2024-09-20T18:54:19.619397+0000","last_undegraded":"2024-09-20T18:54:19.619397+0000","last_fullsized":"2024-09-20T18:54:19.619397+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-20T18:53:22.616166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:53:22.616166+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:25.872696+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0010996199999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":270,"num_read_kb":265,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":208,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.806000+0000","last_change":"2024-09-20T18:52:02.981845+0000","last_active":"2024-09-20T18:54:18.806000+0000","last_peered":"2024-09-20T18:54:18.806000+0000","last_clean":"2024-09-20T18:54:18.806000+0000","last_became_active":"2024-09-20T18:49:49.961066+0000","last_became_peered":"2024-09-20T18:49:49.961066+0000","last_unstale":"2024-09-20T18:54:18.806000+0000","last_undegraded":"2024-09-20T18:54:18.806000+0000","last_fullsized":"2024-09-20T18:54:18.806000+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:02.981762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:02.981762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:58.477818+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":213,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.807316+0000","last_change":"2024-09-20T18:53:49.227504+0000","last_active":"2024-09-20T18:54:18.807316+0000","last_peered":"2024-09-20T18:54:18.807316+0000","last_clean":"2024-09-20T18:54:18.807316+0000","last_became_active":"2024-09-20T18:49:51.970343+0000","last_became_peered":"2024-09-20T18:49:51.970343+0000","last_unstale":"2024-09-20T18:54:18.807316+0000","last_undegraded":"2024-09-20T18:54:18.807316+0000","last_fullsized":"2024-09-20T18:54:18.807316+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:49.227436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:49.227436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:07.422442+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"157'15177","reported_seq":15413,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:21.267476+0000","last_change":"2024-09-20T18:52:13.298242+0000","last_active":"2024-09-20T18:54:21.267476+0000","last_peered":"2024-09-20T18:54:21.267476+0000","last_clean":"2024-09-20T18:54:21.267476+0000","last_became_active":"2024-09-20T18:50:49.842310+0000","last_became_peered":"2024-09-20T18:50:49.842310+0000","last_unstale":"2024-09-20T18:54:21.267476+0000","last_undegraded":"2024-09-20T18:54:21.267476+0000","last_fullsized":"2024-09-20T18:54:21.267476+0000","mapping_epoch":83,"log_start":"132'9600","ondisk_log_start":"132'9600","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"116'6867","last_scrub_stamp":"2024-09-20T18:52:13.297990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:52:13.297990+0000","objects_scrubbed":6775,"log_size":5577,"log_dups_size":0,"ondisk_log_size":5577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:40.850770+0000","scrub_duration":3018,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14221312,"num_objects":217,"num_object_clones":0,"num_object_copies":651,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":217,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15177,"num_write_kb":492608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":281,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.803442+0000","last_change":"2024-09-20T18:49:48.498345+0000","last_active":"2024-09-20T18:54:18.803442+0000","last_peered":"2024-09-20T18:54:18.803442+0000","last_clean":"2024-09-20T18:54:18.803442+0000","last_became_active":"2024-09-20T18:49:48.497936+0000","last_became_peered":"2024-09-20T18:49:48.497936+0000","last_unstale":"2024-09-20T18:54:18.803442+0000","last_undegraded":"2024-09-20T18:54:18.803442+0000","last_fullsized":"2024-09-20T18:54:18.803442+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:59.631793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":225,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.802584+0000","last_change":"2024-09-20T18:53:13.145127+0000","last_active":"2024-09-20T18:54:18.802584+0000","last_peered":"2024-09-20T18:54:18.802584+0000","last_clean":"2024-09-20T18:54:18.802584+0000","last_became_active":"2024-09-20T18:49:45.941136+0000","last_became_peered":"2024-09-20T18:49:45.941136+0000","last_unstale":"2024-09-20T18:54:18.802584+0000","last_undegraded":"2024-09-20T18:54:18.802584+0000","last_fullsized":"2024-09-20T18:54:18.802584+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:13.144896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:13.144896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":263,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.791079+0000","last_change":"2024-09-20T18:52:11.887878+0000","last_active":"2024-09-20T18:54:18.791079+0000","last_peered":"2024-09-20T18:54:18.791079+0000","last_clean":"2024-09-20T18:54:18.791079+0000","last_became_active":"2024-09-20T18:49:36.545306+0000","last_became_peered":"2024-09-20T18:49:36.545306+0000","last_unstale":"2024-09-20T18:54:18.791079+0000","last_undegraded":"2024-09-20T18:54:18.791079+0000","last_fullsized":"2024-09-20T18:54:18.791079+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T18:52:11.887733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T18:52:11.887733+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:32.438028+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":208,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.791248+0000","last_change":"2024-09-20T18:51:22.852942+0000","last_active":"2024-09-20T18:54:18.791248+0000","last_peered":"2024-09-20T18:54:18.791248+0000","last_clean":"2024-09-20T18:54:18.791248+0000","last_became_active":"2024-09-20T18:49:49.956128+0000","last_became_peered":"2024-09-20T18:49:49.956128+0000","last_unstale":"2024-09-20T18:54:18.791248+0000","last_undegraded":"2024-09-20T18:54:18.791248+0000","last_fullsized":"2024-09-20T18:54:18.791248+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:22.852869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:22.852869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:37.516435+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":212,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.805921+0000","last_change":"2024-09-20T18:53:45.306565+0000","last_active":"2024-09-20T18:54:18.805921+0000","last_peered":"2024-09-20T18:54:18.805921+0000","last_clean":"2024-09-20T18:54:18.805921+0000","last_became_active":"2024-09-20T18:49:51.971332+0000","last_became_peered":"2024-09-20T18:49:51.971332+0000","last_unstale":"2024-09-20T18:54:18.805921+0000","last_undegraded":"2024-09-20T18:54:18.805921+0000","last_fullsized":"2024-09-20T18:54:18.805921+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:45.306514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:45.306514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:10.093423+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"157'14996","reported_seq":15309,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:20.818467+0000","last_change":"2024-09-20T18:54:10.006345+0000","last_active":"2024-09-20T18:54:20.818467+0000","last_peered":"2024-09-20T18:54:20.818467+0000","last_clean":"2024-09-20T18:54:20.818467+0000","last_became_active":"2024-09-20T18:50:49.842483+0000","last_became_peered":"2024-09-20T18:50:49.842483+0000","last_unstale":"2024-09-20T18:54:20.818467+0000","last_undegraded":"2024-09-20T18:54:20.818467+0000","last_fullsized":"2024-09-20T18:54:20.818467+0000","mapping_epoch":83,"log_start":"127'8700","ondisk_log_start":"127'8700","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'14301","last_scrub_stamp":"2024-09-20T18:54:10.006103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:54:10.006103+0000","objects_scrubbed":959,"log_size":6296,"log_dups_size":0,"ondisk_log_size":6296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:21.427623+0000","scrub_duration":279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16711704,"num_objects":256,"num_object_clones":0,"num_object_copies":768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14996,"num_write_kb":488001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":232,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.797587+0000","last_change":"2024-09-20T18:51:14.397996+0000","last_active":"2024-09-20T18:54:18.797587+0000","last_peered":"2024-09-20T18:54:18.797587+0000","last_clean":"2024-09-20T18:54:18.797587+0000","last_became_active":"2024-09-20T18:49:47.950073+0000","last_became_peered":"2024-09-20T18:49:47.950073+0000","last_unstale":"2024-09-20T18:54:18.797587+0000","last_undegraded":"2024-09-20T18:54:18.797587+0000","last_fullsized":"2024-09-20T18:54:18.797587+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:51:14.397930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.397930+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:32.825060+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":217,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.797795+0000","last_change":"2024-09-20T18:51:18.410601+0000","last_active":"2024-09-20T18:54:18.797795+0000","last_peered":"2024-09-20T18:54:18.797795+0000","last_clean":"2024-09-20T18:54:18.797795+0000","last_became_active":"2024-09-20T18:49:45.944579+0000","last_became_peered":"2024-09-20T18:49:45.944579+0000","last_unstale":"2024-09-20T18:54:18.797795+0000","last_undegraded":"2024-09-20T18:54:18.797795+0000","last_fullsized":"2024-09-20T18:54:18.797795+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:18.410447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:18.410447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:29.855147+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":223,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.805858+0000","last_change":"2024-09-20T18:54:11.401857+0000","last_active":"2024-09-20T18:54:18.805858+0000","last_peered":"2024-09-20T18:54:18.805858+0000","last_clean":"2024-09-20T18:54:18.805858+0000","last_became_active":"2024-09-20T18:49:49.954391+0000","last_became_peered":"2024-09-20T18:49:49.954391+0000","last_unstale":"2024-09-20T18:54:18.805858+0000","last_undegraded":"2024-09-20T18:54:18.805858+0000","last_fullsized":"2024-09-20T18:54:18.805858+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:11.401777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:11.401777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:15.740623+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":205,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.806828+0000","last_change":"2024-09-20T18:52:05.021228+0000","last_active":"2024-09-20T18:54:18.806828+0000","last_peered":"2024-09-20T18:54:18.806828+0000","last_clean":"2024-09-20T18:54:18.806828+0000","last_became_active":"2024-09-20T18:49:52.139959+0000","last_became_peered":"2024-09-20T18:49:52.139959+0000","last_unstale":"2024-09-20T18:54:18.806828+0000","last_undegraded":"2024-09-20T18:54:18.806828+0000","last_fullsized":"2024-09-20T18:54:18.806828+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:05.021071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.021071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:01.216837+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"157'15019","reported_seq":15229,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:21.051862+0000","last_change":"2024-09-20T18:53:22.451111+0000","last_active":"2024-09-20T18:54:21.051862+0000","last_peered":"2024-09-20T18:54:21.051862+0000","last_clean":"2024-09-20T18:54:21.051862+0000","last_became_active":"2024-09-20T18:50:49.841683+0000","last_became_peered":"2024-09-20T18:50:49.841683+0000","last_unstale":"2024-09-20T18:54:21.051862+0000","last_undegraded":"2024-09-20T18:54:21.051862+0000","last_fullsized":"2024-09-20T18:54:21.051862+0000","mapping_epoch":83,"log_start":"125'8200","ondisk_log_start":"125'8200","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'11259","last_scrub_stamp":"2024-09-20T18:53:22.450961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:53:22.450961+0000","objects_scrubbed":4058,"log_size":6819,"log_dups_size":0,"ondisk_log_size":6819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:26.478768+0000","scrub_duration":1718,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15663104,"num_objects":239,"num_object_clones":0,"num_object_copies":717,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15019,"num_write_kb":488256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":249,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.797761+0000","last_change":"2024-09-20T18:52:02.944536+0000","last_active":"2024-09-20T18:54:18.797761+0000","last_peered":"2024-09-20T18:54:18.797761+0000","last_clean":"2024-09-20T18:54:18.797761+0000","last_became_active":"2024-09-20T18:49:47.947636+0000","last_became_peered":"2024-09-20T18:49:47.947636+0000","last_unstale":"2024-09-20T18:54:18.797761+0000","last_undegraded":"2024-09-20T18:54:18.797761+0000","last_fullsized":"2024-09-20T18:54:18.797761+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-20T18:52:02.944434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:02.944434+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:30.835162+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":238,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.803328+0000","last_change":"2024-09-20T18:53:23.472809+0000","last_active":"2024-09-20T18:54:18.803328+0000","last_peered":"2024-09-20T18:54:18.803328+0000","last_clean":"2024-09-20T18:54:18.803328+0000","last_became_active":"2024-09-20T18:51:55.596689+0000","last_became_peered":"2024-09-20T18:51:55.596689+0000","last_unstale":"2024-09-20T18:54:18.803328+0000","last_undegraded":"2024-09-20T18:54:18.803328+0000","last_fullsized":"2024-09-20T18:54:18.803328+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:23.472757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:23.472757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:39.082278+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":217,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.807611+0000","last_change":"2024-09-20T18:54:11.377907+0000","last_active":"2024-09-20T18:54:18.807611+0000","last_peered":"2024-09-20T18:54:18.807611+0000","last_clean":"2024-09-20T18:54:18.807611+0000","last_became_active":"2024-09-20T18:49:49.959160+0000","last_became_peered":"2024-09-20T18:49:49.959160+0000","last_unstale":"2024-09-20T18:54:18.807611+0000","last_undegraded":"2024-09-20T18:54:18.807611+0000","last_fullsized":"2024-09-20T18:54:18.807611+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:11.376951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:11.376951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:35.227288+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":213,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.801541+0000","last_change":"2024-09-20T18:52:35.635897+0000","last_active":"2024-09-20T18:54:18.801541+0000","last_peered":"2024-09-20T18:54:18.801541+0000","last_clean":"2024-09-20T18:54:18.801541+0000","last_became_active":"2024-09-20T18:49:51.970108+0000","last_became_peered":"2024-09-20T18:49:51.970108+0000","last_unstale":"2024-09-20T18:54:18.801541+0000","last_undegraded":"2024-09-20T18:54:18.801541+0000","last_fullsized":"2024-09-20T18:54:18.801541+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:35.635818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:35.635818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:56.872031+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"157'15252","reported_seq":15536,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:21.252677+0000","last_change":"2024-09-20T18:54:16.253302+0000","last_active":"2024-09-20T18:54:21.252677+0000","last_peered":"2024-09-20T18:54:21.252677+0000","last_clean":"2024-09-20T18:54:21.252677+0000","last_became_active":"2024-09-20T18:50:49.841628+0000","last_became_peered":"2024-09-20T18:50:49.841628+0000","last_unstale":"2024-09-20T18:54:21.252677+0000","last_undegraded":"2024-09-20T18:54:21.252677+0000","last_fullsized":"2024-09-20T18:54:21.252677+0000","mapping_epoch":83,"log_start":"132'9600","ondisk_log_start":"132'9600","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"156'14931","last_scrub_stamp":"2024-09-20T18:54:16.252969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:54:16.252969+0000","objects_scrubbed":558,"log_size":5652,"log_dups_size":0,"ondisk_log_size":5652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:23.832787+0000","scrub_duration":322,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14680064,"num_objects":224,"num_object_clones":0,"num_object_copies":672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15252,"num_write_kb":495232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":236,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.797632+0000","last_change":"2024-09-20T18:54:17.219635+0000","last_active":"2024-09-20T18:54:18.797632+0000","last_peered":"2024-09-20T18:54:18.797632+0000","last_clean":"2024-09-20T18:54:18.797632+0000","last_became_active":"2024-09-20T18:49:47.947126+0000","last_became_peered":"2024-09-20T18:49:47.947126+0000","last_unstale":"2024-09-20T18:54:18.797632+0000","last_undegraded":"2024-09-20T18:54:18.797632+0000","last_fullsized":"2024-09-20T18:54:18.797632+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:54:17.219548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:17.219548+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:45.354404+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":225,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.800420+0000","last_change":"2024-09-20T18:52:46.005172+0000","last_active":"2024-09-20T18:54:18.800420+0000","last_peered":"2024-09-20T18:54:18.800420+0000","last_clean":"2024-09-20T18:54:18.800420+0000","last_became_active":"2024-09-20T18:49:45.937264+0000","last_became_peered":"2024-09-20T18:49:45.937264+0000","last_unstale":"2024-09-20T18:54:18.800420+0000","last_undegraded":"2024-09-20T18:54:18.800420+0000","last_fullsized":"2024-09-20T18:54:18.800420+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:46.005109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:46.005109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":282,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:21.046062+0000","last_change":"2024-09-20T18:52:40.647833+0000","last_active":"2024-09-20T18:54:21.046062+0000","last_peered":"2024-09-20T18:54:21.046062+0000","last_clean":"2024-09-20T18:54:21.046062+0000","last_became_active":"2024-09-20T18:49:49.957005+0000","last_became_peered":"2024-09-20T18:49:49.957005+0000","last_unstale":"2024-09-20T18:54:21.046062+0000","last_undegraded":"2024-09-20T18:54:21.046062+0000","last_fullsized":"2024-09-20T18:54:21.046062+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:52:40.647761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:40.647761+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:07.843426+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":204,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.797404+0000","last_change":"2024-09-20T18:52:10.024610+0000","last_active":"2024-09-20T18:54:18.797404+0000","last_peered":"2024-09-20T18:54:18.797404+0000","last_clean":"2024-09-20T18:54:18.797404+0000","last_became_active":"2024-09-20T18:49:51.969355+0000","last_became_peered":"2024-09-20T18:49:51.969355+0000","last_unstale":"2024-09-20T18:54:18.797404+0000","last_undegraded":"2024-09-20T18:54:18.797404+0000","last_fullsized":"2024-09-20T18:54:18.797404+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:10.024557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:10.024557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:38.850775+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.8","version":"157'15197","reported_seq":15435,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:20.996550+0000","last_change":"2024-09-20T18:52:16.931599+0000","last_active":"2024-09-20T18:54:20.996550+0000","last_peered":"2024-09-20T18:54:20.996550+0000","last_clean":"2024-09-20T18:54:20.996550+0000","last_became_active":"2024-09-20T18:50:49.844028+0000","last_became_peered":"2024-09-20T18:50:49.844028+0000","last_unstale":"2024-09-20T18:54:20.996550+0000","last_undegraded":"2024-09-20T18:54:20.996550+0000","last_fullsized":"2024-09-20T18:54:20.996550+0000","mapping_epoch":83,"log_start":"126'8280","ondisk_log_start":"126'8280","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"118'7065","last_scrub_stamp":"2024-09-20T18:52:16.931418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:52:16.931418+0000","objects_scrubbed":7008,"log_size":6917,"log_dups_size":0,"ondisk_log_size":6917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:31.738778+0000","scrub_duration":2274,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15269888,"num_objects":233,"num_object_clones":0,"num_object_copies":699,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15197,"num_write_kb":493760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":236,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.795934+0000","last_change":"2024-09-20T18:53:17.863709+0000","last_active":"2024-09-20T18:54:18.795934+0000","last_peered":"2024-09-20T18:54:18.795934+0000","last_clean":"2024-09-20T18:54:18.795934+0000","last_became_active":"2024-09-20T18:49:47.946008+0000","last_became_peered":"2024-09-20T18:49:47.946008+0000","last_unstale":"2024-09-20T18:54:18.795934+0000","last_undegraded":"2024-09-20T18:54:18.795934+0000","last_fullsized":"2024-09-20T18:54:18.795934+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:53:17.863623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:17.863623+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:35.672196+0000","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":224,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.805582+0000","last_change":"2024-09-20T18:52:17.120051+0000","last_active":"2024-09-20T18:54:18.805582+0000","last_peered":"2024-09-20T18:54:18.805582+0000","last_clean":"2024-09-20T18:54:18.805582+0000","last_became_active":"2024-09-20T18:49:45.941406+0000","last_became_peered":"2024-09-20T18:49:45.941406+0000","last_unstale":"2024-09-20T18:54:18.805582+0000","last_undegraded":"2024-09-20T18:54:18.805582+0000","last_fullsized":"2024-09-20T18:54:18.805582+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:17.119986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:17.119986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:55.385828+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":218,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.801969+0000","last_change":"2024-09-20T18:53:27.722068+0000","last_active":"2024-09-20T18:54:18.801969+0000","last_peered":"2024-09-20T18:54:18.801969+0000","last_clean":"2024-09-20T18:54:18.801969+0000","last_became_active":"2024-09-20T18:49:49.956767+0000","last_became_peered":"2024-09-20T18:49:49.956767+0000","last_unstale":"2024-09-20T18:54:18.801969+0000","last_undegraded":"2024-09-20T18:54:18.801969+0000","last_fullsized":"2024-09-20T18:54:18.801969+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:27.721940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:27.721940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:42.447352+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":204,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.801806+0000","last_change":"2024-09-20T18:52:46.004013+0000","last_active":"2024-09-20T18:54:18.801806+0000","last_peered":"2024-09-20T18:54:18.801806+0000","last_clean":"2024-09-20T18:54:18.801806+0000","last_became_active":"2024-09-20T18:49:51.964098+0000","last_became_peered":"2024-09-20T18:49:51.964098+0000","last_unstale":"2024-09-20T18:54:18.801806+0000","last_undegraded":"2024-09-20T18:54:18.801806+0000","last_fullsized":"2024-09-20T18:54:18.801806+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:46.003836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:46.003836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"157'15027","reported_seq":15296,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:21.277691+0000","last_change":"2024-09-20T18:53:43.753759+0000","last_active":"2024-09-20T18:54:21.277691+0000","last_peered":"2024-09-20T18:54:21.277691+0000","last_clean":"2024-09-20T18:54:21.277691+0000","last_became_active":"2024-09-20T18:50:49.840023+0000","last_became_peered":"2024-09-20T18:50:49.840023+0000","last_unstale":"2024-09-20T18:54:21.277691+0000","last_undegraded":"2024-09-20T18:54:21.277691+0000","last_fullsized":"2024-09-20T18:54:21.277691+0000","mapping_epoch":83,"log_start":"132'9400","ondisk_log_start":"132'9400","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'12545","last_scrub_stamp":"2024-09-20T18:53:43.753508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:53:43.753508+0000","objects_scrubbed":2735,"log_size":5627,"log_dups_size":0,"ondisk_log_size":5627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:44.964709+0000","scrub_duration":892,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15269888,"num_objects":233,"num_object_clones":0,"num_object_copies":699,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15027,"num_write_kb":488320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":251,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.803086+0000","last_change":"2024-09-20T18:53:24.458093+0000","last_active":"2024-09-20T18:54:18.803086+0000","last_peered":"2024-09-20T18:54:18.803086+0000","last_clean":"2024-09-20T18:54:18.803086+0000","last_became_active":"2024-09-20T18:49:48.498625+0000","last_became_peered":"2024-09-20T18:49:48.498625+0000","last_unstale":"2024-09-20T18:54:18.803086+0000","last_undegraded":"2024-09-20T18:54:18.803086+0000","last_fullsized":"2024-09-20T18:54:18.803086+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-20T18:53:24.458005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:24.458005+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:31.316232+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":224,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.791691+0000","last_change":"2024-09-20T18:53:20.564464+0000","last_active":"2024-09-20T18:54:18.791691+0000","last_peered":"2024-09-20T18:54:18.791691+0000","last_clean":"2024-09-20T18:54:18.791691+0000","last_became_active":"2024-09-20T18:49:45.941448+0000","last_became_peered":"2024-09-20T18:49:45.941448+0000","last_unstale":"2024-09-20T18:54:18.791691+0000","last_undegraded":"2024-09-20T18:54:18.791691+0000","last_fullsized":"2024-09-20T18:54:18.791691+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:20.564413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:20.564413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:38.260460+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":218,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.802680+0000","last_change":"2024-09-20T18:52:36.662703+0000","last_active":"2024-09-20T18:54:18.802680+0000","last_peered":"2024-09-20T18:54:18.802680+0000","last_clean":"2024-09-20T18:54:18.802680+0000","last_became_active":"2024-09-20T18:49:49.957558+0000","last_became_peered":"2024-09-20T18:49:49.957558+0000","last_unstale":"2024-09-20T18:54:18.802680+0000","last_undegraded":"2024-09-20T18:54:18.802680+0000","last_fullsized":"2024-09-20T18:54:18.802680+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:36.662524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:36.662524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:58.801684+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":220,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.791021+0000","last_change":"2024-09-20T18:53:47.370152+0000","last_active":"2024-09-20T18:54:18.791021+0000","last_peered":"2024-09-20T18:54:18.791021+0000","last_clean":"2024-09-20T18:54:18.791021+0000","last_became_active":"2024-09-20T18:49:51.970798+0000","last_became_peered":"2024-09-20T18:49:51.970798+0000","last_unstale":"2024-09-20T18:54:18.791021+0000","last_undegraded":"2024-09-20T18:54:18.791021+0000","last_fullsized":"2024-09-20T18:54:18.791021+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:47.370108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:47.370108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:53.917763+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.a","version":"157'15088","reported_seq":15248,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:21.235750+0000","last_change":"2024-09-20T18:50:49.842445+0000","last_active":"2024-09-20T18:54:21.235750+0000","last_peered":"2024-09-20T18:54:21.235750+0000","last_clean":"2024-09-20T18:54:21.235750+0000","last_became_active":"2024-09-20T18:50:49.842179+0000","last_became_peered":"2024-09-20T18:50:49.842179+0000","last_unstale":"2024-09-20T18:54:21.235750+0000","last_undegraded":"2024-09-20T18:54:21.235750+0000","last_fullsized":"2024-09-20T18:54:21.235750+0000","mapping_epoch":83,"log_start":"136'10400","ondisk_log_start":"136'10400","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:50:48.819660+0000","objects_scrubbed":0,"log_size":4688,"log_dups_size":0,"ondisk_log_size":4688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:29.169615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14417920,"num_objects":220,"num_object_clones":0,"num_object_copies":660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15088,"num_write_kb":489856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":244,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.797303+0000","last_change":"2024-09-20T18:53:52.898818+0000","last_active":"2024-09-20T18:54:18.797303+0000","last_peered":"2024-09-20T18:54:18.797303+0000","last_clean":"2024-09-20T18:54:18.797303+0000","last_became_active":"2024-09-20T18:49:47.949116+0000","last_became_peered":"2024-09-20T18:49:47.949116+0000","last_unstale":"2024-09-20T18:54:18.797303+0000","last_undegraded":"2024-09-20T18:54:18.797303+0000","last_fullsized":"2024-09-20T18:54:18.797303+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:53:52.898760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:52.898760+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:53.100006+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":224,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.791776+0000","last_change":"2024-09-20T18:53:21.578905+0000","last_active":"2024-09-20T18:54:18.791776+0000","last_peered":"2024-09-20T18:54:18.791776+0000","last_clean":"2024-09-20T18:54:18.791776+0000","last_became_active":"2024-09-20T18:49:45.944858+0000","last_became_peered":"2024-09-20T18:49:45.944858+0000","last_unstale":"2024-09-20T18:54:18.791776+0000","last_undegraded":"2024-09-20T18:54:18.791776+0000","last_fullsized":"2024-09-20T18:54:18.791776+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:21.578845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:21.578845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:47.899680+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":210,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.805304+0000","last_change":"2024-09-20T18:52:12.674502+0000","last_active":"2024-09-20T18:54:18.805304+0000","last_peered":"2024-09-20T18:54:18.805304+0000","last_clean":"2024-09-20T18:54:18.805304+0000","last_became_active":"2024-09-20T18:49:49.954944+0000","last_became_peered":"2024-09-20T18:49:49.954944+0000","last_unstale":"2024-09-20T18:54:18.805304+0000","last_undegraded":"2024-09-20T18:54:18.805304+0000","last_fullsized":"2024-09-20T18:54:18.805304+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:12.674203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:12.674203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:41.868905+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":205,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.800158+0000","last_change":"2024-09-20T18:52:09.017828+0000","last_active":"2024-09-20T18:54:18.800158+0000","last_peered":"2024-09-20T18:54:18.800158+0000","last_clean":"2024-09-20T18:54:18.800158+0000","last_became_active":"2024-09-20T18:49:51.972970+0000","last_became_peered":"2024-09-20T18:49:51.972970+0000","last_unstale":"2024-09-20T18:54:18.800158+0000","last_undegraded":"2024-09-20T18:54:18.800158+0000","last_fullsized":"2024-09-20T18:54:18.800158+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:09.017759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:09.017759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:02.178343+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"157'15047","reported_seq":15285,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:20.970867+0000","last_change":"2024-09-20T18:52:15.505711+0000","last_active":"2024-09-20T18:54:20.970867+0000","last_peered":"2024-09-20T18:54:20.970867+0000","last_clean":"2024-09-20T18:54:20.970867+0000","last_became_active":"2024-09-20T18:50:49.843516+0000","last_became_peered":"2024-09-20T18:50:49.843516+0000","last_unstale":"2024-09-20T18:54:20.970867+0000","last_undegraded":"2024-09-20T18:54:20.970867+0000","last_fullsized":"2024-09-20T18:54:20.970867+0000","mapping_epoch":83,"log_start":"134'10000","ondisk_log_start":"134'10000","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'6958","last_scrub_stamp":"2024-09-20T18:52:15.505628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:50:48.819660+0000","last_clean_scrub_stamp":"2024-09-20T18:52:15.505628+0000","objects_scrubbed":6900,"log_size":5047,"log_dups_size":0,"ondisk_log_size":5047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:29.307707+0000","scrub_duration":2196,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14090240,"num_objects":215,"num_object_clones":0,"num_object_copies":645,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":215,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15047,"num_write_kb":488384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":245,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.805210+0000","last_change":"2024-09-20T18:54:12.373389+0000","last_active":"2024-09-20T18:54:18.805210+0000","last_peered":"2024-09-20T18:54:18.805210+0000","last_clean":"2024-09-20T18:54:18.805210+0000","last_became_active":"2024-09-20T18:49:48.499847+0000","last_became_peered":"2024-09-20T18:49:48.499847+0000","last_unstale":"2024-09-20T18:54:18.805210+0000","last_undegraded":"2024-09-20T18:54:18.805210+0000","last_fullsized":"2024-09-20T18:54:18.805210+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:54:12.373309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:12.373309+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:26.708173+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":244,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:19.403873+0000","last_change":"2024-09-20T18:54:19.403828+0000","last_active":"2024-09-20T18:54:19.403873+0000","last_peered":"2024-09-20T18:54:19.403873+0000","last_clean":"2024-09-20T18:54:19.403873+0000","last_became_active":"2024-09-20T18:49:45.944256+0000","last_became_peered":"2024-09-20T18:49:45.944256+0000","last_unstale":"2024-09-20T18:54:19.403873+0000","last_undegraded":"2024-09-20T18:54:19.403873+0000","last_fullsized":"2024-09-20T18:54:19.403873+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-20T18:54:19.403731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:19.403731+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:21.004220+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":271,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.791080+0000","last_change":"2024-09-20T18:52:05.788988+0000","last_active":"2024-09-20T18:54:18.791080+0000","last_peered":"2024-09-20T18:54:18.791080+0000","last_clean":"2024-09-20T18:54:18.791080+0000","last_became_active":"2024-09-20T18:49:49.958411+0000","last_became_peered":"2024-09-20T18:49:49.958411+0000","last_unstale":"2024-09-20T18:54:18.791080+0000","last_undegraded":"2024-09-20T18:54:18.791080+0000","last_fullsized":"2024-09-20T18:54:18.791080+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:52:05.788924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.788924+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:39.468023+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":204,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.805341+0000","last_change":"2024-09-20T18:51:56.970415+0000","last_active":"2024-09-20T18:54:18.805341+0000","last_peered":"2024-09-20T18:54:18.805341+0000","last_clean":"2024-09-20T18:54:18.805341+0000","last_became_active":"2024-09-20T18:49:52.139685+0000","last_became_peered":"2024-09-20T18:49:52.139685+0000","last_unstale":"2024-09-20T18:54:18.805341+0000","last_undegraded":"2024-09-20T18:54:18.805341+0000","last_fullsized":"2024-09-20T18:54:18.805341+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:56.970349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:56.970349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:08.388529+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":231,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.803070+0000","last_change":"2024-09-20T18:54:18.763523+0000","last_active":"2024-09-20T18:54:18.803070+0000","last_peered":"2024-09-20T18:54:18.803070+0000","last_clean":"2024-09-20T18:54:18.803070+0000","last_became_active":"2024-09-20T18:49:45.940060+0000","last_became_peered":"2024-09-20T18:49:45.940060+0000","last_unstale":"2024-09-20T18:54:18.803070+0000","last_undegraded":"2024-09-20T18:54:18.803070+0000","last_fullsized":"2024-09-20T18:54:18.803070+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:18.763474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:18.763474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:24.982459+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":221,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.802413+0000","last_change":"2024-09-20T18:53:27.186982+0000","last_active":"2024-09-20T18:54:18.802413+0000","last_peered":"2024-09-20T18:54:18.802413+0000","last_clean":"2024-09-20T18:54:18.802413+0000","last_became_active":"2024-09-20T18:49:48.500662+0000","last_became_peered":"2024-09-20T18:49:48.500662+0000","last_unstale":"2024-09-20T18:54:18.802413+0000","last_undegraded":"2024-09-20T18:54:18.802413+0000","last_fullsized":"2024-09-20T18:54:18.802413+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-20T18:53:27.186631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:27.186631+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:42.463558+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":215,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.807399+0000","last_change":"2024-09-20T18:53:16.220554+0000","last_active":"2024-09-20T18:54:18.807399+0000","last_peered":"2024-09-20T18:54:18.807399+0000","last_clean":"2024-09-20T18:54:18.807399+0000","last_became_active":"2024-09-20T18:49:49.956749+0000","last_became_peered":"2024-09-20T18:49:49.956749+0000","last_unstale":"2024-09-20T18:54:18.807399+0000","last_undegraded":"2024-09-20T18:54:18.807399+0000","last_fullsized":"2024-09-20T18:54:18.807399+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:16.220490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:16.220490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:36.759843+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":211,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.791334+0000","last_change":"2024-09-20T18:53:19.591395+0000","last_active":"2024-09-20T18:54:18.791334+0000","last_peered":"2024-09-20T18:54:18.791334+0000","last_clean":"2024-09-20T18:54:18.791334+0000","last_became_active":"2024-09-20T18:49:52.142291+0000","last_became_peered":"2024-09-20T18:49:52.142291+0000","last_unstale":"2024-09-20T18:54:18.791334+0000","last_undegraded":"2024-09-20T18:54:18.791334+0000","last_fullsized":"2024-09-20T18:54:18.791334+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:19.591346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:19.591346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:40.474867+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":227,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.800230+0000","last_change":"2024-09-20T18:51:23.065382+0000","last_active":"2024-09-20T18:54:18.800230+0000","last_peered":"2024-09-20T18:54:18.800230+0000","last_clean":"2024-09-20T18:54:18.800230+0000","last_became_active":"2024-09-20T18:49:48.500803+0000","last_became_peered":"2024-09-20T18:49:48.500803+0000","last_unstale":"2024-09-20T18:54:18.800230+0000","last_undegraded":"2024-09-20T18:54:18.800230+0000","last_fullsized":"2024-09-20T18:54:18.800230+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:51:23.064536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:23.064536+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:39.863201+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":215,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.791074+0000","last_change":"2024-09-20T18:51:30.982535+0000","last_active":"2024-09-20T18:54:18.791074+0000","last_peered":"2024-09-20T18:54:18.791074+0000","last_clean":"2024-09-20T18:54:18.791074+0000","last_became_active":"2024-09-20T18:49:45.946600+0000","last_became_peered":"2024-09-20T18:49:45.946600+0000","last_unstale":"2024-09-20T18:54:18.791074+0000","last_undegraded":"2024-09-20T18:54:18.791074+0000","last_fullsized":"2024-09-20T18:54:18.791074+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:30.982454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:30.982454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:04.374723+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":216,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.802278+0000","last_change":"2024-09-20T18:53:50.253721+0000","last_active":"2024-09-20T18:54:18.802278+0000","last_peered":"2024-09-20T18:54:18.802278+0000","last_clean":"2024-09-20T18:54:18.802278+0000","last_became_active":"2024-09-20T18:49:49.961947+0000","last_became_peered":"2024-09-20T18:49:49.961947+0000","last_unstale":"2024-09-20T18:54:18.802278+0000","last_undegraded":"2024-09-20T18:54:18.802278+0000","last_fullsized":"2024-09-20T18:54:18.802278+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:50.253660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:50.253660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:15.284040+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"66'1","reported_seq":222,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:20.681878+0000","last_change":"2024-09-20T18:54:20.681878+0000","last_active":"2024-09-20T18:54:20.681878+0000","last_peered":"2024-09-20T18:54:20.681878+0000","last_clean":"2024-09-20T18:54:20.681878+0000","last_became_active":"2024-09-20T18:49:51.965121+0000","last_became_peered":"2024-09-20T18:49:51.965121+0000","last_unstale":"2024-09-20T18:54:20.681878+0000","last_undegraded":"2024-09-20T18:54:20.681878+0000","last_fullsized":"2024-09-20T18:54:20.681878+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T18:54:20.681790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:20.681790+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:31.186086+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":219,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.805796+0000","last_change":"2024-09-20T18:49:48.501359+0000","last_active":"2024-09-20T18:54:18.805796+0000","last_peered":"2024-09-20T18:54:18.805796+0000","last_clean":"2024-09-20T18:54:18.805796+0000","last_became_active":"2024-09-20T18:49:48.501011+0000","last_became_peered":"2024-09-20T18:49:48.501011+0000","last_unstale":"2024-09-20T18:54:18.805796+0000","last_undegraded":"2024-09-20T18:54:18.805796+0000","last_fullsized":"2024-09-20T18:54:18.805796+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:05.423302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":223,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.803141+0000","last_change":"2024-09-20T18:54:11.896547+0000","last_active":"2024-09-20T18:54:18.803141+0000","last_peered":"2024-09-20T18:54:18.803141+0000","last_clean":"2024-09-20T18:54:18.803141+0000","last_became_active":"2024-09-20T18:49:45.938511+0000","last_became_peered":"2024-09-20T18:49:45.938511+0000","last_unstale":"2024-09-20T18:54:18.803141+0000","last_undegraded":"2024-09-20T18:54:18.803141+0000","last_fullsized":"2024-09-20T18:54:18.803141+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:11.896470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:11.896470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:22.211500+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":280,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.806970+0000","last_change":"2024-09-20T18:53:47.337912+0000","last_active":"2024-09-20T18:54:18.806970+0000","last_peered":"2024-09-20T18:54:18.806970+0000","last_clean":"2024-09-20T18:54:18.806970+0000","last_became_active":"2024-09-20T18:49:49.960550+0000","last_became_peered":"2024-09-20T18:49:49.960550+0000","last_unstale":"2024-09-20T18:54:18.806970+0000","last_undegraded":"2024-09-20T18:54:18.806970+0000","last_fullsized":"2024-09-20T18:54:18.806970+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:53:47.337658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:47.337658+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:01.412695+0000","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":212,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.797806+0000","last_change":"2024-09-20T18:53:48.839130+0000","last_active":"2024-09-20T18:54:18.797806+0000","last_peered":"2024-09-20T18:54:18.797806+0000","last_clean":"2024-09-20T18:54:18.797806+0000","last_became_active":"2024-09-20T18:49:51.966254+0000","last_became_peered":"2024-09-20T18:49:51.966254+0000","last_unstale":"2024-09-20T18:54:18.797806+0000","last_undegraded":"2024-09-20T18:54:18.797806+0000","last_fullsized":"2024-09-20T18:54:18.797806+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:48.839015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:48.839015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:07.481453+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":246,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.807539+0000","last_change":"2024-09-20T18:51:15.223913+0000","last_active":"2024-09-20T18:54:18.807539+0000","last_peered":"2024-09-20T18:54:18.807539+0000","last_clean":"2024-09-20T18:54:18.807539+0000","last_became_active":"2024-09-20T18:49:58.997494+0000","last_became_peered":"2024-09-20T18:49:58.997494+0000","last_unstale":"2024-09-20T18:54:18.807539+0000","last_undegraded":"2024-09-20T18:54:18.807539+0000","last_fullsized":"2024-09-20T18:54:18.807539+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:51:15.223826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:15.223826+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:07.187139+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":223,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.791286+0000","last_change":"2024-09-20T18:53:27.589481+0000","last_active":"2024-09-20T18:54:18.791286+0000","last_peered":"2024-09-20T18:54:18.791286+0000","last_clean":"2024-09-20T18:54:18.791286+0000","last_became_active":"2024-09-20T18:49:45.949492+0000","last_became_peered":"2024-09-20T18:49:45.949492+0000","last_unstale":"2024-09-20T18:54:18.791286+0000","last_undegraded":"2024-09-20T18:54:18.791286+0000","last_fullsized":"2024-09-20T18:54:18.791286+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:27.589413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:27.589413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:48.562654+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":278,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.805699+0000","last_change":"2024-09-20T18:54:17.339848+0000","last_active":"2024-09-20T18:54:18.805699+0000","last_peered":"2024-09-20T18:54:18.805699+0000","last_clean":"2024-09-20T18:54:18.805699+0000","last_became_active":"2024-09-20T18:49:49.962243+0000","last_became_peered":"2024-09-20T18:49:49.962243+0000","last_unstale":"2024-09-20T18:54:18.805699+0000","last_undegraded":"2024-09-20T18:54:18.805699+0000","last_fullsized":"2024-09-20T18:54:18.805699+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:54:17.339761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:17.339761+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:22.040146+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":211,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.799198+0000","last_change":"2024-09-20T18:53:13.863129+0000","last_active":"2024-09-20T18:54:18.799198+0000","last_peered":"2024-09-20T18:54:18.799198+0000","last_clean":"2024-09-20T18:54:18.799198+0000","last_became_active":"2024-09-20T18:49:51.969093+0000","last_became_peered":"2024-09-20T18:49:51.969093+0000","last_unstale":"2024-09-20T18:54:18.799198+0000","last_undegraded":"2024-09-20T18:54:18.799198+0000","last_fullsized":"2024-09-20T18:54:18.799198+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:13.863061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:13.863061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:37.505050+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":239,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.797266+0000","last_change":"2024-09-20T18:53:43.879001+0000","last_active":"2024-09-20T18:54:18.797266+0000","last_peered":"2024-09-20T18:54:18.797266+0000","last_clean":"2024-09-20T18:54:18.797266+0000","last_became_active":"2024-09-20T18:49:47.949703+0000","last_became_peered":"2024-09-20T18:49:47.949703+0000","last_unstale":"2024-09-20T18:54:18.797266+0000","last_undegraded":"2024-09-20T18:54:18.797266+0000","last_fullsized":"2024-09-20T18:54:18.797266+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:53:43.878921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:43.878921+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:02.026666+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":216,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.797792+0000","last_change":"2024-09-20T18:52:05.985622+0000","last_active":"2024-09-20T18:54:18.797792+0000","last_peered":"2024-09-20T18:54:18.797792+0000","last_clean":"2024-09-20T18:54:18.797792+0000","last_became_active":"2024-09-20T18:49:45.939949+0000","last_became_peered":"2024-09-20T18:49:45.939949+0000","last_unstale":"2024-09-20T18:54:18.797792+0000","last_undegraded":"2024-09-20T18:54:18.797792+0000","last_fullsized":"2024-09-20T18:54:18.797792+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:05.985551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.985551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:36.823731+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":208,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.812580+0000","last_change":"2024-09-20T18:51:14.271954+0000","last_active":"2024-09-20T18:54:18.812580+0000","last_peered":"2024-09-20T18:54:18.812580+0000","last_clean":"2024-09-20T18:54:18.812580+0000","last_became_active":"2024-09-20T18:49:49.957067+0000","last_became_peered":"2024-09-20T18:49:49.957067+0000","last_unstale":"2024-09-20T18:54:18.812580+0000","last_undegraded":"2024-09-20T18:54:18.812580+0000","last_fullsized":"2024-09-20T18:54:18.812580+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:14.271719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.271719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:31.305762+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":204,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.803026+0000","last_change":"2024-09-20T18:51:14.210438+0000","last_active":"2024-09-20T18:54:18.803026+0000","last_peered":"2024-09-20T18:54:18.803026+0000","last_clean":"2024-09-20T18:54:18.803026+0000","last_became_active":"2024-09-20T18:49:51.966915+0000","last_became_peered":"2024-09-20T18:49:51.966915+0000","last_unstale":"2024-09-20T18:54:18.803026+0000","last_undegraded":"2024-09-20T18:54:18.803026+0000","last_fullsized":"2024-09-20T18:54:18.803026+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:14.210358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.210358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:04.198046+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":244,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.812766+0000","last_change":"2024-09-20T18:53:53.264294+0000","last_active":"2024-09-20T18:54:18.812766+0000","last_peered":"2024-09-20T18:54:18.812766+0000","last_clean":"2024-09-20T18:54:18.812766+0000","last_became_active":"2024-09-20T18:49:47.953310+0000","last_became_peered":"2024-09-20T18:49:47.953310+0000","last_unstale":"2024-09-20T18:54:18.812766+0000","last_undegraded":"2024-09-20T18:54:18.812766+0000","last_fullsized":"2024-09-20T18:54:18.812766+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:53:53.263976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:53.263976+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:04.916734+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":215,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.791559+0000","last_change":"2024-09-20T18:52:34.834333+0000","last_active":"2024-09-20T18:54:18.791559+0000","last_peered":"2024-09-20T18:54:18.791559+0000","last_clean":"2024-09-20T18:54:18.791559+0000","last_became_active":"2024-09-20T18:49:45.944357+0000","last_became_peered":"2024-09-20T18:49:45.944357+0000","last_unstale":"2024-09-20T18:54:18.791559+0000","last_undegraded":"2024-09-20T18:54:18.791559+0000","last_fullsized":"2024-09-20T18:54:18.791559+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:34.834273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:34.834273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:42.456584+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":216,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.800201+0000","last_change":"2024-09-20T18:54:18.108239+0000","last_active":"2024-09-20T18:54:18.800201+0000","last_peered":"2024-09-20T18:54:18.800201+0000","last_clean":"2024-09-20T18:54:18.800201+0000","last_became_active":"2024-09-20T18:49:49.962523+0000","last_became_peered":"2024-09-20T18:49:49.962523+0000","last_unstale":"2024-09-20T18:54:18.800201+0000","last_undegraded":"2024-09-20T18:54:18.800201+0000","last_fullsized":"2024-09-20T18:54:18.800201+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:18.108175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:18.108175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:33.189789+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":211,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.794091+0000","last_change":"2024-09-20T18:52:38.137485+0000","last_active":"2024-09-20T18:54:18.794091+0000","last_peered":"2024-09-20T18:54:18.794091+0000","last_clean":"2024-09-20T18:54:18.794091+0000","last_became_active":"2024-09-20T18:49:51.967313+0000","last_became_peered":"2024-09-20T18:49:51.967313+0000","last_unstale":"2024-09-20T18:54:18.794091+0000","last_undegraded":"2024-09-20T18:54:18.794091+0000","last_fullsized":"2024-09-20T18:54:18.794091+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:38.137431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:38.137431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":239,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:21.106745+0000","last_change":"2024-09-20T18:54:21.106745+0000","last_active":"2024-09-20T18:54:21.106745+0000","last_peered":"2024-09-20T18:54:21.106745+0000","last_clean":"2024-09-20T18:54:21.106745+0000","last_became_active":"2024-09-20T18:49:48.499111+0000","last_became_peered":"2024-09-20T18:49:48.499111+0000","last_unstale":"2024-09-20T18:54:21.106745+0000","last_undegraded":"2024-09-20T18:54:21.106745+0000","last_fullsized":"2024-09-20T18:54:21.106745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:54:21.106677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:21.106677+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:38.509931+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":223,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.807405+0000","last_change":"2024-09-20T18:53:48.301845+0000","last_active":"2024-09-20T18:54:18.807405+0000","last_peered":"2024-09-20T18:54:18.807405+0000","last_clean":"2024-09-20T18:54:18.807405+0000","last_became_active":"2024-09-20T18:49:45.942105+0000","last_became_peered":"2024-09-20T18:49:45.942105+0000","last_unstale":"2024-09-20T18:54:18.807405+0000","last_undegraded":"2024-09-20T18:54:18.807405+0000","last_fullsized":"2024-09-20T18:54:18.807405+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:48.301801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:48.301801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:06.232022+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":223,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.790681+0000","last_change":"2024-09-20T18:53:46.329643+0000","last_active":"2024-09-20T18:54:18.790681+0000","last_peered":"2024-09-20T18:54:18.790681+0000","last_clean":"2024-09-20T18:54:18.790681+0000","last_became_active":"2024-09-20T18:49:49.958159+0000","last_became_peered":"2024-09-20T18:49:49.958159+0000","last_unstale":"2024-09-20T18:54:18.790681+0000","last_undegraded":"2024-09-20T18:54:18.790681+0000","last_fullsized":"2024-09-20T18:54:18.790681+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:46.329547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:46.329547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:09.181263+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":204,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.800334+0000","last_change":"2024-09-20T18:52:43.118886+0000","last_active":"2024-09-20T18:54:18.800334+0000","last_peered":"2024-09-20T18:54:18.800334+0000","last_clean":"2024-09-20T18:54:18.800334+0000","last_became_active":"2024-09-20T18:49:52.139978+0000","last_became_peered":"2024-09-20T18:49:52.139978+0000","last_unstale":"2024-09-20T18:54:18.800334+0000","last_undegraded":"2024-09-20T18:54:18.800334+0000","last_fullsized":"2024-09-20T18:54:18.800334+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:43.118805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:43.118805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":226,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.813326+0000","last_change":"2024-09-20T18:53:43.253204+0000","last_active":"2024-09-20T18:54:18.813326+0000","last_peered":"2024-09-20T18:54:18.813326+0000","last_clean":"2024-09-20T18:54:18.813326+0000","last_became_active":"2024-09-20T18:49:47.954777+0000","last_became_peered":"2024-09-20T18:49:47.954777+0000","last_unstale":"2024-09-20T18:54:18.813326+0000","last_undegraded":"2024-09-20T18:54:18.813326+0000","last_fullsized":"2024-09-20T18:54:18.813326+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-20T18:53:43.253132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:43.253132+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:00.654449+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":223,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.800424+0000","last_change":"2024-09-20T18:53:16.921320+0000","last_active":"2024-09-20T18:54:18.800424+0000","last_peered":"2024-09-20T18:54:18.800424+0000","last_clean":"2024-09-20T18:54:18.800424+0000","last_became_active":"2024-09-20T18:49:45.940988+0000","last_became_peered":"2024-09-20T18:49:45.940988+0000","last_unstale":"2024-09-20T18:54:18.800424+0000","last_undegraded":"2024-09-20T18:54:18.800424+0000","last_fullsized":"2024-09-20T18:54:18.800424+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:16.921263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:16.921263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:35.775476+0000","scrub_duration":137,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":223,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.803056+0000","last_change":"2024-09-20T18:53:52.635213+0000","last_active":"2024-09-20T18:54:18.803056+0000","last_peered":"2024-09-20T18:54:18.803056+0000","last_clean":"2024-09-20T18:54:18.803056+0000","last_became_active":"2024-09-20T18:49:49.954502+0000","last_became_peered":"2024-09-20T18:49:49.954502+0000","last_unstale":"2024-09-20T18:54:18.803056+0000","last_undegraded":"2024-09-20T18:54:18.803056+0000","last_fullsized":"2024-09-20T18:54:18.803056+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:52.635138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:52.635138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:08.278673+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":203,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.790916+0000","last_change":"2024-09-20T18:51:09.761331+0000","last_active":"2024-09-20T18:54:18.790916+0000","last_peered":"2024-09-20T18:54:18.790916+0000","last_clean":"2024-09-20T18:54:18.790916+0000","last_became_active":"2024-09-20T18:49:51.968662+0000","last_became_peered":"2024-09-20T18:49:51.968662+0000","last_unstale":"2024-09-20T18:54:18.790916+0000","last_undegraded":"2024-09-20T18:54:18.790916+0000","last_fullsized":"2024-09-20T18:54:18.790916+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:09.761283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:09.761283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:30.451288+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":196,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.808039+0000","last_change":"2024-09-20T18:49:51.968476+0000","last_active":"2024-09-20T18:54:18.808039+0000","last_peered":"2024-09-20T18:54:18.808039+0000","last_clean":"2024-09-20T18:54:18.808039+0000","last_became_active":"2024-09-20T18:49:51.968216+0000","last_became_peered":"2024-09-20T18:49:51.968216+0000","last_unstale":"2024-09-20T18:54:18.808039+0000","last_undegraded":"2024-09-20T18:54:18.808039+0000","last_fullsized":"2024-09-20T18:54:18.808039+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:00.220813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":216,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.813574+0000","last_change":"2024-09-20T18:52:41.106558+0000","last_active":"2024-09-20T18:54:18.813574+0000","last_peered":"2024-09-20T18:54:18.813574+0000","last_clean":"2024-09-20T18:54:18.813574+0000","last_became_active":"2024-09-20T18:49:45.937992+0000","last_became_peered":"2024-09-20T18:49:45.937992+0000","last_unstale":"2024-09-20T18:54:18.813574+0000","last_undegraded":"2024-09-20T18:54:18.813574+0000","last_fullsized":"2024-09-20T18:54:18.813574+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:41.106332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:41.106332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":231,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.803665+0000","last_change":"2024-09-20T18:51:14.545122+0000","last_active":"2024-09-20T18:54:18.803665+0000","last_peered":"2024-09-20T18:54:18.803665+0000","last_clean":"2024-09-20T18:54:18.803665+0000","last_became_active":"2024-09-20T18:49:47.949582+0000","last_became_peered":"2024-09-20T18:49:47.949582+0000","last_unstale":"2024-09-20T18:54:18.803665+0000","last_undegraded":"2024-09-20T18:54:18.803665+0000","last_fullsized":"2024-09-20T18:54:18.803665+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:51:14.545043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.545043+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:40.491358+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":208,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-20T18:54:18.799941+0000","last_change":"2024-09-20T18:51:28.120100+0000","last_active":"2024-09-20T18:54:18.799941+0000","last_peered":"2024-09-20T18:54:18.799941+0000","last_clean":"2024-09-20T18:54:18.799941+0000","last_became_active":"2024-09-20T18:49:49.957582+0000","last_became_peered":"2024-09-20T18:49:49.957582+0000","last_unstale":"2024-09-20T18:54:18.799941+0000","last_undegraded":"2024-09-20T18:54:18.799941+0000","last_fullsized":"2024-09-20T18:54:18.799941+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:28.119694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:28.119694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:32.813511+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":247595032,"num_objects":3779,"num_object_clones":0,"num_object_copies":11337,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3779,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":241821,"num_write_kb":7859137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":747253760,"data_stored":747241544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":90015,"ondisk_log_size":90015,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":270,"num_read_kb":265,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266747,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":199676,"kb_used_data":92136,"kb_used_omap":1,"kb_used_meta":107518,"kb_avail":93545476,"statfs":{"total":95995035648,"available":95790567424,"internally_reserved":0,"allocated":94347264,"data_stored":93940564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":110098890},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528386,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":181864,"kb_used_data":82648,"kb_used_omap":1,"kb_used_meta":99198,"kb_avail":93563288,"statfs":{"total":95995035648,"available":95808806912,"internally_reserved":0,"allocated":84631552,"data_stored":84240598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":101579206},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757319,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":185060,"kb_used_data":73364,"kb_used_omap":1,"kb_used_meta":111678,"kb_avail":93560092,"statfs":{"total":95995035648,"available":95805534208,"internally_reserved":0,"allocated":75124736,"data_stored":74737669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":114358725},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986252,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":201672,"kb_used_data":98292,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":93543480,"statfs":{"total":95995035648,"available":95788523520,"internally_reserved":0,"allocated":100651008,"data_stored":100232425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105839045},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182483,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":294016,"kb_used_data":136368,"kb_used_omap":1,"kb_used_meta":157502,"kb_avail":93451136,"statfs":{"total":95995035648,"available":95693963264,"internally_reserved":0,"allocated":139640832,"data_stored":139225189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":161282503},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,7],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":232808,"kb_used_data":103000,"kb_used_omap":1,"kb_used_meta":129726,"kb_avail":93512344,"statfs":{"total":95995035648,"available":95756640256,"internally_reserved":0,"allocated":105472000,"data_stored":105081416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132839877},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,3,2,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":239328,"kb_used_data":109580,"kb_used_omap":1,"kb_used_meta":129726,"kb_avail":93505824,"statfs":{"total":95995035648,"available":95749963776,"internally_reserved":0,"allocated":112209920,"data_stored":111767296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132839877},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738468,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":296892,"kb_used_data":126700,"kb_used_omap":1,"kb_used_meta":169982,"kb_avail":93448260,"statfs":{"total":95995035648,"available":95691018240,"internally_reserved":0,"allocated":129740800,"data_stored":129329724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":174062021},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":116264960,"data_stored":116260888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":93065216,"data_stored":93061144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":93257728,"data_stored":93257728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":131530752,"data_stored":131530752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":86048768,"data_stored":86048768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":64225280,"data_stored":64225280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":80613376,"data_stored":80609304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":82247680,"data_stored":82247680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T18:54:24.710 INFO:tasks.cephadm.ceph_manager.ceph:would drop below min_pgs, proj -9, currently 1 2024-09-20T18:54:24.767 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:24.805 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:24.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:24.806 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:24.823 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Removed 122799 objects 2024-09-20T18:54:24.823 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Clean up completed and total clean up time :117.277 2024-09-20T18:54:24.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:24 smithi083 ceph-mon[29421]: pgmap v210: 148 pgs: 148 active+clean; 237 MiB data, 1.7 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.29k op/s 2024-09-20T18:54:24.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:24 smithi083 ceph-mon[29421]: 3.0 scrub starts 2024-09-20T18:54:24.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:24 smithi083 ceph-mon[29421]: 3.0 scrub ok 2024-09-20T18:54:24.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:24 smithi083 ceph-mon[29421]: 3.1b scrub starts 2024-09-20T18:54:24.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:24 smithi083 ceph-mon[29421]: 3.1b scrub ok 2024-09-20T18:54:24.905 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:24 smithi083 ceph-mon[36260]: pgmap v210: 148 pgs: 148 active+clean; 237 MiB data, 1.7 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.29k op/s 2024-09-20T18:54:24.905 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:24 smithi083 ceph-mon[36260]: 3.0 scrub starts 2024-09-20T18:54:24.905 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:24 smithi083 ceph-mon[36260]: 3.0 scrub ok 2024-09-20T18:54:24.905 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:24 smithi083 ceph-mon[36260]: 3.1b scrub starts 2024-09-20T18:54:24.905 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:24 smithi083 ceph-mon[36260]: 3.1b scrub ok 2024-09-20T18:54:24.906 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:24.927 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:24.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:24.928 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:25.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:24 smithi142 ceph-mon[32508]: pgmap v210: 148 pgs: 148 active+clean; 237 MiB data, 1.7 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.29k op/s 2024-09-20T18:54:25.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:24 smithi142 ceph-mon[32508]: 3.0 scrub starts 2024-09-20T18:54:25.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:24 smithi142 ceph-mon[32508]: 3.0 scrub ok 2024-09-20T18:54:25.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:24 smithi142 ceph-mon[32508]: 3.1b scrub starts 2024-09-20T18:54:25.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:24 smithi142 ceph-mon[32508]: 3.1b scrub ok 2024-09-20T18:54:25.029 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:54:25.050 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:25.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:25.051 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:54:25.062 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_0 2024-09-20T18:54:25.062 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd pool rm unique_pool_0 unique_pool_0 --yes-i-really-really-mean-it 2024-09-20T18:54:25.151 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:54:25.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:25.173 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:25.173 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:54:25.249 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:54:25.250 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:54:25.250 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:54:25.250 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:54:25.250 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:54:25.250 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:54:25.250 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:54:25.250 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:54:25.250 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:54:25.250 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:54:25.251 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:54:25.251 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:54:25.251 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:54:25.251 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:54:25.251 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:54:25.251 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:54:25.274 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:25.295 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:25.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:25.296 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:25.397 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:25.418 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:25.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:25.419 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:25.520 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:25.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:25.545 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:25.545 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:25.646 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:25.671 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:25.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:25.672 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:25.773 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:25.798 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:25.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:25.800 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:25.900 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:25.925 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:25.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:25.926 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:26.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:25 smithi142 ceph-mon[32508]: from='client.24910 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:54:26.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:25 smithi142 ceph-mon[32508]: pgmap v211: 148 pgs: 148 active+clean; 237 MiB data, 1.7 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.05k op/s 2024-09-20T18:54:26.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:25 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1838594184' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:54:26.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:25 smithi142 ceph-mon[32508]: 6.9 scrub starts 2024-09-20T18:54:26.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:25 smithi142 ceph-mon[32508]: 6.9 scrub ok 2024-09-20T18:54:26.027 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:26.050 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:26.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:26.051 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:26.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:25 smithi083 ceph-mon[29421]: from='client.24910 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:54:26.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:25 smithi083 ceph-mon[29421]: pgmap v211: 148 pgs: 148 active+clean; 237 MiB data, 1.7 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.05k op/s 2024-09-20T18:54:26.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:25 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1838594184' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:54:26.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:25 smithi083 ceph-mon[29421]: 6.9 scrub starts 2024-09-20T18:54:26.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:25 smithi083 ceph-mon[29421]: 6.9 scrub ok 2024-09-20T18:54:26.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:25 smithi083 ceph-mon[36260]: from='client.24910 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:54:26.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:25 smithi083 ceph-mon[36260]: pgmap v211: 148 pgs: 148 active+clean; 237 MiB data, 1.7 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.05k op/s 2024-09-20T18:54:26.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:25 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1838594184' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:54:26.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:25 smithi083 ceph-mon[36260]: 6.9 scrub starts 2024-09-20T18:54:26.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:25 smithi083 ceph-mon[36260]: 6.9 scrub ok 2024-09-20T18:54:26.152 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:26.176 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:26.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:26.176 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:26.195 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:54:26.278 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:26.300 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:26.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:26.301 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:26.402 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:26.424 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:26.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:26.425 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:26.526 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:26.557 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:26.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:26.558 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:26.658 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:26.681 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:26.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:26.682 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:26.783 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:26.806 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:26.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:26.807 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:26.907 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:26.930 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:26.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:26.931 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:27.023 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:54:26 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:54:27.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:26 smithi142 ceph-mon[32508]: 5.19 scrub starts 2024-09-20T18:54:27.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:26 smithi142 ceph-mon[32508]: 5.19 scrub ok 2024-09-20T18:54:27.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:26 smithi142 ceph-mon[32508]: 2.0 scrub starts 2024-09-20T18:54:27.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:26 smithi142 ceph-mon[32508]: 2.0 scrub ok 2024-09-20T18:54:27.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:26 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:54:27.031 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:27.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:26 smithi083 ceph-mon[36260]: 5.19 scrub starts 2024-09-20T18:54:27.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:26 smithi083 ceph-mon[36260]: 5.19 scrub ok 2024-09-20T18:54:27.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:26 smithi083 ceph-mon[36260]: 2.0 scrub starts 2024-09-20T18:54:27.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:26 smithi083 ceph-mon[36260]: 2.0 scrub ok 2024-09-20T18:54:27.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:26 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:54:27.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:26 smithi083 ceph-mon[29421]: 5.19 scrub starts 2024-09-20T18:54:27.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:26 smithi083 ceph-mon[29421]: 5.19 scrub ok 2024-09-20T18:54:27.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:26 smithi083 ceph-mon[29421]: 2.0 scrub starts 2024-09-20T18:54:27.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:26 smithi083 ceph-mon[29421]: 2.0 scrub ok 2024-09-20T18:54:27.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:26 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:54:27.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:27.055 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:27.055 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:27.156 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:54:27.179 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:27.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:27.179 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:54:27.280 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:27.305 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:27.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:27.306 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:27.315 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T18:54:27.315 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:54:27.325 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:54:27.406 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:27.427 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:27.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:27.428 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:27.529 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:27.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:27.551 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:27.551 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:27.651 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:27.673 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:27.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:27.674 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:27.701 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-20T18:54:27.775 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:27.801 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:27.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:27.802 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:27.902 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:27.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:27.927 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:27.928 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:27.961 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:27 smithi083 ceph-mon[29421]: pgmap v212: 148 pgs: 148 active+clean; 583 KiB data, 969 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 913 op/s 2024-09-20T18:54:27.961 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:27 smithi083 ceph-mon[29421]: 6.10 scrub starts 2024-09-20T18:54:27.961 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:27 smithi083 ceph-mon[29421]: 6.10 scrub ok 2024-09-20T18:54:27.961 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:27 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3645892856' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:54:27.961 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:27 smithi083 ceph-mon[36260]: pgmap v212: 148 pgs: 148 active+clean; 583 KiB data, 969 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 913 op/s 2024-09-20T18:54:27.961 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:27 smithi083 ceph-mon[36260]: 6.10 scrub starts 2024-09-20T18:54:27.962 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:27 smithi083 ceph-mon[36260]: 6.10 scrub ok 2024-09-20T18:54:27.962 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:27 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3645892856' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:54:27.990 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_historic_ops 2024-09-20T18:54:28.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:27 smithi142 ceph-mon[32508]: pgmap v212: 148 pgs: 148 active+clean; 583 KiB data, 969 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 913 op/s 2024-09-20T18:54:28.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:27 smithi142 ceph-mon[32508]: 6.10 scrub starts 2024-09-20T18:54:28.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:27 smithi142 ceph-mon[32508]: 6.10 scrub ok 2024-09-20T18:54:28.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:27 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3645892856' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:54:28.029 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:28.051 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:28.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:28.052 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:28.152 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:28.173 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:28.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:28.174 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:28.275 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:28.301 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:28.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:28.302 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:28.402 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:28.425 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:28.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:28.426 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:28.526 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:28.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:28.549 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:28.549 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:28.650 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:28.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:28.672 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:28.672 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:28.702 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'unique_pool_0' removed 2024-09-20T18:54:28.773 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:28.796 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:28.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:28.797 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:28.898 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:28.921 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:28.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:28.922 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:29.023 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:54:29.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:28 smithi142 ceph-mon[32508]: 7.e scrub starts 2024-09-20T18:54:29.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:28 smithi142 ceph-mon[32508]: 7.e scrub ok 2024-09-20T18:54:29.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:28 smithi142 ceph-mon[32508]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:54:29.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:28 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3645892856' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T18:54:29.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:28 smithi142 ceph-mon[32508]: osdmap e158: 8 total, 8 up, 8 in 2024-09-20T18:54:29.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:28 smithi142 ceph-mon[32508]: 3.1 scrub starts 2024-09-20T18:54:29.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:28 smithi142 ceph-mon[32508]: 3.1 scrub ok 2024-09-20T18:54:29.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:28 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1850331615' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true} : dispatch 2024-09-20T18:54:29.047 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:29.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:29.048 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:54:29.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:28 smithi083 ceph-mon[29421]: 7.e scrub starts 2024-09-20T18:54:29.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:28 smithi083 ceph-mon[29421]: 7.e scrub ok 2024-09-20T18:54:29.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:28 smithi083 ceph-mon[29421]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:54:29.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:28 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3645892856' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T18:54:29.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:28 smithi083 ceph-mon[29421]: osdmap e158: 8 total, 8 up, 8 in 2024-09-20T18:54:29.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:28 smithi083 ceph-mon[29421]: 3.1 scrub starts 2024-09-20T18:54:29.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:28 smithi083 ceph-mon[29421]: 3.1 scrub ok 2024-09-20T18:54:29.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:28 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1850331615' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true} : dispatch 2024-09-20T18:54:29.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:28 smithi083 ceph-mon[36260]: 7.e scrub starts 2024-09-20T18:54:29.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:28 smithi083 ceph-mon[36260]: 7.e scrub ok 2024-09-20T18:54:29.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:28 smithi083 ceph-mon[36260]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:54:29.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:28 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3645892856' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T18:54:29.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:28 smithi083 ceph-mon[36260]: osdmap e158: 8 total, 8 up, 8 in 2024-09-20T18:54:29.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:28 smithi083 ceph-mon[36260]: 3.1 scrub starts 2024-09-20T18:54:29.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:28 smithi083 ceph-mon[36260]: 3.1 scrub ok 2024-09-20T18:54:29.055 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:28 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1850331615' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true} : dispatch 2024-09-20T18:54:29.130 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T18:54:29.148 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:29.172 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:29.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:29.173 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:29.273 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:29.296 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:29.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:29.297 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:29.299 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:54:29.399 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:29.420 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:29.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:29.421 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:29.521 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:54:29.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:29.548 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:29.548 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:54:29.649 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:29.674 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:29.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:29.675 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:29.685 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-20T18:54:29.686 INFO:tasks.radosbench:Beginning radosbench... 2024-09-20T18:54:29.686 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_1 2024-09-20T18:54:29.687 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd pool create unique_pool_1 16 2024-09-20T18:54:29.711 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-20T18:54:29.711 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T18:54:29.711 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-20T18:54:29.712 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-20T18:54:29.712 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-20T18:54:29.712 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.1 config set filestore_inject_stall 3 2024-09-20T18:54:29.776 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:29.802 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:29.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:29.803 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:29.903 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:29.929 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:29 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1850331615' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]': finished 2024-09-20T18:54:29.929 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:29 smithi083 ceph-mon[36260]: osdmap e159: 8 total, 8 up, 8 in 2024-09-20T18:54:29.929 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:29 smithi083 ceph-mon[36260]: pgmap v215: 132 pgs: 132 active+clean; 583 KiB data, 969 MiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-20T18:54:29.929 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:29 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1850331615' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]': finished 2024-09-20T18:54:29.929 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:29 smithi083 ceph-mon[29421]: osdmap e159: 8 total, 8 up, 8 in 2024-09-20T18:54:29.930 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:29 smithi083 ceph-mon[29421]: pgmap v215: 132 pgs: 132 active+clean; 583 KiB data, 969 MiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-20T18:54:29.930 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:29.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:29.931 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:30.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:29 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1850331615' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]': finished 2024-09-20T18:54:30.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:29 smithi142 ceph-mon[32508]: osdmap e159: 8 total, 8 up, 8 in 2024-09-20T18:54:30.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:29 smithi142 ceph-mon[32508]: pgmap v215: 132 pgs: 132 active+clean; 583 KiB data, 969 MiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-20T18:54:30.032 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:54:30.056 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:30.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:30.057 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:54:30.158 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:30.188 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:30.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:30.189 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:30.198 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T18:54:30.198 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:54:30.202 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:54:30.203 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:54:30.206 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:54:30.290 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:30.311 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:30.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:30.312 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:30.413 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:30.439 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:30.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:30.440 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:30.540 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:54:30.564 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:30.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:30.565 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:54:30.665 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:30.687 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:30.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:30.688 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:30.789 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:30.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:30.810 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:30.811 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:30.857 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:30 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:54:30.858 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:30 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:54:30.858 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:54:30 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:54:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:54:30.912 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:30.933 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:30.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:30.934 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:30.965 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:54:30.965 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:54:30.972 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:54:30.972 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:54:30.988 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:54:30.988 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:54:31.004 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:54:31.004 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:54:31.020 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:54:31.021 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:54:31.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:30 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:54:31.035 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:31.038 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:54:31.038 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:54:31.054 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:54:31.054 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:54:31.056 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:31.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:31.057 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:31.071 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:54:31.071 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:54:31.090 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T18:54:31.158 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:31.179 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:31.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:31.180 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:31.280 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:31.302 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:31.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:31.303 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:31.404 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:31.426 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:31.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:31.427 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:31.528 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:31.550 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:31.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:31.551 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:31.651 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:31.674 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:31.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:31.675 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:31.775 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:31.797 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:31.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:31.798 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:31.898 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:31.904 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T18:54:31.905 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "" 2024-09-20T18:54:31.905 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T18:54:31.923 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:31.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:31.924 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:32.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:31 smithi142 ceph-mon[32508]: pgmap v216: 132 pgs: 132 active+clean; 583 KiB data, 969 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-20T18:54:32.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:31 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2338933875' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:54:32.025 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:54:32.048 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:32.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:32.049 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:54:32.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:31 smithi083 ceph-mon[29421]: pgmap v216: 132 pgs: 132 active+clean; 583 KiB data, 969 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-20T18:54:32.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:31 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2338933875' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:54:32.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:31 smithi083 ceph-mon[36260]: pgmap v216: 132 pgs: 132 active+clean; 583 KiB data, 969 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-20T18:54:32.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:31 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2338933875' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:54:32.150 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:54:32.173 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:32.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:32.174 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:54:32.275 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:54:32.298 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:32.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:32.299 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:54:32.401 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:32.423 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:32.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:32.424 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:32.494 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-20T18:54:32.525 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:54:32.551 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:32.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:32.552 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:54:32.653 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:32.677 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:32.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:32.678 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:32.779 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:32.800 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:32.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:32.801 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:32.902 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:32.928 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:32.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:32.929 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:32.931 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'unique_pool_1' created 2024-09-20T18:54:33.030 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:54:33.055 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:33.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:33.056 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:54:33.157 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:33.179 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:33.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:33.180 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:33.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:32 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/4051293787' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16} : dispatch 2024-09-20T18:54:33.280 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:33.301 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:33.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:33.302 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:33.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:32 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/4051293787' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16} : dispatch 2024-09-20T18:54:33.303 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:32 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/4051293787' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16} : dispatch 2024-09-20T18:54:33.403 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:33.427 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:33.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:33.428 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:33.529 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:33.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:33.556 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:33.556 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:33.657 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:33.679 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:33.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:33.680 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:33.781 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:33.807 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:33.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:33.808 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:33.831 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd pool application enable unique_pool_1 rados --yes-i-really-mean-it || true 2024-09-20T18:54:33.908 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:33.929 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:33.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:33.930 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:34.031 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:34.052 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:34.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:34.053 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:34.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:33 smithi083 ceph-mon[29421]: pgmap v217: 132 pgs: 132 active+clean; 583 KiB data, 986 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-20T18:54:34.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:33 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/4051293787' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]': finished 2024-09-20T18:54:34.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:33 smithi083 ceph-mon[29421]: osdmap e160: 8 total, 8 up, 8 in 2024-09-20T18:54:34.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:33 smithi083 ceph-mon[29421]: osdmap e161: 8 total, 8 up, 8 in 2024-09-20T18:54:34.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:33 smithi083 ceph-mon[36260]: pgmap v217: 132 pgs: 132 active+clean; 583 KiB data, 986 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-20T18:54:34.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:33 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/4051293787' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]': finished 2024-09-20T18:54:34.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:33 smithi083 ceph-mon[36260]: osdmap e160: 8 total, 8 up, 8 in 2024-09-20T18:54:34.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:33 smithi083 ceph-mon[36260]: osdmap e161: 8 total, 8 up, 8 in 2024-09-20T18:54:34.153 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:54:34.179 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:34.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:34.180 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:54:34.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:33 smithi142 ceph-mon[32508]: pgmap v217: 132 pgs: 132 active+clean; 583 KiB data, 986 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-20T18:54:34.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:33 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/4051293787' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]': finished 2024-09-20T18:54:34.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:33 smithi142 ceph-mon[32508]: osdmap e160: 8 total, 8 up, 8 in 2024-09-20T18:54:34.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:33 smithi142 ceph-mon[32508]: osdmap e161: 8 total, 8 up, 8 in 2024-09-20T18:54:34.281 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:34.305 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:34.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:34.306 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:34.337 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:54:34.337 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:54:34.407 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:34.432 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:34.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:34.433 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:34.534 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:34.558 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:34.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:34.559 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:34.660 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:34.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:34.682 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:34.682 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:34.783 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:34.804 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:34.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:34.805 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:34.906 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:34.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:34.928 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:34.928 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:35.028 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:35.049 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:35.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:35.049 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:35.150 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:35.175 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T18:54:35.176 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:35.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:35.177 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:35.257 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T18:54:35.257 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-20T18:54:35.257 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-20T18:54:35.257 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T18:54:35.278 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:35.300 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:35.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:35.301 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:35.402 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:54:35.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:35.429 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:35.429 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:54:35.530 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:35.556 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:35.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:35.557 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:35.657 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:54:35.682 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:35.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:35.683 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:54:35.784 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:35.789 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:54:35.810 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:35.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:35.811 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:35.821 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_blocked_ops 2024-09-20T18:54:35.911 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:35.937 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:35.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:35.938 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:35.954 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-20T18:54:36.040 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:36.063 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:36.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:36.064 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:36.071 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:35 smithi083 ceph-mon[29421]: pgmap v220: 148 pgs: 16 unknown, 132 active+clean; 583 KiB data, 986 MiB used, 714 GiB / 715 GiB avail; 830 B/s rd, 0 op/s 2024-09-20T18:54:36.071 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:35 smithi083 ceph-mon[29421]: osdmap e162: 8 total, 8 up, 8 in 2024-09-20T18:54:36.071 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:35 smithi083 ceph-mon[29421]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-20T18:54:36.071 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:35 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/69764364' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:54:36.071 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:35 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:54:36.072 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:35 smithi083 ceph-mon[36260]: pgmap v220: 148 pgs: 16 unknown, 132 active+clean; 583 KiB data, 986 MiB used, 714 GiB / 715 GiB avail; 830 B/s rd, 0 op/s 2024-09-20T18:54:36.072 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:35 smithi083 ceph-mon[36260]: osdmap e162: 8 total, 8 up, 8 in 2024-09-20T18:54:36.072 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:35 smithi083 ceph-mon[36260]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-20T18:54:36.072 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:35 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/69764364' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:54:36.072 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:35 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:54:36.166 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:36.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:36.189 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:36.189 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:36.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:35 smithi142 ceph-mon[32508]: pgmap v220: 148 pgs: 16 unknown, 132 active+clean; 583 KiB data, 986 MiB used, 714 GiB / 715 GiB avail; 830 B/s rd, 0 op/s 2024-09-20T18:54:36.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:35 smithi142 ceph-mon[32508]: osdmap e162: 8 total, 8 up, 8 in 2024-09-20T18:54:36.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:35 smithi142 ceph-mon[32508]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-20T18:54:36.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:35 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/69764364' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:54:36.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:35 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:54:36.290 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:36.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:36.314 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:36.314 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:36.415 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:36.436 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:36.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:36.437 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:36.538 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:54:36.563 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:36.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:36.563 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:54:36.664 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:36.685 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:36.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:36.686 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:36.787 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:36.813 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:36.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:36.814 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:36.914 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:36.936 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:54:36 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:54:36.938 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:36.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:36.939 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:36.981 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:54:37.003 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:54:37.040 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:37.062 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:37.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:37.063 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:37.093 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:36 smithi083 ceph-mon[29421]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:54:37.093 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:36 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T18:54:37.094 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:36 smithi083 ceph-mon[29421]: osdmap e163: 8 total, 8 up, 8 in 2024-09-20T18:54:37.094 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:36 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:54:37.094 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:36 smithi083 ceph-mon[36260]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:54:37.094 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:36 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T18:54:37.094 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:36 smithi083 ceph-mon[36260]: osdmap e163: 8 total, 8 up, 8 in 2024-09-20T18:54:37.094 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:36 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:54:37.164 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:54:37.190 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:37.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:37.191 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:54:37.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:36 smithi142 ceph-mon[32508]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:54:37.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:36 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T18:54:37.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:36 smithi142 ceph-mon[32508]: osdmap e163: 8 total, 8 up, 8 in 2024-09-20T18:54:37.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:36 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:54:37.292 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:37.312 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:37.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:37.313 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:37.414 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:37.440 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:37.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:37.441 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:37.542 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:37.567 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:37.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:37.568 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:37.614 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:54:37.614 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:54:37.631 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:54:37.631 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:54:37.646 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:54:37.647 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:54:37.662 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:54:37.663 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:54:37.669 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:37.679 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:54:37.679 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:54:37.694 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:37.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:37.695 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:37.696 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:54:37.696 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:54:37.714 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:54:37.715 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:54:37.731 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:54:37.731 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:54:37.796 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:54:37.800 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T18:54:37.800 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:54:37.820 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:37.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:37.821 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:54:37.922 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:37.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:37.943 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:37.944 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:37.972 INFO:teuthology.orchestra.run.smithi083.stderr:enabled application 'rados' on pool 'unique_pool_1' 2024-09-20T18:54:38.045 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:38.065 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:38.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:38.066 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:38.167 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:38.192 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:38.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:38.193 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:38.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:37 smithi142 ceph-mon[32508]: pgmap v223: 148 pgs: 148 active+clean; 583 KiB data, 990 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T18:54:38.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:37 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3578522141' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-20T18:54:38.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:37 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-20T18:54:38.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:37 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2822445349' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:54:38.294 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:38.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:37 smithi083 ceph-mon[29421]: pgmap v223: 148 pgs: 148 active+clean; 583 KiB data, 990 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T18:54:38.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:37 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3578522141' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-20T18:54:38.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:37 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-20T18:54:38.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:37 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2822445349' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:54:38.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:37 smithi083 ceph-mon[36260]: pgmap v223: 148 pgs: 148 active+clean; 583 KiB data, 990 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T18:54:38.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:37 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3578522141' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-20T18:54:38.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:37 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-20T18:54:38.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:37 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2822445349' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:54:38.317 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:38.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:38.318 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:38.419 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:38.440 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:38.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:38.441 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:38.542 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:54:38.566 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:38.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:38.567 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:54:38.668 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:38.680 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T18:54:38.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:38.690 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:38.690 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:38.792 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:54:38.817 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:38.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:38.818 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:54:38.919 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:38.944 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:38.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:38.945 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:39.046 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:54:39.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:38 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-20T18:54:39.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:38 smithi083 ceph-mon[29421]: osdmap e164: 8 total, 8 up, 8 in 2024-09-20T18:54:39.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:38 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-20T18:54:39.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:38 smithi083 ceph-mon[36260]: osdmap e164: 8 total, 8 up, 8 in 2024-09-20T18:54:39.070 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:39.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:39.071 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:54:39.173 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:39.203 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:39.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:39.204 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:39.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:38 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-20T18:54:39.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:38 smithi142 ceph-mon[32508]: osdmap e164: 8 total, 8 up, 8 in 2024-09-20T18:54:39.305 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:39.326 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:39.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:39.327 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:39.369 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-20T18:54:39.369 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T18:54:39.427 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:39.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:39.450 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:39.450 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:39.551 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:39.572 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:39.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:39.573 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:39.604 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:54:39.674 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:54:39.701 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:39.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:39.702 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:54:39.803 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:54:39.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:39.829 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:39.830 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:54:39.930 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:39.953 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:39.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:39.953 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:40.055 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:40.077 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:40.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:40.078 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:40.095 DEBUG:teuthology.orchestra.run.smithi083:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_1 bench 90 write' 2024-09-20T18:54:40.096 INFO:tasks.thrashosds.thrasher:Shrinking pool unique_pool_1 2024-09-20T18:54:40.096 INFO:tasks.cephadm.ceph_manager.ceph:contract_pool unique_pool_1 by 10 min 8 2024-09-20T18:54:40.096 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T18:54:40.098 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-20T18:54:40.098 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-20T18:54:40.154 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:hints = 1 2024-09-20T18:54:40.154 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-20T18:54:40.154 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Object prefix: benchmark_data_smithi083_97422 2024-09-20T18:54:40.155 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-20T18:54:40.155 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 0 0 0 0 0 0 - 0 2024-09-20T18:54:40.179 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:40.221 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:40.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:40.222 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:40.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:39 smithi142 ceph-mon[32508]: pgmap v225: 148 pgs: 148 active+clean; 583 KiB data, 990 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-20T18:54:40.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:39 smithi142 ceph-mon[32508]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-20T18:54:40.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:39 smithi083 ceph-mon[29421]: pgmap v225: 148 pgs: 148 active+clean; 583 KiB data, 990 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-20T18:54:40.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:39 smithi083 ceph-mon[29421]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-20T18:54:40.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:39 smithi083 ceph-mon[36260]: pgmap v225: 148 pgs: 148 active+clean; 583 KiB data, 990 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-20T18:54:40.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:39 smithi083 ceph-mon[36260]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-20T18:54:40.323 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:40.365 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:40.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:40.367 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:40.468 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:40.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:40.506 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:40.506 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:40.542 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T18:54:40.607 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:40.658 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:40.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:40.660 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:40.678 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_historic_ops 2024-09-20T18:54:40.761 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:40.789 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:40.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:40.790 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:40.790 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:54:40.891 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:54:40.917 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:40.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:40.918 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:54:40.946 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:54:40 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:54:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:54:40.980 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-20T18:54:41.019 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:41.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:41.059 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:41.059 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:41.160 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:41.166 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 1 16 2769 2753 170.057 172.062 0.0188502 0.00570855 2024-09-20T18:54:41.183 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:41.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:41.184 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:41.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:40 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:54:41.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:40 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2516140873' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:54:41.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:40 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:54:41.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:40 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:54:41.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:40 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:54:41.285 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:41.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:40 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:54:41.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:40 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2516140873' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:54:41.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:40 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:54:41.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:40 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:54:41.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:40 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:54:41.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:40 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:54:41.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:40 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2516140873' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:54:41.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:40 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:54:41.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:40 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:54:41.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:40 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:54:41.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:41.322 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:41.322 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:41.423 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:41.449 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:41.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:41.451 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:41.552 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:41.588 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:41.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:41.590 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:41.690 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:41.727 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:41.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:41.728 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:41.829 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:41.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:41.862 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:41.863 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:41.963 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:42.001 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:42.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:42.003 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:42.024 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:41 smithi083 ceph-mon[29421]: pgmap v226: 148 pgs: 148 active+clean; 46 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 7.6 MiB/s wr, 123 op/s 2024-09-20T18:54:42.024 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:41 smithi083 ceph-mon[29421]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:54:42.024 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:41 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2516140873' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T18:54:42.024 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:41 smithi083 ceph-mon[29421]: osdmap e165: 8 total, 8 up, 8 in 2024-09-20T18:54:42.024 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:41 smithi083 ceph-mon[36260]: pgmap v226: 148 pgs: 148 active+clean; 46 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 7.6 MiB/s wr, 123 op/s 2024-09-20T18:54:42.024 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:41 smithi083 ceph-mon[36260]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:54:42.024 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:41 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2516140873' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T18:54:42.024 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:41 smithi083 ceph-mon[36260]: osdmap e165: 8 total, 8 up, 8 in 2024-09-20T18:54:42.104 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:42.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:42.144 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:42.144 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:42.166 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 2 16 6020 6004 186.518 203.188 0.00258733 0.00534762 2024-09-20T18:54:42.245 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:54:42.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:41 smithi142 ceph-mon[32508]: pgmap v226: 148 pgs: 148 active+clean; 46 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 7.6 MiB/s wr, 123 op/s 2024-09-20T18:54:42.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:41 smithi142 ceph-mon[32508]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:54:42.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:41 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2516140873' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T18:54:42.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:41 smithi142 ceph-mon[32508]: osdmap e165: 8 total, 8 up, 8 in 2024-09-20T18:54:42.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:42.275 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:42.276 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:54:42.334 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:54:42.335 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-20T18:54:42.376 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:42.411 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:42.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:42.412 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:42.513 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:42.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:42.552 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:42.552 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:42.653 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:42.684 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:42.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:42.685 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:42.785 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:54:42.819 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:42.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:42.820 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:54:42.921 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:42.957 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:42.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:42.958 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:43.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:43 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:54:43.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:43 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:54:43.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:43 smithi083 ceph-mon[29421]: 6.e scrub starts 2024-09-20T18:54:43.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:43 smithi083 ceph-mon[29421]: 6.e scrub ok 2024-09-20T18:54:43.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:43 smithi083 ceph-mon[29421]: from='client.25064 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:54:43.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:43 smithi083 ceph-mon[29421]: 3.15 scrub starts 2024-09-20T18:54:43.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:43 smithi083 ceph-mon[29421]: 3.15 scrub ok 2024-09-20T18:54:43.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:43 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:54:43.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:43 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:54:43.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:43 smithi083 ceph-mon[36260]: 6.e scrub starts 2024-09-20T18:54:43.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:43 smithi083 ceph-mon[36260]: 6.e scrub ok 2024-09-20T18:54:43.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:43 smithi083 ceph-mon[36260]: from='client.25064 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:54:43.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:43 smithi083 ceph-mon[36260]: 3.15 scrub starts 2024-09-20T18:54:43.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:43 smithi083 ceph-mon[36260]: 3.15 scrub ok 2024-09-20T18:54:43.059 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:54:43.091 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:43.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:43.092 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:54:43.166 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 3 16 9222 9206 191.03 200.125 0.00405929 0.00522542 2024-09-20T18:54:43.192 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:43.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:43.232 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:43.232 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:43.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:43 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:54:43.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:43 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:54:43.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:43 smithi142 ceph-mon[32508]: 6.e scrub starts 2024-09-20T18:54:43.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:43 smithi142 ceph-mon[32508]: 6.e scrub ok 2024-09-20T18:54:43.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:43 smithi142 ceph-mon[32508]: from='client.25064 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:54:43.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:43 smithi142 ceph-mon[32508]: 3.15 scrub starts 2024-09-20T18:54:43.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:43 smithi142 ceph-mon[32508]: 3.15 scrub ok 2024-09-20T18:54:43.333 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:43.365 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:43.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:43.366 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:43.432 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":227,"stamp":"2024-09-20T18:54:40.966324+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":48699931,"num_objects":933,"num_object_clones":0,"num_object_copies":2799,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":933,"num_whiteouts":0,"num_read":1138,"num_read_kb":995,"num_write":1283,"num_write_kb":48649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":590808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1286,"ondisk_log_size":1286,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":429,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":1070904,"kb_used_data":73052,"kb_used_omap":12,"kb_used_meta":997619,"kb_avail":748890312,"statfs":{"total":767960285184,"available":766863679488,"internally_reserved":0,"allocated":74805248,"data_stored":71701549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1021562416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":48103424,"num_objects":734,"num_object_clones":0,"num_object_copies":2202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":734,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":734,"num_write_kb":46976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.047578"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":236,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:35.938059+0000","last_change":"2024-09-20T18:54:13.958541+0000","last_active":"2024-09-20T18:54:35.938059+0000","last_peered":"2024-09-20T18:54:35.938059+0000","last_clean":"2024-09-20T18:54:35.938059+0000","last_became_active":"2024-09-20T18:49:45.943201+0000","last_became_peered":"2024-09-20T18:49:45.943201+0000","last_unstale":"2024-09-20T18:54:35.938059+0000","last_undegraded":"2024-09-20T18:54:35.938059+0000","last_fullsized":"2024-09-20T18:54:35.938059+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:13.958355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:13.958355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:27.449343+0000","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":268,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:37.955209+0000","last_change":"2024-09-20T18:53:15.772300+0000","last_active":"2024-09-20T18:54:37.955209+0000","last_peered":"2024-09-20T18:54:37.955209+0000","last_clean":"2024-09-20T18:54:37.955209+0000","last_became_active":"2024-09-20T18:52:15.609648+0000","last_became_peered":"2024-09-20T18:52:15.609648+0000","last_unstale":"2024-09-20T18:54:37.955209+0000","last_undegraded":"2024-09-20T18:54:37.955209+0000","last_fullsized":"2024-09-20T18:54:37.955209+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:53:15.772228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:15.772228+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":37,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":234,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-20T18:54:34.923274+0000","last_change":"2024-09-20T18:54:25.361988+0000","last_active":"2024-09-20T18:54:34.923274+0000","last_peered":"2024-09-20T18:54:34.923274+0000","last_clean":"2024-09-20T18:54:34.923274+0000","last_became_active":"2024-09-20T18:49:49.959514+0000","last_became_peered":"2024-09-20T18:49:49.959514+0000","last_unstale":"2024-09-20T18:54:34.923274+0000","last_undegraded":"2024-09-20T18:54:34.923274+0000","last_fullsized":"2024-09-20T18:54:34.923274+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:25.361935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:25.361935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:49.454156+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":234,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:37.955399+0000","last_change":"2024-09-20T18:54:15.221404+0000","last_active":"2024-09-20T18:54:37.955399+0000","last_peered":"2024-09-20T18:54:37.955399+0000","last_clean":"2024-09-20T18:54:37.955399+0000","last_became_active":"2024-09-20T18:49:51.969421+0000","last_became_peered":"2024-09-20T18:49:51.969421+0000","last_unstale":"2024-09-20T18:54:37.955399+0000","last_undegraded":"2024-09-20T18:54:37.955399+0000","last_fullsized":"2024-09-20T18:54:37.955399+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:15.221342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:15.221342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:21.616726+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":224,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:36.183542+0000","last_change":"2024-09-20T18:53:46.283952+0000","last_active":"2024-09-20T18:54:36.183542+0000","last_peered":"2024-09-20T18:54:36.183542+0000","last_clean":"2024-09-20T18:54:36.183542+0000","last_became_active":"2024-09-20T18:49:52.141967+0000","last_became_peered":"2024-09-20T18:49:52.141967+0000","last_unstale":"2024-09-20T18:54:36.183542+0000","last_undegraded":"2024-09-20T18:54:36.183542+0000","last_fullsized":"2024-09-20T18:54:36.183542+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:46.283881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:46.283881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:49.076672+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":229,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:36.184024+0000","last_change":"2024-09-20T18:51:25.208165+0000","last_active":"2024-09-20T18:54:36.184024+0000","last_peered":"2024-09-20T18:54:36.184024+0000","last_clean":"2024-09-20T18:54:36.184024+0000","last_became_active":"2024-09-20T18:49:45.943607+0000","last_became_peered":"2024-09-20T18:49:45.943607+0000","last_unstale":"2024-09-20T18:54:36.184024+0000","last_undegraded":"2024-09-20T18:54:36.184024+0000","last_fullsized":"2024-09-20T18:54:36.184024+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:25.208073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:25.208073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:09.117437+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":249,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:36.183086+0000","last_change":"2024-09-20T18:51:07.130137+0000","last_active":"2024-09-20T18:54:36.183086+0000","last_peered":"2024-09-20T18:54:36.183086+0000","last_clean":"2024-09-20T18:54:36.183086+0000","last_became_active":"2024-09-20T18:49:47.952150+0000","last_became_peered":"2024-09-20T18:49:47.952150+0000","last_unstale":"2024-09-20T18:54:36.183086+0000","last_undegraded":"2024-09-20T18:54:36.183086+0000","last_fullsized":"2024-09-20T18:54:36.183086+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:51:07.130062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:07.130062+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:36.182256+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":231,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:37.955201+0000","last_change":"2024-09-20T18:54:11.246973+0000","last_active":"2024-09-20T18:54:37.955201+0000","last_peered":"2024-09-20T18:54:37.955201+0000","last_clean":"2024-09-20T18:54:37.955201+0000","last_became_active":"2024-09-20T18:49:49.956445+0000","last_became_peered":"2024-09-20T18:49:49.956445+0000","last_unstale":"2024-09-20T18:54:37.955201+0000","last_undegraded":"2024-09-20T18:54:37.955201+0000","last_fullsized":"2024-09-20T18:54:37.955201+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:11.246922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:11.246922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:18.822524+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"59'1","reported_seq":251,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:38.272960+0000","last_change":"2024-09-20T18:54:15.326527+0000","last_active":"2024-09-20T18:54:38.272960+0000","last_peered":"2024-09-20T18:54:38.272960+0000","last_clean":"2024-09-20T18:54:38.272960+0000","last_became_active":"2024-09-20T18:49:45.946171+0000","last_became_peered":"2024-09-20T18:49:45.946171+0000","last_unstale":"2024-09-20T18:54:38.272960+0000","last_undegraded":"2024-09-20T18:54:38.272960+0000","last_fullsized":"2024-09-20T18:54:38.272960+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:54:15.326476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:15.326476+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:27.762998+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":250,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:37.955692+0000","last_change":"2024-09-20T18:53:19.836082+0000","last_active":"2024-09-20T18:54:37.955692+0000","last_peered":"2024-09-20T18:54:37.955692+0000","last_clean":"2024-09-20T18:54:37.955692+0000","last_became_active":"2024-09-20T18:49:47.947448+0000","last_became_peered":"2024-09-20T18:49:47.947448+0000","last_unstale":"2024-09-20T18:54:37.955692+0000","last_undegraded":"2024-09-20T18:54:37.955692+0000","last_fullsized":"2024-09-20T18:54:37.955692+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:53:19.835996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:19.835996+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:39.939119+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":229,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:38.272488+0000","last_change":"2024-09-20T18:52:36.183932+0000","last_active":"2024-09-20T18:54:38.272488+0000","last_peered":"2024-09-20T18:54:38.272488+0000","last_clean":"2024-09-20T18:54:38.272488+0000","last_became_active":"2024-09-20T18:49:49.961188+0000","last_became_peered":"2024-09-20T18:49:49.961188+0000","last_unstale":"2024-09-20T18:54:38.272488+0000","last_undegraded":"2024-09-20T18:54:38.272488+0000","last_fullsized":"2024-09-20T18:54:38.272488+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:36.183875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:36.183875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:01.443173+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":207,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:35.936880+0000","last_change":"2024-09-20T18:49:51.966957+0000","last_active":"2024-09-20T18:54:35.936880+0000","last_peered":"2024-09-20T18:54:35.936880+0000","last_clean":"2024-09-20T18:54:35.936880+0000","last_became_active":"2024-09-20T18:49:51.966715+0000","last_became_peered":"2024-09-20T18:49:51.966715+0000","last_unstale":"2024-09-20T18:54:35.936880+0000","last_undegraded":"2024-09-20T18:54:35.936880+0000","last_fullsized":"2024-09-20T18:54:35.936880+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:01.897842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":229,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:38.272871+0000","last_change":"2024-09-20T18:52:00.970719+0000","last_active":"2024-09-20T18:54:38.272871+0000","last_peered":"2024-09-20T18:54:38.272871+0000","last_clean":"2024-09-20T18:54:38.272871+0000","last_became_active":"2024-09-20T18:49:45.944703+0000","last_became_peered":"2024-09-20T18:49:45.944703+0000","last_unstale":"2024-09-20T18:54:38.272871+0000","last_undegraded":"2024-09-20T18:54:38.272871+0000","last_fullsized":"2024-09-20T18:54:38.272871+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:00.970650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:00.970650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:09.393254+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":262,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:37.955415+0000","last_change":"2024-09-20T18:51:26.494194+0000","last_active":"2024-09-20T18:54:37.955415+0000","last_peered":"2024-09-20T18:54:37.955415+0000","last_clean":"2024-09-20T18:54:37.955415+0000","last_became_active":"2024-09-20T18:51:23.813399+0000","last_became_peered":"2024-09-20T18:51:23.813399+0000","last_unstale":"2024-09-20T18:54:37.955415+0000","last_undegraded":"2024-09-20T18:54:37.955415+0000","last_fullsized":"2024-09-20T18:54:37.955415+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-20T18:51:26.494118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:26.494118+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":229,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:38.157439+0000","last_change":"2024-09-20T18:52:38.799542+0000","last_active":"2024-09-20T18:54:38.157439+0000","last_peered":"2024-09-20T18:54:38.157439+0000","last_clean":"2024-09-20T18:54:38.157439+0000","last_became_active":"2024-09-20T18:49:49.959696+0000","last_became_peered":"2024-09-20T18:49:49.959696+0000","last_unstale":"2024-09-20T18:54:38.157439+0000","last_undegraded":"2024-09-20T18:54:38.157439+0000","last_fullsized":"2024-09-20T18:54:38.157439+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:38.799487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:38.799487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:06.357517+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":225,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:38.272225+0000","last_change":"2024-09-20T18:53:21.273835+0000","last_active":"2024-09-20T18:54:38.272225+0000","last_peered":"2024-09-20T18:54:38.272225+0000","last_clean":"2024-09-20T18:54:38.272225+0000","last_became_active":"2024-09-20T18:49:51.970663+0000","last_became_peered":"2024-09-20T18:49:51.970663+0000","last_unstale":"2024-09-20T18:54:38.272225+0000","last_undegraded":"2024-09-20T18:54:38.272225+0000","last_fullsized":"2024-09-20T18:54:38.272225+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:21.273788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:21.273788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:31.322423+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":227,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:37.956190+0000","last_change":"2024-09-20T18:53:53.940250+0000","last_active":"2024-09-20T18:54:37.956190+0000","last_peered":"2024-09-20T18:54:37.956190+0000","last_clean":"2024-09-20T18:54:37.956190+0000","last_became_active":"2024-09-20T18:49:52.141170+0000","last_became_peered":"2024-09-20T18:49:52.141170+0000","last_unstale":"2024-09-20T18:54:37.956190+0000","last_undegraded":"2024-09-20T18:54:37.956190+0000","last_fullsized":"2024-09-20T18:54:37.956190+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:53.940191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:53.940191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:59.342447+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":235,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:35.936995+0000","last_change":"2024-09-20T18:54:23.984024+0000","last_active":"2024-09-20T18:54:35.936995+0000","last_peered":"2024-09-20T18:54:35.936995+0000","last_clean":"2024-09-20T18:54:35.936995+0000","last_became_active":"2024-09-20T18:49:45.950480+0000","last_became_peered":"2024-09-20T18:49:45.950480+0000","last_unstale":"2024-09-20T18:54:35.936995+0000","last_undegraded":"2024-09-20T18:54:35.936995+0000","last_fullsized":"2024-09-20T18:54:35.936995+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:23.983958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:23.983958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:30.113894+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":258,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:35.937694+0000","last_change":"2024-09-20T18:53:26.762207+0000","last_active":"2024-09-20T18:54:35.937694+0000","last_peered":"2024-09-20T18:54:35.937694+0000","last_clean":"2024-09-20T18:54:35.937694+0000","last_became_active":"2024-09-20T18:49:47.950939+0000","last_became_peered":"2024-09-20T18:49:47.950939+0000","last_unstale":"2024-09-20T18:54:35.937694+0000","last_undegraded":"2024-09-20T18:54:35.937694+0000","last_fullsized":"2024-09-20T18:54:35.937694+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:53:26.762131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:26.762131+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:38.227744+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":218,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-20T18:54:34.923886+0000","last_change":"2024-09-20T18:51:17.248705+0000","last_active":"2024-09-20T18:54:34.923886+0000","last_peered":"2024-09-20T18:54:34.923886+0000","last_clean":"2024-09-20T18:54:34.923886+0000","last_became_active":"2024-09-20T18:49:49.958518+0000","last_became_peered":"2024-09-20T18:49:49.958518+0000","last_unstale":"2024-09-20T18:54:34.923886+0000","last_undegraded":"2024-09-20T18:54:34.923886+0000","last_fullsized":"2024-09-20T18:54:34.923886+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:17.248634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:17.248634+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:35.297969+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":225,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:36.183971+0000","last_change":"2024-09-20T18:53:54.259243+0000","last_active":"2024-09-20T18:54:36.183971+0000","last_peered":"2024-09-20T18:54:36.183971+0000","last_clean":"2024-09-20T18:54:36.183971+0000","last_became_active":"2024-09-20T18:49:52.140040+0000","last_became_peered":"2024-09-20T18:49:52.140040+0000","last_unstale":"2024-09-20T18:54:36.183971+0000","last_undegraded":"2024-09-20T18:54:36.183971+0000","last_fullsized":"2024-09-20T18:54:36.183971+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:54.259174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:54.259174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:06.632880+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"59'1","reported_seq":244,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:37.955800+0000","last_change":"2024-09-20T18:53:42.840884+0000","last_active":"2024-09-20T18:54:37.955800+0000","last_peered":"2024-09-20T18:54:37.955800+0000","last_clean":"2024-09-20T18:54:37.955800+0000","last_became_active":"2024-09-20T18:49:45.940100+0000","last_became_peered":"2024-09-20T18:49:45.940100+0000","last_unstale":"2024-09-20T18:54:37.955800+0000","last_undegraded":"2024-09-20T18:54:37.955800+0000","last_fullsized":"2024-09-20T18:54:37.955800+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:53:42.840800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:42.840800+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:56.760029+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":239,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:36.183758+0000","last_change":"2024-09-20T18:49:47.951215+0000","last_active":"2024-09-20T18:54:36.183758+0000","last_peered":"2024-09-20T18:54:36.183758+0000","last_clean":"2024-09-20T18:54:36.183758+0000","last_became_active":"2024-09-20T18:49:47.950909+0000","last_became_peered":"2024-09-20T18:49:47.950909+0000","last_unstale":"2024-09-20T18:54:36.183758+0000","last_undegraded":"2024-09-20T18:54:36.183758+0000","last_fullsized":"2024-09-20T18:54:36.183758+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:00.257976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":230,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:37.955607+0000","last_change":"2024-09-20T18:53:12.820752+0000","last_active":"2024-09-20T18:54:37.955607+0000","last_peered":"2024-09-20T18:54:37.955607+0000","last_clean":"2024-09-20T18:54:37.955607+0000","last_became_active":"2024-09-20T18:49:49.959102+0000","last_became_peered":"2024-09-20T18:49:49.959102+0000","last_unstale":"2024-09-20T18:54:37.955607+0000","last_undegraded":"2024-09-20T18:54:37.955607+0000","last_fullsized":"2024-09-20T18:54:37.955607+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:12.820697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:12.820697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":226,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:38.157991+0000","last_change":"2024-09-20T18:52:15.826819+0000","last_active":"2024-09-20T18:54:38.157991+0000","last_peered":"2024-09-20T18:54:38.157991+0000","last_clean":"2024-09-20T18:54:38.157991+0000","last_became_active":"2024-09-20T18:49:51.970058+0000","last_became_peered":"2024-09-20T18:49:51.970058+0000","last_unstale":"2024-09-20T18:54:38.157991+0000","last_undegraded":"2024-09-20T18:54:38.157991+0000","last_fullsized":"2024-09-20T18:54:38.157991+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T18:52:15.826638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:15.826638+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:34.458571+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":227,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-20T18:54:34.923396+0000","last_change":"2024-09-20T18:52:47.158384+0000","last_active":"2024-09-20T18:54:34.923396+0000","last_peered":"2024-09-20T18:54:34.923396+0000","last_clean":"2024-09-20T18:54:34.923396+0000","last_became_active":"2024-09-20T18:49:45.933078+0000","last_became_peered":"2024-09-20T18:49:45.933078+0000","last_unstale":"2024-09-20T18:54:34.923396+0000","last_undegraded":"2024-09-20T18:54:34.923396+0000","last_fullsized":"2024-09-20T18:54:34.923396+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:52:47.158260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:47.158260+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:05.321731+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":247,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:35.938355+0000","last_change":"2024-09-20T18:54:17.919123+0000","last_active":"2024-09-20T18:54:35.938355+0000","last_peered":"2024-09-20T18:54:35.938355+0000","last_clean":"2024-09-20T18:54:35.938355+0000","last_became_active":"2024-09-20T18:49:48.500809+0000","last_became_peered":"2024-09-20T18:49:48.500809+0000","last_unstale":"2024-09-20T18:54:35.938355+0000","last_undegraded":"2024-09-20T18:54:35.938355+0000","last_fullsized":"2024-09-20T18:54:35.938355+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:54:17.919055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:17.919055+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:19.776366+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":300,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:38.017376+0000","last_change":"2024-09-20T18:53:44.524093+0000","last_active":"2024-09-20T18:54:38.017376+0000","last_peered":"2024-09-20T18:54:38.017376+0000","last_clean":"2024-09-20T18:54:38.017376+0000","last_became_active":"2024-09-20T18:49:49.954772+0000","last_became_peered":"2024-09-20T18:49:49.954772+0000","last_unstale":"2024-09-20T18:54:38.017376+0000","last_undegraded":"2024-09-20T18:54:38.017376+0000","last_fullsized":"2024-09-20T18:54:38.017376+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:53:44.524040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:44.524040+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:03.085945+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-20T18:54:34.923562+0000","last_change":"2024-09-20T18:54:33.928915+0000","last_active":"2024-09-20T18:54:34.923562+0000","last_peered":"2024-09-20T18:54:34.923562+0000","last_clean":"2024-09-20T18:54:34.923562+0000","last_became_active":"2024-09-20T18:54:33.928692+0000","last_became_peered":"2024-09-20T18:54:33.928692+0000","last_unstale":"2024-09-20T18:54:34.923562+0000","last_undegraded":"2024-09-20T18:54:34.923562+0000","last_fullsized":"2024-09-20T18:54:34.923562+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:59.293385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":242,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:35.937969+0000","last_change":"2024-09-20T18:53:48.782289+0000","last_active":"2024-09-20T18:54:35.937969+0000","last_peered":"2024-09-20T18:54:35.937969+0000","last_clean":"2024-09-20T18:54:35.937969+0000","last_became_active":"2024-09-20T18:49:58.992302+0000","last_became_peered":"2024-09-20T18:49:58.992302+0000","last_unstale":"2024-09-20T18:54:35.937969+0000","last_undegraded":"2024-09-20T18:54:35.937969+0000","last_fullsized":"2024-09-20T18:54:35.937969+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:53:48.782192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:48.782192+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:04.815274+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":229,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:36.183490+0000","last_change":"2024-09-20T18:53:26.175148+0000","last_active":"2024-09-20T18:54:36.183490+0000","last_peered":"2024-09-20T18:54:36.183490+0000","last_clean":"2024-09-20T18:54:36.183490+0000","last_became_active":"2024-09-20T18:49:45.936104+0000","last_became_peered":"2024-09-20T18:49:45.936104+0000","last_unstale":"2024-09-20T18:54:36.183490+0000","last_undegraded":"2024-09-20T18:54:36.183490+0000","last_fullsized":"2024-09-20T18:54:36.183490+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:26.175083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:26.175083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:49.575901+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":301,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:37.956270+0000","last_change":"2024-09-20T18:54:14.292660+0000","last_active":"2024-09-20T18:54:37.956270+0000","last_peered":"2024-09-20T18:54:37.956270+0000","last_clean":"2024-09-20T18:54:37.956270+0000","last_became_active":"2024-09-20T18:49:49.960697+0000","last_became_peered":"2024-09-20T18:49:49.960697+0000","last_unstale":"2024-09-20T18:54:37.956270+0000","last_undegraded":"2024-09-20T18:54:37.956270+0000","last_fullsized":"2024-09-20T18:54:37.956270+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:54:14.292610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:14.292610+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:35.787441+0000","scrub_duration":34,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":226,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:38.272735+0000","last_change":"2024-09-20T18:54:10.381171+0000","last_active":"2024-09-20T18:54:38.272735+0000","last_peered":"2024-09-20T18:54:38.272735+0000","last_clean":"2024-09-20T18:54:38.272735+0000","last_became_active":"2024-09-20T18:49:51.971399+0000","last_became_peered":"2024-09-20T18:49:51.971399+0000","last_unstale":"2024-09-20T18:54:38.272735+0000","last_undegraded":"2024-09-20T18:54:38.272735+0000","last_fullsized":"2024-09-20T18:54:38.272735+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:10.381108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:10.381108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:32.368649+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":16,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:35.937687+0000","last_change":"2024-09-20T18:54:34.311256+0000","last_active":"2024-09-20T18:54:35.937687+0000","last_peered":"2024-09-20T18:54:35.937687+0000","last_clean":"2024-09-20T18:54:35.937687+0000","last_became_active":"2024-09-20T18:54:34.310904+0000","last_became_peered":"2024-09-20T18:54:34.310904+0000","last_unstale":"2024-09-20T18:54:35.937687+0000","last_undegraded":"2024-09-20T18:54:35.937687+0000","last_fullsized":"2024-09-20T18:54:35.937687+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:36.652900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":265,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:36.183397+0000","last_change":"2024-09-20T18:54:22.077299+0000","last_active":"2024-09-20T18:54:36.183397+0000","last_peered":"2024-09-20T18:54:36.183397+0000","last_clean":"2024-09-20T18:54:36.183397+0000","last_became_active":"2024-09-20T18:49:48.499189+0000","last_became_peered":"2024-09-20T18:49:48.499189+0000","last_unstale":"2024-09-20T18:54:36.183397+0000","last_undegraded":"2024-09-20T18:54:36.183397+0000","last_fullsized":"2024-09-20T18:54:36.183397+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-20T18:54:22.077212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:22.077212+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:52.022959+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":221,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:36.184187+0000","last_change":"2024-09-20T18:49:45.940816+0000","last_active":"2024-09-20T18:54:36.184187+0000","last_peered":"2024-09-20T18:54:36.184187+0000","last_clean":"2024-09-20T18:54:36.184187+0000","last_became_active":"2024-09-20T18:49:45.940544+0000","last_became_peered":"2024-09-20T18:49:45.940544+0000","last_unstale":"2024-09-20T18:54:36.184187+0000","last_undegraded":"2024-09-20T18:54:36.184187+0000","last_fullsized":"2024-09-20T18:54:36.184187+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:56.293250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":231,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:37.955219+0000","last_change":"2024-09-20T18:53:09.779280+0000","last_active":"2024-09-20T18:54:37.955219+0000","last_peered":"2024-09-20T18:54:37.955219+0000","last_clean":"2024-09-20T18:54:37.955219+0000","last_became_active":"2024-09-20T18:49:49.963073+0000","last_became_peered":"2024-09-20T18:49:49.963073+0000","last_unstale":"2024-09-20T18:54:37.955219+0000","last_undegraded":"2024-09-20T18:54:37.955219+0000","last_fullsized":"2024-09-20T18:54:37.955219+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:09.779228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:09.779228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:39.433897+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":227,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:37.955644+0000","last_change":"2024-09-20T18:53:27.796357+0000","last_active":"2024-09-20T18:54:37.955644+0000","last_peered":"2024-09-20T18:54:37.955644+0000","last_clean":"2024-09-20T18:54:37.955644+0000","last_became_active":"2024-09-20T18:49:51.968793+0000","last_became_peered":"2024-09-20T18:49:51.968793+0000","last_unstale":"2024-09-20T18:54:37.955644+0000","last_undegraded":"2024-09-20T18:54:37.955644+0000","last_fullsized":"2024-09-20T18:54:37.955644+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:27.796289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:27.796289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:48.888717+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"164'112","reported_seq":131,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:40.783703+0000","last_change":"2024-09-20T18:54:34.311680+0000","last_active":"2024-09-20T18:54:40.783703+0000","last_peered":"2024-09-20T18:54:40.783703+0000","last_clean":"2024-09-20T18:54:40.783703+0000","last_became_active":"2024-09-20T18:54:34.311241+0000","last_became_peered":"2024-09-20T18:54:34.311241+0000","last_unstale":"2024-09-20T18:54:40.783703+0000","last_undegraded":"2024-09-20T18:54:40.783703+0000","last_fullsized":"2024-09-20T18:54:40.783703+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:38.100923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7340032,"num_objects":112,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":264,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:37.955286+0000","last_change":"2024-09-20T18:53:44.879133+0000","last_active":"2024-09-20T18:54:37.955286+0000","last_peered":"2024-09-20T18:54:37.955286+0000","last_clean":"2024-09-20T18:54:37.955286+0000","last_became_active":"2024-09-20T18:49:47.941895+0000","last_became_peered":"2024-09-20T18:49:47.941895+0000","last_unstale":"2024-09-20T18:54:37.955286+0000","last_undegraded":"2024-09-20T18:54:37.955286+0000","last_fullsized":"2024-09-20T18:54:37.955286+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-20T18:53:44.879048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:44.879048+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:52.038099+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":229,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:35.938686+0000","last_change":"2024-09-20T18:51:27.187060+0000","last_active":"2024-09-20T18:54:35.938686+0000","last_peered":"2024-09-20T18:54:35.938686+0000","last_clean":"2024-09-20T18:54:35.938686+0000","last_became_active":"2024-09-20T18:49:45.940469+0000","last_became_peered":"2024-09-20T18:49:45.940469+0000","last_unstale":"2024-09-20T18:54:35.938686+0000","last_undegraded":"2024-09-20T18:54:35.938686+0000","last_fullsized":"2024-09-20T18:54:35.938686+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:51:27.186991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:27.186991+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:58.816707+0000","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"66'8","reported_seq":296,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:36.183297+0000","last_change":"2024-09-20T18:53:44.293165+0000","last_active":"2024-09-20T18:54:36.183297+0000","last_peered":"2024-09-20T18:54:36.183297+0000","last_clean":"2024-09-20T18:54:36.183297+0000","last_became_active":"2024-09-20T18:49:49.956133+0000","last_became_peered":"2024-09-20T18:49:49.956133+0000","last_unstale":"2024-09-20T18:54:36.183297+0000","last_undegraded":"2024-09-20T18:54:36.183297+0000","last_fullsized":"2024-09-20T18:54:36.183297+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:53:44.293094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:44.293094+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:47.176037+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":226,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:38.157753+0000","last_change":"2024-09-20T18:52:42.880767+0000","last_active":"2024-09-20T18:54:38.157753+0000","last_peered":"2024-09-20T18:54:38.157753+0000","last_clean":"2024-09-20T18:54:38.157753+0000","last_became_active":"2024-09-20T18:49:52.142113+0000","last_became_peered":"2024-09-20T18:49:52.142113+0000","last_unstale":"2024-09-20T18:54:38.157753+0000","last_undegraded":"2024-09-20T18:54:38.157753+0000","last_fullsized":"2024-09-20T18:54:38.157753+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:42.880724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:42.880724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:08.373192+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":16,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:35.937926+0000","last_change":"2024-09-20T18:54:34.309939+0000","last_active":"2024-09-20T18:54:35.937926+0000","last_peered":"2024-09-20T18:54:35.937926+0000","last_clean":"2024-09-20T18:54:35.937926+0000","last_became_active":"2024-09-20T18:54:34.309630+0000","last_became_peered":"2024-09-20T18:54:34.309630+0000","last_unstale":"2024-09-20T18:54:35.937926+0000","last_undegraded":"2024-09-20T18:54:35.937926+0000","last_fullsized":"2024-09-20T18:54:35.937926+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:43.427534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":246,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-20T18:54:34.923510+0000","last_change":"2024-09-20T18:52:39.246125+0000","last_active":"2024-09-20T18:54:34.923510+0000","last_peered":"2024-09-20T18:54:34.923510+0000","last_clean":"2024-09-20T18:54:34.923510+0000","last_became_active":"2024-09-20T18:49:47.949626+0000","last_became_peered":"2024-09-20T18:49:47.949626+0000","last_unstale":"2024-09-20T18:54:34.923510+0000","last_undegraded":"2024-09-20T18:54:34.923510+0000","last_fullsized":"2024-09-20T18:54:34.923510+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:52:39.246050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:39.246050+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:09.285670+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":246,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:38.273022+0000","last_change":"2024-09-20T18:54:18.326947+0000","last_active":"2024-09-20T18:54:38.273022+0000","last_peered":"2024-09-20T18:54:38.273022+0000","last_clean":"2024-09-20T18:54:38.273022+0000","last_became_active":"2024-09-20T18:49:45.942496+0000","last_became_peered":"2024-09-20T18:49:45.942496+0000","last_unstale":"2024-09-20T18:54:38.273022+0000","last_undegraded":"2024-09-20T18:54:38.273022+0000","last_fullsized":"2024-09-20T18:54:38.273022+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:18.326875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:18.326875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:33.692074+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":287,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:35.936643+0000","last_change":"2024-09-20T18:51:16.088407+0000","last_active":"2024-09-20T18:54:35.936643+0000","last_peered":"2024-09-20T18:54:35.936643+0000","last_clean":"2024-09-20T18:54:35.936643+0000","last_became_active":"2024-09-20T18:49:49.959365+0000","last_became_peered":"2024-09-20T18:49:49.959365+0000","last_unstale":"2024-09-20T18:54:35.936643+0000","last_undegraded":"2024-09-20T18:54:35.936643+0000","last_fullsized":"2024-09-20T18:54:35.936643+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:51:16.088343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:16.088343+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":208,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:35.937010+0000","last_change":"2024-09-20T18:49:51.965733+0000","last_active":"2024-09-20T18:54:35.937010+0000","last_peered":"2024-09-20T18:54:35.937010+0000","last_clean":"2024-09-20T18:54:35.937010+0000","last_became_active":"2024-09-20T18:49:51.964780+0000","last_became_peered":"2024-09-20T18:49:51.964780+0000","last_unstale":"2024-09-20T18:54:35.937010+0000","last_undegraded":"2024-09-20T18:54:35.937010+0000","last_fullsized":"2024-09-20T18:54:35.937010+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:56.818438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":16,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:36.184146+0000","last_change":"2024-09-20T18:54:34.309809+0000","last_active":"2024-09-20T18:54:36.184146+0000","last_peered":"2024-09-20T18:54:36.184146+0000","last_clean":"2024-09-20T18:54:36.184146+0000","last_became_active":"2024-09-20T18:54:34.309231+0000","last_became_peered":"2024-09-20T18:54:34.309231+0000","last_unstale":"2024-09-20T18:54:36.184146+0000","last_undegraded":"2024-09-20T18:54:36.184146+0000","last_fullsized":"2024-09-20T18:54:36.184146+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:50.212649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":268,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:35.936789+0000","last_change":"2024-09-20T18:54:14.923895+0000","last_active":"2024-09-20T18:54:35.936789+0000","last_peered":"2024-09-20T18:54:35.936789+0000","last_clean":"2024-09-20T18:54:35.936789+0000","last_became_active":"2024-09-20T18:49:48.497736+0000","last_became_peered":"2024-09-20T18:49:48.497736+0000","last_unstale":"2024-09-20T18:54:35.936789+0000","last_undegraded":"2024-09-20T18:54:35.936789+0000","last_fullsized":"2024-09-20T18:54:35.936789+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-20T18:54:14.923682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:14.923682+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:38.986258+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":219,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-20T18:54:34.923917+0000","last_change":"2024-09-20T18:49:45.942589+0000","last_active":"2024-09-20T18:54:34.923917+0000","last_peered":"2024-09-20T18:54:34.923917+0000","last_clean":"2024-09-20T18:54:34.923917+0000","last_became_active":"2024-09-20T18:49:45.942253+0000","last_became_peered":"2024-09-20T18:49:45.942253+0000","last_unstale":"2024-09-20T18:54:34.923917+0000","last_undegraded":"2024-09-20T18:54:34.923917+0000","last_fullsized":"2024-09-20T18:54:34.923917+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:59.256981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":230,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:38.017677+0000","last_change":"2024-09-20T18:53:12.347123+0000","last_active":"2024-09-20T18:54:38.017677+0000","last_peered":"2024-09-20T18:54:38.017677+0000","last_clean":"2024-09-20T18:54:38.017677+0000","last_became_active":"2024-09-20T18:49:49.954299+0000","last_became_peered":"2024-09-20T18:49:49.954299+0000","last_unstale":"2024-09-20T18:54:38.017677+0000","last_undegraded":"2024-09-20T18:54:38.017677+0000","last_fullsized":"2024-09-20T18:54:38.017677+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:12.347044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:12.347044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:41.912430+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":223,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-20T18:54:34.923273+0000","last_change":"2024-09-20T18:53:25.362907+0000","last_active":"2024-09-20T18:54:34.923273+0000","last_peered":"2024-09-20T18:54:34.923273+0000","last_clean":"2024-09-20T18:54:34.923273+0000","last_became_active":"2024-09-20T18:49:52.139981+0000","last_became_peered":"2024-09-20T18:49:52.139981+0000","last_unstale":"2024-09-20T18:54:34.923273+0000","last_undegraded":"2024-09-20T18:54:34.923273+0000","last_fullsized":"2024-09-20T18:54:34.923273+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:25.362828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:25.362828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:55.107515+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"164'131","reported_seq":150,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:40.815741+0000","last_change":"2024-09-20T18:54:34.311561+0000","last_active":"2024-09-20T18:54:40.815741+0000","last_peered":"2024-09-20T18:54:40.815741+0000","last_clean":"2024-09-20T18:54:40.815741+0000","last_became_active":"2024-09-20T18:54:34.311181+0000","last_became_peered":"2024-09-20T18:54:34.311181+0000","last_unstale":"2024-09-20T18:54:40.815741+0000","last_undegraded":"2024-09-20T18:54:40.815741+0000","last_fullsized":"2024-09-20T18:54:40.815741+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:02.829367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8585216,"num_objects":131,"num_object_clones":0,"num_object_copies":393,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":131,"num_write_kb":8384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"66'28","reported_seq":329,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-20T18:54:34.923782+0000","last_change":"2024-09-20T18:51:57.293659+0000","last_active":"2024-09-20T18:54:34.923782+0000","last_peered":"2024-09-20T18:54:34.923782+0000","last_clean":"2024-09-20T18:54:34.923782+0000","last_became_active":"2024-09-20T18:49:47.955335+0000","last_became_peered":"2024-09-20T18:49:47.955335+0000","last_unstale":"2024-09-20T18:54:34.923782+0000","last_undegraded":"2024-09-20T18:54:34.923782+0000","last_fullsized":"2024-09-20T18:54:34.923782+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'28","last_scrub_stamp":"2024-09-20T18:51:57.293574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:57.293574+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:08.276804+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":223,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:37.955786+0000","last_change":"2024-09-20T18:49:45.945275+0000","last_active":"2024-09-20T18:54:37.955786+0000","last_peered":"2024-09-20T18:54:37.955786+0000","last_clean":"2024-09-20T18:54:37.955786+0000","last_became_active":"2024-09-20T18:49:45.944992+0000","last_became_peered":"2024-09-20T18:49:45.944992+0000","last_unstale":"2024-09-20T18:54:37.955786+0000","last_undegraded":"2024-09-20T18:54:37.955786+0000","last_fullsized":"2024-09-20T18:54:37.955786+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:01.052027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":256,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:38.272574+0000","last_change":"2024-09-20T18:53:49.323451+0000","last_active":"2024-09-20T18:54:38.272574+0000","last_peered":"2024-09-20T18:54:38.272574+0000","last_clean":"2024-09-20T18:54:38.272574+0000","last_became_active":"2024-09-20T18:49:44.936027+0000","last_became_peered":"2024-09-20T18:49:44.936027+0000","last_unstale":"2024-09-20T18:54:38.272574+0000","last_undegraded":"2024-09-20T18:54:38.272574+0000","last_fullsized":"2024-09-20T18:54:38.272574+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-20T18:53:49.323385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:53:49.323385+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:50.958195+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.0012781159999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":228,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:35.937765+0000","last_change":"2024-09-20T18:53:12.897519+0000","last_active":"2024-09-20T18:54:35.937765+0000","last_peered":"2024-09-20T18:54:35.937765+0000","last_clean":"2024-09-20T18:54:35.937765+0000","last_became_active":"2024-09-20T18:49:49.953971+0000","last_became_peered":"2024-09-20T18:49:49.953971+0000","last_unstale":"2024-09-20T18:54:35.937765+0000","last_undegraded":"2024-09-20T18:54:35.937765+0000","last_fullsized":"2024-09-20T18:54:35.937765+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:12.897051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:12.897051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:39.168470+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":209,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:36.183832+0000","last_change":"2024-09-20T18:49:51.969702+0000","last_active":"2024-09-20T18:54:36.183832+0000","last_peered":"2024-09-20T18:54:36.183832+0000","last_clean":"2024-09-20T18:54:36.183832+0000","last_became_active":"2024-09-20T18:49:51.969400+0000","last_became_peered":"2024-09-20T18:49:51.969400+0000","last_unstale":"2024-09-20T18:54:36.183832+0000","last_undegraded":"2024-09-20T18:54:36.183832+0000","last_fullsized":"2024-09-20T18:54:36.183832+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:36.285843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":16,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:36.183530+0000","last_change":"2024-09-20T18:54:33.926785+0000","last_active":"2024-09-20T18:54:36.183530+0000","last_peered":"2024-09-20T18:54:36.183530+0000","last_clean":"2024-09-20T18:54:36.183530+0000","last_became_active":"2024-09-20T18:54:33.925750+0000","last_became_peered":"2024-09-20T18:54:33.925750+0000","last_unstale":"2024-09-20T18:54:36.183530+0000","last_undegraded":"2024-09-20T18:54:36.183530+0000","last_fullsized":"2024-09-20T18:54:36.183530+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:39.953856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":258,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-20T18:54:34.924149+0000","last_change":"2024-09-20T18:53:48.214566+0000","last_active":"2024-09-20T18:54:34.924149+0000","last_peered":"2024-09-20T18:54:34.924149+0000","last_clean":"2024-09-20T18:54:34.924149+0000","last_became_active":"2024-09-20T18:49:47.950904+0000","last_became_peered":"2024-09-20T18:49:47.950904+0000","last_unstale":"2024-09-20T18:54:34.924149+0000","last_undegraded":"2024-09-20T18:54:34.924149+0000","last_fullsized":"2024-09-20T18:54:34.924149+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-20T18:53:48.214491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:48.214491+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:15.858989+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":243,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-20T18:54:34.923245+0000","last_change":"2024-09-20T18:54:23.359948+0000","last_active":"2024-09-20T18:54:34.923245+0000","last_peered":"2024-09-20T18:54:34.923245+0000","last_clean":"2024-09-20T18:54:34.923245+0000","last_became_active":"2024-09-20T18:49:45.939884+0000","last_became_peered":"2024-09-20T18:49:45.939884+0000","last_unstale":"2024-09-20T18:54:34.923245+0000","last_undegraded":"2024-09-20T18:54:34.923245+0000","last_fullsized":"2024-09-20T18:54:34.923245+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:23.359769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:23.359769+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:52.487977+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":239,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:35.937027+0000","last_change":"2024-09-20T18:52:07.718766+0000","last_active":"2024-09-20T18:54:35.937027+0000","last_peered":"2024-09-20T18:54:35.937027+0000","last_clean":"2024-09-20T18:54:35.937027+0000","last_became_active":"2024-09-20T18:49:44.937135+0000","last_became_peered":"2024-09-20T18:49:44.937135+0000","last_unstale":"2024-09-20T18:54:35.937027+0000","last_undegraded":"2024-09-20T18:54:35.937027+0000","last_fullsized":"2024-09-20T18:54:35.937027+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:52:07.718646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:52:07.718646+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:37.805116+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.00095558800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":222,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:35.938723+0000","last_change":"2024-09-20T18:51:59.796450+0000","last_active":"2024-09-20T18:54:35.938723+0000","last_peered":"2024-09-20T18:54:35.938723+0000","last_clean":"2024-09-20T18:54:35.938723+0000","last_became_active":"2024-09-20T18:49:49.954424+0000","last_became_peered":"2024-09-20T18:49:49.954424+0000","last_unstale":"2024-09-20T18:54:35.938723+0000","last_undegraded":"2024-09-20T18:54:35.938723+0000","last_fullsized":"2024-09-20T18:54:35.938723+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:59.796383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:59.796383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:06.819934+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":218,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:38.158513+0000","last_change":"2024-09-20T18:52:39.814324+0000","last_active":"2024-09-20T18:54:38.158513+0000","last_peered":"2024-09-20T18:54:38.158513+0000","last_clean":"2024-09-20T18:54:38.158513+0000","last_became_active":"2024-09-20T18:49:52.139789+0000","last_became_peered":"2024-09-20T18:49:52.139789+0000","last_unstale":"2024-09-20T18:54:38.158513+0000","last_undegraded":"2024-09-20T18:54:38.158513+0000","last_fullsized":"2024-09-20T18:54:38.158513+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:39.814259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:39.814259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:05.336625+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-20T18:54:34.923583+0000","last_change":"2024-09-20T18:54:34.311774+0000","last_active":"2024-09-20T18:54:34.923583+0000","last_peered":"2024-09-20T18:54:34.923583+0000","last_clean":"2024-09-20T18:54:34.923583+0000","last_became_active":"2024-09-20T18:54:34.311487+0000","last_became_peered":"2024-09-20T18:54:34.311487+0000","last_unstale":"2024-09-20T18:54:34.923583+0000","last_undegraded":"2024-09-20T18:54:34.923583+0000","last_fullsized":"2024-09-20T18:54:34.923583+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:51.921461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":258,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:35.938483+0000","last_change":"2024-09-20T18:53:47.786753+0000","last_active":"2024-09-20T18:54:35.938483+0000","last_peered":"2024-09-20T18:54:35.938483+0000","last_clean":"2024-09-20T18:54:35.938483+0000","last_became_active":"2024-09-20T18:49:47.949230+0000","last_became_peered":"2024-09-20T18:49:47.949230+0000","last_unstale":"2024-09-20T18:54:35.938483+0000","last_undegraded":"2024-09-20T18:54:35.938483+0000","last_fullsized":"2024-09-20T18:54:35.938483+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:53:47.786681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:47.786681+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:54.774464+0000","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":244,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:35.937465+0000","last_change":"2024-09-20T18:54:27.905856+0000","last_active":"2024-09-20T18:54:35.937465+0000","last_peered":"2024-09-20T18:54:35.937465+0000","last_clean":"2024-09-20T18:54:35.937465+0000","last_became_active":"2024-09-20T18:49:45.939197+0000","last_became_peered":"2024-09-20T18:49:45.939197+0000","last_unstale":"2024-09-20T18:54:35.937465+0000","last_undegraded":"2024-09-20T18:54:35.937465+0000","last_fullsized":"2024-09-20T18:54:35.937465+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:27.905803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:27.905803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:44.644028+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":547,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:39.648423+0000","last_change":"2024-09-20T18:54:26.359887+0000","last_active":"2024-09-20T18:54:39.648423+0000","last_peered":"2024-09-20T18:54:39.648423+0000","last_clean":"2024-09-20T18:54:39.648423+0000","last_became_active":"2024-09-20T18:49:44.936819+0000","last_became_peered":"2024-09-20T18:49:44.936819+0000","last_unstale":"2024-09-20T18:54:39.648423+0000","last_undegraded":"2024-09-20T18:54:39.648423+0000","last_fullsized":"2024-09-20T18:54:39.648423+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-20T18:54:26.359807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:54:26.359807+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:31.668091+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0010996199999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":290,"num_read_kb":285,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":222,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:38.271905+0000","last_change":"2024-09-20T18:52:02.981845+0000","last_active":"2024-09-20T18:54:38.271905+0000","last_peered":"2024-09-20T18:54:38.271905+0000","last_clean":"2024-09-20T18:54:38.271905+0000","last_became_active":"2024-09-20T18:49:49.961066+0000","last_became_peered":"2024-09-20T18:49:49.961066+0000","last_unstale":"2024-09-20T18:54:38.271905+0000","last_undegraded":"2024-09-20T18:54:38.271905+0000","last_fullsized":"2024-09-20T18:54:38.271905+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:02.981762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:02.981762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:58.477818+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":223,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-20T18:54:34.923517+0000","last_change":"2024-09-20T18:53:49.227504+0000","last_active":"2024-09-20T18:54:34.923517+0000","last_peered":"2024-09-20T18:54:34.923517+0000","last_clean":"2024-09-20T18:54:34.923517+0000","last_became_active":"2024-09-20T18:49:51.970343+0000","last_became_peered":"2024-09-20T18:49:51.970343+0000","last_unstale":"2024-09-20T18:54:34.923517+0000","last_undegraded":"2024-09-20T18:54:34.923517+0000","last_fullsized":"2024-09-20T18:54:34.923517+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:49.227436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:49.227436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:07.422442+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"164'117","reported_seq":135,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:40.837675+0000","last_change":"2024-09-20T18:54:33.928788+0000","last_active":"2024-09-20T18:54:40.837675+0000","last_peered":"2024-09-20T18:54:40.837675+0000","last_clean":"2024-09-20T18:54:40.837675+0000","last_became_active":"2024-09-20T18:54:33.928422+0000","last_became_peered":"2024-09-20T18:54:33.928422+0000","last_unstale":"2024-09-20T18:54:40.837675+0000","last_undegraded":"2024-09-20T18:54:40.837675+0000","last_fullsized":"2024-09-20T18:54:40.837675+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:38.755459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7667712,"num_objects":117,"num_object_clones":0,"num_object_copies":351,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":117,"num_write_kb":7488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":299,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:38.018293+0000","last_change":"2024-09-20T18:49:48.498345+0000","last_active":"2024-09-20T18:54:38.018293+0000","last_peered":"2024-09-20T18:54:38.018293+0000","last_clean":"2024-09-20T18:54:38.018293+0000","last_became_active":"2024-09-20T18:49:48.497936+0000","last_became_peered":"2024-09-20T18:49:48.497936+0000","last_unstale":"2024-09-20T18:54:38.018293+0000","last_undegraded":"2024-09-20T18:54:38.018293+0000","last_fullsized":"2024-09-20T18:54:38.018293+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:59.631793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":237,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:36.183591+0000","last_change":"2024-09-20T18:53:13.145127+0000","last_active":"2024-09-20T18:54:36.183591+0000","last_peered":"2024-09-20T18:54:36.183591+0000","last_clean":"2024-09-20T18:54:36.183591+0000","last_became_active":"2024-09-20T18:49:45.941136+0000","last_became_peered":"2024-09-20T18:49:45.941136+0000","last_unstale":"2024-09-20T18:54:36.183591+0000","last_undegraded":"2024-09-20T18:54:36.183591+0000","last_fullsized":"2024-09-20T18:54:36.183591+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:13.144896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:13.144896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":277,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:38.158612+0000","last_change":"2024-09-20T18:52:11.887878+0000","last_active":"2024-09-20T18:54:38.158612+0000","last_peered":"2024-09-20T18:54:38.158612+0000","last_clean":"2024-09-20T18:54:38.158612+0000","last_became_active":"2024-09-20T18:49:36.545306+0000","last_became_peered":"2024-09-20T18:49:36.545306+0000","last_unstale":"2024-09-20T18:54:38.158612+0000","last_undegraded":"2024-09-20T18:54:38.158612+0000","last_fullsized":"2024-09-20T18:54:38.158612+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T18:52:11.887733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T18:52:11.887733+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:37.449933+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":222,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:38.157464+0000","last_change":"2024-09-20T18:51:22.852942+0000","last_active":"2024-09-20T18:54:38.157464+0000","last_peered":"2024-09-20T18:54:38.157464+0000","last_clean":"2024-09-20T18:54:38.157464+0000","last_became_active":"2024-09-20T18:49:49.956128+0000","last_became_peered":"2024-09-20T18:49:49.956128+0000","last_unstale":"2024-09-20T18:54:38.157464+0000","last_undegraded":"2024-09-20T18:54:38.157464+0000","last_fullsized":"2024-09-20T18:54:38.157464+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:22.852869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:22.852869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":226,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:38.273059+0000","last_change":"2024-09-20T18:53:45.306565+0000","last_active":"2024-09-20T18:54:38.273059+0000","last_peered":"2024-09-20T18:54:38.273059+0000","last_clean":"2024-09-20T18:54:38.273059+0000","last_became_active":"2024-09-20T18:49:51.971332+0000","last_became_peered":"2024-09-20T18:49:51.971332+0000","last_unstale":"2024-09-20T18:54:38.273059+0000","last_undegraded":"2024-09-20T18:54:38.273059+0000","last_fullsized":"2024-09-20T18:54:38.273059+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:45.306514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:45.306514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:10.093423+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"164'114","reported_seq":133,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:40.778426+0000","last_change":"2024-09-20T18:54:33.929366+0000","last_active":"2024-09-20T18:54:40.778426+0000","last_peered":"2024-09-20T18:54:40.778426+0000","last_clean":"2024-09-20T18:54:40.778426+0000","last_became_active":"2024-09-20T18:54:33.929072+0000","last_became_peered":"2024-09-20T18:54:33.929072+0000","last_unstale":"2024-09-20T18:54:40.778426+0000","last_undegraded":"2024-09-20T18:54:40.778426+0000","last_fullsized":"2024-09-20T18:54:40.778426+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:50.299673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7471104,"num_objects":114,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114,"num_write_kb":7296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":246,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:37.956379+0000","last_change":"2024-09-20T18:51:14.397996+0000","last_active":"2024-09-20T18:54:37.956379+0000","last_peered":"2024-09-20T18:54:37.956379+0000","last_clean":"2024-09-20T18:54:37.956379+0000","last_became_active":"2024-09-20T18:49:47.950073+0000","last_became_peered":"2024-09-20T18:49:47.950073+0000","last_unstale":"2024-09-20T18:54:37.956379+0000","last_undegraded":"2024-09-20T18:54:37.956379+0000","last_fullsized":"2024-09-20T18:54:37.956379+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:51:14.397930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.397930+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:36.090967+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":231,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:37.955608+0000","last_change":"2024-09-20T18:51:18.410601+0000","last_active":"2024-09-20T18:54:37.955608+0000","last_peered":"2024-09-20T18:54:37.955608+0000","last_clean":"2024-09-20T18:54:37.955608+0000","last_became_active":"2024-09-20T18:49:45.944579+0000","last_became_peered":"2024-09-20T18:49:45.944579+0000","last_unstale":"2024-09-20T18:54:37.955608+0000","last_undegraded":"2024-09-20T18:54:37.955608+0000","last_fullsized":"2024-09-20T18:54:37.955608+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:18.410447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:18.410447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:30.148528+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":238,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:38.271914+0000","last_change":"2024-09-20T18:54:11.401857+0000","last_active":"2024-09-20T18:54:38.271914+0000","last_peered":"2024-09-20T18:54:38.271914+0000","last_clean":"2024-09-20T18:54:38.271914+0000","last_became_active":"2024-09-20T18:49:49.954391+0000","last_became_peered":"2024-09-20T18:49:49.954391+0000","last_unstale":"2024-09-20T18:54:38.271914+0000","last_undegraded":"2024-09-20T18:54:38.271914+0000","last_fullsized":"2024-09-20T18:54:38.271914+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:11.401777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:11.401777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:15.740623+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":217,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:36.183594+0000","last_change":"2024-09-20T18:52:05.021228+0000","last_active":"2024-09-20T18:54:36.183594+0000","last_peered":"2024-09-20T18:54:36.183594+0000","last_clean":"2024-09-20T18:54:36.183594+0000","last_became_active":"2024-09-20T18:49:52.139959+0000","last_became_peered":"2024-09-20T18:49:52.139959+0000","last_unstale":"2024-09-20T18:54:36.183594+0000","last_undegraded":"2024-09-20T18:54:36.183594+0000","last_fullsized":"2024-09-20T18:54:36.183594+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:05.021071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.021071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:01.216837+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-20T18:54:34.923208+0000","last_change":"2024-09-20T18:54:34.310237+0000","last_active":"2024-09-20T18:54:34.923208+0000","last_peered":"2024-09-20T18:54:34.923208+0000","last_clean":"2024-09-20T18:54:34.923208+0000","last_became_active":"2024-09-20T18:54:34.309781+0000","last_became_peered":"2024-09-20T18:54:34.309781+0000","last_unstale":"2024-09-20T18:54:34.923208+0000","last_undegraded":"2024-09-20T18:54:34.923208+0000","last_fullsized":"2024-09-20T18:54:34.923208+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:53.715983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":263,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:37.956135+0000","last_change":"2024-09-20T18:52:02.944536+0000","last_active":"2024-09-20T18:54:37.956135+0000","last_peered":"2024-09-20T18:54:37.956135+0000","last_clean":"2024-09-20T18:54:37.956135+0000","last_became_active":"2024-09-20T18:49:47.947636+0000","last_became_peered":"2024-09-20T18:49:47.947636+0000","last_unstale":"2024-09-20T18:54:37.956135+0000","last_undegraded":"2024-09-20T18:54:37.956135+0000","last_fullsized":"2024-09-20T18:54:37.956135+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-20T18:52:02.944434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:02.944434+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:31.181686+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":252,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:38.017980+0000","last_change":"2024-09-20T18:53:23.472809+0000","last_active":"2024-09-20T18:54:38.017980+0000","last_peered":"2024-09-20T18:54:38.017980+0000","last_clean":"2024-09-20T18:54:38.017980+0000","last_became_active":"2024-09-20T18:51:55.596689+0000","last_became_peered":"2024-09-20T18:51:55.596689+0000","last_unstale":"2024-09-20T18:54:38.017980+0000","last_undegraded":"2024-09-20T18:54:38.017980+0000","last_fullsized":"2024-09-20T18:54:38.017980+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:23.472757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:23.472757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:39.082278+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":227,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-20T18:54:34.923607+0000","last_change":"2024-09-20T18:54:11.377907+0000","last_active":"2024-09-20T18:54:34.923607+0000","last_peered":"2024-09-20T18:54:34.923607+0000","last_clean":"2024-09-20T18:54:34.923607+0000","last_became_active":"2024-09-20T18:49:49.959160+0000","last_became_peered":"2024-09-20T18:49:49.959160+0000","last_unstale":"2024-09-20T18:54:34.923607+0000","last_undegraded":"2024-09-20T18:54:34.923607+0000","last_fullsized":"2024-09-20T18:54:34.923607+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:11.376951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:11.376951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:35.227288+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":226,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:35.938112+0000","last_change":"2024-09-20T18:52:35.635897+0000","last_active":"2024-09-20T18:54:35.938112+0000","last_peered":"2024-09-20T18:54:35.938112+0000","last_clean":"2024-09-20T18:54:35.938112+0000","last_became_active":"2024-09-20T18:49:51.970108+0000","last_became_peered":"2024-09-20T18:49:51.970108+0000","last_unstale":"2024-09-20T18:54:35.938112+0000","last_undegraded":"2024-09-20T18:54:35.938112+0000","last_fullsized":"2024-09-20T18:54:35.938112+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:35.635818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:35.635818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:56.872031+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":16,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:36.183226+0000","last_change":"2024-09-20T18:54:33.929552+0000","last_active":"2024-09-20T18:54:36.183226+0000","last_peered":"2024-09-20T18:54:36.183226+0000","last_clean":"2024-09-20T18:54:36.183226+0000","last_became_active":"2024-09-20T18:54:33.929208+0000","last_became_peered":"2024-09-20T18:54:33.929208+0000","last_unstale":"2024-09-20T18:54:36.183226+0000","last_undegraded":"2024-09-20T18:54:36.183226+0000","last_fullsized":"2024-09-20T18:54:36.183226+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:57.831114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":250,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:37.955726+0000","last_change":"2024-09-20T18:54:17.219635+0000","last_active":"2024-09-20T18:54:37.955726+0000","last_peered":"2024-09-20T18:54:37.955726+0000","last_clean":"2024-09-20T18:54:37.955726+0000","last_became_active":"2024-09-20T18:49:47.947126+0000","last_became_peered":"2024-09-20T18:49:47.947126+0000","last_unstale":"2024-09-20T18:54:37.955726+0000","last_undegraded":"2024-09-20T18:54:37.955726+0000","last_fullsized":"2024-09-20T18:54:37.955726+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:54:17.219548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:17.219548+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:45.354404+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":237,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:36.183757+0000","last_change":"2024-09-20T18:52:46.005172+0000","last_active":"2024-09-20T18:54:36.183757+0000","last_peered":"2024-09-20T18:54:36.183757+0000","last_clean":"2024-09-20T18:54:36.183757+0000","last_became_active":"2024-09-20T18:49:45.937264+0000","last_became_peered":"2024-09-20T18:49:45.937264+0000","last_unstale":"2024-09-20T18:54:36.183757+0000","last_undegraded":"2024-09-20T18:54:36.183757+0000","last_fullsized":"2024-09-20T18:54:36.183757+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:46.005109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:46.005109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:32.252493+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":297,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:35.937286+0000","last_change":"2024-09-20T18:52:40.647833+0000","last_active":"2024-09-20T18:54:35.937286+0000","last_peered":"2024-09-20T18:54:35.937286+0000","last_clean":"2024-09-20T18:54:35.937286+0000","last_became_active":"2024-09-20T18:49:49.957005+0000","last_became_peered":"2024-09-20T18:49:49.957005+0000","last_unstale":"2024-09-20T18:54:35.937286+0000","last_undegraded":"2024-09-20T18:54:35.937286+0000","last_fullsized":"2024-09-20T18:54:35.937286+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:52:40.647761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:40.647761+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:07.843426+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":219,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:37.956087+0000","last_change":"2024-09-20T18:52:10.024610+0000","last_active":"2024-09-20T18:54:37.956087+0000","last_peered":"2024-09-20T18:54:37.956087+0000","last_clean":"2024-09-20T18:54:37.956087+0000","last_became_active":"2024-09-20T18:49:51.969355+0000","last_became_peered":"2024-09-20T18:49:51.969355+0000","last_unstale":"2024-09-20T18:54:37.956087+0000","last_undegraded":"2024-09-20T18:54:37.956087+0000","last_fullsized":"2024-09-20T18:54:37.956087+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:10.024557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:10.024557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:38.850775+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":14,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-20T18:54:34.924044+0000","last_change":"2024-09-20T18:54:33.928333+0000","last_active":"2024-09-20T18:54:34.924044+0000","last_peered":"2024-09-20T18:54:34.924044+0000","last_clean":"2024-09-20T18:54:34.924044+0000","last_became_active":"2024-09-20T18:54:33.927986+0000","last_became_peered":"2024-09-20T18:54:33.927986+0000","last_unstale":"2024-09-20T18:54:34.924044+0000","last_undegraded":"2024-09-20T18:54:34.924044+0000","last_fullsized":"2024-09-20T18:54:34.924044+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:49.567842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":248,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:35.936696+0000","last_change":"2024-09-20T18:53:17.863709+0000","last_active":"2024-09-20T18:54:35.936696+0000","last_peered":"2024-09-20T18:54:35.936696+0000","last_clean":"2024-09-20T18:54:35.936696+0000","last_became_active":"2024-09-20T18:49:47.946008+0000","last_became_peered":"2024-09-20T18:49:47.946008+0000","last_unstale":"2024-09-20T18:54:35.936696+0000","last_undegraded":"2024-09-20T18:54:35.936696+0000","last_fullsized":"2024-09-20T18:54:35.936696+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:53:17.863623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:17.863623+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":238,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:38.272960+0000","last_change":"2024-09-20T18:52:17.120051+0000","last_active":"2024-09-20T18:54:38.272960+0000","last_peered":"2024-09-20T18:54:38.272960+0000","last_clean":"2024-09-20T18:54:38.272960+0000","last_became_active":"2024-09-20T18:49:45.941406+0000","last_became_peered":"2024-09-20T18:49:45.941406+0000","last_unstale":"2024-09-20T18:54:38.272960+0000","last_undegraded":"2024-09-20T18:54:38.272960+0000","last_fullsized":"2024-09-20T18:54:38.272960+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:17.119986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:17.119986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:55.385828+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":230,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:35.937352+0000","last_change":"2024-09-20T18:53:27.722068+0000","last_active":"2024-09-20T18:54:35.937352+0000","last_peered":"2024-09-20T18:54:35.937352+0000","last_clean":"2024-09-20T18:54:35.937352+0000","last_became_active":"2024-09-20T18:49:49.956767+0000","last_became_peered":"2024-09-20T18:49:49.956767+0000","last_unstale":"2024-09-20T18:54:35.937352+0000","last_undegraded":"2024-09-20T18:54:35.937352+0000","last_fullsized":"2024-09-20T18:54:35.937352+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:27.721940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:27.721940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:42.447352+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":224,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:35.937398+0000","last_change":"2024-09-20T18:54:24.976627+0000","last_active":"2024-09-20T18:54:35.937398+0000","last_peered":"2024-09-20T18:54:35.937398+0000","last_clean":"2024-09-20T18:54:35.937398+0000","last_became_active":"2024-09-20T18:49:51.964098+0000","last_became_peered":"2024-09-20T18:49:51.964098+0000","last_unstale":"2024-09-20T18:54:35.937398+0000","last_undegraded":"2024-09-20T18:54:35.937398+0000","last_fullsized":"2024-09-20T18:54:35.937398+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:24.976584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:24.976584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:48.697647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":16,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:35.938151+0000","last_change":"2024-09-20T18:54:34.311509+0000","last_active":"2024-09-20T18:54:35.938151+0000","last_peered":"2024-09-20T18:54:35.938151+0000","last_clean":"2024-09-20T18:54:35.938151+0000","last_became_active":"2024-09-20T18:54:34.311203+0000","last_became_peered":"2024-09-20T18:54:34.311203+0000","last_unstale":"2024-09-20T18:54:35.938151+0000","last_undegraded":"2024-09-20T18:54:35.938151+0000","last_fullsized":"2024-09-20T18:54:35.938151+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:49.200605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":265,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:38.017764+0000","last_change":"2024-09-20T18:53:24.458093+0000","last_active":"2024-09-20T18:54:38.017764+0000","last_peered":"2024-09-20T18:54:38.017764+0000","last_clean":"2024-09-20T18:54:38.017764+0000","last_became_active":"2024-09-20T18:49:48.498625+0000","last_became_peered":"2024-09-20T18:49:48.498625+0000","last_unstale":"2024-09-20T18:54:38.017764+0000","last_undegraded":"2024-09-20T18:54:38.017764+0000","last_fullsized":"2024-09-20T18:54:38.017764+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-20T18:53:24.458005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:24.458005+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:31.731536+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":238,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:38.158748+0000","last_change":"2024-09-20T18:53:20.564464+0000","last_active":"2024-09-20T18:54:38.158748+0000","last_peered":"2024-09-20T18:54:38.158748+0000","last_clean":"2024-09-20T18:54:38.158748+0000","last_became_active":"2024-09-20T18:49:45.941448+0000","last_became_peered":"2024-09-20T18:49:45.941448+0000","last_unstale":"2024-09-20T18:54:38.158748+0000","last_undegraded":"2024-09-20T18:54:38.158748+0000","last_fullsized":"2024-09-20T18:54:38.158748+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:20.564413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:20.564413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:38.260460+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":230,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:35.936628+0000","last_change":"2024-09-20T18:52:36.662703+0000","last_active":"2024-09-20T18:54:35.936628+0000","last_peered":"2024-09-20T18:54:35.936628+0000","last_clean":"2024-09-20T18:54:35.936628+0000","last_became_active":"2024-09-20T18:49:49.957558+0000","last_became_peered":"2024-09-20T18:49:49.957558+0000","last_unstale":"2024-09-20T18:54:35.936628+0000","last_undegraded":"2024-09-20T18:54:35.936628+0000","last_fullsized":"2024-09-20T18:54:35.936628+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:36.662524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:36.662524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:58.801684+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":234,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:38.158451+0000","last_change":"2024-09-20T18:53:47.370152+0000","last_active":"2024-09-20T18:54:38.158451+0000","last_peered":"2024-09-20T18:54:38.158451+0000","last_clean":"2024-09-20T18:54:38.158451+0000","last_became_active":"2024-09-20T18:49:51.970798+0000","last_became_peered":"2024-09-20T18:49:51.970798+0000","last_unstale":"2024-09-20T18:54:38.158451+0000","last_undegraded":"2024-09-20T18:54:38.158451+0000","last_fullsized":"2024-09-20T18:54:38.158451+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:47.370108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:47.370108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:53.917763+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"164'128","reported_seq":147,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:40.827197+0000","last_change":"2024-09-20T18:54:34.311996+0000","last_active":"2024-09-20T18:54:40.827197+0000","last_peered":"2024-09-20T18:54:40.827197+0000","last_clean":"2024-09-20T18:54:40.827197+0000","last_became_active":"2024-09-20T18:54:34.311675+0000","last_became_peered":"2024-09-20T18:54:34.311675+0000","last_unstale":"2024-09-20T18:54:40.827197+0000","last_undegraded":"2024-09-20T18:54:40.827197+0000","last_fullsized":"2024-09-20T18:54:40.827197+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:57.294768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":128,"num_object_clones":0,"num_object_copies":384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":128,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":128,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":258,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:37.955457+0000","last_change":"2024-09-20T18:53:52.898818+0000","last_active":"2024-09-20T18:54:37.955457+0000","last_peered":"2024-09-20T18:54:37.955457+0000","last_clean":"2024-09-20T18:54:37.955457+0000","last_became_active":"2024-09-20T18:49:47.949116+0000","last_became_peered":"2024-09-20T18:49:47.949116+0000","last_unstale":"2024-09-20T18:54:37.955457+0000","last_undegraded":"2024-09-20T18:54:37.955457+0000","last_fullsized":"2024-09-20T18:54:37.955457+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:53:52.898760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:52.898760+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:53.100006+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":238,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:38.158625+0000","last_change":"2024-09-20T18:53:21.578905+0000","last_active":"2024-09-20T18:54:38.158625+0000","last_peered":"2024-09-20T18:54:38.158625+0000","last_clean":"2024-09-20T18:54:38.158625+0000","last_became_active":"2024-09-20T18:49:45.944858+0000","last_became_peered":"2024-09-20T18:49:45.944858+0000","last_unstale":"2024-09-20T18:54:38.158625+0000","last_undegraded":"2024-09-20T18:54:38.158625+0000","last_fullsized":"2024-09-20T18:54:38.158625+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:21.578845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:21.578845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:47.899680+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":222,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:35.936620+0000","last_change":"2024-09-20T18:52:12.674502+0000","last_active":"2024-09-20T18:54:35.936620+0000","last_peered":"2024-09-20T18:54:35.936620+0000","last_clean":"2024-09-20T18:54:35.936620+0000","last_became_active":"2024-09-20T18:49:49.954944+0000","last_became_peered":"2024-09-20T18:49:49.954944+0000","last_unstale":"2024-09-20T18:54:35.936620+0000","last_undegraded":"2024-09-20T18:54:35.936620+0000","last_fullsized":"2024-09-20T18:54:35.936620+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:12.674203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:12.674203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:41.868905+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":217,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:36.183582+0000","last_change":"2024-09-20T18:52:09.017828+0000","last_active":"2024-09-20T18:54:36.183582+0000","last_peered":"2024-09-20T18:54:36.183582+0000","last_clean":"2024-09-20T18:54:36.183582+0000","last_became_active":"2024-09-20T18:49:51.972970+0000","last_became_peered":"2024-09-20T18:49:51.972970+0000","last_unstale":"2024-09-20T18:54:36.183582+0000","last_undegraded":"2024-09-20T18:54:36.183582+0000","last_fullsized":"2024-09-20T18:54:36.183582+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:09.017759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:09.017759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:02.178343+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.4","version":"164'132","reported_seq":151,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:40.817219+0000","last_change":"2024-09-20T18:54:34.309442+0000","last_active":"2024-09-20T18:54:40.817219+0000","last_peered":"2024-09-20T18:54:40.817219+0000","last_clean":"2024-09-20T18:54:40.817219+0000","last_became_active":"2024-09-20T18:54:34.309113+0000","last_became_peered":"2024-09-20T18:54:34.309113+0000","last_unstale":"2024-09-20T18:54:40.817219+0000","last_undegraded":"2024-09-20T18:54:40.817219+0000","last_fullsized":"2024-09-20T18:54:40.817219+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:57.624679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8650752,"num_objects":132,"num_object_clones":0,"num_object_copies":396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":132,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":8448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":259,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:38.272302+0000","last_change":"2024-09-20T18:54:12.373389+0000","last_active":"2024-09-20T18:54:38.272302+0000","last_peered":"2024-09-20T18:54:38.272302+0000","last_clean":"2024-09-20T18:54:38.272302+0000","last_became_active":"2024-09-20T18:49:48.499847+0000","last_became_peered":"2024-09-20T18:49:48.499847+0000","last_unstale":"2024-09-20T18:54:38.272302+0000","last_undegraded":"2024-09-20T18:54:38.272302+0000","last_fullsized":"2024-09-20T18:54:38.272302+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:54:12.373309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:12.373309+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:26.708173+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":258,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:38.158697+0000","last_change":"2024-09-20T18:54:19.403828+0000","last_active":"2024-09-20T18:54:38.158697+0000","last_peered":"2024-09-20T18:54:38.158697+0000","last_clean":"2024-09-20T18:54:38.158697+0000","last_became_active":"2024-09-20T18:49:45.944256+0000","last_became_peered":"2024-09-20T18:49:45.944256+0000","last_unstale":"2024-09-20T18:54:38.158697+0000","last_undegraded":"2024-09-20T18:54:38.158697+0000","last_fullsized":"2024-09-20T18:54:38.158697+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-20T18:54:19.403731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:19.403731+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:21.004220+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":289,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:38.157439+0000","last_change":"2024-09-20T18:52:05.788988+0000","last_active":"2024-09-20T18:54:38.157439+0000","last_peered":"2024-09-20T18:54:38.157439+0000","last_clean":"2024-09-20T18:54:38.157439+0000","last_became_active":"2024-09-20T18:49:49.958411+0000","last_became_peered":"2024-09-20T18:49:49.958411+0000","last_unstale":"2024-09-20T18:54:38.157439+0000","last_undegraded":"2024-09-20T18:54:38.157439+0000","last_fullsized":"2024-09-20T18:54:38.157439+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:52:05.788924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.788924+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:39.468023+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":218,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:38.272368+0000","last_change":"2024-09-20T18:51:56.970415+0000","last_active":"2024-09-20T18:54:38.272368+0000","last_peered":"2024-09-20T18:54:38.272368+0000","last_clean":"2024-09-20T18:54:38.272368+0000","last_became_active":"2024-09-20T18:49:52.139685+0000","last_became_peered":"2024-09-20T18:49:52.139685+0000","last_unstale":"2024-09-20T18:54:38.272368+0000","last_undegraded":"2024-09-20T18:54:38.272368+0000","last_fullsized":"2024-09-20T18:54:38.272368+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:56.970349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:56.970349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:08.388529+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":246,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:38.017597+0000","last_change":"2024-09-20T18:54:18.763523+0000","last_active":"2024-09-20T18:54:38.017597+0000","last_peered":"2024-09-20T18:54:38.017597+0000","last_clean":"2024-09-20T18:54:38.017597+0000","last_became_active":"2024-09-20T18:49:45.940060+0000","last_became_peered":"2024-09-20T18:49:45.940060+0000","last_unstale":"2024-09-20T18:54:38.017597+0000","last_undegraded":"2024-09-20T18:54:38.017597+0000","last_fullsized":"2024-09-20T18:54:38.017597+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:18.763474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:18.763474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:24.982459+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":233,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:36.184083+0000","last_change":"2024-09-20T18:53:27.186982+0000","last_active":"2024-09-20T18:54:36.184083+0000","last_peered":"2024-09-20T18:54:36.184083+0000","last_clean":"2024-09-20T18:54:36.184083+0000","last_became_active":"2024-09-20T18:49:48.500662+0000","last_became_peered":"2024-09-20T18:49:48.500662+0000","last_unstale":"2024-09-20T18:54:36.184083+0000","last_undegraded":"2024-09-20T18:54:36.184083+0000","last_fullsized":"2024-09-20T18:54:36.184083+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-20T18:53:27.186631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:27.186631+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:42.463558+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":230,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:38.272081+0000","last_change":"2024-09-20T18:53:16.220554+0000","last_active":"2024-09-20T18:54:38.272081+0000","last_peered":"2024-09-20T18:54:38.272081+0000","last_clean":"2024-09-20T18:54:38.272081+0000","last_became_active":"2024-09-20T18:49:49.956749+0000","last_became_peered":"2024-09-20T18:49:49.956749+0000","last_unstale":"2024-09-20T18:54:38.272081+0000","last_undegraded":"2024-09-20T18:54:38.272081+0000","last_fullsized":"2024-09-20T18:54:38.272081+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:16.220490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:16.220490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:38.270912+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":225,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:38.158487+0000","last_change":"2024-09-20T18:53:19.591395+0000","last_active":"2024-09-20T18:54:38.158487+0000","last_peered":"2024-09-20T18:54:38.158487+0000","last_clean":"2024-09-20T18:54:38.158487+0000","last_became_active":"2024-09-20T18:49:52.142291+0000","last_became_peered":"2024-09-20T18:49:52.142291+0000","last_unstale":"2024-09-20T18:54:38.158487+0000","last_undegraded":"2024-09-20T18:54:38.158487+0000","last_fullsized":"2024-09-20T18:54:38.158487+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:19.591346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:19.591346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:40.474867+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":239,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:35.938582+0000","last_change":"2024-09-20T18:51:23.065382+0000","last_active":"2024-09-20T18:54:35.938582+0000","last_peered":"2024-09-20T18:54:35.938582+0000","last_clean":"2024-09-20T18:54:35.938582+0000","last_became_active":"2024-09-20T18:49:48.500803+0000","last_became_peered":"2024-09-20T18:49:48.500803+0000","last_unstale":"2024-09-20T18:54:35.938582+0000","last_undegraded":"2024-09-20T18:54:35.938582+0000","last_fullsized":"2024-09-20T18:54:35.938582+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:51:23.064536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:23.064536+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:39.863201+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":230,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:38.158187+0000","last_change":"2024-09-20T18:51:30.982535+0000","last_active":"2024-09-20T18:54:38.158187+0000","last_peered":"2024-09-20T18:54:38.158187+0000","last_clean":"2024-09-20T18:54:38.158187+0000","last_became_active":"2024-09-20T18:49:45.946600+0000","last_became_peered":"2024-09-20T18:49:45.946600+0000","last_unstale":"2024-09-20T18:54:38.158187+0000","last_undegraded":"2024-09-20T18:54:38.158187+0000","last_fullsized":"2024-09-20T18:54:38.158187+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:30.982454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:30.982454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:04.374723+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":228,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:36.184003+0000","last_change":"2024-09-20T18:53:50.253721+0000","last_active":"2024-09-20T18:54:36.184003+0000","last_peered":"2024-09-20T18:54:36.184003+0000","last_clean":"2024-09-20T18:54:36.184003+0000","last_became_active":"2024-09-20T18:49:49.961947+0000","last_became_peered":"2024-09-20T18:49:49.961947+0000","last_unstale":"2024-09-20T18:54:36.184003+0000","last_undegraded":"2024-09-20T18:54:36.184003+0000","last_fullsized":"2024-09-20T18:54:36.184003+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:50.253660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:50.253660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:15.284040+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"66'1","reported_seq":235,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:35.936559+0000","last_change":"2024-09-20T18:54:20.681878+0000","last_active":"2024-09-20T18:54:35.936559+0000","last_peered":"2024-09-20T18:54:35.936559+0000","last_clean":"2024-09-20T18:54:35.936559+0000","last_became_active":"2024-09-20T18:49:51.965121+0000","last_became_peered":"2024-09-20T18:49:51.965121+0000","last_unstale":"2024-09-20T18:54:35.936559+0000","last_undegraded":"2024-09-20T18:54:35.936559+0000","last_fullsized":"2024-09-20T18:54:35.936559+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T18:54:20.681790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:20.681790+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:31.186086+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":234,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:38.272889+0000","last_change":"2024-09-20T18:49:48.501359+0000","last_active":"2024-09-20T18:54:38.272889+0000","last_peered":"2024-09-20T18:54:38.272889+0000","last_clean":"2024-09-20T18:54:38.272889+0000","last_became_active":"2024-09-20T18:49:48.501011+0000","last_became_peered":"2024-09-20T18:49:48.501011+0000","last_unstale":"2024-09-20T18:54:38.272889+0000","last_undegraded":"2024-09-20T18:54:38.272889+0000","last_fullsized":"2024-09-20T18:54:38.272889+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:05.423302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":236,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:35.936645+0000","last_change":"2024-09-20T18:54:11.896547+0000","last_active":"2024-09-20T18:54:35.936645+0000","last_peered":"2024-09-20T18:54:35.936645+0000","last_clean":"2024-09-20T18:54:35.936645+0000","last_became_active":"2024-09-20T18:49:45.938511+0000","last_became_peered":"2024-09-20T18:49:45.938511+0000","last_unstale":"2024-09-20T18:54:35.936645+0000","last_undegraded":"2024-09-20T18:54:35.936645+0000","last_fullsized":"2024-09-20T18:54:35.936645+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:11.896470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:11.896470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:22.211500+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":296,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:36.183747+0000","last_change":"2024-09-20T18:53:47.337912+0000","last_active":"2024-09-20T18:54:36.183747+0000","last_peered":"2024-09-20T18:54:36.183747+0000","last_clean":"2024-09-20T18:54:36.183747+0000","last_became_active":"2024-09-20T18:49:49.960550+0000","last_became_peered":"2024-09-20T18:49:49.960550+0000","last_unstale":"2024-09-20T18:54:36.183747+0000","last_undegraded":"2024-09-20T18:54:36.183747+0000","last_fullsized":"2024-09-20T18:54:36.183747+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:53:47.337658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:47.337658+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:01.412695+0000","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":226,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:37.956241+0000","last_change":"2024-09-20T18:53:48.839130+0000","last_active":"2024-09-20T18:54:37.956241+0000","last_peered":"2024-09-20T18:54:37.956241+0000","last_clean":"2024-09-20T18:54:37.956241+0000","last_became_active":"2024-09-20T18:49:51.966254+0000","last_became_peered":"2024-09-20T18:49:51.966254+0000","last_unstale":"2024-09-20T18:54:37.956241+0000","last_undegraded":"2024-09-20T18:54:37.956241+0000","last_fullsized":"2024-09-20T18:54:37.956241+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:48.839015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:48.839015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:07.481453+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":259,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:36.183817+0000","last_change":"2024-09-20T18:51:15.223913+0000","last_active":"2024-09-20T18:54:36.183817+0000","last_peered":"2024-09-20T18:54:36.183817+0000","last_clean":"2024-09-20T18:54:36.183817+0000","last_became_active":"2024-09-20T18:49:58.997494+0000","last_became_peered":"2024-09-20T18:49:58.997494+0000","last_unstale":"2024-09-20T18:54:36.183817+0000","last_undegraded":"2024-09-20T18:54:36.183817+0000","last_fullsized":"2024-09-20T18:54:36.183817+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:51:15.223826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:15.223826+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:07.187139+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":238,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:38.158261+0000","last_change":"2024-09-20T18:53:27.589481+0000","last_active":"2024-09-20T18:54:38.158261+0000","last_peered":"2024-09-20T18:54:38.158261+0000","last_clean":"2024-09-20T18:54:38.158261+0000","last_became_active":"2024-09-20T18:49:45.949492+0000","last_became_peered":"2024-09-20T18:49:45.949492+0000","last_unstale":"2024-09-20T18:54:38.158261+0000","last_undegraded":"2024-09-20T18:54:38.158261+0000","last_fullsized":"2024-09-20T18:54:38.158261+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:27.589413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:27.589413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:48.562654+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":297,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:38.271920+0000","last_change":"2024-09-20T18:54:17.339848+0000","last_active":"2024-09-20T18:54:38.271920+0000","last_peered":"2024-09-20T18:54:38.271920+0000","last_clean":"2024-09-20T18:54:38.271920+0000","last_became_active":"2024-09-20T18:49:49.962243+0000","last_became_peered":"2024-09-20T18:49:49.962243+0000","last_unstale":"2024-09-20T18:54:38.271920+0000","last_undegraded":"2024-09-20T18:54:38.271920+0000","last_fullsized":"2024-09-20T18:54:38.271920+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:54:17.339761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:17.339761+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:22.040146+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":223,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:35.937974+0000","last_change":"2024-09-20T18:53:13.863129+0000","last_active":"2024-09-20T18:54:35.937974+0000","last_peered":"2024-09-20T18:54:35.937974+0000","last_clean":"2024-09-20T18:54:35.937974+0000","last_became_active":"2024-09-20T18:49:51.969093+0000","last_became_peered":"2024-09-20T18:49:51.969093+0000","last_unstale":"2024-09-20T18:54:35.937974+0000","last_undegraded":"2024-09-20T18:54:35.937974+0000","last_fullsized":"2024-09-20T18:54:35.937974+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:13.863061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:13.863061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:37.505050+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":254,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:37.955423+0000","last_change":"2024-09-20T18:53:43.879001+0000","last_active":"2024-09-20T18:54:37.955423+0000","last_peered":"2024-09-20T18:54:37.955423+0000","last_clean":"2024-09-20T18:54:37.955423+0000","last_became_active":"2024-09-20T18:49:47.949703+0000","last_became_peered":"2024-09-20T18:49:47.949703+0000","last_unstale":"2024-09-20T18:54:37.955423+0000","last_undegraded":"2024-09-20T18:54:37.955423+0000","last_fullsized":"2024-09-20T18:54:37.955423+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:53:43.878921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:43.878921+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:02.026666+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":231,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:37.955906+0000","last_change":"2024-09-20T18:52:05.985622+0000","last_active":"2024-09-20T18:54:37.955906+0000","last_peered":"2024-09-20T18:54:37.955906+0000","last_clean":"2024-09-20T18:54:37.955906+0000","last_became_active":"2024-09-20T18:49:45.939949+0000","last_became_peered":"2024-09-20T18:49:45.939949+0000","last_unstale":"2024-09-20T18:54:37.955906+0000","last_undegraded":"2024-09-20T18:54:37.955906+0000","last_fullsized":"2024-09-20T18:54:37.955906+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:05.985551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.985551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":219,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-20T18:54:34.923369+0000","last_change":"2024-09-20T18:51:14.271954+0000","last_active":"2024-09-20T18:54:34.923369+0000","last_peered":"2024-09-20T18:54:34.923369+0000","last_clean":"2024-09-20T18:54:34.923369+0000","last_became_active":"2024-09-20T18:49:49.957067+0000","last_became_peered":"2024-09-20T18:49:49.957067+0000","last_unstale":"2024-09-20T18:54:34.923369+0000","last_undegraded":"2024-09-20T18:54:34.923369+0000","last_fullsized":"2024-09-20T18:54:34.923369+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:14.271719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.271719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:34.371697+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":217,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:36.183278+0000","last_change":"2024-09-20T18:51:14.210438+0000","last_active":"2024-09-20T18:54:36.183278+0000","last_peered":"2024-09-20T18:54:36.183278+0000","last_clean":"2024-09-20T18:54:36.183278+0000","last_became_active":"2024-09-20T18:49:51.966915+0000","last_became_peered":"2024-09-20T18:49:51.966915+0000","last_unstale":"2024-09-20T18:54:36.183278+0000","last_undegraded":"2024-09-20T18:54:36.183278+0000","last_fullsized":"2024-09-20T18:54:36.183278+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:14.210358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.210358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:04.198046+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":255,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-20T18:54:34.923604+0000","last_change":"2024-09-20T18:53:53.264294+0000","last_active":"2024-09-20T18:54:34.923604+0000","last_peered":"2024-09-20T18:54:34.923604+0000","last_clean":"2024-09-20T18:54:34.923604+0000","last_became_active":"2024-09-20T18:49:47.953310+0000","last_became_peered":"2024-09-20T18:49:47.953310+0000","last_unstale":"2024-09-20T18:54:34.923604+0000","last_undegraded":"2024-09-20T18:54:34.923604+0000","last_fullsized":"2024-09-20T18:54:34.923604+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:53:53.263976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:53.263976+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:04.916734+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":230,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:38.158729+0000","last_change":"2024-09-20T18:52:34.834333+0000","last_active":"2024-09-20T18:54:38.158729+0000","last_peered":"2024-09-20T18:54:38.158729+0000","last_clean":"2024-09-20T18:54:38.158729+0000","last_became_active":"2024-09-20T18:49:45.944357+0000","last_became_peered":"2024-09-20T18:49:45.944357+0000","last_unstale":"2024-09-20T18:54:38.158729+0000","last_undegraded":"2024-09-20T18:54:38.158729+0000","last_fullsized":"2024-09-20T18:54:38.158729+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:34.834273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:34.834273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:42.456584+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":229,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:36.183698+0000","last_change":"2024-09-20T18:54:18.108239+0000","last_active":"2024-09-20T18:54:36.183698+0000","last_peered":"2024-09-20T18:54:36.183698+0000","last_clean":"2024-09-20T18:54:36.183698+0000","last_became_active":"2024-09-20T18:49:49.962523+0000","last_became_peered":"2024-09-20T18:49:49.962523+0000","last_unstale":"2024-09-20T18:54:36.183698+0000","last_undegraded":"2024-09-20T18:54:36.183698+0000","last_fullsized":"2024-09-20T18:54:36.183698+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:18.108175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:18.108175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:33.189789+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":232,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:35.936784+0000","last_change":"2024-09-20T18:54:26.882441+0000","last_active":"2024-09-20T18:54:35.936784+0000","last_peered":"2024-09-20T18:54:35.936784+0000","last_clean":"2024-09-20T18:54:35.936784+0000","last_became_active":"2024-09-20T18:49:51.967313+0000","last_became_peered":"2024-09-20T18:49:51.967313+0000","last_unstale":"2024-09-20T18:54:35.936784+0000","last_undegraded":"2024-09-20T18:54:35.936784+0000","last_fullsized":"2024-09-20T18:54:35.936784+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:26.882375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:26.882375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:41.905086+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":252,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:36.183378+0000","last_change":"2024-09-20T18:54:21.106745+0000","last_active":"2024-09-20T18:54:36.183378+0000","last_peered":"2024-09-20T18:54:36.183378+0000","last_clean":"2024-09-20T18:54:36.183378+0000","last_became_active":"2024-09-20T18:49:48.499111+0000","last_became_peered":"2024-09-20T18:49:48.499111+0000","last_unstale":"2024-09-20T18:54:36.183378+0000","last_undegraded":"2024-09-20T18:54:36.183378+0000","last_fullsized":"2024-09-20T18:54:36.183378+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:54:21.106677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:21.106677+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:38.509931+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":238,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:38.272894+0000","last_change":"2024-09-20T18:53:48.301845+0000","last_active":"2024-09-20T18:54:38.272894+0000","last_peered":"2024-09-20T18:54:38.272894+0000","last_clean":"2024-09-20T18:54:38.272894+0000","last_became_active":"2024-09-20T18:49:45.942105+0000","last_became_peered":"2024-09-20T18:49:45.942105+0000","last_unstale":"2024-09-20T18:54:38.272894+0000","last_undegraded":"2024-09-20T18:54:38.272894+0000","last_fullsized":"2024-09-20T18:54:38.272894+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:48.301801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:48.301801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:06.232022+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":238,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:38.157385+0000","last_change":"2024-09-20T18:53:46.329643+0000","last_active":"2024-09-20T18:54:38.157385+0000","last_peered":"2024-09-20T18:54:38.157385+0000","last_clean":"2024-09-20T18:54:38.157385+0000","last_became_active":"2024-09-20T18:49:49.958159+0000","last_became_peered":"2024-09-20T18:49:49.958159+0000","last_unstale":"2024-09-20T18:54:38.157385+0000","last_undegraded":"2024-09-20T18:54:38.157385+0000","last_fullsized":"2024-09-20T18:54:38.157385+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:46.329547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:46.329547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:09.181263+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":217,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:36.183826+0000","last_change":"2024-09-20T18:52:43.118886+0000","last_active":"2024-09-20T18:54:36.183826+0000","last_peered":"2024-09-20T18:54:36.183826+0000","last_clean":"2024-09-20T18:54:36.183826+0000","last_became_active":"2024-09-20T18:49:52.139978+0000","last_became_peered":"2024-09-20T18:49:52.139978+0000","last_unstale":"2024-09-20T18:54:36.183826+0000","last_undegraded":"2024-09-20T18:54:36.183826+0000","last_fullsized":"2024-09-20T18:54:36.183826+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:43.118805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:43.118805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":239,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:36.183302+0000","last_change":"2024-09-20T18:53:43.253204+0000","last_active":"2024-09-20T18:54:36.183302+0000","last_peered":"2024-09-20T18:54:36.183302+0000","last_clean":"2024-09-20T18:54:36.183302+0000","last_became_active":"2024-09-20T18:49:47.954777+0000","last_became_peered":"2024-09-20T18:49:47.954777+0000","last_unstale":"2024-09-20T18:54:36.183302+0000","last_undegraded":"2024-09-20T18:54:36.183302+0000","last_fullsized":"2024-09-20T18:54:36.183302+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-20T18:53:43.253132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:43.253132+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:00.654449+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":235,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:35.938145+0000","last_change":"2024-09-20T18:53:16.921320+0000","last_active":"2024-09-20T18:54:35.938145+0000","last_peered":"2024-09-20T18:54:35.938145+0000","last_clean":"2024-09-20T18:54:35.938145+0000","last_became_active":"2024-09-20T18:49:45.940988+0000","last_became_peered":"2024-09-20T18:49:45.940988+0000","last_unstale":"2024-09-20T18:54:35.938145+0000","last_undegraded":"2024-09-20T18:54:35.938145+0000","last_fullsized":"2024-09-20T18:54:35.938145+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:16.921263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:16.921263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":137,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":238,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:38.017319+0000","last_change":"2024-09-20T18:53:52.635213+0000","last_active":"2024-09-20T18:54:38.017319+0000","last_peered":"2024-09-20T18:54:38.017319+0000","last_clean":"2024-09-20T18:54:38.017319+0000","last_became_active":"2024-09-20T18:49:49.954502+0000","last_became_peered":"2024-09-20T18:49:49.954502+0000","last_unstale":"2024-09-20T18:54:38.017319+0000","last_undegraded":"2024-09-20T18:54:38.017319+0000","last_fullsized":"2024-09-20T18:54:38.017319+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:52.635138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:52.635138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:08.278673+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":218,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:38.157658+0000","last_change":"2024-09-20T18:51:09.761331+0000","last_active":"2024-09-20T18:54:38.157658+0000","last_peered":"2024-09-20T18:54:38.157658+0000","last_clean":"2024-09-20T18:54:38.157658+0000","last_became_active":"2024-09-20T18:49:51.968662+0000","last_became_peered":"2024-09-20T18:49:51.968662+0000","last_unstale":"2024-09-20T18:54:38.157658+0000","last_undegraded":"2024-09-20T18:54:38.157658+0000","last_fullsized":"2024-09-20T18:54:38.157658+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:09.761283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:09.761283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:33.490708+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":206,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-20T18:54:34.923629+0000","last_change":"2024-09-20T18:49:51.968476+0000","last_active":"2024-09-20T18:54:34.923629+0000","last_peered":"2024-09-20T18:54:34.923629+0000","last_clean":"2024-09-20T18:54:34.923629+0000","last_became_active":"2024-09-20T18:49:51.968216+0000","last_became_peered":"2024-09-20T18:49:51.968216+0000","last_unstale":"2024-09-20T18:54:34.923629+0000","last_undegraded":"2024-09-20T18:54:34.923629+0000","last_fullsized":"2024-09-20T18:54:34.923629+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:00.220813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":228,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:36.183451+0000","last_change":"2024-09-20T18:52:41.106558+0000","last_active":"2024-09-20T18:54:36.183451+0000","last_peered":"2024-09-20T18:54:36.183451+0000","last_clean":"2024-09-20T18:54:36.183451+0000","last_became_active":"2024-09-20T18:49:45.937992+0000","last_became_peered":"2024-09-20T18:49:45.937992+0000","last_unstale":"2024-09-20T18:54:36.183451+0000","last_undegraded":"2024-09-20T18:54:36.183451+0000","last_fullsized":"2024-09-20T18:54:36.183451+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:41.106332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:41.106332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:28.211547+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":245,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-20T18:54:38.018308+0000","last_change":"2024-09-20T18:51:14.545122+0000","last_active":"2024-09-20T18:54:38.018308+0000","last_peered":"2024-09-20T18:54:38.018308+0000","last_clean":"2024-09-20T18:54:38.018308+0000","last_became_active":"2024-09-20T18:49:47.949582+0000","last_became_peered":"2024-09-20T18:49:47.949582+0000","last_unstale":"2024-09-20T18:54:38.018308+0000","last_undegraded":"2024-09-20T18:54:38.018308+0000","last_fullsized":"2024-09-20T18:54:38.018308+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:51:14.545043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.545043+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:40.491358+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":220,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-20T18:54:35.938198+0000","last_change":"2024-09-20T18:51:28.120100+0000","last_active":"2024-09-20T18:54:35.938198+0000","last_peered":"2024-09-20T18:54:35.938198+0000","last_clean":"2024-09-20T18:54:35.938198+0000","last_became_active":"2024-09-20T18:49:49.957582+0000","last_became_peered":"2024-09-20T18:49:49.957582+0000","last_unstale":"2024-09-20T18:54:35.938198+0000","last_undegraded":"2024-09-20T18:54:35.938198+0000","last_fullsized":"2024-09-20T18:54:35.938198+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:28.119694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:28.119694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:34.833464+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":48103424,"num_objects":734,"num_object_clones":0,"num_object_copies":2202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":734,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":734,"num_write_kb":46976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":185401344,"data_stored":185401344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":734,"ondisk_log_size":734,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":290,"num_read_kb":285,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266751,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":121112,"kb_used_data":21832,"kb_used_omap":1,"kb_used_meta":99198,"kb_avail":93624040,"statfs":{"total":95995035648,"available":95871016960,"internally_reserved":0,"allocated":22355968,"data_stored":21962382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":101579210},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528390,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":145284,"kb_used_data":37748,"kb_used_omap":1,"kb_used_meta":107518,"kb_avail":93599868,"statfs":{"total":95995035648,"available":95846264832,"internally_reserved":0,"allocated":38653952,"data_stored":38280184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":110098886},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757323,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":110728,"kb_used_data":7348,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":93634424,"statfs":{"total":95995035648,"available":95881650176,"internally_reserved":0,"allocated":7524352,"data_stored":7150399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105839045},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986256,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":110052,"kb_used_data":2516,"kb_used_omap":1,"kb_used_meta":107518,"kb_avail":93635100,"statfs":{"total":95995035648,"available":95882342400,"internally_reserved":0,"allocated":2576384,"data_stored":2170915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":110098885},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182486,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":184652,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":183870,"kb_avail":93560500,"statfs":{"total":95995035648,"available":95805952000,"internally_reserved":0,"allocated":778240,"data_stored":386312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":188283335},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411419,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":118008,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":117246,"kb_avail":93627144,"statfs":{"total":95995035648,"available":95874195456,"internally_reserved":0,"allocated":753664,"data_stored":386795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":120060357},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574945,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":122216,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":121406,"kb_avail":93622936,"statfs":{"total":95995035648,"available":95869886464,"internally_reserved":0,"allocated":802816,"data_stored":387979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":124320197},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738471,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":158852,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":157502,"kb_avail":93586300,"statfs":{"total":95995035648,"available":95832371200,"internally_reserved":0,"allocated":1359872,"data_stored":976583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":161282501},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":40566784,"data_stored":40566784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":63635456,"data_stored":63635456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40501248,"data_stored":40501248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40697856,"data_stored":40697856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T18:54:43.438 INFO:tasks.cephadm.ceph_manager.ceph:would drop below min_pgs, proj 6, currently 16 2024-09-20T18:54:43.466 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:54:43.495 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:43.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:43.496 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:54:43.505 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:54:43.505 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:54:43.597 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:43.630 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:43.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:43.631 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:43.732 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:43.768 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:43.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:43.770 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:43.870 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:54:43.892 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:43.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:43.893 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:54:43.994 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:44.031 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:44.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:44.032 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:44.133 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:44.167 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 4 16 12147 12131 188.978 182.812 0.0119264 0.00528029 2024-09-20T18:54:44.172 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:44.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:44.173 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:44.190 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:44 smithi083 ceph-mon[29421]: 5.8 scrub starts 2024-09-20T18:54:44.190 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:44 smithi083 ceph-mon[29421]: 5.8 scrub ok 2024-09-20T18:54:44.190 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:44 smithi083 ceph-mon[29421]: pgmap v228: 148 pgs: 148 active+clean; 158 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 739 B/s rd, 23 MiB/s wr, 365 op/s 2024-09-20T18:54:44.191 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:44 smithi083 ceph-mon[29421]: osdmap e166: 8 total, 8 up, 8 in 2024-09-20T18:54:44.191 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:44 smithi083 ceph-mon[36260]: 5.8 scrub starts 2024-09-20T18:54:44.191 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:44 smithi083 ceph-mon[36260]: 5.8 scrub ok 2024-09-20T18:54:44.191 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:44 smithi083 ceph-mon[36260]: pgmap v228: 148 pgs: 148 active+clean; 158 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 739 B/s rd, 23 MiB/s wr, 365 op/s 2024-09-20T18:54:44.191 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:44 smithi083 ceph-mon[36260]: osdmap e166: 8 total, 8 up, 8 in 2024-09-20T18:54:44.274 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:54:44.298 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T18:54:44.298 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:54:44.304 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:44.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:44.305 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:54:44.406 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:44.443 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:44.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:44.444 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:44.500 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T18:54:44.523 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:44 smithi142 ceph-mon[32508]: 5.8 scrub starts 2024-09-20T18:54:44.523 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:44 smithi142 ceph-mon[32508]: 5.8 scrub ok 2024-09-20T18:54:44.523 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:44 smithi142 ceph-mon[32508]: pgmap v228: 148 pgs: 148 active+clean; 158 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 739 B/s rd, 23 MiB/s wr, 365 op/s 2024-09-20T18:54:44.523 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:44 smithi142 ceph-mon[32508]: osdmap e166: 8 total, 8 up, 8 in 2024-09-20T18:54:44.544 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:44.585 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:44.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:44.586 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:44.687 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:44.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:44.731 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:44.731 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:44.817 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:54:44.817 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:54:44.831 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:44.855 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:54:44.856 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:54:44.876 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:44.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:44.876 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:44.883 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:54:44.883 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:54:44.899 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:54:44.900 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:54:44.916 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:54:44.916 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:54:44.933 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:54:44.933 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:54:44.967 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:54:44.968 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:54:44.977 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:44.999 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:54:44.999 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:54:45.007 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:45.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:45.008 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:45.109 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:45.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:45.131 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:45.131 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:45.167 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 5 16 14485 14469 180.425 146.125 0.00640092 0.0055357 2024-09-20T18:54:45.232 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:54:45.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:45.265 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:45.265 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:54:45.303 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:45 smithi083 ceph-mon[36260]: 5.a scrub starts 2024-09-20T18:54:45.303 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:45 smithi083 ceph-mon[36260]: 5.a scrub ok 2024-09-20T18:54:45.303 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:45 smithi083 ceph-mon[36260]: 4.17 scrub starts 2024-09-20T18:54:45.303 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:45 smithi083 ceph-mon[36260]: 4.17 scrub ok 2024-09-20T18:54:45.303 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:45 smithi083 ceph-mon[36260]: 5.1c scrub starts 2024-09-20T18:54:45.303 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:45 smithi083 ceph-mon[36260]: 5.1c scrub ok 2024-09-20T18:54:45.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:45 smithi083 ceph-mon[36260]: 3.d scrub starts 2024-09-20T18:54:45.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:45 smithi083 ceph-mon[36260]: 3.d scrub ok 2024-09-20T18:54:45.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:45 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/15769812' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:54:45.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:45 smithi083 ceph-mon[29421]: 5.a scrub starts 2024-09-20T18:54:45.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:45 smithi083 ceph-mon[29421]: 5.a scrub ok 2024-09-20T18:54:45.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:45 smithi083 ceph-mon[29421]: 4.17 scrub starts 2024-09-20T18:54:45.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:45 smithi083 ceph-mon[29421]: 4.17 scrub ok 2024-09-20T18:54:45.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:45 smithi083 ceph-mon[29421]: 5.1c scrub starts 2024-09-20T18:54:45.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:45 smithi083 ceph-mon[29421]: 5.1c scrub ok 2024-09-20T18:54:45.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:45 smithi083 ceph-mon[29421]: 3.d scrub starts 2024-09-20T18:54:45.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:45 smithi083 ceph-mon[29421]: 3.d scrub ok 2024-09-20T18:54:45.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:45 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/15769812' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:54:45.366 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:45.402 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:45.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:45.403 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:45.504 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:54:45.523 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:45 smithi142 ceph-mon[32508]: 5.a scrub starts 2024-09-20T18:54:45.523 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:45 smithi142 ceph-mon[32508]: 5.a scrub ok 2024-09-20T18:54:45.523 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:45 smithi142 ceph-mon[32508]: 4.17 scrub starts 2024-09-20T18:54:45.523 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:45 smithi142 ceph-mon[32508]: 4.17 scrub ok 2024-09-20T18:54:45.523 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:45 smithi142 ceph-mon[32508]: 5.1c scrub starts 2024-09-20T18:54:45.523 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:45 smithi142 ceph-mon[32508]: 5.1c scrub ok 2024-09-20T18:54:45.523 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:45 smithi142 ceph-mon[32508]: 3.d scrub starts 2024-09-20T18:54:45.523 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:45 smithi142 ceph-mon[32508]: 3.d scrub ok 2024-09-20T18:54:45.524 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:45 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/15769812' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:54:45.537 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:45.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:45.538 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:54:45.639 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:45.678 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:45.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:45.679 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:45.780 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:45.815 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:45.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:45.816 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:45.883 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-20T18:54:45.917 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:54:45.962 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:45.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:45.963 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:54:46.064 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:54:46.101 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:46.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:46.102 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:54:46.167 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 6 16 17413 17397 180.831 183 0.00563959 0.00552314 2024-09-20T18:54:46.202 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:46.212 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:46 smithi083 ceph-mon[29421]: 4.1f scrub starts 2024-09-20T18:54:46.212 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:46 smithi083 ceph-mon[29421]: 4.1f scrub ok 2024-09-20T18:54:46.212 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:46 smithi083 ceph-mon[29421]: 4.16 scrub starts 2024-09-20T18:54:46.212 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:46 smithi083 ceph-mon[29421]: 4.16 scrub ok 2024-09-20T18:54:46.212 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:46 smithi083 ceph-mon[29421]: pgmap v230: 148 pgs: 148 active+clean; 158 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 741 B/s rd, 23 MiB/s wr, 366 op/s 2024-09-20T18:54:46.212 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:46 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:54:46.212 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:46 smithi083 ceph-mon[36260]: 4.1f scrub starts 2024-09-20T18:54:46.212 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:46 smithi083 ceph-mon[36260]: 4.1f scrub ok 2024-09-20T18:54:46.213 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:46 smithi083 ceph-mon[36260]: 4.16 scrub starts 2024-09-20T18:54:46.213 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:46 smithi083 ceph-mon[36260]: 4.16 scrub ok 2024-09-20T18:54:46.213 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:46 smithi083 ceph-mon[36260]: pgmap v230: 148 pgs: 148 active+clean; 158 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 741 B/s rd, 23 MiB/s wr, 366 op/s 2024-09-20T18:54:46.213 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:46 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:54:46.243 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:46.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:46.245 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:46.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:46 smithi142 ceph-mon[32508]: 4.1f scrub starts 2024-09-20T18:54:46.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:46 smithi142 ceph-mon[32508]: 4.1f scrub ok 2024-09-20T18:54:46.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:46 smithi142 ceph-mon[32508]: 4.16 scrub starts 2024-09-20T18:54:46.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:46 smithi142 ceph-mon[32508]: 4.16 scrub ok 2024-09-20T18:54:46.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:46 smithi142 ceph-mon[32508]: pgmap v230: 148 pgs: 148 active+clean; 158 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 741 B/s rd, 23 MiB/s wr, 366 op/s 2024-09-20T18:54:46.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:46 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:54:46.293 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:54:46.345 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:54:46.380 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:46.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:46.381 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:54:46.482 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:54:46.519 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:46.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:46.520 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:54:46.620 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:54:46.627 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:54:46.656 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:46.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:46.657 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:54:46.750 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:54:46 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:54:46.758 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:46.799 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:46.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:46.800 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:46.900 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:46.925 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T18:54:46.925 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-09-20T18:54:46.926 INFO:teuthology.orchestra.run.smithi142.stdout: "num_ops": 0 2024-09-20T18:54:46.926 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T18:54:46.934 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:46.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:46.935 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:47.036 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:47.088 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:47.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:47.089 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:47.168 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 7 16 19823 19807 176.521 150.625 0.00554512 0.00565902 2024-09-20T18:54:47.190 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:47.224 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:47.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:47.225 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:47.326 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:47.359 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:47.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:47.360 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:47.455 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:47 smithi142 ceph-mon[32508]: 3.a scrub starts 2024-09-20T18:54:47.455 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:47 smithi142 ceph-mon[32508]: 3.a scrub ok 2024-09-20T18:54:47.455 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:47 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:54:47.460 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:47.497 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:47.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:47.498 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:47.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:47 smithi083 ceph-mon[29421]: 3.a scrub starts 2024-09-20T18:54:47.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:47 smithi083 ceph-mon[29421]: 3.a scrub ok 2024-09-20T18:54:47.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:47 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:54:47.553 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:47 smithi083 ceph-mon[36260]: 3.a scrub starts 2024-09-20T18:54:47.553 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:47 smithi083 ceph-mon[36260]: 3.a scrub ok 2024-09-20T18:54:47.553 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:47 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:54:47.599 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:47.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:47.624 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:47.624 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:47.725 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:54:47.759 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:47.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:47.760 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:54:47.861 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:47.894 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:47.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:47.895 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:47.996 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:48.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:48.037 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:48.037 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:48.118 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_blocked_ops 2024-09-20T18:54:48.138 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:48.167 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 8 16 22113 22097 172.352 143.125 0.00571346 0.00579546 2024-09-20T18:54:48.177 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:48.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:48.178 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:48.279 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:48.300 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:48.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:48.302 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:48.303 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:48 smithi083 ceph-mon[36260]: pgmap v231: 148 pgs: 148 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 133 MiB/s wr, 2.13k op/s 2024-09-20T18:54:48.303 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:48 smithi083 ceph-mon[36260]: 5.0 scrub starts 2024-09-20T18:54:48.303 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:48 smithi083 ceph-mon[36260]: 5.0 scrub ok 2024-09-20T18:54:48.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:48 smithi083 ceph-mon[29421]: pgmap v231: 148 pgs: 148 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 133 MiB/s wr, 2.13k op/s 2024-09-20T18:54:48.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:48 smithi083 ceph-mon[29421]: 5.0 scrub starts 2024-09-20T18:54:48.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:48 smithi083 ceph-mon[29421]: 5.0 scrub ok 2024-09-20T18:54:48.403 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:48.439 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-20T18:54:48.439 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T18:54:48.439 INFO:tasks.thrashosds.thrasher:Setting full ratio to .001 2024-09-20T18:54:48.440 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set-full-ratio .001 2024-09-20T18:54:48.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:48.441 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:48.441 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:48.517 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:54:48.523 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:48 smithi142 ceph-mon[32508]: pgmap v231: 148 pgs: 148 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 133 MiB/s wr, 2.13k op/s 2024-09-20T18:54:48.523 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:48 smithi142 ceph-mon[32508]: 5.0 scrub starts 2024-09-20T18:54:48.523 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:48 smithi142 ceph-mon[32508]: 5.0 scrub ok 2024-09-20T18:54:48.542 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:48.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:48.564 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:48.564 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:48.590 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T18:54:48.665 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:48.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:48.712 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:48.713 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:48.813 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:48.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:48.854 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:48.855 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:48.955 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:48.994 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:48.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:48.995 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:49.071 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-20T18:54:49.093 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:54:49.096 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:49.147 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:49.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:49.148 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:49.167 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 9 16 23896 23880 165.592 111.438 0.00388283 0.00601914 2024-09-20T18:54:49.249 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:54:49.281 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:49.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:49.282 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:54:49.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:49 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2367800594' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:54:49.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:49 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:54:49.303 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:49 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2367800594' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:54:49.303 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:49 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:54:49.384 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:49.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:49.413 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:49.413 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:49.514 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:49.522 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:49 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2367800594' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:54:49.523 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:49 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:54:49.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:49.551 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:49.551 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:49.652 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:49.688 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:49.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:49.689 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:49.790 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:49.826 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:49.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:49.827 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:49.927 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:49.960 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:49.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:49.961 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:50.061 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:50.096 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:50.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:50.097 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:50.168 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 10 16 25958 25942 161.923 128.875 0.0072375 0.00617042 2024-09-20T18:54:50.198 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:50.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:50.226 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:50.226 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:50.327 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:54:50.352 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:50.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:50.353 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:54:50.454 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:50.490 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:50.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:50.491 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:50.523 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:50 smithi142 ceph-mon[32508]: pgmap v232: 148 pgs: 148 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 128 MiB/s wr, 2.04k op/s 2024-09-20T18:54:50.523 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:50 smithi142 ceph-mon[32508]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T18:54:50.523 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:50 smithi142 ceph-mon[32508]: Cluster is now healthy 2024-09-20T18:54:50.523 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:50 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T18:54:50.523 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:50 smithi142 ceph-mon[32508]: osdmap e167: 8 total, 8 up, 8 in 2024-09-20T18:54:50.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:50 smithi083 ceph-mon[29421]: pgmap v232: 148 pgs: 148 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 128 MiB/s wr, 2.04k op/s 2024-09-20T18:54:50.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:50 smithi083 ceph-mon[29421]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T18:54:50.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:50 smithi083 ceph-mon[29421]: Cluster is now healthy 2024-09-20T18:54:50.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:50 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T18:54:50.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:50 smithi083 ceph-mon[29421]: osdmap e167: 8 total, 8 up, 8 in 2024-09-20T18:54:50.553 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:50 smithi083 ceph-mon[36260]: pgmap v232: 148 pgs: 148 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 128 MiB/s wr, 2.04k op/s 2024-09-20T18:54:50.553 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:50 smithi083 ceph-mon[36260]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T18:54:50.554 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:50 smithi083 ceph-mon[36260]: Cluster is now healthy 2024-09-20T18:54:50.554 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:50 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T18:54:50.554 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:50 smithi083 ceph-mon[36260]: osdmap e167: 8 total, 8 up, 8 in 2024-09-20T18:54:50.592 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:50.629 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:50.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:50.630 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:50.730 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:50.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:50.766 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:50.766 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:50.867 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:54:50.902 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:50.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:50.903 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:54:51.004 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:51.026 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T18:54:51.026 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-09-20T18:54:51.026 INFO:teuthology.orchestra.run.smithi142.stdout: "complaint_time": 30, 2024-09-20T18:54:51.027 INFO:teuthology.orchestra.run.smithi142.stdout: "num_blocked_ops": 0 2024-09-20T18:54:51.027 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T18:54:51.041 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:51.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:51.042 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:51.052 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:54:50 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:54:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:54:51.143 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:51.170 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 11 16 28153 28137 159.638 137.188 0.00504556 0.00625814 2024-09-20T18:54:51.177 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:51.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:51.178 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:51.235 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:54:51.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:51 smithi142 ceph-mon[32508]: 4.18 scrub starts 2024-09-20T18:54:51.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:51 smithi142 ceph-mon[32508]: 4.18 scrub ok 2024-09-20T18:54:51.279 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:51.320 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:51.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:51.322 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:51.368 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:51 smithi083 ceph-mon[29421]: 4.18 scrub starts 2024-09-20T18:54:51.369 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:51 smithi083 ceph-mon[29421]: 4.18 scrub ok 2024-09-20T18:54:51.369 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:51 smithi083 ceph-mon[36260]: 4.18 scrub starts 2024-09-20T18:54:51.369 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:51 smithi083 ceph-mon[36260]: 4.18 scrub ok 2024-09-20T18:54:51.423 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:51.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:51.453 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:51.453 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:51.554 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:51.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:51.588 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:51.588 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:51.689 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:51.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:51.723 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:51.724 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:51.824 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:54:51.859 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:51.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:51.860 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:54:51.917 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_historic_ops 2024-09-20T18:54:51.960 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:52.000 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:52.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:52.000 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:52.101 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:52.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:52.135 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:52.136 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:52.170 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 12 15 30308 30293 157.566 134.75 0.00282139 0.00634125 2024-09-20T18:54:52.236 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:52.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:52.271 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:52.271 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:52.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:52 smithi142 ceph-mon[32508]: 5.2 scrub starts 2024-09-20T18:54:52.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:52 smithi142 ceph-mon[32508]: 5.2 scrub ok 2024-09-20T18:54:52.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:52 smithi142 ceph-mon[32508]: pgmap v234: 148 pgs: 148 active+clean; 1.3 GiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 146 MiB/s wr, 2.33k op/s 2024-09-20T18:54:52.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:52 smithi142 ceph-mon[32508]: 6.1b scrub starts 2024-09-20T18:54:52.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:52 smithi142 ceph-mon[32508]: 6.1b scrub ok 2024-09-20T18:54:52.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:52 smithi142 ceph-mon[32508]: 2.2 scrub starts 2024-09-20T18:54:52.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:52 smithi142 ceph-mon[32508]: 2.2 scrub ok 2024-09-20T18:54:52.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:52 smithi142 ceph-mon[32508]: 5.1 scrub starts 2024-09-20T18:54:52.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:52 smithi142 ceph-mon[32508]: 5.1 scrub ok 2024-09-20T18:54:52.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:52 smithi142 ceph-mon[32508]: 6.15 scrub starts 2024-09-20T18:54:52.274 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:52 smithi142 ceph-mon[32508]: 6.15 scrub ok 2024-09-20T18:54:52.318 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:54:52.372 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:52.389 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:52 smithi083 ceph-mon[29421]: 5.2 scrub starts 2024-09-20T18:54:52.389 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:52 smithi083 ceph-mon[29421]: 5.2 scrub ok 2024-09-20T18:54:52.390 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:52 smithi083 ceph-mon[29421]: pgmap v234: 148 pgs: 148 active+clean; 1.3 GiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 146 MiB/s wr, 2.33k op/s 2024-09-20T18:54:52.390 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:52 smithi083 ceph-mon[29421]: 6.1b scrub starts 2024-09-20T18:54:52.390 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:52 smithi083 ceph-mon[29421]: 6.1b scrub ok 2024-09-20T18:54:52.390 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:52 smithi083 ceph-mon[29421]: 2.2 scrub starts 2024-09-20T18:54:52.390 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:52 smithi083 ceph-mon[29421]: 2.2 scrub ok 2024-09-20T18:54:52.390 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:52 smithi083 ceph-mon[29421]: 5.1 scrub starts 2024-09-20T18:54:52.390 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:52 smithi083 ceph-mon[29421]: 5.1 scrub ok 2024-09-20T18:54:52.390 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:52 smithi083 ceph-mon[29421]: 6.15 scrub starts 2024-09-20T18:54:52.390 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:52 smithi083 ceph-mon[29421]: 6.15 scrub ok 2024-09-20T18:54:52.391 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:52 smithi083 ceph-mon[36260]: 5.2 scrub starts 2024-09-20T18:54:52.391 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:52 smithi083 ceph-mon[36260]: 5.2 scrub ok 2024-09-20T18:54:52.391 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:52 smithi083 ceph-mon[36260]: pgmap v234: 148 pgs: 148 active+clean; 1.3 GiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 146 MiB/s wr, 2.33k op/s 2024-09-20T18:54:52.391 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:52 smithi083 ceph-mon[36260]: 6.1b scrub starts 2024-09-20T18:54:52.391 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:52 smithi083 ceph-mon[36260]: 6.1b scrub ok 2024-09-20T18:54:52.391 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:52 smithi083 ceph-mon[36260]: 2.2 scrub starts 2024-09-20T18:54:52.391 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:52 smithi083 ceph-mon[36260]: 2.2 scrub ok 2024-09-20T18:54:52.391 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:52 smithi083 ceph-mon[36260]: 5.1 scrub starts 2024-09-20T18:54:52.391 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:52 smithi083 ceph-mon[36260]: 5.1 scrub ok 2024-09-20T18:54:52.391 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:52 smithi083 ceph-mon[36260]: 6.15 scrub starts 2024-09-20T18:54:52.391 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:52 smithi083 ceph-mon[36260]: 6.15 scrub ok 2024-09-20T18:54:52.407 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:52.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:52.409 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:52.509 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:52.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:52.546 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:52.546 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:52.648 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:52.684 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:52.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:52.685 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:52.786 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:54:52.814 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:52.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:52.815 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:54:52.915 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:52.944 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:52.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:52.946 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:53.046 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:53.080 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:53.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:53.081 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:53.116 INFO:teuthology.orchestra.run.smithi083.stderr:osd set-full-ratio 0.001 2024-09-20T18:54:53.134 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:53 smithi142 ceph-mon[32508]: 3.e scrub starts 2024-09-20T18:54:53.134 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:53 smithi142 ceph-mon[32508]: 3.e scrub ok 2024-09-20T18:54:53.134 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:53 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1888091433' entity='client.admin' cmd={"prefix": "osd set-full-ratio", "ratio": 0.001} : dispatch 2024-09-20T18:54:53.170 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 13 15 32184 32169 154.467 117.25 0.00396922 0.00646907 2024-09-20T18:54:53.182 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:53.217 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:53.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:53.218 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:53.319 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:53.355 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:53.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:53.356 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:53.395 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:53 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mon-a[29417]: 2024-09-20T18:54:53.078+0000 7f6c0b9cb640 -1 log_channel(cluster) log [ERR] : Health check failed: full ratio(s) out of order (OSD_OUT_OF_ORDER_FULL) 2024-09-20T18:54:53.396 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:53 smithi083 ceph-mon[29421]: 3.e scrub starts 2024-09-20T18:54:53.396 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:53 smithi083 ceph-mon[29421]: 3.e scrub ok 2024-09-20T18:54:53.396 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:53 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1888091433' entity='client.admin' cmd={"prefix": "osd set-full-ratio", "ratio": 0.001} : dispatch 2024-09-20T18:54:53.396 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:53 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set-full-ratio", "ratio": 0.001} : dispatch 2024-09-20T18:54:53.396 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:53 smithi083 ceph-mon[36260]: 3.e scrub starts 2024-09-20T18:54:53.396 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:53 smithi083 ceph-mon[36260]: 3.e scrub ok 2024-09-20T18:54:53.396 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:53 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1888091433' entity='client.admin' cmd={"prefix": "osd set-full-ratio", "ratio": 0.001} : dispatch 2024-09-20T18:54:53.396 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:53 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set-full-ratio", "ratio": 0.001} : dispatch 2024-09-20T18:54:53.419 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T18:54:53.419 INFO:teuthology.orchestra.run.smithi142.stdout: "size": 20, 2024-09-20T18:54:53.419 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 600, 2024-09-20T18:54:53.419 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-09-20T18:54:53.419 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.420 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:23109 7.1 7:8a12d0b9:::benchmark_data_smithi083_70643_object23108:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-20T18:54:53.420 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:06.089353+0000", 2024-09-20T18:54:53.420 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 226.90212286100001, 2024-09-20T18:54:53.420 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.087697153, 2024-09-20T18:54:53.420 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:54:53.420 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:54:53.420 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:54:53.420 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:54:53.420 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:54:53.420 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:54:53.420 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 23109 2024-09-20T18:54:53.420 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.420 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:54:53.421 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.421 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:54:53.421 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.089353+0000", 2024-09-20T18:54:53.421 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:54:53.421 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.421 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.421 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:54:53.421 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.089353+0000", 2024-09-20T18:54:53.421 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:54:53.421 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.421 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.421 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:54:53.421 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.089354+0000", 2024-09-20T18:54:53.421 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.358e-06 2024-09-20T18:54:53.422 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.422 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.422 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:54:53.422 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.089526+0000", 2024-09-20T18:54:53.422 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000171077 2024-09-20T18:54:53.422 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.422 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.422 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:54:53.422 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.089541+0000", 2024-09-20T18:54:53.422 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5634999999999999e-05 2024-09-20T18:54:53.422 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.422 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.422 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:54:53.423 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.089549+0000", 2024-09-20T18:54:53.423 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.8189999999999992e-06 2024-09-20T18:54:53.423 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.423 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.423 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:54:53.423 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.089602+0000", 2024-09-20T18:54:53.423 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.2988000000000002e-05 2024-09-20T18:54:53.423 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.423 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.423 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:54:53.423 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.089719+0000", 2024-09-20T18:54:53.423 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000117374 2024-09-20T18:54:53.423 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.423 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.424 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,6", 2024-09-20T18:54:53.424 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.089889+0000", 2024-09-20T18:54:53.424 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00016921300000000001 2024-09-20T18:54:53.424 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.424 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.424 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:54:53.424 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.091577+0000", 2024-09-20T18:54:53.424 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001688316 2024-09-20T18:54:53.424 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.424 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.424 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:53.424 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.092076+0000", 2024-09-20T18:54:53.424 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000498705 2024-09-20T18:54:53.425 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.425 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.425 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:53.425 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.176935+0000", 2024-09-20T18:54:53.425 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.084859749999999998 2024-09-20T18:54:53.425 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.425 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.425 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:54:53.425 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.176999+0000", 2024-09-20T18:54:53.425 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.3960000000000004e-05 2024-09-20T18:54:53.425 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.425 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.425 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:54:53.426 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.177050+0000", 2024-09-20T18:54:53.426 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0958000000000002e-05 2024-09-20T18:54:53.426 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:54:53.426 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:54:53.426 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:54:53.426 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.426 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.426 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:25818 7.1 7:8b1d7f7e:::benchmark_data_smithi083_70643_object25817:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:54:53.426 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:07.637397+0000", 2024-09-20T18:54:53.426 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 225.35407920500001, 2024-09-20T18:54:53.426 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.069413953, 2024-09-20T18:54:53.426 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:54:53.426 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:54:53.426 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:54:53.427 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:54:53.427 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:54:53.427 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:54:53.427 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 25818 2024-09-20T18:54:53.427 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.427 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:54:53.427 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.427 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:54:53.427 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.637397+0000", 2024-09-20T18:54:53.427 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:54:53.427 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.427 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.427 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:54:53.427 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.637397+0000", 2024-09-20T18:54:53.428 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:54:53.428 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.428 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.428 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:54:53.428 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.637398+0000", 2024-09-20T18:54:53.428 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.254e-06 2024-09-20T18:54:53.428 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.428 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.428 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:54:53.428 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.637441+0000", 2024-09-20T18:54:53.428 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.2985000000000003e-05 2024-09-20T18:54:53.428 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.428 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.428 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:54:53.429 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.637450+0000", 2024-09-20T18:54:53.429 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.7409999999999998e-06 2024-09-20T18:54:53.429 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.429 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.429 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:54:53.429 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.637454+0000", 2024-09-20T18:54:53.429 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.5229999999999997e-06 2024-09-20T18:54:53.429 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.429 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.429 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:54:53.429 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.637494+0000", 2024-09-20T18:54:53.429 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9595999999999998e-05 2024-09-20T18:54:53.430 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.430 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.430 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:54:53.430 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.637606+0000", 2024-09-20T18:54:53.430 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000111779 2024-09-20T18:54:53.430 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.430 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.430 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,6", 2024-09-20T18:54:53.430 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.637776+0000", 2024-09-20T18:54:53.430 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00017017900000000001 2024-09-20T18:54:53.430 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.430 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.430 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:54:53.431 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.638868+0000", 2024-09-20T18:54:53.431 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0010925830000000001 2024-09-20T18:54:53.431 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.431 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.431 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:53.431 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.639923+0000", 2024-09-20T18:54:53.431 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00105495 2024-09-20T18:54:53.431 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.431 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.431 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:53.431 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.706721+0000", 2024-09-20T18:54:53.431 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.066797422999999995 2024-09-20T18:54:53.431 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.432 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.432 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:54:53.432 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.706772+0000", 2024-09-20T18:54:53.432 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0788999999999998e-05 2024-09-20T18:54:53.432 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.432 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.432 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:54:53.432 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.706811+0000", 2024-09-20T18:54:53.432 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9150999999999998e-05 2024-09-20T18:54:53.432 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:54:53.432 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:54:53.432 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:54:53.432 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.433 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.433 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:26463 7.1 7:878a0722:::benchmark_data_smithi083_70643_object26462:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:54:53.433 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:07.994856+0000", 2024-09-20T18:54:53.433 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 224.99661970400001, 2024-09-20T18:54:53.433 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13761967, 2024-09-20T18:54:53.433 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:54:53.433 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:54:53.433 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:54:53.433 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:54:53.433 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:54:53.433 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:54:53.433 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 26463 2024-09-20T18:54:53.433 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.433 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:54:53.433 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.434 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:54:53.434 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.994856+0000", 2024-09-20T18:54:53.434 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:54:53.434 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.434 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.434 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:54:53.434 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.994856+0000", 2024-09-20T18:54:53.434 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:54:53.434 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.434 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.434 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:54:53.434 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.994858+0000", 2024-09-20T18:54:53.435 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.649e-06 2024-09-20T18:54:53.435 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.435 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.435 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:54:53.435 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.994924+0000", 2024-09-20T18:54:53.435 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.6484999999999998e-05 2024-09-20T18:54:53.435 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.435 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.435 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:54:53.435 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.994937+0000", 2024-09-20T18:54:53.435 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3084e-05 2024-09-20T18:54:53.435 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.435 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.435 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:54:53.435 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.994945+0000", 2024-09-20T18:54:53.436 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.4889999999999998e-06 2024-09-20T18:54:53.436 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.436 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.436 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:54:53.436 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.994984+0000", 2024-09-20T18:54:53.436 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9270999999999997e-05 2024-09-20T18:54:53.436 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.436 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.436 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:54:53.436 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.995073+0000", 2024-09-20T18:54:53.436 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.8747999999999998e-05 2024-09-20T18:54:53.436 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.436 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.436 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,6", 2024-09-20T18:54:53.437 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.995161+0000", 2024-09-20T18:54:53.437 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.7538000000000001e-05 2024-09-20T18:54:53.437 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.437 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.437 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:54:53.437 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.996461+0000", 2024-09-20T18:54:53.437 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0013008060000000001 2024-09-20T18:54:53.437 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.437 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.437 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:53.437 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.996788+0000", 2024-09-20T18:54:53.437 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00032643100000000002 2024-09-20T18:54:53.437 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.437 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.438 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:53.438 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.132406+0000", 2024-09-20T18:54:53.438 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13561836799999999 2024-09-20T18:54:53.438 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.438 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.438 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:54:53.438 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.132445+0000", 2024-09-20T18:54:53.438 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.8640999999999998e-05 2024-09-20T18:54:53.438 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.438 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.438 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:54:53.438 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.132476+0000", 2024-09-20T18:54:53.438 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.116e-05 2024-09-20T18:54:53.438 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:54:53.439 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:54:53.439 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:54:53.439 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.439 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.439 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:26470 7.1 7:85a6a71e:::benchmark_data_smithi083_70643_object26469:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:54:53.439 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:07.997258+0000", 2024-09-20T18:54:53.439 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 224.99421836100001, 2024-09-20T18:54:53.439 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13539621399999999, 2024-09-20T18:54:53.439 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:54:53.439 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:54:53.439 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:54:53.439 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:54:53.439 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:54:53.439 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:54:53.440 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 26470 2024-09-20T18:54:53.440 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.440 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:54:53.440 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.440 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:54:53.440 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.997258+0000", 2024-09-20T18:54:53.440 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:54:53.440 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.440 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.440 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:54:53.440 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.997258+0000", 2024-09-20T18:54:53.440 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:54:53.440 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.440 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.440 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:54:53.441 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.997259+0000", 2024-09-20T18:54:53.441 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6810000000000001e-06 2024-09-20T18:54:53.441 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.441 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.441 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:54:53.441 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.997317+0000", 2024-09-20T18:54:53.441 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8174000000000002e-05 2024-09-20T18:54:53.441 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.441 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.441 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:54:53.441 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.997332+0000", 2024-09-20T18:54:53.441 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4687e-05 2024-09-20T18:54:53.441 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.441 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.442 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:54:53.442 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.997340+0000", 2024-09-20T18:54:53.442 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.4089999999999997e-06 2024-09-20T18:54:53.442 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.442 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.442 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:54:53.442 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.997395+0000", 2024-09-20T18:54:53.442 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5318000000000001e-05 2024-09-20T18:54:53.442 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.442 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.442 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:54:53.442 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.997488+0000", 2024-09-20T18:54:53.442 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.2930000000000006e-05 2024-09-20T18:54:53.443 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.443 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.443 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,6", 2024-09-20T18:54:53.443 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.997587+0000", 2024-09-20T18:54:53.443 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.9167999999999996e-05 2024-09-20T18:54:53.443 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.443 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.443 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:54:53.443 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.998680+0000", 2024-09-20T18:54:53.443 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001092759 2024-09-20T18:54:53.443 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.443 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.443 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:53.443 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.001637+0000", 2024-09-20T18:54:53.443 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.002956991 2024-09-20T18:54:53.444 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.444 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.444 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:53.444 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.132549+0000", 2024-09-20T18:54:53.444 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13091224700000001 2024-09-20T18:54:53.444 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.444 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.444 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:54:53.444 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.132609+0000", 2024-09-20T18:54:53.444 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0123e-05 2024-09-20T18:54:53.444 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.444 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.444 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:54:53.444 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.132654+0000", 2024-09-20T18:54:53.445 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.4727000000000001e-05 2024-09-20T18:54:53.445 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:54:53.445 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:54:53.445 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:54:53.445 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.445 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.445 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:27565 7.1 7:8e95a048:::benchmark_data_smithi083_70643_object27564:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:54:53.445 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:08.738835+0000", 2024-09-20T18:54:53.445 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 224.25264131899999, 2024-09-20T18:54:53.445 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.103678123, 2024-09-20T18:54:53.445 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:54:53.445 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:54:53.445 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:54:53.445 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:54:53.445 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:54:53.446 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:54:53.446 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 27565 2024-09-20T18:54:53.446 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.446 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:54:53.446 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.446 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:54:53.446 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.738835+0000", 2024-09-20T18:54:53.446 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:54:53.446 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.446 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.446 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:54:53.446 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.738835+0000", 2024-09-20T18:54:53.446 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:54:53.447 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.447 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.447 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:54:53.447 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.738836+0000", 2024-09-20T18:54:53.447 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.451e-06 2024-09-20T18:54:53.447 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.447 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.447 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:54:53.447 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.738941+0000", 2024-09-20T18:54:53.447 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00010476000000000001 2024-09-20T18:54:53.447 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.447 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.447 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:54:53.448 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.738949+0000", 2024-09-20T18:54:53.448 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.1869999999999997e-06 2024-09-20T18:54:53.448 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.448 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.448 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:54:53.448 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.738955+0000", 2024-09-20T18:54:53.448 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.908e-06 2024-09-20T18:54:53.448 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.448 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.448 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:54:53.448 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.739267+0000", 2024-09-20T18:54:53.448 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00031249899999999999 2024-09-20T18:54:53.448 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.448 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.449 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:54:53.449 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.739392+0000", 2024-09-20T18:54:53.449 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012450699999999999 2024-09-20T18:54:53.449 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.449 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.449 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,6", 2024-09-20T18:54:53.449 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.739517+0000", 2024-09-20T18:54:53.449 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012527400000000001 2024-09-20T18:54:53.449 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.449 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.449 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:54:53.449 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.740572+0000", 2024-09-20T18:54:53.449 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001054452 2024-09-20T18:54:53.449 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.450 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.450 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:53.450 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.741601+0000", 2024-09-20T18:54:53.450 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0010288160000000001 2024-09-20T18:54:53.450 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.450 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.450 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:53.450 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.842412+0000", 2024-09-20T18:54:53.450 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100811759 2024-09-20T18:54:53.450 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.450 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.450 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:54:53.450 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.842469+0000", 2024-09-20T18:54:53.450 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.7169000000000001e-05 2024-09-20T18:54:53.451 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.451 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.451 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:54:53.451 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.842513+0000", 2024-09-20T18:54:53.451 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3340999999999997e-05 2024-09-20T18:54:53.451 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:54:53.451 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:54:53.451 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:54:53.451 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.451 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.451 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:31526 7.1 7:8e4a4ffd:::benchmark_data_smithi083_70643_object31525:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T18:54:53.451 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:10.995167+0000", 2024-09-20T18:54:53.452 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 221.996309043, 2024-09-20T18:54:53.452 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.064702838999999998, 2024-09-20T18:54:53.452 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:54:53.452 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:54:53.452 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:54:53.452 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:54:53.452 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:54:53.452 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:54:53.452 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 31526 2024-09-20T18:54:53.452 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.452 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:54:53.452 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.452 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:54:53.452 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:10.995167+0000", 2024-09-20T18:54:53.453 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:54:53.453 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.453 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.453 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:54:53.453 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:10.995167+0000", 2024-09-20T18:54:53.453 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:54:53.453 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.453 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.453 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:54:53.453 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:10.995169+0000", 2024-09-20T18:54:53.453 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.784e-06 2024-09-20T18:54:53.453 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.453 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.453 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:54:53.454 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:10.995212+0000", 2024-09-20T18:54:53.454 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.2911000000000003e-05 2024-09-20T18:54:53.454 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.454 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.454 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:54:53.454 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:10.995227+0000", 2024-09-20T18:54:53.454 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5078e-05 2024-09-20T18:54:53.454 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.454 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.454 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:54:53.454 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:10.995235+0000", 2024-09-20T18:54:53.454 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.9929999999999995e-06 2024-09-20T18:54:53.454 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.455 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.455 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:54:53.455 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:10.995282+0000", 2024-09-20T18:54:53.455 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7355999999999999e-05 2024-09-20T18:54:53.455 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.455 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.455 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:54:53.455 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:10.995400+0000", 2024-09-20T18:54:53.455 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000118262 2024-09-20T18:54:53.455 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.455 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.455 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,6", 2024-09-20T18:54:53.455 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:10.995613+0000", 2024-09-20T18:54:53.455 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000213048 2024-09-20T18:54:53.456 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.456 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.456 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:54:53.456 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:10.996704+0000", 2024-09-20T18:54:53.456 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0010906290000000001 2024-09-20T18:54:53.456 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.456 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.456 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:53.456 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:10.997397+0000", 2024-09-20T18:54:53.456 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00069284900000000005 2024-09-20T18:54:53.456 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.456 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.456 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:53.457 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:11.059767+0000", 2024-09-20T18:54:53.457 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.062369804000000001 2024-09-20T18:54:53.457 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.457 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.457 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:54:53.457 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:11.059820+0000", 2024-09-20T18:54:53.457 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.342e-05 2024-09-20T18:54:53.457 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.457 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.457 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:54:53.457 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:11.059870+0000", 2024-09-20T18:54:53.457 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.9704999999999999e-05 2024-09-20T18:54:53.457 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:54:53.457 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:54:53.457 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:54:53.458 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.458 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.458 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:32908 7.1 7:8a214de6:::benchmark_data_smithi083_70643_object32907:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T18:54:53.458 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:11.728778+0000", 2024-09-20T18:54:53.458 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 221.26269825599999, 2024-09-20T18:54:53.458 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.062731522999999997, 2024-09-20T18:54:53.458 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:54:53.458 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:54:53.458 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:54:53.458 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:54:53.458 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:54:53.458 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:54:53.458 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 32908 2024-09-20T18:54:53.459 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.459 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:54:53.459 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.459 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:54:53.459 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:11.728778+0000", 2024-09-20T18:54:53.459 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:54:53.459 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.459 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.459 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:54:53.459 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:11.728778+0000", 2024-09-20T18:54:53.459 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:54:53.459 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.459 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.460 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:54:53.460 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:11.728780+0000", 2024-09-20T18:54:53.460 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8160000000000001e-06 2024-09-20T18:54:53.460 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.460 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.460 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:54:53.460 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:11.728921+0000", 2024-09-20T18:54:53.460 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00014168199999999999 2024-09-20T18:54:53.460 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.460 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.460 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:54:53.460 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:11.728935+0000", 2024-09-20T18:54:53.460 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3423e-05 2024-09-20T18:54:53.460 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.461 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.461 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:54:53.461 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:11.728942+0000", 2024-09-20T18:54:53.461 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.1459999999999999e-06 2024-09-20T18:54:53.461 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.461 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.461 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:54:53.461 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:11.729001+0000", 2024-09-20T18:54:53.461 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9005000000000002e-05 2024-09-20T18:54:53.461 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.461 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.461 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:54:53.461 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:11.729100+0000", 2024-09-20T18:54:53.461 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.8869999999999994e-05 2024-09-20T18:54:53.462 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.462 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.462 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,6", 2024-09-20T18:54:53.462 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:11.729218+0000", 2024-09-20T18:54:53.462 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000118084 2024-09-20T18:54:53.462 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.462 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.462 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:54:53.462 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:11.730418+0000", 2024-09-20T18:54:53.462 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001199961 2024-09-20T18:54:53.462 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.462 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.462 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:53.463 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:11.732927+0000", 2024-09-20T18:54:53.463 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.002509627 2024-09-20T18:54:53.463 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.463 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.463 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:53.463 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:11.791406+0000", 2024-09-20T18:54:53.463 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.058478324999999998 2024-09-20T18:54:53.463 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.463 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.463 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:54:53.463 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:11.791462+0000", 2024-09-20T18:54:53.463 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6659000000000001e-05 2024-09-20T18:54:53.463 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.463 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.464 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:54:53.464 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:11.791509+0000", 2024-09-20T18:54:53.464 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.6925000000000002e-05 2024-09-20T18:54:53.464 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:54:53.464 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:54:53.464 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:54:53.464 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.464 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.464 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:101255 7.1 7:80ab34f1:::benchmark_data_smithi083_70643_object101254:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-20T18:54:53.464 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:05.038883+0000", 2024-09-20T18:54:53.464 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 167.952593283, 2024-09-20T18:54:53.464 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.064111823999999998, 2024-09-20T18:54:53.464 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:54:53.465 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:54:53.465 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:54:53.465 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:54:53.465 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:54:53.465 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:54:53.465 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 101255 2024-09-20T18:54:53.465 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.465 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:54:53.465 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.465 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:54:53.465 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:05.038883+0000", 2024-09-20T18:54:53.465 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:54:53.465 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.465 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.466 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:54:53.466 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:05.038883+0000", 2024-09-20T18:54:53.466 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:54:53.466 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.466 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.466 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:54:53.466 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:05.038883+0000", 2024-09-20T18:54:53.466 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.6899999999999997e-07 2024-09-20T18:54:53.466 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.466 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.466 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:54:53.466 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:05.038920+0000", 2024-09-20T18:54:53.466 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6785000000000001e-05 2024-09-20T18:54:53.466 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.467 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.467 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:54:53.467 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:05.038928+0000", 2024-09-20T18:54:53.467 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.2069999999999995e-06 2024-09-20T18:54:53.467 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.467 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.467 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:54:53.467 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:05.038934+0000", 2024-09-20T18:54:53.467 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0299999999999999e-06 2024-09-20T18:54:53.467 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.467 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.467 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:54:53.467 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:05.041619+0000", 2024-09-20T18:54:53.468 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0026841819999999998 2024-09-20T18:54:53.468 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.468 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.468 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:54:53.468 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:05.041716+0000", 2024-09-20T18:54:53.468 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.7922000000000002e-05 2024-09-20T18:54:53.468 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.468 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.468 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,6", 2024-09-20T18:54:53.468 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:05.041844+0000", 2024-09-20T18:54:53.468 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000127033 2024-09-20T18:54:53.468 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.468 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.468 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:54:53.469 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:05.044356+0000", 2024-09-20T18:54:53.469 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0025127579999999999 2024-09-20T18:54:53.469 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.469 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.469 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:53.469 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:05.044617+0000", 2024-09-20T18:54:53.469 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000261165 2024-09-20T18:54:53.469 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.469 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.469 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:53.469 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:05.102905+0000", 2024-09-20T18:54:53.469 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.058287525 2024-09-20T18:54:53.469 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.469 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.470 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:54:53.470 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:05.102959+0000", 2024-09-20T18:54:53.470 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4398000000000003e-05 2024-09-20T18:54:53.470 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.470 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.470 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:54:53.470 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:05.102995+0000", 2024-09-20T18:54:53.470 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.5150000000000001e-05 2024-09-20T18:54:53.470 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:54:53.470 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:54:53.470 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:54:53.470 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.470 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.471 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:120202 7.1 7:8a289ff9:::benchmark_data_smithi083_70643_object120201:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T18:54:53.471 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:24.982702+0000", 2024-09-20T18:54:53.471 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 148.008774472, 2024-09-20T18:54:53.471 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.096958845000000002, 2024-09-20T18:54:53.471 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:54:53.471 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:54:53.471 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:54:53.471 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:54:53.471 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:54:53.471 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:54:53.471 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 120202 2024-09-20T18:54:53.471 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.471 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:54:53.471 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.472 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:54:53.472 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.982702+0000", 2024-09-20T18:54:53.472 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:54:53.472 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.472 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.472 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:54:53.472 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.982702+0000", 2024-09-20T18:54:53.472 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:54:53.472 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.472 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.472 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:54:53.472 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.982703+0000", 2024-09-20T18:54:53.472 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9199999999999998e-06 2024-09-20T18:54:53.472 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.473 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.473 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:54:53.473 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.982746+0000", 2024-09-20T18:54:53.473 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.2809000000000003e-05 2024-09-20T18:54:53.473 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.473 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.473 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:54:53.473 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.982757+0000", 2024-09-20T18:54:53.473 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0412e-05 2024-09-20T18:54:53.473 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.473 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.473 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:54:53.473 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.982762+0000", 2024-09-20T18:54:53.473 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6080000000000003e-06 2024-09-20T18:54:53.474 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.474 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.474 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:54:53.474 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.982802+0000", 2024-09-20T18:54:53.474 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9857000000000001e-05 2024-09-20T18:54:53.474 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.474 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.474 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:54:53.474 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.982899+0000", 2024-09-20T18:54:53.474 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.7187000000000003e-05 2024-09-20T18:54:53.475 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.475 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.475 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,6", 2024-09-20T18:54:53.475 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.982999+0000", 2024-09-20T18:54:53.475 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.9246999999999994e-05 2024-09-20T18:54:53.475 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.475 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.475 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:54:53.475 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.985215+0000", 2024-09-20T18:54:53.475 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0022165829999999998 2024-09-20T18:54:53.475 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.475 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.476 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:53.476 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.010015+0000", 2024-09-20T18:54:53.476 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.024799796999999998 2024-09-20T18:54:53.476 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.476 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.476 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:53.476 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.079514+0000", 2024-09-20T18:54:53.476 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.069499218000000001 2024-09-20T18:54:53.476 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.476 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.476 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:54:53.476 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.079616+0000", 2024-09-20T18:54:53.476 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000102236 2024-09-20T18:54:53.476 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.477 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.477 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:54:53.477 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.079660+0000", 2024-09-20T18:54:53.477 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3971000000000003e-05 2024-09-20T18:54:53.477 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:54:53.477 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:54:53.477 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:54:53.477 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.477 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.477 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:120207 7.1 7:8d667e1f:::benchmark_data_smithi083_70643_object120206:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T18:54:53.477 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:24.990072+0000", 2024-09-20T18:54:53.477 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 148.001404205, 2024-09-20T18:54:53.477 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.089731845000000005, 2024-09-20T18:54:53.477 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:54:53.478 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:54:53.478 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:54:53.478 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:54:53.478 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:54:53.478 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:54:53.478 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 120207 2024-09-20T18:54:53.478 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.478 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:54:53.478 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.478 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:54:53.478 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.990072+0000", 2024-09-20T18:54:53.478 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:54:53.478 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.478 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.479 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:54:53.479 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.990072+0000", 2024-09-20T18:54:53.479 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:54:53.479 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.479 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.479 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:54:53.479 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.990073+0000", 2024-09-20T18:54:53.479 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6950000000000001e-06 2024-09-20T18:54:53.479 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.479 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.479 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:54:53.479 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.990122+0000", 2024-09-20T18:54:53.479 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8696999999999998e-05 2024-09-20T18:54:53.480 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.480 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.480 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:54:53.480 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.009979+0000", 2024-09-20T18:54:53.480 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.019857099999999999 2024-09-20T18:54:53.480 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.480 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.480 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:54:53.480 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.009989+0000", 2024-09-20T18:54:53.480 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.3000000000000007e-06 2024-09-20T18:54:53.480 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.480 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.480 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:54:53.480 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.010053+0000", 2024-09-20T18:54:53.481 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4024999999999998e-05 2024-09-20T18:54:53.481 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.481 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.481 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:54:53.481 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.010170+0000", 2024-09-20T18:54:53.481 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000117617 2024-09-20T18:54:53.481 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.481 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.481 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,6", 2024-09-20T18:54:53.481 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.010268+0000", 2024-09-20T18:54:53.481 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.7713000000000004e-05 2024-09-20T18:54:53.481 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.481 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.481 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:54:53.482 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.012425+0000", 2024-09-20T18:54:53.482 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0021567829999999998 2024-09-20T18:54:53.482 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.482 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.482 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:53.482 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.012576+0000", 2024-09-20T18:54:53.482 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015128 2024-09-20T18:54:53.482 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.482 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.482 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:53.482 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.079714+0000", 2024-09-20T18:54:53.482 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.067138113999999999 2024-09-20T18:54:53.482 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.483 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.483 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:54:53.483 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.079774+0000", 2024-09-20T18:54:53.483 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0402000000000002e-05 2024-09-20T18:54:53.483 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.483 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.483 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:54:53.483 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.079804+0000", 2024-09-20T18:54:53.483 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9119e-05 2024-09-20T18:54:53.483 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:54:53.483 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:54:53.483 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:54:53.483 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.483 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.484 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:139464 7.1 7:8d273f11:::benchmark_data_smithi083_70643_object16662:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-20T18:54:53.484 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:43.118758+0000", 2024-09-20T18:54:53.484 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 129.87271843100001, 2024-09-20T18:54:53.484 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.094121622000000002, 2024-09-20T18:54:53.484 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:54:53.484 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:54:53.484 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:54:53.484 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:54:53.484 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:54:53.484 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:54:53.484 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 139464 2024-09-20T18:54:53.484 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.484 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:54:53.484 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.485 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:54:53.485 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.118758+0000", 2024-09-20T18:54:53.485 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:54:53.485 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.485 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.485 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:54:53.485 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.118758+0000", 2024-09-20T18:54:53.485 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:54:53.485 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.485 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.485 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:54:53.485 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.118759+0000", 2024-09-20T18:54:53.485 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5659999999999999e-06 2024-09-20T18:54:53.486 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.486 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.486 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:54:53.486 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.118769+0000", 2024-09-20T18:54:53.486 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0091e-05 2024-09-20T18:54:53.486 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.486 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.486 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:54:53.486 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.118782+0000", 2024-09-20T18:54:53.486 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3293e-05 2024-09-20T18:54:53.486 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.486 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.486 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:54:53.486 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.118790+0000", 2024-09-20T18:54:53.487 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.0380000000000002e-06 2024-09-20T18:54:53.487 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.487 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.487 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:54:53.487 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.118845+0000", 2024-09-20T18:54:53.487 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5640999999999998e-05 2024-09-20T18:54:53.487 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.487 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.487 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:54:53.487 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.118969+0000", 2024-09-20T18:54:53.487 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012337599999999999 2024-09-20T18:54:53.487 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.487 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.487 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,6", 2024-09-20T18:54:53.488 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.119100+0000", 2024-09-20T18:54:53.488 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000130994 2024-09-20T18:54:53.488 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.488 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.488 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:53.488 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.123341+0000", 2024-09-20T18:54:53.488 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.004240995 2024-09-20T18:54:53.488 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.488 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.488 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:54:53.488 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.123423+0000", 2024-09-20T18:54:53.488 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.2300999999999997e-05 2024-09-20T18:54:53.488 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.488 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.489 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:53.489 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.212798+0000", 2024-09-20T18:54:53.489 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.089375316999999996 2024-09-20T18:54:53.489 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.489 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.489 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:54:53.489 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.212835+0000", 2024-09-20T18:54:53.489 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6758000000000003e-05 2024-09-20T18:54:53.489 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.489 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.489 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:54:53.489 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.212879+0000", 2024-09-20T18:54:53.490 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.4252000000000003e-05 2024-09-20T18:54:53.490 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:54:53.490 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:54:53.490 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:54:53.490 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.490 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.490 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:139471 7.1 7:8b47cc00:::benchmark_data_smithi083_70643_object16669:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-20T18:54:53.490 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:43.123464+0000", 2024-09-20T18:54:53.490 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 129.86801204400001, 2024-09-20T18:54:53.490 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.089504035999999995, 2024-09-20T18:54:53.490 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:54:53.490 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:54:53.490 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:54:53.490 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:54:53.491 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:54:53.491 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:54:53.491 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 139471 2024-09-20T18:54:53.491 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.491 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:54:53.491 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.491 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:54:53.491 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.123464+0000", 2024-09-20T18:54:53.491 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:54:53.491 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.491 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.491 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:54:53.491 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.123464+0000", 2024-09-20T18:54:53.491 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:54:53.492 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.492 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.492 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:54:53.492 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.123465+0000", 2024-09-20T18:54:53.492 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.471e-06 2024-09-20T18:54:53.492 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.492 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.492 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:54:53.492 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.123474+0000", 2024-09-20T18:54:53.492 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.67e-06 2024-09-20T18:54:53.492 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.492 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.492 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:54:53.492 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.123487+0000", 2024-09-20T18:54:53.493 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3134e-05 2024-09-20T18:54:53.493 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.493 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.493 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:54:53.493 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.123494+0000", 2024-09-20T18:54:53.493 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5950000000000004e-06 2024-09-20T18:54:53.493 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.493 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.493 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:54:53.493 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.123544+0000", 2024-09-20T18:54:53.493 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0099999999999998e-05 2024-09-20T18:54:53.493 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.493 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.493 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:54:53.494 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.123661+0000", 2024-09-20T18:54:53.494 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000116925 2024-09-20T18:54:53.494 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.494 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.494 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,6", 2024-09-20T18:54:53.494 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.123772+0000", 2024-09-20T18:54:53.494 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011149700000000001 2024-09-20T18:54:53.494 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.494 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.494 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:54:53.494 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.126783+0000", 2024-09-20T18:54:53.494 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00301031 2024-09-20T18:54:53.494 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.495 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.495 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:53.495 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.127290+0000", 2024-09-20T18:54:53.495 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00050741800000000004 2024-09-20T18:54:53.495 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.495 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.495 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:53.495 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.212914+0000", 2024-09-20T18:54:53.495 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.085624152999999995 2024-09-20T18:54:53.495 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.495 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.495 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:54:53.495 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.212947+0000", 2024-09-20T18:54:53.495 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.2781999999999999e-05 2024-09-20T18:54:53.495 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.496 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.496 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:54:53.496 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.212968+0000", 2024-09-20T18:54:53.496 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0981000000000001e-05 2024-09-20T18:54:53.496 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:54:53.496 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:54:53.496 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:54:53.496 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.496 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.496 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:156706 7.1 7:8e235435:::benchmark_data_smithi083_70643_object33904:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133)", 2024-09-20T18:54:53.496 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:59.417396+0000", 2024-09-20T18:54:53.496 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 113.574080471, 2024-09-20T18:54:53.496 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.065579346999999996, 2024-09-20T18:54:53.496 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:54:53.497 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:54:53.497 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:54:53.497 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:54:53.497 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:54:53.497 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:54:53.497 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 156706 2024-09-20T18:54:53.497 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.497 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:54:53.497 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.497 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:54:53.497 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:59.417396+0000", 2024-09-20T18:54:53.497 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:54:53.497 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.497 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.498 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:54:53.498 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:59.417396+0000", 2024-09-20T18:54:53.498 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:54:53.498 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.498 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.498 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:54:53.498 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:59.417397+0000", 2024-09-20T18:54:53.498 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5379999999999999e-06 2024-09-20T18:54:53.498 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.498 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.498 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:54:53.498 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:59.417402+0000", 2024-09-20T18:54:53.498 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.5369999999999999e-06 2024-09-20T18:54:53.498 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.499 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.499 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:54:53.499 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:59.417412+0000", 2024-09-20T18:54:53.499 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0573e-05 2024-09-20T18:54:53.499 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.499 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.499 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:54:53.499 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:59.417418+0000", 2024-09-20T18:54:53.499 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3510000000000001e-06 2024-09-20T18:54:53.499 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.499 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.499 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:54:53.499 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:59.417454+0000", 2024-09-20T18:54:53.499 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6559e-05 2024-09-20T18:54:53.499 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.500 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.500 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:54:53.500 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:59.417542+0000", 2024-09-20T18:54:53.500 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.7867000000000004e-05 2024-09-20T18:54:53.500 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.500 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.500 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,6", 2024-09-20T18:54:53.500 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:59.417632+0000", 2024-09-20T18:54:53.500 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.9689000000000001e-05 2024-09-20T18:54:53.500 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.500 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.500 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:54:53.500 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:59.419875+0000", 2024-09-20T18:54:53.500 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0022428919999999998 2024-09-20T18:54:53.501 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.501 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.501 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:53.501 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:59.419955+0000", 2024-09-20T18:54:53.501 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.0074999999999995e-05 2024-09-20T18:54:53.501 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.501 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.501 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:53.501 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:59.482908+0000", 2024-09-20T18:54:53.501 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.062953141000000004 2024-09-20T18:54:53.501 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.501 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.501 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:54:53.501 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:59.482943+0000", 2024-09-20T18:54:53.501 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4876999999999998e-05 2024-09-20T18:54:53.502 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.502 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.502 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:54:53.502 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:59.482975+0000", 2024-09-20T18:54:53.502 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.2248e-05 2024-09-20T18:54:53.502 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:54:53.502 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:54:53.502 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:54:53.502 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.502 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.502 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:188393 7.1 7:83b2a712:::benchmark_data_smithi083_70643_object65591:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-20T18:54:53.502 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:53:29.674052+0000", 2024-09-20T18:54:53.502 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 83.317424423999995, 2024-09-20T18:54:53.503 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.087096657999999993, 2024-09-20T18:54:53.503 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:54:53.503 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:54:53.503 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:54:53.503 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:54:53.503 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:54:53.503 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:54:53.503 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 188393 2024-09-20T18:54:53.503 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.503 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:54:53.503 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.503 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:54:53.503 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.674052+0000", 2024-09-20T18:54:53.503 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:54:53.504 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.504 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.504 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:54:53.504 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.674052+0000", 2024-09-20T18:54:53.504 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:54:53.504 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.504 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.504 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:54:53.504 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.674053+0000", 2024-09-20T18:54:53.504 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.722e-06 2024-09-20T18:54:53.504 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.504 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.504 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:54:53.504 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.674058+0000", 2024-09-20T18:54:53.505 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8860000000000003e-06 2024-09-20T18:54:53.505 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.505 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.505 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:54:53.505 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.674071+0000", 2024-09-20T18:54:53.505 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3071000000000001e-05 2024-09-20T18:54:53.505 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.505 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.505 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:54:53.505 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.674079+0000", 2024-09-20T18:54:53.505 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.3390000000000004e-06 2024-09-20T18:54:53.505 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.505 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.505 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:54:53.506 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.674133+0000", 2024-09-20T18:54:53.506 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4320000000000002e-05 2024-09-20T18:54:53.506 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.506 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.506 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:54:53.506 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.674257+0000", 2024-09-20T18:54:53.506 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000124282 2024-09-20T18:54:53.506 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.506 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.506 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,6", 2024-09-20T18:54:53.506 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.674377+0000", 2024-09-20T18:54:53.506 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000120233 2024-09-20T18:54:53.506 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.506 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.507 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:53.507 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.677167+0000", 2024-09-20T18:54:53.507 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0027899330000000001 2024-09-20T18:54:53.507 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.507 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.507 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:54:53.507 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.682503+0000", 2024-09-20T18:54:53.507 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0053358820000000001 2024-09-20T18:54:53.507 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.507 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.507 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:53.507 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.761074+0000", 2024-09-20T18:54:53.507 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.078571252999999994 2024-09-20T18:54:53.507 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.508 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.508 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:54:53.508 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.761110+0000", 2024-09-20T18:54:53.508 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.5768e-05 2024-09-20T18:54:53.508 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.508 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.508 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:54:53.508 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.761148+0000", 2024-09-20T18:54:53.508 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.7969000000000002e-05 2024-09-20T18:54:53.508 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:54:53.508 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:54:53.508 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:54:53.508 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.508 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.509 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:188400 7.1 7:8dbba07a:::benchmark_data_smithi083_70643_object65598:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-20T18:54:53.509 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:53:29.679529+0000", 2024-09-20T18:54:53.509 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 83.311947097000001, 2024-09-20T18:54:53.509 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.081768355000000001, 2024-09-20T18:54:53.509 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:54:53.509 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:54:53.509 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:54:53.509 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:54:53.509 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:54:53.509 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:54:53.509 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 188400 2024-09-20T18:54:53.509 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.509 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:54:53.509 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.509 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:54:53.510 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.679529+0000", 2024-09-20T18:54:53.510 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:54:53.510 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.510 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.510 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:54:53.510 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.679529+0000", 2024-09-20T18:54:53.510 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:54:53.510 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.510 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.510 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:54:53.510 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.679530+0000", 2024-09-20T18:54:53.510 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5379999999999999e-06 2024-09-20T18:54:53.510 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.510 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.511 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:54:53.511 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.679539+0000", 2024-09-20T18:54:53.511 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.9400000000000008e-06 2024-09-20T18:54:53.511 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.511 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.511 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:54:53.511 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.679555+0000", 2024-09-20T18:54:53.511 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5308999999999999e-05 2024-09-20T18:54:53.511 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.511 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.511 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:54:53.511 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.679562+0000", 2024-09-20T18:54:53.511 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.9009999999999997e-06 2024-09-20T18:54:53.511 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.512 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.512 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:54:53.512 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.679677+0000", 2024-09-20T18:54:53.512 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000115882 2024-09-20T18:54:53.512 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.512 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.512 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:54:53.512 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.679809+0000", 2024-09-20T18:54:53.512 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013178699999999999 2024-09-20T18:54:53.512 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.512 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.512 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,6", 2024-09-20T18:54:53.512 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.679935+0000", 2024-09-20T18:54:53.512 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012615799999999999 2024-09-20T18:54:53.513 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.513 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.513 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:54:53.513 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.682690+0000", 2024-09-20T18:54:53.513 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.002754322 2024-09-20T18:54:53.513 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.513 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.513 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:53.513 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.683012+0000", 2024-09-20T18:54:53.513 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00032239600000000002 2024-09-20T18:54:53.513 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.513 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.513 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:53.513 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.761198+0000", 2024-09-20T18:54:53.513 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.078186126999999994 2024-09-20T18:54:53.514 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.514 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.514 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:54:53.514 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.761247+0000", 2024-09-20T18:54:53.514 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8720000000000001e-05 2024-09-20T18:54:53.514 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.514 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.514 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:54:53.514 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.761297+0000", 2024-09-20T18:54:53.514 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0275000000000002e-05 2024-09-20T18:54:53.514 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:54:53.514 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:54:53.514 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:54:53.514 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.515 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.515 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.24955.0:20800 8.2 8:432f30b8:::benchmark_data_smithi083_97422_object20799:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e166)", 2024-09-20T18:54:53.515 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:54:47.519031+0000", 2024-09-20T18:54:53.515 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 5.4724447659999997, 2024-09-20T18:54:53.515 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10456202000000001, 2024-09-20T18:54:53.515 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:54:53.515 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:54:53.515 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:54:53.515 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:54:53.515 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.24955", 2024-09-20T18:54:53.515 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:54:53.515 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 20800 2024-09-20T18:54:53.515 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.515 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:54:53.516 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.516 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:54:53.516 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:47.519031+0000", 2024-09-20T18:54:53.516 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:54:53.516 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.516 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.516 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:54:53.516 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:47.519031+0000", 2024-09-20T18:54:53.516 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:54:53.516 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.516 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.516 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:54:53.516 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:47.519038+0000", 2024-09-20T18:54:53.516 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.9419999999999998e-06 2024-09-20T18:54:53.517 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.517 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.517 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:54:53.517 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:47.519078+0000", 2024-09-20T18:54:53.517 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9749000000000002e-05 2024-09-20T18:54:53.517 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.517 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.517 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:54:53.517 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:47.519096+0000", 2024-09-20T18:54:53.517 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.844e-05 2024-09-20T18:54:53.517 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.517 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.517 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:54:53.517 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:47.519106+0000", 2024-09-20T18:54:53.517 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.3179999999999996e-06 2024-09-20T18:54:53.518 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.518 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.518 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:54:53.518 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:47.519166+0000", 2024-09-20T18:54:53.518 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.054e-05 2024-09-20T18:54:53.518 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.518 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.518 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:54:53.518 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:47.519300+0000", 2024-09-20T18:54:53.518 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013389399999999999 2024-09-20T18:54:53.518 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.518 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.518 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:54:53.518 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:47.519448+0000", 2024-09-20T18:54:53.519 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00014763499999999999 2024-09-20T18:54:53.519 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.519 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.519 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:54:53.519 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:47.520929+0000", 2024-09-20T18:54:53.519 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001480931 2024-09-20T18:54:53.519 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.519 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.519 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:53.519 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:47.521905+0000", 2024-09-20T18:54:53.519 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00097665300000000007 2024-09-20T18:54:53.519 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.519 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.519 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:53.520 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:47.623479+0000", 2024-09-20T18:54:53.520 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.101573859 2024-09-20T18:54:53.520 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.520 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.520 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:54:53.520 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:47.623542+0000", 2024-09-20T18:54:53.520 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2996999999999993e-05 2024-09-20T18:54:53.520 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.520 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.520 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:54:53.520 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:47.623593+0000", 2024-09-20T18:54:53.520 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.1062e-05 2024-09-20T18:54:53.520 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:54:53.521 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:54:53.521 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:54:53.521 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.521 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.521 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.24955.0:23894 8.2 8:47a50b33:::benchmark_data_smithi083_97422_object23893:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e167)", 2024-09-20T18:54:53.521 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:54:49.130428+0000", 2024-09-20T18:54:53.521 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 3.8610477620000001, 2024-09-20T18:54:53.521 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.080818846999999999, 2024-09-20T18:54:53.521 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:54:53.521 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:54:53.521 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:54:53.521 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:54:53.521 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.24955", 2024-09-20T18:54:53.521 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:54:53.522 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 23894 2024-09-20T18:54:53.522 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.522 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:54:53.522 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.522 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:54:53.522 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.130428+0000", 2024-09-20T18:54:53.522 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:54:53.522 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.522 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.522 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:54:53.522 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.130428+0000", 2024-09-20T18:54:53.522 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:54:53.522 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.522 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.523 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:54:53.523 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.130430+0000", 2024-09-20T18:54:53.523 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.415e-06 2024-09-20T18:54:53.523 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.523 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.523 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:54:53.523 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.130477+0000", 2024-09-20T18:54:53.523 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7138999999999997e-05 2024-09-20T18:54:53.523 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.523 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.523 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:54:53.523 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.130489+0000", 2024-09-20T18:54:53.523 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2546e-05 2024-09-20T18:54:53.523 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.524 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.524 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:54:53.524 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.130496+0000", 2024-09-20T18:54:53.524 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.1860000000000003e-06 2024-09-20T18:54:53.524 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.524 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.524 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:54:53.524 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.130547+0000", 2024-09-20T18:54:53.524 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0518e-05 2024-09-20T18:54:53.524 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.524 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.524 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:54:53.524 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.130696+0000", 2024-09-20T18:54:53.524 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00014871799999999999 2024-09-20T18:54:53.525 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.525 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.525 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:54:53.525 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.130834+0000", 2024-09-20T18:54:53.525 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000138288 2024-09-20T18:54:53.525 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.525 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.525 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:54:53.525 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.131853+0000", 2024-09-20T18:54:53.525 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001018808 2024-09-20T18:54:53.525 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.525 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.525 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:53.525 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.132010+0000", 2024-09-20T18:54:53.526 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015724099999999999 2024-09-20T18:54:53.526 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.526 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.526 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:53.526 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.211173+0000", 2024-09-20T18:54:53.526 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.079163270999999993 2024-09-20T18:54:53.526 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.526 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.526 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:54:53.526 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.211213+0000", 2024-09-20T18:54:53.526 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9239000000000002e-05 2024-09-20T18:54:53.526 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.526 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.527 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:54:53.527 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.211247+0000", 2024-09-20T18:54:53.527 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4477999999999998e-05 2024-09-20T18:54:53.527 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:54:53.527 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:54:53.527 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:54:53.527 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.527 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.527 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.24955.0:25354 8.2 8:431f5c8d:::benchmark_data_smithi083_97422_object25353:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e167)", 2024-09-20T18:54:53.527 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:54:49.818860+0000", 2024-09-20T18:54:53.527 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 3.172615489, 2024-09-20T18:54:53.527 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.068573554999999994, 2024-09-20T18:54:53.527 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:54:53.527 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:54:53.528 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:54:53.528 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:54:53.528 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.24955", 2024-09-20T18:54:53.528 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:54:53.528 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 25354 2024-09-20T18:54:53.528 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.528 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:54:53.528 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.528 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:54:53.528 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.818860+0000", 2024-09-20T18:54:53.528 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:54:53.528 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.528 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.528 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:54:53.529 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.818860+0000", 2024-09-20T18:54:53.529 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:54:53.529 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.529 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.529 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:54:53.529 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.818862+0000", 2024-09-20T18:54:53.529 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5880000000000001e-06 2024-09-20T18:54:53.529 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.529 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.529 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:53 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set-full-ratio", "ratio": 0.001} : dispatch 2024-09-20T18:54:53.530 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:54:53.530 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.818904+0000", 2024-09-20T18:54:53.530 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.2258000000000001e-05 2024-09-20T18:54:53.530 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.530 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.530 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:54:53.530 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.818932+0000", 2024-09-20T18:54:53.530 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.7183999999999999e-05 2024-09-20T18:54:53.530 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.530 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.531 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:54:53.531 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.818939+0000", 2024-09-20T18:54:53.531 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.6850000000000001e-06 2024-09-20T18:54:53.531 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.531 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.531 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:54:53.531 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.818990+0000", 2024-09-20T18:54:53.531 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.1088999999999998e-05 2024-09-20T18:54:53.531 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.531 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.531 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:54:53.531 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.819101+0000", 2024-09-20T18:54:53.531 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000110309 2024-09-20T18:54:53.531 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.532 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.532 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:54:53.532 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.819214+0000", 2024-09-20T18:54:53.532 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000113566 2024-09-20T18:54:53.532 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.532 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.532 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:54:53.532 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.821435+0000", 2024-09-20T18:54:53.532 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.002220783 2024-09-20T18:54:53.532 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.532 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.532 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:53.532 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.821473+0000", 2024-09-20T18:54:53.532 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.7561999999999997e-05 2024-09-20T18:54:53.533 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.533 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.533 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:53.533 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.887334+0000", 2024-09-20T18:54:53.533 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.065861193999999998 2024-09-20T18:54:53.533 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.533 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.533 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:54:53.533 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.887394+0000", 2024-09-20T18:54:53.533 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0296e-05 2024-09-20T18:54:53.533 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.533 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.533 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:54:53.533 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.887434+0000", 2024-09-20T18:54:53.534 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0040999999999998e-05 2024-09-20T18:54:53.534 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:54:53.534 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:54:53.534 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:54:53.534 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.534 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.534 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.24955.0:25356 8.2 8:47b04c19:::benchmark_data_smithi083_97422_object25355:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e167)", 2024-09-20T18:54:53.534 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:54:49.820191+0000", 2024-09-20T18:54:53.534 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 3.1712849240000001, 2024-09-20T18:54:53.534 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.067379626999999997, 2024-09-20T18:54:53.534 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:54:53.534 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:54:53.534 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:54:53.535 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:54:53.535 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.24955", 2024-09-20T18:54:53.535 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:54:53.535 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 25356 2024-09-20T18:54:53.535 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.535 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:54:53.535 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.535 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:54:53.535 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.820191+0000", 2024-09-20T18:54:53.535 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:54:53.535 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.535 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.535 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:54:53.535 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.820191+0000", 2024-09-20T18:54:53.536 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:54:53.536 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.536 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.536 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:54:53.536 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.820192+0000", 2024-09-20T18:54:53.536 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1119999999999999e-06 2024-09-20T18:54:53.536 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.536 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.536 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:54:53.536 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.820257+0000", 2024-09-20T18:54:53.536 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5166000000000006e-05 2024-09-20T18:54:53.536 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.536 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.536 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:54:53.537 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.820267+0000", 2024-09-20T18:54:53.537 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0141e-05 2024-09-20T18:54:53.537 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.537 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.537 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:54:53.537 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.820275+0000", 2024-09-20T18:54:53.537 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.6739999999999997e-06 2024-09-20T18:54:53.537 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.537 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.537 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:54:53.537 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.820329+0000", 2024-09-20T18:54:53.537 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3477000000000003e-05 2024-09-20T18:54:53.537 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.537 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.538 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:54:53.538 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.820454+0000", 2024-09-20T18:54:53.538 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012544499999999999 2024-09-20T18:54:53.538 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.538 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.538 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:54:53.538 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.820647+0000", 2024-09-20T18:54:53.538 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000193115 2024-09-20T18:54:53.538 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.538 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.538 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:54:53.538 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.821557+0000", 2024-09-20T18:54:53.538 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00091003399999999999 2024-09-20T18:54:53.539 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.539 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.539 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:53.539 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.822268+0000", 2024-09-20T18:54:53.539 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00071042999999999996 2024-09-20T18:54:53.539 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.539 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.539 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:53.539 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.887481+0000", 2024-09-20T18:54:53.539 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.065213244000000004 2024-09-20T18:54:53.539 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.539 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.539 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:54:53.540 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.887534+0000", 2024-09-20T18:54:53.540 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3168999999999999e-05 2024-09-20T18:54:53.540 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.540 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.540 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:54:53.540 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.887571+0000", 2024-09-20T18:54:53.540 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6619999999999998e-05 2024-09-20T18:54:53.540 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:54:53.540 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:54:53.540 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:54:53.540 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.540 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.540 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.24955.0:25363 8.2 8:449cd64b:::benchmark_data_smithi083_97422_object25362:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e167)", 2024-09-20T18:54:53.540 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:54:49.822512+0000", 2024-09-20T18:54:53.541 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 3.1689642, 2024-09-20T18:54:53.541 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.065187761999999996, 2024-09-20T18:54:53.541 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:54:53.541 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:54:53.541 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:54:53.541 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:54:53.541 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.24955", 2024-09-20T18:54:53.541 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:54:53.541 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 25363 2024-09-20T18:54:53.541 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.541 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:54:53.541 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.541 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:54:53.541 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.822512+0000", 2024-09-20T18:54:53.541 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:54:53.542 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.542 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.542 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:54:53.542 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.822512+0000", 2024-09-20T18:54:53.542 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:54:53.542 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.542 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.542 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:54:53.542 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.822513+0000", 2024-09-20T18:54:53.542 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.0500000000000002e-07 2024-09-20T18:54:53.542 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.542 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.542 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:54:53.543 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.822575+0000", 2024-09-20T18:54:53.543 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.1909000000000006e-05 2024-09-20T18:54:53.543 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.543 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.543 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:54:53.543 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.822582+0000", 2024-09-20T18:54:53.543 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.8099999999999998e-06 2024-09-20T18:54:53.543 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.543 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.543 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:54:53.543 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.822587+0000", 2024-09-20T18:54:53.543 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.5820000000000003e-06 2024-09-20T18:54:53.543 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.543 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.544 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:54:53.544 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.822620+0000", 2024-09-20T18:54:53.544 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.2903e-05 2024-09-20T18:54:53.544 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.544 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.544 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:54:53.544 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.822690+0000", 2024-09-20T18:54:53.544 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.9673999999999997e-05 2024-09-20T18:54:53.544 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.544 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.544 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:54:53.544 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.822767+0000", 2024-09-20T18:54:53.544 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.7056000000000002e-05 2024-09-20T18:54:53.544 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.545 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.545 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:54:53.545 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.823404+0000", 2024-09-20T18:54:53.545 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00063742599999999996 2024-09-20T18:54:53.545 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.545 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.545 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:53.545 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.881843+0000", 2024-09-20T18:54:53.545 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.058438543000000003 2024-09-20T18:54:53.545 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.545 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.545 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:54:53.545 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.887616+0000", 2024-09-20T18:54:53.545 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0057731140000000002 2024-09-20T18:54:53.546 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.546 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.546 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:54:53.546 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.887664+0000", 2024-09-20T18:54:53.546 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8368000000000002e-05 2024-09-20T18:54:53.546 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:54:53.546 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:54:53.546 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:54:53.546 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:49.887700+0000", 2024-09-20T18:54:53.546 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.5472000000000002e-05 2024-09-20T18:54:53.546 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:54:53.546 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:54:53.546 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:54:53.546 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:54:53.547 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:54:53.547 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T18:54:53.547 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:54:53.575 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:53.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:53.576 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:54:53.677 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:53.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:53.699 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:53.699 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:53.800 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:53.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:53.842 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:53.842 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:53.942 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:53.976 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:53.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:53.977 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:54.077 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:54.112 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:54.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:54.113 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:54.116 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T18:54:54.171 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 14 16 34181 34165 152.346 124.75 0.00880626 0.00655829 2024-09-20T18:54:54.213 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:54.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:54.235 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:54.235 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:54.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:54 smithi142 ceph-mon[32508]: pgmap v235: 148 pgs: 148 active+clean; 1.7 GiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1.0 KiB/s rd, 161 MiB/s wr, 2.57k op/s 2024-09-20T18:54:54.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:54 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:54:54.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:54 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:54:54.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:54 smithi142 ceph-mon[32508]: Health check failed: full ratio(s) out of order (OSD_OUT_OF_ORDER_FULL) 2024-09-20T18:54:54.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:54 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set-full-ratio", "ratio": 0.001}]': finished 2024-09-20T18:54:54.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:54 smithi142 ceph-mon[32508]: osdmap e168: 8 total, 8 up, 8 in 2024-09-20T18:54:54.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:54 smithi142 ceph-mon[32508]: 4.1a scrub starts 2024-09-20T18:54:54.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:54 smithi142 ceph-mon[32508]: 4.1a scrub ok 2024-09-20T18:54:54.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:54 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/897398990' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:54:54.337 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:54.368 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:54.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:54.369 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:54.455 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:54 smithi083 ceph-mon[29421]: pgmap v235: 148 pgs: 148 active+clean; 1.7 GiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1.0 KiB/s rd, 161 MiB/s wr, 2.57k op/s 2024-09-20T18:54:54.586 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:54 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:54:54.586 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:54 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:54:54.586 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:54 smithi083 ceph-mon[29421]: Health check failed: full ratio(s) out of order (OSD_OUT_OF_ORDER_FULL) 2024-09-20T18:54:54.586 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:54 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set-full-ratio", "ratio": 0.001}]': finished 2024-09-20T18:54:54.586 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:54 smithi083 ceph-mon[29421]: osdmap e168: 8 total, 8 up, 8 in 2024-09-20T18:54:54.586 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:54 smithi083 ceph-mon[29421]: 4.1a scrub starts 2024-09-20T18:54:54.587 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:54 smithi083 ceph-mon[29421]: 4.1a scrub ok 2024-09-20T18:54:54.587 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:54 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/897398990' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:54:54.587 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:54:54.587 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:54:54.587 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:54:54.587 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:54:54.587 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:54:54.587 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:54:54.588 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:54:54.588 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:54:54.588 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:54:54.588 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:54:54.588 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:54:54.588 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:54:54.588 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:54:54.588 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:54:54.588 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:54:54.588 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:54:54.589 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:54 smithi083 ceph-mon[36260]: pgmap v235: 148 pgs: 148 active+clean; 1.7 GiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1.0 KiB/s rd, 161 MiB/s wr, 2.57k op/s 2024-09-20T18:54:54.589 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:54 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:54:54.589 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:54 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:54:54.589 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:54 smithi083 ceph-mon[36260]: Health check failed: full ratio(s) out of order (OSD_OUT_OF_ORDER_FULL) 2024-09-20T18:54:54.589 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:54 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set-full-ratio", "ratio": 0.001}]': finished 2024-09-20T18:54:54.589 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:54 smithi083 ceph-mon[36260]: osdmap e168: 8 total, 8 up, 8 in 2024-09-20T18:54:54.589 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:54 smithi083 ceph-mon[36260]: 4.1a scrub starts 2024-09-20T18:54:54.589 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:54 smithi083 ceph-mon[36260]: 4.1a scrub ok 2024-09-20T18:54:54.589 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:54 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/897398990' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:54:54.589 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:54.616 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:54.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:54.617 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:54.718 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:54.754 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:54.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:54.755 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:54.856 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:54.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:54.879 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:54.879 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:54.980 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:55.016 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:55.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:55.017 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:55.118 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:55.153 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:55.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:55.154 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:55.170 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 15 16 35893 35877 149.325 107 0.00697883 0.0066909 2024-09-20T18:54:55.255 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:54:55.290 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:55.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:55.290 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:54:55.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:55 smithi083 ceph-mon[29421]: 3.8 scrub starts 2024-09-20T18:54:55.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:55 smithi083 ceph-mon[29421]: 3.8 scrub ok 2024-09-20T18:54:55.303 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:55 smithi083 ceph-mon[36260]: 3.8 scrub starts 2024-09-20T18:54:55.303 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:55 smithi083 ceph-mon[36260]: 3.8 scrub ok 2024-09-20T18:54:55.391 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:55.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:55.416 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:55.416 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:55.517 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:55.522 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:55 smithi142 ceph-mon[32508]: 3.8 scrub starts 2024-09-20T18:54:55.522 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:55 smithi142 ceph-mon[32508]: 3.8 scrub ok 2024-09-20T18:54:55.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:55.541 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:55.541 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:55.642 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:55.674 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:55.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:55.675 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:55.776 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:55.813 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:55.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:55.814 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:55.915 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:54:55.946 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:55.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:55.948 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:54:56.048 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:54:56.076 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:56.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:56.077 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:54:56.171 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 16 16 37411 37395 145.925 94.875 0.0056492 0.00684695 2024-09-20T18:54:56.178 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:54:56.209 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:56.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:56.210 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:54:56.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:56 smithi083 ceph-mon[29421]: pgmap v237: 148 pgs: 148 active+clean; 1.7 GiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 84 MiB/s wr, 1.35k op/s 2024-09-20T18:54:56.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:56 smithi083 ceph-mon[29421]: 4.b scrub starts 2024-09-20T18:54:56.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:56 smithi083 ceph-mon[29421]: 4.b scrub ok 2024-09-20T18:54:56.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:56 smithi083 ceph-mon[29421]: 6.2 scrub starts 2024-09-20T18:54:56.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:56 smithi083 ceph-mon[29421]: 6.2 scrub ok 2024-09-20T18:54:56.303 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:56 smithi083 ceph-mon[36260]: pgmap v237: 148 pgs: 148 active+clean; 1.7 GiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 84 MiB/s wr, 1.35k op/s 2024-09-20T18:54:56.303 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:56 smithi083 ceph-mon[36260]: 4.b scrub starts 2024-09-20T18:54:56.303 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:56 smithi083 ceph-mon[36260]: 4.b scrub ok 2024-09-20T18:54:56.303 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:56 smithi083 ceph-mon[36260]: 6.2 scrub starts 2024-09-20T18:54:56.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:56 smithi083 ceph-mon[36260]: 6.2 scrub ok 2024-09-20T18:54:56.311 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:56.332 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:56.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:56.333 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:56.433 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:56.468 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:56.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:56.469 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:56.523 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:56 smithi142 ceph-mon[32508]: pgmap v237: 148 pgs: 148 active+clean; 1.7 GiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 84 MiB/s wr, 1.35k op/s 2024-09-20T18:54:56.523 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:56 smithi142 ceph-mon[32508]: 4.b scrub starts 2024-09-20T18:54:56.523 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:56 smithi142 ceph-mon[32508]: 4.b scrub ok 2024-09-20T18:54:56.523 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:56 smithi142 ceph-mon[32508]: 6.2 scrub starts 2024-09-20T18:54:56.523 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:56 smithi142 ceph-mon[32508]: 6.2 scrub ok 2024-09-20T18:54:56.570 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:56.604 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:56.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:56.605 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:56.705 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:54:56.738 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:56.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:56.739 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:54:56.822 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-20T18:54:56.840 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:56.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:56.886 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:56.887 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:56.987 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:57.001 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:54:57.022 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:54:56 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:54:57.025 INFO:tasks.thrashosds.thrasher:Setting full ratio back to .95 2024-09-20T18:54:57.025 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set-full-ratio .95 2024-09-20T18:54:57.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:57.043 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:57.043 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:57.144 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:54:57.171 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 17 16 39189 39173 143.879 111.125 0.00783835 0.00694419 2024-09-20T18:54:57.179 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:54:57.183 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:57.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:57.184 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:54:57.285 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:54:57.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:57 smithi083 ceph-mon[29421]: 3.17 scrub starts 2024-09-20T18:54:57.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:57 smithi083 ceph-mon[29421]: 3.17 scrub ok 2024-09-20T18:54:57.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:57 smithi083 ceph-mon[29421]: 3.c scrub starts 2024-09-20T18:54:57.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:57 smithi083 ceph-mon[29421]: 3.c scrub ok 2024-09-20T18:54:57.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:57 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:54:57.303 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:57 smithi083 ceph-mon[36260]: 3.17 scrub starts 2024-09-20T18:54:57.303 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:57 smithi083 ceph-mon[36260]: 3.17 scrub ok 2024-09-20T18:54:57.303 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:57 smithi083 ceph-mon[36260]: 3.c scrub starts 2024-09-20T18:54:57.303 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:57 smithi083 ceph-mon[36260]: 3.c scrub ok 2024-09-20T18:54:57.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:57 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:54:57.321 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:57.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:57.322 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:54:57.423 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:54:57.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:57.463 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:57.464 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:54:57.482 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:57 smithi142 ceph-mon[32508]: 3.17 scrub starts 2024-09-20T18:54:57.483 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:57 smithi142 ceph-mon[32508]: 3.17 scrub ok 2024-09-20T18:54:57.483 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:57 smithi142 ceph-mon[32508]: 3.c scrub starts 2024-09-20T18:54:57.483 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:57 smithi142 ceph-mon[32508]: 3.c scrub ok 2024-09-20T18:54:57.483 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:57 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:54:57.565 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:54:57.595 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:57.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:57.596 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:54:57.697 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:54:57.723 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:57.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:57.724 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:54:57.796 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T18:54:57.797 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:54:57.825 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:57.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:57.863 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:57.863 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:57.964 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:58.003 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:58.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:58.004 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:58.105 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:58.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:58.154 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:58.155 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:58.171 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 18 16 41068 41052 142.409 117.438 0.00207153 0.00701205 2024-09-20T18:54:58.256 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:58.298 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:58.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:58.299 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:58.400 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:54:58.417 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:54:58.430 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:58.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:58.431 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:54:58.532 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:58.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:58.577 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:58.577 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:58.665 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:58 smithi142 ceph-mon[32508]: pgmap v238: 148 pgs: 148 active+clean; 2.3 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 158 MiB/s wr, 2.53k op/s 2024-09-20T18:54:58.665 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:58 smithi142 ceph-mon[32508]: 3.7 scrub starts 2024-09-20T18:54:58.665 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:58 smithi142 ceph-mon[32508]: 3.7 scrub ok 2024-09-20T18:54:58.665 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:58 smithi142 ceph-mon[32508]: 4.1 scrub starts 2024-09-20T18:54:58.665 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:58 smithi142 ceph-mon[32508]: 4.1 scrub ok 2024-09-20T18:54:58.666 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:58 smithi142 ceph-mon[32508]: 6.1 scrub starts 2024-09-20T18:54:58.666 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:58 smithi142 ceph-mon[32508]: 6.1 scrub ok 2024-09-20T18:54:58.666 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:58 smithi142 ceph-mon[32508]: 3.13 scrub starts 2024-09-20T18:54:58.666 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:58 smithi142 ceph-mon[32508]: 3.13 scrub ok 2024-09-20T18:54:58.678 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:54:58.682 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:58 smithi083 ceph-mon[29421]: pgmap v238: 148 pgs: 148 active+clean; 2.3 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 158 MiB/s wr, 2.53k op/s 2024-09-20T18:54:58.683 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:58 smithi083 ceph-mon[29421]: 3.7 scrub starts 2024-09-20T18:54:58.683 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:58 smithi083 ceph-mon[29421]: 3.7 scrub ok 2024-09-20T18:54:58.683 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:58 smithi083 ceph-mon[29421]: 4.1 scrub starts 2024-09-20T18:54:58.683 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:58 smithi083 ceph-mon[29421]: 4.1 scrub ok 2024-09-20T18:54:58.683 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:58 smithi083 ceph-mon[29421]: 6.1 scrub starts 2024-09-20T18:54:58.683 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:58 smithi083 ceph-mon[29421]: 6.1 scrub ok 2024-09-20T18:54:58.683 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:58 smithi083 ceph-mon[29421]: 3.13 scrub starts 2024-09-20T18:54:58.683 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:58 smithi083 ceph-mon[29421]: 3.13 scrub ok 2024-09-20T18:54:58.683 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:58 smithi083 ceph-mon[36260]: pgmap v238: 148 pgs: 148 active+clean; 2.3 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 158 MiB/s wr, 2.53k op/s 2024-09-20T18:54:58.683 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:58 smithi083 ceph-mon[36260]: 3.7 scrub starts 2024-09-20T18:54:58.683 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:58 smithi083 ceph-mon[36260]: 3.7 scrub ok 2024-09-20T18:54:58.684 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:58 smithi083 ceph-mon[36260]: 4.1 scrub starts 2024-09-20T18:54:58.684 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:58 smithi083 ceph-mon[36260]: 4.1 scrub ok 2024-09-20T18:54:58.684 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:58 smithi083 ceph-mon[36260]: 6.1 scrub starts 2024-09-20T18:54:58.684 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:58 smithi083 ceph-mon[36260]: 6.1 scrub ok 2024-09-20T18:54:58.684 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:58 smithi083 ceph-mon[36260]: 3.13 scrub starts 2024-09-20T18:54:58.684 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:58 smithi083 ceph-mon[36260]: 3.13 scrub ok 2024-09-20T18:54:58.709 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:58.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:58.710 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:54:58.745 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_blocked_ops 2024-09-20T18:54:58.810 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:54:58.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:58.873 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:58.873 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:54:58.974 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:54:58.990 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T18:54:59.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:59.005 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:59.005 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:54:59.106 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:59.148 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:59.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:59.149 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:59.161 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:54:59.171 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 19 16 42773 42757 140.523 106.562 0.00742106 0.00711161 2024-09-20T18:54:59.250 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:54:59.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:59.291 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:59.291 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:54:59.376 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-20T18:54:59.392 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:54:59.439 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:59.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:59.440 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:54:59.541 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:54:59.578 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:59.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:59.579 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:54:59.587 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:59 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2717556290' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:54:59.587 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:59 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:54:59.587 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:59 smithi142 ceph-mon[32508]: 5.7 scrub starts 2024-09-20T18:54:59.587 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:54:59 smithi142 ceph-mon[32508]: 5.7 scrub ok 2024-09-20T18:54:59.680 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:54:59.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:59.714 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:54:59.714 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:54:59.756 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T18:54:59.756 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:54:59.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:59 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2717556290' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:54:59.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:59 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:54:59.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:59 smithi083 ceph-mon[29421]: 5.7 scrub starts 2024-09-20T18:54:59.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:54:59 smithi083 ceph-mon[29421]: 5.7 scrub ok 2024-09-20T18:54:59.803 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:59 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2717556290' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:54:59.803 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:59 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:54:59.803 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:59 smithi083 ceph-mon[36260]: 5.7 scrub starts 2024-09-20T18:54:59.803 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:54:59 smithi083 ceph-mon[36260]: 5.7 scrub ok 2024-09-20T18:54:59.816 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:59.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:59.858 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:59.858 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:54:59.959 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:54:59.979 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:54:59.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:54:59.980 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:55:00.081 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:00.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:00.117 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:00.118 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:00.171 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:2024-09-20T18:55:00.156499+0000 min lat: 0.00111821 max lat: 0.124999 avg lat: 0.00719271 2024-09-20T18:55:00.171 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-20T18:55:00.171 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 20 16 44511 44495 138.929 108.625 0.00588392 0.00719271 2024-09-20T18:55:00.219 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:00.252 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:00.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:00.253 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:00.354 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:00.390 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:00.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:00.391 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:00.397 INFO:teuthology.orchestra.run.smithi083.stderr:osd set-full-ratio 0.95 2024-09-20T18:55:00.492 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:00.527 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:00.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:00.528 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:00.546 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_historic_ops 2024-09-20T18:55:00.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:00 smithi083 ceph-mon[29421]: pgmap v239: 148 pgs: 148 active+clean; 2.3 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 1.0 KiB/s rd, 129 MiB/s wr, 2.06k op/s 2024-09-20T18:55:00.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:00 smithi083 ceph-mon[29421]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:55:00.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:00 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T18:55:00.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:00 smithi083 ceph-mon[29421]: osdmap e169: 8 total, 8 up, 8 in 2024-09-20T18:55:00.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:00 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3711733181' entity='client.admin' cmd={"prefix": "osd set-full-ratio", "ratio": 0.95} : dispatch 2024-09-20T18:55:00.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:00 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set-full-ratio", "ratio": 0.95} : dispatch 2024-09-20T18:55:00.553 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:00 smithi083 ceph-mon[36260]: pgmap v239: 148 pgs: 148 active+clean; 2.3 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 1.0 KiB/s rd, 129 MiB/s wr, 2.06k op/s 2024-09-20T18:55:00.553 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:00 smithi083 ceph-mon[36260]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:55:00.553 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:00 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T18:55:00.553 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:00 smithi083 ceph-mon[36260]: osdmap e169: 8 total, 8 up, 8 in 2024-09-20T18:55:00.553 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:00 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3711733181' entity='client.admin' cmd={"prefix": "osd set-full-ratio", "ratio": 0.95} : dispatch 2024-09-20T18:55:00.553 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:00 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set-full-ratio", "ratio": 0.95} : dispatch 2024-09-20T18:55:00.629 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:00.660 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:00.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:00.662 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:00.763 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:00.772 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:00 smithi142 ceph-mon[32508]: pgmap v239: 148 pgs: 148 active+clean; 2.3 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 1.0 KiB/s rd, 129 MiB/s wr, 2.06k op/s 2024-09-20T18:55:00.772 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:00 smithi142 ceph-mon[32508]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:55:00.772 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:00 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T18:55:00.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:00 smithi142 ceph-mon[32508]: osdmap e169: 8 total, 8 up, 8 in 2024-09-20T18:55:00.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:00 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3711733181' entity='client.admin' cmd={"prefix": "osd set-full-ratio", "ratio": 0.95} : dispatch 2024-09-20T18:55:00.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:00 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set-full-ratio", "ratio": 0.95} : dispatch 2024-09-20T18:55:00.801 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:00.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:00.802 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:00.903 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:55:00.936 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:00.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:00.937 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:55:00.944 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:55:01.037 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:01.052 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:55:00 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:55:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:55:01.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:01.079 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:01.080 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:01.171 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 21 15 46373 46358 137.858 116.438 0.00913163 0.00724844 2024-09-20T18:55:01.180 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:01.218 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:01.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:01.219 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:01.320 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:01.342 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:01.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:01.343 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:01.444 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:55:01.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:01.480 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:01.480 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:55:01.544 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T18:55:01.544 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:55:01.581 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:01.615 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:01.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:01.616 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:01.717 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:01.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:01.752 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:01.752 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:01.772 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:01 smithi142 ceph-mon[32508]: Health check cleared: OSD_OUT_OF_ORDER_FULL (was: full ratio(s) out of order) 2024-09-20T18:55:01.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:01 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set-full-ratio", "ratio": 0.95}]': finished 2024-09-20T18:55:01.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:01 smithi142 ceph-mon[32508]: osdmap e170: 8 total, 8 up, 8 in 2024-09-20T18:55:01.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:01 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:55:01.802 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:01 smithi083 ceph-mon[29421]: Health check cleared: OSD_OUT_OF_ORDER_FULL (was: full ratio(s) out of order) 2024-09-20T18:55:01.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:01 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set-full-ratio", "ratio": 0.95}]': finished 2024-09-20T18:55:01.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:01 smithi083 ceph-mon[29421]: osdmap e170: 8 total, 8 up, 8 in 2024-09-20T18:55:01.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:01 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:55:01.803 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:01 smithi083 ceph-mon[36260]: Health check cleared: OSD_OUT_OF_ORDER_FULL (was: full ratio(s) out of order) 2024-09-20T18:55:01.803 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:01 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set-full-ratio", "ratio": 0.95}]': finished 2024-09-20T18:55:01.803 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:01 smithi083 ceph-mon[36260]: osdmap e170: 8 total, 8 up, 8 in 2024-09-20T18:55:01.803 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:01 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:55:01.853 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:01.883 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:01.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:01.884 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:01.985 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:02.007 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:02.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:02.008 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:02.109 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:02.145 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:02.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:02.146 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:02.171 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 22 16 48142 48126 136.615 110.5 0.00666296 0.00731531 2024-09-20T18:55:02.247 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:02.290 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:02.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:02.291 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:02.295 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-20T18:55:02.361 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:55:02.391 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:02.424 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:02.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:02.425 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:02.526 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:02.560 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:02.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:02.561 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:02.662 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:55:02.712 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T18:55:02.718 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:55:02.719 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:02.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:02.735 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:55:02.802 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:02 smithi083 ceph-mon[29421]: pgmap v242: 148 pgs: 148 active+clean; 2.5 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 1.3 KiB/s rd, 103 MiB/s wr, 1.64k op/s 2024-09-20T18:55:02.803 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:02 smithi083 ceph-mon[36260]: pgmap v242: 148 pgs: 148 active+clean; 2.5 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 1.3 KiB/s rd, 103 MiB/s wr, 1.64k op/s 2024-09-20T18:55:02.836 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:55:02.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:02.884 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:02.884 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:55:02.985 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:02.989 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:02 smithi142 ceph-mon[32508]: pgmap v242: 148 pgs: 148 active+clean; 2.5 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 1.3 KiB/s rd, 103 MiB/s wr, 1.64k op/s 2024-09-20T18:55:03.023 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:03.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:03.024 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:03.025 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:55:03.025 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:55:03.067 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:55:03.067 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:55:03.096 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:55:03.096 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:55:03.124 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:03.131 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:55:03.131 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:55:03.167 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:55:03.167 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:55:03.167 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:03.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:03.168 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:03.172 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 23 16 49749 49733 135.042 100.438 0.0172437 0.00740012 2024-09-20T18:55:03.175 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:55:03.203 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:55:03.204 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:55:03.235 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:55:03.235 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:55:03.268 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:55:03.268 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:55:03.269 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:03.292 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T18:55:03.292 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-09-20T18:55:03.292 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.292 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.24955.0:49916 8.5 8:a93e4468:::benchmark_data_smithi083_97422_object49915:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e170)", 2024-09-20T18:55:03.292 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:55:03.268437+0000", 2024-09-20T18:55:03.293 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.0072056560000000004, 2024-09-20T18:55:03.293 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0072379039999999999, 2024-09-20T18:55:03.293 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:55:03.293 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:55:03.293 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "waiting for sub ops", 2024-09-20T18:55:03.293 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:55:03.293 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.24955", 2024-09-20T18:55:03.293 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:55:03.293 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 49916 2024-09-20T18:55:03.293 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:03.293 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:55:03.293 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.293 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:55:03.293 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:03.268437+0000", 2024-09-20T18:55:03.294 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:03.294 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:03.294 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.294 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:55:03.294 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:03.268437+0000", 2024-09-20T18:55:03.294 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:03.294 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:03.294 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.294 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:55:03.294 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:03.268439+0000", 2024-09-20T18:55:03.294 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1959999999999998e-06 2024-09-20T18:55:03.294 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:03.295 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.295 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:55:03.295 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:03.268475+0000", 2024-09-20T18:55:03.295 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.5321000000000003e-05 2024-09-20T18:55:03.295 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:03.295 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.295 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:55:03.295 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:03.268488+0000", 2024-09-20T18:55:03.295 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3808000000000001e-05 2024-09-20T18:55:03.295 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:03.295 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.295 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:55:03.295 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:03.268496+0000", 2024-09-20T18:55:03.295 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5299999999999999e-06 2024-09-20T18:55:03.295 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:03.295 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.296 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:55:03.296 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:03.268562+0000", 2024-09-20T18:55:03.296 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.6126999999999999e-05 2024-09-20T18:55:03.296 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:03.296 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.296 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:55:03.296 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:03.268705+0000", 2024-09-20T18:55:03.296 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000143287 2024-09-20T18:55:03.296 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:03.296 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.296 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 4,7", 2024-09-20T18:55:03.296 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:03.268835+0000", 2024-09-20T18:55:03.296 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012951099999999999 2024-09-20T18:55:03.296 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:03.296 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.296 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:55:03.297 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:03.273320+0000", 2024-09-20T18:55:03.297 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0044849240000000004 2024-09-20T18:55:03.297 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:03.297 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.297 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:03.297 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:03.275337+0000", 2024-09-20T18:55:03.297 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0020169250000000001 2024-09-20T18:55:03.297 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:03.297 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.297 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:03.297 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:03.275667+0000", 2024-09-20T18:55:03.297 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00033047300000000002 2024-09-20T18:55:03.297 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:03.297 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:55:03.297 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:03.297 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:03.298 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.298 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.24955.0:49915 8.3 e170/160 8:c034b9f7:::benchmark_data_smithi083_97422_object49914:head v 170'3045, mlcod=170'3043)", 2024-09-20T18:55:03.298 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:55:03.269269+0000", 2024-09-20T18:55:03.298 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.0063739770000000003, 2024-09-20T18:55:03.298 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0064973929999999997, 2024-09-20T18:55:03.298 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:55:03.298 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:55:03.298 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-09-20T18:55:03.298 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:55:03.298 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.298 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:55:03.298 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:03.269269+0000", 2024-09-20T18:55:03.298 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:03.299 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:03.299 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.299 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:55:03.299 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:03.269269+0000", 2024-09-20T18:55:03.299 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:03.299 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:03.299 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.299 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:55:03.299 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:03.269270+0000", 2024-09-20T18:55:03.299 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.9699999999999995e-07 2024-09-20T18:55:03.299 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:03.299 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.299 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:55:03.300 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:03.269300+0000", 2024-09-20T18:55:03.300 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.0833000000000001e-05 2024-09-20T18:55:03.300 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:03.300 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.300 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:55:03.300 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:03.269312+0000", 2024-09-20T18:55:03.300 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1992e-05 2024-09-20T18:55:03.300 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:03.300 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.300 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:55:03.300 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:03.269319+0000", 2024-09-20T18:55:03.300 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.3690000000000003e-06 2024-09-20T18:55:03.300 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:03.300 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.301 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:55:03.301 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:03.269450+0000", 2024-09-20T18:55:03.301 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013101000000000001 2024-09-20T18:55:03.301 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:03.301 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.301 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:55:03.301 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:03.269476+0000", 2024-09-20T18:55:03.301 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.5939e-05 2024-09-20T18:55:03.301 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:03.301 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:55:03.302 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:03.302 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:03.302 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.302 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.24955.0:49920 8.5 8:aa9d8566:::benchmark_data_smithi083_97422_object49919:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e170)", 2024-09-20T18:55:03.302 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:55:03.269489+0000", 2024-09-20T18:55:03.302 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.0061541, 2024-09-20T18:55:03.302 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0063350339999999998, 2024-09-20T18:55:03.302 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:55:03.302 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:55:03.302 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "waiting for sub ops", 2024-09-20T18:55:03.302 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:55:03.302 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.24955", 2024-09-20T18:55:03.302 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:55:03.302 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 49920 2024-09-20T18:55:03.303 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:03.303 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:55:03.303 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.303 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:55:03.303 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:03.269489+0000", 2024-09-20T18:55:03.303 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:03.303 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:03.303 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.303 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:55:03.303 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:03.269489+0000", 2024-09-20T18:55:03.303 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:03.303 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:03.303 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.303 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:55:03.304 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:03.269490+0000", 2024-09-20T18:55:03.304 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2309999999999999e-06 2024-09-20T18:55:03.304 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:03.304 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.304 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:55:03.304 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:03.269546+0000", 2024-09-20T18:55:03.304 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6613999999999997e-05 2024-09-20T18:55:03.304 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:03.304 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.304 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:55:03.304 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:03.269556+0000", 2024-09-20T18:55:03.304 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.4490000000000005e-06 2024-09-20T18:55:03.304 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:03.304 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.304 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:55:03.305 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:03.269563+0000", 2024-09-20T18:55:03.305 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.8159999999999996e-06 2024-09-20T18:55:03.305 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:03.305 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.305 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:55:03.305 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:03.270870+0000", 2024-09-20T18:55:03.305 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001307372 2024-09-20T18:55:03.305 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:03.305 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.305 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:55:03.305 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:03.270978+0000", 2024-09-20T18:55:03.305 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000107941 2024-09-20T18:55:03.305 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:03.305 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.305 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 4,7", 2024-09-20T18:55:03.306 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:03.271117+0000", 2024-09-20T18:55:03.306 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013918200000000001 2024-09-20T18:55:03.306 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:03.306 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.306 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:55:03.306 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:03.273327+0000", 2024-09-20T18:55:03.306 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0022094179999999999 2024-09-20T18:55:03.306 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:03.306 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:55:03.306 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:03.306 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:03.306 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.306 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.24955.0:49928 8.9 8:9a5b2305:::benchmark_data_smithi083_97422_object49927:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e170)", 2024-09-20T18:55:03.307 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:55:03.274967+0000", 2024-09-20T18:55:03.307 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.000675404, 2024-09-20T18:55:03.307 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00093019900000000004, 2024-09-20T18:55:03.307 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:55:03.307 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:55:03.307 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "waiting for sub ops", 2024-09-20T18:55:03.307 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:55:03.307 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.24955", 2024-09-20T18:55:03.307 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:55:03.307 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 49928 2024-09-20T18:55:03.307 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:03.307 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:55:03.307 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.307 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:55:03.307 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:03.274967+0000", 2024-09-20T18:55:03.308 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:03.308 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:03.308 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.308 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:55:03.308 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:03.274967+0000", 2024-09-20T18:55:03.308 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:03.308 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:03.308 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.308 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:55:03.308 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:03.274969+0000", 2024-09-20T18:55:03.308 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0020000000000001e-06 2024-09-20T18:55:03.308 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:03.308 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.308 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:55:03.308 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:03.275054+0000", 2024-09-20T18:55:03.309 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.4456000000000006e-05 2024-09-20T18:55:03.309 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:03.309 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.309 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:55:03.309 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:03.275070+0000", 2024-09-20T18:55:03.309 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6663e-05 2024-09-20T18:55:03.309 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:03.309 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.309 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:55:03.309 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:03.275078+0000", 2024-09-20T18:55:03.309 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.8520000000000004e-06 2024-09-20T18:55:03.309 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:03.309 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.309 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:55:03.310 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:03.275132+0000", 2024-09-20T18:55:03.310 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3470000000000001e-05 2024-09-20T18:55:03.310 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:03.310 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.310 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:55:03.310 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:03.275308+0000", 2024-09-20T18:55:03.310 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00017583400000000001 2024-09-20T18:55:03.310 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:03.310 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.310 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:55:03.310 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:03.275517+0000", 2024-09-20T18:55:03.310 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000208953 2024-09-20T18:55:03.310 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:03.310 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:55:03.311 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:03.311 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:03.311 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.311 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop_reply(client.24955.0:49920 8.5 e170/160 ondisk, result = 0)", 2024-09-20T18:55:03.311 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:55:03.275522+0000", 2024-09-20T18:55:03.311 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.00012057000000000001, 2024-09-20T18:55:03.311 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00042794300000000002, 2024-09-20T18:55:03.311 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:55:03.311 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:55:03.311 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-09-20T18:55:03.311 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:55:03.311 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.311 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:55:03.312 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:03.275522+0000", 2024-09-20T18:55:03.312 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:03.312 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:03.312 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.312 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:55:03.312 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:03.275522+0000", 2024-09-20T18:55:03.312 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:03.312 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:03.312 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.312 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:55:03.312 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:03.275522+0000", 2024-09-20T18:55:03.312 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.7599999999999998e-07 2024-09-20T18:55:03.312 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:03.312 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.312 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:55:03.312 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:03.275524+0000", 2024-09-20T18:55:03.313 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0600000000000002e-06 2024-09-20T18:55:03.313 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:03.313 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.313 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:55:03.313 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:03.275530+0000", 2024-09-20T18:55:03.313 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0549999999999999e-06 2024-09-20T18:55:03.313 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:03.313 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.313 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:55:03.313 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:03.275540+0000", 2024-09-20T18:55:03.313 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0558e-05 2024-09-20T18:55:03.313 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:03.313 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.313 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:55:03.313 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:03.275826+0000", 2024-09-20T18:55:03.314 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00028584699999999999 2024-09-20T18:55:03.314 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:03.314 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:03.314 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:55:03.314 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:03.275831+0000", 2024-09-20T18:55:03.314 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.2009999999999998e-06 2024-09-20T18:55:03.314 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:03.314 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:55:03.314 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:03.314 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:03.314 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-09-20T18:55:03.314 INFO:teuthology.orchestra.run.smithi142.stdout: "num_ops": 5 2024-09-20T18:55:03.314 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T18:55:03.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:03.315 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:03.316 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:03.416 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:03.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:03.447 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:03.447 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:03.548 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:03.584 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:03.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:03.585 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:03.686 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:03.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:03.721 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:03.721 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:03.790 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:03 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1019716116' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:55:03.790 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:03 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1019716116' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:55:03.821 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:03.861 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:03.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:03.863 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:03.963 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:55:03.997 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:03.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:03.998 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:55:04.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:03 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1019716116' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:55:04.099 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:04.130 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:04.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:04.131 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:04.154 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_blocked_ops 2024-09-20T18:55:04.171 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 24 16 51443 51427 133.827 105.875 0.00503857 0.00746738 2024-09-20T18:55:04.233 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:04.267 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:04.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:04.268 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:04.369 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:04.405 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:04.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:04.406 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:04.507 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:04.546 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:55:04.551 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:04.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:04.552 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:04.653 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:04.683 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:04.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:04.684 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:04.785 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:04.802 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:04 smithi083 ceph-mon[29421]: pgmap v243: 148 pgs: 148 active+clean; 2.8 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 143 MiB/s wr, 2.29k op/s 2024-09-20T18:55:04.802 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:04 smithi083 ceph-mon[36260]: pgmap v243: 148 pgs: 148 active+clean; 2.8 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 143 MiB/s wr, 2.29k op/s 2024-09-20T18:55:04.819 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:04.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:04.819 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:04.920 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:04.956 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:04.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:04.957 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:05.018 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:04 smithi142 ceph-mon[32508]: pgmap v243: 148 pgs: 148 active+clean; 2.8 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 143 MiB/s wr, 2.29k op/s 2024-09-20T18:55:05.057 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:55:05.094 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:05.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:05.095 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:55:05.171 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 25 16 53124 53108 132.677 105.062 0.00787282 0.00753243 2024-09-20T18:55:05.177 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T18:55:05.177 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-09-20T18:55:05.177 INFO:teuthology.orchestra.run.smithi142.stdout: "complaint_time": 30, 2024-09-20T18:55:05.177 INFO:teuthology.orchestra.run.smithi142.stdout: "num_blocked_ops": 0 2024-09-20T18:55:05.178 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T18:55:05.196 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:05.230 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:05.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:05.231 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:05.332 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:55:05.370 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:05.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:05.371 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:55:05.471 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:55:05.503 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:05.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:05.504 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:55:05.604 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:05.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:05.637 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:05.637 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:05.691 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-20T18:55:05.738 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:05.772 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:05.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:05.773 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:05.874 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:05.907 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:05.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:05.908 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:05.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:05 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1370192379' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:55:06.009 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:06.033 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:05 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1370192379' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:55:06.033 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:05 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1370192379' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:55:06.043 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_historic_ops 2024-09-20T18:55:06.047 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:06.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:06.047 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:06.148 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:06.171 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 26 16 54827 54811 131.668 106.438 0.0126994 0.00758956 2024-09-20T18:55:06.188 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:06.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:06.189 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:06.250 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T18:55:06.290 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:55:06.336 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:06.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:06.337 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:55:06.437 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:55:06.438 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:06.481 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:06.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:06.482 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:06.584 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:06.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:06.644 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:06.644 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:06.693 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:55:06.745 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:06.788 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:06.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:06.789 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:06.887 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:06 smithi142 ceph-mon[32508]: pgmap v244: 148 pgs: 148 active+clean; 2.8 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 639 B/s rd, 69 MiB/s wr, 1.11k op/s 2024-09-20T18:55:06.888 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:06 smithi142 ceph-mon[32508]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:55:06.888 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:06 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1370192379' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T18:55:06.888 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:06 smithi142 ceph-mon[32508]: osdmap e171: 8 total, 8 up, 8 in 2024-09-20T18:55:06.888 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:55:06 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:55:06.890 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:06.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:06.928 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:06.929 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:06.988 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:06 smithi083 ceph-mon[29421]: pgmap v244: 148 pgs: 148 active+clean; 2.8 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 639 B/s rd, 69 MiB/s wr, 1.11k op/s 2024-09-20T18:55:06.988 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:06 smithi083 ceph-mon[29421]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:55:06.988 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:06 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1370192379' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T18:55:06.988 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:06 smithi083 ceph-mon[29421]: osdmap e171: 8 total, 8 up, 8 in 2024-09-20T18:55:06.988 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:06 smithi083 ceph-mon[36260]: pgmap v244: 148 pgs: 148 active+clean; 2.8 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 639 B/s rd, 69 MiB/s wr, 1.11k op/s 2024-09-20T18:55:06.988 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:06 smithi083 ceph-mon[36260]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:55:06.988 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:06 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1370192379' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T18:55:06.989 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:06 smithi083 ceph-mon[36260]: osdmap e171: 8 total, 8 up, 8 in 2024-09-20T18:55:07.029 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:07.037 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T18:55:07.037 INFO:teuthology.orchestra.run.smithi142.stdout: "size": 20, 2024-09-20T18:55:07.037 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 600, 2024-09-20T18:55:07.038 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-09-20T18:55:07.038 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.038 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:24945 7.7 7:e97e81cf:::benchmark_data_smithi083_70643_object24944:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:55:07.038 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:06.996434+0000", 2024-09-20T18:55:07.038 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 240.02350981800001, 2024-09-20T18:55:07.038 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.117472831, 2024-09-20T18:55:07.038 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:55:07.038 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:55:07.038 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:07.038 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:55:07.038 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:55:07.038 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:07.039 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 24945 2024-09-20T18:55:07.039 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.039 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:55:07.039 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.039 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:55:07.039 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.996434+0000", 2024-09-20T18:55:07.039 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:07.039 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.039 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.039 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:55:07.039 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.996434+0000", 2024-09-20T18:55:07.039 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:07.039 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.039 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.040 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:55:07.040 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.996435+0000", 2024-09-20T18:55:07.040 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4869999999999999e-06 2024-09-20T18:55:07.040 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.040 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.040 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:55:07.040 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.996477+0000", 2024-09-20T18:55:07.040 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.1601999999999999e-05 2024-09-20T18:55:07.040 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.040 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.040 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:55:07.040 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.996488+0000", 2024-09-20T18:55:07.040 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0893999999999999e-05 2024-09-20T18:55:07.040 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.041 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.041 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:55:07.041 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.996495+0000", 2024-09-20T18:55:07.041 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.1720000000000001e-06 2024-09-20T18:55:07.041 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.041 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.041 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:55:07.041 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.996709+0000", 2024-09-20T18:55:07.041 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00021389099999999999 2024-09-20T18:55:07.041 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.041 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.041 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:55:07.041 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.996918+0000", 2024-09-20T18:55:07.041 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00020888100000000001 2024-09-20T18:55:07.042 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.042 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.042 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,3", 2024-09-20T18:55:07.042 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.997122+0000", 2024-09-20T18:55:07.042 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00020468 2024-09-20T18:55:07.042 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.042 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.042 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:55:07.042 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.004028+0000", 2024-09-20T18:55:07.042 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0069058180000000002 2024-09-20T18:55:07.042 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.042 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.042 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:07.042 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.004092+0000", 2024-09-20T18:55:07.043 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4323000000000001e-05 2024-09-20T18:55:07.043 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.043 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.043 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:07.043 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.113828+0000", 2024-09-20T18:55:07.043 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.109735151 2024-09-20T18:55:07.043 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.043 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.043 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:55:07.043 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.113870+0000", 2024-09-20T18:55:07.043 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.2258000000000001e-05 2024-09-20T18:55:07.043 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.043 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.043 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:55:07.044 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.113906+0000", 2024-09-20T18:55:07.044 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6674000000000002e-05 2024-09-20T18:55:07.044 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:07.044 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:55:07.044 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:07.044 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.044 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.044 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:24951 7.7 7:ebe82687:::benchmark_data_smithi083_70643_object24950:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:55:07.044 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:06.997097+0000", 2024-09-20T18:55:07.044 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 240.022846851, 2024-09-20T18:55:07.044 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.12447738799999999, 2024-09-20T18:55:07.044 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:55:07.044 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:55:07.044 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:07.045 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:55:07.045 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:55:07.045 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:07.045 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 24951 2024-09-20T18:55:07.045 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.045 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:55:07.045 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.045 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:55:07.045 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.997097+0000", 2024-09-20T18:55:07.045 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:07.045 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.045 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.045 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:55:07.045 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.997097+0000", 2024-09-20T18:55:07.046 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:07.046 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.046 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.046 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:55:07.046 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.997098+0000", 2024-09-20T18:55:07.046 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.082e-06 2024-09-20T18:55:07.046 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.046 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.046 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:55:07.046 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.997210+0000", 2024-09-20T18:55:07.046 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011240500000000001 2024-09-20T18:55:07.046 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.046 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.047 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:55:07.047 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.997223+0000", 2024-09-20T18:55:07.047 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2609000000000001e-05 2024-09-20T18:55:07.047 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.047 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.047 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:55:07.047 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.997230+0000", 2024-09-20T18:55:07.047 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5700000000000004e-06 2024-09-20T18:55:07.047 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.047 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.047 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:55:07.047 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.000085+0000", 2024-09-20T18:55:07.047 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.002855013 2024-09-20T18:55:07.048 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.048 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.048 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:55:07.048 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.000200+0000", 2024-09-20T18:55:07.048 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011476600000000001 2024-09-20T18:55:07.048 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.048 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.048 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,3", 2024-09-20T18:55:07.048 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.001210+0000", 2024-09-20T18:55:07.048 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0010098850000000001 2024-09-20T18:55:07.048 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.048 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.048 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:55:07.049 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.005107+0000", 2024-09-20T18:55:07.049 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0038972730000000001 2024-09-20T18:55:07.049 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.049 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.049 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:07.049 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.103702+0000", 2024-09-20T18:55:07.049 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.098595123000000007 2024-09-20T18:55:07.049 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.049 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.049 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:07.049 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.121471+0000", 2024-09-20T18:55:07.049 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.017768227000000001 2024-09-20T18:55:07.049 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.050 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.050 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:55:07.050 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.121533+0000", 2024-09-20T18:55:07.050 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2489000000000004e-05 2024-09-20T18:55:07.050 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.050 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.050 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:55:07.050 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.121574+0000", 2024-09-20T18:55:07.050 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0945999999999997e-05 2024-09-20T18:55:07.050 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:07.050 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:55:07.050 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:07.050 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.050 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.051 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:24956 7.7 7:e91ba513:::benchmark_data_smithi083_70643_object24955:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:55:07.051 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:06.999473+0000", 2024-09-20T18:55:07.051 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 240.02047074000001, 2024-09-20T18:55:07.051 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.122248832, 2024-09-20T18:55:07.051 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:55:07.051 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:55:07.051 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:07.051 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:55:07.052 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:55:07.052 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:07.052 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 24956 2024-09-20T18:55:07.052 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.052 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:55:07.052 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.052 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:55:07.052 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.999473+0000", 2024-09-20T18:55:07.052 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:07.052 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.052 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.052 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:55:07.052 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.999473+0000", 2024-09-20T18:55:07.052 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:07.052 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.053 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.053 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:55:07.053 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.999475+0000", 2024-09-20T18:55:07.053 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9750000000000001e-06 2024-09-20T18:55:07.053 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.053 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.053 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:55:07.053 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.999545+0000", 2024-09-20T18:55:07.053 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.0104000000000005e-05 2024-09-20T18:55:07.053 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.053 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.054 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:55:07.054 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.999580+0000", 2024-09-20T18:55:07.054 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.5284000000000003e-05 2024-09-20T18:55:07.054 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.054 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.054 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:55:07.054 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.999590+0000", 2024-09-20T18:55:07.054 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.6739999999999992e-06 2024-09-20T18:55:07.054 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.054 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.054 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:55:07.054 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002183+0000", 2024-09-20T18:55:07.054 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0025934899999999999 2024-09-20T18:55:07.054 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.055 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.055 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:55:07.055 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002933+0000", 2024-09-20T18:55:07.055 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00074923700000000004 2024-09-20T18:55:07.055 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.055 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.055 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,3", 2024-09-20T18:55:07.055 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.003100+0000", 2024-09-20T18:55:07.055 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000167651 2024-09-20T18:55:07.055 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.055 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.055 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:55:07.055 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.005113+0000", 2024-09-20T18:55:07.055 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0020131749999999999 2024-09-20T18:55:07.056 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.056 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.056 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:07.056 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.104767+0000", 2024-09-20T18:55:07.056 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.099653505000000003 2024-09-20T18:55:07.056 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.056 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.056 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:07.056 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.121622+0000", 2024-09-20T18:55:07.056 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.016855098999999998 2024-09-20T18:55:07.056 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.056 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.056 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:55:07.056 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.121681+0000", 2024-09-20T18:55:07.056 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9199e-05 2024-09-20T18:55:07.057 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.057 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.057 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:55:07.057 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.121722+0000", 2024-09-20T18:55:07.057 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0438999999999997e-05 2024-09-20T18:55:07.057 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:07.057 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:55:07.057 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:07.057 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.057 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.057 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:24960 7.c 7:3cbb268e:::benchmark_data_smithi083_70643_object24959:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:55:07.057 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:07.001793+0000", 2024-09-20T18:55:07.057 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 240.01814999199999, 2024-09-20T18:55:07.057 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.117162266, 2024-09-20T18:55:07.058 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:55:07.058 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:55:07.058 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:07.058 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:55:07.058 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:55:07.058 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:07.058 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 24960 2024-09-20T18:55:07.058 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.058 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:55:07.058 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.058 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:55:07.058 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.001793+0000", 2024-09-20T18:55:07.058 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:07.059 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.059 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.059 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:55:07.059 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.001793+0000", 2024-09-20T18:55:07.059 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:07.059 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.059 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.059 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:55:07.059 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.001795+0000", 2024-09-20T18:55:07.059 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3069999999999999e-06 2024-09-20T18:55:07.059 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.059 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.059 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:55:07.059 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.001853+0000", 2024-09-20T18:55:07.060 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8372999999999998e-05 2024-09-20T18:55:07.060 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.060 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.060 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:55:07.060 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.001865+0000", 2024-09-20T18:55:07.060 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2004999999999999e-05 2024-09-20T18:55:07.060 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.060 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.060 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:55:07.060 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.001873+0000", 2024-09-20T18:55:07.060 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.362e-06 2024-09-20T18:55:07.060 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.060 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.060 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:55:07.061 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.001933+0000", 2024-09-20T18:55:07.061 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9998999999999999e-05 2024-09-20T18:55:07.061 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.061 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.061 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:55:07.061 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002069+0000", 2024-09-20T18:55:07.061 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013620799999999999 2024-09-20T18:55:07.061 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.061 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.061 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:55:07.061 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002203+0000", 2024-09-20T18:55:07.061 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000134048 2024-09-20T18:55:07.061 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.061 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.062 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:07.062 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.004679+0000", 2024-09-20T18:55:07.062 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0024765479999999999 2024-09-20T18:55:07.062 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.062 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.062 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:55:07.062 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.005474+0000", 2024-09-20T18:55:07.062 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00079475399999999995 2024-09-20T18:55:07.062 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.062 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.062 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:07.062 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.118886+0000", 2024-09-20T18:55:07.062 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.11341159200000001 2024-09-20T18:55:07.062 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.063 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.063 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:55:07.063 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.118928+0000", 2024-09-20T18:55:07.063 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.2460999999999999e-05 2024-09-20T18:55:07.063 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.063 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.063 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:55:07.063 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.118956+0000", 2024-09-20T18:55:07.063 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.7608999999999999e-05 2024-09-20T18:55:07.063 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:07.063 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:55:07.063 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:07.063 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.064 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.064 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:24961 7.c 7:35ca9a87:::benchmark_data_smithi083_70643_object24960:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:55:07.064 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:07.001951+0000", 2024-09-20T18:55:07.064 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 240.01799278300001, 2024-09-20T18:55:07.064 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.11716043299999999, 2024-09-20T18:55:07.064 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:55:07.064 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:55:07.064 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:07.064 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:55:07.064 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:55:07.064 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:07.064 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 24961 2024-09-20T18:55:07.064 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.064 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:55:07.064 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.065 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:55:07.065 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.001951+0000", 2024-09-20T18:55:07.065 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:07.065 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.065 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.065 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:55:07.065 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.001951+0000", 2024-09-20T18:55:07.065 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:07.065 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.065 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.065 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:55:07.065 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.001951+0000", 2024-09-20T18:55:07.065 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.7499999999999999e-07 2024-09-20T18:55:07.066 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.066 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.066 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:55:07.066 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002093+0000", 2024-09-20T18:55:07.066 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000141433 2024-09-20T18:55:07.066 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.066 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.066 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:55:07.066 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002103+0000", 2024-09-20T18:55:07.066 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.9979999999999998e-06 2024-09-20T18:55:07.066 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.066 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.066 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:55:07.067 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002110+0000", 2024-09-20T18:55:07.067 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.6170000000000003e-06 2024-09-20T18:55:07.067 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.067 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.067 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:55:07.067 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.003810+0000", 2024-09-20T18:55:07.067 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001700221 2024-09-20T18:55:07.067 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.067 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.067 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:55:07.067 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.003895+0000", 2024-09-20T18:55:07.067 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.4886e-05 2024-09-20T18:55:07.067 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.068 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.068 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:55:07.068 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.003978+0000", 2024-09-20T18:55:07.068 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.3812000000000003e-05 2024-09-20T18:55:07.068 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.068 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.068 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:55:07.068 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.005480+0000", 2024-09-20T18:55:07.068 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001501245 2024-09-20T18:55:07.068 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.068 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.068 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:07.068 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.005508+0000", 2024-09-20T18:55:07.068 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.8668000000000001e-05 2024-09-20T18:55:07.069 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.069 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.069 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:07.069 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.119009+0000", 2024-09-20T18:55:07.069 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.113500961 2024-09-20T18:55:07.069 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.069 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.069 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:55:07.069 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.119073+0000", 2024-09-20T18:55:07.069 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.3378000000000002e-05 2024-09-20T18:55:07.069 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.069 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.069 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:55:07.069 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.119111+0000", 2024-09-20T18:55:07.070 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.8439000000000002e-05 2024-09-20T18:55:07.070 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:07.070 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:55:07.070 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:07.070 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.070 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.070 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:24962 7.7 7:efd119ca:::benchmark_data_smithi083_70643_object24961:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:55:07.070 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:07.002480+0000", 2024-09-20T18:55:07.070 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 240.01746309200001, 2024-09-20T18:55:07.070 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.11963515, 2024-09-20T18:55:07.070 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:55:07.070 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:55:07.070 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:07.070 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:55:07.071 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:55:07.071 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:07.071 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 24962 2024-09-20T18:55:07.071 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.071 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:55:07.071 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.071 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:55:07.071 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002480+0000", 2024-09-20T18:55:07.071 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:07.071 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.071 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.071 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:55:07.071 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002480+0000", 2024-09-20T18:55:07.072 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:07.072 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.072 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.072 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:55:07.072 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002481+0000", 2024-09-20T18:55:07.072 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0860000000000001e-06 2024-09-20T18:55:07.072 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.072 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.072 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:55:07.072 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002511+0000", 2024-09-20T18:55:07.072 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9499999999999999e-05 2024-09-20T18:55:07.072 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.072 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.072 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:55:07.073 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002522+0000", 2024-09-20T18:55:07.073 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0652000000000001e-05 2024-09-20T18:55:07.073 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.073 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.073 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:55:07.073 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002529+0000", 2024-09-20T18:55:07.073 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.1319999999999996e-06 2024-09-20T18:55:07.073 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.073 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.073 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:55:07.073 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.004125+0000", 2024-09-20T18:55:07.073 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0015959570000000001 2024-09-20T18:55:07.073 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.073 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.074 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:55:07.074 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.004232+0000", 2024-09-20T18:55:07.074 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000107767 2024-09-20T18:55:07.074 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.074 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.074 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,3", 2024-09-20T18:55:07.074 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.004366+0000", 2024-09-20T18:55:07.074 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013360299999999999 2024-09-20T18:55:07.074 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.074 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.074 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:55:07.074 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.005279+0000", 2024-09-20T18:55:07.074 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00091312100000000003 2024-09-20T18:55:07.074 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.075 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.075 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:07.075 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.104849+0000", 2024-09-20T18:55:07.075 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.099569569999999996 2024-09-20T18:55:07.075 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.075 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.075 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:07.075 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.122006+0000", 2024-09-20T18:55:07.075 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.017157531 2024-09-20T18:55:07.075 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.075 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.075 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:55:07.075 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.122071+0000", 2024-09-20T18:55:07.075 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4372e-05 2024-09-20T18:55:07.076 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.076 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.076 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:55:07.076 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.122116+0000", 2024-09-20T18:55:07.076 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.4858999999999999e-05 2024-09-20T18:55:07.076 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:07.076 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:55:07.076 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:07.076 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.076 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.076 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:24963 7.c 7:38caa154:::benchmark_data_smithi083_70643_object24962:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:55:07.076 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:07.002541+0000", 2024-09-20T18:55:07.076 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 240.017402613, 2024-09-20T18:55:07.076 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.11664609500000001, 2024-09-20T18:55:07.076 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:55:07.077 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:55:07.077 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:07.077 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:55:07.077 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:55:07.077 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:07.077 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 24963 2024-09-20T18:55:07.077 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.077 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:55:07.077 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.077 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:55:07.077 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002541+0000", 2024-09-20T18:55:07.077 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:07.077 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.077 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.077 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:55:07.078 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002541+0000", 2024-09-20T18:55:07.078 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:07.078 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.078 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.078 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:55:07.078 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002541+0000", 2024-09-20T18:55:07.078 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.2900000000000004e-07 2024-09-20T18:55:07.078 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.078 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.078 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:55:07.078 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002569+0000", 2024-09-20T18:55:07.079 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.7866999999999999e-05 2024-09-20T18:55:07.079 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.079 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.079 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:55:07.079 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002576+0000", 2024-09-20T18:55:07.080 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.0929999999999997e-06 2024-09-20T18:55:07.080 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.080 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.080 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:55:07.080 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002582+0000", 2024-09-20T18:55:07.080 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.575e-06 2024-09-20T18:55:07.080 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.080 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.080 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:55:07.080 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.004714+0000", 2024-09-20T18:55:07.080 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0021320430000000001 2024-09-20T18:55:07.080 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.080 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.081 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:55:07.081 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.004828+0000", 2024-09-20T18:55:07.081 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000113632 2024-09-20T18:55:07.081 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.081 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.081 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:55:07.081 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.004947+0000", 2024-09-20T18:55:07.081 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000119367 2024-09-20T18:55:07.081 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.081 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.081 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:55:07.081 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.005749+0000", 2024-09-20T18:55:07.081 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000801898 2024-09-20T18:55:07.081 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.081 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.081 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:07.082 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.005930+0000", 2024-09-20T18:55:07.082 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000180686 2024-09-20T18:55:07.082 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.082 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.082 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:07.082 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.119138+0000", 2024-09-20T18:55:07.082 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.11320831200000001 2024-09-20T18:55:07.082 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.082 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.082 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:55:07.082 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.119166+0000", 2024-09-20T18:55:07.082 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.8121000000000001e-05 2024-09-20T18:55:07.083 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.083 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.083 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:55:07.083 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.119187+0000", 2024-09-20T18:55:07.083 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0971999999999998e-05 2024-09-20T18:55:07.083 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:07.083 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:55:07.083 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:07.083 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.083 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.083 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:25344 7.7 7:ead30b22:::benchmark_data_smithi083_70643_object25343:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:55:07.083 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:07.270133+0000", 2024-09-20T18:55:07.083 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 239.74981046799999, 2024-09-20T18:55:07.083 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.17431155100000001, 2024-09-20T18:55:07.084 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:55:07.084 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:55:07.084 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:07.084 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:55:07.084 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:55:07.084 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:07.084 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 25344 2024-09-20T18:55:07.084 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.084 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:55:07.084 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.084 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:55:07.084 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.270133+0000", 2024-09-20T18:55:07.084 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:07.084 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.084 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.085 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:55:07.085 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.270133+0000", 2024-09-20T18:55:07.085 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:07.085 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.085 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.085 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:55:07.085 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.270134+0000", 2024-09-20T18:55:07.085 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4619999999999999e-06 2024-09-20T18:55:07.085 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.085 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.085 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:55:07.085 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.270200+0000", 2024-09-20T18:55:07.085 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5998000000000001e-05 2024-09-20T18:55:07.086 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.086 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.086 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:55:07.086 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.270214+0000", 2024-09-20T18:55:07.086 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3631e-05 2024-09-20T18:55:07.086 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.086 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.086 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:55:07.086 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.270222+0000", 2024-09-20T18:55:07.086 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.6149999999999999e-06 2024-09-20T18:55:07.086 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.086 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.086 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:55:07.087 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.270277+0000", 2024-09-20T18:55:07.087 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4981000000000001e-05 2024-09-20T18:55:07.087 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.087 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.087 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:55:07.087 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.270400+0000", 2024-09-20T18:55:07.087 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012379000000000001 2024-09-20T18:55:07.087 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.087 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.087 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,3", 2024-09-20T18:55:07.087 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.270534+0000", 2024-09-20T18:55:07.087 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013307600000000001 2024-09-20T18:55:07.087 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.088 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.088 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:55:07.088 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.271704+0000", 2024-09-20T18:55:07.088 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001170152 2024-09-20T18:55:07.088 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.088 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.088 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:07.088 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.442076+0000", 2024-09-20T18:55:07.088 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.170371932 2024-09-20T18:55:07.088 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.088 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.088 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:07.088 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.444327+0000", 2024-09-20T18:55:07.088 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.002251454 2024-09-20T18:55:07.088 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.089 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.089 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:55:07.089 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.444386+0000", 2024-09-20T18:55:07.089 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8579999999999998e-05 2024-09-20T18:55:07.089 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.089 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.089 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:55:07.089 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.444445+0000", 2024-09-20T18:55:07.089 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8879999999999999e-05 2024-09-20T18:55:07.089 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:07.089 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:55:07.089 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:07.089 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.089 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.090 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:26469 7.7 7:edf96f0c:::benchmark_data_smithi083_70643_object26468:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:55:07.090 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:07.996717+0000", 2024-09-20T18:55:07.090 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 239.02322694200001, 2024-09-20T18:55:07.090 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13515669799999999, 2024-09-20T18:55:07.090 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:55:07.090 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:55:07.090 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:07.090 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:55:07.090 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:55:07.090 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:07.090 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 26469 2024-09-20T18:55:07.090 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.090 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:55:07.090 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.091 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:55:07.091 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.996717+0000", 2024-09-20T18:55:07.091 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:07.091 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.091 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.091 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:55:07.091 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.996717+0000", 2024-09-20T18:55:07.091 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:07.091 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.091 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.091 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:55:07.091 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.996718+0000", 2024-09-20T18:55:07.091 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6020000000000001e-06 2024-09-20T18:55:07.091 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.091 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.092 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:55:07.092 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.996814+0000", 2024-09-20T18:55:07.092 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.5480000000000001e-05 2024-09-20T18:55:07.092 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.092 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.092 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:55:07.092 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.996827+0000", 2024-09-20T18:55:07.092 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3749e-05 2024-09-20T18:55:07.092 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.092 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.092 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:55:07.092 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.996835+0000", 2024-09-20T18:55:07.092 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.2309999999999999e-06 2024-09-20T18:55:07.092 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.092 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.092 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:55:07.093 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.996897+0000", 2024-09-20T18:55:07.093 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2515999999999996e-05 2024-09-20T18:55:07.093 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.093 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.093 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:55:07.093 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.996992+0000", 2024-09-20T18:55:07.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:07.094 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:07.094 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:07.095 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.4870000000000005e-05 2024-09-20T18:55:07.095 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.095 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.095 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,3", 2024-09-20T18:55:07.095 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.997083+0000", 2024-09-20T18:55:07.095 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.1144999999999996e-05 2024-09-20T18:55:07.095 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.095 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.095 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:55:07.095 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.998108+0000", 2024-09-20T18:55:07.095 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0010251679999999999 2024-09-20T18:55:07.096 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.096 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.096 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:07.096 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.998717+0000", 2024-09-20T18:55:07.096 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00060899999999999995 2024-09-20T18:55:07.096 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.096 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.096 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:07.096 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.131755+0000", 2024-09-20T18:55:07.096 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13303793799999999 2024-09-20T18:55:07.096 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.096 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.096 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:55:07.096 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.131820+0000", 2024-09-20T18:55:07.096 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4678999999999995e-05 2024-09-20T18:55:07.097 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.097 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.097 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:55:07.097 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.131873+0000", 2024-09-20T18:55:07.097 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3319999999999998e-05 2024-09-20T18:55:07.097 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:07.097 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:55:07.097 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:07.097 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.097 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.097 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:26472 7.c 7:326250cc:::benchmark_data_smithi083_70643_object26471:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:55:07.097 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:07.999252+0000", 2024-09-20T18:55:07.097 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 239.0206914, 2024-09-20T18:55:07.098 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.133437312, 2024-09-20T18:55:07.098 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:55:07.098 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:55:07.098 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:07.098 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:55:07.098 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:55:07.098 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:07.098 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 26472 2024-09-20T18:55:07.098 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.098 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:55:07.098 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.098 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:55:07.098 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.999252+0000", 2024-09-20T18:55:07.098 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:07.099 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.099 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.099 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:55:07.099 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.999252+0000", 2024-09-20T18:55:07.099 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:07.099 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.099 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.099 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:55:07.099 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.999253+0000", 2024-09-20T18:55:07.099 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.44e-07 2024-09-20T18:55:07.099 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.099 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.099 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:55:07.099 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.999278+0000", 2024-09-20T18:55:07.100 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.5661e-05 2024-09-20T18:55:07.100 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.100 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.100 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:55:07.100 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.999287+0000", 2024-09-20T18:55:07.100 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.2600000000000005e-06 2024-09-20T18:55:07.100 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.100 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.100 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:55:07.100 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.999293+0000", 2024-09-20T18:55:07.100 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4609999999999996e-06 2024-09-20T18:55:07.100 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.100 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.100 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:55:07.101 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.999366+0000", 2024-09-20T18:55:07.101 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.2695999999999997e-05 2024-09-20T18:55:07.101 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.101 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.101 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:55:07.101 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.999475+0000", 2024-09-20T18:55:07.101 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000109472 2024-09-20T18:55:07.101 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.101 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.101 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:55:07.101 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.000511+0000", 2024-09-20T18:55:07.101 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0010355 2024-09-20T18:55:07.101 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.101 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.102 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:55:07.102 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.002097+0000", 2024-09-20T18:55:07.102 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0015861429999999999 2024-09-20T18:55:07.102 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.102 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.102 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:07.102 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.002781+0000", 2024-09-20T18:55:07.102 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000684039 2024-09-20T18:55:07.102 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.102 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.102 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:07.102 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.132602+0000", 2024-09-20T18:55:07.102 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.129820825 2024-09-20T18:55:07.102 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.103 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.103 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:55:07.103 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.132652+0000", 2024-09-20T18:55:07.103 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0139999999999998e-05 2024-09-20T18:55:07.103 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.103 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.103 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:55:07.103 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.132689+0000", 2024-09-20T18:55:07.103 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.7471000000000001e-05 2024-09-20T18:55:07.103 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:07.103 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:55:07.103 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:07.103 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.103 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.104 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:26478 7.d 7:b12f12bc:::benchmark_data_smithi083_70643_object26477:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:55:07.104 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:08.003662+0000", 2024-09-20T18:55:07.104 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 239.01628153600001, 2024-09-20T18:55:07.104 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.15004061199999999, 2024-09-20T18:55:07.104 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:55:07.104 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:55:07.104 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:07.104 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:55:07.104 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:55:07.104 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:07.104 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 26478 2024-09-20T18:55:07.104 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.104 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:55:07.104 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.105 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:55:07.105 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003662+0000", 2024-09-20T18:55:07.105 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:07.105 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.105 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.105 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:55:07.105 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003662+0000", 2024-09-20T18:55:07.105 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:07.105 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.105 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.105 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:55:07.105 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003664+0000", 2024-09-20T18:55:07.105 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8959999999999999e-06 2024-09-20T18:55:07.105 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.105 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.106 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:55:07.106 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003707+0000", 2024-09-20T18:55:07.106 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3288999999999998e-05 2024-09-20T18:55:07.106 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.106 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.106 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:55:07.106 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003724+0000", 2024-09-20T18:55:07.106 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6529e-05 2024-09-20T18:55:07.106 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.106 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.106 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:55:07.106 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003732+0000", 2024-09-20T18:55:07.106 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.7679999999999997e-06 2024-09-20T18:55:07.107 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.107 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.107 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:55:07.107 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003800+0000", 2024-09-20T18:55:07.107 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.7904999999999994e-05 2024-09-20T18:55:07.107 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.107 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.107 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:55:07.107 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003941+0000", 2024-09-20T18:55:07.107 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000140731 2024-09-20T18:55:07.107 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.107 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.107 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,1", 2024-09-20T18:55:07.107 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.004096+0000", 2024-09-20T18:55:07.108 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015452700000000001 2024-09-20T18:55:07.108 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.108 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.108 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:55:07.108 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.005860+0000", 2024-09-20T18:55:07.108 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001764383 2024-09-20T18:55:07.108 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.108 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.108 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:07.108 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.005912+0000", 2024-09-20T18:55:07.108 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.2519000000000003e-05 2024-09-20T18:55:07.108 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.108 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.108 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:07.108 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.153622+0000", 2024-09-20T18:55:07.108 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.147709641 2024-09-20T18:55:07.109 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.109 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.109 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:55:07.109 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.153662+0000", 2024-09-20T18:55:07.109 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9966000000000003e-05 2024-09-20T18:55:07.109 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.109 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.109 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:55:07.109 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.153703+0000", 2024-09-20T18:55:07.109 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0457999999999998e-05 2024-09-20T18:55:07.109 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:07.109 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:55:07.109 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:07.109 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.109 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.110 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:26479 7.d 7:bdad06c1:::benchmark_data_smithi083_70643_object26478:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:55:07.110 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:08.003826+0000", 2024-09-20T18:55:07.110 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 239.016117374, 2024-09-20T18:55:07.110 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.15140265899999999, 2024-09-20T18:55:07.110 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:55:07.110 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:55:07.110 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:07.110 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:55:07.110 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:55:07.110 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:07.110 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 26479 2024-09-20T18:55:07.110 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.110 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:55:07.110 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.111 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:55:07.111 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003826+0000", 2024-09-20T18:55:07.111 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:07.111 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.111 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.111 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:55:07.111 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003826+0000", 2024-09-20T18:55:07.111 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:07.111 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.111 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.111 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:55:07.111 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003827+0000", 2024-09-20T18:55:07.111 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.8100000000000002e-07 2024-09-20T18:55:07.112 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.112 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.112 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:55:07.112 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003876+0000", 2024-09-20T18:55:07.112 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8888999999999999e-05 2024-09-20T18:55:07.112 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.112 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.112 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:55:07.112 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003883+0000", 2024-09-20T18:55:07.112 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.8909999999999998e-06 2024-09-20T18:55:07.112 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.112 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.112 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:55:07.112 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003887+0000", 2024-09-20T18:55:07.113 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.5909999999999998e-06 2024-09-20T18:55:07.113 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.113 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.113 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:55:07.113 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.005030+0000", 2024-09-20T18:55:07.113 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0011424289999999999 2024-09-20T18:55:07.113 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.113 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.113 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:55:07.113 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.005164+0000", 2024-09-20T18:55:07.113 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000134537 2024-09-20T18:55:07.113 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.113 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.113 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,1", 2024-09-20T18:55:07.114 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.005242+0000", 2024-09-20T18:55:07.114 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.7956999999999999e-05 2024-09-20T18:55:07.114 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.114 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.114 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:07.114 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.006119+0000", 2024-09-20T18:55:07.114 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00087690700000000001 2024-09-20T18:55:07.114 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.114 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.114 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:55:07.114 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.006138+0000", 2024-09-20T18:55:07.114 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8450000000000001e-05 2024-09-20T18:55:07.114 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.115 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.115 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:07.115 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.155098+0000", 2024-09-20T18:55:07.115 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.14896046900000001 2024-09-20T18:55:07.115 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.115 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.115 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:55:07.115 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.155153+0000", 2024-09-20T18:55:07.115 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5058e-05 2024-09-20T18:55:07.115 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.115 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.115 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:55:07.115 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.155229+0000", 2024-09-20T18:55:07.116 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5699999999999997e-05 2024-09-20T18:55:07.116 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:07.116 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:55:07.116 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:07.116 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.116 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.116 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:26481 7.7 7:eecb08c1:::benchmark_data_smithi083_70643_object26480:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:55:07.116 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:08.006431+0000", 2024-09-20T18:55:07.116 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 239.013512717, 2024-09-20T18:55:07.116 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.14993074200000001, 2024-09-20T18:55:07.116 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:55:07.116 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:55:07.116 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:07.117 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:55:07.117 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:55:07.117 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:07.117 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 26481 2024-09-20T18:55:07.117 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.117 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:55:07.117 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.117 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:55:07.117 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.006431+0000", 2024-09-20T18:55:07.117 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:07.117 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.117 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.117 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:55:07.118 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.006431+0000", 2024-09-20T18:55:07.118 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:07.118 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.118 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.118 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:55:07.118 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.006432+0000", 2024-09-20T18:55:07.118 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.291e-06 2024-09-20T18:55:07.118 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.118 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.118 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:55:07.118 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.006487+0000", 2024-09-20T18:55:07.118 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5058e-05 2024-09-20T18:55:07.118 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.119 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.119 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:55:07.119 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.006497+0000", 2024-09-20T18:55:07.119 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.8579999999999995e-06 2024-09-20T18:55:07.119 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.119 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.119 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:55:07.119 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.006502+0000", 2024-09-20T18:55:07.119 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7140000000000001e-06 2024-09-20T18:55:07.119 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.119 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.119 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:55:07.119 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.006544+0000", 2024-09-20T18:55:07.119 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.2518000000000002e-05 2024-09-20T18:55:07.120 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.120 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.120 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:55:07.120 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.006636+0000", 2024-09-20T18:55:07.120 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.1865000000000002e-05 2024-09-20T18:55:07.120 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.120 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.120 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,3", 2024-09-20T18:55:07.120 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.006753+0000", 2024-09-20T18:55:07.120 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011691100000000001 2024-09-20T18:55:07.120 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.120 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.120 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:55:07.120 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.007675+0000", 2024-09-20T18:55:07.121 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00092171199999999999 2024-09-20T18:55:07.121 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.121 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.121 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:07.121 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.132841+0000", 2024-09-20T18:55:07.121 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.125165949 2024-09-20T18:55:07.121 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.121 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.121 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:07.121 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.156244+0000", 2024-09-20T18:55:07.121 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.023402896999999999 2024-09-20T18:55:07.121 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.121 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.121 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:55:07.121 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.156321+0000", 2024-09-20T18:55:07.122 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.7451000000000001e-05 2024-09-20T18:55:07.122 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.122 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.122 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:55:07.122 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.156362+0000", 2024-09-20T18:55:07.122 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0518000000000001e-05 2024-09-20T18:55:07.122 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:07.122 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:55:07.122 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:07.122 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.122 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.122 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:120182 7.d 7:b9824c9e:::benchmark_data_smithi083_70643_object120181:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T18:55:07.122 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:24.962568+0000", 2024-09-20T18:55:07.123 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 162.05737544499999, 2024-09-20T18:55:07.123 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.116782992, 2024-09-20T18:55:07.123 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:55:07.123 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:55:07.123 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:07.123 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:55:07.123 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:55:07.123 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:07.123 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 120182 2024-09-20T18:55:07.123 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.123 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:55:07.123 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.123 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:55:07.123 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962568+0000", 2024-09-20T18:55:07.123 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:07.124 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.124 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.124 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:55:07.124 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962568+0000", 2024-09-20T18:55:07.124 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:07.124 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.124 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.124 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:55:07.124 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962569+0000", 2024-09-20T18:55:07.124 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4160000000000001e-06 2024-09-20T18:55:07.124 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.124 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.124 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:55:07.124 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962623+0000", 2024-09-20T18:55:07.124 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4024000000000003e-05 2024-09-20T18:55:07.124 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.125 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.125 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:55:07.125 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962636+0000", 2024-09-20T18:55:07.125 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2149e-05 2024-09-20T18:55:07.125 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.125 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.125 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:55:07.125 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962643+0000", 2024-09-20T18:55:07.125 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.554e-06 2024-09-20T18:55:07.125 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.125 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.125 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:55:07.125 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962684+0000", 2024-09-20T18:55:07.125 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.1139e-05 2024-09-20T18:55:07.125 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.126 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.126 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:55:07.126 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962772+0000", 2024-09-20T18:55:07.126 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.7750999999999994e-05 2024-09-20T18:55:07.126 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.126 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.126 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,1", 2024-09-20T18:55:07.126 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962859+0000", 2024-09-20T18:55:07.126 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.6434999999999995e-05 2024-09-20T18:55:07.126 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.126 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.126 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:55:07.126 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.966204+0000", 2024-09-20T18:55:07.126 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0033459029999999999 2024-09-20T18:55:07.126 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.127 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.127 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:07.127 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.970405+0000", 2024-09-20T18:55:07.127 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0042008729999999999 2024-09-20T18:55:07.127 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.127 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.127 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:07.127 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.079209+0000", 2024-09-20T18:55:07.127 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10880400899999999 2024-09-20T18:55:07.127 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.127 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.127 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:55:07.127 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.079303+0000", 2024-09-20T18:55:07.127 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.4022000000000002e-05 2024-09-20T18:55:07.128 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.128 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.128 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:55:07.128 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.079351+0000", 2024-09-20T18:55:07.128 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7716999999999997e-05 2024-09-20T18:55:07.128 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:07.128 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:55:07.128 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:07.128 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.128 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.128 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:120184 7.c 7:3c34b9f0:::benchmark_data_smithi083_70643_object120183:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T18:55:07.128 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:24.962667+0000", 2024-09-20T18:55:07.128 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 162.057276621, 2024-09-20T18:55:07.128 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.113312679, 2024-09-20T18:55:07.129 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:55:07.129 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:55:07.129 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:07.129 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:55:07.129 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:55:07.129 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:07.129 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 120184 2024-09-20T18:55:07.129 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.129 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:55:07.129 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.129 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:55:07.129 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962667+0000", 2024-09-20T18:55:07.130 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:07.130 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.130 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.130 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:55:07.130 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962667+0000", 2024-09-20T18:55:07.130 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:07.130 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.130 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.130 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:55:07.130 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962667+0000", 2024-09-20T18:55:07.130 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8400000000000004e-07 2024-09-20T18:55:07.130 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.130 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.130 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:55:07.131 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962702+0000", 2024-09-20T18:55:07.131 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4734000000000003e-05 2024-09-20T18:55:07.131 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.131 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.131 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:55:07.131 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962710+0000", 2024-09-20T18:55:07.131 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.0900000000000005e-06 2024-09-20T18:55:07.131 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.131 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.131 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:55:07.131 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962716+0000", 2024-09-20T18:55:07.131 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.1820000000000003e-06 2024-09-20T18:55:07.131 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.131 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.131 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:55:07.132 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962771+0000", 2024-09-20T18:55:07.132 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4447000000000002e-05 2024-09-20T18:55:07.132 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.132 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.132 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:55:07.132 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962883+0000", 2024-09-20T18:55:07.132 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000111996 2024-09-20T18:55:07.132 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.132 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.132 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:55:07.132 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.963021+0000", 2024-09-20T18:55:07.132 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013835799999999999 2024-09-20T18:55:07.132 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.132 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.133 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:07.133 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.966241+0000", 2024-09-20T18:55:07.133 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0032201299999999999 2024-09-20T18:55:07.133 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.133 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.133 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:55:07.133 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.966257+0000", 2024-09-20T18:55:07.133 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5325e-05 2024-09-20T18:55:07.133 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.133 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.133 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:07.133 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.075873+0000", 2024-09-20T18:55:07.133 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.109616093 2024-09-20T18:55:07.133 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.134 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.134 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:55:07.134 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.075932+0000", 2024-09-20T18:55:07.134 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9487999999999997e-05 2024-09-20T18:55:07.134 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.134 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.134 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:55:07.134 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.075980+0000", 2024-09-20T18:55:07.134 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7252000000000001e-05 2024-09-20T18:55:07.134 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:07.134 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:55:07.134 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:07.134 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.134 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.135 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.14808.0:120191 7.b e121/83 7:d45eee6a:::benchmark_data_smithi083_70643_object120190:head v 121'7491, mlcod=121'7489)", 2024-09-20T18:55:07.135 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:24.971948+0000", 2024-09-20T18:55:07.135 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 162.04799565299999, 2024-09-20T18:55:07.135 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.111765014, 2024-09-20T18:55:07.135 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:55:07.135 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:55:07.135 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:07.135 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:55:07.135 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.135 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:55:07.135 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.971948+0000", 2024-09-20T18:55:07.135 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:07.135 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.135 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.136 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:55:07.136 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.971948+0000", 2024-09-20T18:55:07.136 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:07.136 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.136 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.136 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:55:07.136 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.971949+0000", 2024-09-20T18:55:07.136 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.7400000000000002e-07 2024-09-20T18:55:07.136 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.136 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.136 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:55:07.136 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.971990+0000", 2024-09-20T18:55:07.136 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.1825e-05 2024-09-20T18:55:07.136 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.137 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.137 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:55:07.137 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.073923+0000", 2024-09-20T18:55:07.137 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.101932732 2024-09-20T18:55:07.137 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.137 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.137 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:55:07.137 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.073931+0000", 2024-09-20T18:55:07.137 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.1510000000000004e-06 2024-09-20T18:55:07.137 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.137 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.137 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:55:07.137 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.073984+0000", 2024-09-20T18:55:07.137 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.2679000000000002e-05 2024-09-20T18:55:07.138 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.138 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.138 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:55:07.138 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.074004+0000", 2024-09-20T18:55:07.138 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9882e-05 2024-09-20T18:55:07.138 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.138 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.138 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:55:07.138 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.083676+0000", 2024-09-20T18:55:07.138 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0096721900000000006 2024-09-20T18:55:07.138 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.138 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.138 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:55:07.138 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.083713+0000", 2024-09-20T18:55:07.139 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6780999999999999e-05 2024-09-20T18:55:07.139 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:07.139 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:55:07.139 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:07.139 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.139 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.139 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:120194 7.d 7:b6298d4c:::benchmark_data_smithi083_70643_object120193:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T18:55:07.139 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:24.977580+0000", 2024-09-20T18:55:07.139 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 162.04236383400001, 2024-09-20T18:55:07.139 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.111515778, 2024-09-20T18:55:07.139 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:55:07.139 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:55:07.139 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:07.140 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:55:07.140 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:55:07.140 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:07.140 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 120194 2024-09-20T18:55:07.140 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.140 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:55:07.140 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.140 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:55:07.140 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.977580+0000", 2024-09-20T18:55:07.140 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:07.140 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.140 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.140 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:55:07.140 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.977580+0000", 2024-09-20T18:55:07.140 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:07.141 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.141 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.141 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:55:07.141 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.977582+0000", 2024-09-20T18:55:07.141 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9520000000000001e-06 2024-09-20T18:55:07.141 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.141 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.141 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:55:07.141 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.977642+0000", 2024-09-20T18:55:07.141 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0204000000000002e-05 2024-09-20T18:55:07.141 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.141 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.141 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:55:07.141 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.072252+0000", 2024-09-20T18:55:07.141 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.094610194999999994 2024-09-20T18:55:07.142 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.142 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.142 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:55:07.142 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.072287+0000", 2024-09-20T18:55:07.142 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4542999999999998e-05 2024-09-20T18:55:07.142 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.142 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.142 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:55:07.142 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.072751+0000", 2024-09-20T18:55:07.142 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000464289 2024-09-20T18:55:07.142 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.142 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.142 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:55:07.143 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.072883+0000", 2024-09-20T18:55:07.143 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013254099999999999 2024-09-20T18:55:07.143 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.143 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.143 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,1", 2024-09-20T18:55:07.143 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.073544+0000", 2024-09-20T18:55:07.143 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00066074700000000005 2024-09-20T18:55:07.143 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.143 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.143 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:07.143 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.088934+0000", 2024-09-20T18:55:07.143 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.015389551 2024-09-20T18:55:07.143 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.143 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.144 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:07.144 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.088971+0000", 2024-09-20T18:55:07.144 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.7249000000000002e-05 2024-09-20T18:55:07.144 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.144 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.144 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:55:07.144 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.088990+0000", 2024-09-20T18:55:07.144 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8770000000000002e-05 2024-09-20T18:55:07.144 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.144 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.144 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:55:07.144 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.089045+0000", 2024-09-20T18:55:07.144 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5386999999999997e-05 2024-09-20T18:55:07.144 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.145 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.145 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:55:07.145 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.089095+0000", 2024-09-20T18:55:07.145 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0349999999999997e-05 2024-09-20T18:55:07.145 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:07.145 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:55:07.145 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:07.145 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.145 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.145 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:139218 7.d 7:bef6f051:::benchmark_data_smithi083_70643_object16416:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-20T18:55:07.145 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:42.802191+0000", 2024-09-20T18:55:07.145 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 144.21775263800001, 2024-09-20T18:55:07.145 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.12718874399999999, 2024-09-20T18:55:07.145 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:55:07.145 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:55:07.145 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:07.146 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:55:07.146 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:55:07.146 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:07.146 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 139218 2024-09-20T18:55:07.146 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.146 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:55:07.146 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.146 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:55:07.146 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.802191+0000", 2024-09-20T18:55:07.146 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:07.146 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.146 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.146 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:55:07.146 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.802191+0000", 2024-09-20T18:55:07.146 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:07.147 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.147 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.147 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:55:07.147 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.802192+0000", 2024-09-20T18:55:07.147 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0759999999999999e-06 2024-09-20T18:55:07.147 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.147 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.147 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:55:07.147 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.802196+0000", 2024-09-20T18:55:07.147 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.7309999999999999e-06 2024-09-20T18:55:07.147 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.147 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.147 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:55:07.147 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.802203+0000", 2024-09-20T18:55:07.148 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.4599999999999997e-06 2024-09-20T18:55:07.148 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.148 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.148 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:55:07.148 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.802209+0000", 2024-09-20T18:55:07.148 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5550000000000002e-06 2024-09-20T18:55:07.148 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.148 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.148 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:55:07.148 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.802250+0000", 2024-09-20T18:55:07.148 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.1502999999999999e-05 2024-09-20T18:55:07.148 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.148 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.149 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:55:07.149 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.802322+0000", 2024-09-20T18:55:07.149 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.1723000000000004e-05 2024-09-20T18:55:07.149 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.149 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.149 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,1", 2024-09-20T18:55:07.149 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.802390+0000", 2024-09-20T18:55:07.150 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.8120000000000005e-05 2024-09-20T18:55:07.150 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.150 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.150 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:07.150 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.807367+0000", 2024-09-20T18:55:07.150 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0049773769999999998 2024-09-20T18:55:07.150 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.150 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.150 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:55:07.150 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.807382+0000", 2024-09-20T18:55:07.150 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4667e-05 2024-09-20T18:55:07.150 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.150 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.150 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:07.151 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.929326+0000", 2024-09-20T18:55:07.151 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.121944356 2024-09-20T18:55:07.151 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.151 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.151 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:55:07.151 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.929356+0000", 2024-09-20T18:55:07.151 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9995e-05 2024-09-20T18:55:07.151 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.151 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.151 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:55:07.151 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.929380+0000", 2024-09-20T18:55:07.151 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3181e-05 2024-09-20T18:55:07.151 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:07.151 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:55:07.152 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:07.152 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.152 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.152 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:139220 7.d 7:b6a17747:::benchmark_data_smithi083_70643_object16418:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-20T18:55:07.152 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:42.803402+0000", 2024-09-20T18:55:07.152 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 144.216541447, 2024-09-20T18:55:07.152 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.129377032, 2024-09-20T18:55:07.152 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:55:07.152 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:55:07.152 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:07.152 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:55:07.152 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:55:07.152 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:07.152 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 139220 2024-09-20T18:55:07.153 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.153 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:55:07.153 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.153 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:55:07.153 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.803402+0000", 2024-09-20T18:55:07.153 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:07.153 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.153 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.153 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:55:07.153 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.803402+0000", 2024-09-20T18:55:07.153 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:07.153 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.153 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.153 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:55:07.154 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.803403+0000", 2024-09-20T18:55:07.154 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.097e-06 2024-09-20T18:55:07.154 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.154 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.154 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:55:07.154 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.803408+0000", 2024-09-20T18:55:07.154 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.561e-06 2024-09-20T18:55:07.154 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.154 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.154 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:55:07.154 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.803419+0000", 2024-09-20T18:55:07.154 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1591e-05 2024-09-20T18:55:07.154 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.155 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.155 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:55:07.155 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.803426+0000", 2024-09-20T18:55:07.155 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.9160000000000004e-06 2024-09-20T18:55:07.155 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.155 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.155 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:55:07.155 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.805298+0000", 2024-09-20T18:55:07.155 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0018719780000000001 2024-09-20T18:55:07.155 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.155 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.155 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:55:07.155 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.805418+0000", 2024-09-20T18:55:07.155 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000119739 2024-09-20T18:55:07.156 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.156 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.156 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,1", 2024-09-20T18:55:07.156 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.805543+0000", 2024-09-20T18:55:07.156 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012481699999999999 2024-09-20T18:55:07.156 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.156 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.156 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:55:07.156 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.807413+0000", 2024-09-20T18:55:07.156 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001870678 2024-09-20T18:55:07.156 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.156 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.156 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:07.156 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.808024+0000", 2024-09-20T18:55:07.156 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00061090700000000005 2024-09-20T18:55:07.156 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.157 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.157 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:07.157 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.932673+0000", 2024-09-20T18:55:07.157 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.12464900299999999 2024-09-20T18:55:07.157 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.157 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.157 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:55:07.157 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.932739+0000", 2024-09-20T18:55:07.157 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5979e-05 2024-09-20T18:55:07.157 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.157 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.157 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:55:07.157 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.932779+0000", 2024-09-20T18:55:07.157 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9765999999999998e-05 2024-09-20T18:55:07.158 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:07.158 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:55:07.158 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:07.158 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.158 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.158 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:139224 7.7 7:e3ad72f8:::benchmark_data_smithi083_70643_object16422:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-20T18:55:07.158 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:42.806758+0000", 2024-09-20T18:55:07.158 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 144.21318544100001, 2024-09-20T18:55:07.158 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.121962579, 2024-09-20T18:55:07.158 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:55:07.158 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:55:07.158 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:07.158 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:55:07.159 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:55:07.159 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:07.159 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 139224 2024-09-20T18:55:07.159 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.159 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:55:07.159 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.159 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:55:07.159 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806758+0000", 2024-09-20T18:55:07.159 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:07.159 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.159 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.159 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:55:07.159 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806758+0000", 2024-09-20T18:55:07.159 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:07.159 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.160 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.160 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:55:07.160 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806759+0000", 2024-09-20T18:55:07.160 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.065e-06 2024-09-20T18:55:07.160 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.160 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.160 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:55:07.160 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806764+0000", 2024-09-20T18:55:07.160 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8010000000000003e-06 2024-09-20T18:55:07.160 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.160 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.160 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:55:07.160 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806777+0000", 2024-09-20T18:55:07.160 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3292000000000001e-05 2024-09-20T18:55:07.160 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.161 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.161 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:55:07.161 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806785+0000", 2024-09-20T18:55:07.161 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.9959999999999993e-06 2024-09-20T18:55:07.161 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.161 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.161 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:55:07.161 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806840+0000", 2024-09-20T18:55:07.161 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4308000000000002e-05 2024-09-20T18:55:07.161 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.161 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.161 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:55:07.161 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806948+0000", 2024-09-20T18:55:07.161 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00010843799999999999 2024-09-20T18:55:07.161 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.162 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.162 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,3", 2024-09-20T18:55:07.162 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.807066+0000", 2024-09-20T18:55:07.162 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000117766 2024-09-20T18:55:07.162 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.162 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.162 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:55:07.162 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.809368+0000", 2024-09-20T18:55:07.162 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0023018130000000002 2024-09-20T18:55:07.162 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.162 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.162 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:07.162 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.809785+0000", 2024-09-20T18:55:07.162 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00041736699999999998 2024-09-20T18:55:07.163 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.163 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.163 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:07.163 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.928629+0000", 2024-09-20T18:55:07.163 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.118843746 2024-09-20T18:55:07.163 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.163 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.163 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:55:07.163 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.928687+0000", 2024-09-20T18:55:07.163 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8418999999999998e-05 2024-09-20T18:55:07.163 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:07.163 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:07.163 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:55:07.163 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.928721+0000", 2024-09-20T18:55:07.164 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.3568000000000001e-05 2024-09-20T18:55:07.164 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:07.164 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:55:07.164 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:07.164 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:07.164 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:55:07.164 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T18:55:07.171 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 27 16 56356 56340 130.331 95.5625 0.0066337 0.00766741 2024-09-20T18:55:07.195 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:07.222 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:07.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:07.224 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:07.302 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:55:07.302 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:55:07.311 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-20T18:55:07.312 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T18:55:07.312 INFO:tasks.thrashosds.thrasher:fixing pg num pool .mgr 2024-09-20T18:55:07.312 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T18:55:07.324 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:07.341 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:55:07.341 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:55:07.355 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:07.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:07.357 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:07.379 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:55:07.379 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:55:07.418 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:55:07.419 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:55:07.454 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:55:07.454 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:55:07.457 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:07.488 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:55:07.488 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:55:07.494 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:07.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:07.495 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:07.524 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:55:07.525 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:55:07.555 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:55:07.555 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:55:07.596 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:55:07.648 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:07.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:07.649 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:55:07.749 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:07.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:07.786 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:07.786 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:07.887 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:07.919 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:07.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:07.920 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:07.994 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:07 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:55:07.994 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:07 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1323999622' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:55:07.994 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:07 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:55:07.995 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:07 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1323999622' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:55:08.015 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:07 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:55:08.016 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:07 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1323999622' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:55:08.021 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:08.054 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:08.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:08.055 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:08.155 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:08.172 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 28 16 57888 57872 129.096 95.75 0.00771314 0.00774095 2024-09-20T18:55:08.187 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:08.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:08.188 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:08.238 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-20T18:55:08.289 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:08.324 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:08.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:08.325 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:08.426 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:08.435 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:55:08.476 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:08.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:08.478 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:08.504 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T18:55:08.578 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:08.611 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:55:08.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:08.622 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:08.622 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:08.722 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:55:08.775 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:08.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:08.790 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:55:08.891 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:55:08.898 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:08 smithi083 ceph-mon[29421]: pgmap v246: 148 pgs: 148 active+clean; 3.3 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.3 KiB/s rd, 144 MiB/s wr, 2.31k op/s 2024-09-20T18:55:08.898 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:08 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:55:08.899 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:08 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:55:08.899 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:08 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:55:08.899 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:08 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:55:08.899 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:08 smithi083 ceph-mon[29421]: osdmap e172: 8 total, 8 up, 8 in 2024-09-20T18:55:08.899 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:08 smithi083 ceph-mon[36260]: pgmap v246: 148 pgs: 148 active+clean; 3.3 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.3 KiB/s rd, 144 MiB/s wr, 2.31k op/s 2024-09-20T18:55:08.899 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:08 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:55:08.899 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:08 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:55:08.899 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:08 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:55:08.899 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:08 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:55:08.899 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:08 smithi083 ceph-mon[36260]: osdmap e172: 8 total, 8 up, 8 in 2024-09-20T18:55:08.933 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:08.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:08.934 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:55:08.968 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:08 smithi142 ceph-mon[32508]: pgmap v246: 148 pgs: 148 active+clean; 3.3 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.3 KiB/s rd, 144 MiB/s wr, 2.31k op/s 2024-09-20T18:55:08.968 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:08 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:55:08.968 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:08 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:55:08.968 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:08 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:55:08.968 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:08 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:55:08.968 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:08 smithi142 ceph-mon[32508]: osdmap e172: 8 total, 8 up, 8 in 2024-09-20T18:55:09.035 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:55:09.081 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:09.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:09.081 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:55:09.141 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:55:09.172 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 29 16 59315 59299 127.719 89.1875 0.0142186 0.00782463 2024-09-20T18:55:09.183 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:09.220 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:09.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:09.221 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:09.322 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:09.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:09.365 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:09.365 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:09.420 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:55:09.425 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-20T18:55:09.465 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:09.466 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T18:55:09.466 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:55:09.501 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:09.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:09.502 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:09.603 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:09.635 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:09.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:09.636 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:09.737 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:09.772 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:09.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:09.773 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:09.874 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:10.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:09 smithi142 ceph-mon[32508]: pgmap v248: 148 pgs: 148 active+clean; 3.3 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 639 B/s rd, 110 MiB/s wr, 1.76k op/s 2024-09-20T18:55:10.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:09 smithi142 ceph-mon[32508]: from='client.25109 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:55:10.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:09 smithi083 ceph-mon[29421]: pgmap v248: 148 pgs: 148 active+clean; 3.3 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 639 B/s rd, 110 MiB/s wr, 1.76k op/s 2024-09-20T18:55:10.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:09 smithi083 ceph-mon[29421]: from='client.25109 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:55:10.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:09 smithi083 ceph-mon[36260]: pgmap v248: 148 pgs: 148 active+clean; 3.3 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 639 B/s rd, 110 MiB/s wr, 1.76k op/s 2024-09-20T18:55:10.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:09 smithi083 ceph-mon[36260]: from='client.25109 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:55:10.061 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:10.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:10.063 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:10.163 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:10.190 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 30 16 60808 60792 126.572 93.3125 0.00849122 0.00789505 2024-09-20T18:55:10.191 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_blocked_ops 2024-09-20T18:55:10.191 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":248,"stamp":"2024-09-20T18:55:08.864481+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3582401051,"num_objects":54853,"num_object_clones":0,"num_object_copies":164559,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54853,"num_whiteouts":0,"num_read":1163,"num_read_kb":1020,"num_write":55203,"num_write_kb":3499529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":590808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55206,"ondisk_log_size":55206,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":11711772,"kb_used_data":10332244,"kb_used_omap":12,"kb_used_meta":1379379,"kb_avail":738249444,"statfs":{"total":767960285184,"available":755967430656,"internally_reserved":0,"allocated":10580217856,"data_stored":10576799353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1412484656},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,2,7,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":921305088,"num_objects":14058,"num_object_clones":0,"num_object_copies":42174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14058,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":14058,"num_write_kb":899712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002485"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":252,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.693316+0000","last_change":"2024-09-20T18:54:13.958541+0000","last_active":"2024-09-20T18:55:05.693316+0000","last_peered":"2024-09-20T18:55:05.693316+0000","last_clean":"2024-09-20T18:55:05.693316+0000","last_became_active":"2024-09-20T18:49:45.943201+0000","last_became_peered":"2024-09-20T18:49:45.943201+0000","last_unstale":"2024-09-20T18:55:05.693316+0000","last_undegraded":"2024-09-20T18:55:05.693316+0000","last_fullsized":"2024-09-20T18:55:05.693316+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:13.958355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:13.958355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:27.449343+0000","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":290,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.681771+0000","last_change":"2024-09-20T18:54:50.231699+0000","last_active":"2024-09-20T18:55:05.681771+0000","last_peered":"2024-09-20T18:55:05.681771+0000","last_clean":"2024-09-20T18:55:05.681771+0000","last_became_active":"2024-09-20T18:52:15.609648+0000","last_became_peered":"2024-09-20T18:52:15.609648+0000","last_unstale":"2024-09-20T18:55:05.681771+0000","last_undegraded":"2024-09-20T18:55:05.681771+0000","last_fullsized":"2024-09-20T18:55:05.681771+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:50.231620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:50.231620+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:06.505661+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":253,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.688858+0000","last_change":"2024-09-20T18:54:25.361988+0000","last_active":"2024-09-20T18:55:05.688858+0000","last_peered":"2024-09-20T18:55:05.688858+0000","last_clean":"2024-09-20T18:55:05.688858+0000","last_became_active":"2024-09-20T18:49:49.959514+0000","last_became_peered":"2024-09-20T18:49:49.959514+0000","last_unstale":"2024-09-20T18:55:05.688858+0000","last_undegraded":"2024-09-20T18:55:05.688858+0000","last_fullsized":"2024-09-20T18:55:05.688858+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:25.361935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:25.361935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:49.454156+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":249,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.682261+0000","last_change":"2024-09-20T18:54:15.221404+0000","last_active":"2024-09-20T18:55:05.682261+0000","last_peered":"2024-09-20T18:55:05.682261+0000","last_clean":"2024-09-20T18:55:05.682261+0000","last_became_active":"2024-09-20T18:49:51.969421+0000","last_became_peered":"2024-09-20T18:49:51.969421+0000","last_unstale":"2024-09-20T18:55:05.682261+0000","last_undegraded":"2024-09-20T18:55:05.682261+0000","last_fullsized":"2024-09-20T18:55:05.682261+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:15.221342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:15.221342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:21.616726+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":249,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.685967+0000","last_change":"2024-09-20T18:54:51.216247+0000","last_active":"2024-09-20T18:55:05.685967+0000","last_peered":"2024-09-20T18:55:05.685967+0000","last_clean":"2024-09-20T18:55:05.685967+0000","last_became_active":"2024-09-20T18:49:52.141967+0000","last_became_peered":"2024-09-20T18:49:52.141967+0000","last_unstale":"2024-09-20T18:55:05.685967+0000","last_undegraded":"2024-09-20T18:55:05.685967+0000","last_fullsized":"2024-09-20T18:55:05.685967+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.216188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.216188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:56.724979+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":245,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.687178+0000","last_change":"2024-09-20T18:51:25.208165+0000","last_active":"2024-09-20T18:55:05.687178+0000","last_peered":"2024-09-20T18:55:05.687178+0000","last_clean":"2024-09-20T18:55:05.687178+0000","last_became_active":"2024-09-20T18:49:45.943607+0000","last_became_peered":"2024-09-20T18:49:45.943607+0000","last_unstale":"2024-09-20T18:55:05.687178+0000","last_undegraded":"2024-09-20T18:55:05.687178+0000","last_fullsized":"2024-09-20T18:55:05.687178+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:25.208073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:25.208073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:09.117437+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":266,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.686203+0000","last_change":"2024-09-20T18:51:07.130137+0000","last_active":"2024-09-20T18:55:05.686203+0000","last_peered":"2024-09-20T18:55:05.686203+0000","last_clean":"2024-09-20T18:55:05.686203+0000","last_became_active":"2024-09-20T18:49:47.952150+0000","last_became_peered":"2024-09-20T18:49:47.952150+0000","last_unstale":"2024-09-20T18:55:05.686203+0000","last_undegraded":"2024-09-20T18:55:05.686203+0000","last_fullsized":"2024-09-20T18:55:05.686203+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:51:07.130062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:07.130062+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:36.182256+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":245,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.681683+0000","last_change":"2024-09-20T18:54:11.246973+0000","last_active":"2024-09-20T18:55:05.681683+0000","last_peered":"2024-09-20T18:55:05.681683+0000","last_clean":"2024-09-20T18:55:05.681683+0000","last_became_active":"2024-09-20T18:49:49.956445+0000","last_became_peered":"2024-09-20T18:49:49.956445+0000","last_unstale":"2024-09-20T18:55:05.681683+0000","last_undegraded":"2024-09-20T18:55:05.681683+0000","last_fullsized":"2024-09-20T18:55:05.681683+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:11.246922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:11.246922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:18.822524+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"59'1","reported_seq":266,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.678500+0000","last_change":"2024-09-20T18:54:15.326527+0000","last_active":"2024-09-20T18:55:05.678500+0000","last_peered":"2024-09-20T18:55:05.678500+0000","last_clean":"2024-09-20T18:55:05.678500+0000","last_became_active":"2024-09-20T18:49:45.946171+0000","last_became_peered":"2024-09-20T18:49:45.946171+0000","last_unstale":"2024-09-20T18:55:05.678500+0000","last_undegraded":"2024-09-20T18:55:05.678500+0000","last_fullsized":"2024-09-20T18:55:05.678500+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:54:15.326476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:15.326476+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:27.762998+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":273,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.682436+0000","last_change":"2024-09-20T18:54:53.241889+0000","last_active":"2024-09-20T18:55:05.682436+0000","last_peered":"2024-09-20T18:55:05.682436+0000","last_clean":"2024-09-20T18:55:05.682436+0000","last_became_active":"2024-09-20T18:49:47.947448+0000","last_became_peered":"2024-09-20T18:49:47.947448+0000","last_unstale":"2024-09-20T18:55:05.682436+0000","last_undegraded":"2024-09-20T18:55:05.682436+0000","last_fullsized":"2024-09-20T18:55:05.682436+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:53.241802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:53.241802+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:59.700784+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":244,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.678108+0000","last_change":"2024-09-20T18:52:36.183932+0000","last_active":"2024-09-20T18:55:05.678108+0000","last_peered":"2024-09-20T18:55:05.678108+0000","last_clean":"2024-09-20T18:55:05.678108+0000","last_became_active":"2024-09-20T18:49:49.961188+0000","last_became_peered":"2024-09-20T18:49:49.961188+0000","last_unstale":"2024-09-20T18:55:05.678108+0000","last_undegraded":"2024-09-20T18:55:05.678108+0000","last_fullsized":"2024-09-20T18:55:05.678108+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:36.183875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:36.183875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:05.164316+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":224,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.689606+0000","last_change":"2024-09-20T18:49:51.966957+0000","last_active":"2024-09-20T18:55:05.689606+0000","last_peered":"2024-09-20T18:55:05.689606+0000","last_clean":"2024-09-20T18:55:05.689606+0000","last_became_active":"2024-09-20T18:49:51.966715+0000","last_became_peered":"2024-09-20T18:49:51.966715+0000","last_unstale":"2024-09-20T18:55:05.689606+0000","last_undegraded":"2024-09-20T18:55:05.689606+0000","last_fullsized":"2024-09-20T18:55:05.689606+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:01.984995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":244,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.678332+0000","last_change":"2024-09-20T18:52:00.970719+0000","last_active":"2024-09-20T18:55:05.678332+0000","last_peered":"2024-09-20T18:55:05.678332+0000","last_clean":"2024-09-20T18:55:05.678332+0000","last_became_active":"2024-09-20T18:49:45.944703+0000","last_became_peered":"2024-09-20T18:49:45.944703+0000","last_unstale":"2024-09-20T18:55:05.678332+0000","last_undegraded":"2024-09-20T18:55:05.678332+0000","last_fullsized":"2024-09-20T18:55:05.678332+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:00.970650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:00.970650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:09.393254+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":277,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.681891+0000","last_change":"2024-09-20T18:51:26.494194+0000","last_active":"2024-09-20T18:55:05.681891+0000","last_peered":"2024-09-20T18:55:05.681891+0000","last_clean":"2024-09-20T18:55:05.681891+0000","last_became_active":"2024-09-20T18:51:23.813399+0000","last_became_peered":"2024-09-20T18:51:23.813399+0000","last_unstale":"2024-09-20T18:55:05.681891+0000","last_undegraded":"2024-09-20T18:55:05.681891+0000","last_fullsized":"2024-09-20T18:55:05.681891+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-20T18:51:26.494118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:26.494118+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:39.154315+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":244,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.687079+0000","last_change":"2024-09-20T18:52:38.799542+0000","last_active":"2024-09-20T18:55:05.687079+0000","last_peered":"2024-09-20T18:55:05.687079+0000","last_clean":"2024-09-20T18:55:05.687079+0000","last_became_active":"2024-09-20T18:49:49.959696+0000","last_became_peered":"2024-09-20T18:49:49.959696+0000","last_unstale":"2024-09-20T18:55:05.687079+0000","last_undegraded":"2024-09-20T18:55:05.687079+0000","last_fullsized":"2024-09-20T18:55:05.687079+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:38.799487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:38.799487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:06.357517+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":240,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.677926+0000","last_change":"2024-09-20T18:53:21.273835+0000","last_active":"2024-09-20T18:55:05.677926+0000","last_peered":"2024-09-20T18:55:05.677926+0000","last_clean":"2024-09-20T18:55:05.677926+0000","last_became_active":"2024-09-20T18:49:51.970663+0000","last_became_peered":"2024-09-20T18:49:51.970663+0000","last_unstale":"2024-09-20T18:55:05.677926+0000","last_undegraded":"2024-09-20T18:55:05.677926+0000","last_fullsized":"2024-09-20T18:55:05.677926+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:21.273788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:21.273788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:31.322423+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":241,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.682891+0000","last_change":"2024-09-20T18:53:53.940250+0000","last_active":"2024-09-20T18:55:05.682891+0000","last_peered":"2024-09-20T18:55:05.682891+0000","last_clean":"2024-09-20T18:55:05.682891+0000","last_became_active":"2024-09-20T18:49:52.141170+0000","last_became_peered":"2024-09-20T18:49:52.141170+0000","last_unstale":"2024-09-20T18:55:05.682891+0000","last_undegraded":"2024-09-20T18:55:05.682891+0000","last_fullsized":"2024-09-20T18:55:05.682891+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:53.940191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:53.940191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":252,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.690668+0000","last_change":"2024-09-20T18:54:23.984024+0000","last_active":"2024-09-20T18:55:05.690668+0000","last_peered":"2024-09-20T18:55:05.690668+0000","last_clean":"2024-09-20T18:55:05.690668+0000","last_became_active":"2024-09-20T18:49:45.950480+0000","last_became_peered":"2024-09-20T18:49:45.950480+0000","last_unstale":"2024-09-20T18:55:05.690668+0000","last_undegraded":"2024-09-20T18:55:05.690668+0000","last_fullsized":"2024-09-20T18:55:05.690668+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:23.983958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:23.983958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:30.113894+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":275,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.688344+0000","last_change":"2024-09-20T18:53:26.762207+0000","last_active":"2024-09-20T18:55:05.688344+0000","last_peered":"2024-09-20T18:55:05.688344+0000","last_clean":"2024-09-20T18:55:05.688344+0000","last_became_active":"2024-09-20T18:49:47.950939+0000","last_became_peered":"2024-09-20T18:49:47.950939+0000","last_unstale":"2024-09-20T18:55:05.688344+0000","last_undegraded":"2024-09-20T18:55:05.688344+0000","last_fullsized":"2024-09-20T18:55:05.688344+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:53:26.762131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:26.762131+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:39.382504+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":237,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.690734+0000","last_change":"2024-09-20T18:51:17.248705+0000","last_active":"2024-09-20T18:55:05.690734+0000","last_peered":"2024-09-20T18:55:05.690734+0000","last_clean":"2024-09-20T18:55:05.690734+0000","last_became_active":"2024-09-20T18:49:49.958518+0000","last_became_peered":"2024-09-20T18:49:49.958518+0000","last_unstale":"2024-09-20T18:55:05.690734+0000","last_undegraded":"2024-09-20T18:55:05.690734+0000","last_fullsized":"2024-09-20T18:55:05.690734+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:17.248634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:17.248634+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:36.366317+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":241,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.687620+0000","last_change":"2024-09-20T18:53:54.259243+0000","last_active":"2024-09-20T18:55:05.687620+0000","last_peered":"2024-09-20T18:55:05.687620+0000","last_clean":"2024-09-20T18:55:05.687620+0000","last_became_active":"2024-09-20T18:49:52.140040+0000","last_became_peered":"2024-09-20T18:49:52.140040+0000","last_unstale":"2024-09-20T18:55:05.687620+0000","last_undegraded":"2024-09-20T18:55:05.687620+0000","last_fullsized":"2024-09-20T18:55:05.687620+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:54.259174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:54.259174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:06.632880+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"59'1","reported_seq":259,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.682472+0000","last_change":"2024-09-20T18:53:42.840884+0000","last_active":"2024-09-20T18:55:05.682472+0000","last_peered":"2024-09-20T18:55:05.682472+0000","last_clean":"2024-09-20T18:55:05.682472+0000","last_became_active":"2024-09-20T18:49:45.940100+0000","last_became_peered":"2024-09-20T18:49:45.940100+0000","last_unstale":"2024-09-20T18:55:05.682472+0000","last_undegraded":"2024-09-20T18:55:05.682472+0000","last_fullsized":"2024-09-20T18:55:05.682472+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:53:42.840800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:42.840800+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":256,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.686690+0000","last_change":"2024-09-20T18:49:47.951215+0000","last_active":"2024-09-20T18:55:05.686690+0000","last_peered":"2024-09-20T18:55:05.686690+0000","last_clean":"2024-09-20T18:55:05.686690+0000","last_became_active":"2024-09-20T18:49:47.950909+0000","last_became_peered":"2024-09-20T18:49:47.950909+0000","last_unstale":"2024-09-20T18:55:05.686690+0000","last_undegraded":"2024-09-20T18:55:05.686690+0000","last_fullsized":"2024-09-20T18:55:05.686690+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:02.359480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":253,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.682614+0000","last_change":"2024-09-20T18:54:44.239273+0000","last_active":"2024-09-20T18:55:05.682614+0000","last_peered":"2024-09-20T18:55:05.682614+0000","last_clean":"2024-09-20T18:55:05.682614+0000","last_became_active":"2024-09-20T18:49:49.959102+0000","last_became_peered":"2024-09-20T18:49:49.959102+0000","last_unstale":"2024-09-20T18:55:05.682614+0000","last_undegraded":"2024-09-20T18:55:05.682614+0000","last_fullsized":"2024-09-20T18:55:05.682614+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:44.239226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.239226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:47.649861+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":241,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.686985+0000","last_change":"2024-09-20T18:52:15.826819+0000","last_active":"2024-09-20T18:55:05.686985+0000","last_peered":"2024-09-20T18:55:05.686985+0000","last_clean":"2024-09-20T18:55:05.686985+0000","last_became_active":"2024-09-20T18:49:51.970058+0000","last_became_peered":"2024-09-20T18:49:51.970058+0000","last_unstale":"2024-09-20T18:55:05.686985+0000","last_undegraded":"2024-09-20T18:55:05.686985+0000","last_fullsized":"2024-09-20T18:55:05.686985+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T18:52:15.826638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:15.826638+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:34.458571+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":246,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.688986+0000","last_change":"2024-09-20T18:52:47.158384+0000","last_active":"2024-09-20T18:55:05.688986+0000","last_peered":"2024-09-20T18:55:05.688986+0000","last_clean":"2024-09-20T18:55:05.688986+0000","last_became_active":"2024-09-20T18:49:45.933078+0000","last_became_peered":"2024-09-20T18:49:45.933078+0000","last_unstale":"2024-09-20T18:55:05.688986+0000","last_undegraded":"2024-09-20T18:55:05.688986+0000","last_fullsized":"2024-09-20T18:55:05.688986+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:52:47.158260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:47.158260+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:05.613512+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":263,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.694625+0000","last_change":"2024-09-20T18:54:17.919123+0000","last_active":"2024-09-20T18:55:05.694625+0000","last_peered":"2024-09-20T18:55:05.694625+0000","last_clean":"2024-09-20T18:55:05.694625+0000","last_became_active":"2024-09-20T18:49:48.500809+0000","last_became_peered":"2024-09-20T18:49:48.500809+0000","last_unstale":"2024-09-20T18:55:05.694625+0000","last_undegraded":"2024-09-20T18:55:05.694625+0000","last_fullsized":"2024-09-20T18:55:05.694625+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:54:17.919055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:17.919055+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:19.776366+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":319,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.686310+0000","last_change":"2024-09-20T18:53:44.524093+0000","last_active":"2024-09-20T18:55:05.686310+0000","last_peered":"2024-09-20T18:55:05.686310+0000","last_clean":"2024-09-20T18:55:05.686310+0000","last_became_active":"2024-09-20T18:49:49.954772+0000","last_became_peered":"2024-09-20T18:49:49.954772+0000","last_unstale":"2024-09-20T18:55:05.686310+0000","last_undegraded":"2024-09-20T18:55:05.686310+0000","last_fullsized":"2024-09-20T18:55:05.686310+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:53:44.524040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:44.524040+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:05.658458+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"171'3347","reported_seq":3380,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:06.016252+0000","last_change":"2024-09-20T18:54:33.928915+0000","last_active":"2024-09-20T18:55:06.016252+0000","last_peered":"2024-09-20T18:55:06.016252+0000","last_clean":"2024-09-20T18:55:06.016252+0000","last_became_active":"2024-09-20T18:54:33.928692+0000","last_became_peered":"2024-09-20T18:54:33.928692+0000","last_unstale":"2024-09-20T18:55:06.016252+0000","last_undegraded":"2024-09-20T18:55:06.016252+0000","last_fullsized":"2024-09-20T18:55:06.016252+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":3347,"log_dups_size":0,"ondisk_log_size":3347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:35.597538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":219348992,"num_objects":3347,"num_object_clones":0,"num_object_copies":10041,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3347,"num_write_kb":214208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":258,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.687127+0000","last_change":"2024-09-20T18:53:48.782289+0000","last_active":"2024-09-20T18:55:05.687127+0000","last_peered":"2024-09-20T18:55:05.687127+0000","last_clean":"2024-09-20T18:55:05.687127+0000","last_became_active":"2024-09-20T18:49:58.992302+0000","last_became_peered":"2024-09-20T18:49:58.992302+0000","last_unstale":"2024-09-20T18:55:05.687127+0000","last_undegraded":"2024-09-20T18:55:05.687127+0000","last_fullsized":"2024-09-20T18:55:05.687127+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:53:48.782192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:48.782192+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":253,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.686435+0000","last_change":"2024-09-20T18:54:54.288199+0000","last_active":"2024-09-20T18:55:05.686435+0000","last_peered":"2024-09-20T18:55:05.686435+0000","last_clean":"2024-09-20T18:55:05.686435+0000","last_became_active":"2024-09-20T18:49:45.936104+0000","last_became_peered":"2024-09-20T18:49:45.936104+0000","last_unstale":"2024-09-20T18:55:05.686435+0000","last_undegraded":"2024-09-20T18:55:05.686435+0000","last_fullsized":"2024-09-20T18:55:05.686435+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:54.288117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:54.288117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:07.484422+0000","scrub_duration":46,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":320,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.683078+0000","last_change":"2024-09-20T18:54:14.292660+0000","last_active":"2024-09-20T18:55:05.683078+0000","last_peered":"2024-09-20T18:55:05.683078+0000","last_clean":"2024-09-20T18:55:05.683078+0000","last_became_active":"2024-09-20T18:49:49.960697+0000","last_became_peered":"2024-09-20T18:49:49.960697+0000","last_unstale":"2024-09-20T18:55:05.683078+0000","last_undegraded":"2024-09-20T18:55:05.683078+0000","last_fullsized":"2024-09-20T18:55:05.683078+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:54:14.292610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:14.292610+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:35.787441+0000","scrub_duration":34,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":240,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.678192+0000","last_change":"2024-09-20T18:54:10.381171+0000","last_active":"2024-09-20T18:55:05.678192+0000","last_peered":"2024-09-20T18:55:05.678192+0000","last_clean":"2024-09-20T18:55:05.678192+0000","last_became_active":"2024-09-20T18:49:51.971399+0000","last_became_peered":"2024-09-20T18:49:51.971399+0000","last_unstale":"2024-09-20T18:55:05.678192+0000","last_undegraded":"2024-09-20T18:55:05.678192+0000","last_fullsized":"2024-09-20T18:55:05.678192+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:10.381108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:10.381108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:32.368649+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.c","version":"171'3404","reported_seq":3437,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:06.314356+0000","last_change":"2024-09-20T18:54:34.311256+0000","last_active":"2024-09-20T18:55:06.314356+0000","last_peered":"2024-09-20T18:55:06.314356+0000","last_clean":"2024-09-20T18:55:06.314356+0000","last_became_active":"2024-09-20T18:54:34.310904+0000","last_became_peered":"2024-09-20T18:54:34.310904+0000","last_unstale":"2024-09-20T18:55:06.314356+0000","last_undegraded":"2024-09-20T18:55:06.314356+0000","last_fullsized":"2024-09-20T18:55:06.314356+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":3404,"log_dups_size":0,"ondisk_log_size":3404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:38.039908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":223084544,"num_objects":3404,"num_object_clones":0,"num_object_copies":10212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3404,"num_write_kb":217856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":281,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.686195+0000","last_change":"2024-09-20T18:54:22.077299+0000","last_active":"2024-09-20T18:55:05.686195+0000","last_peered":"2024-09-20T18:55:05.686195+0000","last_clean":"2024-09-20T18:55:05.686195+0000","last_became_active":"2024-09-20T18:49:48.499189+0000","last_became_peered":"2024-09-20T18:49:48.499189+0000","last_unstale":"2024-09-20T18:55:05.686195+0000","last_undegraded":"2024-09-20T18:55:05.686195+0000","last_fullsized":"2024-09-20T18:55:05.686195+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-20T18:54:22.077212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:22.077212+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:52.022959+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":245,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.688061+0000","last_change":"2024-09-20T18:54:57.237740+0000","last_active":"2024-09-20T18:55:05.688061+0000","last_peered":"2024-09-20T18:55:05.688061+0000","last_clean":"2024-09-20T18:55:05.688061+0000","last_became_active":"2024-09-20T18:49:45.940544+0000","last_became_peered":"2024-09-20T18:49:45.940544+0000","last_unstale":"2024-09-20T18:55:05.688061+0000","last_undegraded":"2024-09-20T18:55:05.688061+0000","last_fullsized":"2024-09-20T18:55:05.688061+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.237675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.237675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:25.957896+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":253,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.681981+0000","last_change":"2024-09-20T18:54:51.229251+0000","last_active":"2024-09-20T18:55:05.681981+0000","last_peered":"2024-09-20T18:55:05.681981+0000","last_clean":"2024-09-20T18:55:05.681981+0000","last_became_active":"2024-09-20T18:49:49.963073+0000","last_became_peered":"2024-09-20T18:49:49.963073+0000","last_unstale":"2024-09-20T18:55:05.681981+0000","last_undegraded":"2024-09-20T18:55:05.681981+0000","last_fullsized":"2024-09-20T18:55:05.681981+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.229201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.229201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:51.292800+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":249,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.682414+0000","last_change":"2024-09-20T18:54:55.323937+0000","last_active":"2024-09-20T18:55:05.682414+0000","last_peered":"2024-09-20T18:55:05.682414+0000","last_clean":"2024-09-20T18:55:05.682414+0000","last_became_active":"2024-09-20T18:49:51.968793+0000","last_became_peered":"2024-09-20T18:49:51.968793+0000","last_unstale":"2024-09-20T18:55:05.682414+0000","last_undegraded":"2024-09-20T18:55:05.682414+0000","last_fullsized":"2024-09-20T18:55:05.682414+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:55.323692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:55.323692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:56.443325+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"171'3308","reported_seq":3340,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.835397+0000","last_change":"2024-09-20T18:54:34.311680+0000","last_active":"2024-09-20T18:55:05.835397+0000","last_peered":"2024-09-20T18:55:05.835397+0000","last_clean":"2024-09-20T18:55:05.835397+0000","last_became_active":"2024-09-20T18:54:34.311241+0000","last_became_peered":"2024-09-20T18:54:34.311241+0000","last_unstale":"2024-09-20T18:55:05.835397+0000","last_undegraded":"2024-09-20T18:55:05.835397+0000","last_fullsized":"2024-09-20T18:55:05.835397+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":3308,"log_dups_size":0,"ondisk_log_size":3308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:38.100923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":216793088,"num_objects":3308,"num_object_clones":0,"num_object_copies":9924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3308,"num_write_kb":211712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":286,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.682066+0000","last_change":"2024-09-20T18:54:57.394980+0000","last_active":"2024-09-20T18:55:05.682066+0000","last_peered":"2024-09-20T18:55:05.682066+0000","last_clean":"2024-09-20T18:55:05.682066+0000","last_became_active":"2024-09-20T18:49:47.941895+0000","last_became_peered":"2024-09-20T18:49:47.941895+0000","last_unstale":"2024-09-20T18:55:05.682066+0000","last_undegraded":"2024-09-20T18:55:05.682066+0000","last_fullsized":"2024-09-20T18:55:05.682066+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-20T18:54:57.394893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.394893+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:21.366105+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":245,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.694876+0000","last_change":"2024-09-20T18:51:27.187060+0000","last_active":"2024-09-20T18:55:05.694876+0000","last_peered":"2024-09-20T18:55:05.694876+0000","last_clean":"2024-09-20T18:55:05.694876+0000","last_became_active":"2024-09-20T18:49:45.940469+0000","last_became_peered":"2024-09-20T18:49:45.940469+0000","last_unstale":"2024-09-20T18:55:05.694876+0000","last_undegraded":"2024-09-20T18:55:05.694876+0000","last_fullsized":"2024-09-20T18:55:05.694876+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:51:27.186991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:27.186991+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"66'8","reported_seq":326,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.686028+0000","last_change":"2024-09-20T18:54:47.223010+0000","last_active":"2024-09-20T18:55:05.686028+0000","last_peered":"2024-09-20T18:55:05.686028+0000","last_clean":"2024-09-20T18:55:05.686028+0000","last_became_active":"2024-09-20T18:49:49.956133+0000","last_became_peered":"2024-09-20T18:49:49.956133+0000","last_unstale":"2024-09-20T18:55:05.686028+0000","last_undegraded":"2024-09-20T18:55:05.686028+0000","last_fullsized":"2024-09-20T18:55:05.686028+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:54:47.222937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:47.222937+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:16.538233+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":240,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.686740+0000","last_change":"2024-09-20T18:52:42.880767+0000","last_active":"2024-09-20T18:55:05.686740+0000","last_peered":"2024-09-20T18:55:05.686740+0000","last_clean":"2024-09-20T18:55:05.686740+0000","last_became_active":"2024-09-20T18:49:52.142113+0000","last_became_peered":"2024-09-20T18:49:52.142113+0000","last_unstale":"2024-09-20T18:55:05.686740+0000","last_undegraded":"2024-09-20T18:55:05.686740+0000","last_fullsized":"2024-09-20T18:55:05.686740+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:42.880724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:42.880724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:08.373192+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"171'3434","reported_seq":3467,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:06.319406+0000","last_change":"2024-09-20T18:54:34.309939+0000","last_active":"2024-09-20T18:55:06.319406+0000","last_peered":"2024-09-20T18:55:06.319406+0000","last_clean":"2024-09-20T18:55:06.319406+0000","last_became_active":"2024-09-20T18:54:34.309630+0000","last_became_peered":"2024-09-20T18:54:34.309630+0000","last_unstale":"2024-09-20T18:55:06.319406+0000","last_undegraded":"2024-09-20T18:55:06.319406+0000","last_fullsized":"2024-09-20T18:55:06.319406+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":3434,"log_dups_size":0,"ondisk_log_size":3434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:38.295295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":225050624,"num_objects":3434,"num_object_clones":0,"num_object_copies":10302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3434,"num_write_kb":219776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":264,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.688980+0000","last_change":"2024-09-20T18:52:39.246125+0000","last_active":"2024-09-20T18:55:05.688980+0000","last_peered":"2024-09-20T18:55:05.688980+0000","last_clean":"2024-09-20T18:55:05.688980+0000","last_became_active":"2024-09-20T18:49:47.949626+0000","last_became_peered":"2024-09-20T18:49:47.949626+0000","last_unstale":"2024-09-20T18:55:05.688980+0000","last_undegraded":"2024-09-20T18:55:05.688980+0000","last_fullsized":"2024-09-20T18:55:05.688980+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:52:39.246050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:39.246050+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:09.285670+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":260,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.678587+0000","last_change":"2024-09-20T18:54:18.326947+0000","last_active":"2024-09-20T18:55:05.678587+0000","last_peered":"2024-09-20T18:55:05.678587+0000","last_clean":"2024-09-20T18:55:05.678587+0000","last_became_active":"2024-09-20T18:49:45.942496+0000","last_became_peered":"2024-09-20T18:49:45.942496+0000","last_unstale":"2024-09-20T18:55:05.678587+0000","last_undegraded":"2024-09-20T18:55:05.678587+0000","last_fullsized":"2024-09-20T18:55:05.678587+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:18.326875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:18.326875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:33.692074+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":309,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.690330+0000","last_change":"2024-09-20T18:51:16.088407+0000","last_active":"2024-09-20T18:55:05.690330+0000","last_peered":"2024-09-20T18:55:05.690330+0000","last_clean":"2024-09-20T18:55:05.690330+0000","last_became_active":"2024-09-20T18:49:49.959365+0000","last_became_peered":"2024-09-20T18:49:49.959365+0000","last_unstale":"2024-09-20T18:55:05.690330+0000","last_undegraded":"2024-09-20T18:55:05.690330+0000","last_fullsized":"2024-09-20T18:55:05.690330+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:51:16.088343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:16.088343+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:37.794322+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":224,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.689740+0000","last_change":"2024-09-20T18:49:51.965733+0000","last_active":"2024-09-20T18:55:05.689740+0000","last_peered":"2024-09-20T18:55:05.689740+0000","last_clean":"2024-09-20T18:55:05.689740+0000","last_became_active":"2024-09-20T18:49:51.964780+0000","last_became_peered":"2024-09-20T18:49:51.964780+0000","last_unstale":"2024-09-20T18:55:05.689740+0000","last_undegraded":"2024-09-20T18:55:05.689740+0000","last_fullsized":"2024-09-20T18:55:05.689740+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"171'3525","reported_seq":3557,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:06.297853+0000","last_change":"2024-09-20T18:54:34.309809+0000","last_active":"2024-09-20T18:55:06.297853+0000","last_peered":"2024-09-20T18:55:06.297853+0000","last_clean":"2024-09-20T18:55:06.297853+0000","last_became_active":"2024-09-20T18:54:34.309231+0000","last_became_peered":"2024-09-20T18:54:34.309231+0000","last_unstale":"2024-09-20T18:55:06.297853+0000","last_undegraded":"2024-09-20T18:55:06.297853+0000","last_fullsized":"2024-09-20T18:55:06.297853+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":3525,"log_dups_size":0,"ondisk_log_size":3525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:53.255575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":231014400,"num_objects":3525,"num_object_clones":0,"num_object_copies":10575,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3525,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3525,"num_write_kb":225600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":284,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.690481+0000","last_change":"2024-09-20T18:54:14.923895+0000","last_active":"2024-09-20T18:55:05.690481+0000","last_peered":"2024-09-20T18:55:05.690481+0000","last_clean":"2024-09-20T18:55:05.690481+0000","last_became_active":"2024-09-20T18:49:48.497736+0000","last_became_peered":"2024-09-20T18:49:48.497736+0000","last_unstale":"2024-09-20T18:55:05.690481+0000","last_undegraded":"2024-09-20T18:55:05.690481+0000","last_fullsized":"2024-09-20T18:55:05.690481+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-20T18:54:14.923682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:14.923682+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:38.986258+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":237,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.691158+0000","last_change":"2024-09-20T18:49:45.942589+0000","last_active":"2024-09-20T18:55:05.691158+0000","last_peered":"2024-09-20T18:55:05.691158+0000","last_clean":"2024-09-20T18:55:05.691158+0000","last_became_active":"2024-09-20T18:49:45.942253+0000","last_became_peered":"2024-09-20T18:49:45.942253+0000","last_unstale":"2024-09-20T18:55:05.691158+0000","last_undegraded":"2024-09-20T18:55:05.691158+0000","last_fullsized":"2024-09-20T18:55:05.691158+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:59.467754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":252,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.685991+0000","last_change":"2024-09-20T18:54:50.610923+0000","last_active":"2024-09-20T18:55:05.685991+0000","last_peered":"2024-09-20T18:55:05.685991+0000","last_clean":"2024-09-20T18:55:05.685991+0000","last_became_active":"2024-09-20T18:49:49.954299+0000","last_became_peered":"2024-09-20T18:49:49.954299+0000","last_unstale":"2024-09-20T18:55:05.685991+0000","last_undegraded":"2024-09-20T18:55:05.685991+0000","last_fullsized":"2024-09-20T18:55:05.685991+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:50.610871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:50.610871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:15.062322+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":249,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.688760+0000","last_change":"2024-09-20T18:54:57.403031+0000","last_active":"2024-09-20T18:55:05.688760+0000","last_peered":"2024-09-20T18:55:05.688760+0000","last_clean":"2024-09-20T18:55:05.688760+0000","last_became_active":"2024-09-20T18:49:52.139981+0000","last_became_peered":"2024-09-20T18:49:52.139981+0000","last_unstale":"2024-09-20T18:55:05.688760+0000","last_undegraded":"2024-09-20T18:55:05.688760+0000","last_fullsized":"2024-09-20T18:55:05.688760+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.402841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.402841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:00.589752+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"171'3431","reported_seq":3464,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.834260+0000","last_change":"2024-09-20T18:54:34.311561+0000","last_active":"2024-09-20T18:55:05.834260+0000","last_peered":"2024-09-20T18:55:05.834260+0000","last_clean":"2024-09-20T18:55:05.834260+0000","last_became_active":"2024-09-20T18:54:34.311181+0000","last_became_peered":"2024-09-20T18:54:34.311181+0000","last_unstale":"2024-09-20T18:55:05.834260+0000","last_undegraded":"2024-09-20T18:55:05.834260+0000","last_fullsized":"2024-09-20T18:55:05.834260+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":3431,"log_dups_size":0,"ondisk_log_size":3431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:02.829367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":224854016,"num_objects":3431,"num_object_clones":0,"num_object_copies":10293,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3431,"num_write_kb":219584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"66'28","reported_seq":353,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.691074+0000","last_change":"2024-09-20T18:51:57.293659+0000","last_active":"2024-09-20T18:55:05.691074+0000","last_peered":"2024-09-20T18:55:05.691074+0000","last_clean":"2024-09-20T18:55:05.691074+0000","last_became_active":"2024-09-20T18:49:47.955335+0000","last_became_peered":"2024-09-20T18:49:47.955335+0000","last_unstale":"2024-09-20T18:55:05.691074+0000","last_undegraded":"2024-09-20T18:55:05.691074+0000","last_fullsized":"2024-09-20T18:55:05.691074+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'28","last_scrub_stamp":"2024-09-20T18:51:57.293574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:57.293574+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:08.276804+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":237,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.683337+0000","last_change":"2024-09-20T18:49:45.945275+0000","last_active":"2024-09-20T18:55:05.683337+0000","last_peered":"2024-09-20T18:55:05.683337+0000","last_clean":"2024-09-20T18:55:05.683337+0000","last_became_active":"2024-09-20T18:49:45.944992+0000","last_became_peered":"2024-09-20T18:49:45.944992+0000","last_unstale":"2024-09-20T18:55:05.683337+0000","last_undegraded":"2024-09-20T18:55:05.683337+0000","last_fullsized":"2024-09-20T18:55:05.683337+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:04.353741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":278,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.678611+0000","last_change":"2024-09-20T18:54:51.227606+0000","last_active":"2024-09-20T18:55:05.678611+0000","last_peered":"2024-09-20T18:55:05.678611+0000","last_clean":"2024-09-20T18:55:05.678611+0000","last_became_active":"2024-09-20T18:49:44.936027+0000","last_became_peered":"2024-09-20T18:49:44.936027+0000","last_unstale":"2024-09-20T18:55:05.678611+0000","last_undegraded":"2024-09-20T18:55:05.678611+0000","last_fullsized":"2024-09-20T18:55:05.678611+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-20T18:54:51.227531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.227531+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:06.005839+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0012781159999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":244,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.690718+0000","last_change":"2024-09-20T18:53:12.897519+0000","last_active":"2024-09-20T18:55:05.690718+0000","last_peered":"2024-09-20T18:55:05.690718+0000","last_clean":"2024-09-20T18:55:05.690718+0000","last_became_active":"2024-09-20T18:49:49.953971+0000","last_became_peered":"2024-09-20T18:49:49.953971+0000","last_unstale":"2024-09-20T18:55:05.690718+0000","last_undegraded":"2024-09-20T18:55:05.690718+0000","last_fullsized":"2024-09-20T18:55:05.690718+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:12.897051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:12.897051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":225,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.686944+0000","last_change":"2024-09-20T18:49:51.969702+0000","last_active":"2024-09-20T18:55:05.686944+0000","last_peered":"2024-09-20T18:55:05.686944+0000","last_clean":"2024-09-20T18:55:05.686944+0000","last_became_active":"2024-09-20T18:49:51.969400+0000","last_became_peered":"2024-09-20T18:49:51.969400+0000","last_unstale":"2024-09-20T18:55:05.686944+0000","last_undegraded":"2024-09-20T18:55:05.686944+0000","last_fullsized":"2024-09-20T18:55:05.686944+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:37.214034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"171'3425","reported_seq":3458,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:06.272431+0000","last_change":"2024-09-20T18:54:33.926785+0000","last_active":"2024-09-20T18:55:06.272431+0000","last_peered":"2024-09-20T18:55:06.272431+0000","last_clean":"2024-09-20T18:55:06.272431+0000","last_became_active":"2024-09-20T18:54:33.925750+0000","last_became_peered":"2024-09-20T18:54:33.925750+0000","last_unstale":"2024-09-20T18:55:06.272431+0000","last_undegraded":"2024-09-20T18:55:06.272431+0000","last_fullsized":"2024-09-20T18:55:06.272431+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":3425,"log_dups_size":0,"ondisk_log_size":3425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:39.195256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":224460800,"num_objects":3425,"num_object_clones":0,"num_object_copies":10275,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3425,"num_write_kb":219200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":276,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.688664+0000","last_change":"2024-09-20T18:53:48.214566+0000","last_active":"2024-09-20T18:55:05.688664+0000","last_peered":"2024-09-20T18:55:05.688664+0000","last_clean":"2024-09-20T18:55:05.688664+0000","last_became_active":"2024-09-20T18:49:47.950904+0000","last_became_peered":"2024-09-20T18:49:47.950904+0000","last_unstale":"2024-09-20T18:55:05.688664+0000","last_undegraded":"2024-09-20T18:55:05.688664+0000","last_fullsized":"2024-09-20T18:55:05.688664+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-20T18:53:48.214491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:48.214491+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:15.858989+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":261,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.687871+0000","last_change":"2024-09-20T18:54:23.359948+0000","last_active":"2024-09-20T18:55:05.687871+0000","last_peered":"2024-09-20T18:55:05.687871+0000","last_clean":"2024-09-20T18:55:05.687871+0000","last_became_active":"2024-09-20T18:49:45.939884+0000","last_became_peered":"2024-09-20T18:49:45.939884+0000","last_unstale":"2024-09-20T18:55:05.687871+0000","last_undegraded":"2024-09-20T18:55:05.687871+0000","last_fullsized":"2024-09-20T18:55:05.687871+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:23.359769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:23.359769+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:52.487977+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":255,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.687698+0000","last_change":"2024-09-20T18:52:07.718766+0000","last_active":"2024-09-20T18:55:05.687698+0000","last_peered":"2024-09-20T18:55:05.687698+0000","last_clean":"2024-09-20T18:55:05.687698+0000","last_became_active":"2024-09-20T18:49:44.937135+0000","last_became_peered":"2024-09-20T18:49:44.937135+0000","last_unstale":"2024-09-20T18:55:05.687698+0000","last_undegraded":"2024-09-20T18:55:05.687698+0000","last_fullsized":"2024-09-20T18:55:05.687698+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:52:07.718646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:52:07.718646+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:38.344013+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.00095558800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":238,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.688637+0000","last_change":"2024-09-20T18:51:59.796450+0000","last_active":"2024-09-20T18:55:05.688637+0000","last_peered":"2024-09-20T18:55:05.688637+0000","last_clean":"2024-09-20T18:55:05.688637+0000","last_became_active":"2024-09-20T18:49:49.954424+0000","last_became_peered":"2024-09-20T18:49:49.954424+0000","last_unstale":"2024-09-20T18:55:05.688637+0000","last_undegraded":"2024-09-20T18:55:05.688637+0000","last_fullsized":"2024-09-20T18:55:05.688637+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:59.796383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:59.796383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:06.819934+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":232,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.686915+0000","last_change":"2024-09-20T18:52:39.814324+0000","last_active":"2024-09-20T18:55:05.686915+0000","last_peered":"2024-09-20T18:55:05.686915+0000","last_clean":"2024-09-20T18:55:05.686915+0000","last_became_active":"2024-09-20T18:49:52.139789+0000","last_became_peered":"2024-09-20T18:49:52.139789+0000","last_unstale":"2024-09-20T18:55:05.686915+0000","last_undegraded":"2024-09-20T18:55:05.686915+0000","last_fullsized":"2024-09-20T18:55:05.686915+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:39.814259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:39.814259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"171'3493","reported_seq":3526,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.999323+0000","last_change":"2024-09-20T18:54:34.311774+0000","last_active":"2024-09-20T18:55:05.999323+0000","last_peered":"2024-09-20T18:55:05.999323+0000","last_clean":"2024-09-20T18:55:05.999323+0000","last_became_active":"2024-09-20T18:54:34.311487+0000","last_became_peered":"2024-09-20T18:54:34.311487+0000","last_unstale":"2024-09-20T18:55:05.999323+0000","last_undegraded":"2024-09-20T18:55:05.999323+0000","last_fullsized":"2024-09-20T18:55:05.999323+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":3493,"log_dups_size":0,"ondisk_log_size":3493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:40.976131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":228917248,"num_objects":3493,"num_object_clones":0,"num_object_copies":10479,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3493,"num_write_kb":223552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":274,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.694707+0000","last_change":"2024-09-20T18:53:47.786753+0000","last_active":"2024-09-20T18:55:05.694707+0000","last_peered":"2024-09-20T18:55:05.694707+0000","last_clean":"2024-09-20T18:55:05.694707+0000","last_became_active":"2024-09-20T18:49:47.949230+0000","last_became_peered":"2024-09-20T18:49:47.949230+0000","last_unstale":"2024-09-20T18:55:05.694707+0000","last_undegraded":"2024-09-20T18:55:05.694707+0000","last_fullsized":"2024-09-20T18:55:05.694707+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:53:47.786681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:47.786681+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":260,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.690892+0000","last_change":"2024-09-20T18:54:27.905856+0000","last_active":"2024-09-20T18:55:05.690892+0000","last_peered":"2024-09-20T18:55:05.690892+0000","last_clean":"2024-09-20T18:55:05.690892+0000","last_became_active":"2024-09-20T18:49:45.939197+0000","last_became_peered":"2024-09-20T18:49:45.939197+0000","last_unstale":"2024-09-20T18:55:05.690892+0000","last_undegraded":"2024-09-20T18:55:05.690892+0000","last_fullsized":"2024-09-20T18:55:05.690892+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:27.905803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:27.905803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:44.644028+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":586,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.687683+0000","last_change":"2024-09-20T18:54:26.359887+0000","last_active":"2024-09-20T18:55:05.687683+0000","last_peered":"2024-09-20T18:55:05.687683+0000","last_clean":"2024-09-20T18:55:05.687683+0000","last_became_active":"2024-09-20T18:49:44.936819+0000","last_became_peered":"2024-09-20T18:49:44.936819+0000","last_unstale":"2024-09-20T18:55:05.687683+0000","last_undegraded":"2024-09-20T18:55:05.687683+0000","last_fullsized":"2024-09-20T18:55:05.687683+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-20T18:54:26.359807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:54:26.359807+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:31.668091+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0010996199999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":315,"num_read_kb":310,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":244,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.678825+0000","last_change":"2024-09-20T18:54:59.151228+0000","last_active":"2024-09-20T18:55:05.678825+0000","last_peered":"2024-09-20T18:55:05.678825+0000","last_clean":"2024-09-20T18:55:05.678825+0000","last_became_active":"2024-09-20T18:49:49.961066+0000","last_became_peered":"2024-09-20T18:49:49.961066+0000","last_unstale":"2024-09-20T18:55:05.678825+0000","last_undegraded":"2024-09-20T18:55:05.678825+0000","last_fullsized":"2024-09-20T18:55:05.678825+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:59.150063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:59.150063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:04.007443+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":241,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.688453+0000","last_change":"2024-09-20T18:53:49.227504+0000","last_active":"2024-09-20T18:55:05.688453+0000","last_peered":"2024-09-20T18:55:05.688453+0000","last_clean":"2024-09-20T18:55:05.688453+0000","last_became_active":"2024-09-20T18:49:51.970343+0000","last_became_peered":"2024-09-20T18:49:51.970343+0000","last_unstale":"2024-09-20T18:55:05.688453+0000","last_undegraded":"2024-09-20T18:55:05.688453+0000","last_fullsized":"2024-09-20T18:55:05.688453+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:49.227436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:49.227436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:07.422442+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"171'3405","reported_seq":3438,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.824701+0000","last_change":"2024-09-20T18:54:33.928788+0000","last_active":"2024-09-20T18:55:05.824701+0000","last_peered":"2024-09-20T18:55:05.824701+0000","last_clean":"2024-09-20T18:55:05.824701+0000","last_became_active":"2024-09-20T18:54:33.928422+0000","last_became_peered":"2024-09-20T18:54:33.928422+0000","last_unstale":"2024-09-20T18:55:05.824701+0000","last_undegraded":"2024-09-20T18:55:05.824701+0000","last_fullsized":"2024-09-20T18:55:05.824701+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":3405,"log_dups_size":0,"ondisk_log_size":3405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:38.755459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":223150080,"num_objects":3405,"num_object_clones":0,"num_object_copies":10215,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3405,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3405,"num_write_kb":217920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":318,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.686420+0000","last_change":"2024-09-20T18:49:48.498345+0000","last_active":"2024-09-20T18:55:05.686420+0000","last_peered":"2024-09-20T18:55:05.686420+0000","last_clean":"2024-09-20T18:55:05.686420+0000","last_became_active":"2024-09-20T18:49:48.497936+0000","last_became_peered":"2024-09-20T18:49:48.497936+0000","last_unstale":"2024-09-20T18:55:05.686420+0000","last_undegraded":"2024-09-20T18:55:05.686420+0000","last_fullsized":"2024-09-20T18:55:05.686420+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:59.677116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":253,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.685761+0000","last_change":"2024-09-20T18:53:13.145127+0000","last_active":"2024-09-20T18:55:05.685761+0000","last_peered":"2024-09-20T18:55:05.685761+0000","last_clean":"2024-09-20T18:55:05.685761+0000","last_became_active":"2024-09-20T18:49:45.941136+0000","last_became_peered":"2024-09-20T18:49:45.941136+0000","last_unstale":"2024-09-20T18:55:05.685761+0000","last_undegraded":"2024-09-20T18:55:05.685761+0000","last_fullsized":"2024-09-20T18:55:05.685761+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:13.144896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:13.144896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:40.235212+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":291,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.687837+0000","last_change":"2024-09-20T18:52:11.887878+0000","last_active":"2024-09-20T18:55:05.687837+0000","last_peered":"2024-09-20T18:55:05.687837+0000","last_clean":"2024-09-20T18:55:05.687837+0000","last_became_active":"2024-09-20T18:49:36.545306+0000","last_became_peered":"2024-09-20T18:49:36.545306+0000","last_unstale":"2024-09-20T18:55:05.687837+0000","last_undegraded":"2024-09-20T18:55:05.687837+0000","last_fullsized":"2024-09-20T18:55:05.687837+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T18:52:11.887733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T18:52:11.887733+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:37.449933+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":236,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.687191+0000","last_change":"2024-09-20T18:51:22.852942+0000","last_active":"2024-09-20T18:55:05.687191+0000","last_peered":"2024-09-20T18:55:05.687191+0000","last_clean":"2024-09-20T18:55:05.687191+0000","last_became_active":"2024-09-20T18:49:49.956128+0000","last_became_peered":"2024-09-20T18:49:49.956128+0000","last_unstale":"2024-09-20T18:55:05.687191+0000","last_undegraded":"2024-09-20T18:55:05.687191+0000","last_fullsized":"2024-09-20T18:55:05.687191+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:22.852869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:22.852869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:39.467142+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":240,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.678674+0000","last_change":"2024-09-20T18:53:45.306565+0000","last_active":"2024-09-20T18:55:05.678674+0000","last_peered":"2024-09-20T18:55:05.678674+0000","last_clean":"2024-09-20T18:55:05.678674+0000","last_became_active":"2024-09-20T18:49:51.971332+0000","last_became_peered":"2024-09-20T18:49:51.971332+0000","last_unstale":"2024-09-20T18:55:05.678674+0000","last_undegraded":"2024-09-20T18:55:05.678674+0000","last_fullsized":"2024-09-20T18:55:05.678674+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:45.306514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:45.306514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:10.093423+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"171'3395","reported_seq":3426,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.802685+0000","last_change":"2024-09-20T18:54:33.929366+0000","last_active":"2024-09-20T18:55:05.802685+0000","last_peered":"2024-09-20T18:55:05.802685+0000","last_clean":"2024-09-20T18:55:05.802685+0000","last_became_active":"2024-09-20T18:54:33.929072+0000","last_became_peered":"2024-09-20T18:54:33.929072+0000","last_unstale":"2024-09-20T18:55:05.802685+0000","last_undegraded":"2024-09-20T18:55:05.802685+0000","last_fullsized":"2024-09-20T18:55:05.802685+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":3395,"log_dups_size":0,"ondisk_log_size":3395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:50.299673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222494720,"num_objects":3395,"num_object_clones":0,"num_object_copies":10185,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3395,"num_write_kb":217280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":260,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.682978+0000","last_change":"2024-09-20T18:51:14.397996+0000","last_active":"2024-09-20T18:55:05.682978+0000","last_peered":"2024-09-20T18:55:05.682978+0000","last_clean":"2024-09-20T18:55:05.682978+0000","last_became_active":"2024-09-20T18:49:47.950073+0000","last_became_peered":"2024-09-20T18:49:47.950073+0000","last_unstale":"2024-09-20T18:55:05.682978+0000","last_undegraded":"2024-09-20T18:55:05.682978+0000","last_fullsized":"2024-09-20T18:55:05.682978+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:51:14.397930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.397930+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:36.090967+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":245,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.682396+0000","last_change":"2024-09-20T18:51:18.410601+0000","last_active":"2024-09-20T18:55:05.682396+0000","last_peered":"2024-09-20T18:55:05.682396+0000","last_clean":"2024-09-20T18:55:05.682396+0000","last_became_active":"2024-09-20T18:49:45.944579+0000","last_became_peered":"2024-09-20T18:49:45.944579+0000","last_unstale":"2024-09-20T18:55:05.682396+0000","last_undegraded":"2024-09-20T18:55:05.682396+0000","last_fullsized":"2024-09-20T18:55:05.682396+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:18.410447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:18.410447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:30.148528+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":252,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.678532+0000","last_change":"2024-09-20T18:54:11.401857+0000","last_active":"2024-09-20T18:55:05.678532+0000","last_peered":"2024-09-20T18:55:05.678532+0000","last_clean":"2024-09-20T18:55:05.678532+0000","last_became_active":"2024-09-20T18:49:49.954391+0000","last_became_peered":"2024-09-20T18:49:49.954391+0000","last_unstale":"2024-09-20T18:55:05.678532+0000","last_undegraded":"2024-09-20T18:55:05.678532+0000","last_fullsized":"2024-09-20T18:55:05.678532+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:11.401777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:11.401777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:15.740623+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":233,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.685456+0000","last_change":"2024-09-20T18:52:05.021228+0000","last_active":"2024-09-20T18:55:05.685456+0000","last_peered":"2024-09-20T18:55:05.685456+0000","last_clean":"2024-09-20T18:55:05.685456+0000","last_became_active":"2024-09-20T18:49:52.139959+0000","last_became_peered":"2024-09-20T18:49:52.139959+0000","last_unstale":"2024-09-20T18:55:05.685456+0000","last_undegraded":"2024-09-20T18:55:05.685456+0000","last_fullsized":"2024-09-20T18:55:05.685456+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:05.021071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.021071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1","version":"171'3372","reported_seq":3404,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:06.021776+0000","last_change":"2024-09-20T18:54:34.310237+0000","last_active":"2024-09-20T18:55:06.021776+0000","last_peered":"2024-09-20T18:55:06.021776+0000","last_clean":"2024-09-20T18:55:06.021776+0000","last_became_active":"2024-09-20T18:54:34.309781+0000","last_became_peered":"2024-09-20T18:54:34.309781+0000","last_unstale":"2024-09-20T18:55:06.021776+0000","last_undegraded":"2024-09-20T18:55:06.021776+0000","last_fullsized":"2024-09-20T18:55:06.021776+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":3372,"log_dups_size":0,"ondisk_log_size":3372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:01.398759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220987392,"num_objects":3372,"num_object_clones":0,"num_object_copies":10116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3372,"num_write_kb":215808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":277,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.682811+0000","last_change":"2024-09-20T18:52:02.944536+0000","last_active":"2024-09-20T18:55:05.682811+0000","last_peered":"2024-09-20T18:55:05.682811+0000","last_clean":"2024-09-20T18:55:05.682811+0000","last_became_active":"2024-09-20T18:49:47.947636+0000","last_became_peered":"2024-09-20T18:49:47.947636+0000","last_unstale":"2024-09-20T18:55:05.682811+0000","last_undegraded":"2024-09-20T18:55:05.682811+0000","last_fullsized":"2024-09-20T18:55:05.682811+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-20T18:52:02.944434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:02.944434+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:31.181686+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":274,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.686250+0000","last_change":"2024-09-20T18:54:45.669502+0000","last_active":"2024-09-20T18:55:05.686250+0000","last_peered":"2024-09-20T18:55:05.686250+0000","last_clean":"2024-09-20T18:55:05.686250+0000","last_became_active":"2024-09-20T18:51:55.596689+0000","last_became_peered":"2024-09-20T18:51:55.596689+0000","last_unstale":"2024-09-20T18:55:05.686250+0000","last_undegraded":"2024-09-20T18:55:05.686250+0000","last_fullsized":"2024-09-20T18:55:05.686250+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:45.669427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:45.669427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:58.242177+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":245,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.690943+0000","last_change":"2024-09-20T18:54:11.377907+0000","last_active":"2024-09-20T18:55:05.690943+0000","last_peered":"2024-09-20T18:55:05.690943+0000","last_clean":"2024-09-20T18:55:05.690943+0000","last_became_active":"2024-09-20T18:49:49.959160+0000","last_became_peered":"2024-09-20T18:49:49.959160+0000","last_unstale":"2024-09-20T18:55:05.690943+0000","last_undegraded":"2024-09-20T18:55:05.690943+0000","last_fullsized":"2024-09-20T18:55:05.690943+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:11.376951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:11.376951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:35.227288+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":242,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.687255+0000","last_change":"2024-09-20T18:52:35.635897+0000","last_active":"2024-09-20T18:55:05.687255+0000","last_peered":"2024-09-20T18:55:05.687255+0000","last_clean":"2024-09-20T18:55:05.687255+0000","last_became_active":"2024-09-20T18:49:51.970108+0000","last_became_peered":"2024-09-20T18:49:51.970108+0000","last_unstale":"2024-09-20T18:55:05.687255+0000","last_undegraded":"2024-09-20T18:55:05.687255+0000","last_fullsized":"2024-09-20T18:55:05.687255+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:35.635818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:35.635818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:02.492360+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"171'3369","reported_seq":3402,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:06.285555+0000","last_change":"2024-09-20T18:54:33.929552+0000","last_active":"2024-09-20T18:55:06.285555+0000","last_peered":"2024-09-20T18:55:06.285555+0000","last_clean":"2024-09-20T18:55:06.285555+0000","last_became_active":"2024-09-20T18:54:33.929208+0000","last_became_peered":"2024-09-20T18:54:33.929208+0000","last_unstale":"2024-09-20T18:55:06.285555+0000","last_undegraded":"2024-09-20T18:55:06.285555+0000","last_fullsized":"2024-09-20T18:55:06.285555+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":3369,"log_dups_size":0,"ondisk_log_size":3369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:59.332404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220790784,"num_objects":3369,"num_object_clones":0,"num_object_copies":10107,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3369,"num_write_kb":215616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":264,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.682750+0000","last_change":"2024-09-20T18:54:17.219635+0000","last_active":"2024-09-20T18:55:05.682750+0000","last_peered":"2024-09-20T18:55:05.682750+0000","last_clean":"2024-09-20T18:55:05.682750+0000","last_became_active":"2024-09-20T18:49:47.947126+0000","last_became_peered":"2024-09-20T18:49:47.947126+0000","last_unstale":"2024-09-20T18:55:05.682750+0000","last_undegraded":"2024-09-20T18:55:05.682750+0000","last_fullsized":"2024-09-20T18:55:05.682750+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:54:17.219548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:17.219548+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:45.354404+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":253,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.686342+0000","last_change":"2024-09-20T18:52:46.005172+0000","last_active":"2024-09-20T18:55:05.686342+0000","last_peered":"2024-09-20T18:55:05.686342+0000","last_clean":"2024-09-20T18:55:05.686342+0000","last_became_active":"2024-09-20T18:49:45.937264+0000","last_became_peered":"2024-09-20T18:49:45.937264+0000","last_unstale":"2024-09-20T18:55:05.686342+0000","last_undegraded":"2024-09-20T18:55:05.686342+0000","last_fullsized":"2024-09-20T18:55:05.686342+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:46.005109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:46.005109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:32.252493+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":319,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.687261+0000","last_change":"2024-09-20T18:52:40.647833+0000","last_active":"2024-09-20T18:55:05.687261+0000","last_peered":"2024-09-20T18:55:05.687261+0000","last_clean":"2024-09-20T18:55:05.687261+0000","last_became_active":"2024-09-20T18:49:49.957005+0000","last_became_peered":"2024-09-20T18:49:49.957005+0000","last_unstale":"2024-09-20T18:55:05.687261+0000","last_undegraded":"2024-09-20T18:55:05.687261+0000","last_fullsized":"2024-09-20T18:55:05.687261+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:52:40.647761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:40.647761+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:07.843426+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":241,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.682804+0000","last_change":"2024-09-20T18:54:42.182875+0000","last_active":"2024-09-20T18:55:05.682804+0000","last_peered":"2024-09-20T18:55:05.682804+0000","last_clean":"2024-09-20T18:55:05.682804+0000","last_became_active":"2024-09-20T18:49:51.969355+0000","last_became_peered":"2024-09-20T18:49:51.969355+0000","last_unstale":"2024-09-20T18:55:05.682804+0000","last_undegraded":"2024-09-20T18:55:05.682804+0000","last_fullsized":"2024-09-20T18:55:05.682804+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:42.182809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:42.182809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:58.169396+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"171'3416","reported_seq":3449,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:06.017166+0000","last_change":"2024-09-20T18:54:33.928333+0000","last_active":"2024-09-20T18:55:06.017166+0000","last_peered":"2024-09-20T18:55:06.017166+0000","last_clean":"2024-09-20T18:55:06.017166+0000","last_became_active":"2024-09-20T18:54:33.927986+0000","last_became_peered":"2024-09-20T18:54:33.927986+0000","last_unstale":"2024-09-20T18:55:06.017166+0000","last_undegraded":"2024-09-20T18:55:06.017166+0000","last_fullsized":"2024-09-20T18:55:06.017166+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":3416,"log_dups_size":0,"ondisk_log_size":3416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:48.616895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":223870976,"num_objects":3416,"num_object_clones":0,"num_object_copies":10248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3416,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3416,"num_write_kb":218624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":272,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.690383+0000","last_change":"2024-09-20T18:54:54.897991+0000","last_active":"2024-09-20T18:55:05.690383+0000","last_peered":"2024-09-20T18:55:05.690383+0000","last_clean":"2024-09-20T18:55:05.690383+0000","last_became_active":"2024-09-20T18:49:47.946008+0000","last_became_peered":"2024-09-20T18:49:47.946008+0000","last_unstale":"2024-09-20T18:55:05.690383+0000","last_undegraded":"2024-09-20T18:55:05.690383+0000","last_fullsized":"2024-09-20T18:55:05.690383+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:54.897933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:54.897933+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:13.253681+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":260,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.678558+0000","last_change":"2024-09-20T18:54:56.212449+0000","last_active":"2024-09-20T18:55:05.678558+0000","last_peered":"2024-09-20T18:55:05.678558+0000","last_clean":"2024-09-20T18:55:05.678558+0000","last_became_active":"2024-09-20T18:49:45.941406+0000","last_became_peered":"2024-09-20T18:49:45.941406+0000","last_unstale":"2024-09-20T18:55:05.678558+0000","last_undegraded":"2024-09-20T18:55:05.678558+0000","last_fullsized":"2024-09-20T18:55:05.678558+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:56.212378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:56.212378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:57.729598+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":254,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.686844+0000","last_change":"2024-09-20T18:54:43.379898+0000","last_active":"2024-09-20T18:55:05.686844+0000","last_peered":"2024-09-20T18:55:05.686844+0000","last_clean":"2024-09-20T18:55:05.686844+0000","last_became_active":"2024-09-20T18:49:49.956767+0000","last_became_peered":"2024-09-20T18:49:49.956767+0000","last_unstale":"2024-09-20T18:55:05.686844+0000","last_undegraded":"2024-09-20T18:55:05.686844+0000","last_fullsized":"2024-09-20T18:55:05.686844+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:43.379706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:43.379706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:11.401014+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":240,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.690800+0000","last_change":"2024-09-20T18:54:24.976627+0000","last_active":"2024-09-20T18:55:05.690800+0000","last_peered":"2024-09-20T18:55:05.690800+0000","last_clean":"2024-09-20T18:55:05.690800+0000","last_became_active":"2024-09-20T18:49:51.964098+0000","last_became_peered":"2024-09-20T18:49:51.964098+0000","last_unstale":"2024-09-20T18:55:05.690800+0000","last_undegraded":"2024-09-20T18:55:05.690800+0000","last_fullsized":"2024-09-20T18:55:05.690800+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:24.976584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:24.976584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:48.697647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"171'3470","reported_seq":3502,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:06.326295+0000","last_change":"2024-09-20T18:54:34.311509+0000","last_active":"2024-09-20T18:55:06.326295+0000","last_peered":"2024-09-20T18:55:06.326295+0000","last_clean":"2024-09-20T18:55:06.326295+0000","last_became_active":"2024-09-20T18:54:34.311203+0000","last_became_peered":"2024-09-20T18:54:34.311203+0000","last_unstale":"2024-09-20T18:55:06.326295+0000","last_undegraded":"2024-09-20T18:55:06.326295+0000","last_fullsized":"2024-09-20T18:55:06.326295+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":3470,"log_dups_size":0,"ondisk_log_size":3470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:51.286355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":227409920,"num_objects":3470,"num_object_clones":0,"num_object_copies":10410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3470,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3470,"num_write_kb":222080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":279,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.686081+0000","last_change":"2024-09-20T18:53:24.458093+0000","last_active":"2024-09-20T18:55:05.686081+0000","last_peered":"2024-09-20T18:55:05.686081+0000","last_clean":"2024-09-20T18:55:05.686081+0000","last_became_active":"2024-09-20T18:49:48.498625+0000","last_became_peered":"2024-09-20T18:49:48.498625+0000","last_unstale":"2024-09-20T18:55:05.686081+0000","last_undegraded":"2024-09-20T18:55:05.686081+0000","last_fullsized":"2024-09-20T18:55:05.686081+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-20T18:53:24.458005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:24.458005+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:31.731536+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":260,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.688891+0000","last_change":"2024-09-20T18:54:44.501240+0000","last_active":"2024-09-20T18:55:05.688891+0000","last_peered":"2024-09-20T18:55:05.688891+0000","last_clean":"2024-09-20T18:55:05.688891+0000","last_became_active":"2024-09-20T18:49:45.941448+0000","last_became_peered":"2024-09-20T18:49:45.941448+0000","last_unstale":"2024-09-20T18:55:05.688891+0000","last_undegraded":"2024-09-20T18:55:05.688891+0000","last_fullsized":"2024-09-20T18:55:05.688891+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:44.501192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.501192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:49.235854+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":246,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.686580+0000","last_change":"2024-09-20T18:52:36.662703+0000","last_active":"2024-09-20T18:55:05.686580+0000","last_peered":"2024-09-20T18:55:05.686580+0000","last_clean":"2024-09-20T18:55:05.686580+0000","last_became_active":"2024-09-20T18:49:49.957558+0000","last_became_peered":"2024-09-20T18:49:49.957558+0000","last_unstale":"2024-09-20T18:55:05.686580+0000","last_undegraded":"2024-09-20T18:55:05.686580+0000","last_fullsized":"2024-09-20T18:55:05.686580+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:36.662524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:36.662524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":248,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.687761+0000","last_change":"2024-09-20T18:53:47.370152+0000","last_active":"2024-09-20T18:55:05.687761+0000","last_peered":"2024-09-20T18:55:05.687761+0000","last_clean":"2024-09-20T18:55:05.687761+0000","last_became_active":"2024-09-20T18:49:51.970798+0000","last_became_peered":"2024-09-20T18:49:51.970798+0000","last_unstale":"2024-09-20T18:55:05.687761+0000","last_undegraded":"2024-09-20T18:55:05.687761+0000","last_fullsized":"2024-09-20T18:55:05.687761+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:47.370108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:47.370108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:00.606252+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"171'3480","reported_seq":3513,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.824814+0000","last_change":"2024-09-20T18:54:34.311996+0000","last_active":"2024-09-20T18:55:05.824814+0000","last_peered":"2024-09-20T18:55:05.824814+0000","last_clean":"2024-09-20T18:55:05.824814+0000","last_became_active":"2024-09-20T18:54:34.311675+0000","last_became_peered":"2024-09-20T18:54:34.311675+0000","last_unstale":"2024-09-20T18:55:05.824814+0000","last_undegraded":"2024-09-20T18:55:05.824814+0000","last_fullsized":"2024-09-20T18:55:05.824814+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":3480,"log_dups_size":0,"ondisk_log_size":3480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:57.294768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":228065280,"num_objects":3480,"num_object_clones":0,"num_object_copies":10440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3480,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3480,"num_write_kb":222720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":272,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.682252+0000","last_change":"2024-09-20T18:53:52.898818+0000","last_active":"2024-09-20T18:55:05.682252+0000","last_peered":"2024-09-20T18:55:05.682252+0000","last_clean":"2024-09-20T18:55:05.682252+0000","last_became_active":"2024-09-20T18:49:47.949116+0000","last_became_peered":"2024-09-20T18:49:47.949116+0000","last_unstale":"2024-09-20T18:55:05.682252+0000","last_undegraded":"2024-09-20T18:55:05.682252+0000","last_fullsized":"2024-09-20T18:55:05.682252+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:53:52.898760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:52.898760+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:00.383125+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":260,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.687758+0000","last_change":"2024-09-20T18:54:52.650798+0000","last_active":"2024-09-20T18:55:05.687758+0000","last_peered":"2024-09-20T18:55:05.687758+0000","last_clean":"2024-09-20T18:55:05.687758+0000","last_became_active":"2024-09-20T18:49:45.944858+0000","last_became_peered":"2024-09-20T18:49:45.944858+0000","last_unstale":"2024-09-20T18:55:05.687758+0000","last_undegraded":"2024-09-20T18:55:05.687758+0000","last_fullsized":"2024-09-20T18:55:05.687758+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:52.650756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:52.650756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:03.257522+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":246,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.686230+0000","last_change":"2024-09-20T18:54:42.349879+0000","last_active":"2024-09-20T18:55:05.686230+0000","last_peered":"2024-09-20T18:55:05.686230+0000","last_clean":"2024-09-20T18:55:05.686230+0000","last_became_active":"2024-09-20T18:49:49.954944+0000","last_became_peered":"2024-09-20T18:49:49.954944+0000","last_unstale":"2024-09-20T18:55:05.686230+0000","last_undegraded":"2024-09-20T18:55:05.686230+0000","last_fullsized":"2024-09-20T18:55:05.686230+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:42.349807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:42.349807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:06.061918+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":233,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.686048+0000","last_change":"2024-09-20T18:52:09.017828+0000","last_active":"2024-09-20T18:55:05.686048+0000","last_peered":"2024-09-20T18:55:05.686048+0000","last_clean":"2024-09-20T18:55:05.686048+0000","last_became_active":"2024-09-20T18:49:51.972970+0000","last_became_peered":"2024-09-20T18:49:51.972970+0000","last_unstale":"2024-09-20T18:55:05.686048+0000","last_undegraded":"2024-09-20T18:55:05.686048+0000","last_fullsized":"2024-09-20T18:55:05.686048+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:09.017759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:09.017759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.4","version":"171'3380","reported_seq":3412,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.839073+0000","last_change":"2024-09-20T18:54:34.309442+0000","last_active":"2024-09-20T18:55:05.839073+0000","last_peered":"2024-09-20T18:55:05.839073+0000","last_clean":"2024-09-20T18:55:05.839073+0000","last_became_active":"2024-09-20T18:54:34.309113+0000","last_became_peered":"2024-09-20T18:54:34.309113+0000","last_unstale":"2024-09-20T18:55:05.839073+0000","last_undegraded":"2024-09-20T18:55:05.839073+0000","last_fullsized":"2024-09-20T18:55:05.839073+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":3380,"log_dups_size":0,"ondisk_log_size":3380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:57.624679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":221511680,"num_objects":3380,"num_object_clones":0,"num_object_copies":10140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3380,"num_write_kb":216320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":273,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.677922+0000","last_change":"2024-09-20T18:54:12.373389+0000","last_active":"2024-09-20T18:55:05.677922+0000","last_peered":"2024-09-20T18:55:05.677922+0000","last_clean":"2024-09-20T18:55:05.677922+0000","last_became_active":"2024-09-20T18:49:48.499847+0000","last_became_peered":"2024-09-20T18:49:48.499847+0000","last_unstale":"2024-09-20T18:55:05.677922+0000","last_undegraded":"2024-09-20T18:55:05.677922+0000","last_fullsized":"2024-09-20T18:55:05.677922+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:54:12.373309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:12.373309+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:26.708173+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":272,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.687505+0000","last_change":"2024-09-20T18:54:19.403828+0000","last_active":"2024-09-20T18:55:05.687505+0000","last_peered":"2024-09-20T18:55:05.687505+0000","last_clean":"2024-09-20T18:55:05.687505+0000","last_became_active":"2024-09-20T18:49:45.944256+0000","last_became_peered":"2024-09-20T18:49:45.944256+0000","last_unstale":"2024-09-20T18:55:05.687505+0000","last_undegraded":"2024-09-20T18:55:05.687505+0000","last_fullsized":"2024-09-20T18:55:05.687505+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-20T18:54:19.403731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:19.403731+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:21.004220+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":308,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.686593+0000","last_change":"2024-09-20T18:52:05.788988+0000","last_active":"2024-09-20T18:55:05.686593+0000","last_peered":"2024-09-20T18:55:05.686593+0000","last_clean":"2024-09-20T18:55:05.686593+0000","last_became_active":"2024-09-20T18:49:49.958411+0000","last_became_peered":"2024-09-20T18:49:49.958411+0000","last_unstale":"2024-09-20T18:55:05.686593+0000","last_undegraded":"2024-09-20T18:55:05.686593+0000","last_fullsized":"2024-09-20T18:55:05.686593+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:52:05.788924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.788924+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:40.485906+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":232,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.678049+0000","last_change":"2024-09-20T18:51:56.970415+0000","last_active":"2024-09-20T18:55:05.678049+0000","last_peered":"2024-09-20T18:55:05.678049+0000","last_clean":"2024-09-20T18:55:05.678049+0000","last_became_active":"2024-09-20T18:49:52.139685+0000","last_became_peered":"2024-09-20T18:49:52.139685+0000","last_unstale":"2024-09-20T18:55:05.678049+0000","last_undegraded":"2024-09-20T18:55:05.678049+0000","last_fullsized":"2024-09-20T18:55:05.678049+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:56.970349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:56.970349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:08.388529+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":260,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.685938+0000","last_change":"2024-09-20T18:54:18.763523+0000","last_active":"2024-09-20T18:55:05.685938+0000","last_peered":"2024-09-20T18:55:05.685938+0000","last_clean":"2024-09-20T18:55:05.685938+0000","last_became_active":"2024-09-20T18:49:45.940060+0000","last_became_peered":"2024-09-20T18:49:45.940060+0000","last_unstale":"2024-09-20T18:55:05.685938+0000","last_undegraded":"2024-09-20T18:55:05.685938+0000","last_fullsized":"2024-09-20T18:55:05.685938+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:18.763474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:18.763474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:24.982459+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":258,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.687836+0000","last_change":"2024-09-20T18:54:44.194161+0000","last_active":"2024-09-20T18:55:05.687836+0000","last_peered":"2024-09-20T18:55:05.687836+0000","last_clean":"2024-09-20T18:55:05.687836+0000","last_became_active":"2024-09-20T18:49:48.500662+0000","last_became_peered":"2024-09-20T18:49:48.500662+0000","last_unstale":"2024-09-20T18:55:05.687836+0000","last_undegraded":"2024-09-20T18:55:05.687836+0000","last_fullsized":"2024-09-20T18:55:05.687836+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-20T18:54:44.194089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.194089+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:52.901957+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":244,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.678437+0000","last_change":"2024-09-20T18:53:16.220554+0000","last_active":"2024-09-20T18:55:05.678437+0000","last_peered":"2024-09-20T18:55:05.678437+0000","last_clean":"2024-09-20T18:55:05.678437+0000","last_became_active":"2024-09-20T18:49:49.956749+0000","last_became_peered":"2024-09-20T18:49:49.956749+0000","last_unstale":"2024-09-20T18:55:05.678437+0000","last_undegraded":"2024-09-20T18:55:05.678437+0000","last_fullsized":"2024-09-20T18:55:05.678437+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:16.220490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:16.220490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:38.270912+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":248,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.686799+0000","last_change":"2024-09-20T18:54:51.637785+0000","last_active":"2024-09-20T18:55:05.686799+0000","last_peered":"2024-09-20T18:55:05.686799+0000","last_clean":"2024-09-20T18:55:05.686799+0000","last_became_active":"2024-09-20T18:49:52.142291+0000","last_became_peered":"2024-09-20T18:49:52.142291+0000","last_unstale":"2024-09-20T18:55:05.686799+0000","last_undegraded":"2024-09-20T18:55:05.686799+0000","last_fullsized":"2024-09-20T18:55:05.686799+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.637739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.637739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:08.854973+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":264,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.694786+0000","last_change":"2024-09-20T18:54:44.830872+0000","last_active":"2024-09-20T18:55:05.694786+0000","last_peered":"2024-09-20T18:55:05.694786+0000","last_clean":"2024-09-20T18:55:05.694786+0000","last_became_active":"2024-09-20T18:49:48.500803+0000","last_became_peered":"2024-09-20T18:49:48.500803+0000","last_unstale":"2024-09-20T18:55:05.694786+0000","last_undegraded":"2024-09-20T18:55:05.694786+0000","last_fullsized":"2024-09-20T18:55:05.694786+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:44.830790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.830790+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:07.369453+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":244,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.686797+0000","last_change":"2024-09-20T18:51:30.982535+0000","last_active":"2024-09-20T18:55:05.686797+0000","last_peered":"2024-09-20T18:55:05.686797+0000","last_clean":"2024-09-20T18:55:05.686797+0000","last_became_active":"2024-09-20T18:49:45.946600+0000","last_became_peered":"2024-09-20T18:49:45.946600+0000","last_unstale":"2024-09-20T18:55:05.686797+0000","last_undegraded":"2024-09-20T18:55:05.686797+0000","last_fullsized":"2024-09-20T18:55:05.686797+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:30.982454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:30.982454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":245,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.687700+0000","last_change":"2024-09-20T18:53:50.253721+0000","last_active":"2024-09-20T18:55:05.687700+0000","last_peered":"2024-09-20T18:55:05.687700+0000","last_clean":"2024-09-20T18:55:05.687700+0000","last_became_active":"2024-09-20T18:49:49.961947+0000","last_became_peered":"2024-09-20T18:49:49.961947+0000","last_unstale":"2024-09-20T18:55:05.687700+0000","last_undegraded":"2024-09-20T18:55:05.687700+0000","last_fullsized":"2024-09-20T18:55:05.687700+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:50.253660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:50.253660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:15.284040+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"66'1","reported_seq":251,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.687910+0000","last_change":"2024-09-20T18:54:20.681878+0000","last_active":"2024-09-20T18:55:05.687910+0000","last_peered":"2024-09-20T18:55:05.687910+0000","last_clean":"2024-09-20T18:55:05.687910+0000","last_became_active":"2024-09-20T18:49:51.965121+0000","last_became_peered":"2024-09-20T18:49:51.965121+0000","last_unstale":"2024-09-20T18:55:05.687910+0000","last_undegraded":"2024-09-20T18:55:05.687910+0000","last_fullsized":"2024-09-20T18:55:05.687910+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T18:54:20.681790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:20.681790+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:31.186086+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":248,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.678626+0000","last_change":"2024-09-20T18:49:48.501359+0000","last_active":"2024-09-20T18:55:05.678626+0000","last_peered":"2024-09-20T18:55:05.678626+0000","last_clean":"2024-09-20T18:55:05.678626+0000","last_became_active":"2024-09-20T18:49:48.501011+0000","last_became_peered":"2024-09-20T18:49:48.501011+0000","last_unstale":"2024-09-20T18:55:05.678626+0000","last_undegraded":"2024-09-20T18:55:05.678626+0000","last_fullsized":"2024-09-20T18:55:05.678626+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":252,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.690464+0000","last_change":"2024-09-20T18:54:11.896547+0000","last_active":"2024-09-20T18:55:05.690464+0000","last_peered":"2024-09-20T18:55:05.690464+0000","last_clean":"2024-09-20T18:55:05.690464+0000","last_became_active":"2024-09-20T18:49:45.938511+0000","last_became_peered":"2024-09-20T18:49:45.938511+0000","last_unstale":"2024-09-20T18:55:05.690464+0000","last_undegraded":"2024-09-20T18:55:05.690464+0000","last_fullsized":"2024-09-20T18:55:05.690464+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:11.896470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:11.896470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:22.211500+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":318,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.686735+0000","last_change":"2024-09-20T18:53:47.337912+0000","last_active":"2024-09-20T18:55:05.686735+0000","last_peered":"2024-09-20T18:55:05.686735+0000","last_clean":"2024-09-20T18:55:05.686735+0000","last_became_active":"2024-09-20T18:49:49.960550+0000","last_became_peered":"2024-09-20T18:49:49.960550+0000","last_unstale":"2024-09-20T18:55:05.686735+0000","last_undegraded":"2024-09-20T18:55:05.686735+0000","last_fullsized":"2024-09-20T18:55:05.686735+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:53:47.337658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:47.337658+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":241,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.683021+0000","last_change":"2024-09-20T18:53:48.839130+0000","last_active":"2024-09-20T18:55:05.683021+0000","last_peered":"2024-09-20T18:55:05.683021+0000","last_clean":"2024-09-20T18:55:05.683021+0000","last_became_active":"2024-09-20T18:49:51.966254+0000","last_became_peered":"2024-09-20T18:49:51.966254+0000","last_unstale":"2024-09-20T18:55:05.683021+0000","last_undegraded":"2024-09-20T18:55:05.683021+0000","last_fullsized":"2024-09-20T18:55:05.683021+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:48.839015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:48.839015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:07.481453+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":275,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.686885+0000","last_change":"2024-09-20T18:51:15.223913+0000","last_active":"2024-09-20T18:55:05.686885+0000","last_peered":"2024-09-20T18:55:05.686885+0000","last_clean":"2024-09-20T18:55:05.686885+0000","last_became_active":"2024-09-20T18:49:58.997494+0000","last_became_peered":"2024-09-20T18:49:58.997494+0000","last_unstale":"2024-09-20T18:55:05.686885+0000","last_undegraded":"2024-09-20T18:55:05.686885+0000","last_fullsized":"2024-09-20T18:55:05.686885+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:51:15.223826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:15.223826+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:07.187139+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":260,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.687073+0000","last_change":"2024-09-20T18:54:57.632458+0000","last_active":"2024-09-20T18:55:05.687073+0000","last_peered":"2024-09-20T18:55:05.687073+0000","last_clean":"2024-09-20T18:55:05.687073+0000","last_became_active":"2024-09-20T18:49:45.949492+0000","last_became_peered":"2024-09-20T18:49:45.949492+0000","last_unstale":"2024-09-20T18:55:05.687073+0000","last_undegraded":"2024-09-20T18:55:05.687073+0000","last_fullsized":"2024-09-20T18:55:05.687073+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.632380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.632380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:58.895820+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":316,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.678372+0000","last_change":"2024-09-20T18:54:17.339848+0000","last_active":"2024-09-20T18:55:05.678372+0000","last_peered":"2024-09-20T18:55:05.678372+0000","last_clean":"2024-09-20T18:55:05.678372+0000","last_became_active":"2024-09-20T18:49:49.962243+0000","last_became_peered":"2024-09-20T18:49:49.962243+0000","last_unstale":"2024-09-20T18:55:05.678372+0000","last_undegraded":"2024-09-20T18:55:05.678372+0000","last_fullsized":"2024-09-20T18:55:05.678372+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:54:17.339761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:17.339761+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:22.040146+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":240,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.694348+0000","last_change":"2024-09-20T18:53:13.863129+0000","last_active":"2024-09-20T18:55:05.694348+0000","last_peered":"2024-09-20T18:55:05.694348+0000","last_clean":"2024-09-20T18:55:05.694348+0000","last_became_active":"2024-09-20T18:49:51.969093+0000","last_became_peered":"2024-09-20T18:49:51.969093+0000","last_unstale":"2024-09-20T18:55:05.694348+0000","last_undegraded":"2024-09-20T18:55:05.694348+0000","last_fullsized":"2024-09-20T18:55:05.694348+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:13.863061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:13.863061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":268,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.681979+0000","last_change":"2024-09-20T18:53:43.879001+0000","last_active":"2024-09-20T18:55:05.681979+0000","last_peered":"2024-09-20T18:55:05.681979+0000","last_clean":"2024-09-20T18:55:05.681979+0000","last_became_active":"2024-09-20T18:49:47.949703+0000","last_became_peered":"2024-09-20T18:49:47.949703+0000","last_unstale":"2024-09-20T18:55:05.681979+0000","last_undegraded":"2024-09-20T18:55:05.681979+0000","last_fullsized":"2024-09-20T18:55:05.681979+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:53:43.878921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:43.878921+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":245,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.682888+0000","last_change":"2024-09-20T18:52:05.985622+0000","last_active":"2024-09-20T18:55:05.682888+0000","last_peered":"2024-09-20T18:55:05.682888+0000","last_clean":"2024-09-20T18:55:05.682888+0000","last_became_active":"2024-09-20T18:49:45.939949+0000","last_became_peered":"2024-09-20T18:49:45.939949+0000","last_unstale":"2024-09-20T18:55:05.682888+0000","last_undegraded":"2024-09-20T18:55:05.682888+0000","last_fullsized":"2024-09-20T18:55:05.682888+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:05.985551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.985551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:40.169255+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":237,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.688242+0000","last_change":"2024-09-20T18:51:14.271954+0000","last_active":"2024-09-20T18:55:05.688242+0000","last_peered":"2024-09-20T18:55:05.688242+0000","last_clean":"2024-09-20T18:55:05.688242+0000","last_became_active":"2024-09-20T18:49:49.957067+0000","last_became_peered":"2024-09-20T18:49:49.957067+0000","last_unstale":"2024-09-20T18:55:05.688242+0000","last_undegraded":"2024-09-20T18:55:05.688242+0000","last_fullsized":"2024-09-20T18:55:05.688242+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:14.271719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.271719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:34.371697+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":233,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.685457+0000","last_change":"2024-09-20T18:51:14.210438+0000","last_active":"2024-09-20T18:55:05.685457+0000","last_peered":"2024-09-20T18:55:05.685457+0000","last_clean":"2024-09-20T18:55:05.685457+0000","last_became_active":"2024-09-20T18:49:51.966915+0000","last_became_peered":"2024-09-20T18:49:51.966915+0000","last_unstale":"2024-09-20T18:55:05.685457+0000","last_undegraded":"2024-09-20T18:55:05.685457+0000","last_fullsized":"2024-09-20T18:55:05.685457+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:14.210358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.210358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:04.317164+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":273,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.689065+0000","last_change":"2024-09-20T18:53:53.264294+0000","last_active":"2024-09-20T18:55:05.689065+0000","last_peered":"2024-09-20T18:55:05.689065+0000","last_clean":"2024-09-20T18:55:05.689065+0000","last_became_active":"2024-09-20T18:49:47.953310+0000","last_became_peered":"2024-09-20T18:49:47.953310+0000","last_unstale":"2024-09-20T18:55:05.689065+0000","last_undegraded":"2024-09-20T18:55:05.689065+0000","last_fullsized":"2024-09-20T18:55:05.689065+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:53:53.263976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:53.263976+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":252,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.688763+0000","last_change":"2024-09-20T18:54:42.487465+0000","last_active":"2024-09-20T18:55:05.688763+0000","last_peered":"2024-09-20T18:55:05.688763+0000","last_clean":"2024-09-20T18:55:05.688763+0000","last_became_active":"2024-09-20T18:49:45.944357+0000","last_became_peered":"2024-09-20T18:49:45.944357+0000","last_unstale":"2024-09-20T18:55:05.688763+0000","last_undegraded":"2024-09-20T18:55:05.688763+0000","last_fullsized":"2024-09-20T18:55:05.688763+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:42.487403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:42.487403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:04.597583+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":245,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.686133+0000","last_change":"2024-09-20T18:54:18.108239+0000","last_active":"2024-09-20T18:55:05.686133+0000","last_peered":"2024-09-20T18:55:05.686133+0000","last_clean":"2024-09-20T18:55:05.686133+0000","last_became_active":"2024-09-20T18:49:49.962523+0000","last_became_peered":"2024-09-20T18:49:49.962523+0000","last_unstale":"2024-09-20T18:55:05.686133+0000","last_undegraded":"2024-09-20T18:55:05.686133+0000","last_fullsized":"2024-09-20T18:55:05.686133+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:18.108175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:18.108175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:33.189789+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":248,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.689458+0000","last_change":"2024-09-20T18:54:26.882441+0000","last_active":"2024-09-20T18:55:05.689458+0000","last_peered":"2024-09-20T18:55:05.689458+0000","last_clean":"2024-09-20T18:55:05.689458+0000","last_became_active":"2024-09-20T18:49:51.967313+0000","last_became_peered":"2024-09-20T18:49:51.967313+0000","last_unstale":"2024-09-20T18:55:05.689458+0000","last_undegraded":"2024-09-20T18:55:05.689458+0000","last_fullsized":"2024-09-20T18:55:05.689458+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:26.882375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:26.882375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:41.905086+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":268,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.685956+0000","last_change":"2024-09-20T18:54:21.106745+0000","last_active":"2024-09-20T18:55:05.685956+0000","last_peered":"2024-09-20T18:55:05.685956+0000","last_clean":"2024-09-20T18:55:05.685956+0000","last_became_active":"2024-09-20T18:49:48.499111+0000","last_became_peered":"2024-09-20T18:49:48.499111+0000","last_unstale":"2024-09-20T18:55:05.685956+0000","last_undegraded":"2024-09-20T18:55:05.685956+0000","last_fullsized":"2024-09-20T18:55:05.685956+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:54:21.106677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:21.106677+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:38.509931+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":252,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.678467+0000","last_change":"2024-09-20T18:53:48.301845+0000","last_active":"2024-09-20T18:55:05.678467+0000","last_peered":"2024-09-20T18:55:05.678467+0000","last_clean":"2024-09-20T18:55:05.678467+0000","last_became_active":"2024-09-20T18:49:45.942105+0000","last_became_peered":"2024-09-20T18:49:45.942105+0000","last_unstale":"2024-09-20T18:55:05.678467+0000","last_undegraded":"2024-09-20T18:55:05.678467+0000","last_fullsized":"2024-09-20T18:55:05.678467+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:48.301801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:48.301801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:06.232022+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":252,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.687054+0000","last_change":"2024-09-20T18:53:46.329643+0000","last_active":"2024-09-20T18:55:05.687054+0000","last_peered":"2024-09-20T18:55:05.687054+0000","last_clean":"2024-09-20T18:55:05.687054+0000","last_became_active":"2024-09-20T18:49:49.958159+0000","last_became_peered":"2024-09-20T18:49:49.958159+0000","last_unstale":"2024-09-20T18:55:05.687054+0000","last_undegraded":"2024-09-20T18:55:05.687054+0000","last_fullsized":"2024-09-20T18:55:05.687054+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:46.329547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:46.329547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:09.181263+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":233,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.686261+0000","last_change":"2024-09-20T18:52:43.118886+0000","last_active":"2024-09-20T18:55:05.686261+0000","last_peered":"2024-09-20T18:55:05.686261+0000","last_clean":"2024-09-20T18:55:05.686261+0000","last_became_active":"2024-09-20T18:49:52.139978+0000","last_became_peered":"2024-09-20T18:49:52.139978+0000","last_unstale":"2024-09-20T18:55:05.686261+0000","last_undegraded":"2024-09-20T18:55:05.686261+0000","last_fullsized":"2024-09-20T18:55:05.686261+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:43.118805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:43.118805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:38.224953+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":255,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.686081+0000","last_change":"2024-09-20T18:53:43.253204+0000","last_active":"2024-09-20T18:55:05.686081+0000","last_peered":"2024-09-20T18:55:05.686081+0000","last_clean":"2024-09-20T18:55:05.686081+0000","last_became_active":"2024-09-20T18:49:47.954777+0000","last_became_peered":"2024-09-20T18:49:47.954777+0000","last_unstale":"2024-09-20T18:55:05.686081+0000","last_undegraded":"2024-09-20T18:55:05.686081+0000","last_fullsized":"2024-09-20T18:55:05.686081+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-20T18:53:43.253132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:43.253132+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":260,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.693391+0000","last_change":"2024-09-20T18:54:55.900487+0000","last_active":"2024-09-20T18:55:05.693391+0000","last_peered":"2024-09-20T18:55:05.693391+0000","last_clean":"2024-09-20T18:55:05.693391+0000","last_became_active":"2024-09-20T18:49:45.940988+0000","last_became_peered":"2024-09-20T18:49:45.940988+0000","last_unstale":"2024-09-20T18:55:05.693391+0000","last_undegraded":"2024-09-20T18:55:05.693391+0000","last_fullsized":"2024-09-20T18:55:05.693391+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:55.899611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:55.899611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:09.905336+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":252,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.686589+0000","last_change":"2024-09-20T18:53:52.635213+0000","last_active":"2024-09-20T18:55:05.686589+0000","last_peered":"2024-09-20T18:55:05.686589+0000","last_clean":"2024-09-20T18:55:05.686589+0000","last_became_active":"2024-09-20T18:49:49.954502+0000","last_became_peered":"2024-09-20T18:49:49.954502+0000","last_unstale":"2024-09-20T18:55:05.686589+0000","last_undegraded":"2024-09-20T18:55:05.686589+0000","last_fullsized":"2024-09-20T18:55:05.686589+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:52.635138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:52.635138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:08.278673+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":232,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.686435+0000","last_change":"2024-09-20T18:51:09.761331+0000","last_active":"2024-09-20T18:55:05.686435+0000","last_peered":"2024-09-20T18:55:05.686435+0000","last_clean":"2024-09-20T18:55:05.686435+0000","last_became_active":"2024-09-20T18:49:51.968662+0000","last_became_peered":"2024-09-20T18:49:51.968662+0000","last_unstale":"2024-09-20T18:55:05.686435+0000","last_undegraded":"2024-09-20T18:55:05.686435+0000","last_fullsized":"2024-09-20T18:55:05.686435+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:09.761283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:09.761283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:33.490708+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":225,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.688077+0000","last_change":"2024-09-20T18:49:51.968476+0000","last_active":"2024-09-20T18:55:05.688077+0000","last_peered":"2024-09-20T18:55:05.688077+0000","last_clean":"2024-09-20T18:55:05.688077+0000","last_became_active":"2024-09-20T18:49:51.968216+0000","last_became_peered":"2024-09-20T18:49:51.968216+0000","last_unstale":"2024-09-20T18:55:05.688077+0000","last_undegraded":"2024-09-20T18:55:05.688077+0000","last_fullsized":"2024-09-20T18:55:05.688077+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:00.509304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":245,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.686342+0000","last_change":"2024-09-20T18:52:41.106558+0000","last_active":"2024-09-20T18:55:05.686342+0000","last_peered":"2024-09-20T18:55:05.686342+0000","last_clean":"2024-09-20T18:55:05.686342+0000","last_became_active":"2024-09-20T18:49:45.937992+0000","last_became_peered":"2024-09-20T18:49:45.937992+0000","last_unstale":"2024-09-20T18:55:05.686342+0000","last_undegraded":"2024-09-20T18:55:05.686342+0000","last_fullsized":"2024-09-20T18:55:05.686342+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:41.106332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:41.106332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:28.211547+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":267,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.688805+0000","last_change":"2024-09-20T18:54:44.719730+0000","last_active":"2024-09-20T18:55:05.688805+0000","last_peered":"2024-09-20T18:55:05.688805+0000","last_clean":"2024-09-20T18:55:05.688805+0000","last_became_active":"2024-09-20T18:49:47.949582+0000","last_became_peered":"2024-09-20T18:49:47.949582+0000","last_unstale":"2024-09-20T18:55:05.688805+0000","last_undegraded":"2024-09-20T18:55:05.688805+0000","last_fullsized":"2024-09-20T18:55:05.688805+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:54:44.719676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.719676+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:06.922997+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":236,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-20T18:55:05.694557+0000","last_change":"2024-09-20T18:51:28.120100+0000","last_active":"2024-09-20T18:55:05.694557+0000","last_peered":"2024-09-20T18:55:05.694557+0000","last_clean":"2024-09-20T18:55:05.694557+0000","last_became_active":"2024-09-20T18:49:49.957582+0000","last_became_peered":"2024-09-20T18:49:49.957582+0000","last_unstale":"2024-09-20T18:55:05.694557+0000","last_undegraded":"2024-09-20T18:55:05.694557+0000","last_fullsized":"2024-09-20T18:55:05.694557+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:28.119694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:28.119694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:34.833464+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":3581804544,"num_objects":54654,"num_object_clones":0,"num_object_copies":163962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54654,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":54654,"num_write_kb":3497856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":10729422848,"data_stored":10729422848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54654,"ondisk_log_size":54654,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":315,"num_read_kb":310,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266756,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1232304,"kb_used_data":1084508,"kb_used_omap":1,"kb_used_meta":147774,"kb_avail":92512848,"statfs":{"total":95995035648,"available":94733156352,"internally_reserved":0,"allocated":1110536192,"data_stored":1110104458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":151321034},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528395,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1224268,"kb_used_data":1084796,"kb_used_omap":1,"kb_used_meta":139454,"kb_avail":92520884,"statfs":{"total":95995035648,"available":94741385216,"internally_reserved":0,"allocated":1110831104,"data_stored":1110424652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":142801350},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757328,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1842316,"kb_used_data":1694524,"kb_used_omap":1,"kb_used_meta":147774,"kb_avail":91902836,"statfs":{"total":95995035648,"available":94108504064,"internally_reserved":0,"allocated":1735192576,"data_stored":1734785939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151321029},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986261,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1219180,"kb_used_data":1075548,"kb_used_omap":1,"kb_used_meta":143614,"kb_avail":92525972,"statfs":{"total":95995035648,"available":94746595328,"internally_reserved":0,"allocated":1101361152,"data_stored":1100922999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":147061189},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182492,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1509672,"kb_used_data":1284120,"kb_used_omap":1,"kb_used_meta":225534,"kb_avail":92235480,"statfs":{"total":95995035648,"available":94449131520,"internally_reserved":0,"allocated":1314938880,"data_stored":1314503667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":230947271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411425,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1450832,"kb_used_data":1293312,"kb_used_omap":1,"kb_used_meta":157502,"kb_avail":92294320,"statfs":{"total":95995035648,"available":94509383680,"internally_reserved":0,"allocated":1324351488,"data_stored":1323941334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":161282501},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574951,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1675456,"kb_used_data":1509616,"kb_used_omap":1,"kb_used_meta":165822,"kb_avail":92069696,"statfs":{"total":95995035648,"available":94279368704,"internally_reserved":0,"allocated":1545846784,"data_stored":1545388662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":169802181},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,3,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738477,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1557744,"kb_used_data":1305820,"kb_used_omap":1,"kb_used_meta":251902,"kb_avail":92187408,"statfs":{"total":95995035648,"available":94399905792,"internally_reserved":0,"allocated":1337159680,"data_stored":1336727642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":257948101},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1350238208,"data_stored":1350238208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1564868608,"data_stored":1564868608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1344012288,"data_stored":1344012288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1352794112,"data_stored":1352794112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1114767360,"data_stored":1114767360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1773273088,"data_stored":1773273088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1116209152,"data_stored":1116209152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1113260032,"data_stored":1113260032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T18:55:10.197 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd pool set .mgr pgp_num 1 2024-09-20T18:55:10.204 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:10.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:10.205 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:10.307 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:55:10.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:10.355 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:10.355 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:55:10.388 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T18:55:10.456 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:10.487 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:10.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:10.488 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:10.569 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:55:10.589 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:55:10.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:10.644 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:10.644 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:55:10.695 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:55:10.745 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:55:10.807 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:10.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:10.808 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:55:10.909 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:10.943 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:10.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:10.944 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:11.027 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:55:10 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:55:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:55:11.045 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:11.083 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:11.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:11.084 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:11.141 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T18:55:11.141 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:55:11.172 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 31 16 61986 61970 124.865 73.625 0.0124628 0.00800406 2024-09-20T18:55:11.185 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:55:11.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:11.212 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:11.212 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:55:11.313 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:11.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:11.342 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:11.342 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:11.346 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:55:11.443 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:11.471 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:11.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:11.472 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:11.572 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:55:11.610 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:11.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:11.611 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:55:11.696 INFO:teuthology.orchestra.run.smithi083.stderr:set pool 1 pgp_num to 1 2024-09-20T18:55:11.701 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-20T18:55:11.713 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:11.730 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:11 smithi083 ceph-mon[29421]: osdmap e173: 8 total, 8 up, 8 in 2024-09-20T18:55:11.731 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:11 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3252107696' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:55:11.731 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:11 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:55:11.731 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:11 smithi083 ceph-mon[36260]: osdmap e173: 8 total, 8 up, 8 in 2024-09-20T18:55:11.731 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:11 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3252107696' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:55:11.731 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:11 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:55:11.742 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:11.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:11.743 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:11.844 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:11.871 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:11.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:11.872 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:11.973 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:11.974 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:55:11.974 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:55:12.003 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:12.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:12.005 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:12.010 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:55:12.011 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:55:12.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:11 smithi142 ceph-mon[32508]: osdmap e173: 8 total, 8 up, 8 in 2024-09-20T18:55:12.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:11 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3252107696' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:55:12.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:11 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:55:12.045 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:55:12.045 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:55:12.078 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:55:12.078 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:55:12.105 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:55:12.113 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:55:12.113 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:55:12.143 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:12.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:12.144 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:55:12.147 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:55:12.147 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:55:12.172 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 32 16 63315 63299 123.559 83.0625 0.0130806 0.0080877 2024-09-20T18:55:12.224 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:55:12.225 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:55:12.245 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:55:12.260 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:55:12.260 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:55:12.284 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:12.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:12.285 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:55:12.385 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:12.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:12.422 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:12.422 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:12.523 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:12.552 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:12.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:12.553 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:12.654 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:12.684 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:12.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:12.685 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:12.786 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:12.828 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:12.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:12.829 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:12.930 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:55:12.967 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:12.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:12.968 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:55:13.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:12 smithi142 ceph-mon[32508]: pgmap v250: 148 pgs: 148 active+clean; 3.4 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1.7 KiB/s rd, 103 MiB/s wr, 1.64k op/s 2024-09-20T18:55:13.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:12 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/622903200' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"} : dispatch 2024-09-20T18:55:13.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:12 smithi142 ceph-mon[32508]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:55:13.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:12 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T18:55:13.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:12 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/622903200' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"}]': finished 2024-09-20T18:55:13.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:12 smithi142 ceph-mon[32508]: osdmap e174: 8 total, 8 up, 8 in 2024-09-20T18:55:13.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:12 smithi142 ceph-mon[32508]: 6.16 scrub starts 2024-09-20T18:55:13.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:12 smithi142 ceph-mon[32508]: 6.16 scrub ok 2024-09-20T18:55:13.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:12 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2910947585' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:55:13.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:12 smithi142 ceph-mon[32508]: 4.15 scrub starts 2024-09-20T18:55:13.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:12 smithi142 ceph-mon[32508]: 4.15 scrub ok 2024-09-20T18:55:13.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:12 smithi142 ceph-mon[32508]: 6.17 scrub starts 2024-09-20T18:55:13.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:12 smithi142 ceph-mon[32508]: 6.17 scrub ok 2024-09-20T18:55:13.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:12 smithi083 ceph-mon[29421]: pgmap v250: 148 pgs: 148 active+clean; 3.4 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1.7 KiB/s rd, 103 MiB/s wr, 1.64k op/s 2024-09-20T18:55:13.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:12 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/622903200' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"} : dispatch 2024-09-20T18:55:13.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:12 smithi083 ceph-mon[29421]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:55:13.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:12 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T18:55:13.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:12 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/622903200' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"}]': finished 2024-09-20T18:55:13.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:12 smithi083 ceph-mon[29421]: osdmap e174: 8 total, 8 up, 8 in 2024-09-20T18:55:13.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:12 smithi083 ceph-mon[29421]: 6.16 scrub starts 2024-09-20T18:55:13.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:12 smithi083 ceph-mon[29421]: 6.16 scrub ok 2024-09-20T18:55:13.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:12 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2910947585' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:55:13.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:12 smithi083 ceph-mon[29421]: 4.15 scrub starts 2024-09-20T18:55:13.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:12 smithi083 ceph-mon[29421]: 4.15 scrub ok 2024-09-20T18:55:13.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:12 smithi083 ceph-mon[29421]: 6.17 scrub starts 2024-09-20T18:55:13.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:12 smithi083 ceph-mon[29421]: 6.17 scrub ok 2024-09-20T18:55:13.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:12 smithi083 ceph-mon[36260]: pgmap v250: 148 pgs: 148 active+clean; 3.4 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1.7 KiB/s rd, 103 MiB/s wr, 1.64k op/s 2024-09-20T18:55:13.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:12 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/622903200' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"} : dispatch 2024-09-20T18:55:13.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:12 smithi083 ceph-mon[36260]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:55:13.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:12 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T18:55:13.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:12 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/622903200' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"}]': finished 2024-09-20T18:55:13.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:12 smithi083 ceph-mon[36260]: osdmap e174: 8 total, 8 up, 8 in 2024-09-20T18:55:13.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:12 smithi083 ceph-mon[36260]: 6.16 scrub starts 2024-09-20T18:55:13.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:12 smithi083 ceph-mon[36260]: 6.16 scrub ok 2024-09-20T18:55:13.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:12 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2910947585' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:55:13.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:12 smithi083 ceph-mon[36260]: 4.15 scrub starts 2024-09-20T18:55:13.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:12 smithi083 ceph-mon[36260]: 4.15 scrub ok 2024-09-20T18:55:13.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:12 smithi083 ceph-mon[36260]: 6.17 scrub starts 2024-09-20T18:55:13.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:12 smithi083 ceph-mon[36260]: 6.17 scrub ok 2024-09-20T18:55:13.069 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:13.107 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:13.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:13.108 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:13.172 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 33 16 64736 64720 122.506 88.8125 0.00828367 0.00815767 2024-09-20T18:55:13.210 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:13.240 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:13.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:13.241 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:13.342 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:13.372 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:13.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:13.372 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:13.473 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:13.505 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:13.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:13.506 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:13.607 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:13.643 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:13.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:13.644 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:13.745 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:13.775 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:13.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:13.776 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:13.850 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_historic_ops 2024-09-20T18:55:13.877 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:13.911 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:13.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:13.912 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:14.013 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:14.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:14.050 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:14.050 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:14.151 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:14.174 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 34 16 66215 66199 121.619 92.4375 0.0069143 0.00821764 2024-09-20T18:55:14.184 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:14.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:14.185 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:14.269 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:55:14.285 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:14.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:14.332 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:14.333 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:14.433 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:14.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:14.455 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:14.455 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:14.555 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:14.589 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:14.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:14.591 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:14.692 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:55:14.727 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:14.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:14.729 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:55:14.829 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:14.859 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:14.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:14.860 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:14.929 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:14 smithi142 ceph-mon[32508]: pgmap v252: 148 pgs: 148 active+clean; 3.8 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 853 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-20T18:55:14.929 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:14 smithi142 ceph-mon[32508]: 5.b scrub starts 2024-09-20T18:55:14.929 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:14 smithi142 ceph-mon[32508]: 5.b scrub ok 2024-09-20T18:55:14.929 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:14 smithi142 ceph-mon[32508]: 5.5 scrub starts 2024-09-20T18:55:14.929 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:14 smithi142 ceph-mon[32508]: 5.5 scrub ok 2024-09-20T18:55:14.929 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:14 smithi142 ceph-mon[32508]: 6.a scrub starts 2024-09-20T18:55:14.929 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:14 smithi142 ceph-mon[32508]: 6.a scrub ok 2024-09-20T18:55:14.961 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:14.986 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:14.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:14.987 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:15.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:14 smithi083 ceph-mon[29421]: pgmap v252: 148 pgs: 148 active+clean; 3.8 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 853 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-20T18:55:15.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:14 smithi083 ceph-mon[29421]: 5.b scrub starts 2024-09-20T18:55:15.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:14 smithi083 ceph-mon[29421]: 5.b scrub ok 2024-09-20T18:55:15.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:14 smithi083 ceph-mon[29421]: 5.5 scrub starts 2024-09-20T18:55:15.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:14 smithi083 ceph-mon[29421]: 5.5 scrub ok 2024-09-20T18:55:15.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:14 smithi083 ceph-mon[29421]: 6.a scrub starts 2024-09-20T18:55:15.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:14 smithi083 ceph-mon[29421]: 6.a scrub ok 2024-09-20T18:55:15.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:14 smithi083 ceph-mon[36260]: pgmap v252: 148 pgs: 148 active+clean; 3.8 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 853 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-20T18:55:15.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:14 smithi083 ceph-mon[36260]: 5.b scrub starts 2024-09-20T18:55:15.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:14 smithi083 ceph-mon[36260]: 5.b scrub ok 2024-09-20T18:55:15.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:14 smithi083 ceph-mon[36260]: 5.5 scrub starts 2024-09-20T18:55:15.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:14 smithi083 ceph-mon[36260]: 5.5 scrub ok 2024-09-20T18:55:15.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:14 smithi083 ceph-mon[36260]: 6.a scrub starts 2024-09-20T18:55:15.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:14 smithi083 ceph-mon[36260]: 6.a scrub ok 2024-09-20T18:55:15.088 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:15.124 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:15.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:15.125 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:15.173 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 35 16 67616 67600 120.646 87.5625 0.00858833 0.00828355 2024-09-20T18:55:15.200 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T18:55:15.200 INFO:teuthology.orchestra.run.smithi142.stdout: "size": 20, 2024-09-20T18:55:15.200 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 600, 2024-09-20T18:55:15.200 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-09-20T18:55:15.200 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.200 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:25342 7.5 7:ae6f0d57:::benchmark_data_smithi083_70643_object25341:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:55:15.201 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:07.269377+0000", 2024-09-20T18:55:15.201 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 247.91332281800001, 2024-09-20T18:55:15.201 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.17285128899999999, 2024-09-20T18:55:15.201 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:55:15.201 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:55:15.201 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:15.201 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:55:15.201 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:55:15.201 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:15.201 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 25342 2024-09-20T18:55:15.201 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.201 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:55:15.202 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.202 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:55:15.202 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.269377+0000", 2024-09-20T18:55:15.202 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:15.202 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.202 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.202 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:55:15.202 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.269377+0000", 2024-09-20T18:55:15.202 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:15.202 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.202 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.202 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:55:15.202 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.269378+0000", 2024-09-20T18:55:15.203 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.615e-06 2024-09-20T18:55:15.203 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.203 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.203 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:55:15.203 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.269401+0000", 2024-09-20T18:55:15.203 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3159e-05 2024-09-20T18:55:15.203 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.203 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.203 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:55:15.203 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.269417+0000", 2024-09-20T18:55:15.203 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5760000000000002e-05 2024-09-20T18:55:15.203 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.203 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.204 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:55:15.204 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.269425+0000", 2024-09-20T18:55:15.204 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.6190000000000002e-06 2024-09-20T18:55:15.204 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.204 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.204 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:55:15.204 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.269487+0000", 2024-09-20T18:55:15.204 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2053999999999999e-05 2024-09-20T18:55:15.204 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.204 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.204 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:55:15.204 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.269633+0000", 2024-09-20T18:55:15.204 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00014581899999999999 2024-09-20T18:55:15.205 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.205 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.205 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,2", 2024-09-20T18:55:15.205 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.269777+0000", 2024-09-20T18:55:15.205 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00014390599999999999 2024-09-20T18:55:15.205 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.205 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.205 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:55:15.205 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.270815+0000", 2024-09-20T18:55:15.205 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0010380400000000001 2024-09-20T18:55:15.205 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.205 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.205 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:15.205 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.272683+0000", 2024-09-20T18:55:15.206 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0018686810000000001 2024-09-20T18:55:15.206 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.206 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.206 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:15.206 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.442115+0000", 2024-09-20T18:55:15.206 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.16943219800000001 2024-09-20T18:55:15.206 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.206 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.206 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:55:15.206 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.442185+0000", 2024-09-20T18:55:15.206 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.9690999999999994e-05 2024-09-20T18:55:15.206 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.206 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.207 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:55:15.207 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.442228+0000", 2024-09-20T18:55:15.207 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.2747000000000003e-05 2024-09-20T18:55:15.207 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:15.207 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:55:15.207 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:15.207 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.207 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.207 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:27567 7.5 7:a34bb3a0:::benchmark_data_smithi083_70643_object27566:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:55:15.207 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:08.739688+0000", 2024-09-20T18:55:15.207 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 246.44301120899999, 2024-09-20T18:55:15.207 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.105546902, 2024-09-20T18:55:15.207 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:55:15.207 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:55:15.208 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:15.208 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:55:15.208 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:55:15.208 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:15.208 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 27567 2024-09-20T18:55:15.208 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.208 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:55:15.208 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.208 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:55:15.208 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.739688+0000", 2024-09-20T18:55:15.208 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:15.208 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.208 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.208 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:55:15.209 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.739688+0000", 2024-09-20T18:55:15.209 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:15.209 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.209 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.209 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:55:15.209 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.739690+0000", 2024-09-20T18:55:15.209 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0499999999999999e-06 2024-09-20T18:55:15.209 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.209 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.209 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:55:15.209 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.739730+0000", 2024-09-20T18:55:15.209 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0210000000000003e-05 2024-09-20T18:55:15.209 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.209 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.209 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:55:15.210 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.739751+0000", 2024-09-20T18:55:15.210 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0585e-05 2024-09-20T18:55:15.210 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.210 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.210 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:55:15.210 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.739760+0000", 2024-09-20T18:55:15.210 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.2189999999999993e-06 2024-09-20T18:55:15.210 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.210 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.210 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:55:15.210 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.739852+0000", 2024-09-20T18:55:15.210 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.1718999999999994e-05 2024-09-20T18:55:15.210 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.210 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.211 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:55:15.211 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.739994+0000", 2024-09-20T18:55:15.211 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000141621 2024-09-20T18:55:15.211 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.211 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.211 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,2", 2024-09-20T18:55:15.211 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.740130+0000", 2024-09-20T18:55:15.211 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000135885 2024-09-20T18:55:15.211 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.211 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.211 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:55:15.211 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.741313+0000", 2024-09-20T18:55:15.211 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001183332 2024-09-20T18:55:15.211 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.212 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.212 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:15.212 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.841823+0000", 2024-09-20T18:55:15.212 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10051009499999999 2024-09-20T18:55:15.212 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.212 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.212 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:15.212 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.845143+0000", 2024-09-20T18:55:15.212 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0033201680000000001 2024-09-20T18:55:15.212 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.212 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.212 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:55:15.212 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.845194+0000", 2024-09-20T18:55:15.212 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.117e-05 2024-09-20T18:55:15.212 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.213 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.213 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:55:15.213 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.845235+0000", 2024-09-20T18:55:15.213 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0847999999999999e-05 2024-09-20T18:55:15.213 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:15.213 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:55:15.213 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:15.213 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.213 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.213 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:31048 7.5 7:a90359a4:::benchmark_data_smithi083_70643_object31047:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T18:55:15.213 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:10.677095+0000", 2024-09-20T18:55:15.213 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 244.5056046, 2024-09-20T18:55:15.213 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.085187867, 2024-09-20T18:55:15.213 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:55:15.214 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:55:15.214 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:15.214 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:55:15.214 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:55:15.215 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:15.215 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 31048 2024-09-20T18:55:15.215 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.215 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:55:15.215 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.215 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:55:15.215 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:10.677095+0000", 2024-09-20T18:55:15.215 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:15.215 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.215 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.215 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:55:15.215 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:10.677095+0000", 2024-09-20T18:55:15.215 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:15.216 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.216 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.216 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:55:15.216 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:10.677097+0000", 2024-09-20T18:55:15.216 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.784e-06 2024-09-20T18:55:15.216 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.216 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.216 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:55:15.216 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:10.677180+0000", 2024-09-20T18:55:15.216 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.3275999999999993e-05 2024-09-20T18:55:15.216 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.216 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.216 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:55:15.216 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:10.677192+0000", 2024-09-20T18:55:15.217 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1780000000000001e-05 2024-09-20T18:55:15.217 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.217 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.217 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:55:15.217 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:10.677199+0000", 2024-09-20T18:55:15.217 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.401e-06 2024-09-20T18:55:15.217 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.217 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.217 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:55:15.217 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:10.677267+0000", 2024-09-20T18:55:15.217 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.7537000000000001e-05 2024-09-20T18:55:15.217 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.217 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.217 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:55:15.218 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:10.677389+0000", 2024-09-20T18:55:15.218 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000122831 2024-09-20T18:55:15.218 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.218 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.218 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,2", 2024-09-20T18:55:15.218 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:10.677533+0000", 2024-09-20T18:55:15.218 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000143296 2024-09-20T18:55:15.218 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.218 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.218 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:55:15.218 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:10.678630+0000", 2024-09-20T18:55:15.218 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001097102 2024-09-20T18:55:15.218 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.218 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.218 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:15.219 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:10.679160+0000", 2024-09-20T18:55:15.219 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00053048699999999995 2024-09-20T18:55:15.219 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.219 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.219 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:15.219 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:10.762155+0000", 2024-09-20T18:55:15.219 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.082995034999999995 2024-09-20T18:55:15.219 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.219 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.219 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:55:15.219 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:10.762229+0000", 2024-09-20T18:55:15.219 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.3431999999999998e-05 2024-09-20T18:55:15.219 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.219 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.220 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:55:15.220 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:10.762283+0000", 2024-09-20T18:55:15.220 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3906000000000002e-05 2024-09-20T18:55:15.220 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:15.220 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:55:15.220 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:15.220 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.220 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.220 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:31052 7.5 7:aa0acc57:::benchmark_data_smithi083_70643_object31051:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T18:55:15.220 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:10.680120+0000", 2024-09-20T18:55:15.220 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 244.502579631, 2024-09-20T18:55:15.220 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.082632122000000002, 2024-09-20T18:55:15.220 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:55:15.221 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:55:15.221 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:15.221 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:55:15.221 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:55:15.221 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:15.221 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 31052 2024-09-20T18:55:15.221 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.221 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:55:15.221 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.221 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:55:15.221 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:10.680120+0000", 2024-09-20T18:55:15.221 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:15.221 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.222 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.222 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:55:15.222 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:10.680120+0000", 2024-09-20T18:55:15.222 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:15.222 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.222 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.222 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:55:15.222 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:10.680121+0000", 2024-09-20T18:55:15.222 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4100000000000001e-06 2024-09-20T18:55:15.222 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.222 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.222 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:55:15.222 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:10.680181+0000", 2024-09-20T18:55:15.222 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0129e-05 2024-09-20T18:55:15.223 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.223 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.223 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:55:15.223 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:10.680196+0000", 2024-09-20T18:55:15.223 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4562000000000001e-05 2024-09-20T18:55:15.223 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.223 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.223 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:55:15.223 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:10.680204+0000", 2024-09-20T18:55:15.223 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.2039999999999997e-06 2024-09-20T18:55:15.223 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.223 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.223 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:55:15.223 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:10.680260+0000", 2024-09-20T18:55:15.223 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6382999999999999e-05 2024-09-20T18:55:15.224 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.224 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.224 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:55:15.224 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:10.680378+0000", 2024-09-20T18:55:15.224 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000117069 2024-09-20T18:55:15.224 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.224 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.224 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,2", 2024-09-20T18:55:15.224 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:10.680518+0000", 2024-09-20T18:55:15.224 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000139994 2024-09-20T18:55:15.224 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.224 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.224 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:55:15.224 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:10.681377+0000", 2024-09-20T18:55:15.225 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00085957399999999999 2024-09-20T18:55:15.225 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.225 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.225 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:15.225 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:10.682232+0000", 2024-09-20T18:55:15.225 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00085484600000000003 2024-09-20T18:55:15.225 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.225 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.225 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:15.225 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:10.762653+0000", 2024-09-20T18:55:15.225 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.080420924000000005 2024-09-20T18:55:15.225 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.225 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.225 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:55:15.225 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:10.762709+0000", 2024-09-20T18:55:15.225 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5665000000000003e-05 2024-09-20T18:55:15.225 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.226 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.226 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:55:15.226 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:10.762752+0000", 2024-09-20T18:55:15.226 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3362000000000003e-05 2024-09-20T18:55:15.226 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:15.226 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:55:15.226 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:15.226 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.226 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.226 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:65463 7.5 7:ac697569:::benchmark_data_smithi083_70643_object65462:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-20T18:55:15.226 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:32.823693+0000", 2024-09-20T18:55:15.226 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 222.35900606499999, 2024-09-20T18:55:15.226 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.061018204999999999, 2024-09-20T18:55:15.227 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:55:15.227 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:55:15.227 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:15.227 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:55:15.227 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:55:15.227 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:15.227 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 65463 2024-09-20T18:55:15.227 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.227 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:55:15.227 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.227 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:55:15.227 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:32.823693+0000", 2024-09-20T18:55:15.227 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:15.227 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.228 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.228 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:55:15.228 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:32.823693+0000", 2024-09-20T18:55:15.228 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:15.228 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.228 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.228 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:55:15.228 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:32.823695+0000", 2024-09-20T18:55:15.228 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6929999999999999e-06 2024-09-20T18:55:15.228 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.228 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.228 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:55:15.228 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:32.823745+0000", 2024-09-20T18:55:15.228 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.9996e-05 2024-09-20T18:55:15.229 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.229 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.229 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:55:15.229 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:32.823756+0000", 2024-09-20T18:55:15.229 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1173e-05 2024-09-20T18:55:15.229 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.229 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.229 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:55:15.229 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:32.823762+0000", 2024-09-20T18:55:15.229 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0789999999999997e-06 2024-09-20T18:55:15.229 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.229 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.229 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:55:15.229 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:32.823804+0000", 2024-09-20T18:55:15.230 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.1459999999999999e-05 2024-09-20T18:55:15.230 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.230 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.230 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:55:15.230 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:32.823915+0000", 2024-09-20T18:55:15.230 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011095900000000001 2024-09-20T18:55:15.230 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.230 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.230 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,2", 2024-09-20T18:55:15.230 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:32.824008+0000", 2024-09-20T18:55:15.230 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.2955999999999996e-05 2024-09-20T18:55:15.230 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.230 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.230 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:55:15.231 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:32.825610+0000", 2024-09-20T18:55:15.231 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001602184 2024-09-20T18:55:15.231 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.231 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.231 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:15.231 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:32.827387+0000", 2024-09-20T18:55:15.231 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0017775899999999999 2024-09-20T18:55:15.231 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.231 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.231 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:15.231 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:32.884601+0000", 2024-09-20T18:55:15.231 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.057213631000000001 2024-09-20T18:55:15.231 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.231 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.232 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:55:15.232 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:32.884661+0000", 2024-09-20T18:55:15.232 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9416999999999997e-05 2024-09-20T18:55:15.232 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.232 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.232 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:55:15.232 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:32.884712+0000", 2024-09-20T18:55:15.232 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.1066999999999997e-05 2024-09-20T18:55:15.232 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:15.232 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:55:15.232 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:15.232 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.232 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.232 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:65675 7.5 7:ad801d92:::benchmark_data_smithi083_70643_object65674:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-20T18:55:15.232 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:33.015438+0000", 2024-09-20T18:55:15.233 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 222.167261333, 2024-09-20T18:55:15.233 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.065255200999999999, 2024-09-20T18:55:15.233 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:55:15.233 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:55:15.233 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:15.233 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:55:15.233 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:55:15.233 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:15.233 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 65675 2024-09-20T18:55:15.233 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.233 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:55:15.233 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.233 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:55:15.233 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:33.015438+0000", 2024-09-20T18:55:15.233 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:15.234 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.234 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.234 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:55:15.234 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:33.015438+0000", 2024-09-20T18:55:15.234 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:15.234 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.234 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.234 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:55:15.234 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:33.015440+0000", 2024-09-20T18:55:15.234 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.488e-06 2024-09-20T18:55:15.234 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.234 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.234 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:55:15.234 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:33.015486+0000", 2024-09-20T18:55:15.235 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.5985000000000001e-05 2024-09-20T18:55:15.235 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.235 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.235 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:55:15.235 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:33.015499+0000", 2024-09-20T18:55:15.235 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3645999999999999e-05 2024-09-20T18:55:15.235 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.235 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.235 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:55:15.235 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:33.015507+0000", 2024-09-20T18:55:15.235 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.4440000000000002e-06 2024-09-20T18:55:15.235 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.235 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.235 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:55:15.236 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:33.015623+0000", 2024-09-20T18:55:15.236 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000116731 2024-09-20T18:55:15.236 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.236 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.236 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:55:15.236 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:33.015719+0000", 2024-09-20T18:55:15.236 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.5208000000000001e-05 2024-09-20T18:55:15.236 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.236 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.236 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,2", 2024-09-20T18:55:15.236 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:33.015805+0000", 2024-09-20T18:55:15.236 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.5940999999999996e-05 2024-09-20T18:55:15.236 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.236 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.236 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:15.236 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:33.017847+0000", 2024-09-20T18:55:15.237 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0020427119999999999 2024-09-20T18:55:15.237 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.237 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.237 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:55:15.237 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:33.017882+0000", 2024-09-20T18:55:15.237 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.5046000000000003e-05 2024-09-20T18:55:15.237 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.237 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.237 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:15.237 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:33.080598+0000", 2024-09-20T18:55:15.237 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.062715897000000007 2024-09-20T18:55:15.237 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.237 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.237 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:55:15.238 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:33.080651+0000", 2024-09-20T18:55:15.238 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.2342999999999997e-05 2024-09-20T18:55:15.238 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.238 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.238 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:55:15.238 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:33.080693+0000", 2024-09-20T18:55:15.238 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.2759999999999997e-05 2024-09-20T18:55:15.238 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:15.238 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:55:15.238 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:15.238 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.238 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.238 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:70181 7.5 7:af77a2a1:::benchmark_data_smithi083_70643_object70180:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-20T18:55:15.238 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:36.479489+0000", 2024-09-20T18:55:15.239 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 218.70321054600001, 2024-09-20T18:55:15.239 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.060138616999999998, 2024-09-20T18:55:15.239 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:55:15.239 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:55:15.239 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:15.239 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:55:15.239 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:55:15.239 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:15.239 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 70181 2024-09-20T18:55:15.239 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.239 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:55:15.239 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.239 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:55:15.239 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:36.479489+0000", 2024-09-20T18:55:15.239 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:15.240 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.240 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.240 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:55:15.240 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:36.479489+0000", 2024-09-20T18:55:15.240 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:15.240 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.240 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.240 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:55:15.240 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:36.479491+0000", 2024-09-20T18:55:15.240 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.621e-06 2024-09-20T18:55:15.240 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.240 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.240 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:55:15.240 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:36.479523+0000", 2024-09-20T18:55:15.241 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.2268999999999999e-05 2024-09-20T18:55:15.241 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.241 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.241 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:55:15.241 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:36.479537+0000", 2024-09-20T18:55:15.241 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4496e-05 2024-09-20T18:55:15.241 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.241 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.241 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:55:15.241 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:36.479557+0000", 2024-09-20T18:55:15.241 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0152999999999998e-05 2024-09-20T18:55:15.241 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.241 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.241 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:55:15.242 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:36.479622+0000", 2024-09-20T18:55:15.242 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4264999999999995e-05 2024-09-20T18:55:15.242 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.242 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.242 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:55:15.242 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:36.479741+0000", 2024-09-20T18:55:15.242 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000119628 2024-09-20T18:55:15.242 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.242 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.242 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,2", 2024-09-20T18:55:15.242 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:36.479884+0000", 2024-09-20T18:55:15.242 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000143139 2024-09-20T18:55:15.242 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.242 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.243 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:55:15.243 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:36.481968+0000", 2024-09-20T18:55:15.243 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0020836470000000001 2024-09-20T18:55:15.243 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.243 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.243 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:15.243 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:36.535380+0000", 2024-09-20T18:55:15.243 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.053411827000000002 2024-09-20T18:55:15.243 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.243 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.243 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:15.243 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:36.539556+0000", 2024-09-20T18:55:15.243 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0041761580000000001 2024-09-20T18:55:15.243 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.243 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.244 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:55:15.244 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:36.539590+0000", 2024-09-20T18:55:15.244 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4236999999999998e-05 2024-09-20T18:55:15.244 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.244 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.244 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:55:15.244 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:36.539628+0000", 2024-09-20T18:55:15.244 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.7177e-05 2024-09-20T18:55:15.244 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:15.244 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:55:15.244 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:15.244 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.244 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.244 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:87113 7.5 7:ab6e1c85:::benchmark_data_smithi083_70643_object87112:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-20T18:55:15.244 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:51.409827+0000", 2024-09-20T18:55:15.245 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 203.772872337, 2024-09-20T18:55:15.245 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.058788189999999997, 2024-09-20T18:55:15.245 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:55:15.245 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:55:15.245 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:15.245 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:55:15.245 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:55:15.245 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:15.245 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 87113 2024-09-20T18:55:15.245 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.245 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:55:15.245 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.245 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:55:15.245 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:51.409827+0000", 2024-09-20T18:55:15.246 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:15.246 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.246 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.246 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:55:15.246 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:51.409827+0000", 2024-09-20T18:55:15.246 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:15.246 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.246 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.246 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:55:15.246 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:51.409828+0000", 2024-09-20T18:55:15.246 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.387e-06 2024-09-20T18:55:15.246 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.246 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.246 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:55:15.247 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:51.409875+0000", 2024-09-20T18:55:15.247 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.6919000000000003e-05 2024-09-20T18:55:15.247 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.247 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.247 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:55:15.247 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:51.409886+0000", 2024-09-20T18:55:15.247 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0376e-05 2024-09-20T18:55:15.247 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.247 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.247 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:55:15.247 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:51.409891+0000", 2024-09-20T18:55:15.247 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5330000000000002e-06 2024-09-20T18:55:15.247 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.247 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.248 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:55:15.248 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:51.409948+0000", 2024-09-20T18:55:15.248 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6882999999999997e-05 2024-09-20T18:55:15.248 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.248 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.248 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:55:15.248 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:51.410052+0000", 2024-09-20T18:55:15.248 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00010357999999999999 2024-09-20T18:55:15.248 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.248 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.248 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,2", 2024-09-20T18:55:15.248 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:51.410145+0000", 2024-09-20T18:55:15.248 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.3202999999999995e-05 2024-09-20T18:55:15.248 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.249 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.249 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:55:15.249 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:51.414357+0000", 2024-09-20T18:55:15.249 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.004212141 2024-09-20T18:55:15.249 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.249 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.249 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:15.249 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:51.449211+0000", 2024-09-20T18:55:15.249 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.034853497999999997 2024-09-20T18:55:15.249 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.249 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.249 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:15.249 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:51.468517+0000", 2024-09-20T18:55:15.249 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.019306001999999999 2024-09-20T18:55:15.249 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.250 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.250 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:55:15.250 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:51.468571+0000", 2024-09-20T18:55:15.250 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4546999999999998e-05 2024-09-20T18:55:15.250 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.250 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.250 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:55:15.250 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:51.468615+0000", 2024-09-20T18:55:15.250 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.4121e-05 2024-09-20T18:55:15.250 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:15.250 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:55:15.250 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:15.250 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.250 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.251 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:87118 7.5 7:a846fb95:::benchmark_data_smithi083_70643_object87117:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-20T18:55:15.251 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:51.411744+0000", 2024-09-20T18:55:15.251 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 203.77095502700001, 2024-09-20T18:55:15.251 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.05961172, 2024-09-20T18:55:15.251 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:55:15.251 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:55:15.251 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:15.251 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:55:15.251 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:55:15.251 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:15.251 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 87118 2024-09-20T18:55:15.251 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.251 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:55:15.251 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.251 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:55:15.252 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:51.411744+0000", 2024-09-20T18:55:15.252 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:15.252 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.252 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.252 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:55:15.252 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:51.411744+0000", 2024-09-20T18:55:15.252 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:15.252 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.252 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.252 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:55:15.252 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:51.411746+0000", 2024-09-20T18:55:15.252 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.483e-06 2024-09-20T18:55:15.252 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.252 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.252 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:55:15.253 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:51.411833+0000", 2024-09-20T18:55:15.253 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.7247e-05 2024-09-20T18:55:15.253 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.253 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.253 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:55:15.253 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:51.411846+0000", 2024-09-20T18:55:15.253 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2829e-05 2024-09-20T18:55:15.253 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.253 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.253 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:55:15.253 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:51.411854+0000", 2024-09-20T18:55:15.253 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.6520000000000006e-06 2024-09-20T18:55:15.253 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.253 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.254 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:55:15.254 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:51.412164+0000", 2024-09-20T18:55:15.254 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00031006399999999998 2024-09-20T18:55:15.254 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.254 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.254 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:55:15.254 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:51.412281+0000", 2024-09-20T18:55:15.254 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000117614 2024-09-20T18:55:15.254 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.254 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.254 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:15.256 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,2", 2024-09-20T18:55:15.256 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:51.412422+0000", 2024-09-20T18:55:15.256 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00014072800000000001 2024-09-20T18:55:15.256 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.256 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.256 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:55:15.256 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:51.414453+0000", 2024-09-20T18:55:15.256 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0020310229999999999 2024-09-20T18:55:15.256 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.256 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.256 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:15.256 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:51.449252+0000", 2024-09-20T18:55:15.256 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.034798612999999999 2024-09-20T18:55:15.257 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.257 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.257 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:15.257 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:51.471252+0000", 2024-09-20T18:55:15.257 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.021999872 2024-09-20T18:55:15.257 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.257 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.257 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:55:15.257 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:51.471309+0000", 2024-09-20T18:55:15.257 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.7954000000000001e-05 2024-09-20T18:55:15.257 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.257 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.257 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:55:15.257 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:51.471356+0000", 2024-09-20T18:55:15.257 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.6641000000000003e-05 2024-09-20T18:55:15.258 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:15.258 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:55:15.258 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:15.258 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.258 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.258 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:106209 7.5 7:afb00736:::benchmark_data_smithi083_70643_object106208:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-20T18:55:15.258 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:09.770099+0000", 2024-09-20T18:55:15.258 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 185.41260019999999, 2024-09-20T18:55:15.258 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.082658245000000005, 2024-09-20T18:55:15.258 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:55:15.258 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:55:15.258 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:15.258 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:55:15.259 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:55:15.259 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:15.259 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 106209 2024-09-20T18:55:15.259 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.259 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:55:15.259 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.259 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:55:15.259 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:09.770099+0000", 2024-09-20T18:55:15.259 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:15.259 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.259 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.259 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:55:15.259 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:09.770099+0000", 2024-09-20T18:55:15.259 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:15.260 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.260 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.260 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:55:15.260 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:09.770101+0000", 2024-09-20T18:55:15.260 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.739e-06 2024-09-20T18:55:15.260 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.260 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.260 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:55:15.260 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:09.770163+0000", 2024-09-20T18:55:15.260 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.1950000000000001e-05 2024-09-20T18:55:15.260 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.260 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.260 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:55:15.260 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:09.770176+0000", 2024-09-20T18:55:15.261 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2819999999999999e-05 2024-09-20T18:55:15.261 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.261 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.261 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:55:15.261 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:09.770186+0000", 2024-09-20T18:55:15.261 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.7999999999999993e-06 2024-09-20T18:55:15.261 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.261 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.261 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:55:15.261 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:09.775249+0000", 2024-09-20T18:55:15.261 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0050633589999999999 2024-09-20T18:55:15.261 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.261 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.261 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:55:15.262 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:09.775344+0000", 2024-09-20T18:55:15.262 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.5000999999999994e-05 2024-09-20T18:55:15.262 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.262 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.262 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,2", 2024-09-20T18:55:15.262 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:09.775494+0000", 2024-09-20T18:55:15.262 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015025999999999999 2024-09-20T18:55:15.262 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.262 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.262 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:55:15.262 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:09.780123+0000", 2024-09-20T18:55:15.262 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0046290139999999999 2024-09-20T18:55:15.262 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.262 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.262 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:15.263 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:09.780165+0000", 2024-09-20T18:55:15.263 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.2150999999999997e-05 2024-09-20T18:55:15.263 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.263 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.263 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:15.263 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:09.852635+0000", 2024-09-20T18:55:15.263 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.072469322000000003 2024-09-20T18:55:15.263 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.263 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.263 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:55:15.263 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:09.852693+0000", 2024-09-20T18:55:15.263 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8251000000000002e-05 2024-09-20T18:55:15.263 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.263 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.264 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:55:15.264 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:09.852757+0000", 2024-09-20T18:55:15.264 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4578000000000004e-05 2024-09-20T18:55:15.264 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:15.264 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:55:15.264 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:15.264 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.264 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.264 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:106215 7.5 7:a4ef0704:::benchmark_data_smithi083_70643_object106214:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-20T18:55:15.264 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:09.775460+0000", 2024-09-20T18:55:15.264 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 185.40723965000001, 2024-09-20T18:55:15.264 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.077432612999999997, 2024-09-20T18:55:15.264 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:55:15.264 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:55:15.264 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:15.265 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:55:15.265 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:55:15.265 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:15.265 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 106215 2024-09-20T18:55:15.265 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.265 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:55:15.265 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.265 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:55:15.265 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:09.775460+0000", 2024-09-20T18:55:15.265 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:15.265 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.265 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.265 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:55:15.265 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:09.775460+0000", 2024-09-20T18:55:15.266 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:15.266 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.266 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.266 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:55:15.266 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:09.775461+0000", 2024-09-20T18:55:15.266 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1939999999999999e-06 2024-09-20T18:55:15.266 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.266 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.266 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:55:15.266 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:09.775515+0000", 2024-09-20T18:55:15.266 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4104999999999998e-05 2024-09-20T18:55:15.266 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.266 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.266 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:55:15.266 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:09.775530+0000", 2024-09-20T18:55:15.267 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4899000000000001e-05 2024-09-20T18:55:15.267 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.267 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.267 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:55:15.267 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:09.775536+0000", 2024-09-20T18:55:15.267 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6679999999999998e-06 2024-09-20T18:55:15.267 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.267 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.267 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:55:15.267 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:09.777399+0000", 2024-09-20T18:55:15.267 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0018631769999999999 2024-09-20T18:55:15.267 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.267 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.267 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:55:15.267 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:09.777497+0000", 2024-09-20T18:55:15.268 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.8312000000000003e-05 2024-09-20T18:55:15.268 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.268 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.268 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,2", 2024-09-20T18:55:15.268 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:09.777600+0000", 2024-09-20T18:55:15.268 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00010309799999999999 2024-09-20T18:55:15.268 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.268 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.268 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:55:15.268 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:09.780129+0000", 2024-09-20T18:55:15.268 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0025291599999999999 2024-09-20T18:55:15.268 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.268 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.268 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:15.268 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:09.780220+0000", 2024-09-20T18:55:15.269 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.0451999999999994e-05 2024-09-20T18:55:15.269 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.269 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.269 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:15.269 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:09.852806+0000", 2024-09-20T18:55:15.269 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.072586237999999997 2024-09-20T18:55:15.269 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.269 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.269 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:55:15.269 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:09.852847+0000", 2024-09-20T18:55:15.269 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0760999999999998e-05 2024-09-20T18:55:15.269 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.269 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.269 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:55:15.270 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:09.852892+0000", 2024-09-20T18:55:15.270 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.5549e-05 2024-09-20T18:55:15.270 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:15.270 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:55:15.270 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:15.270 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.270 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.270 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:106222 7.5 7:a144fe28:::benchmark_data_smithi083_70643_object106221:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-20T18:55:15.270 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:09.785282+0000", 2024-09-20T18:55:15.270 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 185.39741712700001, 2024-09-20T18:55:15.270 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.068234718, 2024-09-20T18:55:15.270 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:55:15.270 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:55:15.270 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:15.271 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:55:15.271 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:55:15.271 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:15.271 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 106222 2024-09-20T18:55:15.271 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.271 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:55:15.271 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.271 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:55:15.271 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:09.785282+0000", 2024-09-20T18:55:15.271 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:15.271 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.271 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.271 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:55:15.271 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:09.785282+0000", 2024-09-20T18:55:15.271 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:15.272 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.272 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.272 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:55:15.272 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:09.785284+0000", 2024-09-20T18:55:15.272 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6190000000000001e-06 2024-09-20T18:55:15.272 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.272 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.272 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:55:15.272 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:09.785324+0000", 2024-09-20T18:55:15.272 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9576999999999997e-05 2024-09-20T18:55:15.272 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.272 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.272 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:55:15.272 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:09.785335+0000", 2024-09-20T18:55:15.273 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1928e-05 2024-09-20T18:55:15.273 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.273 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.273 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:55:15.273 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:09.785341+0000", 2024-09-20T18:55:15.273 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0299999999999999e-06 2024-09-20T18:55:15.273 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.273 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.273 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:55:15.273 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:09.785390+0000", 2024-09-20T18:55:15.273 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8804000000000002e-05 2024-09-20T18:55:15.273 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.273 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.273 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:55:15.273 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:09.785478+0000", 2024-09-20T18:55:15.274 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.8171000000000007e-05 2024-09-20T18:55:15.274 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.274 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.274 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,2", 2024-09-20T18:55:15.274 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:09.785579+0000", 2024-09-20T18:55:15.274 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000100907 2024-09-20T18:55:15.274 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.274 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.274 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:55:15.274 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:09.787490+0000", 2024-09-20T18:55:15.274 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0019103569999999999 2024-09-20T18:55:15.274 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.274 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.274 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:15.275 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:09.787650+0000", 2024-09-20T18:55:15.275 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00016057199999999999 2024-09-20T18:55:15.275 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.275 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.275 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:15.275 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:09.853449+0000", 2024-09-20T18:55:15.275 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.065798762999999996 2024-09-20T18:55:15.275 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.275 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.275 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:55:15.275 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:09.853479+0000", 2024-09-20T18:55:15.275 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.0273999999999998e-05 2024-09-20T18:55:15.275 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.275 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.276 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:55:15.276 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:09.853517+0000", 2024-09-20T18:55:15.276 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.7716000000000003e-05 2024-09-20T18:55:15.276 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:15.276 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:55:15.276 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:15.276 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.276 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.276 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:139222 7.5 7:a5a82ccf:::benchmark_data_smithi083_70643_object16420:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-20T18:55:15.276 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:42.806570+0000", 2024-09-20T18:55:15.276 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 152.37612989100001, 2024-09-20T18:55:15.276 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.12715717200000001, 2024-09-20T18:55:15.276 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:55:15.276 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:55:15.276 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:15.277 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:55:15.277 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:55:15.277 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:15.277 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 139222 2024-09-20T18:55:15.277 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.277 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:55:15.277 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.277 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:55:15.277 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806570+0000", 2024-09-20T18:55:15.277 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:15.277 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.277 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.277 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:55:15.278 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806570+0000", 2024-09-20T18:55:15.278 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:15.278 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.278 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.278 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:55:15.278 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806571+0000", 2024-09-20T18:55:15.278 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.621e-06 2024-09-20T18:55:15.278 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.278 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.278 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:55:15.278 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806576+0000", 2024-09-20T18:55:15.278 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3449999999999999e-06 2024-09-20T18:55:15.278 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.278 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.279 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:55:15.279 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806587+0000", 2024-09-20T18:55:15.279 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1375e-05 2024-09-20T18:55:15.279 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.279 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.279 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:55:15.279 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806595+0000", 2024-09-20T18:55:15.279 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.7449999999999995e-06 2024-09-20T18:55:15.279 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.279 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.279 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:55:15.279 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806658+0000", 2024-09-20T18:55:15.279 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.3601000000000003e-05 2024-09-20T18:55:15.279 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.279 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.280 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:55:15.280 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806734+0000", 2024-09-20T18:55:15.280 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5519000000000007e-05 2024-09-20T18:55:15.280 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.280 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.280 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,2", 2024-09-20T18:55:15.280 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806809+0000", 2024-09-20T18:55:15.280 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5587000000000007e-05 2024-09-20T18:55:15.280 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.280 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.280 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:55:15.280 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.810258+0000", 2024-09-20T18:55:15.280 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0034485219999999999 2024-09-20T18:55:15.280 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.281 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.281 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:15.281 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.810281+0000", 2024-09-20T18:55:15.281 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2999000000000001e-05 2024-09-20T18:55:15.281 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.281 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.281 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:15.281 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.933655+0000", 2024-09-20T18:55:15.281 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.12337419299999999 2024-09-20T18:55:15.281 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.281 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.281 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:55:15.281 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.933685+0000", 2024-09-20T18:55:15.281 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.0219e-05 2024-09-20T18:55:15.282 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.282 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.282 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:55:15.282 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.933727+0000", 2024-09-20T18:55:15.282 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.1446000000000003e-05 2024-09-20T18:55:15.282 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:15.282 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:55:15.282 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:15.282 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.282 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.282 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:139223 7.5 7:ae1b4edf:::benchmark_data_smithi083_70643_object16421:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-20T18:55:15.282 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:42.806736+0000", 2024-09-20T18:55:15.282 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 152.375963181, 2024-09-20T18:55:15.283 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.127118553, 2024-09-20T18:55:15.283 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:55:15.283 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:55:15.283 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:15.283 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:55:15.283 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:55:15.283 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:15.283 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 139223 2024-09-20T18:55:15.283 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.283 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:55:15.283 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.283 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:55:15.283 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806736+0000", 2024-09-20T18:55:15.283 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:15.284 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.284 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.284 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:55:15.284 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806736+0000", 2024-09-20T18:55:15.284 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:15.284 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.284 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.284 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:55:15.284 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806737+0000", 2024-09-20T18:55:15.284 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0529999999999999e-06 2024-09-20T18:55:15.284 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.284 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.284 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:55:15.284 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806742+0000", 2024-09-20T18:55:15.285 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3749999999999996e-06 2024-09-20T18:55:15.285 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.285 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.285 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:55:15.285 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806752+0000", 2024-09-20T18:55:15.285 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0730999999999999e-05 2024-09-20T18:55:15.285 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.285 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.285 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:55:15.285 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806760+0000", 2024-09-20T18:55:15.285 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.4569999999999999e-06 2024-09-20T18:55:15.285 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.285 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.286 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:55:15.286 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.808698+0000", 2024-09-20T18:55:15.286 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0019381380000000001 2024-09-20T18:55:15.286 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.286 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.286 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:55:15.286 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.808770+0000", 2024-09-20T18:55:15.286 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.1746000000000001e-05 2024-09-20T18:55:15.286 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.286 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.286 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,2", 2024-09-20T18:55:15.286 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.808835+0000", 2024-09-20T18:55:15.286 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4832999999999994e-05 2024-09-20T18:55:15.286 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.287 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.287 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:15.287 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.810390+0000", 2024-09-20T18:55:15.287 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001555463 2024-09-20T18:55:15.287 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.287 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.287 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:55:15.287 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.810400+0000", 2024-09-20T18:55:15.287 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.8239999999999995e-06 2024-09-20T18:55:15.287 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.287 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.287 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:15.287 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.933775+0000", 2024-09-20T18:55:15.287 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.12337550799999999 2024-09-20T18:55:15.288 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.288 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.288 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:55:15.288 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.933814+0000", 2024-09-20T18:55:15.288 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.8874000000000001e-05 2024-09-20T18:55:15.288 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.288 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.288 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:55:15.288 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.933855+0000", 2024-09-20T18:55:15.288 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0550999999999999e-05 2024-09-20T18:55:15.288 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:15.288 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:55:15.288 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:15.288 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.288 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.289 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:139483 7.5 7:ab7602ac:::benchmark_data_smithi083_70643_object16681:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-20T18:55:15.289 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:43.132737+0000", 2024-09-20T18:55:15.289 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 152.04996291500001, 2024-09-20T18:55:15.289 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.076706779000000003, 2024-09-20T18:55:15.289 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:55:15.289 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:55:15.289 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:15.289 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:55:15.289 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:55:15.289 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:15.289 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 139483 2024-09-20T18:55:15.289 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.289 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:55:15.289 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.290 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:55:15.290 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.132737+0000", 2024-09-20T18:55:15.290 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:15.290 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.290 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.290 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:55:15.290 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.132737+0000", 2024-09-20T18:55:15.290 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:15.290 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.290 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.290 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:55:15.290 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.132738+0000", 2024-09-20T18:55:15.290 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3659999999999999e-06 2024-09-20T18:55:15.290 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.291 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.291 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:55:15.291 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.132743+0000", 2024-09-20T18:55:15.291 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0209999999999999e-06 2024-09-20T18:55:15.291 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.291 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.291 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:55:15.291 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.132754+0000", 2024-09-20T18:55:15.291 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1034e-05 2024-09-20T18:55:15.291 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.291 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.291 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:55:15.291 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.132760+0000", 2024-09-20T18:55:15.291 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4679999999999997e-06 2024-09-20T18:55:15.291 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.292 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.292 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:55:15.292 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.132813+0000", 2024-09-20T18:55:15.292 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.2913e-05 2024-09-20T18:55:15.292 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.292 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.292 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:55:15.292 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.132929+0000", 2024-09-20T18:55:15.292 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011612099999999999 2024-09-20T18:55:15.292 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.292 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.292 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,2", 2024-09-20T18:55:15.292 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.133070+0000", 2024-09-20T18:55:15.292 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000140149 2024-09-20T18:55:15.292 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.293 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.293 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:55:15.293 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.135000+0000", 2024-09-20T18:55:15.293 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0019306270000000001 2024-09-20T18:55:15.293 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.293 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.293 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:15.293 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.135371+0000", 2024-09-20T18:55:15.293 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000371157 2024-09-20T18:55:15.293 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.293 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.293 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:15.293 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.209339+0000", 2024-09-20T18:55:15.293 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.073968107000000005 2024-09-20T18:55:15.293 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.294 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.294 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:55:15.294 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.209396+0000", 2024-09-20T18:55:15.294 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6085000000000003e-05 2024-09-20T18:55:15.294 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.294 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.294 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:55:15.294 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:43.209443+0000", 2024-09-20T18:55:15.294 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7731000000000001e-05 2024-09-20T18:55:15.294 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:15.294 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:55:15.294 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:15.294 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.294 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.295 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:175264 7.5 7:acde70e2:::benchmark_data_smithi083_70643_object52462:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137)", 2024-09-20T18:55:15.295 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:53:16.777008+0000", 2024-09-20T18:55:15.295 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 118.40569155, 2024-09-20T18:55:15.295 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.078417799999999996, 2024-09-20T18:55:15.295 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:55:15.295 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:55:15.295 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:15.295 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:55:15.295 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:55:15.296 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T18:55:15.297 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:15.297 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 175264 2024-09-20T18:55:15.297 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.297 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:55:15.297 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.297 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:55:15.298 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:16.777008+0000", 2024-09-20T18:55:15.298 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:15.298 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.298 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.298 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:55:15.298 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:16.777008+0000", 2024-09-20T18:55:15.298 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:15.298 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.298 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.298 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:55:15.298 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:16.777009+0000", 2024-09-20T18:55:15.298 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.291e-06 2024-09-20T18:55:15.298 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.298 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.299 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:55:15.299 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:16.777014+0000", 2024-09-20T18:55:15.299 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.5240000000000002e-06 2024-09-20T18:55:15.299 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.299 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.299 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:55:15.299 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:16.777024+0000", 2024-09-20T18:55:15.299 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.9590000000000007e-06 2024-09-20T18:55:15.299 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.299 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.299 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:55:15.299 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:16.777029+0000", 2024-09-20T18:55:15.299 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4639999999999997e-06 2024-09-20T18:55:15.299 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.300 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.300 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:55:15.300 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:16.777078+0000", 2024-09-20T18:55:15.300 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8375999999999999e-05 2024-09-20T18:55:15.300 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.300 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.300 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:55:15.300 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:16.777155+0000", 2024-09-20T18:55:15.300 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.7843000000000007e-05 2024-09-20T18:55:15.300 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.300 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.300 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,2", 2024-09-20T18:55:15.300 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:16.777224+0000", 2024-09-20T18:55:15.300 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.8349999999999994e-05 2024-09-20T18:55:15.301 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.301 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.301 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:55:15.301 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:16.779488+0000", 2024-09-20T18:55:15.301 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0022647589999999999 2024-09-20T18:55:15.301 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.301 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.301 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:15.301 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:16.780969+0000", 2024-09-20T18:55:15.301 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0014805599999999999 2024-09-20T18:55:15.301 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.301 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.301 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:15.301 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:16.855281+0000", 2024-09-20T18:55:15.301 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.074311782000000007 2024-09-20T18:55:15.302 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.302 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.302 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:55:15.302 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:16.855336+0000", 2024-09-20T18:55:15.302 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5247999999999997e-05 2024-09-20T18:55:15.302 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.302 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.302 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:55:15.302 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:16.855426+0000", 2024-09-20T18:55:15.302 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.9643999999999998e-05 2024-09-20T18:55:15.302 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:15.302 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:55:15.302 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:15.302 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.303 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.303 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:188403 7.5 7:a701844f:::benchmark_data_smithi083_70643_object65601:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-20T18:55:15.303 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:53:29.679766+0000", 2024-09-20T18:55:15.303 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 105.50293354599999, 2024-09-20T18:55:15.303 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.080671427000000004, 2024-09-20T18:55:15.303 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:55:15.303 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:55:15.303 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:15.303 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:55:15.303 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:55:15.303 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:15.303 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 188403 2024-09-20T18:55:15.303 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.303 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:55:15.303 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.304 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:55:15.304 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.679766+0000", 2024-09-20T18:55:15.304 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:15.304 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.304 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.304 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:55:15.304 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.679766+0000", 2024-09-20T18:55:15.304 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:15.304 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.304 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.304 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:55:15.304 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.679768+0000", 2024-09-20T18:55:15.304 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.657e-06 2024-09-20T18:55:15.305 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.305 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.305 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:55:15.305 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.679773+0000", 2024-09-20T18:55:15.305 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5149999999999997e-06 2024-09-20T18:55:15.305 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.305 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.305 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:55:15.305 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.679788+0000", 2024-09-20T18:55:15.305 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4508999999999999e-05 2024-09-20T18:55:15.305 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.305 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.305 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:55:15.305 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.679796+0000", 2024-09-20T18:55:15.305 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.2090000000000005e-06 2024-09-20T18:55:15.306 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.306 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.306 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:55:15.306 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.679860+0000", 2024-09-20T18:55:15.306 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4585999999999995e-05 2024-09-20T18:55:15.306 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.306 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.306 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:55:15.306 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.679964+0000", 2024-09-20T18:55:15.306 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00010348099999999999 2024-09-20T18:55:15.306 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.306 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.306 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,2", 2024-09-20T18:55:15.306 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.680087+0000", 2024-09-20T18:55:15.307 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000123607 2024-09-20T18:55:15.307 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.307 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.307 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:55:15.307 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.682715+0000", 2024-09-20T18:55:15.307 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0026271530000000001 2024-09-20T18:55:15.307 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.307 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.307 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:15.307 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.682896+0000", 2024-09-20T18:55:15.307 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00018158000000000001 2024-09-20T18:55:15.307 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.307 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.307 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:15.308 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.760364+0000", 2024-09-20T18:55:15.308 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.077467822000000006 2024-09-20T18:55:15.308 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.308 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.308 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:55:15.308 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.760397+0000", 2024-09-20T18:55:15.308 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.3024000000000001e-05 2024-09-20T18:55:15.308 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.308 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.308 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:55:15.308 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:29.760437+0000", 2024-09-20T18:55:15.308 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0284000000000003e-05 2024-09-20T18:55:15.308 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:15.308 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:55:15.308 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:15.309 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.309 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.309 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:194702 7.5 7:ab43130b:::benchmark_data_smithi083_70643_object71900:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e144)", 2024-09-20T18:55:15.309 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:53:35.709984+0000", 2024-09-20T18:55:15.309 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 99.472715125999997, 2024-09-20T18:55:15.309 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.060538688, 2024-09-20T18:55:15.309 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:55:15.309 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:55:15.309 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:15.309 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:55:15.309 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:55:15.309 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:15.309 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 194702 2024-09-20T18:55:15.309 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.310 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:55:15.310 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.310 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:55:15.310 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:35.709984+0000", 2024-09-20T18:55:15.310 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:15.310 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.310 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.310 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:55:15.310 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:35.709984+0000", 2024-09-20T18:55:15.310 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:15.310 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.310 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.310 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:55:15.310 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:35.709986+0000", 2024-09-20T18:55:15.311 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.221e-06 2024-09-20T18:55:15.311 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.311 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.311 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:55:15.311 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:35.709990+0000", 2024-09-20T18:55:15.311 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7380000000000002e-06 2024-09-20T18:55:15.311 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.311 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.311 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:55:15.311 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:35.710001+0000", 2024-09-20T18:55:15.311 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.114e-05 2024-09-20T18:55:15.311 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.311 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.311 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:55:15.312 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:35.710008+0000", 2024-09-20T18:55:15.312 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4840000000000001e-06 2024-09-20T18:55:15.312 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.312 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.312 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:55:15.312 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:35.710065+0000", 2024-09-20T18:55:15.312 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.7194999999999997e-05 2024-09-20T18:55:15.312 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.312 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.312 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:55:15.312 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:35.710137+0000", 2024-09-20T18:55:15.312 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.2218000000000005e-05 2024-09-20T18:55:15.312 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.312 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.313 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,2", 2024-09-20T18:55:15.313 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:35.710481+0000", 2024-09-20T18:55:15.313 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00034402 2024-09-20T18:55:15.313 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.313 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.313 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:55:15.313 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:35.713184+0000", 2024-09-20T18:55:15.313 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0027023749999999999 2024-09-20T18:55:15.313 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.313 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.313 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:15.313 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:35.713281+0000", 2024-09-20T18:55:15.313 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.7528000000000005e-05 2024-09-20T18:55:15.313 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.313 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.314 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:15.314 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:35.770368+0000", 2024-09-20T18:55:15.314 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.057087233000000001 2024-09-20T18:55:15.314 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.314 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.314 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:55:15.314 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:35.770454+0000", 2024-09-20T18:55:15.314 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.5760999999999994e-05 2024-09-20T18:55:15.314 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.314 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.314 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:55:15.314 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:35.770523+0000", 2024-09-20T18:55:15.314 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.8775000000000005e-05 2024-09-20T18:55:15.314 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:15.315 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:55:15.315 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:15.315 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.315 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.315 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:197552 7.5 7:a595dabc:::benchmark_data_smithi083_70643_object74750:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e144)", 2024-09-20T18:55:15.315 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:53:38.356465+0000", 2024-09-20T18:55:15.315 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 96.826234616999997, 2024-09-20T18:55:15.315 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.073240718999999996, 2024-09-20T18:55:15.315 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:55:15.315 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:55:15.315 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:15.315 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:55:15.315 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:55:15.315 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:15.315 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 197552 2024-09-20T18:55:15.316 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.316 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:55:15.316 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.316 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:55:15.316 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:38.356465+0000", 2024-09-20T18:55:15.316 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:15.316 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.316 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.316 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:55:15.316 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:38.356465+0000", 2024-09-20T18:55:15.316 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:15.316 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.316 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.316 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:55:15.317 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:38.356466+0000", 2024-09-20T18:55:15.317 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.184e-06 2024-09-20T18:55:15.317 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.317 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.317 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:55:15.317 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:38.356470+0000", 2024-09-20T18:55:15.317 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.4510000000000002e-06 2024-09-20T18:55:15.317 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.317 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.317 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:55:15.317 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:38.356481+0000", 2024-09-20T18:55:15.317 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0054999999999999e-05 2024-09-20T18:55:15.317 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.317 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.318 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:55:15.318 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:38.356486+0000", 2024-09-20T18:55:15.318 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8130000000000001e-06 2024-09-20T18:55:15.318 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.318 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.318 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:55:15.318 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:38.356530+0000", 2024-09-20T18:55:15.318 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3407e-05 2024-09-20T18:55:15.318 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.318 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.318 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:55:15.318 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:38.356609+0000", 2024-09-20T18:55:15.318 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.9006999999999998e-05 2024-09-20T18:55:15.318 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.318 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.319 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,2", 2024-09-20T18:55:15.319 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:38.356683+0000", 2024-09-20T18:55:15.319 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.3915e-05 2024-09-20T18:55:15.319 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.319 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.319 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:55:15.319 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:38.358551+0000", 2024-09-20T18:55:15.319 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0018683230000000001 2024-09-20T18:55:15.319 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.319 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.319 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:15.319 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:38.389027+0000", 2024-09-20T18:55:15.319 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.030476281000000001 2024-09-20T18:55:15.319 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.320 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.320 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:15.320 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:38.429598+0000", 2024-09-20T18:55:15.320 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.040570978000000001 2024-09-20T18:55:15.320 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.320 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.320 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:55:15.320 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:38.429668+0000", 2024-09-20T18:55:15.320 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.9380000000000003e-05 2024-09-20T18:55:15.320 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.320 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.320 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:55:15.320 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:38.429706+0000", 2024-09-20T18:55:15.320 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.7925e-05 2024-09-20T18:55:15.321 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:15.321 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:55:15.321 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:15.321 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.321 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.321 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:210322 7.5 7:a7198242:::benchmark_data_smithi083_70643_object87520:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-20T18:55:15.321 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:53:50.952233+0000", 2024-09-20T18:55:15.321 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 84.230466379000006, 2024-09-20T18:55:15.321 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.060714255000000002, 2024-09-20T18:55:15.321 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:55:15.321 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:55:15.321 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:15.321 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:55:15.321 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:55:15.321 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:15.322 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 210322 2024-09-20T18:55:15.322 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.322 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:55:15.322 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.322 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:55:15.322 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:50.952233+0000", 2024-09-20T18:55:15.322 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:15.322 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.322 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.322 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:55:15.322 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:50.952233+0000", 2024-09-20T18:55:15.322 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:55:15.322 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.322 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.323 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:55:15.323 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:50.952234+0000", 2024-09-20T18:55:15.323 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2720000000000001e-06 2024-09-20T18:55:15.323 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.323 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.323 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:55:15.323 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:50.952238+0000", 2024-09-20T18:55:15.323 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9090000000000002e-06 2024-09-20T18:55:15.323 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.323 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.323 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:55:15.323 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:50.952246+0000", 2024-09-20T18:55:15.323 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.2549999999999997e-06 2024-09-20T18:55:15.323 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.323 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.324 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:55:15.324 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:50.952251+0000", 2024-09-20T18:55:15.324 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.1439999999999996e-06 2024-09-20T18:55:15.324 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.324 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.324 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:55:15.324 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:50.952293+0000", 2024-09-20T18:55:15.324 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.2157999999999999e-05 2024-09-20T18:55:15.324 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.324 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.324 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:55:15.324 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:50.952376+0000", 2024-09-20T18:55:15.324 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.3054999999999997e-05 2024-09-20T18:55:15.324 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.324 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.325 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,2", 2024-09-20T18:55:15.325 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:50.952447+0000", 2024-09-20T18:55:15.325 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.0858999999999994e-05 2024-09-20T18:55:15.325 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.325 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.325 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:55:15.325 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:50.954418+0000", 2024-09-20T18:55:15.325 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001971427 2024-09-20T18:55:15.325 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.325 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.325 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:15.325 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:50.956444+0000", 2024-09-20T18:55:15.325 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0020258139999999999 2024-09-20T18:55:15.326 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.326 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.326 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:15.326 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:51.012883+0000", 2024-09-20T18:55:15.326 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.056438727000000001 2024-09-20T18:55:15.326 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.326 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.326 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:55:15.326 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:51.012918+0000", 2024-09-20T18:55:15.326 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.5395999999999998e-05 2024-09-20T18:55:15.326 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:55:15.326 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:55:15.326 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:55:15.326 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:53:51.012947+0000", 2024-09-20T18:55:15.326 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9238999999999999e-05 2024-09-20T18:55:15.327 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:15.327 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:55:15.327 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:15.327 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:55:15.327 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:55:15.327 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T18:55:15.330 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:15.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:15.331 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:15.432 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:15.468 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:15.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:15.470 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:15.570 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:55:15.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:15.611 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:15.611 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:55:15.679 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:15 smithi083 ceph-mon[29421]: 1.0 scrub starts 2024-09-20T18:55:15.679 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:15 smithi083 ceph-mon[29421]: 1.0 scrub ok 2024-09-20T18:55:15.679 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:15 smithi083 ceph-mon[29421]: 4.6 scrub starts 2024-09-20T18:55:15.679 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:15 smithi083 ceph-mon[29421]: 4.6 scrub ok 2024-09-20T18:55:15.679 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:15 smithi083 ceph-mon[29421]: 3.1c scrub starts 2024-09-20T18:55:15.679 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:15 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:55:15.679 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:15 smithi083 ceph-mon[29421]: osdmap e175: 8 total, 8 up, 8 in 2024-09-20T18:55:15.712 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:15.726 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:55:15.746 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:15.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:15.747 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:15.848 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:15.889 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:15.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:15.890 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:15.990 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:16.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:15 smithi142 ceph-mon[32508]: 1.0 scrub starts 2024-09-20T18:55:16.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:15 smithi142 ceph-mon[32508]: 1.0 scrub ok 2024-09-20T18:55:16.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:15 smithi142 ceph-mon[32508]: 4.6 scrub starts 2024-09-20T18:55:16.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:15 smithi142 ceph-mon[32508]: 4.6 scrub ok 2024-09-20T18:55:16.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:15 smithi142 ceph-mon[32508]: 3.1c scrub starts 2024-09-20T18:55:16.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:15 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:55:16.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:15 smithi142 ceph-mon[32508]: osdmap e175: 8 total, 8 up, 8 in 2024-09-20T18:55:16.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:16.024 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:16.024 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:16.038 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:15 smithi083 ceph-mon[36260]: 1.0 scrub starts 2024-09-20T18:55:16.039 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:15 smithi083 ceph-mon[36260]: 1.0 scrub ok 2024-09-20T18:55:16.039 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:15 smithi083 ceph-mon[36260]: 4.6 scrub starts 2024-09-20T18:55:16.039 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:15 smithi083 ceph-mon[36260]: 4.6 scrub ok 2024-09-20T18:55:16.039 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:15 smithi083 ceph-mon[36260]: 3.1c scrub starts 2024-09-20T18:55:16.039 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:15 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:55:16.039 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:15 smithi083 ceph-mon[36260]: osdmap e175: 8 total, 8 up, 8 in 2024-09-20T18:55:16.124 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:16.151 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-20T18:55:16.164 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:16.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:16.165 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:16.173 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 36 16 68973 68957 119.651 84.8125 0.0217041 0.00835292 2024-09-20T18:55:16.265 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:16.295 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:16.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:16.296 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:16.381 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:55:16.382 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:55:16.396 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:16.420 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T18:55:16.420 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:55:16.420 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:55:16.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:16.450 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:16.450 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:16.456 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:55:16.456 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:55:16.489 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:55:16.490 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:55:16.533 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:55:16.533 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:55:16.551 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:16.566 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:55:16.567 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:55:16.578 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:16.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:16.579 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:16.599 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:55:16.599 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:55:16.614 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:55:16.635 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:55:16.635 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:55:16.680 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:55:16.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:16.724 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:16.724 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:55:16.825 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:16.852 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:16.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:16.853 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:16.954 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:16.987 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:16.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:16.988 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:16.994 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:16 smithi083 ceph-mon[29421]: pgmap v253: 148 pgs: 148 active+clean; 3.8 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 712 B/s rd, 63 MiB/s wr, 1.00k op/s 2024-09-20T18:55:16.994 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:16 smithi083 ceph-mon[29421]: 3.1c scrub ok 2024-09-20T18:55:16.995 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:16 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/155211026' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:55:16.995 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:16 smithi083 ceph-mon[36260]: pgmap v253: 148 pgs: 148 active+clean; 3.8 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 712 B/s rd, 63 MiB/s wr, 1.00k op/s 2024-09-20T18:55:16.995 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:16 smithi083 ceph-mon[36260]: 3.1c scrub ok 2024-09-20T18:55:16.995 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:16 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/155211026' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:55:17.022 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:55:16 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:55:17.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:16 smithi142 ceph-mon[32508]: pgmap v253: 148 pgs: 148 active+clean; 3.8 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 712 B/s rd, 63 MiB/s wr, 1.00k op/s 2024-09-20T18:55:17.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:16 smithi142 ceph-mon[32508]: 3.1c scrub ok 2024-09-20T18:55:17.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:16 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/155211026' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:55:17.089 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:17.123 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:17.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:17.124 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:17.174 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 37 15 70215 70200 118.517 77.6875 0.0106849 0.00843308 2024-09-20T18:55:17.225 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:17.257 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:17.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:17.258 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:17.358 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:55:17.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:17.397 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:17.397 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:55:17.498 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:17.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:17.527 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:17.528 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:17.628 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:17.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:17.663 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:17.664 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:17.764 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:17.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:17.798 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:17.799 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:17.899 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:17.921 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:17.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:17.922 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:18.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:17 smithi142 ceph-mon[32508]: 4.4 scrub starts 2024-09-20T18:55:18.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:17 smithi142 ceph-mon[32508]: 4.4 scrub ok 2024-09-20T18:55:18.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:17 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:55:18.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:17 smithi142 ceph-mon[32508]: 6.b scrub starts 2024-09-20T18:55:18.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:17 smithi142 ceph-mon[32508]: 6.b scrub ok 2024-09-20T18:55:18.023 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:18.025 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:17 smithi083 ceph-mon[29421]: 4.4 scrub starts 2024-09-20T18:55:18.026 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:17 smithi083 ceph-mon[29421]: 4.4 scrub ok 2024-09-20T18:55:18.026 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:17 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:55:18.026 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:17 smithi083 ceph-mon[29421]: 6.b scrub starts 2024-09-20T18:55:18.026 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:17 smithi083 ceph-mon[29421]: 6.b scrub ok 2024-09-20T18:55:18.026 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:17 smithi083 ceph-mon[36260]: 4.4 scrub starts 2024-09-20T18:55:18.026 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:17 smithi083 ceph-mon[36260]: 4.4 scrub ok 2024-09-20T18:55:18.026 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:17 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:55:18.026 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:17 smithi083 ceph-mon[36260]: 6.b scrub starts 2024-09-20T18:55:18.026 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:17 smithi083 ceph-mon[36260]: 6.b scrub ok 2024-09-20T18:55:18.064 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:18.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:18.065 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:18.165 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:18.175 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 38 15 71466 71451 117.451 78.1875 0.00906415 0.0085096 2024-09-20T18:55:18.188 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T18:55:18.188 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:55:18.190 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:18.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:18.191 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:18.197 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:55:18.292 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:18.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:18.335 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:18.335 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:18.436 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:18.471 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:18.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:18.472 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:18.573 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:18.610 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:18.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:18.611 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:18.712 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:18.750 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:18.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:18.751 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:18.802 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:18 smithi083 ceph-mon[29421]: pgmap v255: 148 pgs: 148 active+clean; 4.2 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.6 KiB/s rd, 143 MiB/s wr, 2.28k op/s 2024-09-20T18:55:18.802 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:18 smithi083 ceph-mon[29421]: 5.d scrub starts 2024-09-20T18:55:18.802 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:18 smithi083 ceph-mon[29421]: 5.d scrub ok 2024-09-20T18:55:18.802 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:18 smithi083 ceph-mon[36260]: pgmap v255: 148 pgs: 148 active+clean; 4.2 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.6 KiB/s rd, 143 MiB/s wr, 2.28k op/s 2024-09-20T18:55:18.803 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:18 smithi083 ceph-mon[36260]: 5.d scrub starts 2024-09-20T18:55:18.803 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:18 smithi083 ceph-mon[36260]: 5.d scrub ok 2024-09-20T18:55:18.852 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:18.878 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:18.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:18.879 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:18.980 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:19.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:18 smithi142 ceph-mon[32508]: pgmap v255: 148 pgs: 148 active+clean; 4.2 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.6 KiB/s rd, 143 MiB/s wr, 2.28k op/s 2024-09-20T18:55:19.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:18 smithi142 ceph-mon[32508]: 5.d scrub starts 2024-09-20T18:55:19.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:18 smithi142 ceph-mon[32508]: 5.d scrub ok 2024-09-20T18:55:19.023 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:19.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:19.024 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:19.125 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:19.149 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:19.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:19.150 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:19.172 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_blocked_ops 2024-09-20T18:55:19.175 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 39 16 72878 72862 116.701 88.1875 0.00640717 0.00856403 2024-09-20T18:55:19.251 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:55:19.307 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:19.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:19.308 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:55:19.408 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:19.431 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:19.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:19.432 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:19.533 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:19.540 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-20T18:55:19.540 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T18:55:19.540 INFO:tasks.thrashosds.thrasher:Removing osd 6, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-20T18:55:19.540 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd out 6 2024-09-20T18:55:19.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:19.570 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:19.570 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:19.671 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:55:19.728 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:19.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:19.729 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:55:19.802 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:19 smithi083 ceph-mon[29421]: 4.f scrub starts 2024-09-20T18:55:19.802 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:19 smithi083 ceph-mon[29421]: 4.f scrub ok 2024-09-20T18:55:19.802 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:19 smithi083 ceph-mon[29421]: pgmap v256: 148 pgs: 148 active+clean; 4.2 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 639 B/s rd, 100 MiB/s wr, 1.60k op/s 2024-09-20T18:55:19.802 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:19 smithi083 ceph-mon[36260]: 4.f scrub starts 2024-09-20T18:55:19.803 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:19 smithi083 ceph-mon[36260]: 4.f scrub ok 2024-09-20T18:55:19.803 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:19 smithi083 ceph-mon[36260]: pgmap v256: 148 pgs: 148 active+clean; 4.2 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 639 B/s rd, 100 MiB/s wr, 1.60k op/s 2024-09-20T18:55:19.830 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:19.867 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:19.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:19.868 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:19.969 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:20.012 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:20.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:20.013 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:20.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:19 smithi142 ceph-mon[32508]: 4.f scrub starts 2024-09-20T18:55:20.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:19 smithi142 ceph-mon[32508]: 4.f scrub ok 2024-09-20T18:55:20.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:19 smithi142 ceph-mon[32508]: pgmap v256: 148 pgs: 148 active+clean; 4.2 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 639 B/s rd, 100 MiB/s wr, 1.60k op/s 2024-09-20T18:55:20.114 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:20.138 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:20.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:20.139 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:20.176 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:2024-09-20T18:55:20.162243+0000 min lat: 0.00111821 max lat: 0.124999 avg lat: 0.00863417 2024-09-20T18:55:20.176 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-20T18:55:20.176 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 40 16 74139 74123 115.752 78.8125 0.0111645 0.00863417 2024-09-20T18:55:20.240 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:55:20.276 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:20.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:20.277 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:55:20.377 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:20.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:20.400 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:20.400 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:20.501 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:55:20.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:20.537 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:20.538 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:55:20.638 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:55:20.674 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:20.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:20.675 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:55:20.776 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:20.811 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:20.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:20.812 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:20.913 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:20.948 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:20.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:20.949 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:21.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:20 smithi142 ceph-mon[32508]: 3.16 scrub starts 2024-09-20T18:55:21.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:20 smithi142 ceph-mon[32508]: 3.16 scrub ok 2024-09-20T18:55:21.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:20 smithi142 ceph-mon[32508]: 4.2 scrub starts 2024-09-20T18:55:21.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:20 smithi142 ceph-mon[32508]: 4.2 scrub ok 2024-09-20T18:55:21.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:20 smithi142 ceph-mon[32508]: 4.1e scrub starts 2024-09-20T18:55:21.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:20 smithi142 ceph-mon[32508]: 4.1e scrub ok 2024-09-20T18:55:21.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:20 smithi142 ceph-mon[32508]: 4.10 scrub starts 2024-09-20T18:55:21.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:20 smithi142 ceph-mon[32508]: 4.10 scrub ok 2024-09-20T18:55:21.023 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:20 smithi142 ceph-mon[32508]: osdmap e176: 8 total, 8 up, 8 in 2024-09-20T18:55:21.049 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:55:21.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:20 smithi083 ceph-mon[29421]: 3.16 scrub starts 2024-09-20T18:55:21.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:20 smithi083 ceph-mon[29421]: 3.16 scrub ok 2024-09-20T18:55:21.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:20 smithi083 ceph-mon[29421]: 4.2 scrub starts 2024-09-20T18:55:21.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:20 smithi083 ceph-mon[29421]: 4.2 scrub ok 2024-09-20T18:55:21.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:20 smithi083 ceph-mon[29421]: 4.1e scrub starts 2024-09-20T18:55:21.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:20 smithi083 ceph-mon[29421]: 4.1e scrub ok 2024-09-20T18:55:21.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:20 smithi083 ceph-mon[29421]: 4.10 scrub starts 2024-09-20T18:55:21.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:20 smithi083 ceph-mon[29421]: 4.10 scrub ok 2024-09-20T18:55:21.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:20 smithi083 ceph-mon[29421]: osdmap e176: 8 total, 8 up, 8 in 2024-09-20T18:55:21.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:20 smithi083 ceph-mon[36260]: 3.16 scrub starts 2024-09-20T18:55:21.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:20 smithi083 ceph-mon[36260]: 3.16 scrub ok 2024-09-20T18:55:21.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:20 smithi083 ceph-mon[36260]: 4.2 scrub starts 2024-09-20T18:55:21.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:20 smithi083 ceph-mon[36260]: 4.2 scrub ok 2024-09-20T18:55:21.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:20 smithi083 ceph-mon[36260]: 4.1e scrub starts 2024-09-20T18:55:21.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:20 smithi083 ceph-mon[36260]: 4.1e scrub ok 2024-09-20T18:55:21.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:20 smithi083 ceph-mon[36260]: 4.10 scrub starts 2024-09-20T18:55:21.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:20 smithi083 ceph-mon[36260]: 4.10 scrub ok 2024-09-20T18:55:21.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:20 smithi083 ceph-mon[36260]: osdmap e176: 8 total, 8 up, 8 in 2024-09-20T18:55:21.054 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:55:20 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:55:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:55:21.075 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:21.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:21.075 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:55:21.176 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:55:21.177 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 41 16 75239 75223 114.605 68.75 0.0137699 0.00872129 2024-09-20T18:55:21.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:21.210 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:21.210 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:55:21.311 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:21.333 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:21.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:21.334 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:21.435 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:21.468 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:21.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:21.469 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:21.569 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:55:21.605 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:21.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:21.606 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:55:21.707 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:55:21.742 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:21.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:21.743 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:55:21.843 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:21.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:21.878 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:21.878 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:21.979 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:55:22.019 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:22.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:22.020 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:55:22.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:21 smithi142 ceph-mon[32508]: pgmap v258: 148 pgs: 148 active+clean; 4.3 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 64 MiB/s wr, 1.03k op/s 2024-09-20T18:55:22.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:21 smithi083 ceph-mon[29421]: pgmap v258: 148 pgs: 148 active+clean; 4.3 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 64 MiB/s wr, 1.03k op/s 2024-09-20T18:55:22.052 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:21 smithi083 ceph-mon[36260]: pgmap v258: 148 pgs: 148 active+clean; 4.3 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 64 MiB/s wr, 1.03k op/s 2024-09-20T18:55:22.121 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:22.153 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:22.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:22.154 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:22.176 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 42 16 76368 76352 113.557 70.5625 0.00862029 0.00880125 2024-09-20T18:55:22.256 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:22.286 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:22.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:22.287 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:22.388 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:22.420 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:22.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:22.421 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:22.521 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:22.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:22.556 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:22.556 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:22.658 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:22.694 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:22.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:22.695 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:22.796 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:22.825 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:22.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:22.826 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:22.927 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:22.967 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:22.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:22.968 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:23.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:22 smithi142 ceph-mon[32508]: 6.4 scrub starts 2024-09-20T18:55:23.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:22 smithi142 ceph-mon[32508]: 6.4 scrub ok 2024-09-20T18:55:23.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:22 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:55:23.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:22 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:55:23.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:22 smithi083 ceph-mon[29421]: 6.4 scrub starts 2024-09-20T18:55:23.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:22 smithi083 ceph-mon[29421]: 6.4 scrub ok 2024-09-20T18:55:23.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:22 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:55:23.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:22 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:55:23.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:22 smithi083 ceph-mon[36260]: 6.4 scrub starts 2024-09-20T18:55:23.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:22 smithi083 ceph-mon[36260]: 6.4 scrub ok 2024-09-20T18:55:23.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:22 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:55:23.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:22 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:55:23.069 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:23.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:23.101 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:23.101 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:23.177 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 43 16 77688 77672 112.834 82.5 0.0115588 0.00885816 2024-09-20T18:55:23.202 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:23.228 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:23.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:23.229 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:23.330 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:23.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:23.364 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:23.365 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:23.466 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:23.488 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:23.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:23.489 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:23.590 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:23.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:23.615 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:23.615 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:23.716 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:23.748 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:23.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:23.749 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:23.850 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:23.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:23.888 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:23.888 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:23.989 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:24.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:23 smithi142 ceph-mon[32508]: osdmap e177: 8 total, 8 up, 8 in 2024-09-20T18:55:24.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:23 smithi142 ceph-mon[32508]: pgmap v260: 148 pgs: 148 active+clean; 4.6 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.4 KiB/s rd, 114 MiB/s wr, 1.83k op/s 2024-09-20T18:55:24.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:23 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:55:24.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:23 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:55:24.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:24.025 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:24.025 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:24.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:23 smithi083 ceph-mon[29421]: osdmap e177: 8 total, 8 up, 8 in 2024-09-20T18:55:24.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:23 smithi083 ceph-mon[29421]: pgmap v260: 148 pgs: 148 active+clean; 4.6 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.4 KiB/s rd, 114 MiB/s wr, 1.83k op/s 2024-09-20T18:55:24.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:23 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:55:24.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:23 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:55:24.052 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:23 smithi083 ceph-mon[36260]: osdmap e177: 8 total, 8 up, 8 in 2024-09-20T18:55:24.052 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:23 smithi083 ceph-mon[36260]: pgmap v260: 148 pgs: 148 active+clean; 4.6 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.4 KiB/s rd, 114 MiB/s wr, 1.83k op/s 2024-09-20T18:55:24.052 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:23 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:55:24.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:23 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:55:24.126 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:24.160 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:24.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:24.161 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:24.177 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 44 16 78847 78831 111.916 72.4375 0.00504874 0.0089301 2024-09-20T18:55:24.262 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:24.299 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:24.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:24.300 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:24.401 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:24.434 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:24.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:24.435 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:24.495 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:55:24.535 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:24.536 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:55:24.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:24.569 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:24.569 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:24.670 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:24.714 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:24.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:24.716 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:24.767 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-20T18:55:24.817 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:55:24.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:24.858 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:24.858 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:55:24.901 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:24 smithi083 ceph-mon[29421]: osdmap e178: 8 total, 8 up, 8 in 2024-09-20T18:55:24.901 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:24 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3972998274' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:55:24.902 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:24 smithi083 ceph-mon[36260]: osdmap e178: 8 total, 8 up, 8 in 2024-09-20T18:55:24.902 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:24 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3972998274' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:55:24.959 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:24.989 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:24.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:24.990 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:25.090 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:25.107 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T18:55:25.107 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:55:25.119 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:25.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:25.120 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:25.156 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T18:55:25.177 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 45 16 80090 80074 111.155 77.6875 0.00793735 0.00899157 2024-09-20T18:55:25.229 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:25.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:25.257 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:25.258 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:25.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:24 smithi142 ceph-mon[32508]: osdmap e178: 8 total, 8 up, 8 in 2024-09-20T18:55:25.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:24 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3972998274' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:55:25.358 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:55:25.404 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:25.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:25.405 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:55:25.506 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:25.541 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:25.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:25.542 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:25.643 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:25.670 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:25.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:25.672 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:25.772 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:25.810 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:25.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:25.811 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:25.912 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:25.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:25.942 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:25.942 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:26.021 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:25 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:55:26.021 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:25 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:55:26.021 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:25 smithi083 ceph-mon[29421]: 3.f scrub starts 2024-09-20T18:55:26.021 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:25 smithi083 ceph-mon[29421]: 3.f scrub ok 2024-09-20T18:55:26.021 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:25 smithi083 ceph-mon[29421]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T18:55:26.021 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:25 smithi083 ceph-mon[29421]: Cluster is now healthy 2024-09-20T18:55:26.022 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:25 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3972998274' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T18:55:26.022 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:25 smithi083 ceph-mon[29421]: osdmap e179: 8 total, 8 up, 8 in 2024-09-20T18:55:26.022 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:25 smithi083 ceph-mon[29421]: pgmap v263: 148 pgs: 148 active+clean; 4.6 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 92 MiB/s wr, 1.48k op/s 2024-09-20T18:55:26.022 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:25 smithi083 ceph-mon[29421]: osdmap e180: 8 total, 8 up, 8 in 2024-09-20T18:55:26.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:25 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:55:26.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:25 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:55:26.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:25 smithi083 ceph-mon[36260]: 3.f scrub starts 2024-09-20T18:55:26.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:25 smithi083 ceph-mon[36260]: 3.f scrub ok 2024-09-20T18:55:26.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:25 smithi083 ceph-mon[36260]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T18:55:26.023 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:25 smithi083 ceph-mon[36260]: Cluster is now healthy 2024-09-20T18:55:26.023 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:25 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3972998274' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T18:55:26.023 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:25 smithi083 ceph-mon[36260]: osdmap e179: 8 total, 8 up, 8 in 2024-09-20T18:55:26.023 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:25 smithi083 ceph-mon[36260]: pgmap v263: 148 pgs: 148 active+clean; 4.6 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 92 MiB/s wr, 1.48k op/s 2024-09-20T18:55:26.023 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:25 smithi083 ceph-mon[36260]: osdmap e180: 8 total, 8 up, 8 in 2024-09-20T18:55:26.042 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:26.072 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:26.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:26.073 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:26.174 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:26.177 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 46 16 81349 81333 110.45 78.6875 0.0131669 0.00904933 2024-09-20T18:55:26.207 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:26.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:26.208 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:26.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:25 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:55:26.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:25 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:55:26.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:25 smithi142 ceph-mon[32508]: 3.f scrub starts 2024-09-20T18:55:26.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:25 smithi142 ceph-mon[32508]: 3.f scrub ok 2024-09-20T18:55:26.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:25 smithi142 ceph-mon[32508]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T18:55:26.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:25 smithi142 ceph-mon[32508]: Cluster is now healthy 2024-09-20T18:55:26.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:25 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3972998274' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T18:55:26.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:25 smithi142 ceph-mon[32508]: osdmap e179: 8 total, 8 up, 8 in 2024-09-20T18:55:26.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:25 smithi142 ceph-mon[32508]: pgmap v263: 148 pgs: 148 active+clean; 4.6 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 92 MiB/s wr, 1.48k op/s 2024-09-20T18:55:26.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:25 smithi142 ceph-mon[32508]: osdmap e180: 8 total, 8 up, 8 in 2024-09-20T18:55:26.309 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:26.345 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:26.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:26.346 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:26.447 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:26.479 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:26.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:26.481 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:26.581 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:26.608 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:26.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:26.609 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:26.710 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:26.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:26.741 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:26.742 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:26.842 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:55:26.887 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:26.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:26.888 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:55:26.988 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:26.992 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:26 smithi083 ceph-mon[29421]: 6.7 scrub starts 2024-09-20T18:55:26.992 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:26 smithi083 ceph-mon[29421]: 6.7 scrub ok 2024-09-20T18:55:26.992 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:26 smithi083 ceph-mon[29421]: 4.7 scrub starts 2024-09-20T18:55:26.992 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:26 smithi083 ceph-mon[29421]: 4.7 scrub ok 2024-09-20T18:55:26.992 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:26 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:55:26.993 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:26 smithi083 ceph-mon[36260]: 6.7 scrub starts 2024-09-20T18:55:26.993 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:26 smithi083 ceph-mon[36260]: 6.7 scrub ok 2024-09-20T18:55:26.993 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:26 smithi083 ceph-mon[36260]: 4.7 scrub starts 2024-09-20T18:55:26.993 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:26 smithi083 ceph-mon[36260]: 4.7 scrub ok 2024-09-20T18:55:26.993 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:26 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:55:27.016 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:27.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:27.017 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:27.021 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:55:26 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:55:27.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:26 smithi142 ceph-mon[32508]: 6.7 scrub starts 2024-09-20T18:55:27.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:26 smithi142 ceph-mon[32508]: 6.7 scrub ok 2024-09-20T18:55:27.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:26 smithi142 ceph-mon[32508]: 4.7 scrub starts 2024-09-20T18:55:27.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:26 smithi142 ceph-mon[32508]: 4.7 scrub ok 2024-09-20T18:55:27.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:26 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:55:27.119 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:27.134 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_historic_ops 2024-09-20T18:55:27.158 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:27.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:27.159 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:27.177 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 47 16 82579 82563 109.735 76.875 0.0167792 0.00910774 2024-09-20T18:55:27.202 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:55:27.260 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:27.286 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:27.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:27.287 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:27.301 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:26 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mon-a[29417]: 2024-09-20T18:55:26.975+0000 7f6c091c6640 -1 mon.a@0(leader).osd e180 definitely_dead 0 2024-09-20T18:55:27.387 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:27.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:27.413 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:27.414 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:27.514 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:55:27.565 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:27.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:27.567 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:55:27.667 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:55:27.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:27.706 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:27.706 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:55:27.807 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:27.808 INFO:teuthology.orchestra.run.smithi083.stderr:marked out osd.6. 2024-09-20T18:55:27.842 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:27.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:27.843 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:27.887 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:55:27.887 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:55:27.904 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:55:27.904 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:55:27.920 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:55:27.920 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:55:27.935 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:55:27.936 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:55:27.944 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:27.952 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:55:27.952 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:55:27.953 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:27 smithi083 ceph-mon[29421]: 3.10 scrub starts 2024-09-20T18:55:27.953 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:27 smithi083 ceph-mon[29421]: 3.10 scrub ok 2024-09-20T18:55:27.953 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:27 smithi083 ceph-mon[29421]: pgmap v265: 148 pgs: 148 active+clean; 5.0 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 93 MiB/s wr, 1.50k op/s 2024-09-20T18:55:27.953 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:27 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2324101095' entity='client.admin' cmd={"prefix": "osd out", "ids": ["6"]} : dispatch 2024-09-20T18:55:27.953 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:27 smithi083 ceph-mon[29421]: Client client.admin marked osd.6 out, while it was still marked up 2024-09-20T18:55:27.953 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:27 smithi083 ceph-mon[29421]: 3.12 scrub starts 2024-09-20T18:55:27.953 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:27 smithi083 ceph-mon[29421]: 3.12 scrub ok 2024-09-20T18:55:27.953 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:27 smithi083 ceph-mon[29421]: 5.14 scrub starts 2024-09-20T18:55:27.953 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:27 smithi083 ceph-mon[29421]: 5.14 scrub ok 2024-09-20T18:55:27.954 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:27 smithi083 ceph-mon[36260]: 3.10 scrub starts 2024-09-20T18:55:27.954 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:27 smithi083 ceph-mon[36260]: 3.10 scrub ok 2024-09-20T18:55:27.954 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:27 smithi083 ceph-mon[36260]: pgmap v265: 148 pgs: 148 active+clean; 5.0 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 93 MiB/s wr, 1.50k op/s 2024-09-20T18:55:27.954 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:27 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2324101095' entity='client.admin' cmd={"prefix": "osd out", "ids": ["6"]} : dispatch 2024-09-20T18:55:27.954 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:27 smithi083 ceph-mon[36260]: Client client.admin marked osd.6 out, while it was still marked up 2024-09-20T18:55:27.954 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:27 smithi083 ceph-mon[36260]: 3.12 scrub starts 2024-09-20T18:55:27.954 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:27 smithi083 ceph-mon[36260]: 3.12 scrub ok 2024-09-20T18:55:27.954 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:27 smithi083 ceph-mon[36260]: 5.14 scrub starts 2024-09-20T18:55:27.954 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:27 smithi083 ceph-mon[36260]: 5.14 scrub ok 2024-09-20T18:55:27.966 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:27.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:27.967 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:27.969 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:55:27.969 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:55:27.985 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:55:27.985 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:55:28.002 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:55:28.002 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:55:28.067 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:28.071 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T18:55:28.088 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:28.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:28.089 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:28.177 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 48 16 83340 83324 108.441 47.5625 0.00281906 0.00915598 2024-09-20T18:55:28.190 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:28.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:28.211 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:28.212 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:28.271 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:27 smithi142 ceph-mon[32508]: 3.10 scrub starts 2024-09-20T18:55:28.271 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:27 smithi142 ceph-mon[32508]: 3.10 scrub ok 2024-09-20T18:55:28.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:27 smithi142 ceph-mon[32508]: pgmap v265: 148 pgs: 148 active+clean; 5.0 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 93 MiB/s wr, 1.50k op/s 2024-09-20T18:55:28.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:27 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2324101095' entity='client.admin' cmd={"prefix": "osd out", "ids": ["6"]} : dispatch 2024-09-20T18:55:28.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:27 smithi142 ceph-mon[32508]: Client client.admin marked osd.6 out, while it was still marked up 2024-09-20T18:55:28.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:27 smithi142 ceph-mon[32508]: 3.12 scrub starts 2024-09-20T18:55:28.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:27 smithi142 ceph-mon[32508]: 3.12 scrub ok 2024-09-20T18:55:28.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:27 smithi142 ceph-mon[32508]: 5.14 scrub starts 2024-09-20T18:55:28.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:27 smithi142 ceph-mon[32508]: 5.14 scrub ok 2024-09-20T18:55:28.313 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:55:28.334 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:28.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:28.335 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:55:28.436 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:28.456 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:28.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:28.458 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:28.558 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:28.580 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:28.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:28.581 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:28.682 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:28.705 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:28.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:28.706 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:28.798 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:55:28.808 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:28.842 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:28.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:28.843 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:28.946 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:28.968 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:28.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:28.969 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:29.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:28 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2324101095' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["6"]}]': finished 2024-09-20T18:55:29.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:28 smithi083 ceph-mon[29421]: osdmap e181: 8 total, 8 up, 7 in 2024-09-20T18:55:29.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:28 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2357094381' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:55:29.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:28 smithi083 ceph-mon[29421]: 3.1f scrub starts 2024-09-20T18:55:29.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:28 smithi083 ceph-mon[29421]: 3.1f scrub ok 2024-09-20T18:55:29.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:28 smithi083 ceph-mon[29421]: 3.18 scrub starts 2024-09-20T18:55:29.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:28 smithi083 ceph-mon[29421]: 3.18 scrub ok 2024-09-20T18:55:29.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:28 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2324101095' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["6"]}]': finished 2024-09-20T18:55:29.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:28 smithi083 ceph-mon[36260]: osdmap e181: 8 total, 8 up, 7 in 2024-09-20T18:55:29.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:28 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2357094381' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:55:29.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:28 smithi083 ceph-mon[36260]: 3.1f scrub starts 2024-09-20T18:55:29.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:28 smithi083 ceph-mon[36260]: 3.1f scrub ok 2024-09-20T18:55:29.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:28 smithi083 ceph-mon[36260]: 3.18 scrub starts 2024-09-20T18:55:29.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:28 smithi083 ceph-mon[36260]: 3.18 scrub ok 2024-09-20T18:55:29.069 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:29.091 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:29.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:29.092 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:29.177 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 49 16 83340 83324 106.228 0 - 0.00915598 2024-09-20T18:55:29.193 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:29.216 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:29.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:29.217 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:29.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:28 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2324101095' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["6"]}]': finished 2024-09-20T18:55:29.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:28 smithi142 ceph-mon[32508]: osdmap e181: 8 total, 8 up, 7 in 2024-09-20T18:55:29.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:28 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2357094381' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:55:29.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:28 smithi142 ceph-mon[32508]: 3.1f scrub starts 2024-09-20T18:55:29.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:28 smithi142 ceph-mon[32508]: 3.1f scrub ok 2024-09-20T18:55:29.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:28 smithi142 ceph-mon[32508]: 3.18 scrub starts 2024-09-20T18:55:29.273 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:28 smithi142 ceph-mon[32508]: 3.18 scrub ok 2024-09-20T18:55:29.317 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:29.339 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:29.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:29.340 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:29.441 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:29.462 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:29.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:29.463 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:29.563 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:55:29.585 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:29.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:29.586 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:55:29.687 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:29.706 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T18:55:29.706 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-20T18:55:29.707 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-20T18:55:29.707 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-20T18:55:29.707 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.707 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14442.0:1 3.12 3:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-20T18:55:29.707 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:49:45.769138+0000", 2024-09-20T18:55:29.707 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 343.92169849099997, 2024-09-20T18:55:29.707 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.16996755799999999, 2024-09-20T18:55:29.707 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:55:29.707 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:55:29.707 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-20T18:55:29.707 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:55:29.707 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14442", 2024-09-20T18:55:29.707 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/3586269208", 2024-09-20T18:55:29.708 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 1 2024-09-20T18:55:29.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.708 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:55:29.708 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.708 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:55:29.708 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:49:45.769138+0000", 2024-09-20T18:55:29.708 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:29.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.708 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.708 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:55:29.708 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:49:45.769138+0000", 2024-09-20T18:55:29.708 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:29.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.708 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.709 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:55:29.709 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:49:45.769145+0000", 2024-09-20T18:55:29.709 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8450000000000003e-06 2024-09-20T18:55:29.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.709 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.709 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:55:29.709 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:49:45.769151+0000", 2024-09-20T18:55:29.709 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2989999999999996e-06 2024-09-20T18:55:29.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.709 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.709 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:55:29.709 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:49:45.769160+0000", 2024-09-20T18:55:29.709 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0089999999999996e-06 2024-09-20T18:55:29.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.710 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.710 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:55:29.710 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:49:45.769168+0000", 2024-09-20T18:55:29.710 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5490000000000006e-06 2024-09-20T18:55:29.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.710 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.710 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:55:29.710 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:49:45.769240+0000", 2024-09-20T18:55:29.710 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2081000000000003e-05 2024-09-20T18:55:29.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.710 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.710 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:55:29.711 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:49:45.769247+0000", 2024-09-20T18:55:29.711 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5309999999999998e-06 2024-09-20T18:55:29.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.711 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.711 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:55:29.711 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:49:45.939030+0000", 2024-09-20T18:55:29.711 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.169782715 2024-09-20T18:55:29.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.711 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.711 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:55:29.711 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:49:45.939105+0000", 2024-09-20T18:55:29.711 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5529000000000001e-05 2024-09-20T18:55:29.711 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:29.712 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:55:29.712 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:29.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.712 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.712 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:25335 7.6 7:63a06433:::benchmark_data_smithi083_70643_object25334:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:55:29.712 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.266613+0000", 2024-09-20T18:55:29.712 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 262.42422263499998, 2024-09-20T18:55:29.712 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17508610599999999, 2024-09-20T18:55:29.712 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:55:29.712 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:55:29.712 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:29.712 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:55:29.712 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:55:29.713 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:29.713 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25335 2024-09-20T18:55:29.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.713 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:55:29.713 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.713 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:55:29.713 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266613+0000", 2024-09-20T18:55:29.713 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:29.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.713 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.713 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:55:29.713 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266613+0000", 2024-09-20T18:55:29.713 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:29.714 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.714 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.714 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:55:29.714 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266615+0000", 2024-09-20T18:55:29.714 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2240000000000001e-06 2024-09-20T18:55:29.714 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.714 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.714 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:55:29.714 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266641+0000", 2024-09-20T18:55:29.714 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6848000000000001e-05 2024-09-20T18:55:29.714 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.714 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.714 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:55:29.714 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266650+0000", 2024-09-20T18:55:29.715 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.3569999999999998e-06 2024-09-20T18:55:29.715 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.715 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.715 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:55:29.715 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266656+0000", 2024-09-20T18:55:29.715 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0839999999999997e-06 2024-09-20T18:55:29.715 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.715 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.715 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:55:29.715 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266923+0000", 2024-09-20T18:55:29.715 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00026740999999999999 2024-09-20T18:55:29.715 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.715 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.716 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:55:29.716 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.267055+0000", 2024-09-20T18:55:29.716 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013152800000000001 2024-09-20T18:55:29.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.716 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.716 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:55:29.716 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.267176+0000", 2024-09-20T18:55:29.716 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012149799999999999 2024-09-20T18:55:29.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.716 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.716 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:55:29.716 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269576+0000", 2024-09-20T18:55:29.716 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002400157 2024-09-20T18:55:29.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.717 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.717 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:29.717 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441551+0000", 2024-09-20T18:55:29.717 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17197433400000001 2024-09-20T18:55:29.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.717 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.717 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:29.717 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441597+0000", 2024-09-20T18:55:29.717 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5986999999999999e-05 2024-09-20T18:55:29.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.717 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.717 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:55:29.717 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441650+0000", 2024-09-20T18:55:29.717 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.325e-05 2024-09-20T18:55:29.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.718 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.718 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:55:29.718 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441699+0000", 2024-09-20T18:55:29.718 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9428999999999997e-05 2024-09-20T18:55:29.718 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:29.718 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:55:29.718 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:29.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.718 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.718 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:25337 7.f 7:fd2b3e49:::benchmark_data_smithi083_70643_object25336:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:55:29.719 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.268131+0000", 2024-09-20T18:55:29.719 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 262.42270540599998, 2024-09-20T18:55:29.719 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17632631300000001, 2024-09-20T18:55:29.719 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:55:29.719 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:55:29.719 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:29.719 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:55:29.720 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:55:29.720 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:29.720 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25337 2024-09-20T18:55:29.720 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.720 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:55:29.720 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.720 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:55:29.720 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268131+0000", 2024-09-20T18:55:29.720 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:29.720 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.720 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.720 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:55:29.720 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268131+0000", 2024-09-20T18:55:29.720 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:29.721 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.721 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.721 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:55:29.721 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268132+0000", 2024-09-20T18:55:29.721 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3659999999999999e-06 2024-09-20T18:55:29.721 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.721 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.721 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:55:29.721 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268162+0000", 2024-09-20T18:55:29.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:29.723 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9572000000000001e-05 2024-09-20T18:55:29.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.723 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.723 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:55:29.723 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268173+0000", 2024-09-20T18:55:29.723 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.132e-05 2024-09-20T18:55:29.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.723 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.723 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:55:29.723 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268200+0000", 2024-09-20T18:55:29.723 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7081999999999999e-05 2024-09-20T18:55:29.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.724 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.724 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:55:29.724 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268365+0000", 2024-09-20T18:55:29.724 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016553299999999999 2024-09-20T18:55:29.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.724 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.724 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:55:29.724 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268480+0000", 2024-09-20T18:55:29.724 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000114595 2024-09-20T18:55:29.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.724 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.724 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:55:29.724 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268598+0000", 2024-09-20T18:55:29.725 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000117909 2024-09-20T18:55:29.725 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.725 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.725 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:55:29.725 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270238+0000", 2024-09-20T18:55:29.725 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001639604 2024-09-20T18:55:29.725 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.725 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.725 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:29.725 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441916+0000", 2024-09-20T18:55:29.725 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.171678468 2024-09-20T18:55:29.725 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.725 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.726 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:29.726 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444363+0000", 2024-09-20T18:55:29.726 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0024469090000000001 2024-09-20T18:55:29.726 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.726 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.726 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:55:29.726 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444423+0000", 2024-09-20T18:55:29.726 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9645999999999998e-05 2024-09-20T18:55:29.726 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.726 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.726 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:55:29.726 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444457+0000", 2024-09-20T18:55:29.727 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4309e-05 2024-09-20T18:55:29.727 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:29.727 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:55:29.727 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:29.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.727 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.727 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:25338 7.9 7:9270090d:::benchmark_data_smithi083_70643_object25337:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:55:29.727 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.268223+0000", 2024-09-20T18:55:29.727 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 262.42261344799999, 2024-09-20T18:55:29.727 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.173517488, 2024-09-20T18:55:29.727 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:55:29.727 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:55:29.727 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:29.727 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:55:29.728 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:55:29.728 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:29.728 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25338 2024-09-20T18:55:29.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.728 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:55:29.728 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.728 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:55:29.728 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268223+0000", 2024-09-20T18:55:29.728 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:29.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.728 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.728 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:55:29.728 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268223+0000", 2024-09-20T18:55:29.728 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:29.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.729 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.729 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:55:29.729 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268223+0000", 2024-09-20T18:55:29.729 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2799999999999996e-07 2024-09-20T18:55:29.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.729 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.729 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:55:29.729 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268277+0000", 2024-09-20T18:55:29.729 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4076000000000002e-05 2024-09-20T18:55:29.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.729 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.729 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:55:29.730 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268284+0000", 2024-09-20T18:55:29.730 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1849999999999998e-06 2024-09-20T18:55:29.730 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.730 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.730 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:55:29.730 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268290+0000", 2024-09-20T18:55:29.730 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0390000000000001e-06 2024-09-20T18:55:29.730 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.730 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.730 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:55:29.730 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268372+0000", 2024-09-20T18:55:29.730 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.1027999999999997e-05 2024-09-20T18:55:29.730 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.731 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.731 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:55:29.731 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268482+0000", 2024-09-20T18:55:29.731 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000110979 2024-09-20T18:55:29.731 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.731 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.731 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,4", 2024-09-20T18:55:29.731 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268615+0000", 2024-09-20T18:55:29.731 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013288199999999999 2024-09-20T18:55:29.731 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.731 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.731 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:29.731 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271414+0000", 2024-09-20T18:55:29.731 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0027988100000000001 2024-09-20T18:55:29.732 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.732 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.732 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:55:29.732 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271646+0000", 2024-09-20T18:55:29.732 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000231449 2024-09-20T18:55:29.732 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.732 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.732 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:29.732 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271757+0000", 2024-09-20T18:55:29.732 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011127400000000001 2024-09-20T18:55:29.732 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.732 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.732 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:55:29.732 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441704+0000", 2024-09-20T18:55:29.733 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.16994677499999999 2024-09-20T18:55:29.733 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.733 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.733 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:55:29.733 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441740+0000", 2024-09-20T18:55:29.733 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6362999999999997e-05 2024-09-20T18:55:29.733 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:29.733 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:55:29.733 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:29.733 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.733 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.733 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:25340 7.6 7:6dd1b43c:::benchmark_data_smithi083_70643_object25339:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:55:29.733 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.268308+0000", 2024-09-20T18:55:29.734 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 262.42252752399997, 2024-09-20T18:55:29.734 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17672299299999999, 2024-09-20T18:55:29.734 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:55:29.734 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:55:29.734 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:29.734 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:55:29.734 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:55:29.734 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:29.734 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25340 2024-09-20T18:55:29.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.734 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:55:29.734 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.734 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:55:29.735 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268308+0000", 2024-09-20T18:55:29.735 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:29.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.735 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.735 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:55:29.735 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268308+0000", 2024-09-20T18:55:29.735 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:29.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.735 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.735 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:55:29.735 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268309+0000", 2024-09-20T18:55:29.735 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3499999999999996e-07 2024-09-20T18:55:29.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.735 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.735 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:55:29.736 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268332+0000", 2024-09-20T18:55:29.736 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2741000000000001e-05 2024-09-20T18:55:29.736 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.736 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.736 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:55:29.736 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268341+0000", 2024-09-20T18:55:29.736 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7630000000000006e-06 2024-09-20T18:55:29.736 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.736 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.736 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:55:29.736 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268346+0000", 2024-09-20T18:55:29.736 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7790000000000001e-06 2024-09-20T18:55:29.736 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.736 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.736 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:55:29.736 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268450+0000", 2024-09-20T18:55:29.737 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000103745 2024-09-20T18:55:29.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.737 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.737 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:55:29.737 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268553+0000", 2024-09-20T18:55:29.737 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000102575 2024-09-20T18:55:29.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.737 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.737 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:55:29.737 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268667+0000", 2024-09-20T18:55:29.737 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000114449 2024-09-20T18:55:29.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.737 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.737 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:55:29.737 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270201+0000", 2024-09-20T18:55:29.738 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001533834 2024-09-20T18:55:29.738 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.738 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.738 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:29.738 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441748+0000", 2024-09-20T18:55:29.738 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.171546802 2024-09-20T18:55:29.738 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.738 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.738 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:29.738 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444881+0000", 2024-09-20T18:55:29.738 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0031329589999999998 2024-09-20T18:55:29.738 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.738 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.738 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:55:29.739 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444940+0000", 2024-09-20T18:55:29.739 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9416999999999997e-05 2024-09-20T18:55:29.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.739 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.739 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:55:29.739 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445031+0000", 2024-09-20T18:55:29.739 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1393999999999999e-05 2024-09-20T18:55:29.739 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:29.739 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:55:29.739 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:29.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.739 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.739 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:25343 7.6 7:61f2dd3b:::benchmark_data_smithi083_70643_object25342:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:55:29.739 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.270643+0000", 2024-09-20T18:55:29.740 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 262.42019286499999, 2024-09-20T18:55:29.740 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17748241200000001, 2024-09-20T18:55:29.740 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:55:29.740 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:55:29.740 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:29.740 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:55:29.740 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:55:29.740 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:29.740 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25343 2024-09-20T18:55:29.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.740 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:55:29.740 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.740 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:55:29.740 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270643+0000", 2024-09-20T18:55:29.741 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:29.741 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.741 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.741 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:55:29.741 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270643+0000", 2024-09-20T18:55:29.741 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:29.741 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.741 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.741 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:55:29.741 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270645+0000", 2024-09-20T18:55:29.741 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4810000000000001e-06 2024-09-20T18:55:29.741 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.741 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.741 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:55:29.741 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270664+0000", 2024-09-20T18:55:29.742 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9709999999999999e-05 2024-09-20T18:55:29.742 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.742 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.742 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:55:29.742 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270674+0000", 2024-09-20T18:55:29.742 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6120000000000005e-06 2024-09-20T18:55:29.742 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.742 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.742 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:55:29.742 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270680+0000", 2024-09-20T18:55:29.742 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7089999999999999e-06 2024-09-20T18:55:29.742 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.742 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.742 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:55:29.743 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270841+0000", 2024-09-20T18:55:29.743 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016147500000000001 2024-09-20T18:55:29.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.743 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.743 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:55:29.743 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271023+0000", 2024-09-20T18:55:29.743 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000181836 2024-09-20T18:55:29.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.743 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.743 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:55:29.743 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271282+0000", 2024-09-20T18:55:29.743 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00025898300000000001 2024-09-20T18:55:29.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.743 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.744 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:55:29.744 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441716+0000", 2024-09-20T18:55:29.744 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17043377800000001 2024-09-20T18:55:29.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.744 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.744 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:29.744 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441787+0000", 2024-09-20T18:55:29.744 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1465999999999996e-05 2024-09-20T18:55:29.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.744 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.744 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:29.744 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.448030+0000", 2024-09-20T18:55:29.744 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0062427089999999999 2024-09-20T18:55:29.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.745 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.745 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:55:29.745 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.448081+0000", 2024-09-20T18:55:29.745 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1255000000000003e-05 2024-09-20T18:55:29.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.745 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.745 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:55:29.745 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.448126+0000", 2024-09-20T18:55:29.745 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4397999999999997e-05 2024-09-20T18:55:29.745 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:29.745 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:55:29.745 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:29.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.746 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.746 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:25344 7.7 e93/83 7:ead30b22:::benchmark_data_smithi083_70643_object25343:head v 93'1577, mlcod=93'1575)", 2024-09-20T18:55:29.746 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.270793+0000", 2024-09-20T18:55:29.746 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 262.42004285799999, 2024-09-20T18:55:29.746 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17078492100000001, 2024-09-20T18:55:29.746 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:55:29.746 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:55:29.746 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:29.746 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:55:29.746 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.746 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:55:29.746 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270793+0000", 2024-09-20T18:55:29.746 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:29.746 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.747 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.747 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:55:29.747 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270793+0000", 2024-09-20T18:55:29.747 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:29.747 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.747 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.747 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:55:29.747 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270794+0000", 2024-09-20T18:55:29.747 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9100000000000004e-07 2024-09-20T18:55:29.747 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.747 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.747 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:55:29.747 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270929+0000", 2024-09-20T18:55:29.747 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000134795 2024-09-20T18:55:29.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.748 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:55:29.748 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270941+0000", 2024-09-20T18:55:29.748 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2431e-05 2024-09-20T18:55:29.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.748 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:55:29.748 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270948+0000", 2024-09-20T18:55:29.748 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8109999999999997e-06 2024-09-20T18:55:29.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.748 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:55:29.748 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271000+0000", 2024-09-20T18:55:29.748 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2122e-05 2024-09-20T18:55:29.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.749 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.749 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:55:29.749 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271019+0000", 2024-09-20T18:55:29.749 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9490000000000001e-05 2024-09-20T18:55:29.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.749 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.749 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:55:29.749 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441531+0000", 2024-09-20T18:55:29.749 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.170511842 2024-09-20T18:55:29.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.749 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.749 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:55:29.749 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441578+0000", 2024-09-20T18:55:29.750 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6839000000000003e-05 2024-09-20T18:55:29.750 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:29.750 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:55:29.750 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:29.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.750 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.750 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:25335 7.6 e93/83 ondisk, result = 0)", 2024-09-20T18:55:29.750 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.270924+0000", 2024-09-20T18:55:29.750 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 262.419912071, 2024-09-20T18:55:29.750 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.170638343, 2024-09-20T18:55:29.750 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:55:29.750 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:55:29.750 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:55:29.750 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:55:29.750 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.751 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:55:29.751 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270924+0000", 2024-09-20T18:55:29.751 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:29.751 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.751 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.751 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:55:29.751 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270924+0000", 2024-09-20T18:55:29.751 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:29.751 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.751 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.751 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:55:29.751 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270925+0000", 2024-09-20T18:55:29.751 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8899999999999999e-07 2024-09-20T18:55:29.751 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.752 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.752 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:55:29.752 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270929+0000", 2024-09-20T18:55:29.752 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9829999999999998e-06 2024-09-20T18:55:29.752 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.752 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.752 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:55:29.752 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270936+0000", 2024-09-20T18:55:29.752 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5900000000000002e-06 2024-09-20T18:55:29.752 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.752 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.752 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:55:29.752 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270942+0000", 2024-09-20T18:55:29.752 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8420000000000001e-06 2024-09-20T18:55:29.752 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.753 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.753 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:55:29.753 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441532+0000", 2024-09-20T18:55:29.753 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17059043900000001 2024-09-20T18:55:29.753 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.753 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.753 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:55:29.753 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441543+0000", 2024-09-20T18:55:29.753 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0549e-05 2024-09-20T18:55:29.753 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.753 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.753 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:55:29.753 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441562+0000", 2024-09-20T18:55:29.753 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9250999999999999e-05 2024-09-20T18:55:29.753 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:29.754 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:55:29.754 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:29.754 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.754 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.754 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:25345 7.4 7:2557809c:::benchmark_data_smithi083_70643_object25344:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:55:29.754 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.270978+0000", 2024-09-20T18:55:29.754 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 262.41985779100003, 2024-09-20T18:55:29.754 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.170924456, 2024-09-20T18:55:29.754 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:55:29.754 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:55:29.754 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:29.754 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:55:29.754 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:55:29.754 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:29.755 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25345 2024-09-20T18:55:29.755 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.755 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:55:29.755 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.755 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:55:29.755 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270978+0000", 2024-09-20T18:55:29.755 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:29.755 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.755 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.755 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:55:29.755 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270978+0000", 2024-09-20T18:55:29.755 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:29.755 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.755 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.755 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:55:29.756 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270979+0000", 2024-09-20T18:55:29.756 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9400000000000004e-07 2024-09-20T18:55:29.756 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.756 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.756 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:55:29.756 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271001+0000", 2024-09-20T18:55:29.756 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1520000000000001e-05 2024-09-20T18:55:29.756 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.756 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.756 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:55:29.756 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271008+0000", 2024-09-20T18:55:29.756 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4699999999999996e-06 2024-09-20T18:55:29.756 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.756 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.756 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:55:29.757 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271015+0000", 2024-09-20T18:55:29.757 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3529999999999997e-06 2024-09-20T18:55:29.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.757 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.757 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:55:29.757 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271108+0000", 2024-09-20T18:55:29.757 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2194999999999994e-05 2024-09-20T18:55:29.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.757 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.757 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:55:29.757 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271218+0000", 2024-09-20T18:55:29.757 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000110797 2024-09-20T18:55:29.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.757 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.758 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,4", 2024-09-20T18:55:29.758 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271334+0000", 2024-09-20T18:55:29.758 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115202 2024-09-20T18:55:29.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.758 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.758 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:29.758 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441629+0000", 2024-09-20T18:55:29.758 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.170295053 2024-09-20T18:55:29.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.758 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.758 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:55:29.758 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441647+0000", 2024-09-20T18:55:29.758 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8465999999999999e-05 2024-09-20T18:55:29.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.758 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.759 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:29.759 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441814+0000", 2024-09-20T18:55:29.759 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016683799999999999 2024-09-20T18:55:29.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.759 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.759 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:55:29.759 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441869+0000", 2024-09-20T18:55:29.759 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4693e-05 2024-09-20T18:55:29.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.759 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.759 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:55:29.759 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441903+0000", 2024-09-20T18:55:29.759 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3974999999999999e-05 2024-09-20T18:55:29.759 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:29.760 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:55:29.760 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:29.760 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.760 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.760 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:25335 7.6 e93/83 ondisk, result = 0)", 2024-09-20T18:55:29.760 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.270989+0000", 2024-09-20T18:55:29.760 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 262.41984724399998, 2024-09-20T18:55:29.760 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17072014499999999, 2024-09-20T18:55:29.760 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:55:29.760 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:55:29.760 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:55:29.760 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:55:29.760 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.760 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:55:29.760 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270989+0000", 2024-09-20T18:55:29.761 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:29.761 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.761 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.761 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:55:29.761 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270989+0000", 2024-09-20T18:55:29.761 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:29.761 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.761 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.761 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:55:29.761 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270990+0000", 2024-09-20T18:55:29.761 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.1200000000000002e-07 2024-09-20T18:55:29.761 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.761 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.761 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:55:29.762 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270994+0000", 2024-09-20T18:55:29.762 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2119999999999997e-06 2024-09-20T18:55:29.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.762 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:55:29.762 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271010+0000", 2024-09-20T18:55:29.762 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6070999999999999e-05 2024-09-20T18:55:29.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.762 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:55:29.762 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271016+0000", 2024-09-20T18:55:29.762 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2859999999999996e-06 2024-09-20T18:55:29.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.762 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:55:29.763 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441586+0000", 2024-09-20T18:55:29.763 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17057012499999999 2024-09-20T18:55:29.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.763 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.763 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:55:29.763 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441592+0000", 2024-09-20T18:55:29.763 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8690000000000001e-06 2024-09-20T18:55:29.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.763 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.763 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:55:29.763 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441709+0000", 2024-09-20T18:55:29.763 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011677 2024-09-20T18:55:29.763 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:29.763 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:55:29.763 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:29.764 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.764 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.764 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:25342 7.5 e93/83 7:ae6f0d57:::benchmark_data_smithi083_70643_object25341:head v 93'1574, mlcod=93'1572)", 2024-09-20T18:55:29.764 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.271037+0000", 2024-09-20T18:55:29.764 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 262.41979908000002, 2024-09-20T18:55:29.764 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.170762196, 2024-09-20T18:55:29.764 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:55:29.764 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:55:29.764 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:29.764 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:55:29.764 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.764 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:55:29.764 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271037+0000", 2024-09-20T18:55:29.764 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:29.765 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.765 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.765 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:55:29.765 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271037+0000", 2024-09-20T18:55:29.765 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:29.765 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.765 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.765 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:55:29.765 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271037+0000", 2024-09-20T18:55:29.765 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9700000000000002e-07 2024-09-20T18:55:29.765 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.765 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.765 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:55:29.765 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271069+0000", 2024-09-20T18:55:29.766 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1647999999999999e-05 2024-09-20T18:55:29.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.766 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.766 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:55:29.766 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271075+0000", 2024-09-20T18:55:29.766 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0889999999999996e-06 2024-09-20T18:55:29.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.766 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.766 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:55:29.766 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271080+0000", 2024-09-20T18:55:29.766 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5129999999999998e-06 2024-09-20T18:55:29.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.766 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.767 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:55:29.767 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271147+0000", 2024-09-20T18:55:29.767 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7470000000000003e-05 2024-09-20T18:55:29.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.767 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.767 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:55:29.767 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271165+0000", 2024-09-20T18:55:29.767 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8292e-05 2024-09-20T18:55:29.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.767 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.767 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:55:29.767 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441764+0000", 2024-09-20T18:55:29.767 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.170598789 2024-09-20T18:55:29.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.767 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.768 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:55:29.768 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441799+0000", 2024-09-20T18:55:29.768 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4997999999999999e-05 2024-09-20T18:55:29.768 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:29.768 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:55:29.768 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:29.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.768 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.768 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:25338 7.9 e93/83 ondisk, result = 0)", 2024-09-20T18:55:29.768 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.271688+0000", 2024-09-20T18:55:29.768 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 262.41914843799998, 2024-09-20T18:55:29.768 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.170058869, 2024-09-20T18:55:29.768 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:55:29.768 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:55:29.769 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:55:29.769 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:55:29.769 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.769 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:55:29.769 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271688+0000", 2024-09-20T18:55:29.769 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:29.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.769 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.769 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:55:29.769 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271688+0000", 2024-09-20T18:55:29.769 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:29.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.769 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.770 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:55:29.770 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271688+0000", 2024-09-20T18:55:29.770 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5600000000000005e-07 2024-09-20T18:55:29.770 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.770 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.770 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:55:29.770 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271692+0000", 2024-09-20T18:55:29.770 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.844e-06 2024-09-20T18:55:29.770 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.770 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.770 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:55:29.770 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271700+0000", 2024-09-20T18:55:29.770 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2619999999999998e-06 2024-09-20T18:55:29.770 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.771 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:55:29.771 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271706+0000", 2024-09-20T18:55:29.771 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8019999999999997e-06 2024-09-20T18:55:29.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.771 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:55:29.771 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271746+0000", 2024-09-20T18:55:29.771 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0163999999999997e-05 2024-09-20T18:55:29.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.771 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:55:29.771 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271752+0000", 2024-09-20T18:55:29.771 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.896e-06 2024-09-20T18:55:29.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.772 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.772 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:55:29.772 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441746+0000", 2024-09-20T18:55:29.772 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.16999414500000001 2024-09-20T18:55:29.772 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:29.772 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:55:29.772 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:29.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.772 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.772 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:25340 7.6 e93/83 ondisk, result = 0)", 2024-09-20T18:55:29.772 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.272049+0000", 2024-09-20T18:55:29.772 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 262.41878706300002, 2024-09-20T18:55:29.772 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.16970538299999999, 2024-09-20T18:55:29.772 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:55:29.773 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:55:29.773 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:55:29.773 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:55:29.773 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.773 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:55:29.773 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.272049+0000", 2024-09-20T18:55:29.773 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:29.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.773 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.773 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:55:29.773 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.272049+0000", 2024-09-20T18:55:29.773 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:29.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.773 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.774 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:55:29.774 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.272049+0000", 2024-09-20T18:55:29.774 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7999999999999996e-07 2024-09-20T18:55:29.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.774 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:55:29.774 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.272053+0000", 2024-09-20T18:55:29.774 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4970000000000001e-06 2024-09-20T18:55:29.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.774 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:55:29.774 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441627+0000", 2024-09-20T18:55:29.774 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.169574321 2024-09-20T18:55:29.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.775 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:55:29.775 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441637+0000", 2024-09-20T18:55:29.775 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9750000000000002e-06 2024-09-20T18:55:29.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.775 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:55:29.775 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441738+0000", 2024-09-20T18:55:29.775 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000100342 2024-09-20T18:55:29.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.775 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:55:29.775 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441743+0000", 2024-09-20T18:55:29.775 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4280000000000004e-06 2024-09-20T18:55:29.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.776 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.776 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:55:29.776 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441754+0000", 2024-09-20T18:55:29.776 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.134e-05 2024-09-20T18:55:29.776 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:29.776 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:55:29.776 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:29.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.776 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.776 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:25345 7.4 e93/83 ondisk, result = 0)", 2024-09-20T18:55:29.776 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.272463+0000", 2024-09-20T18:55:29.776 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 262.41837259300002, 2024-09-20T18:55:29.776 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.169185207, 2024-09-20T18:55:29.776 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:55:29.777 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:55:29.777 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:55:29.777 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:55:29.777 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.777 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:55:29.777 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.272463+0000", 2024-09-20T18:55:29.777 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:29.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.777 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.777 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:55:29.777 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.272463+0000", 2024-09-20T18:55:29.777 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:29.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.777 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.777 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:55:29.778 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.272464+0000", 2024-09-20T18:55:29.778 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:29.778 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:29.779 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9999999999999997e-07 2024-09-20T18:55:29.779 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.779 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.779 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:55:29.779 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.272468+0000", 2024-09-20T18:55:29.779 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9779999999999999e-06 2024-09-20T18:55:29.779 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.779 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.779 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:55:29.779 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441420+0000", 2024-09-20T18:55:29.780 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.16895175900000001 2024-09-20T18:55:29.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.780 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.780 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:55:29.780 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441438+0000", 2024-09-20T18:55:29.780 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8244999999999999e-05 2024-09-20T18:55:29.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.780 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.780 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:55:29.780 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441615+0000", 2024-09-20T18:55:29.780 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017724900000000001 2024-09-20T18:55:29.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.780 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.781 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:55:29.781 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441623+0000", 2024-09-20T18:55:29.781 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.6399999999999997e-06 2024-09-20T18:55:29.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.781 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:55:29.781 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.441649+0000", 2024-09-20T18:55:29.781 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5636e-05 2024-09-20T18:55:29.781 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:29.781 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:55:29.781 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:29.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.782 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:26465 7.6 7:63edb752:::benchmark_data_smithi083_70643_object26464:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:55:29.782 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.995659+0000", 2024-09-20T18:55:29.782 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 261.69517709299998, 2024-09-20T18:55:29.782 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.159355475, 2024-09-20T18:55:29.782 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:55:29.782 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:55:29.782 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:29.782 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:55:29.782 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:55:29.782 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:29.782 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 26465 2024-09-20T18:55:29.782 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.782 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:55:29.782 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.783 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:55:29.783 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.995659+0000", 2024-09-20T18:55:29.783 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:29.783 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.783 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.783 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:55:29.783 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.995659+0000", 2024-09-20T18:55:29.783 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:29.783 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.783 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.783 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:55:29.783 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.995660+0000", 2024-09-20T18:55:29.783 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2410000000000001e-06 2024-09-20T18:55:29.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.784 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.784 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:55:29.784 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.995684+0000", 2024-09-20T18:55:29.784 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3417999999999998e-05 2024-09-20T18:55:29.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.784 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.784 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:55:29.784 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.995693+0000", 2024-09-20T18:55:29.784 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8519999999999999e-06 2024-09-20T18:55:29.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.784 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.784 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:55:29.784 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.995700+0000", 2024-09-20T18:55:29.785 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.849e-06 2024-09-20T18:55:29.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.785 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:55:29.785 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.996430+0000", 2024-09-20T18:55:29.785 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00072999800000000004 2024-09-20T18:55:29.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.785 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:55:29.785 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.996548+0000", 2024-09-20T18:55:29.785 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000117537 2024-09-20T18:55:29.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.786 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:55:29.786 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.996682+0000", 2024-09-20T18:55:29.786 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000134609 2024-09-20T18:55:29.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.786 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.786 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:55:29.786 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.998822+0000", 2024-09-20T18:55:29.786 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0021392630000000002 2024-09-20T18:55:29.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.786 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.786 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:29.786 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.003162+0000", 2024-09-20T18:55:29.786 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0043400319999999997 2024-09-20T18:55:29.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.787 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.787 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:29.787 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.154547+0000", 2024-09-20T18:55:29.787 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.151385617 2024-09-20T18:55:29.787 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.787 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.787 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:55:29.787 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.154601+0000", 2024-09-20T18:55:29.787 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4146e-05 2024-09-20T18:55:29.787 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.787 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.787 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:55:29.787 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.155014+0000", 2024-09-20T18:55:29.788 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000412913 2024-09-20T18:55:29.788 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:29.788 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:55:29.788 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:29.788 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.788 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.788 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:26466 7.6 7:641ccaab:::benchmark_data_smithi083_70643_object26465:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:55:29.788 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.995720+0000", 2024-09-20T18:55:29.788 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 261.69511584499998, 2024-09-20T18:55:29.788 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.15942645999999999, 2024-09-20T18:55:29.788 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:55:29.788 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:55:29.788 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:29.788 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:55:29.788 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:55:29.789 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:29.789 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 26466 2024-09-20T18:55:29.789 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.789 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:55:29.789 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.789 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:55:29.789 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.995720+0000", 2024-09-20T18:55:29.789 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:29.789 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.789 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.789 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:55:29.789 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.995720+0000", 2024-09-20T18:55:29.789 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:29.789 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.790 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.790 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:55:29.790 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.995721+0000", 2024-09-20T18:55:29.790 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5199999999999997e-07 2024-09-20T18:55:29.790 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.790 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.790 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:55:29.790 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.995773+0000", 2024-09-20T18:55:29.790 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2679999999999997e-05 2024-09-20T18:55:29.790 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.790 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.790 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:55:29.790 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.995780+0000", 2024-09-20T18:55:29.790 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6880000000000002e-06 2024-09-20T18:55:29.791 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.791 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.791 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:55:29.791 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.995785+0000", 2024-09-20T18:55:29.791 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3129999999999998e-06 2024-09-20T18:55:29.791 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.791 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.791 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:55:29.791 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.997812+0000", 2024-09-20T18:55:29.791 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0020264710000000002 2024-09-20T18:55:29.791 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.791 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.791 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:55:29.791 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.997923+0000", 2024-09-20T18:55:29.792 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000110788 2024-09-20T18:55:29.792 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.792 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.792 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:55:29.792 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.998039+0000", 2024-09-20T18:55:29.792 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115961 2024-09-20T18:55:29.792 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.792 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.792 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:55:29.792 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.999810+0000", 2024-09-20T18:55:29.792 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001771051 2024-09-20T18:55:29.792 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.792 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.792 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:29.793 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.003194+0000", 2024-09-20T18:55:29.793 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003383952 2024-09-20T18:55:29.793 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.793 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.793 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:29.793 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.155059+0000", 2024-09-20T18:55:29.793 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.151865219 2024-09-20T18:55:29.793 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.793 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.793 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:55:29.793 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.155110+0000", 2024-09-20T18:55:29.793 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1004999999999997e-05 2024-09-20T18:55:29.793 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.793 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.794 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:55:29.794 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.155147+0000", 2024-09-20T18:55:29.794 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6779999999999997e-05 2024-09-20T18:55:29.794 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:29.794 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:55:29.794 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:29.794 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.794 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.794 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:26467 7.6 7:6fa2bfec:::benchmark_data_smithi083_70643_object26466:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:55:29.794 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.995794+0000", 2024-09-20T18:55:29.794 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 261.69504243199998, 2024-09-20T18:55:29.794 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.15946484, 2024-09-20T18:55:29.794 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:55:29.795 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:55:29.795 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:29.795 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:55:29.795 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:55:29.795 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:29.795 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 26467 2024-09-20T18:55:29.795 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.795 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:55:29.795 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.795 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:55:29.795 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.995794+0000", 2024-09-20T18:55:29.795 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:29.795 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.795 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.795 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:55:29.796 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.995794+0000", 2024-09-20T18:55:29.796 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:29.796 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.796 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.796 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:55:29.796 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.995794+0000", 2024-09-20T18:55:29.796 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4000000000000002e-07 2024-09-20T18:55:29.796 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.796 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.796 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:55:29.796 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.995813+0000", 2024-09-20T18:55:29.796 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9134999999999999e-05 2024-09-20T18:55:29.796 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.796 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.796 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:55:29.797 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.995819+0000", 2024-09-20T18:55:29.797 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.524e-06 2024-09-20T18:55:29.797 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.797 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.797 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:55:29.797 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.995823+0000", 2024-09-20T18:55:29.797 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6789999999999996e-06 2024-09-20T18:55:29.797 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.797 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.797 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:55:29.797 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.998856+0000", 2024-09-20T18:55:29.797 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0030330299999999999 2024-09-20T18:55:29.797 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.797 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.798 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:55:29.798 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.998973+0000", 2024-09-20T18:55:29.798 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000116214 2024-09-20T18:55:29.798 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.798 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.798 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:55:29.798 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.999103+0000", 2024-09-20T18:55:29.798 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013050200000000001 2024-09-20T18:55:29.798 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.798 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.798 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:55:29.798 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.000382+0000", 2024-09-20T18:55:29.798 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001278532 2024-09-20T18:55:29.798 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.799 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.799 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:29.799 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.003226+0000", 2024-09-20T18:55:29.799 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0028442160000000001 2024-09-20T18:55:29.799 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.799 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.799 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:29.799 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.155183+0000", 2024-09-20T18:55:29.799 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.15195683700000001 2024-09-20T18:55:29.799 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.799 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.799 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:55:29.799 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.155225+0000", 2024-09-20T18:55:29.799 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2528999999999999e-05 2024-09-20T18:55:29.800 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.800 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.800 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:55:29.800 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.155258+0000", 2024-09-20T18:55:29.800 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3201999999999998e-05 2024-09-20T18:55:29.800 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:29.800 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:55:29.800 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:29.800 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.800 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.800 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:26474 7.f 7:f7988c88:::benchmark_data_smithi083_70643_object26473:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:55:29.800 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.998994+0000", 2024-09-20T18:55:29.800 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 261.69184164199999, 2024-09-20T18:55:29.800 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.157028792, 2024-09-20T18:55:29.800 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:55:29.801 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:55:29.801 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:29.801 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:55:29.801 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:55:29.801 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:29.801 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 26474 2024-09-20T18:55:29.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.801 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:55:29.801 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.801 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:55:29.801 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.998994+0000", 2024-09-20T18:55:29.801 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:29.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.801 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.802 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:55:29.802 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.998994+0000", 2024-09-20T18:55:29.802 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:29.802 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.802 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.802 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:55:29.802 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.998995+0000", 2024-09-20T18:55:29.802 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0380000000000001e-06 2024-09-20T18:55:29.802 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.802 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.802 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:55:29.802 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.999012+0000", 2024-09-20T18:55:29.802 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6949e-05 2024-09-20T18:55:29.802 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.803 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.803 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:55:29.803 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.999021+0000", 2024-09-20T18:55:29.803 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7239999999999998e-06 2024-09-20T18:55:29.803 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.803 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.803 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:55:29.803 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.999026+0000", 2024-09-20T18:55:29.803 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7539999999999997e-06 2024-09-20T18:55:29.803 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.803 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.803 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:55:29.803 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.999072+0000", 2024-09-20T18:55:29.803 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5920000000000001e-05 2024-09-20T18:55:29.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.804 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.804 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:55:29.804 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.999192+0000", 2024-09-20T18:55:29.804 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000119943 2024-09-20T18:55:29.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.804 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.804 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:55:29.804 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.999269+0000", 2024-09-20T18:55:29.804 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.6956e-05 2024-09-20T18:55:29.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.804 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.804 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:55:29.804 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.000374+0000", 2024-09-20T18:55:29.805 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0011057829999999999 2024-09-20T18:55:29.805 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.805 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.805 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:29.805 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.001757+0000", 2024-09-20T18:55:29.805 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0013829529999999999 2024-09-20T18:55:29.805 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.805 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.805 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:29.805 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.155951+0000", 2024-09-20T18:55:29.805 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.15419313300000001 2024-09-20T18:55:29.805 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.805 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.805 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:55:29.805 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.155990+0000", 2024-09-20T18:55:29.806 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9714e-05 2024-09-20T18:55:29.806 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.806 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.806 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:55:29.806 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.156023+0000", 2024-09-20T18:55:29.806 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2925000000000001e-05 2024-09-20T18:55:29.806 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:29.806 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:55:29.806 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:29.806 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.806 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.806 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:26476 7.6 7:696c1e7a:::benchmark_data_smithi083_70643_object26475:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:55:29.806 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:08.002180+0000", 2024-09-20T18:55:29.807 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 261.68865634999997, 2024-09-20T18:55:29.807 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.15334672699999999, 2024-09-20T18:55:29.807 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:55:29.807 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:55:29.807 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:29.807 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:55:29.807 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:55:29.807 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:29.807 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 26476 2024-09-20T18:55:29.807 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.807 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:55:29.807 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.807 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:55:29.807 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.002180+0000", 2024-09-20T18:55:29.808 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:29.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.808 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.808 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:55:29.808 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.002180+0000", 2024-09-20T18:55:29.808 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:29.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.808 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.808 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:55:29.808 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.002181+0000", 2024-09-20T18:55:29.808 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.173e-06 2024-09-20T18:55:29.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.808 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.808 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:55:29.808 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.002198+0000", 2024-09-20T18:55:29.809 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6895999999999999e-05 2024-09-20T18:55:29.809 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.809 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.809 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:55:29.809 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.002205+0000", 2024-09-20T18:55:29.809 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.6590000000000007e-06 2024-09-20T18:55:29.809 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.809 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.809 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:55:29.809 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.002210+0000", 2024-09-20T18:55:29.809 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6099999999999999e-06 2024-09-20T18:55:29.809 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.809 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.809 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:55:29.809 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.002251+0000", 2024-09-20T18:55:29.810 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0828000000000003e-05 2024-09-20T18:55:29.810 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.810 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.810 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:55:29.810 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.002333+0000", 2024-09-20T18:55:29.810 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2410000000000005e-05 2024-09-20T18:55:29.810 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.810 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.810 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:55:29.810 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.002416+0000", 2024-09-20T18:55:29.810 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2363999999999999e-05 2024-09-20T18:55:29.810 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.810 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.810 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:55:29.811 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.003273+0000", 2024-09-20T18:55:29.811 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00085788100000000001 2024-09-20T18:55:29.811 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.811 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.811 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:29.811 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.004070+0000", 2024-09-20T18:55:29.811 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000796644 2024-09-20T18:55:29.811 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.811 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.811 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:29.811 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.155292+0000", 2024-09-20T18:55:29.811 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.15122216099999999 2024-09-20T18:55:29.811 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.811 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.811 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:55:29.812 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.155494+0000", 2024-09-20T18:55:29.812 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020213300000000001 2024-09-20T18:55:29.812 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.812 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.812 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:55:29.812 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.155526+0000", 2024-09-20T18:55:29.812 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1968000000000003e-05 2024-09-20T18:55:29.812 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:29.812 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:55:29.812 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:29.812 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.812 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.812 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:26479 7.d e93/83 7:bdad06c1:::benchmark_data_smithi083_70643_object26478:head v 93'1700, mlcod=93'1697)", 2024-09-20T18:55:29.812 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:08.005291+0000", 2024-09-20T18:55:29.812 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 261.685545359, 2024-09-20T18:55:29.813 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.14962600100000001, 2024-09-20T18:55:29.813 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:55:29.813 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:55:29.813 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:29.813 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:55:29.813 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.813 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:55:29.813 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.005291+0000", 2024-09-20T18:55:29.813 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:29.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.813 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.813 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:55:29.813 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.005291+0000", 2024-09-20T18:55:29.813 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:29.814 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.814 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.814 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:55:29.814 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.005291+0000", 2024-09-20T18:55:29.814 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1999999999999999e-07 2024-09-20T18:55:29.814 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.814 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.814 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:55:29.814 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.005402+0000", 2024-09-20T18:55:29.814 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000110463 2024-09-20T18:55:29.814 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.814 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.814 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:55:29.814 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.153269+0000", 2024-09-20T18:55:29.815 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.14786682100000001 2024-09-20T18:55:29.815 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.815 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.815 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:55:29.815 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.153279+0000", 2024-09-20T18:55:29.815 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0431999999999999e-05 2024-09-20T18:55:29.815 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.815 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.815 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:55:29.815 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.153448+0000", 2024-09-20T18:55:29.815 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016901899999999999 2024-09-20T18:55:29.815 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.815 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.815 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:55:29.815 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.153470+0000", 2024-09-20T18:55:29.816 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1747999999999999e-05 2024-09-20T18:55:29.816 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.816 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.816 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:55:29.816 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.154874+0000", 2024-09-20T18:55:29.816 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001403767 2024-09-20T18:55:29.816 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:29.816 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:29.816 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:55:29.816 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.154917+0000", 2024-09-20T18:55:29.816 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3130999999999998e-05 2024-09-20T18:55:29.816 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:29.816 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:55:29.816 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:29.816 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:29.817 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:55:29.817 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T18:55:29.879 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:55:29.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:29.915 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:29.915 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:55:30.016 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:30.047 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:30.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:30.048 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:30.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:29 smithi083 ceph-mon[36260]: 5.15 scrub starts 2024-09-20T18:55:30.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:29 smithi083 ceph-mon[36260]: 5.15 scrub ok 2024-09-20T18:55:30.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:29 smithi083 ceph-mon[36260]: osdmap e182: 8 total, 8 up, 7 in 2024-09-20T18:55:30.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:29 smithi083 ceph-mon[36260]: pgmap v268: 148 pgs: 148 active+clean; 5.0 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1.2 KiB/s rd, 94 MiB/s wr, 1.50k op/s 2024-09-20T18:55:30.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:29 smithi083 ceph-mon[36260]: 5.3 scrub starts 2024-09-20T18:55:30.054 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:29 smithi083 ceph-mon[36260]: 5.3 scrub ok 2024-09-20T18:55:30.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:29 smithi083 ceph-mon[29421]: 5.15 scrub starts 2024-09-20T18:55:30.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:29 smithi083 ceph-mon[29421]: 5.15 scrub ok 2024-09-20T18:55:30.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:29 smithi083 ceph-mon[29421]: osdmap e182: 8 total, 8 up, 7 in 2024-09-20T18:55:30.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:29 smithi083 ceph-mon[29421]: pgmap v268: 148 pgs: 148 active+clean; 5.0 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1.2 KiB/s rd, 94 MiB/s wr, 1.50k op/s 2024-09-20T18:55:30.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:29 smithi083 ceph-mon[29421]: 5.3 scrub starts 2024-09-20T18:55:30.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:29 smithi083 ceph-mon[29421]: 5.3 scrub ok 2024-09-20T18:55:30.148 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:30.177 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 50 16 83455 83439 104.248 3.59375 0.0188896 0.0095861 2024-09-20T18:55:30.180 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:30.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:30.181 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:30.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:29 smithi142 ceph-mon[32508]: 5.15 scrub starts 2024-09-20T18:55:30.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:29 smithi142 ceph-mon[32508]: 5.15 scrub ok 2024-09-20T18:55:30.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:29 smithi142 ceph-mon[32508]: osdmap e182: 8 total, 8 up, 7 in 2024-09-20T18:55:30.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:29 smithi142 ceph-mon[32508]: pgmap v268: 148 pgs: 148 active+clean; 5.0 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1.2 KiB/s rd, 94 MiB/s wr, 1.50k op/s 2024-09-20T18:55:30.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:29 smithi142 ceph-mon[32508]: 5.3 scrub starts 2024-09-20T18:55:30.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:29 smithi142 ceph-mon[32508]: 5.3 scrub ok 2024-09-20T18:55:30.283 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:30.305 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:30.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:30.306 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:30.406 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:30.438 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:30.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:30.439 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:30.540 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:30.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:30.569 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:30.569 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:30.670 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:30.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:30.699 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:30.699 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:30.726 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T18:55:30.764 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-20T18:55:30.767 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:55:30.801 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:30.825 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:30.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:30.826 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:30.927 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:30.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:30.985 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:30.986 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:31.052 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:55:30 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:55:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:55:31.052 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:30 smithi083 ceph-mon[36260]: 3.1d scrub starts 2024-09-20T18:55:31.052 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:30 smithi083 ceph-mon[36260]: 3.1d scrub ok 2024-09-20T18:55:31.052 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:30 smithi083 ceph-mon[36260]: 3.11 scrub starts 2024-09-20T18:55:31.052 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:30 smithi083 ceph-mon[36260]: 3.11 scrub ok 2024-09-20T18:55:31.052 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:30 smithi083 ceph-mon[36260]: 8.3 starting backfill to osd.2 from (0'0,0'0] MAX to 180'5097 2024-09-20T18:55:31.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:30 smithi083 ceph-mon[36260]: 8.7 starting backfill to osd.2 from (0'0,0'0] MAX to 180'5286 2024-09-20T18:55:31.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:30 smithi083 ceph-mon[36260]: osdmap e183: 8 total, 8 up, 7 in 2024-09-20T18:55:31.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:30 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:55:31.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:30 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:55:31.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:30 smithi083 ceph-mon[36260]: osdmap e184: 8 total, 8 up, 7 in 2024-09-20T18:55:31.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:30 smithi083 ceph-mon[29421]: 3.1d scrub starts 2024-09-20T18:55:31.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:30 smithi083 ceph-mon[29421]: 3.1d scrub ok 2024-09-20T18:55:31.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:30 smithi083 ceph-mon[29421]: 3.11 scrub starts 2024-09-20T18:55:31.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:30 smithi083 ceph-mon[29421]: 3.11 scrub ok 2024-09-20T18:55:31.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:30 smithi083 ceph-mon[29421]: 8.3 starting backfill to osd.2 from (0'0,0'0] MAX to 180'5097 2024-09-20T18:55:31.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:30 smithi083 ceph-mon[29421]: 8.7 starting backfill to osd.2 from (0'0,0'0] MAX to 180'5286 2024-09-20T18:55:31.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:30 smithi083 ceph-mon[29421]: osdmap e183: 8 total, 8 up, 7 in 2024-09-20T18:55:31.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:30 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:55:31.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:30 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:55:31.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:30 smithi083 ceph-mon[29421]: osdmap e184: 8 total, 8 up, 7 in 2024-09-20T18:55:31.087 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:31.113 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:31.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:31.114 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:31.178 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 51 16 84259 84243 103.188 50.25 0.00745392 0.00968235 2024-09-20T18:55:31.215 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:31.243 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:31.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:31.245 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:31.271 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:30 smithi142 ceph-mon[32508]: 3.1d scrub starts 2024-09-20T18:55:31.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:30 smithi142 ceph-mon[32508]: 3.1d scrub ok 2024-09-20T18:55:31.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:30 smithi142 ceph-mon[32508]: 3.11 scrub starts 2024-09-20T18:55:31.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:30 smithi142 ceph-mon[32508]: 3.11 scrub ok 2024-09-20T18:55:31.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:30 smithi142 ceph-mon[32508]: 8.3 starting backfill to osd.2 from (0'0,0'0] MAX to 180'5097 2024-09-20T18:55:31.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:30 smithi142 ceph-mon[32508]: 8.7 starting backfill to osd.2 from (0'0,0'0] MAX to 180'5286 2024-09-20T18:55:31.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:30 smithi142 ceph-mon[32508]: osdmap e183: 8 total, 8 up, 7 in 2024-09-20T18:55:31.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:30 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:55:31.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:30 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:55:31.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:30 smithi142 ceph-mon[32508]: osdmap e184: 8 total, 8 up, 7 in 2024-09-20T18:55:31.346 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:31.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:31.397 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:31.398 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:31.469 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:55:31.499 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:31.514 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:55:31.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:31.564 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:31.564 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:31.665 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:31.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:31.716 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:31.716 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:31.817 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:31.845 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-20T18:55:31.847 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:31.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:31.848 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:31.910 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:31 smithi083 ceph-mon[29421]: pgmap v271: 148 pgs: 1 active+recovering+undersized+remapped, 147 active+clean; 5.0 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1023 B/s rd, 5.9 MiB/s wr, 95 op/s; 5362/245475 objects misplaced (2.184%); 608 KiB/s, 16 objects/s recovering 2024-09-20T18:55:31.910 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:31 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1690933094' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:55:31.910 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:31 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:55:31.910 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:31 smithi083 ceph-mon[36260]: pgmap v271: 148 pgs: 1 active+recovering+undersized+remapped, 147 active+clean; 5.0 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1023 B/s rd, 5.9 MiB/s wr, 95 op/s; 5362/245475 objects misplaced (2.184%); 608 KiB/s, 16 objects/s recovering 2024-09-20T18:55:31.910 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:31 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1690933094' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:55:31.911 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:31 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:55:31.948 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:31.986 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:31.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:31.987 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:32.088 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:32.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:32.123 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:32.123 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:32.179 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 52 16 84989 84973 102.081 45.625 0.037121 0.00978988 2024-09-20T18:55:32.224 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:32.250 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:32.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:32.251 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:32.271 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:31 smithi142 ceph-mon[32508]: pgmap v271: 148 pgs: 1 active+recovering+undersized+remapped, 147 active+clean; 5.0 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1023 B/s rd, 5.9 MiB/s wr, 95 op/s; 5362/245475 objects misplaced (2.184%); 608 KiB/s, 16 objects/s recovering 2024-09-20T18:55:32.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:31 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1690933094' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:55:32.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:31 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:55:32.351 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:32.380 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:32.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:32.381 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:32.482 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:32.510 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:32.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:32.511 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:32.612 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:32.638 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:32.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:32.639 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:32.740 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:32.765 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:32.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:32.766 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:32.866 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:32.894 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:32.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:32.895 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:32.996 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:33.018 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:33.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:33.019 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:33.051 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:32 smithi083 ceph-mon[29421]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:55:33.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:32 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T18:55:33.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:32 smithi083 ceph-mon[29421]: osdmap e185: 8 total, 8 up, 7 in 2024-09-20T18:55:33.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:32 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:55:33.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:32 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:55:33.052 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:32 smithi083 ceph-mon[36260]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:55:33.052 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:32 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T18:55:33.052 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:32 smithi083 ceph-mon[36260]: osdmap e185: 8 total, 8 up, 7 in 2024-09-20T18:55:33.052 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:32 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:55:33.052 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:32 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:55:33.120 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:33.157 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:33.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:33.158 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:33.178 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 53 16 85828 85812 101.145 52.4375 0.00946168 0.00988229 2024-09-20T18:55:33.259 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:55:33.271 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:32 smithi142 ceph-mon[32508]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:55:33.271 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:32 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T18:55:33.271 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:32 smithi142 ceph-mon[32508]: osdmap e185: 8 total, 8 up, 7 in 2024-09-20T18:55:33.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:32 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:55:33.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:32 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:55:33.294 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:33.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:33.295 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:55:33.396 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:55:33.435 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:33.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:33.436 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:55:33.537 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:33.578 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:33.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:33.579 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:33.652 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:55:33.652 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:55:33.681 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:55:33.688 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:55:33.688 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:55:33.726 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:33.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:33.727 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:55:33.728 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:55:33.728 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:55:33.765 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:55:33.765 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:55:33.804 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:55:33.804 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:55:33.828 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:33.849 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:55:33.849 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:55:33.856 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:33.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:33.857 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:33.861 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T18:55:33.861 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:55:33.883 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:55:33.883 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:55:33.926 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:55:33.926 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:55:33.957 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:33.984 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:33.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:33.985 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:34.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:33 smithi083 ceph-mon[29421]: osdmap e186: 8 total, 8 up, 7 in 2024-09-20T18:55:34.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:33 smithi083 ceph-mon[29421]: pgmap v274: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 143 active+clean; 5.1 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 511 B/s rd, 46 MiB/s wr, 744 op/s; 26055/253269 objects misplaced (10.287%); 3.5 MiB/s, 72 objects/s recovering 2024-09-20T18:55:34.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:33 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1562030287' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:55:34.052 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:33 smithi083 ceph-mon[36260]: osdmap e186: 8 total, 8 up, 7 in 2024-09-20T18:55:34.052 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:33 smithi083 ceph-mon[36260]: pgmap v274: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 143 active+clean; 5.1 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 511 B/s rd, 46 MiB/s wr, 744 op/s; 26055/253269 objects misplaced (10.287%); 3.5 MiB/s, 72 objects/s recovering 2024-09-20T18:55:34.052 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:33 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1562030287' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:55:34.086 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:34.122 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:34.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:34.123 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:34.178 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 54 16 86658 86642 100.233 51.875 0.0211681 0.00997183 2024-09-20T18:55:34.224 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:34.259 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:34.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:34.260 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:34.271 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:33 smithi142 ceph-mon[32508]: osdmap e186: 8 total, 8 up, 7 in 2024-09-20T18:55:34.271 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:33 smithi142 ceph-mon[32508]: pgmap v274: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 143 active+clean; 5.1 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 511 B/s rd, 46 MiB/s wr, 744 op/s; 26055/253269 objects misplaced (10.287%); 3.5 MiB/s, 72 objects/s recovering 2024-09-20T18:55:34.271 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:33 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1562030287' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:55:34.275 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 7] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-20T18:55:34.275 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd reweight 0 1 2024-09-20T18:55:34.361 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:34.389 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:34.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:34.390 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:34.490 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:34.514 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:34.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:34.516 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:34.616 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:34.690 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:34.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:34.691 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:34.792 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:34.822 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:34.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:34.823 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:34.924 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:34.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:34.951 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:34.952 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:35.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:34 smithi083 ceph-mon[29421]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T18:55:35.006 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:34 smithi083 ceph-mon[36260]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T18:55:35.052 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:35.078 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:35.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:35.079 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:35.179 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 55 16 87553 87537 99.4278 55.9375 0.0128911 0.0100523 2024-09-20T18:55:35.180 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:35.219 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:35.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:35.220 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:35.271 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:34 smithi142 ceph-mon[32508]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T18:55:35.321 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:35.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:35.363 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:35.363 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:35.464 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:55:35.498 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:35.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:35.499 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:55:35.599 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:35.624 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:35.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:35.625 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:35.726 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:35.756 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:35.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:35.757 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:35.857 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:35.885 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:35.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:35.886 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:35.917 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_blocked_ops 2024-09-20T18:55:35.987 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:55:36.039 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:55:36.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:36.050 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:36.050 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:55:36.151 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:36.179 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:36.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:36.180 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:36.180 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 56 16 88398 88382 98.5938 52.8125 0.00449452 0.0101347 2024-09-20T18:55:36.281 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:36.306 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:36.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:36.307 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:36.408 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:36.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:36.436 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:36.436 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:36.537 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:36.562 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:36.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:36.564 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:36.664 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:36.688 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:36.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:36.689 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:36.693 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T18:55:36.719 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:55:36.768 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:36 smithi083 ceph-mon[29421]: pgmap v275: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 143 active+clean; 5.1 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 404 B/s rd, 37 MiB/s wr, 587 op/s; 26055/253269 objects misplaced (10.287%); 2.8 MiB/s, 57 objects/s recovering 2024-09-20T18:55:36.768 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:36 smithi083 ceph-mon[36260]: pgmap v275: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 143 active+clean; 5.1 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 404 B/s rd, 37 MiB/s wr, 587 op/s; 26055/253269 objects misplaced (10.287%); 2.8 MiB/s, 57 objects/s recovering 2024-09-20T18:55:36.790 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:36.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:36.854 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:36.854 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:36.955 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:36.980 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:36.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:36.981 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:37.021 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:36 smithi142 ceph-mon[32508]: pgmap v275: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 143 active+clean; 5.1 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 404 B/s rd, 37 MiB/s wr, 587 op/s; 26055/253269 objects misplaced (10.287%); 2.8 MiB/s, 57 objects/s recovering 2024-09-20T18:55:37.021 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:55:36 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:55:37.082 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:37.135 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:37.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:37.136 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:37.180 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 57 16 89151 89135 97.6889 47.0625 0.0147207 0.0102307 2024-09-20T18:55:37.237 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:37.264 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:37.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:37.265 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:37.363 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T18:55:37.363 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:55:37.365 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:37.383 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:55:37.406 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:37.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:37.407 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:37.468 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T18:55:37.508 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:37.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:37.538 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:37.538 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:37.639 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:37.694 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.0 to 1 (10000) 2024-09-20T18:55:37.697 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:37.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:37.698 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:37.798 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:37.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:37.842 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:37.842 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:37.943 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:37.967 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:37.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:37.968 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:38.021 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:37 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:55:38.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:37 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1209667375' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-20T18:55:38.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:37 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-20T18:55:38.051 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:37 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:55:38.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:37 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1209667375' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-20T18:55:38.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:37 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-20T18:55:38.052 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:37 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:55:38.052 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:37 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1209667375' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-20T18:55:38.052 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:37 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-20T18:55:38.069 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:38.095 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:38.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:38.096 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:38.180 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 58 15 89990 89975 96.91 52.5 0.00883566 0.0103135 2024-09-20T18:55:38.197 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:55:38.236 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:38.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:38.237 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:55:38.338 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:55:38.372 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:38.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:38.373 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:55:38.437 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_historic_ops 2024-09-20T18:55:38.473 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:55:38.529 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:38.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:38.530 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:55:38.630 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:55:38.677 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:38.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:38.679 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:55:38.779 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:38.816 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:38.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:38.817 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:38.918 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:38.958 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:38.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:38.959 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:39.021 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:38 smithi142 ceph-mon[32508]: pgmap v276: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 143 active+clean; 5.4 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1.1 KiB/s rd, 72 MiB/s wr, 1.15k op/s; 25481/265707 objects misplaced (9.590%); 13 MiB/s, 213 objects/s recovering 2024-09-20T18:55:39.021 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:38 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-20T18:55:39.021 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:38 smithi142 ceph-mon[32508]: osdmap e187: 8 total, 8 up, 7 in 2024-09-20T18:55:39.042 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:38 smithi083 ceph-mon[29421]: pgmap v276: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 143 active+clean; 5.4 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1.1 KiB/s rd, 72 MiB/s wr, 1.15k op/s; 25481/265707 objects misplaced (9.590%); 13 MiB/s, 213 objects/s recovering 2024-09-20T18:55:39.042 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:38 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-20T18:55:39.042 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:38 smithi083 ceph-mon[29421]: osdmap e187: 8 total, 8 up, 7 in 2024-09-20T18:55:39.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:38 smithi083 ceph-mon[36260]: pgmap v276: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 143 active+clean; 5.4 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1.1 KiB/s rd, 72 MiB/s wr, 1.15k op/s; 25481/265707 objects misplaced (9.590%); 13 MiB/s, 213 objects/s recovering 2024-09-20T18:55:39.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:38 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-20T18:55:39.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:38 smithi083 ceph-mon[36260]: osdmap e187: 8 total, 8 up, 7 in 2024-09-20T18:55:39.060 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:55:39.104 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:39.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:39.105 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:55:39.180 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 59 16 90802 90786 96.1268 50.6875 0.00423292 0.0103958 2024-09-20T18:55:39.206 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:39.229 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:39.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:39.230 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:39.331 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:39.364 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:39.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:39.365 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:39.466 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:55:39.507 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:39.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:39.508 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:55:39.609 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:39.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:39.639 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:39.639 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:39.739 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:39.772 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:39.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:39.773 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:39.869 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:39 smithi142 ceph-mon[32508]: pgmap v278: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 143 active+clean; 5.4 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 725 B/s rd, 37 MiB/s wr, 588 op/s; 25481/265707 objects misplaced (9.590%); 9.2 MiB/s, 146 objects/s recovering 2024-09-20T18:55:39.873 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:39.907 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:39.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:39.908 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:39.946 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd reweight 1 1 2024-09-20T18:55:39.995 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:39 smithi083 ceph-mon[29421]: pgmap v278: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 143 active+clean; 5.4 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 725 B/s rd, 37 MiB/s wr, 588 op/s; 25481/265707 objects misplaced (9.590%); 9.2 MiB/s, 146 objects/s recovering 2024-09-20T18:55:39.998 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:39 smithi083 ceph-mon[36260]: pgmap v278: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 143 active+clean; 5.4 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 725 B/s rd, 37 MiB/s wr, 588 op/s; 25481/265707 objects misplaced (9.590%); 9.2 MiB/s, 146 objects/s recovering 2024-09-20T18:55:40.009 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:40.036 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:55:40.038 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:40.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:40.039 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:40.055 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:55:40.140 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:40.164 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:40.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:40.165 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:40.181 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:2024-09-20T18:55:40.167462+0000 min lat: 0.00111821 max lat: 2.23065 avg lat: 0.0104752 2024-09-20T18:55:40.181 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-20T18:55:40.181 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 60 16 91615 91599 95.3713 50.8125 0.00764803 0.0104752 2024-09-20T18:55:40.266 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:40.317 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:40.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:40.318 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:40.419 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:40.449 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:40.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:40.450 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:40.551 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:40.587 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:40.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:40.588 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:40.677 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-20T18:55:40.689 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:55:40.762 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:40.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:40.763 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:55:40.864 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:40.865 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:40 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1415782025' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:55:40.865 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:40 smithi083 ceph-mon[29421]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:55:40.865 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:40 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1415782025' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T18:55:40.865 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:40 smithi083 ceph-mon[29421]: osdmap e188: 8 total, 8 up, 7 in 2024-09-20T18:55:40.865 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:55:40 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:55:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:55:40.865 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:40 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1415782025' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:55:40.866 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:40 smithi083 ceph-mon[36260]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:55:40.866 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:40 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1415782025' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T18:55:40.866 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:40 smithi083 ceph-mon[36260]: osdmap e188: 8 total, 8 up, 7 in 2024-09-20T18:55:40.904 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:40.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:40.905 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:41.005 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:41.021 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:40 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1415782025' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:55:41.021 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:40 smithi142 ceph-mon[32508]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:55:41.021 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:40 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1415782025' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T18:55:41.021 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:40 smithi142 ceph-mon[32508]: osdmap e188: 8 total, 8 up, 7 in 2024-09-20T18:55:41.033 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:41.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:41.033 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:41.135 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:41.167 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:55:41.168 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:55:41.179 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:41.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:41.180 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:41.181 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 61 16 92446 92430 94.6594 51.9375 0.0307515 0.010558 2024-09-20T18:55:41.201 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:55:41.201 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:55:41.239 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:55:41.239 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:55:41.275 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:55:41.275 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:55:41.281 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:55:41.304 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:55:41.304 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:55:41.316 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:41.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:41.318 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:55:41.338 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:55:41.338 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:55:41.369 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:55:41.369 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:55:41.405 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:55:41.406 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:55:41.419 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:41.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:41.476 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:41.476 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:41.577 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:41.613 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:41.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:41.614 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:41.715 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:41.754 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:41.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:41.755 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:41.855 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:41.883 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:41.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:41.884 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:41.985 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:42.020 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:42.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:42.021 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:42.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:41 smithi142 ceph-mon[32508]: pgmap v280: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 143 active+clean; 5.5 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1.2 KiB/s rd, 44 MiB/s wr, 708 op/s; 25125/270240 objects misplaced (9.297%); 11 MiB/s, 173 objects/s recovering 2024-09-20T18:55:42.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:41 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1450774726' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:55:42.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:41 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:55:42.051 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:41 smithi083 ceph-mon[29421]: pgmap v280: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 143 active+clean; 5.5 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1.2 KiB/s rd, 44 MiB/s wr, 708 op/s; 25125/270240 objects misplaced (9.297%); 11 MiB/s, 173 objects/s recovering 2024-09-20T18:55:42.051 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:41 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1450774726' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:55:42.051 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:41 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:55:42.052 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:41 smithi083 ceph-mon[36260]: pgmap v280: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 143 active+clean; 5.5 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1.2 KiB/s rd, 44 MiB/s wr, 708 op/s; 25125/270240 objects misplaced (9.297%); 11 MiB/s, 173 objects/s recovering 2024-09-20T18:55:42.052 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:41 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1450774726' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:55:42.052 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:41 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:55:42.122 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:55:42.163 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:42.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:42.164 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:55:42.181 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 62 16 93241 93225 93.9342 49.6875 0.0131443 0.0106403 2024-09-20T18:55:42.266 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:42.292 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:42.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:42.293 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:42.394 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:42.467 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:42.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:42.468 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:42.569 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:42.598 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:42.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:42.599 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:42.700 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:55:42.725 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T18:55:42.725 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-20T18:55:42.725 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-20T18:55:42.725 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-20T18:55:42.725 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.725 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14158.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-20T18:55:42.725 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:47:16.702121+0000", 2024-09-20T18:55:42.725 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 506.00403924, 2024-09-20T18:55:42.725 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.46027232899999998, 2024-09-20T18:55:42.725 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:55:42.725 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:55:42.726 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:42.726 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:55:42.726 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14158", 2024-09-20T18:55:42.726 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2161728215", 2024-09-20T18:55:42.726 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 1 2024-09-20T18:55:42.726 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.726 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:55:42.726 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.726 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:55:42.726 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:47:16.702121+0000", 2024-09-20T18:55:42.726 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:42.726 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.726 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.726 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:55:42.727 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:47:16.702121+0000", 2024-09-20T18:55:42.727 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:42.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.727 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.727 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:55:42.727 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:47:16.702122+0000", 2024-09-20T18:55:42.727 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9900000000000009e-07 2024-09-20T18:55:42.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.727 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.727 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:55:42.727 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:47:16.702135+0000", 2024-09-20T18:55:42.727 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2087e-05 2024-09-20T18:55:42.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.727 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.728 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:55:42.728 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:47:16.702143+0000", 2024-09-20T18:55:42.728 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5420000000000005e-06 2024-09-20T18:55:42.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.728 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.728 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:55:42.728 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:47:16.702150+0000", 2024-09-20T18:55:42.728 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1910000000000003e-06 2024-09-20T18:55:42.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.728 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.728 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:55:42.728 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:47:16.702193+0000", 2024-09-20T18:55:42.728 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2398999999999998e-05 2024-09-20T18:55:42.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.729 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.729 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:55:42.729 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:47:16.702198+0000", 2024-09-20T18:55:42.729 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4009999999999996e-06 2024-09-20T18:55:42.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.729 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.729 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:55:42.729 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:47:17.161275+0000", 2024-09-20T18:55:42.729 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.45907645200000002 2024-09-20T18:55:42.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.729 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.729 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:55:42.729 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:47:17.161417+0000", 2024-09-20T18:55:42.729 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014258900000000001 2024-09-20T18:55:42.730 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.730 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.730 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,2", 2024-09-20T18:55:42.730 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:47:17.161566+0000", 2024-09-20T18:55:42.730 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014886599999999999 2024-09-20T18:55:42.730 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.730 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.730 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:55:42.730 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:47:17.162060+0000", 2024-09-20T18:55:42.730 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00049417099999999995 2024-09-20T18:55:42.730 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.730 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.730 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:42.731 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:47:17.162256+0000", 2024-09-20T18:55:42.731 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019619200000000001 2024-09-20T18:55:42.731 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.731 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.731 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:42.731 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:47:17.162303+0000", 2024-09-20T18:55:42.731 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6159999999999999e-05 2024-09-20T18:55:42.731 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.731 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.731 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:55:42.731 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:47:17.162357+0000", 2024-09-20T18:55:42.731 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4236e-05 2024-09-20T18:55:42.731 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.731 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.732 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:55:42.732 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:47:17.162394+0000", 2024-09-20T18:55:42.732 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7044e-05 2024-09-20T18:55:42.732 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:42.732 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:55:42.732 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:42.732 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.732 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.732 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:120191 7.b 7:d45eee6a:::benchmark_data_smithi083_70643_object120190:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T18:55:42.732 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:52:24.970674+0000", 2024-09-20T18:55:42.733 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 197.73548657800001, 2024-09-20T18:55:42.733 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.115151315, 2024-09-20T18:55:42.733 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:55:42.733 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:55:42.733 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:42.733 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:55:42.733 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:55:42.733 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:42.733 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 120191 2024-09-20T18:55:42.733 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.733 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:55:42.733 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.733 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:55:42.733 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.970674+0000", 2024-09-20T18:55:42.734 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:42.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.734 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.734 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:55:42.734 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.970674+0000", 2024-09-20T18:55:42.734 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:42.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.734 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.734 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:55:42.734 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.970676+0000", 2024-09-20T18:55:42.734 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.796e-06 2024-09-20T18:55:42.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.734 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.735 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:55:42.735 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.970706+0000", 2024-09-20T18:55:42.735 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9938e-05 2024-09-20T18:55:42.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.735 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.735 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:55:42.735 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.970721+0000", 2024-09-20T18:55:42.735 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4657999999999999e-05 2024-09-20T18:55:42.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.735 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.735 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:55:42.735 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.970729+0000", 2024-09-20T18:55:42.735 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.994e-06 2024-09-20T18:55:42.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.736 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.736 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:55:42.736 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.970786+0000", 2024-09-20T18:55:42.736 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7113e-05 2024-09-20T18:55:42.736 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.736 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.736 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:55:42.736 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.971178+0000", 2024-09-20T18:55:42.736 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00039247099999999998 2024-09-20T18:55:42.736 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.736 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.736 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,6", 2024-09-20T18:55:42.736 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.971619+0000", 2024-09-20T18:55:42.736 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000440956 2024-09-20T18:55:42.736 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.737 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.737 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:42.737 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.975170+0000", 2024-09-20T18:55:42.737 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0035509000000000001 2024-09-20T18:55:42.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.737 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.737 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:55:42.737 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.975661+0000", 2024-09-20T18:55:42.737 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00049051900000000002 2024-09-20T18:55:42.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.737 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.737 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:42.737 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.085675+0000", 2024-09-20T18:55:42.737 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.110014127 2024-09-20T18:55:42.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.738 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.738 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:55:42.738 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.085748+0000", 2024-09-20T18:55:42.738 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3512999999999999e-05 2024-09-20T18:55:42.738 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.738 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.738 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:55:42.738 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.085825+0000", 2024-09-20T18:55:42.738 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7330000000000007e-05 2024-09-20T18:55:42.738 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:42.738 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:55:42.738 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:42.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.739 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.739 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:120201 7.b 7:d1c07770:::benchmark_data_smithi083_70643_object120200:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T18:55:42.739 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:52:24.982656+0000", 2024-09-20T18:55:42.739 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 197.72350466399999, 2024-09-20T18:55:42.739 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.103339863, 2024-09-20T18:55:42.739 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:55:42.739 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:55:42.739 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:42.739 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:55:42.739 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:55:42.739 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:42.739 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 120201 2024-09-20T18:55:42.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.740 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:55:42.740 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.740 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:55:42.740 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.982656+0000", 2024-09-20T18:55:42.740 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:42.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.740 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.740 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:55:42.740 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.982656+0000", 2024-09-20T18:55:42.740 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:42.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.740 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.740 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:55:42.740 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.982658+0000", 2024-09-20T18:55:42.740 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.77e-06 2024-09-20T18:55:42.741 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.741 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.741 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:55:42.741 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.982682+0000", 2024-09-20T18:55:42.741 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4309999999999999e-05 2024-09-20T18:55:42.741 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.741 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.741 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:55:42.741 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.982698+0000", 2024-09-20T18:55:42.741 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5389000000000001e-05 2024-09-20T18:55:42.741 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.741 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.741 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:55:42.741 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.982705+0000", 2024-09-20T18:55:42.742 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9019999999999999e-06 2024-09-20T18:55:42.742 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.742 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.742 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:55:42.742 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.982858+0000", 2024-09-20T18:55:42.742 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015291400000000001 2024-09-20T18:55:42.742 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.742 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.742 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:55:42.742 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.983315+0000", 2024-09-20T18:55:42.742 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000456563 2024-09-20T18:55:42.742 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.742 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.742 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,6", 2024-09-20T18:55:42.743 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.983727+0000", 2024-09-20T18:55:42.743 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00041162000000000002 2024-09-20T18:55:42.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.743 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.743 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:42.743 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.993189+0000", 2024-09-20T18:55:42.743 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0094625750000000008 2024-09-20T18:55:42.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.743 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.743 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:55:42.743 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.993200+0000", 2024-09-20T18:55:42.743 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0866000000000001e-05 2024-09-20T18:55:42.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.743 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.744 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:42.744 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.085878+0000", 2024-09-20T18:55:42.744 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.092678153999999999 2024-09-20T18:55:42.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.744 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.744 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:55:42.744 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.085938+0000", 2024-09-20T18:55:42.744 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9991999999999998e-05 2024-09-20T18:55:42.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.744 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.744 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:55:42.744 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.085996+0000", 2024-09-20T18:55:42.744 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7807999999999999e-05 2024-09-20T18:55:42.744 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:42.745 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:55:42.745 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:42.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.745 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.745 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:120205 7.b 7:d694ab9d:::benchmark_data_smithi083_70643_object120204:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T18:55:42.745 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:52:24.983125+0000", 2024-09-20T18:55:42.745 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 197.72303609400001, 2024-09-20T18:55:42.745 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.10302963699999999, 2024-09-20T18:55:42.745 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:55:42.745 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:55:42.745 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:42.745 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:55:42.745 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:55:42.745 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:55:42.746 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 120205 2024-09-20T18:55:42.746 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.746 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:55:42.746 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.746 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:55:42.746 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.983125+0000", 2024-09-20T18:55:42.746 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:42.746 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.746 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.746 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:55:42.746 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.983125+0000", 2024-09-20T18:55:42.746 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:42.747 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.747 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.747 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:55:42.747 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.983126+0000", 2024-09-20T18:55:42.747 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6899999999999996e-07 2024-09-20T18:55:42.747 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.747 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.747 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:55:42.747 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.983153+0000", 2024-09-20T18:55:42.747 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7821e-05 2024-09-20T18:55:42.747 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.747 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.747 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:55:42.747 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.983165+0000", 2024-09-20T18:55:42.747 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1443999999999999e-05 2024-09-20T18:55:42.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.748 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:55:42.748 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.983172+0000", 2024-09-20T18:55:42.748 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4070000000000004e-06 2024-09-20T18:55:42.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.748 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:55:42.748 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.989732+0000", 2024-09-20T18:55:42.748 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00656012 2024-09-20T18:55:42.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.748 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:55:42.748 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.989873+0000", 2024-09-20T18:55:42.748 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000141011 2024-09-20T18:55:42.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.749 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.749 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,6", 2024-09-20T18:55:42.749 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.990287+0000", 2024-09-20T18:55:42.749 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00041316299999999997 2024-09-20T18:55:42.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.749 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.749 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:55:42.749 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.993386+0000", 2024-09-20T18:55:42.749 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0030997239999999999 2024-09-20T18:55:42.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.749 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.749 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:42.749 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:24.995564+0000", 2024-09-20T18:55:42.750 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0021777810000000002 2024-09-20T18:55:42.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.750 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.750 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:42.750 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.086036+0000", 2024-09-20T18:55:42.750 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.090471996999999998 2024-09-20T18:55:42.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.750 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.750 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:55:42.750 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.086120+0000", 2024-09-20T18:55:42.750 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4123999999999996e-05 2024-09-20T18:55:42.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.750 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.750 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:55:42.751 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:25.086154+0000", 2024-09-20T18:55:42.751 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4075999999999997e-05 2024-09-20T18:55:42.751 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:42.751 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:55:42.751 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:42.751 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.751 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.751 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:14427 8.7 8:e293b0e9:::benchmark_data_smithi083_97422_object14426:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e166)", 2024-09-20T18:55:42.751 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:54:45.059514+0000", 2024-09-20T18:55:42.751 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 57.646647151000003, 2024-09-20T18:55:42.751 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.081623460999999994, 2024-09-20T18:55:42.751 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:55:42.751 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:55:42.752 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:42.752 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:55:42.752 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:55:42.752 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:55:42.753 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 14427 2024-09-20T18:55:42.753 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.753 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:55:42.753 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.753 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:55:42.753 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.059514+0000", 2024-09-20T18:55:42.753 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:42.753 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.753 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.753 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:55:42.753 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.059514+0000", 2024-09-20T18:55:42.753 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:42.753 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.753 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.754 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:55:42.754 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.059515+0000", 2024-09-20T18:55:42.754 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4799999999999997e-07 2024-09-20T18:55:42.754 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.754 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.754 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:55:42.754 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.059544+0000", 2024-09-20T18:55:42.754 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9852000000000001e-05 2024-09-20T18:55:42.754 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.754 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.754 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:55:42.754 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.059555+0000", 2024-09-20T18:55:42.754 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0273e-05 2024-09-20T18:55:42.754 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.755 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.755 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:55:42.755 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.059561+0000", 2024-09-20T18:55:42.755 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8340000000000001e-06 2024-09-20T18:55:42.755 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.755 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.755 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:55:42.755 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.060343+0000", 2024-09-20T18:55:42.755 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00078104200000000002 2024-09-20T18:55:42.755 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.755 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.755 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:55:42.755 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.060450+0000", 2024-09-20T18:55:42.755 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000107429 2024-09-20T18:55:42.755 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.756 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.756 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3,6", 2024-09-20T18:55:42.756 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.060572+0000", 2024-09-20T18:55:42.756 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121922 2024-09-20T18:55:42.756 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.756 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.756 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:55:42.756 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.062088+0000", 2024-09-20T18:55:42.756 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0015165409999999999 2024-09-20T18:55:42.756 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.756 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.756 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:42.756 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.141011+0000", 2024-09-20T18:55:42.756 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.078922292000000005 2024-09-20T18:55:42.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.757 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.757 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:42.757 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.141043+0000", 2024-09-20T18:55:42.757 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2514000000000001e-05 2024-09-20T18:55:42.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.757 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.757 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:55:42.757 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.141093+0000", 2024-09-20T18:55:42.757 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9806999999999999e-05 2024-09-20T18:55:42.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.757 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.757 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:55:42.757 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.141137+0000", 2024-09-20T18:55:42.757 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4007000000000001e-05 2024-09-20T18:55:42.758 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:42.758 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:55:42.758 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:42.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.758 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.758 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:14432 8.1 8:85904a75:::benchmark_data_smithi083_97422_object14431:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e166)", 2024-09-20T18:55:42.758 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:54:45.060657+0000", 2024-09-20T18:55:42.758 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 57.645503843, 2024-09-20T18:55:42.758 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.083750514999999998, 2024-09-20T18:55:42.758 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:55:42.758 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:55:42.758 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:42.758 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:55:42.758 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:55:42.759 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:55:42.759 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 14432 2024-09-20T18:55:42.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.759 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:55:42.759 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.759 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:55:42.759 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.060657+0000", 2024-09-20T18:55:42.759 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:42.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.759 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.759 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:55:42.759 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.060657+0000", 2024-09-20T18:55:42.759 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:42.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.760 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.760 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:55:42.760 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.060658+0000", 2024-09-20T18:55:42.760 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.034e-06 2024-09-20T18:55:42.760 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.760 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.760 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:55:42.760 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.060680+0000", 2024-09-20T18:55:42.760 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1617999999999999e-05 2024-09-20T18:55:42.760 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.760 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.760 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:55:42.760 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.060690+0000", 2024-09-20T18:55:42.760 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0271e-05 2024-09-20T18:55:42.760 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.761 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.761 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:55:42.761 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.060696+0000", 2024-09-20T18:55:42.761 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5490000000000003e-06 2024-09-20T18:55:42.761 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.761 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.761 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:55:42.761 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.060797+0000", 2024-09-20T18:55:42.761 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000100851 2024-09-20T18:55:42.761 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.761 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.761 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:55:42.761 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.060906+0000", 2024-09-20T18:55:42.761 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000109022 2024-09-20T18:55:42.761 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.762 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,2", 2024-09-20T18:55:42.762 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.061396+0000", 2024-09-20T18:55:42.762 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00049000300000000003 2024-09-20T18:55:42.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.762 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:42.762 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.141547+0000", 2024-09-20T18:55:42.762 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.080150684999999999 2024-09-20T18:55:42.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.762 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:55:42.762 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.141558+0000", 2024-09-20T18:55:42.762 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0917999999999999e-05 2024-09-20T18:55:42.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.763 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.763 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:42.763 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.144045+0000", 2024-09-20T18:55:42.763 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0024874749999999998 2024-09-20T18:55:42.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.763 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.763 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:55:42.763 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.144364+0000", 2024-09-20T18:55:42.763 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00031859599999999998 2024-09-20T18:55:42.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.763 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.763 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:55:42.763 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.144407+0000", 2024-09-20T18:55:42.763 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3492999999999998e-05 2024-09-20T18:55:42.764 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:42.764 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:55:42.764 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:42.764 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.764 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.764 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:14436 8.1 8:8394f3bb:::benchmark_data_smithi083_97422_object14435:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e166)", 2024-09-20T18:55:42.764 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:54:45.062241+0000", 2024-09-20T18:55:42.764 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 57.643920041999998, 2024-09-20T18:55:42.764 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088328078000000004, 2024-09-20T18:55:42.764 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:55:42.764 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:55:42.764 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:42.764 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:55:42.764 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:55:42.765 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:55:42.765 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 14436 2024-09-20T18:55:42.765 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.765 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:55:42.765 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.765 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:55:42.765 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.062241+0000", 2024-09-20T18:55:42.765 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:42.765 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.765 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.765 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:55:42.765 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.062241+0000", 2024-09-20T18:55:42.765 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:42.765 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.765 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.766 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:55:42.766 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.062242+0000", 2024-09-20T18:55:42.766 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0139999999999999e-06 2024-09-20T18:55:42.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.766 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.766 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:55:42.766 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.062264+0000", 2024-09-20T18:55:42.766 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2453999999999999e-05 2024-09-20T18:55:42.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.766 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.766 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:55:42.766 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.062272+0000", 2024-09-20T18:55:42.766 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9999999999999996e-06 2024-09-20T18:55:42.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.767 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.767 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:55:42.767 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.062279+0000", 2024-09-20T18:55:42.767 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5270000000000004e-06 2024-09-20T18:55:42.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.767 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.767 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:55:42.767 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.062351+0000", 2024-09-20T18:55:42.767 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1960999999999998e-05 2024-09-20T18:55:42.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.767 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.767 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:55:42.767 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.062460+0000", 2024-09-20T18:55:42.767 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000109555 2024-09-20T18:55:42.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.768 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.768 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,2", 2024-09-20T18:55:42.768 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.062587+0000", 2024-09-20T18:55:42.768 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012651200000000001 2024-09-20T18:55:42.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.768 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.768 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:42.768 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.144450+0000", 2024-09-20T18:55:42.768 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.081863135000000004 2024-09-20T18:55:42.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.768 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.768 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:42.768 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.144566+0000", 2024-09-20T18:55:42.768 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000116153 2024-09-20T18:55:42.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.769 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.769 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:55:42.769 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.150479+0000", 2024-09-20T18:55:42.769 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059128130000000003 2024-09-20T18:55:42.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.769 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.769 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:55:42.769 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.150526+0000", 2024-09-20T18:55:42.769 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7423000000000003e-05 2024-09-20T18:55:42.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.769 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.769 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:55:42.769 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.150569+0000", 2024-09-20T18:55:42.769 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2531000000000003e-05 2024-09-20T18:55:42.770 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:42.770 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:55:42.770 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:42.770 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.770 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.770 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:14439 8.3 8:c3b86491:::benchmark_data_smithi083_97422_object14438:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e166)", 2024-09-20T18:55:42.770 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:54:45.063495+0000", 2024-09-20T18:55:42.770 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 57.642665874999999, 2024-09-20T18:55:42.770 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.082542584000000002, 2024-09-20T18:55:42.770 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:55:42.770 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:55:42.770 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:42.770 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:55:42.770 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:55:42.771 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:55:42.771 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 14439 2024-09-20T18:55:42.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.771 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:55:42.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.771 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:55:42.771 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.063495+0000", 2024-09-20T18:55:42.771 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:42.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.771 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:55:42.771 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.063495+0000", 2024-09-20T18:55:42.771 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:42.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.772 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.772 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:55:42.772 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.063496+0000", 2024-09-20T18:55:42.772 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8599999999999997e-07 2024-09-20T18:55:42.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.772 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.772 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:55:42.772 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.063513+0000", 2024-09-20T18:55:42.772 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7453e-05 2024-09-20T18:55:42.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.772 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.772 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:55:42.772 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.141141+0000", 2024-09-20T18:55:42.772 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.077628100000000005 2024-09-20T18:55:42.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.773 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.773 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:55:42.773 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.141153+0000", 2024-09-20T18:55:42.773 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2075999999999999e-05 2024-09-20T18:55:42.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.773 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.773 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:55:42.773 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.141901+0000", 2024-09-20T18:55:42.773 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00074799299999999997 2024-09-20T18:55:42.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.773 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.773 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:55:42.773 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.141999+0000", 2024-09-20T18:55:42.774 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7962999999999997e-05 2024-09-20T18:55:42.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.774 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,6", 2024-09-20T18:55:42.774 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.142126+0000", 2024-09-20T18:55:42.774 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012721 2024-09-20T18:55:42.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.774 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:55:42.774 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.143904+0000", 2024-09-20T18:55:42.774 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001778047 2024-09-20T18:55:42.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.774 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:42.775 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.145899+0000", 2024-09-20T18:55:42.775 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0019940439999999999 2024-09-20T18:55:42.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.775 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:42.775 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.145944+0000", 2024-09-20T18:55:42.775 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5828999999999998e-05 2024-09-20T18:55:42.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.775 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:55:42.775 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.145997+0000", 2024-09-20T18:55:42.775 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3018e-05 2024-09-20T18:55:42.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.776 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:55:42.776 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:45.146037+0000", 2024-09-20T18:55:42.776 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0064999999999997e-05 2024-09-20T18:55:42.776 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:42.776 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:55:42.776 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:42.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.776 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.776 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:20779 8.a 8:5422596e:::benchmark_data_smithi083_97422_object20778:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e166)", 2024-09-20T18:55:42.776 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:54:47.509702+0000", 2024-09-20T18:55:42.776 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 55.196459177999998, 2024-09-20T18:55:42.776 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.082059367999999994, 2024-09-20T18:55:42.776 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:55:42.776 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:55:42.776 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:42.777 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:55:42.777 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:55:42.777 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:55:42.777 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 20779 2024-09-20T18:55:42.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.777 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:55:42.777 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.777 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:55:42.777 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:47.509702+0000", 2024-09-20T18:55:42.777 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:42.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.777 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.777 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:55:42.777 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:47.509702+0000", 2024-09-20T18:55:42.777 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:42.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.778 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.778 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:55:42.778 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:47.509703+0000", 2024-09-20T18:55:42.778 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7000000000000003e-07 2024-09-20T18:55:42.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.778 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.778 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:55:42.778 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:47.509736+0000", 2024-09-20T18:55:42.778 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3093000000000003e-05 2024-09-20T18:55:42.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.778 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.778 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:55:42.778 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:47.509746+0000", 2024-09-20T18:55:42.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:42.780 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:42.780 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:55:42.781 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0176999999999999e-05 2024-09-20T18:55:42.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.781 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:55:42.781 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:47.509753+0000", 2024-09-20T18:55:42.781 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2579999999999998e-06 2024-09-20T18:55:42.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.781 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:55:42.781 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:47.509795+0000", 2024-09-20T18:55:42.781 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1517999999999998e-05 2024-09-20T18:55:42.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.782 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:55:42.782 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:47.509894+0000", 2024-09-20T18:55:42.782 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9729999999999996e-05 2024-09-20T18:55:42.782 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.782 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.782 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,5", 2024-09-20T18:55:42.782 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:47.510059+0000", 2024-09-20T18:55:42.782 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016481699999999999 2024-09-20T18:55:42.782 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.782 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.782 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:55:42.782 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:47.510993+0000", 2024-09-20T18:55:42.782 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00093369500000000001 2024-09-20T18:55:42.782 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.783 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.783 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:42.783 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:47.511411+0000", 2024-09-20T18:55:42.783 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00041772599999999999 2024-09-20T18:55:42.783 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.783 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.783 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:42.783 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:47.591661+0000", 2024-09-20T18:55:42.783 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.080250552000000003 2024-09-20T18:55:42.783 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.783 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.783 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:55:42.783 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:47.591714+0000", 2024-09-20T18:55:42.783 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3254999999999997e-05 2024-09-20T18:55:42.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.784 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.784 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:55:42.784 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:47.591761+0000", 2024-09-20T18:55:42.784 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6576999999999998e-05 2024-09-20T18:55:42.784 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:42.784 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:55:42.784 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:42.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.784 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.784 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:23888 8.a 8:5b264bcf:::benchmark_data_smithi083_97422_object23887:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e167)", 2024-09-20T18:55:42.784 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:54:49.128149+0000", 2024-09-20T18:55:42.784 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 53.578012211000001, 2024-09-20T18:55:42.784 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.082606120000000005, 2024-09-20T18:55:42.785 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:55:42.785 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:55:42.785 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:42.785 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:55:42.785 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:55:42.785 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:55:42.785 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 23888 2024-09-20T18:55:42.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.785 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:55:42.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.785 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:55:42.785 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.128149+0000", 2024-09-20T18:55:42.785 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:42.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.786 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.786 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:55:42.786 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.128149+0000", 2024-09-20T18:55:42.786 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:42.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.786 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.786 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:55:42.786 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.128150+0000", 2024-09-20T18:55:42.786 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.356e-06 2024-09-20T18:55:42.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.786 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.786 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:55:42.786 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.128172+0000", 2024-09-20T18:55:42.786 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2252e-05 2024-09-20T18:55:42.787 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.787 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.787 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:55:42.787 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.128183+0000", 2024-09-20T18:55:42.787 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0624e-05 2024-09-20T18:55:42.787 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.787 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.787 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:55:42.787 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.128190+0000", 2024-09-20T18:55:42.787 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8539999999999999e-06 2024-09-20T18:55:42.787 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.787 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.787 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:55:42.787 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.128240+0000", 2024-09-20T18:55:42.788 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9920000000000003e-05 2024-09-20T18:55:42.788 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.788 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.788 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:55:42.788 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.128351+0000", 2024-09-20T18:55:42.788 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000111543 2024-09-20T18:55:42.788 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.788 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.788 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,5", 2024-09-20T18:55:42.788 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.128464+0000", 2024-09-20T18:55:42.788 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011314700000000001 2024-09-20T18:55:42.788 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.788 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.788 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:55:42.788 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.129546+0000", 2024-09-20T18:55:42.789 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0010820459999999999 2024-09-20T18:55:42.789 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.789 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.789 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:42.789 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.210405+0000", 2024-09-20T18:55:42.789 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.080858340000000001 2024-09-20T18:55:42.789 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.789 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.789 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:42.789 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.210667+0000", 2024-09-20T18:55:42.789 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00026220200000000001 2024-09-20T18:55:42.789 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.789 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.789 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:55:42.790 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.210717+0000", 2024-09-20T18:55:42.790 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0368999999999998e-05 2024-09-20T18:55:42.790 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.790 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.790 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:55:42.790 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.210755+0000", 2024-09-20T18:55:42.790 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7466999999999999e-05 2024-09-20T18:55:42.790 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:42.790 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:55:42.790 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:42.790 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.790 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.790 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.24955.0:23884 8.f e167/160 8:fdf3dc24:::benchmark_data_smithi083_97422_object23883:head v 167'1519, mlcod=167'1517)", 2024-09-20T18:55:42.790 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:54:49.128800+0000", 2024-09-20T18:55:42.791 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 53.577361197000002, 2024-09-20T18:55:42.791 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088377654, 2024-09-20T18:55:42.791 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:55:42.791 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:55:42.791 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:42.791 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:55:42.791 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.791 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:55:42.791 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.128800+0000", 2024-09-20T18:55:42.791 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:42.791 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.791 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.791 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:55:42.791 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.128800+0000", 2024-09-20T18:55:42.791 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:42.792 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.792 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.792 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:55:42.792 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.128800+0000", 2024-09-20T18:55:42.792 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8100000000000001e-07 2024-09-20T18:55:42.792 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.792 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.792 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:55:42.792 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.128834+0000", 2024-09-20T18:55:42.792 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3460999999999997e-05 2024-09-20T18:55:42.792 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.792 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.792 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:55:42.792 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.128843+0000", 2024-09-20T18:55:42.792 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4579999999999999e-06 2024-09-20T18:55:42.793 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.793 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.793 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:55:42.793 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.128858+0000", 2024-09-20T18:55:42.793 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5078e-05 2024-09-20T18:55:42.793 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.793 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.793 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:55:42.793 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.128943+0000", 2024-09-20T18:55:42.793 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4929e-05 2024-09-20T18:55:42.793 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.793 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.793 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:55:42.793 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.129026+0000", 2024-09-20T18:55:42.794 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2237999999999994e-05 2024-09-20T18:55:42.794 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.794 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.794 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:55:42.794 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.217130+0000", 2024-09-20T18:55:42.794 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088104772999999997 2024-09-20T18:55:42.794 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.794 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.794 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:55:42.794 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.217177+0000", 2024-09-20T18:55:42.794 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6835999999999997e-05 2024-09-20T18:55:42.795 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:42.795 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:55:42.795 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:42.795 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.795 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.795 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:23890 8.1 8:8d066e57:::benchmark_data_smithi083_97422_object23889:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e167)", 2024-09-20T18:55:42.795 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:54:49.129625+0000", 2024-09-20T18:55:42.795 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 53.576535499999999, 2024-09-20T18:55:42.795 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.085897344, 2024-09-20T18:55:42.795 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:55:42.795 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:55:42.795 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:42.795 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:55:42.795 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:55:42.796 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:55:42.796 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 23890 2024-09-20T18:55:42.796 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.796 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:55:42.796 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.796 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:55:42.796 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.129625+0000", 2024-09-20T18:55:42.796 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:42.796 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.796 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.796 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:55:42.796 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.129625+0000", 2024-09-20T18:55:42.796 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:42.796 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.796 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.797 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:55:42.797 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.129627+0000", 2024-09-20T18:55:42.797 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.454e-06 2024-09-20T18:55:42.797 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.797 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.797 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:55:42.797 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.129651+0000", 2024-09-20T18:55:42.797 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4091999999999999e-05 2024-09-20T18:55:42.797 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.797 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.797 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:55:42.797 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.129662+0000", 2024-09-20T18:55:42.797 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1412e-05 2024-09-20T18:55:42.797 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.797 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.798 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:55:42.798 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.129669+0000", 2024-09-20T18:55:42.798 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9659999999999999e-06 2024-09-20T18:55:42.798 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.798 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.798 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:55:42.798 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.129813+0000", 2024-09-20T18:55:42.798 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014401600000000001 2024-09-20T18:55:42.798 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.798 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.798 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:55:42.798 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.208665+0000", 2024-09-20T18:55:42.798 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.078852169 2024-09-20T18:55:42.798 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.799 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.799 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,2", 2024-09-20T18:55:42.799 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.208815+0000", 2024-09-20T18:55:42.799 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000150023 2024-09-20T18:55:42.799 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.799 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.799 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:55:42.799 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.210426+0000", 2024-09-20T18:55:42.799 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00161086 2024-09-20T18:55:42.799 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.799 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.799 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:42.799 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.211441+0000", 2024-09-20T18:55:42.799 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0010149149999999999 2024-09-20T18:55:42.800 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.800 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.800 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:42.800 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.215002+0000", 2024-09-20T18:55:42.800 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0035611409999999999 2024-09-20T18:55:42.800 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.800 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.800 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:55:42.800 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.215471+0000", 2024-09-20T18:55:42.800 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00046846099999999999 2024-09-20T18:55:42.800 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.800 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.800 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:55:42.801 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.215523+0000", 2024-09-20T18:55:42.801 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1835000000000001e-05 2024-09-20T18:55:42.801 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:42.801 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:55:42.801 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:42.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.801 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.801 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:23891 8.7 8:ebd0e138:::benchmark_data_smithi083_97422_object23890:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e167)", 2024-09-20T18:55:42.801 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:54:49.129687+0000", 2024-09-20T18:55:42.801 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 53.576473982000003, 2024-09-20T18:55:42.801 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087685760000000001, 2024-09-20T18:55:42.801 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:55:42.801 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:55:42.801 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:42.801 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:55:42.802 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:55:42.802 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:55:42.802 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 23891 2024-09-20T18:55:42.802 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.802 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:55:42.802 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.802 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:55:42.802 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.129687+0000", 2024-09-20T18:55:42.802 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:42.802 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.802 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.802 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:55:42.802 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.129687+0000", 2024-09-20T18:55:42.803 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:42.803 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.803 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.803 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:55:42.803 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.129687+0000", 2024-09-20T18:55:42.803 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2e-07 2024-09-20T18:55:42.803 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.803 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.803 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:55:42.803 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.129706+0000", 2024-09-20T18:55:42.803 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8644e-05 2024-09-20T18:55:42.803 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.803 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.803 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:55:42.803 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.129712+0000", 2024-09-20T18:55:42.804 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1099999999999999e-06 2024-09-20T18:55:42.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.804 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.804 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:55:42.804 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.129717+0000", 2024-09-20T18:55:42.804 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8810000000000004e-06 2024-09-20T18:55:42.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.804 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.804 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:55:42.804 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.129763+0000", 2024-09-20T18:55:42.804 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5977999999999999e-05 2024-09-20T18:55:42.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.804 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.804 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:55:42.805 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.209846+0000", 2024-09-20T18:55:42.805 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.080082634999999999 2024-09-20T18:55:42.805 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.805 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.805 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3,6", 2024-09-20T18:55:42.805 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.209999+0000", 2024-09-20T18:55:42.805 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015369299999999999 2024-09-20T18:55:42.805 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.805 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.805 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:42.805 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.215885+0000", 2024-09-20T18:55:42.805 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058861590000000002 2024-09-20T18:55:42.806 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.806 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.806 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:42.806 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.216185+0000", 2024-09-20T18:55:42.806 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00030002300000000001 2024-09-20T18:55:42.806 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.806 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.806 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:55:42.806 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.217281+0000", 2024-09-20T18:55:42.806 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0010956099999999999 2024-09-20T18:55:42.806 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.806 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.806 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:55:42.806 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.217330+0000", 2024-09-20T18:55:42.807 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8748000000000002e-05 2024-09-20T18:55:42.807 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.807 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.807 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:55:42.807 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.217373+0000", 2024-09-20T18:55:42.807 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2759000000000002e-05 2024-09-20T18:55:42.807 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:42.807 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:55:42.807 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:42.807 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.807 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.807 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:23892 8.7 8:efa838ec:::benchmark_data_smithi083_97422_object23891:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e167)", 2024-09-20T18:55:42.807 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:54:49.129733+0000", 2024-09-20T18:55:42.807 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 53.576428157999999, 2024-09-20T18:55:42.808 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.087725431000000006, 2024-09-20T18:55:42.808 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:55:42.808 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:55:42.808 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:42.808 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:55:42.808 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:55:42.808 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:55:42.808 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 23892 2024-09-20T18:55:42.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.808 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:55:42.808 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.808 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:55:42.808 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.129733+0000", 2024-09-20T18:55:42.808 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:42.809 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.809 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.809 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:55:42.809 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.129733+0000", 2024-09-20T18:55:42.809 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:42.809 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.809 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.809 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:55:42.809 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.129733+0000", 2024-09-20T18:55:42.809 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4400000000000001e-07 2024-09-20T18:55:42.809 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.809 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.809 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:55:42.809 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.129751+0000", 2024-09-20T18:55:42.810 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8297000000000001e-05 2024-09-20T18:55:42.810 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.810 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.810 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:55:42.810 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.129757+0000", 2024-09-20T18:55:42.810 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7370000000000003e-06 2024-09-20T18:55:42.810 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.810 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.810 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:55:42.810 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.129762+0000", 2024-09-20T18:55:42.810 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8930000000000004e-06 2024-09-20T18:55:42.810 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.810 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.810 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:55:42.811 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.212019+0000", 2024-09-20T18:55:42.811 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.082257489000000003 2024-09-20T18:55:42.811 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.811 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.811 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:55:42.811 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.212122+0000", 2024-09-20T18:55:42.811 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000102533 2024-09-20T18:55:42.811 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.811 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.811 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3,6", 2024-09-20T18:55:42.811 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.212240+0000", 2024-09-20T18:55:42.811 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000117605 2024-09-20T18:55:42.811 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.811 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.811 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:42.812 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.215926+0000", 2024-09-20T18:55:42.812 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0036867789999999998 2024-09-20T18:55:42.812 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.812 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.812 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:42.812 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.217222+0000", 2024-09-20T18:55:42.812 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001295301 2024-09-20T18:55:42.812 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.812 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.812 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:55:42.812 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.217381+0000", 2024-09-20T18:55:42.812 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015948399999999999 2024-09-20T18:55:42.812 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.812 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.813 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:55:42.813 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.217424+0000", 2024-09-20T18:55:42.813 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3328000000000003e-05 2024-09-20T18:55:42.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.813 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.813 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:55:42.813 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:54:49.217458+0000", 2024-09-20T18:55:42.813 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3541000000000003e-05 2024-09-20T18:55:42.813 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:42.813 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:55:42.813 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:42.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.813 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.813 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:85534 8.a 8:56c6418e:::benchmark_data_smithi083_97422_object85533:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e185)", 2024-09-20T18:55:42.813 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:55:32.749628+0000", 2024-09-20T18:55:42.814 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 9.9565324769999997, 2024-09-20T18:55:42.814 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.108697138, 2024-09-20T18:55:42.814 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:55:42.814 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:55:42.814 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:42.814 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:55:42.814 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:55:42.814 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:55:42.814 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 85534 2024-09-20T18:55:42.814 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.814 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:55:42.814 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.814 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:55:42.814 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.749628+0000", 2024-09-20T18:55:42.814 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:42.815 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.815 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.815 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:55:42.815 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.749628+0000", 2024-09-20T18:55:42.815 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:42.815 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.815 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.815 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:55:42.815 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.749630+0000", 2024-09-20T18:55:42.815 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6509999999999999e-06 2024-09-20T18:55:42.815 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.815 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.815 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:55:42.815 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.749651+0000", 2024-09-20T18:55:42.816 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1566999999999999e-05 2024-09-20T18:55:42.816 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.816 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.816 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:55:42.816 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.749662+0000", 2024-09-20T18:55:42.816 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0242e-05 2024-09-20T18:55:42.816 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.816 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.816 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:55:42.816 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.749668+0000", 2024-09-20T18:55:42.816 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6059999999999999e-06 2024-09-20T18:55:42.816 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.816 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.816 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:55:42.817 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.752395+0000", 2024-09-20T18:55:42.817 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0027269999999999998 2024-09-20T18:55:42.817 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.817 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.817 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:55:42.817 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.752970+0000", 2024-09-20T18:55:42.817 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00057473599999999995 2024-09-20T18:55:42.817 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.817 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.817 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,5", 2024-09-20T18:55:42.817 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.753097+0000", 2024-09-20T18:55:42.817 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000127391 2024-09-20T18:55:42.817 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.817 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.817 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:55:42.818 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.755412+0000", 2024-09-20T18:55:42.818 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0023147659999999998 2024-09-20T18:55:42.818 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.818 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.818 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:42.818 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.855565+0000", 2024-09-20T18:55:42.818 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.10015248 2024-09-20T18:55:42.818 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.818 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.818 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:42.818 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.858229+0000", 2024-09-20T18:55:42.818 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0026640940000000001 2024-09-20T18:55:42.818 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.819 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.819 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:55:42.819 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.858283+0000", 2024-09-20T18:55:42.819 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4379000000000002e-05 2024-09-20T18:55:42.819 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.819 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.819 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:55:42.819 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.858325+0000", 2024-09-20T18:55:42.819 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2225999999999999e-05 2024-09-20T18:55:42.819 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:42.819 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:55:42.819 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:42.819 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.819 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.820 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:85537 8.3 8:c2a20bfd:::benchmark_data_smithi083_97422_object85536:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e185)", 2024-09-20T18:55:42.820 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:55:32.749686+0000", 2024-09-20T18:55:42.820 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 9.9564751820000001, 2024-09-20T18:55:42.820 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.111365674, 2024-09-20T18:55:42.820 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:55:42.820 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:55:42.820 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:42.820 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:55:42.820 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:55:42.820 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:55:42.820 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 85537 2024-09-20T18:55:42.820 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.820 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:55:42.821 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.821 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:55:42.821 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.749686+0000", 2024-09-20T18:55:42.821 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:42.821 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.822 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.822 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:55:42.822 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.749686+0000", 2024-09-20T18:55:42.822 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:42.822 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.822 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.822 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:55:42.822 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.749686+0000", 2024-09-20T18:55:42.822 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7300000000000001e-07 2024-09-20T18:55:42.822 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.822 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.822 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:55:42.822 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.749704+0000", 2024-09-20T18:55:42.823 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7626e-05 2024-09-20T18:55:42.823 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.823 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.823 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:55:42.823 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.749709+0000", 2024-09-20T18:55:42.823 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5130000000000004e-06 2024-09-20T18:55:42.823 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.823 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.823 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:55:42.823 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.749714+0000", 2024-09-20T18:55:42.823 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4139999999999996e-06 2024-09-20T18:55:42.823 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.823 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.823 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:55:42.823 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.750397+0000", 2024-09-20T18:55:42.824 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00068380199999999996 2024-09-20T18:55:42.824 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.824 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.824 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:55:42.824 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.750520+0000", 2024-09-20T18:55:42.824 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000122516 2024-09-20T18:55:42.824 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.824 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.824 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,5,6", 2024-09-20T18:55:42.824 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.750661+0000", 2024-09-20T18:55:42.824 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014089800000000001 2024-09-20T18:55:42.824 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.824 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.824 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:42.825 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.755638+0000", 2024-09-20T18:55:42.825 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049767199999999996 2024-09-20T18:55:42.825 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.825 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.825 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:42.825 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.756500+0000", 2024-09-20T18:55:42.825 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00086244700000000002 2024-09-20T18:55:42.825 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.825 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.825 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:42.825 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.756547+0000", 2024-09-20T18:55:42.825 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6971999999999997e-05 2024-09-20T18:55:42.825 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.825 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.826 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:55:42.826 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.860879+0000", 2024-09-20T18:55:42.826 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.104331983 2024-09-20T18:55:42.826 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.826 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.826 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:55:42.826 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.860938+0000", 2024-09-20T18:55:42.826 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9268999999999998e-05 2024-09-20T18:55:42.826 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.826 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.826 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:55:42.826 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.861051+0000", 2024-09-20T18:55:42.826 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000113041 2024-09-20T18:55:42.826 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:42.827 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:55:42.827 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:42.827 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.827 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.827 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.24955.0:85539 8.b e185/160 8:d35f6e4c:::benchmark_data_smithi083_97422_object85538:head v 185'5368, mlcod=185'5365)", 2024-09-20T18:55:42.827 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:55:32.751299+0000", 2024-09-20T18:55:42.827 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 9.9548616750000001, 2024-09-20T18:55:42.827 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.109880872, 2024-09-20T18:55:42.827 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:55:42.827 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:55:42.827 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:42.827 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:55:42.827 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.827 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:55:42.828 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.751299+0000", 2024-09-20T18:55:42.828 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:42.828 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.828 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.828 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:55:42.828 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.751299+0000", 2024-09-20T18:55:42.828 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:42.828 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.828 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.828 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:55:42.828 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.751299+0000", 2024-09-20T18:55:42.828 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2399999999999999e-07 2024-09-20T18:55:42.828 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.828 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.829 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:55:42.829 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.751320+0000", 2024-09-20T18:55:42.829 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0324000000000001e-05 2024-09-20T18:55:42.829 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.829 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.829 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:55:42.829 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.751326+0000", 2024-09-20T18:55:42.829 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5889999999999999e-06 2024-09-20T18:55:42.829 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.829 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.829 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:55:42.829 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.751331+0000", 2024-09-20T18:55:42.829 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5730000000000001e-06 2024-09-20T18:55:42.829 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.830 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.830 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:55:42.830 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.753270+0000", 2024-09-20T18:55:42.830 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001938925 2024-09-20T18:55:42.830 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.830 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.830 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:55:42.830 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.753287+0000", 2024-09-20T18:55:42.830 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6844e-05 2024-09-20T18:55:42.830 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.830 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.830 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:55:42.830 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.861119+0000", 2024-09-20T18:55:42.830 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.107831865 2024-09-20T18:55:42.831 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.831 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.831 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:55:42.831 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.861180+0000", 2024-09-20T18:55:42.831 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1328000000000006e-05 2024-09-20T18:55:42.831 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:42.831 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:55:42.831 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:42.831 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.831 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.831 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:85542 8.3 8:cafa02d3:::benchmark_data_smithi083_97422_object85541:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e185)", 2024-09-20T18:55:42.831 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:55:32.756596+0000", 2024-09-20T18:55:42.831 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 9.9495648629999991, 2024-09-20T18:55:42.831 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.115124421, 2024-09-20T18:55:42.831 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:55:42.832 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:55:42.832 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:42.832 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:55:42.832 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:55:42.832 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:55:42.832 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 85542 2024-09-20T18:55:42.832 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.832 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:55:42.832 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.832 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:55:42.832 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.756596+0000", 2024-09-20T18:55:42.832 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:42.832 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.832 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.832 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:55:42.833 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.756596+0000", 2024-09-20T18:55:42.833 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:42.833 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.833 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.833 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:55:42.833 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.756597+0000", 2024-09-20T18:55:42.833 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.344e-06 2024-09-20T18:55:42.833 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.833 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.833 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:55:42.833 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.756617+0000", 2024-09-20T18:55:42.833 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9287999999999999e-05 2024-09-20T18:55:42.833 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.833 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.833 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:55:42.834 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.756628+0000", 2024-09-20T18:55:42.834 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1026000000000001e-05 2024-09-20T18:55:42.834 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.834 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.834 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:55:42.834 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.756635+0000", 2024-09-20T18:55:42.834 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7819999999999994e-06 2024-09-20T18:55:42.834 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.834 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.834 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:55:42.834 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.756830+0000", 2024-09-20T18:55:42.834 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019426900000000001 2024-09-20T18:55:42.834 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.834 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.834 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:55:42.835 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.756959+0000", 2024-09-20T18:55:42.835 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000129445 2024-09-20T18:55:42.835 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.835 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.835 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,5,6", 2024-09-20T18:55:42.835 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.856373+0000", 2024-09-20T18:55:42.835 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.099413827999999996 2024-09-20T18:55:42.835 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.835 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.835 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:55:42.835 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.861188+0000", 2024-09-20T18:55:42.835 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0048153989999999997 2024-09-20T18:55:42.835 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.835 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.836 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:42.836 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.865907+0000", 2024-09-20T18:55:42.836 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.004719226 2024-09-20T18:55:42.836 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.836 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.836 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:42.836 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.865949+0000", 2024-09-20T18:55:42.836 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1958e-05 2024-09-20T18:55:42.836 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.836 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.836 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:42.836 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.871462+0000", 2024-09-20T18:55:42.836 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0055120689999999996 2024-09-20T18:55:42.836 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.837 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.837 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:55:42.837 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.871552+0000", 2024-09-20T18:55:42.837 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0488999999999994e-05 2024-09-20T18:55:42.837 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.837 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.837 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:55:42.837 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.871720+0000", 2024-09-20T18:55:42.837 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016829799999999999 2024-09-20T18:55:42.837 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:42.837 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:55:42.837 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:42.837 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.837 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.837 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.24955.0:85534 8.a e185/160 ondisk, result = 0)", 2024-09-20T18:55:42.838 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:55:32.757361+0000", 2024-09-20T18:55:42.838 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 9.9487999990000002, 2024-09-20T18:55:42.838 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098211889999999996, 2024-09-20T18:55:42.838 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:55:42.838 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:55:42.838 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:55:42.838 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:55:42.838 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.838 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:55:42.838 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.757361+0000", 2024-09-20T18:55:42.838 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:42.838 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.838 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.838 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:55:42.839 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.757361+0000", 2024-09-20T18:55:42.839 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:42.839 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.839 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.839 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:55:42.839 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.757362+0000", 2024-09-20T18:55:42.839 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5000000000000001e-07 2024-09-20T18:55:42.839 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.839 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.839 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:55:42.839 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.757366+0000", 2024-09-20T18:55:42.839 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1500000000000001e-06 2024-09-20T18:55:42.839 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.839 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.839 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:55:42.840 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.855471+0000", 2024-09-20T18:55:42.840 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098105214999999996 2024-09-20T18:55:42.840 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.840 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.840 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:55:42.840 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.855486+0000", 2024-09-20T18:55:42.840 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4847e-05 2024-09-20T18:55:42.840 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.840 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.840 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:55:42.840 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.855550+0000", 2024-09-20T18:55:42.840 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3899000000000006e-05 2024-09-20T18:55:42.840 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.840 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.841 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:55:42.841 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.855558+0000", 2024-09-20T18:55:42.841 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.1629999999999996e-06 2024-09-20T18:55:42.841 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.841 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.841 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:55:42.841 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.855573+0000", 2024-09-20T18:55:42.841 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4766000000000001e-05 2024-09-20T18:55:42.841 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:42.841 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:55:42.841 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:42.841 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.841 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.841 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:85550 8.1 8:89baf99d:::benchmark_data_smithi083_97422_object85549:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e185)", 2024-09-20T18:55:42.842 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:55:32.766674+0000", 2024-09-20T18:55:42.842 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 9.9394863440000005, 2024-09-20T18:55:42.842 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.10435720699999999, 2024-09-20T18:55:42.842 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:55:42.842 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:55:42.842 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:55:42.842 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:55:42.842 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:55:42.842 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:55:42.842 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 85550 2024-09-20T18:55:42.842 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.842 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:55:42.842 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.842 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:55:42.843 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.766674+0000", 2024-09-20T18:55:42.843 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:42.843 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.843 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.843 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:55:42.843 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.766674+0000", 2024-09-20T18:55:42.843 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:42.843 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.843 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.843 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:55:42.843 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.766677+0000", 2024-09-20T18:55:42.843 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.711e-06 2024-09-20T18:55:42.843 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.843 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.844 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:55:42.844 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.766714+0000", 2024-09-20T18:55:42.844 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7190000000000001e-05 2024-09-20T18:55:42.844 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.844 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.844 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:55:42.844 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.855473+0000", 2024-09-20T18:55:42.844 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088758879999999998 2024-09-20T18:55:42.844 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.844 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.844 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:55:42.844 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.855487+0000", 2024-09-20T18:55:42.844 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.413e-05 2024-09-20T18:55:42.844 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.845 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.845 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:55:42.845 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.855626+0000", 2024-09-20T18:55:42.845 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000139092 2024-09-20T18:55:42.845 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.845 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.845 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:55:42.845 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.855744+0000", 2024-09-20T18:55:42.845 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000117812 2024-09-20T18:55:42.845 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.845 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.845 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,2", 2024-09-20T18:55:42.845 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.855891+0000", 2024-09-20T18:55:42.845 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014677900000000001 2024-09-20T18:55:42.845 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.845 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.846 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:55:42.846 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.864260+0000", 2024-09-20T18:55:42.846 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.008368749 2024-09-20T18:55:42.846 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.846 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.846 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:42.846 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.870859+0000", 2024-09-20T18:55:42.846 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006599359 2024-09-20T18:55:42.846 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.846 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.846 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:55:42.846 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.870898+0000", 2024-09-20T18:55:42.846 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8473000000000002e-05 2024-09-20T18:55:42.846 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.847 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.847 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:55:42.847 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.870983+0000", 2024-09-20T18:55:42.847 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4958999999999998e-05 2024-09-20T18:55:42.847 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:42.847 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:42.847 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:55:42.847 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:32.871032+0000", 2024-09-20T18:55:42.847 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9073000000000003e-05 2024-09-20T18:55:42.847 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:42.847 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:55:42.847 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:42.847 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:42.847 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:55:42.847 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T18:55:42.881 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:42.905 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:42.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:42.906 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:43.007 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:43.034 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:43.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:43.035 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:43.051 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:42 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:55:43.051 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:42 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:55:43.051 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:42 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:55:43.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:42 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:55:43.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:42 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:55:43.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:42 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:55:43.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:42 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:55:43.052 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:42 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:55:43.052 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:42 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:55:43.052 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:42 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:55:43.052 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:42 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:55:43.052 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:42 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:55:43.052 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:42 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:55:43.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:42 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:55:43.136 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:43.160 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:43.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:43.161 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:43.181 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 63 15 94014 93999 93.2113 48.375 0.0105786 0.010723 2024-09-20T18:55:43.261 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:43.271 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:42 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:55:43.271 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:42 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:55:43.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:42 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:55:43.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:42 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:55:43.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:42 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:55:43.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:42 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:55:43.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:42 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:55:43.289 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:43.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:43.290 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:43.391 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:43.421 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:43.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:43.422 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:43.522 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:43.548 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:43.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:43.549 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:43.650 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:43.693 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:43.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:43.694 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:43.795 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:55:43.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:43.841 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:43.842 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:55:43.942 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:43.957 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:55:43.967 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:43.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:43.969 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:44.070 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:55:44.092 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:43 smithi083 ceph-mon[29421]: pgmap v281: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 143 active+clean; 5.6 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 64 MiB/s wr, 1.02k op/s; 24937/277827 objects misplaced (8.976%); 16 MiB/s, 251 objects/s recovering 2024-09-20T18:55:44.092 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:43 smithi083 ceph-mon[36260]: pgmap v281: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 143 active+clean; 5.6 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 64 MiB/s wr, 1.02k op/s; 24937/277827 objects misplaced (8.976%); 16 MiB/s, 251 objects/s recovering 2024-09-20T18:55:44.117 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:44.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:44.118 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:55:44.181 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 64 16 94900 94884 92.6192 55.3125 0.0157293 0.0107911 2024-09-20T18:55:44.219 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:44.246 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:44.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:44.247 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:44.271 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:43 smithi142 ceph-mon[32508]: pgmap v281: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 143 active+clean; 5.6 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 64 MiB/s wr, 1.02k op/s; 24937/277827 objects misplaced (8.976%); 16 MiB/s, 251 objects/s recovering 2024-09-20T18:55:44.348 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:55:44.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:44.392 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:44.392 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:55:44.430 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T18:55:44.494 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:44.524 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:44.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:44.525 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:44.575 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-20T18:55:44.626 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:44.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:44.674 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:44.674 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:44.775 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:44.834 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:44.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:44.835 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:44.936 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:44.990 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:44.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:44.991 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:45.092 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:45.117 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:45.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:45.118 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:45.188 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 65 16 95666 95650 91.924 47.875 0.0151035 0.0108708 2024-09-20T18:55:45.188 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:55:45.219 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:45.277 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:45.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:45.278 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:45.379 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:45.405 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:45.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:45.406 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:45.499 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:55:45.507 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:45.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:45.529 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:45.529 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:45.556 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T18:55:45.630 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:55:45.710 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:45.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:45.712 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:55:45.813 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:45.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:45.858 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:45.858 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:45.959 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:46.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:46.005 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:46.005 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:46.106 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:55:46.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:46.142 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:46.143 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:55:46.186 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 66 16 96346 96330 91.1755 42.5 0.0119209 0.0109622 2024-09-20T18:55:46.244 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:46.282 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:46.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:46.283 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:46.383 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:55:46.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:46.420 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:46.420 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:55:46.514 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.1 to 1 (10000) 2024-09-20T18:55:46.516 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-20T18:55:46.521 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:46.546 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:46.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:46.547 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:46.647 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:46.678 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:46.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:46.679 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:46.780 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:46.802 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:46 smithi083 ceph-mon[29421]: pgmap v282: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 143 active+clean; 5.6 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 639 B/s rd, 32 MiB/s wr, 505 op/s; 24937/277827 objects misplaced (8.976%); 7.6 MiB/s, 121 objects/s recovering 2024-09-20T18:55:46.802 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:46 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:55:46.802 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:46 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:55:46.802 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:46 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2093155829' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-20T18:55:46.802 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:46 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1620338741' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:55:46.802 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:46 smithi083 ceph-mon[36260]: pgmap v282: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 143 active+clean; 5.6 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 639 B/s rd, 32 MiB/s wr, 505 op/s; 24937/277827 objects misplaced (8.976%); 7.6 MiB/s, 121 objects/s recovering 2024-09-20T18:55:46.802 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:46 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:55:46.802 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:46 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:55:46.802 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:46 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2093155829' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-20T18:55:46.803 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:46 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1620338741' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:55:46.812 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:46.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:46.813 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:46.914 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:46.942 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:46.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:46.943 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:47.021 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:55:46 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:55:47.021 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:46 smithi142 ceph-mon[32508]: pgmap v282: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 143 active+clean; 5.6 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 639 B/s rd, 32 MiB/s wr, 505 op/s; 24937/277827 objects misplaced (8.976%); 7.6 MiB/s, 121 objects/s recovering 2024-09-20T18:55:47.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:46 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:55:47.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:46 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:55:47.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:46 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2093155829' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-20T18:55:47.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:46 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1620338741' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:55:47.043 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:47.064 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:47.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:47.064 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:47.165 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:55:47.186 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 67 15 97103 97088 90.5219 47.375 0.00480494 0.0110421 2024-09-20T18:55:47.205 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:47.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:47.207 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:55:47.307 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:47.339 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:47.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:47.340 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:47.441 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:47.480 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:47.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:47.481 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:47.581 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:47.609 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:47.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:47.610 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:47.711 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:47.737 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:47.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:47.738 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:47.801 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:47 smithi083 ceph-mon[29421]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:55:47.801 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:47 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2093155829' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-20T18:55:47.801 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:47 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1620338741' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T18:55:47.802 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:47 smithi083 ceph-mon[29421]: osdmap e189: 8 total, 8 up, 7 in 2024-09-20T18:55:47.802 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:47 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:55:47.802 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:47 smithi083 ceph-mon[36260]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:55:47.802 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:47 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2093155829' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-20T18:55:47.802 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:47 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1620338741' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T18:55:47.802 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:47 smithi083 ceph-mon[36260]: osdmap e189: 8 total, 8 up, 7 in 2024-09-20T18:55:47.803 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:47 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:55:47.838 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:55:47.880 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:47.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:47.881 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:55:47.982 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:48.007 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:48.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:48.008 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:48.021 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:47 smithi142 ceph-mon[32508]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:55:48.021 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:47 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2093155829' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-20T18:55:48.021 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:47 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1620338741' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T18:55:48.021 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:47 smithi142 ceph-mon[32508]: osdmap e189: 8 total, 8 up, 7 in 2024-09-20T18:55:48.021 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:47 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:55:48.109 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:48.133 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:48.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:48.134 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:48.187 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 68 16 97931 97915 89.9505 51.6875 0.0175021 0.011109 2024-09-20T18:55:48.235 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:48.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:48.273 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:48.274 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:48.374 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:48.402 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:48.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:48.403 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:48.504 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:48.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:48.542 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:48.543 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:48.643 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:48.670 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:48.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:48.671 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:48.772 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:48.801 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:48.802 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:48 smithi083 ceph-mon[29421]: pgmap v284: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 143 active+clean; 5.9 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.2 KiB/s rd, 62 MiB/s wr, 994 op/s; 24426/289551 objects misplaced (8.436%); 15 MiB/s, 238 objects/s recovering 2024-09-20T18:55:48.802 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:48 smithi083 ceph-mon[36260]: pgmap v284: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 143 active+clean; 5.9 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.2 KiB/s rd, 62 MiB/s wr, 994 op/s; 24426/289551 objects misplaced (8.436%); 15 MiB/s, 238 objects/s recovering 2024-09-20T18:55:48.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:48.803 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:48.903 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:48.924 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:48.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:48.925 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:49.021 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:48 smithi142 ceph-mon[32508]: pgmap v284: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 143 active+clean; 5.9 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.2 KiB/s rd, 62 MiB/s wr, 994 op/s; 24426/289551 objects misplaced (8.436%); 15 MiB/s, 238 objects/s recovering 2024-09-20T18:55:49.025 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:49.053 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:49.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:49.054 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:49.155 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:55:49.187 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 69 16 98808 98792 89.4414 54.8125 0.0125867 0.0111754 2024-09-20T18:55:49.192 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:49.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:49.193 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:55:49.294 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:55:49.326 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:49.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:49.328 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:55:49.428 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:49.458 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:49.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:49.459 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:49.560 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:49.583 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:49.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:49.584 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:49.685 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:49.719 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:49.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:49.720 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:49.821 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:55:49.858 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:49.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:49.859 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:55:49.959 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:49.989 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:49.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:49.990 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:50.091 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:55:50.125 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:50.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:50.126 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:55:50.187 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 70 16 99599 99583 88.87 49.4375 0.00961616 0.0112472 2024-09-20T18:55:50.226 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:50.250 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:50.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:50.251 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:50.352 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:55:50.385 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:50.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:50.387 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:55:50.488 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:50.526 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:50.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:50.527 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:50.627 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:50.650 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:50.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:50.651 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:50.752 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:50.778 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:50.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:50.779 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:50.801 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:50 smithi083 ceph-mon[29421]: pgmap v285: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 143 active+clean; 5.9 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.2 KiB/s rd, 61 MiB/s wr, 969 op/s; 24426/289551 objects misplaced (8.436%); 15 MiB/s, 232 objects/s recovering 2024-09-20T18:55:50.801 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:50 smithi083 ceph-mon[36260]: pgmap v285: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 143 active+clean; 5.9 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.2 KiB/s rd, 61 MiB/s wr, 969 op/s; 24426/289551 objects misplaced (8.436%); 15 MiB/s, 232 objects/s recovering 2024-09-20T18:55:50.802 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:55:50 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:55:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:55:50.880 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:50.916 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:50.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:50.917 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:51.018 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:51.021 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:50 smithi142 ceph-mon[32508]: pgmap v285: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 143 active+clean; 5.9 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.2 KiB/s rd, 61 MiB/s wr, 969 op/s; 24426/289551 objects misplaced (8.436%); 15 MiB/s, 232 objects/s recovering 2024-09-20T18:55:51.044 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:51.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:51.046 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:51.146 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:51.172 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:51.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:51.173 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:51.187 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 71 15 100422 100407 88.3439 51.5 0.00677358 0.0113145 2024-09-20T18:55:51.274 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:51.301 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:51.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:51.302 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:51.403 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:51.436 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:51.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:51.438 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:51.538 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:51.564 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:51.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:51.565 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:51.665 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:51.704 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:51.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:51.705 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:51.731 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:51 smithi083 ceph-mon[29421]: osdmap e190: 8 total, 8 up, 7 in 2024-09-20T18:55:51.732 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:51 smithi083 ceph-mon[36260]: osdmap e190: 8 total, 8 up, 7 in 2024-09-20T18:55:51.792 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T18:55:51.793 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-20T18:55:51.793 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.793 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.24955.0:100879 8.6 e190/182 8:6dab6812:::benchmark_data_smithi083_97422_object100878:head v 190'6274, mlcod=0'0)", 2024-09-20T18:55:51.793 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:55:51.742946+0000", 2024-09-20T18:55:51.793 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.034826026000000003, 2024-09-20T18:55:51.793 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.034851685, 2024-09-20T18:55:51.793 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:55:51.793 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:55:51.793 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:55:51.793 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:55:51.793 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.793 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:55:51.794 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.742946+0000", 2024-09-20T18:55:51.794 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:51.794 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.794 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.794 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:55:51.794 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.742946+0000", 2024-09-20T18:55:51.794 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:51.794 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.794 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.794 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:55:51.794 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.742947+0000", 2024-09-20T18:55:51.794 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0699999999999998e-07 2024-09-20T18:55:51.794 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.794 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.794 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:55:51.795 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.742969+0000", 2024-09-20T18:55:51.795 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2461999999999999e-05 2024-09-20T18:55:51.795 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.795 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.795 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:55:51.795 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.742977+0000", 2024-09-20T18:55:51.795 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8699999999999992e-06 2024-09-20T18:55:51.795 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.795 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.795 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:55:51.795 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.742981+0000", 2024-09-20T18:55:51.795 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4399999999999998e-06 2024-09-20T18:55:51.795 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.795 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.795 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:55:51.796 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.765542+0000", 2024-09-20T18:55:51.796 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.022560799999999999 2024-09-20T18:55:51.796 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.796 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.796 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:55:51.796 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.765565+0000", 2024-09-20T18:55:51.796 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3195000000000001e-05 2024-09-20T18:55:51.796 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:51.796 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:55:51.796 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:51.796 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.796 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.796 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.24955.0:100897 8.e e190/160 8:7cecd767:::benchmark_data_smithi083_97422_object100896:head v 190'6298, mlcod=190'6295)", 2024-09-20T18:55:51.796 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:55:51.763910+0000", 2024-09-20T18:55:51.797 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.013862423, 2024-09-20T18:55:51.797 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.013956758999999999, 2024-09-20T18:55:51.797 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:55:51.797 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:55:51.797 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:55:51.797 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:55:51.797 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.797 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:55:51.797 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.763910+0000", 2024-09-20T18:55:51.797 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:51.797 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.797 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.797 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:55:51.797 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.763910+0000", 2024-09-20T18:55:51.797 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:51.797 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.798 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.798 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:55:51.798 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.763911+0000", 2024-09-20T18:55:51.798 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.082e-06 2024-09-20T18:55:51.798 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.798 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.798 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:55:51.798 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.763937+0000", 2024-09-20T18:55:51.798 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6078999999999999e-05 2024-09-20T18:55:51.798 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.798 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.798 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:55:51.798 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.763948+0000", 2024-09-20T18:55:51.798 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1144999999999999e-05 2024-09-20T18:55:51.798 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.799 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.799 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:55:51.799 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.763956+0000", 2024-09-20T18:55:51.799 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4660000000000002e-06 2024-09-20T18:55:51.799 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.799 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.799 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:55:51.799 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.764004+0000", 2024-09-20T18:55:51.799 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8489999999999998e-05 2024-09-20T18:55:51.799 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.799 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.799 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:55:51.799 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.764023+0000", 2024-09-20T18:55:51.800 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8600000000000001e-05 2024-09-20T18:55:51.800 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:51.800 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:55:51.800 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:51.800 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.800 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.800 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.24955.0:100899 8.6 e190/182 8:6b9ff441:::benchmark_data_smithi083_97422_object100898:head v 190'6275, mlcod=0'0)", 2024-09-20T18:55:51.800 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:55:51.763974+0000", 2024-09-20T18:55:51.800 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.013798292, 2024-09-20T18:55:51.800 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.013943954, 2024-09-20T18:55:51.800 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:55:51.800 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:55:51.801 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:55:51.801 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:55:51.801 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.801 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:55:51.801 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.763974+0000", 2024-09-20T18:55:51.801 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:51.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.801 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.801 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:55:51.801 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.763974+0000", 2024-09-20T18:55:51.801 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:51.802 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.802 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.802 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:55:51.802 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.763974+0000", 2024-09-20T18:55:51.802 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9499999999999998e-07 2024-09-20T18:55:51.802 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.802 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.802 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:55:51.802 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.763995+0000", 2024-09-20T18:55:51.802 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1064e-05 2024-09-20T18:55:51.802 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.802 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.802 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:55:51.802 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.764002+0000", 2024-09-20T18:55:51.802 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6649999999999997e-06 2024-09-20T18:55:51.803 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.803 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.803 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:55:51.803 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.764008+0000", 2024-09-20T18:55:51.803 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6470000000000003e-06 2024-09-20T18:55:51.803 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.803 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.803 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:55:51.803 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.771820+0000", 2024-09-20T18:55:51.803 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0078122510000000001 2024-09-20T18:55:51.803 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.803 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.803 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:55:51.803 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.771920+0000", 2024-09-20T18:55:51.804 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9859999999999996e-05 2024-09-20T18:55:51.804 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:51.804 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:55:51.804 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:51.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.804 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.804 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "MOSDPGPush(8.6 190/182 [PushOp(8:618aef16:::benchmark_data_smithi083_97422_object5069:head, version: 165'319, data_included: [0~65536], data_size: 65536, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(8:618aef16:::benchmark_data_smithi083_97422_object5069:head@165'319, size: 65536, copy_subset: [(0, 65536)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 65536, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-20T18:55:51.804 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:55:51.764017+0000", 2024-09-20T18:55:51.804 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.013755705999999999, 2024-09-20T18:55:51.804 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.013957713, 2024-09-20T18:55:51.804 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:55:51.804 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:55:51.804 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "queued for pg", 2024-09-20T18:55:51.805 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:55:51.805 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.805 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:55:51.805 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.764017+0000", 2024-09-20T18:55:51.805 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:51.805 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.805 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.805 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:55:51.805 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.764017+0000", 2024-09-20T18:55:51.805 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:51.805 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.805 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.805 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:55:51.805 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.764017+0000", 2024-09-20T18:55:51.806 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3500000000000002e-07 2024-09-20T18:55:51.806 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.806 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.806 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:55:51.807 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:55:51.807 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.764047+0000", 2024-09-20T18:55:51.807 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0406999999999999e-05 2024-09-20T18:55:51.807 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.807 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.807 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:55:51.807 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.764067+0000", 2024-09-20T18:55:51.807 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9332000000000001e-05 2024-09-20T18:55:51.807 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.807 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.807 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:55:51.808 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.764083+0000", 2024-09-20T18:55:51.808 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6205000000000002e-05 2024-09-20T18:55:51.808 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:51.808 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:55:51.808 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:51.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.808 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.808 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.24955.0:100903 8.e e190/160 8:76ce4959:::benchmark_data_smithi083_97422_object100902:head v 190'6299, mlcod=190'6295)", 2024-09-20T18:55:51.808 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:55:51.765587+0000", 2024-09-20T18:55:51.808 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.01218517, 2024-09-20T18:55:51.808 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.012411333, 2024-09-20T18:55:51.808 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:55:51.808 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:55:51.809 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:55:51.809 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:55:51.809 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.809 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:55:51.809 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.765587+0000", 2024-09-20T18:55:51.809 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:51.809 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.809 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.809 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:55:51.809 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.765587+0000", 2024-09-20T18:55:51.809 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:51.809 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.809 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.809 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:55:51.810 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.765588+0000", 2024-09-20T18:55:51.810 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8299999999999995e-07 2024-09-20T18:55:51.810 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.810 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.810 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:55:51.810 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.765640+0000", 2024-09-20T18:55:51.810 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1734000000000003e-05 2024-09-20T18:55:51.810 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.810 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.810 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:55:51.810 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.765650+0000", 2024-09-20T18:55:51.810 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8230000000000006e-06 2024-09-20T18:55:51.811 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.811 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.811 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:55:51.811 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.765657+0000", 2024-09-20T18:55:51.811 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9260000000000003e-06 2024-09-20T18:55:51.811 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.811 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.811 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:55:51.811 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.771747+0000", 2024-09-20T18:55:51.811 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006090977 2024-09-20T18:55:51.811 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.811 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.811 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:55:51.812 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.771768+0000", 2024-09-20T18:55:51.812 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0112999999999999e-05 2024-09-20T18:55:51.812 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:51.812 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:55:51.812 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:51.812 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.812 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.812 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.24955.0:100920 8.7 e190/182 8:e9e4fb2d:::benchmark_data_smithi083_97422_object100919:head v 190'6350, mlcod=190'6348)", 2024-09-20T18:55:51.812 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:55:51.775269+0000", 2024-09-20T18:55:51.812 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0025031319999999999, 2024-09-20T18:55:51.812 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0027544969999999998, 2024-09-20T18:55:51.812 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:55:51.812 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:55:51.812 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:55:51.812 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:55:51.812 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.813 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:55:51.813 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.775269+0000", 2024-09-20T18:55:51.813 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:51.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.813 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.813 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:55:51.813 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.775269+0000", 2024-09-20T18:55:51.813 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:51.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.813 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.813 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:55:51.813 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.775270+0000", 2024-09-20T18:55:51.813 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2699999999999999e-06 2024-09-20T18:55:51.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.813 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.814 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:55:51.814 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.775277+0000", 2024-09-20T18:55:51.814 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9659999999999999e-06 2024-09-20T18:55:51.814 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.814 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.814 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:55:51.814 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.775294+0000", 2024-09-20T18:55:51.814 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6424999999999999e-05 2024-09-20T18:55:51.814 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.814 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.814 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:55:51.814 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.775302+0000", 2024-09-20T18:55:51.815 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.1939999999999998e-06 2024-09-20T18:55:51.815 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.815 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.815 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:55:51.815 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.775503+0000", 2024-09-20T18:55:51.815 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00020140100000000001 2024-09-20T18:55:51.815 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.815 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.815 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:55:51.815 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.775560+0000", 2024-09-20T18:55:51.815 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6753999999999999e-05 2024-09-20T18:55:51.815 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:51.815 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:55:51.815 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:51.816 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.816 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.816 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.24955.0:100847 8.6 e190/182 8:6ef0e76c:::benchmark_data_smithi083_97422_object100846:head v 190'6273, mlcod=0'0)", 2024-09-20T18:55:51.816 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:55:51.742815+0000", 2024-09-20T18:55:51.816 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.034957486000000003, 2024-09-20T18:55:51.816 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.035248433000000003, 2024-09-20T18:55:51.816 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:55:51.816 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:55:51.816 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:55:51.816 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:55:51.816 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.816 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:55:51.816 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.742815+0000", 2024-09-20T18:55:51.816 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:51.817 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.817 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.817 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:55:51.817 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.742815+0000", 2024-09-20T18:55:51.817 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:55:51.817 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.817 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.817 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:55:51.817 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.742816+0000", 2024-09-20T18:55:51.817 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.003e-06 2024-09-20T18:55:51.817 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.817 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.817 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:55:51.817 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.742844+0000", 2024-09-20T18:55:51.818 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8558e-05 2024-09-20T18:55:51.818 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.818 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.818 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:55:51.818 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.742866+0000", 2024-09-20T18:55:51.818 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1994e-05 2024-09-20T18:55:51.818 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.818 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.818 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:55:51.818 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.742874+0000", 2024-09-20T18:55:51.818 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5830000000000001e-06 2024-09-20T18:55:51.818 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.818 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.818 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:55:51.819 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.745889+0000", 2024-09-20T18:55:51.819 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0030148499999999999 2024-09-20T18:55:51.819 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:55:51.819 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:55:51.819 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:55:51.819 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:55:51.745914+0000", 2024-09-20T18:55:51.819 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4706e-05 2024-09-20T18:55:51.819 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:51.819 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:55:51.819 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:51.819 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:55:51.819 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T18:55:51.819 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 7 2024-09-20T18:55:51.819 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T18:55:51.863 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:51.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:51.864 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:55:51.965 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:52.006 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:52.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:52.007 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:52.021 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:51 smithi142 ceph-mon[32508]: osdmap e190: 8 total, 8 up, 7 in 2024-09-20T18:55:52.108 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:52.137 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:52.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:52.138 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:52.187 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 72 16 101279 101263 87.8601 53.5 0.00891342 0.0113763 2024-09-20T18:55:52.238 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:52.281 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:52.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:52.282 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:52.383 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:52.405 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:52.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:52.406 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:52.507 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:55:52.533 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:52.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:52.534 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:55:52.635 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:52.665 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:52.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:52.666 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:52.739 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_blocked_ops 2024-09-20T18:55:52.767 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:52.791 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:52.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:52.792 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:52.892 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:55:52.941 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:52.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:52.942 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:55:52.958 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:52 smithi083 ceph-mon[29421]: pgmap v287: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 143 active+clean; 5.9 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.2 KiB/s rd, 34 MiB/s wr, 549 op/s; 24254/290994 objects misplaced (8.335%); 8.6 MiB/s, 137 objects/s recovering 2024-09-20T18:55:52.959 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:52 smithi083 ceph-mon[36260]: pgmap v287: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 143 active+clean; 5.9 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.2 KiB/s rd, 34 MiB/s wr, 549 op/s; 24254/290994 objects misplaced (8.335%); 8.6 MiB/s, 137 objects/s recovering 2024-09-20T18:55:53.021 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:52 smithi142 ceph-mon[32508]: pgmap v287: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 143 active+clean; 5.9 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.2 KiB/s rd, 34 MiB/s wr, 549 op/s; 24254/290994 objects misplaced (8.335%); 8.6 MiB/s, 137 objects/s recovering 2024-09-20T18:55:53.042 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:53.072 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:53.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:53.073 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:53.173 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:53.189 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 73 16 102085 102069 87.3447 50.375 0.00544981 0.0114434 2024-09-20T18:55:53.212 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:53.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:53.213 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:53.265 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd reweight 2 1 2024-09-20T18:55:53.289 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:55:53.313 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:53.340 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:53.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:53.341 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:53.360 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:55:53.443 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:53.473 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:53.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:53.474 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:53.512 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T18:55:53.575 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:53.600 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:53.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:53.601 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:53.702 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:55:53.750 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:53.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:53.751 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:55:53.852 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:55:53.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:53.912 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:53.912 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:55:53.917 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:53 smithi083 ceph-mon[29421]: pgmap v288: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 143 active+clean; 6.1 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.2 KiB/s rd, 62 MiB/s wr, 994 op/s; 23789/301671 objects misplaced (7.886%); 15 MiB/s, 244 objects/s recovering 2024-09-20T18:55:53.917 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:53 smithi083 ceph-mon[36260]: pgmap v288: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 143 active+clean; 6.1 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.2 KiB/s rd, 62 MiB/s wr, 994 op/s; 23789/301671 objects misplaced (7.886%); 15 MiB/s, 244 objects/s recovering 2024-09-20T18:55:54.013 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:54.021 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:53 smithi142 ceph-mon[32508]: pgmap v288: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 143 active+clean; 6.1 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.2 KiB/s rd, 62 MiB/s wr, 994 op/s; 23789/301671 objects misplaced (7.886%); 15 MiB/s, 244 objects/s recovering 2024-09-20T18:55:54.045 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:54.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:54.046 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:54.147 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:55:54.186 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:54.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:54.187 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:55:54.189 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 74 16 102793 102777 86.7625 44.25 0.0259685 0.0115211 2024-09-20T18:55:54.288 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:54.313 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:54.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:54.314 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:54.415 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:55:54.455 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:54.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:54.456 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:55:54.557 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:55:54.600 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:54.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:54.601 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:55:54.702 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:55:54.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:54.749 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:54.749 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:55:54.850 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:54.904 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:54.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:54.905 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:55.006 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:55.030 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:55.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:55.031 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:55.131 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:55:55.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:55.159 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:55.159 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:55:55.189 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 75 16 103680 103664 86.3449 55.4375 0.0656165 0.0115736 2024-09-20T18:55:55.260 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:55.288 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:55.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:55.289 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:55.390 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:55:55.435 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:55.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:55.436 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:55:55.536 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:55.564 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:55.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:55.565 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:55.666 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:55:55.700 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:55.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:55.701 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:55:55.802 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:55.831 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:55.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:55.832 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:55.879 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:55:55.879 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:55:55.880 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:55:55.880 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:55:55.880 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:55:55.880 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:55:55.880 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:55:55.880 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:55:55.880 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:55:55.880 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:55:55.880 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:55:55.880 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:55:55.881 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:55:55.881 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:55:55.881 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:55:55.881 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:55:55.932 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:56.005 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:56.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:56.007 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:56.107 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:55:56.138 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:56.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:56.138 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:55:56.189 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 76 15 104576 104561 85.9466 56.0625 0.0114053 0.0116295 2024-09-20T18:55:56.239 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:56.265 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:56.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:56.266 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:56.271 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:55 smithi142 ceph-mon[32508]: pgmap v289: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 143 active+clean; 6.1 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 610 B/s rd, 30 MiB/s wr, 482 op/s; 23789/301671 objects misplaced (7.886%); 7.7 MiB/s, 122 objects/s recovering 2024-09-20T18:55:56.271 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:55 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3592125455' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:55:56.301 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:55 smithi083 ceph-mon[29421]: pgmap v289: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 143 active+clean; 6.1 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 610 B/s rd, 30 MiB/s wr, 482 op/s; 23789/301671 objects misplaced (7.886%); 7.7 MiB/s, 122 objects/s recovering 2024-09-20T18:55:56.301 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:55 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3592125455' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:55:56.302 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:55 smithi083 ceph-mon[36260]: pgmap v289: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 143 active+clean; 6.1 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 610 B/s rd, 30 MiB/s wr, 482 op/s; 23789/301671 objects misplaced (7.886%); 7.7 MiB/s, 122 objects/s recovering 2024-09-20T18:55:56.302 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:55 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3592125455' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:55:56.367 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:56.398 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:56.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:56.399 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:56.500 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:56.527 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:56.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:56.527 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:56.628 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:55:56.669 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:56.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:56.670 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:55:56.771 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:56.796 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:56.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:56.797 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:56.898 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:56.922 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:56.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:56.924 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:56.968 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:55:56 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:55:57.024 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:57.050 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:57.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:57.051 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:57.152 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:55:57.189 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 77 16 105502 105486 85.5813 57.8125 0.0173434 0.0116795 2024-09-20T18:55:57.193 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:57.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:57.194 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:55:57.223 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:56 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:55:57.223 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:56 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:55:57.271 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:56 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:55:57.295 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:57.320 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:57.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:57.321 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:57.395 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T18:55:57.395 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:55:57.420 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:55:57.422 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:55:57.464 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:57.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:57.465 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:55:57.566 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:57.594 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:57.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:57.595 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:57.696 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:57.727 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:57.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:57.728 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:57.828 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:57.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:57.860 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:57.860 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:57.961 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:57.994 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:57.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:57.995 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:58.051 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:57 smithi083 ceph-mon[29421]: pgmap v290: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 143 active+clean; 6.4 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1023 B/s rd, 52 MiB/s wr, 826 op/s; 23142/314340 objects misplaced (7.362%); 13 MiB/s, 206 objects/s recovering 2024-09-20T18:55:58.051 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:57 smithi083 ceph-mon[36260]: pgmap v290: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 143 active+clean; 6.4 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1023 B/s rd, 52 MiB/s wr, 826 op/s; 23142/314340 objects misplaced (7.362%); 13 MiB/s, 206 objects/s recovering 2024-09-20T18:55:58.097 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:55:58.141 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:58.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:58.142 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:55:58.189 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 78 16 106416 106400 85.2165 57.125 0.00591974 0.0117291 2024-09-20T18:55:58.244 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:58.262 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_historic_ops 2024-09-20T18:55:58.271 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:57 smithi142 ceph-mon[32508]: pgmap v290: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 143 active+clean; 6.4 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1023 B/s rd, 52 MiB/s wr, 826 op/s; 23142/314340 objects misplaced (7.362%); 13 MiB/s, 206 objects/s recovering 2024-09-20T18:55:58.287 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:58.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:58.288 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:58.389 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:58.447 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:58.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:58.448 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:58.457 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:55:58.549 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:58.573 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:58.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:58.574 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:58.675 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:58.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:58.719 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:58.719 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:58.820 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:58.848 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:58.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:58.849 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:58.950 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:55:58.978 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:58.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:58.979 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:55:59.080 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:55:59.085 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T18:55:59.109 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:59.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:59.110 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:55:59.189 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 79 16 107238 107222 84.7883 51.375 0.0176456 0.0117886 2024-09-20T18:55:59.211 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:59.238 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:59.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:59.239 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:59.340 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:55:59.365 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:59.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:59.366 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:55:59.467 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:59.513 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:59.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:59.514 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:59.615 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:59.639 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:59.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:59.640 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:55:59.741 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:55:59.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:59.782 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:55:59.782 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:55:59.883 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:55:59.915 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:55:59.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:55:59.916 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:00.020 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:00.048 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:00.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:00.049 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:00.150 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:00.177 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:00.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:00.178 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:00.189 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:2024-09-20T18:56:00.176207+0000 min lat: 0.00111821 max lat: 2.23065 avg lat: 0.0118512 2024-09-20T18:56:00.189 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-20T18:56:00.189 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 80 16 108024 108008 84.3426 49.125 0.00947349 0.0118512 2024-09-20T18:56:00.271 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:59 smithi142 ceph-mon[32508]: pgmap v291: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 143 active+clean; 6.4 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1023 B/s rd, 52 MiB/s wr, 826 op/s; 23142/314340 objects misplaced (7.362%); 13 MiB/s, 206 objects/s recovering 2024-09-20T18:56:00.271 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:59 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:56:00.271 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:59 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:56:00.271 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:59 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2131308976' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-20T18:56:00.271 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:55:59 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-20T18:56:00.279 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:00.301 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:59 smithi083 ceph-mon[29421]: pgmap v291: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 143 active+clean; 6.4 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1023 B/s rd, 52 MiB/s wr, 826 op/s; 23142/314340 objects misplaced (7.362%); 13 MiB/s, 206 objects/s recovering 2024-09-20T18:56:00.301 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:59 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:56:00.301 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:59 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:56:00.301 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:59 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2131308976' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-20T18:56:00.301 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:55:59 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-20T18:56:00.302 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:55:59 smithi083 ceph-mon[36260]: pgmap v291: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 143 active+clean; 6.4 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1023 B/s rd, 52 MiB/s wr, 826 op/s; 23142/314340 objects misplaced (7.362%); 13 MiB/s, 206 objects/s recovering 2024-09-20T18:56:00.302 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:00 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:56:00.302 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:00 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:56:00.302 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:00 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2131308976' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-20T18:56:00.302 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:00 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-20T18:56:00.305 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:00.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:00.306 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:00.405 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.2 to 1 (10000) 2024-09-20T18:56:00.407 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:56:00.442 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:00.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:00.443 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:56:00.543 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:00.579 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:00.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:00.580 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:00.681 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:00.708 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:00.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:00.710 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:00.810 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:00.839 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:00.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:00.840 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:00.940 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:00.966 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:00.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:00.967 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:01.029 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:56:00 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:56:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:56:01.068 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:56:01.105 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:01.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:01.107 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:56:01.189 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 81 16 108910 108894 83.985 55.375 0.00429793 0.0119016 2024-09-20T18:56:01.207 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:01.247 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:01.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:01.248 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:01.301 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:01 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-20T18:56:01.301 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:01 smithi083 ceph-mon[29421]: osdmap e191: 8 total, 8 up, 7 in 2024-09-20T18:56:01.301 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:01 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:56:01.301 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:01 smithi083 ceph-mon[29421]: osdmap e192: 8 total, 8 up, 7 in 2024-09-20T18:56:01.301 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:01 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-20T18:56:01.301 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:01 smithi083 ceph-mon[36260]: osdmap e191: 8 total, 8 up, 7 in 2024-09-20T18:56:01.301 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:01 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:56:01.301 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:01 smithi083 ceph-mon[36260]: osdmap e192: 8 total, 8 up, 7 in 2024-09-20T18:56:01.349 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:01.377 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:01.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:01.378 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:01.479 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:56:01.490 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd reweight 3 1 2024-09-20T18:56:01.517 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:56:01.521 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:01 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-20T18:56:01.521 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:01 smithi142 ceph-mon[32508]: osdmap e191: 8 total, 8 up, 7 in 2024-09-20T18:56:01.521 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:01 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:56:01.521 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:01 smithi142 ceph-mon[32508]: osdmap e192: 8 total, 8 up, 7 in 2024-09-20T18:56:01.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:01.563 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:01.563 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:56:01.664 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:56:01.719 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:01.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:01.720 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:56:01.821 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:01.843 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:01.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:01.844 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:01.945 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:56:01.987 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:01.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:01.988 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:56:02.089 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:02.115 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:02.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:02.116 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:02.132 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:56:02.190 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 82 16 109590 109574 83.4788 42.5 0.0240247 0.0119733 2024-09-20T18:56:02.201 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:56:02.201 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:56:02.217 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:02.242 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:02.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:02.243 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:02.243 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:56:02.244 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:56:02.275 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:56:02.275 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:56:02.319 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:56:02.319 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:56:02.343 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:02.349 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:56:02.350 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:56:02.369 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:02.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:02.370 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:02.385 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:56:02.385 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:56:02.416 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:56:02.417 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:56:02.456 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:56:02.456 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:56:02.471 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:56:02.512 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:02.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:02.513 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:56:02.614 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:56:02.655 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:02.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:02.656 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:56:02.718 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-20T18:56:02.757 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:02.766 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T18:56:02.767 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-20T18:56:02.767 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-20T18:56:02.767 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-20T18:56:02.767 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.767 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:27549 7.e 7:77436293:::benchmark_data_smithi083_70643_object27548:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:56:02.767 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:08.731657+0000", 2024-09-20T18:56:02.767 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 294.01916376299999, 2024-09-20T18:56:02.767 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.113586293, 2024-09-20T18:56:02.767 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:56:02.767 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:56:02.767 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:56:02.767 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:56:02.767 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:56:02.767 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:56:02.768 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 27549 2024-09-20T18:56:02.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.768 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:56:02.768 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.768 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:56:02.768 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.731657+0000", 2024-09-20T18:56:02.768 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:02.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.768 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.768 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:56:02.768 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.731657+0000", 2024-09-20T18:56:02.768 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:02.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.769 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.769 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:56:02.769 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.731658+0000", 2024-09-20T18:56:02.769 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1400000000000001e-06 2024-09-20T18:56:02.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.769 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.769 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:56:02.769 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.731686+0000", 2024-09-20T18:56:02.769 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7840000000000001e-05 2024-09-20T18:56:02.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.769 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.769 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:56:02.769 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.731697+0000", 2024-09-20T18:56:02.770 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0774999999999999e-05 2024-09-20T18:56:02.770 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.770 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.770 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:56:02.770 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.731704+0000", 2024-09-20T18:56:02.770 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1740000000000003e-06 2024-09-20T18:56:02.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.771 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:56:02.771 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.731757+0000", 2024-09-20T18:56:02.771 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3214000000000002e-05 2024-09-20T18:56:02.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.771 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:56:02.771 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.731876+0000", 2024-09-20T18:56:02.771 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000118829 2024-09-20T18:56:02.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.771 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:56:02.772 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.732005+0000", 2024-09-20T18:56:02.772 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000128937 2024-09-20T18:56:02.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.772 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.772 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:56:02.772 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733315+0000", 2024-09-20T18:56:02.772 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001309943 2024-09-20T18:56:02.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.772 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.772 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:02.772 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843149+0000", 2024-09-20T18:56:02.772 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.109834472 2024-09-20T18:56:02.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.772 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.772 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:02.773 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843225+0000", 2024-09-20T18:56:02.773 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5304999999999998e-05 2024-09-20T18:56:02.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.773 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.773 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:56:02.773 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.844447+0000", 2024-09-20T18:56:02.773 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0012227239999999999 2024-09-20T18:56:02.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.773 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.773 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:56:02.773 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.845243+0000", 2024-09-20T18:56:02.773 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00079593999999999997 2024-09-20T18:56:02.774 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:02.774 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:56:02.774 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:02.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.774 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:27551 7.8 7:1bc4112c:::benchmark_data_smithi083_70643_object27550:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:56:02.774 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:08.733037+0000", 2024-09-20T18:56:02.774 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 294.017783974, 2024-09-20T18:56:02.774 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.110337717, 2024-09-20T18:56:02.774 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:56:02.774 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:56:02.774 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:56:02.775 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:56:02.775 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:56:02.775 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:56:02.775 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 27551 2024-09-20T18:56:02.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.775 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:56:02.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.775 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:56:02.775 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733037+0000", 2024-09-20T18:56:02.775 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:02.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.775 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:56:02.776 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733037+0000", 2024-09-20T18:56:02.776 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:02.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.776 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.776 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:56:02.776 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733038+0000", 2024-09-20T18:56:02.776 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.497e-06 2024-09-20T18:56:02.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.776 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.776 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:56:02.776 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733062+0000", 2024-09-20T18:56:02.776 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3498000000000001e-05 2024-09-20T18:56:02.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.776 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.776 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:56:02.777 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733072+0000", 2024-09-20T18:56:02.777 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0173000000000001e-05 2024-09-20T18:56:02.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.777 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.777 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:56:02.777 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733079+0000", 2024-09-20T18:56:02.777 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5370000000000003e-06 2024-09-20T18:56:02.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.777 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.777 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:56:02.777 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733187+0000", 2024-09-20T18:56:02.777 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010888 2024-09-20T18:56:02.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.777 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.777 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:56:02.778 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733330+0000", 2024-09-20T18:56:02.778 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000142206 2024-09-20T18:56:02.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.778 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.778 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4,7", 2024-09-20T18:56:02.778 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733549+0000", 2024-09-20T18:56:02.778 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0002198 2024-09-20T18:56:02.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.778 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.778 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:56:02.778 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.735541+0000", 2024-09-20T18:56:02.778 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0019915129999999999 2024-09-20T18:56:02.779 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.779 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.779 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:02.779 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.842311+0000", 2024-09-20T18:56:02.779 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.106769612 2024-09-20T18:56:02.779 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.779 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.779 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:02.779 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.842372+0000", 2024-09-20T18:56:02.779 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0968000000000003e-05 2024-09-20T18:56:02.779 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.779 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.779 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:56:02.779 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843315+0000", 2024-09-20T18:56:02.779 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00094376300000000002 2024-09-20T18:56:02.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.780 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.780 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:56:02.780 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843375+0000", 2024-09-20T18:56:02.780 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.927e-05 2024-09-20T18:56:02.780 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:02.780 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:56:02.780 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:02.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.780 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.780 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:27555 7.e 7:73709da2:::benchmark_data_smithi083_70643_object27554:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:56:02.780 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:08.733695+0000", 2024-09-20T18:56:02.780 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 294.017125851, 2024-09-20T18:56:02.780 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.11168107300000001, 2024-09-20T18:56:02.781 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:56:02.781 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:56:02.781 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:56:02.781 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:56:02.781 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:56:02.781 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:56:02.782 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 27555 2024-09-20T18:56:02.782 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.782 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:56:02.782 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.782 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:56:02.782 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733695+0000", 2024-09-20T18:56:02.782 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:02.782 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.782 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.782 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:56:02.782 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733695+0000", 2024-09-20T18:56:02.782 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:02.783 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.783 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.783 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:56:02.783 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733696+0000", 2024-09-20T18:56:02.783 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0200000000000001e-07 2024-09-20T18:56:02.783 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.783 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.783 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:56:02.783 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733718+0000", 2024-09-20T18:56:02.783 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2226999999999999e-05 2024-09-20T18:56:02.783 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.783 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.783 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:56:02.783 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733727+0000", 2024-09-20T18:56:02.784 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6049999999999998e-06 2024-09-20T18:56:02.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.784 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.784 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:56:02.784 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733734+0000", 2024-09-20T18:56:02.784 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0639999999999996e-06 2024-09-20T18:56:02.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.784 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.784 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:56:02.784 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733789+0000", 2024-09-20T18:56:02.784 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5087999999999998e-05 2024-09-20T18:56:02.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.784 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.785 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:56:02.785 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733920+0000", 2024-09-20T18:56:02.785 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013146800000000001 2024-09-20T18:56:02.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.785 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:56:02.785 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733995+0000", 2024-09-20T18:56:02.785 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4664000000000002e-05 2024-09-20T18:56:02.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.785 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:56:02.785 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.735538+0000", 2024-09-20T18:56:02.786 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001542671 2024-09-20T18:56:02.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.786 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.786 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:02.786 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843187+0000", 2024-09-20T18:56:02.786 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.107649921 2024-09-20T18:56:02.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.786 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.786 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:02.786 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.845296+0000", 2024-09-20T18:56:02.786 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0021081920000000001 2024-09-20T18:56:02.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.787 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.787 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:56:02.787 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.845341+0000", 2024-09-20T18:56:02.787 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5141e-05 2024-09-20T18:56:02.787 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.787 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.787 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:56:02.787 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.845376+0000", 2024-09-20T18:56:02.787 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.523e-05 2024-09-20T18:56:02.787 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:02.787 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:56:02.787 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:02.787 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.788 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.788 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:27560 7.e 7:7d7d0b94:::benchmark_data_smithi083_70643_object27559:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:56:02.788 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:08.736128+0000", 2024-09-20T18:56:02.788 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 294.01469325199997, 2024-09-20T18:56:02.788 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.112670376, 2024-09-20T18:56:02.788 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:56:02.788 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:56:02.788 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:56:02.788 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:56:02.788 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:56:02.788 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:56:02.788 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 27560 2024-09-20T18:56:02.788 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.788 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:56:02.789 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.789 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:56:02.789 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736128+0000", 2024-09-20T18:56:02.789 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:02.789 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.789 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.789 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:56:02.789 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736128+0000", 2024-09-20T18:56:02.789 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:02.789 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.789 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.789 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:56:02.789 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736129+0000", 2024-09-20T18:56:02.789 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.251e-06 2024-09-20T18:56:02.789 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.790 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.790 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:56:02.790 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736150+0000", 2024-09-20T18:56:02.790 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1450999999999998e-05 2024-09-20T18:56:02.790 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.790 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.790 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:56:02.790 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736161+0000", 2024-09-20T18:56:02.790 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0682e-05 2024-09-20T18:56:02.790 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.790 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.790 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:56:02.790 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736168+0000", 2024-09-20T18:56:02.791 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8549999999999996e-06 2024-09-20T18:56:02.791 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.791 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.791 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:56:02.791 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736218+0000", 2024-09-20T18:56:02.791 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0497000000000001e-05 2024-09-20T18:56:02.791 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.791 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.791 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:56:02.791 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736327+0000", 2024-09-20T18:56:02.791 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010837599999999999 2024-09-20T18:56:02.791 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.791 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.791 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:56:02.791 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.841982+0000", 2024-09-20T18:56:02.792 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.105654846 2024-09-20T18:56:02.792 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.792 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.792 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:56:02.792 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.845386+0000", 2024-09-20T18:56:02.792 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0034042719999999999 2024-09-20T18:56:02.792 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.792 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.792 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:02.792 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.845423+0000", 2024-09-20T18:56:02.792 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7160000000000003e-05 2024-09-20T18:56:02.792 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.792 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.792 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:02.792 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.848702+0000", 2024-09-20T18:56:02.793 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0032792099999999999 2024-09-20T18:56:02.793 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.793 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.793 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:56:02.793 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.848758+0000", 2024-09-20T18:56:02.793 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.537e-05 2024-09-20T18:56:02.793 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.793 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.793 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:56:02.793 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.848798+0000", 2024-09-20T18:56:02.793 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0405999999999999e-05 2024-09-20T18:56:02.793 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:02.793 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:56:02.793 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:02.793 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.794 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.794 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:27559 7.a e93/83 7:5f89880a:::benchmark_data_smithi083_70643_object27558:head v 93'1704, mlcod=93'1702)", 2024-09-20T18:56:02.794 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:08.736205+0000", 2024-09-20T18:56:02.794 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 294.01461572300002, 2024-09-20T18:56:02.794 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.108390217, 2024-09-20T18:56:02.794 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:56:02.794 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:56:02.794 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:56:02.794 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:56:02.794 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.794 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:56:02.794 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736205+0000", 2024-09-20T18:56:02.794 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:02.795 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.795 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.795 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:56:02.795 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736205+0000", 2024-09-20T18:56:02.795 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:02.795 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.795 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.795 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:56:02.795 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736206+0000", 2024-09-20T18:56:02.795 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4000000000000001e-07 2024-09-20T18:56:02.795 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.795 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.795 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:56:02.795 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736226+0000", 2024-09-20T18:56:02.795 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0426999999999999e-05 2024-09-20T18:56:02.796 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.796 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.796 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:56:02.796 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736236+0000", 2024-09-20T18:56:02.796 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.781e-06 2024-09-20T18:56:02.796 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.796 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.796 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:56:02.796 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736242+0000", 2024-09-20T18:56:02.796 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0179999999999998e-06 2024-09-20T18:56:02.796 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.796 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.796 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:56:02.796 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736307+0000", 2024-09-20T18:56:02.797 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5042000000000005e-05 2024-09-20T18:56:02.797 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.797 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.797 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:56:02.797 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736325+0000", 2024-09-20T18:56:02.797 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7895999999999999e-05 2024-09-20T18:56:02.797 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.797 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.797 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:56:02.797 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.844517+0000", 2024-09-20T18:56:02.797 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.10819235100000001 2024-09-20T18:56:02.797 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.797 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.797 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:56:02.797 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.844595+0000", 2024-09-20T18:56:02.797 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7961999999999997e-05 2024-09-20T18:56:02.798 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:02.798 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:56:02.798 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:02.798 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.798 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.798 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:27549 7.e e93/83 ondisk, result = 0)", 2024-09-20T18:56:02.798 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:08.736243+0000", 2024-09-20T18:56:02.798 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 294.01457812899997, 2024-09-20T18:56:02.798 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.106913722, 2024-09-20T18:56:02.798 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:56:02.798 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:56:02.798 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:56:02.798 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:56:02.798 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.799 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:56:02.799 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736243+0000", 2024-09-20T18:56:02.799 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:02.799 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.799 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.799 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:56:02.799 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736243+0000", 2024-09-20T18:56:02.799 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:02.799 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.799 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.799 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:56:02.799 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736243+0000", 2024-09-20T18:56:02.799 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3200000000000004e-07 2024-09-20T18:56:02.799 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.800 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.800 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:56:02.800 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736248+0000", 2024-09-20T18:56:02.800 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2780000000000004e-06 2024-09-20T18:56:02.800 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.800 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.800 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:56:02.800 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736257+0000", 2024-09-20T18:56:02.800 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2790000000000004e-06 2024-09-20T18:56:02.800 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.800 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.800 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:56:02.800 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736263+0000", 2024-09-20T18:56:02.800 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3570000000000003e-06 2024-09-20T18:56:02.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.801 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.801 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:56:02.801 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843136+0000", 2024-09-20T18:56:02.801 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.10687292499999999 2024-09-20T18:56:02.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.801 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.801 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:56:02.801 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843143+0000", 2024-09-20T18:56:02.801 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2300000000000002e-06 2024-09-20T18:56:02.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.801 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.801 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:56:02.801 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843156+0000", 2024-09-20T18:56:02.802 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2921e-05 2024-09-20T18:56:02.802 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:02.802 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:56:02.802 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:02.802 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.802 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.802 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:27555 7.e e93/83 ondisk, result = 0)", 2024-09-20T18:56:02.802 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:08.736486+0000", 2024-09-20T18:56:02.802 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 294.01433504099998, 2024-09-20T18:56:02.802 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.106708079, 2024-09-20T18:56:02.802 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:56:02.802 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:56:02.802 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:56:02.803 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:56:02.803 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.803 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:56:02.803 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736486+0000", 2024-09-20T18:56:02.803 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:02.803 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.803 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.803 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:56:02.803 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736486+0000", 2024-09-20T18:56:02.803 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:02.803 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.803 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.803 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:56:02.803 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736486+0000", 2024-09-20T18:56:02.803 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8100000000000002e-07 2024-09-20T18:56:02.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.804 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.804 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:56:02.804 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736490+0000", 2024-09-20T18:56:02.804 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.546e-06 2024-09-20T18:56:02.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.804 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.804 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:56:02.804 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.841491+0000", 2024-09-20T18:56:02.804 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.105001423 2024-09-20T18:56:02.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.804 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.804 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:56:02.804 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.841503+0000", 2024-09-20T18:56:02.804 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1399e-05 2024-09-20T18:56:02.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.805 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.805 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:56:02.805 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843177+0000", 2024-09-20T18:56:02.805 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0016741900000000001 2024-09-20T18:56:02.805 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.805 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.805 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:56:02.805 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843183+0000", 2024-09-20T18:56:02.805 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7209999999999999e-06 2024-09-20T18:56:02.805 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.805 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.805 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:56:02.805 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843194+0000", 2024-09-20T18:56:02.805 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1119e-05 2024-09-20T18:56:02.805 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:02.805 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:56:02.806 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:02.806 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.806 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.806 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:27551 7.8 e93/83 ondisk, result = 0)", 2024-09-20T18:56:02.806 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:08.736884+0000", 2024-09-20T18:56:02.806 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 294.01393724299999, 2024-09-20T18:56:02.806 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.105442044, 2024-09-20T18:56:02.806 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:56:02.806 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:56:02.806 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:56:02.806 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:56:02.806 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.806 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:56:02.806 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736884+0000", 2024-09-20T18:56:02.807 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:02.807 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.807 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.807 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:56:02.807 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736884+0000", 2024-09-20T18:56:02.807 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:02.807 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.807 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.807 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:56:02.807 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736884+0000", 2024-09-20T18:56:02.807 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6499999999999999e-07 2024-09-20T18:56:02.807 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.807 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.807 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:56:02.808 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736888+0000", 2024-09-20T18:56:02.808 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7170000000000002e-06 2024-09-20T18:56:02.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.808 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.808 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:56:02.808 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.841517+0000", 2024-09-20T18:56:02.808 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.104628836 2024-09-20T18:56:02.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.808 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.808 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:56:02.808 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.841525+0000", 2024-09-20T18:56:02.808 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7110000000000001e-06 2024-09-20T18:56:02.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.808 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.808 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:56:02.809 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.842286+0000", 2024-09-20T18:56:02.809 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00076098500000000005 2024-09-20T18:56:02.809 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.809 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.809 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:56:02.809 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.842304+0000", 2024-09-20T18:56:02.809 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7898e-05 2024-09-20T18:56:02.809 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.809 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.809 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:56:02.809 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.842326+0000", 2024-09-20T18:56:02.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:02.811 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1331999999999998e-05 2024-09-20T18:56:02.811 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:02.811 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:56:02.811 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:02.811 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.811 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.811 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:27571 7.8 7:1e0af83a:::benchmark_data_smithi083_70643_object27570:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:56:02.811 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:08.741453+0000", 2024-09-20T18:56:02.811 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 294.00936803899998, 2024-09-20T18:56:02.811 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.103905603, 2024-09-20T18:56:02.811 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:56:02.812 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:56:02.812 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:56:02.812 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:56:02.812 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:56:02.812 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:56:02.812 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 27571 2024-09-20T18:56:02.812 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.812 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:56:02.812 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.812 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:56:02.813 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.741453+0000", 2024-09-20T18:56:02.813 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:02.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.813 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.813 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:56:02.813 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.741453+0000", 2024-09-20T18:56:02.813 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:02.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.814 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.814 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:56:02.814 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.741455+0000", 2024-09-20T18:56:02.814 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4669999999999998e-06 2024-09-20T18:56:02.814 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.814 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.814 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:56:02.814 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.741483+0000", 2024-09-20T18:56:02.814 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7472999999999999e-05 2024-09-20T18:56:02.815 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.815 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.815 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:56:02.815 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.841438+0000", 2024-09-20T18:56:02.815 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.099955024000000003 2024-09-20T18:56:02.815 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.815 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.815 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:56:02.815 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.841457+0000", 2024-09-20T18:56:02.816 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9264999999999999e-05 2024-09-20T18:56:02.816 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.816 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.816 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:56:02.816 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843412+0000", 2024-09-20T18:56:02.816 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0019553869999999998 2024-09-20T18:56:02.816 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.816 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.817 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:56:02.817 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843559+0000", 2024-09-20T18:56:02.817 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014645399999999999 2024-09-20T18:56:02.817 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.817 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.817 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4,7", 2024-09-20T18:56:02.817 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843706+0000", 2024-09-20T18:56:02.817 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014695600000000001 2024-09-20T18:56:02.817 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.818 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.818 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:56:02.818 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.844855+0000", 2024-09-20T18:56:02.818 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001149424 2024-09-20T18:56:02.818 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.818 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.818 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:02.818 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.845215+0000", 2024-09-20T18:56:02.819 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00035930100000000002 2024-09-20T18:56:02.819 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.819 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.819 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:02.819 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.845270+0000", 2024-09-20T18:56:02.819 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5461999999999999e-05 2024-09-20T18:56:02.819 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.819 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.819 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:56:02.820 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.845321+0000", 2024-09-20T18:56:02.820 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1360000000000003e-05 2024-09-20T18:56:02.820 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.820 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.820 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:56:02.820 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.845358+0000", 2024-09-20T18:56:02.820 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7030000000000003e-05 2024-09-20T18:56:02.820 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:02.820 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:56:02.821 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:02.821 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.821 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.821 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:117083 7.e 7:7035ec2c:::benchmark_data_smithi083_70643_object117082:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-20T18:56:02.821 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:52:21.789905+0000", 2024-09-20T18:56:02.821 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 220.96091595300001, 2024-09-20T18:56:02.821 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.1098601, 2024-09-20T18:56:02.821 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:56:02.821 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:56:02.822 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:56:02.822 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:56:02.822 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:56:02.822 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:56:02.822 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 117083 2024-09-20T18:56:02.822 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.822 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:56:02.822 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.823 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:56:02.823 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.789905+0000", 2024-09-20T18:56:02.823 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:02.823 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.823 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.823 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:56:02.823 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.789905+0000", 2024-09-20T18:56:02.823 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:02.824 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.824 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.824 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:56:02.824 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.789906+0000", 2024-09-20T18:56:02.824 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4109999999999999e-06 2024-09-20T18:56:02.824 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.824 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.824 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:56:02.824 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.789929+0000", 2024-09-20T18:56:02.824 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3212000000000001e-05 2024-09-20T18:56:02.825 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.825 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.825 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:56:02.825 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.789941+0000", 2024-09-20T18:56:02.825 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1103e-05 2024-09-20T18:56:02.825 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.825 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.825 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:56:02.825 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.789948+0000", 2024-09-20T18:56:02.825 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.968e-06 2024-09-20T18:56:02.825 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.825 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.826 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:56:02.826 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.790013+0000", 2024-09-20T18:56:02.826 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5518000000000005e-05 2024-09-20T18:56:02.826 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.826 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.826 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:56:02.826 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.790130+0000", 2024-09-20T18:56:02.826 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000116906 2024-09-20T18:56:02.826 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.826 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.826 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:56:02.826 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.790270+0000", 2024-09-20T18:56:02.826 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013975700000000001 2024-09-20T18:56:02.826 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.826 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.827 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:56:02.827 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.793553+0000", 2024-09-20T18:56:02.827 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003283589 2024-09-20T18:56:02.827 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.827 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.827 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:02.827 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.799223+0000", 2024-09-20T18:56:02.827 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0056701340000000003 2024-09-20T18:56:02.827 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.827 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.827 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:02.827 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.899580+0000", 2024-09-20T18:56:02.827 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.100356374 2024-09-20T18:56:02.827 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.828 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.828 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:56:02.828 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.899639+0000", 2024-09-20T18:56:02.828 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8990000000000003e-05 2024-09-20T18:56:02.828 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.828 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.828 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:56:02.828 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.899765+0000", 2024-09-20T18:56:02.828 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000126138 2024-09-20T18:56:02.828 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:02.828 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:56:02.828 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:02.828 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.828 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.829 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:117091 7.e 7:791f482a:::benchmark_data_smithi083_70643_object117090:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-20T18:56:02.829 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:52:21.794291+0000", 2024-09-20T18:56:02.829 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 220.95653024500001, 2024-09-20T18:56:02.829 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.108790281, 2024-09-20T18:56:02.829 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:56:02.829 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:56:02.829 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:56:02.829 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:56:02.829 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:56:02.829 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:56:02.829 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 117091 2024-09-20T18:56:02.829 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.829 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:56:02.830 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.830 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:56:02.830 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.794291+0000", 2024-09-20T18:56:02.830 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:02.830 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.830 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.830 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:56:02.830 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.794291+0000", 2024-09-20T18:56:02.830 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:02.830 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.830 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.830 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:56:02.830 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.794292+0000", 2024-09-20T18:56:02.830 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.579e-06 2024-09-20T18:56:02.831 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.831 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.831 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:56:02.831 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.794317+0000", 2024-09-20T18:56:02.831 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4895000000000001e-05 2024-09-20T18:56:02.831 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.831 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.831 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:56:02.831 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.794329+0000", 2024-09-20T18:56:02.831 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1905e-05 2024-09-20T18:56:02.831 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.831 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.831 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:56:02.831 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.794337+0000", 2024-09-20T18:56:02.832 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8590000000000005e-06 2024-09-20T18:56:02.832 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.832 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.832 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:56:02.832 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.794397+0000", 2024-09-20T18:56:02.832 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0531e-05 2024-09-20T18:56:02.832 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.832 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.832 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:56:02.832 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.794533+0000", 2024-09-20T18:56:02.832 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013557200000000001 2024-09-20T18:56:02.832 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.832 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.832 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:56:02.832 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.794680+0000", 2024-09-20T18:56:02.833 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014704799999999999 2024-09-20T18:56:02.833 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.833 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.833 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:56:02.833 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.799870+0000", 2024-09-20T18:56:02.833 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0051902720000000001 2024-09-20T18:56:02.833 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.833 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.833 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:02.833 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.804125+0000", 2024-09-20T18:56:02.833 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0042546779999999996 2024-09-20T18:56:02.833 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.833 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.833 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:02.833 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.902983+0000", 2024-09-20T18:56:02.834 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098858370000000001 2024-09-20T18:56:02.834 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.834 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.834 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:56:02.834 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.903040+0000", 2024-09-20T18:56:02.834 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7136999999999999e-05 2024-09-20T18:56:02.834 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.834 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.834 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:56:02.834 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.903081+0000", 2024-09-20T18:56:02.834 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0435000000000002e-05 2024-09-20T18:56:02.834 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:02.834 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:56:02.835 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:02.835 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.835 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.835 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:187757 7.e 7:70705a0e:::benchmark_data_smithi083_70643_object64955:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-20T18:56:02.835 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:53:29.029730+0000", 2024-09-20T18:56:02.835 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 153.72109031400001, 2024-09-20T18:56:02.835 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.137882645, 2024-09-20T18:56:02.835 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:56:02.835 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:56:02.835 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:56:02.835 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:56:02.835 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:56:02.835 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:56:02.835 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 187757 2024-09-20T18:56:02.835 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.836 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:56:02.836 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.836 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:56:02.836 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.029730+0000", 2024-09-20T18:56:02.836 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:02.836 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.836 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.836 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:56:02.836 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.029730+0000", 2024-09-20T18:56:02.836 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:02.836 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.836 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.836 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:56:02.836 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.029732+0000", 2024-09-20T18:56:02.836 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.415e-06 2024-09-20T18:56:02.837 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.837 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.837 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:56:02.837 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.029737+0000", 2024-09-20T18:56:02.837 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0239999999999997e-06 2024-09-20T18:56:02.837 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.837 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.837 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:56:02.837 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.029748+0000", 2024-09-20T18:56:02.837 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0920999999999999e-05 2024-09-20T18:56:02.837 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.837 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.837 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:56:02.837 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.029755+0000", 2024-09-20T18:56:02.838 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1430000000000001e-06 2024-09-20T18:56:02.838 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.838 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.838 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:56:02.838 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.030815+0000", 2024-09-20T18:56:02.838 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0010603590000000001 2024-09-20T18:56:02.838 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.838 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.838 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:56:02.838 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.030930+0000", 2024-09-20T18:56:02.838 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000114171 2024-09-20T18:56:02.838 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.838 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.838 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:56:02.839 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.031080+0000", 2024-09-20T18:56:02.839 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015042000000000001 2024-09-20T18:56:02.839 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.839 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.839 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:56:02.839 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.036072+0000", 2024-09-20T18:56:02.839 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049921159999999996 2024-09-20T18:56:02.839 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.839 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.839 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:02.839 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.037324+0000", 2024-09-20T18:56:02.839 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0012515930000000001 2024-09-20T18:56:02.840 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.840 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.840 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:02.840 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045902+0000", 2024-09-20T18:56:02.840 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0085779580000000001 2024-09-20T18:56:02.840 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.840 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.840 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:56:02.840 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.167564+0000", 2024-09-20T18:56:02.840 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.121662143 2024-09-20T18:56:02.840 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.840 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.840 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:56:02.840 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.167613+0000", 2024-09-20T18:56:02.841 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9382000000000003e-05 2024-09-20T18:56:02.841 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:02.841 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:56:02.841 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:02.841 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.841 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.841 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:187765 7.e 7:7e1cec20:::benchmark_data_smithi083_70643_object64963:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-20T18:56:02.841 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:53:29.040056+0000", 2024-09-20T18:56:02.841 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 153.71076522800001, 2024-09-20T18:56:02.841 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.132718637, 2024-09-20T18:56:02.841 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:56:02.841 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:56:02.841 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:56:02.841 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:56:02.841 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:56:02.842 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:56:02.842 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 187765 2024-09-20T18:56:02.842 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.842 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:56:02.842 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.842 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:56:02.842 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.040056+0000", 2024-09-20T18:56:02.842 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:02.842 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.842 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.842 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:56:02.842 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.040056+0000", 2024-09-20T18:56:02.842 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:02.842 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.842 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.843 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:56:02.843 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.040057+0000", 2024-09-20T18:56:02.843 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.528e-06 2024-09-20T18:56:02.843 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.843 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.843 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:56:02.843 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.040063+0000", 2024-09-20T18:56:02.843 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8170000000000004e-06 2024-09-20T18:56:02.843 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.843 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.843 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:56:02.843 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.040074+0000", 2024-09-20T18:56:02.843 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1253000000000001e-05 2024-09-20T18:56:02.843 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.844 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.844 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:56:02.844 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.040081+0000", 2024-09-20T18:56:02.844 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1389999999999998e-06 2024-09-20T18:56:02.844 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.844 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.844 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:56:02.844 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.040134+0000", 2024-09-20T18:56:02.844 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2549000000000001e-05 2024-09-20T18:56:02.844 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.844 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.844 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:56:02.844 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.040246+0000", 2024-09-20T18:56:02.844 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000112271 2024-09-20T18:56:02.844 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.845 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.845 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:56:02.845 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.040354+0000", 2024-09-20T18:56:02.845 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010791 2024-09-20T18:56:02.845 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.845 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.845 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:02.845 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045867+0000", 2024-09-20T18:56:02.845 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0055125449999999998 2024-09-20T18:56:02.845 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.845 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.845 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:56:02.845 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.172613+0000", 2024-09-20T18:56:02.845 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.12674679599999999 2024-09-20T18:56:02.846 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.846 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.846 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:02.846 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.172691+0000", 2024-09-20T18:56:02.846 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7255000000000005e-05 2024-09-20T18:56:02.846 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.846 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.846 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:56:02.846 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.172735+0000", 2024-09-20T18:56:02.846 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4069000000000001e-05 2024-09-20T18:56:02.846 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.846 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.846 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:56:02.846 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.172774+0000", 2024-09-20T18:56:02.847 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9505000000000002e-05 2024-09-20T18:56:02.847 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:02.847 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:56:02.847 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:02.847 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.847 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.847 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:187763 7.3 e141/83 7:c71c27bb:::benchmark_data_smithi083_70643_object64961:head v 141'11587, mlcod=141'11584)", 2024-09-20T18:56:02.847 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:53:29.043149+0000", 2024-09-20T18:56:02.847 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 153.70767154399999, 2024-09-20T18:56:02.847 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.125479285, 2024-09-20T18:56:02.847 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:56:02.847 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:56:02.847 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:56:02.847 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:56:02.847 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.848 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:56:02.848 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043149+0000", 2024-09-20T18:56:02.848 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:02.848 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.848 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.848 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:56:02.848 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043149+0000", 2024-09-20T18:56:02.848 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:02.848 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.848 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.848 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:56:02.848 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043150+0000", 2024-09-20T18:56:02.848 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7800000000000001e-07 2024-09-20T18:56:02.848 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.849 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.849 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:56:02.849 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043154+0000", 2024-09-20T18:56:02.849 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6480000000000002e-06 2024-09-20T18:56:02.849 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.849 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.849 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:56:02.849 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043162+0000", 2024-09-20T18:56:02.849 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1810000000000004e-06 2024-09-20T18:56:02.849 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.849 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.849 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:56:02.849 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043167+0000", 2024-09-20T18:56:02.849 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4500000000000003e-06 2024-09-20T18:56:02.849 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.850 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.850 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:56:02.850 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043227+0000", 2024-09-20T18:56:02.850 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9697999999999997e-05 2024-09-20T18:56:02.850 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.850 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.850 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:56:02.850 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043243+0000", 2024-09-20T18:56:02.850 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6461999999999999e-05 2024-09-20T18:56:02.850 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.850 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.850 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:56:02.850 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.168513+0000", 2024-09-20T18:56:02.850 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.125269624 2024-09-20T18:56:02.850 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.851 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.851 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:56:02.851 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.168629+0000", 2024-09-20T18:56:02.851 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011564400000000001 2024-09-20T18:56:02.851 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:02.851 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:56:02.851 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:02.851 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.851 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.851 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:187773 7.e 7:7168b707:::benchmark_data_smithi083_70643_object64971:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-20T18:56:02.851 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:53:29.043782+0000", 2024-09-20T18:56:02.851 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 153.70703881599999, 2024-09-20T18:56:02.851 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.14065076300000001, 2024-09-20T18:56:02.851 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:56:02.852 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:56:02.852 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:56:02.852 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:56:02.852 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:56:02.852 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:56:02.852 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 187773 2024-09-20T18:56:02.852 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.852 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:56:02.852 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.852 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:56:02.852 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043782+0000", 2024-09-20T18:56:02.852 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:02.852 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.852 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.852 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:56:02.853 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043782+0000", 2024-09-20T18:56:02.853 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:02.853 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.853 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.853 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:56:02.853 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043784+0000", 2024-09-20T18:56:02.853 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5260000000000001e-06 2024-09-20T18:56:02.853 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.853 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.853 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:56:02.853 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043788+0000", 2024-09-20T18:56:02.853 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.481e-06 2024-09-20T18:56:02.853 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.853 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.853 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:56:02.854 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043798+0000", 2024-09-20T18:56:02.854 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0393e-05 2024-09-20T18:56:02.854 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.854 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.854 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:56:02.854 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043805+0000", 2024-09-20T18:56:02.854 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5030000000000002e-06 2024-09-20T18:56:02.854 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.854 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.854 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:56:02.854 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.167654+0000", 2024-09-20T18:56:02.854 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.123848659 2024-09-20T18:56:02.854 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.855 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.855 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:56:02.855 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.167881+0000", 2024-09-20T18:56:02.855 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000227698 2024-09-20T18:56:02.855 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.855 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.855 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:56:02.855 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.167981+0000", 2024-09-20T18:56:02.855 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9611999999999994e-05 2024-09-20T18:56:02.855 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.855 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.855 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:02.855 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.172817+0000", 2024-09-20T18:56:02.855 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0048360829999999997 2024-09-20T18:56:02.856 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.856 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.856 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:56:02.856 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.172828+0000", 2024-09-20T18:56:02.856 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1280999999999999e-05 2024-09-20T18:56:02.856 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.856 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.856 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:02.856 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:02.857 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:02.857 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.184218+0000", 2024-09-20T18:56:02.857 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011389882 2024-09-20T18:56:02.857 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.857 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.857 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:56:02.858 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.184387+0000", 2024-09-20T18:56:02.858 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000168992 2024-09-20T18:56:02.858 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.858 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.858 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:56:02.858 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.184433+0000", 2024-09-20T18:56:02.858 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5652999999999998e-05 2024-09-20T18:56:02.858 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:02.858 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:56:02.858 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:02.858 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.858 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.859 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:187757 7.e e141/83 ondisk, result = 0)", 2024-09-20T18:56:02.859 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:53:29.045594+0000", 2024-09-20T18:56:02.859 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 153.70522686199999, 2024-09-20T18:56:02.859 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.12203122600000001, 2024-09-20T18:56:02.859 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:56:02.859 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:56:02.859 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:56:02.859 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:56:02.859 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.859 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:56:02.859 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045594+0000", 2024-09-20T18:56:02.859 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:02.859 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.860 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.860 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:56:02.860 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045594+0000", 2024-09-20T18:56:02.860 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:02.860 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.860 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.860 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:56:02.860 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045595+0000", 2024-09-20T18:56:02.860 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6499999999999998e-07 2024-09-20T18:56:02.860 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.860 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.860 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:56:02.860 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045599+0000", 2024-09-20T18:56:02.860 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6160000000000004e-06 2024-09-20T18:56:02.861 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.861 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.861 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:56:02.861 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045609+0000", 2024-09-20T18:56:02.861 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2269999999999999e-06 2024-09-20T18:56:02.861 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.861 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.861 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:56:02.861 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045615+0000", 2024-09-20T18:56:02.861 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1349999999999997e-06 2024-09-20T18:56:02.861 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.861 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.862 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:56:02.862 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045892+0000", 2024-09-20T18:56:02.862 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000276724 2024-09-20T18:56:02.862 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.862 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.862 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:56:02.862 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045897+0000", 2024-09-20T18:56:02.862 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2970000000000003e-06 2024-09-20T18:56:02.862 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.862 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.862 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:56:02.862 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.167625+0000", 2024-09-20T18:56:02.862 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.12172836200000001 2024-09-20T18:56:02.863 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:02.863 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:56:02.863 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:02.863 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.863 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.863 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:187772 7.a e141/83 7:5948c6a4:::benchmark_data_smithi083_70643_object64970:head v 141'11671, mlcod=141'11669)", 2024-09-20T18:56:02.863 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:53:29.045936+0000", 2024-09-20T18:56:02.863 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 153.704885166, 2024-09-20T18:56:02.863 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.12458983799999999, 2024-09-20T18:56:02.863 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:56:02.863 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:56:02.863 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:56:02.863 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:56:02.863 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.864 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:56:02.864 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045936+0000", 2024-09-20T18:56:02.864 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:02.864 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.864 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.864 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:56:02.864 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045936+0000", 2024-09-20T18:56:02.864 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:02.864 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.864 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.864 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:56:02.864 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045936+0000", 2024-09-20T18:56:02.864 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2900000000000003e-07 2024-09-20T18:56:02.865 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.865 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.865 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:56:02.865 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045942+0000", 2024-09-20T18:56:02.865 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.66e-06 2024-09-20T18:56:02.865 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.865 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.865 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:56:02.865 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.165480+0000", 2024-09-20T18:56:02.865 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.119537582 2024-09-20T18:56:02.865 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.865 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.865 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:56:02.865 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.165496+0000", 2024-09-20T18:56:02.866 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6249999999999999e-05 2024-09-20T18:56:02.866 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.866 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.866 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:56:02.866 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.165558+0000", 2024-09-20T18:56:02.866 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2120999999999997e-05 2024-09-20T18:56:02.866 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.866 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.866 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:56:02.866 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.165582+0000", 2024-09-20T18:56:02.866 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4409e-05 2024-09-20T18:56:02.866 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.866 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.867 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:56:02.867 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.170489+0000", 2024-09-20T18:56:02.867 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049066860000000004 2024-09-20T18:56:02.867 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.867 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.867 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:56:02.867 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.170525+0000", 2024-09-20T18:56:02.867 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6400999999999999e-05 2024-09-20T18:56:02.867 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:02.867 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:56:02.867 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:02.867 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.867 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.867 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:187774 7.e 7:778445cb:::benchmark_data_smithi083_70643_object64972:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-20T18:56:02.868 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:53:29.046524+0000", 2024-09-20T18:56:02.868 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 153.70429674100001, 2024-09-20T18:56:02.868 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.15650777099999999, 2024-09-20T18:56:02.868 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:56:02.868 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:56:02.868 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:56:02.868 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:56:02.868 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:56:02.868 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:56:02.868 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 187774 2024-09-20T18:56:02.868 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.868 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:56:02.868 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.868 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:56:02.868 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.046524+0000", 2024-09-20T18:56:02.869 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:02.869 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.869 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.869 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:56:02.869 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.046524+0000", 2024-09-20T18:56:02.869 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:02.869 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.869 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.869 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:56:02.869 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.046526+0000", 2024-09-20T18:56:02.869 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.494e-06 2024-09-20T18:56:02.870 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.870 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.870 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:56:02.870 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.046531+0000", 2024-09-20T18:56:02.870 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4840000000000003e-06 2024-09-20T18:56:02.870 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.870 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.870 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:56:02.870 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.165656+0000", 2024-09-20T18:56:02.870 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.11912532100000001 2024-09-20T18:56:02.870 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.870 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.870 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:56:02.870 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.165666+0000", 2024-09-20T18:56:02.871 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1789999999999997e-06 2024-09-20T18:56:02.871 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.871 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.871 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:56:02.871 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.173061+0000", 2024-09-20T18:56:02.871 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0073953969999999997 2024-09-20T18:56:02.871 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.871 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.871 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:56:02.871 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.173154+0000", 2024-09-20T18:56:02.871 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3113000000000001e-05 2024-09-20T18:56:02.871 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.871 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.871 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:56:02.872 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.173250+0000", 2024-09-20T18:56:02.872 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5493000000000002e-05 2024-09-20T18:56:02.872 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.872 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.872 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:02.872 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.180360+0000", 2024-09-20T18:56:02.872 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0071109220000000004 2024-09-20T18:56:02.872 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.872 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.872 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:56:02.872 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.188345+0000", 2024-09-20T18:56:02.872 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0079849840000000005 2024-09-20T18:56:02.872 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.872 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.873 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:02.873 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.202813+0000", 2024-09-20T18:56:02.873 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.014467647 2024-09-20T18:56:02.873 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.873 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.873 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:56:02.873 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.202985+0000", 2024-09-20T18:56:02.873 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017191999999999999 2024-09-20T18:56:02.873 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.873 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.873 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:56:02.873 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.203032+0000", 2024-09-20T18:56:02.873 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6817000000000003e-05 2024-09-20T18:56:02.873 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:02.874 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:56:02.874 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:02.874 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.874 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.874 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:187765 7.e e141/83 ondisk, result = 0)", 2024-09-20T18:56:02.874 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:53:29.046987+0000", 2024-09-20T18:56:02.874 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 153.703833742, 2024-09-20T18:56:02.874 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.12579221700000001, 2024-09-20T18:56:02.874 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:56:02.874 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:56:02.874 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:56:02.874 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:56:02.874 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.875 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:56:02.875 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.046987+0000", 2024-09-20T18:56:02.875 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:02.875 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.875 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.875 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:56:02.875 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.046987+0000", 2024-09-20T18:56:02.875 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:02.875 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.875 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.875 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:56:02.875 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.046988+0000", 2024-09-20T18:56:02.875 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6499999999999999e-07 2024-09-20T18:56:02.876 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.876 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.876 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:56:02.876 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.046992+0000", 2024-09-20T18:56:02.876 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9650000000000002e-06 2024-09-20T18:56:02.876 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.876 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.876 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:56:02.876 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.165597+0000", 2024-09-20T18:56:02.876 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.118605087 2024-09-20T18:56:02.876 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.876 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.876 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:56:02.876 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.165603+0000", 2024-09-20T18:56:02.877 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0100000000000001e-06 2024-09-20T18:56:02.877 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.877 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.877 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:56:02.877 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.172679+0000", 2024-09-20T18:56:02.877 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0070759810000000003 2024-09-20T18:56:02.877 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.877 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.877 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:56:02.877 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.172686+0000", 2024-09-20T18:56:02.877 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9959999999999996e-06 2024-09-20T18:56:02.877 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.877 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.877 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:56:02.877 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.172779+0000", 2024-09-20T18:56:02.878 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3512999999999997e-05 2024-09-20T18:56:02.878 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:02.878 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:56:02.878 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:02.878 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.878 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.878 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:197546 7.e 7:71d75882:::benchmark_data_smithi083_70643_object74744:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e144)", 2024-09-20T18:56:02.878 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:53:38.347654+0000", 2024-09-20T18:56:02.878 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 144.40316662999999, 2024-09-20T18:56:02.878 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.103736439, 2024-09-20T18:56:02.878 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:56:02.878 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:56:02.879 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:56:02.879 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:56:02.879 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:56:02.879 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:56:02.879 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 197546 2024-09-20T18:56:02.879 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.879 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:56:02.879 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.879 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:56:02.879 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:38.347654+0000", 2024-09-20T18:56:02.879 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:02.879 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.879 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.879 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:56:02.880 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:38.347654+0000", 2024-09-20T18:56:02.880 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:02.880 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.880 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.880 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:56:02.880 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:38.347655+0000", 2024-09-20T18:56:02.880 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1510000000000001e-06 2024-09-20T18:56:02.880 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.880 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.880 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:56:02.880 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:38.347660+0000", 2024-09-20T18:56:02.880 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6210000000000003e-06 2024-09-20T18:56:02.880 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.880 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.881 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:56:02.881 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:38.347671+0000", 2024-09-20T18:56:02.881 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1399999999999999e-05 2024-09-20T18:56:02.881 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.881 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.881 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:56:02.881 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:38.347679+0000", 2024-09-20T18:56:02.881 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.1200000000000002e-06 2024-09-20T18:56:02.881 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.881 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.881 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:56:02.881 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:38.392261+0000", 2024-09-20T18:56:02.881 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044581335 2024-09-20T18:56:02.881 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.882 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.882 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:56:02.882 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:38.392366+0000", 2024-09-20T18:56:02.882 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010553299999999999 2024-09-20T18:56:02.882 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.882 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.882 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:56:02.882 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:38.392476+0000", 2024-09-20T18:56:02.882 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000110093 2024-09-20T18:56:02.882 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.882 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.882 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:56:02.882 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:38.395765+0000", 2024-09-20T18:56:02.882 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0032884979999999999 2024-09-20T18:56:02.883 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.883 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.883 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:02.883 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:38.399134+0000", 2024-09-20T18:56:02.883 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003369244 2024-09-20T18:56:02.883 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.883 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.883 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:02.883 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:38.451307+0000", 2024-09-20T18:56:02.883 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.052172832000000002 2024-09-20T18:56:02.883 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.883 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.883 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:56:02.883 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:38.451351+0000", 2024-09-20T18:56:02.884 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3890999999999997e-05 2024-09-20T18:56:02.884 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:02.884 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:02.884 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:56:02.884 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:38.451391+0000", 2024-09-20T18:56:02.884 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9721000000000001e-05 2024-09-20T18:56:02.884 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:02.884 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:56:02.884 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:02.884 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:02.884 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:56:02.884 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T18:56:02.958 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:02.983 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:02.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:02.984 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:03.021 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:02 smithi142 ceph-mon[32508]: pgmap v294: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 143 active+clean; 6.4 GiB data, 19 GiB used, 606 GiB / 626 GiB avail; 1.2 KiB/s rd, 37 MiB/s wr, 594 op/s; 22949/315912 objects misplaced (7.264%); 9.6 MiB/s, 153 objects/s recovering 2024-09-20T18:56:03.021 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:02 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1022711106' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:56:03.021 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:02 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:56:03.021 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:02 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3990079200' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:56:03.031 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:02 smithi083 ceph-mon[36260]: pgmap v294: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 143 active+clean; 6.4 GiB data, 19 GiB used, 606 GiB / 626 GiB avail; 1.2 KiB/s rd, 37 MiB/s wr, 594 op/s; 22949/315912 objects misplaced (7.264%); 9.6 MiB/s, 153 objects/s recovering 2024-09-20T18:56:03.031 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:02 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1022711106' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:56:03.031 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:02 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:56:03.032 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:02 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3990079200' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:56:03.032 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:02 smithi083 ceph-mon[29421]: pgmap v294: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 143 active+clean; 6.4 GiB data, 19 GiB used, 606 GiB / 626 GiB avail; 1.2 KiB/s rd, 37 MiB/s wr, 594 op/s; 22949/315912 objects misplaced (7.264%); 9.6 MiB/s, 153 objects/s recovering 2024-09-20T18:56:03.032 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:02 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1022711106' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:56:03.033 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:02 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:56:03.033 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:02 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3990079200' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:56:03.085 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:03.111 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:03.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:03.112 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:03.190 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 83 16 110391 110375 83.0762 50.0625 0.00734338 0.0120316 2024-09-20T18:56:03.212 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:03.239 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:03.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:03.240 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:03.341 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:03.375 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:03.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:03.376 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:03.477 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:03.505 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:03.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:03.506 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:03.606 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:03.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:03.639 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:03.640 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:03.740 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:03.765 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:03.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:03.766 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:03.866 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:03.894 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:03.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:03.895 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:03.996 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:04.020 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:03 smithi142 ceph-mon[32508]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T18:56:04.021 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:03 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T18:56:04.021 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:03 smithi142 ceph-mon[32508]: osdmap e193: 8 total, 8 up, 7 in 2024-09-20T18:56:04.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:04.024 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:04.025 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:04.051 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:03 smithi083 ceph-mon[29421]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T18:56:04.051 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:03 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T18:56:04.051 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:03 smithi083 ceph-mon[29421]: osdmap e193: 8 total, 8 up, 7 in 2024-09-20T18:56:04.051 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:03 smithi083 ceph-mon[36260]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T18:56:04.052 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:03 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T18:56:04.052 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:03 smithi083 ceph-mon[36260]: osdmap e193: 8 total, 8 up, 7 in 2024-09-20T18:56:04.074 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:56:04.126 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:04.154 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:04.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:04.156 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:04.190 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 84 16 111286 111270 82.7531 55.9375 0.00551997 0.0120793 2024-09-20T18:56:04.256 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:04.298 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:04.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:04.299 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:04.400 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:04.438 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:04.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:04.439 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:04.540 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:56:04.578 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:04.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:04.579 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:56:04.626 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-20T18:56:04.680 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:04.706 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:04.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:04.707 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:04.808 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:56:04.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:04.862 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:04.862 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:56:04.942 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:04 smithi083 ceph-mon[29421]: pgmap v296: 148 pgs: 2 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 143 active+clean; 6.6 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 853 B/s rd, 45 MiB/s wr, 716 op/s; 22337/327228 objects misplaced (6.826%); 12 MiB/s, 197 objects/s recovering 2024-09-20T18:56:04.942 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:04 smithi083 ceph-mon[29421]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-09-20T18:56:04.942 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:04 smithi083 ceph-mon[29421]: Cluster is now healthy 2024-09-20T18:56:04.943 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:04 smithi083 ceph-mon[36260]: pgmap v296: 148 pgs: 2 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 143 active+clean; 6.6 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 853 B/s rd, 45 MiB/s wr, 716 op/s; 22337/327228 objects misplaced (6.826%); 12 MiB/s, 197 objects/s recovering 2024-09-20T18:56:04.943 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:04 smithi083 ceph-mon[36260]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-09-20T18:56:04.943 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:04 smithi083 ceph-mon[36260]: Cluster is now healthy 2024-09-20T18:56:04.963 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:05.021 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:05.021 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:04 smithi142 ceph-mon[32508]: pgmap v296: 148 pgs: 2 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 143 active+clean; 6.6 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 853 B/s rd, 45 MiB/s wr, 716 op/s; 22337/327228 objects misplaced (6.826%); 12 MiB/s, 197 objects/s recovering 2024-09-20T18:56:05.021 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:04 smithi142 ceph-mon[32508]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-09-20T18:56:05.022 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:04 smithi142 ceph-mon[32508]: Cluster is now healthy 2024-09-20T18:56:05.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:05.022 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:05.123 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:05.150 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:05.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:05.151 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:05.190 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 85 16 112132 112116 82.4017 52.875 0.0382342 0.0121302 2024-09-20T18:56:05.251 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:05.277 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:05.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:05.278 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:05.378 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:05.405 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:05.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:05.407 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:05.507 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:05.534 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:05.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:05.535 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:05.636 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:56:05.672 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:05.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:05.673 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:56:05.759 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T18:56:05.774 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:05.798 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:05.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:05.799 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:05.900 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:56:05.930 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:56:05.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:05.994 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:05.994 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:56:06.095 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:56:06.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:06.158 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:06.158 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:56:06.194 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 86 16 112878 112862 81.9824 46.625 0.0159474 0.0121927 2024-09-20T18:56:06.259 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:06.286 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:06.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:06.287 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:06.388 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:06.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:06.446 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:06.446 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:06.547 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:56:06.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:06.593 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:06.593 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:56:06.693 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:56:06.728 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.3 to 1 (10000) 2024-09-20T18:56:06.740 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:06.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:06.741 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:56:06.841 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:06.863 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:06.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:06.864 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:06.965 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:56:06.998 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:06.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:06.999 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:56:07.021 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:56:06 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:56:07.021 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:06 smithi142 ceph-mon[32508]: pgmap v297: 148 pgs: 2 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 143 active+clean; 6.6 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 852 B/s rd, 45 MiB/s wr, 716 op/s; 22337/327228 objects misplaced (6.826%); 12 MiB/s, 197 objects/s recovering 2024-09-20T18:56:07.021 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:06 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:56:07.021 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:06 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:56:07.021 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:06 smithi142 ceph-mon[32508]: osdmap e194: 8 total, 8 up, 7 in 2024-09-20T18:56:07.021 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:06 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3725441020' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-20T18:56:07.051 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:06 smithi083 ceph-mon[29421]: pgmap v297: 148 pgs: 2 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 143 active+clean; 6.6 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 852 B/s rd, 45 MiB/s wr, 716 op/s; 22337/327228 objects misplaced (6.826%); 12 MiB/s, 197 objects/s recovering 2024-09-20T18:56:07.051 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:06 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:56:07.051 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:06 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:56:07.051 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:06 smithi083 ceph-mon[29421]: osdmap e194: 8 total, 8 up, 7 in 2024-09-20T18:56:07.051 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:06 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3725441020' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-20T18:56:07.051 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:06 smithi083 ceph-mon[36260]: pgmap v297: 148 pgs: 2 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 143 active+clean; 6.6 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 852 B/s rd, 45 MiB/s wr, 716 op/s; 22337/327228 objects misplaced (6.826%); 12 MiB/s, 197 objects/s recovering 2024-09-20T18:56:07.051 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:06 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:56:07.051 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:06 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:56:07.051 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:06 smithi083 ceph-mon[36260]: osdmap e194: 8 total, 8 up, 7 in 2024-09-20T18:56:07.051 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:06 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3725441020' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-20T18:56:07.100 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:07.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:07.132 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:07.132 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:07.194 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 87 16 113624 113608 81.5761 46.625 0.0068144 0.0122522 2024-09-20T18:56:07.232 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:56:07.271 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:07.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:07.272 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:56:07.323 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T18:56:07.373 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:56:07.421 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:07.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:07.422 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:56:07.523 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:56:07.567 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:07.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:07.568 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:56:07.668 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:07.698 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:07.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:07.699 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:07.800 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:07.829 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:07.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:07.830 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:07.931 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:56:07.970 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:07.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:07.971 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:56:08.020 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:07 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:56:08.021 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:07 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3725441020' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-20T18:56:08.021 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:07 smithi142 ceph-mon[32508]: osdmap e195: 8 total, 8 up, 7 in 2024-09-20T18:56:08.021 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:07 smithi142 ceph-mon[32508]: pgmap v300: 148 pgs: 2 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 143 active+clean; 6.9 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 852 B/s rd, 81 MiB/s wr, 1.29k op/s; 21105/339210 objects misplaced (6.222%); 26 MiB/s, 423 objects/s recovering 2024-09-20T18:56:08.051 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:07 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:56:08.051 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:07 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3725441020' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-20T18:56:08.051 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:07 smithi083 ceph-mon[29421]: osdmap e195: 8 total, 8 up, 7 in 2024-09-20T18:56:08.051 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:07 smithi083 ceph-mon[29421]: pgmap v300: 148 pgs: 2 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 143 active+clean; 6.9 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 852 B/s rd, 81 MiB/s wr, 1.29k op/s; 21105/339210 objects misplaced (6.222%); 26 MiB/s, 423 objects/s recovering 2024-09-20T18:56:08.051 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:07 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:56:08.051 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:07 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3725441020' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-20T18:56:08.051 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:07 smithi083 ceph-mon[36260]: osdmap e195: 8 total, 8 up, 7 in 2024-09-20T18:56:08.051 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:07 smithi083 ceph-mon[36260]: pgmap v300: 148 pgs: 2 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 143 active+clean; 6.9 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 852 B/s rd, 81 MiB/s wr, 1.29k op/s; 21105/339210 objects misplaced (6.222%); 26 MiB/s, 423 objects/s recovering 2024-09-20T18:56:08.072 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:08.100 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:08.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:08.101 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:08.194 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 88 16 114449 114433 81.2351 51.5625 0.013433 0.0123051 2024-09-20T18:56:08.202 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:56:08.238 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:08.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:08.239 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:56:08.340 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:56:08.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:08.380 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:08.380 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:56:08.473 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T18:56:08.473 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-20T18:56:08.473 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.473 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.24955.0:114249 8.8 e195/160 8:1cea2238:::benchmark_data_smithi083_97422_object114248:head v 195'7193, mlcod=195'7191)", 2024-09-20T18:56:08.473 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:07.910855+0000", 2024-09-20T18:56:08.473 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0065110080000000004, 2024-09-20T18:56:08.473 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065382180000000002, 2024-09-20T18:56:08.473 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:56:08.473 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:56:08.473 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:56:08.473 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:56:08.473 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.474 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:56:08.474 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.910855+0000", 2024-09-20T18:56:08.474 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:08.474 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.474 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.474 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:56:08.474 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.910855+0000", 2024-09-20T18:56:08.474 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:08.474 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.474 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.474 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:56:08.474 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.910856+0000", 2024-09-20T18:56:08.474 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1100000000000004e-07 2024-09-20T18:56:08.474 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.475 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.475 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:56:08.475 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.910885+0000", 2024-09-20T18:56:08.475 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9252999999999999e-05 2024-09-20T18:56:08.475 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.475 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.475 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:56:08.475 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.910896+0000", 2024-09-20T18:56:08.475 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0988000000000001e-05 2024-09-20T18:56:08.475 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.475 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.475 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:56:08.475 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.910904+0000", 2024-09-20T18:56:08.475 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2849999999999997e-06 2024-09-20T18:56:08.476 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.476 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.476 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:56:08.476 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.913182+0000", 2024-09-20T18:56:08.476 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0022781210000000001 2024-09-20T18:56:08.476 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.476 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.476 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:56:08.476 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.913196+0000", 2024-09-20T18:56:08.476 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4771e-05 2024-09-20T18:56:08.476 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:08.476 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:56:08.476 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:08.476 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.477 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.477 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:114257 8.0 8:0bf7f690:::benchmark_data_smithi083_97422_object114256:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e195)", 2024-09-20T18:56:08.477 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:07.914687+0000", 2024-09-20T18:56:08.477 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0026788939999999998, 2024-09-20T18:56:08.477 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002768983, 2024-09-20T18:56:08.477 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:56:08.477 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:56:08.477 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "waiting for sub ops", 2024-09-20T18:56:08.477 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:56:08.477 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:56:08.477 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:56:08.477 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 114257 2024-09-20T18:56:08.478 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.478 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:56:08.478 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.478 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:56:08.478 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.914687+0000", 2024-09-20T18:56:08.478 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:08.478 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.478 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.478 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:56:08.478 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.914687+0000", 2024-09-20T18:56:08.478 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:08.478 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.478 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.478 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:56:08.478 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.914689+0000", 2024-09-20T18:56:08.479 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.699e-06 2024-09-20T18:56:08.479 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.479 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.479 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:56:08.479 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.914721+0000", 2024-09-20T18:56:08.479 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2561999999999998e-05 2024-09-20T18:56:08.479 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.479 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.479 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:56:08.479 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.914734+0000", 2024-09-20T18:56:08.479 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2232000000000001e-05 2024-09-20T18:56:08.479 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.479 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.480 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:56:08.480 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.914742+0000", 2024-09-20T18:56:08.480 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2819999999999996e-06 2024-09-20T18:56:08.480 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.480 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.480 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:56:08.480 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.914796+0000", 2024-09-20T18:56:08.480 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4296000000000003e-05 2024-09-20T18:56:08.480 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.480 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.480 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:56:08.480 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.914926+0000", 2024-09-20T18:56:08.480 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012951299999999999 2024-09-20T18:56:08.480 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.480 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.481 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,5", 2024-09-20T18:56:08.481 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.915090+0000", 2024-09-20T18:56:08.481 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000164473 2024-09-20T18:56:08.481 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:08.481 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:56:08.481 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:08.481 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.481 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.481 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.24955.0:114254 8.7 e195/182 8:edca3a18:::benchmark_data_smithi083_97422_object114253:head v 195'7159, mlcod=195'7157)", 2024-09-20T18:56:08.481 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:07.916751+0000", 2024-09-20T18:56:08.481 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.00061516399999999997, 2024-09-20T18:56:08.481 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00077074800000000003, 2024-09-20T18:56:08.481 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:56:08.482 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:56:08.482 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:56:08.482 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:56:08.482 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.482 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:56:08.482 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.916751+0000", 2024-09-20T18:56:08.482 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:08.482 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.482 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.482 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:56:08.482 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.916751+0000", 2024-09-20T18:56:08.482 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:08.482 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.482 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.483 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:56:08.483 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.916752+0000", 2024-09-20T18:56:08.483 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.3900000000000004e-07 2024-09-20T18:56:08.483 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.483 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.483 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:56:08.483 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.916784+0000", 2024-09-20T18:56:08.483 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1745999999999997e-05 2024-09-20T18:56:08.483 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.483 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.483 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:56:08.483 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.916798+0000", 2024-09-20T18:56:08.483 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4174e-05 2024-09-20T18:56:08.483 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.483 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.484 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:56:08.484 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.916805+0000", 2024-09-20T18:56:08.484 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7389999999999999e-06 2024-09-20T18:56:08.484 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.484 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.484 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:56:08.484 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.916961+0000", 2024-09-20T18:56:08.484 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015523300000000001 2024-09-20T18:56:08.484 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.484 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.484 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:56:08.484 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.916983+0000", 2024-09-20T18:56:08.484 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2758000000000001e-05 2024-09-20T18:56:08.484 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:08.485 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:56:08.485 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:08.485 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.485 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.485 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.24955.0:114241 8.0 e195/160)", 2024-09-20T18:56:08.485 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:07.916825+0000", 2024-09-20T18:56:08.485 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.00054118000000000005, 2024-09-20T18:56:08.485 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00073896400000000005, 2024-09-20T18:56:08.485 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:56:08.485 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:56:08.485 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "queued for pg", 2024-09-20T18:56:08.485 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:56:08.485 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.485 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:56:08.486 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.916825+0000", 2024-09-20T18:56:08.486 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:08.486 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.486 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.486 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:56:08.486 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.916825+0000", 2024-09-20T18:56:08.486 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:08.486 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.486 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.486 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:56:08.486 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.916825+0000", 2024-09-20T18:56:08.486 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9499999999999998e-07 2024-09-20T18:56:08.486 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.486 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.487 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:56:08.487 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.916828+0000", 2024-09-20T18:56:08.487 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7039999999999999e-06 2024-09-20T18:56:08.487 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.487 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.487 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:56:08.487 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.916833+0000", 2024-09-20T18:56:08.487 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2100000000000001e-06 2024-09-20T18:56:08.487 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.487 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.487 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:56:08.487 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.916838+0000", 2024-09-20T18:56:08.487 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6639999999999997e-06 2024-09-20T18:56:08.488 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:08.488 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:56:08.488 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:08.488 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.488 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.488 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:114241 8.0 8:04b5c8c9:::benchmark_data_smithi083_97422_object114240:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e195)", 2024-09-20T18:56:08.488 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:07.905242+0000", 2024-09-20T18:56:08.488 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.012124084, 2024-09-20T18:56:08.488 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.012371010999999999, 2024-09-20T18:56:08.488 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:56:08.488 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:56:08.488 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "waiting for sub ops", 2024-09-20T18:56:08.488 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:56:08.488 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:56:08.488 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:56:08.488 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 114241 2024-09-20T18:56:08.489 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.489 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:56:08.489 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.489 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:56:08.489 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.905242+0000", 2024-09-20T18:56:08.489 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:08.489 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.489 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.489 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:56:08.489 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.905242+0000", 2024-09-20T18:56:08.489 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:08.489 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.489 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.490 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:56:08.490 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.905244+0000", 2024-09-20T18:56:08.490 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.579e-06 2024-09-20T18:56:08.490 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.490 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.490 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:56:08.490 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.905269+0000", 2024-09-20T18:56:08.490 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5088000000000001e-05 2024-09-20T18:56:08.490 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.490 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.490 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:56:08.490 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.905288+0000", 2024-09-20T18:56:08.490 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9581000000000001e-05 2024-09-20T18:56:08.491 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.491 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.491 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:56:08.491 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.905297+0000", 2024-09-20T18:56:08.491 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5830000000000007e-06 2024-09-20T18:56:08.491 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.491 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.491 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:56:08.491 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.906210+0000", 2024-09-20T18:56:08.491 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00091322899999999999 2024-09-20T18:56:08.491 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.491 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.491 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:56:08.491 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.906368+0000", 2024-09-20T18:56:08.492 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015825799999999999 2024-09-20T18:56:08.492 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.492 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.492 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,5", 2024-09-20T18:56:08.492 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.906533+0000", 2024-09-20T18:56:08.492 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000164733 2024-09-20T18:56:08.492 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.492 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.492 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:56:08.492 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.913132+0000", 2024-09-20T18:56:08.492 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0065987240000000003 2024-09-20T18:56:08.492 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.492 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.492 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:08.492 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.913159+0000", 2024-09-20T18:56:08.493 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7277999999999999e-05 2024-09-20T18:56:08.493 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:08.493 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:56:08.493 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:08.493 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.493 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.493 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.24955.0:114246 8.9 e195/182 8:94a0ae0c:::benchmark_data_smithi083_97422_object114245:head v 195'7150, mlcod=0'0)", 2024-09-20T18:56:08.493 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:07.905332+0000", 2024-09-20T18:56:08.493 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.012034573, 2024-09-20T18:56:08.493 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.012354675000000001, 2024-09-20T18:56:08.493 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:56:08.493 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:56:08.493 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:56:08.493 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:56:08.493 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.493 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:56:08.494 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.905332+0000", 2024-09-20T18:56:08.494 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:08.494 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.494 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.494 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:56:08.494 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.905332+0000", 2024-09-20T18:56:08.494 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:08.494 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.494 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.494 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:56:08.494 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.905332+0000", 2024-09-20T18:56:08.494 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0500000000000004e-07 2024-09-20T18:56:08.494 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.494 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.494 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:56:08.495 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.905364+0000", 2024-09-20T18:56:08.495 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2325e-05 2024-09-20T18:56:08.495 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.495 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.495 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:56:08.495 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.905372+0000", 2024-09-20T18:56:08.495 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.277e-06 2024-09-20T18:56:08.495 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.495 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.495 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:56:08.495 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.905376+0000", 2024-09-20T18:56:08.495 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7600000000000002e-06 2024-09-20T18:56:08.495 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.495 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.496 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:56:08.496 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.905713+0000", 2024-09-20T18:56:08.496 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00033634000000000002 2024-09-20T18:56:08.496 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.496 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.496 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:56:08.496 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.905735+0000", 2024-09-20T18:56:08.496 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2265000000000001e-05 2024-09-20T18:56:08.496 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:08.496 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:56:08.496 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:08.496 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.496 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.496 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.24955.0:114247 8.9 e195/182 8:9249520f:::benchmark_data_smithi083_97422_object114246:head v 195'7151, mlcod=0'0)", 2024-09-20T18:56:08.496 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:07.905390+0000", 2024-09-20T18:56:08.497 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.011975668, 2024-09-20T18:56:08.497 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.012343207, 2024-09-20T18:56:08.497 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:56:08.497 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:56:08.497 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:56:08.497 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:56:08.497 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.497 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:56:08.497 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.905390+0000", 2024-09-20T18:56:08.497 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:08.497 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.497 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.497 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:56:08.497 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.905390+0000", 2024-09-20T18:56:08.497 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:08.498 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.498 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.498 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:56:08.498 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.905391+0000", 2024-09-20T18:56:08.498 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.65e-07 2024-09-20T18:56:08.498 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.498 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.498 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:56:08.498 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.905421+0000", 2024-09-20T18:56:08.498 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0094e-05 2024-09-20T18:56:08.498 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.498 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.498 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:56:08.498 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.905426+0000", 2024-09-20T18:56:08.498 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1610000000000002e-06 2024-09-20T18:56:08.499 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.499 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.499 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:56:08.499 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.905430+0000", 2024-09-20T18:56:08.499 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1400000000000002e-06 2024-09-20T18:56:08.499 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.499 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.499 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:56:08.499 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.912884+0000", 2024-09-20T18:56:08.499 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074543839999999997 2024-09-20T18:56:08.499 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.499 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.499 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:56:08.499 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.912931+0000", 2024-09-20T18:56:08.500 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6851000000000003e-05 2024-09-20T18:56:08.500 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:08.500 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:56:08.500 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:08.500 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.500 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.500 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:114248 8.0 8:0a48010d:::benchmark_data_smithi083_97422_object114247:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e195)", 2024-09-20T18:56:08.500 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:07.905702+0000", 2024-09-20T18:56:08.500 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.011663685999999999, 2024-09-20T18:56:08.500 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.012110239, 2024-09-20T18:56:08.500 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:56:08.500 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:56:08.500 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "waiting for sub ops", 2024-09-20T18:56:08.500 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:56:08.501 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:56:08.501 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:56:08.501 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 114248 2024-09-20T18:56:08.501 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.501 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:56:08.501 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.501 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:56:08.501 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.905702+0000", 2024-09-20T18:56:08.501 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:08.501 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.501 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.501 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:56:08.501 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.905702+0000", 2024-09-20T18:56:08.501 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:08.502 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.502 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.502 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:56:08.502 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.905704+0000", 2024-09-20T18:56:08.502 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2720000000000001e-06 2024-09-20T18:56:08.502 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.502 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.502 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:56:08.502 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.905726+0000", 2024-09-20T18:56:08.502 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2691999999999999e-05 2024-09-20T18:56:08.502 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.502 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.502 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:56:08.502 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.905740+0000", 2024-09-20T18:56:08.503 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3186e-05 2024-09-20T18:56:08.503 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.503 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.503 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:56:08.503 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.905748+0000", 2024-09-20T18:56:08.503 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6249999999999996e-06 2024-09-20T18:56:08.503 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.503 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.503 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:56:08.503 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.909342+0000", 2024-09-20T18:56:08.503 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0035935759999999998 2024-09-20T18:56:08.503 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.503 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.503 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:56:08.503 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.909482+0000", 2024-09-20T18:56:08.504 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014023 2024-09-20T18:56:08.504 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:08.504 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:08.504 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,5", 2024-09-20T18:56:08.504 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:07.909633+0000", 2024-09-20T18:56:08.504 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015064200000000001 2024-09-20T18:56:08.504 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:08.504 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:56:08.504 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:08.504 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:08.504 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T18:56:08.504 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 8 2024-09-20T18:56:08.504 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T18:56:08.505 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:08.531 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:08.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:08.532 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:08.632 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:56:08.671 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:08.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:08.673 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:56:08.773 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:08.799 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:08.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:08.800 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:08.901 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:56:08.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:08.940 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:08.940 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:56:09.041 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:09.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:09.083 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:09.083 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:09.184 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:09.194 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 89 16 115300 115284 80.9201 53.1875 0.00835938 0.0123515 2024-09-20T18:56:09.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:09.211 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:09.211 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:09.312 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:09.337 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:09.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:09.338 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:09.438 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:56:09.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:09.479 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:09.480 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:56:09.580 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:09.587 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_blocked_ops 2024-09-20T18:56:09.606 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:09.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:09.607 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:09.708 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:09.759 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:09.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:09.760 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:09.861 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:56:09.911 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:09.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:09.912 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:56:10.013 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:56:10.032 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:56:10.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:10.078 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:10.078 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:56:10.107 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd reweight 4 1 2024-09-20T18:56:10.112 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:56:10.179 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:56:10.194 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Total time run: 90.0295 2024-09-20T18:56:10.194 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Total writes made: 116088 2024-09-20T18:56:10.194 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Write size: 65536 2024-09-20T18:56:10.194 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Object size: 65536 2024-09-20T18:56:10.194 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Bandwidth (MB/sec): 80.5903 2024-09-20T18:56:10.194 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Stddev Bandwidth: 40.3326 2024-09-20T18:56:10.194 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Max bandwidth (MB/sec): 203.188 2024-09-20T18:56:10.194 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Min bandwidth (MB/sec): 0 2024-09-20T18:56:10.194 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Average IOPS: 1289 2024-09-20T18:56:10.194 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Stddev IOPS: 645.333 2024-09-20T18:56:10.195 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Max IOPS: 3251 2024-09-20T18:56:10.195 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Min IOPS: 0 2024-09-20T18:56:10.195 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Average Latency(s): 0.012402 2024-09-20T18:56:10.195 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Stddev Latency(s): 0.0277238 2024-09-20T18:56:10.195 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Max latency(s): 2.23065 2024-09-20T18:56:10.195 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Min latency(s): 0.00111821 2024-09-20T18:56:10.200 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Cleaning up (deleting benchmark objects) 2024-09-20T18:56:10.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:10.213 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:10.213 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:56:10.220 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:09 smithi083 ceph-mon[29421]: pgmap v301: 148 pgs: 2 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 143 active+clean; 6.9 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 828 B/s rd, 40 MiB/s wr, 647 op/s; 21105/339210 objects misplaced (6.222%); 16 MiB/s, 250 objects/s recovering 2024-09-20T18:56:10.220 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:09 smithi083 ceph-mon[36260]: pgmap v301: 148 pgs: 2 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 143 active+clean; 6.9 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 828 B/s rd, 40 MiB/s wr, 647 op/s; 21105/339210 objects misplaced (6.222%); 16 MiB/s, 250 objects/s recovering 2024-09-20T18:56:10.270 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:09 smithi142 ceph-mon[32508]: pgmap v301: 148 pgs: 2 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 143 active+clean; 6.9 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 828 B/s rd, 40 MiB/s wr, 647 op/s; 21105/339210 objects misplaced (6.222%); 16 MiB/s, 250 objects/s recovering 2024-09-20T18:56:10.314 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:10.363 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:10.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:10.364 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:10.465 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:10.491 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:10.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:10.492 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:10.593 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:56:10.629 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:10.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:10.629 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:56:10.730 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:56:10.767 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:10.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:10.768 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:56:10.786 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:56:10 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:56:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:56:10.870 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:10.895 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:10.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:10.896 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:10.972 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:56:10.972 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:56:10.997 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:11.016 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:56:11.017 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:56:11.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:11.024 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:11.024 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:11.050 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:10 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:56:11.051 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:10 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:56:11.051 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:10 smithi083 ceph-mon[29421]: osdmap e196: 8 total, 8 up, 7 in 2024-09-20T18:56:11.051 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:10 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3225314793' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:56:11.051 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:10 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:56:11.051 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:10 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:56:11.051 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:10 smithi083 ceph-mon[36260]: osdmap e196: 8 total, 8 up, 7 in 2024-09-20T18:56:11.051 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:10 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3225314793' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:56:11.079 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:56:11.079 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:56:11.124 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:56:11.126 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:56:11.126 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:56:11.164 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:56:11.164 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:56:11.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:11.176 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:11.176 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:56:11.195 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:56:11.195 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:56:11.235 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:56:11.236 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:56:11.262 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:56:11.270 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:10 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:56:11.270 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:10 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:56:11.271 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:10 smithi142 ceph-mon[32508]: osdmap e196: 8 total, 8 up, 7 in 2024-09-20T18:56:11.271 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:10 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3225314793' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:56:11.277 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:11.277 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:56:11.278 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:56:11.300 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:11.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:11.301 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:11.402 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:11.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:11.456 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:11.456 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:11.544 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:56:11.557 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:11.596 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:11.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:11.597 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:11.698 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:11.724 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:11.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:11.725 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:11.825 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:11.855 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:11.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:11.856 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:11.956 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:56:11.991 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:11.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:11.992 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:56:12.005 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-20T18:56:12.093 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:56:12.117 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:12.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:12.118 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:56:12.218 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:12.247 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:12.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:12.247 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:12.270 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:11 smithi142 ceph-mon[32508]: pgmap v303: 148 pgs: 2 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 143 active+clean; 6.9 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1.7 KiB/s rd, 46 MiB/s wr, 755 op/s; 20900/340359 objects misplaced (6.141%); 18 MiB/s, 291 objects/s recovering 2024-09-20T18:56:12.270 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:11 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1389853595' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:56:12.270 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:11 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:56:12.301 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:11 smithi083 ceph-mon[29421]: pgmap v303: 148 pgs: 2 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 143 active+clean; 6.9 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1.7 KiB/s rd, 46 MiB/s wr, 755 op/s; 20900/340359 objects misplaced (6.141%); 18 MiB/s, 291 objects/s recovering 2024-09-20T18:56:12.301 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:11 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1389853595' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:56:12.301 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:11 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:56:12.301 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:11 smithi083 ceph-mon[36260]: pgmap v303: 148 pgs: 2 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 143 active+clean; 6.9 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1.7 KiB/s rd, 46 MiB/s wr, 755 op/s; 20900/340359 objects misplaced (6.141%); 18 MiB/s, 291 objects/s recovering 2024-09-20T18:56:12.301 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:11 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1389853595' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:56:12.301 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:11 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:56:12.349 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:12.375 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:12.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:12.376 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:12.477 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:12.503 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:12.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:12.504 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:12.605 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:12.649 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:12.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:12.650 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:12.751 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:12.779 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:12.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:12.780 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:12.881 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:56:12.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:12.917 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:12.917 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:56:13.018 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:13.048 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:13.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:13.049 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:13.149 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:13.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:13.184 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:13.184 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:13.270 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:12 smithi142 ceph-mon[32508]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:56:13.271 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:12 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T18:56:13.271 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:12 smithi142 ceph-mon[32508]: osdmap e197: 8 total, 8 up, 7 in 2024-09-20T18:56:13.285 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:13.289 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:12 smithi083 ceph-mon[29421]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:56:13.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:12 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T18:56:13.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:12 smithi083 ceph-mon[29421]: osdmap e197: 8 total, 8 up, 7 in 2024-09-20T18:56:13.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:12 smithi083 ceph-mon[36260]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:56:13.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:12 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T18:56:13.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:12 smithi083 ceph-mon[36260]: osdmap e197: 8 total, 8 up, 7 in 2024-09-20T18:56:13.313 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:13.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:13.314 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:13.415 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:56:13.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:13.458 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:13.458 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:56:13.559 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:56:13.602 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:13.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:13.603 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:56:13.703 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:13.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:13.732 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:13.732 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:13.833 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:56:13.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:13.873 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:13.873 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:56:13.974 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:14.004 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:14.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:14.005 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:14.106 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:14.133 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:14.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:14.134 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:14.235 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:14.262 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:14.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:14.263 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:14.265 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:13 smithi083 ceph-mon[36260]: pgmap v305: 148 pgs: 2 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 143 active+clean; 7.0 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 994 B/s rd, 33 MiB/s wr, 653 op/s; 19760/346431 objects misplaced (5.704%); 15 MiB/s, 241 objects/s recovering 2024-09-20T18:56:14.265 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:13 smithi083 ceph-mon[29421]: pgmap v305: 148 pgs: 2 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 143 active+clean; 7.0 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 994 B/s rd, 33 MiB/s wr, 653 op/s; 19760/346431 objects misplaced (5.704%); 15 MiB/s, 241 objects/s recovering 2024-09-20T18:56:14.270 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:13 smithi142 ceph-mon[32508]: pgmap v305: 148 pgs: 2 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 143 active+clean; 7.0 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 994 B/s rd, 33 MiB/s wr, 653 op/s; 19760/346431 objects misplaced (5.704%); 15 MiB/s, 241 objects/s recovering 2024-09-20T18:56:14.364 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:14.389 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:14.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:14.390 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:14.491 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:56:14.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:14.535 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:14.535 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:56:14.586 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T18:56:14.586 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:56:14.636 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:14.662 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:14.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:14.663 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:14.764 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:56:14.817 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:14.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:14.818 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:56:14.919 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:56:14.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:14.956 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:14.957 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:56:15.057 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:15.084 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:15.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:15.086 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:15.125 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T18:56:15.186 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:56:15.256 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:15.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:15.257 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:56:15.358 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:15.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:15.411 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:15.411 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:15.420 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_historic_ops 2024-09-20T18:56:15.512 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:56:15.567 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:15.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:15.568 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:56:15.669 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:56:15.719 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:15.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:15.720 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:56:15.821 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:15.848 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:15.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:15.849 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:15.950 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:16.005 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:16.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:16.006 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:16.077 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:56:16.107 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:16.131 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:16.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:16.132 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:16.138 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:56:16.233 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:56:16.274 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:16.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:16.275 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:56:16.318 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T18:56:16.375 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:56:16.438 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:16.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:16.484 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:56:16.585 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:16.609 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:16.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:16.610 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:16.705 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.4 to 1 (10000) 2024-09-20T18:56:16.711 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:56:16.733 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:16 smithi083 ceph-mon[29421]: pgmap v306: 148 pgs: 2 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 143 active+clean; 7.0 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 767 B/s rd, 25 MiB/s wr, 504 op/s; 19760/346431 objects misplaced (5.704%); 12 MiB/s, 186 objects/s recovering 2024-09-20T18:56:16.733 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:16 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:56:16.733 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:16 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:56:16.733 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:16 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:56:16.733 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:16 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:56:16.733 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:16 smithi083 ceph-mon[29421]: osdmap e198: 8 total, 8 up, 7 in 2024-09-20T18:56:16.733 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:16 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3495776063' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-20T18:56:16.733 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:16 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-20T18:56:16.734 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:16 smithi083 ceph-mon[36260]: pgmap v306: 148 pgs: 2 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 143 active+clean; 7.0 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 767 B/s rd, 25 MiB/s wr, 504 op/s; 19760/346431 objects misplaced (5.704%); 12 MiB/s, 186 objects/s recovering 2024-09-20T18:56:16.734 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:16 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:56:16.734 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:16 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:56:16.734 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:16 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:56:16.734 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:16 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:56:16.734 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:16 smithi083 ceph-mon[36260]: osdmap e198: 8 total, 8 up, 7 in 2024-09-20T18:56:16.734 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:16 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3495776063' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-20T18:56:16.734 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:16 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-20T18:56:16.756 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:16.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:16.757 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:56:16.858 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:56:16.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:16.897 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:16.897 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:56:16.997 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:17.020 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:56:16 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:56:17.021 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:16 smithi142 ceph-mon[32508]: pgmap v306: 148 pgs: 2 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 143 active+clean; 7.0 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 767 B/s rd, 25 MiB/s wr, 504 op/s; 19760/346431 objects misplaced (5.704%); 12 MiB/s, 186 objects/s recovering 2024-09-20T18:56:17.021 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:16 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:56:17.021 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:16 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:56:17.021 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:16 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:56:17.021 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:16 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:56:17.021 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:16 smithi142 ceph-mon[32508]: osdmap e198: 8 total, 8 up, 7 in 2024-09-20T18:56:17.021 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:16 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3495776063' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-20T18:56:17.021 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:16 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-20T18:56:17.024 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:17.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:17.025 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:17.126 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:56:17.167 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:17.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:17.168 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:56:17.269 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:17.290 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:56:17.290 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:56:17.293 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:17.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:17.294 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:17.326 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:56:17.326 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:56:17.364 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:56:17.364 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:56:17.394 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:56:17.402 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:56:17.402 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:56:17.431 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:17.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:17.432 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:56:17.437 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:56:17.437 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:56:17.466 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:56:17.466 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:56:17.497 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:56:17.497 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:56:17.529 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:56:17.529 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:56:17.533 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:56:17.575 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:17.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:17.576 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:56:17.677 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:56:17.712 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:17.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:17.713 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:56:17.814 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:56:17.833 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd reweight 5 1 2024-09-20T18:56:17.851 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:17.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:17.852 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:56:17.953 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:17.957 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:17 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-20T18:56:17.957 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:17 smithi083 ceph-mon[29421]: osdmap e199: 8 total, 8 up, 7 in 2024-09-20T18:56:17.957 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:17 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2801171575' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:56:17.957 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:17 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-20T18:56:17.957 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:17 smithi083 ceph-mon[36260]: osdmap e199: 8 total, 8 up, 7 in 2024-09-20T18:56:17.958 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:17 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2801171575' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:56:17.958 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T18:56:17.958 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:56:17.978 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:17.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:17.979 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:18.020 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:17 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-20T18:56:18.020 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:17 smithi142 ceph-mon[32508]: osdmap e199: 8 total, 8 up, 7 in 2024-09-20T18:56:18.021 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:17 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2801171575' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:56:18.080 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:18.131 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:18.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:18.132 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:18.232 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:56:18.270 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:18.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:18.271 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:56:18.372 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:56:18.409 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:18.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:18.410 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:56:18.511 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:56:18.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:18.544 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:18.544 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:56:18.645 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:18.671 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:18.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:18.673 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:18.773 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:56:18.802 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:18.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:18.803 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:56:18.904 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:18.928 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:18.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:18.929 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:19.020 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:18 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:56:19.021 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:18 smithi142 ceph-mon[32508]: pgmap v309: 148 pgs: 2 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 143 active+clean; 6.8 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1023 B/s rd, 29 MiB/s wr, 1.27k op/s; 17922/333963 objects misplaced (5.366%); 30 MiB/s, 477 objects/s recovering 2024-09-20T18:56:19.030 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:19.051 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:18 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:56:19.051 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:18 smithi083 ceph-mon[36260]: pgmap v309: 148 pgs: 2 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 143 active+clean; 6.8 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1023 B/s rd, 29 MiB/s wr, 1.27k op/s; 17922/333963 objects misplaced (5.366%); 30 MiB/s, 477 objects/s recovering 2024-09-20T18:56:19.051 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:18 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:56:19.051 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:18 smithi083 ceph-mon[29421]: pgmap v309: 148 pgs: 2 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 143 active+clean; 6.8 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1023 B/s rd, 29 MiB/s wr, 1.27k op/s; 17922/333963 objects misplaced (5.366%); 30 MiB/s, 477 objects/s recovering 2024-09-20T18:56:19.054 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:19.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:19.056 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:19.156 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:19.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:19.188 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:19.188 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:19.288 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:19.317 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:19.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:19.318 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:19.419 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:19.447 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:19.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:19.448 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:19.548 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:56:19.597 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:19.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:19.598 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:56:19.619 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-20T18:56:19.699 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:19.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:19.736 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:19.736 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:19.837 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:19.873 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:19.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:19.874 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:19.968 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:56:19.975 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:20.006 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:20.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:20.007 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:20.108 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:56:20.151 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:20.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:20.152 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:56:20.252 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:20.291 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:56:20.313 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:20.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:20.314 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:20.332 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:56:20.415 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:20.444 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:20.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:20.445 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:20.546 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:20.594 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:20.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:20.596 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:20.696 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:20.727 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:20.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:20.728 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:20.828 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:20.856 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:20.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:20.857 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:20.957 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:20.963 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:20 smithi083 ceph-mon[29421]: pgmap v310: 148 pgs: 2 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 143 active+clean; 6.8 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 740 B/s rd, 0 B/s wr, 601 op/s; 17922/333963 objects misplaced (5.366%); 14 MiB/s, 227 objects/s recovering 2024-09-20T18:56:20.963 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:20 smithi083 ceph-mon[29421]: osdmap e200: 8 total, 8 up, 7 in 2024-09-20T18:56:20.963 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:20 smithi083 ceph-mon[36260]: pgmap v310: 148 pgs: 2 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 143 active+clean; 6.8 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 740 B/s rd, 0 B/s wr, 601 op/s; 17922/333963 objects misplaced (5.366%); 14 MiB/s, 227 objects/s recovering 2024-09-20T18:56:20.964 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:20 smithi083 ceph-mon[36260]: osdmap e200: 8 total, 8 up, 7 in 2024-09-20T18:56:20.964 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:56:20 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:56:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:56:20.982 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:20.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:20.983 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:21.020 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:20 smithi142 ceph-mon[32508]: pgmap v310: 148 pgs: 2 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 143 active+clean; 6.8 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 740 B/s rd, 0 B/s wr, 601 op/s; 17922/333963 objects misplaced (5.366%); 14 MiB/s, 227 objects/s recovering 2024-09-20T18:56:21.020 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:20 smithi142 ceph-mon[32508]: osdmap e200: 8 total, 8 up, 7 in 2024-09-20T18:56:21.084 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:21.112 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:21.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:21.113 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:21.213 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:21.254 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:21.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:21.255 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:21.355 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:21.381 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:21.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:21.382 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:21.482 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:56:21.522 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:21.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:21.524 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:56:21.625 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:21.648 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:21.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:21.649 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:21.750 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:56:21.779 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-20T18:56:21.790 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:21.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:21.791 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:56:21.892 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:56:21.927 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:21.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:21.929 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:56:22.020 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:21 smithi142 ceph-mon[32508]: pgmap v312: 148 pgs: 2 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 143 active+clean; 6.8 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 694 op/s; 17922/333963 objects misplaced (5.366%); 16 MiB/s, 262 objects/s recovering 2024-09-20T18:56:22.020 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:21 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1955004867' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:56:22.020 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:21 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:56:22.029 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:22.050 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:21 smithi083 ceph-mon[29421]: pgmap v312: 148 pgs: 2 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 143 active+clean; 6.8 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 694 op/s; 17922/333963 objects misplaced (5.366%); 16 MiB/s, 262 objects/s recovering 2024-09-20T18:56:22.050 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:21 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1955004867' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:56:22.051 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:21 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:56:22.051 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:21 smithi083 ceph-mon[36260]: pgmap v312: 148 pgs: 2 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 143 active+clean; 6.8 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 694 op/s; 17922/333963 objects misplaced (5.366%); 16 MiB/s, 262 objects/s recovering 2024-09-20T18:56:22.051 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:21 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1955004867' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:56:22.051 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:21 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:56:22.055 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:22.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:22.058 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:22.159 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:22.182 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:22.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:22.183 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:22.265 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T18:56:22.284 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:22.309 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:22.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:22.310 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:22.411 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:56:22.435 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:22.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:22.436 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:56:22.537 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:22.560 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:22.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:22.562 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:22.662 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:56:22.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:22.694 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:22.695 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:56:22.795 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:22.844 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:22.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:22.845 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:22.946 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:22.984 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:22.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:22.985 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:23.015 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:22 smithi142 ceph-mon[32508]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:56:23.015 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:22 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T18:56:23.016 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:22 smithi142 ceph-mon[32508]: osdmap e201: 8 total, 8 up, 7 in 2024-09-20T18:56:23.016 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:22 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2126726097' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-20T18:56:23.016 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:22 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-20T18:56:23.050 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:22 smithi083 ceph-mon[29421]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:56:23.050 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:22 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T18:56:23.051 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:22 smithi083 ceph-mon[29421]: osdmap e201: 8 total, 8 up, 7 in 2024-09-20T18:56:23.051 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:22 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2126726097' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-20T18:56:23.051 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:22 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-20T18:56:23.051 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:22 smithi083 ceph-mon[36260]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:56:23.051 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:22 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T18:56:23.051 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:22 smithi083 ceph-mon[36260]: osdmap e201: 8 total, 8 up, 7 in 2024-09-20T18:56:23.051 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:22 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2126726097' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-20T18:56:23.051 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:22 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-20T18:56:23.073 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T18:56:23.073 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-09-20T18:56:23.073 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:23.073 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.24955.0:126604 8.2 8:427cecf0:::benchmark_data_smithi083_97422_object10513:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e202)", 2024-09-20T18:56:23.073 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:56:23.047924+0000", 2024-09-20T18:56:23.073 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.010450624, 2024-09-20T18:56:23.073 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.010472354999999999, 2024-09-20T18:56:23.073 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:56:23.073 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:56:23.073 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "waiting for sub ops", 2024-09-20T18:56:23.074 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:56:23.074 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.24955", 2024-09-20T18:56:23.074 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:56:23.074 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 126604 2024-09-20T18:56:23.074 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:23.074 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:56:23.074 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:23.074 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:56:23.074 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:23.047924+0000", 2024-09-20T18:56:23.074 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:56:23.074 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:23.074 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:23.074 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:56:23.075 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:23.047924+0000", 2024-09-20T18:56:23.075 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:56:23.075 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:23.075 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:23.075 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:56:23.075 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:23.047925+0000", 2024-09-20T18:56:23.075 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4160000000000001e-06 2024-09-20T18:56:23.075 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:23.075 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:23.075 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:56:23.075 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:23.047931+0000", 2024-09-20T18:56:23.075 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5210000000000002e-06 2024-09-20T18:56:23.075 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:23.075 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:23.075 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:56:23.076 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:23.047944+0000", 2024-09-20T18:56:23.076 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3040000000000001e-05 2024-09-20T18:56:23.076 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:23.076 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:23.076 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:56:23.076 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:23.047952+0000", 2024-09-20T18:56:23.076 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.7889999999999995e-06 2024-09-20T18:56:23.076 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:23.076 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:23.076 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:56:23.076 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:23.047997+0000", 2024-09-20T18:56:23.076 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.5309999999999998e-05 2024-09-20T18:56:23.076 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:23.077 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:23.077 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:56:23.077 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:23.048185+0000", 2024-09-20T18:56:23.077 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00018751600000000001 2024-09-20T18:56:23.077 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:23.077 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:23.077 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:56:23.077 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:23.048293+0000", 2024-09-20T18:56:23.077 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000108826 2024-09-20T18:56:23.077 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:23.077 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:23.078 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:56:23.078 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:23.051156+0000", 2024-09-20T18:56:23.078 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0028629950000000001 2024-09-20T18:56:23.078 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:23.078 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:23.078 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:23.078 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:23.051616+0000", 2024-09-20T18:56:23.078 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00046009099999999998 2024-09-20T18:56:23.078 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:56:23.078 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:56:23.078 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:56:23.078 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:56:23.078 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-09-20T18:56:23.079 INFO:teuthology.orchestra.run.smithi142.stdout: "num_ops": 1 2024-09-20T18:56:23.079 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T18:56:23.086 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:23.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:23.120 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:23.120 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:23.133 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.5 to 1 (10000) 2024-09-20T18:56:23.222 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:23.249 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:23.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:23.250 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:23.351 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:23.393 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:23.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:23.394 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:23.495 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:23.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:23.531 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:23.531 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:23.632 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:23.654 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:23.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:23.655 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:23.756 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:23.786 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:23.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:23.787 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:23.888 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:23.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:23.910 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:23.911 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:24.012 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:24.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:24.039 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:24.039 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:24.050 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:23 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-20T18:56:24.050 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:23 smithi083 ceph-mon[29421]: osdmap e202: 8 total, 8 up, 7 in 2024-09-20T18:56:24.051 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:23 smithi083 ceph-mon[29421]: pgmap v315: 148 pgs: 2 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 143 active+clean; 6.5 GiB data, 20 GiB used, 605 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 716 op/s; 16008/321072 objects misplaced (4.986%); 17 MiB/s, 279 objects/s recovering 2024-09-20T18:56:24.051 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:23 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-20T18:56:24.051 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:23 smithi083 ceph-mon[36260]: osdmap e202: 8 total, 8 up, 7 in 2024-09-20T18:56:24.051 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:23 smithi083 ceph-mon[36260]: pgmap v315: 148 pgs: 2 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 143 active+clean; 6.5 GiB data, 20 GiB used, 605 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 716 op/s; 16008/321072 objects misplaced (4.986%); 17 MiB/s, 279 objects/s recovering 2024-09-20T18:56:24.142 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:24.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:24.177 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:24.177 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:24.270 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:23 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-20T18:56:24.271 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:23 smithi142 ceph-mon[32508]: osdmap e202: 8 total, 8 up, 7 in 2024-09-20T18:56:24.271 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:23 smithi142 ceph-mon[32508]: pgmap v315: 148 pgs: 2 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 143 active+clean; 6.5 GiB data, 20 GiB used, 605 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 716 op/s; 16008/321072 objects misplaced (4.986%); 17 MiB/s, 279 objects/s recovering 2024-09-20T18:56:24.278 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:56:24.319 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:24.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:24.320 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:56:24.421 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:24.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:24.452 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:24.452 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:24.552 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:56:24.593 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:24.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:24.594 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:56:24.695 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:24.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:24.723 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:24.723 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:24.824 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:24.854 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:24.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:24.855 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:24.955 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:24.981 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:24.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:24.982 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:25.083 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:25.106 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:25.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:25.107 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:25.208 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:25.230 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:25.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:25.231 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:25.262 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_blocked_ops 2024-09-20T18:56:25.332 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:56:25.357 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:25.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:25.359 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:56:25.460 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:56:25.487 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:25.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:25.488 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:56:25.589 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:25.591 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:56:25.614 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:25.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:25.615 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:25.716 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:25.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:25.745 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:25.745 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:25.846 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:25.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:25.883 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:25.883 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:25.984 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:56:26.021 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:25 smithi142 ceph-mon[32508]: osdmap e203: 8 total, 8 up, 7 in 2024-09-20T18:56:26.021 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:25 smithi142 ceph-mon[32508]: pgmap v317: 148 pgs: 2 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 143 active+clean; 6.5 GiB data, 20 GiB used, 605 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.02k op/s; 16008/321072 objects misplaced (4.986%); 25 MiB/s, 397 objects/s recovering 2024-09-20T18:56:26.021 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:25 smithi142 ceph-mon[32508]: osdmap e204: 8 total, 8 up, 7 in 2024-09-20T18:56:26.024 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd reweight 7 1 2024-09-20T18:56:26.045 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:26.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:26.046 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:56:26.050 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:56:26.089 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:25 smithi083 ceph-mon[36260]: osdmap e203: 8 total, 8 up, 7 in 2024-09-20T18:56:26.089 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:25 smithi083 ceph-mon[36260]: pgmap v317: 148 pgs: 2 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 143 active+clean; 6.5 GiB data, 20 GiB used, 605 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.02k op/s; 16008/321072 objects misplaced (4.986%); 25 MiB/s, 397 objects/s recovering 2024-09-20T18:56:26.089 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:25 smithi083 ceph-mon[36260]: osdmap e204: 8 total, 8 up, 7 in 2024-09-20T18:56:26.089 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:25 smithi083 ceph-mon[29421]: osdmap e203: 8 total, 8 up, 7 in 2024-09-20T18:56:26.090 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:25 smithi083 ceph-mon[29421]: pgmap v317: 148 pgs: 2 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 143 active+clean; 6.5 GiB data, 20 GiB used, 605 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.02k op/s; 16008/321072 objects misplaced (4.986%); 25 MiB/s, 397 objects/s recovering 2024-09-20T18:56:26.090 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:25 smithi083 ceph-mon[29421]: osdmap e204: 8 total, 8 up, 7 in 2024-09-20T18:56:26.147 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:26.168 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T18:56:26.169 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-09-20T18:56:26.169 INFO:teuthology.orchestra.run.smithi142.stdout: "complaint_time": 30, 2024-09-20T18:56:26.169 INFO:teuthology.orchestra.run.smithi142.stdout: "num_blocked_ops": 0 2024-09-20T18:56:26.169 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T18:56:26.182 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:26.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:26.183 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:26.284 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:26.312 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:26.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:26.313 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:26.413 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:26.438 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:26.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:26.439 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:26.540 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:26.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:26.568 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:26.568 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:26.669 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:26.694 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:26.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:26.695 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:26.733 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:56:26.733 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:56:26.760 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:56:26.771 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:56:26.771 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:56:26.796 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:26.806 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:56:26.807 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:56:26.820 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:26.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:26.821 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:26.838 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:56:26.838 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:56:26.870 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:56:26.870 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:56:26.878 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_historic_ops 2024-09-20T18:56:26.903 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:26 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2034317380' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:56:26.904 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:26 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2034317380' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:56:26.904 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:56:26.904 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:56:26.922 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:26.937 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:56:26.937 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:56:26.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:26.967 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:26.967 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:26.979 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:56:26.980 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:56:27.020 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:56:26 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:56:27.020 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:26 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2034317380' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:56:27.068 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:27.096 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:27.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:27.097 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:27.198 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:27.203 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:56:27.231 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:27.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:27.232 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:27.333 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:27.361 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:27.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:27.362 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:27.462 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:27.489 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:27.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:27.490 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:27.493 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T18:56:27.590 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:27.614 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:27.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:27.615 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:27.716 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:27.771 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T18:56:27.771 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:56:27.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:27.773 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:27.773 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:27.874 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:27.899 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:27.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:27.900 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:28.001 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:28.020 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:27 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:56:28.020 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:27 smithi142 ceph-mon[32508]: pgmap v319: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 144 active+clean; 6.3 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 995 B/s rd, 0 B/s wr, 658 op/s; 13912/310920 objects misplaced (4.474%); 24 MiB/s, 379 objects/s recovering 2024-09-20T18:56:28.026 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:28.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:28.027 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:28.050 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:27 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:56:28.051 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:27 smithi083 ceph-mon[29421]: pgmap v319: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 144 active+clean; 6.3 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 995 B/s rd, 0 B/s wr, 658 op/s; 13912/310920 objects misplaced (4.474%); 24 MiB/s, 379 objects/s recovering 2024-09-20T18:56:28.051 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:27 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:56:28.051 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:27 smithi083 ceph-mon[36260]: pgmap v319: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 144 active+clean; 6.3 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 995 B/s rd, 0 B/s wr, 658 op/s; 13912/310920 objects misplaced (4.474%); 24 MiB/s, 379 objects/s recovering 2024-09-20T18:56:28.127 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:28.159 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:28.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:28.161 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:28.261 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:56:28.264 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:56:28.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:28.304 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:28.304 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:56:28.405 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:28.428 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:28.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:28.429 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:28.530 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:28.556 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:28.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:28.557 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:28.658 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:28.699 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:28.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:28.700 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:28.801 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:28.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:28.832 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:28.832 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:28.839 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.7 to 1 (10000) 2024-09-20T18:56:28.932 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:56:28.967 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:28.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:28.968 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:56:29.050 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:28 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3825287512' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-20T18:56:29.050 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:28 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3825287512' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-20T18:56:29.069 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:29.095 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:29.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:29.096 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:29.129 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:28 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3825287512' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-20T18:56:29.197 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:29.224 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:29.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:29.225 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:29.326 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:29.351 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:29.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:29.352 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:29.453 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:56:29.485 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:29.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:29.486 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:56:29.587 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:56:29.612 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-20T18:56:29.625 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:29.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:29.626 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:56:29.727 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:56:29.765 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:29.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:29.766 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:56:29.858 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-20T18:56:29.867 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:29.895 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:29.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:29.896 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:29.959 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:56:29.996 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:56:30.031 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:30.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:30.032 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:56:30.126 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:29 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3825287512' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-20T18:56:30.126 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:29 smithi142 ceph-mon[32508]: osdmap e205: 8 total, 8 up, 7 in 2024-09-20T18:56:30.126 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:29 smithi142 ceph-mon[32508]: pgmap v321: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 144 active+clean; 6.3 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 564 op/s; 13912/310920 objects misplaced (4.474%); 20 MiB/s, 325 objects/s recovering 2024-09-20T18:56:30.126 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:29 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3541330271' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:56:30.126 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:29 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:56:30.128 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T18:56:30.133 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:30.159 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:30.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:30.160 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:30.171 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:29 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3825287512' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-20T18:56:30.171 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:29 smithi083 ceph-mon[29421]: osdmap e205: 8 total, 8 up, 7 in 2024-09-20T18:56:30.171 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:29 smithi083 ceph-mon[29421]: pgmap v321: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 144 active+clean; 6.3 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 564 op/s; 13912/310920 objects misplaced (4.474%); 20 MiB/s, 325 objects/s recovering 2024-09-20T18:56:30.171 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:29 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3541330271' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:56:30.171 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:29 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:56:30.172 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:29 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3825287512' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-20T18:56:30.172 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:29 smithi083 ceph-mon[36260]: osdmap e205: 8 total, 8 up, 7 in 2024-09-20T18:56:30.172 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:29 smithi083 ceph-mon[36260]: pgmap v321: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 144 active+clean; 6.3 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 564 op/s; 13912/310920 objects misplaced (4.474%); 20 MiB/s, 325 objects/s recovering 2024-09-20T18:56:30.172 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:29 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3541330271' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:56:30.172 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:29 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:56:30.260 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:30.288 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:30.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:30.289 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:30.390 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:30.422 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:30.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:30.423 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:30.524 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:56:30.533 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T18:56:30.534 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:56:30.545 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:30.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:30.546 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:56:30.646 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:30.673 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:30.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:30.674 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:30.775 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:30.803 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:30.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:30.804 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:30.905 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:30.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:30.940 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:30.940 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:31.041 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:56:31.050 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:30 smithi083 ceph-mon[29421]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:56:31.050 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:30 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T18:56:31.050 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:30 smithi083 ceph-mon[29421]: osdmap e206: 8 total, 8 up, 7 in 2024-09-20T18:56:31.050 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:30 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:56:31.051 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:56:30 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:56:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:56:31.051 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:30 smithi083 ceph-mon[36260]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:56:31.051 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:30 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T18:56:31.051 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:30 smithi083 ceph-mon[36260]: osdmap e206: 8 total, 8 up, 7 in 2024-09-20T18:56:31.051 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:30 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:56:31.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:31.084 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:31.084 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:56:31.185 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:56:31.214 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:31.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:31.215 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:56:31.251 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:30 smithi142 ceph-mon[32508]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:56:31.251 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:30 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T18:56:31.252 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:30 smithi142 ceph-mon[32508]: osdmap e206: 8 total, 8 up, 7 in 2024-09-20T18:56:31.252 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:30 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:56:31.316 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:31.352 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:31.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:31.353 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:31.360 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_blocked_ops 2024-09-20T18:56:31.455 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:56:31.489 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:31.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:31.490 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:56:31.591 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:31.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:31.630 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:31.630 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:31.731 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:56:31.732 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:56:31.765 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:31.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:31.766 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:56:31.867 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:31.895 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:31.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:31.897 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:31.998 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:56:32.034 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:32.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:32.036 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:56:32.136 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:32.160 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:31 smithi142 ceph-mon[32508]: pgmap v323: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 144 active+clean; 6.3 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 656 op/s; 13683/309273 objects misplaced (4.424%); 23 MiB/s, 363 objects/s recovering 2024-09-20T18:56:32.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:32.173 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:32.173 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:32.274 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:56:32.296 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T18:56:32.296 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:56:32.301 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:31 smithi083 ceph-mon[29421]: pgmap v323: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 144 active+clean; 6.3 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 656 op/s; 13683/309273 objects misplaced (4.424%); 23 MiB/s, 363 objects/s recovering 2024-09-20T18:56:32.301 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:31 smithi083 ceph-mon[36260]: pgmap v323: 148 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 144 active+clean; 6.3 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 656 op/s; 13683/309273 objects misplaced (4.424%); 23 MiB/s, 363 objects/s recovering 2024-09-20T18:56:32.312 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:32.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:32.313 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:56:32.414 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:32.446 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:32.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:32.447 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:32.548 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:32.578 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:32.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:32.579 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:32.680 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:56:32.719 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:32.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:32.721 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:56:32.821 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:56:32.859 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:32.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:32.860 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:56:32.960 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:32.976 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:32 smithi142 ceph-mon[32508]: Health check failed: Degraded data redundancy: 3 pgs undersized (PG_DEGRADED) 2024-09-20T18:56:32.991 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:32.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:32.992 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:33.023 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_historic_ops 2024-09-20T18:56:33.050 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:32 smithi083 ceph-mon[29421]: Health check failed: Degraded data redundancy: 3 pgs undersized (PG_DEGRADED) 2024-09-20T18:56:33.050 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:32 smithi083 ceph-mon[36260]: Health check failed: Degraded data redundancy: 3 pgs undersized (PG_DEGRADED) 2024-09-20T18:56:33.093 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:56:33.132 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:33.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:33.133 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:56:33.235 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:33.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:33.269 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:33.269 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:33.341 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-09-20T18:56:33.341 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T18:56:33.371 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:33.398 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:33.398 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:56:33.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:33.399 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:33.441 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:56:33.500 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:33.525 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:33.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:33.526 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:33.626 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:56:33.673 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:33.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:33.675 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:56:33.776 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:33.801 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:33.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:33.802 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:33.903 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:56:33.946 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:33.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:33.947 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:56:33.962 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T18:56:33.962 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:56:34.048 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:34.071 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:34.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:34.072 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:34.128 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:56:34.151 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:33 smithi083 ceph-mon[29421]: pgmap v324: 148 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 144 active+clean; 6.0 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.07k op/s; 148/297906 objects degraded (0.050%); 12543/297906 objects misplaced (4.210%); 26 MiB/s, 420 objects/s recovering 2024-09-20T18:56:34.151 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:56:34.151 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:56:34.152 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:33 smithi083 ceph-mon[36260]: pgmap v324: 148 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 144 active+clean; 6.0 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.07k op/s; 148/297906 objects degraded (0.050%); 12543/297906 objects misplaced (4.210%); 26 MiB/s, 420 objects/s recovering 2024-09-20T18:56:34.173 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:56:34.194 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:56:34.194 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:56:34.222 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:34.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:34.223 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:56:34.229 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:56:34.229 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:56:34.270 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:33 smithi142 ceph-mon[32508]: pgmap v324: 148 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 144 active+clean; 6.0 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.07k op/s; 148/297906 objects degraded (0.050%); 12543/297906 objects misplaced (4.210%); 26 MiB/s, 420 objects/s recovering 2024-09-20T18:56:34.270 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:56:34.271 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:56:34.303 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:56:34.303 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:56:34.324 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:34.340 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:56:34.340 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:56:34.353 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:34.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:34.354 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:34.373 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:56:34.374 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:56:34.406 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:56:34.407 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:56:34.455 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:34.483 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:34.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:34.484 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:34.585 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:34.606 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:34.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:34.607 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:34.708 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:56:34.748 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:34.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:34.749 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:56:34.849 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:34.876 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:34.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:34.878 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:34.979 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-20T18:56:34.979 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:56:35.016 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:35.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:35.017 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:56:35.118 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:35.139 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:35.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:35.140 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:35.215 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T18:56:35.240 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:56:35.263 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:34 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3724256088' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:56:35.263 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:34 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3724256088' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:56:35.270 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:34 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3724256088' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:56:35.293 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:56:35.301 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:35.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:35.302 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:56:35.402 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:35.430 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:35.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:35.431 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:35.532 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:35.570 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:35.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:35.571 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:35.672 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:35.702 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:35.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:35.704 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:35.805 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:35.837 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:35.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:35.838 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:35.939 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:56:35.979 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:35.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:35.980 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:56:36.081 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:56:36.115 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:36.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:36.116 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:56:36.217 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:56:36.256 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:36.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:36.257 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:56:36.282 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T18:56:36.282 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-09-20T18:56:36.282 INFO:teuthology.orchestra.run.smithi142.stdout: "num_ops": 0 2024-09-20T18:56:36.283 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T18:56:36.357 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:36.389 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:36.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:36.390 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:36.491 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:56:36.527 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:36.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:36.528 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:56:36.628 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:56:36.668 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:36.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:36.669 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:56:36.770 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:56:36.801 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:36.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:36.802 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:56:36.903 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:56:36.926 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:36.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:36.927 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:56:37.020 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:56:36 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:56:37.020 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:36 smithi142 ceph-mon[32508]: pgmap v325: 148 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 144 active+clean; 6.0 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 542 op/s; 148/297906 objects degraded (0.050%); 12543/297906 objects misplaced (4.210%); 8.5 MiB/s, 135 objects/s recovering 2024-09-20T18:56:37.020 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:36 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:56:37.020 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:36 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:56:37.028 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:37.050 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:36 smithi083 ceph-mon[29421]: pgmap v325: 148 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 144 active+clean; 6.0 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 542 op/s; 148/297906 objects degraded (0.050%); 12543/297906 objects misplaced (4.210%); 8.5 MiB/s, 135 objects/s recovering 2024-09-20T18:56:37.050 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:36 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:56:37.050 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:36 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:56:37.051 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:36 smithi083 ceph-mon[36260]: pgmap v325: 148 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 144 active+clean; 6.0 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 542 op/s; 148/297906 objects degraded (0.050%); 12543/297906 objects misplaced (4.210%); 8.5 MiB/s, 135 objects/s recovering 2024-09-20T18:56:37.051 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:36 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:56:37.051 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:36 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:56:37.055 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:37.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:37.056 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:37.157 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:37.166 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_blocked_ops 2024-09-20T18:56:37.182 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:37.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:37.183 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:37.284 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:37.316 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:37.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:37.317 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:37.418 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:37.449 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:37.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:37.450 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:37.545 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:56:37.551 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:56:37.588 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:37.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:37.589 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:56:37.690 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:37.715 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:37.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:37.716 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:37.817 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:56:37.853 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:37.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:37.854 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:56:37.955 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:37.979 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:37 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:56:37.979 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:37 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:56:37.979 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:37 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:56:37.979 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:37 smithi142 ceph-mon[32508]: osdmap e207: 8 total, 8 up, 7 in 2024-09-20T18:56:37.979 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:37 smithi142 ceph-mon[32508]: pgmap v327: 148 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 144 active+clean; 5.8 GiB data, 18 GiB used, 607 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.09k op/s; 354/284766 objects degraded (0.124%); 11200/284766 objects misplaced (3.933%); 16 MiB/s, 262 objects/s recovering 2024-09-20T18:56:37.979 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:37 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:56:37.979 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:37 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:56:37.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:37.983 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:37.984 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:38.084 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:38.115 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:38.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:38.117 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:38.119 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T18:56:38.119 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-09-20T18:56:38.119 INFO:teuthology.orchestra.run.smithi142.stdout: "complaint_time": 30, 2024-09-20T18:56:38.119 INFO:teuthology.orchestra.run.smithi142.stdout: "num_blocked_ops": 0 2024-09-20T18:56:38.120 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T18:56:38.217 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:38.242 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:38.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:38.242 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:38.300 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:37 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:56:38.301 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:37 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:56:38.301 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:37 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:56:38.301 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:37 smithi083 ceph-mon[29421]: osdmap e207: 8 total, 8 up, 7 in 2024-09-20T18:56:38.301 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:37 smithi083 ceph-mon[29421]: pgmap v327: 148 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 144 active+clean; 5.8 GiB data, 18 GiB used, 607 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.09k op/s; 354/284766 objects degraded (0.124%); 11200/284766 objects misplaced (3.933%); 16 MiB/s, 262 objects/s recovering 2024-09-20T18:56:38.301 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:37 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:56:38.301 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:37 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:56:38.301 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:37 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:56:38.301 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:37 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:56:38.301 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:37 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:56:38.301 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:37 smithi083 ceph-mon[36260]: osdmap e207: 8 total, 8 up, 7 in 2024-09-20T18:56:38.302 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:37 smithi083 ceph-mon[36260]: pgmap v327: 148 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 144 active+clean; 5.8 GiB data, 18 GiB used, 607 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.09k op/s; 354/284766 objects degraded (0.124%); 11200/284766 objects misplaced (3.933%); 16 MiB/s, 262 objects/s recovering 2024-09-20T18:56:38.302 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:37 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:56:38.302 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:37 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:56:38.343 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:56:38.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:38.382 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:38.382 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:56:38.403 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:56:38.410 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-20T18:56:38.483 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:38.521 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:38.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:38.522 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:38.623 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:38.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:38.653 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:38.653 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:38.754 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:38.791 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:38.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:38.792 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:38.814 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_historic_ops 2024-09-20T18:56:38.893 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:38.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:38.925 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:38.925 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:39.020 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:38 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 354/284766 objects degraded (0.124%), 1 pg degraded, 3 pgs undersized (PG_DEGRADED) 2024-09-20T18:56:39.020 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:38 smithi142 ceph-mon[32508]: osdmap e208: 8 total, 8 up, 7 in 2024-09-20T18:56:39.020 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:38 smithi142 ceph-mon[32508]: from='client.25238 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:56:39.027 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:39.029 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:38 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 354/284766 objects degraded (0.124%), 1 pg degraded, 3 pgs undersized (PG_DEGRADED) 2024-09-20T18:56:39.030 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:38 smithi083 ceph-mon[29421]: osdmap e208: 8 total, 8 up, 7 in 2024-09-20T18:56:39.030 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:38 smithi083 ceph-mon[29421]: from='client.25238 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:56:39.035 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:38 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 354/284766 objects degraded (0.124%), 1 pg degraded, 3 pgs undersized (PG_DEGRADED) 2024-09-20T18:56:39.035 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:38 smithi083 ceph-mon[36260]: osdmap e208: 8 total, 8 up, 7 in 2024-09-20T18:56:39.035 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:38 smithi083 ceph-mon[36260]: from='client.25238 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:56:39.062 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:39.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:39.063 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:39.164 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:56:39.168 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:56:39.205 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:39.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:39.206 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:56:39.307 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:39.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:39.354 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:39.354 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:39.455 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:39.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:39.495 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:39.496 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:39.533 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":328,"stamp":"2024-09-20T18:56:37.879084+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6208297523,"num_objects":94922,"num_object_clones":0,"num_object_copies":284766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":354,"num_objects_misplaced":11200,"num_objects_unfound":0,"num_objects_dirty":94922,"num_whiteouts":0,"num_read":1251,"num_read_kb":1108,"num_write":138004,"num_write_kb":7431306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16849,"num_bytes_recovered":1099172809,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74844,"ondisk_log_size":74844,"up":444,"acting":441,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":19332796,"kb_used_data":17537292,"kb_used_omap":10,"kb_used_meta":1795253,"kb_avail":636883268,"statfs":{"total":671965249536,"available":652168466432,"internally_reserved":0,"allocated":17958187008,"data_stored":17953791307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11158,"internal_metadata":1838339178},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,4,3,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-571342848,"num_objects":-8718,"num_object_clones":0,"num_object_copies":-26154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":354,"num_objects_misplaced":-2712,"num_objects_unfound":0,"num_objects_dirty":-8718,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":8718,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2100,"num_bytes_recovered":137625600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.041865"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":331,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.866961+0000","last_change":"2024-09-20T18:55:28.037494+0000","last_active":"2024-09-20T18:56:29.866961+0000","last_peered":"2024-09-20T18:56:29.866961+0000","last_clean":"2024-09-20T18:56:29.866961+0000","last_became_active":"2024-09-20T18:49:45.943201+0000","last_became_peered":"2024-09-20T18:49:45.943201+0000","last_unstale":"2024-09-20T18:56:29.866961+0000","last_undegraded":"2024-09-20T18:56:29.866961+0000","last_fullsized":"2024-09-20T18:56:29.866961+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:28.037449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:28.037449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:53.606960+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":377,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.841264+0000","last_change":"2024-09-20T18:55:28.838729+0000","last_active":"2024-09-20T18:56:29.841264+0000","last_peered":"2024-09-20T18:56:29.841264+0000","last_clean":"2024-09-20T18:56:29.841264+0000","last_became_active":"2024-09-20T18:55:28.819637+0000","last_became_peered":"2024-09-20T18:55:28.819637+0000","last_unstale":"2024-09-20T18:56:29.841264+0000","last_undegraded":"2024-09-20T18:56:29.841264+0000","last_fullsized":"2024-09-20T18:56:29.841264+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:50.231620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:50.231620+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":323,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.847371+0000","last_change":"2024-09-20T18:54:25.361988+0000","last_active":"2024-09-20T18:56:29.847371+0000","last_peered":"2024-09-20T18:56:29.847371+0000","last_clean":"2024-09-20T18:56:29.847371+0000","last_became_active":"2024-09-20T18:49:49.959514+0000","last_became_peered":"2024-09-20T18:49:49.959514+0000","last_unstale":"2024-09-20T18:56:29.847371+0000","last_undegraded":"2024-09-20T18:56:29.847371+0000","last_fullsized":"2024-09-20T18:56:29.847371+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:25.361935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:25.361935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":319,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.841796+0000","last_change":"2024-09-20T18:54:15.221404+0000","last_active":"2024-09-20T18:56:29.841796+0000","last_peered":"2024-09-20T18:56:29.841796+0000","last_clean":"2024-09-20T18:56:29.841796+0000","last_became_active":"2024-09-20T18:49:51.969421+0000","last_became_peered":"2024-09-20T18:49:51.969421+0000","last_unstale":"2024-09-20T18:56:29.841796+0000","last_undegraded":"2024-09-20T18:56:29.841796+0000","last_fullsized":"2024-09-20T18:56:29.841796+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:15.221342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:15.221342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":326,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.853237+0000","last_change":"2024-09-20T18:55:28.822034+0000","last_active":"2024-09-20T18:56:29.853237+0000","last_peered":"2024-09-20T18:56:29.853237+0000","last_clean":"2024-09-20T18:56:29.853237+0000","last_became_active":"2024-09-20T18:55:28.821717+0000","last_became_peered":"2024-09-20T18:55:28.821717+0000","last_unstale":"2024-09-20T18:56:29.853237+0000","last_undegraded":"2024-09-20T18:56:29.853237+0000","last_fullsized":"2024-09-20T18:56:29.853237+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.216188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.216188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":323,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.853823+0000","last_change":"2024-09-20T18:55:28.825144+0000","last_active":"2024-09-20T18:56:29.853823+0000","last_peered":"2024-09-20T18:56:29.853823+0000","last_clean":"2024-09-20T18:56:29.853823+0000","last_became_active":"2024-09-20T18:55:28.824826+0000","last_became_peered":"2024-09-20T18:55:28.824826+0000","last_unstale":"2024-09-20T18:56:29.853823+0000","last_undegraded":"2024-09-20T18:56:29.853823+0000","last_fullsized":"2024-09-20T18:56:29.853823+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:25.208073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:25.208073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":336,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.856493+0000","last_change":"2024-09-20T18:51:07.130137+0000","last_active":"2024-09-20T18:56:29.856493+0000","last_peered":"2024-09-20T18:56:29.856493+0000","last_clean":"2024-09-20T18:56:29.856493+0000","last_became_active":"2024-09-20T18:49:47.952150+0000","last_became_peered":"2024-09-20T18:49:47.952150+0000","last_unstale":"2024-09-20T18:56:29.856493+0000","last_undegraded":"2024-09-20T18:56:29.856493+0000","last_fullsized":"2024-09-20T18:56:29.856493+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:51:07.130062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:07.130062+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":322,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.841174+0000","last_change":"2024-09-20T18:55:28.820753+0000","last_active":"2024-09-20T18:56:29.841174+0000","last_peered":"2024-09-20T18:56:29.841174+0000","last_clean":"2024-09-20T18:56:29.841174+0000","last_became_active":"2024-09-20T18:55:28.820362+0000","last_became_peered":"2024-09-20T18:55:28.820362+0000","last_unstale":"2024-09-20T18:56:29.841174+0000","last_undegraded":"2024-09-20T18:56:29.841174+0000","last_fullsized":"2024-09-20T18:56:29.841174+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:11.246922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:11.246922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"59'1","reported_seq":355,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.852615+0000","last_change":"2024-09-20T18:55:29.340140+0000","last_active":"2024-09-20T18:56:29.852615+0000","last_peered":"2024-09-20T18:56:29.852615+0000","last_clean":"2024-09-20T18:56:29.852615+0000","last_became_active":"2024-09-20T18:55:28.833386+0000","last_became_peered":"2024-09-20T18:55:28.833386+0000","last_unstale":"2024-09-20T18:56:29.852615+0000","last_undegraded":"2024-09-20T18:56:29.852615+0000","last_fullsized":"2024-09-20T18:56:29.852615+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:55:29.340083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:29.340083+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:41.552449+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":343,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.841956+0000","last_change":"2024-09-20T18:54:53.241889+0000","last_active":"2024-09-20T18:56:29.841956+0000","last_peered":"2024-09-20T18:56:29.841956+0000","last_clean":"2024-09-20T18:56:29.841956+0000","last_became_active":"2024-09-20T18:49:47.947448+0000","last_became_peered":"2024-09-20T18:49:47.947448+0000","last_unstale":"2024-09-20T18:56:29.841956+0000","last_undegraded":"2024-09-20T18:56:29.841956+0000","last_fullsized":"2024-09-20T18:56:29.841956+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:53.241802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:53.241802+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":314,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.852655+0000","last_change":"2024-09-20T18:52:36.183932+0000","last_active":"2024-09-20T18:56:29.852655+0000","last_peered":"2024-09-20T18:56:29.852655+0000","last_clean":"2024-09-20T18:56:29.852655+0000","last_became_active":"2024-09-20T18:49:49.961188+0000","last_became_peered":"2024-09-20T18:49:49.961188+0000","last_unstale":"2024-09-20T18:56:29.852655+0000","last_undegraded":"2024-09-20T18:56:29.852655+0000","last_fullsized":"2024-09-20T18:56:29.852655+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:36.183875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:36.183875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":294,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.865546+0000","last_change":"2024-09-20T18:49:51.966957+0000","last_active":"2024-09-20T18:56:29.865546+0000","last_peered":"2024-09-20T18:56:29.865546+0000","last_clean":"2024-09-20T18:56:29.865546+0000","last_became_active":"2024-09-20T18:49:51.966715+0000","last_became_peered":"2024-09-20T18:49:51.966715+0000","last_unstale":"2024-09-20T18:56:29.865546+0000","last_undegraded":"2024-09-20T18:56:29.865546+0000","last_fullsized":"2024-09-20T18:56:29.865546+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":322,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.851436+0000","last_change":"2024-09-20T18:55:15.252839+0000","last_active":"2024-09-20T18:56:29.851436+0000","last_peered":"2024-09-20T18:56:29.851436+0000","last_clean":"2024-09-20T18:56:29.851436+0000","last_became_active":"2024-09-20T18:49:45.944703+0000","last_became_peered":"2024-09-20T18:49:45.944703+0000","last_unstale":"2024-09-20T18:56:29.851436+0000","last_undegraded":"2024-09-20T18:56:29.851436+0000","last_fullsized":"2024-09-20T18:56:29.851436+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:15.252782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:15.252782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:43.639912+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":347,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.841800+0000","last_change":"2024-09-20T18:51:26.494194+0000","last_active":"2024-09-20T18:56:29.841800+0000","last_peered":"2024-09-20T18:56:29.841800+0000","last_clean":"2024-09-20T18:56:29.841800+0000","last_became_active":"2024-09-20T18:51:23.813399+0000","last_became_peered":"2024-09-20T18:51:23.813399+0000","last_unstale":"2024-09-20T18:56:29.841800+0000","last_undegraded":"2024-09-20T18:56:29.841800+0000","last_fullsized":"2024-09-20T18:56:29.841800+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-20T18:51:26.494118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:26.494118+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":314,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.844614+0000","last_change":"2024-09-20T18:52:38.799542+0000","last_active":"2024-09-20T18:56:29.844614+0000","last_peered":"2024-09-20T18:56:29.844614+0000","last_clean":"2024-09-20T18:56:29.844614+0000","last_became_active":"2024-09-20T18:49:49.959696+0000","last_became_peered":"2024-09-20T18:49:49.959696+0000","last_unstale":"2024-09-20T18:56:29.844614+0000","last_undegraded":"2024-09-20T18:56:29.844614+0000","last_fullsized":"2024-09-20T18:56:29.844614+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:38.799487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:38.799487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":310,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.853082+0000","last_change":"2024-09-20T18:53:21.273835+0000","last_active":"2024-09-20T18:56:29.853082+0000","last_peered":"2024-09-20T18:56:29.853082+0000","last_clean":"2024-09-20T18:56:29.853082+0000","last_became_active":"2024-09-20T18:49:51.970663+0000","last_became_peered":"2024-09-20T18:49:51.970663+0000","last_unstale":"2024-09-20T18:56:29.853082+0000","last_undegraded":"2024-09-20T18:56:29.853082+0000","last_fullsized":"2024-09-20T18:56:29.853082+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:21.273788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:21.273788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":319,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.842301+0000","last_change":"2024-09-20T18:55:28.828417+0000","last_active":"2024-09-20T18:56:29.842301+0000","last_peered":"2024-09-20T18:56:29.842301+0000","last_clean":"2024-09-20T18:56:29.842301+0000","last_became_active":"2024-09-20T18:55:28.827435+0000","last_became_peered":"2024-09-20T18:55:28.827435+0000","last_unstale":"2024-09-20T18:56:29.842301+0000","last_undegraded":"2024-09-20T18:56:29.842301+0000","last_fullsized":"2024-09-20T18:56:29.842301+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:53.940191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:53.940191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":322,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.866624+0000","last_change":"2024-09-20T18:54:23.984024+0000","last_active":"2024-09-20T18:56:29.866624+0000","last_peered":"2024-09-20T18:56:29.866624+0000","last_clean":"2024-09-20T18:56:29.866624+0000","last_became_active":"2024-09-20T18:49:45.950480+0000","last_became_peered":"2024-09-20T18:49:45.950480+0000","last_unstale":"2024-09-20T18:56:29.866624+0000","last_undegraded":"2024-09-20T18:56:29.866624+0000","last_fullsized":"2024-09-20T18:56:29.866624+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:23.983958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:23.983958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":345,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.845546+0000","last_change":"2024-09-20T18:53:26.762207+0000","last_active":"2024-09-20T18:56:29.845546+0000","last_peered":"2024-09-20T18:56:29.845546+0000","last_clean":"2024-09-20T18:56:29.845546+0000","last_became_active":"2024-09-20T18:49:47.950939+0000","last_became_peered":"2024-09-20T18:49:47.950939+0000","last_unstale":"2024-09-20T18:56:29.845546+0000","last_undegraded":"2024-09-20T18:56:29.845546+0000","last_fullsized":"2024-09-20T18:56:29.845546+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:53:26.762131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:26.762131+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":307,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.847873+0000","last_change":"2024-09-20T18:51:17.248705+0000","last_active":"2024-09-20T18:56:29.847873+0000","last_peered":"2024-09-20T18:56:29.847873+0000","last_clean":"2024-09-20T18:56:29.847873+0000","last_became_active":"2024-09-20T18:49:49.958518+0000","last_became_peered":"2024-09-20T18:49:49.958518+0000","last_unstale":"2024-09-20T18:56:29.847873+0000","last_undegraded":"2024-09-20T18:56:29.847873+0000","last_fullsized":"2024-09-20T18:56:29.847873+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:17.248634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:17.248634+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":319,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.863702+0000","last_change":"2024-09-20T18:55:28.833468+0000","last_active":"2024-09-20T18:56:29.863702+0000","last_peered":"2024-09-20T18:56:29.863702+0000","last_clean":"2024-09-20T18:56:29.863702+0000","last_became_active":"2024-09-20T18:55:28.833133+0000","last_became_peered":"2024-09-20T18:55:28.833133+0000","last_unstale":"2024-09-20T18:56:29.863702+0000","last_undegraded":"2024-09-20T18:56:29.863702+0000","last_fullsized":"2024-09-20T18:56:29.863702+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:54.259174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:54.259174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"59'1","reported_seq":329,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.842001+0000","last_change":"2024-09-20T18:53:42.840884+0000","last_active":"2024-09-20T18:56:29.842001+0000","last_peered":"2024-09-20T18:56:29.842001+0000","last_clean":"2024-09-20T18:56:29.842001+0000","last_became_active":"2024-09-20T18:49:45.940100+0000","last_became_peered":"2024-09-20T18:49:45.940100+0000","last_unstale":"2024-09-20T18:56:29.842001+0000","last_undegraded":"2024-09-20T18:56:29.842001+0000","last_fullsized":"2024-09-20T18:56:29.842001+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:53:42.840800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:42.840800+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":326,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.852974+0000","last_change":"2024-09-20T18:49:47.951215+0000","last_active":"2024-09-20T18:56:29.852974+0000","last_peered":"2024-09-20T18:56:29.852974+0000","last_clean":"2024-09-20T18:56:29.852974+0000","last_became_active":"2024-09-20T18:49:47.950909+0000","last_became_peered":"2024-09-20T18:49:47.950909+0000","last_unstale":"2024-09-20T18:56:29.852974+0000","last_undegraded":"2024-09-20T18:56:29.852974+0000","last_fullsized":"2024-09-20T18:56:29.852974+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":323,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.841934+0000","last_change":"2024-09-20T18:54:44.239273+0000","last_active":"2024-09-20T18:56:29.841934+0000","last_peered":"2024-09-20T18:56:29.841934+0000","last_clean":"2024-09-20T18:56:29.841934+0000","last_became_active":"2024-09-20T18:49:49.959102+0000","last_became_peered":"2024-09-20T18:49:49.959102+0000","last_unstale":"2024-09-20T18:56:29.841934+0000","last_undegraded":"2024-09-20T18:56:29.841934+0000","last_fullsized":"2024-09-20T18:56:29.841934+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:44.239226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.239226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":311,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.845079+0000","last_change":"2024-09-20T18:52:15.826819+0000","last_active":"2024-09-20T18:56:29.845079+0000","last_peered":"2024-09-20T18:56:29.845079+0000","last_clean":"2024-09-20T18:56:29.845079+0000","last_became_active":"2024-09-20T18:49:51.970058+0000","last_became_peered":"2024-09-20T18:49:51.970058+0000","last_unstale":"2024-09-20T18:56:29.845079+0000","last_undegraded":"2024-09-20T18:56:29.845079+0000","last_fullsized":"2024-09-20T18:56:29.845079+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T18:52:15.826638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:15.826638+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":316,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.847232+0000","last_change":"2024-09-20T18:52:47.158384+0000","last_active":"2024-09-20T18:56:29.847232+0000","last_peered":"2024-09-20T18:56:29.847232+0000","last_clean":"2024-09-20T18:56:29.847232+0000","last_became_active":"2024-09-20T18:49:45.933078+0000","last_became_peered":"2024-09-20T18:49:45.933078+0000","last_unstale":"2024-09-20T18:56:29.847232+0000","last_undegraded":"2024-09-20T18:56:29.847232+0000","last_fullsized":"2024-09-20T18:56:29.847232+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:52:47.158260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:47.158260+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":342,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.880272+0000","last_change":"2024-09-20T18:55:19.996522+0000","last_active":"2024-09-20T18:56:29.880272+0000","last_peered":"2024-09-20T18:56:29.880272+0000","last_clean":"2024-09-20T18:56:29.880272+0000","last_became_active":"2024-09-20T18:49:48.500809+0000","last_became_peered":"2024-09-20T18:49:48.500809+0000","last_unstale":"2024-09-20T18:56:29.880272+0000","last_undegraded":"2024-09-20T18:56:29.880272+0000","last_fullsized":"2024-09-20T18:56:29.880272+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:55:19.996328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:19.996328+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:35.334334+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"184'9","reported_seq":107,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.842287+0000","last_change":"2024-09-20T18:55:28.845188+0000","last_active":"2024-09-20T18:56:29.842287+0000","last_peered":"2024-09-20T18:56:29.842287+0000","last_clean":"2024-09-20T18:56:29.842287+0000","last_became_active":"2024-09-20T18:55:28.820941+0000","last_became_peered":"2024-09-20T18:55:28.820941+0000","last_unstale":"2024-09-20T18:56:29.842287+0000","last_undegraded":"2024-09-20T18:56:29.842287+0000","last_fullsized":"2024-09-20T18:56:29.842287+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:53:44.524040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:44.524040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3","version":"206'8386","reported_seq":20821,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:36.221118+0000","last_change":"2024-09-20T18:56:25.697520+0000","last_active":"2024-09-20T18:56:36.221118+0000","last_peered":"2024-09-20T18:56:36.221118+0000","last_clean":"2024-09-20T18:56:36.221118+0000","last_became_active":"2024-09-20T18:56:25.687391+0000","last_became_peered":"2024-09-20T18:56:25.687391+0000","last_unstale":"2024-09-20T18:56:36.221118+0000","last_undegraded":"2024-09-20T18:56:36.221118+0000","last_fullsized":"2024-09-20T18:56:36.221118+0000","mapping_epoch":203,"log_start":"174'4072","ondisk_log_start":"174'4072","created":160,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4314,"log_dups_size":0,"ondisk_log_size":4314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":387317760,"num_objects":5910,"num_object_clones":0,"num_object_copies":17730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5910,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8386,"num_write_kb":457472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6156,"num_bytes_recovered":403439616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":336,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.847324+0000","last_change":"2024-09-20T18:55:18.581290+0000","last_active":"2024-09-20T18:56:29.847324+0000","last_peered":"2024-09-20T18:56:29.847324+0000","last_clean":"2024-09-20T18:56:29.847324+0000","last_became_active":"2024-09-20T18:49:58.992302+0000","last_became_peered":"2024-09-20T18:49:58.992302+0000","last_unstale":"2024-09-20T18:56:29.847324+0000","last_undegraded":"2024-09-20T18:56:29.847324+0000","last_fullsized":"2024-09-20T18:56:29.847324+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:55:18.581200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:18.581200+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:33.641712+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":324,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.858388+0000","last_change":"2024-09-20T18:54:54.288199+0000","last_active":"2024-09-20T18:56:29.858388+0000","last_peered":"2024-09-20T18:56:29.858388+0000","last_clean":"2024-09-20T18:56:29.858388+0000","last_became_active":"2024-09-20T18:49:45.936104+0000","last_became_peered":"2024-09-20T18:49:45.936104+0000","last_unstale":"2024-09-20T18:56:29.858388+0000","last_undegraded":"2024-09-20T18:56:29.858388+0000","last_fullsized":"2024-09-20T18:56:29.858388+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:54.288117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:54.288117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":46,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":409,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:31.072297+0000","last_change":"2024-09-20T18:54:14.292660+0000","last_active":"2024-09-20T18:56:31.072297+0000","last_peered":"2024-09-20T18:56:31.072297+0000","last_clean":"2024-09-20T18:56:31.072297+0000","last_became_active":"2024-09-20T18:49:49.960697+0000","last_became_peered":"2024-09-20T18:49:49.960697+0000","last_unstale":"2024-09-20T18:56:31.072297+0000","last_undegraded":"2024-09-20T18:56:31.072297+0000","last_fullsized":"2024-09-20T18:56:31.072297+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:54:14.292610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:14.292610+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":34,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":311,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.851726+0000","last_change":"2024-09-20T18:54:10.381171+0000","last_active":"2024-09-20T18:56:29.851726+0000","last_peered":"2024-09-20T18:56:29.851726+0000","last_clean":"2024-09-20T18:56:29.851726+0000","last_became_active":"2024-09-20T18:49:51.971399+0000","last_became_peered":"2024-09-20T18:49:51.971399+0000","last_unstale":"2024-09-20T18:56:29.851726+0000","last_undegraded":"2024-09-20T18:56:29.851726+0000","last_fullsized":"2024-09-20T18:56:29.851726+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:10.381108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:10.381108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.c","version":"206'8513","reported_seq":8616,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:36.587859+0000","last_change":"2024-09-20T18:54:34.311256+0000","last_active":"2024-09-20T18:56:36.587859+0000","last_peered":"2024-09-20T18:56:36.587859+0000","last_clean":"2024-09-20T18:56:36.587859+0000","last_became_active":"2024-09-20T18:54:34.310904+0000","last_became_peered":"2024-09-20T18:54:34.310904+0000","last_unstale":"2024-09-20T18:56:36.587859+0000","last_undegraded":"2024-09-20T18:56:36.587859+0000","last_fullsized":"2024-09-20T18:56:36.587859+0000","mapping_epoch":160,"log_start":"172'3611","ondisk_log_start":"172'3611","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4902,"log_dups_size":0,"ondisk_log_size":4902,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":385286144,"num_objects":5879,"num_object_clones":0,"num_object_copies":17637,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5879,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8513,"num_write_kb":460544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":352,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.858171+0000","last_change":"2024-09-20T18:54:22.077299+0000","last_active":"2024-09-20T18:56:29.858171+0000","last_peered":"2024-09-20T18:56:29.858171+0000","last_clean":"2024-09-20T18:56:29.858171+0000","last_became_active":"2024-09-20T18:49:48.499189+0000","last_became_peered":"2024-09-20T18:49:48.499189+0000","last_unstale":"2024-09-20T18:56:29.858171+0000","last_undegraded":"2024-09-20T18:56:29.858171+0000","last_fullsized":"2024-09-20T18:56:29.858171+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-20T18:54:22.077212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:22.077212+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":316,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.864264+0000","last_change":"2024-09-20T18:54:57.237740+0000","last_active":"2024-09-20T18:56:29.864264+0000","last_peered":"2024-09-20T18:56:29.864264+0000","last_clean":"2024-09-20T18:56:29.864264+0000","last_became_active":"2024-09-20T18:49:45.940544+0000","last_became_peered":"2024-09-20T18:49:45.940544+0000","last_unstale":"2024-09-20T18:56:29.864264+0000","last_undegraded":"2024-09-20T18:56:29.864264+0000","last_fullsized":"2024-09-20T18:56:29.864264+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.237675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.237675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":324,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.841117+0000","last_change":"2024-09-20T18:54:51.229251+0000","last_active":"2024-09-20T18:56:29.841117+0000","last_peered":"2024-09-20T18:56:29.841117+0000","last_clean":"2024-09-20T18:56:29.841117+0000","last_became_active":"2024-09-20T18:49:49.963073+0000","last_became_peered":"2024-09-20T18:49:49.963073+0000","last_unstale":"2024-09-20T18:56:29.841117+0000","last_undegraded":"2024-09-20T18:56:29.841117+0000","last_fullsized":"2024-09-20T18:56:29.841117+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.229201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.229201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":320,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.841905+0000","last_change":"2024-09-20T18:54:55.323937+0000","last_active":"2024-09-20T18:56:29.841905+0000","last_peered":"2024-09-20T18:56:29.841905+0000","last_clean":"2024-09-20T18:56:29.841905+0000","last_became_active":"2024-09-20T18:49:51.968793+0000","last_became_peered":"2024-09-20T18:49:51.968793+0000","last_unstale":"2024-09-20T18:56:29.841905+0000","last_undegraded":"2024-09-20T18:56:29.841905+0000","last_fullsized":"2024-09-20T18:56:29.841905+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:55.323692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:55.323692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"206'8488","reported_seq":8592,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:35.821634+0000","last_change":"2024-09-20T18:54:34.311680+0000","last_active":"2024-09-20T18:56:35.821634+0000","last_peered":"2024-09-20T18:56:35.821634+0000","last_clean":"2024-09-20T18:56:35.821634+0000","last_became_active":"2024-09-20T18:54:34.311241+0000","last_became_peered":"2024-09-20T18:54:34.311241+0000","last_unstale":"2024-09-20T18:56:35.821634+0000","last_undegraded":"2024-09-20T18:56:35.821634+0000","last_fullsized":"2024-09-20T18:56:35.821634+0000","mapping_epoch":160,"log_start":"171'3446","ondisk_log_start":"171'3446","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":5042,"log_dups_size":0,"ondisk_log_size":5042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":385548312,"num_objects":5884,"num_object_clones":0,"num_object_copies":17652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5884,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8488,"num_write_kb":459841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":380,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.841228+0000","last_change":"2024-09-20T18:55:28.853375+0000","last_active":"2024-09-20T18:56:29.841228+0000","last_peered":"2024-09-20T18:56:29.841228+0000","last_clean":"2024-09-20T18:56:29.841228+0000","last_became_active":"2024-09-20T18:55:28.831057+0000","last_became_peered":"2024-09-20T18:55:28.831057+0000","last_unstale":"2024-09-20T18:56:29.841228+0000","last_undegraded":"2024-09-20T18:56:29.841228+0000","last_fullsized":"2024-09-20T18:56:29.841228+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-20T18:54:57.394893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.394893+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":316,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.880472+0000","last_change":"2024-09-20T18:51:27.187060+0000","last_active":"2024-09-20T18:56:29.880472+0000","last_peered":"2024-09-20T18:56:29.880472+0000","last_clean":"2024-09-20T18:56:29.880472+0000","last_became_active":"2024-09-20T18:49:45.940469+0000","last_became_peered":"2024-09-20T18:49:45.940469+0000","last_unstale":"2024-09-20T18:56:29.880472+0000","last_undegraded":"2024-09-20T18:56:29.880472+0000","last_fullsized":"2024-09-20T18:56:29.880472+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:51:27.186991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:27.186991+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"66'8","reported_seq":415,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:31.073517+0000","last_change":"2024-09-20T18:54:47.223010+0000","last_active":"2024-09-20T18:56:31.073517+0000","last_peered":"2024-09-20T18:56:31.073517+0000","last_clean":"2024-09-20T18:56:31.073517+0000","last_became_active":"2024-09-20T18:49:49.956133+0000","last_became_peered":"2024-09-20T18:49:49.956133+0000","last_unstale":"2024-09-20T18:56:31.073517+0000","last_undegraded":"2024-09-20T18:56:31.073517+0000","last_fullsized":"2024-09-20T18:56:31.073517+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:54:47.222937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:47.222937+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":318,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.844751+0000","last_change":"2024-09-20T18:55:28.819147+0000","last_active":"2024-09-20T18:56:29.844751+0000","last_peered":"2024-09-20T18:56:29.844751+0000","last_clean":"2024-09-20T18:56:29.844751+0000","last_became_active":"2024-09-20T18:55:28.818804+0000","last_became_peered":"2024-09-20T18:55:28.818804+0000","last_unstale":"2024-09-20T18:56:29.844751+0000","last_undegraded":"2024-09-20T18:56:29.844751+0000","last_fullsized":"2024-09-20T18:56:29.844751+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:42.880724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:42.880724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"206'8526","reported_seq":8627,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:36.581065+0000","last_change":"2024-09-20T18:54:34.309939+0000","last_active":"2024-09-20T18:56:36.581065+0000","last_peered":"2024-09-20T18:56:36.581065+0000","last_clean":"2024-09-20T18:56:36.581065+0000","last_became_active":"2024-09-20T18:54:34.309630+0000","last_became_peered":"2024-09-20T18:54:34.309630+0000","last_unstale":"2024-09-20T18:56:36.581065+0000","last_undegraded":"2024-09-20T18:56:36.581065+0000","last_fullsized":"2024-09-20T18:56:36.581065+0000","mapping_epoch":160,"log_start":"172'3649","ondisk_log_start":"172'3649","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4877,"log_dups_size":0,"ondisk_log_size":4877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":385089536,"num_objects":5876,"num_object_clones":0,"num_object_copies":17628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5876,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8526,"num_write_kb":460864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":343,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.847482+0000","last_change":"2024-09-20T18:55:19.541011+0000","last_active":"2024-09-20T18:56:29.847482+0000","last_peered":"2024-09-20T18:56:29.847482+0000","last_clean":"2024-09-20T18:56:29.847482+0000","last_became_active":"2024-09-20T18:49:47.949626+0000","last_became_peered":"2024-09-20T18:49:47.949626+0000","last_unstale":"2024-09-20T18:56:29.847482+0000","last_undegraded":"2024-09-20T18:56:29.847482+0000","last_fullsized":"2024-09-20T18:56:29.847482+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:55:19.540812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:19.540812+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":331,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.852824+0000","last_change":"2024-09-20T18:54:18.326947+0000","last_active":"2024-09-20T18:56:29.852824+0000","last_peered":"2024-09-20T18:56:29.852824+0000","last_clean":"2024-09-20T18:56:29.852824+0000","last_became_active":"2024-09-20T18:49:45.942496+0000","last_became_peered":"2024-09-20T18:49:45.942496+0000","last_unstale":"2024-09-20T18:56:29.852824+0000","last_undegraded":"2024-09-20T18:56:29.852824+0000","last_fullsized":"2024-09-20T18:56:29.852824+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:18.326875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:18.326875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"184'9","reported_seq":417,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:31.073234+0000","last_change":"2024-09-20T18:55:29.010062+0000","last_active":"2024-09-20T18:56:31.073234+0000","last_peered":"2024-09-20T18:56:31.073234+0000","last_clean":"2024-09-20T18:56:31.073234+0000","last_became_active":"2024-09-20T18:55:28.830643+0000","last_became_peered":"2024-09-20T18:55:28.830643+0000","last_unstale":"2024-09-20T18:56:31.073234+0000","last_undegraded":"2024-09-20T18:56:31.073234+0000","last_fullsized":"2024-09-20T18:56:31.073234+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:29.010015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:29.010015+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":295,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.865665+0000","last_change":"2024-09-20T18:49:51.965733+0000","last_active":"2024-09-20T18:56:29.865665+0000","last_peered":"2024-09-20T18:56:29.865665+0000","last_clean":"2024-09-20T18:56:29.865665+0000","last_became_active":"2024-09-20T18:49:51.964780+0000","last_became_peered":"2024-09-20T18:49:51.964780+0000","last_unstale":"2024-09-20T18:56:29.865665+0000","last_undegraded":"2024-09-20T18:56:29.865665+0000","last_fullsized":"2024-09-20T18:56:29.865665+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"206'8727","reported_seq":8830,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:36.396066+0000","last_change":"2024-09-20T18:54:34.309809+0000","last_active":"2024-09-20T18:56:36.396066+0000","last_peered":"2024-09-20T18:56:36.396066+0000","last_clean":"2024-09-20T18:56:36.396066+0000","last_became_active":"2024-09-20T18:54:34.309231+0000","last_became_peered":"2024-09-20T18:54:34.309231+0000","last_unstale":"2024-09-20T18:56:36.396066+0000","last_undegraded":"2024-09-20T18:56:36.396066+0000","last_fullsized":"2024-09-20T18:56:36.396066+0000","mapping_epoch":160,"log_start":"175'4432","ondisk_log_start":"175'4432","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4295,"log_dups_size":0,"ondisk_log_size":4295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":392364032,"num_objects":5987,"num_object_clones":0,"num_object_copies":17961,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5987,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8727,"num_write_kb":470848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":355,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.866474+0000","last_change":"2024-09-20T18:54:14.923895+0000","last_active":"2024-09-20T18:56:29.866474+0000","last_peered":"2024-09-20T18:56:29.866474+0000","last_clean":"2024-09-20T18:56:29.866474+0000","last_became_active":"2024-09-20T18:49:48.497736+0000","last_became_peered":"2024-09-20T18:49:48.497736+0000","last_unstale":"2024-09-20T18:56:29.866474+0000","last_undegraded":"2024-09-20T18:56:29.866474+0000","last_fullsized":"2024-09-20T18:56:29.866474+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-20T18:54:14.923682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:14.923682+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":308,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.847528+0000","last_change":"2024-09-20T18:49:45.942589+0000","last_active":"2024-09-20T18:56:29.847528+0000","last_peered":"2024-09-20T18:56:29.847528+0000","last_clean":"2024-09-20T18:56:29.847528+0000","last_became_active":"2024-09-20T18:49:45.942253+0000","last_became_peered":"2024-09-20T18:49:45.942253+0000","last_unstale":"2024-09-20T18:56:29.847528+0000","last_undegraded":"2024-09-20T18:56:29.847528+0000","last_fullsized":"2024-09-20T18:56:29.847528+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":61,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.868720+0000","last_change":"2024-09-20T18:55:28.830954+0000","last_active":"2024-09-20T18:56:29.868720+0000","last_peered":"2024-09-20T18:56:29.868720+0000","last_clean":"2024-09-20T18:56:29.868720+0000","last_became_active":"2024-09-20T18:55:28.830645+0000","last_became_peered":"2024-09-20T18:55:28.830645+0000","last_unstale":"2024-09-20T18:56:29.868720+0000","last_undegraded":"2024-09-20T18:56:29.868720+0000","last_fullsized":"2024-09-20T18:56:29.868720+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:50.610871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:50.610871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":327,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.847051+0000","last_change":"2024-09-20T18:55:28.822362+0000","last_active":"2024-09-20T18:56:29.847051+0000","last_peered":"2024-09-20T18:56:29.847051+0000","last_clean":"2024-09-20T18:56:29.847051+0000","last_became_active":"2024-09-20T18:55:28.821588+0000","last_became_peered":"2024-09-20T18:55:28.821588+0000","last_unstale":"2024-09-20T18:56:29.847051+0000","last_undegraded":"2024-09-20T18:56:29.847051+0000","last_fullsized":"2024-09-20T18:56:29.847051+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.402841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.402841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"206'8590","reported_seq":8693,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:35.818124+0000","last_change":"2024-09-20T18:54:34.311561+0000","last_active":"2024-09-20T18:56:35.818124+0000","last_peered":"2024-09-20T18:56:35.818124+0000","last_clean":"2024-09-20T18:56:35.818124+0000","last_became_active":"2024-09-20T18:54:34.311181+0000","last_became_peered":"2024-09-20T18:54:34.311181+0000","last_unstale":"2024-09-20T18:56:35.818124+0000","last_undegraded":"2024-09-20T18:56:35.818124+0000","last_fullsized":"2024-09-20T18:56:35.818124+0000","mapping_epoch":160,"log_start":"171'3494","ondisk_log_start":"171'3494","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":5096,"log_dups_size":0,"ondisk_log_size":5096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393740288,"num_objects":6008,"num_object_clones":0,"num_object_copies":18024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6008,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8590,"num_write_kb":467136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"66'28","reported_seq":451,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:31.074761+0000","last_change":"2024-09-20T18:55:16.527267+0000","last_active":"2024-09-20T18:56:31.074761+0000","last_peered":"2024-09-20T18:56:31.074761+0000","last_clean":"2024-09-20T18:56:31.074761+0000","last_became_active":"2024-09-20T18:49:47.955335+0000","last_became_peered":"2024-09-20T18:49:47.955335+0000","last_unstale":"2024-09-20T18:56:31.074761+0000","last_undegraded":"2024-09-20T18:56:31.074761+0000","last_fullsized":"2024-09-20T18:56:31.074761+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'28","last_scrub_stamp":"2024-09-20T18:55:16.527165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:16.527165+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":315,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.842248+0000","last_change":"2024-09-20T18:55:28.832303+0000","last_active":"2024-09-20T18:56:29.842248+0000","last_peered":"2024-09-20T18:56:29.842248+0000","last_clean":"2024-09-20T18:56:29.842248+0000","last_became_active":"2024-09-20T18:55:28.831574+0000","last_became_peered":"2024-09-20T18:55:28.831574+0000","last_unstale":"2024-09-20T18:56:29.842248+0000","last_undegraded":"2024-09-20T18:56:29.842248+0000","last_fullsized":"2024-09-20T18:56:29.842248+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1],"acting":[4,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":360,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.852853+0000","last_change":"2024-09-20T18:55:28.854615+0000","last_active":"2024-09-20T18:56:29.852853+0000","last_peered":"2024-09-20T18:56:29.852853+0000","last_clean":"2024-09-20T18:56:29.852853+0000","last_became_active":"2024-09-20T18:55:28.837226+0000","last_became_peered":"2024-09-20T18:55:28.837226+0000","last_unstale":"2024-09-20T18:56:29.852853+0000","last_undegraded":"2024-09-20T18:56:29.852853+0000","last_fullsized":"2024-09-20T18:56:29.852853+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-20T18:54:51.227531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.227531+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0012781159999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":323,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.870545+0000","last_change":"2024-09-20T18:55:13.949348+0000","last_active":"2024-09-20T18:56:29.870545+0000","last_peered":"2024-09-20T18:56:29.870545+0000","last_clean":"2024-09-20T18:56:29.870545+0000","last_became_active":"2024-09-20T18:49:49.953971+0000","last_became_peered":"2024-09-20T18:49:49.953971+0000","last_unstale":"2024-09-20T18:56:29.870545+0000","last_undegraded":"2024-09-20T18:56:29.870545+0000","last_fullsized":"2024-09-20T18:56:29.870545+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:13.949269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:13.949269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":296,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.853331+0000","last_change":"2024-09-20T18:49:51.969702+0000","last_active":"2024-09-20T18:56:29.853331+0000","last_peered":"2024-09-20T18:56:29.853331+0000","last_clean":"2024-09-20T18:56:29.853331+0000","last_became_active":"2024-09-20T18:49:51.969400+0000","last_became_peered":"2024-09-20T18:49:51.969400+0000","last_unstale":"2024-09-20T18:56:29.853331+0000","last_undegraded":"2024-09-20T18:56:29.853331+0000","last_fullsized":"2024-09-20T18:56:29.853331+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"206'8708","reported_seq":8811,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:36.353743+0000","last_change":"2024-09-20T18:54:33.926785+0000","last_active":"2024-09-20T18:56:36.353743+0000","last_peered":"2024-09-20T18:56:36.353743+0000","last_clean":"2024-09-20T18:56:36.353743+0000","last_became_active":"2024-09-20T18:54:33.925750+0000","last_became_peered":"2024-09-20T18:54:33.925750+0000","last_unstale":"2024-09-20T18:56:36.353743+0000","last_undegraded":"2024-09-20T18:56:36.353743+0000","last_fullsized":"2024-09-20T18:56:36.353743+0000","mapping_epoch":160,"log_start":"175'4340","ondisk_log_start":"175'4340","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4368,"log_dups_size":0,"ondisk_log_size":4368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393871360,"num_objects":6010,"num_object_clones":0,"num_object_copies":18030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8708,"num_write_kb":470976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":355,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.849208+0000","last_change":"2024-09-20T18:55:25.702762+0000","last_active":"2024-09-20T18:56:29.849208+0000","last_peered":"2024-09-20T18:56:29.849208+0000","last_clean":"2024-09-20T18:56:29.849208+0000","last_became_active":"2024-09-20T18:49:47.950904+0000","last_became_peered":"2024-09-20T18:49:47.950904+0000","last_unstale":"2024-09-20T18:56:29.849208+0000","last_undegraded":"2024-09-20T18:56:29.849208+0000","last_fullsized":"2024-09-20T18:56:29.849208+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-20T18:55:25.702596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:25.702596+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":339,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.848287+0000","last_change":"2024-09-20T18:55:28.830832+0000","last_active":"2024-09-20T18:56:29.848287+0000","last_peered":"2024-09-20T18:56:29.848287+0000","last_clean":"2024-09-20T18:56:29.848287+0000","last_became_active":"2024-09-20T18:55:28.830269+0000","last_became_peered":"2024-09-20T18:55:28.830269+0000","last_unstale":"2024-09-20T18:56:29.848287+0000","last_undegraded":"2024-09-20T18:56:29.848287+0000","last_fullsized":"2024-09-20T18:56:29.848287+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:23.359769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:23.359769+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":326,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.845481+0000","last_change":"2024-09-20T18:52:07.718766+0000","last_active":"2024-09-20T18:56:29.845481+0000","last_peered":"2024-09-20T18:56:29.845481+0000","last_clean":"2024-09-20T18:56:29.845481+0000","last_became_active":"2024-09-20T18:49:44.937135+0000","last_became_peered":"2024-09-20T18:49:44.937135+0000","last_unstale":"2024-09-20T18:56:29.845481+0000","last_undegraded":"2024-09-20T18:56:29.845481+0000","last_fullsized":"2024-09-20T18:56:29.845481+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:52:07.718646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:52:07.718646+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.00095558800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":309,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.846982+0000","last_change":"2024-09-20T18:51:59.796450+0000","last_active":"2024-09-20T18:56:29.846982+0000","last_peered":"2024-09-20T18:56:29.846982+0000","last_clean":"2024-09-20T18:56:29.846982+0000","last_became_active":"2024-09-20T18:49:49.954424+0000","last_became_peered":"2024-09-20T18:49:49.954424+0000","last_unstale":"2024-09-20T18:56:29.846982+0000","last_undegraded":"2024-09-20T18:56:29.846982+0000","last_fullsized":"2024-09-20T18:56:29.846982+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:59.796383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:59.796383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":310,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.845843+0000","last_change":"2024-09-20T18:55:28.826659+0000","last_active":"2024-09-20T18:56:29.845843+0000","last_peered":"2024-09-20T18:56:29.845843+0000","last_clean":"2024-09-20T18:56:29.845843+0000","last_became_active":"2024-09-20T18:55:28.825812+0000","last_became_peered":"2024-09-20T18:55:28.825812+0000","last_unstale":"2024-09-20T18:56:29.845843+0000","last_undegraded":"2024-09-20T18:56:29.845843+0000","last_fullsized":"2024-09-20T18:56:29.845843+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:39.814259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:39.814259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"206'8689","reported_seq":8792,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:36.224241+0000","last_change":"2024-09-20T18:54:34.311774+0000","last_active":"2024-09-20T18:56:36.224241+0000","last_peered":"2024-09-20T18:56:36.224241+0000","last_clean":"2024-09-20T18:56:36.224241+0000","last_became_active":"2024-09-20T18:54:34.311487+0000","last_became_peered":"2024-09-20T18:54:34.311487+0000","last_unstale":"2024-09-20T18:56:36.224241+0000","last_undegraded":"2024-09-20T18:56:36.224241+0000","last_fullsized":"2024-09-20T18:56:36.224241+0000","mapping_epoch":160,"log_start":"175'4392","ondisk_log_start":"175'4392","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4297,"log_dups_size":0,"ondisk_log_size":4297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":388562944,"num_objects":5929,"num_object_clones":0,"num_object_copies":17787,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8689,"num_write_kb":467776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":353,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.880338+0000","last_change":"2024-09-20T18:55:14.951950+0000","last_active":"2024-09-20T18:56:29.880338+0000","last_peered":"2024-09-20T18:56:29.880338+0000","last_clean":"2024-09-20T18:56:29.880338+0000","last_became_active":"2024-09-20T18:49:47.949230+0000","last_became_peered":"2024-09-20T18:49:47.949230+0000","last_unstale":"2024-09-20T18:56:29.880338+0000","last_undegraded":"2024-09-20T18:56:29.880338+0000","last_fullsized":"2024-09-20T18:56:29.880338+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:55:14.951892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:14.951892+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:36.283330+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":331,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.871683+0000","last_change":"2024-09-20T18:54:27.905856+0000","last_active":"2024-09-20T18:56:29.871683+0000","last_peered":"2024-09-20T18:56:29.871683+0000","last_clean":"2024-09-20T18:56:29.871683+0000","last_became_active":"2024-09-20T18:49:45.939197+0000","last_became_peered":"2024-09-20T18:49:45.939197+0000","last_unstale":"2024-09-20T18:56:29.871683+0000","last_undegraded":"2024-09-20T18:56:29.871683+0000","last_fullsized":"2024-09-20T18:56:29.871683+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:27.905803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:27.905803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":747,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:34.788384+0000","last_change":"2024-09-20T18:54:26.359887+0000","last_active":"2024-09-20T18:56:34.788384+0000","last_peered":"2024-09-20T18:56:34.788384+0000","last_clean":"2024-09-20T18:56:34.788384+0000","last_became_active":"2024-09-20T18:49:44.936819+0000","last_became_peered":"2024-09-20T18:49:44.936819+0000","last_unstale":"2024-09-20T18:56:34.788384+0000","last_undegraded":"2024-09-20T18:56:34.788384+0000","last_fullsized":"2024-09-20T18:56:34.788384+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-20T18:54:26.359807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:54:26.359807+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0010996199999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":405,"num_read_kb":400,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":315,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.852047+0000","last_change":"2024-09-20T18:54:59.151228+0000","last_active":"2024-09-20T18:56:29.852047+0000","last_peered":"2024-09-20T18:56:29.852047+0000","last_clean":"2024-09-20T18:56:29.852047+0000","last_became_active":"2024-09-20T18:49:49.961066+0000","last_became_peered":"2024-09-20T18:49:49.961066+0000","last_unstale":"2024-09-20T18:56:29.852047+0000","last_undegraded":"2024-09-20T18:56:29.852047+0000","last_fullsized":"2024-09-20T18:56:29.852047+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:59.150063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:59.150063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":320,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.847193+0000","last_change":"2024-09-20T18:55:21.629867+0000","last_active":"2024-09-20T18:56:29.847193+0000","last_peered":"2024-09-20T18:56:29.847193+0000","last_clean":"2024-09-20T18:56:29.847193+0000","last_became_active":"2024-09-20T18:49:51.970343+0000","last_became_peered":"2024-09-20T18:49:51.970343+0000","last_unstale":"2024-09-20T18:56:29.847193+0000","last_undegraded":"2024-09-20T18:56:29.847193+0000","last_fullsized":"2024-09-20T18:56:29.847193+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:21.629800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:21.629800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:40.736438+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"206'8653","reported_seq":14663,"reported_epoch":206,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-20T18:56:35.905343+0000","last_change":"2024-09-20T18:55:30.088909+0000","last_active":"2024-09-20T18:56:35.905343+0000","last_peered":"2024-09-20T18:56:35.905343+0000","last_clean":"2024-09-20T18:55:27.745898+0000","last_became_active":"2024-09-20T18:55:29.995624+0000","last_became_peered":"2024-09-20T18:55:29.995624+0000","last_unstale":"2024-09-20T18:56:35.905343+0000","last_undegraded":"2024-09-20T18:56:35.905343+0000","last_fullsized":"2024-09-20T18:55:29.815609+0000","mapping_epoch":182,"log_start":"172'3606","ondisk_log_start":"172'3606","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":5047,"log_dups_size":0,"ondisk_log_size":5047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":385286144,"num_objects":5879,"num_object_clones":0,"num_object_copies":17637,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2253,"num_objects_unfound":0,"num_objects_dirty":5879,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8653,"num_write_kb":465024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2953,"num_bytes_recovered":193527808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0],"avail_no_missing":["5","0","6"],"object_location_counts":[{"shards":"0,5","objects":3626},{"shards":"0,5,6","objects":2253}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"184'17","reported_seq":136,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.845866+0000","last_change":"2024-09-20T18:55:28.856602+0000","last_active":"2024-09-20T18:56:29.845866+0000","last_peered":"2024-09-20T18:56:29.845866+0000","last_clean":"2024-09-20T18:56:29.845866+0000","last_became_active":"2024-09-20T18:55:28.831024+0000","last_became_peered":"2024-09-20T18:55:28.831024+0000","last_unstale":"2024-09-20T18:56:29.845866+0000","last_undegraded":"2024-09-20T18:56:29.845866+0000","last_fullsized":"2024-09-20T18:56:29.845866+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":24,"num_read_kb":14,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":331,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.852037+0000","last_change":"2024-09-20T18:55:28.830447+0000","last_active":"2024-09-20T18:56:29.852037+0000","last_peered":"2024-09-20T18:56:29.852037+0000","last_clean":"2024-09-20T18:56:29.852037+0000","last_became_active":"2024-09-20T18:55:28.830147+0000","last_became_peered":"2024-09-20T18:55:28.830147+0000","last_unstale":"2024-09-20T18:56:29.852037+0000","last_undegraded":"2024-09-20T18:56:29.852037+0000","last_fullsized":"2024-09-20T18:56:29.852037+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:13.144896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:13.144896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":384,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.844931+0000","last_change":"2024-09-20T18:55:28.844060+0000","last_active":"2024-09-20T18:56:29.844931+0000","last_peered":"2024-09-20T18:56:29.844931+0000","last_clean":"2024-09-20T18:56:29.844931+0000","last_became_active":"2024-09-20T18:55:28.829325+0000","last_became_peered":"2024-09-20T18:55:28.829325+0000","last_unstale":"2024-09-20T18:56:29.844931+0000","last_undegraded":"2024-09-20T18:56:29.844931+0000","last_fullsized":"2024-09-20T18:56:29.844931+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T18:55:13.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T18:55:13.696607+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":307,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.845204+0000","last_change":"2024-09-20T18:51:22.852942+0000","last_active":"2024-09-20T18:56:29.845204+0000","last_peered":"2024-09-20T18:56:29.845204+0000","last_clean":"2024-09-20T18:56:29.845204+0000","last_became_active":"2024-09-20T18:49:49.956128+0000","last_became_peered":"2024-09-20T18:49:49.956128+0000","last_unstale":"2024-09-20T18:56:29.845204+0000","last_undegraded":"2024-09-20T18:56:29.845204+0000","last_fullsized":"2024-09-20T18:56:29.845204+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:22.852869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:22.852869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":319,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.851824+0000","last_change":"2024-09-20T18:55:25.281435+0000","last_active":"2024-09-20T18:56:29.851824+0000","last_peered":"2024-09-20T18:56:29.851824+0000","last_clean":"2024-09-20T18:56:29.851824+0000","last_became_active":"2024-09-20T18:49:51.971332+0000","last_became_peered":"2024-09-20T18:49:51.971332+0000","last_unstale":"2024-09-20T18:56:29.851824+0000","last_undegraded":"2024-09-20T18:56:29.851824+0000","last_fullsized":"2024-09-20T18:56:29.851824+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:25.281366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:25.281366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"206'8594","reported_seq":8697,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:35.805249+0000","last_change":"2024-09-20T18:54:33.929366+0000","last_active":"2024-09-20T18:56:35.805249+0000","last_peered":"2024-09-20T18:56:35.805249+0000","last_clean":"2024-09-20T18:56:35.805249+0000","last_became_active":"2024-09-20T18:54:33.929072+0000","last_became_peered":"2024-09-20T18:54:33.929072+0000","last_unstale":"2024-09-20T18:56:35.805249+0000","last_undegraded":"2024-09-20T18:56:35.805249+0000","last_fullsized":"2024-09-20T18:56:35.805249+0000","mapping_epoch":160,"log_start":"174'4002","ondisk_log_start":"174'4002","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4592,"log_dups_size":0,"ondisk_log_size":4592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":391118848,"num_objects":5968,"num_object_clones":0,"num_object_copies":17904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5968,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8594,"num_write_kb":465984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":350,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.842507+0000","last_change":"2024-09-20T18:55:28.840105+0000","last_active":"2024-09-20T18:56:29.842507+0000","last_peered":"2024-09-20T18:56:29.842507+0000","last_clean":"2024-09-20T18:56:29.842507+0000","last_became_active":"2024-09-20T18:55:28.818269+0000","last_became_peered":"2024-09-20T18:55:28.818269+0000","last_unstale":"2024-09-20T18:56:29.842507+0000","last_undegraded":"2024-09-20T18:56:29.842507+0000","last_fullsized":"2024-09-20T18:56:29.842507+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:51:14.397930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.397930+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":316,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.841478+0000","last_change":"2024-09-20T18:51:18.410601+0000","last_active":"2024-09-20T18:56:29.841478+0000","last_peered":"2024-09-20T18:56:29.841478+0000","last_clean":"2024-09-20T18:56:29.841478+0000","last_became_active":"2024-09-20T18:49:45.944579+0000","last_became_peered":"2024-09-20T18:49:45.944579+0000","last_unstale":"2024-09-20T18:56:29.841478+0000","last_undegraded":"2024-09-20T18:56:29.841478+0000","last_fullsized":"2024-09-20T18:56:29.841478+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:18.410447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:18.410447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":329,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.851684+0000","last_change":"2024-09-20T18:55:28.830634+0000","last_active":"2024-09-20T18:56:29.851684+0000","last_peered":"2024-09-20T18:56:29.851684+0000","last_clean":"2024-09-20T18:56:29.851684+0000","last_became_active":"2024-09-20T18:55:28.827785+0000","last_became_peered":"2024-09-20T18:55:28.827785+0000","last_unstale":"2024-09-20T18:56:29.851684+0000","last_undegraded":"2024-09-20T18:56:29.851684+0000","last_fullsized":"2024-09-20T18:56:29.851684+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:11.401777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:11.401777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":311,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.851792+0000","last_change":"2024-09-20T18:55:28.833649+0000","last_active":"2024-09-20T18:56:29.851792+0000","last_peered":"2024-09-20T18:56:29.851792+0000","last_clean":"2024-09-20T18:56:29.851792+0000","last_became_active":"2024-09-20T18:55:28.833115+0000","last_became_peered":"2024-09-20T18:55:28.833115+0000","last_unstale":"2024-09-20T18:56:29.851792+0000","last_undegraded":"2024-09-20T18:56:29.851792+0000","last_fullsized":"2024-09-20T18:56:29.851792+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:05.021071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.021071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1","version":"206'8540","reported_seq":8643,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:36.215547+0000","last_change":"2024-09-20T18:54:34.310237+0000","last_active":"2024-09-20T18:56:36.215547+0000","last_peered":"2024-09-20T18:56:36.215547+0000","last_clean":"2024-09-20T18:56:36.215547+0000","last_became_active":"2024-09-20T18:54:34.309781+0000","last_became_peered":"2024-09-20T18:54:34.309781+0000","last_unstale":"2024-09-20T18:56:36.215547+0000","last_undegraded":"2024-09-20T18:56:36.215547+0000","last_fullsized":"2024-09-20T18:56:36.215547+0000","mapping_epoch":160,"log_start":"175'4224","ondisk_log_start":"175'4224","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4316,"log_dups_size":0,"ondisk_log_size":4316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":386138112,"num_objects":5892,"num_object_clones":0,"num_object_copies":17676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5892,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8540,"num_write_kb":461824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":348,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.846024+0000","last_change":"2024-09-20T18:52:02.944536+0000","last_active":"2024-09-20T18:56:29.846024+0000","last_peered":"2024-09-20T18:56:29.846024+0000","last_clean":"2024-09-20T18:56:29.846024+0000","last_became_active":"2024-09-20T18:49:47.947636+0000","last_became_peered":"2024-09-20T18:49:47.947636+0000","last_unstale":"2024-09-20T18:56:29.846024+0000","last_undegraded":"2024-09-20T18:56:29.846024+0000","last_fullsized":"2024-09-20T18:56:29.846024+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-20T18:52:02.944434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:02.944434+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":185,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.841701+0000","last_change":"2024-09-20T18:55:28.822330+0000","last_active":"2024-09-20T18:56:29.841701+0000","last_peered":"2024-09-20T18:56:29.841701+0000","last_clean":"2024-09-20T18:56:29.841701+0000","last_became_active":"2024-09-20T18:55:28.821959+0000","last_became_peered":"2024-09-20T18:55:28.821959+0000","last_unstale":"2024-09-20T18:56:29.841701+0000","last_undegraded":"2024-09-20T18:56:29.841701+0000","last_fullsized":"2024-09-20T18:56:29.841701+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:45.669427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:45.669427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":316,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.847307+0000","last_change":"2024-09-20T18:54:11.377907+0000","last_active":"2024-09-20T18:56:29.847307+0000","last_peered":"2024-09-20T18:56:29.847307+0000","last_clean":"2024-09-20T18:56:29.847307+0000","last_became_active":"2024-09-20T18:49:49.959160+0000","last_became_peered":"2024-09-20T18:49:49.959160+0000","last_unstale":"2024-09-20T18:56:29.847307+0000","last_undegraded":"2024-09-20T18:56:29.847307+0000","last_fullsized":"2024-09-20T18:56:29.847307+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:11.376951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:11.376951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":312,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.847467+0000","last_change":"2024-09-20T18:52:35.635897+0000","last_active":"2024-09-20T18:56:29.847467+0000","last_peered":"2024-09-20T18:56:29.847467+0000","last_clean":"2024-09-20T18:56:29.847467+0000","last_became_active":"2024-09-20T18:49:51.970108+0000","last_became_peered":"2024-09-20T18:49:51.970108+0000","last_unstale":"2024-09-20T18:56:29.847467+0000","last_undegraded":"2024-09-20T18:56:29.847467+0000","last_fullsized":"2024-09-20T18:56:29.847467+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:35.635818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:35.635818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"206'8417","reported_seq":8521,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:36.380773+0000","last_change":"2024-09-20T18:54:33.929552+0000","last_active":"2024-09-20T18:56:36.380773+0000","last_peered":"2024-09-20T18:56:36.380773+0000","last_clean":"2024-09-20T18:56:36.380773+0000","last_became_active":"2024-09-20T18:54:33.929208+0000","last_became_peered":"2024-09-20T18:54:33.929208+0000","last_unstale":"2024-09-20T18:56:36.380773+0000","last_undegraded":"2024-09-20T18:56:36.380773+0000","last_fullsized":"2024-09-20T18:56:36.380773+0000","mapping_epoch":160,"log_start":"174'4121","ondisk_log_start":"174'4121","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4296,"log_dups_size":0,"ondisk_log_size":4296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":380698624,"num_objects":5809,"num_object_clones":0,"num_object_copies":17427,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5809,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8417,"num_write_kb":455232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":354,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.842093+0000","last_change":"2024-09-20T18:55:28.846773+0000","last_active":"2024-09-20T18:56:29.842093+0000","last_peered":"2024-09-20T18:56:29.842093+0000","last_clean":"2024-09-20T18:56:29.842093+0000","last_became_active":"2024-09-20T18:55:28.817101+0000","last_became_peered":"2024-09-20T18:55:28.817101+0000","last_unstale":"2024-09-20T18:56:29.842093+0000","last_undegraded":"2024-09-20T18:56:29.842093+0000","last_fullsized":"2024-09-20T18:56:29.842093+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:54:17.219548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:17.219548+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":324,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.853629+0000","last_change":"2024-09-20T18:52:46.005172+0000","last_active":"2024-09-20T18:56:29.853629+0000","last_peered":"2024-09-20T18:56:29.853629+0000","last_clean":"2024-09-20T18:56:29.853629+0000","last_became_active":"2024-09-20T18:49:45.937264+0000","last_became_peered":"2024-09-20T18:49:45.937264+0000","last_unstale":"2024-09-20T18:56:29.853629+0000","last_undegraded":"2024-09-20T18:56:29.853629+0000","last_fullsized":"2024-09-20T18:56:29.853629+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:46.005109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:46.005109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":416,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:36.074022+0000","last_change":"2024-09-20T18:55:17.549660+0000","last_active":"2024-09-20T18:56:36.074022+0000","last_peered":"2024-09-20T18:56:36.074022+0000","last_clean":"2024-09-20T18:56:36.074022+0000","last_became_active":"2024-09-20T18:49:49.957005+0000","last_became_peered":"2024-09-20T18:49:49.957005+0000","last_unstale":"2024-09-20T18:56:36.074022+0000","last_undegraded":"2024-09-20T18:56:36.074022+0000","last_fullsized":"2024-09-20T18:56:36.074022+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:17.549598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:17.549598+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:45.590247+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":311,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.842230+0000","last_change":"2024-09-20T18:54:42.182875+0000","last_active":"2024-09-20T18:56:29.842230+0000","last_peered":"2024-09-20T18:56:29.842230+0000","last_clean":"2024-09-20T18:56:29.842230+0000","last_became_active":"2024-09-20T18:49:51.969355+0000","last_became_peered":"2024-09-20T18:49:51.969355+0000","last_unstale":"2024-09-20T18:56:29.842230+0000","last_undegraded":"2024-09-20T18:56:29.842230+0000","last_fullsized":"2024-09-20T18:56:29.842230+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:42.182809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:42.182809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"206'8623","reported_seq":16686,"reported_epoch":206,"state":"active+remapped+backfilling","last_fresh":"2024-09-20T18:56:36.231658+0000","last_change":"2024-09-20T18:55:59.938306+0000","last_active":"2024-09-20T18:56:36.231658+0000","last_peered":"2024-09-20T18:56:36.231658+0000","last_clean":"2024-09-20T18:55:27.792821+0000","last_became_active":"2024-09-20T18:55:29.915245+0000","last_became_peered":"2024-09-20T18:55:29.915245+0000","last_unstale":"2024-09-20T18:56:36.231658+0000","last_undegraded":"2024-09-20T18:56:36.231658+0000","last_fullsized":"2024-09-20T18:56:36.231658+0000","mapping_epoch":182,"log_start":"175'4361","ondisk_log_start":"175'4361","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4262,"log_dups_size":0,"ondisk_log_size":4262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":388431872,"num_objects":5927,"num_object_clones":0,"num_object_copies":17781,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2402,"num_objects_unfound":0,"num_objects_dirty":5927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8623,"num_write_kb":465600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3975,"num_bytes_recovered":260505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,6],"avail_no_missing":["1","3","6"],"object_location_counts":[{"shards":"1,3,6","objects":5927}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":343,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.866435+0000","last_change":"2024-09-20T18:54:54.897991+0000","last_active":"2024-09-20T18:56:29.866435+0000","last_peered":"2024-09-20T18:56:29.866435+0000","last_clean":"2024-09-20T18:56:29.866435+0000","last_became_active":"2024-09-20T18:49:47.946008+0000","last_became_peered":"2024-09-20T18:49:47.946008+0000","last_unstale":"2024-09-20T18:56:29.866435+0000","last_undegraded":"2024-09-20T18:56:29.866435+0000","last_fullsized":"2024-09-20T18:56:29.866435+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:54.897933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:54.897933+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":338,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.851547+0000","last_change":"2024-09-20T18:55:28.833453+0000","last_active":"2024-09-20T18:56:29.851547+0000","last_peered":"2024-09-20T18:56:29.851547+0000","last_clean":"2024-09-20T18:56:29.851547+0000","last_became_active":"2024-09-20T18:55:28.833095+0000","last_became_peered":"2024-09-20T18:55:28.833095+0000","last_unstale":"2024-09-20T18:56:29.851547+0000","last_undegraded":"2024-09-20T18:56:29.851547+0000","last_fullsized":"2024-09-20T18:56:29.851547+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:56.212378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:56.212378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":325,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.847194+0000","last_change":"2024-09-20T18:54:43.379898+0000","last_active":"2024-09-20T18:56:29.847194+0000","last_peered":"2024-09-20T18:56:29.847194+0000","last_clean":"2024-09-20T18:56:29.847194+0000","last_became_active":"2024-09-20T18:49:49.956767+0000","last_became_peered":"2024-09-20T18:49:49.956767+0000","last_unstale":"2024-09-20T18:56:29.847194+0000","last_undegraded":"2024-09-20T18:56:29.847194+0000","last_fullsized":"2024-09-20T18:56:29.847194+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:43.379706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:43.379706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":311,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.871602+0000","last_change":"2024-09-20T18:54:24.976627+0000","last_active":"2024-09-20T18:56:29.871602+0000","last_peered":"2024-09-20T18:56:29.871602+0000","last_clean":"2024-09-20T18:56:29.871602+0000","last_became_active":"2024-09-20T18:49:51.964098+0000","last_became_peered":"2024-09-20T18:49:51.964098+0000","last_unstale":"2024-09-20T18:56:29.871602+0000","last_undegraded":"2024-09-20T18:56:29.871602+0000","last_fullsized":"2024-09-20T18:56:29.871602+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:24.976584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:24.976584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"206'8669","reported_seq":10959,"reported_epoch":206,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-20T18:56:36.580559+0000","last_change":"2024-09-20T18:56:27.871592+0000","last_active":"2024-09-20T18:56:36.580559+0000","last_peered":"2024-09-20T18:56:36.580559+0000","last_clean":"2024-09-20T18:55:27.791097+0000","last_became_active":"2024-09-20T18:55:29.979932+0000","last_became_peered":"2024-09-20T18:55:29.979932+0000","last_unstale":"2024-09-20T18:56:36.580559+0000","last_undegraded":"2024-09-20T18:56:27.867865+0000","last_fullsized":"2024-09-20T18:55:29.815116+0000","mapping_epoch":182,"log_start":"172'3740","ondisk_log_start":"172'3740","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4929,"log_dups_size":0,"ondisk_log_size":4929,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":378732544,"num_objects":5779,"num_object_clones":0,"num_object_copies":17337,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":354,"num_objects_misplaced":3732,"num_objects_unfound":0,"num_objects_dirty":5779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8669,"num_write_kb":462336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1092,"num_bytes_recovered":71565312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7],"avail_no_missing":["0","6","7"],"object_location_counts":[{"shards":"0,6,7","objects":3732},{"shards":"0,7","objects":1693}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":132,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.847390+0000","last_change":"2024-09-20T18:55:28.855793+0000","last_active":"2024-09-20T18:56:29.847390+0000","last_peered":"2024-09-20T18:56:29.847390+0000","last_clean":"2024-09-20T18:56:29.847390+0000","last_became_active":"2024-09-20T18:55:28.828563+0000","last_became_peered":"2024-09-20T18:55:28.828563+0000","last_unstale":"2024-09-20T18:56:29.847390+0000","last_undegraded":"2024-09-20T18:56:29.847390+0000","last_fullsized":"2024-09-20T18:56:29.847390+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-20T18:53:24.458005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:24.458005+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":31,"num_read_kb":20,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,5],"acting":[1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":338,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.846126+0000","last_change":"2024-09-20T18:55:28.835767+0000","last_active":"2024-09-20T18:56:29.846126+0000","last_peered":"2024-09-20T18:56:29.846126+0000","last_clean":"2024-09-20T18:56:29.846126+0000","last_became_active":"2024-09-20T18:55:28.835149+0000","last_became_peered":"2024-09-20T18:55:28.835149+0000","last_unstale":"2024-09-20T18:56:29.846126+0000","last_undegraded":"2024-09-20T18:56:29.846126+0000","last_fullsized":"2024-09-20T18:56:29.846126+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:44.501192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.501192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":325,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.844794+0000","last_change":"2024-09-20T18:55:13.577772+0000","last_active":"2024-09-20T18:56:29.844794+0000","last_peered":"2024-09-20T18:56:29.844794+0000","last_clean":"2024-09-20T18:56:29.844794+0000","last_became_active":"2024-09-20T18:49:49.957558+0000","last_became_peered":"2024-09-20T18:49:49.957558+0000","last_unstale":"2024-09-20T18:56:29.844794+0000","last_undegraded":"2024-09-20T18:56:29.844794+0000","last_fullsized":"2024-09-20T18:56:29.844794+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:13.577699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:13.577699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:35.011344+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":319,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.844804+0000","last_change":"2024-09-20T18:53:47.370152+0000","last_active":"2024-09-20T18:56:29.844804+0000","last_peered":"2024-09-20T18:56:29.844804+0000","last_clean":"2024-09-20T18:56:29.844804+0000","last_became_active":"2024-09-20T18:49:51.970798+0000","last_became_peered":"2024-09-20T18:49:51.970798+0000","last_unstale":"2024-09-20T18:56:29.844804+0000","last_undegraded":"2024-09-20T18:56:29.844804+0000","last_fullsized":"2024-09-20T18:56:29.844804+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:47.370108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:47.370108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"206'8852","reported_seq":14115,"reported_epoch":206,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-20T18:56:35.874486+0000","last_change":"2024-09-20T18:55:30.067593+0000","last_active":"2024-09-20T18:56:35.874486+0000","last_peered":"2024-09-20T18:56:35.874486+0000","last_clean":"2024-09-20T18:55:27.760955+0000","last_became_active":"2024-09-20T18:55:30.004922+0000","last_became_peered":"2024-09-20T18:55:30.004922+0000","last_unstale":"2024-09-20T18:56:35.874486+0000","last_undegraded":"2024-09-20T18:56:35.874486+0000","last_fullsized":"2024-09-20T18:55:29.819726+0000","mapping_epoch":182,"log_start":"174'4246","ondisk_log_start":"174'4246","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4606,"log_dups_size":0,"ondisk_log_size":4606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":401997824,"num_objects":6134,"num_object_clones":0,"num_object_copies":18402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2813,"num_objects_unfound":0,"num_objects_dirty":6134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8852,"num_write_kb":479552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2578,"num_bytes_recovered":168951808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7],"avail_no_missing":["4","6","7"],"object_location_counts":[{"shards":"4,6,7","objects":2813},{"shards":"4,7","objects":3321}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":343,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.841344+0000","last_change":"2024-09-20T18:53:52.898818+0000","last_active":"2024-09-20T18:56:29.841344+0000","last_peered":"2024-09-20T18:56:29.841344+0000","last_clean":"2024-09-20T18:56:29.841344+0000","last_became_active":"2024-09-20T18:49:47.949116+0000","last_became_peered":"2024-09-20T18:49:47.949116+0000","last_unstale":"2024-09-20T18:56:29.841344+0000","last_undegraded":"2024-09-20T18:56:29.841344+0000","last_fullsized":"2024-09-20T18:56:29.841344+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:53:52.898760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:52.898760+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":331,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.845535+0000","last_change":"2024-09-20T18:54:52.650798+0000","last_active":"2024-09-20T18:56:29.845535+0000","last_peered":"2024-09-20T18:56:29.845535+0000","last_clean":"2024-09-20T18:56:29.845535+0000","last_became_active":"2024-09-20T18:49:45.944858+0000","last_became_peered":"2024-09-20T18:49:45.944858+0000","last_unstale":"2024-09-20T18:56:29.845535+0000","last_undegraded":"2024-09-20T18:56:29.845535+0000","last_fullsized":"2024-09-20T18:56:29.845535+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:52.650756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:52.650756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":317,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.844874+0000","last_change":"2024-09-20T18:54:42.349879+0000","last_active":"2024-09-20T18:56:29.844874+0000","last_peered":"2024-09-20T18:56:29.844874+0000","last_clean":"2024-09-20T18:56:29.844874+0000","last_became_active":"2024-09-20T18:49:49.954944+0000","last_became_peered":"2024-09-20T18:49:49.954944+0000","last_unstale":"2024-09-20T18:56:29.844874+0000","last_undegraded":"2024-09-20T18:56:29.844874+0000","last_fullsized":"2024-09-20T18:56:29.844874+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:42.349807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:42.349807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":312,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.853354+0000","last_change":"2024-09-20T18:55:17.341862+0000","last_active":"2024-09-20T18:56:29.853354+0000","last_peered":"2024-09-20T18:56:29.853354+0000","last_clean":"2024-09-20T18:56:29.853354+0000","last_became_active":"2024-09-20T18:49:51.972970+0000","last_became_peered":"2024-09-20T18:49:51.972970+0000","last_unstale":"2024-09-20T18:56:29.853354+0000","last_undegraded":"2024-09-20T18:56:29.853354+0000","last_fullsized":"2024-09-20T18:56:29.853354+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:17.341793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:17.341793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.4","version":"206'8480","reported_seq":8583,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:35.813220+0000","last_change":"2024-09-20T18:54:34.309442+0000","last_active":"2024-09-20T18:56:35.813220+0000","last_peered":"2024-09-20T18:56:35.813220+0000","last_clean":"2024-09-20T18:56:35.813220+0000","last_became_active":"2024-09-20T18:54:34.309113+0000","last_became_peered":"2024-09-20T18:54:34.309113+0000","last_unstale":"2024-09-20T18:56:35.813220+0000","last_undegraded":"2024-09-20T18:56:35.813220+0000","last_fullsized":"2024-09-20T18:56:35.813220+0000","mapping_epoch":160,"log_start":"171'3431","ondisk_log_start":"171'3431","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":5049,"log_dups_size":0,"ondisk_log_size":5049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":383516672,"num_objects":5852,"num_object_clones":0,"num_object_copies":17556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5852,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8480,"num_write_kb":458624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":367,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.853540+0000","last_change":"2024-09-20T18:55:28.847010+0000","last_active":"2024-09-20T18:56:29.853540+0000","last_peered":"2024-09-20T18:56:29.853540+0000","last_clean":"2024-09-20T18:56:29.853540+0000","last_became_active":"2024-09-20T18:55:28.824801+0000","last_became_peered":"2024-09-20T18:55:28.824801+0000","last_unstale":"2024-09-20T18:56:29.853540+0000","last_undegraded":"2024-09-20T18:56:29.853540+0000","last_fullsized":"2024-09-20T18:56:29.853540+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:54:12.373309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:12.373309+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":351,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.845198+0000","last_change":"2024-09-20T18:55:24.729301+0000","last_active":"2024-09-20T18:56:29.845198+0000","last_peered":"2024-09-20T18:56:29.845198+0000","last_clean":"2024-09-20T18:56:29.845198+0000","last_became_active":"2024-09-20T18:49:45.944256+0000","last_became_peered":"2024-09-20T18:49:45.944256+0000","last_unstale":"2024-09-20T18:56:29.845198+0000","last_undegraded":"2024-09-20T18:56:29.845198+0000","last_fullsized":"2024-09-20T18:56:29.845198+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-20T18:55:24.729219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:24.729219+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"184'9","reported_seq":408,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.844838+0000","last_change":"2024-09-20T18:55:28.824771+0000","last_active":"2024-09-20T18:56:29.844838+0000","last_peered":"2024-09-20T18:56:29.844838+0000","last_clean":"2024-09-20T18:56:29.844838+0000","last_became_active":"2024-09-20T18:55:28.818085+0000","last_became_peered":"2024-09-20T18:55:28.818085+0000","last_unstale":"2024-09-20T18:56:29.844838+0000","last_undegraded":"2024-09-20T18:56:29.844838+0000","last_fullsized":"2024-09-20T18:56:29.844838+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:52:05.788924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.788924+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":318,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.852201+0000","last_change":"2024-09-20T18:55:28.832276+0000","last_active":"2024-09-20T18:56:29.852201+0000","last_peered":"2024-09-20T18:56:29.852201+0000","last_clean":"2024-09-20T18:56:29.852201+0000","last_became_active":"2024-09-20T18:55:28.830380+0000","last_became_peered":"2024-09-20T18:55:28.830380+0000","last_unstale":"2024-09-20T18:56:29.852201+0000","last_undegraded":"2024-09-20T18:56:29.852201+0000","last_fullsized":"2024-09-20T18:56:29.852201+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:14.247398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:14.247398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:30.298253+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":61,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.865736+0000","last_change":"2024-09-20T18:55:28.835344+0000","last_active":"2024-09-20T18:56:29.865736+0000","last_peered":"2024-09-20T18:56:29.865736+0000","last_clean":"2024-09-20T18:56:29.865736+0000","last_became_active":"2024-09-20T18:55:28.835010+0000","last_became_peered":"2024-09-20T18:55:28.835010+0000","last_unstale":"2024-09-20T18:56:29.865736+0000","last_undegraded":"2024-09-20T18:56:29.865736+0000","last_fullsized":"2024-09-20T18:56:29.865736+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:26.721735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:26.721735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:49.502875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":328,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.863956+0000","last_change":"2024-09-20T18:54:44.194161+0000","last_active":"2024-09-20T18:56:29.863956+0000","last_peered":"2024-09-20T18:56:29.863956+0000","last_clean":"2024-09-20T18:56:29.863956+0000","last_became_active":"2024-09-20T18:49:48.500662+0000","last_became_peered":"2024-09-20T18:49:48.500662+0000","last_unstale":"2024-09-20T18:56:29.863956+0000","last_undegraded":"2024-09-20T18:56:29.863956+0000","last_fullsized":"2024-09-20T18:56:29.863956+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-20T18:54:44.194089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.194089+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":314,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.851317+0000","last_change":"2024-09-20T18:53:16.220554+0000","last_active":"2024-09-20T18:56:29.851317+0000","last_peered":"2024-09-20T18:56:29.851317+0000","last_clean":"2024-09-20T18:56:29.851317+0000","last_became_active":"2024-09-20T18:49:49.956749+0000","last_became_peered":"2024-09-20T18:49:49.956749+0000","last_unstale":"2024-09-20T18:56:29.851317+0000","last_undegraded":"2024-09-20T18:56:29.851317+0000","last_fullsized":"2024-09-20T18:56:29.851317+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:16.220490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:16.220490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":325,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.845836+0000","last_change":"2024-09-20T18:55:28.827897+0000","last_active":"2024-09-20T18:56:29.845836+0000","last_peered":"2024-09-20T18:56:29.845836+0000","last_clean":"2024-09-20T18:56:29.845836+0000","last_became_active":"2024-09-20T18:55:28.827418+0000","last_became_peered":"2024-09-20T18:55:28.827418+0000","last_unstale":"2024-09-20T18:56:29.845836+0000","last_undegraded":"2024-09-20T18:56:29.845836+0000","last_fullsized":"2024-09-20T18:56:29.845836+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.637739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.637739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":334,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.880405+0000","last_change":"2024-09-20T18:54:44.830872+0000","last_active":"2024-09-20T18:56:29.880405+0000","last_peered":"2024-09-20T18:56:29.880405+0000","last_clean":"2024-09-20T18:56:29.880405+0000","last_became_active":"2024-09-20T18:49:48.500803+0000","last_became_peered":"2024-09-20T18:49:48.500803+0000","last_unstale":"2024-09-20T18:56:29.880405+0000","last_undegraded":"2024-09-20T18:56:29.880405+0000","last_fullsized":"2024-09-20T18:56:29.880405+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:44.830790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.830790+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":329,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.844933+0000","last_change":"2024-09-20T18:55:29.751718+0000","last_active":"2024-09-20T18:56:29.844933+0000","last_peered":"2024-09-20T18:56:29.844933+0000","last_clean":"2024-09-20T18:56:29.844933+0000","last_became_active":"2024-09-20T18:55:28.833799+0000","last_became_peered":"2024-09-20T18:55:28.833799+0000","last_unstale":"2024-09-20T18:56:29.844933+0000","last_undegraded":"2024-09-20T18:56:29.844933+0000","last_fullsized":"2024-09-20T18:56:29.844933+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:29.751642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:29.751642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:40.748541+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":315,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.863799+0000","last_change":"2024-09-20T18:53:50.253721+0000","last_active":"2024-09-20T18:56:29.863799+0000","last_peered":"2024-09-20T18:56:29.863799+0000","last_clean":"2024-09-20T18:56:29.863799+0000","last_became_active":"2024-09-20T18:49:49.961947+0000","last_became_peered":"2024-09-20T18:49:49.961947+0000","last_unstale":"2024-09-20T18:56:29.863799+0000","last_undegraded":"2024-09-20T18:56:29.863799+0000","last_fullsized":"2024-09-20T18:56:29.863799+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:50.253660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:50.253660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"66'1","reported_seq":321,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.845185+0000","last_change":"2024-09-20T18:54:20.681878+0000","last_active":"2024-09-20T18:56:29.845185+0000","last_peered":"2024-09-20T18:56:29.845185+0000","last_clean":"2024-09-20T18:56:29.845185+0000","last_became_active":"2024-09-20T18:49:51.965121+0000","last_became_peered":"2024-09-20T18:49:51.965121+0000","last_unstale":"2024-09-20T18:56:29.845185+0000","last_undegraded":"2024-09-20T18:56:29.845185+0000","last_fullsized":"2024-09-20T18:56:29.845185+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T18:54:20.681790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:20.681790+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":326,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.852677+0000","last_change":"2024-09-20T18:55:12.205118+0000","last_active":"2024-09-20T18:56:29.852677+0000","last_peered":"2024-09-20T18:56:29.852677+0000","last_clean":"2024-09-20T18:56:29.852677+0000","last_became_active":"2024-09-20T18:49:48.501011+0000","last_became_peered":"2024-09-20T18:49:48.501011+0000","last_unstale":"2024-09-20T18:56:29.852677+0000","last_undegraded":"2024-09-20T18:56:29.852677+0000","last_fullsized":"2024-09-20T18:56:29.852677+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:55:12.205069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:12.205069+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":330,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.868867+0000","last_change":"2024-09-20T18:55:27.028184+0000","last_active":"2024-09-20T18:56:29.868867+0000","last_peered":"2024-09-20T18:56:29.868867+0000","last_clean":"2024-09-20T18:56:29.868867+0000","last_became_active":"2024-09-20T18:49:45.938511+0000","last_became_peered":"2024-09-20T18:49:45.938511+0000","last_unstale":"2024-09-20T18:56:29.868867+0000","last_undegraded":"2024-09-20T18:56:29.868867+0000","last_fullsized":"2024-09-20T18:56:29.868867+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:27.028126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:27.028126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:47.494086+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":414,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:31.073266+0000","last_change":"2024-09-20T18:55:27.284156+0000","last_active":"2024-09-20T18:56:31.073266+0000","last_peered":"2024-09-20T18:56:31.073266+0000","last_clean":"2024-09-20T18:56:31.073266+0000","last_became_active":"2024-09-20T18:49:49.960550+0000","last_became_peered":"2024-09-20T18:49:49.960550+0000","last_unstale":"2024-09-20T18:56:31.073266+0000","last_undegraded":"2024-09-20T18:56:31.073266+0000","last_fullsized":"2024-09-20T18:56:31.073266+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:27.284086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:27.284086+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:48.225116+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":319,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.842053+0000","last_change":"2024-09-20T18:55:12.348079+0000","last_active":"2024-09-20T18:56:29.842053+0000","last_peered":"2024-09-20T18:56:29.842053+0000","last_clean":"2024-09-20T18:56:29.842053+0000","last_became_active":"2024-09-20T18:49:51.966254+0000","last_became_peered":"2024-09-20T18:49:51.966254+0000","last_unstale":"2024-09-20T18:56:29.842053+0000","last_undegraded":"2024-09-20T18:56:29.842053+0000","last_fullsized":"2024-09-20T18:56:29.842053+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:12.347993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:12.347993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:32.305729+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":345,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.857683+0000","last_change":"2024-09-20T18:51:15.223913+0000","last_active":"2024-09-20T18:56:29.857683+0000","last_peered":"2024-09-20T18:56:29.857683+0000","last_clean":"2024-09-20T18:56:29.857683+0000","last_became_active":"2024-09-20T18:49:58.997494+0000","last_became_peered":"2024-09-20T18:49:58.997494+0000","last_unstale":"2024-09-20T18:56:29.857683+0000","last_undegraded":"2024-09-20T18:56:29.857683+0000","last_fullsized":"2024-09-20T18:56:29.857683+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:51:15.223826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:15.223826+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":330,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.844912+0000","last_change":"2024-09-20T18:54:57.632458+0000","last_active":"2024-09-20T18:56:29.844912+0000","last_peered":"2024-09-20T18:56:29.844912+0000","last_clean":"2024-09-20T18:56:29.844912+0000","last_became_active":"2024-09-20T18:49:45.949492+0000","last_became_peered":"2024-09-20T18:49:45.949492+0000","last_unstale":"2024-09-20T18:56:29.844912+0000","last_undegraded":"2024-09-20T18:56:29.844912+0000","last_fullsized":"2024-09-20T18:56:29.844912+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.632380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.632380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":412,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.852450+0000","last_change":"2024-09-20T18:55:28.302353+0000","last_active":"2024-09-20T18:56:29.852450+0000","last_peered":"2024-09-20T18:56:29.852450+0000","last_clean":"2024-09-20T18:56:29.852450+0000","last_became_active":"2024-09-20T18:49:49.962243+0000","last_became_peered":"2024-09-20T18:49:49.962243+0000","last_unstale":"2024-09-20T18:56:29.852450+0000","last_undegraded":"2024-09-20T18:56:29.852450+0000","last_fullsized":"2024-09-20T18:56:29.852450+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:28.302303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:28.302303+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:55.176573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":318,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.880112+0000","last_change":"2024-09-20T18:55:11.918284+0000","last_active":"2024-09-20T18:56:29.880112+0000","last_peered":"2024-09-20T18:56:29.880112+0000","last_clean":"2024-09-20T18:56:29.880112+0000","last_became_active":"2024-09-20T18:49:51.969093+0000","last_became_peered":"2024-09-20T18:49:51.969093+0000","last_unstale":"2024-09-20T18:56:29.880112+0000","last_undegraded":"2024-09-20T18:56:29.880112+0000","last_fullsized":"2024-09-20T18:56:29.880112+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:11.918210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:11.918210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":357,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.841895+0000","last_change":"2024-09-20T18:55:28.854249+0000","last_active":"2024-09-20T18:56:29.841895+0000","last_peered":"2024-09-20T18:56:29.841895+0000","last_clean":"2024-09-20T18:56:29.841895+0000","last_became_active":"2024-09-20T18:55:28.826721+0000","last_became_peered":"2024-09-20T18:55:28.826721+0000","last_unstale":"2024-09-20T18:56:29.841895+0000","last_undegraded":"2024-09-20T18:56:29.841895+0000","last_fullsized":"2024-09-20T18:56:29.841895+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:53:43.878921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:43.878921+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":322,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.842451+0000","last_change":"2024-09-20T18:55:28.822690+0000","last_active":"2024-09-20T18:56:29.842451+0000","last_peered":"2024-09-20T18:56:29.842451+0000","last_clean":"2024-09-20T18:56:29.842451+0000","last_became_active":"2024-09-20T18:55:28.822258+0000","last_became_peered":"2024-09-20T18:55:28.822258+0000","last_unstale":"2024-09-20T18:56:29.842451+0000","last_undegraded":"2024-09-20T18:56:29.842451+0000","last_fullsized":"2024-09-20T18:56:29.842451+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:05.985551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.985551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":307,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.847442+0000","last_change":"2024-09-20T18:51:14.271954+0000","last_active":"2024-09-20T18:56:29.847442+0000","last_peered":"2024-09-20T18:56:29.847442+0000","last_clean":"2024-09-20T18:56:29.847442+0000","last_became_active":"2024-09-20T18:49:49.957067+0000","last_became_peered":"2024-09-20T18:49:49.957067+0000","last_unstale":"2024-09-20T18:56:29.847442+0000","last_undegraded":"2024-09-20T18:56:29.847442+0000","last_fullsized":"2024-09-20T18:56:29.847442+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:14.271719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.271719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":303,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.856075+0000","last_change":"2024-09-20T18:51:14.210438+0000","last_active":"2024-09-20T18:56:29.856075+0000","last_peered":"2024-09-20T18:56:29.856075+0000","last_clean":"2024-09-20T18:56:29.856075+0000","last_became_active":"2024-09-20T18:49:51.966915+0000","last_became_peered":"2024-09-20T18:49:51.966915+0000","last_unstale":"2024-09-20T18:56:29.856075+0000","last_undegraded":"2024-09-20T18:56:29.856075+0000","last_fullsized":"2024-09-20T18:56:29.856075+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:14.210358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.210358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":360,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.847521+0000","last_change":"2024-09-20T18:55:28.838698+0000","last_active":"2024-09-20T18:56:29.847521+0000","last_peered":"2024-09-20T18:56:29.847521+0000","last_clean":"2024-09-20T18:56:29.847521+0000","last_became_active":"2024-09-20T18:55:28.827023+0000","last_became_peered":"2024-09-20T18:55:28.827023+0000","last_unstale":"2024-09-20T18:56:29.847521+0000","last_undegraded":"2024-09-20T18:56:29.847521+0000","last_fullsized":"2024-09-20T18:56:29.847521+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:53:53.263976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:53.263976+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":322,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.845977+0000","last_change":"2024-09-20T18:54:42.487465+0000","last_active":"2024-09-20T18:56:29.845977+0000","last_peered":"2024-09-20T18:56:29.845977+0000","last_clean":"2024-09-20T18:56:29.845977+0000","last_became_active":"2024-09-20T18:49:45.944357+0000","last_became_peered":"2024-09-20T18:49:45.944357+0000","last_unstale":"2024-09-20T18:56:29.845977+0000","last_undegraded":"2024-09-20T18:56:29.845977+0000","last_fullsized":"2024-09-20T18:56:29.845977+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:42.487403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:42.487403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":315,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.853393+0000","last_change":"2024-09-20T18:54:18.108239+0000","last_active":"2024-09-20T18:56:29.853393+0000","last_peered":"2024-09-20T18:56:29.853393+0000","last_clean":"2024-09-20T18:56:29.853393+0000","last_became_active":"2024-09-20T18:49:49.962523+0000","last_became_peered":"2024-09-20T18:49:49.962523+0000","last_unstale":"2024-09-20T18:56:29.853393+0000","last_undegraded":"2024-09-20T18:56:29.853393+0000","last_fullsized":"2024-09-20T18:56:29.853393+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:18.108175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:18.108175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":318,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.865374+0000","last_change":"2024-09-20T18:54:26.882441+0000","last_active":"2024-09-20T18:56:29.865374+0000","last_peered":"2024-09-20T18:56:29.865374+0000","last_clean":"2024-09-20T18:56:29.865374+0000","last_became_active":"2024-09-20T18:49:51.967313+0000","last_became_peered":"2024-09-20T18:49:51.967313+0000","last_unstale":"2024-09-20T18:56:29.865374+0000","last_undegraded":"2024-09-20T18:56:29.865374+0000","last_fullsized":"2024-09-20T18:56:29.865374+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:26.882375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:26.882375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":357,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.856252+0000","last_change":"2024-09-20T18:55:28.843363+0000","last_active":"2024-09-20T18:56:29.856252+0000","last_peered":"2024-09-20T18:56:29.856252+0000","last_clean":"2024-09-20T18:56:29.856252+0000","last_became_active":"2024-09-20T18:55:28.830801+0000","last_became_peered":"2024-09-20T18:55:28.830801+0000","last_unstale":"2024-09-20T18:56:29.856252+0000","last_undegraded":"2024-09-20T18:56:29.856252+0000","last_fullsized":"2024-09-20T18:56:29.856252+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:54:21.106677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:21.106677+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":330,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.851400+0000","last_change":"2024-09-20T18:55:19.258723+0000","last_active":"2024-09-20T18:56:29.851400+0000","last_peered":"2024-09-20T18:56:29.851400+0000","last_clean":"2024-09-20T18:56:29.851400+0000","last_became_active":"2024-09-20T18:49:45.942105+0000","last_became_peered":"2024-09-20T18:49:45.942105+0000","last_unstale":"2024-09-20T18:56:29.851400+0000","last_undegraded":"2024-09-20T18:56:29.851400+0000","last_fullsized":"2024-09-20T18:56:29.851400+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:19.258674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:19.258674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:33.782760+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":329,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.844389+0000","last_change":"2024-09-20T18:55:28.825004+0000","last_active":"2024-09-20T18:56:29.844389+0000","last_peered":"2024-09-20T18:56:29.844389+0000","last_clean":"2024-09-20T18:56:29.844389+0000","last_became_active":"2024-09-20T18:55:28.824355+0000","last_became_peered":"2024-09-20T18:55:28.824355+0000","last_unstale":"2024-09-20T18:56:29.844389+0000","last_undegraded":"2024-09-20T18:56:29.844389+0000","last_fullsized":"2024-09-20T18:56:29.844389+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:46.329547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:46.329547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":310,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.853517+0000","last_change":"2024-09-20T18:55:28.825607+0000","last_active":"2024-09-20T18:56:29.853517+0000","last_peered":"2024-09-20T18:56:29.853517+0000","last_clean":"2024-09-20T18:56:29.853517+0000","last_became_active":"2024-09-20T18:55:28.825296+0000","last_became_peered":"2024-09-20T18:55:28.825296+0000","last_unstale":"2024-09-20T18:56:29.853517+0000","last_undegraded":"2024-09-20T18:56:29.853517+0000","last_fullsized":"2024-09-20T18:56:29.853517+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:43.118805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:43.118805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":346,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.858096+0000","last_change":"2024-09-20T18:55:28.836789+0000","last_active":"2024-09-20T18:56:29.858096+0000","last_peered":"2024-09-20T18:56:29.858096+0000","last_clean":"2024-09-20T18:56:29.858096+0000","last_became_active":"2024-09-20T18:55:28.824579+0000","last_became_peered":"2024-09-20T18:55:28.824579+0000","last_unstale":"2024-09-20T18:56:29.858096+0000","last_undegraded":"2024-09-20T18:56:29.858096+0000","last_fullsized":"2024-09-20T18:56:29.858096+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-20T18:55:20.282193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:20.282193+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:40.485847+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":330,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.867218+0000","last_change":"2024-09-20T18:54:55.900487+0000","last_active":"2024-09-20T18:56:29.867218+0000","last_peered":"2024-09-20T18:56:29.867218+0000","last_clean":"2024-09-20T18:56:29.867218+0000","last_became_active":"2024-09-20T18:49:45.940988+0000","last_became_peered":"2024-09-20T18:49:45.940988+0000","last_unstale":"2024-09-20T18:56:29.867218+0000","last_undegraded":"2024-09-20T18:56:29.867218+0000","last_fullsized":"2024-09-20T18:56:29.867218+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:55.899611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:55.899611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":61,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.841049+0000","last_change":"2024-09-20T18:55:28.832864+0000","last_active":"2024-09-20T18:56:29.841049+0000","last_peered":"2024-09-20T18:56:29.841049+0000","last_clean":"2024-09-20T18:56:29.841049+0000","last_became_active":"2024-09-20T18:55:28.832534+0000","last_became_peered":"2024-09-20T18:55:28.832534+0000","last_unstale":"2024-09-20T18:56:29.841049+0000","last_undegraded":"2024-09-20T18:56:29.841049+0000","last_fullsized":"2024-09-20T18:56:29.841049+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:52.635138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:52.635138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":302,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.844553+0000","last_change":"2024-09-20T18:51:09.761331+0000","last_active":"2024-09-20T18:56:29.844553+0000","last_peered":"2024-09-20T18:56:29.844553+0000","last_clean":"2024-09-20T18:56:29.844553+0000","last_became_active":"2024-09-20T18:49:51.968662+0000","last_became_peered":"2024-09-20T18:49:51.968662+0000","last_unstale":"2024-09-20T18:56:29.844553+0000","last_undegraded":"2024-09-20T18:56:29.844553+0000","last_fullsized":"2024-09-20T18:56:29.844553+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:09.761283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:09.761283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":295,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.847663+0000","last_change":"2024-09-20T18:49:51.968476+0000","last_active":"2024-09-20T18:56:29.847663+0000","last_peered":"2024-09-20T18:56:29.847663+0000","last_clean":"2024-09-20T18:56:29.847663+0000","last_became_active":"2024-09-20T18:49:51.968216+0000","last_became_peered":"2024-09-20T18:49:51.968216+0000","last_unstale":"2024-09-20T18:56:29.847663+0000","last_undegraded":"2024-09-20T18:56:29.847663+0000","last_fullsized":"2024-09-20T18:56:29.847663+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":323,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.858306+0000","last_change":"2024-09-20T18:55:28.295445+0000","last_active":"2024-09-20T18:56:29.858306+0000","last_peered":"2024-09-20T18:56:29.858306+0000","last_clean":"2024-09-20T18:56:29.858306+0000","last_became_active":"2024-09-20T18:49:45.937992+0000","last_became_peered":"2024-09-20T18:49:45.937992+0000","last_unstale":"2024-09-20T18:56:29.858306+0000","last_undegraded":"2024-09-20T18:56:29.858306+0000","last_fullsized":"2024-09-20T18:56:29.858306+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:28.295399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:28.295399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:39.921452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":112,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.851907+0000","last_change":"2024-09-20T18:55:28.852468+0000","last_active":"2024-09-20T18:56:29.851907+0000","last_peered":"2024-09-20T18:56:29.851907+0000","last_clean":"2024-09-20T18:56:29.851907+0000","last_became_active":"2024-09-20T18:55:28.834675+0000","last_became_peered":"2024-09-20T18:55:28.834675+0000","last_unstale":"2024-09-20T18:56:29.851907+0000","last_undegraded":"2024-09-20T18:56:29.851907+0000","last_fullsized":"2024-09-20T18:56:29.851907+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:54:44.719676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.719676+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":12,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":307,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.880199+0000","last_change":"2024-09-20T18:51:28.120100+0000","last_active":"2024-09-20T18:56:29.880199+0000","last_peered":"2024-09-20T18:56:29.880199+0000","last_clean":"2024-09-20T18:56:29.880199+0000","last_became_active":"2024-09-20T18:49:49.957582+0000","last_became_peered":"2024-09-20T18:49:49.957582+0000","last_unstale":"2024-09-20T18:56:29.880199+0000","last_undegraded":"2024-09-20T18:56:29.880199+0000","last_fullsized":"2024-09-20T18:56:29.880199+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:28.119694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:28.119694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":6207701016,"num_objects":94723,"num_object_clones":0,"num_object_copies":284169,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":354,"num_objects_misplaced":11200,"num_objects_unfound":0,"num_objects_dirty":94723,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":137455,"num_write_kb":7429633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16754,"num_bytes_recovered":1097990144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":19727069184,"data_stored":19727056968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74288,"ondisk_log_size":74288,"up":48,"acting":45,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":67,"ondisk_log_size":67,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":695,"num_read_kb":452,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":1474,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":12140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":394,"ondisk_log_size":394,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":5275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":405,"num_read_kb":400,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266774,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2079252,"kb_used_data":1907844,"kb_used_omap":1,"kb_used_meta":171390,"kb_avail":91665900,"statfs":{"total":95995035648,"available":93865881600,"internally_reserved":0,"allocated":1953632256,"data_stored":1953011165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":175503818},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":0,"seq":193273528413,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757346,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3267084,"kb_used_data":3033212,"kb_used_omap":1,"kb_used_meta":233854,"kb_avail":90478068,"statfs":{"total":95995035648,"available":92649541632,"internally_reserved":0,"allocated":3106009088,"data_stored":3105395672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":239466949},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986279,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2104672,"kb_used_data":1920784,"kb_used_omap":1,"kb_used_meta":183870,"kb_avail":91640480,"statfs":{"total":95995035648,"available":93839851520,"internally_reserved":0,"allocated":1966882816,"data_stored":1966249928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":188283333},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182510,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2835040,"kb_used_data":2524752,"kb_used_omap":1,"kb_used_meta":310206,"kb_avail":90910112,"statfs":{"total":95995035648,"available":93091954688,"internally_reserved":0,"allocated":2585346048,"data_stored":2584712114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":317651399},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411443,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3331268,"kb_used_data":3007156,"kb_used_omap":1,"kb_used_meta":324094,"kb_avail":90413884,"statfs":{"total":95995035648,"available":92583817216,"internally_reserved":0,"allocated":3079327744,"data_stored":3078722769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":331872709},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,3,2,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574969,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2919096,"kb_used_data":2667112,"kb_used_omap":1,"kb_used_meta":251902,"kb_avail":90826056,"statfs":{"total":95995035648,"available":93005881344,"internally_reserved":0,"allocated":2731122688,"data_stored":2730466605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":257948101},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738495,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2796384,"kb_used_data":2476432,"kb_used_omap":1,"kb_used_meta":319934,"kb_avail":90948768,"statfs":{"total":95995035648,"available":93131538432,"internally_reserved":0,"allocated":2535866368,"data_stored":2535233054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":327612869},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2523660288,"data_stored":2523660288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2719088640,"data_stored":2719088640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3075280896,"data_stored":3075276824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2570715136,"data_stored":2570715136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1959919616,"data_stored":1959919616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3099529216,"data_stored":3099525144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1827405824,"data_stored":1827405824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1951469568,"data_stored":1951465496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T18:56:39.540 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T18:56:39.576 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:56:39.596 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:39.622 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:39.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:39.623 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:39.705 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T18:56:39.706 INFO:teuthology.orchestra.run.smithi142.stdout: "size": 20, 2024-09-20T18:56:39.706 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 600, 2024-09-20T18:56:39.706 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-09-20T18:56:39.706 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.706 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:24945 7.7 7:e97e81cf:::benchmark_data_smithi083_70643_object24944:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:56:39.706 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:06.996434+0000", 2024-09-20T18:56:39.706 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 332.69478237700002, 2024-09-20T18:56:39.706 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.117472831, 2024-09-20T18:56:39.706 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:56:39.706 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:56:39.706 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:56:39.706 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:56:39.707 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:56:39.707 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:56:39.707 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 24945 2024-09-20T18:56:39.707 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.707 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:56:39.707 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.707 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:56:39.707 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.996434+0000", 2024-09-20T18:56:39.707 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:56:39.707 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.707 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.707 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:56:39.707 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.996434+0000", 2024-09-20T18:56:39.708 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:56:39.708 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.708 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.708 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:56:39.708 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.996435+0000", 2024-09-20T18:56:39.708 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4869999999999999e-06 2024-09-20T18:56:39.708 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.708 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.708 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:56:39.708 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.996477+0000", 2024-09-20T18:56:39.708 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.1601999999999999e-05 2024-09-20T18:56:39.708 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.708 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.708 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:56:39.708 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.996488+0000", 2024-09-20T18:56:39.709 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0893999999999999e-05 2024-09-20T18:56:39.709 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.709 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.709 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:56:39.709 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.996495+0000", 2024-09-20T18:56:39.709 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.1720000000000001e-06 2024-09-20T18:56:39.709 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.709 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.709 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:56:39.709 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.996709+0000", 2024-09-20T18:56:39.709 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00021389099999999999 2024-09-20T18:56:39.709 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.709 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.710 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:56:39.710 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.996918+0000", 2024-09-20T18:56:39.710 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00020888100000000001 2024-09-20T18:56:39.710 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.710 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.710 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,3", 2024-09-20T18:56:39.710 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.997122+0000", 2024-09-20T18:56:39.710 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00020468 2024-09-20T18:56:39.710 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.710 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.710 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:56:39.710 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.004028+0000", 2024-09-20T18:56:39.710 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0069058180000000002 2024-09-20T18:56:39.710 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.711 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.711 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:39.711 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.004092+0000", 2024-09-20T18:56:39.711 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4323000000000001e-05 2024-09-20T18:56:39.711 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.711 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.711 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:39.711 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.113828+0000", 2024-09-20T18:56:39.711 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.109735151 2024-09-20T18:56:39.711 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.711 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.711 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:56:39.711 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.113870+0000", 2024-09-20T18:56:39.711 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.2258000000000001e-05 2024-09-20T18:56:39.711 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.712 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.712 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:56:39.712 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.113906+0000", 2024-09-20T18:56:39.712 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6674000000000002e-05 2024-09-20T18:56:39.712 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:56:39.712 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:56:39.712 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:56:39.712 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.712 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.712 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:24951 7.7 7:ebe82687:::benchmark_data_smithi083_70643_object24950:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:56:39.712 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:06.997097+0000", 2024-09-20T18:56:39.712 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 332.69411940999998, 2024-09-20T18:56:39.712 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.12447738799999999, 2024-09-20T18:56:39.712 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:56:39.713 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:56:39.713 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:56:39.713 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:56:39.713 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:56:39.713 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:56:39.713 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 24951 2024-09-20T18:56:39.713 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.713 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:56:39.713 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.713 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:56:39.713 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.997097+0000", 2024-09-20T18:56:39.713 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:56:39.713 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.713 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.714 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:56:39.714 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.997097+0000", 2024-09-20T18:56:39.714 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:56:39.714 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.714 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.714 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:56:39.714 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.997098+0000", 2024-09-20T18:56:39.714 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.082e-06 2024-09-20T18:56:39.714 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.714 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.714 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:56:39.714 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.997210+0000", 2024-09-20T18:56:39.714 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011240500000000001 2024-09-20T18:56:39.714 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.715 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.715 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:56:39.715 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.997223+0000", 2024-09-20T18:56:39.715 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2609000000000001e-05 2024-09-20T18:56:39.715 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.715 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.715 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:56:39.715 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.997230+0000", 2024-09-20T18:56:39.715 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5700000000000004e-06 2024-09-20T18:56:39.715 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.715 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.715 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:56:39.715 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.000085+0000", 2024-09-20T18:56:39.715 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.002855013 2024-09-20T18:56:39.716 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.716 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.716 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:56:39.716 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.000200+0000", 2024-09-20T18:56:39.716 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011476600000000001 2024-09-20T18:56:39.716 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.716 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.716 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,3", 2024-09-20T18:56:39.716 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.001210+0000", 2024-09-20T18:56:39.716 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0010098850000000001 2024-09-20T18:56:39.716 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.716 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.716 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:56:39.716 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.005107+0000", 2024-09-20T18:56:39.717 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0038972730000000001 2024-09-20T18:56:39.717 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.717 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.717 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:39.717 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.103702+0000", 2024-09-20T18:56:39.717 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.098595123000000007 2024-09-20T18:56:39.717 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.717 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.717 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:39.717 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.121471+0000", 2024-09-20T18:56:39.717 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.017768227000000001 2024-09-20T18:56:39.717 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.717 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.718 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:56:39.718 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.121533+0000", 2024-09-20T18:56:39.718 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2489000000000004e-05 2024-09-20T18:56:39.718 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.718 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.718 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:56:39.718 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.121574+0000", 2024-09-20T18:56:39.718 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0945999999999997e-05 2024-09-20T18:56:39.718 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:56:39.718 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:56:39.718 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:56:39.718 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.718 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.718 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:24956 7.7 7:e91ba513:::benchmark_data_smithi083_70643_object24955:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:56:39.719 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:06.999473+0000", 2024-09-20T18:56:39.719 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 332.691743299, 2024-09-20T18:56:39.719 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.122248832, 2024-09-20T18:56:39.719 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:56:39.719 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:56:39.719 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:56:39.720 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:56:39.720 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:56:39.720 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:56:39.720 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 24956 2024-09-20T18:56:39.720 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.720 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:56:39.720 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.720 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:56:39.720 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.999473+0000", 2024-09-20T18:56:39.720 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:56:39.720 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.721 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.721 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:56:39.721 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.999473+0000", 2024-09-20T18:56:39.721 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:56:39.721 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.721 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.721 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:56:39.721 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.999475+0000", 2024-09-20T18:56:39.721 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9750000000000001e-06 2024-09-20T18:56:39.721 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.721 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.721 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:56:39.721 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.999545+0000", 2024-09-20T18:56:39.722 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.0104000000000005e-05 2024-09-20T18:56:39.722 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.722 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.722 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:56:39.722 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.999580+0000", 2024-09-20T18:56:39.722 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.5284000000000003e-05 2024-09-20T18:56:39.722 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.722 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.722 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:56:39.722 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.999590+0000", 2024-09-20T18:56:39.722 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.6739999999999992e-06 2024-09-20T18:56:39.722 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.722 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.722 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:56:39.723 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002183+0000", 2024-09-20T18:56:39.723 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0025934899999999999 2024-09-20T18:56:39.723 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.723 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.723 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:56:39.723 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002933+0000", 2024-09-20T18:56:39.723 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00074923700000000004 2024-09-20T18:56:39.723 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.723 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.723 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,3", 2024-09-20T18:56:39.723 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.003100+0000", 2024-09-20T18:56:39.723 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000167651 2024-09-20T18:56:39.723 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.723 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.724 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:56:39.724 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.005113+0000", 2024-09-20T18:56:39.724 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0020131749999999999 2024-09-20T18:56:39.724 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.724 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.724 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:39.724 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.104767+0000", 2024-09-20T18:56:39.724 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.099653505000000003 2024-09-20T18:56:39.724 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.724 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.724 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:39.724 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.121622+0000", 2024-09-20T18:56:39.724 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.016855098999999998 2024-09-20T18:56:39.724 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.725 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.725 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:56:39.725 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.121681+0000", 2024-09-20T18:56:39.725 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9199e-05 2024-09-20T18:56:39.725 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.725 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.725 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:56:39.725 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.121722+0000", 2024-09-20T18:56:39.725 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0438999999999997e-05 2024-09-20T18:56:39.725 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:56:39.725 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:56:39.725 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:56:39.725 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.726 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.726 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:24960 7.c 7:3cbb268e:::benchmark_data_smithi083_70643_object24959:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:56:39.726 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:07.001793+0000", 2024-09-20T18:56:39.726 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 332.68942255100001, 2024-09-20T18:56:39.726 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.117162266, 2024-09-20T18:56:39.726 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:56:39.726 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:56:39.726 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:56:39.726 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:56:39.726 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:56:39.726 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:56:39.726 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 24960 2024-09-20T18:56:39.726 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.727 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:56:39.727 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.727 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:56:39.727 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.001793+0000", 2024-09-20T18:56:39.727 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:56:39.727 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.727 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.727 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:56:39.727 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.001793+0000", 2024-09-20T18:56:39.727 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:56:39.727 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.727 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.727 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:56:39.727 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.001795+0000", 2024-09-20T18:56:39.728 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3069999999999999e-06 2024-09-20T18:56:39.728 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.728 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.728 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:56:39.728 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.001853+0000", 2024-09-20T18:56:39.728 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8372999999999998e-05 2024-09-20T18:56:39.728 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.728 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.728 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:56:39.728 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.001865+0000", 2024-09-20T18:56:39.728 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2004999999999999e-05 2024-09-20T18:56:39.728 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.728 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.728 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:56:39.729 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.001873+0000", 2024-09-20T18:56:39.729 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.362e-06 2024-09-20T18:56:39.729 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.729 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.729 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:56:39.729 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.001933+0000", 2024-09-20T18:56:39.729 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9998999999999999e-05 2024-09-20T18:56:39.729 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.729 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.729 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:56:39.729 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002069+0000", 2024-09-20T18:56:39.729 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013620799999999999 2024-09-20T18:56:39.729 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.729 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.730 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:56:39.730 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002203+0000", 2024-09-20T18:56:39.730 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000134048 2024-09-20T18:56:39.730 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.730 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.730 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:39.730 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.004679+0000", 2024-09-20T18:56:39.730 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0024765479999999999 2024-09-20T18:56:39.730 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.730 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.730 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:56:39.730 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.005474+0000", 2024-09-20T18:56:39.730 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00079475399999999995 2024-09-20T18:56:39.731 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.731 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.731 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:39.731 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.118886+0000", 2024-09-20T18:56:39.731 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.11341159200000001 2024-09-20T18:56:39.731 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.731 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.731 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:56:39.731 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.118928+0000", 2024-09-20T18:56:39.731 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.2460999999999999e-05 2024-09-20T18:56:39.731 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.731 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.731 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:56:39.731 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.118956+0000", 2024-09-20T18:56:39.731 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.7608999999999999e-05 2024-09-20T18:56:39.732 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:56:39.732 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:56:39.732 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:56:39.732 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.732 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.732 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:24961 7.c 7:35ca9a87:::benchmark_data_smithi083_70643_object24960:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:56:39.732 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:07.001951+0000", 2024-09-20T18:56:39.732 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 332.689265342, 2024-09-20T18:56:39.732 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.11716043299999999, 2024-09-20T18:56:39.732 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:56:39.732 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:56:39.732 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:56:39.732 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:56:39.732 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:56:39.733 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:56:39.733 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 24961 2024-09-20T18:56:39.733 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.733 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:56:39.733 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.733 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:56:39.733 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.001951+0000", 2024-09-20T18:56:39.733 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:56:39.733 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.733 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.733 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:56:39.733 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.001951+0000", 2024-09-20T18:56:39.733 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:56:39.733 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.733 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.733 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:56:39.734 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.001951+0000", 2024-09-20T18:56:39.734 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.7499999999999999e-07 2024-09-20T18:56:39.734 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.734 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.734 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:56:39.734 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002093+0000", 2024-09-20T18:56:39.734 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000141433 2024-09-20T18:56:39.734 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.734 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.734 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:56:39.734 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002103+0000", 2024-09-20T18:56:39.734 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.9979999999999998e-06 2024-09-20T18:56:39.734 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.734 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.734 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:56:39.735 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002110+0000", 2024-09-20T18:56:39.735 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.6170000000000003e-06 2024-09-20T18:56:39.735 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.735 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.735 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:56:39.735 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.003810+0000", 2024-09-20T18:56:39.735 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001700221 2024-09-20T18:56:39.735 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.735 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.735 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:56:39.735 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.003895+0000", 2024-09-20T18:56:39.735 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.4886e-05 2024-09-20T18:56:39.735 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.735 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.736 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:56:39.736 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.003978+0000", 2024-09-20T18:56:39.736 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.3812000000000003e-05 2024-09-20T18:56:39.736 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.736 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.736 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:56:39.736 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.005480+0000", 2024-09-20T18:56:39.736 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001501245 2024-09-20T18:56:39.736 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.736 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.736 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:39.736 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.005508+0000", 2024-09-20T18:56:39.736 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.8668000000000001e-05 2024-09-20T18:56:39.736 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.736 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.737 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:39.737 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.119009+0000", 2024-09-20T18:56:39.737 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.113500961 2024-09-20T18:56:39.737 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.737 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.737 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:56:39.737 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.119073+0000", 2024-09-20T18:56:39.737 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.3378000000000002e-05 2024-09-20T18:56:39.737 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.737 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.737 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:56:39.737 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.119111+0000", 2024-09-20T18:56:39.737 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.8439000000000002e-05 2024-09-20T18:56:39.737 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:56:39.737 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:56:39.738 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:56:39.738 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.738 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.738 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:24962 7.7 7:efd119ca:::benchmark_data_smithi083_70643_object24961:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:56:39.738 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:07.002480+0000", 2024-09-20T18:56:39.738 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 332.688735651, 2024-09-20T18:56:39.738 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.11963515, 2024-09-20T18:56:39.738 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:56:39.738 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:56:39.738 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:56:39.738 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:56:39.738 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:56:39.738 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:56:39.738 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 24962 2024-09-20T18:56:39.739 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.739 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:56:39.739 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.739 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:56:39.739 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002480+0000", 2024-09-20T18:56:39.739 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:56:39.739 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.739 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.739 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:56:39.739 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002480+0000", 2024-09-20T18:56:39.739 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:56:39.739 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.739 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.740 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:56:39.740 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002481+0000", 2024-09-20T18:56:39.740 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0860000000000001e-06 2024-09-20T18:56:39.740 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.740 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.740 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:56:39.740 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002511+0000", 2024-09-20T18:56:39.740 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9499999999999999e-05 2024-09-20T18:56:39.740 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.740 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.740 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:56:39.740 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002522+0000", 2024-09-20T18:56:39.740 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0652000000000001e-05 2024-09-20T18:56:39.740 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.740 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.741 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:56:39.741 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002529+0000", 2024-09-20T18:56:39.741 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.1319999999999996e-06 2024-09-20T18:56:39.741 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.741 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.741 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:56:39.741 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.004125+0000", 2024-09-20T18:56:39.741 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0015959570000000001 2024-09-20T18:56:39.741 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.741 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.741 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:56:39.741 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.004232+0000", 2024-09-20T18:56:39.741 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000107767 2024-09-20T18:56:39.742 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.742 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.742 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,3", 2024-09-20T18:56:39.742 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.004366+0000", 2024-09-20T18:56:39.742 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013360299999999999 2024-09-20T18:56:39.742 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.742 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.742 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:56:39.742 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.005279+0000", 2024-09-20T18:56:39.742 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00091312100000000003 2024-09-20T18:56:39.742 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.742 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.742 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:39.743 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.104849+0000", 2024-09-20T18:56:39.743 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.099569569999999996 2024-09-20T18:56:39.743 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.743 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.743 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:39.743 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.122006+0000", 2024-09-20T18:56:39.743 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.017157531 2024-09-20T18:56:39.743 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.743 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.743 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:56:39.743 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.122071+0000", 2024-09-20T18:56:39.743 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4372e-05 2024-09-20T18:56:39.743 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.743 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.744 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:56:39.744 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.122116+0000", 2024-09-20T18:56:39.744 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.4858999999999999e-05 2024-09-20T18:56:39.744 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:56:39.744 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:56:39.744 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:56:39.744 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.744 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.744 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:24963 7.c 7:38caa154:::benchmark_data_smithi083_70643_object24962:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:56:39.744 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:07.002541+0000", 2024-09-20T18:56:39.744 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 332.68867517199999, 2024-09-20T18:56:39.744 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.11664609500000001, 2024-09-20T18:56:39.744 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:56:39.744 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:56:39.745 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:56:39.745 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:56:39.745 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:56:39.745 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:56:39.745 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 24963 2024-09-20T18:56:39.745 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.745 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:56:39.745 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.745 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:56:39.745 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002541+0000", 2024-09-20T18:56:39.745 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:56:39.745 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.745 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.745 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:56:39.746 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002541+0000", 2024-09-20T18:56:39.746 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:56:39.746 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.746 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.746 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:56:39.746 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002541+0000", 2024-09-20T18:56:39.746 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.2900000000000004e-07 2024-09-20T18:56:39.746 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.746 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.746 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:56:39.746 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002569+0000", 2024-09-20T18:56:39.746 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.7866999999999999e-05 2024-09-20T18:56:39.746 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.746 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.747 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:56:39.747 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002576+0000", 2024-09-20T18:56:39.747 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.0929999999999997e-06 2024-09-20T18:56:39.747 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.747 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.747 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:56:39.747 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.002582+0000", 2024-09-20T18:56:39.747 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.575e-06 2024-09-20T18:56:39.747 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.747 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.747 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:56:39.747 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.004714+0000", 2024-09-20T18:56:39.748 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0021320430000000001 2024-09-20T18:56:39.748 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.748 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.748 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:56:39.748 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.004828+0000", 2024-09-20T18:56:39.748 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000113632 2024-09-20T18:56:39.748 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.748 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.748 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:56:39.748 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.004947+0000", 2024-09-20T18:56:39.748 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000119367 2024-09-20T18:56:39.748 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.748 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.748 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:56:39.748 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.005749+0000", 2024-09-20T18:56:39.749 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000801898 2024-09-20T18:56:39.749 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.749 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.749 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:39.749 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.005930+0000", 2024-09-20T18:56:39.749 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000180686 2024-09-20T18:56:39.749 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.749 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.749 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:39.749 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.119138+0000", 2024-09-20T18:56:39.749 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.11320831200000001 2024-09-20T18:56:39.749 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.749 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.750 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:56:39.750 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.119166+0000", 2024-09-20T18:56:39.750 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.8121000000000001e-05 2024-09-20T18:56:39.750 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.750 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.750 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:56:39.750 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.119187+0000", 2024-09-20T18:56:39.750 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0971999999999998e-05 2024-09-20T18:56:39.750 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:56:39.750 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:56:39.750 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:56:39.750 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.750 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.750 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:25344 7.7 7:ead30b22:::benchmark_data_smithi083_70643_object25343:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:56:39.750 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:07.270133+0000", 2024-09-20T18:56:39.751 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 332.42108302700001, 2024-09-20T18:56:39.751 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.17431155100000001, 2024-09-20T18:56:39.751 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:56:39.751 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:56:39.751 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:56:39.751 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:56:39.751 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:56:39.751 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:56:39.751 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 25344 2024-09-20T18:56:39.751 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.751 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:56:39.751 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.751 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:56:39.751 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.270133+0000", 2024-09-20T18:56:39.751 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:56:39.751 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.752 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.752 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:56:39.752 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.270133+0000", 2024-09-20T18:56:39.752 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:56:39.752 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.752 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.752 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:56:39.752 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.270134+0000", 2024-09-20T18:56:39.752 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4619999999999999e-06 2024-09-20T18:56:39.752 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.752 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.752 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:56:39.752 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.270200+0000", 2024-09-20T18:56:39.752 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5998000000000001e-05 2024-09-20T18:56:39.752 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.752 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.753 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:56:39.753 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.270214+0000", 2024-09-20T18:56:39.753 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3631e-05 2024-09-20T18:56:39.753 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.753 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.753 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:56:39.753 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.270222+0000", 2024-09-20T18:56:39.753 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.6149999999999999e-06 2024-09-20T18:56:39.753 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.753 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.753 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:56:39.753 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.270277+0000", 2024-09-20T18:56:39.753 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4981000000000001e-05 2024-09-20T18:56:39.753 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.753 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.753 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:56:39.754 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.270400+0000", 2024-09-20T18:56:39.754 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012379000000000001 2024-09-20T18:56:39.754 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.754 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.754 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,3", 2024-09-20T18:56:39.754 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.270534+0000", 2024-09-20T18:56:39.754 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013307600000000001 2024-09-20T18:56:39.754 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.754 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.754 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:56:39.754 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.271704+0000", 2024-09-20T18:56:39.754 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001170152 2024-09-20T18:56:39.754 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.754 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.755 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:39.755 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.442076+0000", 2024-09-20T18:56:39.755 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.170371932 2024-09-20T18:56:39.755 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.755 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.755 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:39.755 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.444327+0000", 2024-09-20T18:56:39.755 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.002251454 2024-09-20T18:56:39.755 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.755 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.755 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:56:39.755 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.444386+0000", 2024-09-20T18:56:39.755 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8579999999999998e-05 2024-09-20T18:56:39.755 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.755 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.756 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:56:39.756 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.444445+0000", 2024-09-20T18:56:39.756 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8879999999999999e-05 2024-09-20T18:56:39.756 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:56:39.756 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:56:39.756 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:56:39.756 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.756 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.756 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:26469 7.7 7:edf96f0c:::benchmark_data_smithi083_70643_object26468:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:56:39.756 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:07.996717+0000", 2024-09-20T18:56:39.756 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 331.694499501, 2024-09-20T18:56:39.756 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13515669799999999, 2024-09-20T18:56:39.756 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:56:39.756 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:56:39.757 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:56:39.757 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:56:39.757 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:56:39.757 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:56:39.757 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 26469 2024-09-20T18:56:39.757 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.757 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:56:39.757 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.757 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:56:39.757 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.996717+0000", 2024-09-20T18:56:39.757 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:56:39.757 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.757 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.757 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:56:39.757 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.996717+0000", 2024-09-20T18:56:39.758 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:56:39.758 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.758 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.758 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:56:39.758 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.996718+0000", 2024-09-20T18:56:39.758 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6020000000000001e-06 2024-09-20T18:56:39.758 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.758 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.758 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:56:39.758 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.996814+0000", 2024-09-20T18:56:39.758 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.5480000000000001e-05 2024-09-20T18:56:39.758 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.758 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.758 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:56:39.758 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.996827+0000", 2024-09-20T18:56:39.759 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3749e-05 2024-09-20T18:56:39.759 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.759 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.759 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:56:39.759 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.996835+0000", 2024-09-20T18:56:39.759 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.2309999999999999e-06 2024-09-20T18:56:39.759 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.759 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.759 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:56:39.759 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.996897+0000", 2024-09-20T18:56:39.759 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2515999999999996e-05 2024-09-20T18:56:39.759 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.759 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.759 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:56:39.759 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.996992+0000", 2024-09-20T18:56:39.760 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:39.761 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.4870000000000005e-05 2024-09-20T18:56:39.761 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.761 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.761 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,3", 2024-09-20T18:56:39.761 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.997083+0000", 2024-09-20T18:56:39.761 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.1144999999999996e-05 2024-09-20T18:56:39.761 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.761 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.762 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:56:39.762 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.998108+0000", 2024-09-20T18:56:39.762 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0010251679999999999 2024-09-20T18:56:39.762 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.762 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.762 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:39.762 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.998717+0000", 2024-09-20T18:56:39.762 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00060899999999999995 2024-09-20T18:56:39.762 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.762 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.762 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:39.762 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.131755+0000", 2024-09-20T18:56:39.762 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13303793799999999 2024-09-20T18:56:39.762 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.763 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.763 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:56:39.763 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.131820+0000", 2024-09-20T18:56:39.763 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4678999999999995e-05 2024-09-20T18:56:39.763 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.763 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.763 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:56:39.763 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.131873+0000", 2024-09-20T18:56:39.763 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3319999999999998e-05 2024-09-20T18:56:39.763 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:56:39.763 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:56:39.763 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:56:39.763 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.763 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.763 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:26472 7.c 7:326250cc:::benchmark_data_smithi083_70643_object26471:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:56:39.764 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:07.999252+0000", 2024-09-20T18:56:39.764 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 331.69196395900002, 2024-09-20T18:56:39.764 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.133437312, 2024-09-20T18:56:39.764 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:56:39.764 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:56:39.764 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:56:39.764 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:56:39.764 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:56:39.764 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:56:39.764 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 26472 2024-09-20T18:56:39.764 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.764 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:56:39.764 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.764 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:56:39.764 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.999252+0000", 2024-09-20T18:56:39.765 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:56:39.765 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.765 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.765 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:56:39.765 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.999252+0000", 2024-09-20T18:56:39.765 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:56:39.765 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.765 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.765 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:56:39.765 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.999253+0000", 2024-09-20T18:56:39.765 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.44e-07 2024-09-20T18:56:39.765 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.765 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.765 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:56:39.765 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.999278+0000", 2024-09-20T18:56:39.765 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.5661e-05 2024-09-20T18:56:39.766 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.766 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.766 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:56:39.766 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.999287+0000", 2024-09-20T18:56:39.766 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.2600000000000005e-06 2024-09-20T18:56:39.766 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.766 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.766 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:56:39.766 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.999293+0000", 2024-09-20T18:56:39.766 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4609999999999996e-06 2024-09-20T18:56:39.766 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.766 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.766 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:56:39.766 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.999366+0000", 2024-09-20T18:56:39.766 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.2695999999999997e-05 2024-09-20T18:56:39.767 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.767 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.767 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:56:39.767 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.999475+0000", 2024-09-20T18:56:39.767 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000109472 2024-09-20T18:56:39.767 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.767 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.767 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:56:39.767 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.000511+0000", 2024-09-20T18:56:39.767 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0010355 2024-09-20T18:56:39.767 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.767 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.767 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:56:39.768 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.002097+0000", 2024-09-20T18:56:39.768 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0015861429999999999 2024-09-20T18:56:39.768 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.768 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.768 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:39.768 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.002781+0000", 2024-09-20T18:56:39.768 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000684039 2024-09-20T18:56:39.768 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.768 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.768 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:39.768 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.132602+0000", 2024-09-20T18:56:39.768 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.129820825 2024-09-20T18:56:39.768 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.768 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.769 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:56:39.769 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.132652+0000", 2024-09-20T18:56:39.769 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0139999999999998e-05 2024-09-20T18:56:39.769 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.769 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.769 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:56:39.769 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.132689+0000", 2024-09-20T18:56:39.769 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.7471000000000001e-05 2024-09-20T18:56:39.769 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:56:39.769 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:56:39.769 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:56:39.769 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.769 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.770 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:26478 7.d 7:b12f12bc:::benchmark_data_smithi083_70643_object26477:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:56:39.770 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:08.003662+0000", 2024-09-20T18:56:39.770 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 331.687554095, 2024-09-20T18:56:39.770 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.15004061199999999, 2024-09-20T18:56:39.770 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:56:39.770 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:56:39.770 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:56:39.770 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:56:39.770 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:56:39.770 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:56:39.770 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 26478 2024-09-20T18:56:39.770 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.770 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:56:39.770 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.771 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:56:39.771 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003662+0000", 2024-09-20T18:56:39.771 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:56:39.771 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.771 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.771 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:56:39.771 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003662+0000", 2024-09-20T18:56:39.771 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:56:39.771 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.771 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.771 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:56:39.771 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003664+0000", 2024-09-20T18:56:39.771 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8959999999999999e-06 2024-09-20T18:56:39.771 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.772 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.772 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:56:39.772 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003707+0000", 2024-09-20T18:56:39.772 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3288999999999998e-05 2024-09-20T18:56:39.772 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.772 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.772 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:56:39.772 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003724+0000", 2024-09-20T18:56:39.772 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6529e-05 2024-09-20T18:56:39.772 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.772 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.772 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:56:39.772 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003732+0000", 2024-09-20T18:56:39.773 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.7679999999999997e-06 2024-09-20T18:56:39.773 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.773 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.773 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:56:39.773 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003800+0000", 2024-09-20T18:56:39.773 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.7904999999999994e-05 2024-09-20T18:56:39.773 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.773 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.773 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:56:39.773 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003941+0000", 2024-09-20T18:56:39.773 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000140731 2024-09-20T18:56:39.773 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.773 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.773 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,1", 2024-09-20T18:56:39.774 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.004096+0000", 2024-09-20T18:56:39.774 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015452700000000001 2024-09-20T18:56:39.774 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.774 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.774 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:56:39.774 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.005860+0000", 2024-09-20T18:56:39.774 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001764383 2024-09-20T18:56:39.774 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.774 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.774 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:39.774 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.005912+0000", 2024-09-20T18:56:39.774 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.2519000000000003e-05 2024-09-20T18:56:39.774 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.775 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.775 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:39.775 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.153622+0000", 2024-09-20T18:56:39.775 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.147709641 2024-09-20T18:56:39.775 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.775 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.775 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:56:39.775 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.153662+0000", 2024-09-20T18:56:39.775 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9966000000000003e-05 2024-09-20T18:56:39.775 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.775 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.775 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:56:39.775 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.153703+0000", 2024-09-20T18:56:39.776 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0457999999999998e-05 2024-09-20T18:56:39.776 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:56:39.776 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:56:39.776 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:56:39.776 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.776 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.776 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:26479 7.d 7:bdad06c1:::benchmark_data_smithi083_70643_object26478:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:56:39.776 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:08.003826+0000", 2024-09-20T18:56:39.776 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 331.68738993300002, 2024-09-20T18:56:39.776 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.15140265899999999, 2024-09-20T18:56:39.776 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:56:39.776 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:56:39.776 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:56:39.776 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:56:39.777 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:56:39.777 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:56:39.777 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 26479 2024-09-20T18:56:39.777 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.777 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:56:39.777 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.777 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:56:39.777 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003826+0000", 2024-09-20T18:56:39.777 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:56:39.777 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.777 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.777 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:56:39.777 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003826+0000", 2024-09-20T18:56:39.777 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:56:39.777 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.778 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.778 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:56:39.778 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003827+0000", 2024-09-20T18:56:39.778 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.8100000000000002e-07 2024-09-20T18:56:39.778 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.778 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.778 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:56:39.778 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003876+0000", 2024-09-20T18:56:39.778 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8888999999999999e-05 2024-09-20T18:56:39.778 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.778 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.778 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:56:39.778 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003883+0000", 2024-09-20T18:56:39.778 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.8909999999999998e-06 2024-09-20T18:56:39.779 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.779 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.779 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:56:39.779 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.003887+0000", 2024-09-20T18:56:39.779 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.5909999999999998e-06 2024-09-20T18:56:39.779 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.779 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.779 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:56:39.779 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.005030+0000", 2024-09-20T18:56:39.779 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0011424289999999999 2024-09-20T18:56:39.779 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.779 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.779 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:56:39.779 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.005164+0000", 2024-09-20T18:56:39.780 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000134537 2024-09-20T18:56:39.780 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.780 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.780 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,1", 2024-09-20T18:56:39.780 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.005242+0000", 2024-09-20T18:56:39.780 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.7956999999999999e-05 2024-09-20T18:56:39.780 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.780 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.780 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:39.780 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.006119+0000", 2024-09-20T18:56:39.780 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00087690700000000001 2024-09-20T18:56:39.780 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.780 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.780 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:56:39.781 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.006138+0000", 2024-09-20T18:56:39.781 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8450000000000001e-05 2024-09-20T18:56:39.781 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.781 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.781 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:39.781 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.155098+0000", 2024-09-20T18:56:39.781 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.14896046900000001 2024-09-20T18:56:39.781 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.781 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.781 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:56:39.781 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.155153+0000", 2024-09-20T18:56:39.781 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5058e-05 2024-09-20T18:56:39.781 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.781 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.782 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:56:39.782 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.155229+0000", 2024-09-20T18:56:39.782 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5699999999999997e-05 2024-09-20T18:56:39.782 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:56:39.782 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:56:39.782 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:56:39.782 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.782 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.782 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:26481 7.7 7:eecb08c1:::benchmark_data_smithi083_70643_object26480:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:56:39.782 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:08.006431+0000", 2024-09-20T18:56:39.782 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 331.68478527600001, 2024-09-20T18:56:39.782 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.14993074200000001, 2024-09-20T18:56:39.782 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:56:39.782 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:56:39.783 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:56:39.783 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:56:39.783 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:56:39.783 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:56:39.783 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 26481 2024-09-20T18:56:39.783 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.783 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:56:39.783 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.783 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:56:39.783 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.006431+0000", 2024-09-20T18:56:39.783 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:56:39.783 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.783 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.783 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:56:39.784 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.006431+0000", 2024-09-20T18:56:39.784 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:56:39.784 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.784 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.784 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:56:39.784 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.006432+0000", 2024-09-20T18:56:39.784 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.291e-06 2024-09-20T18:56:39.784 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.784 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.784 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:56:39.784 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.006487+0000", 2024-09-20T18:56:39.784 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5058e-05 2024-09-20T18:56:39.784 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.784 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.784 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:56:39.785 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.006497+0000", 2024-09-20T18:56:39.785 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.8579999999999995e-06 2024-09-20T18:56:39.785 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.785 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.785 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:56:39.785 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.006502+0000", 2024-09-20T18:56:39.785 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7140000000000001e-06 2024-09-20T18:56:39.785 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.785 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.785 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:56:39.785 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.006544+0000", 2024-09-20T18:56:39.785 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.2518000000000002e-05 2024-09-20T18:56:39.785 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.786 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.786 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:56:39.786 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.006636+0000", 2024-09-20T18:56:39.786 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.1865000000000002e-05 2024-09-20T18:56:39.786 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.786 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.786 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,3", 2024-09-20T18:56:39.786 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.006753+0000", 2024-09-20T18:56:39.786 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011691100000000001 2024-09-20T18:56:39.786 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.786 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.786 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:56:39.786 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.007675+0000", 2024-09-20T18:56:39.786 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00092171199999999999 2024-09-20T18:56:39.787 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.787 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.787 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:39.787 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.132841+0000", 2024-09-20T18:56:39.787 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.125165949 2024-09-20T18:56:39.787 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.787 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.787 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:39.787 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.156244+0000", 2024-09-20T18:56:39.787 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.023402896999999999 2024-09-20T18:56:39.787 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.787 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.787 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:56:39.787 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.156321+0000", 2024-09-20T18:56:39.788 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.7451000000000001e-05 2024-09-20T18:56:39.788 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.788 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.788 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:56:39.788 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.156362+0000", 2024-09-20T18:56:39.788 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0518000000000001e-05 2024-09-20T18:56:39.788 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:56:39.788 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:56:39.788 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:56:39.788 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.788 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.788 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:120182 7.d 7:b9824c9e:::benchmark_data_smithi083_70643_object120181:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T18:56:39.788 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:24.962568+0000", 2024-09-20T18:56:39.789 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 254.72864800400001, 2024-09-20T18:56:39.789 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.116782992, 2024-09-20T18:56:39.789 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:56:39.789 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:56:39.789 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:56:39.789 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:56:39.789 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:56:39.789 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:56:39.789 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 120182 2024-09-20T18:56:39.789 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.789 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:56:39.789 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.789 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:56:39.789 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962568+0000", 2024-09-20T18:56:39.790 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:56:39.790 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.790 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.790 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:56:39.790 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962568+0000", 2024-09-20T18:56:39.790 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:56:39.790 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.790 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.790 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:56:39.790 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962569+0000", 2024-09-20T18:56:39.790 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4160000000000001e-06 2024-09-20T18:56:39.790 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.790 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.790 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:56:39.790 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962623+0000", 2024-09-20T18:56:39.791 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4024000000000003e-05 2024-09-20T18:56:39.791 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.791 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.791 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:56:39.791 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962636+0000", 2024-09-20T18:56:39.791 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2149e-05 2024-09-20T18:56:39.791 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.791 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.791 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:56:39.791 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962643+0000", 2024-09-20T18:56:39.791 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.554e-06 2024-09-20T18:56:39.791 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.791 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.791 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:56:39.791 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962684+0000", 2024-09-20T18:56:39.792 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.1139e-05 2024-09-20T18:56:39.792 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.792 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.792 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:56:39.792 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962772+0000", 2024-09-20T18:56:39.792 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.7750999999999994e-05 2024-09-20T18:56:39.792 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.792 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.792 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,1", 2024-09-20T18:56:39.792 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962859+0000", 2024-09-20T18:56:39.792 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.6434999999999995e-05 2024-09-20T18:56:39.792 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.792 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.792 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:56:39.792 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.966204+0000", 2024-09-20T18:56:39.793 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0033459029999999999 2024-09-20T18:56:39.793 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.793 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.793 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:39.793 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.970405+0000", 2024-09-20T18:56:39.793 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0042008729999999999 2024-09-20T18:56:39.793 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.793 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.793 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:39.793 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.079209+0000", 2024-09-20T18:56:39.793 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10880400899999999 2024-09-20T18:56:39.793 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.793 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.793 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:56:39.793 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.079303+0000", 2024-09-20T18:56:39.794 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.4022000000000002e-05 2024-09-20T18:56:39.794 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.794 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.794 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:56:39.794 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.079351+0000", 2024-09-20T18:56:39.794 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7716999999999997e-05 2024-09-20T18:56:39.794 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:56:39.794 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:56:39.794 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:56:39.794 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.794 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.794 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:120184 7.c 7:3c34b9f0:::benchmark_data_smithi083_70643_object120183:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T18:56:39.794 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:24.962667+0000", 2024-09-20T18:56:39.794 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 254.72854917999999, 2024-09-20T18:56:39.794 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.113312679, 2024-09-20T18:56:39.795 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:56:39.795 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:56:39.795 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:56:39.796 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:56:39.796 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:56:39.796 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:56:39.796 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 120184 2024-09-20T18:56:39.796 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.796 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:56:39.796 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.797 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:56:39.797 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962667+0000", 2024-09-20T18:56:39.797 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:56:39.797 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.797 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.797 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:56:39.797 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962667+0000", 2024-09-20T18:56:39.797 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:56:39.797 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.797 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.797 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:56:39.797 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962667+0000", 2024-09-20T18:56:39.797 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8400000000000004e-07 2024-09-20T18:56:39.797 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.798 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.798 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:56:39.798 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962702+0000", 2024-09-20T18:56:39.798 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4734000000000003e-05 2024-09-20T18:56:39.798 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.798 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.798 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:56:39.798 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962710+0000", 2024-09-20T18:56:39.798 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.0900000000000005e-06 2024-09-20T18:56:39.798 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.798 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.798 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:56:39.799 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962716+0000", 2024-09-20T18:56:39.799 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.1820000000000003e-06 2024-09-20T18:56:39.799 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.799 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.799 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:56:39.799 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962771+0000", 2024-09-20T18:56:39.799 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4447000000000002e-05 2024-09-20T18:56:39.799 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.799 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.799 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:56:39.799 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962883+0000", 2024-09-20T18:56:39.799 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000111996 2024-09-20T18:56:39.799 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.799 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.800 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:56:39.800 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.963021+0000", 2024-09-20T18:56:39.800 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013835799999999999 2024-09-20T18:56:39.800 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.800 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.800 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:39.800 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.966241+0000", 2024-09-20T18:56:39.800 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0032201299999999999 2024-09-20T18:56:39.800 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.800 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.800 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:56:39.800 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.966257+0000", 2024-09-20T18:56:39.800 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5325e-05 2024-09-20T18:56:39.800 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.801 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.801 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:39.801 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.075873+0000", 2024-09-20T18:56:39.801 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.109616093 2024-09-20T18:56:39.801 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.801 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.801 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:56:39.801 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.075932+0000", 2024-09-20T18:56:39.801 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9487999999999997e-05 2024-09-20T18:56:39.801 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.801 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.801 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:56:39.801 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.075980+0000", 2024-09-20T18:56:39.801 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7252000000000001e-05 2024-09-20T18:56:39.801 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:56:39.802 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:56:39.802 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:56:39.802 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.802 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.802 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.14808.0:120191 7.b e121/83 7:d45eee6a:::benchmark_data_smithi083_70643_object120190:head v 121'7491, mlcod=121'7489)", 2024-09-20T18:56:39.802 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:24.971948+0000", 2024-09-20T18:56:39.802 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 254.719268212, 2024-09-20T18:56:39.802 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.111765014, 2024-09-20T18:56:39.802 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:56:39.802 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:56:39.802 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:56:39.802 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:56:39.802 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.803 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:56:39.803 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.971948+0000", 2024-09-20T18:56:39.803 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:56:39.803 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.803 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.803 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:56:39.803 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.971948+0000", 2024-09-20T18:56:39.803 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:56:39.803 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.803 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.803 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:56:39.803 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.971949+0000", 2024-09-20T18:56:39.803 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.7400000000000002e-07 2024-09-20T18:56:39.803 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.803 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.804 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:56:39.804 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.971990+0000", 2024-09-20T18:56:39.804 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.1825e-05 2024-09-20T18:56:39.804 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.804 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.804 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:56:39.804 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.073923+0000", 2024-09-20T18:56:39.804 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.101932732 2024-09-20T18:56:39.804 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.804 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.804 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:56:39.804 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.073931+0000", 2024-09-20T18:56:39.804 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.1510000000000004e-06 2024-09-20T18:56:39.804 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.804 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.805 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:56:39.805 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.073984+0000", 2024-09-20T18:56:39.805 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.2679000000000002e-05 2024-09-20T18:56:39.805 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.805 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.805 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:56:39.805 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.074004+0000", 2024-09-20T18:56:39.805 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9882e-05 2024-09-20T18:56:39.805 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.805 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.805 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:56:39.805 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.083676+0000", 2024-09-20T18:56:39.805 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0096721900000000006 2024-09-20T18:56:39.805 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.806 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.806 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:56:39.806 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.083713+0000", 2024-09-20T18:56:39.806 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6780999999999999e-05 2024-09-20T18:56:39.806 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:56:39.806 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:56:39.806 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:56:39.806 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.806 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.806 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:120194 7.d 7:b6298d4c:::benchmark_data_smithi083_70643_object120193:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T18:56:39.806 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:24.977580+0000", 2024-09-20T18:56:39.806 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 254.713636393, 2024-09-20T18:56:39.806 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.111515778, 2024-09-20T18:56:39.806 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:56:39.807 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:56:39.807 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:56:39.807 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:56:39.807 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:56:39.807 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:56:39.807 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 120194 2024-09-20T18:56:39.807 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.807 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:56:39.807 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.807 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:56:39.807 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.977580+0000", 2024-09-20T18:56:39.807 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:56:39.807 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.807 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.808 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:56:39.808 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.977580+0000", 2024-09-20T18:56:39.808 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:56:39.808 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.808 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.808 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:56:39.808 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.977582+0000", 2024-09-20T18:56:39.808 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9520000000000001e-06 2024-09-20T18:56:39.808 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.808 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.808 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:56:39.808 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.977642+0000", 2024-09-20T18:56:39.808 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0204000000000002e-05 2024-09-20T18:56:39.808 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.809 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.809 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:56:39.809 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.072252+0000", 2024-09-20T18:56:39.809 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.094610194999999994 2024-09-20T18:56:39.809 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.809 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.809 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:56:39.809 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.072287+0000", 2024-09-20T18:56:39.809 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4542999999999998e-05 2024-09-20T18:56:39.809 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.809 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.809 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:56:39.809 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.072751+0000", 2024-09-20T18:56:39.809 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000464289 2024-09-20T18:56:39.810 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.810 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.810 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:56:39.810 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.072883+0000", 2024-09-20T18:56:39.810 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013254099999999999 2024-09-20T18:56:39.810 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.810 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.810 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,1", 2024-09-20T18:56:39.810 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.073544+0000", 2024-09-20T18:56:39.810 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00066074700000000005 2024-09-20T18:56:39.810 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.810 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.810 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:39.810 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.088934+0000", 2024-09-20T18:56:39.811 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.015389551 2024-09-20T18:56:39.811 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.811 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.811 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:39.811 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.088971+0000", 2024-09-20T18:56:39.811 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.7249000000000002e-05 2024-09-20T18:56:39.811 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.811 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.811 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:56:39.811 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.088990+0000", 2024-09-20T18:56:39.811 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8770000000000002e-05 2024-09-20T18:56:39.811 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.812 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.812 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:56:39.812 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.089045+0000", 2024-09-20T18:56:39.812 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5386999999999997e-05 2024-09-20T18:56:39.812 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.812 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.812 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:56:39.812 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.089095+0000", 2024-09-20T18:56:39.812 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0349999999999997e-05 2024-09-20T18:56:39.812 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:56:39.812 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:56:39.812 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:56:39.812 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.813 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.813 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:139218 7.d 7:bef6f051:::benchmark_data_smithi083_70643_object16416:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-20T18:56:39.813 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:42.802191+0000", 2024-09-20T18:56:39.813 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 236.889025197, 2024-09-20T18:56:39.813 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.12718874399999999, 2024-09-20T18:56:39.813 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:56:39.813 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:56:39.813 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:56:39.813 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:56:39.813 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:56:39.813 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:56:39.813 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 139218 2024-09-20T18:56:39.814 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.814 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:56:39.814 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.814 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:56:39.814 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.802191+0000", 2024-09-20T18:56:39.814 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:56:39.814 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.814 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.814 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:56:39.814 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.802191+0000", 2024-09-20T18:56:39.814 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:56:39.814 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.814 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.814 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:56:39.815 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.802192+0000", 2024-09-20T18:56:39.815 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0759999999999999e-06 2024-09-20T18:56:39.815 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.815 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.815 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:56:39.815 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.802196+0000", 2024-09-20T18:56:39.815 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.7309999999999999e-06 2024-09-20T18:56:39.815 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.815 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.815 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:56:39.815 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.802203+0000", 2024-09-20T18:56:39.815 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.4599999999999997e-06 2024-09-20T18:56:39.815 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.815 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.816 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:56:39.816 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.802209+0000", 2024-09-20T18:56:39.816 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5550000000000002e-06 2024-09-20T18:56:39.816 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.816 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.816 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:56:39.816 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.802250+0000", 2024-09-20T18:56:39.816 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.1502999999999999e-05 2024-09-20T18:56:39.816 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.816 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.816 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:56:39.816 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.802322+0000", 2024-09-20T18:56:39.816 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.1723000000000004e-05 2024-09-20T18:56:39.817 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.817 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.817 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,1", 2024-09-20T18:56:39.817 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.802390+0000", 2024-09-20T18:56:39.817 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.8120000000000005e-05 2024-09-20T18:56:39.817 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.817 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.817 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:39.817 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.807367+0000", 2024-09-20T18:56:39.817 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0049773769999999998 2024-09-20T18:56:39.817 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.817 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.817 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:56:39.817 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.807382+0000", 2024-09-20T18:56:39.818 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4667e-05 2024-09-20T18:56:39.818 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.818 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.818 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:39.818 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.929326+0000", 2024-09-20T18:56:39.818 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.121944356 2024-09-20T18:56:39.818 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.818 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.818 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:56:39.818 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.929356+0000", 2024-09-20T18:56:39.818 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9995e-05 2024-09-20T18:56:39.818 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.818 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.819 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:56:39.819 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.929380+0000", 2024-09-20T18:56:39.819 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3181e-05 2024-09-20T18:56:39.819 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:56:39.819 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:56:39.819 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:56:39.819 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.819 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.819 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:139220 7.d 7:b6a17747:::benchmark_data_smithi083_70643_object16418:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-20T18:56:39.819 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:42.803402+0000", 2024-09-20T18:56:39.819 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 236.88781400600001, 2024-09-20T18:56:39.819 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.129377032, 2024-09-20T18:56:39.819 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:56:39.819 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:56:39.820 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:56:39.820 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:56:39.820 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:56:39.820 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:56:39.820 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 139220 2024-09-20T18:56:39.820 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.820 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:56:39.820 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.820 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:56:39.820 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.803402+0000", 2024-09-20T18:56:39.820 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:56:39.820 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.820 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.820 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:56:39.820 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.803402+0000", 2024-09-20T18:56:39.820 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:56:39.821 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.821 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.821 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:56:39.821 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.803403+0000", 2024-09-20T18:56:39.821 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.097e-06 2024-09-20T18:56:39.821 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.821 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.821 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:56:39.821 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.803408+0000", 2024-09-20T18:56:39.821 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.561e-06 2024-09-20T18:56:39.821 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.821 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.821 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:56:39.821 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.803419+0000", 2024-09-20T18:56:39.821 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1591e-05 2024-09-20T18:56:39.822 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.822 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.822 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:56:39.822 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.803426+0000", 2024-09-20T18:56:39.822 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.9160000000000004e-06 2024-09-20T18:56:39.822 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.822 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.822 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:56:39.822 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.805298+0000", 2024-09-20T18:56:39.822 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0018719780000000001 2024-09-20T18:56:39.822 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.822 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.822 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:56:39.822 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.805418+0000", 2024-09-20T18:56:39.823 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000119739 2024-09-20T18:56:39.823 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.823 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.823 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,1", 2024-09-20T18:56:39.823 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.805543+0000", 2024-09-20T18:56:39.823 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012481699999999999 2024-09-20T18:56:39.823 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.823 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.823 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:56:39.823 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.807413+0000", 2024-09-20T18:56:39.823 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001870678 2024-09-20T18:56:39.823 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.823 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.823 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:39.823 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.808024+0000", 2024-09-20T18:56:39.824 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00061090700000000005 2024-09-20T18:56:39.824 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.824 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.824 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:39.824 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.932673+0000", 2024-09-20T18:56:39.824 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.12464900299999999 2024-09-20T18:56:39.824 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.824 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.824 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:56:39.824 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.932739+0000", 2024-09-20T18:56:39.824 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5979e-05 2024-09-20T18:56:39.824 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.824 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.824 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:56:39.825 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.932779+0000", 2024-09-20T18:56:39.825 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9765999999999998e-05 2024-09-20T18:56:39.825 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:56:39.825 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:56:39.825 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:56:39.825 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.825 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.825 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:139224 7.7 7:e3ad72f8:::benchmark_data_smithi083_70643_object16422:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-20T18:56:39.825 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:42.806758+0000", 2024-09-20T18:56:39.825 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 236.884458, 2024-09-20T18:56:39.825 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.121962579, 2024-09-20T18:56:39.825 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:56:39.825 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:56:39.825 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:56:39.825 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:56:39.826 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:56:39.826 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:56:39.826 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 139224 2024-09-20T18:56:39.826 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.826 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:56:39.826 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.826 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:56:39.826 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806758+0000", 2024-09-20T18:56:39.826 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:56:39.826 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.826 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.826 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:56:39.826 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806758+0000", 2024-09-20T18:56:39.826 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:56:39.827 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.827 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.827 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:56:39.827 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806759+0000", 2024-09-20T18:56:39.827 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.065e-06 2024-09-20T18:56:39.827 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.827 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.827 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:56:39.827 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806764+0000", 2024-09-20T18:56:39.827 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8010000000000003e-06 2024-09-20T18:56:39.827 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.827 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.827 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:56:39.827 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806777+0000", 2024-09-20T18:56:39.828 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3292000000000001e-05 2024-09-20T18:56:39.828 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.828 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.828 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:56:39.828 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806785+0000", 2024-09-20T18:56:39.828 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.9959999999999993e-06 2024-09-20T18:56:39.828 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.828 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.828 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:56:39.828 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806840+0000", 2024-09-20T18:56:39.828 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4308000000000002e-05 2024-09-20T18:56:39.828 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.828 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.828 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:56:39.829 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806948+0000", 2024-09-20T18:56:39.829 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00010843799999999999 2024-09-20T18:56:39.829 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.829 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.829 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,3", 2024-09-20T18:56:39.829 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.807066+0000", 2024-09-20T18:56:39.829 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000117766 2024-09-20T18:56:39.829 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.829 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.829 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:56:39.829 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.809368+0000", 2024-09-20T18:56:39.829 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0023018130000000002 2024-09-20T18:56:39.829 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.830 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.830 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:39.830 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.809785+0000", 2024-09-20T18:56:39.830 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00041736699999999998 2024-09-20T18:56:39.830 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.830 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.830 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:56:39.830 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.928629+0000", 2024-09-20T18:56:39.830 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.118843746 2024-09-20T18:56:39.830 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.830 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.830 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:56:39.830 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.928687+0000", 2024-09-20T18:56:39.830 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8418999999999998e-05 2024-09-20T18:56:39.831 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:39.831 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:39.831 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:56:39.831 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.928721+0000", 2024-09-20T18:56:39.831 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.3568000000000001e-05 2024-09-20T18:56:39.831 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:56:39.831 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:56:39.831 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:56:39.831 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:56:39.831 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:56:39.831 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T18:56:39.851 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:39.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:39.852 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:39.953 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:56:39.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:39.996 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:39.996 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:56:40.019 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:39 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:56:40.019 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:39 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:56:40.019 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:39 smithi083 ceph-mon[29421]: osdmap e209: 8 total, 8 up, 7 in 2024-09-20T18:56:40.019 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:39 smithi083 ceph-mon[29421]: pgmap v330: 148 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 144 active+clean; 5.8 GiB data, 18 GiB used, 607 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 730 op/s; 354/284766 objects degraded (0.124%); 11200/284766 objects misplaced (3.933%); 11 MiB/s, 168 objects/s recovering 2024-09-20T18:56:40.019 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:39 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:56:40.020 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:39 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:56:40.020 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:39 smithi083 ceph-mon[36260]: osdmap e209: 8 total, 8 up, 7 in 2024-09-20T18:56:40.020 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:39 smithi083 ceph-mon[36260]: pgmap v330: 148 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 144 active+clean; 5.8 GiB data, 18 GiB used, 607 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 730 op/s; 354/284766 objects degraded (0.124%); 11200/284766 objects misplaced (3.933%); 11 MiB/s, 168 objects/s recovering 2024-09-20T18:56:40.097 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:56:40.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:40.139 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:40.140 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:56:40.226 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:56:40.240 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:40.270 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:39 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:56:40.270 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:39 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:56:40.270 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:39 smithi142 ceph-mon[32508]: osdmap e209: 8 total, 8 up, 7 in 2024-09-20T18:56:40.270 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:39 smithi142 ceph-mon[32508]: pgmap v330: 148 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 144 active+clean; 5.8 GiB data, 18 GiB used, 607 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 730 op/s; 354/284766 objects degraded (0.124%); 11200/284766 objects misplaced (3.933%); 11 MiB/s, 168 objects/s recovering 2024-09-20T18:56:40.310 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:40.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:40.311 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:40.412 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:40.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:40.465 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:40.465 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:40.566 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:40.594 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:40.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:40.595 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:40.696 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:40.709 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-20T18:56:40.725 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:40.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:40.726 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:40.760 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T18:56:40.826 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:56:40.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:40.869 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:40.869 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:56:40.969 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:40 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1170931111' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:56:40.970 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:40 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1170931111' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:56:40.970 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:56:40 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:56:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:56:40.970 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:40.995 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:40.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:40.996 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:41.020 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:40 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1170931111' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:56:41.020 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:40 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:56:41.020 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:40 smithi142 ceph-mon[32508]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T18:56:41.020 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:40 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T18:56:41.020 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:40 smithi142 ceph-mon[32508]: osdmap e210: 8 total, 8 up, 7 in 2024-09-20T18:56:41.098 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:41.127 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:41.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:41.128 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:41.229 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:41.257 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:41.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:41.258 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:41.300 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:40 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:56:41.300 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:40 smithi083 ceph-mon[29421]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T18:56:41.300 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:40 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T18:56:41.300 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:40 smithi083 ceph-mon[29421]: osdmap e210: 8 total, 8 up, 7 in 2024-09-20T18:56:41.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:40 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:56:41.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:40 smithi083 ceph-mon[36260]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T18:56:41.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:40 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T18:56:41.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:40 smithi083 ceph-mon[36260]: osdmap e210: 8 total, 8 up, 7 in 2024-09-20T18:56:41.359 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:41.385 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:41.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:41.386 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:41.487 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:41.519 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:41.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:41.520 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:41.621 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:41.648 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:41.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:41.649 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:41.749 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:56:41.798 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:41.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:41.799 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:56:41.899 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:41.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:41.928 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:41.928 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:41.962 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:56:41.963 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-20T18:56:42.028 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:42.061 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:42.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:42.063 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:42.164 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:42.196 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:42.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:42.197 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:42.298 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:56:42.334 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:42.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:42.335 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:56:42.436 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:56:42.472 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:42.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:42.473 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:56:42.573 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:42.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:42.605 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:42.606 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:42.676 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-20T18:56:42.706 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:42.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:42.748 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:42.748 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:42.849 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:42.883 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:42.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:42.884 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:42.985 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:43.017 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:43.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:43.018 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:43.020 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:42 smithi142 ceph-mon[32508]: pgmap v332: 148 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 144 active+clean; 5.8 GiB data, 18 GiB used, 607 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s; 354/284766 objects degraded (0.124%); 11200/284766 objects misplaced (3.933%) 2024-09-20T18:56:43.050 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:42 smithi083 ceph-mon[29421]: pgmap v332: 148 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 144 active+clean; 5.8 GiB data, 18 GiB used, 607 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s; 354/284766 objects degraded (0.124%); 11200/284766 objects misplaced (3.933%) 2024-09-20T18:56:43.050 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:42 smithi083 ceph-mon[36260]: pgmap v332: 148 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 144 active+clean; 5.8 GiB data, 18 GiB used, 607 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s; 354/284766 objects degraded (0.124%); 11200/284766 objects misplaced (3.933%) 2024-09-20T18:56:43.051 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:56:43.119 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:43.147 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:43.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:43.150 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:43.250 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:43.282 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:43.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:43.282 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:43.383 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:43.413 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:43.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:43.413 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:43.514 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:56:43.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:43.559 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:43.560 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:56:43.613 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T18:56:43.614 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-09-20T18:56:43.614 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:43.614 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.24955.0:144119 8.8 e210/160 8:16b78a9a:::benchmark_data_smithi083_97422_object28028:head v 210'9049, mlcod=210'9046)", 2024-09-20T18:56:43.614 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:56:43.597387+0000", 2024-09-20T18:56:43.614 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.00027682999999999999, 2024-09-20T18:56:43.614 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000303269, 2024-09-20T18:56:43.614 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:56:43.614 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:56:43.614 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-09-20T18:56:43.614 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:56:43.614 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:43.615 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:56:43.615 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:43.597387+0000", 2024-09-20T18:56:43.615 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:56:43.615 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:43.615 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:43.615 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:56:43.615 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:43.597387+0000", 2024-09-20T18:56:43.615 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:56:43.615 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:43.615 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:43.615 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:56:43.616 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:43.597388+0000", 2024-09-20T18:56:43.616 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.204e-06 2024-09-20T18:56:43.616 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:43.616 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:43.616 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:56:43.616 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:43.597396+0000", 2024-09-20T18:56:43.617 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.7149999999999998e-06 2024-09-20T18:56:43.617 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:43.617 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:43.617 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:56:43.617 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:43.597414+0000", 2024-09-20T18:56:43.617 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8301999999999999e-05 2024-09-20T18:56:43.617 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:43.617 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:43.618 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:56:43.618 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:43.597423+0000", 2024-09-20T18:56:43.618 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.1030000000000008e-06 2024-09-20T18:56:43.618 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:43.618 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:43.618 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:56:43.618 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:43.597497+0000", 2024-09-20T18:56:43.618 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.4239999999999994e-05 2024-09-20T18:56:43.618 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:56:43.618 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:56:43.618 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:56:43.618 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:43.597518+0000", 2024-09-20T18:56:43.618 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1046000000000001e-05 2024-09-20T18:56:43.618 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:56:43.619 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:56:43.619 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:56:43.619 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:56:43.619 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-09-20T18:56:43.619 INFO:teuthology.orchestra.run.smithi142.stdout: "num_ops": 1 2024-09-20T18:56:43.619 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T18:56:43.660 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:43.693 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:43.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:43.697 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:43.732 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:43 smithi142 ceph-mon[32508]: from='client.25111 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:56:43.732 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:43 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:56:43.798 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:43.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:43.822 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:43.822 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:43.923 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:56:43.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:43.961 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:43.961 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:56:44.050 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:43 smithi083 ceph-mon[29421]: from='client.25111 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:56:44.050 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:43 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:56:44.050 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:43 smithi083 ceph-mon[36260]: from='client.25111 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:56:44.050 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:43 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:56:44.062 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:44.085 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:44.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:44.086 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:44.187 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:44.193 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:56:44.209 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:44.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:44.210 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:44.219 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-09-20T18:56:40.903234+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6208297523,"num_objects":94922,"num_object_clones":0,"num_object_copies":284766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":354,"num_objects_misplaced":11200,"num_objects_unfound":0,"num_objects_dirty":94922,"num_whiteouts":0,"num_read":1256,"num_read_kb":1113,"num_write":138004,"num_write_kb":7431306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16849,"num_bytes_recovered":1099172809,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74844,"ondisk_log_size":74844,"up":444,"acting":441,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":19252916,"kb_used_data":17449084,"kb_used_omap":10,"kb_used_meta":1803573,"kb_avail":636963148,"statfs":{"total":671965249536,"available":652250263552,"internally_reserved":0,"allocated":17867862016,"data_stored":17863445139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11158,"internal_metadata":1846858858},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,4,3,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.039486"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":331,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.866961+0000","last_change":"2024-09-20T18:55:28.037494+0000","last_active":"2024-09-20T18:56:29.866961+0000","last_peered":"2024-09-20T18:56:29.866961+0000","last_clean":"2024-09-20T18:56:29.866961+0000","last_became_active":"2024-09-20T18:49:45.943201+0000","last_became_peered":"2024-09-20T18:49:45.943201+0000","last_unstale":"2024-09-20T18:56:29.866961+0000","last_undegraded":"2024-09-20T18:56:29.866961+0000","last_fullsized":"2024-09-20T18:56:29.866961+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:28.037449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:28.037449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:53.606960+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":377,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.841264+0000","last_change":"2024-09-20T18:55:28.838729+0000","last_active":"2024-09-20T18:56:29.841264+0000","last_peered":"2024-09-20T18:56:29.841264+0000","last_clean":"2024-09-20T18:56:29.841264+0000","last_became_active":"2024-09-20T18:55:28.819637+0000","last_became_peered":"2024-09-20T18:55:28.819637+0000","last_unstale":"2024-09-20T18:56:29.841264+0000","last_undegraded":"2024-09-20T18:56:29.841264+0000","last_fullsized":"2024-09-20T18:56:29.841264+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:50.231620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:50.231620+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":323,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.847371+0000","last_change":"2024-09-20T18:54:25.361988+0000","last_active":"2024-09-20T18:56:29.847371+0000","last_peered":"2024-09-20T18:56:29.847371+0000","last_clean":"2024-09-20T18:56:29.847371+0000","last_became_active":"2024-09-20T18:49:49.959514+0000","last_became_peered":"2024-09-20T18:49:49.959514+0000","last_unstale":"2024-09-20T18:56:29.847371+0000","last_undegraded":"2024-09-20T18:56:29.847371+0000","last_fullsized":"2024-09-20T18:56:29.847371+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:25.361935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:25.361935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":319,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.841796+0000","last_change":"2024-09-20T18:54:15.221404+0000","last_active":"2024-09-20T18:56:29.841796+0000","last_peered":"2024-09-20T18:56:29.841796+0000","last_clean":"2024-09-20T18:56:29.841796+0000","last_became_active":"2024-09-20T18:49:51.969421+0000","last_became_peered":"2024-09-20T18:49:51.969421+0000","last_unstale":"2024-09-20T18:56:29.841796+0000","last_undegraded":"2024-09-20T18:56:29.841796+0000","last_fullsized":"2024-09-20T18:56:29.841796+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:15.221342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:15.221342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":326,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.853237+0000","last_change":"2024-09-20T18:55:28.822034+0000","last_active":"2024-09-20T18:56:29.853237+0000","last_peered":"2024-09-20T18:56:29.853237+0000","last_clean":"2024-09-20T18:56:29.853237+0000","last_became_active":"2024-09-20T18:55:28.821717+0000","last_became_peered":"2024-09-20T18:55:28.821717+0000","last_unstale":"2024-09-20T18:56:29.853237+0000","last_undegraded":"2024-09-20T18:56:29.853237+0000","last_fullsized":"2024-09-20T18:56:29.853237+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.216188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.216188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":323,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.853823+0000","last_change":"2024-09-20T18:55:28.825144+0000","last_active":"2024-09-20T18:56:29.853823+0000","last_peered":"2024-09-20T18:56:29.853823+0000","last_clean":"2024-09-20T18:56:29.853823+0000","last_became_active":"2024-09-20T18:55:28.824826+0000","last_became_peered":"2024-09-20T18:55:28.824826+0000","last_unstale":"2024-09-20T18:56:29.853823+0000","last_undegraded":"2024-09-20T18:56:29.853823+0000","last_fullsized":"2024-09-20T18:56:29.853823+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:25.208073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:25.208073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":336,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.856493+0000","last_change":"2024-09-20T18:51:07.130137+0000","last_active":"2024-09-20T18:56:29.856493+0000","last_peered":"2024-09-20T18:56:29.856493+0000","last_clean":"2024-09-20T18:56:29.856493+0000","last_became_active":"2024-09-20T18:49:47.952150+0000","last_became_peered":"2024-09-20T18:49:47.952150+0000","last_unstale":"2024-09-20T18:56:29.856493+0000","last_undegraded":"2024-09-20T18:56:29.856493+0000","last_fullsized":"2024-09-20T18:56:29.856493+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:51:07.130062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:07.130062+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":322,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.841174+0000","last_change":"2024-09-20T18:55:28.820753+0000","last_active":"2024-09-20T18:56:29.841174+0000","last_peered":"2024-09-20T18:56:29.841174+0000","last_clean":"2024-09-20T18:56:29.841174+0000","last_became_active":"2024-09-20T18:55:28.820362+0000","last_became_peered":"2024-09-20T18:55:28.820362+0000","last_unstale":"2024-09-20T18:56:29.841174+0000","last_undegraded":"2024-09-20T18:56:29.841174+0000","last_fullsized":"2024-09-20T18:56:29.841174+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:11.246922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:11.246922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"59'1","reported_seq":355,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.852615+0000","last_change":"2024-09-20T18:55:29.340140+0000","last_active":"2024-09-20T18:56:29.852615+0000","last_peered":"2024-09-20T18:56:29.852615+0000","last_clean":"2024-09-20T18:56:29.852615+0000","last_became_active":"2024-09-20T18:55:28.833386+0000","last_became_peered":"2024-09-20T18:55:28.833386+0000","last_unstale":"2024-09-20T18:56:29.852615+0000","last_undegraded":"2024-09-20T18:56:29.852615+0000","last_fullsized":"2024-09-20T18:56:29.852615+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:55:29.340083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:29.340083+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:41.552449+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":343,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.841956+0000","last_change":"2024-09-20T18:54:53.241889+0000","last_active":"2024-09-20T18:56:29.841956+0000","last_peered":"2024-09-20T18:56:29.841956+0000","last_clean":"2024-09-20T18:56:29.841956+0000","last_became_active":"2024-09-20T18:49:47.947448+0000","last_became_peered":"2024-09-20T18:49:47.947448+0000","last_unstale":"2024-09-20T18:56:29.841956+0000","last_undegraded":"2024-09-20T18:56:29.841956+0000","last_fullsized":"2024-09-20T18:56:29.841956+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:53.241802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:53.241802+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":314,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.852655+0000","last_change":"2024-09-20T18:52:36.183932+0000","last_active":"2024-09-20T18:56:29.852655+0000","last_peered":"2024-09-20T18:56:29.852655+0000","last_clean":"2024-09-20T18:56:29.852655+0000","last_became_active":"2024-09-20T18:49:49.961188+0000","last_became_peered":"2024-09-20T18:49:49.961188+0000","last_unstale":"2024-09-20T18:56:29.852655+0000","last_undegraded":"2024-09-20T18:56:29.852655+0000","last_fullsized":"2024-09-20T18:56:29.852655+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:36.183875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:36.183875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":294,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.865546+0000","last_change":"2024-09-20T18:49:51.966957+0000","last_active":"2024-09-20T18:56:29.865546+0000","last_peered":"2024-09-20T18:56:29.865546+0000","last_clean":"2024-09-20T18:56:29.865546+0000","last_became_active":"2024-09-20T18:49:51.966715+0000","last_became_peered":"2024-09-20T18:49:51.966715+0000","last_unstale":"2024-09-20T18:56:29.865546+0000","last_undegraded":"2024-09-20T18:56:29.865546+0000","last_fullsized":"2024-09-20T18:56:29.865546+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":322,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.851436+0000","last_change":"2024-09-20T18:55:15.252839+0000","last_active":"2024-09-20T18:56:29.851436+0000","last_peered":"2024-09-20T18:56:29.851436+0000","last_clean":"2024-09-20T18:56:29.851436+0000","last_became_active":"2024-09-20T18:49:45.944703+0000","last_became_peered":"2024-09-20T18:49:45.944703+0000","last_unstale":"2024-09-20T18:56:29.851436+0000","last_undegraded":"2024-09-20T18:56:29.851436+0000","last_fullsized":"2024-09-20T18:56:29.851436+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:15.252782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:15.252782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:43.639912+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":347,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.841800+0000","last_change":"2024-09-20T18:51:26.494194+0000","last_active":"2024-09-20T18:56:29.841800+0000","last_peered":"2024-09-20T18:56:29.841800+0000","last_clean":"2024-09-20T18:56:29.841800+0000","last_became_active":"2024-09-20T18:51:23.813399+0000","last_became_peered":"2024-09-20T18:51:23.813399+0000","last_unstale":"2024-09-20T18:56:29.841800+0000","last_undegraded":"2024-09-20T18:56:29.841800+0000","last_fullsized":"2024-09-20T18:56:29.841800+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-20T18:51:26.494118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:26.494118+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":322,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.692160+0000","last_change":"2024-09-20T18:52:38.799542+0000","last_active":"2024-09-20T18:56:40.692160+0000","last_peered":"2024-09-20T18:56:40.692160+0000","last_clean":"2024-09-20T18:56:40.692160+0000","last_became_active":"2024-09-20T18:49:49.959696+0000","last_became_peered":"2024-09-20T18:49:49.959696+0000","last_unstale":"2024-09-20T18:56:40.692160+0000","last_undegraded":"2024-09-20T18:56:40.692160+0000","last_fullsized":"2024-09-20T18:56:40.692160+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:38.799487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:38.799487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":310,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.853082+0000","last_change":"2024-09-20T18:53:21.273835+0000","last_active":"2024-09-20T18:56:29.853082+0000","last_peered":"2024-09-20T18:56:29.853082+0000","last_clean":"2024-09-20T18:56:29.853082+0000","last_became_active":"2024-09-20T18:49:51.970663+0000","last_became_peered":"2024-09-20T18:49:51.970663+0000","last_unstale":"2024-09-20T18:56:29.853082+0000","last_undegraded":"2024-09-20T18:56:29.853082+0000","last_fullsized":"2024-09-20T18:56:29.853082+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:21.273788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:21.273788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":319,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.842301+0000","last_change":"2024-09-20T18:55:28.828417+0000","last_active":"2024-09-20T18:56:29.842301+0000","last_peered":"2024-09-20T18:56:29.842301+0000","last_clean":"2024-09-20T18:56:29.842301+0000","last_became_active":"2024-09-20T18:55:28.827435+0000","last_became_peered":"2024-09-20T18:55:28.827435+0000","last_unstale":"2024-09-20T18:56:29.842301+0000","last_undegraded":"2024-09-20T18:56:29.842301+0000","last_fullsized":"2024-09-20T18:56:29.842301+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:53.940191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:53.940191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":322,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.866624+0000","last_change":"2024-09-20T18:54:23.984024+0000","last_active":"2024-09-20T18:56:29.866624+0000","last_peered":"2024-09-20T18:56:29.866624+0000","last_clean":"2024-09-20T18:56:29.866624+0000","last_became_active":"2024-09-20T18:49:45.950480+0000","last_became_peered":"2024-09-20T18:49:45.950480+0000","last_unstale":"2024-09-20T18:56:29.866624+0000","last_undegraded":"2024-09-20T18:56:29.866624+0000","last_fullsized":"2024-09-20T18:56:29.866624+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:23.983958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:23.983958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":345,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.845546+0000","last_change":"2024-09-20T18:53:26.762207+0000","last_active":"2024-09-20T18:56:29.845546+0000","last_peered":"2024-09-20T18:56:29.845546+0000","last_clean":"2024-09-20T18:56:29.845546+0000","last_became_active":"2024-09-20T18:49:47.950939+0000","last_became_peered":"2024-09-20T18:49:47.950939+0000","last_unstale":"2024-09-20T18:56:29.845546+0000","last_undegraded":"2024-09-20T18:56:29.845546+0000","last_fullsized":"2024-09-20T18:56:29.845546+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:53:26.762131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:26.762131+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":307,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.847873+0000","last_change":"2024-09-20T18:51:17.248705+0000","last_active":"2024-09-20T18:56:29.847873+0000","last_peered":"2024-09-20T18:56:29.847873+0000","last_clean":"2024-09-20T18:56:29.847873+0000","last_became_active":"2024-09-20T18:49:49.958518+0000","last_became_peered":"2024-09-20T18:49:49.958518+0000","last_unstale":"2024-09-20T18:56:29.847873+0000","last_undegraded":"2024-09-20T18:56:29.847873+0000","last_fullsized":"2024-09-20T18:56:29.847873+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:17.248634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:17.248634+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":319,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.863702+0000","last_change":"2024-09-20T18:55:28.833468+0000","last_active":"2024-09-20T18:56:29.863702+0000","last_peered":"2024-09-20T18:56:29.863702+0000","last_clean":"2024-09-20T18:56:29.863702+0000","last_became_active":"2024-09-20T18:55:28.833133+0000","last_became_peered":"2024-09-20T18:55:28.833133+0000","last_unstale":"2024-09-20T18:56:29.863702+0000","last_undegraded":"2024-09-20T18:56:29.863702+0000","last_fullsized":"2024-09-20T18:56:29.863702+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:54.259174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:54.259174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"59'1","reported_seq":329,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.842001+0000","last_change":"2024-09-20T18:53:42.840884+0000","last_active":"2024-09-20T18:56:29.842001+0000","last_peered":"2024-09-20T18:56:29.842001+0000","last_clean":"2024-09-20T18:56:29.842001+0000","last_became_active":"2024-09-20T18:49:45.940100+0000","last_became_peered":"2024-09-20T18:49:45.940100+0000","last_unstale":"2024-09-20T18:56:29.842001+0000","last_undegraded":"2024-09-20T18:56:29.842001+0000","last_fullsized":"2024-09-20T18:56:29.842001+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:53:42.840800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:42.840800+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":326,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.852974+0000","last_change":"2024-09-20T18:49:47.951215+0000","last_active":"2024-09-20T18:56:29.852974+0000","last_peered":"2024-09-20T18:56:29.852974+0000","last_clean":"2024-09-20T18:56:29.852974+0000","last_became_active":"2024-09-20T18:49:47.950909+0000","last_became_peered":"2024-09-20T18:49:47.950909+0000","last_unstale":"2024-09-20T18:56:29.852974+0000","last_undegraded":"2024-09-20T18:56:29.852974+0000","last_fullsized":"2024-09-20T18:56:29.852974+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":323,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.841934+0000","last_change":"2024-09-20T18:54:44.239273+0000","last_active":"2024-09-20T18:56:29.841934+0000","last_peered":"2024-09-20T18:56:29.841934+0000","last_clean":"2024-09-20T18:56:29.841934+0000","last_became_active":"2024-09-20T18:49:49.959102+0000","last_became_peered":"2024-09-20T18:49:49.959102+0000","last_unstale":"2024-09-20T18:56:29.841934+0000","last_undegraded":"2024-09-20T18:56:29.841934+0000","last_fullsized":"2024-09-20T18:56:29.841934+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:44.239226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.239226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":319,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.692562+0000","last_change":"2024-09-20T18:52:15.826819+0000","last_active":"2024-09-20T18:56:40.692562+0000","last_peered":"2024-09-20T18:56:40.692562+0000","last_clean":"2024-09-20T18:56:40.692562+0000","last_became_active":"2024-09-20T18:49:51.970058+0000","last_became_peered":"2024-09-20T18:49:51.970058+0000","last_unstale":"2024-09-20T18:56:40.692562+0000","last_undegraded":"2024-09-20T18:56:40.692562+0000","last_fullsized":"2024-09-20T18:56:40.692562+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T18:52:15.826638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:15.826638+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":316,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.847232+0000","last_change":"2024-09-20T18:52:47.158384+0000","last_active":"2024-09-20T18:56:29.847232+0000","last_peered":"2024-09-20T18:56:29.847232+0000","last_clean":"2024-09-20T18:56:29.847232+0000","last_became_active":"2024-09-20T18:49:45.933078+0000","last_became_peered":"2024-09-20T18:49:45.933078+0000","last_unstale":"2024-09-20T18:56:29.847232+0000","last_undegraded":"2024-09-20T18:56:29.847232+0000","last_fullsized":"2024-09-20T18:56:29.847232+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:52:47.158260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:47.158260+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":342,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.880272+0000","last_change":"2024-09-20T18:55:19.996522+0000","last_active":"2024-09-20T18:56:29.880272+0000","last_peered":"2024-09-20T18:56:29.880272+0000","last_clean":"2024-09-20T18:56:29.880272+0000","last_became_active":"2024-09-20T18:49:48.500809+0000","last_became_peered":"2024-09-20T18:49:48.500809+0000","last_unstale":"2024-09-20T18:56:29.880272+0000","last_undegraded":"2024-09-20T18:56:29.880272+0000","last_fullsized":"2024-09-20T18:56:29.880272+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:55:19.996328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:19.996328+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:35.334334+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"184'9","reported_seq":107,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.842287+0000","last_change":"2024-09-20T18:55:28.845188+0000","last_active":"2024-09-20T18:56:29.842287+0000","last_peered":"2024-09-20T18:56:29.842287+0000","last_clean":"2024-09-20T18:56:29.842287+0000","last_became_active":"2024-09-20T18:55:28.820941+0000","last_became_peered":"2024-09-20T18:55:28.820941+0000","last_unstale":"2024-09-20T18:56:29.842287+0000","last_undegraded":"2024-09-20T18:56:29.842287+0000","last_fullsized":"2024-09-20T18:56:29.842287+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:53:44.524040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:44.524040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3","version":"206'8386","reported_seq":20821,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:36.221118+0000","last_change":"2024-09-20T18:56:25.697520+0000","last_active":"2024-09-20T18:56:36.221118+0000","last_peered":"2024-09-20T18:56:36.221118+0000","last_clean":"2024-09-20T18:56:36.221118+0000","last_became_active":"2024-09-20T18:56:25.687391+0000","last_became_peered":"2024-09-20T18:56:25.687391+0000","last_unstale":"2024-09-20T18:56:36.221118+0000","last_undegraded":"2024-09-20T18:56:36.221118+0000","last_fullsized":"2024-09-20T18:56:36.221118+0000","mapping_epoch":203,"log_start":"174'4072","ondisk_log_start":"174'4072","created":160,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4314,"log_dups_size":0,"ondisk_log_size":4314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":387317760,"num_objects":5910,"num_object_clones":0,"num_object_copies":17730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5910,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8386,"num_write_kb":457472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6156,"num_bytes_recovered":403439616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":336,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.847324+0000","last_change":"2024-09-20T18:55:18.581290+0000","last_active":"2024-09-20T18:56:29.847324+0000","last_peered":"2024-09-20T18:56:29.847324+0000","last_clean":"2024-09-20T18:56:29.847324+0000","last_became_active":"2024-09-20T18:49:58.992302+0000","last_became_peered":"2024-09-20T18:49:58.992302+0000","last_unstale":"2024-09-20T18:56:29.847324+0000","last_undegraded":"2024-09-20T18:56:29.847324+0000","last_fullsized":"2024-09-20T18:56:29.847324+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:55:18.581200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:18.581200+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:33.641712+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":324,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.858388+0000","last_change":"2024-09-20T18:54:54.288199+0000","last_active":"2024-09-20T18:56:29.858388+0000","last_peered":"2024-09-20T18:56:29.858388+0000","last_clean":"2024-09-20T18:56:29.858388+0000","last_became_active":"2024-09-20T18:49:45.936104+0000","last_became_peered":"2024-09-20T18:49:45.936104+0000","last_unstale":"2024-09-20T18:56:29.858388+0000","last_undegraded":"2024-09-20T18:56:29.858388+0000","last_fullsized":"2024-09-20T18:56:29.858388+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:54.288117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:54.288117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":46,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":409,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:31.072297+0000","last_change":"2024-09-20T18:54:14.292660+0000","last_active":"2024-09-20T18:56:31.072297+0000","last_peered":"2024-09-20T18:56:31.072297+0000","last_clean":"2024-09-20T18:56:31.072297+0000","last_became_active":"2024-09-20T18:49:49.960697+0000","last_became_peered":"2024-09-20T18:49:49.960697+0000","last_unstale":"2024-09-20T18:56:31.072297+0000","last_undegraded":"2024-09-20T18:56:31.072297+0000","last_fullsized":"2024-09-20T18:56:31.072297+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:54:14.292610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:14.292610+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":34,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":311,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.851726+0000","last_change":"2024-09-20T18:54:10.381171+0000","last_active":"2024-09-20T18:56:29.851726+0000","last_peered":"2024-09-20T18:56:29.851726+0000","last_clean":"2024-09-20T18:56:29.851726+0000","last_became_active":"2024-09-20T18:49:51.971399+0000","last_became_peered":"2024-09-20T18:49:51.971399+0000","last_unstale":"2024-09-20T18:56:29.851726+0000","last_undegraded":"2024-09-20T18:56:29.851726+0000","last_fullsized":"2024-09-20T18:56:29.851726+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:10.381108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:10.381108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.c","version":"206'8513","reported_seq":8616,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:36.587859+0000","last_change":"2024-09-20T18:54:34.311256+0000","last_active":"2024-09-20T18:56:36.587859+0000","last_peered":"2024-09-20T18:56:36.587859+0000","last_clean":"2024-09-20T18:56:36.587859+0000","last_became_active":"2024-09-20T18:54:34.310904+0000","last_became_peered":"2024-09-20T18:54:34.310904+0000","last_unstale":"2024-09-20T18:56:36.587859+0000","last_undegraded":"2024-09-20T18:56:36.587859+0000","last_fullsized":"2024-09-20T18:56:36.587859+0000","mapping_epoch":160,"log_start":"172'3611","ondisk_log_start":"172'3611","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4902,"log_dups_size":0,"ondisk_log_size":4902,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":385286144,"num_objects":5879,"num_object_clones":0,"num_object_copies":17637,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5879,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8513,"num_write_kb":460544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":352,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.858171+0000","last_change":"2024-09-20T18:54:22.077299+0000","last_active":"2024-09-20T18:56:29.858171+0000","last_peered":"2024-09-20T18:56:29.858171+0000","last_clean":"2024-09-20T18:56:29.858171+0000","last_became_active":"2024-09-20T18:49:48.499189+0000","last_became_peered":"2024-09-20T18:49:48.499189+0000","last_unstale":"2024-09-20T18:56:29.858171+0000","last_undegraded":"2024-09-20T18:56:29.858171+0000","last_fullsized":"2024-09-20T18:56:29.858171+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-20T18:54:22.077212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:22.077212+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":316,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.864264+0000","last_change":"2024-09-20T18:54:57.237740+0000","last_active":"2024-09-20T18:56:29.864264+0000","last_peered":"2024-09-20T18:56:29.864264+0000","last_clean":"2024-09-20T18:56:29.864264+0000","last_became_active":"2024-09-20T18:49:45.940544+0000","last_became_peered":"2024-09-20T18:49:45.940544+0000","last_unstale":"2024-09-20T18:56:29.864264+0000","last_undegraded":"2024-09-20T18:56:29.864264+0000","last_fullsized":"2024-09-20T18:56:29.864264+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.237675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.237675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":324,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.841117+0000","last_change":"2024-09-20T18:54:51.229251+0000","last_active":"2024-09-20T18:56:29.841117+0000","last_peered":"2024-09-20T18:56:29.841117+0000","last_clean":"2024-09-20T18:56:29.841117+0000","last_became_active":"2024-09-20T18:49:49.963073+0000","last_became_peered":"2024-09-20T18:49:49.963073+0000","last_unstale":"2024-09-20T18:56:29.841117+0000","last_undegraded":"2024-09-20T18:56:29.841117+0000","last_fullsized":"2024-09-20T18:56:29.841117+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.229201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.229201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":320,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.841905+0000","last_change":"2024-09-20T18:54:55.323937+0000","last_active":"2024-09-20T18:56:29.841905+0000","last_peered":"2024-09-20T18:56:29.841905+0000","last_clean":"2024-09-20T18:56:29.841905+0000","last_became_active":"2024-09-20T18:49:51.968793+0000","last_became_peered":"2024-09-20T18:49:51.968793+0000","last_unstale":"2024-09-20T18:56:29.841905+0000","last_undegraded":"2024-09-20T18:56:29.841905+0000","last_fullsized":"2024-09-20T18:56:29.841905+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:55.323692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:55.323692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"206'8488","reported_seq":8592,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:35.821634+0000","last_change":"2024-09-20T18:54:34.311680+0000","last_active":"2024-09-20T18:56:35.821634+0000","last_peered":"2024-09-20T18:56:35.821634+0000","last_clean":"2024-09-20T18:56:35.821634+0000","last_became_active":"2024-09-20T18:54:34.311241+0000","last_became_peered":"2024-09-20T18:54:34.311241+0000","last_unstale":"2024-09-20T18:56:35.821634+0000","last_undegraded":"2024-09-20T18:56:35.821634+0000","last_fullsized":"2024-09-20T18:56:35.821634+0000","mapping_epoch":160,"log_start":"171'3446","ondisk_log_start":"171'3446","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":5042,"log_dups_size":0,"ondisk_log_size":5042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":385548312,"num_objects":5884,"num_object_clones":0,"num_object_copies":17652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5884,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8488,"num_write_kb":459841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":380,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.841228+0000","last_change":"2024-09-20T18:55:28.853375+0000","last_active":"2024-09-20T18:56:29.841228+0000","last_peered":"2024-09-20T18:56:29.841228+0000","last_clean":"2024-09-20T18:56:29.841228+0000","last_became_active":"2024-09-20T18:55:28.831057+0000","last_became_peered":"2024-09-20T18:55:28.831057+0000","last_unstale":"2024-09-20T18:56:29.841228+0000","last_undegraded":"2024-09-20T18:56:29.841228+0000","last_fullsized":"2024-09-20T18:56:29.841228+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-20T18:54:57.394893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.394893+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":316,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.880472+0000","last_change":"2024-09-20T18:51:27.187060+0000","last_active":"2024-09-20T18:56:29.880472+0000","last_peered":"2024-09-20T18:56:29.880472+0000","last_clean":"2024-09-20T18:56:29.880472+0000","last_became_active":"2024-09-20T18:49:45.940469+0000","last_became_peered":"2024-09-20T18:49:45.940469+0000","last_unstale":"2024-09-20T18:56:29.880472+0000","last_undegraded":"2024-09-20T18:56:29.880472+0000","last_fullsized":"2024-09-20T18:56:29.880472+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:51:27.186991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:27.186991+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"66'8","reported_seq":415,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:31.073517+0000","last_change":"2024-09-20T18:54:47.223010+0000","last_active":"2024-09-20T18:56:31.073517+0000","last_peered":"2024-09-20T18:56:31.073517+0000","last_clean":"2024-09-20T18:56:31.073517+0000","last_became_active":"2024-09-20T18:49:49.956133+0000","last_became_peered":"2024-09-20T18:49:49.956133+0000","last_unstale":"2024-09-20T18:56:31.073517+0000","last_undegraded":"2024-09-20T18:56:31.073517+0000","last_fullsized":"2024-09-20T18:56:31.073517+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:54:47.222937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:47.222937+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":326,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.691582+0000","last_change":"2024-09-20T18:55:28.819147+0000","last_active":"2024-09-20T18:56:40.691582+0000","last_peered":"2024-09-20T18:56:40.691582+0000","last_clean":"2024-09-20T18:56:40.691582+0000","last_became_active":"2024-09-20T18:55:28.818804+0000","last_became_peered":"2024-09-20T18:55:28.818804+0000","last_unstale":"2024-09-20T18:56:40.691582+0000","last_undegraded":"2024-09-20T18:56:40.691582+0000","last_fullsized":"2024-09-20T18:56:40.691582+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:42.880724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:42.880724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"206'8526","reported_seq":8627,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:36.581065+0000","last_change":"2024-09-20T18:54:34.309939+0000","last_active":"2024-09-20T18:56:36.581065+0000","last_peered":"2024-09-20T18:56:36.581065+0000","last_clean":"2024-09-20T18:56:36.581065+0000","last_became_active":"2024-09-20T18:54:34.309630+0000","last_became_peered":"2024-09-20T18:54:34.309630+0000","last_unstale":"2024-09-20T18:56:36.581065+0000","last_undegraded":"2024-09-20T18:56:36.581065+0000","last_fullsized":"2024-09-20T18:56:36.581065+0000","mapping_epoch":160,"log_start":"172'3649","ondisk_log_start":"172'3649","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4877,"log_dups_size":0,"ondisk_log_size":4877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":385089536,"num_objects":5876,"num_object_clones":0,"num_object_copies":17628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5876,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8526,"num_write_kb":460864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":343,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.847482+0000","last_change":"2024-09-20T18:55:19.541011+0000","last_active":"2024-09-20T18:56:29.847482+0000","last_peered":"2024-09-20T18:56:29.847482+0000","last_clean":"2024-09-20T18:56:29.847482+0000","last_became_active":"2024-09-20T18:49:47.949626+0000","last_became_peered":"2024-09-20T18:49:47.949626+0000","last_unstale":"2024-09-20T18:56:29.847482+0000","last_undegraded":"2024-09-20T18:56:29.847482+0000","last_fullsized":"2024-09-20T18:56:29.847482+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:55:19.540812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:19.540812+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":331,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.852824+0000","last_change":"2024-09-20T18:54:18.326947+0000","last_active":"2024-09-20T18:56:29.852824+0000","last_peered":"2024-09-20T18:56:29.852824+0000","last_clean":"2024-09-20T18:56:29.852824+0000","last_became_active":"2024-09-20T18:49:45.942496+0000","last_became_peered":"2024-09-20T18:49:45.942496+0000","last_unstale":"2024-09-20T18:56:29.852824+0000","last_undegraded":"2024-09-20T18:56:29.852824+0000","last_fullsized":"2024-09-20T18:56:29.852824+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:18.326875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:18.326875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"184'9","reported_seq":417,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:31.073234+0000","last_change":"2024-09-20T18:55:29.010062+0000","last_active":"2024-09-20T18:56:31.073234+0000","last_peered":"2024-09-20T18:56:31.073234+0000","last_clean":"2024-09-20T18:56:31.073234+0000","last_became_active":"2024-09-20T18:55:28.830643+0000","last_became_peered":"2024-09-20T18:55:28.830643+0000","last_unstale":"2024-09-20T18:56:31.073234+0000","last_undegraded":"2024-09-20T18:56:31.073234+0000","last_fullsized":"2024-09-20T18:56:31.073234+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:29.010015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:29.010015+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":295,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.865665+0000","last_change":"2024-09-20T18:49:51.965733+0000","last_active":"2024-09-20T18:56:29.865665+0000","last_peered":"2024-09-20T18:56:29.865665+0000","last_clean":"2024-09-20T18:56:29.865665+0000","last_became_active":"2024-09-20T18:49:51.964780+0000","last_became_peered":"2024-09-20T18:49:51.964780+0000","last_unstale":"2024-09-20T18:56:29.865665+0000","last_undegraded":"2024-09-20T18:56:29.865665+0000","last_fullsized":"2024-09-20T18:56:29.865665+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"206'8727","reported_seq":8830,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:36.396066+0000","last_change":"2024-09-20T18:54:34.309809+0000","last_active":"2024-09-20T18:56:36.396066+0000","last_peered":"2024-09-20T18:56:36.396066+0000","last_clean":"2024-09-20T18:56:36.396066+0000","last_became_active":"2024-09-20T18:54:34.309231+0000","last_became_peered":"2024-09-20T18:54:34.309231+0000","last_unstale":"2024-09-20T18:56:36.396066+0000","last_undegraded":"2024-09-20T18:56:36.396066+0000","last_fullsized":"2024-09-20T18:56:36.396066+0000","mapping_epoch":160,"log_start":"175'4432","ondisk_log_start":"175'4432","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4295,"log_dups_size":0,"ondisk_log_size":4295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":392364032,"num_objects":5987,"num_object_clones":0,"num_object_copies":17961,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5987,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8727,"num_write_kb":470848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":355,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.866474+0000","last_change":"2024-09-20T18:54:14.923895+0000","last_active":"2024-09-20T18:56:29.866474+0000","last_peered":"2024-09-20T18:56:29.866474+0000","last_clean":"2024-09-20T18:56:29.866474+0000","last_became_active":"2024-09-20T18:49:48.497736+0000","last_became_peered":"2024-09-20T18:49:48.497736+0000","last_unstale":"2024-09-20T18:56:29.866474+0000","last_undegraded":"2024-09-20T18:56:29.866474+0000","last_fullsized":"2024-09-20T18:56:29.866474+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-20T18:54:14.923682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:14.923682+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":308,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.847528+0000","last_change":"2024-09-20T18:49:45.942589+0000","last_active":"2024-09-20T18:56:29.847528+0000","last_peered":"2024-09-20T18:56:29.847528+0000","last_clean":"2024-09-20T18:56:29.847528+0000","last_became_active":"2024-09-20T18:49:45.942253+0000","last_became_peered":"2024-09-20T18:49:45.942253+0000","last_unstale":"2024-09-20T18:56:29.847528+0000","last_undegraded":"2024-09-20T18:56:29.847528+0000","last_fullsized":"2024-09-20T18:56:29.847528+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":61,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.868720+0000","last_change":"2024-09-20T18:55:28.830954+0000","last_active":"2024-09-20T18:56:29.868720+0000","last_peered":"2024-09-20T18:56:29.868720+0000","last_clean":"2024-09-20T18:56:29.868720+0000","last_became_active":"2024-09-20T18:55:28.830645+0000","last_became_peered":"2024-09-20T18:55:28.830645+0000","last_unstale":"2024-09-20T18:56:29.868720+0000","last_undegraded":"2024-09-20T18:56:29.868720+0000","last_fullsized":"2024-09-20T18:56:29.868720+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:50.610871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:50.610871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":327,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.847051+0000","last_change":"2024-09-20T18:55:28.822362+0000","last_active":"2024-09-20T18:56:29.847051+0000","last_peered":"2024-09-20T18:56:29.847051+0000","last_clean":"2024-09-20T18:56:29.847051+0000","last_became_active":"2024-09-20T18:55:28.821588+0000","last_became_peered":"2024-09-20T18:55:28.821588+0000","last_unstale":"2024-09-20T18:56:29.847051+0000","last_undegraded":"2024-09-20T18:56:29.847051+0000","last_fullsized":"2024-09-20T18:56:29.847051+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.402841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.402841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"206'8590","reported_seq":8693,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:35.818124+0000","last_change":"2024-09-20T18:54:34.311561+0000","last_active":"2024-09-20T18:56:35.818124+0000","last_peered":"2024-09-20T18:56:35.818124+0000","last_clean":"2024-09-20T18:56:35.818124+0000","last_became_active":"2024-09-20T18:54:34.311181+0000","last_became_peered":"2024-09-20T18:54:34.311181+0000","last_unstale":"2024-09-20T18:56:35.818124+0000","last_undegraded":"2024-09-20T18:56:35.818124+0000","last_fullsized":"2024-09-20T18:56:35.818124+0000","mapping_epoch":160,"log_start":"171'3494","ondisk_log_start":"171'3494","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":5096,"log_dups_size":0,"ondisk_log_size":5096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393740288,"num_objects":6008,"num_object_clones":0,"num_object_copies":18024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6008,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8590,"num_write_kb":467136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"66'28","reported_seq":451,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:31.074761+0000","last_change":"2024-09-20T18:55:16.527267+0000","last_active":"2024-09-20T18:56:31.074761+0000","last_peered":"2024-09-20T18:56:31.074761+0000","last_clean":"2024-09-20T18:56:31.074761+0000","last_became_active":"2024-09-20T18:49:47.955335+0000","last_became_peered":"2024-09-20T18:49:47.955335+0000","last_unstale":"2024-09-20T18:56:31.074761+0000","last_undegraded":"2024-09-20T18:56:31.074761+0000","last_fullsized":"2024-09-20T18:56:31.074761+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'28","last_scrub_stamp":"2024-09-20T18:55:16.527165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:16.527165+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":315,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.842248+0000","last_change":"2024-09-20T18:55:28.832303+0000","last_active":"2024-09-20T18:56:29.842248+0000","last_peered":"2024-09-20T18:56:29.842248+0000","last_clean":"2024-09-20T18:56:29.842248+0000","last_became_active":"2024-09-20T18:55:28.831574+0000","last_became_peered":"2024-09-20T18:55:28.831574+0000","last_unstale":"2024-09-20T18:56:29.842248+0000","last_undegraded":"2024-09-20T18:56:29.842248+0000","last_fullsized":"2024-09-20T18:56:29.842248+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1],"acting":[4,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":360,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.852853+0000","last_change":"2024-09-20T18:55:28.854615+0000","last_active":"2024-09-20T18:56:29.852853+0000","last_peered":"2024-09-20T18:56:29.852853+0000","last_clean":"2024-09-20T18:56:29.852853+0000","last_became_active":"2024-09-20T18:55:28.837226+0000","last_became_peered":"2024-09-20T18:55:28.837226+0000","last_unstale":"2024-09-20T18:56:29.852853+0000","last_undegraded":"2024-09-20T18:56:29.852853+0000","last_fullsized":"2024-09-20T18:56:29.852853+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-20T18:54:51.227531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.227531+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0012781159999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":323,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.870545+0000","last_change":"2024-09-20T18:55:13.949348+0000","last_active":"2024-09-20T18:56:29.870545+0000","last_peered":"2024-09-20T18:56:29.870545+0000","last_clean":"2024-09-20T18:56:29.870545+0000","last_became_active":"2024-09-20T18:49:49.953971+0000","last_became_peered":"2024-09-20T18:49:49.953971+0000","last_unstale":"2024-09-20T18:56:29.870545+0000","last_undegraded":"2024-09-20T18:56:29.870545+0000","last_fullsized":"2024-09-20T18:56:29.870545+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:13.949269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:13.949269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":296,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.853331+0000","last_change":"2024-09-20T18:49:51.969702+0000","last_active":"2024-09-20T18:56:29.853331+0000","last_peered":"2024-09-20T18:56:29.853331+0000","last_clean":"2024-09-20T18:56:29.853331+0000","last_became_active":"2024-09-20T18:49:51.969400+0000","last_became_peered":"2024-09-20T18:49:51.969400+0000","last_unstale":"2024-09-20T18:56:29.853331+0000","last_undegraded":"2024-09-20T18:56:29.853331+0000","last_fullsized":"2024-09-20T18:56:29.853331+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"206'8708","reported_seq":8811,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:36.353743+0000","last_change":"2024-09-20T18:54:33.926785+0000","last_active":"2024-09-20T18:56:36.353743+0000","last_peered":"2024-09-20T18:56:36.353743+0000","last_clean":"2024-09-20T18:56:36.353743+0000","last_became_active":"2024-09-20T18:54:33.925750+0000","last_became_peered":"2024-09-20T18:54:33.925750+0000","last_unstale":"2024-09-20T18:56:36.353743+0000","last_undegraded":"2024-09-20T18:56:36.353743+0000","last_fullsized":"2024-09-20T18:56:36.353743+0000","mapping_epoch":160,"log_start":"175'4340","ondisk_log_start":"175'4340","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4368,"log_dups_size":0,"ondisk_log_size":4368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393871360,"num_objects":6010,"num_object_clones":0,"num_object_copies":18030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8708,"num_write_kb":470976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":355,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.849208+0000","last_change":"2024-09-20T18:55:25.702762+0000","last_active":"2024-09-20T18:56:29.849208+0000","last_peered":"2024-09-20T18:56:29.849208+0000","last_clean":"2024-09-20T18:56:29.849208+0000","last_became_active":"2024-09-20T18:49:47.950904+0000","last_became_peered":"2024-09-20T18:49:47.950904+0000","last_unstale":"2024-09-20T18:56:29.849208+0000","last_undegraded":"2024-09-20T18:56:29.849208+0000","last_fullsized":"2024-09-20T18:56:29.849208+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-20T18:55:25.702596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:25.702596+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":339,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.848287+0000","last_change":"2024-09-20T18:55:28.830832+0000","last_active":"2024-09-20T18:56:29.848287+0000","last_peered":"2024-09-20T18:56:29.848287+0000","last_clean":"2024-09-20T18:56:29.848287+0000","last_became_active":"2024-09-20T18:55:28.830269+0000","last_became_peered":"2024-09-20T18:55:28.830269+0000","last_unstale":"2024-09-20T18:56:29.848287+0000","last_undegraded":"2024-09-20T18:56:29.848287+0000","last_fullsized":"2024-09-20T18:56:29.848287+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:23.359769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:23.359769+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":326,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.845481+0000","last_change":"2024-09-20T18:52:07.718766+0000","last_active":"2024-09-20T18:56:29.845481+0000","last_peered":"2024-09-20T18:56:29.845481+0000","last_clean":"2024-09-20T18:56:29.845481+0000","last_became_active":"2024-09-20T18:49:44.937135+0000","last_became_peered":"2024-09-20T18:49:44.937135+0000","last_unstale":"2024-09-20T18:56:29.845481+0000","last_undegraded":"2024-09-20T18:56:29.845481+0000","last_fullsized":"2024-09-20T18:56:29.845481+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:52:07.718646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:52:07.718646+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.00095558800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":309,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.846982+0000","last_change":"2024-09-20T18:51:59.796450+0000","last_active":"2024-09-20T18:56:29.846982+0000","last_peered":"2024-09-20T18:56:29.846982+0000","last_clean":"2024-09-20T18:56:29.846982+0000","last_became_active":"2024-09-20T18:49:49.954424+0000","last_became_peered":"2024-09-20T18:49:49.954424+0000","last_unstale":"2024-09-20T18:56:29.846982+0000","last_undegraded":"2024-09-20T18:56:29.846982+0000","last_fullsized":"2024-09-20T18:56:29.846982+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:59.796383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:59.796383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":318,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.692755+0000","last_change":"2024-09-20T18:55:28.826659+0000","last_active":"2024-09-20T18:56:40.692755+0000","last_peered":"2024-09-20T18:56:40.692755+0000","last_clean":"2024-09-20T18:56:40.692755+0000","last_became_active":"2024-09-20T18:55:28.825812+0000","last_became_peered":"2024-09-20T18:55:28.825812+0000","last_unstale":"2024-09-20T18:56:40.692755+0000","last_undegraded":"2024-09-20T18:56:40.692755+0000","last_fullsized":"2024-09-20T18:56:40.692755+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:39.814259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:39.814259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"206'8689","reported_seq":8792,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:36.224241+0000","last_change":"2024-09-20T18:54:34.311774+0000","last_active":"2024-09-20T18:56:36.224241+0000","last_peered":"2024-09-20T18:56:36.224241+0000","last_clean":"2024-09-20T18:56:36.224241+0000","last_became_active":"2024-09-20T18:54:34.311487+0000","last_became_peered":"2024-09-20T18:54:34.311487+0000","last_unstale":"2024-09-20T18:56:36.224241+0000","last_undegraded":"2024-09-20T18:56:36.224241+0000","last_fullsized":"2024-09-20T18:56:36.224241+0000","mapping_epoch":160,"log_start":"175'4392","ondisk_log_start":"175'4392","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4297,"log_dups_size":0,"ondisk_log_size":4297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":388562944,"num_objects":5929,"num_object_clones":0,"num_object_copies":17787,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8689,"num_write_kb":467776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":353,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.880338+0000","last_change":"2024-09-20T18:55:14.951950+0000","last_active":"2024-09-20T18:56:29.880338+0000","last_peered":"2024-09-20T18:56:29.880338+0000","last_clean":"2024-09-20T18:56:29.880338+0000","last_became_active":"2024-09-20T18:49:47.949230+0000","last_became_peered":"2024-09-20T18:49:47.949230+0000","last_unstale":"2024-09-20T18:56:29.880338+0000","last_undegraded":"2024-09-20T18:56:29.880338+0000","last_fullsized":"2024-09-20T18:56:29.880338+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:55:14.951892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:14.951892+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:36.283330+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":331,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.871683+0000","last_change":"2024-09-20T18:54:27.905856+0000","last_active":"2024-09-20T18:56:29.871683+0000","last_peered":"2024-09-20T18:56:29.871683+0000","last_clean":"2024-09-20T18:56:29.871683+0000","last_became_active":"2024-09-20T18:49:45.939197+0000","last_became_peered":"2024-09-20T18:49:45.939197+0000","last_unstale":"2024-09-20T18:56:29.871683+0000","last_undegraded":"2024-09-20T18:56:29.871683+0000","last_fullsized":"2024-09-20T18:56:29.871683+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:27.905803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:27.905803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":760,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.691632+0000","last_change":"2024-09-20T18:54:26.359887+0000","last_active":"2024-09-20T18:56:40.691632+0000","last_peered":"2024-09-20T18:56:40.691632+0000","last_clean":"2024-09-20T18:56:40.691632+0000","last_became_active":"2024-09-20T18:49:44.936819+0000","last_became_peered":"2024-09-20T18:49:44.936819+0000","last_unstale":"2024-09-20T18:56:40.691632+0000","last_undegraded":"2024-09-20T18:56:40.691632+0000","last_fullsized":"2024-09-20T18:56:40.691632+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-20T18:54:26.359807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:54:26.359807+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0010996199999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":410,"num_read_kb":405,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":315,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.852047+0000","last_change":"2024-09-20T18:54:59.151228+0000","last_active":"2024-09-20T18:56:29.852047+0000","last_peered":"2024-09-20T18:56:29.852047+0000","last_clean":"2024-09-20T18:56:29.852047+0000","last_became_active":"2024-09-20T18:49:49.961066+0000","last_became_peered":"2024-09-20T18:49:49.961066+0000","last_unstale":"2024-09-20T18:56:29.852047+0000","last_undegraded":"2024-09-20T18:56:29.852047+0000","last_fullsized":"2024-09-20T18:56:29.852047+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:59.150063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:59.150063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":320,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.847193+0000","last_change":"2024-09-20T18:55:21.629867+0000","last_active":"2024-09-20T18:56:29.847193+0000","last_peered":"2024-09-20T18:56:29.847193+0000","last_clean":"2024-09-20T18:56:29.847193+0000","last_became_active":"2024-09-20T18:49:51.970343+0000","last_became_peered":"2024-09-20T18:49:51.970343+0000","last_unstale":"2024-09-20T18:56:29.847193+0000","last_undegraded":"2024-09-20T18:56:29.847193+0000","last_fullsized":"2024-09-20T18:56:29.847193+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:21.629800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:21.629800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:40.736438+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"206'8653","reported_seq":14663,"reported_epoch":206,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-20T18:56:35.905343+0000","last_change":"2024-09-20T18:55:30.088909+0000","last_active":"2024-09-20T18:56:35.905343+0000","last_peered":"2024-09-20T18:56:35.905343+0000","last_clean":"2024-09-20T18:55:27.745898+0000","last_became_active":"2024-09-20T18:55:29.995624+0000","last_became_peered":"2024-09-20T18:55:29.995624+0000","last_unstale":"2024-09-20T18:56:35.905343+0000","last_undegraded":"2024-09-20T18:56:35.905343+0000","last_fullsized":"2024-09-20T18:55:29.815609+0000","mapping_epoch":182,"log_start":"172'3606","ondisk_log_start":"172'3606","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":5047,"log_dups_size":0,"ondisk_log_size":5047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":385286144,"num_objects":5879,"num_object_clones":0,"num_object_copies":17637,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2253,"num_objects_unfound":0,"num_objects_dirty":5879,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8653,"num_write_kb":465024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2953,"num_bytes_recovered":193527808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0],"avail_no_missing":["5","0","6"],"object_location_counts":[{"shards":"0,5","objects":3626},{"shards":"0,5,6","objects":2253}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"184'17","reported_seq":136,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.845866+0000","last_change":"2024-09-20T18:55:28.856602+0000","last_active":"2024-09-20T18:56:29.845866+0000","last_peered":"2024-09-20T18:56:29.845866+0000","last_clean":"2024-09-20T18:56:29.845866+0000","last_became_active":"2024-09-20T18:55:28.831024+0000","last_became_peered":"2024-09-20T18:55:28.831024+0000","last_unstale":"2024-09-20T18:56:29.845866+0000","last_undegraded":"2024-09-20T18:56:29.845866+0000","last_fullsized":"2024-09-20T18:56:29.845866+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":24,"num_read_kb":14,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":331,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.852037+0000","last_change":"2024-09-20T18:55:28.830447+0000","last_active":"2024-09-20T18:56:29.852037+0000","last_peered":"2024-09-20T18:56:29.852037+0000","last_clean":"2024-09-20T18:56:29.852037+0000","last_became_active":"2024-09-20T18:55:28.830147+0000","last_became_peered":"2024-09-20T18:55:28.830147+0000","last_unstale":"2024-09-20T18:56:29.852037+0000","last_undegraded":"2024-09-20T18:56:29.852037+0000","last_fullsized":"2024-09-20T18:56:29.852037+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:13.144896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:13.144896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":392,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.691683+0000","last_change":"2024-09-20T18:55:28.844060+0000","last_active":"2024-09-20T18:56:40.691683+0000","last_peered":"2024-09-20T18:56:40.691683+0000","last_clean":"2024-09-20T18:56:40.691683+0000","last_became_active":"2024-09-20T18:55:28.829325+0000","last_became_peered":"2024-09-20T18:55:28.829325+0000","last_unstale":"2024-09-20T18:56:40.691683+0000","last_undegraded":"2024-09-20T18:56:40.691683+0000","last_fullsized":"2024-09-20T18:56:40.691683+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T18:55:13.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T18:55:13.696607+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":315,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.692781+0000","last_change":"2024-09-20T18:51:22.852942+0000","last_active":"2024-09-20T18:56:40.692781+0000","last_peered":"2024-09-20T18:56:40.692781+0000","last_clean":"2024-09-20T18:56:40.692781+0000","last_became_active":"2024-09-20T18:49:49.956128+0000","last_became_peered":"2024-09-20T18:49:49.956128+0000","last_unstale":"2024-09-20T18:56:40.692781+0000","last_undegraded":"2024-09-20T18:56:40.692781+0000","last_fullsized":"2024-09-20T18:56:40.692781+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:22.852869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:22.852869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":319,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.851824+0000","last_change":"2024-09-20T18:55:25.281435+0000","last_active":"2024-09-20T18:56:29.851824+0000","last_peered":"2024-09-20T18:56:29.851824+0000","last_clean":"2024-09-20T18:56:29.851824+0000","last_became_active":"2024-09-20T18:49:51.971332+0000","last_became_peered":"2024-09-20T18:49:51.971332+0000","last_unstale":"2024-09-20T18:56:29.851824+0000","last_undegraded":"2024-09-20T18:56:29.851824+0000","last_fullsized":"2024-09-20T18:56:29.851824+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:25.281366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:25.281366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"206'8594","reported_seq":8697,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:35.805249+0000","last_change":"2024-09-20T18:54:33.929366+0000","last_active":"2024-09-20T18:56:35.805249+0000","last_peered":"2024-09-20T18:56:35.805249+0000","last_clean":"2024-09-20T18:56:35.805249+0000","last_became_active":"2024-09-20T18:54:33.929072+0000","last_became_peered":"2024-09-20T18:54:33.929072+0000","last_unstale":"2024-09-20T18:56:35.805249+0000","last_undegraded":"2024-09-20T18:56:35.805249+0000","last_fullsized":"2024-09-20T18:56:35.805249+0000","mapping_epoch":160,"log_start":"174'4002","ondisk_log_start":"174'4002","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4592,"log_dups_size":0,"ondisk_log_size":4592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":391118848,"num_objects":5968,"num_object_clones":0,"num_object_copies":17904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5968,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8594,"num_write_kb":465984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":350,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.842507+0000","last_change":"2024-09-20T18:55:28.840105+0000","last_active":"2024-09-20T18:56:29.842507+0000","last_peered":"2024-09-20T18:56:29.842507+0000","last_clean":"2024-09-20T18:56:29.842507+0000","last_became_active":"2024-09-20T18:55:28.818269+0000","last_became_peered":"2024-09-20T18:55:28.818269+0000","last_unstale":"2024-09-20T18:56:29.842507+0000","last_undegraded":"2024-09-20T18:56:29.842507+0000","last_fullsized":"2024-09-20T18:56:29.842507+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:51:14.397930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.397930+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":316,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.841478+0000","last_change":"2024-09-20T18:51:18.410601+0000","last_active":"2024-09-20T18:56:29.841478+0000","last_peered":"2024-09-20T18:56:29.841478+0000","last_clean":"2024-09-20T18:56:29.841478+0000","last_became_active":"2024-09-20T18:49:45.944579+0000","last_became_peered":"2024-09-20T18:49:45.944579+0000","last_unstale":"2024-09-20T18:56:29.841478+0000","last_undegraded":"2024-09-20T18:56:29.841478+0000","last_fullsized":"2024-09-20T18:56:29.841478+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:18.410447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:18.410447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":329,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.851684+0000","last_change":"2024-09-20T18:55:28.830634+0000","last_active":"2024-09-20T18:56:29.851684+0000","last_peered":"2024-09-20T18:56:29.851684+0000","last_clean":"2024-09-20T18:56:29.851684+0000","last_became_active":"2024-09-20T18:55:28.827785+0000","last_became_peered":"2024-09-20T18:55:28.827785+0000","last_unstale":"2024-09-20T18:56:29.851684+0000","last_undegraded":"2024-09-20T18:56:29.851684+0000","last_fullsized":"2024-09-20T18:56:29.851684+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:11.401777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:11.401777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":311,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.851792+0000","last_change":"2024-09-20T18:55:28.833649+0000","last_active":"2024-09-20T18:56:29.851792+0000","last_peered":"2024-09-20T18:56:29.851792+0000","last_clean":"2024-09-20T18:56:29.851792+0000","last_became_active":"2024-09-20T18:55:28.833115+0000","last_became_peered":"2024-09-20T18:55:28.833115+0000","last_unstale":"2024-09-20T18:56:29.851792+0000","last_undegraded":"2024-09-20T18:56:29.851792+0000","last_fullsized":"2024-09-20T18:56:29.851792+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:05.021071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.021071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1","version":"206'8540","reported_seq":8643,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:36.215547+0000","last_change":"2024-09-20T18:54:34.310237+0000","last_active":"2024-09-20T18:56:36.215547+0000","last_peered":"2024-09-20T18:56:36.215547+0000","last_clean":"2024-09-20T18:56:36.215547+0000","last_became_active":"2024-09-20T18:54:34.309781+0000","last_became_peered":"2024-09-20T18:54:34.309781+0000","last_unstale":"2024-09-20T18:56:36.215547+0000","last_undegraded":"2024-09-20T18:56:36.215547+0000","last_fullsized":"2024-09-20T18:56:36.215547+0000","mapping_epoch":160,"log_start":"175'4224","ondisk_log_start":"175'4224","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4316,"log_dups_size":0,"ondisk_log_size":4316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":386138112,"num_objects":5892,"num_object_clones":0,"num_object_copies":17676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5892,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8540,"num_write_kb":461824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":348,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.846024+0000","last_change":"2024-09-20T18:52:02.944536+0000","last_active":"2024-09-20T18:56:29.846024+0000","last_peered":"2024-09-20T18:56:29.846024+0000","last_clean":"2024-09-20T18:56:29.846024+0000","last_became_active":"2024-09-20T18:49:47.947636+0000","last_became_peered":"2024-09-20T18:49:47.947636+0000","last_unstale":"2024-09-20T18:56:29.846024+0000","last_undegraded":"2024-09-20T18:56:29.846024+0000","last_fullsized":"2024-09-20T18:56:29.846024+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-20T18:52:02.944434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:02.944434+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":185,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.841701+0000","last_change":"2024-09-20T18:55:28.822330+0000","last_active":"2024-09-20T18:56:29.841701+0000","last_peered":"2024-09-20T18:56:29.841701+0000","last_clean":"2024-09-20T18:56:29.841701+0000","last_became_active":"2024-09-20T18:55:28.821959+0000","last_became_peered":"2024-09-20T18:55:28.821959+0000","last_unstale":"2024-09-20T18:56:29.841701+0000","last_undegraded":"2024-09-20T18:56:29.841701+0000","last_fullsized":"2024-09-20T18:56:29.841701+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:45.669427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:45.669427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":316,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.847307+0000","last_change":"2024-09-20T18:54:11.377907+0000","last_active":"2024-09-20T18:56:29.847307+0000","last_peered":"2024-09-20T18:56:29.847307+0000","last_clean":"2024-09-20T18:56:29.847307+0000","last_became_active":"2024-09-20T18:49:49.959160+0000","last_became_peered":"2024-09-20T18:49:49.959160+0000","last_unstale":"2024-09-20T18:56:29.847307+0000","last_undegraded":"2024-09-20T18:56:29.847307+0000","last_fullsized":"2024-09-20T18:56:29.847307+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:11.376951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:11.376951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":312,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.847467+0000","last_change":"2024-09-20T18:52:35.635897+0000","last_active":"2024-09-20T18:56:29.847467+0000","last_peered":"2024-09-20T18:56:29.847467+0000","last_clean":"2024-09-20T18:56:29.847467+0000","last_became_active":"2024-09-20T18:49:51.970108+0000","last_became_peered":"2024-09-20T18:49:51.970108+0000","last_unstale":"2024-09-20T18:56:29.847467+0000","last_undegraded":"2024-09-20T18:56:29.847467+0000","last_fullsized":"2024-09-20T18:56:29.847467+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:35.635818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:35.635818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"206'8417","reported_seq":8521,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:36.380773+0000","last_change":"2024-09-20T18:54:33.929552+0000","last_active":"2024-09-20T18:56:36.380773+0000","last_peered":"2024-09-20T18:56:36.380773+0000","last_clean":"2024-09-20T18:56:36.380773+0000","last_became_active":"2024-09-20T18:54:33.929208+0000","last_became_peered":"2024-09-20T18:54:33.929208+0000","last_unstale":"2024-09-20T18:56:36.380773+0000","last_undegraded":"2024-09-20T18:56:36.380773+0000","last_fullsized":"2024-09-20T18:56:36.380773+0000","mapping_epoch":160,"log_start":"174'4121","ondisk_log_start":"174'4121","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4296,"log_dups_size":0,"ondisk_log_size":4296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":380698624,"num_objects":5809,"num_object_clones":0,"num_object_copies":17427,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5809,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8417,"num_write_kb":455232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":354,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.842093+0000","last_change":"2024-09-20T18:55:28.846773+0000","last_active":"2024-09-20T18:56:29.842093+0000","last_peered":"2024-09-20T18:56:29.842093+0000","last_clean":"2024-09-20T18:56:29.842093+0000","last_became_active":"2024-09-20T18:55:28.817101+0000","last_became_peered":"2024-09-20T18:55:28.817101+0000","last_unstale":"2024-09-20T18:56:29.842093+0000","last_undegraded":"2024-09-20T18:56:29.842093+0000","last_fullsized":"2024-09-20T18:56:29.842093+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:54:17.219548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:17.219548+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":324,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.853629+0000","last_change":"2024-09-20T18:52:46.005172+0000","last_active":"2024-09-20T18:56:29.853629+0000","last_peered":"2024-09-20T18:56:29.853629+0000","last_clean":"2024-09-20T18:56:29.853629+0000","last_became_active":"2024-09-20T18:49:45.937264+0000","last_became_peered":"2024-09-20T18:49:45.937264+0000","last_unstale":"2024-09-20T18:56:29.853629+0000","last_undegraded":"2024-09-20T18:56:29.853629+0000","last_fullsized":"2024-09-20T18:56:29.853629+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:46.005109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:46.005109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":416,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:36.074022+0000","last_change":"2024-09-20T18:55:17.549660+0000","last_active":"2024-09-20T18:56:36.074022+0000","last_peered":"2024-09-20T18:56:36.074022+0000","last_clean":"2024-09-20T18:56:36.074022+0000","last_became_active":"2024-09-20T18:49:49.957005+0000","last_became_peered":"2024-09-20T18:49:49.957005+0000","last_unstale":"2024-09-20T18:56:36.074022+0000","last_undegraded":"2024-09-20T18:56:36.074022+0000","last_fullsized":"2024-09-20T18:56:36.074022+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:17.549598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:17.549598+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:45.590247+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":311,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.842230+0000","last_change":"2024-09-20T18:54:42.182875+0000","last_active":"2024-09-20T18:56:29.842230+0000","last_peered":"2024-09-20T18:56:29.842230+0000","last_clean":"2024-09-20T18:56:29.842230+0000","last_became_active":"2024-09-20T18:49:51.969355+0000","last_became_peered":"2024-09-20T18:49:51.969355+0000","last_unstale":"2024-09-20T18:56:29.842230+0000","last_undegraded":"2024-09-20T18:56:29.842230+0000","last_fullsized":"2024-09-20T18:56:29.842230+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:42.182809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:42.182809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"206'8623","reported_seq":16686,"reported_epoch":206,"state":"active+remapped+backfilling","last_fresh":"2024-09-20T18:56:36.231658+0000","last_change":"2024-09-20T18:55:59.938306+0000","last_active":"2024-09-20T18:56:36.231658+0000","last_peered":"2024-09-20T18:56:36.231658+0000","last_clean":"2024-09-20T18:55:27.792821+0000","last_became_active":"2024-09-20T18:55:29.915245+0000","last_became_peered":"2024-09-20T18:55:29.915245+0000","last_unstale":"2024-09-20T18:56:36.231658+0000","last_undegraded":"2024-09-20T18:56:36.231658+0000","last_fullsized":"2024-09-20T18:56:36.231658+0000","mapping_epoch":182,"log_start":"175'4361","ondisk_log_start":"175'4361","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4262,"log_dups_size":0,"ondisk_log_size":4262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":388431872,"num_objects":5927,"num_object_clones":0,"num_object_copies":17781,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2402,"num_objects_unfound":0,"num_objects_dirty":5927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8623,"num_write_kb":465600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3975,"num_bytes_recovered":260505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,6],"avail_no_missing":["1","3","6"],"object_location_counts":[{"shards":"1,3,6","objects":5927}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":343,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.866435+0000","last_change":"2024-09-20T18:54:54.897991+0000","last_active":"2024-09-20T18:56:29.866435+0000","last_peered":"2024-09-20T18:56:29.866435+0000","last_clean":"2024-09-20T18:56:29.866435+0000","last_became_active":"2024-09-20T18:49:47.946008+0000","last_became_peered":"2024-09-20T18:49:47.946008+0000","last_unstale":"2024-09-20T18:56:29.866435+0000","last_undegraded":"2024-09-20T18:56:29.866435+0000","last_fullsized":"2024-09-20T18:56:29.866435+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:54.897933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:54.897933+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":338,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.851547+0000","last_change":"2024-09-20T18:55:28.833453+0000","last_active":"2024-09-20T18:56:29.851547+0000","last_peered":"2024-09-20T18:56:29.851547+0000","last_clean":"2024-09-20T18:56:29.851547+0000","last_became_active":"2024-09-20T18:55:28.833095+0000","last_became_peered":"2024-09-20T18:55:28.833095+0000","last_unstale":"2024-09-20T18:56:29.851547+0000","last_undegraded":"2024-09-20T18:56:29.851547+0000","last_fullsized":"2024-09-20T18:56:29.851547+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:56.212378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:56.212378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":325,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.847194+0000","last_change":"2024-09-20T18:54:43.379898+0000","last_active":"2024-09-20T18:56:29.847194+0000","last_peered":"2024-09-20T18:56:29.847194+0000","last_clean":"2024-09-20T18:56:29.847194+0000","last_became_active":"2024-09-20T18:49:49.956767+0000","last_became_peered":"2024-09-20T18:49:49.956767+0000","last_unstale":"2024-09-20T18:56:29.847194+0000","last_undegraded":"2024-09-20T18:56:29.847194+0000","last_fullsized":"2024-09-20T18:56:29.847194+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:43.379706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:43.379706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":311,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.871602+0000","last_change":"2024-09-20T18:54:24.976627+0000","last_active":"2024-09-20T18:56:29.871602+0000","last_peered":"2024-09-20T18:56:29.871602+0000","last_clean":"2024-09-20T18:56:29.871602+0000","last_became_active":"2024-09-20T18:49:51.964098+0000","last_became_peered":"2024-09-20T18:49:51.964098+0000","last_unstale":"2024-09-20T18:56:29.871602+0000","last_undegraded":"2024-09-20T18:56:29.871602+0000","last_fullsized":"2024-09-20T18:56:29.871602+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:24.976584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:24.976584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"206'8669","reported_seq":10959,"reported_epoch":206,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-20T18:56:36.580559+0000","last_change":"2024-09-20T18:56:27.871592+0000","last_active":"2024-09-20T18:56:36.580559+0000","last_peered":"2024-09-20T18:56:36.580559+0000","last_clean":"2024-09-20T18:55:27.791097+0000","last_became_active":"2024-09-20T18:55:29.979932+0000","last_became_peered":"2024-09-20T18:55:29.979932+0000","last_unstale":"2024-09-20T18:56:36.580559+0000","last_undegraded":"2024-09-20T18:56:27.867865+0000","last_fullsized":"2024-09-20T18:55:29.815116+0000","mapping_epoch":182,"log_start":"172'3740","ondisk_log_start":"172'3740","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4929,"log_dups_size":0,"ondisk_log_size":4929,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":378732544,"num_objects":5779,"num_object_clones":0,"num_object_copies":17337,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":354,"num_objects_misplaced":3732,"num_objects_unfound":0,"num_objects_dirty":5779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8669,"num_write_kb":462336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1092,"num_bytes_recovered":71565312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7],"avail_no_missing":["0","6","7"],"object_location_counts":[{"shards":"0,6,7","objects":3732},{"shards":"0,7","objects":1693}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":132,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.847390+0000","last_change":"2024-09-20T18:55:28.855793+0000","last_active":"2024-09-20T18:56:29.847390+0000","last_peered":"2024-09-20T18:56:29.847390+0000","last_clean":"2024-09-20T18:56:29.847390+0000","last_became_active":"2024-09-20T18:55:28.828563+0000","last_became_peered":"2024-09-20T18:55:28.828563+0000","last_unstale":"2024-09-20T18:56:29.847390+0000","last_undegraded":"2024-09-20T18:56:29.847390+0000","last_fullsized":"2024-09-20T18:56:29.847390+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-20T18:53:24.458005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:24.458005+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":31,"num_read_kb":20,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,5],"acting":[1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":346,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.693103+0000","last_change":"2024-09-20T18:55:28.835767+0000","last_active":"2024-09-20T18:56:40.693103+0000","last_peered":"2024-09-20T18:56:40.693103+0000","last_clean":"2024-09-20T18:56:40.693103+0000","last_became_active":"2024-09-20T18:55:28.835149+0000","last_became_peered":"2024-09-20T18:55:28.835149+0000","last_unstale":"2024-09-20T18:56:40.693103+0000","last_undegraded":"2024-09-20T18:56:40.693103+0000","last_fullsized":"2024-09-20T18:56:40.693103+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:44.501192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.501192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":325,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.844794+0000","last_change":"2024-09-20T18:55:13.577772+0000","last_active":"2024-09-20T18:56:29.844794+0000","last_peered":"2024-09-20T18:56:29.844794+0000","last_clean":"2024-09-20T18:56:29.844794+0000","last_became_active":"2024-09-20T18:49:49.957558+0000","last_became_peered":"2024-09-20T18:49:49.957558+0000","last_unstale":"2024-09-20T18:56:29.844794+0000","last_undegraded":"2024-09-20T18:56:29.844794+0000","last_fullsized":"2024-09-20T18:56:29.844794+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:13.577699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:13.577699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:35.011344+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":327,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.691623+0000","last_change":"2024-09-20T18:53:47.370152+0000","last_active":"2024-09-20T18:56:40.691623+0000","last_peered":"2024-09-20T18:56:40.691623+0000","last_clean":"2024-09-20T18:56:40.691623+0000","last_became_active":"2024-09-20T18:49:51.970798+0000","last_became_peered":"2024-09-20T18:49:51.970798+0000","last_unstale":"2024-09-20T18:56:40.691623+0000","last_undegraded":"2024-09-20T18:56:40.691623+0000","last_fullsized":"2024-09-20T18:56:40.691623+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:47.370108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:47.370108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"206'8852","reported_seq":14115,"reported_epoch":206,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-20T18:56:35.874486+0000","last_change":"2024-09-20T18:55:30.067593+0000","last_active":"2024-09-20T18:56:35.874486+0000","last_peered":"2024-09-20T18:56:35.874486+0000","last_clean":"2024-09-20T18:55:27.760955+0000","last_became_active":"2024-09-20T18:55:30.004922+0000","last_became_peered":"2024-09-20T18:55:30.004922+0000","last_unstale":"2024-09-20T18:56:35.874486+0000","last_undegraded":"2024-09-20T18:56:35.874486+0000","last_fullsized":"2024-09-20T18:55:29.819726+0000","mapping_epoch":182,"log_start":"174'4246","ondisk_log_start":"174'4246","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4606,"log_dups_size":0,"ondisk_log_size":4606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":401997824,"num_objects":6134,"num_object_clones":0,"num_object_copies":18402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2813,"num_objects_unfound":0,"num_objects_dirty":6134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8852,"num_write_kb":479552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2578,"num_bytes_recovered":168951808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7],"avail_no_missing":["4","6","7"],"object_location_counts":[{"shards":"4,6,7","objects":2813},{"shards":"4,7","objects":3321}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":343,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.841344+0000","last_change":"2024-09-20T18:53:52.898818+0000","last_active":"2024-09-20T18:56:29.841344+0000","last_peered":"2024-09-20T18:56:29.841344+0000","last_clean":"2024-09-20T18:56:29.841344+0000","last_became_active":"2024-09-20T18:49:47.949116+0000","last_became_peered":"2024-09-20T18:49:47.949116+0000","last_unstale":"2024-09-20T18:56:29.841344+0000","last_undegraded":"2024-09-20T18:56:29.841344+0000","last_fullsized":"2024-09-20T18:56:29.841344+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:53:52.898760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:52.898760+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":339,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.693048+0000","last_change":"2024-09-20T18:54:52.650798+0000","last_active":"2024-09-20T18:56:40.693048+0000","last_peered":"2024-09-20T18:56:40.693048+0000","last_clean":"2024-09-20T18:56:40.693048+0000","last_became_active":"2024-09-20T18:49:45.944858+0000","last_became_peered":"2024-09-20T18:49:45.944858+0000","last_unstale":"2024-09-20T18:56:40.693048+0000","last_undegraded":"2024-09-20T18:56:40.693048+0000","last_fullsized":"2024-09-20T18:56:40.693048+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:52.650756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:52.650756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":317,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.844874+0000","last_change":"2024-09-20T18:54:42.349879+0000","last_active":"2024-09-20T18:56:29.844874+0000","last_peered":"2024-09-20T18:56:29.844874+0000","last_clean":"2024-09-20T18:56:29.844874+0000","last_became_active":"2024-09-20T18:49:49.954944+0000","last_became_peered":"2024-09-20T18:49:49.954944+0000","last_unstale":"2024-09-20T18:56:29.844874+0000","last_undegraded":"2024-09-20T18:56:29.844874+0000","last_fullsized":"2024-09-20T18:56:29.844874+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:42.349807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:42.349807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":312,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.853354+0000","last_change":"2024-09-20T18:55:17.341862+0000","last_active":"2024-09-20T18:56:29.853354+0000","last_peered":"2024-09-20T18:56:29.853354+0000","last_clean":"2024-09-20T18:56:29.853354+0000","last_became_active":"2024-09-20T18:49:51.972970+0000","last_became_peered":"2024-09-20T18:49:51.972970+0000","last_unstale":"2024-09-20T18:56:29.853354+0000","last_undegraded":"2024-09-20T18:56:29.853354+0000","last_fullsized":"2024-09-20T18:56:29.853354+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:17.341793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:17.341793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.4","version":"206'8480","reported_seq":8583,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:35.813220+0000","last_change":"2024-09-20T18:54:34.309442+0000","last_active":"2024-09-20T18:56:35.813220+0000","last_peered":"2024-09-20T18:56:35.813220+0000","last_clean":"2024-09-20T18:56:35.813220+0000","last_became_active":"2024-09-20T18:54:34.309113+0000","last_became_peered":"2024-09-20T18:54:34.309113+0000","last_unstale":"2024-09-20T18:56:35.813220+0000","last_undegraded":"2024-09-20T18:56:35.813220+0000","last_fullsized":"2024-09-20T18:56:35.813220+0000","mapping_epoch":160,"log_start":"171'3431","ondisk_log_start":"171'3431","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":5049,"log_dups_size":0,"ondisk_log_size":5049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":383516672,"num_objects":5852,"num_object_clones":0,"num_object_copies":17556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5852,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8480,"num_write_kb":458624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":367,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.853540+0000","last_change":"2024-09-20T18:55:28.847010+0000","last_active":"2024-09-20T18:56:29.853540+0000","last_peered":"2024-09-20T18:56:29.853540+0000","last_clean":"2024-09-20T18:56:29.853540+0000","last_became_active":"2024-09-20T18:55:28.824801+0000","last_became_peered":"2024-09-20T18:55:28.824801+0000","last_unstale":"2024-09-20T18:56:29.853540+0000","last_undegraded":"2024-09-20T18:56:29.853540+0000","last_fullsized":"2024-09-20T18:56:29.853540+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:54:12.373309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:12.373309+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":359,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.692860+0000","last_change":"2024-09-20T18:55:24.729301+0000","last_active":"2024-09-20T18:56:40.692860+0000","last_peered":"2024-09-20T18:56:40.692860+0000","last_clean":"2024-09-20T18:56:40.692860+0000","last_became_active":"2024-09-20T18:49:45.944256+0000","last_became_peered":"2024-09-20T18:49:45.944256+0000","last_unstale":"2024-09-20T18:56:40.692860+0000","last_undegraded":"2024-09-20T18:56:40.692860+0000","last_fullsized":"2024-09-20T18:56:40.692860+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-20T18:55:24.729219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:24.729219+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"184'9","reported_seq":417,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.691783+0000","last_change":"2024-09-20T18:55:28.824771+0000","last_active":"2024-09-20T18:56:40.691783+0000","last_peered":"2024-09-20T18:56:40.691783+0000","last_clean":"2024-09-20T18:56:40.691783+0000","last_became_active":"2024-09-20T18:55:28.818085+0000","last_became_peered":"2024-09-20T18:55:28.818085+0000","last_unstale":"2024-09-20T18:56:40.691783+0000","last_undegraded":"2024-09-20T18:56:40.691783+0000","last_fullsized":"2024-09-20T18:56:40.691783+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:52:05.788924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.788924+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":318,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.852201+0000","last_change":"2024-09-20T18:55:28.832276+0000","last_active":"2024-09-20T18:56:29.852201+0000","last_peered":"2024-09-20T18:56:29.852201+0000","last_clean":"2024-09-20T18:56:29.852201+0000","last_became_active":"2024-09-20T18:55:28.830380+0000","last_became_peered":"2024-09-20T18:55:28.830380+0000","last_unstale":"2024-09-20T18:56:29.852201+0000","last_undegraded":"2024-09-20T18:56:29.852201+0000","last_fullsized":"2024-09-20T18:56:29.852201+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:14.247398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:14.247398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:30.298253+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":61,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.865736+0000","last_change":"2024-09-20T18:55:28.835344+0000","last_active":"2024-09-20T18:56:29.865736+0000","last_peered":"2024-09-20T18:56:29.865736+0000","last_clean":"2024-09-20T18:56:29.865736+0000","last_became_active":"2024-09-20T18:55:28.835010+0000","last_became_peered":"2024-09-20T18:55:28.835010+0000","last_unstale":"2024-09-20T18:56:29.865736+0000","last_undegraded":"2024-09-20T18:56:29.865736+0000","last_fullsized":"2024-09-20T18:56:29.865736+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:26.721735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:26.721735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:49.502875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":328,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.863956+0000","last_change":"2024-09-20T18:54:44.194161+0000","last_active":"2024-09-20T18:56:29.863956+0000","last_peered":"2024-09-20T18:56:29.863956+0000","last_clean":"2024-09-20T18:56:29.863956+0000","last_became_active":"2024-09-20T18:49:48.500662+0000","last_became_peered":"2024-09-20T18:49:48.500662+0000","last_unstale":"2024-09-20T18:56:29.863956+0000","last_undegraded":"2024-09-20T18:56:29.863956+0000","last_fullsized":"2024-09-20T18:56:29.863956+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-20T18:54:44.194089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.194089+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":314,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.851317+0000","last_change":"2024-09-20T18:53:16.220554+0000","last_active":"2024-09-20T18:56:29.851317+0000","last_peered":"2024-09-20T18:56:29.851317+0000","last_clean":"2024-09-20T18:56:29.851317+0000","last_became_active":"2024-09-20T18:49:49.956749+0000","last_became_peered":"2024-09-20T18:49:49.956749+0000","last_unstale":"2024-09-20T18:56:29.851317+0000","last_undegraded":"2024-09-20T18:56:29.851317+0000","last_fullsized":"2024-09-20T18:56:29.851317+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:16.220490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:16.220490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":333,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.692657+0000","last_change":"2024-09-20T18:55:28.827897+0000","last_active":"2024-09-20T18:56:40.692657+0000","last_peered":"2024-09-20T18:56:40.692657+0000","last_clean":"2024-09-20T18:56:40.692657+0000","last_became_active":"2024-09-20T18:55:28.827418+0000","last_became_peered":"2024-09-20T18:55:28.827418+0000","last_unstale":"2024-09-20T18:56:40.692657+0000","last_undegraded":"2024-09-20T18:56:40.692657+0000","last_fullsized":"2024-09-20T18:56:40.692657+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.637739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.637739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":334,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.880405+0000","last_change":"2024-09-20T18:54:44.830872+0000","last_active":"2024-09-20T18:56:29.880405+0000","last_peered":"2024-09-20T18:56:29.880405+0000","last_clean":"2024-09-20T18:56:29.880405+0000","last_became_active":"2024-09-20T18:49:48.500803+0000","last_became_peered":"2024-09-20T18:49:48.500803+0000","last_unstale":"2024-09-20T18:56:29.880405+0000","last_undegraded":"2024-09-20T18:56:29.880405+0000","last_fullsized":"2024-09-20T18:56:29.880405+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:44.830790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.830790+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":337,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.691972+0000","last_change":"2024-09-20T18:55:29.751718+0000","last_active":"2024-09-20T18:56:40.691972+0000","last_peered":"2024-09-20T18:56:40.691972+0000","last_clean":"2024-09-20T18:56:40.691972+0000","last_became_active":"2024-09-20T18:55:28.833799+0000","last_became_peered":"2024-09-20T18:55:28.833799+0000","last_unstale":"2024-09-20T18:56:40.691972+0000","last_undegraded":"2024-09-20T18:56:40.691972+0000","last_fullsized":"2024-09-20T18:56:40.691972+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:29.751642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:29.751642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:40.748541+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":315,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.863799+0000","last_change":"2024-09-20T18:53:50.253721+0000","last_active":"2024-09-20T18:56:29.863799+0000","last_peered":"2024-09-20T18:56:29.863799+0000","last_clean":"2024-09-20T18:56:29.863799+0000","last_became_active":"2024-09-20T18:49:49.961947+0000","last_became_peered":"2024-09-20T18:49:49.961947+0000","last_unstale":"2024-09-20T18:56:29.863799+0000","last_undegraded":"2024-09-20T18:56:29.863799+0000","last_fullsized":"2024-09-20T18:56:29.863799+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:50.253660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:50.253660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"66'1","reported_seq":321,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.845185+0000","last_change":"2024-09-20T18:54:20.681878+0000","last_active":"2024-09-20T18:56:29.845185+0000","last_peered":"2024-09-20T18:56:29.845185+0000","last_clean":"2024-09-20T18:56:29.845185+0000","last_became_active":"2024-09-20T18:49:51.965121+0000","last_became_peered":"2024-09-20T18:49:51.965121+0000","last_unstale":"2024-09-20T18:56:29.845185+0000","last_undegraded":"2024-09-20T18:56:29.845185+0000","last_fullsized":"2024-09-20T18:56:29.845185+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T18:54:20.681790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:20.681790+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":326,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.852677+0000","last_change":"2024-09-20T18:55:12.205118+0000","last_active":"2024-09-20T18:56:29.852677+0000","last_peered":"2024-09-20T18:56:29.852677+0000","last_clean":"2024-09-20T18:56:29.852677+0000","last_became_active":"2024-09-20T18:49:48.501011+0000","last_became_peered":"2024-09-20T18:49:48.501011+0000","last_unstale":"2024-09-20T18:56:29.852677+0000","last_undegraded":"2024-09-20T18:56:29.852677+0000","last_fullsized":"2024-09-20T18:56:29.852677+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:55:12.205069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:12.205069+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":330,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.868867+0000","last_change":"2024-09-20T18:55:27.028184+0000","last_active":"2024-09-20T18:56:29.868867+0000","last_peered":"2024-09-20T18:56:29.868867+0000","last_clean":"2024-09-20T18:56:29.868867+0000","last_became_active":"2024-09-20T18:49:45.938511+0000","last_became_peered":"2024-09-20T18:49:45.938511+0000","last_unstale":"2024-09-20T18:56:29.868867+0000","last_undegraded":"2024-09-20T18:56:29.868867+0000","last_fullsized":"2024-09-20T18:56:29.868867+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:27.028126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:27.028126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:47.494086+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":414,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:31.073266+0000","last_change":"2024-09-20T18:55:27.284156+0000","last_active":"2024-09-20T18:56:31.073266+0000","last_peered":"2024-09-20T18:56:31.073266+0000","last_clean":"2024-09-20T18:56:31.073266+0000","last_became_active":"2024-09-20T18:49:49.960550+0000","last_became_peered":"2024-09-20T18:49:49.960550+0000","last_unstale":"2024-09-20T18:56:31.073266+0000","last_undegraded":"2024-09-20T18:56:31.073266+0000","last_fullsized":"2024-09-20T18:56:31.073266+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:27.284086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:27.284086+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:48.225116+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":319,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.842053+0000","last_change":"2024-09-20T18:55:12.348079+0000","last_active":"2024-09-20T18:56:29.842053+0000","last_peered":"2024-09-20T18:56:29.842053+0000","last_clean":"2024-09-20T18:56:29.842053+0000","last_became_active":"2024-09-20T18:49:51.966254+0000","last_became_peered":"2024-09-20T18:49:51.966254+0000","last_unstale":"2024-09-20T18:56:29.842053+0000","last_undegraded":"2024-09-20T18:56:29.842053+0000","last_fullsized":"2024-09-20T18:56:29.842053+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:12.347993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:12.347993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:32.305729+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":345,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.857683+0000","last_change":"2024-09-20T18:51:15.223913+0000","last_active":"2024-09-20T18:56:29.857683+0000","last_peered":"2024-09-20T18:56:29.857683+0000","last_clean":"2024-09-20T18:56:29.857683+0000","last_became_active":"2024-09-20T18:49:58.997494+0000","last_became_peered":"2024-09-20T18:49:58.997494+0000","last_unstale":"2024-09-20T18:56:29.857683+0000","last_undegraded":"2024-09-20T18:56:29.857683+0000","last_fullsized":"2024-09-20T18:56:29.857683+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:51:15.223826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:15.223826+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":338,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.692538+0000","last_change":"2024-09-20T18:54:57.632458+0000","last_active":"2024-09-20T18:56:40.692538+0000","last_peered":"2024-09-20T18:56:40.692538+0000","last_clean":"2024-09-20T18:56:40.692538+0000","last_became_active":"2024-09-20T18:49:45.949492+0000","last_became_peered":"2024-09-20T18:49:45.949492+0000","last_unstale":"2024-09-20T18:56:40.692538+0000","last_undegraded":"2024-09-20T18:56:40.692538+0000","last_fullsized":"2024-09-20T18:56:40.692538+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.632380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.632380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":412,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.852450+0000","last_change":"2024-09-20T18:55:28.302353+0000","last_active":"2024-09-20T18:56:29.852450+0000","last_peered":"2024-09-20T18:56:29.852450+0000","last_clean":"2024-09-20T18:56:29.852450+0000","last_became_active":"2024-09-20T18:49:49.962243+0000","last_became_peered":"2024-09-20T18:49:49.962243+0000","last_unstale":"2024-09-20T18:56:29.852450+0000","last_undegraded":"2024-09-20T18:56:29.852450+0000","last_fullsized":"2024-09-20T18:56:29.852450+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:28.302303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:28.302303+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:55.176573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":318,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.880112+0000","last_change":"2024-09-20T18:55:11.918284+0000","last_active":"2024-09-20T18:56:29.880112+0000","last_peered":"2024-09-20T18:56:29.880112+0000","last_clean":"2024-09-20T18:56:29.880112+0000","last_became_active":"2024-09-20T18:49:51.969093+0000","last_became_peered":"2024-09-20T18:49:51.969093+0000","last_unstale":"2024-09-20T18:56:29.880112+0000","last_undegraded":"2024-09-20T18:56:29.880112+0000","last_fullsized":"2024-09-20T18:56:29.880112+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:11.918210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:11.918210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":357,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.841895+0000","last_change":"2024-09-20T18:55:28.854249+0000","last_active":"2024-09-20T18:56:29.841895+0000","last_peered":"2024-09-20T18:56:29.841895+0000","last_clean":"2024-09-20T18:56:29.841895+0000","last_became_active":"2024-09-20T18:55:28.826721+0000","last_became_peered":"2024-09-20T18:55:28.826721+0000","last_unstale":"2024-09-20T18:56:29.841895+0000","last_undegraded":"2024-09-20T18:56:29.841895+0000","last_fullsized":"2024-09-20T18:56:29.841895+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:53:43.878921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:43.878921+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":322,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.842451+0000","last_change":"2024-09-20T18:55:28.822690+0000","last_active":"2024-09-20T18:56:29.842451+0000","last_peered":"2024-09-20T18:56:29.842451+0000","last_clean":"2024-09-20T18:56:29.842451+0000","last_became_active":"2024-09-20T18:55:28.822258+0000","last_became_peered":"2024-09-20T18:55:28.822258+0000","last_unstale":"2024-09-20T18:56:29.842451+0000","last_undegraded":"2024-09-20T18:56:29.842451+0000","last_fullsized":"2024-09-20T18:56:29.842451+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:05.985551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.985551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":307,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.847442+0000","last_change":"2024-09-20T18:51:14.271954+0000","last_active":"2024-09-20T18:56:29.847442+0000","last_peered":"2024-09-20T18:56:29.847442+0000","last_clean":"2024-09-20T18:56:29.847442+0000","last_became_active":"2024-09-20T18:49:49.957067+0000","last_became_peered":"2024-09-20T18:49:49.957067+0000","last_unstale":"2024-09-20T18:56:29.847442+0000","last_undegraded":"2024-09-20T18:56:29.847442+0000","last_fullsized":"2024-09-20T18:56:29.847442+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:14.271719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.271719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":303,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.856075+0000","last_change":"2024-09-20T18:51:14.210438+0000","last_active":"2024-09-20T18:56:29.856075+0000","last_peered":"2024-09-20T18:56:29.856075+0000","last_clean":"2024-09-20T18:56:29.856075+0000","last_became_active":"2024-09-20T18:49:51.966915+0000","last_became_peered":"2024-09-20T18:49:51.966915+0000","last_unstale":"2024-09-20T18:56:29.856075+0000","last_undegraded":"2024-09-20T18:56:29.856075+0000","last_fullsized":"2024-09-20T18:56:29.856075+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:14.210358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.210358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":360,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.847521+0000","last_change":"2024-09-20T18:55:28.838698+0000","last_active":"2024-09-20T18:56:29.847521+0000","last_peered":"2024-09-20T18:56:29.847521+0000","last_clean":"2024-09-20T18:56:29.847521+0000","last_became_active":"2024-09-20T18:55:28.827023+0000","last_became_peered":"2024-09-20T18:55:28.827023+0000","last_unstale":"2024-09-20T18:56:29.847521+0000","last_undegraded":"2024-09-20T18:56:29.847521+0000","last_fullsized":"2024-09-20T18:56:29.847521+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:53:53.263976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:53.263976+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":330,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.692947+0000","last_change":"2024-09-20T18:54:42.487465+0000","last_active":"2024-09-20T18:56:40.692947+0000","last_peered":"2024-09-20T18:56:40.692947+0000","last_clean":"2024-09-20T18:56:40.692947+0000","last_became_active":"2024-09-20T18:49:45.944357+0000","last_became_peered":"2024-09-20T18:49:45.944357+0000","last_unstale":"2024-09-20T18:56:40.692947+0000","last_undegraded":"2024-09-20T18:56:40.692947+0000","last_fullsized":"2024-09-20T18:56:40.692947+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:42.487403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:42.487403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":315,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.853393+0000","last_change":"2024-09-20T18:54:18.108239+0000","last_active":"2024-09-20T18:56:29.853393+0000","last_peered":"2024-09-20T18:56:29.853393+0000","last_clean":"2024-09-20T18:56:29.853393+0000","last_became_active":"2024-09-20T18:49:49.962523+0000","last_became_peered":"2024-09-20T18:49:49.962523+0000","last_unstale":"2024-09-20T18:56:29.853393+0000","last_undegraded":"2024-09-20T18:56:29.853393+0000","last_fullsized":"2024-09-20T18:56:29.853393+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:18.108175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:18.108175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":318,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.865374+0000","last_change":"2024-09-20T18:54:26.882441+0000","last_active":"2024-09-20T18:56:29.865374+0000","last_peered":"2024-09-20T18:56:29.865374+0000","last_clean":"2024-09-20T18:56:29.865374+0000","last_became_active":"2024-09-20T18:49:51.967313+0000","last_became_peered":"2024-09-20T18:49:51.967313+0000","last_unstale":"2024-09-20T18:56:29.865374+0000","last_undegraded":"2024-09-20T18:56:29.865374+0000","last_fullsized":"2024-09-20T18:56:29.865374+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:26.882375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:26.882375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":357,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.856252+0000","last_change":"2024-09-20T18:55:28.843363+0000","last_active":"2024-09-20T18:56:29.856252+0000","last_peered":"2024-09-20T18:56:29.856252+0000","last_clean":"2024-09-20T18:56:29.856252+0000","last_became_active":"2024-09-20T18:55:28.830801+0000","last_became_peered":"2024-09-20T18:55:28.830801+0000","last_unstale":"2024-09-20T18:56:29.856252+0000","last_undegraded":"2024-09-20T18:56:29.856252+0000","last_fullsized":"2024-09-20T18:56:29.856252+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:54:21.106677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:21.106677+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":330,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.851400+0000","last_change":"2024-09-20T18:55:19.258723+0000","last_active":"2024-09-20T18:56:29.851400+0000","last_peered":"2024-09-20T18:56:29.851400+0000","last_clean":"2024-09-20T18:56:29.851400+0000","last_became_active":"2024-09-20T18:49:45.942105+0000","last_became_peered":"2024-09-20T18:49:45.942105+0000","last_unstale":"2024-09-20T18:56:29.851400+0000","last_undegraded":"2024-09-20T18:56:29.851400+0000","last_fullsized":"2024-09-20T18:56:29.851400+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:19.258674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:19.258674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:33.782760+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":337,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.691415+0000","last_change":"2024-09-20T18:55:28.825004+0000","last_active":"2024-09-20T18:56:40.691415+0000","last_peered":"2024-09-20T18:56:40.691415+0000","last_clean":"2024-09-20T18:56:40.691415+0000","last_became_active":"2024-09-20T18:55:28.824355+0000","last_became_peered":"2024-09-20T18:55:28.824355+0000","last_unstale":"2024-09-20T18:56:40.691415+0000","last_undegraded":"2024-09-20T18:56:40.691415+0000","last_fullsized":"2024-09-20T18:56:40.691415+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:46.329547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:46.329547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":310,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.853517+0000","last_change":"2024-09-20T18:55:28.825607+0000","last_active":"2024-09-20T18:56:29.853517+0000","last_peered":"2024-09-20T18:56:29.853517+0000","last_clean":"2024-09-20T18:56:29.853517+0000","last_became_active":"2024-09-20T18:55:28.825296+0000","last_became_peered":"2024-09-20T18:55:28.825296+0000","last_unstale":"2024-09-20T18:56:29.853517+0000","last_undegraded":"2024-09-20T18:56:29.853517+0000","last_fullsized":"2024-09-20T18:56:29.853517+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:43.118805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:43.118805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":346,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.858096+0000","last_change":"2024-09-20T18:55:28.836789+0000","last_active":"2024-09-20T18:56:29.858096+0000","last_peered":"2024-09-20T18:56:29.858096+0000","last_clean":"2024-09-20T18:56:29.858096+0000","last_became_active":"2024-09-20T18:55:28.824579+0000","last_became_peered":"2024-09-20T18:55:28.824579+0000","last_unstale":"2024-09-20T18:56:29.858096+0000","last_undegraded":"2024-09-20T18:56:29.858096+0000","last_fullsized":"2024-09-20T18:56:29.858096+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-20T18:55:20.282193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:20.282193+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:40.485847+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":330,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.867218+0000","last_change":"2024-09-20T18:54:55.900487+0000","last_active":"2024-09-20T18:56:29.867218+0000","last_peered":"2024-09-20T18:56:29.867218+0000","last_clean":"2024-09-20T18:56:29.867218+0000","last_became_active":"2024-09-20T18:49:45.940988+0000","last_became_peered":"2024-09-20T18:49:45.940988+0000","last_unstale":"2024-09-20T18:56:29.867218+0000","last_undegraded":"2024-09-20T18:56:29.867218+0000","last_fullsized":"2024-09-20T18:56:29.867218+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:55.899611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:55.899611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":61,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.841049+0000","last_change":"2024-09-20T18:55:28.832864+0000","last_active":"2024-09-20T18:56:29.841049+0000","last_peered":"2024-09-20T18:56:29.841049+0000","last_clean":"2024-09-20T18:56:29.841049+0000","last_became_active":"2024-09-20T18:55:28.832534+0000","last_became_peered":"2024-09-20T18:55:28.832534+0000","last_unstale":"2024-09-20T18:56:29.841049+0000","last_undegraded":"2024-09-20T18:56:29.841049+0000","last_fullsized":"2024-09-20T18:56:29.841049+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:52.635138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:52.635138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":310,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.692079+0000","last_change":"2024-09-20T18:51:09.761331+0000","last_active":"2024-09-20T18:56:40.692079+0000","last_peered":"2024-09-20T18:56:40.692079+0000","last_clean":"2024-09-20T18:56:40.692079+0000","last_became_active":"2024-09-20T18:49:51.968662+0000","last_became_peered":"2024-09-20T18:49:51.968662+0000","last_unstale":"2024-09-20T18:56:40.692079+0000","last_undegraded":"2024-09-20T18:56:40.692079+0000","last_fullsized":"2024-09-20T18:56:40.692079+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:09.761283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:09.761283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":295,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.847663+0000","last_change":"2024-09-20T18:49:51.968476+0000","last_active":"2024-09-20T18:56:29.847663+0000","last_peered":"2024-09-20T18:56:29.847663+0000","last_clean":"2024-09-20T18:56:29.847663+0000","last_became_active":"2024-09-20T18:49:51.968216+0000","last_became_peered":"2024-09-20T18:49:51.968216+0000","last_unstale":"2024-09-20T18:56:29.847663+0000","last_undegraded":"2024-09-20T18:56:29.847663+0000","last_fullsized":"2024-09-20T18:56:29.847663+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":323,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.858306+0000","last_change":"2024-09-20T18:55:28.295445+0000","last_active":"2024-09-20T18:56:29.858306+0000","last_peered":"2024-09-20T18:56:29.858306+0000","last_clean":"2024-09-20T18:56:29.858306+0000","last_became_active":"2024-09-20T18:49:45.937992+0000","last_became_peered":"2024-09-20T18:49:45.937992+0000","last_unstale":"2024-09-20T18:56:29.858306+0000","last_undegraded":"2024-09-20T18:56:29.858306+0000","last_fullsized":"2024-09-20T18:56:29.858306+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:28.295399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:28.295399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:39.921452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":112,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.851907+0000","last_change":"2024-09-20T18:55:28.852468+0000","last_active":"2024-09-20T18:56:29.851907+0000","last_peered":"2024-09-20T18:56:29.851907+0000","last_clean":"2024-09-20T18:56:29.851907+0000","last_became_active":"2024-09-20T18:55:28.834675+0000","last_became_peered":"2024-09-20T18:55:28.834675+0000","last_unstale":"2024-09-20T18:56:29.851907+0000","last_undegraded":"2024-09-20T18:56:29.851907+0000","last_fullsized":"2024-09-20T18:56:29.851907+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:54:44.719676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.719676+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":12,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":307,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-20T18:56:29.880199+0000","last_change":"2024-09-20T18:51:28.120100+0000","last_active":"2024-09-20T18:56:29.880199+0000","last_peered":"2024-09-20T18:56:29.880199+0000","last_clean":"2024-09-20T18:56:29.880199+0000","last_became_active":"2024-09-20T18:49:49.957582+0000","last_became_peered":"2024-09-20T18:49:49.957582+0000","last_unstale":"2024-09-20T18:56:29.880199+0000","last_undegraded":"2024-09-20T18:56:29.880199+0000","last_fullsized":"2024-09-20T18:56:29.880199+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:28.119694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:28.119694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":6207701016,"num_objects":94723,"num_object_clones":0,"num_object_copies":284169,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":354,"num_objects_misplaced":11200,"num_objects_unfound":0,"num_objects_dirty":94723,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":137455,"num_write_kb":7429633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16754,"num_bytes_recovered":1097990144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":19609432064,"data_stored":19609419848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74288,"ondisk_log_size":74288,"up":48,"acting":45,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":67,"ondisk_log_size":67,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":695,"num_read_kb":452,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":1474,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":12140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":394,"ondisk_log_size":394,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":5275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":410,"num_read_kb":405,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266775,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1999372,"kb_used_data":1819636,"kb_used_omap":1,"kb_used_meta":179710,"kb_avail":91745780,"statfs":{"total":95995035648,"available":93947678720,"internally_reserved":0,"allocated":1863307264,"data_stored":1862664997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":184023498},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":0,"seq":193273528414,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757346,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3267084,"kb_used_data":3033212,"kb_used_omap":1,"kb_used_meta":233854,"kb_avail":90478068,"statfs":{"total":95995035648,"available":92649541632,"internally_reserved":0,"allocated":3106009088,"data_stored":3105395672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":239466949},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986279,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2104672,"kb_used_data":1920784,"kb_used_omap":1,"kb_used_meta":183870,"kb_avail":91640480,"statfs":{"total":95995035648,"available":93839851520,"internally_reserved":0,"allocated":1966882816,"data_stored":1966249928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":188283333},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182510,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2835040,"kb_used_data":2524752,"kb_used_omap":1,"kb_used_meta":310206,"kb_avail":90910112,"statfs":{"total":95995035648,"available":93091954688,"internally_reserved":0,"allocated":2585346048,"data_stored":2584712114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":317651399},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411443,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3331268,"kb_used_data":3007156,"kb_used_omap":1,"kb_used_meta":324094,"kb_avail":90413884,"statfs":{"total":95995035648,"available":92583817216,"internally_reserved":0,"allocated":3079327744,"data_stored":3078722769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":331872709},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,3,2,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574969,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2919096,"kb_used_data":2667112,"kb_used_omap":1,"kb_used_meta":251902,"kb_avail":90826056,"statfs":{"total":95995035648,"available":93005881344,"internally_reserved":0,"allocated":2731122688,"data_stored":2730466605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":257948101},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738495,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2796384,"kb_used_data":2476432,"kb_used_omap":1,"kb_used_meta":319934,"kb_avail":90948768,"statfs":{"total":95995035648,"available":93131538432,"internally_reserved":0,"allocated":2535866368,"data_stored":2535233054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":327612869},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2523660288,"data_stored":2523660288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2719088640,"data_stored":2719088640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3075280896,"data_stored":3075276824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2570715136,"data_stored":2570715136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1959919616,"data_stored":1959919616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3099529216,"data_stored":3099525144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1800142848,"data_stored":1800142848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1861095424,"data_stored":1861091352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T18:56:44.226 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph status --format=json 2024-09-20T18:56:44.311 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:44.330 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_blocked_ops 2024-09-20T18:56:44.339 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:44.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:44.341 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:44.402 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T18:56:44.441 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:56:44.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:44.500 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:44.500 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:56:44.600 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:44.625 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:44.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:44.626 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:44.639 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:56:44.727 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:44.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:44.797 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:44.797 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:44.898 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:44.924 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:44.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:44.925 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:44.932 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:56:44.932 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:56:44.971 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:56:44.971 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:56:44.993 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:56:45.009 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:56:45.009 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:56:45.010 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:44 smithi083 ceph-mon[29421]: pgmap v333: 148 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 144 active+clean; 5.5 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 738 op/s; 544/271485 objects degraded (0.200%); 9727/271485 objects misplaced (3.583%); 12 MiB/s, 197 objects/s recovering 2024-09-20T18:56:45.010 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:44 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 544/271485 objects degraded (0.200%), 1 pg degraded, 3 pgs undersized (PG_DEGRADED) 2024-09-20T18:56:45.010 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:44 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:56:45.010 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:44 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:56:45.010 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:44 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:56:45.013 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:44 smithi083 ceph-mon[36260]: pgmap v333: 148 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 144 active+clean; 5.5 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 738 op/s; 544/271485 objects degraded (0.200%); 9727/271485 objects misplaced (3.583%); 12 MiB/s, 197 objects/s recovering 2024-09-20T18:56:45.013 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:44 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 544/271485 objects degraded (0.200%), 1 pg degraded, 3 pgs undersized (PG_DEGRADED) 2024-09-20T18:56:45.013 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:44 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:56:45.013 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:44 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:56:45.013 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:44 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:56:45.020 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:44 smithi142 ceph-mon[32508]: pgmap v333: 148 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 144 active+clean; 5.5 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 738 op/s; 544/271485 objects degraded (0.200%); 9727/271485 objects misplaced (3.583%); 12 MiB/s, 197 objects/s recovering 2024-09-20T18:56:45.020 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:44 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 544/271485 objects degraded (0.200%), 1 pg degraded, 3 pgs undersized (PG_DEGRADED) 2024-09-20T18:56:45.020 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:44 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:56:45.020 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:44 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:56:45.020 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:44 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:56:45.025 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:45.035 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:56:45.049 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:56:45.049 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:56:45.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:45.077 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:45.077 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:45.082 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:56:45.082 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:56:45.117 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:56:45.118 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:56:45.147 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:56:45.147 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:56:45.179 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:56:45.186 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:56:45.187 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:56:45.188 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T18:56:45.188 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:56:45.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:45.224 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:45.224 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:56:45.325 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:56:45.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:45.369 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:45.369 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:56:45.470 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:45.507 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:45.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:45.508 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:45.609 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:56:45.647 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:45.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:45.648 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:56:45.749 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:45.777 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:45.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:45.779 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:45.880 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:45.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:45.918 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:45.918 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:45.963 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:45 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2557247917' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:56:45.963 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:45 smithi083 ceph-mon[29421]: pgmap v334: 148 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 144 active+clean; 5.5 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 728 B/s rd, 0 B/s wr, 630 op/s; 544/271485 objects degraded (0.200%); 9727/271485 objects misplaced (3.583%); 11 MiB/s, 168 objects/s recovering 2024-09-20T18:56:45.963 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:45 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:56:45.963 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:45 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:56:45.963 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:45 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2557247917' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:56:45.963 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:45 smithi083 ceph-mon[36260]: pgmap v334: 148 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 144 active+clean; 5.5 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 728 B/s rd, 0 B/s wr, 630 op/s; 544/271485 objects degraded (0.200%); 9727/271485 objects misplaced (3.583%); 11 MiB/s, 168 objects/s recovering 2024-09-20T18:56:45.963 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:45 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:56:45.964 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:45 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:56:45.974 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:45 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2557247917' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:56:45.974 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:45 smithi142 ceph-mon[32508]: pgmap v334: 148 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 144 active+clean; 5.5 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 728 B/s rd, 0 B/s wr, 630 op/s; 544/271485 objects degraded (0.200%); 9727/271485 objects misplaced (3.583%); 11 MiB/s, 168 objects/s recovering 2024-09-20T18:56:45.974 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:45 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:56:45.974 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:45 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:56:46.012 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_historic_ops 2024-09-20T18:56:46.019 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:56:46.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:46.066 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:46.066 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:56:46.168 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:46.202 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:46.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:46.203 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:46.304 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:46.357 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:46.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:46.358 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:46.360 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:56:46.459 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:46.501 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:46.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:46.502 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:46.602 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:56:46.640 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:46.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:46.641 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:56:46.742 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:56:46.778 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:46.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:46.779 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:56:46.880 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:56:46.912 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:46.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:46.913 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:56:47.004 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:56:47.004 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"1ed5a50a-7780-11ef-baf5-efdc52797490","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 713/258132 objects degraded (0.276%), 1 pg degraded, 3 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":668,"monmap":{"epoch":3,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":210,"num_osds":8,"num_up_osds":8,"osd_up_since":1726858174,"num_in_osds":7,"osd_in_since":1726858527,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":144},{"state_name":"active+recovering+undersized+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1},{"state_name":"active+remapped+backfilling","count":1}],"num_pgs":148,"num_pools":7,"num_objects":86044,"data_bytes":5626468915,"bytes_used":18267123712,"bytes_avail":653698125824,"bytes_total":671965249536,"degraded_objects":713,"degraded_total":258132,"degraded_ratio":0.0027621526970697165,"misplaced_objects":8437,"misplaced_total":258132,"misplaced_ratio":0.032684827917499576,"recovering_objects_per_sec":278,"recovering_bytes_per_sec":18239764,"recovering_keys_per_sec":0,"num_objects_recovered":2233,"num_bytes_recovered":146341888,"num_keys_recovered":0,"read_bytes_sec":1276,"write_bytes_sec":0,"read_op_per_sec":1,"write_op_per_sec":1106},"fsmap":{"epoch":1,"btime":"2024-09-20T18:44:10:399616+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"services":{"dashboard":"https://172.21.15.83:8443/","prometheus":"http://172.21.15.83:9283/"}},"servicemap":{"epoch":14,"modified":"2024-09-20T18:51:16.768808+0000","services":{"rgw":{"daemons":{"summary":"","14460":{"start_epoch":12,"start_stamp":"2024-09-20T18:49:53.053671+0000","gid":14460,"addr":"172.21.15.83:0/1555339426","metadata":{"arch":"x86_64","ceph_release":"squid","ceph_version":"ceph version 19.3.0-5099-g88deadb8 (88deadb8082f4a33832e228ccefc59cedba158a5) squid (dev)","ceph_version_short":"19.3.0-5099-g88deadb8","container_hostname":"smithi083","container_image":"quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2","cpu":"Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz","distro":"centos","distro_description":"CentOS Stream 9","distro_version":"9","frontend_config#0":"beast port=80","frontend_type#0":"beast","hostname":"smithi083","id":"foo.a","kernel_description":"#1 SMP PREEMPT_DYNAMIC Thu Sep 12 15:49:37 UTC 2024","kernel_version":"5.14.0-508.el9.x86_64","mem_swap_kb":"0","mem_total_kb":"32487292","num_handles":"1","os":"Linux","pid":"2","realm_id":"","realm_name":"","zone_id":"41426759-9535-4245-bcfe-8d158f74a134","zone_name":"default","zonegroup_id":"fb472636-52d3-4587-9964-925ef38835b6","zonegroup_name":"default"},"task_status":{}}}}}},"progress_events":{}} 2024-09-20T18:56:47.013 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:56:47.020 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:56:46 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:56:47.055 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:47.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:47.056 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:56:47.157 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:56:47.209 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:47.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:47.210 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:56:47.310 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:47.340 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:47.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:47.341 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:47.442 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:56:47.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:47.478 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:47.478 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:56:47.579 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:56:47.615 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:47.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:47.616 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:56:47.717 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:47.742 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:47.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:47.743 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:47.843 INFO:tasks.cephadm.ceph_manager.ceph:making progress, resetting timeout 2024-09-20T18:56:47.843 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T18:56:47.844 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:47.865 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:47.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:47.866 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:47.967 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:47.986 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-20T18:56:47.995 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:47.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:47.997 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:48.015 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T18:56:48.050 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:47 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:56:48.050 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:47 smithi083 ceph-mon[29421]: pgmap v335: 148 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 144 active+clean; 5.2 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.11k op/s; 713/258132 objects degraded (0.276%); 8437/258132 objects misplaced (3.268%); 17 MiB/s, 278 objects/s recovering 2024-09-20T18:56:48.050 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:47 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/762789614' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-09-20T18:56:48.050 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:47 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2668967383' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:56:48.050 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:47 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:56:48.051 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:47 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:56:48.051 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:47 smithi083 ceph-mon[36260]: pgmap v335: 148 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 144 active+clean; 5.2 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.11k op/s; 713/258132 objects degraded (0.276%); 8437/258132 objects misplaced (3.268%); 17 MiB/s, 278 objects/s recovering 2024-09-20T18:56:48.051 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:47 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/762789614' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-09-20T18:56:48.051 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:47 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2668967383' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:56:48.051 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:47 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:56:48.098 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:48.122 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:48.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:48.123 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:48.223 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:48.247 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:48.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:48.248 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:48.270 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:47 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:56:48.270 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:47 smithi142 ceph-mon[32508]: pgmap v335: 148 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 144 active+clean; 5.2 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.11k op/s; 713/258132 objects degraded (0.276%); 8437/258132 objects misplaced (3.268%); 17 MiB/s, 278 objects/s recovering 2024-09-20T18:56:48.270 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:47 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/762789614' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-09-20T18:56:48.270 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:47 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2668967383' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:56:48.270 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:47 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:56:48.350 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:48.375 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:48.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:48.376 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:48.477 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:48.501 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:48.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:48.502 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:48.603 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:48.632 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:48.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:48.633 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:48.734 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:56:48.768 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:48.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:48.769 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:56:48.870 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:56:48.904 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:48.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:48.905 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:56:49.005 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:56:49.006 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:49.017 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:56:49.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:49.028 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:49.028 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:49.129 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:56:49.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:49.175 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:49.175 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:56:49.270 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:48 smithi142 ceph-mon[32508]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:56:49.270 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:49 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T18:56:49.270 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:49 smithi142 ceph-mon[32508]: osdmap e211: 8 total, 8 up, 7 in 2024-09-20T18:56:49.276 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:56:49.300 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:48 smithi083 ceph-mon[29421]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:56:49.300 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:48 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T18:56:49.300 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:48 smithi083 ceph-mon[29421]: osdmap e211: 8 total, 8 up, 7 in 2024-09-20T18:56:49.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:48 smithi083 ceph-mon[36260]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:56:49.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:48 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T18:56:49.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:48 smithi083 ceph-mon[36260]: osdmap e211: 8 total, 8 up, 7 in 2024-09-20T18:56:49.301 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:49.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:49.303 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:56:49.403 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:49.450 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:49.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:49.451 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:49.552 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:49.581 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:49.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:49.582 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:49.683 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:49.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:49.728 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:49.728 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:49.829 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:49.851 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:49.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:49.852 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:49.953 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:49.984 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:49.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:49.985 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:50.086 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:50.114 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:50.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:50.115 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:50.216 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:56:50.253 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:50.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:50.255 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:56:50.269 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:49 smithi142 ceph-mon[32508]: pgmap v337: 148 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 144 active+clean; 5.2 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.08k op/s; 713/258132 objects degraded (0.276%); 8437/258132 objects misplaced (3.268%); 17 MiB/s, 271 objects/s recovering 2024-09-20T18:56:50.300 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:49 smithi083 ceph-mon[29421]: pgmap v337: 148 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 144 active+clean; 5.2 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.08k op/s; 713/258132 objects degraded (0.276%); 8437/258132 objects misplaced (3.268%); 17 MiB/s, 271 objects/s recovering 2024-09-20T18:56:50.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:50 smithi083 ceph-mon[36260]: pgmap v337: 148 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 144 active+clean; 5.2 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.08k op/s; 713/258132 objects degraded (0.276%); 8437/258132 objects misplaced (3.268%); 17 MiB/s, 271 objects/s recovering 2024-09-20T18:56:50.355 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:56:50.396 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:50.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:50.398 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:56:50.499 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:56:50.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:50.537 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:50.537 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:56:50.638 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:50.665 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:50.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:50.666 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:50.766 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:50.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:50.803 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:50.803 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:50.903 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:50.909 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:56:50.911 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-20T18:56:50.927 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:50.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:50.928 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:51.001 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T18:56:51.016 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:56:50 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:56:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:56:51.029 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:51.052 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:51.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:51.053 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:51.104 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:56:51.105 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:56:51.130 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:56:51.130 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:56:51.153 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:51.157 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:56:51.157 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:56:51.180 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:51.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:51.181 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:51.192 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:56:51.192 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:56:51.230 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:56:51.231 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:56:51.267 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:51 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 713/258132 objects degraded (0.276%), 1 pg degraded, 3 pgs undersized (PG_DEGRADED) 2024-09-20T18:56:51.267 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:56:51.268 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:56:51.268 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:51 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 713/258132 objects degraded (0.276%), 1 pg degraded, 3 pgs undersized (PG_DEGRADED) 2024-09-20T18:56:51.269 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:51 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 713/258132 objects degraded (0.276%), 1 pg degraded, 3 pgs undersized (PG_DEGRADED) 2024-09-20T18:56:51.283 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:51.306 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:56:51.306 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:56:51.314 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T18:56:51.314 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:56:51.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:51.341 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:51.342 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:51.343 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:56:51.344 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:56:51.442 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:51.469 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:51.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:51.470 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:51.571 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:56:51.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:51.607 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:51.607 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:56:51.708 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:51.735 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:51.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:51.736 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:51.836 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:51.838 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":337,"stamp":"2024-09-20T18:56:48.908465+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5626468915,"num_objects":86044,"num_object_clones":0,"num_object_copies":258132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":713,"num_objects_misplaced":8437,"num_objects_unfound":0,"num_objects_dirty":86044,"num_whiteouts":0,"num_read":1261,"num_read_kb":1118,"num_write":146882,"num_write_kb":7431306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19082,"num_bytes_recovered":1245514697,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74722,"ondisk_log_size":74722,"up":444,"acting":441,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":17838988,"kb_used_data":16029212,"kb_used_omap":10,"kb_used_meta":1809141,"kb_avail":638377076,"statfs":{"total":671965249536,"available":653698125824,"internally_reserved":0,"allocated":16413913088,"data_stored":16409369155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11158,"internal_metadata":1852560490},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[8,0,1,7,11,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-581828608,"num_objects":-8878,"num_object_clones":0,"num_object_copies":-26634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":359,"num_objects_misplaced":-2763,"num_objects_unfound":0,"num_objects_dirty":-8878,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":8878,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2233,"num_bytes_recovered":146341888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.219317"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":339,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.699108+0000","last_change":"2024-09-20T18:55:28.037494+0000","last_active":"2024-09-20T18:56:40.699108+0000","last_peered":"2024-09-20T18:56:40.699108+0000","last_clean":"2024-09-20T18:56:40.699108+0000","last_became_active":"2024-09-20T18:49:45.943201+0000","last_became_peered":"2024-09-20T18:49:45.943201+0000","last_unstale":"2024-09-20T18:56:40.699108+0000","last_undegraded":"2024-09-20T18:56:40.699108+0000","last_fullsized":"2024-09-20T18:56:40.699108+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:28.037449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:28.037449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:53.606960+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":385,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.695659+0000","last_change":"2024-09-20T18:55:28.838729+0000","last_active":"2024-09-20T18:56:40.695659+0000","last_peered":"2024-09-20T18:56:40.695659+0000","last_clean":"2024-09-20T18:56:40.695659+0000","last_became_active":"2024-09-20T18:55:28.819637+0000","last_became_peered":"2024-09-20T18:55:28.819637+0000","last_unstale":"2024-09-20T18:56:40.695659+0000","last_undegraded":"2024-09-20T18:56:40.695659+0000","last_fullsized":"2024-09-20T18:56:40.695659+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:50.231620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:50.231620+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":331,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.691756+0000","last_change":"2024-09-20T18:54:25.361988+0000","last_active":"2024-09-20T18:56:40.691756+0000","last_peered":"2024-09-20T18:56:40.691756+0000","last_clean":"2024-09-20T18:56:40.691756+0000","last_became_active":"2024-09-20T18:49:49.959514+0000","last_became_peered":"2024-09-20T18:49:49.959514+0000","last_unstale":"2024-09-20T18:56:40.691756+0000","last_undegraded":"2024-09-20T18:56:40.691756+0000","last_fullsized":"2024-09-20T18:56:40.691756+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:25.361935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:25.361935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":327,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.695868+0000","last_change":"2024-09-20T18:54:15.221404+0000","last_active":"2024-09-20T18:56:40.695868+0000","last_peered":"2024-09-20T18:56:40.695868+0000","last_clean":"2024-09-20T18:56:40.695868+0000","last_became_active":"2024-09-20T18:49:51.969421+0000","last_became_peered":"2024-09-20T18:49:51.969421+0000","last_unstale":"2024-09-20T18:56:40.695868+0000","last_undegraded":"2024-09-20T18:56:40.695868+0000","last_fullsized":"2024-09-20T18:56:40.695868+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:15.221342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:15.221342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":334,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.697000+0000","last_change":"2024-09-20T18:55:28.822034+0000","last_active":"2024-09-20T18:56:40.697000+0000","last_peered":"2024-09-20T18:56:40.697000+0000","last_clean":"2024-09-20T18:56:40.697000+0000","last_became_active":"2024-09-20T18:55:28.821717+0000","last_became_peered":"2024-09-20T18:55:28.821717+0000","last_unstale":"2024-09-20T18:56:40.697000+0000","last_undegraded":"2024-09-20T18:56:40.697000+0000","last_fullsized":"2024-09-20T18:56:40.697000+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.216188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.216188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":331,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.697324+0000","last_change":"2024-09-20T18:55:28.825144+0000","last_active":"2024-09-20T18:56:40.697324+0000","last_peered":"2024-09-20T18:56:40.697324+0000","last_clean":"2024-09-20T18:56:40.697324+0000","last_became_active":"2024-09-20T18:55:28.824826+0000","last_became_peered":"2024-09-20T18:55:28.824826+0000","last_unstale":"2024-09-20T18:56:40.697324+0000","last_undegraded":"2024-09-20T18:56:40.697324+0000","last_fullsized":"2024-09-20T18:56:40.697324+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:25.208073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:25.208073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":344,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.697148+0000","last_change":"2024-09-20T18:51:07.130137+0000","last_active":"2024-09-20T18:56:40.697148+0000","last_peered":"2024-09-20T18:56:40.697148+0000","last_clean":"2024-09-20T18:56:40.697148+0000","last_became_active":"2024-09-20T18:49:47.952150+0000","last_became_peered":"2024-09-20T18:49:47.952150+0000","last_unstale":"2024-09-20T18:56:40.697148+0000","last_undegraded":"2024-09-20T18:56:40.697148+0000","last_fullsized":"2024-09-20T18:56:40.697148+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:51:07.130062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:07.130062+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":330,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.695564+0000","last_change":"2024-09-20T18:55:28.820753+0000","last_active":"2024-09-20T18:56:40.695564+0000","last_peered":"2024-09-20T18:56:40.695564+0000","last_clean":"2024-09-20T18:56:40.695564+0000","last_became_active":"2024-09-20T18:55:28.820362+0000","last_became_peered":"2024-09-20T18:55:28.820362+0000","last_unstale":"2024-09-20T18:56:40.695564+0000","last_undegraded":"2024-09-20T18:56:40.695564+0000","last_fullsized":"2024-09-20T18:56:40.695564+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:11.246922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:11.246922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"59'1","reported_seq":363,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.690906+0000","last_change":"2024-09-20T18:55:29.340140+0000","last_active":"2024-09-20T18:56:40.690906+0000","last_peered":"2024-09-20T18:56:40.690906+0000","last_clean":"2024-09-20T18:56:40.690906+0000","last_became_active":"2024-09-20T18:55:28.833386+0000","last_became_peered":"2024-09-20T18:55:28.833386+0000","last_unstale":"2024-09-20T18:56:40.690906+0000","last_undegraded":"2024-09-20T18:56:40.690906+0000","last_fullsized":"2024-09-20T18:56:40.690906+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:55:29.340083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:29.340083+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:41.552449+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":351,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.695952+0000","last_change":"2024-09-20T18:54:53.241889+0000","last_active":"2024-09-20T18:56:40.695952+0000","last_peered":"2024-09-20T18:56:40.695952+0000","last_clean":"2024-09-20T18:56:40.695952+0000","last_became_active":"2024-09-20T18:49:47.947448+0000","last_became_peered":"2024-09-20T18:49:47.947448+0000","last_unstale":"2024-09-20T18:56:40.695952+0000","last_undegraded":"2024-09-20T18:56:40.695952+0000","last_fullsized":"2024-09-20T18:56:40.695952+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:53.241802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:53.241802+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":322,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.689491+0000","last_change":"2024-09-20T18:52:36.183932+0000","last_active":"2024-09-20T18:56:40.689491+0000","last_peered":"2024-09-20T18:56:40.689491+0000","last_clean":"2024-09-20T18:56:40.689491+0000","last_became_active":"2024-09-20T18:49:49.961188+0000","last_became_peered":"2024-09-20T18:49:49.961188+0000","last_unstale":"2024-09-20T18:56:40.689491+0000","last_undegraded":"2024-09-20T18:56:40.689491+0000","last_fullsized":"2024-09-20T18:56:40.689491+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:36.183875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:36.183875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":302,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.696012+0000","last_change":"2024-09-20T18:49:51.966957+0000","last_active":"2024-09-20T18:56:40.696012+0000","last_peered":"2024-09-20T18:56:40.696012+0000","last_clean":"2024-09-20T18:56:40.696012+0000","last_became_active":"2024-09-20T18:49:51.966715+0000","last_became_peered":"2024-09-20T18:49:51.966715+0000","last_unstale":"2024-09-20T18:56:40.696012+0000","last_undegraded":"2024-09-20T18:56:40.696012+0000","last_fullsized":"2024-09-20T18:56:40.696012+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":330,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.690356+0000","last_change":"2024-09-20T18:55:15.252839+0000","last_active":"2024-09-20T18:56:40.690356+0000","last_peered":"2024-09-20T18:56:40.690356+0000","last_clean":"2024-09-20T18:56:40.690356+0000","last_became_active":"2024-09-20T18:49:45.944703+0000","last_became_peered":"2024-09-20T18:49:45.944703+0000","last_unstale":"2024-09-20T18:56:40.690356+0000","last_undegraded":"2024-09-20T18:56:40.690356+0000","last_fullsized":"2024-09-20T18:56:40.690356+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:15.252782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:15.252782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:43.639912+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":355,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.695028+0000","last_change":"2024-09-20T18:51:26.494194+0000","last_active":"2024-09-20T18:56:40.695028+0000","last_peered":"2024-09-20T18:56:40.695028+0000","last_clean":"2024-09-20T18:56:40.695028+0000","last_became_active":"2024-09-20T18:51:23.813399+0000","last_became_peered":"2024-09-20T18:51:23.813399+0000","last_unstale":"2024-09-20T18:56:40.695028+0000","last_undegraded":"2024-09-20T18:56:40.695028+0000","last_fullsized":"2024-09-20T18:56:40.695028+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-20T18:51:26.494118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:26.494118+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":322,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.692160+0000","last_change":"2024-09-20T18:52:38.799542+0000","last_active":"2024-09-20T18:56:40.692160+0000","last_peered":"2024-09-20T18:56:40.692160+0000","last_clean":"2024-09-20T18:56:40.692160+0000","last_became_active":"2024-09-20T18:49:49.959696+0000","last_became_peered":"2024-09-20T18:49:49.959696+0000","last_unstale":"2024-09-20T18:56:40.692160+0000","last_undegraded":"2024-09-20T18:56:40.692160+0000","last_fullsized":"2024-09-20T18:56:40.692160+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:38.799487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:38.799487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":318,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.692046+0000","last_change":"2024-09-20T18:53:21.273835+0000","last_active":"2024-09-20T18:56:40.692046+0000","last_peered":"2024-09-20T18:56:40.692046+0000","last_clean":"2024-09-20T18:56:40.692046+0000","last_became_active":"2024-09-20T18:49:51.970663+0000","last_became_peered":"2024-09-20T18:49:51.970663+0000","last_unstale":"2024-09-20T18:56:40.692046+0000","last_undegraded":"2024-09-20T18:56:40.692046+0000","last_fullsized":"2024-09-20T18:56:40.692046+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:21.273788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:21.273788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":327,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.696556+0000","last_change":"2024-09-20T18:55:28.828417+0000","last_active":"2024-09-20T18:56:40.696556+0000","last_peered":"2024-09-20T18:56:40.696556+0000","last_clean":"2024-09-20T18:56:40.696556+0000","last_became_active":"2024-09-20T18:55:28.827435+0000","last_became_peered":"2024-09-20T18:55:28.827435+0000","last_unstale":"2024-09-20T18:56:40.696556+0000","last_undegraded":"2024-09-20T18:56:40.696556+0000","last_fullsized":"2024-09-20T18:56:40.696556+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:53.940191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:53.940191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":330,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.698882+0000","last_change":"2024-09-20T18:54:23.984024+0000","last_active":"2024-09-20T18:56:40.698882+0000","last_peered":"2024-09-20T18:56:40.698882+0000","last_clean":"2024-09-20T18:56:40.698882+0000","last_became_active":"2024-09-20T18:49:45.950480+0000","last_became_peered":"2024-09-20T18:49:45.950480+0000","last_unstale":"2024-09-20T18:56:40.698882+0000","last_undegraded":"2024-09-20T18:56:40.698882+0000","last_fullsized":"2024-09-20T18:56:40.698882+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:23.983958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:23.983958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":353,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.699997+0000","last_change":"2024-09-20T18:53:26.762207+0000","last_active":"2024-09-20T18:56:40.699997+0000","last_peered":"2024-09-20T18:56:40.699997+0000","last_clean":"2024-09-20T18:56:40.699997+0000","last_became_active":"2024-09-20T18:49:47.950939+0000","last_became_peered":"2024-09-20T18:49:47.950939+0000","last_unstale":"2024-09-20T18:56:40.699997+0000","last_undegraded":"2024-09-20T18:56:40.699997+0000","last_fullsized":"2024-09-20T18:56:40.699997+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:53:26.762131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:26.762131+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":315,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.691700+0000","last_change":"2024-09-20T18:51:17.248705+0000","last_active":"2024-09-20T18:56:40.691700+0000","last_peered":"2024-09-20T18:56:40.691700+0000","last_clean":"2024-09-20T18:56:40.691700+0000","last_became_active":"2024-09-20T18:49:49.958518+0000","last_became_peered":"2024-09-20T18:49:49.958518+0000","last_unstale":"2024-09-20T18:56:40.691700+0000","last_undegraded":"2024-09-20T18:56:40.691700+0000","last_fullsized":"2024-09-20T18:56:40.691700+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:17.248634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:17.248634+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":327,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.696659+0000","last_change":"2024-09-20T18:55:28.833468+0000","last_active":"2024-09-20T18:56:40.696659+0000","last_peered":"2024-09-20T18:56:40.696659+0000","last_clean":"2024-09-20T18:56:40.696659+0000","last_became_active":"2024-09-20T18:55:28.833133+0000","last_became_peered":"2024-09-20T18:55:28.833133+0000","last_unstale":"2024-09-20T18:56:40.696659+0000","last_undegraded":"2024-09-20T18:56:40.696659+0000","last_fullsized":"2024-09-20T18:56:40.696659+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:54.259174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:54.259174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"59'1","reported_seq":337,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.695998+0000","last_change":"2024-09-20T18:53:42.840884+0000","last_active":"2024-09-20T18:56:40.695998+0000","last_peered":"2024-09-20T18:56:40.695998+0000","last_clean":"2024-09-20T18:56:40.695998+0000","last_became_active":"2024-09-20T18:49:45.940100+0000","last_became_peered":"2024-09-20T18:49:45.940100+0000","last_unstale":"2024-09-20T18:56:40.695998+0000","last_undegraded":"2024-09-20T18:56:40.695998+0000","last_fullsized":"2024-09-20T18:56:40.695998+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:53:42.840800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:42.840800+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":334,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.696606+0000","last_change":"2024-09-20T18:49:47.951215+0000","last_active":"2024-09-20T18:56:40.696606+0000","last_peered":"2024-09-20T18:56:40.696606+0000","last_clean":"2024-09-20T18:56:40.696606+0000","last_became_active":"2024-09-20T18:49:47.950909+0000","last_became_peered":"2024-09-20T18:49:47.950909+0000","last_unstale":"2024-09-20T18:56:40.696606+0000","last_undegraded":"2024-09-20T18:56:40.696606+0000","last_fullsized":"2024-09-20T18:56:40.696606+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":331,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.695179+0000","last_change":"2024-09-20T18:54:44.239273+0000","last_active":"2024-09-20T18:56:40.695179+0000","last_peered":"2024-09-20T18:56:40.695179+0000","last_clean":"2024-09-20T18:56:40.695179+0000","last_became_active":"2024-09-20T18:49:49.959102+0000","last_became_peered":"2024-09-20T18:49:49.959102+0000","last_unstale":"2024-09-20T18:56:40.695179+0000","last_undegraded":"2024-09-20T18:56:40.695179+0000","last_fullsized":"2024-09-20T18:56:40.695179+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:44.239226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.239226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":319,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.692562+0000","last_change":"2024-09-20T18:52:15.826819+0000","last_active":"2024-09-20T18:56:40.692562+0000","last_peered":"2024-09-20T18:56:40.692562+0000","last_clean":"2024-09-20T18:56:40.692562+0000","last_became_active":"2024-09-20T18:49:51.970058+0000","last_became_peered":"2024-09-20T18:49:51.970058+0000","last_unstale":"2024-09-20T18:56:40.692562+0000","last_undegraded":"2024-09-20T18:56:40.692562+0000","last_fullsized":"2024-09-20T18:56:40.692562+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T18:52:15.826638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:15.826638+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":324,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.692368+0000","last_change":"2024-09-20T18:52:47.158384+0000","last_active":"2024-09-20T18:56:40.692368+0000","last_peered":"2024-09-20T18:56:40.692368+0000","last_clean":"2024-09-20T18:56:40.692368+0000","last_became_active":"2024-09-20T18:49:45.933078+0000","last_became_peered":"2024-09-20T18:49:45.933078+0000","last_unstale":"2024-09-20T18:56:40.692368+0000","last_undegraded":"2024-09-20T18:56:40.692368+0000","last_fullsized":"2024-09-20T18:56:40.692368+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:52:47.158260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:47.158260+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":350,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.702377+0000","last_change":"2024-09-20T18:55:19.996522+0000","last_active":"2024-09-20T18:56:40.702377+0000","last_peered":"2024-09-20T18:56:40.702377+0000","last_clean":"2024-09-20T18:56:40.702377+0000","last_became_active":"2024-09-20T18:49:48.500809+0000","last_became_peered":"2024-09-20T18:49:48.500809+0000","last_unstale":"2024-09-20T18:56:40.702377+0000","last_undegraded":"2024-09-20T18:56:40.702377+0000","last_fullsized":"2024-09-20T18:56:40.702377+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:55:19.996328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:19.996328+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"184'9","reported_seq":117,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.696406+0000","last_change":"2024-09-20T18:55:28.845188+0000","last_active":"2024-09-20T18:56:40.696406+0000","last_peered":"2024-09-20T18:56:40.696406+0000","last_clean":"2024-09-20T18:56:40.696406+0000","last_became_active":"2024-09-20T18:55:28.820941+0000","last_became_peered":"2024-09-20T18:55:28.820941+0000","last_unstale":"2024-09-20T18:56:40.696406+0000","last_undegraded":"2024-09-20T18:56:40.696406+0000","last_fullsized":"2024-09-20T18:56:40.696406+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:53:44.524040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:44.524040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3","version":"210'8912","reported_seq":21356,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:46.214345+0000","last_change":"2024-09-20T18:56:25.697520+0000","last_active":"2024-09-20T18:56:46.214345+0000","last_peered":"2024-09-20T18:56:46.214345+0000","last_clean":"2024-09-20T18:56:46.214345+0000","last_became_active":"2024-09-20T18:56:25.687391+0000","last_became_peered":"2024-09-20T18:56:25.687391+0000","last_unstale":"2024-09-20T18:56:46.214345+0000","last_undegraded":"2024-09-20T18:56:46.214345+0000","last_fullsized":"2024-09-20T18:56:46.214345+0000","mapping_epoch":203,"log_start":"176'4672","ondisk_log_start":"176'4672","created":160,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4240,"log_dups_size":0,"ondisk_log_size":4240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":352845824,"num_objects":5384,"num_object_clones":0,"num_object_copies":16152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5384,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8912,"num_write_kb":457472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6156,"num_bytes_recovered":403439616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":345,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.699573+0000","last_change":"2024-09-20T18:55:18.581290+0000","last_active":"2024-09-20T18:56:40.699573+0000","last_peered":"2024-09-20T18:56:40.699573+0000","last_clean":"2024-09-20T18:56:40.699573+0000","last_became_active":"2024-09-20T18:49:58.992302+0000","last_became_peered":"2024-09-20T18:49:58.992302+0000","last_unstale":"2024-09-20T18:56:40.699573+0000","last_undegraded":"2024-09-20T18:56:40.699573+0000","last_fullsized":"2024-09-20T18:56:40.699573+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:55:18.581200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:18.581200+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":332,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.697874+0000","last_change":"2024-09-20T18:54:54.288199+0000","last_active":"2024-09-20T18:56:40.697874+0000","last_peered":"2024-09-20T18:56:40.697874+0000","last_clean":"2024-09-20T18:56:40.697874+0000","last_became_active":"2024-09-20T18:49:45.936104+0000","last_became_peered":"2024-09-20T18:49:45.936104+0000","last_unstale":"2024-09-20T18:56:40.697874+0000","last_undegraded":"2024-09-20T18:56:40.697874+0000","last_fullsized":"2024-09-20T18:56:40.697874+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:54.288117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:54.288117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":46,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":419,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.696657+0000","last_change":"2024-09-20T18:54:14.292660+0000","last_active":"2024-09-20T18:56:40.696657+0000","last_peered":"2024-09-20T18:56:40.696657+0000","last_clean":"2024-09-20T18:56:40.696657+0000","last_became_active":"2024-09-20T18:49:49.960697+0000","last_became_peered":"2024-09-20T18:49:49.960697+0000","last_unstale":"2024-09-20T18:56:40.696657+0000","last_undegraded":"2024-09-20T18:56:40.696657+0000","last_fullsized":"2024-09-20T18:56:40.696657+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:54:14.292610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:14.292610+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":34,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":319,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.689782+0000","last_change":"2024-09-20T18:54:10.381171+0000","last_active":"2024-09-20T18:56:40.689782+0000","last_peered":"2024-09-20T18:56:40.689782+0000","last_clean":"2024-09-20T18:56:40.689782+0000","last_became_active":"2024-09-20T18:49:51.971399+0000","last_became_peered":"2024-09-20T18:49:51.971399+0000","last_unstale":"2024-09-20T18:56:40.689782+0000","last_undegraded":"2024-09-20T18:56:40.689782+0000","last_fullsized":"2024-09-20T18:56:40.689782+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:10.381108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:10.381108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.c","version":"210'9072","reported_seq":9182,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:46.644999+0000","last_change":"2024-09-20T18:54:34.311256+0000","last_active":"2024-09-20T18:56:46.644999+0000","last_peered":"2024-09-20T18:56:46.644999+0000","last_clean":"2024-09-20T18:56:46.644999+0000","last_became_active":"2024-09-20T18:54:34.310904+0000","last_became_peered":"2024-09-20T18:54:34.310904+0000","last_unstale":"2024-09-20T18:56:46.644999+0000","last_undegraded":"2024-09-20T18:56:46.644999+0000","last_fullsized":"2024-09-20T18:56:46.644999+0000","mapping_epoch":160,"log_start":"175'4211","ondisk_log_start":"175'4211","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4861,"log_dups_size":0,"ondisk_log_size":4861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":348651520,"num_objects":5320,"num_object_clones":0,"num_object_copies":15960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9072,"num_write_kb":460544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":360,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.696634+0000","last_change":"2024-09-20T18:54:22.077299+0000","last_active":"2024-09-20T18:56:40.696634+0000","last_peered":"2024-09-20T18:56:40.696634+0000","last_clean":"2024-09-20T18:56:40.696634+0000","last_became_active":"2024-09-20T18:49:48.499189+0000","last_became_peered":"2024-09-20T18:49:48.499189+0000","last_unstale":"2024-09-20T18:56:40.696634+0000","last_undegraded":"2024-09-20T18:56:40.696634+0000","last_fullsized":"2024-09-20T18:56:40.696634+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-20T18:54:22.077212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:22.077212+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":324,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.698452+0000","last_change":"2024-09-20T18:54:57.237740+0000","last_active":"2024-09-20T18:56:40.698452+0000","last_peered":"2024-09-20T18:56:40.698452+0000","last_clean":"2024-09-20T18:56:40.698452+0000","last_became_active":"2024-09-20T18:49:45.940544+0000","last_became_peered":"2024-09-20T18:49:45.940544+0000","last_unstale":"2024-09-20T18:56:40.698452+0000","last_undegraded":"2024-09-20T18:56:40.698452+0000","last_fullsized":"2024-09-20T18:56:40.698452+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.237675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.237675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":332,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.694625+0000","last_change":"2024-09-20T18:54:51.229251+0000","last_active":"2024-09-20T18:56:40.694625+0000","last_peered":"2024-09-20T18:56:40.694625+0000","last_clean":"2024-09-20T18:56:40.694625+0000","last_became_active":"2024-09-20T18:49:49.963073+0000","last_became_peered":"2024-09-20T18:49:49.963073+0000","last_unstale":"2024-09-20T18:56:40.694625+0000","last_undegraded":"2024-09-20T18:56:40.694625+0000","last_fullsized":"2024-09-20T18:56:40.694625+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.229201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.229201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":328,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.695921+0000","last_change":"2024-09-20T18:54:55.323937+0000","last_active":"2024-09-20T18:56:40.695921+0000","last_peered":"2024-09-20T18:56:40.695921+0000","last_clean":"2024-09-20T18:56:40.695921+0000","last_became_active":"2024-09-20T18:49:51.968793+0000","last_became_peered":"2024-09-20T18:49:51.968793+0000","last_unstale":"2024-09-20T18:56:40.695921+0000","last_undegraded":"2024-09-20T18:56:40.695921+0000","last_fullsized":"2024-09-20T18:56:40.695921+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:55.323692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:55.323692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"210'9043","reported_seq":9155,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:45.867172+0000","last_change":"2024-09-20T18:54:34.311680+0000","last_active":"2024-09-20T18:56:45.867172+0000","last_peered":"2024-09-20T18:56:45.867172+0000","last_clean":"2024-09-20T18:56:45.867172+0000","last_became_active":"2024-09-20T18:54:34.311241+0000","last_became_peered":"2024-09-20T18:54:34.311241+0000","last_unstale":"2024-09-20T18:56:45.867172+0000","last_undegraded":"2024-09-20T18:56:45.867172+0000","last_fullsized":"2024-09-20T18:56:45.867172+0000","mapping_epoch":160,"log_start":"174'3946","ondisk_log_start":"174'3946","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":5097,"log_dups_size":0,"ondisk_log_size":5097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":349175832,"num_objects":5329,"num_object_clones":0,"num_object_copies":15987,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5329,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9043,"num_write_kb":459841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":388,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.694705+0000","last_change":"2024-09-20T18:55:28.853375+0000","last_active":"2024-09-20T18:56:40.694705+0000","last_peered":"2024-09-20T18:56:40.694705+0000","last_clean":"2024-09-20T18:56:40.694705+0000","last_became_active":"2024-09-20T18:55:28.831057+0000","last_became_peered":"2024-09-20T18:55:28.831057+0000","last_unstale":"2024-09-20T18:56:40.694705+0000","last_undegraded":"2024-09-20T18:56:40.694705+0000","last_fullsized":"2024-09-20T18:56:40.694705+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-20T18:54:57.394893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.394893+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":324,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.702610+0000","last_change":"2024-09-20T18:51:27.187060+0000","last_active":"2024-09-20T18:56:40.702610+0000","last_peered":"2024-09-20T18:56:40.702610+0000","last_clean":"2024-09-20T18:56:40.702610+0000","last_became_active":"2024-09-20T18:49:45.940469+0000","last_became_peered":"2024-09-20T18:49:45.940469+0000","last_unstale":"2024-09-20T18:56:40.702610+0000","last_undegraded":"2024-09-20T18:56:40.702610+0000","last_fullsized":"2024-09-20T18:56:40.702610+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:51:27.186991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:27.186991+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"66'8","reported_seq":425,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:41.079385+0000","last_change":"2024-09-20T18:54:47.223010+0000","last_active":"2024-09-20T18:56:41.079385+0000","last_peered":"2024-09-20T18:56:41.079385+0000","last_clean":"2024-09-20T18:56:41.079385+0000","last_became_active":"2024-09-20T18:49:49.956133+0000","last_became_peered":"2024-09-20T18:49:49.956133+0000","last_unstale":"2024-09-20T18:56:41.079385+0000","last_undegraded":"2024-09-20T18:56:41.079385+0000","last_fullsized":"2024-09-20T18:56:41.079385+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:54:47.222937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:47.222937+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":326,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.691582+0000","last_change":"2024-09-20T18:55:28.819147+0000","last_active":"2024-09-20T18:56:40.691582+0000","last_peered":"2024-09-20T18:56:40.691582+0000","last_clean":"2024-09-20T18:56:40.691582+0000","last_became_active":"2024-09-20T18:55:28.818804+0000","last_became_peered":"2024-09-20T18:55:28.818804+0000","last_unstale":"2024-09-20T18:56:40.691582+0000","last_undegraded":"2024-09-20T18:56:40.691582+0000","last_fullsized":"2024-09-20T18:56:40.691582+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:42.880724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:42.880724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"210'9096","reported_seq":9207,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:46.606251+0000","last_change":"2024-09-20T18:54:34.309939+0000","last_active":"2024-09-20T18:56:46.606251+0000","last_peered":"2024-09-20T18:56:46.606251+0000","last_clean":"2024-09-20T18:56:46.606251+0000","last_became_active":"2024-09-20T18:54:34.309630+0000","last_became_peered":"2024-09-20T18:54:34.309630+0000","last_unstale":"2024-09-20T18:56:46.606251+0000","last_undegraded":"2024-09-20T18:56:46.606251+0000","last_fullsized":"2024-09-20T18:56:46.606251+0000","mapping_epoch":160,"log_start":"175'4249","ondisk_log_start":"175'4249","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4847,"log_dups_size":0,"ondisk_log_size":4847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":347734016,"num_objects":5306,"num_object_clones":0,"num_object_copies":15918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9096,"num_write_kb":460864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":351,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.691236+0000","last_change":"2024-09-20T18:55:19.541011+0000","last_active":"2024-09-20T18:56:40.691236+0000","last_peered":"2024-09-20T18:56:40.691236+0000","last_clean":"2024-09-20T18:56:40.691236+0000","last_became_active":"2024-09-20T18:49:47.949626+0000","last_became_peered":"2024-09-20T18:49:47.949626+0000","last_unstale":"2024-09-20T18:56:40.691236+0000","last_undegraded":"2024-09-20T18:56:40.691236+0000","last_fullsized":"2024-09-20T18:56:40.691236+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:55:19.540812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:19.540812+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":339,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.691062+0000","last_change":"2024-09-20T18:54:18.326947+0000","last_active":"2024-09-20T18:56:40.691062+0000","last_peered":"2024-09-20T18:56:40.691062+0000","last_clean":"2024-09-20T18:56:40.691062+0000","last_became_active":"2024-09-20T18:49:45.942496+0000","last_became_peered":"2024-09-20T18:49:45.942496+0000","last_unstale":"2024-09-20T18:56:40.691062+0000","last_undegraded":"2024-09-20T18:56:40.691062+0000","last_fullsized":"2024-09-20T18:56:40.691062+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:18.326875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:18.326875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"184'9","reported_seq":427,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:41.076992+0000","last_change":"2024-09-20T18:55:29.010062+0000","last_active":"2024-09-20T18:56:41.076992+0000","last_peered":"2024-09-20T18:56:41.076992+0000","last_clean":"2024-09-20T18:56:41.076992+0000","last_became_active":"2024-09-20T18:55:28.830643+0000","last_became_peered":"2024-09-20T18:55:28.830643+0000","last_unstale":"2024-09-20T18:56:41.076992+0000","last_undegraded":"2024-09-20T18:56:41.076992+0000","last_fullsized":"2024-09-20T18:56:41.076992+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:29.010015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:29.010015+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":303,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.696137+0000","last_change":"2024-09-20T18:49:51.965733+0000","last_active":"2024-09-20T18:56:40.696137+0000","last_peered":"2024-09-20T18:56:40.696137+0000","last_clean":"2024-09-20T18:56:40.696137+0000","last_became_active":"2024-09-20T18:49:51.964780+0000","last_became_peered":"2024-09-20T18:49:51.964780+0000","last_unstale":"2024-09-20T18:56:40.696137+0000","last_undegraded":"2024-09-20T18:56:40.696137+0000","last_fullsized":"2024-09-20T18:56:40.696137+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"210'9316","reported_seq":9427,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:46.443905+0000","last_change":"2024-09-20T18:54:34.309809+0000","last_active":"2024-09-20T18:56:46.443905+0000","last_peered":"2024-09-20T18:56:46.443905+0000","last_clean":"2024-09-20T18:56:46.443905+0000","last_became_active":"2024-09-20T18:54:34.309231+0000","last_became_peered":"2024-09-20T18:54:34.309231+0000","last_unstale":"2024-09-20T18:56:46.443905+0000","last_undegraded":"2024-09-20T18:56:46.443905+0000","last_fullsized":"2024-09-20T18:56:46.443905+0000","mapping_epoch":160,"log_start":"177'4932","ondisk_log_start":"177'4932","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4384,"log_dups_size":0,"ondisk_log_size":4384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":353763328,"num_objects":5398,"num_object_clones":0,"num_object_copies":16194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5398,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9316,"num_write_kb":470848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":363,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.698633+0000","last_change":"2024-09-20T18:54:14.923895+0000","last_active":"2024-09-20T18:56:40.698633+0000","last_peered":"2024-09-20T18:56:40.698633+0000","last_clean":"2024-09-20T18:56:40.698633+0000","last_became_active":"2024-09-20T18:49:48.497736+0000","last_became_peered":"2024-09-20T18:49:48.497736+0000","last_unstale":"2024-09-20T18:56:40.698633+0000","last_undegraded":"2024-09-20T18:56:40.698633+0000","last_fullsized":"2024-09-20T18:56:40.698633+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-20T18:54:14.923682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:14.923682+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":316,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.691334+0000","last_change":"2024-09-20T18:49:45.942589+0000","last_active":"2024-09-20T18:56:40.691334+0000","last_peered":"2024-09-20T18:56:40.691334+0000","last_clean":"2024-09-20T18:56:40.691334+0000","last_became_active":"2024-09-20T18:49:45.942253+0000","last_became_peered":"2024-09-20T18:49:45.942253+0000","last_unstale":"2024-09-20T18:56:40.691334+0000","last_undegraded":"2024-09-20T18:56:40.691334+0000","last_fullsized":"2024-09-20T18:56:40.691334+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":69,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.697098+0000","last_change":"2024-09-20T18:55:28.830954+0000","last_active":"2024-09-20T18:56:40.697098+0000","last_peered":"2024-09-20T18:56:40.697098+0000","last_clean":"2024-09-20T18:56:40.697098+0000","last_became_active":"2024-09-20T18:55:28.830645+0000","last_became_peered":"2024-09-20T18:55:28.830645+0000","last_unstale":"2024-09-20T18:56:40.697098+0000","last_undegraded":"2024-09-20T18:56:40.697098+0000","last_fullsized":"2024-09-20T18:56:40.697098+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:50.610871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:50.610871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":335,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.691679+0000","last_change":"2024-09-20T18:55:28.822362+0000","last_active":"2024-09-20T18:56:40.691679+0000","last_peered":"2024-09-20T18:56:40.691679+0000","last_clean":"2024-09-20T18:56:40.691679+0000","last_became_active":"2024-09-20T18:55:28.821588+0000","last_became_peered":"2024-09-20T18:55:28.821588+0000","last_unstale":"2024-09-20T18:56:40.691679+0000","last_undegraded":"2024-09-20T18:56:40.691679+0000","last_fullsized":"2024-09-20T18:56:40.691679+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.402841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.402841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"210'9160","reported_seq":9270,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:45.922665+0000","last_change":"2024-09-20T18:54:34.311561+0000","last_active":"2024-09-20T18:56:45.922665+0000","last_peered":"2024-09-20T18:56:45.922665+0000","last_clean":"2024-09-20T18:56:45.922665+0000","last_became_active":"2024-09-20T18:54:34.311181+0000","last_became_peered":"2024-09-20T18:54:34.311181+0000","last_unstale":"2024-09-20T18:56:45.922665+0000","last_undegraded":"2024-09-20T18:56:45.922665+0000","last_fullsized":"2024-09-20T18:56:45.922665+0000","mapping_epoch":160,"log_start":"174'4094","ondisk_log_start":"174'4094","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":5066,"log_dups_size":0,"ondisk_log_size":5066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":356384768,"num_objects":5438,"num_object_clones":0,"num_object_copies":16314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5438,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9160,"num_write_kb":467136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"66'28","reported_seq":461,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:41.076339+0000","last_change":"2024-09-20T18:55:16.527267+0000","last_active":"2024-09-20T18:56:41.076339+0000","last_peered":"2024-09-20T18:56:41.076339+0000","last_clean":"2024-09-20T18:56:41.076339+0000","last_became_active":"2024-09-20T18:49:47.955335+0000","last_became_peered":"2024-09-20T18:49:47.955335+0000","last_unstale":"2024-09-20T18:56:41.076339+0000","last_undegraded":"2024-09-20T18:56:41.076339+0000","last_fullsized":"2024-09-20T18:56:41.076339+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'28","last_scrub_stamp":"2024-09-20T18:55:16.527165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:16.527165+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":323,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.695880+0000","last_change":"2024-09-20T18:55:28.832303+0000","last_active":"2024-09-20T18:56:40.695880+0000","last_peered":"2024-09-20T18:56:40.695880+0000","last_clean":"2024-09-20T18:56:40.695880+0000","last_became_active":"2024-09-20T18:55:28.831574+0000","last_became_peered":"2024-09-20T18:55:28.831574+0000","last_unstale":"2024-09-20T18:56:40.695880+0000","last_undegraded":"2024-09-20T18:56:40.695880+0000","last_fullsized":"2024-09-20T18:56:40.695880+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1],"acting":[4,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":368,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.690365+0000","last_change":"2024-09-20T18:55:28.854615+0000","last_active":"2024-09-20T18:56:40.690365+0000","last_peered":"2024-09-20T18:56:40.690365+0000","last_clean":"2024-09-20T18:56:40.690365+0000","last_became_active":"2024-09-20T18:55:28.837226+0000","last_became_peered":"2024-09-20T18:55:28.837226+0000","last_unstale":"2024-09-20T18:56:40.690365+0000","last_undegraded":"2024-09-20T18:56:40.690365+0000","last_fullsized":"2024-09-20T18:56:40.690365+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-20T18:54:51.227531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.227531+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0012781159999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":331,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.697899+0000","last_change":"2024-09-20T18:55:13.949348+0000","last_active":"2024-09-20T18:56:40.697899+0000","last_peered":"2024-09-20T18:56:40.697899+0000","last_clean":"2024-09-20T18:56:40.697899+0000","last_became_active":"2024-09-20T18:49:49.953971+0000","last_became_peered":"2024-09-20T18:49:49.953971+0000","last_unstale":"2024-09-20T18:56:40.697899+0000","last_undegraded":"2024-09-20T18:56:40.697899+0000","last_fullsized":"2024-09-20T18:56:40.697899+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:13.949269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:13.949269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":304,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.696926+0000","last_change":"2024-09-20T18:49:51.969702+0000","last_active":"2024-09-20T18:56:40.696926+0000","last_peered":"2024-09-20T18:56:40.696926+0000","last_clean":"2024-09-20T18:56:40.696926+0000","last_became_active":"2024-09-20T18:49:51.969400+0000","last_became_peered":"2024-09-20T18:49:51.969400+0000","last_unstale":"2024-09-20T18:56:40.696926+0000","last_undegraded":"2024-09-20T18:56:40.696926+0000","last_fullsized":"2024-09-20T18:56:40.696926+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"210'9279","reported_seq":9390,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:46.441093+0000","last_change":"2024-09-20T18:54:33.926785+0000","last_active":"2024-09-20T18:56:46.441093+0000","last_peered":"2024-09-20T18:56:46.441093+0000","last_clean":"2024-09-20T18:56:46.441093+0000","last_became_active":"2024-09-20T18:54:33.925750+0000","last_became_peered":"2024-09-20T18:54:33.925750+0000","last_unstale":"2024-09-20T18:56:46.441093+0000","last_undegraded":"2024-09-20T18:56:46.441093+0000","last_fullsized":"2024-09-20T18:56:46.441093+0000","mapping_epoch":160,"log_start":"178'4940","ondisk_log_start":"178'4940","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4339,"log_dups_size":0,"ondisk_log_size":4339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":356450304,"num_objects":5439,"num_object_clones":0,"num_object_copies":16317,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5439,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9279,"num_write_kb":470976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":363,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.706581+0000","last_change":"2024-09-20T18:55:25.702762+0000","last_active":"2024-09-20T18:56:40.706581+0000","last_peered":"2024-09-20T18:56:40.706581+0000","last_clean":"2024-09-20T18:56:40.706581+0000","last_became_active":"2024-09-20T18:49:47.950904+0000","last_became_peered":"2024-09-20T18:49:47.950904+0000","last_unstale":"2024-09-20T18:56:40.706581+0000","last_undegraded":"2024-09-20T18:56:40.706581+0000","last_fullsized":"2024-09-20T18:56:40.706581+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-20T18:55:25.702596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:25.702596+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":347,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.693084+0000","last_change":"2024-09-20T18:55:28.830832+0000","last_active":"2024-09-20T18:56:40.693084+0000","last_peered":"2024-09-20T18:56:40.693084+0000","last_clean":"2024-09-20T18:56:40.693084+0000","last_became_active":"2024-09-20T18:55:28.830269+0000","last_became_peered":"2024-09-20T18:55:28.830269+0000","last_unstale":"2024-09-20T18:56:40.693084+0000","last_undegraded":"2024-09-20T18:56:40.693084+0000","last_fullsized":"2024-09-20T18:56:40.693084+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:23.359769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:23.359769+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":334,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.693454+0000","last_change":"2024-09-20T18:52:07.718766+0000","last_active":"2024-09-20T18:56:40.693454+0000","last_peered":"2024-09-20T18:56:40.693454+0000","last_clean":"2024-09-20T18:56:40.693454+0000","last_became_active":"2024-09-20T18:49:44.937135+0000","last_became_peered":"2024-09-20T18:49:44.937135+0000","last_unstale":"2024-09-20T18:56:40.693454+0000","last_undegraded":"2024-09-20T18:56:40.693454+0000","last_fullsized":"2024-09-20T18:56:40.693454+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:52:07.718646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:52:07.718646+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.00095558800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":317,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.700199+0000","last_change":"2024-09-20T18:51:59.796450+0000","last_active":"2024-09-20T18:56:40.700199+0000","last_peered":"2024-09-20T18:56:40.700199+0000","last_clean":"2024-09-20T18:56:40.700199+0000","last_became_active":"2024-09-20T18:49:49.954424+0000","last_became_peered":"2024-09-20T18:49:49.954424+0000","last_unstale":"2024-09-20T18:56:40.700199+0000","last_undegraded":"2024-09-20T18:56:40.700199+0000","last_fullsized":"2024-09-20T18:56:40.700199+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:59.796383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:59.796383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":318,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.692755+0000","last_change":"2024-09-20T18:55:28.826659+0000","last_active":"2024-09-20T18:56:40.692755+0000","last_peered":"2024-09-20T18:56:40.692755+0000","last_clean":"2024-09-20T18:56:40.692755+0000","last_became_active":"2024-09-20T18:55:28.825812+0000","last_became_peered":"2024-09-20T18:55:28.825812+0000","last_unstale":"2024-09-20T18:56:40.692755+0000","last_undegraded":"2024-09-20T18:56:40.692755+0000","last_fullsized":"2024-09-20T18:56:40.692755+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:39.814259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:39.814259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"210'9267","reported_seq":9378,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:46.226030+0000","last_change":"2024-09-20T18:54:34.311774+0000","last_active":"2024-09-20T18:56:46.226030+0000","last_peered":"2024-09-20T18:56:46.226030+0000","last_clean":"2024-09-20T18:56:46.226030+0000","last_became_active":"2024-09-20T18:54:34.311487+0000","last_became_peered":"2024-09-20T18:54:34.311487+0000","last_unstale":"2024-09-20T18:56:46.226030+0000","last_undegraded":"2024-09-20T18:56:46.226030+0000","last_fullsized":"2024-09-20T18:56:46.226030+0000","mapping_epoch":160,"log_start":"177'4992","ondisk_log_start":"177'4992","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4275,"log_dups_size":0,"ondisk_log_size":4275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":350683136,"num_objects":5351,"num_object_clones":0,"num_object_copies":16053,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5351,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9267,"num_write_kb":467776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":361,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.702456+0000","last_change":"2024-09-20T18:55:14.951950+0000","last_active":"2024-09-20T18:56:40.702456+0000","last_peered":"2024-09-20T18:56:40.702456+0000","last_clean":"2024-09-20T18:56:40.702456+0000","last_became_active":"2024-09-20T18:49:47.949230+0000","last_became_peered":"2024-09-20T18:49:47.949230+0000","last_unstale":"2024-09-20T18:56:40.702456+0000","last_undegraded":"2024-09-20T18:56:40.702456+0000","last_fullsized":"2024-09-20T18:56:40.702456+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:55:14.951892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:14.951892+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":339,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.698340+0000","last_change":"2024-09-20T18:54:27.905856+0000","last_active":"2024-09-20T18:56:40.698340+0000","last_peered":"2024-09-20T18:56:40.698340+0000","last_clean":"2024-09-20T18:56:40.698340+0000","last_became_active":"2024-09-20T18:49:45.939197+0000","last_became_peered":"2024-09-20T18:49:45.939197+0000","last_unstale":"2024-09-20T18:56:40.698340+0000","last_undegraded":"2024-09-20T18:56:40.698340+0000","last_fullsized":"2024-09-20T18:56:40.698340+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:27.905803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:27.905803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":765,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:44.800477+0000","last_change":"2024-09-20T18:54:26.359887+0000","last_active":"2024-09-20T18:56:44.800477+0000","last_peered":"2024-09-20T18:56:44.800477+0000","last_clean":"2024-09-20T18:56:44.800477+0000","last_became_active":"2024-09-20T18:49:44.936819+0000","last_became_peered":"2024-09-20T18:49:44.936819+0000","last_unstale":"2024-09-20T18:56:44.800477+0000","last_undegraded":"2024-09-20T18:56:44.800477+0000","last_fullsized":"2024-09-20T18:56:44.800477+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-20T18:54:26.359807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:54:26.359807+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0010996199999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":415,"num_read_kb":410,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":323,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.690621+0000","last_change":"2024-09-20T18:54:59.151228+0000","last_active":"2024-09-20T18:56:40.690621+0000","last_peered":"2024-09-20T18:56:40.690621+0000","last_clean":"2024-09-20T18:56:40.690621+0000","last_became_active":"2024-09-20T18:49:49.961066+0000","last_became_peered":"2024-09-20T18:49:49.961066+0000","last_unstale":"2024-09-20T18:56:40.690621+0000","last_undegraded":"2024-09-20T18:56:40.690621+0000","last_fullsized":"2024-09-20T18:56:40.690621+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:59.150063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:59.150063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":328,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.691008+0000","last_change":"2024-09-20T18:55:21.629867+0000","last_active":"2024-09-20T18:56:40.691008+0000","last_peered":"2024-09-20T18:56:40.691008+0000","last_clean":"2024-09-20T18:56:40.691008+0000","last_became_active":"2024-09-20T18:49:51.970343+0000","last_became_peered":"2024-09-20T18:49:51.970343+0000","last_unstale":"2024-09-20T18:56:40.691008+0000","last_undegraded":"2024-09-20T18:56:40.691008+0000","last_fullsized":"2024-09-20T18:56:40.691008+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:21.629800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:21.629800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:40.736438+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"210'9223","reported_seq":16408,"reported_epoch":210,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-20T18:56:45.914958+0000","last_change":"2024-09-20T18:55:30.088909+0000","last_active":"2024-09-20T18:56:45.914958+0000","last_peered":"2024-09-20T18:56:45.914958+0000","last_clean":"2024-09-20T18:55:27.745898+0000","last_became_active":"2024-09-20T18:55:29.995624+0000","last_became_peered":"2024-09-20T18:55:29.995624+0000","last_unstale":"2024-09-20T18:56:45.914958+0000","last_undegraded":"2024-09-20T18:56:45.914958+0000","last_fullsized":"2024-09-20T18:55:29.815609+0000","mapping_epoch":182,"log_start":"174'4206","ondisk_log_start":"174'4206","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":5017,"log_dups_size":0,"ondisk_log_size":5017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":347930624,"num_objects":5309,"num_object_clones":0,"num_object_copies":15927,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1671,"num_objects_unfound":0,"num_objects_dirty":5309,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9223,"num_write_kb":465024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":231669760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0],"avail_no_missing":["5","0","6"],"object_location_counts":[{"shards":"0,5","objects":3638},{"shards":"0,5,6","objects":1671}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"184'17","reported_seq":146,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.699360+0000","last_change":"2024-09-20T18:55:28.856602+0000","last_active":"2024-09-20T18:56:40.699360+0000","last_peered":"2024-09-20T18:56:40.699360+0000","last_clean":"2024-09-20T18:56:40.699360+0000","last_became_active":"2024-09-20T18:55:28.831024+0000","last_became_peered":"2024-09-20T18:55:28.831024+0000","last_unstale":"2024-09-20T18:56:40.699360+0000","last_undegraded":"2024-09-20T18:56:40.699360+0000","last_fullsized":"2024-09-20T18:56:40.699360+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":24,"num_read_kb":14,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":339,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.696403+0000","last_change":"2024-09-20T18:55:28.830447+0000","last_active":"2024-09-20T18:56:40.696403+0000","last_peered":"2024-09-20T18:56:40.696403+0000","last_clean":"2024-09-20T18:56:40.696403+0000","last_became_active":"2024-09-20T18:55:28.830147+0000","last_became_peered":"2024-09-20T18:55:28.830147+0000","last_unstale":"2024-09-20T18:56:40.696403+0000","last_undegraded":"2024-09-20T18:56:40.696403+0000","last_fullsized":"2024-09-20T18:56:40.696403+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:13.144896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:13.144896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":392,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.691683+0000","last_change":"2024-09-20T18:55:28.844060+0000","last_active":"2024-09-20T18:56:40.691683+0000","last_peered":"2024-09-20T18:56:40.691683+0000","last_clean":"2024-09-20T18:56:40.691683+0000","last_became_active":"2024-09-20T18:55:28.829325+0000","last_became_peered":"2024-09-20T18:55:28.829325+0000","last_unstale":"2024-09-20T18:56:40.691683+0000","last_undegraded":"2024-09-20T18:56:40.691683+0000","last_fullsized":"2024-09-20T18:56:40.691683+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T18:55:13.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T18:55:13.696607+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":315,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.692781+0000","last_change":"2024-09-20T18:51:22.852942+0000","last_active":"2024-09-20T18:56:40.692781+0000","last_peered":"2024-09-20T18:56:40.692781+0000","last_clean":"2024-09-20T18:56:40.692781+0000","last_became_active":"2024-09-20T18:49:49.956128+0000","last_became_peered":"2024-09-20T18:49:49.956128+0000","last_unstale":"2024-09-20T18:56:40.692781+0000","last_undegraded":"2024-09-20T18:56:40.692781+0000","last_fullsized":"2024-09-20T18:56:40.692781+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:22.852869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:22.852869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":327,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.690429+0000","last_change":"2024-09-20T18:55:25.281435+0000","last_active":"2024-09-20T18:56:40.690429+0000","last_peered":"2024-09-20T18:56:40.690429+0000","last_clean":"2024-09-20T18:56:40.690429+0000","last_became_active":"2024-09-20T18:49:51.971332+0000","last_became_peered":"2024-09-20T18:49:51.971332+0000","last_unstale":"2024-09-20T18:56:40.690429+0000","last_undegraded":"2024-09-20T18:56:40.690429+0000","last_fullsized":"2024-09-20T18:56:40.690429+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:25.281366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:25.281366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"210'9170","reported_seq":9282,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:45.902429+0000","last_change":"2024-09-20T18:54:33.929366+0000","last_active":"2024-09-20T18:56:45.902429+0000","last_peered":"2024-09-20T18:56:45.902429+0000","last_clean":"2024-09-20T18:56:45.902429+0000","last_became_active":"2024-09-20T18:54:33.929072+0000","last_became_peered":"2024-09-20T18:54:33.929072+0000","last_unstale":"2024-09-20T18:56:45.902429+0000","last_undegraded":"2024-09-20T18:56:45.902429+0000","last_fullsized":"2024-09-20T18:56:45.902429+0000","mapping_epoch":160,"log_start":"175'4602","ondisk_log_start":"175'4602","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4568,"log_dups_size":0,"ondisk_log_size":4568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":353370112,"num_objects":5392,"num_object_clones":0,"num_object_copies":16176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9170,"num_write_kb":465984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":358,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.696656+0000","last_change":"2024-09-20T18:55:28.840105+0000","last_active":"2024-09-20T18:56:40.696656+0000","last_peered":"2024-09-20T18:56:40.696656+0000","last_clean":"2024-09-20T18:56:40.696656+0000","last_became_active":"2024-09-20T18:55:28.818269+0000","last_became_peered":"2024-09-20T18:55:28.818269+0000","last_unstale":"2024-09-20T18:56:40.696656+0000","last_undegraded":"2024-09-20T18:56:40.696656+0000","last_fullsized":"2024-09-20T18:56:40.696656+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:51:14.397930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.397930+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":324,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.695070+0000","last_change":"2024-09-20T18:51:18.410601+0000","last_active":"2024-09-20T18:56:40.695070+0000","last_peered":"2024-09-20T18:56:40.695070+0000","last_clean":"2024-09-20T18:56:40.695070+0000","last_became_active":"2024-09-20T18:49:45.944579+0000","last_became_peered":"2024-09-20T18:49:45.944579+0000","last_unstale":"2024-09-20T18:56:40.695070+0000","last_undegraded":"2024-09-20T18:56:40.695070+0000","last_fullsized":"2024-09-20T18:56:40.695070+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:18.410447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:18.410447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":338,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.690335+0000","last_change":"2024-09-20T18:55:28.830634+0000","last_active":"2024-09-20T18:56:40.690335+0000","last_peered":"2024-09-20T18:56:40.690335+0000","last_clean":"2024-09-20T18:56:40.690335+0000","last_became_active":"2024-09-20T18:55:28.827785+0000","last_became_peered":"2024-09-20T18:55:28.827785+0000","last_unstale":"2024-09-20T18:56:40.690335+0000","last_undegraded":"2024-09-20T18:56:40.690335+0000","last_fullsized":"2024-09-20T18:56:40.690335+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:11.401777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:11.401777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":319,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.696696+0000","last_change":"2024-09-20T18:55:28.833649+0000","last_active":"2024-09-20T18:56:40.696696+0000","last_peered":"2024-09-20T18:56:40.696696+0000","last_clean":"2024-09-20T18:56:40.696696+0000","last_became_active":"2024-09-20T18:55:28.833115+0000","last_became_peered":"2024-09-20T18:55:28.833115+0000","last_unstale":"2024-09-20T18:56:40.696696+0000","last_undegraded":"2024-09-20T18:56:40.696696+0000","last_fullsized":"2024-09-20T18:56:40.696696+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:05.021071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.021071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1","version":"210'9073","reported_seq":9185,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:46.201089+0000","last_change":"2024-09-20T18:54:34.310237+0000","last_active":"2024-09-20T18:56:46.201089+0000","last_peered":"2024-09-20T18:56:46.201089+0000","last_clean":"2024-09-20T18:56:46.201089+0000","last_became_active":"2024-09-20T18:54:34.309781+0000","last_became_peered":"2024-09-20T18:54:34.309781+0000","last_unstale":"2024-09-20T18:56:46.201089+0000","last_undegraded":"2024-09-20T18:56:46.201089+0000","last_fullsized":"2024-09-20T18:56:46.201089+0000","mapping_epoch":160,"log_start":"177'4824","ondisk_log_start":"177'4824","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4249,"log_dups_size":0,"ondisk_log_size":4249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":351207424,"num_objects":5359,"num_object_clones":0,"num_object_copies":16077,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9073,"num_write_kb":461824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":356,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.699422+0000","last_change":"2024-09-20T18:52:02.944536+0000","last_active":"2024-09-20T18:56:40.699422+0000","last_peered":"2024-09-20T18:56:40.699422+0000","last_clean":"2024-09-20T18:56:40.699422+0000","last_became_active":"2024-09-20T18:49:47.947636+0000","last_became_peered":"2024-09-20T18:49:47.947636+0000","last_unstale":"2024-09-20T18:56:40.699422+0000","last_undegraded":"2024-09-20T18:56:40.699422+0000","last_fullsized":"2024-09-20T18:56:40.699422+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-20T18:52:02.944434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:02.944434+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":193,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.695583+0000","last_change":"2024-09-20T18:55:28.822330+0000","last_active":"2024-09-20T18:56:40.695583+0000","last_peered":"2024-09-20T18:56:40.695583+0000","last_clean":"2024-09-20T18:56:40.695583+0000","last_became_active":"2024-09-20T18:55:28.821959+0000","last_became_peered":"2024-09-20T18:55:28.821959+0000","last_unstale":"2024-09-20T18:56:40.695583+0000","last_undegraded":"2024-09-20T18:56:40.695583+0000","last_fullsized":"2024-09-20T18:56:40.695583+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:45.669427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:45.669427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":324,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.691094+0000","last_change":"2024-09-20T18:54:11.377907+0000","last_active":"2024-09-20T18:56:40.691094+0000","last_peered":"2024-09-20T18:56:40.691094+0000","last_clean":"2024-09-20T18:56:40.691094+0000","last_became_active":"2024-09-20T18:49:49.959160+0000","last_became_peered":"2024-09-20T18:49:49.959160+0000","last_unstale":"2024-09-20T18:56:40.691094+0000","last_undegraded":"2024-09-20T18:56:40.691094+0000","last_fullsized":"2024-09-20T18:56:40.691094+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:11.376951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:11.376951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":321,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.699754+0000","last_change":"2024-09-20T18:52:35.635897+0000","last_active":"2024-09-20T18:56:40.699754+0000","last_peered":"2024-09-20T18:56:40.699754+0000","last_clean":"2024-09-20T18:56:40.699754+0000","last_became_active":"2024-09-20T18:49:51.970108+0000","last_became_peered":"2024-09-20T18:49:51.970108+0000","last_unstale":"2024-09-20T18:56:40.699754+0000","last_undegraded":"2024-09-20T18:56:40.699754+0000","last_fullsized":"2024-09-20T18:56:40.699754+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:35.635818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:35.635818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"210'8968","reported_seq":9080,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:46.436466+0000","last_change":"2024-09-20T18:54:33.929552+0000","last_active":"2024-09-20T18:56:46.436466+0000","last_peered":"2024-09-20T18:56:46.436466+0000","last_clean":"2024-09-20T18:56:46.436466+0000","last_became_active":"2024-09-20T18:54:33.929208+0000","last_became_peered":"2024-09-20T18:54:33.929208+0000","last_unstale":"2024-09-20T18:56:46.436466+0000","last_undegraded":"2024-09-20T18:56:46.436466+0000","last_fullsized":"2024-09-20T18:56:46.436466+0000","mapping_epoch":160,"log_start":"176'4621","ondisk_log_start":"176'4621","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4347,"log_dups_size":0,"ondisk_log_size":4347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":344588288,"num_objects":5258,"num_object_clones":0,"num_object_copies":15774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8968,"num_write_kb":455232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":362,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.695320+0000","last_change":"2024-09-20T18:55:28.846773+0000","last_active":"2024-09-20T18:56:40.695320+0000","last_peered":"2024-09-20T18:56:40.695320+0000","last_clean":"2024-09-20T18:56:40.695320+0000","last_became_active":"2024-09-20T18:55:28.817101+0000","last_became_peered":"2024-09-20T18:55:28.817101+0000","last_unstale":"2024-09-20T18:56:40.695320+0000","last_undegraded":"2024-09-20T18:56:40.695320+0000","last_fullsized":"2024-09-20T18:56:40.695320+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:54:17.219548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:17.219548+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":332,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.697372+0000","last_change":"2024-09-20T18:52:46.005172+0000","last_active":"2024-09-20T18:56:40.697372+0000","last_peered":"2024-09-20T18:56:40.697372+0000","last_clean":"2024-09-20T18:56:40.697372+0000","last_became_active":"2024-09-20T18:49:45.937264+0000","last_became_peered":"2024-09-20T18:49:45.937264+0000","last_unstale":"2024-09-20T18:56:40.697372+0000","last_undegraded":"2024-09-20T18:56:40.697372+0000","last_fullsized":"2024-09-20T18:56:40.697372+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:46.005109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:46.005109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":426,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:41.080709+0000","last_change":"2024-09-20T18:55:17.549660+0000","last_active":"2024-09-20T18:56:41.080709+0000","last_peered":"2024-09-20T18:56:41.080709+0000","last_clean":"2024-09-20T18:56:41.080709+0000","last_became_active":"2024-09-20T18:49:49.957005+0000","last_became_peered":"2024-09-20T18:49:49.957005+0000","last_unstale":"2024-09-20T18:56:41.080709+0000","last_undegraded":"2024-09-20T18:56:41.080709+0000","last_fullsized":"2024-09-20T18:56:41.080709+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:17.549598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:17.549598+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":320,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.696508+0000","last_change":"2024-09-20T18:54:42.182875+0000","last_active":"2024-09-20T18:56:40.696508+0000","last_peered":"2024-09-20T18:56:40.696508+0000","last_clean":"2024-09-20T18:56:40.696508+0000","last_became_active":"2024-09-20T18:49:51.969355+0000","last_became_peered":"2024-09-20T18:49:51.969355+0000","last_unstale":"2024-09-20T18:56:40.696508+0000","last_undegraded":"2024-09-20T18:56:40.696508+0000","last_fullsized":"2024-09-20T18:56:40.696508+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:42.182809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:42.182809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"210'9158","reported_seq":19217,"reported_epoch":210,"state":"active+remapped+backfilling","last_fresh":"2024-09-20T18:56:46.249617+0000","last_change":"2024-09-20T18:55:59.938306+0000","last_active":"2024-09-20T18:56:46.249617+0000","last_peered":"2024-09-20T18:56:46.249617+0000","last_clean":"2024-09-20T18:55:27.792821+0000","last_became_active":"2024-09-20T18:55:29.915245+0000","last_became_peered":"2024-09-20T18:55:29.915245+0000","last_unstale":"2024-09-20T18:56:46.249617+0000","last_undegraded":"2024-09-20T18:56:46.249617+0000","last_fullsized":"2024-09-20T18:56:46.249617+0000","mapping_epoch":182,"log_start":"176'4861","ondisk_log_start":"176'4861","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4297,"log_dups_size":0,"ondisk_log_size":4297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":353370112,"num_objects":5392,"num_object_clones":0,"num_object_copies":16176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1237,"num_objects_unfound":0,"num_objects_dirty":5392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9158,"num_write_kb":465600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4969,"num_bytes_recovered":325648384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,6],"avail_no_missing":["1","3","6"],"object_location_counts":[{"shards":"1,3,6","objects":5392}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":351,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.698086+0000","last_change":"2024-09-20T18:54:54.897991+0000","last_active":"2024-09-20T18:56:40.698086+0000","last_peered":"2024-09-20T18:56:40.698086+0000","last_clean":"2024-09-20T18:56:40.698086+0000","last_became_active":"2024-09-20T18:49:47.946008+0000","last_became_peered":"2024-09-20T18:49:47.946008+0000","last_unstale":"2024-09-20T18:56:40.698086+0000","last_undegraded":"2024-09-20T18:56:40.698086+0000","last_fullsized":"2024-09-20T18:56:40.698086+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:54.897933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:54.897933+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":346,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.689813+0000","last_change":"2024-09-20T18:55:28.833453+0000","last_active":"2024-09-20T18:56:40.689813+0000","last_peered":"2024-09-20T18:56:40.689813+0000","last_clean":"2024-09-20T18:56:40.689813+0000","last_became_active":"2024-09-20T18:55:28.833095+0000","last_became_peered":"2024-09-20T18:55:28.833095+0000","last_unstale":"2024-09-20T18:56:40.689813+0000","last_undegraded":"2024-09-20T18:56:40.689813+0000","last_fullsized":"2024-09-20T18:56:40.689813+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:56.212378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:56.212378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":333,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.693262+0000","last_change":"2024-09-20T18:54:43.379898+0000","last_active":"2024-09-20T18:56:40.693262+0000","last_peered":"2024-09-20T18:56:40.693262+0000","last_clean":"2024-09-20T18:56:40.693262+0000","last_became_active":"2024-09-20T18:49:49.956767+0000","last_became_peered":"2024-09-20T18:49:49.956767+0000","last_unstale":"2024-09-20T18:56:40.693262+0000","last_undegraded":"2024-09-20T18:56:40.693262+0000","last_fullsized":"2024-09-20T18:56:40.693262+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:43.379706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:43.379706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":319,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.698256+0000","last_change":"2024-09-20T18:54:24.976627+0000","last_active":"2024-09-20T18:56:40.698256+0000","last_peered":"2024-09-20T18:56:40.698256+0000","last_clean":"2024-09-20T18:56:40.698256+0000","last_became_active":"2024-09-20T18:49:51.964098+0000","last_became_peered":"2024-09-20T18:49:51.964098+0000","last_unstale":"2024-09-20T18:56:40.698256+0000","last_undegraded":"2024-09-20T18:56:40.698256+0000","last_fullsized":"2024-09-20T18:56:40.698256+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:24.976584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:24.976584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"210'9204","reported_seq":11851,"reported_epoch":210,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-20T18:56:46.649190+0000","last_change":"2024-09-20T18:56:27.871592+0000","last_active":"2024-09-20T18:56:46.649190+0000","last_peered":"2024-09-20T18:56:46.649190+0000","last_clean":"2024-09-20T18:55:27.791097+0000","last_became_active":"2024-09-20T18:55:29.979932+0000","last_became_peered":"2024-09-20T18:55:29.979932+0000","last_unstale":"2024-09-20T18:56:46.649190+0000","last_undegraded":"2024-09-20T18:56:27.867865+0000","last_fullsized":"2024-09-20T18:55:29.815116+0000","mapping_epoch":182,"log_start":"175'4340","ondisk_log_start":"175'4340","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4864,"log_dups_size":0,"ondisk_log_size":4864,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":343670784,"num_objects":5244,"num_object_clones":0,"num_object_copies":15732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":713,"num_objects_misplaced":3198,"num_objects_unfound":0,"num_objects_dirty":5244,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9204,"num_write_kb":462336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1267,"num_bytes_recovered":83034112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7],"avail_no_missing":["0","6","7"],"object_location_counts":[{"shards":"0,6,7","objects":3198},{"shards":"0,7","objects":1333}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":140,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.691148+0000","last_change":"2024-09-20T18:55:28.855793+0000","last_active":"2024-09-20T18:56:40.691148+0000","last_peered":"2024-09-20T18:56:40.691148+0000","last_clean":"2024-09-20T18:56:40.691148+0000","last_became_active":"2024-09-20T18:55:28.828563+0000","last_became_peered":"2024-09-20T18:55:28.828563+0000","last_unstale":"2024-09-20T18:56:40.691148+0000","last_undegraded":"2024-09-20T18:56:40.691148+0000","last_fullsized":"2024-09-20T18:56:40.691148+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-20T18:53:24.458005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:24.458005+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":31,"num_read_kb":20,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,5],"acting":[1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":346,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.693103+0000","last_change":"2024-09-20T18:55:28.835767+0000","last_active":"2024-09-20T18:56:40.693103+0000","last_peered":"2024-09-20T18:56:40.693103+0000","last_clean":"2024-09-20T18:56:40.693103+0000","last_became_active":"2024-09-20T18:55:28.835149+0000","last_became_peered":"2024-09-20T18:55:28.835149+0000","last_unstale":"2024-09-20T18:56:40.693103+0000","last_undegraded":"2024-09-20T18:56:40.693103+0000","last_fullsized":"2024-09-20T18:56:40.693103+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:44.501192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.501192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":333,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.693232+0000","last_change":"2024-09-20T18:55:13.577772+0000","last_active":"2024-09-20T18:56:40.693232+0000","last_peered":"2024-09-20T18:56:40.693232+0000","last_clean":"2024-09-20T18:56:40.693232+0000","last_became_active":"2024-09-20T18:49:49.957558+0000","last_became_peered":"2024-09-20T18:49:49.957558+0000","last_unstale":"2024-09-20T18:56:40.693232+0000","last_undegraded":"2024-09-20T18:56:40.693232+0000","last_fullsized":"2024-09-20T18:56:40.693232+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:13.577699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:13.577699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":327,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.691623+0000","last_change":"2024-09-20T18:53:47.370152+0000","last_active":"2024-09-20T18:56:40.691623+0000","last_peered":"2024-09-20T18:56:40.691623+0000","last_clean":"2024-09-20T18:56:40.691623+0000","last_became_active":"2024-09-20T18:49:51.970798+0000","last_became_peered":"2024-09-20T18:49:51.970798+0000","last_unstale":"2024-09-20T18:56:40.691623+0000","last_undegraded":"2024-09-20T18:56:40.691623+0000","last_fullsized":"2024-09-20T18:56:40.691623+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:47.370108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:47.370108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"210'9367","reported_seq":15602,"reported_epoch":210,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-20T18:56:45.883905+0000","last_change":"2024-09-20T18:55:30.067593+0000","last_active":"2024-09-20T18:56:45.883905+0000","last_peered":"2024-09-20T18:56:45.883905+0000","last_clean":"2024-09-20T18:55:27.760955+0000","last_became_active":"2024-09-20T18:55:30.004922+0000","last_became_peered":"2024-09-20T18:55:30.004922+0000","last_unstale":"2024-09-20T18:56:45.883905+0000","last_undegraded":"2024-09-20T18:56:45.883905+0000","last_fullsized":"2024-09-20T18:55:29.819726+0000","mapping_epoch":182,"log_start":"175'4746","ondisk_log_start":"175'4746","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4621,"log_dups_size":0,"ondisk_log_size":4621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":368246784,"num_objects":5619,"num_object_clones":0,"num_object_copies":16857,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2331,"num_objects_unfound":0,"num_objects_dirty":5619,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9367,"num_write_kb":479552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3060,"num_bytes_recovered":200540160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7],"avail_no_missing":["4","6","7"],"object_location_counts":[{"shards":"4,6,7","objects":2331},{"shards":"4,7","objects":3288}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":351,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.694886+0000","last_change":"2024-09-20T18:53:52.898818+0000","last_active":"2024-09-20T18:56:40.694886+0000","last_peered":"2024-09-20T18:56:40.694886+0000","last_clean":"2024-09-20T18:56:40.694886+0000","last_became_active":"2024-09-20T18:49:47.949116+0000","last_became_peered":"2024-09-20T18:49:47.949116+0000","last_unstale":"2024-09-20T18:56:40.694886+0000","last_undegraded":"2024-09-20T18:56:40.694886+0000","last_fullsized":"2024-09-20T18:56:40.694886+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:53:52.898760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:52.898760+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":339,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.693048+0000","last_change":"2024-09-20T18:54:52.650798+0000","last_active":"2024-09-20T18:56:40.693048+0000","last_peered":"2024-09-20T18:56:40.693048+0000","last_clean":"2024-09-20T18:56:40.693048+0000","last_became_active":"2024-09-20T18:49:45.944858+0000","last_became_peered":"2024-09-20T18:49:45.944858+0000","last_unstale":"2024-09-20T18:56:40.693048+0000","last_undegraded":"2024-09-20T18:56:40.693048+0000","last_fullsized":"2024-09-20T18:56:40.693048+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:52.650756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:52.650756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":325,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.692516+0000","last_change":"2024-09-20T18:54:42.349879+0000","last_active":"2024-09-20T18:56:40.692516+0000","last_peered":"2024-09-20T18:56:40.692516+0000","last_clean":"2024-09-20T18:56:40.692516+0000","last_became_active":"2024-09-20T18:49:49.954944+0000","last_became_peered":"2024-09-20T18:49:49.954944+0000","last_unstale":"2024-09-20T18:56:40.692516+0000","last_undegraded":"2024-09-20T18:56:40.692516+0000","last_fullsized":"2024-09-20T18:56:40.692516+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:42.349807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:42.349807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":320,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.697080+0000","last_change":"2024-09-20T18:55:17.341862+0000","last_active":"2024-09-20T18:56:40.697080+0000","last_peered":"2024-09-20T18:56:40.697080+0000","last_clean":"2024-09-20T18:56:40.697080+0000","last_became_active":"2024-09-20T18:49:51.972970+0000","last_became_peered":"2024-09-20T18:49:51.972970+0000","last_unstale":"2024-09-20T18:56:40.697080+0000","last_undegraded":"2024-09-20T18:56:40.697080+0000","last_fullsized":"2024-09-20T18:56:40.697080+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:17.341793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:17.341793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.4","version":"210'9025","reported_seq":9137,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:45.883245+0000","last_change":"2024-09-20T18:54:34.309442+0000","last_active":"2024-09-20T18:56:45.883245+0000","last_peered":"2024-09-20T18:56:45.883245+0000","last_clean":"2024-09-20T18:56:45.883245+0000","last_became_active":"2024-09-20T18:54:34.309113+0000","last_became_peered":"2024-09-20T18:54:34.309113+0000","last_unstale":"2024-09-20T18:56:45.883245+0000","last_undegraded":"2024-09-20T18:56:45.883245+0000","last_fullsized":"2024-09-20T18:56:45.883245+0000","mapping_epoch":160,"log_start":"174'3931","ondisk_log_start":"174'3931","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":5094,"log_dups_size":0,"ondisk_log_size":5094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":347799552,"num_objects":5307,"num_object_clones":0,"num_object_copies":15921,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9025,"num_write_kb":458624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":375,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.689597+0000","last_change":"2024-09-20T18:55:28.847010+0000","last_active":"2024-09-20T18:56:40.689597+0000","last_peered":"2024-09-20T18:56:40.689597+0000","last_clean":"2024-09-20T18:56:40.689597+0000","last_became_active":"2024-09-20T18:55:28.824801+0000","last_became_peered":"2024-09-20T18:55:28.824801+0000","last_unstale":"2024-09-20T18:56:40.689597+0000","last_undegraded":"2024-09-20T18:56:40.689597+0000","last_fullsized":"2024-09-20T18:56:40.689597+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:54:12.373309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:12.373309+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":359,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.692860+0000","last_change":"2024-09-20T18:55:24.729301+0000","last_active":"2024-09-20T18:56:40.692860+0000","last_peered":"2024-09-20T18:56:40.692860+0000","last_clean":"2024-09-20T18:56:40.692860+0000","last_became_active":"2024-09-20T18:49:45.944256+0000","last_became_peered":"2024-09-20T18:49:45.944256+0000","last_unstale":"2024-09-20T18:56:40.692860+0000","last_undegraded":"2024-09-20T18:56:40.692860+0000","last_fullsized":"2024-09-20T18:56:40.692860+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-20T18:55:24.729219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:24.729219+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"184'9","reported_seq":418,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.691829+0000","last_change":"2024-09-20T18:55:28.824771+0000","last_active":"2024-09-20T18:56:40.691829+0000","last_peered":"2024-09-20T18:56:40.691829+0000","last_clean":"2024-09-20T18:56:40.691829+0000","last_became_active":"2024-09-20T18:55:28.818085+0000","last_became_peered":"2024-09-20T18:55:28.818085+0000","last_unstale":"2024-09-20T18:56:40.691829+0000","last_undegraded":"2024-09-20T18:56:40.691829+0000","last_fullsized":"2024-09-20T18:56:40.691829+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:52:05.788924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.788924+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":326,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.689764+0000","last_change":"2024-09-20T18:55:28.832276+0000","last_active":"2024-09-20T18:56:40.689764+0000","last_peered":"2024-09-20T18:56:40.689764+0000","last_clean":"2024-09-20T18:56:40.689764+0000","last_became_active":"2024-09-20T18:55:28.830380+0000","last_became_peered":"2024-09-20T18:55:28.830380+0000","last_unstale":"2024-09-20T18:56:40.689764+0000","last_undegraded":"2024-09-20T18:56:40.689764+0000","last_fullsized":"2024-09-20T18:56:40.689764+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:14.247398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:14.247398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":69,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.696207+0000","last_change":"2024-09-20T18:55:28.835344+0000","last_active":"2024-09-20T18:56:40.696207+0000","last_peered":"2024-09-20T18:56:40.696207+0000","last_clean":"2024-09-20T18:56:40.696207+0000","last_became_active":"2024-09-20T18:55:28.835010+0000","last_became_peered":"2024-09-20T18:55:28.835010+0000","last_unstale":"2024-09-20T18:56:40.696207+0000","last_undegraded":"2024-09-20T18:56:40.696207+0000","last_fullsized":"2024-09-20T18:56:40.696207+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:26.721735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:26.721735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:49.502875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":336,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.698285+0000","last_change":"2024-09-20T18:54:44.194161+0000","last_active":"2024-09-20T18:56:40.698285+0000","last_peered":"2024-09-20T18:56:40.698285+0000","last_clean":"2024-09-20T18:56:40.698285+0000","last_became_active":"2024-09-20T18:49:48.500662+0000","last_became_peered":"2024-09-20T18:49:48.500662+0000","last_unstale":"2024-09-20T18:56:40.698285+0000","last_undegraded":"2024-09-20T18:56:40.698285+0000","last_fullsized":"2024-09-20T18:56:40.698285+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-20T18:54:44.194089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.194089+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":322,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.690104+0000","last_change":"2024-09-20T18:53:16.220554+0000","last_active":"2024-09-20T18:56:40.690104+0000","last_peered":"2024-09-20T18:56:40.690104+0000","last_clean":"2024-09-20T18:56:40.690104+0000","last_became_active":"2024-09-20T18:49:49.956749+0000","last_became_peered":"2024-09-20T18:49:49.956749+0000","last_unstale":"2024-09-20T18:56:40.690104+0000","last_undegraded":"2024-09-20T18:56:40.690104+0000","last_fullsized":"2024-09-20T18:56:40.690104+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:16.220490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:16.220490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":333,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.692657+0000","last_change":"2024-09-20T18:55:28.827897+0000","last_active":"2024-09-20T18:56:40.692657+0000","last_peered":"2024-09-20T18:56:40.692657+0000","last_clean":"2024-09-20T18:56:40.692657+0000","last_became_active":"2024-09-20T18:55:28.827418+0000","last_became_peered":"2024-09-20T18:55:28.827418+0000","last_unstale":"2024-09-20T18:56:40.692657+0000","last_undegraded":"2024-09-20T18:56:40.692657+0000","last_fullsized":"2024-09-20T18:56:40.692657+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.637739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.637739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":342,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.702534+0000","last_change":"2024-09-20T18:54:44.830872+0000","last_active":"2024-09-20T18:56:40.702534+0000","last_peered":"2024-09-20T18:56:40.702534+0000","last_clean":"2024-09-20T18:56:40.702534+0000","last_became_active":"2024-09-20T18:49:48.500803+0000","last_became_peered":"2024-09-20T18:49:48.500803+0000","last_unstale":"2024-09-20T18:56:40.702534+0000","last_undegraded":"2024-09-20T18:56:40.702534+0000","last_fullsized":"2024-09-20T18:56:40.702534+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:44.830790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.830790+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":337,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.691972+0000","last_change":"2024-09-20T18:55:29.751718+0000","last_active":"2024-09-20T18:56:40.691972+0000","last_peered":"2024-09-20T18:56:40.691972+0000","last_clean":"2024-09-20T18:56:40.691972+0000","last_became_active":"2024-09-20T18:55:28.833799+0000","last_became_peered":"2024-09-20T18:55:28.833799+0000","last_unstale":"2024-09-20T18:56:40.691972+0000","last_undegraded":"2024-09-20T18:56:40.691972+0000","last_fullsized":"2024-09-20T18:56:40.691972+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:29.751642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:29.751642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:40.748541+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":323,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.696747+0000","last_change":"2024-09-20T18:53:50.253721+0000","last_active":"2024-09-20T18:56:40.696747+0000","last_peered":"2024-09-20T18:56:40.696747+0000","last_clean":"2024-09-20T18:56:40.696747+0000","last_became_active":"2024-09-20T18:49:49.961947+0000","last_became_peered":"2024-09-20T18:49:49.961947+0000","last_unstale":"2024-09-20T18:56:40.696747+0000","last_undegraded":"2024-09-20T18:56:40.696747+0000","last_fullsized":"2024-09-20T18:56:40.696747+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:50.253660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:50.253660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"66'1","reported_seq":329,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.693733+0000","last_change":"2024-09-20T18:54:20.681878+0000","last_active":"2024-09-20T18:56:40.693733+0000","last_peered":"2024-09-20T18:56:40.693733+0000","last_clean":"2024-09-20T18:56:40.693733+0000","last_became_active":"2024-09-20T18:49:51.965121+0000","last_became_peered":"2024-09-20T18:49:51.965121+0000","last_unstale":"2024-09-20T18:56:40.693733+0000","last_undegraded":"2024-09-20T18:56:40.693733+0000","last_fullsized":"2024-09-20T18:56:40.693733+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T18:54:20.681790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:20.681790+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":334,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.690831+0000","last_change":"2024-09-20T18:55:12.205118+0000","last_active":"2024-09-20T18:56:40.690831+0000","last_peered":"2024-09-20T18:56:40.690831+0000","last_clean":"2024-09-20T18:56:40.690831+0000","last_became_active":"2024-09-20T18:49:48.501011+0000","last_became_peered":"2024-09-20T18:49:48.501011+0000","last_unstale":"2024-09-20T18:56:40.690831+0000","last_undegraded":"2024-09-20T18:56:40.690831+0000","last_fullsized":"2024-09-20T18:56:40.690831+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:55:12.205069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:12.205069+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":338,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.697246+0000","last_change":"2024-09-20T18:55:27.028184+0000","last_active":"2024-09-20T18:56:40.697246+0000","last_peered":"2024-09-20T18:56:40.697246+0000","last_clean":"2024-09-20T18:56:40.697246+0000","last_became_active":"2024-09-20T18:49:45.938511+0000","last_became_peered":"2024-09-20T18:49:45.938511+0000","last_unstale":"2024-09-20T18:56:40.697246+0000","last_undegraded":"2024-09-20T18:56:40.697246+0000","last_fullsized":"2024-09-20T18:56:40.697246+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:27.028126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:27.028126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:47.494086+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":424,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:41.077609+0000","last_change":"2024-09-20T18:55:27.284156+0000","last_active":"2024-09-20T18:56:41.077609+0000","last_peered":"2024-09-20T18:56:41.077609+0000","last_clean":"2024-09-20T18:56:41.077609+0000","last_became_active":"2024-09-20T18:49:49.960550+0000","last_became_peered":"2024-09-20T18:49:49.960550+0000","last_unstale":"2024-09-20T18:56:41.077609+0000","last_undegraded":"2024-09-20T18:56:41.077609+0000","last_fullsized":"2024-09-20T18:56:41.077609+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:27.284086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:27.284086+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:48.225116+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":327,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.696269+0000","last_change":"2024-09-20T18:55:12.348079+0000","last_active":"2024-09-20T18:56:40.696269+0000","last_peered":"2024-09-20T18:56:40.696269+0000","last_clean":"2024-09-20T18:56:40.696269+0000","last_became_active":"2024-09-20T18:49:51.966254+0000","last_became_peered":"2024-09-20T18:49:51.966254+0000","last_unstale":"2024-09-20T18:56:40.696269+0000","last_undegraded":"2024-09-20T18:56:40.696269+0000","last_fullsized":"2024-09-20T18:56:40.696269+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:12.347993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:12.347993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":353,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.698003+0000","last_change":"2024-09-20T18:51:15.223913+0000","last_active":"2024-09-20T18:56:40.698003+0000","last_peered":"2024-09-20T18:56:40.698003+0000","last_clean":"2024-09-20T18:56:40.698003+0000","last_became_active":"2024-09-20T18:49:58.997494+0000","last_became_peered":"2024-09-20T18:49:58.997494+0000","last_unstale":"2024-09-20T18:56:40.698003+0000","last_undegraded":"2024-09-20T18:56:40.698003+0000","last_fullsized":"2024-09-20T18:56:40.698003+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:51:15.223826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:15.223826+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":338,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.692538+0000","last_change":"2024-09-20T18:54:57.632458+0000","last_active":"2024-09-20T18:56:40.692538+0000","last_peered":"2024-09-20T18:56:40.692538+0000","last_clean":"2024-09-20T18:56:40.692538+0000","last_became_active":"2024-09-20T18:49:45.949492+0000","last_became_peered":"2024-09-20T18:49:45.949492+0000","last_unstale":"2024-09-20T18:56:40.692538+0000","last_undegraded":"2024-09-20T18:56:40.692538+0000","last_fullsized":"2024-09-20T18:56:40.692538+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.632380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.632380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":422,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.690028+0000","last_change":"2024-09-20T18:55:28.302353+0000","last_active":"2024-09-20T18:56:40.690028+0000","last_peered":"2024-09-20T18:56:40.690028+0000","last_clean":"2024-09-20T18:56:40.690028+0000","last_became_active":"2024-09-20T18:49:49.962243+0000","last_became_peered":"2024-09-20T18:49:49.962243+0000","last_unstale":"2024-09-20T18:56:40.690028+0000","last_undegraded":"2024-09-20T18:56:40.690028+0000","last_fullsized":"2024-09-20T18:56:40.690028+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:28.302303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:28.302303+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:55.176573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":326,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.699281+0000","last_change":"2024-09-20T18:55:11.918284+0000","last_active":"2024-09-20T18:56:40.699281+0000","last_peered":"2024-09-20T18:56:40.699281+0000","last_clean":"2024-09-20T18:56:40.699281+0000","last_became_active":"2024-09-20T18:49:51.969093+0000","last_became_peered":"2024-09-20T18:49:51.969093+0000","last_unstale":"2024-09-20T18:56:40.699281+0000","last_undegraded":"2024-09-20T18:56:40.699281+0000","last_fullsized":"2024-09-20T18:56:40.699281+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:11.918210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:11.918210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":365,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.695140+0000","last_change":"2024-09-20T18:55:28.854249+0000","last_active":"2024-09-20T18:56:40.695140+0000","last_peered":"2024-09-20T18:56:40.695140+0000","last_clean":"2024-09-20T18:56:40.695140+0000","last_became_active":"2024-09-20T18:55:28.826721+0000","last_became_peered":"2024-09-20T18:55:28.826721+0000","last_unstale":"2024-09-20T18:56:40.695140+0000","last_undegraded":"2024-09-20T18:56:40.695140+0000","last_fullsized":"2024-09-20T18:56:40.695140+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:53:43.878921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:43.878921+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":331,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.695539+0000","last_change":"2024-09-20T18:55:28.822690+0000","last_active":"2024-09-20T18:56:40.695539+0000","last_peered":"2024-09-20T18:56:40.695539+0000","last_clean":"2024-09-20T18:56:40.695539+0000","last_became_active":"2024-09-20T18:55:28.822258+0000","last_became_peered":"2024-09-20T18:55:28.822258+0000","last_unstale":"2024-09-20T18:56:40.695539+0000","last_undegraded":"2024-09-20T18:56:40.695539+0000","last_fullsized":"2024-09-20T18:56:40.695539+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:05.985551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.985551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":315,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.691065+0000","last_change":"2024-09-20T18:51:14.271954+0000","last_active":"2024-09-20T18:56:40.691065+0000","last_peered":"2024-09-20T18:56:40.691065+0000","last_clean":"2024-09-20T18:56:40.691065+0000","last_became_active":"2024-09-20T18:49:49.957067+0000","last_became_peered":"2024-09-20T18:49:49.957067+0000","last_unstale":"2024-09-20T18:56:40.691065+0000","last_undegraded":"2024-09-20T18:56:40.691065+0000","last_fullsized":"2024-09-20T18:56:40.691065+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:14.271719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.271719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":311,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.696640+0000","last_change":"2024-09-20T18:51:14.210438+0000","last_active":"2024-09-20T18:56:40.696640+0000","last_peered":"2024-09-20T18:56:40.696640+0000","last_clean":"2024-09-20T18:56:40.696640+0000","last_became_active":"2024-09-20T18:49:51.966915+0000","last_became_peered":"2024-09-20T18:49:51.966915+0000","last_unstale":"2024-09-20T18:56:40.696640+0000","last_undegraded":"2024-09-20T18:56:40.696640+0000","last_fullsized":"2024-09-20T18:56:40.696640+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:14.210358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.210358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":368,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.691651+0000","last_change":"2024-09-20T18:55:28.838698+0000","last_active":"2024-09-20T18:56:40.691651+0000","last_peered":"2024-09-20T18:56:40.691651+0000","last_clean":"2024-09-20T18:56:40.691651+0000","last_became_active":"2024-09-20T18:55:28.827023+0000","last_became_peered":"2024-09-20T18:55:28.827023+0000","last_unstale":"2024-09-20T18:56:40.691651+0000","last_undegraded":"2024-09-20T18:56:40.691651+0000","last_fullsized":"2024-09-20T18:56:40.691651+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:53:53.263976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:53.263976+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":330,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.692947+0000","last_change":"2024-09-20T18:54:42.487465+0000","last_active":"2024-09-20T18:56:40.692947+0000","last_peered":"2024-09-20T18:56:40.692947+0000","last_clean":"2024-09-20T18:56:40.692947+0000","last_became_active":"2024-09-20T18:49:45.944357+0000","last_became_peered":"2024-09-20T18:49:45.944357+0000","last_unstale":"2024-09-20T18:56:40.692947+0000","last_undegraded":"2024-09-20T18:56:40.692947+0000","last_fullsized":"2024-09-20T18:56:40.692947+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:42.487403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:42.487403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":323,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.697156+0000","last_change":"2024-09-20T18:54:18.108239+0000","last_active":"2024-09-20T18:56:40.697156+0000","last_peered":"2024-09-20T18:56:40.697156+0000","last_clean":"2024-09-20T18:56:40.697156+0000","last_became_active":"2024-09-20T18:49:49.962523+0000","last_became_peered":"2024-09-20T18:49:49.962523+0000","last_unstale":"2024-09-20T18:56:40.697156+0000","last_undegraded":"2024-09-20T18:56:40.697156+0000","last_fullsized":"2024-09-20T18:56:40.697156+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:18.108175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:18.108175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":326,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.695874+0000","last_change":"2024-09-20T18:54:26.882441+0000","last_active":"2024-09-20T18:56:40.695874+0000","last_peered":"2024-09-20T18:56:40.695874+0000","last_clean":"2024-09-20T18:56:40.695874+0000","last_became_active":"2024-09-20T18:49:51.967313+0000","last_became_peered":"2024-09-20T18:49:51.967313+0000","last_unstale":"2024-09-20T18:56:40.695874+0000","last_undegraded":"2024-09-20T18:56:40.695874+0000","last_fullsized":"2024-09-20T18:56:40.695874+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:26.882375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:26.882375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":366,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.696856+0000","last_change":"2024-09-20T18:55:28.843363+0000","last_active":"2024-09-20T18:56:40.696856+0000","last_peered":"2024-09-20T18:56:40.696856+0000","last_clean":"2024-09-20T18:56:40.696856+0000","last_became_active":"2024-09-20T18:55:28.830801+0000","last_became_peered":"2024-09-20T18:55:28.830801+0000","last_unstale":"2024-09-20T18:56:40.696856+0000","last_undegraded":"2024-09-20T18:56:40.696856+0000","last_fullsized":"2024-09-20T18:56:40.696856+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:54:21.106677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:21.106677+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":338,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.690193+0000","last_change":"2024-09-20T18:55:19.258723+0000","last_active":"2024-09-20T18:56:40.690193+0000","last_peered":"2024-09-20T18:56:40.690193+0000","last_clean":"2024-09-20T18:56:40.690193+0000","last_became_active":"2024-09-20T18:49:45.942105+0000","last_became_peered":"2024-09-20T18:49:45.942105+0000","last_unstale":"2024-09-20T18:56:40.690193+0000","last_undegraded":"2024-09-20T18:56:40.690193+0000","last_fullsized":"2024-09-20T18:56:40.690193+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:19.258674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:19.258674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":337,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.691415+0000","last_change":"2024-09-20T18:55:28.825004+0000","last_active":"2024-09-20T18:56:40.691415+0000","last_peered":"2024-09-20T18:56:40.691415+0000","last_clean":"2024-09-20T18:56:40.691415+0000","last_became_active":"2024-09-20T18:55:28.824355+0000","last_became_peered":"2024-09-20T18:55:28.824355+0000","last_unstale":"2024-09-20T18:56:40.691415+0000","last_undegraded":"2024-09-20T18:56:40.691415+0000","last_fullsized":"2024-09-20T18:56:40.691415+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:46.329547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:46.329547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":318,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.697286+0000","last_change":"2024-09-20T18:55:28.825607+0000","last_active":"2024-09-20T18:56:40.697286+0000","last_peered":"2024-09-20T18:56:40.697286+0000","last_clean":"2024-09-20T18:56:40.697286+0000","last_became_active":"2024-09-20T18:55:28.825296+0000","last_became_peered":"2024-09-20T18:55:28.825296+0000","last_unstale":"2024-09-20T18:56:40.697286+0000","last_undegraded":"2024-09-20T18:56:40.697286+0000","last_fullsized":"2024-09-20T18:56:40.697286+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:43.118805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:43.118805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":355,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.696580+0000","last_change":"2024-09-20T18:55:28.836789+0000","last_active":"2024-09-20T18:56:40.696580+0000","last_peered":"2024-09-20T18:56:40.696580+0000","last_clean":"2024-09-20T18:56:40.696580+0000","last_became_active":"2024-09-20T18:55:28.824579+0000","last_became_peered":"2024-09-20T18:55:28.824579+0000","last_unstale":"2024-09-20T18:56:40.696580+0000","last_undegraded":"2024-09-20T18:56:40.696580+0000","last_fullsized":"2024-09-20T18:56:40.696580+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-20T18:55:20.282193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:20.282193+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":338,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.699190+0000","last_change":"2024-09-20T18:54:55.900487+0000","last_active":"2024-09-20T18:56:40.699190+0000","last_peered":"2024-09-20T18:56:40.699190+0000","last_clean":"2024-09-20T18:56:40.699190+0000","last_became_active":"2024-09-20T18:49:45.940988+0000","last_became_peered":"2024-09-20T18:49:45.940988+0000","last_unstale":"2024-09-20T18:56:40.699190+0000","last_undegraded":"2024-09-20T18:56:40.699190+0000","last_fullsized":"2024-09-20T18:56:40.699190+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:55.899611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:55.899611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":69,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.694476+0000","last_change":"2024-09-20T18:55:28.832864+0000","last_active":"2024-09-20T18:56:40.694476+0000","last_peered":"2024-09-20T18:56:40.694476+0000","last_clean":"2024-09-20T18:56:40.694476+0000","last_became_active":"2024-09-20T18:55:28.832534+0000","last_became_peered":"2024-09-20T18:55:28.832534+0000","last_unstale":"2024-09-20T18:56:40.694476+0000","last_undegraded":"2024-09-20T18:56:40.694476+0000","last_fullsized":"2024-09-20T18:56:40.694476+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:52.635138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:52.635138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":310,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.692079+0000","last_change":"2024-09-20T18:51:09.761331+0000","last_active":"2024-09-20T18:56:40.692079+0000","last_peered":"2024-09-20T18:56:40.692079+0000","last_clean":"2024-09-20T18:56:40.692079+0000","last_became_active":"2024-09-20T18:49:51.968662+0000","last_became_peered":"2024-09-20T18:49:51.968662+0000","last_unstale":"2024-09-20T18:56:40.692079+0000","last_undegraded":"2024-09-20T18:56:40.692079+0000","last_fullsized":"2024-09-20T18:56:40.692079+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:09.761283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:09.761283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":303,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.691473+0000","last_change":"2024-09-20T18:49:51.968476+0000","last_active":"2024-09-20T18:56:40.691473+0000","last_peered":"2024-09-20T18:56:40.691473+0000","last_clean":"2024-09-20T18:56:40.691473+0000","last_became_active":"2024-09-20T18:49:51.968216+0000","last_became_peered":"2024-09-20T18:49:51.968216+0000","last_unstale":"2024-09-20T18:56:40.691473+0000","last_undegraded":"2024-09-20T18:56:40.691473+0000","last_fullsized":"2024-09-20T18:56:40.691473+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":331,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.696773+0000","last_change":"2024-09-20T18:55:28.295445+0000","last_active":"2024-09-20T18:56:40.696773+0000","last_peered":"2024-09-20T18:56:40.696773+0000","last_clean":"2024-09-20T18:56:40.696773+0000","last_became_active":"2024-09-20T18:49:45.937992+0000","last_became_peered":"2024-09-20T18:49:45.937992+0000","last_unstale":"2024-09-20T18:56:40.696773+0000","last_undegraded":"2024-09-20T18:56:40.696773+0000","last_fullsized":"2024-09-20T18:56:40.696773+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:28.295399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:28.295399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":120,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.690506+0000","last_change":"2024-09-20T18:55:28.852468+0000","last_active":"2024-09-20T18:56:40.690506+0000","last_peered":"2024-09-20T18:56:40.690506+0000","last_clean":"2024-09-20T18:56:40.690506+0000","last_became_active":"2024-09-20T18:55:28.834675+0000","last_became_peered":"2024-09-20T18:55:28.834675+0000","last_unstale":"2024-09-20T18:56:40.690506+0000","last_undegraded":"2024-09-20T18:56:40.690506+0000","last_fullsized":"2024-09-20T18:56:40.690506+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:54:44.719676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.719676+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":12,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":315,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T18:56:40.702271+0000","last_change":"2024-09-20T18:51:28.120100+0000","last_active":"2024-09-20T18:56:40.702271+0000","last_peered":"2024-09-20T18:56:40.702271+0000","last_clean":"2024-09-20T18:56:40.702271+0000","last_became_active":"2024-09-20T18:49:49.957582+0000","last_became_peered":"2024-09-20T18:49:49.957582+0000","last_unstale":"2024-09-20T18:56:40.702271+0000","last_undegraded":"2024-09-20T18:56:40.702271+0000","last_fullsized":"2024-09-20T18:56:40.702271+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:28.119694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:28.119694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":5625872408,"num_objects":85845,"num_object_clones":0,"num_object_copies":257535,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":713,"num_objects_misplaced":8437,"num_objects_unfound":0,"num_objects_dirty":85845,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":146333,"num_write_kb":7429633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18987,"num_bytes_recovered":1244332032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18108002304,"data_stored":18107990088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74166,"ondisk_log_size":74166,"up":48,"acting":45,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":67,"ondisk_log_size":67,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":695,"num_read_kb":452,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":1474,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":12140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":394,"ondisk_log_size":394,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":5275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":415,"num_read_kb":410,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266776,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1918148,"kb_used_data":1734260,"kb_used_omap":1,"kb_used_meta":183870,"kb_avail":91827004,"statfs":{"total":95995035648,"available":94030852096,"internally_reserved":0,"allocated":1775882240,"data_stored":1775239973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":188283338},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":0,"seq":193273528415,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757348,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3004476,"kb_used_data":2752556,"kb_used_omap":1,"kb_used_meta":251902,"kb_avail":90740676,"statfs":{"total":95995035648,"available":92918452224,"internally_reserved":0,"allocated":2818617344,"data_stored":2817982752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":257948101},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,0,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986281,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1924880,"kb_used_data":1745152,"kb_used_omap":1,"kb_used_meta":179710,"kb_avail":91820272,"statfs":{"total":95995035648,"available":94023958528,"internally_reserved":0,"allocated":1787035648,"data_stored":1786381584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":184023493},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182512,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2625296,"kb_used_data":2314816,"kb_used_omap":1,"kb_used_meta":310206,"kb_avail":91119856,"statfs":{"total":95995035648,"available":93306732544,"internally_reserved":0,"allocated":2370371584,"data_stored":2369716474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":317651399},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411445,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3113652,"kb_used_data":2797796,"kb_used_omap":1,"kb_used_meta":315774,"kb_avail":90631500,"statfs":{"total":95995035648,"available":92806656000,"internally_reserved":0,"allocated":2864943104,"data_stored":2864316953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":323353029},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574971,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2671528,"kb_used_data":2423640,"kb_used_omap":1,"kb_used_meta":247742,"kb_avail":91073624,"statfs":{"total":95995035648,"available":93259390976,"internally_reserved":0,"allocated":2481807360,"data_stored":2481130101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":253688261},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,1,3,4,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738497,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2581008,"kb_used_data":2260992,"kb_used_omap":1,"kb_used_meta":319934,"kb_avail":91164144,"statfs":{"total":95995035648,"available":93352083456,"internally_reserved":0,"allocated":2315255808,"data_stored":2314601318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":327612869},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2300575744,"data_stored":2300575744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2465398784,"data_stored":2465398784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2860716032,"data_stored":2860711960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2350645248,"data_stored":2350645248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1778253824,"data_stored":1778253824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2807107584,"data_stored":2807103512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1773404160,"data_stored":1773404160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1771900928,"data_stored":1771896856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T18:56:51.865 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:51.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:51.866 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:51.967 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:56:52.008 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:52.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:52.010 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:56:52.111 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:52.143 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:52.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:52.144 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:52.245 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:52.270 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:52 smithi142 ceph-mon[32508]: from='client.25265 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:56:52.270 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:52 smithi142 ceph-mon[32508]: pgmap v338: 148 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 144 active+clean; 5.2 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 888 op/s; 713/258132 objects degraded (0.276%); 8437/258132 objects misplaced (3.268%); 14 MiB/s, 223 objects/s recovering 2024-09-20T18:56:52.270 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:52 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2446957910' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:56:52.272 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:52.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:52.273 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:52.300 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:52 smithi083 ceph-mon[29421]: from='client.25265 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:56:52.300 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:52 smithi083 ceph-mon[29421]: pgmap v338: 148 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 144 active+clean; 5.2 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 888 op/s; 713/258132 objects degraded (0.276%); 8437/258132 objects misplaced (3.268%); 14 MiB/s, 223 objects/s recovering 2024-09-20T18:56:52.300 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:52 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2446957910' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:56:52.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:52 smithi083 ceph-mon[36260]: from='client.25265 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:56:52.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:52 smithi083 ceph-mon[36260]: pgmap v338: 148 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 144 active+clean; 5.2 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 888 op/s; 713/258132 objects degraded (0.276%); 8437/258132 objects misplaced (3.268%); 14 MiB/s, 223 objects/s recovering 2024-09-20T18:56:52.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:52 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2446957910' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:56:52.374 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:56:52.417 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:52.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:52.418 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:56:52.519 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:52.543 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:52.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:52.544 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:52.645 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:56:52.686 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:52.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:52.687 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:56:52.787 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:52.814 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:52.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:52.815 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:52.916 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:52.917 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:56:52.940 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:52.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:52.941 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:53.042 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:53.067 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:53.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:53.068 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:53.113 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-20T18:56:53.170 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:56:53.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:53.214 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:53.214 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:56:53.315 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:56:53.355 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:53.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:53.356 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:56:53.457 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:53.480 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:53.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:53.481 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:53.582 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:53.605 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:53.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:53.606 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:53.684 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:56:53.707 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:53.733 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:53.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:53.735 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:53.835 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:56:53.883 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:53.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:53.884 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:56:53.985 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:54.010 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:54.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:54.011 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:54.020 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-20T18:56:54.112 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:54.139 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:54.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:54.140 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:54.241 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:54.266 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:54.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:54.267 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:54.300 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:54 smithi083 ceph-mon[29421]: pgmap v339: 148 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 144 active+clean; 5.0 GiB data, 16 GiB used, 609 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 869 op/s; 866/245421 objects degraded (0.353%); 7041/245421 objects misplaced (2.869%); 14 MiB/s, 223 objects/s recovering 2024-09-20T18:56:54.300 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:54 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1091869420' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:56:54.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:54 smithi083 ceph-mon[36260]: pgmap v339: 148 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 144 active+clean; 5.0 GiB data, 16 GiB used, 609 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 869 op/s; 866/245421 objects degraded (0.353%); 7041/245421 objects misplaced (2.869%); 14 MiB/s, 223 objects/s recovering 2024-09-20T18:56:54.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:54 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1091869420' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:56:54.368 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:54.393 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:54.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:54.394 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:54.494 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:54.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:54 smithi142 ceph-mon[32508]: pgmap v339: 148 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 144 active+clean; 5.0 GiB data, 16 GiB used, 609 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 869 op/s; 866/245421 objects degraded (0.353%); 7041/245421 objects misplaced (2.869%); 14 MiB/s, 223 objects/s recovering 2024-09-20T18:56:54.520 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:54 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1091869420' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:56:54.523 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:54.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:54.524 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:54.625 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:56:54.658 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:54.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:54.659 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:56:54.760 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:54.789 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:54.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:54.790 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:54.826 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T18:56:54.849 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T18:56:54.890 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:54.917 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:54.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:54.918 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:55.019 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:55.044 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:55.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:55.045 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:55.146 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:55.170 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:55.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:55.171 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:55.272 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:56:55.273 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:55 smithi083 ceph-mon[29421]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:56:55.273 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:55 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1091869420' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T18:56:55.273 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:55 smithi083 ceph-mon[29421]: osdmap e212: 8 total, 8 up, 7 in 2024-09-20T18:56:55.277 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:55 smithi083 ceph-mon[36260]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:56:55.277 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:55 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1091869420' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T18:56:55.278 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:55 smithi083 ceph-mon[36260]: osdmap e212: 8 total, 8 up, 7 in 2024-09-20T18:56:55.332 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:55.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:55.334 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:56:55.434 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:56:55.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:55.473 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:55.474 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:56:55.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:55 smithi142 ceph-mon[32508]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:56:55.520 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:55 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1091869420' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T18:56:55.520 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:55 smithi142 ceph-mon[32508]: osdmap e212: 8 total, 8 up, 7 in 2024-09-20T18:56:55.574 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:55.599 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:55.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:55.600 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:55.700 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:55.726 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:55.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:55.727 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:55.828 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:55.849 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:55.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:55.850 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:55.951 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:56:55.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:55.992 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:55.993 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:56:56.093 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:56:56.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:56.115 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:56.115 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:56:56.216 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:56.255 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:56.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:56.256 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:56.299 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:56 smithi083 ceph-mon[29421]: pgmap v341: 148 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 144 active+clean; 5.0 GiB data, 16 GiB used, 609 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 529 op/s; 866/245421 objects degraded (0.353%); 7041/245421 objects misplaced (2.869%); 9.3 MiB/s, 148 objects/s recovering 2024-09-20T18:56:56.300 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:56 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 866/245421 objects degraded (0.353%), 1 pg degraded, 3 pgs undersized (PG_DEGRADED) 2024-09-20T18:56:56.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:56 smithi083 ceph-mon[36260]: pgmap v341: 148 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 144 active+clean; 5.0 GiB data, 16 GiB used, 609 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 529 op/s; 866/245421 objects degraded (0.353%); 7041/245421 objects misplaced (2.869%); 9.3 MiB/s, 148 objects/s recovering 2024-09-20T18:56:56.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:56 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 866/245421 objects degraded (0.353%), 1 pg degraded, 3 pgs undersized (PG_DEGRADED) 2024-09-20T18:56:56.357 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:56.394 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:56.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:56.395 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:56.496 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:56:56.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:56 smithi142 ceph-mon[32508]: pgmap v341: 148 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 144 active+clean; 5.0 GiB data, 16 GiB used, 609 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 529 op/s; 866/245421 objects degraded (0.353%); 7041/245421 objects misplaced (2.869%); 9.3 MiB/s, 148 objects/s recovering 2024-09-20T18:56:56.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:56 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 866/245421 objects degraded (0.353%), 1 pg degraded, 3 pgs undersized (PG_DEGRADED) 2024-09-20T18:56:56.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:56.539 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:56.540 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:56:56.640 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:56.678 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:56.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:56.679 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:56.780 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:56.823 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:56.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:56.827 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:56.927 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:56.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:56.950 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:56.950 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:57.051 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:57.075 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:57.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:57.076 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:57.177 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:57.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:57.218 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:57.218 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:57.270 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:56:56 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:56:57.319 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:57.347 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:57.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:57.348 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:57.449 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:57.481 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:57.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:57.482 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:57.583 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:57.609 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:57.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:57.610 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:57.711 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:56:57.749 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:57.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:57.750 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:56:57.851 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:56:57.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:57.886 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:57.886 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:56:57.987 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:58.013 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:58.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:58.014 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:56:58.115 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:56:58.153 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:58.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:58.154 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:56:58.254 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:58.291 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:58.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:58.292 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:58.299 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:58 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:56:58.300 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:56:58 smithi083 ceph-mon[29421]: pgmap v342: 148 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 144 active+clean; 4.7 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 935 op/s; 1012/233064 objects degraded (0.434%); 5783/233064 objects misplaced (2.481%); 16 MiB/s, 254 objects/s recovering 2024-09-20T18:56:58.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:58 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:56:58.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:56:58 smithi083 ceph-mon[36260]: pgmap v342: 148 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 144 active+clean; 4.7 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 935 op/s; 1012/233064 objects degraded (0.434%); 5783/233064 objects misplaced (2.481%); 16 MiB/s, 254 objects/s recovering 2024-09-20T18:56:58.393 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:58.418 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:58.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:58.419 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:58.519 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:58.520 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:58 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:56:58.520 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:56:58 smithi142 ceph-mon[32508]: pgmap v342: 148 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 144 active+clean; 4.7 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 935 op/s; 1012/233064 objects degraded (0.434%); 5783/233064 objects misplaced (2.481%); 16 MiB/s, 254 objects/s recovering 2024-09-20T18:56:58.549 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:58.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:58.550 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:58.651 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:56:58.678 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:58.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:58.679 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:56:58.780 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:56:58.817 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:58.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:58.818 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:56:58.919 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:56:58.956 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:58.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:58.957 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:56:59.058 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:59.083 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:59.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:59.084 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:59.185 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:59.197 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T18:56:59.197 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-20T18:56:59.197 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:59.197 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.24955.0:157534 8.5 e213/182 8:a7288f40:::benchmark_data_smithi083_97422_object41443:head v 213'10126, mlcod=0'0)", 2024-09-20T18:56:59.197 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.174911+0000", 2024-09-20T18:56:59.197 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0088478529999999993, 2024-09-20T18:56:59.197 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0088755819999999999, 2024-09-20T18:56:59.197 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:56:59.197 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:56:59.197 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:56:59.197 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:56:59.198 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:59.198 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:56:59.198 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.174911+0000", 2024-09-20T18:56:59.198 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:59.198 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:59.198 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:59.198 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:56:59.198 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.174911+0000", 2024-09-20T18:56:59.198 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:59.198 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:59.198 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:59.198 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:56:59.198 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.174912+0000", 2024-09-20T18:56:59.198 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3499999999999995e-07 2024-09-20T18:56:59.199 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:59.199 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:59.199 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:56:59.199 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.174918+0000", 2024-09-20T18:56:59.199 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2650000000000002e-06 2024-09-20T18:56:59.199 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:59.199 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:59.199 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:56:59.199 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.174929+0000", 2024-09-20T18:56:59.199 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1223000000000001e-05 2024-09-20T18:56:59.199 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:59.199 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:59.199 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:56:59.199 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.174936+0000", 2024-09-20T18:56:59.199 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1169999999999998e-06 2024-09-20T18:56:59.200 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:59.200 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:59.200 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:56:59.200 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.177194+0000", 2024-09-20T18:56:59.200 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0022580970000000001 2024-09-20T18:56:59.200 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:59.200 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:59.200 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:56:59.200 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.177216+0000", 2024-09-20T18:56:59.200 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.156e-05 2024-09-20T18:56:59.200 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:59.200 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:56:59.200 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:59.200 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:59.201 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:59.201 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "MOSDPGPushReply(8.6 213/182 [PushReplyOp(8:6bb980c1:::benchmark_data_smithi083_97422_object41459:head)])", 2024-09-20T18:56:59.201 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.178497+0000", 2024-09-20T18:56:59.201 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.005261155, 2024-09-20T18:56:59.201 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.005339964, 2024-09-20T18:56:59.201 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:56:59.201 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:56:59.201 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-20T18:56:59.201 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:56:59.201 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:59.201 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:56:59.201 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.178497+0000", 2024-09-20T18:56:59.201 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:59.201 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:59.202 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:59.202 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:56:59.202 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.178497+0000", 2024-09-20T18:56:59.202 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:59.202 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:59.202 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:59.202 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:56:59.202 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.178498+0000", 2024-09-20T18:56:59.202 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4099999999999997e-07 2024-09-20T18:56:59.202 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:59.202 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:59.202 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:56:59.202 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.178506+0000", 2024-09-20T18:56:59.202 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3919999999999997e-06 2024-09-20T18:56:59.203 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:59.203 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:59.203 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:56:59.203 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.178518+0000", 2024-09-20T18:56:59.203 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2027e-05 2024-09-20T18:56:59.203 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:59.203 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:59.203 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:56:59.203 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.178527+0000", 2024-09-20T18:56:59.203 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4720000000000001e-06 2024-09-20T18:56:59.203 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:59.203 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:59.203 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:56:59.203 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.178622+0000", 2024-09-20T18:56:59.203 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4246999999999994e-05 2024-09-20T18:56:59.204 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:59.204 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:56:59.204 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:59.204 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:59.204 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:59.204 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.24955.0:157539 8.5 e213/182 8:a58e3e70:::benchmark_data_smithi083_97422_object41448:head v 213'10127, mlcod=0'0)", 2024-09-20T18:56:59.204 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.179052+0000", 2024-09-20T18:56:59.204 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0047068079999999998, 2024-09-20T18:56:59.204 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0048342460000000004, 2024-09-20T18:56:59.204 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:56:59.204 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:56:59.204 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:56:59.204 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:56:59.204 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:59.204 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:56:59.205 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.179052+0000", 2024-09-20T18:56:59.205 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:59.205 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:59.205 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:59.205 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:56:59.205 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.179052+0000", 2024-09-20T18:56:59.205 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:56:59.205 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:59.205 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:59.205 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:56:59.205 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.179053+0000", 2024-09-20T18:56:59.205 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.018e-06 2024-09-20T18:56:59.205 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:59.205 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:59.205 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:56:59.206 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.179060+0000", 2024-09-20T18:56:59.206 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5440000000000001e-06 2024-09-20T18:56:59.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:59.206 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:59.206 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:56:59.206 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.179072+0000", 2024-09-20T18:56:59.206 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1416999999999999e-05 2024-09-20T18:56:59.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:59.206 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:59.206 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:56:59.206 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.179079+0000", 2024-09-20T18:56:59.206 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3499999999999999e-06 2024-09-20T18:56:59.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:59.206 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:59.207 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:56:59.207 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.181371+0000", 2024-09-20T18:56:59.207 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002291651 2024-09-20T18:56:59.207 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:56:59.207 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:56:59.207 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:56:59.207 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.181393+0000", 2024-09-20T18:56:59.207 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2469000000000001e-05 2024-09-20T18:56:59.207 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:59.207 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:56:59.207 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:59.207 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:56:59.207 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T18:56:59.208 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 3 2024-09-20T18:56:59.208 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T18:56:59.233 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:59.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:59.234 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:59.335 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:56:59.358 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:59.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:59.359 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:56:59.460 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:56:59.481 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:59.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:59.482 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:56:59.583 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:56:59.583 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:56:59.584 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:56:59.613 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:59.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:59.614 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:56:59.715 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:56:59.738 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:59.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:59.739 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:56:59.840 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:56:59.864 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:56:59.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:59.865 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:56:59.965 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:56:59.987 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:56:59.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:56:59.988 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:00.089 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:00.094 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_blocked_ops 2024-09-20T18:57:00.127 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:00.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:00.128 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:00.228 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:00.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:00.281 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:00.281 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:00.382 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:00.425 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:00 smithi083 ceph-mon[36260]: pgmap v343: 148 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 144 active+clean; 4.7 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 836 op/s; 1012/233064 objects degraded (0.434%); 5783/233064 objects misplaced (2.481%); 14 MiB/s, 227 objects/s recovering 2024-09-20T18:57:00.425 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:00 smithi083 ceph-mon[36260]: osdmap e213: 8 total, 8 up, 7 in 2024-09-20T18:57:00.425 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:00 smithi083 ceph-mon[29421]: pgmap v343: 148 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 144 active+clean; 4.7 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 836 op/s; 1012/233064 objects degraded (0.434%); 5783/233064 objects misplaced (2.481%); 14 MiB/s, 227 objects/s recovering 2024-09-20T18:57:00.425 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:00 smithi083 ceph-mon[29421]: osdmap e213: 8 total, 8 up, 7 in 2024-09-20T18:57:00.446 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T18:57:00.467 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:00.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:00.468 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:00.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:00 smithi142 ceph-mon[32508]: pgmap v343: 148 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 144 active+clean; 4.7 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 836 op/s; 1012/233064 objects degraded (0.434%); 5783/233064 objects misplaced (2.481%); 14 MiB/s, 227 objects/s recovering 2024-09-20T18:57:00.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:00 smithi142 ceph-mon[32508]: osdmap e213: 8 total, 8 up, 7 in 2024-09-20T18:57:00.569 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:00.592 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:00.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:00.593 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:00.694 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:57:00.721 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:00.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:00.722 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:57:00.822 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:00.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:00.848 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:00.849 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:00.949 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:57:00.952 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:57:00 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:57:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:57:00.972 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:00.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:00.973 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:57:01.074 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:01.097 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:01.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:01.100 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:01.147 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:57:01.147 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:57:01.201 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:57:01.204 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:57:01.204 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:57:01.205 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:01 smithi083 ceph-mon[29421]: osdmap e214: 8 total, 8 up, 7 in 2024-09-20T18:57:01.205 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:01 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:57:01.205 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:01 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:57:01.205 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:01 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 1012/233064 objects degraded (0.434%), 1 pg degraded, 3 pgs undersized (PG_DEGRADED) 2024-09-20T18:57:01.205 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:01 smithi083 ceph-mon[36260]: osdmap e214: 8 total, 8 up, 7 in 2024-09-20T18:57:01.205 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:01 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:57:01.205 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:01 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:57:01.205 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:01 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 1012/233064 objects degraded (0.434%), 1 pg degraded, 3 pgs undersized (PG_DEGRADED) 2024-09-20T18:57:01.224 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:01.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:01.225 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:57:01.240 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:57:01.240 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:57:01.279 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:57:01.279 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:57:01.298 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:57:01.312 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:57:01.322 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:57:01.322 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:57:01.326 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:01.350 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:01.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:01.351 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:01.364 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:57:01.364 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:57:01.377 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:57:01.386 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-20T18:57:01.400 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:57:01.400 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:57:01.439 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:57:01.439 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:57:01.452 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:01.496 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:01.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:01.496 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:01.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:01 smithi142 ceph-mon[32508]: osdmap e214: 8 total, 8 up, 7 in 2024-09-20T18:57:01.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:01 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:57:01.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:01 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:57:01.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:01 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 1012/233064 objects degraded (0.434%), 1 pg degraded, 3 pgs undersized (PG_DEGRADED) 2024-09-20T18:57:01.597 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:01.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:01.635 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:01.635 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:01.736 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:01.780 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:01.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:01.782 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:01.882 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:01.915 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:01.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:01.916 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:02.017 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:02.059 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:02.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:02.060 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:02.160 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:02.190 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:02.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:02.191 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:02.292 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:02.300 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:02 smithi083 ceph-mon[29421]: pgmap v346: 148 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 144 active+clean; 4.7 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 597 op/s; 1012/233064 objects degraded (0.434%); 5783/233064 objects misplaced (2.481%); 9.9 MiB/s, 157 objects/s recovering 2024-09-20T18:57:02.300 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:02 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1897038502' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:57:02.300 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:02 smithi083 ceph-mon[29421]: from='client.25144 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:57:02.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:02 smithi083 ceph-mon[36260]: pgmap v346: 148 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 144 active+clean; 4.7 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 597 op/s; 1012/233064 objects degraded (0.434%); 5783/233064 objects misplaced (2.481%); 9.9 MiB/s, 157 objects/s recovering 2024-09-20T18:57:02.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:02 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1897038502' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:57:02.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:02 smithi083 ceph-mon[36260]: from='client.25144 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:57:02.318 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:02.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:02.319 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:02.419 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:02.445 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:02.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:02.446 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:02.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:02 smithi142 ceph-mon[32508]: pgmap v346: 148 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 144 active+clean; 4.7 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 597 op/s; 1012/233064 objects degraded (0.434%); 5783/233064 objects misplaced (2.481%); 9.9 MiB/s, 157 objects/s recovering 2024-09-20T18:57:02.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:02 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1897038502' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:57:02.520 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:02 smithi142 ceph-mon[32508]: from='client.25144 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:57:02.547 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:02.569 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:02.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:02.570 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:02.671 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:02.705 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:02.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:02.706 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:02.807 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:02.834 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:02.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:02.835 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:02.936 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:02.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:02.975 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:02.975 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:03.076 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:03.116 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:03.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:03.117 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:03.218 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:03.260 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:03.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:03.261 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:03.362 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:03.387 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:03.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:03.387 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:03.488 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:57:03.512 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:03.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:03.513 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:57:03.613 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:03.639 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:03.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:03.640 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:03.741 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:03.777 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:03.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:03.778 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:03.879 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:03.918 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:03.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:03.919 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:04.020 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:04.043 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:04.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:04.044 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:04.145 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:04.171 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:04.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:04.172 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:04.272 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:04.308 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:04.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:04.309 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:04.409 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:57:04.436 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:04.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:04.437 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:57:04.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:04 smithi142 ceph-mon[32508]: pgmap v347: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 145 active+clean; 4.5 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.01k op/s; 1083/221190 objects degraded (0.490%); 4445/221190 objects misplaced (2.010%); 18 MiB/s, 293 objects/s recovering 2024-09-20T18:57:04.538 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:04.550 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:04 smithi083 ceph-mon[29421]: pgmap v347: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 145 active+clean; 4.5 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.01k op/s; 1083/221190 objects degraded (0.490%); 4445/221190 objects misplaced (2.010%); 18 MiB/s, 293 objects/s recovering 2024-09-20T18:57:04.550 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:04 smithi083 ceph-mon[36260]: pgmap v347: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 145 active+clean; 4.5 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.01k op/s; 1083/221190 objects degraded (0.490%); 4445/221190 objects misplaced (2.010%); 18 MiB/s, 293 objects/s recovering 2024-09-20T18:57:04.577 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:04.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:04.578 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:04.679 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:04.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:04.734 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:04.734 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:04.836 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:04.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:04.879 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:04.880 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:04.980 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:05.005 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:05.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:05.006 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:05.107 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:05.149 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:05.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:05.150 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:05.251 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:05.276 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:05.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:05.278 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:05.379 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:05.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:05.420 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:05.420 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:05.521 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:57:05.543 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:05.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:05.544 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:57:05.549 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2024-09-20T18:57:00.918320+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5078850099,"num_objects":77688,"num_object_clones":0,"num_object_copies":233064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1012,"num_objects_misplaced":5783,"num_objects_unfound":0,"num_objects_dirty":77688,"num_whiteouts":0,"num_read":1276,"num_read_kb":1133,"num_write":155238,"num_write_kb":7431306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21359,"num_bytes_recovered":1394740169,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74778,"ondisk_log_size":74778,"up":444,"acting":441,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":16405692,"kb_used_data":14500272,"kb_used_omap":10,"kb_used_meta":1904949,"kb_avail":639810372,"statfs":{"total":671965249536,"available":655165820928,"internally_reserved":0,"allocated":14848278528,"data_stored":14844919979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11158,"internal_metadata":1950667882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,3,3,6,7,1,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-269942784,"num_objects":-4119,"num_object_clones":0,"num_object_copies":-12357,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":146,"num_objects_misplaced":-1258,"num_objects_unfound":0,"num_objects_dirty":-4119,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4119,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1089,"num_bytes_recovered":71368704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.909315"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":343,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.025007+0000","last_change":"2024-09-20T18:55:28.037494+0000","last_active":"2024-09-20T18:56:54.025007+0000","last_peered":"2024-09-20T18:56:54.025007+0000","last_clean":"2024-09-20T18:56:54.025007+0000","last_became_active":"2024-09-20T18:49:45.943201+0000","last_became_peered":"2024-09-20T18:49:45.943201+0000","last_unstale":"2024-09-20T18:56:54.025007+0000","last_undegraded":"2024-09-20T18:56:54.025007+0000","last_fullsized":"2024-09-20T18:56:54.025007+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:28.037449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:28.037449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":389,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.007278+0000","last_change":"2024-09-20T18:55:28.838729+0000","last_active":"2024-09-20T18:56:54.007278+0000","last_peered":"2024-09-20T18:56:54.007278+0000","last_clean":"2024-09-20T18:56:54.007278+0000","last_became_active":"2024-09-20T18:55:28.819637+0000","last_became_peered":"2024-09-20T18:55:28.819637+0000","last_unstale":"2024-09-20T18:56:54.007278+0000","last_undegraded":"2024-09-20T18:56:54.007278+0000","last_fullsized":"2024-09-20T18:56:54.007278+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:50.231620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:50.231620+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":335,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.033542+0000","last_change":"2024-09-20T18:54:25.361988+0000","last_active":"2024-09-20T18:56:54.033542+0000","last_peered":"2024-09-20T18:56:54.033542+0000","last_clean":"2024-09-20T18:56:54.033542+0000","last_became_active":"2024-09-20T18:49:49.959514+0000","last_became_peered":"2024-09-20T18:49:49.959514+0000","last_unstale":"2024-09-20T18:56:54.033542+0000","last_undegraded":"2024-09-20T18:56:54.033542+0000","last_fullsized":"2024-09-20T18:56:54.033542+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:25.361935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:25.361935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":331,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.007773+0000","last_change":"2024-09-20T18:54:15.221404+0000","last_active":"2024-09-20T18:56:54.007773+0000","last_peered":"2024-09-20T18:56:54.007773+0000","last_clean":"2024-09-20T18:56:54.007773+0000","last_became_active":"2024-09-20T18:49:51.969421+0000","last_became_peered":"2024-09-20T18:49:51.969421+0000","last_unstale":"2024-09-20T18:56:54.007773+0000","last_undegraded":"2024-09-20T18:56:54.007773+0000","last_fullsized":"2024-09-20T18:56:54.007773+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:15.221342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:15.221342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":338,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.017500+0000","last_change":"2024-09-20T18:55:28.822034+0000","last_active":"2024-09-20T18:56:54.017500+0000","last_peered":"2024-09-20T18:56:54.017500+0000","last_clean":"2024-09-20T18:56:54.017500+0000","last_became_active":"2024-09-20T18:55:28.821717+0000","last_became_peered":"2024-09-20T18:55:28.821717+0000","last_unstale":"2024-09-20T18:56:54.017500+0000","last_undegraded":"2024-09-20T18:56:54.017500+0000","last_fullsized":"2024-09-20T18:56:54.017500+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.216188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.216188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":335,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.017393+0000","last_change":"2024-09-20T18:55:28.825144+0000","last_active":"2024-09-20T18:56:54.017393+0000","last_peered":"2024-09-20T18:56:54.017393+0000","last_clean":"2024-09-20T18:56:54.017393+0000","last_became_active":"2024-09-20T18:55:28.824826+0000","last_became_peered":"2024-09-20T18:55:28.824826+0000","last_unstale":"2024-09-20T18:56:54.017393+0000","last_undegraded":"2024-09-20T18:56:54.017393+0000","last_fullsized":"2024-09-20T18:56:54.017393+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:25.208073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:25.208073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":348,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.023466+0000","last_change":"2024-09-20T18:51:07.130137+0000","last_active":"2024-09-20T18:56:54.023466+0000","last_peered":"2024-09-20T18:56:54.023466+0000","last_clean":"2024-09-20T18:56:54.023466+0000","last_became_active":"2024-09-20T18:49:47.952150+0000","last_became_peered":"2024-09-20T18:49:47.952150+0000","last_unstale":"2024-09-20T18:56:54.023466+0000","last_undegraded":"2024-09-20T18:56:54.023466+0000","last_fullsized":"2024-09-20T18:56:54.023466+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:51:07.130062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:07.130062+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":334,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.007250+0000","last_change":"2024-09-20T18:55:28.820753+0000","last_active":"2024-09-20T18:56:54.007250+0000","last_peered":"2024-09-20T18:56:54.007250+0000","last_clean":"2024-09-20T18:56:54.007250+0000","last_became_active":"2024-09-20T18:55:28.820362+0000","last_became_peered":"2024-09-20T18:55:28.820362+0000","last_unstale":"2024-09-20T18:56:54.007250+0000","last_undegraded":"2024-09-20T18:56:54.007250+0000","last_fullsized":"2024-09-20T18:56:54.007250+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:11.246922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:11.246922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"59'1","reported_seq":367,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.031501+0000","last_change":"2024-09-20T18:55:29.340140+0000","last_active":"2024-09-20T18:56:54.031501+0000","last_peered":"2024-09-20T18:56:54.031501+0000","last_clean":"2024-09-20T18:56:54.031501+0000","last_became_active":"2024-09-20T18:55:28.833386+0000","last_became_peered":"2024-09-20T18:55:28.833386+0000","last_unstale":"2024-09-20T18:56:54.031501+0000","last_undegraded":"2024-09-20T18:56:54.031501+0000","last_fullsized":"2024-09-20T18:56:54.031501+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:55:29.340083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:29.340083+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":355,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.008071+0000","last_change":"2024-09-20T18:54:53.241889+0000","last_active":"2024-09-20T18:56:54.008071+0000","last_peered":"2024-09-20T18:56:54.008071+0000","last_clean":"2024-09-20T18:56:54.008071+0000","last_became_active":"2024-09-20T18:49:47.947448+0000","last_became_peered":"2024-09-20T18:49:47.947448+0000","last_unstale":"2024-09-20T18:56:54.008071+0000","last_undegraded":"2024-09-20T18:56:54.008071+0000","last_fullsized":"2024-09-20T18:56:54.008071+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:53.241802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:53.241802+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":326,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.030941+0000","last_change":"2024-09-20T18:52:36.183932+0000","last_active":"2024-09-20T18:56:54.030941+0000","last_peered":"2024-09-20T18:56:54.030941+0000","last_clean":"2024-09-20T18:56:54.030941+0000","last_became_active":"2024-09-20T18:49:49.961188+0000","last_became_peered":"2024-09-20T18:49:49.961188+0000","last_unstale":"2024-09-20T18:56:54.030941+0000","last_undegraded":"2024-09-20T18:56:54.030941+0000","last_fullsized":"2024-09-20T18:56:54.030941+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:36.183875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:36.183875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":306,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.022714+0000","last_change":"2024-09-20T18:49:51.966957+0000","last_active":"2024-09-20T18:56:54.022714+0000","last_peered":"2024-09-20T18:56:54.022714+0000","last_clean":"2024-09-20T18:56:54.022714+0000","last_became_active":"2024-09-20T18:49:51.966715+0000","last_became_peered":"2024-09-20T18:49:51.966715+0000","last_unstale":"2024-09-20T18:56:54.022714+0000","last_undegraded":"2024-09-20T18:56:54.022714+0000","last_fullsized":"2024-09-20T18:56:54.022714+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":334,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.031192+0000","last_change":"2024-09-20T18:55:15.252839+0000","last_active":"2024-09-20T18:56:54.031192+0000","last_peered":"2024-09-20T18:56:54.031192+0000","last_clean":"2024-09-20T18:56:54.031192+0000","last_became_active":"2024-09-20T18:49:45.944703+0000","last_became_peered":"2024-09-20T18:49:45.944703+0000","last_unstale":"2024-09-20T18:56:54.031192+0000","last_undegraded":"2024-09-20T18:56:54.031192+0000","last_fullsized":"2024-09-20T18:56:54.031192+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:15.252782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:15.252782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":359,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.007697+0000","last_change":"2024-09-20T18:51:26.494194+0000","last_active":"2024-09-20T18:56:54.007697+0000","last_peered":"2024-09-20T18:56:54.007697+0000","last_clean":"2024-09-20T18:56:54.007697+0000","last_became_active":"2024-09-20T18:51:23.813399+0000","last_became_peered":"2024-09-20T18:51:23.813399+0000","last_unstale":"2024-09-20T18:56:54.007697+0000","last_undegraded":"2024-09-20T18:56:54.007697+0000","last_fullsized":"2024-09-20T18:56:54.007697+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-20T18:51:26.494118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:26.494118+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":330,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-20T18:57:00.099324+0000","last_change":"2024-09-20T18:52:38.799542+0000","last_active":"2024-09-20T18:57:00.099324+0000","last_peered":"2024-09-20T18:57:00.099324+0000","last_clean":"2024-09-20T18:57:00.099324+0000","last_became_active":"2024-09-20T18:49:49.959696+0000","last_became_peered":"2024-09-20T18:49:49.959696+0000","last_unstale":"2024-09-20T18:57:00.099324+0000","last_undegraded":"2024-09-20T18:57:00.099324+0000","last_fullsized":"2024-09-20T18:57:00.099324+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:38.799487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:38.799487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":322,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.030919+0000","last_change":"2024-09-20T18:53:21.273835+0000","last_active":"2024-09-20T18:56:54.030919+0000","last_peered":"2024-09-20T18:56:54.030919+0000","last_clean":"2024-09-20T18:56:54.030919+0000","last_became_active":"2024-09-20T18:49:51.970663+0000","last_became_peered":"2024-09-20T18:49:51.970663+0000","last_unstale":"2024-09-20T18:56:54.030919+0000","last_undegraded":"2024-09-20T18:56:54.030919+0000","last_fullsized":"2024-09-20T18:56:54.030919+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:21.273788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:21.273788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":331,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.007556+0000","last_change":"2024-09-20T18:55:28.828417+0000","last_active":"2024-09-20T18:56:54.007556+0000","last_peered":"2024-09-20T18:56:54.007556+0000","last_clean":"2024-09-20T18:56:54.007556+0000","last_became_active":"2024-09-20T18:55:28.827435+0000","last_became_peered":"2024-09-20T18:55:28.827435+0000","last_unstale":"2024-09-20T18:56:54.007556+0000","last_undegraded":"2024-09-20T18:56:54.007556+0000","last_fullsized":"2024-09-20T18:56:54.007556+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:53.940191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:53.940191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":334,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.023825+0000","last_change":"2024-09-20T18:54:23.984024+0000","last_active":"2024-09-20T18:56:54.023825+0000","last_peered":"2024-09-20T18:56:54.023825+0000","last_clean":"2024-09-20T18:56:54.023825+0000","last_became_active":"2024-09-20T18:49:45.950480+0000","last_became_peered":"2024-09-20T18:49:45.950480+0000","last_unstale":"2024-09-20T18:56:54.023825+0000","last_undegraded":"2024-09-20T18:56:54.023825+0000","last_fullsized":"2024-09-20T18:56:54.023825+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:23.983958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:23.983958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":357,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.052610+0000","last_change":"2024-09-20T18:53:26.762207+0000","last_active":"2024-09-20T18:56:54.052610+0000","last_peered":"2024-09-20T18:56:54.052610+0000","last_clean":"2024-09-20T18:56:54.052610+0000","last_became_active":"2024-09-20T18:49:47.950939+0000","last_became_peered":"2024-09-20T18:49:47.950939+0000","last_unstale":"2024-09-20T18:56:54.052610+0000","last_undegraded":"2024-09-20T18:56:54.052610+0000","last_fullsized":"2024-09-20T18:56:54.052610+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:53:26.762131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:26.762131+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":319,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.029168+0000","last_change":"2024-09-20T18:51:17.248705+0000","last_active":"2024-09-20T18:56:54.029168+0000","last_peered":"2024-09-20T18:56:54.029168+0000","last_clean":"2024-09-20T18:56:54.029168+0000","last_became_active":"2024-09-20T18:49:49.958518+0000","last_became_peered":"2024-09-20T18:49:49.958518+0000","last_unstale":"2024-09-20T18:56:54.029168+0000","last_undegraded":"2024-09-20T18:56:54.029168+0000","last_fullsized":"2024-09-20T18:56:54.029168+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:17.248634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:17.248634+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":331,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.036511+0000","last_change":"2024-09-20T18:55:28.833468+0000","last_active":"2024-09-20T18:56:54.036511+0000","last_peered":"2024-09-20T18:56:54.036511+0000","last_clean":"2024-09-20T18:56:54.036511+0000","last_became_active":"2024-09-20T18:55:28.833133+0000","last_became_peered":"2024-09-20T18:55:28.833133+0000","last_unstale":"2024-09-20T18:56:54.036511+0000","last_undegraded":"2024-09-20T18:56:54.036511+0000","last_fullsized":"2024-09-20T18:56:54.036511+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:54.259174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:54.259174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"59'1","reported_seq":341,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.008224+0000","last_change":"2024-09-20T18:53:42.840884+0000","last_active":"2024-09-20T18:56:54.008224+0000","last_peered":"2024-09-20T18:56:54.008224+0000","last_clean":"2024-09-20T18:56:54.008224+0000","last_became_active":"2024-09-20T18:49:45.940100+0000","last_became_peered":"2024-09-20T18:49:45.940100+0000","last_unstale":"2024-09-20T18:56:54.008224+0000","last_undegraded":"2024-09-20T18:56:54.008224+0000","last_fullsized":"2024-09-20T18:56:54.008224+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:53:42.840800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:42.840800+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":338,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.017054+0000","last_change":"2024-09-20T18:49:47.951215+0000","last_active":"2024-09-20T18:56:54.017054+0000","last_peered":"2024-09-20T18:56:54.017054+0000","last_clean":"2024-09-20T18:56:54.017054+0000","last_became_active":"2024-09-20T18:49:47.950909+0000","last_became_peered":"2024-09-20T18:49:47.950909+0000","last_unstale":"2024-09-20T18:56:54.017054+0000","last_undegraded":"2024-09-20T18:56:54.017054+0000","last_fullsized":"2024-09-20T18:56:54.017054+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":335,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.007492+0000","last_change":"2024-09-20T18:54:44.239273+0000","last_active":"2024-09-20T18:56:54.007492+0000","last_peered":"2024-09-20T18:56:54.007492+0000","last_clean":"2024-09-20T18:56:54.007492+0000","last_became_active":"2024-09-20T18:49:49.959102+0000","last_became_peered":"2024-09-20T18:49:49.959102+0000","last_unstale":"2024-09-20T18:56:54.007492+0000","last_undegraded":"2024-09-20T18:56:54.007492+0000","last_fullsized":"2024-09-20T18:56:54.007492+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:44.239226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.239226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":327,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-20T18:57:00.100094+0000","last_change":"2024-09-20T18:52:15.826819+0000","last_active":"2024-09-20T18:57:00.100094+0000","last_peered":"2024-09-20T18:57:00.100094+0000","last_clean":"2024-09-20T18:57:00.100094+0000","last_became_active":"2024-09-20T18:49:51.970058+0000","last_became_peered":"2024-09-20T18:49:51.970058+0000","last_unstale":"2024-09-20T18:57:00.100094+0000","last_undegraded":"2024-09-20T18:57:00.100094+0000","last_fullsized":"2024-09-20T18:57:00.100094+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T18:52:15.826638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:15.826638+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":328,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.033622+0000","last_change":"2024-09-20T18:52:47.158384+0000","last_active":"2024-09-20T18:56:54.033622+0000","last_peered":"2024-09-20T18:56:54.033622+0000","last_clean":"2024-09-20T18:56:54.033622+0000","last_became_active":"2024-09-20T18:49:45.933078+0000","last_became_peered":"2024-09-20T18:49:45.933078+0000","last_unstale":"2024-09-20T18:56:54.033622+0000","last_undegraded":"2024-09-20T18:56:54.033622+0000","last_fullsized":"2024-09-20T18:56:54.033622+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:52:47.158260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:47.158260+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":354,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.040965+0000","last_change":"2024-09-20T18:55:19.996522+0000","last_active":"2024-09-20T18:56:54.040965+0000","last_peered":"2024-09-20T18:56:54.040965+0000","last_clean":"2024-09-20T18:56:54.040965+0000","last_became_active":"2024-09-20T18:49:48.500809+0000","last_became_peered":"2024-09-20T18:49:48.500809+0000","last_unstale":"2024-09-20T18:56:54.040965+0000","last_undegraded":"2024-09-20T18:56:54.040965+0000","last_fullsized":"2024-09-20T18:56:54.040965+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:55:19.996328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:19.996328+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"184'9","reported_seq":123,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.007695+0000","last_change":"2024-09-20T18:55:28.845188+0000","last_active":"2024-09-20T18:56:54.007695+0000","last_peered":"2024-09-20T18:56:54.007695+0000","last_clean":"2024-09-20T18:56:54.007695+0000","last_became_active":"2024-09-20T18:55:28.820941+0000","last_became_peered":"2024-09-20T18:55:28.820941+0000","last_unstale":"2024-09-20T18:56:54.007695+0000","last_undegraded":"2024-09-20T18:56:54.007695+0000","last_fullsized":"2024-09-20T18:56:54.007695+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:53:44.524040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:44.524040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3","version":"212'9452","reported_seq":21900,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:56.246488+0000","last_change":"2024-09-20T18:56:25.697520+0000","last_active":"2024-09-20T18:56:56.246488+0000","last_peered":"2024-09-20T18:56:56.246488+0000","last_clean":"2024-09-20T18:56:56.246488+0000","last_became_active":"2024-09-20T18:56:25.687391+0000","last_became_peered":"2024-09-20T18:56:25.687391+0000","last_unstale":"2024-09-20T18:56:56.246488+0000","last_undegraded":"2024-09-20T18:56:56.246488+0000","last_fullsized":"2024-09-20T18:56:56.246488+0000","mapping_epoch":203,"log_start":"184'5172","ondisk_log_start":"184'5172","created":160,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4280,"log_dups_size":0,"ondisk_log_size":4280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":317456384,"num_objects":4844,"num_object_clones":0,"num_object_copies":14532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9452,"num_write_kb":457472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6156,"num_bytes_recovered":403439616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":349,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.057468+0000","last_change":"2024-09-20T18:55:18.581290+0000","last_active":"2024-09-20T18:56:54.057468+0000","last_peered":"2024-09-20T18:56:54.057468+0000","last_clean":"2024-09-20T18:56:54.057468+0000","last_became_active":"2024-09-20T18:49:58.992302+0000","last_became_peered":"2024-09-20T18:49:58.992302+0000","last_unstale":"2024-09-20T18:56:54.057468+0000","last_undegraded":"2024-09-20T18:56:54.057468+0000","last_fullsized":"2024-09-20T18:56:54.057468+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:55:18.581200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:18.581200+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":336,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.035564+0000","last_change":"2024-09-20T18:54:54.288199+0000","last_active":"2024-09-20T18:56:54.035564+0000","last_peered":"2024-09-20T18:56:54.035564+0000","last_clean":"2024-09-20T18:56:54.035564+0000","last_became_active":"2024-09-20T18:49:45.936104+0000","last_became_peered":"2024-09-20T18:49:45.936104+0000","last_unstale":"2024-09-20T18:56:54.035564+0000","last_undegraded":"2024-09-20T18:56:54.035564+0000","last_fullsized":"2024-09-20T18:56:54.035564+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:54.288117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:54.288117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":46,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":425,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.007641+0000","last_change":"2024-09-20T18:54:14.292660+0000","last_active":"2024-09-20T18:56:54.007641+0000","last_peered":"2024-09-20T18:56:54.007641+0000","last_clean":"2024-09-20T18:56:54.007641+0000","last_became_active":"2024-09-20T18:49:49.960697+0000","last_became_peered":"2024-09-20T18:49:49.960697+0000","last_unstale":"2024-09-20T18:56:54.007641+0000","last_undegraded":"2024-09-20T18:56:54.007641+0000","last_fullsized":"2024-09-20T18:56:54.007641+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:54:14.292610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:14.292610+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":34,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":323,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.031401+0000","last_change":"2024-09-20T18:54:10.381171+0000","last_active":"2024-09-20T18:56:54.031401+0000","last_peered":"2024-09-20T18:56:54.031401+0000","last_clean":"2024-09-20T18:56:54.031401+0000","last_became_active":"2024-09-20T18:49:51.971399+0000","last_became_peered":"2024-09-20T18:49:51.971399+0000","last_unstale":"2024-09-20T18:56:54.031401+0000","last_undegraded":"2024-09-20T18:56:54.031401+0000","last_fullsized":"2024-09-20T18:56:54.031401+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:10.381108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:10.381108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.c","version":"212'9582","reported_seq":9698,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:56.561833+0000","last_change":"2024-09-20T18:54:34.311256+0000","last_active":"2024-09-20T18:56:56.561833+0000","last_peered":"2024-09-20T18:56:56.561833+0000","last_clean":"2024-09-20T18:56:56.561833+0000","last_became_active":"2024-09-20T18:54:34.310904+0000","last_became_peered":"2024-09-20T18:54:34.310904+0000","last_unstale":"2024-09-20T18:56:56.561833+0000","last_undegraded":"2024-09-20T18:56:56.561833+0000","last_fullsized":"2024-09-20T18:56:56.561833+0000","mapping_epoch":160,"log_start":"176'4711","ondisk_log_start":"176'4711","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4871,"log_dups_size":0,"ondisk_log_size":4871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":315228160,"num_objects":4810,"num_object_clones":0,"num_object_copies":14430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9582,"num_write_kb":460544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":364,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.035343+0000","last_change":"2024-09-20T18:54:22.077299+0000","last_active":"2024-09-20T18:56:54.035343+0000","last_peered":"2024-09-20T18:56:54.035343+0000","last_clean":"2024-09-20T18:56:54.035343+0000","last_became_active":"2024-09-20T18:49:48.499189+0000","last_became_peered":"2024-09-20T18:49:48.499189+0000","last_unstale":"2024-09-20T18:56:54.035343+0000","last_undegraded":"2024-09-20T18:56:54.035343+0000","last_fullsized":"2024-09-20T18:56:54.035343+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-20T18:54:22.077212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:22.077212+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":328,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.040331+0000","last_change":"2024-09-20T18:54:57.237740+0000","last_active":"2024-09-20T18:56:54.040331+0000","last_peered":"2024-09-20T18:56:54.040331+0000","last_clean":"2024-09-20T18:56:54.040331+0000","last_became_active":"2024-09-20T18:49:45.940544+0000","last_became_peered":"2024-09-20T18:49:45.940544+0000","last_unstale":"2024-09-20T18:56:54.040331+0000","last_undegraded":"2024-09-20T18:56:54.040331+0000","last_fullsized":"2024-09-20T18:56:54.040331+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.237675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.237675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":336,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.007277+0000","last_change":"2024-09-20T18:54:51.229251+0000","last_active":"2024-09-20T18:56:54.007277+0000","last_peered":"2024-09-20T18:56:54.007277+0000","last_clean":"2024-09-20T18:56:54.007277+0000","last_became_active":"2024-09-20T18:49:49.963073+0000","last_became_peered":"2024-09-20T18:49:49.963073+0000","last_unstale":"2024-09-20T18:56:54.007277+0000","last_undegraded":"2024-09-20T18:56:54.007277+0000","last_fullsized":"2024-09-20T18:56:54.007277+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.229201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.229201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":332,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.008004+0000","last_change":"2024-09-20T18:54:55.323937+0000","last_active":"2024-09-20T18:56:54.008004+0000","last_peered":"2024-09-20T18:56:54.008004+0000","last_clean":"2024-09-20T18:56:54.008004+0000","last_became_active":"2024-09-20T18:49:51.968793+0000","last_became_peered":"2024-09-20T18:49:51.968793+0000","last_unstale":"2024-09-20T18:56:54.008004+0000","last_undegraded":"2024-09-20T18:56:54.008004+0000","last_fullsized":"2024-09-20T18:56:54.008004+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:55.323692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:55.323692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"212'9542","reported_seq":9658,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:55.916410+0000","last_change":"2024-09-20T18:54:34.311680+0000","last_active":"2024-09-20T18:56:55.916410+0000","last_peered":"2024-09-20T18:56:55.916410+0000","last_clean":"2024-09-20T18:56:55.916410+0000","last_became_active":"2024-09-20T18:54:34.311241+0000","last_became_peered":"2024-09-20T18:54:34.311241+0000","last_unstale":"2024-09-20T18:56:55.916410+0000","last_undegraded":"2024-09-20T18:56:55.916410+0000","last_fullsized":"2024-09-20T18:56:55.916410+0000","mapping_epoch":160,"log_start":"175'4446","ondisk_log_start":"175'4446","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":5096,"log_dups_size":0,"ondisk_log_size":5096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":316473368,"num_objects":4830,"num_object_clones":0,"num_object_copies":14490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4830,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9542,"num_write_kb":459841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":392,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.007363+0000","last_change":"2024-09-20T18:55:28.853375+0000","last_active":"2024-09-20T18:56:54.007363+0000","last_peered":"2024-09-20T18:56:54.007363+0000","last_clean":"2024-09-20T18:56:54.007363+0000","last_became_active":"2024-09-20T18:55:28.831057+0000","last_became_peered":"2024-09-20T18:55:28.831057+0000","last_unstale":"2024-09-20T18:56:54.007363+0000","last_undegraded":"2024-09-20T18:56:54.007363+0000","last_fullsized":"2024-09-20T18:56:54.007363+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-20T18:54:57.394893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.394893+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":328,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.041212+0000","last_change":"2024-09-20T18:51:27.187060+0000","last_active":"2024-09-20T18:56:54.041212+0000","last_peered":"2024-09-20T18:56:54.041212+0000","last_clean":"2024-09-20T18:56:54.041212+0000","last_became_active":"2024-09-20T18:49:45.940469+0000","last_became_peered":"2024-09-20T18:49:45.940469+0000","last_unstale":"2024-09-20T18:56:54.041212+0000","last_undegraded":"2024-09-20T18:56:54.041212+0000","last_fullsized":"2024-09-20T18:56:54.041212+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:51:27.186991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:27.186991+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"66'8","reported_seq":431,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.035196+0000","last_change":"2024-09-20T18:54:47.223010+0000","last_active":"2024-09-20T18:56:54.035196+0000","last_peered":"2024-09-20T18:56:54.035196+0000","last_clean":"2024-09-20T18:56:54.035196+0000","last_became_active":"2024-09-20T18:49:49.956133+0000","last_became_peered":"2024-09-20T18:49:49.956133+0000","last_unstale":"2024-09-20T18:56:54.035196+0000","last_undegraded":"2024-09-20T18:56:54.035196+0000","last_fullsized":"2024-09-20T18:56:54.035196+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:54:47.222937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:47.222937+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":334,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-20T18:57:00.099543+0000","last_change":"2024-09-20T18:55:28.819147+0000","last_active":"2024-09-20T18:57:00.099543+0000","last_peered":"2024-09-20T18:57:00.099543+0000","last_clean":"2024-09-20T18:57:00.099543+0000","last_became_active":"2024-09-20T18:55:28.818804+0000","last_became_peered":"2024-09-20T18:55:28.818804+0000","last_unstale":"2024-09-20T18:57:00.099543+0000","last_undegraded":"2024-09-20T18:57:00.099543+0000","last_fullsized":"2024-09-20T18:57:00.099543+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:42.880724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:42.880724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"212'9617","reported_seq":9733,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:56.730353+0000","last_change":"2024-09-20T18:54:34.309939+0000","last_active":"2024-09-20T18:56:56.730353+0000","last_peered":"2024-09-20T18:56:56.730353+0000","last_clean":"2024-09-20T18:56:56.730353+0000","last_became_active":"2024-09-20T18:54:34.309630+0000","last_became_peered":"2024-09-20T18:54:34.309630+0000","last_unstale":"2024-09-20T18:56:56.730353+0000","last_undegraded":"2024-09-20T18:56:56.730353+0000","last_fullsized":"2024-09-20T18:56:56.730353+0000","mapping_epoch":160,"log_start":"176'4749","ondisk_log_start":"176'4749","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4868,"log_dups_size":0,"ondisk_log_size":4868,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":313589760,"num_objects":4785,"num_object_clones":0,"num_object_copies":14355,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9617,"num_write_kb":460864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":355,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.029353+0000","last_change":"2024-09-20T18:55:19.541011+0000","last_active":"2024-09-20T18:56:54.029353+0000","last_peered":"2024-09-20T18:56:54.029353+0000","last_clean":"2024-09-20T18:56:54.029353+0000","last_became_active":"2024-09-20T18:49:47.949626+0000","last_became_peered":"2024-09-20T18:49:47.949626+0000","last_unstale":"2024-09-20T18:56:54.029353+0000","last_undegraded":"2024-09-20T18:56:54.029353+0000","last_fullsized":"2024-09-20T18:56:54.029353+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:55:19.540812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:19.540812+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":343,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.031580+0000","last_change":"2024-09-20T18:54:18.326947+0000","last_active":"2024-09-20T18:56:54.031580+0000","last_peered":"2024-09-20T18:56:54.031580+0000","last_clean":"2024-09-20T18:56:54.031580+0000","last_became_active":"2024-09-20T18:49:45.942496+0000","last_became_peered":"2024-09-20T18:49:45.942496+0000","last_unstale":"2024-09-20T18:56:54.031580+0000","last_undegraded":"2024-09-20T18:56:54.031580+0000","last_fullsized":"2024-09-20T18:56:54.031580+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:18.326875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:18.326875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"184'9","reported_seq":433,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.023541+0000","last_change":"2024-09-20T18:55:29.010062+0000","last_active":"2024-09-20T18:56:54.023541+0000","last_peered":"2024-09-20T18:56:54.023541+0000","last_clean":"2024-09-20T18:56:54.023541+0000","last_became_active":"2024-09-20T18:55:28.830643+0000","last_became_peered":"2024-09-20T18:55:28.830643+0000","last_unstale":"2024-09-20T18:56:54.023541+0000","last_undegraded":"2024-09-20T18:56:54.023541+0000","last_fullsized":"2024-09-20T18:56:54.023541+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:29.010015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:29.010015+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":307,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.022859+0000","last_change":"2024-09-20T18:49:51.965733+0000","last_active":"2024-09-20T18:56:54.022859+0000","last_peered":"2024-09-20T18:56:54.022859+0000","last_clean":"2024-09-20T18:56:54.022859+0000","last_became_active":"2024-09-20T18:49:51.964780+0000","last_became_peered":"2024-09-20T18:49:51.964780+0000","last_unstale":"2024-09-20T18:56:54.022859+0000","last_undegraded":"2024-09-20T18:56:54.022859+0000","last_fullsized":"2024-09-20T18:56:54.022859+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"212'9852","reported_seq":9968,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:56.422711+0000","last_change":"2024-09-20T18:54:34.309809+0000","last_active":"2024-09-20T18:56:56.422711+0000","last_peered":"2024-09-20T18:56:56.422711+0000","last_clean":"2024-09-20T18:56:56.422711+0000","last_became_active":"2024-09-20T18:54:34.309231+0000","last_became_peered":"2024-09-20T18:54:34.309231+0000","last_unstale":"2024-09-20T18:56:56.422711+0000","last_undegraded":"2024-09-20T18:56:56.422711+0000","last_fullsized":"2024-09-20T18:56:56.422711+0000","mapping_epoch":160,"log_start":"186'5532","ondisk_log_start":"186'5532","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4320,"log_dups_size":0,"ondisk_log_size":4320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":318636032,"num_objects":4862,"num_object_clones":0,"num_object_copies":14586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9852,"num_write_kb":470848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":367,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.023660+0000","last_change":"2024-09-20T18:54:14.923895+0000","last_active":"2024-09-20T18:56:54.023660+0000","last_peered":"2024-09-20T18:56:54.023660+0000","last_clean":"2024-09-20T18:56:54.023660+0000","last_became_active":"2024-09-20T18:49:48.497736+0000","last_became_peered":"2024-09-20T18:49:48.497736+0000","last_unstale":"2024-09-20T18:56:54.023660+0000","last_undegraded":"2024-09-20T18:56:54.023660+0000","last_fullsized":"2024-09-20T18:56:54.023660+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-20T18:54:14.923682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:14.923682+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":320,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.029415+0000","last_change":"2024-09-20T18:49:45.942589+0000","last_active":"2024-09-20T18:56:54.029415+0000","last_peered":"2024-09-20T18:56:54.029415+0000","last_clean":"2024-09-20T18:56:54.029415+0000","last_became_active":"2024-09-20T18:49:45.942253+0000","last_became_peered":"2024-09-20T18:49:45.942253+0000","last_unstale":"2024-09-20T18:56:54.029415+0000","last_undegraded":"2024-09-20T18:56:54.029415+0000","last_fullsized":"2024-09-20T18:56:54.029415+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":73,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.025296+0000","last_change":"2024-09-20T18:55:28.830954+0000","last_active":"2024-09-20T18:56:54.025296+0000","last_peered":"2024-09-20T18:56:54.025296+0000","last_clean":"2024-09-20T18:56:54.025296+0000","last_became_active":"2024-09-20T18:55:28.830645+0000","last_became_peered":"2024-09-20T18:55:28.830645+0000","last_unstale":"2024-09-20T18:56:54.025296+0000","last_undegraded":"2024-09-20T18:56:54.025296+0000","last_fullsized":"2024-09-20T18:56:54.025296+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:50.610871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:50.610871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":339,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.033512+0000","last_change":"2024-09-20T18:55:28.822362+0000","last_active":"2024-09-20T18:56:54.033512+0000","last_peered":"2024-09-20T18:56:54.033512+0000","last_clean":"2024-09-20T18:56:54.033512+0000","last_became_active":"2024-09-20T18:55:28.821588+0000","last_became_peered":"2024-09-20T18:55:28.821588+0000","last_unstale":"2024-09-20T18:56:54.033512+0000","last_undegraded":"2024-09-20T18:56:54.033512+0000","last_fullsized":"2024-09-20T18:56:54.033512+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.402841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.402841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"212'9723","reported_seq":9839,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:55.893972+0000","last_change":"2024-09-20T18:54:34.311561+0000","last_active":"2024-09-20T18:56:55.893972+0000","last_peered":"2024-09-20T18:56:55.893972+0000","last_clean":"2024-09-20T18:56:55.893972+0000","last_became_active":"2024-09-20T18:54:34.311181+0000","last_became_peered":"2024-09-20T18:54:34.311181+0000","last_unstale":"2024-09-20T18:56:55.893972+0000","last_undegraded":"2024-09-20T18:56:55.893972+0000","last_fullsized":"2024-09-20T18:56:55.893972+0000","mapping_epoch":160,"log_start":"176'4694","ondisk_log_start":"176'4694","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":5029,"log_dups_size":0,"ondisk_log_size":5029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":319488000,"num_objects":4875,"num_object_clones":0,"num_object_copies":14625,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9723,"num_write_kb":467136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"66'28","reported_seq":467,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.029319+0000","last_change":"2024-09-20T18:55:16.527267+0000","last_active":"2024-09-20T18:56:54.029319+0000","last_peered":"2024-09-20T18:56:54.029319+0000","last_clean":"2024-09-20T18:56:54.029319+0000","last_became_active":"2024-09-20T18:49:47.955335+0000","last_became_peered":"2024-09-20T18:49:47.955335+0000","last_unstale":"2024-09-20T18:56:54.029319+0000","last_undegraded":"2024-09-20T18:56:54.029319+0000","last_fullsized":"2024-09-20T18:56:54.029319+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'28","last_scrub_stamp":"2024-09-20T18:55:16.527165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:16.527165+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":327,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.008215+0000","last_change":"2024-09-20T18:55:28.832303+0000","last_active":"2024-09-20T18:56:54.008215+0000","last_peered":"2024-09-20T18:56:54.008215+0000","last_clean":"2024-09-20T18:56:54.008215+0000","last_became_active":"2024-09-20T18:55:28.831574+0000","last_became_peered":"2024-09-20T18:55:28.831574+0000","last_unstale":"2024-09-20T18:56:54.008215+0000","last_undegraded":"2024-09-20T18:56:54.008215+0000","last_fullsized":"2024-09-20T18:56:54.008215+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1],"acting":[4,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":372,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.031226+0000","last_change":"2024-09-20T18:55:28.854615+0000","last_active":"2024-09-20T18:56:54.031226+0000","last_peered":"2024-09-20T18:56:54.031226+0000","last_clean":"2024-09-20T18:56:54.031226+0000","last_became_active":"2024-09-20T18:55:28.837226+0000","last_became_peered":"2024-09-20T18:55:28.837226+0000","last_unstale":"2024-09-20T18:56:54.031226+0000","last_undegraded":"2024-09-20T18:56:54.031226+0000","last_fullsized":"2024-09-20T18:56:54.031226+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-20T18:54:51.227531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.227531+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0012781159999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":335,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.026275+0000","last_change":"2024-09-20T18:55:13.949348+0000","last_active":"2024-09-20T18:56:54.026275+0000","last_peered":"2024-09-20T18:56:54.026275+0000","last_clean":"2024-09-20T18:56:54.026275+0000","last_became_active":"2024-09-20T18:49:49.953971+0000","last_became_peered":"2024-09-20T18:49:49.953971+0000","last_unstale":"2024-09-20T18:56:54.026275+0000","last_undegraded":"2024-09-20T18:56:54.026275+0000","last_fullsized":"2024-09-20T18:56:54.026275+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:13.949269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:13.949269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":308,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.015988+0000","last_change":"2024-09-20T18:49:51.969702+0000","last_active":"2024-09-20T18:56:54.015988+0000","last_peered":"2024-09-20T18:56:54.015988+0000","last_clean":"2024-09-20T18:56:54.015988+0000","last_became_active":"2024-09-20T18:49:51.969400+0000","last_became_peered":"2024-09-20T18:49:51.969400+0000","last_unstale":"2024-09-20T18:56:54.015988+0000","last_undegraded":"2024-09-20T18:56:54.015988+0000","last_fullsized":"2024-09-20T18:56:54.015988+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"212'9795","reported_seq":9911,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:56.464239+0000","last_change":"2024-09-20T18:54:33.926785+0000","last_active":"2024-09-20T18:56:56.464239+0000","last_peered":"2024-09-20T18:56:56.464239+0000","last_clean":"2024-09-20T18:56:56.464239+0000","last_became_active":"2024-09-20T18:54:33.925750+0000","last_became_peered":"2024-09-20T18:54:33.925750+0000","last_unstale":"2024-09-20T18:56:56.464239+0000","last_undegraded":"2024-09-20T18:56:56.464239+0000","last_fullsized":"2024-09-20T18:56:56.464239+0000","mapping_epoch":160,"log_start":"186'5440","ondisk_log_start":"186'5440","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4355,"log_dups_size":0,"ondisk_log_size":4355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":322633728,"num_objects":4923,"num_object_clones":0,"num_object_copies":14769,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9795,"num_write_kb":470976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":367,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.034801+0000","last_change":"2024-09-20T18:55:25.702762+0000","last_active":"2024-09-20T18:56:54.034801+0000","last_peered":"2024-09-20T18:56:54.034801+0000","last_clean":"2024-09-20T18:56:54.034801+0000","last_became_active":"2024-09-20T18:49:47.950904+0000","last_became_peered":"2024-09-20T18:49:47.950904+0000","last_unstale":"2024-09-20T18:56:54.034801+0000","last_undegraded":"2024-09-20T18:56:54.034801+0000","last_fullsized":"2024-09-20T18:56:54.034801+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-20T18:55:25.702596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:25.702596+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":351,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.031048+0000","last_change":"2024-09-20T18:55:28.830832+0000","last_active":"2024-09-20T18:56:54.031048+0000","last_peered":"2024-09-20T18:56:54.031048+0000","last_clean":"2024-09-20T18:56:54.031048+0000","last_became_active":"2024-09-20T18:55:28.830269+0000","last_became_peered":"2024-09-20T18:55:28.830269+0000","last_unstale":"2024-09-20T18:56:54.031048+0000","last_undegraded":"2024-09-20T18:56:54.031048+0000","last_fullsized":"2024-09-20T18:56:54.031048+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:23.359769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:23.359769+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":338,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.050473+0000","last_change":"2024-09-20T18:52:07.718766+0000","last_active":"2024-09-20T18:56:54.050473+0000","last_peered":"2024-09-20T18:56:54.050473+0000","last_clean":"2024-09-20T18:56:54.050473+0000","last_became_active":"2024-09-20T18:49:44.937135+0000","last_became_peered":"2024-09-20T18:49:44.937135+0000","last_unstale":"2024-09-20T18:56:54.050473+0000","last_undegraded":"2024-09-20T18:56:54.050473+0000","last_fullsized":"2024-09-20T18:56:54.050473+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:52:07.718646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:52:07.718646+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.00095558800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":321,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.049912+0000","last_change":"2024-09-20T18:51:59.796450+0000","last_active":"2024-09-20T18:56:54.049912+0000","last_peered":"2024-09-20T18:56:54.049912+0000","last_clean":"2024-09-20T18:56:54.049912+0000","last_became_active":"2024-09-20T18:49:49.954424+0000","last_became_peered":"2024-09-20T18:49:49.954424+0000","last_unstale":"2024-09-20T18:56:54.049912+0000","last_undegraded":"2024-09-20T18:56:54.049912+0000","last_fullsized":"2024-09-20T18:56:54.049912+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:59.796383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:59.796383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":326,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-20T18:57:00.099832+0000","last_change":"2024-09-20T18:55:28.826659+0000","last_active":"2024-09-20T18:57:00.099832+0000","last_peered":"2024-09-20T18:57:00.099832+0000","last_clean":"2024-09-20T18:57:00.099832+0000","last_became_active":"2024-09-20T18:55:28.825812+0000","last_became_peered":"2024-09-20T18:55:28.825812+0000","last_unstale":"2024-09-20T18:57:00.099832+0000","last_undegraded":"2024-09-20T18:57:00.099832+0000","last_fullsized":"2024-09-20T18:57:00.099832+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:39.814259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:39.814259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"212'9786","reported_seq":9902,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:56.260861+0000","last_change":"2024-09-20T18:54:34.311774+0000","last_active":"2024-09-20T18:56:56.260861+0000","last_peered":"2024-09-20T18:56:56.260861+0000","last_clean":"2024-09-20T18:56:56.260861+0000","last_became_active":"2024-09-20T18:54:34.311487+0000","last_became_peered":"2024-09-20T18:54:34.311487+0000","last_unstale":"2024-09-20T18:56:56.260861+0000","last_undegraded":"2024-09-20T18:56:56.260861+0000","last_fullsized":"2024-09-20T18:56:56.260861+0000","mapping_epoch":160,"log_start":"186'5492","ondisk_log_start":"186'5492","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4294,"log_dups_size":0,"ondisk_log_size":4294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":316669952,"num_objects":4832,"num_object_clones":0,"num_object_copies":14496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9786,"num_write_kb":467776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":365,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.041041+0000","last_change":"2024-09-20T18:55:14.951950+0000","last_active":"2024-09-20T18:56:54.041041+0000","last_peered":"2024-09-20T18:56:54.041041+0000","last_clean":"2024-09-20T18:56:54.041041+0000","last_became_active":"2024-09-20T18:49:47.949230+0000","last_became_peered":"2024-09-20T18:49:47.949230+0000","last_unstale":"2024-09-20T18:56:54.041041+0000","last_undegraded":"2024-09-20T18:56:54.041041+0000","last_fullsized":"2024-09-20T18:56:54.041041+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:55:14.951892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:14.951892+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":343,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.024150+0000","last_change":"2024-09-20T18:54:27.905856+0000","last_active":"2024-09-20T18:56:54.024150+0000","last_peered":"2024-09-20T18:56:54.024150+0000","last_clean":"2024-09-20T18:56:54.024150+0000","last_became_active":"2024-09-20T18:49:45.939197+0000","last_became_peered":"2024-09-20T18:49:45.939197+0000","last_unstale":"2024-09-20T18:56:54.024150+0000","last_undegraded":"2024-09-20T18:56:54.024150+0000","last_fullsized":"2024-09-20T18:56:54.024150+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:27.905803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:27.905803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":788,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-20T18:57:00.099368+0000","last_change":"2024-09-20T18:54:26.359887+0000","last_active":"2024-09-20T18:57:00.099368+0000","last_peered":"2024-09-20T18:57:00.099368+0000","last_clean":"2024-09-20T18:57:00.099368+0000","last_became_active":"2024-09-20T18:49:44.936819+0000","last_became_peered":"2024-09-20T18:49:44.936819+0000","last_unstale":"2024-09-20T18:57:00.099368+0000","last_undegraded":"2024-09-20T18:57:00.099368+0000","last_fullsized":"2024-09-20T18:57:00.099368+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-20T18:54:26.359807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:54:26.359807+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0010996199999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":430,"num_read_kb":425,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":327,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.031451+0000","last_change":"2024-09-20T18:54:59.151228+0000","last_active":"2024-09-20T18:56:54.031451+0000","last_peered":"2024-09-20T18:56:54.031451+0000","last_clean":"2024-09-20T18:56:54.031451+0000","last_became_active":"2024-09-20T18:49:49.961066+0000","last_became_peered":"2024-09-20T18:49:49.961066+0000","last_unstale":"2024-09-20T18:56:54.031451+0000","last_undegraded":"2024-09-20T18:56:54.031451+0000","last_fullsized":"2024-09-20T18:56:54.031451+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:59.150063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:59.150063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":332,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.029056+0000","last_change":"2024-09-20T18:55:21.629867+0000","last_active":"2024-09-20T18:56:54.029056+0000","last_peered":"2024-09-20T18:56:54.029056+0000","last_clean":"2024-09-20T18:56:54.029056+0000","last_became_active":"2024-09-20T18:49:51.970343+0000","last_became_peered":"2024-09-20T18:49:51.970343+0000","last_unstale":"2024-09-20T18:56:54.029056+0000","last_undegraded":"2024-09-20T18:56:54.029056+0000","last_fullsized":"2024-09-20T18:56:54.029056+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:21.629800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:21.629800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"212'9703","reported_seq":18126,"reported_epoch":212,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-20T18:56:55.915227+0000","last_change":"2024-09-20T18:55:30.088909+0000","last_active":"2024-09-20T18:56:55.915227+0000","last_peered":"2024-09-20T18:56:55.915227+0000","last_clean":"2024-09-20T18:55:27.745898+0000","last_became_active":"2024-09-20T18:55:29.995624+0000","last_became_peered":"2024-09-20T18:55:29.995624+0000","last_unstale":"2024-09-20T18:56:55.915227+0000","last_undegraded":"2024-09-20T18:56:55.915227+0000","last_fullsized":"2024-09-20T18:55:29.815609+0000","mapping_epoch":182,"log_start":"175'4606","ondisk_log_start":"175'4606","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":5097,"log_dups_size":0,"ondisk_log_size":5097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":316473344,"num_objects":4829,"num_object_clones":0,"num_object_copies":14487,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1054,"num_objects_unfound":0,"num_objects_dirty":4829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9703,"num_write_kb":465024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4152,"num_bytes_recovered":272105472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0],"avail_no_missing":["5","0","6"],"object_location_counts":[{"shards":"0,5","objects":3775},{"shards":"0,5,6","objects":1054}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"184'17","reported_seq":152,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.010025+0000","last_change":"2024-09-20T18:55:28.856602+0000","last_active":"2024-09-20T18:56:54.010025+0000","last_peered":"2024-09-20T18:56:54.010025+0000","last_clean":"2024-09-20T18:56:54.010025+0000","last_became_active":"2024-09-20T18:55:28.831024+0000","last_became_peered":"2024-09-20T18:55:28.831024+0000","last_unstale":"2024-09-20T18:56:54.010025+0000","last_undegraded":"2024-09-20T18:56:54.010025+0000","last_fullsized":"2024-09-20T18:56:54.010025+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":24,"num_read_kb":14,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":343,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.015398+0000","last_change":"2024-09-20T18:55:28.830447+0000","last_active":"2024-09-20T18:56:54.015398+0000","last_peered":"2024-09-20T18:56:54.015398+0000","last_clean":"2024-09-20T18:56:54.015398+0000","last_became_active":"2024-09-20T18:55:28.830147+0000","last_became_peered":"2024-09-20T18:55:28.830147+0000","last_unstale":"2024-09-20T18:56:54.015398+0000","last_undegraded":"2024-09-20T18:56:54.015398+0000","last_fullsized":"2024-09-20T18:56:54.015398+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:13.144896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:13.144896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":400,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-20T18:57:00.099559+0000","last_change":"2024-09-20T18:55:28.844060+0000","last_active":"2024-09-20T18:57:00.099559+0000","last_peered":"2024-09-20T18:57:00.099559+0000","last_clean":"2024-09-20T18:57:00.099559+0000","last_became_active":"2024-09-20T18:55:28.829325+0000","last_became_peered":"2024-09-20T18:55:28.829325+0000","last_unstale":"2024-09-20T18:57:00.099559+0000","last_undegraded":"2024-09-20T18:57:00.099559+0000","last_fullsized":"2024-09-20T18:57:00.099559+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T18:55:13.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T18:55:13.696607+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":323,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-20T18:57:00.100585+0000","last_change":"2024-09-20T18:51:22.852942+0000","last_active":"2024-09-20T18:57:00.100585+0000","last_peered":"2024-09-20T18:57:00.100585+0000","last_clean":"2024-09-20T18:57:00.100585+0000","last_became_active":"2024-09-20T18:49:49.956128+0000","last_became_peered":"2024-09-20T18:49:49.956128+0000","last_unstale":"2024-09-20T18:57:00.100585+0000","last_undegraded":"2024-09-20T18:57:00.100585+0000","last_fullsized":"2024-09-20T18:57:00.100585+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:22.852869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:22.852869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":331,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.031334+0000","last_change":"2024-09-20T18:55:25.281435+0000","last_active":"2024-09-20T18:56:54.031334+0000","last_peered":"2024-09-20T18:56:54.031334+0000","last_clean":"2024-09-20T18:56:54.031334+0000","last_became_active":"2024-09-20T18:49:51.971332+0000","last_became_peered":"2024-09-20T18:49:51.971332+0000","last_unstale":"2024-09-20T18:56:54.031334+0000","last_undegraded":"2024-09-20T18:56:54.031334+0000","last_fullsized":"2024-09-20T18:56:54.031334+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:25.281366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:25.281366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"212'9714","reported_seq":9830,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:55.900319+0000","last_change":"2024-09-20T18:54:33.929366+0000","last_active":"2024-09-20T18:56:55.900319+0000","last_peered":"2024-09-20T18:56:55.900319+0000","last_clean":"2024-09-20T18:56:55.900319+0000","last_became_active":"2024-09-20T18:54:33.929072+0000","last_became_peered":"2024-09-20T18:54:33.929072+0000","last_unstale":"2024-09-20T18:56:55.900319+0000","last_undegraded":"2024-09-20T18:56:55.900319+0000","last_fullsized":"2024-09-20T18:56:55.900319+0000","mapping_epoch":160,"log_start":"180'5102","ondisk_log_start":"180'5102","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4612,"log_dups_size":0,"ondisk_log_size":4612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":317718528,"num_objects":4848,"num_object_clones":0,"num_object_copies":14544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9714,"num_write_kb":465984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":362,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.007754+0000","last_change":"2024-09-20T18:55:28.840105+0000","last_active":"2024-09-20T18:56:54.007754+0000","last_peered":"2024-09-20T18:56:54.007754+0000","last_clean":"2024-09-20T18:56:54.007754+0000","last_became_active":"2024-09-20T18:55:28.818269+0000","last_became_peered":"2024-09-20T18:55:28.818269+0000","last_unstale":"2024-09-20T18:56:54.007754+0000","last_undegraded":"2024-09-20T18:56:54.007754+0000","last_fullsized":"2024-09-20T18:56:54.007754+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:51:14.397930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.397930+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":328,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.007780+0000","last_change":"2024-09-20T18:51:18.410601+0000","last_active":"2024-09-20T18:56:54.007780+0000","last_peered":"2024-09-20T18:56:54.007780+0000","last_clean":"2024-09-20T18:56:54.007780+0000","last_became_active":"2024-09-20T18:49:45.944579+0000","last_became_peered":"2024-09-20T18:49:45.944579+0000","last_unstale":"2024-09-20T18:56:54.007780+0000","last_undegraded":"2024-09-20T18:56:54.007780+0000","last_fullsized":"2024-09-20T18:56:54.007780+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:18.410447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:18.410447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":342,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.031238+0000","last_change":"2024-09-20T18:55:28.830634+0000","last_active":"2024-09-20T18:56:54.031238+0000","last_peered":"2024-09-20T18:56:54.031238+0000","last_clean":"2024-09-20T18:56:54.031238+0000","last_became_active":"2024-09-20T18:55:28.827785+0000","last_became_peered":"2024-09-20T18:55:28.827785+0000","last_unstale":"2024-09-20T18:56:54.031238+0000","last_undegraded":"2024-09-20T18:56:54.031238+0000","last_fullsized":"2024-09-20T18:56:54.031238+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:11.401777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:11.401777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":323,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.018002+0000","last_change":"2024-09-20T18:55:28.833649+0000","last_active":"2024-09-20T18:56:54.018002+0000","last_peered":"2024-09-20T18:56:54.018002+0000","last_clean":"2024-09-20T18:56:54.018002+0000","last_became_active":"2024-09-20T18:55:28.833115+0000","last_became_peered":"2024-09-20T18:55:28.833115+0000","last_unstale":"2024-09-20T18:56:54.018002+0000","last_undegraded":"2024-09-20T18:56:54.018002+0000","last_fullsized":"2024-09-20T18:56:54.018002+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:05.021071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.021071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1","version":"212'9609","reported_seq":9725,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:56.243091+0000","last_change":"2024-09-20T18:54:34.310237+0000","last_active":"2024-09-20T18:56:56.243091+0000","last_peered":"2024-09-20T18:56:56.243091+0000","last_clean":"2024-09-20T18:56:56.243091+0000","last_became_active":"2024-09-20T18:54:34.309781+0000","last_became_peered":"2024-09-20T18:54:34.309781+0000","last_unstale":"2024-09-20T18:56:56.243091+0000","last_undegraded":"2024-09-20T18:56:56.243091+0000","last_fullsized":"2024-09-20T18:56:56.243091+0000","mapping_epoch":160,"log_start":"186'5324","ondisk_log_start":"186'5324","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4285,"log_dups_size":0,"ondisk_log_size":4285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":316080128,"num_objects":4823,"num_object_clones":0,"num_object_copies":14469,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9609,"num_write_kb":461824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":360,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.010080+0000","last_change":"2024-09-20T18:52:02.944536+0000","last_active":"2024-09-20T18:56:54.010080+0000","last_peered":"2024-09-20T18:56:54.010080+0000","last_clean":"2024-09-20T18:56:54.010080+0000","last_became_active":"2024-09-20T18:49:47.947636+0000","last_became_peered":"2024-09-20T18:49:47.947636+0000","last_unstale":"2024-09-20T18:56:54.010080+0000","last_undegraded":"2024-09-20T18:56:54.010080+0000","last_fullsized":"2024-09-20T18:56:54.010080+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-20T18:52:02.944434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:02.944434+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":197,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.007628+0000","last_change":"2024-09-20T18:55:28.822330+0000","last_active":"2024-09-20T18:56:54.007628+0000","last_peered":"2024-09-20T18:56:54.007628+0000","last_clean":"2024-09-20T18:56:54.007628+0000","last_became_active":"2024-09-20T18:55:28.821959+0000","last_became_peered":"2024-09-20T18:55:28.821959+0000","last_unstale":"2024-09-20T18:56:54.007628+0000","last_undegraded":"2024-09-20T18:56:54.007628+0000","last_fullsized":"2024-09-20T18:56:54.007628+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:45.669427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:45.669427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":328,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.029140+0000","last_change":"2024-09-20T18:54:11.377907+0000","last_active":"2024-09-20T18:56:54.029140+0000","last_peered":"2024-09-20T18:56:54.029140+0000","last_clean":"2024-09-20T18:56:54.029140+0000","last_became_active":"2024-09-20T18:49:49.959160+0000","last_became_peered":"2024-09-20T18:49:49.959160+0000","last_unstale":"2024-09-20T18:56:54.029140+0000","last_undegraded":"2024-09-20T18:56:54.029140+0000","last_fullsized":"2024-09-20T18:56:54.029140+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:11.376951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:11.376951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":325,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.057670+0000","last_change":"2024-09-20T18:52:35.635897+0000","last_active":"2024-09-20T18:56:54.057670+0000","last_peered":"2024-09-20T18:56:54.057670+0000","last_clean":"2024-09-20T18:56:54.057670+0000","last_became_active":"2024-09-20T18:49:51.970108+0000","last_became_peered":"2024-09-20T18:49:51.970108+0000","last_unstale":"2024-09-20T18:56:54.057670+0000","last_undegraded":"2024-09-20T18:56:54.057670+0000","last_fullsized":"2024-09-20T18:56:54.057670+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:35.635818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:35.635818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"212'9482","reported_seq":9598,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:56.424192+0000","last_change":"2024-09-20T18:54:33.929552+0000","last_active":"2024-09-20T18:56:56.424192+0000","last_peered":"2024-09-20T18:56:56.424192+0000","last_clean":"2024-09-20T18:56:56.424192+0000","last_became_active":"2024-09-20T18:54:33.929208+0000","last_became_peered":"2024-09-20T18:54:33.929208+0000","last_unstale":"2024-09-20T18:56:56.424192+0000","last_undegraded":"2024-09-20T18:56:56.424192+0000","last_fullsized":"2024-09-20T18:56:56.424192+0000","mapping_epoch":160,"log_start":"183'5121","ondisk_log_start":"183'5121","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4361,"log_dups_size":0,"ondisk_log_size":4361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":310902784,"num_objects":4744,"num_object_clones":0,"num_object_copies":14232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9482,"num_write_kb":455232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":366,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.007948+0000","last_change":"2024-09-20T18:55:28.846773+0000","last_active":"2024-09-20T18:56:54.007948+0000","last_peered":"2024-09-20T18:56:54.007948+0000","last_clean":"2024-09-20T18:56:54.007948+0000","last_became_active":"2024-09-20T18:55:28.817101+0000","last_became_peered":"2024-09-20T18:55:28.817101+0000","last_unstale":"2024-09-20T18:56:54.007948+0000","last_undegraded":"2024-09-20T18:56:54.007948+0000","last_fullsized":"2024-09-20T18:56:54.007948+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:54:17.219548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:17.219548+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":336,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.017882+0000","last_change":"2024-09-20T18:52:46.005172+0000","last_active":"2024-09-20T18:56:54.017882+0000","last_peered":"2024-09-20T18:56:54.017882+0000","last_clean":"2024-09-20T18:56:54.017882+0000","last_became_active":"2024-09-20T18:49:45.937264+0000","last_became_peered":"2024-09-20T18:49:45.937264+0000","last_unstale":"2024-09-20T18:56:54.017882+0000","last_undegraded":"2024-09-20T18:56:54.017882+0000","last_fullsized":"2024-09-20T18:56:54.017882+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:46.005109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:46.005109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":432,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.049082+0000","last_change":"2024-09-20T18:55:17.549660+0000","last_active":"2024-09-20T18:56:54.049082+0000","last_peered":"2024-09-20T18:56:54.049082+0000","last_clean":"2024-09-20T18:56:54.049082+0000","last_became_active":"2024-09-20T18:49:49.957005+0000","last_became_peered":"2024-09-20T18:49:49.957005+0000","last_unstale":"2024-09-20T18:56:54.049082+0000","last_undegraded":"2024-09-20T18:56:54.049082+0000","last_fullsized":"2024-09-20T18:56:54.049082+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:17.549598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:17.549598+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":324,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.007462+0000","last_change":"2024-09-20T18:54:42.182875+0000","last_active":"2024-09-20T18:56:54.007462+0000","last_peered":"2024-09-20T18:56:54.007462+0000","last_clean":"2024-09-20T18:56:54.007462+0000","last_became_active":"2024-09-20T18:49:51.969355+0000","last_became_peered":"2024-09-20T18:49:51.969355+0000","last_unstale":"2024-09-20T18:56:54.007462+0000","last_undegraded":"2024-09-20T18:56:54.007462+0000","last_fullsized":"2024-09-20T18:56:54.007462+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:42.182809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:42.182809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"212'9699","reported_seq":21685,"reported_epoch":212,"state":"active+remapped+backfilling","last_fresh":"2024-09-20T18:56:56.271163+0000","last_change":"2024-09-20T18:55:59.938306+0000","last_active":"2024-09-20T18:56:56.271163+0000","last_peered":"2024-09-20T18:56:56.271163+0000","last_clean":"2024-09-20T18:55:27.792821+0000","last_became_active":"2024-09-20T18:55:29.915245+0000","last_became_peered":"2024-09-20T18:55:29.915245+0000","last_unstale":"2024-09-20T18:56:56.271163+0000","last_undegraded":"2024-09-20T18:56:56.271163+0000","last_fullsized":"2024-09-20T18:56:56.271163+0000","mapping_epoch":182,"log_start":"186'5461","ondisk_log_start":"186'5461","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4238,"log_dups_size":0,"ondisk_log_size":4238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":317915136,"num_objects":4851,"num_object_clones":0,"num_object_copies":14553,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":198,"num_objects_unfound":0,"num_objects_dirty":4851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9699,"num_write_kb":465600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5930,"num_bytes_recovered":388628480,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,6],"avail_no_missing":["1","3","6"],"object_location_counts":[{"shards":"1,3,6","objects":4851}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":355,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.023582+0000","last_change":"2024-09-20T18:54:54.897991+0000","last_active":"2024-09-20T18:56:54.023582+0000","last_peered":"2024-09-20T18:56:54.023582+0000","last_clean":"2024-09-20T18:56:54.023582+0000","last_became_active":"2024-09-20T18:49:47.946008+0000","last_became_peered":"2024-09-20T18:49:47.946008+0000","last_unstale":"2024-09-20T18:56:54.023582+0000","last_undegraded":"2024-09-20T18:56:54.023582+0000","last_fullsized":"2024-09-20T18:56:54.023582+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:54.897933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:54.897933+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":350,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.031279+0000","last_change":"2024-09-20T18:55:28.833453+0000","last_active":"2024-09-20T18:56:54.031279+0000","last_peered":"2024-09-20T18:56:54.031279+0000","last_clean":"2024-09-20T18:56:54.031279+0000","last_became_active":"2024-09-20T18:55:28.833095+0000","last_became_peered":"2024-09-20T18:55:28.833095+0000","last_unstale":"2024-09-20T18:56:54.031279+0000","last_undegraded":"2024-09-20T18:56:54.031279+0000","last_fullsized":"2024-09-20T18:56:54.031279+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:56.212378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:56.212378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":337,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.051699+0000","last_change":"2024-09-20T18:54:43.379898+0000","last_active":"2024-09-20T18:56:54.051699+0000","last_peered":"2024-09-20T18:56:54.051699+0000","last_clean":"2024-09-20T18:56:54.051699+0000","last_became_active":"2024-09-20T18:49:49.956767+0000","last_became_peered":"2024-09-20T18:49:49.956767+0000","last_unstale":"2024-09-20T18:56:54.051699+0000","last_undegraded":"2024-09-20T18:56:54.051699+0000","last_fullsized":"2024-09-20T18:56:54.051699+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:43.379706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:43.379706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":323,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.024075+0000","last_change":"2024-09-20T18:54:24.976627+0000","last_active":"2024-09-20T18:56:54.024075+0000","last_peered":"2024-09-20T18:56:54.024075+0000","last_clean":"2024-09-20T18:56:54.024075+0000","last_became_active":"2024-09-20T18:49:51.964098+0000","last_became_peered":"2024-09-20T18:49:51.964098+0000","last_unstale":"2024-09-20T18:56:54.024075+0000","last_undegraded":"2024-09-20T18:56:54.024075+0000","last_fullsized":"2024-09-20T18:56:54.024075+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:24.976584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:24.976584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"212'9705","reported_seq":12764,"reported_epoch":212,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-20T18:56:56.734851+0000","last_change":"2024-09-20T18:56:27.871592+0000","last_active":"2024-09-20T18:56:56.734851+0000","last_peered":"2024-09-20T18:56:56.734851+0000","last_clean":"2024-09-20T18:55:27.791097+0000","last_became_active":"2024-09-20T18:55:29.979932+0000","last_became_peered":"2024-09-20T18:55:29.979932+0000","last_unstale":"2024-09-20T18:56:56.734851+0000","last_undegraded":"2024-09-20T18:56:27.867865+0000","last_fullsized":"2024-09-20T18:55:29.815116+0000","mapping_epoch":182,"log_start":"177'4840","ondisk_log_start":"177'4840","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4865,"log_dups_size":0,"ondisk_log_size":4865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":310837248,"num_objects":4743,"num_object_clones":0,"num_object_copies":14229,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1012,"num_objects_misplaced":2696,"num_objects_unfound":0,"num_objects_dirty":4743,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9705,"num_write_kb":462336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1470,"num_bytes_recovered":96337920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7],"avail_no_missing":["0","6","7"],"object_location_counts":[{"shards":"0,6,7","objects":2696},{"shards":"0,7","objects":1035}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":144,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.029266+0000","last_change":"2024-09-20T18:55:28.855793+0000","last_active":"2024-09-20T18:56:54.029266+0000","last_peered":"2024-09-20T18:56:54.029266+0000","last_clean":"2024-09-20T18:56:54.029266+0000","last_became_active":"2024-09-20T18:55:28.828563+0000","last_became_peered":"2024-09-20T18:55:28.828563+0000","last_unstale":"2024-09-20T18:56:54.029266+0000","last_undegraded":"2024-09-20T18:56:54.029266+0000","last_fullsized":"2024-09-20T18:56:54.029266+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-20T18:53:24.458005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:24.458005+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":31,"num_read_kb":20,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,5],"acting":[1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":354,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-20T18:57:00.099997+0000","last_change":"2024-09-20T18:55:28.835767+0000","last_active":"2024-09-20T18:57:00.099997+0000","last_peered":"2024-09-20T18:57:00.099997+0000","last_clean":"2024-09-20T18:57:00.099997+0000","last_became_active":"2024-09-20T18:55:28.835149+0000","last_became_peered":"2024-09-20T18:55:28.835149+0000","last_unstale":"2024-09-20T18:57:00.099997+0000","last_undegraded":"2024-09-20T18:57:00.099997+0000","last_fullsized":"2024-09-20T18:57:00.099997+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:44.501192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.501192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":337,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.049797+0000","last_change":"2024-09-20T18:55:13.577772+0000","last_active":"2024-09-20T18:56:54.049797+0000","last_peered":"2024-09-20T18:56:54.049797+0000","last_clean":"2024-09-20T18:56:54.049797+0000","last_became_active":"2024-09-20T18:49:49.957558+0000","last_became_peered":"2024-09-20T18:49:49.957558+0000","last_unstale":"2024-09-20T18:56:54.049797+0000","last_undegraded":"2024-09-20T18:56:54.049797+0000","last_fullsized":"2024-09-20T18:56:54.049797+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:13.577699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:13.577699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":335,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-20T18:57:00.099427+0000","last_change":"2024-09-20T18:53:47.370152+0000","last_active":"2024-09-20T18:57:00.099427+0000","last_peered":"2024-09-20T18:57:00.099427+0000","last_clean":"2024-09-20T18:57:00.099427+0000","last_became_active":"2024-09-20T18:49:51.970798+0000","last_became_peered":"2024-09-20T18:49:51.970798+0000","last_unstale":"2024-09-20T18:57:00.099427+0000","last_undegraded":"2024-09-20T18:57:00.099427+0000","last_fullsized":"2024-09-20T18:57:00.099427+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:47.370108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:47.370108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"212'9913","reported_seq":17144,"reported_epoch":212,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-20T18:56:55.915364+0000","last_change":"2024-09-20T18:55:30.067593+0000","last_active":"2024-09-20T18:56:55.915364+0000","last_peered":"2024-09-20T18:56:55.915364+0000","last_clean":"2024-09-20T18:55:27.760955+0000","last_became_active":"2024-09-20T18:55:30.004922+0000","last_became_peered":"2024-09-20T18:55:30.004922+0000","last_unstale":"2024-09-20T18:56:55.915364+0000","last_undegraded":"2024-09-20T18:56:55.915364+0000","last_fullsized":"2024-09-20T18:55:29.819726+0000","mapping_epoch":182,"log_start":"180'5346","ondisk_log_start":"180'5346","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4567,"log_dups_size":0,"ondisk_log_size":4567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":332464128,"num_objects":5073,"num_object_clones":0,"num_object_copies":15219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1835,"num_objects_unfound":0,"num_objects_dirty":5073,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9913,"num_write_kb":479552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3556,"num_bytes_recovered":233046016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7],"avail_no_missing":["4","6","7"],"object_location_counts":[{"shards":"4,6,7","objects":1835},{"shards":"4,7","objects":3238}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":355,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.007534+0000","last_change":"2024-09-20T18:53:52.898818+0000","last_active":"2024-09-20T18:56:54.007534+0000","last_peered":"2024-09-20T18:56:54.007534+0000","last_clean":"2024-09-20T18:56:54.007534+0000","last_became_active":"2024-09-20T18:49:47.949116+0000","last_became_peered":"2024-09-20T18:49:47.949116+0000","last_unstale":"2024-09-20T18:56:54.007534+0000","last_undegraded":"2024-09-20T18:56:54.007534+0000","last_fullsized":"2024-09-20T18:56:54.007534+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:53:52.898760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:52.898760+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":347,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-20T18:57:00.102301+0000","last_change":"2024-09-20T18:54:52.650798+0000","last_active":"2024-09-20T18:57:00.102301+0000","last_peered":"2024-09-20T18:57:00.102301+0000","last_clean":"2024-09-20T18:57:00.102301+0000","last_became_active":"2024-09-20T18:49:45.944858+0000","last_became_peered":"2024-09-20T18:49:45.944858+0000","last_unstale":"2024-09-20T18:57:00.102301+0000","last_undegraded":"2024-09-20T18:57:00.102301+0000","last_fullsized":"2024-09-20T18:57:00.102301+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:52.650756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:52.650756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":329,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.048885+0000","last_change":"2024-09-20T18:54:42.349879+0000","last_active":"2024-09-20T18:56:54.048885+0000","last_peered":"2024-09-20T18:56:54.048885+0000","last_clean":"2024-09-20T18:56:54.048885+0000","last_became_active":"2024-09-20T18:49:49.954944+0000","last_became_peered":"2024-09-20T18:49:49.954944+0000","last_unstale":"2024-09-20T18:56:54.048885+0000","last_undegraded":"2024-09-20T18:56:54.048885+0000","last_fullsized":"2024-09-20T18:56:54.048885+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:42.349807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:42.349807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":324,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.017579+0000","last_change":"2024-09-20T18:55:17.341862+0000","last_active":"2024-09-20T18:56:54.017579+0000","last_peered":"2024-09-20T18:56:54.017579+0000","last_clean":"2024-09-20T18:56:54.017579+0000","last_became_active":"2024-09-20T18:49:51.972970+0000","last_became_peered":"2024-09-20T18:49:51.972970+0000","last_unstale":"2024-09-20T18:56:54.017579+0000","last_undegraded":"2024-09-20T18:56:54.017579+0000","last_fullsized":"2024-09-20T18:56:54.017579+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:17.341793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:17.341793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.4","version":"212'9515","reported_seq":9631,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:55.842484+0000","last_change":"2024-09-20T18:54:34.309442+0000","last_active":"2024-09-20T18:56:55.842484+0000","last_peered":"2024-09-20T18:56:55.842484+0000","last_clean":"2024-09-20T18:56:55.842484+0000","last_became_active":"2024-09-20T18:54:34.309113+0000","last_became_peered":"2024-09-20T18:54:34.309113+0000","last_unstale":"2024-09-20T18:56:55.842484+0000","last_undegraded":"2024-09-20T18:56:55.842484+0000","last_fullsized":"2024-09-20T18:56:55.842484+0000","mapping_epoch":160,"log_start":"175'4431","ondisk_log_start":"175'4431","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":5084,"log_dups_size":0,"ondisk_log_size":5084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":315686912,"num_objects":4817,"num_object_clones":0,"num_object_copies":14451,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4817,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9515,"num_write_kb":458624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":379,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.030988+0000","last_change":"2024-09-20T18:55:28.847010+0000","last_active":"2024-09-20T18:56:54.030988+0000","last_peered":"2024-09-20T18:56:54.030988+0000","last_clean":"2024-09-20T18:56:54.030988+0000","last_became_active":"2024-09-20T18:55:28.824801+0000","last_became_peered":"2024-09-20T18:55:28.824801+0000","last_unstale":"2024-09-20T18:56:54.030988+0000","last_undegraded":"2024-09-20T18:56:54.030988+0000","last_fullsized":"2024-09-20T18:56:54.030988+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:54:12.373309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:12.373309+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":367,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-20T18:57:00.099923+0000","last_change":"2024-09-20T18:55:24.729301+0000","last_active":"2024-09-20T18:57:00.099923+0000","last_peered":"2024-09-20T18:57:00.099923+0000","last_clean":"2024-09-20T18:57:00.099923+0000","last_became_active":"2024-09-20T18:49:45.944256+0000","last_became_peered":"2024-09-20T18:49:45.944256+0000","last_unstale":"2024-09-20T18:57:00.099923+0000","last_undegraded":"2024-09-20T18:57:00.099923+0000","last_fullsized":"2024-09-20T18:57:00.099923+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-20T18:55:24.729219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:24.729219+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"184'9","reported_seq":429,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-20T18:57:00.099363+0000","last_change":"2024-09-20T18:55:28.824771+0000","last_active":"2024-09-20T18:57:00.099363+0000","last_peered":"2024-09-20T18:57:00.099363+0000","last_clean":"2024-09-20T18:57:00.099363+0000","last_became_active":"2024-09-20T18:55:28.818085+0000","last_became_peered":"2024-09-20T18:55:28.818085+0000","last_unstale":"2024-09-20T18:57:00.099363+0000","last_undegraded":"2024-09-20T18:57:00.099363+0000","last_fullsized":"2024-09-20T18:57:00.099363+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:52:05.788924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.788924+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":330,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.030815+0000","last_change":"2024-09-20T18:55:28.832276+0000","last_active":"2024-09-20T18:56:54.030815+0000","last_peered":"2024-09-20T18:56:54.030815+0000","last_clean":"2024-09-20T18:56:54.030815+0000","last_became_active":"2024-09-20T18:55:28.830380+0000","last_became_peered":"2024-09-20T18:55:28.830380+0000","last_unstale":"2024-09-20T18:56:54.030815+0000","last_undegraded":"2024-09-20T18:56:54.030815+0000","last_fullsized":"2024-09-20T18:56:54.030815+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:14.247398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:14.247398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":73,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.022930+0000","last_change":"2024-09-20T18:55:28.835344+0000","last_active":"2024-09-20T18:56:54.022930+0000","last_peered":"2024-09-20T18:56:54.022930+0000","last_clean":"2024-09-20T18:56:54.022930+0000","last_became_active":"2024-09-20T18:55:28.835010+0000","last_became_peered":"2024-09-20T18:55:28.835010+0000","last_unstale":"2024-09-20T18:56:54.022930+0000","last_undegraded":"2024-09-20T18:56:54.022930+0000","last_fullsized":"2024-09-20T18:56:54.022930+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:26.721735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:26.721735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":340,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.036737+0000","last_change":"2024-09-20T18:54:44.194161+0000","last_active":"2024-09-20T18:56:54.036737+0000","last_peered":"2024-09-20T18:56:54.036737+0000","last_clean":"2024-09-20T18:56:54.036737+0000","last_became_active":"2024-09-20T18:49:48.500662+0000","last_became_peered":"2024-09-20T18:49:48.500662+0000","last_unstale":"2024-09-20T18:56:54.036737+0000","last_undegraded":"2024-09-20T18:56:54.036737+0000","last_fullsized":"2024-09-20T18:56:54.036737+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-20T18:54:44.194089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.194089+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":326,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.031314+0000","last_change":"2024-09-20T18:53:16.220554+0000","last_active":"2024-09-20T18:56:54.031314+0000","last_peered":"2024-09-20T18:56:54.031314+0000","last_clean":"2024-09-20T18:56:54.031314+0000","last_became_active":"2024-09-20T18:49:49.956749+0000","last_became_peered":"2024-09-20T18:49:49.956749+0000","last_unstale":"2024-09-20T18:56:54.031314+0000","last_undegraded":"2024-09-20T18:56:54.031314+0000","last_fullsized":"2024-09-20T18:56:54.031314+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:16.220490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:16.220490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":341,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-20T18:57:00.099779+0000","last_change":"2024-09-20T18:55:28.827897+0000","last_active":"2024-09-20T18:57:00.099779+0000","last_peered":"2024-09-20T18:57:00.099779+0000","last_clean":"2024-09-20T18:57:00.099779+0000","last_became_active":"2024-09-20T18:55:28.827418+0000","last_became_peered":"2024-09-20T18:55:28.827418+0000","last_unstale":"2024-09-20T18:57:00.099779+0000","last_undegraded":"2024-09-20T18:57:00.099779+0000","last_fullsized":"2024-09-20T18:57:00.099779+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.637739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.637739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":346,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.041115+0000","last_change":"2024-09-20T18:54:44.830872+0000","last_active":"2024-09-20T18:56:54.041115+0000","last_peered":"2024-09-20T18:56:54.041115+0000","last_clean":"2024-09-20T18:56:54.041115+0000","last_became_active":"2024-09-20T18:49:48.500803+0000","last_became_peered":"2024-09-20T18:49:48.500803+0000","last_unstale":"2024-09-20T18:56:54.041115+0000","last_undegraded":"2024-09-20T18:56:54.041115+0000","last_fullsized":"2024-09-20T18:56:54.041115+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:44.830790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.830790+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":346,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-20T18:57:00.099461+0000","last_change":"2024-09-20T18:55:29.751718+0000","last_active":"2024-09-20T18:57:00.099461+0000","last_peered":"2024-09-20T18:57:00.099461+0000","last_clean":"2024-09-20T18:57:00.099461+0000","last_became_active":"2024-09-20T18:55:28.833799+0000","last_became_peered":"2024-09-20T18:55:28.833799+0000","last_unstale":"2024-09-20T18:57:00.099461+0000","last_undegraded":"2024-09-20T18:57:00.099461+0000","last_fullsized":"2024-09-20T18:57:00.099461+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:29.751642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:29.751642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":327,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.036599+0000","last_change":"2024-09-20T18:53:50.253721+0000","last_active":"2024-09-20T18:56:54.036599+0000","last_peered":"2024-09-20T18:56:54.036599+0000","last_clean":"2024-09-20T18:56:54.036599+0000","last_became_active":"2024-09-20T18:49:49.961947+0000","last_became_peered":"2024-09-20T18:49:49.961947+0000","last_unstale":"2024-09-20T18:56:54.036599+0000","last_undegraded":"2024-09-20T18:56:54.036599+0000","last_fullsized":"2024-09-20T18:56:54.036599+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:50.253660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:50.253660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"66'1","reported_seq":334,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.052326+0000","last_change":"2024-09-20T18:54:20.681878+0000","last_active":"2024-09-20T18:56:54.052326+0000","last_peered":"2024-09-20T18:56:54.052326+0000","last_clean":"2024-09-20T18:56:54.052326+0000","last_became_active":"2024-09-20T18:49:51.965121+0000","last_became_peered":"2024-09-20T18:49:51.965121+0000","last_unstale":"2024-09-20T18:56:54.052326+0000","last_undegraded":"2024-09-20T18:56:54.052326+0000","last_fullsized":"2024-09-20T18:56:54.052326+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T18:54:20.681790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:20.681790+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":338,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.031457+0000","last_change":"2024-09-20T18:55:12.205118+0000","last_active":"2024-09-20T18:56:54.031457+0000","last_peered":"2024-09-20T18:56:54.031457+0000","last_clean":"2024-09-20T18:56:54.031457+0000","last_became_active":"2024-09-20T18:49:48.501011+0000","last_became_peered":"2024-09-20T18:49:48.501011+0000","last_unstale":"2024-09-20T18:56:54.031457+0000","last_undegraded":"2024-09-20T18:56:54.031457+0000","last_fullsized":"2024-09-20T18:56:54.031457+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:55:12.205069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:12.205069+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":343,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.025637+0000","last_change":"2024-09-20T18:55:27.028184+0000","last_active":"2024-09-20T18:56:54.025637+0000","last_peered":"2024-09-20T18:56:54.025637+0000","last_clean":"2024-09-20T18:56:54.025637+0000","last_became_active":"2024-09-20T18:49:45.938511+0000","last_became_peered":"2024-09-20T18:49:45.938511+0000","last_unstale":"2024-09-20T18:56:54.025637+0000","last_undegraded":"2024-09-20T18:56:54.025637+0000","last_fullsized":"2024-09-20T18:56:54.025637+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:27.028126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:27.028126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":431,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:56.077003+0000","last_change":"2024-09-20T18:55:27.284156+0000","last_active":"2024-09-20T18:56:56.077003+0000","last_peered":"2024-09-20T18:56:56.077003+0000","last_clean":"2024-09-20T18:56:56.077003+0000","last_became_active":"2024-09-20T18:49:49.960550+0000","last_became_peered":"2024-09-20T18:49:49.960550+0000","last_unstale":"2024-09-20T18:56:56.077003+0000","last_undegraded":"2024-09-20T18:56:56.077003+0000","last_fullsized":"2024-09-20T18:56:56.077003+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:27.284086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:27.284086+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":331,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.007667+0000","last_change":"2024-09-20T18:55:12.348079+0000","last_active":"2024-09-20T18:56:54.007667+0000","last_peered":"2024-09-20T18:56:54.007667+0000","last_clean":"2024-09-20T18:56:54.007667+0000","last_became_active":"2024-09-20T18:49:51.966254+0000","last_became_peered":"2024-09-20T18:49:51.966254+0000","last_unstale":"2024-09-20T18:56:54.007667+0000","last_undegraded":"2024-09-20T18:56:54.007667+0000","last_fullsized":"2024-09-20T18:56:54.007667+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:12.347993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:12.347993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":358,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.024666+0000","last_change":"2024-09-20T18:51:15.223913+0000","last_active":"2024-09-20T18:56:54.024666+0000","last_peered":"2024-09-20T18:56:54.024666+0000","last_clean":"2024-09-20T18:56:54.024666+0000","last_became_active":"2024-09-20T18:49:58.997494+0000","last_became_peered":"2024-09-20T18:49:58.997494+0000","last_unstale":"2024-09-20T18:56:54.024666+0000","last_undegraded":"2024-09-20T18:56:54.024666+0000","last_fullsized":"2024-09-20T18:56:54.024666+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:51:15.223826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:15.223826+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":347,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-20T18:57:00.099781+0000","last_change":"2024-09-20T18:54:57.632458+0000","last_active":"2024-09-20T18:57:00.099781+0000","last_peered":"2024-09-20T18:57:00.099781+0000","last_clean":"2024-09-20T18:57:00.099781+0000","last_became_active":"2024-09-20T18:49:45.949492+0000","last_became_peered":"2024-09-20T18:49:45.949492+0000","last_unstale":"2024-09-20T18:57:00.099781+0000","last_undegraded":"2024-09-20T18:57:00.099781+0000","last_fullsized":"2024-09-20T18:57:00.099781+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.632380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.632380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":428,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.031160+0000","last_change":"2024-09-20T18:55:28.302353+0000","last_active":"2024-09-20T18:56:54.031160+0000","last_peered":"2024-09-20T18:56:54.031160+0000","last_clean":"2024-09-20T18:56:54.031160+0000","last_became_active":"2024-09-20T18:49:49.962243+0000","last_became_peered":"2024-09-20T18:49:49.962243+0000","last_unstale":"2024-09-20T18:56:54.031160+0000","last_undegraded":"2024-09-20T18:56:54.031160+0000","last_fullsized":"2024-09-20T18:56:54.031160+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:28.302303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:28.302303+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:55.176573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":330,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.025973+0000","last_change":"2024-09-20T18:55:11.918284+0000","last_active":"2024-09-20T18:56:54.025973+0000","last_peered":"2024-09-20T18:56:54.025973+0000","last_clean":"2024-09-20T18:56:54.025973+0000","last_became_active":"2024-09-20T18:49:51.969093+0000","last_became_peered":"2024-09-20T18:49:51.969093+0000","last_unstale":"2024-09-20T18:56:54.025973+0000","last_undegraded":"2024-09-20T18:56:54.025973+0000","last_fullsized":"2024-09-20T18:56:54.025973+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:11.918210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:11.918210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":370,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.007954+0000","last_change":"2024-09-20T18:55:28.854249+0000","last_active":"2024-09-20T18:56:54.007954+0000","last_peered":"2024-09-20T18:56:54.007954+0000","last_clean":"2024-09-20T18:56:54.007954+0000","last_became_active":"2024-09-20T18:55:28.826721+0000","last_became_peered":"2024-09-20T18:55:28.826721+0000","last_unstale":"2024-09-20T18:56:54.007954+0000","last_undegraded":"2024-09-20T18:56:54.007954+0000","last_fullsized":"2024-09-20T18:56:54.007954+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:53:43.878921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:43.878921+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":335,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.008158+0000","last_change":"2024-09-20T18:55:28.822690+0000","last_active":"2024-09-20T18:56:54.008158+0000","last_peered":"2024-09-20T18:56:54.008158+0000","last_clean":"2024-09-20T18:56:54.008158+0000","last_became_active":"2024-09-20T18:55:28.822258+0000","last_became_peered":"2024-09-20T18:55:28.822258+0000","last_unstale":"2024-09-20T18:56:54.008158+0000","last_undegraded":"2024-09-20T18:56:54.008158+0000","last_fullsized":"2024-09-20T18:56:54.008158+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:05.985551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.985551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":320,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.029182+0000","last_change":"2024-09-20T18:51:14.271954+0000","last_active":"2024-09-20T18:56:54.029182+0000","last_peered":"2024-09-20T18:56:54.029182+0000","last_clean":"2024-09-20T18:56:54.029182+0000","last_became_active":"2024-09-20T18:49:49.957067+0000","last_became_peered":"2024-09-20T18:49:49.957067+0000","last_unstale":"2024-09-20T18:56:54.029182+0000","last_undegraded":"2024-09-20T18:56:54.029182+0000","last_fullsized":"2024-09-20T18:56:54.029182+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:14.271719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.271719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":316,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.023056+0000","last_change":"2024-09-20T18:51:14.210438+0000","last_active":"2024-09-20T18:56:54.023056+0000","last_peered":"2024-09-20T18:56:54.023056+0000","last_clean":"2024-09-20T18:56:54.023056+0000","last_became_active":"2024-09-20T18:49:51.966915+0000","last_became_peered":"2024-09-20T18:49:51.966915+0000","last_unstale":"2024-09-20T18:56:54.023056+0000","last_undegraded":"2024-09-20T18:56:54.023056+0000","last_fullsized":"2024-09-20T18:56:54.023056+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:14.210358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.210358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":373,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.029437+0000","last_change":"2024-09-20T18:55:28.838698+0000","last_active":"2024-09-20T18:56:54.029437+0000","last_peered":"2024-09-20T18:56:54.029437+0000","last_clean":"2024-09-20T18:56:54.029437+0000","last_became_active":"2024-09-20T18:55:28.827023+0000","last_became_peered":"2024-09-20T18:55:28.827023+0000","last_unstale":"2024-09-20T18:56:54.029437+0000","last_undegraded":"2024-09-20T18:56:54.029437+0000","last_fullsized":"2024-09-20T18:56:54.029437+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:53:53.263976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:53.263976+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":338,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-20T18:57:00.099926+0000","last_change":"2024-09-20T18:54:42.487465+0000","last_active":"2024-09-20T18:57:00.099926+0000","last_peered":"2024-09-20T18:57:00.099926+0000","last_clean":"2024-09-20T18:57:00.099926+0000","last_became_active":"2024-09-20T18:49:45.944357+0000","last_became_peered":"2024-09-20T18:49:45.944357+0000","last_unstale":"2024-09-20T18:57:00.099926+0000","last_undegraded":"2024-09-20T18:57:00.099926+0000","last_fullsized":"2024-09-20T18:57:00.099926+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:42.487403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:42.487403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":328,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.017685+0000","last_change":"2024-09-20T18:54:18.108239+0000","last_active":"2024-09-20T18:56:54.017685+0000","last_peered":"2024-09-20T18:56:54.017685+0000","last_clean":"2024-09-20T18:56:54.017685+0000","last_became_active":"2024-09-20T18:49:49.962523+0000","last_became_peered":"2024-09-20T18:49:49.962523+0000","last_unstale":"2024-09-20T18:56:54.017685+0000","last_undegraded":"2024-09-20T18:56:54.017685+0000","last_fullsized":"2024-09-20T18:56:54.017685+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:18.108175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:18.108175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":331,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.022613+0000","last_change":"2024-09-20T18:54:26.882441+0000","last_active":"2024-09-20T18:56:54.022613+0000","last_peered":"2024-09-20T18:56:54.022613+0000","last_clean":"2024-09-20T18:56:54.022613+0000","last_became_active":"2024-09-20T18:49:51.967313+0000","last_became_peered":"2024-09-20T18:49:51.967313+0000","last_unstale":"2024-09-20T18:56:54.022613+0000","last_undegraded":"2024-09-20T18:56:54.022613+0000","last_fullsized":"2024-09-20T18:56:54.022613+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:26.882375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:26.882375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":370,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.023208+0000","last_change":"2024-09-20T18:55:28.843363+0000","last_active":"2024-09-20T18:56:54.023208+0000","last_peered":"2024-09-20T18:56:54.023208+0000","last_clean":"2024-09-20T18:56:54.023208+0000","last_became_active":"2024-09-20T18:55:28.830801+0000","last_became_peered":"2024-09-20T18:55:28.830801+0000","last_unstale":"2024-09-20T18:56:54.023208+0000","last_undegraded":"2024-09-20T18:56:54.023208+0000","last_fullsized":"2024-09-20T18:56:54.023208+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:54:21.106677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:21.106677+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":342,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.031306+0000","last_change":"2024-09-20T18:55:19.258723+0000","last_active":"2024-09-20T18:56:54.031306+0000","last_peered":"2024-09-20T18:56:54.031306+0000","last_clean":"2024-09-20T18:56:54.031306+0000","last_became_active":"2024-09-20T18:49:45.942105+0000","last_became_peered":"2024-09-20T18:49:45.942105+0000","last_unstale":"2024-09-20T18:56:54.031306+0000","last_undegraded":"2024-09-20T18:56:54.031306+0000","last_fullsized":"2024-09-20T18:56:54.031306+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:19.258674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:19.258674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":346,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-20T18:57:00.099147+0000","last_change":"2024-09-20T18:55:28.825004+0000","last_active":"2024-09-20T18:57:00.099147+0000","last_peered":"2024-09-20T18:57:00.099147+0000","last_clean":"2024-09-20T18:57:00.099147+0000","last_became_active":"2024-09-20T18:55:28.824355+0000","last_became_peered":"2024-09-20T18:55:28.824355+0000","last_unstale":"2024-09-20T18:57:00.099147+0000","last_undegraded":"2024-09-20T18:57:00.099147+0000","last_fullsized":"2024-09-20T18:57:00.099147+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:46.329547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:46.329547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":323,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.017808+0000","last_change":"2024-09-20T18:55:28.825607+0000","last_active":"2024-09-20T18:56:54.017808+0000","last_peered":"2024-09-20T18:56:54.017808+0000","last_clean":"2024-09-20T18:56:54.017808+0000","last_became_active":"2024-09-20T18:55:28.825296+0000","last_became_peered":"2024-09-20T18:55:28.825296+0000","last_unstale":"2024-09-20T18:56:54.017808+0000","last_undegraded":"2024-09-20T18:56:54.017808+0000","last_fullsized":"2024-09-20T18:56:54.017808+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:43.118805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:43.118805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":359,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.035249+0000","last_change":"2024-09-20T18:55:28.836789+0000","last_active":"2024-09-20T18:56:54.035249+0000","last_peered":"2024-09-20T18:56:54.035249+0000","last_clean":"2024-09-20T18:56:54.035249+0000","last_became_active":"2024-09-20T18:55:28.824579+0000","last_became_peered":"2024-09-20T18:55:28.824579+0000","last_unstale":"2024-09-20T18:56:54.035249+0000","last_undegraded":"2024-09-20T18:56:54.035249+0000","last_fullsized":"2024-09-20T18:56:54.035249+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-20T18:55:20.282193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:20.282193+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":342,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.025155+0000","last_change":"2024-09-20T18:54:55.900487+0000","last_active":"2024-09-20T18:56:54.025155+0000","last_peered":"2024-09-20T18:56:54.025155+0000","last_clean":"2024-09-20T18:56:54.025155+0000","last_became_active":"2024-09-20T18:49:45.940988+0000","last_became_peered":"2024-09-20T18:49:45.940988+0000","last_unstale":"2024-09-20T18:56:54.025155+0000","last_undegraded":"2024-09-20T18:56:54.025155+0000","last_fullsized":"2024-09-20T18:56:54.025155+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:55.899611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:55.899611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":73,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.007182+0000","last_change":"2024-09-20T18:55:28.832864+0000","last_active":"2024-09-20T18:56:54.007182+0000","last_peered":"2024-09-20T18:56:54.007182+0000","last_clean":"2024-09-20T18:56:54.007182+0000","last_became_active":"2024-09-20T18:55:28.832534+0000","last_became_peered":"2024-09-20T18:55:28.832534+0000","last_unstale":"2024-09-20T18:56:54.007182+0000","last_undegraded":"2024-09-20T18:56:54.007182+0000","last_fullsized":"2024-09-20T18:56:54.007182+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:52.635138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:52.635138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":319,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-20T18:57:00.099398+0000","last_change":"2024-09-20T18:51:09.761331+0000","last_active":"2024-09-20T18:57:00.099398+0000","last_peered":"2024-09-20T18:57:00.099398+0000","last_clean":"2024-09-20T18:57:00.099398+0000","last_became_active":"2024-09-20T18:49:51.968662+0000","last_became_peered":"2024-09-20T18:49:51.968662+0000","last_unstale":"2024-09-20T18:57:00.099398+0000","last_undegraded":"2024-09-20T18:57:00.099398+0000","last_fullsized":"2024-09-20T18:57:00.099398+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:09.761283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:09.761283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":307,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.028907+0000","last_change":"2024-09-20T18:49:51.968476+0000","last_active":"2024-09-20T18:56:54.028907+0000","last_peered":"2024-09-20T18:56:54.028907+0000","last_clean":"2024-09-20T18:56:54.028907+0000","last_became_active":"2024-09-20T18:49:51.968216+0000","last_became_peered":"2024-09-20T18:49:51.968216+0000","last_unstale":"2024-09-20T18:56:54.028907+0000","last_undegraded":"2024-09-20T18:56:54.028907+0000","last_fullsized":"2024-09-20T18:56:54.028907+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":335,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.035484+0000","last_change":"2024-09-20T18:55:28.295445+0000","last_active":"2024-09-20T18:56:54.035484+0000","last_peered":"2024-09-20T18:56:54.035484+0000","last_clean":"2024-09-20T18:56:54.035484+0000","last_became_active":"2024-09-20T18:49:45.937992+0000","last_became_peered":"2024-09-20T18:49:45.937992+0000","last_unstale":"2024-09-20T18:56:54.035484+0000","last_undegraded":"2024-09-20T18:56:54.035484+0000","last_fullsized":"2024-09-20T18:56:54.035484+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:28.295399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:28.295399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":124,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.031422+0000","last_change":"2024-09-20T18:55:28.852468+0000","last_active":"2024-09-20T18:56:54.031422+0000","last_peered":"2024-09-20T18:56:54.031422+0000","last_clean":"2024-09-20T18:56:54.031422+0000","last_became_active":"2024-09-20T18:55:28.834675+0000","last_became_peered":"2024-09-20T18:55:28.834675+0000","last_unstale":"2024-09-20T18:56:54.031422+0000","last_undegraded":"2024-09-20T18:56:54.031422+0000","last_fullsized":"2024-09-20T18:56:54.031422+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:54:44.719676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.719676+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":12,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":319,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-20T18:56:54.039903+0000","last_change":"2024-09-20T18:51:28.120100+0000","last_active":"2024-09-20T18:56:54.039903+0000","last_peered":"2024-09-20T18:56:54.039903+0000","last_clean":"2024-09-20T18:56:54.039903+0000","last_became_active":"2024-09-20T18:49:49.957582+0000","last_became_peered":"2024-09-20T18:49:49.957582+0000","last_unstale":"2024-09-20T18:56:54.039903+0000","last_undegraded":"2024-09-20T18:56:54.039903+0000","last_fullsized":"2024-09-20T18:56:54.039903+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:28.119694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:28.119694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":5078253592,"num_objects":77489,"num_object_clones":0,"num_object_copies":232467,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1012,"num_objects_misplaced":5783,"num_objects_unfound":0,"num_objects_dirty":77489,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":154689,"num_write_kb":7429633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21264,"num_bytes_recovered":1393557504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16476418048,"data_stored":16476405832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74222,"ondisk_log_size":74222,"up":48,"acting":45,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":67,"ondisk_log_size":67,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":695,"num_read_kb":452,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":1474,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":12140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":394,"ondisk_log_size":394,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":5275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":430,"num_read_kb":425,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266779,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1674844,"kb_used_data":1490856,"kb_used_omap":1,"kb_used_meta":183870,"kb_avail":92070308,"statfs":{"total":95995035648,"available":94279995392,"internally_reserved":0,"allocated":1526636544,"data_stored":1526262045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":188283338},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":0,"seq":193273528418,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757350,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2737828,"kb_used_data":2485724,"kb_used_omap":1,"kb_used_meta":251902,"kb_avail":91007324,"statfs":{"total":95995035648,"available":93191499776,"internally_reserved":0,"allocated":2545381376,"data_stored":2544899720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":257948101},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986283,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1743168,"kb_used_data":1571760,"kb_used_omap":1,"kb_used_meta":171390,"kb_avail":92001984,"statfs":{"total":95995035648,"available":94210031616,"internally_reserved":0,"allocated":1609482240,"data_stored":1608981112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":175503813},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182514,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2461312,"kb_used_data":2114992,"kb_used_omap":1,"kb_used_meta":346302,"kb_avail":91283840,"statfs":{"total":95995035648,"available":93474652160,"internally_reserved":0,"allocated":2165751808,"data_stored":2165249634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":354613703},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411447,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2951396,"kb_used_data":2595284,"kb_used_omap":1,"kb_used_meta":356030,"kb_avail":90793756,"statfs":{"total":95995035648,"available":92972806144,"internally_reserved":0,"allocated":2657570816,"data_stored":2657097601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":364575173},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574973,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2462488,"kb_used_data":2182792,"kb_used_omap":1,"kb_used_meta":279678,"kb_avail":91282664,"statfs":{"total":95995035648,"available":93473447936,"internally_reserved":0,"allocated":2235179008,"data_stored":2234654685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":286390725},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738499,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2374656,"kb_used_data":2058864,"kb_used_omap":1,"kb_used_meta":315774,"kb_avail":91370496,"statfs":{"total":95995035648,"available":93563387904,"internally_reserved":0,"allocated":2108276736,"data_stored":2107775182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":323353029},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,6,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2093350912,"data_stored":2093350912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2222391296,"data_stored":2222391296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2655260672,"data_stored":2655256600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2150170624,"data_stored":2150170624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1606483968,"data_stored":1606483968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2533822464,"data_stored":2533818392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1692729344,"data_stored":1692729344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1522208768,"data_stored":1522204696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T18:57:05.555 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph status --format=json 2024-09-20T18:57:05.645 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:05.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:05.686 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:05.686 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:05.787 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:05.788 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T18:57:05.788 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:57:05.858 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:05.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:05.859 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:05.960 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:05.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:05.995 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:05.995 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:06.096 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:06.100 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-20T18:57:06.122 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:06.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:06.123 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:06.223 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:06.248 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:06.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:06.249 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:06.350 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:06.381 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:06.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:06.383 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:06.483 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:06.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:06 smithi142 ceph-mon[32508]: pgmap v348: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 145 active+clean; 4.5 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 495 op/s; 1083/221190 objects degraded (0.490%); 4445/221190 objects misplaced (2.010%); 9.9 MiB/s, 157 objects/s recovering 2024-09-20T18:57:06.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:06 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1028940622' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:57:06.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:06 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:57:06.520 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:06 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 1083/221190 objects degraded (0.490%), 1 pg degraded, 3 pgs undersized (PG_DEGRADED) 2024-09-20T18:57:06.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:06.521 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:06.521 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:06.549 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:06 smithi083 ceph-mon[29421]: pgmap v348: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 145 active+clean; 4.5 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 495 op/s; 1083/221190 objects degraded (0.490%); 4445/221190 objects misplaced (2.010%); 9.9 MiB/s, 157 objects/s recovering 2024-09-20T18:57:06.549 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:06 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1028940622' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:57:06.549 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:06 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:57:06.549 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:06 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 1083/221190 objects degraded (0.490%), 1 pg degraded, 3 pgs undersized (PG_DEGRADED) 2024-09-20T18:57:06.550 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:06 smithi083 ceph-mon[36260]: pgmap v348: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 145 active+clean; 4.5 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 495 op/s; 1083/221190 objects degraded (0.490%); 4445/221190 objects misplaced (2.010%); 9.9 MiB/s, 157 objects/s recovering 2024-09-20T18:57:06.550 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:06 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1028940622' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:57:06.550 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:06 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:57:06.550 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:06 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 1083/221190 objects degraded (0.490%), 1 pg degraded, 3 pgs undersized (PG_DEGRADED) 2024-09-20T18:57:06.622 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:06.637 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_historic_ops 2024-09-20T18:57:06.657 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:06.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:06.658 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:06.737 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T18:57:06.759 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:06.783 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:06.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:06.784 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:06.885 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:06.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:06.947 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:06.947 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:07.048 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:07.075 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:07.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:07.076 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:07.113 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:57:06 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:57:07.177 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:07.201 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:07.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:07.202 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:07.300 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:07 smithi083 ceph-mon[29421]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:57:07.300 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:07 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T18:57:07.300 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:07 smithi083 ceph-mon[29421]: osdmap e215: 8 total, 8 up, 7 in 2024-09-20T18:57:07.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:07 smithi083 ceph-mon[36260]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:57:07.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:07 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T18:57:07.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:07 smithi083 ceph-mon[36260]: osdmap e215: 8 total, 8 up, 7 in 2024-09-20T18:57:07.303 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:07.327 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:07.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:07.328 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:07.429 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:07.472 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:07.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:07.473 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:07.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:07 smithi142 ceph-mon[32508]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:57:07.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:07 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T18:57:07.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:07 smithi142 ceph-mon[32508]: osdmap e215: 8 total, 8 up, 7 in 2024-09-20T18:57:07.574 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:07.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:07.701 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:07.702 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:07.802 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:07.836 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:07.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:07.837 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:07.937 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:07.975 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:07.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:07.976 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:08.077 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:08.114 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:08.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:08.115 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:08.174 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:57:08.216 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:08.267 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:08.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:08.268 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:08.368 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:08.401 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:08.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:08.402 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:08.451 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:08 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:57:08.451 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:08 smithi083 ceph-mon[29421]: pgmap v350: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 145 active+clean; 4.2 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.12k op/s; 1244/206511 objects degraded (0.602%); 3464/206511 objects misplaced (1.677%); 17 MiB/s, 264 objects/s recovering 2024-09-20T18:57:08.451 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:08 smithi083 ceph-mon[29421]: osdmap e216: 8 total, 8 up, 7 in 2024-09-20T18:57:08.451 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:08 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:57:08.452 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:08 smithi083 ceph-mon[36260]: pgmap v350: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 145 active+clean; 4.2 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.12k op/s; 1244/206511 objects degraded (0.602%); 3464/206511 objects misplaced (1.677%); 17 MiB/s, 264 objects/s recovering 2024-09-20T18:57:08.452 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:08 smithi083 ceph-mon[36260]: osdmap e216: 8 total, 8 up, 7 in 2024-09-20T18:57:08.503 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:08.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:08 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:57:08.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:08 smithi142 ceph-mon[32508]: pgmap v350: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 145 active+clean; 4.2 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.12k op/s; 1244/206511 objects degraded (0.602%); 3464/206511 objects misplaced (1.677%); 17 MiB/s, 264 objects/s recovering 2024-09-20T18:57:08.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:08 smithi142 ceph-mon[32508]: osdmap e216: 8 total, 8 up, 7 in 2024-09-20T18:57:08.525 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:08.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:08.526 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:08.627 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:08.649 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:08.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:08.650 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:08.708 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:57:08.752 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:08.778 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:08.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:08.779 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:08.880 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:08.902 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:08.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:08.903 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:09.004 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:09.007 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:57:09.007 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"1ed5a50a-7780-11ef-baf5-efdc52797490","health":{"status":"HEALTH_WARN","checks":{"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"nodeep-scrub flag(s) set","count":12},"muted":false},"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 1244/206511 objects degraded (0.602%), 1 pg degraded, 3 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":690,"monmap":{"epoch":3,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":216,"num_osds":8,"num_up_osds":8,"osd_up_since":1726858174,"num_in_osds":7,"osd_in_since":1726858527,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":145},{"state_name":"active+recovering+undersized+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":148,"num_pools":7,"num_objects":68837,"data_bytes":4498790963,"bytes_used":15336038400,"bytes_avail":656629211136,"bytes_total":671965249536,"degraded_objects":1244,"degraded_total":206511,"degraded_ratio":0.0060238921897622887,"misplaced_objects":3464,"misplaced_total":206511,"misplaced_ratio":0.016773924875672482,"recovering_objects_per_sec":264,"recovering_bytes_per_sec":17351191,"recovering_keys_per_sec":0,"num_objects_recovered":2084,"num_bytes_recovered":136577024,"num_keys_recovered":0,"read_bytes_sec":1300,"write_bytes_sec":0,"read_op_per_sec":1,"write_op_per_sec":1124},"fsmap":{"epoch":1,"btime":"2024-09-20T18:44:10:399616+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"services":{"dashboard":"https://172.21.15.83:8443/","prometheus":"http://172.21.15.83:9283/"}},"servicemap":{"epoch":14,"modified":"2024-09-20T18:51:16.768808+0000","services":{"rgw":{"daemons":{"summary":"","14460":{"start_epoch":12,"start_stamp":"2024-09-20T18:49:53.053671+0000","gid":14460,"addr":"172.21.15.83:0/1555339426","metadata":{"arch":"x86_64","ceph_release":"squid","ceph_version":"ceph version 19.3.0-5099-g88deadb8 (88deadb8082f4a33832e228ccefc59cedba158a5) squid (dev)","ceph_version_short":"19.3.0-5099-g88deadb8","container_hostname":"smithi083","container_image":"quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2","cpu":"Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz","distro":"centos","distro_description":"CentOS Stream 9","distro_version":"9","frontend_config#0":"beast port=80","frontend_type#0":"beast","hostname":"smithi083","id":"foo.a","kernel_description":"#1 SMP PREEMPT_DYNAMIC Thu Sep 12 15:49:37 UTC 2024","kernel_version":"5.14.0-508.el9.x86_64","mem_swap_kb":"0","mem_total_kb":"32487292","num_handles":"1","os":"Linux","pid":"2","realm_id":"","realm_name":"","zone_id":"41426759-9535-4245-bcfe-8d158f74a134","zone_name":"default","zonegroup_id":"fb472636-52d3-4587-9964-925ef38835b6","zonegroup_name":"default"},"task_status":{}}}}}},"progress_events":{}} 2024-09-20T18:57:09.028 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:09.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:09.029 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:09.130 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:09.154 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:09.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:09.155 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:09.256 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:09.288 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:09.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:09.289 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:09.381 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:09 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1664194521' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-09-20T18:57:09.381 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:09 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1664194521' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-09-20T18:57:09.390 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:09.432 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:09.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:09.433 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:09.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:09 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1664194521' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-09-20T18:57:09.534 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:09.543 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T18:57:09.543 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:57:09.559 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:09.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:09.560 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:09.661 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:09.686 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:09.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:09.688 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:09.788 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:09.813 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:09.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:09.814 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:09.915 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:09.950 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:09.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:09.951 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:10.052 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:10.080 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:10.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:10.081 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:10.176 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T18:57:10.178 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:57:10.181 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:10.215 INFO:tasks.cephadm.ceph_manager.ceph:making progress, resetting timeout 2024-09-20T18:57:10.215 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T18:57:10.304 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:10.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:10.305 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:10.406 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:10.420 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:10 smithi142 ceph-mon[32508]: 5.1b scrub starts 2024-09-20T18:57:10.420 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:10 smithi142 ceph-mon[32508]: 5.1b scrub ok 2024-09-20T18:57:10.420 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:10 smithi142 ceph-mon[32508]: pgmap v352: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 145 active+clean; 4.2 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 1.11k op/s; 1244/206511 objects degraded (0.602%); 3464/206511 objects misplaced (1.677%); 16 MiB/s, 260 objects/s recovering 2024-09-20T18:57:10.420 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:10 smithi142 ceph-mon[32508]: osdmap e217: 8 total, 8 up, 7 in 2024-09-20T18:57:10.420 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:10 smithi142 ceph-mon[32508]: 3.4 scrub starts 2024-09-20T18:57:10.420 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:10 smithi142 ceph-mon[32508]: 3.4 scrub ok 2024-09-20T18:57:10.428 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:10.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:10.429 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:10.491 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:10 smithi083 ceph-mon[36260]: 5.1b scrub starts 2024-09-20T18:57:10.491 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:10 smithi083 ceph-mon[36260]: 5.1b scrub ok 2024-09-20T18:57:10.491 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:10 smithi083 ceph-mon[36260]: pgmap v352: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 145 active+clean; 4.2 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 1.11k op/s; 1244/206511 objects degraded (0.602%); 3464/206511 objects misplaced (1.677%); 16 MiB/s, 260 objects/s recovering 2024-09-20T18:57:10.491 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:10 smithi083 ceph-mon[36260]: osdmap e217: 8 total, 8 up, 7 in 2024-09-20T18:57:10.492 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:10 smithi083 ceph-mon[36260]: 3.4 scrub starts 2024-09-20T18:57:10.492 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:10 smithi083 ceph-mon[36260]: 3.4 scrub ok 2024-09-20T18:57:10.492 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:10 smithi083 ceph-mon[29421]: 5.1b scrub starts 2024-09-20T18:57:10.492 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:10 smithi083 ceph-mon[29421]: 5.1b scrub ok 2024-09-20T18:57:10.492 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:10 smithi083 ceph-mon[29421]: pgmap v352: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 145 active+clean; 4.2 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 1.11k op/s; 1244/206511 objects degraded (0.602%); 3464/206511 objects misplaced (1.677%); 16 MiB/s, 260 objects/s recovering 2024-09-20T18:57:10.492 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:10 smithi083 ceph-mon[29421]: osdmap e217: 8 total, 8 up, 7 in 2024-09-20T18:57:10.492 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:10 smithi083 ceph-mon[29421]: 3.4 scrub starts 2024-09-20T18:57:10.492 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:10 smithi083 ceph-mon[29421]: 3.4 scrub ok 2024-09-20T18:57:10.530 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:10.555 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:10.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:10.557 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:10.658 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:10.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:10.703 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:10.703 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:10.799 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:57:10 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:57:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:57:10.804 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:10.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:10.836 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:10.836 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:10.937 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:10.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:10.974 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:10.974 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:11.007 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-20T18:57:11.074 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:57:11.102 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:11.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:11.103 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:57:11.203 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:11.269 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:11 smithi142 ceph-mon[32508]: 4.1d scrub starts 2024-09-20T18:57:11.269 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:11 smithi142 ceph-mon[32508]: 4.1d scrub ok 2024-09-20T18:57:11.270 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:11 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:57:11.270 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:11 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:57:11.270 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:11 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 1244/206511 objects degraded (0.602%), 1 pg degraded, 3 pgs undersized (PG_DEGRADED) 2024-09-20T18:57:11.270 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:11 smithi142 ceph-mon[32508]: 6.1d scrub starts 2024-09-20T18:57:11.270 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:11 smithi142 ceph-mon[32508]: 6.1d scrub ok 2024-09-20T18:57:11.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:11.279 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:11.279 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:11.287 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:11 smithi083 ceph-mon[29421]: 4.1d scrub starts 2024-09-20T18:57:11.287 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:11 smithi083 ceph-mon[29421]: 4.1d scrub ok 2024-09-20T18:57:11.287 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:11 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:57:11.287 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:11 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:57:11.287 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:11 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 1244/206511 objects degraded (0.602%), 1 pg degraded, 3 pgs undersized (PG_DEGRADED) 2024-09-20T18:57:11.288 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:11 smithi083 ceph-mon[29421]: 6.1d scrub starts 2024-09-20T18:57:11.288 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:11 smithi083 ceph-mon[29421]: 6.1d scrub ok 2024-09-20T18:57:11.288 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:11 smithi083 ceph-mon[36260]: 4.1d scrub starts 2024-09-20T18:57:11.288 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:11 smithi083 ceph-mon[36260]: 4.1d scrub ok 2024-09-20T18:57:11.288 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:11 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:57:11.288 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:11 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:57:11.288 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:11 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 1244/206511 objects degraded (0.602%), 1 pg degraded, 3 pgs undersized (PG_DEGRADED) 2024-09-20T18:57:11.288 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:11 smithi083 ceph-mon[36260]: 6.1d scrub starts 2024-09-20T18:57:11.288 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:11 smithi083 ceph-mon[36260]: 6.1d scrub ok 2024-09-20T18:57:11.290 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:57:11.290 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:57:11.313 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:57:11.331 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:57:11.331 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:57:11.332 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:57:11.375 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:57:11.376 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:57:11.380 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:11.405 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:11.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:11.406 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:11.415 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:57:11.415 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:57:11.448 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:57:11.448 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:57:11.477 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:57:11.477 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:57:11.507 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:11.516 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:57:11.516 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:57:11.548 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:57:11.548 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:57:11.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:11.561 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:11.562 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:11.662 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:11.691 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:11.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:11.692 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:11.793 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:11.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:11.821 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:11.821 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:11.922 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:11.950 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:11.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:11.951 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:12.051 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:12.080 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:12.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:12.081 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:12.182 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:12.219 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:12.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:12.220 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:12.269 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:12 smithi142 ceph-mon[32508]: pgmap v354: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 145 active+clean; 4.2 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 816 op/s; 1244/206511 objects degraded (0.602%); 3464/206511 objects misplaced (1.677%); 8.5 MiB/s, 136 objects/s recovering 2024-09-20T18:57:12.269 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:12 smithi142 ceph-mon[32508]: osdmap e218: 8 total, 8 up, 7 in 2024-09-20T18:57:12.269 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:12 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3054141647' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:57:12.269 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:12 smithi142 ceph-mon[32508]: 5.12 scrub starts 2024-09-20T18:57:12.270 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:12 smithi142 ceph-mon[32508]: 5.12 scrub ok 2024-09-20T18:57:12.299 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:12 smithi083 ceph-mon[29421]: pgmap v354: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 145 active+clean; 4.2 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 816 op/s; 1244/206511 objects degraded (0.602%); 3464/206511 objects misplaced (1.677%); 8.5 MiB/s, 136 objects/s recovering 2024-09-20T18:57:12.299 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:12 smithi083 ceph-mon[29421]: osdmap e218: 8 total, 8 up, 7 in 2024-09-20T18:57:12.300 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:12 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3054141647' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:57:12.300 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:12 smithi083 ceph-mon[29421]: 5.12 scrub starts 2024-09-20T18:57:12.300 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:12 smithi083 ceph-mon[29421]: 5.12 scrub ok 2024-09-20T18:57:12.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:12 smithi083 ceph-mon[36260]: pgmap v354: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 145 active+clean; 4.2 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 816 op/s; 1244/206511 objects degraded (0.602%); 3464/206511 objects misplaced (1.677%); 8.5 MiB/s, 136 objects/s recovering 2024-09-20T18:57:12.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:12 smithi083 ceph-mon[36260]: osdmap e218: 8 total, 8 up, 7 in 2024-09-20T18:57:12.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:12 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3054141647' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:57:12.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:12 smithi083 ceph-mon[36260]: 5.12 scrub starts 2024-09-20T18:57:12.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:12 smithi083 ceph-mon[36260]: 5.12 scrub ok 2024-09-20T18:57:12.321 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:12.362 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:12.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:12.363 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:12.464 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:57:12.489 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:12.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:12.490 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:57:12.591 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:12.617 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:12.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:12.617 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:12.718 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:12.756 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:12.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:12.757 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:12.858 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:12.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:12.883 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:12.883 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:12.984 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:13.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:13.020 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:13.020 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:13.121 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:57:13.129 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:57:13.144 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:13.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:13.145 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:57:13.245 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:13.269 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:13.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:13.270 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:13.370 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:13.397 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:13.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:13.398 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:13.496 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:57:13.499 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-20T18:57:13.500 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:13.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:13.543 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:13.544 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:13.645 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:13.672 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:13.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:13.673 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:13.774 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:13.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:13.810 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:13.810 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:13.911 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:57:13.938 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:13.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:13.942 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:57:14.043 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:14.068 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:14.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:14.069 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:14.167 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-20T18:57:14.169 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:14.217 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:14.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:14.218 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:14.319 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:14.342 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:14.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:14.343 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:14.399 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T18:57:14.434 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:14 smithi083 ceph-mon[29421]: 6.6 scrub starts 2024-09-20T18:57:14.434 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:14 smithi083 ceph-mon[29421]: 6.6 scrub ok 2024-09-20T18:57:14.434 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:14 smithi083 ceph-mon[29421]: pgmap v356: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 146 active+clean; 3.9 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 697 op/s; 1360/193971 objects degraded (0.701%); 2638/193971 objects misplaced (1.360%); 7.4 MiB/s, 118 objects/s recovering 2024-09-20T18:57:14.434 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:14 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/874343179' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:57:14.434 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:14 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:57:14.434 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:14 smithi083 ceph-mon[29421]: from='client.25171 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:57:14.435 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:14 smithi083 ceph-mon[36260]: 6.6 scrub starts 2024-09-20T18:57:14.435 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:14 smithi083 ceph-mon[36260]: 6.6 scrub ok 2024-09-20T18:57:14.435 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:14 smithi083 ceph-mon[36260]: pgmap v356: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 146 active+clean; 3.9 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 697 op/s; 1360/193971 objects degraded (0.701%); 2638/193971 objects misplaced (1.360%); 7.4 MiB/s, 118 objects/s recovering 2024-09-20T18:57:14.435 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:14 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/874343179' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:57:14.435 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:14 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:57:14.435 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:14 smithi083 ceph-mon[36260]: from='client.25171 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:57:14.444 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:14.476 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:14.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:14.477 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:14.522 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:14 smithi142 ceph-mon[32508]: 6.6 scrub starts 2024-09-20T18:57:14.522 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:14 smithi142 ceph-mon[32508]: 6.6 scrub ok 2024-09-20T18:57:14.522 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:14 smithi142 ceph-mon[32508]: pgmap v356: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 146 active+clean; 3.9 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 697 op/s; 1360/193971 objects degraded (0.701%); 2638/193971 objects misplaced (1.360%); 7.4 MiB/s, 118 objects/s recovering 2024-09-20T18:57:14.522 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:14 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/874343179' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:57:14.523 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:14 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:57:14.523 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:14 smithi142 ceph-mon[32508]: from='client.25171 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:57:14.578 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:14.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:14.603 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:14.603 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:14.704 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:57:14.726 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:14.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:14.727 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:57:14.828 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:14.851 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:14.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:14.852 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:14.887 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":356,"stamp":"2024-09-20T18:57:12.922692+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4224850483,"num_objects":64657,"num_object_clones":0,"num_object_copies":193971,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1360,"num_objects_misplaced":2638,"num_objects_unfound":0,"num_objects_dirty":64657,"num_whiteouts":0,"num_read":1286,"num_read_kb":1143,"num_write":168269,"num_write_kb":7431306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24153,"num_bytes_recovered":1577847753,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74809,"ondisk_log_size":74809,"up":444,"acting":442,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":14224796,"kb_used_data":12282012,"kb_used_omap":10,"kb_used_meta":1942453,"kb_avail":641991268,"statfs":{"total":671965249536,"available":657399058432,"internally_reserved":0,"allocated":12576780288,"data_stored":12573986955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11158,"internal_metadata":1989071978},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-273940480,"num_objects":-4180,"num_object_clones":0,"num_object_copies":-12540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":116,"num_objects_misplaced":-826,"num_objects_unfound":0,"num_objects_dirty":-4180,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4180,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":710,"num_bytes_recovered":46530560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002058"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":355,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.140898+0000","last_change":"2024-09-20T18:55:28.037494+0000","last_active":"2024-09-20T18:57:11.140898+0000","last_peered":"2024-09-20T18:57:11.140898+0000","last_clean":"2024-09-20T18:57:11.140898+0000","last_became_active":"2024-09-20T18:49:45.943201+0000","last_became_peered":"2024-09-20T18:49:45.943201+0000","last_unstale":"2024-09-20T18:57:11.140898+0000","last_undegraded":"2024-09-20T18:57:11.140898+0000","last_fullsized":"2024-09-20T18:57:11.140898+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:28.037449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:28.037449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":400,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.114635+0000","last_change":"2024-09-20T18:55:28.838729+0000","last_active":"2024-09-20T18:57:09.114635+0000","last_peered":"2024-09-20T18:57:09.114635+0000","last_clean":"2024-09-20T18:57:09.114635+0000","last_became_active":"2024-09-20T18:55:28.819637+0000","last_became_peered":"2024-09-20T18:55:28.819637+0000","last_unstale":"2024-09-20T18:57:09.114635+0000","last_undegraded":"2024-09-20T18:57:09.114635+0000","last_fullsized":"2024-09-20T18:57:09.114635+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:50.231620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:50.231620+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":348,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.152057+0000","last_change":"2024-09-20T18:54:25.361988+0000","last_active":"2024-09-20T18:57:11.152057+0000","last_peered":"2024-09-20T18:57:11.152057+0000","last_clean":"2024-09-20T18:57:11.152057+0000","last_became_active":"2024-09-20T18:49:49.959514+0000","last_became_peered":"2024-09-20T18:49:49.959514+0000","last_unstale":"2024-09-20T18:57:11.152057+0000","last_undegraded":"2024-09-20T18:57:11.152057+0000","last_fullsized":"2024-09-20T18:57:11.152057+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:25.361935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:25.361935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":341,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.114833+0000","last_change":"2024-09-20T18:54:15.221404+0000","last_active":"2024-09-20T18:57:09.114833+0000","last_peered":"2024-09-20T18:57:09.114833+0000","last_clean":"2024-09-20T18:57:09.114833+0000","last_became_active":"2024-09-20T18:49:51.969421+0000","last_became_peered":"2024-09-20T18:49:51.969421+0000","last_unstale":"2024-09-20T18:57:09.114833+0000","last_undegraded":"2024-09-20T18:57:09.114833+0000","last_fullsized":"2024-09-20T18:57:09.114833+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:15.221342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:15.221342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":350,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.153068+0000","last_change":"2024-09-20T18:55:28.822034+0000","last_active":"2024-09-20T18:57:11.153068+0000","last_peered":"2024-09-20T18:57:11.153068+0000","last_clean":"2024-09-20T18:57:11.153068+0000","last_became_active":"2024-09-20T18:55:28.821717+0000","last_became_peered":"2024-09-20T18:55:28.821717+0000","last_unstale":"2024-09-20T18:57:11.153068+0000","last_undegraded":"2024-09-20T18:57:11.153068+0000","last_fullsized":"2024-09-20T18:57:11.153068+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.216188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.216188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":347,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.151091+0000","last_change":"2024-09-20T18:55:28.825144+0000","last_active":"2024-09-20T18:57:11.151091+0000","last_peered":"2024-09-20T18:57:11.151091+0000","last_clean":"2024-09-20T18:57:11.151091+0000","last_became_active":"2024-09-20T18:55:28.824826+0000","last_became_peered":"2024-09-20T18:55:28.824826+0000","last_unstale":"2024-09-20T18:57:11.151091+0000","last_undegraded":"2024-09-20T18:57:11.151091+0000","last_fullsized":"2024-09-20T18:57:11.151091+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:25.208073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:25.208073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":361,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.151365+0000","last_change":"2024-09-20T18:51:07.130137+0000","last_active":"2024-09-20T18:57:11.151365+0000","last_peered":"2024-09-20T18:57:11.151365+0000","last_clean":"2024-09-20T18:57:11.151365+0000","last_became_active":"2024-09-20T18:49:47.952150+0000","last_became_peered":"2024-09-20T18:49:47.952150+0000","last_unstale":"2024-09-20T18:57:11.151365+0000","last_undegraded":"2024-09-20T18:57:11.151365+0000","last_fullsized":"2024-09-20T18:57:11.151365+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:51:07.130062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:07.130062+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":345,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.114598+0000","last_change":"2024-09-20T18:55:28.820753+0000","last_active":"2024-09-20T18:57:09.114598+0000","last_peered":"2024-09-20T18:57:09.114598+0000","last_clean":"2024-09-20T18:57:09.114598+0000","last_became_active":"2024-09-20T18:55:28.820362+0000","last_became_peered":"2024-09-20T18:55:28.820362+0000","last_unstale":"2024-09-20T18:57:09.114598+0000","last_undegraded":"2024-09-20T18:57:09.114598+0000","last_fullsized":"2024-09-20T18:57:09.114598+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:11.246922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:11.246922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"59'1","reported_seq":377,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.119168+0000","last_change":"2024-09-20T18:55:29.340140+0000","last_active":"2024-09-20T18:57:09.119168+0000","last_peered":"2024-09-20T18:57:09.119168+0000","last_clean":"2024-09-20T18:57:09.119168+0000","last_became_active":"2024-09-20T18:55:28.833386+0000","last_became_peered":"2024-09-20T18:55:28.833386+0000","last_unstale":"2024-09-20T18:57:09.119168+0000","last_undegraded":"2024-09-20T18:57:09.119168+0000","last_fullsized":"2024-09-20T18:57:09.119168+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:55:29.340083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:29.340083+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":365,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.115068+0000","last_change":"2024-09-20T18:54:53.241889+0000","last_active":"2024-09-20T18:57:09.115068+0000","last_peered":"2024-09-20T18:57:09.115068+0000","last_clean":"2024-09-20T18:57:09.115068+0000","last_became_active":"2024-09-20T18:49:47.947448+0000","last_became_peered":"2024-09-20T18:49:47.947448+0000","last_unstale":"2024-09-20T18:57:09.115068+0000","last_undegraded":"2024-09-20T18:57:09.115068+0000","last_fullsized":"2024-09-20T18:57:09.115068+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:53.241802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:53.241802+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":344,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.118801+0000","last_change":"2024-09-20T18:57:08.539466+0000","last_active":"2024-09-20T18:57:09.118801+0000","last_peered":"2024-09-20T18:57:09.118801+0000","last_clean":"2024-09-20T18:57:09.118801+0000","last_became_active":"2024-09-20T18:49:49.961188+0000","last_became_peered":"2024-09-20T18:49:49.961188+0000","last_unstale":"2024-09-20T18:57:09.118801+0000","last_undegraded":"2024-09-20T18:57:09.118801+0000","last_fullsized":"2024-09-20T18:57:09.118801+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:08.539418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:08.539418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:13.102215+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":319,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.139387+0000","last_change":"2024-09-20T18:49:51.966957+0000","last_active":"2024-09-20T18:57:11.139387+0000","last_peered":"2024-09-20T18:57:11.139387+0000","last_clean":"2024-09-20T18:57:11.139387+0000","last_became_active":"2024-09-20T18:49:51.966715+0000","last_became_peered":"2024-09-20T18:49:51.966715+0000","last_unstale":"2024-09-20T18:57:11.139387+0000","last_undegraded":"2024-09-20T18:57:11.139387+0000","last_fullsized":"2024-09-20T18:57:11.139387+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":344,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.119042+0000","last_change":"2024-09-20T18:55:15.252839+0000","last_active":"2024-09-20T18:57:09.119042+0000","last_peered":"2024-09-20T18:57:09.119042+0000","last_clean":"2024-09-20T18:57:09.119042+0000","last_became_active":"2024-09-20T18:49:45.944703+0000","last_became_peered":"2024-09-20T18:49:45.944703+0000","last_unstale":"2024-09-20T18:57:09.119042+0000","last_undegraded":"2024-09-20T18:57:09.119042+0000","last_fullsized":"2024-09-20T18:57:09.119042+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:15.252782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:15.252782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":369,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.114823+0000","last_change":"2024-09-20T18:51:26.494194+0000","last_active":"2024-09-20T18:57:09.114823+0000","last_peered":"2024-09-20T18:57:09.114823+0000","last_clean":"2024-09-20T18:57:09.114823+0000","last_became_active":"2024-09-20T18:51:23.813399+0000","last_became_peered":"2024-09-20T18:51:23.813399+0000","last_unstale":"2024-09-20T18:57:09.114823+0000","last_undegraded":"2024-09-20T18:57:09.114823+0000","last_fullsized":"2024-09-20T18:57:09.114823+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-20T18:51:26.494118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:26.494118+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":336,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.149999+0000","last_change":"2024-09-20T18:52:38.799542+0000","last_active":"2024-09-20T18:57:09.149999+0000","last_peered":"2024-09-20T18:57:09.149999+0000","last_clean":"2024-09-20T18:57:09.149999+0000","last_became_active":"2024-09-20T18:49:49.959696+0000","last_became_peered":"2024-09-20T18:49:49.959696+0000","last_unstale":"2024-09-20T18:57:09.149999+0000","last_undegraded":"2024-09-20T18:57:09.149999+0000","last_fullsized":"2024-09-20T18:57:09.149999+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:38.799487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:38.799487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":333,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.118982+0000","last_change":"2024-09-20T18:53:21.273835+0000","last_active":"2024-09-20T18:57:09.118982+0000","last_peered":"2024-09-20T18:57:09.118982+0000","last_clean":"2024-09-20T18:57:09.118982+0000","last_became_active":"2024-09-20T18:49:51.970663+0000","last_became_peered":"2024-09-20T18:49:51.970663+0000","last_unstale":"2024-09-20T18:57:09.118982+0000","last_undegraded":"2024-09-20T18:57:09.118982+0000","last_fullsized":"2024-09-20T18:57:09.118982+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:21.273788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:21.273788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":341,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.115248+0000","last_change":"2024-09-20T18:55:28.828417+0000","last_active":"2024-09-20T18:57:09.115248+0000","last_peered":"2024-09-20T18:57:09.115248+0000","last_clean":"2024-09-20T18:57:09.115248+0000","last_became_active":"2024-09-20T18:55:28.827435+0000","last_became_peered":"2024-09-20T18:55:28.827435+0000","last_unstale":"2024-09-20T18:57:09.115248+0000","last_undegraded":"2024-09-20T18:57:09.115248+0000","last_fullsized":"2024-09-20T18:57:09.115248+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:53.940191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:53.940191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":346,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.140531+0000","last_change":"2024-09-20T18:54:23.984024+0000","last_active":"2024-09-20T18:57:11.140531+0000","last_peered":"2024-09-20T18:57:11.140531+0000","last_clean":"2024-09-20T18:57:11.140531+0000","last_became_active":"2024-09-20T18:49:45.950480+0000","last_became_peered":"2024-09-20T18:49:45.950480+0000","last_unstale":"2024-09-20T18:57:11.140531+0000","last_undegraded":"2024-09-20T18:57:11.140531+0000","last_fullsized":"2024-09-20T18:57:11.140531+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:23.983958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:23.983958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":369,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.155067+0000","last_change":"2024-09-20T18:53:26.762207+0000","last_active":"2024-09-20T18:57:11.155067+0000","last_peered":"2024-09-20T18:57:11.155067+0000","last_clean":"2024-09-20T18:57:11.155067+0000","last_became_active":"2024-09-20T18:49:47.950939+0000","last_became_peered":"2024-09-20T18:49:47.950939+0000","last_unstale":"2024-09-20T18:57:11.155067+0000","last_undegraded":"2024-09-20T18:57:11.155067+0000","last_fullsized":"2024-09-20T18:57:11.155067+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:53:26.762131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:26.762131+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":331,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.153002+0000","last_change":"2024-09-20T18:51:17.248705+0000","last_active":"2024-09-20T18:57:11.153002+0000","last_peered":"2024-09-20T18:57:11.153002+0000","last_clean":"2024-09-20T18:57:11.153002+0000","last_became_active":"2024-09-20T18:49:49.958518+0000","last_became_peered":"2024-09-20T18:49:49.958518+0000","last_unstale":"2024-09-20T18:57:11.153002+0000","last_undegraded":"2024-09-20T18:57:11.153002+0000","last_fullsized":"2024-09-20T18:57:11.153002+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:17.248634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:17.248634+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":343,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.157943+0000","last_change":"2024-09-20T18:55:28.833468+0000","last_active":"2024-09-20T18:57:11.157943+0000","last_peered":"2024-09-20T18:57:11.157943+0000","last_clean":"2024-09-20T18:57:11.157943+0000","last_became_active":"2024-09-20T18:55:28.833133+0000","last_became_peered":"2024-09-20T18:55:28.833133+0000","last_unstale":"2024-09-20T18:57:11.157943+0000","last_undegraded":"2024-09-20T18:57:11.157943+0000","last_fullsized":"2024-09-20T18:57:11.157943+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:54.259174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:54.259174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"59'1","reported_seq":351,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.115235+0000","last_change":"2024-09-20T18:53:42.840884+0000","last_active":"2024-09-20T18:57:09.115235+0000","last_peered":"2024-09-20T18:57:09.115235+0000","last_clean":"2024-09-20T18:57:09.115235+0000","last_became_active":"2024-09-20T18:49:45.940100+0000","last_became_peered":"2024-09-20T18:49:45.940100+0000","last_unstale":"2024-09-20T18:57:09.115235+0000","last_undegraded":"2024-09-20T18:57:09.115235+0000","last_fullsized":"2024-09-20T18:57:09.115235+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:53:42.840800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:42.840800+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":358,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.151590+0000","last_change":"2024-09-20T18:57:09.819080+0000","last_active":"2024-09-20T18:57:11.151590+0000","last_peered":"2024-09-20T18:57:11.151590+0000","last_clean":"2024-09-20T18:57:11.151590+0000","last_became_active":"2024-09-20T18:49:47.950909+0000","last_became_peered":"2024-09-20T18:49:47.950909+0000","last_unstale":"2024-09-20T18:57:11.151590+0000","last_undegraded":"2024-09-20T18:57:11.151590+0000","last_fullsized":"2024-09-20T18:57:11.151590+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:57:09.818999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:09.818999+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:17.216750+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":345,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.114915+0000","last_change":"2024-09-20T18:54:44.239273+0000","last_active":"2024-09-20T18:57:09.114915+0000","last_peered":"2024-09-20T18:57:09.114915+0000","last_clean":"2024-09-20T18:57:09.114915+0000","last_became_active":"2024-09-20T18:49:49.959102+0000","last_became_peered":"2024-09-20T18:49:49.959102+0000","last_unstale":"2024-09-20T18:57:09.114915+0000","last_undegraded":"2024-09-20T18:57:09.114915+0000","last_fullsized":"2024-09-20T18:57:09.114915+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:44.239226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.239226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":333,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.150218+0000","last_change":"2024-09-20T18:52:15.826819+0000","last_active":"2024-09-20T18:57:09.150218+0000","last_peered":"2024-09-20T18:57:09.150218+0000","last_clean":"2024-09-20T18:57:09.150218+0000","last_became_active":"2024-09-20T18:49:51.970058+0000","last_became_peered":"2024-09-20T18:49:51.970058+0000","last_unstale":"2024-09-20T18:57:09.150218+0000","last_undegraded":"2024-09-20T18:57:09.150218+0000","last_fullsized":"2024-09-20T18:57:09.150218+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T18:52:15.826638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:15.826638+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":341,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.152631+0000","last_change":"2024-09-20T18:52:47.158384+0000","last_active":"2024-09-20T18:57:11.152631+0000","last_peered":"2024-09-20T18:57:11.152631+0000","last_clean":"2024-09-20T18:57:11.152631+0000","last_became_active":"2024-09-20T18:49:45.933078+0000","last_became_peered":"2024-09-20T18:49:45.933078+0000","last_unstale":"2024-09-20T18:57:11.152631+0000","last_undegraded":"2024-09-20T18:57:11.152631+0000","last_fullsized":"2024-09-20T18:57:11.152631+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:52:47.158260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:47.158260+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":366,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.159748+0000","last_change":"2024-09-20T18:55:19.996522+0000","last_active":"2024-09-20T18:57:11.159748+0000","last_peered":"2024-09-20T18:57:11.159748+0000","last_clean":"2024-09-20T18:57:11.159748+0000","last_became_active":"2024-09-20T18:49:48.500809+0000","last_became_peered":"2024-09-20T18:49:48.500809+0000","last_unstale":"2024-09-20T18:57:11.159748+0000","last_undegraded":"2024-09-20T18:57:11.159748+0000","last_fullsized":"2024-09-20T18:57:11.159748+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:55:19.996328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:19.996328+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"184'9","reported_seq":136,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.115295+0000","last_change":"2024-09-20T18:55:28.845188+0000","last_active":"2024-09-20T18:57:09.115295+0000","last_peered":"2024-09-20T18:57:09.115295+0000","last_clean":"2024-09-20T18:57:09.115295+0000","last_became_active":"2024-09-20T18:55:28.820941+0000","last_became_peered":"2024-09-20T18:55:28.820941+0000","last_unstale":"2024-09-20T18:57:09.115295+0000","last_undegraded":"2024-09-20T18:57:09.115295+0000","last_fullsized":"2024-09-20T18:57:09.115295+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:53:44.524040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:44.524040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3","version":"218'10309","reported_seq":22769,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.245881+0000","last_change":"2024-09-20T18:56:25.697520+0000","last_active":"2024-09-20T18:57:11.245881+0000","last_peered":"2024-09-20T18:57:11.245881+0000","last_clean":"2024-09-20T18:57:11.245881+0000","last_became_active":"2024-09-20T18:56:25.687391+0000","last_became_peered":"2024-09-20T18:56:25.687391+0000","last_unstale":"2024-09-20T18:57:11.245881+0000","last_undegraded":"2024-09-20T18:57:11.245881+0000","last_fullsized":"2024-09-20T18:57:11.245881+0000","mapping_epoch":203,"log_start":"189'6072","ondisk_log_start":"189'6072","created":160,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4237,"log_dups_size":0,"ondisk_log_size":4237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":261292032,"num_objects":3987,"num_object_clones":0,"num_object_copies":11961,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3987,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10309,"num_write_kb":457472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6156,"num_bytes_recovered":403439616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":361,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.160477+0000","last_change":"2024-09-20T18:55:18.581290+0000","last_active":"2024-09-20T18:57:11.160477+0000","last_peered":"2024-09-20T18:57:11.160477+0000","last_clean":"2024-09-20T18:57:11.160477+0000","last_became_active":"2024-09-20T18:49:58.992302+0000","last_became_peered":"2024-09-20T18:49:58.992302+0000","last_unstale":"2024-09-20T18:57:11.160477+0000","last_undegraded":"2024-09-20T18:57:11.160477+0000","last_fullsized":"2024-09-20T18:57:11.160477+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:55:18.581200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:18.581200+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":348,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.156363+0000","last_change":"2024-09-20T18:54:54.288199+0000","last_active":"2024-09-20T18:57:11.156363+0000","last_peered":"2024-09-20T18:57:11.156363+0000","last_clean":"2024-09-20T18:57:11.156363+0000","last_became_active":"2024-09-20T18:49:45.936104+0000","last_became_peered":"2024-09-20T18:49:45.936104+0000","last_unstale":"2024-09-20T18:57:11.156363+0000","last_undegraded":"2024-09-20T18:57:11.156363+0000","last_fullsized":"2024-09-20T18:57:11.156363+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:54.288117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:54.288117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":46,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":438,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.115323+0000","last_change":"2024-09-20T18:54:14.292660+0000","last_active":"2024-09-20T18:57:09.115323+0000","last_peered":"2024-09-20T18:57:09.115323+0000","last_clean":"2024-09-20T18:57:09.115323+0000","last_became_active":"2024-09-20T18:49:49.960697+0000","last_became_peered":"2024-09-20T18:49:49.960697+0000","last_unstale":"2024-09-20T18:57:09.115323+0000","last_undegraded":"2024-09-20T18:57:09.115323+0000","last_fullsized":"2024-09-20T18:57:09.115323+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:54:14.292610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:14.292610+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":34,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":333,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.118732+0000","last_change":"2024-09-20T18:54:10.381171+0000","last_active":"2024-09-20T18:57:09.118732+0000","last_peered":"2024-09-20T18:57:09.118732+0000","last_clean":"2024-09-20T18:57:09.118732+0000","last_became_active":"2024-09-20T18:49:51.971399+0000","last_became_peered":"2024-09-20T18:49:51.971399+0000","last_unstale":"2024-09-20T18:57:09.118732+0000","last_undegraded":"2024-09-20T18:57:09.118732+0000","last_fullsized":"2024-09-20T18:57:09.118732+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:10.381108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:10.381108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.c","version":"218'10429","reported_seq":10557,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.799998+0000","last_change":"2024-09-20T18:54:34.311256+0000","last_active":"2024-09-20T18:57:11.799998+0000","last_peered":"2024-09-20T18:57:11.799998+0000","last_clean":"2024-09-20T18:57:11.799998+0000","last_became_active":"2024-09-20T18:54:34.310904+0000","last_became_peered":"2024-09-20T18:54:34.310904+0000","last_unstale":"2024-09-20T18:57:11.799998+0000","last_undegraded":"2024-09-20T18:57:11.799998+0000","last_fullsized":"2024-09-20T18:57:11.799998+0000","mapping_epoch":160,"log_start":"186'5511","ondisk_log_start":"186'5511","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4918,"log_dups_size":0,"ondisk_log_size":4918,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":259719168,"num_objects":3963,"num_object_clones":0,"num_object_copies":11889,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10429,"num_write_kb":460544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":376,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.156140+0000","last_change":"2024-09-20T18:54:22.077299+0000","last_active":"2024-09-20T18:57:11.156140+0000","last_peered":"2024-09-20T18:57:11.156140+0000","last_clean":"2024-09-20T18:57:11.156140+0000","last_became_active":"2024-09-20T18:49:48.499189+0000","last_became_peered":"2024-09-20T18:49:48.499189+0000","last_unstale":"2024-09-20T18:57:11.156140+0000","last_undegraded":"2024-09-20T18:57:11.156140+0000","last_fullsized":"2024-09-20T18:57:11.156140+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-20T18:54:22.077212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:22.077212+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":340,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.158393+0000","last_change":"2024-09-20T18:54:57.237740+0000","last_active":"2024-09-20T18:57:11.158393+0000","last_peered":"2024-09-20T18:57:11.158393+0000","last_clean":"2024-09-20T18:57:11.158393+0000","last_became_active":"2024-09-20T18:49:45.940544+0000","last_became_peered":"2024-09-20T18:49:45.940544+0000","last_unstale":"2024-09-20T18:57:11.158393+0000","last_undegraded":"2024-09-20T18:57:11.158393+0000","last_fullsized":"2024-09-20T18:57:11.158393+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.237675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.237675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":346,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.114667+0000","last_change":"2024-09-20T18:54:51.229251+0000","last_active":"2024-09-20T18:57:09.114667+0000","last_peered":"2024-09-20T18:57:09.114667+0000","last_clean":"2024-09-20T18:57:09.114667+0000","last_became_active":"2024-09-20T18:49:49.963073+0000","last_became_peered":"2024-09-20T18:49:49.963073+0000","last_unstale":"2024-09-20T18:57:09.114667+0000","last_undegraded":"2024-09-20T18:57:09.114667+0000","last_fullsized":"2024-09-20T18:57:09.114667+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.229201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.229201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":342,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.115133+0000","last_change":"2024-09-20T18:54:55.323937+0000","last_active":"2024-09-20T18:57:09.115133+0000","last_peered":"2024-09-20T18:57:09.115133+0000","last_clean":"2024-09-20T18:57:09.115133+0000","last_became_active":"2024-09-20T18:49:51.968793+0000","last_became_peered":"2024-09-20T18:49:51.968793+0000","last_unstale":"2024-09-20T18:57:09.115133+0000","last_undegraded":"2024-09-20T18:57:09.115133+0000","last_fullsized":"2024-09-20T18:57:09.115133+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:55.323692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:55.323692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"217'10312","reported_seq":10439,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:10.915923+0000","last_change":"2024-09-20T18:54:34.311680+0000","last_active":"2024-09-20T18:57:10.915923+0000","last_peered":"2024-09-20T18:57:10.915923+0000","last_clean":"2024-09-20T18:57:10.915923+0000","last_became_active":"2024-09-20T18:54:34.311241+0000","last_became_peered":"2024-09-20T18:54:34.311241+0000","last_unstale":"2024-09-20T18:57:10.915923+0000","last_undegraded":"2024-09-20T18:57:10.915923+0000","last_fullsized":"2024-09-20T18:57:10.915923+0000","mapping_epoch":160,"log_start":"184'5246","ondisk_log_start":"184'5246","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":5066,"log_dups_size":0,"ondisk_log_size":5066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":266010648,"num_objects":4060,"num_object_clones":0,"num_object_copies":12180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4060,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10312,"num_write_kb":459841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":402,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.114707+0000","last_change":"2024-09-20T18:55:28.853375+0000","last_active":"2024-09-20T18:57:09.114707+0000","last_peered":"2024-09-20T18:57:09.114707+0000","last_clean":"2024-09-20T18:57:09.114707+0000","last_became_active":"2024-09-20T18:55:28.831057+0000","last_became_peered":"2024-09-20T18:55:28.831057+0000","last_unstale":"2024-09-20T18:57:09.114707+0000","last_undegraded":"2024-09-20T18:57:09.114707+0000","last_fullsized":"2024-09-20T18:57:09.114707+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-20T18:54:57.394893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.394893+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":340,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.160063+0000","last_change":"2024-09-20T18:51:27.187060+0000","last_active":"2024-09-20T18:57:11.160063+0000","last_peered":"2024-09-20T18:57:11.160063+0000","last_clean":"2024-09-20T18:57:11.160063+0000","last_became_active":"2024-09-20T18:49:45.940469+0000","last_became_peered":"2024-09-20T18:49:45.940469+0000","last_unstale":"2024-09-20T18:57:11.160063+0000","last_undegraded":"2024-09-20T18:57:11.160063+0000","last_fullsized":"2024-09-20T18:57:11.160063+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:51:27.186991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:27.186991+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"66'8","reported_seq":446,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.155953+0000","last_change":"2024-09-20T18:54:47.223010+0000","last_active":"2024-09-20T18:57:11.155953+0000","last_peered":"2024-09-20T18:57:11.155953+0000","last_clean":"2024-09-20T18:57:11.155953+0000","last_became_active":"2024-09-20T18:49:49.956133+0000","last_became_peered":"2024-09-20T18:49:49.956133+0000","last_unstale":"2024-09-20T18:57:11.155953+0000","last_undegraded":"2024-09-20T18:57:11.155953+0000","last_fullsized":"2024-09-20T18:57:11.155953+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:54:47.222937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:47.222937+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":340,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.150154+0000","last_change":"2024-09-20T18:55:28.819147+0000","last_active":"2024-09-20T18:57:09.150154+0000","last_peered":"2024-09-20T18:57:09.150154+0000","last_clean":"2024-09-20T18:57:09.150154+0000","last_became_active":"2024-09-20T18:55:28.818804+0000","last_became_peered":"2024-09-20T18:55:28.818804+0000","last_unstale":"2024-09-20T18:57:09.150154+0000","last_undegraded":"2024-09-20T18:57:09.150154+0000","last_fullsized":"2024-09-20T18:57:09.150154+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:42.880724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:42.880724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"218'10451","reported_seq":10578,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.810610+0000","last_change":"2024-09-20T18:54:34.309939+0000","last_active":"2024-09-20T18:57:11.810610+0000","last_peered":"2024-09-20T18:57:11.810610+0000","last_clean":"2024-09-20T18:57:11.810610+0000","last_became_active":"2024-09-20T18:54:34.309630+0000","last_became_peered":"2024-09-20T18:54:34.309630+0000","last_unstale":"2024-09-20T18:57:11.810610+0000","last_undegraded":"2024-09-20T18:57:11.810610+0000","last_fullsized":"2024-09-20T18:57:11.810610+0000","mapping_epoch":160,"log_start":"186'5549","ondisk_log_start":"186'5549","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4902,"log_dups_size":0,"ondisk_log_size":4902,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":258932736,"num_objects":3951,"num_object_clones":0,"num_object_copies":11853,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10451,"num_write_kb":460864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":367,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.152621+0000","last_change":"2024-09-20T18:55:19.541011+0000","last_active":"2024-09-20T18:57:11.152621+0000","last_peered":"2024-09-20T18:57:11.152621+0000","last_clean":"2024-09-20T18:57:11.152621+0000","last_became_active":"2024-09-20T18:49:47.949626+0000","last_became_peered":"2024-09-20T18:49:47.949626+0000","last_unstale":"2024-09-20T18:57:11.152621+0000","last_undegraded":"2024-09-20T18:57:11.152621+0000","last_fullsized":"2024-09-20T18:57:11.152621+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:55:19.540812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:19.540812+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":353,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.119313+0000","last_change":"2024-09-20T18:54:18.326947+0000","last_active":"2024-09-20T18:57:09.119313+0000","last_peered":"2024-09-20T18:57:09.119313+0000","last_clean":"2024-09-20T18:57:09.119313+0000","last_became_active":"2024-09-20T18:49:45.942496+0000","last_became_peered":"2024-09-20T18:49:45.942496+0000","last_unstale":"2024-09-20T18:57:09.119313+0000","last_undegraded":"2024-09-20T18:57:09.119313+0000","last_fullsized":"2024-09-20T18:57:09.119313+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:18.326875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:18.326875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"184'9","reported_seq":448,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.140200+0000","last_change":"2024-09-20T18:55:29.010062+0000","last_active":"2024-09-20T18:57:11.140200+0000","last_peered":"2024-09-20T18:57:11.140200+0000","last_clean":"2024-09-20T18:57:11.140200+0000","last_became_active":"2024-09-20T18:55:28.830643+0000","last_became_peered":"2024-09-20T18:55:28.830643+0000","last_unstale":"2024-09-20T18:57:11.140200+0000","last_undegraded":"2024-09-20T18:57:11.140200+0000","last_fullsized":"2024-09-20T18:57:11.140200+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:29.010015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:29.010015+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":319,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.139473+0000","last_change":"2024-09-20T18:49:51.965733+0000","last_active":"2024-09-20T18:57:11.139473+0000","last_peered":"2024-09-20T18:57:11.139473+0000","last_clean":"2024-09-20T18:57:11.139473+0000","last_became_active":"2024-09-20T18:49:51.964780+0000","last_became_peered":"2024-09-20T18:49:51.964780+0000","last_unstale":"2024-09-20T18:57:11.139473+0000","last_undegraded":"2024-09-20T18:57:11.139473+0000","last_fullsized":"2024-09-20T18:57:11.139473+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"218'10680","reported_seq":10808,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.432516+0000","last_change":"2024-09-20T18:54:34.309809+0000","last_active":"2024-09-20T18:57:11.432516+0000","last_peered":"2024-09-20T18:57:11.432516+0000","last_clean":"2024-09-20T18:57:11.432516+0000","last_became_active":"2024-09-20T18:54:34.309231+0000","last_became_peered":"2024-09-20T18:54:34.309231+0000","last_unstale":"2024-09-20T18:57:11.432516+0000","last_undegraded":"2024-09-20T18:57:11.432516+0000","last_fullsized":"2024-09-20T18:57:11.432516+0000","mapping_epoch":160,"log_start":"189'6332","ondisk_log_start":"189'6332","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4348,"log_dups_size":0,"ondisk_log_size":4348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":264372224,"num_objects":4034,"num_object_clones":0,"num_object_copies":12102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4034,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10680,"num_write_kb":470848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":379,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.140365+0000","last_change":"2024-09-20T18:54:14.923895+0000","last_active":"2024-09-20T18:57:11.140365+0000","last_peered":"2024-09-20T18:57:11.140365+0000","last_clean":"2024-09-20T18:57:11.140365+0000","last_became_active":"2024-09-20T18:49:48.497736+0000","last_became_peered":"2024-09-20T18:49:48.497736+0000","last_unstale":"2024-09-20T18:57:11.140365+0000","last_undegraded":"2024-09-20T18:57:11.140365+0000","last_fullsized":"2024-09-20T18:57:11.140365+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-20T18:54:14.923682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:14.923682+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":340,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.152790+0000","last_change":"2024-09-20T18:57:09.940634+0000","last_active":"2024-09-20T18:57:11.152790+0000","last_peered":"2024-09-20T18:57:11.152790+0000","last_clean":"2024-09-20T18:57:11.152790+0000","last_became_active":"2024-09-20T18:49:45.942253+0000","last_became_peered":"2024-09-20T18:49:45.942253+0000","last_unstale":"2024-09-20T18:57:11.152790+0000","last_undegraded":"2024-09-20T18:57:11.152790+0000","last_fullsized":"2024-09-20T18:57:11.152790+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:09.940517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:09.940517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:18.833936+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":85,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.140172+0000","last_change":"2024-09-20T18:55:28.830954+0000","last_active":"2024-09-20T18:57:11.140172+0000","last_peered":"2024-09-20T18:57:11.140172+0000","last_clean":"2024-09-20T18:57:11.140172+0000","last_became_active":"2024-09-20T18:55:28.830645+0000","last_became_peered":"2024-09-20T18:55:28.830645+0000","last_unstale":"2024-09-20T18:57:11.140172+0000","last_undegraded":"2024-09-20T18:57:11.140172+0000","last_fullsized":"2024-09-20T18:57:11.140172+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:50.610871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:50.610871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":351,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.151980+0000","last_change":"2024-09-20T18:55:28.822362+0000","last_active":"2024-09-20T18:57:11.151980+0000","last_peered":"2024-09-20T18:57:11.151980+0000","last_clean":"2024-09-20T18:57:11.151980+0000","last_became_active":"2024-09-20T18:55:28.821588+0000","last_became_peered":"2024-09-20T18:55:28.821588+0000","last_unstale":"2024-09-20T18:57:11.151980+0000","last_undegraded":"2024-09-20T18:57:11.151980+0000","last_fullsized":"2024-09-20T18:57:11.151980+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.402841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.402841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"217'10519","reported_seq":10645,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:10.890489+0000","last_change":"2024-09-20T18:54:34.311561+0000","last_active":"2024-09-20T18:57:10.890489+0000","last_peered":"2024-09-20T18:57:10.890489+0000","last_clean":"2024-09-20T18:57:10.890489+0000","last_became_active":"2024-09-20T18:54:34.311181+0000","last_became_peered":"2024-09-20T18:54:34.311181+0000","last_unstale":"2024-09-20T18:57:10.890489+0000","last_undegraded":"2024-09-20T18:57:10.890489+0000","last_fullsized":"2024-09-20T18:57:10.890489+0000","mapping_epoch":160,"log_start":"186'5494","ondisk_log_start":"186'5494","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":5025,"log_dups_size":0,"ondisk_log_size":5025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":267321344,"num_objects":4079,"num_object_clones":0,"num_object_copies":12237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10519,"num_write_kb":467136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"66'28","reported_seq":482,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.152628+0000","last_change":"2024-09-20T18:55:16.527267+0000","last_active":"2024-09-20T18:57:11.152628+0000","last_peered":"2024-09-20T18:57:11.152628+0000","last_clean":"2024-09-20T18:57:11.152628+0000","last_became_active":"2024-09-20T18:49:47.955335+0000","last_became_peered":"2024-09-20T18:49:47.955335+0000","last_unstale":"2024-09-20T18:57:11.152628+0000","last_undegraded":"2024-09-20T18:57:11.152628+0000","last_fullsized":"2024-09-20T18:57:11.152628+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'28","last_scrub_stamp":"2024-09-20T18:55:16.527165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:16.527165+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":337,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.115241+0000","last_change":"2024-09-20T18:55:28.832303+0000","last_active":"2024-09-20T18:57:09.115241+0000","last_peered":"2024-09-20T18:57:09.115241+0000","last_clean":"2024-09-20T18:57:09.115241+0000","last_became_active":"2024-09-20T18:55:28.831574+0000","last_became_peered":"2024-09-20T18:55:28.831574+0000","last_unstale":"2024-09-20T18:57:09.115241+0000","last_undegraded":"2024-09-20T18:57:09.115241+0000","last_fullsized":"2024-09-20T18:57:09.115241+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1],"acting":[4,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":382,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.118947+0000","last_change":"2024-09-20T18:55:28.854615+0000","last_active":"2024-09-20T18:57:09.118947+0000","last_peered":"2024-09-20T18:57:09.118947+0000","last_clean":"2024-09-20T18:57:09.118947+0000","last_became_active":"2024-09-20T18:55:28.837226+0000","last_became_peered":"2024-09-20T18:55:28.837226+0000","last_unstale":"2024-09-20T18:57:09.118947+0000","last_undegraded":"2024-09-20T18:57:09.118947+0000","last_fullsized":"2024-09-20T18:57:09.118947+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-20T18:54:51.227531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.227531+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0012781159999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":347,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.140719+0000","last_change":"2024-09-20T18:55:13.949348+0000","last_active":"2024-09-20T18:57:11.140719+0000","last_peered":"2024-09-20T18:57:11.140719+0000","last_clean":"2024-09-20T18:57:11.140719+0000","last_became_active":"2024-09-20T18:49:49.953971+0000","last_became_peered":"2024-09-20T18:49:49.953971+0000","last_unstale":"2024-09-20T18:57:11.140719+0000","last_undegraded":"2024-09-20T18:57:11.140719+0000","last_fullsized":"2024-09-20T18:57:11.140719+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:13.949269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:13.949269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":320,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.150870+0000","last_change":"2024-09-20T18:49:51.969702+0000","last_active":"2024-09-20T18:57:11.150870+0000","last_peered":"2024-09-20T18:57:11.150870+0000","last_clean":"2024-09-20T18:57:11.150870+0000","last_became_active":"2024-09-20T18:49:51.969400+0000","last_became_peered":"2024-09-20T18:49:51.969400+0000","last_unstale":"2024-09-20T18:57:11.150870+0000","last_undegraded":"2024-09-20T18:57:11.150870+0000","last_fullsized":"2024-09-20T18:57:11.150870+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"218'10583","reported_seq":10711,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.408240+0000","last_change":"2024-09-20T18:54:33.926785+0000","last_active":"2024-09-20T18:57:11.408240+0000","last_peered":"2024-09-20T18:57:11.408240+0000","last_clean":"2024-09-20T18:57:11.408240+0000","last_became_active":"2024-09-20T18:54:33.925750+0000","last_became_peered":"2024-09-20T18:54:33.925750+0000","last_unstale":"2024-09-20T18:57:11.408240+0000","last_undegraded":"2024-09-20T18:57:11.408240+0000","last_fullsized":"2024-09-20T18:57:11.408240+0000","mapping_epoch":160,"log_start":"189'6240","ondisk_log_start":"189'6240","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4343,"log_dups_size":0,"ondisk_log_size":4343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":270991360,"num_objects":4135,"num_object_clones":0,"num_object_copies":12405,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10583,"num_write_kb":470976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":379,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.158563+0000","last_change":"2024-09-20T18:55:25.702762+0000","last_active":"2024-09-20T18:57:11.158563+0000","last_peered":"2024-09-20T18:57:11.158563+0000","last_clean":"2024-09-20T18:57:11.158563+0000","last_became_active":"2024-09-20T18:49:47.950904+0000","last_became_peered":"2024-09-20T18:49:47.950904+0000","last_unstale":"2024-09-20T18:57:11.158563+0000","last_undegraded":"2024-09-20T18:57:11.158563+0000","last_fullsized":"2024-09-20T18:57:11.158563+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-20T18:55:25.702596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:25.702596+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":363,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.152706+0000","last_change":"2024-09-20T18:55:28.830832+0000","last_active":"2024-09-20T18:57:11.152706+0000","last_peered":"2024-09-20T18:57:11.152706+0000","last_clean":"2024-09-20T18:57:11.152706+0000","last_became_active":"2024-09-20T18:55:28.830269+0000","last_became_peered":"2024-09-20T18:55:28.830269+0000","last_unstale":"2024-09-20T18:57:11.152706+0000","last_undegraded":"2024-09-20T18:57:11.152706+0000","last_fullsized":"2024-09-20T18:57:11.152706+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:23.359769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:23.359769+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":350,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.153982+0000","last_change":"2024-09-20T18:52:07.718766+0000","last_active":"2024-09-20T18:57:11.153982+0000","last_peered":"2024-09-20T18:57:11.153982+0000","last_clean":"2024-09-20T18:57:11.153982+0000","last_became_active":"2024-09-20T18:49:44.937135+0000","last_became_peered":"2024-09-20T18:49:44.937135+0000","last_unstale":"2024-09-20T18:57:11.153982+0000","last_undegraded":"2024-09-20T18:57:11.153982+0000","last_fullsized":"2024-09-20T18:57:11.153982+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:52:07.718646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:52:07.718646+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.00095558800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":333,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.156769+0000","last_change":"2024-09-20T18:51:59.796450+0000","last_active":"2024-09-20T18:57:11.156769+0000","last_peered":"2024-09-20T18:57:11.156769+0000","last_clean":"2024-09-20T18:57:11.156769+0000","last_became_active":"2024-09-20T18:49:49.954424+0000","last_became_peered":"2024-09-20T18:49:49.954424+0000","last_unstale":"2024-09-20T18:57:11.156769+0000","last_undegraded":"2024-09-20T18:57:11.156769+0000","last_fullsized":"2024-09-20T18:57:11.156769+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:59.796383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:59.796383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":332,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.150987+0000","last_change":"2024-09-20T18:55:28.826659+0000","last_active":"2024-09-20T18:57:09.150987+0000","last_peered":"2024-09-20T18:57:09.150987+0000","last_clean":"2024-09-20T18:57:09.150987+0000","last_became_active":"2024-09-20T18:55:28.825812+0000","last_became_peered":"2024-09-20T18:55:28.825812+0000","last_unstale":"2024-09-20T18:57:09.150987+0000","last_undegraded":"2024-09-20T18:57:09.150987+0000","last_fullsized":"2024-09-20T18:57:09.150987+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:39.814259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:39.814259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"218'10629","reported_seq":10757,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.191806+0000","last_change":"2024-09-20T18:54:34.311774+0000","last_active":"2024-09-20T18:57:11.191806+0000","last_peered":"2024-09-20T18:57:11.191806+0000","last_clean":"2024-09-20T18:57:11.191806+0000","last_became_active":"2024-09-20T18:54:34.311487+0000","last_became_peered":"2024-09-20T18:54:34.311487+0000","last_unstale":"2024-09-20T18:57:11.191806+0000","last_undegraded":"2024-09-20T18:57:11.191806+0000","last_fullsized":"2024-09-20T18:57:11.191806+0000","mapping_epoch":160,"log_start":"190'6392","ondisk_log_start":"190'6392","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4237,"log_dups_size":0,"ondisk_log_size":4237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":261423104,"num_objects":3989,"num_object_clones":0,"num_object_copies":11967,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10629,"num_write_kb":467776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":377,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.159856+0000","last_change":"2024-09-20T18:55:14.951950+0000","last_active":"2024-09-20T18:57:11.159856+0000","last_peered":"2024-09-20T18:57:11.159856+0000","last_clean":"2024-09-20T18:57:11.159856+0000","last_became_active":"2024-09-20T18:49:47.949230+0000","last_became_peered":"2024-09-20T18:49:47.949230+0000","last_unstale":"2024-09-20T18:57:11.159856+0000","last_undegraded":"2024-09-20T18:57:11.159856+0000","last_fullsized":"2024-09-20T18:57:11.159856+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:55:14.951892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:14.951892+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":355,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.145722+0000","last_change":"2024-09-20T18:54:27.905856+0000","last_active":"2024-09-20T18:57:11.145722+0000","last_peered":"2024-09-20T18:57:11.145722+0000","last_clean":"2024-09-20T18:57:11.145722+0000","last_became_active":"2024-09-20T18:49:45.939197+0000","last_became_peered":"2024-09-20T18:49:45.939197+0000","last_unstale":"2024-09-20T18:57:11.145722+0000","last_undegraded":"2024-09-20T18:57:11.145722+0000","last_fullsized":"2024-09-20T18:57:11.145722+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:27.905803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:27.905803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":804,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.842430+0000","last_change":"2024-09-20T18:54:26.359887+0000","last_active":"2024-09-20T18:57:09.842430+0000","last_peered":"2024-09-20T18:57:09.842430+0000","last_clean":"2024-09-20T18:57:09.842430+0000","last_became_active":"2024-09-20T18:49:44.936819+0000","last_became_peered":"2024-09-20T18:49:44.936819+0000","last_unstale":"2024-09-20T18:57:09.842430+0000","last_undegraded":"2024-09-20T18:57:09.842430+0000","last_fullsized":"2024-09-20T18:57:09.842430+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-20T18:54:26.359807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:54:26.359807+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0010996199999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":440,"num_read_kb":435,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":337,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.119496+0000","last_change":"2024-09-20T18:54:59.151228+0000","last_active":"2024-09-20T18:57:09.119496+0000","last_peered":"2024-09-20T18:57:09.119496+0000","last_clean":"2024-09-20T18:57:09.119496+0000","last_became_active":"2024-09-20T18:49:49.961066+0000","last_became_peered":"2024-09-20T18:49:49.961066+0000","last_unstale":"2024-09-20T18:57:09.119496+0000","last_undegraded":"2024-09-20T18:57:09.119496+0000","last_fullsized":"2024-09-20T18:57:09.119496+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:59.150063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:59.150063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":344,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.151822+0000","last_change":"2024-09-20T18:55:21.629867+0000","last_active":"2024-09-20T18:57:11.151822+0000","last_peered":"2024-09-20T18:57:11.151822+0000","last_clean":"2024-09-20T18:57:11.151822+0000","last_became_active":"2024-09-20T18:49:51.970343+0000","last_became_peered":"2024-09-20T18:49:51.970343+0000","last_unstale":"2024-09-20T18:57:11.151822+0000","last_undegraded":"2024-09-20T18:57:11.151822+0000","last_fullsized":"2024-09-20T18:57:11.151822+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:21.629800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:21.629800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"217'10489","reported_seq":21038,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:10.909527+0000","last_change":"2024-09-20T18:57:09.139489+0000","last_active":"2024-09-20T18:57:10.909527+0000","last_peered":"2024-09-20T18:57:10.909527+0000","last_clean":"2024-09-20T18:57:10.909527+0000","last_became_active":"2024-09-20T18:57:09.121592+0000","last_became_peered":"2024-09-20T18:57:09.121592+0000","last_unstale":"2024-09-20T18:57:10.909527+0000","last_undegraded":"2024-09-20T18:57:10.909527+0000","last_fullsized":"2024-09-20T18:57:10.909527+0000","mapping_epoch":216,"log_start":"186'5406","ondisk_log_start":"186'5406","created":160,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":5083,"log_dups_size":0,"ondisk_log_size":5083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":264962048,"num_objects":4043,"num_object_clones":0,"num_object_copies":12129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10489,"num_write_kb":465024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5206,"num_bytes_recovered":341180416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"184'17","reported_seq":165,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.116668+0000","last_change":"2024-09-20T18:55:28.856602+0000","last_active":"2024-09-20T18:57:09.116668+0000","last_peered":"2024-09-20T18:57:09.116668+0000","last_clean":"2024-09-20T18:57:09.116668+0000","last_became_active":"2024-09-20T18:55:28.831024+0000","last_became_peered":"2024-09-20T18:55:28.831024+0000","last_unstale":"2024-09-20T18:57:09.116668+0000","last_undegraded":"2024-09-20T18:57:09.116668+0000","last_fullsized":"2024-09-20T18:57:09.116668+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":24,"num_read_kb":14,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":355,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.151971+0000","last_change":"2024-09-20T18:55:28.830447+0000","last_active":"2024-09-20T18:57:11.151971+0000","last_peered":"2024-09-20T18:57:11.151971+0000","last_clean":"2024-09-20T18:57:11.151971+0000","last_became_active":"2024-09-20T18:55:28.830147+0000","last_became_peered":"2024-09-20T18:55:28.830147+0000","last_unstale":"2024-09-20T18:57:11.151971+0000","last_undegraded":"2024-09-20T18:57:11.151971+0000","last_fullsized":"2024-09-20T18:57:11.151971+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:13.144896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:13.144896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":406,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.149887+0000","last_change":"2024-09-20T18:55:28.844060+0000","last_active":"2024-09-20T18:57:09.149887+0000","last_peered":"2024-09-20T18:57:09.149887+0000","last_clean":"2024-09-20T18:57:09.149887+0000","last_became_active":"2024-09-20T18:55:28.829325+0000","last_became_peered":"2024-09-20T18:55:28.829325+0000","last_unstale":"2024-09-20T18:57:09.149887+0000","last_undegraded":"2024-09-20T18:57:09.149887+0000","last_fullsized":"2024-09-20T18:57:09.149887+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T18:55:13.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T18:55:13.696607+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":329,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.150398+0000","last_change":"2024-09-20T18:51:22.852942+0000","last_active":"2024-09-20T18:57:09.150398+0000","last_peered":"2024-09-20T18:57:09.150398+0000","last_clean":"2024-09-20T18:57:09.150398+0000","last_became_active":"2024-09-20T18:49:49.956128+0000","last_became_peered":"2024-09-20T18:49:49.956128+0000","last_unstale":"2024-09-20T18:57:09.150398+0000","last_undegraded":"2024-09-20T18:57:09.150398+0000","last_fullsized":"2024-09-20T18:57:09.150398+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:22.852869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:22.852869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":341,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.119406+0000","last_change":"2024-09-20T18:55:25.281435+0000","last_active":"2024-09-20T18:57:09.119406+0000","last_peered":"2024-09-20T18:57:09.119406+0000","last_clean":"2024-09-20T18:57:09.119406+0000","last_became_active":"2024-09-20T18:49:51.971332+0000","last_became_peered":"2024-09-20T18:49:51.971332+0000","last_unstale":"2024-09-20T18:57:09.119406+0000","last_undegraded":"2024-09-20T18:57:09.119406+0000","last_fullsized":"2024-09-20T18:57:09.119406+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:25.281366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:25.281366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"217'10504","reported_seq":10630,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:10.928516+0000","last_change":"2024-09-20T18:54:33.929366+0000","last_active":"2024-09-20T18:57:10.928516+0000","last_peered":"2024-09-20T18:57:10.928516+0000","last_clean":"2024-09-20T18:57:10.928516+0000","last_became_active":"2024-09-20T18:54:33.929072+0000","last_became_peered":"2024-09-20T18:54:33.929072+0000","last_unstale":"2024-09-20T18:57:10.928516+0000","last_undegraded":"2024-09-20T18:57:10.928516+0000","last_fullsized":"2024-09-20T18:57:10.928516+0000","mapping_epoch":160,"log_start":"188'5902","ondisk_log_start":"188'5902","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4602,"log_dups_size":0,"ondisk_log_size":4602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":265945088,"num_objects":4058,"num_object_clones":0,"num_object_copies":12174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10504,"num_write_kb":465984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":372,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.115353+0000","last_change":"2024-09-20T18:55:28.840105+0000","last_active":"2024-09-20T18:57:09.115353+0000","last_peered":"2024-09-20T18:57:09.115353+0000","last_clean":"2024-09-20T18:57:09.115353+0000","last_became_active":"2024-09-20T18:55:28.818269+0000","last_became_peered":"2024-09-20T18:55:28.818269+0000","last_unstale":"2024-09-20T18:57:09.115353+0000","last_undegraded":"2024-09-20T18:57:09.115353+0000","last_fullsized":"2024-09-20T18:57:09.115353+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:51:14.397930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.397930+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":338,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.114951+0000","last_change":"2024-09-20T18:51:18.410601+0000","last_active":"2024-09-20T18:57:09.114951+0000","last_peered":"2024-09-20T18:57:09.114951+0000","last_clean":"2024-09-20T18:57:09.114951+0000","last_became_active":"2024-09-20T18:49:45.944579+0000","last_became_peered":"2024-09-20T18:49:45.944579+0000","last_unstale":"2024-09-20T18:57:09.114951+0000","last_undegraded":"2024-09-20T18:57:09.114951+0000","last_fullsized":"2024-09-20T18:57:09.114951+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:18.410447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:18.410447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":352,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.119340+0000","last_change":"2024-09-20T18:55:28.830634+0000","last_active":"2024-09-20T18:57:09.119340+0000","last_peered":"2024-09-20T18:57:09.119340+0000","last_clean":"2024-09-20T18:57:09.119340+0000","last_became_active":"2024-09-20T18:55:28.827785+0000","last_became_peered":"2024-09-20T18:55:28.827785+0000","last_unstale":"2024-09-20T18:57:09.119340+0000","last_undegraded":"2024-09-20T18:57:09.119340+0000","last_fullsized":"2024-09-20T18:57:09.119340+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:11.401777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:11.401777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":335,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.152511+0000","last_change":"2024-09-20T18:55:28.833649+0000","last_active":"2024-09-20T18:57:11.152511+0000","last_peered":"2024-09-20T18:57:11.152511+0000","last_clean":"2024-09-20T18:57:11.152511+0000","last_became_active":"2024-09-20T18:55:28.833115+0000","last_became_peered":"2024-09-20T18:55:28.833115+0000","last_unstale":"2024-09-20T18:57:11.152511+0000","last_undegraded":"2024-09-20T18:57:11.152511+0000","last_fullsized":"2024-09-20T18:57:11.152511+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:05.021071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.021071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1","version":"218'10413","reported_seq":10541,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.266435+0000","last_change":"2024-09-20T18:54:34.310237+0000","last_active":"2024-09-20T18:57:11.266435+0000","last_peered":"2024-09-20T18:57:11.266435+0000","last_clean":"2024-09-20T18:57:11.266435+0000","last_became_active":"2024-09-20T18:54:34.309781+0000","last_became_peered":"2024-09-20T18:54:34.309781+0000","last_unstale":"2024-09-20T18:57:11.266435+0000","last_undegraded":"2024-09-20T18:57:11.266435+0000","last_fullsized":"2024-09-20T18:57:11.266435+0000","mapping_epoch":160,"log_start":"189'6124","ondisk_log_start":"189'6124","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4289,"log_dups_size":0,"ondisk_log_size":4289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":263389184,"num_objects":4019,"num_object_clones":0,"num_object_copies":12057,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4019,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10413,"num_write_kb":461824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":370,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.116818+0000","last_change":"2024-09-20T18:52:02.944536+0000","last_active":"2024-09-20T18:57:09.116818+0000","last_peered":"2024-09-20T18:57:09.116818+0000","last_clean":"2024-09-20T18:57:09.116818+0000","last_became_active":"2024-09-20T18:49:47.947636+0000","last_became_peered":"2024-09-20T18:49:47.947636+0000","last_unstale":"2024-09-20T18:57:09.116818+0000","last_undegraded":"2024-09-20T18:57:09.116818+0000","last_fullsized":"2024-09-20T18:57:09.116818+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-20T18:52:02.944434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:02.944434+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":207,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.114721+0000","last_change":"2024-09-20T18:55:28.822330+0000","last_active":"2024-09-20T18:57:09.114721+0000","last_peered":"2024-09-20T18:57:09.114721+0000","last_clean":"2024-09-20T18:57:09.114721+0000","last_became_active":"2024-09-20T18:55:28.821959+0000","last_became_peered":"2024-09-20T18:55:28.821959+0000","last_unstale":"2024-09-20T18:57:09.114721+0000","last_undegraded":"2024-09-20T18:57:09.114721+0000","last_fullsized":"2024-09-20T18:57:09.114721+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:45.669427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:45.669427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":340,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.152477+0000","last_change":"2024-09-20T18:54:11.377907+0000","last_active":"2024-09-20T18:57:11.152477+0000","last_peered":"2024-09-20T18:57:11.152477+0000","last_clean":"2024-09-20T18:57:11.152477+0000","last_became_active":"2024-09-20T18:49:49.959160+0000","last_became_peered":"2024-09-20T18:49:49.959160+0000","last_unstale":"2024-09-20T18:57:11.152477+0000","last_undegraded":"2024-09-20T18:57:11.152477+0000","last_fullsized":"2024-09-20T18:57:11.152477+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:11.376951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:11.376951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":337,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.160640+0000","last_change":"2024-09-20T18:52:35.635897+0000","last_active":"2024-09-20T18:57:11.160640+0000","last_peered":"2024-09-20T18:57:11.160640+0000","last_clean":"2024-09-20T18:57:11.160640+0000","last_became_active":"2024-09-20T18:49:51.970108+0000","last_became_peered":"2024-09-20T18:49:51.970108+0000","last_unstale":"2024-09-20T18:57:11.160640+0000","last_undegraded":"2024-09-20T18:57:11.160640+0000","last_fullsized":"2024-09-20T18:57:11.160640+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:35.635818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:35.635818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"218'10292","reported_seq":10420,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.426349+0000","last_change":"2024-09-20T18:54:33.929552+0000","last_active":"2024-09-20T18:57:11.426349+0000","last_peered":"2024-09-20T18:57:11.426349+0000","last_clean":"2024-09-20T18:57:11.426349+0000","last_became_active":"2024-09-20T18:54:33.929208+0000","last_became_peered":"2024-09-20T18:54:33.929208+0000","last_unstale":"2024-09-20T18:57:11.426349+0000","last_undegraded":"2024-09-20T18:57:11.426349+0000","last_fullsized":"2024-09-20T18:57:11.426349+0000","mapping_epoch":160,"log_start":"189'5921","ondisk_log_start":"189'5921","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4371,"log_dups_size":0,"ondisk_log_size":4371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":257818624,"num_objects":3934,"num_object_clones":0,"num_object_copies":11802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3934,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10292,"num_write_kb":455232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":376,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.115021+0000","last_change":"2024-09-20T18:55:28.846773+0000","last_active":"2024-09-20T18:57:09.115021+0000","last_peered":"2024-09-20T18:57:09.115021+0000","last_clean":"2024-09-20T18:57:09.115021+0000","last_became_active":"2024-09-20T18:55:28.817101+0000","last_became_peered":"2024-09-20T18:55:28.817101+0000","last_unstale":"2024-09-20T18:57:09.115021+0000","last_undegraded":"2024-09-20T18:57:09.115021+0000","last_fullsized":"2024-09-20T18:57:09.115021+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:54:17.219548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:17.219548+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":348,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.153404+0000","last_change":"2024-09-20T18:52:46.005172+0000","last_active":"2024-09-20T18:57:11.153404+0000","last_peered":"2024-09-20T18:57:11.153404+0000","last_clean":"2024-09-20T18:57:11.153404+0000","last_became_active":"2024-09-20T18:49:45.937264+0000","last_became_peered":"2024-09-20T18:49:45.937264+0000","last_unstale":"2024-09-20T18:57:11.153404+0000","last_undegraded":"2024-09-20T18:57:11.153404+0000","last_fullsized":"2024-09-20T18:57:11.153404+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:46.005109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:46.005109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":447,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.166018+0000","last_change":"2024-09-20T18:55:17.549660+0000","last_active":"2024-09-20T18:57:11.166018+0000","last_peered":"2024-09-20T18:57:11.166018+0000","last_clean":"2024-09-20T18:57:11.166018+0000","last_became_active":"2024-09-20T18:49:49.957005+0000","last_became_peered":"2024-09-20T18:49:49.957005+0000","last_unstale":"2024-09-20T18:57:11.166018+0000","last_undegraded":"2024-09-20T18:57:11.166018+0000","last_fullsized":"2024-09-20T18:57:11.166018+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:17.549598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:17.549598+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":334,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.115162+0000","last_change":"2024-09-20T18:54:42.182875+0000","last_active":"2024-09-20T18:57:09.115162+0000","last_peered":"2024-09-20T18:57:09.115162+0000","last_clean":"2024-09-20T18:57:09.115162+0000","last_became_active":"2024-09-20T18:49:51.969355+0000","last_became_peered":"2024-09-20T18:49:51.969355+0000","last_unstale":"2024-09-20T18:57:09.115162+0000","last_undegraded":"2024-09-20T18:57:09.115162+0000","last_fullsized":"2024-09-20T18:57:09.115162+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:42.182809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:42.182809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"218'10511","reported_seq":22907,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.286330+0000","last_change":"2024-09-20T18:57:00.083015+0000","last_active":"2024-09-20T18:57:11.286330+0000","last_peered":"2024-09-20T18:57:11.286330+0000","last_clean":"2024-09-20T18:57:11.286330+0000","last_became_active":"2024-09-20T18:57:00.060960+0000","last_became_peered":"2024-09-20T18:57:00.060960+0000","last_unstale":"2024-09-20T18:57:11.286330+0000","last_undegraded":"2024-09-20T18:57:11.286330+0000","last_fullsized":"2024-09-20T18:57:11.286330+0000","mapping_epoch":213,"log_start":"189'6261","ondisk_log_start":"189'6261","created":160,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4250,"log_dups_size":0,"ondisk_log_size":4250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":264699904,"num_objects":4039,"num_object_clones":0,"num_object_copies":12117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10511,"num_write_kb":465600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6125,"num_bytes_recovered":401408000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":367,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.140285+0000","last_change":"2024-09-20T18:54:54.897991+0000","last_active":"2024-09-20T18:57:11.140285+0000","last_peered":"2024-09-20T18:57:11.140285+0000","last_clean":"2024-09-20T18:57:11.140285+0000","last_became_active":"2024-09-20T18:49:47.946008+0000","last_became_peered":"2024-09-20T18:49:47.946008+0000","last_unstale":"2024-09-20T18:57:11.140285+0000","last_undegraded":"2024-09-20T18:57:11.140285+0000","last_fullsized":"2024-09-20T18:57:11.140285+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:54.897933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:54.897933+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":360,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.119139+0000","last_change":"2024-09-20T18:55:28.833453+0000","last_active":"2024-09-20T18:57:09.119139+0000","last_peered":"2024-09-20T18:57:09.119139+0000","last_clean":"2024-09-20T18:57:09.119139+0000","last_became_active":"2024-09-20T18:55:28.833095+0000","last_became_peered":"2024-09-20T18:55:28.833095+0000","last_unstale":"2024-09-20T18:57:09.119139+0000","last_undegraded":"2024-09-20T18:57:09.119139+0000","last_fullsized":"2024-09-20T18:57:09.119139+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:56.212378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:56.212378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":349,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.154378+0000","last_change":"2024-09-20T18:54:43.379898+0000","last_active":"2024-09-20T18:57:11.154378+0000","last_peered":"2024-09-20T18:57:11.154378+0000","last_clean":"2024-09-20T18:57:11.154378+0000","last_became_active":"2024-09-20T18:49:49.956767+0000","last_became_peered":"2024-09-20T18:49:49.956767+0000","last_unstale":"2024-09-20T18:57:11.154378+0000","last_undegraded":"2024-09-20T18:57:11.154378+0000","last_fullsized":"2024-09-20T18:57:11.154378+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:43.379706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:43.379706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":335,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.145635+0000","last_change":"2024-09-20T18:54:24.976627+0000","last_active":"2024-09-20T18:57:11.145635+0000","last_peered":"2024-09-20T18:57:11.145635+0000","last_clean":"2024-09-20T18:57:11.145635+0000","last_became_active":"2024-09-20T18:49:51.964098+0000","last_became_peered":"2024-09-20T18:49:51.964098+0000","last_unstale":"2024-09-20T18:57:11.145635+0000","last_undegraded":"2024-09-20T18:57:11.145635+0000","last_fullsized":"2024-09-20T18:57:11.145635+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:24.976584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:24.976584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"218'10464","reported_seq":14354,"reported_epoch":218,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-20T18:57:11.805285+0000","last_change":"2024-09-20T18:56:27.871592+0000","last_active":"2024-09-20T18:57:11.805285+0000","last_peered":"2024-09-20T18:57:11.805285+0000","last_clean":"2024-09-20T18:55:27.791097+0000","last_became_active":"2024-09-20T18:55:29.979932+0000","last_became_peered":"2024-09-20T18:55:29.979932+0000","last_unstale":"2024-09-20T18:57:11.805285+0000","last_undegraded":"2024-09-20T18:56:27.867865+0000","last_fullsized":"2024-09-20T18:55:29.815116+0000","mapping_epoch":182,"log_start":"186'5540","ondisk_log_start":"186'5540","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4924,"log_dups_size":0,"ondisk_log_size":4924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":261095424,"num_objects":3984,"num_object_clones":0,"num_object_copies":11952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1360,"num_objects_misplaced":1938,"num_objects_unfound":0,"num_objects_dirty":3984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10464,"num_write_kb":462336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1880,"num_bytes_recovered":123207680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7],"avail_no_missing":["0","6","7"],"object_location_counts":[{"shards":"0,6,7","objects":1938},{"shards":"0,7","objects":686}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":156,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.152518+0000","last_change":"2024-09-20T18:55:28.855793+0000","last_active":"2024-09-20T18:57:11.152518+0000","last_peered":"2024-09-20T18:57:11.152518+0000","last_clean":"2024-09-20T18:57:11.152518+0000","last_became_active":"2024-09-20T18:55:28.828563+0000","last_became_peered":"2024-09-20T18:55:28.828563+0000","last_unstale":"2024-09-20T18:57:11.152518+0000","last_undegraded":"2024-09-20T18:57:11.152518+0000","last_fullsized":"2024-09-20T18:57:11.152518+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-20T18:53:24.458005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:24.458005+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":31,"num_read_kb":20,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,5],"acting":[1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":360,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.151109+0000","last_change":"2024-09-20T18:55:28.835767+0000","last_active":"2024-09-20T18:57:09.151109+0000","last_peered":"2024-09-20T18:57:09.151109+0000","last_clean":"2024-09-20T18:57:09.151109+0000","last_became_active":"2024-09-20T18:55:28.835149+0000","last_became_peered":"2024-09-20T18:55:28.835149+0000","last_unstale":"2024-09-20T18:57:09.151109+0000","last_undegraded":"2024-09-20T18:57:09.151109+0000","last_fullsized":"2024-09-20T18:57:09.151109+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:44.501192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.501192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":349,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.154082+0000","last_change":"2024-09-20T18:55:13.577772+0000","last_active":"2024-09-20T18:57:11.154082+0000","last_peered":"2024-09-20T18:57:11.154082+0000","last_clean":"2024-09-20T18:57:11.154082+0000","last_became_active":"2024-09-20T18:49:49.957558+0000","last_became_peered":"2024-09-20T18:49:49.957558+0000","last_unstale":"2024-09-20T18:57:11.154082+0000","last_undegraded":"2024-09-20T18:57:11.154082+0000","last_fullsized":"2024-09-20T18:57:11.154082+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:13.577699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:13.577699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":341,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.149813+0000","last_change":"2024-09-20T18:53:47.370152+0000","last_active":"2024-09-20T18:57:09.149813+0000","last_peered":"2024-09-20T18:57:09.149813+0000","last_clean":"2024-09-20T18:57:09.149813+0000","last_became_active":"2024-09-20T18:49:51.970798+0000","last_became_peered":"2024-09-20T18:49:51.970798+0000","last_unstale":"2024-09-20T18:57:09.149813+0000","last_undegraded":"2024-09-20T18:57:09.149813+0000","last_fullsized":"2024-09-20T18:57:09.149813+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:47.370108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:47.370108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"217'10785","reported_seq":20295,"reported_epoch":217,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-20T18:57:10.935165+0000","last_change":"2024-09-20T18:55:30.067593+0000","last_active":"2024-09-20T18:57:10.935165+0000","last_peered":"2024-09-20T18:57:10.935165+0000","last_clean":"2024-09-20T18:55:27.760955+0000","last_became_active":"2024-09-20T18:55:30.004922+0000","last_became_peered":"2024-09-20T18:55:30.004922+0000","last_unstale":"2024-09-20T18:57:10.935165+0000","last_undegraded":"2024-09-20T18:57:10.935165+0000","last_fullsized":"2024-09-20T18:55:29.819726+0000","mapping_epoch":182,"log_start":"188'6146","ondisk_log_start":"188'6146","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4639,"log_dups_size":0,"ondisk_log_size":4639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":275316736,"num_objects":4201,"num_object_clones":0,"num_object_copies":12603,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":700,"num_objects_unfound":0,"num_objects_dirty":4201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10785,"num_write_kb":479552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4691,"num_bytes_recovered":307429376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7],"avail_no_missing":["4","6","7"],"object_location_counts":[{"shards":"4,6,7","objects":700},{"shards":"4,7","objects":3501}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":365,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.114855+0000","last_change":"2024-09-20T18:53:52.898818+0000","last_active":"2024-09-20T18:57:09.114855+0000","last_peered":"2024-09-20T18:57:09.114855+0000","last_clean":"2024-09-20T18:57:09.114855+0000","last_became_active":"2024-09-20T18:49:47.949116+0000","last_became_peered":"2024-09-20T18:49:47.949116+0000","last_unstale":"2024-09-20T18:57:09.114855+0000","last_undegraded":"2024-09-20T18:57:09.114855+0000","last_fullsized":"2024-09-20T18:57:09.114855+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:53:52.898760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:52.898760+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":353,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.150668+0000","last_change":"2024-09-20T18:54:52.650798+0000","last_active":"2024-09-20T18:57:09.150668+0000","last_peered":"2024-09-20T18:57:09.150668+0000","last_clean":"2024-09-20T18:57:09.150668+0000","last_became_active":"2024-09-20T18:49:45.944858+0000","last_became_peered":"2024-09-20T18:49:45.944858+0000","last_unstale":"2024-09-20T18:57:09.150668+0000","last_undegraded":"2024-09-20T18:57:09.150668+0000","last_fullsized":"2024-09-20T18:57:09.150668+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:52.650756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:52.650756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":341,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.153706+0000","last_change":"2024-09-20T18:54:42.349879+0000","last_active":"2024-09-20T18:57:11.153706+0000","last_peered":"2024-09-20T18:57:11.153706+0000","last_clean":"2024-09-20T18:57:11.153706+0000","last_became_active":"2024-09-20T18:49:49.954944+0000","last_became_peered":"2024-09-20T18:49:49.954944+0000","last_unstale":"2024-09-20T18:57:11.153706+0000","last_undegraded":"2024-09-20T18:57:11.153706+0000","last_fullsized":"2024-09-20T18:57:11.153706+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:42.349807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:42.349807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":336,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.153147+0000","last_change":"2024-09-20T18:55:17.341862+0000","last_active":"2024-09-20T18:57:11.153147+0000","last_peered":"2024-09-20T18:57:11.153147+0000","last_clean":"2024-09-20T18:57:11.153147+0000","last_became_active":"2024-09-20T18:49:51.972970+0000","last_became_peered":"2024-09-20T18:49:51.972970+0000","last_unstale":"2024-09-20T18:57:11.153147+0000","last_undegraded":"2024-09-20T18:57:11.153147+0000","last_fullsized":"2024-09-20T18:57:11.153147+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:17.341793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:17.341793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.4","version":"217'10350","reported_seq":10475,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:10.930552+0000","last_change":"2024-09-20T18:54:34.309442+0000","last_active":"2024-09-20T18:57:10.930552+0000","last_peered":"2024-09-20T18:57:10.930552+0000","last_clean":"2024-09-20T18:57:10.930552+0000","last_became_active":"2024-09-20T18:54:34.309113+0000","last_became_peered":"2024-09-20T18:54:34.309113+0000","last_unstale":"2024-09-20T18:57:10.930552+0000","last_undegraded":"2024-09-20T18:57:10.930552+0000","last_fullsized":"2024-09-20T18:57:10.930552+0000","mapping_epoch":160,"log_start":"186'5331","ondisk_log_start":"186'5331","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":5019,"log_dups_size":0,"ondisk_log_size":5019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":260964352,"num_objects":3982,"num_object_clones":0,"num_object_copies":11946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10350,"num_write_kb":458624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":389,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.118439+0000","last_change":"2024-09-20T18:55:28.847010+0000","last_active":"2024-09-20T18:57:09.118439+0000","last_peered":"2024-09-20T18:57:09.118439+0000","last_clean":"2024-09-20T18:57:09.118439+0000","last_became_active":"2024-09-20T18:55:28.824801+0000","last_became_peered":"2024-09-20T18:55:28.824801+0000","last_unstale":"2024-09-20T18:57:09.118439+0000","last_undegraded":"2024-09-20T18:57:09.118439+0000","last_fullsized":"2024-09-20T18:57:09.118439+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:54:12.373309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:12.373309+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":373,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.150493+0000","last_change":"2024-09-20T18:55:24.729301+0000","last_active":"2024-09-20T18:57:09.150493+0000","last_peered":"2024-09-20T18:57:09.150493+0000","last_clean":"2024-09-20T18:57:09.150493+0000","last_became_active":"2024-09-20T18:49:45.944256+0000","last_became_peered":"2024-09-20T18:49:45.944256+0000","last_unstale":"2024-09-20T18:57:09.150493+0000","last_undegraded":"2024-09-20T18:57:09.150493+0000","last_fullsized":"2024-09-20T18:57:09.150493+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-20T18:55:24.729219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:24.729219+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"184'9","reported_seq":437,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.149967+0000","last_change":"2024-09-20T18:55:28.824771+0000","last_active":"2024-09-20T18:57:09.149967+0000","last_peered":"2024-09-20T18:57:09.149967+0000","last_clean":"2024-09-20T18:57:09.149967+0000","last_became_active":"2024-09-20T18:55:28.818085+0000","last_became_peered":"2024-09-20T18:55:28.818085+0000","last_unstale":"2024-09-20T18:57:09.149967+0000","last_undegraded":"2024-09-20T18:57:09.149967+0000","last_fullsized":"2024-09-20T18:57:09.149967+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:52:05.788924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.788924+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":340,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.118513+0000","last_change":"2024-09-20T18:55:28.832276+0000","last_active":"2024-09-20T18:57:09.118513+0000","last_peered":"2024-09-20T18:57:09.118513+0000","last_clean":"2024-09-20T18:57:09.118513+0000","last_became_active":"2024-09-20T18:55:28.830380+0000","last_became_peered":"2024-09-20T18:55:28.830380+0000","last_unstale":"2024-09-20T18:57:09.118513+0000","last_undegraded":"2024-09-20T18:57:09.118513+0000","last_fullsized":"2024-09-20T18:57:09.118513+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:14.247398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:14.247398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":85,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.139560+0000","last_change":"2024-09-20T18:55:28.835344+0000","last_active":"2024-09-20T18:57:11.139560+0000","last_peered":"2024-09-20T18:57:11.139560+0000","last_clean":"2024-09-20T18:57:11.139560+0000","last_became_active":"2024-09-20T18:55:28.835010+0000","last_became_peered":"2024-09-20T18:55:28.835010+0000","last_unstale":"2024-09-20T18:57:11.139560+0000","last_undegraded":"2024-09-20T18:57:11.139560+0000","last_fullsized":"2024-09-20T18:57:11.139560+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:26.721735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:26.721735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":353,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.158169+0000","last_change":"2024-09-20T18:54:44.194161+0000","last_active":"2024-09-20T18:57:11.158169+0000","last_peered":"2024-09-20T18:57:11.158169+0000","last_clean":"2024-09-20T18:57:11.158169+0000","last_became_active":"2024-09-20T18:49:48.500662+0000","last_became_peered":"2024-09-20T18:49:48.500662+0000","last_unstale":"2024-09-20T18:57:11.158169+0000","last_undegraded":"2024-09-20T18:57:11.158169+0000","last_fullsized":"2024-09-20T18:57:11.158169+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-20T18:54:44.194089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.194089+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":337,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.119052+0000","last_change":"2024-09-20T18:53:16.220554+0000","last_active":"2024-09-20T18:57:09.119052+0000","last_peered":"2024-09-20T18:57:09.119052+0000","last_clean":"2024-09-20T18:57:09.119052+0000","last_became_active":"2024-09-20T18:49:49.956749+0000","last_became_peered":"2024-09-20T18:49:49.956749+0000","last_unstale":"2024-09-20T18:57:09.119052+0000","last_undegraded":"2024-09-20T18:57:09.119052+0000","last_fullsized":"2024-09-20T18:57:09.119052+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:16.220490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:16.220490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":348,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.150969+0000","last_change":"2024-09-20T18:55:28.827897+0000","last_active":"2024-09-20T18:57:09.150969+0000","last_peered":"2024-09-20T18:57:09.150969+0000","last_clean":"2024-09-20T18:57:09.150969+0000","last_became_active":"2024-09-20T18:55:28.827418+0000","last_became_peered":"2024-09-20T18:55:28.827418+0000","last_unstale":"2024-09-20T18:57:09.150969+0000","last_undegraded":"2024-09-20T18:57:09.150969+0000","last_fullsized":"2024-09-20T18:57:09.150969+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.637739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.637739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":359,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.159958+0000","last_change":"2024-09-20T18:54:44.830872+0000","last_active":"2024-09-20T18:57:11.159958+0000","last_peered":"2024-09-20T18:57:11.159958+0000","last_clean":"2024-09-20T18:57:11.159958+0000","last_became_active":"2024-09-20T18:49:48.500803+0000","last_became_peered":"2024-09-20T18:49:48.500803+0000","last_unstale":"2024-09-20T18:57:11.159958+0000","last_undegraded":"2024-09-20T18:57:11.159958+0000","last_fullsized":"2024-09-20T18:57:11.159958+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:44.830790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.830790+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":352,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.150121+0000","last_change":"2024-09-20T18:55:29.751718+0000","last_active":"2024-09-20T18:57:09.150121+0000","last_peered":"2024-09-20T18:57:09.150121+0000","last_clean":"2024-09-20T18:57:09.150121+0000","last_became_active":"2024-09-20T18:55:28.833799+0000","last_became_peered":"2024-09-20T18:55:28.833799+0000","last_unstale":"2024-09-20T18:57:09.150121+0000","last_undegraded":"2024-09-20T18:57:09.150121+0000","last_fullsized":"2024-09-20T18:57:09.150121+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:29.751642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:29.751642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":340,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.158064+0000","last_change":"2024-09-20T18:53:50.253721+0000","last_active":"2024-09-20T18:57:11.158064+0000","last_peered":"2024-09-20T18:57:11.158064+0000","last_clean":"2024-09-20T18:57:11.158064+0000","last_became_active":"2024-09-20T18:49:49.961947+0000","last_became_peered":"2024-09-20T18:49:49.961947+0000","last_unstale":"2024-09-20T18:57:11.158064+0000","last_undegraded":"2024-09-20T18:57:11.158064+0000","last_fullsized":"2024-09-20T18:57:11.158064+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:50.253660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:50.253660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"66'1","reported_seq":346,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.154652+0000","last_change":"2024-09-20T18:54:20.681878+0000","last_active":"2024-09-20T18:57:11.154652+0000","last_peered":"2024-09-20T18:57:11.154652+0000","last_clean":"2024-09-20T18:57:11.154652+0000","last_became_active":"2024-09-20T18:49:51.965121+0000","last_became_peered":"2024-09-20T18:49:51.965121+0000","last_unstale":"2024-09-20T18:57:11.154652+0000","last_undegraded":"2024-09-20T18:57:11.154652+0000","last_fullsized":"2024-09-20T18:57:11.154652+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T18:54:20.681790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:20.681790+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":349,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.119081+0000","last_change":"2024-09-20T18:55:12.205118+0000","last_active":"2024-09-20T18:57:09.119081+0000","last_peered":"2024-09-20T18:57:09.119081+0000","last_clean":"2024-09-20T18:57:09.119081+0000","last_became_active":"2024-09-20T18:49:48.501011+0000","last_became_peered":"2024-09-20T18:49:48.501011+0000","last_unstale":"2024-09-20T18:57:09.119081+0000","last_undegraded":"2024-09-20T18:57:09.119081+0000","last_fullsized":"2024-09-20T18:57:09.119081+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:55:12.205069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:12.205069+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":355,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.140309+0000","last_change":"2024-09-20T18:55:27.028184+0000","last_active":"2024-09-20T18:57:11.140309+0000","last_peered":"2024-09-20T18:57:11.140309+0000","last_clean":"2024-09-20T18:57:11.140309+0000","last_became_active":"2024-09-20T18:49:45.938511+0000","last_became_peered":"2024-09-20T18:49:45.938511+0000","last_unstale":"2024-09-20T18:57:11.140309+0000","last_undegraded":"2024-09-20T18:57:11.140309+0000","last_fullsized":"2024-09-20T18:57:11.140309+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:27.028126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:27.028126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":446,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.152662+0000","last_change":"2024-09-20T18:55:27.284156+0000","last_active":"2024-09-20T18:57:11.152662+0000","last_peered":"2024-09-20T18:57:11.152662+0000","last_clean":"2024-09-20T18:57:11.152662+0000","last_became_active":"2024-09-20T18:49:49.960550+0000","last_became_peered":"2024-09-20T18:49:49.960550+0000","last_unstale":"2024-09-20T18:57:11.152662+0000","last_undegraded":"2024-09-20T18:57:11.152662+0000","last_fullsized":"2024-09-20T18:57:11.152662+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:27.284086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:27.284086+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":342,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.115324+0000","last_change":"2024-09-20T18:55:12.348079+0000","last_active":"2024-09-20T18:57:09.115324+0000","last_peered":"2024-09-20T18:57:09.115324+0000","last_clean":"2024-09-20T18:57:09.115324+0000","last_became_active":"2024-09-20T18:49:51.966254+0000","last_became_peered":"2024-09-20T18:49:51.966254+0000","last_unstale":"2024-09-20T18:57:09.115324+0000","last_undegraded":"2024-09-20T18:57:09.115324+0000","last_fullsized":"2024-09-20T18:57:09.115324+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:12.347993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:12.347993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":370,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.152370+0000","last_change":"2024-09-20T18:51:15.223913+0000","last_active":"2024-09-20T18:57:11.152370+0000","last_peered":"2024-09-20T18:57:11.152370+0000","last_clean":"2024-09-20T18:57:11.152370+0000","last_became_active":"2024-09-20T18:49:58.997494+0000","last_became_peered":"2024-09-20T18:49:58.997494+0000","last_unstale":"2024-09-20T18:57:11.152370+0000","last_undegraded":"2024-09-20T18:57:11.152370+0000","last_fullsized":"2024-09-20T18:57:11.152370+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:51:15.223826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:15.223826+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":353,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.150315+0000","last_change":"2024-09-20T18:54:57.632458+0000","last_active":"2024-09-20T18:57:09.150315+0000","last_peered":"2024-09-20T18:57:09.150315+0000","last_clean":"2024-09-20T18:57:09.150315+0000","last_became_active":"2024-09-20T18:49:45.949492+0000","last_became_peered":"2024-09-20T18:49:45.949492+0000","last_unstale":"2024-09-20T18:57:09.150315+0000","last_undegraded":"2024-09-20T18:57:09.150315+0000","last_fullsized":"2024-09-20T18:57:09.150315+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.632380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.632380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":442,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.119250+0000","last_change":"2024-09-20T18:55:28.302353+0000","last_active":"2024-09-20T18:57:09.119250+0000","last_peered":"2024-09-20T18:57:09.119250+0000","last_clean":"2024-09-20T18:57:09.119250+0000","last_became_active":"2024-09-20T18:49:49.962243+0000","last_became_peered":"2024-09-20T18:49:49.962243+0000","last_unstale":"2024-09-20T18:57:09.119250+0000","last_undegraded":"2024-09-20T18:57:09.119250+0000","last_fullsized":"2024-09-20T18:57:09.119250+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:28.302303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:28.302303+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":343,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.157093+0000","last_change":"2024-09-20T18:55:11.918284+0000","last_active":"2024-09-20T18:57:11.157093+0000","last_peered":"2024-09-20T18:57:11.157093+0000","last_clean":"2024-09-20T18:57:11.157093+0000","last_became_active":"2024-09-20T18:49:51.969093+0000","last_became_peered":"2024-09-20T18:49:51.969093+0000","last_unstale":"2024-09-20T18:57:11.157093+0000","last_undegraded":"2024-09-20T18:57:11.157093+0000","last_fullsized":"2024-09-20T18:57:11.157093+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:11.918210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:11.918210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":380,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.114916+0000","last_change":"2024-09-20T18:55:28.854249+0000","last_active":"2024-09-20T18:57:09.114916+0000","last_peered":"2024-09-20T18:57:09.114916+0000","last_clean":"2024-09-20T18:57:09.114916+0000","last_became_active":"2024-09-20T18:55:28.826721+0000","last_became_peered":"2024-09-20T18:55:28.826721+0000","last_unstale":"2024-09-20T18:57:09.114916+0000","last_undegraded":"2024-09-20T18:57:09.114916+0000","last_fullsized":"2024-09-20T18:57:09.114916+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:53:43.878921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:43.878921+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":345,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.115147+0000","last_change":"2024-09-20T18:55:28.822690+0000","last_active":"2024-09-20T18:57:09.115147+0000","last_peered":"2024-09-20T18:57:09.115147+0000","last_clean":"2024-09-20T18:57:09.115147+0000","last_became_active":"2024-09-20T18:55:28.822258+0000","last_became_peered":"2024-09-20T18:55:28.822258+0000","last_unstale":"2024-09-20T18:57:09.115147+0000","last_undegraded":"2024-09-20T18:57:09.115147+0000","last_fullsized":"2024-09-20T18:57:09.115147+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:05.985551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.985551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":332,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.152418+0000","last_change":"2024-09-20T18:51:14.271954+0000","last_active":"2024-09-20T18:57:11.152418+0000","last_peered":"2024-09-20T18:57:11.152418+0000","last_clean":"2024-09-20T18:57:11.152418+0000","last_became_active":"2024-09-20T18:49:49.957067+0000","last_became_peered":"2024-09-20T18:49:49.957067+0000","last_unstale":"2024-09-20T18:57:11.152418+0000","last_undegraded":"2024-09-20T18:57:11.152418+0000","last_fullsized":"2024-09-20T18:57:11.152418+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:14.271719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.271719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":328,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.150659+0000","last_change":"2024-09-20T18:51:14.210438+0000","last_active":"2024-09-20T18:57:11.150659+0000","last_peered":"2024-09-20T18:57:11.150659+0000","last_clean":"2024-09-20T18:57:11.150659+0000","last_became_active":"2024-09-20T18:49:51.966915+0000","last_became_peered":"2024-09-20T18:49:51.966915+0000","last_unstale":"2024-09-20T18:57:11.150659+0000","last_undegraded":"2024-09-20T18:57:11.150659+0000","last_fullsized":"2024-09-20T18:57:11.150659+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:14.210358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.210358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":385,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.152716+0000","last_change":"2024-09-20T18:55:28.838698+0000","last_active":"2024-09-20T18:57:11.152716+0000","last_peered":"2024-09-20T18:57:11.152716+0000","last_clean":"2024-09-20T18:57:11.152716+0000","last_became_active":"2024-09-20T18:55:28.827023+0000","last_became_peered":"2024-09-20T18:55:28.827023+0000","last_unstale":"2024-09-20T18:57:11.152716+0000","last_undegraded":"2024-09-20T18:57:11.152716+0000","last_fullsized":"2024-09-20T18:57:11.152716+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:53:53.263976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:53.263976+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":345,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.151063+0000","last_change":"2024-09-20T18:54:42.487465+0000","last_active":"2024-09-20T18:57:09.151063+0000","last_peered":"2024-09-20T18:57:09.151063+0000","last_clean":"2024-09-20T18:57:09.151063+0000","last_became_active":"2024-09-20T18:49:45.944357+0000","last_became_peered":"2024-09-20T18:49:45.944357+0000","last_unstale":"2024-09-20T18:57:09.151063+0000","last_undegraded":"2024-09-20T18:57:09.151063+0000","last_fullsized":"2024-09-20T18:57:09.151063+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:42.487403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:42.487403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":340,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.153215+0000","last_change":"2024-09-20T18:54:18.108239+0000","last_active":"2024-09-20T18:57:11.153215+0000","last_peered":"2024-09-20T18:57:11.153215+0000","last_clean":"2024-09-20T18:57:11.153215+0000","last_became_active":"2024-09-20T18:49:49.962523+0000","last_became_peered":"2024-09-20T18:49:49.962523+0000","last_unstale":"2024-09-20T18:57:11.153215+0000","last_undegraded":"2024-09-20T18:57:11.153215+0000","last_fullsized":"2024-09-20T18:57:11.153215+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:18.108175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:18.108175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":343,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.139210+0000","last_change":"2024-09-20T18:54:26.882441+0000","last_active":"2024-09-20T18:57:11.139210+0000","last_peered":"2024-09-20T18:57:11.139210+0000","last_clean":"2024-09-20T18:57:11.139210+0000","last_became_active":"2024-09-20T18:49:51.967313+0000","last_became_peered":"2024-09-20T18:49:51.967313+0000","last_unstale":"2024-09-20T18:57:11.139210+0000","last_undegraded":"2024-09-20T18:57:11.139210+0000","last_fullsized":"2024-09-20T18:57:11.139210+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:26.882375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:26.882375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":382,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.150807+0000","last_change":"2024-09-20T18:55:28.843363+0000","last_active":"2024-09-20T18:57:11.150807+0000","last_peered":"2024-09-20T18:57:11.150807+0000","last_clean":"2024-09-20T18:57:11.150807+0000","last_became_active":"2024-09-20T18:55:28.830801+0000","last_became_peered":"2024-09-20T18:55:28.830801+0000","last_unstale":"2024-09-20T18:57:11.150807+0000","last_undegraded":"2024-09-20T18:57:11.150807+0000","last_fullsized":"2024-09-20T18:57:11.150807+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:54:21.106677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:21.106677+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":353,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.119147+0000","last_change":"2024-09-20T18:55:19.258723+0000","last_active":"2024-09-20T18:57:09.119147+0000","last_peered":"2024-09-20T18:57:09.119147+0000","last_clean":"2024-09-20T18:57:09.119147+0000","last_became_active":"2024-09-20T18:49:45.942105+0000","last_became_peered":"2024-09-20T18:49:45.942105+0000","last_unstale":"2024-09-20T18:57:09.119147+0000","last_undegraded":"2024-09-20T18:57:09.119147+0000","last_fullsized":"2024-09-20T18:57:09.119147+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:19.258674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:19.258674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":352,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.149622+0000","last_change":"2024-09-20T18:55:28.825004+0000","last_active":"2024-09-20T18:57:09.149622+0000","last_peered":"2024-09-20T18:57:09.149622+0000","last_clean":"2024-09-20T18:57:09.149622+0000","last_became_active":"2024-09-20T18:55:28.824355+0000","last_became_peered":"2024-09-20T18:55:28.824355+0000","last_unstale":"2024-09-20T18:57:09.149622+0000","last_undegraded":"2024-09-20T18:57:09.149622+0000","last_fullsized":"2024-09-20T18:57:09.149622+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:46.329547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:46.329547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":335,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.153332+0000","last_change":"2024-09-20T18:55:28.825607+0000","last_active":"2024-09-20T18:57:11.153332+0000","last_peered":"2024-09-20T18:57:11.153332+0000","last_clean":"2024-09-20T18:57:11.153332+0000","last_became_active":"2024-09-20T18:55:28.825296+0000","last_became_peered":"2024-09-20T18:55:28.825296+0000","last_unstale":"2024-09-20T18:57:11.153332+0000","last_undegraded":"2024-09-20T18:57:11.153332+0000","last_fullsized":"2024-09-20T18:57:11.153332+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:43.118805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:43.118805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":371,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.156047+0000","last_change":"2024-09-20T18:55:28.836789+0000","last_active":"2024-09-20T18:57:11.156047+0000","last_peered":"2024-09-20T18:57:11.156047+0000","last_clean":"2024-09-20T18:57:11.156047+0000","last_became_active":"2024-09-20T18:55:28.824579+0000","last_became_peered":"2024-09-20T18:55:28.824579+0000","last_unstale":"2024-09-20T18:57:11.156047+0000","last_undegraded":"2024-09-20T18:57:11.156047+0000","last_fullsized":"2024-09-20T18:57:11.156047+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-20T18:55:20.282193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:20.282193+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":355,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.140972+0000","last_change":"2024-09-20T18:54:55.900487+0000","last_active":"2024-09-20T18:57:11.140972+0000","last_peered":"2024-09-20T18:57:11.140972+0000","last_clean":"2024-09-20T18:57:11.140972+0000","last_became_active":"2024-09-20T18:49:45.940988+0000","last_became_peered":"2024-09-20T18:49:45.940988+0000","last_unstale":"2024-09-20T18:57:11.140972+0000","last_undegraded":"2024-09-20T18:57:11.140972+0000","last_fullsized":"2024-09-20T18:57:11.140972+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:55.899611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:55.899611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":83,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.114595+0000","last_change":"2024-09-20T18:55:28.832864+0000","last_active":"2024-09-20T18:57:09.114595+0000","last_peered":"2024-09-20T18:57:09.114595+0000","last_clean":"2024-09-20T18:57:09.114595+0000","last_became_active":"2024-09-20T18:55:28.832534+0000","last_became_peered":"2024-09-20T18:55:28.832534+0000","last_unstale":"2024-09-20T18:57:09.114595+0000","last_undegraded":"2024-09-20T18:57:09.114595+0000","last_fullsized":"2024-09-20T18:57:09.114595+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:52.635138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:52.635138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":325,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.149685+0000","last_change":"2024-09-20T18:51:09.761331+0000","last_active":"2024-09-20T18:57:09.149685+0000","last_peered":"2024-09-20T18:57:09.149685+0000","last_clean":"2024-09-20T18:57:09.149685+0000","last_became_active":"2024-09-20T18:49:51.968662+0000","last_became_peered":"2024-09-20T18:49:51.968662+0000","last_unstale":"2024-09-20T18:57:09.149685+0000","last_undegraded":"2024-09-20T18:57:09.149685+0000","last_fullsized":"2024-09-20T18:57:09.149685+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:09.761283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:09.761283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":327,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.152293+0000","last_change":"2024-09-20T18:57:10.970843+0000","last_active":"2024-09-20T18:57:11.152293+0000","last_peered":"2024-09-20T18:57:11.152293+0000","last_clean":"2024-09-20T18:57:11.152293+0000","last_became_active":"2024-09-20T18:49:51.968216+0000","last_became_peered":"2024-09-20T18:49:51.968216+0000","last_unstale":"2024-09-20T18:57:11.152293+0000","last_undegraded":"2024-09-20T18:57:11.152293+0000","last_fullsized":"2024-09-20T18:57:11.152293+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:10.970779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:10.970779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:38.733512+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":348,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.156283+0000","last_change":"2024-09-20T18:55:28.295445+0000","last_active":"2024-09-20T18:57:11.156283+0000","last_peered":"2024-09-20T18:57:11.156283+0000","last_clean":"2024-09-20T18:57:11.156283+0000","last_became_active":"2024-09-20T18:49:45.937992+0000","last_became_peered":"2024-09-20T18:49:45.937992+0000","last_unstale":"2024-09-20T18:57:11.156283+0000","last_undegraded":"2024-09-20T18:57:11.156283+0000","last_fullsized":"2024-09-20T18:57:11.156283+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:28.295399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:28.295399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":134,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-20T18:57:09.119410+0000","last_change":"2024-09-20T18:55:28.852468+0000","last_active":"2024-09-20T18:57:09.119410+0000","last_peered":"2024-09-20T18:57:09.119410+0000","last_clean":"2024-09-20T18:57:09.119410+0000","last_became_active":"2024-09-20T18:55:28.834675+0000","last_became_peered":"2024-09-20T18:55:28.834675+0000","last_unstale":"2024-09-20T18:57:09.119410+0000","last_undegraded":"2024-09-20T18:57:09.119410+0000","last_fullsized":"2024-09-20T18:57:09.119410+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:54:44.719676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.719676+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":12,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":331,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-20T18:57:11.159651+0000","last_change":"2024-09-20T18:51:28.120100+0000","last_active":"2024-09-20T18:57:11.159651+0000","last_peered":"2024-09-20T18:57:11.159651+0000","last_clean":"2024-09-20T18:57:11.159651+0000","last_became_active":"2024-09-20T18:49:49.957582+0000","last_became_peered":"2024-09-20T18:49:49.957582+0000","last_unstale":"2024-09-20T18:57:11.159651+0000","last_undegraded":"2024-09-20T18:57:11.159651+0000","last_fullsized":"2024-09-20T18:57:11.159651+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:28.119694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:28.119694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":4224253976,"num_objects":64458,"num_object_clones":0,"num_object_copies":193374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1360,"num_objects_misplaced":2638,"num_objects_unfound":0,"num_objects_dirty":64458,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":167720,"num_write_kb":7429633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24058,"num_bytes_recovered":1576665088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":14159917056,"data_stored":14159904840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74253,"ondisk_log_size":74253,"up":48,"acting":46,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":67,"ondisk_log_size":67,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":695,"num_read_kb":452,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":1474,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":12140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":394,"ondisk_log_size":394,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":5275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":440,"num_read_kb":435,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266781,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1521440,"kb_used_data":1309772,"kb_used_omap":1,"kb_used_meta":211646,"kb_avail":92223712,"statfs":{"total":95995035648,"available":94437081088,"internally_reserved":0,"allocated":1341206528,"data_stored":1340815877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":216725962},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":0,"seq":193273528420,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757353,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2316440,"kb_used_data":2072836,"kb_used_omap":1,"kb_used_meta":243582,"kb_avail":91428712,"statfs":{"total":95995035648,"available":93623001088,"internally_reserved":0,"allocated":2122584064,"data_stored":2122201088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":249428421},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986286,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1482664,"kb_used_data":1311256,"kb_used_omap":1,"kb_used_meta":171390,"kb_avail":92262488,"statfs":{"total":95995035648,"available":94476787712,"internally_reserved":0,"allocated":1342726144,"data_stored":1342323696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":175503813},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182517,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2159400,"kb_used_data":1821400,"kb_used_omap":1,"kb_used_meta":337982,"kb_avail":91585752,"statfs":{"total":95995035648,"available":93783810048,"internally_reserved":0,"allocated":1865113600,"data_stored":1864710106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":346094023},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411450,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2538908,"kb_used_data":2186824,"kb_used_omap":1,"kb_used_meta":351870,"kb_avail":91206244,"statfs":{"total":95995035648,"available":93395193856,"internally_reserved":0,"allocated":2239307776,"data_stored":2238927885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":360315333},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574976,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2095168,"kb_used_data":1811312,"kb_used_omap":1,"kb_used_meta":283838,"kb_avail":91649984,"statfs":{"total":95995035648,"available":93849583616,"internally_reserved":0,"allocated":1854783488,"data_stored":1854357845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":290650565},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738502,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2110776,"kb_used_data":1768612,"kb_used_omap":1,"kb_used_meta":342142,"kb_avail":91634376,"statfs":{"total":95995035648,"available":93833601024,"internally_reserved":0,"allocated":1811058688,"data_stored":1810650458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":350353861},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1796734976,"data_stored":1796734976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1845100544,"data_stored":1845100544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2236878848,"data_stored":2236874776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1847066624,"data_stored":1847066624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1337262080,"data_stored":1337262080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2110001152,"data_stored":2109997080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1649541120,"data_stored":1649541120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1337331712,"data_stored":1337327640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T18:57:14.952 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:14.976 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:14.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:14.977 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:15.078 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:15.103 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:15.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:15.104 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:15.205 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:15.231 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T18:57:15.231 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-20T18:57:15.231 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.232 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:171533 8.a 8:5e004ffe:::benchmark_data_smithi083_97422_object55442:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e219)", 2024-09-20T18:57:15.232 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:57:15.206387+0000", 2024-09-20T18:57:15.232 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.010131032999999999, 2024-09-20T18:57:15.232 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010160366000000001, 2024-09-20T18:57:15.232 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:57:15.232 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:57:15.232 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "waiting for sub ops", 2024-09-20T18:57:15.232 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:57:15.232 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:57:15.232 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:57:15.232 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 171533 2024-09-20T18:57:15.232 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.232 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:57:15.232 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.233 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:57:15.233 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.206387+0000", 2024-09-20T18:57:15.233 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:15.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.233 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.233 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:57:15.233 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.206387+0000", 2024-09-20T18:57:15.233 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:15.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.233 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.233 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:57:15.233 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.206389+0000", 2024-09-20T18:57:15.233 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5230000000000001e-06 2024-09-20T18:57:15.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.234 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.234 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:57:15.234 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.206394+0000", 2024-09-20T18:57:15.234 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7119999999999997e-06 2024-09-20T18:57:15.234 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.234 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.234 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:57:15.234 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.206409+0000", 2024-09-20T18:57:15.234 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4263999999999999e-05 2024-09-20T18:57:15.234 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.235 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.235 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:57:15.235 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.206416+0000", 2024-09-20T18:57:15.235 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7109999999999995e-06 2024-09-20T18:57:15.235 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.235 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.235 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:15.235 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.206474+0000", 2024-09-20T18:57:15.236 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7192999999999999e-05 2024-09-20T18:57:15.236 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.236 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.236 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:57:15.236 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.206609+0000", 2024-09-20T18:57:15.236 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013546200000000001 2024-09-20T18:57:15.236 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.236 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.237 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,5", 2024-09-20T18:57:15.237 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.206733+0000", 2024-09-20T18:57:15.237 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012423999999999999 2024-09-20T18:57:15.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.237 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.237 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:15.237 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.209962+0000", 2024-09-20T18:57:15.237 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0032288909999999998 2024-09-20T18:57:15.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.238 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.238 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:57:15.238 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.209975+0000", 2024-09-20T18:57:15.238 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2995e-05 2024-09-20T18:57:15.238 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:15.238 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:15.239 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:15.239 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.239 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.239 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:171534 8.3 8:cf404627:::benchmark_data_smithi083_97422_object55443:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e219)", 2024-09-20T18:57:15.239 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:57:15.206470+0000", 2024-09-20T18:57:15.239 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.010048408999999999, 2024-09-20T18:57:15.239 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010161762, 2024-09-20T18:57:15.239 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:57:15.240 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:57:15.240 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "waiting for sub ops", 2024-09-20T18:57:15.240 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:57:15.240 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:57:15.240 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:57:15.240 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 171534 2024-09-20T18:57:15.240 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.240 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:57:15.240 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.241 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:57:15.241 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.206470+0000", 2024-09-20T18:57:15.241 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:15.241 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.241 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.241 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:57:15.241 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.206470+0000", 2024-09-20T18:57:15.241 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:15.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.242 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.242 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:57:15.242 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.206471+0000", 2024-09-20T18:57:15.242 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4499999999999997e-07 2024-09-20T18:57:15.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.242 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.242 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:57:15.243 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.206474+0000", 2024-09-20T18:57:15.243 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.123e-06 2024-09-20T18:57:15.243 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.243 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.243 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:57:15.243 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.206481+0000", 2024-09-20T18:57:15.243 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1679999999999998e-06 2024-09-20T18:57:15.243 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.244 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.244 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:57:15.244 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.206486+0000", 2024-09-20T18:57:15.244 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3380000000000004e-06 2024-09-20T18:57:15.244 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.244 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.244 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:15.244 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.206538+0000", 2024-09-20T18:57:15.244 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1481999999999999e-05 2024-09-20T18:57:15.245 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.245 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.245 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:57:15.245 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.206640+0000", 2024-09-20T18:57:15.245 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000102803 2024-09-20T18:57:15.245 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.245 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.245 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,5", 2024-09-20T18:57:15.245 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.208538+0000", 2024-09-20T18:57:15.246 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0018974459999999999 2024-09-20T18:57:15.246 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:15.246 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:15.246 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:15.246 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.246 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.246 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:171537 8.3 8:c75c7759:::benchmark_data_smithi083_97422_object55446:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e219)", 2024-09-20T18:57:15.246 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:57:15.206692+0000", 2024-09-20T18:57:15.247 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0098258489999999993, 2024-09-20T18:57:15.247 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010004245, 2024-09-20T18:57:15.247 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:57:15.247 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:57:15.247 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "waiting for sub ops", 2024-09-20T18:57:15.247 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:57:15.247 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:57:15.247 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:57:15.248 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 171537 2024-09-20T18:57:15.248 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.248 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:57:15.248 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.248 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:57:15.248 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.206692+0000", 2024-09-20T18:57:15.248 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:15.248 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.249 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.249 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:57:15.249 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.206692+0000", 2024-09-20T18:57:15.249 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:15.249 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.249 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.249 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:57:15.249 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.206693+0000", 2024-09-20T18:57:15.249 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4099999999999998e-07 2024-09-20T18:57:15.250 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.250 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.250 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:57:15.250 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.206697+0000", 2024-09-20T18:57:15.250 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5080000000000001e-06 2024-09-20T18:57:15.250 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.250 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.250 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:57:15.250 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.206703+0000", 2024-09-20T18:57:15.251 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4030000000000004e-06 2024-09-20T18:57:15.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.251 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.251 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:57:15.251 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.206709+0000", 2024-09-20T18:57:15.251 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7259999999999999e-06 2024-09-20T18:57:15.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.251 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.251 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:15.251 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.212111+0000", 2024-09-20T18:57:15.251 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0054026539999999998 2024-09-20T18:57:15.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.251 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.251 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:57:15.252 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.212220+0000", 2024-09-20T18:57:15.252 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000108242 2024-09-20T18:57:15.252 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.252 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.252 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,5", 2024-09-20T18:57:15.252 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.212330+0000", 2024-09-20T18:57:15.252 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011042700000000001 2024-09-20T18:57:15.252 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:15.252 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:15.252 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:15.252 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.252 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.252 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:171538 8.1 8:8207091d:::benchmark_data_smithi083_97422_object55447:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e219)", 2024-09-20T18:57:15.252 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:57:15.206760+0000", 2024-09-20T18:57:15.253 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0097585599999999995, 2024-09-20T18:57:15.253 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010005504, 2024-09-20T18:57:15.253 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:57:15.253 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:57:15.253 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "waiting for sub ops", 2024-09-20T18:57:15.253 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:57:15.253 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:57:15.253 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:57:15.253 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 171538 2024-09-20T18:57:15.253 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.253 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:57:15.253 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.253 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:57:15.253 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.206760+0000", 2024-09-20T18:57:15.254 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:15.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.254 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.254 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:57:15.254 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.206760+0000", 2024-09-20T18:57:15.254 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:15.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.254 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.254 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:57:15.254 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.206760+0000", 2024-09-20T18:57:15.254 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3000000000000001e-07 2024-09-20T18:57:15.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.254 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.255 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:57:15.255 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.206763+0000", 2024-09-20T18:57:15.255 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7219999999999999e-06 2024-09-20T18:57:15.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.255 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.255 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:57:15.255 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.206769+0000", 2024-09-20T18:57:15.255 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7910000000000001e-06 2024-09-20T18:57:15.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.255 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.255 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:57:15.255 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.206774+0000", 2024-09-20T18:57:15.256 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.938e-06 2024-09-20T18:57:15.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.256 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.256 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:15.256 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.206813+0000", 2024-09-20T18:57:15.256 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9138999999999999e-05 2024-09-20T18:57:15.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.256 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.256 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:57:15.256 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.206932+0000", 2024-09-20T18:57:15.256 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000119048 2024-09-20T18:57:15.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.256 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.256 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,2", 2024-09-20T18:57:15.257 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.207026+0000", 2024-09-20T18:57:15.257 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4621000000000001e-05 2024-09-20T18:57:15.257 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.257 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.257 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:57:15.257 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.209570+0000", 2024-09-20T18:57:15.257 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002543029 2024-09-20T18:57:15.257 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:15.257 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:15.257 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:15.257 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.257 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.257 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:171539 8.3 8:c37609fd:::benchmark_data_smithi083_97422_object55448:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e219)", 2024-09-20T18:57:15.257 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:57:15.209138+0000", 2024-09-20T18:57:15.258 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0073804480000000004, 2024-09-20T18:57:15.258 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0076961099999999999, 2024-09-20T18:57:15.258 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:57:15.258 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:57:15.258 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "waiting for sub ops", 2024-09-20T18:57:15.258 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:57:15.258 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:57:15.258 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:57:15.258 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 171539 2024-09-20T18:57:15.258 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.258 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:57:15.258 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.258 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:57:15.259 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.209138+0000", 2024-09-20T18:57:15.259 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:15.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.259 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:57:15.259 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.209138+0000", 2024-09-20T18:57:15.259 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:15.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.259 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:57:15.259 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.209139+0000", 2024-09-20T18:57:15.259 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6279999999999999e-06 2024-09-20T18:57:15.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.260 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:57:15.260 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.209145+0000", 2024-09-20T18:57:15.260 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6899999999999997e-06 2024-09-20T18:57:15.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.260 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.260 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:57:15.260 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.209156+0000", 2024-09-20T18:57:15.260 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0891e-05 2024-09-20T18:57:15.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.260 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.260 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:57:15.260 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.209164+0000", 2024-09-20T18:57:15.260 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2290000000000003e-06 2024-09-20T18:57:15.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.261 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.261 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:15.261 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.215510+0000", 2024-09-20T18:57:15.261 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063457069999999999 2024-09-20T18:57:15.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.261 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.261 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:57:15.261 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.215630+0000", 2024-09-20T18:57:15.261 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012011 2024-09-20T18:57:15.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.261 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.261 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,5", 2024-09-20T18:57:15.261 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.215750+0000", 2024-09-20T18:57:15.261 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000120024 2024-09-20T18:57:15.261 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:15.262 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:15.262 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:15.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.262 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.262 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.24955.0:171531 8.b e219/160)", 2024-09-20T18:57:15.262 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:57:15.212237+0000", 2024-09-20T18:57:15.262 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0042811469999999999, 2024-09-20T18:57:15.262 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0046640980000000002, 2024-09-20T18:57:15.262 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:57:15.262 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:57:15.262 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "queued for pg", 2024-09-20T18:57:15.262 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:57:15.262 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.262 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:57:15.263 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.212237+0000", 2024-09-20T18:57:15.263 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:15.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.263 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.263 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:57:15.263 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.212237+0000", 2024-09-20T18:57:15.263 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:15.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.263 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.263 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:57:15.263 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.212238+0000", 2024-09-20T18:57:15.263 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1960000000000001e-06 2024-09-20T18:57:15.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.263 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.264 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:57:15.264 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.212247+0000", 2024-09-20T18:57:15.264 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4630000000000001e-06 2024-09-20T18:57:15.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.264 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:57:15.264 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.212259+0000", 2024-09-20T18:57:15.264 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.234e-05 2024-09-20T18:57:15.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.264 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:57:15.264 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.212267+0000", 2024-09-20T18:57:15.264 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.038e-06 2024-09-20T18:57:15.264 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:15.265 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:15.265 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:15.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.265 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.24955.0:171532 8.b e219/160)", 2024-09-20T18:57:15.265 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:57:15.212308+0000", 2024-09-20T18:57:15.265 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0042107220000000001, 2024-09-20T18:57:15.265 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0046240470000000001, 2024-09-20T18:57:15.265 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:57:15.265 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:57:15.265 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "queued for pg", 2024-09-20T18:57:15.265 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:57:15.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.265 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:57:15.266 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.212308+0000", 2024-09-20T18:57:15.266 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:15.266 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.266 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.266 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:57:15.266 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.212308+0000", 2024-09-20T18:57:15.266 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:15.266 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.266 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.267 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:57:15.267 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.212308+0000", 2024-09-20T18:57:15.267 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.85e-07 2024-09-20T18:57:15.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.267 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.267 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:57:15.267 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.212314+0000", 2024-09-20T18:57:15.267 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4659999999999996e-06 2024-09-20T18:57:15.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.267 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.267 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:57:15.268 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.212319+0000", 2024-09-20T18:57:15.268 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.622e-06 2024-09-20T18:57:15.268 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.268 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.268 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:57:15.268 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.212323+0000", 2024-09-20T18:57:15.268 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8279999999999996e-06 2024-09-20T18:57:15.268 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:15.268 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:15.268 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:15.268 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.268 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.268 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.24955.0:171534 8.3 e219/203)", 2024-09-20T18:57:15.268 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:57:15.212337+0000", 2024-09-20T18:57:15.269 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0041812619999999998, 2024-09-20T18:57:15.269 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0046319229999999996, 2024-09-20T18:57:15.269 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:57:15.269 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:57:15.269 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "queued for pg", 2024-09-20T18:57:15.269 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:57:15.269 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.269 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:57:15.269 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.212337+0000", 2024-09-20T18:57:15.269 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:15.269 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.269 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.269 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:57:15.269 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.212337+0000", 2024-09-20T18:57:15.270 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:15.270 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.270 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.270 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:57:15.270 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.212337+0000", 2024-09-20T18:57:15.270 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.01e-07 2024-09-20T18:57:15.270 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.270 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.270 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:57:15.270 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.212340+0000", 2024-09-20T18:57:15.270 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5629999999999999e-06 2024-09-20T18:57:15.270 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.270 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.270 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:57:15.271 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.212345+0000", 2024-09-20T18:57:15.271 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.434e-06 2024-09-20T18:57:15.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.271 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.271 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:57:15.271 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.212350+0000", 2024-09-20T18:57:15.271 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1330000000000001e-06 2024-09-20T18:57:15.271 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:15.271 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:15.271 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:15.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.271 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.271 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.24955.0:171537 8.3 e219/203)", 2024-09-20T18:57:15.271 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:57:15.215527+0000", 2024-09-20T18:57:15.272 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.00099089899999999995, 2024-09-20T18:57:15.272 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014744669999999999, 2024-09-20T18:57:15.272 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:57:15.272 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:57:15.272 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "queued for pg", 2024-09-20T18:57:15.272 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:57:15.272 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.272 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:57:15.272 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.215527+0000", 2024-09-20T18:57:15.272 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:15.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.272 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.272 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:57:15.272 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.215527+0000", 2024-09-20T18:57:15.272 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:15.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.273 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.273 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:57:15.273 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.215528+0000", 2024-09-20T18:57:15.273 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9100000000000003e-07 2024-09-20T18:57:15.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.273 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.273 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:57:15.273 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.215532+0000", 2024-09-20T18:57:15.273 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.366e-06 2024-09-20T18:57:15.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.273 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.273 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:57:15.273 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.215542+0000", 2024-09-20T18:57:15.273 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0893999999999999e-05 2024-09-20T18:57:15.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.274 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.274 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:57:15.274 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.215549+0000", 2024-09-20T18:57:15.274 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9140000000000002e-06 2024-09-20T18:57:15.274 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:15.274 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:15.274 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:15.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.274 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.274 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.24955.0:171534 8.3 e219/203)", 2024-09-20T18:57:15.274 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:57:15.215580+0000", 2024-09-20T18:57:15.274 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.00093839100000000001, 2024-09-20T18:57:15.274 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001455278, 2024-09-20T18:57:15.274 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:57:15.275 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:57:15.275 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "queued for pg", 2024-09-20T18:57:15.275 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:57:15.275 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.275 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:57:15.275 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.215580+0000", 2024-09-20T18:57:15.275 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:15.275 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.275 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.275 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:57:15.275 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.215580+0000", 2024-09-20T18:57:15.275 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:15.275 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.275 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.275 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:57:15.276 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.215580+0000", 2024-09-20T18:57:15.276 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.58e-07 2024-09-20T18:57:15.276 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.276 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.276 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:57:15.276 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.215583+0000", 2024-09-20T18:57:15.276 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3970000000000001e-06 2024-09-20T18:57:15.276 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.276 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.276 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:57:15.276 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.215589+0000", 2024-09-20T18:57:15.276 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.93e-06 2024-09-20T18:57:15.276 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:15.277 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:15.277 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:57:15.277 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:15.215593+0000", 2024-09-20T18:57:15.277 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4309999999999996e-06 2024-09-20T18:57:15.277 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:15.277 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:15.277 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:15.277 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:15.277 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T18:57:15.277 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 10 2024-09-20T18:57:15.278 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T18:57:15.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:15.279 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:15.279 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:15.380 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:15.405 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:15.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:15.406 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:15.507 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:15.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:15 smithi142 ceph-mon[32508]: 6.19 scrub starts 2024-09-20T18:57:15.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:15 smithi142 ceph-mon[32508]: 6.19 scrub ok 2024-09-20T18:57:15.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:15 smithi142 ceph-mon[32508]: 4.19 scrub starts 2024-09-20T18:57:15.520 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:15 smithi142 ceph-mon[32508]: 4.19 scrub ok 2024-09-20T18:57:15.520 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:15 smithi142 ceph-mon[32508]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T18:57:15.520 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:15 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T18:57:15.520 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:15 smithi142 ceph-mon[32508]: osdmap e219: 8 total, 8 up, 7 in 2024-09-20T18:57:15.520 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:15 smithi142 ceph-mon[32508]: 5.1d scrub starts 2024-09-20T18:57:15.520 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:15 smithi142 ceph-mon[32508]: 5.1d scrub ok 2024-09-20T18:57:15.542 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:15.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:15.543 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:15.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:15 smithi083 ceph-mon[36260]: 6.19 scrub starts 2024-09-20T18:57:15.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:15 smithi083 ceph-mon[36260]: 6.19 scrub ok 2024-09-20T18:57:15.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:15 smithi083 ceph-mon[36260]: 4.19 scrub starts 2024-09-20T18:57:15.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:15 smithi083 ceph-mon[36260]: 4.19 scrub ok 2024-09-20T18:57:15.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:15 smithi083 ceph-mon[36260]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T18:57:15.550 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:15 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T18:57:15.550 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:15 smithi083 ceph-mon[36260]: osdmap e219: 8 total, 8 up, 7 in 2024-09-20T18:57:15.550 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:15 smithi083 ceph-mon[36260]: 5.1d scrub starts 2024-09-20T18:57:15.550 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:15 smithi083 ceph-mon[36260]: 5.1d scrub ok 2024-09-20T18:57:15.550 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:15 smithi083 ceph-mon[29421]: 6.19 scrub starts 2024-09-20T18:57:15.550 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:15 smithi083 ceph-mon[29421]: 6.19 scrub ok 2024-09-20T18:57:15.550 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:15 smithi083 ceph-mon[29421]: 4.19 scrub starts 2024-09-20T18:57:15.550 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:15 smithi083 ceph-mon[29421]: 4.19 scrub ok 2024-09-20T18:57:15.550 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:15 smithi083 ceph-mon[29421]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T18:57:15.550 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:15 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T18:57:15.550 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:15 smithi083 ceph-mon[29421]: osdmap e219: 8 total, 8 up, 7 in 2024-09-20T18:57:15.551 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:15 smithi083 ceph-mon[29421]: 5.1d scrub starts 2024-09-20T18:57:15.551 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:15 smithi083 ceph-mon[29421]: 5.1d scrub ok 2024-09-20T18:57:15.643 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:15.678 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:15.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:15.680 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:15.780 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:15.806 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:15.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:15.807 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:15.908 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:15.949 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:15.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:15.950 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:16.051 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:16.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:16.088 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:16.088 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:16.189 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:16.223 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:16.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:16.224 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:16.325 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:16.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:16.360 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:16.361 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:16.461 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:16.511 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:16.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:16.512 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:16.612 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:16.654 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:16.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:16.655 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:16.756 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:16.799 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:16 smithi083 ceph-mon[29421]: pgmap v358: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 146 active+clean; 3.9 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 697 op/s; 1360/193971 objects degraded (0.701%); 2638/193971 objects misplaced (1.360%); 7.4 MiB/s, 118 objects/s recovering 2024-09-20T18:57:16.799 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:16 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:57:16.799 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:16 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:57:16.799 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:16 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 1360/193971 objects degraded (0.701%), 1 pg degraded, 2 pgs undersized (PG_DEGRADED) 2024-09-20T18:57:16.800 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:16.800 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:16 smithi083 ceph-mon[36260]: pgmap v358: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 146 active+clean; 3.9 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 697 op/s; 1360/193971 objects degraded (0.701%); 2638/193971 objects misplaced (1.360%); 7.4 MiB/s, 118 objects/s recovering 2024-09-20T18:57:16.800 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:16 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:57:16.800 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:16 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:57:16.800 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:16 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 1360/193971 objects degraded (0.701%), 1 pg degraded, 2 pgs undersized (PG_DEGRADED) 2024-09-20T18:57:16.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:16.801 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:16.874 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_blocked_ops 2024-09-20T18:57:16.893 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:57:16.902 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:16.959 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:16.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:16.960 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:17.019 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:57:16 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:57:17.019 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:16 smithi142 ceph-mon[32508]: pgmap v358: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 146 active+clean; 3.9 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 697 op/s; 1360/193971 objects degraded (0.701%); 2638/193971 objects misplaced (1.360%); 7.4 MiB/s, 118 objects/s recovering 2024-09-20T18:57:17.019 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:16 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:57:17.019 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:16 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:57:17.019 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:16 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 1360/193971 objects degraded (0.701%), 1 pg degraded, 2 pgs undersized (PG_DEGRADED) 2024-09-20T18:57:17.061 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:17.087 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:17.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:17.088 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:17.188 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:17.215 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:17.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:17.216 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:17.317 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:17.354 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:17.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:17.355 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:17.456 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:17.493 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:17.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:17.494 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:17.578 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:57:17.585 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:57:17.585 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:57:17.596 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:17.618 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:17.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:17.619 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:17.623 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:57:17.623 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:57:17.670 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:57:17.670 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:57:17.701 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:57:17.701 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:57:17.720 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:17.736 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:57:17.737 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:57:17.764 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:17.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:17.765 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:17.773 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:57:17.774 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:57:17.806 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:57:17.806 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:57:17.838 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:57:17.838 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:57:17.866 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:17.894 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T18:57:17.909 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:17.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:17.910 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:17.932 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:17 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:57:17.932 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:17 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:57:17.932 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:17 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/4228053990' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:57:17.932 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:17 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:57:17.932 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:17 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:57:17.932 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:17 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/4228053990' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:57:18.012 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:18.019 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:17 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:57:18.019 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:17 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:57:18.019 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:17 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/4228053990' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:57:18.038 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:18.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:18.039 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:18.140 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:18.148 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T18:57:18.164 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:18.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:18.165 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:18.266 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:18.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:18.326 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:18.326 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:18.361 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T18:57:18.361 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:57:18.427 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:18.453 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:18.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:18.454 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:18.555 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:18.579 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:18.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:18.580 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:18.681 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:57:18.706 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:18.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:18.707 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:57:18.808 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:18.843 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:18.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:18.844 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:18.945 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:18.970 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:18.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:18.971 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:19.019 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:18 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:57:19.019 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:18 smithi142 ceph-mon[32508]: 6.11 scrub starts 2024-09-20T18:57:19.019 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:18 smithi142 ceph-mon[32508]: 6.11 scrub ok 2024-09-20T18:57:19.019 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:18 smithi142 ceph-mon[32508]: pgmap v359: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 146 active+clean; 3.6 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.17k op/s; 1625/179121 objects degraded (0.907%); 1861/179121 objects misplaced (1.039%); 9.8 MiB/s, 156 objects/s recovering 2024-09-20T18:57:19.019 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:18 smithi142 ceph-mon[32508]: osdmap e220: 8 total, 8 up, 7 in 2024-09-20T18:57:19.049 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:18 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:57:19.049 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:18 smithi083 ceph-mon[29421]: 6.11 scrub starts 2024-09-20T18:57:19.049 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:18 smithi083 ceph-mon[29421]: 6.11 scrub ok 2024-09-20T18:57:19.049 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:18 smithi083 ceph-mon[29421]: pgmap v359: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 146 active+clean; 3.6 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.17k op/s; 1625/179121 objects degraded (0.907%); 1861/179121 objects misplaced (1.039%); 9.8 MiB/s, 156 objects/s recovering 2024-09-20T18:57:19.050 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:18 smithi083 ceph-mon[29421]: osdmap e220: 8 total, 8 up, 7 in 2024-09-20T18:57:19.050 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:18 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:57:19.050 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:18 smithi083 ceph-mon[36260]: 6.11 scrub starts 2024-09-20T18:57:19.050 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:18 smithi083 ceph-mon[36260]: 6.11 scrub ok 2024-09-20T18:57:19.050 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:18 smithi083 ceph-mon[36260]: pgmap v359: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 146 active+clean; 3.6 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.17k op/s; 1625/179121 objects degraded (0.907%); 1861/179121 objects misplaced (1.039%); 9.8 MiB/s, 156 objects/s recovering 2024-09-20T18:57:19.050 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:18 smithi083 ceph-mon[36260]: osdmap e220: 8 total, 8 up, 7 in 2024-09-20T18:57:19.072 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:19.097 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:19.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:19.098 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:19.199 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:19.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:19.239 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:19.239 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:19.339 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:19.360 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:19.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:19.361 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:19.462 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:19.496 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:19.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:19.497 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:19.598 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:19.639 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:19.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:19.640 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:19.741 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:19.764 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:19.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:19.765 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:19.866 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:19.890 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:19.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:19.891 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:19.992 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:20.013 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:20.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:20.014 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:20.019 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:19 smithi142 ceph-mon[32508]: 3.19 scrub starts 2024-09-20T18:57:20.019 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:19 smithi142 ceph-mon[32508]: 3.19 scrub ok 2024-09-20T18:57:20.049 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:19 smithi083 ceph-mon[29421]: 3.19 scrub starts 2024-09-20T18:57:20.049 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:19 smithi083 ceph-mon[29421]: 3.19 scrub ok 2024-09-20T18:57:20.049 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:19 smithi083 ceph-mon[36260]: 3.19 scrub starts 2024-09-20T18:57:20.049 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:19 smithi083 ceph-mon[36260]: 3.19 scrub ok 2024-09-20T18:57:20.115 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:20.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:20.137 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:20.137 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:20.238 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:20.259 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:20.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:20.260 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:20.360 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:20.384 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:20.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:20.385 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:20.485 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:20.507 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:20.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:20.508 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:20.609 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:20.631 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:20.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:20.632 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:20.733 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:20.756 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:20.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:20.757 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:20.857 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:20.897 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:20.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:20.898 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:20.923 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T18:57:20.981 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:20 smithi083 ceph-mon[29421]: pgmap v361: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 146 active+clean; 3.6 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 657 B/s rd, 0 B/s wr, 971 op/s; 1625/179121 objects degraded (0.907%); 1861/179121 objects misplaced (1.039%); 5.3 MiB/s, 85 objects/s recovering 2024-09-20T18:57:20.981 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:20 smithi083 ceph-mon[29421]: osdmap e221: 8 total, 8 up, 7 in 2024-09-20T18:57:20.981 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:20 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 1625/179121 objects degraded (0.907%), 1 pg degraded, 2 pgs undersized (PG_DEGRADED) 2024-09-20T18:57:20.982 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:57:20 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:57:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:57:20.982 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:20 smithi083 ceph-mon[36260]: pgmap v361: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 146 active+clean; 3.6 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 657 B/s rd, 0 B/s wr, 971 op/s; 1625/179121 objects degraded (0.907%); 1861/179121 objects misplaced (1.039%); 5.3 MiB/s, 85 objects/s recovering 2024-09-20T18:57:20.982 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:20 smithi083 ceph-mon[36260]: osdmap e221: 8 total, 8 up, 7 in 2024-09-20T18:57:20.982 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:20 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 1625/179121 objects degraded (0.907%), 1 pg degraded, 2 pgs undersized (PG_DEGRADED) 2024-09-20T18:57:20.998 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:21.019 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:20 smithi142 ceph-mon[32508]: pgmap v361: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 146 active+clean; 3.6 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 657 B/s rd, 0 B/s wr, 971 op/s; 1625/179121 objects degraded (0.907%); 1861/179121 objects misplaced (1.039%); 5.3 MiB/s, 85 objects/s recovering 2024-09-20T18:57:21.019 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:20 smithi142 ceph-mon[32508]: osdmap e221: 8 total, 8 up, 7 in 2024-09-20T18:57:21.019 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:20 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 1625/179121 objects degraded (0.907%), 1 pg degraded, 2 pgs undersized (PG_DEGRADED) 2024-09-20T18:57:21.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:21.047 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:21.048 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:21.148 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:21.199 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:21.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:21.200 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:21.300 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:21.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:21.343 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:21.343 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:21.444 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:21.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:21.486 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:21.486 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:21.587 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:21.609 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:21.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:21.610 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:21.711 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:21.734 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:21.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:21.735 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:21.835 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:21.854 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_historic_ops 2024-09-20T18:57:21.858 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:21.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:21.859 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:21.885 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:57:21.942 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:57:21.960 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:57:21.984 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:21.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:21.985 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:57:22.019 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:21 smithi142 ceph-mon[32508]: 4.14 scrub starts 2024-09-20T18:57:22.019 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:21 smithi142 ceph-mon[32508]: 4.14 scrub ok 2024-09-20T18:57:22.019 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:21 smithi142 ceph-mon[32508]: osdmap e222: 8 total, 8 up, 7 in 2024-09-20T18:57:22.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:21 smithi083 ceph-mon[29421]: 4.14 scrub starts 2024-09-20T18:57:22.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:21 smithi083 ceph-mon[29421]: 4.14 scrub ok 2024-09-20T18:57:22.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:21 smithi083 ceph-mon[29421]: osdmap e222: 8 total, 8 up, 7 in 2024-09-20T18:57:22.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:21 smithi083 ceph-mon[36260]: 4.14 scrub starts 2024-09-20T18:57:22.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:21 smithi083 ceph-mon[36260]: 4.14 scrub ok 2024-09-20T18:57:22.053 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:21 smithi083 ceph-mon[36260]: osdmap e222: 8 total, 8 up, 7 in 2024-09-20T18:57:22.085 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:22.130 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:22.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:22.131 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:22.232 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:22.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:22.318 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:22.318 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:22.419 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:22.461 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:22.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:22.462 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:22.563 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:22.587 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:22.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:22.588 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:22.688 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:57:22.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:22.715 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:22.715 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:57:22.815 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:22.840 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:22.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:22.841 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:22.860 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:57:22.942 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:22.973 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:22 smithi083 ceph-mon[29421]: pgmap v364: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 146 active+clean; 3.6 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 825 op/s; 1625/179121 objects degraded (0.907%); 1861/179121 objects misplaced (1.039%); 5.3 MiB/s, 85 objects/s recovering 2024-09-20T18:57:22.973 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:22 smithi083 ceph-mon[29421]: 3.3 scrub starts 2024-09-20T18:57:22.973 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:22 smithi083 ceph-mon[29421]: 3.3 scrub ok 2024-09-20T18:57:22.973 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:22 smithi083 ceph-mon[36260]: pgmap v364: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 146 active+clean; 3.6 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 825 op/s; 1625/179121 objects degraded (0.907%); 1861/179121 objects misplaced (1.039%); 5.3 MiB/s, 85 objects/s recovering 2024-09-20T18:57:22.973 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:22 smithi083 ceph-mon[36260]: 3.3 scrub starts 2024-09-20T18:57:22.973 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:22 smithi083 ceph-mon[36260]: 3.3 scrub ok 2024-09-20T18:57:22.982 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:22.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:22.983 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:23.019 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:22 smithi142 ceph-mon[32508]: pgmap v364: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 146 active+clean; 3.6 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 825 op/s; 1625/179121 objects degraded (0.907%); 1861/179121 objects misplaced (1.039%); 5.3 MiB/s, 85 objects/s recovering 2024-09-20T18:57:23.019 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:22 smithi142 ceph-mon[32508]: 3.3 scrub starts 2024-09-20T18:57:23.019 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:22 smithi142 ceph-mon[32508]: 3.3 scrub ok 2024-09-20T18:57:23.083 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:23.123 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:23.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:23.124 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:23.225 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:23.261 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:57:23.264 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-20T18:57:23.282 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:23.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:23.283 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:23.384 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:23.429 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:23.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:23.430 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:23.531 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:23.556 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:23.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:23.557 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:23.658 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:23.686 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:23.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:23.687 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:23.766 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-20T18:57:23.788 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:23.811 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:23.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:23.812 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:23.874 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:23 smithi083 ceph-mon[29421]: 4.a scrub starts 2024-09-20T18:57:23.874 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:23 smithi083 ceph-mon[29421]: 4.a scrub ok 2024-09-20T18:57:23.874 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:23 smithi083 ceph-mon[29421]: 3.1e scrub starts 2024-09-20T18:57:23.874 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:23 smithi083 ceph-mon[29421]: 3.1e scrub ok 2024-09-20T18:57:23.874 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:23 smithi083 ceph-mon[29421]: 4.5 scrub starts 2024-09-20T18:57:23.874 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:23 smithi083 ceph-mon[29421]: 4.5 scrub ok 2024-09-20T18:57:23.874 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:23 smithi083 ceph-mon[29421]: pgmap v365: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 3.4 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 686 op/s; 1736/166770 objects degraded (1.041%); 1281/166770 objects misplaced (0.768%); 4.9 MiB/s, 78 objects/s recovering 2024-09-20T18:57:23.875 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:23 smithi083 ceph-mon[29421]: from='client.25177 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:57:23.875 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:23 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3301676281' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:57:23.875 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:23 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:57:23.875 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:23 smithi083 ceph-mon[36260]: 4.a scrub starts 2024-09-20T18:57:23.875 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:23 smithi083 ceph-mon[36260]: 4.a scrub ok 2024-09-20T18:57:23.875 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:23 smithi083 ceph-mon[36260]: 3.1e scrub starts 2024-09-20T18:57:23.875 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:23 smithi083 ceph-mon[36260]: 3.1e scrub ok 2024-09-20T18:57:23.875 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:23 smithi083 ceph-mon[36260]: 4.5 scrub starts 2024-09-20T18:57:23.875 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:23 smithi083 ceph-mon[36260]: 4.5 scrub ok 2024-09-20T18:57:23.875 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:23 smithi083 ceph-mon[36260]: pgmap v365: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 3.4 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 686 op/s; 1736/166770 objects degraded (1.041%); 1281/166770 objects misplaced (0.768%); 4.9 MiB/s, 78 objects/s recovering 2024-09-20T18:57:23.875 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:23 smithi083 ceph-mon[36260]: from='client.25177 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:57:23.875 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:23 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3301676281' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:57:23.875 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:23 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:57:23.913 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:23.942 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:57:23.942 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:57:23.951 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:23.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:23.952 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:23.982 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:57:23.982 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:57:24.019 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:23 smithi142 ceph-mon[32508]: 4.a scrub starts 2024-09-20T18:57:24.019 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:23 smithi142 ceph-mon[32508]: 4.a scrub ok 2024-09-20T18:57:24.019 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:23 smithi142 ceph-mon[32508]: 3.1e scrub starts 2024-09-20T18:57:24.019 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:23 smithi142 ceph-mon[32508]: 3.1e scrub ok 2024-09-20T18:57:24.019 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:23 smithi142 ceph-mon[32508]: 4.5 scrub starts 2024-09-20T18:57:24.019 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:23 smithi142 ceph-mon[32508]: 4.5 scrub ok 2024-09-20T18:57:24.019 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:23 smithi142 ceph-mon[32508]: pgmap v365: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 3.4 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 686 op/s; 1736/166770 objects degraded (1.041%); 1281/166770 objects misplaced (0.768%); 4.9 MiB/s, 78 objects/s recovering 2024-09-20T18:57:24.019 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:23 smithi142 ceph-mon[32508]: from='client.25177 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:57:24.019 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:23 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3301676281' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:57:24.020 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:23 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:57:24.020 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:57:24.020 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:57:24.053 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:57:24.053 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:57:24.053 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:24.087 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:57:24.087 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:57:24.094 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:24.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:24.095 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:24.122 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:57:24.122 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:57:24.153 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:57:24.153 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:57:24.167 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:57:24.195 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:24.199 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:57:24.199 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:57:24.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:24.251 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:24.252 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:24.352 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:24.391 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:24.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:24.392 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:24.493 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:24.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:24.531 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:24.531 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:24.632 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:24.656 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:24.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:24.656 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:24.757 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:57:24.783 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:24.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:24.784 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:57:24.885 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:24.927 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:24.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:24.928 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:25.019 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:24 smithi142 ceph-mon[32508]: 6.c scrub starts 2024-09-20T18:57:25.019 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:24 smithi142 ceph-mon[32508]: 6.c scrub ok 2024-09-20T18:57:25.019 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:24 smithi142 ceph-mon[32508]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:57:25.019 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:24 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T18:57:25.019 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:24 smithi142 ceph-mon[32508]: osdmap e223: 8 total, 8 up, 7 in 2024-09-20T18:57:25.019 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:24 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3426208908' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:57:25.029 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:57:25.049 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:24 smithi083 ceph-mon[29421]: 6.c scrub starts 2024-09-20T18:57:25.049 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:24 smithi083 ceph-mon[29421]: 6.c scrub ok 2024-09-20T18:57:25.049 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:24 smithi083 ceph-mon[29421]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:57:25.049 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:24 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T18:57:25.049 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:24 smithi083 ceph-mon[29421]: osdmap e223: 8 total, 8 up, 7 in 2024-09-20T18:57:25.049 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:24 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3426208908' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:57:25.050 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:24 smithi083 ceph-mon[36260]: 6.c scrub starts 2024-09-20T18:57:25.050 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:24 smithi083 ceph-mon[36260]: 6.c scrub ok 2024-09-20T18:57:25.050 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:24 smithi083 ceph-mon[36260]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:57:25.050 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:24 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T18:57:25.050 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:24 smithi083 ceph-mon[36260]: osdmap e223: 8 total, 8 up, 7 in 2024-09-20T18:57:25.050 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:24 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3426208908' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:57:25.055 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:25.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:25.056 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:57:25.157 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:25.183 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:25.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:25.184 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:25.284 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:25.305 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:25.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:25.306 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:25.407 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:25.439 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:25.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:25.440 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:25.540 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:57:25.565 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:25.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:25.566 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:57:25.666 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:25.692 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:25.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:25.693 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:25.793 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:25.832 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:25.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:25.833 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:25.934 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:57:25.958 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:25.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:25.959 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:57:26.019 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:25 smithi142 ceph-mon[32508]: pgmap v367: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 3.4 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 686 op/s; 1736/166770 objects degraded (1.041%); 1281/166770 objects misplaced (0.768%); 4.9 MiB/s, 78 objects/s recovering 2024-09-20T18:57:26.019 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:25 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 1736/166770 objects degraded (1.041%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T18:57:26.049 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:25 smithi083 ceph-mon[29421]: pgmap v367: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 3.4 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 686 op/s; 1736/166770 objects degraded (1.041%); 1281/166770 objects misplaced (0.768%); 4.9 MiB/s, 78 objects/s recovering 2024-09-20T18:57:26.049 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:25 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 1736/166770 objects degraded (1.041%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T18:57:26.049 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:25 smithi083 ceph-mon[36260]: pgmap v367: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 3.4 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 686 op/s; 1736/166770 objects degraded (1.041%); 1281/166770 objects misplaced (0.768%); 4.9 MiB/s, 78 objects/s recovering 2024-09-20T18:57:26.049 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:25 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 1736/166770 objects degraded (1.041%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T18:57:26.060 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:26.081 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:26.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:26.082 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:26.183 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:57:26.211 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:26.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:26.211 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:57:26.312 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:26.334 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:26.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:26.334 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:26.435 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:26.477 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:26.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:26.478 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:26.578 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:26.599 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:26.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:26.600 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:26.701 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:26.736 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:26.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:26.738 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:26.838 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:26.863 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:26.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:26.863 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:26.964 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:27.004 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:27.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:27.005 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:27.106 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:27.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:27.148 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:27.149 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:27.249 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:27.268 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:57:26 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:57:27.276 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:27.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:27.277 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:27.378 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:57:27.402 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:27.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:27.403 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:57:27.504 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:57:27.527 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:27.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:27.528 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:57:27.628 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:27.649 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:27.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:27.650 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:27.751 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:27.775 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:27.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:27.776 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:27.877 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:27.910 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:27.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:27.911 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:28.011 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:57:28.034 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:28.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:28.035 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:57:28.135 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:28.158 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:28.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:28.159 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:28.259 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:28.269 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:27 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:57:28.269 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:27 smithi142 ceph-mon[32508]: pgmap v368: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 3.1 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.26k op/s; 1845/151920 objects degraded (1.214%); 1054/151920 objects misplaced (0.694%); 5.1 MiB/s, 81 objects/s recovering 2024-09-20T18:57:28.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:28.296 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:28.296 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:28.299 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:27 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:57:28.299 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:27 smithi083 ceph-mon[29421]: pgmap v368: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 3.1 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.26k op/s; 1845/151920 objects degraded (1.214%); 1054/151920 objects misplaced (0.694%); 5.1 MiB/s, 81 objects/s recovering 2024-09-20T18:57:28.299 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:27 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:57:28.299 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:27 smithi083 ceph-mon[36260]: pgmap v368: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 3.1 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.26k op/s; 1845/151920 objects degraded (1.214%); 1054/151920 objects misplaced (0.694%); 5.1 MiB/s, 81 objects/s recovering 2024-09-20T18:57:28.397 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:28.419 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:28.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:28.420 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:28.521 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:28.554 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:28.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:28.555 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:28.655 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:28.692 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:28.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:28.693 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:28.794 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:28.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:28.830 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:28.830 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:28.931 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:28.964 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:28.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:28.965 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:29.066 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:29.105 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:29.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:29.106 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:29.207 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:29.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:29.233 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:29.233 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:29.245 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2024-09-20T18:57:22.930660+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3630635571,"num_objects":55590,"num_object_clones":0,"num_object_copies":166770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1736,"num_objects_misplaced":1281,"num_objects_unfound":0,"num_objects_dirty":55590,"num_whiteouts":0,"num_read":1296,"num_read_kb":1153,"num_write":177336,"num_write_kb":7431306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25134,"num_bytes_recovered":1642138569,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74676,"ondisk_log_size":74676,"up":444,"acting":443,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":12591024,"kb_used_data":10621968,"kb_used_omap":10,"kb_used_meta":1968821,"kb_avail":643625040,"statfs":{"total":671965249536,"available":659072040960,"internally_reserved":0,"allocated":10876895232,"data_stored":10873934808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11158,"internal_metadata":2016072810},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,5],"upper_bound":16},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-269811712,"num_objects":-4117,"num_object_clones":0,"num_object_copies":-12351,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":111,"num_objects_misplaced":-580,"num_objects_unfound":0,"num_objects_dirty":-4117,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4117,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":469,"num_bytes_recovered":30736384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.005497"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":363,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.719322+0000","last_change":"2024-09-20T18:55:28.037494+0000","last_active":"2024-09-20T18:57:20.719322+0000","last_peered":"2024-09-20T18:57:20.719322+0000","last_clean":"2024-09-20T18:57:20.719322+0000","last_became_active":"2024-09-20T18:49:45.943201+0000","last_became_peered":"2024-09-20T18:49:45.943201+0000","last_unstale":"2024-09-20T18:57:20.719322+0000","last_undegraded":"2024-09-20T18:57:20.719322+0000","last_fullsized":"2024-09-20T18:57:20.719322+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:28.037449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:28.037449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":410,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.714286+0000","last_change":"2024-09-20T18:55:28.838729+0000","last_active":"2024-09-20T18:57:20.714286+0000","last_peered":"2024-09-20T18:57:20.714286+0000","last_clean":"2024-09-20T18:57:20.714286+0000","last_became_active":"2024-09-20T18:55:28.819637+0000","last_became_peered":"2024-09-20T18:55:28.819637+0000","last_unstale":"2024-09-20T18:57:20.714286+0000","last_undegraded":"2024-09-20T18:57:20.714286+0000","last_fullsized":"2024-09-20T18:57:20.714286+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:50.231620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:50.231620+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":356,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.760265+0000","last_change":"2024-09-20T18:54:25.361988+0000","last_active":"2024-09-20T18:57:20.760265+0000","last_peered":"2024-09-20T18:57:20.760265+0000","last_clean":"2024-09-20T18:57:20.760265+0000","last_became_active":"2024-09-20T18:49:49.959514+0000","last_became_peered":"2024-09-20T18:49:49.959514+0000","last_unstale":"2024-09-20T18:57:20.760265+0000","last_undegraded":"2024-09-20T18:57:20.760265+0000","last_fullsized":"2024-09-20T18:57:20.760265+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:25.361935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:25.361935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":351,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.714558+0000","last_change":"2024-09-20T18:54:15.221404+0000","last_active":"2024-09-20T18:57:20.714558+0000","last_peered":"2024-09-20T18:57:20.714558+0000","last_clean":"2024-09-20T18:57:20.714558+0000","last_became_active":"2024-09-20T18:49:51.969421+0000","last_became_peered":"2024-09-20T18:49:51.969421+0000","last_unstale":"2024-09-20T18:57:20.714558+0000","last_undegraded":"2024-09-20T18:57:20.714558+0000","last_fullsized":"2024-09-20T18:57:20.714558+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:15.221342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:15.221342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":359,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.758546+0000","last_change":"2024-09-20T18:55:28.822034+0000","last_active":"2024-09-20T18:57:20.758546+0000","last_peered":"2024-09-20T18:57:20.758546+0000","last_clean":"2024-09-20T18:57:20.758546+0000","last_became_active":"2024-09-20T18:55:28.821717+0000","last_became_peered":"2024-09-20T18:55:28.821717+0000","last_unstale":"2024-09-20T18:57:20.758546+0000","last_undegraded":"2024-09-20T18:57:20.758546+0000","last_fullsized":"2024-09-20T18:57:20.758546+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.216188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.216188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":355,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.759453+0000","last_change":"2024-09-20T18:55:28.825144+0000","last_active":"2024-09-20T18:57:20.759453+0000","last_peered":"2024-09-20T18:57:20.759453+0000","last_clean":"2024-09-20T18:57:20.759453+0000","last_became_active":"2024-09-20T18:55:28.824826+0000","last_became_peered":"2024-09-20T18:55:28.824826+0000","last_unstale":"2024-09-20T18:57:20.759453+0000","last_undegraded":"2024-09-20T18:57:20.759453+0000","last_fullsized":"2024-09-20T18:57:20.759453+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:25.208073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:25.208073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":377,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.757923+0000","last_change":"2024-09-20T18:57:13.774827+0000","last_active":"2024-09-20T18:57:20.757923+0000","last_peered":"2024-09-20T18:57:20.757923+0000","last_clean":"2024-09-20T18:57:20.757923+0000","last_became_active":"2024-09-20T18:49:47.952150+0000","last_became_peered":"2024-09-20T18:49:47.952150+0000","last_unstale":"2024-09-20T18:57:20.757923+0000","last_undegraded":"2024-09-20T18:57:20.757923+0000","last_fullsized":"2024-09-20T18:57:20.757923+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:57:13.774305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:13.774305+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:25.245744+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":355,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.714272+0000","last_change":"2024-09-20T18:55:28.820753+0000","last_active":"2024-09-20T18:57:20.714272+0000","last_peered":"2024-09-20T18:57:20.714272+0000","last_clean":"2024-09-20T18:57:20.714272+0000","last_became_active":"2024-09-20T18:55:28.820362+0000","last_became_peered":"2024-09-20T18:55:28.820362+0000","last_unstale":"2024-09-20T18:57:20.714272+0000","last_undegraded":"2024-09-20T18:57:20.714272+0000","last_fullsized":"2024-09-20T18:57:20.714272+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:11.246922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:11.246922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"59'1","reported_seq":388,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.761831+0000","last_change":"2024-09-20T18:55:29.340140+0000","last_active":"2024-09-20T18:57:20.761831+0000","last_peered":"2024-09-20T18:57:20.761831+0000","last_clean":"2024-09-20T18:57:20.761831+0000","last_became_active":"2024-09-20T18:55:28.833386+0000","last_became_peered":"2024-09-20T18:55:28.833386+0000","last_unstale":"2024-09-20T18:57:20.761831+0000","last_undegraded":"2024-09-20T18:57:20.761831+0000","last_fullsized":"2024-09-20T18:57:20.761831+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:55:29.340083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:29.340083+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":376,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.715419+0000","last_change":"2024-09-20T18:54:53.241889+0000","last_active":"2024-09-20T18:57:20.715419+0000","last_peered":"2024-09-20T18:57:20.715419+0000","last_clean":"2024-09-20T18:57:20.715419+0000","last_became_active":"2024-09-20T18:49:47.947448+0000","last_became_peered":"2024-09-20T18:49:47.947448+0000","last_unstale":"2024-09-20T18:57:20.715419+0000","last_undegraded":"2024-09-20T18:57:20.715419+0000","last_fullsized":"2024-09-20T18:57:20.715419+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:53.241802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:53.241802+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":355,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.761132+0000","last_change":"2024-09-20T18:57:08.539466+0000","last_active":"2024-09-20T18:57:20.761132+0000","last_peered":"2024-09-20T18:57:20.761132+0000","last_clean":"2024-09-20T18:57:20.761132+0000","last_became_active":"2024-09-20T18:49:49.961188+0000","last_became_peered":"2024-09-20T18:49:49.961188+0000","last_unstale":"2024-09-20T18:57:20.761132+0000","last_undegraded":"2024-09-20T18:57:20.761132+0000","last_fullsized":"2024-09-20T18:57:20.761132+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:08.539418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:08.539418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:13.102215+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":327,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.715168+0000","last_change":"2024-09-20T18:49:51.966957+0000","last_active":"2024-09-20T18:57:20.715168+0000","last_peered":"2024-09-20T18:57:20.715168+0000","last_clean":"2024-09-20T18:57:20.715168+0000","last_became_active":"2024-09-20T18:49:51.966715+0000","last_became_peered":"2024-09-20T18:49:51.966715+0000","last_unstale":"2024-09-20T18:57:20.715168+0000","last_undegraded":"2024-09-20T18:57:20.715168+0000","last_fullsized":"2024-09-20T18:57:20.715168+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":355,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.761147+0000","last_change":"2024-09-20T18:55:15.252839+0000","last_active":"2024-09-20T18:57:20.761147+0000","last_peered":"2024-09-20T18:57:20.761147+0000","last_clean":"2024-09-20T18:57:20.761147+0000","last_became_active":"2024-09-20T18:49:45.944703+0000","last_became_peered":"2024-09-20T18:49:45.944703+0000","last_unstale":"2024-09-20T18:57:20.761147+0000","last_undegraded":"2024-09-20T18:57:20.761147+0000","last_fullsized":"2024-09-20T18:57:20.761147+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:15.252782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:15.252782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":380,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.714697+0000","last_change":"2024-09-20T18:51:26.494194+0000","last_active":"2024-09-20T18:57:20.714697+0000","last_peered":"2024-09-20T18:57:20.714697+0000","last_clean":"2024-09-20T18:57:20.714697+0000","last_became_active":"2024-09-20T18:51:23.813399+0000","last_became_peered":"2024-09-20T18:51:23.813399+0000","last_unstale":"2024-09-20T18:57:20.714697+0000","last_undegraded":"2024-09-20T18:57:20.714697+0000","last_fullsized":"2024-09-20T18:57:20.714697+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-20T18:51:26.494118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:26.494118+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":347,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.717055+0000","last_change":"2024-09-20T18:52:38.799542+0000","last_active":"2024-09-20T18:57:20.717055+0000","last_peered":"2024-09-20T18:57:20.717055+0000","last_clean":"2024-09-20T18:57:20.717055+0000","last_became_active":"2024-09-20T18:49:49.959696+0000","last_became_peered":"2024-09-20T18:49:49.959696+0000","last_unstale":"2024-09-20T18:57:20.717055+0000","last_undegraded":"2024-09-20T18:57:20.717055+0000","last_fullsized":"2024-09-20T18:57:20.717055+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:38.799487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:38.799487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":351,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.760680+0000","last_change":"2024-09-20T18:57:13.587827+0000","last_active":"2024-09-20T18:57:20.760680+0000","last_peered":"2024-09-20T18:57:20.760680+0000","last_clean":"2024-09-20T18:57:20.760680+0000","last_became_active":"2024-09-20T18:49:51.970663+0000","last_became_peered":"2024-09-20T18:49:51.970663+0000","last_unstale":"2024-09-20T18:57:20.760680+0000","last_undegraded":"2024-09-20T18:57:20.760680+0000","last_fullsized":"2024-09-20T18:57:20.760680+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:13.587774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:13.587774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:15.272228+0000","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":351,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.715187+0000","last_change":"2024-09-20T18:55:28.828417+0000","last_active":"2024-09-20T18:57:20.715187+0000","last_peered":"2024-09-20T18:57:20.715187+0000","last_clean":"2024-09-20T18:57:20.715187+0000","last_became_active":"2024-09-20T18:55:28.827435+0000","last_became_peered":"2024-09-20T18:55:28.827435+0000","last_unstale":"2024-09-20T18:57:20.715187+0000","last_undegraded":"2024-09-20T18:57:20.715187+0000","last_fullsized":"2024-09-20T18:57:20.715187+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:53.940191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:53.940191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":355,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.719021+0000","last_change":"2024-09-20T18:54:23.984024+0000","last_active":"2024-09-20T18:57:20.719021+0000","last_peered":"2024-09-20T18:57:20.719021+0000","last_clean":"2024-09-20T18:57:20.719021+0000","last_became_active":"2024-09-20T18:49:45.950480+0000","last_became_peered":"2024-09-20T18:49:45.950480+0000","last_unstale":"2024-09-20T18:57:20.719021+0000","last_undegraded":"2024-09-20T18:57:20.719021+0000","last_fullsized":"2024-09-20T18:57:20.719021+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:23.983958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:23.983958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":378,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.716444+0000","last_change":"2024-09-20T18:53:26.762207+0000","last_active":"2024-09-20T18:57:20.716444+0000","last_peered":"2024-09-20T18:57:20.716444+0000","last_clean":"2024-09-20T18:57:20.716444+0000","last_became_active":"2024-09-20T18:49:47.950939+0000","last_became_peered":"2024-09-20T18:49:47.950939+0000","last_unstale":"2024-09-20T18:57:20.716444+0000","last_undegraded":"2024-09-20T18:57:20.716444+0000","last_fullsized":"2024-09-20T18:57:20.716444+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:53:26.762131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:26.762131+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":348,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.758619+0000","last_change":"2024-09-20T18:57:14.941518+0000","last_active":"2024-09-20T18:57:20.758619+0000","last_peered":"2024-09-20T18:57:20.758619+0000","last_clean":"2024-09-20T18:57:20.758619+0000","last_became_active":"2024-09-20T18:49:49.958518+0000","last_became_peered":"2024-09-20T18:49:49.958518+0000","last_unstale":"2024-09-20T18:57:20.758619+0000","last_undegraded":"2024-09-20T18:57:20.758619+0000","last_fullsized":"2024-09-20T18:57:20.758619+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:14.941458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:14.941458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:37.988570+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":351,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.759596+0000","last_change":"2024-09-20T18:55:28.833468+0000","last_active":"2024-09-20T18:57:20.759596+0000","last_peered":"2024-09-20T18:57:20.759596+0000","last_clean":"2024-09-20T18:57:20.759596+0000","last_became_active":"2024-09-20T18:55:28.833133+0000","last_became_peered":"2024-09-20T18:55:28.833133+0000","last_unstale":"2024-09-20T18:57:20.759596+0000","last_undegraded":"2024-09-20T18:57:20.759596+0000","last_fullsized":"2024-09-20T18:57:20.759596+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:54.259174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:54.259174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"59'1","reported_seq":362,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.714787+0000","last_change":"2024-09-20T18:53:42.840884+0000","last_active":"2024-09-20T18:57:20.714787+0000","last_peered":"2024-09-20T18:57:20.714787+0000","last_clean":"2024-09-20T18:57:20.714787+0000","last_became_active":"2024-09-20T18:49:45.940100+0000","last_became_peered":"2024-09-20T18:49:45.940100+0000","last_unstale":"2024-09-20T18:57:20.714787+0000","last_undegraded":"2024-09-20T18:57:20.714787+0000","last_fullsized":"2024-09-20T18:57:20.714787+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:53:42.840800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:42.840800+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":367,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.758886+0000","last_change":"2024-09-20T18:57:09.819080+0000","last_active":"2024-09-20T18:57:20.758886+0000","last_peered":"2024-09-20T18:57:20.758886+0000","last_clean":"2024-09-20T18:57:20.758886+0000","last_became_active":"2024-09-20T18:49:47.950909+0000","last_became_peered":"2024-09-20T18:49:47.950909+0000","last_unstale":"2024-09-20T18:57:20.758886+0000","last_undegraded":"2024-09-20T18:57:20.758886+0000","last_fullsized":"2024-09-20T18:57:20.758886+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:57:09.818999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:09.818999+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:17.216750+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":356,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.714912+0000","last_change":"2024-09-20T18:54:44.239273+0000","last_active":"2024-09-20T18:57:20.714912+0000","last_peered":"2024-09-20T18:57:20.714912+0000","last_clean":"2024-09-20T18:57:20.714912+0000","last_became_active":"2024-09-20T18:49:49.959102+0000","last_became_peered":"2024-09-20T18:49:49.959102+0000","last_unstale":"2024-09-20T18:57:20.714912+0000","last_undegraded":"2024-09-20T18:57:20.714912+0000","last_fullsized":"2024-09-20T18:57:20.714912+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:44.239226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.239226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":343,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.717196+0000","last_change":"2024-09-20T18:52:15.826819+0000","last_active":"2024-09-20T18:57:20.717196+0000","last_peered":"2024-09-20T18:57:20.717196+0000","last_clean":"2024-09-20T18:57:20.717196+0000","last_became_active":"2024-09-20T18:49:51.970058+0000","last_became_peered":"2024-09-20T18:49:51.970058+0000","last_unstale":"2024-09-20T18:57:20.717196+0000","last_undegraded":"2024-09-20T18:57:20.717196+0000","last_fullsized":"2024-09-20T18:57:20.717196+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T18:52:15.826638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:15.826638+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":357,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.760396+0000","last_change":"2024-09-20T18:57:18.032140+0000","last_active":"2024-09-20T18:57:20.760396+0000","last_peered":"2024-09-20T18:57:20.760396+0000","last_clean":"2024-09-20T18:57:20.760396+0000","last_became_active":"2024-09-20T18:49:45.933078+0000","last_became_peered":"2024-09-20T18:49:45.933078+0000","last_unstale":"2024-09-20T18:57:20.760396+0000","last_undegraded":"2024-09-20T18:57:20.760396+0000","last_fullsized":"2024-09-20T18:57:20.760396+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:57:18.032067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:18.032067+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:42.727785+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":374,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.719758+0000","last_change":"2024-09-20T18:55:19.996522+0000","last_active":"2024-09-20T18:57:20.719758+0000","last_peered":"2024-09-20T18:57:20.719758+0000","last_clean":"2024-09-20T18:57:20.719758+0000","last_became_active":"2024-09-20T18:49:48.500809+0000","last_became_peered":"2024-09-20T18:49:48.500809+0000","last_unstale":"2024-09-20T18:57:20.719758+0000","last_undegraded":"2024-09-20T18:57:20.719758+0000","last_fullsized":"2024-09-20T18:57:20.719758+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:55:19.996328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:19.996328+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"184'9","reported_seq":148,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.715204+0000","last_change":"2024-09-20T18:55:28.845188+0000","last_active":"2024-09-20T18:57:20.715204+0000","last_peered":"2024-09-20T18:57:20.715204+0000","last_clean":"2024-09-20T18:57:20.715204+0000","last_became_active":"2024-09-20T18:55:28.820941+0000","last_became_peered":"2024-09-20T18:55:28.820941+0000","last_unstale":"2024-09-20T18:57:20.715204+0000","last_undegraded":"2024-09-20T18:57:20.715204+0000","last_fullsized":"2024-09-20T18:57:20.715204+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:53:44.524040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:44.524040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3","version":"222'10857","reported_seq":23325,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:21.226874+0000","last_change":"2024-09-20T18:56:25.697520+0000","last_active":"2024-09-20T18:57:21.226874+0000","last_peered":"2024-09-20T18:57:21.226874+0000","last_clean":"2024-09-20T18:57:21.226874+0000","last_became_active":"2024-09-20T18:56:25.687391+0000","last_became_peered":"2024-09-20T18:56:25.687391+0000","last_unstale":"2024-09-20T18:57:21.226874+0000","last_undegraded":"2024-09-20T18:57:21.226874+0000","last_fullsized":"2024-09-20T18:57:21.226874+0000","mapping_epoch":203,"log_start":"190'6572","ondisk_log_start":"190'6572","created":160,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4285,"log_dups_size":0,"ondisk_log_size":4285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":225378304,"num_objects":3439,"num_object_clones":0,"num_object_copies":10317,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3439,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10857,"num_write_kb":457472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6156,"num_bytes_recovered":403439616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":369,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.716612+0000","last_change":"2024-09-20T18:55:18.581290+0000","last_active":"2024-09-20T18:57:20.716612+0000","last_peered":"2024-09-20T18:57:20.716612+0000","last_clean":"2024-09-20T18:57:20.716612+0000","last_became_active":"2024-09-20T18:49:58.992302+0000","last_became_peered":"2024-09-20T18:49:58.992302+0000","last_unstale":"2024-09-20T18:57:20.716612+0000","last_undegraded":"2024-09-20T18:57:20.716612+0000","last_fullsized":"2024-09-20T18:57:20.716612+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:55:18.581200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:18.581200+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":356,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.757882+0000","last_change":"2024-09-20T18:54:54.288199+0000","last_active":"2024-09-20T18:57:20.757882+0000","last_peered":"2024-09-20T18:57:20.757882+0000","last_clean":"2024-09-20T18:57:20.757882+0000","last_became_active":"2024-09-20T18:49:45.936104+0000","last_became_peered":"2024-09-20T18:49:45.936104+0000","last_unstale":"2024-09-20T18:57:20.757882+0000","last_undegraded":"2024-09-20T18:57:20.757882+0000","last_fullsized":"2024-09-20T18:57:20.757882+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:54.288117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:54.288117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":46,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":450,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.715313+0000","last_change":"2024-09-20T18:54:14.292660+0000","last_active":"2024-09-20T18:57:20.715313+0000","last_peered":"2024-09-20T18:57:20.715313+0000","last_clean":"2024-09-20T18:57:20.715313+0000","last_became_active":"2024-09-20T18:49:49.960697+0000","last_became_peered":"2024-09-20T18:49:49.960697+0000","last_unstale":"2024-09-20T18:57:20.715313+0000","last_undegraded":"2024-09-20T18:57:20.715313+0000","last_fullsized":"2024-09-20T18:57:20.715313+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:54:14.292610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:14.292610+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":34,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":343,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.761406+0000","last_change":"2024-09-20T18:54:10.381171+0000","last_active":"2024-09-20T18:57:20.761406+0000","last_peered":"2024-09-20T18:57:20.761406+0000","last_clean":"2024-09-20T18:57:20.761406+0000","last_became_active":"2024-09-20T18:49:51.971399+0000","last_became_peered":"2024-09-20T18:49:51.971399+0000","last_unstale":"2024-09-20T18:57:20.761406+0000","last_undegraded":"2024-09-20T18:57:20.761406+0000","last_fullsized":"2024-09-20T18:57:20.761406+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:10.381108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:10.381108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.c","version":"222'10976","reported_seq":11112,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:21.781967+0000","last_change":"2024-09-20T18:54:34.311256+0000","last_active":"2024-09-20T18:57:21.781967+0000","last_peered":"2024-09-20T18:57:21.781967+0000","last_clean":"2024-09-20T18:57:21.781967+0000","last_became_active":"2024-09-20T18:54:34.310904+0000","last_became_peered":"2024-09-20T18:54:34.310904+0000","last_unstale":"2024-09-20T18:57:21.781967+0000","last_undegraded":"2024-09-20T18:57:21.781967+0000","last_fullsized":"2024-09-20T18:57:21.781967+0000","mapping_epoch":160,"log_start":"189'6111","ondisk_log_start":"189'6111","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4865,"log_dups_size":0,"ondisk_log_size":4865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":223870976,"num_objects":3416,"num_object_clones":0,"num_object_copies":10248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3416,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10976,"num_write_kb":460544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":384,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.757785+0000","last_change":"2024-09-20T18:54:22.077299+0000","last_active":"2024-09-20T18:57:20.757785+0000","last_peered":"2024-09-20T18:57:20.757785+0000","last_clean":"2024-09-20T18:57:20.757785+0000","last_became_active":"2024-09-20T18:49:48.499189+0000","last_became_peered":"2024-09-20T18:49:48.499189+0000","last_unstale":"2024-09-20T18:57:20.757785+0000","last_undegraded":"2024-09-20T18:57:20.757785+0000","last_fullsized":"2024-09-20T18:57:20.757785+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-20T18:54:22.077212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:22.077212+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":348,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.760042+0000","last_change":"2024-09-20T18:54:57.237740+0000","last_active":"2024-09-20T18:57:20.760042+0000","last_peered":"2024-09-20T18:57:20.760042+0000","last_clean":"2024-09-20T18:57:20.760042+0000","last_became_active":"2024-09-20T18:49:45.940544+0000","last_became_peered":"2024-09-20T18:49:45.940544+0000","last_unstale":"2024-09-20T18:57:20.760042+0000","last_undegraded":"2024-09-20T18:57:20.760042+0000","last_fullsized":"2024-09-20T18:57:20.760042+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.237675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.237675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":356,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.714315+0000","last_change":"2024-09-20T18:54:51.229251+0000","last_active":"2024-09-20T18:57:20.714315+0000","last_peered":"2024-09-20T18:57:20.714315+0000","last_clean":"2024-09-20T18:57:20.714315+0000","last_became_active":"2024-09-20T18:49:49.963073+0000","last_became_peered":"2024-09-20T18:49:49.963073+0000","last_unstale":"2024-09-20T18:57:20.714315+0000","last_undegraded":"2024-09-20T18:57:20.714315+0000","last_fullsized":"2024-09-20T18:57:20.714315+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.229201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.229201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":352,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.714691+0000","last_change":"2024-09-20T18:54:55.323937+0000","last_active":"2024-09-20T18:57:20.714691+0000","last_peered":"2024-09-20T18:57:20.714691+0000","last_clean":"2024-09-20T18:57:20.714691+0000","last_became_active":"2024-09-20T18:49:51.968793+0000","last_became_peered":"2024-09-20T18:49:51.968793+0000","last_unstale":"2024-09-20T18:57:20.714691+0000","last_undegraded":"2024-09-20T18:57:20.714691+0000","last_fullsized":"2024-09-20T18:57:20.714691+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:55.323692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:55.323692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"222'10919","reported_seq":11056,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.928853+0000","last_change":"2024-09-20T18:54:34.311680+0000","last_active":"2024-09-20T18:57:20.928853+0000","last_peered":"2024-09-20T18:57:20.928853+0000","last_clean":"2024-09-20T18:57:20.928853+0000","last_became_active":"2024-09-20T18:54:34.311241+0000","last_became_peered":"2024-09-20T18:54:34.311241+0000","last_unstale":"2024-09-20T18:57:20.928853+0000","last_undegraded":"2024-09-20T18:57:20.928853+0000","last_fullsized":"2024-09-20T18:57:20.928853+0000","mapping_epoch":160,"log_start":"188'5846","ondisk_log_start":"188'5846","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":5073,"log_dups_size":0,"ondisk_log_size":5073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":226230296,"num_objects":3453,"num_object_clones":0,"num_object_copies":10359,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3453,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10919,"num_write_kb":459841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":412,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.714323+0000","last_change":"2024-09-20T18:55:28.853375+0000","last_active":"2024-09-20T18:57:20.714323+0000","last_peered":"2024-09-20T18:57:20.714323+0000","last_clean":"2024-09-20T18:57:20.714323+0000","last_became_active":"2024-09-20T18:55:28.831057+0000","last_became_peered":"2024-09-20T18:55:28.831057+0000","last_unstale":"2024-09-20T18:57:20.714323+0000","last_undegraded":"2024-09-20T18:57:20.714323+0000","last_fullsized":"2024-09-20T18:57:20.714323+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-20T18:54:57.394893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.394893+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":348,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.720071+0000","last_change":"2024-09-20T18:51:27.187060+0000","last_active":"2024-09-20T18:57:20.720071+0000","last_peered":"2024-09-20T18:57:20.720071+0000","last_clean":"2024-09-20T18:57:20.720071+0000","last_became_active":"2024-09-20T18:49:45.940469+0000","last_became_peered":"2024-09-20T18:49:45.940469+0000","last_unstale":"2024-09-20T18:57:20.720071+0000","last_undegraded":"2024-09-20T18:57:20.720071+0000","last_fullsized":"2024-09-20T18:57:20.720071+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:51:27.186991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:27.186991+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"66'8","reported_seq":456,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.757658+0000","last_change":"2024-09-20T18:54:47.223010+0000","last_active":"2024-09-20T18:57:20.757658+0000","last_peered":"2024-09-20T18:57:20.757658+0000","last_clean":"2024-09-20T18:57:20.757658+0000","last_became_active":"2024-09-20T18:49:49.956133+0000","last_became_peered":"2024-09-20T18:49:49.956133+0000","last_unstale":"2024-09-20T18:57:20.757658+0000","last_undegraded":"2024-09-20T18:57:20.757658+0000","last_fullsized":"2024-09-20T18:57:20.757658+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:54:47.222937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:47.222937+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":350,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.717243+0000","last_change":"2024-09-20T18:55:28.819147+0000","last_active":"2024-09-20T18:57:20.717243+0000","last_peered":"2024-09-20T18:57:20.717243+0000","last_clean":"2024-09-20T18:57:20.717243+0000","last_became_active":"2024-09-20T18:55:28.818804+0000","last_became_peered":"2024-09-20T18:55:28.818804+0000","last_unstale":"2024-09-20T18:57:20.717243+0000","last_undegraded":"2024-09-20T18:57:20.717243+0000","last_fullsized":"2024-09-20T18:57:20.717243+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:42.880724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:42.880724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"222'10994","reported_seq":11129,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:21.788953+0000","last_change":"2024-09-20T18:54:34.309939+0000","last_active":"2024-09-20T18:57:21.788953+0000","last_peered":"2024-09-20T18:57:21.788953+0000","last_clean":"2024-09-20T18:57:21.788953+0000","last_became_active":"2024-09-20T18:54:34.309630+0000","last_became_peered":"2024-09-20T18:54:34.309630+0000","last_unstale":"2024-09-20T18:57:21.788953+0000","last_undegraded":"2024-09-20T18:57:21.788953+0000","last_fullsized":"2024-09-20T18:57:21.788953+0000","mapping_epoch":160,"log_start":"189'6149","ondisk_log_start":"189'6149","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4845,"log_dups_size":0,"ondisk_log_size":4845,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":223346688,"num_objects":3408,"num_object_clones":0,"num_object_copies":10224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10994,"num_write_kb":460864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":375,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.759992+0000","last_change":"2024-09-20T18:55:19.541011+0000","last_active":"2024-09-20T18:57:20.759992+0000","last_peered":"2024-09-20T18:57:20.759992+0000","last_clean":"2024-09-20T18:57:20.759992+0000","last_became_active":"2024-09-20T18:49:47.949626+0000","last_became_peered":"2024-09-20T18:49:47.949626+0000","last_unstale":"2024-09-20T18:57:20.759992+0000","last_undegraded":"2024-09-20T18:57:20.759992+0000","last_fullsized":"2024-09-20T18:57:20.759992+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:55:19.540812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:19.540812+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":363,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.761874+0000","last_change":"2024-09-20T18:54:18.326947+0000","last_active":"2024-09-20T18:57:20.761874+0000","last_peered":"2024-09-20T18:57:20.761874+0000","last_clean":"2024-09-20T18:57:20.761874+0000","last_became_active":"2024-09-20T18:49:45.942496+0000","last_became_peered":"2024-09-20T18:49:45.942496+0000","last_unstale":"2024-09-20T18:57:20.761874+0000","last_undegraded":"2024-09-20T18:57:20.761874+0000","last_fullsized":"2024-09-20T18:57:20.761874+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:18.326875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:18.326875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"184'9","reported_seq":458,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.718793+0000","last_change":"2024-09-20T18:55:29.010062+0000","last_active":"2024-09-20T18:57:20.718793+0000","last_peered":"2024-09-20T18:57:20.718793+0000","last_clean":"2024-09-20T18:57:20.718793+0000","last_became_active":"2024-09-20T18:55:28.830643+0000","last_became_peered":"2024-09-20T18:55:28.830643+0000","last_unstale":"2024-09-20T18:57:20.718793+0000","last_undegraded":"2024-09-20T18:57:20.718793+0000","last_fullsized":"2024-09-20T18:57:20.718793+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:29.010015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:29.010015+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":327,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.715319+0000","last_change":"2024-09-20T18:49:51.965733+0000","last_active":"2024-09-20T18:57:20.715319+0000","last_peered":"2024-09-20T18:57:20.715319+0000","last_clean":"2024-09-20T18:57:20.715319+0000","last_became_active":"2024-09-20T18:49:51.964780+0000","last_became_peered":"2024-09-20T18:49:51.964780+0000","last_unstale":"2024-09-20T18:57:20.715319+0000","last_undegraded":"2024-09-20T18:57:20.715319+0000","last_fullsized":"2024-09-20T18:57:20.715319+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"222'11257","reported_seq":11393,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:21.493676+0000","last_change":"2024-09-20T18:54:34.309809+0000","last_active":"2024-09-20T18:57:21.493676+0000","last_peered":"2024-09-20T18:57:21.493676+0000","last_clean":"2024-09-20T18:57:21.493676+0000","last_became_active":"2024-09-20T18:54:34.309231+0000","last_became_peered":"2024-09-20T18:54:34.309231+0000","last_unstale":"2024-09-20T18:57:21.493676+0000","last_undegraded":"2024-09-20T18:57:21.493676+0000","last_fullsized":"2024-09-20T18:57:21.493676+0000","mapping_epoch":160,"log_start":"192'6932","ondisk_log_start":"192'6932","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4325,"log_dups_size":0,"ondisk_log_size":4325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":226557952,"num_objects":3457,"num_object_clones":0,"num_object_copies":10371,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3457,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11257,"num_write_kb":470848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":387,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.718844+0000","last_change":"2024-09-20T18:54:14.923895+0000","last_active":"2024-09-20T18:57:20.718844+0000","last_peered":"2024-09-20T18:57:20.718844+0000","last_clean":"2024-09-20T18:57:20.718844+0000","last_became_active":"2024-09-20T18:49:48.497736+0000","last_became_peered":"2024-09-20T18:49:48.497736+0000","last_unstale":"2024-09-20T18:57:20.718844+0000","last_undegraded":"2024-09-20T18:57:20.718844+0000","last_fullsized":"2024-09-20T18:57:20.718844+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-20T18:54:14.923682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:14.923682+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":348,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.759670+0000","last_change":"2024-09-20T18:57:09.940634+0000","last_active":"2024-09-20T18:57:20.759670+0000","last_peered":"2024-09-20T18:57:20.759670+0000","last_clean":"2024-09-20T18:57:20.759670+0000","last_became_active":"2024-09-20T18:49:45.942253+0000","last_became_peered":"2024-09-20T18:49:45.942253+0000","last_unstale":"2024-09-20T18:57:20.759670+0000","last_undegraded":"2024-09-20T18:57:20.759670+0000","last_fullsized":"2024-09-20T18:57:20.759670+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:09.940517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:09.940517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:18.833936+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":93,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.718476+0000","last_change":"2024-09-20T18:55:28.830954+0000","last_active":"2024-09-20T18:57:20.718476+0000","last_peered":"2024-09-20T18:57:20.718476+0000","last_clean":"2024-09-20T18:57:20.718476+0000","last_became_active":"2024-09-20T18:55:28.830645+0000","last_became_peered":"2024-09-20T18:55:28.830645+0000","last_unstale":"2024-09-20T18:57:20.718476+0000","last_undegraded":"2024-09-20T18:57:20.718476+0000","last_fullsized":"2024-09-20T18:57:20.718476+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:50.610871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:50.610871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":359,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.760190+0000","last_change":"2024-09-20T18:55:28.822362+0000","last_active":"2024-09-20T18:57:20.760190+0000","last_peered":"2024-09-20T18:57:20.760190+0000","last_clean":"2024-09-20T18:57:20.760190+0000","last_became_active":"2024-09-20T18:55:28.821588+0000","last_became_peered":"2024-09-20T18:55:28.821588+0000","last_unstale":"2024-09-20T18:57:20.760190+0000","last_undegraded":"2024-09-20T18:57:20.760190+0000","last_fullsized":"2024-09-20T18:57:20.760190+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.402841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.402841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"222'11113","reported_seq":11249,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.893873+0000","last_change":"2024-09-20T18:54:34.311561+0000","last_active":"2024-09-20T18:57:20.893873+0000","last_peered":"2024-09-20T18:57:20.893873+0000","last_clean":"2024-09-20T18:57:20.893873+0000","last_became_active":"2024-09-20T18:54:34.311181+0000","last_became_peered":"2024-09-20T18:54:34.311181+0000","last_unstale":"2024-09-20T18:57:20.893873+0000","last_undegraded":"2024-09-20T18:57:20.893873+0000","last_fullsized":"2024-09-20T18:57:20.893873+0000","mapping_epoch":160,"log_start":"189'6094","ondisk_log_start":"189'6094","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":5019,"log_dups_size":0,"ondisk_log_size":5019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":228392960,"num_objects":3485,"num_object_clones":0,"num_object_copies":10455,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11113,"num_write_kb":467136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"66'28","reported_seq":492,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.759583+0000","last_change":"2024-09-20T18:55:16.527267+0000","last_active":"2024-09-20T18:57:20.759583+0000","last_peered":"2024-09-20T18:57:20.759583+0000","last_clean":"2024-09-20T18:57:20.759583+0000","last_became_active":"2024-09-20T18:49:47.955335+0000","last_became_peered":"2024-09-20T18:49:47.955335+0000","last_unstale":"2024-09-20T18:57:20.759583+0000","last_undegraded":"2024-09-20T18:57:20.759583+0000","last_fullsized":"2024-09-20T18:57:20.759583+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'28","last_scrub_stamp":"2024-09-20T18:55:16.527165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:16.527165+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":347,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.715208+0000","last_change":"2024-09-20T18:55:28.832303+0000","last_active":"2024-09-20T18:57:20.715208+0000","last_peered":"2024-09-20T18:57:20.715208+0000","last_clean":"2024-09-20T18:57:20.715208+0000","last_became_active":"2024-09-20T18:55:28.831574+0000","last_became_peered":"2024-09-20T18:55:28.831574+0000","last_unstale":"2024-09-20T18:57:20.715208+0000","last_undegraded":"2024-09-20T18:57:20.715208+0000","last_fullsized":"2024-09-20T18:57:20.715208+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:49:44.925938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1],"acting":[4,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":392,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.762058+0000","last_change":"2024-09-20T18:55:28.854615+0000","last_active":"2024-09-20T18:57:20.762058+0000","last_peered":"2024-09-20T18:57:20.762058+0000","last_clean":"2024-09-20T18:57:20.762058+0000","last_became_active":"2024-09-20T18:55:28.837226+0000","last_became_peered":"2024-09-20T18:55:28.837226+0000","last_unstale":"2024-09-20T18:57:20.762058+0000","last_undegraded":"2024-09-20T18:57:20.762058+0000","last_fullsized":"2024-09-20T18:57:20.762058+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-20T18:54:51.227531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.227531+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0012781159999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":355,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.719267+0000","last_change":"2024-09-20T18:55:13.949348+0000","last_active":"2024-09-20T18:57:20.719267+0000","last_peered":"2024-09-20T18:57:20.719267+0000","last_clean":"2024-09-20T18:57:20.719267+0000","last_became_active":"2024-09-20T18:49:49.953971+0000","last_became_peered":"2024-09-20T18:49:49.953971+0000","last_unstale":"2024-09-20T18:57:20.719267+0000","last_undegraded":"2024-09-20T18:57:20.719267+0000","last_fullsized":"2024-09-20T18:57:20.719267+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:13.949269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:13.949269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":336,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.759206+0000","last_change":"2024-09-20T18:57:12.744116+0000","last_active":"2024-09-20T18:57:20.759206+0000","last_peered":"2024-09-20T18:57:20.759206+0000","last_clean":"2024-09-20T18:57:20.759206+0000","last_became_active":"2024-09-20T18:49:51.969400+0000","last_became_peered":"2024-09-20T18:49:51.969400+0000","last_unstale":"2024-09-20T18:57:20.759206+0000","last_undegraded":"2024-09-20T18:57:20.759206+0000","last_fullsized":"2024-09-20T18:57:20.759206+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:12.744040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:12.744040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:17.964285+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"222'11159","reported_seq":11295,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:21.509877+0000","last_change":"2024-09-20T18:54:33.926785+0000","last_active":"2024-09-20T18:57:21.509877+0000","last_peered":"2024-09-20T18:57:21.509877+0000","last_clean":"2024-09-20T18:57:21.509877+0000","last_became_active":"2024-09-20T18:54:33.925750+0000","last_became_peered":"2024-09-20T18:54:33.925750+0000","last_unstale":"2024-09-20T18:57:21.509877+0000","last_undegraded":"2024-09-20T18:57:21.509877+0000","last_fullsized":"2024-09-20T18:57:21.509877+0000","mapping_epoch":160,"log_start":"191'6840","ondisk_log_start":"191'6840","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4319,"log_dups_size":0,"ondisk_log_size":4319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":233242624,"num_objects":3559,"num_object_clones":0,"num_object_copies":10677,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3559,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11159,"num_write_kb":470976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":387,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.761416+0000","last_change":"2024-09-20T18:55:25.702762+0000","last_active":"2024-09-20T18:57:20.761416+0000","last_peered":"2024-09-20T18:57:20.761416+0000","last_clean":"2024-09-20T18:57:20.761416+0000","last_became_active":"2024-09-20T18:49:47.950904+0000","last_became_peered":"2024-09-20T18:49:47.950904+0000","last_unstale":"2024-09-20T18:57:20.761416+0000","last_undegraded":"2024-09-20T18:57:20.761416+0000","last_fullsized":"2024-09-20T18:57:20.761416+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-20T18:55:25.702596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:25.702596+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":371,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.758626+0000","last_change":"2024-09-20T18:55:28.830832+0000","last_active":"2024-09-20T18:57:20.758626+0000","last_peered":"2024-09-20T18:57:20.758626+0000","last_clean":"2024-09-20T18:57:20.758626+0000","last_became_active":"2024-09-20T18:55:28.830269+0000","last_became_peered":"2024-09-20T18:55:28.830269+0000","last_unstale":"2024-09-20T18:57:20.758626+0000","last_undegraded":"2024-09-20T18:57:20.758626+0000","last_fullsized":"2024-09-20T18:57:20.758626+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:23.359769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:23.359769+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":358,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.716087+0000","last_change":"2024-09-20T18:52:07.718766+0000","last_active":"2024-09-20T18:57:20.716087+0000","last_peered":"2024-09-20T18:57:20.716087+0000","last_clean":"2024-09-20T18:57:20.716087+0000","last_became_active":"2024-09-20T18:49:44.937135+0000","last_became_peered":"2024-09-20T18:49:44.937135+0000","last_unstale":"2024-09-20T18:57:20.716087+0000","last_undegraded":"2024-09-20T18:57:20.716087+0000","last_fullsized":"2024-09-20T18:57:20.716087+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:52:07.718646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:52:07.718646+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.00095558800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":341,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.717099+0000","last_change":"2024-09-20T18:51:59.796450+0000","last_active":"2024-09-20T18:57:20.717099+0000","last_peered":"2024-09-20T18:57:20.717099+0000","last_clean":"2024-09-20T18:57:20.717099+0000","last_became_active":"2024-09-20T18:49:49.954424+0000","last_became_peered":"2024-09-20T18:49:49.954424+0000","last_unstale":"2024-09-20T18:57:20.717099+0000","last_undegraded":"2024-09-20T18:57:20.717099+0000","last_fullsized":"2024-09-20T18:57:20.717099+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:59.796383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:59.796383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":342,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.717656+0000","last_change":"2024-09-20T18:55:28.826659+0000","last_active":"2024-09-20T18:57:20.717656+0000","last_peered":"2024-09-20T18:57:20.717656+0000","last_clean":"2024-09-20T18:57:20.717656+0000","last_became_active":"2024-09-20T18:55:28.825812+0000","last_became_peered":"2024-09-20T18:55:28.825812+0000","last_unstale":"2024-09-20T18:57:20.717656+0000","last_undegraded":"2024-09-20T18:57:20.717656+0000","last_fullsized":"2024-09-20T18:57:20.717656+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:39.814259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:39.814259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"222'11179","reported_seq":11315,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:21.252774+0000","last_change":"2024-09-20T18:54:34.311774+0000","last_active":"2024-09-20T18:57:21.252774+0000","last_peered":"2024-09-20T18:57:21.252774+0000","last_clean":"2024-09-20T18:57:21.252774+0000","last_became_active":"2024-09-20T18:54:34.311487+0000","last_became_peered":"2024-09-20T18:54:34.311487+0000","last_unstale":"2024-09-20T18:57:21.252774+0000","last_undegraded":"2024-09-20T18:57:21.252774+0000","last_fullsized":"2024-09-20T18:57:21.252774+0000","mapping_epoch":160,"log_start":"191'6892","ondisk_log_start":"191'6892","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4287,"log_dups_size":0,"ondisk_log_size":4287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":225378304,"num_objects":3439,"num_object_clones":0,"num_object_copies":10317,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3439,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11179,"num_write_kb":467776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":385,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.719874+0000","last_change":"2024-09-20T18:55:14.951950+0000","last_active":"2024-09-20T18:57:20.719874+0000","last_peered":"2024-09-20T18:57:20.719874+0000","last_clean":"2024-09-20T18:57:20.719874+0000","last_became_active":"2024-09-20T18:49:47.949230+0000","last_became_peered":"2024-09-20T18:49:47.949230+0000","last_unstale":"2024-09-20T18:57:20.719874+0000","last_undegraded":"2024-09-20T18:57:20.719874+0000","last_fullsized":"2024-09-20T18:57:20.719874+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:55:14.951892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:14.951892+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":363,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.718061+0000","last_change":"2024-09-20T18:54:27.905856+0000","last_active":"2024-09-20T18:57:20.718061+0000","last_peered":"2024-09-20T18:57:20.718061+0000","last_clean":"2024-09-20T18:57:20.718061+0000","last_became_active":"2024-09-20T18:49:45.939197+0000","last_became_peered":"2024-09-20T18:49:45.939197+0000","last_unstale":"2024-09-20T18:57:20.718061+0000","last_undegraded":"2024-09-20T18:57:20.718061+0000","last_fullsized":"2024-09-20T18:57:20.718061+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:27.905803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:27.905803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":824,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.717284+0000","last_change":"2024-09-20T18:54:26.359887+0000","last_active":"2024-09-20T18:57:20.717284+0000","last_peered":"2024-09-20T18:57:20.717284+0000","last_clean":"2024-09-20T18:57:20.717284+0000","last_became_active":"2024-09-20T18:49:44.936819+0000","last_became_peered":"2024-09-20T18:49:44.936819+0000","last_unstale":"2024-09-20T18:57:20.717284+0000","last_undegraded":"2024-09-20T18:57:20.717284+0000","last_fullsized":"2024-09-20T18:57:20.717284+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-20T18:54:26.359807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:54:26.359807+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0010996199999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":450,"num_read_kb":445,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":347,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.761816+0000","last_change":"2024-09-20T18:54:59.151228+0000","last_active":"2024-09-20T18:57:20.761816+0000","last_peered":"2024-09-20T18:57:20.761816+0000","last_clean":"2024-09-20T18:57:20.761816+0000","last_became_active":"2024-09-20T18:49:49.961066+0000","last_became_peered":"2024-09-20T18:49:49.961066+0000","last_unstale":"2024-09-20T18:57:20.761816+0000","last_undegraded":"2024-09-20T18:57:20.761816+0000","last_fullsized":"2024-09-20T18:57:20.761816+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:59.150063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:59.150063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":352,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.759301+0000","last_change":"2024-09-20T18:55:21.629867+0000","last_active":"2024-09-20T18:57:20.759301+0000","last_peered":"2024-09-20T18:57:20.759301+0000","last_clean":"2024-09-20T18:57:20.759301+0000","last_became_active":"2024-09-20T18:49:51.970343+0000","last_became_peered":"2024-09-20T18:49:51.970343+0000","last_unstale":"2024-09-20T18:57:20.759301+0000","last_undegraded":"2024-09-20T18:57:20.759301+0000","last_fullsized":"2024-09-20T18:57:20.759301+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:21.629800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:21.629800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"222'11062","reported_seq":21621,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.920985+0000","last_change":"2024-09-20T18:57:09.139489+0000","last_active":"2024-09-20T18:57:20.920985+0000","last_peered":"2024-09-20T18:57:20.920985+0000","last_clean":"2024-09-20T18:57:20.920985+0000","last_became_active":"2024-09-20T18:57:09.121592+0000","last_became_peered":"2024-09-20T18:57:09.121592+0000","last_unstale":"2024-09-20T18:57:20.920985+0000","last_undegraded":"2024-09-20T18:57:20.920985+0000","last_fullsized":"2024-09-20T18:57:20.920985+0000","mapping_epoch":216,"log_start":"188'6006","ondisk_log_start":"188'6006","created":160,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":5056,"log_dups_size":0,"ondisk_log_size":5056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":227409920,"num_objects":3470,"num_object_clones":0,"num_object_copies":10410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3470,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11062,"num_write_kb":465024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5206,"num_bytes_recovered":341180416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"184'17","reported_seq":177,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.715135+0000","last_change":"2024-09-20T18:55:28.856602+0000","last_active":"2024-09-20T18:57:20.715135+0000","last_peered":"2024-09-20T18:57:20.715135+0000","last_clean":"2024-09-20T18:57:20.715135+0000","last_became_active":"2024-09-20T18:55:28.831024+0000","last_became_peered":"2024-09-20T18:55:28.831024+0000","last_unstale":"2024-09-20T18:57:20.715135+0000","last_undegraded":"2024-09-20T18:57:20.715135+0000","last_fullsized":"2024-09-20T18:57:20.715135+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:49:46.928922+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":24,"num_read_kb":14,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":363,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.757948+0000","last_change":"2024-09-20T18:55:28.830447+0000","last_active":"2024-09-20T18:57:20.757948+0000","last_peered":"2024-09-20T18:57:20.757948+0000","last_clean":"2024-09-20T18:57:20.757948+0000","last_became_active":"2024-09-20T18:55:28.830147+0000","last_became_peered":"2024-09-20T18:55:28.830147+0000","last_unstale":"2024-09-20T18:57:20.757948+0000","last_undegraded":"2024-09-20T18:57:20.757948+0000","last_fullsized":"2024-09-20T18:57:20.757948+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:13.144896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:13.144896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":416,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.717084+0000","last_change":"2024-09-20T18:55:28.844060+0000","last_active":"2024-09-20T18:57:20.717084+0000","last_peered":"2024-09-20T18:57:20.717084+0000","last_clean":"2024-09-20T18:57:20.717084+0000","last_became_active":"2024-09-20T18:55:28.829325+0000","last_became_peered":"2024-09-20T18:55:28.829325+0000","last_unstale":"2024-09-20T18:57:20.717084+0000","last_undegraded":"2024-09-20T18:57:20.717084+0000","last_fullsized":"2024-09-20T18:57:20.717084+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T18:55:13.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T18:55:13.696607+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":339,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.717457+0000","last_change":"2024-09-20T18:51:22.852942+0000","last_active":"2024-09-20T18:57:20.717457+0000","last_peered":"2024-09-20T18:57:20.717457+0000","last_clean":"2024-09-20T18:57:20.717457+0000","last_became_active":"2024-09-20T18:49:49.956128+0000","last_became_peered":"2024-09-20T18:49:49.956128+0000","last_unstale":"2024-09-20T18:57:20.717457+0000","last_undegraded":"2024-09-20T18:57:20.717457+0000","last_fullsized":"2024-09-20T18:57:20.717457+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:22.852869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:22.852869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":351,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.761595+0000","last_change":"2024-09-20T18:55:25.281435+0000","last_active":"2024-09-20T18:57:20.761595+0000","last_peered":"2024-09-20T18:57:20.761595+0000","last_clean":"2024-09-20T18:57:20.761595+0000","last_became_active":"2024-09-20T18:49:51.971332+0000","last_became_peered":"2024-09-20T18:49:51.971332+0000","last_unstale":"2024-09-20T18:57:20.761595+0000","last_undegraded":"2024-09-20T18:57:20.761595+0000","last_fullsized":"2024-09-20T18:57:20.761595+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:25.281366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:25.281366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"222'11048","reported_seq":11184,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.923113+0000","last_change":"2024-09-20T18:54:33.929366+0000","last_active":"2024-09-20T18:57:20.923113+0000","last_peered":"2024-09-20T18:57:20.923113+0000","last_clean":"2024-09-20T18:57:20.923113+0000","last_became_active":"2024-09-20T18:54:33.929072+0000","last_became_peered":"2024-09-20T18:54:33.929072+0000","last_unstale":"2024-09-20T18:57:20.923113+0000","last_undegraded":"2024-09-20T18:57:20.923113+0000","last_fullsized":"2024-09-20T18:57:20.923113+0000","mapping_epoch":160,"log_start":"190'6502","ondisk_log_start":"190'6502","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4546,"log_dups_size":0,"ondisk_log_size":4546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":230293504,"num_objects":3514,"num_object_clones":0,"num_object_copies":10542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11048,"num_write_kb":465984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":382,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.715426+0000","last_change":"2024-09-20T18:55:28.840105+0000","last_active":"2024-09-20T18:57:20.715426+0000","last_peered":"2024-09-20T18:57:20.715426+0000","last_clean":"2024-09-20T18:57:20.715426+0000","last_became_active":"2024-09-20T18:55:28.818269+0000","last_became_peered":"2024-09-20T18:55:28.818269+0000","last_unstale":"2024-09-20T18:57:20.715426+0000","last_undegraded":"2024-09-20T18:57:20.715426+0000","last_fullsized":"2024-09-20T18:57:20.715426+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:51:14.397930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.397930+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":348,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.714561+0000","last_change":"2024-09-20T18:51:18.410601+0000","last_active":"2024-09-20T18:57:20.714561+0000","last_peered":"2024-09-20T18:57:20.714561+0000","last_clean":"2024-09-20T18:57:20.714561+0000","last_became_active":"2024-09-20T18:49:45.944579+0000","last_became_peered":"2024-09-20T18:49:45.944579+0000","last_unstale":"2024-09-20T18:57:20.714561+0000","last_undegraded":"2024-09-20T18:57:20.714561+0000","last_fullsized":"2024-09-20T18:57:20.714561+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:18.410447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:18.410447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":362,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.761470+0000","last_change":"2024-09-20T18:55:28.830634+0000","last_active":"2024-09-20T18:57:20.761470+0000","last_peered":"2024-09-20T18:57:20.761470+0000","last_clean":"2024-09-20T18:57:20.761470+0000","last_became_active":"2024-09-20T18:55:28.827785+0000","last_became_peered":"2024-09-20T18:55:28.827785+0000","last_unstale":"2024-09-20T18:57:20.761470+0000","last_undegraded":"2024-09-20T18:57:20.761470+0000","last_fullsized":"2024-09-20T18:57:20.761470+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:11.401777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:11.401777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":343,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.759608+0000","last_change":"2024-09-20T18:55:28.833649+0000","last_active":"2024-09-20T18:57:20.759608+0000","last_peered":"2024-09-20T18:57:20.759608+0000","last_clean":"2024-09-20T18:57:20.759608+0000","last_became_active":"2024-09-20T18:55:28.833115+0000","last_became_peered":"2024-09-20T18:55:28.833115+0000","last_unstale":"2024-09-20T18:57:20.759608+0000","last_undegraded":"2024-09-20T18:57:20.759608+0000","last_fullsized":"2024-09-20T18:57:20.759608+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:05.021071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.021071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1","version":"222'10949","reported_seq":11085,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:21.292413+0000","last_change":"2024-09-20T18:54:34.310237+0000","last_active":"2024-09-20T18:57:21.292413+0000","last_peered":"2024-09-20T18:57:21.292413+0000","last_clean":"2024-09-20T18:57:21.292413+0000","last_became_active":"2024-09-20T18:54:34.309781+0000","last_became_peered":"2024-09-20T18:54:34.309781+0000","last_unstale":"2024-09-20T18:57:21.292413+0000","last_undegraded":"2024-09-20T18:57:21.292413+0000","last_fullsized":"2024-09-20T18:57:21.292413+0000","mapping_epoch":160,"log_start":"190'6624","ondisk_log_start":"190'6624","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4325,"log_dups_size":0,"ondisk_log_size":4325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":228261888,"num_objects":3483,"num_object_clones":0,"num_object_copies":10449,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10949,"num_write_kb":461824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":380,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.715701+0000","last_change":"2024-09-20T18:52:02.944536+0000","last_active":"2024-09-20T18:57:20.715701+0000","last_peered":"2024-09-20T18:57:20.715701+0000","last_clean":"2024-09-20T18:57:20.715701+0000","last_became_active":"2024-09-20T18:49:47.947636+0000","last_became_peered":"2024-09-20T18:49:47.947636+0000","last_unstale":"2024-09-20T18:57:20.715701+0000","last_undegraded":"2024-09-20T18:57:20.715701+0000","last_fullsized":"2024-09-20T18:57:20.715701+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-20T18:52:02.944434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:02.944434+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":217,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.714470+0000","last_change":"2024-09-20T18:55:28.822330+0000","last_active":"2024-09-20T18:57:20.714470+0000","last_peered":"2024-09-20T18:57:20.714470+0000","last_clean":"2024-09-20T18:57:20.714470+0000","last_became_active":"2024-09-20T18:55:28.821959+0000","last_became_peered":"2024-09-20T18:55:28.821959+0000","last_unstale":"2024-09-20T18:57:20.714470+0000","last_undegraded":"2024-09-20T18:57:20.714470+0000","last_fullsized":"2024-09-20T18:57:20.714470+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:45.669427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:45.669427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":348,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.759386+0000","last_change":"2024-09-20T18:54:11.377907+0000","last_active":"2024-09-20T18:57:20.759386+0000","last_peered":"2024-09-20T18:57:20.759386+0000","last_clean":"2024-09-20T18:57:20.759386+0000","last_became_active":"2024-09-20T18:49:49.959160+0000","last_became_peered":"2024-09-20T18:49:49.959160+0000","last_unstale":"2024-09-20T18:57:20.759386+0000","last_undegraded":"2024-09-20T18:57:20.759386+0000","last_fullsized":"2024-09-20T18:57:20.759386+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:11.376951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:11.376951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":345,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.716748+0000","last_change":"2024-09-20T18:52:35.635897+0000","last_active":"2024-09-20T18:57:20.716748+0000","last_peered":"2024-09-20T18:57:20.716748+0000","last_clean":"2024-09-20T18:57:20.716748+0000","last_became_active":"2024-09-20T18:49:51.970108+0000","last_became_peered":"2024-09-20T18:49:51.970108+0000","last_unstale":"2024-09-20T18:57:20.716748+0000","last_undegraded":"2024-09-20T18:57:20.716748+0000","last_fullsized":"2024-09-20T18:57:20.716748+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:35.635818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:35.635818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"222'10863","reported_seq":10999,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:21.485409+0000","last_change":"2024-09-20T18:54:33.929552+0000","last_active":"2024-09-20T18:57:21.485409+0000","last_peered":"2024-09-20T18:57:21.485409+0000","last_clean":"2024-09-20T18:57:21.485409+0000","last_became_active":"2024-09-20T18:54:33.929208+0000","last_became_peered":"2024-09-20T18:54:33.929208+0000","last_unstale":"2024-09-20T18:57:21.485409+0000","last_undegraded":"2024-09-20T18:57:21.485409+0000","last_fullsized":"2024-09-20T18:57:21.485409+0000","mapping_epoch":160,"log_start":"190'6521","ondisk_log_start":"190'6521","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4342,"log_dups_size":0,"ondisk_log_size":4342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220397568,"num_objects":3363,"num_object_clones":0,"num_object_copies":10089,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3363,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10863,"num_write_kb":455232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":386,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.714949+0000","last_change":"2024-09-20T18:55:28.846773+0000","last_active":"2024-09-20T18:57:20.714949+0000","last_peered":"2024-09-20T18:57:20.714949+0000","last_clean":"2024-09-20T18:57:20.714949+0000","last_became_active":"2024-09-20T18:55:28.817101+0000","last_became_peered":"2024-09-20T18:55:28.817101+0000","last_unstale":"2024-09-20T18:57:20.714949+0000","last_undegraded":"2024-09-20T18:57:20.714949+0000","last_fullsized":"2024-09-20T18:57:20.714949+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:54:17.219548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:17.219548+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":356,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.758951+0000","last_change":"2024-09-20T18:52:46.005172+0000","last_active":"2024-09-20T18:57:20.758951+0000","last_peered":"2024-09-20T18:57:20.758951+0000","last_clean":"2024-09-20T18:57:20.758951+0000","last_became_active":"2024-09-20T18:49:45.937264+0000","last_became_peered":"2024-09-20T18:49:45.937264+0000","last_unstale":"2024-09-20T18:57:20.758951+0000","last_undegraded":"2024-09-20T18:57:20.758951+0000","last_fullsized":"2024-09-20T18:57:20.758951+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:46.005109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:46.005109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":457,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.716318+0000","last_change":"2024-09-20T18:55:17.549660+0000","last_active":"2024-09-20T18:57:20.716318+0000","last_peered":"2024-09-20T18:57:20.716318+0000","last_clean":"2024-09-20T18:57:20.716318+0000","last_became_active":"2024-09-20T18:49:49.957005+0000","last_became_peered":"2024-09-20T18:49:49.957005+0000","last_unstale":"2024-09-20T18:57:20.716318+0000","last_undegraded":"2024-09-20T18:57:20.716318+0000","last_fullsized":"2024-09-20T18:57:20.716318+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:17.549598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:17.549598+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":344,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.714940+0000","last_change":"2024-09-20T18:54:42.182875+0000","last_active":"2024-09-20T18:57:20.714940+0000","last_peered":"2024-09-20T18:57:20.714940+0000","last_clean":"2024-09-20T18:57:20.714940+0000","last_became_active":"2024-09-20T18:49:51.969355+0000","last_became_peered":"2024-09-20T18:49:51.969355+0000","last_unstale":"2024-09-20T18:57:20.714940+0000","last_undegraded":"2024-09-20T18:57:20.714940+0000","last_fullsized":"2024-09-20T18:57:20.714940+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:42.182809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:42.182809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"222'11103","reported_seq":23507,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:21.285277+0000","last_change":"2024-09-20T18:57:00.083015+0000","last_active":"2024-09-20T18:57:21.285277+0000","last_peered":"2024-09-20T18:57:21.285277+0000","last_clean":"2024-09-20T18:57:21.285277+0000","last_became_active":"2024-09-20T18:57:00.060960+0000","last_became_peered":"2024-09-20T18:57:00.060960+0000","last_unstale":"2024-09-20T18:57:21.285277+0000","last_undegraded":"2024-09-20T18:57:21.285277+0000","last_fullsized":"2024-09-20T18:57:21.285277+0000","mapping_epoch":213,"log_start":"192'6861","ondisk_log_start":"192'6861","created":160,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4242,"log_dups_size":0,"ondisk_log_size":4242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":225902592,"num_objects":3447,"num_object_clones":0,"num_object_copies":10341,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11103,"num_write_kb":465600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6125,"num_bytes_recovered":401408000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":375,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.718809+0000","last_change":"2024-09-20T18:54:54.897991+0000","last_active":"2024-09-20T18:57:20.718809+0000","last_peered":"2024-09-20T18:57:20.718809+0000","last_clean":"2024-09-20T18:57:20.718809+0000","last_became_active":"2024-09-20T18:49:47.946008+0000","last_became_peered":"2024-09-20T18:49:47.946008+0000","last_unstale":"2024-09-20T18:57:20.718809+0000","last_undegraded":"2024-09-20T18:57:20.718809+0000","last_fullsized":"2024-09-20T18:57:20.718809+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:54.897933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:54.897933+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":370,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.761277+0000","last_change":"2024-09-20T18:55:28.833453+0000","last_active":"2024-09-20T18:57:20.761277+0000","last_peered":"2024-09-20T18:57:20.761277+0000","last_clean":"2024-09-20T18:57:20.761277+0000","last_became_active":"2024-09-20T18:55:28.833095+0000","last_became_peered":"2024-09-20T18:55:28.833095+0000","last_unstale":"2024-09-20T18:57:20.761277+0000","last_undegraded":"2024-09-20T18:57:20.761277+0000","last_fullsized":"2024-09-20T18:57:20.761277+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:56.212378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:56.212378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":357,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.715588+0000","last_change":"2024-09-20T18:54:43.379898+0000","last_active":"2024-09-20T18:57:20.715588+0000","last_peered":"2024-09-20T18:57:20.715588+0000","last_clean":"2024-09-20T18:57:20.715588+0000","last_became_active":"2024-09-20T18:49:49.956767+0000","last_became_peered":"2024-09-20T18:49:49.956767+0000","last_unstale":"2024-09-20T18:57:20.715588+0000","last_undegraded":"2024-09-20T18:57:20.715588+0000","last_fullsized":"2024-09-20T18:57:20.715588+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:43.379706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:43.379706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":343,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.717965+0000","last_change":"2024-09-20T18:54:24.976627+0000","last_active":"2024-09-20T18:57:20.717965+0000","last_peered":"2024-09-20T18:57:20.717965+0000","last_clean":"2024-09-20T18:57:20.717965+0000","last_became_active":"2024-09-20T18:49:51.964098+0000","last_became_peered":"2024-09-20T18:49:51.964098+0000","last_unstale":"2024-09-20T18:57:20.717965+0000","last_undegraded":"2024-09-20T18:57:20.717965+0000","last_fullsized":"2024-09-20T18:57:20.717965+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:24.976584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:24.976584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"222'11068","reported_seq":15529,"reported_epoch":222,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-20T18:57:21.821256+0000","last_change":"2024-09-20T18:56:27.871592+0000","last_active":"2024-09-20T18:57:21.821256+0000","last_peered":"2024-09-20T18:57:21.821256+0000","last_clean":"2024-09-20T18:55:27.791097+0000","last_became_active":"2024-09-20T18:55:29.979932+0000","last_became_peered":"2024-09-20T18:55:29.979932+0000","last_unstale":"2024-09-20T18:57:21.821256+0000","last_undegraded":"2024-09-20T18:56:27.867865+0000","last_fullsized":"2024-09-20T18:55:29.815116+0000","mapping_epoch":182,"log_start":"189'6140","ondisk_log_start":"189'6140","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4928,"log_dups_size":0,"ondisk_log_size":4928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":221511680,"num_objects":3380,"num_object_clones":0,"num_object_copies":10140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1736,"num_objects_misplaced":1281,"num_objects_unfound":0,"num_objects_dirty":3380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11068,"num_write_kb":462336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2161,"num_bytes_recovered":141623296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7],"avail_no_missing":["0","6","7"],"object_location_counts":[{"shards":"0,6,7","objects":1281},{"shards":"0,7","objects":363}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":164,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.759902+0000","last_change":"2024-09-20T18:55:28.855793+0000","last_active":"2024-09-20T18:57:20.759902+0000","last_peered":"2024-09-20T18:57:20.759902+0000","last_clean":"2024-09-20T18:57:20.759902+0000","last_became_active":"2024-09-20T18:55:28.828563+0000","last_became_peered":"2024-09-20T18:55:28.828563+0000","last_unstale":"2024-09-20T18:57:20.759902+0000","last_undegraded":"2024-09-20T18:57:20.759902+0000","last_fullsized":"2024-09-20T18:57:20.759902+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-20T18:53:24.458005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:24.458005+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":31,"num_read_kb":20,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,5],"acting":[1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":370,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.717937+0000","last_change":"2024-09-20T18:55:28.835767+0000","last_active":"2024-09-20T18:57:20.717937+0000","last_peered":"2024-09-20T18:57:20.717937+0000","last_clean":"2024-09-20T18:57:20.717937+0000","last_became_active":"2024-09-20T18:55:28.835149+0000","last_became_peered":"2024-09-20T18:55:28.835149+0000","last_unstale":"2024-09-20T18:57:20.717937+0000","last_undegraded":"2024-09-20T18:57:20.717937+0000","last_fullsized":"2024-09-20T18:57:20.717937+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:44.501192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.501192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":357,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.717763+0000","last_change":"2024-09-20T18:55:13.577772+0000","last_active":"2024-09-20T18:57:20.717763+0000","last_peered":"2024-09-20T18:57:20.717763+0000","last_clean":"2024-09-20T18:57:20.717763+0000","last_became_active":"2024-09-20T18:49:49.957558+0000","last_became_peered":"2024-09-20T18:49:49.957558+0000","last_unstale":"2024-09-20T18:57:20.717763+0000","last_undegraded":"2024-09-20T18:57:20.717763+0000","last_fullsized":"2024-09-20T18:57:20.717763+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:13.577699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:13.577699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":351,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.716967+0000","last_change":"2024-09-20T18:53:47.370152+0000","last_active":"2024-09-20T18:57:20.716967+0000","last_peered":"2024-09-20T18:57:20.716967+0000","last_clean":"2024-09-20T18:57:20.716967+0000","last_became_active":"2024-09-20T18:49:51.970798+0000","last_became_peered":"2024-09-20T18:49:51.970798+0000","last_unstale":"2024-09-20T18:57:20.716967+0000","last_undegraded":"2024-09-20T18:57:20.716967+0000","last_fullsized":"2024-09-20T18:57:20.716967+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:47.370108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:47.370108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"222'11378","reported_seq":22307,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.908283+0000","last_change":"2024-09-20T18:57:20.750334+0000","last_active":"2024-09-20T18:57:20.908283+0000","last_peered":"2024-09-20T18:57:20.908283+0000","last_clean":"2024-09-20T18:57:20.908283+0000","last_became_active":"2024-09-20T18:57:20.718369+0000","last_became_peered":"2024-09-20T18:57:20.718369+0000","last_unstale":"2024-09-20T18:57:20.908283+0000","last_undegraded":"2024-09-20T18:57:20.908283+0000","last_fullsized":"2024-09-20T18:57:20.908283+0000","mapping_epoch":221,"log_start":"190'6746","ondisk_log_start":"190'6746","created":160,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4632,"log_dups_size":0,"ondisk_log_size":4632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":236453888,"num_objects":3608,"num_object_clones":0,"num_object_copies":10824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11378,"num_write_kb":479552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5391,"num_bytes_recovered":353304576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":375,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.714486+0000","last_change":"2024-09-20T18:53:52.898818+0000","last_active":"2024-09-20T18:57:20.714486+0000","last_peered":"2024-09-20T18:57:20.714486+0000","last_clean":"2024-09-20T18:57:20.714486+0000","last_became_active":"2024-09-20T18:49:47.949116+0000","last_became_peered":"2024-09-20T18:49:47.949116+0000","last_unstale":"2024-09-20T18:57:20.714486+0000","last_undegraded":"2024-09-20T18:57:20.714486+0000","last_fullsized":"2024-09-20T18:57:20.714486+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:53:52.898760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:52.898760+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":363,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.717724+0000","last_change":"2024-09-20T18:54:52.650798+0000","last_active":"2024-09-20T18:57:20.717724+0000","last_peered":"2024-09-20T18:57:20.717724+0000","last_clean":"2024-09-20T18:57:20.717724+0000","last_became_active":"2024-09-20T18:49:45.944858+0000","last_became_peered":"2024-09-20T18:49:45.944858+0000","last_unstale":"2024-09-20T18:57:20.717724+0000","last_undegraded":"2024-09-20T18:57:20.717724+0000","last_fullsized":"2024-09-20T18:57:20.717724+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:52.650756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:52.650756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":349,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.715373+0000","last_change":"2024-09-20T18:54:42.349879+0000","last_active":"2024-09-20T18:57:20.715373+0000","last_peered":"2024-09-20T18:57:20.715373+0000","last_clean":"2024-09-20T18:57:20.715373+0000","last_became_active":"2024-09-20T18:49:49.954944+0000","last_became_peered":"2024-09-20T18:49:49.954944+0000","last_unstale":"2024-09-20T18:57:20.715373+0000","last_undegraded":"2024-09-20T18:57:20.715373+0000","last_fullsized":"2024-09-20T18:57:20.715373+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:42.349807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:42.349807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":344,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.758595+0000","last_change":"2024-09-20T18:55:17.341862+0000","last_active":"2024-09-20T18:57:20.758595+0000","last_peered":"2024-09-20T18:57:20.758595+0000","last_clean":"2024-09-20T18:57:20.758595+0000","last_became_active":"2024-09-20T18:49:51.972970+0000","last_became_peered":"2024-09-20T18:49:51.972970+0000","last_unstale":"2024-09-20T18:57:20.758595+0000","last_undegraded":"2024-09-20T18:57:20.758595+0000","last_fullsized":"2024-09-20T18:57:20.758595+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:17.341793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:17.341793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.4","version":"222'10862","reported_seq":10998,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.906326+0000","last_change":"2024-09-20T18:54:34.309442+0000","last_active":"2024-09-20T18:57:20.906326+0000","last_peered":"2024-09-20T18:57:20.906326+0000","last_clean":"2024-09-20T18:57:20.906326+0000","last_became_active":"2024-09-20T18:54:34.309113+0000","last_became_peered":"2024-09-20T18:54:34.309113+0000","last_unstale":"2024-09-20T18:57:20.906326+0000","last_undegraded":"2024-09-20T18:57:20.906326+0000","last_fullsized":"2024-09-20T18:57:20.906326+0000","mapping_epoch":160,"log_start":"188'5831","ondisk_log_start":"188'5831","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":5031,"log_dups_size":0,"ondisk_log_size":5031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":227409920,"num_objects":3470,"num_object_clones":0,"num_object_copies":10410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3470,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10862,"num_write_kb":458624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":399,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.763044+0000","last_change":"2024-09-20T18:55:28.847010+0000","last_active":"2024-09-20T18:57:20.763044+0000","last_peered":"2024-09-20T18:57:20.763044+0000","last_clean":"2024-09-20T18:57:20.763044+0000","last_became_active":"2024-09-20T18:55:28.824801+0000","last_became_peered":"2024-09-20T18:55:28.824801+0000","last_unstale":"2024-09-20T18:57:20.763044+0000","last_undegraded":"2024-09-20T18:57:20.763044+0000","last_fullsized":"2024-09-20T18:57:20.763044+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:54:12.373309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:12.373309+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":383,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.717559+0000","last_change":"2024-09-20T18:55:24.729301+0000","last_active":"2024-09-20T18:57:20.717559+0000","last_peered":"2024-09-20T18:57:20.717559+0000","last_clean":"2024-09-20T18:57:20.717559+0000","last_became_active":"2024-09-20T18:49:45.944256+0000","last_became_peered":"2024-09-20T18:49:45.944256+0000","last_unstale":"2024-09-20T18:57:20.717559+0000","last_undegraded":"2024-09-20T18:57:20.717559+0000","last_fullsized":"2024-09-20T18:57:20.717559+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-20T18:55:24.729219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:24.729219+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"184'9","reported_seq":449,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.717008+0000","last_change":"2024-09-20T18:55:28.824771+0000","last_active":"2024-09-20T18:57:20.717008+0000","last_peered":"2024-09-20T18:57:20.717008+0000","last_clean":"2024-09-20T18:57:20.717008+0000","last_became_active":"2024-09-20T18:55:28.818085+0000","last_became_peered":"2024-09-20T18:55:28.818085+0000","last_unstale":"2024-09-20T18:57:20.717008+0000","last_undegraded":"2024-09-20T18:57:20.717008+0000","last_fullsized":"2024-09-20T18:57:20.717008+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:52:05.788924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.788924+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":350,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.760760+0000","last_change":"2024-09-20T18:55:28.832276+0000","last_active":"2024-09-20T18:57:20.760760+0000","last_peered":"2024-09-20T18:57:20.760760+0000","last_clean":"2024-09-20T18:57:20.760760+0000","last_became_active":"2024-09-20T18:55:28.830380+0000","last_became_peered":"2024-09-20T18:55:28.830380+0000","last_unstale":"2024-09-20T18:57:20.760760+0000","last_undegraded":"2024-09-20T18:57:20.760760+0000","last_fullsized":"2024-09-20T18:57:20.760760+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:14.247398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:14.247398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":93,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.715395+0000","last_change":"2024-09-20T18:55:28.835344+0000","last_active":"2024-09-20T18:57:20.715395+0000","last_peered":"2024-09-20T18:57:20.715395+0000","last_clean":"2024-09-20T18:57:20.715395+0000","last_became_active":"2024-09-20T18:55:28.835010+0000","last_became_peered":"2024-09-20T18:55:28.835010+0000","last_unstale":"2024-09-20T18:57:20.715395+0000","last_undegraded":"2024-09-20T18:57:20.715395+0000","last_fullsized":"2024-09-20T18:57:20.715395+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:26.721735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:26.721735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":361,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.759827+0000","last_change":"2024-09-20T18:54:44.194161+0000","last_active":"2024-09-20T18:57:20.759827+0000","last_peered":"2024-09-20T18:57:20.759827+0000","last_clean":"2024-09-20T18:57:20.759827+0000","last_became_active":"2024-09-20T18:49:48.500662+0000","last_became_peered":"2024-09-20T18:49:48.500662+0000","last_unstale":"2024-09-20T18:57:20.759827+0000","last_undegraded":"2024-09-20T18:57:20.759827+0000","last_fullsized":"2024-09-20T18:57:20.759827+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-20T18:54:44.194089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.194089+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":347,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.761199+0000","last_change":"2024-09-20T18:53:16.220554+0000","last_active":"2024-09-20T18:57:20.761199+0000","last_peered":"2024-09-20T18:57:20.761199+0000","last_clean":"2024-09-20T18:57:20.761199+0000","last_became_active":"2024-09-20T18:49:49.956749+0000","last_became_peered":"2024-09-20T18:49:49.956749+0000","last_unstale":"2024-09-20T18:57:20.761199+0000","last_undegraded":"2024-09-20T18:57:20.761199+0000","last_fullsized":"2024-09-20T18:57:20.761199+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:16.220490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:16.220490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":358,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.717567+0000","last_change":"2024-09-20T18:55:28.827897+0000","last_active":"2024-09-20T18:57:20.717567+0000","last_peered":"2024-09-20T18:57:20.717567+0000","last_clean":"2024-09-20T18:57:20.717567+0000","last_became_active":"2024-09-20T18:55:28.827418+0000","last_became_peered":"2024-09-20T18:55:28.827418+0000","last_unstale":"2024-09-20T18:57:20.717567+0000","last_undegraded":"2024-09-20T18:57:20.717567+0000","last_fullsized":"2024-09-20T18:57:20.717567+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.637739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.637739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":367,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.719977+0000","last_change":"2024-09-20T18:54:44.830872+0000","last_active":"2024-09-20T18:57:20.719977+0000","last_peered":"2024-09-20T18:57:20.719977+0000","last_clean":"2024-09-20T18:57:20.719977+0000","last_became_active":"2024-09-20T18:49:48.500803+0000","last_became_peered":"2024-09-20T18:49:48.500803+0000","last_unstale":"2024-09-20T18:57:20.719977+0000","last_undegraded":"2024-09-20T18:57:20.719977+0000","last_fullsized":"2024-09-20T18:57:20.719977+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:44.830790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.830790+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":362,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.717127+0000","last_change":"2024-09-20T18:55:29.751718+0000","last_active":"2024-09-20T18:57:20.717127+0000","last_peered":"2024-09-20T18:57:20.717127+0000","last_clean":"2024-09-20T18:57:20.717127+0000","last_became_active":"2024-09-20T18:55:28.833799+0000","last_became_peered":"2024-09-20T18:55:28.833799+0000","last_unstale":"2024-09-20T18:57:20.717127+0000","last_undegraded":"2024-09-20T18:57:20.717127+0000","last_fullsized":"2024-09-20T18:57:20.717127+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:29.751642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:29.751642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":348,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.759683+0000","last_change":"2024-09-20T18:53:50.253721+0000","last_active":"2024-09-20T18:57:20.759683+0000","last_peered":"2024-09-20T18:57:20.759683+0000","last_clean":"2024-09-20T18:57:20.759683+0000","last_became_active":"2024-09-20T18:49:49.961947+0000","last_became_peered":"2024-09-20T18:49:49.961947+0000","last_unstale":"2024-09-20T18:57:20.759683+0000","last_undegraded":"2024-09-20T18:57:20.759683+0000","last_fullsized":"2024-09-20T18:57:20.759683+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:50.253660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:50.253660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"66'1","reported_seq":354,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.715935+0000","last_change":"2024-09-20T18:54:20.681878+0000","last_active":"2024-09-20T18:57:20.715935+0000","last_peered":"2024-09-20T18:57:20.715935+0000","last_clean":"2024-09-20T18:57:20.715935+0000","last_became_active":"2024-09-20T18:49:51.965121+0000","last_became_peered":"2024-09-20T18:49:51.965121+0000","last_unstale":"2024-09-20T18:57:20.715935+0000","last_undegraded":"2024-09-20T18:57:20.715935+0000","last_fullsized":"2024-09-20T18:57:20.715935+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T18:54:20.681790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:20.681790+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":359,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.761757+0000","last_change":"2024-09-20T18:55:12.205118+0000","last_active":"2024-09-20T18:57:20.761757+0000","last_peered":"2024-09-20T18:57:20.761757+0000","last_clean":"2024-09-20T18:57:20.761757+0000","last_became_active":"2024-09-20T18:49:48.501011+0000","last_became_peered":"2024-09-20T18:49:48.501011+0000","last_unstale":"2024-09-20T18:57:20.761757+0000","last_undegraded":"2024-09-20T18:57:20.761757+0000","last_fullsized":"2024-09-20T18:57:20.761757+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:55:12.205069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:12.205069+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":363,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.718640+0000","last_change":"2024-09-20T18:55:27.028184+0000","last_active":"2024-09-20T18:57:20.718640+0000","last_peered":"2024-09-20T18:57:20.718640+0000","last_clean":"2024-09-20T18:57:20.718640+0000","last_became_active":"2024-09-20T18:49:45.938511+0000","last_became_peered":"2024-09-20T18:49:45.938511+0000","last_unstale":"2024-09-20T18:57:20.718640+0000","last_undegraded":"2024-09-20T18:57:20.718640+0000","last_fullsized":"2024-09-20T18:57:20.718640+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:27.028126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:27.028126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":456,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.759889+0000","last_change":"2024-09-20T18:55:27.284156+0000","last_active":"2024-09-20T18:57:20.759889+0000","last_peered":"2024-09-20T18:57:20.759889+0000","last_clean":"2024-09-20T18:57:20.759889+0000","last_became_active":"2024-09-20T18:49:49.960550+0000","last_became_peered":"2024-09-20T18:49:49.960550+0000","last_unstale":"2024-09-20T18:57:20.759889+0000","last_undegraded":"2024-09-20T18:57:20.759889+0000","last_fullsized":"2024-09-20T18:57:20.759889+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:27.284086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:27.284086+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":352,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.715178+0000","last_change":"2024-09-20T18:55:12.348079+0000","last_active":"2024-09-20T18:57:20.715178+0000","last_peered":"2024-09-20T18:57:20.715178+0000","last_clean":"2024-09-20T18:57:20.715178+0000","last_became_active":"2024-09-20T18:49:51.966254+0000","last_became_peered":"2024-09-20T18:49:51.966254+0000","last_unstale":"2024-09-20T18:57:20.715178+0000","last_undegraded":"2024-09-20T18:57:20.715178+0000","last_fullsized":"2024-09-20T18:57:20.715178+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:12.347993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:12.347993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":386,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.760151+0000","last_change":"2024-09-20T18:57:20.701592+0000","last_active":"2024-09-20T18:57:20.760151+0000","last_peered":"2024-09-20T18:57:20.760151+0000","last_clean":"2024-09-20T18:57:20.760151+0000","last_became_active":"2024-09-20T18:49:58.997494+0000","last_became_peered":"2024-09-20T18:49:58.997494+0000","last_unstale":"2024-09-20T18:57:20.760151+0000","last_undegraded":"2024-09-20T18:57:20.760151+0000","last_fullsized":"2024-09-20T18:57:20.760151+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:57:20.701543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:20.701543+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:44.352173+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":363,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.717431+0000","last_change":"2024-09-20T18:54:57.632458+0000","last_active":"2024-09-20T18:57:20.717431+0000","last_peered":"2024-09-20T18:57:20.717431+0000","last_clean":"2024-09-20T18:57:20.717431+0000","last_became_active":"2024-09-20T18:49:45.949492+0000","last_became_peered":"2024-09-20T18:49:45.949492+0000","last_unstale":"2024-09-20T18:57:20.717431+0000","last_undegraded":"2024-09-20T18:57:20.717431+0000","last_fullsized":"2024-09-20T18:57:20.717431+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.632380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.632380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":454,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.761536+0000","last_change":"2024-09-20T18:55:28.302353+0000","last_active":"2024-09-20T18:57:20.761536+0000","last_peered":"2024-09-20T18:57:20.761536+0000","last_clean":"2024-09-20T18:57:20.761536+0000","last_became_active":"2024-09-20T18:49:49.962243+0000","last_became_peered":"2024-09-20T18:49:49.962243+0000","last_unstale":"2024-09-20T18:57:20.761536+0000","last_undegraded":"2024-09-20T18:57:20.761536+0000","last_fullsized":"2024-09-20T18:57:20.761536+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:28.302303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:28.302303+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":351,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.718412+0000","last_change":"2024-09-20T18:55:11.918284+0000","last_active":"2024-09-20T18:57:20.718412+0000","last_peered":"2024-09-20T18:57:20.718412+0000","last_clean":"2024-09-20T18:57:20.718412+0000","last_became_active":"2024-09-20T18:49:51.969093+0000","last_became_peered":"2024-09-20T18:49:51.969093+0000","last_unstale":"2024-09-20T18:57:20.718412+0000","last_undegraded":"2024-09-20T18:57:20.718412+0000","last_fullsized":"2024-09-20T18:57:20.718412+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:11.918210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:11.918210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":390,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.714804+0000","last_change":"2024-09-20T18:55:28.854249+0000","last_active":"2024-09-20T18:57:20.714804+0000","last_peered":"2024-09-20T18:57:20.714804+0000","last_clean":"2024-09-20T18:57:20.714804+0000","last_became_active":"2024-09-20T18:55:28.826721+0000","last_became_peered":"2024-09-20T18:55:28.826721+0000","last_unstale":"2024-09-20T18:57:20.714804+0000","last_undegraded":"2024-09-20T18:57:20.714804+0000","last_fullsized":"2024-09-20T18:57:20.714804+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:53:43.878921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:43.878921+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":355,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.715082+0000","last_change":"2024-09-20T18:55:28.822690+0000","last_active":"2024-09-20T18:57:20.715082+0000","last_peered":"2024-09-20T18:57:20.715082+0000","last_clean":"2024-09-20T18:57:20.715082+0000","last_became_active":"2024-09-20T18:55:28.822258+0000","last_became_peered":"2024-09-20T18:55:28.822258+0000","last_unstale":"2024-09-20T18:57:20.715082+0000","last_undegraded":"2024-09-20T18:57:20.715082+0000","last_fullsized":"2024-09-20T18:57:20.715082+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:05.985551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.985551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":348,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.758329+0000","last_change":"2024-09-20T18:57:11.986605+0000","last_active":"2024-09-20T18:57:20.758329+0000","last_peered":"2024-09-20T18:57:20.758329+0000","last_clean":"2024-09-20T18:57:20.758329+0000","last_became_active":"2024-09-20T18:49:49.957067+0000","last_became_peered":"2024-09-20T18:49:49.957067+0000","last_unstale":"2024-09-20T18:57:20.758329+0000","last_undegraded":"2024-09-20T18:57:20.758329+0000","last_fullsized":"2024-09-20T18:57:20.758329+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:11.986528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:11.986528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:36.007001+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":344,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.757673+0000","last_change":"2024-09-20T18:57:16.789412+0000","last_active":"2024-09-20T18:57:20.757673+0000","last_peered":"2024-09-20T18:57:20.757673+0000","last_clean":"2024-09-20T18:57:20.757673+0000","last_became_active":"2024-09-20T18:49:51.966915+0000","last_became_peered":"2024-09-20T18:49:51.966915+0000","last_unstale":"2024-09-20T18:57:20.757673+0000","last_undegraded":"2024-09-20T18:57:20.757673+0000","last_fullsized":"2024-09-20T18:57:20.757673+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:16.789095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:16.789095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:45.821909+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":393,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.760076+0000","last_change":"2024-09-20T18:55:28.838698+0000","last_active":"2024-09-20T18:57:20.760076+0000","last_peered":"2024-09-20T18:57:20.760076+0000","last_clean":"2024-09-20T18:57:20.760076+0000","last_became_active":"2024-09-20T18:55:28.827023+0000","last_became_peered":"2024-09-20T18:55:28.827023+0000","last_unstale":"2024-09-20T18:57:20.760076+0000","last_undegraded":"2024-09-20T18:57:20.760076+0000","last_fullsized":"2024-09-20T18:57:20.760076+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:53:53.263976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:53.263976+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":355,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.717870+0000","last_change":"2024-09-20T18:54:42.487465+0000","last_active":"2024-09-20T18:57:20.717870+0000","last_peered":"2024-09-20T18:57:20.717870+0000","last_clean":"2024-09-20T18:57:20.717870+0000","last_became_active":"2024-09-20T18:49:45.944357+0000","last_became_peered":"2024-09-20T18:49:45.944357+0000","last_unstale":"2024-09-20T18:57:20.717870+0000","last_undegraded":"2024-09-20T18:57:20.717870+0000","last_fullsized":"2024-09-20T18:57:20.717870+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:42.487403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:42.487403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":348,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.758681+0000","last_change":"2024-09-20T18:54:18.108239+0000","last_active":"2024-09-20T18:57:20.758681+0000","last_peered":"2024-09-20T18:57:20.758681+0000","last_clean":"2024-09-20T18:57:20.758681+0000","last_became_active":"2024-09-20T18:49:49.962523+0000","last_became_peered":"2024-09-20T18:49:49.962523+0000","last_unstale":"2024-09-20T18:57:20.758681+0000","last_undegraded":"2024-09-20T18:57:20.758681+0000","last_fullsized":"2024-09-20T18:57:20.758681+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:18.108175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:18.108175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":351,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.715019+0000","last_change":"2024-09-20T18:54:26.882441+0000","last_active":"2024-09-20T18:57:20.715019+0000","last_peered":"2024-09-20T18:57:20.715019+0000","last_clean":"2024-09-20T18:57:20.715019+0000","last_became_active":"2024-09-20T18:49:51.967313+0000","last_became_peered":"2024-09-20T18:49:51.967313+0000","last_unstale":"2024-09-20T18:57:20.715019+0000","last_undegraded":"2024-09-20T18:57:20.715019+0000","last_fullsized":"2024-09-20T18:57:20.715019+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:26.882375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:26.882375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":390,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.757747+0000","last_change":"2024-09-20T18:55:28.843363+0000","last_active":"2024-09-20T18:57:20.757747+0000","last_peered":"2024-09-20T18:57:20.757747+0000","last_clean":"2024-09-20T18:57:20.757747+0000","last_became_active":"2024-09-20T18:55:28.830801+0000","last_became_peered":"2024-09-20T18:55:28.830801+0000","last_unstale":"2024-09-20T18:57:20.757747+0000","last_undegraded":"2024-09-20T18:57:20.757747+0000","last_fullsized":"2024-09-20T18:57:20.757747+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:54:21.106677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:21.106677+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":363,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.761276+0000","last_change":"2024-09-20T18:55:19.258723+0000","last_active":"2024-09-20T18:57:20.761276+0000","last_peered":"2024-09-20T18:57:20.761276+0000","last_clean":"2024-09-20T18:57:20.761276+0000","last_became_active":"2024-09-20T18:49:45.942105+0000","last_became_peered":"2024-09-20T18:49:45.942105+0000","last_unstale":"2024-09-20T18:57:20.761276+0000","last_undegraded":"2024-09-20T18:57:20.761276+0000","last_fullsized":"2024-09-20T18:57:20.761276+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:19.258674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:19.258674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":362,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.716709+0000","last_change":"2024-09-20T18:55:28.825004+0000","last_active":"2024-09-20T18:57:20.716709+0000","last_peered":"2024-09-20T18:57:20.716709+0000","last_clean":"2024-09-20T18:57:20.716709+0000","last_became_active":"2024-09-20T18:55:28.824355+0000","last_became_peered":"2024-09-20T18:55:28.824355+0000","last_unstale":"2024-09-20T18:57:20.716709+0000","last_undegraded":"2024-09-20T18:57:20.716709+0000","last_fullsized":"2024-09-20T18:57:20.716709+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:46.329547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:46.329547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":343,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.758860+0000","last_change":"2024-09-20T18:55:28.825607+0000","last_active":"2024-09-20T18:57:20.758860+0000","last_peered":"2024-09-20T18:57:20.758860+0000","last_clean":"2024-09-20T18:57:20.758860+0000","last_became_active":"2024-09-20T18:55:28.825296+0000","last_became_peered":"2024-09-20T18:55:28.825296+0000","last_unstale":"2024-09-20T18:57:20.758860+0000","last_undegraded":"2024-09-20T18:57:20.758860+0000","last_fullsized":"2024-09-20T18:57:20.758860+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:43.118805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:43.118805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":379,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.757623+0000","last_change":"2024-09-20T18:55:28.836789+0000","last_active":"2024-09-20T18:57:20.757623+0000","last_peered":"2024-09-20T18:57:20.757623+0000","last_clean":"2024-09-20T18:57:20.757623+0000","last_became_active":"2024-09-20T18:55:28.824579+0000","last_became_peered":"2024-09-20T18:55:28.824579+0000","last_unstale":"2024-09-20T18:57:20.757623+0000","last_undegraded":"2024-09-20T18:57:20.757623+0000","last_fullsized":"2024-09-20T18:57:20.757623+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-20T18:55:20.282193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:20.282193+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":363,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.719438+0000","last_change":"2024-09-20T18:54:55.900487+0000","last_active":"2024-09-20T18:57:20.719438+0000","last_peered":"2024-09-20T18:57:20.719438+0000","last_clean":"2024-09-20T18:57:20.719438+0000","last_became_active":"2024-09-20T18:49:45.940988+0000","last_became_peered":"2024-09-20T18:49:45.940988+0000","last_unstale":"2024-09-20T18:57:20.719438+0000","last_undegraded":"2024-09-20T18:57:20.719438+0000","last_fullsized":"2024-09-20T18:57:20.719438+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:55.899611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:55.899611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":93,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.714204+0000","last_change":"2024-09-20T18:55:28.832864+0000","last_active":"2024-09-20T18:57:20.714204+0000","last_peered":"2024-09-20T18:57:20.714204+0000","last_clean":"2024-09-20T18:57:20.714204+0000","last_became_active":"2024-09-20T18:55:28.832534+0000","last_became_peered":"2024-09-20T18:55:28.832534+0000","last_unstale":"2024-09-20T18:57:20.714204+0000","last_undegraded":"2024-09-20T18:57:20.714204+0000","last_fullsized":"2024-09-20T18:57:20.714204+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:52.635138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:52.635138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":335,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.716914+0000","last_change":"2024-09-20T18:51:09.761331+0000","last_active":"2024-09-20T18:57:20.716914+0000","last_peered":"2024-09-20T18:57:20.716914+0000","last_clean":"2024-09-20T18:57:20.716914+0000","last_became_active":"2024-09-20T18:49:51.968662+0000","last_became_peered":"2024-09-20T18:49:51.968662+0000","last_unstale":"2024-09-20T18:57:20.716914+0000","last_undegraded":"2024-09-20T18:57:20.716914+0000","last_fullsized":"2024-09-20T18:57:20.716914+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:09.761283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:09.761283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":335,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.758295+0000","last_change":"2024-09-20T18:57:10.970843+0000","last_active":"2024-09-20T18:57:20.758295+0000","last_peered":"2024-09-20T18:57:20.758295+0000","last_clean":"2024-09-20T18:57:20.758295+0000","last_became_active":"2024-09-20T18:49:51.968216+0000","last_became_peered":"2024-09-20T18:49:51.968216+0000","last_unstale":"2024-09-20T18:57:20.758295+0000","last_undegraded":"2024-09-20T18:57:20.758295+0000","last_fullsized":"2024-09-20T18:57:20.758295+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:10.970779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:10.970779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:38.733512+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":356,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.757846+0000","last_change":"2024-09-20T18:55:28.295445+0000","last_active":"2024-09-20T18:57:20.757846+0000","last_peered":"2024-09-20T18:57:20.757846+0000","last_clean":"2024-09-20T18:57:20.757846+0000","last_became_active":"2024-09-20T18:49:45.937992+0000","last_became_peered":"2024-09-20T18:49:45.937992+0000","last_unstale":"2024-09-20T18:57:20.757846+0000","last_undegraded":"2024-09-20T18:57:20.757846+0000","last_fullsized":"2024-09-20T18:57:20.757846+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:28.295399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:28.295399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":144,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.761679+0000","last_change":"2024-09-20T18:55:28.852468+0000","last_active":"2024-09-20T18:57:20.761679+0000","last_peered":"2024-09-20T18:57:20.761679+0000","last_clean":"2024-09-20T18:57:20.761679+0000","last_became_active":"2024-09-20T18:55:28.834675+0000","last_became_peered":"2024-09-20T18:55:28.834675+0000","last_unstale":"2024-09-20T18:57:20.761679+0000","last_undegraded":"2024-09-20T18:57:20.761679+0000","last_fullsized":"2024-09-20T18:57:20.761679+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:54:44.719676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.719676+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":12,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":339,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-20T18:57:20.719631+0000","last_change":"2024-09-20T18:51:28.120100+0000","last_active":"2024-09-20T18:57:20.719631+0000","last_peered":"2024-09-20T18:57:20.719631+0000","last_clean":"2024-09-20T18:57:20.719631+0000","last_became_active":"2024-09-20T18:49:49.957582+0000","last_became_peered":"2024-09-20T18:49:49.957582+0000","last_unstale":"2024-09-20T18:57:20.719631+0000","last_undegraded":"2024-09-20T18:57:20.719631+0000","last_fullsized":"2024-09-20T18:57:20.719631+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:28.119694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:28.119694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":3630039064,"num_objects":55391,"num_object_clones":0,"num_object_copies":166173,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1736,"num_objects_misplaced":1281,"num_objects_unfound":0,"num_objects_dirty":55391,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":176787,"num_write_kb":7429633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25039,"num_bytes_recovered":1640955904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12398374912,"data_stored":12398362696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74120,"ondisk_log_size":74120,"up":48,"acting":47,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":67,"ondisk_log_size":67,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":695,"num_read_kb":452,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":1474,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":12140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":394,"ondisk_log_size":394,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":5275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":450,"num_read_kb":445,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266783,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1328864,"kb_used_data":1121352,"kb_used_omap":1,"kb_used_meta":207486,"kb_avail":92416288,"statfs":{"total":95995035648,"available":94634278912,"internally_reserved":0,"allocated":1148264448,"data_stored":1147846846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":212466122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":0,"seq":193273528422,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757355,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2027272,"kb_used_data":1779508,"kb_used_omap":1,"kb_used_meta":247742,"kb_avail":91717880,"statfs":{"total":95995035648,"available":93919109120,"internally_reserved":0,"allocated":1822216192,"data_stored":1821811692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":253688261},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986288,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1335148,"kb_used_data":1133204,"kb_used_omap":1,"kb_used_meta":201918,"kb_avail":92410004,"statfs":{"total":95995035648,"available":94627844096,"internally_reserved":0,"allocated":1160400896,"data_stored":1159971497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":206764485},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182519,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1898524,"kb_used_data":1564680,"kb_used_omap":1,"kb_used_meta":333822,"kb_avail":91846628,"statfs":{"total":95995035648,"available":94050947072,"internally_reserved":0,"allocated":1602232320,"data_stored":1601807302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":341834183},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411452,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2259536,"kb_used_data":1907640,"kb_used_omap":1,"kb_used_meta":351870,"kb_avail":91485616,"statfs":{"total":95995035648,"available":93681270784,"internally_reserved":0,"allocated":1953423360,"data_stored":1953021874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":360315333},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574978,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1841540,"kb_used_data":1557676,"kb_used_omap":1,"kb_used_meta":283838,"kb_avail":91903612,"statfs":{"total":95995035648,"available":94109298688,"internally_reserved":0,"allocated":1595060224,"data_stored":1594607630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":290650565},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,5],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738504,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1900140,"kb_used_data":1557908,"kb_used_omap":1,"kb_used_meta":342142,"kb_avail":91845012,"statfs":{"total":95995035648,"available":94049292288,"internally_reserved":0,"allocated":1595297792,"data_stored":1594867967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":350353861},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1577517056,"data_stored":1577517056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1587806208,"data_stored":1587806208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1948717056,"data_stored":1948712984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1581645824,"data_stored":1581645824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1155399680,"data_stored":1155399680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1814499328,"data_stored":1814495256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1588658176,"data_stored":1588658176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1144131584,"data_stored":1144127512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T18:57:29.306 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph status --format=json 2024-09-20T18:57:29.334 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:29.405 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:29.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:29.406 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:29.507 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:29.552 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:29.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:29.553 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:29.653 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:29.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:29.696 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:29.696 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:29.797 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:29.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:29.839 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:29.840 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:29.940 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:57:29.966 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:29.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:29.967 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:57:30.067 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:30.094 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:30.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:30.095 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:30.123 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:57:30.196 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:30.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:29 smithi083 ceph-mon[29421]: pgmap v369: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 3.1 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.10k op/s; 1845/151920 objects degraded (1.214%); 1054/151920 objects misplaced (0.694%); 4.5 MiB/s, 71 objects/s recovering 2024-09-20T18:57:30.258 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:29 smithi083 ceph-mon[36260]: pgmap v369: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 3.1 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.10k op/s; 1845/151920 objects degraded (1.214%); 1054/151920 objects misplaced (0.694%); 4.5 MiB/s, 71 objects/s recovering 2024-09-20T18:57:30.260 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:30.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:30.261 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:30.269 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:29 smithi142 ceph-mon[32508]: pgmap v369: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 3.1 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.10k op/s; 1845/151920 objects degraded (1.214%); 1054/151920 objects misplaced (0.694%); 4.5 MiB/s, 71 objects/s recovering 2024-09-20T18:57:30.362 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:30.386 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:30.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:30.387 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:30.488 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:30.512 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:30.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:30.513 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:30.614 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:30.657 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:30.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:30.658 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:30.697 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T18:57:30.697 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-20T18:57:30.697 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-20T18:57:30.697 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-20T18:57:30.698 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.698 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:128076 8.3 8:ca202ecf:::benchmark_data_smithi083_97422_object11985:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e203)", 2024-09-20T18:57:30.698 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:24.802138+0000", 2024-09-20T18:57:30.698 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 65.880524109000007, 2024-09-20T18:57:30.698 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.90051086899999999, 2024-09-20T18:57:30.698 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:57:30.698 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:57:30.698 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:57:30.698 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:57:30.698 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:57:30.698 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:57:30.698 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 128076 2024-09-20T18:57:30.698 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.699 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:57:30.699 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.699 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:57:30.699 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.802138+0000", 2024-09-20T18:57:30.699 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:30.699 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.699 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.699 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:57:30.700 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.802138+0000", 2024-09-20T18:57:30.700 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:30.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.700 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.700 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:57:30.700 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.802139+0000", 2024-09-20T18:57:30.700 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8199999999999999e-07 2024-09-20T18:57:30.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.701 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.701 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:57:30.701 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.802143+0000", 2024-09-20T18:57:30.701 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0500000000000002e-06 2024-09-20T18:57:30.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.701 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.701 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:57:30.701 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.802151+0000", 2024-09-20T18:57:30.701 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8860000000000004e-06 2024-09-20T18:57:30.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.701 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.701 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:57:30.702 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.802157+0000", 2024-09-20T18:57:30.702 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.105e-06 2024-09-20T18:57:30.702 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.702 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.702 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:30.702 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.802209+0000", 2024-09-20T18:57:30.702 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1229e-05 2024-09-20T18:57:30.702 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.702 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.702 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:57:30.702 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.802215+0000", 2024-09-20T18:57:30.702 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8700000000000003e-06 2024-09-20T18:57:30.702 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.703 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.703 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:30.703 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.694932+0000", 2024-09-20T18:57:30.703 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.89271625700000001 2024-09-20T18:57:30.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.703 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.703 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:57:30.703 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.695043+0000", 2024-09-20T18:57:30.703 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000111789 2024-09-20T18:57:30.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.703 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.703 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,5", 2024-09-20T18:57:30.703 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.695170+0000", 2024-09-20T18:57:30.703 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000126255 2024-09-20T18:57:30.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.704 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.704 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:30.704 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.700200+0000", 2024-09-20T18:57:30.704 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0050300939999999997 2024-09-20T18:57:30.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.704 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.704 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:30.704 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.700261+0000", 2024-09-20T18:57:30.704 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1340000000000006e-05 2024-09-20T18:57:30.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.704 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.704 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:57:30.704 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.702559+0000", 2024-09-20T18:57:30.705 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002297707 2024-09-20T18:57:30.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.705 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.705 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:57:30.705 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.702606+0000", 2024-09-20T18:57:30.705 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7037999999999999e-05 2024-09-20T18:57:30.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.705 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.705 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:57:30.705 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.702649+0000", 2024-09-20T18:57:30.705 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3467000000000002e-05 2024-09-20T18:57:30.705 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:30.705 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:30.705 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:30.706 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.706 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.706 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:128090 8.3 8:c10076ee:::benchmark_data_smithi083_97422_object11999:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e203)", 2024-09-20T18:57:30.706 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:24.815492+0000", 2024-09-20T18:57:30.706 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 65.867170861000005, 2024-09-20T18:57:30.706 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88975575799999995, 2024-09-20T18:57:30.706 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:57:30.706 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:57:30.706 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:57:30.706 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:57:30.706 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:57:30.706 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:57:30.706 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 128090 2024-09-20T18:57:30.707 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.707 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:57:30.707 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.707 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:57:30.707 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.815492+0000", 2024-09-20T18:57:30.707 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:30.707 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.707 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.707 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:57:30.707 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.815492+0000", 2024-09-20T18:57:30.707 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:30.707 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.707 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.707 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:57:30.708 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.815493+0000", 2024-09-20T18:57:30.708 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2920000000000001e-06 2024-09-20T18:57:30.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.708 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.708 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:57:30.708 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.815497+0000", 2024-09-20T18:57:30.708 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8349999999999997e-06 2024-09-20T18:57:30.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.708 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.708 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:57:30.708 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.815505+0000", 2024-09-20T18:57:30.708 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2830000000000001e-06 2024-09-20T18:57:30.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.708 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.709 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:57:30.709 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.815512+0000", 2024-09-20T18:57:30.709 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8499999999999996e-06 2024-09-20T18:57:30.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.709 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.709 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:30.709 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.815561+0000", 2024-09-20T18:57:30.709 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8566000000000002e-05 2024-09-20T18:57:30.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.709 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.709 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:57:30.709 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.815568+0000", 2024-09-20T18:57:30.709 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.746e-06 2024-09-20T18:57:30.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.710 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.710 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:30.710 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.697779+0000", 2024-09-20T18:57:30.710 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88221116799999999 2024-09-20T18:57:30.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.710 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.710 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:57:30.710 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.697904+0000", 2024-09-20T18:57:30.710 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012453900000000001 2024-09-20T18:57:30.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.710 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.710 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,5", 2024-09-20T18:57:30.710 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.698026+0000", 2024-09-20T18:57:30.710 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121783 2024-09-20T18:57:30.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.711 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.711 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:30.711 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.705094+0000", 2024-09-20T18:57:30.711 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.007068146 2024-09-20T18:57:30.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.711 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.711 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:30.711 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.705140+0000", 2024-09-20T18:57:30.711 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6332000000000003e-05 2024-09-20T18:57:30.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.711 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.711 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:57:30.711 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.705154+0000", 2024-09-20T18:57:30.711 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3516000000000001e-05 2024-09-20T18:57:30.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.712 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.712 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:57:30.712 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.705203+0000", 2024-09-20T18:57:30.712 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.918e-05 2024-09-20T18:57:30.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.712 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.713 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:57:30.713 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.705247+0000", 2024-09-20T18:57:30.713 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4521999999999998e-05 2024-09-20T18:57:30.713 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:30.713 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:30.713 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:30.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.713 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.713 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:128136 8.3 8:c6a9dcdc:::benchmark_data_smithi083_97422_object12045:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e203)", 2024-09-20T18:57:30.713 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:24.875803+0000", 2024-09-20T18:57:30.714 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 65.806859262000003, 2024-09-20T18:57:30.714 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.82965221499999997, 2024-09-20T18:57:30.714 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:57:30.714 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:57:30.714 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:57:30.714 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:57:30.714 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:57:30.714 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:57:30.714 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 128136 2024-09-20T18:57:30.714 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.714 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:57:30.714 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.714 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:57:30.714 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.875803+0000", 2024-09-20T18:57:30.715 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:30.715 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.715 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.715 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:57:30.715 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.875803+0000", 2024-09-20T18:57:30.715 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:30.715 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.715 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.715 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:57:30.715 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.875804+0000", 2024-09-20T18:57:30.715 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.51e-07 2024-09-20T18:57:30.715 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.715 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.716 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:57:30.716 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.875806+0000", 2024-09-20T18:57:30.716 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6000000000000001e-06 2024-09-20T18:57:30.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.716 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.716 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:57:30.716 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.875812+0000", 2024-09-20T18:57:30.716 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3290000000000001e-06 2024-09-20T18:57:30.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.716 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.716 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:57:30.716 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.875817+0000", 2024-09-20T18:57:30.716 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0969999999999996e-06 2024-09-20T18:57:30.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.716 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.717 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:30.717 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.875977+0000", 2024-09-20T18:57:30.717 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015977400000000001 2024-09-20T18:57:30.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.717 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.717 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:57:30.717 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.875985+0000", 2024-09-20T18:57:30.717 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9019999999999999e-06 2024-09-20T18:57:30.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.717 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.717 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:30.717 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.700308+0000", 2024-09-20T18:57:30.717 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.82432304000000001 2024-09-20T18:57:30.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.718 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.718 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:57:30.718 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.700431+0000", 2024-09-20T18:57:30.718 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012312099999999999 2024-09-20T18:57:30.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.718 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.718 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,5", 2024-09-20T18:57:30.718 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.700535+0000", 2024-09-20T18:57:30.718 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010469 2024-09-20T18:57:30.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.718 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.718 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:57:30.718 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.705256+0000", 2024-09-20T18:57:30.719 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0047210610000000004 2024-09-20T18:57:30.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.719 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.719 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:30.719 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.705309+0000", 2024-09-20T18:57:30.719 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2534000000000002e-05 2024-09-20T18:57:30.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.719 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.719 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:30.719 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.705365+0000", 2024-09-20T18:57:30.719 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5844999999999998e-05 2024-09-20T18:57:30.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.719 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.719 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:57:30.720 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.705415+0000", 2024-09-20T18:57:30.720 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0297000000000003e-05 2024-09-20T18:57:30.720 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.720 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.720 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:57:30.720 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.705456+0000", 2024-09-20T18:57:30.720 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0373999999999997e-05 2024-09-20T18:57:30.720 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:30.720 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:30.720 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:30.720 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.720 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.720 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:128165 8.3 8:ccd2385f:::benchmark_data_smithi083_97422_object12074:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e203)", 2024-09-20T18:57:30.721 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:24.912336+0000", 2024-09-20T18:57:30.721 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 65.770327068, 2024-09-20T18:57:30.721 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.79332345500000001, 2024-09-20T18:57:30.721 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:57:30.721 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:57:30.721 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:57:30.721 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:57:30.721 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:57:30.721 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:57:30.721 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 128165 2024-09-20T18:57:30.721 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.721 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:57:30.721 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.721 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:57:30.721 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.912336+0000", 2024-09-20T18:57:30.722 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:30.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.722 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.722 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:57:30.722 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.912336+0000", 2024-09-20T18:57:30.722 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:30.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.722 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.722 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:57:30.722 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.912337+0000", 2024-09-20T18:57:30.722 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.466e-06 2024-09-20T18:57:30.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.722 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.722 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:57:30.723 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.912342+0000", 2024-09-20T18:57:30.723 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4330000000000003e-06 2024-09-20T18:57:30.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.723 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.723 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:57:30.723 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.912354+0000", 2024-09-20T18:57:30.723 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1620999999999999e-05 2024-09-20T18:57:30.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.723 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.723 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:57:30.723 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.912362+0000", 2024-09-20T18:57:30.723 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.6320000000000008e-06 2024-09-20T18:57:30.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.724 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.724 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:30.724 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.912419+0000", 2024-09-20T18:57:30.724 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7051e-05 2024-09-20T18:57:30.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.724 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.724 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:57:30.724 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.912428+0000", 2024-09-20T18:57:30.724 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3749999999999992e-06 2024-09-20T18:57:30.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.724 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.724 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:30.724 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.702684+0000", 2024-09-20T18:57:30.724 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.79025554899999995 2024-09-20T18:57:30.725 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.725 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.725 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:57:30.725 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.702784+0000", 2024-09-20T18:57:30.725 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000100629 2024-09-20T18:57:30.725 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.725 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.725 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,5", 2024-09-20T18:57:30.725 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.702908+0000", 2024-09-20T18:57:30.725 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012386 2024-09-20T18:57:30.725 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.725 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.725 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:57:30.725 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.705262+0000", 2024-09-20T18:57:30.725 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0023542070000000001 2024-09-20T18:57:30.726 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.726 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.726 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:30.726 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.705502+0000", 2024-09-20T18:57:30.726 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000239693 2024-09-20T18:57:30.726 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.726 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.726 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:30.726 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.705546+0000", 2024-09-20T18:57:30.726 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4231999999999999e-05 2024-09-20T18:57:30.726 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.726 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.726 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:57:30.726 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.705594+0000", 2024-09-20T18:57:30.727 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7484999999999997e-05 2024-09-20T18:57:30.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.727 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.727 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:57:30.727 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.705659+0000", 2024-09-20T18:57:30.727 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5222000000000007e-05 2024-09-20T18:57:30.727 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:30.727 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:30.727 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:30.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.727 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.727 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:128167 8.3 8:c28a9a26:::benchmark_data_smithi083_97422_object12076:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e203)", 2024-09-20T18:57:30.727 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:24.912455+0000", 2024-09-20T18:57:30.728 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 65.770207945999999, 2024-09-20T18:57:30.728 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.80018460800000002, 2024-09-20T18:57:30.728 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:57:30.728 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:57:30.728 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:57:30.728 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:57:30.728 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:57:30.728 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:57:30.728 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 128167 2024-09-20T18:57:30.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.728 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:57:30.728 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.728 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:57:30.728 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.912455+0000", 2024-09-20T18:57:30.729 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:30.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.729 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.729 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:57:30.729 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.912455+0000", 2024-09-20T18:57:30.729 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:30.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.729 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.729 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:57:30.729 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.912456+0000", 2024-09-20T18:57:30.729 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6500000000000008e-07 2024-09-20T18:57:30.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.729 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.729 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:57:30.730 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.912460+0000", 2024-09-20T18:57:30.730 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7029999999999997e-06 2024-09-20T18:57:30.730 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.730 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.730 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:57:30.730 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.912470+0000", 2024-09-20T18:57:30.730 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2469999999999997e-06 2024-09-20T18:57:30.730 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.730 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.730 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:57:30.730 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.912476+0000", 2024-09-20T18:57:30.730 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6059999999999999e-06 2024-09-20T18:57:30.730 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.731 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.731 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:30.731 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.912515+0000", 2024-09-20T18:57:30.731 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9229e-05 2024-09-20T18:57:30.731 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.731 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.731 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:57:30.731 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.912522+0000", 2024-09-20T18:57:30.731 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.866e-06 2024-09-20T18:57:30.731 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.731 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.731 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:30.731 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.705721+0000", 2024-09-20T18:57:30.731 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.79319880300000001 2024-09-20T18:57:30.732 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.732 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.732 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:57:30.732 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.705826+0000", 2024-09-20T18:57:30.732 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010511299999999999 2024-09-20T18:57:30.732 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.732 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.732 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,5", 2024-09-20T18:57:30.732 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.705947+0000", 2024-09-20T18:57:30.732 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000120405 2024-09-20T18:57:30.732 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.732 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.732 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:30.732 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.710142+0000", 2024-09-20T18:57:30.733 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0041950599999999996 2024-09-20T18:57:30.733 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.733 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.733 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:57:30.733 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.712464+0000", 2024-09-20T18:57:30.733 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0023219450000000002 2024-09-20T18:57:30.733 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.733 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.733 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:30.733 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.712570+0000", 2024-09-20T18:57:30.733 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106347 2024-09-20T18:57:30.733 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.733 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.734 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:57:30.734 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.712610+0000", 2024-09-20T18:57:30.734 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0302999999999997e-05 2024-09-20T18:57:30.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.734 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.734 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:57:30.734 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.712639+0000", 2024-09-20T18:57:30.734 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9016000000000001e-05 2024-09-20T18:57:30.734 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:30.734 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:30.734 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:30.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.734 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.734 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:128171 8.3 8:c9f7dd59:::benchmark_data_smithi083_97422_object12080:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e203)", 2024-09-20T18:57:30.735 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:24.917207+0000", 2024-09-20T18:57:30.735 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 65.765455333000006, 2024-09-20T18:57:30.735 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.80280542799999999, 2024-09-20T18:57:30.735 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:57:30.735 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:57:30.735 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:57:30.735 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:57:30.735 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:57:30.735 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:57:30.735 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 128171 2024-09-20T18:57:30.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.735 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:57:30.735 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.736 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:57:30.736 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.917207+0000", 2024-09-20T18:57:30.736 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:30.736 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.736 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.736 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:57:30.736 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.917207+0000", 2024-09-20T18:57:30.736 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:30.736 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.736 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.736 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:57:30.736 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.917209+0000", 2024-09-20T18:57:30.736 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5740000000000001e-06 2024-09-20T18:57:30.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.737 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.737 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:57:30.737 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.917215+0000", 2024-09-20T18:57:30.737 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2689999999999996e-06 2024-09-20T18:57:30.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.737 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.737 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:57:30.737 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.917229+0000", 2024-09-20T18:57:30.737 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3686000000000001e-05 2024-09-20T18:57:30.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.737 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.737 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:57:30.737 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.917236+0000", 2024-09-20T18:57:30.738 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1690000000000003e-06 2024-09-20T18:57:30.738 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.738 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.738 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:30.738 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.917348+0000", 2024-09-20T18:57:30.738 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000111573 2024-09-20T18:57:30.738 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.738 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.738 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:57:30.738 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.917357+0000", 2024-09-20T18:57:30.738 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6269999999999995e-06 2024-09-20T18:57:30.738 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.738 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.739 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:30.739 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.708009+0000", 2024-09-20T18:57:30.739 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.79065208899999995 2024-09-20T18:57:30.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.739 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.739 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:57:30.739 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.708102+0000", 2024-09-20T18:57:30.739 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3206000000000002e-05 2024-09-20T18:57:30.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.739 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.739 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,5", 2024-09-20T18:57:30.739 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.708240+0000", 2024-09-20T18:57:30.739 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013709700000000001 2024-09-20T18:57:30.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.740 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.740 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:30.740 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.712514+0000", 2024-09-20T18:57:30.740 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0042744580000000001 2024-09-20T18:57:30.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.740 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.740 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:57:30.740 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.712649+0000", 2024-09-20T18:57:30.740 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013514899999999999 2024-09-20T18:57:30.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.740 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.740 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:30.740 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.719853+0000", 2024-09-20T18:57:30.740 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0072033790000000002 2024-09-20T18:57:30.741 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.741 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.741 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:57:30.741 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.719980+0000", 2024-09-20T18:57:30.741 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000127341 2024-09-20T18:57:30.741 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.741 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.741 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:57:30.741 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.720013+0000", 2024-09-20T18:57:30.742 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2811000000000001e-05 2024-09-20T18:57:30.742 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:30.742 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:30.742 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:30.742 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.742 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.742 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:128174 8.3 8:c0429fd7:::benchmark_data_smithi083_97422_object12083:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e203)", 2024-09-20T18:57:30.742 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:24.921868+0000", 2024-09-20T18:57:30.742 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 65.760794101000002, 2024-09-20T18:57:30.742 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.79834304, 2024-09-20T18:57:30.742 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:57:30.742 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:57:30.742 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:57:30.742 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:57:30.743 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:57:30.743 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:57:30.743 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 128174 2024-09-20T18:57:30.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.743 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:57:30.743 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.743 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:57:30.743 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.921868+0000", 2024-09-20T18:57:30.743 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:30.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.743 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.743 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:57:30.743 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.921868+0000", 2024-09-20T18:57:30.743 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:30.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.744 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.744 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:57:30.744 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.921870+0000", 2024-09-20T18:57:30.744 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7740000000000001e-06 2024-09-20T18:57:30.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.744 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.744 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:57:30.744 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.921875+0000", 2024-09-20T18:57:30.744 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0839999999999999e-06 2024-09-20T18:57:30.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.744 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.744 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:57:30.744 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.921889+0000", 2024-09-20T18:57:30.744 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.393e-05 2024-09-20T18:57:30.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.745 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.745 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:57:30.745 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.921897+0000", 2024-09-20T18:57:30.745 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8639999999999996e-06 2024-09-20T18:57:30.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.745 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.745 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:30.745 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.921969+0000", 2024-09-20T18:57:30.745 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1894e-05 2024-09-20T18:57:30.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.745 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.746 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:57:30.746 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.921981+0000", 2024-09-20T18:57:30.746 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2194e-05 2024-09-20T18:57:30.746 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.746 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.746 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:30.746 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.710174+0000", 2024-09-20T18:57:30.746 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.78819257200000004 2024-09-20T18:57:30.746 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.746 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.746 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:57:30.746 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.710282+0000", 2024-09-20T18:57:30.746 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000108021 2024-09-20T18:57:30.747 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.747 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.747 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,5", 2024-09-20T18:57:30.747 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.710394+0000", 2024-09-20T18:57:30.747 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000112661 2024-09-20T18:57:30.747 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.747 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.747 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:30.747 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.712692+0000", 2024-09-20T18:57:30.747 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0022973109999999998 2024-09-20T18:57:30.747 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.747 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.747 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:57:30.747 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.717533+0000", 2024-09-20T18:57:30.747 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0048411959999999999 2024-09-20T18:57:30.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.748 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:30.748 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.720119+0000", 2024-09-20T18:57:30.748 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0025860269999999999 2024-09-20T18:57:30.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.748 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:57:30.748 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.720171+0000", 2024-09-20T18:57:30.748 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2120999999999998e-05 2024-09-20T18:57:30.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.748 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:57:30.748 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.720212+0000", 2024-09-20T18:57:30.749 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0391e-05 2024-09-20T18:57:30.749 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:30.749 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:30.749 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:30.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.749 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.749 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157429 8.7 8:ee8a5b93:::benchmark_data_smithi083_97422_object41338:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e212)", 2024-09-20T18:57:30.749 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.048178+0000", 2024-09-20T18:57:30.749 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.634484495999999, 2024-09-20T18:57:30.749 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.034527355, 2024-09-20T18:57:30.749 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:57:30.749 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:57:30.749 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-20T18:57:30.749 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:57:30.750 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:57:30.750 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:57:30.750 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157429 2024-09-20T18:57:30.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.750 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:57:30.750 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.750 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:57:30.750 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.048178+0000", 2024-09-20T18:57:30.750 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:30.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.750 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.750 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:57:30.750 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.048178+0000", 2024-09-20T18:57:30.750 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:30.751 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.751 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.751 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:57:30.751 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.048180+0000", 2024-09-20T18:57:30.751 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.618e-06 2024-09-20T18:57:30.751 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.751 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.751 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:57:30.751 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.048187+0000", 2024-09-20T18:57:30.751 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8240000000000002e-06 2024-09-20T18:57:30.751 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.751 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.751 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:57:30.751 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.048205+0000", 2024-09-20T18:57:30.752 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8717000000000001e-05 2024-09-20T18:57:30.752 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.752 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.752 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:57:30.752 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.048215+0000", 2024-09-20T18:57:30.752 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.022e-05 2024-09-20T18:57:30.752 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.752 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.752 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:30.752 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.048294+0000", 2024-09-20T18:57:30.752 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8088000000000002e-05 2024-09-20T18:57:30.752 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.752 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.752 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:57:30.753 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.048409+0000", 2024-09-20T18:57:30.753 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115146 2024-09-20T18:57:30.753 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.753 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.753 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3,6", 2024-09-20T18:57:30.753 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.048517+0000", 2024-09-20T18:57:30.753 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010874099999999999 2024-09-20T18:57:30.753 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.753 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.753 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:30.753 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.059282+0000", 2024-09-20T18:57:30.753 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010764078 2024-09-20T18:57:30.753 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.754 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.754 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:57:30.754 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.059297+0000", 2024-09-20T18:57:30.754 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5773e-05 2024-09-20T18:57:30.754 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.754 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.754 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:30.754 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.059343+0000", 2024-09-20T18:57:30.754 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5816000000000003e-05 2024-09-20T18:57:30.754 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.754 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.754 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:30.754 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.070249+0000", 2024-09-20T18:57:30.754 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010906381999999999 2024-09-20T18:57:30.754 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.755 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.755 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:57:30.755 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.070260+0000", 2024-09-20T18:57:30.756 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0404000000000001e-05 2024-09-20T18:57:30.756 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.756 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.756 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:30.756 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.082592+0000", 2024-09-20T18:57:30.756 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0123322640000001 2024-09-20T18:57:30.756 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.756 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.756 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:57:30.756 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.082705+0000", 2024-09-20T18:57:30.756 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011328399999999999 2024-09-20T18:57:30.756 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:30.756 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:30.757 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:30.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.757 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.757 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157435 8.7 8:e8101dbf:::benchmark_data_smithi083_97422_object41344:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e212)", 2024-09-20T18:57:30.757 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.053885+0000", 2024-09-20T18:57:30.757 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.628777266, 2024-09-20T18:57:30.757 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0294991419999999, 2024-09-20T18:57:30.757 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:57:30.757 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:57:30.757 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-20T18:57:30.757 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:57:30.757 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:57:30.757 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:57:30.757 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157435 2024-09-20T18:57:30.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.758 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:57:30.758 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.758 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:57:30.758 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053885+0000", 2024-09-20T18:57:30.758 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:30.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.758 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.758 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:57:30.758 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053885+0000", 2024-09-20T18:57:30.758 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:30.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.758 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.759 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:57:30.759 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053887+0000", 2024-09-20T18:57:30.759 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.68e-06 2024-09-20T18:57:30.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.759 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.759 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:57:30.759 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053894+0000", 2024-09-20T18:57:30.759 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5939999999999999e-06 2024-09-20T18:57:30.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.759 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.759 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:57:30.759 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053907+0000", 2024-09-20T18:57:30.759 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3599e-05 2024-09-20T18:57:30.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.760 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.760 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:57:30.760 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053916+0000", 2024-09-20T18:57:30.760 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2029999999999998e-06 2024-09-20T18:57:30.760 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.760 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.760 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:30.760 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.054004+0000", 2024-09-20T18:57:30.760 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8026000000000001e-05 2024-09-20T18:57:30.760 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.760 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.760 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:57:30.760 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.054123+0000", 2024-09-20T18:57:30.760 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000119046 2024-09-20T18:57:30.761 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.761 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.761 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3,6", 2024-09-20T18:57:30.761 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.056711+0000", 2024-09-20T18:57:30.761 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0025878049999999999 2024-09-20T18:57:30.761 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.761 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.761 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:57:30.761 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.059354+0000", 2024-09-20T18:57:30.761 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0026431969999999999 2024-09-20T18:57:30.761 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.761 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.761 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:30.761 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.070287+0000", 2024-09-20T18:57:30.762 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010932261 2024-09-20T18:57:30.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.762 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:57:30.762 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.070294+0000", 2024-09-20T18:57:30.762 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0029999999999997e-06 2024-09-20T18:57:30.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.762 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:30.762 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.083330+0000", 2024-09-20T18:57:30.762 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.013036592 2024-09-20T18:57:30.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.763 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:57:30.763 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.083384+0000", 2024-09-20T18:57:30.763 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4135999999999998e-05 2024-09-20T18:57:30.763 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:30.763 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:30.763 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:30.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.763 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.763 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157436 8.7 8:e2843ea0:::benchmark_data_smithi083_97422_object41345:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e212)", 2024-09-20T18:57:30.763 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.053939+0000", 2024-09-20T18:57:30.763 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.628723735000001, 2024-09-20T18:57:30.763 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.029517123, 2024-09-20T18:57:30.763 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:57:30.763 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:57:30.764 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-20T18:57:30.764 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:57:30.764 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:57:30.764 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:57:30.764 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157436 2024-09-20T18:57:30.764 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.764 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:57:30.764 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.764 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:57:30.764 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053939+0000", 2024-09-20T18:57:30.764 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:30.764 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.764 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.764 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:57:30.765 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053939+0000", 2024-09-20T18:57:30.765 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:30.765 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.765 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.765 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:57:30.765 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053939+0000", 2024-09-20T18:57:30.765 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9100000000000004e-07 2024-09-20T18:57:30.765 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.765 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.765 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:57:30.765 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053942+0000", 2024-09-20T18:57:30.765 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7980000000000001e-06 2024-09-20T18:57:30.765 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.765 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.766 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:57:30.766 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053949+0000", 2024-09-20T18:57:30.766 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8580000000000002e-06 2024-09-20T18:57:30.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.766 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.766 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:57:30.766 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053955+0000", 2024-09-20T18:57:30.766 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1449999999999996e-06 2024-09-20T18:57:30.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.766 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.766 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:30.766 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.060057+0000", 2024-09-20T18:57:30.766 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0061021729999999998 2024-09-20T18:57:30.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.767 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.767 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:57:30.767 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.060177+0000", 2024-09-20T18:57:30.767 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000120013 2024-09-20T18:57:30.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.767 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.767 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3,6", 2024-09-20T18:57:30.767 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.060294+0000", 2024-09-20T18:57:30.767 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000117056 2024-09-20T18:57:30.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.767 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.767 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:30.767 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.070317+0000", 2024-09-20T18:57:30.767 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010022778 2024-09-20T18:57:30.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.768 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.768 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:57:30.768 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.070324+0000", 2024-09-20T18:57:30.768 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4300000000000003e-06 2024-09-20T18:57:30.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.768 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.768 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:30.768 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.083413+0000", 2024-09-20T18:57:30.768 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0130890960000001 2024-09-20T18:57:30.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.768 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.768 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:57:30.768 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.083456+0000", 2024-09-20T18:57:30.769 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3185000000000001e-05 2024-09-20T18:57:30.769 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:30.769 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:30.769 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:30.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.769 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.769 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157449 8.7 8:e0d43d9e:::benchmark_data_smithi083_97422_object41358:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e213)", 2024-09-20T18:57:30.769 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.075612+0000", 2024-09-20T18:57:30.769 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.607050886, 2024-09-20T18:57:30.769 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.016308008, 2024-09-20T18:57:30.769 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:57:30.769 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:57:30.769 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:57:30.769 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:57:30.770 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:57:30.770 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:57:30.770 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157449 2024-09-20T18:57:30.770 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.770 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:57:30.770 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.770 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:57:30.770 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.075612+0000", 2024-09-20T18:57:30.770 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:30.770 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.770 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.770 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:57:30.770 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.075612+0000", 2024-09-20T18:57:30.770 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:30.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.771 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:57:30.771 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.075613+0000", 2024-09-20T18:57:30.771 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1570000000000001e-06 2024-09-20T18:57:30.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.771 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:57:30.771 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.075618+0000", 2024-09-20T18:57:30.771 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7899999999999999e-06 2024-09-20T18:57:30.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.771 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:57:30.771 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.075627+0000", 2024-09-20T18:57:30.772 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2569999999999996e-06 2024-09-20T18:57:30.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.772 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.772 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:57:30.772 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.075662+0000", 2024-09-20T18:57:30.772 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5439999999999999e-05 2024-09-20T18:57:30.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.772 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.772 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:30.772 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.078479+0000", 2024-09-20T18:57:30.772 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0028168910000000002 2024-09-20T18:57:30.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.772 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.773 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:57:30.773 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.078488+0000", 2024-09-20T18:57:30.773 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4069999999999993e-06 2024-09-20T18:57:30.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.773 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.773 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:30.773 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.083476+0000", 2024-09-20T18:57:30.773 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.004987893 2024-09-20T18:57:30.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.773 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.773 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:57:30.773 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.083567+0000", 2024-09-20T18:57:30.773 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1598e-05 2024-09-20T18:57:30.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.774 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3", 2024-09-20T18:57:30.774 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.083675+0000", 2024-09-20T18:57:30.774 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000107701 2024-09-20T18:57:30.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.774 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:30.774 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.091482+0000", 2024-09-20T18:57:30.774 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0078073639999999998 2024-09-20T18:57:30.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.774 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:57:30.774 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.091668+0000", 2024-09-20T18:57:30.775 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018606100000000001 2024-09-20T18:57:30.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.775 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:30.775 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.091718+0000", 2024-09-20T18:57:30.775 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9830000000000002e-05 2024-09-20T18:57:30.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.775 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:57:30.775 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.091878+0000", 2024-09-20T18:57:30.775 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000160166 2024-09-20T18:57:30.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.775 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:57:30.776 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.091920+0000", 2024-09-20T18:57:30.776 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1452999999999998e-05 2024-09-20T18:57:30.776 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:30.776 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:30.776 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:30.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.776 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.776 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157451 8.7 8:ea555b5d:::benchmark_data_smithi083_97422_object41360:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e213)", 2024-09-20T18:57:30.776 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.080725+0000", 2024-09-20T18:57:30.776 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.601937919000001, 2024-09-20T18:57:30.776 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0181950740000001, 2024-09-20T18:57:30.776 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:57:30.776 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:57:30.776 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:57:30.777 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:57:30.777 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:57:30.777 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:57:30.777 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157451 2024-09-20T18:57:30.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.777 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:57:30.777 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.777 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:57:30.777 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.080725+0000", 2024-09-20T18:57:30.777 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:30.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.777 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.777 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:57:30.777 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.080725+0000", 2024-09-20T18:57:30.778 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:30.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.778 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.778 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:57:30.778 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.080726+0000", 2024-09-20T18:57:30.778 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3459999999999999e-06 2024-09-20T18:57:30.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.778 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.778 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:57:30.778 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.080731+0000", 2024-09-20T18:57:30.778 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3619999999999997e-06 2024-09-20T18:57:30.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.778 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.778 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:57:30.778 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.080742+0000", 2024-09-20T18:57:30.779 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0302000000000001e-05 2024-09-20T18:57:30.779 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.779 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.779 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:57:30.779 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.080757+0000", 2024-09-20T18:57:30.779 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4955e-05 2024-09-20T18:57:30.779 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.779 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.779 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:30.779 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.080819+0000", 2024-09-20T18:57:30.779 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2310999999999993e-05 2024-09-20T18:57:30.779 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.779 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.779 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:57:30.780 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.080826+0000", 2024-09-20T18:57:30.780 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1069999999999999e-06 2024-09-20T18:57:30.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.780 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.780 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:30.780 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.085855+0000", 2024-09-20T18:57:30.780 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0050288679999999 2024-09-20T18:57:30.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.780 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.780 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:57:30.780 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.085947+0000", 2024-09-20T18:57:30.780 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1911999999999997e-05 2024-09-20T18:57:30.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.780 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.781 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3", 2024-09-20T18:57:30.781 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.086042+0000", 2024-09-20T18:57:30.781 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5234000000000003e-05 2024-09-20T18:57:30.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.781 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:57:30.781 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.091673+0000", 2024-09-20T18:57:30.781 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0056306079999999996 2024-09-20T18:57:30.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.781 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:30.781 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.098793+0000", 2024-09-20T18:57:30.781 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.007120637 2024-09-20T18:57:30.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.782 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:30.782 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.098828+0000", 2024-09-20T18:57:30.782 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4405e-05 2024-09-20T18:57:30.782 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.782 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.782 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:57:30.782 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.098882+0000", 2024-09-20T18:57:30.782 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4481000000000002e-05 2024-09-20T18:57:30.782 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.782 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.782 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:57:30.782 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.098920+0000", 2024-09-20T18:57:30.782 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7546000000000003e-05 2024-09-20T18:57:30.782 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:30.783 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:30.783 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:30.783 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.783 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.783 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157481 8.7 8:ebfa0664:::benchmark_data_smithi083_97422_object41390:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e213)", 2024-09-20T18:57:30.783 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.116650+0000", 2024-09-20T18:57:30.783 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.566012405999999, 2024-09-20T18:57:30.783 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.98244276600000002, 2024-09-20T18:57:30.783 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:57:30.783 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:57:30.783 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:57:30.783 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:57:30.783 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:57:30.783 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:57:30.784 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157481 2024-09-20T18:57:30.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.784 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:57:30.784 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.784 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:57:30.784 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.116650+0000", 2024-09-20T18:57:30.784 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:30.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.784 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.784 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:57:30.784 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.116650+0000", 2024-09-20T18:57:30.784 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:30.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.784 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.784 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:57:30.785 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.116652+0000", 2024-09-20T18:57:30.785 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5680000000000001e-06 2024-09-20T18:57:30.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.785 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:57:30.785 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.116658+0000", 2024-09-20T18:57:30.785 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4309999999999999e-06 2024-09-20T18:57:30.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.785 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:57:30.785 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.116671+0000", 2024-09-20T18:57:30.785 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2384e-05 2024-09-20T18:57:30.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.786 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:57:30.786 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.116678+0000", 2024-09-20T18:57:30.786 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8690000000000004e-06 2024-09-20T18:57:30.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.786 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.786 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:30.786 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.116734+0000", 2024-09-20T18:57:30.786 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6041000000000001e-05 2024-09-20T18:57:30.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.786 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.786 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:57:30.786 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.116742+0000", 2024-09-20T18:57:30.786 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8189999999999992e-06 2024-09-20T18:57:30.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.787 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.787 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:30.787 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.088569+0000", 2024-09-20T18:57:30.787 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97182719100000003 2024-09-20T18:57:30.787 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.787 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.787 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:57:30.787 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.088666+0000", 2024-09-20T18:57:30.787 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6897000000000004e-05 2024-09-20T18:57:30.787 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.787 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.787 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3", 2024-09-20T18:57:30.787 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.088770+0000", 2024-09-20T18:57:30.788 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010320299999999999 2024-09-20T18:57:30.788 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.788 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.788 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:57:30.788 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.091677+0000", 2024-09-20T18:57:30.788 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0029076900000000001 2024-09-20T18:57:30.788 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.788 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.788 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:30.788 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.098961+0000", 2024-09-20T18:57:30.788 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0072835080000000002 2024-09-20T18:57:30.788 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.788 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.788 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:30.789 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.099000+0000", 2024-09-20T18:57:30.789 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.892e-05 2024-09-20T18:57:30.789 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.789 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.789 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:57:30.789 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.099055+0000", 2024-09-20T18:57:30.789 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5554999999999999e-05 2024-09-20T18:57:30.789 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.789 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.789 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:57:30.789 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.099093+0000", 2024-09-20T18:57:30.789 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.769e-05 2024-09-20T18:57:30.789 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:30.789 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:30.790 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:30.790 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.790 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.790 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157496 8.7 8:efb55f87:::benchmark_data_smithi083_97422_object41405:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e213)", 2024-09-20T18:57:30.790 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.130402+0000", 2024-09-20T18:57:30.790 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.552260688, 2024-09-20T18:57:30.790 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.99286820799999997, 2024-09-20T18:57:30.790 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:57:30.790 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:57:30.790 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:57:30.790 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:57:30.790 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:57:30.790 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:57:30.790 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157496 2024-09-20T18:57:30.790 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.791 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:57:30.791 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.791 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:57:30.791 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.130402+0000", 2024-09-20T18:57:30.791 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:30.791 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.791 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.791 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:57:30.791 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.130402+0000", 2024-09-20T18:57:30.791 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:30.791 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.791 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.791 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:57:30.791 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.130404+0000", 2024-09-20T18:57:30.792 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.748e-06 2024-09-20T18:57:30.792 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.792 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.792 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:57:30.792 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.130410+0000", 2024-09-20T18:57:30.792 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2720000000000003e-06 2024-09-20T18:57:30.792 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.792 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.792 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:57:30.792 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.130426+0000", 2024-09-20T18:57:30.792 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6557000000000001e-05 2024-09-20T18:57:30.792 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.792 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.792 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:57:30.793 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.130436+0000", 2024-09-20T18:57:30.793 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4320000000000005e-06 2024-09-20T18:57:30.793 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.793 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.793 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:30.793 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.130499+0000", 2024-09-20T18:57:30.793 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3149000000000001e-05 2024-09-20T18:57:30.793 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.793 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.793 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:57:30.793 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.130508+0000", 2024-09-20T18:57:30.793 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1619999999999997e-06 2024-09-20T18:57:30.793 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.793 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.794 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:30.794 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.091952+0000", 2024-09-20T18:57:30.794 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.96144401999999995 2024-09-20T18:57:30.794 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.794 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.794 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:57:30.794 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.092057+0000", 2024-09-20T18:57:30.794 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000104268 2024-09-20T18:57:30.794 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.794 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.794 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3", 2024-09-20T18:57:30.794 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.092163+0000", 2024-09-20T18:57:30.794 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106545 2024-09-20T18:57:30.794 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.795 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.795 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:30.795 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.099133+0000", 2024-09-20T18:57:30.795 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0069702990000000001 2024-09-20T18:57:30.795 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.795 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.795 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:30.795 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.104449+0000", 2024-09-20T18:57:30.795 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0053152920000000001 2024-09-20T18:57:30.795 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.795 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.795 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:57:30.795 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.123091+0000", 2024-09-20T18:57:30.795 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.018642757999999999 2024-09-20T18:57:30.796 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.796 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.796 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:57:30.796 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.123225+0000", 2024-09-20T18:57:30.796 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000133881 2024-09-20T18:57:30.796 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.796 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.796 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:57:30.796 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.123270+0000", 2024-09-20T18:57:30.796 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4824999999999999e-05 2024-09-20T18:57:30.796 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:30.796 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:30.796 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:30.796 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.797 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.797 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:30.798 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157528 8.7 8:e4768593:::benchmark_data_smithi083_97422_object41437:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e213)", 2024-09-20T18:57:30.798 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.165392+0000", 2024-09-20T18:57:30.798 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.51727039, 2024-09-20T18:57:30.798 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.958042846, 2024-09-20T18:57:30.798 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:57:30.798 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:57:30.798 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:57:30.798 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:57:30.798 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:57:30.798 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:57:30.799 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157528 2024-09-20T18:57:30.799 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.799 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:57:30.799 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.799 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:57:30.799 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.165392+0000", 2024-09-20T18:57:30.799 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:30.799 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.799 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.799 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:57:30.799 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.165392+0000", 2024-09-20T18:57:30.799 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:30.799 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.800 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.800 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:57:30.800 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.165394+0000", 2024-09-20T18:57:30.800 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.77e-06 2024-09-20T18:57:30.800 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.800 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.800 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:57:30.800 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.165400+0000", 2024-09-20T18:57:30.800 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2099999999999998e-06 2024-09-20T18:57:30.800 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.800 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.801 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:57:30.801 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.165419+0000", 2024-09-20T18:57:30.801 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9046999999999999e-05 2024-09-20T18:57:30.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.801 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.801 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:57:30.801 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.165428+0000", 2024-09-20T18:57:30.801 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1030000000000008e-06 2024-09-20T18:57:30.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.801 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.801 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:30.801 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.166013+0000", 2024-09-20T18:57:30.801 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000584868 2024-09-20T18:57:30.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.802 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.802 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:57:30.802 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.166022+0000", 2024-09-20T18:57:30.802 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7090000000000008e-06 2024-09-20T18:57:30.802 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.802 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.802 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:30.802 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.095256+0000", 2024-09-20T18:57:30.802 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.92923407700000005 2024-09-20T18:57:30.802 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.802 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.802 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:57:30.802 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.095349+0000", 2024-09-20T18:57:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2724000000000002e-05 2024-09-20T18:57:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3", 2024-09-20T18:57:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.095688+0000", 2024-09-20T18:57:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000339379 2024-09-20T18:57:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.104489+0000", 2024-09-20T18:57:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0088013359999999999 2024-09-20T18:57:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.803 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.804 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:30.804 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.106738+0000", 2024-09-20T18:57:30.804 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0022487029999999999 2024-09-20T18:57:30.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.804 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.804 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:57:30.804 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.123280+0000", 2024-09-20T18:57:30.804 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.016541613 2024-09-20T18:57:30.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.804 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.804 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:57:30.804 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.123398+0000", 2024-09-20T18:57:30.804 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011812299999999999 2024-09-20T18:57:30.805 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.805 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.805 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:57:30.805 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.123435+0000", 2024-09-20T18:57:30.805 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7184000000000002e-05 2024-09-20T18:57:30.805 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:30.805 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:30.805 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:30.805 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.805 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.805 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157535 8.7 8:ed2a8870:::benchmark_data_smithi083_97422_object41444:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e213)", 2024-09-20T18:57:30.805 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.173838+0000", 2024-09-20T18:57:30.805 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.508824778000001, 2024-09-20T18:57:30.805 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.95335319799999996, 2024-09-20T18:57:30.806 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:57:30.806 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:57:30.806 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:57:30.806 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:57:30.806 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:57:30.806 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:57:30.806 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157535 2024-09-20T18:57:30.806 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.806 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:57:30.806 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.806 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:57:30.806 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.173838+0000", 2024-09-20T18:57:30.806 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:30.806 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.807 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.807 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:57:30.807 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.173838+0000", 2024-09-20T18:57:30.807 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:30.807 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.807 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.807 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:57:30.807 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.173839+0000", 2024-09-20T18:57:30.807 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5340000000000001e-06 2024-09-20T18:57:30.807 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.807 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.807 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:57:30.807 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.173846+0000", 2024-09-20T18:57:30.808 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2910000000000004e-06 2024-09-20T18:57:30.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.808 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.808 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:57:30.808 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.173867+0000", 2024-09-20T18:57:30.808 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0928e-05 2024-09-20T18:57:30.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.808 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.808 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:57:30.808 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.173874+0000", 2024-09-20T18:57:30.808 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.6990000000000003e-06 2024-09-20T18:57:30.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.808 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.808 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:30.808 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.173935+0000", 2024-09-20T18:57:30.809 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0899000000000001e-05 2024-09-20T18:57:30.809 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.809 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.809 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:57:30.809 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.173944+0000", 2024-09-20T18:57:30.809 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5369999999999997e-06 2024-09-20T18:57:30.809 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.809 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.809 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:30.809 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.099161+0000", 2024-09-20T18:57:30.809 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.92521724299999997 2024-09-20T18:57:30.809 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.809 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.809 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:57:30.810 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.099277+0000", 2024-09-20T18:57:30.810 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115689 2024-09-20T18:57:30.810 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.810 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.810 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3", 2024-09-20T18:57:30.810 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.099390+0000", 2024-09-20T18:57:30.810 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011385199999999999 2024-09-20T18:57:30.810 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.810 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.810 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:30.810 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.106780+0000", 2024-09-20T18:57:30.810 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.007389367 2024-09-20T18:57:30.810 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.811 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.811 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:30.811 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.114134+0000", 2024-09-20T18:57:30.811 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0073541429999999996 2024-09-20T18:57:30.811 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.811 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.811 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:57:30.811 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.123441+0000", 2024-09-20T18:57:30.811 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0093066620000000003 2024-09-20T18:57:30.811 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.811 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.811 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:57:30.811 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.127133+0000", 2024-09-20T18:57:30.812 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0036927879999999998 2024-09-20T18:57:30.812 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.812 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.812 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:57:30.812 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.127191+0000", 2024-09-20T18:57:30.812 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7565999999999997e-05 2024-09-20T18:57:30.812 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:30.812 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:30.812 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:30.812 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.812 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.812 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157545 8.7 8:e3edf173:::benchmark_data_smithi083_97422_object41454:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e213)", 2024-09-20T18:57:30.812 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.188692+0000", 2024-09-20T18:57:30.813 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.493970173000001, 2024-09-20T18:57:30.813 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.93865532500000004, 2024-09-20T18:57:30.813 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:57:30.813 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:57:30.813 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:57:30.813 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:57:30.813 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:57:30.813 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:57:30.813 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157545 2024-09-20T18:57:30.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.813 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:57:30.813 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.813 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:57:30.813 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.188692+0000", 2024-09-20T18:57:30.814 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:30.814 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.814 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.814 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:57:30.814 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.188692+0000", 2024-09-20T18:57:30.814 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:30.814 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.814 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.814 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:57:30.814 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.188694+0000", 2024-09-20T18:57:30.814 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.869e-06 2024-09-20T18:57:30.814 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.815 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.815 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:57:30.815 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.188701+0000", 2024-09-20T18:57:30.815 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9829999999999999e-06 2024-09-20T18:57:30.815 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.815 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.815 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:57:30.815 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.188718+0000", 2024-09-20T18:57:30.815 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6750999999999999e-05 2024-09-20T18:57:30.815 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.815 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.815 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:57:30.815 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.188727+0000", 2024-09-20T18:57:30.815 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6449999999999994e-06 2024-09-20T18:57:30.816 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.816 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.816 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:30.816 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.188800+0000", 2024-09-20T18:57:30.816 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3022999999999995e-05 2024-09-20T18:57:30.816 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.816 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.816 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:57:30.816 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.188810+0000", 2024-09-20T18:57:30.816 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0019e-05 2024-09-20T18:57:30.816 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.816 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.816 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:30.816 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.101615+0000", 2024-09-20T18:57:30.817 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.91280537699999997 2024-09-20T18:57:30.817 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.817 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.817 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:57:30.817 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.101713+0000", 2024-09-20T18:57:30.817 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7870999999999999e-05 2024-09-20T18:57:30.817 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.817 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.817 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3", 2024-09-20T18:57:30.817 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.101823+0000", 2024-09-20T18:57:30.817 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010982 2024-09-20T18:57:30.817 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.817 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.817 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:30.818 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.106819+0000", 2024-09-20T18:57:30.818 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00499592 2024-09-20T18:57:30.818 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.818 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.818 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:30.818 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.114173+0000", 2024-09-20T18:57:30.818 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0073544980000000001 2024-09-20T18:57:30.818 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.818 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.818 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:57:30.818 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.127205+0000", 2024-09-20T18:57:30.818 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.013031839 2024-09-20T18:57:30.818 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.819 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.819 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:57:30.819 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.127309+0000", 2024-09-20T18:57:30.819 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010435899999999999 2024-09-20T18:57:30.819 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.819 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.819 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:57:30.819 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.127348+0000", 2024-09-20T18:57:30.819 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8350999999999999e-05 2024-09-20T18:57:30.819 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:30.819 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:30.819 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:30.819 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.819 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.820 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157575 8.7 8:e03ab630:::benchmark_data_smithi083_97422_object41484:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e213)", 2024-09-20T18:57:30.820 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.236943+0000", 2024-09-20T18:57:30.820 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.445719556, 2024-09-20T18:57:30.820 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.89067574599999999, 2024-09-20T18:57:30.820 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:57:30.820 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:57:30.820 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:57:30.820 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:57:30.820 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:57:30.820 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:57:30.820 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157575 2024-09-20T18:57:30.820 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.820 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:57:30.820 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.821 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:57:30.821 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.236943+0000", 2024-09-20T18:57:30.821 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:30.821 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.821 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.821 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:57:30.821 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.236943+0000", 2024-09-20T18:57:30.821 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:30.821 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.821 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.821 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:57:30.821 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.236945+0000", 2024-09-20T18:57:30.821 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.584e-06 2024-09-20T18:57:30.822 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.822 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.822 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:57:30.822 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.236949+0000", 2024-09-20T18:57:30.822 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.673e-06 2024-09-20T18:57:30.822 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.822 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.822 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:57:30.822 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.236963+0000", 2024-09-20T18:57:30.822 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4209000000000001e-05 2024-09-20T18:57:30.822 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.822 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.822 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:57:30.822 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.236970+0000", 2024-09-20T18:57:30.823 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0029999999999997e-06 2024-09-20T18:57:30.823 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.823 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.823 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:30.823 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.237078+0000", 2024-09-20T18:57:30.823 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000107026 2024-09-20T18:57:30.823 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.823 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.823 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:57:30.823 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.237085+0000", 2024-09-20T18:57:30.823 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4669999999999998e-06 2024-09-20T18:57:30.823 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.823 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.823 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:30.824 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.106857+0000", 2024-09-20T18:57:30.824 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.86977158899999996 2024-09-20T18:57:30.824 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.824 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.824 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:57:30.824 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.106967+0000", 2024-09-20T18:57:30.824 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011026700000000001 2024-09-20T18:57:30.824 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.824 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.824 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3", 2024-09-20T18:57:30.824 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.107090+0000", 2024-09-20T18:57:30.824 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012295399999999999 2024-09-20T18:57:30.824 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.825 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.825 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:57:30.825 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.127396+0000", 2024-09-20T18:57:30.825 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.020306026000000001 2024-09-20T18:57:30.825 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.825 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.825 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:30.825 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.127459+0000", 2024-09-20T18:57:30.825 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3322000000000001e-05 2024-09-20T18:57:30.825 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.825 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.825 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:30.825 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.127496+0000", 2024-09-20T18:57:30.825 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7156999999999997e-05 2024-09-20T18:57:30.825 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.826 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.826 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:57:30.826 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.127581+0000", 2024-09-20T18:57:30.826 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4679999999999996e-05 2024-09-20T18:57:30.826 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.826 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.826 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:57:30.826 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:57:30 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:57:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:57:30.827 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.127619+0000", 2024-09-20T18:57:30.827 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7789e-05 2024-09-20T18:57:30.827 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:30.827 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:30.827 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:30.827 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.827 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.827 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157577 8.7 8:ed15ec18:::benchmark_data_smithi083_97422_object41486:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e213)", 2024-09-20T18:57:30.827 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.239083+0000", 2024-09-20T18:57:30.827 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.443579389, 2024-09-20T18:57:30.828 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88873236799999999, 2024-09-20T18:57:30.828 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:57:30.828 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:57:30.828 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:57:30.828 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:57:30.828 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:57:30.828 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:57:30.828 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157577 2024-09-20T18:57:30.828 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.828 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:57:30.828 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.828 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:57:30.828 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.239083+0000", 2024-09-20T18:57:30.829 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:30.829 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.829 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.829 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:57:30.829 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.239083+0000", 2024-09-20T18:57:30.829 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:30.829 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.829 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.829 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:57:30.829 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.239084+0000", 2024-09-20T18:57:30.829 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5599999999999995e-07 2024-09-20T18:57:30.829 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.829 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.830 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:57:30.830 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.239087+0000", 2024-09-20T18:57:30.830 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8990000000000001e-06 2024-09-20T18:57:30.830 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.830 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.830 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:57:30.830 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.239094+0000", 2024-09-20T18:57:30.830 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5859999999999999e-06 2024-09-20T18:57:30.830 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.830 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.830 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:57:30.830 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.239100+0000", 2024-09-20T18:57:30.830 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7490000000000004e-06 2024-09-20T18:57:30.831 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.831 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.831 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:30.831 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.239149+0000", 2024-09-20T18:57:30.831 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9202000000000001e-05 2024-09-20T18:57:30.831 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.831 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.831 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:57:30.831 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.239157+0000", 2024-09-20T18:57:30.831 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.276e-06 2024-09-20T18:57:30.831 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.831 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.831 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:30.831 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.109327+0000", 2024-09-20T18:57:30.832 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87016914899999998 2024-09-20T18:57:30.832 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.832 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.832 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:57:30.832 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.109435+0000", 2024-09-20T18:57:30.832 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010798099999999999 2024-09-20T18:57:30.832 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.832 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.832 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3", 2024-09-20T18:57:30.832 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.109546+0000", 2024-09-20T18:57:30.832 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011176299999999999 2024-09-20T18:57:30.832 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.832 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.832 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:57:30.833 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.127401+0000", 2024-09-20T18:57:30.833 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.017854716999999999 2024-09-20T18:57:30.833 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.833 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.833 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:30.833 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.127661+0000", 2024-09-20T18:57:30.833 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00026038800000000001 2024-09-20T18:57:30.833 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.833 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.833 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:30.833 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.127697+0000", 2024-09-20T18:57:30.833 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5666e-05 2024-09-20T18:57:30.833 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.833 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.834 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:57:30.834 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.127779+0000", 2024-09-20T18:57:30.834 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.1860000000000006e-05 2024-09-20T18:57:30.834 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.834 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.834 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:57:30.834 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.127816+0000", 2024-09-20T18:57:30.834 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6575999999999997e-05 2024-09-20T18:57:30.834 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:30.834 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:30.834 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:30.834 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.834 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.835 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157637 8.7 8:ee232675:::benchmark_data_smithi083_97422_object41546:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e213)", 2024-09-20T18:57:30.835 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.327404+0000", 2024-09-20T18:57:30.835 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 31.355258773999999, 2024-09-20T18:57:30.835 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.80573648099999995, 2024-09-20T18:57:30.835 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:57:30.835 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:57:30.835 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:57:30.835 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:57:30.835 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:57:30.835 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:57:30.835 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157637 2024-09-20T18:57:30.835 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.835 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:57:30.835 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.836 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:57:30.836 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.327404+0000", 2024-09-20T18:57:30.836 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:30.836 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.836 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.836 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:57:30.836 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.327404+0000", 2024-09-20T18:57:30.836 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:30.836 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.836 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.836 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:57:30.836 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.327405+0000", 2024-09-20T18:57:30.836 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3179999999999999e-06 2024-09-20T18:57:30.836 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.837 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.837 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:57:30.837 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.327410+0000", 2024-09-20T18:57:30.837 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.062e-06 2024-09-20T18:57:30.837 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.837 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.837 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:57:30.837 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.327422+0000", 2024-09-20T18:57:30.837 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.167e-05 2024-09-20T18:57:30.837 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.837 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.837 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:57:30.837 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.327429+0000", 2024-09-20T18:57:30.837 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3520000000000001e-06 2024-09-20T18:57:30.837 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.838 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.838 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:30.838 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.327497+0000", 2024-09-20T18:57:30.838 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7898000000000006e-05 2024-09-20T18:57:30.838 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.838 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.838 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:57:30.838 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.327508+0000", 2024-09-20T18:57:30.838 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1209999999999999e-05 2024-09-20T18:57:30.838 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.838 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.838 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:30.838 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.111708+0000", 2024-09-20T18:57:30.839 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.78419992000000005 2024-09-20T18:57:30.839 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.839 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.839 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:57:30.839 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.111816+0000", 2024-09-20T18:57:30.839 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010796100000000001 2024-09-20T18:57:30.839 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.839 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.839 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3", 2024-09-20T18:57:30.839 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.111932+0000", 2024-09-20T18:57:30.839 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011589199999999999 2024-09-20T18:57:30.839 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.839 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.839 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:57:30.840 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.127406+0000", 2024-09-20T18:57:30.840 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.01547343 2024-09-20T18:57:30.840 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.840 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.840 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:30.840 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.127869+0000", 2024-09-20T18:57:30.840 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00046379199999999998 2024-09-20T18:57:30.840 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.840 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.840 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:30.840 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.132820+0000", 2024-09-20T18:57:30.840 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049506630000000001 2024-09-20T18:57:30.840 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.840 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.841 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:57:30.841 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.133087+0000", 2024-09-20T18:57:30.841 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00026739899999999998 2024-09-20T18:57:30.841 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:30.841 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:30.841 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:57:30.841 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.133140+0000", 2024-09-20T18:57:30.841 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2914000000000002e-05 2024-09-20T18:57:30.841 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:30.841 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:30.841 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:30.841 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:30.841 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:30.842 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T18:57:30.864 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:30.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:30.865 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:30.966 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:30.993 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:30.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:30.994 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:31.049 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:30 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 1845/151920 objects degraded (1.214%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T18:57:31.049 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:30 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:57:31.049 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:30 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:57:31.049 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:31 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 1845/151920 objects degraded (1.214%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T18:57:31.049 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:31 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:57:31.049 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:31 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:57:31.095 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:57:31.117 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:31.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:31.118 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:57:31.219 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:31.259 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:31.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:31.260 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:31.269 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:30 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 1845/151920 objects degraded (1.214%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T18:57:31.269 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:30 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:57:31.269 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:30 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:57:31.360 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:31.374 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T18:57:31.385 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:31.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:31.386 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:31.486 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:31.550 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:31.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:31.551 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:31.652 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:31.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:31.703 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:31.703 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:31.737 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:57:31.737 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"1ed5a50a-7780-11ef-baf5-efdc52797490","health":{"status":"HEALTH_WARN","checks":{"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"noscrub flag(s) set","count":7},"muted":false},"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 1845/151920 objects degraded (1.214%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":713,"monmap":{"epoch":3,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":223,"num_osds":8,"num_up_osds":8,"osd_up_since":1726858174,"num_in_osds":7,"osd_in_since":1726858527,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":147},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":148,"num_pools":7,"num_objects":50640,"data_bytes":3306232371,"bytes_used":11919147008,"bytes_avail":660046102528,"bytes_total":671965249536,"degraded_objects":1845,"degraded_total":151920,"degraded_ratio":0.012144549763033176,"misplaced_objects":1054,"misplaced_total":151920,"misplaced_ratio":0.0069378620326487625,"recovering_objects_per_sec":58,"recovering_bytes_per_sec":3844961,"recovering_keys_per_sec":0,"num_objects_recovered":587,"num_bytes_recovered":38469632,"num_keys_recovered":0,"read_bytes_sec":1023,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":906},"fsmap":{"epoch":1,"btime":"2024-09-20T18:44:10:399616+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"services":{"dashboard":"https://172.21.15.83:8443/","prometheus":"http://172.21.15.83:9283/"}},"servicemap":{"epoch":14,"modified":"2024-09-20T18:51:16.768808+0000","services":{"rgw":{"daemons":{"summary":"","14460":{"start_epoch":12,"start_stamp":"2024-09-20T18:49:53.053671+0000","gid":14460,"addr":"172.21.15.83:0/1555339426","metadata":{"arch":"x86_64","ceph_release":"squid","ceph_version":"ceph version 19.3.0-5099-g88deadb8 (88deadb8082f4a33832e228ccefc59cedba158a5) squid (dev)","ceph_version_short":"19.3.0-5099-g88deadb8","container_hostname":"smithi083","container_image":"quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2","cpu":"Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz","distro":"centos","distro_description":"CentOS Stream 9","distro_version":"9","frontend_config#0":"beast port=80","frontend_type#0":"beast","hostname":"smithi083","id":"foo.a","kernel_description":"#1 SMP PREEMPT_DYNAMIC Thu Sep 12 15:49:37 UTC 2024","kernel_version":"5.14.0-508.el9.x86_64","mem_swap_kb":"0","mem_total_kb":"32487292","num_handles":"1","os":"Linux","pid":"2","realm_id":"","realm_name":"","zone_id":"41426759-9535-4245-bcfe-8d158f74a134","zone_name":"default","zonegroup_id":"fb472636-52d3-4587-9964-925ef38835b6","zonegroup_name":"default"},"task_status":{}}}}}},"progress_events":{}} 2024-09-20T18:57:31.802 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T18:57:31.804 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:57:31.829 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:31.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:31.830 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:57:31.930 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:57:31.955 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:31.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:31.956 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:57:32.057 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:32.080 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:32.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:32.082 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:32.182 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:32.210 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:32.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:32.211 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:32.299 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:32 smithi083 ceph-mon[29421]: pgmap v370: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 3.1 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 907 op/s; 1845/151920 objects degraded (1.214%); 1054/151920 objects misplaced (0.694%); 3.7 MiB/s, 58 objects/s recovering 2024-09-20T18:57:32.299 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:32 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:57:32.299 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:32 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:57:32.299 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:32 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2678925359' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-09-20T18:57:32.299 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:32 smithi083 ceph-mon[36260]: pgmap v370: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 3.1 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 907 op/s; 1845/151920 objects degraded (1.214%); 1054/151920 objects misplaced (0.694%); 3.7 MiB/s, 58 objects/s recovering 2024-09-20T18:57:32.299 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:32 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:57:32.299 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:32 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:57:32.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:32 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2678925359' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-09-20T18:57:32.312 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:32.351 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:32.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:32.352 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:32.453 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:57:32.477 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:32.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:32.478 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:57:32.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:32 smithi142 ceph-mon[32508]: pgmap v370: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 3.1 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 907 op/s; 1845/151920 objects degraded (1.214%); 1054/151920 objects misplaced (0.694%); 3.7 MiB/s, 58 objects/s recovering 2024-09-20T18:57:32.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:32 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:57:32.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:32 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:57:32.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:32 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2678925359' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-09-20T18:57:32.580 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:32.620 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:32.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:32.621 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:32.721 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:57:32.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:32.747 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:32.747 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:57:32.848 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:32.873 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:32.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:32.874 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:32.886 INFO:tasks.cephadm.ceph_manager.ceph:making progress, resetting timeout 2024-09-20T18:57:32.886 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T18:57:32.974 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:33.000 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:33.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:33.002 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:33.102 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:57:33.126 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:33.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:33.127 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:57:33.228 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:33.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:33.271 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:33.271 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:33.299 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:33 smithi083 ceph-mon[29421]: osdmap e224: 8 total, 8 up, 7 in 2024-09-20T18:57:33.299 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:33 smithi083 ceph-mon[36260]: osdmap e224: 8 total, 8 up, 7 in 2024-09-20T18:57:33.321 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:57:33.372 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:33.374 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-20T18:57:33.396 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:33.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:33.397 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:33.498 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:33.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:33 smithi142 ceph-mon[32508]: osdmap e224: 8 total, 8 up, 7 in 2024-09-20T18:57:33.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:33.543 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:33.543 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:33.644 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:33.667 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:33.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:33.669 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:33.769 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:33.798 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:33.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:33.799 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:33.900 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:33.923 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:33.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:33.924 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:33.925 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:57:33.995 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:57:34.003 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:57:34.004 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:57:34.023 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:57:34.024 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:34.054 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:57:34.054 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:57:34.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:34.067 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:34.067 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:34.087 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:57:34.087 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:57:34.120 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:57:34.121 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:57:34.152 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:57:34.153 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:57:34.168 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:34.185 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:57:34.186 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:57:34.186 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:34 smithi083 ceph-mon[29421]: pgmap v372: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 2.8 GiB data, 10 GiB used, 616 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.10k op/s; 2045/136350 objects degraded (1.500%); 740/136350 objects misplaced (0.543%); 1.6 MiB/s, 25 objects/s recovering 2024-09-20T18:57:34.186 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:34 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3113924832' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:57:34.187 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:34 smithi083 ceph-mon[36260]: pgmap v372: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 2.8 GiB data, 10 GiB used, 616 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.10k op/s; 2045/136350 objects degraded (1.500%); 740/136350 objects misplaced (0.543%); 1.6 MiB/s, 25 objects/s recovering 2024-09-20T18:57:34.187 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:34 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3113924832' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:57:34.192 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:34.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:34.193 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:34.220 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:57:34.220 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:57:34.248 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:57:34.248 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:57:34.294 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:34.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:34.393 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:34.393 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:34.494 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:34.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:34 smithi142 ceph-mon[32508]: pgmap v372: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 2.8 GiB data, 10 GiB used, 616 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.10k op/s; 2045/136350 objects degraded (1.500%); 740/136350 objects misplaced (0.543%); 1.6 MiB/s, 25 objects/s recovering 2024-09-20T18:57:34.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:34 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3113924832' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:57:34.522 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:34.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:34.523 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:34.624 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:34.651 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:34.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:34.652 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:34.753 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:34.780 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:34.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:34.781 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:34.882 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:34.908 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:34.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:34.909 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:35.009 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:35.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:35.049 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:35.050 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:35.150 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:57:35.174 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:35.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:35.175 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:57:35.276 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:35.314 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:35.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:35.315 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:35.416 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:35.451 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:35.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:35.453 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:35.553 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:35.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:35.579 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:35.579 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:35.680 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:35.704 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:35.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:35.705 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:35.805 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:35.846 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:35.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:35.847 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:35.947 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:35.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:35.975 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:35.975 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:36.076 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:36.114 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:36.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:36.116 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:36.216 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:36.243 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:36 smithi083 ceph-mon[29421]: pgmap v373: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 2.8 GiB data, 10 GiB used, 616 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.01k op/s; 2045/136350 objects degraded (1.500%); 740/136350 objects misplaced (0.543%); 1.4 MiB/s, 23 objects/s recovering 2024-09-20T18:57:36.243 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:36 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 2045/136350 objects degraded (1.500%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T18:57:36.243 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:36 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3622851071' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:57:36.243 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:36 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:57:36.243 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:36 smithi083 ceph-mon[36260]: pgmap v373: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 2.8 GiB data, 10 GiB used, 616 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.01k op/s; 2045/136350 objects degraded (1.500%); 740/136350 objects misplaced (0.543%); 1.4 MiB/s, 23 objects/s recovering 2024-09-20T18:57:36.243 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:36 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 2045/136350 objects degraded (1.500%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T18:57:36.244 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:36 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3622851071' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:57:36.244 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:36 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:57:36.244 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:36.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:36.245 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:36.346 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:36.384 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:36.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:36.385 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:36.485 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:36.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:36 smithi142 ceph-mon[32508]: pgmap v373: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 2.8 GiB data, 10 GiB used, 616 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.01k op/s; 2045/136350 objects degraded (1.500%); 740/136350 objects misplaced (0.543%); 1.4 MiB/s, 23 objects/s recovering 2024-09-20T18:57:36.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:36 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 2045/136350 objects degraded (1.500%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T18:57:36.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:36 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3622851071' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:57:36.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:36 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:57:36.528 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:36.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:36.529 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:36.630 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:36.669 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:36.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:36.670 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:36.731 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-20T18:57:36.772 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:36.796 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:36.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:36.797 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:36.898 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:36.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:36.932 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:36.933 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:37.033 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:57:37.058 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:37.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:37.059 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:57:37.160 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:37.195 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:37.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:37.196 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:37.268 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:57:36 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:57:37.269 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:37 smithi142 ceph-mon[32508]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:57:37.269 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:37 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T18:57:37.269 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:37 smithi142 ceph-mon[32508]: osdmap e225: 8 total, 8 up, 7 in 2024-09-20T18:57:37.296 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:57:37.299 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:37 smithi083 ceph-mon[29421]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:57:37.299 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:37 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T18:57:37.299 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:37 smithi083 ceph-mon[29421]: osdmap e225: 8 total, 8 up, 7 in 2024-09-20T18:57:37.299 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:37 smithi083 ceph-mon[36260]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:57:37.299 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:37 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T18:57:37.299 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:37 smithi083 ceph-mon[36260]: osdmap e225: 8 total, 8 up, 7 in 2024-09-20T18:57:37.319 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:37.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:37.320 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:57:37.421 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:37.460 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:37.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:37.461 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:37.562 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:37.608 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:37.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:37.609 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:37.709 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:37.734 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:37.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:37.735 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:37.835 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:37.860 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:37.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:37.861 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:37.961 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:37.990 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:37.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:37.991 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:38.024 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T18:57:38.056 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:57:38.061 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-20T18:57:38.092 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:38.116 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:38.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:38.117 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:38.218 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:38.240 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:38.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:38.241 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:38.342 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:38.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:38.364 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:38.365 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:38.465 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:38.487 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:38.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:38.488 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:38.588 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:38.624 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:38.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:38.625 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:38.726 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:38.750 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:38.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:38.751 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:38.851 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:38.876 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:38.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:38.877 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:38.978 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:38.982 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:38 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:57:38.982 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:38 smithi083 ceph-mon[29421]: pgmap v375: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 2.4 GiB data, 9.3 GiB used, 617 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.30k op/s; 2256/120633 objects degraded (1.870%); 401/120633 objects misplaced (0.332%); 1.9 MiB/s, 30 objects/s recovering 2024-09-20T18:57:38.982 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:38 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:57:38.982 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:38 smithi083 ceph-mon[36260]: pgmap v375: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 2.4 GiB data, 9.3 GiB used, 617 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.30k op/s; 2256/120633 objects degraded (1.870%); 401/120633 objects misplaced (0.332%); 1.9 MiB/s, 30 objects/s recovering 2024-09-20T18:57:39.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:39.011 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:39.011 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:39.018 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:38 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:57:39.019 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:38 smithi142 ceph-mon[32508]: pgmap v375: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 2.4 GiB data, 9.3 GiB used, 617 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.30k op/s; 2256/120633 objects degraded (1.870%); 401/120633 objects misplaced (0.332%); 1.9 MiB/s, 30 objects/s recovering 2024-09-20T18:57:39.111 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:57:39.138 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:39.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:39.140 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:57:39.241 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:39.279 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:39.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:39.282 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:39.382 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:39.422 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:39.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:39.423 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:39.513 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T18:57:39.514 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:57:39.524 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:39.548 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:39.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:39.549 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:39.650 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:39.690 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:39.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:39.691 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:39.792 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:39.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:39.830 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:39.831 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:39.931 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:39.977 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:39.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:39.978 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:40.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:39 smithi083 ceph-mon[29421]: from='client.25337 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:57:40.007 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:39 smithi083 ceph-mon[36260]: from='client.25337 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:57:40.018 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:39 smithi142 ceph-mon[32508]: from='client.25337 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:57:40.080 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:40.122 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:40.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:40.123 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:40.224 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:40.246 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:40.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:40.247 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:40.348 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:40.371 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:40.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:40.372 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:40.473 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:40.498 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:40.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:40.499 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:40.600 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:40.627 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:40.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:40.628 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:40.728 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:40.770 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:40.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:40.771 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:40.818 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_blocked_ops 2024-09-20T18:57:40.872 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:40.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:40.911 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:40.911 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:41.012 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:41.018 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:40 smithi142 ceph-mon[32508]: pgmap v376: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 2.4 GiB data, 9.3 GiB used, 617 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 1.30k op/s; 2256/120633 objects degraded (1.870%); 401/120633 objects misplaced (0.332%); 1.9 MiB/s, 30 objects/s recovering 2024-09-20T18:57:41.018 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:40 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 2256/120633 objects degraded (1.870%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T18:57:41.019 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:40 smithi142 ceph-mon[32508]: osdmap e226: 8 total, 8 up, 7 in 2024-09-20T18:57:41.037 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:41.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:41.038 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:41.049 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:40 smithi083 ceph-mon[29421]: pgmap v376: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 2.4 GiB data, 9.3 GiB used, 617 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 1.30k op/s; 2256/120633 objects degraded (1.870%); 401/120633 objects misplaced (0.332%); 1.9 MiB/s, 30 objects/s recovering 2024-09-20T18:57:41.049 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:40 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 2256/120633 objects degraded (1.870%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T18:57:41.049 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:40 smithi083 ceph-mon[29421]: osdmap e226: 8 total, 8 up, 7 in 2024-09-20T18:57:41.049 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:57:40 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:57:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:57:41.049 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:40 smithi083 ceph-mon[36260]: pgmap v376: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 2.4 GiB data, 9.3 GiB used, 617 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 1.30k op/s; 2256/120633 objects degraded (1.870%); 401/120633 objects misplaced (0.332%); 1.9 MiB/s, 30 objects/s recovering 2024-09-20T18:57:41.050 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:40 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 2256/120633 objects degraded (1.870%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T18:57:41.050 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:40 smithi083 ceph-mon[36260]: osdmap e226: 8 total, 8 up, 7 in 2024-09-20T18:57:41.139 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:41.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:41.189 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:41.189 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:41.290 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:41.318 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:41.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:41.319 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:41.419 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:41.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:41.455 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:41.455 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:41.556 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:41.586 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:41.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:41.587 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:41.594 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":375,"stamp":"2024-09-20T18:57:36.935895+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2622757427,"num_objects":40211,"num_object_clones":0,"num_object_copies":120633,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2256,"num_objects_misplaced":401,"num_objects_unfound":0,"num_objects_dirty":40211,"num_whiteouts":0,"num_read":1311,"num_read_kb":1168,"num_write":192715,"num_write_kb":7431306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25494,"num_bytes_recovered":1665731529,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74755,"ondisk_log_size":74755,"up":444,"acting":443,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":9755456,"kb_used_data":7733648,"kb_used_omap":10,"kb_used_meta":2021557,"kb_avail":646460608,"statfs":{"total":671965249536,"available":661975662592,"internally_reserved":0,"allocated":7919255552,"data_stored":7916208534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11158,"internal_metadata":2070074474},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-683474944,"num_objects":-10429,"num_object_clones":0,"num_object_copies":-31287,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":411,"num_objects_misplaced":-653,"num_objects_unfound":0,"num_objects_dirty":-10429,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":10429,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":15859712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002333"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":369,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T18:57:36.724535+0000","last_change":"2024-09-20T18:55:28.037494+0000","last_active":"2024-09-20T18:57:36.724535+0000","last_peered":"2024-09-20T18:57:36.724535+0000","last_clean":"2024-09-20T18:57:36.724535+0000","last_became_active":"2024-09-20T18:49:45.943201+0000","last_became_peered":"2024-09-20T18:49:45.943201+0000","last_unstale":"2024-09-20T18:57:36.724535+0000","last_undegraded":"2024-09-20T18:57:36.724535+0000","last_fullsized":"2024-09-20T18:57:36.724535+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:28.037449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:28.037449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":414,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.013682+0000","last_change":"2024-09-20T18:55:28.838729+0000","last_active":"2024-09-20T18:57:32.013682+0000","last_peered":"2024-09-20T18:57:32.013682+0000","last_clean":"2024-09-20T18:57:32.013682+0000","last_became_active":"2024-09-20T18:55:28.819637+0000","last_became_peered":"2024-09-20T18:55:28.819637+0000","last_unstale":"2024-09-20T18:57:32.013682+0000","last_undegraded":"2024-09-20T18:57:32.013682+0000","last_fullsized":"2024-09-20T18:57:32.013682+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:50.231620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:50.231620+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":360,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.014711+0000","last_change":"2024-09-20T18:54:25.361988+0000","last_active":"2024-09-20T18:57:32.014711+0000","last_peered":"2024-09-20T18:57:32.014711+0000","last_clean":"2024-09-20T18:57:32.014711+0000","last_became_active":"2024-09-20T18:49:49.959514+0000","last_became_peered":"2024-09-20T18:49:49.959514+0000","last_unstale":"2024-09-20T18:57:32.014711+0000","last_undegraded":"2024-09-20T18:57:32.014711+0000","last_fullsized":"2024-09-20T18:57:32.014711+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:25.361935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:25.361935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":356,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.014158+0000","last_change":"2024-09-20T18:54:15.221404+0000","last_active":"2024-09-20T18:57:32.014158+0000","last_peered":"2024-09-20T18:57:32.014158+0000","last_clean":"2024-09-20T18:57:32.014158+0000","last_became_active":"2024-09-20T18:49:51.969421+0000","last_became_peered":"2024-09-20T18:49:51.969421+0000","last_unstale":"2024-09-20T18:57:32.014158+0000","last_undegraded":"2024-09-20T18:57:32.014158+0000","last_fullsized":"2024-09-20T18:57:32.014158+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:15.221342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:15.221342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":363,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.020402+0000","last_change":"2024-09-20T18:55:28.822034+0000","last_active":"2024-09-20T18:57:32.020402+0000","last_peered":"2024-09-20T18:57:32.020402+0000","last_clean":"2024-09-20T18:57:32.020402+0000","last_became_active":"2024-09-20T18:55:28.821717+0000","last_became_peered":"2024-09-20T18:55:28.821717+0000","last_unstale":"2024-09-20T18:57:32.020402+0000","last_undegraded":"2024-09-20T18:57:32.020402+0000","last_fullsized":"2024-09-20T18:57:32.020402+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.216188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.216188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":367,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.018074+0000","last_change":"2024-09-20T18:57:22.694362+0000","last_active":"2024-09-20T18:57:32.018074+0000","last_peered":"2024-09-20T18:57:32.018074+0000","last_clean":"2024-09-20T18:57:32.018074+0000","last_became_active":"2024-09-20T18:55:28.824826+0000","last_became_peered":"2024-09-20T18:55:28.824826+0000","last_unstale":"2024-09-20T18:57:32.018074+0000","last_undegraded":"2024-09-20T18:57:32.018074+0000","last_fullsized":"2024-09-20T18:57:32.018074+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:22.694295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:22.694295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:33.473592+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":381,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.018843+0000","last_change":"2024-09-20T18:57:13.774827+0000","last_active":"2024-09-20T18:57:32.018843+0000","last_peered":"2024-09-20T18:57:32.018843+0000","last_clean":"2024-09-20T18:57:32.018843+0000","last_became_active":"2024-09-20T18:49:47.952150+0000","last_became_peered":"2024-09-20T18:49:47.952150+0000","last_unstale":"2024-09-20T18:57:32.018843+0000","last_undegraded":"2024-09-20T18:57:32.018843+0000","last_fullsized":"2024-09-20T18:57:32.018843+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:57:13.774305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:13.774305+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:25.245744+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":359,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.013676+0000","last_change":"2024-09-20T18:55:28.820753+0000","last_active":"2024-09-20T18:57:32.013676+0000","last_peered":"2024-09-20T18:57:32.013676+0000","last_clean":"2024-09-20T18:57:32.013676+0000","last_became_active":"2024-09-20T18:55:28.820362+0000","last_became_peered":"2024-09-20T18:55:28.820362+0000","last_unstale":"2024-09-20T18:57:32.013676+0000","last_undegraded":"2024-09-20T18:57:32.013676+0000","last_fullsized":"2024-09-20T18:57:32.013676+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:11.246922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:11.246922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"59'1","reported_seq":392,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.011045+0000","last_change":"2024-09-20T18:55:29.340140+0000","last_active":"2024-09-20T18:57:32.011045+0000","last_peered":"2024-09-20T18:57:32.011045+0000","last_clean":"2024-09-20T18:57:32.011045+0000","last_became_active":"2024-09-20T18:55:28.833386+0000","last_became_peered":"2024-09-20T18:55:28.833386+0000","last_unstale":"2024-09-20T18:57:32.011045+0000","last_undegraded":"2024-09-20T18:57:32.011045+0000","last_fullsized":"2024-09-20T18:57:32.011045+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:55:29.340083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:29.340083+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":380,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.014326+0000","last_change":"2024-09-20T18:54:53.241889+0000","last_active":"2024-09-20T18:57:32.014326+0000","last_peered":"2024-09-20T18:57:32.014326+0000","last_clean":"2024-09-20T18:57:32.014326+0000","last_became_active":"2024-09-20T18:49:47.947448+0000","last_became_peered":"2024-09-20T18:49:47.947448+0000","last_unstale":"2024-09-20T18:57:32.014326+0000","last_undegraded":"2024-09-20T18:57:32.014326+0000","last_fullsized":"2024-09-20T18:57:32.014326+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:53.241802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:53.241802+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":359,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.010685+0000","last_change":"2024-09-20T18:57:08.539466+0000","last_active":"2024-09-20T18:57:32.010685+0000","last_peered":"2024-09-20T18:57:32.010685+0000","last_clean":"2024-09-20T18:57:32.010685+0000","last_became_active":"2024-09-20T18:49:49.961188+0000","last_became_peered":"2024-09-20T18:49:49.961188+0000","last_unstale":"2024-09-20T18:57:32.010685+0000","last_undegraded":"2024-09-20T18:57:32.010685+0000","last_fullsized":"2024-09-20T18:57:32.010685+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:08.539418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:08.539418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:13.102215+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":333,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T18:57:36.723176+0000","last_change":"2024-09-20T18:49:51.966957+0000","last_active":"2024-09-20T18:57:36.723176+0000","last_peered":"2024-09-20T18:57:36.723176+0000","last_clean":"2024-09-20T18:57:36.723176+0000","last_became_active":"2024-09-20T18:49:51.966715+0000","last_became_peered":"2024-09-20T18:49:51.966715+0000","last_unstale":"2024-09-20T18:57:36.723176+0000","last_undegraded":"2024-09-20T18:57:36.723176+0000","last_fullsized":"2024-09-20T18:57:36.723176+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":359,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.010807+0000","last_change":"2024-09-20T18:55:15.252839+0000","last_active":"2024-09-20T18:57:32.010807+0000","last_peered":"2024-09-20T18:57:32.010807+0000","last_clean":"2024-09-20T18:57:32.010807+0000","last_became_active":"2024-09-20T18:49:45.944703+0000","last_became_peered":"2024-09-20T18:49:45.944703+0000","last_unstale":"2024-09-20T18:57:32.010807+0000","last_undegraded":"2024-09-20T18:57:32.010807+0000","last_fullsized":"2024-09-20T18:57:32.010807+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:15.252782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:15.252782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":384,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.013807+0000","last_change":"2024-09-20T18:51:26.494194+0000","last_active":"2024-09-20T18:57:32.013807+0000","last_peered":"2024-09-20T18:57:32.013807+0000","last_clean":"2024-09-20T18:57:32.013807+0000","last_became_active":"2024-09-20T18:51:23.813399+0000","last_became_peered":"2024-09-20T18:51:23.813399+0000","last_unstale":"2024-09-20T18:57:32.013807+0000","last_undegraded":"2024-09-20T18:57:32.013807+0000","last_fullsized":"2024-09-20T18:57:32.013807+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-20T18:51:26.494118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:26.494118+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:31.887636+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":351,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.004973+0000","last_change":"2024-09-20T18:52:38.799542+0000","last_active":"2024-09-20T18:57:32.004973+0000","last_peered":"2024-09-20T18:57:32.004973+0000","last_clean":"2024-09-20T18:57:32.004973+0000","last_became_active":"2024-09-20T18:49:49.959696+0000","last_became_peered":"2024-09-20T18:49:49.959696+0000","last_unstale":"2024-09-20T18:57:32.004973+0000","last_undegraded":"2024-09-20T18:57:32.004973+0000","last_fullsized":"2024-09-20T18:57:32.004973+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:38.799487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:38.799487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":355,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.011079+0000","last_change":"2024-09-20T18:57:13.587827+0000","last_active":"2024-09-20T18:57:32.011079+0000","last_peered":"2024-09-20T18:57:32.011079+0000","last_clean":"2024-09-20T18:57:32.011079+0000","last_became_active":"2024-09-20T18:49:51.970663+0000","last_became_peered":"2024-09-20T18:49:51.970663+0000","last_unstale":"2024-09-20T18:57:32.011079+0000","last_undegraded":"2024-09-20T18:57:32.011079+0000","last_fullsized":"2024-09-20T18:57:32.011079+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:13.587774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:13.587774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:15.272228+0000","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":355,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.014288+0000","last_change":"2024-09-20T18:55:28.828417+0000","last_active":"2024-09-20T18:57:32.014288+0000","last_peered":"2024-09-20T18:57:32.014288+0000","last_clean":"2024-09-20T18:57:32.014288+0000","last_became_active":"2024-09-20T18:55:28.827435+0000","last_became_peered":"2024-09-20T18:55:28.827435+0000","last_unstale":"2024-09-20T18:57:32.014288+0000","last_undegraded":"2024-09-20T18:57:32.014288+0000","last_fullsized":"2024-09-20T18:57:32.014288+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:53.940191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:53.940191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":361,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T18:57:36.724488+0000","last_change":"2024-09-20T18:54:23.984024+0000","last_active":"2024-09-20T18:57:36.724488+0000","last_peered":"2024-09-20T18:57:36.724488+0000","last_clean":"2024-09-20T18:57:36.724488+0000","last_became_active":"2024-09-20T18:49:45.950480+0000","last_became_peered":"2024-09-20T18:49:45.950480+0000","last_unstale":"2024-09-20T18:57:36.724488+0000","last_undegraded":"2024-09-20T18:57:36.724488+0000","last_fullsized":"2024-09-20T18:57:36.724488+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:23.983958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:23.983958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":382,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.010668+0000","last_change":"2024-09-20T18:53:26.762207+0000","last_active":"2024-09-20T18:57:32.010668+0000","last_peered":"2024-09-20T18:57:32.010668+0000","last_clean":"2024-09-20T18:57:32.010668+0000","last_became_active":"2024-09-20T18:49:47.950939+0000","last_became_peered":"2024-09-20T18:49:47.950939+0000","last_unstale":"2024-09-20T18:57:32.010668+0000","last_undegraded":"2024-09-20T18:57:32.010668+0000","last_fullsized":"2024-09-20T18:57:32.010668+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:53:26.762131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:26.762131+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":352,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.015221+0000","last_change":"2024-09-20T18:57:14.941518+0000","last_active":"2024-09-20T18:57:32.015221+0000","last_peered":"2024-09-20T18:57:32.015221+0000","last_clean":"2024-09-20T18:57:32.015221+0000","last_became_active":"2024-09-20T18:49:49.958518+0000","last_became_peered":"2024-09-20T18:49:49.958518+0000","last_unstale":"2024-09-20T18:57:32.015221+0000","last_undegraded":"2024-09-20T18:57:32.015221+0000","last_fullsized":"2024-09-20T18:57:32.015221+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:14.941458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:14.941458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:37.988570+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":355,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.020972+0000","last_change":"2024-09-20T18:55:28.833468+0000","last_active":"2024-09-20T18:57:32.020972+0000","last_peered":"2024-09-20T18:57:32.020972+0000","last_clean":"2024-09-20T18:57:32.020972+0000","last_became_active":"2024-09-20T18:55:28.833133+0000","last_became_peered":"2024-09-20T18:55:28.833133+0000","last_unstale":"2024-09-20T18:57:32.020972+0000","last_undegraded":"2024-09-20T18:57:32.020972+0000","last_fullsized":"2024-09-20T18:57:32.020972+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:54.259174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:54.259174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:30.790542+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"59'1","reported_seq":366,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.014435+0000","last_change":"2024-09-20T18:53:42.840884+0000","last_active":"2024-09-20T18:57:32.014435+0000","last_peered":"2024-09-20T18:57:32.014435+0000","last_clean":"2024-09-20T18:57:32.014435+0000","last_became_active":"2024-09-20T18:49:45.940100+0000","last_became_peered":"2024-09-20T18:49:45.940100+0000","last_unstale":"2024-09-20T18:57:32.014435+0000","last_undegraded":"2024-09-20T18:57:32.014435+0000","last_fullsized":"2024-09-20T18:57:32.014435+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:53:42.840800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:42.840800+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":371,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.018298+0000","last_change":"2024-09-20T18:57:09.819080+0000","last_active":"2024-09-20T18:57:32.018298+0000","last_peered":"2024-09-20T18:57:32.018298+0000","last_clean":"2024-09-20T18:57:32.018298+0000","last_became_active":"2024-09-20T18:49:47.950909+0000","last_became_peered":"2024-09-20T18:49:47.950909+0000","last_unstale":"2024-09-20T18:57:32.018298+0000","last_undegraded":"2024-09-20T18:57:32.018298+0000","last_fullsized":"2024-09-20T18:57:32.018298+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:57:09.818999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:09.818999+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:17.216750+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":360,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.014423+0000","last_change":"2024-09-20T18:54:44.239273+0000","last_active":"2024-09-20T18:57:32.014423+0000","last_peered":"2024-09-20T18:57:32.014423+0000","last_clean":"2024-09-20T18:57:32.014423+0000","last_became_active":"2024-09-20T18:49:49.959102+0000","last_became_peered":"2024-09-20T18:49:49.959102+0000","last_unstale":"2024-09-20T18:57:32.014423+0000","last_undegraded":"2024-09-20T18:57:32.014423+0000","last_fullsized":"2024-09-20T18:57:32.014423+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:44.239226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.239226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":348,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.005189+0000","last_change":"2024-09-20T18:52:15.826819+0000","last_active":"2024-09-20T18:57:32.005189+0000","last_peered":"2024-09-20T18:57:32.005189+0000","last_clean":"2024-09-20T18:57:32.005189+0000","last_became_active":"2024-09-20T18:49:51.970058+0000","last_became_peered":"2024-09-20T18:49:51.970058+0000","last_unstale":"2024-09-20T18:57:32.005189+0000","last_undegraded":"2024-09-20T18:57:32.005189+0000","last_fullsized":"2024-09-20T18:57:32.005189+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T18:52:15.826638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:15.826638+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":361,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.014844+0000","last_change":"2024-09-20T18:57:18.032140+0000","last_active":"2024-09-20T18:57:32.014844+0000","last_peered":"2024-09-20T18:57:32.014844+0000","last_clean":"2024-09-20T18:57:32.014844+0000","last_became_active":"2024-09-20T18:49:45.933078+0000","last_became_peered":"2024-09-20T18:49:45.933078+0000","last_unstale":"2024-09-20T18:57:32.014844+0000","last_undegraded":"2024-09-20T18:57:32.014844+0000","last_fullsized":"2024-09-20T18:57:32.014844+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:57:18.032067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:18.032067+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:42.727785+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":380,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T18:57:36.727753+0000","last_change":"2024-09-20T18:55:19.996522+0000","last_active":"2024-09-20T18:57:36.727753+0000","last_peered":"2024-09-20T18:57:36.727753+0000","last_clean":"2024-09-20T18:57:36.727753+0000","last_became_active":"2024-09-20T18:49:48.500809+0000","last_became_peered":"2024-09-20T18:49:48.500809+0000","last_unstale":"2024-09-20T18:57:36.727753+0000","last_undegraded":"2024-09-20T18:57:36.727753+0000","last_fullsized":"2024-09-20T18:57:36.727753+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:55:19.996328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:19.996328+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"184'9","reported_seq":156,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.014536+0000","last_change":"2024-09-20T18:55:28.845188+0000","last_active":"2024-09-20T18:57:32.014536+0000","last_peered":"2024-09-20T18:57:32.014536+0000","last_clean":"2024-09-20T18:57:32.014536+0000","last_became_active":"2024-09-20T18:55:28.820941+0000","last_became_peered":"2024-09-20T18:55:28.820941+0000","last_unstale":"2024-09-20T18:57:32.014536+0000","last_undegraded":"2024-09-20T18:57:32.014536+0000","last_fullsized":"2024-09-20T18:57:32.014536+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:53:44.524040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:44.524040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3","version":"224'11815","reported_seq":24287,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:36.363937+0000","last_change":"2024-09-20T18:56:25.697520+0000","last_active":"2024-09-20T18:57:36.363937+0000","last_peered":"2024-09-20T18:57:36.363937+0000","last_clean":"2024-09-20T18:57:36.363937+0000","last_became_active":"2024-09-20T18:56:25.687391+0000","last_became_peered":"2024-09-20T18:56:25.687391+0000","last_unstale":"2024-09-20T18:57:36.363937+0000","last_undegraded":"2024-09-20T18:57:36.363937+0000","last_fullsized":"2024-09-20T18:57:36.363937+0000","mapping_epoch":203,"log_start":"199'7572","ondisk_log_start":"199'7572","created":160,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4243,"log_dups_size":0,"ondisk_log_size":4243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":162594816,"num_objects":2481,"num_object_clones":0,"num_object_copies":7443,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2481,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11815,"num_write_kb":457472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6156,"num_bytes_recovered":403439616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":373,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.011070+0000","last_change":"2024-09-20T18:55:18.581290+0000","last_active":"2024-09-20T18:57:32.011070+0000","last_peered":"2024-09-20T18:57:32.011070+0000","last_clean":"2024-09-20T18:57:32.011070+0000","last_became_active":"2024-09-20T18:49:58.992302+0000","last_became_peered":"2024-09-20T18:49:58.992302+0000","last_unstale":"2024-09-20T18:57:32.011070+0000","last_undegraded":"2024-09-20T18:57:32.011070+0000","last_fullsized":"2024-09-20T18:57:32.011070+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:55:18.581200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:18.581200+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":360,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.025548+0000","last_change":"2024-09-20T18:54:54.288199+0000","last_active":"2024-09-20T18:57:32.025548+0000","last_peered":"2024-09-20T18:57:32.025548+0000","last_clean":"2024-09-20T18:57:32.025548+0000","last_became_active":"2024-09-20T18:49:45.936104+0000","last_became_peered":"2024-09-20T18:49:45.936104+0000","last_unstale":"2024-09-20T18:57:32.025548+0000","last_undegraded":"2024-09-20T18:57:32.025548+0000","last_fullsized":"2024-09-20T18:57:32.025548+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:54.288117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:54.288117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":46,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":457,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.014489+0000","last_change":"2024-09-20T18:54:14.292660+0000","last_active":"2024-09-20T18:57:32.014489+0000","last_peered":"2024-09-20T18:57:32.014489+0000","last_clean":"2024-09-20T18:57:32.014489+0000","last_became_active":"2024-09-20T18:49:49.960697+0000","last_became_peered":"2024-09-20T18:49:49.960697+0000","last_unstale":"2024-09-20T18:57:32.014489+0000","last_undegraded":"2024-09-20T18:57:32.014489+0000","last_fullsized":"2024-09-20T18:57:32.014489+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:54:14.292610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:14.292610+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":34,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":347,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.010632+0000","last_change":"2024-09-20T18:54:10.381171+0000","last_active":"2024-09-20T18:57:32.010632+0000","last_peered":"2024-09-20T18:57:32.010632+0000","last_clean":"2024-09-20T18:57:32.010632+0000","last_became_active":"2024-09-20T18:49:51.971399+0000","last_became_peered":"2024-09-20T18:49:51.971399+0000","last_unstale":"2024-09-20T18:57:32.010632+0000","last_undegraded":"2024-09-20T18:57:32.010632+0000","last_fullsized":"2024-09-20T18:57:32.010632+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:10.381108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:10.381108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:30.527012+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.c","version":"225'11934","reported_seq":12076,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T18:57:36.868630+0000","last_change":"2024-09-20T18:54:34.311256+0000","last_active":"2024-09-20T18:57:36.868630+0000","last_peered":"2024-09-20T18:57:36.868630+0000","last_clean":"2024-09-20T18:57:36.868630+0000","last_became_active":"2024-09-20T18:54:34.310904+0000","last_became_peered":"2024-09-20T18:54:34.310904+0000","last_unstale":"2024-09-20T18:57:36.868630+0000","last_undegraded":"2024-09-20T18:57:36.868630+0000","last_fullsized":"2024-09-20T18:57:36.868630+0000","mapping_epoch":160,"log_start":"194'7011","ondisk_log_start":"194'7011","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4923,"log_dups_size":0,"ondisk_log_size":4923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":161087488,"num_objects":2458,"num_object_clones":0,"num_object_copies":7374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11934,"num_write_kb":460544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":388,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.025341+0000","last_change":"2024-09-20T18:54:22.077299+0000","last_active":"2024-09-20T18:57:32.025341+0000","last_peered":"2024-09-20T18:57:32.025341+0000","last_clean":"2024-09-20T18:57:32.025341+0000","last_became_active":"2024-09-20T18:49:48.499189+0000","last_became_peered":"2024-09-20T18:49:48.499189+0000","last_unstale":"2024-09-20T18:57:32.025341+0000","last_undegraded":"2024-09-20T18:57:32.025341+0000","last_fullsized":"2024-09-20T18:57:32.025341+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-20T18:54:22.077212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:22.077212+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":352,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.021418+0000","last_change":"2024-09-20T18:54:57.237740+0000","last_active":"2024-09-20T18:57:32.021418+0000","last_peered":"2024-09-20T18:57:32.021418+0000","last_clean":"2024-09-20T18:57:32.021418+0000","last_became_active":"2024-09-20T18:49:45.940544+0000","last_became_peered":"2024-09-20T18:49:45.940544+0000","last_unstale":"2024-09-20T18:57:32.021418+0000","last_undegraded":"2024-09-20T18:57:32.021418+0000","last_fullsized":"2024-09-20T18:57:32.021418+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.237675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.237675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":360,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.013696+0000","last_change":"2024-09-20T18:54:51.229251+0000","last_active":"2024-09-20T18:57:32.013696+0000","last_peered":"2024-09-20T18:57:32.013696+0000","last_clean":"2024-09-20T18:57:32.013696+0000","last_became_active":"2024-09-20T18:49:49.963073+0000","last_became_peered":"2024-09-20T18:49:49.963073+0000","last_unstale":"2024-09-20T18:57:32.013696+0000","last_undegraded":"2024-09-20T18:57:32.013696+0000","last_fullsized":"2024-09-20T18:57:32.013696+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.229201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.229201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":356,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.014328+0000","last_change":"2024-09-20T18:54:55.323937+0000","last_active":"2024-09-20T18:57:32.014328+0000","last_peered":"2024-09-20T18:57:32.014328+0000","last_clean":"2024-09-20T18:57:32.014328+0000","last_became_active":"2024-09-20T18:49:51.968793+0000","last_became_peered":"2024-09-20T18:49:51.968793+0000","last_unstale":"2024-09-20T18:57:32.014328+0000","last_undegraded":"2024-09-20T18:57:32.014328+0000","last_fullsized":"2024-09-20T18:57:32.014328+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:55.323692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:55.323692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"224'11875","reported_seq":12016,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:35.890467+0000","last_change":"2024-09-20T18:54:34.311680+0000","last_active":"2024-09-20T18:57:35.890467+0000","last_peered":"2024-09-20T18:57:35.890467+0000","last_clean":"2024-09-20T18:57:35.890467+0000","last_became_active":"2024-09-20T18:54:34.311241+0000","last_became_peered":"2024-09-20T18:54:34.311241+0000","last_unstale":"2024-09-20T18:57:35.890467+0000","last_undegraded":"2024-09-20T18:57:35.890467+0000","last_fullsized":"2024-09-20T18:57:35.890467+0000","mapping_epoch":160,"log_start":"193'6846","ondisk_log_start":"193'6846","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":5029,"log_dups_size":0,"ondisk_log_size":5029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163577880,"num_objects":2497,"num_object_clones":0,"num_object_copies":7491,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2497,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11875,"num_write_kb":459841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":416,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.013731+0000","last_change":"2024-09-20T18:55:28.853375+0000","last_active":"2024-09-20T18:57:32.013731+0000","last_peered":"2024-09-20T18:57:32.013731+0000","last_clean":"2024-09-20T18:57:32.013731+0000","last_became_active":"2024-09-20T18:55:28.831057+0000","last_became_peered":"2024-09-20T18:55:28.831057+0000","last_unstale":"2024-09-20T18:57:32.013731+0000","last_undegraded":"2024-09-20T18:57:32.013731+0000","last_fullsized":"2024-09-20T18:57:32.013731+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-20T18:54:57.394893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.394893+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":354,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T18:57:36.727979+0000","last_change":"2024-09-20T18:51:27.187060+0000","last_active":"2024-09-20T18:57:36.727979+0000","last_peered":"2024-09-20T18:57:36.727979+0000","last_clean":"2024-09-20T18:57:36.727979+0000","last_became_active":"2024-09-20T18:49:45.940469+0000","last_became_peered":"2024-09-20T18:49:45.940469+0000","last_unstale":"2024-09-20T18:57:36.727979+0000","last_undegraded":"2024-09-20T18:57:36.727979+0000","last_fullsized":"2024-09-20T18:57:36.727979+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:51:27.186991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:27.186991+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"66'8","reported_seq":463,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.025206+0000","last_change":"2024-09-20T18:54:47.223010+0000","last_active":"2024-09-20T18:57:32.025206+0000","last_peered":"2024-09-20T18:57:32.025206+0000","last_clean":"2024-09-20T18:57:32.025206+0000","last_became_active":"2024-09-20T18:49:49.956133+0000","last_became_peered":"2024-09-20T18:49:49.956133+0000","last_unstale":"2024-09-20T18:57:32.025206+0000","last_undegraded":"2024-09-20T18:57:32.025206+0000","last_fullsized":"2024-09-20T18:57:32.025206+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:54:47.222937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:47.222937+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":354,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.005236+0000","last_change":"2024-09-20T18:55:28.819147+0000","last_active":"2024-09-20T18:57:32.005236+0000","last_peered":"2024-09-20T18:57:32.005236+0000","last_clean":"2024-09-20T18:57:32.005236+0000","last_became_active":"2024-09-20T18:55:28.818804+0000","last_became_peered":"2024-09-20T18:55:28.818804+0000","last_unstale":"2024-09-20T18:57:32.005236+0000","last_undegraded":"2024-09-20T18:57:32.005236+0000","last_fullsized":"2024-09-20T18:57:32.005236+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:42.880724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:42.880724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"225'11934","reported_seq":12076,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T18:57:36.844127+0000","last_change":"2024-09-20T18:54:34.309939+0000","last_active":"2024-09-20T18:57:36.844127+0000","last_peered":"2024-09-20T18:57:36.844127+0000","last_clean":"2024-09-20T18:57:36.844127+0000","last_became_active":"2024-09-20T18:54:34.309630+0000","last_became_peered":"2024-09-20T18:54:34.309630+0000","last_unstale":"2024-09-20T18:57:36.844127+0000","last_undegraded":"2024-09-20T18:57:36.844127+0000","last_fullsized":"2024-09-20T18:57:36.844127+0000","mapping_epoch":160,"log_start":"195'7049","ondisk_log_start":"195'7049","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4885,"log_dups_size":0,"ondisk_log_size":4885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":161742848,"num_objects":2468,"num_object_clones":0,"num_object_copies":7404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2468,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11934,"num_write_kb":460864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":379,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.013593+0000","last_change":"2024-09-20T18:55:19.541011+0000","last_active":"2024-09-20T18:57:32.013593+0000","last_peered":"2024-09-20T18:57:32.013593+0000","last_clean":"2024-09-20T18:57:32.013593+0000","last_became_active":"2024-09-20T18:49:47.949626+0000","last_became_peered":"2024-09-20T18:49:47.949626+0000","last_unstale":"2024-09-20T18:57:32.013593+0000","last_undegraded":"2024-09-20T18:57:32.013593+0000","last_fullsized":"2024-09-20T18:57:32.013593+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:55:19.540812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:19.540812+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":367,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.011108+0000","last_change":"2024-09-20T18:54:18.326947+0000","last_active":"2024-09-20T18:57:32.011108+0000","last_peered":"2024-09-20T18:57:32.011108+0000","last_clean":"2024-09-20T18:57:32.011108+0000","last_became_active":"2024-09-20T18:49:45.942496+0000","last_became_peered":"2024-09-20T18:49:45.942496+0000","last_unstale":"2024-09-20T18:57:32.011108+0000","last_undegraded":"2024-09-20T18:57:32.011108+0000","last_fullsized":"2024-09-20T18:57:32.011108+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:18.326875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:18.326875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"184'9","reported_seq":467,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T18:57:36.724128+0000","last_change":"2024-09-20T18:55:29.010062+0000","last_active":"2024-09-20T18:57:36.724128+0000","last_peered":"2024-09-20T18:57:36.724128+0000","last_clean":"2024-09-20T18:57:36.724128+0000","last_became_active":"2024-09-20T18:55:28.830643+0000","last_became_peered":"2024-09-20T18:55:28.830643+0000","last_unstale":"2024-09-20T18:57:36.724128+0000","last_undegraded":"2024-09-20T18:57:36.724128+0000","last_fullsized":"2024-09-20T18:57:36.724128+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:29.010015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:29.010015+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":333,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T18:57:36.723318+0000","last_change":"2024-09-20T18:49:51.965733+0000","last_active":"2024-09-20T18:57:36.723318+0000","last_peered":"2024-09-20T18:57:36.723318+0000","last_clean":"2024-09-20T18:57:36.723318+0000","last_became_active":"2024-09-20T18:49:51.964780+0000","last_became_peered":"2024-09-20T18:49:51.964780+0000","last_unstale":"2024-09-20T18:57:36.723318+0000","last_undegraded":"2024-09-20T18:57:36.723318+0000","last_fullsized":"2024-09-20T18:57:36.723318+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"224'12220","reported_seq":12360,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:36.516362+0000","last_change":"2024-09-20T18:54:34.309809+0000","last_active":"2024-09-20T18:57:36.516362+0000","last_peered":"2024-09-20T18:57:36.516362+0000","last_clean":"2024-09-20T18:57:36.516362+0000","last_became_active":"2024-09-20T18:54:34.309231+0000","last_became_peered":"2024-09-20T18:54:34.309231+0000","last_unstale":"2024-09-20T18:57:36.516362+0000","last_undegraded":"2024-09-20T18:57:36.516362+0000","last_fullsized":"2024-09-20T18:57:36.516362+0000","mapping_epoch":160,"log_start":"200'7932","ondisk_log_start":"200'7932","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4288,"log_dups_size":0,"ondisk_log_size":4288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163446784,"num_objects":2494,"num_object_clones":0,"num_object_copies":7482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2494,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12220,"num_write_kb":470848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":393,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T18:57:36.724325+0000","last_change":"2024-09-20T18:54:14.923895+0000","last_active":"2024-09-20T18:57:36.724325+0000","last_peered":"2024-09-20T18:57:36.724325+0000","last_clean":"2024-09-20T18:57:36.724325+0000","last_became_active":"2024-09-20T18:49:48.497736+0000","last_became_peered":"2024-09-20T18:49:48.497736+0000","last_unstale":"2024-09-20T18:57:36.724325+0000","last_undegraded":"2024-09-20T18:57:36.724325+0000","last_fullsized":"2024-09-20T18:57:36.724325+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-20T18:54:14.923682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:14.923682+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":352,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.013410+0000","last_change":"2024-09-20T18:57:09.940634+0000","last_active":"2024-09-20T18:57:32.013410+0000","last_peered":"2024-09-20T18:57:32.013410+0000","last_clean":"2024-09-20T18:57:32.013410+0000","last_became_active":"2024-09-20T18:49:45.942253+0000","last_became_peered":"2024-09-20T18:49:45.942253+0000","last_unstale":"2024-09-20T18:57:32.013410+0000","last_undegraded":"2024-09-20T18:57:32.013410+0000","last_fullsized":"2024-09-20T18:57:32.013410+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:09.940517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:09.940517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:18.833936+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":100,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T18:57:36.724108+0000","last_change":"2024-09-20T18:55:28.830954+0000","last_active":"2024-09-20T18:57:36.724108+0000","last_peered":"2024-09-20T18:57:36.724108+0000","last_clean":"2024-09-20T18:57:36.724108+0000","last_became_active":"2024-09-20T18:55:28.830645+0000","last_became_peered":"2024-09-20T18:55:28.830645+0000","last_unstale":"2024-09-20T18:57:36.724108+0000","last_undegraded":"2024-09-20T18:57:36.724108+0000","last_fullsized":"2024-09-20T18:57:36.724108+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:50.610871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:50.610871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":363,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.014628+0000","last_change":"2024-09-20T18:55:28.822362+0000","last_active":"2024-09-20T18:57:32.014628+0000","last_peered":"2024-09-20T18:57:32.014628+0000","last_clean":"2024-09-20T18:57:32.014628+0000","last_became_active":"2024-09-20T18:55:28.821588+0000","last_became_peered":"2024-09-20T18:55:28.821588+0000","last_unstale":"2024-09-20T18:57:32.014628+0000","last_undegraded":"2024-09-20T18:57:32.014628+0000","last_fullsized":"2024-09-20T18:57:32.014628+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.402841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.402841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"224'12033","reported_seq":12173,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:35.918153+0000","last_change":"2024-09-20T18:54:34.311561+0000","last_active":"2024-09-20T18:57:35.918153+0000","last_peered":"2024-09-20T18:57:35.918153+0000","last_clean":"2024-09-20T18:57:35.918153+0000","last_became_active":"2024-09-20T18:54:34.311181+0000","last_became_peered":"2024-09-20T18:54:34.311181+0000","last_unstale":"2024-09-20T18:57:35.918153+0000","last_undegraded":"2024-09-20T18:57:35.918153+0000","last_fullsized":"2024-09-20T18:57:35.918153+0000","mapping_epoch":160,"log_start":"193'6994","ondisk_log_start":"193'6994","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":5039,"log_dups_size":0,"ondisk_log_size":5039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":168099840,"num_objects":2565,"num_object_clones":0,"num_object_copies":7695,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12033,"num_write_kb":467136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"66'28","reported_seq":499,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.013318+0000","last_change":"2024-09-20T18:55:16.527267+0000","last_active":"2024-09-20T18:57:32.013318+0000","last_peered":"2024-09-20T18:57:32.013318+0000","last_clean":"2024-09-20T18:57:32.013318+0000","last_became_active":"2024-09-20T18:49:47.955335+0000","last_became_peered":"2024-09-20T18:49:47.955335+0000","last_unstale":"2024-09-20T18:57:32.013318+0000","last_undegraded":"2024-09-20T18:57:32.013318+0000","last_fullsized":"2024-09-20T18:57:32.013318+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'28","last_scrub_stamp":"2024-09-20T18:55:16.527165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:16.527165+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":359,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.014474+0000","last_change":"2024-09-20T18:57:21.807755+0000","last_active":"2024-09-20T18:57:32.014474+0000","last_peered":"2024-09-20T18:57:32.014474+0000","last_clean":"2024-09-20T18:57:32.014474+0000","last_became_active":"2024-09-20T18:55:28.831574+0000","last_became_peered":"2024-09-20T18:55:28.831574+0000","last_unstale":"2024-09-20T18:57:32.014474+0000","last_undegraded":"2024-09-20T18:57:32.014474+0000","last_fullsized":"2024-09-20T18:57:32.014474+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:21.807703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:21.807703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:42.821495+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1],"acting":[4,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":396,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.012172+0000","last_change":"2024-09-20T18:55:28.854615+0000","last_active":"2024-09-20T18:57:32.012172+0000","last_peered":"2024-09-20T18:57:32.012172+0000","last_clean":"2024-09-20T18:57:32.012172+0000","last_became_active":"2024-09-20T18:55:28.837226+0000","last_became_peered":"2024-09-20T18:55:28.837226+0000","last_unstale":"2024-09-20T18:57:32.012172+0000","last_undegraded":"2024-09-20T18:57:32.012172+0000","last_fullsized":"2024-09-20T18:57:32.012172+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-20T18:54:51.227531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.227531+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0012781159999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":361,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T18:57:36.727031+0000","last_change":"2024-09-20T18:55:13.949348+0000","last_active":"2024-09-20T18:57:36.727031+0000","last_peered":"2024-09-20T18:57:36.727031+0000","last_clean":"2024-09-20T18:57:36.727031+0000","last_became_active":"2024-09-20T18:49:49.953971+0000","last_became_peered":"2024-09-20T18:49:49.953971+0000","last_unstale":"2024-09-20T18:57:36.727031+0000","last_undegraded":"2024-09-20T18:57:36.727031+0000","last_fullsized":"2024-09-20T18:57:36.727031+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:13.949269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:13.949269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":340,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.017795+0000","last_change":"2024-09-20T18:57:12.744116+0000","last_active":"2024-09-20T18:57:32.017795+0000","last_peered":"2024-09-20T18:57:32.017795+0000","last_clean":"2024-09-20T18:57:32.017795+0000","last_became_active":"2024-09-20T18:49:51.969400+0000","last_became_peered":"2024-09-20T18:49:51.969400+0000","last_unstale":"2024-09-20T18:57:32.017795+0000","last_undegraded":"2024-09-20T18:57:32.017795+0000","last_fullsized":"2024-09-20T18:57:32.017795+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:12.744040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:12.744040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:17.964285+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"224'12155","reported_seq":12295,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:36.446729+0000","last_change":"2024-09-20T18:54:33.926785+0000","last_active":"2024-09-20T18:57:36.446729+0000","last_peered":"2024-09-20T18:57:36.446729+0000","last_clean":"2024-09-20T18:57:36.446729+0000","last_became_active":"2024-09-20T18:54:33.925750+0000","last_became_peered":"2024-09-20T18:54:33.925750+0000","last_unstale":"2024-09-20T18:57:36.446729+0000","last_undegraded":"2024-09-20T18:57:36.446729+0000","last_fullsized":"2024-09-20T18:57:36.446729+0000","mapping_epoch":160,"log_start":"199'7840","ondisk_log_start":"199'7840","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4315,"log_dups_size":0,"ondisk_log_size":4315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":167968768,"num_objects":2563,"num_object_clones":0,"num_object_copies":7689,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12155,"num_write_kb":470976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":391,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.014317+0000","last_change":"2024-09-20T18:55:25.702762+0000","last_active":"2024-09-20T18:57:32.014317+0000","last_peered":"2024-09-20T18:57:32.014317+0000","last_clean":"2024-09-20T18:57:32.014317+0000","last_became_active":"2024-09-20T18:49:47.950904+0000","last_became_peered":"2024-09-20T18:49:47.950904+0000","last_unstale":"2024-09-20T18:57:32.014317+0000","last_undegraded":"2024-09-20T18:57:32.014317+0000","last_fullsized":"2024-09-20T18:57:32.014317+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-20T18:55:25.702596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:25.702596+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":375,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.013497+0000","last_change":"2024-09-20T18:55:28.830832+0000","last_active":"2024-09-20T18:57:32.013497+0000","last_peered":"2024-09-20T18:57:32.013497+0000","last_clean":"2024-09-20T18:57:32.013497+0000","last_became_active":"2024-09-20T18:55:28.830269+0000","last_became_peered":"2024-09-20T18:55:28.830269+0000","last_unstale":"2024-09-20T18:57:32.013497+0000","last_undegraded":"2024-09-20T18:57:32.013497+0000","last_fullsized":"2024-09-20T18:57:32.013497+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:23.359769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:23.359769+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:27.139436+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":362,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.010918+0000","last_change":"2024-09-20T18:52:07.718766+0000","last_active":"2024-09-20T18:57:32.010918+0000","last_peered":"2024-09-20T18:57:32.010918+0000","last_clean":"2024-09-20T18:57:32.010918+0000","last_became_active":"2024-09-20T18:49:44.937135+0000","last_became_peered":"2024-09-20T18:49:44.937135+0000","last_unstale":"2024-09-20T18:57:32.010918+0000","last_undegraded":"2024-09-20T18:57:32.010918+0000","last_fullsized":"2024-09-20T18:57:32.010918+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:52:07.718646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:52:07.718646+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.00095558800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":345,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.020842+0000","last_change":"2024-09-20T18:51:59.796450+0000","last_active":"2024-09-20T18:57:32.020842+0000","last_peered":"2024-09-20T18:57:32.020842+0000","last_clean":"2024-09-20T18:57:32.020842+0000","last_became_active":"2024-09-20T18:49:49.954424+0000","last_became_peered":"2024-09-20T18:49:49.954424+0000","last_unstale":"2024-09-20T18:57:32.020842+0000","last_undegraded":"2024-09-20T18:57:32.020842+0000","last_fullsized":"2024-09-20T18:57:32.020842+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:59.796383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:59.796383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":346,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.005646+0000","last_change":"2024-09-20T18:55:28.826659+0000","last_active":"2024-09-20T18:57:32.005646+0000","last_peered":"2024-09-20T18:57:32.005646+0000","last_clean":"2024-09-20T18:57:32.005646+0000","last_became_active":"2024-09-20T18:55:28.825812+0000","last_became_peered":"2024-09-20T18:55:28.825812+0000","last_unstale":"2024-09-20T18:57:32.005646+0000","last_undegraded":"2024-09-20T18:57:32.005646+0000","last_fullsized":"2024-09-20T18:57:32.005646+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:39.814259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:39.814259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"224'12157","reported_seq":12297,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:36.338903+0000","last_change":"2024-09-20T18:54:34.311774+0000","last_active":"2024-09-20T18:57:36.338903+0000","last_peered":"2024-09-20T18:57:36.338903+0000","last_clean":"2024-09-20T18:57:36.338903+0000","last_became_active":"2024-09-20T18:54:34.311487+0000","last_became_peered":"2024-09-20T18:54:34.311487+0000","last_unstale":"2024-09-20T18:57:36.338903+0000","last_undegraded":"2024-09-20T18:57:36.338903+0000","last_fullsized":"2024-09-20T18:57:36.338903+0000","mapping_epoch":160,"log_start":"200'7892","ondisk_log_start":"200'7892","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4265,"log_dups_size":0,"ondisk_log_size":4265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":161284096,"num_objects":2461,"num_object_clones":0,"num_object_copies":7383,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2461,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12157,"num_write_kb":467776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":391,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T18:57:36.727828+0000","last_change":"2024-09-20T18:55:14.951950+0000","last_active":"2024-09-20T18:57:36.727828+0000","last_peered":"2024-09-20T18:57:36.727828+0000","last_clean":"2024-09-20T18:57:36.727828+0000","last_became_active":"2024-09-20T18:49:47.949230+0000","last_became_peered":"2024-09-20T18:49:47.949230+0000","last_unstale":"2024-09-20T18:57:36.727828+0000","last_undegraded":"2024-09-20T18:57:36.727828+0000","last_fullsized":"2024-09-20T18:57:36.727828+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:55:14.951892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:14.951892+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":369,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T18:57:36.727742+0000","last_change":"2024-09-20T18:54:27.905856+0000","last_active":"2024-09-20T18:57:36.727742+0000","last_peered":"2024-09-20T18:57:36.727742+0000","last_clean":"2024-09-20T18:57:36.727742+0000","last_became_active":"2024-09-20T18:49:45.939197+0000","last_became_peered":"2024-09-20T18:49:45.939197+0000","last_unstale":"2024-09-20T18:57:36.727742+0000","last_undegraded":"2024-09-20T18:57:36.727742+0000","last_fullsized":"2024-09-20T18:57:36.727742+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:27.905803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:27.905803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":843,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:34.867363+0000","last_change":"2024-09-20T18:54:26.359887+0000","last_active":"2024-09-20T18:57:34.867363+0000","last_peered":"2024-09-20T18:57:34.867363+0000","last_clean":"2024-09-20T18:57:34.867363+0000","last_became_active":"2024-09-20T18:49:44.936819+0000","last_became_peered":"2024-09-20T18:49:44.936819+0000","last_unstale":"2024-09-20T18:57:34.867363+0000","last_undegraded":"2024-09-20T18:57:34.867363+0000","last_fullsized":"2024-09-20T18:57:34.867363+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-20T18:54:26.359807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:54:26.359807+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0010996199999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":465,"num_read_kb":460,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":351,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.011386+0000","last_change":"2024-09-20T18:54:59.151228+0000","last_active":"2024-09-20T18:57:32.011386+0000","last_peered":"2024-09-20T18:57:32.011386+0000","last_clean":"2024-09-20T18:57:32.011386+0000","last_became_active":"2024-09-20T18:49:49.961066+0000","last_became_peered":"2024-09-20T18:49:49.961066+0000","last_unstale":"2024-09-20T18:57:32.011386+0000","last_undegraded":"2024-09-20T18:57:32.011386+0000","last_fullsized":"2024-09-20T18:57:32.011386+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:59.150063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:59.150063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":356,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.013053+0000","last_change":"2024-09-20T18:55:21.629867+0000","last_active":"2024-09-20T18:57:32.013053+0000","last_peered":"2024-09-20T18:57:32.013053+0000","last_clean":"2024-09-20T18:57:32.013053+0000","last_became_active":"2024-09-20T18:49:51.970343+0000","last_became_peered":"2024-09-20T18:49:51.970343+0000","last_unstale":"2024-09-20T18:57:32.013053+0000","last_undegraded":"2024-09-20T18:57:32.013053+0000","last_fullsized":"2024-09-20T18:57:32.013053+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:21.629800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:21.629800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"224'12036","reported_seq":22599,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:35.934410+0000","last_change":"2024-09-20T18:57:09.139489+0000","last_active":"2024-09-20T18:57:35.934410+0000","last_peered":"2024-09-20T18:57:35.934410+0000","last_clean":"2024-09-20T18:57:35.934410+0000","last_became_active":"2024-09-20T18:57:09.121592+0000","last_became_peered":"2024-09-20T18:57:09.121592+0000","last_unstale":"2024-09-20T18:57:35.934410+0000","last_undegraded":"2024-09-20T18:57:35.934410+0000","last_fullsized":"2024-09-20T18:57:35.934410+0000","mapping_epoch":216,"log_start":"193'7006","ondisk_log_start":"193'7006","created":160,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":5030,"log_dups_size":0,"ondisk_log_size":5030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:35.584332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163577856,"num_objects":2496,"num_object_clones":0,"num_object_copies":7488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12036,"num_write_kb":465024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5206,"num_bytes_recovered":341180416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"184'17","reported_seq":193,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.014346+0000","last_change":"2024-09-20T18:57:22.816240+0000","last_active":"2024-09-20T18:57:32.014346+0000","last_peered":"2024-09-20T18:57:32.014346+0000","last_clean":"2024-09-20T18:57:32.014346+0000","last_became_active":"2024-09-20T18:55:28.831024+0000","last_became_peered":"2024-09-20T18:55:28.831024+0000","last_unstale":"2024-09-20T18:57:32.014346+0000","last_undegraded":"2024-09-20T18:57:32.014346+0000","last_fullsized":"2024-09-20T18:57:32.014346+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'17","last_scrub_stamp":"2024-09-20T18:57:22.816154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:22.816154+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:28.727289+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":24,"num_read_kb":14,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":367,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.017304+0000","last_change":"2024-09-20T18:55:28.830447+0000","last_active":"2024-09-20T18:57:32.017304+0000","last_peered":"2024-09-20T18:57:32.017304+0000","last_clean":"2024-09-20T18:57:32.017304+0000","last_became_active":"2024-09-20T18:55:28.830147+0000","last_became_peered":"2024-09-20T18:55:28.830147+0000","last_unstale":"2024-09-20T18:57:32.017304+0000","last_undegraded":"2024-09-20T18:57:32.017304+0000","last_fullsized":"2024-09-20T18:57:32.017304+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:13.144896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:13.144896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:28.731761+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":420,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.005088+0000","last_change":"2024-09-20T18:55:28.844060+0000","last_active":"2024-09-20T18:57:32.005088+0000","last_peered":"2024-09-20T18:57:32.005088+0000","last_clean":"2024-09-20T18:57:32.005088+0000","last_became_active":"2024-09-20T18:55:28.829325+0000","last_became_peered":"2024-09-20T18:55:28.829325+0000","last_unstale":"2024-09-20T18:57:32.005088+0000","last_undegraded":"2024-09-20T18:57:32.005088+0000","last_fullsized":"2024-09-20T18:57:32.005088+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T18:55:13.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T18:55:13.696607+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":343,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.005426+0000","last_change":"2024-09-20T18:51:22.852942+0000","last_active":"2024-09-20T18:57:32.005426+0000","last_peered":"2024-09-20T18:57:32.005426+0000","last_clean":"2024-09-20T18:57:32.005426+0000","last_became_active":"2024-09-20T18:49:49.956128+0000","last_became_peered":"2024-09-20T18:49:49.956128+0000","last_unstale":"2024-09-20T18:57:32.005426+0000","last_undegraded":"2024-09-20T18:57:32.005426+0000","last_fullsized":"2024-09-20T18:57:32.005426+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:22.852869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:22.852869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":355,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.011177+0000","last_change":"2024-09-20T18:55:25.281435+0000","last_active":"2024-09-20T18:57:32.011177+0000","last_peered":"2024-09-20T18:57:32.011177+0000","last_clean":"2024-09-20T18:57:32.011177+0000","last_became_active":"2024-09-20T18:49:51.971332+0000","last_became_peered":"2024-09-20T18:49:51.971332+0000","last_unstale":"2024-09-20T18:57:32.011177+0000","last_undegraded":"2024-09-20T18:57:32.011177+0000","last_fullsized":"2024-09-20T18:57:32.011177+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:25.281366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:25.281366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"224'12032","reported_seq":12172,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:35.919691+0000","last_change":"2024-09-20T18:54:33.929366+0000","last_active":"2024-09-20T18:57:35.919691+0000","last_peered":"2024-09-20T18:57:35.919691+0000","last_clean":"2024-09-20T18:57:35.919691+0000","last_became_active":"2024-09-20T18:54:33.929072+0000","last_became_peered":"2024-09-20T18:54:33.929072+0000","last_unstale":"2024-09-20T18:57:35.919691+0000","last_undegraded":"2024-09-20T18:57:35.919691+0000","last_fullsized":"2024-09-20T18:57:35.919691+0000","mapping_epoch":160,"log_start":"197'7402","ondisk_log_start":"197'7402","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4630,"log_dups_size":0,"ondisk_log_size":4630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":165806080,"num_objects":2530,"num_object_clones":0,"num_object_copies":7590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12032,"num_write_kb":465984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":386,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.014385+0000","last_change":"2024-09-20T18:55:28.840105+0000","last_active":"2024-09-20T18:57:32.014385+0000","last_peered":"2024-09-20T18:57:32.014385+0000","last_clean":"2024-09-20T18:57:32.014385+0000","last_became_active":"2024-09-20T18:55:28.818269+0000","last_became_peered":"2024-09-20T18:55:28.818269+0000","last_unstale":"2024-09-20T18:57:32.014385+0000","last_undegraded":"2024-09-20T18:57:32.014385+0000","last_fullsized":"2024-09-20T18:57:32.014385+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:51:14.397930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.397930+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:23.803752+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":352,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.014096+0000","last_change":"2024-09-20T18:51:18.410601+0000","last_active":"2024-09-20T18:57:32.014096+0000","last_peered":"2024-09-20T18:57:32.014096+0000","last_clean":"2024-09-20T18:57:32.014096+0000","last_became_active":"2024-09-20T18:49:45.944579+0000","last_became_peered":"2024-09-20T18:49:45.944579+0000","last_unstale":"2024-09-20T18:57:32.014096+0000","last_undegraded":"2024-09-20T18:57:32.014096+0000","last_fullsized":"2024-09-20T18:57:32.014096+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:18.410447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:18.410447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:29.917821+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":366,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.011030+0000","last_change":"2024-09-20T18:55:28.830634+0000","last_active":"2024-09-20T18:57:32.011030+0000","last_peered":"2024-09-20T18:57:32.011030+0000","last_clean":"2024-09-20T18:57:32.011030+0000","last_became_active":"2024-09-20T18:55:28.827785+0000","last_became_peered":"2024-09-20T18:55:28.827785+0000","last_unstale":"2024-09-20T18:57:32.011030+0000","last_undegraded":"2024-09-20T18:57:32.011030+0000","last_fullsized":"2024-09-20T18:57:32.011030+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:11.401777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:11.401777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":355,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.019892+0000","last_change":"2024-09-20T18:57:23.721754+0000","last_active":"2024-09-20T18:57:32.019892+0000","last_peered":"2024-09-20T18:57:32.019892+0000","last_clean":"2024-09-20T18:57:32.019892+0000","last_became_active":"2024-09-20T18:55:28.833115+0000","last_became_peered":"2024-09-20T18:55:28.833115+0000","last_unstale":"2024-09-20T18:57:32.019892+0000","last_undegraded":"2024-09-20T18:57:32.019892+0000","last_fullsized":"2024-09-20T18:57:32.019892+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:23.721643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:23.721643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:25.737391+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1","version":"224'11922","reported_seq":12062,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:36.338102+0000","last_change":"2024-09-20T18:54:34.310237+0000","last_active":"2024-09-20T18:57:36.338102+0000","last_peered":"2024-09-20T18:57:36.338102+0000","last_clean":"2024-09-20T18:57:36.338102+0000","last_became_active":"2024-09-20T18:54:34.309781+0000","last_became_peered":"2024-09-20T18:54:34.309781+0000","last_unstale":"2024-09-20T18:57:36.338102+0000","last_undegraded":"2024-09-20T18:57:36.338102+0000","last_fullsized":"2024-09-20T18:57:36.338102+0000","mapping_epoch":160,"log_start":"199'7624","ondisk_log_start":"199'7624","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4298,"log_dups_size":0,"ondisk_log_size":4298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":164495360,"num_objects":2510,"num_object_clones":0,"num_object_copies":7530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11922,"num_write_kb":461824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":384,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.014333+0000","last_change":"2024-09-20T18:52:02.944536+0000","last_active":"2024-09-20T18:57:32.014333+0000","last_peered":"2024-09-20T18:57:32.014333+0000","last_clean":"2024-09-20T18:57:32.014333+0000","last_became_active":"2024-09-20T18:49:47.947636+0000","last_became_peered":"2024-09-20T18:49:47.947636+0000","last_unstale":"2024-09-20T18:57:32.014333+0000","last_undegraded":"2024-09-20T18:57:32.014333+0000","last_fullsized":"2024-09-20T18:57:32.014333+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-20T18:52:02.944434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:02.944434+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":221,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.014065+0000","last_change":"2024-09-20T18:55:28.822330+0000","last_active":"2024-09-20T18:57:32.014065+0000","last_peered":"2024-09-20T18:57:32.014065+0000","last_clean":"2024-09-20T18:57:32.014065+0000","last_became_active":"2024-09-20T18:55:28.821959+0000","last_became_peered":"2024-09-20T18:55:28.821959+0000","last_unstale":"2024-09-20T18:57:32.014065+0000","last_undegraded":"2024-09-20T18:57:32.014065+0000","last_fullsized":"2024-09-20T18:57:32.014065+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:45.669427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:45.669427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":352,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.013141+0000","last_change":"2024-09-20T18:54:11.377907+0000","last_active":"2024-09-20T18:57:32.013141+0000","last_peered":"2024-09-20T18:57:32.013141+0000","last_clean":"2024-09-20T18:57:32.013141+0000","last_became_active":"2024-09-20T18:49:49.959160+0000","last_became_peered":"2024-09-20T18:49:49.959160+0000","last_unstale":"2024-09-20T18:57:32.013141+0000","last_undegraded":"2024-09-20T18:57:32.013141+0000","last_fullsized":"2024-09-20T18:57:32.013141+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:11.376951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:11.376951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:29.061620+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":349,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.011200+0000","last_change":"2024-09-20T18:52:35.635897+0000","last_active":"2024-09-20T18:57:32.011200+0000","last_peered":"2024-09-20T18:57:32.011200+0000","last_clean":"2024-09-20T18:57:32.011200+0000","last_became_active":"2024-09-20T18:49:51.970108+0000","last_became_peered":"2024-09-20T18:49:51.970108+0000","last_unstale":"2024-09-20T18:57:32.011200+0000","last_undegraded":"2024-09-20T18:57:32.011200+0000","last_fullsized":"2024-09-20T18:57:32.011200+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:35.635818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:35.635818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"224'11792","reported_seq":11927,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:36.493783+0000","last_change":"2024-09-20T18:54:33.929552+0000","last_active":"2024-09-20T18:57:36.493783+0000","last_peered":"2024-09-20T18:57:36.493783+0000","last_clean":"2024-09-20T18:57:36.493783+0000","last_became_active":"2024-09-20T18:54:33.929208+0000","last_became_peered":"2024-09-20T18:54:33.929208+0000","last_unstale":"2024-09-20T18:57:36.493783+0000","last_undegraded":"2024-09-20T18:57:36.493783+0000","last_fullsized":"2024-09-20T18:57:36.493783+0000","mapping_epoch":160,"log_start":"198'7421","ondisk_log_start":"198'7421","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4371,"log_dups_size":0,"ondisk_log_size":4371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":159514624,"num_objects":2434,"num_object_clones":0,"num_object_copies":7302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11792,"num_write_kb":455232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":390,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.014591+0000","last_change":"2024-09-20T18:55:28.846773+0000","last_active":"2024-09-20T18:57:32.014591+0000","last_peered":"2024-09-20T18:57:32.014591+0000","last_clean":"2024-09-20T18:57:32.014591+0000","last_became_active":"2024-09-20T18:55:28.817101+0000","last_became_peered":"2024-09-20T18:55:28.817101+0000","last_unstale":"2024-09-20T18:57:32.014591+0000","last_undegraded":"2024-09-20T18:57:32.014591+0000","last_fullsized":"2024-09-20T18:57:32.014591+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:54:17.219548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:17.219548+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":360,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.020710+0000","last_change":"2024-09-20T18:52:46.005172+0000","last_active":"2024-09-20T18:57:32.020710+0000","last_peered":"2024-09-20T18:57:32.020710+0000","last_clean":"2024-09-20T18:57:32.020710+0000","last_became_active":"2024-09-20T18:49:45.937264+0000","last_became_peered":"2024-09-20T18:49:45.937264+0000","last_unstale":"2024-09-20T18:57:32.020710+0000","last_undegraded":"2024-09-20T18:57:32.020710+0000","last_fullsized":"2024-09-20T18:57:32.020710+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:46.005109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:46.005109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:27.755958+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":464,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.011359+0000","last_change":"2024-09-20T18:55:17.549660+0000","last_active":"2024-09-20T18:57:32.011359+0000","last_peered":"2024-09-20T18:57:32.011359+0000","last_clean":"2024-09-20T18:57:32.011359+0000","last_became_active":"2024-09-20T18:49:49.957005+0000","last_became_peered":"2024-09-20T18:49:49.957005+0000","last_unstale":"2024-09-20T18:57:32.011359+0000","last_undegraded":"2024-09-20T18:57:32.011359+0000","last_fullsized":"2024-09-20T18:57:32.011359+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:17.549598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:17.549598+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":348,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.014194+0000","last_change":"2024-09-20T18:54:42.182875+0000","last_active":"2024-09-20T18:57:32.014194+0000","last_peered":"2024-09-20T18:57:32.014194+0000","last_clean":"2024-09-20T18:57:32.014194+0000","last_became_active":"2024-09-20T18:49:51.969355+0000","last_became_peered":"2024-09-20T18:49:51.969355+0000","last_unstale":"2024-09-20T18:57:32.014194+0000","last_undegraded":"2024-09-20T18:57:32.014194+0000","last_fullsized":"2024-09-20T18:57:32.014194+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:42.182809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:42.182809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"224'12077","reported_seq":24485,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:36.254146+0000","last_change":"2024-09-20T18:57:00.083015+0000","last_active":"2024-09-20T18:57:36.254146+0000","last_peered":"2024-09-20T18:57:36.254146+0000","last_clean":"2024-09-20T18:57:36.254146+0000","last_became_active":"2024-09-20T18:57:00.060960+0000","last_became_peered":"2024-09-20T18:57:00.060960+0000","last_unstale":"2024-09-20T18:57:36.254146+0000","last_undegraded":"2024-09-20T18:57:36.254146+0000","last_fullsized":"2024-09-20T18:57:36.254146+0000","mapping_epoch":213,"log_start":"199'7761","ondisk_log_start":"199'7761","created":160,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4316,"log_dups_size":0,"ondisk_log_size":4316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:34.055353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":162070528,"num_objects":2473,"num_object_clones":0,"num_object_copies":7419,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12077,"num_write_kb":465600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6125,"num_bytes_recovered":401408000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":381,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T18:57:36.724251+0000","last_change":"2024-09-20T18:54:54.897991+0000","last_active":"2024-09-20T18:57:36.724251+0000","last_peered":"2024-09-20T18:57:36.724251+0000","last_clean":"2024-09-20T18:57:36.724251+0000","last_became_active":"2024-09-20T18:49:47.946008+0000","last_became_peered":"2024-09-20T18:49:47.946008+0000","last_unstale":"2024-09-20T18:57:36.724251+0000","last_undegraded":"2024-09-20T18:57:36.724251+0000","last_fullsized":"2024-09-20T18:57:36.724251+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:54.897933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:54.897933+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":374,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.010890+0000","last_change":"2024-09-20T18:55:28.833453+0000","last_active":"2024-09-20T18:57:32.010890+0000","last_peered":"2024-09-20T18:57:32.010890+0000","last_clean":"2024-09-20T18:57:32.010890+0000","last_became_active":"2024-09-20T18:55:28.833095+0000","last_became_peered":"2024-09-20T18:55:28.833095+0000","last_unstale":"2024-09-20T18:57:32.010890+0000","last_undegraded":"2024-09-20T18:57:32.010890+0000","last_fullsized":"2024-09-20T18:57:32.010890+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:56.212378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:56.212378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":361,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.009793+0000","last_change":"2024-09-20T18:54:43.379898+0000","last_active":"2024-09-20T18:57:32.009793+0000","last_peered":"2024-09-20T18:57:32.009793+0000","last_clean":"2024-09-20T18:57:32.009793+0000","last_became_active":"2024-09-20T18:49:49.956767+0000","last_became_peered":"2024-09-20T18:49:49.956767+0000","last_unstale":"2024-09-20T18:57:32.009793+0000","last_undegraded":"2024-09-20T18:57:32.009793+0000","last_fullsized":"2024-09-20T18:57:32.009793+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:43.379706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:43.379706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":349,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T18:57:36.727652+0000","last_change":"2024-09-20T18:54:24.976627+0000","last_active":"2024-09-20T18:57:36.727652+0000","last_peered":"2024-09-20T18:57:36.727652+0000","last_clean":"2024-09-20T18:57:36.727652+0000","last_became_active":"2024-09-20T18:49:51.964098+0000","last_became_peered":"2024-09-20T18:49:51.964098+0000","last_unstale":"2024-09-20T18:57:36.727652+0000","last_undegraded":"2024-09-20T18:57:36.727652+0000","last_fullsized":"2024-09-20T18:57:36.727652+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:24.976584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:24.976584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"225'12001","reported_seq":17185,"reported_epoch":225,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-20T18:57:36.898327+0000","last_change":"2024-09-20T18:56:27.871592+0000","last_active":"2024-09-20T18:57:36.898327+0000","last_peered":"2024-09-20T18:57:36.898327+0000","last_clean":"2024-09-20T18:55:27.791097+0000","last_became_active":"2024-09-20T18:55:29.979932+0000","last_became_peered":"2024-09-20T18:55:29.979932+0000","last_unstale":"2024-09-20T18:57:36.898327+0000","last_undegraded":"2024-09-20T18:56:27.867865+0000","last_fullsized":"2024-09-20T18:55:29.815116+0000","mapping_epoch":182,"log_start":"195'7140","ondisk_log_start":"195'7140","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4861,"log_dups_size":0,"ondisk_log_size":4861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":160366592,"num_objects":2447,"num_object_clones":0,"num_object_copies":7341,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2256,"num_objects_misplaced":401,"num_objects_unfound":0,"num_objects_dirty":2447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12001,"num_write_kb":462336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2521,"num_bytes_recovered":165216256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7],"avail_no_missing":["0","6","7"],"object_location_counts":[{"shards":"0,6,7","objects":401},{"shards":"0,7","objects":-210}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":176,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.013504+0000","last_change":"2024-09-20T18:57:22.033602+0000","last_active":"2024-09-20T18:57:32.013504+0000","last_peered":"2024-09-20T18:57:32.013504+0000","last_clean":"2024-09-20T18:57:32.013504+0000","last_became_active":"2024-09-20T18:55:28.828563+0000","last_became_peered":"2024-09-20T18:55:28.828563+0000","last_unstale":"2024-09-20T18:57:32.013504+0000","last_undegraded":"2024-09-20T18:57:32.013504+0000","last_fullsized":"2024-09-20T18:57:32.013504+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-20T18:57:22.033475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:22.033475+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:43.998101+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":31,"num_read_kb":20,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,5],"acting":[1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":374,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.005794+0000","last_change":"2024-09-20T18:55:28.835767+0000","last_active":"2024-09-20T18:57:32.005794+0000","last_peered":"2024-09-20T18:57:32.005794+0000","last_clean":"2024-09-20T18:57:32.005794+0000","last_became_active":"2024-09-20T18:55:28.835149+0000","last_became_peered":"2024-09-20T18:55:28.835149+0000","last_unstale":"2024-09-20T18:57:32.005794+0000","last_undegraded":"2024-09-20T18:57:32.005794+0000","last_fullsized":"2024-09-20T18:57:32.005794+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:44.501192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.501192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":361,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.011081+0000","last_change":"2024-09-20T18:55:13.577772+0000","last_active":"2024-09-20T18:57:32.011081+0000","last_peered":"2024-09-20T18:57:32.011081+0000","last_clean":"2024-09-20T18:57:32.011081+0000","last_became_active":"2024-09-20T18:49:49.957558+0000","last_became_peered":"2024-09-20T18:49:49.957558+0000","last_unstale":"2024-09-20T18:57:32.011081+0000","last_undegraded":"2024-09-20T18:57:32.011081+0000","last_fullsized":"2024-09-20T18:57:32.011081+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:13.577699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:13.577699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":355,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.004993+0000","last_change":"2024-09-20T18:53:47.370152+0000","last_active":"2024-09-20T18:57:32.004993+0000","last_peered":"2024-09-20T18:57:32.004993+0000","last_clean":"2024-09-20T18:57:32.004993+0000","last_became_active":"2024-09-20T18:49:51.970798+0000","last_became_peered":"2024-09-20T18:49:51.970798+0000","last_unstale":"2024-09-20T18:57:32.004993+0000","last_undegraded":"2024-09-20T18:57:32.004993+0000","last_fullsized":"2024-09-20T18:57:32.004993+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:47.370108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:47.370108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"224'12374","reported_seq":23307,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:35.918336+0000","last_change":"2024-09-20T18:57:20.750334+0000","last_active":"2024-09-20T18:57:35.918336+0000","last_peered":"2024-09-20T18:57:35.918336+0000","last_clean":"2024-09-20T18:57:35.918336+0000","last_became_active":"2024-09-20T18:57:20.718369+0000","last_became_peered":"2024-09-20T18:57:20.718369+0000","last_unstale":"2024-09-20T18:57:35.918336+0000","last_undegraded":"2024-09-20T18:57:35.918336+0000","last_fullsized":"2024-09-20T18:57:35.918336+0000","mapping_epoch":221,"log_start":"197'7746","ondisk_log_start":"197'7746","created":160,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4628,"log_dups_size":0,"ondisk_log_size":4628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":171180032,"num_objects":2612,"num_object_clones":0,"num_object_copies":7836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12374,"num_write_kb":479552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5391,"num_bytes_recovered":353304576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":379,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.013905+0000","last_change":"2024-09-20T18:53:52.898818+0000","last_active":"2024-09-20T18:57:32.013905+0000","last_peered":"2024-09-20T18:57:32.013905+0000","last_clean":"2024-09-20T18:57:32.013905+0000","last_became_active":"2024-09-20T18:49:47.949116+0000","last_became_peered":"2024-09-20T18:49:47.949116+0000","last_unstale":"2024-09-20T18:57:32.013905+0000","last_undegraded":"2024-09-20T18:57:32.013905+0000","last_fullsized":"2024-09-20T18:57:32.013905+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:53:52.898760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:52.898760+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":367,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.005553+0000","last_change":"2024-09-20T18:54:52.650798+0000","last_active":"2024-09-20T18:57:32.005553+0000","last_peered":"2024-09-20T18:57:32.005553+0000","last_clean":"2024-09-20T18:57:32.005553+0000","last_became_active":"2024-09-20T18:49:45.944858+0000","last_became_peered":"2024-09-20T18:49:45.944858+0000","last_unstale":"2024-09-20T18:57:32.005553+0000","last_undegraded":"2024-09-20T18:57:32.005553+0000","last_fullsized":"2024-09-20T18:57:32.005553+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:52.650756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:52.650756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":353,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.009933+0000","last_change":"2024-09-20T18:54:42.349879+0000","last_active":"2024-09-20T18:57:32.009933+0000","last_peered":"2024-09-20T18:57:32.009933+0000","last_clean":"2024-09-20T18:57:32.009933+0000","last_became_active":"2024-09-20T18:49:49.954944+0000","last_became_peered":"2024-09-20T18:49:49.954944+0000","last_unstale":"2024-09-20T18:57:32.009933+0000","last_undegraded":"2024-09-20T18:57:32.009933+0000","last_fullsized":"2024-09-20T18:57:32.009933+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:42.349807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:42.349807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":348,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.020494+0000","last_change":"2024-09-20T18:55:17.341862+0000","last_active":"2024-09-20T18:57:32.020494+0000","last_peered":"2024-09-20T18:57:32.020494+0000","last_clean":"2024-09-20T18:57:32.020494+0000","last_became_active":"2024-09-20T18:49:51.972970+0000","last_became_peered":"2024-09-20T18:49:51.972970+0000","last_unstale":"2024-09-20T18:57:32.020494+0000","last_undegraded":"2024-09-20T18:57:32.020494+0000","last_fullsized":"2024-09-20T18:57:32.020494+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:17.341793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:17.341793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.4","version":"224'11809","reported_seq":11949,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:35.867794+0000","last_change":"2024-09-20T18:54:34.309442+0000","last_active":"2024-09-20T18:57:35.867794+0000","last_peered":"2024-09-20T18:57:35.867794+0000","last_clean":"2024-09-20T18:57:35.867794+0000","last_became_active":"2024-09-20T18:54:34.309113+0000","last_became_peered":"2024-09-20T18:54:34.309113+0000","last_unstale":"2024-09-20T18:57:35.867794+0000","last_undegraded":"2024-09-20T18:57:35.867794+0000","last_fullsized":"2024-09-20T18:57:35.867794+0000","mapping_epoch":160,"log_start":"192'6731","ondisk_log_start":"192'6731","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":5078,"log_dups_size":0,"ondisk_log_size":5078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":165347328,"num_objects":2523,"num_object_clones":0,"num_object_copies":7569,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2523,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11809,"num_write_kb":458624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":403,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.011275+0000","last_change":"2024-09-20T18:55:28.847010+0000","last_active":"2024-09-20T18:57:32.011275+0000","last_peered":"2024-09-20T18:57:32.011275+0000","last_clean":"2024-09-20T18:57:32.011275+0000","last_became_active":"2024-09-20T18:55:28.824801+0000","last_became_peered":"2024-09-20T18:55:28.824801+0000","last_unstale":"2024-09-20T18:57:32.011275+0000","last_undegraded":"2024-09-20T18:57:32.011275+0000","last_fullsized":"2024-09-20T18:57:32.011275+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:54:12.373309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:12.373309+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:29.516079+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":387,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.005427+0000","last_change":"2024-09-20T18:55:24.729301+0000","last_active":"2024-09-20T18:57:32.005427+0000","last_peered":"2024-09-20T18:57:32.005427+0000","last_clean":"2024-09-20T18:57:32.005427+0000","last_became_active":"2024-09-20T18:49:45.944256+0000","last_became_peered":"2024-09-20T18:49:45.944256+0000","last_unstale":"2024-09-20T18:57:32.005427+0000","last_undegraded":"2024-09-20T18:57:32.005427+0000","last_fullsized":"2024-09-20T18:57:32.005427+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-20T18:55:24.729219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:24.729219+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"184'9","reported_seq":456,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.005129+0000","last_change":"2024-09-20T18:55:28.824771+0000","last_active":"2024-09-20T18:57:32.005129+0000","last_peered":"2024-09-20T18:57:32.005129+0000","last_clean":"2024-09-20T18:57:32.005129+0000","last_became_active":"2024-09-20T18:55:28.818085+0000","last_became_peered":"2024-09-20T18:55:28.818085+0000","last_unstale":"2024-09-20T18:57:32.005129+0000","last_undegraded":"2024-09-20T18:57:32.005129+0000","last_fullsized":"2024-09-20T18:57:32.005129+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:52:05.788924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.788924+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":354,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.010822+0000","last_change":"2024-09-20T18:55:28.832276+0000","last_active":"2024-09-20T18:57:32.010822+0000","last_peered":"2024-09-20T18:57:32.010822+0000","last_clean":"2024-09-20T18:57:32.010822+0000","last_became_active":"2024-09-20T18:55:28.830380+0000","last_became_peered":"2024-09-20T18:55:28.830380+0000","last_unstale":"2024-09-20T18:57:32.010822+0000","last_undegraded":"2024-09-20T18:57:32.010822+0000","last_fullsized":"2024-09-20T18:57:32.010822+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:14.247398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:14.247398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":99,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T18:57:36.723399+0000","last_change":"2024-09-20T18:55:28.835344+0000","last_active":"2024-09-20T18:57:36.723399+0000","last_peered":"2024-09-20T18:57:36.723399+0000","last_clean":"2024-09-20T18:57:36.723399+0000","last_became_active":"2024-09-20T18:55:28.835010+0000","last_became_peered":"2024-09-20T18:55:28.835010+0000","last_unstale":"2024-09-20T18:57:36.723399+0000","last_undegraded":"2024-09-20T18:57:36.723399+0000","last_fullsized":"2024-09-20T18:57:36.723399+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:26.721735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:26.721735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":365,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.021188+0000","last_change":"2024-09-20T18:54:44.194161+0000","last_active":"2024-09-20T18:57:32.021188+0000","last_peered":"2024-09-20T18:57:32.021188+0000","last_clean":"2024-09-20T18:57:32.021188+0000","last_became_active":"2024-09-20T18:49:48.500662+0000","last_became_peered":"2024-09-20T18:49:48.500662+0000","last_unstale":"2024-09-20T18:57:32.021188+0000","last_undegraded":"2024-09-20T18:57:32.021188+0000","last_fullsized":"2024-09-20T18:57:32.021188+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-20T18:54:44.194089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.194089+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":351,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.010888+0000","last_change":"2024-09-20T18:53:16.220554+0000","last_active":"2024-09-20T18:57:32.010888+0000","last_peered":"2024-09-20T18:57:32.010888+0000","last_clean":"2024-09-20T18:57:32.010888+0000","last_became_active":"2024-09-20T18:49:49.956749+0000","last_became_peered":"2024-09-20T18:49:49.956749+0000","last_unstale":"2024-09-20T18:57:32.010888+0000","last_undegraded":"2024-09-20T18:57:32.010888+0000","last_fullsized":"2024-09-20T18:57:32.010888+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:16.220490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:16.220490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:27.545117+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":362,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.005421+0000","last_change":"2024-09-20T18:55:28.827897+0000","last_active":"2024-09-20T18:57:32.005421+0000","last_peered":"2024-09-20T18:57:32.005421+0000","last_clean":"2024-09-20T18:57:32.005421+0000","last_became_active":"2024-09-20T18:55:28.827418+0000","last_became_peered":"2024-09-20T18:55:28.827418+0000","last_unstale":"2024-09-20T18:57:32.005421+0000","last_undegraded":"2024-09-20T18:57:32.005421+0000","last_fullsized":"2024-09-20T18:57:32.005421+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.637739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.637739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":373,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T18:57:36.727907+0000","last_change":"2024-09-20T18:54:44.830872+0000","last_active":"2024-09-20T18:57:36.727907+0000","last_peered":"2024-09-20T18:57:36.727907+0000","last_clean":"2024-09-20T18:57:36.727907+0000","last_became_active":"2024-09-20T18:49:48.500803+0000","last_became_peered":"2024-09-20T18:49:48.500803+0000","last_unstale":"2024-09-20T18:57:36.727907+0000","last_undegraded":"2024-09-20T18:57:36.727907+0000","last_fullsized":"2024-09-20T18:57:36.727907+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:44.830790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.830790+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":366,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.005659+0000","last_change":"2024-09-20T18:55:29.751718+0000","last_active":"2024-09-20T18:57:32.005659+0000","last_peered":"2024-09-20T18:57:32.005659+0000","last_clean":"2024-09-20T18:57:32.005659+0000","last_became_active":"2024-09-20T18:55:28.833799+0000","last_became_peered":"2024-09-20T18:55:28.833799+0000","last_unstale":"2024-09-20T18:57:32.005659+0000","last_undegraded":"2024-09-20T18:57:32.005659+0000","last_fullsized":"2024-09-20T18:57:32.005659+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:29.751642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:29.751642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":352,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.021051+0000","last_change":"2024-09-20T18:53:50.253721+0000","last_active":"2024-09-20T18:57:32.021051+0000","last_peered":"2024-09-20T18:57:32.021051+0000","last_clean":"2024-09-20T18:57:32.021051+0000","last_became_active":"2024-09-20T18:49:49.961947+0000","last_became_peered":"2024-09-20T18:49:49.961947+0000","last_unstale":"2024-09-20T18:57:32.021051+0000","last_undegraded":"2024-09-20T18:57:32.021051+0000","last_fullsized":"2024-09-20T18:57:32.021051+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:50.253660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:50.253660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"66'1","reported_seq":358,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.010234+0000","last_change":"2024-09-20T18:54:20.681878+0000","last_active":"2024-09-20T18:57:32.010234+0000","last_peered":"2024-09-20T18:57:32.010234+0000","last_clean":"2024-09-20T18:57:32.010234+0000","last_became_active":"2024-09-20T18:49:51.965121+0000","last_became_peered":"2024-09-20T18:49:51.965121+0000","last_unstale":"2024-09-20T18:57:32.010234+0000","last_undegraded":"2024-09-20T18:57:32.010234+0000","last_fullsized":"2024-09-20T18:57:32.010234+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T18:54:20.681790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:20.681790+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":363,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.010961+0000","last_change":"2024-09-20T18:55:12.205118+0000","last_active":"2024-09-20T18:57:32.010961+0000","last_peered":"2024-09-20T18:57:32.010961+0000","last_clean":"2024-09-20T18:57:32.010961+0000","last_became_active":"2024-09-20T18:49:48.501011+0000","last_became_peered":"2024-09-20T18:49:48.501011+0000","last_unstale":"2024-09-20T18:57:32.010961+0000","last_undegraded":"2024-09-20T18:57:32.010961+0000","last_fullsized":"2024-09-20T18:57:32.010961+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:55:12.205069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:12.205069+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":369,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T18:57:36.724217+0000","last_change":"2024-09-20T18:55:27.028184+0000","last_active":"2024-09-20T18:57:36.724217+0000","last_peered":"2024-09-20T18:57:36.724217+0000","last_clean":"2024-09-20T18:57:36.724217+0000","last_became_active":"2024-09-20T18:49:45.938511+0000","last_became_peered":"2024-09-20T18:49:45.938511+0000","last_unstale":"2024-09-20T18:57:36.724217+0000","last_undegraded":"2024-09-20T18:57:36.724217+0000","last_fullsized":"2024-09-20T18:57:36.724217+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:27.028126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:27.028126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":463,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.020106+0000","last_change":"2024-09-20T18:55:27.284156+0000","last_active":"2024-09-20T18:57:32.020106+0000","last_peered":"2024-09-20T18:57:32.020106+0000","last_clean":"2024-09-20T18:57:32.020106+0000","last_became_active":"2024-09-20T18:49:49.960550+0000","last_became_peered":"2024-09-20T18:49:49.960550+0000","last_unstale":"2024-09-20T18:57:32.020106+0000","last_undegraded":"2024-09-20T18:57:32.020106+0000","last_fullsized":"2024-09-20T18:57:32.020106+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:27.284086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:27.284086+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":356,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.014421+0000","last_change":"2024-09-20T18:55:12.348079+0000","last_active":"2024-09-20T18:57:32.014421+0000","last_peered":"2024-09-20T18:57:32.014421+0000","last_clean":"2024-09-20T18:57:32.014421+0000","last_became_active":"2024-09-20T18:49:51.966254+0000","last_became_peered":"2024-09-20T18:49:51.966254+0000","last_unstale":"2024-09-20T18:57:32.014421+0000","last_undegraded":"2024-09-20T18:57:32.014421+0000","last_fullsized":"2024-09-20T18:57:32.014421+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:12.347993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:12.347993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":390,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.020261+0000","last_change":"2024-09-20T18:57:20.701592+0000","last_active":"2024-09-20T18:57:32.020261+0000","last_peered":"2024-09-20T18:57:32.020261+0000","last_clean":"2024-09-20T18:57:32.020261+0000","last_became_active":"2024-09-20T18:49:58.997494+0000","last_became_peered":"2024-09-20T18:49:58.997494+0000","last_unstale":"2024-09-20T18:57:32.020261+0000","last_undegraded":"2024-09-20T18:57:32.020261+0000","last_fullsized":"2024-09-20T18:57:32.020261+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:57:20.701543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:20.701543+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:44.352173+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":367,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.005509+0000","last_change":"2024-09-20T18:54:57.632458+0000","last_active":"2024-09-20T18:57:32.005509+0000","last_peered":"2024-09-20T18:57:32.005509+0000","last_clean":"2024-09-20T18:57:32.005509+0000","last_became_active":"2024-09-20T18:49:45.949492+0000","last_became_peered":"2024-09-20T18:49:45.949492+0000","last_unstale":"2024-09-20T18:57:32.005509+0000","last_undegraded":"2024-09-20T18:57:32.005509+0000","last_fullsized":"2024-09-20T18:57:32.005509+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.632380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.632380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":461,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.010795+0000","last_change":"2024-09-20T18:55:28.302353+0000","last_active":"2024-09-20T18:57:32.010795+0000","last_peered":"2024-09-20T18:57:32.010795+0000","last_clean":"2024-09-20T18:57:32.010795+0000","last_became_active":"2024-09-20T18:49:49.962243+0000","last_became_peered":"2024-09-20T18:49:49.962243+0000","last_unstale":"2024-09-20T18:57:32.010795+0000","last_undegraded":"2024-09-20T18:57:32.010795+0000","last_fullsized":"2024-09-20T18:57:32.010795+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:28.302303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:28.302303+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":357,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T18:57:36.725040+0000","last_change":"2024-09-20T18:55:11.918284+0000","last_active":"2024-09-20T18:57:36.725040+0000","last_peered":"2024-09-20T18:57:36.725040+0000","last_clean":"2024-09-20T18:57:36.725040+0000","last_became_active":"2024-09-20T18:49:51.969093+0000","last_became_peered":"2024-09-20T18:49:51.969093+0000","last_unstale":"2024-09-20T18:57:36.725040+0000","last_undegraded":"2024-09-20T18:57:36.725040+0000","last_fullsized":"2024-09-20T18:57:36.725040+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:11.918210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:11.918210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":394,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.013848+0000","last_change":"2024-09-20T18:55:28.854249+0000","last_active":"2024-09-20T18:57:32.013848+0000","last_peered":"2024-09-20T18:57:32.013848+0000","last_clean":"2024-09-20T18:57:32.013848+0000","last_became_active":"2024-09-20T18:55:28.826721+0000","last_became_peered":"2024-09-20T18:55:28.826721+0000","last_unstale":"2024-09-20T18:57:32.013848+0000","last_undegraded":"2024-09-20T18:57:32.013848+0000","last_fullsized":"2024-09-20T18:57:32.013848+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:53:43.878921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:43.878921+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":359,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.014650+0000","last_change":"2024-09-20T18:55:28.822690+0000","last_active":"2024-09-20T18:57:32.014650+0000","last_peered":"2024-09-20T18:57:32.014650+0000","last_clean":"2024-09-20T18:57:32.014650+0000","last_became_active":"2024-09-20T18:55:28.822258+0000","last_became_peered":"2024-09-20T18:55:28.822258+0000","last_unstale":"2024-09-20T18:57:32.014650+0000","last_undegraded":"2024-09-20T18:57:32.014650+0000","last_fullsized":"2024-09-20T18:57:32.014650+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:05.985551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.985551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":352,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.013250+0000","last_change":"2024-09-20T18:57:11.986605+0000","last_active":"2024-09-20T18:57:32.013250+0000","last_peered":"2024-09-20T18:57:32.013250+0000","last_clean":"2024-09-20T18:57:32.013250+0000","last_became_active":"2024-09-20T18:49:49.957067+0000","last_became_peered":"2024-09-20T18:49:49.957067+0000","last_unstale":"2024-09-20T18:57:32.013250+0000","last_undegraded":"2024-09-20T18:57:32.013250+0000","last_fullsized":"2024-09-20T18:57:32.013250+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:11.986528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:11.986528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:36.007001+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":348,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.018517+0000","last_change":"2024-09-20T18:57:16.789412+0000","last_active":"2024-09-20T18:57:32.018517+0000","last_peered":"2024-09-20T18:57:32.018517+0000","last_clean":"2024-09-20T18:57:32.018517+0000","last_became_active":"2024-09-20T18:49:51.966915+0000","last_became_peered":"2024-09-20T18:49:51.966915+0000","last_unstale":"2024-09-20T18:57:32.018517+0000","last_undegraded":"2024-09-20T18:57:32.018517+0000","last_fullsized":"2024-09-20T18:57:32.018517+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:16.789095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:16.789095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:45.821909+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":397,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.013675+0000","last_change":"2024-09-20T18:55:28.838698+0000","last_active":"2024-09-20T18:57:32.013675+0000","last_peered":"2024-09-20T18:57:32.013675+0000","last_clean":"2024-09-20T18:57:32.013675+0000","last_became_active":"2024-09-20T18:55:28.827023+0000","last_became_peered":"2024-09-20T18:55:28.827023+0000","last_unstale":"2024-09-20T18:57:32.013675+0000","last_undegraded":"2024-09-20T18:57:32.013675+0000","last_fullsized":"2024-09-20T18:57:32.013675+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:53:53.263976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:53.263976+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:26.131891+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":359,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.005736+0000","last_change":"2024-09-20T18:54:42.487465+0000","last_active":"2024-09-20T18:57:32.005736+0000","last_peered":"2024-09-20T18:57:32.005736+0000","last_clean":"2024-09-20T18:57:32.005736+0000","last_became_active":"2024-09-20T18:49:45.944357+0000","last_became_peered":"2024-09-20T18:49:45.944357+0000","last_unstale":"2024-09-20T18:57:32.005736+0000","last_undegraded":"2024-09-20T18:57:32.005736+0000","last_fullsized":"2024-09-20T18:57:32.005736+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:42.487403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:42.487403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":352,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.020577+0000","last_change":"2024-09-20T18:54:18.108239+0000","last_active":"2024-09-20T18:57:32.020577+0000","last_peered":"2024-09-20T18:57:32.020577+0000","last_clean":"2024-09-20T18:57:32.020577+0000","last_became_active":"2024-09-20T18:49:49.962523+0000","last_became_peered":"2024-09-20T18:49:49.962523+0000","last_unstale":"2024-09-20T18:57:32.020577+0000","last_undegraded":"2024-09-20T18:57:32.020577+0000","last_fullsized":"2024-09-20T18:57:32.020577+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:18.108175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:18.108175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":357,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T18:57:36.722928+0000","last_change":"2024-09-20T18:54:26.882441+0000","last_active":"2024-09-20T18:57:36.722928+0000","last_peered":"2024-09-20T18:57:36.722928+0000","last_clean":"2024-09-20T18:57:36.722928+0000","last_became_active":"2024-09-20T18:49:51.967313+0000","last_became_peered":"2024-09-20T18:49:51.967313+0000","last_unstale":"2024-09-20T18:57:36.722928+0000","last_undegraded":"2024-09-20T18:57:36.722928+0000","last_fullsized":"2024-09-20T18:57:36.722928+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:26.882375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:26.882375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":394,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.018632+0000","last_change":"2024-09-20T18:55:28.843363+0000","last_active":"2024-09-20T18:57:32.018632+0000","last_peered":"2024-09-20T18:57:32.018632+0000","last_clean":"2024-09-20T18:57:32.018632+0000","last_became_active":"2024-09-20T18:55:28.830801+0000","last_became_peered":"2024-09-20T18:55:28.830801+0000","last_unstale":"2024-09-20T18:57:32.018632+0000","last_undegraded":"2024-09-20T18:57:32.018632+0000","last_fullsized":"2024-09-20T18:57:32.018632+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:54:21.106677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:21.106677+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":367,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.011061+0000","last_change":"2024-09-20T18:55:19.258723+0000","last_active":"2024-09-20T18:57:32.011061+0000","last_peered":"2024-09-20T18:57:32.011061+0000","last_clean":"2024-09-20T18:57:32.011061+0000","last_became_active":"2024-09-20T18:49:45.942105+0000","last_became_peered":"2024-09-20T18:49:45.942105+0000","last_unstale":"2024-09-20T18:57:32.011061+0000","last_undegraded":"2024-09-20T18:57:32.011061+0000","last_fullsized":"2024-09-20T18:57:32.011061+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:19.258674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:19.258674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":366,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.004818+0000","last_change":"2024-09-20T18:55:28.825004+0000","last_active":"2024-09-20T18:57:32.004818+0000","last_peered":"2024-09-20T18:57:32.004818+0000","last_clean":"2024-09-20T18:57:32.004818+0000","last_became_active":"2024-09-20T18:55:28.824355+0000","last_became_peered":"2024-09-20T18:55:28.824355+0000","last_unstale":"2024-09-20T18:57:32.004818+0000","last_undegraded":"2024-09-20T18:57:32.004818+0000","last_fullsized":"2024-09-20T18:57:32.004818+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:46.329547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:46.329547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":347,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.020690+0000","last_change":"2024-09-20T18:55:28.825607+0000","last_active":"2024-09-20T18:57:32.020690+0000","last_peered":"2024-09-20T18:57:32.020690+0000","last_clean":"2024-09-20T18:57:32.020690+0000","last_became_active":"2024-09-20T18:55:28.825296+0000","last_became_peered":"2024-09-20T18:55:28.825296+0000","last_unstale":"2024-09-20T18:57:32.020690+0000","last_undegraded":"2024-09-20T18:57:32.020690+0000","last_fullsized":"2024-09-20T18:57:32.020690+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:43.118805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:43.118805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:25.734066+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":383,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.025254+0000","last_change":"2024-09-20T18:55:28.836789+0000","last_active":"2024-09-20T18:57:32.025254+0000","last_peered":"2024-09-20T18:57:32.025254+0000","last_clean":"2024-09-20T18:57:32.025254+0000","last_became_active":"2024-09-20T18:55:28.824579+0000","last_became_peered":"2024-09-20T18:55:28.824579+0000","last_unstale":"2024-09-20T18:57:32.025254+0000","last_undegraded":"2024-09-20T18:57:32.025254+0000","last_fullsized":"2024-09-20T18:57:32.025254+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-20T18:55:20.282193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:20.282193+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":369,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T18:57:36.724829+0000","last_change":"2024-09-20T18:54:55.900487+0000","last_active":"2024-09-20T18:57:36.724829+0000","last_peered":"2024-09-20T18:57:36.724829+0000","last_clean":"2024-09-20T18:57:36.724829+0000","last_became_active":"2024-09-20T18:49:45.940988+0000","last_became_peered":"2024-09-20T18:49:45.940988+0000","last_unstale":"2024-09-20T18:57:36.724829+0000","last_undegraded":"2024-09-20T18:57:36.724829+0000","last_fullsized":"2024-09-20T18:57:36.724829+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:55.899611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:55.899611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":97,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.013628+0000","last_change":"2024-09-20T18:55:28.832864+0000","last_active":"2024-09-20T18:57:32.013628+0000","last_peered":"2024-09-20T18:57:32.013628+0000","last_clean":"2024-09-20T18:57:32.013628+0000","last_became_active":"2024-09-20T18:55:28.832534+0000","last_became_peered":"2024-09-20T18:55:28.832534+0000","last_unstale":"2024-09-20T18:57:32.013628+0000","last_undegraded":"2024-09-20T18:57:32.013628+0000","last_fullsized":"2024-09-20T18:57:32.013628+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:52.635138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:52.635138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":339,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.004946+0000","last_change":"2024-09-20T18:51:09.761331+0000","last_active":"2024-09-20T18:57:32.004946+0000","last_peered":"2024-09-20T18:57:32.004946+0000","last_clean":"2024-09-20T18:57:32.004946+0000","last_became_active":"2024-09-20T18:49:51.968662+0000","last_became_peered":"2024-09-20T18:49:51.968662+0000","last_unstale":"2024-09-20T18:57:32.004946+0000","last_undegraded":"2024-09-20T18:57:32.004946+0000","last_fullsized":"2024-09-20T18:57:32.004946+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:09.761283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:09.761283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":340,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.014977+0000","last_change":"2024-09-20T18:57:10.970843+0000","last_active":"2024-09-20T18:57:32.014977+0000","last_peered":"2024-09-20T18:57:32.014977+0000","last_clean":"2024-09-20T18:57:32.014977+0000","last_became_active":"2024-09-20T18:49:51.968216+0000","last_became_peered":"2024-09-20T18:49:51.968216+0000","last_unstale":"2024-09-20T18:57:32.014977+0000","last_undegraded":"2024-09-20T18:57:32.014977+0000","last_fullsized":"2024-09-20T18:57:32.014977+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:10.970779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:10.970779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:38.733512+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":360,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.025482+0000","last_change":"2024-09-20T18:55:28.295445+0000","last_active":"2024-09-20T18:57:32.025482+0000","last_peered":"2024-09-20T18:57:32.025482+0000","last_clean":"2024-09-20T18:57:32.025482+0000","last_became_active":"2024-09-20T18:49:45.937992+0000","last_became_peered":"2024-09-20T18:49:45.937992+0000","last_unstale":"2024-09-20T18:57:32.025482+0000","last_undegraded":"2024-09-20T18:57:32.025482+0000","last_fullsized":"2024-09-20T18:57:32.025482+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:28.295399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:28.295399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":149,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-20T18:57:32.011317+0000","last_change":"2024-09-20T18:55:28.852468+0000","last_active":"2024-09-20T18:57:32.011317+0000","last_peered":"2024-09-20T18:57:32.011317+0000","last_clean":"2024-09-20T18:57:32.011317+0000","last_became_active":"2024-09-20T18:55:28.834675+0000","last_became_peered":"2024-09-20T18:55:28.834675+0000","last_unstale":"2024-09-20T18:57:32.011317+0000","last_undegraded":"2024-09-20T18:57:32.011317+0000","last_fullsized":"2024-09-20T18:57:32.011317+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:54:44.719676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.719676+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":12,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":345,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T18:57:36.727680+0000","last_change":"2024-09-20T18:51:28.120100+0000","last_active":"2024-09-20T18:57:36.727680+0000","last_peered":"2024-09-20T18:57:36.727680+0000","last_clean":"2024-09-20T18:57:36.727680+0000","last_became_active":"2024-09-20T18:49:49.957582+0000","last_became_peered":"2024-09-20T18:49:49.957582+0000","last_unstale":"2024-09-20T18:57:36.727680+0000","last_undegraded":"2024-09-20T18:57:36.727680+0000","last_fullsized":"2024-09-20T18:57:36.727680+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:28.119694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:28.119694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":2622160920,"num_objects":40012,"num_object_clones":0,"num_object_copies":120036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2256,"num_objects_misplaced":401,"num_objects_unfound":0,"num_objects_dirty":40012,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":192166,"num_write_kb":7429633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25399,"num_bytes_recovered":1664548864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":9312481280,"data_stored":9312469064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74199,"ondisk_log_size":74199,"up":48,"acting":47,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":67,"ondisk_log_size":67,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":695,"num_read_kb":452,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":1474,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":12140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":394,"ondisk_log_size":394,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":5275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":465,"num_read_kb":460,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266786,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1030704,"kb_used_data":819040,"kb_used_omap":1,"kb_used_meta":211646,"kb_avail":92714448,"statfs":{"total":95995035648,"available":94939594752,"internally_reserved":0,"allocated":838696960,"data_stored":838268538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":216725962},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":0,"seq":193273528425,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757358,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1540584,"kb_used_data":1296984,"kb_used_omap":1,"kb_used_meta":243582,"kb_avail":92204568,"statfs":{"total":95995035648,"available":94417477632,"internally_reserved":0,"allocated":1328111616,"data_stored":1327690869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":249428421},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986291,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1027580,"kb_used_data":820076,"kb_used_omap":1,"kb_used_meta":207486,"kb_avail":92717572,"statfs":{"total":95995035648,"available":94942793728,"internally_reserved":0,"allocated":839757824,"data_stored":839317605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":212466117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182522,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1482108,"kb_used_data":1139948,"kb_used_omap":1,"kb_used_meta":342142,"kb_avail":92263044,"statfs":{"total":95995035648,"available":94477357056,"internally_reserved":0,"allocated":1167306752,"data_stored":1166865487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":350353863},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411455,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1795552,"kb_used_data":1413136,"kb_used_omap":1,"kb_used_meta":382398,"kb_avail":91949600,"statfs":{"total":95995035648,"available":94156390400,"internally_reserved":0,"allocated":1447051264,"data_stored":1446638958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":391576005},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574981,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1415188,"kb_used_data":1127108,"kb_used_omap":1,"kb_used_meta":287998,"kb_avail":92329964,"statfs":{"total":95995035648,"available":94545883136,"internally_reserved":0,"allocated":1154158592,"data_stored":1153695178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":294910405},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738507,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1463740,"kb_used_data":1117356,"kb_used_omap":1,"kb_used_meta":346302,"kb_avail":92281412,"statfs":{"total":95995035648,"available":94496165888,"internally_reserved":0,"allocated":1144172544,"data_stored":1143731899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":354613701},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1130299392,"data_stored":1130299392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1143275520,"data_stored":1143275520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1439764480,"data_stored":1439760408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1139539968,"data_stored":1139539968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":837877760,"data_stored":837877760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1315835904,"data_stored":1315831832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1470693376,"data_stored":1470693376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":835194880,"data_stored":835190808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T18:57:41.683 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:57:41.688 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:41.713 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:57:41.734 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:41.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:41.735 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:41.799 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:41 smithi083 ceph-mon[36260]: pgmap v378: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 2.4 GiB data, 9.2 GiB used, 617 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 655 op/s; 2256/120633 objects degraded (1.870%); 401/120633 objects misplaced (0.332%); 1024 KiB/s, 15 objects/s recovering 2024-09-20T18:57:41.800 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:41 smithi083 ceph-mon[29421]: pgmap v378: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 2.4 GiB data, 9.2 GiB used, 617 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 655 op/s; 2256/120633 objects degraded (1.870%); 401/120633 objects misplaced (0.332%); 1024 KiB/s, 15 objects/s recovering 2024-09-20T18:57:41.836 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:41.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:41.881 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:41.881 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:41.982 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:42.008 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T18:57:42.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:42.009 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:42.009 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:42.018 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:41 smithi142 ceph-mon[32508]: pgmap v378: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 2.4 GiB data, 9.2 GiB used, 617 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 655 op/s; 2256/120633 objects degraded (1.870%); 401/120633 objects misplaced (0.332%); 1024 KiB/s, 15 objects/s recovering 2024-09-20T18:57:42.111 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:42.138 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:42.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:42.139 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:42.240 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:57:42.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:42.268 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:42.269 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:57:42.369 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:42.397 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:42.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:42.398 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:42.420 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T18:57:42.420 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:57:42.499 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:42.546 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:42.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:42.547 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:42.648 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:42.684 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:42.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:42.686 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:42.786 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:42.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:42.820 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:42.820 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:42.921 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:42.948 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:42.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:42.949 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:43.050 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:57:43.080 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:43.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:43.081 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:57:43.182 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:43.207 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:43.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:43.208 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:43.249 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_historic_ops 2024-09-20T18:57:43.308 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:43.334 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:43.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:43.335 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:43.436 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:43.459 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:43.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:43.460 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:43.560 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:43.585 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:57:43.585 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:57:43.589 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:57:43.620 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:57:43.620 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:57:43.630 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:43.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:43.631 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:43.665 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:57:43.666 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:57:43.703 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:57:43.703 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:57:43.733 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:43.735 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:57:43.735 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:57:43.758 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:43.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:43.759 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:43.774 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:57:43.774 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:57:43.810 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:57:43.810 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:57:43.840 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:57:43.841 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:57:43.860 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:43.894 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:43.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:43.896 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:43.997 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:44.022 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:44.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:44.023 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:44.124 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:44.161 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:44.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:44.162 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:44.263 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:44.287 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:44.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:44.288 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:44.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:44 smithi083 ceph-mon[29421]: pgmap v379: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 2.1 GiB data, 8.4 GiB used, 617 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.33k op/s; 2458/104394 objects degraded (2.355%); 69/104394 objects misplaced (0.066%); 2.0 MiB/s, 32 objects/s recovering 2024-09-20T18:57:44.294 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:44 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3757266299' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:57:44.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:44 smithi083 ceph-mon[36260]: pgmap v379: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 2.1 GiB data, 8.4 GiB used, 617 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.33k op/s; 2458/104394 objects degraded (2.355%); 69/104394 objects misplaced (0.066%); 2.0 MiB/s, 32 objects/s recovering 2024-09-20T18:57:44.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:44 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3757266299' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:57:44.388 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:44.415 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:44.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:44.416 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:44.517 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:44.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:44 smithi142 ceph-mon[32508]: pgmap v379: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 2.1 GiB data, 8.4 GiB used, 617 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.33k op/s; 2458/104394 objects degraded (2.355%); 69/104394 objects misplaced (0.066%); 2.0 MiB/s, 32 objects/s recovering 2024-09-20T18:57:44.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:44 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3757266299' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:57:44.542 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:44.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:44.543 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:44.601 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T18:57:44.644 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:44.668 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:44.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:44.669 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:44.770 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:44.795 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:44.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:44.796 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:44.890 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:57:44.896 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:44.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:44.952 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:44.952 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:45.053 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:45.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:45.100 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:45.100 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:45.201 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:45.224 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:45.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:45.225 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:45.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:45 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:57:45.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:45 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:57:45.325 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:57:45.347 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:45.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:45.348 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:57:45.449 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:45.488 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:45.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:45.490 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:45.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:45 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:57:45.588 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T18:57:45.589 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-20T18:57:45.589 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-20T18:57:45.589 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-20T18:57:45.589 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.589 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:27549 7.e 7:77436293:::benchmark_data_smithi083_70643_object27548:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:57:45.589 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:08.731657+0000", 2024-09-20T18:57:45.589 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 396.84317862199998, 2024-09-20T18:57:45.589 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.113586293, 2024-09-20T18:57:45.589 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:57:45.589 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:57:45.589 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:57:45.589 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:57:45.589 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:57:45.590 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:57:45.590 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 27549 2024-09-20T18:57:45.590 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.590 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:57:45.590 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.590 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:57:45.590 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.731657+0000", 2024-09-20T18:57:45.590 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:45.590 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.590 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.590 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:57:45.590 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.731657+0000", 2024-09-20T18:57:45.591 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:45.591 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.591 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.591 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:57:45.591 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.731658+0000", 2024-09-20T18:57:45.591 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1400000000000001e-06 2024-09-20T18:57:45.591 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.591 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.591 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:57:45.591 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.731686+0000", 2024-09-20T18:57:45.591 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7840000000000001e-05 2024-09-20T18:57:45.591 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.591 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.591 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:57:45.592 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.731697+0000", 2024-09-20T18:57:45.592 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0774999999999999e-05 2024-09-20T18:57:45.592 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.592 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.592 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:57:45.592 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.731704+0000", 2024-09-20T18:57:45.592 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1740000000000003e-06 2024-09-20T18:57:45.592 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.592 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.592 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:45.592 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.731757+0000", 2024-09-20T18:57:45.592 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3214000000000002e-05 2024-09-20T18:57:45.592 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.593 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.593 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:57:45.593 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.731876+0000", 2024-09-20T18:57:45.593 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000118829 2024-09-20T18:57:45.593 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.593 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.593 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:57:45.593 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.732005+0000", 2024-09-20T18:57:45.593 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000128937 2024-09-20T18:57:45.593 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.593 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.593 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:57:45.593 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733315+0000", 2024-09-20T18:57:45.593 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001309943 2024-09-20T18:57:45.594 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.594 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.594 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:45.594 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843149+0000", 2024-09-20T18:57:45.594 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.109834472 2024-09-20T18:57:45.594 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.594 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.594 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:45.594 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843225+0000", 2024-09-20T18:57:45.594 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5304999999999998e-05 2024-09-20T18:57:45.594 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.594 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.594 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:57:45.595 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.844447+0000", 2024-09-20T18:57:45.595 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0012227239999999999 2024-09-20T18:57:45.595 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.595 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.595 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:57:45.595 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.845243+0000", 2024-09-20T18:57:45.595 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00079593999999999997 2024-09-20T18:57:45.595 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:45.595 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:45.595 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:45.595 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.595 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.595 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:27551 7.8 7:1bc4112c:::benchmark_data_smithi083_70643_object27550:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:57:45.596 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:08.733037+0000", 2024-09-20T18:57:45.596 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 396.84179883299998, 2024-09-20T18:57:45.596 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.110337717, 2024-09-20T18:57:45.596 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:57:45.596 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:57:45.596 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:57:45.596 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:57:45.596 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:57:45.596 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:57:45.596 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 27551 2024-09-20T18:57:45.596 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.596 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:57:45.596 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.596 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:57:45.597 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733037+0000", 2024-09-20T18:57:45.597 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:45.597 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.597 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.597 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:57:45.597 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733037+0000", 2024-09-20T18:57:45.597 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:45.597 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.597 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.597 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:57:45.597 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733038+0000", 2024-09-20T18:57:45.597 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.497e-06 2024-09-20T18:57:45.597 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.597 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.598 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:57:45.598 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733062+0000", 2024-09-20T18:57:45.598 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3498000000000001e-05 2024-09-20T18:57:45.598 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.598 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.598 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:57:45.598 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733072+0000", 2024-09-20T18:57:45.598 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0173000000000001e-05 2024-09-20T18:57:45.598 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.598 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.598 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:57:45.598 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733079+0000", 2024-09-20T18:57:45.598 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5370000000000003e-06 2024-09-20T18:57:45.598 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.599 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.599 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:45.599 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733187+0000", 2024-09-20T18:57:45.599 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010888 2024-09-20T18:57:45.599 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.599 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.599 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:57:45.599 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733330+0000", 2024-09-20T18:57:45.599 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000142206 2024-09-20T18:57:45.599 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.599 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.599 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4,7", 2024-09-20T18:57:45.599 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733549+0000", 2024-09-20T18:57:45.599 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0002198 2024-09-20T18:57:45.600 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.600 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.600 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:57:45.600 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.735541+0000", 2024-09-20T18:57:45.600 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0019915129999999999 2024-09-20T18:57:45.600 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.600 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.600 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:45.600 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.842311+0000", 2024-09-20T18:57:45.600 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.106769612 2024-09-20T18:57:45.600 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.600 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.600 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:45.600 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.842372+0000", 2024-09-20T18:57:45.601 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0968000000000003e-05 2024-09-20T18:57:45.601 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.601 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.601 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:57:45.601 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843315+0000", 2024-09-20T18:57:45.601 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00094376300000000002 2024-09-20T18:57:45.601 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.601 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.601 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:57:45.601 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843375+0000", 2024-09-20T18:57:45.601 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.927e-05 2024-09-20T18:57:45.601 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:45.601 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:45.601 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:45.602 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.602 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.602 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:27555 7.e 7:73709da2:::benchmark_data_smithi083_70643_object27554:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:57:45.602 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:08.733695+0000", 2024-09-20T18:57:45.602 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 396.84114070999999, 2024-09-20T18:57:45.602 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.11168107300000001, 2024-09-20T18:57:45.602 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:57:45.602 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:57:45.602 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:57:45.602 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:57:45.603 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:45.603 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:57:45.603 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:57:45.604 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 27555 2024-09-20T18:57:45.604 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.604 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:57:45.604 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.604 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:57:45.604 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733695+0000", 2024-09-20T18:57:45.604 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:45.604 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.604 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.604 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:57:45.604 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733695+0000", 2024-09-20T18:57:45.604 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:45.604 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.604 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.605 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:57:45.605 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733696+0000", 2024-09-20T18:57:45.605 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0200000000000001e-07 2024-09-20T18:57:45.605 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.605 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.605 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:57:45.605 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733718+0000", 2024-09-20T18:57:45.605 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2226999999999999e-05 2024-09-20T18:57:45.605 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.605 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.605 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:57:45.605 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733727+0000", 2024-09-20T18:57:45.605 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6049999999999998e-06 2024-09-20T18:57:45.605 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.606 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.606 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:57:45.606 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733734+0000", 2024-09-20T18:57:45.606 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0639999999999996e-06 2024-09-20T18:57:45.606 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.606 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.606 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:45.606 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733789+0000", 2024-09-20T18:57:45.606 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5087999999999998e-05 2024-09-20T18:57:45.606 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.606 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.606 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:57:45.606 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733920+0000", 2024-09-20T18:57:45.607 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013146800000000001 2024-09-20T18:57:45.607 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.607 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.607 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:57:45.607 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733995+0000", 2024-09-20T18:57:45.607 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4664000000000002e-05 2024-09-20T18:57:45.607 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.607 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.607 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:57:45.607 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.735538+0000", 2024-09-20T18:57:45.607 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001542671 2024-09-20T18:57:45.607 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.607 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.607 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:45.608 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843187+0000", 2024-09-20T18:57:45.608 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.107649921 2024-09-20T18:57:45.608 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.608 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.608 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:45.608 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.845296+0000", 2024-09-20T18:57:45.608 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0021081920000000001 2024-09-20T18:57:45.608 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.608 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.608 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:57:45.608 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.845341+0000", 2024-09-20T18:57:45.608 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5141e-05 2024-09-20T18:57:45.608 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.608 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.609 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:57:45.609 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.845376+0000", 2024-09-20T18:57:45.609 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.523e-05 2024-09-20T18:57:45.609 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:45.609 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:45.609 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:45.609 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.609 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.609 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:27560 7.e 7:7d7d0b94:::benchmark_data_smithi083_70643_object27559:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:57:45.609 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:08.736128+0000", 2024-09-20T18:57:45.609 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 396.83870811100002, 2024-09-20T18:57:45.609 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.112670376, 2024-09-20T18:57:45.609 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:57:45.609 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:57:45.610 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:57:45.610 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:57:45.610 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:57:45.610 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:57:45.610 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 27560 2024-09-20T18:57:45.610 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.610 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:57:45.610 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.610 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:57:45.610 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736128+0000", 2024-09-20T18:57:45.610 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:45.610 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.610 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.611 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:57:45.611 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736128+0000", 2024-09-20T18:57:45.611 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:45.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.611 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.611 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:57:45.611 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736129+0000", 2024-09-20T18:57:45.611 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.251e-06 2024-09-20T18:57:45.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.611 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.611 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:57:45.611 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736150+0000", 2024-09-20T18:57:45.611 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1450999999999998e-05 2024-09-20T18:57:45.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.612 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.612 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:57:45.612 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736161+0000", 2024-09-20T18:57:45.612 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0682e-05 2024-09-20T18:57:45.612 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.612 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.612 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:57:45.612 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736168+0000", 2024-09-20T18:57:45.612 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8549999999999996e-06 2024-09-20T18:57:45.612 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.612 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.612 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:45.612 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736218+0000", 2024-09-20T18:57:45.613 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0497000000000001e-05 2024-09-20T18:57:45.613 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.613 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.613 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:57:45.613 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736327+0000", 2024-09-20T18:57:45.613 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010837599999999999 2024-09-20T18:57:45.613 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.613 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.613 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:57:45.613 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.841982+0000", 2024-09-20T18:57:45.613 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.105654846 2024-09-20T18:57:45.613 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.613 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.613 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:57:45.614 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.845386+0000", 2024-09-20T18:57:45.614 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0034042719999999999 2024-09-20T18:57:45.614 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.614 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.614 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:45.614 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.845423+0000", 2024-09-20T18:57:45.614 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7160000000000003e-05 2024-09-20T18:57:45.614 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.614 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.614 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:45.614 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.848702+0000", 2024-09-20T18:57:45.614 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0032792099999999999 2024-09-20T18:57:45.614 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.614 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.615 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:57:45.615 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.848758+0000", 2024-09-20T18:57:45.615 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.537e-05 2024-09-20T18:57:45.615 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.615 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.615 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:57:45.615 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.848798+0000", 2024-09-20T18:57:45.615 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0405999999999999e-05 2024-09-20T18:57:45.615 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:45.615 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:45.615 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:45.615 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.615 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.615 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:27559 7.a e93/83 7:5f89880a:::benchmark_data_smithi083_70643_object27558:head v 93'1704, mlcod=93'1702)", 2024-09-20T18:57:45.616 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:08.736205+0000", 2024-09-20T18:57:45.616 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 396.83863058200001, 2024-09-20T18:57:45.616 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.108390217, 2024-09-20T18:57:45.616 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:57:45.616 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:57:45.616 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:57:45.616 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:57:45.616 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.616 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:57:45.616 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736205+0000", 2024-09-20T18:57:45.616 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:45.616 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.616 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.616 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:57:45.617 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736205+0000", 2024-09-20T18:57:45.617 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:45.617 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.617 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.617 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:57:45.617 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736206+0000", 2024-09-20T18:57:45.617 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4000000000000001e-07 2024-09-20T18:57:45.617 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.617 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.617 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:57:45.617 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736226+0000", 2024-09-20T18:57:45.617 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0426999999999999e-05 2024-09-20T18:57:45.617 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.617 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.617 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:57:45.618 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736236+0000", 2024-09-20T18:57:45.618 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.781e-06 2024-09-20T18:57:45.618 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.618 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.618 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:57:45.618 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736242+0000", 2024-09-20T18:57:45.618 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0179999999999998e-06 2024-09-20T18:57:45.618 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.618 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.618 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:45.618 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736307+0000", 2024-09-20T18:57:45.618 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5042000000000005e-05 2024-09-20T18:57:45.618 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.618 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.619 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:57:45.619 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736325+0000", 2024-09-20T18:57:45.619 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7895999999999999e-05 2024-09-20T18:57:45.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.619 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.619 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:57:45.619 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.844517+0000", 2024-09-20T18:57:45.619 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.10819235100000001 2024-09-20T18:57:45.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.619 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.619 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:57:45.619 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.844595+0000", 2024-09-20T18:57:45.619 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7961999999999997e-05 2024-09-20T18:57:45.620 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:45.620 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:45.620 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:45.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.620 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.620 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:27549 7.e e93/83 ondisk, result = 0)", 2024-09-20T18:57:45.620 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:08.736243+0000", 2024-09-20T18:57:45.620 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 396.83859298800002, 2024-09-20T18:57:45.620 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.106913722, 2024-09-20T18:57:45.620 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:57:45.620 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:57:45.620 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:57:45.620 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:57:45.621 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.621 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:57:45.621 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736243+0000", 2024-09-20T18:57:45.621 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:45.621 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.621 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.621 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:57:45.621 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736243+0000", 2024-09-20T18:57:45.621 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:45.621 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.621 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.621 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:57:45.621 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736243+0000", 2024-09-20T18:57:45.622 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3200000000000004e-07 2024-09-20T18:57:45.622 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.622 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.622 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:57:45.622 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736248+0000", 2024-09-20T18:57:45.622 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2780000000000004e-06 2024-09-20T18:57:45.622 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.622 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.622 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:57:45.622 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736257+0000", 2024-09-20T18:57:45.622 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2790000000000004e-06 2024-09-20T18:57:45.622 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.622 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.622 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:57:45.623 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736263+0000", 2024-09-20T18:57:45.623 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3570000000000003e-06 2024-09-20T18:57:45.623 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.623 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.623 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:45.623 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843136+0000", 2024-09-20T18:57:45.623 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.10687292499999999 2024-09-20T18:57:45.623 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.623 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.623 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:57:45.623 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843143+0000", 2024-09-20T18:57:45.623 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2300000000000002e-06 2024-09-20T18:57:45.623 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.623 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.624 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:57:45.624 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843156+0000", 2024-09-20T18:57:45.624 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2921e-05 2024-09-20T18:57:45.624 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:45.624 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:45.624 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:45.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.624 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.624 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:27555 7.e e93/83 ondisk, result = 0)", 2024-09-20T18:57:45.624 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:08.736486+0000", 2024-09-20T18:57:45.624 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 396.83834990000003, 2024-09-20T18:57:45.624 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.106708079, 2024-09-20T18:57:45.624 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:57:45.624 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:57:45.625 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:57:45.625 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:57:45.625 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.625 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:57:45.625 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736486+0000", 2024-09-20T18:57:45.625 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:45.625 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.625 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.625 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:57:45.625 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736486+0000", 2024-09-20T18:57:45.625 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:45.625 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.625 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.625 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:57:45.625 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736486+0000", 2024-09-20T18:57:45.626 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8100000000000002e-07 2024-09-20T18:57:45.626 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.626 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.626 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:57:45.626 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736490+0000", 2024-09-20T18:57:45.626 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.546e-06 2024-09-20T18:57:45.626 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.626 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.626 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:57:45.626 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.841491+0000", 2024-09-20T18:57:45.626 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.105001423 2024-09-20T18:57:45.626 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.626 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.627 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:57:45.627 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.841503+0000", 2024-09-20T18:57:45.627 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1399e-05 2024-09-20T18:57:45.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.627 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.627 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:45.627 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843177+0000", 2024-09-20T18:57:45.627 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0016741900000000001 2024-09-20T18:57:45.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.627 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.627 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:57:45.627 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843183+0000", 2024-09-20T18:57:45.627 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7209999999999999e-06 2024-09-20T18:57:45.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.627 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.628 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:57:45.628 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843194+0000", 2024-09-20T18:57:45.628 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1119e-05 2024-09-20T18:57:45.628 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:45.628 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:45.628 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:45.628 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.628 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.628 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:27551 7.8 e93/83 ondisk, result = 0)", 2024-09-20T18:57:45.628 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:08.736884+0000", 2024-09-20T18:57:45.628 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 396.83795210199997, 2024-09-20T18:57:45.628 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.105442044, 2024-09-20T18:57:45.628 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:57:45.628 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:57:45.629 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:57:45.629 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:57:45.629 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.629 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:57:45.629 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736884+0000", 2024-09-20T18:57:45.629 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:45.629 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.629 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.629 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:57:45.629 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736884+0000", 2024-09-20T18:57:45.629 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:45.629 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.629 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.629 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:57:45.629 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736884+0000", 2024-09-20T18:57:45.630 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6499999999999999e-07 2024-09-20T18:57:45.630 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.630 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.630 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:57:45.630 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736888+0000", 2024-09-20T18:57:45.630 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7170000000000002e-06 2024-09-20T18:57:45.630 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.630 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.630 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:57:45.630 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.841517+0000", 2024-09-20T18:57:45.630 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.104628836 2024-09-20T18:57:45.630 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.630 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.630 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:57:45.631 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.841525+0000", 2024-09-20T18:57:45.631 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7110000000000001e-06 2024-09-20T18:57:45.631 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.631 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.631 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:45.631 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.842286+0000", 2024-09-20T18:57:45.631 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00076098500000000005 2024-09-20T18:57:45.631 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.631 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.631 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:57:45.631 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.842304+0000", 2024-09-20T18:57:45.631 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7898e-05 2024-09-20T18:57:45.631 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.631 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.632 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:57:45.632 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.842326+0000", 2024-09-20T18:57:45.632 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:57:45.633 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1331999999999998e-05 2024-09-20T18:57:45.633 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:45.633 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:45.633 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:45.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.633 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.633 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:27571 7.8 7:1e0af83a:::benchmark_data_smithi083_70643_object27570:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:57:45.633 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:08.741453+0000", 2024-09-20T18:57:45.633 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 396.83338289800002, 2024-09-20T18:57:45.634 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.103905603, 2024-09-20T18:57:45.634 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:57:45.634 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:57:45.634 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:57:45.634 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:57:45.634 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:57:45.634 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:57:45.634 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 27571 2024-09-20T18:57:45.634 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.634 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:57:45.634 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.634 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:57:45.634 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.741453+0000", 2024-09-20T18:57:45.635 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:45.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.635 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.635 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:57:45.635 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.741453+0000", 2024-09-20T18:57:45.635 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:45.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.635 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.635 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:57:45.635 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.741455+0000", 2024-09-20T18:57:45.635 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4669999999999998e-06 2024-09-20T18:57:45.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.635 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.635 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:57:45.636 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.741483+0000", 2024-09-20T18:57:45.636 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7472999999999999e-05 2024-09-20T18:57:45.636 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.636 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.636 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:57:45.636 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.841438+0000", 2024-09-20T18:57:45.636 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.099955024000000003 2024-09-20T18:57:45.636 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.636 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.636 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:57:45.636 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.841457+0000", 2024-09-20T18:57:45.636 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9264999999999999e-05 2024-09-20T18:57:45.636 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.636 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.636 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:45.637 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843412+0000", 2024-09-20T18:57:45.637 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0019553869999999998 2024-09-20T18:57:45.637 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.637 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.637 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:57:45.637 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843559+0000", 2024-09-20T18:57:45.637 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014645399999999999 2024-09-20T18:57:45.637 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.637 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.637 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4,7", 2024-09-20T18:57:45.637 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843706+0000", 2024-09-20T18:57:45.637 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014695600000000001 2024-09-20T18:57:45.637 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.637 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.638 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:57:45.638 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.844855+0000", 2024-09-20T18:57:45.638 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001149424 2024-09-20T18:57:45.638 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.638 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.638 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:45.638 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.845215+0000", 2024-09-20T18:57:45.638 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00035930100000000002 2024-09-20T18:57:45.638 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.638 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.638 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:45.638 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.845270+0000", 2024-09-20T18:57:45.638 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5461999999999999e-05 2024-09-20T18:57:45.638 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.639 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.639 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:57:45.639 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.845321+0000", 2024-09-20T18:57:45.639 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1360000000000003e-05 2024-09-20T18:57:45.639 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.639 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.639 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:57:45.639 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.845358+0000", 2024-09-20T18:57:45.639 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7030000000000003e-05 2024-09-20T18:57:45.639 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:45.639 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:45.639 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:45.639 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.640 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.640 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:117083 7.e 7:7035ec2c:::benchmark_data_smithi083_70643_object117082:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-20T18:57:45.640 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:52:21.789905+0000", 2024-09-20T18:57:45.640 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 323.78493081200003, 2024-09-20T18:57:45.640 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.1098601, 2024-09-20T18:57:45.640 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:57:45.640 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:57:45.640 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:57:45.640 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:57:45.640 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:57:45.640 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:57:45.640 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 117083 2024-09-20T18:57:45.640 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.640 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:57:45.641 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.641 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:57:45.641 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.789905+0000", 2024-09-20T18:57:45.641 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:45.641 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.641 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.641 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:57:45.641 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.789905+0000", 2024-09-20T18:57:45.641 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:45.641 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.641 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.641 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:57:45.641 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.789906+0000", 2024-09-20T18:57:45.641 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4109999999999999e-06 2024-09-20T18:57:45.641 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.642 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.642 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:57:45.642 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.789929+0000", 2024-09-20T18:57:45.642 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3212000000000001e-05 2024-09-20T18:57:45.642 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.642 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.642 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:57:45.642 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.789941+0000", 2024-09-20T18:57:45.642 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1103e-05 2024-09-20T18:57:45.642 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.642 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.642 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:57:45.642 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.789948+0000", 2024-09-20T18:57:45.643 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.968e-06 2024-09-20T18:57:45.643 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.643 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.643 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:45.643 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.790013+0000", 2024-09-20T18:57:45.643 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5518000000000005e-05 2024-09-20T18:57:45.643 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.643 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.643 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:57:45.643 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.790130+0000", 2024-09-20T18:57:45.643 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000116906 2024-09-20T18:57:45.643 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.643 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.643 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:57:45.643 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.790270+0000", 2024-09-20T18:57:45.644 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013975700000000001 2024-09-20T18:57:45.644 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.644 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.644 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:57:45.644 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.793553+0000", 2024-09-20T18:57:45.644 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003283589 2024-09-20T18:57:45.644 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.644 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.644 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:45.644 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.799223+0000", 2024-09-20T18:57:45.644 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0056701340000000003 2024-09-20T18:57:45.644 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.644 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.644 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:45.645 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.899580+0000", 2024-09-20T18:57:45.645 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.100356374 2024-09-20T18:57:45.645 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.645 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.645 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:57:45.645 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.899639+0000", 2024-09-20T18:57:45.645 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8990000000000003e-05 2024-09-20T18:57:45.645 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.645 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.645 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:57:45.645 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.899765+0000", 2024-09-20T18:57:45.645 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000126138 2024-09-20T18:57:45.645 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:45.645 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:45.646 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:45.646 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.646 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.646 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:117091 7.e 7:791f482a:::benchmark_data_smithi083_70643_object117090:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-20T18:57:45.646 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:52:21.794291+0000", 2024-09-20T18:57:45.646 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 323.780545104, 2024-09-20T18:57:45.646 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.108790281, 2024-09-20T18:57:45.646 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:57:45.646 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:57:45.646 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:57:45.646 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:57:45.646 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:57:45.646 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:57:45.647 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 117091 2024-09-20T18:57:45.647 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.647 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:57:45.647 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.647 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:57:45.647 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.794291+0000", 2024-09-20T18:57:45.647 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:45.647 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.647 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.647 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:57:45.647 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.794291+0000", 2024-09-20T18:57:45.647 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:45.647 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.647 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.648 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:57:45.648 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.794292+0000", 2024-09-20T18:57:45.648 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.579e-06 2024-09-20T18:57:45.648 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.648 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.648 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:57:45.648 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.794317+0000", 2024-09-20T18:57:45.648 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4895000000000001e-05 2024-09-20T18:57:45.648 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.648 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.648 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:57:45.648 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.794329+0000", 2024-09-20T18:57:45.648 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1905e-05 2024-09-20T18:57:45.649 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.649 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.649 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:57:45.649 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.794337+0000", 2024-09-20T18:57:45.649 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8590000000000005e-06 2024-09-20T18:57:45.649 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.649 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.649 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:45.649 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.794397+0000", 2024-09-20T18:57:45.649 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0531e-05 2024-09-20T18:57:45.649 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.649 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.649 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:57:45.649 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.794533+0000", 2024-09-20T18:57:45.650 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013557200000000001 2024-09-20T18:57:45.650 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.650 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.650 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:57:45.650 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.794680+0000", 2024-09-20T18:57:45.650 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014704799999999999 2024-09-20T18:57:45.650 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.650 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.650 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:57:45.650 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.799870+0000", 2024-09-20T18:57:45.650 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0051902720000000001 2024-09-20T18:57:45.650 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.650 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.650 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:45.651 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.804125+0000", 2024-09-20T18:57:45.651 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0042546779999999996 2024-09-20T18:57:45.651 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.651 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.651 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:45.651 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.902983+0000", 2024-09-20T18:57:45.651 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098858370000000001 2024-09-20T18:57:45.651 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.651 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.651 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:57:45.651 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.903040+0000", 2024-09-20T18:57:45.651 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7136999999999999e-05 2024-09-20T18:57:45.651 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.652 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.652 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:57:45.652 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.903081+0000", 2024-09-20T18:57:45.652 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0435000000000002e-05 2024-09-20T18:57:45.652 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:45.652 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:45.652 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:45.652 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.652 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.652 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:187757 7.e 7:70705a0e:::benchmark_data_smithi083_70643_object64955:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-20T18:57:45.652 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:53:29.029730+0000", 2024-09-20T18:57:45.652 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 256.54510517300002, 2024-09-20T18:57:45.652 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.137882645, 2024-09-20T18:57:45.652 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:57:45.653 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:57:45.653 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:57:45.653 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:57:45.653 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:57:45.653 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:57:45.653 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 187757 2024-09-20T18:57:45.653 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.653 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:57:45.653 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.653 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:57:45.653 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.029730+0000", 2024-09-20T18:57:45.653 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:45.653 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.653 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.653 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:57:45.654 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.029730+0000", 2024-09-20T18:57:45.654 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:45.654 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.654 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.654 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:57:45.654 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.029732+0000", 2024-09-20T18:57:45.654 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.415e-06 2024-09-20T18:57:45.654 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.654 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.654 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:57:45.654 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.029737+0000", 2024-09-20T18:57:45.654 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0239999999999997e-06 2024-09-20T18:57:45.655 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.655 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.655 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:57:45.655 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.029748+0000", 2024-09-20T18:57:45.655 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0920999999999999e-05 2024-09-20T18:57:45.655 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.655 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.655 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:57:45.655 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.029755+0000", 2024-09-20T18:57:45.655 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1430000000000001e-06 2024-09-20T18:57:45.655 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.655 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.655 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:45.655 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.030815+0000", 2024-09-20T18:57:45.656 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0010603590000000001 2024-09-20T18:57:45.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.656 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.656 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:57:45.656 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.030930+0000", 2024-09-20T18:57:45.656 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000114171 2024-09-20T18:57:45.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.656 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.656 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:57:45.656 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.031080+0000", 2024-09-20T18:57:45.656 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015042000000000001 2024-09-20T18:57:45.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.656 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.656 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:57:45.657 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.036072+0000", 2024-09-20T18:57:45.657 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049921159999999996 2024-09-20T18:57:45.657 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.657 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.657 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:45.657 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.037324+0000", 2024-09-20T18:57:45.657 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0012515930000000001 2024-09-20T18:57:45.657 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.657 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.657 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:45.657 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045902+0000", 2024-09-20T18:57:45.657 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0085779580000000001 2024-09-20T18:57:45.657 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.657 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.658 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:57:45.658 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.167564+0000", 2024-09-20T18:57:45.658 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.121662143 2024-09-20T18:57:45.658 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.658 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.658 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:57:45.658 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.167613+0000", 2024-09-20T18:57:45.658 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9382000000000003e-05 2024-09-20T18:57:45.658 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:45.658 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:45.658 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:45.658 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.658 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.658 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:187765 7.e 7:7e1cec20:::benchmark_data_smithi083_70643_object64963:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-20T18:57:45.659 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:53:29.040056+0000", 2024-09-20T18:57:45.659 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 256.534780087, 2024-09-20T18:57:45.659 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.132718637, 2024-09-20T18:57:45.659 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:57:45.659 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:57:45.659 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:57:45.659 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:57:45.659 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:57:45.659 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:57:45.659 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 187765 2024-09-20T18:57:45.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.659 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:57:45.659 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.659 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:57:45.660 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.040056+0000", 2024-09-20T18:57:45.660 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:45.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.660 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.660 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:57:45.660 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.040056+0000", 2024-09-20T18:57:45.660 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:45.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.660 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.660 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:57:45.662 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.040057+0000", 2024-09-20T18:57:45.662 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.528e-06 2024-09-20T18:57:45.662 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.662 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.662 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:57:45.662 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.040063+0000", 2024-09-20T18:57:45.662 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8170000000000004e-06 2024-09-20T18:57:45.662 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.662 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.662 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:57:45.662 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.040074+0000", 2024-09-20T18:57:45.663 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1253000000000001e-05 2024-09-20T18:57:45.663 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.663 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.663 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:57:45.663 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.040081+0000", 2024-09-20T18:57:45.663 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1389999999999998e-06 2024-09-20T18:57:45.663 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.663 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.663 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:45.663 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.040134+0000", 2024-09-20T18:57:45.663 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2549000000000001e-05 2024-09-20T18:57:45.663 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.663 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.663 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:57:45.664 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.040246+0000", 2024-09-20T18:57:45.664 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000112271 2024-09-20T18:57:45.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.664 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.664 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:57:45.664 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.040354+0000", 2024-09-20T18:57:45.664 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010791 2024-09-20T18:57:45.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.664 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.664 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:45.664 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045867+0000", 2024-09-20T18:57:45.664 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0055125449999999998 2024-09-20T18:57:45.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.664 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.665 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:57:45.665 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.172613+0000", 2024-09-20T18:57:45.665 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.12674679599999999 2024-09-20T18:57:45.665 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.665 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.665 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:45.665 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.172691+0000", 2024-09-20T18:57:45.665 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7255000000000005e-05 2024-09-20T18:57:45.665 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.665 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.665 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:57:45.665 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.172735+0000", 2024-09-20T18:57:45.665 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4069000000000001e-05 2024-09-20T18:57:45.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.666 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.666 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:57:45.666 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.172774+0000", 2024-09-20T18:57:45.666 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9505000000000002e-05 2024-09-20T18:57:45.666 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:45.666 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:45.666 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:45.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.666 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.666 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:187763 7.3 e141/83 7:c71c27bb:::benchmark_data_smithi083_70643_object64961:head v 141'11587, mlcod=141'11584)", 2024-09-20T18:57:45.666 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:53:29.043149+0000", 2024-09-20T18:57:45.666 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 256.53168640299998, 2024-09-20T18:57:45.666 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.125479285, 2024-09-20T18:57:45.666 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:57:45.667 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:57:45.667 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:57:45.667 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:57:45.667 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.667 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:57:45.667 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043149+0000", 2024-09-20T18:57:45.667 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:45.667 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.667 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.667 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:57:45.667 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043149+0000", 2024-09-20T18:57:45.667 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:45.667 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.667 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.668 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:57:45.668 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043150+0000", 2024-09-20T18:57:45.668 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7800000000000001e-07 2024-09-20T18:57:45.668 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.668 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.668 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:57:45.668 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043154+0000", 2024-09-20T18:57:45.668 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6480000000000002e-06 2024-09-20T18:57:45.668 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.668 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.668 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:57:45.668 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043162+0000", 2024-09-20T18:57:45.668 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1810000000000004e-06 2024-09-20T18:57:45.668 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.668 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.669 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:57:45.669 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043167+0000", 2024-09-20T18:57:45.669 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4500000000000003e-06 2024-09-20T18:57:45.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.669 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.669 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:45.669 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043227+0000", 2024-09-20T18:57:45.669 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9697999999999997e-05 2024-09-20T18:57:45.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.669 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.669 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:57:45.669 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043243+0000", 2024-09-20T18:57:45.669 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6461999999999999e-05 2024-09-20T18:57:45.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.670 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.670 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:57:45.670 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.168513+0000", 2024-09-20T18:57:45.670 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.125269624 2024-09-20T18:57:45.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.670 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.670 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:57:45.670 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.168629+0000", 2024-09-20T18:57:45.670 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011564400000000001 2024-09-20T18:57:45.670 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:45.670 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:45.670 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:45.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.671 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.671 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:187773 7.e 7:7168b707:::benchmark_data_smithi083_70643_object64971:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-20T18:57:45.671 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:53:29.043782+0000", 2024-09-20T18:57:45.671 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 256.53105367500001, 2024-09-20T18:57:45.671 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.14065076300000001, 2024-09-20T18:57:45.671 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:57:45.671 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:57:45.671 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:57:45.671 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:57:45.671 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:57:45.671 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:57:45.671 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 187773 2024-09-20T18:57:45.671 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.671 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:57:45.671 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.672 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:57:45.672 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043782+0000", 2024-09-20T18:57:45.672 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:45.672 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.672 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.672 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:57:45.672 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043782+0000", 2024-09-20T18:57:45.672 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:45.672 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.672 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.672 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:57:45.672 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043784+0000", 2024-09-20T18:57:45.672 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5260000000000001e-06 2024-09-20T18:57:45.673 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.673 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.673 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:57:45.673 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043788+0000", 2024-09-20T18:57:45.673 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.481e-06 2024-09-20T18:57:45.673 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.673 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.673 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:57:45.673 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043798+0000", 2024-09-20T18:57:45.673 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0393e-05 2024-09-20T18:57:45.673 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.673 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.673 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:57:45.674 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043805+0000", 2024-09-20T18:57:45.674 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5030000000000002e-06 2024-09-20T18:57:45.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.674 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.674 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:45.674 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.167654+0000", 2024-09-20T18:57:45.674 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.123848659 2024-09-20T18:57:45.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.674 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.674 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:57:45.674 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.167881+0000", 2024-09-20T18:57:45.674 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000227698 2024-09-20T18:57:45.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.675 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.675 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:57:45.675 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.167981+0000", 2024-09-20T18:57:45.675 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9611999999999994e-05 2024-09-20T18:57:45.675 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.675 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.675 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:45.675 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.172817+0000", 2024-09-20T18:57:45.675 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0048360829999999997 2024-09-20T18:57:45.675 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.675 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.675 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:57:45.675 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.172828+0000", 2024-09-20T18:57:45.675 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1280999999999999e-05 2024-09-20T18:57:45.675 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.676 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.676 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:45.676 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.184218+0000", 2024-09-20T18:57:45.676 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011389882 2024-09-20T18:57:45.676 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.676 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.676 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:57:45.676 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.184387+0000", 2024-09-20T18:57:45.676 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000168992 2024-09-20T18:57:45.676 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.676 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.676 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:57:45.676 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.184433+0000", 2024-09-20T18:57:45.677 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5652999999999998e-05 2024-09-20T18:57:45.677 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:45.677 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:45.677 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:45.677 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.677 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.677 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:187757 7.e e141/83 ondisk, result = 0)", 2024-09-20T18:57:45.677 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:53:29.045594+0000", 2024-09-20T18:57:45.677 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 256.52924172100001, 2024-09-20T18:57:45.677 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.12203122600000001, 2024-09-20T18:57:45.677 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:57:45.677 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:57:45.677 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:57:45.677 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:57:45.677 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.678 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:57:45.678 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045594+0000", 2024-09-20T18:57:45.678 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:45.678 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.678 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.678 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:57:45.678 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045594+0000", 2024-09-20T18:57:45.678 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:45.678 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.678 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.678 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:57:45.678 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045595+0000", 2024-09-20T18:57:45.678 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6499999999999998e-07 2024-09-20T18:57:45.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.679 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.679 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:57:45.679 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045599+0000", 2024-09-20T18:57:45.679 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6160000000000004e-06 2024-09-20T18:57:45.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.679 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.679 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:57:45.679 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045609+0000", 2024-09-20T18:57:45.679 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2269999999999999e-06 2024-09-20T18:57:45.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.679 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.679 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:57:45.679 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045615+0000", 2024-09-20T18:57:45.680 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1349999999999997e-06 2024-09-20T18:57:45.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.680 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.680 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:45.680 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045892+0000", 2024-09-20T18:57:45.680 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000276724 2024-09-20T18:57:45.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.680 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.680 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:57:45.680 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045897+0000", 2024-09-20T18:57:45.680 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2970000000000003e-06 2024-09-20T18:57:45.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.680 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.680 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:57:45.681 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.167625+0000", 2024-09-20T18:57:45.681 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.12172836200000001 2024-09-20T18:57:45.681 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:45.681 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:45.681 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:45.681 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.681 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.681 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:187772 7.a e141/83 7:5948c6a4:::benchmark_data_smithi083_70643_object64970:head v 141'11671, mlcod=141'11669)", 2024-09-20T18:57:45.681 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:53:29.045936+0000", 2024-09-20T18:57:45.681 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 256.52890002499998, 2024-09-20T18:57:45.681 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.12458983799999999, 2024-09-20T18:57:45.681 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:57:45.681 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:57:45.681 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:57:45.681 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:57:45.682 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.682 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:57:45.682 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045936+0000", 2024-09-20T18:57:45.682 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:45.682 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.682 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.682 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:57:45.682 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045936+0000", 2024-09-20T18:57:45.682 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:45.682 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.682 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.682 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:57:45.682 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045936+0000", 2024-09-20T18:57:45.682 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2900000000000003e-07 2024-09-20T18:57:45.682 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.685 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.686 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:57:45.686 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045942+0000", 2024-09-20T18:57:45.686 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.66e-06 2024-09-20T18:57:45.686 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.686 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.686 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:57:45.686 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.165480+0000", 2024-09-20T18:57:45.686 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.119537582 2024-09-20T18:57:45.686 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.686 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.686 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:57:45.686 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.165496+0000", 2024-09-20T18:57:45.686 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6249999999999999e-05 2024-09-20T18:57:45.687 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.687 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.687 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:45.687 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.165558+0000", 2024-09-20T18:57:45.687 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2120999999999997e-05 2024-09-20T18:57:45.687 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.687 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.687 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:57:45.687 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.165582+0000", 2024-09-20T18:57:45.687 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4409e-05 2024-09-20T18:57:45.687 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.687 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.687 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:57:45.687 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.170489+0000", 2024-09-20T18:57:45.688 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049066860000000004 2024-09-20T18:57:45.688 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.688 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.688 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:57:45.688 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.170525+0000", 2024-09-20T18:57:45.688 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6400999999999999e-05 2024-09-20T18:57:45.688 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:45.688 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:45.688 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:45.688 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.688 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.688 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:187774 7.e 7:778445cb:::benchmark_data_smithi083_70643_object64972:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-20T18:57:45.688 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:53:29.046524+0000", 2024-09-20T18:57:45.689 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 256.52831159999999, 2024-09-20T18:57:45.689 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.15650777099999999, 2024-09-20T18:57:45.689 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:57:45.689 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:57:45.689 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:57:45.689 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:57:45.689 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:57:45.689 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:57:45.689 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 187774 2024-09-20T18:57:45.689 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.689 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:57:45.689 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.689 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:57:45.689 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.046524+0000", 2024-09-20T18:57:45.690 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:45.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.690 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.690 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:57:45.690 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.046524+0000", 2024-09-20T18:57:45.690 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:45.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.690 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.690 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:57:45.690 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.046526+0000", 2024-09-20T18:57:45.690 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.494e-06 2024-09-20T18:57:45.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.690 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.690 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:57:45.691 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.046531+0000", 2024-09-20T18:57:45.691 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4840000000000003e-06 2024-09-20T18:57:45.691 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.691 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.691 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:57:45.691 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.165656+0000", 2024-09-20T18:57:45.691 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.11912532100000001 2024-09-20T18:57:45.691 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.691 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.691 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:57:45.691 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.165666+0000", 2024-09-20T18:57:45.691 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1789999999999997e-06 2024-09-20T18:57:45.691 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.691 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.692 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:45.692 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.173061+0000", 2024-09-20T18:57:45.692 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0073953969999999997 2024-09-20T18:57:45.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.692 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.692 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:57:45.692 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.173154+0000", 2024-09-20T18:57:45.692 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3113000000000001e-05 2024-09-20T18:57:45.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.692 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.692 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:57:45.692 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.173250+0000", 2024-09-20T18:57:45.692 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5493000000000002e-05 2024-09-20T18:57:45.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.693 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.693 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:45.693 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.180360+0000", 2024-09-20T18:57:45.693 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0071109220000000004 2024-09-20T18:57:45.693 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.693 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.693 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:57:45.693 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.188345+0000", 2024-09-20T18:57:45.693 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0079849840000000005 2024-09-20T18:57:45.693 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.693 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.693 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:45.693 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.202813+0000", 2024-09-20T18:57:45.694 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.014467647 2024-09-20T18:57:45.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.694 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.694 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:57:45.694 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.202985+0000", 2024-09-20T18:57:45.694 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017191999999999999 2024-09-20T18:57:45.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.694 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.694 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:57:45.694 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.203032+0000", 2024-09-20T18:57:45.694 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6817000000000003e-05 2024-09-20T18:57:45.694 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:45.694 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:45.694 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:45.695 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.695 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.695 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:187765 7.e e141/83 ondisk, result = 0)", 2024-09-20T18:57:45.695 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:53:29.046987+0000", 2024-09-20T18:57:45.695 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 256.52784860100002, 2024-09-20T18:57:45.695 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.12579221700000001, 2024-09-20T18:57:45.695 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:57:45.695 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:57:45.695 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:57:45.695 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:57:45.695 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.695 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:57:45.695 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.046987+0000", 2024-09-20T18:57:45.696 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:45.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.696 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.696 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:57:45.696 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.046987+0000", 2024-09-20T18:57:45.696 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:45.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.696 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.696 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:57:45.696 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.046988+0000", 2024-09-20T18:57:45.696 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6499999999999999e-07 2024-09-20T18:57:45.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.696 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.696 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:57:45.697 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.046992+0000", 2024-09-20T18:57:45.697 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9650000000000002e-06 2024-09-20T18:57:45.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.697 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.697 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:57:45.697 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.165597+0000", 2024-09-20T18:57:45.697 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.118605087 2024-09-20T18:57:45.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.697 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.697 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:57:45.697 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.165603+0000", 2024-09-20T18:57:45.697 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0100000000000001e-06 2024-09-20T18:57:45.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.697 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.697 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:45.698 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.172679+0000", 2024-09-20T18:57:45.698 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0070759810000000003 2024-09-20T18:57:45.698 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.698 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.698 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:57:45.698 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.172686+0000", 2024-09-20T18:57:45.698 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9959999999999996e-06 2024-09-20T18:57:45.698 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.698 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.698 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:57:45.698 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.172779+0000", 2024-09-20T18:57:45.698 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3512999999999997e-05 2024-09-20T18:57:45.698 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:45.699 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:45.699 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:45.699 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.699 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.699 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:197546 7.e 7:71d75882:::benchmark_data_smithi083_70643_object74744:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e144)", 2024-09-20T18:57:45.699 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:53:38.347654+0000", 2024-09-20T18:57:45.699 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 247.227181489, 2024-09-20T18:57:45.699 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.103736439, 2024-09-20T18:57:45.699 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:57:45.699 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:57:45.699 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:57:45.699 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:57:45.699 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:57:45.699 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:57:45.700 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 197546 2024-09-20T18:57:45.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.700 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:57:45.700 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.700 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:57:45.700 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:38.347654+0000", 2024-09-20T18:57:45.700 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:45.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.700 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.700 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:57:45.700 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:38.347654+0000", 2024-09-20T18:57:45.700 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:57:45.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.700 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.701 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:57:45.701 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:38.347655+0000", 2024-09-20T18:57:45.701 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1510000000000001e-06 2024-09-20T18:57:45.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.701 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.701 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:57:45.701 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:38.347660+0000", 2024-09-20T18:57:45.701 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6210000000000003e-06 2024-09-20T18:57:45.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.701 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.701 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:57:45.701 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:38.347671+0000", 2024-09-20T18:57:45.701 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1399999999999999e-05 2024-09-20T18:57:45.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.702 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.702 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:57:45.702 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:38.347679+0000", 2024-09-20T18:57:45.702 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.1200000000000002e-06 2024-09-20T18:57:45.702 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.702 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.702 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:57:45.702 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:38.392261+0000", 2024-09-20T18:57:45.702 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.044581335 2024-09-20T18:57:45.702 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.702 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.702 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:57:45.702 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:38.392366+0000", 2024-09-20T18:57:45.702 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010553299999999999 2024-09-20T18:57:45.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.703 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.703 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:57:45.703 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:38.392476+0000", 2024-09-20T18:57:45.703 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000110093 2024-09-20T18:57:45.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.703 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.703 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:57:45.703 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:38.395765+0000", 2024-09-20T18:57:45.703 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0032884979999999999 2024-09-20T18:57:45.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.703 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.703 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:45.703 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:38.399134+0000", 2024-09-20T18:57:45.704 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003369244 2024-09-20T18:57:45.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.704 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.704 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:57:45.704 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:38.451307+0000", 2024-09-20T18:57:45.704 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.052172832000000002 2024-09-20T18:57:45.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.704 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.704 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:57:45.704 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:38.451351+0000", 2024-09-20T18:57:45.704 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3890999999999997e-05 2024-09-20T18:57:45.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:57:45.704 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:57:45.704 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:57:45.705 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:38.451391+0000", 2024-09-20T18:57:45.705 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9721000000000001e-05 2024-09-20T18:57:45.705 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:45.705 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:45.705 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:45.705 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:57:45.705 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:57:45.705 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T18:57:45.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:45.706 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:45.706 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:45.731 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-20T18:57:45.806 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:45.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:45.848 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:45.848 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:45.949 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:45.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:45.989 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:45.989 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:46.090 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:46.124 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:46.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:46.125 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:46.226 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:46.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:46.263 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:46.263 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:46.299 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:46 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:57:46.299 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:46 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:57:46.299 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:46 smithi083 ceph-mon[29421]: pgmap v380: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 2.1 GiB data, 8.4 GiB used, 617 GiB / 626 GiB avail; 622 B/s rd, 0 B/s wr, 781 op/s; 2458/104394 objects degraded (2.355%); 69/104394 objects misplaced (0.066%); 2.0 MiB/s, 31 objects/s recovering 2024-09-20T18:57:46.299 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:46 smithi083 ceph-mon[29421]: osdmap e227: 8 total, 8 up, 7 in 2024-09-20T18:57:46.299 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:46 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1276599549' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:57:46.299 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:46 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:57:46.299 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:46 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:57:46.299 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:46 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:57:46.299 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:46 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 2458/104394 objects degraded (2.355%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T18:57:46.299 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:46 smithi083 ceph-mon[29421]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:57:46.300 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:46 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1276599549' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T18:57:46.300 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:46 smithi083 ceph-mon[29421]: osdmap e228: 8 total, 8 up, 7 in 2024-09-20T18:57:46.300 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:46 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:57:46.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:46 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:57:46.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:46 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:57:46.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:46 smithi083 ceph-mon[36260]: pgmap v380: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 2.1 GiB data, 8.4 GiB used, 617 GiB / 626 GiB avail; 622 B/s rd, 0 B/s wr, 781 op/s; 2458/104394 objects degraded (2.355%); 69/104394 objects misplaced (0.066%); 2.0 MiB/s, 31 objects/s recovering 2024-09-20T18:57:46.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:46 smithi083 ceph-mon[36260]: osdmap e227: 8 total, 8 up, 7 in 2024-09-20T18:57:46.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:46 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1276599549' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:57:46.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:46 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:57:46.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:46 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:57:46.301 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:46 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:57:46.301 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:46 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 2458/104394 objects degraded (2.355%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T18:57:46.301 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:46 smithi083 ceph-mon[36260]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:57:46.301 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:46 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1276599549' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T18:57:46.301 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:46 smithi083 ceph-mon[36260]: osdmap e228: 8 total, 8 up, 7 in 2024-09-20T18:57:46.301 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:46 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:57:46.364 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:46.388 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:46.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:46.389 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:46.490 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:46.515 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:46.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:46.516 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:46.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:46 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:57:46.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:46 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:57:46.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:46 smithi142 ceph-mon[32508]: pgmap v380: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 2.1 GiB data, 8.4 GiB used, 617 GiB / 626 GiB avail; 622 B/s rd, 0 B/s wr, 781 op/s; 2458/104394 objects degraded (2.355%); 69/104394 objects misplaced (0.066%); 2.0 MiB/s, 31 objects/s recovering 2024-09-20T18:57:46.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:46 smithi142 ceph-mon[32508]: osdmap e227: 8 total, 8 up, 7 in 2024-09-20T18:57:46.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:46 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1276599549' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:57:46.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:46 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:57:46.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:46 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:57:46.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:46 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:57:46.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:46 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 2458/104394 objects degraded (2.355%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T18:57:46.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:46 smithi142 ceph-mon[32508]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:57:46.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:46 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1276599549' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T18:57:46.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:46 smithi142 ceph-mon[32508]: osdmap e228: 8 total, 8 up, 7 in 2024-09-20T18:57:46.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:46 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:57:46.617 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:46.643 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:46.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:46.644 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:46.648 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T18:57:46.745 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:46.771 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:46.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:46.772 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:46.873 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:46.895 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:46.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:46.896 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:46.997 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:47.022 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:47.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:47.023 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:47.124 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:47.152 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:47.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:47.153 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:47.254 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:47.268 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:57:46 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:57:47.291 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:47.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:47.292 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:47.393 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:47.438 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:47.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:47.439 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:47.540 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:47.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:47.583 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:47.583 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:47.680 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-20T18:57:47.684 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:47.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:47.770 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:47.770 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:47.871 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:57:47.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:47.895 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:47.895 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:57:47.996 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:48.017 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:48.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:48.018 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:48.070 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:57:48.119 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:48.174 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:48.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:48.175 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:48.216 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:48 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:57:48.216 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:48 smithi083 ceph-mon[29421]: 5.17 scrub starts 2024-09-20T18:57:48.216 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:48 smithi083 ceph-mon[29421]: 5.17 scrub ok 2024-09-20T18:57:48.216 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:48 smithi083 ceph-mon[29421]: pgmap v383: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 1.8 GiB data, 7.4 GiB used, 618 GiB / 626 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1.69k op/s; 2382/89049 objects degraded (2.675%); 1.6 MiB/s, 44 objects/s recovering 2024-09-20T18:57:48.216 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:48 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:57:48.216 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:48 smithi083 ceph-mon[36260]: 5.17 scrub starts 2024-09-20T18:57:48.216 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:48 smithi083 ceph-mon[36260]: 5.17 scrub ok 2024-09-20T18:57:48.216 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:48 smithi083 ceph-mon[36260]: pgmap v383: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 1.8 GiB data, 7.4 GiB used, 618 GiB / 626 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1.69k op/s; 2382/89049 objects degraded (2.675%); 1.6 MiB/s, 44 objects/s recovering 2024-09-20T18:57:48.276 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:57:48.300 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:48.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:48.301 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:57:48.309 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:57:48.317 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-20T18:57:48.402 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:48.443 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:48.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:48.444 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:48.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:48 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:57:48.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:48 smithi142 ceph-mon[32508]: 5.17 scrub starts 2024-09-20T18:57:48.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:48 smithi142 ceph-mon[32508]: 5.17 scrub ok 2024-09-20T18:57:48.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:48 smithi142 ceph-mon[32508]: pgmap v383: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 1.8 GiB data, 7.4 GiB used, 618 GiB / 626 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1.69k op/s; 2382/89049 objects degraded (2.675%); 1.6 MiB/s, 44 objects/s recovering 2024-09-20T18:57:48.545 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:48.586 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:48.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:48.587 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:48.688 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:48.734 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:48.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:48.735 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:48.825 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:57:48.836 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:48.859 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:48.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:48.860 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:48.901 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:57:48.902 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:57:48.936 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:57:48.937 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:57:48.961 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:48.973 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:57:48.974 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:57:49.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:49.007 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:49.008 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:49.008 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:57:49.008 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:57:49.042 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:57:49.042 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:57:49.075 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:57:49.076 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:57:49.109 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:49.110 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:57:49.111 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:57:49.133 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:49.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:49.134 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:49.142 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:57:49.143 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:57:49.236 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:49.239 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T18:57:49.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:49.274 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:49.274 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:49.340 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:49 smithi083 ceph-mon[29421]: 8.8 scrub starts 2024-09-20T18:57:49.340 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:49 smithi083 ceph-mon[29421]: 8.8 scrub ok 2024-09-20T18:57:49.340 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:49 smithi083 ceph-mon[29421]: from='client.25213 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:57:49.340 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:49 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2037193382' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:57:49.347 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:49 smithi083 ceph-mon[36260]: 8.8 scrub starts 2024-09-20T18:57:49.347 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:49 smithi083 ceph-mon[36260]: 8.8 scrub ok 2024-09-20T18:57:49.347 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:49 smithi083 ceph-mon[36260]: from='client.25213 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:57:49.347 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:49 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2037193382' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:57:49.376 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:49.393 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":383,"stamp":"2024-09-20T18:57:46.940813+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1932794419,"num_objects":29683,"num_object_clones":0,"num_object_copies":89049,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29683,"num_whiteouts":0,"num_read":1321,"num_read_kb":1178,"num_write":203243,"num_write_kb":7431306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25769,"num_bytes_recovered":1676479433,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74783,"ondisk_log_size":74783,"up":444,"acting":443,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":7806300,"kb_used_data":5748296,"kb_used_omap":10,"kb_used_meta":2057653,"kb_avail":648409764,"statfs":{"total":671965249536,"available":663971598336,"internally_reserved":0,"allocated":5886255104,"data_stored":5883067126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11158,"internal_metadata":2107036778},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,6,4,5,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-689963008,"num_objects":-10528,"num_object_clones":0,"num_object_copies":-31584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":126,"num_objects_misplaced":-401,"num_objects_unfound":0,"num_objects_dirty":-10528,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":10528,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":275,"num_bytes_recovered":10747904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.223788"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":375,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.727785+0000","last_change":"2024-09-20T18:55:28.037494+0000","last_active":"2024-09-20T18:57:45.727785+0000","last_peered":"2024-09-20T18:57:45.727785+0000","last_clean":"2024-09-20T18:57:45.727785+0000","last_became_active":"2024-09-20T18:49:45.943201+0000","last_became_peered":"2024-09-20T18:49:45.943201+0000","last_unstale":"2024-09-20T18:57:45.727785+0000","last_undegraded":"2024-09-20T18:57:45.727785+0000","last_fullsized":"2024-09-20T18:57:45.727785+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:28.037449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:28.037449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":422,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.718556+0000","last_change":"2024-09-20T18:55:28.838729+0000","last_active":"2024-09-20T18:57:45.718556+0000","last_peered":"2024-09-20T18:57:45.718556+0000","last_clean":"2024-09-20T18:57:45.718556+0000","last_became_active":"2024-09-20T18:55:28.819637+0000","last_became_peered":"2024-09-20T18:55:28.819637+0000","last_unstale":"2024-09-20T18:57:45.718556+0000","last_undegraded":"2024-09-20T18:57:45.718556+0000","last_fullsized":"2024-09-20T18:57:45.718556+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:50.231620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:50.231620+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":368,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.737539+0000","last_change":"2024-09-20T18:54:25.361988+0000","last_active":"2024-09-20T18:57:45.737539+0000","last_peered":"2024-09-20T18:57:45.737539+0000","last_clean":"2024-09-20T18:57:45.737539+0000","last_became_active":"2024-09-20T18:49:49.959514+0000","last_became_peered":"2024-09-20T18:49:49.959514+0000","last_unstale":"2024-09-20T18:57:45.737539+0000","last_undegraded":"2024-09-20T18:57:45.737539+0000","last_fullsized":"2024-09-20T18:57:45.737539+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:25.361935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:25.361935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":364,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.719463+0000","last_change":"2024-09-20T18:54:15.221404+0000","last_active":"2024-09-20T18:57:45.719463+0000","last_peered":"2024-09-20T18:57:45.719463+0000","last_clean":"2024-09-20T18:57:45.719463+0000","last_became_active":"2024-09-20T18:49:51.969421+0000","last_became_peered":"2024-09-20T18:49:51.969421+0000","last_unstale":"2024-09-20T18:57:45.719463+0000","last_undegraded":"2024-09-20T18:57:45.719463+0000","last_fullsized":"2024-09-20T18:57:45.719463+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:15.221342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:15.221342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":371,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.731824+0000","last_change":"2024-09-20T18:55:28.822034+0000","last_active":"2024-09-20T18:57:45.731824+0000","last_peered":"2024-09-20T18:57:45.731824+0000","last_clean":"2024-09-20T18:57:45.731824+0000","last_became_active":"2024-09-20T18:55:28.821717+0000","last_became_peered":"2024-09-20T18:55:28.821717+0000","last_unstale":"2024-09-20T18:57:45.731824+0000","last_undegraded":"2024-09-20T18:57:45.731824+0000","last_fullsized":"2024-09-20T18:57:45.731824+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.216188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.216188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":375,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.732287+0000","last_change":"2024-09-20T18:57:22.694362+0000","last_active":"2024-09-20T18:57:45.732287+0000","last_peered":"2024-09-20T18:57:45.732287+0000","last_clean":"2024-09-20T18:57:45.732287+0000","last_became_active":"2024-09-20T18:55:28.824826+0000","last_became_peered":"2024-09-20T18:55:28.824826+0000","last_unstale":"2024-09-20T18:57:45.732287+0000","last_undegraded":"2024-09-20T18:57:45.732287+0000","last_fullsized":"2024-09-20T18:57:45.732287+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:22.694295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:22.694295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:33.473592+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":389,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.731765+0000","last_change":"2024-09-20T18:57:13.774827+0000","last_active":"2024-09-20T18:57:45.731765+0000","last_peered":"2024-09-20T18:57:45.731765+0000","last_clean":"2024-09-20T18:57:45.731765+0000","last_became_active":"2024-09-20T18:49:47.952150+0000","last_became_peered":"2024-09-20T18:49:47.952150+0000","last_unstale":"2024-09-20T18:57:45.731765+0000","last_undegraded":"2024-09-20T18:57:45.731765+0000","last_fullsized":"2024-09-20T18:57:45.731765+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:57:13.774305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:13.774305+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:25.245744+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":367,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.718479+0000","last_change":"2024-09-20T18:55:28.820753+0000","last_active":"2024-09-20T18:57:45.718479+0000","last_peered":"2024-09-20T18:57:45.718479+0000","last_clean":"2024-09-20T18:57:45.718479+0000","last_became_active":"2024-09-20T18:55:28.820362+0000","last_became_peered":"2024-09-20T18:55:28.820362+0000","last_unstale":"2024-09-20T18:57:45.718479+0000","last_undegraded":"2024-09-20T18:57:45.718479+0000","last_fullsized":"2024-09-20T18:57:45.718479+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:11.246922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:11.246922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"59'1","reported_seq":400,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.737586+0000","last_change":"2024-09-20T18:55:29.340140+0000","last_active":"2024-09-20T18:57:45.737586+0000","last_peered":"2024-09-20T18:57:45.737586+0000","last_clean":"2024-09-20T18:57:45.737586+0000","last_became_active":"2024-09-20T18:55:28.833386+0000","last_became_peered":"2024-09-20T18:55:28.833386+0000","last_unstale":"2024-09-20T18:57:45.737586+0000","last_undegraded":"2024-09-20T18:57:45.737586+0000","last_fullsized":"2024-09-20T18:57:45.737586+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:55:29.340083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:29.340083+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":388,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.719746+0000","last_change":"2024-09-20T18:54:53.241889+0000","last_active":"2024-09-20T18:57:45.719746+0000","last_peered":"2024-09-20T18:57:45.719746+0000","last_clean":"2024-09-20T18:57:45.719746+0000","last_became_active":"2024-09-20T18:49:47.947448+0000","last_became_peered":"2024-09-20T18:49:47.947448+0000","last_unstale":"2024-09-20T18:57:45.719746+0000","last_undegraded":"2024-09-20T18:57:45.719746+0000","last_fullsized":"2024-09-20T18:57:45.719746+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:53.241802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:53.241802+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":367,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.737250+0000","last_change":"2024-09-20T18:57:08.539466+0000","last_active":"2024-09-20T18:57:45.737250+0000","last_peered":"2024-09-20T18:57:45.737250+0000","last_clean":"2024-09-20T18:57:45.737250+0000","last_became_active":"2024-09-20T18:49:49.961188+0000","last_became_peered":"2024-09-20T18:49:49.961188+0000","last_unstale":"2024-09-20T18:57:45.737250+0000","last_undegraded":"2024-09-20T18:57:45.737250+0000","last_fullsized":"2024-09-20T18:57:45.737250+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:08.539418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:08.539418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:13.102215+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":339,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.723723+0000","last_change":"2024-09-20T18:49:51.966957+0000","last_active":"2024-09-20T18:57:45.723723+0000","last_peered":"2024-09-20T18:57:45.723723+0000","last_clean":"2024-09-20T18:57:45.723723+0000","last_became_active":"2024-09-20T18:49:51.966715+0000","last_became_peered":"2024-09-20T18:49:51.966715+0000","last_unstale":"2024-09-20T18:57:45.723723+0000","last_undegraded":"2024-09-20T18:57:45.723723+0000","last_fullsized":"2024-09-20T18:57:45.723723+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":367,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.737388+0000","last_change":"2024-09-20T18:55:15.252839+0000","last_active":"2024-09-20T18:57:45.737388+0000","last_peered":"2024-09-20T18:57:45.737388+0000","last_clean":"2024-09-20T18:57:45.737388+0000","last_became_active":"2024-09-20T18:49:45.944703+0000","last_became_peered":"2024-09-20T18:49:45.944703+0000","last_unstale":"2024-09-20T18:57:45.737388+0000","last_undegraded":"2024-09-20T18:57:45.737388+0000","last_fullsized":"2024-09-20T18:57:45.737388+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:15.252782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:15.252782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":392,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.720086+0000","last_change":"2024-09-20T18:51:26.494194+0000","last_active":"2024-09-20T18:57:45.720086+0000","last_peered":"2024-09-20T18:57:45.720086+0000","last_clean":"2024-09-20T18:57:45.720086+0000","last_became_active":"2024-09-20T18:51:23.813399+0000","last_became_peered":"2024-09-20T18:51:23.813399+0000","last_unstale":"2024-09-20T18:57:45.720086+0000","last_undegraded":"2024-09-20T18:57:45.720086+0000","last_fullsized":"2024-09-20T18:57:45.720086+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-20T18:51:26.494118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:26.494118+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:31.887636+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":359,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.721515+0000","last_change":"2024-09-20T18:52:38.799542+0000","last_active":"2024-09-20T18:57:45.721515+0000","last_peered":"2024-09-20T18:57:45.721515+0000","last_clean":"2024-09-20T18:57:45.721515+0000","last_became_active":"2024-09-20T18:49:49.959696+0000","last_became_peered":"2024-09-20T18:49:49.959696+0000","last_unstale":"2024-09-20T18:57:45.721515+0000","last_undegraded":"2024-09-20T18:57:45.721515+0000","last_fullsized":"2024-09-20T18:57:45.721515+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:38.799487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:38.799487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":363,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.737135+0000","last_change":"2024-09-20T18:57:13.587827+0000","last_active":"2024-09-20T18:57:45.737135+0000","last_peered":"2024-09-20T18:57:45.737135+0000","last_clean":"2024-09-20T18:57:45.737135+0000","last_became_active":"2024-09-20T18:49:51.970663+0000","last_became_peered":"2024-09-20T18:49:51.970663+0000","last_unstale":"2024-09-20T18:57:45.737135+0000","last_undegraded":"2024-09-20T18:57:45.737135+0000","last_fullsized":"2024-09-20T18:57:45.737135+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:13.587774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:13.587774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:15.272228+0000","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":363,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.720017+0000","last_change":"2024-09-20T18:55:28.828417+0000","last_active":"2024-09-20T18:57:45.720017+0000","last_peered":"2024-09-20T18:57:45.720017+0000","last_clean":"2024-09-20T18:57:45.720017+0000","last_became_active":"2024-09-20T18:55:28.827435+0000","last_became_peered":"2024-09-20T18:55:28.827435+0000","last_unstale":"2024-09-20T18:57:45.720017+0000","last_undegraded":"2024-09-20T18:57:45.720017+0000","last_fullsized":"2024-09-20T18:57:45.720017+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:53.940191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:53.940191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":367,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.725649+0000","last_change":"2024-09-20T18:54:23.984024+0000","last_active":"2024-09-20T18:57:45.725649+0000","last_peered":"2024-09-20T18:57:45.725649+0000","last_clean":"2024-09-20T18:57:45.725649+0000","last_became_active":"2024-09-20T18:49:45.950480+0000","last_became_peered":"2024-09-20T18:49:45.950480+0000","last_unstale":"2024-09-20T18:57:45.725649+0000","last_undegraded":"2024-09-20T18:57:45.725649+0000","last_fullsized":"2024-09-20T18:57:45.725649+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:23.983958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:23.983958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":390,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.730967+0000","last_change":"2024-09-20T18:53:26.762207+0000","last_active":"2024-09-20T18:57:45.730967+0000","last_peered":"2024-09-20T18:57:45.730967+0000","last_clean":"2024-09-20T18:57:45.730967+0000","last_became_active":"2024-09-20T18:49:47.950939+0000","last_became_peered":"2024-09-20T18:49:47.950939+0000","last_unstale":"2024-09-20T18:57:45.730967+0000","last_undegraded":"2024-09-20T18:57:45.730967+0000","last_fullsized":"2024-09-20T18:57:45.730967+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:53:26.762131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:26.762131+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":360,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.734082+0000","last_change":"2024-09-20T18:57:14.941518+0000","last_active":"2024-09-20T18:57:45.734082+0000","last_peered":"2024-09-20T18:57:45.734082+0000","last_clean":"2024-09-20T18:57:45.734082+0000","last_became_active":"2024-09-20T18:49:49.958518+0000","last_became_peered":"2024-09-20T18:49:49.958518+0000","last_unstale":"2024-09-20T18:57:45.734082+0000","last_undegraded":"2024-09-20T18:57:45.734082+0000","last_fullsized":"2024-09-20T18:57:45.734082+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:14.941458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:14.941458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:37.988570+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":363,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.734967+0000","last_change":"2024-09-20T18:55:28.833468+0000","last_active":"2024-09-20T18:57:45.734967+0000","last_peered":"2024-09-20T18:57:45.734967+0000","last_clean":"2024-09-20T18:57:45.734967+0000","last_became_active":"2024-09-20T18:55:28.833133+0000","last_became_peered":"2024-09-20T18:55:28.833133+0000","last_unstale":"2024-09-20T18:57:45.734967+0000","last_undegraded":"2024-09-20T18:57:45.734967+0000","last_fullsized":"2024-09-20T18:57:45.734967+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:54.259174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:54.259174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:30.790542+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"59'1","reported_seq":374,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.719784+0000","last_change":"2024-09-20T18:53:42.840884+0000","last_active":"2024-09-20T18:57:45.719784+0000","last_peered":"2024-09-20T18:57:45.719784+0000","last_clean":"2024-09-20T18:57:45.719784+0000","last_became_active":"2024-09-20T18:49:45.940100+0000","last_became_peered":"2024-09-20T18:49:45.940100+0000","last_unstale":"2024-09-20T18:57:45.719784+0000","last_undegraded":"2024-09-20T18:57:45.719784+0000","last_fullsized":"2024-09-20T18:57:45.719784+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:53:42.840800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:42.840800+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":379,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.731536+0000","last_change":"2024-09-20T18:57:09.819080+0000","last_active":"2024-09-20T18:57:45.731536+0000","last_peered":"2024-09-20T18:57:45.731536+0000","last_clean":"2024-09-20T18:57:45.731536+0000","last_became_active":"2024-09-20T18:49:47.950909+0000","last_became_peered":"2024-09-20T18:49:47.950909+0000","last_unstale":"2024-09-20T18:57:45.731536+0000","last_undegraded":"2024-09-20T18:57:45.731536+0000","last_fullsized":"2024-09-20T18:57:45.731536+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:57:09.818999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:09.818999+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:17.216750+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":368,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.719588+0000","last_change":"2024-09-20T18:54:44.239273+0000","last_active":"2024-09-20T18:57:45.719588+0000","last_peered":"2024-09-20T18:57:45.719588+0000","last_clean":"2024-09-20T18:57:45.719588+0000","last_became_active":"2024-09-20T18:49:49.959102+0000","last_became_peered":"2024-09-20T18:49:49.959102+0000","last_unstale":"2024-09-20T18:57:45.719588+0000","last_undegraded":"2024-09-20T18:57:45.719588+0000","last_fullsized":"2024-09-20T18:57:45.719588+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:44.239226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.239226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":356,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.721845+0000","last_change":"2024-09-20T18:52:15.826819+0000","last_active":"2024-09-20T18:57:45.721845+0000","last_peered":"2024-09-20T18:57:45.721845+0000","last_clean":"2024-09-20T18:57:45.721845+0000","last_became_active":"2024-09-20T18:49:51.970058+0000","last_became_peered":"2024-09-20T18:49:51.970058+0000","last_unstale":"2024-09-20T18:57:45.721845+0000","last_undegraded":"2024-09-20T18:57:45.721845+0000","last_fullsized":"2024-09-20T18:57:45.721845+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T18:52:15.826638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:15.826638+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":369,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.737684+0000","last_change":"2024-09-20T18:57:18.032140+0000","last_active":"2024-09-20T18:57:45.737684+0000","last_peered":"2024-09-20T18:57:45.737684+0000","last_clean":"2024-09-20T18:57:45.737684+0000","last_became_active":"2024-09-20T18:49:45.933078+0000","last_became_peered":"2024-09-20T18:49:45.933078+0000","last_unstale":"2024-09-20T18:57:45.737684+0000","last_undegraded":"2024-09-20T18:57:45.737684+0000","last_fullsized":"2024-09-20T18:57:45.737684+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:57:18.032067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:18.032067+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:42.727785+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":386,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.727600+0000","last_change":"2024-09-20T18:55:19.996522+0000","last_active":"2024-09-20T18:57:45.727600+0000","last_peered":"2024-09-20T18:57:45.727600+0000","last_clean":"2024-09-20T18:57:45.727600+0000","last_became_active":"2024-09-20T18:49:48.500809+0000","last_became_peered":"2024-09-20T18:49:48.500809+0000","last_unstale":"2024-09-20T18:57:45.727600+0000","last_undegraded":"2024-09-20T18:57:45.727600+0000","last_fullsized":"2024-09-20T18:57:45.727600+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:55:19.996328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:19.996328+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"184'9","reported_seq":166,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.719252+0000","last_change":"2024-09-20T18:55:28.845188+0000","last_active":"2024-09-20T18:57:45.719252+0000","last_peered":"2024-09-20T18:57:45.719252+0000","last_clean":"2024-09-20T18:57:45.719252+0000","last_became_active":"2024-09-20T18:55:28.820941+0000","last_became_peered":"2024-09-20T18:55:28.820941+0000","last_unstale":"2024-09-20T18:57:45.719252+0000","last_undegraded":"2024-09-20T18:57:45.719252+0000","last_fullsized":"2024-09-20T18:57:45.719252+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:53:44.524040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:44.524040+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3","version":"228'12431","reported_seq":24909,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:46.346965+0000","last_change":"2024-09-20T18:56:25.697520+0000","last_active":"2024-09-20T18:57:46.346965+0000","last_peered":"2024-09-20T18:57:46.346965+0000","last_clean":"2024-09-20T18:57:46.346965+0000","last_became_active":"2024-09-20T18:56:25.687391+0000","last_became_peered":"2024-09-20T18:56:25.687391+0000","last_unstale":"2024-09-20T18:57:46.346965+0000","last_undegraded":"2024-09-20T18:57:46.346965+0000","last_fullsized":"2024-09-20T18:57:46.346965+0000","mapping_epoch":203,"log_start":"206'8172","ondisk_log_start":"206'8172","created":160,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4259,"log_dups_size":0,"ondisk_log_size":4259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":122224640,"num_objects":1865,"num_object_clones":0,"num_object_copies":5595,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12431,"num_write_kb":457472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6156,"num_bytes_recovered":403439616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":381,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.730769+0000","last_change":"2024-09-20T18:55:18.581290+0000","last_active":"2024-09-20T18:57:45.730769+0000","last_peered":"2024-09-20T18:57:45.730769+0000","last_clean":"2024-09-20T18:57:45.730769+0000","last_became_active":"2024-09-20T18:49:58.992302+0000","last_became_peered":"2024-09-20T18:49:58.992302+0000","last_unstale":"2024-09-20T18:57:45.730769+0000","last_undegraded":"2024-09-20T18:57:45.730769+0000","last_fullsized":"2024-09-20T18:57:45.730769+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:55:18.581200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:18.581200+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":368,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.733873+0000","last_change":"2024-09-20T18:54:54.288199+0000","last_active":"2024-09-20T18:57:45.733873+0000","last_peered":"2024-09-20T18:57:45.733873+0000","last_clean":"2024-09-20T18:57:45.733873+0000","last_became_active":"2024-09-20T18:49:45.936104+0000","last_became_peered":"2024-09-20T18:49:45.936104+0000","last_unstale":"2024-09-20T18:57:45.733873+0000","last_undegraded":"2024-09-20T18:57:45.733873+0000","last_fullsized":"2024-09-20T18:57:45.733873+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:54.288117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:54.288117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":46,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":467,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.720044+0000","last_change":"2024-09-20T18:54:14.292660+0000","last_active":"2024-09-20T18:57:45.720044+0000","last_peered":"2024-09-20T18:57:45.720044+0000","last_clean":"2024-09-20T18:57:45.720044+0000","last_became_active":"2024-09-20T18:49:49.960697+0000","last_became_peered":"2024-09-20T18:49:49.960697+0000","last_unstale":"2024-09-20T18:57:45.720044+0000","last_undegraded":"2024-09-20T18:57:45.720044+0000","last_fullsized":"2024-09-20T18:57:45.720044+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:54:14.292610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:14.292610+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":34,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":355,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.737342+0000","last_change":"2024-09-20T18:54:10.381171+0000","last_active":"2024-09-20T18:57:45.737342+0000","last_peered":"2024-09-20T18:57:45.737342+0000","last_clean":"2024-09-20T18:57:45.737342+0000","last_became_active":"2024-09-20T18:49:51.971399+0000","last_became_peered":"2024-09-20T18:49:51.971399+0000","last_unstale":"2024-09-20T18:57:45.737342+0000","last_undegraded":"2024-09-20T18:57:45.737342+0000","last_fullsized":"2024-09-20T18:57:45.737342+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:10.381108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:10.381108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:30.527012+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.c","version":"228'12613","reported_seq":12761,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:46.891003+0000","last_change":"2024-09-20T18:54:34.311256+0000","last_active":"2024-09-20T18:57:46.891003+0000","last_peered":"2024-09-20T18:57:46.891003+0000","last_clean":"2024-09-20T18:57:46.891003+0000","last_became_active":"2024-09-20T18:54:34.310904+0000","last_became_peered":"2024-09-20T18:54:34.310904+0000","last_unstale":"2024-09-20T18:57:46.891003+0000","last_undegraded":"2024-09-20T18:57:46.891003+0000","last_fullsized":"2024-09-20T18:57:46.891003+0000","mapping_epoch":160,"log_start":"199'7711","ondisk_log_start":"199'7711","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4902,"log_dups_size":0,"ondisk_log_size":4902,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":116588544,"num_objects":1779,"num_object_clones":0,"num_object_copies":5337,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12613,"num_write_kb":460544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":396,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.733658+0000","last_change":"2024-09-20T18:54:22.077299+0000","last_active":"2024-09-20T18:57:45.733658+0000","last_peered":"2024-09-20T18:57:45.733658+0000","last_clean":"2024-09-20T18:57:45.733658+0000","last_became_active":"2024-09-20T18:49:48.499189+0000","last_became_peered":"2024-09-20T18:49:48.499189+0000","last_unstale":"2024-09-20T18:57:45.733658+0000","last_undegraded":"2024-09-20T18:57:45.733658+0000","last_fullsized":"2024-09-20T18:57:45.733658+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-20T18:54:22.077212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:22.077212+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":360,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.735428+0000","last_change":"2024-09-20T18:54:57.237740+0000","last_active":"2024-09-20T18:57:45.735428+0000","last_peered":"2024-09-20T18:57:45.735428+0000","last_clean":"2024-09-20T18:57:45.735428+0000","last_became_active":"2024-09-20T18:49:45.940544+0000","last_became_peered":"2024-09-20T18:49:45.940544+0000","last_unstale":"2024-09-20T18:57:45.735428+0000","last_undegraded":"2024-09-20T18:57:45.735428+0000","last_fullsized":"2024-09-20T18:57:45.735428+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.237675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.237675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":368,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.718680+0000","last_change":"2024-09-20T18:54:51.229251+0000","last_active":"2024-09-20T18:57:45.718680+0000","last_peered":"2024-09-20T18:57:45.718680+0000","last_clean":"2024-09-20T18:57:45.718680+0000","last_became_active":"2024-09-20T18:49:49.963073+0000","last_became_peered":"2024-09-20T18:49:49.963073+0000","last_unstale":"2024-09-20T18:57:45.718680+0000","last_undegraded":"2024-09-20T18:57:45.718680+0000","last_fullsized":"2024-09-20T18:57:45.718680+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.229201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.229201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":364,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.719669+0000","last_change":"2024-09-20T18:54:55.323937+0000","last_active":"2024-09-20T18:57:45.719669+0000","last_peered":"2024-09-20T18:57:45.719669+0000","last_clean":"2024-09-20T18:57:45.719669+0000","last_became_active":"2024-09-20T18:49:51.968793+0000","last_became_peered":"2024-09-20T18:49:51.968793+0000","last_unstale":"2024-09-20T18:57:45.719669+0000","last_undegraded":"2024-09-20T18:57:45.719669+0000","last_fullsized":"2024-09-20T18:57:45.719669+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:55.323692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:55.323692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"228'12519","reported_seq":12668,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.923930+0000","last_change":"2024-09-20T18:54:34.311680+0000","last_active":"2024-09-20T18:57:45.923930+0000","last_peered":"2024-09-20T18:57:45.923930+0000","last_clean":"2024-09-20T18:57:45.923930+0000","last_became_active":"2024-09-20T18:54:34.311241+0000","last_became_peered":"2024-09-20T18:54:34.311241+0000","last_unstale":"2024-09-20T18:57:45.923930+0000","last_undegraded":"2024-09-20T18:57:45.923930+0000","last_fullsized":"2024-09-20T18:57:45.923930+0000","mapping_epoch":160,"log_start":"197'7446","ondisk_log_start":"197'7446","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":5073,"log_dups_size":0,"ondisk_log_size":5073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:36.546826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121372696,"num_objects":1853,"num_object_clones":0,"num_object_copies":5559,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1853,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12519,"num_write_kb":459841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":424,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.718795+0000","last_change":"2024-09-20T18:55:28.853375+0000","last_active":"2024-09-20T18:57:45.718795+0000","last_peered":"2024-09-20T18:57:45.718795+0000","last_clean":"2024-09-20T18:57:45.718795+0000","last_became_active":"2024-09-20T18:55:28.831057+0000","last_became_peered":"2024-09-20T18:55:28.831057+0000","last_unstale":"2024-09-20T18:57:45.718795+0000","last_undegraded":"2024-09-20T18:57:45.718795+0000","last_fullsized":"2024-09-20T18:57:45.718795+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-20T18:54:57.394893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.394893+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":360,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.727843+0000","last_change":"2024-09-20T18:51:27.187060+0000","last_active":"2024-09-20T18:57:45.727843+0000","last_peered":"2024-09-20T18:57:45.727843+0000","last_clean":"2024-09-20T18:57:45.727843+0000","last_became_active":"2024-09-20T18:49:45.940469+0000","last_became_peered":"2024-09-20T18:49:45.940469+0000","last_unstale":"2024-09-20T18:57:45.727843+0000","last_undegraded":"2024-09-20T18:57:45.727843+0000","last_fullsized":"2024-09-20T18:57:45.727843+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:51:27.186991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:27.186991+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"66'8","reported_seq":473,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.733515+0000","last_change":"2024-09-20T18:54:47.223010+0000","last_active":"2024-09-20T18:57:45.733515+0000","last_peered":"2024-09-20T18:57:45.733515+0000","last_clean":"2024-09-20T18:57:45.733515+0000","last_became_active":"2024-09-20T18:49:49.956133+0000","last_became_peered":"2024-09-20T18:49:49.956133+0000","last_unstale":"2024-09-20T18:57:45.733515+0000","last_undegraded":"2024-09-20T18:57:45.733515+0000","last_fullsized":"2024-09-20T18:57:45.733515+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:54:47.222937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:47.222937+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":362,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.721519+0000","last_change":"2024-09-20T18:55:28.819147+0000","last_active":"2024-09-20T18:57:45.721519+0000","last_peered":"2024-09-20T18:57:45.721519+0000","last_clean":"2024-09-20T18:57:45.721519+0000","last_became_active":"2024-09-20T18:55:28.818804+0000","last_became_peered":"2024-09-20T18:55:28.818804+0000","last_unstale":"2024-09-20T18:57:45.721519+0000","last_undegraded":"2024-09-20T18:57:45.721519+0000","last_fullsized":"2024-09-20T18:57:45.721519+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:42.880724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:42.880724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"228'12635","reported_seq":12782,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:46.889350+0000","last_change":"2024-09-20T18:54:34.309939+0000","last_active":"2024-09-20T18:57:46.889350+0000","last_peered":"2024-09-20T18:57:46.889350+0000","last_clean":"2024-09-20T18:57:46.889350+0000","last_became_active":"2024-09-20T18:54:34.309630+0000","last_became_peered":"2024-09-20T18:54:34.309630+0000","last_unstale":"2024-09-20T18:57:46.889350+0000","last_undegraded":"2024-09-20T18:57:46.889350+0000","last_fullsized":"2024-09-20T18:57:46.889350+0000","mapping_epoch":160,"log_start":"200'7749","ondisk_log_start":"200'7749","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4886,"log_dups_size":0,"ondisk_log_size":4886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115802112,"num_objects":1767,"num_object_clones":0,"num_object_copies":5301,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12635,"num_write_kb":460864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":387,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.737926+0000","last_change":"2024-09-20T18:55:19.541011+0000","last_active":"2024-09-20T18:57:45.737926+0000","last_peered":"2024-09-20T18:57:45.737926+0000","last_clean":"2024-09-20T18:57:45.737926+0000","last_became_active":"2024-09-20T18:49:47.949626+0000","last_became_peered":"2024-09-20T18:49:47.949626+0000","last_unstale":"2024-09-20T18:57:45.737926+0000","last_undegraded":"2024-09-20T18:57:45.737926+0000","last_fullsized":"2024-09-20T18:57:45.737926+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:55:19.540812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:19.540812+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":375,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.737683+0000","last_change":"2024-09-20T18:54:18.326947+0000","last_active":"2024-09-20T18:57:45.737683+0000","last_peered":"2024-09-20T18:57:45.737683+0000","last_clean":"2024-09-20T18:57:45.737683+0000","last_became_active":"2024-09-20T18:49:45.942496+0000","last_became_peered":"2024-09-20T18:49:45.942496+0000","last_unstale":"2024-09-20T18:57:45.737683+0000","last_undegraded":"2024-09-20T18:57:45.737683+0000","last_fullsized":"2024-09-20T18:57:45.737683+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:18.326875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:18.326875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:32.505949+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"184'9","reported_seq":475,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.725347+0000","last_change":"2024-09-20T18:55:29.010062+0000","last_active":"2024-09-20T18:57:45.725347+0000","last_peered":"2024-09-20T18:57:45.725347+0000","last_clean":"2024-09-20T18:57:45.725347+0000","last_became_active":"2024-09-20T18:55:28.830643+0000","last_became_peered":"2024-09-20T18:55:28.830643+0000","last_unstale":"2024-09-20T18:57:45.725347+0000","last_undegraded":"2024-09-20T18:57:45.725347+0000","last_fullsized":"2024-09-20T18:57:45.725347+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:29.010015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:29.010015+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":339,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.723884+0000","last_change":"2024-09-20T18:49:51.965733+0000","last_active":"2024-09-20T18:57:45.723884+0000","last_peered":"2024-09-20T18:57:45.723884+0000","last_clean":"2024-09-20T18:57:45.723884+0000","last_became_active":"2024-09-20T18:49:51.964780+0000","last_became_peered":"2024-09-20T18:49:51.964780+0000","last_unstale":"2024-09-20T18:57:45.723884+0000","last_undegraded":"2024-09-20T18:57:45.723884+0000","last_fullsized":"2024-09-20T18:57:45.723884+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"228'12901","reported_seq":13049,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:46.534137+0000","last_change":"2024-09-20T18:54:34.309809+0000","last_active":"2024-09-20T18:57:46.534137+0000","last_peered":"2024-09-20T18:57:46.534137+0000","last_clean":"2024-09-20T18:57:46.534137+0000","last_became_active":"2024-09-20T18:54:34.309231+0000","last_became_peered":"2024-09-20T18:54:34.309231+0000","last_unstale":"2024-09-20T18:57:46.534137+0000","last_undegraded":"2024-09-20T18:57:46.534137+0000","last_fullsized":"2024-09-20T18:57:46.534137+0000","mapping_epoch":160,"log_start":"206'8532","ondisk_log_start":"206'8532","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4369,"log_dups_size":0,"ondisk_log_size":4369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118816768,"num_objects":1813,"num_object_clones":0,"num_object_copies":5439,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12901,"num_write_kb":470848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":399,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.725479+0000","last_change":"2024-09-20T18:54:14.923895+0000","last_active":"2024-09-20T18:57:45.725479+0000","last_peered":"2024-09-20T18:57:45.725479+0000","last_clean":"2024-09-20T18:57:45.725479+0000","last_became_active":"2024-09-20T18:49:48.497736+0000","last_became_peered":"2024-09-20T18:49:48.497736+0000","last_unstale":"2024-09-20T18:57:45.725479+0000","last_undegraded":"2024-09-20T18:57:45.725479+0000","last_fullsized":"2024-09-20T18:57:45.725479+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-20T18:54:14.923682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:14.923682+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":360,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.734487+0000","last_change":"2024-09-20T18:57:09.940634+0000","last_active":"2024-09-20T18:57:45.734487+0000","last_peered":"2024-09-20T18:57:45.734487+0000","last_clean":"2024-09-20T18:57:45.734487+0000","last_became_active":"2024-09-20T18:49:45.942253+0000","last_became_peered":"2024-09-20T18:49:45.942253+0000","last_unstale":"2024-09-20T18:57:45.734487+0000","last_undegraded":"2024-09-20T18:57:45.734487+0000","last_fullsized":"2024-09-20T18:57:45.734487+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:09.940517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:09.940517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:18.833936+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":106,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.724132+0000","last_change":"2024-09-20T18:55:28.830954+0000","last_active":"2024-09-20T18:57:45.724132+0000","last_peered":"2024-09-20T18:57:45.724132+0000","last_clean":"2024-09-20T18:57:45.724132+0000","last_became_active":"2024-09-20T18:55:28.830645+0000","last_became_peered":"2024-09-20T18:55:28.830645+0000","last_unstale":"2024-09-20T18:57:45.724132+0000","last_undegraded":"2024-09-20T18:57:45.724132+0000","last_fullsized":"2024-09-20T18:57:45.724132+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:50.610871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:50.610871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":371,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.737453+0000","last_change":"2024-09-20T18:55:28.822362+0000","last_active":"2024-09-20T18:57:45.737453+0000","last_peered":"2024-09-20T18:57:45.737453+0000","last_clean":"2024-09-20T18:57:45.737453+0000","last_became_active":"2024-09-20T18:55:28.821588+0000","last_became_peered":"2024-09-20T18:55:28.821588+0000","last_unstale":"2024-09-20T18:57:45.737453+0000","last_undegraded":"2024-09-20T18:57:45.737453+0000","last_fullsized":"2024-09-20T18:57:45.737453+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.402841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.402841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"228'12673","reported_seq":12821,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.932219+0000","last_change":"2024-09-20T18:54:34.311561+0000","last_active":"2024-09-20T18:57:45.932219+0000","last_peered":"2024-09-20T18:57:45.932219+0000","last_clean":"2024-09-20T18:57:45.932219+0000","last_became_active":"2024-09-20T18:54:34.311181+0000","last_became_peered":"2024-09-20T18:54:34.311181+0000","last_unstale":"2024-09-20T18:57:45.932219+0000","last_undegraded":"2024-09-20T18:57:45.932219+0000","last_fullsized":"2024-09-20T18:57:45.932219+0000","mapping_epoch":160,"log_start":"198'7594","ondisk_log_start":"198'7594","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":5079,"log_dups_size":0,"ondisk_log_size":5079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":126156800,"num_objects":1925,"num_object_clones":0,"num_object_copies":5775,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12673,"num_write_kb":467136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"66'28","reported_seq":509,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.734392+0000","last_change":"2024-09-20T18:55:16.527267+0000","last_active":"2024-09-20T18:57:45.734392+0000","last_peered":"2024-09-20T18:57:45.734392+0000","last_clean":"2024-09-20T18:57:45.734392+0000","last_became_active":"2024-09-20T18:49:47.955335+0000","last_became_peered":"2024-09-20T18:49:47.955335+0000","last_unstale":"2024-09-20T18:57:45.734392+0000","last_undegraded":"2024-09-20T18:57:45.734392+0000","last_fullsized":"2024-09-20T18:57:45.734392+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'28","last_scrub_stamp":"2024-09-20T18:55:16.527165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:16.527165+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":367,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.720283+0000","last_change":"2024-09-20T18:57:21.807755+0000","last_active":"2024-09-20T18:57:45.720283+0000","last_peered":"2024-09-20T18:57:45.720283+0000","last_clean":"2024-09-20T18:57:45.720283+0000","last_became_active":"2024-09-20T18:55:28.831574+0000","last_became_peered":"2024-09-20T18:55:28.831574+0000","last_unstale":"2024-09-20T18:57:45.720283+0000","last_undegraded":"2024-09-20T18:57:45.720283+0000","last_fullsized":"2024-09-20T18:57:45.720283+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:21.807703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:21.807703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:42.821495+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1],"acting":[4,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":404,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.737619+0000","last_change":"2024-09-20T18:55:28.854615+0000","last_active":"2024-09-20T18:57:45.737619+0000","last_peered":"2024-09-20T18:57:45.737619+0000","last_clean":"2024-09-20T18:57:45.737619+0000","last_became_active":"2024-09-20T18:55:28.837226+0000","last_became_peered":"2024-09-20T18:55:28.837226+0000","last_unstale":"2024-09-20T18:57:45.737619+0000","last_undegraded":"2024-09-20T18:57:45.737619+0000","last_fullsized":"2024-09-20T18:57:45.737619+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-20T18:54:51.227531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.227531+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:43.598074+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0012781159999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":367,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.724016+0000","last_change":"2024-09-20T18:55:13.949348+0000","last_active":"2024-09-20T18:57:45.724016+0000","last_peered":"2024-09-20T18:57:45.724016+0000","last_clean":"2024-09-20T18:57:45.724016+0000","last_became_active":"2024-09-20T18:49:49.953971+0000","last_became_peered":"2024-09-20T18:49:49.953971+0000","last_unstale":"2024-09-20T18:57:45.724016+0000","last_undegraded":"2024-09-20T18:57:45.724016+0000","last_fullsized":"2024-09-20T18:57:45.724016+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:13.949269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:13.949269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":348,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.731949+0000","last_change":"2024-09-20T18:57:12.744116+0000","last_active":"2024-09-20T18:57:45.731949+0000","last_peered":"2024-09-20T18:57:45.731949+0000","last_clean":"2024-09-20T18:57:45.731949+0000","last_became_active":"2024-09-20T18:49:51.969400+0000","last_became_peered":"2024-09-20T18:49:51.969400+0000","last_unstale":"2024-09-20T18:57:45.731949+0000","last_undegraded":"2024-09-20T18:57:45.731949+0000","last_fullsized":"2024-09-20T18:57:45.731949+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:12.744040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:12.744040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:17.964285+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"228'12812","reported_seq":12960,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:46.492125+0000","last_change":"2024-09-20T18:54:33.926785+0000","last_active":"2024-09-20T18:57:46.492125+0000","last_peered":"2024-09-20T18:57:46.492125+0000","last_clean":"2024-09-20T18:57:46.492125+0000","last_became_active":"2024-09-20T18:54:33.925750+0000","last_became_peered":"2024-09-20T18:54:33.925750+0000","last_unstale":"2024-09-20T18:57:46.492125+0000","last_undegraded":"2024-09-20T18:57:46.492125+0000","last_fullsized":"2024-09-20T18:57:46.492125+0000","mapping_epoch":160,"log_start":"206'8440","ondisk_log_start":"206'8440","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4372,"log_dups_size":0,"ondisk_log_size":4372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":124911616,"num_objects":1906,"num_object_clones":0,"num_object_copies":5718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12812,"num_write_kb":470976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":399,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.734678+0000","last_change":"2024-09-20T18:55:25.702762+0000","last_active":"2024-09-20T18:57:45.734678+0000","last_peered":"2024-09-20T18:57:45.734678+0000","last_clean":"2024-09-20T18:57:45.734678+0000","last_became_active":"2024-09-20T18:49:47.950904+0000","last_became_peered":"2024-09-20T18:49:47.950904+0000","last_unstale":"2024-09-20T18:57:45.734678+0000","last_undegraded":"2024-09-20T18:57:45.734678+0000","last_fullsized":"2024-09-20T18:57:45.734678+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-20T18:55:25.702596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:25.702596+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":383,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.742095+0000","last_change":"2024-09-20T18:55:28.830832+0000","last_active":"2024-09-20T18:57:45.742095+0000","last_peered":"2024-09-20T18:57:45.742095+0000","last_clean":"2024-09-20T18:57:45.742095+0000","last_became_active":"2024-09-20T18:55:28.830269+0000","last_became_peered":"2024-09-20T18:55:28.830269+0000","last_unstale":"2024-09-20T18:57:45.742095+0000","last_undegraded":"2024-09-20T18:57:45.742095+0000","last_fullsized":"2024-09-20T18:57:45.742095+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:23.359769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:23.359769+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:27.139436+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":370,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.730687+0000","last_change":"2024-09-20T18:52:07.718766+0000","last_active":"2024-09-20T18:57:45.730687+0000","last_peered":"2024-09-20T18:57:45.730687+0000","last_clean":"2024-09-20T18:57:45.730687+0000","last_became_active":"2024-09-20T18:49:44.937135+0000","last_became_peered":"2024-09-20T18:49:44.937135+0000","last_unstale":"2024-09-20T18:57:45.730687+0000","last_undegraded":"2024-09-20T18:57:45.730687+0000","last_fullsized":"2024-09-20T18:57:45.730687+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:52:07.718646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:52:07.718646+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.00095558800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":353,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.732775+0000","last_change":"2024-09-20T18:51:59.796450+0000","last_active":"2024-09-20T18:57:45.732775+0000","last_peered":"2024-09-20T18:57:45.732775+0000","last_clean":"2024-09-20T18:57:45.732775+0000","last_became_active":"2024-09-20T18:49:49.954424+0000","last_became_peered":"2024-09-20T18:49:49.954424+0000","last_unstale":"2024-09-20T18:57:45.732775+0000","last_undegraded":"2024-09-20T18:57:45.732775+0000","last_fullsized":"2024-09-20T18:57:45.732775+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:59.796383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:59.796383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":354,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.722671+0000","last_change":"2024-09-20T18:55:28.826659+0000","last_active":"2024-09-20T18:57:45.722671+0000","last_peered":"2024-09-20T18:57:45.722671+0000","last_clean":"2024-09-20T18:57:45.722671+0000","last_became_active":"2024-09-20T18:55:28.825812+0000","last_became_peered":"2024-09-20T18:55:28.825812+0000","last_unstale":"2024-09-20T18:57:45.722671+0000","last_undegraded":"2024-09-20T18:57:45.722671+0000","last_fullsized":"2024-09-20T18:57:45.722671+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:39.814259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:39.814259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"228'12819","reported_seq":12967,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:46.329937+0000","last_change":"2024-09-20T18:54:34.311774+0000","last_active":"2024-09-20T18:57:46.329937+0000","last_peered":"2024-09-20T18:57:46.329937+0000","last_clean":"2024-09-20T18:57:46.329937+0000","last_became_active":"2024-09-20T18:54:34.311487+0000","last_became_peered":"2024-09-20T18:54:34.311487+0000","last_unstale":"2024-09-20T18:57:46.329937+0000","last_undegraded":"2024-09-20T18:57:46.329937+0000","last_fullsized":"2024-09-20T18:57:46.329937+0000","mapping_epoch":160,"log_start":"206'8592","ondisk_log_start":"206'8592","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4227,"log_dups_size":0,"ondisk_log_size":4227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:43.066892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117899264,"num_objects":1799,"num_object_clones":0,"num_object_copies":5397,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1799,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12819,"num_write_kb":467776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":397,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.727680+0000","last_change":"2024-09-20T18:55:14.951950+0000","last_active":"2024-09-20T18:57:45.727680+0000","last_peered":"2024-09-20T18:57:45.727680+0000","last_clean":"2024-09-20T18:57:45.727680+0000","last_became_active":"2024-09-20T18:49:47.949230+0000","last_became_peered":"2024-09-20T18:49:47.949230+0000","last_unstale":"2024-09-20T18:57:45.727680+0000","last_undegraded":"2024-09-20T18:57:45.727680+0000","last_fullsized":"2024-09-20T18:57:45.727680+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:55:14.951892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:14.951892+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":375,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.728583+0000","last_change":"2024-09-20T18:54:27.905856+0000","last_active":"2024-09-20T18:57:45.728583+0000","last_peered":"2024-09-20T18:57:45.728583+0000","last_clean":"2024-09-20T18:57:45.728583+0000","last_became_active":"2024-09-20T18:49:45.939197+0000","last_became_peered":"2024-09-20T18:49:45.939197+0000","last_unstale":"2024-09-20T18:57:45.728583+0000","last_undegraded":"2024-09-20T18:57:45.728583+0000","last_fullsized":"2024-09-20T18:57:45.728583+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:27.905803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:27.905803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":862,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.879468+0000","last_change":"2024-09-20T18:54:26.359887+0000","last_active":"2024-09-20T18:57:45.879468+0000","last_peered":"2024-09-20T18:57:45.879468+0000","last_clean":"2024-09-20T18:57:45.879468+0000","last_became_active":"2024-09-20T18:49:44.936819+0000","last_became_peered":"2024-09-20T18:49:44.936819+0000","last_unstale":"2024-09-20T18:57:45.879468+0000","last_undegraded":"2024-09-20T18:57:45.879468+0000","last_fullsized":"2024-09-20T18:57:45.879468+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-20T18:54:26.359807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:54:26.359807+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0010996199999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":475,"num_read_kb":470,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":359,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.737960+0000","last_change":"2024-09-20T18:54:59.151228+0000","last_active":"2024-09-20T18:57:45.737960+0000","last_peered":"2024-09-20T18:57:45.737960+0000","last_clean":"2024-09-20T18:57:45.737960+0000","last_became_active":"2024-09-20T18:49:49.961066+0000","last_became_peered":"2024-09-20T18:49:49.961066+0000","last_unstale":"2024-09-20T18:57:45.737960+0000","last_undegraded":"2024-09-20T18:57:45.737960+0000","last_fullsized":"2024-09-20T18:57:45.737960+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:59.150063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:59.150063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":364,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.734107+0000","last_change":"2024-09-20T18:55:21.629867+0000","last_active":"2024-09-20T18:57:45.734107+0000","last_peered":"2024-09-20T18:57:45.734107+0000","last_clean":"2024-09-20T18:57:45.734107+0000","last_became_active":"2024-09-20T18:49:51.970343+0000","last_became_peered":"2024-09-20T18:49:51.970343+0000","last_unstale":"2024-09-20T18:57:45.734107+0000","last_undegraded":"2024-09-20T18:57:45.734107+0000","last_fullsized":"2024-09-20T18:57:45.734107+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:21.629800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:21.629800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"228'12648","reported_seq":23220,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.925551+0000","last_change":"2024-09-20T18:57:09.139489+0000","last_active":"2024-09-20T18:57:45.925551+0000","last_peered":"2024-09-20T18:57:45.925551+0000","last_clean":"2024-09-20T18:57:45.925551+0000","last_became_active":"2024-09-20T18:57:09.121592+0000","last_became_peered":"2024-09-20T18:57:09.121592+0000","last_unstale":"2024-09-20T18:57:45.925551+0000","last_undegraded":"2024-09-20T18:57:45.925551+0000","last_fullsized":"2024-09-20T18:57:45.925551+0000","mapping_epoch":216,"log_start":"198'7606","ondisk_log_start":"198'7606","created":160,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":5042,"log_dups_size":0,"ondisk_log_size":5042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:35.584332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123469824,"num_objects":1884,"num_object_clones":0,"num_object_copies":5652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1884,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12648,"num_write_kb":465024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5206,"num_bytes_recovered":341180416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"184'17","reported_seq":203,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.722458+0000","last_change":"2024-09-20T18:57:22.816240+0000","last_active":"2024-09-20T18:57:45.722458+0000","last_peered":"2024-09-20T18:57:45.722458+0000","last_clean":"2024-09-20T18:57:45.722458+0000","last_became_active":"2024-09-20T18:55:28.831024+0000","last_became_peered":"2024-09-20T18:55:28.831024+0000","last_unstale":"2024-09-20T18:57:45.722458+0000","last_undegraded":"2024-09-20T18:57:45.722458+0000","last_fullsized":"2024-09-20T18:57:45.722458+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'17","last_scrub_stamp":"2024-09-20T18:57:22.816154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:22.816154+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:28.727289+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":24,"num_read_kb":14,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":375,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.731527+0000","last_change":"2024-09-20T18:55:28.830447+0000","last_active":"2024-09-20T18:57:45.731527+0000","last_peered":"2024-09-20T18:57:45.731527+0000","last_clean":"2024-09-20T18:57:45.731527+0000","last_became_active":"2024-09-20T18:55:28.830147+0000","last_became_peered":"2024-09-20T18:55:28.830147+0000","last_unstale":"2024-09-20T18:57:45.731527+0000","last_undegraded":"2024-09-20T18:57:45.731527+0000","last_fullsized":"2024-09-20T18:57:45.731527+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:13.144896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:13.144896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:28.731761+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":429,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.721686+0000","last_change":"2024-09-20T18:55:28.844060+0000","last_active":"2024-09-20T18:57:45.721686+0000","last_peered":"2024-09-20T18:57:45.721686+0000","last_clean":"2024-09-20T18:57:45.721686+0000","last_became_active":"2024-09-20T18:55:28.829325+0000","last_became_peered":"2024-09-20T18:55:28.829325+0000","last_unstale":"2024-09-20T18:57:45.721686+0000","last_undegraded":"2024-09-20T18:57:45.721686+0000","last_fullsized":"2024-09-20T18:57:45.721686+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T18:55:13.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T18:55:13.696607+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":351,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.721955+0000","last_change":"2024-09-20T18:51:22.852942+0000","last_active":"2024-09-20T18:57:45.721955+0000","last_peered":"2024-09-20T18:57:45.721955+0000","last_clean":"2024-09-20T18:57:45.721955+0000","last_became_active":"2024-09-20T18:49:49.956128+0000","last_became_peered":"2024-09-20T18:49:49.956128+0000","last_unstale":"2024-09-20T18:57:45.721955+0000","last_undegraded":"2024-09-20T18:57:45.721955+0000","last_fullsized":"2024-09-20T18:57:45.721955+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:22.852869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:22.852869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":363,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.737882+0000","last_change":"2024-09-20T18:55:25.281435+0000","last_active":"2024-09-20T18:57:45.737882+0000","last_peered":"2024-09-20T18:57:45.737882+0000","last_clean":"2024-09-20T18:57:45.737882+0000","last_became_active":"2024-09-20T18:49:51.971332+0000","last_became_peered":"2024-09-20T18:49:51.971332+0000","last_unstale":"2024-09-20T18:57:45.737882+0000","last_undegraded":"2024-09-20T18:57:45.737882+0000","last_fullsized":"2024-09-20T18:57:45.737882+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:25.281366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:25.281366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"228'12712","reported_seq":12860,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.884264+0000","last_change":"2024-09-20T18:54:33.929366+0000","last_active":"2024-09-20T18:57:45.884264+0000","last_peered":"2024-09-20T18:57:45.884264+0000","last_clean":"2024-09-20T18:57:45.884264+0000","last_became_active":"2024-09-20T18:54:33.929072+0000","last_became_peered":"2024-09-20T18:54:33.929072+0000","last_unstale":"2024-09-20T18:57:45.884264+0000","last_undegraded":"2024-09-20T18:57:45.884264+0000","last_fullsized":"2024-09-20T18:57:45.884264+0000","mapping_epoch":160,"log_start":"204'8102","ondisk_log_start":"204'8102","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4610,"log_dups_size":0,"ondisk_log_size":4610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121241600,"num_objects":1850,"num_object_clones":0,"num_object_copies":5550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12712,"num_write_kb":465984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":394,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.720136+0000","last_change":"2024-09-20T18:55:28.840105+0000","last_active":"2024-09-20T18:57:45.720136+0000","last_peered":"2024-09-20T18:57:45.720136+0000","last_clean":"2024-09-20T18:57:45.720136+0000","last_became_active":"2024-09-20T18:55:28.818269+0000","last_became_peered":"2024-09-20T18:55:28.818269+0000","last_unstale":"2024-09-20T18:57:45.720136+0000","last_undegraded":"2024-09-20T18:57:45.720136+0000","last_fullsized":"2024-09-20T18:57:45.720136+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:51:14.397930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.397930+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:23.803752+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":360,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.719124+0000","last_change":"2024-09-20T18:51:18.410601+0000","last_active":"2024-09-20T18:57:45.719124+0000","last_peered":"2024-09-20T18:57:45.719124+0000","last_clean":"2024-09-20T18:57:45.719124+0000","last_became_active":"2024-09-20T18:49:45.944579+0000","last_became_peered":"2024-09-20T18:49:45.944579+0000","last_unstale":"2024-09-20T18:57:45.719124+0000","last_undegraded":"2024-09-20T18:57:45.719124+0000","last_fullsized":"2024-09-20T18:57:45.719124+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:18.410447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:18.410447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:29.917821+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":374,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.737794+0000","last_change":"2024-09-20T18:55:28.830634+0000","last_active":"2024-09-20T18:57:45.737794+0000","last_peered":"2024-09-20T18:57:45.737794+0000","last_clean":"2024-09-20T18:57:45.737794+0000","last_became_active":"2024-09-20T18:55:28.827785+0000","last_became_peered":"2024-09-20T18:55:28.827785+0000","last_unstale":"2024-09-20T18:57:45.737794+0000","last_undegraded":"2024-09-20T18:57:45.737794+0000","last_fullsized":"2024-09-20T18:57:45.737794+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:11.401777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:11.401777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:37.550724+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":363,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.732428+0000","last_change":"2024-09-20T18:57:23.721754+0000","last_active":"2024-09-20T18:57:45.732428+0000","last_peered":"2024-09-20T18:57:45.732428+0000","last_clean":"2024-09-20T18:57:45.732428+0000","last_became_active":"2024-09-20T18:55:28.833115+0000","last_became_peered":"2024-09-20T18:55:28.833115+0000","last_unstale":"2024-09-20T18:57:45.732428+0000","last_undegraded":"2024-09-20T18:57:45.732428+0000","last_fullsized":"2024-09-20T18:57:45.732428+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:23.721643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:23.721643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:25.737391+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1","version":"228'12568","reported_seq":12716,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:46.323107+0000","last_change":"2024-09-20T18:54:34.310237+0000","last_active":"2024-09-20T18:57:46.323107+0000","last_peered":"2024-09-20T18:57:46.323107+0000","last_clean":"2024-09-20T18:57:46.323107+0000","last_became_active":"2024-09-20T18:54:34.309781+0000","last_became_peered":"2024-09-20T18:54:34.309781+0000","last_unstale":"2024-09-20T18:57:46.323107+0000","last_undegraded":"2024-09-20T18:57:46.323107+0000","last_fullsized":"2024-09-20T18:57:46.323107+0000","mapping_epoch":160,"log_start":"206'8324","ondisk_log_start":"206'8324","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4244,"log_dups_size":0,"ondisk_log_size":4244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":122159104,"num_objects":1864,"num_object_clones":0,"num_object_copies":5592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12568,"num_write_kb":461824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":392,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.722633+0000","last_change":"2024-09-20T18:52:02.944536+0000","last_active":"2024-09-20T18:57:45.722633+0000","last_peered":"2024-09-20T18:57:45.722633+0000","last_clean":"2024-09-20T18:57:45.722633+0000","last_became_active":"2024-09-20T18:49:47.947636+0000","last_became_peered":"2024-09-20T18:49:47.947636+0000","last_unstale":"2024-09-20T18:57:45.722633+0000","last_undegraded":"2024-09-20T18:57:45.722633+0000","last_fullsized":"2024-09-20T18:57:45.722633+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-20T18:52:02.944434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:02.944434+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:34.896863+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":230,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.719342+0000","last_change":"2024-09-20T18:55:28.822330+0000","last_active":"2024-09-20T18:57:45.719342+0000","last_peered":"2024-09-20T18:57:45.719342+0000","last_clean":"2024-09-20T18:57:45.719342+0000","last_became_active":"2024-09-20T18:55:28.821959+0000","last_became_peered":"2024-09-20T18:55:28.821959+0000","last_unstale":"2024-09-20T18:57:45.719342+0000","last_undegraded":"2024-09-20T18:57:45.719342+0000","last_fullsized":"2024-09-20T18:57:45.719342+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:45.669427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:45.669427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":360,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.734205+0000","last_change":"2024-09-20T18:54:11.377907+0000","last_active":"2024-09-20T18:57:45.734205+0000","last_peered":"2024-09-20T18:57:45.734205+0000","last_clean":"2024-09-20T18:57:45.734205+0000","last_became_active":"2024-09-20T18:49:49.959160+0000","last_became_peered":"2024-09-20T18:49:49.959160+0000","last_unstale":"2024-09-20T18:57:45.734205+0000","last_undegraded":"2024-09-20T18:57:45.734205+0000","last_fullsized":"2024-09-20T18:57:45.734205+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:11.376951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:11.376951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:29.061620+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":357,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.733034+0000","last_change":"2024-09-20T18:52:35.635897+0000","last_active":"2024-09-20T18:57:45.733034+0000","last_peered":"2024-09-20T18:57:45.733034+0000","last_clean":"2024-09-20T18:57:45.733034+0000","last_became_active":"2024-09-20T18:49:51.970108+0000","last_became_peered":"2024-09-20T18:49:51.970108+0000","last_unstale":"2024-09-20T18:57:45.733034+0000","last_undegraded":"2024-09-20T18:57:45.733034+0000","last_fullsized":"2024-09-20T18:57:45.733034+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:35.635818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:35.635818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"228'12423","reported_seq":12570,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:46.538513+0000","last_change":"2024-09-20T18:54:33.929552+0000","last_active":"2024-09-20T18:57:46.538513+0000","last_peered":"2024-09-20T18:57:46.538513+0000","last_clean":"2024-09-20T18:57:46.538513+0000","last_became_active":"2024-09-20T18:54:33.929208+0000","last_became_peered":"2024-09-20T18:54:33.929208+0000","last_unstale":"2024-09-20T18:57:46.538513+0000","last_undegraded":"2024-09-20T18:57:46.538513+0000","last_fullsized":"2024-09-20T18:57:46.538513+0000","mapping_epoch":160,"log_start":"206'8121","ondisk_log_start":"206'8121","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4302,"log_dups_size":0,"ondisk_log_size":4302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118161408,"num_objects":1803,"num_object_clones":0,"num_object_copies":5409,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12423,"num_write_kb":455232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":398,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.719664+0000","last_change":"2024-09-20T18:55:28.846773+0000","last_active":"2024-09-20T18:57:45.719664+0000","last_peered":"2024-09-20T18:57:45.719664+0000","last_clean":"2024-09-20T18:57:45.719664+0000","last_became_active":"2024-09-20T18:55:28.817101+0000","last_became_peered":"2024-09-20T18:55:28.817101+0000","last_unstale":"2024-09-20T18:57:45.719664+0000","last_undegraded":"2024-09-20T18:57:45.719664+0000","last_fullsized":"2024-09-20T18:57:45.719664+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:54:17.219548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:17.219548+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":368,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.732237+0000","last_change":"2024-09-20T18:52:46.005172+0000","last_active":"2024-09-20T18:57:45.732237+0000","last_peered":"2024-09-20T18:57:45.732237+0000","last_clean":"2024-09-20T18:57:45.732237+0000","last_became_active":"2024-09-20T18:49:45.937264+0000","last_became_peered":"2024-09-20T18:49:45.937264+0000","last_unstale":"2024-09-20T18:57:45.732237+0000","last_undegraded":"2024-09-20T18:57:45.732237+0000","last_fullsized":"2024-09-20T18:57:45.732237+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:46.005109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:46.005109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:27.755958+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":474,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.730289+0000","last_change":"2024-09-20T18:55:17.549660+0000","last_active":"2024-09-20T18:57:45.730289+0000","last_peered":"2024-09-20T18:57:45.730289+0000","last_clean":"2024-09-20T18:57:45.730289+0000","last_became_active":"2024-09-20T18:49:49.957005+0000","last_became_peered":"2024-09-20T18:49:49.957005+0000","last_unstale":"2024-09-20T18:57:45.730289+0000","last_undegraded":"2024-09-20T18:57:45.730289+0000","last_fullsized":"2024-09-20T18:57:45.730289+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:17.549598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:17.549598+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":356,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.719952+0000","last_change":"2024-09-20T18:54:42.182875+0000","last_active":"2024-09-20T18:57:45.719952+0000","last_peered":"2024-09-20T18:57:45.719952+0000","last_clean":"2024-09-20T18:57:45.719952+0000","last_became_active":"2024-09-20T18:49:51.969355+0000","last_became_peered":"2024-09-20T18:49:51.969355+0000","last_unstale":"2024-09-20T18:57:45.719952+0000","last_undegraded":"2024-09-20T18:57:45.719952+0000","last_fullsized":"2024-09-20T18:57:45.719952+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:42.182809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:42.182809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"228'12765","reported_seq":25181,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:46.315702+0000","last_change":"2024-09-20T18:57:00.083015+0000","last_active":"2024-09-20T18:57:46.315702+0000","last_peered":"2024-09-20T18:57:46.315702+0000","last_clean":"2024-09-20T18:57:46.315702+0000","last_became_active":"2024-09-20T18:57:00.060960+0000","last_became_peered":"2024-09-20T18:57:00.060960+0000","last_unstale":"2024-09-20T18:57:46.315702+0000","last_undegraded":"2024-09-20T18:57:46.315702+0000","last_fullsized":"2024-09-20T18:57:46.315702+0000","mapping_epoch":213,"log_start":"206'8461","ondisk_log_start":"206'8461","created":160,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4304,"log_dups_size":0,"ondisk_log_size":4304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:34.055353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":116981760,"num_objects":1785,"num_object_clones":0,"num_object_copies":5355,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12765,"num_write_kb":465600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6125,"num_bytes_recovered":401408000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":387,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.725399+0000","last_change":"2024-09-20T18:54:54.897991+0000","last_active":"2024-09-20T18:57:45.725399+0000","last_peered":"2024-09-20T18:57:45.725399+0000","last_clean":"2024-09-20T18:57:45.725399+0000","last_became_active":"2024-09-20T18:49:47.946008+0000","last_became_peered":"2024-09-20T18:49:47.946008+0000","last_unstale":"2024-09-20T18:57:45.725399+0000","last_undegraded":"2024-09-20T18:57:45.725399+0000","last_fullsized":"2024-09-20T18:57:45.725399+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:54.897933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:54.897933+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":382,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.737534+0000","last_change":"2024-09-20T18:55:28.833453+0000","last_active":"2024-09-20T18:57:45.737534+0000","last_peered":"2024-09-20T18:57:45.737534+0000","last_clean":"2024-09-20T18:57:45.737534+0000","last_became_active":"2024-09-20T18:55:28.833095+0000","last_became_peered":"2024-09-20T18:55:28.833095+0000","last_unstale":"2024-09-20T18:57:45.737534+0000","last_undegraded":"2024-09-20T18:57:45.737534+0000","last_fullsized":"2024-09-20T18:57:45.737534+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:56.212378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:56.212378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":369,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.731090+0000","last_change":"2024-09-20T18:54:43.379898+0000","last_active":"2024-09-20T18:57:45.731090+0000","last_peered":"2024-09-20T18:57:45.731090+0000","last_clean":"2024-09-20T18:57:45.731090+0000","last_became_active":"2024-09-20T18:49:49.956767+0000","last_became_peered":"2024-09-20T18:49:49.956767+0000","last_unstale":"2024-09-20T18:57:45.731090+0000","last_undegraded":"2024-09-20T18:57:45.731090+0000","last_fullsized":"2024-09-20T18:57:45.731090+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:43.379706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:43.379706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":355,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.728508+0000","last_change":"2024-09-20T18:54:24.976627+0000","last_active":"2024-09-20T18:57:45.728508+0000","last_peered":"2024-09-20T18:57:45.728508+0000","last_clean":"2024-09-20T18:57:45.728508+0000","last_became_active":"2024-09-20T18:49:51.964098+0000","last_became_peered":"2024-09-20T18:49:51.964098+0000","last_unstale":"2024-09-20T18:57:45.728508+0000","last_undegraded":"2024-09-20T18:57:45.728508+0000","last_fullsized":"2024-09-20T18:57:45.728508+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:24.976584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:24.976584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"228'12632","reported_seq":18375,"reported_epoch":228,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-20T18:57:46.889092+0000","last_change":"2024-09-20T18:56:27.871592+0000","last_active":"2024-09-20T18:57:46.889092+0000","last_peered":"2024-09-20T18:57:46.889092+0000","last_clean":"2024-09-20T18:55:27.791097+0000","last_became_active":"2024-09-20T18:55:29.979932+0000","last_became_peered":"2024-09-20T18:55:29.979932+0000","last_unstale":"2024-09-20T18:57:46.889092+0000","last_undegraded":"2024-09-20T18:56:27.867865+0000","last_fullsized":"2024-09-20T18:55:29.815116+0000","mapping_epoch":182,"log_start":"199'7740","ondisk_log_start":"199'7740","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4892,"log_dups_size":0,"ondisk_log_size":4892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":119013376,"num_objects":1816,"num_object_clones":0,"num_object_copies":5448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1816,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12632,"num_write_kb":462336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2796,"num_bytes_recovered":175964160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7],"avail_no_missing":["0","6","7"],"object_location_counts":[{"shards":"0,7","objects":-566}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":185,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.737835+0000","last_change":"2024-09-20T18:57:22.033602+0000","last_active":"2024-09-20T18:57:45.737835+0000","last_peered":"2024-09-20T18:57:45.737835+0000","last_clean":"2024-09-20T18:57:45.737835+0000","last_became_active":"2024-09-20T18:55:28.828563+0000","last_became_peered":"2024-09-20T18:55:28.828563+0000","last_unstale":"2024-09-20T18:57:45.737835+0000","last_undegraded":"2024-09-20T18:57:45.737835+0000","last_fullsized":"2024-09-20T18:57:45.737835+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-20T18:57:22.033475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:22.033475+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:43.998101+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":31,"num_read_kb":20,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,5],"acting":[1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":382,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.722847+0000","last_change":"2024-09-20T18:55:28.835767+0000","last_active":"2024-09-20T18:57:45.722847+0000","last_peered":"2024-09-20T18:57:45.722847+0000","last_clean":"2024-09-20T18:57:45.722847+0000","last_became_active":"2024-09-20T18:55:28.835149+0000","last_became_peered":"2024-09-20T18:55:28.835149+0000","last_unstale":"2024-09-20T18:57:45.722847+0000","last_undegraded":"2024-09-20T18:57:45.722847+0000","last_fullsized":"2024-09-20T18:57:45.722847+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:44.501192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.501192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":369,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.730373+0000","last_change":"2024-09-20T18:55:13.577772+0000","last_active":"2024-09-20T18:57:45.730373+0000","last_peered":"2024-09-20T18:57:45.730373+0000","last_clean":"2024-09-20T18:57:45.730373+0000","last_became_active":"2024-09-20T18:49:49.957558+0000","last_became_peered":"2024-09-20T18:49:49.957558+0000","last_unstale":"2024-09-20T18:57:45.730373+0000","last_undegraded":"2024-09-20T18:57:45.730373+0000","last_fullsized":"2024-09-20T18:57:45.730373+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:13.577699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:13.577699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":363,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.721482+0000","last_change":"2024-09-20T18:53:47.370152+0000","last_active":"2024-09-20T18:57:45.721482+0000","last_peered":"2024-09-20T18:57:45.721482+0000","last_clean":"2024-09-20T18:57:45.721482+0000","last_became_active":"2024-09-20T18:49:51.970798+0000","last_became_peered":"2024-09-20T18:49:51.970798+0000","last_unstale":"2024-09-20T18:57:45.721482+0000","last_undegraded":"2024-09-20T18:57:45.721482+0000","last_fullsized":"2024-09-20T18:57:45.721482+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:47.370108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:47.370108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"228'13078","reported_seq":24020,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.932543+0000","last_change":"2024-09-20T18:57:20.750334+0000","last_active":"2024-09-20T18:57:45.932543+0000","last_peered":"2024-09-20T18:57:45.932543+0000","last_clean":"2024-09-20T18:57:45.932543+0000","last_became_active":"2024-09-20T18:57:20.718369+0000","last_became_peered":"2024-09-20T18:57:20.718369+0000","last_unstale":"2024-09-20T18:57:45.932543+0000","last_undegraded":"2024-09-20T18:57:45.932543+0000","last_fullsized":"2024-09-20T18:57:45.932543+0000","mapping_epoch":221,"log_start":"204'8446","ondisk_log_start":"204'8446","created":160,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4632,"log_dups_size":0,"ondisk_log_size":4632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":125042688,"num_objects":1908,"num_object_clones":0,"num_object_copies":5724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1908,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13078,"num_write_kb":479552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5391,"num_bytes_recovered":353304576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":387,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.718979+0000","last_change":"2024-09-20T18:53:52.898818+0000","last_active":"2024-09-20T18:57:45.718979+0000","last_peered":"2024-09-20T18:57:45.718979+0000","last_clean":"2024-09-20T18:57:45.718979+0000","last_became_active":"2024-09-20T18:49:47.949116+0000","last_became_peered":"2024-09-20T18:49:47.949116+0000","last_unstale":"2024-09-20T18:57:45.718979+0000","last_undegraded":"2024-09-20T18:57:45.718979+0000","last_fullsized":"2024-09-20T18:57:45.718979+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:53:52.898760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:52.898760+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:42.918366+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":375,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.722276+0000","last_change":"2024-09-20T18:54:52.650798+0000","last_active":"2024-09-20T18:57:45.722276+0000","last_peered":"2024-09-20T18:57:45.722276+0000","last_clean":"2024-09-20T18:57:45.722276+0000","last_became_active":"2024-09-20T18:49:45.944858+0000","last_became_peered":"2024-09-20T18:49:45.944858+0000","last_unstale":"2024-09-20T18:57:45.722276+0000","last_undegraded":"2024-09-20T18:57:45.722276+0000","last_fullsized":"2024-09-20T18:57:45.722276+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:52.650756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:52.650756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":361,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.737118+0000","last_change":"2024-09-20T18:54:42.349879+0000","last_active":"2024-09-20T18:57:45.737118+0000","last_peered":"2024-09-20T18:57:45.737118+0000","last_clean":"2024-09-20T18:57:45.737118+0000","last_became_active":"2024-09-20T18:49:49.954944+0000","last_became_peered":"2024-09-20T18:49:49.954944+0000","last_unstale":"2024-09-20T18:57:45.737118+0000","last_undegraded":"2024-09-20T18:57:45.737118+0000","last_fullsized":"2024-09-20T18:57:45.737118+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:42.349807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:42.349807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":356,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.731911+0000","last_change":"2024-09-20T18:55:17.341862+0000","last_active":"2024-09-20T18:57:45.731911+0000","last_peered":"2024-09-20T18:57:45.731911+0000","last_clean":"2024-09-20T18:57:45.731911+0000","last_became_active":"2024-09-20T18:49:51.972970+0000","last_became_peered":"2024-09-20T18:49:51.972970+0000","last_unstale":"2024-09-20T18:57:45.731911+0000","last_undegraded":"2024-09-20T18:57:45.731911+0000","last_fullsized":"2024-09-20T18:57:45.731911+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:17.341793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:17.341793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.4","version":"228'12465","reported_seq":12613,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.923809+0000","last_change":"2024-09-20T18:54:34.309442+0000","last_active":"2024-09-20T18:57:45.923809+0000","last_peered":"2024-09-20T18:57:45.923809+0000","last_clean":"2024-09-20T18:57:45.923809+0000","last_became_active":"2024-09-20T18:54:34.309113+0000","last_became_peered":"2024-09-20T18:54:34.309113+0000","last_unstale":"2024-09-20T18:57:45.923809+0000","last_undegraded":"2024-09-20T18:57:45.923809+0000","last_fullsized":"2024-09-20T18:57:45.923809+0000","mapping_epoch":160,"log_start":"197'7431","ondisk_log_start":"197'7431","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":5034,"log_dups_size":0,"ondisk_log_size":5034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:40.627488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":122355712,"num_objects":1867,"num_object_clones":0,"num_object_copies":5601,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12465,"num_write_kb":458624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":411,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.736947+0000","last_change":"2024-09-20T18:55:28.847010+0000","last_active":"2024-09-20T18:57:45.736947+0000","last_peered":"2024-09-20T18:57:45.736947+0000","last_clean":"2024-09-20T18:57:45.736947+0000","last_became_active":"2024-09-20T18:55:28.824801+0000","last_became_peered":"2024-09-20T18:55:28.824801+0000","last_unstale":"2024-09-20T18:57:45.736947+0000","last_undegraded":"2024-09-20T18:57:45.736947+0000","last_fullsized":"2024-09-20T18:57:45.736947+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:54:12.373309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:12.373309+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:29.516079+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":395,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.722050+0000","last_change":"2024-09-20T18:55:24.729301+0000","last_active":"2024-09-20T18:57:45.722050+0000","last_peered":"2024-09-20T18:57:45.722050+0000","last_clean":"2024-09-20T18:57:45.722050+0000","last_became_active":"2024-09-20T18:49:45.944256+0000","last_became_peered":"2024-09-20T18:49:45.944256+0000","last_unstale":"2024-09-20T18:57:45.722050+0000","last_undegraded":"2024-09-20T18:57:45.722050+0000","last_fullsized":"2024-09-20T18:57:45.722050+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-20T18:55:24.729219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:24.729219+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"184'9","reported_seq":466,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.721575+0000","last_change":"2024-09-20T18:55:28.824771+0000","last_active":"2024-09-20T18:57:45.721575+0000","last_peered":"2024-09-20T18:57:45.721575+0000","last_clean":"2024-09-20T18:57:45.721575+0000","last_became_active":"2024-09-20T18:55:28.818085+0000","last_became_peered":"2024-09-20T18:55:28.818085+0000","last_unstale":"2024-09-20T18:57:45.721575+0000","last_undegraded":"2024-09-20T18:57:45.721575+0000","last_fullsized":"2024-09-20T18:57:45.721575+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:52:05.788924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.788924+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":362,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.737064+0000","last_change":"2024-09-20T18:55:28.832276+0000","last_active":"2024-09-20T18:57:45.737064+0000","last_peered":"2024-09-20T18:57:45.737064+0000","last_clean":"2024-09-20T18:57:45.737064+0000","last_became_active":"2024-09-20T18:55:28.830380+0000","last_became_peered":"2024-09-20T18:55:28.830380+0000","last_unstale":"2024-09-20T18:57:45.737064+0000","last_undegraded":"2024-09-20T18:57:45.737064+0000","last_fullsized":"2024-09-20T18:57:45.737064+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:14.247398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:14.247398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":105,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.727909+0000","last_change":"2024-09-20T18:55:28.835344+0000","last_active":"2024-09-20T18:57:45.727909+0000","last_peered":"2024-09-20T18:57:45.727909+0000","last_clean":"2024-09-20T18:57:45.727909+0000","last_became_active":"2024-09-20T18:55:28.835010+0000","last_became_peered":"2024-09-20T18:55:28.835010+0000","last_unstale":"2024-09-20T18:57:45.727909+0000","last_undegraded":"2024-09-20T18:57:45.727909+0000","last_fullsized":"2024-09-20T18:57:45.727909+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:26.721735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:26.721735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":373,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.735190+0000","last_change":"2024-09-20T18:54:44.194161+0000","last_active":"2024-09-20T18:57:45.735190+0000","last_peered":"2024-09-20T18:57:45.735190+0000","last_clean":"2024-09-20T18:57:45.735190+0000","last_became_active":"2024-09-20T18:49:48.500662+0000","last_became_peered":"2024-09-20T18:49:48.500662+0000","last_unstale":"2024-09-20T18:57:45.735190+0000","last_undegraded":"2024-09-20T18:57:45.735190+0000","last_fullsized":"2024-09-20T18:57:45.735190+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-20T18:54:44.194089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.194089+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":359,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.737504+0000","last_change":"2024-09-20T18:53:16.220554+0000","last_active":"2024-09-20T18:57:45.737504+0000","last_peered":"2024-09-20T18:57:45.737504+0000","last_clean":"2024-09-20T18:57:45.737504+0000","last_became_active":"2024-09-20T18:49:49.956749+0000","last_became_peered":"2024-09-20T18:49:49.956749+0000","last_unstale":"2024-09-20T18:57:45.737504+0000","last_undegraded":"2024-09-20T18:57:45.737504+0000","last_fullsized":"2024-09-20T18:57:45.737504+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:16.220490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:16.220490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:27.545117+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":370,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.722663+0000","last_change":"2024-09-20T18:55:28.827897+0000","last_active":"2024-09-20T18:57:45.722663+0000","last_peered":"2024-09-20T18:57:45.722663+0000","last_clean":"2024-09-20T18:57:45.722663+0000","last_became_active":"2024-09-20T18:55:28.827418+0000","last_became_peered":"2024-09-20T18:55:28.827418+0000","last_unstale":"2024-09-20T18:57:45.722663+0000","last_undegraded":"2024-09-20T18:57:45.722663+0000","last_fullsized":"2024-09-20T18:57:45.722663+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.637739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.637739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":379,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.727756+0000","last_change":"2024-09-20T18:54:44.830872+0000","last_active":"2024-09-20T18:57:45.727756+0000","last_peered":"2024-09-20T18:57:45.727756+0000","last_clean":"2024-09-20T18:57:45.727756+0000","last_became_active":"2024-09-20T18:49:48.500803+0000","last_became_peered":"2024-09-20T18:49:48.500803+0000","last_unstale":"2024-09-20T18:57:45.727756+0000","last_undegraded":"2024-09-20T18:57:45.727756+0000","last_fullsized":"2024-09-20T18:57:45.727756+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:44.830790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.830790+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":374,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.721776+0000","last_change":"2024-09-20T18:55:29.751718+0000","last_active":"2024-09-20T18:57:45.721776+0000","last_peered":"2024-09-20T18:57:45.721776+0000","last_clean":"2024-09-20T18:57:45.721776+0000","last_became_active":"2024-09-20T18:55:28.833799+0000","last_became_peered":"2024-09-20T18:55:28.833799+0000","last_unstale":"2024-09-20T18:57:45.721776+0000","last_undegraded":"2024-09-20T18:57:45.721776+0000","last_fullsized":"2024-09-20T18:57:45.721776+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:29.751642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:29.751642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":360,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.735051+0000","last_change":"2024-09-20T18:53:50.253721+0000","last_active":"2024-09-20T18:57:45.735051+0000","last_peered":"2024-09-20T18:57:45.735051+0000","last_clean":"2024-09-20T18:57:45.735051+0000","last_became_active":"2024-09-20T18:49:49.961947+0000","last_became_peered":"2024-09-20T18:49:49.961947+0000","last_unstale":"2024-09-20T18:57:45.735051+0000","last_undegraded":"2024-09-20T18:57:45.735051+0000","last_fullsized":"2024-09-20T18:57:45.735051+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:50.253660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:50.253660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"66'1","reported_seq":366,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.730489+0000","last_change":"2024-09-20T18:54:20.681878+0000","last_active":"2024-09-20T18:57:45.730489+0000","last_peered":"2024-09-20T18:57:45.730489+0000","last_clean":"2024-09-20T18:57:45.730489+0000","last_became_active":"2024-09-20T18:49:51.965121+0000","last_became_peered":"2024-09-20T18:49:51.965121+0000","last_unstale":"2024-09-20T18:57:45.730489+0000","last_undegraded":"2024-09-20T18:57:45.730489+0000","last_fullsized":"2024-09-20T18:57:45.730489+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T18:54:20.681790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:20.681790+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":371,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.737542+0000","last_change":"2024-09-20T18:55:12.205118+0000","last_active":"2024-09-20T18:57:45.737542+0000","last_peered":"2024-09-20T18:57:45.737542+0000","last_clean":"2024-09-20T18:57:45.737542+0000","last_became_active":"2024-09-20T18:49:48.501011+0000","last_became_peered":"2024-09-20T18:49:48.501011+0000","last_unstale":"2024-09-20T18:57:45.737542+0000","last_undegraded":"2024-09-20T18:57:45.737542+0000","last_fullsized":"2024-09-20T18:57:45.737542+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:55:12.205069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:12.205069+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":375,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.724268+0000","last_change":"2024-09-20T18:55:27.028184+0000","last_active":"2024-09-20T18:57:45.724268+0000","last_peered":"2024-09-20T18:57:45.724268+0000","last_clean":"2024-09-20T18:57:45.724268+0000","last_became_active":"2024-09-20T18:49:45.938511+0000","last_became_peered":"2024-09-20T18:49:45.938511+0000","last_unstale":"2024-09-20T18:57:45.724268+0000","last_undegraded":"2024-09-20T18:57:45.724268+0000","last_fullsized":"2024-09-20T18:57:45.724268+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:27.028126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:27.028126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":473,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.732743+0000","last_change":"2024-09-20T18:55:27.284156+0000","last_active":"2024-09-20T18:57:45.732743+0000","last_peered":"2024-09-20T18:57:45.732743+0000","last_clean":"2024-09-20T18:57:45.732743+0000","last_became_active":"2024-09-20T18:49:49.960550+0000","last_became_peered":"2024-09-20T18:49:49.960550+0000","last_unstale":"2024-09-20T18:57:45.732743+0000","last_undegraded":"2024-09-20T18:57:45.732743+0000","last_fullsized":"2024-09-20T18:57:45.732743+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:27.284086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:27.284086+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":364,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.719171+0000","last_change":"2024-09-20T18:55:12.348079+0000","last_active":"2024-09-20T18:57:45.719171+0000","last_peered":"2024-09-20T18:57:45.719171+0000","last_clean":"2024-09-20T18:57:45.719171+0000","last_became_active":"2024-09-20T18:49:51.966254+0000","last_became_peered":"2024-09-20T18:49:51.966254+0000","last_unstale":"2024-09-20T18:57:45.719171+0000","last_undegraded":"2024-09-20T18:57:45.719171+0000","last_fullsized":"2024-09-20T18:57:45.719171+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:12.347993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:12.347993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":398,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.732910+0000","last_change":"2024-09-20T18:57:20.701592+0000","last_active":"2024-09-20T18:57:45.732910+0000","last_peered":"2024-09-20T18:57:45.732910+0000","last_clean":"2024-09-20T18:57:45.732910+0000","last_became_active":"2024-09-20T18:49:58.997494+0000","last_became_peered":"2024-09-20T18:49:58.997494+0000","last_unstale":"2024-09-20T18:57:45.732910+0000","last_undegraded":"2024-09-20T18:57:45.732910+0000","last_fullsized":"2024-09-20T18:57:45.732910+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:57:20.701543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:20.701543+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:44.352173+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":375,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.721876+0000","last_change":"2024-09-20T18:54:57.632458+0000","last_active":"2024-09-20T18:57:45.721876+0000","last_peered":"2024-09-20T18:57:45.721876+0000","last_clean":"2024-09-20T18:57:45.721876+0000","last_became_active":"2024-09-20T18:49:45.949492+0000","last_became_peered":"2024-09-20T18:49:45.949492+0000","last_unstale":"2024-09-20T18:57:45.721876+0000","last_undegraded":"2024-09-20T18:57:45.721876+0000","last_fullsized":"2024-09-20T18:57:45.721876+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.632380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.632380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":471,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.737449+0000","last_change":"2024-09-20T18:55:28.302353+0000","last_active":"2024-09-20T18:57:45.737449+0000","last_peered":"2024-09-20T18:57:45.737449+0000","last_clean":"2024-09-20T18:57:45.737449+0000","last_became_active":"2024-09-20T18:49:49.962243+0000","last_became_peered":"2024-09-20T18:49:49.962243+0000","last_unstale":"2024-09-20T18:57:45.737449+0000","last_undegraded":"2024-09-20T18:57:45.737449+0000","last_fullsized":"2024-09-20T18:57:45.737449+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:28.302303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:28.302303+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":363,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.724631+0000","last_change":"2024-09-20T18:55:11.918284+0000","last_active":"2024-09-20T18:57:45.724631+0000","last_peered":"2024-09-20T18:57:45.724631+0000","last_clean":"2024-09-20T18:57:45.724631+0000","last_became_active":"2024-09-20T18:49:51.969093+0000","last_became_peered":"2024-09-20T18:49:51.969093+0000","last_unstale":"2024-09-20T18:57:45.724631+0000","last_undegraded":"2024-09-20T18:57:45.724631+0000","last_fullsized":"2024-09-20T18:57:45.724631+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:11.918210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:11.918210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":402,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.720011+0000","last_change":"2024-09-20T18:55:28.854249+0000","last_active":"2024-09-20T18:57:45.720011+0000","last_peered":"2024-09-20T18:57:45.720011+0000","last_clean":"2024-09-20T18:57:45.720011+0000","last_became_active":"2024-09-20T18:55:28.826721+0000","last_became_peered":"2024-09-20T18:55:28.826721+0000","last_unstale":"2024-09-20T18:57:45.720011+0000","last_undegraded":"2024-09-20T18:57:45.720011+0000","last_fullsized":"2024-09-20T18:57:45.720011+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:53:43.878921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:43.878921+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":367,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.719862+0000","last_change":"2024-09-20T18:55:28.822690+0000","last_active":"2024-09-20T18:57:45.719862+0000","last_peered":"2024-09-20T18:57:45.719862+0000","last_clean":"2024-09-20T18:57:45.719862+0000","last_became_active":"2024-09-20T18:55:28.822258+0000","last_became_peered":"2024-09-20T18:55:28.822258+0000","last_unstale":"2024-09-20T18:57:45.719862+0000","last_undegraded":"2024-09-20T18:57:45.719862+0000","last_fullsized":"2024-09-20T18:57:45.719862+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:05.985551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.985551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:37.926823+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":360,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.737757+0000","last_change":"2024-09-20T18:57:11.986605+0000","last_active":"2024-09-20T18:57:45.737757+0000","last_peered":"2024-09-20T18:57:45.737757+0000","last_clean":"2024-09-20T18:57:45.737757+0000","last_became_active":"2024-09-20T18:49:49.957067+0000","last_became_peered":"2024-09-20T18:49:49.957067+0000","last_unstale":"2024-09-20T18:57:45.737757+0000","last_undegraded":"2024-09-20T18:57:45.737757+0000","last_fullsized":"2024-09-20T18:57:45.737757+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:11.986528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:11.986528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:36.007001+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":356,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.731375+0000","last_change":"2024-09-20T18:57:16.789412+0000","last_active":"2024-09-20T18:57:45.731375+0000","last_peered":"2024-09-20T18:57:45.731375+0000","last_clean":"2024-09-20T18:57:45.731375+0000","last_became_active":"2024-09-20T18:49:51.966915+0000","last_became_peered":"2024-09-20T18:49:51.966915+0000","last_unstale":"2024-09-20T18:57:45.731375+0000","last_undegraded":"2024-09-20T18:57:45.731375+0000","last_fullsized":"2024-09-20T18:57:45.731375+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:16.789095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:16.789095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:45.821909+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":405,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.738008+0000","last_change":"2024-09-20T18:55:28.838698+0000","last_active":"2024-09-20T18:57:45.738008+0000","last_peered":"2024-09-20T18:57:45.738008+0000","last_clean":"2024-09-20T18:57:45.738008+0000","last_became_active":"2024-09-20T18:55:28.827023+0000","last_became_peered":"2024-09-20T18:55:28.827023+0000","last_unstale":"2024-09-20T18:57:45.738008+0000","last_undegraded":"2024-09-20T18:57:45.738008+0000","last_fullsized":"2024-09-20T18:57:45.738008+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:53:53.263976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:53.263976+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:26.131891+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":367,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.722787+0000","last_change":"2024-09-20T18:54:42.487465+0000","last_active":"2024-09-20T18:57:45.722787+0000","last_peered":"2024-09-20T18:57:45.722787+0000","last_clean":"2024-09-20T18:57:45.722787+0000","last_became_active":"2024-09-20T18:49:45.944357+0000","last_became_peered":"2024-09-20T18:49:45.944357+0000","last_unstale":"2024-09-20T18:57:45.722787+0000","last_undegraded":"2024-09-20T18:57:45.722787+0000","last_fullsized":"2024-09-20T18:57:45.722787+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:42.487403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:42.487403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":360,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.731987+0000","last_change":"2024-09-20T18:54:18.108239+0000","last_active":"2024-09-20T18:57:45.731987+0000","last_peered":"2024-09-20T18:57:45.731987+0000","last_clean":"2024-09-20T18:57:45.731987+0000","last_became_active":"2024-09-20T18:49:49.962523+0000","last_became_peered":"2024-09-20T18:49:49.962523+0000","last_unstale":"2024-09-20T18:57:45.731987+0000","last_undegraded":"2024-09-20T18:57:45.731987+0000","last_fullsized":"2024-09-20T18:57:45.731987+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:18.108175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:18.108175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":363,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.723561+0000","last_change":"2024-09-20T18:54:26.882441+0000","last_active":"2024-09-20T18:57:45.723561+0000","last_peered":"2024-09-20T18:57:45.723561+0000","last_clean":"2024-09-20T18:57:45.723561+0000","last_became_active":"2024-09-20T18:49:51.967313+0000","last_became_peered":"2024-09-20T18:49:51.967313+0000","last_unstale":"2024-09-20T18:57:45.723561+0000","last_undegraded":"2024-09-20T18:57:45.723561+0000","last_fullsized":"2024-09-20T18:57:45.723561+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:26.882375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:26.882375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":402,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.731521+0000","last_change":"2024-09-20T18:55:28.843363+0000","last_active":"2024-09-20T18:57:45.731521+0000","last_peered":"2024-09-20T18:57:45.731521+0000","last_clean":"2024-09-20T18:57:45.731521+0000","last_became_active":"2024-09-20T18:55:28.830801+0000","last_became_peered":"2024-09-20T18:55:28.830801+0000","last_unstale":"2024-09-20T18:57:45.731521+0000","last_undegraded":"2024-09-20T18:57:45.731521+0000","last_fullsized":"2024-09-20T18:57:45.731521+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:54:21.106677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:21.106677+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":375,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.737598+0000","last_change":"2024-09-20T18:55:19.258723+0000","last_active":"2024-09-20T18:57:45.737598+0000","last_peered":"2024-09-20T18:57:45.737598+0000","last_clean":"2024-09-20T18:57:45.737598+0000","last_became_active":"2024-09-20T18:49:45.942105+0000","last_became_peered":"2024-09-20T18:49:45.942105+0000","last_unstale":"2024-09-20T18:57:45.737598+0000","last_undegraded":"2024-09-20T18:57:45.737598+0000","last_fullsized":"2024-09-20T18:57:45.737598+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:19.258674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:19.258674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":374,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.721289+0000","last_change":"2024-09-20T18:55:28.825004+0000","last_active":"2024-09-20T18:57:45.721289+0000","last_peered":"2024-09-20T18:57:45.721289+0000","last_clean":"2024-09-20T18:57:45.721289+0000","last_became_active":"2024-09-20T18:55:28.824355+0000","last_became_peered":"2024-09-20T18:55:28.824355+0000","last_unstale":"2024-09-20T18:57:45.721289+0000","last_undegraded":"2024-09-20T18:57:45.721289+0000","last_fullsized":"2024-09-20T18:57:45.721289+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:46.329547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:46.329547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":355,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.732117+0000","last_change":"2024-09-20T18:55:28.825607+0000","last_active":"2024-09-20T18:57:45.732117+0000","last_peered":"2024-09-20T18:57:45.732117+0000","last_clean":"2024-09-20T18:57:45.732117+0000","last_became_active":"2024-09-20T18:55:28.825296+0000","last_became_peered":"2024-09-20T18:55:28.825296+0000","last_unstale":"2024-09-20T18:57:45.732117+0000","last_undegraded":"2024-09-20T18:57:45.732117+0000","last_fullsized":"2024-09-20T18:57:45.732117+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:43.118805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:43.118805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:25.734066+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":391,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.733570+0000","last_change":"2024-09-20T18:55:28.836789+0000","last_active":"2024-09-20T18:57:45.733570+0000","last_peered":"2024-09-20T18:57:45.733570+0000","last_clean":"2024-09-20T18:57:45.733570+0000","last_became_active":"2024-09-20T18:55:28.824579+0000","last_became_peered":"2024-09-20T18:55:28.824579+0000","last_unstale":"2024-09-20T18:57:45.733570+0000","last_undegraded":"2024-09-20T18:57:45.733570+0000","last_fullsized":"2024-09-20T18:57:45.733570+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-20T18:55:20.282193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:20.282193+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":375,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.727897+0000","last_change":"2024-09-20T18:54:55.900487+0000","last_active":"2024-09-20T18:57:45.727897+0000","last_peered":"2024-09-20T18:57:45.727897+0000","last_clean":"2024-09-20T18:57:45.727897+0000","last_became_active":"2024-09-20T18:49:45.940988+0000","last_became_peered":"2024-09-20T18:49:45.940988+0000","last_unstale":"2024-09-20T18:57:45.727897+0000","last_undegraded":"2024-09-20T18:57:45.727897+0000","last_fullsized":"2024-09-20T18:57:45.727897+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:55.899611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:55.899611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":105,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.718289+0000","last_change":"2024-09-20T18:55:28.832864+0000","last_active":"2024-09-20T18:57:45.718289+0000","last_peered":"2024-09-20T18:57:45.718289+0000","last_clean":"2024-09-20T18:57:45.718289+0000","last_became_active":"2024-09-20T18:55:28.832534+0000","last_became_peered":"2024-09-20T18:55:28.832534+0000","last_unstale":"2024-09-20T18:57:45.718289+0000","last_undegraded":"2024-09-20T18:57:45.718289+0000","last_fullsized":"2024-09-20T18:57:45.718289+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:52.635138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:52.635138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":347,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.721462+0000","last_change":"2024-09-20T18:51:09.761331+0000","last_active":"2024-09-20T18:57:45.721462+0000","last_peered":"2024-09-20T18:57:45.721462+0000","last_clean":"2024-09-20T18:57:45.721462+0000","last_became_active":"2024-09-20T18:49:51.968662+0000","last_became_peered":"2024-09-20T18:49:51.968662+0000","last_unstale":"2024-09-20T18:57:45.721462+0000","last_undegraded":"2024-09-20T18:57:45.721462+0000","last_fullsized":"2024-09-20T18:57:45.721462+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:09.761283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:09.761283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":348,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.733837+0000","last_change":"2024-09-20T18:57:10.970843+0000","last_active":"2024-09-20T18:57:45.733837+0000","last_peered":"2024-09-20T18:57:45.733837+0000","last_clean":"2024-09-20T18:57:45.733837+0000","last_became_active":"2024-09-20T18:49:51.968216+0000","last_became_peered":"2024-09-20T18:49:51.968216+0000","last_unstale":"2024-09-20T18:57:45.733837+0000","last_undegraded":"2024-09-20T18:57:45.733837+0000","last_fullsized":"2024-09-20T18:57:45.733837+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:10.970779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:10.970779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:38.733512+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":368,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.733789+0000","last_change":"2024-09-20T18:55:28.295445+0000","last_active":"2024-09-20T18:57:45.733789+0000","last_peered":"2024-09-20T18:57:45.733789+0000","last_clean":"2024-09-20T18:57:45.733789+0000","last_became_active":"2024-09-20T18:49:45.937992+0000","last_became_peered":"2024-09-20T18:49:45.937992+0000","last_unstale":"2024-09-20T18:57:45.733789+0000","last_undegraded":"2024-09-20T18:57:45.733789+0000","last_fullsized":"2024-09-20T18:57:45.733789+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:28.295399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:28.295399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":157,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.737902+0000","last_change":"2024-09-20T18:55:28.852468+0000","last_active":"2024-09-20T18:57:45.737902+0000","last_peered":"2024-09-20T18:57:45.737902+0000","last_clean":"2024-09-20T18:57:45.737902+0000","last_became_active":"2024-09-20T18:55:28.834675+0000","last_became_peered":"2024-09-20T18:55:28.834675+0000","last_unstale":"2024-09-20T18:57:45.737902+0000","last_undegraded":"2024-09-20T18:57:45.737902+0000","last_fullsized":"2024-09-20T18:57:45.737902+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:54:44.719676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.719676+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:38.535671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":12,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":351,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-20T18:57:45.727504+0000","last_change":"2024-09-20T18:51:28.120100+0000","last_active":"2024-09-20T18:57:45.727504+0000","last_peered":"2024-09-20T18:57:45.727504+0000","last_clean":"2024-09-20T18:57:45.727504+0000","last_became_active":"2024-09-20T18:49:49.957582+0000","last_became_peered":"2024-09-20T18:49:49.957582+0000","last_unstale":"2024-09-20T18:57:45.727504+0000","last_undegraded":"2024-09-20T18:57:45.727504+0000","last_fullsized":"2024-09-20T18:57:45.727504+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:28.119694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:28.119694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":1932197912,"num_objects":29484,"num_object_clones":0,"num_object_copies":88452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29484,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":202694,"num_write_kb":7429633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25674,"num_bytes_recovered":1675296768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":7202287616,"data_stored":7202275400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74227,"ondisk_log_size":74227,"up":48,"acting":47,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":67,"ondisk_log_size":67,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":695,"num_read_kb":452,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":1474,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":12140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":394,"ondisk_log_size":394,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":5275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":475,"num_read_kb":470,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266788,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":819352,"kb_used_data":611780,"kb_used_omap":1,"kb_used_meta":207486,"kb_avail":92925800,"statfs":{"total":95995035648,"available":95156019200,"internally_reserved":0,"allocated":626462720,"data_stored":626018043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":212466122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":0,"seq":193273528427,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757360,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1235408,"kb_used_data":969596,"kb_used_omap":1,"kb_used_meta":265790,"kb_avail":92509744,"statfs":{"total":95995035648,"available":94729977856,"internally_reserved":0,"allocated":992866304,"data_stored":992429302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":272169413},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986293,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":823984,"kb_used_data":603996,"kb_used_omap":1,"kb_used_meta":219966,"kb_avail":92921168,"statfs":{"total":95995035648,"available":95151276032,"internally_reserved":0,"allocated":618491904,"data_stored":618029995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":225245637},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182524,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1203892,"kb_used_data":843676,"kb_used_omap":1,"kb_used_meta":360190,"kb_avail":92541260,"statfs":{"total":95995035648,"available":94762250240,"internally_reserved":0,"allocated":863924224,"data_stored":863461269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":368835015},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411457,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1431064,"kb_used_data":1061120,"kb_used_omap":1,"kb_used_meta":369918,"kb_avail":92314088,"statfs":{"total":95995035648,"available":94529626112,"internally_reserved":0,"allocated":1086586880,"data_stored":1086152884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":378796485},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574983,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1123716,"kb_used_data":835700,"kb_used_omap":1,"kb_used_meta":287998,"kb_avail":92621436,"statfs":{"total":95995035648,"available":94844350464,"internally_reserved":0,"allocated":855756800,"data_stored":855271696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":294910405},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738509,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1168884,"kb_used_data":822428,"kb_used_omap":1,"kb_used_meta":346302,"kb_avail":92576268,"statfs":{"total":95995035648,"available":94798098432,"internally_reserved":0,"allocated":842166272,"data_stored":841703937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":354613701},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,3,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":827457536,"data_stored":827457536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":843644928,"data_stored":843644928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1082855424,"data_stored":1082851352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":841809920,"data_stored":841809920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":616693760,"data_stored":616693760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":979308544,"data_stored":979304472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1392050176,"data_stored":1392050176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":618467328,"data_stored":618463256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T18:57:49.400 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph status --format=json 2024-09-20T18:57:49.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:49.401 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:49.402 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:49.503 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:49.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:49 smithi142 ceph-mon[32508]: 8.8 scrub starts 2024-09-20T18:57:49.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:49 smithi142 ceph-mon[32508]: 8.8 scrub ok 2024-09-20T18:57:49.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:49 smithi142 ceph-mon[32508]: from='client.25213 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:57:49.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:49 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2037193382' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:57:49.568 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:49.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:49.569 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:49.670 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:49.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:49.715 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:49.715 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:49.816 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:49.870 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:49.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:49.871 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:49.972 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:57:50.000 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:50.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:50.001 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:57:50.102 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:50.126 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:50.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:50.127 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:50.204 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T18:57:50.204 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:57:50.228 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:50.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:50.270 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:50.270 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:50.298 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:50 smithi083 ceph-mon[29421]: 5.1f scrub starts 2024-09-20T18:57:50.299 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:50 smithi083 ceph-mon[29421]: 5.1f scrub ok 2024-09-20T18:57:50.299 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:50 smithi083 ceph-mon[29421]: pgmap v384: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 1.8 GiB data, 7.4 GiB used, 618 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 1.31k op/s; 2382/89049 objects degraded (2.675%); 1.3 MiB/s, 34 objects/s recovering 2024-09-20T18:57:50.299 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:50 smithi083 ceph-mon[29421]: 5.19 scrub starts 2024-09-20T18:57:50.299 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:50 smithi083 ceph-mon[29421]: 5.19 scrub ok 2024-09-20T18:57:50.299 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:50 smithi083 ceph-mon[36260]: 5.1f scrub starts 2024-09-20T18:57:50.299 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:50 smithi083 ceph-mon[36260]: 5.1f scrub ok 2024-09-20T18:57:50.299 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:50 smithi083 ceph-mon[36260]: pgmap v384: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 1.8 GiB data, 7.4 GiB used, 618 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 1.31k op/s; 2382/89049 objects degraded (2.675%); 1.3 MiB/s, 34 objects/s recovering 2024-09-20T18:57:50.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:50 smithi083 ceph-mon[36260]: 5.19 scrub starts 2024-09-20T18:57:50.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:50 smithi083 ceph-mon[36260]: 5.19 scrub ok 2024-09-20T18:57:50.371 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:50.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:50.407 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:50.407 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:50.508 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:50.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:50 smithi142 ceph-mon[32508]: 5.1f scrub starts 2024-09-20T18:57:50.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:50 smithi142 ceph-mon[32508]: 5.1f scrub ok 2024-09-20T18:57:50.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:50 smithi142 ceph-mon[32508]: pgmap v384: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 1.8 GiB data, 7.4 GiB used, 618 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 1.31k op/s; 2382/89049 objects degraded (2.675%); 1.3 MiB/s, 34 objects/s recovering 2024-09-20T18:57:50.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:50 smithi142 ceph-mon[32508]: 5.19 scrub starts 2024-09-20T18:57:50.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:50 smithi142 ceph-mon[32508]: 5.19 scrub ok 2024-09-20T18:57:50.553 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:50.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:50.555 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:50.655 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:50.680 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:50.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:50.682 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:50.783 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:50.819 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:50.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:50.820 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:50.921 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:50.947 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:50.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:50.948 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:51.049 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:51.050 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:57:50 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:57:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:57:51.103 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:51.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:51.104 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:51.205 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:51.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:51.248 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:51.248 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:51.349 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:57:51.374 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:51.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:51.375 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:57:51.447 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:51 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:57:51.447 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:51 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:57:51.447 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:51 smithi083 ceph-mon[29421]: 8.1 scrub starts 2024-09-20T18:57:51.447 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:51 smithi083 ceph-mon[29421]: osdmap e229: 8 total, 8 up, 7 in 2024-09-20T18:57:51.447 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:51 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 2382/89049 objects degraded (2.675%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T18:57:51.447 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:51 smithi083 ceph-mon[29421]: osdmap e230: 8 total, 8 up, 7 in 2024-09-20T18:57:51.448 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:51 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:57:51.448 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:51 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:57:51.448 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:51 smithi083 ceph-mon[36260]: 8.1 scrub starts 2024-09-20T18:57:51.448 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:51 smithi083 ceph-mon[36260]: osdmap e229: 8 total, 8 up, 7 in 2024-09-20T18:57:51.448 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:51 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 2382/89049 objects degraded (2.675%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T18:57:51.448 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:51 smithi083 ceph-mon[36260]: osdmap e230: 8 total, 8 up, 7 in 2024-09-20T18:57:51.475 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:51.497 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_blocked_ops 2024-09-20T18:57:51.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:51 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:57:51.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:51 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:57:51.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:51 smithi142 ceph-mon[32508]: 8.1 scrub starts 2024-09-20T18:57:51.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:51 smithi142 ceph-mon[32508]: osdmap e229: 8 total, 8 up, 7 in 2024-09-20T18:57:51.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:51 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 2382/89049 objects degraded (2.675%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T18:57:51.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:51 smithi142 ceph-mon[32508]: osdmap e230: 8 total, 8 up, 7 in 2024-09-20T18:57:51.520 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:51.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:51.521 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:51.579 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:57:51.583 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:57:51.621 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:57:51.645 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:51.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:51.646 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:57:51.747 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:51.772 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:51.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:51.773 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:51.874 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:57:51.897 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:51.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:51.898 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:57:51.998 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:52.022 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:52.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:52.024 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:52.125 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:52.165 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:52.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:52.166 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:52.267 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:52.268 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:52 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:57:52.268 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:52 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:57:52.268 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:52 smithi142 ceph-mon[32508]: 8.1 scrub ok 2024-09-20T18:57:52.268 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:52 smithi142 ceph-mon[32508]: 4.13 scrub starts 2024-09-20T18:57:52.269 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:52 smithi142 ceph-mon[32508]: 4.13 scrub ok 2024-09-20T18:57:52.269 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:52 smithi142 ceph-mon[32508]: pgmap v387: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 1.8 GiB data, 7.4 GiB used, 618 GiB / 626 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 866 op/s; 2382/89049 objects degraded (2.675%); 368 KiB/s, 24 objects/s recovering 2024-09-20T18:57:52.269 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:52 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:57:52.269 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:52 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:57:52.319 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:52.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:52.320 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:52.421 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:52.444 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:52.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:52.446 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:52.464 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:52 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:57:52.465 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:52 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:57:52.465 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:52 smithi083 ceph-mon[29421]: 8.1 scrub ok 2024-09-20T18:57:52.465 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:52 smithi083 ceph-mon[29421]: 4.13 scrub starts 2024-09-20T18:57:52.465 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:52 smithi083 ceph-mon[29421]: 4.13 scrub ok 2024-09-20T18:57:52.465 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:52 smithi083 ceph-mon[29421]: pgmap v387: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 1.8 GiB data, 7.4 GiB used, 618 GiB / 626 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 866 op/s; 2382/89049 objects degraded (2.675%); 368 KiB/s, 24 objects/s recovering 2024-09-20T18:57:52.465 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:52 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:57:52.465 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:52 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:57:52.465 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:52 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:57:52.465 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:52 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:57:52.465 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:52 smithi083 ceph-mon[36260]: 8.1 scrub ok 2024-09-20T18:57:52.465 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:52 smithi083 ceph-mon[36260]: 4.13 scrub starts 2024-09-20T18:57:52.465 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:52 smithi083 ceph-mon[36260]: 4.13 scrub ok 2024-09-20T18:57:52.466 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:52 smithi083 ceph-mon[36260]: pgmap v387: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 1.8 GiB data, 7.4 GiB used, 618 GiB / 626 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 866 op/s; 2382/89049 objects degraded (2.675%); 368 KiB/s, 24 objects/s recovering 2024-09-20T18:57:52.466 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:52 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:57:52.466 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:52 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:57:52.538 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T18:57:52.547 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:52.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:52.570 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:52.570 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:52.671 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:52.748 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:57:52.748 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"1ed5a50a-7780-11ef-baf5-efdc52797490","health":{"status":"HEALTH_WARN","checks":{"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"nodeep-scrub flag(s) set","count":12},"muted":false},"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2382/89049 objects degraded (2.675%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":734,"monmap":{"epoch":3,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":231,"num_osds":8,"num_up_osds":8,"osd_up_since":1726858174,"num_in_osds":7,"osd_in_since":1726858527,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":147},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":148,"num_pools":7,"num_objects":29683,"data_bytes":1932794419,"bytes_used":7892684800,"bytes_avail":664072564736,"bytes_total":671965249536,"degraded_objects":2382,"degraded_total":89049,"degraded_ratio":0.026749317791328369,"recovering_objects_per_sec":24,"recovering_bytes_per_sec":376639,"recovering_keys_per_sec":0,"num_objects_recovered":145,"num_bytes_recovered":2228224,"num_keys_recovered":0,"read_bytes_sec":1557,"write_bytes_sec":0,"read_op_per_sec":1,"write_op_per_sec":864},"fsmap":{"epoch":1,"btime":"2024-09-20T18:44:10:399616+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"services":{"dashboard":"https://172.21.15.83:8443/","prometheus":"http://172.21.15.83:9283/"}},"servicemap":{"epoch":14,"modified":"2024-09-20T18:51:16.768808+0000","services":{"rgw":{"daemons":{"summary":"","14460":{"start_epoch":12,"start_stamp":"2024-09-20T18:49:53.053671+0000","gid":14460,"addr":"172.21.15.83:0/1555339426","metadata":{"arch":"x86_64","ceph_release":"squid","ceph_version":"ceph version 19.3.0-5099-g88deadb8 (88deadb8082f4a33832e228ccefc59cedba158a5) squid (dev)","ceph_version_short":"19.3.0-5099-g88deadb8","container_hostname":"smithi083","container_image":"quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2","cpu":"Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz","distro":"centos","distro_description":"CentOS Stream 9","distro_version":"9","frontend_config#0":"beast port=80","frontend_type#0":"beast","hostname":"smithi083","id":"foo.a","kernel_description":"#1 SMP PREEMPT_DYNAMIC Thu Sep 12 15:49:37 UTC 2024","kernel_version":"5.14.0-508.el9.x86_64","mem_swap_kb":"0","mem_total_kb":"32487292","num_handles":"1","os":"Linux","pid":"2","realm_id":"","realm_name":"","zone_id":"41426759-9535-4245-bcfe-8d158f74a134","zone_name":"default","zonegroup_id":"fb472636-52d3-4587-9964-925ef38835b6","zonegroup_name":"default"},"task_status":{}}}}}},"progress_events":{}} 2024-09-20T18:57:52.755 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:52.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:52.756 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:52.857 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:52.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:52.881 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:52.881 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:52.982 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:53.033 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:53.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:53.034 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:53.135 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:57:53.157 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:53.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:53.158 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:57:53.259 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:53.268 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:57:53.282 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:53.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:53.283 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:53.384 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:53.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:53 smithi083 ceph-mon[29421]: osdmap e231: 8 total, 8 up, 7 in 2024-09-20T18:57:53.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:53 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2216117573' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-09-20T18:57:53.405 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:53 smithi083 ceph-mon[36260]: osdmap e231: 8 total, 8 up, 7 in 2024-09-20T18:57:53.405 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:53 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2216117573' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-09-20T18:57:53.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:53.443 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:53.443 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:53.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:53 smithi142 ceph-mon[32508]: osdmap e231: 8 total, 8 up, 7 in 2024-09-20T18:57:53.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:53 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2216117573' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-09-20T18:57:53.544 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:53.586 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:53.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:53.587 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:53.688 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:53.715 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:53.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:53.716 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:53.816 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:53.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:53.851 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:53.851 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:53.952 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:53.977 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:53.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:53.978 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:54.079 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:57:54.098 INFO:tasks.cephadm.ceph_manager.ceph:making progress, resetting timeout 2024-09-20T18:57:54.098 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T18:57:54.104 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:54.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:54.105 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:57:54.148 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-20T18:57:54.206 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:54.250 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:54.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:54.251 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:54.352 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:54.376 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:54.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:54.377 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:54.478 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:54.500 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:54.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:54.501 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:54.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:54 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:57:54.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:54 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:57:54.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:54 smithi142 ceph-mon[32508]: 8.3 scrub starts 2024-09-20T18:57:54.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:54 smithi142 ceph-mon[32508]: 5.11 scrub starts 2024-09-20T18:57:54.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:54 smithi142 ceph-mon[32508]: 5.11 scrub ok 2024-09-20T18:57:54.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:54 smithi142 ceph-mon[32508]: pgmap v389: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 1.5 GiB data, 6.5 GiB used, 619 GiB / 626 GiB avail; 681 B/s rd, 0 B/s wr, 867 op/s; 2225/73419 objects degraded (3.031%); 0 B/s, 26 objects/s recovering 2024-09-20T18:57:54.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:54 smithi142 ceph-mon[32508]: osdmap e232: 8 total, 8 up, 7 in 2024-09-20T18:57:54.520 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:54 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/419147843' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:57:54.520 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:54 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:57:54.520 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:54 smithi142 ceph-mon[32508]: 8.3 scrub ok 2024-09-20T18:57:54.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:54 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:57:54.549 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:54 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:57:54.549 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:54 smithi083 ceph-mon[29421]: 8.3 scrub starts 2024-09-20T18:57:54.549 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:54 smithi083 ceph-mon[29421]: 5.11 scrub starts 2024-09-20T18:57:54.549 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:54 smithi083 ceph-mon[29421]: 5.11 scrub ok 2024-09-20T18:57:54.549 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:54 smithi083 ceph-mon[29421]: pgmap v389: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 1.5 GiB data, 6.5 GiB used, 619 GiB / 626 GiB avail; 681 B/s rd, 0 B/s wr, 867 op/s; 2225/73419 objects degraded (3.031%); 0 B/s, 26 objects/s recovering 2024-09-20T18:57:54.549 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:54 smithi083 ceph-mon[29421]: osdmap e232: 8 total, 8 up, 7 in 2024-09-20T18:57:54.549 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:54 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/419147843' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:57:54.549 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:54 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:57:54.549 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:54 smithi083 ceph-mon[29421]: 8.3 scrub ok 2024-09-20T18:57:54.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:54 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:57:54.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:54 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:57:54.550 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:54 smithi083 ceph-mon[36260]: 8.3 scrub starts 2024-09-20T18:57:54.550 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:54 smithi083 ceph-mon[36260]: 5.11 scrub starts 2024-09-20T18:57:54.550 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:54 smithi083 ceph-mon[36260]: 5.11 scrub ok 2024-09-20T18:57:54.550 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:54 smithi083 ceph-mon[36260]: pgmap v389: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 1.5 GiB data, 6.5 GiB used, 619 GiB / 626 GiB avail; 681 B/s rd, 0 B/s wr, 867 op/s; 2225/73419 objects degraded (3.031%); 0 B/s, 26 objects/s recovering 2024-09-20T18:57:54.550 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:54 smithi083 ceph-mon[36260]: osdmap e232: 8 total, 8 up, 7 in 2024-09-20T18:57:54.550 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:54 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/419147843' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:57:54.550 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:54 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:57:54.550 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:54 smithi083 ceph-mon[36260]: 8.3 scrub ok 2024-09-20T18:57:54.602 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:57:54.630 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:54.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:54.631 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:57:54.732 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:54.760 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:54.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:54.761 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:54.861 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:54.889 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:54.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:54.890 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:54.991 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:55.023 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:55.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:55.024 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:55.125 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:55.164 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:55.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:55.165 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:55.266 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:55.301 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:55.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:55.302 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:55.403 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:55.426 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:55.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:55.427 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:55.511 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:57:55.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:55 smithi142 ceph-mon[32508]: 3.1a scrub starts 2024-09-20T18:57:55.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:55 smithi142 ceph-mon[32508]: 3.1a scrub ok 2024-09-20T18:57:55.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:55 smithi142 ceph-mon[32508]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T18:57:55.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:55 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T18:57:55.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:55 smithi142 ceph-mon[32508]: osdmap e233: 8 total, 8 up, 7 in 2024-09-20T18:57:55.528 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:55.548 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:55 smithi083 ceph-mon[36260]: 3.1a scrub starts 2024-09-20T18:57:55.548 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:55 smithi083 ceph-mon[36260]: 3.1a scrub ok 2024-09-20T18:57:55.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:55 smithi083 ceph-mon[36260]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T18:57:55.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:55 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T18:57:55.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:55 smithi083 ceph-mon[36260]: osdmap e233: 8 total, 8 up, 7 in 2024-09-20T18:57:55.549 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:55 smithi083 ceph-mon[29421]: 3.1a scrub starts 2024-09-20T18:57:55.549 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:55 smithi083 ceph-mon[29421]: 3.1a scrub ok 2024-09-20T18:57:55.549 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:55 smithi083 ceph-mon[29421]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T18:57:55.549 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:55 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T18:57:55.549 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:55 smithi083 ceph-mon[29421]: osdmap e233: 8 total, 8 up, 7 in 2024-09-20T18:57:55.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:55.585 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:55.586 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:55.686 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:55.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:55.729 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:55.729 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:55.830 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:55.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:55.868 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:55.868 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:55.969 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:56.008 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:56.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:56.009 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:56.110 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:57:56.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:56.136 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:56.136 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:57:56.237 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:56.244 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:56 smithi083 ceph-mon[29421]: 5.7 scrub starts 2024-09-20T18:57:56.244 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:56 smithi083 ceph-mon[29421]: 5.7 scrub ok 2024-09-20T18:57:56.244 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:56 smithi083 ceph-mon[29421]: 4.11 scrub starts 2024-09-20T18:57:56.244 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:56 smithi083 ceph-mon[29421]: 4.11 scrub ok 2024-09-20T18:57:56.244 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:56 smithi083 ceph-mon[29421]: pgmap v392: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 1.5 GiB data, 6.5 GiB used, 619 GiB / 626 GiB avail; 968 B/s rd, 0 B/s wr, 1.23k op/s; 2225/73419 objects degraded (3.031%); 0 B/s, 37 objects/s recovering 2024-09-20T18:57:56.244 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:56 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 2225/73419 objects degraded (3.031%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T18:57:56.245 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:56 smithi083 ceph-mon[36260]: 5.7 scrub starts 2024-09-20T18:57:56.245 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:56 smithi083 ceph-mon[36260]: 5.7 scrub ok 2024-09-20T18:57:56.245 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:56 smithi083 ceph-mon[36260]: 4.11 scrub starts 2024-09-20T18:57:56.245 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:56 smithi083 ceph-mon[36260]: 4.11 scrub ok 2024-09-20T18:57:56.245 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:56 smithi083 ceph-mon[36260]: pgmap v392: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 1.5 GiB data, 6.5 GiB used, 619 GiB / 626 GiB avail; 968 B/s rd, 0 B/s wr, 1.23k op/s; 2225/73419 objects degraded (3.031%); 0 B/s, 37 objects/s recovering 2024-09-20T18:57:56.245 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:56 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 2225/73419 objects degraded (3.031%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T18:57:56.259 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:56.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:56.260 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:56.360 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:56.398 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:56.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:56.399 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:56.500 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:56.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:56 smithi142 ceph-mon[32508]: 5.7 scrub starts 2024-09-20T18:57:56.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:56 smithi142 ceph-mon[32508]: 5.7 scrub ok 2024-09-20T18:57:56.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:56 smithi142 ceph-mon[32508]: 4.11 scrub starts 2024-09-20T18:57:56.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:56 smithi142 ceph-mon[32508]: 4.11 scrub ok 2024-09-20T18:57:56.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:56 smithi142 ceph-mon[32508]: pgmap v392: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 1.5 GiB data, 6.5 GiB used, 619 GiB / 626 GiB avail; 968 B/s rd, 0 B/s wr, 1.23k op/s; 2225/73419 objects degraded (3.031%); 0 B/s, 37 objects/s recovering 2024-09-20T18:57:56.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:56 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 2225/73419 objects degraded (3.031%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T18:57:56.535 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:56.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:56.536 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:56.637 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:56.673 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:56.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:56.674 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:56.775 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:57:56.801 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:56.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:56.803 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:57:56.886 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T18:57:56.887 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:57:56.903 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:56.947 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:56.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:56.948 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:57.049 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:57.071 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:57.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:57.072 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:57.151 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:57:56 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:57:57.173 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:57.195 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:57.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:57.197 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:57.298 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:57.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:57.334 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:57.334 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:57.435 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:57.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:57.473 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:57.474 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:57.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:57 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2162874349' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:57:57.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:57 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2162874349' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:57:57.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:57 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2162874349' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:57:57.574 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:57.620 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:57.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:57.621 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:57.722 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:57.759 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:57.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:57.760 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:57.860 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:57.893 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:57.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:57.895 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:57.995 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:58.020 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:58.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:58.021 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:58.122 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:58.158 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:58.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:58.159 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:58.260 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:57:58.294 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:58.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:58.295 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:57:58.395 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:58.434 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:58.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:58.435 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:58.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:58 smithi142 ceph-mon[32508]: 3.c scrub starts 2024-09-20T18:57:58.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:58 smithi142 ceph-mon[32508]: 3.c scrub ok 2024-09-20T18:57:58.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:58 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:57:58.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:58 smithi142 ceph-mon[32508]: pgmap v393: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 1.2 GiB data, 5.6 GiB used, 620 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 1.68k op/s; 2080/58827 objects degraded (3.536%); 0 B/s, 50 objects/s recovering 2024-09-20T18:57:58.535 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:57:58.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:58 smithi083 ceph-mon[29421]: 3.c scrub starts 2024-09-20T18:57:58.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:58 smithi083 ceph-mon[29421]: 3.c scrub ok 2024-09-20T18:57:58.549 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:58 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:57:58.549 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:58 smithi083 ceph-mon[29421]: pgmap v393: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 1.2 GiB data, 5.6 GiB used, 620 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 1.68k op/s; 2080/58827 objects degraded (3.536%); 0 B/s, 50 objects/s recovering 2024-09-20T18:57:58.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:58 smithi083 ceph-mon[36260]: 3.c scrub starts 2024-09-20T18:57:58.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:58 smithi083 ceph-mon[36260]: 3.c scrub ok 2024-09-20T18:57:58.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:58 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:57:58.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:58 smithi083 ceph-mon[36260]: pgmap v393: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 1.2 GiB data, 5.6 GiB used, 620 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 1.68k op/s; 2080/58827 objects degraded (3.536%); 0 B/s, 50 objects/s recovering 2024-09-20T18:57:58.560 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:58.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:58.561 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:57:58.661 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:58.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:58.700 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:58.700 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:58.801 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:57:58.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:58.877 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:58.877 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:57:58.978 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:59.012 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:59.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:59.013 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:59.114 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:57:59.138 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:59.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:59.139 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:57:59.240 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:57:59.263 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:59.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:59.264 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:57:59.365 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:57:59.393 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:59.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:59.394 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:57:59.495 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:59.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:59 smithi142 ceph-mon[32508]: 5.13 scrub starts 2024-09-20T18:57:59.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:59 smithi142 ceph-mon[32508]: 5.13 scrub ok 2024-09-20T18:57:59.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:59 smithi142 ceph-mon[32508]: 6.1 scrub starts 2024-09-20T18:57:59.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:57:59 smithi142 ceph-mon[32508]: 6.1 scrub ok 2024-09-20T18:57:59.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:59.532 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:59.532 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:59.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:59 smithi083 ceph-mon[29421]: 5.13 scrub starts 2024-09-20T18:57:59.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:59 smithi083 ceph-mon[29421]: 5.13 scrub ok 2024-09-20T18:57:59.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:59 smithi083 ceph-mon[29421]: 6.1 scrub starts 2024-09-20T18:57:59.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:57:59 smithi083 ceph-mon[29421]: 6.1 scrub ok 2024-09-20T18:57:59.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:59 smithi083 ceph-mon[36260]: 5.13 scrub starts 2024-09-20T18:57:59.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:59 smithi083 ceph-mon[36260]: 5.13 scrub ok 2024-09-20T18:57:59.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:59 smithi083 ceph-mon[36260]: 6.1 scrub starts 2024-09-20T18:57:59.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:57:59 smithi083 ceph-mon[36260]: 6.1 scrub ok 2024-09-20T18:57:59.634 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:57:59.655 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:57:59.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:59.656 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:57:59.757 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:59.790 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:59.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:59.791 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:57:59.892 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:57:59.927 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:57:59.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:57:59.928 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:00.029 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:00.051 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:00.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:00.052 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:00.152 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:00.180 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:00.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:00.182 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:00.268 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:58:00.268 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:58:00.269 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:58:00.269 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:58:00.269 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:58:00.269 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:58:00.270 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:58:00.270 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:58:00.270 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:58:00.270 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:58:00.270 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:58:00.270 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:58:00.271 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:58:00.271 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:58:00.271 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:58:00.272 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:58:00.284 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:00.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:00.322 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:00.322 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:00.423 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:00.444 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:00.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:00.445 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:00.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:00 smithi142 ceph-mon[32508]: pgmap v394: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 1.2 GiB data, 5.6 GiB used, 620 GiB / 626 GiB avail; 746 B/s rd, 0 B/s wr, 709 op/s; 2080/58827 objects degraded (3.536%); 0 B/s, 21 objects/s recovering 2024-09-20T18:58:00.547 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:58:00.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:00 smithi083 ceph-mon[29421]: pgmap v394: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 1.2 GiB data, 5.6 GiB used, 620 GiB / 626 GiB avail; 746 B/s rd, 0 B/s wr, 709 op/s; 2080/58827 objects degraded (3.536%); 0 B/s, 21 objects/s recovering 2024-09-20T18:58:00.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:00 smithi083 ceph-mon[36260]: pgmap v394: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 1.2 GiB data, 5.6 GiB used, 620 GiB / 626 GiB avail; 746 B/s rd, 0 B/s wr, 709 op/s; 2080/58827 objects degraded (3.536%); 0 B/s, 21 objects/s recovering 2024-09-20T18:58:00.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:00.584 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:00.584 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:58:00.685 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:00.714 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:00.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:00.715 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:00.816 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:00.839 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:00.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:00.840 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:00.941 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:58:00.972 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:00.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:00.973 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:58:01.048 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:58:00 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:58:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:58:01.074 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:01.116 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:01.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:01.117 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:01.218 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:58:01.239 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:01.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:01.240 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:58:01.341 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:01.381 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:01.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:01.382 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:01.483 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:01.507 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:01.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:01.508 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:01.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:01 smithi142 ceph-mon[32508]: 4.15 scrub starts 2024-09-20T18:58:01.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:01 smithi142 ceph-mon[32508]: 4.15 scrub ok 2024-09-20T18:58:01.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:01 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 2080/58827 objects degraded (3.536%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T18:58:01.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:01 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:58:01.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:01 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:58:01.548 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:01 smithi083 ceph-mon[36260]: 4.15 scrub starts 2024-09-20T18:58:01.548 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:01 smithi083 ceph-mon[36260]: 4.15 scrub ok 2024-09-20T18:58:01.548 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:01 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 2080/58827 objects degraded (3.536%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T18:58:01.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:01 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:58:01.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:01 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:58:01.549 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:01 smithi083 ceph-mon[29421]: 4.15 scrub starts 2024-09-20T18:58:01.549 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:01 smithi083 ceph-mon[29421]: 4.15 scrub ok 2024-09-20T18:58:01.549 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:01 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 2080/58827 objects degraded (3.536%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T18:58:01.549 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:01 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:58:01.549 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:01 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:58:01.610 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:58:01.647 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:01.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:01.648 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:58:01.748 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:58:01.787 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:01.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:01.789 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:58:01.889 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:01.911 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:01.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:01.912 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:02.013 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:02.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:02.035 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:02.035 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:02.136 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:02.160 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:02.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:02.161 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:02.261 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:02.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:02.285 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:02.286 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:02.386 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:58:02.410 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:02.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:02.411 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:58:02.512 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:02.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:02 smithi142 ceph-mon[32508]: 6.1e scrub starts 2024-09-20T18:58:02.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:02 smithi142 ceph-mon[32508]: 6.1e scrub ok 2024-09-20T18:58:02.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:02 smithi142 ceph-mon[32508]: pgmap v395: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 1.1 GiB data, 5.3 GiB used, 621 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 842 op/s; 2080/53217 objects degraded (3.909%); 0 B/s, 18 objects/s recovering 2024-09-20T18:58:02.538 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:02.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:02.539 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:02.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:02 smithi083 ceph-mon[29421]: 6.1e scrub starts 2024-09-20T18:58:02.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:02 smithi083 ceph-mon[29421]: 6.1e scrub ok 2024-09-20T18:58:02.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:02 smithi083 ceph-mon[29421]: pgmap v395: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 1.1 GiB data, 5.3 GiB used, 621 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 842 op/s; 2080/53217 objects degraded (3.909%); 0 B/s, 18 objects/s recovering 2024-09-20T18:58:02.548 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:02 smithi083 ceph-mon[36260]: 6.1e scrub starts 2024-09-20T18:58:02.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:02 smithi083 ceph-mon[36260]: 6.1e scrub ok 2024-09-20T18:58:02.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:02 smithi083 ceph-mon[36260]: pgmap v395: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 1.1 GiB data, 5.3 GiB used, 621 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 842 op/s; 2080/53217 objects degraded (3.909%); 0 B/s, 18 objects/s recovering 2024-09-20T18:58:02.640 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:02.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:02.680 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:02.681 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:02.782 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:58:02.821 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:02.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:02.821 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:58:02.922 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:02.948 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:02.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:02.949 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:03.049 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:03.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:03.078 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:03.078 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:03.179 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:03.214 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:03.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:03.215 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:03.315 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:03.353 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:03.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:03.354 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:03.455 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:03.480 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:03.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:03.481 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:03.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:03 smithi142 ceph-mon[32508]: 6.1a scrub starts 2024-09-20T18:58:03.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:03 smithi142 ceph-mon[32508]: 6.1a scrub ok 2024-09-20T18:58:03.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:03 smithi142 ceph-mon[32508]: 8.b scrub starts 2024-09-20T18:58:03.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:03 smithi142 ceph-mon[32508]: 8.b scrub ok 2024-09-20T18:58:03.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:03 smithi142 ceph-mon[32508]: 4.2 scrub starts 2024-09-20T18:58:03.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:03 smithi142 ceph-mon[32508]: 4.2 scrub ok 2024-09-20T18:58:03.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:03 smithi083 ceph-mon[29421]: 6.1a scrub starts 2024-09-20T18:58:03.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:03 smithi083 ceph-mon[29421]: 6.1a scrub ok 2024-09-20T18:58:03.549 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:03 smithi083 ceph-mon[29421]: 8.b scrub starts 2024-09-20T18:58:03.549 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:03 smithi083 ceph-mon[29421]: 8.b scrub ok 2024-09-20T18:58:03.549 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:03 smithi083 ceph-mon[29421]: 4.2 scrub starts 2024-09-20T18:58:03.549 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:03 smithi083 ceph-mon[29421]: 4.2 scrub ok 2024-09-20T18:58:03.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:03 smithi083 ceph-mon[36260]: 6.1a scrub starts 2024-09-20T18:58:03.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:03 smithi083 ceph-mon[36260]: 6.1a scrub ok 2024-09-20T18:58:03.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:03 smithi083 ceph-mon[36260]: 8.b scrub starts 2024-09-20T18:58:03.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:03 smithi083 ceph-mon[36260]: 8.b scrub ok 2024-09-20T18:58:03.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:03 smithi083 ceph-mon[36260]: 4.2 scrub starts 2024-09-20T18:58:03.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:03 smithi083 ceph-mon[36260]: 4.2 scrub ok 2024-09-20T18:58:03.582 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:03.610 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:03.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:03.611 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:03.711 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:03.735 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:03.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:03.736 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:03.837 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:03.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:03.876 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:03.876 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:03.977 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:04.017 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:04.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:04.017 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:04.118 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:04.160 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:04.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:04.161 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:04.262 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:04.305 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:04.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:04.306 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:04.407 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:04.453 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:04.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:04.454 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:04.478 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:04 smithi083 ceph-mon[29421]: pgmap v396: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 908 MiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 992 op/s; 1945/44133 objects degraded (4.407%); 0 B/s, 28 objects/s recovering 2024-09-20T18:58:04.479 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:04 smithi083 ceph-mon[36260]: pgmap v396: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 908 MiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 992 op/s; 1945/44133 objects degraded (4.407%); 0 B/s, 28 objects/s recovering 2024-09-20T18:58:04.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:04 smithi142 ceph-mon[32508]: pgmap v396: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 908 MiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 992 op/s; 1945/44133 objects degraded (4.407%); 0 B/s, 28 objects/s recovering 2024-09-20T18:58:04.554 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:04.581 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:04.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:04.581 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:04.682 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:04.711 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:04.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:04.712 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:04.812 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:04.849 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:04.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:04.850 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:04.895 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_historic_ops 2024-09-20T18:58:04.950 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:04.971 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:58:05.021 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:05.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:05.022 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:05.124 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:05.149 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:05.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:05.151 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:05.251 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:05.300 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:05.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:05.301 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:05.402 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:05.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:05.428 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:05.428 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:05.453 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:05 smithi083 ceph-mon[29421]: 6.7 scrub starts 2024-09-20T18:58:05.453 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:05 smithi083 ceph-mon[29421]: 6.7 scrub ok 2024-09-20T18:58:05.453 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:05 smithi083 ceph-mon[36260]: 6.7 scrub starts 2024-09-20T18:58:05.454 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:05 smithi083 ceph-mon[36260]: 6.7 scrub ok 2024-09-20T18:58:05.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:05 smithi142 ceph-mon[32508]: 6.7 scrub starts 2024-09-20T18:58:05.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:05 smithi142 ceph-mon[32508]: 6.7 scrub ok 2024-09-20T18:58:05.529 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:05.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:05.568 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:05.568 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:05.614 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:58:05.669 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:05.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:05.725 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:05.725 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:05.766 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T18:58:05.826 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:05.849 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:05.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:05.850 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:05.951 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:58:05.997 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:58:06.001 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-20T18:58:06.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:06.054 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:06.055 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:58:06.155 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:06.182 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:06.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:06.184 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:06.285 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:06.298 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:06 smithi083 ceph-mon[29421]: pgmap v397: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 908 MiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 946 B/s rd, 0 B/s wr, 902 op/s; 1945/44133 objects degraded (4.407%); 0 B/s, 25 objects/s recovering 2024-09-20T18:58:06.298 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:06 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 1945/44133 objects degraded (4.407%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T18:58:06.299 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:06 smithi083 ceph-mon[29421]: 4.7 scrub starts 2024-09-20T18:58:06.299 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:06 smithi083 ceph-mon[29421]: 4.7 scrub ok 2024-09-20T18:58:06.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:06 smithi083 ceph-mon[36260]: pgmap v397: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 908 MiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 946 B/s rd, 0 B/s wr, 902 op/s; 1945/44133 objects degraded (4.407%); 0 B/s, 25 objects/s recovering 2024-09-20T18:58:06.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:06 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 1945/44133 objects degraded (4.407%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T18:58:06.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:06 smithi083 ceph-mon[36260]: 4.7 scrub starts 2024-09-20T18:58:06.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:06 smithi083 ceph-mon[36260]: 4.7 scrub ok 2024-09-20T18:58:06.326 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:06.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:06.328 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:06.381 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T18:58:06.381 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-20T18:58:06.382 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-20T18:58:06.382 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-20T18:58:06.382 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.382 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:25327 7.3 7:cbe7b718:::benchmark_data_smithi083_70643_object25326:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:58:06.382 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.263028+0000", 2024-09-20T18:58:06.382 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 419.10214027500001, 2024-09-20T18:58:06.382 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.1801133, 2024-09-20T18:58:06.382 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:58:06.382 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:58:06.382 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:06.382 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:58:06.382 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:58:06.383 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:58:06.383 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25327 2024-09-20T18:58:06.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.383 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:58:06.383 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.383 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:58:06.383 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263028+0000", 2024-09-20T18:58:06.383 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:06.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.383 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.383 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:58:06.383 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263028+0000", 2024-09-20T18:58:06.383 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:06.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.384 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.384 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:58:06.384 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263029+0000", 2024-09-20T18:58:06.384 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1200000000000001e-06 2024-09-20T18:58:06.384 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.384 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.384 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:58:06.384 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263050+0000", 2024-09-20T18:58:06.384 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1019999999999999e-05 2024-09-20T18:58:06.384 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.384 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.384 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:58:06.384 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263060+0000", 2024-09-20T18:58:06.385 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0049e-05 2024-09-20T18:58:06.385 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.385 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.385 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:58:06.385 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263069+0000", 2024-09-20T18:58:06.385 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8999999999999995e-06 2024-09-20T18:58:06.385 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.385 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.385 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:06.385 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263770+0000", 2024-09-20T18:58:06.385 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00070160100000000005 2024-09-20T18:58:06.385 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.386 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.386 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:58:06.386 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263876+0000", 2024-09-20T18:58:06.386 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010538400000000001 2024-09-20T18:58:06.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.386 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.386 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,4", 2024-09-20T18:58:06.386 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263991+0000", 2024-09-20T18:58:06.386 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115151 2024-09-20T18:58:06.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.386 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.386 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:58:06.386 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.265911+0000", 2024-09-20T18:58:06.386 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0019197319999999999 2024-09-20T18:58:06.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.387 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.387 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:06.387 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.267644+0000", 2024-09-20T18:58:06.387 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0017335600000000001 2024-09-20T18:58:06.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.387 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.387 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:06.387 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443038+0000", 2024-09-20T18:58:06.387 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17539418100000001 2024-09-20T18:58:06.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.387 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.388 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:58:06.388 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443095+0000", 2024-09-20T18:58:06.388 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6972999999999998e-05 2024-09-20T18:58:06.388 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.388 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.388 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:58:06.388 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443141+0000", 2024-09-20T18:58:06.388 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5629e-05 2024-09-20T18:58:06.388 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:06.388 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:58:06.388 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:06.388 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.388 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.389 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:25329 7.3 7:c0dcc395:::benchmark_data_smithi083_70643_object25328:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:58:06.389 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.263576+0000", 2024-09-20T18:58:06.389 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 419.10159208499999, 2024-09-20T18:58:06.389 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17968018699999999, 2024-09-20T18:58:06.389 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:58:06.389 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:58:06.389 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:06.389 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:58:06.389 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:58:06.389 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:58:06.389 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25329 2024-09-20T18:58:06.389 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.389 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:58:06.389 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.389 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:58:06.390 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263576+0000", 2024-09-20T18:58:06.390 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:06.390 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.390 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.390 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:58:06.390 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263576+0000", 2024-09-20T18:58:06.390 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:06.390 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.390 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.390 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:58:06.390 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263577+0000", 2024-09-20T18:58:06.390 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.3799999999999996e-07 2024-09-20T18:58:06.390 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.390 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.390 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:58:06.391 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263599+0000", 2024-09-20T18:58:06.391 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2333000000000001e-05 2024-09-20T18:58:06.391 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.391 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.391 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:58:06.391 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263608+0000", 2024-09-20T18:58:06.391 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5690000000000005e-06 2024-09-20T18:58:06.391 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.391 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.391 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:58:06.391 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263614+0000", 2024-09-20T18:58:06.391 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6449999999999999e-06 2024-09-20T18:58:06.391 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.391 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.391 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:06.392 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.264856+0000", 2024-09-20T18:58:06.392 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001241486 2024-09-20T18:58:06.392 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.392 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.392 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:58:06.392 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.264959+0000", 2024-09-20T18:58:06.392 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000103037 2024-09-20T18:58:06.392 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.392 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.392 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,4", 2024-09-20T18:58:06.392 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.265077+0000", 2024-09-20T18:58:06.392 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000118163 2024-09-20T18:58:06.392 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.392 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.392 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:58:06.392 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266326+0000", 2024-09-20T18:58:06.393 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001248929 2024-09-20T18:58:06.393 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.393 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.393 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:06.393 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.267696+0000", 2024-09-20T18:58:06.393 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001369924 2024-09-20T18:58:06.393 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.393 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.393 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:06.393 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443180+0000", 2024-09-20T18:58:06.393 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17548422 2024-09-20T18:58:06.393 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.393 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.393 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:58:06.393 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443222+0000", 2024-09-20T18:58:06.394 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2426999999999999e-05 2024-09-20T18:58:06.394 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.394 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.394 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:58:06.394 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443256+0000", 2024-09-20T18:58:06.394 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3615999999999998e-05 2024-09-20T18:58:06.394 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:06.394 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:58:06.394 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:06.394 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.394 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.394 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:25332 7.2 7:4a99327a:::benchmark_data_smithi083_70643_object25331:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:58:06.394 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.266138+0000", 2024-09-20T18:58:06.394 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 419.099029621, 2024-09-20T18:58:06.395 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17949907200000001, 2024-09-20T18:58:06.395 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:58:06.395 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:58:06.395 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:06.395 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:58:06.395 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:58:06.395 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:58:06.395 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25332 2024-09-20T18:58:06.395 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.395 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:58:06.395 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.395 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:58:06.395 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266138+0000", 2024-09-20T18:58:06.395 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:06.396 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.396 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.396 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:58:06.396 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266138+0000", 2024-09-20T18:58:06.396 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:06.396 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.396 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.396 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:58:06.396 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266140+0000", 2024-09-20T18:58:06.396 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1480000000000001e-06 2024-09-20T18:58:06.396 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.396 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.396 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:58:06.396 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266160+0000", 2024-09-20T18:58:06.397 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0908999999999999e-05 2024-09-20T18:58:06.397 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.397 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.397 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:58:06.397 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266170+0000", 2024-09-20T18:58:06.397 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1390000000000001e-06 2024-09-20T18:58:06.397 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.397 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.397 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:58:06.397 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266176+0000", 2024-09-20T18:58:06.397 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3740000000000003e-06 2024-09-20T18:58:06.397 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.397 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.397 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:06.398 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266234+0000", 2024-09-20T18:58:06.398 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8084999999999997e-05 2024-09-20T18:58:06.398 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.398 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.398 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:58:06.398 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266342+0000", 2024-09-20T18:58:06.398 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000108448 2024-09-20T18:58:06.398 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.398 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.398 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T18:58:06.398 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266470+0000", 2024-09-20T18:58:06.398 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012756700000000001 2024-09-20T18:58:06.398 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.398 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.399 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:06.399 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444956+0000", 2024-09-20T18:58:06.399 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.178485641 2024-09-20T18:58:06.399 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.399 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.399 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:06.399 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445026+0000", 2024-09-20T18:58:06.399 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0778999999999994e-05 2024-09-20T18:58:06.399 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.399 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.399 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:58:06.400 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445522+0000", 2024-09-20T18:58:06.400 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00049521500000000002 2024-09-20T18:58:06.400 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.400 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.400 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:58:06.400 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445598+0000", 2024-09-20T18:58:06.400 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5993000000000002e-05 2024-09-20T18:58:06.400 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.400 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.400 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:58:06.400 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445637+0000", 2024-09-20T18:58:06.400 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9774000000000002e-05 2024-09-20T18:58:06.400 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:06.400 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:58:06.400 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:06.401 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.401 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.401 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:25333 7.2 7:4c00f64e:::benchmark_data_smithi083_70643_object25332:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:58:06.401 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.266192+0000", 2024-09-20T18:58:06.401 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 419.09897575600002, 2024-09-20T18:58:06.401 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17952103699999999, 2024-09-20T18:58:06.401 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:58:06.401 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:58:06.401 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:06.401 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:58:06.401 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:58:06.401 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:58:06.401 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25333 2024-09-20T18:58:06.401 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.402 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:58:06.402 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.402 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:58:06.402 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266192+0000", 2024-09-20T18:58:06.402 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:06.402 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.402 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.402 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:58:06.402 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266192+0000", 2024-09-20T18:58:06.402 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:06.402 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.402 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.402 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:58:06.402 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266193+0000", 2024-09-20T18:58:06.403 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0999999999999998e-07 2024-09-20T18:58:06.403 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.405 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.405 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:58:06.406 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266211+0000", 2024-09-20T18:58:06.406 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8587e-05 2024-09-20T18:58:06.406 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.406 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.406 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:58:06.406 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266218+0000", 2024-09-20T18:58:06.406 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7340000000000002e-06 2024-09-20T18:58:06.406 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.406 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.406 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:58:06.406 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266224+0000", 2024-09-20T18:58:06.407 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6590000000000004e-06 2024-09-20T18:58:06.407 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.407 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.407 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:06.407 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268185+0000", 2024-09-20T18:58:06.407 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0019615240000000001 2024-09-20T18:58:06.407 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.407 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.407 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:58:06.407 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268326+0000", 2024-09-20T18:58:06.407 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014032399999999999 2024-09-20T18:58:06.407 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.407 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.408 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T18:58:06.408 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268450+0000", 2024-09-20T18:58:06.408 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012426599999999999 2024-09-20T18:58:06.408 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.408 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.408 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:06.408 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444991+0000", 2024-09-20T18:58:06.408 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17654108199999999 2024-09-20T18:58:06.408 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.408 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.408 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:06.408 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445507+0000", 2024-09-20T18:58:06.408 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00051583599999999999 2024-09-20T18:58:06.408 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.408 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.409 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:58:06.409 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445645+0000", 2024-09-20T18:58:06.409 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000137959 2024-09-20T18:58:06.409 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.409 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.409 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:58:06.409 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445682+0000", 2024-09-20T18:58:06.409 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7455e-05 2024-09-20T18:58:06.409 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.409 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.409 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:58:06.409 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445713+0000", 2024-09-20T18:58:06.409 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1000999999999997e-05 2024-09-20T18:58:06.409 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:06.409 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:58:06.409 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:06.410 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.410 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.410 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:25336 7.2 7:4be582b4:::benchmark_data_smithi083_70643_object25335:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:58:06.410 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.267773+0000", 2024-09-20T18:58:06.410 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 419.09739490200002, 2024-09-20T18:58:06.410 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.18517899500000001, 2024-09-20T18:58:06.410 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:58:06.410 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:58:06.410 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:06.410 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:58:06.410 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:58:06.410 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:58:06.410 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25336 2024-09-20T18:58:06.411 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.411 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:58:06.411 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.411 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:58:06.411 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.267773+0000", 2024-09-20T18:58:06.411 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:06.411 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.411 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.411 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:58:06.411 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.267773+0000", 2024-09-20T18:58:06.411 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:06.411 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.411 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.412 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:58:06.412 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.267774+0000", 2024-09-20T18:58:06.412 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.13e-06 2024-09-20T18:58:06.412 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.413 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.413 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:58:06.413 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.267797+0000", 2024-09-20T18:58:06.413 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2852000000000001e-05 2024-09-20T18:58:06.413 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.413 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.413 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:58:06.413 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.267806+0000", 2024-09-20T18:58:06.413 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4519999999999997e-06 2024-09-20T18:58:06.413 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.413 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.413 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:58:06.413 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.267812+0000", 2024-09-20T18:58:06.414 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3489999999999997e-06 2024-09-20T18:58:06.414 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.414 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.414 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:06.414 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270828+0000", 2024-09-20T18:58:06.414 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0030158020000000002 2024-09-20T18:58:06.414 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.414 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.414 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:58:06.414 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443376+0000", 2024-09-20T18:58:06.414 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17254844 2024-09-20T18:58:06.414 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.414 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.414 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T18:58:06.415 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443495+0000", 2024-09-20T18:58:06.415 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000119338 2024-09-20T18:58:06.415 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.415 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.415 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:58:06.415 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.446411+0000", 2024-09-20T18:58:06.415 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0029155869999999999 2024-09-20T18:58:06.415 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.415 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.415 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:06.415 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.452802+0000", 2024-09-20T18:58:06.415 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063913169999999997 2024-09-20T18:58:06.415 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.415 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.416 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:06.416 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.452870+0000", 2024-09-20T18:58:06.416 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7824999999999995e-05 2024-09-20T18:58:06.416 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.416 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.416 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:58:06.416 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.452915+0000", 2024-09-20T18:58:06.416 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4815999999999999e-05 2024-09-20T18:58:06.416 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.416 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.416 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:58:06.416 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.452952+0000", 2024-09-20T18:58:06.416 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7086999999999999e-05 2024-09-20T18:58:06.416 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:06.417 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:58:06.417 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:06.417 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.417 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.417 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:25337 7.f e93/83 7:fd2b3e49:::benchmark_data_smithi083_70643_object25336:head v 93'1586, mlcod=93'1584)", 2024-09-20T18:58:06.417 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.269129+0000", 2024-09-20T18:58:06.417 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 419.09603858999998, 2024-09-20T18:58:06.417 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.174986899, 2024-09-20T18:58:06.417 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:58:06.417 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:58:06.417 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:06.417 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:58:06.417 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.417 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:58:06.418 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269129+0000", 2024-09-20T18:58:06.418 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:06.418 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.418 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.418 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:58:06.418 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269129+0000", 2024-09-20T18:58:06.418 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:06.418 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.418 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.418 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:58:06.418 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269130+0000", 2024-09-20T18:58:06.418 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8000000000000001e-07 2024-09-20T18:58:06.418 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.419 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.419 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:58:06.419 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269151+0000", 2024-09-20T18:58:06.419 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1593000000000001e-05 2024-09-20T18:58:06.419 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.419 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.419 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:58:06.419 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269158+0000", 2024-09-20T18:58:06.419 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1069999999999999e-06 2024-09-20T18:58:06.419 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.419 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.419 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:58:06.419 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269163+0000", 2024-09-20T18:58:06.419 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7029999999999997e-06 2024-09-20T18:58:06.419 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.420 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.420 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:06.420 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269349+0000", 2024-09-20T18:58:06.420 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000185331 2024-09-20T18:58:06.420 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.420 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.420 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:58:06.420 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269365+0000", 2024-09-20T18:58:06.420 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6424000000000001e-05 2024-09-20T18:58:06.420 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.421 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.421 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:58:06.421 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271621+0000", 2024-09-20T18:58:06.421 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0022557200000000001 2024-09-20T18:58:06.421 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.421 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.421 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:58:06.421 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444116+0000", 2024-09-20T18:58:06.421 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17249564100000001 2024-09-20T18:58:06.421 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:06.421 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:58:06.421 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:06.421 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.421 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.421 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:25340 7.6 e93/83 7:6dd1b43c:::benchmark_data_smithi083_70643_object25339:head v 93'1582, mlcod=93'1579)", 2024-09-20T18:58:06.422 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.269179+0000", 2024-09-20T18:58:06.422 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 419.09598874900001, 2024-09-20T18:58:06.422 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.173948506, 2024-09-20T18:58:06.422 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:58:06.422 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:58:06.422 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:06.422 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:58:06.422 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.422 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:58:06.422 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269179+0000", 2024-09-20T18:58:06.422 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:06.422 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.422 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.423 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:58:06.423 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269179+0000", 2024-09-20T18:58:06.423 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:06.423 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.423 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.423 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:58:06.423 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269179+0000", 2024-09-20T18:58:06.423 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.48e-07 2024-09-20T18:58:06.423 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.423 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.423 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:58:06.423 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269198+0000", 2024-09-20T18:58:06.423 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8972999999999999e-05 2024-09-20T18:58:06.423 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.423 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.423 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:58:06.424 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269204+0000", 2024-09-20T18:58:06.424 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0440000000000003e-06 2024-09-20T18:58:06.424 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.424 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.424 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:58:06.424 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269207+0000", 2024-09-20T18:58:06.424 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6789999999999998e-06 2024-09-20T18:58:06.424 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.424 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.424 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:06.424 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270103+0000", 2024-09-20T18:58:06.424 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00089533500000000003 2024-09-20T18:58:06.424 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.424 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.424 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:58:06.424 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270120+0000", 2024-09-20T18:58:06.425 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7455000000000001e-05 2024-09-20T18:58:06.425 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.425 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.425 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:58:06.425 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443096+0000", 2024-09-20T18:58:06.425 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17297637699999999 2024-09-20T18:58:06.425 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.425 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.425 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:58:06.425 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443128+0000", 2024-09-20T18:58:06.425 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1395e-05 2024-09-20T18:58:06.425 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:06.425 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:58:06.425 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:06.425 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.426 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.426 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:25341 7.a 7:5bb50a52:::benchmark_data_smithi083_70643_object25340:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:58:06.426 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.269273+0000", 2024-09-20T18:58:06.426 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 419.09589451900001, 2024-09-20T18:58:06.426 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.18597625000000001, 2024-09-20T18:58:06.426 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:58:06.426 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:58:06.426 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:06.426 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:58:06.426 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:58:06.426 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:58:06.426 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25341 2024-09-20T18:58:06.426 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.426 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:58:06.427 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.427 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:58:06.427 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269273+0000", 2024-09-20T18:58:06.427 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:06.427 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.427 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.427 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:58:06.427 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269273+0000", 2024-09-20T18:58:06.427 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:06.427 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.427 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.427 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:58:06.427 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269274+0000", 2024-09-20T18:58:06.427 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0139999999999999e-06 2024-09-20T18:58:06.427 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.428 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.428 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:58:06.428 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269293+0000", 2024-09-20T18:58:06.428 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8499e-05 2024-09-20T18:58:06.428 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.428 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.428 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:58:06.428 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269301+0000", 2024-09-20T18:58:06.428 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8429999999999993e-06 2024-09-20T18:58:06.428 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.428 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.428 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:58:06.429 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269306+0000", 2024-09-20T18:58:06.429 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2809999999999999e-06 2024-09-20T18:58:06.429 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.429 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.429 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:06.429 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445057+0000", 2024-09-20T18:58:06.429 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17575099 2024-09-20T18:58:06.429 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.429 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.429 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:58:06.429 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445166+0000", 2024-09-20T18:58:06.429 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000109335 2024-09-20T18:58:06.429 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.429 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.429 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,5", 2024-09-20T18:58:06.429 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445293+0000", 2024-09-20T18:58:06.430 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000126995 2024-09-20T18:58:06.430 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.430 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.430 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:06.430 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.452767+0000", 2024-09-20T18:58:06.430 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074736009999999999 2024-09-20T18:58:06.430 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.430 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.430 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:06.430 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.452832+0000", 2024-09-20T18:58:06.430 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4817e-05 2024-09-20T18:58:06.430 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.430 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.430 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:58:06.430 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.455135+0000", 2024-09-20T18:58:06.431 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0023034459999999998 2024-09-20T18:58:06.431 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.431 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.431 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:58:06.431 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.455211+0000", 2024-09-20T18:58:06.431 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5896999999999995e-05 2024-09-20T18:58:06.431 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.431 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.431 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:58:06.431 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.455250+0000", 2024-09-20T18:58:06.431 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8532000000000003e-05 2024-09-20T18:58:06.431 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:06.431 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:58:06.431 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:06.432 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.432 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.432 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:25332 7.2 e93/83 ondisk, result = 0)", 2024-09-20T18:58:06.432 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.270341+0000", 2024-09-20T18:58:06.432 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 419.09482745700001, 2024-09-20T18:58:06.432 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.174621416, 2024-09-20T18:58:06.432 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:58:06.432 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:58:06.432 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:58:06.432 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:58:06.432 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.432 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:58:06.432 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270341+0000", 2024-09-20T18:58:06.433 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:06.433 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.433 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.433 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:58:06.433 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270341+0000", 2024-09-20T18:58:06.433 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:06.433 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.433 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.433 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:58:06.433 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270341+0000", 2024-09-20T18:58:06.433 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6000000000000004e-07 2024-09-20T18:58:06.433 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.433 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.433 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:58:06.433 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270345+0000", 2024-09-20T18:58:06.433 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.444e-06 2024-09-20T18:58:06.434 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.434 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.434 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:58:06.434 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270353+0000", 2024-09-20T18:58:06.434 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5580000000000001e-06 2024-09-20T18:58:06.434 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.434 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.434 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:58:06.434 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270359+0000", 2024-09-20T18:58:06.434 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7459999999999997e-06 2024-09-20T18:58:06.434 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.434 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.434 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:06.434 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444944+0000", 2024-09-20T18:58:06.434 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.174585351 2024-09-20T18:58:06.435 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.435 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.435 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:58:06.435 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444950+0000", 2024-09-20T18:58:06.435 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0530000000000003e-06 2024-09-20T18:58:06.435 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.435 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.435 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:58:06.435 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444962+0000", 2024-09-20T18:58:06.435 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1704e-05 2024-09-20T18:58:06.435 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:06.435 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:58:06.435 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:06.435 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.436 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.436 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:25333 7.2 e93/83 ondisk, result = 0)", 2024-09-20T18:58:06.436 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.270458+0000", 2024-09-20T18:58:06.436 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 419.09471006500002, 2024-09-20T18:58:06.436 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17453908100000001, 2024-09-20T18:58:06.436 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:58:06.436 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:58:06.436 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:58:06.436 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:58:06.436 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.436 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:58:06.436 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270458+0000", 2024-09-20T18:58:06.436 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:06.436 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.437 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.437 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:58:06.437 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270458+0000", 2024-09-20T18:58:06.437 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:06.437 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.437 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.437 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:58:06.437 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270458+0000", 2024-09-20T18:58:06.437 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3499999999999996e-07 2024-09-20T18:58:06.437 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.437 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.437 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:58:06.437 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270462+0000", 2024-09-20T18:58:06.438 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1499999999999999e-06 2024-09-20T18:58:06.438 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.438 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.438 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:58:06.438 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270469+0000", 2024-09-20T18:58:06.438 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.695e-06 2024-09-20T18:58:06.438 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.438 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.438 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:58:06.438 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270475+0000", 2024-09-20T18:58:06.438 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5470000000000004e-06 2024-09-20T18:58:06.438 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.438 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.438 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:06.438 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444981+0000", 2024-09-20T18:58:06.439 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17450622299999999 2024-09-20T18:58:06.439 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.439 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.439 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:58:06.439 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444987+0000", 2024-09-20T18:58:06.439 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5219999999999998e-06 2024-09-20T18:58:06.439 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.439 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.439 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:58:06.439 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444997+0000", 2024-09-20T18:58:06.439 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0409e-05 2024-09-20T18:58:06.439 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:06.439 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:58:06.439 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:06.439 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.439 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.440 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:25332 7.2 e93/83 ondisk, result = 0)", 2024-09-20T18:58:06.440 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.270547+0000", 2024-09-20T18:58:06.440 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 419.094621383, 2024-09-20T18:58:06.440 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.174485849, 2024-09-20T18:58:06.440 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:58:06.440 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:58:06.440 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:58:06.440 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:58:06.440 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.440 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:58:06.440 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270547+0000", 2024-09-20T18:58:06.440 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:06.440 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.441 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.441 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:58:06.441 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270547+0000", 2024-09-20T18:58:06.441 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:06.441 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.441 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.441 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:58:06.441 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270547+0000", 2024-09-20T18:58:06.441 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7599999999999997e-07 2024-09-20T18:58:06.441 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.441 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.441 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:58:06.441 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270550+0000", 2024-09-20T18:58:06.441 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7470000000000001e-06 2024-09-20T18:58:06.441 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.442 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.442 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:58:06.442 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270557+0000", 2024-09-20T18:58:06.442 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2620000000000001e-06 2024-09-20T18:58:06.442 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.442 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.442 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:58:06.442 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270562+0000", 2024-09-20T18:58:06.442 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1839999999999998e-06 2024-09-20T18:58:06.442 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.442 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.442 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:06.442 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445016+0000", 2024-09-20T18:58:06.442 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17445396699999999 2024-09-20T18:58:06.443 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.443 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.443 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:58:06.443 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445022+0000", 2024-09-20T18:58:06.443 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7559999999999996e-06 2024-09-20T18:58:06.443 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.443 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.443 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:58:06.443 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445032+0000", 2024-09-20T18:58:06.443 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0457e-05 2024-09-20T18:58:06.443 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:06.443 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:58:06.443 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:06.443 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.443 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.444 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:25343 7.6 e93/83 7:61f2dd3b:::benchmark_data_smithi083_70643_object25342:head v 93'1583, mlcod=93'1579)", 2024-09-20T18:58:06.444 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.271493+0000", 2024-09-20T18:58:06.444 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 419.09367531499998, 2024-09-20T18:58:06.444 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17491087999999999, 2024-09-20T18:58:06.444 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:58:06.444 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:58:06.444 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:06.444 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:58:06.444 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.444 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:58:06.444 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271493+0000", 2024-09-20T18:58:06.444 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:06.444 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.444 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.445 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:58:06.445 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271493+0000", 2024-09-20T18:58:06.445 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:06.445 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.445 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.445 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:58:06.445 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271493+0000", 2024-09-20T18:58:06.445 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8400000000000004e-07 2024-09-20T18:58:06.445 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.445 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.445 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:58:06.445 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271513+0000", 2024-09-20T18:58:06.445 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0126999999999999e-05 2024-09-20T18:58:06.445 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.445 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.446 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:58:06.446 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443960+0000", 2024-09-20T18:58:06.446 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.172446983 2024-09-20T18:58:06.446 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.446 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.446 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:58:06.446 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443970+0000", 2024-09-20T18:58:06.446 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5119999999999997e-06 2024-09-20T18:58:06.446 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.446 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.446 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:06.446 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444547+0000", 2024-09-20T18:58:06.446 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00057698700000000005 2024-09-20T18:58:06.446 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.447 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.447 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:58:06.447 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444578+0000", 2024-09-20T18:58:06.447 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0956e-05 2024-09-20T18:58:06.447 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.447 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.447 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:58:06.447 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.446361+0000", 2024-09-20T18:58:06.447 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001783473 2024-09-20T18:58:06.447 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.447 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.447 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:58:06.447 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.446404+0000", 2024-09-20T18:58:06.447 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2157999999999999e-05 2024-09-20T18:58:06.447 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:06.448 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:58:06.448 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:06.448 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.448 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.448 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.24955.0:218155 8.7 e233/213 8:ebcbd19d:::benchmark_data_smithi083_97422_object102064:head v 233'13685, mlcod=233'13682)", 2024-09-20T18:58:06.448 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:58:01.869876+0000", 2024-09-20T18:58:06.448 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.4952918200000003, 2024-09-20T18:58:06.448 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.218638782, 2024-09-20T18:58:06.448 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:58:06.448 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:58:06.448 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:06.448 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:58:06.448 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.448 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:58:06.449 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.869876+0000", 2024-09-20T18:58:06.449 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:06.449 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.449 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.449 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:58:06.449 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.869876+0000", 2024-09-20T18:58:06.449 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:06.449 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.449 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.449 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:58:06.449 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.869877+0000", 2024-09-20T18:58:06.449 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9999999999999996e-07 2024-09-20T18:58:06.449 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.449 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.449 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:58:06.450 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.869883+0000", 2024-09-20T18:58:06.450 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7819999999999999e-06 2024-09-20T18:58:06.450 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.450 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.450 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:58:06.450 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.869891+0000", 2024-09-20T18:58:06.450 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6289999999999999e-06 2024-09-20T18:58:06.450 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.450 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.450 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:58:06.450 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.869897+0000", 2024-09-20T18:58:06.450 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.049e-06 2024-09-20T18:58:06.450 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.450 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.451 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:06.451 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.874096+0000", 2024-09-20T18:58:06.451 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0041987489999999999 2024-09-20T18:58:06.451 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.451 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.451 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:58:06.451 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.874117+0000", 2024-09-20T18:58:06.451 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0361999999999999e-05 2024-09-20T18:58:06.451 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.451 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.451 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:58:06.451 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.088464+0000", 2024-09-20T18:58:06.452 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.21434697999999999 2024-09-20T18:58:06.452 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.452 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.452 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:58:06.452 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.088515+0000", 2024-09-20T18:58:06.452 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1431000000000003e-05 2024-09-20T18:58:06.452 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:06.452 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:58:06.452 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:06.452 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.452 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.452 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:218167 8.f 8:f8df719b:::benchmark_data_smithi083_97422_object102076:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-20T18:58:06.452 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:58:01.876042+0000", 2024-09-20T18:58:06.453 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.4891255269999997, 2024-09-20T18:58:06.453 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.212910085, 2024-09-20T18:58:06.453 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:58:06.453 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:58:06.453 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:06.453 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:58:06.453 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:58:06.453 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:58:06.453 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 218167 2024-09-20T18:58:06.453 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.453 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:58:06.453 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.453 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:58:06.454 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.876042+0000", 2024-09-20T18:58:06.454 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:06.455 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:06.455 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.455 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.455 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:58:06.455 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.876042+0000", 2024-09-20T18:58:06.455 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:06.455 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.455 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.455 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:58:06.455 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.876044+0000", 2024-09-20T18:58:06.456 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1340000000000001e-06 2024-09-20T18:58:06.456 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.456 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.456 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:58:06.456 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.876047+0000", 2024-09-20T18:58:06.456 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9969999999999999e-06 2024-09-20T18:58:06.456 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.456 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.456 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:58:06.456 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.876054+0000", 2024-09-20T18:58:06.456 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1620000000000002e-06 2024-09-20T18:58:06.456 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.456 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.457 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:58:06.457 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.876059+0000", 2024-09-20T18:58:06.457 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0610000000000003e-06 2024-09-20T18:58:06.457 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.457 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.457 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:06.457 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.876376+0000", 2024-09-20T18:58:06.457 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00031752499999999998 2024-09-20T18:58:06.457 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.457 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.457 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:58:06.457 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.876544+0000", 2024-09-20T18:58:06.458 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000167732 2024-09-20T18:58:06.458 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.458 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.458 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T18:58:06.458 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.876643+0000", 2024-09-20T18:58:06.458 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9339000000000006e-05 2024-09-20T18:58:06.458 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.458 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.458 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:06.458 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.885374+0000", 2024-09-20T18:58:06.458 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0087305390000000007 2024-09-20T18:58:06.458 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.458 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.458 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:06.458 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.081897+0000", 2024-09-20T18:58:06.459 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.19652308099999999 2024-09-20T18:58:06.459 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.459 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.459 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:58:06.459 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.088624+0000", 2024-09-20T18:58:06.459 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0067268320000000003 2024-09-20T18:58:06.459 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.459 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.459 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:58:06.459 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.088808+0000", 2024-09-20T18:58:06.459 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018438999999999999 2024-09-20T18:58:06.459 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.459 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.459 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:58:06.460 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.088953+0000", 2024-09-20T18:58:06.460 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014429300000000001 2024-09-20T18:58:06.460 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:06.460 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:58:06.460 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:06.460 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.460 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.460 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:218168 8.f 8:fa798818:::benchmark_data_smithi083_97422_object102077:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-20T18:58:06.460 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:58:01.879290+0000", 2024-09-20T18:58:06.460 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.4858783249999998, 2024-09-20T18:58:06.460 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.209845379, 2024-09-20T18:58:06.460 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:58:06.460 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:58:06.460 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:06.461 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:58:06.461 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:58:06.461 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:58:06.461 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 218168 2024-09-20T18:58:06.461 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.461 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:58:06.461 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.461 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:58:06.461 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.879290+0000", 2024-09-20T18:58:06.461 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:06.461 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.461 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.462 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:58:06.462 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.879290+0000", 2024-09-20T18:58:06.462 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:06.462 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.462 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.462 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:58:06.462 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.879291+0000", 2024-09-20T18:58:06.462 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4890000000000001e-06 2024-09-20T18:58:06.462 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.462 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.462 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:58:06.462 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.879296+0000", 2024-09-20T18:58:06.462 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9479999999999999e-06 2024-09-20T18:58:06.462 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.463 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.463 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:58:06.463 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.879306+0000", 2024-09-20T18:58:06.463 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6749999999999997e-06 2024-09-20T18:58:06.463 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.463 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.463 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:58:06.463 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.879313+0000", 2024-09-20T18:58:06.463 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1570000000000003e-06 2024-09-20T18:58:06.463 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.463 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.463 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:06.463 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.879650+0000", 2024-09-20T18:58:06.463 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00033674399999999999 2024-09-20T18:58:06.463 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.464 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.464 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:58:06.464 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.879933+0000", 2024-09-20T18:58:06.464 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00028346099999999999 2024-09-20T18:58:06.464 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.464 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.464 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T18:58:06.464 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.880034+0000", 2024-09-20T18:58:06.464 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000100523 2024-09-20T18:58:06.464 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.464 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.464 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:06.464 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.086134+0000", 2024-09-20T18:58:06.464 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.20610060399999999 2024-09-20T18:58:06.464 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.464 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.465 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:06.465 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.088528+0000", 2024-09-20T18:58:06.465 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0023938850000000001 2024-09-20T18:58:06.465 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.465 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.465 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:58:06.465 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.089043+0000", 2024-09-20T18:58:06.465 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000514921 2024-09-20T18:58:06.465 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.465 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.465 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:58:06.465 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.089099+0000", 2024-09-20T18:58:06.465 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5773000000000003e-05 2024-09-20T18:58:06.465 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.465 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.465 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:58:06.466 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.089135+0000", 2024-09-20T18:58:06.466 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6198999999999997e-05 2024-09-20T18:58:06.466 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:06.466 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:58:06.466 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:06.466 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.466 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.466 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:218173 8.f 8:fc626e9a:::benchmark_data_smithi083_97422_object102082:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-20T18:58:06.466 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:58:01.879894+0000", 2024-09-20T18:58:06.466 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.4852735069999996, 2024-09-20T18:58:06.466 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.22879112400000001, 2024-09-20T18:58:06.466 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:58:06.466 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:58:06.466 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:06.466 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:58:06.466 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:58:06.467 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:58:06.467 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 218173 2024-09-20T18:58:06.467 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.467 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:58:06.467 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.467 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:58:06.467 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.879894+0000", 2024-09-20T18:58:06.467 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:06.467 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.467 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.467 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:58:06.467 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.879894+0000", 2024-09-20T18:58:06.467 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:06.467 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.467 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.468 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:58:06.468 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.879895+0000", 2024-09-20T18:58:06.468 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3600000000000003e-07 2024-09-20T18:58:06.468 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.468 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.468 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:58:06.468 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.879899+0000", 2024-09-20T18:58:06.468 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3500000000000001e-06 2024-09-20T18:58:06.468 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.468 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.468 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:58:06.468 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.879907+0000", 2024-09-20T18:58:06.468 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.943e-06 2024-09-20T18:58:06.468 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.468 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.469 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:58:06.469 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.879913+0000", 2024-09-20T18:58:06.469 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3910000000000003e-06 2024-09-20T18:58:06.469 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.469 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.469 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:06.469 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.095128+0000", 2024-09-20T18:58:06.469 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.21521499199999999 2024-09-20T18:58:06.469 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.469 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.469 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:58:06.469 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.095243+0000", 2024-09-20T18:58:06.469 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011465599999999999 2024-09-20T18:58:06.469 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.469 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.470 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T18:58:06.470 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.095357+0000", 2024-09-20T18:58:06.470 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000114868 2024-09-20T18:58:06.470 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.470 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.470 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:06.470 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.104639+0000", 2024-09-20T18:58:06.470 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0092811200000000003 2024-09-20T18:58:06.470 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.470 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.470 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:58:06.470 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.105329+0000", 2024-09-20T18:58:06.470 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000690195 2024-09-20T18:58:06.470 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.470 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.471 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:06.471 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.108593+0000", 2024-09-20T18:58:06.471 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003263967 2024-09-20T18:58:06.471 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.471 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.471 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:58:06.471 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.108646+0000", 2024-09-20T18:58:06.471 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3495999999999997e-05 2024-09-20T18:58:06.471 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.471 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.471 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:58:06.471 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.108686+0000", 2024-09-20T18:58:06.471 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9409999999999997e-05 2024-09-20T18:58:06.471 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:06.471 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:58:06.472 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:06.472 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.472 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.472 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.24955.0:218167 8.f e233/160 ondisk, result = 0)", 2024-09-20T18:58:06.472 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:58:01.881107+0000", 2024-09-20T18:58:06.472 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.4840610820000002, 2024-09-20T18:58:06.472 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.20784897499999999, 2024-09-20T18:58:06.472 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:58:06.472 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:58:06.472 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:58:06.472 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:58:06.472 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.472 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:58:06.472 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.881107+0000", 2024-09-20T18:58:06.472 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:06.473 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.473 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.473 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:58:06.473 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.881107+0000", 2024-09-20T18:58:06.473 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:06.473 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.473 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.473 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:58:06.473 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.881108+0000", 2024-09-20T18:58:06.473 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7400000000000002e-07 2024-09-20T18:58:06.473 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.473 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.473 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:58:06.473 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.881112+0000", 2024-09-20T18:58:06.474 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5260000000000004e-06 2024-09-20T18:58:06.474 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.474 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.474 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:58:06.474 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.881121+0000", 2024-09-20T18:58:06.474 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0469999999999999e-06 2024-09-20T18:58:06.474 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.474 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.474 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:58:06.474 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.881127+0000", 2024-09-20T18:58:06.474 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9970000000000004e-06 2024-09-20T18:58:06.474 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.474 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.475 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:06.475 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.081884+0000", 2024-09-20T18:58:06.475 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.200756394 2024-09-20T18:58:06.475 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.475 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.475 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:58:06.475 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.081891+0000", 2024-09-20T18:58:06.475 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.6109999999999996e-06 2024-09-20T18:58:06.475 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.475 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.475 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:58:06.475 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.088956+0000", 2024-09-20T18:58:06.475 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0070646260000000001 2024-09-20T18:58:06.476 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:06.476 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:58:06.476 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:06.476 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.476 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.476 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.24955.0:218171 8.7 e233/213 8:e3fe301b:::benchmark_data_smithi083_97422_object102080:head v 233'13686, mlcod=233'13682)", 2024-09-20T18:58:06.476 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:58:01.882984+0000", 2024-09-20T18:58:06.476 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.4821836929999996, 2024-09-20T18:58:06.476 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.21396309799999999, 2024-09-20T18:58:06.476 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:58:06.476 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:58:06.476 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:06.476 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:58:06.476 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.477 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:58:06.477 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.882984+0000", 2024-09-20T18:58:06.477 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:06.477 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.477 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.477 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:58:06.477 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.882984+0000", 2024-09-20T18:58:06.477 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:06.477 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.477 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.477 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:58:06.477 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.882985+0000", 2024-09-20T18:58:06.477 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5799999999999999e-07 2024-09-20T18:58:06.478 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.478 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.478 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:58:06.478 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.882990+0000", 2024-09-20T18:58:06.478 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0919999999999997e-06 2024-09-20T18:58:06.478 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.478 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.478 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:58:06.478 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.882997+0000", 2024-09-20T18:58:06.478 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9269999999999999e-06 2024-09-20T18:58:06.478 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.478 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.478 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:58:06.478 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.883003+0000", 2024-09-20T18:58:06.479 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2319999999999998e-06 2024-09-20T18:58:06.479 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.479 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.479 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:06.479 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.080830+0000", 2024-09-20T18:58:06.479 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.197826429 2024-09-20T18:58:06.479 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.479 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.479 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:58:06.479 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.080862+0000", 2024-09-20T18:58:06.479 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2858999999999998e-05 2024-09-20T18:58:06.479 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.479 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.479 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:58:06.480 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.096909+0000", 2024-09-20T18:58:06.480 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.016046662 2024-09-20T18:58:06.480 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.480 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.480 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:58:06.480 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.096947+0000", 2024-09-20T18:58:06.480 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8238999999999998e-05 2024-09-20T18:58:06.480 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:06.480 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:58:06.480 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:06.480 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.480 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.480 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.24955.0:218179 8.7 e233/213 8:e44c0f9a:::benchmark_data_smithi083_97422_object102088:head v 233'13687, mlcod=233'13683)", 2024-09-20T18:58:06.480 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:58:01.885843+0000", 2024-09-20T18:58:06.480 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.4793254679999999, 2024-09-20T18:58:06.481 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.211128274, 2024-09-20T18:58:06.481 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:58:06.481 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:58:06.481 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:06.481 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:58:06.481 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.481 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:58:06.481 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.885843+0000", 2024-09-20T18:58:06.481 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:06.481 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.481 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.481 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:58:06.481 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.885843+0000", 2024-09-20T18:58:06.481 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:06.482 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.482 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.482 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:58:06.482 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.885843+0000", 2024-09-20T18:58:06.482 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5999999999999991e-07 2024-09-20T18:58:06.482 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.482 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.482 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:58:06.482 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.885858+0000", 2024-09-20T18:58:06.482 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4286e-05 2024-09-20T18:58:06.482 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.482 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.482 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:58:06.482 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.080663+0000", 2024-09-20T18:58:06.483 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.19480504600000001 2024-09-20T18:58:06.483 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.483 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.483 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:58:06.483 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.080676+0000", 2024-09-20T18:58:06.483 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3417999999999999e-05 2024-09-20T18:58:06.483 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.483 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.483 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:06.483 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.083322+0000", 2024-09-20T18:58:06.483 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0026456880000000002 2024-09-20T18:58:06.483 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.483 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.483 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:58:06.484 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.083341+0000", 2024-09-20T18:58:06.484 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9539999999999999e-05 2024-09-20T18:58:06.484 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.484 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.484 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:58:06.484 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.096951+0000", 2024-09-20T18:58:06.484 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.013609677000000001 2024-09-20T18:58:06.484 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.484 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.484 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:58:06.484 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.096971+0000", 2024-09-20T18:58:06.484 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9658999999999999e-05 2024-09-20T18:58:06.484 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:06.484 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:58:06.485 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:06.485 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.485 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.485 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.24955.0:218168 8.f e233/160 ondisk, result = 0)", 2024-09-20T18:58:06.485 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:58:01.888899+0000", 2024-09-20T18:58:06.485 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 4.4762690110000003, 2024-09-20T18:58:06.485 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.19963995100000001, 2024-09-20T18:58:06.485 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:58:06.485 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:58:06.485 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:58:06.485 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:58:06.485 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.485 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:58:06.485 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.888899+0000", 2024-09-20T18:58:06.486 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:06.486 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.486 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.486 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:58:06.486 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.888899+0000", 2024-09-20T18:58:06.486 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:06.486 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.486 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.486 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:58:06.486 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.888900+0000", 2024-09-20T18:58:06.486 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0400000000000005e-07 2024-09-20T18:58:06.486 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.486 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.487 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:58:06.487 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.888906+0000", 2024-09-20T18:58:06.487 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.784e-06 2024-09-20T18:58:06.487 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.487 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.487 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:58:06.487 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.080754+0000", 2024-09-20T18:58:06.487 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.191848466 2024-09-20T18:58:06.487 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.487 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.487 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:58:06.487 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.080764+0000", 2024-09-20T18:58:06.487 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5359999999999998e-06 2024-09-20T18:58:06.488 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.488 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.488 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:06.488 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.088464+0000", 2024-09-20T18:58:06.488 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0077003829999999999 2024-09-20T18:58:06.488 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.488 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.488 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:58:06.488 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.088499+0000", 2024-09-20T18:58:06.488 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.485e-05 2024-09-20T18:58:06.488 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:06.488 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:06.488 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:58:06.488 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.088539+0000", 2024-09-20T18:58:06.489 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0027999999999997e-05 2024-09-20T18:58:06.489 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:06.489 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:58:06.489 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:06.489 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:06.489 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:58:06.489 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T18:58:06.514 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:06.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:06.514 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:06.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:06 smithi142 ceph-mon[32508]: pgmap v397: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 908 MiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 946 B/s rd, 0 B/s wr, 902 op/s; 1945/44133 objects degraded (4.407%); 0 B/s, 25 objects/s recovering 2024-09-20T18:58:06.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:06 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 1945/44133 objects degraded (4.407%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T18:58:06.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:06 smithi142 ceph-mon[32508]: 4.7 scrub starts 2024-09-20T18:58:06.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:06 smithi142 ceph-mon[32508]: 4.7 scrub ok 2024-09-20T18:58:06.616 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:06.659 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:06.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:06.660 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:06.761 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:06.801 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:06.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:06.802 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:06.902 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:06.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:06.940 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:06.940 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:06.944 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T18:58:07.041 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:07.064 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:07.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:07.065 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:07.166 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:07.189 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:07.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:07.190 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:07.220 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:58:06 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:58:07.285 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:07 smithi083 ceph-mon[29421]: 6.a scrub starts 2024-09-20T18:58:07.285 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:07 smithi083 ceph-mon[29421]: 6.a scrub ok 2024-09-20T18:58:07.285 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:07 smithi083 ceph-mon[29421]: from='client.25376 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:58:07.285 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:07 smithi083 ceph-mon[29421]: 4.4 scrub starts 2024-09-20T18:58:07.285 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:07 smithi083 ceph-mon[29421]: 4.4 scrub ok 2024-09-20T18:58:07.286 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":397,"stamp":"2024-09-20T18:58:04.957054+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":951589427,"num_objects":14711,"num_object_clones":0,"num_object_copies":44133,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14711,"num_whiteouts":0,"num_read":1335,"num_read_kb":1192,"num_write":218215,"num_write_kb":7431306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26206,"num_bytes_recovered":1676479433,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74955,"ondisk_log_size":74955,"up":444,"acting":443,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":4923380,"kb_used_data":2855620,"kb_used_omap":10,"kb_used_meta":2067381,"kb_avail":651292684,"statfs":{"total":671965249536,"available":666923708416,"internally_reserved":0,"allocated":2924154880,"data_stored":2920767207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11158,"internal_metadata":2116998250},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,7,4,4,0,0,0,4,6],"upper_bound":512},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-639762432,"num_objects":-9762,"num_object_clones":0,"num_object_copies":-29286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-280,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-9762,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":9762,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.823214"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":385,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.160096+0000","last_change":"2024-09-20T18:55:28.037494+0000","last_active":"2024-09-20T18:57:54.160096+0000","last_peered":"2024-09-20T18:57:54.160096+0000","last_clean":"2024-09-20T18:57:54.160096+0000","last_became_active":"2024-09-20T18:49:45.943201+0000","last_became_peered":"2024-09-20T18:49:45.943201+0000","last_unstale":"2024-09-20T18:57:54.160096+0000","last_undegraded":"2024-09-20T18:57:54.160096+0000","last_fullsized":"2024-09-20T18:57:54.160096+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:28.037449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:28.037449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":432,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.133704+0000","last_change":"2024-09-20T18:55:28.838729+0000","last_active":"2024-09-20T18:57:54.133704+0000","last_peered":"2024-09-20T18:57:54.133704+0000","last_clean":"2024-09-20T18:57:54.133704+0000","last_became_active":"2024-09-20T18:55:28.819637+0000","last_became_peered":"2024-09-20T18:55:28.819637+0000","last_unstale":"2024-09-20T18:57:54.133704+0000","last_undegraded":"2024-09-20T18:57:54.133704+0000","last_fullsized":"2024-09-20T18:57:54.133704+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:50.231620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:50.231620+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":386,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.162837+0000","last_change":"2024-09-20T18:57:49.016078+0000","last_active":"2024-09-20T18:57:54.162837+0000","last_peered":"2024-09-20T18:57:54.162837+0000","last_clean":"2024-09-20T18:57:54.162837+0000","last_became_active":"2024-09-20T18:49:49.959514+0000","last_became_peered":"2024-09-20T18:49:49.959514+0000","last_unstale":"2024-09-20T18:57:54.162837+0000","last_undegraded":"2024-09-20T18:57:54.162837+0000","last_fullsized":"2024-09-20T18:57:54.162837+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:49.016006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:49.016006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:13.297987+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":374,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.134096+0000","last_change":"2024-09-20T18:54:15.221404+0000","last_active":"2024-09-20T18:57:54.134096+0000","last_peered":"2024-09-20T18:57:54.134096+0000","last_clean":"2024-09-20T18:57:54.134096+0000","last_became_active":"2024-09-20T18:49:51.969421+0000","last_became_peered":"2024-09-20T18:49:51.969421+0000","last_unstale":"2024-09-20T18:57:54.134096+0000","last_undegraded":"2024-09-20T18:57:54.134096+0000","last_fullsized":"2024-09-20T18:57:54.134096+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:15.221342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:15.221342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":381,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.143633+0000","last_change":"2024-09-20T18:55:28.822034+0000","last_active":"2024-09-20T18:57:54.143633+0000","last_peered":"2024-09-20T18:57:54.143633+0000","last_clean":"2024-09-20T18:57:54.143633+0000","last_became_active":"2024-09-20T18:55:28.821717+0000","last_became_peered":"2024-09-20T18:55:28.821717+0000","last_unstale":"2024-09-20T18:57:54.143633+0000","last_undegraded":"2024-09-20T18:57:54.143633+0000","last_fullsized":"2024-09-20T18:57:54.143633+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.216188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.216188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":386,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.142788+0000","last_change":"2024-09-20T18:57:22.694362+0000","last_active":"2024-09-20T18:57:54.142788+0000","last_peered":"2024-09-20T18:57:54.142788+0000","last_clean":"2024-09-20T18:57:54.142788+0000","last_became_active":"2024-09-20T18:55:28.824826+0000","last_became_peered":"2024-09-20T18:55:28.824826+0000","last_unstale":"2024-09-20T18:57:54.142788+0000","last_undegraded":"2024-09-20T18:57:54.142788+0000","last_fullsized":"2024-09-20T18:57:54.142788+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:22.694295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:22.694295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:33.473592+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":399,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.141935+0000","last_change":"2024-09-20T18:57:13.774827+0000","last_active":"2024-09-20T18:57:54.141935+0000","last_peered":"2024-09-20T18:57:54.141935+0000","last_clean":"2024-09-20T18:57:54.141935+0000","last_became_active":"2024-09-20T18:49:47.952150+0000","last_became_peered":"2024-09-20T18:49:47.952150+0000","last_unstale":"2024-09-20T18:57:54.141935+0000","last_undegraded":"2024-09-20T18:57:54.141935+0000","last_fullsized":"2024-09-20T18:57:54.141935+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:57:13.774305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:13.774305+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:25.245744+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":377,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.133696+0000","last_change":"2024-09-20T18:55:28.820753+0000","last_active":"2024-09-20T18:57:54.133696+0000","last_peered":"2024-09-20T18:57:54.133696+0000","last_clean":"2024-09-20T18:57:54.133696+0000","last_became_active":"2024-09-20T18:55:28.820362+0000","last_became_peered":"2024-09-20T18:55:28.820362+0000","last_unstale":"2024-09-20T18:57:54.133696+0000","last_undegraded":"2024-09-20T18:57:54.133696+0000","last_fullsized":"2024-09-20T18:57:54.133696+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:11.246922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:11.246922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"59'1","reported_seq":410,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.146699+0000","last_change":"2024-09-20T18:55:29.340140+0000","last_active":"2024-09-20T18:57:54.146699+0000","last_peered":"2024-09-20T18:57:54.146699+0000","last_clean":"2024-09-20T18:57:54.146699+0000","last_became_active":"2024-09-20T18:55:28.833386+0000","last_became_peered":"2024-09-20T18:55:28.833386+0000","last_unstale":"2024-09-20T18:57:54.146699+0000","last_undegraded":"2024-09-20T18:57:54.146699+0000","last_fullsized":"2024-09-20T18:57:54.146699+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:55:29.340083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:29.340083+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":398,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.134401+0000","last_change":"2024-09-20T18:54:53.241889+0000","last_active":"2024-09-20T18:57:54.134401+0000","last_peered":"2024-09-20T18:57:54.134401+0000","last_clean":"2024-09-20T18:57:54.134401+0000","last_became_active":"2024-09-20T18:49:47.947448+0000","last_became_peered":"2024-09-20T18:49:47.947448+0000","last_unstale":"2024-09-20T18:57:54.134401+0000","last_undegraded":"2024-09-20T18:57:54.134401+0000","last_fullsized":"2024-09-20T18:57:54.134401+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:53.241802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:53.241802+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":377,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.145999+0000","last_change":"2024-09-20T18:57:08.539466+0000","last_active":"2024-09-20T18:57:54.145999+0000","last_peered":"2024-09-20T18:57:54.145999+0000","last_clean":"2024-09-20T18:57:54.145999+0000","last_became_active":"2024-09-20T18:49:49.961188+0000","last_became_peered":"2024-09-20T18:49:49.961188+0000","last_unstale":"2024-09-20T18:57:54.145999+0000","last_undegraded":"2024-09-20T18:57:54.145999+0000","last_fullsized":"2024-09-20T18:57:54.145999+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:08.539418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:08.539418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:13.102215+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":349,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.161262+0000","last_change":"2024-09-20T18:49:51.966957+0000","last_active":"2024-09-20T18:57:54.161262+0000","last_peered":"2024-09-20T18:57:54.161262+0000","last_clean":"2024-09-20T18:57:54.161262+0000","last_became_active":"2024-09-20T18:49:51.966715+0000","last_became_peered":"2024-09-20T18:49:51.966715+0000","last_unstale":"2024-09-20T18:57:54.161262+0000","last_undegraded":"2024-09-20T18:57:54.161262+0000","last_fullsized":"2024-09-20T18:57:54.161262+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":377,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.148831+0000","last_change":"2024-09-20T18:55:15.252839+0000","last_active":"2024-09-20T18:57:54.148831+0000","last_peered":"2024-09-20T18:57:54.148831+0000","last_clean":"2024-09-20T18:57:54.148831+0000","last_became_active":"2024-09-20T18:49:45.944703+0000","last_became_peered":"2024-09-20T18:49:45.944703+0000","last_unstale":"2024-09-20T18:57:54.148831+0000","last_undegraded":"2024-09-20T18:57:54.148831+0000","last_fullsized":"2024-09-20T18:57:54.148831+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:15.252782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:15.252782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":402,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.133995+0000","last_change":"2024-09-20T18:51:26.494194+0000","last_active":"2024-09-20T18:57:54.133995+0000","last_peered":"2024-09-20T18:57:54.133995+0000","last_clean":"2024-09-20T18:57:54.133995+0000","last_became_active":"2024-09-20T18:51:23.813399+0000","last_became_peered":"2024-09-20T18:51:23.813399+0000","last_unstale":"2024-09-20T18:57:54.133995+0000","last_undegraded":"2024-09-20T18:57:54.133995+0000","last_fullsized":"2024-09-20T18:57:54.133995+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-20T18:51:26.494118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:26.494118+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:31.887636+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":369,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.147940+0000","last_change":"2024-09-20T18:52:38.799542+0000","last_active":"2024-09-20T18:57:54.147940+0000","last_peered":"2024-09-20T18:57:54.147940+0000","last_clean":"2024-09-20T18:57:54.147940+0000","last_became_active":"2024-09-20T18:49:49.959696+0000","last_became_peered":"2024-09-20T18:49:49.959696+0000","last_unstale":"2024-09-20T18:57:54.147940+0000","last_undegraded":"2024-09-20T18:57:54.147940+0000","last_fullsized":"2024-09-20T18:57:54.147940+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:38.799487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:38.799487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":373,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.146587+0000","last_change":"2024-09-20T18:57:13.587827+0000","last_active":"2024-09-20T18:57:54.146587+0000","last_peered":"2024-09-20T18:57:54.146587+0000","last_clean":"2024-09-20T18:57:54.146587+0000","last_became_active":"2024-09-20T18:49:51.970663+0000","last_became_peered":"2024-09-20T18:49:51.970663+0000","last_unstale":"2024-09-20T18:57:54.146587+0000","last_undegraded":"2024-09-20T18:57:54.146587+0000","last_fullsized":"2024-09-20T18:57:54.146587+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:13.587774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:13.587774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:15.272228+0000","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":382,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:58:00.906697+0000","last_change":"2024-09-20T18:58:00.906697+0000","last_active":"2024-09-20T18:58:00.906697+0000","last_peered":"2024-09-20T18:58:00.906697+0000","last_clean":"2024-09-20T18:58:00.906697+0000","last_became_active":"2024-09-20T18:55:28.827435+0000","last_became_peered":"2024-09-20T18:55:28.827435+0000","last_unstale":"2024-09-20T18:58:00.906697+0000","last_undegraded":"2024-09-20T18:58:00.906697+0000","last_fullsized":"2024-09-20T18:58:00.906697+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:00.906652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:00.906652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:23.434725+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":377,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.162013+0000","last_change":"2024-09-20T18:54:23.984024+0000","last_active":"2024-09-20T18:57:54.162013+0000","last_peered":"2024-09-20T18:57:54.162013+0000","last_clean":"2024-09-20T18:57:54.162013+0000","last_became_active":"2024-09-20T18:49:45.950480+0000","last_became_peered":"2024-09-20T18:49:45.950480+0000","last_unstale":"2024-09-20T18:57:54.162013+0000","last_undegraded":"2024-09-20T18:57:54.162013+0000","last_fullsized":"2024-09-20T18:57:54.162013+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:23.983958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:23.983958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":400,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.165114+0000","last_change":"2024-09-20T18:53:26.762207+0000","last_active":"2024-09-20T18:57:54.165114+0000","last_peered":"2024-09-20T18:57:54.165114+0000","last_clean":"2024-09-20T18:57:54.165114+0000","last_became_active":"2024-09-20T18:49:47.950939+0000","last_became_peered":"2024-09-20T18:49:47.950939+0000","last_unstale":"2024-09-20T18:57:54.165114+0000","last_undegraded":"2024-09-20T18:57:54.165114+0000","last_fullsized":"2024-09-20T18:57:54.165114+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:53:26.762131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:26.762131+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":370,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.166553+0000","last_change":"2024-09-20T18:57:14.941518+0000","last_active":"2024-09-20T18:57:54.166553+0000","last_peered":"2024-09-20T18:57:54.166553+0000","last_clean":"2024-09-20T18:57:54.166553+0000","last_became_active":"2024-09-20T18:49:49.958518+0000","last_became_peered":"2024-09-20T18:49:49.958518+0000","last_unstale":"2024-09-20T18:57:54.166553+0000","last_undegraded":"2024-09-20T18:57:54.166553+0000","last_fullsized":"2024-09-20T18:57:54.166553+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:14.941458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:14.941458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:37.988570+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":373,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.144359+0000","last_change":"2024-09-20T18:55:28.833468+0000","last_active":"2024-09-20T18:57:54.144359+0000","last_peered":"2024-09-20T18:57:54.144359+0000","last_clean":"2024-09-20T18:57:54.144359+0000","last_became_active":"2024-09-20T18:55:28.833133+0000","last_became_peered":"2024-09-20T18:55:28.833133+0000","last_unstale":"2024-09-20T18:57:54.144359+0000","last_undegraded":"2024-09-20T18:57:54.144359+0000","last_fullsized":"2024-09-20T18:57:54.144359+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:54.259174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:54.259174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:30.790542+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"59'1","reported_seq":392,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.134430+0000","last_change":"2024-09-20T18:57:53.918080+0000","last_active":"2024-09-20T18:57:54.134430+0000","last_peered":"2024-09-20T18:57:54.134430+0000","last_clean":"2024-09-20T18:57:54.134430+0000","last_became_active":"2024-09-20T18:49:45.940100+0000","last_became_peered":"2024-09-20T18:49:45.940100+0000","last_unstale":"2024-09-20T18:57:54.134430+0000","last_undegraded":"2024-09-20T18:57:54.134430+0000","last_fullsized":"2024-09-20T18:57:54.134430+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:57:53.918022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:53.918022+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:03.242427+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":389,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.142243+0000","last_change":"2024-09-20T18:57:09.819080+0000","last_active":"2024-09-20T18:57:54.142243+0000","last_peered":"2024-09-20T18:57:54.142243+0000","last_clean":"2024-09-20T18:57:54.142243+0000","last_became_active":"2024-09-20T18:49:47.950909+0000","last_became_peered":"2024-09-20T18:49:47.950909+0000","last_unstale":"2024-09-20T18:57:54.142243+0000","last_undegraded":"2024-09-20T18:57:54.142243+0000","last_fullsized":"2024-09-20T18:57:54.142243+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:57:09.818999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:09.818999+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:17.216750+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":378,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.133946+0000","last_change":"2024-09-20T18:54:44.239273+0000","last_active":"2024-09-20T18:57:54.133946+0000","last_peered":"2024-09-20T18:57:54.133946+0000","last_clean":"2024-09-20T18:57:54.133946+0000","last_became_active":"2024-09-20T18:49:49.959102+0000","last_became_peered":"2024-09-20T18:49:49.959102+0000","last_unstale":"2024-09-20T18:57:54.133946+0000","last_undegraded":"2024-09-20T18:57:54.133946+0000","last_fullsized":"2024-09-20T18:57:54.133946+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:44.239226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.239226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":366,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.148177+0000","last_change":"2024-09-20T18:52:15.826819+0000","last_active":"2024-09-20T18:57:54.148177+0000","last_peered":"2024-09-20T18:57:54.148177+0000","last_clean":"2024-09-20T18:57:54.148177+0000","last_became_active":"2024-09-20T18:49:51.970058+0000","last_became_peered":"2024-09-20T18:49:51.970058+0000","last_unstale":"2024-09-20T18:57:54.148177+0000","last_undegraded":"2024-09-20T18:57:54.148177+0000","last_fullsized":"2024-09-20T18:57:54.148177+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T18:52:15.826638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:15.826638+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":379,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.162995+0000","last_change":"2024-09-20T18:57:18.032140+0000","last_active":"2024-09-20T18:57:54.162995+0000","last_peered":"2024-09-20T18:57:54.162995+0000","last_clean":"2024-09-20T18:57:54.162995+0000","last_became_active":"2024-09-20T18:49:45.933078+0000","last_became_peered":"2024-09-20T18:49:45.933078+0000","last_unstale":"2024-09-20T18:57:54.162995+0000","last_undegraded":"2024-09-20T18:57:54.162995+0000","last_fullsized":"2024-09-20T18:57:54.162995+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:57:18.032067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:18.032067+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:42.727785+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":397,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.163486+0000","last_change":"2024-09-20T18:55:19.996522+0000","last_active":"2024-09-20T18:57:54.163486+0000","last_peered":"2024-09-20T18:57:54.163486+0000","last_clean":"2024-09-20T18:57:54.163486+0000","last_became_active":"2024-09-20T18:49:48.500809+0000","last_became_peered":"2024-09-20T18:49:48.500809+0000","last_unstale":"2024-09-20T18:57:54.163486+0000","last_undegraded":"2024-09-20T18:57:54.163486+0000","last_fullsized":"2024-09-20T18:57:54.163486+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:55:19.996328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:19.996328+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"184'9","reported_seq":187,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.134376+0000","last_change":"2024-09-20T18:57:48.853376+0000","last_active":"2024-09-20T18:57:54.134376+0000","last_peered":"2024-09-20T18:57:54.134376+0000","last_clean":"2024-09-20T18:57:54.134376+0000","last_became_active":"2024-09-20T18:55:28.820941+0000","last_became_peered":"2024-09-20T18:55:28.820941+0000","last_unstale":"2024-09-20T18:57:54.134376+0000","last_undegraded":"2024-09-20T18:57:54.134376+0000","last_fullsized":"2024-09-20T18:57:54.134376+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'9","last_scrub_stamp":"2024-09-20T18:57:48.853302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:48.853302+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:06.295025+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3","version":"233'13334","reported_seq":25848,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:58:01.384720+0000","last_change":"2024-09-20T18:57:53.578343+0000","last_active":"2024-09-20T18:58:01.384720+0000","last_peered":"2024-09-20T18:58:01.384720+0000","last_clean":"2024-09-20T18:58:01.384720+0000","last_became_active":"2024-09-20T18:56:25.687391+0000","last_became_peered":"2024-09-20T18:56:25.687391+0000","last_unstale":"2024-09-20T18:58:01.384720+0000","last_undegraded":"2024-09-20T18:58:01.384720+0000","last_fullsized":"2024-09-20T18:58:01.384720+0000","mapping_epoch":203,"log_start":"211'9072","ondisk_log_start":"211'9072","created":160,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'12840","last_scrub_stamp":"2024-09-20T18:57:53.578263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:57:53.578263+0000","objects_scrubbed":1475,"log_size":4262,"log_dups_size":0,"ondisk_log_size":4262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:55.743448+0000","scrub_duration":660,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63045632,"num_objects":962,"num_object_clones":0,"num_object_copies":2886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13334,"num_write_kb":457472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6156,"num_bytes_recovered":403439616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":391,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.164686+0000","last_change":"2024-09-20T18:55:18.581290+0000","last_active":"2024-09-20T18:57:54.164686+0000","last_peered":"2024-09-20T18:57:54.164686+0000","last_clean":"2024-09-20T18:57:54.164686+0000","last_became_active":"2024-09-20T18:49:58.992302+0000","last_became_peered":"2024-09-20T18:49:58.992302+0000","last_unstale":"2024-09-20T18:57:54.164686+0000","last_undegraded":"2024-09-20T18:57:54.164686+0000","last_fullsized":"2024-09-20T18:57:54.164686+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:55:18.581200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:18.581200+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":378,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.151273+0000","last_change":"2024-09-20T18:54:54.288199+0000","last_active":"2024-09-20T18:57:54.151273+0000","last_peered":"2024-09-20T18:57:54.151273+0000","last_clean":"2024-09-20T18:57:54.151273+0000","last_became_active":"2024-09-20T18:49:45.936104+0000","last_became_peered":"2024-09-20T18:49:45.936104+0000","last_unstale":"2024-09-20T18:57:54.151273+0000","last_undegraded":"2024-09-20T18:57:54.151273+0000","last_fullsized":"2024-09-20T18:57:54.151273+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:54.288117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:54.288117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":46,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":480,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.134511+0000","last_change":"2024-09-20T18:54:14.292660+0000","last_active":"2024-09-20T18:57:54.134511+0000","last_peered":"2024-09-20T18:57:54.134511+0000","last_clean":"2024-09-20T18:57:54.134511+0000","last_became_active":"2024-09-20T18:49:49.960697+0000","last_became_peered":"2024-09-20T18:49:49.960697+0000","last_unstale":"2024-09-20T18:57:54.134511+0000","last_undegraded":"2024-09-20T18:57:54.134511+0000","last_fullsized":"2024-09-20T18:57:54.134511+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:54:14.292610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:14.292610+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":34,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":365,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.146304+0000","last_change":"2024-09-20T18:54:10.381171+0000","last_active":"2024-09-20T18:57:54.146304+0000","last_peered":"2024-09-20T18:57:54.146304+0000","last_clean":"2024-09-20T18:57:54.146304+0000","last_became_active":"2024-09-20T18:49:51.971399+0000","last_became_peered":"2024-09-20T18:49:51.971399+0000","last_unstale":"2024-09-20T18:57:54.146304+0000","last_undegraded":"2024-09-20T18:57:54.146304+0000","last_fullsized":"2024-09-20T18:57:54.146304+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:10.381108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:10.381108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:30.527012+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.c","version":"233'13542","reported_seq":13700,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:58:01.843500+0000","last_change":"2024-09-20T18:54:34.311256+0000","last_active":"2024-09-20T18:58:01.843500+0000","last_peered":"2024-09-20T18:58:01.843500+0000","last_clean":"2024-09-20T18:58:01.843500+0000","last_became_active":"2024-09-20T18:54:34.310904+0000","last_became_peered":"2024-09-20T18:54:34.310904+0000","last_unstale":"2024-09-20T18:58:01.843500+0000","last_undegraded":"2024-09-20T18:58:01.843500+0000","last_fullsized":"2024-09-20T18:58:01.843500+0000","mapping_epoch":160,"log_start":"208'8611","ondisk_log_start":"208'8611","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4931,"log_dups_size":0,"ondisk_log_size":4931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55705600,"num_objects":850,"num_object_clones":0,"num_object_copies":2550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13542,"num_write_kb":460544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":407,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.150299+0000","last_change":"2024-09-20T18:54:22.077299+0000","last_active":"2024-09-20T18:57:54.150299+0000","last_peered":"2024-09-20T18:57:54.150299+0000","last_clean":"2024-09-20T18:57:54.150299+0000","last_became_active":"2024-09-20T18:49:48.499189+0000","last_became_peered":"2024-09-20T18:49:48.499189+0000","last_unstale":"2024-09-20T18:57:54.150299+0000","last_undegraded":"2024-09-20T18:57:54.150299+0000","last_fullsized":"2024-09-20T18:57:54.150299+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-20T18:54:22.077212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:22.077212+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":370,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.145818+0000","last_change":"2024-09-20T18:54:57.237740+0000","last_active":"2024-09-20T18:57:54.145818+0000","last_peered":"2024-09-20T18:57:54.145818+0000","last_clean":"2024-09-20T18:57:54.145818+0000","last_became_active":"2024-09-20T18:49:45.940544+0000","last_became_peered":"2024-09-20T18:49:45.940544+0000","last_unstale":"2024-09-20T18:57:54.145818+0000","last_undegraded":"2024-09-20T18:57:54.145818+0000","last_fullsized":"2024-09-20T18:57:54.145818+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.237675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.237675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":378,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.133700+0000","last_change":"2024-09-20T18:54:51.229251+0000","last_active":"2024-09-20T18:57:54.133700+0000","last_peered":"2024-09-20T18:57:54.133700+0000","last_clean":"2024-09-20T18:57:54.133700+0000","last_became_active":"2024-09-20T18:49:49.963073+0000","last_became_peered":"2024-09-20T18:49:49.963073+0000","last_unstale":"2024-09-20T18:57:54.133700+0000","last_undegraded":"2024-09-20T18:57:54.133700+0000","last_fullsized":"2024-09-20T18:57:54.133700+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.229201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.229201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":374,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.134212+0000","last_change":"2024-09-20T18:54:55.323937+0000","last_active":"2024-09-20T18:57:54.134212+0000","last_peered":"2024-09-20T18:57:54.134212+0000","last_clean":"2024-09-20T18:57:54.134212+0000","last_became_active":"2024-09-20T18:49:51.968793+0000","last_became_peered":"2024-09-20T18:49:51.968793+0000","last_unstale":"2024-09-20T18:57:54.134212+0000","last_undegraded":"2024-09-20T18:57:54.134212+0000","last_fullsized":"2024-09-20T18:57:54.134212+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:55.323692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:55.323692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"233'13443","reported_seq":13602,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:58:00.935743+0000","last_change":"2024-09-20T18:54:34.311680+0000","last_active":"2024-09-20T18:58:00.935743+0000","last_peered":"2024-09-20T18:58:00.935743+0000","last_clean":"2024-09-20T18:58:00.935743+0000","last_became_active":"2024-09-20T18:54:34.311241+0000","last_became_peered":"2024-09-20T18:54:34.311241+0000","last_unstale":"2024-09-20T18:58:00.935743+0000","last_undegraded":"2024-09-20T18:58:00.935743+0000","last_fullsized":"2024-09-20T18:58:00.935743+0000","mapping_epoch":160,"log_start":"206'8346","ondisk_log_start":"206'8346","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":5097,"log_dups_size":0,"ondisk_log_size":5097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:36.546826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60817432,"num_objects":929,"num_object_clones":0,"num_object_copies":2787,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":929,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13443,"num_write_kb":459841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":434,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.133736+0000","last_change":"2024-09-20T18:55:28.853375+0000","last_active":"2024-09-20T18:57:54.133736+0000","last_peered":"2024-09-20T18:57:54.133736+0000","last_clean":"2024-09-20T18:57:54.133736+0000","last_became_active":"2024-09-20T18:55:28.831057+0000","last_became_peered":"2024-09-20T18:55:28.831057+0000","last_unstale":"2024-09-20T18:57:54.133736+0000","last_undegraded":"2024-09-20T18:57:54.133736+0000","last_fullsized":"2024-09-20T18:57:54.133736+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-20T18:54:57.394893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.394893+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":370,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.162963+0000","last_change":"2024-09-20T18:51:27.187060+0000","last_active":"2024-09-20T18:57:54.162963+0000","last_peered":"2024-09-20T18:57:54.162963+0000","last_clean":"2024-09-20T18:57:54.162963+0000","last_became_active":"2024-09-20T18:49:45.940469+0000","last_became_peered":"2024-09-20T18:49:45.940469+0000","last_unstale":"2024-09-20T18:57:54.162963+0000","last_undegraded":"2024-09-20T18:57:54.162963+0000","last_fullsized":"2024-09-20T18:57:54.162963+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:51:27.186991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:27.186991+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"66'8","reported_seq":487,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:56.565169+0000","last_change":"2024-09-20T18:54:47.223010+0000","last_active":"2024-09-20T18:57:56.565169+0000","last_peered":"2024-09-20T18:57:56.565169+0000","last_clean":"2024-09-20T18:57:56.565169+0000","last_became_active":"2024-09-20T18:49:49.956133+0000","last_became_peered":"2024-09-20T18:49:49.956133+0000","last_unstale":"2024-09-20T18:57:56.565169+0000","last_undegraded":"2024-09-20T18:57:56.565169+0000","last_fullsized":"2024-09-20T18:57:56.565169+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:54:47.222937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:47.222937+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":372,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.147948+0000","last_change":"2024-09-20T18:55:28.819147+0000","last_active":"2024-09-20T18:57:54.147948+0000","last_peered":"2024-09-20T18:57:54.147948+0000","last_clean":"2024-09-20T18:57:54.147948+0000","last_became_active":"2024-09-20T18:55:28.818804+0000","last_became_peered":"2024-09-20T18:55:28.818804+0000","last_unstale":"2024-09-20T18:57:54.147948+0000","last_undegraded":"2024-09-20T18:57:54.147948+0000","last_fullsized":"2024-09-20T18:57:54.147948+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:42.880724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:42.880724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"233'13563","reported_seq":13719,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:58:02.084107+0000","last_change":"2024-09-20T18:54:34.309939+0000","last_active":"2024-09-20T18:58:02.084107+0000","last_peered":"2024-09-20T18:58:02.084107+0000","last_clean":"2024-09-20T18:58:02.084107+0000","last_became_active":"2024-09-20T18:54:34.309630+0000","last_became_peered":"2024-09-20T18:54:34.309630+0000","last_unstale":"2024-09-20T18:58:02.084107+0000","last_undegraded":"2024-09-20T18:58:02.084107+0000","last_fullsized":"2024-09-20T18:58:02.084107+0000","mapping_epoch":160,"log_start":"208'8649","ondisk_log_start":"208'8649","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4914,"log_dups_size":0,"ondisk_log_size":4914,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54984704,"num_objects":839,"num_object_clones":0,"num_object_copies":2517,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13563,"num_write_kb":460864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":397,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.164080+0000","last_change":"2024-09-20T18:55:19.541011+0000","last_active":"2024-09-20T18:57:54.164080+0000","last_peered":"2024-09-20T18:57:54.164080+0000","last_clean":"2024-09-20T18:57:54.164080+0000","last_became_active":"2024-09-20T18:49:47.949626+0000","last_became_peered":"2024-09-20T18:49:47.949626+0000","last_unstale":"2024-09-20T18:57:54.164080+0000","last_undegraded":"2024-09-20T18:57:54.164080+0000","last_fullsized":"2024-09-20T18:57:54.164080+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:55:19.540812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:19.540812+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":385,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.146828+0000","last_change":"2024-09-20T18:54:18.326947+0000","last_active":"2024-09-20T18:57:54.146828+0000","last_peered":"2024-09-20T18:57:54.146828+0000","last_clean":"2024-09-20T18:57:54.146828+0000","last_became_active":"2024-09-20T18:49:45.942496+0000","last_became_peered":"2024-09-20T18:49:45.942496+0000","last_unstale":"2024-09-20T18:57:54.146828+0000","last_undegraded":"2024-09-20T18:57:54.146828+0000","last_fullsized":"2024-09-20T18:57:54.146828+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:18.326875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:18.326875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:32.505949+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"184'9","reported_seq":488,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:56.090561+0000","last_change":"2024-09-20T18:55:29.010062+0000","last_active":"2024-09-20T18:57:56.090561+0000","last_peered":"2024-09-20T18:57:56.090561+0000","last_clean":"2024-09-20T18:57:56.090561+0000","last_became_active":"2024-09-20T18:55:28.830643+0000","last_became_peered":"2024-09-20T18:55:28.830643+0000","last_unstale":"2024-09-20T18:57:56.090561+0000","last_undegraded":"2024-09-20T18:57:56.090561+0000","last_fullsized":"2024-09-20T18:57:56.090561+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:29.010015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:29.010015+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":350,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.159917+0000","last_change":"2024-09-20T18:49:51.965733+0000","last_active":"2024-09-20T18:57:54.159917+0000","last_peered":"2024-09-20T18:57:54.159917+0000","last_clean":"2024-09-20T18:57:54.159917+0000","last_became_active":"2024-09-20T18:49:51.964780+0000","last_became_peered":"2024-09-20T18:49:51.964780+0000","last_unstale":"2024-09-20T18:57:54.159917+0000","last_undegraded":"2024-09-20T18:57:54.159917+0000","last_fullsized":"2024-09-20T18:57:54.159917+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"233'13866","reported_seq":14024,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:58:01.480489+0000","last_change":"2024-09-20T18:54:34.309809+0000","last_active":"2024-09-20T18:58:01.480489+0000","last_peered":"2024-09-20T18:58:01.480489+0000","last_clean":"2024-09-20T18:58:01.480489+0000","last_became_active":"2024-09-20T18:54:34.309231+0000","last_became_peered":"2024-09-20T18:54:34.309231+0000","last_unstale":"2024-09-20T18:58:01.480489+0000","last_undegraded":"2024-09-20T18:58:01.480489+0000","last_fullsized":"2024-09-20T18:58:01.480489+0000","mapping_epoch":160,"log_start":"211'9532","ondisk_log_start":"211'9532","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4334,"log_dups_size":0,"ondisk_log_size":4334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55574528,"num_objects":848,"num_object_clones":0,"num_object_copies":2544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13866,"num_write_kb":470848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":409,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.160953+0000","last_change":"2024-09-20T18:54:14.923895+0000","last_active":"2024-09-20T18:57:54.160953+0000","last_peered":"2024-09-20T18:57:54.160953+0000","last_clean":"2024-09-20T18:57:54.160953+0000","last_became_active":"2024-09-20T18:49:48.497736+0000","last_became_peered":"2024-09-20T18:49:48.497736+0000","last_unstale":"2024-09-20T18:57:54.160953+0000","last_undegraded":"2024-09-20T18:57:54.160953+0000","last_fullsized":"2024-09-20T18:57:54.160953+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-20T18:54:14.923682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:14.923682+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":370,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.166094+0000","last_change":"2024-09-20T18:57:09.940634+0000","last_active":"2024-09-20T18:57:54.166094+0000","last_peered":"2024-09-20T18:57:54.166094+0000","last_clean":"2024-09-20T18:57:54.166094+0000","last_became_active":"2024-09-20T18:49:45.942253+0000","last_became_peered":"2024-09-20T18:49:45.942253+0000","last_unstale":"2024-09-20T18:57:54.166094+0000","last_undegraded":"2024-09-20T18:57:54.166094+0000","last_fullsized":"2024-09-20T18:57:54.166094+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:09.940517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:09.940517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:18.833936+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":116,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.160514+0000","last_change":"2024-09-20T18:55:28.830954+0000","last_active":"2024-09-20T18:57:54.160514+0000","last_peered":"2024-09-20T18:57:54.160514+0000","last_clean":"2024-09-20T18:57:54.160514+0000","last_became_active":"2024-09-20T18:55:28.830645+0000","last_became_peered":"2024-09-20T18:55:28.830645+0000","last_unstale":"2024-09-20T18:57:54.160514+0000","last_undegraded":"2024-09-20T18:57:54.160514+0000","last_fullsized":"2024-09-20T18:57:54.160514+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:50.610871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:50.610871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":389,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:58.973310+0000","last_change":"2024-09-20T18:57:58.973310+0000","last_active":"2024-09-20T18:57:58.973310+0000","last_peered":"2024-09-20T18:57:58.973310+0000","last_clean":"2024-09-20T18:57:58.973310+0000","last_became_active":"2024-09-20T18:55:28.821588+0000","last_became_peered":"2024-09-20T18:55:28.821588+0000","last_unstale":"2024-09-20T18:57:58.973310+0000","last_undegraded":"2024-09-20T18:57:58.973310+0000","last_fullsized":"2024-09-20T18:57:58.973310+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:58.973176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:58.973176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:03.568279+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"233'13633","reported_seq":13817,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:58:00.920782+0000","last_change":"2024-09-20T18:57:47.843800+0000","last_active":"2024-09-20T18:58:00.920782+0000","last_peered":"2024-09-20T18:58:00.920782+0000","last_clean":"2024-09-20T18:58:00.920782+0000","last_became_active":"2024-09-20T18:54:34.311181+0000","last_became_peered":"2024-09-20T18:54:34.311181+0000","last_unstale":"2024-09-20T18:58:00.920782+0000","last_undegraded":"2024-09-20T18:58:00.920782+0000","last_fullsized":"2024-09-20T18:58:00.920782+0000","mapping_epoch":160,"log_start":"206'8594","ondisk_log_start":"206'8594","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'12813","last_scrub_stamp":"2024-09-20T18:57:47.843722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:57:47.843722+0000","objects_scrubbed":1796,"log_size":5039,"log_dups_size":0,"ondisk_log_size":5039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:52.400285+0000","scrub_duration":406,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63242240,"num_objects":965,"num_object_clones":0,"num_object_copies":2895,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":965,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13633,"num_write_kb":467136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"66'28","reported_seq":522,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:56.091375+0000","last_change":"2024-09-20T18:55:16.527267+0000","last_active":"2024-09-20T18:57:56.091375+0000","last_peered":"2024-09-20T18:57:56.091375+0000","last_clean":"2024-09-20T18:57:56.091375+0000","last_became_active":"2024-09-20T18:49:47.955335+0000","last_became_peered":"2024-09-20T18:49:47.955335+0000","last_unstale":"2024-09-20T18:57:56.091375+0000","last_undegraded":"2024-09-20T18:57:56.091375+0000","last_fullsized":"2024-09-20T18:57:56.091375+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'28","last_scrub_stamp":"2024-09-20T18:55:16.527165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:16.527165+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":377,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.134315+0000","last_change":"2024-09-20T18:57:21.807755+0000","last_active":"2024-09-20T18:57:54.134315+0000","last_peered":"2024-09-20T18:57:54.134315+0000","last_clean":"2024-09-20T18:57:54.134315+0000","last_became_active":"2024-09-20T18:55:28.831574+0000","last_became_peered":"2024-09-20T18:55:28.831574+0000","last_unstale":"2024-09-20T18:57:54.134315+0000","last_undegraded":"2024-09-20T18:57:54.134315+0000","last_fullsized":"2024-09-20T18:57:54.134315+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:21.807703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:21.807703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:42.821495+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1],"acting":[4,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":414,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.146186+0000","last_change":"2024-09-20T18:55:28.854615+0000","last_active":"2024-09-20T18:57:54.146186+0000","last_peered":"2024-09-20T18:57:54.146186+0000","last_clean":"2024-09-20T18:57:54.146186+0000","last_became_active":"2024-09-20T18:55:28.837226+0000","last_became_peered":"2024-09-20T18:55:28.837226+0000","last_unstale":"2024-09-20T18:57:54.146186+0000","last_undegraded":"2024-09-20T18:57:54.146186+0000","last_fullsized":"2024-09-20T18:57:54.146186+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-20T18:54:51.227531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.227531+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:43.598074+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0012781159999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":377,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.162174+0000","last_change":"2024-09-20T18:55:13.949348+0000","last_active":"2024-09-20T18:57:54.162174+0000","last_peered":"2024-09-20T18:57:54.162174+0000","last_clean":"2024-09-20T18:57:54.162174+0000","last_became_active":"2024-09-20T18:49:49.953971+0000","last_became_peered":"2024-09-20T18:49:49.953971+0000","last_unstale":"2024-09-20T18:57:54.162174+0000","last_undegraded":"2024-09-20T18:57:54.162174+0000","last_fullsized":"2024-09-20T18:57:54.162174+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:13.949269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:13.949269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":358,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.142571+0000","last_change":"2024-09-20T18:57:12.744116+0000","last_active":"2024-09-20T18:57:54.142571+0000","last_peered":"2024-09-20T18:57:54.142571+0000","last_clean":"2024-09-20T18:57:54.142571+0000","last_became_active":"2024-09-20T18:49:51.969400+0000","last_became_peered":"2024-09-20T18:49:51.969400+0000","last_unstale":"2024-09-20T18:57:54.142571+0000","last_undegraded":"2024-09-20T18:57:54.142571+0000","last_fullsized":"2024-09-20T18:57:54.142571+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:12.744040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:12.744040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:17.964285+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"233'13778","reported_seq":13936,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:58:01.503151+0000","last_change":"2024-09-20T18:54:33.926785+0000","last_active":"2024-09-20T18:58:01.503151+0000","last_peered":"2024-09-20T18:58:01.503151+0000","last_clean":"2024-09-20T18:58:01.503151+0000","last_became_active":"2024-09-20T18:54:33.925750+0000","last_became_peered":"2024-09-20T18:54:33.925750+0000","last_unstale":"2024-09-20T18:58:01.503151+0000","last_undegraded":"2024-09-20T18:58:01.503151+0000","last_fullsized":"2024-09-20T18:58:01.503151+0000","mapping_epoch":160,"log_start":"211'9440","ondisk_log_start":"211'9440","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4338,"log_dups_size":0,"ondisk_log_size":4338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61603840,"num_objects":940,"num_object_clones":0,"num_object_copies":2820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":940,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13778,"num_write_kb":470976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":409,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.169677+0000","last_change":"2024-09-20T18:55:25.702762+0000","last_active":"2024-09-20T18:57:54.169677+0000","last_peered":"2024-09-20T18:57:54.169677+0000","last_clean":"2024-09-20T18:57:54.169677+0000","last_became_active":"2024-09-20T18:49:47.950904+0000","last_became_peered":"2024-09-20T18:49:47.950904+0000","last_unstale":"2024-09-20T18:57:54.169677+0000","last_undegraded":"2024-09-20T18:57:54.169677+0000","last_fullsized":"2024-09-20T18:57:54.169677+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-20T18:55:25.702596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:25.702596+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":394,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.163293+0000","last_change":"2024-09-20T18:55:28.830832+0000","last_active":"2024-09-20T18:57:54.163293+0000","last_peered":"2024-09-20T18:57:54.163293+0000","last_clean":"2024-09-20T18:57:54.163293+0000","last_became_active":"2024-09-20T18:55:28.830269+0000","last_became_peered":"2024-09-20T18:55:28.830269+0000","last_unstale":"2024-09-20T18:57:54.163293+0000","last_undegraded":"2024-09-20T18:57:54.163293+0000","last_fullsized":"2024-09-20T18:57:54.163293+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:23.359769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:23.359769+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:27.139436+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":381,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.163676+0000","last_change":"2024-09-20T18:52:07.718766+0000","last_active":"2024-09-20T18:57:54.163676+0000","last_peered":"2024-09-20T18:57:54.163676+0000","last_clean":"2024-09-20T18:57:54.163676+0000","last_became_active":"2024-09-20T18:49:44.937135+0000","last_became_peered":"2024-09-20T18:49:44.937135+0000","last_unstale":"2024-09-20T18:57:54.163676+0000","last_undegraded":"2024-09-20T18:57:54.163676+0000","last_fullsized":"2024-09-20T18:57:54.163676+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:52:07.718646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:52:07.718646+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.00095558800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":363,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.170778+0000","last_change":"2024-09-20T18:51:59.796450+0000","last_active":"2024-09-20T18:57:54.170778+0000","last_peered":"2024-09-20T18:57:54.170778+0000","last_clean":"2024-09-20T18:57:54.170778+0000","last_became_active":"2024-09-20T18:49:49.954424+0000","last_became_peered":"2024-09-20T18:49:49.954424+0000","last_unstale":"2024-09-20T18:57:54.170778+0000","last_undegraded":"2024-09-20T18:57:54.170778+0000","last_fullsized":"2024-09-20T18:57:54.170778+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:59.796383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:59.796383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":364,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.148783+0000","last_change":"2024-09-20T18:55:28.826659+0000","last_active":"2024-09-20T18:57:54.148783+0000","last_peered":"2024-09-20T18:57:54.148783+0000","last_clean":"2024-09-20T18:57:54.148783+0000","last_became_active":"2024-09-20T18:55:28.825812+0000","last_became_peered":"2024-09-20T18:55:28.825812+0000","last_unstale":"2024-09-20T18:57:54.148783+0000","last_undegraded":"2024-09-20T18:57:54.148783+0000","last_fullsized":"2024-09-20T18:57:54.148783+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:39.814259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:39.814259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"233'13748","reported_seq":13906,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:58:01.377582+0000","last_change":"2024-09-20T18:54:34.311774+0000","last_active":"2024-09-20T18:58:01.377582+0000","last_peered":"2024-09-20T18:58:01.377582+0000","last_clean":"2024-09-20T18:58:01.377582+0000","last_became_active":"2024-09-20T18:54:34.311487+0000","last_became_peered":"2024-09-20T18:54:34.311487+0000","last_unstale":"2024-09-20T18:58:01.377582+0000","last_undegraded":"2024-09-20T18:58:01.377582+0000","last_fullsized":"2024-09-20T18:58:01.377582+0000","mapping_epoch":160,"log_start":"211'9492","ondisk_log_start":"211'9492","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4256,"log_dups_size":0,"ondisk_log_size":4256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:43.066892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57016320,"num_objects":870,"num_object_clones":0,"num_object_copies":2610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13748,"num_write_kb":467776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":407,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.163568+0000","last_change":"2024-09-20T18:55:14.951950+0000","last_active":"2024-09-20T18:57:54.163568+0000","last_peered":"2024-09-20T18:57:54.163568+0000","last_clean":"2024-09-20T18:57:54.163568+0000","last_became_active":"2024-09-20T18:49:47.949230+0000","last_became_peered":"2024-09-20T18:49:47.949230+0000","last_unstale":"2024-09-20T18:57:54.163568+0000","last_undegraded":"2024-09-20T18:57:54.163568+0000","last_fullsized":"2024-09-20T18:57:54.163568+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:55:14.951892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:14.951892+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":385,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.160683+0000","last_change":"2024-09-20T18:54:27.905856+0000","last_active":"2024-09-20T18:57:54.160683+0000","last_peered":"2024-09-20T18:57:54.160683+0000","last_clean":"2024-09-20T18:57:54.160683+0000","last_became_active":"2024-09-20T18:49:45.939197+0000","last_became_peered":"2024-09-20T18:49:45.939197+0000","last_unstale":"2024-09-20T18:57:54.160683+0000","last_undegraded":"2024-09-20T18:57:54.160683+0000","last_fullsized":"2024-09-20T18:57:54.160683+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:27.905803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:27.905803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":886,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:58.893322+0000","last_change":"2024-09-20T18:54:26.359887+0000","last_active":"2024-09-20T18:57:58.893322+0000","last_peered":"2024-09-20T18:57:58.893322+0000","last_clean":"2024-09-20T18:57:58.893322+0000","last_became_active":"2024-09-20T18:49:44.936819+0000","last_became_peered":"2024-09-20T18:49:44.936819+0000","last_unstale":"2024-09-20T18:57:58.893322+0000","last_undegraded":"2024-09-20T18:57:58.893322+0000","last_fullsized":"2024-09-20T18:57:58.893322+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-20T18:54:26.359807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:54:26.359807+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0010996199999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":489,"num_read_kb":484,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":377,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.362085+0000","last_change":"2024-09-20T18:57:54.362085+0000","last_active":"2024-09-20T18:57:54.362085+0000","last_peered":"2024-09-20T18:57:54.362085+0000","last_clean":"2024-09-20T18:57:54.362085+0000","last_became_active":"2024-09-20T18:49:49.961066+0000","last_became_peered":"2024-09-20T18:49:49.961066+0000","last_unstale":"2024-09-20T18:57:54.362085+0000","last_undegraded":"2024-09-20T18:57:54.362085+0000","last_fullsized":"2024-09-20T18:57:54.362085+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:54.362038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:54.362038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:08.560302+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":374,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.165688+0000","last_change":"2024-09-20T18:55:21.629867+0000","last_active":"2024-09-20T18:57:54.165688+0000","last_peered":"2024-09-20T18:57:54.165688+0000","last_clean":"2024-09-20T18:57:54.165688+0000","last_became_active":"2024-09-20T18:49:51.970343+0000","last_became_peered":"2024-09-20T18:49:51.970343+0000","last_unstale":"2024-09-20T18:57:54.165688+0000","last_undegraded":"2024-09-20T18:57:54.165688+0000","last_fullsized":"2024-09-20T18:57:54.165688+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:21.629800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:21.629800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"233'13592","reported_seq":24174,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:58:00.932651+0000","last_change":"2024-09-20T18:57:09.139489+0000","last_active":"2024-09-20T18:58:00.932651+0000","last_peered":"2024-09-20T18:58:00.932651+0000","last_clean":"2024-09-20T18:58:00.932651+0000","last_became_active":"2024-09-20T18:57:09.121592+0000","last_became_peered":"2024-09-20T18:57:09.121592+0000","last_unstale":"2024-09-20T18:58:00.932651+0000","last_undegraded":"2024-09-20T18:58:00.932651+0000","last_fullsized":"2024-09-20T18:58:00.932651+0000","mapping_epoch":216,"log_start":"206'8506","ondisk_log_start":"206'8506","created":160,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":5086,"log_dups_size":0,"ondisk_log_size":5086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:35.584332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61603840,"num_objects":940,"num_object_clones":0,"num_object_copies":2820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":940,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13592,"num_write_kb":465024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5206,"num_bytes_recovered":341180416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"184'17","reported_seq":216,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.134240+0000","last_change":"2024-09-20T18:57:22.816240+0000","last_active":"2024-09-20T18:57:54.134240+0000","last_peered":"2024-09-20T18:57:54.134240+0000","last_clean":"2024-09-20T18:57:54.134240+0000","last_became_active":"2024-09-20T18:55:28.831024+0000","last_became_peered":"2024-09-20T18:55:28.831024+0000","last_unstale":"2024-09-20T18:57:54.134240+0000","last_undegraded":"2024-09-20T18:57:54.134240+0000","last_fullsized":"2024-09-20T18:57:54.134240+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'17","last_scrub_stamp":"2024-09-20T18:57:22.816154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:22.816154+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:28.727289+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":24,"num_read_kb":14,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":385,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.141997+0000","last_change":"2024-09-20T18:55:28.830447+0000","last_active":"2024-09-20T18:57:54.141997+0000","last_peered":"2024-09-20T18:57:54.141997+0000","last_clean":"2024-09-20T18:57:54.141997+0000","last_became_active":"2024-09-20T18:55:28.830147+0000","last_became_peered":"2024-09-20T18:55:28.830147+0000","last_unstale":"2024-09-20T18:57:54.141997+0000","last_undegraded":"2024-09-20T18:57:54.141997+0000","last_fullsized":"2024-09-20T18:57:54.141997+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:13.144896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:13.144896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:28.731761+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":439,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.148883+0000","last_change":"2024-09-20T18:55:28.844060+0000","last_active":"2024-09-20T18:57:54.148883+0000","last_peered":"2024-09-20T18:57:54.148883+0000","last_clean":"2024-09-20T18:57:54.148883+0000","last_became_active":"2024-09-20T18:55:28.829325+0000","last_became_peered":"2024-09-20T18:55:28.829325+0000","last_unstale":"2024-09-20T18:57:54.148883+0000","last_undegraded":"2024-09-20T18:57:54.148883+0000","last_fullsized":"2024-09-20T18:57:54.148883+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T18:55:13.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T18:55:13.696607+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":361,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.148577+0000","last_change":"2024-09-20T18:51:22.852942+0000","last_active":"2024-09-20T18:57:54.148577+0000","last_peered":"2024-09-20T18:57:54.148577+0000","last_clean":"2024-09-20T18:57:54.148577+0000","last_became_active":"2024-09-20T18:49:49.956128+0000","last_became_peered":"2024-09-20T18:49:49.956128+0000","last_unstale":"2024-09-20T18:57:54.148577+0000","last_undegraded":"2024-09-20T18:57:54.148577+0000","last_fullsized":"2024-09-20T18:57:54.148577+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:22.852869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:22.852869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":373,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.146436+0000","last_change":"2024-09-20T18:55:25.281435+0000","last_active":"2024-09-20T18:57:54.146436+0000","last_peered":"2024-09-20T18:57:54.146436+0000","last_clean":"2024-09-20T18:57:54.146436+0000","last_became_active":"2024-09-20T18:49:51.971332+0000","last_became_peered":"2024-09-20T18:49:51.971332+0000","last_unstale":"2024-09-20T18:57:54.146436+0000","last_undegraded":"2024-09-20T18:57:54.146436+0000","last_fullsized":"2024-09-20T18:57:54.146436+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:25.281366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:25.281366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"233'13611","reported_seq":13769,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:58:00.943840+0000","last_change":"2024-09-20T18:54:33.929366+0000","last_active":"2024-09-20T18:58:00.943840+0000","last_peered":"2024-09-20T18:58:00.943840+0000","last_clean":"2024-09-20T18:58:00.943840+0000","last_became_active":"2024-09-20T18:54:33.929072+0000","last_became_peered":"2024-09-20T18:54:33.929072+0000","last_unstale":"2024-09-20T18:58:00.943840+0000","last_undegraded":"2024-09-20T18:58:00.943840+0000","last_fullsized":"2024-09-20T18:58:00.943840+0000","mapping_epoch":160,"log_start":"210'9002","ondisk_log_start":"210'9002","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4609,"log_dups_size":0,"ondisk_log_size":4609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62324736,"num_objects":951,"num_object_clones":0,"num_object_copies":2853,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13611,"num_write_kb":465984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":404,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.134499+0000","last_change":"2024-09-20T18:55:28.840105+0000","last_active":"2024-09-20T18:57:54.134499+0000","last_peered":"2024-09-20T18:57:54.134499+0000","last_clean":"2024-09-20T18:57:54.134499+0000","last_became_active":"2024-09-20T18:55:28.818269+0000","last_became_peered":"2024-09-20T18:55:28.818269+0000","last_unstale":"2024-09-20T18:57:54.134499+0000","last_undegraded":"2024-09-20T18:57:54.134499+0000","last_fullsized":"2024-09-20T18:57:54.134499+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:51:14.397930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.397930+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:23.803752+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":370,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.133973+0000","last_change":"2024-09-20T18:51:18.410601+0000","last_active":"2024-09-20T18:57:54.133973+0000","last_peered":"2024-09-20T18:57:54.133973+0000","last_clean":"2024-09-20T18:57:54.133973+0000","last_became_active":"2024-09-20T18:49:45.944579+0000","last_became_peered":"2024-09-20T18:49:45.944579+0000","last_unstale":"2024-09-20T18:57:54.133973+0000","last_undegraded":"2024-09-20T18:57:54.133973+0000","last_fullsized":"2024-09-20T18:57:54.133973+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:18.410447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:18.410447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:29.917821+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":384,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.146365+0000","last_change":"2024-09-20T18:55:28.830634+0000","last_active":"2024-09-20T18:57:54.146365+0000","last_peered":"2024-09-20T18:57:54.146365+0000","last_clean":"2024-09-20T18:57:54.146365+0000","last_became_active":"2024-09-20T18:55:28.827785+0000","last_became_peered":"2024-09-20T18:55:28.827785+0000","last_unstale":"2024-09-20T18:57:54.146365+0000","last_undegraded":"2024-09-20T18:57:54.146365+0000","last_fullsized":"2024-09-20T18:57:54.146365+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:11.401777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:11.401777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:37.550724+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":373,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.142091+0000","last_change":"2024-09-20T18:57:23.721754+0000","last_active":"2024-09-20T18:57:54.142091+0000","last_peered":"2024-09-20T18:57:54.142091+0000","last_clean":"2024-09-20T18:57:54.142091+0000","last_became_active":"2024-09-20T18:55:28.833115+0000","last_became_peered":"2024-09-20T18:55:28.833115+0000","last_unstale":"2024-09-20T18:57:54.142091+0000","last_undegraded":"2024-09-20T18:57:54.142091+0000","last_fullsized":"2024-09-20T18:57:54.142091+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:23.721643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:23.721643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:25.737391+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1","version":"233'13545","reported_seq":13728,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:58:01.405056+0000","last_change":"2024-09-20T18:57:50.704022+0000","last_active":"2024-09-20T18:58:01.405056+0000","last_peered":"2024-09-20T18:58:01.405056+0000","last_clean":"2024-09-20T18:58:01.405056+0000","last_became_active":"2024-09-20T18:54:34.309781+0000","last_became_peered":"2024-09-20T18:54:34.309781+0000","last_unstale":"2024-09-20T18:58:01.405056+0000","last_undegraded":"2024-09-20T18:58:01.405056+0000","last_fullsized":"2024-09-20T18:58:01.405056+0000","mapping_epoch":160,"log_start":"211'9224","ondisk_log_start":"211'9224","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'12866","last_scrub_stamp":"2024-09-20T18:57:50.703702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:57:50.703702+0000","objects_scrubbed":1599,"log_size":4321,"log_dups_size":0,"ondisk_log_size":4321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:00.179362+0000","scrub_duration":719,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58130432,"num_objects":887,"num_object_clones":0,"num_object_copies":2661,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13545,"num_write_kb":461824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":402,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.134298+0000","last_change":"2024-09-20T18:52:02.944536+0000","last_active":"2024-09-20T18:57:54.134298+0000","last_peered":"2024-09-20T18:57:54.134298+0000","last_clean":"2024-09-20T18:57:54.134298+0000","last_became_active":"2024-09-20T18:49:47.947636+0000","last_became_peered":"2024-09-20T18:49:47.947636+0000","last_unstale":"2024-09-20T18:57:54.134298+0000","last_undegraded":"2024-09-20T18:57:54.134298+0000","last_fullsized":"2024-09-20T18:57:54.134298+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-20T18:52:02.944434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:02.944434+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:34.896863+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":240,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.133838+0000","last_change":"2024-09-20T18:55:28.822330+0000","last_active":"2024-09-20T18:57:54.133838+0000","last_peered":"2024-09-20T18:57:54.133838+0000","last_clean":"2024-09-20T18:57:54.133838+0000","last_became_active":"2024-09-20T18:55:28.821959+0000","last_became_peered":"2024-09-20T18:55:28.821959+0000","last_unstale":"2024-09-20T18:57:54.133838+0000","last_undegraded":"2024-09-20T18:57:54.133838+0000","last_fullsized":"2024-09-20T18:57:54.133838+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:45.669427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:45.669427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":370,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.165787+0000","last_change":"2024-09-20T18:54:11.377907+0000","last_active":"2024-09-20T18:57:54.165787+0000","last_peered":"2024-09-20T18:57:54.165787+0000","last_clean":"2024-09-20T18:57:54.165787+0000","last_became_active":"2024-09-20T18:49:49.959160+0000","last_became_peered":"2024-09-20T18:49:49.959160+0000","last_unstale":"2024-09-20T18:57:54.165787+0000","last_undegraded":"2024-09-20T18:57:54.165787+0000","last_fullsized":"2024-09-20T18:57:54.165787+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:11.376951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:11.376951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:29.061620+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":367,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.164831+0000","last_change":"2024-09-20T18:52:35.635897+0000","last_active":"2024-09-20T18:57:54.164831+0000","last_peered":"2024-09-20T18:57:54.164831+0000","last_clean":"2024-09-20T18:57:54.164831+0000","last_became_active":"2024-09-20T18:49:51.970108+0000","last_became_peered":"2024-09-20T18:49:51.970108+0000","last_unstale":"2024-09-20T18:57:54.164831+0000","last_undegraded":"2024-09-20T18:57:54.164831+0000","last_fullsized":"2024-09-20T18:57:54.164831+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:35.635818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:35.635818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"233'13362","reported_seq":13520,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:58:01.487982+0000","last_change":"2024-09-20T18:54:33.929552+0000","last_active":"2024-09-20T18:58:01.487982+0000","last_peered":"2024-09-20T18:58:01.487982+0000","last_clean":"2024-09-20T18:58:01.487982+0000","last_became_active":"2024-09-20T18:54:33.929208+0000","last_became_peered":"2024-09-20T18:54:33.929208+0000","last_unstale":"2024-09-20T18:58:01.487982+0000","last_undegraded":"2024-09-20T18:58:01.487982+0000","last_fullsized":"2024-09-20T18:58:01.487982+0000","mapping_epoch":160,"log_start":"210'9021","ondisk_log_start":"210'9021","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4341,"log_dups_size":0,"ondisk_log_size":4341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56623104,"num_objects":864,"num_object_clones":0,"num_object_copies":2592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13362,"num_write_kb":455232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":408,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.134079+0000","last_change":"2024-09-20T18:55:28.846773+0000","last_active":"2024-09-20T18:57:54.134079+0000","last_peered":"2024-09-20T18:57:54.134079+0000","last_clean":"2024-09-20T18:57:54.134079+0000","last_became_active":"2024-09-20T18:55:28.817101+0000","last_became_peered":"2024-09-20T18:55:28.817101+0000","last_unstale":"2024-09-20T18:57:54.134079+0000","last_undegraded":"2024-09-20T18:57:54.134079+0000","last_fullsized":"2024-09-20T18:57:54.134079+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:54:17.219548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:17.219548+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":378,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.144006+0000","last_change":"2024-09-20T18:52:46.005172+0000","last_active":"2024-09-20T18:57:54.144006+0000","last_peered":"2024-09-20T18:57:54.144006+0000","last_clean":"2024-09-20T18:57:54.144006+0000","last_became_active":"2024-09-20T18:49:45.937264+0000","last_became_peered":"2024-09-20T18:49:45.937264+0000","last_unstale":"2024-09-20T18:57:54.144006+0000","last_undegraded":"2024-09-20T18:57:54.144006+0000","last_fullsized":"2024-09-20T18:57:54.144006+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:46.005109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:46.005109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:27.755958+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":487,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:56.092031+0000","last_change":"2024-09-20T18:55:17.549660+0000","last_active":"2024-09-20T18:57:56.092031+0000","last_peered":"2024-09-20T18:57:56.092031+0000","last_clean":"2024-09-20T18:57:56.092031+0000","last_became_active":"2024-09-20T18:49:49.957005+0000","last_became_peered":"2024-09-20T18:49:49.957005+0000","last_unstale":"2024-09-20T18:57:56.092031+0000","last_undegraded":"2024-09-20T18:57:56.092031+0000","last_fullsized":"2024-09-20T18:57:56.092031+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:17.549598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:17.549598+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":366,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.134338+0000","last_change":"2024-09-20T18:54:42.182875+0000","last_active":"2024-09-20T18:57:54.134338+0000","last_peered":"2024-09-20T18:57:54.134338+0000","last_clean":"2024-09-20T18:57:54.134338+0000","last_became_active":"2024-09-20T18:49:51.969355+0000","last_became_peered":"2024-09-20T18:49:51.969355+0000","last_unstale":"2024-09-20T18:57:54.134338+0000","last_undegraded":"2024-09-20T18:57:54.134338+0000","last_fullsized":"2024-09-20T18:57:54.134338+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:42.182809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:42.182809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"233'13667","reported_seq":26093,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:58:01.376841+0000","last_change":"2024-09-20T18:57:00.083015+0000","last_active":"2024-09-20T18:58:01.376841+0000","last_peered":"2024-09-20T18:58:01.376841+0000","last_clean":"2024-09-20T18:58:01.376841+0000","last_became_active":"2024-09-20T18:57:00.060960+0000","last_became_peered":"2024-09-20T18:57:00.060960+0000","last_unstale":"2024-09-20T18:58:01.376841+0000","last_undegraded":"2024-09-20T18:58:01.376841+0000","last_fullsized":"2024-09-20T18:58:01.376841+0000","mapping_epoch":213,"log_start":"211'9361","ondisk_log_start":"211'9361","created":160,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4306,"log_dups_size":0,"ondisk_log_size":4306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:34.055353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57868288,"num_objects":883,"num_object_clones":0,"num_object_copies":2649,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13667,"num_write_kb":465600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6125,"num_bytes_recovered":401408000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":397,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.160870+0000","last_change":"2024-09-20T18:54:54.897991+0000","last_active":"2024-09-20T18:57:54.160870+0000","last_peered":"2024-09-20T18:57:54.160870+0000","last_clean":"2024-09-20T18:57:54.160870+0000","last_became_active":"2024-09-20T18:49:47.946008+0000","last_became_peered":"2024-09-20T18:49:47.946008+0000","last_unstale":"2024-09-20T18:57:54.160870+0000","last_undegraded":"2024-09-20T18:57:54.160870+0000","last_fullsized":"2024-09-20T18:57:54.160870+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:54.897933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:54.897933+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":400,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:56.405392+0000","last_change":"2024-09-20T18:57:56.405392+0000","last_active":"2024-09-20T18:57:56.405392+0000","last_peered":"2024-09-20T18:57:56.405392+0000","last_clean":"2024-09-20T18:57:56.405392+0000","last_became_active":"2024-09-20T18:55:28.833095+0000","last_became_peered":"2024-09-20T18:55:28.833095+0000","last_unstale":"2024-09-20T18:57:56.405392+0000","last_undegraded":"2024-09-20T18:57:56.405392+0000","last_fullsized":"2024-09-20T18:57:56.405392+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:56.405349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:56.405349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:22.442121+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":379,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.162569+0000","last_change":"2024-09-20T18:54:43.379898+0000","last_active":"2024-09-20T18:57:54.162569+0000","last_peered":"2024-09-20T18:57:54.162569+0000","last_clean":"2024-09-20T18:57:54.162569+0000","last_became_active":"2024-09-20T18:49:49.956767+0000","last_became_peered":"2024-09-20T18:49:49.956767+0000","last_unstale":"2024-09-20T18:57:54.162569+0000","last_undegraded":"2024-09-20T18:57:54.162569+0000","last_fullsized":"2024-09-20T18:57:54.162569+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:43.379706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:43.379706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":365,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.160590+0000","last_change":"2024-09-20T18:54:24.976627+0000","last_active":"2024-09-20T18:57:54.160590+0000","last_peered":"2024-09-20T18:57:54.160590+0000","last_clean":"2024-09-20T18:57:54.160590+0000","last_became_active":"2024-09-20T18:49:51.964098+0000","last_became_peered":"2024-09-20T18:49:51.964098+0000","last_unstale":"2024-09-20T18:57:54.160590+0000","last_undegraded":"2024-09-20T18:57:54.160590+0000","last_fullsized":"2024-09-20T18:57:54.160590+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:24.976584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:24.976584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"233'13565","reported_seq":20192,"reported_epoch":233,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-20T18:58:01.883546+0000","last_change":"2024-09-20T18:56:27.871592+0000","last_active":"2024-09-20T18:58:01.883546+0000","last_peered":"2024-09-20T18:58:01.883546+0000","last_clean":"2024-09-20T18:55:27.791097+0000","last_became_active":"2024-09-20T18:55:29.979932+0000","last_became_peered":"2024-09-20T18:55:29.979932+0000","last_unstale":"2024-09-20T18:58:01.883546+0000","last_undegraded":"2024-09-20T18:56:27.867865+0000","last_fullsized":"2024-09-20T18:55:29.815116+0000","mapping_epoch":182,"log_start":"206'8640","ondisk_log_start":"206'8640","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4925,"log_dups_size":0,"ondisk_log_size":4925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57868288,"num_objects":883,"num_object_clones":0,"num_object_copies":2649,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13565,"num_write_kb":462336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3233,"num_bytes_recovered":175964160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7],"avail_no_missing":["0","6","7"],"object_location_counts":[{"shards":"0,7","objects":-1062}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":195,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.164456+0000","last_change":"2024-09-20T18:57:22.033602+0000","last_active":"2024-09-20T18:57:54.164456+0000","last_peered":"2024-09-20T18:57:54.164456+0000","last_clean":"2024-09-20T18:57:54.164456+0000","last_became_active":"2024-09-20T18:55:28.828563+0000","last_became_peered":"2024-09-20T18:55:28.828563+0000","last_unstale":"2024-09-20T18:57:54.164456+0000","last_undegraded":"2024-09-20T18:57:54.164456+0000","last_fullsized":"2024-09-20T18:57:54.164456+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-20T18:57:22.033475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:22.033475+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:43.998101+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":31,"num_read_kb":20,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,5],"acting":[1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":392,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.148971+0000","last_change":"2024-09-20T18:55:28.835767+0000","last_active":"2024-09-20T18:57:54.148971+0000","last_peered":"2024-09-20T18:57:54.148971+0000","last_clean":"2024-09-20T18:57:54.148971+0000","last_became_active":"2024-09-20T18:55:28.835149+0000","last_became_peered":"2024-09-20T18:55:28.835149+0000","last_unstale":"2024-09-20T18:57:54.148971+0000","last_undegraded":"2024-09-20T18:57:54.148971+0000","last_fullsized":"2024-09-20T18:57:54.148971+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:44.501192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.501192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":379,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.164116+0000","last_change":"2024-09-20T18:55:13.577772+0000","last_active":"2024-09-20T18:57:54.164116+0000","last_peered":"2024-09-20T18:57:54.164116+0000","last_clean":"2024-09-20T18:57:54.164116+0000","last_became_active":"2024-09-20T18:49:49.957558+0000","last_became_peered":"2024-09-20T18:49:49.957558+0000","last_unstale":"2024-09-20T18:57:54.164116+0000","last_undegraded":"2024-09-20T18:57:54.164116+0000","last_fullsized":"2024-09-20T18:57:54.164116+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:13.577699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:13.577699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":373,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.148787+0000","last_change":"2024-09-20T18:53:47.370152+0000","last_active":"2024-09-20T18:57:54.148787+0000","last_peered":"2024-09-20T18:57:54.148787+0000","last_clean":"2024-09-20T18:57:54.148787+0000","last_became_active":"2024-09-20T18:49:51.970798+0000","last_became_peered":"2024-09-20T18:49:51.970798+0000","last_unstale":"2024-09-20T18:57:54.148787+0000","last_undegraded":"2024-09-20T18:57:54.148787+0000","last_fullsized":"2024-09-20T18:57:54.148787+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:47.370108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:47.370108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"233'14009","reported_seq":24961,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:58:00.934213+0000","last_change":"2024-09-20T18:57:20.750334+0000","last_active":"2024-09-20T18:58:00.934213+0000","last_peered":"2024-09-20T18:58:00.934213+0000","last_clean":"2024-09-20T18:58:00.934213+0000","last_became_active":"2024-09-20T18:57:20.718369+0000","last_became_peered":"2024-09-20T18:57:20.718369+0000","last_unstale":"2024-09-20T18:58:00.934213+0000","last_undegraded":"2024-09-20T18:58:00.934213+0000","last_fullsized":"2024-09-20T18:58:00.934213+0000","mapping_epoch":221,"log_start":"210'9446","ondisk_log_start":"210'9446","created":160,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4563,"log_dups_size":0,"ondisk_log_size":4563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64028672,"num_objects":977,"num_object_clones":0,"num_object_copies":2931,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14009,"num_write_kb":479552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5391,"num_bytes_recovered":353304576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":397,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.133840+0000","last_change":"2024-09-20T18:53:52.898818+0000","last_active":"2024-09-20T18:57:54.133840+0000","last_peered":"2024-09-20T18:57:54.133840+0000","last_clean":"2024-09-20T18:57:54.133840+0000","last_became_active":"2024-09-20T18:49:47.949116+0000","last_became_peered":"2024-09-20T18:49:47.949116+0000","last_unstale":"2024-09-20T18:57:54.133840+0000","last_undegraded":"2024-09-20T18:57:54.133840+0000","last_fullsized":"2024-09-20T18:57:54.133840+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:53:52.898760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:52.898760+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:42.918366+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":385,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.148576+0000","last_change":"2024-09-20T18:54:52.650798+0000","last_active":"2024-09-20T18:57:54.148576+0000","last_peered":"2024-09-20T18:57:54.148576+0000","last_clean":"2024-09-20T18:57:54.148576+0000","last_became_active":"2024-09-20T18:49:45.944858+0000","last_became_peered":"2024-09-20T18:49:45.944858+0000","last_unstale":"2024-09-20T18:57:54.148576+0000","last_undegraded":"2024-09-20T18:57:54.148576+0000","last_fullsized":"2024-09-20T18:57:54.148576+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:52.650756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:52.650756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":371,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.162136+0000","last_change":"2024-09-20T18:54:42.349879+0000","last_active":"2024-09-20T18:57:54.162136+0000","last_peered":"2024-09-20T18:57:54.162136+0000","last_clean":"2024-09-20T18:57:54.162136+0000","last_became_active":"2024-09-20T18:49:49.954944+0000","last_became_peered":"2024-09-20T18:49:49.954944+0000","last_unstale":"2024-09-20T18:57:54.162136+0000","last_undegraded":"2024-09-20T18:57:54.162136+0000","last_fullsized":"2024-09-20T18:57:54.162136+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:42.349807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:42.349807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":366,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.143716+0000","last_change":"2024-09-20T18:55:17.341862+0000","last_active":"2024-09-20T18:57:54.143716+0000","last_peered":"2024-09-20T18:57:54.143716+0000","last_clean":"2024-09-20T18:57:54.143716+0000","last_became_active":"2024-09-20T18:49:51.972970+0000","last_became_peered":"2024-09-20T18:49:51.972970+0000","last_unstale":"2024-09-20T18:57:54.143716+0000","last_undegraded":"2024-09-20T18:57:54.143716+0000","last_fullsized":"2024-09-20T18:57:54.143716+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:17.341793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:17.341793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.4","version":"233'13408","reported_seq":13564,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:58:00.914183+0000","last_change":"2024-09-20T18:54:34.309442+0000","last_active":"2024-09-20T18:58:00.914183+0000","last_peered":"2024-09-20T18:58:00.914183+0000","last_clean":"2024-09-20T18:58:00.914183+0000","last_became_active":"2024-09-20T18:54:34.309113+0000","last_became_peered":"2024-09-20T18:54:34.309113+0000","last_unstale":"2024-09-20T18:58:00.914183+0000","last_undegraded":"2024-09-20T18:58:00.914183+0000","last_fullsized":"2024-09-20T18:58:00.914183+0000","mapping_epoch":160,"log_start":"206'8331","ondisk_log_start":"206'8331","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":5077,"log_dups_size":0,"ondisk_log_size":5077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:40.627488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60555264,"num_objects":924,"num_object_clones":0,"num_object_copies":2772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13408,"num_write_kb":458624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":421,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.149012+0000","last_change":"2024-09-20T18:55:28.847010+0000","last_active":"2024-09-20T18:57:54.149012+0000","last_peered":"2024-09-20T18:57:54.149012+0000","last_clean":"2024-09-20T18:57:54.149012+0000","last_became_active":"2024-09-20T18:55:28.824801+0000","last_became_peered":"2024-09-20T18:55:28.824801+0000","last_unstale":"2024-09-20T18:57:54.149012+0000","last_undegraded":"2024-09-20T18:57:54.149012+0000","last_fullsized":"2024-09-20T18:57:54.149012+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:54:12.373309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:12.373309+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:29.516079+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":405,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.148592+0000","last_change":"2024-09-20T18:55:24.729301+0000","last_active":"2024-09-20T18:57:54.148592+0000","last_peered":"2024-09-20T18:57:54.148592+0000","last_clean":"2024-09-20T18:57:54.148592+0000","last_became_active":"2024-09-20T18:49:45.944256+0000","last_became_peered":"2024-09-20T18:49:45.944256+0000","last_unstale":"2024-09-20T18:57:54.148592+0000","last_undegraded":"2024-09-20T18:57:54.148592+0000","last_fullsized":"2024-09-20T18:57:54.148592+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-20T18:55:24.729219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:24.729219+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"184'9","reported_seq":479,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.148042+0000","last_change":"2024-09-20T18:55:28.824771+0000","last_active":"2024-09-20T18:57:54.148042+0000","last_peered":"2024-09-20T18:57:54.148042+0000","last_clean":"2024-09-20T18:57:54.148042+0000","last_became_active":"2024-09-20T18:55:28.818085+0000","last_became_peered":"2024-09-20T18:55:28.818085+0000","last_unstale":"2024-09-20T18:57:54.148042+0000","last_undegraded":"2024-09-20T18:57:54.148042+0000","last_fullsized":"2024-09-20T18:57:54.148042+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:52:05.788924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.788924+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":372,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.145744+0000","last_change":"2024-09-20T18:55:28.832276+0000","last_active":"2024-09-20T18:57:54.145744+0000","last_peered":"2024-09-20T18:57:54.145744+0000","last_clean":"2024-09-20T18:57:54.145744+0000","last_became_active":"2024-09-20T18:55:28.830380+0000","last_became_peered":"2024-09-20T18:55:28.830380+0000","last_unstale":"2024-09-20T18:57:54.145744+0000","last_undegraded":"2024-09-20T18:57:54.145744+0000","last_fullsized":"2024-09-20T18:57:54.145744+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:14.247398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:14.247398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":116,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.159986+0000","last_change":"2024-09-20T18:55:28.835344+0000","last_active":"2024-09-20T18:57:54.159986+0000","last_peered":"2024-09-20T18:57:54.159986+0000","last_clean":"2024-09-20T18:57:54.159986+0000","last_became_active":"2024-09-20T18:55:28.835010+0000","last_became_peered":"2024-09-20T18:55:28.835010+0000","last_unstale":"2024-09-20T18:57:54.159986+0000","last_undegraded":"2024-09-20T18:57:54.159986+0000","last_fullsized":"2024-09-20T18:57:54.159986+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:26.721735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:26.721735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":383,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.145722+0000","last_change":"2024-09-20T18:54:44.194161+0000","last_active":"2024-09-20T18:57:54.145722+0000","last_peered":"2024-09-20T18:57:54.145722+0000","last_clean":"2024-09-20T18:57:54.145722+0000","last_became_active":"2024-09-20T18:49:48.500662+0000","last_became_peered":"2024-09-20T18:49:48.500662+0000","last_unstale":"2024-09-20T18:57:54.145722+0000","last_undegraded":"2024-09-20T18:57:54.145722+0000","last_fullsized":"2024-09-20T18:57:54.145722+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-20T18:54:44.194089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.194089+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":369,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.146207+0000","last_change":"2024-09-20T18:53:16.220554+0000","last_active":"2024-09-20T18:57:54.146207+0000","last_peered":"2024-09-20T18:57:54.146207+0000","last_clean":"2024-09-20T18:57:54.146207+0000","last_became_active":"2024-09-20T18:49:49.956749+0000","last_became_peered":"2024-09-20T18:49:49.956749+0000","last_unstale":"2024-09-20T18:57:54.146207+0000","last_undegraded":"2024-09-20T18:57:54.146207+0000","last_fullsized":"2024-09-20T18:57:54.146207+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:16.220490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:16.220490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:27.545117+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":380,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.148737+0000","last_change":"2024-09-20T18:55:28.827897+0000","last_active":"2024-09-20T18:57:54.148737+0000","last_peered":"2024-09-20T18:57:54.148737+0000","last_clean":"2024-09-20T18:57:54.148737+0000","last_became_active":"2024-09-20T18:55:28.827418+0000","last_became_peered":"2024-09-20T18:55:28.827418+0000","last_unstale":"2024-09-20T18:57:54.148737+0000","last_undegraded":"2024-09-20T18:57:54.148737+0000","last_fullsized":"2024-09-20T18:57:54.148737+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.637739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.637739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":389,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.163652+0000","last_change":"2024-09-20T18:54:44.830872+0000","last_active":"2024-09-20T18:57:54.163652+0000","last_peered":"2024-09-20T18:57:54.163652+0000","last_clean":"2024-09-20T18:57:54.163652+0000","last_became_active":"2024-09-20T18:49:48.500803+0000","last_became_peered":"2024-09-20T18:49:48.500803+0000","last_unstale":"2024-09-20T18:57:54.163652+0000","last_undegraded":"2024-09-20T18:57:54.163652+0000","last_fullsized":"2024-09-20T18:57:54.163652+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:44.830790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.830790+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":384,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.148051+0000","last_change":"2024-09-20T18:55:29.751718+0000","last_active":"2024-09-20T18:57:54.148051+0000","last_peered":"2024-09-20T18:57:54.148051+0000","last_clean":"2024-09-20T18:57:54.148051+0000","last_became_active":"2024-09-20T18:55:28.833799+0000","last_became_peered":"2024-09-20T18:55:28.833799+0000","last_unstale":"2024-09-20T18:57:54.148051+0000","last_undegraded":"2024-09-20T18:57:54.148051+0000","last_fullsized":"2024-09-20T18:57:54.148051+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:29.751642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:29.751642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":378,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.144437+0000","last_change":"2024-09-20T18:57:46.898929+0000","last_active":"2024-09-20T18:57:54.144437+0000","last_peered":"2024-09-20T18:57:54.144437+0000","last_clean":"2024-09-20T18:57:54.144437+0000","last_became_active":"2024-09-20T18:49:49.961947+0000","last_became_peered":"2024-09-20T18:49:49.961947+0000","last_unstale":"2024-09-20T18:57:54.144437+0000","last_undegraded":"2024-09-20T18:57:54.144437+0000","last_fullsized":"2024-09-20T18:57:54.144437+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:46.898862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:46.898862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:06.640666+0000","scrub_duration":38,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"66'1","reported_seq":376,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.163822+0000","last_change":"2024-09-20T18:54:20.681878+0000","last_active":"2024-09-20T18:57:54.163822+0000","last_peered":"2024-09-20T18:57:54.163822+0000","last_clean":"2024-09-20T18:57:54.163822+0000","last_became_active":"2024-09-20T18:49:51.965121+0000","last_became_peered":"2024-09-20T18:49:51.965121+0000","last_unstale":"2024-09-20T18:57:54.163822+0000","last_undegraded":"2024-09-20T18:57:54.163822+0000","last_fullsized":"2024-09-20T18:57:54.163822+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T18:54:20.681790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:20.681790+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":389,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:59.430353+0000","last_change":"2024-09-20T18:57:59.430353+0000","last_active":"2024-09-20T18:57:59.430353+0000","last_peered":"2024-09-20T18:57:59.430353+0000","last_clean":"2024-09-20T18:57:59.430353+0000","last_became_active":"2024-09-20T18:49:48.501011+0000","last_became_peered":"2024-09-20T18:49:48.501011+0000","last_unstale":"2024-09-20T18:57:59.430353+0000","last_undegraded":"2024-09-20T18:57:59.430353+0000","last_fullsized":"2024-09-20T18:57:59.430353+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:57:59.430294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:59.430294+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:21.333603+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":385,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.160661+0000","last_change":"2024-09-20T18:55:27.028184+0000","last_active":"2024-09-20T18:57:54.160661+0000","last_peered":"2024-09-20T18:57:54.160661+0000","last_clean":"2024-09-20T18:57:54.160661+0000","last_became_active":"2024-09-20T18:49:45.938511+0000","last_became_peered":"2024-09-20T18:49:45.938511+0000","last_unstale":"2024-09-20T18:57:54.160661+0000","last_undegraded":"2024-09-20T18:57:54.160661+0000","last_fullsized":"2024-09-20T18:57:54.160661+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:27.028126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:27.028126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":486,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:56.090709+0000","last_change":"2024-09-20T18:55:27.284156+0000","last_active":"2024-09-20T18:57:56.090709+0000","last_peered":"2024-09-20T18:57:56.090709+0000","last_clean":"2024-09-20T18:57:56.090709+0000","last_became_active":"2024-09-20T18:49:49.960550+0000","last_became_peered":"2024-09-20T18:49:49.960550+0000","last_unstale":"2024-09-20T18:57:56.090709+0000","last_undegraded":"2024-09-20T18:57:56.090709+0000","last_fullsized":"2024-09-20T18:57:56.090709+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:27.284086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:27.284086+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":374,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.134187+0000","last_change":"2024-09-20T18:55:12.348079+0000","last_active":"2024-09-20T18:57:54.134187+0000","last_peered":"2024-09-20T18:57:54.134187+0000","last_clean":"2024-09-20T18:57:54.134187+0000","last_became_active":"2024-09-20T18:49:51.966254+0000","last_became_peered":"2024-09-20T18:49:51.966254+0000","last_unstale":"2024-09-20T18:57:54.134187+0000","last_undegraded":"2024-09-20T18:57:54.134187+0000","last_fullsized":"2024-09-20T18:57:54.134187+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:12.347993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:12.347993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":408,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.142497+0000","last_change":"2024-09-20T18:57:20.701592+0000","last_active":"2024-09-20T18:57:54.142497+0000","last_peered":"2024-09-20T18:57:54.142497+0000","last_clean":"2024-09-20T18:57:54.142497+0000","last_became_active":"2024-09-20T18:49:58.997494+0000","last_became_peered":"2024-09-20T18:49:58.997494+0000","last_unstale":"2024-09-20T18:57:54.142497+0000","last_undegraded":"2024-09-20T18:57:54.142497+0000","last_fullsized":"2024-09-20T18:57:54.142497+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:57:20.701543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:20.701543+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:44.352173+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":385,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.148300+0000","last_change":"2024-09-20T18:54:57.632458+0000","last_active":"2024-09-20T18:57:54.148300+0000","last_peered":"2024-09-20T18:57:54.148300+0000","last_clean":"2024-09-20T18:57:54.148300+0000","last_became_active":"2024-09-20T18:49:45.949492+0000","last_became_peered":"2024-09-20T18:49:45.949492+0000","last_unstale":"2024-09-20T18:57:54.148300+0000","last_undegraded":"2024-09-20T18:57:54.148300+0000","last_fullsized":"2024-09-20T18:57:54.148300+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.632380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.632380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":484,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.146468+0000","last_change":"2024-09-20T18:55:28.302353+0000","last_active":"2024-09-20T18:57:54.146468+0000","last_peered":"2024-09-20T18:57:54.146468+0000","last_clean":"2024-09-20T18:57:54.146468+0000","last_became_active":"2024-09-20T18:49:49.962243+0000","last_became_peered":"2024-09-20T18:49:49.962243+0000","last_unstale":"2024-09-20T18:57:54.146468+0000","last_undegraded":"2024-09-20T18:57:54.146468+0000","last_fullsized":"2024-09-20T18:57:54.146468+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:28.302303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:28.302303+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":373,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.163155+0000","last_change":"2024-09-20T18:55:11.918284+0000","last_active":"2024-09-20T18:57:54.163155+0000","last_peered":"2024-09-20T18:57:54.163155+0000","last_clean":"2024-09-20T18:57:54.163155+0000","last_became_active":"2024-09-20T18:49:51.969093+0000","last_became_peered":"2024-09-20T18:49:51.969093+0000","last_unstale":"2024-09-20T18:57:54.163155+0000","last_undegraded":"2024-09-20T18:57:54.163155+0000","last_fullsized":"2024-09-20T18:57:54.163155+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:11.918210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:11.918210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":420,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.134017+0000","last_change":"2024-09-20T18:57:50.899576+0000","last_active":"2024-09-20T18:57:54.134017+0000","last_peered":"2024-09-20T18:57:54.134017+0000","last_clean":"2024-09-20T18:57:54.134017+0000","last_became_active":"2024-09-20T18:55:28.826721+0000","last_became_peered":"2024-09-20T18:55:28.826721+0000","last_unstale":"2024-09-20T18:57:54.134017+0000","last_undegraded":"2024-09-20T18:57:54.134017+0000","last_fullsized":"2024-09-20T18:57:54.134017+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:57:50.899514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:50.899514+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:12.210010+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":377,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.134350+0000","last_change":"2024-09-20T18:55:28.822690+0000","last_active":"2024-09-20T18:57:54.134350+0000","last_peered":"2024-09-20T18:57:54.134350+0000","last_clean":"2024-09-20T18:57:54.134350+0000","last_became_active":"2024-09-20T18:55:28.822258+0000","last_became_peered":"2024-09-20T18:55:28.822258+0000","last_unstale":"2024-09-20T18:57:54.134350+0000","last_undegraded":"2024-09-20T18:57:54.134350+0000","last_fullsized":"2024-09-20T18:57:54.134350+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:05.985551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.985551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:37.926823+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":370,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.162832+0000","last_change":"2024-09-20T18:57:11.986605+0000","last_active":"2024-09-20T18:57:54.162832+0000","last_peered":"2024-09-20T18:57:54.162832+0000","last_clean":"2024-09-20T18:57:54.162832+0000","last_became_active":"2024-09-20T18:49:49.957067+0000","last_became_peered":"2024-09-20T18:49:49.957067+0000","last_unstale":"2024-09-20T18:57:54.162832+0000","last_undegraded":"2024-09-20T18:57:54.162832+0000","last_fullsized":"2024-09-20T18:57:54.162832+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:11.986528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:11.986528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:36.007001+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":366,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.141521+0000","last_change":"2024-09-20T18:57:16.789412+0000","last_active":"2024-09-20T18:57:54.141521+0000","last_peered":"2024-09-20T18:57:54.141521+0000","last_clean":"2024-09-20T18:57:54.141521+0000","last_became_active":"2024-09-20T18:49:51.966915+0000","last_became_peered":"2024-09-20T18:49:51.966915+0000","last_unstale":"2024-09-20T18:57:54.141521+0000","last_undegraded":"2024-09-20T18:57:54.141521+0000","last_fullsized":"2024-09-20T18:57:54.141521+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:16.789095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:16.789095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:45.821909+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":415,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.164180+0000","last_change":"2024-09-20T18:55:28.838698+0000","last_active":"2024-09-20T18:57:54.164180+0000","last_peered":"2024-09-20T18:57:54.164180+0000","last_clean":"2024-09-20T18:57:54.164180+0000","last_became_active":"2024-09-20T18:55:28.827023+0000","last_became_peered":"2024-09-20T18:55:28.827023+0000","last_unstale":"2024-09-20T18:57:54.164180+0000","last_undegraded":"2024-09-20T18:57:54.164180+0000","last_fullsized":"2024-09-20T18:57:54.164180+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:53:53.263976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:53.263976+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:26.131891+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":377,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.148892+0000","last_change":"2024-09-20T18:54:42.487465+0000","last_active":"2024-09-20T18:57:54.148892+0000","last_peered":"2024-09-20T18:57:54.148892+0000","last_clean":"2024-09-20T18:57:54.148892+0000","last_became_active":"2024-09-20T18:49:45.944357+0000","last_became_peered":"2024-09-20T18:49:45.944357+0000","last_unstale":"2024-09-20T18:57:54.148892+0000","last_undegraded":"2024-09-20T18:57:54.148892+0000","last_fullsized":"2024-09-20T18:57:54.148892+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:42.487403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:42.487403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":378,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:57.915090+0000","last_change":"2024-09-20T18:57:57.915090+0000","last_active":"2024-09-20T18:57:57.915090+0000","last_peered":"2024-09-20T18:57:57.915090+0000","last_clean":"2024-09-20T18:57:57.915090+0000","last_became_active":"2024-09-20T18:49:49.962523+0000","last_became_peered":"2024-09-20T18:49:49.962523+0000","last_unstale":"2024-09-20T18:57:57.915090+0000","last_undegraded":"2024-09-20T18:57:57.915090+0000","last_fullsized":"2024-09-20T18:57:57.915090+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:57.914913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:57.914913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:01.797575+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":373,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.159650+0000","last_change":"2024-09-20T18:54:26.882441+0000","last_active":"2024-09-20T18:57:54.159650+0000","last_peered":"2024-09-20T18:57:54.159650+0000","last_clean":"2024-09-20T18:57:54.159650+0000","last_became_active":"2024-09-20T18:49:51.967313+0000","last_became_peered":"2024-09-20T18:49:51.967313+0000","last_unstale":"2024-09-20T18:57:54.159650+0000","last_undegraded":"2024-09-20T18:57:54.159650+0000","last_fullsized":"2024-09-20T18:57:54.159650+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:26.882375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:26.882375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":420,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.862785+0000","last_change":"2024-09-20T18:57:54.862785+0000","last_active":"2024-09-20T18:57:54.862785+0000","last_peered":"2024-09-20T18:57:54.862785+0000","last_clean":"2024-09-20T18:57:54.862785+0000","last_became_active":"2024-09-20T18:55:28.830801+0000","last_became_peered":"2024-09-20T18:55:28.830801+0000","last_unstale":"2024-09-20T18:57:54.862785+0000","last_undegraded":"2024-09-20T18:57:54.862785+0000","last_fullsized":"2024-09-20T18:57:54.862785+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:57:54.862329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:54.862329+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:09.269532+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":385,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.146754+0000","last_change":"2024-09-20T18:55:19.258723+0000","last_active":"2024-09-20T18:57:54.146754+0000","last_peered":"2024-09-20T18:57:54.146754+0000","last_clean":"2024-09-20T18:57:54.146754+0000","last_became_active":"2024-09-20T18:49:45.942105+0000","last_became_peered":"2024-09-20T18:49:45.942105+0000","last_unstale":"2024-09-20T18:57:54.146754+0000","last_undegraded":"2024-09-20T18:57:54.146754+0000","last_fullsized":"2024-09-20T18:57:54.146754+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:19.258674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:19.258674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":384,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.148168+0000","last_change":"2024-09-20T18:55:28.825004+0000","last_active":"2024-09-20T18:57:54.148168+0000","last_peered":"2024-09-20T18:57:54.148168+0000","last_clean":"2024-09-20T18:57:54.148168+0000","last_became_active":"2024-09-20T18:55:28.824355+0000","last_became_peered":"2024-09-20T18:55:28.824355+0000","last_unstale":"2024-09-20T18:57:54.148168+0000","last_undegraded":"2024-09-20T18:57:54.148168+0000","last_fullsized":"2024-09-20T18:57:54.148168+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:46.329547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:46.329547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":365,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.143928+0000","last_change":"2024-09-20T18:55:28.825607+0000","last_active":"2024-09-20T18:57:54.143928+0000","last_peered":"2024-09-20T18:57:54.143928+0000","last_clean":"2024-09-20T18:57:54.143928+0000","last_became_active":"2024-09-20T18:55:28.825296+0000","last_became_peered":"2024-09-20T18:55:28.825296+0000","last_unstale":"2024-09-20T18:57:54.143928+0000","last_undegraded":"2024-09-20T18:57:54.143928+0000","last_fullsized":"2024-09-20T18:57:54.143928+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:43.118805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:43.118805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:25.734066+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":401,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.149930+0000","last_change":"2024-09-20T18:55:28.836789+0000","last_active":"2024-09-20T18:57:54.149930+0000","last_peered":"2024-09-20T18:57:54.149930+0000","last_clean":"2024-09-20T18:57:54.149930+0000","last_became_active":"2024-09-20T18:55:28.824579+0000","last_became_peered":"2024-09-20T18:55:28.824579+0000","last_unstale":"2024-09-20T18:57:54.149930+0000","last_undegraded":"2024-09-20T18:57:54.149930+0000","last_fullsized":"2024-09-20T18:57:54.149930+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-20T18:55:20.282193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:20.282193+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":385,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.160180+0000","last_change":"2024-09-20T18:54:55.900487+0000","last_active":"2024-09-20T18:57:54.160180+0000","last_peered":"2024-09-20T18:57:54.160180+0000","last_clean":"2024-09-20T18:57:54.160180+0000","last_became_active":"2024-09-20T18:49:45.940988+0000","last_became_peered":"2024-09-20T18:49:45.940988+0000","last_unstale":"2024-09-20T18:57:54.160180+0000","last_undegraded":"2024-09-20T18:57:54.160180+0000","last_fullsized":"2024-09-20T18:57:54.160180+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:55.899611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:55.899611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":124,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.133629+0000","last_change":"2024-09-20T18:57:52.929891+0000","last_active":"2024-09-20T18:57:54.133629+0000","last_peered":"2024-09-20T18:57:54.133629+0000","last_clean":"2024-09-20T18:57:54.133629+0000","last_became_active":"2024-09-20T18:55:28.832534+0000","last_became_peered":"2024-09-20T18:55:28.832534+0000","last_unstale":"2024-09-20T18:57:54.133629+0000","last_undegraded":"2024-09-20T18:57:54.133629+0000","last_fullsized":"2024-09-20T18:57:54.133629+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:52.929828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:52.929828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:13.316360+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":357,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.147822+0000","last_change":"2024-09-20T18:51:09.761331+0000","last_active":"2024-09-20T18:57:54.147822+0000","last_peered":"2024-09-20T18:57:54.147822+0000","last_clean":"2024-09-20T18:57:54.147822+0000","last_became_active":"2024-09-20T18:49:51.968662+0000","last_became_peered":"2024-09-20T18:49:51.968662+0000","last_unstale":"2024-09-20T18:57:54.147822+0000","last_undegraded":"2024-09-20T18:57:54.147822+0000","last_fullsized":"2024-09-20T18:57:54.147822+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:09.761283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:09.761283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":358,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.166259+0000","last_change":"2024-09-20T18:57:10.970843+0000","last_active":"2024-09-20T18:57:54.166259+0000","last_peered":"2024-09-20T18:57:54.166259+0000","last_clean":"2024-09-20T18:57:54.166259+0000","last_became_active":"2024-09-20T18:49:51.968216+0000","last_became_peered":"2024-09-20T18:49:51.968216+0000","last_unstale":"2024-09-20T18:57:54.166259+0000","last_undegraded":"2024-09-20T18:57:54.166259+0000","last_fullsized":"2024-09-20T18:57:54.166259+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:10.970779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:10.970779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:38.733512+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":378,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.150916+0000","last_change":"2024-09-20T18:55:28.295445+0000","last_active":"2024-09-20T18:57:54.150916+0000","last_peered":"2024-09-20T18:57:54.150916+0000","last_clean":"2024-09-20T18:57:54.150916+0000","last_became_active":"2024-09-20T18:49:45.937992+0000","last_became_peered":"2024-09-20T18:49:45.937992+0000","last_unstale":"2024-09-20T18:57:54.150916+0000","last_undegraded":"2024-09-20T18:57:54.150916+0000","last_fullsized":"2024-09-20T18:57:54.150916+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:28.295399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:28.295399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":167,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.146467+0000","last_change":"2024-09-20T18:55:28.852468+0000","last_active":"2024-09-20T18:57:54.146467+0000","last_peered":"2024-09-20T18:57:54.146467+0000","last_clean":"2024-09-20T18:57:54.146467+0000","last_became_active":"2024-09-20T18:55:28.834675+0000","last_became_peered":"2024-09-20T18:55:28.834675+0000","last_unstale":"2024-09-20T18:57:54.146467+0000","last_undegraded":"2024-09-20T18:57:54.146467+0000","last_fullsized":"2024-09-20T18:57:54.146467+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:54:44.719676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.719676+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:38.535671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":12,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":362,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.163404+0000","last_change":"2024-09-20T18:51:28.120100+0000","last_active":"2024-09-20T18:57:54.163404+0000","last_peered":"2024-09-20T18:57:54.163404+0000","last_clean":"2024-09-20T18:57:54.163404+0000","last_became_active":"2024-09-20T18:49:49.957582+0000","last_became_peered":"2024-09-20T18:49:49.957582+0000","last_unstale":"2024-09-20T18:57:54.163404+0000","last_undegraded":"2024-09-20T18:57:54.163404+0000","last_fullsized":"2024-09-20T18:57:54.163404+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:28.119694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:28.119694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":950992920,"num_objects":14512,"num_object_clones":0,"num_object_copies":43536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14512,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":217666,"num_write_kb":7429633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26111,"num_bytes_recovered":1675296768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4135792640,"data_stored":4135780424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74399,"ondisk_log_size":74399,"up":48,"acting":47,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":67,"ondisk_log_size":67,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":695,"num_read_kb":452,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":1474,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":12140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":394,"ondisk_log_size":394,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":5275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":489,"num_read_kb":484,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266791,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":515740,"kb_used_data":304076,"kb_used_omap":1,"kb_used_meta":211646,"kb_avail":93229412,"statfs":{"total":95995035648,"available":95466917888,"internally_reserved":0,"allocated":311373824,"data_stored":310896737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":216725962},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":0,"seq":193273528430,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757363,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":751252,"kb_used_data":489604,"kb_used_omap":1,"kb_used_meta":261630,"kb_avail":92993900,"statfs":{"total":95995035648,"available":95225753600,"internally_reserved":0,"allocated":501354496,"data_stored":500885084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":267909573},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,1,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986296,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":528040,"kb_used_data":303896,"kb_used_omap":1,"kb_used_meta":224126,"kb_avail":93217112,"statfs":{"total":95995035648,"available":95454322688,"internally_reserved":0,"allocated":311189504,"data_stored":310700620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":229505477},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182527,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":781352,"kb_used_data":416920,"kb_used_omap":1,"kb_used_meta":364350,"kb_avail":92963800,"statfs":{"total":95995035648,"available":95194931200,"internally_reserved":0,"allocated":426926080,"data_stored":426436150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":373094855},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411460,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":890828,"kb_used_data":520892,"kb_used_omap":1,"kb_used_meta":369918,"kb_avail":92854324,"statfs":{"total":95995035648,"available":95082827776,"internally_reserved":0,"allocated":533393408,"data_stored":532932437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":378796485},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574986,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":713792,"kb_used_data":415920,"kb_used_omap":1,"kb_used_meta":297726,"kb_avail":93031360,"statfs":{"total":95995035648,"available":95264112640,"internally_reserved":0,"allocated":425902080,"data_stored":425390001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":304871877},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738512,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":742376,"kb_used_data":404312,"kb_used_omap":1,"kb_used_meta":337982,"kb_avail":93002776,"statfs":{"total":95995035648,"available":95234842624,"internally_reserved":0,"allocated":414015488,"data_stored":413526178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":346094021},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,4,6],"upper_bound":512},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":401735680,"data_stored":401735680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":411697152,"data_stored":411697152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":531501056,"data_stored":531496984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":408027136,"data_stored":408027136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":309133312,"data_stored":309133312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":490606592,"data_stored":490602520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1274085376,"data_stored":1274085376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":309006336,"data_stored":309002264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T18:58:07.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:07 smithi083 ceph-mon[36260]: 6.a scrub starts 2024-09-20T18:58:07.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:07 smithi083 ceph-mon[36260]: 6.a scrub ok 2024-09-20T18:58:07.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:07 smithi083 ceph-mon[36260]: from='client.25376 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:58:07.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:07 smithi083 ceph-mon[36260]: 4.4 scrub starts 2024-09-20T18:58:07.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:07 smithi083 ceph-mon[36260]: 4.4 scrub ok 2024-09-20T18:58:07.294 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:07.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:07.336 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:07.336 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:07.437 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:07.475 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:07.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:07.476 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:07.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:07 smithi142 ceph-mon[32508]: 6.a scrub starts 2024-09-20T18:58:07.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:07 smithi142 ceph-mon[32508]: 6.a scrub ok 2024-09-20T18:58:07.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:07 smithi142 ceph-mon[32508]: from='client.25376 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:58:07.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:07 smithi142 ceph-mon[32508]: 4.4 scrub starts 2024-09-20T18:58:07.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:07 smithi142 ceph-mon[32508]: 4.4 scrub ok 2024-09-20T18:58:07.577 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:07.614 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:07.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:07.615 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:07.716 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:07.743 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:07.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:07.744 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:07.845 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:07.875 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:07.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:07.876 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:07.977 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:08.003 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:08.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:08.004 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:08.105 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:58:08.140 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:08.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:08.141 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:58:08.242 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:08.266 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:08.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:08.267 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:08.367 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:08.392 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:08.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:08.393 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:08.492 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:08 smithi083 ceph-mon[29421]: 3.16 scrub starts 2024-09-20T18:58:08.492 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:08 smithi083 ceph-mon[29421]: 3.16 scrub ok 2024-09-20T18:58:08.492 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:08 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:58:08.492 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:08 smithi083 ceph-mon[29421]: 5.18 scrub starts 2024-09-20T18:58:08.492 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:08 smithi083 ceph-mon[29421]: 5.18 scrub ok 2024-09-20T18:58:08.492 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:08 smithi083 ceph-mon[29421]: pgmap v398: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 658 MiB data, 4.0 GiB used, 622 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.15k op/s; 1945/32163 objects degraded (6.047%); 0 B/s, 23 objects/s recovering 2024-09-20T18:58:08.492 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:08 smithi083 ceph-mon[29421]: 6.4 scrub starts 2024-09-20T18:58:08.492 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:08 smithi083 ceph-mon[29421]: 6.4 scrub ok 2024-09-20T18:58:08.492 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:08 smithi083 ceph-mon[29421]: 4.0 scrub starts 2024-09-20T18:58:08.493 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:08 smithi083 ceph-mon[29421]: 4.0 scrub ok 2024-09-20T18:58:08.493 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:08 smithi083 ceph-mon[36260]: 3.16 scrub starts 2024-09-20T18:58:08.493 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:08 smithi083 ceph-mon[36260]: 3.16 scrub ok 2024-09-20T18:58:08.493 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:08 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:58:08.493 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:08 smithi083 ceph-mon[36260]: 5.18 scrub starts 2024-09-20T18:58:08.493 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:08 smithi083 ceph-mon[36260]: 5.18 scrub ok 2024-09-20T18:58:08.493 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:08 smithi083 ceph-mon[36260]: pgmap v398: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 658 MiB data, 4.0 GiB used, 622 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.15k op/s; 1945/32163 objects degraded (6.047%); 0 B/s, 23 objects/s recovering 2024-09-20T18:58:08.493 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:08 smithi083 ceph-mon[36260]: 6.4 scrub starts 2024-09-20T18:58:08.493 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:08 smithi083 ceph-mon[36260]: 6.4 scrub ok 2024-09-20T18:58:08.493 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:08 smithi083 ceph-mon[36260]: 4.0 scrub starts 2024-09-20T18:58:08.494 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:08 smithi083 ceph-mon[36260]: 4.0 scrub ok 2024-09-20T18:58:08.494 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:08.517 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:08.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:08.518 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:08.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:08 smithi142 ceph-mon[32508]: 3.16 scrub starts 2024-09-20T18:58:08.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:08 smithi142 ceph-mon[32508]: 3.16 scrub ok 2024-09-20T18:58:08.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:08 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:58:08.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:08 smithi142 ceph-mon[32508]: 5.18 scrub starts 2024-09-20T18:58:08.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:08 smithi142 ceph-mon[32508]: 5.18 scrub ok 2024-09-20T18:58:08.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:08 smithi142 ceph-mon[32508]: pgmap v398: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 658 MiB data, 4.0 GiB used, 622 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.15k op/s; 1945/32163 objects degraded (6.047%); 0 B/s, 23 objects/s recovering 2024-09-20T18:58:08.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:08 smithi142 ceph-mon[32508]: 6.4 scrub starts 2024-09-20T18:58:08.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:08 smithi142 ceph-mon[32508]: 6.4 scrub ok 2024-09-20T18:58:08.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:08 smithi142 ceph-mon[32508]: 4.0 scrub starts 2024-09-20T18:58:08.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:08 smithi142 ceph-mon[32508]: 4.0 scrub ok 2024-09-20T18:58:08.544 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-20T18:58:08.619 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:08.628 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:58:08.684 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:08.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:08.685 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:08.694 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:58:08.785 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:08.816 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:08.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:08.817 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:08.886 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:58:08.918 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:08.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:08.948 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:08.948 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:09.049 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:09.072 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:09.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:09.073 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:09.174 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:09.221 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:09.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:09.222 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:09.268 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:09 smithi142 ceph-mon[32508]: 8.5 scrub starts 2024-09-20T18:58:09.268 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:09 smithi142 ceph-mon[32508]: 8.5 scrub ok 2024-09-20T18:58:09.268 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:09 smithi142 ceph-mon[32508]: 4.17 scrub starts 2024-09-20T18:58:09.268 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:09 smithi142 ceph-mon[32508]: 4.17 scrub ok 2024-09-20T18:58:09.323 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:09.355 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:09.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:09.357 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:09.430 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:09 smithi083 ceph-mon[29421]: 8.5 scrub starts 2024-09-20T18:58:09.430 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:09 smithi083 ceph-mon[29421]: 8.5 scrub ok 2024-09-20T18:58:09.430 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:09 smithi083 ceph-mon[29421]: 4.17 scrub starts 2024-09-20T18:58:09.430 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:09 smithi083 ceph-mon[29421]: 4.17 scrub ok 2024-09-20T18:58:09.431 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:09 smithi083 ceph-mon[36260]: 8.5 scrub starts 2024-09-20T18:58:09.431 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:09 smithi083 ceph-mon[36260]: 8.5 scrub ok 2024-09-20T18:58:09.431 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:09 smithi083 ceph-mon[36260]: 4.17 scrub starts 2024-09-20T18:58:09.431 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:09 smithi083 ceph-mon[36260]: 4.17 scrub ok 2024-09-20T18:58:09.457 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:58:09.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:09.503 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:09.503 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:58:09.604 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:58:09.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:09.648 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:09.649 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:58:09.673 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T18:58:09.673 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-09-20T18:58:09.673 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:09.673 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.24955.0:225706 8.5 8:a86809b5:::benchmark_data_smithi083_97422_object109615:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-20T18:58:09.673 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:58:09.389148+0000", 2024-09-20T18:58:09.673 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.0081129989999999992, 2024-09-20T18:58:09.673 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0081405130000000003, 2024-09-20T18:58:09.673 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:58:09.674 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:58:09.674 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "waiting for sub ops", 2024-09-20T18:58:09.674 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:58:09.674 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.24955", 2024-09-20T18:58:09.674 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:58:09.674 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 225706 2024-09-20T18:58:09.674 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:09.674 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:58:09.674 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:09.674 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:58:09.674 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:09.389148+0000", 2024-09-20T18:58:09.674 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:58:09.674 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:09.674 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:09.675 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:58:09.675 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:09.389148+0000", 2024-09-20T18:58:09.675 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:58:09.675 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:09.675 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:09.675 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:58:09.675 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:09.389149+0000", 2024-09-20T18:58:09.675 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5710000000000001e-06 2024-09-20T18:58:09.675 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:09.675 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:09.675 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:58:09.675 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:09.389155+0000", 2024-09-20T18:58:09.675 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6699999999999999e-06 2024-09-20T18:58:09.675 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:09.676 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:09.676 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:58:09.676 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:09.389166+0000", 2024-09-20T18:58:09.676 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1079e-05 2024-09-20T18:58:09.676 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:09.676 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:09.676 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:58:09.676 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:09.389172+0000", 2024-09-20T18:58:09.676 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8749999999999997e-06 2024-09-20T18:58:09.676 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:09.676 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:09.676 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:58:09.676 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:09.389223+0000", 2024-09-20T18:58:09.676 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.1456000000000003e-05 2024-09-20T18:58:09.677 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:09.677 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:09.677 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:58:09.677 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:09.389333+0000", 2024-09-20T18:58:09.677 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00010966199999999999 2024-09-20T18:58:09.677 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:09.677 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:09.677 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,7", 2024-09-20T18:58:09.677 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:09.389441+0000", 2024-09-20T18:58:09.677 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000107707 2024-09-20T18:58:09.677 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:09.677 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:09.677 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:58:09.677 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:09.391354+0000", 2024-09-20T18:58:09.677 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001913699 2024-09-20T18:58:09.677 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:09.678 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:09.678 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:09.678 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:09.391735+0000", 2024-09-20T18:58:09.678 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00038099999999999999 2024-09-20T18:58:09.678 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:58:09.678 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:58:09.678 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:58:09.678 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:58:09.678 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-09-20T18:58:09.678 INFO:teuthology.orchestra.run.smithi142.stdout: "num_ops": 1 2024-09-20T18:58:09.678 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T18:58:09.750 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:09.767 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:58:09.767 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:58:09.776 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:09.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:09.777 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:09.808 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:58:09.808 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:58:09.846 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:58:09.846 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:58:09.878 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:58:09.879 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:58:09.879 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:58:09.910 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:58:09.910 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:58:09.919 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:09.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:09.921 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:58:09.948 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:58:09.948 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:58:09.979 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:58:09.979 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:58:10.015 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:58:10.015 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:58:10.022 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:10.060 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:10.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:10.061 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:10.162 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:10.201 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:10.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:10.202 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:10.249 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-20T18:58:10.293 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T18:58:10.302 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:10.329 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:10.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:10.330 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:10.431 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:10.457 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:10.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:10.458 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:10.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:10 smithi142 ceph-mon[32508]: pgmap v399: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 602 MiB data, 3.9 GiB used, 622 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 815 op/s; 1770/29490 objects degraded (6.002%); 0 B/s, 25 objects/s recovering 2024-09-20T18:58:10.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:10 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/361636754' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:58:10.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:10 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1852088077' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:58:10.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:10 smithi142 ceph-mon[32508]: 8.f scrub starts 2024-09-20T18:58:10.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:10 smithi083 ceph-mon[29421]: pgmap v399: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 602 MiB data, 3.9 GiB used, 622 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 815 op/s; 1770/29490 objects degraded (6.002%); 0 B/s, 25 objects/s recovering 2024-09-20T18:58:10.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:10 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/361636754' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:58:10.549 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:10 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1852088077' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:58:10.549 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:10 smithi083 ceph-mon[29421]: 8.f scrub starts 2024-09-20T18:58:10.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:10 smithi083 ceph-mon[36260]: pgmap v399: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 602 MiB data, 3.9 GiB used, 622 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 815 op/s; 1770/29490 objects degraded (6.002%); 0 B/s, 25 objects/s recovering 2024-09-20T18:58:10.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:10 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/361636754' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:58:10.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:10 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1852088077' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:58:10.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:10 smithi083 ceph-mon[36260]: 8.f scrub starts 2024-09-20T18:58:10.559 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:10.581 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:10.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:10.582 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:10.644 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_blocked_ops 2024-09-20T18:58:10.683 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:58:10.725 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:10.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:10.726 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:58:10.827 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:10.859 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:10.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:10.859 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:10.960 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:58:10.987 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:58:11.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:11.003 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:11.004 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:58:11.046 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:58:10 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:58:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:58:11.104 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:11.131 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:11.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:11.132 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:11.233 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:11.257 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:11.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:11.258 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:11.298 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:11 smithi083 ceph-mon[29421]: 3.1d scrub starts 2024-09-20T18:58:11.298 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:11 smithi083 ceph-mon[29421]: 3.1d scrub ok 2024-09-20T18:58:11.299 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:11 smithi083 ceph-mon[29421]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:58:11.299 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:11 smithi083 ceph-mon[29421]: 8.f scrub ok 2024-09-20T18:58:11.299 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:11 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/361636754' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T18:58:11.299 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:11 smithi083 ceph-mon[29421]: osdmap e234: 8 total, 8 up, 7 in 2024-09-20T18:58:11.299 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:11 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 1770/29490 objects degraded (6.002%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T18:58:11.299 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:11 smithi083 ceph-mon[29421]: osdmap e235: 8 total, 8 up, 7 in 2024-09-20T18:58:11.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:11 smithi083 ceph-mon[36260]: 3.1d scrub starts 2024-09-20T18:58:11.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:11 smithi083 ceph-mon[36260]: 3.1d scrub ok 2024-09-20T18:58:11.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:11 smithi083 ceph-mon[36260]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:58:11.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:11 smithi083 ceph-mon[36260]: 8.f scrub ok 2024-09-20T18:58:11.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:11 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/361636754' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T18:58:11.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:11 smithi083 ceph-mon[36260]: osdmap e234: 8 total, 8 up, 7 in 2024-09-20T18:58:11.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:11 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 1770/29490 objects degraded (6.002%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T18:58:11.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:11 smithi083 ceph-mon[36260]: osdmap e235: 8 total, 8 up, 7 in 2024-09-20T18:58:11.358 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:11.392 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:11.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:11.393 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:11.417 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:11 smithi142 ceph-mon[32508]: 3.1d scrub starts 2024-09-20T18:58:11.417 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:11 smithi142 ceph-mon[32508]: 3.1d scrub ok 2024-09-20T18:58:11.417 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:11 smithi142 ceph-mon[32508]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:58:11.418 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:11 smithi142 ceph-mon[32508]: 8.f scrub ok 2024-09-20T18:58:11.418 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:11 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/361636754' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T18:58:11.418 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:11 smithi142 ceph-mon[32508]: osdmap e234: 8 total, 8 up, 7 in 2024-09-20T18:58:11.418 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:11 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 1770/29490 objects degraded (6.002%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T18:58:11.418 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:11 smithi142 ceph-mon[32508]: osdmap e235: 8 total, 8 up, 7 in 2024-09-20T18:58:11.493 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:11.530 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:11.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:11.531 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:11.544 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T18:58:11.544 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:58:11.614 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:58:11.632 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:11.684 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:11.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:11.685 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:11.786 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:11.811 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:11.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:11.812 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:11.913 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:11.960 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:11.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:11.961 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:12.062 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:12.087 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:12.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:12.088 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:12.189 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:12.215 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:12.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:12.216 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:12.283 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_historic_ops 2024-09-20T18:58:12.317 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:12.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:12.354 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:12.354 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:12.454 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:12.481 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:12.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:12.482 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:12.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:12 smithi142 ceph-mon[32508]: pgmap v402: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 560 MiB data, 3.7 GiB used, 622 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 696 op/s; 1770/27441 objects degraded (6.450%); 0 B/s, 21 objects/s recovering 2024-09-20T18:58:12.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:12 smithi083 ceph-mon[29421]: pgmap v402: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 560 MiB data, 3.7 GiB used, 622 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 696 op/s; 1770/27441 objects degraded (6.450%); 0 B/s, 21 objects/s recovering 2024-09-20T18:58:12.548 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:12 smithi083 ceph-mon[36260]: pgmap v402: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 560 MiB data, 3.7 GiB used, 622 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 696 op/s; 1770/27441 objects degraded (6.450%); 0 B/s, 21 objects/s recovering 2024-09-20T18:58:12.583 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:12.607 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:58:12.609 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-20T18:58:12.610 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:58:12.619 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:12.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:12.620 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:12.721 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:12.745 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:12.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:12.746 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:12.847 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:12.883 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:12.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:12.884 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:12.985 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:13.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:13.021 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:13.021 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:13.104 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T18:58:13.122 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:13.178 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T18:58:13.178 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:58:13.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:13.189 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:13.189 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:13.290 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:13.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:13.335 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:13.335 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:13.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:13 smithi142 ceph-mon[32508]: from='client.25252 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:58:13.418 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:13 smithi083 ceph-mon[29421]: from='client.25252 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:58:13.418 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:13 smithi083 ceph-mon[36260]: from='client.25252 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:58:13.436 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:13.463 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:13.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:13.464 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:13.566 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:13.578 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T18:58:13.594 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:13.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:13.596 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:13.696 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:13.720 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:13.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:13.721 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:13.822 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:13.837 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-20T18:58:13.881 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:13.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:13.882 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:13.983 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:14.008 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:14.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:14.009 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:14.110 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:14.136 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:58:14.138 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:14.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:14.140 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:14.237 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":402,"stamp":"2024-09-20T18:58:10.959289+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":586947123,"num_objects":9147,"num_object_clones":0,"num_object_copies":27441,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1770,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9147,"num_whiteouts":0,"num_read":1345,"num_read_kb":1202,"num_write":223779,"num_write_kb":7431306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26381,"num_bytes_recovered":1676479433,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74719,"ondisk_log_size":74719,"up":444,"acting":443,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":3841124,"kb_used_data":1720744,"kb_used_omap":10,"kb_used_meta":2120117,"kb_avail":652374940,"statfs":{"total":671965249536,"available":668031938560,"internally_reserved":0,"allocated":1762041856,"data_stored":1758637878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11158,"internal_metadata":2170999914},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,5,8,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-364642304,"num_objects":-5564,"num_object_clones":0,"num_object_copies":-16692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-175,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5564,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":5564,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":175,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003136"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":385,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.160096+0000","last_change":"2024-09-20T18:55:28.037494+0000","last_active":"2024-09-20T18:57:54.160096+0000","last_peered":"2024-09-20T18:57:54.160096+0000","last_clean":"2024-09-20T18:57:54.160096+0000","last_became_active":"2024-09-20T18:49:45.943201+0000","last_became_peered":"2024-09-20T18:49:45.943201+0000","last_unstale":"2024-09-20T18:57:54.160096+0000","last_undegraded":"2024-09-20T18:57:54.160096+0000","last_fullsized":"2024-09-20T18:57:54.160096+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:28.037449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:28.037449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":436,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-20T18:58:10.730502+0000","last_change":"2024-09-20T18:55:28.838729+0000","last_active":"2024-09-20T18:58:10.730502+0000","last_peered":"2024-09-20T18:58:10.730502+0000","last_clean":"2024-09-20T18:58:10.730502+0000","last_became_active":"2024-09-20T18:55:28.819637+0000","last_became_peered":"2024-09-20T18:55:28.819637+0000","last_unstale":"2024-09-20T18:58:10.730502+0000","last_undegraded":"2024-09-20T18:58:10.730502+0000","last_fullsized":"2024-09-20T18:58:10.730502+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:50.231620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:50.231620+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":386,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.162837+0000","last_change":"2024-09-20T18:57:49.016078+0000","last_active":"2024-09-20T18:57:54.162837+0000","last_peered":"2024-09-20T18:57:54.162837+0000","last_clean":"2024-09-20T18:57:54.162837+0000","last_became_active":"2024-09-20T18:49:49.959514+0000","last_became_peered":"2024-09-20T18:49:49.959514+0000","last_unstale":"2024-09-20T18:57:54.162837+0000","last_undegraded":"2024-09-20T18:57:54.162837+0000","last_fullsized":"2024-09-20T18:57:54.162837+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:49.016006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:49.016006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:13.297987+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":386,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-20T18:58:10.732440+0000","last_change":"2024-09-20T18:58:02.084435+0000","last_active":"2024-09-20T18:58:10.732440+0000","last_peered":"2024-09-20T18:58:10.732440+0000","last_clean":"2024-09-20T18:58:10.732440+0000","last_became_active":"2024-09-20T18:49:51.969421+0000","last_became_peered":"2024-09-20T18:49:51.969421+0000","last_unstale":"2024-09-20T18:58:10.732440+0000","last_undegraded":"2024-09-20T18:58:10.732440+0000","last_fullsized":"2024-09-20T18:58:10.732440+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:02.084375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:02.084375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:20.422560+0000","scrub_duration":197,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":381,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.143633+0000","last_change":"2024-09-20T18:55:28.822034+0000","last_active":"2024-09-20T18:57:54.143633+0000","last_peered":"2024-09-20T18:57:54.143633+0000","last_clean":"2024-09-20T18:57:54.143633+0000","last_became_active":"2024-09-20T18:55:28.821717+0000","last_became_peered":"2024-09-20T18:55:28.821717+0000","last_unstale":"2024-09-20T18:57:54.143633+0000","last_undegraded":"2024-09-20T18:57:54.143633+0000","last_fullsized":"2024-09-20T18:57:54.143633+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.216188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.216188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":386,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.142788+0000","last_change":"2024-09-20T18:57:22.694362+0000","last_active":"2024-09-20T18:57:54.142788+0000","last_peered":"2024-09-20T18:57:54.142788+0000","last_clean":"2024-09-20T18:57:54.142788+0000","last_became_active":"2024-09-20T18:55:28.824826+0000","last_became_peered":"2024-09-20T18:55:28.824826+0000","last_unstale":"2024-09-20T18:57:54.142788+0000","last_undegraded":"2024-09-20T18:57:54.142788+0000","last_fullsized":"2024-09-20T18:57:54.142788+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:22.694295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:22.694295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:33.473592+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":399,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.141935+0000","last_change":"2024-09-20T18:57:13.774827+0000","last_active":"2024-09-20T18:57:54.141935+0000","last_peered":"2024-09-20T18:57:54.141935+0000","last_clean":"2024-09-20T18:57:54.141935+0000","last_became_active":"2024-09-20T18:49:47.952150+0000","last_became_peered":"2024-09-20T18:49:47.952150+0000","last_unstale":"2024-09-20T18:57:54.141935+0000","last_undegraded":"2024-09-20T18:57:54.141935+0000","last_fullsized":"2024-09-20T18:57:54.141935+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:57:13.774305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:13.774305+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:25.245744+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":389,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-20T18:58:10.730406+0000","last_change":"2024-09-20T18:58:06.846086+0000","last_active":"2024-09-20T18:58:10.730406+0000","last_peered":"2024-09-20T18:58:10.730406+0000","last_clean":"2024-09-20T18:58:10.730406+0000","last_became_active":"2024-09-20T18:55:28.820362+0000","last_became_peered":"2024-09-20T18:55:28.820362+0000","last_unstale":"2024-09-20T18:58:10.730406+0000","last_undegraded":"2024-09-20T18:58:10.730406+0000","last_fullsized":"2024-09-20T18:58:10.730406+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:06.846018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:06.846018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:33.625936+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"59'1","reported_seq":410,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.146699+0000","last_change":"2024-09-20T18:55:29.340140+0000","last_active":"2024-09-20T18:57:54.146699+0000","last_peered":"2024-09-20T18:57:54.146699+0000","last_clean":"2024-09-20T18:57:54.146699+0000","last_became_active":"2024-09-20T18:55:28.833386+0000","last_became_peered":"2024-09-20T18:55:28.833386+0000","last_unstale":"2024-09-20T18:57:54.146699+0000","last_undegraded":"2024-09-20T18:57:54.146699+0000","last_fullsized":"2024-09-20T18:57:54.146699+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:55:29.340083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:29.340083+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":402,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-20T18:58:10.732839+0000","last_change":"2024-09-20T18:54:53.241889+0000","last_active":"2024-09-20T18:58:10.732839+0000","last_peered":"2024-09-20T18:58:10.732839+0000","last_clean":"2024-09-20T18:58:10.732839+0000","last_became_active":"2024-09-20T18:49:47.947448+0000","last_became_peered":"2024-09-20T18:49:47.947448+0000","last_unstale":"2024-09-20T18:58:10.732839+0000","last_undegraded":"2024-09-20T18:58:10.732839+0000","last_fullsized":"2024-09-20T18:58:10.732839+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:53.241802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:53.241802+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":377,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.145999+0000","last_change":"2024-09-20T18:57:08.539466+0000","last_active":"2024-09-20T18:57:54.145999+0000","last_peered":"2024-09-20T18:57:54.145999+0000","last_clean":"2024-09-20T18:57:54.145999+0000","last_became_active":"2024-09-20T18:49:49.961188+0000","last_became_peered":"2024-09-20T18:49:49.961188+0000","last_unstale":"2024-09-20T18:57:54.145999+0000","last_undegraded":"2024-09-20T18:57:54.145999+0000","last_fullsized":"2024-09-20T18:57:54.145999+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:08.539418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:08.539418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:13.102215+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":349,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.161262+0000","last_change":"2024-09-20T18:49:51.966957+0000","last_active":"2024-09-20T18:57:54.161262+0000","last_peered":"2024-09-20T18:57:54.161262+0000","last_clean":"2024-09-20T18:57:54.161262+0000","last_became_active":"2024-09-20T18:49:51.966715+0000","last_became_peered":"2024-09-20T18:49:51.966715+0000","last_unstale":"2024-09-20T18:57:54.161262+0000","last_undegraded":"2024-09-20T18:57:54.161262+0000","last_fullsized":"2024-09-20T18:57:54.161262+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":377,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.148831+0000","last_change":"2024-09-20T18:55:15.252839+0000","last_active":"2024-09-20T18:57:54.148831+0000","last_peered":"2024-09-20T18:57:54.148831+0000","last_clean":"2024-09-20T18:57:54.148831+0000","last_became_active":"2024-09-20T18:49:45.944703+0000","last_became_peered":"2024-09-20T18:49:45.944703+0000","last_unstale":"2024-09-20T18:57:54.148831+0000","last_undegraded":"2024-09-20T18:57:54.148831+0000","last_fullsized":"2024-09-20T18:57:54.148831+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:15.252782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:15.252782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":406,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-20T18:58:10.731520+0000","last_change":"2024-09-20T18:51:26.494194+0000","last_active":"2024-09-20T18:58:10.731520+0000","last_peered":"2024-09-20T18:58:10.731520+0000","last_clean":"2024-09-20T18:58:10.731520+0000","last_became_active":"2024-09-20T18:51:23.813399+0000","last_became_peered":"2024-09-20T18:51:23.813399+0000","last_unstale":"2024-09-20T18:58:10.731520+0000","last_undegraded":"2024-09-20T18:58:10.731520+0000","last_fullsized":"2024-09-20T18:58:10.731520+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-20T18:51:26.494118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:26.494118+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:31.887636+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":373,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-20T18:58:10.738187+0000","last_change":"2024-09-20T18:52:38.799542+0000","last_active":"2024-09-20T18:58:10.738187+0000","last_peered":"2024-09-20T18:58:10.738187+0000","last_clean":"2024-09-20T18:58:10.738187+0000","last_became_active":"2024-09-20T18:49:49.959696+0000","last_became_peered":"2024-09-20T18:49:49.959696+0000","last_unstale":"2024-09-20T18:58:10.738187+0000","last_undegraded":"2024-09-20T18:58:10.738187+0000","last_fullsized":"2024-09-20T18:58:10.738187+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:38.799487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:38.799487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":373,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.146587+0000","last_change":"2024-09-20T18:57:13.587827+0000","last_active":"2024-09-20T18:57:54.146587+0000","last_peered":"2024-09-20T18:57:54.146587+0000","last_clean":"2024-09-20T18:57:54.146587+0000","last_became_active":"2024-09-20T18:49:51.970663+0000","last_became_peered":"2024-09-20T18:49:51.970663+0000","last_unstale":"2024-09-20T18:57:54.146587+0000","last_undegraded":"2024-09-20T18:57:54.146587+0000","last_fullsized":"2024-09-20T18:57:54.146587+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:13.587774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:13.587774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:15.272228+0000","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":386,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-20T18:58:10.731349+0000","last_change":"2024-09-20T18:58:00.906697+0000","last_active":"2024-09-20T18:58:10.731349+0000","last_peered":"2024-09-20T18:58:10.731349+0000","last_clean":"2024-09-20T18:58:10.731349+0000","last_became_active":"2024-09-20T18:55:28.827435+0000","last_became_peered":"2024-09-20T18:55:28.827435+0000","last_unstale":"2024-09-20T18:58:10.731349+0000","last_undegraded":"2024-09-20T18:58:10.731349+0000","last_fullsized":"2024-09-20T18:58:10.731349+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:00.906652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:00.906652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:23.434725+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":377,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.162013+0000","last_change":"2024-09-20T18:54:23.984024+0000","last_active":"2024-09-20T18:57:54.162013+0000","last_peered":"2024-09-20T18:57:54.162013+0000","last_clean":"2024-09-20T18:57:54.162013+0000","last_became_active":"2024-09-20T18:49:45.950480+0000","last_became_peered":"2024-09-20T18:49:45.950480+0000","last_unstale":"2024-09-20T18:57:54.162013+0000","last_undegraded":"2024-09-20T18:57:54.162013+0000","last_fullsized":"2024-09-20T18:57:54.162013+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:23.983958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:23.983958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":400,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.165114+0000","last_change":"2024-09-20T18:53:26.762207+0000","last_active":"2024-09-20T18:57:54.165114+0000","last_peered":"2024-09-20T18:57:54.165114+0000","last_clean":"2024-09-20T18:57:54.165114+0000","last_became_active":"2024-09-20T18:49:47.950939+0000","last_became_peered":"2024-09-20T18:49:47.950939+0000","last_unstale":"2024-09-20T18:57:54.165114+0000","last_undegraded":"2024-09-20T18:57:54.165114+0000","last_fullsized":"2024-09-20T18:57:54.165114+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:53:26.762131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:26.762131+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":370,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.166553+0000","last_change":"2024-09-20T18:57:14.941518+0000","last_active":"2024-09-20T18:57:54.166553+0000","last_peered":"2024-09-20T18:57:54.166553+0000","last_clean":"2024-09-20T18:57:54.166553+0000","last_became_active":"2024-09-20T18:49:49.958518+0000","last_became_peered":"2024-09-20T18:49:49.958518+0000","last_unstale":"2024-09-20T18:57:54.166553+0000","last_undegraded":"2024-09-20T18:57:54.166553+0000","last_fullsized":"2024-09-20T18:57:54.166553+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:14.941458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:14.941458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:37.988570+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":373,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.144359+0000","last_change":"2024-09-20T18:55:28.833468+0000","last_active":"2024-09-20T18:57:54.144359+0000","last_peered":"2024-09-20T18:57:54.144359+0000","last_clean":"2024-09-20T18:57:54.144359+0000","last_became_active":"2024-09-20T18:55:28.833133+0000","last_became_peered":"2024-09-20T18:55:28.833133+0000","last_unstale":"2024-09-20T18:57:54.144359+0000","last_undegraded":"2024-09-20T18:57:54.144359+0000","last_fullsized":"2024-09-20T18:57:54.144359+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:54.259174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:54.259174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:30.790542+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"59'1","reported_seq":396,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-20T18:58:10.732921+0000","last_change":"2024-09-20T18:57:53.918080+0000","last_active":"2024-09-20T18:58:10.732921+0000","last_peered":"2024-09-20T18:58:10.732921+0000","last_clean":"2024-09-20T18:58:10.732921+0000","last_became_active":"2024-09-20T18:49:45.940100+0000","last_became_peered":"2024-09-20T18:49:45.940100+0000","last_unstale":"2024-09-20T18:58:10.732921+0000","last_undegraded":"2024-09-20T18:58:10.732921+0000","last_fullsized":"2024-09-20T18:58:10.732921+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:57:53.918022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:53.918022+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:03.242427+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":389,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.142243+0000","last_change":"2024-09-20T18:57:09.819080+0000","last_active":"2024-09-20T18:57:54.142243+0000","last_peered":"2024-09-20T18:57:54.142243+0000","last_clean":"2024-09-20T18:57:54.142243+0000","last_became_active":"2024-09-20T18:49:47.950909+0000","last_became_peered":"2024-09-20T18:49:47.950909+0000","last_unstale":"2024-09-20T18:57:54.142243+0000","last_undegraded":"2024-09-20T18:57:54.142243+0000","last_fullsized":"2024-09-20T18:57:54.142243+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:57:09.818999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:09.818999+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:17.216750+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":382,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-20T18:58:10.732078+0000","last_change":"2024-09-20T18:54:44.239273+0000","last_active":"2024-09-20T18:58:10.732078+0000","last_peered":"2024-09-20T18:58:10.732078+0000","last_clean":"2024-09-20T18:58:10.732078+0000","last_became_active":"2024-09-20T18:49:49.959102+0000","last_became_peered":"2024-09-20T18:49:49.959102+0000","last_unstale":"2024-09-20T18:58:10.732078+0000","last_undegraded":"2024-09-20T18:58:10.732078+0000","last_fullsized":"2024-09-20T18:58:10.732078+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:44.239226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.239226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":370,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-20T18:58:10.739557+0000","last_change":"2024-09-20T18:52:15.826819+0000","last_active":"2024-09-20T18:58:10.739557+0000","last_peered":"2024-09-20T18:58:10.739557+0000","last_clean":"2024-09-20T18:58:10.739557+0000","last_became_active":"2024-09-20T18:49:51.970058+0000","last_became_peered":"2024-09-20T18:49:51.970058+0000","last_unstale":"2024-09-20T18:58:10.739557+0000","last_undegraded":"2024-09-20T18:58:10.739557+0000","last_fullsized":"2024-09-20T18:58:10.739557+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T18:52:15.826638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:15.826638+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":379,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.162995+0000","last_change":"2024-09-20T18:57:18.032140+0000","last_active":"2024-09-20T18:57:54.162995+0000","last_peered":"2024-09-20T18:57:54.162995+0000","last_clean":"2024-09-20T18:57:54.162995+0000","last_became_active":"2024-09-20T18:49:45.933078+0000","last_became_peered":"2024-09-20T18:49:45.933078+0000","last_unstale":"2024-09-20T18:57:54.162995+0000","last_undegraded":"2024-09-20T18:57:54.162995+0000","last_fullsized":"2024-09-20T18:57:54.162995+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:57:18.032067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:18.032067+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:42.727785+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":397,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.163486+0000","last_change":"2024-09-20T18:55:19.996522+0000","last_active":"2024-09-20T18:57:54.163486+0000","last_peered":"2024-09-20T18:57:54.163486+0000","last_clean":"2024-09-20T18:57:54.163486+0000","last_became_active":"2024-09-20T18:49:48.500809+0000","last_became_peered":"2024-09-20T18:49:48.500809+0000","last_unstale":"2024-09-20T18:57:54.163486+0000","last_undegraded":"2024-09-20T18:57:54.163486+0000","last_fullsized":"2024-09-20T18:57:54.163486+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:55:19.996328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:19.996328+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"184'9","reported_seq":193,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-20T18:58:10.730907+0000","last_change":"2024-09-20T18:57:48.853376+0000","last_active":"2024-09-20T18:58:10.730907+0000","last_peered":"2024-09-20T18:58:10.730907+0000","last_clean":"2024-09-20T18:58:10.730907+0000","last_became_active":"2024-09-20T18:55:28.820941+0000","last_became_peered":"2024-09-20T18:55:28.820941+0000","last_unstale":"2024-09-20T18:58:10.730907+0000","last_undegraded":"2024-09-20T18:58:10.730907+0000","last_fullsized":"2024-09-20T18:58:10.730907+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'9","last_scrub_stamp":"2024-09-20T18:57:48.853302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:48.853302+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:06.295025+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3","version":"233'13658","reported_seq":26172,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:58:06.407801+0000","last_change":"2024-09-20T18:57:53.578343+0000","last_active":"2024-09-20T18:58:06.407801+0000","last_peered":"2024-09-20T18:58:06.407801+0000","last_clean":"2024-09-20T18:58:06.407801+0000","last_became_active":"2024-09-20T18:56:25.687391+0000","last_became_peered":"2024-09-20T18:56:25.687391+0000","last_unstale":"2024-09-20T18:58:06.407801+0000","last_undegraded":"2024-09-20T18:58:06.407801+0000","last_fullsized":"2024-09-20T18:58:06.407801+0000","mapping_epoch":203,"log_start":"212'9372","ondisk_log_start":"212'9372","created":160,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'12840","last_scrub_stamp":"2024-09-20T18:57:53.578263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:57:53.578263+0000","objects_scrubbed":1475,"log_size":4286,"log_dups_size":0,"ondisk_log_size":4286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:55.743448+0000","scrub_duration":660,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41811968,"num_objects":638,"num_object_clones":0,"num_object_copies":1914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13658,"num_write_kb":457472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6156,"num_bytes_recovered":403439616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":391,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.164686+0000","last_change":"2024-09-20T18:55:18.581290+0000","last_active":"2024-09-20T18:57:54.164686+0000","last_peered":"2024-09-20T18:57:54.164686+0000","last_clean":"2024-09-20T18:57:54.164686+0000","last_became_active":"2024-09-20T18:49:58.992302+0000","last_became_peered":"2024-09-20T18:49:58.992302+0000","last_unstale":"2024-09-20T18:57:54.164686+0000","last_undegraded":"2024-09-20T18:57:54.164686+0000","last_fullsized":"2024-09-20T18:57:54.164686+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:55:18.581200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:18.581200+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":378,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.151273+0000","last_change":"2024-09-20T18:54:54.288199+0000","last_active":"2024-09-20T18:57:54.151273+0000","last_peered":"2024-09-20T18:57:54.151273+0000","last_clean":"2024-09-20T18:57:54.151273+0000","last_became_active":"2024-09-20T18:49:45.936104+0000","last_became_peered":"2024-09-20T18:49:45.936104+0000","last_unstale":"2024-09-20T18:57:54.151273+0000","last_undegraded":"2024-09-20T18:57:54.151273+0000","last_fullsized":"2024-09-20T18:57:54.151273+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:54.288117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:54.288117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":46,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":486,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-20T18:58:10.731436+0000","last_change":"2024-09-20T18:54:14.292660+0000","last_active":"2024-09-20T18:58:10.731436+0000","last_peered":"2024-09-20T18:58:10.731436+0000","last_clean":"2024-09-20T18:58:10.731436+0000","last_became_active":"2024-09-20T18:49:49.960697+0000","last_became_peered":"2024-09-20T18:49:49.960697+0000","last_unstale":"2024-09-20T18:58:10.731436+0000","last_undegraded":"2024-09-20T18:58:10.731436+0000","last_fullsized":"2024-09-20T18:58:10.731436+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:54:14.292610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:14.292610+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":34,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":365,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.146304+0000","last_change":"2024-09-20T18:54:10.381171+0000","last_active":"2024-09-20T18:57:54.146304+0000","last_peered":"2024-09-20T18:57:54.146304+0000","last_clean":"2024-09-20T18:57:54.146304+0000","last_became_active":"2024-09-20T18:49:51.971399+0000","last_became_peered":"2024-09-20T18:49:51.971399+0000","last_unstale":"2024-09-20T18:57:54.146304+0000","last_undegraded":"2024-09-20T18:57:54.146304+0000","last_fullsized":"2024-09-20T18:57:54.146304+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:10.381108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:10.381108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:30.527012+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.c","version":"233'13847","reported_seq":14004,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:58:07.108046+0000","last_change":"2024-09-20T18:54:34.311256+0000","last_active":"2024-09-20T18:58:07.108046+0000","last_peered":"2024-09-20T18:58:07.108046+0000","last_clean":"2024-09-20T18:58:07.108046+0000","last_became_active":"2024-09-20T18:54:34.310904+0000","last_became_peered":"2024-09-20T18:54:34.310904+0000","last_unstale":"2024-09-20T18:58:07.108046+0000","last_undegraded":"2024-09-20T18:58:07.108046+0000","last_fullsized":"2024-09-20T18:58:07.108046+0000","mapping_epoch":160,"log_start":"210'8911","ondisk_log_start":"210'8911","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4936,"log_dups_size":0,"ondisk_log_size":4936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35717120,"num_objects":545,"num_object_clones":0,"num_object_copies":1635,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13847,"num_write_kb":460544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":407,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.150299+0000","last_change":"2024-09-20T18:54:22.077299+0000","last_active":"2024-09-20T18:57:54.150299+0000","last_peered":"2024-09-20T18:57:54.150299+0000","last_clean":"2024-09-20T18:57:54.150299+0000","last_became_active":"2024-09-20T18:49:48.499189+0000","last_became_peered":"2024-09-20T18:49:48.499189+0000","last_unstale":"2024-09-20T18:57:54.150299+0000","last_undegraded":"2024-09-20T18:57:54.150299+0000","last_fullsized":"2024-09-20T18:57:54.150299+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-20T18:54:22.077212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:22.077212+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":370,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.145818+0000","last_change":"2024-09-20T18:54:57.237740+0000","last_active":"2024-09-20T18:57:54.145818+0000","last_peered":"2024-09-20T18:57:54.145818+0000","last_clean":"2024-09-20T18:57:54.145818+0000","last_became_active":"2024-09-20T18:49:45.940544+0000","last_became_peered":"2024-09-20T18:49:45.940544+0000","last_unstale":"2024-09-20T18:57:54.145818+0000","last_undegraded":"2024-09-20T18:57:54.145818+0000","last_fullsized":"2024-09-20T18:57:54.145818+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.237675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.237675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":383,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-20T18:58:10.730799+0000","last_change":"2024-09-20T18:54:51.229251+0000","last_active":"2024-09-20T18:58:10.730799+0000","last_peered":"2024-09-20T18:58:10.730799+0000","last_clean":"2024-09-20T18:58:10.730799+0000","last_became_active":"2024-09-20T18:49:49.963073+0000","last_became_peered":"2024-09-20T18:49:49.963073+0000","last_unstale":"2024-09-20T18:58:10.730799+0000","last_undegraded":"2024-09-20T18:58:10.730799+0000","last_fullsized":"2024-09-20T18:58:10.730799+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.229201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.229201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":378,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-20T18:58:10.732716+0000","last_change":"2024-09-20T18:54:55.323937+0000","last_active":"2024-09-20T18:58:10.732716+0000","last_peered":"2024-09-20T18:58:10.732716+0000","last_clean":"2024-09-20T18:58:10.732716+0000","last_became_active":"2024-09-20T18:49:51.968793+0000","last_became_peered":"2024-09-20T18:49:51.968793+0000","last_unstale":"2024-09-20T18:58:10.732716+0000","last_undegraded":"2024-09-20T18:58:10.732716+0000","last_fullsized":"2024-09-20T18:58:10.732716+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:55.323692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:55.323692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"233'13753","reported_seq":13912,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:58:05.932356+0000","last_change":"2024-09-20T18:54:34.311680+0000","last_active":"2024-09-20T18:58:05.932356+0000","last_peered":"2024-09-20T18:58:05.932356+0000","last_clean":"2024-09-20T18:58:05.932356+0000","last_became_active":"2024-09-20T18:54:34.311241+0000","last_became_peered":"2024-09-20T18:54:34.311241+0000","last_unstale":"2024-09-20T18:58:05.932356+0000","last_undegraded":"2024-09-20T18:58:05.932356+0000","last_fullsized":"2024-09-20T18:58:05.932356+0000","mapping_epoch":160,"log_start":"209'8746","ondisk_log_start":"209'8746","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":5007,"log_dups_size":0,"ondisk_log_size":5007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:36.546826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40501272,"num_objects":619,"num_object_clones":0,"num_object_copies":1857,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":619,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13753,"num_write_kb":459841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":439,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-20T18:58:10.730879+0000","last_change":"2024-09-20T18:55:28.853375+0000","last_active":"2024-09-20T18:58:10.730879+0000","last_peered":"2024-09-20T18:58:10.730879+0000","last_clean":"2024-09-20T18:58:10.730879+0000","last_became_active":"2024-09-20T18:55:28.831057+0000","last_became_peered":"2024-09-20T18:55:28.831057+0000","last_unstale":"2024-09-20T18:58:10.730879+0000","last_undegraded":"2024-09-20T18:58:10.730879+0000","last_fullsized":"2024-09-20T18:58:10.730879+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-20T18:54:57.394893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.394893+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":370,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.162963+0000","last_change":"2024-09-20T18:51:27.187060+0000","last_active":"2024-09-20T18:57:54.162963+0000","last_peered":"2024-09-20T18:57:54.162963+0000","last_clean":"2024-09-20T18:57:54.162963+0000","last_became_active":"2024-09-20T18:49:45.940469+0000","last_became_peered":"2024-09-20T18:49:45.940469+0000","last_unstale":"2024-09-20T18:57:54.162963+0000","last_undegraded":"2024-09-20T18:57:54.162963+0000","last_fullsized":"2024-09-20T18:57:54.162963+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:51:27.186991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:27.186991+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"66'8","reported_seq":488,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:58:01.091652+0000","last_change":"2024-09-20T18:54:47.223010+0000","last_active":"2024-09-20T18:58:01.091652+0000","last_peered":"2024-09-20T18:58:01.091652+0000","last_clean":"2024-09-20T18:58:01.091652+0000","last_became_active":"2024-09-20T18:49:49.956133+0000","last_became_peered":"2024-09-20T18:49:49.956133+0000","last_unstale":"2024-09-20T18:58:01.091652+0000","last_undegraded":"2024-09-20T18:58:01.091652+0000","last_fullsized":"2024-09-20T18:58:01.091652+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:54:47.222937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:47.222937+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":376,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-20T18:58:10.738449+0000","last_change":"2024-09-20T18:55:28.819147+0000","last_active":"2024-09-20T18:58:10.738449+0000","last_peered":"2024-09-20T18:58:10.738449+0000","last_clean":"2024-09-20T18:58:10.738449+0000","last_became_active":"2024-09-20T18:55:28.818804+0000","last_became_peered":"2024-09-20T18:55:28.818804+0000","last_unstale":"2024-09-20T18:58:10.738449+0000","last_undegraded":"2024-09-20T18:58:10.738449+0000","last_fullsized":"2024-09-20T18:58:10.738449+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:42.880724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:42.880724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"233'13854","reported_seq":14012,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:58:07.083425+0000","last_change":"2024-09-20T18:54:34.309939+0000","last_active":"2024-09-20T18:58:07.083425+0000","last_peered":"2024-09-20T18:58:07.083425+0000","last_clean":"2024-09-20T18:58:07.083425+0000","last_became_active":"2024-09-20T18:54:34.309630+0000","last_became_peered":"2024-09-20T18:54:34.309630+0000","last_unstale":"2024-09-20T18:58:07.083425+0000","last_undegraded":"2024-09-20T18:58:07.083425+0000","last_fullsized":"2024-09-20T18:58:07.083425+0000","mapping_epoch":160,"log_start":"210'8949","ondisk_log_start":"210'8949","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4905,"log_dups_size":0,"ondisk_log_size":4905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35913728,"num_objects":548,"num_object_clones":0,"num_object_copies":1644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":548,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13854,"num_write_kb":460864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":405,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:58:03.053521+0000","last_change":"2024-09-20T18:58:03.053521+0000","last_active":"2024-09-20T18:58:03.053521+0000","last_peered":"2024-09-20T18:58:03.053521+0000","last_clean":"2024-09-20T18:58:03.053521+0000","last_became_active":"2024-09-20T18:49:47.949626+0000","last_became_peered":"2024-09-20T18:49:47.949626+0000","last_unstale":"2024-09-20T18:58:03.053521+0000","last_undegraded":"2024-09-20T18:58:03.053521+0000","last_fullsized":"2024-09-20T18:58:03.053521+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:58:03.053310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:03.053310+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:05.671653+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":385,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.146828+0000","last_change":"2024-09-20T18:54:18.326947+0000","last_active":"2024-09-20T18:57:54.146828+0000","last_peered":"2024-09-20T18:57:54.146828+0000","last_clean":"2024-09-20T18:57:54.146828+0000","last_became_active":"2024-09-20T18:49:45.942496+0000","last_became_peered":"2024-09-20T18:49:45.942496+0000","last_unstale":"2024-09-20T18:57:54.146828+0000","last_undegraded":"2024-09-20T18:57:54.146828+0000","last_fullsized":"2024-09-20T18:57:54.146828+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:18.326875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:18.326875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:32.505949+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"184'9","reported_seq":489,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:58:01.093240+0000","last_change":"2024-09-20T18:55:29.010062+0000","last_active":"2024-09-20T18:58:01.093240+0000","last_peered":"2024-09-20T18:58:01.093240+0000","last_clean":"2024-09-20T18:58:01.093240+0000","last_became_active":"2024-09-20T18:55:28.830643+0000","last_became_peered":"2024-09-20T18:55:28.830643+0000","last_unstale":"2024-09-20T18:58:01.093240+0000","last_undegraded":"2024-09-20T18:58:01.093240+0000","last_fullsized":"2024-09-20T18:58:01.093240+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:29.010015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:29.010015+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":350,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.159917+0000","last_change":"2024-09-20T18:49:51.965733+0000","last_active":"2024-09-20T18:57:54.159917+0000","last_peered":"2024-09-20T18:57:54.159917+0000","last_clean":"2024-09-20T18:57:54.159917+0000","last_became_active":"2024-09-20T18:49:51.964780+0000","last_became_peered":"2024-09-20T18:49:51.964780+0000","last_unstale":"2024-09-20T18:57:54.159917+0000","last_undegraded":"2024-09-20T18:57:54.159917+0000","last_fullsized":"2024-09-20T18:57:54.159917+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"233'14152","reported_seq":14309,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:58:06.513918+0000","last_change":"2024-09-20T18:54:34.309809+0000","last_active":"2024-09-20T18:58:06.513918+0000","last_peered":"2024-09-20T18:58:06.513918+0000","last_clean":"2024-09-20T18:58:06.513918+0000","last_became_active":"2024-09-20T18:54:34.309231+0000","last_became_peered":"2024-09-20T18:54:34.309231+0000","last_unstale":"2024-09-20T18:58:06.513918+0000","last_undegraded":"2024-09-20T18:58:06.513918+0000","last_fullsized":"2024-09-20T18:58:06.513918+0000","mapping_epoch":160,"log_start":"212'9832","ondisk_log_start":"212'9832","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4320,"log_dups_size":0,"ondisk_log_size":4320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36831232,"num_objects":562,"num_object_clones":0,"num_object_copies":1686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":562,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14152,"num_write_kb":470848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":409,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.160953+0000","last_change":"2024-09-20T18:54:14.923895+0000","last_active":"2024-09-20T18:57:54.160953+0000","last_peered":"2024-09-20T18:57:54.160953+0000","last_clean":"2024-09-20T18:57:54.160953+0000","last_became_active":"2024-09-20T18:49:48.497736+0000","last_became_peered":"2024-09-20T18:49:48.497736+0000","last_unstale":"2024-09-20T18:57:54.160953+0000","last_undegraded":"2024-09-20T18:57:54.160953+0000","last_fullsized":"2024-09-20T18:57:54.160953+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-20T18:54:14.923682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:14.923682+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":370,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.166094+0000","last_change":"2024-09-20T18:57:09.940634+0000","last_active":"2024-09-20T18:57:54.166094+0000","last_peered":"2024-09-20T18:57:54.166094+0000","last_clean":"2024-09-20T18:57:54.166094+0000","last_became_active":"2024-09-20T18:49:45.942253+0000","last_became_peered":"2024-09-20T18:49:45.942253+0000","last_unstale":"2024-09-20T18:57:54.166094+0000","last_undegraded":"2024-09-20T18:57:54.166094+0000","last_fullsized":"2024-09-20T18:57:54.166094+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:09.940517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:09.940517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:18.833936+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":116,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.160514+0000","last_change":"2024-09-20T18:55:28.830954+0000","last_active":"2024-09-20T18:57:54.160514+0000","last_peered":"2024-09-20T18:57:54.160514+0000","last_clean":"2024-09-20T18:57:54.160514+0000","last_became_active":"2024-09-20T18:55:28.830645+0000","last_became_peered":"2024-09-20T18:55:28.830645+0000","last_unstale":"2024-09-20T18:57:54.160514+0000","last_undegraded":"2024-09-20T18:57:54.160514+0000","last_fullsized":"2024-09-20T18:57:54.160514+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:50.610871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:50.610871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":389,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:58.973310+0000","last_change":"2024-09-20T18:57:58.973310+0000","last_active":"2024-09-20T18:57:58.973310+0000","last_peered":"2024-09-20T18:57:58.973310+0000","last_clean":"2024-09-20T18:57:58.973310+0000","last_became_active":"2024-09-20T18:55:28.821588+0000","last_became_peered":"2024-09-20T18:55:28.821588+0000","last_unstale":"2024-09-20T18:57:58.973310+0000","last_undegraded":"2024-09-20T18:57:58.973310+0000","last_fullsized":"2024-09-20T18:57:58.973310+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:58.973176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:58.973176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:03.568279+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"233'13941","reported_seq":14125,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:58:05.909143+0000","last_change":"2024-09-20T18:57:47.843800+0000","last_active":"2024-09-20T18:58:05.909143+0000","last_peered":"2024-09-20T18:58:05.909143+0000","last_clean":"2024-09-20T18:58:05.909143+0000","last_became_active":"2024-09-20T18:54:34.311181+0000","last_became_peered":"2024-09-20T18:54:34.311181+0000","last_unstale":"2024-09-20T18:58:05.909143+0000","last_undegraded":"2024-09-20T18:58:05.909143+0000","last_fullsized":"2024-09-20T18:58:05.909143+0000","mapping_epoch":160,"log_start":"210'8894","ondisk_log_start":"210'8894","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'12813","last_scrub_stamp":"2024-09-20T18:57:47.843722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:57:47.843722+0000","objects_scrubbed":1796,"log_size":5047,"log_dups_size":0,"ondisk_log_size":5047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:52.400285+0000","scrub_duration":406,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43057152,"num_objects":657,"num_object_clones":0,"num_object_copies":1971,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13941,"num_write_kb":467136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"66'28","reported_seq":523,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:58:01.091783+0000","last_change":"2024-09-20T18:55:16.527267+0000","last_active":"2024-09-20T18:58:01.091783+0000","last_peered":"2024-09-20T18:58:01.091783+0000","last_clean":"2024-09-20T18:58:01.091783+0000","last_became_active":"2024-09-20T18:49:47.955335+0000","last_became_peered":"2024-09-20T18:49:47.955335+0000","last_unstale":"2024-09-20T18:58:01.091783+0000","last_undegraded":"2024-09-20T18:58:01.091783+0000","last_fullsized":"2024-09-20T18:58:01.091783+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'28","last_scrub_stamp":"2024-09-20T18:55:16.527165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:16.527165+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":382,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-20T18:58:10.731867+0000","last_change":"2024-09-20T18:57:21.807755+0000","last_active":"2024-09-20T18:58:10.731867+0000","last_peered":"2024-09-20T18:58:10.731867+0000","last_clean":"2024-09-20T18:58:10.731867+0000","last_became_active":"2024-09-20T18:55:28.831574+0000","last_became_peered":"2024-09-20T18:55:28.831574+0000","last_unstale":"2024-09-20T18:58:10.731867+0000","last_undegraded":"2024-09-20T18:58:10.731867+0000","last_fullsized":"2024-09-20T18:58:10.731867+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:21.807703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:21.807703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:42.821495+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1],"acting":[4,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":415,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.146230+0000","last_change":"2024-09-20T18:55:28.854615+0000","last_active":"2024-09-20T18:57:54.146230+0000","last_peered":"2024-09-20T18:57:54.146230+0000","last_clean":"2024-09-20T18:57:54.146230+0000","last_became_active":"2024-09-20T18:55:28.837226+0000","last_became_peered":"2024-09-20T18:55:28.837226+0000","last_unstale":"2024-09-20T18:57:54.146230+0000","last_undegraded":"2024-09-20T18:57:54.146230+0000","last_fullsized":"2024-09-20T18:57:54.146230+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-20T18:54:51.227531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.227531+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:43.598074+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0012781159999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":377,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.162174+0000","last_change":"2024-09-20T18:55:13.949348+0000","last_active":"2024-09-20T18:57:54.162174+0000","last_peered":"2024-09-20T18:57:54.162174+0000","last_clean":"2024-09-20T18:57:54.162174+0000","last_became_active":"2024-09-20T18:49:49.953971+0000","last_became_peered":"2024-09-20T18:49:49.953971+0000","last_unstale":"2024-09-20T18:57:54.162174+0000","last_undegraded":"2024-09-20T18:57:54.162174+0000","last_fullsized":"2024-09-20T18:57:54.162174+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:13.949269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:13.949269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":358,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.142571+0000","last_change":"2024-09-20T18:57:12.744116+0000","last_active":"2024-09-20T18:57:54.142571+0000","last_peered":"2024-09-20T18:57:54.142571+0000","last_clean":"2024-09-20T18:57:54.142571+0000","last_became_active":"2024-09-20T18:49:51.969400+0000","last_became_peered":"2024-09-20T18:49:51.969400+0000","last_unstale":"2024-09-20T18:57:54.142571+0000","last_undegraded":"2024-09-20T18:57:54.142571+0000","last_fullsized":"2024-09-20T18:57:54.142571+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:12.744040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:12.744040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:17.964285+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"233'14113","reported_seq":14288,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:58:06.546821+0000","last_change":"2024-09-20T18:58:02.207422+0000","last_active":"2024-09-20T18:58:06.546821+0000","last_peered":"2024-09-20T18:58:06.546821+0000","last_clean":"2024-09-20T18:58:06.546821+0000","last_became_active":"2024-09-20T18:54:33.925750+0000","last_became_peered":"2024-09-20T18:54:33.925750+0000","last_unstale":"2024-09-20T18:58:06.546821+0000","last_undegraded":"2024-09-20T18:58:06.546821+0000","last_fullsized":"2024-09-20T18:58:06.546821+0000","mapping_epoch":160,"log_start":"212'9740","ondisk_log_start":"212'9740","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"233'13810","last_scrub_stamp":"2024-09-20T18:58:02.207357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:58:02.207357+0000","objects_scrubbed":908,"log_size":4373,"log_dups_size":0,"ondisk_log_size":4373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:23.632342+0000","scrub_duration":122,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39649280,"num_objects":605,"num_object_clones":0,"num_object_copies":1815,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":605,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14113,"num_write_kb":470976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":417,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:58:06.079117+0000","last_change":"2024-09-20T18:58:06.079117+0000","last_active":"2024-09-20T18:58:06.079117+0000","last_peered":"2024-09-20T18:58:06.079117+0000","last_clean":"2024-09-20T18:58:06.079117+0000","last_became_active":"2024-09-20T18:49:47.950904+0000","last_became_peered":"2024-09-20T18:49:47.950904+0000","last_unstale":"2024-09-20T18:58:06.079117+0000","last_undegraded":"2024-09-20T18:58:06.079117+0000","last_fullsized":"2024-09-20T18:58:06.079117+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-20T18:58:06.078352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:06.078352+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:26.574178+0000","scrub_duration":59,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":394,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.163293+0000","last_change":"2024-09-20T18:55:28.830832+0000","last_active":"2024-09-20T18:57:54.163293+0000","last_peered":"2024-09-20T18:57:54.163293+0000","last_clean":"2024-09-20T18:57:54.163293+0000","last_became_active":"2024-09-20T18:55:28.830269+0000","last_became_peered":"2024-09-20T18:55:28.830269+0000","last_unstale":"2024-09-20T18:57:54.163293+0000","last_undegraded":"2024-09-20T18:57:54.163293+0000","last_fullsized":"2024-09-20T18:57:54.163293+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:23.359769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:23.359769+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:27.139436+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":381,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.163676+0000","last_change":"2024-09-20T18:52:07.718766+0000","last_active":"2024-09-20T18:57:54.163676+0000","last_peered":"2024-09-20T18:57:54.163676+0000","last_clean":"2024-09-20T18:57:54.163676+0000","last_became_active":"2024-09-20T18:49:44.937135+0000","last_became_peered":"2024-09-20T18:49:44.937135+0000","last_unstale":"2024-09-20T18:57:54.163676+0000","last_undegraded":"2024-09-20T18:57:54.163676+0000","last_fullsized":"2024-09-20T18:57:54.163676+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:52:07.718646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:52:07.718646+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.00095558800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":363,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.170778+0000","last_change":"2024-09-20T18:51:59.796450+0000","last_active":"2024-09-20T18:57:54.170778+0000","last_peered":"2024-09-20T18:57:54.170778+0000","last_clean":"2024-09-20T18:57:54.170778+0000","last_became_active":"2024-09-20T18:49:49.954424+0000","last_became_peered":"2024-09-20T18:49:49.954424+0000","last_unstale":"2024-09-20T18:57:54.170778+0000","last_undegraded":"2024-09-20T18:57:54.170778+0000","last_fullsized":"2024-09-20T18:57:54.170778+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:59.796383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:59.796383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":368,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-20T18:58:10.739638+0000","last_change":"2024-09-20T18:55:28.826659+0000","last_active":"2024-09-20T18:58:10.739638+0000","last_peered":"2024-09-20T18:58:10.739638+0000","last_clean":"2024-09-20T18:58:10.739638+0000","last_became_active":"2024-09-20T18:55:28.825812+0000","last_became_peered":"2024-09-20T18:55:28.825812+0000","last_unstale":"2024-09-20T18:58:10.739638+0000","last_undegraded":"2024-09-20T18:58:10.739638+0000","last_fullsized":"2024-09-20T18:58:10.739638+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:39.814259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:39.814259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"233'14069","reported_seq":14226,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:58:06.378126+0000","last_change":"2024-09-20T18:54:34.311774+0000","last_active":"2024-09-20T18:58:06.378126+0000","last_peered":"2024-09-20T18:58:06.378126+0000","last_clean":"2024-09-20T18:58:06.378126+0000","last_became_active":"2024-09-20T18:54:34.311487+0000","last_became_peered":"2024-09-20T18:54:34.311487+0000","last_unstale":"2024-09-20T18:58:06.378126+0000","last_undegraded":"2024-09-20T18:58:06.378126+0000","last_fullsized":"2024-09-20T18:58:06.378126+0000","mapping_epoch":160,"log_start":"212'9792","ondisk_log_start":"212'9792","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4277,"log_dups_size":0,"ondisk_log_size":4277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:43.066892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35979264,"num_objects":549,"num_object_clones":0,"num_object_copies":1647,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14069,"num_write_kb":467776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":407,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.163568+0000","last_change":"2024-09-20T18:55:14.951950+0000","last_active":"2024-09-20T18:57:54.163568+0000","last_peered":"2024-09-20T18:57:54.163568+0000","last_clean":"2024-09-20T18:57:54.163568+0000","last_became_active":"2024-09-20T18:49:47.949230+0000","last_became_peered":"2024-09-20T18:49:47.949230+0000","last_unstale":"2024-09-20T18:57:54.163568+0000","last_undegraded":"2024-09-20T18:57:54.163568+0000","last_fullsized":"2024-09-20T18:57:54.163568+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:55:14.951892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:14.951892+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":385,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.160683+0000","last_change":"2024-09-20T18:54:27.905856+0000","last_active":"2024-09-20T18:57:54.160683+0000","last_peered":"2024-09-20T18:57:54.160683+0000","last_clean":"2024-09-20T18:57:54.160683+0000","last_became_active":"2024-09-20T18:49:45.939197+0000","last_became_peered":"2024-09-20T18:49:45.939197+0000","last_unstale":"2024-09-20T18:57:54.160683+0000","last_undegraded":"2024-09-20T18:57:54.160683+0000","last_fullsized":"2024-09-20T18:57:54.160683+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:27.905803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:27.905803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":900,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-20T18:58:10.738101+0000","last_change":"2024-09-20T18:54:26.359887+0000","last_active":"2024-09-20T18:58:10.738101+0000","last_peered":"2024-09-20T18:58:10.738101+0000","last_clean":"2024-09-20T18:58:10.738101+0000","last_became_active":"2024-09-20T18:49:44.936819+0000","last_became_peered":"2024-09-20T18:49:44.936819+0000","last_unstale":"2024-09-20T18:58:10.738101+0000","last_undegraded":"2024-09-20T18:58:10.738101+0000","last_fullsized":"2024-09-20T18:58:10.738101+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-20T18:54:26.359807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:54:26.359807+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0010996199999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":499,"num_read_kb":494,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":377,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.362085+0000","last_change":"2024-09-20T18:57:54.362085+0000","last_active":"2024-09-20T18:57:54.362085+0000","last_peered":"2024-09-20T18:57:54.362085+0000","last_clean":"2024-09-20T18:57:54.362085+0000","last_became_active":"2024-09-20T18:49:49.961066+0000","last_became_peered":"2024-09-20T18:49:49.961066+0000","last_unstale":"2024-09-20T18:57:54.362085+0000","last_undegraded":"2024-09-20T18:57:54.362085+0000","last_fullsized":"2024-09-20T18:57:54.362085+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:54.362038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:54.362038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:08.560302+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":374,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.165688+0000","last_change":"2024-09-20T18:55:21.629867+0000","last_active":"2024-09-20T18:57:54.165688+0000","last_peered":"2024-09-20T18:57:54.165688+0000","last_clean":"2024-09-20T18:57:54.165688+0000","last_became_active":"2024-09-20T18:49:51.970343+0000","last_became_peered":"2024-09-20T18:49:51.970343+0000","last_unstale":"2024-09-20T18:57:54.165688+0000","last_undegraded":"2024-09-20T18:57:54.165688+0000","last_fullsized":"2024-09-20T18:57:54.165688+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:21.629800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:21.629800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"233'13912","reported_seq":24494,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:58:05.889775+0000","last_change":"2024-09-20T18:57:09.139489+0000","last_active":"2024-09-20T18:58:05.889775+0000","last_peered":"2024-09-20T18:58:05.889775+0000","last_clean":"2024-09-20T18:58:05.889775+0000","last_became_active":"2024-09-20T18:57:09.121592+0000","last_became_peered":"2024-09-20T18:57:09.121592+0000","last_unstale":"2024-09-20T18:58:05.889775+0000","last_undegraded":"2024-09-20T18:58:05.889775+0000","last_fullsized":"2024-09-20T18:58:05.889775+0000","mapping_epoch":216,"log_start":"209'8906","ondisk_log_start":"209'8906","created":160,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":5006,"log_dups_size":0,"ondisk_log_size":5006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:35.584332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40632320,"num_objects":620,"num_object_clones":0,"num_object_copies":1860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13912,"num_write_kb":465024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5206,"num_bytes_recovered":341180416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"184'17","reported_seq":222,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-20T18:58:10.731605+0000","last_change":"2024-09-20T18:57:22.816240+0000","last_active":"2024-09-20T18:58:10.731605+0000","last_peered":"2024-09-20T18:58:10.731605+0000","last_clean":"2024-09-20T18:58:10.731605+0000","last_became_active":"2024-09-20T18:55:28.831024+0000","last_became_peered":"2024-09-20T18:55:28.831024+0000","last_unstale":"2024-09-20T18:58:10.731605+0000","last_undegraded":"2024-09-20T18:58:10.731605+0000","last_fullsized":"2024-09-20T18:58:10.731605+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'17","last_scrub_stamp":"2024-09-20T18:57:22.816154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:22.816154+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:28.727289+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":24,"num_read_kb":14,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":385,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.141997+0000","last_change":"2024-09-20T18:55:28.830447+0000","last_active":"2024-09-20T18:57:54.141997+0000","last_peered":"2024-09-20T18:57:54.141997+0000","last_clean":"2024-09-20T18:57:54.141997+0000","last_became_active":"2024-09-20T18:55:28.830147+0000","last_became_peered":"2024-09-20T18:55:28.830147+0000","last_unstale":"2024-09-20T18:57:54.141997+0000","last_undegraded":"2024-09-20T18:57:54.141997+0000","last_fullsized":"2024-09-20T18:57:54.141997+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:13.144896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:13.144896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:28.731761+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":443,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-20T18:58:10.738286+0000","last_change":"2024-09-20T18:55:28.844060+0000","last_active":"2024-09-20T18:58:10.738286+0000","last_peered":"2024-09-20T18:58:10.738286+0000","last_clean":"2024-09-20T18:58:10.738286+0000","last_became_active":"2024-09-20T18:55:28.829325+0000","last_became_peered":"2024-09-20T18:55:28.829325+0000","last_unstale":"2024-09-20T18:58:10.738286+0000","last_undegraded":"2024-09-20T18:58:10.738286+0000","last_fullsized":"2024-09-20T18:58:10.738286+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T18:55:13.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T18:55:13.696607+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":366,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-20T18:58:10.739778+0000","last_change":"2024-09-20T18:51:22.852942+0000","last_active":"2024-09-20T18:58:10.739778+0000","last_peered":"2024-09-20T18:58:10.739778+0000","last_clean":"2024-09-20T18:58:10.739778+0000","last_became_active":"2024-09-20T18:49:49.956128+0000","last_became_peered":"2024-09-20T18:49:49.956128+0000","last_unstale":"2024-09-20T18:58:10.739778+0000","last_undegraded":"2024-09-20T18:58:10.739778+0000","last_fullsized":"2024-09-20T18:58:10.739778+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:22.852869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:22.852869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":381,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:58:03.523127+0000","last_change":"2024-09-20T18:58:03.523127+0000","last_active":"2024-09-20T18:58:03.523127+0000","last_peered":"2024-09-20T18:58:03.523127+0000","last_clean":"2024-09-20T18:58:03.523127+0000","last_became_active":"2024-09-20T18:49:51.971332+0000","last_became_peered":"2024-09-20T18:49:51.971332+0000","last_unstale":"2024-09-20T18:58:03.523127+0000","last_undegraded":"2024-09-20T18:58:03.523127+0000","last_fullsized":"2024-09-20T18:58:03.523127+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:03.523078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:03.523078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:14.059180+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"235'14255","reported_seq":14417,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-20T18:58:10.934729+0000","last_change":"2024-09-20T18:54:33.929366+0000","last_active":"2024-09-20T18:58:10.934729+0000","last_peered":"2024-09-20T18:58:10.934729+0000","last_clean":"2024-09-20T18:58:10.934729+0000","last_became_active":"2024-09-20T18:54:33.929072+0000","last_became_peered":"2024-09-20T18:54:33.929072+0000","last_unstale":"2024-09-20T18:58:10.934729+0000","last_undegraded":"2024-09-20T18:58:10.934729+0000","last_fullsized":"2024-09-20T18:58:10.934729+0000","mapping_epoch":160,"log_start":"212'9702","ondisk_log_start":"212'9702","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4553,"log_dups_size":0,"ondisk_log_size":4553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20119552,"num_objects":307,"num_object_clones":0,"num_object_copies":921,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14255,"num_write_kb":465984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":408,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-20T18:58:10.731527+0000","last_change":"2024-09-20T18:55:28.840105+0000","last_active":"2024-09-20T18:58:10.731527+0000","last_peered":"2024-09-20T18:58:10.731527+0000","last_clean":"2024-09-20T18:58:10.731527+0000","last_became_active":"2024-09-20T18:55:28.818269+0000","last_became_peered":"2024-09-20T18:55:28.818269+0000","last_unstale":"2024-09-20T18:58:10.731527+0000","last_undegraded":"2024-09-20T18:58:10.731527+0000","last_fullsized":"2024-09-20T18:58:10.731527+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:51:14.397930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.397930+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:23.803752+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":374,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-20T18:58:10.731145+0000","last_change":"2024-09-20T18:51:18.410601+0000","last_active":"2024-09-20T18:58:10.731145+0000","last_peered":"2024-09-20T18:58:10.731145+0000","last_clean":"2024-09-20T18:58:10.731145+0000","last_became_active":"2024-09-20T18:49:45.944579+0000","last_became_peered":"2024-09-20T18:49:45.944579+0000","last_unstale":"2024-09-20T18:58:10.731145+0000","last_undegraded":"2024-09-20T18:58:10.731145+0000","last_fullsized":"2024-09-20T18:58:10.731145+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:18.410447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:18.410447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:29.917821+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":384,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.146365+0000","last_change":"2024-09-20T18:55:28.830634+0000","last_active":"2024-09-20T18:57:54.146365+0000","last_peered":"2024-09-20T18:57:54.146365+0000","last_clean":"2024-09-20T18:57:54.146365+0000","last_became_active":"2024-09-20T18:55:28.827785+0000","last_became_peered":"2024-09-20T18:55:28.827785+0000","last_unstale":"2024-09-20T18:57:54.146365+0000","last_undegraded":"2024-09-20T18:57:54.146365+0000","last_fullsized":"2024-09-20T18:57:54.146365+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:11.401777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:11.401777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:37.550724+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":373,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.142091+0000","last_change":"2024-09-20T18:57:23.721754+0000","last_active":"2024-09-20T18:57:54.142091+0000","last_peered":"2024-09-20T18:57:54.142091+0000","last_clean":"2024-09-20T18:57:54.142091+0000","last_became_active":"2024-09-20T18:55:28.833115+0000","last_became_peered":"2024-09-20T18:55:28.833115+0000","last_unstale":"2024-09-20T18:57:54.142091+0000","last_undegraded":"2024-09-20T18:57:54.142091+0000","last_fullsized":"2024-09-20T18:57:54.142091+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:23.721643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:23.721643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:25.737391+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1","version":"233'13859","reported_seq":14042,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:58:06.384846+0000","last_change":"2024-09-20T18:57:50.704022+0000","last_active":"2024-09-20T18:58:06.384846+0000","last_peered":"2024-09-20T18:58:06.384846+0000","last_clean":"2024-09-20T18:58:06.384846+0000","last_became_active":"2024-09-20T18:54:34.309781+0000","last_became_peered":"2024-09-20T18:54:34.309781+0000","last_unstale":"2024-09-20T18:58:06.384846+0000","last_undegraded":"2024-09-20T18:58:06.384846+0000","last_fullsized":"2024-09-20T18:58:06.384846+0000","mapping_epoch":160,"log_start":"212'9624","ondisk_log_start":"212'9624","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'12866","last_scrub_stamp":"2024-09-20T18:57:50.703702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:57:50.703702+0000","objects_scrubbed":1599,"log_size":4235,"log_dups_size":0,"ondisk_log_size":4235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:00.179362+0000","scrub_duration":719,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37552128,"num_objects":573,"num_object_clones":0,"num_object_copies":1719,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13859,"num_write_kb":461824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":406,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-20T18:58:10.731748+0000","last_change":"2024-09-20T18:52:02.944536+0000","last_active":"2024-09-20T18:58:10.731748+0000","last_peered":"2024-09-20T18:58:10.731748+0000","last_clean":"2024-09-20T18:58:10.731748+0000","last_became_active":"2024-09-20T18:49:47.947636+0000","last_became_peered":"2024-09-20T18:49:47.947636+0000","last_unstale":"2024-09-20T18:58:10.731748+0000","last_undegraded":"2024-09-20T18:58:10.731748+0000","last_fullsized":"2024-09-20T18:58:10.731748+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-20T18:52:02.944434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:02.944434+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:34.896863+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":244,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-20T18:58:10.732311+0000","last_change":"2024-09-20T18:55:28.822330+0000","last_active":"2024-09-20T18:58:10.732311+0000","last_peered":"2024-09-20T18:58:10.732311+0000","last_clean":"2024-09-20T18:58:10.732311+0000","last_became_active":"2024-09-20T18:55:28.821959+0000","last_became_peered":"2024-09-20T18:55:28.821959+0000","last_unstale":"2024-09-20T18:58:10.732311+0000","last_undegraded":"2024-09-20T18:58:10.732311+0000","last_fullsized":"2024-09-20T18:58:10.732311+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:45.669427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:45.669427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":370,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.165787+0000","last_change":"2024-09-20T18:54:11.377907+0000","last_active":"2024-09-20T18:57:54.165787+0000","last_peered":"2024-09-20T18:57:54.165787+0000","last_clean":"2024-09-20T18:57:54.165787+0000","last_became_active":"2024-09-20T18:49:49.959160+0000","last_became_peered":"2024-09-20T18:49:49.959160+0000","last_unstale":"2024-09-20T18:57:54.165787+0000","last_undegraded":"2024-09-20T18:57:54.165787+0000","last_fullsized":"2024-09-20T18:57:54.165787+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:11.376951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:11.376951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:29.061620+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":367,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.164831+0000","last_change":"2024-09-20T18:52:35.635897+0000","last_active":"2024-09-20T18:57:54.164831+0000","last_peered":"2024-09-20T18:57:54.164831+0000","last_clean":"2024-09-20T18:57:54.164831+0000","last_became_active":"2024-09-20T18:49:51.970108+0000","last_became_peered":"2024-09-20T18:49:51.970108+0000","last_unstale":"2024-09-20T18:57:54.164831+0000","last_undegraded":"2024-09-20T18:57:54.164831+0000","last_fullsized":"2024-09-20T18:57:54.164831+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:35.635818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:35.635818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"233'13652","reported_seq":13810,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:58:06.555776+0000","last_change":"2024-09-20T18:54:33.929552+0000","last_active":"2024-09-20T18:58:06.555776+0000","last_peered":"2024-09-20T18:58:06.555776+0000","last_clean":"2024-09-20T18:58:06.555776+0000","last_became_active":"2024-09-20T18:54:33.929208+0000","last_became_peered":"2024-09-20T18:54:33.929208+0000","last_unstale":"2024-09-20T18:58:06.555776+0000","last_undegraded":"2024-09-20T18:58:06.555776+0000","last_fullsized":"2024-09-20T18:58:06.555776+0000","mapping_epoch":160,"log_start":"211'9321","ondisk_log_start":"211'9321","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4331,"log_dups_size":0,"ondisk_log_size":4331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37617664,"num_objects":574,"num_object_clones":0,"num_object_copies":1722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":574,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13652,"num_write_kb":455232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":412,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-20T18:58:10.732115+0000","last_change":"2024-09-20T18:55:28.846773+0000","last_active":"2024-09-20T18:58:10.732115+0000","last_peered":"2024-09-20T18:58:10.732115+0000","last_clean":"2024-09-20T18:58:10.732115+0000","last_became_active":"2024-09-20T18:55:28.817101+0000","last_became_peered":"2024-09-20T18:55:28.817101+0000","last_unstale":"2024-09-20T18:58:10.732115+0000","last_undegraded":"2024-09-20T18:58:10.732115+0000","last_fullsized":"2024-09-20T18:58:10.732115+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:54:17.219548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:17.219548+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":378,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.144006+0000","last_change":"2024-09-20T18:52:46.005172+0000","last_active":"2024-09-20T18:57:54.144006+0000","last_peered":"2024-09-20T18:57:54.144006+0000","last_clean":"2024-09-20T18:57:54.144006+0000","last_became_active":"2024-09-20T18:49:45.937264+0000","last_became_peered":"2024-09-20T18:49:45.937264+0000","last_unstale":"2024-09-20T18:57:54.144006+0000","last_undegraded":"2024-09-20T18:57:54.144006+0000","last_fullsized":"2024-09-20T18:57:54.144006+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:46.005109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:46.005109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:27.755958+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":488,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:58:01.092759+0000","last_change":"2024-09-20T18:55:17.549660+0000","last_active":"2024-09-20T18:58:01.092759+0000","last_peered":"2024-09-20T18:58:01.092759+0000","last_clean":"2024-09-20T18:58:01.092759+0000","last_became_active":"2024-09-20T18:49:49.957005+0000","last_became_peered":"2024-09-20T18:49:49.957005+0000","last_unstale":"2024-09-20T18:58:01.092759+0000","last_undegraded":"2024-09-20T18:58:01.092759+0000","last_fullsized":"2024-09-20T18:58:01.092759+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:17.549598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:17.549598+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":370,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-20T18:58:10.731251+0000","last_change":"2024-09-20T18:54:42.182875+0000","last_active":"2024-09-20T18:58:10.731251+0000","last_peered":"2024-09-20T18:58:10.731251+0000","last_clean":"2024-09-20T18:58:10.731251+0000","last_became_active":"2024-09-20T18:49:51.969355+0000","last_became_peered":"2024-09-20T18:49:51.969355+0000","last_unstale":"2024-09-20T18:58:10.731251+0000","last_undegraded":"2024-09-20T18:58:10.731251+0000","last_fullsized":"2024-09-20T18:58:10.731251+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:42.182809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:42.182809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"233'13960","reported_seq":26386,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:58:06.397755+0000","last_change":"2024-09-20T18:57:00.083015+0000","last_active":"2024-09-20T18:58:06.397755+0000","last_peered":"2024-09-20T18:58:06.397755+0000","last_clean":"2024-09-20T18:58:06.397755+0000","last_became_active":"2024-09-20T18:57:00.060960+0000","last_became_peered":"2024-09-20T18:57:00.060960+0000","last_unstale":"2024-09-20T18:58:06.397755+0000","last_undegraded":"2024-09-20T18:58:06.397755+0000","last_fullsized":"2024-09-20T18:58:06.397755+0000","mapping_epoch":213,"log_start":"212'9661","ondisk_log_start":"212'9661","created":160,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4299,"log_dups_size":0,"ondisk_log_size":4299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:34.055353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38666240,"num_objects":590,"num_object_clones":0,"num_object_copies":1770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13960,"num_write_kb":465600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6125,"num_bytes_recovered":401408000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":397,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.160870+0000","last_change":"2024-09-20T18:54:54.897991+0000","last_active":"2024-09-20T18:57:54.160870+0000","last_peered":"2024-09-20T18:57:54.160870+0000","last_clean":"2024-09-20T18:57:54.160870+0000","last_became_active":"2024-09-20T18:49:47.946008+0000","last_became_peered":"2024-09-20T18:49:47.946008+0000","last_unstale":"2024-09-20T18:57:54.160870+0000","last_undegraded":"2024-09-20T18:57:54.160870+0000","last_fullsized":"2024-09-20T18:57:54.160870+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:54.897933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:54.897933+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":400,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:56.405392+0000","last_change":"2024-09-20T18:57:56.405392+0000","last_active":"2024-09-20T18:57:56.405392+0000","last_peered":"2024-09-20T18:57:56.405392+0000","last_clean":"2024-09-20T18:57:56.405392+0000","last_became_active":"2024-09-20T18:55:28.833095+0000","last_became_peered":"2024-09-20T18:55:28.833095+0000","last_unstale":"2024-09-20T18:57:56.405392+0000","last_undegraded":"2024-09-20T18:57:56.405392+0000","last_fullsized":"2024-09-20T18:57:56.405392+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:56.405349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:56.405349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:22.442121+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":379,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.162569+0000","last_change":"2024-09-20T18:54:43.379898+0000","last_active":"2024-09-20T18:57:54.162569+0000","last_peered":"2024-09-20T18:57:54.162569+0000","last_clean":"2024-09-20T18:57:54.162569+0000","last_became_active":"2024-09-20T18:49:49.956767+0000","last_became_peered":"2024-09-20T18:49:49.956767+0000","last_unstale":"2024-09-20T18:57:54.162569+0000","last_undegraded":"2024-09-20T18:57:54.162569+0000","last_fullsized":"2024-09-20T18:57:54.162569+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:43.379706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:43.379706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":365,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.160590+0000","last_change":"2024-09-20T18:54:24.976627+0000","last_active":"2024-09-20T18:57:54.160590+0000","last_peered":"2024-09-20T18:57:54.160590+0000","last_clean":"2024-09-20T18:57:54.160590+0000","last_became_active":"2024-09-20T18:49:51.964098+0000","last_became_peered":"2024-09-20T18:49:51.964098+0000","last_unstale":"2024-09-20T18:57:54.160590+0000","last_undegraded":"2024-09-20T18:57:54.160590+0000","last_fullsized":"2024-09-20T18:57:54.160590+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:24.976584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:24.976584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"233'13860","reported_seq":20836,"reported_epoch":233,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-20T18:58:07.093289+0000","last_change":"2024-09-20T18:56:27.871592+0000","last_active":"2024-09-20T18:58:07.093289+0000","last_peered":"2024-09-20T18:58:07.093289+0000","last_clean":"2024-09-20T18:55:27.791097+0000","last_became_active":"2024-09-20T18:55:29.979932+0000","last_became_peered":"2024-09-20T18:55:29.979932+0000","last_unstale":"2024-09-20T18:58:07.093289+0000","last_undegraded":"2024-09-20T18:56:27.867865+0000","last_fullsized":"2024-09-20T18:55:29.815116+0000","mapping_epoch":182,"log_start":"210'8940","ondisk_log_start":"210'8940","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4920,"log_dups_size":0,"ondisk_log_size":4920,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38535168,"num_objects":588,"num_object_clones":0,"num_object_copies":1764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1770,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13860,"num_write_kb":462336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3408,"num_bytes_recovered":175964160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7],"avail_no_missing":["0","6","7"],"object_location_counts":[{"shards":"0,7","objects":-1182}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":195,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.164456+0000","last_change":"2024-09-20T18:57:22.033602+0000","last_active":"2024-09-20T18:57:54.164456+0000","last_peered":"2024-09-20T18:57:54.164456+0000","last_clean":"2024-09-20T18:57:54.164456+0000","last_became_active":"2024-09-20T18:55:28.828563+0000","last_became_peered":"2024-09-20T18:55:28.828563+0000","last_unstale":"2024-09-20T18:57:54.164456+0000","last_undegraded":"2024-09-20T18:57:54.164456+0000","last_fullsized":"2024-09-20T18:57:54.164456+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-20T18:57:22.033475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:22.033475+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:43.998101+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":31,"num_read_kb":20,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,5],"acting":[1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":396,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-20T18:58:10.740086+0000","last_change":"2024-09-20T18:55:28.835767+0000","last_active":"2024-09-20T18:58:10.740086+0000","last_peered":"2024-09-20T18:58:10.740086+0000","last_clean":"2024-09-20T18:58:10.740086+0000","last_became_active":"2024-09-20T18:55:28.835149+0000","last_became_peered":"2024-09-20T18:55:28.835149+0000","last_unstale":"2024-09-20T18:58:10.740086+0000","last_undegraded":"2024-09-20T18:58:10.740086+0000","last_fullsized":"2024-09-20T18:58:10.740086+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:44.501192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.501192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":379,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.164116+0000","last_change":"2024-09-20T18:55:13.577772+0000","last_active":"2024-09-20T18:57:54.164116+0000","last_peered":"2024-09-20T18:57:54.164116+0000","last_clean":"2024-09-20T18:57:54.164116+0000","last_became_active":"2024-09-20T18:49:49.957558+0000","last_became_peered":"2024-09-20T18:49:49.957558+0000","last_unstale":"2024-09-20T18:57:54.164116+0000","last_undegraded":"2024-09-20T18:57:54.164116+0000","last_fullsized":"2024-09-20T18:57:54.164116+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:13.577699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:13.577699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":377,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-20T18:58:10.738203+0000","last_change":"2024-09-20T18:53:47.370152+0000","last_active":"2024-09-20T18:58:10.738203+0000","last_peered":"2024-09-20T18:58:10.738203+0000","last_clean":"2024-09-20T18:58:10.738203+0000","last_became_active":"2024-09-20T18:49:51.970798+0000","last_became_peered":"2024-09-20T18:49:51.970798+0000","last_unstale":"2024-09-20T18:58:10.738203+0000","last_undegraded":"2024-09-20T18:58:10.738203+0000","last_fullsized":"2024-09-20T18:58:10.738203+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:47.370108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:47.370108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"235'14657","reported_seq":25627,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-20T18:58:10.925666+0000","last_change":"2024-09-20T18:58:07.953371+0000","last_active":"2024-09-20T18:58:10.925666+0000","last_peered":"2024-09-20T18:58:10.925666+0000","last_clean":"2024-09-20T18:58:10.925666+0000","last_became_active":"2024-09-20T18:57:20.718369+0000","last_became_peered":"2024-09-20T18:57:20.718369+0000","last_unstale":"2024-09-20T18:58:10.925666+0000","last_undegraded":"2024-09-20T18:58:10.925666+0000","last_fullsized":"2024-09-20T18:58:10.925666+0000","mapping_epoch":221,"log_start":"212'10046","ondisk_log_start":"212'10046","created":160,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"233'14450","last_scrub_stamp":"2024-09-20T18:58:07.953275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:58:07.953275+0000","objects_scrubbed":542,"log_size":4611,"log_dups_size":0,"ondisk_log_size":4611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:22.081382+0000","scrub_duration":135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21561344,"num_objects":329,"num_object_clones":0,"num_object_copies":987,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14657,"num_write_kb":479552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5391,"num_bytes_recovered":353304576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":401,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-20T18:58:10.731010+0000","last_change":"2024-09-20T18:53:52.898818+0000","last_active":"2024-09-20T18:58:10.731010+0000","last_peered":"2024-09-20T18:58:10.731010+0000","last_clean":"2024-09-20T18:58:10.731010+0000","last_became_active":"2024-09-20T18:49:47.949116+0000","last_became_peered":"2024-09-20T18:49:47.949116+0000","last_unstale":"2024-09-20T18:58:10.731010+0000","last_undegraded":"2024-09-20T18:58:10.731010+0000","last_fullsized":"2024-09-20T18:58:10.731010+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:53:52.898760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:52.898760+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:42.918366+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":389,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-20T18:58:10.743504+0000","last_change":"2024-09-20T18:54:52.650798+0000","last_active":"2024-09-20T18:58:10.743504+0000","last_peered":"2024-09-20T18:58:10.743504+0000","last_clean":"2024-09-20T18:58:10.743504+0000","last_became_active":"2024-09-20T18:49:45.944858+0000","last_became_peered":"2024-09-20T18:49:45.944858+0000","last_unstale":"2024-09-20T18:58:10.743504+0000","last_undegraded":"2024-09-20T18:58:10.743504+0000","last_fullsized":"2024-09-20T18:58:10.743504+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:52.650756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:52.650756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":371,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.162136+0000","last_change":"2024-09-20T18:54:42.349879+0000","last_active":"2024-09-20T18:57:54.162136+0000","last_peered":"2024-09-20T18:57:54.162136+0000","last_clean":"2024-09-20T18:57:54.162136+0000","last_became_active":"2024-09-20T18:49:49.954944+0000","last_became_peered":"2024-09-20T18:49:49.954944+0000","last_unstale":"2024-09-20T18:57:54.162136+0000","last_undegraded":"2024-09-20T18:57:54.162136+0000","last_fullsized":"2024-09-20T18:57:54.162136+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:42.349807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:42.349807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":366,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.143716+0000","last_change":"2024-09-20T18:55:17.341862+0000","last_active":"2024-09-20T18:57:54.143716+0000","last_peered":"2024-09-20T18:57:54.143716+0000","last_clean":"2024-09-20T18:57:54.143716+0000","last_became_active":"2024-09-20T18:49:51.972970+0000","last_became_peered":"2024-09-20T18:49:51.972970+0000","last_unstale":"2024-09-20T18:57:54.143716+0000","last_undegraded":"2024-09-20T18:57:54.143716+0000","last_fullsized":"2024-09-20T18:57:54.143716+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:17.341793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:17.341793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.4","version":"233'13688","reported_seq":13846,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:58:05.939617+0000","last_change":"2024-09-20T18:54:34.309442+0000","last_active":"2024-09-20T18:58:05.939617+0000","last_peered":"2024-09-20T18:58:05.939617+0000","last_clean":"2024-09-20T18:58:05.939617+0000","last_became_active":"2024-09-20T18:54:34.309113+0000","last_became_peered":"2024-09-20T18:54:34.309113+0000","last_unstale":"2024-09-20T18:58:05.939617+0000","last_undegraded":"2024-09-20T18:58:05.939617+0000","last_fullsized":"2024-09-20T18:58:05.939617+0000","mapping_epoch":160,"log_start":"208'8631","ondisk_log_start":"208'8631","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":5057,"log_dups_size":0,"ondisk_log_size":5057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:40.627488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42205184,"num_objects":644,"num_object_clones":0,"num_object_copies":1932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13688,"num_write_kb":458624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":421,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.149012+0000","last_change":"2024-09-20T18:55:28.847010+0000","last_active":"2024-09-20T18:57:54.149012+0000","last_peered":"2024-09-20T18:57:54.149012+0000","last_clean":"2024-09-20T18:57:54.149012+0000","last_became_active":"2024-09-20T18:55:28.824801+0000","last_became_peered":"2024-09-20T18:55:28.824801+0000","last_unstale":"2024-09-20T18:57:54.149012+0000","last_undegraded":"2024-09-20T18:57:54.149012+0000","last_fullsized":"2024-09-20T18:57:54.149012+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:54:12.373309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:12.373309+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:29.516079+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":409,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-20T18:58:10.739397+0000","last_change":"2024-09-20T18:55:24.729301+0000","last_active":"2024-09-20T18:58:10.739397+0000","last_peered":"2024-09-20T18:58:10.739397+0000","last_clean":"2024-09-20T18:58:10.739397+0000","last_became_active":"2024-09-20T18:49:45.944256+0000","last_became_peered":"2024-09-20T18:49:45.944256+0000","last_unstale":"2024-09-20T18:58:10.739397+0000","last_undegraded":"2024-09-20T18:58:10.739397+0000","last_fullsized":"2024-09-20T18:58:10.739397+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-20T18:55:24.729219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:24.729219+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"184'9","reported_seq":485,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-20T18:58:10.738945+0000","last_change":"2024-09-20T18:55:28.824771+0000","last_active":"2024-09-20T18:58:10.738945+0000","last_peered":"2024-09-20T18:58:10.738945+0000","last_clean":"2024-09-20T18:58:10.738945+0000","last_became_active":"2024-09-20T18:55:28.818085+0000","last_became_peered":"2024-09-20T18:55:28.818085+0000","last_unstale":"2024-09-20T18:58:10.738945+0000","last_undegraded":"2024-09-20T18:58:10.738945+0000","last_fullsized":"2024-09-20T18:58:10.738945+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:52:05.788924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.788924+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":380,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:58:05.523446+0000","last_change":"2024-09-20T18:58:05.523446+0000","last_active":"2024-09-20T18:58:05.523446+0000","last_peered":"2024-09-20T18:58:05.523446+0000","last_clean":"2024-09-20T18:58:05.523446+0000","last_became_active":"2024-09-20T18:55:28.830380+0000","last_became_peered":"2024-09-20T18:55:28.830380+0000","last_unstale":"2024-09-20T18:58:05.523446+0000","last_undegraded":"2024-09-20T18:58:05.523446+0000","last_fullsized":"2024-09-20T18:58:05.523446+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:05.523359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:05.523359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:28.858918+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":116,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.159986+0000","last_change":"2024-09-20T18:55:28.835344+0000","last_active":"2024-09-20T18:57:54.159986+0000","last_peered":"2024-09-20T18:57:54.159986+0000","last_clean":"2024-09-20T18:57:54.159986+0000","last_became_active":"2024-09-20T18:55:28.835010+0000","last_became_peered":"2024-09-20T18:55:28.835010+0000","last_unstale":"2024-09-20T18:57:54.159986+0000","last_undegraded":"2024-09-20T18:57:54.159986+0000","last_fullsized":"2024-09-20T18:57:54.159986+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:26.721735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:26.721735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":383,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.145722+0000","last_change":"2024-09-20T18:54:44.194161+0000","last_active":"2024-09-20T18:57:54.145722+0000","last_peered":"2024-09-20T18:57:54.145722+0000","last_clean":"2024-09-20T18:57:54.145722+0000","last_became_active":"2024-09-20T18:49:48.500662+0000","last_became_peered":"2024-09-20T18:49:48.500662+0000","last_unstale":"2024-09-20T18:57:54.145722+0000","last_undegraded":"2024-09-20T18:57:54.145722+0000","last_fullsized":"2024-09-20T18:57:54.145722+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-20T18:54:44.194089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.194089+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":369,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.146207+0000","last_change":"2024-09-20T18:53:16.220554+0000","last_active":"2024-09-20T18:57:54.146207+0000","last_peered":"2024-09-20T18:57:54.146207+0000","last_clean":"2024-09-20T18:57:54.146207+0000","last_became_active":"2024-09-20T18:49:49.956749+0000","last_became_peered":"2024-09-20T18:49:49.956749+0000","last_unstale":"2024-09-20T18:57:54.146207+0000","last_undegraded":"2024-09-20T18:57:54.146207+0000","last_fullsized":"2024-09-20T18:57:54.146207+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:16.220490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:16.220490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:27.545117+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":384,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-20T18:58:10.739542+0000","last_change":"2024-09-20T18:55:28.827897+0000","last_active":"2024-09-20T18:58:10.739542+0000","last_peered":"2024-09-20T18:58:10.739542+0000","last_clean":"2024-09-20T18:58:10.739542+0000","last_became_active":"2024-09-20T18:55:28.827418+0000","last_became_peered":"2024-09-20T18:55:28.827418+0000","last_unstale":"2024-09-20T18:58:10.739542+0000","last_undegraded":"2024-09-20T18:58:10.739542+0000","last_fullsized":"2024-09-20T18:58:10.739542+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.637739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.637739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":389,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.163652+0000","last_change":"2024-09-20T18:54:44.830872+0000","last_active":"2024-09-20T18:57:54.163652+0000","last_peered":"2024-09-20T18:57:54.163652+0000","last_clean":"2024-09-20T18:57:54.163652+0000","last_became_active":"2024-09-20T18:49:48.500803+0000","last_became_peered":"2024-09-20T18:49:48.500803+0000","last_unstale":"2024-09-20T18:57:54.163652+0000","last_undegraded":"2024-09-20T18:57:54.163652+0000","last_fullsized":"2024-09-20T18:57:54.163652+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:44.830790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.830790+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":388,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-20T18:58:10.739122+0000","last_change":"2024-09-20T18:55:29.751718+0000","last_active":"2024-09-20T18:58:10.739122+0000","last_peered":"2024-09-20T18:58:10.739122+0000","last_clean":"2024-09-20T18:58:10.739122+0000","last_became_active":"2024-09-20T18:55:28.833799+0000","last_became_peered":"2024-09-20T18:55:28.833799+0000","last_unstale":"2024-09-20T18:58:10.739122+0000","last_undegraded":"2024-09-20T18:58:10.739122+0000","last_fullsized":"2024-09-20T18:58:10.739122+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:29.751642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:29.751642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":378,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.144437+0000","last_change":"2024-09-20T18:57:46.898929+0000","last_active":"2024-09-20T18:57:54.144437+0000","last_peered":"2024-09-20T18:57:54.144437+0000","last_clean":"2024-09-20T18:57:54.144437+0000","last_became_active":"2024-09-20T18:49:49.961947+0000","last_became_peered":"2024-09-20T18:49:49.961947+0000","last_unstale":"2024-09-20T18:57:54.144437+0000","last_undegraded":"2024-09-20T18:57:54.144437+0000","last_fullsized":"2024-09-20T18:57:54.144437+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:46.898862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:46.898862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:06.640666+0000","scrub_duration":38,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"66'1","reported_seq":376,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.163822+0000","last_change":"2024-09-20T18:54:20.681878+0000","last_active":"2024-09-20T18:57:54.163822+0000","last_peered":"2024-09-20T18:57:54.163822+0000","last_clean":"2024-09-20T18:57:54.163822+0000","last_became_active":"2024-09-20T18:49:51.965121+0000","last_became_peered":"2024-09-20T18:49:51.965121+0000","last_unstale":"2024-09-20T18:57:54.163822+0000","last_undegraded":"2024-09-20T18:57:54.163822+0000","last_fullsized":"2024-09-20T18:57:54.163822+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T18:54:20.681790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:20.681790+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":389,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:59.430353+0000","last_change":"2024-09-20T18:57:59.430353+0000","last_active":"2024-09-20T18:57:59.430353+0000","last_peered":"2024-09-20T18:57:59.430353+0000","last_clean":"2024-09-20T18:57:59.430353+0000","last_became_active":"2024-09-20T18:49:48.501011+0000","last_became_peered":"2024-09-20T18:49:48.501011+0000","last_unstale":"2024-09-20T18:57:59.430353+0000","last_undegraded":"2024-09-20T18:57:59.430353+0000","last_fullsized":"2024-09-20T18:57:59.430353+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:57:59.430294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:59.430294+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:21.333603+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":385,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.160661+0000","last_change":"2024-09-20T18:55:27.028184+0000","last_active":"2024-09-20T18:57:54.160661+0000","last_peered":"2024-09-20T18:57:54.160661+0000","last_clean":"2024-09-20T18:57:54.160661+0000","last_became_active":"2024-09-20T18:49:45.938511+0000","last_became_peered":"2024-09-20T18:49:45.938511+0000","last_unstale":"2024-09-20T18:57:54.160661+0000","last_undegraded":"2024-09-20T18:57:54.160661+0000","last_fullsized":"2024-09-20T18:57:54.160661+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:27.028126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:27.028126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":487,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:58:01.091384+0000","last_change":"2024-09-20T18:55:27.284156+0000","last_active":"2024-09-20T18:58:01.091384+0000","last_peered":"2024-09-20T18:58:01.091384+0000","last_clean":"2024-09-20T18:58:01.091384+0000","last_became_active":"2024-09-20T18:49:49.960550+0000","last_became_peered":"2024-09-20T18:49:49.960550+0000","last_unstale":"2024-09-20T18:58:01.091384+0000","last_undegraded":"2024-09-20T18:58:01.091384+0000","last_fullsized":"2024-09-20T18:58:01.091384+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:27.284086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:27.284086+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":378,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-20T18:58:10.730811+0000","last_change":"2024-09-20T18:55:12.348079+0000","last_active":"2024-09-20T18:58:10.730811+0000","last_peered":"2024-09-20T18:58:10.730811+0000","last_clean":"2024-09-20T18:58:10.730811+0000","last_became_active":"2024-09-20T18:49:51.966254+0000","last_became_peered":"2024-09-20T18:49:51.966254+0000","last_unstale":"2024-09-20T18:58:10.730811+0000","last_undegraded":"2024-09-20T18:58:10.730811+0000","last_fullsized":"2024-09-20T18:58:10.730811+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:12.347993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:12.347993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":408,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.142497+0000","last_change":"2024-09-20T18:57:20.701592+0000","last_active":"2024-09-20T18:57:54.142497+0000","last_peered":"2024-09-20T18:57:54.142497+0000","last_clean":"2024-09-20T18:57:54.142497+0000","last_became_active":"2024-09-20T18:49:58.997494+0000","last_became_peered":"2024-09-20T18:49:58.997494+0000","last_unstale":"2024-09-20T18:57:54.142497+0000","last_undegraded":"2024-09-20T18:57:54.142497+0000","last_fullsized":"2024-09-20T18:57:54.142497+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:57:20.701543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:20.701543+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:44.352173+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":389,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-20T18:58:10.738719+0000","last_change":"2024-09-20T18:54:57.632458+0000","last_active":"2024-09-20T18:58:10.738719+0000","last_peered":"2024-09-20T18:58:10.738719+0000","last_clean":"2024-09-20T18:58:10.738719+0000","last_became_active":"2024-09-20T18:49:45.949492+0000","last_became_peered":"2024-09-20T18:49:45.949492+0000","last_unstale":"2024-09-20T18:58:10.738719+0000","last_undegraded":"2024-09-20T18:58:10.738719+0000","last_fullsized":"2024-09-20T18:58:10.738719+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.632380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.632380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":485,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:56.090409+0000","last_change":"2024-09-20T18:55:28.302353+0000","last_active":"2024-09-20T18:57:56.090409+0000","last_peered":"2024-09-20T18:57:56.090409+0000","last_clean":"2024-09-20T18:57:56.090409+0000","last_became_active":"2024-09-20T18:49:49.962243+0000","last_became_peered":"2024-09-20T18:49:49.962243+0000","last_unstale":"2024-09-20T18:57:56.090409+0000","last_undegraded":"2024-09-20T18:57:56.090409+0000","last_fullsized":"2024-09-20T18:57:56.090409+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:28.302303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:28.302303+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":373,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.163155+0000","last_change":"2024-09-20T18:55:11.918284+0000","last_active":"2024-09-20T18:57:54.163155+0000","last_peered":"2024-09-20T18:57:54.163155+0000","last_clean":"2024-09-20T18:57:54.163155+0000","last_became_active":"2024-09-20T18:49:51.969093+0000","last_became_peered":"2024-09-20T18:49:51.969093+0000","last_unstale":"2024-09-20T18:57:54.163155+0000","last_undegraded":"2024-09-20T18:57:54.163155+0000","last_fullsized":"2024-09-20T18:57:54.163155+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:11.918210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:11.918210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":424,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-20T18:58:10.731611+0000","last_change":"2024-09-20T18:57:50.899576+0000","last_active":"2024-09-20T18:58:10.731611+0000","last_peered":"2024-09-20T18:58:10.731611+0000","last_clean":"2024-09-20T18:58:10.731611+0000","last_became_active":"2024-09-20T18:55:28.826721+0000","last_became_peered":"2024-09-20T18:55:28.826721+0000","last_unstale":"2024-09-20T18:58:10.731611+0000","last_undegraded":"2024-09-20T18:58:10.731611+0000","last_fullsized":"2024-09-20T18:58:10.731611+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:57:50.899514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:50.899514+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:12.210010+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":381,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-20T18:58:10.732220+0000","last_change":"2024-09-20T18:55:28.822690+0000","last_active":"2024-09-20T18:58:10.732220+0000","last_peered":"2024-09-20T18:58:10.732220+0000","last_clean":"2024-09-20T18:58:10.732220+0000","last_became_active":"2024-09-20T18:55:28.822258+0000","last_became_peered":"2024-09-20T18:55:28.822258+0000","last_unstale":"2024-09-20T18:58:10.732220+0000","last_undegraded":"2024-09-20T18:58:10.732220+0000","last_fullsized":"2024-09-20T18:58:10.732220+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:05.985551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.985551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:37.926823+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":370,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.162832+0000","last_change":"2024-09-20T18:57:11.986605+0000","last_active":"2024-09-20T18:57:54.162832+0000","last_peered":"2024-09-20T18:57:54.162832+0000","last_clean":"2024-09-20T18:57:54.162832+0000","last_became_active":"2024-09-20T18:49:49.957067+0000","last_became_peered":"2024-09-20T18:49:49.957067+0000","last_unstale":"2024-09-20T18:57:54.162832+0000","last_undegraded":"2024-09-20T18:57:54.162832+0000","last_fullsized":"2024-09-20T18:57:54.162832+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:11.986528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:11.986528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:36.007001+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":366,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.141521+0000","last_change":"2024-09-20T18:57:16.789412+0000","last_active":"2024-09-20T18:57:54.141521+0000","last_peered":"2024-09-20T18:57:54.141521+0000","last_clean":"2024-09-20T18:57:54.141521+0000","last_became_active":"2024-09-20T18:49:51.966915+0000","last_became_peered":"2024-09-20T18:49:51.966915+0000","last_unstale":"2024-09-20T18:57:54.141521+0000","last_undegraded":"2024-09-20T18:57:54.141521+0000","last_fullsized":"2024-09-20T18:57:54.141521+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:16.789095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:16.789095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:45.821909+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":415,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.164180+0000","last_change":"2024-09-20T18:55:28.838698+0000","last_active":"2024-09-20T18:57:54.164180+0000","last_peered":"2024-09-20T18:57:54.164180+0000","last_clean":"2024-09-20T18:57:54.164180+0000","last_became_active":"2024-09-20T18:55:28.827023+0000","last_became_peered":"2024-09-20T18:55:28.827023+0000","last_unstale":"2024-09-20T18:57:54.164180+0000","last_undegraded":"2024-09-20T18:57:54.164180+0000","last_fullsized":"2024-09-20T18:57:54.164180+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:53:53.263976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:53.263976+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:26.131891+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":381,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-20T18:58:10.739825+0000","last_change":"2024-09-20T18:54:42.487465+0000","last_active":"2024-09-20T18:58:10.739825+0000","last_peered":"2024-09-20T18:58:10.739825+0000","last_clean":"2024-09-20T18:58:10.739825+0000","last_became_active":"2024-09-20T18:49:45.944357+0000","last_became_peered":"2024-09-20T18:49:45.944357+0000","last_unstale":"2024-09-20T18:58:10.739825+0000","last_undegraded":"2024-09-20T18:58:10.739825+0000","last_fullsized":"2024-09-20T18:58:10.739825+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:42.487403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:42.487403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":378,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:57.915090+0000","last_change":"2024-09-20T18:57:57.915090+0000","last_active":"2024-09-20T18:57:57.915090+0000","last_peered":"2024-09-20T18:57:57.915090+0000","last_clean":"2024-09-20T18:57:57.915090+0000","last_became_active":"2024-09-20T18:49:49.962523+0000","last_became_peered":"2024-09-20T18:49:49.962523+0000","last_unstale":"2024-09-20T18:57:57.915090+0000","last_undegraded":"2024-09-20T18:57:57.915090+0000","last_fullsized":"2024-09-20T18:57:57.915090+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:57.914913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:57.914913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:01.797575+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":373,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.159650+0000","last_change":"2024-09-20T18:54:26.882441+0000","last_active":"2024-09-20T18:57:54.159650+0000","last_peered":"2024-09-20T18:57:54.159650+0000","last_clean":"2024-09-20T18:57:54.159650+0000","last_became_active":"2024-09-20T18:49:51.967313+0000","last_became_peered":"2024-09-20T18:49:51.967313+0000","last_unstale":"2024-09-20T18:57:54.159650+0000","last_undegraded":"2024-09-20T18:57:54.159650+0000","last_fullsized":"2024-09-20T18:57:54.159650+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:26.882375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:26.882375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":420,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.862785+0000","last_change":"2024-09-20T18:57:54.862785+0000","last_active":"2024-09-20T18:57:54.862785+0000","last_peered":"2024-09-20T18:57:54.862785+0000","last_clean":"2024-09-20T18:57:54.862785+0000","last_became_active":"2024-09-20T18:55:28.830801+0000","last_became_peered":"2024-09-20T18:55:28.830801+0000","last_unstale":"2024-09-20T18:57:54.862785+0000","last_undegraded":"2024-09-20T18:57:54.862785+0000","last_fullsized":"2024-09-20T18:57:54.862785+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:57:54.862329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:54.862329+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:09.269532+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":385,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.146754+0000","last_change":"2024-09-20T18:55:19.258723+0000","last_active":"2024-09-20T18:57:54.146754+0000","last_peered":"2024-09-20T18:57:54.146754+0000","last_clean":"2024-09-20T18:57:54.146754+0000","last_became_active":"2024-09-20T18:49:45.942105+0000","last_became_peered":"2024-09-20T18:49:45.942105+0000","last_unstale":"2024-09-20T18:57:54.146754+0000","last_undegraded":"2024-09-20T18:57:54.146754+0000","last_fullsized":"2024-09-20T18:57:54.146754+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:19.258674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:19.258674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":388,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-20T18:58:10.737767+0000","last_change":"2024-09-20T18:55:28.825004+0000","last_active":"2024-09-20T18:58:10.737767+0000","last_peered":"2024-09-20T18:58:10.737767+0000","last_clean":"2024-09-20T18:58:10.737767+0000","last_became_active":"2024-09-20T18:55:28.824355+0000","last_became_peered":"2024-09-20T18:55:28.824355+0000","last_unstale":"2024-09-20T18:58:10.737767+0000","last_undegraded":"2024-09-20T18:58:10.737767+0000","last_fullsized":"2024-09-20T18:58:10.737767+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:46.329547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:46.329547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":365,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.143928+0000","last_change":"2024-09-20T18:55:28.825607+0000","last_active":"2024-09-20T18:57:54.143928+0000","last_peered":"2024-09-20T18:57:54.143928+0000","last_clean":"2024-09-20T18:57:54.143928+0000","last_became_active":"2024-09-20T18:55:28.825296+0000","last_became_peered":"2024-09-20T18:55:28.825296+0000","last_unstale":"2024-09-20T18:57:54.143928+0000","last_undegraded":"2024-09-20T18:57:54.143928+0000","last_fullsized":"2024-09-20T18:57:54.143928+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:43.118805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:43.118805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:25.734066+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":401,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.149930+0000","last_change":"2024-09-20T18:55:28.836789+0000","last_active":"2024-09-20T18:57:54.149930+0000","last_peered":"2024-09-20T18:57:54.149930+0000","last_clean":"2024-09-20T18:57:54.149930+0000","last_became_active":"2024-09-20T18:55:28.824579+0000","last_became_peered":"2024-09-20T18:55:28.824579+0000","last_unstale":"2024-09-20T18:57:54.149930+0000","last_undegraded":"2024-09-20T18:57:54.149930+0000","last_fullsized":"2024-09-20T18:57:54.149930+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-20T18:55:20.282193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:20.282193+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":385,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.160180+0000","last_change":"2024-09-20T18:54:55.900487+0000","last_active":"2024-09-20T18:57:54.160180+0000","last_peered":"2024-09-20T18:57:54.160180+0000","last_clean":"2024-09-20T18:57:54.160180+0000","last_became_active":"2024-09-20T18:49:45.940988+0000","last_became_peered":"2024-09-20T18:49:45.940988+0000","last_unstale":"2024-09-20T18:57:54.160180+0000","last_undegraded":"2024-09-20T18:57:54.160180+0000","last_fullsized":"2024-09-20T18:57:54.160180+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:55.899611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:55.899611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":128,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-20T18:58:10.730612+0000","last_change":"2024-09-20T18:57:52.929891+0000","last_active":"2024-09-20T18:58:10.730612+0000","last_peered":"2024-09-20T18:58:10.730612+0000","last_clean":"2024-09-20T18:58:10.730612+0000","last_became_active":"2024-09-20T18:55:28.832534+0000","last_became_peered":"2024-09-20T18:55:28.832534+0000","last_unstale":"2024-09-20T18:58:10.730612+0000","last_undegraded":"2024-09-20T18:58:10.730612+0000","last_fullsized":"2024-09-20T18:58:10.730612+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:52.929828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:52.929828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:13.316360+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":361,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-20T18:58:10.738106+0000","last_change":"2024-09-20T18:51:09.761331+0000","last_active":"2024-09-20T18:58:10.738106+0000","last_peered":"2024-09-20T18:58:10.738106+0000","last_clean":"2024-09-20T18:58:10.738106+0000","last_became_active":"2024-09-20T18:49:51.968662+0000","last_became_peered":"2024-09-20T18:49:51.968662+0000","last_unstale":"2024-09-20T18:58:10.738106+0000","last_undegraded":"2024-09-20T18:58:10.738106+0000","last_fullsized":"2024-09-20T18:58:10.738106+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:09.761283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:09.761283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":358,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.166259+0000","last_change":"2024-09-20T18:57:10.970843+0000","last_active":"2024-09-20T18:57:54.166259+0000","last_peered":"2024-09-20T18:57:54.166259+0000","last_clean":"2024-09-20T18:57:54.166259+0000","last_became_active":"2024-09-20T18:49:51.968216+0000","last_became_peered":"2024-09-20T18:49:51.968216+0000","last_unstale":"2024-09-20T18:57:54.166259+0000","last_undegraded":"2024-09-20T18:57:54.166259+0000","last_fullsized":"2024-09-20T18:57:54.166259+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:10.970779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:10.970779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:38.733512+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":378,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.150916+0000","last_change":"2024-09-20T18:55:28.295445+0000","last_active":"2024-09-20T18:57:54.150916+0000","last_peered":"2024-09-20T18:57:54.150916+0000","last_clean":"2024-09-20T18:57:54.150916+0000","last_became_active":"2024-09-20T18:49:45.937992+0000","last_became_peered":"2024-09-20T18:49:45.937992+0000","last_unstale":"2024-09-20T18:57:54.150916+0000","last_undegraded":"2024-09-20T18:57:54.150916+0000","last_fullsized":"2024-09-20T18:57:54.150916+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:28.295399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:28.295399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":167,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.146467+0000","last_change":"2024-09-20T18:55:28.852468+0000","last_active":"2024-09-20T18:57:54.146467+0000","last_peered":"2024-09-20T18:57:54.146467+0000","last_clean":"2024-09-20T18:57:54.146467+0000","last_became_active":"2024-09-20T18:55:28.834675+0000","last_became_peered":"2024-09-20T18:55:28.834675+0000","last_unstale":"2024-09-20T18:57:54.146467+0000","last_undegraded":"2024-09-20T18:57:54.146467+0000","last_fullsized":"2024-09-20T18:57:54.146467+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:54:44.719676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.719676+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:38.535671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":12,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":362,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-20T18:57:54.163404+0000","last_change":"2024-09-20T18:51:28.120100+0000","last_active":"2024-09-20T18:57:54.163404+0000","last_peered":"2024-09-20T18:57:54.163404+0000","last_clean":"2024-09-20T18:57:54.163404+0000","last_became_active":"2024-09-20T18:49:49.957582+0000","last_became_peered":"2024-09-20T18:49:49.957582+0000","last_unstale":"2024-09-20T18:57:54.163404+0000","last_undegraded":"2024-09-20T18:57:54.163404+0000","last_fullsized":"2024-09-20T18:57:54.163404+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:28.119694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:28.119694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":586350616,"num_objects":8948,"num_object_clones":0,"num_object_copies":26844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1770,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8948,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":223230,"num_write_kb":7429633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26286,"num_bytes_recovered":1675296768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2882482176,"data_stored":2882469960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74163,"ondisk_log_size":74163,"up":48,"acting":47,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":67,"ondisk_log_size":67,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":695,"num_read_kb":452,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":1474,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":12140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":394,"ondisk_log_size":394,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":5275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":499,"num_read_kb":494,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266793,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":315692,"kb_used_data":108184,"kb_used_omap":1,"kb_used_meta":207486,"kb_avail":93429460,"statfs":{"total":95995035648,"available":95671767040,"internally_reserved":0,"allocated":110780416,"data_stored":110297894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":212466122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":0,"seq":193273528432,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757364,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":609044,"kb_used_data":334916,"kb_used_omap":1,"kb_used_meta":274110,"kb_avail":93136108,"statfs":{"total":95995035648,"available":95371374592,"internally_reserved":0,"allocated":342953984,"data_stored":342484572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":280689093},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986298,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":326344,"kb_used_data":106288,"kb_used_omap":1,"kb_used_meta":219966,"kb_avail":93418808,"statfs":{"total":95995035648,"available":95660859392,"internally_reserved":0,"allocated":108838912,"data_stored":108339158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":225245637},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182528,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":633000,"kb_used_data":276952,"kb_used_omap":1,"kb_used_meta":356030,"kb_avail":93112152,"statfs":{"total":95995035648,"available":95346843648,"internally_reserved":0,"allocated":283598848,"data_stored":283108918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":364575175},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411461,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":745612,"kb_used_data":349308,"kb_used_omap":1,"kb_used_meta":396286,"kb_avail":92999540,"statfs":{"total":95995035648,"available":95231528960,"internally_reserved":0,"allocated":357691392,"data_stored":357230421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":405797317},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,3,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574987,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":581120,"kb_used_data":275056,"kb_used_omap":1,"kb_used_meta":306046,"kb_avail":93164032,"statfs":{"total":95995035648,"available":95399968768,"internally_reserved":0,"allocated":281657344,"data_stored":281145265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":313391557},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738513,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":630312,"kb_used_data":270040,"kb_used_omap":1,"kb_used_meta":360190,"kb_avail":93114840,"statfs":{"total":95995035648,"available":95349596160,"internally_reserved":0,"allocated":276520960,"data_stored":276031650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":368835013},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":258998272,"data_stored":258998272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":269811712,"data_stored":269811712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":354881536,"data_stored":354877464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":268763136,"data_stored":268763136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":102432768,"data_stored":102432768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":330436608,"data_stored":330432536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1195442176,"data_stored":1195442176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":101715968,"data_stored":101711896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T18:58:14.244 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph status --format=json 2024-09-20T18:58:14.244 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:14.269 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:14.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:14.270 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:14.293 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:58:14.303 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:58:14.371 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:14.415 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:14.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:14.416 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:14.446 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:14 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:58:14.446 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:14 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:58:14.446 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:14 smithi083 ceph-mon[29421]: pgmap v403: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 268 MiB data, 2.9 GiB used, 623 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.28k op/s; 1602/13458 objects degraded (11.904%); 0 B/s, 42 objects/s recovering 2024-09-20T18:58:14.446 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:14 smithi083 ceph-mon[29421]: osdmap e236: 8 total, 8 up, 7 in 2024-09-20T18:58:14.447 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:14 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:58:14.447 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:14 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:58:14.447 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:14 smithi083 ceph-mon[36260]: pgmap v403: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 268 MiB data, 2.9 GiB used, 623 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.28k op/s; 1602/13458 objects degraded (11.904%); 0 B/s, 42 objects/s recovering 2024-09-20T18:58:14.447 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:14 smithi083 ceph-mon[36260]: osdmap e236: 8 total, 8 up, 7 in 2024-09-20T18:58:14.518 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:14.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:14 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:58:14.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:14 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:58:14.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:14 smithi142 ceph-mon[32508]: pgmap v403: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 268 MiB data, 2.9 GiB used, 623 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.28k op/s; 1602/13458 objects degraded (11.904%); 0 B/s, 42 objects/s recovering 2024-09-20T18:58:14.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:14 smithi142 ceph-mon[32508]: osdmap e236: 8 total, 8 up, 7 in 2024-09-20T18:58:14.554 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:14.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:14.556 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:14.656 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:14.685 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:14.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:14.686 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:14.721 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T18:58:14.721 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:58:14.789 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:14.824 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:14.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:14.825 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:14.925 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:14.950 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:14.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:14.951 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:15.052 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:15.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:15.091 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:15.091 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:15.192 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:15.195 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:58:15.196 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:58:15.217 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:15.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:15.218 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:15.234 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:58:15.235 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:58:15.275 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:58:15.275 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:58:15.318 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:15.321 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:58:15.322 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:58:15.355 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:58:15.355 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:58:15.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:15.359 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:15.359 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:15.404 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_blocked_ops 2024-09-20T18:58:15.410 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:58:15.411 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:58:15.443 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:58:15.444 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:58:15.449 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Removed 116088 objects 2024-09-20T18:58:15.450 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Clean up completed and total clean up time :125.252 2024-09-20T18:58:15.454 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:58:15.460 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:15.465 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:58:15.465 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:58:15.481 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:15.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:15.482 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:15.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:15 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1976003273' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:58:15.548 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:15 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1976003273' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:58:15.583 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:15.606 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:15.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:15.607 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:15.661 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:58:15.708 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:58:15.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:15.730 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:15.731 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:58:15.737 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-20T18:58:15.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:15 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1976003273' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:58:15.831 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:15.853 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:15.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:15.854 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:15.955 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:15.977 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:15.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:15.978 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:16.079 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:16.100 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:16.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:16.101 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:16.139 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T18:58:16.139 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-09-20T18:58:16.139 INFO:teuthology.orchestra.run.smithi142.stdout: "complaint_time": 30, 2024-09-20T18:58:16.139 INFO:teuthology.orchestra.run.smithi142.stdout: "num_blocked_ops": 0 2024-09-20T18:58:16.140 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T18:58:16.202 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:58:16.223 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:16.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:16.224 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:58:16.325 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:16.347 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:16.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:16.348 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:16.443 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:16 smithi142 ceph-mon[32508]: pgmap v405: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 268 MiB data, 2.9 GiB used, 623 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 891 op/s; 1602/13458 objects degraded (11.904%); 0 B/s, 27 objects/s recovering 2024-09-20T18:58:16.443 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:16 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1660780863' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:58:16.443 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:16 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 1602/13458 objects degraded (11.904%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T18:58:16.443 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:16 smithi142 ceph-mon[32508]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:58:16.443 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:16 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1660780863' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T18:58:16.443 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:16 smithi142 ceph-mon[32508]: osdmap e237: 8 total, 8 up, 7 in 2024-09-20T18:58:16.443 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:16 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:58:16.443 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:16 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:58:16.449 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:16.470 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:16.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:16.471 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:16.548 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:16 smithi083 ceph-mon[36260]: pgmap v405: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 268 MiB data, 2.9 GiB used, 623 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 891 op/s; 1602/13458 objects degraded (11.904%); 0 B/s, 27 objects/s recovering 2024-09-20T18:58:16.548 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:16 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1660780863' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:58:16.548 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:16 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 1602/13458 objects degraded (11.904%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T18:58:16.548 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:16 smithi083 ceph-mon[36260]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:58:16.548 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:16 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1660780863' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T18:58:16.548 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:16 smithi083 ceph-mon[36260]: osdmap e237: 8 total, 8 up, 7 in 2024-09-20T18:58:16.548 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:16 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:58:16.548 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:16 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:58:16.549 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:16 smithi083 ceph-mon[29421]: pgmap v405: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 268 MiB data, 2.9 GiB used, 623 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 891 op/s; 1602/13458 objects degraded (11.904%); 0 B/s, 27 objects/s recovering 2024-09-20T18:58:16.549 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:16 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1660780863' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:58:16.549 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:16 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 1602/13458 objects degraded (11.904%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T18:58:16.549 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:16 smithi083 ceph-mon[29421]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:58:16.549 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:16 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1660780863' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T18:58:16.549 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:16 smithi083 ceph-mon[29421]: osdmap e237: 8 total, 8 up, 7 in 2024-09-20T18:58:16.549 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:16 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:58:16.549 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:16 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:58:16.572 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:16.593 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:16.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:16.594 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:16.695 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:16.717 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:16.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:16.718 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:16.778 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_historic_ops 2024-09-20T18:58:16.820 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:16.841 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:16.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:16.842 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:16.943 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:16.965 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:16.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:16.966 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:17.017 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:58:16 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:58:17.028 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:58:17.068 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:17.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:17.090 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:17.090 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:17.191 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:17.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:17.213 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:17.213 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:17.314 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:17.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:17.338 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:17.339 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:17.439 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:17.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:17.462 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:17.463 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:17.523 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T18:58:17.524 INFO:teuthology.orchestra.run.smithi142.stdout: "size": 20, 2024-09-20T18:58:17.524 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 600, 2024-09-20T18:58:17.524 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-09-20T18:58:17.524 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.524 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:24949 7.0 7:017c073e:::benchmark_data_smithi083_70643_object24948:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:58:17.524 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:06.996980+0000", 2024-09-20T18:58:17.524 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 430.51387161299999, 2024-09-20T18:58:17.524 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.117885157, 2024-09-20T18:58:17.524 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:58:17.524 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:58:17.524 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:17.525 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:58:17.525 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:58:17.525 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:58:17.525 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 24949 2024-09-20T18:58:17.525 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.525 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:58:17.525 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.525 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:58:17.525 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.996980+0000", 2024-09-20T18:58:17.525 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:58:17.525 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.525 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.525 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:58:17.525 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.996980+0000", 2024-09-20T18:58:17.526 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:58:17.526 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.526 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.526 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:58:17.526 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.996982+0000", 2024-09-20T18:58:17.526 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.849e-06 2024-09-20T18:58:17.526 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.526 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.526 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:58:17.526 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.997040+0000", 2024-09-20T18:58:17.526 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8103000000000003e-05 2024-09-20T18:58:17.526 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.526 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.527 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:58:17.527 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.997052+0000", 2024-09-20T18:58:17.527 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2515e-05 2024-09-20T18:58:17.527 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.527 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.527 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:58:17.527 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.997060+0000", 2024-09-20T18:58:17.527 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.4240000000000004e-06 2024-09-20T18:58:17.527 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.527 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.527 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:58:17.527 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.997107+0000", 2024-09-20T18:58:17.527 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7579e-05 2024-09-20T18:58:17.527 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.528 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.528 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:58:17.528 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.997260+0000", 2024-09-20T18:58:17.528 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015316099999999999 2024-09-20T18:58:17.528 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.528 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.528 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,3", 2024-09-20T18:58:17.528 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.997397+0000", 2024-09-20T18:58:17.528 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000136737 2024-09-20T18:58:17.528 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.528 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.528 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:58:17.528 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:06.998701+0000", 2024-09-20T18:58:17.529 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001303413 2024-09-20T18:58:17.529 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.529 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.529 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:17.529 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.000575+0000", 2024-09-20T18:58:17.529 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0018739990000000001 2024-09-20T18:58:17.529 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.529 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.529 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:17.529 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.114762+0000", 2024-09-20T18:58:17.529 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.114187739 2024-09-20T18:58:17.529 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.529 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.529 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:58:17.530 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.114816+0000", 2024-09-20T18:58:17.530 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3541000000000001e-05 2024-09-20T18:58:17.530 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.530 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.530 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:58:17.530 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.114865+0000", 2024-09-20T18:58:17.530 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.9097000000000001e-05 2024-09-20T18:58:17.530 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:58:17.530 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:58:17.530 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:58:17.530 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.530 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.530 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:27568 7.0 7:04b5e4aa:::benchmark_data_smithi083_70643_object27567:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:58:17.531 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:08.739355+0000", 2024-09-20T18:58:17.531 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 428.77149603300001, 2024-09-20T18:58:17.531 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10425282499999999, 2024-09-20T18:58:17.531 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:58:17.531 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:58:17.531 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:17.531 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:58:17.531 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:58:17.531 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:58:17.531 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 27568 2024-09-20T18:58:17.531 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.531 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:58:17.531 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.531 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:58:17.532 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.739355+0000", 2024-09-20T18:58:17.532 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:58:17.532 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.532 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.532 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:58:17.532 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.739355+0000", 2024-09-20T18:58:17.532 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:58:17.532 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.532 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.532 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:58:17.532 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.739357+0000", 2024-09-20T18:58:17.532 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.271e-06 2024-09-20T18:58:17.532 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.532 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.533 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:58:17.533 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.739407+0000", 2024-09-20T18:58:17.533 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0322999999999999e-05 2024-09-20T18:58:17.533 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.533 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.533 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:58:17.533 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.739414+0000", 2024-09-20T18:58:17.533 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.486e-06 2024-09-20T18:58:17.533 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.533 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.533 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:58:17.533 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.739420+0000", 2024-09-20T18:58:17.533 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.733e-06 2024-09-20T18:58:17.533 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.533 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.534 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:58:17.534 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.740847+0000", 2024-09-20T18:58:17.534 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001427085 2024-09-20T18:58:17.534 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.534 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.534 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:58:17.534 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.741111+0000", 2024-09-20T18:58:17.534 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000263837 2024-09-20T18:58:17.534 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.534 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.534 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,3", 2024-09-20T18:58:17.534 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.741241+0000", 2024-09-20T18:58:17.534 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012974800000000001 2024-09-20T18:58:17.535 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.535 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.535 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:58:17.535 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.743321+0000", 2024-09-20T18:58:17.535 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0020804349999999998 2024-09-20T18:58:17.535 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.535 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.535 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:17.535 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.743507+0000", 2024-09-20T18:58:17.535 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00018593400000000001 2024-09-20T18:58:17.535 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.535 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.535 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:17.535 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.843476+0000", 2024-09-20T18:58:17.536 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.099968571000000006 2024-09-20T18:58:17.536 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.536 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.536 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:58:17.536 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.843566+0000", 2024-09-20T18:58:17.536 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.9741999999999996e-05 2024-09-20T18:58:17.536 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.536 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.536 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:58:17.536 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.843608+0000", 2024-09-20T18:58:17.536 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.2660000000000002e-05 2024-09-20T18:58:17.536 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:58:17.536 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:58:17.536 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:58:17.537 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.537 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.537 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:27569 7.0 7:0c21f1cc:::benchmark_data_smithi083_70643_object27568:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:58:17.537 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:08.739536+0000", 2024-09-20T18:58:17.537 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 428.77131588200001, 2024-09-20T18:58:17.537 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.105744901, 2024-09-20T18:58:17.537 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:58:17.537 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:58:17.537 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:17.537 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:58:17.537 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:58:17.538 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:58:17.538 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 27569 2024-09-20T18:58:17.538 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.539 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:58:17.539 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.539 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:58:17.539 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.739536+0000", 2024-09-20T18:58:17.539 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:58:17.539 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.539 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.539 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:58:17.539 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.739536+0000", 2024-09-20T18:58:17.539 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:58:17.539 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.539 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.540 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:58:17.540 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.739537+0000", 2024-09-20T18:58:17.540 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.6099999999999999e-07 2024-09-20T18:58:17.540 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.540 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.540 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:58:17.540 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.739598+0000", 2024-09-20T18:58:17.540 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.1107999999999998e-05 2024-09-20T18:58:17.540 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.540 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.540 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:58:17.540 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.739609+0000", 2024-09-20T18:58:17.540 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1109e-05 2024-09-20T18:58:17.540 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.541 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.541 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:58:17.541 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.739616+0000", 2024-09-20T18:58:17.541 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.0380000000000002e-06 2024-09-20T18:58:17.541 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.541 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.541 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:58:17.541 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.742252+0000", 2024-09-20T18:58:17.541 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0026361100000000001 2024-09-20T18:58:17.541 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.541 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.541 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:58:17.541 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.742350+0000", 2024-09-20T18:58:17.541 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.7936000000000006e-05 2024-09-20T18:58:17.542 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.542 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.542 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,3", 2024-09-20T18:58:17.542 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.742525+0000", 2024-09-20T18:58:17.542 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00017482199999999999 2024-09-20T18:58:17.542 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.542 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.542 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:58:17.542 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.743518+0000", 2024-09-20T18:58:17.542 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00099339400000000009 2024-09-20T18:58:17.542 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.542 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.542 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:17.542 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.743552+0000", 2024-09-20T18:58:17.543 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4109000000000002e-05 2024-09-20T18:58:17.543 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.543 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.543 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:17.543 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.845182+0000", 2024-09-20T18:58:17.543 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10163016800000001 2024-09-20T18:58:17.543 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.543 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.543 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:58:17.543 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.845239+0000", 2024-09-20T18:58:17.543 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6252000000000003e-05 2024-09-20T18:58:17.543 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.543 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.544 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:58:17.544 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.845280+0000", 2024-09-20T18:58:17.544 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.1894000000000002e-05 2024-09-20T18:58:17.544 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:58:17.544 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:58:17.544 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:58:17.544 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.544 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.544 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.24955.0:83350 8.9 8:9099de8d:::benchmark_data_smithi083_97422_object83349:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e183)", 2024-09-20T18:58:17.544 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:55:29.957756+0000", 2024-09-20T18:58:17.544 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 167.553095708, 2024-09-20T18:58:17.544 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10443042900000001, 2024-09-20T18:58:17.544 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:58:17.544 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:58:17.545 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:17.545 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:58:17.545 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.24955", 2024-09-20T18:58:17.545 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:58:17.545 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 83350 2024-09-20T18:58:17.545 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.545 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:58:17.545 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.545 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:58:17.545 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:29.957756+0000", 2024-09-20T18:58:17.545 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:58:17.545 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.545 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.545 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:58:17.546 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:29.957756+0000", 2024-09-20T18:58:17.546 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:58:17.546 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.546 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.546 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:58:17.546 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:29.957757+0000", 2024-09-20T18:58:17.546 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4079999999999999e-06 2024-09-20T18:58:17.546 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.546 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.546 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:58:17.546 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:29.957834+0000", 2024-09-20T18:58:17.546 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.6378999999999995e-05 2024-09-20T18:58:17.546 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.547 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.547 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:58:17.547 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:29.957842+0000", 2024-09-20T18:58:17.547 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.6379999999999993e-06 2024-09-20T18:58:17.547 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.547 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.547 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:58:17.547 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:29.957847+0000", 2024-09-20T18:58:17.547 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3610000000000003e-06 2024-09-20T18:58:17.547 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.547 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.547 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:58:17.547 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:29.957903+0000", 2024-09-20T18:58:17.547 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6619999999999997e-05 2024-09-20T18:58:17.548 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.548 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.548 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T18:58:17.548 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:29.957912+0000", 2024-09-20T18:58:17.548 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.1700000000000003e-06 2024-09-20T18:58:17.548 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.548 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.548 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:58:17.548 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:30.017150+0000", 2024-09-20T18:58:17.548 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.059237680000000001 2024-09-20T18:58:17.548 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.548 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.548 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:58:17.548 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:30.017286+0000", 2024-09-20T18:58:17.549 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013576899999999999 2024-09-20T18:58:17.549 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.549 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.549 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,3", 2024-09-20T18:58:17.549 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:30.017401+0000", 2024-09-20T18:58:17.549 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011528599999999999 2024-09-20T18:58:17.549 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.549 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.549 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:58:17.549 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:30.021235+0000", 2024-09-20T18:58:17.549 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0038338420000000001 2024-09-20T18:58:17.549 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.549 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.549 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:17.550 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:30.037163+0000", 2024-09-20T18:58:17.550 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.015927782000000001 2024-09-20T18:58:17.550 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.550 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.550 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:17.550 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:30.061255+0000", 2024-09-20T18:58:17.550 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.024092638999999999 2024-09-20T18:58:17.550 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.550 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.550 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:58:17.550 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:30.061292+0000", 2024-09-20T18:58:17.550 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6686000000000001e-05 2024-09-20T18:58:17.550 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.550 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.550 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:58:17.551 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:30.062186+0000", 2024-09-20T18:58:17.551 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000894169 2024-09-20T18:58:17.551 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:58:17.551 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:58:17.551 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:58:17.551 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.551 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.551 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.24955.0:98825 8.8 8:133d99cf:::benchmark_data_smithi083_97422_object98824:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e189)", 2024-09-20T18:58:17.551 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:55:49.196602+0000", 2024-09-20T18:58:17.551 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 148.31424900600001, 2024-09-20T18:58:17.551 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.11128930200000001, 2024-09-20T18:58:17.551 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:58:17.551 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:58:17.552 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:17.552 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:58:17.552 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.24955", 2024-09-20T18:58:17.552 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:58:17.552 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 98825 2024-09-20T18:58:17.552 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.552 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:58:17.552 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.552 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:58:17.552 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:49.196602+0000", 2024-09-20T18:58:17.552 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:58:17.552 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.552 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.552 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:58:17.553 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:49.196602+0000", 2024-09-20T18:58:17.553 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:58:17.553 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.553 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.553 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:58:17.553 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:49.196604+0000", 2024-09-20T18:58:17.553 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5689999999999999e-06 2024-09-20T18:58:17.553 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.553 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.553 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:58:17.553 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:49.196658+0000", 2024-09-20T18:58:17.553 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4064000000000003e-05 2024-09-20T18:58:17.553 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.553 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.554 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:58:17.554 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:49.196671+0000", 2024-09-20T18:58:17.554 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2611e-05 2024-09-20T18:58:17.554 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.554 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.554 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:58:17.554 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:49.196679+0000", 2024-09-20T18:58:17.554 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.9160000000000001e-06 2024-09-20T18:58:17.554 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.554 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.554 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:58:17.554 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:49.196745+0000", 2024-09-20T18:58:17.554 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.6551999999999996e-05 2024-09-20T18:58:17.554 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.555 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.555 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:58:17.555 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:49.196877+0000", 2024-09-20T18:58:17.555 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013138099999999999 2024-09-20T18:58:17.555 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.555 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.555 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:58:17.555 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:49.197023+0000", 2024-09-20T18:58:17.555 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00014689 2024-09-20T18:58:17.555 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.555 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.555 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:58:17.555 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:49.199812+0000", 2024-09-20T18:58:17.555 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0027884989999999998 2024-09-20T18:58:17.555 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.556 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.556 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:17.556 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:49.200058+0000", 2024-09-20T18:58:17.556 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00024575299999999998 2024-09-20T18:58:17.556 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.556 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.556 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:17.556 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:49.307800+0000", 2024-09-20T18:58:17.556 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10774196799999999 2024-09-20T18:58:17.556 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.556 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.556 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:58:17.556 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:49.307848+0000", 2024-09-20T18:58:17.556 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8096000000000002e-05 2024-09-20T18:58:17.556 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.557 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.557 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:58:17.557 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:49.307892+0000", 2024-09-20T18:58:17.557 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.4002999999999999e-05 2024-09-20T18:58:17.557 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:58:17.557 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:58:17.557 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:58:17.557 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.557 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.557 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.24955.0:98835 8.8 8:179535b3:::benchmark_data_smithi083_97422_object98834:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e189)", 2024-09-20T18:58:17.557 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:55:49.206528+0000", 2024-09-20T18:58:17.557 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 148.30432332399999, 2024-09-20T18:58:17.557 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.107683369, 2024-09-20T18:58:17.558 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:58:17.558 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:58:17.558 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:17.558 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:58:17.558 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.24955", 2024-09-20T18:58:17.558 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:58:17.558 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 98835 2024-09-20T18:58:17.558 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.558 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:58:17.558 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.558 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:58:17.558 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:49.206528+0000", 2024-09-20T18:58:17.558 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:58:17.559 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.559 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.559 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:58:17.559 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:49.206528+0000", 2024-09-20T18:58:17.559 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:58:17.559 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.559 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.559 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:58:17.559 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:49.206530+0000", 2024-09-20T18:58:17.559 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.407e-06 2024-09-20T18:58:17.559 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.559 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.559 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:58:17.559 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:49.206583+0000", 2024-09-20T18:58:17.560 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3637000000000002e-05 2024-09-20T18:58:17.560 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.560 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.560 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:58:17.560 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:49.206594+0000", 2024-09-20T18:58:17.560 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0973e-05 2024-09-20T18:58:17.560 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.560 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.560 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:58:17.560 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:49.206601+0000", 2024-09-20T18:58:17.560 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.3209999999999999e-06 2024-09-20T18:58:17.560 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.560 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.560 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:58:17.561 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:49.206657+0000", 2024-09-20T18:58:17.561 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5522999999999997e-05 2024-09-20T18:58:17.561 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.561 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.561 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:58:17.561 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:49.206754+0000", 2024-09-20T18:58:17.561 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.6677999999999998e-05 2024-09-20T18:58:17.561 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.561 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.561 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:58:17.561 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:49.206868+0000", 2024-09-20T18:58:17.561 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000114348 2024-09-20T18:58:17.561 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.561 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.562 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:58:17.562 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:49.209148+0000", 2024-09-20T18:58:17.562 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0022796629999999999 2024-09-20T18:58:17.562 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.562 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.562 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:17.562 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:49.209291+0000", 2024-09-20T18:58:17.562 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00014355600000000001 2024-09-20T18:58:17.562 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.562 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.562 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:17.562 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:49.314145+0000", 2024-09-20T18:58:17.562 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.104853539 2024-09-20T18:58:17.562 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.563 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.563 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:58:17.563 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:49.314179+0000", 2024-09-20T18:58:17.563 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.3794999999999997e-05 2024-09-20T18:58:17.563 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.563 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.563 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:58:17.563 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:49.314211+0000", 2024-09-20T18:58:17.563 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.2929000000000003e-05 2024-09-20T18:58:17.563 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:58:17.563 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:58:17.563 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:58:17.563 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.563 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.564 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.24955.0:101455 8.8 8:1ef316d4:::benchmark_data_smithi083_97422_object101454:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e190)", 2024-09-20T18:58:17.564 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:55:52.356987+0000", 2024-09-20T18:58:17.564 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 145.153863984, 2024-09-20T18:58:17.564 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.15453507499999999, 2024-09-20T18:58:17.564 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:58:17.564 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:58:17.564 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:17.564 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:58:17.564 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.24955", 2024-09-20T18:58:17.564 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:58:17.564 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 101455 2024-09-20T18:58:17.564 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.564 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:58:17.564 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.565 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:58:17.565 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.356987+0000", 2024-09-20T18:58:17.565 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:58:17.565 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.565 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.565 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:58:17.565 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.356987+0000", 2024-09-20T18:58:17.565 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:58:17.565 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.565 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.565 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:58:17.565 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.356989+0000", 2024-09-20T18:58:17.565 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3170000000000001e-06 2024-09-20T18:58:17.565 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.566 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.566 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:58:17.566 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.357036+0000", 2024-09-20T18:58:17.566 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7540999999999998e-05 2024-09-20T18:58:17.566 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.566 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.566 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:58:17.566 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.357047+0000", 2024-09-20T18:58:17.566 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1123e-05 2024-09-20T18:58:17.566 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.566 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.566 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:58:17.566 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.357053+0000", 2024-09-20T18:58:17.567 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4450000000000004e-06 2024-09-20T18:58:17.567 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.567 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.567 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:58:17.567 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.357246+0000", 2024-09-20T18:58:17.567 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00019330700000000001 2024-09-20T18:58:17.567 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.567 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.567 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:58:17.567 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.357342+0000", 2024-09-20T18:58:17.567 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.5838e-05 2024-09-20T18:58:17.567 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.567 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.567 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:58:17.568 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.357443+0000", 2024-09-20T18:58:17.568 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000100537 2024-09-20T18:58:17.568 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.568 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.568 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:58:17.568 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.360271+0000", 2024-09-20T18:58:17.568 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0028288330000000002 2024-09-20T18:58:17.568 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.568 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.568 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:17.568 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.363086+0000", 2024-09-20T18:58:17.568 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0028142620000000001 2024-09-20T18:58:17.568 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.569 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.569 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:17.569 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.511417+0000", 2024-09-20T18:58:17.569 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.148331466 2024-09-20T18:58:17.569 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.569 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.569 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:58:17.569 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.511477+0000", 2024-09-20T18:58:17.569 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9539e-05 2024-09-20T18:58:17.569 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.569 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.569 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:58:17.569 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.511523+0000", 2024-09-20T18:58:17.569 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.5867e-05 2024-09-20T18:58:17.569 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:58:17.570 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:58:17.570 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:58:17.570 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.570 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.570 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.24955.0:101481 8.8 8:1b23d429:::benchmark_data_smithi083_97422_object101480:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e190)", 2024-09-20T18:58:17.570 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:55:52.396131+0000", 2024-09-20T18:58:17.570 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 145.114720541, 2024-09-20T18:58:17.570 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.119361123, 2024-09-20T18:58:17.570 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:58:17.570 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:58:17.570 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:17.570 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:58:17.570 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.24955", 2024-09-20T18:58:17.571 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:58:17.571 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 101481 2024-09-20T18:58:17.571 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.571 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:58:17.571 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.571 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:58:17.571 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.396131+0000", 2024-09-20T18:58:17.571 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:58:17.571 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.571 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.571 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:58:17.571 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.396131+0000", 2024-09-20T18:58:17.571 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:58:17.572 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.572 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.572 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:58:17.572 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.396132+0000", 2024-09-20T18:58:17.572 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2410000000000001e-06 2024-09-20T18:58:17.572 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.572 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.572 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:58:17.572 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.396177+0000", 2024-09-20T18:58:17.572 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.4685999999999999e-05 2024-09-20T18:58:17.572 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.572 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.572 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:58:17.572 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.396185+0000", 2024-09-20T18:58:17.573 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.1990000000000006e-06 2024-09-20T18:58:17.573 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.573 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.573 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:58:17.573 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.396190+0000", 2024-09-20T18:58:17.573 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0649999999999998e-06 2024-09-20T18:58:17.573 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.573 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.573 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:58:17.573 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.396234+0000", 2024-09-20T18:58:17.573 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3869000000000003e-05 2024-09-20T18:58:17.573 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.573 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.573 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:58:17.573 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.396347+0000", 2024-09-20T18:58:17.574 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000113338 2024-09-20T18:58:17.574 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.574 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.574 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:58:17.574 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.396440+0000", 2024-09-20T18:58:17.574 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.2737000000000003e-05 2024-09-20T18:58:17.574 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.574 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.574 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:58:17.574 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.398764+0000", 2024-09-20T18:58:17.574 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0023238270000000001 2024-09-20T18:58:17.574 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.574 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.575 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:17.575 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.400458+0000", 2024-09-20T18:58:17.575 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001693754 2024-09-20T18:58:17.575 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.575 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.575 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:17.575 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.515393+0000", 2024-09-20T18:58:17.575 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.11493568799999999 2024-09-20T18:58:17.575 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.575 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.575 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:58:17.575 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.515448+0000", 2024-09-20T18:58:17.575 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4861000000000002e-05 2024-09-20T18:58:17.575 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.576 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.576 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:58:17.576 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.515492+0000", 2024-09-20T18:58:17.576 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3857999999999999e-05 2024-09-20T18:58:17.576 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:58:17.576 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:58:17.576 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:58:17.576 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.576 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.576 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.24955.0:136159 8.9 8:96b2d3d3:::benchmark_data_smithi083_97422_object20068:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e206)", 2024-09-20T18:58:17.576 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:56:34.540612+0000", 2024-09-20T18:58:17.576 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 102.970238968, 2024-09-20T18:58:17.576 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.14784939699999999, 2024-09-20T18:58:17.576 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:58:17.576 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:58:17.577 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:17.577 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:58:17.577 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.24955", 2024-09-20T18:58:17.577 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:58:17.577 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 136159 2024-09-20T18:58:17.577 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.577 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:58:17.577 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.577 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:58:17.577 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:34.540612+0000", 2024-09-20T18:58:17.577 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:58:17.577 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.577 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.577 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:58:17.577 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:34.540612+0000", 2024-09-20T18:58:17.578 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:58:17.578 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.578 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.578 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:58:17.578 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:34.540614+0000", 2024-09-20T18:58:17.578 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5549999999999999e-06 2024-09-20T18:58:17.578 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.578 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.578 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:58:17.578 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:34.540619+0000", 2024-09-20T18:58:17.578 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4829999999999999e-06 2024-09-20T18:58:17.578 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.578 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.578 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:58:17.579 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:34.540633+0000", 2024-09-20T18:58:17.579 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3679999999999999e-05 2024-09-20T18:58:17.579 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.579 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.579 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:58:17.579 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:34.540640+0000", 2024-09-20T18:58:17.579 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.1119999999999998e-06 2024-09-20T18:58:17.579 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.579 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.579 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:58:17.579 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:34.544885+0000", 2024-09-20T18:58:17.579 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.004244879 2024-09-20T18:58:17.579 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.580 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.580 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:58:17.580 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:34.545042+0000", 2024-09-20T18:58:17.580 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000157269 2024-09-20T18:58:17.580 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.580 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.580 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,3", 2024-09-20T18:58:17.580 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:34.545192+0000", 2024-09-20T18:58:17.580 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00014966299999999999 2024-09-20T18:58:17.580 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.580 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.580 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:17.580 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:34.550597+0000", 2024-09-20T18:58:17.580 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0054044949999999996 2024-09-20T18:58:17.581 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.581 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.581 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:58:17.581 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:34.550608+0000", 2024-09-20T18:58:17.581 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1688999999999999e-05 2024-09-20T18:58:17.581 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.581 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.581 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:17.581 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:34.687369+0000", 2024-09-20T18:58:17.581 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13676118900000001 2024-09-20T18:58:17.581 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.581 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.581 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:58:17.581 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:34.687406+0000", 2024-09-20T18:58:17.582 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6105000000000001e-05 2024-09-20T18:58:17.582 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.582 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.582 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:58:17.582 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:34.688462+0000", 2024-09-20T18:58:17.582 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0010562779999999999 2024-09-20T18:58:17.582 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:58:17.582 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:58:17.582 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:58:17.582 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.582 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.582 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.24955.0:136160 8.8 8:1f254e81:::benchmark_data_smithi083_97422_object20069:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e206)", 2024-09-20T18:58:17.582 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:56:34.540654+0000", 2024-09-20T18:58:17.583 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 102.97019706, 2024-09-20T18:58:17.583 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.139256357, 2024-09-20T18:58:17.583 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:58:17.583 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:58:17.583 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:17.583 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:58:17.583 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.24955", 2024-09-20T18:58:17.583 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:58:17.583 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 136160 2024-09-20T18:58:17.583 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.583 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:58:17.583 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.583 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:58:17.583 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:34.540654+0000", 2024-09-20T18:58:17.583 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:58:17.584 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.584 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.584 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:58:17.584 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:34.540654+0000", 2024-09-20T18:58:17.584 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:58:17.584 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.584 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.584 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:58:17.584 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:34.540655+0000", 2024-09-20T18:58:17.584 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.2599999999999998e-07 2024-09-20T18:58:17.584 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.584 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.584 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:58:17.584 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:34.540657+0000", 2024-09-20T18:58:17.585 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2189999999999998e-06 2024-09-20T18:58:17.585 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.585 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.585 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:58:17.585 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:34.540662+0000", 2024-09-20T18:58:17.585 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.2809999999999999e-06 2024-09-20T18:58:17.585 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.585 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.585 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:58:17.585 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:34.540667+0000", 2024-09-20T18:58:17.585 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.4950000000000002e-06 2024-09-20T18:58:17.585 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.585 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.585 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:58:17.585 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:34.540719+0000", 2024-09-20T18:58:17.586 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.2549999999999997e-05 2024-09-20T18:58:17.586 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.586 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.586 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:58:17.586 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:34.540821+0000", 2024-09-20T18:58:17.586 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000101835 2024-09-20T18:58:17.586 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.586 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.586 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:58:17.586 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:34.540920+0000", 2024-09-20T18:58:17.586 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.8510000000000004e-05 2024-09-20T18:58:17.586 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.586 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.587 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:17.587 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:34.547932+0000", 2024-09-20T18:58:17.587 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0070126980000000004 2024-09-20T18:58:17.587 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.587 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.587 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:58:17.587 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:34.547949+0000", 2024-09-20T18:58:17.587 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6708e-05 2024-09-20T18:58:17.587 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.587 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.587 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:17.587 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:34.679800+0000", 2024-09-20T18:58:17.587 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13185100799999999 2024-09-20T18:58:17.587 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.588 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.588 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:58:17.588 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:34.679861+0000", 2024-09-20T18:58:17.588 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.1344000000000001e-05 2024-09-20T18:58:17.588 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.588 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.588 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:58:17.588 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:34.679911+0000", 2024-09-20T18:58:17.588 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.9283000000000002e-05 2024-09-20T18:58:17.588 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:58:17.588 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:58:17.588 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:58:17.588 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.588 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.589 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.24955.0:136161 8.8 8:1b17fcea:::benchmark_data_smithi083_97422_object20070:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e206)", 2024-09-20T18:58:17.589 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:56:34.542682+0000", 2024-09-20T18:58:17.589 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 102.968169571, 2024-09-20T18:58:17.589 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.14475237399999999, 2024-09-20T18:58:17.589 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:58:17.589 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:58:17.589 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:17.589 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:58:17.589 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.24955", 2024-09-20T18:58:17.589 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:58:17.589 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 136161 2024-09-20T18:58:17.589 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.589 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:58:17.589 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.590 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:58:17.590 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:34.542682+0000", 2024-09-20T18:58:17.590 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:58:17.590 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.590 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.590 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:58:17.590 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:34.542682+0000", 2024-09-20T18:58:17.590 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:58:17.590 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.590 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.590 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:58:17.590 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:34.542684+0000", 2024-09-20T18:58:17.590 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9920000000000002e-06 2024-09-20T18:58:17.590 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.591 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.591 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:58:17.591 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:34.542691+0000", 2024-09-20T18:58:17.591 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.2359999999999998e-06 2024-09-20T18:58:17.591 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.591 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.591 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:58:17.591 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:34.542703+0000", 2024-09-20T18:58:17.591 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1868999999999999e-05 2024-09-20T18:58:17.591 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.591 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.591 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:58:17.591 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:34.542712+0000", 2024-09-20T18:58:17.592 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.0440000000000001e-06 2024-09-20T18:58:17.592 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.592 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.592 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:58:17.592 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:34.545239+0000", 2024-09-20T18:58:17.592 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0025272929999999999 2024-09-20T18:58:17.592 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.592 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.592 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:58:17.592 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:34.545321+0000", 2024-09-20T18:58:17.592 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.1273999999999995e-05 2024-09-20T18:58:17.592 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.592 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.592 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:58:17.592 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:34.545396+0000", 2024-09-20T18:58:17.593 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5677000000000001e-05 2024-09-20T18:58:17.593 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.593 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.593 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:17.594 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:58:17.595 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:34.548400+0000", 2024-09-20T18:58:17.595 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0030033659999999999 2024-09-20T18:58:17.595 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.595 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.595 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:17.595 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:34.548494+0000", 2024-09-20T18:58:17.595 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.4623999999999994e-05 2024-09-20T18:58:17.595 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.595 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.595 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:17.595 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:34.687363+0000", 2024-09-20T18:58:17.595 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13886921899999999 2024-09-20T18:58:17.595 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.596 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.596 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:58:17.596 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:34.687403+0000", 2024-09-20T18:58:17.596 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9561999999999998e-05 2024-09-20T18:58:17.596 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.596 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.596 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:58:17.596 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:34.687434+0000", 2024-09-20T18:58:17.596 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.1217999999999998e-05 2024-09-20T18:58:17.596 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:58:17.596 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:58:17.596 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:58:17.596 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.597 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.597 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.24955.0:181696 8.8 8:1f4c5b80:::benchmark_data_smithi083_97422_object65605:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e223)", 2024-09-20T18:58:17.597 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:57:26.223182+0000", 2024-09-20T18:58:17.597 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 51.287668928999999, 2024-09-20T18:58:17.597 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13381210499999999, 2024-09-20T18:58:17.597 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:58:17.597 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:58:17.597 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:17.597 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:58:17.597 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.24955", 2024-09-20T18:58:17.597 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:58:17.597 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 181696 2024-09-20T18:58:17.597 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.598 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:58:17.598 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.598 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:58:17.598 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:26.223182+0000", 2024-09-20T18:58:17.598 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:58:17.598 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.598 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.598 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:58:17.598 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:26.223182+0000", 2024-09-20T18:58:17.598 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:58:17.598 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.598 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.598 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:58:17.598 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:26.223184+0000", 2024-09-20T18:58:17.599 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.438e-06 2024-09-20T18:58:17.599 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.599 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.599 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:58:17.599 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:26.223188+0000", 2024-09-20T18:58:17.599 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0829999999999997e-06 2024-09-20T18:58:17.599 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.599 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.599 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:58:17.599 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:26.223198+0000", 2024-09-20T18:58:17.599 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.7550000000000006e-06 2024-09-20T18:58:17.599 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.599 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.599 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:58:17.600 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:26.223205+0000", 2024-09-20T18:58:17.600 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.0430000000000002e-06 2024-09-20T18:58:17.600 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.600 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.600 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:58:17.600 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:26.223261+0000", 2024-09-20T18:58:17.600 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6437000000000002e-05 2024-09-20T18:58:17.600 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.600 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.600 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:58:17.600 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:26.223372+0000", 2024-09-20T18:58:17.600 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000111191 2024-09-20T18:58:17.600 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.600 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.601 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:58:17.601 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:26.223485+0000", 2024-09-20T18:58:17.601 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000112083 2024-09-20T18:58:17.601 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.601 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.601 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:58:17.601 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:26.225558+0000", 2024-09-20T18:58:17.601 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0020733790000000002 2024-09-20T18:58:17.601 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.601 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.601 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:17.601 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:26.225674+0000", 2024-09-20T18:58:17.601 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011594 2024-09-20T18:58:17.601 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.602 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.602 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:17.602 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:26.356929+0000", 2024-09-20T18:58:17.602 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.131255338 2024-09-20T18:58:17.602 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.602 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.602 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:58:17.602 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:26.356965+0000", 2024-09-20T18:58:17.602 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6093000000000002e-05 2024-09-20T18:58:17.602 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.602 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.602 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:58:17.602 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:26.356995+0000", 2024-09-20T18:58:17.602 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9325000000000001e-05 2024-09-20T18:58:17.603 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:58:17.603 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:58:17.603 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:58:17.603 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.603 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.603 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.24955.0:184217 8.9 8:925da0f0:::benchmark_data_smithi083_97422_object68126:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e223)", 2024-09-20T18:58:17.603 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:57:28.697597+0000", 2024-09-20T18:58:17.603 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 48.813254309999998, 2024-09-20T18:58:17.603 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.105086083, 2024-09-20T18:58:17.603 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:58:17.603 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:58:17.603 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:17.603 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:58:17.603 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.24955", 2024-09-20T18:58:17.604 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:58:17.604 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 184217 2024-09-20T18:58:17.604 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.604 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:58:17.604 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.604 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:58:17.604 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:28.697597+0000", 2024-09-20T18:58:17.604 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:58:17.604 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.604 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.604 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:58:17.604 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:28.697597+0000", 2024-09-20T18:58:17.604 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:58:17.605 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.605 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.605 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:58:17.605 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:28.697598+0000", 2024-09-20T18:58:17.605 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.311e-06 2024-09-20T18:58:17.605 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.605 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.605 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:58:17.605 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:28.697603+0000", 2024-09-20T18:58:17.605 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.233e-06 2024-09-20T18:58:17.605 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.605 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.605 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:58:17.605 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:28.697614+0000", 2024-09-20T18:58:17.606 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1443e-05 2024-09-20T18:58:17.606 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.606 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.606 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:58:17.606 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:28.697621+0000", 2024-09-20T18:58:17.606 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.7630000000000003e-06 2024-09-20T18:58:17.606 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.606 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.606 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:58:17.606 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:28.697678+0000", 2024-09-20T18:58:17.606 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.7593000000000002e-05 2024-09-20T18:58:17.606 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.606 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.606 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:58:17.607 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:28.697790+0000", 2024-09-20T18:58:17.607 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011181299999999999 2024-09-20T18:58:17.607 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.607 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.607 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,3", 2024-09-20T18:58:17.607 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:28.697904+0000", 2024-09-20T18:58:17.607 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011382099999999999 2024-09-20T18:58:17.607 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.607 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.607 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:58:17.607 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:28.700165+0000", 2024-09-20T18:58:17.607 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.002260494 2024-09-20T18:58:17.607 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.607 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.607 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:17.607 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:28.701693+0000", 2024-09-20T18:58:17.608 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001528424 2024-09-20T18:58:17.608 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.608 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.608 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:17.608 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:28.802588+0000", 2024-09-20T18:58:17.608 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100895122 2024-09-20T18:58:17.608 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.608 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.608 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:58:17.608 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:28.802641+0000", 2024-09-20T18:58:17.608 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.2609999999999999e-05 2024-09-20T18:58:17.608 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.608 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.608 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:58:17.608 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:28.802683+0000", 2024-09-20T18:58:17.609 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.2456000000000001e-05 2024-09-20T18:58:17.609 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:58:17.609 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:58:17.609 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:58:17.609 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.609 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.609 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.24955.0:216997 8.8 8:15211c0b:::benchmark_data_smithi083_97422_object100906:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-20T18:58:17.609 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:58:00.356580+0000", 2024-09-20T18:58:17.609 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 17.154271172000001, 2024-09-20T18:58:17.609 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.15243809699999999, 2024-09-20T18:58:17.609 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:58:17.609 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:58:17.609 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:17.609 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:58:17.609 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.24955", 2024-09-20T18:58:17.609 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:58:17.610 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 216997 2024-09-20T18:58:17.610 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.610 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:58:17.610 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.610 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:58:17.610 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.356580+0000", 2024-09-20T18:58:17.610 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:58:17.610 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.610 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.610 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:58:17.610 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.356580+0000", 2024-09-20T18:58:17.610 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:58:17.610 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.611 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.611 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:58:17.611 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.356581+0000", 2024-09-20T18:58:17.611 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.5600000000000004e-07 2024-09-20T18:58:17.611 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.611 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.611 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:58:17.611 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.356585+0000", 2024-09-20T18:58:17.611 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6830000000000001e-06 2024-09-20T18:58:17.611 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.611 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.611 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:58:17.611 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.356592+0000", 2024-09-20T18:58:17.611 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.2989999999999999e-06 2024-09-20T18:58:17.612 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.612 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.612 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:58:17.612 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.356596+0000", 2024-09-20T18:58:17.612 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.2200000000000003e-06 2024-09-20T18:58:17.612 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.612 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.612 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:58:17.612 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.356635+0000", 2024-09-20T18:58:17.612 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.8488000000000001e-05 2024-09-20T18:58:17.612 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.612 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.612 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:58:17.613 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.356714+0000", 2024-09-20T18:58:17.613 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.9194000000000001e-05 2024-09-20T18:58:17.613 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.613 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.613 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:58:17.613 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.356788+0000", 2024-09-20T18:58:17.613 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.3506999999999999e-05 2024-09-20T18:58:17.613 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.613 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.613 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:58:17.613 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.361427+0000", 2024-09-20T18:58:17.613 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0046395230000000004 2024-09-20T18:58:17.613 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.613 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.614 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:17.614 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.361478+0000", 2024-09-20T18:58:17.614 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.1079999999999999e-05 2024-09-20T18:58:17.614 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.614 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.614 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:17.614 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.508937+0000", 2024-09-20T18:58:17.614 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.147458535 2024-09-20T18:58:17.614 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.614 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.614 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:58:17.614 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.508970+0000", 2024-09-20T18:58:17.614 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.3633999999999997e-05 2024-09-20T18:58:17.614 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.615 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.615 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:58:17.615 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.509018+0000", 2024-09-20T18:58:17.615 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8078000000000003e-05 2024-09-20T18:58:17.615 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:58:17.615 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:58:17.615 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:58:17.615 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.615 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.615 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.24955.0:216998 8.9 8:937dd68f:::benchmark_data_smithi083_97422_object100907:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-20T18:58:17.615 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:58:00.357004+0000", 2024-09-20T18:58:17.615 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 17.153847273, 2024-09-20T18:58:17.615 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.17862109500000001, 2024-09-20T18:58:17.616 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:58:17.616 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:58:17.616 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:17.616 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:58:17.616 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.24955", 2024-09-20T18:58:17.616 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:58:17.616 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 216998 2024-09-20T18:58:17.616 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.616 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:58:17.616 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.616 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:58:17.616 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.357004+0000", 2024-09-20T18:58:17.616 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:58:17.616 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.617 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.617 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:58:17.617 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.357004+0000", 2024-09-20T18:58:17.617 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:58:17.617 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.617 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.617 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:58:17.617 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.357005+0000", 2024-09-20T18:58:17.617 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.3699999999999999e-07 2024-09-20T18:58:17.617 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.617 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.617 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:58:17.617 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.357012+0000", 2024-09-20T18:58:17.618 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.8499999999999996e-06 2024-09-20T18:58:17.618 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.618 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.618 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:58:17.618 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.357022+0000", 2024-09-20T18:58:17.618 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0003000000000001e-05 2024-09-20T18:58:17.618 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.618 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.618 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:58:17.618 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.357027+0000", 2024-09-20T18:58:17.618 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0719999999999999e-06 2024-09-20T18:58:17.618 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.618 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.619 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:58:17.619 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.357083+0000", 2024-09-20T18:58:17.619 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5696999999999999e-05 2024-09-20T18:58:17.619 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.619 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.619 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:58:17.619 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.357154+0000", 2024-09-20T18:58:17.619 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.0908000000000006e-05 2024-09-20T18:58:17.619 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.619 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.619 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,3", 2024-09-20T18:58:17.619 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.357220+0000", 2024-09-20T18:58:17.619 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.69e-05 2024-09-20T18:58:17.619 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.620 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.620 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:58:17.620 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.359367+0000", 2024-09-20T18:58:17.620 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0021462349999999998 2024-09-20T18:58:17.620 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.620 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.620 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:17.620 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.506531+0000", 2024-09-20T18:58:17.620 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.14716431599999999 2024-09-20T18:58:17.620 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.620 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.620 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:17.620 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.535536+0000", 2024-09-20T18:58:17.620 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.029005388999999999 2024-09-20T18:58:17.621 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.621 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.621 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:58:17.621 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.535586+0000", 2024-09-20T18:58:17.621 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.9508000000000001e-05 2024-09-20T18:58:17.621 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.621 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.621 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:58:17.621 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.535625+0000", 2024-09-20T18:58:17.621 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9379999999999999e-05 2024-09-20T18:58:17.621 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:58:17.621 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:58:17.621 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:58:17.621 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.622 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.622 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.24955.0:216999 8.8 8:175ad6b7:::benchmark_data_smithi083_97422_object100908:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-20T18:58:17.622 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:58:00.357483+0000", 2024-09-20T18:58:17.622 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 17.153368267000001, 2024-09-20T18:58:17.622 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.15187231700000001, 2024-09-20T18:58:17.622 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:58:17.622 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:58:17.622 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:17.622 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:58:17.622 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.24955", 2024-09-20T18:58:17.622 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:58:17.622 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 216999 2024-09-20T18:58:17.622 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.623 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:58:17.623 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.623 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:58:17.623 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.357483+0000", 2024-09-20T18:58:17.623 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:58:17.623 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.623 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.623 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:58:17.623 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.357483+0000", 2024-09-20T18:58:17.623 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:58:17.623 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.623 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.623 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:58:17.623 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.357484+0000", 2024-09-20T18:58:17.624 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.7599999999999996e-07 2024-09-20T18:58:17.624 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.624 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.624 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:58:17.624 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.357488+0000", 2024-09-20T18:58:17.624 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.2799999999999997e-06 2024-09-20T18:58:17.624 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.624 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.624 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:58:17.624 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.357498+0000", 2024-09-20T18:58:17.624 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0135e-05 2024-09-20T18:58:17.624 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.624 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.624 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:58:17.625 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.357505+0000", 2024-09-20T18:58:17.625 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2179999999999996e-06 2024-09-20T18:58:17.625 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.625 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.625 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:58:17.625 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.358843+0000", 2024-09-20T18:58:17.625 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0013381370000000001 2024-09-20T18:58:17.625 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.625 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.625 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:58:17.625 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.358939+0000", 2024-09-20T18:58:17.625 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.6062000000000003e-05 2024-09-20T18:58:17.625 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.625 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.626 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:58:17.626 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.359038+0000", 2024-09-20T18:58:17.626 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.8949000000000004e-05 2024-09-20T18:58:17.626 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.626 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.626 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:58:17.626 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.361563+0000", 2024-09-20T18:58:17.626 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0025256789999999999 2024-09-20T18:58:17.626 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.626 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.626 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:17.626 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.361647+0000", 2024-09-20T18:58:17.626 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.3138000000000003e-05 2024-09-20T18:58:17.627 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.627 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.627 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:17.627 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.509287+0000", 2024-09-20T18:58:17.627 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.14763997400000001 2024-09-20T18:58:17.627 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.627 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.627 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:58:17.627 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.509323+0000", 2024-09-20T18:58:17.627 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6279999999999998e-05 2024-09-20T18:58:17.627 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.627 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.627 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:58:17.628 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.509355+0000", 2024-09-20T18:58:17.628 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.2688999999999998e-05 2024-09-20T18:58:17.628 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:58:17.628 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:58:17.628 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:58:17.628 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.628 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.628 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.24955.0:217250 8.8 8:1dc5b14b:::benchmark_data_smithi083_97422_object101159:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-20T18:58:17.628 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:58:00.735048+0000", 2024-09-20T18:58:17.628 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 16.775803394, 2024-09-20T18:58:17.628 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.170735638, 2024-09-20T18:58:17.628 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:58:17.628 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:58:17.628 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:17.628 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:58:17.629 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.24955", 2024-09-20T18:58:17.629 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:58:17.629 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 217250 2024-09-20T18:58:17.629 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.629 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:58:17.629 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.629 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:58:17.629 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.735048+0000", 2024-09-20T18:58:17.629 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:58:17.629 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.629 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.629 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:58:17.629 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.735048+0000", 2024-09-20T18:58:17.630 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:58:17.630 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.630 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.630 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:58:17.630 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.735049+0000", 2024-09-20T18:58:17.630 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.8199999999999999e-07 2024-09-20T18:58:17.630 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.630 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.630 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:58:17.630 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.735052+0000", 2024-09-20T18:58:17.630 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.1769999999999998e-06 2024-09-20T18:58:17.630 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.630 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.630 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:58:17.631 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.735059+0000", 2024-09-20T18:58:17.631 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.3880000000000002e-06 2024-09-20T18:58:17.631 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.631 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.631 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:58:17.631 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.735065+0000", 2024-09-20T18:58:17.631 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.356e-06 2024-09-20T18:58:17.631 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.631 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.631 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:58:17.631 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.735101+0000", 2024-09-20T18:58:17.631 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6295999999999999e-05 2024-09-20T18:58:17.631 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.632 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.632 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:58:17.632 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.735176+0000", 2024-09-20T18:58:17.632 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5452999999999997e-05 2024-09-20T18:58:17.632 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.632 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.632 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:58:17.632 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.735245+0000", 2024-09-20T18:58:17.632 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.8423000000000005e-05 2024-09-20T18:58:17.632 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.632 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.632 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:58:17.632 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.739097+0000", 2024-09-20T18:58:17.632 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0038518889999999998 2024-09-20T18:58:17.633 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.633 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.633 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:17.633 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.741853+0000", 2024-09-20T18:58:17.633 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0027559759999999998 2024-09-20T18:58:17.633 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.633 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.633 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:17.633 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.905687+0000", 2024-09-20T18:58:17.633 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.16383436500000001 2024-09-20T18:58:17.633 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.633 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.633 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:58:17.633 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.905738+0000", 2024-09-20T18:58:17.634 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.1344000000000002e-05 2024-09-20T18:58:17.634 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.634 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.634 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:58:17.634 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.905784+0000", 2024-09-20T18:58:17.634 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.5289e-05 2024-09-20T18:58:17.634 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:58:17.634 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:58:17.634 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:58:17.634 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.634 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.634 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.24955.0:217256 8.9 8:91d912b3:::benchmark_data_smithi083_97422_object101165:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-20T18:58:17.634 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:58:00.744869+0000", 2024-09-20T18:58:17.635 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 16.765982404999999, 2024-09-20T18:58:17.635 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.165686044, 2024-09-20T18:58:17.635 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:58:17.635 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:58:17.635 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:17.635 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:58:17.635 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.24955", 2024-09-20T18:58:17.635 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:58:17.635 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 217256 2024-09-20T18:58:17.635 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.635 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:58:17.635 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.635 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:58:17.635 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.744869+0000", 2024-09-20T18:58:17.636 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:58:17.636 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.636 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.636 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:58:17.636 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.744869+0000", 2024-09-20T18:58:17.636 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:58:17.636 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.636 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.636 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:58:17.636 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.744870+0000", 2024-09-20T18:58:17.636 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.11e-06 2024-09-20T18:58:17.636 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.636 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.636 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:58:17.637 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.744874+0000", 2024-09-20T18:58:17.637 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3340000000000003e-06 2024-09-20T18:58:17.637 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.637 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.637 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:58:17.637 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.744884+0000", 2024-09-20T18:58:17.637 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.9380000000000004e-06 2024-09-20T18:58:17.637 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.637 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.637 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:58:17.637 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.744890+0000", 2024-09-20T18:58:17.637 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.1769999999999997e-06 2024-09-20T18:58:17.637 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.638 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.638 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:58:17.638 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.744930+0000", 2024-09-20T18:58:17.638 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0553999999999998e-05 2024-09-20T18:58:17.638 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.638 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.638 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:58:17.638 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.745009+0000", 2024-09-20T18:58:17.638 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.9320000000000006e-05 2024-09-20T18:58:17.638 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.638 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.638 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,3", 2024-09-20T18:58:17.638 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.745092+0000", 2024-09-20T18:58:17.639 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.2943999999999997e-05 2024-09-20T18:58:17.639 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.639 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.639 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:58:17.639 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.747268+0000", 2024-09-20T18:58:17.639 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0021758060000000002 2024-09-20T18:58:17.639 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.639 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.639 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:17.639 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.905711+0000", 2024-09-20T18:58:17.639 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.15844275999999999 2024-09-20T18:58:17.639 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.639 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.639 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:17.639 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.910495+0000", 2024-09-20T18:58:17.640 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0047838530000000002 2024-09-20T18:58:17.640 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.640 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.640 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:58:17.640 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.910528+0000", 2024-09-20T18:58:17.640 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.3102000000000003e-05 2024-09-20T18:58:17.640 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.640 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.640 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:58:17.640 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.910555+0000", 2024-09-20T18:58:17.640 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.7146000000000001e-05 2024-09-20T18:58:17.640 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:58:17.640 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:58:17.640 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:58:17.641 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.641 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.641 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.24955.0:217257 8.8 8:1a8671bd:::benchmark_data_smithi083_97422_object101166:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-20T18:58:17.641 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:58:00.746932+0000", 2024-09-20T18:58:17.641 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 16.763919224999999, 2024-09-20T18:58:17.641 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.15894323599999999, 2024-09-20T18:58:17.641 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:58:17.641 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:58:17.641 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:17.641 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:58:17.641 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.24955", 2024-09-20T18:58:17.641 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:58:17.641 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 217257 2024-09-20T18:58:17.641 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.641 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:58:17.642 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.642 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:58:17.642 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.746932+0000", 2024-09-20T18:58:17.642 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:58:17.642 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.642 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.642 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:58:17.642 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.746932+0000", 2024-09-20T18:58:17.642 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:58:17.642 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.642 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.642 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:58:17.642 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.746933+0000", 2024-09-20T18:58:17.643 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.1800000000000004e-07 2024-09-20T18:58:17.643 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.643 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.643 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:58:17.643 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.746937+0000", 2024-09-20T18:58:17.643 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0690000000000003e-06 2024-09-20T18:58:17.643 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.643 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.643 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:58:17.643 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.746946+0000", 2024-09-20T18:58:17.643 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.0149999999999992e-06 2024-09-20T18:58:17.643 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.643 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.643 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:58:17.644 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.746951+0000", 2024-09-20T18:58:17.644 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.5070000000000002e-06 2024-09-20T18:58:17.644 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.644 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.644 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:58:17.644 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.746999+0000", 2024-09-20T18:58:17.644 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8591999999999998e-05 2024-09-20T18:58:17.644 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.644 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.644 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:58:17.644 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.747090+0000", 2024-09-20T18:58:17.644 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.0733e-05 2024-09-20T18:58:17.644 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.644 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.645 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,7", 2024-09-20T18:58:17.645 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.747189+0000", 2024-09-20T18:58:17.645 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.8814000000000006e-05 2024-09-20T18:58:17.645 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.645 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.645 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:58:17.645 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.749423+0000", 2024-09-20T18:58:17.645 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0022339209999999998 2024-09-20T18:58:17.645 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.645 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.645 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:17.645 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.749564+0000", 2024-09-20T18:58:17.645 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00014107400000000001 2024-09-20T18:58:17.645 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.646 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.646 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:17.646 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.905817+0000", 2024-09-20T18:58:17.646 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.15625285799999999 2024-09-20T18:58:17.646 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.646 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.646 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:58:17.646 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.905854+0000", 2024-09-20T18:58:17.646 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.7738999999999999e-05 2024-09-20T18:58:17.646 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.646 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.646 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:58:17.646 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:00.905875+0000", 2024-09-20T18:58:17.646 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0996e-05 2024-09-20T18:58:17.647 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:58:17.647 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:58:17.647 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:58:17.647 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.647 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.647 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.24955.0:217587 8.9 8:90ca5dcf:::benchmark_data_smithi083_97422_object101496:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-20T18:58:17.647 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:58:01.173763+0000", 2024-09-20T18:58:17.647 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 16.337088394999999, 2024-09-20T18:58:17.647 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.161994314, 2024-09-20T18:58:17.647 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:58:17.647 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:58:17.647 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:17.648 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:58:17.648 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.24955", 2024-09-20T18:58:17.648 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:58:17.648 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 217587 2024-09-20T18:58:17.648 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.648 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:58:17.648 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.648 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:58:17.648 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:01.173763+0000", 2024-09-20T18:58:17.648 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:58:17.648 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.648 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.648 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:58:17.648 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:01.173763+0000", 2024-09-20T18:58:17.648 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:58:17.649 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.649 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.649 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:58:17.649 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:01.173765+0000", 2024-09-20T18:58:17.649 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.81e-06 2024-09-20T18:58:17.649 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.649 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.649 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:58:17.649 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:01.173770+0000", 2024-09-20T18:58:17.649 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0930000000000002e-06 2024-09-20T18:58:17.649 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.649 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.649 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:58:17.649 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:01.173782+0000", 2024-09-20T18:58:17.649 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1953000000000001e-05 2024-09-20T18:58:17.650 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.650 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.650 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:58:17.650 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:01.173789+0000", 2024-09-20T18:58:17.650 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.2599999999999999e-06 2024-09-20T18:58:17.650 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.650 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.650 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:58:17.650 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:01.173840+0000", 2024-09-20T18:58:17.650 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0875999999999999e-05 2024-09-20T18:58:17.650 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.650 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.650 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:58:17.651 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:01.173946+0000", 2024-09-20T18:58:17.651 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00010552000000000001 2024-09-20T18:58:17.651 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.651 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.651 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,3", 2024-09-20T18:58:17.651 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:01.174030+0000", 2024-09-20T18:58:17.651 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.4599999999999996e-05 2024-09-20T18:58:17.651 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.651 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.651 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:58:17.651 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:01.176071+0000", 2024-09-20T18:58:17.651 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0020405319999999998 2024-09-20T18:58:17.651 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.651 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.651 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:17.652 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:01.176573+0000", 2024-09-20T18:58:17.652 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000502072 2024-09-20T18:58:17.652 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.652 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.652 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:17.652 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:01.335703+0000", 2024-09-20T18:58:17.652 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.15913028800000001 2024-09-20T18:58:17.652 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.652 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.652 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:58:17.652 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:01.335733+0000", 2024-09-20T18:58:17.652 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.0321e-05 2024-09-20T18:58:17.652 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:58:17.652 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:58:17.653 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:58:17.653 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:01.335757+0000", 2024-09-20T18:58:17.653 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3989e-05 2024-09-20T18:58:17.653 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:58:17.653 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:58:17.653 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:58:17.653 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:58:17.653 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:58:17.653 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T18:58:17.674 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:17.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:17.675 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:17.776 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:17.796 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:17.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:17.797 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:17.800 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:58:17.800 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"1ed5a50a-7780-11ef-baf5-efdc52797490","health":{"status":"HEALTH_WARN","checks":{"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"noscrub,nodeep-scrub flag(s) set","count":20},"muted":false},"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 1602/2592 objects degraded (61.806%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":759,"monmap":{"epoch":3,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":237,"num_osds":8,"num_up_osds":8,"osd_up_since":1726858174,"num_in_osds":7,"osd_in_since":1726858527,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":147},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":148,"num_pools":7,"num_objects":864,"data_bytes":44177947,"bytes_used":2291920896,"bytes_avail":669673328640,"bytes_total":671965249536,"degraded_objects":1602,"degraded_total":2592,"degraded_ratio":0.61805555555555558,"recovering_objects_per_sec":26,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":168,"num_bytes_recovered":0,"num_keys_recovered":0,"read_bytes_sec":1642,"write_bytes_sec":0,"read_op_per_sec":1,"write_op_per_sec":1438},"fsmap":{"epoch":1,"btime":"2024-09-20T18:44:10:399616+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"services":{"dashboard":"https://172.21.15.83:8443/","prometheus":"http://172.21.15.83:9283/"}},"servicemap":{"epoch":14,"modified":"2024-09-20T18:51:16.768808+0000","services":{"rgw":{"daemons":{"summary":"","14460":{"start_epoch":12,"start_stamp":"2024-09-20T18:49:53.053671+0000","gid":14460,"addr":"172.21.15.83:0/1555339426","metadata":{"arch":"x86_64","ceph_release":"squid","ceph_version":"ceph version 19.3.0-5099-g88deadb8 (88deadb8082f4a33832e228ccefc59cedba158a5) squid (dev)","ceph_version_short":"19.3.0-5099-g88deadb8","container_hostname":"smithi083","container_image":"quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2","cpu":"Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz","distro":"centos","distro_description":"CentOS Stream 9","distro_version":"9","frontend_config#0":"beast port=80","frontend_type#0":"beast","hostname":"smithi083","id":"foo.a","kernel_description":"#1 SMP PREEMPT_DYNAMIC Thu Sep 12 15:49:37 UTC 2024","kernel_version":"5.14.0-508.el9.x86_64","mem_swap_kb":"0","mem_total_kb":"32487292","num_handles":"1","os":"Linux","pid":"2","realm_id":"","realm_name":"","zone_id":"41426759-9535-4245-bcfe-8d158f74a134","zone_name":"default","zonegroup_id":"fb472636-52d3-4587-9964-925ef38835b6","zonegroup_name":"default"},"task_status":{}}}}}},"progress_events":{}} 2024-09-20T18:58:17.898 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:17.920 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:17.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:17.921 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:18.022 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:18.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:18.044 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:18.044 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:18.145 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:18.166 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:18.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:18.167 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:18.201 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T18:58:18.268 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:18.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:18.291 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:18.292 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:18.392 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:18.417 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:18.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:18.418 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:18.478 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:18 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:58:18.478 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:18 smithi083 ceph-mon[29421]: pgmap v407: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 42 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1.44k op/s; 1602/2592 objects degraded (61.806%); 0 B/s, 26 objects/s recovering 2024-09-20T18:58:18.478 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:18 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2943771309' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-09-20T18:58:18.478 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:18 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:58:18.478 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:18 smithi083 ceph-mon[36260]: pgmap v407: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 42 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1.44k op/s; 1602/2592 objects degraded (61.806%); 0 B/s, 26 objects/s recovering 2024-09-20T18:58:18.478 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:18 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2943771309' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-09-20T18:58:18.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:18 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:58:18.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:18 smithi142 ceph-mon[32508]: pgmap v407: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 42 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1.44k op/s; 1602/2592 objects degraded (61.806%); 0 B/s, 26 objects/s recovering 2024-09-20T18:58:18.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:18 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2943771309' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-09-20T18:58:18.518 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:18.539 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:18.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:18.540 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:18.590 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-20T18:58:18.641 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:18.662 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:18.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:18.663 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:18.764 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:18.786 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:18.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:18.787 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:18.835 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:58:18.887 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:18.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:18.909 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:18.910 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:19.010 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:19.029 INFO:tasks.cephadm.ceph_manager.ceph:making progress, resetting timeout 2024-09-20T18:58:19.029 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T18:58:19.033 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:19.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:19.034 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:19.077 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T18:58:19.106 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:58:19.135 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:19.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:19.156 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:19.157 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:19.257 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:19.282 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:19.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:19.283 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:19.383 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:19.409 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:19.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:19.410 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:19.510 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:58:19.532 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:19.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:19.534 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:58:19.561 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T18:58:19.562 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-09-20T18:58:19.562 INFO:teuthology.orchestra.run.smithi142.stdout: "num_ops": 0 2024-09-20T18:58:19.562 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T18:58:19.633 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:58:19.634 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:58:19.635 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:19.635 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:58:19.636 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:58:19.659 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:19.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:19.660 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:19.661 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:58:19.661 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:58:19.675 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:58:19.680 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:58:19.680 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:58:19.680 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:58:19.706 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:58:19.707 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:58:19.726 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:58:19.727 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:58:19.740 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_1 2024-09-20T18:58:19.740 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd pool rm unique_pool_1 unique_pool_1 --yes-i-really-really-mean-it 2024-09-20T18:58:19.741 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:58:19.741 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:58:19.757 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:58:19.757 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:58:19.761 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:19.782 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:19.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:19.783 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:19.884 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:19.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:19.907 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:19.907 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:20.008 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:20.029 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:20.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:20.031 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:20.131 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:20.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:20.153 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:20.154 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:20.179 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_blocked_ops 2024-09-20T18:58:20.255 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:20.276 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:20.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:20.277 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:20.377 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:20.399 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:20.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:20.400 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:20.410 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:20 smithi083 ceph-mon[29421]: pgmap v408: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 583 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 1.12k op/s; 974/597 objects degraded (163.149%); 0 B/s, 99 objects/s recovering 2024-09-20T18:58:20.410 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:20 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3083381239' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:58:20.411 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:20 smithi083 ceph-mon[36260]: pgmap v408: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 583 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 1.12k op/s; 974/597 objects degraded (163.149%); 0 B/s, 99 objects/s recovering 2024-09-20T18:58:20.411 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:20 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3083381239' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:58:20.429 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:58:20.501 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:20.522 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:20.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:20.523 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:20.550 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:20 smithi142 ceph-mon[32508]: pgmap v408: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 583 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 1.12k op/s; 974/597 objects degraded (163.149%); 0 B/s, 99 objects/s recovering 2024-09-20T18:58:20.551 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:20 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3083381239' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:58:20.624 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:58:20.646 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:20.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:20.647 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:58:20.750 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:20.772 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:20.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:20.773 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:20.874 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:20.897 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:20.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:20.898 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:20.974 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T18:58:20.974 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:58:20.998 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:21.009 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:58:21.010 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:58:20 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:58:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:58:21.015 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-20T18:58:21.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:21.020 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:21.021 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:21.121 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:21.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:21.144 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:21.144 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:21.245 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:21.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:21.268 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:21.268 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:21.368 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:21.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:21.390 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:21.391 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:21.491 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:21.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:21.513 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:21.514 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:21.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:21 smithi083 ceph-mon[29421]: osdmap e238: 8 total, 8 up, 7 in 2024-09-20T18:58:21.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:21 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 974/597 objects degraded (163.149%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T18:58:21.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:21 smithi083 ceph-mon[29421]: osdmap e239: 8 total, 8 up, 7 in 2024-09-20T18:58:21.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:21 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/70674786' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:58:21.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:21 smithi083 ceph-mon[36260]: osdmap e238: 8 total, 8 up, 7 in 2024-09-20T18:58:21.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:21 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 974/597 objects degraded (163.149%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T18:58:21.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:21 smithi083 ceph-mon[36260]: osdmap e239: 8 total, 8 up, 7 in 2024-09-20T18:58:21.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:21 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/70674786' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:58:21.551 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:21 smithi142 ceph-mon[32508]: osdmap e238: 8 total, 8 up, 7 in 2024-09-20T18:58:21.552 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:21 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 974/597 objects degraded (163.149%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T18:58:21.552 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:21 smithi142 ceph-mon[32508]: osdmap e239: 8 total, 8 up, 7 in 2024-09-20T18:58:21.552 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:21 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/70674786' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:58:21.578 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_historic_ops 2024-09-20T18:58:21.615 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:21.641 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:21.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:21.642 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:21.742 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:21.751 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-20T18:58:21.765 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:58:21.770 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:21.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:21.771 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:21.828 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:58:21.871 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:21.892 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:21.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:21.893 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:21.994 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:22.015 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:22.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:22.016 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:22.116 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:22.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:22.138 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:22.139 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:22.204 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":411,"stamp":"2024-09-20T18:58:20.962931+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":974,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":1355,"num_read_kb":1212,"num_write":232727,"num_write_kb":7431306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27177,"num_bytes_recovered":1676479433,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74667,"ondisk_log_size":74667,"up":444,"acting":443,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":2127712,"kb_used_data":11480,"kb_used_omap":10,"kb_used_meta":2115957,"kb_avail":654088352,"statfs":{"total":671965249536,"available":669786472448,"internally_reserved":0,"allocated":11755520,"data_stored":8218726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11158,"internal_metadata":2166740074},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-280887320,"num_objects":-4287,"num_object_clones":0,"num_object_copies":-12861,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-628,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4287,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4287,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":628,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001718"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":394,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:16.351758+0000","last_change":"2024-09-20T18:55:28.037494+0000","last_active":"2024-09-20T18:58:16.351758+0000","last_peered":"2024-09-20T18:58:16.351758+0000","last_clean":"2024-09-20T18:58:16.351758+0000","last_became_active":"2024-09-20T18:49:45.943201+0000","last_became_peered":"2024-09-20T18:49:45.943201+0000","last_unstale":"2024-09-20T18:58:16.351758+0000","last_undegraded":"2024-09-20T18:58:16.351758+0000","last_fullsized":"2024-09-20T18:58:16.351758+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:28.037449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:28.037449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":444,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-20T18:58:20.729999+0000","last_change":"2024-09-20T18:55:28.838729+0000","last_active":"2024-09-20T18:58:20.729999+0000","last_peered":"2024-09-20T18:58:20.729999+0000","last_clean":"2024-09-20T18:58:20.729999+0000","last_became_active":"2024-09-20T18:55:28.819637+0000","last_became_peered":"2024-09-20T18:55:28.819637+0000","last_unstale":"2024-09-20T18:58:20.729999+0000","last_undegraded":"2024-09-20T18:58:20.729999+0000","last_fullsized":"2024-09-20T18:58:20.729999+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:50.231620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:50.231620+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":394,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:15.731432+0000","last_change":"2024-09-20T18:57:49.016078+0000","last_active":"2024-09-20T18:58:15.731432+0000","last_peered":"2024-09-20T18:58:15.731432+0000","last_clean":"2024-09-20T18:58:15.731432+0000","last_became_active":"2024-09-20T18:49:49.959514+0000","last_became_peered":"2024-09-20T18:49:49.959514+0000","last_unstale":"2024-09-20T18:58:15.731432+0000","last_undegraded":"2024-09-20T18:58:15.731432+0000","last_fullsized":"2024-09-20T18:58:15.731432+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:49.016006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:49.016006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:13.297987+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":394,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-20T18:58:20.730287+0000","last_change":"2024-09-20T18:58:02.084435+0000","last_active":"2024-09-20T18:58:20.730287+0000","last_peered":"2024-09-20T18:58:20.730287+0000","last_clean":"2024-09-20T18:58:20.730287+0000","last_became_active":"2024-09-20T18:49:51.969421+0000","last_became_peered":"2024-09-20T18:49:51.969421+0000","last_unstale":"2024-09-20T18:58:20.730287+0000","last_undegraded":"2024-09-20T18:58:20.730287+0000","last_fullsized":"2024-09-20T18:58:20.730287+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:02.084375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:02.084375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:20.422560+0000","scrub_duration":197,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":389,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:15.731665+0000","last_change":"2024-09-20T18:55:28.822034+0000","last_active":"2024-09-20T18:58:15.731665+0000","last_peered":"2024-09-20T18:58:15.731665+0000","last_clean":"2024-09-20T18:58:15.731665+0000","last_became_active":"2024-09-20T18:55:28.821717+0000","last_became_peered":"2024-09-20T18:55:28.821717+0000","last_unstale":"2024-09-20T18:58:15.731665+0000","last_undegraded":"2024-09-20T18:58:15.731665+0000","last_fullsized":"2024-09-20T18:58:15.731665+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.216188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.216188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:13.051959+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":394,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:15.729807+0000","last_change":"2024-09-20T18:57:22.694362+0000","last_active":"2024-09-20T18:58:15.729807+0000","last_peered":"2024-09-20T18:58:15.729807+0000","last_clean":"2024-09-20T18:58:15.729807+0000","last_became_active":"2024-09-20T18:55:28.824826+0000","last_became_peered":"2024-09-20T18:55:28.824826+0000","last_unstale":"2024-09-20T18:58:15.729807+0000","last_undegraded":"2024-09-20T18:58:15.729807+0000","last_fullsized":"2024-09-20T18:58:15.729807+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:22.694295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:22.694295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:33.473592+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":407,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:15.730376+0000","last_change":"2024-09-20T18:57:13.774827+0000","last_active":"2024-09-20T18:58:15.730376+0000","last_peered":"2024-09-20T18:58:15.730376+0000","last_clean":"2024-09-20T18:58:15.730376+0000","last_became_active":"2024-09-20T18:49:47.952150+0000","last_became_peered":"2024-09-20T18:49:47.952150+0000","last_unstale":"2024-09-20T18:58:15.730376+0000","last_undegraded":"2024-09-20T18:58:15.730376+0000","last_fullsized":"2024-09-20T18:58:15.730376+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:57:13.774305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:13.774305+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:25.245744+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":397,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-20T18:58:20.729989+0000","last_change":"2024-09-20T18:58:06.846086+0000","last_active":"2024-09-20T18:58:20.729989+0000","last_peered":"2024-09-20T18:58:20.729989+0000","last_clean":"2024-09-20T18:58:20.729989+0000","last_became_active":"2024-09-20T18:55:28.820362+0000","last_became_peered":"2024-09-20T18:55:28.820362+0000","last_unstale":"2024-09-20T18:58:20.729989+0000","last_undegraded":"2024-09-20T18:58:20.729989+0000","last_fullsized":"2024-09-20T18:58:20.729989+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:06.846018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:06.846018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:33.625936+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"59'1","reported_seq":426,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:16.442659+0000","last_change":"2024-09-20T18:58:09.472495+0000","last_active":"2024-09-20T18:58:16.442659+0000","last_peered":"2024-09-20T18:58:16.442659+0000","last_clean":"2024-09-20T18:58:16.442659+0000","last_became_active":"2024-09-20T18:55:28.833386+0000","last_became_peered":"2024-09-20T18:55:28.833386+0000","last_unstale":"2024-09-20T18:58:16.442659+0000","last_undegraded":"2024-09-20T18:58:16.442659+0000","last_fullsized":"2024-09-20T18:58:16.442659+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:58:09.472360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:09.472360+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:29.957740+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":410,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-20T18:58:20.730514+0000","last_change":"2024-09-20T18:54:53.241889+0000","last_active":"2024-09-20T18:58:20.730514+0000","last_peered":"2024-09-20T18:58:20.730514+0000","last_clean":"2024-09-20T18:58:20.730514+0000","last_became_active":"2024-09-20T18:49:47.947448+0000","last_became_peered":"2024-09-20T18:49:47.947448+0000","last_unstale":"2024-09-20T18:58:20.730514+0000","last_undegraded":"2024-09-20T18:58:20.730514+0000","last_fullsized":"2024-09-20T18:58:20.730514+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:53.241802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:53.241802+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":385,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:16.442411+0000","last_change":"2024-09-20T18:57:08.539466+0000","last_active":"2024-09-20T18:58:16.442411+0000","last_peered":"2024-09-20T18:58:16.442411+0000","last_clean":"2024-09-20T18:58:16.442411+0000","last_became_active":"2024-09-20T18:49:49.961188+0000","last_became_peered":"2024-09-20T18:49:49.961188+0000","last_unstale":"2024-09-20T18:58:16.442411+0000","last_undegraded":"2024-09-20T18:58:16.442411+0000","last_fullsized":"2024-09-20T18:58:16.442411+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:08.539418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:08.539418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":357,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:16.350660+0000","last_change":"2024-09-20T18:49:51.966957+0000","last_active":"2024-09-20T18:58:16.350660+0000","last_peered":"2024-09-20T18:58:16.350660+0000","last_clean":"2024-09-20T18:58:16.350660+0000","last_became_active":"2024-09-20T18:49:51.966715+0000","last_became_peered":"2024-09-20T18:49:51.966715+0000","last_unstale":"2024-09-20T18:58:16.350660+0000","last_undegraded":"2024-09-20T18:58:16.350660+0000","last_fullsized":"2024-09-20T18:58:16.350660+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":385,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:16.442682+0000","last_change":"2024-09-20T18:55:15.252839+0000","last_active":"2024-09-20T18:58:16.442682+0000","last_peered":"2024-09-20T18:58:16.442682+0000","last_clean":"2024-09-20T18:58:16.442682+0000","last_became_active":"2024-09-20T18:49:45.944703+0000","last_became_peered":"2024-09-20T18:49:45.944703+0000","last_unstale":"2024-09-20T18:58:16.442682+0000","last_undegraded":"2024-09-20T18:58:16.442682+0000","last_fullsized":"2024-09-20T18:58:16.442682+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:15.252782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:15.252782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:15.451958+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":414,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-20T18:58:20.730576+0000","last_change":"2024-09-20T18:51:26.494194+0000","last_active":"2024-09-20T18:58:20.730576+0000","last_peered":"2024-09-20T18:58:20.730576+0000","last_clean":"2024-09-20T18:58:20.730576+0000","last_became_active":"2024-09-20T18:51:23.813399+0000","last_became_peered":"2024-09-20T18:51:23.813399+0000","last_unstale":"2024-09-20T18:58:20.730576+0000","last_undegraded":"2024-09-20T18:58:20.730576+0000","last_fullsized":"2024-09-20T18:58:20.730576+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-20T18:51:26.494118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:26.494118+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:31.887636+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":381,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-20T18:58:20.734754+0000","last_change":"2024-09-20T18:52:38.799542+0000","last_active":"2024-09-20T18:58:20.734754+0000","last_peered":"2024-09-20T18:58:20.734754+0000","last_clean":"2024-09-20T18:58:20.734754+0000","last_became_active":"2024-09-20T18:49:49.959696+0000","last_became_peered":"2024-09-20T18:49:49.959696+0000","last_unstale":"2024-09-20T18:58:20.734754+0000","last_undegraded":"2024-09-20T18:58:20.734754+0000","last_fullsized":"2024-09-20T18:58:20.734754+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:38.799487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:38.799487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":381,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:16.441594+0000","last_change":"2024-09-20T18:57:13.587827+0000","last_active":"2024-09-20T18:58:16.441594+0000","last_peered":"2024-09-20T18:58:16.441594+0000","last_clean":"2024-09-20T18:58:16.441594+0000","last_became_active":"2024-09-20T18:49:51.970663+0000","last_became_peered":"2024-09-20T18:49:51.970663+0000","last_unstale":"2024-09-20T18:58:16.441594+0000","last_undegraded":"2024-09-20T18:58:16.441594+0000","last_fullsized":"2024-09-20T18:58:16.441594+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:13.587774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:13.587774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":394,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-20T18:58:20.730735+0000","last_change":"2024-09-20T18:58:00.906697+0000","last_active":"2024-09-20T18:58:20.730735+0000","last_peered":"2024-09-20T18:58:20.730735+0000","last_clean":"2024-09-20T18:58:20.730735+0000","last_became_active":"2024-09-20T18:55:28.827435+0000","last_became_peered":"2024-09-20T18:55:28.827435+0000","last_unstale":"2024-09-20T18:58:20.730735+0000","last_undegraded":"2024-09-20T18:58:20.730735+0000","last_fullsized":"2024-09-20T18:58:20.730735+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:00.906652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:00.906652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:23.434725+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":385,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:16.351538+0000","last_change":"2024-09-20T18:54:23.984024+0000","last_active":"2024-09-20T18:58:16.351538+0000","last_peered":"2024-09-20T18:58:16.351538+0000","last_clean":"2024-09-20T18:58:16.351538+0000","last_became_active":"2024-09-20T18:49:45.950480+0000","last_became_peered":"2024-09-20T18:49:45.950480+0000","last_unstale":"2024-09-20T18:58:16.351538+0000","last_undegraded":"2024-09-20T18:58:16.351538+0000","last_fullsized":"2024-09-20T18:58:16.351538+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:23.983958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:23.983958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":406,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-20T18:58:13.254950+0000","last_change":"2024-09-20T18:53:26.762207+0000","last_active":"2024-09-20T18:58:13.254950+0000","last_peered":"2024-09-20T18:58:13.254950+0000","last_clean":"2024-09-20T18:58:13.254950+0000","last_became_active":"2024-09-20T18:49:47.950939+0000","last_became_peered":"2024-09-20T18:49:47.950939+0000","last_unstale":"2024-09-20T18:58:13.254950+0000","last_undegraded":"2024-09-20T18:58:13.254950+0000","last_fullsized":"2024-09-20T18:58:13.254950+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:53:26.762131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:26.762131+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":378,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:15.731864+0000","last_change":"2024-09-20T18:57:14.941518+0000","last_active":"2024-09-20T18:58:15.731864+0000","last_peered":"2024-09-20T18:58:15.731864+0000","last_clean":"2024-09-20T18:58:15.731864+0000","last_became_active":"2024-09-20T18:49:49.958518+0000","last_became_peered":"2024-09-20T18:49:49.958518+0000","last_unstale":"2024-09-20T18:58:15.731864+0000","last_undegraded":"2024-09-20T18:58:15.731864+0000","last_fullsized":"2024-09-20T18:58:15.731864+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:14.941458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:14.941458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:37.988570+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":382,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:15.731143+0000","last_change":"2024-09-20T18:55:28.833468+0000","last_active":"2024-09-20T18:58:15.731143+0000","last_peered":"2024-09-20T18:58:15.731143+0000","last_clean":"2024-09-20T18:58:15.731143+0000","last_became_active":"2024-09-20T18:55:28.833133+0000","last_became_peered":"2024-09-20T18:55:28.833133+0000","last_unstale":"2024-09-20T18:58:15.731143+0000","last_undegraded":"2024-09-20T18:58:15.731143+0000","last_fullsized":"2024-09-20T18:58:15.731143+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:54.259174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:54.259174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:30.790542+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"59'1","reported_seq":404,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-20T18:58:20.730546+0000","last_change":"2024-09-20T18:57:53.918080+0000","last_active":"2024-09-20T18:58:20.730546+0000","last_peered":"2024-09-20T18:58:20.730546+0000","last_clean":"2024-09-20T18:58:20.730546+0000","last_became_active":"2024-09-20T18:49:45.940100+0000","last_became_peered":"2024-09-20T18:49:45.940100+0000","last_unstale":"2024-09-20T18:58:20.730546+0000","last_undegraded":"2024-09-20T18:58:20.730546+0000","last_fullsized":"2024-09-20T18:58:20.730546+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:57:53.918022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:53.918022+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:03.242427+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":397,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:15.729232+0000","last_change":"2024-09-20T18:57:09.819080+0000","last_active":"2024-09-20T18:58:15.729232+0000","last_peered":"2024-09-20T18:58:15.729232+0000","last_clean":"2024-09-20T18:58:15.729232+0000","last_became_active":"2024-09-20T18:49:47.950909+0000","last_became_peered":"2024-09-20T18:49:47.950909+0000","last_unstale":"2024-09-20T18:58:15.729232+0000","last_undegraded":"2024-09-20T18:58:15.729232+0000","last_fullsized":"2024-09-20T18:58:15.729232+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:57:09.818999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:09.818999+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:17.216750+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":390,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-20T18:58:20.730722+0000","last_change":"2024-09-20T18:54:44.239273+0000","last_active":"2024-09-20T18:58:20.730722+0000","last_peered":"2024-09-20T18:58:20.730722+0000","last_clean":"2024-09-20T18:58:20.730722+0000","last_became_active":"2024-09-20T18:49:49.959102+0000","last_became_peered":"2024-09-20T18:49:49.959102+0000","last_unstale":"2024-09-20T18:58:20.730722+0000","last_undegraded":"2024-09-20T18:58:20.730722+0000","last_fullsized":"2024-09-20T18:58:20.730722+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:44.239226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.239226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:17.660601+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":378,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-20T18:58:20.734990+0000","last_change":"2024-09-20T18:52:15.826819+0000","last_active":"2024-09-20T18:58:20.734990+0000","last_peered":"2024-09-20T18:58:20.734990+0000","last_clean":"2024-09-20T18:58:20.734990+0000","last_became_active":"2024-09-20T18:49:51.970058+0000","last_became_peered":"2024-09-20T18:49:51.970058+0000","last_unstale":"2024-09-20T18:58:20.734990+0000","last_undegraded":"2024-09-20T18:58:20.734990+0000","last_fullsized":"2024-09-20T18:58:20.734990+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T18:52:15.826638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:15.826638+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":387,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:15.731562+0000","last_change":"2024-09-20T18:57:18.032140+0000","last_active":"2024-09-20T18:58:15.731562+0000","last_peered":"2024-09-20T18:58:15.731562+0000","last_clean":"2024-09-20T18:58:15.731562+0000","last_became_active":"2024-09-20T18:49:45.933078+0000","last_became_peered":"2024-09-20T18:49:45.933078+0000","last_unstale":"2024-09-20T18:58:15.731562+0000","last_undegraded":"2024-09-20T18:58:15.731562+0000","last_fullsized":"2024-09-20T18:58:15.731562+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:57:18.032067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:18.032067+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:42.727785+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":405,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:16.351783+0000","last_change":"2024-09-20T18:55:19.996522+0000","last_active":"2024-09-20T18:58:16.351783+0000","last_peered":"2024-09-20T18:58:16.351783+0000","last_clean":"2024-09-20T18:58:16.351783+0000","last_became_active":"2024-09-20T18:49:48.500809+0000","last_became_peered":"2024-09-20T18:49:48.500809+0000","last_unstale":"2024-09-20T18:58:16.351783+0000","last_undegraded":"2024-09-20T18:58:16.351783+0000","last_fullsized":"2024-09-20T18:58:16.351783+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:55:19.996328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:19.996328+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"184'9","reported_seq":203,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-20T18:58:20.730880+0000","last_change":"2024-09-20T18:57:48.853376+0000","last_active":"2024-09-20T18:58:20.730880+0000","last_peered":"2024-09-20T18:58:20.730880+0000","last_clean":"2024-09-20T18:58:20.730880+0000","last_became_active":"2024-09-20T18:55:28.820941+0000","last_became_peered":"2024-09-20T18:55:28.820941+0000","last_unstale":"2024-09-20T18:58:20.730880+0000","last_undegraded":"2024-09-20T18:58:20.730880+0000","last_fullsized":"2024-09-20T18:58:20.730880+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'9","last_scrub_stamp":"2024-09-20T18:57:48.853302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:48.853302+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:06.295025+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3","version":"236'14296","reported_seq":26818,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:15.730541+0000","last_change":"2024-09-20T18:57:53.578343+0000","last_active":"2024-09-20T18:58:15.730541+0000","last_peered":"2024-09-20T18:58:15.730541+0000","last_clean":"2024-09-20T18:58:15.730541+0000","last_became_active":"2024-09-20T18:56:25.687391+0000","last_became_peered":"2024-09-20T18:56:25.687391+0000","last_unstale":"2024-09-20T18:58:15.730541+0000","last_undegraded":"2024-09-20T18:58:15.730541+0000","last_fullsized":"2024-09-20T18:58:15.730541+0000","mapping_epoch":203,"log_start":"214'9972","ondisk_log_start":"214'9972","created":160,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'12840","last_scrub_stamp":"2024-09-20T18:57:53.578263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:57:53.578263+0000","objects_scrubbed":1475,"log_size":4324,"log_dups_size":0,"ondisk_log_size":4324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:55.743448+0000","scrub_duration":660,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14296,"num_write_kb":457472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6156,"num_bytes_recovered":403439616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":397,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-20T18:58:13.260530+0000","last_change":"2024-09-20T18:55:18.581290+0000","last_active":"2024-09-20T18:58:13.260530+0000","last_peered":"2024-09-20T18:58:13.260530+0000","last_clean":"2024-09-20T18:58:13.260530+0000","last_became_active":"2024-09-20T18:49:58.992302+0000","last_became_peered":"2024-09-20T18:49:58.992302+0000","last_unstale":"2024-09-20T18:58:13.260530+0000","last_undegraded":"2024-09-20T18:58:13.260530+0000","last_fullsized":"2024-09-20T18:58:13.260530+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:55:18.581200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:18.581200+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":386,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:15.730548+0000","last_change":"2024-09-20T18:54:54.288199+0000","last_active":"2024-09-20T18:58:15.730548+0000","last_peered":"2024-09-20T18:58:15.730548+0000","last_clean":"2024-09-20T18:58:15.730548+0000","last_became_active":"2024-09-20T18:49:45.936104+0000","last_became_peered":"2024-09-20T18:49:45.936104+0000","last_unstale":"2024-09-20T18:58:15.730548+0000","last_undegraded":"2024-09-20T18:58:15.730548+0000","last_fullsized":"2024-09-20T18:58:15.730548+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:54.288117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:54.288117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":46,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":496,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-20T18:58:20.730821+0000","last_change":"2024-09-20T18:54:14.292660+0000","last_active":"2024-09-20T18:58:20.730821+0000","last_peered":"2024-09-20T18:58:20.730821+0000","last_clean":"2024-09-20T18:58:20.730821+0000","last_became_active":"2024-09-20T18:49:49.960697+0000","last_became_peered":"2024-09-20T18:49:49.960697+0000","last_unstale":"2024-09-20T18:58:20.730821+0000","last_undegraded":"2024-09-20T18:58:20.730821+0000","last_fullsized":"2024-09-20T18:58:20.730821+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:54:14.292610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:14.292610+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:10.776813+0000","scrub_duration":34,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":373,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:16.441741+0000","last_change":"2024-09-20T18:54:10.381171+0000","last_active":"2024-09-20T18:58:16.441741+0000","last_peered":"2024-09-20T18:58:16.441741+0000","last_clean":"2024-09-20T18:58:16.441741+0000","last_became_active":"2024-09-20T18:49:51.971399+0000","last_became_peered":"2024-09-20T18:49:51.971399+0000","last_unstale":"2024-09-20T18:58:16.441741+0000","last_undegraded":"2024-09-20T18:58:16.441741+0000","last_fullsized":"2024-09-20T18:58:16.441741+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:10.381108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:10.381108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:30.527012+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.c","version":"236'14392","reported_seq":14558,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:16.351346+0000","last_change":"2024-09-20T18:54:34.311256+0000","last_active":"2024-09-20T18:58:16.351346+0000","last_peered":"2024-09-20T18:58:16.351346+0000","last_clean":"2024-09-20T18:58:16.351346+0000","last_became_active":"2024-09-20T18:54:34.310904+0000","last_became_peered":"2024-09-20T18:54:34.310904+0000","last_unstale":"2024-09-20T18:58:16.351346+0000","last_undegraded":"2024-09-20T18:58:16.351346+0000","last_fullsized":"2024-09-20T18:58:16.351346+0000","mapping_epoch":160,"log_start":"212'9511","ondisk_log_start":"212'9511","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4881,"log_dups_size":0,"ondisk_log_size":4881,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14392,"num_write_kb":460544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":423,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:15.730340+0000","last_change":"2024-09-20T18:58:08.025426+0000","last_active":"2024-09-20T18:58:15.730340+0000","last_peered":"2024-09-20T18:58:15.730340+0000","last_clean":"2024-09-20T18:58:15.730340+0000","last_became_active":"2024-09-20T18:49:48.499189+0000","last_became_peered":"2024-09-20T18:49:48.499189+0000","last_unstale":"2024-09-20T18:58:15.730340+0000","last_undegraded":"2024-09-20T18:58:15.730340+0000","last_fullsized":"2024-09-20T18:58:15.730340+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-20T18:58:08.025202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:08.025202+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:16.521840+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":378,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:15.731894+0000","last_change":"2024-09-20T18:54:57.237740+0000","last_active":"2024-09-20T18:58:15.731894+0000","last_peered":"2024-09-20T18:58:15.731894+0000","last_clean":"2024-09-20T18:58:15.731894+0000","last_became_active":"2024-09-20T18:49:45.940544+0000","last_became_peered":"2024-09-20T18:49:45.940544+0000","last_unstale":"2024-09-20T18:58:15.731894+0000","last_undegraded":"2024-09-20T18:58:15.731894+0000","last_fullsized":"2024-09-20T18:58:15.731894+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.237675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.237675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":391,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-20T18:58:20.729988+0000","last_change":"2024-09-20T18:54:51.229251+0000","last_active":"2024-09-20T18:58:20.729988+0000","last_peered":"2024-09-20T18:58:20.729988+0000","last_clean":"2024-09-20T18:58:20.729988+0000","last_became_active":"2024-09-20T18:49:49.963073+0000","last_became_peered":"2024-09-20T18:49:49.963073+0000","last_unstale":"2024-09-20T18:58:20.729988+0000","last_undegraded":"2024-09-20T18:58:20.729988+0000","last_fullsized":"2024-09-20T18:58:20.729988+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.229201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.229201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:20.666517+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":387,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-20T18:58:20.730476+0000","last_change":"2024-09-20T18:54:55.323937+0000","last_active":"2024-09-20T18:58:20.730476+0000","last_peered":"2024-09-20T18:58:20.730476+0000","last_clean":"2024-09-20T18:58:20.730476+0000","last_became_active":"2024-09-20T18:49:51.968793+0000","last_became_peered":"2024-09-20T18:49:51.968793+0000","last_unstale":"2024-09-20T18:58:20.730476+0000","last_undegraded":"2024-09-20T18:58:20.730476+0000","last_fullsized":"2024-09-20T18:58:20.730476+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:55.323692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:55.323692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"236'14372","reported_seq":14539,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:16.442436+0000","last_change":"2024-09-20T18:54:34.311680+0000","last_active":"2024-09-20T18:58:16.442436+0000","last_peered":"2024-09-20T18:58:16.442436+0000","last_clean":"2024-09-20T18:58:16.442436+0000","last_became_active":"2024-09-20T18:54:34.311241+0000","last_became_peered":"2024-09-20T18:54:34.311241+0000","last_unstale":"2024-09-20T18:58:16.442436+0000","last_undegraded":"2024-09-20T18:58:16.442436+0000","last_fullsized":"2024-09-20T18:58:16.442436+0000","mapping_epoch":160,"log_start":"211'9346","ondisk_log_start":"211'9346","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":5026,"log_dups_size":0,"ondisk_log_size":5026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:36.546826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14372,"num_write_kb":459841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":447,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-20T18:58:20.730027+0000","last_change":"2024-09-20T18:55:28.853375+0000","last_active":"2024-09-20T18:58:20.730027+0000","last_peered":"2024-09-20T18:58:20.730027+0000","last_clean":"2024-09-20T18:58:20.730027+0000","last_became_active":"2024-09-20T18:55:28.831057+0000","last_became_peered":"2024-09-20T18:55:28.831057+0000","last_unstale":"2024-09-20T18:58:20.730027+0000","last_undegraded":"2024-09-20T18:58:20.730027+0000","last_fullsized":"2024-09-20T18:58:20.730027+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-20T18:54:57.394893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.394893+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":378,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:16.351982+0000","last_change":"2024-09-20T18:51:27.187060+0000","last_active":"2024-09-20T18:58:16.351982+0000","last_peered":"2024-09-20T18:58:16.351982+0000","last_clean":"2024-09-20T18:58:16.351982+0000","last_became_active":"2024-09-20T18:49:45.940469+0000","last_became_peered":"2024-09-20T18:49:45.940469+0000","last_unstale":"2024-09-20T18:58:16.351982+0000","last_undegraded":"2024-09-20T18:58:16.351982+0000","last_fullsized":"2024-09-20T18:58:16.351982+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:51:27.186991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:27.186991+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"66'8","reported_seq":498,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:15.730203+0000","last_change":"2024-09-20T18:54:47.223010+0000","last_active":"2024-09-20T18:58:15.730203+0000","last_peered":"2024-09-20T18:58:15.730203+0000","last_clean":"2024-09-20T18:58:15.730203+0000","last_became_active":"2024-09-20T18:49:49.956133+0000","last_became_peered":"2024-09-20T18:49:49.956133+0000","last_unstale":"2024-09-20T18:58:15.730203+0000","last_undegraded":"2024-09-20T18:58:15.730203+0000","last_fullsized":"2024-09-20T18:58:15.730203+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:54:47.222937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:47.222937+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":385,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-20T18:58:20.734822+0000","last_change":"2024-09-20T18:55:28.819147+0000","last_active":"2024-09-20T18:58:20.734822+0000","last_peered":"2024-09-20T18:58:20.734822+0000","last_clean":"2024-09-20T18:58:20.734822+0000","last_became_active":"2024-09-20T18:55:28.818804+0000","last_became_peered":"2024-09-20T18:55:28.818804+0000","last_unstale":"2024-09-20T18:58:20.734822+0000","last_undegraded":"2024-09-20T18:58:20.734822+0000","last_fullsized":"2024-09-20T18:58:20.734822+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:42.880724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:42.880724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"236'14402","reported_seq":14568,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:16.351563+0000","last_change":"2024-09-20T18:54:34.309939+0000","last_active":"2024-09-20T18:58:16.351563+0000","last_peered":"2024-09-20T18:58:16.351563+0000","last_clean":"2024-09-20T18:58:16.351563+0000","last_became_active":"2024-09-20T18:54:34.309630+0000","last_became_peered":"2024-09-20T18:54:34.309630+0000","last_unstale":"2024-09-20T18:58:16.351563+0000","last_undegraded":"2024-09-20T18:58:16.351563+0000","last_fullsized":"2024-09-20T18:58:16.351563+0000","mapping_epoch":160,"log_start":"212'9549","ondisk_log_start":"212'9549","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4853,"log_dups_size":0,"ondisk_log_size":4853,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14402,"num_write_kb":460864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":414,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:15.731082+0000","last_change":"2024-09-20T18:58:03.053521+0000","last_active":"2024-09-20T18:58:15.731082+0000","last_peered":"2024-09-20T18:58:15.731082+0000","last_clean":"2024-09-20T18:58:15.731082+0000","last_became_active":"2024-09-20T18:49:47.949626+0000","last_became_peered":"2024-09-20T18:49:47.949626+0000","last_unstale":"2024-09-20T18:58:15.731082+0000","last_undegraded":"2024-09-20T18:58:15.731082+0000","last_fullsized":"2024-09-20T18:58:15.731082+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:58:03.053310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:03.053310+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:05.671653+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":394,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:16.442798+0000","last_change":"2024-09-20T18:54:18.326947+0000","last_active":"2024-09-20T18:58:16.442798+0000","last_peered":"2024-09-20T18:58:16.442798+0000","last_clean":"2024-09-20T18:58:16.442798+0000","last_became_active":"2024-09-20T18:49:45.942496+0000","last_became_peered":"2024-09-20T18:49:45.942496+0000","last_unstale":"2024-09-20T18:58:16.442798+0000","last_undegraded":"2024-09-20T18:58:16.442798+0000","last_fullsized":"2024-09-20T18:58:16.442798+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:18.326875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:18.326875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:32.505949+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"184'9","reported_seq":500,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:16.351286+0000","last_change":"2024-09-20T18:55:29.010062+0000","last_active":"2024-09-20T18:58:16.351286+0000","last_peered":"2024-09-20T18:58:16.351286+0000","last_clean":"2024-09-20T18:58:16.351286+0000","last_became_active":"2024-09-20T18:55:28.830643+0000","last_became_peered":"2024-09-20T18:55:28.830643+0000","last_unstale":"2024-09-20T18:58:16.351286+0000","last_undegraded":"2024-09-20T18:58:16.351286+0000","last_fullsized":"2024-09-20T18:58:16.351286+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:29.010015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:29.010015+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":358,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:16.350646+0000","last_change":"2024-09-20T18:49:51.965733+0000","last_active":"2024-09-20T18:58:16.350646+0000","last_peered":"2024-09-20T18:58:16.350646+0000","last_clean":"2024-09-20T18:58:16.350646+0000","last_became_active":"2024-09-20T18:49:51.964780+0000","last_became_peered":"2024-09-20T18:49:51.964780+0000","last_unstale":"2024-09-20T18:58:16.350646+0000","last_undegraded":"2024-09-20T18:58:16.350646+0000","last_fullsized":"2024-09-20T18:58:16.350646+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:49:50.942419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"236'14714","reported_seq":14892,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:15.731734+0000","last_change":"2024-09-20T18:58:10.233639+0000","last_active":"2024-09-20T18:58:15.731734+0000","last_peered":"2024-09-20T18:58:15.731734+0000","last_clean":"2024-09-20T18:58:15.731734+0000","last_became_active":"2024-09-20T18:54:34.309231+0000","last_became_peered":"2024-09-20T18:54:34.309231+0000","last_unstale":"2024-09-20T18:58:15.731734+0000","last_undegraded":"2024-09-20T18:58:15.731734+0000","last_fullsized":"2024-09-20T18:58:15.731734+0000","mapping_epoch":160,"log_start":"214'10332","ondisk_log_start":"214'10332","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"233'14387","last_scrub_stamp":"2024-09-20T18:58:10.233473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:58:10.233473+0000","objects_scrubbed":334,"log_size":4382,"log_dups_size":0,"ondisk_log_size":4382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:16.682043+0000","scrub_duration":224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14714,"num_write_kb":470848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":418,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:16.351422+0000","last_change":"2024-09-20T18:54:14.923895+0000","last_active":"2024-09-20T18:58:16.351422+0000","last_peered":"2024-09-20T18:58:16.351422+0000","last_clean":"2024-09-20T18:58:16.351422+0000","last_became_active":"2024-09-20T18:49:48.497736+0000","last_became_peered":"2024-09-20T18:49:48.497736+0000","last_unstale":"2024-09-20T18:58:16.351422+0000","last_undegraded":"2024-09-20T18:58:16.351422+0000","last_fullsized":"2024-09-20T18:58:16.351422+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-20T18:54:14.923682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:14.923682+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":379,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:15.729746+0000","last_change":"2024-09-20T18:57:09.940634+0000","last_active":"2024-09-20T18:58:15.729746+0000","last_peered":"2024-09-20T18:58:15.729746+0000","last_clean":"2024-09-20T18:58:15.729746+0000","last_became_active":"2024-09-20T18:49:45.942253+0000","last_became_peered":"2024-09-20T18:49:45.942253+0000","last_unstale":"2024-09-20T18:58:15.729746+0000","last_undegraded":"2024-09-20T18:58:15.729746+0000","last_fullsized":"2024-09-20T18:58:15.729746+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:09.940517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:09.940517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:18.833936+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":124,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:16.351251+0000","last_change":"2024-09-20T18:55:28.830954+0000","last_active":"2024-09-20T18:58:16.351251+0000","last_peered":"2024-09-20T18:58:16.351251+0000","last_clean":"2024-09-20T18:58:16.351251+0000","last_became_active":"2024-09-20T18:55:28.830645+0000","last_became_peered":"2024-09-20T18:55:28.830645+0000","last_unstale":"2024-09-20T18:58:16.351251+0000","last_undegraded":"2024-09-20T18:58:16.351251+0000","last_fullsized":"2024-09-20T18:58:16.351251+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:50.610871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:50.610871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":398,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:15.731125+0000","last_change":"2024-09-20T18:57:58.973310+0000","last_active":"2024-09-20T18:58:15.731125+0000","last_peered":"2024-09-20T18:58:15.731125+0000","last_clean":"2024-09-20T18:58:15.731125+0000","last_became_active":"2024-09-20T18:55:28.821588+0000","last_became_peered":"2024-09-20T18:55:28.821588+0000","last_unstale":"2024-09-20T18:58:15.731125+0000","last_undegraded":"2024-09-20T18:58:15.731125+0000","last_fullsized":"2024-09-20T18:58:15.731125+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:58.973176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:58.973176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:03.568279+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"236'14598","reported_seq":14790,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:16.441948+0000","last_change":"2024-09-20T18:57:47.843800+0000","last_active":"2024-09-20T18:58:16.441948+0000","last_peered":"2024-09-20T18:58:16.441948+0000","last_clean":"2024-09-20T18:58:16.441948+0000","last_became_active":"2024-09-20T18:54:34.311181+0000","last_became_peered":"2024-09-20T18:54:34.311181+0000","last_unstale":"2024-09-20T18:58:16.441948+0000","last_undegraded":"2024-09-20T18:58:16.441948+0000","last_fullsized":"2024-09-20T18:58:16.441948+0000","mapping_epoch":160,"log_start":"211'9594","ondisk_log_start":"211'9594","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'12813","last_scrub_stamp":"2024-09-20T18:57:47.843722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:57:47.843722+0000","objects_scrubbed":1796,"log_size":5004,"log_dups_size":0,"ondisk_log_size":5004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:52.400285+0000","scrub_duration":406,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14598,"num_write_kb":467136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"66'28","reported_seq":542,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:15.729655+0000","last_change":"2024-09-20T18:58:07.018449+0000","last_active":"2024-09-20T18:58:15.729655+0000","last_peered":"2024-09-20T18:58:15.729655+0000","last_clean":"2024-09-20T18:58:15.729655+0000","last_became_active":"2024-09-20T18:49:47.955335+0000","last_became_peered":"2024-09-20T18:49:47.955335+0000","last_unstale":"2024-09-20T18:58:15.729655+0000","last_undegraded":"2024-09-20T18:58:15.729655+0000","last_fullsized":"2024-09-20T18:58:15.729655+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'28","last_scrub_stamp":"2024-09-20T18:58:07.018340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:07.018340+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:28.256912+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":390,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-20T18:58:20.731006+0000","last_change":"2024-09-20T18:57:21.807755+0000","last_active":"2024-09-20T18:58:20.731006+0000","last_peered":"2024-09-20T18:58:20.731006+0000","last_clean":"2024-09-20T18:58:20.731006+0000","last_became_active":"2024-09-20T18:55:28.831574+0000","last_became_peered":"2024-09-20T18:55:28.831574+0000","last_unstale":"2024-09-20T18:58:20.731006+0000","last_undegraded":"2024-09-20T18:58:20.731006+0000","last_fullsized":"2024-09-20T18:58:20.731006+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:21.807703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:21.807703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:42.821495+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1],"acting":[4,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":423,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:16.442476+0000","last_change":"2024-09-20T18:55:28.854615+0000","last_active":"2024-09-20T18:58:16.442476+0000","last_peered":"2024-09-20T18:58:16.442476+0000","last_clean":"2024-09-20T18:58:16.442476+0000","last_became_active":"2024-09-20T18:55:28.837226+0000","last_became_peered":"2024-09-20T18:55:28.837226+0000","last_unstale":"2024-09-20T18:58:16.442476+0000","last_undegraded":"2024-09-20T18:58:16.442476+0000","last_fullsized":"2024-09-20T18:58:16.442476+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-20T18:54:51.227531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.227531+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:43.598074+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0012781159999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":386,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:16.351651+0000","last_change":"2024-09-20T18:55:13.949348+0000","last_active":"2024-09-20T18:58:16.351651+0000","last_peered":"2024-09-20T18:58:16.351651+0000","last_clean":"2024-09-20T18:58:16.351651+0000","last_became_active":"2024-09-20T18:49:49.953971+0000","last_became_peered":"2024-09-20T18:49:49.953971+0000","last_unstale":"2024-09-20T18:58:16.351651+0000","last_undegraded":"2024-09-20T18:58:16.351651+0000","last_fullsized":"2024-09-20T18:58:16.351651+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:13.949269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:13.949269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":366,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:15.729578+0000","last_change":"2024-09-20T18:57:12.744116+0000","last_active":"2024-09-20T18:58:15.729578+0000","last_peered":"2024-09-20T18:58:15.729578+0000","last_clean":"2024-09-20T18:58:15.729578+0000","last_became_active":"2024-09-20T18:49:51.969400+0000","last_became_peered":"2024-09-20T18:49:51.969400+0000","last_unstale":"2024-09-20T18:58:15.729578+0000","last_undegraded":"2024-09-20T18:58:15.729578+0000","last_fullsized":"2024-09-20T18:58:15.729578+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:12.744040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:12.744040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:17.964285+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"236'14718","reported_seq":14901,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:15.731619+0000","last_change":"2024-09-20T18:58:02.207422+0000","last_active":"2024-09-20T18:58:15.731619+0000","last_peered":"2024-09-20T18:58:15.731619+0000","last_clean":"2024-09-20T18:58:15.731619+0000","last_became_active":"2024-09-20T18:54:33.925750+0000","last_became_peered":"2024-09-20T18:54:33.925750+0000","last_unstale":"2024-09-20T18:58:15.731619+0000","last_undegraded":"2024-09-20T18:58:15.731619+0000","last_fullsized":"2024-09-20T18:58:15.731619+0000","mapping_epoch":160,"log_start":"215'10340","ondisk_log_start":"215'10340","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"233'13810","last_scrub_stamp":"2024-09-20T18:58:02.207357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:58:02.207357+0000","objects_scrubbed":908,"log_size":4378,"log_dups_size":0,"ondisk_log_size":4378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:23.632342+0000","scrub_duration":122,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14718,"num_write_kb":470976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":425,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:15.729749+0000","last_change":"2024-09-20T18:58:06.079117+0000","last_active":"2024-09-20T18:58:15.729749+0000","last_peered":"2024-09-20T18:58:15.729749+0000","last_clean":"2024-09-20T18:58:15.729749+0000","last_became_active":"2024-09-20T18:49:47.950904+0000","last_became_peered":"2024-09-20T18:49:47.950904+0000","last_unstale":"2024-09-20T18:58:15.729749+0000","last_undegraded":"2024-09-20T18:58:15.729749+0000","last_fullsized":"2024-09-20T18:58:15.729749+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-20T18:58:06.078352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:06.078352+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:26.574178+0000","scrub_duration":59,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":402,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:15.729271+0000","last_change":"2024-09-20T18:55:28.830832+0000","last_active":"2024-09-20T18:58:15.729271+0000","last_peered":"2024-09-20T18:58:15.729271+0000","last_clean":"2024-09-20T18:58:15.729271+0000","last_became_active":"2024-09-20T18:55:28.830269+0000","last_became_peered":"2024-09-20T18:55:28.830269+0000","last_unstale":"2024-09-20T18:58:15.729271+0000","last_undegraded":"2024-09-20T18:58:15.729271+0000","last_fullsized":"2024-09-20T18:58:15.729271+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:23.359769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:23.359769+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:27.139436+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":387,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-20T18:58:13.254833+0000","last_change":"2024-09-20T18:52:07.718766+0000","last_active":"2024-09-20T18:58:13.254833+0000","last_peered":"2024-09-20T18:58:13.254833+0000","last_clean":"2024-09-20T18:58:13.254833+0000","last_became_active":"2024-09-20T18:49:44.937135+0000","last_became_peered":"2024-09-20T18:49:44.937135+0000","last_unstale":"2024-09-20T18:58:13.254833+0000","last_undegraded":"2024-09-20T18:58:13.254833+0000","last_fullsized":"2024-09-20T18:58:13.254833+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:52:07.718646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:52:07.718646+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.00095558800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":369,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-20T18:58:13.260710+0000","last_change":"2024-09-20T18:51:59.796450+0000","last_active":"2024-09-20T18:58:13.260710+0000","last_peered":"2024-09-20T18:58:13.260710+0000","last_clean":"2024-09-20T18:58:13.260710+0000","last_became_active":"2024-09-20T18:49:49.954424+0000","last_became_peered":"2024-09-20T18:49:49.954424+0000","last_unstale":"2024-09-20T18:58:13.260710+0000","last_undegraded":"2024-09-20T18:58:13.260710+0000","last_fullsized":"2024-09-20T18:58:13.260710+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:59.796383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:59.796383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":377,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-20T18:58:20.750220+0000","last_change":"2024-09-20T18:55:28.826659+0000","last_active":"2024-09-20T18:58:20.750220+0000","last_peered":"2024-09-20T18:58:20.750220+0000","last_clean":"2024-09-20T18:58:20.750220+0000","last_became_active":"2024-09-20T18:55:28.825812+0000","last_became_peered":"2024-09-20T18:55:28.825812+0000","last_unstale":"2024-09-20T18:58:20.750220+0000","last_undegraded":"2024-09-20T18:58:20.750220+0000","last_fullsized":"2024-09-20T18:58:20.750220+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:39.814259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:39.814259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"236'14618","reported_seq":14784,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:15.730727+0000","last_change":"2024-09-20T18:54:34.311774+0000","last_active":"2024-09-20T18:58:15.730727+0000","last_peered":"2024-09-20T18:58:15.730727+0000","last_clean":"2024-09-20T18:58:15.730727+0000","last_became_active":"2024-09-20T18:54:34.311487+0000","last_became_peered":"2024-09-20T18:54:34.311487+0000","last_unstale":"2024-09-20T18:58:15.730727+0000","last_undegraded":"2024-09-20T18:58:15.730727+0000","last_fullsized":"2024-09-20T18:58:15.730727+0000","mapping_epoch":160,"log_start":"215'10392","ondisk_log_start":"215'10392","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4226,"log_dups_size":0,"ondisk_log_size":4226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:43.066892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14618,"num_write_kb":467776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":415,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:16.351864+0000","last_change":"2024-09-20T18:55:14.951950+0000","last_active":"2024-09-20T18:58:16.351864+0000","last_peered":"2024-09-20T18:58:16.351864+0000","last_clean":"2024-09-20T18:58:16.351864+0000","last_became_active":"2024-09-20T18:49:47.949230+0000","last_became_peered":"2024-09-20T18:49:47.949230+0000","last_unstale":"2024-09-20T18:58:16.351864+0000","last_undegraded":"2024-09-20T18:58:16.351864+0000","last_fullsized":"2024-09-20T18:58:16.351864+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:55:14.951892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:14.951892+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":394,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:16.351231+0000","last_change":"2024-09-20T18:54:27.905856+0000","last_active":"2024-09-20T18:58:16.351231+0000","last_peered":"2024-09-20T18:58:16.351231+0000","last_clean":"2024-09-20T18:58:16.351231+0000","last_became_active":"2024-09-20T18:49:45.939197+0000","last_became_peered":"2024-09-20T18:49:45.939197+0000","last_unstale":"2024-09-20T18:58:16.351231+0000","last_undegraded":"2024-09-20T18:58:16.351231+0000","last_fullsized":"2024-09-20T18:58:16.351231+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:27.905803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:27.905803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":918,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-20T18:58:20.734835+0000","last_change":"2024-09-20T18:54:26.359887+0000","last_active":"2024-09-20T18:58:20.734835+0000","last_peered":"2024-09-20T18:58:20.734835+0000","last_clean":"2024-09-20T18:58:20.734835+0000","last_became_active":"2024-09-20T18:49:44.936819+0000","last_became_peered":"2024-09-20T18:49:44.936819+0000","last_unstale":"2024-09-20T18:58:20.734835+0000","last_undegraded":"2024-09-20T18:58:20.734835+0000","last_fullsized":"2024-09-20T18:58:20.734835+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-20T18:54:26.359807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:54:26.359807+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0010996199999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":509,"num_read_kb":504,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":385,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:16.442925+0000","last_change":"2024-09-20T18:57:54.362085+0000","last_active":"2024-09-20T18:58:16.442925+0000","last_peered":"2024-09-20T18:58:16.442925+0000","last_clean":"2024-09-20T18:58:16.442925+0000","last_became_active":"2024-09-20T18:49:49.961066+0000","last_became_peered":"2024-09-20T18:49:49.961066+0000","last_unstale":"2024-09-20T18:58:16.442925+0000","last_undegraded":"2024-09-20T18:58:16.442925+0000","last_fullsized":"2024-09-20T18:58:16.442925+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:54.362038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:54.362038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:08.560302+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":391,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:15.729451+0000","last_change":"2024-09-20T18:58:08.019154+0000","last_active":"2024-09-20T18:58:15.729451+0000","last_peered":"2024-09-20T18:58:15.729451+0000","last_clean":"2024-09-20T18:58:15.729451+0000","last_became_active":"2024-09-20T18:49:51.970343+0000","last_became_peered":"2024-09-20T18:49:51.970343+0000","last_unstale":"2024-09-20T18:58:15.729451+0000","last_undegraded":"2024-09-20T18:58:15.729451+0000","last_fullsized":"2024-09-20T18:58:15.729451+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:08.019092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:08.019092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:32.791415+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"236'14532","reported_seq":25122,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:16.441963+0000","last_change":"2024-09-20T18:57:09.139489+0000","last_active":"2024-09-20T18:58:16.441963+0000","last_peered":"2024-09-20T18:58:16.441963+0000","last_clean":"2024-09-20T18:58:16.441963+0000","last_became_active":"2024-09-20T18:57:09.121592+0000","last_became_peered":"2024-09-20T18:57:09.121592+0000","last_unstale":"2024-09-20T18:58:16.441963+0000","last_undegraded":"2024-09-20T18:58:16.441963+0000","last_fullsized":"2024-09-20T18:58:16.441963+0000","mapping_epoch":216,"log_start":"211'9506","ondisk_log_start":"211'9506","created":160,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":5026,"log_dups_size":0,"ondisk_log_size":5026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:35.584332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14532,"num_write_kb":465024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5206,"num_bytes_recovered":341180416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"184'17","reported_seq":232,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-20T18:58:20.731168+0000","last_change":"2024-09-20T18:57:22.816240+0000","last_active":"2024-09-20T18:58:20.731168+0000","last_peered":"2024-09-20T18:58:20.731168+0000","last_clean":"2024-09-20T18:58:20.731168+0000","last_became_active":"2024-09-20T18:55:28.831024+0000","last_became_peered":"2024-09-20T18:55:28.831024+0000","last_unstale":"2024-09-20T18:58:20.731168+0000","last_undegraded":"2024-09-20T18:58:20.731168+0000","last_fullsized":"2024-09-20T18:58:20.731168+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'17","last_scrub_stamp":"2024-09-20T18:57:22.816154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:22.816154+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:28.727289+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":24,"num_read_kb":14,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":394,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:15.731445+0000","last_change":"2024-09-20T18:55:28.830447+0000","last_active":"2024-09-20T18:58:15.731445+0000","last_peered":"2024-09-20T18:58:15.731445+0000","last_clean":"2024-09-20T18:58:15.731445+0000","last_became_active":"2024-09-20T18:55:28.830147+0000","last_became_peered":"2024-09-20T18:55:28.830147+0000","last_unstale":"2024-09-20T18:58:15.731445+0000","last_undegraded":"2024-09-20T18:58:15.731445+0000","last_fullsized":"2024-09-20T18:58:15.731445+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:13.144896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:13.144896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:28.731761+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":451,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-20T18:58:20.735039+0000","last_change":"2024-09-20T18:55:28.844060+0000","last_active":"2024-09-20T18:58:20.735039+0000","last_peered":"2024-09-20T18:58:20.735039+0000","last_clean":"2024-09-20T18:58:20.735039+0000","last_became_active":"2024-09-20T18:55:28.829325+0000","last_became_peered":"2024-09-20T18:55:28.829325+0000","last_unstale":"2024-09-20T18:58:20.735039+0000","last_undegraded":"2024-09-20T18:58:20.735039+0000","last_fullsized":"2024-09-20T18:58:20.735039+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T18:55:13.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T18:55:13.696607+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":374,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-20T18:58:20.749950+0000","last_change":"2024-09-20T18:51:22.852942+0000","last_active":"2024-09-20T18:58:20.749950+0000","last_peered":"2024-09-20T18:58:20.749950+0000","last_clean":"2024-09-20T18:58:20.749950+0000","last_became_active":"2024-09-20T18:49:49.956128+0000","last_became_peered":"2024-09-20T18:49:49.956128+0000","last_unstale":"2024-09-20T18:58:20.749950+0000","last_undegraded":"2024-09-20T18:58:20.749950+0000","last_fullsized":"2024-09-20T18:58:20.749950+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:22.852869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:22.852869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":389,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:16.441728+0000","last_change":"2024-09-20T18:58:03.523127+0000","last_active":"2024-09-20T18:58:16.441728+0000","last_peered":"2024-09-20T18:58:16.441728+0000","last_clean":"2024-09-20T18:58:16.441728+0000","last_became_active":"2024-09-20T18:49:51.971332+0000","last_became_peered":"2024-09-20T18:49:51.971332+0000","last_unstale":"2024-09-20T18:58:16.441728+0000","last_undegraded":"2024-09-20T18:58:16.441728+0000","last_fullsized":"2024-09-20T18:58:16.441728+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:03.523078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:03.523078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:14.059180+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"236'14562","reported_seq":14732,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-20T18:58:20.730089+0000","last_change":"2024-09-20T18:54:33.929366+0000","last_active":"2024-09-20T18:58:20.730089+0000","last_peered":"2024-09-20T18:58:20.730089+0000","last_clean":"2024-09-20T18:58:20.730089+0000","last_became_active":"2024-09-20T18:54:33.929072+0000","last_became_peered":"2024-09-20T18:54:33.929072+0000","last_unstale":"2024-09-20T18:58:20.730089+0000","last_undegraded":"2024-09-20T18:58:20.730089+0000","last_fullsized":"2024-09-20T18:58:20.730089+0000","mapping_epoch":160,"log_start":"214'10002","ondisk_log_start":"214'10002","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4560,"log_dups_size":0,"ondisk_log_size":4560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:19.627729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14562,"num_write_kb":465984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":416,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-20T18:58:20.730871+0000","last_change":"2024-09-20T18:55:28.840105+0000","last_active":"2024-09-20T18:58:20.730871+0000","last_peered":"2024-09-20T18:58:20.730871+0000","last_clean":"2024-09-20T18:58:20.730871+0000","last_became_active":"2024-09-20T18:55:28.818269+0000","last_became_peered":"2024-09-20T18:55:28.818269+0000","last_unstale":"2024-09-20T18:58:20.730871+0000","last_undegraded":"2024-09-20T18:58:20.730871+0000","last_fullsized":"2024-09-20T18:58:20.730871+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:51:14.397930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:14.397930+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:23.803752+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":382,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-20T18:58:20.730257+0000","last_change":"2024-09-20T18:51:18.410601+0000","last_active":"2024-09-20T18:58:20.730257+0000","last_peered":"2024-09-20T18:58:20.730257+0000","last_clean":"2024-09-20T18:58:20.730257+0000","last_became_active":"2024-09-20T18:49:45.944579+0000","last_became_peered":"2024-09-20T18:49:45.944579+0000","last_unstale":"2024-09-20T18:58:20.730257+0000","last_undegraded":"2024-09-20T18:58:20.730257+0000","last_fullsized":"2024-09-20T18:58:20.730257+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:18.410447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:18.410447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:29.917821+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":392,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:16.442803+0000","last_change":"2024-09-20T18:55:28.830634+0000","last_active":"2024-09-20T18:58:16.442803+0000","last_peered":"2024-09-20T18:58:16.442803+0000","last_clean":"2024-09-20T18:58:16.442803+0000","last_became_active":"2024-09-20T18:55:28.827785+0000","last_became_peered":"2024-09-20T18:55:28.827785+0000","last_unstale":"2024-09-20T18:58:16.442803+0000","last_undegraded":"2024-09-20T18:58:16.442803+0000","last_fullsized":"2024-09-20T18:58:16.442803+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:11.401777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:11.401777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:37.550724+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":381,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:15.729079+0000","last_change":"2024-09-20T18:57:23.721754+0000","last_active":"2024-09-20T18:58:15.729079+0000","last_peered":"2024-09-20T18:58:15.729079+0000","last_clean":"2024-09-20T18:58:15.729079+0000","last_became_active":"2024-09-20T18:55:28.833115+0000","last_became_peered":"2024-09-20T18:55:28.833115+0000","last_unstale":"2024-09-20T18:58:15.729079+0000","last_undegraded":"2024-09-20T18:58:15.729079+0000","last_fullsized":"2024-09-20T18:58:15.729079+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:23.721643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:23.721643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:25.737391+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1","version":"236'14432","reported_seq":14623,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:15.731334+0000","last_change":"2024-09-20T18:57:50.704022+0000","last_active":"2024-09-20T18:58:15.731334+0000","last_peered":"2024-09-20T18:58:15.731334+0000","last_clean":"2024-09-20T18:58:15.731334+0000","last_became_active":"2024-09-20T18:54:34.309781+0000","last_became_peered":"2024-09-20T18:54:34.309781+0000","last_unstale":"2024-09-20T18:58:15.731334+0000","last_undegraded":"2024-09-20T18:58:15.731334+0000","last_fullsized":"2024-09-20T18:58:15.731334+0000","mapping_epoch":160,"log_start":"214'10124","ondisk_log_start":"214'10124","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'12866","last_scrub_stamp":"2024-09-20T18:57:50.703702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:57:50.703702+0000","objects_scrubbed":1599,"log_size":4308,"log_dups_size":0,"ondisk_log_size":4308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:00.179362+0000","scrub_duration":719,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14432,"num_write_kb":461824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":414,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-20T18:58:20.731266+0000","last_change":"2024-09-20T18:52:02.944536+0000","last_active":"2024-09-20T18:58:20.731266+0000","last_peered":"2024-09-20T18:58:20.731266+0000","last_clean":"2024-09-20T18:58:20.731266+0000","last_became_active":"2024-09-20T18:49:47.947636+0000","last_became_peered":"2024-09-20T18:49:47.947636+0000","last_unstale":"2024-09-20T18:58:20.731266+0000","last_undegraded":"2024-09-20T18:58:20.731266+0000","last_fullsized":"2024-09-20T18:58:20.731266+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-20T18:52:02.944434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:02.944434+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:34.896863+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":252,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-20T18:58:20.730173+0000","last_change":"2024-09-20T18:55:28.822330+0000","last_active":"2024-09-20T18:58:20.730173+0000","last_peered":"2024-09-20T18:58:20.730173+0000","last_clean":"2024-09-20T18:58:20.730173+0000","last_became_active":"2024-09-20T18:55:28.821959+0000","last_became_peered":"2024-09-20T18:55:28.821959+0000","last_unstale":"2024-09-20T18:58:20.730173+0000","last_undegraded":"2024-09-20T18:58:20.730173+0000","last_fullsized":"2024-09-20T18:58:20.730173+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:45.669427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:45.669427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":378,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:15.729491+0000","last_change":"2024-09-20T18:54:11.377907+0000","last_active":"2024-09-20T18:58:15.729491+0000","last_peered":"2024-09-20T18:58:15.729491+0000","last_clean":"2024-09-20T18:58:15.729491+0000","last_became_active":"2024-09-20T18:49:49.959160+0000","last_became_peered":"2024-09-20T18:49:49.959160+0000","last_unstale":"2024-09-20T18:58:15.729491+0000","last_undegraded":"2024-09-20T18:58:15.729491+0000","last_fullsized":"2024-09-20T18:58:15.729491+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:11.376951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:11.376951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:29.061620+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":373,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-20T18:58:13.268840+0000","last_change":"2024-09-20T18:52:35.635897+0000","last_active":"2024-09-20T18:58:13.268840+0000","last_peered":"2024-09-20T18:58:13.268840+0000","last_clean":"2024-09-20T18:58:13.268840+0000","last_became_active":"2024-09-20T18:49:51.970108+0000","last_became_peered":"2024-09-20T18:49:51.970108+0000","last_unstale":"2024-09-20T18:58:13.268840+0000","last_undegraded":"2024-09-20T18:58:13.268840+0000","last_fullsized":"2024-09-20T18:58:13.268840+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:35.635818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:35.635818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"236'14226","reported_seq":14392,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:15.729999+0000","last_change":"2024-09-20T18:54:33.929552+0000","last_active":"2024-09-20T18:58:15.729999+0000","last_peered":"2024-09-20T18:58:15.729999+0000","last_clean":"2024-09-20T18:58:15.729999+0000","last_became_active":"2024-09-20T18:54:33.929208+0000","last_became_peered":"2024-09-20T18:54:33.929208+0000","last_unstale":"2024-09-20T18:58:15.729999+0000","last_undegraded":"2024-09-20T18:58:15.729999+0000","last_fullsized":"2024-09-20T18:58:15.729999+0000","mapping_epoch":160,"log_start":"214'9921","ondisk_log_start":"214'9921","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4305,"log_dups_size":0,"ondisk_log_size":4305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:12.009343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14226,"num_write_kb":455232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":420,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-20T18:58:20.730830+0000","last_change":"2024-09-20T18:55:28.846773+0000","last_active":"2024-09-20T18:58:20.730830+0000","last_peered":"2024-09-20T18:58:20.730830+0000","last_clean":"2024-09-20T18:58:20.730830+0000","last_became_active":"2024-09-20T18:55:28.817101+0000","last_became_peered":"2024-09-20T18:55:28.817101+0000","last_unstale":"2024-09-20T18:58:20.730830+0000","last_undegraded":"2024-09-20T18:58:20.730830+0000","last_fullsized":"2024-09-20T18:58:20.730830+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:54:17.219548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:17.219548+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:12.738146+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":386,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:15.731898+0000","last_change":"2024-09-20T18:52:46.005172+0000","last_active":"2024-09-20T18:58:15.731898+0000","last_peered":"2024-09-20T18:58:15.731898+0000","last_clean":"2024-09-20T18:58:15.731898+0000","last_became_active":"2024-09-20T18:49:45.937264+0000","last_became_peered":"2024-09-20T18:49:45.937264+0000","last_unstale":"2024-09-20T18:58:15.731898+0000","last_undegraded":"2024-09-20T18:58:15.731898+0000","last_fullsized":"2024-09-20T18:58:15.731898+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:46.005109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:46.005109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:27.755958+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":496,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-20T18:58:13.256575+0000","last_change":"2024-09-20T18:55:17.549660+0000","last_active":"2024-09-20T18:58:13.256575+0000","last_peered":"2024-09-20T18:58:13.256575+0000","last_clean":"2024-09-20T18:58:13.256575+0000","last_became_active":"2024-09-20T18:49:49.957005+0000","last_became_peered":"2024-09-20T18:49:49.957005+0000","last_unstale":"2024-09-20T18:58:13.256575+0000","last_undegraded":"2024-09-20T18:58:13.256575+0000","last_fullsized":"2024-09-20T18:58:13.256575+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:17.549598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:17.549598+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":378,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-20T18:58:20.730707+0000","last_change":"2024-09-20T18:54:42.182875+0000","last_active":"2024-09-20T18:58:20.730707+0000","last_peered":"2024-09-20T18:58:20.730707+0000","last_clean":"2024-09-20T18:58:20.730707+0000","last_became_active":"2024-09-20T18:49:51.969355+0000","last_became_peered":"2024-09-20T18:49:51.969355+0000","last_unstale":"2024-09-20T18:58:20.730707+0000","last_undegraded":"2024-09-20T18:58:20.730707+0000","last_fullsized":"2024-09-20T18:58:20.730707+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:42.182809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:42.182809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"236'14550","reported_seq":26984,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:15.729520+0000","last_change":"2024-09-20T18:57:00.083015+0000","last_active":"2024-09-20T18:58:15.729520+0000","last_peered":"2024-09-20T18:58:15.729520+0000","last_clean":"2024-09-20T18:58:15.729520+0000","last_became_active":"2024-09-20T18:57:00.060960+0000","last_became_peered":"2024-09-20T18:57:00.060960+0000","last_unstale":"2024-09-20T18:58:15.729520+0000","last_undegraded":"2024-09-20T18:58:15.729520+0000","last_fullsized":"2024-09-20T18:58:15.729520+0000","mapping_epoch":213,"log_start":"215'10261","ondisk_log_start":"215'10261","created":160,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4289,"log_dups_size":0,"ondisk_log_size":4289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:34.055353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14550,"num_write_kb":465600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6125,"num_bytes_recovered":401408000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":405,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:16.351379+0000","last_change":"2024-09-20T18:54:54.897991+0000","last_active":"2024-09-20T18:58:16.351379+0000","last_peered":"2024-09-20T18:58:16.351379+0000","last_clean":"2024-09-20T18:58:16.351379+0000","last_became_active":"2024-09-20T18:49:47.946008+0000","last_became_peered":"2024-09-20T18:49:47.946008+0000","last_unstale":"2024-09-20T18:58:16.351379+0000","last_undegraded":"2024-09-20T18:58:16.351379+0000","last_fullsized":"2024-09-20T18:58:16.351379+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:54.897933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:54.897933+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":408,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:16.442723+0000","last_change":"2024-09-20T18:57:56.405392+0000","last_active":"2024-09-20T18:58:16.442723+0000","last_peered":"2024-09-20T18:58:16.442723+0000","last_clean":"2024-09-20T18:58:16.442723+0000","last_became_active":"2024-09-20T18:55:28.833095+0000","last_became_peered":"2024-09-20T18:55:28.833095+0000","last_unstale":"2024-09-20T18:58:16.442723+0000","last_undegraded":"2024-09-20T18:58:16.442723+0000","last_fullsized":"2024-09-20T18:58:16.442723+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:56.405349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:56.405349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:22.442121+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":385,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-20T18:58:13.253632+0000","last_change":"2024-09-20T18:54:43.379898+0000","last_active":"2024-09-20T18:58:13.253632+0000","last_peered":"2024-09-20T18:58:13.253632+0000","last_clean":"2024-09-20T18:58:13.253632+0000","last_became_active":"2024-09-20T18:49:49.956767+0000","last_became_peered":"2024-09-20T18:49:49.956767+0000","last_unstale":"2024-09-20T18:58:13.253632+0000","last_undegraded":"2024-09-20T18:58:13.253632+0000","last_fullsized":"2024-09-20T18:58:13.253632+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:43.379706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:43.379706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":373,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:16.350683+0000","last_change":"2024-09-20T18:54:24.976627+0000","last_active":"2024-09-20T18:58:16.350683+0000","last_peered":"2024-09-20T18:58:16.350683+0000","last_clean":"2024-09-20T18:58:16.350683+0000","last_became_active":"2024-09-20T18:49:51.964098+0000","last_became_peered":"2024-09-20T18:49:51.964098+0000","last_unstale":"2024-09-20T18:58:16.350683+0000","last_undegraded":"2024-09-20T18:58:16.350683+0000","last_fullsized":"2024-09-20T18:58:16.350683+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:24.976584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:24.976584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"236'14448","reported_seq":23025,"reported_epoch":237,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-20T18:58:17.121168+0000","last_change":"2024-09-20T18:56:27.871592+0000","last_active":"2024-09-20T18:58:17.121168+0000","last_peered":"2024-09-20T18:58:17.121168+0000","last_clean":"2024-09-20T18:55:27.791097+0000","last_became_active":"2024-09-20T18:55:29.979932+0000","last_became_peered":"2024-09-20T18:55:29.979932+0000","last_unstale":"2024-09-20T18:58:17.121168+0000","last_undegraded":"2024-09-20T18:56:27.867865+0000","last_fullsized":"2024-09-20T18:55:29.815116+0000","mapping_epoch":182,"log_start":"211'9540","ondisk_log_start":"211'9540","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":4908,"log_dups_size":0,"ondisk_log_size":4908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":974,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14448,"num_write_kb":462336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4204,"num_bytes_recovered":175964160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7],"avail_no_missing":["0","6","7"],"object_location_counts":[{"shards":"0,7","objects":-974}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":203,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:15.731022+0000","last_change":"2024-09-20T18:57:22.033602+0000","last_active":"2024-09-20T18:58:15.731022+0000","last_peered":"2024-09-20T18:58:15.731022+0000","last_clean":"2024-09-20T18:58:15.731022+0000","last_became_active":"2024-09-20T18:55:28.828563+0000","last_became_peered":"2024-09-20T18:55:28.828563+0000","last_unstale":"2024-09-20T18:58:15.731022+0000","last_undegraded":"2024-09-20T18:58:15.731022+0000","last_fullsized":"2024-09-20T18:58:15.731022+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-20T18:57:22.033475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:22.033475+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:43.998101+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":31,"num_read_kb":20,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,5],"acting":[1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":404,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-20T18:58:20.750427+0000","last_change":"2024-09-20T18:55:28.835767+0000","last_active":"2024-09-20T18:58:20.750427+0000","last_peered":"2024-09-20T18:58:20.750427+0000","last_clean":"2024-09-20T18:58:20.750427+0000","last_became_active":"2024-09-20T18:55:28.835149+0000","last_became_peered":"2024-09-20T18:55:28.835149+0000","last_unstale":"2024-09-20T18:58:20.750427+0000","last_undegraded":"2024-09-20T18:58:20.750427+0000","last_fullsized":"2024-09-20T18:58:20.750427+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:44.501192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.501192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":385,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-20T18:58:13.255477+0000","last_change":"2024-09-20T18:55:13.577772+0000","last_active":"2024-09-20T18:58:13.255477+0000","last_peered":"2024-09-20T18:58:13.255477+0000","last_clean":"2024-09-20T18:58:13.255477+0000","last_became_active":"2024-09-20T18:49:49.957558+0000","last_became_peered":"2024-09-20T18:49:49.957558+0000","last_unstale":"2024-09-20T18:58:13.255477+0000","last_undegraded":"2024-09-20T18:58:13.255477+0000","last_fullsized":"2024-09-20T18:58:13.255477+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:13.577699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:13.577699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":385,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-20T18:58:20.735061+0000","last_change":"2024-09-20T18:53:47.370152+0000","last_active":"2024-09-20T18:58:20.735061+0000","last_peered":"2024-09-20T18:58:20.735061+0000","last_clean":"2024-09-20T18:58:20.735061+0000","last_became_active":"2024-09-20T18:49:51.970798+0000","last_became_peered":"2024-09-20T18:49:51.970798+0000","last_unstale":"2024-09-20T18:58:20.735061+0000","last_undegraded":"2024-09-20T18:58:20.735061+0000","last_fullsized":"2024-09-20T18:58:20.735061+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:47.370108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:47.370108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"236'14986","reported_seq":25964,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-20T18:58:20.730800+0000","last_change":"2024-09-20T18:58:07.953371+0000","last_active":"2024-09-20T18:58:20.730800+0000","last_peered":"2024-09-20T18:58:20.730800+0000","last_clean":"2024-09-20T18:58:20.730800+0000","last_became_active":"2024-09-20T18:57:20.718369+0000","last_became_peered":"2024-09-20T18:57:20.718369+0000","last_unstale":"2024-09-20T18:58:20.730800+0000","last_undegraded":"2024-09-20T18:58:20.730800+0000","last_fullsized":"2024-09-20T18:58:20.730800+0000","mapping_epoch":221,"log_start":"214'10346","ondisk_log_start":"214'10346","created":160,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"233'14450","last_scrub_stamp":"2024-09-20T18:58:07.953275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:58:07.953275+0000","objects_scrubbed":542,"log_size":4640,"log_dups_size":0,"ondisk_log_size":4640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:22.081382+0000","scrub_duration":135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14986,"num_write_kb":479552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5391,"num_bytes_recovered":353304576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":409,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-20T18:58:20.730164+0000","last_change":"2024-09-20T18:53:52.898818+0000","last_active":"2024-09-20T18:58:20.730164+0000","last_peered":"2024-09-20T18:58:20.730164+0000","last_clean":"2024-09-20T18:58:20.730164+0000","last_became_active":"2024-09-20T18:49:47.949116+0000","last_became_peered":"2024-09-20T18:49:47.949116+0000","last_unstale":"2024-09-20T18:58:20.730164+0000","last_undegraded":"2024-09-20T18:58:20.730164+0000","last_fullsized":"2024-09-20T18:58:20.730164+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:53:52.898760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:52.898760+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:42.918366+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":397,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-20T18:58:20.750132+0000","last_change":"2024-09-20T18:54:52.650798+0000","last_active":"2024-09-20T18:58:20.750132+0000","last_peered":"2024-09-20T18:58:20.750132+0000","last_clean":"2024-09-20T18:58:20.750132+0000","last_became_active":"2024-09-20T18:49:45.944858+0000","last_became_peered":"2024-09-20T18:49:45.944858+0000","last_unstale":"2024-09-20T18:58:20.750132+0000","last_undegraded":"2024-09-20T18:58:20.750132+0000","last_fullsized":"2024-09-20T18:58:20.750132+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:52.650756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:52.650756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":377,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-20T18:58:13.253251+0000","last_change":"2024-09-20T18:54:42.349879+0000","last_active":"2024-09-20T18:58:13.253251+0000","last_peered":"2024-09-20T18:58:13.253251+0000","last_clean":"2024-09-20T18:58:13.253251+0000","last_became_active":"2024-09-20T18:49:49.954944+0000","last_became_peered":"2024-09-20T18:49:49.954944+0000","last_unstale":"2024-09-20T18:58:13.253251+0000","last_undegraded":"2024-09-20T18:58:13.253251+0000","last_fullsized":"2024-09-20T18:58:13.253251+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:42.349807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:42.349807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":374,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:15.731708+0000","last_change":"2024-09-20T18:55:17.341862+0000","last_active":"2024-09-20T18:58:15.731708+0000","last_peered":"2024-09-20T18:58:15.731708+0000","last_clean":"2024-09-20T18:58:15.731708+0000","last_became_active":"2024-09-20T18:49:51.972970+0000","last_became_peered":"2024-09-20T18:49:51.972970+0000","last_unstale":"2024-09-20T18:58:15.731708+0000","last_undegraded":"2024-09-20T18:58:15.731708+0000","last_fullsized":"2024-09-20T18:58:15.731708+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:17.341793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:17.341793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.4","version":"236'14332","reported_seq":14498,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:16.442340+0000","last_change":"2024-09-20T18:54:34.309442+0000","last_active":"2024-09-20T18:58:16.442340+0000","last_peered":"2024-09-20T18:58:16.442340+0000","last_clean":"2024-09-20T18:58:16.442340+0000","last_became_active":"2024-09-20T18:54:34.309113+0000","last_became_peered":"2024-09-20T18:54:34.309113+0000","last_unstale":"2024-09-20T18:58:16.442340+0000","last_undegraded":"2024-09-20T18:58:16.442340+0000","last_fullsized":"2024-09-20T18:58:16.442340+0000","mapping_epoch":160,"log_start":"211'9331","ondisk_log_start":"211'9331","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:54:32.911954+0000","last_clean_scrub_stamp":"2024-09-20T18:54:32.911954+0000","objects_scrubbed":0,"log_size":5001,"log_dups_size":0,"ondisk_log_size":5001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:40.627488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14332,"num_write_kb":458624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":429,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:16.442058+0000","last_change":"2024-09-20T18:55:28.847010+0000","last_active":"2024-09-20T18:58:16.442058+0000","last_peered":"2024-09-20T18:58:16.442058+0000","last_clean":"2024-09-20T18:58:16.442058+0000","last_became_active":"2024-09-20T18:55:28.824801+0000","last_became_peered":"2024-09-20T18:55:28.824801+0000","last_unstale":"2024-09-20T18:58:16.442058+0000","last_undegraded":"2024-09-20T18:58:16.442058+0000","last_fullsized":"2024-09-20T18:58:16.442058+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:54:12.373309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:12.373309+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:29.516079+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":417,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-20T18:58:20.750081+0000","last_change":"2024-09-20T18:55:24.729301+0000","last_active":"2024-09-20T18:58:20.750081+0000","last_peered":"2024-09-20T18:58:20.750081+0000","last_clean":"2024-09-20T18:58:20.750081+0000","last_became_active":"2024-09-20T18:49:45.944256+0000","last_became_peered":"2024-09-20T18:49:45.944256+0000","last_unstale":"2024-09-20T18:58:20.750081+0000","last_undegraded":"2024-09-20T18:58:20.750081+0000","last_fullsized":"2024-09-20T18:58:20.750081+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-20T18:55:24.729219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:24.729219+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"184'9","reported_seq":495,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-20T18:58:20.734830+0000","last_change":"2024-09-20T18:55:28.824771+0000","last_active":"2024-09-20T18:58:20.734830+0000","last_peered":"2024-09-20T18:58:20.734830+0000","last_clean":"2024-09-20T18:58:20.734830+0000","last_became_active":"2024-09-20T18:55:28.818085+0000","last_became_peered":"2024-09-20T18:55:28.818085+0000","last_unstale":"2024-09-20T18:58:20.734830+0000","last_undegraded":"2024-09-20T18:58:20.734830+0000","last_fullsized":"2024-09-20T18:58:20.734830+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:52:05.788924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.788924+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":388,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:16.442177+0000","last_change":"2024-09-20T18:58:05.523446+0000","last_active":"2024-09-20T18:58:16.442177+0000","last_peered":"2024-09-20T18:58:16.442177+0000","last_clean":"2024-09-20T18:58:16.442177+0000","last_became_active":"2024-09-20T18:55:28.830380+0000","last_became_peered":"2024-09-20T18:55:28.830380+0000","last_unstale":"2024-09-20T18:58:16.442177+0000","last_undegraded":"2024-09-20T18:58:16.442177+0000","last_fullsized":"2024-09-20T18:58:16.442177+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:05.523359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:05.523359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:28.858918+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":124,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:16.351016+0000","last_change":"2024-09-20T18:55:28.835344+0000","last_active":"2024-09-20T18:58:16.351016+0000","last_peered":"2024-09-20T18:58:16.351016+0000","last_clean":"2024-09-20T18:58:16.351016+0000","last_became_active":"2024-09-20T18:55:28.835010+0000","last_became_peered":"2024-09-20T18:55:28.835010+0000","last_unstale":"2024-09-20T18:58:16.351016+0000","last_undegraded":"2024-09-20T18:58:16.351016+0000","last_fullsized":"2024-09-20T18:58:16.351016+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:26.721735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:26.721735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":399,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:15.731641+0000","last_change":"2024-09-20T18:58:09.063555+0000","last_active":"2024-09-20T18:58:15.731641+0000","last_peered":"2024-09-20T18:58:15.731641+0000","last_clean":"2024-09-20T18:58:15.731641+0000","last_became_active":"2024-09-20T18:49:48.500662+0000","last_became_peered":"2024-09-20T18:49:48.500662+0000","last_unstale":"2024-09-20T18:58:15.731641+0000","last_undegraded":"2024-09-20T18:58:15.731641+0000","last_fullsized":"2024-09-20T18:58:15.731641+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-20T18:58:09.063481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:09.063481+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:21.861420+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":377,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:16.442698+0000","last_change":"2024-09-20T18:53:16.220554+0000","last_active":"2024-09-20T18:58:16.442698+0000","last_peered":"2024-09-20T18:58:16.442698+0000","last_clean":"2024-09-20T18:58:16.442698+0000","last_became_active":"2024-09-20T18:49:49.956749+0000","last_became_peered":"2024-09-20T18:49:49.956749+0000","last_unstale":"2024-09-20T18:58:16.442698+0000","last_undegraded":"2024-09-20T18:58:16.442698+0000","last_fullsized":"2024-09-20T18:58:16.442698+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:16.220490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:16.220490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:27.545117+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":392,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-20T18:58:20.750134+0000","last_change":"2024-09-20T18:55:28.827897+0000","last_active":"2024-09-20T18:58:20.750134+0000","last_peered":"2024-09-20T18:58:20.750134+0000","last_clean":"2024-09-20T18:58:20.750134+0000","last_became_active":"2024-09-20T18:55:28.827418+0000","last_became_peered":"2024-09-20T18:55:28.827418+0000","last_unstale":"2024-09-20T18:58:20.750134+0000","last_undegraded":"2024-09-20T18:58:20.750134+0000","last_fullsized":"2024-09-20T18:58:20.750134+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.637739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.637739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":397,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:16.351925+0000","last_change":"2024-09-20T18:54:44.830872+0000","last_active":"2024-09-20T18:58:16.351925+0000","last_peered":"2024-09-20T18:58:16.351925+0000","last_clean":"2024-09-20T18:58:16.351925+0000","last_became_active":"2024-09-20T18:49:48.500803+0000","last_became_peered":"2024-09-20T18:49:48.500803+0000","last_unstale":"2024-09-20T18:58:16.351925+0000","last_undegraded":"2024-09-20T18:58:16.351925+0000","last_fullsized":"2024-09-20T18:58:16.351925+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:44.830790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.830790+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":396,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-20T18:58:20.734977+0000","last_change":"2024-09-20T18:55:29.751718+0000","last_active":"2024-09-20T18:58:20.734977+0000","last_peered":"2024-09-20T18:58:20.734977+0000","last_clean":"2024-09-20T18:58:20.734977+0000","last_became_active":"2024-09-20T18:55:28.833799+0000","last_became_peered":"2024-09-20T18:55:28.833799+0000","last_unstale":"2024-09-20T18:58:20.734977+0000","last_undegraded":"2024-09-20T18:58:20.734977+0000","last_fullsized":"2024-09-20T18:58:20.734977+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:29.751642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:29.751642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":386,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:15.731497+0000","last_change":"2024-09-20T18:57:46.898929+0000","last_active":"2024-09-20T18:58:15.731497+0000","last_peered":"2024-09-20T18:58:15.731497+0000","last_clean":"2024-09-20T18:58:15.731497+0000","last_became_active":"2024-09-20T18:49:49.961947+0000","last_became_peered":"2024-09-20T18:49:49.961947+0000","last_unstale":"2024-09-20T18:58:15.731497+0000","last_undegraded":"2024-09-20T18:58:15.731497+0000","last_fullsized":"2024-09-20T18:58:15.731497+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:46.898862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:46.898862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:06.640666+0000","scrub_duration":38,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"66'1","reported_seq":382,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-20T18:58:13.254369+0000","last_change":"2024-09-20T18:54:20.681878+0000","last_active":"2024-09-20T18:58:13.254369+0000","last_peered":"2024-09-20T18:58:13.254369+0000","last_clean":"2024-09-20T18:58:13.254369+0000","last_became_active":"2024-09-20T18:49:51.965121+0000","last_became_peered":"2024-09-20T18:49:51.965121+0000","last_unstale":"2024-09-20T18:58:13.254369+0000","last_undegraded":"2024-09-20T18:58:13.254369+0000","last_fullsized":"2024-09-20T18:58:13.254369+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T18:54:20.681790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:20.681790+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":397,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:16.442796+0000","last_change":"2024-09-20T18:57:59.430353+0000","last_active":"2024-09-20T18:58:16.442796+0000","last_peered":"2024-09-20T18:58:16.442796+0000","last_clean":"2024-09-20T18:58:16.442796+0000","last_became_active":"2024-09-20T18:49:48.501011+0000","last_became_peered":"2024-09-20T18:49:48.501011+0000","last_unstale":"2024-09-20T18:58:16.442796+0000","last_undegraded":"2024-09-20T18:58:16.442796+0000","last_fullsized":"2024-09-20T18:58:16.442796+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:57:59.430294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:59.430294+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:21.333603+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":393,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:16.351346+0000","last_change":"2024-09-20T18:55:27.028184+0000","last_active":"2024-09-20T18:58:16.351346+0000","last_peered":"2024-09-20T18:58:16.351346+0000","last_clean":"2024-09-20T18:58:16.351346+0000","last_became_active":"2024-09-20T18:49:45.938511+0000","last_became_peered":"2024-09-20T18:49:45.938511+0000","last_unstale":"2024-09-20T18:58:16.351346+0000","last_undegraded":"2024-09-20T18:58:16.351346+0000","last_fullsized":"2024-09-20T18:58:16.351346+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:27.028126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:27.028126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":497,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:15.729337+0000","last_change":"2024-09-20T18:55:27.284156+0000","last_active":"2024-09-20T18:58:15.729337+0000","last_peered":"2024-09-20T18:58:15.729337+0000","last_clean":"2024-09-20T18:58:15.729337+0000","last_became_active":"2024-09-20T18:49:49.960550+0000","last_became_peered":"2024-09-20T18:49:49.960550+0000","last_unstale":"2024-09-20T18:58:15.729337+0000","last_undegraded":"2024-09-20T18:58:15.729337+0000","last_fullsized":"2024-09-20T18:58:15.729337+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:27.284086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:27.284086+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":386,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-20T18:58:20.730902+0000","last_change":"2024-09-20T18:55:12.348079+0000","last_active":"2024-09-20T18:58:20.730902+0000","last_peered":"2024-09-20T18:58:20.730902+0000","last_clean":"2024-09-20T18:58:20.730902+0000","last_became_active":"2024-09-20T18:49:51.966254+0000","last_became_peered":"2024-09-20T18:49:51.966254+0000","last_unstale":"2024-09-20T18:58:20.730902+0000","last_undegraded":"2024-09-20T18:58:20.730902+0000","last_fullsized":"2024-09-20T18:58:20.730902+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:12.347993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:12.347993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":416,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:15.729498+0000","last_change":"2024-09-20T18:57:20.701592+0000","last_active":"2024-09-20T18:58:15.729498+0000","last_peered":"2024-09-20T18:58:15.729498+0000","last_clean":"2024-09-20T18:58:15.729498+0000","last_became_active":"2024-09-20T18:49:58.997494+0000","last_became_peered":"2024-09-20T18:49:58.997494+0000","last_unstale":"2024-09-20T18:58:15.729498+0000","last_undegraded":"2024-09-20T18:58:15.729498+0000","last_fullsized":"2024-09-20T18:58:15.729498+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:57:20.701543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:20.701543+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:44.352173+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":397,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-20T18:58:20.735062+0000","last_change":"2024-09-20T18:54:57.632458+0000","last_active":"2024-09-20T18:58:20.735062+0000","last_peered":"2024-09-20T18:58:20.735062+0000","last_clean":"2024-09-20T18:58:20.735062+0000","last_became_active":"2024-09-20T18:49:45.949492+0000","last_became_peered":"2024-09-20T18:49:45.949492+0000","last_unstale":"2024-09-20T18:58:20.735062+0000","last_undegraded":"2024-09-20T18:58:20.735062+0000","last_fullsized":"2024-09-20T18:58:20.735062+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.632380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.632380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":496,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:16.442542+0000","last_change":"2024-09-20T18:55:28.302353+0000","last_active":"2024-09-20T18:58:16.442542+0000","last_peered":"2024-09-20T18:58:16.442542+0000","last_clean":"2024-09-20T18:58:16.442542+0000","last_became_active":"2024-09-20T18:49:49.962243+0000","last_became_peered":"2024-09-20T18:49:49.962243+0000","last_unstale":"2024-09-20T18:58:16.442542+0000","last_undegraded":"2024-09-20T18:58:16.442542+0000","last_fullsized":"2024-09-20T18:58:16.442542+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:28.302303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:28.302303+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":381,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:16.350813+0000","last_change":"2024-09-20T18:55:11.918284+0000","last_active":"2024-09-20T18:58:16.350813+0000","last_peered":"2024-09-20T18:58:16.350813+0000","last_clean":"2024-09-20T18:58:16.350813+0000","last_became_active":"2024-09-20T18:49:51.969093+0000","last_became_peered":"2024-09-20T18:49:51.969093+0000","last_unstale":"2024-09-20T18:58:16.350813+0000","last_undegraded":"2024-09-20T18:58:16.350813+0000","last_fullsized":"2024-09-20T18:58:16.350813+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:11.918210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:11.918210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":432,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-20T18:58:20.730670+0000","last_change":"2024-09-20T18:57:50.899576+0000","last_active":"2024-09-20T18:58:20.730670+0000","last_peered":"2024-09-20T18:58:20.730670+0000","last_clean":"2024-09-20T18:58:20.730670+0000","last_became_active":"2024-09-20T18:55:28.826721+0000","last_became_peered":"2024-09-20T18:55:28.826721+0000","last_unstale":"2024-09-20T18:58:20.730670+0000","last_undegraded":"2024-09-20T18:58:20.730670+0000","last_fullsized":"2024-09-20T18:58:20.730670+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:57:50.899514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:50.899514+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:12.210010+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":389,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-20T18:58:20.730976+0000","last_change":"2024-09-20T18:55:28.822690+0000","last_active":"2024-09-20T18:58:20.730976+0000","last_peered":"2024-09-20T18:58:20.730976+0000","last_clean":"2024-09-20T18:58:20.730976+0000","last_became_active":"2024-09-20T18:55:28.822258+0000","last_became_peered":"2024-09-20T18:55:28.822258+0000","last_unstale":"2024-09-20T18:58:20.730976+0000","last_undegraded":"2024-09-20T18:58:20.730976+0000","last_fullsized":"2024-09-20T18:58:20.730976+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:05.985551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.985551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:37.926823+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":378,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:15.730931+0000","last_change":"2024-09-20T18:57:11.986605+0000","last_active":"2024-09-20T18:58:15.730931+0000","last_peered":"2024-09-20T18:58:15.730931+0000","last_clean":"2024-09-20T18:58:15.730931+0000","last_became_active":"2024-09-20T18:49:49.957067+0000","last_became_peered":"2024-09-20T18:49:49.957067+0000","last_unstale":"2024-09-20T18:58:15.730931+0000","last_undegraded":"2024-09-20T18:58:15.730931+0000","last_fullsized":"2024-09-20T18:58:15.730931+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:11.986528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:11.986528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:36.007001+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":374,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:15.729990+0000","last_change":"2024-09-20T18:57:16.789412+0000","last_active":"2024-09-20T18:58:15.729990+0000","last_peered":"2024-09-20T18:58:15.729990+0000","last_clean":"2024-09-20T18:58:15.729990+0000","last_became_active":"2024-09-20T18:49:51.966915+0000","last_became_peered":"2024-09-20T18:49:51.966915+0000","last_unstale":"2024-09-20T18:58:15.729990+0000","last_undegraded":"2024-09-20T18:58:15.729990+0000","last_fullsized":"2024-09-20T18:58:15.729990+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:16.789095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:16.789095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:45.821909+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":423,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:15.731120+0000","last_change":"2024-09-20T18:55:28.838698+0000","last_active":"2024-09-20T18:58:15.731120+0000","last_peered":"2024-09-20T18:58:15.731120+0000","last_clean":"2024-09-20T18:58:15.731120+0000","last_became_active":"2024-09-20T18:55:28.827023+0000","last_became_peered":"2024-09-20T18:55:28.827023+0000","last_unstale":"2024-09-20T18:58:15.731120+0000","last_undegraded":"2024-09-20T18:58:15.731120+0000","last_fullsized":"2024-09-20T18:58:15.731120+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:53:53.263976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:53.263976+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:26.131891+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":389,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-20T18:58:20.750326+0000","last_change":"2024-09-20T18:54:42.487465+0000","last_active":"2024-09-20T18:58:20.750326+0000","last_peered":"2024-09-20T18:58:20.750326+0000","last_clean":"2024-09-20T18:58:20.750326+0000","last_became_active":"2024-09-20T18:49:45.944357+0000","last_became_peered":"2024-09-20T18:49:45.944357+0000","last_unstale":"2024-09-20T18:58:20.750326+0000","last_undegraded":"2024-09-20T18:58:20.750326+0000","last_fullsized":"2024-09-20T18:58:20.750326+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:42.487403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:42.487403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":386,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:15.731748+0000","last_change":"2024-09-20T18:57:57.915090+0000","last_active":"2024-09-20T18:58:15.731748+0000","last_peered":"2024-09-20T18:58:15.731748+0000","last_clean":"2024-09-20T18:58:15.731748+0000","last_became_active":"2024-09-20T18:49:49.962523+0000","last_became_peered":"2024-09-20T18:49:49.962523+0000","last_unstale":"2024-09-20T18:58:15.731748+0000","last_undegraded":"2024-09-20T18:58:15.731748+0000","last_fullsized":"2024-09-20T18:58:15.731748+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:57.914913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:57.914913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:01.797575+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":381,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:16.350760+0000","last_change":"2024-09-20T18:54:26.882441+0000","last_active":"2024-09-20T18:58:16.350760+0000","last_peered":"2024-09-20T18:58:16.350760+0000","last_clean":"2024-09-20T18:58:16.350760+0000","last_became_active":"2024-09-20T18:49:51.967313+0000","last_became_peered":"2024-09-20T18:49:51.967313+0000","last_unstale":"2024-09-20T18:58:16.350760+0000","last_undegraded":"2024-09-20T18:58:16.350760+0000","last_fullsized":"2024-09-20T18:58:16.350760+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:26.882375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:26.882375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":428,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:15.730127+0000","last_change":"2024-09-20T18:57:54.862785+0000","last_active":"2024-09-20T18:58:15.730127+0000","last_peered":"2024-09-20T18:58:15.730127+0000","last_clean":"2024-09-20T18:58:15.730127+0000","last_became_active":"2024-09-20T18:55:28.830801+0000","last_became_peered":"2024-09-20T18:55:28.830801+0000","last_unstale":"2024-09-20T18:58:15.730127+0000","last_undegraded":"2024-09-20T18:58:15.730127+0000","last_fullsized":"2024-09-20T18:58:15.730127+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:57:54.862329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:54.862329+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:09.269532+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":401,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:16.442727+0000","last_change":"2024-09-20T18:58:06.479419+0000","last_active":"2024-09-20T18:58:16.442727+0000","last_peered":"2024-09-20T18:58:16.442727+0000","last_clean":"2024-09-20T18:58:16.442727+0000","last_became_active":"2024-09-20T18:49:45.942105+0000","last_became_peered":"2024-09-20T18:49:45.942105+0000","last_unstale":"2024-09-20T18:58:16.442727+0000","last_undegraded":"2024-09-20T18:58:16.442727+0000","last_fullsized":"2024-09-20T18:58:16.442727+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:06.479364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:06.479364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:26.337228+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":396,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-20T18:58:20.734594+0000","last_change":"2024-09-20T18:55:28.825004+0000","last_active":"2024-09-20T18:58:20.734594+0000","last_peered":"2024-09-20T18:58:20.734594+0000","last_clean":"2024-09-20T18:58:20.734594+0000","last_became_active":"2024-09-20T18:55:28.824355+0000","last_became_peered":"2024-09-20T18:55:28.824355+0000","last_unstale":"2024-09-20T18:58:20.734594+0000","last_undegraded":"2024-09-20T18:58:20.734594+0000","last_fullsized":"2024-09-20T18:58:20.734594+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:46.329547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:46.329547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":373,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:15.731831+0000","last_change":"2024-09-20T18:55:28.825607+0000","last_active":"2024-09-20T18:58:15.731831+0000","last_peered":"2024-09-20T18:58:15.731831+0000","last_clean":"2024-09-20T18:58:15.731831+0000","last_became_active":"2024-09-20T18:55:28.825296+0000","last_became_peered":"2024-09-20T18:55:28.825296+0000","last_unstale":"2024-09-20T18:58:15.731831+0000","last_undegraded":"2024-09-20T18:58:15.731831+0000","last_fullsized":"2024-09-20T18:58:15.731831+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:43.118805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:43.118805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:25.734066+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":409,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:15.730252+0000","last_change":"2024-09-20T18:55:28.836789+0000","last_active":"2024-09-20T18:58:15.730252+0000","last_peered":"2024-09-20T18:58:15.730252+0000","last_clean":"2024-09-20T18:58:15.730252+0000","last_became_active":"2024-09-20T18:55:28.824579+0000","last_became_peered":"2024-09-20T18:55:28.824579+0000","last_unstale":"2024-09-20T18:58:15.730252+0000","last_undegraded":"2024-09-20T18:58:15.730252+0000","last_fullsized":"2024-09-20T18:58:15.730252+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-20T18:55:20.282193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:20.282193+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":393,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:16.351807+0000","last_change":"2024-09-20T18:54:55.900487+0000","last_active":"2024-09-20T18:58:16.351807+0000","last_peered":"2024-09-20T18:58:16.351807+0000","last_clean":"2024-09-20T18:58:16.351807+0000","last_became_active":"2024-09-20T18:49:45.940988+0000","last_became_peered":"2024-09-20T18:49:45.940988+0000","last_unstale":"2024-09-20T18:58:16.351807+0000","last_undegraded":"2024-09-20T18:58:16.351807+0000","last_fullsized":"2024-09-20T18:58:16.351807+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:55.899611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:55.899611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":136,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-20T18:58:20.729912+0000","last_change":"2024-09-20T18:57:52.929891+0000","last_active":"2024-09-20T18:58:20.729912+0000","last_peered":"2024-09-20T18:58:20.729912+0000","last_clean":"2024-09-20T18:58:20.729912+0000","last_became_active":"2024-09-20T18:55:28.832534+0000","last_became_peered":"2024-09-20T18:55:28.832534+0000","last_unstale":"2024-09-20T18:58:20.729912+0000","last_undegraded":"2024-09-20T18:58:20.729912+0000","last_fullsized":"2024-09-20T18:58:20.729912+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:52.929828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:52.929828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:13.316360+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":369,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-20T18:58:20.734703+0000","last_change":"2024-09-20T18:51:09.761331+0000","last_active":"2024-09-20T18:58:20.734703+0000","last_peered":"2024-09-20T18:58:20.734703+0000","last_clean":"2024-09-20T18:58:20.734703+0000","last_became_active":"2024-09-20T18:49:51.968662+0000","last_became_peered":"2024-09-20T18:49:51.968662+0000","last_unstale":"2024-09-20T18:58:20.734703+0000","last_undegraded":"2024-09-20T18:58:20.734703+0000","last_fullsized":"2024-09-20T18:58:20.734703+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:09.761283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:51:09.761283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":366,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:15.731273+0000","last_change":"2024-09-20T18:57:10.970843+0000","last_active":"2024-09-20T18:58:15.731273+0000","last_peered":"2024-09-20T18:58:15.731273+0000","last_clean":"2024-09-20T18:58:15.731273+0000","last_became_active":"2024-09-20T18:49:51.968216+0000","last_became_peered":"2024-09-20T18:49:51.968216+0000","last_unstale":"2024-09-20T18:58:15.731273+0000","last_undegraded":"2024-09-20T18:58:15.731273+0000","last_fullsized":"2024-09-20T18:58:15.731273+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:10.970779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:10.970779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:38.733512+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":386,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:15.730472+0000","last_change":"2024-09-20T18:55:28.295445+0000","last_active":"2024-09-20T18:58:15.730472+0000","last_peered":"2024-09-20T18:58:15.730472+0000","last_clean":"2024-09-20T18:58:15.730472+0000","last_became_active":"2024-09-20T18:49:45.937992+0000","last_became_peered":"2024-09-20T18:49:45.937992+0000","last_unstale":"2024-09-20T18:58:15.730472+0000","last_undegraded":"2024-09-20T18:58:15.730472+0000","last_fullsized":"2024-09-20T18:58:15.730472+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:28.295399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:28.295399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":175,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:16.442902+0000","last_change":"2024-09-20T18:55:28.852468+0000","last_active":"2024-09-20T18:58:16.442902+0000","last_peered":"2024-09-20T18:58:16.442902+0000","last_clean":"2024-09-20T18:58:16.442902+0000","last_became_active":"2024-09-20T18:55:28.834675+0000","last_became_peered":"2024-09-20T18:55:28.834675+0000","last_unstale":"2024-09-20T18:58:16.442902+0000","last_undegraded":"2024-09-20T18:58:16.442902+0000","last_fullsized":"2024-09-20T18:58:16.442902+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:54:44.719676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.719676+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:38.535671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":12,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":370,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-20T18:58:16.351694+0000","last_change":"2024-09-20T18:51:28.120100+0000","last_active":"2024-09-20T18:58:16.351694+0000","last_peered":"2024-09-20T18:58:16.351694+0000","last_clean":"2024-09-20T18:58:16.351694+0000","last_became_active":"2024-09-20T18:49:49.957582+0000","last_became_peered":"2024-09-20T18:49:49.957582+0000","last_unstale":"2024-09-20T18:58:16.351694+0000","last_undegraded":"2024-09-20T18:58:16.351694+0000","last_fullsized":"2024-09-20T18:58:16.351694+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:28.119694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:28.119694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":974,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":232178,"num_write_kb":7429633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27082,"num_bytes_recovered":1675296768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1116798976,"data_stored":1116798976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74111,"ondisk_log_size":74111,"up":48,"acting":47,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":67,"ondisk_log_size":67,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":695,"num_read_kb":452,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":1474,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":12140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":394,"ondisk_log_size":394,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":5275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":509,"num_read_kb":504,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266795,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":213272,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":211646,"kb_avail":93531880,"statfs":{"total":95995035648,"available":95776645120,"internally_reserved":0,"allocated":1642496,"data_stored":1142356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":216725962},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":0,"seq":193273528434,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757367,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":279300,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":278270,"kb_avail":93465852,"statfs":{"total":95995035648,"available":95709032448,"internally_reserved":0,"allocated":1032192,"data_stored":545162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":284948933},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986300,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":208552,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":207486,"kb_avail":93536600,"statfs":{"total":95995035648,"available":95781478400,"internally_reserved":0,"allocated":1069056,"data_stored":553047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":212466117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182530,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":357084,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":356030,"kb_avail":93388068,"statfs":{"total":95995035648,"available":95629381632,"internally_reserved":0,"allocated":1056768,"data_stored":545148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":364575175},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411463,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":404144,"kb_used_data":3552,"kb_used_omap":1,"kb_used_meta":400446,"kb_avail":93341008,"statfs":{"total":95995035648,"available":95581192192,"internally_reserved":0,"allocated":3637248,"data_stored":3160022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":410057157},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574989,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":303540,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":301886,"kb_avail":93441612,"statfs":{"total":95995035648,"available":95684210688,"internally_reserved":0,"allocated":1671168,"data_stored":1137399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":309131717},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738515,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":361820,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":360190,"kb_avail":93383332,"statfs":{"total":95995035648,"available":95624531968,"internally_reserved":0,"allocated":1646592,"data_stored":1135592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":368835013},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1116798976,"data_stored":1116798976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T18:58:22.240 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:22.263 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:22.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:22.264 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:22.347 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T18:58:22.347 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:58:22.364 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:22.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:22.387 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:22.387 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:22.488 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:22.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:22 smithi083 ceph-mon[29421]: pgmap v411: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 583 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 715 op/s; 974/597 objects degraded (163.149%); 0 B/s, 104 objects/s recovering 2024-09-20T18:58:22.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:22 smithi083 ceph-mon[29421]: from='client.15534 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:58:22.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:22 smithi083 ceph-mon[29421]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:58:22.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:22 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/70674786' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T18:58:22.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:22 smithi083 ceph-mon[29421]: osdmap e240: 8 total, 8 up, 7 in 2024-09-20T18:58:22.507 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:22 smithi083 ceph-mon[36260]: pgmap v411: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 583 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 715 op/s; 974/597 objects degraded (163.149%); 0 B/s, 104 objects/s recovering 2024-09-20T18:58:22.507 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:22 smithi083 ceph-mon[36260]: from='client.15534 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:58:22.507 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:22 smithi083 ceph-mon[36260]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:58:22.507 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:22 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/70674786' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T18:58:22.507 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:22 smithi083 ceph-mon[36260]: osdmap e240: 8 total, 8 up, 7 in 2024-09-20T18:58:22.509 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:22.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:22.510 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:22.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:22 smithi142 ceph-mon[32508]: pgmap v411: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 147 active+clean; 583 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 715 op/s; 974/597 objects degraded (163.149%); 0 B/s, 104 objects/s recovering 2024-09-20T18:58:22.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:22 smithi142 ceph-mon[32508]: from='client.15534 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:58:22.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:22 smithi142 ceph-mon[32508]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:58:22.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:22 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/70674786' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T18:58:22.520 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:22 smithi142 ceph-mon[32508]: osdmap e240: 8 total, 8 up, 7 in 2024-09-20T18:58:22.611 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:22.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:22.634 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:22.635 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:22.735 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:22.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:22.760 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:22.760 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:22.860 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:22.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:22.882 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:22.882 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:22.919 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-20T18:58:22.983 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:23.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:23.005 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:23.005 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:23.106 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:23.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:23.132 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:23.132 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:23.169 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:58:23.233 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:23.255 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:23.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:23.256 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:23.352 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:23 smithi083 ceph-mon[29421]: 6.18 scrub starts 2024-09-20T18:58:23.352 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:23 smithi083 ceph-mon[29421]: 6.18 scrub ok 2024-09-20T18:58:23.352 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:23 smithi083 ceph-mon[29421]: 6.12 scrub starts 2024-09-20T18:58:23.352 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:23 smithi083 ceph-mon[29421]: 6.12 scrub ok 2024-09-20T18:58:23.352 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:23 smithi083 ceph-mon[36260]: 6.18 scrub starts 2024-09-20T18:58:23.352 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:23 smithi083 ceph-mon[36260]: 6.18 scrub ok 2024-09-20T18:58:23.353 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:23 smithi083 ceph-mon[36260]: 6.12 scrub starts 2024-09-20T18:58:23.353 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:23 smithi083 ceph-mon[36260]: 6.12 scrub ok 2024-09-20T18:58:23.357 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:23.379 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:23.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:23.380 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:23.481 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:23.486 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T18:58:23.502 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:23.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:23.502 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:23.554 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:23 smithi142 ceph-mon[32508]: 6.18 scrub starts 2024-09-20T18:58:23.554 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:23 smithi142 ceph-mon[32508]: 6.18 scrub ok 2024-09-20T18:58:23.554 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:23 smithi142 ceph-mon[32508]: 6.12 scrub starts 2024-09-20T18:58:23.554 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:23 smithi142 ceph-mon[32508]: 6.12 scrub ok 2024-09-20T18:58:23.604 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:23.625 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:23.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:23.626 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:23.655 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T18:58:23.655 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:58:23.726 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:23.749 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:23.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:23.750 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:23.758 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:58:23.851 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:23.872 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:23.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:23.873 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:23.974 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:23.994 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:23.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:23.995 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:24.096 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:24.118 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:24.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:24.119 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:24.220 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:24.241 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:24.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:24.242 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:24.284 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_blocked_ops 2024-09-20T18:58:24.310 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'unique_pool_1' removed 2024-09-20T18:58:24.343 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:24.389 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:24.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:24.390 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:24.490 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:24.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:24 smithi142 ceph-mon[32508]: pgmap v413: 148 pgs: 148 active+clean; 583 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 111 op/s; 0 B/s, 266 objects/s recovering 2024-09-20T18:58:24.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:24 smithi142 ceph-mon[32508]: 5.0 scrub starts 2024-09-20T18:58:24.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:24 smithi142 ceph-mon[32508]: 5.0 scrub ok 2024-09-20T18:58:24.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:24 smithi142 ceph-mon[32508]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 974/597 objects degraded (163.149%), 1 pg degraded, 1 pg undersized) 2024-09-20T18:58:24.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:24 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3192998830' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true} : dispatch 2024-09-20T18:58:24.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:24 smithi142 ceph-mon[32508]: 5.4 scrub starts 2024-09-20T18:58:24.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:24 smithi142 ceph-mon[32508]: 5.4 scrub ok 2024-09-20T18:58:24.520 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:24.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:24.521 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:24.598 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:58:24.622 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:24.643 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:24.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:24.644 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:24.744 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:24.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:24.766 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:24.767 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:24.798 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:24 smithi083 ceph-mon[29421]: pgmap v413: 148 pgs: 148 active+clean; 583 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 111 op/s; 0 B/s, 266 objects/s recovering 2024-09-20T18:58:24.799 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:24 smithi083 ceph-mon[29421]: 5.0 scrub starts 2024-09-20T18:58:24.799 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:24 smithi083 ceph-mon[29421]: 5.0 scrub ok 2024-09-20T18:58:24.799 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:24 smithi083 ceph-mon[29421]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 974/597 objects degraded (163.149%), 1 pg degraded, 1 pg undersized) 2024-09-20T18:58:24.799 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:24 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3192998830' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true} : dispatch 2024-09-20T18:58:24.799 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:24 smithi083 ceph-mon[29421]: 5.4 scrub starts 2024-09-20T18:58:24.799 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:24 smithi083 ceph-mon[29421]: 5.4 scrub ok 2024-09-20T18:58:24.799 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:24 smithi083 ceph-mon[36260]: pgmap v413: 148 pgs: 148 active+clean; 583 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 111 op/s; 0 B/s, 266 objects/s recovering 2024-09-20T18:58:24.799 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:24 smithi083 ceph-mon[36260]: 5.0 scrub starts 2024-09-20T18:58:24.799 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:24 smithi083 ceph-mon[36260]: 5.0 scrub ok 2024-09-20T18:58:24.800 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:24 smithi083 ceph-mon[36260]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 974/597 objects degraded (163.149%), 1 pg degraded, 1 pg undersized) 2024-09-20T18:58:24.800 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:24 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3192998830' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true} : dispatch 2024-09-20T18:58:24.800 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:24 smithi083 ceph-mon[36260]: 5.4 scrub starts 2024-09-20T18:58:24.800 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:24 smithi083 ceph-mon[36260]: 5.4 scrub ok 2024-09-20T18:58:24.852 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T18:58:24.867 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:24.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:24.920 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:24.920 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:25.021 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:25.044 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:25.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:25.045 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:25.145 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:25.168 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:25.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:25.169 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:25.211 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T18:58:25.269 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:25.290 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:25.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:25.291 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:25.392 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:25.412 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:25.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:25.413 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:25.512 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:25 smithi142 ceph-mon[32508]: 6.b scrub starts 2024-09-20T18:58:25.512 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:25 smithi142 ceph-mon[32508]: 6.b scrub ok 2024-09-20T18:58:25.512 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:25 smithi142 ceph-mon[32508]: 2.1 scrub starts 2024-09-20T18:58:25.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:25 smithi142 ceph-mon[32508]: 2.1 scrub ok 2024-09-20T18:58:25.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:25 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3192998830' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]': finished 2024-09-20T18:58:25.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:25 smithi142 ceph-mon[32508]: osdmap e241: 8 total, 8 up, 7 in 2024-09-20T18:58:25.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:25 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:58:25.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:25 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:58:25.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:25 smithi142 ceph-mon[32508]: 4.e scrub starts 2024-09-20T18:58:25.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:25 smithi142 ceph-mon[32508]: 4.e scrub ok 2024-09-20T18:58:25.514 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:25.535 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:25.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:25.537 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:25.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:25 smithi083 ceph-mon[29421]: 6.b scrub starts 2024-09-20T18:58:25.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:25 smithi083 ceph-mon[29421]: 6.b scrub ok 2024-09-20T18:58:25.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:25 smithi083 ceph-mon[29421]: 2.1 scrub starts 2024-09-20T18:58:25.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:25 smithi083 ceph-mon[29421]: 2.1 scrub ok 2024-09-20T18:58:25.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:25 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3192998830' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]': finished 2024-09-20T18:58:25.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:25 smithi083 ceph-mon[29421]: osdmap e241: 8 total, 8 up, 7 in 2024-09-20T18:58:25.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:25 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:58:25.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:25 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:58:25.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:25 smithi083 ceph-mon[29421]: 4.e scrub starts 2024-09-20T18:58:25.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:25 smithi083 ceph-mon[29421]: 4.e scrub ok 2024-09-20T18:58:25.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:25 smithi083 ceph-mon[36260]: 6.b scrub starts 2024-09-20T18:58:25.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:25 smithi083 ceph-mon[36260]: 6.b scrub ok 2024-09-20T18:58:25.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:25 smithi083 ceph-mon[36260]: 2.1 scrub starts 2024-09-20T18:58:25.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:25 smithi083 ceph-mon[36260]: 2.1 scrub ok 2024-09-20T18:58:25.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:25 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3192998830' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]': finished 2024-09-20T18:58:25.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:25 smithi083 ceph-mon[36260]: osdmap e241: 8 total, 8 up, 7 in 2024-09-20T18:58:25.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:25 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:58:25.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:25 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:58:25.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:25 smithi083 ceph-mon[36260]: 4.e scrub starts 2024-09-20T18:58:25.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:25 smithi083 ceph-mon[36260]: 4.e scrub ok 2024-09-20T18:58:25.612 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T18:58:25.612 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:58:25.638 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:25.661 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:25.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:25.662 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:25.763 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:25.785 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:25.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:25.785 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:25.886 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:25.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:25.908 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:25.909 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:26.009 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:26.031 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:26.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:26.031 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:26.132 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:26.154 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:26.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:26.155 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:26.256 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:58:26.280 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:26.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:26.281 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:58:26.368 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_historic_ops 2024-09-20T18:58:26.381 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:26.404 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:26.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:26.405 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:26.506 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:26.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:26 smithi142 ceph-mon[32508]: pgmap v415: 132 pgs: 132 active+clean; 583 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 874 B/s rd, 0 op/s 2024-09-20T18:58:26.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:26 smithi142 ceph-mon[32508]: 3.7 scrub starts 2024-09-20T18:58:26.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:26 smithi142 ceph-mon[32508]: 3.7 scrub ok 2024-09-20T18:58:26.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:26 smithi142 ceph-mon[32508]: 6.f scrub starts 2024-09-20T18:58:26.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:26 smithi142 ceph-mon[32508]: 6.f scrub ok 2024-09-20T18:58:26.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:26 smithi142 ceph-mon[32508]: osdmap e242: 8 total, 8 up, 7 in 2024-09-20T18:58:26.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:26 smithi142 ceph-mon[32508]: 6.1c scrub starts 2024-09-20T18:58:26.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:26 smithi142 ceph-mon[32508]: 6.1c scrub ok 2024-09-20T18:58:26.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:26 smithi142 ceph-mon[32508]: 3.a scrub starts 2024-09-20T18:58:26.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:26 smithi142 ceph-mon[32508]: 3.a scrub ok 2024-09-20T18:58:26.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:26 smithi142 ceph-mon[32508]: osdmap e243: 8 total, 8 up, 7 in 2024-09-20T18:58:26.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:26.528 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:26.528 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:26.617 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:58:26.629 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:26.654 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:26.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:26.655 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:26.756 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:26.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:26.779 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:26.779 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:26.798 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:26 smithi083 ceph-mon[29421]: pgmap v415: 132 pgs: 132 active+clean; 583 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 874 B/s rd, 0 op/s 2024-09-20T18:58:26.798 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:26 smithi083 ceph-mon[29421]: 3.7 scrub starts 2024-09-20T18:58:26.798 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:26 smithi083 ceph-mon[29421]: 3.7 scrub ok 2024-09-20T18:58:26.798 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:26 smithi083 ceph-mon[29421]: 6.f scrub starts 2024-09-20T18:58:26.798 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:26 smithi083 ceph-mon[29421]: 6.f scrub ok 2024-09-20T18:58:26.798 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:26 smithi083 ceph-mon[29421]: osdmap e242: 8 total, 8 up, 7 in 2024-09-20T18:58:26.798 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:26 smithi083 ceph-mon[29421]: 6.1c scrub starts 2024-09-20T18:58:26.798 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:26 smithi083 ceph-mon[29421]: 6.1c scrub ok 2024-09-20T18:58:26.799 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:26 smithi083 ceph-mon[29421]: 3.a scrub starts 2024-09-20T18:58:26.799 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:26 smithi083 ceph-mon[29421]: 3.a scrub ok 2024-09-20T18:58:26.799 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:26 smithi083 ceph-mon[29421]: osdmap e243: 8 total, 8 up, 7 in 2024-09-20T18:58:26.799 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:26 smithi083 ceph-mon[36260]: pgmap v415: 132 pgs: 132 active+clean; 583 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 874 B/s rd, 0 op/s 2024-09-20T18:58:26.799 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:26 smithi083 ceph-mon[36260]: 3.7 scrub starts 2024-09-20T18:58:26.799 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:26 smithi083 ceph-mon[36260]: 3.7 scrub ok 2024-09-20T18:58:26.799 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:26 smithi083 ceph-mon[36260]: 6.f scrub starts 2024-09-20T18:58:26.799 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:26 smithi083 ceph-mon[36260]: 6.f scrub ok 2024-09-20T18:58:26.799 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:26 smithi083 ceph-mon[36260]: osdmap e242: 8 total, 8 up, 7 in 2024-09-20T18:58:26.799 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:26 smithi083 ceph-mon[36260]: 6.1c scrub starts 2024-09-20T18:58:26.799 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:26 smithi083 ceph-mon[36260]: 6.1c scrub ok 2024-09-20T18:58:26.800 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:26 smithi083 ceph-mon[36260]: 3.a scrub starts 2024-09-20T18:58:26.800 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:26 smithi083 ceph-mon[36260]: 3.a scrub ok 2024-09-20T18:58:26.800 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:26 smithi083 ceph-mon[36260]: osdmap e243: 8 total, 8 up, 7 in 2024-09-20T18:58:26.879 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:26.900 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:26.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:26.901 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:27.002 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:27.018 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:58:26 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:58:27.028 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:27.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:27.029 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:27.126 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T18:58:27.127 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:58:27.130 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:27.152 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:27.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:27.153 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:27.253 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:27.274 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:27.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:27.275 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:27.376 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:27.397 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:27.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:27.398 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:27.499 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:27.520 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:27.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:27.521 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:27.546 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:58:27.547 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:58:27.566 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:58:27.566 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:58:27.582 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:58:27.582 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:58:27.583 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:27 smithi083 ceph-mon[29421]: 5.1b scrub starts 2024-09-20T18:58:27.583 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:27 smithi083 ceph-mon[29421]: 5.1b scrub ok 2024-09-20T18:58:27.583 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:27 smithi083 ceph-mon[36260]: 5.1b scrub starts 2024-09-20T18:58:27.583 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:27 smithi083 ceph-mon[36260]: 5.1b scrub ok 2024-09-20T18:58:27.597 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:27 smithi142 ceph-mon[32508]: 5.1b scrub starts 2024-09-20T18:58:27.597 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:27 smithi142 ceph-mon[32508]: 5.1b scrub ok 2024-09-20T18:58:27.598 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:58:27.598 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:58:27.615 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:58:27.615 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:58:27.621 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:27.631 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:58:27.632 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:58:27.642 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:27.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:27.643 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:27.648 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:58:27.648 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:58:27.666 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:58:27.666 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:58:27.743 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:27.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:27.766 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:27.766 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:27.866 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:27.888 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:27.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:27.889 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:27.915 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-20T18:58:27.989 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:28.010 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:28.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:28.011 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:28.112 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:28.133 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:28.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:28.134 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:28.235 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:28.256 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:28.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:28.257 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:28.357 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:58:28.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:28.380 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:28.380 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:58:28.481 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:28.502 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:28.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:28.503 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:28.604 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:28.625 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:28.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:28.626 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:28.726 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:28.749 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:28.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:28.750 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:28.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:28 smithi142 ceph-mon[32508]: 6.19 scrub starts 2024-09-20T18:58:28.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:28 smithi142 ceph-mon[32508]: 6.19 scrub ok 2024-09-20T18:58:28.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:28 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:58:28.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:28 smithi142 ceph-mon[32508]: 6.0 scrub starts 2024-09-20T18:58:28.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:28 smithi142 ceph-mon[32508]: 6.0 scrub ok 2024-09-20T18:58:28.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:28 smithi142 ceph-mon[32508]: pgmap v418: 132 pgs: 132 active+clean; 583 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 784 B/s rd, 0 op/s 2024-09-20T18:58:28.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:28 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/982572890' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:58:28.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:28 smithi142 ceph-mon[32508]: 5.9 scrub starts 2024-09-20T18:58:28.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:28 smithi142 ceph-mon[32508]: 5.9 scrub ok 2024-09-20T18:58:28.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:28 smithi142 ceph-mon[32508]: 4.12 scrub starts 2024-09-20T18:58:28.769 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:28 smithi142 ceph-mon[32508]: 4.12 scrub ok 2024-09-20T18:58:28.798 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:28 smithi083 ceph-mon[29421]: 6.19 scrub starts 2024-09-20T18:58:28.798 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:28 smithi083 ceph-mon[29421]: 6.19 scrub ok 2024-09-20T18:58:28.798 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:28 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:58:28.798 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:28 smithi083 ceph-mon[29421]: 6.0 scrub starts 2024-09-20T18:58:28.798 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:28 smithi083 ceph-mon[29421]: 6.0 scrub ok 2024-09-20T18:58:28.798 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:28 smithi083 ceph-mon[29421]: pgmap v418: 132 pgs: 132 active+clean; 583 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 784 B/s rd, 0 op/s 2024-09-20T18:58:28.798 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:28 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/982572890' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:58:28.799 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:28 smithi083 ceph-mon[29421]: 5.9 scrub starts 2024-09-20T18:58:28.799 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:28 smithi083 ceph-mon[29421]: 5.9 scrub ok 2024-09-20T18:58:28.799 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:28 smithi083 ceph-mon[29421]: 4.12 scrub starts 2024-09-20T18:58:28.799 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:28 smithi083 ceph-mon[29421]: 4.12 scrub ok 2024-09-20T18:58:28.799 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:28 smithi083 ceph-mon[36260]: 6.19 scrub starts 2024-09-20T18:58:28.799 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:28 smithi083 ceph-mon[36260]: 6.19 scrub ok 2024-09-20T18:58:28.799 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:28 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:58:28.799 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:28 smithi083 ceph-mon[36260]: 6.0 scrub starts 2024-09-20T18:58:28.799 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:28 smithi083 ceph-mon[36260]: 6.0 scrub ok 2024-09-20T18:58:28.799 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:28 smithi083 ceph-mon[36260]: pgmap v418: 132 pgs: 132 active+clean; 583 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 784 B/s rd, 0 op/s 2024-09-20T18:58:28.799 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:28 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/982572890' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:58:28.800 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:28 smithi083 ceph-mon[36260]: 5.9 scrub starts 2024-09-20T18:58:28.800 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:28 smithi083 ceph-mon[36260]: 5.9 scrub ok 2024-09-20T18:58:28.800 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:28 smithi083 ceph-mon[36260]: 4.12 scrub starts 2024-09-20T18:58:28.800 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:28 smithi083 ceph-mon[36260]: 4.12 scrub ok 2024-09-20T18:58:28.851 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:28.872 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:28.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:28.874 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:28.969 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:58:28.975 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:28.998 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:28.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:28.999 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:29.005 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-20T18:58:29.006 INFO:tasks.radosbench:Beginning radosbench... 2024-09-20T18:58:29.006 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_2 2024-09-20T18:58:29.006 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd pool create unique_pool_2 16 2024-09-20T18:58:29.024 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:58:29.025 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:58:29.100 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:29.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:29.121 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:29.122 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:29.223 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:29.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:29.249 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:29.249 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:29.350 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:29.374 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:29.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:29.375 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:29.476 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:29.499 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:29.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:29.559 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:29.559 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:29 smithi083 ceph-mon[29421]: 6.2 scrub starts 2024-09-20T18:58:29.559 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:29 smithi083 ceph-mon[29421]: 6.2 scrub ok 2024-09-20T18:58:29.560 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:29 smithi083 ceph-mon[29421]: 5.1e scrub starts 2024-09-20T18:58:29.560 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:29 smithi083 ceph-mon[29421]: 5.1e scrub ok 2024-09-20T18:58:29.560 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:29 smithi083 ceph-mon[29421]: 6.13 scrub starts 2024-09-20T18:58:29.560 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:29 smithi083 ceph-mon[29421]: 6.13 scrub ok 2024-09-20T18:58:29.560 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:29 smithi083 ceph-mon[29421]: 4.1c scrub starts 2024-09-20T18:58:29.560 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:29 smithi083 ceph-mon[29421]: 4.1c scrub ok 2024-09-20T18:58:29.560 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:29 smithi083 ceph-mon[29421]: 3.0 scrub starts 2024-09-20T18:58:29.560 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:29 smithi083 ceph-mon[29421]: 3.0 scrub ok 2024-09-20T18:58:29.560 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:29 smithi083 ceph-mon[36260]: 6.2 scrub starts 2024-09-20T18:58:29.560 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:29 smithi083 ceph-mon[36260]: 6.2 scrub ok 2024-09-20T18:58:29.560 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:29 smithi083 ceph-mon[36260]: 5.1e scrub starts 2024-09-20T18:58:29.560 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:29 smithi083 ceph-mon[36260]: 5.1e scrub ok 2024-09-20T18:58:29.560 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:29 smithi083 ceph-mon[36260]: 6.13 scrub starts 2024-09-20T18:58:29.561 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:29 smithi083 ceph-mon[36260]: 6.13 scrub ok 2024-09-20T18:58:29.561 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:29 smithi083 ceph-mon[36260]: 4.1c scrub starts 2024-09-20T18:58:29.561 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:29 smithi083 ceph-mon[36260]: 4.1c scrub ok 2024-09-20T18:58:29.561 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:29 smithi083 ceph-mon[36260]: 3.0 scrub starts 2024-09-20T18:58:29.561 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:29 smithi083 ceph-mon[36260]: 3.0 scrub ok 2024-09-20T18:58:29.661 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:29.682 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:29.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:29.683 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:29.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:29 smithi142 ceph-mon[32508]: 6.2 scrub starts 2024-09-20T18:58:29.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:29 smithi142 ceph-mon[32508]: 6.2 scrub ok 2024-09-20T18:58:29.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:29 smithi142 ceph-mon[32508]: 5.1e scrub starts 2024-09-20T18:58:29.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:29 smithi142 ceph-mon[32508]: 5.1e scrub ok 2024-09-20T18:58:29.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:29 smithi142 ceph-mon[32508]: 6.13 scrub starts 2024-09-20T18:58:29.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:29 smithi142 ceph-mon[32508]: 6.13 scrub ok 2024-09-20T18:58:29.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:29 smithi142 ceph-mon[32508]: 4.1c scrub starts 2024-09-20T18:58:29.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:29 smithi142 ceph-mon[32508]: 4.1c scrub ok 2024-09-20T18:58:29.769 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:29 smithi142 ceph-mon[32508]: 3.0 scrub starts 2024-09-20T18:58:29.769 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:29 smithi142 ceph-mon[32508]: 3.0 scrub ok 2024-09-20T18:58:29.783 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:29.804 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:29.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:29.805 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:29.906 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:29.931 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:29.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:29.932 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:29.994 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:58:30.033 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:30.057 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:30.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:30.058 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:30.088 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:58:30.090 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-20T18:58:30.159 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:30.184 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:30.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:30.185 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:30.285 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:30.307 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:30.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:30.308 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:30.362 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-20T18:58:30.408 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:30.429 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:30.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:30.430 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:30.446 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T18:58:30.531 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:30.553 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:30.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:30.554 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:30.610 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:30 smithi083 ceph-mon[29421]: 5.16 scrub starts 2024-09-20T18:58:30.611 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:30 smithi083 ceph-mon[29421]: 5.16 scrub ok 2024-09-20T18:58:30.611 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:30 smithi083 ceph-mon[29421]: 6.e scrub starts 2024-09-20T18:58:30.611 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:30 smithi083 ceph-mon[29421]: 6.e scrub ok 2024-09-20T18:58:30.611 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:30 smithi083 ceph-mon[29421]: pgmap v419: 132 pgs: 132 active+clean; 583 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 682 B/s rd, 0 op/s 2024-09-20T18:58:30.611 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:30 smithi083 ceph-mon[29421]: 6.3 scrub starts 2024-09-20T18:58:30.611 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:30 smithi083 ceph-mon[29421]: 6.3 scrub ok 2024-09-20T18:58:30.611 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:30 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3613704502' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:58:30.611 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:30 smithi083 ceph-mon[36260]: 5.16 scrub starts 2024-09-20T18:58:30.611 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:30 smithi083 ceph-mon[36260]: 5.16 scrub ok 2024-09-20T18:58:30.611 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:30 smithi083 ceph-mon[36260]: 6.e scrub starts 2024-09-20T18:58:30.612 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:30 smithi083 ceph-mon[36260]: 6.e scrub ok 2024-09-20T18:58:30.612 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:30 smithi083 ceph-mon[36260]: pgmap v419: 132 pgs: 132 active+clean; 583 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 682 B/s rd, 0 op/s 2024-09-20T18:58:30.612 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:30 smithi083 ceph-mon[36260]: 6.3 scrub starts 2024-09-20T18:58:30.612 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:30 smithi083 ceph-mon[36260]: 6.3 scrub ok 2024-09-20T18:58:30.612 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:30 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3613704502' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:58:30.637 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T18:58:30.637 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-20T18:58:30.637 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-20T18:58:30.637 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T18:58:30.655 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:30.677 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:30.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:30.678 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:30.767 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:30 smithi142 ceph-mon[32508]: 5.16 scrub starts 2024-09-20T18:58:30.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:30 smithi142 ceph-mon[32508]: 5.16 scrub ok 2024-09-20T18:58:30.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:30 smithi142 ceph-mon[32508]: 6.e scrub starts 2024-09-20T18:58:30.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:30 smithi142 ceph-mon[32508]: 6.e scrub ok 2024-09-20T18:58:30.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:30 smithi142 ceph-mon[32508]: pgmap v419: 132 pgs: 132 active+clean; 583 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 682 B/s rd, 0 op/s 2024-09-20T18:58:30.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:30 smithi142 ceph-mon[32508]: 6.3 scrub starts 2024-09-20T18:58:30.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:30 smithi142 ceph-mon[32508]: 6.3 scrub ok 2024-09-20T18:58:30.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:30 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3613704502' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:58:30.779 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:30.802 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:30.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:30.803 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:30.904 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:30.926 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:30.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:30.927 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:30.999 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:58:30 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:58:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:58:31.027 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:58:31.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:31.050 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:31.051 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:58:31.151 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:31.173 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:31.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:31.174 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:31.274 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:31.299 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:31.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:31.300 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:31.401 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:31.426 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:31.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:31.427 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:31.446 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":419,"stamp":"2024-09-20T18:58:28.964895+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":1359,"num_read_kb":1216,"num_write":549,"num_write_kb":1673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":1182665,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":556,"ondisk_log_size":556,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":401,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":2139384,"kb_used_data":9364,"kb_used_omap":10,"kb_used_meta":2129845,"kb_avail":654076680,"statfs":{"total":671965249536,"available":669774520320,"internally_reserved":0,"allocated":9588736,"data_stored":5833718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11158,"internal_metadata":2180961386},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":55,"apply_latency_ms":55,"commit_latency_ns":55000000,"apply_latency_ns":55000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":-232178,"num_write_kb":-7429633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-28056,"num_bytes_recovered":-1675296768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001462"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":406,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:26.104495+0000","last_change":"2024-09-20T18:55:28.037494+0000","last_active":"2024-09-20T18:58:26.104495+0000","last_peered":"2024-09-20T18:58:26.104495+0000","last_clean":"2024-09-20T18:58:26.104495+0000","last_became_active":"2024-09-20T18:49:45.943201+0000","last_became_peered":"2024-09-20T18:49:45.943201+0000","last_unstale":"2024-09-20T18:58:26.104495+0000","last_undegraded":"2024-09-20T18:58:26.104495+0000","last_fullsized":"2024-09-20T18:58:26.104495+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:28.037449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:28.037449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":452,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.731333+0000","last_change":"2024-09-20T18:55:28.838729+0000","last_active":"2024-09-20T18:58:25.731333+0000","last_peered":"2024-09-20T18:58:25.731333+0000","last_clean":"2024-09-20T18:58:25.731333+0000","last_became_active":"2024-09-20T18:55:28.819637+0000","last_became_peered":"2024-09-20T18:55:28.819637+0000","last_unstale":"2024-09-20T18:58:25.731333+0000","last_undegraded":"2024-09-20T18:58:25.731333+0000","last_fullsized":"2024-09-20T18:58:25.731333+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:50.231620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:50.231620+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":405,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.736705+0000","last_change":"2024-09-20T18:57:49.016078+0000","last_active":"2024-09-20T18:58:25.736705+0000","last_peered":"2024-09-20T18:58:25.736705+0000","last_clean":"2024-09-20T18:58:25.736705+0000","last_became_active":"2024-09-20T18:49:49.959514+0000","last_became_peered":"2024-09-20T18:49:49.959514+0000","last_unstale":"2024-09-20T18:58:25.736705+0000","last_undegraded":"2024-09-20T18:58:25.736705+0000","last_fullsized":"2024-09-20T18:58:25.736705+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:49.016006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:49.016006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:13.297987+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":402,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.731507+0000","last_change":"2024-09-20T18:58:02.084435+0000","last_active":"2024-09-20T18:58:25.731507+0000","last_peered":"2024-09-20T18:58:25.731507+0000","last_clean":"2024-09-20T18:58:25.731507+0000","last_became_active":"2024-09-20T18:49:51.969421+0000","last_became_peered":"2024-09-20T18:49:51.969421+0000","last_unstale":"2024-09-20T18:58:25.731507+0000","last_undegraded":"2024-09-20T18:58:25.731507+0000","last_fullsized":"2024-09-20T18:58:25.731507+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:02.084375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:02.084375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:20.422560+0000","scrub_duration":197,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":400,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.736307+0000","last_change":"2024-09-20T18:55:28.822034+0000","last_active":"2024-09-20T18:58:25.736307+0000","last_peered":"2024-09-20T18:58:25.736307+0000","last_clean":"2024-09-20T18:58:25.736307+0000","last_became_active":"2024-09-20T18:55:28.821717+0000","last_became_peered":"2024-09-20T18:55:28.821717+0000","last_unstale":"2024-09-20T18:58:25.736307+0000","last_undegraded":"2024-09-20T18:58:25.736307+0000","last_fullsized":"2024-09-20T18:58:25.736307+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.216188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.216188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:13.051959+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":405,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.772233+0000","last_change":"2024-09-20T18:57:22.694362+0000","last_active":"2024-09-20T18:58:25.772233+0000","last_peered":"2024-09-20T18:58:25.772233+0000","last_clean":"2024-09-20T18:58:25.772233+0000","last_became_active":"2024-09-20T18:55:28.824826+0000","last_became_peered":"2024-09-20T18:55:28.824826+0000","last_unstale":"2024-09-20T18:58:25.772233+0000","last_undegraded":"2024-09-20T18:58:25.772233+0000","last_fullsized":"2024-09-20T18:58:25.772233+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:22.694295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:22.694295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:33.473592+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":418,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.736223+0000","last_change":"2024-09-20T18:57:13.774827+0000","last_active":"2024-09-20T18:58:25.736223+0000","last_peered":"2024-09-20T18:58:25.736223+0000","last_clean":"2024-09-20T18:58:25.736223+0000","last_became_active":"2024-09-20T18:49:47.952150+0000","last_became_peered":"2024-09-20T18:49:47.952150+0000","last_unstale":"2024-09-20T18:58:25.736223+0000","last_undegraded":"2024-09-20T18:58:25.736223+0000","last_fullsized":"2024-09-20T18:58:25.736223+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:57:13.774305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:13.774305+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":405,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.731346+0000","last_change":"2024-09-20T18:58:06.846086+0000","last_active":"2024-09-20T18:58:25.731346+0000","last_peered":"2024-09-20T18:58:25.731346+0000","last_clean":"2024-09-20T18:58:25.731346+0000","last_became_active":"2024-09-20T18:55:28.820362+0000","last_became_peered":"2024-09-20T18:55:28.820362+0000","last_unstale":"2024-09-20T18:58:25.731346+0000","last_undegraded":"2024-09-20T18:58:25.731346+0000","last_fullsized":"2024-09-20T18:58:25.731346+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:06.846018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:06.846018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:33.625936+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"59'1","reported_seq":435,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.308241+0000","last_change":"2024-09-20T18:58:09.472495+0000","last_active":"2024-09-20T18:58:25.308241+0000","last_peered":"2024-09-20T18:58:25.308241+0000","last_clean":"2024-09-20T18:58:25.308241+0000","last_became_active":"2024-09-20T18:55:28.833386+0000","last_became_peered":"2024-09-20T18:55:28.833386+0000","last_unstale":"2024-09-20T18:58:25.308241+0000","last_undegraded":"2024-09-20T18:58:25.308241+0000","last_fullsized":"2024-09-20T18:58:25.308241+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:58:09.472360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:09.472360+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:29.957740+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":418,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.731675+0000","last_change":"2024-09-20T18:54:53.241889+0000","last_active":"2024-09-20T18:58:25.731675+0000","last_peered":"2024-09-20T18:58:25.731675+0000","last_clean":"2024-09-20T18:58:25.731675+0000","last_became_active":"2024-09-20T18:49:47.947448+0000","last_became_peered":"2024-09-20T18:49:47.947448+0000","last_unstale":"2024-09-20T18:58:25.731675+0000","last_undegraded":"2024-09-20T18:58:25.731675+0000","last_fullsized":"2024-09-20T18:58:25.731675+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:53.241802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:53.241802+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":402,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.505818+0000","last_change":"2024-09-20T18:58:25.505818+0000","last_active":"2024-09-20T18:58:25.505818+0000","last_peered":"2024-09-20T18:58:25.505818+0000","last_clean":"2024-09-20T18:58:25.505818+0000","last_became_active":"2024-09-20T18:49:49.961188+0000","last_became_peered":"2024-09-20T18:49:49.961188+0000","last_unstale":"2024-09-20T18:58:25.505818+0000","last_undegraded":"2024-09-20T18:58:25.505818+0000","last_fullsized":"2024-09-20T18:58:25.505818+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:25.505757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:25.505757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:26.562790+0000","scrub_duration":45,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":377,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:26.103721+0000","last_change":"2024-09-20T18:58:21.861893+0000","last_active":"2024-09-20T18:58:26.103721+0000","last_peered":"2024-09-20T18:58:26.103721+0000","last_clean":"2024-09-20T18:58:26.103721+0000","last_became_active":"2024-09-20T18:49:51.966715+0000","last_became_peered":"2024-09-20T18:49:51.966715+0000","last_unstale":"2024-09-20T18:58:26.103721+0000","last_undegraded":"2024-09-20T18:58:26.103721+0000","last_fullsized":"2024-09-20T18:58:26.103721+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:21.861836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:21.861836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:51.627461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":394,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.307813+0000","last_change":"2024-09-20T18:55:15.252839+0000","last_active":"2024-09-20T18:58:25.307813+0000","last_peered":"2024-09-20T18:58:25.307813+0000","last_clean":"2024-09-20T18:58:25.307813+0000","last_became_active":"2024-09-20T18:49:45.944703+0000","last_became_peered":"2024-09-20T18:49:45.944703+0000","last_unstale":"2024-09-20T18:58:25.307813+0000","last_undegraded":"2024-09-20T18:58:25.307813+0000","last_fullsized":"2024-09-20T18:58:25.307813+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:15.252782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:15.252782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:15.451958+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":422,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.731608+0000","last_change":"2024-09-20T18:51:26.494194+0000","last_active":"2024-09-20T18:58:25.731608+0000","last_peered":"2024-09-20T18:58:25.731608+0000","last_clean":"2024-09-20T18:58:25.731608+0000","last_became_active":"2024-09-20T18:51:23.813399+0000","last_became_peered":"2024-09-20T18:51:23.813399+0000","last_unstale":"2024-09-20T18:58:25.731608+0000","last_undegraded":"2024-09-20T18:58:25.731608+0000","last_fullsized":"2024-09-20T18:58:25.731608+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-20T18:51:26.494118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:51:26.494118+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:31.887636+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":387,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.500543+0000","last_change":"2024-09-20T18:52:38.799542+0000","last_active":"2024-09-20T18:58:25.500543+0000","last_peered":"2024-09-20T18:58:25.500543+0000","last_clean":"2024-09-20T18:58:25.500543+0000","last_became_active":"2024-09-20T18:49:49.959696+0000","last_became_peered":"2024-09-20T18:49:49.959696+0000","last_unstale":"2024-09-20T18:58:25.500543+0000","last_undegraded":"2024-09-20T18:58:25.500543+0000","last_fullsized":"2024-09-20T18:58:25.500543+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:38.799487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:38.799487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":390,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.307471+0000","last_change":"2024-09-20T18:57:13.587827+0000","last_active":"2024-09-20T18:58:25.307471+0000","last_peered":"2024-09-20T18:58:25.307471+0000","last_clean":"2024-09-20T18:58:25.307471+0000","last_became_active":"2024-09-20T18:49:51.970663+0000","last_became_peered":"2024-09-20T18:49:51.970663+0000","last_unstale":"2024-09-20T18:58:25.307471+0000","last_undegraded":"2024-09-20T18:58:25.307471+0000","last_fullsized":"2024-09-20T18:58:25.307471+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:13.587774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:13.587774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":402,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.731816+0000","last_change":"2024-09-20T18:58:00.906697+0000","last_active":"2024-09-20T18:58:25.731816+0000","last_peered":"2024-09-20T18:58:25.731816+0000","last_clean":"2024-09-20T18:58:25.731816+0000","last_became_active":"2024-09-20T18:55:28.827435+0000","last_became_peered":"2024-09-20T18:55:28.827435+0000","last_unstale":"2024-09-20T18:58:25.731816+0000","last_undegraded":"2024-09-20T18:58:25.731816+0000","last_fullsized":"2024-09-20T18:58:25.731816+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:00.906652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:00.906652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:23.434725+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":397,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:26.103403+0000","last_change":"2024-09-20T18:54:23.984024+0000","last_active":"2024-09-20T18:58:26.103403+0000","last_peered":"2024-09-20T18:58:26.103403+0000","last_clean":"2024-09-20T18:58:26.103403+0000","last_became_active":"2024-09-20T18:49:45.950480+0000","last_became_peered":"2024-09-20T18:49:45.950480+0000","last_unstale":"2024-09-20T18:58:26.103403+0000","last_undegraded":"2024-09-20T18:58:26.103403+0000","last_fullsized":"2024-09-20T18:58:26.103403+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:23.983958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:23.983958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":420,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:26.273018+0000","last_change":"2024-09-20T18:53:26.762207+0000","last_active":"2024-09-20T18:58:26.273018+0000","last_peered":"2024-09-20T18:58:26.273018+0000","last_clean":"2024-09-20T18:58:26.273018+0000","last_became_active":"2024-09-20T18:49:47.950939+0000","last_became_peered":"2024-09-20T18:49:47.950939+0000","last_unstale":"2024-09-20T18:58:26.273018+0000","last_undegraded":"2024-09-20T18:58:26.273018+0000","last_fullsized":"2024-09-20T18:58:26.273018+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:53:26.762131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:26.762131+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":389,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.772629+0000","last_change":"2024-09-20T18:57:14.941518+0000","last_active":"2024-09-20T18:58:25.772629+0000","last_peered":"2024-09-20T18:58:25.772629+0000","last_clean":"2024-09-20T18:58:25.772629+0000","last_became_active":"2024-09-20T18:49:49.958518+0000","last_became_peered":"2024-09-20T18:49:49.958518+0000","last_unstale":"2024-09-20T18:58:25.772629+0000","last_undegraded":"2024-09-20T18:58:25.772629+0000","last_fullsized":"2024-09-20T18:58:25.772629+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:14.941458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:14.941458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:37.988570+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":393,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.772186+0000","last_change":"2024-09-20T18:55:28.833468+0000","last_active":"2024-09-20T18:58:25.772186+0000","last_peered":"2024-09-20T18:58:25.772186+0000","last_clean":"2024-09-20T18:58:25.772186+0000","last_became_active":"2024-09-20T18:55:28.833133+0000","last_became_peered":"2024-09-20T18:55:28.833133+0000","last_unstale":"2024-09-20T18:58:25.772186+0000","last_undegraded":"2024-09-20T18:58:25.772186+0000","last_fullsized":"2024-09-20T18:58:25.772186+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:54.259174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:54.259174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:30.790542+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"59'1","reported_seq":412,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.731713+0000","last_change":"2024-09-20T18:57:53.918080+0000","last_active":"2024-09-20T18:58:25.731713+0000","last_peered":"2024-09-20T18:58:25.731713+0000","last_clean":"2024-09-20T18:58:25.731713+0000","last_became_active":"2024-09-20T18:49:45.940100+0000","last_became_peered":"2024-09-20T18:49:45.940100+0000","last_unstale":"2024-09-20T18:58:25.731713+0000","last_undegraded":"2024-09-20T18:58:25.731713+0000","last_fullsized":"2024-09-20T18:58:25.731713+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:57:53.918022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:53.918022+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:03.242427+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":408,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.771902+0000","last_change":"2024-09-20T18:57:09.819080+0000","last_active":"2024-09-20T18:58:25.771902+0000","last_peered":"2024-09-20T18:58:25.771902+0000","last_clean":"2024-09-20T18:58:25.771902+0000","last_became_active":"2024-09-20T18:49:47.950909+0000","last_became_peered":"2024-09-20T18:49:47.950909+0000","last_unstale":"2024-09-20T18:58:25.771902+0000","last_undegraded":"2024-09-20T18:58:25.771902+0000","last_fullsized":"2024-09-20T18:58:25.771902+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:57:09.818999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:09.818999+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":398,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.731605+0000","last_change":"2024-09-20T18:54:44.239273+0000","last_active":"2024-09-20T18:58:25.731605+0000","last_peered":"2024-09-20T18:58:25.731605+0000","last_clean":"2024-09-20T18:58:25.731605+0000","last_became_active":"2024-09-20T18:49:49.959102+0000","last_became_peered":"2024-09-20T18:49:49.959102+0000","last_unstale":"2024-09-20T18:58:25.731605+0000","last_undegraded":"2024-09-20T18:58:25.731605+0000","last_fullsized":"2024-09-20T18:58:25.731605+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:44.239226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.239226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:17.660601+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":392,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.501434+0000","last_change":"2024-09-20T18:58:25.501434+0000","last_active":"2024-09-20T18:58:25.501434+0000","last_peered":"2024-09-20T18:58:25.501434+0000","last_clean":"2024-09-20T18:58:25.501434+0000","last_became_active":"2024-09-20T18:49:51.970058+0000","last_became_peered":"2024-09-20T18:49:51.970058+0000","last_unstale":"2024-09-20T18:58:25.501434+0000","last_undegraded":"2024-09-20T18:58:25.501434+0000","last_fullsized":"2024-09-20T18:58:25.501434+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T18:58:25.501384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:25.501384+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:36.383692+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":398,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.736890+0000","last_change":"2024-09-20T18:57:18.032140+0000","last_active":"2024-09-20T18:58:25.736890+0000","last_peered":"2024-09-20T18:58:25.736890+0000","last_clean":"2024-09-20T18:58:25.736890+0000","last_became_active":"2024-09-20T18:49:45.933078+0000","last_became_peered":"2024-09-20T18:49:45.933078+0000","last_unstale":"2024-09-20T18:58:25.736890+0000","last_undegraded":"2024-09-20T18:58:25.736890+0000","last_fullsized":"2024-09-20T18:58:25.736890+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:57:18.032067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:18.032067+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:42.727785+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":417,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:26.104394+0000","last_change":"2024-09-20T18:55:19.996522+0000","last_active":"2024-09-20T18:58:26.104394+0000","last_peered":"2024-09-20T18:58:26.104394+0000","last_clean":"2024-09-20T18:58:26.104394+0000","last_became_active":"2024-09-20T18:49:48.500809+0000","last_became_peered":"2024-09-20T18:49:48.500809+0000","last_unstale":"2024-09-20T18:58:26.104394+0000","last_undegraded":"2024-09-20T18:58:26.104394+0000","last_fullsized":"2024-09-20T18:58:26.104394+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:55:19.996328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:19.996328+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"184'9","reported_seq":212,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.732114+0000","last_change":"2024-09-20T18:57:48.853376+0000","last_active":"2024-09-20T18:58:25.732114+0000","last_peered":"2024-09-20T18:58:25.732114+0000","last_clean":"2024-09-20T18:58:25.732114+0000","last_became_active":"2024-09-20T18:55:28.820941+0000","last_became_peered":"2024-09-20T18:55:28.820941+0000","last_unstale":"2024-09-20T18:58:25.732114+0000","last_undegraded":"2024-09-20T18:58:25.732114+0000","last_fullsized":"2024-09-20T18:58:25.732114+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'9","last_scrub_stamp":"2024-09-20T18:57:48.853302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:48.853302+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:06.295025+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":411,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:26.273143+0000","last_change":"2024-09-20T18:55:18.581290+0000","last_active":"2024-09-20T18:58:26.273143+0000","last_peered":"2024-09-20T18:58:26.273143+0000","last_clean":"2024-09-20T18:58:26.273143+0000","last_became_active":"2024-09-20T18:49:58.992302+0000","last_became_peered":"2024-09-20T18:49:58.992302+0000","last_unstale":"2024-09-20T18:58:26.273143+0000","last_undegraded":"2024-09-20T18:58:26.273143+0000","last_fullsized":"2024-09-20T18:58:26.273143+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:55:18.581200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:18.581200+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":397,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.736189+0000","last_change":"2024-09-20T18:54:54.288199+0000","last_active":"2024-09-20T18:58:25.736189+0000","last_peered":"2024-09-20T18:58:25.736189+0000","last_clean":"2024-09-20T18:58:25.736189+0000","last_became_active":"2024-09-20T18:49:45.936104+0000","last_became_peered":"2024-09-20T18:49:45.936104+0000","last_unstale":"2024-09-20T18:58:25.736189+0000","last_undegraded":"2024-09-20T18:58:25.736189+0000","last_fullsized":"2024-09-20T18:58:25.736189+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:54.288117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:54.288117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:21.124624+0000","scrub_duration":46,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":505,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.731871+0000","last_change":"2024-09-20T18:54:14.292660+0000","last_active":"2024-09-20T18:58:25.731871+0000","last_peered":"2024-09-20T18:58:25.731871+0000","last_clean":"2024-09-20T18:58:25.731871+0000","last_became_active":"2024-09-20T18:49:49.960697+0000","last_became_peered":"2024-09-20T18:49:49.960697+0000","last_unstale":"2024-09-20T18:58:25.731871+0000","last_undegraded":"2024-09-20T18:58:25.731871+0000","last_fullsized":"2024-09-20T18:58:25.731871+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:54:14.292610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:14.292610+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:10.776813+0000","scrub_duration":34,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":382,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.307908+0000","last_change":"2024-09-20T18:54:10.381171+0000","last_active":"2024-09-20T18:58:25.307908+0000","last_peered":"2024-09-20T18:58:25.307908+0000","last_clean":"2024-09-20T18:58:25.307908+0000","last_became_active":"2024-09-20T18:49:51.971399+0000","last_became_peered":"2024-09-20T18:49:51.971399+0000","last_unstale":"2024-09-20T18:58:25.307908+0000","last_undegraded":"2024-09-20T18:58:25.307908+0000","last_fullsized":"2024-09-20T18:58:25.307908+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:10.381108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:10.381108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:30.527012+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":434,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.736093+0000","last_change":"2024-09-20T18:58:08.025426+0000","last_active":"2024-09-20T18:58:25.736093+0000","last_peered":"2024-09-20T18:58:25.736093+0000","last_clean":"2024-09-20T18:58:25.736093+0000","last_became_active":"2024-09-20T18:49:48.499189+0000","last_became_peered":"2024-09-20T18:49:48.499189+0000","last_unstale":"2024-09-20T18:58:25.736093+0000","last_undegraded":"2024-09-20T18:58:25.736093+0000","last_fullsized":"2024-09-20T18:58:25.736093+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-20T18:58:08.025202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:08.025202+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:16.521840+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":397,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.772433+0000","last_change":"2024-09-20T18:58:25.159724+0000","last_active":"2024-09-20T18:58:25.772433+0000","last_peered":"2024-09-20T18:58:25.772433+0000","last_clean":"2024-09-20T18:58:25.772433+0000","last_became_active":"2024-09-20T18:49:45.940544+0000","last_became_peered":"2024-09-20T18:49:45.940544+0000","last_unstale":"2024-09-20T18:58:25.772433+0000","last_undegraded":"2024-09-20T18:58:25.772433+0000","last_fullsized":"2024-09-20T18:58:25.772433+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:25.159673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:25.159673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:40.807883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":399,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.731343+0000","last_change":"2024-09-20T18:54:51.229251+0000","last_active":"2024-09-20T18:58:25.731343+0000","last_peered":"2024-09-20T18:58:25.731343+0000","last_clean":"2024-09-20T18:58:25.731343+0000","last_became_active":"2024-09-20T18:49:49.963073+0000","last_became_peered":"2024-09-20T18:49:49.963073+0000","last_unstale":"2024-09-20T18:58:25.731343+0000","last_undegraded":"2024-09-20T18:58:25.731343+0000","last_fullsized":"2024-09-20T18:58:25.731343+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.229201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.229201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:20.666517+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":395,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.731618+0000","last_change":"2024-09-20T18:54:55.323937+0000","last_active":"2024-09-20T18:58:25.731618+0000","last_peered":"2024-09-20T18:58:25.731618+0000","last_clean":"2024-09-20T18:58:25.731618+0000","last_became_active":"2024-09-20T18:49:51.968793+0000","last_became_peered":"2024-09-20T18:49:51.968793+0000","last_unstale":"2024-09-20T18:58:25.731618+0000","last_undegraded":"2024-09-20T18:58:25.731618+0000","last_fullsized":"2024-09-20T18:58:25.731618+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:55.323692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:55.323692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":455,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.731352+0000","last_change":"2024-09-20T18:55:28.853375+0000","last_active":"2024-09-20T18:58:25.731352+0000","last_peered":"2024-09-20T18:58:25.731352+0000","last_clean":"2024-09-20T18:58:25.731352+0000","last_became_active":"2024-09-20T18:55:28.831057+0000","last_became_peered":"2024-09-20T18:55:28.831057+0000","last_unstale":"2024-09-20T18:58:25.731352+0000","last_undegraded":"2024-09-20T18:58:25.731352+0000","last_fullsized":"2024-09-20T18:58:25.731352+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-20T18:54:57.394893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.394893+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":390,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:26.104499+0000","last_change":"2024-09-20T18:51:27.187060+0000","last_active":"2024-09-20T18:58:26.104499+0000","last_peered":"2024-09-20T18:58:26.104499+0000","last_clean":"2024-09-20T18:58:26.104499+0000","last_became_active":"2024-09-20T18:49:45.940469+0000","last_became_peered":"2024-09-20T18:49:45.940469+0000","last_unstale":"2024-09-20T18:58:26.104499+0000","last_undegraded":"2024-09-20T18:58:26.104499+0000","last_fullsized":"2024-09-20T18:58:26.104499+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:51:27.186991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:27.186991+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"66'8","reported_seq":519,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.736005+0000","last_change":"2024-09-20T18:58:23.114419+0000","last_active":"2024-09-20T18:58:25.736005+0000","last_peered":"2024-09-20T18:58:25.736005+0000","last_clean":"2024-09-20T18:58:25.736005+0000","last_became_active":"2024-09-20T18:49:49.956133+0000","last_became_peered":"2024-09-20T18:49:49.956133+0000","last_unstale":"2024-09-20T18:58:25.736005+0000","last_undegraded":"2024-09-20T18:58:25.736005+0000","last_fullsized":"2024-09-20T18:58:25.736005+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:58:23.114371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:23.114371+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:27.894374+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":391,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.500651+0000","last_change":"2024-09-20T18:55:28.819147+0000","last_active":"2024-09-20T18:58:25.500651+0000","last_peered":"2024-09-20T18:58:25.500651+0000","last_clean":"2024-09-20T18:58:25.500651+0000","last_became_active":"2024-09-20T18:55:28.818804+0000","last_became_peered":"2024-09-20T18:55:28.818804+0000","last_unstale":"2024-09-20T18:58:25.500651+0000","last_undegraded":"2024-09-20T18:58:25.500651+0000","last_fullsized":"2024-09-20T18:58:25.500651+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:42.880724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:42.880724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":425,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.736877+0000","last_change":"2024-09-20T18:58:03.053521+0000","last_active":"2024-09-20T18:58:25.736877+0000","last_peered":"2024-09-20T18:58:25.736877+0000","last_clean":"2024-09-20T18:58:25.736877+0000","last_became_active":"2024-09-20T18:49:47.949626+0000","last_became_peered":"2024-09-20T18:49:47.949626+0000","last_unstale":"2024-09-20T18:58:25.736877+0000","last_undegraded":"2024-09-20T18:58:25.736877+0000","last_fullsized":"2024-09-20T18:58:25.736877+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:58:03.053310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:03.053310+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:05.671653+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":403,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.308320+0000","last_change":"2024-09-20T18:54:18.326947+0000","last_active":"2024-09-20T18:58:25.308320+0000","last_peered":"2024-09-20T18:58:25.308320+0000","last_clean":"2024-09-20T18:58:25.308320+0000","last_became_active":"2024-09-20T18:49:45.942496+0000","last_became_peered":"2024-09-20T18:49:45.942496+0000","last_unstale":"2024-09-20T18:58:25.308320+0000","last_undegraded":"2024-09-20T18:58:25.308320+0000","last_fullsized":"2024-09-20T18:58:25.308320+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:18.326875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:18.326875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:32.505949+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"184'9","reported_seq":514,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:26.104171+0000","last_change":"2024-09-20T18:55:29.010062+0000","last_active":"2024-09-20T18:58:26.104171+0000","last_peered":"2024-09-20T18:58:26.104171+0000","last_clean":"2024-09-20T18:58:26.104171+0000","last_became_active":"2024-09-20T18:55:28.830643+0000","last_became_peered":"2024-09-20T18:55:28.830643+0000","last_unstale":"2024-09-20T18:58:26.104171+0000","last_undegraded":"2024-09-20T18:58:26.104171+0000","last_fullsized":"2024-09-20T18:58:26.104171+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:29.010015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:29.010015+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":378,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:26.948165+0000","last_change":"2024-09-20T18:58:26.948165+0000","last_active":"2024-09-20T18:58:26.948165+0000","last_peered":"2024-09-20T18:58:26.948165+0000","last_clean":"2024-09-20T18:58:26.948165+0000","last_became_active":"2024-09-20T18:49:51.964780+0000","last_became_peered":"2024-09-20T18:49:51.964780+0000","last_unstale":"2024-09-20T18:58:26.948165+0000","last_undegraded":"2024-09-20T18:58:26.948165+0000","last_fullsized":"2024-09-20T18:58:26.948165+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:26.948102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:26.948102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:28.716180+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":430,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:26.104262+0000","last_change":"2024-09-20T18:54:14.923895+0000","last_active":"2024-09-20T18:58:26.104262+0000","last_peered":"2024-09-20T18:58:26.104262+0000","last_clean":"2024-09-20T18:58:26.104262+0000","last_became_active":"2024-09-20T18:49:48.497736+0000","last_became_peered":"2024-09-20T18:49:48.497736+0000","last_unstale":"2024-09-20T18:58:26.104262+0000","last_undegraded":"2024-09-20T18:58:26.104262+0000","last_fullsized":"2024-09-20T18:58:26.104262+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-20T18:54:14.923682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:14.923682+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":390,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.772354+0000","last_change":"2024-09-20T18:57:09.940634+0000","last_active":"2024-09-20T18:58:25.772354+0000","last_peered":"2024-09-20T18:58:25.772354+0000","last_clean":"2024-09-20T18:58:25.772354+0000","last_became_active":"2024-09-20T18:49:45.942253+0000","last_became_peered":"2024-09-20T18:49:45.942253+0000","last_unstale":"2024-09-20T18:58:25.772354+0000","last_undegraded":"2024-09-20T18:58:25.772354+0000","last_fullsized":"2024-09-20T18:58:25.772354+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:09.940517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:09.940517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:19.931681+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":136,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:26.104093+0000","last_change":"2024-09-20T18:55:28.830954+0000","last_active":"2024-09-20T18:58:26.104093+0000","last_peered":"2024-09-20T18:58:26.104093+0000","last_clean":"2024-09-20T18:58:26.104093+0000","last_became_active":"2024-09-20T18:55:28.830645+0000","last_became_peered":"2024-09-20T18:55:28.830645+0000","last_unstale":"2024-09-20T18:58:26.104093+0000","last_undegraded":"2024-09-20T18:58:26.104093+0000","last_fullsized":"2024-09-20T18:58:26.104093+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:50.610871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:50.610871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":409,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.736707+0000","last_change":"2024-09-20T18:57:58.973310+0000","last_active":"2024-09-20T18:58:25.736707+0000","last_peered":"2024-09-20T18:58:25.736707+0000","last_clean":"2024-09-20T18:58:25.736707+0000","last_became_active":"2024-09-20T18:55:28.821588+0000","last_became_peered":"2024-09-20T18:55:28.821588+0000","last_unstale":"2024-09-20T18:58:25.736707+0000","last_undegraded":"2024-09-20T18:58:25.736707+0000","last_fullsized":"2024-09-20T18:58:25.736707+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:58.973176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:58.973176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:03.568279+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"66'28","reported_seq":555,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.772257+0000","last_change":"2024-09-20T18:58:07.018449+0000","last_active":"2024-09-20T18:58:25.772257+0000","last_peered":"2024-09-20T18:58:25.772257+0000","last_clean":"2024-09-20T18:58:25.772257+0000","last_became_active":"2024-09-20T18:49:47.955335+0000","last_became_peered":"2024-09-20T18:49:47.955335+0000","last_unstale":"2024-09-20T18:58:25.772257+0000","last_undegraded":"2024-09-20T18:58:25.772257+0000","last_fullsized":"2024-09-20T18:58:25.772257+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'28","last_scrub_stamp":"2024-09-20T18:58:07.018340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:07.018340+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:28.256912+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":398,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.731845+0000","last_change":"2024-09-20T18:57:21.807755+0000","last_active":"2024-09-20T18:58:25.731845+0000","last_peered":"2024-09-20T18:58:25.731845+0000","last_clean":"2024-09-20T18:58:25.731845+0000","last_became_active":"2024-09-20T18:55:28.831574+0000","last_became_peered":"2024-09-20T18:55:28.831574+0000","last_unstale":"2024-09-20T18:58:25.731845+0000","last_undegraded":"2024-09-20T18:58:25.731845+0000","last_fullsized":"2024-09-20T18:58:25.731845+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:21.807703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:21.807703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:42.821495+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1],"acting":[4,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":432,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.307758+0000","last_change":"2024-09-20T18:55:28.854615+0000","last_active":"2024-09-20T18:58:25.307758+0000","last_peered":"2024-09-20T18:58:25.307758+0000","last_clean":"2024-09-20T18:58:25.307758+0000","last_became_active":"2024-09-20T18:55:28.837226+0000","last_became_peered":"2024-09-20T18:55:28.837226+0000","last_unstale":"2024-09-20T18:58:25.307758+0000","last_undegraded":"2024-09-20T18:58:25.307758+0000","last_fullsized":"2024-09-20T18:58:25.307758+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-20T18:54:51.227531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.227531+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:43.598074+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0012781159999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":398,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:26.104368+0000","last_change":"2024-09-20T18:55:13.949348+0000","last_active":"2024-09-20T18:58:26.104368+0000","last_peered":"2024-09-20T18:58:26.104368+0000","last_clean":"2024-09-20T18:58:26.104368+0000","last_became_active":"2024-09-20T18:49:49.953971+0000","last_became_peered":"2024-09-20T18:49:49.953971+0000","last_unstale":"2024-09-20T18:58:26.104368+0000","last_undegraded":"2024-09-20T18:58:26.104368+0000","last_fullsized":"2024-09-20T18:58:26.104368+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:13.949269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:13.949269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":377,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.772075+0000","last_change":"2024-09-20T18:57:12.744116+0000","last_active":"2024-09-20T18:58:25.772075+0000","last_peered":"2024-09-20T18:58:25.772075+0000","last_clean":"2024-09-20T18:58:25.772075+0000","last_became_active":"2024-09-20T18:49:51.969400+0000","last_became_peered":"2024-09-20T18:49:51.969400+0000","last_unstale":"2024-09-20T18:58:25.772075+0000","last_undegraded":"2024-09-20T18:58:25.772075+0000","last_fullsized":"2024-09-20T18:58:25.772075+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:12.744040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:12.744040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":436,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.772793+0000","last_change":"2024-09-20T18:58:06.079117+0000","last_active":"2024-09-20T18:58:25.772793+0000","last_peered":"2024-09-20T18:58:25.772793+0000","last_clean":"2024-09-20T18:58:25.772793+0000","last_became_active":"2024-09-20T18:49:47.950904+0000","last_became_peered":"2024-09-20T18:49:47.950904+0000","last_unstale":"2024-09-20T18:58:25.772793+0000","last_undegraded":"2024-09-20T18:58:25.772793+0000","last_fullsized":"2024-09-20T18:58:25.772793+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-20T18:58:06.078352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:06.078352+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:26.574178+0000","scrub_duration":59,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":413,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.736998+0000","last_change":"2024-09-20T18:55:28.830832+0000","last_active":"2024-09-20T18:58:25.736998+0000","last_peered":"2024-09-20T18:58:25.736998+0000","last_clean":"2024-09-20T18:58:25.736998+0000","last_became_active":"2024-09-20T18:55:28.830269+0000","last_became_peered":"2024-09-20T18:55:28.830269+0000","last_unstale":"2024-09-20T18:58:25.736998+0000","last_undegraded":"2024-09-20T18:58:25.736998+0000","last_fullsized":"2024-09-20T18:58:25.736998+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:23.359769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:23.359769+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:27.139436+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":409,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:26.217395+0000","last_change":"2024-09-20T18:58:24.229997+0000","last_active":"2024-09-20T18:58:26.217395+0000","last_peered":"2024-09-20T18:58:26.217395+0000","last_clean":"2024-09-20T18:58:26.217395+0000","last_became_active":"2024-09-20T18:49:44.937135+0000","last_became_peered":"2024-09-20T18:49:44.937135+0000","last_unstale":"2024-09-20T18:58:26.217395+0000","last_undegraded":"2024-09-20T18:58:26.217395+0000","last_fullsized":"2024-09-20T18:58:26.217395+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:58:24.229938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:58:24.229938+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:47.181712+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00095558800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":384,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:26.273141+0000","last_change":"2024-09-20T18:51:59.796450+0000","last_active":"2024-09-20T18:58:26.273141+0000","last_peered":"2024-09-20T18:58:26.273141+0000","last_clean":"2024-09-20T18:58:26.273141+0000","last_became_active":"2024-09-20T18:49:49.954424+0000","last_became_peered":"2024-09-20T18:49:49.954424+0000","last_unstale":"2024-09-20T18:58:26.273141+0000","last_undegraded":"2024-09-20T18:58:26.273141+0000","last_fullsized":"2024-09-20T18:58:26.273141+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:59.796383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:59.796383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:21.220437+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":383,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.501051+0000","last_change":"2024-09-20T18:55:28.826659+0000","last_active":"2024-09-20T18:58:25.501051+0000","last_peered":"2024-09-20T18:58:25.501051+0000","last_clean":"2024-09-20T18:58:25.501051+0000","last_became_active":"2024-09-20T18:55:28.825812+0000","last_became_peered":"2024-09-20T18:55:28.825812+0000","last_unstale":"2024-09-20T18:58:25.501051+0000","last_undegraded":"2024-09-20T18:58:25.501051+0000","last_fullsized":"2024-09-20T18:58:25.501051+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:39.814259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:39.814259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":427,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:26.104445+0000","last_change":"2024-09-20T18:55:14.951950+0000","last_active":"2024-09-20T18:58:26.104445+0000","last_peered":"2024-09-20T18:58:26.104445+0000","last_clean":"2024-09-20T18:58:26.104445+0000","last_became_active":"2024-09-20T18:49:47.949230+0000","last_became_peered":"2024-09-20T18:49:47.949230+0000","last_unstale":"2024-09-20T18:58:26.104445+0000","last_undegraded":"2024-09-20T18:58:26.104445+0000","last_fullsized":"2024-09-20T18:58:26.104445+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:55:14.951892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:14.951892+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":406,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:26.103513+0000","last_change":"2024-09-20T18:54:27.905856+0000","last_active":"2024-09-20T18:58:26.103513+0000","last_peered":"2024-09-20T18:58:26.103513+0000","last_clean":"2024-09-20T18:58:26.103513+0000","last_became_active":"2024-09-20T18:49:45.939197+0000","last_became_peered":"2024-09-20T18:49:45.939197+0000","last_unstale":"2024-09-20T18:58:26.103513+0000","last_undegraded":"2024-09-20T18:58:26.103513+0000","last_fullsized":"2024-09-20T18:58:26.103513+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:27.905803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:27.905803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":929,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.500413+0000","last_change":"2024-09-20T18:54:26.359887+0000","last_active":"2024-09-20T18:58:25.500413+0000","last_peered":"2024-09-20T18:58:25.500413+0000","last_clean":"2024-09-20T18:58:25.500413+0000","last_became_active":"2024-09-20T18:49:44.936819+0000","last_became_peered":"2024-09-20T18:49:44.936819+0000","last_unstale":"2024-09-20T18:58:25.500413+0000","last_undegraded":"2024-09-20T18:58:25.500413+0000","last_fullsized":"2024-09-20T18:58:25.500413+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-20T18:54:26.359807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:54:26.359807+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0010996199999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":514,"num_read_kb":509,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":394,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.308436+0000","last_change":"2024-09-20T18:57:54.362085+0000","last_active":"2024-09-20T18:58:25.308436+0000","last_peered":"2024-09-20T18:58:25.308436+0000","last_clean":"2024-09-20T18:58:25.308436+0000","last_became_active":"2024-09-20T18:49:49.961066+0000","last_became_peered":"2024-09-20T18:49:49.961066+0000","last_unstale":"2024-09-20T18:58:25.308436+0000","last_undegraded":"2024-09-20T18:58:25.308436+0000","last_fullsized":"2024-09-20T18:58:25.308436+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:54.362038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:54.362038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:08.560302+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":402,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.771514+0000","last_change":"2024-09-20T18:58:08.019154+0000","last_active":"2024-09-20T18:58:25.771514+0000","last_peered":"2024-09-20T18:58:25.771514+0000","last_clean":"2024-09-20T18:58:25.771514+0000","last_became_active":"2024-09-20T18:49:51.970343+0000","last_became_peered":"2024-09-20T18:49:51.970343+0000","last_unstale":"2024-09-20T18:58:25.771514+0000","last_undegraded":"2024-09-20T18:58:25.771514+0000","last_fullsized":"2024-09-20T18:58:25.771514+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:08.019092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:08.019092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:32.791415+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"184'17","reported_seq":241,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.731978+0000","last_change":"2024-09-20T18:57:22.816240+0000","last_active":"2024-09-20T18:58:25.731978+0000","last_peered":"2024-09-20T18:58:25.731978+0000","last_clean":"2024-09-20T18:58:25.731978+0000","last_became_active":"2024-09-20T18:55:28.831024+0000","last_became_peered":"2024-09-20T18:55:28.831024+0000","last_unstale":"2024-09-20T18:58:25.731978+0000","last_undegraded":"2024-09-20T18:58:25.731978+0000","last_fullsized":"2024-09-20T18:58:25.731978+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'17","last_scrub_stamp":"2024-09-20T18:57:22.816154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:22.816154+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:28.727289+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":24,"num_read_kb":14,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":405,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.736356+0000","last_change":"2024-09-20T18:55:28.830447+0000","last_active":"2024-09-20T18:58:25.736356+0000","last_peered":"2024-09-20T18:58:25.736356+0000","last_clean":"2024-09-20T18:58:25.736356+0000","last_became_active":"2024-09-20T18:55:28.830147+0000","last_became_peered":"2024-09-20T18:55:28.830147+0000","last_unstale":"2024-09-20T18:58:25.736356+0000","last_undegraded":"2024-09-20T18:58:25.736356+0000","last_fullsized":"2024-09-20T18:58:25.736356+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:13.144896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:53:13.144896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:28.731761+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":457,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.500507+0000","last_change":"2024-09-20T18:55:28.844060+0000","last_active":"2024-09-20T18:58:25.500507+0000","last_peered":"2024-09-20T18:58:25.500507+0000","last_clean":"2024-09-20T18:58:25.500507+0000","last_became_active":"2024-09-20T18:55:28.829325+0000","last_became_peered":"2024-09-20T18:55:28.829325+0000","last_unstale":"2024-09-20T18:58:25.500507+0000","last_undegraded":"2024-09-20T18:58:25.500507+0000","last_fullsized":"2024-09-20T18:58:25.500507+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T18:55:13.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T18:55:13.696607+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":388,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.500811+0000","last_change":"2024-09-20T18:58:23.545370+0000","last_active":"2024-09-20T18:58:25.500811+0000","last_peered":"2024-09-20T18:58:25.500811+0000","last_clean":"2024-09-20T18:58:25.500811+0000","last_became_active":"2024-09-20T18:49:49.956128+0000","last_became_peered":"2024-09-20T18:49:49.956128+0000","last_unstale":"2024-09-20T18:58:25.500811+0000","last_undegraded":"2024-09-20T18:58:25.500811+0000","last_fullsized":"2024-09-20T18:58:25.500811+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:23.545321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:23.545321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:34.942242+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":398,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.308298+0000","last_change":"2024-09-20T18:58:03.523127+0000","last_active":"2024-09-20T18:58:25.308298+0000","last_peered":"2024-09-20T18:58:25.308298+0000","last_clean":"2024-09-20T18:58:25.308298+0000","last_became_active":"2024-09-20T18:49:51.971332+0000","last_became_peered":"2024-09-20T18:49:51.971332+0000","last_unstale":"2024-09-20T18:58:25.308298+0000","last_undegraded":"2024-09-20T18:58:25.308298+0000","last_fullsized":"2024-09-20T18:58:25.308298+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:03.523078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:03.523078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:14.059180+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":432,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.731916+0000","last_change":"2024-09-20T18:58:24.616639+0000","last_active":"2024-09-20T18:58:25.731916+0000","last_peered":"2024-09-20T18:58:25.731916+0000","last_clean":"2024-09-20T18:58:25.731916+0000","last_became_active":"2024-09-20T18:55:28.818269+0000","last_became_peered":"2024-09-20T18:55:28.818269+0000","last_unstale":"2024-09-20T18:58:25.731916+0000","last_undegraded":"2024-09-20T18:58:25.731916+0000","last_fullsized":"2024-09-20T18:58:25.731916+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:58:24.616580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:24.616580+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:24.623208+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":391,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.731612+0000","last_change":"2024-09-20T18:51:18.410601+0000","last_active":"2024-09-20T18:58:25.731612+0000","last_peered":"2024-09-20T18:58:25.731612+0000","last_clean":"2024-09-20T18:58:25.731612+0000","last_became_active":"2024-09-20T18:49:45.944579+0000","last_became_peered":"2024-09-20T18:49:45.944579+0000","last_unstale":"2024-09-20T18:58:25.731612+0000","last_undegraded":"2024-09-20T18:58:25.731612+0000","last_fullsized":"2024-09-20T18:58:25.731612+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:18.410447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:51:18.410447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:29.917821+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":401,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.308236+0000","last_change":"2024-09-20T18:55:28.830634+0000","last_active":"2024-09-20T18:58:25.308236+0000","last_peered":"2024-09-20T18:58:25.308236+0000","last_clean":"2024-09-20T18:58:25.308236+0000","last_became_active":"2024-09-20T18:55:28.827785+0000","last_became_peered":"2024-09-20T18:55:28.827785+0000","last_unstale":"2024-09-20T18:58:25.308236+0000","last_undegraded":"2024-09-20T18:58:25.308236+0000","last_fullsized":"2024-09-20T18:58:25.308236+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:11.401777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:11.401777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:37.550724+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":392,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.736388+0000","last_change":"2024-09-20T18:57:23.721754+0000","last_active":"2024-09-20T18:58:25.736388+0000","last_peered":"2024-09-20T18:58:25.736388+0000","last_clean":"2024-09-20T18:58:25.736388+0000","last_became_active":"2024-09-20T18:55:28.833115+0000","last_became_peered":"2024-09-20T18:55:28.833115+0000","last_unstale":"2024-09-20T18:58:25.736388+0000","last_undegraded":"2024-09-20T18:58:25.736388+0000","last_fullsized":"2024-09-20T18:58:25.736388+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:23.721643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:23.721643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:25.737391+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":422,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.732049+0000","last_change":"2024-09-20T18:52:02.944536+0000","last_active":"2024-09-20T18:58:25.732049+0000","last_peered":"2024-09-20T18:58:25.732049+0000","last_clean":"2024-09-20T18:58:25.732049+0000","last_became_active":"2024-09-20T18:49:47.947636+0000","last_became_peered":"2024-09-20T18:49:47.947636+0000","last_unstale":"2024-09-20T18:58:25.732049+0000","last_undegraded":"2024-09-20T18:58:25.732049+0000","last_fullsized":"2024-09-20T18:58:25.732049+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-20T18:52:02.944434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:52:02.944434+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:34.896863+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":268,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.731346+0000","last_change":"2024-09-20T18:58:25.649818+0000","last_active":"2024-09-20T18:58:25.731346+0000","last_peered":"2024-09-20T18:58:25.731346+0000","last_clean":"2024-09-20T18:58:25.731346+0000","last_became_active":"2024-09-20T18:55:28.821959+0000","last_became_peered":"2024-09-20T18:55:28.821959+0000","last_unstale":"2024-09-20T18:58:25.731346+0000","last_undegraded":"2024-09-20T18:58:25.731346+0000","last_fullsized":"2024-09-20T18:58:25.731346+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:25.649733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:25.649733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:46.966857+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":389,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.772063+0000","last_change":"2024-09-20T18:54:11.377907+0000","last_active":"2024-09-20T18:58:25.772063+0000","last_peered":"2024-09-20T18:58:25.772063+0000","last_clean":"2024-09-20T18:58:25.772063+0000","last_became_active":"2024-09-20T18:49:49.959160+0000","last_became_peered":"2024-09-20T18:49:49.959160+0000","last_unstale":"2024-09-20T18:58:25.772063+0000","last_undegraded":"2024-09-20T18:58:25.772063+0000","last_fullsized":"2024-09-20T18:58:25.772063+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:11.376951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:11.376951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:29.061620+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":395,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:26.273214+0000","last_change":"2024-09-20T18:58:25.227679+0000","last_active":"2024-09-20T18:58:26.273214+0000","last_peered":"2024-09-20T18:58:26.273214+0000","last_clean":"2024-09-20T18:58:26.273214+0000","last_became_active":"2024-09-20T18:49:51.970108+0000","last_became_peered":"2024-09-20T18:49:51.970108+0000","last_unstale":"2024-09-20T18:58:26.273214+0000","last_undegraded":"2024-09-20T18:58:26.273214+0000","last_fullsized":"2024-09-20T18:58:26.273214+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:25.227630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:25.227630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:27.041924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":428,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.731766+0000","last_change":"2024-09-20T18:55:28.846773+0000","last_active":"2024-09-20T18:58:25.731766+0000","last_peered":"2024-09-20T18:58:25.731766+0000","last_clean":"2024-09-20T18:58:25.731766+0000","last_became_active":"2024-09-20T18:55:28.817101+0000","last_became_peered":"2024-09-20T18:55:28.817101+0000","last_unstale":"2024-09-20T18:58:25.731766+0000","last_undegraded":"2024-09-20T18:58:25.731766+0000","last_fullsized":"2024-09-20T18:58:25.731766+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:54:17.219548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:17.219548+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:12.738146+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":397,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.771635+0000","last_change":"2024-09-20T18:52:46.005172+0000","last_active":"2024-09-20T18:58:25.771635+0000","last_peered":"2024-09-20T18:58:25.771635+0000","last_clean":"2024-09-20T18:58:25.771635+0000","last_became_active":"2024-09-20T18:49:45.937264+0000","last_became_peered":"2024-09-20T18:49:45.937264+0000","last_unstale":"2024-09-20T18:58:25.771635+0000","last_undegraded":"2024-09-20T18:58:25.771635+0000","last_fullsized":"2024-09-20T18:58:25.771635+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:46.005109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:46.005109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:27.755958+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":512,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:26.217679+0000","last_change":"2024-09-20T18:55:17.549660+0000","last_active":"2024-09-20T18:58:26.217679+0000","last_peered":"2024-09-20T18:58:26.217679+0000","last_clean":"2024-09-20T18:58:26.217679+0000","last_became_active":"2024-09-20T18:49:49.957005+0000","last_became_peered":"2024-09-20T18:49:49.957005+0000","last_unstale":"2024-09-20T18:58:26.217679+0000","last_undegraded":"2024-09-20T18:58:26.217679+0000","last_fullsized":"2024-09-20T18:58:26.217679+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:17.549598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:17.549598+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":386,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.731785+0000","last_change":"2024-09-20T18:54:42.182875+0000","last_active":"2024-09-20T18:58:25.731785+0000","last_peered":"2024-09-20T18:58:25.731785+0000","last_clean":"2024-09-20T18:58:25.731785+0000","last_became_active":"2024-09-20T18:49:51.969355+0000","last_became_peered":"2024-09-20T18:49:51.969355+0000","last_unstale":"2024-09-20T18:58:25.731785+0000","last_undegraded":"2024-09-20T18:58:25.731785+0000","last_fullsized":"2024-09-20T18:58:25.731785+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:42.182809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:42.182809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":417,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:26.104222+0000","last_change":"2024-09-20T18:54:54.897991+0000","last_active":"2024-09-20T18:58:26.104222+0000","last_peered":"2024-09-20T18:58:26.104222+0000","last_clean":"2024-09-20T18:58:26.104222+0000","last_became_active":"2024-09-20T18:49:47.946008+0000","last_became_peered":"2024-09-20T18:49:47.946008+0000","last_unstale":"2024-09-20T18:58:26.104222+0000","last_undegraded":"2024-09-20T18:58:26.104222+0000","last_fullsized":"2024-09-20T18:58:26.104222+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:54.897933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:54.897933+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":417,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.307901+0000","last_change":"2024-09-20T18:57:56.405392+0000","last_active":"2024-09-20T18:58:25.307901+0000","last_peered":"2024-09-20T18:58:25.307901+0000","last_clean":"2024-09-20T18:58:25.307901+0000","last_became_active":"2024-09-20T18:55:28.833095+0000","last_became_peered":"2024-09-20T18:55:28.833095+0000","last_unstale":"2024-09-20T18:58:25.307901+0000","last_undegraded":"2024-09-20T18:58:25.307901+0000","last_fullsized":"2024-09-20T18:58:25.307901+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:56.405349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:56.405349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:22.442121+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":399,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:26.217519+0000","last_change":"2024-09-20T18:54:43.379898+0000","last_active":"2024-09-20T18:58:26.217519+0000","last_peered":"2024-09-20T18:58:26.217519+0000","last_clean":"2024-09-20T18:58:26.217519+0000","last_became_active":"2024-09-20T18:49:49.956767+0000","last_became_peered":"2024-09-20T18:49:49.956767+0000","last_unstale":"2024-09-20T18:58:26.217519+0000","last_undegraded":"2024-09-20T18:58:26.217519+0000","last_fullsized":"2024-09-20T18:58:26.217519+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:43.379706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:43.379706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":385,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:26.104043+0000","last_change":"2024-09-20T18:54:24.976627+0000","last_active":"2024-09-20T18:58:26.104043+0000","last_peered":"2024-09-20T18:58:26.104043+0000","last_clean":"2024-09-20T18:58:26.104043+0000","last_became_active":"2024-09-20T18:49:51.964098+0000","last_became_peered":"2024-09-20T18:49:51.964098+0000","last_unstale":"2024-09-20T18:58:26.104043+0000","last_undegraded":"2024-09-20T18:58:26.104043+0000","last_fullsized":"2024-09-20T18:58:26.104043+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:24.976584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:24.976584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":214,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.736777+0000","last_change":"2024-09-20T18:57:22.033602+0000","last_active":"2024-09-20T18:58:25.736777+0000","last_peered":"2024-09-20T18:58:25.736777+0000","last_clean":"2024-09-20T18:58:25.736777+0000","last_became_active":"2024-09-20T18:55:28.828563+0000","last_became_peered":"2024-09-20T18:55:28.828563+0000","last_unstale":"2024-09-20T18:58:25.736777+0000","last_undegraded":"2024-09-20T18:58:25.736777+0000","last_fullsized":"2024-09-20T18:58:25.736777+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-20T18:57:22.033475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:22.033475+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:43.998101+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":31,"num_read_kb":20,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,5],"acting":[1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":410,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.501206+0000","last_change":"2024-09-20T18:55:28.835767+0000","last_active":"2024-09-20T18:58:25.501206+0000","last_peered":"2024-09-20T18:58:25.501206+0000","last_clean":"2024-09-20T18:58:25.501206+0000","last_became_active":"2024-09-20T18:55:28.835149+0000","last_became_peered":"2024-09-20T18:55:28.835149+0000","last_unstale":"2024-09-20T18:58:25.501206+0000","last_undegraded":"2024-09-20T18:58:25.501206+0000","last_fullsized":"2024-09-20T18:58:25.501206+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:44.501192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.501192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":399,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:26.217593+0000","last_change":"2024-09-20T18:55:13.577772+0000","last_active":"2024-09-20T18:58:26.217593+0000","last_peered":"2024-09-20T18:58:26.217593+0000","last_clean":"2024-09-20T18:58:26.217593+0000","last_became_active":"2024-09-20T18:49:49.957558+0000","last_became_peered":"2024-09-20T18:49:49.957558+0000","last_unstale":"2024-09-20T18:58:26.217593+0000","last_undegraded":"2024-09-20T18:58:26.217593+0000","last_fullsized":"2024-09-20T18:58:26.217593+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:13.577699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:13.577699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":391,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.500406+0000","last_change":"2024-09-20T18:53:47.370152+0000","last_active":"2024-09-20T18:58:25.500406+0000","last_peered":"2024-09-20T18:58:25.500406+0000","last_clean":"2024-09-20T18:58:25.500406+0000","last_became_active":"2024-09-20T18:49:51.970798+0000","last_became_peered":"2024-09-20T18:49:51.970798+0000","last_unstale":"2024-09-20T18:58:25.500406+0000","last_undegraded":"2024-09-20T18:58:25.500406+0000","last_fullsized":"2024-09-20T18:58:25.500406+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:47.370108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:47.370108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":417,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.731487+0000","last_change":"2024-09-20T18:53:52.898818+0000","last_active":"2024-09-20T18:58:25.731487+0000","last_peered":"2024-09-20T18:58:25.731487+0000","last_clean":"2024-09-20T18:58:25.731487+0000","last_became_active":"2024-09-20T18:49:47.949116+0000","last_became_peered":"2024-09-20T18:49:47.949116+0000","last_unstale":"2024-09-20T18:58:25.731487+0000","last_undegraded":"2024-09-20T18:58:25.731487+0000","last_fullsized":"2024-09-20T18:58:25.731487+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:53:52.898760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:52.898760+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:42.918366+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":403,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.501023+0000","last_change":"2024-09-20T18:54:52.650798+0000","last_active":"2024-09-20T18:58:25.501023+0000","last_peered":"2024-09-20T18:58:25.501023+0000","last_clean":"2024-09-20T18:58:25.501023+0000","last_became_active":"2024-09-20T18:49:45.944858+0000","last_became_peered":"2024-09-20T18:49:45.944858+0000","last_unstale":"2024-09-20T18:58:25.501023+0000","last_undegraded":"2024-09-20T18:58:25.501023+0000","last_fullsized":"2024-09-20T18:58:25.501023+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:52.650756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:52.650756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":391,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:26.217361+0000","last_change":"2024-09-20T18:54:42.349879+0000","last_active":"2024-09-20T18:58:26.217361+0000","last_peered":"2024-09-20T18:58:26.217361+0000","last_clean":"2024-09-20T18:58:26.217361+0000","last_became_active":"2024-09-20T18:49:49.954944+0000","last_became_peered":"2024-09-20T18:49:49.954944+0000","last_unstale":"2024-09-20T18:58:26.217361+0000","last_undegraded":"2024-09-20T18:58:26.217361+0000","last_fullsized":"2024-09-20T18:58:26.217361+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:42.349807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:42.349807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":393,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.736320+0000","last_change":"2024-09-20T18:58:24.162921+0000","last_active":"2024-09-20T18:58:25.736320+0000","last_peered":"2024-09-20T18:58:25.736320+0000","last_clean":"2024-09-20T18:58:25.736320+0000","last_became_active":"2024-09-20T18:49:51.972970+0000","last_became_peered":"2024-09-20T18:49:51.972970+0000","last_unstale":"2024-09-20T18:58:25.736320+0000","last_undegraded":"2024-09-20T18:58:25.736320+0000","last_fullsized":"2024-09-20T18:58:25.736320+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:24.162879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:24.162879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:46.703799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":439,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.307422+0000","last_change":"2024-09-20T18:55:28.847010+0000","last_active":"2024-09-20T18:58:25.307422+0000","last_peered":"2024-09-20T18:58:25.307422+0000","last_clean":"2024-09-20T18:58:25.307422+0000","last_became_active":"2024-09-20T18:55:28.824801+0000","last_became_peered":"2024-09-20T18:55:28.824801+0000","last_unstale":"2024-09-20T18:58:25.307422+0000","last_undegraded":"2024-09-20T18:58:25.307422+0000","last_fullsized":"2024-09-20T18:58:25.307422+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:54:12.373309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:12.373309+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:29.516079+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":423,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.501002+0000","last_change":"2024-09-20T18:55:24.729301+0000","last_active":"2024-09-20T18:58:25.501002+0000","last_peered":"2024-09-20T18:58:25.501002+0000","last_clean":"2024-09-20T18:58:25.501002+0000","last_became_active":"2024-09-20T18:49:45.944256+0000","last_became_peered":"2024-09-20T18:49:45.944256+0000","last_unstale":"2024-09-20T18:58:25.501002+0000","last_undegraded":"2024-09-20T18:58:25.501002+0000","last_fullsized":"2024-09-20T18:58:25.501002+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-20T18:55:24.729219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:24.729219+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"184'9","reported_seq":502,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.500373+0000","last_change":"2024-09-20T18:55:28.824771+0000","last_active":"2024-09-20T18:58:25.500373+0000","last_peered":"2024-09-20T18:58:25.500373+0000","last_clean":"2024-09-20T18:58:25.500373+0000","last_became_active":"2024-09-20T18:55:28.818085+0000","last_became_peered":"2024-09-20T18:55:28.818085+0000","last_unstale":"2024-09-20T18:58:25.500373+0000","last_undegraded":"2024-09-20T18:58:25.500373+0000","last_fullsized":"2024-09-20T18:58:25.500373+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:52:05.788924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.788924+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":397,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.307459+0000","last_change":"2024-09-20T18:58:05.523446+0000","last_active":"2024-09-20T18:58:25.307459+0000","last_peered":"2024-09-20T18:58:25.307459+0000","last_clean":"2024-09-20T18:58:25.307459+0000","last_became_active":"2024-09-20T18:55:28.830380+0000","last_became_peered":"2024-09-20T18:55:28.830380+0000","last_unstale":"2024-09-20T18:58:25.307459+0000","last_undegraded":"2024-09-20T18:58:25.307459+0000","last_fullsized":"2024-09-20T18:58:25.307459+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:05.523359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:05.523359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:28.858918+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":136,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:26.103801+0000","last_change":"2024-09-20T18:55:28.835344+0000","last_active":"2024-09-20T18:58:26.103801+0000","last_peered":"2024-09-20T18:58:26.103801+0000","last_clean":"2024-09-20T18:58:26.103801+0000","last_became_active":"2024-09-20T18:55:28.835010+0000","last_became_peered":"2024-09-20T18:55:28.835010+0000","last_unstale":"2024-09-20T18:58:26.103801+0000","last_undegraded":"2024-09-20T18:58:26.103801+0000","last_fullsized":"2024-09-20T18:58:26.103801+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:26.721735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:26.721735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":410,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.772388+0000","last_change":"2024-09-20T18:58:09.063555+0000","last_active":"2024-09-20T18:58:25.772388+0000","last_peered":"2024-09-20T18:58:25.772388+0000","last_clean":"2024-09-20T18:58:25.772388+0000","last_became_active":"2024-09-20T18:49:48.500662+0000","last_became_peered":"2024-09-20T18:49:48.500662+0000","last_unstale":"2024-09-20T18:58:25.772388+0000","last_undegraded":"2024-09-20T18:58:25.772388+0000","last_fullsized":"2024-09-20T18:58:25.772388+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-20T18:58:09.063481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:09.063481+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:21.861420+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":386,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.308109+0000","last_change":"2024-09-20T18:53:16.220554+0000","last_active":"2024-09-20T18:58:25.308109+0000","last_peered":"2024-09-20T18:58:25.308109+0000","last_clean":"2024-09-20T18:58:25.308109+0000","last_became_active":"2024-09-20T18:49:49.956749+0000","last_became_peered":"2024-09-20T18:49:49.956749+0000","last_unstale":"2024-09-20T18:58:25.308109+0000","last_undegraded":"2024-09-20T18:58:25.308109+0000","last_fullsized":"2024-09-20T18:58:25.308109+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:16.220490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:16.220490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:27.545117+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":398,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.501045+0000","last_change":"2024-09-20T18:55:28.827897+0000","last_active":"2024-09-20T18:58:25.501045+0000","last_peered":"2024-09-20T18:58:25.501045+0000","last_clean":"2024-09-20T18:58:25.501045+0000","last_became_active":"2024-09-20T18:55:28.827418+0000","last_became_peered":"2024-09-20T18:55:28.827418+0000","last_unstale":"2024-09-20T18:58:25.501045+0000","last_undegraded":"2024-09-20T18:58:25.501045+0000","last_fullsized":"2024-09-20T18:58:25.501045+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.637739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.637739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":409,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:26.104490+0000","last_change":"2024-09-20T18:54:44.830872+0000","last_active":"2024-09-20T18:58:26.104490+0000","last_peered":"2024-09-20T18:58:26.104490+0000","last_clean":"2024-09-20T18:58:26.104490+0000","last_became_active":"2024-09-20T18:49:48.500803+0000","last_became_peered":"2024-09-20T18:49:48.500803+0000","last_unstale":"2024-09-20T18:58:26.104490+0000","last_undegraded":"2024-09-20T18:58:26.104490+0000","last_fullsized":"2024-09-20T18:58:26.104490+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:44.830790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.830790+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":402,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.500471+0000","last_change":"2024-09-20T18:55:29.751718+0000","last_active":"2024-09-20T18:58:25.500471+0000","last_peered":"2024-09-20T18:58:25.500471+0000","last_clean":"2024-09-20T18:58:25.500471+0000","last_became_active":"2024-09-20T18:55:28.833799+0000","last_became_peered":"2024-09-20T18:55:28.833799+0000","last_unstale":"2024-09-20T18:58:25.500471+0000","last_undegraded":"2024-09-20T18:58:25.500471+0000","last_fullsized":"2024-09-20T18:58:25.500471+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:29.751642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:29.751642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":397,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.772296+0000","last_change":"2024-09-20T18:57:46.898929+0000","last_active":"2024-09-20T18:58:25.772296+0000","last_peered":"2024-09-20T18:58:25.772296+0000","last_clean":"2024-09-20T18:58:25.772296+0000","last_became_active":"2024-09-20T18:49:49.961947+0000","last_became_peered":"2024-09-20T18:49:49.961947+0000","last_unstale":"2024-09-20T18:58:25.772296+0000","last_undegraded":"2024-09-20T18:58:25.772296+0000","last_fullsized":"2024-09-20T18:58:25.772296+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:46.898862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:46.898862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:06.640666+0000","scrub_duration":38,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"66'1","reported_seq":396,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:26.217590+0000","last_change":"2024-09-20T18:54:20.681878+0000","last_active":"2024-09-20T18:58:26.217590+0000","last_peered":"2024-09-20T18:58:26.217590+0000","last_clean":"2024-09-20T18:58:26.217590+0000","last_became_active":"2024-09-20T18:49:51.965121+0000","last_became_peered":"2024-09-20T18:49:51.965121+0000","last_unstale":"2024-09-20T18:58:26.217590+0000","last_undegraded":"2024-09-20T18:58:26.217590+0000","last_fullsized":"2024-09-20T18:58:26.217590+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T18:54:20.681790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:20.681790+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":406,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.308145+0000","last_change":"2024-09-20T18:57:59.430353+0000","last_active":"2024-09-20T18:58:25.308145+0000","last_peered":"2024-09-20T18:58:25.308145+0000","last_clean":"2024-09-20T18:58:25.308145+0000","last_became_active":"2024-09-20T18:49:48.501011+0000","last_became_peered":"2024-09-20T18:49:48.501011+0000","last_unstale":"2024-09-20T18:58:25.308145+0000","last_undegraded":"2024-09-20T18:58:25.308145+0000","last_fullsized":"2024-09-20T18:58:25.308145+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:57:59.430294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:59.430294+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:21.333603+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":405,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:26.103398+0000","last_change":"2024-09-20T18:55:27.028184+0000","last_active":"2024-09-20T18:58:26.103398+0000","last_peered":"2024-09-20T18:58:26.103398+0000","last_clean":"2024-09-20T18:58:26.103398+0000","last_became_active":"2024-09-20T18:49:45.938511+0000","last_became_peered":"2024-09-20T18:49:45.938511+0000","last_unstale":"2024-09-20T18:58:26.103398+0000","last_undegraded":"2024-09-20T18:58:26.103398+0000","last_fullsized":"2024-09-20T18:58:26.103398+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:27.028126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:27.028126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":510,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.771736+0000","last_change":"2024-09-20T18:55:27.284156+0000","last_active":"2024-09-20T18:58:25.771736+0000","last_peered":"2024-09-20T18:58:25.771736+0000","last_clean":"2024-09-20T18:58:25.771736+0000","last_became_active":"2024-09-20T18:49:49.960550+0000","last_became_peered":"2024-09-20T18:49:49.960550+0000","last_unstale":"2024-09-20T18:58:25.771736+0000","last_undegraded":"2024-09-20T18:58:25.771736+0000","last_fullsized":"2024-09-20T18:58:25.771736+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:27.284086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:27.284086+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":394,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.732018+0000","last_change":"2024-09-20T18:55:12.348079+0000","last_active":"2024-09-20T18:58:25.732018+0000","last_peered":"2024-09-20T18:58:25.732018+0000","last_clean":"2024-09-20T18:58:25.732018+0000","last_became_active":"2024-09-20T18:49:51.966254+0000","last_became_peered":"2024-09-20T18:49:51.966254+0000","last_unstale":"2024-09-20T18:58:25.732018+0000","last_undegraded":"2024-09-20T18:58:25.732018+0000","last_fullsized":"2024-09-20T18:58:25.732018+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:12.347993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:12.347993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":427,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.771613+0000","last_change":"2024-09-20T18:57:20.701592+0000","last_active":"2024-09-20T18:58:25.771613+0000","last_peered":"2024-09-20T18:58:25.771613+0000","last_clean":"2024-09-20T18:58:25.771613+0000","last_became_active":"2024-09-20T18:49:58.997494+0000","last_became_peered":"2024-09-20T18:49:58.997494+0000","last_unstale":"2024-09-20T18:58:25.771613+0000","last_undegraded":"2024-09-20T18:58:25.771613+0000","last_fullsized":"2024-09-20T18:58:25.771613+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:57:20.701543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:20.701543+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:44.352173+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":403,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.500851+0000","last_change":"2024-09-20T18:54:57.632458+0000","last_active":"2024-09-20T18:58:25.500851+0000","last_peered":"2024-09-20T18:58:25.500851+0000","last_clean":"2024-09-20T18:58:25.500851+0000","last_became_active":"2024-09-20T18:49:45.949492+0000","last_became_peered":"2024-09-20T18:49:45.949492+0000","last_unstale":"2024-09-20T18:58:25.500851+0000","last_undegraded":"2024-09-20T18:58:25.500851+0000","last_fullsized":"2024-09-20T18:58:25.500851+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.632380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.632380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":506,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.307989+0000","last_change":"2024-09-20T18:55:28.302353+0000","last_active":"2024-09-20T18:58:25.307989+0000","last_peered":"2024-09-20T18:58:25.307989+0000","last_clean":"2024-09-20T18:58:25.307989+0000","last_became_active":"2024-09-20T18:49:49.962243+0000","last_became_peered":"2024-09-20T18:49:49.962243+0000","last_unstale":"2024-09-20T18:58:25.307989+0000","last_undegraded":"2024-09-20T18:58:25.307989+0000","last_fullsized":"2024-09-20T18:58:25.307989+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:28.302303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:28.302303+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:17.501674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":393,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:26.104289+0000","last_change":"2024-09-20T18:55:11.918284+0000","last_active":"2024-09-20T18:58:26.104289+0000","last_peered":"2024-09-20T18:58:26.104289+0000","last_clean":"2024-09-20T18:58:26.104289+0000","last_became_active":"2024-09-20T18:49:51.969093+0000","last_became_peered":"2024-09-20T18:49:51.969093+0000","last_unstale":"2024-09-20T18:58:26.104289+0000","last_undegraded":"2024-09-20T18:58:26.104289+0000","last_fullsized":"2024-09-20T18:58:26.104289+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:11.918210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:11.918210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":440,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.731659+0000","last_change":"2024-09-20T18:57:50.899576+0000","last_active":"2024-09-20T18:58:25.731659+0000","last_peered":"2024-09-20T18:58:25.731659+0000","last_clean":"2024-09-20T18:58:25.731659+0000","last_became_active":"2024-09-20T18:55:28.826721+0000","last_became_peered":"2024-09-20T18:55:28.826721+0000","last_unstale":"2024-09-20T18:58:25.731659+0000","last_undegraded":"2024-09-20T18:58:25.731659+0000","last_fullsized":"2024-09-20T18:58:25.731659+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:57:50.899514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:50.899514+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:12.210010+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":397,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.731968+0000","last_change":"2024-09-20T18:55:28.822690+0000","last_active":"2024-09-20T18:58:25.731968+0000","last_peered":"2024-09-20T18:58:25.731968+0000","last_clean":"2024-09-20T18:58:25.731968+0000","last_became_active":"2024-09-20T18:55:28.822258+0000","last_became_peered":"2024-09-20T18:55:28.822258+0000","last_unstale":"2024-09-20T18:58:25.731968+0000","last_undegraded":"2024-09-20T18:58:25.731968+0000","last_fullsized":"2024-09-20T18:58:25.731968+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:05.985551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.985551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:37.926823+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":389,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.736768+0000","last_change":"2024-09-20T18:57:11.986605+0000","last_active":"2024-09-20T18:58:25.736768+0000","last_peered":"2024-09-20T18:58:25.736768+0000","last_clean":"2024-09-20T18:58:25.736768+0000","last_became_active":"2024-09-20T18:49:49.957067+0000","last_became_peered":"2024-09-20T18:49:49.957067+0000","last_unstale":"2024-09-20T18:58:25.736768+0000","last_undegraded":"2024-09-20T18:58:25.736768+0000","last_fullsized":"2024-09-20T18:58:25.736768+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:11.986528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:11.986528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:36.007001+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":385,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.735961+0000","last_change":"2024-09-20T18:57:16.789412+0000","last_active":"2024-09-20T18:58:25.735961+0000","last_peered":"2024-09-20T18:58:25.735961+0000","last_clean":"2024-09-20T18:58:25.735961+0000","last_became_active":"2024-09-20T18:49:51.966915+0000","last_became_peered":"2024-09-20T18:49:51.966915+0000","last_unstale":"2024-09-20T18:58:25.735961+0000","last_undegraded":"2024-09-20T18:58:25.735961+0000","last_fullsized":"2024-09-20T18:58:25.735961+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:16.789095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:16.789095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:45.821909+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":434,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.736899+0000","last_change":"2024-09-20T18:55:28.838698+0000","last_active":"2024-09-20T18:58:25.736899+0000","last_peered":"2024-09-20T18:58:25.736899+0000","last_clean":"2024-09-20T18:58:25.736899+0000","last_became_active":"2024-09-20T18:55:28.827023+0000","last_became_peered":"2024-09-20T18:55:28.827023+0000","last_unstale":"2024-09-20T18:58:25.736899+0000","last_undegraded":"2024-09-20T18:58:25.736899+0000","last_fullsized":"2024-09-20T18:58:25.736899+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:53:53.263976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:53.263976+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:26.131891+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":395,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.501156+0000","last_change":"2024-09-20T18:54:42.487465+0000","last_active":"2024-09-20T18:58:25.501156+0000","last_peered":"2024-09-20T18:58:25.501156+0000","last_clean":"2024-09-20T18:58:25.501156+0000","last_became_active":"2024-09-20T18:49:45.944357+0000","last_became_peered":"2024-09-20T18:49:45.944357+0000","last_unstale":"2024-09-20T18:58:25.501156+0000","last_undegraded":"2024-09-20T18:58:25.501156+0000","last_fullsized":"2024-09-20T18:58:25.501156+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:42.487403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:42.487403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":397,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.736409+0000","last_change":"2024-09-20T18:57:57.915090+0000","last_active":"2024-09-20T18:58:25.736409+0000","last_peered":"2024-09-20T18:58:25.736409+0000","last_clean":"2024-09-20T18:58:25.736409+0000","last_became_active":"2024-09-20T18:49:49.962523+0000","last_became_peered":"2024-09-20T18:49:49.962523+0000","last_unstale":"2024-09-20T18:58:25.736409+0000","last_undegraded":"2024-09-20T18:58:25.736409+0000","last_fullsized":"2024-09-20T18:58:25.736409+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:57.914913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:57.914913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:01.797575+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":393,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:26.103640+0000","last_change":"2024-09-20T18:54:26.882441+0000","last_active":"2024-09-20T18:58:26.103640+0000","last_peered":"2024-09-20T18:58:26.103640+0000","last_clean":"2024-09-20T18:58:26.103640+0000","last_became_active":"2024-09-20T18:49:51.967313+0000","last_became_peered":"2024-09-20T18:49:51.967313+0000","last_unstale":"2024-09-20T18:58:26.103640+0000","last_undegraded":"2024-09-20T18:58:26.103640+0000","last_fullsized":"2024-09-20T18:58:26.103640+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:26.882375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:26.882375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":439,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.736055+0000","last_change":"2024-09-20T18:57:54.862785+0000","last_active":"2024-09-20T18:58:25.736055+0000","last_peered":"2024-09-20T18:58:25.736055+0000","last_clean":"2024-09-20T18:58:25.736055+0000","last_became_active":"2024-09-20T18:55:28.830801+0000","last_became_peered":"2024-09-20T18:55:28.830801+0000","last_unstale":"2024-09-20T18:58:25.736055+0000","last_undegraded":"2024-09-20T18:58:25.736055+0000","last_fullsized":"2024-09-20T18:58:25.736055+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:57:54.862329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:54.862329+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:09.269532+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":410,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.308160+0000","last_change":"2024-09-20T18:58:06.479419+0000","last_active":"2024-09-20T18:58:25.308160+0000","last_peered":"2024-09-20T18:58:25.308160+0000","last_clean":"2024-09-20T18:58:25.308160+0000","last_became_active":"2024-09-20T18:49:45.942105+0000","last_became_peered":"2024-09-20T18:49:45.942105+0000","last_unstale":"2024-09-20T18:58:25.308160+0000","last_undegraded":"2024-09-20T18:58:25.308160+0000","last_fullsized":"2024-09-20T18:58:25.308160+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:06.479364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:06.479364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:26.337228+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":402,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.500123+0000","last_change":"2024-09-20T18:55:28.825004+0000","last_active":"2024-09-20T18:58:25.500123+0000","last_peered":"2024-09-20T18:58:25.500123+0000","last_clean":"2024-09-20T18:58:25.500123+0000","last_became_active":"2024-09-20T18:55:28.824355+0000","last_became_peered":"2024-09-20T18:55:28.824355+0000","last_unstale":"2024-09-20T18:58:25.500123+0000","last_undegraded":"2024-09-20T18:58:25.500123+0000","last_fullsized":"2024-09-20T18:58:25.500123+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:46.329547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:46.329547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":384,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.771543+0000","last_change":"2024-09-20T18:55:28.825607+0000","last_active":"2024-09-20T18:58:25.771543+0000","last_peered":"2024-09-20T18:58:25.771543+0000","last_clean":"2024-09-20T18:58:25.771543+0000","last_became_active":"2024-09-20T18:55:28.825296+0000","last_became_peered":"2024-09-20T18:55:28.825296+0000","last_unstale":"2024-09-20T18:58:25.771543+0000","last_undegraded":"2024-09-20T18:58:25.771543+0000","last_fullsized":"2024-09-20T18:58:25.771543+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:43.118805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:43.118805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":420,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.735977+0000","last_change":"2024-09-20T18:55:28.836789+0000","last_active":"2024-09-20T18:58:25.735977+0000","last_peered":"2024-09-20T18:58:25.735977+0000","last_clean":"2024-09-20T18:58:25.735977+0000","last_became_active":"2024-09-20T18:55:28.824579+0000","last_became_peered":"2024-09-20T18:55:28.824579+0000","last_unstale":"2024-09-20T18:58:25.735977+0000","last_undegraded":"2024-09-20T18:58:25.735977+0000","last_fullsized":"2024-09-20T18:58:25.735977+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-20T18:55:20.282193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:20.282193+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":405,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:26.103466+0000","last_change":"2024-09-20T18:54:55.900487+0000","last_active":"2024-09-20T18:58:26.103466+0000","last_peered":"2024-09-20T18:58:26.103466+0000","last_clean":"2024-09-20T18:58:26.103466+0000","last_became_active":"2024-09-20T18:49:45.940988+0000","last_became_peered":"2024-09-20T18:49:45.940988+0000","last_unstale":"2024-09-20T18:58:26.103466+0000","last_undegraded":"2024-09-20T18:58:26.103466+0000","last_fullsized":"2024-09-20T18:58:26.103466+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:55.899611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:55.899611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":144,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.731252+0000","last_change":"2024-09-20T18:57:52.929891+0000","last_active":"2024-09-20T18:58:25.731252+0000","last_peered":"2024-09-20T18:58:25.731252+0000","last_clean":"2024-09-20T18:58:25.731252+0000","last_became_active":"2024-09-20T18:55:28.832534+0000","last_became_peered":"2024-09-20T18:55:28.832534+0000","last_unstale":"2024-09-20T18:58:25.731252+0000","last_undegraded":"2024-09-20T18:58:25.731252+0000","last_fullsized":"2024-09-20T18:58:25.731252+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:52.929828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:52.929828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:13.316360+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":383,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.500467+0000","last_change":"2024-09-20T18:58:22.558181+0000","last_active":"2024-09-20T18:58:25.500467+0000","last_peered":"2024-09-20T18:58:25.500467+0000","last_clean":"2024-09-20T18:58:25.500467+0000","last_became_active":"2024-09-20T18:49:51.968662+0000","last_became_peered":"2024-09-20T18:49:51.968662+0000","last_unstale":"2024-09-20T18:58:25.500467+0000","last_undegraded":"2024-09-20T18:58:25.500467+0000","last_fullsized":"2024-09-20T18:58:25.500467+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:22.558131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:22.558131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:46.688905+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":377,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.772128+0000","last_change":"2024-09-20T18:57:10.970843+0000","last_active":"2024-09-20T18:58:25.772128+0000","last_peered":"2024-09-20T18:58:25.772128+0000","last_clean":"2024-09-20T18:58:25.772128+0000","last_became_active":"2024-09-20T18:49:51.968216+0000","last_became_peered":"2024-09-20T18:49:51.968216+0000","last_unstale":"2024-09-20T18:58:25.772128+0000","last_undegraded":"2024-09-20T18:58:25.772128+0000","last_fullsized":"2024-09-20T18:58:25.772128+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:10.970779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:10.970779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:38.733512+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":397,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.736159+0000","last_change":"2024-09-20T18:55:28.295445+0000","last_active":"2024-09-20T18:58:25.736159+0000","last_peered":"2024-09-20T18:58:25.736159+0000","last_clean":"2024-09-20T18:58:25.736159+0000","last_became_active":"2024-09-20T18:49:45.937992+0000","last_became_peered":"2024-09-20T18:49:45.937992+0000","last_unstale":"2024-09-20T18:58:25.736159+0000","last_undegraded":"2024-09-20T18:58:25.736159+0000","last_fullsized":"2024-09-20T18:58:25.736159+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:28.295399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:28.295399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":184,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-20T18:58:25.308332+0000","last_change":"2024-09-20T18:55:28.852468+0000","last_active":"2024-09-20T18:58:25.308332+0000","last_peered":"2024-09-20T18:58:25.308332+0000","last_clean":"2024-09-20T18:58:25.308332+0000","last_became_active":"2024-09-20T18:55:28.834675+0000","last_became_peered":"2024-09-20T18:55:28.834675+0000","last_unstale":"2024-09-20T18:58:25.308332+0000","last_undegraded":"2024-09-20T18:58:25.308332+0000","last_fullsized":"2024-09-20T18:58:25.308332+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:54:44.719676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.719676+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:38.535671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":12,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":382,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-20T18:58:26.104364+0000","last_change":"2024-09-20T18:51:28.120100+0000","last_active":"2024-09-20T18:58:26.104364+0000","last_peered":"2024-09-20T18:58:26.104364+0000","last_clean":"2024-09-20T18:58:26.104364+0000","last_became_active":"2024-09-20T18:49:49.957582+0000","last_became_peered":"2024-09-20T18:49:49.957582+0000","last_unstale":"2024-09-20T18:58:26.104364+0000","last_undegraded":"2024-09-20T18:58:26.104364+0000","last_fullsized":"2024-09-20T18:58:26.104364+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:28.119694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:28.119694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":67,"ondisk_log_size":67,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":695,"num_read_kb":452,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":1474,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":12140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":394,"ondisk_log_size":394,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":5275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":514,"num_read_kb":509,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266796,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":231360,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":229694,"kb_avail":93513792,"statfs":{"total":95995035648,"available":95758123008,"internally_reserved":0,"allocated":1683456,"data_stored":1162844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":235207114},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":0,"seq":193273528435,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757368,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":279368,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":278270,"kb_avail":93465784,"statfs":{"total":95995035648,"available":95708962816,"internally_reserved":0,"allocated":1097728,"data_stored":579012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":284948933},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986301,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":208592,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":207486,"kb_avail":93536560,"statfs":{"total":95995035648,"available":95781437440,"internally_reserved":0,"allocated":1110016,"data_stored":573535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":212466117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182532,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":357168,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":356030,"kb_avail":93387984,"statfs":{"total":95995035648,"available":95629295616,"internally_reserved":0,"allocated":1134592,"data_stored":585457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":364575175},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411465,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":397376,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":396286,"kb_avail":93347776,"statfs":{"total":95995035648,"available":95588122624,"internally_reserved":0,"allocated":1089536,"data_stored":579261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":405797317},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574991,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":303616,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":301886,"kb_avail":93441536,"statfs":{"total":95995035648,"available":95684132864,"internally_reserved":0,"allocated":1748992,"data_stored":1177708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":309131717},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738517,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":361904,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":360190,"kb_avail":93383248,"statfs":{"total":95995035648,"available":95624445952,"internally_reserved":0,"allocated":1724416,"data_stored":1175901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":368835013},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T18:58:31.451 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-09-20T18:58:31.451 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T18:58:31.527 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:31.548 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:31.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:31.549 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:31.638 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:31 smithi083 ceph-mon[29421]: 4.1a scrub starts 2024-09-20T18:58:31.638 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:31 smithi083 ceph-mon[29421]: 4.1a scrub ok 2024-09-20T18:58:31.638 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:31 smithi083 ceph-mon[29421]: 3.6 scrub starts 2024-09-20T18:58:31.639 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:31 smithi083 ceph-mon[29421]: 3.6 scrub ok 2024-09-20T18:58:31.639 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:31 smithi083 ceph-mon[29421]: from='client.25445 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:58:31.639 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:31 smithi083 ceph-mon[29421]: 6.14 scrub starts 2024-09-20T18:58:31.639 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:31 smithi083 ceph-mon[29421]: 6.14 scrub ok 2024-09-20T18:58:31.639 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:31 smithi083 ceph-mon[29421]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T18:58:31.639 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:31 smithi083 ceph-mon[29421]: Cluster is now healthy 2024-09-20T18:58:31.639 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:31 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3613704502' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T18:58:31.639 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:31 smithi083 ceph-mon[29421]: osdmap e244: 8 total, 8 up, 7 in 2024-09-20T18:58:31.639 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:31 smithi083 ceph-mon[29421]: 4.8 scrub starts 2024-09-20T18:58:31.639 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:31 smithi083 ceph-mon[29421]: 4.8 scrub ok 2024-09-20T18:58:31.639 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:31 smithi083 ceph-mon[29421]: osdmap e245: 8 total, 8 up, 7 in 2024-09-20T18:58:31.639 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:31 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:58:31.639 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:31 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:58:31.640 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:31 smithi083 ceph-mon[36260]: 4.1a scrub starts 2024-09-20T18:58:31.640 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:31 smithi083 ceph-mon[36260]: 4.1a scrub ok 2024-09-20T18:58:31.640 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:31 smithi083 ceph-mon[36260]: 3.6 scrub starts 2024-09-20T18:58:31.640 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:31 smithi083 ceph-mon[36260]: 3.6 scrub ok 2024-09-20T18:58:31.640 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:31 smithi083 ceph-mon[36260]: from='client.25445 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:58:31.640 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:31 smithi083 ceph-mon[36260]: 6.14 scrub starts 2024-09-20T18:58:31.640 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:31 smithi083 ceph-mon[36260]: 6.14 scrub ok 2024-09-20T18:58:31.640 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:31 smithi083 ceph-mon[36260]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T18:58:31.640 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:31 smithi083 ceph-mon[36260]: Cluster is now healthy 2024-09-20T18:58:31.640 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:31 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3613704502' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T18:58:31.640 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:31 smithi083 ceph-mon[36260]: osdmap e244: 8 total, 8 up, 7 in 2024-09-20T18:58:31.640 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:31 smithi083 ceph-mon[36260]: 4.8 scrub starts 2024-09-20T18:58:31.641 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:31 smithi083 ceph-mon[36260]: 4.8 scrub ok 2024-09-20T18:58:31.641 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:31 smithi083 ceph-mon[36260]: osdmap e245: 8 total, 8 up, 7 in 2024-09-20T18:58:31.641 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:31 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:58:31.641 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:31 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:58:31.650 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:31.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:31.685 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:31.685 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:31.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:31 smithi142 ceph-mon[32508]: 4.1a scrub starts 2024-09-20T18:58:31.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:31 smithi142 ceph-mon[32508]: 4.1a scrub ok 2024-09-20T18:58:31.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:31 smithi142 ceph-mon[32508]: 3.6 scrub starts 2024-09-20T18:58:31.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:31 smithi142 ceph-mon[32508]: 3.6 scrub ok 2024-09-20T18:58:31.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:31 smithi142 ceph-mon[32508]: from='client.25445 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:58:31.769 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:31 smithi142 ceph-mon[32508]: 6.14 scrub starts 2024-09-20T18:58:31.769 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:31 smithi142 ceph-mon[32508]: 6.14 scrub ok 2024-09-20T18:58:31.769 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:31 smithi142 ceph-mon[32508]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T18:58:31.769 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:31 smithi142 ceph-mon[32508]: Cluster is now healthy 2024-09-20T18:58:31.769 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:31 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3613704502' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T18:58:31.769 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:31 smithi142 ceph-mon[32508]: osdmap e244: 8 total, 8 up, 7 in 2024-09-20T18:58:31.769 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:31 smithi142 ceph-mon[32508]: 4.8 scrub starts 2024-09-20T18:58:31.770 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:31 smithi142 ceph-mon[32508]: 4.8 scrub ok 2024-09-20T18:58:31.770 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:31 smithi142 ceph-mon[32508]: osdmap e245: 8 total, 8 up, 7 in 2024-09-20T18:58:31.770 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:31 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:58:31.770 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:31 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:58:31.786 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:31.807 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:31.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:31.808 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:31.909 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:31.930 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:31.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:31.931 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:32.032 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:32.057 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:32.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:32.058 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:32.159 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:32.180 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:32.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:32.181 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:32.281 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:32.307 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:32.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:32.308 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:32.409 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:32.430 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:32.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:32.431 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:32.532 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:32.553 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:32.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:32.554 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:32.655 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:32.667 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:32 smithi083 ceph-mon[36260]: pgmap v422: 132 pgs: 132 active+clean; 583 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.8 KiB/s rd, 1 op/s 2024-09-20T18:58:32.667 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:32 smithi083 ceph-mon[36260]: 3.b scrub starts 2024-09-20T18:58:32.667 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:32 smithi083 ceph-mon[36260]: 3.b scrub ok 2024-09-20T18:58:32.667 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:32 smithi083 ceph-mon[36260]: 6.d scrub starts 2024-09-20T18:58:32.667 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:32 smithi083 ceph-mon[36260]: 6.d scrub ok 2024-09-20T18:58:32.667 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:32 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1632505786' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16} : dispatch 2024-09-20T18:58:32.667 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:32 smithi083 ceph-mon[29421]: pgmap v422: 132 pgs: 132 active+clean; 583 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.8 KiB/s rd, 1 op/s 2024-09-20T18:58:32.667 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:32 smithi083 ceph-mon[29421]: 3.b scrub starts 2024-09-20T18:58:32.667 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:32 smithi083 ceph-mon[29421]: 3.b scrub ok 2024-09-20T18:58:32.668 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:32 smithi083 ceph-mon[29421]: 6.d scrub starts 2024-09-20T18:58:32.668 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:32 smithi083 ceph-mon[29421]: 6.d scrub ok 2024-09-20T18:58:32.668 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:32 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1632505786' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16} : dispatch 2024-09-20T18:58:32.677 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:32.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:32.678 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:32.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:32 smithi142 ceph-mon[32508]: pgmap v422: 132 pgs: 132 active+clean; 583 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.8 KiB/s rd, 1 op/s 2024-09-20T18:58:32.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:32 smithi142 ceph-mon[32508]: 3.b scrub starts 2024-09-20T18:58:32.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:32 smithi142 ceph-mon[32508]: 3.b scrub ok 2024-09-20T18:58:32.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:32 smithi142 ceph-mon[32508]: 6.d scrub starts 2024-09-20T18:58:32.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:32 smithi142 ceph-mon[32508]: 6.d scrub ok 2024-09-20T18:58:32.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:32 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1632505786' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16} : dispatch 2024-09-20T18:58:32.779 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:32.800 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:32.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:32.801 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:32.814 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'unique_pool_2' created 2024-09-20T18:58:32.902 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:32.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:32.925 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:32.925 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:33.026 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:58:33.048 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:33.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:33.049 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:58:33.149 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:33.175 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:33.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:33.176 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:33.189 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_blocked_ops 2024-09-20T18:58:33.278 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:33.303 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:33.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:33.304 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:33.405 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:33.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:33.431 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:33.431 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:33.491 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:33 smithi083 ceph-mon[29421]: 3.2 scrub starts 2024-09-20T18:58:33.491 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:33 smithi083 ceph-mon[29421]: 3.2 scrub ok 2024-09-20T18:58:33.491 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:33 smithi083 ceph-mon[29421]: 5.1a scrub starts 2024-09-20T18:58:33.491 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:33 smithi083 ceph-mon[29421]: 5.1a scrub ok 2024-09-20T18:58:33.492 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:33 smithi083 ceph-mon[29421]: 3.5 scrub starts 2024-09-20T18:58:33.492 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:33 smithi083 ceph-mon[29421]: 3.5 scrub ok 2024-09-20T18:58:33.492 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:33 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1632505786' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]': finished 2024-09-20T18:58:33.492 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:33 smithi083 ceph-mon[29421]: osdmap e246: 8 total, 8 up, 7 in 2024-09-20T18:58:33.492 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:33 smithi083 ceph-mon[36260]: 3.2 scrub starts 2024-09-20T18:58:33.492 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:33 smithi083 ceph-mon[36260]: 3.2 scrub ok 2024-09-20T18:58:33.492 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:33 smithi083 ceph-mon[36260]: 5.1a scrub starts 2024-09-20T18:58:33.492 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:33 smithi083 ceph-mon[36260]: 5.1a scrub ok 2024-09-20T18:58:33.492 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:33 smithi083 ceph-mon[36260]: 3.5 scrub starts 2024-09-20T18:58:33.492 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:33 smithi083 ceph-mon[36260]: 3.5 scrub ok 2024-09-20T18:58:33.492 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:33 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1632505786' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]': finished 2024-09-20T18:58:33.492 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:33 smithi083 ceph-mon[36260]: osdmap e246: 8 total, 8 up, 7 in 2024-09-20T18:58:33.532 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:33.556 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:33.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:33.557 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:33.658 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:33.680 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:33.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:33.681 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:33.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:33 smithi142 ceph-mon[32508]: 3.2 scrub starts 2024-09-20T18:58:33.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:33 smithi142 ceph-mon[32508]: 3.2 scrub ok 2024-09-20T18:58:33.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:33 smithi142 ceph-mon[32508]: 5.1a scrub starts 2024-09-20T18:58:33.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:33 smithi142 ceph-mon[32508]: 5.1a scrub ok 2024-09-20T18:58:33.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:33 smithi142 ceph-mon[32508]: 3.5 scrub starts 2024-09-20T18:58:33.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:33 smithi142 ceph-mon[32508]: 3.5 scrub ok 2024-09-20T18:58:33.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:33 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1632505786' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]': finished 2024-09-20T18:58:33.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:33 smithi142 ceph-mon[32508]: osdmap e246: 8 total, 8 up, 7 in 2024-09-20T18:58:33.783 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:33.806 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:33.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:33.808 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:33.836 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd pool application enable unique_pool_2 rados --yes-i-really-mean-it || true 2024-09-20T18:58:33.868 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:58:33.895 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:58:33.908 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:33.929 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:33.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:33.930 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:34.031 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:34.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:34.053 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:34.053 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:34.154 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:34.177 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:34.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:34.178 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:34.279 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:34.302 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:34.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:34.303 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:34.403 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:34.426 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:34.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:34.427 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:34.447 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:34 smithi083 ceph-mon[29421]: 3.9 scrub starts 2024-09-20T18:58:34.447 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:34 smithi083 ceph-mon[29421]: 3.9 scrub ok 2024-09-20T18:58:34.447 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:34 smithi083 ceph-mon[29421]: pgmap v424: 148 pgs: 16 unknown, 132 active+clean; 583 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T18:58:34.447 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:34 smithi083 ceph-mon[29421]: 5.8 scrub starts 2024-09-20T18:58:34.448 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:34 smithi083 ceph-mon[29421]: 5.8 scrub ok 2024-09-20T18:58:34.448 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:34 smithi083 ceph-mon[29421]: 4.1b scrub starts 2024-09-20T18:58:34.448 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:34 smithi083 ceph-mon[29421]: 4.1b scrub ok 2024-09-20T18:58:34.448 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:34 smithi083 ceph-mon[29421]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-20T18:58:34.448 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:34 smithi083 ceph-mon[29421]: osdmap e247: 8 total, 8 up, 7 in 2024-09-20T18:58:34.448 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:34 smithi083 ceph-mon[36260]: 3.9 scrub starts 2024-09-20T18:58:34.448 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:34 smithi083 ceph-mon[36260]: 3.9 scrub ok 2024-09-20T18:58:34.448 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:34 smithi083 ceph-mon[36260]: pgmap v424: 148 pgs: 16 unknown, 132 active+clean; 583 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T18:58:34.448 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:34 smithi083 ceph-mon[36260]: 5.8 scrub starts 2024-09-20T18:58:34.448 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:34 smithi083 ceph-mon[36260]: 5.8 scrub ok 2024-09-20T18:58:34.448 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:34 smithi083 ceph-mon[36260]: 4.1b scrub starts 2024-09-20T18:58:34.448 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:34 smithi083 ceph-mon[36260]: 4.1b scrub ok 2024-09-20T18:58:34.449 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:34 smithi083 ceph-mon[36260]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-20T18:58:34.449 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:34 smithi083 ceph-mon[36260]: osdmap e247: 8 total, 8 up, 7 in 2024-09-20T18:58:34.466 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:58:34.466 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:58:34.482 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:58:34.483 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:58:34.499 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:58:34.499 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:58:34.515 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:58:34.515 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:58:34.528 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:34.532 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:58:34.532 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:58:34.548 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:58:34.548 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:58:34.552 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:34.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:34.553 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:34.565 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:58:34.566 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:58:34.582 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:58:34.582 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:58:34.654 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:34.675 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:34.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:34.676 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:34.696 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T18:58:34.767 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:34 smithi142 ceph-mon[32508]: 3.9 scrub starts 2024-09-20T18:58:34.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:34 smithi142 ceph-mon[32508]: 3.9 scrub ok 2024-09-20T18:58:34.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:34 smithi142 ceph-mon[32508]: pgmap v424: 148 pgs: 16 unknown, 132 active+clean; 583 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T18:58:34.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:34 smithi142 ceph-mon[32508]: 5.8 scrub starts 2024-09-20T18:58:34.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:34 smithi142 ceph-mon[32508]: 5.8 scrub ok 2024-09-20T18:58:34.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:34 smithi142 ceph-mon[32508]: 4.1b scrub starts 2024-09-20T18:58:34.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:34 smithi142 ceph-mon[32508]: 4.1b scrub ok 2024-09-20T18:58:34.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:34 smithi142 ceph-mon[32508]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-20T18:58:34.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:34 smithi142 ceph-mon[32508]: osdmap e247: 8 total, 8 up, 7 in 2024-09-20T18:58:34.777 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:34.801 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:34.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:34.802 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:34.903 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:34.924 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:34.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:34.925 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:35.026 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:35.051 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:35.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:35.053 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:35.154 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:35.175 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:35.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:35.176 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:35.277 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:58:35.298 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:35.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:35.299 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:58:35.400 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:35.422 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:35.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:35.423 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:35.524 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:35.546 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:35.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:35.547 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:35.570 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:35 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1648261065' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:58:35.570 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:35 smithi083 ceph-mon[29421]: osdmap e248: 8 total, 8 up, 7 in 2024-09-20T18:58:35.570 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T18:58:35.570 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:58:35.570 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:35 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1648261065' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:58:35.570 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:35 smithi083 ceph-mon[36260]: osdmap e248: 8 total, 8 up, 7 in 2024-09-20T18:58:35.648 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:58:35.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:35.670 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:35.670 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:58:35.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:35 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1648261065' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:58:35.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:35 smithi142 ceph-mon[32508]: osdmap e248: 8 total, 8 up, 7 in 2024-09-20T18:58:35.770 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:35.794 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:35.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:35.795 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:35.896 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:35.917 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:35.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:35.918 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:36.019 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:36.044 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:36.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:36.045 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:36.145 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:36.169 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:36.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:36.170 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:36.204 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_historic_ops 2024-09-20T18:58:36.216 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:58:36.221 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:58:36.270 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:36.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:36.293 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:36.294 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:36.394 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:36.417 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:36.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:36.418 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:36.492 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:36 smithi083 ceph-mon[29421]: pgmap v427: 148 pgs: 16 unknown, 132 active+clean; 583 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T18:58:36.492 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:36 smithi083 ceph-mon[29421]: 3.1b scrub starts 2024-09-20T18:58:36.492 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:36 smithi083 ceph-mon[29421]: 3.1b scrub ok 2024-09-20T18:58:36.492 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:36 smithi083 ceph-mon[29421]: 6.1f scrub starts 2024-09-20T18:58:36.493 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:36 smithi083 ceph-mon[29421]: 6.1f scrub ok 2024-09-20T18:58:36.493 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:36 smithi083 ceph-mon[29421]: osdmap e249: 8 total, 8 up, 7 in 2024-09-20T18:58:36.493 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:36 smithi083 ceph-mon[29421]: 3.18 scrub starts 2024-09-20T18:58:36.493 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:36 smithi083 ceph-mon[29421]: 3.18 scrub ok 2024-09-20T18:58:36.493 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:36 smithi083 ceph-mon[29421]: 5.c scrub starts 2024-09-20T18:58:36.493 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:36 smithi083 ceph-mon[29421]: 5.c scrub ok 2024-09-20T18:58:36.493 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:36 smithi083 ceph-mon[36260]: pgmap v427: 148 pgs: 16 unknown, 132 active+clean; 583 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T18:58:36.493 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:36 smithi083 ceph-mon[36260]: 3.1b scrub starts 2024-09-20T18:58:36.493 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:36 smithi083 ceph-mon[36260]: 3.1b scrub ok 2024-09-20T18:58:36.493 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:36 smithi083 ceph-mon[36260]: 6.1f scrub starts 2024-09-20T18:58:36.493 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:36 smithi083 ceph-mon[36260]: 6.1f scrub ok 2024-09-20T18:58:36.494 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:36 smithi083 ceph-mon[36260]: osdmap e249: 8 total, 8 up, 7 in 2024-09-20T18:58:36.494 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:36 smithi083 ceph-mon[36260]: 3.18 scrub starts 2024-09-20T18:58:36.494 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:36 smithi083 ceph-mon[36260]: 3.18 scrub ok 2024-09-20T18:58:36.494 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:36 smithi083 ceph-mon[36260]: 5.c scrub starts 2024-09-20T18:58:36.494 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:36 smithi083 ceph-mon[36260]: 5.c scrub ok 2024-09-20T18:58:36.518 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:36.540 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:36.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:36.541 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:36.642 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:36.663 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:36.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:36.664 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:36.765 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:36.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:36 smithi142 ceph-mon[32508]: pgmap v427: 148 pgs: 16 unknown, 132 active+clean; 583 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T18:58:36.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:36 smithi142 ceph-mon[32508]: 3.1b scrub starts 2024-09-20T18:58:36.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:36 smithi142 ceph-mon[32508]: 3.1b scrub ok 2024-09-20T18:58:36.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:36 smithi142 ceph-mon[32508]: 6.1f scrub starts 2024-09-20T18:58:36.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:36 smithi142 ceph-mon[32508]: 6.1f scrub ok 2024-09-20T18:58:36.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:36 smithi142 ceph-mon[32508]: osdmap e249: 8 total, 8 up, 7 in 2024-09-20T18:58:36.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:36 smithi142 ceph-mon[32508]: 3.18 scrub starts 2024-09-20T18:58:36.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:36 smithi142 ceph-mon[32508]: 3.18 scrub ok 2024-09-20T18:58:36.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:36 smithi142 ceph-mon[32508]: 5.c scrub starts 2024-09-20T18:58:36.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:36 smithi142 ceph-mon[32508]: 5.c scrub ok 2024-09-20T18:58:36.786 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:36.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:36.787 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:36.887 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:36.909 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:36.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:36.910 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:37.011 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:37.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:37.034 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:37.034 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:37.135 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:37.157 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:37.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:37.158 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:37.177 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:58:37.259 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:37.267 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:58:36 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:58:37.280 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:37.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:37.281 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:37.332 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T18:58:37.368 INFO:teuthology.orchestra.run.smithi083.stderr:enabled application 'rados' on pool 'unique_pool_2' 2024-09-20T18:58:37.368 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-20T18:58:37.382 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:37.408 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:37.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:37.409 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:37.510 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:37.531 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:37.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:37.532 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:37.632 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:37.657 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:37.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:37.658 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:37.751 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:37 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3933367625' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-20T18:58:37.751 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:37 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/589012951' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:58:37.751 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:37 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3933367625' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-20T18:58:37.751 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:37 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/589012951' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:58:37.759 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:58:37.767 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:37 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3933367625' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-20T18:58:37.767 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:37 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/589012951' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:58:37.780 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:37.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:37.781 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:58:37.882 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:37.905 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:37.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:37.906 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:38.006 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:38.028 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:38.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:38.030 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:38.130 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:38.152 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:38.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:38.153 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:38.236 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T18:58:38.236 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:58:38.253 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:38.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:38.277 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:38.277 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:38.378 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:38.400 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:38.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:38.401 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:38.502 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:38.523 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:38.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:38.524 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:38.625 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:38.647 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:38.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:38.648 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:38.748 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:38.767 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:38 smithi142 ceph-mon[32508]: 4.d scrub starts 2024-09-20T18:58:38.767 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:38 smithi142 ceph-mon[32508]: 4.d scrub ok 2024-09-20T18:58:38.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:38 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:58:38.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:38 smithi142 ceph-mon[32508]: pgmap v429: 148 pgs: 3 unknown, 145 active+clean; 583 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T18:58:38.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:38 smithi142 ceph-mon[32508]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:58:38.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:38 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3933367625' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-20T18:58:38.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:38 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/589012951' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T18:58:38.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:38 smithi142 ceph-mon[32508]: osdmap e250: 8 total, 8 up, 7 in 2024-09-20T18:58:38.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:38 smithi142 ceph-mon[32508]: 6.5 scrub starts 2024-09-20T18:58:38.771 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:38.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:38.772 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:38.798 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:38 smithi083 ceph-mon[29421]: 4.d scrub starts 2024-09-20T18:58:38.798 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:38 smithi083 ceph-mon[29421]: 4.d scrub ok 2024-09-20T18:58:38.798 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:38 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:58:38.798 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:38 smithi083 ceph-mon[29421]: pgmap v429: 148 pgs: 3 unknown, 145 active+clean; 583 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T18:58:38.798 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:38 smithi083 ceph-mon[29421]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:58:38.798 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:38 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3933367625' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-20T18:58:38.798 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:38 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/589012951' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T18:58:38.798 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:38 smithi083 ceph-mon[29421]: osdmap e250: 8 total, 8 up, 7 in 2024-09-20T18:58:38.798 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:38 smithi083 ceph-mon[29421]: 6.5 scrub starts 2024-09-20T18:58:38.798 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:38 smithi083 ceph-mon[36260]: 4.d scrub starts 2024-09-20T18:58:38.799 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:38 smithi083 ceph-mon[36260]: 4.d scrub ok 2024-09-20T18:58:38.799 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:38 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:58:38.799 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:38 smithi083 ceph-mon[36260]: pgmap v429: 148 pgs: 3 unknown, 145 active+clean; 583 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T18:58:38.799 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:38 smithi083 ceph-mon[36260]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:58:38.799 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:38 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3933367625' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-20T18:58:38.799 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:38 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/589012951' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T18:58:38.799 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:38 smithi083 ceph-mon[36260]: osdmap e250: 8 total, 8 up, 7 in 2024-09-20T18:58:38.799 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:38 smithi083 ceph-mon[36260]: 6.5 scrub starts 2024-09-20T18:58:38.873 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:58:38.894 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:38.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:38.895 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:58:38.995 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:39.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:39.018 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:39.018 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:39.118 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:39.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:39.141 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:39.141 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:39.209 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:58:39.242 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:39.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:39.265 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:39.265 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:39.365 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:39.387 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:39.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:39.388 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:39.489 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:39.511 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:39.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:39.511 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:39.612 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:39.634 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:39.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:39.635 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:39.682 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:39 smithi083 ceph-mon[29421]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-20T18:58:39.683 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:39 smithi083 ceph-mon[36260]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-20T18:58:39.684 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-20T18:58:39.735 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:39.756 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:39.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:39.757 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:39.767 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:39 smithi142 ceph-mon[32508]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-20T18:58:39.858 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:39.880 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:39.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:39.881 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:39.981 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:40.003 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:40.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:40.004 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:40.039 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T18:58:40.105 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:40.126 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:40.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:40.127 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:40.186 DEBUG:teuthology.orchestra.run.smithi083:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_2 bench 90 write' 2024-09-20T18:58:40.187 INFO:tasks.thrashosds.thrasher:fixing pg num pool .mgr 2024-09-20T18:58:40.187 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T18:58:40.189 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-20T18:58:40.190 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-20T18:58:40.228 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:40.246 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:hints = 1 2024-09-20T18:58:40.246 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-20T18:58:40.246 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Object prefix: benchmark_data_smithi083_124552 2024-09-20T18:58:40.246 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-20T18:58:40.246 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 0 0 0 0 0 0 - 0 2024-09-20T18:58:40.251 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:40.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:40.253 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:40.277 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:58:40.277 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:58:40.324 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:58:40.325 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:58:40.353 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:40.357 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:58:40.359 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:58:40.359 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:58:40.379 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:40.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:40.381 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:40.400 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:58:40.400 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:58:40.406 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:58:40.435 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:58:40.435 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:58:40.468 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:58:40.468 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:58:40.482 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:40.495 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:58:40.495 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:58:40.511 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:40.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:40.512 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:40.528 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:58:40.528 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:58:40.529 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:40 smithi083 ceph-mon[29421]: pgmap v431: 148 pgs: 148 active+clean; 583 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 992 B/s rd, 0 op/s 2024-09-20T18:58:40.529 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:40 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2028342956' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:58:40.529 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:40 smithi083 ceph-mon[36260]: pgmap v431: 148 pgs: 148 active+clean; 583 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 992 B/s rd, 0 op/s 2024-09-20T18:58:40.529 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:40 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2028342956' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:58:40.613 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:58:40.649 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:40.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:40.650 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:58:40.751 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:40.769 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:40 smithi142 ceph-mon[32508]: pgmap v431: 148 pgs: 148 active+clean; 583 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 992 B/s rd, 0 op/s 2024-09-20T18:58:40.769 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:40 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2028342956' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:58:40.782 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:40.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:40.783 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:40.797 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:58:40 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:58:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:58:40.884 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:40.915 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:40.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:40.916 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:41.017 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:41.054 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:41.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:41.055 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:41.155 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:58:41.191 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:41.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:41.192 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:58:41.246 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 1 16 3021 3005 187.803 187.812 0.00578048 0.00530163 2024-09-20T18:58:41.293 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:41.329 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:41.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:41.330 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:41.431 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:41.464 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:41.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:41.465 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:41.566 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:41.603 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:41.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:41.605 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:41.626 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T18:58:41.626 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-20T18:58:41.626 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:41.626 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:2899 9.0 9:0019e308:::benchmark_data_smithi083_124552_object2898:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e251)", 2024-09-20T18:58:41.626 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:58:41.199793+0000", 2024-09-20T18:58:41.626 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0033049709999999999, 2024-09-20T18:58:41.627 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0033380889999999998, 2024-09-20T18:58:41.627 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:58:41.627 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:58:41.627 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "waiting for sub ops", 2024-09-20T18:58:41.627 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:58:41.627 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T18:58:41.627 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T18:58:41.627 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2899 2024-09-20T18:58:41.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:41.627 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:58:41.627 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:41.627 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:58:41.627 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:41.199793+0000", 2024-09-20T18:58:41.627 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:41.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:41.628 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:41.628 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:58:41.628 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:41.199793+0000", 2024-09-20T18:58:41.628 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:41.628 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:41.628 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:41.628 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:58:41.628 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:41.199795+0000", 2024-09-20T18:58:41.628 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1790000000000001e-06 2024-09-20T18:58:41.628 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:41.628 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:41.628 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:58:41.628 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:41.199817+0000", 2024-09-20T18:58:41.628 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2894000000000001e-05 2024-09-20T18:58:41.628 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:41.628 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:41.629 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:58:41.629 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:41.199829+0000", 2024-09-20T18:58:41.629 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1626e-05 2024-09-20T18:58:41.629 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:41.629 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:41.629 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:58:41.629 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:41.199857+0000", 2024-09-20T18:58:41.629 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7851000000000002e-05 2024-09-20T18:58:41.629 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:41.629 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:41.629 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:41.629 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:41.200076+0000", 2024-09-20T18:58:41.629 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000218719 2024-09-20T18:58:41.629 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:41.629 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:41.630 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:58:41.630 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:41.200199+0000", 2024-09-20T18:58:41.630 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012375799999999999 2024-09-20T18:58:41.630 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:41.630 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:41.630 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,2", 2024-09-20T18:58:41.630 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:41.200336+0000", 2024-09-20T18:58:41.630 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000136181 2024-09-20T18:58:41.630 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:41.630 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:41.630 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:58:41.630 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:41.201684+0000", 2024-09-20T18:58:41.630 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0013486100000000001 2024-09-20T18:58:41.630 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:41.630 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:58:41.631 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:41.631 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:41.631 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:41.631 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:2904 9.4 9:29e5cdfc:::benchmark_data_smithi083_124552_object2903:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e251)", 2024-09-20T18:58:41.631 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:58:41.200104+0000", 2024-09-20T18:58:41.631 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0029941149999999999, 2024-09-20T18:58:41.631 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0031105019999999998, 2024-09-20T18:58:41.631 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:58:41.631 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:58:41.631 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "waiting for sub ops", 2024-09-20T18:58:41.631 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:58:41.631 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T18:58:41.632 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T18:58:41.632 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2904 2024-09-20T18:58:41.632 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:41.632 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:58:41.632 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:41.632 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:58:41.632 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:41.200104+0000", 2024-09-20T18:58:41.632 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:41.632 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:41.632 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:41.632 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:58:41.632 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:41.200104+0000", 2024-09-20T18:58:41.632 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:41.632 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:41.632 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:41.632 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:58:41.633 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:41.200105+0000", 2024-09-20T18:58:41.633 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0070000000000001e-06 2024-09-20T18:58:41.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:41.633 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:41.633 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:58:41.633 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:41.200126+0000", 2024-09-20T18:58:41.633 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0962999999999999e-05 2024-09-20T18:58:41.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:41.633 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:41.633 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:58:41.633 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:41.200136+0000", 2024-09-20T18:58:41.633 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0068000000000001e-05 2024-09-20T18:58:41.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:41.633 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:41.634 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:58:41.634 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:41.200144+0000", 2024-09-20T18:58:41.634 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7079999999999997e-06 2024-09-20T18:58:41.634 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:41.634 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:41.634 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:41.634 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:41.200331+0000", 2024-09-20T18:58:41.634 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018696899999999999 2024-09-20T18:58:41.634 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:41.634 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:41.634 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:58:41.634 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:41.200449+0000", 2024-09-20T18:58:41.634 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011803699999999999 2024-09-20T18:58:41.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:41.635 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:41.635 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3,4", 2024-09-20T18:58:41.635 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:41.200583+0000", 2024-09-20T18:58:41.635 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013444499999999999 2024-09-20T18:58:41.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:41.635 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:41.635 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:58:41.635 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:41.201607+0000", 2024-09-20T18:58:41.635 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001023693 2024-09-20T18:58:41.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:41.635 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:41.635 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:41.635 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:41.202365+0000", 2024-09-20T18:58:41.636 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00075767800000000002 2024-09-20T18:58:41.636 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:41.636 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:58:41.636 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:41.636 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:41.636 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:41.636 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:2906 9.0 9:00fadebf:::benchmark_data_smithi083_124552_object2905:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e251)", 2024-09-20T18:58:41.636 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:58:41.200788+0000", 2024-09-20T18:58:41.636 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.002309967, 2024-09-20T18:58:41.636 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0025074950000000002, 2024-09-20T18:58:41.636 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:58:41.636 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:58:41.636 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "waiting for sub ops", 2024-09-20T18:58:41.636 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:58:41.636 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T18:58:41.637 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T18:58:41.637 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2906 2024-09-20T18:58:41.637 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:41.637 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:58:41.637 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:41.637 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:58:41.637 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:41.200788+0000", 2024-09-20T18:58:41.637 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:41.637 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:41.637 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:41.637 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:58:41.637 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:41.200788+0000", 2024-09-20T18:58:41.637 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:41.637 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:41.637 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:41.637 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:58:41.638 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:41.200789+0000", 2024-09-20T18:58:41.638 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8400000000000002e-07 2024-09-20T18:58:41.638 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:41.638 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:41.638 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:58:41.638 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:41.200818+0000", 2024-09-20T18:58:41.638 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8342000000000001e-05 2024-09-20T18:58:41.638 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:41.638 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:41.638 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:58:41.638 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:41.200838+0000", 2024-09-20T18:58:41.638 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0621000000000001e-05 2024-09-20T18:58:41.639 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:41.639 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:41.639 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:58:41.639 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:41.200857+0000", 2024-09-20T18:58:41.639 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8300000000000001e-05 2024-09-20T18:58:41.639 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:41.639 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:41.639 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:41.639 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:41.200927+0000", 2024-09-20T18:58:41.639 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0720000000000001e-05 2024-09-20T18:58:41.639 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:41.639 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:41.639 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:58:41.640 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:41.201197+0000", 2024-09-20T18:58:41.640 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00027017499999999999 2024-09-20T18:58:41.640 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:41.640 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:41.640 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,2", 2024-09-20T18:58:41.640 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:41.201328+0000", 2024-09-20T18:58:41.640 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013081099999999999 2024-09-20T18:58:41.640 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:41.640 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:41.640 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:58:41.640 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:41.202406+0000", 2024-09-20T18:58:41.640 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001078028 2024-09-20T18:58:41.640 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:41.640 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:58:41.640 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:41.640 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:41.640 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:41.641 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.25484.0:2899 9.0 e251/246)", 2024-09-20T18:58:41.641 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:58:41.203087+0000", 2024-09-20T18:58:41.641 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 1.1557e-05, 2024-09-20T18:58:41.641 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00027166400000000002, 2024-09-20T18:58:41.641 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:58:41.641 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:58:41.641 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "queued for pg", 2024-09-20T18:58:41.641 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:58:41.641 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:41.641 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:58:41.641 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:41.203087+0000", 2024-09-20T18:58:41.641 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:41.641 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:41.641 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:41.641 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:58:41.642 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:41.203087+0000", 2024-09-20T18:58:41.642 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:41.642 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:41.642 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:41.642 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:58:41.642 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:41.203088+0000", 2024-09-20T18:58:41.642 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8999999999999995e-07 2024-09-20T18:58:41.642 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:41.642 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:41.642 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:58:41.642 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:41.203093+0000", 2024-09-20T18:58:41.642 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.057e-06 2024-09-20T18:58:41.642 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:41.642 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:41.642 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:58:41.643 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:41.203104+0000", 2024-09-20T18:58:41.643 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.118e-05 2024-09-20T18:58:41.643 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:41.643 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:41.643 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:58:41.643 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:41.203110+0000", 2024-09-20T18:58:41.643 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7059999999999998e-06 2024-09-20T18:58:41.643 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:41.643 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:58:41.643 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:41.643 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:41.643 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T18:58:41.643 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 4 2024-09-20T18:58:41.643 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T18:58:41.705 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:41.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:41.744 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:41.745 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:41.846 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:41.878 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:41.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:41.879 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:41.920 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:58:41.980 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:42.014 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:42.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:42.015 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:42.017 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:41 smithi142 ceph-mon[32508]: osdmap e251: 8 total, 8 up, 7 in 2024-09-20T18:58:42.026 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:41 smithi083 ceph-mon[29421]: osdmap e251: 8 total, 8 up, 7 in 2024-09-20T18:58:42.026 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:41 smithi083 ceph-mon[36260]: osdmap e251: 8 total, 8 up, 7 in 2024-09-20T18:58:42.116 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:42.163 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:42.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:42.164 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:42.247 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 2 16 6312 6296 196.729 205.688 0.00409077 0.00507311 2024-09-20T18:58:42.265 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:42.302 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:42.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:42.303 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:42.404 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:42.439 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:42.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:42.440 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:42.540 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:42.572 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:42.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:42.573 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:42.673 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:42.709 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:42.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:42.710 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:42.767 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_blocked_ops 2024-09-20T18:58:42.802 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:42 smithi083 ceph-mon[29421]: pgmap v433: 148 pgs: 148 active+clean; 54 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.7 KiB/s rd, 8.8 MiB/s wr, 142 op/s 2024-09-20T18:58:42.802 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:42 smithi083 ceph-mon[36260]: pgmap v433: 148 pgs: 148 active+clean; 54 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.7 KiB/s rd, 8.8 MiB/s wr, 142 op/s 2024-09-20T18:58:42.811 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:42.854 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:42.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:42.855 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:42.956 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:42.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:42.993 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:42.993 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:43.017 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:42 smithi142 ceph-mon[32508]: pgmap v433: 148 pgs: 148 active+clean; 54 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.7 KiB/s rd, 8.8 MiB/s wr, 142 op/s 2024-09-20T18:58:43.095 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:43.125 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:43.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:43.126 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:43.227 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:43.246 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 3 16 9342 9326 194.27 189.375 0.00893569 0.00513662 2024-09-20T18:58:43.255 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:43.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:43.256 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:43.357 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:43.388 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:43.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:43.389 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:43.489 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:43.517 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:43.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:43.518 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:43.589 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:58:43.619 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:43.648 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:43.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:43.649 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:43.749 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:43.766 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-20T18:58:43.791 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:43.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:43.792 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:43.892 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:43.917 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:43.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:43.918 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:43.928 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:58:43.930 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-20T18:58:43.972 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T18:58:44.011 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:43 smithi083 ceph-mon[29421]: pgmap v434: 148 pgs: 148 active+clean; 222 MiB data, 2.5 GiB used, 623 GiB / 626 GiB avail; 1.4 KiB/s rd, 31 MiB/s wr, 491 op/s 2024-09-20T18:58:44.011 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:43 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1807136789' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:58:44.011 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:43 smithi083 ceph-mon[36260]: pgmap v434: 148 pgs: 148 active+clean; 222 MiB data, 2.5 GiB used, 623 GiB / 626 GiB avail; 1.4 KiB/s rd, 31 MiB/s wr, 491 op/s 2024-09-20T18:58:44.012 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:43 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1807136789' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:58:44.017 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:43 smithi142 ceph-mon[32508]: pgmap v434: 148 pgs: 148 active+clean; 222 MiB data, 2.5 GiB used, 623 GiB / 626 GiB avail; 1.4 KiB/s rd, 31 MiB/s wr, 491 op/s 2024-09-20T18:58:44.017 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:43 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1807136789' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:58:44.019 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:44.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:44.062 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:44.062 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:44.163 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:58:44.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:44.206 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:44.207 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:58:44.247 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 4 15 11810 11795 184.273 154.312 0.0037044 0.00541766 2024-09-20T18:58:44.307 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:44.340 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:44.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:44.341 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:44.442 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:44.485 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:44.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:44.486 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:44.586 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:44.619 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:44.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:44.620 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:44.626 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T18:58:44.627 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-20T18:58:44.627 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-20T18:58:44.627 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-20T18:58:44.627 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T18:58:44.724 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:44.756 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:44.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:44.757 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:44.858 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:44.891 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:44.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:44.892 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:44.993 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:45.017 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:44 smithi142 ceph-mon[32508]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:58:45.018 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:44 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1807136789' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T18:58:45.018 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:44 smithi142 ceph-mon[32508]: osdmap e252: 8 total, 8 up, 7 in 2024-09-20T18:58:45.018 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:44 smithi142 ceph-mon[32508]: from='client.25496 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:58:45.025 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:45.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:45.026 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:45.047 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:44 smithi083 ceph-mon[29421]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:58:45.047 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:44 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1807136789' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T18:58:45.047 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:44 smithi083 ceph-mon[29421]: osdmap e252: 8 total, 8 up, 7 in 2024-09-20T18:58:45.047 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:44 smithi083 ceph-mon[29421]: from='client.25496 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:58:45.048 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:44 smithi083 ceph-mon[36260]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:58:45.048 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:44 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1807136789' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T18:58:45.048 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:44 smithi083 ceph-mon[36260]: osdmap e252: 8 total, 8 up, 7 in 2024-09-20T18:58:45.048 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:44 smithi083 ceph-mon[36260]: from='client.25496 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:58:45.126 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:45.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:45.167 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:45.167 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:45.247 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 5 16 14793 14777 184.69 186.375 0.00545892 0.00540884 2024-09-20T18:58:45.263 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":435,"stamp":"2024-09-20T18:58:43.757631+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":232790555,"num_objects":3742,"num_object_clones":0,"num_object_copies":11226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3742,"num_whiteouts":0,"num_read":1374,"num_read_kb":1231,"num_write":4092,"num_write_kb":228425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":1182665,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4099,"ondisk_log_size":4099,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":2586508,"kb_used_data":412120,"kb_used_omap":10,"kb_used_meta":2174261,"kb_avail":653629556,"statfs":{"total":671965249536,"available":669316665344,"internally_reserved":0,"allocated":422010880,"data_stored":417933475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11158,"internal_metadata":2226443370},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,5,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":232194048,"num_objects":3543,"num_object_clones":0,"num_object_copies":10629,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3543,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":3543,"num_write_kb":226752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.790940"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":422,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.751027+0000","last_change":"2024-09-20T18:55:28.037494+0000","last_active":"2024-09-20T18:58:40.751027+0000","last_peered":"2024-09-20T18:58:40.751027+0000","last_clean":"2024-09-20T18:58:40.751027+0000","last_became_active":"2024-09-20T18:49:45.943201+0000","last_became_peered":"2024-09-20T18:49:45.943201+0000","last_unstale":"2024-09-20T18:58:40.751027+0000","last_undegraded":"2024-09-20T18:58:40.751027+0000","last_fullsized":"2024-09-20T18:58:40.751027+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:28.037449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:28.037449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":468,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.745185+0000","last_change":"2024-09-20T18:55:28.838729+0000","last_active":"2024-09-20T18:58:40.745185+0000","last_peered":"2024-09-20T18:58:40.745185+0000","last_clean":"2024-09-20T18:58:40.745185+0000","last_became_active":"2024-09-20T18:55:28.819637+0000","last_became_peered":"2024-09-20T18:55:28.819637+0000","last_unstale":"2024-09-20T18:58:40.745185+0000","last_undegraded":"2024-09-20T18:58:40.745185+0000","last_fullsized":"2024-09-20T18:58:40.745185+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:50.231620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:50.231620+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:37.754928+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":420,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.743865+0000","last_change":"2024-09-20T18:57:49.016078+0000","last_active":"2024-09-20T18:58:40.743865+0000","last_peered":"2024-09-20T18:58:40.743865+0000","last_clean":"2024-09-20T18:58:40.743865+0000","last_became_active":"2024-09-20T18:49:49.959514+0000","last_became_peered":"2024-09-20T18:49:49.959514+0000","last_unstale":"2024-09-20T18:58:40.743865+0000","last_undegraded":"2024-09-20T18:58:40.743865+0000","last_fullsized":"2024-09-20T18:58:40.743865+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:49.016006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:49.016006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:13.297987+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":418,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.745404+0000","last_change":"2024-09-20T18:58:02.084435+0000","last_active":"2024-09-20T18:58:40.745404+0000","last_peered":"2024-09-20T18:58:40.745404+0000","last_clean":"2024-09-20T18:58:40.745404+0000","last_became_active":"2024-09-20T18:49:51.969421+0000","last_became_peered":"2024-09-20T18:49:51.969421+0000","last_unstale":"2024-09-20T18:58:40.745404+0000","last_undegraded":"2024-09-20T18:58:40.745404+0000","last_fullsized":"2024-09-20T18:58:40.745404+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:02.084375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:02.084375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:20.422560+0000","scrub_duration":197,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":415,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.749985+0000","last_change":"2024-09-20T18:55:28.822034+0000","last_active":"2024-09-20T18:58:40.749985+0000","last_peered":"2024-09-20T18:58:40.749985+0000","last_clean":"2024-09-20T18:58:40.749985+0000","last_became_active":"2024-09-20T18:55:28.821717+0000","last_became_peered":"2024-09-20T18:55:28.821717+0000","last_unstale":"2024-09-20T18:58:40.749985+0000","last_undegraded":"2024-09-20T18:58:40.749985+0000","last_fullsized":"2024-09-20T18:58:40.749985+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.216188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.216188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:13.051959+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":420,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.749782+0000","last_change":"2024-09-20T18:57:22.694362+0000","last_active":"2024-09-20T18:58:40.749782+0000","last_peered":"2024-09-20T18:58:40.749782+0000","last_clean":"2024-09-20T18:58:40.749782+0000","last_became_active":"2024-09-20T18:55:28.824826+0000","last_became_peered":"2024-09-20T18:55:28.824826+0000","last_unstale":"2024-09-20T18:58:40.749782+0000","last_undegraded":"2024-09-20T18:58:40.749782+0000","last_fullsized":"2024-09-20T18:58:40.749782+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:22.694295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:22.694295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":433,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.747375+0000","last_change":"2024-09-20T18:57:13.774827+0000","last_active":"2024-09-20T18:58:40.747375+0000","last_peered":"2024-09-20T18:58:40.747375+0000","last_clean":"2024-09-20T18:58:40.747375+0000","last_became_active":"2024-09-20T18:49:47.952150+0000","last_became_peered":"2024-09-20T18:49:47.952150+0000","last_unstale":"2024-09-20T18:58:40.747375+0000","last_undegraded":"2024-09-20T18:58:40.747375+0000","last_fullsized":"2024-09-20T18:58:40.747375+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:57:13.774305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:13.774305+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":421,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.745276+0000","last_change":"2024-09-20T18:58:06.846086+0000","last_active":"2024-09-20T18:58:40.745276+0000","last_peered":"2024-09-20T18:58:40.745276+0000","last_clean":"2024-09-20T18:58:40.745276+0000","last_became_active":"2024-09-20T18:55:28.820362+0000","last_became_peered":"2024-09-20T18:55:28.820362+0000","last_unstale":"2024-09-20T18:58:40.745276+0000","last_undegraded":"2024-09-20T18:58:40.745276+0000","last_fullsized":"2024-09-20T18:58:40.745276+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:06.846018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:06.846018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:33.625936+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"59'1","reported_seq":453,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.739972+0000","last_change":"2024-09-20T18:58:09.472495+0000","last_active":"2024-09-20T18:58:40.739972+0000","last_peered":"2024-09-20T18:58:40.739972+0000","last_clean":"2024-09-20T18:58:40.739972+0000","last_became_active":"2024-09-20T18:55:28.833386+0000","last_became_peered":"2024-09-20T18:55:28.833386+0000","last_unstale":"2024-09-20T18:58:40.739972+0000","last_undegraded":"2024-09-20T18:58:40.739972+0000","last_fullsized":"2024-09-20T18:58:40.739972+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:58:09.472360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:09.472360+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:29.957740+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":442,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.745734+0000","last_change":"2024-09-20T18:58:29.671959+0000","last_active":"2024-09-20T18:58:40.745734+0000","last_peered":"2024-09-20T18:58:40.745734+0000","last_clean":"2024-09-20T18:58:40.745734+0000","last_became_active":"2024-09-20T18:49:47.947448+0000","last_became_peered":"2024-09-20T18:49:47.947448+0000","last_unstale":"2024-09-20T18:58:40.745734+0000","last_undegraded":"2024-09-20T18:58:40.745734+0000","last_fullsized":"2024-09-20T18:58:40.745734+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:58:29.671887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:29.671887+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:55.089337+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":420,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.739615+0000","last_change":"2024-09-20T18:58:25.505818+0000","last_active":"2024-09-20T18:58:40.739615+0000","last_peered":"2024-09-20T18:58:40.739615+0000","last_clean":"2024-09-20T18:58:40.739615+0000","last_became_active":"2024-09-20T18:49:49.961188+0000","last_became_peered":"2024-09-20T18:49:49.961188+0000","last_unstale":"2024-09-20T18:58:40.739615+0000","last_undegraded":"2024-09-20T18:58:40.739615+0000","last_fullsized":"2024-09-20T18:58:40.739615+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:25.505757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:25.505757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:26.562790+0000","scrub_duration":45,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":393,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.751526+0000","last_change":"2024-09-20T18:58:21.861893+0000","last_active":"2024-09-20T18:58:40.751526+0000","last_peered":"2024-09-20T18:58:40.751526+0000","last_clean":"2024-09-20T18:58:40.751526+0000","last_became_active":"2024-09-20T18:49:51.966715+0000","last_became_peered":"2024-09-20T18:49:51.966715+0000","last_unstale":"2024-09-20T18:58:40.751526+0000","last_undegraded":"2024-09-20T18:58:40.751526+0000","last_fullsized":"2024-09-20T18:58:40.751526+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:21.861836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:21.861836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:51.627461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":412,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.739955+0000","last_change":"2024-09-20T18:55:15.252839+0000","last_active":"2024-09-20T18:58:40.739955+0000","last_peered":"2024-09-20T18:58:40.739955+0000","last_clean":"2024-09-20T18:58:40.739955+0000","last_became_active":"2024-09-20T18:49:45.944703+0000","last_became_peered":"2024-09-20T18:49:45.944703+0000","last_unstale":"2024-09-20T18:58:40.739955+0000","last_undegraded":"2024-09-20T18:58:40.739955+0000","last_fullsized":"2024-09-20T18:58:40.739955+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:15.252782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:15.252782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:15.451958+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":446,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.745261+0000","last_change":"2024-09-20T18:58:33.660957+0000","last_active":"2024-09-20T18:58:40.745261+0000","last_peered":"2024-09-20T18:58:40.745261+0000","last_clean":"2024-09-20T18:58:40.745261+0000","last_became_active":"2024-09-20T18:51:23.813399+0000","last_became_peered":"2024-09-20T18:51:23.813399+0000","last_unstale":"2024-09-20T18:58:40.745261+0000","last_undegraded":"2024-09-20T18:58:40.745261+0000","last_fullsized":"2024-09-20T18:58:40.745261+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-20T18:58:33.660531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:33.660531+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:38.572897+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":413,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.742870+0000","last_change":"2024-09-20T18:58:32.526595+0000","last_active":"2024-09-20T18:58:40.742870+0000","last_peered":"2024-09-20T18:58:40.742870+0000","last_clean":"2024-09-20T18:58:40.742870+0000","last_became_active":"2024-09-20T18:49:49.959696+0000","last_became_peered":"2024-09-20T18:49:49.959696+0000","last_unstale":"2024-09-20T18:58:40.742870+0000","last_undegraded":"2024-09-20T18:58:40.742870+0000","last_fullsized":"2024-09-20T18:58:40.742870+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.526527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.526527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:36.077135+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":416,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.739125+0000","last_change":"2024-09-20T18:58:26.465305+0000","last_active":"2024-09-20T18:58:40.739125+0000","last_peered":"2024-09-20T18:58:40.739125+0000","last_clean":"2024-09-20T18:58:40.739125+0000","last_became_active":"2024-09-20T18:49:51.970663+0000","last_became_peered":"2024-09-20T18:49:51.970663+0000","last_unstale":"2024-09-20T18:58:40.739125+0000","last_undegraded":"2024-09-20T18:58:40.739125+0000","last_fullsized":"2024-09-20T18:58:40.739125+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:26.465264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:26.465264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:51.798999+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":418,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.746703+0000","last_change":"2024-09-20T18:58:00.906697+0000","last_active":"2024-09-20T18:58:40.746703+0000","last_peered":"2024-09-20T18:58:40.746703+0000","last_clean":"2024-09-20T18:58:40.746703+0000","last_became_active":"2024-09-20T18:55:28.827435+0000","last_became_peered":"2024-09-20T18:55:28.827435+0000","last_unstale":"2024-09-20T18:58:40.746703+0000","last_undegraded":"2024-09-20T18:58:40.746703+0000","last_fullsized":"2024-09-20T18:58:40.746703+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:00.906652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:00.906652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:23.434725+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":421,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.751060+0000","last_change":"2024-09-20T18:58:34.996936+0000","last_active":"2024-09-20T18:58:40.751060+0000","last_peered":"2024-09-20T18:58:40.751060+0000","last_clean":"2024-09-20T18:58:40.751060+0000","last_became_active":"2024-09-20T18:49:45.950480+0000","last_became_peered":"2024-09-20T18:49:45.950480+0000","last_unstale":"2024-09-20T18:58:40.751060+0000","last_undegraded":"2024-09-20T18:58:40.751060+0000","last_fullsized":"2024-09-20T18:58:40.751060+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:34.996893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:34.996893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:50.843422+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":444,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.744282+0000","last_change":"2024-09-20T18:58:28.253192+0000","last_active":"2024-09-20T18:58:40.744282+0000","last_peered":"2024-09-20T18:58:40.744282+0000","last_clean":"2024-09-20T18:58:40.744282+0000","last_became_active":"2024-09-20T18:49:47.950939+0000","last_became_peered":"2024-09-20T18:49:47.950939+0000","last_unstale":"2024-09-20T18:58:40.744282+0000","last_undegraded":"2024-09-20T18:58:40.744282+0000","last_fullsized":"2024-09-20T18:58:40.744282+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:58:28.253141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:28.253141+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:55.772150+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":404,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.745263+0000","last_change":"2024-09-20T18:57:14.941518+0000","last_active":"2024-09-20T18:58:40.745263+0000","last_peered":"2024-09-20T18:58:40.745263+0000","last_clean":"2024-09-20T18:58:40.745263+0000","last_became_active":"2024-09-20T18:49:49.958518+0000","last_became_peered":"2024-09-20T18:49:49.958518+0000","last_unstale":"2024-09-20T18:58:40.745263+0000","last_undegraded":"2024-09-20T18:58:40.745263+0000","last_fullsized":"2024-09-20T18:58:40.745263+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:14.941458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:14.941458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":416,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.751221+0000","last_change":"2024-09-20T18:58:35.125496+0000","last_active":"2024-09-20T18:58:40.751221+0000","last_peered":"2024-09-20T18:58:40.751221+0000","last_clean":"2024-09-20T18:58:40.751221+0000","last_became_active":"2024-09-20T18:55:28.833133+0000","last_became_peered":"2024-09-20T18:55:28.833133+0000","last_unstale":"2024-09-20T18:58:40.751221+0000","last_undegraded":"2024-09-20T18:58:40.751221+0000","last_fullsized":"2024-09-20T18:58:40.751221+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:35.125451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:35.125451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:56.313970+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"59'1","reported_seq":428,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.746194+0000","last_change":"2024-09-20T18:57:53.918080+0000","last_active":"2024-09-20T18:58:40.746194+0000","last_peered":"2024-09-20T18:58:40.746194+0000","last_clean":"2024-09-20T18:58:40.746194+0000","last_became_active":"2024-09-20T18:49:45.940100+0000","last_became_peered":"2024-09-20T18:49:45.940100+0000","last_unstale":"2024-09-20T18:58:40.746194+0000","last_undegraded":"2024-09-20T18:58:40.746194+0000","last_fullsized":"2024-09-20T18:58:40.746194+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:57:53.918022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:53.918022+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:03.242427+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":423,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.749161+0000","last_change":"2024-09-20T18:57:09.819080+0000","last_active":"2024-09-20T18:58:40.749161+0000","last_peered":"2024-09-20T18:58:40.749161+0000","last_clean":"2024-09-20T18:58:40.749161+0000","last_became_active":"2024-09-20T18:49:47.950909+0000","last_became_peered":"2024-09-20T18:49:47.950909+0000","last_unstale":"2024-09-20T18:58:40.749161+0000","last_undegraded":"2024-09-20T18:58:40.749161+0000","last_fullsized":"2024-09-20T18:58:40.749161+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:57:09.818999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:09.818999+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":414,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.745993+0000","last_change":"2024-09-20T18:54:44.239273+0000","last_active":"2024-09-20T18:58:40.745993+0000","last_peered":"2024-09-20T18:58:40.745993+0000","last_clean":"2024-09-20T18:58:40.745993+0000","last_became_active":"2024-09-20T18:49:49.959102+0000","last_became_peered":"2024-09-20T18:49:49.959102+0000","last_unstale":"2024-09-20T18:58:40.745993+0000","last_undegraded":"2024-09-20T18:58:40.745993+0000","last_fullsized":"2024-09-20T18:58:40.745993+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:44.239226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.239226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:17.660601+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":410,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.743320+0000","last_change":"2024-09-20T18:58:25.501434+0000","last_active":"2024-09-20T18:58:40.743320+0000","last_peered":"2024-09-20T18:58:40.743320+0000","last_clean":"2024-09-20T18:58:40.743320+0000","last_became_active":"2024-09-20T18:49:51.970058+0000","last_became_peered":"2024-09-20T18:49:51.970058+0000","last_unstale":"2024-09-20T18:58:40.743320+0000","last_undegraded":"2024-09-20T18:58:40.743320+0000","last_fullsized":"2024-09-20T18:58:40.743320+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T18:58:25.501384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:25.501384+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:36.383692+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":413,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.744053+0000","last_change":"2024-09-20T18:57:18.032140+0000","last_active":"2024-09-20T18:58:40.744053+0000","last_peered":"2024-09-20T18:58:40.744053+0000","last_clean":"2024-09-20T18:58:40.744053+0000","last_became_active":"2024-09-20T18:49:45.933078+0000","last_became_peered":"2024-09-20T18:49:45.933078+0000","last_unstale":"2024-09-20T18:58:40.744053+0000","last_undegraded":"2024-09-20T18:58:40.744053+0000","last_fullsized":"2024-09-20T18:58:40.744053+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:57:18.032067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:18.032067+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:42.727785+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":433,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.750327+0000","last_change":"2024-09-20T18:55:19.996522+0000","last_active":"2024-09-20T18:58:40.750327+0000","last_peered":"2024-09-20T18:58:40.750327+0000","last_clean":"2024-09-20T18:58:40.750327+0000","last_became_active":"2024-09-20T18:49:48.500809+0000","last_became_peered":"2024-09-20T18:49:48.500809+0000","last_unstale":"2024-09-20T18:58:40.750327+0000","last_undegraded":"2024-09-20T18:58:40.750327+0000","last_fullsized":"2024-09-20T18:58:40.750327+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:55:19.996328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:19.996328+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"184'9","reported_seq":231,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.746782+0000","last_change":"2024-09-20T18:57:48.853376+0000","last_active":"2024-09-20T18:58:40.746782+0000","last_peered":"2024-09-20T18:58:40.746782+0000","last_clean":"2024-09-20T18:58:40.746782+0000","last_became_active":"2024-09-20T18:55:28.820941+0000","last_became_peered":"2024-09-20T18:55:28.820941+0000","last_unstale":"2024-09-20T18:58:40.746782+0000","last_undegraded":"2024-09-20T18:58:40.746782+0000","last_fullsized":"2024-09-20T18:58:40.746782+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'9","last_scrub_stamp":"2024-09-20T18:57:48.853302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:48.853302+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:06.295025+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"251'281","reported_seq":302,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:41.572776+0000","last_change":"2024-09-20T18:58:33.827583+0000","last_active":"2024-09-20T18:58:41.572776+0000","last_peered":"2024-09-20T18:58:41.572776+0000","last_clean":"2024-09-20T18:58:41.572776+0000","last_became_active":"2024-09-20T18:58:33.827098+0000","last_became_peered":"2024-09-20T18:58:33.827098+0000","last_unstale":"2024-09-20T18:58:41.572776+0000","last_undegraded":"2024-09-20T18:58:41.572776+0000","last_fullsized":"2024-09-20T18:58:41.572776+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:41.122270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18415616,"num_objects":281,"num_object_clones":0,"num_object_copies":843,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":281,"num_write_kb":17984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":427,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.743766+0000","last_change":"2024-09-20T18:55:18.581290+0000","last_active":"2024-09-20T18:58:40.743766+0000","last_peered":"2024-09-20T18:58:40.743766+0000","last_clean":"2024-09-20T18:58:40.743766+0000","last_became_active":"2024-09-20T18:49:58.992302+0000","last_became_peered":"2024-09-20T18:49:58.992302+0000","last_unstale":"2024-09-20T18:58:40.743766+0000","last_undegraded":"2024-09-20T18:58:40.743766+0000","last_fullsized":"2024-09-20T18:58:40.743766+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:55:18.581200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:18.581200+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":413,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.746978+0000","last_change":"2024-09-20T18:54:54.288199+0000","last_active":"2024-09-20T18:58:40.746978+0000","last_peered":"2024-09-20T18:58:40.746978+0000","last_clean":"2024-09-20T18:58:40.746978+0000","last_became_active":"2024-09-20T18:49:45.936104+0000","last_became_peered":"2024-09-20T18:49:45.936104+0000","last_unstale":"2024-09-20T18:58:40.746978+0000","last_undegraded":"2024-09-20T18:58:40.746978+0000","last_fullsized":"2024-09-20T18:58:40.746978+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:54.288117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:54.288117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:21.124624+0000","scrub_duration":46,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":524,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.746793+0000","last_change":"2024-09-20T18:54:14.292660+0000","last_active":"2024-09-20T18:58:40.746793+0000","last_peered":"2024-09-20T18:58:40.746793+0000","last_clean":"2024-09-20T18:58:40.746793+0000","last_became_active":"2024-09-20T18:49:49.960697+0000","last_became_peered":"2024-09-20T18:49:49.960697+0000","last_unstale":"2024-09-20T18:58:40.746793+0000","last_undegraded":"2024-09-20T18:58:40.746793+0000","last_fullsized":"2024-09-20T18:58:40.746793+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:54:14.292610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:14.292610+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:10.776813+0000","scrub_duration":34,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":408,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.739611+0000","last_change":"2024-09-20T18:58:31.493658+0000","last_active":"2024-09-20T18:58:40.739611+0000","last_peered":"2024-09-20T18:58:40.739611+0000","last_clean":"2024-09-20T18:58:40.739611+0000","last_became_active":"2024-09-20T18:49:51.971399+0000","last_became_peered":"2024-09-20T18:49:51.971399+0000","last_unstale":"2024-09-20T18:58:40.739611+0000","last_undegraded":"2024-09-20T18:58:40.739611+0000","last_fullsized":"2024-09-20T18:58:40.739611+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:31.493595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:31.493595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:54.153071+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"251'119","reported_seq":140,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.884251+0000","last_change":"2024-09-20T18:58:33.820158+0000","last_active":"2024-09-20T18:58:40.884251+0000","last_peered":"2024-09-20T18:58:40.884251+0000","last_clean":"2024-09-20T18:58:40.884251+0000","last_became_active":"2024-09-20T18:58:33.819886+0000","last_became_peered":"2024-09-20T18:58:33.819886+0000","last_unstale":"2024-09-20T18:58:40.884251+0000","last_undegraded":"2024-09-20T18:58:40.884251+0000","last_fullsized":"2024-09-20T18:58:40.884251+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:02.637424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7798784,"num_objects":119,"num_object_clones":0,"num_object_copies":357,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119,"num_write_kb":7616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":449,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.746705+0000","last_change":"2024-09-20T18:58:08.025426+0000","last_active":"2024-09-20T18:58:40.746705+0000","last_peered":"2024-09-20T18:58:40.746705+0000","last_clean":"2024-09-20T18:58:40.746705+0000","last_became_active":"2024-09-20T18:49:48.499189+0000","last_became_peered":"2024-09-20T18:49:48.499189+0000","last_unstale":"2024-09-20T18:58:40.746705+0000","last_undegraded":"2024-09-20T18:58:40.746705+0000","last_fullsized":"2024-09-20T18:58:40.746705+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-20T18:58:08.025202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:08.025202+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:16.521840+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":413,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.753504+0000","last_change":"2024-09-20T18:58:25.159724+0000","last_active":"2024-09-20T18:58:40.753504+0000","last_peered":"2024-09-20T18:58:40.753504+0000","last_clean":"2024-09-20T18:58:40.753504+0000","last_became_active":"2024-09-20T18:49:45.940544+0000","last_became_peered":"2024-09-20T18:49:45.940544+0000","last_unstale":"2024-09-20T18:58:40.753504+0000","last_undegraded":"2024-09-20T18:58:40.753504+0000","last_fullsized":"2024-09-20T18:58:40.753504+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:25.159673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:25.159673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:40.807883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":415,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.745397+0000","last_change":"2024-09-20T18:54:51.229251+0000","last_active":"2024-09-20T18:58:40.745397+0000","last_peered":"2024-09-20T18:58:40.745397+0000","last_clean":"2024-09-20T18:58:40.745397+0000","last_became_active":"2024-09-20T18:49:49.963073+0000","last_became_peered":"2024-09-20T18:49:49.963073+0000","last_unstale":"2024-09-20T18:58:40.745397+0000","last_undegraded":"2024-09-20T18:58:40.745397+0000","last_fullsized":"2024-09-20T18:58:40.745397+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.229201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.229201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:20.666517+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":419,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.745642+0000","last_change":"2024-09-20T18:58:27.704583+0000","last_active":"2024-09-20T18:58:40.745642+0000","last_peered":"2024-09-20T18:58:40.745642+0000","last_clean":"2024-09-20T18:58:40.745642+0000","last_became_active":"2024-09-20T18:49:51.968793+0000","last_became_peered":"2024-09-20T18:49:51.968793+0000","last_unstale":"2024-09-20T18:58:40.745642+0000","last_undegraded":"2024-09-20T18:58:40.745642+0000","last_fullsized":"2024-09-20T18:58:40.745642+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:27.704520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:27.704520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:37.550976+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.c","version":"251'299","reported_seq":320,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:41.568044+0000","last_change":"2024-09-20T18:58:33.845752+0000","last_active":"2024-09-20T18:58:41.568044+0000","last_peered":"2024-09-20T18:58:41.568044+0000","last_clean":"2024-09-20T18:58:41.568044+0000","last_became_active":"2024-09-20T18:58:33.845463+0000","last_became_peered":"2024-09-20T18:58:33.845463+0000","last_unstale":"2024-09-20T18:58:41.568044+0000","last_undegraded":"2024-09-20T18:58:41.568044+0000","last_fullsized":"2024-09-20T18:58:41.568044+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":299,"log_dups_size":0,"ondisk_log_size":299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:58.514651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19595264,"num_objects":299,"num_object_clones":0,"num_object_copies":897,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":299,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":299,"num_write_kb":19136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":471,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.745443+0000","last_change":"2024-09-20T18:55:28.853375+0000","last_active":"2024-09-20T18:58:40.745443+0000","last_peered":"2024-09-20T18:58:40.745443+0000","last_clean":"2024-09-20T18:58:40.745443+0000","last_became_active":"2024-09-20T18:55:28.831057+0000","last_became_peered":"2024-09-20T18:55:28.831057+0000","last_unstale":"2024-09-20T18:58:40.745443+0000","last_undegraded":"2024-09-20T18:58:40.745443+0000","last_fullsized":"2024-09-20T18:58:40.745443+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-20T18:54:57.394893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.394893+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":415,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.755246+0000","last_change":"2024-09-20T18:58:29.939909+0000","last_active":"2024-09-20T18:58:40.755246+0000","last_peered":"2024-09-20T18:58:40.755246+0000","last_clean":"2024-09-20T18:58:40.755246+0000","last_became_active":"2024-09-20T18:49:45.940469+0000","last_became_peered":"2024-09-20T18:49:45.940469+0000","last_unstale":"2024-09-20T18:58:40.755246+0000","last_undegraded":"2024-09-20T18:58:40.755246+0000","last_fullsized":"2024-09-20T18:58:40.755246+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:58:29.939813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:29.939813+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:46.132321+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"66'8","reported_seq":537,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.746523+0000","last_change":"2024-09-20T18:58:23.114419+0000","last_active":"2024-09-20T18:58:40.746523+0000","last_peered":"2024-09-20T18:58:40.746523+0000","last_clean":"2024-09-20T18:58:40.746523+0000","last_became_active":"2024-09-20T18:49:49.956133+0000","last_became_peered":"2024-09-20T18:49:49.956133+0000","last_unstale":"2024-09-20T18:58:40.746523+0000","last_undegraded":"2024-09-20T18:58:40.746523+0000","last_fullsized":"2024-09-20T18:58:40.746523+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:58:23.114371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:23.114371+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:27.894374+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":417,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.742939+0000","last_change":"2024-09-20T18:58:29.530066+0000","last_active":"2024-09-20T18:58:40.742939+0000","last_peered":"2024-09-20T18:58:40.742939+0000","last_clean":"2024-09-20T18:58:40.742939+0000","last_became_active":"2024-09-20T18:55:28.818804+0000","last_became_peered":"2024-09-20T18:55:28.818804+0000","last_unstale":"2024-09-20T18:58:40.742939+0000","last_undegraded":"2024-09-20T18:58:40.742939+0000","last_fullsized":"2024-09-20T18:58:40.742939+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:29.530001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:29.530001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:47.518552+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"251'128","reported_seq":147,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.941222+0000","last_change":"2024-09-20T18:58:33.826834+0000","last_active":"2024-09-20T18:58:40.941222+0000","last_peered":"2024-09-20T18:58:40.941222+0000","last_clean":"2024-09-20T18:58:40.941222+0000","last_became_active":"2024-09-20T18:58:33.826568+0000","last_became_peered":"2024-09-20T18:58:33.826568+0000","last_unstale":"2024-09-20T18:58:40.941222+0000","last_undegraded":"2024-09-20T18:58:40.941222+0000","last_fullsized":"2024-09-20T18:58:40.941222+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:50.766101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":128,"num_object_clones":0,"num_object_copies":384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":128,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":128,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":440,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.744841+0000","last_change":"2024-09-20T18:58:03.053521+0000","last_active":"2024-09-20T18:58:40.744841+0000","last_peered":"2024-09-20T18:58:40.744841+0000","last_clean":"2024-09-20T18:58:40.744841+0000","last_became_active":"2024-09-20T18:49:47.949626+0000","last_became_peered":"2024-09-20T18:49:47.949626+0000","last_unstale":"2024-09-20T18:58:40.744841+0000","last_undegraded":"2024-09-20T18:58:40.744841+0000","last_fullsized":"2024-09-20T18:58:40.744841+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:58:03.053310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:03.053310+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:05.671653+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":429,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.740054+0000","last_change":"2024-09-20T18:58:32.528966+0000","last_active":"2024-09-20T18:58:40.740054+0000","last_peered":"2024-09-20T18:58:40.740054+0000","last_clean":"2024-09-20T18:58:40.740054+0000","last_became_active":"2024-09-20T18:49:45.942496+0000","last_became_peered":"2024-09-20T18:49:45.942496+0000","last_unstale":"2024-09-20T18:58:40.740054+0000","last_undegraded":"2024-09-20T18:58:40.740054+0000","last_fullsized":"2024-09-20T18:58:40.740054+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.528310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.528310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:37.417253+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"184'9","reported_seq":533,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.750717+0000","last_change":"2024-09-20T18:55:29.010062+0000","last_active":"2024-09-20T18:58:40.750717+0000","last_peered":"2024-09-20T18:58:40.750717+0000","last_clean":"2024-09-20T18:58:40.750717+0000","last_became_active":"2024-09-20T18:55:28.830643+0000","last_became_peered":"2024-09-20T18:55:28.830643+0000","last_unstale":"2024-09-20T18:58:40.750717+0000","last_undegraded":"2024-09-20T18:58:40.750717+0000","last_fullsized":"2024-09-20T18:58:40.750717+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:29.010015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:29.010015+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":394,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.751724+0000","last_change":"2024-09-20T18:58:26.948165+0000","last_active":"2024-09-20T18:58:40.751724+0000","last_peered":"2024-09-20T18:58:40.751724+0000","last_clean":"2024-09-20T18:58:40.751724+0000","last_became_active":"2024-09-20T18:49:51.964780+0000","last_became_peered":"2024-09-20T18:49:51.964780+0000","last_unstale":"2024-09-20T18:58:40.751724+0000","last_undegraded":"2024-09-20T18:58:40.751724+0000","last_fullsized":"2024-09-20T18:58:40.751724+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:26.948102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:26.948102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:28.716180+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"251'115","reported_seq":136,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.874614+0000","last_change":"2024-09-20T18:58:33.845804+0000","last_active":"2024-09-20T18:58:40.874614+0000","last_peered":"2024-09-20T18:58:40.874614+0000","last_clean":"2024-09-20T18:58:40.874614+0000","last_became_active":"2024-09-20T18:58:33.845493+0000","last_became_peered":"2024-09-20T18:58:33.845493+0000","last_unstale":"2024-09-20T18:58:40.874614+0000","last_undegraded":"2024-09-20T18:58:40.874614+0000","last_fullsized":"2024-09-20T18:58:40.874614+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:59.386071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7536640,"num_objects":115,"num_object_clones":0,"num_object_copies":345,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115,"num_write_kb":7360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":446,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.750801+0000","last_change":"2024-09-20T18:54:14.923895+0000","last_active":"2024-09-20T18:58:40.750801+0000","last_peered":"2024-09-20T18:58:40.750801+0000","last_clean":"2024-09-20T18:58:40.750801+0000","last_became_active":"2024-09-20T18:49:48.497736+0000","last_became_peered":"2024-09-20T18:49:48.497736+0000","last_unstale":"2024-09-20T18:58:40.750801+0000","last_undegraded":"2024-09-20T18:58:40.750801+0000","last_fullsized":"2024-09-20T18:58:40.750801+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-20T18:54:14.923682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:14.923682+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:39.022337+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":405,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.744002+0000","last_change":"2024-09-20T18:57:09.940634+0000","last_active":"2024-09-20T18:58:40.744002+0000","last_peered":"2024-09-20T18:58:40.744002+0000","last_clean":"2024-09-20T18:58:40.744002+0000","last_became_active":"2024-09-20T18:49:45.942253+0000","last_became_peered":"2024-09-20T18:49:45.942253+0000","last_unstale":"2024-09-20T18:58:40.744002+0000","last_undegraded":"2024-09-20T18:58:40.744002+0000","last_fullsized":"2024-09-20T18:58:40.744002+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:09.940517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:09.940517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:19.931681+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":152,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.749655+0000","last_change":"2024-09-20T18:55:28.830954+0000","last_active":"2024-09-20T18:58:40.749655+0000","last_peered":"2024-09-20T18:58:40.749655+0000","last_clean":"2024-09-20T18:58:40.749655+0000","last_became_active":"2024-09-20T18:55:28.830645+0000","last_became_peered":"2024-09-20T18:55:28.830645+0000","last_unstale":"2024-09-20T18:58:40.749655+0000","last_undegraded":"2024-09-20T18:58:40.749655+0000","last_fullsized":"2024-09-20T18:58:40.749655+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:50.610871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:50.610871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":424,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.743766+0000","last_change":"2024-09-20T18:57:58.973310+0000","last_active":"2024-09-20T18:58:40.743766+0000","last_peered":"2024-09-20T18:58:40.743766+0000","last_clean":"2024-09-20T18:58:40.743766+0000","last_became_active":"2024-09-20T18:55:28.821588+0000","last_became_peered":"2024-09-20T18:55:28.821588+0000","last_unstale":"2024-09-20T18:58:40.743766+0000","last_undegraded":"2024-09-20T18:58:40.743766+0000","last_fullsized":"2024-09-20T18:58:40.743766+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:58.973176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:58.973176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:03.568279+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"251'371","reported_seq":392,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:42.112534+0000","last_change":"2024-09-20T18:58:33.822011+0000","last_active":"2024-09-20T18:58:42.112534+0000","last_peered":"2024-09-20T18:58:42.112534+0000","last_clean":"2024-09-20T18:58:42.112534+0000","last_became_active":"2024-09-20T18:58:33.820506+0000","last_became_peered":"2024-09-20T18:58:33.820506+0000","last_unstale":"2024-09-20T18:58:42.112534+0000","last_undegraded":"2024-09-20T18:58:42.112534+0000","last_fullsized":"2024-09-20T18:58:42.112534+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:55.828171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24313856,"num_objects":371,"num_object_clones":0,"num_object_copies":1113,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":371,"num_write_kb":23744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"66'28","reported_seq":573,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.743915+0000","last_change":"2024-09-20T18:58:07.018449+0000","last_active":"2024-09-20T18:58:40.743915+0000","last_peered":"2024-09-20T18:58:40.743915+0000","last_clean":"2024-09-20T18:58:40.743915+0000","last_became_active":"2024-09-20T18:49:47.955335+0000","last_became_peered":"2024-09-20T18:49:47.955335+0000","last_unstale":"2024-09-20T18:58:40.743915+0000","last_undegraded":"2024-09-20T18:58:40.743915+0000","last_fullsized":"2024-09-20T18:58:40.743915+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'28","last_scrub_stamp":"2024-09-20T18:58:07.018340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:07.018340+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:28.256912+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":414,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.746044+0000","last_change":"2024-09-20T18:57:21.807755+0000","last_active":"2024-09-20T18:58:40.746044+0000","last_peered":"2024-09-20T18:58:40.746044+0000","last_clean":"2024-09-20T18:58:40.746044+0000","last_became_active":"2024-09-20T18:55:28.831574+0000","last_became_peered":"2024-09-20T18:55:28.831574+0000","last_unstale":"2024-09-20T18:58:40.746044+0000","last_undegraded":"2024-09-20T18:58:40.746044+0000","last_fullsized":"2024-09-20T18:58:40.746044+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:21.807703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:21.807703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:42.821495+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1],"acting":[4,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":450,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.739578+0000","last_change":"2024-09-20T18:55:28.854615+0000","last_active":"2024-09-20T18:58:40.739578+0000","last_peered":"2024-09-20T18:58:40.739578+0000","last_clean":"2024-09-20T18:58:40.739578+0000","last_became_active":"2024-09-20T18:55:28.837226+0000","last_became_peered":"2024-09-20T18:55:28.837226+0000","last_unstale":"2024-09-20T18:58:40.739578+0000","last_undegraded":"2024-09-20T18:58:40.739578+0000","last_fullsized":"2024-09-20T18:58:40.739578+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-20T18:54:51.227531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.227531+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:43.598074+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0012781159999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":414,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.750037+0000","last_change":"2024-09-20T18:55:13.949348+0000","last_active":"2024-09-20T18:58:40.750037+0000","last_peered":"2024-09-20T18:58:40.750037+0000","last_clean":"2024-09-20T18:58:40.750037+0000","last_became_active":"2024-09-20T18:49:49.953971+0000","last_became_peered":"2024-09-20T18:49:49.953971+0000","last_unstale":"2024-09-20T18:58:40.750037+0000","last_undegraded":"2024-09-20T18:58:40.750037+0000","last_fullsized":"2024-09-20T18:58:40.750037+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:13.949269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:13.949269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":393,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.749547+0000","last_change":"2024-09-20T18:57:12.744116+0000","last_active":"2024-09-20T18:58:40.749547+0000","last_peered":"2024-09-20T18:58:40.749547+0000","last_clean":"2024-09-20T18:58:40.749547+0000","last_became_active":"2024-09-20T18:49:51.969400+0000","last_became_peered":"2024-09-20T18:49:51.969400+0000","last_unstale":"2024-09-20T18:58:40.749547+0000","last_undegraded":"2024-09-20T18:58:40.749547+0000","last_fullsized":"2024-09-20T18:58:40.749547+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:12.744040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:12.744040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"251'122","reported_seq":143,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.941290+0000","last_change":"2024-09-20T18:58:33.829042+0000","last_active":"2024-09-20T18:58:40.941290+0000","last_peered":"2024-09-20T18:58:40.941290+0000","last_clean":"2024-09-20T18:58:40.941290+0000","last_became_active":"2024-09-20T18:58:33.828787+0000","last_became_peered":"2024-09-20T18:58:33.828787+0000","last_unstale":"2024-09-20T18:58:40.941290+0000","last_undegraded":"2024-09-20T18:58:40.941290+0000","last_fullsized":"2024-09-20T18:58:40.941290+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:57.211364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7995392,"num_objects":122,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":122,"num_write_kb":7808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":452,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.744786+0000","last_change":"2024-09-20T18:58:06.079117+0000","last_active":"2024-09-20T18:58:40.744786+0000","last_peered":"2024-09-20T18:58:40.744786+0000","last_clean":"2024-09-20T18:58:40.744786+0000","last_became_active":"2024-09-20T18:49:47.950904+0000","last_became_peered":"2024-09-20T18:49:47.950904+0000","last_unstale":"2024-09-20T18:58:40.744786+0000","last_undegraded":"2024-09-20T18:58:40.744786+0000","last_fullsized":"2024-09-20T18:58:40.744786+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-20T18:58:06.078352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:06.078352+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:26.574178+0000","scrub_duration":59,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":436,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.748060+0000","last_change":"2024-09-20T18:58:28.987535+0000","last_active":"2024-09-20T18:58:40.748060+0000","last_peered":"2024-09-20T18:58:40.748060+0000","last_clean":"2024-09-20T18:58:40.748060+0000","last_became_active":"2024-09-20T18:55:28.830269+0000","last_became_peered":"2024-09-20T18:55:28.830269+0000","last_unstale":"2024-09-20T18:58:40.748060+0000","last_undegraded":"2024-09-20T18:58:40.748060+0000","last_fullsized":"2024-09-20T18:58:40.748060+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:28.987458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:28.987458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:43.384017+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":425,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.742648+0000","last_change":"2024-09-20T18:58:24.229997+0000","last_active":"2024-09-20T18:58:40.742648+0000","last_peered":"2024-09-20T18:58:40.742648+0000","last_clean":"2024-09-20T18:58:40.742648+0000","last_became_active":"2024-09-20T18:49:44.937135+0000","last_became_peered":"2024-09-20T18:49:44.937135+0000","last_unstale":"2024-09-20T18:58:40.742648+0000","last_undegraded":"2024-09-20T18:58:40.742648+0000","last_fullsized":"2024-09-20T18:58:40.742648+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:58:24.229938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:58:24.229938+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:47.181712+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00095558800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":400,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.743525+0000","last_change":"2024-09-20T18:51:59.796450+0000","last_active":"2024-09-20T18:58:40.743525+0000","last_peered":"2024-09-20T18:58:40.743525+0000","last_clean":"2024-09-20T18:58:40.743525+0000","last_became_active":"2024-09-20T18:49:49.954424+0000","last_became_peered":"2024-09-20T18:49:49.954424+0000","last_unstale":"2024-09-20T18:58:40.743525+0000","last_undegraded":"2024-09-20T18:58:40.743525+0000","last_fullsized":"2024-09-20T18:58:40.743525+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:59.796383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:59.796383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:21.220437+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":404,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.743128+0000","last_change":"2024-09-20T18:58:40.743128+0000","last_active":"2024-09-20T18:58:40.743128+0000","last_peered":"2024-09-20T18:58:40.743128+0000","last_clean":"2024-09-20T18:58:40.743128+0000","last_became_active":"2024-09-20T18:55:28.825812+0000","last_became_peered":"2024-09-20T18:55:28.825812+0000","last_unstale":"2024-09-20T18:58:40.743128+0000","last_undegraded":"2024-09-20T18:58:40.743128+0000","last_fullsized":"2024-09-20T18:58:40.743128+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:39.814259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:39.814259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:07.517130+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"251'372","reported_seq":393,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:42.123948+0000","last_change":"2024-09-20T18:58:33.823936+0000","last_active":"2024-09-20T18:58:42.123948+0000","last_peered":"2024-09-20T18:58:42.123948+0000","last_clean":"2024-09-20T18:58:42.123948+0000","last_became_active":"2024-09-20T18:58:33.822678+0000","last_became_peered":"2024-09-20T18:58:33.822678+0000","last_unstale":"2024-09-20T18:58:42.123948+0000","last_undegraded":"2024-09-20T18:58:42.123948+0000","last_fullsized":"2024-09-20T18:58:42.123948+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:58.099759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24379392,"num_objects":372,"num_object_clones":0,"num_object_copies":1116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":372,"num_write_kb":23808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":444,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.750455+0000","last_change":"2024-09-20T18:55:14.951950+0000","last_active":"2024-09-20T18:58:40.750455+0000","last_peered":"2024-09-20T18:58:40.750455+0000","last_clean":"2024-09-20T18:58:40.750455+0000","last_became_active":"2024-09-20T18:49:47.949230+0000","last_became_peered":"2024-09-20T18:49:47.949230+0000","last_unstale":"2024-09-20T18:58:40.750455+0000","last_undegraded":"2024-09-20T18:58:40.750455+0000","last_fullsized":"2024-09-20T18:58:40.750455+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:55:14.951892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:14.951892+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":422,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.750070+0000","last_change":"2024-09-20T18:54:27.905856+0000","last_active":"2024-09-20T18:58:40.750070+0000","last_peered":"2024-09-20T18:58:40.750070+0000","last_clean":"2024-09-20T18:58:40.750070+0000","last_became_active":"2024-09-20T18:49:45.939197+0000","last_became_peered":"2024-09-20T18:49:45.939197+0000","last_unstale":"2024-09-20T18:58:40.750070+0000","last_undegraded":"2024-09-20T18:58:40.750070+0000","last_fullsized":"2024-09-20T18:58:40.750070+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:27.905803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:27.905803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":962,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.742360+0000","last_change":"2024-09-20T18:54:26.359887+0000","last_active":"2024-09-20T18:58:40.742360+0000","last_peered":"2024-09-20T18:58:40.742360+0000","last_clean":"2024-09-20T18:58:40.742360+0000","last_became_active":"2024-09-20T18:49:44.936819+0000","last_became_peered":"2024-09-20T18:49:44.936819+0000","last_unstale":"2024-09-20T18:58:40.742360+0000","last_undegraded":"2024-09-20T18:58:40.742360+0000","last_fullsized":"2024-09-20T18:58:40.742360+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-20T18:54:26.359807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:54:26.359807+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0010996199999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":529,"num_read_kb":524,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":413,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.740488+0000","last_change":"2024-09-20T18:57:54.362085+0000","last_active":"2024-09-20T18:58:40.740488+0000","last_peered":"2024-09-20T18:58:40.740488+0000","last_clean":"2024-09-20T18:58:40.740488+0000","last_became_active":"2024-09-20T18:49:49.961066+0000","last_became_peered":"2024-09-20T18:49:49.961066+0000","last_unstale":"2024-09-20T18:58:40.740488+0000","last_undegraded":"2024-09-20T18:58:40.740488+0000","last_fullsized":"2024-09-20T18:58:40.740488+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:54.362038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:54.362038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:08.560302+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":417,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.743565+0000","last_change":"2024-09-20T18:58:08.019154+0000","last_active":"2024-09-20T18:58:40.743565+0000","last_peered":"2024-09-20T18:58:40.743565+0000","last_clean":"2024-09-20T18:58:40.743565+0000","last_became_active":"2024-09-20T18:49:51.970343+0000","last_became_peered":"2024-09-20T18:49:51.970343+0000","last_unstale":"2024-09-20T18:58:40.743565+0000","last_undegraded":"2024-09-20T18:58:40.743565+0000","last_fullsized":"2024-09-20T18:58:40.743565+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:08.019092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:08.019092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:32.791415+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"251'259","reported_seq":280,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:41.576672+0000","last_change":"2024-09-20T18:58:33.820307+0000","last_active":"2024-09-20T18:58:41.576672+0000","last_peered":"2024-09-20T18:58:41.576672+0000","last_clean":"2024-09-20T18:58:41.576672+0000","last_became_active":"2024-09-20T18:58:33.820027+0000","last_became_peered":"2024-09-20T18:58:33.820027+0000","last_unstale":"2024-09-20T18:58:41.576672+0000","last_undegraded":"2024-09-20T18:58:41.576672+0000","last_fullsized":"2024-09-20T18:58:41.576672+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:41.996589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16973824,"num_objects":259,"num_object_clones":0,"num_object_copies":777,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":259,"num_write_kb":16576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"184'17","reported_seq":260,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.746303+0000","last_change":"2024-09-20T18:57:22.816240+0000","last_active":"2024-09-20T18:58:40.746303+0000","last_peered":"2024-09-20T18:58:40.746303+0000","last_clean":"2024-09-20T18:58:40.746303+0000","last_became_active":"2024-09-20T18:55:28.831024+0000","last_became_peered":"2024-09-20T18:55:28.831024+0000","last_unstale":"2024-09-20T18:58:40.746303+0000","last_undegraded":"2024-09-20T18:58:40.746303+0000","last_fullsized":"2024-09-20T18:58:40.746303+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'17","last_scrub_stamp":"2024-09-20T18:57:22.816154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:22.816154+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":24,"num_read_kb":14,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":428,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.750556+0000","last_change":"2024-09-20T18:58:32.100672+0000","last_active":"2024-09-20T18:58:40.750556+0000","last_peered":"2024-09-20T18:58:40.750556+0000","last_clean":"2024-09-20T18:58:40.750556+0000","last_became_active":"2024-09-20T18:55:28.830147+0000","last_became_peered":"2024-09-20T18:55:28.830147+0000","last_unstale":"2024-09-20T18:58:40.750556+0000","last_undegraded":"2024-09-20T18:58:40.750556+0000","last_fullsized":"2024-09-20T18:58:40.750556+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.100625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.100625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:56.577105+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":475,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.742473+0000","last_change":"2024-09-20T18:55:28.844060+0000","last_active":"2024-09-20T18:58:40.742473+0000","last_peered":"2024-09-20T18:58:40.742473+0000","last_clean":"2024-09-20T18:58:40.742473+0000","last_became_active":"2024-09-20T18:55:28.829325+0000","last_became_peered":"2024-09-20T18:55:28.829325+0000","last_unstale":"2024-09-20T18:58:40.742473+0000","last_undegraded":"2024-09-20T18:58:40.742473+0000","last_fullsized":"2024-09-20T18:58:40.742473+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T18:55:13.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T18:55:13.696607+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":406,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.743435+0000","last_change":"2024-09-20T18:58:23.545370+0000","last_active":"2024-09-20T18:58:40.743435+0000","last_peered":"2024-09-20T18:58:40.743435+0000","last_clean":"2024-09-20T18:58:40.743435+0000","last_became_active":"2024-09-20T18:49:49.956128+0000","last_became_peered":"2024-09-20T18:49:49.956128+0000","last_unstale":"2024-09-20T18:58:40.743435+0000","last_undegraded":"2024-09-20T18:58:40.743435+0000","last_fullsized":"2024-09-20T18:58:40.743435+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:23.545321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:23.545321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:34.942242+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":417,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.740253+0000","last_change":"2024-09-20T18:58:03.523127+0000","last_active":"2024-09-20T18:58:40.740253+0000","last_peered":"2024-09-20T18:58:40.740253+0000","last_clean":"2024-09-20T18:58:40.740253+0000","last_became_active":"2024-09-20T18:49:51.971332+0000","last_became_peered":"2024-09-20T18:49:51.971332+0000","last_unstale":"2024-09-20T18:58:40.740253+0000","last_undegraded":"2024-09-20T18:58:40.740253+0000","last_fullsized":"2024-09-20T18:58:40.740253+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:03.523078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:03.523078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:14.059180+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"251'106","reported_seq":127,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.870363+0000","last_change":"2024-09-20T18:58:33.825100+0000","last_active":"2024-09-20T18:58:40.870363+0000","last_peered":"2024-09-20T18:58:40.870363+0000","last_clean":"2024-09-20T18:58:40.870363+0000","last_became_active":"2024-09-20T18:58:33.824751+0000","last_became_peered":"2024-09-20T18:58:33.824751+0000","last_unstale":"2024-09-20T18:58:40.870363+0000","last_undegraded":"2024-09-20T18:58:40.870363+0000","last_fullsized":"2024-09-20T18:58:40.870363+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:33.144864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6946816,"num_objects":106,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106,"num_write_kb":6784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":448,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.746885+0000","last_change":"2024-09-20T18:58:24.616639+0000","last_active":"2024-09-20T18:58:40.746885+0000","last_peered":"2024-09-20T18:58:40.746885+0000","last_clean":"2024-09-20T18:58:40.746885+0000","last_became_active":"2024-09-20T18:55:28.818269+0000","last_became_peered":"2024-09-20T18:55:28.818269+0000","last_unstale":"2024-09-20T18:58:40.746885+0000","last_undegraded":"2024-09-20T18:58:40.746885+0000","last_fullsized":"2024-09-20T18:58:40.746885+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:58:24.616580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:24.616580+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:24.623208+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":415,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.745626+0000","last_change":"2024-09-20T18:58:32.694078+0000","last_active":"2024-09-20T18:58:40.745626+0000","last_peered":"2024-09-20T18:58:40.745626+0000","last_clean":"2024-09-20T18:58:40.745626+0000","last_became_active":"2024-09-20T18:49:45.944579+0000","last_became_peered":"2024-09-20T18:49:45.944579+0000","last_unstale":"2024-09-20T18:58:40.745626+0000","last_undegraded":"2024-09-20T18:58:40.745626+0000","last_fullsized":"2024-09-20T18:58:40.745626+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.693993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.693993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:48.788982+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":419,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.740154+0000","last_change":"2024-09-20T18:55:28.830634+0000","last_active":"2024-09-20T18:58:40.740154+0000","last_peered":"2024-09-20T18:58:40.740154+0000","last_clean":"2024-09-20T18:58:40.740154+0000","last_became_active":"2024-09-20T18:55:28.827785+0000","last_became_peered":"2024-09-20T18:55:28.827785+0000","last_unstale":"2024-09-20T18:58:40.740154+0000","last_undegraded":"2024-09-20T18:58:40.740154+0000","last_fullsized":"2024-09-20T18:58:40.740154+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:11.401777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:11.401777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:39.410398+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":407,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.751464+0000","last_change":"2024-09-20T18:57:23.721754+0000","last_active":"2024-09-20T18:58:40.751464+0000","last_peered":"2024-09-20T18:58:40.751464+0000","last_clean":"2024-09-20T18:58:40.751464+0000","last_became_active":"2024-09-20T18:55:28.833115+0000","last_became_peered":"2024-09-20T18:55:28.833115+0000","last_unstale":"2024-09-20T18:58:40.751464+0000","last_undegraded":"2024-09-20T18:58:40.751464+0000","last_fullsized":"2024-09-20T18:58:40.751464+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:23.721643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:23.721643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.0","version":"251'219","reported_seq":240,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:41.420261+0000","last_change":"2024-09-20T18:58:33.828344+0000","last_active":"2024-09-20T18:58:41.420261+0000","last_peered":"2024-09-20T18:58:41.420261+0000","last_clean":"2024-09-20T18:58:41.420261+0000","last_became_active":"2024-09-20T18:58:33.828089+0000","last_became_peered":"2024-09-20T18:58:33.828089+0000","last_unstale":"2024-09-20T18:58:41.420261+0000","last_undegraded":"2024-09-20T18:58:41.420261+0000","last_fullsized":"2024-09-20T18:58:41.420261+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:48.784974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14352384,"num_objects":219,"num_object_clones":0,"num_object_copies":657,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":219,"num_write_kb":14016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":446,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.746487+0000","last_change":"2024-09-20T18:58:36.716146+0000","last_active":"2024-09-20T18:58:40.746487+0000","last_peered":"2024-09-20T18:58:40.746487+0000","last_clean":"2024-09-20T18:58:40.746487+0000","last_became_active":"2024-09-20T18:49:47.947636+0000","last_became_peered":"2024-09-20T18:49:47.947636+0000","last_unstale":"2024-09-20T18:58:40.746487+0000","last_undegraded":"2024-09-20T18:58:40.746487+0000","last_fullsized":"2024-09-20T18:58:40.746487+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-20T18:58:36.716079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:36.716079+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:36.836323+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":284,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.745253+0000","last_change":"2024-09-20T18:58:25.649818+0000","last_active":"2024-09-20T18:58:40.745253+0000","last_peered":"2024-09-20T18:58:40.745253+0000","last_clean":"2024-09-20T18:58:40.745253+0000","last_became_active":"2024-09-20T18:55:28.821959+0000","last_became_peered":"2024-09-20T18:55:28.821959+0000","last_unstale":"2024-09-20T18:58:40.745253+0000","last_undegraded":"2024-09-20T18:58:40.745253+0000","last_fullsized":"2024-09-20T18:58:40.745253+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:25.649733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:25.649733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:46.966857+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":412,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.743653+0000","last_change":"2024-09-20T18:58:36.100033+0000","last_active":"2024-09-20T18:58:40.743653+0000","last_peered":"2024-09-20T18:58:40.743653+0000","last_clean":"2024-09-20T18:58:40.743653+0000","last_became_active":"2024-09-20T18:49:49.959160+0000","last_became_peered":"2024-09-20T18:49:49.959160+0000","last_unstale":"2024-09-20T18:58:40.743653+0000","last_undegraded":"2024-09-20T18:58:40.743653+0000","last_fullsized":"2024-09-20T18:58:40.743653+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:36.099985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:36.099985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:56.689064+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":411,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.743902+0000","last_change":"2024-09-20T18:58:25.227679+0000","last_active":"2024-09-20T18:58:40.743902+0000","last_peered":"2024-09-20T18:58:40.743902+0000","last_clean":"2024-09-20T18:58:40.743902+0000","last_became_active":"2024-09-20T18:49:51.970108+0000","last_became_peered":"2024-09-20T18:49:51.970108+0000","last_unstale":"2024-09-20T18:58:40.743902+0000","last_undegraded":"2024-09-20T18:58:40.743902+0000","last_fullsized":"2024-09-20T18:58:40.743902+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:25.227630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:25.227630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:27.041924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1","version":"251'290","reported_seq":311,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:41.578359+0000","last_change":"2024-09-20T18:58:33.845689+0000","last_active":"2024-09-20T18:58:41.578359+0000","last_peered":"2024-09-20T18:58:41.578359+0000","last_clean":"2024-09-20T18:58:41.578359+0000","last_became_active":"2024-09-20T18:58:33.845352+0000","last_became_peered":"2024-09-20T18:58:33.845352+0000","last_unstale":"2024-09-20T18:58:41.578359+0000","last_undegraded":"2024-09-20T18:58:41.578359+0000","last_fullsized":"2024-09-20T18:58:41.578359+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:52.065866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19005440,"num_objects":290,"num_object_clones":0,"num_object_copies":870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":290,"num_write_kb":18560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,4],"acting":[3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":444,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.746091+0000","last_change":"2024-09-20T18:55:28.846773+0000","last_active":"2024-09-20T18:58:40.746091+0000","last_peered":"2024-09-20T18:58:40.746091+0000","last_clean":"2024-09-20T18:58:40.746091+0000","last_became_active":"2024-09-20T18:55:28.817101+0000","last_became_peered":"2024-09-20T18:55:28.817101+0000","last_unstale":"2024-09-20T18:58:40.746091+0000","last_undegraded":"2024-09-20T18:58:40.746091+0000","last_fullsized":"2024-09-20T18:58:40.746091+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:54:17.219548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:17.219548+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:12.738146+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":420,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.750351+0000","last_change":"2024-09-20T18:58:31.126134+0000","last_active":"2024-09-20T18:58:40.750351+0000","last_peered":"2024-09-20T18:58:40.750351+0000","last_clean":"2024-09-20T18:58:40.750351+0000","last_became_active":"2024-09-20T18:49:45.937264+0000","last_became_peered":"2024-09-20T18:49:45.937264+0000","last_unstale":"2024-09-20T18:58:40.750351+0000","last_undegraded":"2024-09-20T18:58:40.750351+0000","last_fullsized":"2024-09-20T18:58:40.750351+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:31.126087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:31.126087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:45.329561+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":531,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.743540+0000","last_change":"2024-09-20T18:55:17.549660+0000","last_active":"2024-09-20T18:58:40.743540+0000","last_peered":"2024-09-20T18:58:40.743540+0000","last_clean":"2024-09-20T18:58:40.743540+0000","last_became_active":"2024-09-20T18:49:49.957005+0000","last_became_peered":"2024-09-20T18:49:49.957005+0000","last_unstale":"2024-09-20T18:58:40.743540+0000","last_undegraded":"2024-09-20T18:58:40.743540+0000","last_fullsized":"2024-09-20T18:58:40.743540+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:17.549598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:17.549598+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":410,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.746616+0000","last_change":"2024-09-20T18:58:28.688247+0000","last_active":"2024-09-20T18:58:40.746616+0000","last_peered":"2024-09-20T18:58:40.746616+0000","last_clean":"2024-09-20T18:58:40.746616+0000","last_became_active":"2024-09-20T18:49:51.969355+0000","last_became_peered":"2024-09-20T18:49:51.969355+0000","last_unstale":"2024-09-20T18:58:40.746616+0000","last_undegraded":"2024-09-20T18:58:40.746616+0000","last_fullsized":"2024-09-20T18:58:40.746616+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:28.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:28.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:31.796237+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"251'129","reported_seq":150,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.954735+0000","last_change":"2024-09-20T18:58:33.826744+0000","last_active":"2024-09-20T18:58:40.954735+0000","last_peered":"2024-09-20T18:58:40.954735+0000","last_clean":"2024-09-20T18:58:40.954735+0000","last_became_active":"2024-09-20T18:58:33.826480+0000","last_became_peered":"2024-09-20T18:58:33.826480+0000","last_unstale":"2024-09-20T18:58:40.954735+0000","last_undegraded":"2024-09-20T18:58:40.954735+0000","last_fullsized":"2024-09-20T18:58:40.954735+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:44.033875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8454144,"num_objects":129,"num_object_clones":0,"num_object_copies":387,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":8256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":433,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.753887+0000","last_change":"2024-09-20T18:54:54.897991+0000","last_active":"2024-09-20T18:58:40.753887+0000","last_peered":"2024-09-20T18:58:40.753887+0000","last_clean":"2024-09-20T18:58:40.753887+0000","last_became_active":"2024-09-20T18:49:47.946008+0000","last_became_peered":"2024-09-20T18:49:47.946008+0000","last_unstale":"2024-09-20T18:58:40.753887+0000","last_undegraded":"2024-09-20T18:58:40.753887+0000","last_fullsized":"2024-09-20T18:58:40.753887+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:54.897933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:54.897933+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":435,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.739904+0000","last_change":"2024-09-20T18:57:56.405392+0000","last_active":"2024-09-20T18:58:40.739904+0000","last_peered":"2024-09-20T18:58:40.739904+0000","last_clean":"2024-09-20T18:58:40.739904+0000","last_became_active":"2024-09-20T18:55:28.833095+0000","last_became_peered":"2024-09-20T18:55:28.833095+0000","last_unstale":"2024-09-20T18:58:40.739904+0000","last_undegraded":"2024-09-20T18:58:40.739904+0000","last_fullsized":"2024-09-20T18:58:40.739904+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:56.405349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:56.405349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:22.442121+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":416,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.743084+0000","last_change":"2024-09-20T18:54:43.379898+0000","last_active":"2024-09-20T18:58:40.743084+0000","last_peered":"2024-09-20T18:58:40.743084+0000","last_clean":"2024-09-20T18:58:40.743084+0000","last_became_active":"2024-09-20T18:49:49.956767+0000","last_became_peered":"2024-09-20T18:49:49.956767+0000","last_unstale":"2024-09-20T18:58:40.743084+0000","last_undegraded":"2024-09-20T18:58:40.743084+0000","last_fullsized":"2024-09-20T18:58:40.743084+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:43.379706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:43.379706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:38.219215+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":402,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.749932+0000","last_change":"2024-09-20T18:54:24.976627+0000","last_active":"2024-09-20T18:58:40.749932+0000","last_peered":"2024-09-20T18:58:40.749932+0000","last_clean":"2024-09-20T18:58:40.749932+0000","last_became_active":"2024-09-20T18:49:51.964098+0000","last_became_peered":"2024-09-20T18:49:51.964098+0000","last_unstale":"2024-09-20T18:58:40.749932+0000","last_undegraded":"2024-09-20T18:58:40.749932+0000","last_fullsized":"2024-09-20T18:58:40.749932+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:24.976584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:24.976584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"251'129","reported_seq":147,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.935017+0000","last_change":"2024-09-20T18:58:33.824658+0000","last_active":"2024-09-20T18:58:40.935017+0000","last_peered":"2024-09-20T18:58:40.935017+0000","last_clean":"2024-09-20T18:58:40.935017+0000","last_became_active":"2024-09-20T18:58:33.824342+0000","last_became_peered":"2024-09-20T18:58:33.824342+0000","last_unstale":"2024-09-20T18:58:40.935017+0000","last_undegraded":"2024-09-20T18:58:40.935017+0000","last_fullsized":"2024-09-20T18:58:40.935017+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:49.056054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8454144,"num_objects":129,"num_object_clones":0,"num_object_copies":387,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":8256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":229,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.744776+0000","last_change":"2024-09-20T18:57:22.033602+0000","last_active":"2024-09-20T18:58:40.744776+0000","last_peered":"2024-09-20T18:58:40.744776+0000","last_clean":"2024-09-20T18:58:40.744776+0000","last_became_active":"2024-09-20T18:55:28.828563+0000","last_became_peered":"2024-09-20T18:55:28.828563+0000","last_unstale":"2024-09-20T18:58:40.744776+0000","last_undegraded":"2024-09-20T18:58:40.744776+0000","last_fullsized":"2024-09-20T18:58:40.744776+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-20T18:57:22.033475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:22.033475+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:43.998101+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":31,"num_read_kb":20,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,5],"acting":[1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":428,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.743856+0000","last_change":"2024-09-20T18:55:28.835767+0000","last_active":"2024-09-20T18:58:40.743856+0000","last_peered":"2024-09-20T18:58:40.743856+0000","last_clean":"2024-09-20T18:58:40.743856+0000","last_became_active":"2024-09-20T18:55:28.835149+0000","last_became_peered":"2024-09-20T18:55:28.835149+0000","last_unstale":"2024-09-20T18:58:40.743856+0000","last_undegraded":"2024-09-20T18:58:40.743856+0000","last_fullsized":"2024-09-20T18:58:40.743856+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:44.501192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.501192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":415,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.743328+0000","last_change":"2024-09-20T18:55:13.577772+0000","last_active":"2024-09-20T18:58:40.743328+0000","last_peered":"2024-09-20T18:58:40.743328+0000","last_clean":"2024-09-20T18:58:40.743328+0000","last_became_active":"2024-09-20T18:49:49.957558+0000","last_became_peered":"2024-09-20T18:49:49.957558+0000","last_unstale":"2024-09-20T18:58:40.743328+0000","last_undegraded":"2024-09-20T18:58:40.743328+0000","last_fullsized":"2024-09-20T18:58:40.743328+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:13.577699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:13.577699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":410,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.742363+0000","last_change":"2024-09-20T18:53:47.370152+0000","last_active":"2024-09-20T18:58:40.742363+0000","last_peered":"2024-09-20T18:58:40.742363+0000","last_clean":"2024-09-20T18:58:40.742363+0000","last_became_active":"2024-09-20T18:49:51.970798+0000","last_became_peered":"2024-09-20T18:49:51.970798+0000","last_unstale":"2024-09-20T18:58:40.742363+0000","last_undegraded":"2024-09-20T18:58:40.742363+0000","last_fullsized":"2024-09-20T18:58:40.742363+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:47.370108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:47.370108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:38.509312+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.4","version":"251'224","reported_seq":245,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:41.405286+0000","last_change":"2024-09-20T18:58:33.846071+0000","last_active":"2024-09-20T18:58:41.405286+0000","last_peered":"2024-09-20T18:58:41.405286+0000","last_clean":"2024-09-20T18:58:41.405286+0000","last_became_active":"2024-09-20T18:58:33.845736+0000","last_became_peered":"2024-09-20T18:58:33.845736+0000","last_unstale":"2024-09-20T18:58:41.405286+0000","last_undegraded":"2024-09-20T18:58:41.405286+0000","last_fullsized":"2024-09-20T18:58:41.405286+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":224,"log_dups_size":0,"ondisk_log_size":224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:36.696830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14680064,"num_objects":224,"num_object_clones":0,"num_object_copies":672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":224,"num_write_kb":14336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":434,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.745548+0000","last_change":"2024-09-20T18:53:52.898818+0000","last_active":"2024-09-20T18:58:40.745548+0000","last_peered":"2024-09-20T18:58:40.745548+0000","last_clean":"2024-09-20T18:58:40.745548+0000","last_became_active":"2024-09-20T18:49:47.949116+0000","last_became_peered":"2024-09-20T18:49:47.949116+0000","last_unstale":"2024-09-20T18:58:40.745548+0000","last_undegraded":"2024-09-20T18:58:40.745548+0000","last_fullsized":"2024-09-20T18:58:40.745548+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:53:52.898760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:52.898760+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:42.918366+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":421,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.743458+0000","last_change":"2024-09-20T18:54:52.650798+0000","last_active":"2024-09-20T18:58:40.743458+0000","last_peered":"2024-09-20T18:58:40.743458+0000","last_clean":"2024-09-20T18:58:40.743458+0000","last_became_active":"2024-09-20T18:49:45.944858+0000","last_became_peered":"2024-09-20T18:49:45.944858+0000","last_unstale":"2024-09-20T18:58:40.743458+0000","last_undegraded":"2024-09-20T18:58:40.743458+0000","last_fullsized":"2024-09-20T18:58:40.743458+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:52.650756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:52.650756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":416,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.742289+0000","last_change":"2024-09-20T18:58:33.180357+0000","last_active":"2024-09-20T18:58:40.742289+0000","last_peered":"2024-09-20T18:58:40.742289+0000","last_clean":"2024-09-20T18:58:40.742289+0000","last_became_active":"2024-09-20T18:49:49.954944+0000","last_became_peered":"2024-09-20T18:49:49.954944+0000","last_unstale":"2024-09-20T18:58:40.742289+0000","last_undegraded":"2024-09-20T18:58:40.742289+0000","last_fullsized":"2024-09-20T18:58:40.742289+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:33.180309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:33.180309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:34.621967+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":408,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.750065+0000","last_change":"2024-09-20T18:58:24.162921+0000","last_active":"2024-09-20T18:58:40.750065+0000","last_peered":"2024-09-20T18:58:40.750065+0000","last_clean":"2024-09-20T18:58:40.750065+0000","last_became_active":"2024-09-20T18:49:51.972970+0000","last_became_peered":"2024-09-20T18:49:51.972970+0000","last_unstale":"2024-09-20T18:58:40.750065+0000","last_undegraded":"2024-09-20T18:58:40.750065+0000","last_fullsized":"2024-09-20T18:58:40.750065+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:24.162879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:24.162879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:46.703799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"251'380","reported_seq":401,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:42.115908+0000","last_change":"2024-09-20T18:58:33.821228+0000","last_active":"2024-09-20T18:58:42.115908+0000","last_peered":"2024-09-20T18:58:42.115908+0000","last_clean":"2024-09-20T18:58:42.115908+0000","last_became_active":"2024-09-20T18:58:33.819949+0000","last_became_peered":"2024-09-20T18:58:33.819949+0000","last_unstale":"2024-09-20T18:58:42.115908+0000","last_undegraded":"2024-09-20T18:58:42.115908+0000","last_fullsized":"2024-09-20T18:58:42.115908+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":380,"log_dups_size":0,"ondisk_log_size":380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:38.576354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24903680,"num_objects":380,"num_object_clones":0,"num_object_copies":1140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":380,"num_write_kb":24320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":465,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.738739+0000","last_change":"2024-09-20T18:58:30.522198+0000","last_active":"2024-09-20T18:58:40.738739+0000","last_peered":"2024-09-20T18:58:40.738739+0000","last_clean":"2024-09-20T18:58:40.738739+0000","last_became_active":"2024-09-20T18:55:28.824801+0000","last_became_peered":"2024-09-20T18:55:28.824801+0000","last_unstale":"2024-09-20T18:58:40.738739+0000","last_undegraded":"2024-09-20T18:58:40.738739+0000","last_fullsized":"2024-09-20T18:58:40.738739+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:58:30.522129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:30.522129+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:33.543607+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":441,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.743428+0000","last_change":"2024-09-20T18:55:24.729301+0000","last_active":"2024-09-20T18:58:40.743428+0000","last_peered":"2024-09-20T18:58:40.743428+0000","last_clean":"2024-09-20T18:58:40.743428+0000","last_became_active":"2024-09-20T18:49:45.944256+0000","last_became_peered":"2024-09-20T18:49:45.944256+0000","last_unstale":"2024-09-20T18:58:40.743428+0000","last_undegraded":"2024-09-20T18:58:40.743428+0000","last_fullsized":"2024-09-20T18:58:40.743428+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-20T18:55:24.729219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:24.729219+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"184'9","reported_seq":532,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.742891+0000","last_change":"2024-09-20T18:58:27.521112+0000","last_active":"2024-09-20T18:58:40.742891+0000","last_peered":"2024-09-20T18:58:40.742891+0000","last_clean":"2024-09-20T18:58:40.742891+0000","last_became_active":"2024-09-20T18:55:28.818085+0000","last_became_peered":"2024-09-20T18:55:28.818085+0000","last_unstale":"2024-09-20T18:58:40.742891+0000","last_undegraded":"2024-09-20T18:58:40.742891+0000","last_fullsized":"2024-09-20T18:58:40.742891+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'9","last_scrub_stamp":"2024-09-20T18:58:27.521051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:27.521051+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:55.700229+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":416,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.739005+0000","last_change":"2024-09-20T18:58:05.523446+0000","last_active":"2024-09-20T18:58:40.739005+0000","last_peered":"2024-09-20T18:58:40.739005+0000","last_clean":"2024-09-20T18:58:40.739005+0000","last_became_active":"2024-09-20T18:55:28.830380+0000","last_became_peered":"2024-09-20T18:55:28.830380+0000","last_unstale":"2024-09-20T18:58:40.739005+0000","last_undegraded":"2024-09-20T18:58:40.739005+0000","last_fullsized":"2024-09-20T18:58:40.739005+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:05.523359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:05.523359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:28.858918+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":152,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.751813+0000","last_change":"2024-09-20T18:55:28.835344+0000","last_active":"2024-09-20T18:58:40.751813+0000","last_peered":"2024-09-20T18:58:40.751813+0000","last_clean":"2024-09-20T18:58:40.751813+0000","last_became_active":"2024-09-20T18:55:28.835010+0000","last_became_peered":"2024-09-20T18:55:28.835010+0000","last_unstale":"2024-09-20T18:58:40.751813+0000","last_undegraded":"2024-09-20T18:58:40.751813+0000","last_fullsized":"2024-09-20T18:58:40.751813+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:26.721735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:26.721735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":425,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.753414+0000","last_change":"2024-09-20T18:58:09.063555+0000","last_active":"2024-09-20T18:58:40.753414+0000","last_peered":"2024-09-20T18:58:40.753414+0000","last_clean":"2024-09-20T18:58:40.753414+0000","last_became_active":"2024-09-20T18:49:48.500662+0000","last_became_peered":"2024-09-20T18:49:48.500662+0000","last_unstale":"2024-09-20T18:58:40.753414+0000","last_undegraded":"2024-09-20T18:58:40.753414+0000","last_fullsized":"2024-09-20T18:58:40.753414+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-20T18:58:09.063481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:09.063481+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:21.861420+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":412,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.739953+0000","last_change":"2024-09-20T18:58:28.460106+0000","last_active":"2024-09-20T18:58:40.739953+0000","last_peered":"2024-09-20T18:58:40.739953+0000","last_clean":"2024-09-20T18:58:40.739953+0000","last_became_active":"2024-09-20T18:49:49.956749+0000","last_became_peered":"2024-09-20T18:49:49.956749+0000","last_unstale":"2024-09-20T18:58:40.739953+0000","last_undegraded":"2024-09-20T18:58:40.739953+0000","last_fullsized":"2024-09-20T18:58:40.739953+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:28.460054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:28.460054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:47.283266+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":416,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.743023+0000","last_change":"2024-09-20T18:55:28.827897+0000","last_active":"2024-09-20T18:58:40.743023+0000","last_peered":"2024-09-20T18:58:40.743023+0000","last_clean":"2024-09-20T18:58:40.743023+0000","last_became_active":"2024-09-20T18:55:28.827418+0000","last_became_peered":"2024-09-20T18:55:28.827418+0000","last_unstale":"2024-09-20T18:58:40.743023+0000","last_undegraded":"2024-09-20T18:58:40.743023+0000","last_fullsized":"2024-09-20T18:58:40.743023+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.637739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.637739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":425,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.750583+0000","last_change":"2024-09-20T18:54:44.830872+0000","last_active":"2024-09-20T18:58:40.750583+0000","last_peered":"2024-09-20T18:58:40.750583+0000","last_clean":"2024-09-20T18:58:40.750583+0000","last_became_active":"2024-09-20T18:49:48.500803+0000","last_became_peered":"2024-09-20T18:49:48.500803+0000","last_unstale":"2024-09-20T18:58:40.750583+0000","last_undegraded":"2024-09-20T18:58:40.750583+0000","last_fullsized":"2024-09-20T18:58:40.750583+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:44.830790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.830790+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":420,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.743172+0000","last_change":"2024-09-20T18:55:29.751718+0000","last_active":"2024-09-20T18:58:40.743172+0000","last_peered":"2024-09-20T18:58:40.743172+0000","last_clean":"2024-09-20T18:58:40.743172+0000","last_became_active":"2024-09-20T18:55:28.833799+0000","last_became_peered":"2024-09-20T18:55:28.833799+0000","last_unstale":"2024-09-20T18:58:40.743172+0000","last_undegraded":"2024-09-20T18:58:40.743172+0000","last_fullsized":"2024-09-20T18:58:40.743172+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:29.751642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:29.751642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":412,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.751221+0000","last_change":"2024-09-20T18:57:46.898929+0000","last_active":"2024-09-20T18:58:40.751221+0000","last_peered":"2024-09-20T18:58:40.751221+0000","last_clean":"2024-09-20T18:58:40.751221+0000","last_became_active":"2024-09-20T18:49:49.961947+0000","last_became_peered":"2024-09-20T18:49:49.961947+0000","last_unstale":"2024-09-20T18:58:40.751221+0000","last_undegraded":"2024-09-20T18:58:40.751221+0000","last_fullsized":"2024-09-20T18:58:40.751221+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:46.898862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:46.898862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:06.640666+0000","scrub_duration":38,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"66'1","reported_seq":420,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.743826+0000","last_change":"2024-09-20T18:58:30.171382+0000","last_active":"2024-09-20T18:58:40.743826+0000","last_peered":"2024-09-20T18:58:40.743826+0000","last_clean":"2024-09-20T18:58:40.743826+0000","last_became_active":"2024-09-20T18:49:51.965121+0000","last_became_peered":"2024-09-20T18:49:51.965121+0000","last_unstale":"2024-09-20T18:58:40.743826+0000","last_undegraded":"2024-09-20T18:58:40.743826+0000","last_fullsized":"2024-09-20T18:58:40.743826+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T18:58:30.171329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:30.171329+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:33.520988+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":424,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.739878+0000","last_change":"2024-09-20T18:57:59.430353+0000","last_active":"2024-09-20T18:58:40.739878+0000","last_peered":"2024-09-20T18:58:40.739878+0000","last_clean":"2024-09-20T18:58:40.739878+0000","last_became_active":"2024-09-20T18:49:48.501011+0000","last_became_peered":"2024-09-20T18:49:48.501011+0000","last_unstale":"2024-09-20T18:58:40.739878+0000","last_undegraded":"2024-09-20T18:58:40.739878+0000","last_fullsized":"2024-09-20T18:58:40.739878+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:57:59.430294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:59.430294+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:21.333603+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":421,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.749865+0000","last_change":"2024-09-20T18:55:27.028184+0000","last_active":"2024-09-20T18:58:40.749865+0000","last_peered":"2024-09-20T18:58:40.749865+0000","last_clean":"2024-09-20T18:58:40.749865+0000","last_became_active":"2024-09-20T18:49:45.938511+0000","last_became_peered":"2024-09-20T18:49:45.938511+0000","last_unstale":"2024-09-20T18:58:40.749865+0000","last_undegraded":"2024-09-20T18:58:40.749865+0000","last_fullsized":"2024-09-20T18:58:40.749865+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:27.028126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:27.028126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":528,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.752672+0000","last_change":"2024-09-20T18:55:27.284156+0000","last_active":"2024-09-20T18:58:40.752672+0000","last_peered":"2024-09-20T18:58:40.752672+0000","last_clean":"2024-09-20T18:58:40.752672+0000","last_became_active":"2024-09-20T18:49:49.960550+0000","last_became_peered":"2024-09-20T18:49:49.960550+0000","last_unstale":"2024-09-20T18:58:40.752672+0000","last_undegraded":"2024-09-20T18:58:40.752672+0000","last_fullsized":"2024-09-20T18:58:40.752672+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:27.284086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:27.284086+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":410,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.746694+0000","last_change":"2024-09-20T18:55:12.348079+0000","last_active":"2024-09-20T18:58:40.746694+0000","last_peered":"2024-09-20T18:58:40.746694+0000","last_clean":"2024-09-20T18:58:40.746694+0000","last_became_active":"2024-09-20T18:49:51.966254+0000","last_became_peered":"2024-09-20T18:49:51.966254+0000","last_unstale":"2024-09-20T18:58:40.746694+0000","last_undegraded":"2024-09-20T18:58:40.746694+0000","last_fullsized":"2024-09-20T18:58:40.746694+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:12.347993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:12.347993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":442,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.752010+0000","last_change":"2024-09-20T18:57:20.701592+0000","last_active":"2024-09-20T18:58:40.752010+0000","last_peered":"2024-09-20T18:58:40.752010+0000","last_clean":"2024-09-20T18:58:40.752010+0000","last_became_active":"2024-09-20T18:49:58.997494+0000","last_became_peered":"2024-09-20T18:49:58.997494+0000","last_unstale":"2024-09-20T18:58:40.752010+0000","last_undegraded":"2024-09-20T18:58:40.752010+0000","last_fullsized":"2024-09-20T18:58:40.752010+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:57:20.701543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:20.701543+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:44.352173+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":421,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.743178+0000","last_change":"2024-09-20T18:54:57.632458+0000","last_active":"2024-09-20T18:58:40.743178+0000","last_peered":"2024-09-20T18:58:40.743178+0000","last_clean":"2024-09-20T18:58:40.743178+0000","last_became_active":"2024-09-20T18:49:45.949492+0000","last_became_peered":"2024-09-20T18:49:45.949492+0000","last_unstale":"2024-09-20T18:58:40.743178+0000","last_undegraded":"2024-09-20T18:58:40.743178+0000","last_fullsized":"2024-09-20T18:58:40.743178+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.632380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.632380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":527,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.739740+0000","last_change":"2024-09-20T18:55:28.302353+0000","last_active":"2024-09-20T18:58:40.739740+0000","last_peered":"2024-09-20T18:58:40.739740+0000","last_clean":"2024-09-20T18:58:40.739740+0000","last_became_active":"2024-09-20T18:49:49.962243+0000","last_became_peered":"2024-09-20T18:49:49.962243+0000","last_unstale":"2024-09-20T18:58:40.739740+0000","last_undegraded":"2024-09-20T18:58:40.739740+0000","last_fullsized":"2024-09-20T18:58:40.739740+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:28.302303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:28.302303+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:17.501674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":409,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.750002+0000","last_change":"2024-09-20T18:55:11.918284+0000","last_active":"2024-09-20T18:58:40.750002+0000","last_peered":"2024-09-20T18:58:40.750002+0000","last_clean":"2024-09-20T18:58:40.750002+0000","last_became_active":"2024-09-20T18:49:51.969093+0000","last_became_peered":"2024-09-20T18:49:51.969093+0000","last_unstale":"2024-09-20T18:58:40.750002+0000","last_undegraded":"2024-09-20T18:58:40.750002+0000","last_fullsized":"2024-09-20T18:58:40.750002+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:11.918210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:11.918210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":456,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.745801+0000","last_change":"2024-09-20T18:57:50.899576+0000","last_active":"2024-09-20T18:58:40.745801+0000","last_peered":"2024-09-20T18:58:40.745801+0000","last_clean":"2024-09-20T18:58:40.745801+0000","last_became_active":"2024-09-20T18:55:28.826721+0000","last_became_peered":"2024-09-20T18:55:28.826721+0000","last_unstale":"2024-09-20T18:58:40.745801+0000","last_undegraded":"2024-09-20T18:58:40.745801+0000","last_fullsized":"2024-09-20T18:58:40.745801+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:57:50.899514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:50.899514+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:12.210010+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":413,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.746336+0000","last_change":"2024-09-20T18:55:28.822690+0000","last_active":"2024-09-20T18:58:40.746336+0000","last_peered":"2024-09-20T18:58:40.746336+0000","last_clean":"2024-09-20T18:58:40.746336+0000","last_became_active":"2024-09-20T18:55:28.822258+0000","last_became_peered":"2024-09-20T18:55:28.822258+0000","last_unstale":"2024-09-20T18:58:40.746336+0000","last_undegraded":"2024-09-20T18:58:40.746336+0000","last_fullsized":"2024-09-20T18:58:40.746336+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:05.985551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:52:05.985551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":404,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.744682+0000","last_change":"2024-09-20T18:57:11.986605+0000","last_active":"2024-09-20T18:58:40.744682+0000","last_peered":"2024-09-20T18:58:40.744682+0000","last_clean":"2024-09-20T18:58:40.744682+0000","last_became_active":"2024-09-20T18:49:49.957067+0000","last_became_peered":"2024-09-20T18:49:49.957067+0000","last_unstale":"2024-09-20T18:58:40.744682+0000","last_undegraded":"2024-09-20T18:58:40.744682+0000","last_fullsized":"2024-09-20T18:58:40.744682+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:11.986528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:11.986528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":400,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.746519+0000","last_change":"2024-09-20T18:57:16.789412+0000","last_active":"2024-09-20T18:58:40.746519+0000","last_peered":"2024-09-20T18:58:40.746519+0000","last_clean":"2024-09-20T18:58:40.746519+0000","last_became_active":"2024-09-20T18:49:51.966915+0000","last_became_peered":"2024-09-20T18:49:51.966915+0000","last_unstale":"2024-09-20T18:58:40.746519+0000","last_undegraded":"2024-09-20T18:58:40.746519+0000","last_fullsized":"2024-09-20T18:58:40.746519+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:16.789095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:16.789095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:45.821909+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":457,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.744900+0000","last_change":"2024-09-20T18:58:28.035512+0000","last_active":"2024-09-20T18:58:40.744900+0000","last_peered":"2024-09-20T18:58:40.744900+0000","last_clean":"2024-09-20T18:58:40.744900+0000","last_became_active":"2024-09-20T18:55:28.827023+0000","last_became_peered":"2024-09-20T18:55:28.827023+0000","last_unstale":"2024-09-20T18:58:40.744900+0000","last_undegraded":"2024-09-20T18:58:40.744900+0000","last_fullsized":"2024-09-20T18:58:40.744900+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:58:28.035450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:28.035450+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:56.559064+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":413,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.743628+0000","last_change":"2024-09-20T18:54:42.487465+0000","last_active":"2024-09-20T18:58:40.743628+0000","last_peered":"2024-09-20T18:58:40.743628+0000","last_clean":"2024-09-20T18:58:40.743628+0000","last_became_active":"2024-09-20T18:49:45.944357+0000","last_became_peered":"2024-09-20T18:49:45.944357+0000","last_unstale":"2024-09-20T18:58:40.743628+0000","last_undegraded":"2024-09-20T18:58:40.743628+0000","last_fullsized":"2024-09-20T18:58:40.743628+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:42.487403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:42.487403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":412,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.750144+0000","last_change":"2024-09-20T18:57:57.915090+0000","last_active":"2024-09-20T18:58:40.750144+0000","last_peered":"2024-09-20T18:58:40.750144+0000","last_clean":"2024-09-20T18:58:40.750144+0000","last_became_active":"2024-09-20T18:49:49.962523+0000","last_became_peered":"2024-09-20T18:49:49.962523+0000","last_unstale":"2024-09-20T18:58:40.750144+0000","last_undegraded":"2024-09-20T18:58:40.750144+0000","last_fullsized":"2024-09-20T18:58:40.750144+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:57.914913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:57.914913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:01.797575+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":409,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.751337+0000","last_change":"2024-09-20T18:54:26.882441+0000","last_active":"2024-09-20T18:58:40.751337+0000","last_peered":"2024-09-20T18:58:40.751337+0000","last_clean":"2024-09-20T18:58:40.751337+0000","last_became_active":"2024-09-20T18:49:51.967313+0000","last_became_peered":"2024-09-20T18:49:51.967313+0000","last_unstale":"2024-09-20T18:58:40.751337+0000","last_undegraded":"2024-09-20T18:58:40.751337+0000","last_fullsized":"2024-09-20T18:58:40.751337+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:26.882375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:26.882375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:40.010268+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":454,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.747125+0000","last_change":"2024-09-20T18:57:54.862785+0000","last_active":"2024-09-20T18:58:40.747125+0000","last_peered":"2024-09-20T18:58:40.747125+0000","last_clean":"2024-09-20T18:58:40.747125+0000","last_became_active":"2024-09-20T18:55:28.830801+0000","last_became_peered":"2024-09-20T18:55:28.830801+0000","last_unstale":"2024-09-20T18:58:40.747125+0000","last_undegraded":"2024-09-20T18:58:40.747125+0000","last_fullsized":"2024-09-20T18:58:40.747125+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:57:54.862329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:54.862329+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:09.269532+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":428,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.739958+0000","last_change":"2024-09-20T18:58:06.479419+0000","last_active":"2024-09-20T18:58:40.739958+0000","last_peered":"2024-09-20T18:58:40.739958+0000","last_clean":"2024-09-20T18:58:40.739958+0000","last_became_active":"2024-09-20T18:49:45.942105+0000","last_became_peered":"2024-09-20T18:49:45.942105+0000","last_unstale":"2024-09-20T18:58:40.739958+0000","last_undegraded":"2024-09-20T18:58:40.739958+0000","last_fullsized":"2024-09-20T18:58:40.739958+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:06.479364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:06.479364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:26.337228+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":420,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.742188+0000","last_change":"2024-09-20T18:55:28.825004+0000","last_active":"2024-09-20T18:58:40.742188+0000","last_peered":"2024-09-20T18:58:40.742188+0000","last_clean":"2024-09-20T18:58:40.742188+0000","last_became_active":"2024-09-20T18:55:28.824355+0000","last_became_peered":"2024-09-20T18:55:28.824355+0000","last_unstale":"2024-09-20T18:58:40.742188+0000","last_undegraded":"2024-09-20T18:58:40.742188+0000","last_fullsized":"2024-09-20T18:58:40.742188+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:46.329547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:46.329547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:39.533283+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":407,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.750272+0000","last_change":"2024-09-20T18:58:28.165763+0000","last_active":"2024-09-20T18:58:40.750272+0000","last_peered":"2024-09-20T18:58:40.750272+0000","last_clean":"2024-09-20T18:58:40.750272+0000","last_became_active":"2024-09-20T18:55:28.825296+0000","last_became_peered":"2024-09-20T18:55:28.825296+0000","last_unstale":"2024-09-20T18:58:40.750272+0000","last_undegraded":"2024-09-20T18:58:40.750272+0000","last_fullsized":"2024-09-20T18:58:40.750272+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:28.165712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:28.165712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:43.081975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":435,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.746579+0000","last_change":"2024-09-20T18:55:28.836789+0000","last_active":"2024-09-20T18:58:40.746579+0000","last_peered":"2024-09-20T18:58:40.746579+0000","last_clean":"2024-09-20T18:58:40.746579+0000","last_became_active":"2024-09-20T18:55:28.824579+0000","last_became_peered":"2024-09-20T18:55:28.824579+0000","last_unstale":"2024-09-20T18:58:40.746579+0000","last_undegraded":"2024-09-20T18:58:40.746579+0000","last_fullsized":"2024-09-20T18:58:40.746579+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-20T18:55:20.282193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:20.282193+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:39.120828+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":421,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.751099+0000","last_change":"2024-09-20T18:54:55.900487+0000","last_active":"2024-09-20T18:58:40.751099+0000","last_peered":"2024-09-20T18:58:40.751099+0000","last_clean":"2024-09-20T18:58:40.751099+0000","last_became_active":"2024-09-20T18:49:45.940988+0000","last_became_peered":"2024-09-20T18:49:45.940988+0000","last_unstale":"2024-09-20T18:58:40.751099+0000","last_undegraded":"2024-09-20T18:58:40.751099+0000","last_fullsized":"2024-09-20T18:58:40.751099+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:55.899611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:55.899611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":160,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.745290+0000","last_change":"2024-09-20T18:57:52.929891+0000","last_active":"2024-09-20T18:58:40.745290+0000","last_peered":"2024-09-20T18:58:40.745290+0000","last_clean":"2024-09-20T18:58:40.745290+0000","last_became_active":"2024-09-20T18:55:28.832534+0000","last_became_peered":"2024-09-20T18:55:28.832534+0000","last_unstale":"2024-09-20T18:58:40.745290+0000","last_undegraded":"2024-09-20T18:58:40.745290+0000","last_fullsized":"2024-09-20T18:58:40.745290+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:52.929828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:52.929828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:13.316360+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":401,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.742828+0000","last_change":"2024-09-20T18:58:22.558181+0000","last_active":"2024-09-20T18:58:40.742828+0000","last_peered":"2024-09-20T18:58:40.742828+0000","last_clean":"2024-09-20T18:58:40.742828+0000","last_became_active":"2024-09-20T18:49:51.968662+0000","last_became_peered":"2024-09-20T18:49:51.968662+0000","last_unstale":"2024-09-20T18:58:40.742828+0000","last_undegraded":"2024-09-20T18:58:40.742828+0000","last_fullsized":"2024-09-20T18:58:40.742828+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:22.558131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:22.558131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:46.688905+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":392,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.745024+0000","last_change":"2024-09-20T18:57:10.970843+0000","last_active":"2024-09-20T18:58:40.745024+0000","last_peered":"2024-09-20T18:58:40.745024+0000","last_clean":"2024-09-20T18:58:40.745024+0000","last_became_active":"2024-09-20T18:49:51.968216+0000","last_became_peered":"2024-09-20T18:49:51.968216+0000","last_unstale":"2024-09-20T18:58:40.745024+0000","last_undegraded":"2024-09-20T18:58:40.745024+0000","last_fullsized":"2024-09-20T18:58:40.745024+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:10.970779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:10.970779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":420,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.746885+0000","last_change":"2024-09-20T18:58:36.077005+0000","last_active":"2024-09-20T18:58:40.746885+0000","last_peered":"2024-09-20T18:58:40.746885+0000","last_clean":"2024-09-20T18:58:40.746885+0000","last_became_active":"2024-09-20T18:49:45.937992+0000","last_became_peered":"2024-09-20T18:49:45.937992+0000","last_unstale":"2024-09-20T18:58:40.746885+0000","last_undegraded":"2024-09-20T18:58:40.746885+0000","last_fullsized":"2024-09-20T18:58:40.746885+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:36.076937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:36.076937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:38.647424+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":202,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.740353+0000","last_change":"2024-09-20T18:55:28.852468+0000","last_active":"2024-09-20T18:58:40.740353+0000","last_peered":"2024-09-20T18:58:40.740353+0000","last_clean":"2024-09-20T18:58:40.740353+0000","last_became_active":"2024-09-20T18:55:28.834675+0000","last_became_peered":"2024-09-20T18:55:28.834675+0000","last_unstale":"2024-09-20T18:58:40.740353+0000","last_undegraded":"2024-09-20T18:58:40.740353+0000","last_fullsized":"2024-09-20T18:58:40.740353+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:54:44.719676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.719676+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":12,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":406,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-20T18:58:40.750205+0000","last_change":"2024-09-20T18:58:27.904636+0000","last_active":"2024-09-20T18:58:40.750205+0000","last_peered":"2024-09-20T18:58:40.750205+0000","last_clean":"2024-09-20T18:58:40.750205+0000","last_became_active":"2024-09-20T18:49:49.957582+0000","last_became_peered":"2024-09-20T18:49:49.957582+0000","last_unstale":"2024-09-20T18:58:40.750205+0000","last_undegraded":"2024-09-20T18:58:40.750205+0000","last_fullsized":"2024-09-20T18:58:40.750205+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:27.904592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:27.904592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:35.767269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":232194048,"num_objects":3543,"num_object_clones":0,"num_object_copies":10629,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3543,"num_write_kb":226752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":638451712,"data_stored":638451712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3543,"ondisk_log_size":3543,"up":48,"acting":48,"num_store_stats":7},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":67,"ondisk_log_size":67,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":695,"num_read_kb":452,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":1474,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":12140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":394,"ondisk_log_size":394,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":5275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":529,"num_read_kb":524,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266799,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":300072,"kb_used_data":30104,"kb_used_omap":1,"kb_used_meta":269950,"kb_avail":93445080,"statfs":{"total":95995035648,"available":95687761920,"internally_reserved":0,"allocated":30826496,"data_stored":30256325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":276429258},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":0,"seq":193273528438,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757371,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":276384,"kb_used_data":14736,"kb_used_omap":1,"kb_used_meta":261630,"kb_avail":93468768,"statfs":{"total":95995035648,"available":95712018432,"internally_reserved":0,"allocated":15089664,"data_stored":14527168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":267909573},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986304,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":235512,"kb_used_data":23848,"kb_used_omap":1,"kb_used_meta":211646,"kb_avail":93509640,"statfs":{"total":95995035648,"available":95753871360,"internally_reserved":0,"allocated":24420352,"data_stored":23834312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":216725957},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182535,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":451976,"kb_used_data":91764,"kb_used_omap":1,"kb_used_meta":360190,"kb_avail":93293176,"statfs":{"total":95995035648,"available":95532212224,"internally_reserved":0,"allocated":93966336,"data_stored":93373786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":368835015},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411468,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":449764,"kb_used_data":49300,"kb_used_omap":1,"kb_used_meta":400446,"kb_avail":93295388,"statfs":{"total":95995035648,"available":95534477312,"internally_reserved":0,"allocated":50483200,"data_stored":49923681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":410057157},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574994,"num_pgs":73,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":397660,"kb_used_data":91596,"kb_used_omap":1,"kb_used_meta":306046,"kb_avail":93347492,"statfs":{"total":95995035648,"available":95587831808,"internally_reserved":0,"allocated":93794304,"data_stored":93179605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":313391557},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738520,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":475140,"kb_used_data":110772,"kb_used_omap":1,"kb_used_meta":364350,"kb_avail":93270012,"statfs":{"total":95995035648,"available":95508492288,"internally_reserved":0,"allocated":113430528,"data_stored":112838598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":373094853},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":123338752,"data_stored":123338752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":134742016,"data_stored":134742016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53936128,"data_stored":53936128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":141426688,"data_stored":141426688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":34734080,"data_stored":34734080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":78184448,"data_stored":78184448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":72089600,"data_stored":72089600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T18:58:45.270 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd pool set .mgr pgp_num 1 2024-09-20T18:58:45.272 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:45.302 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:45.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:45.305 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:45.406 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:45.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:45.452 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:45.452 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:45.553 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:45.584 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:45.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:45.585 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:45.686 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:45.720 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:45.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:45.721 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:45.821 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:45.858 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:45.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:45.859 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:45.938 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_historic_ops 2024-09-20T18:58:45.961 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:46.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:46.025 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:46.025 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:46.126 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:46.156 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:46.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:46.157 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:46.247 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 6 16 17455 17439 181.625 166.375 0.00733716 0.00549786 2024-09-20T18:58:46.259 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:46.288 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:46.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:46.289 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:46.390 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:46.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:46.431 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:46.431 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:46.532 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:58:46.591 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:58:46.604 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:58:46.607 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:58:46.609 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:46.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:46.610 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:58:46.711 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:46.742 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:46.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:46.743 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:46.844 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:46.889 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:46.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:46.890 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:46.985 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:46 smithi083 ceph-mon[29421]: pgmap v436: 148 pgs: 148 active+clean; 222 MiB data, 2.5 GiB used, 623 GiB / 626 GiB avail; 672 B/s rd, 29 MiB/s wr, 466 op/s 2024-09-20T18:58:46.985 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:46 smithi083 ceph-mon[29421]: osdmap e253: 8 total, 8 up, 7 in 2024-09-20T18:58:46.985 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:46 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:58:46.985 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:46 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:58:46.985 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:46 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:58:46.985 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:46 smithi083 ceph-mon[36260]: pgmap v436: 148 pgs: 148 active+clean; 222 MiB data, 2.5 GiB used, 623 GiB / 626 GiB avail; 672 B/s rd, 29 MiB/s wr, 466 op/s 2024-09-20T18:58:46.986 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:46 smithi083 ceph-mon[36260]: osdmap e253: 8 total, 8 up, 7 in 2024-09-20T18:58:46.986 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:46 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:58:46.986 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:46 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:58:46.986 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:46 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:58:46.991 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:47.017 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:58:46 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:58:47.017 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:46 smithi142 ceph-mon[32508]: pgmap v436: 148 pgs: 148 active+clean; 222 MiB data, 2.5 GiB used, 623 GiB / 626 GiB avail; 672 B/s rd, 29 MiB/s wr, 466 op/s 2024-09-20T18:58:47.017 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:46 smithi142 ceph-mon[32508]: osdmap e253: 8 total, 8 up, 7 in 2024-09-20T18:58:47.017 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:46 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:58:47.018 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:46 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:58:47.018 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:46 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:58:47.021 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:47.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:47.022 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:47.122 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:47.151 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:47.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:47.152 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:47.247 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 7 16 19603 19587 174.856 134.25 0.00480924 0.00571192 2024-09-20T18:58:47.253 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:47.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:47.284 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:47.285 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:47.385 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:47.427 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:47.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:47.428 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:47.529 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:47.553 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:47.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:47.554 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:47.654 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:47.692 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:47.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:47.693 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:47.794 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:47.822 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:47.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:47.823 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:47.924 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:47.953 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:47.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:47.954 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:48.017 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:47 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:58:48.017 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:47 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:58:48.017 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:47 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:58:48.017 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:47 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:58:48.018 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:47 smithi142 ceph-mon[32508]: pgmap v438: 148 pgs: 148 active+clean; 951 MiB data, 4.9 GiB used, 621 GiB / 626 GiB avail; 1.6 KiB/s rd, 152 MiB/s wr, 2.44k op/s 2024-09-20T18:58:48.047 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:47 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:58:48.047 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:47 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:58:48.048 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:47 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:58:48.048 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:47 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:58:48.048 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:47 smithi083 ceph-mon[29421]: pgmap v438: 148 pgs: 148 active+clean; 951 MiB data, 4.9 GiB used, 621 GiB / 626 GiB avail; 1.6 KiB/s rd, 152 MiB/s wr, 2.44k op/s 2024-09-20T18:58:48.048 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:47 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:58:48.048 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:47 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:58:48.048 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:47 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:58:48.048 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:47 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:58:48.048 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:47 smithi083 ceph-mon[36260]: pgmap v438: 148 pgs: 148 active+clean; 951 MiB data, 4.9 GiB used, 621 GiB / 626 GiB avail; 1.6 KiB/s rd, 152 MiB/s wr, 2.44k op/s 2024-09-20T18:58:48.056 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:48.087 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:48.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:48.088 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:48.189 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:48.231 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:48.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:48.232 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:48.247 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 8 16 21712 21696 169.475 131.812 0.00995787 0.00589467 2024-09-20T18:58:48.259 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T18:58:48.259 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-20T18:58:48.260 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-20T18:58:48.260 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-20T18:58:48.260 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.260 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:128076 8.3 8:ca202ecf:::benchmark_data_smithi083_97422_object11985:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e203)", 2024-09-20T18:58:48.260 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:24.802138+0000", 2024-09-20T18:58:48.260 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 143.442807219, 2024-09-20T18:58:48.260 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.90051086899999999, 2024-09-20T18:58:48.260 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:58:48.260 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:58:48.260 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:48.260 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:58:48.260 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:58:48.260 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:58:48.261 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 128076 2024-09-20T18:58:48.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.261 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:58:48.261 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.261 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:58:48.261 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.802138+0000", 2024-09-20T18:58:48.261 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:48.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.261 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.261 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:58:48.261 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.802138+0000", 2024-09-20T18:58:48.261 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:48.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.261 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.261 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:58:48.262 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.802139+0000", 2024-09-20T18:58:48.262 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8199999999999999e-07 2024-09-20T18:58:48.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.262 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.262 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:58:48.262 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.802143+0000", 2024-09-20T18:58:48.262 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0500000000000002e-06 2024-09-20T18:58:48.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.262 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.262 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:58:48.262 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.802151+0000", 2024-09-20T18:58:48.262 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8860000000000004e-06 2024-09-20T18:58:48.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.262 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.263 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:58:48.263 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.802157+0000", 2024-09-20T18:58:48.263 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.105e-06 2024-09-20T18:58:48.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.263 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.263 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:48.263 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.802209+0000", 2024-09-20T18:58:48.263 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1229e-05 2024-09-20T18:58:48.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.263 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.263 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:58:48.263 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.802215+0000", 2024-09-20T18:58:48.263 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8700000000000003e-06 2024-09-20T18:58:48.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.264 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:48.264 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.694932+0000", 2024-09-20T18:58:48.264 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.89271625700000001 2024-09-20T18:58:48.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.264 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:58:48.264 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.695043+0000", 2024-09-20T18:58:48.264 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000111789 2024-09-20T18:58:48.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.264 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,5", 2024-09-20T18:58:48.264 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.695170+0000", 2024-09-20T18:58:48.264 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000126255 2024-09-20T18:58:48.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.265 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:48.265 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.700200+0000", 2024-09-20T18:58:48.265 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0050300939999999997 2024-09-20T18:58:48.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.265 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:48.265 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.700261+0000", 2024-09-20T18:58:48.265 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1340000000000006e-05 2024-09-20T18:58:48.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.265 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:58:48.265 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.702559+0000", 2024-09-20T18:58:48.266 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002297707 2024-09-20T18:58:48.266 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.266 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.266 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:58:48.266 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.702606+0000", 2024-09-20T18:58:48.266 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7037999999999999e-05 2024-09-20T18:58:48.266 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.266 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.266 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:58:48.266 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.702649+0000", 2024-09-20T18:58:48.266 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3467000000000002e-05 2024-09-20T18:58:48.266 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:48.266 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:58:48.266 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:48.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.267 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.267 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:128090 8.3 8:c10076ee:::benchmark_data_smithi083_97422_object11999:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e203)", 2024-09-20T18:58:48.267 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:24.815492+0000", 2024-09-20T18:58:48.267 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 143.42945397099999, 2024-09-20T18:58:48.267 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88975575799999995, 2024-09-20T18:58:48.267 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:58:48.267 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:58:48.267 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:48.267 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:58:48.267 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:58:48.267 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:58:48.267 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 128090 2024-09-20T18:58:48.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.268 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:58:48.268 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.268 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:58:48.268 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.815492+0000", 2024-09-20T18:58:48.268 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:48.268 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.268 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.268 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:58:48.268 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.815492+0000", 2024-09-20T18:58:48.268 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:48.268 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.268 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.268 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:58:48.268 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.815493+0000", 2024-09-20T18:58:48.268 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2920000000000001e-06 2024-09-20T18:58:48.269 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.269 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.269 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:58:48.269 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.815497+0000", 2024-09-20T18:58:48.269 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8349999999999997e-06 2024-09-20T18:58:48.269 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.269 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.269 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:58:48.269 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.815505+0000", 2024-09-20T18:58:48.269 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2830000000000001e-06 2024-09-20T18:58:48.269 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.269 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.269 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:58:48.269 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.815512+0000", 2024-09-20T18:58:48.270 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8499999999999996e-06 2024-09-20T18:58:48.270 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.270 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.270 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:48.270 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.815561+0000", 2024-09-20T18:58:48.270 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8566000000000002e-05 2024-09-20T18:58:48.270 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.270 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.270 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:58:48.270 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.815568+0000", 2024-09-20T18:58:48.270 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.746e-06 2024-09-20T18:58:48.270 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.270 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.270 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:48.271 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.697779+0000", 2024-09-20T18:58:48.271 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88221116799999999 2024-09-20T18:58:48.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.271 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.271 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:58:48.271 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.697904+0000", 2024-09-20T18:58:48.271 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012453900000000001 2024-09-20T18:58:48.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.271 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.271 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,5", 2024-09-20T18:58:48.271 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.698026+0000", 2024-09-20T18:58:48.271 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121783 2024-09-20T18:58:48.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.271 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.271 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:48.272 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.705094+0000", 2024-09-20T18:58:48.272 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.007068146 2024-09-20T18:58:48.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.272 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.272 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:48.272 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.705140+0000", 2024-09-20T18:58:48.272 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6332000000000003e-05 2024-09-20T18:58:48.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.272 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.272 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:58:48.272 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.705154+0000", 2024-09-20T18:58:48.272 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3516000000000001e-05 2024-09-20T18:58:48.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.272 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.272 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:58:48.273 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.705203+0000", 2024-09-20T18:58:48.273 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.918e-05 2024-09-20T18:58:48.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.273 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.273 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:58:48.273 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.705247+0000", 2024-09-20T18:58:48.273 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4521999999999998e-05 2024-09-20T18:58:48.273 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:48.273 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:58:48.273 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:48.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.273 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.273 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:128136 8.3 8:c6a9dcdc:::benchmark_data_smithi083_97422_object12045:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e203)", 2024-09-20T18:58:48.273 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:24.875803+0000", 2024-09-20T18:58:48.274 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 143.369142372, 2024-09-20T18:58:48.274 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.82965221499999997, 2024-09-20T18:58:48.274 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:58:48.274 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:58:48.274 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:48.274 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:58:48.274 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:58:48.274 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:58:48.274 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 128136 2024-09-20T18:58:48.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.274 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:58:48.274 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.274 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:58:48.274 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.875803+0000", 2024-09-20T18:58:48.275 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:48.275 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.275 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.275 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:58:48.275 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.875803+0000", 2024-09-20T18:58:48.275 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:48.275 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.275 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.275 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:58:48.275 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.875804+0000", 2024-09-20T18:58:48.275 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.51e-07 2024-09-20T18:58:48.275 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.275 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.276 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:58:48.276 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.875806+0000", 2024-09-20T18:58:48.276 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6000000000000001e-06 2024-09-20T18:58:48.276 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.276 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.276 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:58:48.276 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.875812+0000", 2024-09-20T18:58:48.276 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3290000000000001e-06 2024-09-20T18:58:48.276 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.276 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.276 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:58:48.276 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.875817+0000", 2024-09-20T18:58:48.276 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0969999999999996e-06 2024-09-20T18:58:48.276 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.276 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.277 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:48.277 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.875977+0000", 2024-09-20T18:58:48.277 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015977400000000001 2024-09-20T18:58:48.277 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.277 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.277 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:58:48.277 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.875985+0000", 2024-09-20T18:58:48.277 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9019999999999999e-06 2024-09-20T18:58:48.277 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.277 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.277 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:48.277 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.700308+0000", 2024-09-20T18:58:48.277 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.82432304000000001 2024-09-20T18:58:48.277 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.278 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.278 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:58:48.278 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.700431+0000", 2024-09-20T18:58:48.278 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012312099999999999 2024-09-20T18:58:48.278 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.278 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.278 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,5", 2024-09-20T18:58:48.278 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.700535+0000", 2024-09-20T18:58:48.278 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010469 2024-09-20T18:58:48.278 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.278 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.278 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:58:48.278 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.705256+0000", 2024-09-20T18:58:48.278 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0047210610000000004 2024-09-20T18:58:48.279 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.279 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.279 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:48.279 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.705309+0000", 2024-09-20T18:58:48.279 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2534000000000002e-05 2024-09-20T18:58:48.279 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.279 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.279 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:48.279 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.705365+0000", 2024-09-20T18:58:48.279 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5844999999999998e-05 2024-09-20T18:58:48.279 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.279 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.279 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:58:48.279 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.705415+0000", 2024-09-20T18:58:48.279 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0297000000000003e-05 2024-09-20T18:58:48.280 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.280 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.280 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:58:48.280 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.705456+0000", 2024-09-20T18:58:48.280 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0373999999999997e-05 2024-09-20T18:58:48.280 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:48.280 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:58:48.280 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:48.280 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.280 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.280 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:128165 8.3 8:ccd2385f:::benchmark_data_smithi083_97422_object12074:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e203)", 2024-09-20T18:58:48.280 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:24.912336+0000", 2024-09-20T18:58:48.280 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 143.33261017800001, 2024-09-20T18:58:48.280 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.79332345500000001, 2024-09-20T18:58:48.281 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:58:48.281 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:58:48.281 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:48.281 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:58:48.281 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:58:48.281 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:58:48.281 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 128165 2024-09-20T18:58:48.281 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.281 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:58:48.281 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.281 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:58:48.281 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.912336+0000", 2024-09-20T18:58:48.281 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:48.281 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.281 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.282 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:58:48.282 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.912336+0000", 2024-09-20T18:58:48.282 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:48.282 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.282 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.282 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:58:48.282 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.912337+0000", 2024-09-20T18:58:48.282 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.466e-06 2024-09-20T18:58:48.282 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.282 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.282 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:58:48.282 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.912342+0000", 2024-09-20T18:58:48.282 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4330000000000003e-06 2024-09-20T18:58:48.282 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.283 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.283 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:58:48.283 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.912354+0000", 2024-09-20T18:58:48.283 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1620999999999999e-05 2024-09-20T18:58:48.283 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.283 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.283 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:58:48.283 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.912362+0000", 2024-09-20T18:58:48.283 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.6320000000000008e-06 2024-09-20T18:58:48.283 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.283 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.283 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:48.283 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.912419+0000", 2024-09-20T18:58:48.283 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7051e-05 2024-09-20T18:58:48.283 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.284 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.284 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:58:48.284 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.912428+0000", 2024-09-20T18:58:48.284 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3749999999999992e-06 2024-09-20T18:58:48.284 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.284 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.284 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:48.284 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.702684+0000", 2024-09-20T18:58:48.284 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.79025554899999995 2024-09-20T18:58:48.284 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.284 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.284 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:58:48.284 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.702784+0000", 2024-09-20T18:58:48.284 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000100629 2024-09-20T18:58:48.285 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.285 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.285 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,5", 2024-09-20T18:58:48.285 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.702908+0000", 2024-09-20T18:58:48.285 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012386 2024-09-20T18:58:48.285 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.285 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.285 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:58:48.285 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.705262+0000", 2024-09-20T18:58:48.285 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0023542070000000001 2024-09-20T18:58:48.285 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.285 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.285 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:48.285 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.705502+0000", 2024-09-20T18:58:48.286 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000239693 2024-09-20T18:58:48.286 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.286 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.286 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:48.286 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.705546+0000", 2024-09-20T18:58:48.286 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4231999999999999e-05 2024-09-20T18:58:48.286 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.286 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.286 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:58:48.287 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.705594+0000", 2024-09-20T18:58:48.287 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7484999999999997e-05 2024-09-20T18:58:48.287 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.287 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.287 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:58:48.287 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.705659+0000", 2024-09-20T18:58:48.287 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5222000000000007e-05 2024-09-20T18:58:48.287 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:48.287 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:58:48.287 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:48.288 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.288 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.288 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:128167 8.3 8:c28a9a26:::benchmark_data_smithi083_97422_object12076:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e203)", 2024-09-20T18:58:48.288 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:24.912455+0000", 2024-09-20T18:58:48.288 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 143.33249105600001, 2024-09-20T18:58:48.288 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.80018460800000002, 2024-09-20T18:58:48.288 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:58:48.288 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:58:48.288 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:48.288 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:58:48.288 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:58:48.288 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:58:48.288 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 128167 2024-09-20T18:58:48.289 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.289 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:58:48.289 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.289 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:58:48.289 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.912455+0000", 2024-09-20T18:58:48.289 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:48.289 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.289 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.289 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:58:48.289 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.912455+0000", 2024-09-20T18:58:48.289 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:48.289 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.289 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.289 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:58:48.290 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.912456+0000", 2024-09-20T18:58:48.290 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6500000000000008e-07 2024-09-20T18:58:48.290 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.290 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.290 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:58:48.290 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.912460+0000", 2024-09-20T18:58:48.290 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7029999999999997e-06 2024-09-20T18:58:48.290 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.290 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.290 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:58:48.290 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.912470+0000", 2024-09-20T18:58:48.290 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2469999999999997e-06 2024-09-20T18:58:48.290 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.290 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.290 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:58:48.291 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.912476+0000", 2024-09-20T18:58:48.291 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6059999999999999e-06 2024-09-20T18:58:48.291 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.291 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.291 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:48.291 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.912515+0000", 2024-09-20T18:58:48.291 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9229e-05 2024-09-20T18:58:48.291 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.291 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.291 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:58:48.291 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.912522+0000", 2024-09-20T18:58:48.291 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.866e-06 2024-09-20T18:58:48.291 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.291 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.292 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:48.292 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.705721+0000", 2024-09-20T18:58:48.292 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.79319880300000001 2024-09-20T18:58:48.292 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.292 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.292 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:58:48.292 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.705826+0000", 2024-09-20T18:58:48.292 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010511299999999999 2024-09-20T18:58:48.292 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.292 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.292 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,5", 2024-09-20T18:58:48.292 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.705947+0000", 2024-09-20T18:58:48.292 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000120405 2024-09-20T18:58:48.292 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.293 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.293 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:48.293 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.710142+0000", 2024-09-20T18:58:48.293 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0041950599999999996 2024-09-20T18:58:48.293 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.293 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.293 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:58:48.293 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.712464+0000", 2024-09-20T18:58:48.293 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0023219450000000002 2024-09-20T18:58:48.293 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.293 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.293 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:48.293 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.712570+0000", 2024-09-20T18:58:48.293 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106347 2024-09-20T18:58:48.294 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.294 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.294 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:58:48.294 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.712610+0000", 2024-09-20T18:58:48.294 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0302999999999997e-05 2024-09-20T18:58:48.294 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.294 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.294 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:58:48.294 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.712639+0000", 2024-09-20T18:58:48.294 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9016000000000001e-05 2024-09-20T18:58:48.294 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:48.294 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:58:48.294 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:48.294 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.295 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.295 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:128171 8.3 8:c9f7dd59:::benchmark_data_smithi083_97422_object12080:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e203)", 2024-09-20T18:58:48.295 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:24.917207+0000", 2024-09-20T18:58:48.295 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 143.32773844299999, 2024-09-20T18:58:48.295 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.80280542799999999, 2024-09-20T18:58:48.295 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:58:48.295 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:58:48.295 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:48.295 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:58:48.295 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:58:48.295 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:58:48.295 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 128171 2024-09-20T18:58:48.295 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.295 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:58:48.296 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.296 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:58:48.296 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.917207+0000", 2024-09-20T18:58:48.296 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:48.296 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.296 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.296 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:58:48.296 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.917207+0000", 2024-09-20T18:58:48.296 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:48.296 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.296 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.296 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:58:48.296 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.917209+0000", 2024-09-20T18:58:48.296 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5740000000000001e-06 2024-09-20T18:58:48.296 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.297 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.297 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:58:48.297 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.917215+0000", 2024-09-20T18:58:48.297 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2689999999999996e-06 2024-09-20T18:58:48.297 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.297 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.297 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:58:48.297 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.917229+0000", 2024-09-20T18:58:48.297 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3686000000000001e-05 2024-09-20T18:58:48.297 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.297 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.297 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:58:48.297 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.917236+0000", 2024-09-20T18:58:48.297 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1690000000000003e-06 2024-09-20T18:58:48.298 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.298 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.298 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:48.298 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.917348+0000", 2024-09-20T18:58:48.298 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000111573 2024-09-20T18:58:48.298 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.298 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.298 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:58:48.298 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.917357+0000", 2024-09-20T18:58:48.298 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6269999999999995e-06 2024-09-20T18:58:48.298 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.298 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.298 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:48.298 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.708009+0000", 2024-09-20T18:58:48.298 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.79065208899999995 2024-09-20T18:58:48.299 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.299 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.299 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:58:48.299 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.708102+0000", 2024-09-20T18:58:48.299 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3206000000000002e-05 2024-09-20T18:58:48.299 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.299 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.299 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,5", 2024-09-20T18:58:48.299 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.708240+0000", 2024-09-20T18:58:48.299 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013709700000000001 2024-09-20T18:58:48.299 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.299 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.299 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:48.299 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.712514+0000", 2024-09-20T18:58:48.300 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0042744580000000001 2024-09-20T18:58:48.300 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.300 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.300 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:58:48.300 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.712649+0000", 2024-09-20T18:58:48.300 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013514899999999999 2024-09-20T18:58:48.300 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.300 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.300 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:48.300 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.719853+0000", 2024-09-20T18:58:48.300 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0072033790000000002 2024-09-20T18:58:48.300 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.300 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.300 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:58:48.300 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.719980+0000", 2024-09-20T18:58:48.301 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000127341 2024-09-20T18:58:48.301 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.301 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.301 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:58:48.301 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.720013+0000", 2024-09-20T18:58:48.301 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2811000000000001e-05 2024-09-20T18:58:48.301 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:48.301 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:58:48.301 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:48.301 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.302 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.302 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:128174 8.3 8:c0429fd7:::benchmark_data_smithi083_97422_object12083:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e203)", 2024-09-20T18:58:48.302 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:24.921868+0000", 2024-09-20T18:58:48.302 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 143.323077211, 2024-09-20T18:58:48.302 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.79834304, 2024-09-20T18:58:48.302 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:58:48.302 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:58:48.302 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:48.302 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:58:48.302 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:58:48.302 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:58:48.302 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 128174 2024-09-20T18:58:48.302 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.302 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:58:48.303 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.303 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:58:48.303 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.921868+0000", 2024-09-20T18:58:48.303 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:48.303 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.303 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.303 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:58:48.303 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.921868+0000", 2024-09-20T18:58:48.303 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:48.303 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.303 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.303 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:58:48.303 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.921870+0000", 2024-09-20T18:58:48.303 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7740000000000001e-06 2024-09-20T18:58:48.303 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.304 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.304 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:58:48.304 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.921875+0000", 2024-09-20T18:58:48.304 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0839999999999999e-06 2024-09-20T18:58:48.304 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.304 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.304 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:58:48.304 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.921889+0000", 2024-09-20T18:58:48.304 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.393e-05 2024-09-20T18:58:48.304 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.304 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.304 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:58:48.304 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.921897+0000", 2024-09-20T18:58:48.304 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8639999999999996e-06 2024-09-20T18:58:48.304 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.305 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.305 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:48.305 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.921969+0000", 2024-09-20T18:58:48.305 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1894e-05 2024-09-20T18:58:48.305 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.305 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.305 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:58:48.305 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.921981+0000", 2024-09-20T18:58:48.305 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2194e-05 2024-09-20T18:58:48.305 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.305 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.305 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:48.305 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.710174+0000", 2024-09-20T18:58:48.305 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.78819257200000004 2024-09-20T18:58:48.306 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.306 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.306 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:58:48.306 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.710282+0000", 2024-09-20T18:58:48.306 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000108021 2024-09-20T18:58:48.306 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.306 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.306 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,5", 2024-09-20T18:58:48.306 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.710394+0000", 2024-09-20T18:58:48.306 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000112661 2024-09-20T18:58:48.306 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.306 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.306 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:48.306 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.712692+0000", 2024-09-20T18:58:48.307 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0022973109999999998 2024-09-20T18:58:48.307 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.307 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.307 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:58:48.307 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.717533+0000", 2024-09-20T18:58:48.307 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0048411959999999999 2024-09-20T18:58:48.307 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.307 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.307 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:48.307 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.720119+0000", 2024-09-20T18:58:48.307 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0025860269999999999 2024-09-20T18:58:48.307 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.307 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.307 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:58:48.307 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.720171+0000", 2024-09-20T18:58:48.308 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2120999999999998e-05 2024-09-20T18:58:48.308 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.308 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.308 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:58:48.308 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.720212+0000", 2024-09-20T18:58:48.308 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0391e-05 2024-09-20T18:58:48.308 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:48.308 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:58:48.308 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:48.308 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.308 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.308 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157429 8.7 8:ee8a5b93:::benchmark_data_smithi083_97422_object41338:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e212)", 2024-09-20T18:58:48.308 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.048178+0000", 2024-09-20T18:58:48.308 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 109.19676760599999, 2024-09-20T18:58:48.309 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.034527355, 2024-09-20T18:58:48.309 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:58:48.309 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:58:48.309 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-20T18:58:48.309 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:58:48.309 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:58:48.309 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:58:48.309 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157429 2024-09-20T18:58:48.309 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.309 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:58:48.309 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.309 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:58:48.309 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.048178+0000", 2024-09-20T18:58:48.309 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:48.309 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.310 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.310 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:58:48.310 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.048178+0000", 2024-09-20T18:58:48.310 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:48.310 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.310 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.310 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:58:48.310 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.048180+0000", 2024-09-20T18:58:48.310 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.618e-06 2024-09-20T18:58:48.310 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.310 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.310 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:58:48.311 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.048187+0000", 2024-09-20T18:58:48.311 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8240000000000002e-06 2024-09-20T18:58:48.311 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.311 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.311 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:58:48.311 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.048205+0000", 2024-09-20T18:58:48.311 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8717000000000001e-05 2024-09-20T18:58:48.311 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.311 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.311 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:58:48.311 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.048215+0000", 2024-09-20T18:58:48.311 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.022e-05 2024-09-20T18:58:48.311 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.312 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.312 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:48.312 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.048294+0000", 2024-09-20T18:58:48.312 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8088000000000002e-05 2024-09-20T18:58:48.312 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.312 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.312 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:58:48.312 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.048409+0000", 2024-09-20T18:58:48.312 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115146 2024-09-20T18:58:48.312 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.312 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.312 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3,6", 2024-09-20T18:58:48.312 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.048517+0000", 2024-09-20T18:58:48.312 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010874099999999999 2024-09-20T18:58:48.313 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.313 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.313 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:48.313 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.059282+0000", 2024-09-20T18:58:48.313 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010764078 2024-09-20T18:58:48.313 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.313 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.313 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:58:48.313 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.059297+0000", 2024-09-20T18:58:48.313 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5773e-05 2024-09-20T18:58:48.313 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.313 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.313 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:48.313 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.059343+0000", 2024-09-20T18:58:48.313 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5816000000000003e-05 2024-09-20T18:58:48.314 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.314 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.314 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:48.314 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.070249+0000", 2024-09-20T18:58:48.314 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010906381999999999 2024-09-20T18:58:48.314 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.314 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.314 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:58:48.315 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.070260+0000", 2024-09-20T18:58:48.315 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0404000000000001e-05 2024-09-20T18:58:48.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.315 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.315 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:48.315 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.082592+0000", 2024-09-20T18:58:48.315 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0123322640000001 2024-09-20T18:58:48.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.315 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.316 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:58:48.316 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.082705+0000", 2024-09-20T18:58:48.316 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011328399999999999 2024-09-20T18:58:48.316 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:48.316 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:58:48.316 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:48.316 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.316 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.316 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157435 8.7 8:e8101dbf:::benchmark_data_smithi083_97422_object41344:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e212)", 2024-09-20T18:58:48.316 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.053885+0000", 2024-09-20T18:58:48.316 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 109.191060376, 2024-09-20T18:58:48.316 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0294991419999999, 2024-09-20T18:58:48.316 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:58:48.316 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:58:48.316 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-20T18:58:48.317 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:58:48.317 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:58:48.317 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:58:48.317 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157435 2024-09-20T18:58:48.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.317 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:58:48.317 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.317 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:58:48.317 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053885+0000", 2024-09-20T18:58:48.317 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:48.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.317 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.317 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:58:48.317 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053885+0000", 2024-09-20T18:58:48.318 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:48.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.318 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.318 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:58:48.318 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053887+0000", 2024-09-20T18:58:48.318 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.68e-06 2024-09-20T18:58:48.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.318 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.318 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:58:48.318 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053894+0000", 2024-09-20T18:58:48.318 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5939999999999999e-06 2024-09-20T18:58:48.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.318 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.318 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:58:48.318 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053907+0000", 2024-09-20T18:58:48.319 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3599e-05 2024-09-20T18:58:48.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.319 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.319 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:58:48.319 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053916+0000", 2024-09-20T18:58:48.319 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2029999999999998e-06 2024-09-20T18:58:48.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.319 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.319 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:48.319 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.054004+0000", 2024-09-20T18:58:48.319 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8026000000000001e-05 2024-09-20T18:58:48.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.319 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.319 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:58:48.319 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.054123+0000", 2024-09-20T18:58:48.320 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000119046 2024-09-20T18:58:48.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.320 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.320 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3,6", 2024-09-20T18:58:48.320 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.056711+0000", 2024-09-20T18:58:48.320 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0025878049999999999 2024-09-20T18:58:48.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.320 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.320 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:58:48.320 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.059354+0000", 2024-09-20T18:58:48.320 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0026431969999999999 2024-09-20T18:58:48.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.320 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.320 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:48.320 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.070287+0000", 2024-09-20T18:58:48.321 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010932261 2024-09-20T18:58:48.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.321 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.321 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:58:48.321 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.070294+0000", 2024-09-20T18:58:48.321 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0029999999999997e-06 2024-09-20T18:58:48.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.321 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.321 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:48.321 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.083330+0000", 2024-09-20T18:58:48.321 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.013036592 2024-09-20T18:58:48.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.321 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.321 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:58:48.321 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.083384+0000", 2024-09-20T18:58:48.322 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4135999999999998e-05 2024-09-20T18:58:48.322 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:48.322 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:58:48.322 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:48.322 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.322 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.322 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157436 8.7 8:e2843ea0:::benchmark_data_smithi083_97422_object41345:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e212)", 2024-09-20T18:58:48.322 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.053939+0000", 2024-09-20T18:58:48.322 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 109.191006845, 2024-09-20T18:58:48.322 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.029517123, 2024-09-20T18:58:48.322 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:58:48.322 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:58:48.322 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-20T18:58:48.322 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:58:48.323 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:58:48.323 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:58:48.323 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157436 2024-09-20T18:58:48.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.323 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:58:48.323 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.323 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:58:48.323 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053939+0000", 2024-09-20T18:58:48.323 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:48.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.323 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.323 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:58:48.323 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053939+0000", 2024-09-20T18:58:48.323 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:48.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.324 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.324 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:58:48.324 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053939+0000", 2024-09-20T18:58:48.324 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9100000000000004e-07 2024-09-20T18:58:48.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.324 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.324 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:58:48.324 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053942+0000", 2024-09-20T18:58:48.324 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7980000000000001e-06 2024-09-20T18:58:48.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.324 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.324 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:58:48.324 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053949+0000", 2024-09-20T18:58:48.324 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8580000000000002e-06 2024-09-20T18:58:48.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.325 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.325 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:58:48.325 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053955+0000", 2024-09-20T18:58:48.325 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1449999999999996e-06 2024-09-20T18:58:48.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.325 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.325 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:48.325 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.060057+0000", 2024-09-20T18:58:48.325 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0061021729999999998 2024-09-20T18:58:48.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.325 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.325 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:58:48.325 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.060177+0000", 2024-09-20T18:58:48.325 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000120013 2024-09-20T18:58:48.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.326 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.326 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3,6", 2024-09-20T18:58:48.326 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.060294+0000", 2024-09-20T18:58:48.326 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000117056 2024-09-20T18:58:48.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.326 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.326 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:48.326 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.070317+0000", 2024-09-20T18:58:48.326 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010022778 2024-09-20T18:58:48.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.326 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.326 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:58:48.326 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.070324+0000", 2024-09-20T18:58:48.326 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4300000000000003e-06 2024-09-20T18:58:48.327 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.327 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.327 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:48.327 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.083413+0000", 2024-09-20T18:58:48.327 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0130890960000001 2024-09-20T18:58:48.327 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.327 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.327 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:58:48.327 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.083456+0000", 2024-09-20T18:58:48.327 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3185000000000001e-05 2024-09-20T18:58:48.327 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:48.327 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:58:48.327 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:48.327 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.327 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.328 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157449 8.7 8:e0d43d9e:::benchmark_data_smithi083_97422_object41358:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e213)", 2024-09-20T18:58:48.328 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.075612+0000", 2024-09-20T18:58:48.328 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 109.16933399600001, 2024-09-20T18:58:48.328 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.016308008, 2024-09-20T18:58:48.328 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:58:48.328 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:58:48.328 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:48.328 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:58:48.328 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:58:48.328 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:58:48.328 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157449 2024-09-20T18:58:48.328 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.328 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:58:48.328 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.329 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:58:48.329 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.075612+0000", 2024-09-20T18:58:48.329 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:48.329 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.329 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.329 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:58:48.329 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.075612+0000", 2024-09-20T18:58:48.329 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:48.329 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.329 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.329 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:58:48.329 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.075613+0000", 2024-09-20T18:58:48.329 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1570000000000001e-06 2024-09-20T18:58:48.329 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.329 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.330 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:58:48.330 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.075618+0000", 2024-09-20T18:58:48.330 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7899999999999999e-06 2024-09-20T18:58:48.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.330 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.330 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:58:48.330 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.075627+0000", 2024-09-20T18:58:48.330 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2569999999999996e-06 2024-09-20T18:58:48.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.330 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.330 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:58:48.330 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.075662+0000", 2024-09-20T18:58:48.330 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5439999999999999e-05 2024-09-20T18:58:48.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.331 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.331 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:48.331 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.078479+0000", 2024-09-20T18:58:48.331 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0028168910000000002 2024-09-20T18:58:48.331 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.331 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.331 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:58:48.331 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.078488+0000", 2024-09-20T18:58:48.331 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4069999999999993e-06 2024-09-20T18:58:48.331 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.331 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.331 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:48.331 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.083476+0000", 2024-09-20T18:58:48.331 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.004987893 2024-09-20T18:58:48.331 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.332 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.332 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:58:48.332 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.083567+0000", 2024-09-20T18:58:48.332 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1598e-05 2024-09-20T18:58:48.332 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.332 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.332 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3", 2024-09-20T18:58:48.332 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.083675+0000", 2024-09-20T18:58:48.332 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000107701 2024-09-20T18:58:48.332 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.332 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.332 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:48.332 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.091482+0000", 2024-09-20T18:58:48.332 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0078073639999999998 2024-09-20T18:58:48.332 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.333 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:58:48.333 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.091668+0000", 2024-09-20T18:58:48.333 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018606100000000001 2024-09-20T18:58:48.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.333 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:48.333 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.091718+0000", 2024-09-20T18:58:48.333 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9830000000000002e-05 2024-09-20T18:58:48.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.333 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:58:48.333 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.091878+0000", 2024-09-20T18:58:48.333 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000160166 2024-09-20T18:58:48.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.334 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.334 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:58:48.334 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.091920+0000", 2024-09-20T18:58:48.334 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1452999999999998e-05 2024-09-20T18:58:48.334 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:48.334 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:58:48.334 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:48.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.334 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.334 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157451 8.7 8:ea555b5d:::benchmark_data_smithi083_97422_object41360:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e213)", 2024-09-20T18:58:48.334 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.080725+0000", 2024-09-20T18:58:48.334 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 109.164221029, 2024-09-20T18:58:48.334 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0181950740000001, 2024-09-20T18:58:48.334 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:58:48.335 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:58:48.335 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:48.335 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:58:48.335 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:58:48.335 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:58:48.335 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157451 2024-09-20T18:58:48.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.335 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:58:48.335 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.335 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:58:48.335 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.080725+0000", 2024-09-20T18:58:48.335 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:48.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.335 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.336 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:58:48.336 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.080725+0000", 2024-09-20T18:58:48.336 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:48.336 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.336 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.336 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:58:48.336 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.080726+0000", 2024-09-20T18:58:48.336 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3459999999999999e-06 2024-09-20T18:58:48.336 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.336 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.336 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:58:48.336 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.080731+0000", 2024-09-20T18:58:48.336 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3619999999999997e-06 2024-09-20T18:58:48.336 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.337 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.337 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:58:48.337 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.080742+0000", 2024-09-20T18:58:48.337 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0302000000000001e-05 2024-09-20T18:58:48.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.337 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.337 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:58:48.337 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.080757+0000", 2024-09-20T18:58:48.337 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4955e-05 2024-09-20T18:58:48.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.337 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.337 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:48.337 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.080819+0000", 2024-09-20T18:58:48.337 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2310999999999993e-05 2024-09-20T18:58:48.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.338 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.338 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:58:48.338 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.080826+0000", 2024-09-20T18:58:48.338 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1069999999999999e-06 2024-09-20T18:58:48.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.338 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.338 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:48.338 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.085855+0000", 2024-09-20T18:58:48.338 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0050288679999999 2024-09-20T18:58:48.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.338 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.338 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:58:48.338 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.085947+0000", 2024-09-20T18:58:48.338 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1911999999999997e-05 2024-09-20T18:58:48.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.339 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.339 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3", 2024-09-20T18:58:48.339 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.086042+0000", 2024-09-20T18:58:48.339 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5234000000000003e-05 2024-09-20T18:58:48.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.339 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.339 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:58:48.339 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.091673+0000", 2024-09-20T18:58:48.339 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0056306079999999996 2024-09-20T18:58:48.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.339 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.339 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:48.339 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.098793+0000", 2024-09-20T18:58:48.339 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.007120637 2024-09-20T18:58:48.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.340 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:48.340 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.098828+0000", 2024-09-20T18:58:48.340 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4405e-05 2024-09-20T18:58:48.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.340 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:58:48.340 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.098882+0000", 2024-09-20T18:58:48.340 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4481000000000002e-05 2024-09-20T18:58:48.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.340 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:58:48.340 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.098920+0000", 2024-09-20T18:58:48.340 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7546000000000003e-05 2024-09-20T18:58:48.341 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:48.341 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:58:48.341 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:48.341 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.341 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.341 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157481 8.7 8:ebfa0664:::benchmark_data_smithi083_97422_object41390:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e213)", 2024-09-20T18:58:48.341 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.116650+0000", 2024-09-20T18:58:48.341 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:48.342 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 109.12829551599999, 2024-09-20T18:58:48.342 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.98244276600000002, 2024-09-20T18:58:48.342 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:58:48.342 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:58:48.342 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:48.342 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:58:48.342 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:58:48.343 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:58:48.343 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157481 2024-09-20T18:58:48.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.343 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:58:48.343 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.343 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:58:48.343 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.116650+0000", 2024-09-20T18:58:48.343 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:48.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.343 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.343 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:58:48.343 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.116650+0000", 2024-09-20T18:58:48.343 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:48.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.344 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.344 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:58:48.344 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.116652+0000", 2024-09-20T18:58:48.344 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5680000000000001e-06 2024-09-20T18:58:48.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.344 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.344 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:58:48.344 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.116658+0000", 2024-09-20T18:58:48.344 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4309999999999999e-06 2024-09-20T18:58:48.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.344 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.344 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:58:48.344 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.116671+0000", 2024-09-20T18:58:48.344 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2384e-05 2024-09-20T18:58:48.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.345 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.345 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:58:48.345 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.116678+0000", 2024-09-20T18:58:48.345 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8690000000000004e-06 2024-09-20T18:58:48.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.345 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.345 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:48.345 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.116734+0000", 2024-09-20T18:58:48.345 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6041000000000001e-05 2024-09-20T18:58:48.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.345 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.345 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:58:48.345 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.116742+0000", 2024-09-20T18:58:48.345 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8189999999999992e-06 2024-09-20T18:58:48.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.346 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.346 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:48.346 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.088569+0000", 2024-09-20T18:58:48.346 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97182719100000003 2024-09-20T18:58:48.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.346 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.346 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:58:48.346 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.088666+0000", 2024-09-20T18:58:48.346 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6897000000000004e-05 2024-09-20T18:58:48.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.346 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.346 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3", 2024-09-20T18:58:48.346 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.088770+0000", 2024-09-20T18:58:48.346 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010320299999999999 2024-09-20T18:58:48.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.347 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.347 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:58:48.347 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.091677+0000", 2024-09-20T18:58:48.347 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0029076900000000001 2024-09-20T18:58:48.347 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.347 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.347 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:48.347 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.098961+0000", 2024-09-20T18:58:48.347 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0072835080000000002 2024-09-20T18:58:48.347 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.347 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.347 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:48.347 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.099000+0000", 2024-09-20T18:58:48.347 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.892e-05 2024-09-20T18:58:48.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.348 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.348 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:58:48.348 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.099055+0000", 2024-09-20T18:58:48.348 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5554999999999999e-05 2024-09-20T18:58:48.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.348 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.348 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:58:48.348 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.099093+0000", 2024-09-20T18:58:48.348 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.769e-05 2024-09-20T18:58:48.348 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:48.348 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:58:48.348 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:48.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.349 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.349 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157496 8.7 8:efb55f87:::benchmark_data_smithi083_97422_object41405:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e213)", 2024-09-20T18:58:48.349 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.130402+0000", 2024-09-20T18:58:48.349 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 109.114543798, 2024-09-20T18:58:48.349 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.99286820799999997, 2024-09-20T18:58:48.349 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:58:48.349 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:58:48.349 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:48.349 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:58:48.349 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:58:48.349 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:58:48.349 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157496 2024-09-20T18:58:48.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.349 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:58:48.349 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.350 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:58:48.350 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.130402+0000", 2024-09-20T18:58:48.350 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:48.350 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.350 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.350 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:58:48.350 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.130402+0000", 2024-09-20T18:58:48.350 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:48.350 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.350 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.350 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:58:48.350 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.130404+0000", 2024-09-20T18:58:48.350 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.748e-06 2024-09-20T18:58:48.350 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.351 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.351 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:58:48.351 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.130410+0000", 2024-09-20T18:58:48.351 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2720000000000003e-06 2024-09-20T18:58:48.351 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.351 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.351 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:58:48.351 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.130426+0000", 2024-09-20T18:58:48.351 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6557000000000001e-05 2024-09-20T18:58:48.351 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.351 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.351 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:58:48.351 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.130436+0000", 2024-09-20T18:58:48.351 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4320000000000005e-06 2024-09-20T18:58:48.351 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.352 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.352 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:48.352 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.130499+0000", 2024-09-20T18:58:48.352 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3149000000000001e-05 2024-09-20T18:58:48.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.352 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.352 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:58:48.352 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.130508+0000", 2024-09-20T18:58:48.352 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1619999999999997e-06 2024-09-20T18:58:48.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.352 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.352 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:48.352 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.091952+0000", 2024-09-20T18:58:48.352 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.96144401999999995 2024-09-20T18:58:48.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.353 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.353 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:58:48.353 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.092057+0000", 2024-09-20T18:58:48.353 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000104268 2024-09-20T18:58:48.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.353 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.353 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3", 2024-09-20T18:58:48.353 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.092163+0000", 2024-09-20T18:58:48.353 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106545 2024-09-20T18:58:48.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.353 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.353 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:48.353 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.099133+0000", 2024-09-20T18:58:48.353 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0069702990000000001 2024-09-20T18:58:48.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.354 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.354 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:48.354 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.104449+0000", 2024-09-20T18:58:48.354 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0053152920000000001 2024-09-20T18:58:48.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.354 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.354 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:58:48.354 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.123091+0000", 2024-09-20T18:58:48.354 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.018642757999999999 2024-09-20T18:58:48.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.354 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.354 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:58:48.354 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.123225+0000", 2024-09-20T18:58:48.354 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000133881 2024-09-20T18:58:48.355 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.355 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.355 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:58:48.355 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.123270+0000", 2024-09-20T18:58:48.355 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4824999999999999e-05 2024-09-20T18:58:48.355 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:48.355 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:58:48.355 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:48.355 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.355 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.355 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157528 8.7 8:e4768593:::benchmark_data_smithi083_97422_object41437:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e213)", 2024-09-20T18:58:48.355 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.165392+0000", 2024-09-20T18:58:48.355 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 109.0795535, 2024-09-20T18:58:48.355 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.958042846, 2024-09-20T18:58:48.355 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:58:48.356 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:58:48.356 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:48.356 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:58:48.356 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:58:48.356 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:58:48.356 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157528 2024-09-20T18:58:48.356 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.356 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:58:48.356 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.356 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:58:48.356 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.165392+0000", 2024-09-20T18:58:48.356 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:48.356 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.356 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.357 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:58:48.357 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.165392+0000", 2024-09-20T18:58:48.357 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:48.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.357 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.357 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:58:48.357 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.165394+0000", 2024-09-20T18:58:48.357 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.77e-06 2024-09-20T18:58:48.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.357 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.357 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:58:48.357 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.165400+0000", 2024-09-20T18:58:48.357 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2099999999999998e-06 2024-09-20T18:58:48.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.357 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.358 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:58:48.358 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.165419+0000", 2024-09-20T18:58:48.358 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9046999999999999e-05 2024-09-20T18:58:48.358 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.358 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.358 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:58:48.358 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.165428+0000", 2024-09-20T18:58:48.358 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1030000000000008e-06 2024-09-20T18:58:48.358 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.358 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.358 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:48.358 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.166013+0000", 2024-09-20T18:58:48.358 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000584868 2024-09-20T18:58:48.358 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.359 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.359 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:58:48.359 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.166022+0000", 2024-09-20T18:58:48.359 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7090000000000008e-06 2024-09-20T18:58:48.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.359 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.359 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:48.359 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.095256+0000", 2024-09-20T18:58:48.359 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.92923407700000005 2024-09-20T18:58:48.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.359 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.359 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:58:48.359 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.095349+0000", 2024-09-20T18:58:48.359 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2724000000000002e-05 2024-09-20T18:58:48.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.360 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.360 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3", 2024-09-20T18:58:48.360 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.095688+0000", 2024-09-20T18:58:48.360 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000339379 2024-09-20T18:58:48.360 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.360 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.360 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:48.360 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.104489+0000", 2024-09-20T18:58:48.360 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0088013359999999999 2024-09-20T18:58:48.360 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.360 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.360 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:48.360 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.106738+0000", 2024-09-20T18:58:48.360 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0022487029999999999 2024-09-20T18:58:48.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.361 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.361 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:58:48.361 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.123280+0000", 2024-09-20T18:58:48.361 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.016541613 2024-09-20T18:58:48.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.361 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.361 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:58:48.361 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.123398+0000", 2024-09-20T18:58:48.361 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011812299999999999 2024-09-20T18:58:48.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.361 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.361 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:58:48.361 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.123435+0000", 2024-09-20T18:58:48.361 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7184000000000002e-05 2024-09-20T18:58:48.362 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:48.362 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:58:48.362 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:48.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.362 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.362 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157535 8.7 8:ed2a8870:::benchmark_data_smithi083_97422_object41444:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e213)", 2024-09-20T18:58:48.362 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.173838+0000", 2024-09-20T18:58:48.362 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 109.071107888, 2024-09-20T18:58:48.362 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.95335319799999996, 2024-09-20T18:58:48.362 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:58:48.362 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:58:48.362 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:48.362 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:58:48.362 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:58:48.363 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:58:48.363 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157535 2024-09-20T18:58:48.363 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.363 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:58:48.363 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.363 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:58:48.363 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.173838+0000", 2024-09-20T18:58:48.363 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:48.363 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.363 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.363 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:58:48.363 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.173838+0000", 2024-09-20T18:58:48.363 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:48.363 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.363 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.364 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:58:48.364 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.173839+0000", 2024-09-20T18:58:48.364 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5340000000000001e-06 2024-09-20T18:58:48.364 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.364 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.364 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:58:48.364 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.173846+0000", 2024-09-20T18:58:48.364 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2910000000000004e-06 2024-09-20T18:58:48.364 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.364 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.364 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:58:48.364 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.173867+0000", 2024-09-20T18:58:48.364 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0928e-05 2024-09-20T18:58:48.364 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.364 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.365 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:58:48.365 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.173874+0000", 2024-09-20T18:58:48.365 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.6990000000000003e-06 2024-09-20T18:58:48.365 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.365 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.365 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:48.365 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.173935+0000", 2024-09-20T18:58:48.365 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0899000000000001e-05 2024-09-20T18:58:48.365 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.365 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.365 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:58:48.365 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.173944+0000", 2024-09-20T18:58:48.365 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5369999999999997e-06 2024-09-20T18:58:48.365 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.365 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.366 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:48.366 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.099161+0000", 2024-09-20T18:58:48.366 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.92521724299999997 2024-09-20T18:58:48.366 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.366 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.366 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:58:48.366 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.099277+0000", 2024-09-20T18:58:48.366 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115689 2024-09-20T18:58:48.366 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.366 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.366 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3", 2024-09-20T18:58:48.366 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.099390+0000", 2024-09-20T18:58:48.366 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011385199999999999 2024-09-20T18:58:48.366 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.367 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.367 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:48.367 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.106780+0000", 2024-09-20T18:58:48.367 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.007389367 2024-09-20T18:58:48.367 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.367 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.367 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:48.367 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.114134+0000", 2024-09-20T18:58:48.367 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0073541429999999996 2024-09-20T18:58:48.367 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.367 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.367 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:58:48.367 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.123441+0000", 2024-09-20T18:58:48.367 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0093066620000000003 2024-09-20T18:58:48.367 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.368 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.368 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:58:48.368 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.127133+0000", 2024-09-20T18:58:48.368 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0036927879999999998 2024-09-20T18:58:48.368 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.368 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.368 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:58:48.368 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.127191+0000", 2024-09-20T18:58:48.368 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7565999999999997e-05 2024-09-20T18:58:48.368 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:48.368 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:58:48.369 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:48.369 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.369 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.369 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157545 8.7 8:e3edf173:::benchmark_data_smithi083_97422_object41454:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e213)", 2024-09-20T18:58:48.370 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.188692+0000", 2024-09-20T18:58:48.370 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 109.056253283, 2024-09-20T18:58:48.370 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.93865532500000004, 2024-09-20T18:58:48.370 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:58:48.370 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:58:48.370 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:48.370 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:58:48.370 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:58:48.370 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:58:48.370 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157545 2024-09-20T18:58:48.370 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.370 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:58:48.370 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.370 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:58:48.371 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.188692+0000", 2024-09-20T18:58:48.371 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:48.371 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.371 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.371 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:58:48.371 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.188692+0000", 2024-09-20T18:58:48.371 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:48.371 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.371 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.371 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:58:48.371 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.188694+0000", 2024-09-20T18:58:48.371 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.869e-06 2024-09-20T18:58:48.371 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.371 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.371 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:58:48.372 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.188701+0000", 2024-09-20T18:58:48.372 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9829999999999999e-06 2024-09-20T18:58:48.372 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.372 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.372 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:58:48.372 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.188718+0000", 2024-09-20T18:58:48.372 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6750999999999999e-05 2024-09-20T18:58:48.372 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.372 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.372 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:58:48.372 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.188727+0000", 2024-09-20T18:58:48.372 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6449999999999994e-06 2024-09-20T18:58:48.372 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.372 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.373 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:48.373 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.188800+0000", 2024-09-20T18:58:48.373 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3022999999999995e-05 2024-09-20T18:58:48.373 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.373 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.373 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:58:48.373 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.188810+0000", 2024-09-20T18:58:48.373 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0019e-05 2024-09-20T18:58:48.373 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.373 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.373 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:48.373 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.101615+0000", 2024-09-20T18:58:48.374 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.91280537699999997 2024-09-20T18:58:48.374 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.374 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.374 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:58:48.374 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.101713+0000", 2024-09-20T18:58:48.374 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7870999999999999e-05 2024-09-20T18:58:48.374 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.374 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.374 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3", 2024-09-20T18:58:48.374 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.101823+0000", 2024-09-20T18:58:48.374 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010982 2024-09-20T18:58:48.374 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.374 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.374 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:48.375 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.106819+0000", 2024-09-20T18:58:48.375 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00499592 2024-09-20T18:58:48.375 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.375 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.375 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:48.375 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.114173+0000", 2024-09-20T18:58:48.375 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0073544980000000001 2024-09-20T18:58:48.375 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.375 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.375 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:58:48.375 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.127205+0000", 2024-09-20T18:58:48.375 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.013031839 2024-09-20T18:58:48.375 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.375 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.376 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:58:48.376 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.127309+0000", 2024-09-20T18:58:48.376 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010435899999999999 2024-09-20T18:58:48.376 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.376 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.376 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:58:48.376 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.127348+0000", 2024-09-20T18:58:48.376 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8350999999999999e-05 2024-09-20T18:58:48.376 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:48.376 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:58:48.376 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:48.376 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.376 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.376 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157575 8.7 8:e03ab630:::benchmark_data_smithi083_97422_object41484:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e213)", 2024-09-20T18:58:48.377 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.236943+0000", 2024-09-20T18:58:48.377 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 109.008002666, 2024-09-20T18:58:48.377 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.89067574599999999, 2024-09-20T18:58:48.377 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:58:48.377 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:58:48.377 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:48.377 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:58:48.377 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:58:48.377 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:58:48.377 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157575 2024-09-20T18:58:48.377 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.377 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:58:48.377 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.377 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:58:48.378 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.236943+0000", 2024-09-20T18:58:48.378 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:48.378 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.378 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.378 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:58:48.378 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.236943+0000", 2024-09-20T18:58:48.378 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:48.378 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.378 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.378 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:58:48.378 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.236945+0000", 2024-09-20T18:58:48.378 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.584e-06 2024-09-20T18:58:48.378 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.378 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.379 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:58:48.379 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.236949+0000", 2024-09-20T18:58:48.379 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.673e-06 2024-09-20T18:58:48.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.379 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.379 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:58:48.379 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.236963+0000", 2024-09-20T18:58:48.379 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4209000000000001e-05 2024-09-20T18:58:48.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.379 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.379 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:58:48.379 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.236970+0000", 2024-09-20T18:58:48.379 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0029999999999997e-06 2024-09-20T18:58:48.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.379 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.380 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:48.380 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.237078+0000", 2024-09-20T18:58:48.380 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000107026 2024-09-20T18:58:48.380 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.380 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.380 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:58:48.380 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.237085+0000", 2024-09-20T18:58:48.380 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4669999999999998e-06 2024-09-20T18:58:48.380 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.380 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.380 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:48.380 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.106857+0000", 2024-09-20T18:58:48.380 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.86977158899999996 2024-09-20T18:58:48.380 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.381 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.381 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:58:48.381 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.106967+0000", 2024-09-20T18:58:48.381 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011026700000000001 2024-09-20T18:58:48.381 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.381 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.381 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3", 2024-09-20T18:58:48.381 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.107090+0000", 2024-09-20T18:58:48.381 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012295399999999999 2024-09-20T18:58:48.381 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.381 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.381 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:58:48.381 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.127396+0000", 2024-09-20T18:58:48.381 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.020306026000000001 2024-09-20T18:58:48.381 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.382 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.382 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:48.382 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.127459+0000", 2024-09-20T18:58:48.382 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3322000000000001e-05 2024-09-20T18:58:48.382 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.382 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.382 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:48.382 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.127496+0000", 2024-09-20T18:58:48.382 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7156999999999997e-05 2024-09-20T18:58:48.382 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.382 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.382 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:58:48.382 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.127581+0000", 2024-09-20T18:58:48.382 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4679999999999996e-05 2024-09-20T18:58:48.382 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.383 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.383 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:58:48.383 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.127619+0000", 2024-09-20T18:58:48.383 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7789e-05 2024-09-20T18:58:48.383 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:48.383 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:58:48.383 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:48.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.383 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.383 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157577 8.7 8:ed15ec18:::benchmark_data_smithi083_97422_object41486:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e213)", 2024-09-20T18:58:48.383 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.239083+0000", 2024-09-20T18:58:48.383 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 109.005862499, 2024-09-20T18:58:48.385 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88873236799999999, 2024-09-20T18:58:48.385 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:58:48.385 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:58:48.386 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:48.386 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:58:48.386 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:58:48.386 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:58:48.386 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157577 2024-09-20T18:58:48.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.386 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:58:48.386 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.386 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:58:48.386 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.239083+0000", 2024-09-20T18:58:48.386 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:48.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.386 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.386 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:58:48.387 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.239083+0000", 2024-09-20T18:58:48.387 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:48.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.387 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.387 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:58:48.387 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.239084+0000", 2024-09-20T18:58:48.387 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5599999999999995e-07 2024-09-20T18:58:48.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.387 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.387 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:58:48.387 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.239087+0000", 2024-09-20T18:58:48.387 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8990000000000001e-06 2024-09-20T18:58:48.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.387 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.387 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:58:48.388 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.239094+0000", 2024-09-20T18:58:48.388 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5859999999999999e-06 2024-09-20T18:58:48.388 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.388 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.388 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:58:48.388 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.239100+0000", 2024-09-20T18:58:48.388 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7490000000000004e-06 2024-09-20T18:58:48.388 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.388 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.388 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:48.388 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.239149+0000", 2024-09-20T18:58:48.388 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9202000000000001e-05 2024-09-20T18:58:48.388 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.388 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.388 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:58:48.389 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.239157+0000", 2024-09-20T18:58:48.389 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.276e-06 2024-09-20T18:58:48.389 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.389 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.389 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:48.389 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.109327+0000", 2024-09-20T18:58:48.389 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87016914899999998 2024-09-20T18:58:48.389 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.389 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.389 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:58:48.389 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.109435+0000", 2024-09-20T18:58:48.389 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010798099999999999 2024-09-20T18:58:48.389 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.389 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.389 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3", 2024-09-20T18:58:48.390 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.109546+0000", 2024-09-20T18:58:48.390 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011176299999999999 2024-09-20T18:58:48.390 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.390 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.390 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:58:48.390 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.127401+0000", 2024-09-20T18:58:48.390 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.017854716999999999 2024-09-20T18:58:48.390 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.390 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.390 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:48.390 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.127661+0000", 2024-09-20T18:58:48.390 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00026038800000000001 2024-09-20T18:58:48.390 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.390 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.391 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:48.391 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.127697+0000", 2024-09-20T18:58:48.391 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5666e-05 2024-09-20T18:58:48.391 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.391 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.391 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:58:48.391 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.127779+0000", 2024-09-20T18:58:48.391 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.1860000000000006e-05 2024-09-20T18:58:48.391 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.391 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.391 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:58:48.391 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.127816+0000", 2024-09-20T18:58:48.391 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6575999999999997e-05 2024-09-20T18:58:48.391 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:48.391 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:58:48.392 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:48.392 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.392 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.392 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157637 8.7 8:ee232675:::benchmark_data_smithi083_97422_object41546:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e213)", 2024-09-20T18:58:48.392 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.327404+0000", 2024-09-20T18:58:48.392 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 108.917541884, 2024-09-20T18:58:48.392 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.80573648099999995, 2024-09-20T18:58:48.392 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:58:48.392 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:58:48.392 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:58:48.392 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:58:48.392 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:58:48.392 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:58:48.392 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157637 2024-09-20T18:58:48.393 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.393 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:58:48.393 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.393 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:58:48.393 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.327404+0000", 2024-09-20T18:58:48.393 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:48.393 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.393 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.393 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:58:48.393 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.327404+0000", 2024-09-20T18:58:48.393 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:58:48.393 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.393 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.393 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:58:48.394 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.327405+0000", 2024-09-20T18:58:48.394 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3179999999999999e-06 2024-09-20T18:58:48.394 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.394 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.394 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:58:48.394 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.327410+0000", 2024-09-20T18:58:48.394 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.062e-06 2024-09-20T18:58:48.394 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.394 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.394 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:58:48.394 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.327422+0000", 2024-09-20T18:58:48.394 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.167e-05 2024-09-20T18:58:48.394 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.394 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.395 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:58:48.395 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.327429+0000", 2024-09-20T18:58:48.395 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3520000000000001e-06 2024-09-20T18:58:48.395 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.395 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.395 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:48.395 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.327497+0000", 2024-09-20T18:58:48.395 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7898000000000006e-05 2024-09-20T18:58:48.395 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.395 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.395 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:58:48.395 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.327508+0000", 2024-09-20T18:58:48.395 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1209999999999999e-05 2024-09-20T18:58:48.395 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.395 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.396 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:58:48.396 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.111708+0000", 2024-09-20T18:58:48.396 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.78419992000000005 2024-09-20T18:58:48.396 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.396 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.396 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:58:48.396 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.111816+0000", 2024-09-20T18:58:48.396 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010796100000000001 2024-09-20T18:58:48.396 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.396 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.396 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3", 2024-09-20T18:58:48.396 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.111932+0000", 2024-09-20T18:58:48.396 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011589199999999999 2024-09-20T18:58:48.396 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.397 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.397 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:58:48.397 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.127406+0000", 2024-09-20T18:58:48.397 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.01547343 2024-09-20T18:58:48.397 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.397 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.397 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:48.397 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.127869+0000", 2024-09-20T18:58:48.397 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00046379199999999998 2024-09-20T18:58:48.397 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.397 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.397 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:58:48.397 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.132820+0000", 2024-09-20T18:58:48.397 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049506630000000001 2024-09-20T18:58:48.397 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.398 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.398 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:58:48.398 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.133087+0000", 2024-09-20T18:58:48.398 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00026739899999999998 2024-09-20T18:58:48.398 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:58:48.398 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:58:48.398 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:58:48.398 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.133140+0000", 2024-09-20T18:58:48.398 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2914000000000002e-05 2024-09-20T18:58:48.398 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:48.398 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:58:48.398 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:48.398 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:58:48.398 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:58:48.399 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T18:58:48.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:48.400 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:48.400 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:48.499 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T18:58:48.501 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:48.533 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:48.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:48.534 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:48.635 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:48.665 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:48.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:48.666 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:48.766 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:48.795 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:48.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:48.796 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:48.895 INFO:teuthology.orchestra.run.smithi083.stderr:set pool 1 pgp_num to 1 2024-09-20T18:58:48.897 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:48.937 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:48.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:48.938 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:49.039 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:49.081 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:49.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:49.083 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:49.137 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:48 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1619287741' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"} : dispatch 2024-09-20T18:58:49.137 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:48 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"} : dispatch 2024-09-20T18:58:49.137 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:48 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1619287741' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"} : dispatch 2024-09-20T18:58:49.138 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:48 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"} : dispatch 2024-09-20T18:58:49.184 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:49.216 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:49.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:49.217 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:49.248 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 9 16 23893 23877 165.789 136.312 0.0059211 0.00602634 2024-09-20T18:58:49.267 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:48 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1619287741' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"} : dispatch 2024-09-20T18:58:49.268 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:48 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"} : dispatch 2024-09-20T18:58:49.318 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:49.347 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:49.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:49.348 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:49.449 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:49.490 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:49.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:49.491 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:49.592 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:49.625 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:49.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:49.626 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:49.726 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:58:49.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:49.762 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:49.762 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:58:49.824 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:58:49.824 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:58:49.824 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:58:49.824 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:58:49.824 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:58:49.824 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:58:49.824 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:58:49.825 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:58:49.825 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:58:49.825 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:58:49.825 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:58:49.825 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:58:49.825 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:58:49.825 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:58:49.825 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:58:49.826 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:58:49.863 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:49.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:49.901 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:49.901 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:50.002 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:50.037 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:50.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:50.038 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:50.139 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:50.148 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:49 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"}]': finished 2024-09-20T18:58:50.148 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:49 smithi083 ceph-mon[29421]: osdmap e254: 8 total, 8 up, 7 in 2024-09-20T18:58:50.148 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:49 smithi083 ceph-mon[29421]: pgmap v440: 148 pgs: 148 active+clean; 1.1 GiB data, 5.2 GiB used, 621 GiB / 626 GiB avail; 852 B/s rd, 147 MiB/s wr, 2.36k op/s 2024-09-20T18:58:50.148 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:49 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1165644232' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:58:50.148 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:49 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"}]': finished 2024-09-20T18:58:50.148 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:49 smithi083 ceph-mon[36260]: osdmap e254: 8 total, 8 up, 7 in 2024-09-20T18:58:50.148 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:49 smithi083 ceph-mon[36260]: pgmap v440: 148 pgs: 148 active+clean; 1.1 GiB data, 5.2 GiB used, 621 GiB / 626 GiB avail; 852 B/s rd, 147 MiB/s wr, 2.36k op/s 2024-09-20T18:58:50.148 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:49 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1165644232' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:58:50.178 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:50.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:50.179 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:50.248 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 10 15 26251 26236 163.953 147.438 0.00557988 0.00609262 2024-09-20T18:58:50.267 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:49 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"}]': finished 2024-09-20T18:58:50.267 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:49 smithi142 ceph-mon[32508]: osdmap e254: 8 total, 8 up, 7 in 2024-09-20T18:58:50.267 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:49 smithi142 ceph-mon[32508]: pgmap v440: 148 pgs: 148 active+clean; 1.1 GiB data, 5.2 GiB used, 621 GiB / 626 GiB avail; 852 B/s rd, 147 MiB/s wr, 2.36k op/s 2024-09-20T18:58:50.267 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:49 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1165644232' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:58:50.280 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:50.320 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:50.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:50.321 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:50.422 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:50.454 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:50.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:50.455 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:50.556 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:50.589 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:50.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:50.589 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:50.690 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:50.706 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-20T18:58:50.716 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:50.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:50.717 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:50.817 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:58:50.857 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:50.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:50.858 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:58:50.929 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:58:50 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:58:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:58:50.959 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:50.989 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:50.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:50.990 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:51.090 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:51.120 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:51.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:51.121 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:51.222 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:58:51.250 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 11 16 28529 28513 161.983 142.312 0.00716301 0.00616839 2024-09-20T18:58:51.260 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:51.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:51.261 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:58:51.362 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:51.396 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:51.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:51.397 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:51.498 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:51.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:51.528 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:51.528 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:51.629 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:51.655 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:51.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:51.656 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:51.757 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:51.821 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:51.822 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:51 smithi083 ceph-mon[29421]: osdmap e255: 8 total, 8 up, 7 in 2024-09-20T18:58:51.822 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:51 smithi083 ceph-mon[36260]: osdmap e255: 8 total, 8 up, 7 in 2024-09-20T18:58:51.823 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:58:51.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:51.824 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:51.844 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:58:51.926 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:51.969 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:51.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:51.971 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:52.017 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:51 smithi142 ceph-mon[32508]: osdmap e255: 8 total, 8 up, 7 in 2024-09-20T18:58:52.071 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:52.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:52.109 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:52.109 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:52.210 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:52.235 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:52.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:52.236 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:52.248 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 12 16 30352 30336 157.979 113.938 0.00238657 0.00631363 2024-09-20T18:58:52.336 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:52.365 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:52.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:52.366 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:52.467 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:52.491 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:52.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:52.492 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:52.593 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:58:52.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:52.631 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:52.631 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:58:52.732 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:58:52.769 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:52.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:52.770 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:58:52.870 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:52.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:52.906 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:52.906 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:53.007 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:53.017 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:52 smithi142 ceph-mon[32508]: pgmap v442: 148 pgs: 148 active+clean; 1.4 GiB data, 6.2 GiB used, 620 GiB / 626 GiB avail; 1.7 KiB/s rd, 199 MiB/s wr, 3.18k op/s 2024-09-20T18:58:53.017 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:52 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:58:53.017 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:52 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:58:53.046 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:53.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:53.048 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:53.048 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:52 smithi083 ceph-mon[29421]: pgmap v442: 148 pgs: 148 active+clean; 1.4 GiB data, 6.2 GiB used, 620 GiB / 626 GiB avail; 1.7 KiB/s rd, 199 MiB/s wr, 3.18k op/s 2024-09-20T18:58:53.048 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:52 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:58:53.048 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:52 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:58:53.048 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:52 smithi083 ceph-mon[36260]: pgmap v442: 148 pgs: 148 active+clean; 1.4 GiB data, 6.2 GiB used, 620 GiB / 626 GiB avail; 1.7 KiB/s rd, 199 MiB/s wr, 3.18k op/s 2024-09-20T18:58:53.048 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:52 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:58:53.048 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:52 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:58:53.148 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:53.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:53.177 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:53.177 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:53.248 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 13 16 31987 31971 153.687 102.188 0.0231329 0.00650132 2024-09-20T18:58:53.278 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:53.313 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T18:58:53.315 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:53.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:53.316 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:53.417 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:53.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:53.461 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:53.462 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:53.562 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:53.591 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:53.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:53.592 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:53.692 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:53.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:53.714 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:53.715 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:53.816 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:53.854 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:53.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:53.855 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:53.956 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:58:53.992 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:53.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:53.993 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:58:54.094 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:54.115 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:54.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:54.116 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:54.217 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:58:54.248 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 14 16 33441 33425 149.2 90.875 0.0063723 0.00669736 2024-09-20T18:58:54.254 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:54.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:54.255 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:58:54.267 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:53 smithi142 ceph-mon[32508]: osdmap e256: 8 total, 8 up, 7 in 2024-09-20T18:58:54.267 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:53 smithi142 ceph-mon[32508]: pgmap v444: 148 pgs: 148 active+clean; 1.8 GiB data, 7.3 GiB used, 618 GiB / 626 GiB avail; 852 B/s rd, 142 MiB/s wr, 2.26k op/s 2024-09-20T18:58:54.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:53 smithi083 ceph-mon[29421]: osdmap e256: 8 total, 8 up, 7 in 2024-09-20T18:58:54.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:53 smithi083 ceph-mon[29421]: pgmap v444: 148 pgs: 148 active+clean; 1.8 GiB data, 7.3 GiB used, 618 GiB / 626 GiB avail; 852 B/s rd, 142 MiB/s wr, 2.26k op/s 2024-09-20T18:58:54.297 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:53 smithi083 ceph-mon[36260]: osdmap e256: 8 total, 8 up, 7 in 2024-09-20T18:58:54.298 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:53 smithi083 ceph-mon[36260]: pgmap v444: 148 pgs: 148 active+clean; 1.8 GiB data, 7.3 GiB used, 618 GiB / 626 GiB avail; 852 B/s rd, 142 MiB/s wr, 2.26k op/s 2024-09-20T18:58:54.356 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:54.386 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:54.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:54.387 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:54.488 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:54.520 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:54.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:54.521 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:54.622 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:54.660 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:54.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:54.661 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:54.761 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:54.798 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:54.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:54.799 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:54.897 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-20T18:58:54.899 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:54.934 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:54.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:54.935 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:55.036 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:55.066 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:55.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:55.067 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:55.168 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:58:55.192 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:55.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:55.193 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:58:55.248 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 15 16 35419 35403 147.494 123.625 0.00344171 0.00677267 2024-09-20T18:58:55.267 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:54 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/488015650' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:58:55.267 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:54 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:58:55.267 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:54 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:58:55.294 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:55.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:54 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/488015650' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:58:55.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:54 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:58:55.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:54 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:58:55.297 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:54 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/488015650' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:58:55.297 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:54 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:58:55.298 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:54 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:58:55.330 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:55.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:55.331 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:55.432 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:58:55.466 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:55.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:55.468 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:58:55.568 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:55.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:55.602 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:55.602 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:55.703 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:55.738 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:55.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:55.739 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:55.839 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:55.865 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:55.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:55.866 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:55.966 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:55.998 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:55.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:55.999 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:56.099 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:56.141 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:56.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:56.142 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:56.243 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:58:56.249 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 16 15 37371 37356 145.897 122.062 0.00918287 0.00684929 2024-09-20T18:58:56.267 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:55 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:58:56.267 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:55 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:58:56.267 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:55 smithi142 ceph-mon[32508]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:58:56.267 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:55 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/488015650' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T18:58:56.267 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:55 smithi142 ceph-mon[32508]: osdmap e257: 8 total, 8 up, 7 in 2024-09-20T18:58:56.268 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:55 smithi142 ceph-mon[32508]: pgmap v446: 148 pgs: 148 active+clean; 1.8 GiB data, 7.3 GiB used, 618 GiB / 626 GiB avail; 852 B/s rd, 115 MiB/s wr, 1.85k op/s 2024-09-20T18:58:56.268 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:55 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:58:56.268 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:55 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:58:56.268 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:55 smithi142 ceph-mon[32508]: 6.16 scrub starts 2024-09-20T18:58:56.268 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:55 smithi142 ceph-mon[32508]: 6.16 scrub ok 2024-09-20T18:58:56.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:56.282 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:56.282 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:58:56.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:55 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:58:56.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:55 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:58:56.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:55 smithi083 ceph-mon[29421]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:58:56.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:55 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/488015650' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T18:58:56.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:55 smithi083 ceph-mon[29421]: osdmap e257: 8 total, 8 up, 7 in 2024-09-20T18:58:56.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:55 smithi083 ceph-mon[29421]: pgmap v446: 148 pgs: 148 active+clean; 1.8 GiB data, 7.3 GiB used, 618 GiB / 626 GiB avail; 852 B/s rd, 115 MiB/s wr, 1.85k op/s 2024-09-20T18:58:56.298 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:55 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:58:56.298 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:55 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:58:56.298 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:55 smithi083 ceph-mon[29421]: 6.16 scrub starts 2024-09-20T18:58:56.298 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:55 smithi083 ceph-mon[29421]: 6.16 scrub ok 2024-09-20T18:58:56.298 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:55 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:58:56.298 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:55 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:58:56.298 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:55 smithi083 ceph-mon[36260]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:58:56.298 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:55 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/488015650' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T18:58:56.299 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:55 smithi083 ceph-mon[36260]: osdmap e257: 8 total, 8 up, 7 in 2024-09-20T18:58:56.299 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:55 smithi083 ceph-mon[36260]: pgmap v446: 148 pgs: 148 active+clean; 1.8 GiB data, 7.3 GiB used, 618 GiB / 626 GiB avail; 852 B/s rd, 115 MiB/s wr, 1.85k op/s 2024-09-20T18:58:56.299 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:55 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:58:56.299 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:55 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:58:56.299 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:55 smithi083 ceph-mon[36260]: 6.16 scrub starts 2024-09-20T18:58:56.299 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:55 smithi083 ceph-mon[36260]: 6.16 scrub ok 2024-09-20T18:58:56.382 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:58:56.418 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:56.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:56.419 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:58:56.519 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:56.552 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:56.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:56.553 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:56.654 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:58:56.694 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:56.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:56.695 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:58:56.796 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:56.816 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:56.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:56.817 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:56.825 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 7] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-20T18:58:56.825 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T18:58:56.825 INFO:tasks.thrashosds.thrasher:Adding osd 6 2024-09-20T18:58:56.825 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd in 6 2024-09-20T18:58:56.918 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:58:56.957 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:56.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:56.958 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:58:57.059 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:57.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:57.090 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:57.090 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:57.191 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:57.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:57.232 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:57.232 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:57.250 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 17 16 39192 39176 144.006 113.75 0.0172005 0.00693875 2024-09-20T18:58:57.267 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:58:56 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:58:57.267 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:56 smithi142 ceph-mon[32508]: osdmap e258: 8 total, 8 up, 7 in 2024-09-20T18:58:57.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:56 smithi083 ceph-mon[29421]: osdmap e258: 8 total, 8 up, 7 in 2024-09-20T18:58:57.297 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:56 smithi083 ceph-mon[36260]: osdmap e258: 8 total, 8 up, 7 in 2024-09-20T18:58:57.333 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:57.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:57.362 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:57.362 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:57.463 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:57.493 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:57.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:57.494 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:57.595 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:57.627 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:57.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:57.628 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:57.729 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:57.750 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:57.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:57.751 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:57.852 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:58:57.887 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:57.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:57.888 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:58:57.989 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:58:58.026 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:58.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:58.027 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:58:58.128 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:58.160 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:58.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:58.161 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:58.249 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 18 15 40849 40834 141.762 103.625 0.00497937 0.00704934 2024-09-20T18:58:58.262 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:58.267 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:57 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:58:58.267 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:57 smithi142 ceph-mon[32508]: pgmap v448: 148 pgs: 148 active+clean; 2.2 GiB data, 8.9 GiB used, 617 GiB / 626 GiB avail; 852 B/s rd, 139 MiB/s wr, 2.23k op/s 2024-09-20T18:58:58.267 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:57 smithi142 ceph-mon[32508]: 4.19 scrub starts 2024-09-20T18:58:58.267 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:57 smithi142 ceph-mon[32508]: 4.19 scrub ok 2024-09-20T18:58:58.267 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:57 smithi142 ceph-mon[32508]: 3.f scrub starts 2024-09-20T18:58:58.267 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:58:57 smithi142 ceph-mon[32508]: 3.f scrub ok 2024-09-20T18:58:58.294 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:58.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:58.295 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:58.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:57 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:58:58.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:57 smithi083 ceph-mon[29421]: pgmap v448: 148 pgs: 148 active+clean; 2.2 GiB data, 8.9 GiB used, 617 GiB / 626 GiB avail; 852 B/s rd, 139 MiB/s wr, 2.23k op/s 2024-09-20T18:58:58.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:57 smithi083 ceph-mon[29421]: 4.19 scrub starts 2024-09-20T18:58:58.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:57 smithi083 ceph-mon[29421]: 4.19 scrub ok 2024-09-20T18:58:58.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:57 smithi083 ceph-mon[29421]: 3.f scrub starts 2024-09-20T18:58:58.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:58:57 smithi083 ceph-mon[29421]: 3.f scrub ok 2024-09-20T18:58:58.298 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:57 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:58:58.298 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:57 smithi083 ceph-mon[36260]: pgmap v448: 148 pgs: 148 active+clean; 2.2 GiB data, 8.9 GiB used, 617 GiB / 626 GiB avail; 852 B/s rd, 139 MiB/s wr, 2.23k op/s 2024-09-20T18:58:58.298 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:57 smithi083 ceph-mon[36260]: 4.19 scrub starts 2024-09-20T18:58:58.298 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:57 smithi083 ceph-mon[36260]: 4.19 scrub ok 2024-09-20T18:58:58.298 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:57 smithi083 ceph-mon[36260]: 3.f scrub starts 2024-09-20T18:58:58.298 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:58:57 smithi083 ceph-mon[36260]: 3.f scrub ok 2024-09-20T18:58:58.395 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:58:58.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:58.432 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:58.433 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:58:58.533 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:58.567 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:58.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:58.568 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:58.669 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:58.701 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:58.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:58.703 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:58.803 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:58.831 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:58.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:58.832 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:58.933 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:58:58.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:58.966 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:58.966 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:58:59.067 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:59.104 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:59.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:59.105 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:58:59.206 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:58:59.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:59.237 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:59.237 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:58:59.249 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 19 16 42657 42641 140.245 112.938 0.00682121 0.00712498 2024-09-20T18:58:59.338 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:59.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:59.373 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:59.373 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:59.474 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:58:59.500 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:59.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:59.501 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:58:59.560 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:58:59.602 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:58:59.633 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:58:59.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:59.634 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:58:59.735 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:58:59.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:59.784 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:59.784 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:58:59.885 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:58:59.922 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:58:59.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:58:59.923 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:59:00.024 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:00.057 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:00.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:00.058 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:00.159 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:59:00.193 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:00.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:00.195 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:59:00.249 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:2024-09-20T18:59:00.239863+0000 min lat: 0.00133037 max lat: 0.132461 avg lat: 0.00720714 2024-09-20T18:59:00.249 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-20T18:59:00.249 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 20 16 44389 44373 138.645 108.25 0.0143609 0.00720714 2024-09-20T18:59:00.296 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:00.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:00 smithi083 ceph-mon[29421]: pgmap v449: 148 pgs: 148 active+clean; 2.3 GiB data, 9.1 GiB used, 617 GiB / 626 GiB avail; 826 B/s rd, 89 MiB/s wr, 1.43k op/s 2024-09-20T18:59:00.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:00 smithi083 ceph-mon[29421]: 5.5 scrub starts 2024-09-20T18:59:00.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:00 smithi083 ceph-mon[29421]: 5.5 scrub ok 2024-09-20T18:59:00.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:00 smithi083 ceph-mon[29421]: 3.11 scrub starts 2024-09-20T18:59:00.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:00 smithi083 ceph-mon[29421]: 3.11 scrub ok 2024-09-20T18:59:00.298 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:00 smithi083 ceph-mon[36260]: pgmap v449: 148 pgs: 148 active+clean; 2.3 GiB data, 9.1 GiB used, 617 GiB / 626 GiB avail; 826 B/s rd, 89 MiB/s wr, 1.43k op/s 2024-09-20T18:59:00.298 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:00 smithi083 ceph-mon[36260]: 5.5 scrub starts 2024-09-20T18:59:00.298 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:00 smithi083 ceph-mon[36260]: 5.5 scrub ok 2024-09-20T18:59:00.298 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:00 smithi083 ceph-mon[36260]: 3.11 scrub starts 2024-09-20T18:59:00.298 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:00 smithi083 ceph-mon[36260]: 3.11 scrub ok 2024-09-20T18:59:00.340 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:00.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:00.341 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:00.350 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T18:59:00.350 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:59:00.442 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:00.485 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:00.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:00.486 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:00.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:00 smithi142 ceph-mon[32508]: pgmap v449: 148 pgs: 148 active+clean; 2.3 GiB data, 9.1 GiB used, 617 GiB / 626 GiB avail; 826 B/s rd, 89 MiB/s wr, 1.43k op/s 2024-09-20T18:59:00.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:00 smithi142 ceph-mon[32508]: 5.5 scrub starts 2024-09-20T18:59:00.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:00 smithi142 ceph-mon[32508]: 5.5 scrub ok 2024-09-20T18:59:00.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:00 smithi142 ceph-mon[32508]: 3.11 scrub starts 2024-09-20T18:59:00.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:00 smithi142 ceph-mon[32508]: 3.11 scrub ok 2024-09-20T18:59:00.587 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:00.594 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:59:00.594 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:59:00.621 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:00.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:00.622 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:00.624 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:59:00.624 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:59:00.642 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:59:00.643 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:59:00.679 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:59:00.680 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:59:00.714 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:59:00.714 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:59:00.723 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:00.747 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:59:00.747 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:59:00.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:00.765 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:00.765 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:00.781 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:59:00.781 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:59:00.821 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:59:00.821 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:59:00.865 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:00.901 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:00.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:00.903 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:01.003 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:59:01.035 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:01.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:01.036 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:59:01.042 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:59:00 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:59:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:59:01.137 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:01.172 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:01.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:01.173 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:01.249 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 21 16 46032 46016 136.932 102.688 0.00854278 0.0072981 2024-09-20T18:59:01.274 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:01.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:01 smithi083 ceph-mon[29421]: 5.3 scrub starts 2024-09-20T18:59:01.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:01 smithi083 ceph-mon[29421]: 5.3 scrub ok 2024-09-20T18:59:01.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:01 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3461042565' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:59:01.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:01 smithi083 ceph-mon[29421]: osdmap e259: 8 total, 8 up, 7 in 2024-09-20T18:59:01.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:01 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:59:01.297 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:01 smithi083 ceph-mon[36260]: 5.3 scrub starts 2024-09-20T18:59:01.298 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:01 smithi083 ceph-mon[36260]: 5.3 scrub ok 2024-09-20T18:59:01.298 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:01 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3461042565' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:59:01.298 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:01 smithi083 ceph-mon[36260]: osdmap e259: 8 total, 8 up, 7 in 2024-09-20T18:59:01.298 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:01 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:59:01.303 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:01.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:01.304 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:01.309 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_blocked_ops 2024-09-20T18:59:01.405 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:01.449 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:01.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:01.450 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:01.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:01 smithi142 ceph-mon[32508]: 5.3 scrub starts 2024-09-20T18:59:01.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:01 smithi142 ceph-mon[32508]: 5.3 scrub ok 2024-09-20T18:59:01.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:01 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3461042565' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:59:01.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:01 smithi142 ceph-mon[32508]: osdmap e259: 8 total, 8 up, 7 in 2024-09-20T18:59:01.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:01 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:59:01.523 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T18:59:01.551 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:01.574 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:01.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:01.575 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:01.676 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:01.706 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:01.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:01.707 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:01.808 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:01.852 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:01.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:01.853 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:01.954 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:59:01.983 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:01.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:01.984 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:59:02.084 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:02.121 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:02.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:02.122 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:02.186 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:59:02.223 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:59:02.250 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 22 16 47563 47547 135.054 95.6875 0.00657859 0.00740019 2024-09-20T18:59:02.260 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:02.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:02.261 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:59:02.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:02 smithi083 ceph-mon[29421]: pgmap v451: 148 pgs: 148 active+clean; 2.4 GiB data, 9.5 GiB used, 616 GiB / 626 GiB avail; 1.6 KiB/s rd, 107 MiB/s wr, 1.71k op/s 2024-09-20T18:59:02.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:02 smithi083 ceph-mon[29421]: 6.c scrub starts 2024-09-20T18:59:02.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:02 smithi083 ceph-mon[29421]: 6.c scrub ok 2024-09-20T18:59:02.297 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:02 smithi083 ceph-mon[36260]: pgmap v451: 148 pgs: 148 active+clean; 2.4 GiB data, 9.5 GiB used, 616 GiB / 626 GiB avail; 1.6 KiB/s rd, 107 MiB/s wr, 1.71k op/s 2024-09-20T18:59:02.297 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:02 smithi083 ceph-mon[36260]: 6.c scrub starts 2024-09-20T18:59:02.298 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:02 smithi083 ceph-mon[36260]: 6.c scrub ok 2024-09-20T18:59:02.362 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:59:02.401 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:02.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:02.402 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:59:02.502 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:02.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:02 smithi142 ceph-mon[32508]: pgmap v451: 148 pgs: 148 active+clean; 2.4 GiB data, 9.5 GiB used, 616 GiB / 626 GiB avail; 1.6 KiB/s rd, 107 MiB/s wr, 1.71k op/s 2024-09-20T18:59:02.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:02 smithi142 ceph-mon[32508]: 6.c scrub starts 2024-09-20T18:59:02.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:02 smithi142 ceph-mon[32508]: 6.c scrub ok 2024-09-20T18:59:02.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:02.535 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:02.535 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:02.636 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:02.666 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:02.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:02.667 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:02.768 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:59:02.792 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:02.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:02.793 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:59:02.893 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:59:02.935 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:02.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:02.936 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:59:03.037 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:03.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:03.067 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:03.067 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:03.168 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:59:03.206 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:03.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:03.207 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:59:03.249 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 23 16 49208 49192 133.652 102.812 0.0040039 0.00747655 2024-09-20T18:59:03.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:03 smithi083 ceph-mon[29421]: 4.1e scrub starts 2024-09-20T18:59:03.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:03 smithi083 ceph-mon[29421]: 4.1e scrub ok 2024-09-20T18:59:03.297 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:03 smithi083 ceph-mon[36260]: 4.1e scrub starts 2024-09-20T18:59:03.297 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:03 smithi083 ceph-mon[36260]: 4.1e scrub ok 2024-09-20T18:59:03.308 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:59:03.316 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:59:03.336 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:03.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:03.337 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:59:03.397 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:59:03.438 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:59:03.477 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:03.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:03.478 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:59:03.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:03 smithi142 ceph-mon[32508]: 4.1e scrub starts 2024-09-20T18:59:03.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:03 smithi142 ceph-mon[32508]: 4.1e scrub ok 2024-09-20T18:59:03.579 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:59:03.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:03.618 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:03.619 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:59:03.719 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:59:03.743 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:03.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:03.744 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:59:03.844 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:03.874 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:03.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:03.876 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:03.914 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:03 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mon-a[29417]: 2024-09-20T18:59:03.740+0000 7f6c091c6640 -1 mon.a@0(leader).osd e259 definitely_dead 0 2024-09-20T18:59:03.977 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:03.999 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:04.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:04.000 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:04.076 INFO:teuthology.orchestra.run.smithi083.stderr:marked in osd.6. 2024-09-20T18:59:04.101 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:59:04.140 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:04.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:04.141 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:59:04.212 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:04 smithi083 ceph-mon[29421]: pgmap v452: 148 pgs: 148 active+clean; 2.8 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 1.2 KiB/s rd, 136 MiB/s wr, 2.17k op/s 2024-09-20T18:59:04.212 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:04 smithi083 ceph-mon[29421]: 3.1e scrub starts 2024-09-20T18:59:04.212 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:04 smithi083 ceph-mon[29421]: 3.1e scrub ok 2024-09-20T18:59:04.212 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:04 smithi083 ceph-mon[29421]: 3.1a scrub starts 2024-09-20T18:59:04.212 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:04 smithi083 ceph-mon[29421]: 3.1a scrub ok 2024-09-20T18:59:04.213 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:04 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2448856598' entity='client.admin' cmd={"prefix": "osd in", "ids": ["6"]} : dispatch 2024-09-20T18:59:04.213 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:04 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd in", "ids": ["6"]} : dispatch 2024-09-20T18:59:04.213 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:04 smithi083 ceph-mon[36260]: pgmap v452: 148 pgs: 148 active+clean; 2.8 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 1.2 KiB/s rd, 136 MiB/s wr, 2.17k op/s 2024-09-20T18:59:04.213 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:04 smithi083 ceph-mon[36260]: 3.1e scrub starts 2024-09-20T18:59:04.213 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:04 smithi083 ceph-mon[36260]: 3.1e scrub ok 2024-09-20T18:59:04.213 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:04 smithi083 ceph-mon[36260]: 3.1a scrub starts 2024-09-20T18:59:04.213 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:04 smithi083 ceph-mon[36260]: 3.1a scrub ok 2024-09-20T18:59:04.213 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:04 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2448856598' entity='client.admin' cmd={"prefix": "osd in", "ids": ["6"]} : dispatch 2024-09-20T18:59:04.213 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:04 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd in", "ids": ["6"]} : dispatch 2024-09-20T18:59:04.237 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T18:59:04.238 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-20T18:59:04.238 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-20T18:59:04.238 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-20T18:59:04.238 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T18:59:04.242 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:59:04.249 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 24 16 50523 50507 131.508 82.1875 0.00240845 0.0075617 2024-09-20T18:59:04.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:04.265 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:04.265 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:59:04.367 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:04.389 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:04.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:04.390 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:04.491 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:59:04.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:04.513 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:04.513 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:59:04.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:04 smithi142 ceph-mon[32508]: pgmap v452: 148 pgs: 148 active+clean; 2.8 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 1.2 KiB/s rd, 136 MiB/s wr, 2.17k op/s 2024-09-20T18:59:04.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:04 smithi142 ceph-mon[32508]: 3.1e scrub starts 2024-09-20T18:59:04.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:04 smithi142 ceph-mon[32508]: 3.1e scrub ok 2024-09-20T18:59:04.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:04 smithi142 ceph-mon[32508]: 3.1a scrub starts 2024-09-20T18:59:04.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:04 smithi142 ceph-mon[32508]: 3.1a scrub ok 2024-09-20T18:59:04.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:04 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2448856598' entity='client.admin' cmd={"prefix": "osd in", "ids": ["6"]} : dispatch 2024-09-20T18:59:04.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:04 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd in", "ids": ["6"]} : dispatch 2024-09-20T18:59:04.613 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:04.635 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:04.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:04.636 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:04.737 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:59:04.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:04.759 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:04.759 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:59:04.789 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T18:59:04.859 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:04.880 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:04.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:04.881 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:04.963 INFO:tasks.thrashosds.thrasher:Added osd 6 2024-09-20T18:59:04.982 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:59:05.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:05.003 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:05.003 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:59:05.104 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:05.131 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:05.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:05.132 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:05.233 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:05.250 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 25 16 50523 50507 126.248 0 - 0.0075617 2024-09-20T18:59:05.254 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:05.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:05.255 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:05.356 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:59:05.378 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:05.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:05.379 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:59:05.479 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:05.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:05.502 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:05.502 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:05.513 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:05 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:59:05.513 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:05 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:59:05.513 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:05 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:59:05.514 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:05 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:59:05.514 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:05 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["6"]}]': finished 2024-09-20T18:59:05.514 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:05 smithi083 ceph-mon[29421]: osdmap e260: 8 total, 8 up, 8 in 2024-09-20T18:59:05.514 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:05 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:59:05.514 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:05 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:59:05.514 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:05 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:59:05.514 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:05 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:59:05.514 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:05 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["6"]}]': finished 2024-09-20T18:59:05.514 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:05 smithi083 ceph-mon[36260]: osdmap e260: 8 total, 8 up, 8 in 2024-09-20T18:59:05.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:05 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:59:05.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:05 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:59:05.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:05 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:59:05.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:05 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:59:05.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:05 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["6"]}]': finished 2024-09-20T18:59:05.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:05 smithi142 ceph-mon[32508]: osdmap e260: 8 total, 8 up, 8 in 2024-09-20T18:59:05.540 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_historic_ops 2024-09-20T18:59:05.604 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:05.626 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:05.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:05.627 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:05.727 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:05.750 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:05.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:05.751 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:05.852 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:59:05.859 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:59:05.876 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:05.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:05.877 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:59:05.888 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:59:05.978 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:59:05.999 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:06.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:06.000 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:59:06.073 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-20T18:59:06.101 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:06.127 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:06.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:06.128 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:06.229 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:59:06.250 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 26 16 50549 50533 121.455 0.8125 0.00779007 0.0081455 2024-09-20T18:59:06.262 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:06.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:06.263 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:59:06.363 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:06.400 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:06.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:06.401 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:06.437 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:06 smithi083 ceph-mon[29421]: pgmap v454: 148 pgs: 148 active+clean; 2.8 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 639 B/s rd, 79 MiB/s wr, 1.27k op/s 2024-09-20T18:59:06.437 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:06 smithi083 ceph-mon[29421]: osdmap e261: 8 total, 8 up, 8 in 2024-09-20T18:59:06.437 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:06 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1770169890' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:59:06.437 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:06 smithi083 ceph-mon[36260]: pgmap v454: 148 pgs: 148 active+clean; 2.8 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 639 B/s rd, 79 MiB/s wr, 1.27k op/s 2024-09-20T18:59:06.437 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:06 smithi083 ceph-mon[36260]: osdmap e261: 8 total, 8 up, 8 in 2024-09-20T18:59:06.437 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:06 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1770169890' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:59:06.503 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:59:06.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:06 smithi142 ceph-mon[32508]: pgmap v454: 148 pgs: 148 active+clean; 2.8 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 639 B/s rd, 79 MiB/s wr, 1.27k op/s 2024-09-20T18:59:06.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:06 smithi142 ceph-mon[32508]: osdmap e261: 8 total, 8 up, 8 in 2024-09-20T18:59:06.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:06 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1770169890' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:59:06.544 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:06.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:06.545 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:59:06.614 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T18:59:06.615 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-20T18:59:06.615 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-20T18:59:06.615 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-20T18:59:06.615 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.615 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:27549 7.e 7:77436293:::benchmark_data_smithi083_70643_object27548:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:59:06.615 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:08.731657+0000", 2024-09-20T18:59:06.615 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 477.86464222699999, 2024-09-20T18:59:06.615 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.113586293, 2024-09-20T18:59:06.615 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:59:06.615 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:59:06.615 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:06.615 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:59:06.615 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:59:06.616 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:59:06.616 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 27549 2024-09-20T18:59:06.616 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.616 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:59:06.616 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.616 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:59:06.616 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.731657+0000", 2024-09-20T18:59:06.616 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:06.616 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.616 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.616 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:59:06.616 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.731657+0000", 2024-09-20T18:59:06.616 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:06.616 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.616 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.617 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:59:06.617 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.731658+0000", 2024-09-20T18:59:06.617 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1400000000000001e-06 2024-09-20T18:59:06.617 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.617 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.617 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:59:06.617 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.731686+0000", 2024-09-20T18:59:06.617 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7840000000000001e-05 2024-09-20T18:59:06.617 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.617 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.617 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:59:06.617 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.731697+0000", 2024-09-20T18:59:06.617 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0774999999999999e-05 2024-09-20T18:59:06.617 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.618 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.618 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:59:06.618 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.731704+0000", 2024-09-20T18:59:06.618 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1740000000000003e-06 2024-09-20T18:59:06.618 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.618 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.618 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:59:06.618 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.731757+0000", 2024-09-20T18:59:06.618 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3214000000000002e-05 2024-09-20T18:59:06.618 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.618 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.618 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:59:06.618 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.731876+0000", 2024-09-20T18:59:06.619 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000118829 2024-09-20T18:59:06.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.619 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.619 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:59:06.619 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.732005+0000", 2024-09-20T18:59:06.619 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000128937 2024-09-20T18:59:06.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.619 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.619 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:59:06.619 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733315+0000", 2024-09-20T18:59:06.619 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001309943 2024-09-20T18:59:06.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.619 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.619 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:06.620 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843149+0000", 2024-09-20T18:59:06.620 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.109834472 2024-09-20T18:59:06.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.620 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.620 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:06.620 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843225+0000", 2024-09-20T18:59:06.620 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5304999999999998e-05 2024-09-20T18:59:06.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.620 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.620 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:59:06.620 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.844447+0000", 2024-09-20T18:59:06.620 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0012227239999999999 2024-09-20T18:59:06.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.620 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.620 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:59:06.621 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.845243+0000", 2024-09-20T18:59:06.621 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00079593999999999997 2024-09-20T18:59:06.621 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:06.621 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:06.621 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:06.621 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.621 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.621 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:27551 7.8 7:1bc4112c:::benchmark_data_smithi083_70643_object27550:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:59:06.621 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:08.733037+0000", 2024-09-20T18:59:06.621 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 477.86326243799999, 2024-09-20T18:59:06.621 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.110337717, 2024-09-20T18:59:06.621 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:59:06.621 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:59:06.621 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:06.621 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:59:06.622 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:59:06.622 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:59:06.622 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 27551 2024-09-20T18:59:06.622 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.622 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:59:06.622 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.622 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:59:06.622 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733037+0000", 2024-09-20T18:59:06.622 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:06.622 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.622 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.622 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:59:06.622 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733037+0000", 2024-09-20T18:59:06.622 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:06.623 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.623 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.623 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:59:06.623 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733038+0000", 2024-09-20T18:59:06.623 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.497e-06 2024-09-20T18:59:06.623 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.623 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.623 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:59:06.623 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733062+0000", 2024-09-20T18:59:06.623 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3498000000000001e-05 2024-09-20T18:59:06.623 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.623 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.623 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:59:06.623 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733072+0000", 2024-09-20T18:59:06.623 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0173000000000001e-05 2024-09-20T18:59:06.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.624 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.624 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:59:06.624 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733079+0000", 2024-09-20T18:59:06.624 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5370000000000003e-06 2024-09-20T18:59:06.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.624 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.624 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:59:06.624 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733187+0000", 2024-09-20T18:59:06.624 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010888 2024-09-20T18:59:06.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.624 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.624 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:59:06.624 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733330+0000", 2024-09-20T18:59:06.624 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000142206 2024-09-20T18:59:06.625 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.625 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.625 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4,7", 2024-09-20T18:59:06.625 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733549+0000", 2024-09-20T18:59:06.625 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0002198 2024-09-20T18:59:06.625 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.625 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.625 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:59:06.625 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.735541+0000", 2024-09-20T18:59:06.625 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0019915129999999999 2024-09-20T18:59:06.625 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.625 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.625 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:06.625 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.842311+0000", 2024-09-20T18:59:06.626 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.106769612 2024-09-20T18:59:06.626 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.626 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.626 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:06.626 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.842372+0000", 2024-09-20T18:59:06.626 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0968000000000003e-05 2024-09-20T18:59:06.626 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.626 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.626 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:59:06.626 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843315+0000", 2024-09-20T18:59:06.626 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00094376300000000002 2024-09-20T18:59:06.626 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.626 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.626 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:59:06.626 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843375+0000", 2024-09-20T18:59:06.627 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.927e-05 2024-09-20T18:59:06.627 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:06.627 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:06.627 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:06.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.627 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.627 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:27555 7.e 7:73709da2:::benchmark_data_smithi083_70643_object27554:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:59:06.627 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:08.733695+0000", 2024-09-20T18:59:06.627 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 477.862604315, 2024-09-20T18:59:06.627 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.11168107300000001, 2024-09-20T18:59:06.627 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:59:06.627 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:59:06.627 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:06.627 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:59:06.628 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:59:06.628 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:59:06.628 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 27555 2024-09-20T18:59:06.628 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.628 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:59:06.628 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.628 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:59:06.628 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733695+0000", 2024-09-20T18:59:06.628 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:06.628 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.628 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.628 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:59:06.628 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733695+0000", 2024-09-20T18:59:06.629 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:06.629 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.629 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.629 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:59:06.629 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733696+0000", 2024-09-20T18:59:06.629 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0200000000000001e-07 2024-09-20T18:59:06.629 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.629 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.629 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:59:06.629 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733718+0000", 2024-09-20T18:59:06.629 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2226999999999999e-05 2024-09-20T18:59:06.629 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.629 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.629 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:59:06.629 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733727+0000", 2024-09-20T18:59:06.630 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6049999999999998e-06 2024-09-20T18:59:06.630 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.630 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.630 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:59:06.630 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733734+0000", 2024-09-20T18:59:06.630 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0639999999999996e-06 2024-09-20T18:59:06.630 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.630 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.630 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:59:06.630 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733789+0000", 2024-09-20T18:59:06.630 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5087999999999998e-05 2024-09-20T18:59:06.630 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.630 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.630 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:59:06.631 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733920+0000", 2024-09-20T18:59:06.631 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013146800000000001 2024-09-20T18:59:06.631 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.631 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.631 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:59:06.631 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733995+0000", 2024-09-20T18:59:06.631 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4664000000000002e-05 2024-09-20T18:59:06.631 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.631 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.631 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:59:06.631 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.735538+0000", 2024-09-20T18:59:06.631 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001542671 2024-09-20T18:59:06.631 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.631 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.632 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:06.632 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843187+0000", 2024-09-20T18:59:06.632 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.107649921 2024-09-20T18:59:06.632 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.632 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.632 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:06.632 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.845296+0000", 2024-09-20T18:59:06.632 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0021081920000000001 2024-09-20T18:59:06.632 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.632 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.632 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:59:06.632 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.845341+0000", 2024-09-20T18:59:06.632 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5141e-05 2024-09-20T18:59:06.632 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.632 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.633 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:59:06.633 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.845376+0000", 2024-09-20T18:59:06.633 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.523e-05 2024-09-20T18:59:06.633 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:06.633 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:06.633 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:06.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.633 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.633 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:27560 7.e 7:7d7d0b94:::benchmark_data_smithi083_70643_object27559:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:59:06.633 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:08.736128+0000", 2024-09-20T18:59:06.633 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 477.86017171600002, 2024-09-20T18:59:06.633 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.112670376, 2024-09-20T18:59:06.633 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:59:06.633 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:59:06.633 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:06.634 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:59:06.634 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:59:06.634 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:59:06.634 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 27560 2024-09-20T18:59:06.634 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.634 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:59:06.634 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.634 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:59:06.634 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736128+0000", 2024-09-20T18:59:06.634 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:06.634 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.634 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.634 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:59:06.634 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736128+0000", 2024-09-20T18:59:06.635 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:06.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.635 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.635 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:59:06.635 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736129+0000", 2024-09-20T18:59:06.635 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.251e-06 2024-09-20T18:59:06.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.635 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.635 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:59:06.635 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736150+0000", 2024-09-20T18:59:06.635 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1450999999999998e-05 2024-09-20T18:59:06.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.635 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.635 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:59:06.635 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736161+0000", 2024-09-20T18:59:06.636 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0682e-05 2024-09-20T18:59:06.636 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.636 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.636 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:59:06.636 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736168+0000", 2024-09-20T18:59:06.636 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8549999999999996e-06 2024-09-20T18:59:06.636 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.636 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.636 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:59:06.636 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736218+0000", 2024-09-20T18:59:06.636 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0497000000000001e-05 2024-09-20T18:59:06.636 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.636 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.636 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:59:06.636 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736327+0000", 2024-09-20T18:59:06.637 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010837599999999999 2024-09-20T18:59:06.637 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.637 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.637 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:59:06.637 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.841982+0000", 2024-09-20T18:59:06.637 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.105654846 2024-09-20T18:59:06.637 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.637 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.637 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:59:06.637 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.845386+0000", 2024-09-20T18:59:06.637 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0034042719999999999 2024-09-20T18:59:06.637 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.637 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.637 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:06.637 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.845423+0000", 2024-09-20T18:59:06.638 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7160000000000003e-05 2024-09-20T18:59:06.638 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.638 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.638 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:06.638 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.848702+0000", 2024-09-20T18:59:06.638 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0032792099999999999 2024-09-20T18:59:06.638 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.638 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.638 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:59:06.638 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.848758+0000", 2024-09-20T18:59:06.638 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.537e-05 2024-09-20T18:59:06.638 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.638 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.638 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:59:06.639 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.848798+0000", 2024-09-20T18:59:06.639 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0405999999999999e-05 2024-09-20T18:59:06.639 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:06.639 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:06.639 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:06.639 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.639 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.639 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:27559 7.a e93/83 7:5f89880a:::benchmark_data_smithi083_70643_object27558:head v 93'1704, mlcod=93'1702)", 2024-09-20T18:59:06.639 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:08.736205+0000", 2024-09-20T18:59:06.639 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 477.86009418700002, 2024-09-20T18:59:06.639 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.108390217, 2024-09-20T18:59:06.639 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:59:06.639 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:59:06.639 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:06.639 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:59:06.640 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.640 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:59:06.640 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736205+0000", 2024-09-20T18:59:06.640 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:06.640 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.640 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.640 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:59:06.640 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736205+0000", 2024-09-20T18:59:06.640 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:06.640 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.640 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.640 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:59:06.640 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736206+0000", 2024-09-20T18:59:06.640 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4000000000000001e-07 2024-09-20T18:59:06.640 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.641 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.641 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:59:06.641 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736226+0000", 2024-09-20T18:59:06.641 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0426999999999999e-05 2024-09-20T18:59:06.641 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.641 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.642 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:59:06.642 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736236+0000", 2024-09-20T18:59:06.642 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.781e-06 2024-09-20T18:59:06.642 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.642 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.642 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:59:06.642 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736242+0000", 2024-09-20T18:59:06.642 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0179999999999998e-06 2024-09-20T18:59:06.642 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.642 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.642 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:59:06.642 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736307+0000", 2024-09-20T18:59:06.642 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5042000000000005e-05 2024-09-20T18:59:06.642 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.643 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.643 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:59:06.643 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736325+0000", 2024-09-20T18:59:06.643 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7895999999999999e-05 2024-09-20T18:59:06.643 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.643 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.643 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:59:06.643 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.844517+0000", 2024-09-20T18:59:06.643 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.10819235100000001 2024-09-20T18:59:06.643 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.643 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.643 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:59:06.643 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.844595+0000", 2024-09-20T18:59:06.643 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7961999999999997e-05 2024-09-20T18:59:06.644 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:06.644 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:06.644 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:06.644 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.644 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.644 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:27549 7.e e93/83 ondisk, result = 0)", 2024-09-20T18:59:06.644 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:08.736243+0000", 2024-09-20T18:59:06.644 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 477.86005659300002, 2024-09-20T18:59:06.644 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.106913722, 2024-09-20T18:59:06.644 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:59:06.644 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:59:06.644 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:59:06.644 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:59:06.644 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.644 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:59:06.645 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736243+0000", 2024-09-20T18:59:06.645 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:06.645 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.645 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.645 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:59:06.645 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736243+0000", 2024-09-20T18:59:06.645 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:06.645 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.645 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.645 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:59:06.645 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736243+0000", 2024-09-20T18:59:06.645 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3200000000000004e-07 2024-09-20T18:59:06.645 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.645 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.645 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:59:06.646 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736248+0000", 2024-09-20T18:59:06.646 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2780000000000004e-06 2024-09-20T18:59:06.646 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.646 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.646 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:59:06.646 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736257+0000", 2024-09-20T18:59:06.646 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2790000000000004e-06 2024-09-20T18:59:06.646 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.646 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.646 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:59:06.646 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736263+0000", 2024-09-20T18:59:06.646 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3570000000000003e-06 2024-09-20T18:59:06.646 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.646 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.646 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:59:06.647 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843136+0000", 2024-09-20T18:59:06.647 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.10687292499999999 2024-09-20T18:59:06.647 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.647 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.647 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:59:06.647 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843143+0000", 2024-09-20T18:59:06.647 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2300000000000002e-06 2024-09-20T18:59:06.647 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.647 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.647 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:59:06.647 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843156+0000", 2024-09-20T18:59:06.647 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2921e-05 2024-09-20T18:59:06.647 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:06.647 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:06.648 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:06.648 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.648 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.648 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:27555 7.e e93/83 ondisk, result = 0)", 2024-09-20T18:59:06.648 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:08.736486+0000", 2024-09-20T18:59:06.648 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 477.85981350499998, 2024-09-20T18:59:06.648 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.106708079, 2024-09-20T18:59:06.648 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:59:06.648 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:59:06.648 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:59:06.648 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:59:06.648 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.648 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:59:06.648 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736486+0000", 2024-09-20T18:59:06.648 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:06.648 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.649 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.649 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:59:06.649 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736486+0000", 2024-09-20T18:59:06.649 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:06.649 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.649 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.649 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:59:06.649 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736486+0000", 2024-09-20T18:59:06.649 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8100000000000002e-07 2024-09-20T18:59:06.649 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.649 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.649 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:59:06.649 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736490+0000", 2024-09-20T18:59:06.649 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.546e-06 2024-09-20T18:59:06.649 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.650 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.650 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:59:06.650 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.841491+0000", 2024-09-20T18:59:06.650 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.105001423 2024-09-20T18:59:06.650 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.650 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.650 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:59:06.650 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.841503+0000", 2024-09-20T18:59:06.650 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1399e-05 2024-09-20T18:59:06.650 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.650 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.650 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:59:06.650 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843177+0000", 2024-09-20T18:59:06.650 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0016741900000000001 2024-09-20T18:59:06.651 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.651 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.651 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:59:06.651 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843183+0000", 2024-09-20T18:59:06.651 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7209999999999999e-06 2024-09-20T18:59:06.651 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.651 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.651 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:59:06.651 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843194+0000", 2024-09-20T18:59:06.651 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1119e-05 2024-09-20T18:59:06.651 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:06.651 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:06.651 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:06.651 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.651 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.652 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:27551 7.8 e93/83 ondisk, result = 0)", 2024-09-20T18:59:06.652 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:08.736884+0000", 2024-09-20T18:59:06.652 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 477.85941570699998, 2024-09-20T18:59:06.652 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.105442044, 2024-09-20T18:59:06.652 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:59:06.652 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:59:06.652 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:59:06.652 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:59:06.652 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.652 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:59:06.652 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736884+0000", 2024-09-20T18:59:06.652 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:06.652 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.652 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.652 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:59:06.653 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736884+0000", 2024-09-20T18:59:06.653 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:06.653 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.653 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.653 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:59:06.653 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736884+0000", 2024-09-20T18:59:06.653 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6499999999999999e-07 2024-09-20T18:59:06.653 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.653 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.653 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:59:06.653 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736888+0000", 2024-09-20T18:59:06.653 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7170000000000002e-06 2024-09-20T18:59:06.653 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.653 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.653 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:59:06.653 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.841517+0000", 2024-09-20T18:59:06.653 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.104628836 2024-09-20T18:59:06.654 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.654 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.654 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:59:06.654 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.841525+0000", 2024-09-20T18:59:06.654 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7110000000000001e-06 2024-09-20T18:59:06.654 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.654 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.654 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:59:06.654 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.842286+0000", 2024-09-20T18:59:06.654 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00076098500000000005 2024-09-20T18:59:06.654 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.654 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.654 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:59:06.654 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.842304+0000", 2024-09-20T18:59:06.654 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7898e-05 2024-09-20T18:59:06.654 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.655 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.655 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:59:06.655 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.842326+0000", 2024-09-20T18:59:06.655 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1331999999999998e-05 2024-09-20T18:59:06.655 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:06.655 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:06.655 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:06.655 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.655 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.655 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:27571 7.8 7:1e0af83a:::benchmark_data_smithi083_70643_object27570:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:59:06.655 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:08.741453+0000", 2024-09-20T18:59:06.655 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 477.85484650299998, 2024-09-20T18:59:06.655 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.103905603, 2024-09-20T18:59:06.655 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:59:06.655 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:59:06.655 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:06.656 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:59:06.656 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:59:06.656 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:59:06.656 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 27571 2024-09-20T18:59:06.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.656 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:59:06.656 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.656 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:59:06.656 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.741453+0000", 2024-09-20T18:59:06.656 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:06.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.656 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.656 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:59:06.656 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.741453+0000", 2024-09-20T18:59:06.656 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:06.657 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.657 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.657 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:59:06.657 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.741455+0000", 2024-09-20T18:59:06.657 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4669999999999998e-06 2024-09-20T18:59:06.657 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.657 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.657 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:59:06.657 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.741483+0000", 2024-09-20T18:59:06.657 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7472999999999999e-05 2024-09-20T18:59:06.657 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.657 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.657 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:59:06.657 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.841438+0000", 2024-09-20T18:59:06.657 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.099955024000000003 2024-09-20T18:59:06.657 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.658 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.658 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:59:06.658 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.841457+0000", 2024-09-20T18:59:06.658 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9264999999999999e-05 2024-09-20T18:59:06.658 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.658 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.658 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:59:06.658 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843412+0000", 2024-09-20T18:59:06.658 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0019553869999999998 2024-09-20T18:59:06.658 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.658 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.658 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:59:06.658 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843559+0000", 2024-09-20T18:59:06.658 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014645399999999999 2024-09-20T18:59:06.658 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.658 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.659 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4,7", 2024-09-20T18:59:06.659 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843706+0000", 2024-09-20T18:59:06.659 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014695600000000001 2024-09-20T18:59:06.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.659 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.659 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:59:06.659 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.844855+0000", 2024-09-20T18:59:06.659 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001149424 2024-09-20T18:59:06.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.659 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.659 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:06.659 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.845215+0000", 2024-09-20T18:59:06.659 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00035930100000000002 2024-09-20T18:59:06.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.659 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.660 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:06.660 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.845270+0000", 2024-09-20T18:59:06.660 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5461999999999999e-05 2024-09-20T18:59:06.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.660 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.660 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:59:06.660 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.845321+0000", 2024-09-20T18:59:06.660 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1360000000000003e-05 2024-09-20T18:59:06.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.660 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.660 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:59:06.660 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.845358+0000", 2024-09-20T18:59:06.660 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7030000000000003e-05 2024-09-20T18:59:06.660 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:06.660 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:06.660 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:06.661 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.661 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.661 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:117083 7.e 7:7035ec2c:::benchmark_data_smithi083_70643_object117082:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-20T18:59:06.661 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:52:21.789905+0000", 2024-09-20T18:59:06.661 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 404.80639441699998, 2024-09-20T18:59:06.661 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.1098601, 2024-09-20T18:59:06.661 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:59:06.661 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:59:06.661 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:06.661 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:59:06.661 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:59:06.661 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:59:06.661 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 117083 2024-09-20T18:59:06.661 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.661 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:59:06.661 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.661 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:59:06.662 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.789905+0000", 2024-09-20T18:59:06.662 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:06.662 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.662 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.662 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:59:06.662 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.789905+0000", 2024-09-20T18:59:06.662 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:06.662 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.662 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.662 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:59:06.662 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.789906+0000", 2024-09-20T18:59:06.662 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4109999999999999e-06 2024-09-20T18:59:06.662 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.662 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.663 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:59:06.663 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.789929+0000", 2024-09-20T18:59:06.663 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3212000000000001e-05 2024-09-20T18:59:06.663 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.663 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.663 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:59:06.663 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.789941+0000", 2024-09-20T18:59:06.663 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1103e-05 2024-09-20T18:59:06.663 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.663 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.663 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:59:06.663 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.789948+0000", 2024-09-20T18:59:06.663 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.968e-06 2024-09-20T18:59:06.663 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.663 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.664 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:59:06.664 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.790013+0000", 2024-09-20T18:59:06.664 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5518000000000005e-05 2024-09-20T18:59:06.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.664 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.664 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:59:06.664 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.790130+0000", 2024-09-20T18:59:06.664 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000116906 2024-09-20T18:59:06.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.664 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.664 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:59:06.664 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.790270+0000", 2024-09-20T18:59:06.664 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013975700000000001 2024-09-20T18:59:06.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.664 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.664 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:59:06.665 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.793553+0000", 2024-09-20T18:59:06.665 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003283589 2024-09-20T18:59:06.665 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.665 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.665 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:06.665 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.799223+0000", 2024-09-20T18:59:06.665 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0056701340000000003 2024-09-20T18:59:06.665 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.665 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.665 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:06.665 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.899580+0000", 2024-09-20T18:59:06.665 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.100356374 2024-09-20T18:59:06.665 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.665 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.665 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:59:06.666 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.899639+0000", 2024-09-20T18:59:06.666 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8990000000000003e-05 2024-09-20T18:59:06.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.666 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.666 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:59:06.666 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.899765+0000", 2024-09-20T18:59:06.666 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000126138 2024-09-20T18:59:06.666 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:06.666 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:06.666 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:06.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.666 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.666 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:117091 7.e 7:791f482a:::benchmark_data_smithi083_70643_object117090:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-20T18:59:06.666 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:52:21.794291+0000", 2024-09-20T18:59:06.667 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 404.80200870900001, 2024-09-20T18:59:06.667 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.108790281, 2024-09-20T18:59:06.667 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:59:06.667 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:59:06.667 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:06.667 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:59:06.668 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:59:06.668 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:59:06.668 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:59:06.668 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 117091 2024-09-20T18:59:06.668 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.668 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:59:06.668 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.668 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:59:06.669 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.794291+0000", 2024-09-20T18:59:06.669 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:06.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.669 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.669 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:59:06.669 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.794291+0000", 2024-09-20T18:59:06.669 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:06.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.669 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.669 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:59:06.669 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.794292+0000", 2024-09-20T18:59:06.669 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.579e-06 2024-09-20T18:59:06.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.669 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.669 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:59:06.669 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.794317+0000", 2024-09-20T18:59:06.670 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4895000000000001e-05 2024-09-20T18:59:06.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.670 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.670 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:59:06.670 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.794329+0000", 2024-09-20T18:59:06.670 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1905e-05 2024-09-20T18:59:06.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.670 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.670 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:59:06.670 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.794337+0000", 2024-09-20T18:59:06.670 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8590000000000005e-06 2024-09-20T18:59:06.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.670 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.670 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:59:06.671 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.794397+0000", 2024-09-20T18:59:06.671 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0531e-05 2024-09-20T18:59:06.671 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.671 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.671 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:59:06.671 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.794533+0000", 2024-09-20T18:59:06.671 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013557200000000001 2024-09-20T18:59:06.671 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.671 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.671 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:59:06.671 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.794680+0000", 2024-09-20T18:59:06.671 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014704799999999999 2024-09-20T18:59:06.671 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.671 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.671 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:59:06.671 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.799870+0000", 2024-09-20T18:59:06.672 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0051902720000000001 2024-09-20T18:59:06.672 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.672 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.672 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:06.672 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.804125+0000", 2024-09-20T18:59:06.672 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0042546779999999996 2024-09-20T18:59:06.672 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.672 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.672 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:06.672 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.902983+0000", 2024-09-20T18:59:06.672 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.098858370000000001 2024-09-20T18:59:06.672 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.672 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.672 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:59:06.672 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.903040+0000", 2024-09-20T18:59:06.672 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7136999999999999e-05 2024-09-20T18:59:06.673 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.673 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.673 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:59:06.673 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.903081+0000", 2024-09-20T18:59:06.673 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0435000000000002e-05 2024-09-20T18:59:06.673 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:06.673 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:06.673 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:06.673 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.673 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.673 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:187757 7.e 7:70705a0e:::benchmark_data_smithi083_70643_object64955:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-20T18:59:06.673 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:53:29.029730+0000", 2024-09-20T18:59:06.673 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 337.56656877799998, 2024-09-20T18:59:06.673 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.137882645, 2024-09-20T18:59:06.673 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:59:06.674 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:59:06.674 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:06.674 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:59:06.674 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:59:06.674 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:59:06.674 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 187757 2024-09-20T18:59:06.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.674 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:59:06.674 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.674 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:59:06.674 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.029730+0000", 2024-09-20T18:59:06.674 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:06.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.674 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.674 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:59:06.674 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.029730+0000", 2024-09-20T18:59:06.675 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:06.675 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.675 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.675 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:59:06.675 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.029732+0000", 2024-09-20T18:59:06.675 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.415e-06 2024-09-20T18:59:06.675 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.675 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.675 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:59:06.675 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.029737+0000", 2024-09-20T18:59:06.675 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0239999999999997e-06 2024-09-20T18:59:06.675 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.675 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.675 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:59:06.675 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.029748+0000", 2024-09-20T18:59:06.675 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0920999999999999e-05 2024-09-20T18:59:06.676 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.676 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.676 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:59:06.676 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.029755+0000", 2024-09-20T18:59:06.676 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1430000000000001e-06 2024-09-20T18:59:06.676 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.676 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.676 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:59:06.676 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.030815+0000", 2024-09-20T18:59:06.676 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0010603590000000001 2024-09-20T18:59:06.676 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.676 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.676 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:59:06.676 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.030930+0000", 2024-09-20T18:59:06.676 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000114171 2024-09-20T18:59:06.676 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.677 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.677 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:59:06.677 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.031080+0000", 2024-09-20T18:59:06.677 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015042000000000001 2024-09-20T18:59:06.677 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.677 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.677 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:59:06.677 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.036072+0000", 2024-09-20T18:59:06.677 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049921159999999996 2024-09-20T18:59:06.677 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.677 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.677 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:06.677 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.037324+0000", 2024-09-20T18:59:06.677 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0012515930000000001 2024-09-20T18:59:06.677 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.678 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.678 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:06.678 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045902+0000", 2024-09-20T18:59:06.678 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0085779580000000001 2024-09-20T18:59:06.678 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.678 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.678 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:59:06.678 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.167564+0000", 2024-09-20T18:59:06.678 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.121662143 2024-09-20T18:59:06.678 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.678 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.678 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:59:06.678 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.167613+0000", 2024-09-20T18:59:06.678 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9382000000000003e-05 2024-09-20T18:59:06.678 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:06.678 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:06.679 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:06.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.679 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.679 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:187765 7.e 7:7e1cec20:::benchmark_data_smithi083_70643_object64963:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-20T18:59:06.679 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:53:29.040056+0000", 2024-09-20T18:59:06.679 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 337.55624369200001, 2024-09-20T18:59:06.679 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.132718637, 2024-09-20T18:59:06.679 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:59:06.679 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:59:06.679 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:06.679 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:59:06.679 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:59:06.679 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:59:06.679 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 187765 2024-09-20T18:59:06.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.680 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:59:06.680 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.680 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:59:06.680 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.040056+0000", 2024-09-20T18:59:06.680 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:06.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.680 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.680 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:59:06.680 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.040056+0000", 2024-09-20T18:59:06.680 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:06.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.680 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.680 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:59:06.681 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.040057+0000", 2024-09-20T18:59:06.681 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.528e-06 2024-09-20T18:59:06.681 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.681 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.681 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:59:06.681 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.040063+0000", 2024-09-20T18:59:06.681 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8170000000000004e-06 2024-09-20T18:59:06.681 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.681 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.681 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:59:06.681 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.040074+0000", 2024-09-20T18:59:06.681 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1253000000000001e-05 2024-09-20T18:59:06.682 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.682 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.682 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:59:06.682 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.040081+0000", 2024-09-20T18:59:06.682 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1389999999999998e-06 2024-09-20T18:59:06.682 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.682 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.682 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:59:06.682 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.040134+0000", 2024-09-20T18:59:06.682 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2549000000000001e-05 2024-09-20T18:59:06.682 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.682 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.682 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:59:06.682 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.040246+0000", 2024-09-20T18:59:06.683 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000112271 2024-09-20T18:59:06.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.683 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.683 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:59:06.683 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.040354+0000", 2024-09-20T18:59:06.683 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010791 2024-09-20T18:59:06.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.683 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.683 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:06.683 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045867+0000", 2024-09-20T18:59:06.683 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0055125449999999998 2024-09-20T18:59:06.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.683 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.683 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:59:06.683 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.172613+0000", 2024-09-20T18:59:06.683 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.12674679599999999 2024-09-20T18:59:06.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.684 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.684 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:06.684 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.172691+0000", 2024-09-20T18:59:06.684 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7255000000000005e-05 2024-09-20T18:59:06.684 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.684 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.684 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:59:06.684 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.172735+0000", 2024-09-20T18:59:06.684 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4069000000000001e-05 2024-09-20T18:59:06.684 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.684 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.684 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:59:06.684 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.172774+0000", 2024-09-20T18:59:06.684 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9505000000000002e-05 2024-09-20T18:59:06.685 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:06.685 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:06.685 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:06.685 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.685 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.685 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:187763 7.3 e141/83 7:c71c27bb:::benchmark_data_smithi083_70643_object64961:head v 141'11587, mlcod=141'11584)", 2024-09-20T18:59:06.685 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:53:29.043149+0000", 2024-09-20T18:59:06.685 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 337.55315000799999, 2024-09-20T18:59:06.685 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.125479285, 2024-09-20T18:59:06.685 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:59:06.685 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:59:06.685 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:06.685 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:59:06.685 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.685 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:59:06.685 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043149+0000", 2024-09-20T18:59:06.686 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:06.686 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.686 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.686 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:59:06.686 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043149+0000", 2024-09-20T18:59:06.686 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:06.686 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.686 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.686 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:59:06.686 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043150+0000", 2024-09-20T18:59:06.686 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7800000000000001e-07 2024-09-20T18:59:06.686 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.686 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.686 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:59:06.686 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043154+0000", 2024-09-20T18:59:06.687 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6480000000000002e-06 2024-09-20T18:59:06.687 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.687 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.687 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:59:06.687 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043162+0000", 2024-09-20T18:59:06.687 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1810000000000004e-06 2024-09-20T18:59:06.687 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.687 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.687 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:59:06.687 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043167+0000", 2024-09-20T18:59:06.687 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4500000000000003e-06 2024-09-20T18:59:06.688 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.688 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.688 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:59:06.688 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043227+0000", 2024-09-20T18:59:06.688 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9697999999999997e-05 2024-09-20T18:59:06.688 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.688 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.688 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:59:06.688 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043243+0000", 2024-09-20T18:59:06.688 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6461999999999999e-05 2024-09-20T18:59:06.688 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.688 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.688 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:59:06.688 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.168513+0000", 2024-09-20T18:59:06.688 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.125269624 2024-09-20T18:59:06.689 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.689 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.689 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:59:06.689 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.168629+0000", 2024-09-20T18:59:06.689 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011564400000000001 2024-09-20T18:59:06.689 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:06.689 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:06.689 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:06.689 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.689 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.689 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:187773 7.e 7:7168b707:::benchmark_data_smithi083_70643_object64971:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-20T18:59:06.689 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:53:29.043782+0000", 2024-09-20T18:59:06.689 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 337.55251728000002, 2024-09-20T18:59:06.689 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.14065076300000001, 2024-09-20T18:59:06.689 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:59:06.690 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:59:06.690 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:06.690 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:59:06.690 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:59:06.690 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:59:06.690 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 187773 2024-09-20T18:59:06.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.690 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:59:06.690 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.690 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:59:06.690 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043782+0000", 2024-09-20T18:59:06.690 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:06.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.690 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.690 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:59:06.690 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043782+0000", 2024-09-20T18:59:06.691 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:06.691 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.691 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.691 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:59:06.691 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043784+0000", 2024-09-20T18:59:06.691 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5260000000000001e-06 2024-09-20T18:59:06.691 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.691 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.691 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:59:06.691 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043788+0000", 2024-09-20T18:59:06.691 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.481e-06 2024-09-20T18:59:06.691 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.691 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.691 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:59:06.691 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043798+0000", 2024-09-20T18:59:06.691 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0393e-05 2024-09-20T18:59:06.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.692 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.692 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:59:06.692 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043805+0000", 2024-09-20T18:59:06.692 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5030000000000002e-06 2024-09-20T18:59:06.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.692 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.692 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:59:06.692 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.167654+0000", 2024-09-20T18:59:06.692 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.123848659 2024-09-20T18:59:06.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.692 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.692 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:59:06.692 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.167881+0000", 2024-09-20T18:59:06.692 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000227698 2024-09-20T18:59:06.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.692 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.693 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:59:06.693 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.167981+0000", 2024-09-20T18:59:06.693 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9611999999999994e-05 2024-09-20T18:59:06.693 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.693 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.693 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:06.693 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.172817+0000", 2024-09-20T18:59:06.693 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0048360829999999997 2024-09-20T18:59:06.693 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.693 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.693 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:59:06.693 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.172828+0000", 2024-09-20T18:59:06.693 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1280999999999999e-05 2024-09-20T18:59:06.693 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.693 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.693 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:06.694 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.184218+0000", 2024-09-20T18:59:06.694 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011389882 2024-09-20T18:59:06.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.694 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.694 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:59:06.694 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.184387+0000", 2024-09-20T18:59:06.694 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000168992 2024-09-20T18:59:06.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.694 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.694 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:59:06.694 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.184433+0000", 2024-09-20T18:59:06.694 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5652999999999998e-05 2024-09-20T18:59:06.694 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:06.694 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:06.694 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:06.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.695 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.695 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:187757 7.e e141/83 ondisk, result = 0)", 2024-09-20T18:59:06.695 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:53:29.045594+0000", 2024-09-20T18:59:06.695 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 337.55070532600001, 2024-09-20T18:59:06.695 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.12203122600000001, 2024-09-20T18:59:06.695 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:59:06.695 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:59:06.695 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:59:06.695 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:59:06.695 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.695 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:59:06.695 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045594+0000", 2024-09-20T18:59:06.695 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:06.695 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.695 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.696 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:59:06.696 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045594+0000", 2024-09-20T18:59:06.696 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:06.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.696 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.696 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:59:06.696 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045595+0000", 2024-09-20T18:59:06.696 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6499999999999998e-07 2024-09-20T18:59:06.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.696 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.696 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:59:06.696 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045599+0000", 2024-09-20T18:59:06.696 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6160000000000004e-06 2024-09-20T18:59:06.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.697 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.697 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:59:06.697 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045609+0000", 2024-09-20T18:59:06.697 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2269999999999999e-06 2024-09-20T18:59:06.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.697 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.697 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:59:06.697 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045615+0000", 2024-09-20T18:59:06.697 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1349999999999997e-06 2024-09-20T18:59:06.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.697 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.697 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:59:06.697 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045892+0000", 2024-09-20T18:59:06.697 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000276724 2024-09-20T18:59:06.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.697 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.697 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:59:06.698 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045897+0000", 2024-09-20T18:59:06.698 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2970000000000003e-06 2024-09-20T18:59:06.698 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.698 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.698 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:59:06.698 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.167625+0000", 2024-09-20T18:59:06.698 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.12172836200000001 2024-09-20T18:59:06.698 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:06.698 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:06.698 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:06.698 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.698 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.698 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:187772 7.a e141/83 7:5948c6a4:::benchmark_data_smithi083_70643_object64970:head v 141'11671, mlcod=141'11669)", 2024-09-20T18:59:06.698 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:53:29.045936+0000", 2024-09-20T18:59:06.698 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 337.55036362999999, 2024-09-20T18:59:06.699 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.12458983799999999, 2024-09-20T18:59:06.699 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:59:06.699 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:59:06.699 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:06.699 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:59:06.699 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.699 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:59:06.699 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045936+0000", 2024-09-20T18:59:06.699 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:06.699 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.699 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.699 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:59:06.699 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045936+0000", 2024-09-20T18:59:06.699 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:06.699 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.699 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.699 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:59:06.700 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045936+0000", 2024-09-20T18:59:06.700 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2900000000000003e-07 2024-09-20T18:59:06.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.700 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.700 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:59:06.700 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045942+0000", 2024-09-20T18:59:06.700 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.66e-06 2024-09-20T18:59:06.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.700 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.700 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:59:06.700 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.165480+0000", 2024-09-20T18:59:06.700 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.119537582 2024-09-20T18:59:06.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.700 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.700 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:59:06.700 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.165496+0000", 2024-09-20T18:59:06.701 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6249999999999999e-05 2024-09-20T18:59:06.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.701 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.701 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:59:06.701 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.165558+0000", 2024-09-20T18:59:06.701 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2120999999999997e-05 2024-09-20T18:59:06.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.701 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.701 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:59:06.701 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.165582+0000", 2024-09-20T18:59:06.701 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4409e-05 2024-09-20T18:59:06.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.701 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.701 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:59:06.701 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.170489+0000", 2024-09-20T18:59:06.701 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049066860000000004 2024-09-20T18:59:06.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.702 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.702 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:59:06.702 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.170525+0000", 2024-09-20T18:59:06.702 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6400999999999999e-05 2024-09-20T18:59:06.702 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:06.702 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:06.702 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:06.702 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.702 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.702 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:187774 7.e 7:778445cb:::benchmark_data_smithi083_70643_object64972:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-20T18:59:06.702 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:53:29.046524+0000", 2024-09-20T18:59:06.702 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 337.549775205, 2024-09-20T18:59:06.702 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.15650777099999999, 2024-09-20T18:59:06.702 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:59:06.702 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:59:06.703 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:06.703 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:59:06.703 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:59:06.703 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:59:06.703 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 187774 2024-09-20T18:59:06.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.703 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:59:06.703 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.703 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:59:06.703 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.046524+0000", 2024-09-20T18:59:06.703 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:06.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.703 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.703 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:59:06.703 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.046524+0000", 2024-09-20T18:59:06.703 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:06.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.704 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.704 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:59:06.704 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.046526+0000", 2024-09-20T18:59:06.704 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.494e-06 2024-09-20T18:59:06.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.704 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.704 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:59:06.704 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.046531+0000", 2024-09-20T18:59:06.704 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4840000000000003e-06 2024-09-20T18:59:06.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.704 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.704 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:59:06.704 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.165656+0000", 2024-09-20T18:59:06.704 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.11912532100000001 2024-09-20T18:59:06.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.705 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.705 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:59:06.705 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.165666+0000", 2024-09-20T18:59:06.705 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1789999999999997e-06 2024-09-20T18:59:06.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.705 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.705 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:59:06.705 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.173061+0000", 2024-09-20T18:59:06.705 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0073953969999999997 2024-09-20T18:59:06.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.705 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.705 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:59:06.705 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.173154+0000", 2024-09-20T18:59:06.705 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3113000000000001e-05 2024-09-20T18:59:06.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.705 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.706 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:59:06.706 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.173250+0000", 2024-09-20T18:59:06.706 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5493000000000002e-05 2024-09-20T18:59:06.706 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.706 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.706 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:06.706 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.180360+0000", 2024-09-20T18:59:06.706 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0071109220000000004 2024-09-20T18:59:06.707 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.707 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.707 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:59:06.707 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.188345+0000", 2024-09-20T18:59:06.707 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0079849840000000005 2024-09-20T18:59:06.707 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.707 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.707 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:06.707 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.202813+0000", 2024-09-20T18:59:06.707 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.014467647 2024-09-20T18:59:06.707 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.707 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.707 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:59:06.707 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.202985+0000", 2024-09-20T18:59:06.708 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017191999999999999 2024-09-20T18:59:06.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.708 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.708 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:59:06.708 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.203032+0000", 2024-09-20T18:59:06.708 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6817000000000003e-05 2024-09-20T18:59:06.708 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:06.708 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:06.708 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:06.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.708 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.708 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:187765 7.e e141/83 ondisk, result = 0)", 2024-09-20T18:59:06.708 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:53:29.046987+0000", 2024-09-20T18:59:06.708 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 337.54931220600002, 2024-09-20T18:59:06.708 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.12579221700000001, 2024-09-20T18:59:06.708 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:59:06.709 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:59:06.709 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:59:06.709 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:59:06.709 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.709 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:59:06.709 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.046987+0000", 2024-09-20T18:59:06.709 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:06.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.709 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.709 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:59:06.709 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.046987+0000", 2024-09-20T18:59:06.709 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:06.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.709 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.709 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:59:06.710 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.046988+0000", 2024-09-20T18:59:06.710 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6499999999999999e-07 2024-09-20T18:59:06.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.710 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.710 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:59:06.710 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.046992+0000", 2024-09-20T18:59:06.710 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9650000000000002e-06 2024-09-20T18:59:06.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.710 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.710 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:59:06.710 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.165597+0000", 2024-09-20T18:59:06.710 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.118605087 2024-09-20T18:59:06.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.710 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.710 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:59:06.710 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.165603+0000", 2024-09-20T18:59:06.711 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0100000000000001e-06 2024-09-20T18:59:06.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.711 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.711 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:59:06.711 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.172679+0000", 2024-09-20T18:59:06.711 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0070759810000000003 2024-09-20T18:59:06.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.711 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.711 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:59:06.711 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.172686+0000", 2024-09-20T18:59:06.711 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9959999999999996e-06 2024-09-20T18:59:06.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.711 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.711 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:59:06.711 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.172779+0000", 2024-09-20T18:59:06.712 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3512999999999997e-05 2024-09-20T18:59:06.712 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:06.712 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:06.712 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:06.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.712 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.712 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.24955.0:218218 8.6 e233/182 8:6660910c:::benchmark_data_smithi083_97422_object102127:head v 233'13570, mlcod=210'8949)", 2024-09-20T18:59:06.712 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:58:02.122025+0000", 2024-09-20T18:59:06.712 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 64.474274059999999, 2024-09-20T18:59:06.712 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.112690573, 2024-09-20T18:59:06.712 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:59:06.712 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:59:06.712 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:06.712 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:59:06.712 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.713 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:59:06.713 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.122025+0000", 2024-09-20T18:59:06.713 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:06.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.713 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.713 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:59:06.713 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.122025+0000", 2024-09-20T18:59:06.713 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:06.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.713 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.713 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:59:06.713 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.122026+0000", 2024-09-20T18:59:06.713 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2399999999999996e-07 2024-09-20T18:59:06.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.713 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.714 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:59:06.714 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.122033+0000", 2024-09-20T18:59:06.714 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.562e-06 2024-09-20T18:59:06.714 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.714 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.714 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:59:06.714 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.214763+0000", 2024-09-20T18:59:06.714 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.092730564000000001 2024-09-20T18:59:06.714 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.714 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.714 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:59:06.714 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.214772+0000", 2024-09-20T18:59:06.714 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7979999999999995e-06 2024-09-20T18:59:06.714 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.714 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.715 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:59:06.715 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.214856+0000", 2024-09-20T18:59:06.715 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4146000000000004e-05 2024-09-20T18:59:06.715 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.715 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.715 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:59:06.715 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.214878+0000", 2024-09-20T18:59:06.715 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1722e-05 2024-09-20T18:59:06.715 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.715 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.715 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:59:06.715 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.234612+0000", 2024-09-20T18:59:06.715 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019734267 2024-09-20T18:59:06.715 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:06.715 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:06.715 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:59:06.716 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.234716+0000", 2024-09-20T18:59:06.716 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010359 2024-09-20T18:59:06.716 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:06.716 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:06.716 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:06.716 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:06.716 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:06.716 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T18:59:06.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:06.746 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:06.746 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:59:06.846 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:06.885 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:06.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:06.887 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:06.987 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:59:07.021 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:07.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:07.022 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:59:07.122 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:07.156 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:07.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:07.157 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:07.250 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 27 16 51826 51810 119.912 79.8125 0.0319456 0.00833365 2024-09-20T18:59:07.258 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:07.267 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:59:06 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:59:07.267 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:07 smithi142 ceph-mon[32508]: 4.6 scrub starts 2024-09-20T18:59:07.267 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:07 smithi142 ceph-mon[32508]: 4.6 scrub ok 2024-09-20T18:59:07.267 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:07 smithi142 ceph-mon[32508]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T18:59:07.267 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:07 smithi142 ceph-mon[32508]: Cluster is now healthy 2024-09-20T18:59:07.267 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:07 smithi142 ceph-mon[32508]: 4.2 scrub starts 2024-09-20T18:59:07.267 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:07 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1770169890' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T18:59:07.267 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:07 smithi142 ceph-mon[32508]: 4.2 scrub ok 2024-09-20T18:59:07.267 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:07 smithi142 ceph-mon[32508]: osdmap e262: 8 total, 8 up, 8 in 2024-09-20T18:59:07.268 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:07 smithi142 ceph-mon[32508]: 3.14 scrub starts 2024-09-20T18:59:07.268 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:07 smithi142 ceph-mon[32508]: 3.14 scrub ok 2024-09-20T18:59:07.291 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:07.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:07.292 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:07.393 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:07.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:07.429 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:07.429 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:07.530 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:07.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:07 smithi083 ceph-mon[29421]: 4.6 scrub starts 2024-09-20T18:59:07.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:07 smithi083 ceph-mon[29421]: 4.6 scrub ok 2024-09-20T18:59:07.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:07 smithi083 ceph-mon[29421]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T18:59:07.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:07 smithi083 ceph-mon[29421]: Cluster is now healthy 2024-09-20T18:59:07.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:07 smithi083 ceph-mon[29421]: 4.2 scrub starts 2024-09-20T18:59:07.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:07 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1770169890' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T18:59:07.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:07 smithi083 ceph-mon[29421]: 4.2 scrub ok 2024-09-20T18:59:07.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:07 smithi083 ceph-mon[29421]: osdmap e262: 8 total, 8 up, 8 in 2024-09-20T18:59:07.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:07 smithi083 ceph-mon[29421]: 3.14 scrub starts 2024-09-20T18:59:07.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:07 smithi083 ceph-mon[29421]: 3.14 scrub ok 2024-09-20T18:59:07.548 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:07 smithi083 ceph-mon[36260]: 4.6 scrub starts 2024-09-20T18:59:07.548 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:07 smithi083 ceph-mon[36260]: 4.6 scrub ok 2024-09-20T18:59:07.548 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:07 smithi083 ceph-mon[36260]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T18:59:07.548 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:07 smithi083 ceph-mon[36260]: Cluster is now healthy 2024-09-20T18:59:07.548 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:07 smithi083 ceph-mon[36260]: 4.2 scrub starts 2024-09-20T18:59:07.548 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:07 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1770169890' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T18:59:07.548 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:07 smithi083 ceph-mon[36260]: 4.2 scrub ok 2024-09-20T18:59:07.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:07 smithi083 ceph-mon[36260]: osdmap e262: 8 total, 8 up, 8 in 2024-09-20T18:59:07.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:07 smithi083 ceph-mon[36260]: 3.14 scrub starts 2024-09-20T18:59:07.549 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:07 smithi083 ceph-mon[36260]: 3.14 scrub ok 2024-09-20T18:59:07.561 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:07.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:07.562 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:07.663 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:07.697 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:07.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:07.698 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:07.798 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:59:07.831 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:07.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:07.832 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:59:07.933 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:59:07.967 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:07.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:07.969 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:59:08.069 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:08.130 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:08.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:08.131 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:08.142 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-20T18:59:08.232 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:08.250 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 28 16 53110 53094 118.495 80.25 0.00758149 0.00843303 2024-09-20T18:59:08.262 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:08.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:08.263 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:08.364 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:08.389 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:08.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:08.390 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:08.410 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:08 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:59:08.410 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:08 smithi083 ceph-mon[29421]: pgmap v457: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 3.1 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 852 B/s rd, 113 MiB/s wr, 1.80k op/s; 6432/150951 objects misplaced (4.261%); 267 KiB/s, 12 objects/s recovering 2024-09-20T18:59:08.410 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:08 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:59:08.410 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:08 smithi083 ceph-mon[36260]: pgmap v457: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 3.1 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 852 B/s rd, 113 MiB/s wr, 1.80k op/s; 6432/150951 objects misplaced (4.261%); 267 KiB/s, 12 objects/s recovering 2024-09-20T18:59:08.491 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:59:08.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:08 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:59:08.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:08 smithi142 ceph-mon[32508]: pgmap v457: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 3.1 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 852 B/s rd, 113 MiB/s wr, 1.80k op/s; 6432/150951 objects misplaced (4.261%); 267 KiB/s, 12 objects/s recovering 2024-09-20T18:59:08.520 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:08.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:08.522 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:59:08.622 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:08.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:08.671 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:08.671 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:08.750 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:59:08.751 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:59:08.762 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:59:08.771 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:59:08.800 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:59:08.801 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:59:08.834 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:08.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:08.835 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:59:08.836 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:59:08.836 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:59:08.883 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:59:08.883 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:59:08.922 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:59:08.923 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:59:08.936 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:59:08.954 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:59:08.955 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:59:08.985 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:08.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:08.986 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:59:08.988 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:59:08.988 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:59:09.022 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:59:09.022 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:59:09.087 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:09.125 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:09.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:09.126 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:09.226 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:59:09.250 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 29 16 54009 53993 116.346 56.1875 0.0230195 0.00858846 2024-09-20T18:59:09.255 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:09.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:09.257 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:59:09.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:09 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2927215933' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:59:09.297 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:09 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2927215933' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:59:09.326 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T18:59:09.357 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:09.441 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:09.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:09.442 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:09.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:09 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2927215933' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:59:09.543 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:59:09.569 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:09.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:09.570 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:59:09.671 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:09.696 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:09.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:09.697 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:09.798 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:59:09.824 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:09.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:09.825 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:59:09.926 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:09.964 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 7, 6] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-20T18:59:09.964 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T18:59:09.964 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T18:59:09.982 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:09.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:09.983 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:10.084 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:10.113 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:10.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:10.114 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:10.215 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:59:10.248 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:10.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:10.250 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:59:10.250 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 30 16 55043 55027 114.622 64.625 0.00776565 0.00871656 2024-09-20T18:59:10.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:10 smithi083 ceph-mon[29421]: pgmap v458: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 3.1 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 852 B/s rd, 49 MiB/s wr, 788 op/s; 9445/153303 objects misplaced (6.161%); 672 KiB/s, 18 objects/s recovering 2024-09-20T18:59:10.297 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:10 smithi083 ceph-mon[36260]: pgmap v458: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 3.1 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 852 B/s rd, 49 MiB/s wr, 788 op/s; 9445/153303 objects misplaced (6.161%); 672 KiB/s, 18 objects/s recovering 2024-09-20T18:59:10.351 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:59:10.391 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:10.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:10.393 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:59:10.418 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:10 smithi142 ceph-mon[32508]: pgmap v458: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 3.1 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 852 B/s rd, 49 MiB/s wr, 788 op/s; 9445/153303 objects misplaced (6.161%); 672 KiB/s, 18 objects/s recovering 2024-09-20T18:59:10.493 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:10.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:10.531 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:10.531 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:10.580 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T18:59:10.580 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:59:10.632 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:10.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:10.672 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:10.673 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:10.773 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:10.811 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:10.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:10.812 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:10.913 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:10.956 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:10.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:10.957 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:11.047 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:59:10 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:59:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:59:11.058 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:59:11.092 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:11.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:11.093 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:59:11.194 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:11.230 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:11.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:11.231 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:11.250 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 31 16 56234 56218 113.326 74.4375 0.00534451 0.00881764 2024-09-20T18:59:11.332 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:11.366 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:11.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:11.367 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:11.376 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:11 smithi083 ceph-mon[29421]: 4.c scrub starts 2024-09-20T18:59:11.377 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:11 smithi083 ceph-mon[29421]: 4.c scrub ok 2024-09-20T18:59:11.377 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:11 smithi083 ceph-mon[29421]: osdmap e263: 8 total, 8 up, 8 in 2024-09-20T18:59:11.377 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:11 smithi083 ceph-mon[36260]: 4.c scrub starts 2024-09-20T18:59:11.377 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:11 smithi083 ceph-mon[36260]: 4.c scrub ok 2024-09-20T18:59:11.377 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:11 smithi083 ceph-mon[36260]: osdmap e263: 8 total, 8 up, 8 in 2024-09-20T18:59:11.426 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_blocked_ops 2024-09-20T18:59:11.438 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:59:11.468 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:11.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:11 smithi142 ceph-mon[32508]: 4.c scrub starts 2024-09-20T18:59:11.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:11 smithi142 ceph-mon[32508]: 4.c scrub ok 2024-09-20T18:59:11.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:11 smithi142 ceph-mon[32508]: osdmap e263: 8 total, 8 up, 8 in 2024-09-20T18:59:11.538 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:59:11.540 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:11.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:11.541 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:11.642 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:11.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:11.693 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:11.693 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:11.794 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:59:11.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:11.846 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:11.846 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:59:11.944 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T18:59:11.947 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:59:11.982 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:11.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:11.983 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:59:12.084 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:12.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:12.135 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:12.135 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:12.221 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:12 smithi083 ceph-mon[29421]: pgmap v460: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 3.2 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 1.7 KiB/s rd, 60 MiB/s wr, 956 op/s; 9445/156303 objects misplaced (6.043%); 672 KiB/s, 18 objects/s recovering 2024-09-20T18:59:12.221 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:12 smithi083 ceph-mon[36260]: pgmap v460: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 3.2 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 1.7 KiB/s rd, 60 MiB/s wr, 956 op/s; 9445/156303 objects misplaced (6.043%); 672 KiB/s, 18 objects/s recovering 2024-09-20T18:59:12.235 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:59:12.251 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 32 16 57231 57215 111.731 62.3125 0.00663606 0.0089426 2024-09-20T18:59:12.265 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:12.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:12.266 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:59:12.366 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:59:12.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:12.418 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:12.418 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:59:12.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:12 smithi142 ceph-mon[32508]: pgmap v460: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 3.2 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 1.7 KiB/s rd, 60 MiB/s wr, 956 op/s; 9445/156303 objects misplaced (6.043%); 672 KiB/s, 18 objects/s recovering 2024-09-20T18:59:12.518 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:12.589 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:12.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:12.591 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:12.615 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:59:12.681 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:59:12.692 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:59:12.717 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:12.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:12.718 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:59:12.819 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:59:12.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:12.863 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:12.863 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:59:12.940 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:59:12.943 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-20T18:59:12.964 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:59:12.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:12.992 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:12.993 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:59:13.093 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:59:13.125 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:13.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:13.127 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:59:13.142 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-20T18:59:13.228 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:13.251 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 33 16 58220 58204 110.218 61.8125 0.00409559 0.00906796 2024-09-20T18:59:13.257 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:13.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:13.258 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:13.296 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:13 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1901264883' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:59:13.297 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:13 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1901264883' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:59:13.359 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:13.387 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:13.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:13.389 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:13.489 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:59:13.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:13 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1901264883' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T18:59:13.522 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:13.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:13.523 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:59:13.624 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:59:13.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:13.668 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:13.668 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:59:13.769 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:59:13.802 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:13.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:13.804 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:59:13.904 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:13.939 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:13.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:13.940 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:14.041 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:14.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:14.096 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:14.096 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:14.197 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:14.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:14.236 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:14.236 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:14.251 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 34 16 59424 59408 109.189 75.25 0.00525668 0.00915237 2024-09-20T18:59:14.337 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:14.376 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:14.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:14.377 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:14.393 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:59:14.394 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:59:14.394 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:14 smithi083 ceph-mon[29421]: from='client.15642 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:59:14.394 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:14 smithi083 ceph-mon[29421]: pgmap v461: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 3.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.3 KiB/s rd, 80 MiB/s wr, 1.28k op/s; 8955/169626 objects misplaced (5.279%); 4.4 MiB/s, 76 objects/s recovering 2024-09-20T18:59:14.394 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:14 smithi083 ceph-mon[29421]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:59:14.394 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:14 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1901264883' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T18:59:14.394 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:14 smithi083 ceph-mon[29421]: osdmap e264: 8 total, 8 up, 8 in 2024-09-20T18:59:14.394 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:14 smithi083 ceph-mon[36260]: from='client.15642 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:59:14.395 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:14 smithi083 ceph-mon[36260]: pgmap v461: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 3.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.3 KiB/s rd, 80 MiB/s wr, 1.28k op/s; 8955/169626 objects misplaced (5.279%); 4.4 MiB/s, 76 objects/s recovering 2024-09-20T18:59:14.395 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:14 smithi083 ceph-mon[36260]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:59:14.395 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:14 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1901264883' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T18:59:14.395 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:14 smithi083 ceph-mon[36260]: osdmap e264: 8 total, 8 up, 8 in 2024-09-20T18:59:14.435 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:59:14.436 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:59:14.470 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:59:14.470 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:59:14.478 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:14.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:14.513 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:14.514 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:14.514 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:59:14.514 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:59:14.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:14 smithi142 ceph-mon[32508]: from='client.15642 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:59:14.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:14 smithi142 ceph-mon[32508]: pgmap v461: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 3.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.3 KiB/s rd, 80 MiB/s wr, 1.28k op/s; 8955/169626 objects misplaced (5.279%); 4.4 MiB/s, 76 objects/s recovering 2024-09-20T18:59:14.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:14 smithi142 ceph-mon[32508]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:59:14.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:14 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1901264883' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T18:59:14.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:14 smithi142 ceph-mon[32508]: osdmap e264: 8 total, 8 up, 8 in 2024-09-20T18:59:14.546 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:59:14.547 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:59:14.579 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:59:14.579 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:59:14.611 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:59:14.611 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:59:14.615 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:14.642 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:14.643 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:59:14.643 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:59:14.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:14.644 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:14.745 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:14.772 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:14.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:14.774 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:14.875 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:14.897 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:14.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:14.898 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:14.999 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:15.034 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:15.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:15.035 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:15.043 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T18:59:15.044 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:59:15.136 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:59:15.164 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:15.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:15.166 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:59:15.251 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 35 15 60577 60562 108.13 72.125 0.00666844 0.00924324 2024-09-20T18:59:15.266 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:59:15.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:15 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2725551388' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:59:15.297 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:15 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2725551388' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:59:15.298 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:15.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:15.299 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:59:15.400 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:15.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:15.434 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:15.434 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:15.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:15 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2725551388' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:59:15.535 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:15.573 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:15.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:15.629 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:15.730 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:15.759 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:15.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:15.760 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:15.828 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":460,"stamp":"2024-09-20T18:59:10.983649+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3402045979,"num_objects":52101,"num_object_clones":0,"num_object_copies":156303,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":9445,"num_objects_unfound":0,"num_objects_dirty":52101,"num_whiteouts":0,"num_read":1404,"num_read_kb":1261,"num_write":52451,"num_write_kb":3323401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":5313202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52462,"ondisk_log_size":52462,"up":444,"acting":441,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":490,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":13307188,"kb_used_data":10662872,"kb_used_omap":12,"kb_used_meta":2644147,"kb_avail":736654028,"statfs":{"total":767960285184,"available":754333724672,"internally_reserved":0,"allocated":10918780928,"data_stored":10913729940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":2707607088},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,4,4,4],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":375586816,"num_objects":5731,"num_object_clones":0,"num_object_copies":17193,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":9445,"num_objects_unfound":0,"num_objects_dirty":5731,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":5731,"num_write_kb":366784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":113,"num_bytes_recovered":4130537,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.004609"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":444,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.068043+0000","last_change":"2024-09-20T18:55:28.037494+0000","last_active":"2024-09-20T18:59:06.068043+0000","last_peered":"2024-09-20T18:59:06.068043+0000","last_clean":"2024-09-20T18:59:06.068043+0000","last_became_active":"2024-09-20T18:49:45.943201+0000","last_became_peered":"2024-09-20T18:49:45.943201+0000","last_unstale":"2024-09-20T18:59:06.068043+0000","last_undegraded":"2024-09-20T18:59:06.068043+0000","last_fullsized":"2024-09-20T18:59:06.068043+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:28.037449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:28.037449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":509,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.752415+0000","last_change":"2024-09-20T18:59:05.094771+0000","last_active":"2024-09-20T18:59:10.752415+0000","last_peered":"2024-09-20T18:59:10.752415+0000","last_clean":"2024-09-20T18:59:10.752415+0000","last_became_active":"2024-09-20T18:59:05.076159+0000","last_became_peered":"2024-09-20T18:59:05.076159+0000","last_unstale":"2024-09-20T18:59:10.752415+0000","last_undegraded":"2024-09-20T18:59:10.752415+0000","last_fullsized":"2024-09-20T18:59:10.752415+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:50.231620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:50.231620+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":442,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.067164+0000","last_change":"2024-09-20T18:57:49.016078+0000","last_active":"2024-09-20T18:59:06.067164+0000","last_peered":"2024-09-20T18:59:06.067164+0000","last_clean":"2024-09-20T18:59:06.067164+0000","last_became_active":"2024-09-20T18:49:49.959514+0000","last_became_peered":"2024-09-20T18:49:49.959514+0000","last_unstale":"2024-09-20T18:59:06.067164+0000","last_undegraded":"2024-09-20T18:59:06.067164+0000","last_fullsized":"2024-09-20T18:59:06.067164+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:49.016006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:49.016006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:13.297987+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":442,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.753310+0000","last_change":"2024-09-20T18:58:02.084435+0000","last_active":"2024-09-20T18:59:10.753310+0000","last_peered":"2024-09-20T18:59:10.753310+0000","last_clean":"2024-09-20T18:59:10.753310+0000","last_became_active":"2024-09-20T18:49:51.969421+0000","last_became_peered":"2024-09-20T18:49:51.969421+0000","last_unstale":"2024-09-20T18:59:10.753310+0000","last_undegraded":"2024-09-20T18:59:10.753310+0000","last_fullsized":"2024-09-20T18:59:10.753310+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:02.084375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:02.084375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:20.422560+0000","scrub_duration":197,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":444,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.071915+0000","last_change":"2024-09-20T18:59:05.081431+0000","last_active":"2024-09-20T18:59:06.071915+0000","last_peered":"2024-09-20T18:59:06.071915+0000","last_clean":"2024-09-20T18:59:06.071915+0000","last_became_active":"2024-09-20T18:59:05.081081+0000","last_became_peered":"2024-09-20T18:59:05.081081+0000","last_unstale":"2024-09-20T18:59:06.071915+0000","last_undegraded":"2024-09-20T18:59:06.071915+0000","last_fullsized":"2024-09-20T18:59:06.071915+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.216188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.216188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":457,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.070995+0000","last_change":"2024-09-20T18:59:05.073093+0000","last_active":"2024-09-20T18:59:06.070995+0000","last_peered":"2024-09-20T18:59:06.070995+0000","last_clean":"2024-09-20T18:59:06.070995+0000","last_became_active":"2024-09-20T18:59:05.072675+0000","last_became_peered":"2024-09-20T18:59:05.072675+0000","last_unstale":"2024-09-20T18:59:06.070995+0000","last_undegraded":"2024-09-20T18:59:06.070995+0000","last_fullsized":"2024-09-20T18:59:06.070995+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:03.055929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:59:03.055929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:26.275543+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":463,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.070982+0000","last_change":"2024-09-20T18:58:57.097877+0000","last_active":"2024-09-20T18:59:06.070982+0000","last_peered":"2024-09-20T18:59:06.070982+0000","last_clean":"2024-09-20T18:59:06.070982+0000","last_became_active":"2024-09-20T18:49:47.952150+0000","last_became_peered":"2024-09-20T18:49:47.952150+0000","last_unstale":"2024-09-20T18:59:06.070982+0000","last_undegraded":"2024-09-20T18:59:06.070982+0000","last_fullsized":"2024-09-20T18:59:06.070982+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:58:57.097766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:57.097766+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:06.978089+0000","scrub_duration":38,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":452,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.752327+0000","last_change":"2024-09-20T18:59:05.077076+0000","last_active":"2024-09-20T18:59:10.752327+0000","last_peered":"2024-09-20T18:59:10.752327+0000","last_clean":"2024-09-20T18:59:10.752327+0000","last_became_active":"2024-09-20T18:59:05.076607+0000","last_became_peered":"2024-09-20T18:59:05.076607+0000","last_unstale":"2024-09-20T18:59:10.752327+0000","last_undegraded":"2024-09-20T18:59:10.752327+0000","last_fullsized":"2024-09-20T18:59:10.752327+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:06.846018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:06.846018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:21.441744+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"59'1","reported_seq":484,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-20T18:59:05.111405+0000","last_change":"2024-09-20T18:59:05.111316+0000","last_active":"2024-09-20T18:59:05.111405+0000","last_peered":"2024-09-20T18:59:05.111405+0000","last_clean":"2024-09-20T18:59:05.111405+0000","last_became_active":"2024-09-20T18:59:05.090056+0000","last_became_peered":"2024-09-20T18:59:05.090056+0000","last_unstale":"2024-09-20T18:59:05.111405+0000","last_undegraded":"2024-09-20T18:59:05.111405+0000","last_fullsized":"2024-09-20T18:59:05.111405+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:58:09.472360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:09.472360+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:24.863556+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":466,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.753617+0000","last_change":"2024-09-20T18:58:29.671959+0000","last_active":"2024-09-20T18:59:10.753617+0000","last_peered":"2024-09-20T18:59:10.753617+0000","last_clean":"2024-09-20T18:59:10.753617+0000","last_became_active":"2024-09-20T18:49:47.947448+0000","last_became_peered":"2024-09-20T18:49:47.947448+0000","last_unstale":"2024-09-20T18:59:10.753617+0000","last_undegraded":"2024-09-20T18:59:10.753617+0000","last_fullsized":"2024-09-20T18:59:10.753617+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:58:29.671887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:29.671887+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:55.089337+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":440,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-20T18:59:05.078782+0000","last_change":"2024-09-20T18:58:25.505818+0000","last_active":"2024-09-20T18:59:05.078782+0000","last_peered":"2024-09-20T18:59:05.078782+0000","last_clean":"2024-09-20T18:59:05.078782+0000","last_became_active":"2024-09-20T18:49:49.961188+0000","last_became_peered":"2024-09-20T18:49:49.961188+0000","last_unstale":"2024-09-20T18:59:05.078782+0000","last_undegraded":"2024-09-20T18:59:05.078782+0000","last_fullsized":"2024-09-20T18:59:05.078782+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:25.505757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:25.505757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:26.562790+0000","scrub_duration":45,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":415,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.067323+0000","last_change":"2024-09-20T18:58:21.861893+0000","last_active":"2024-09-20T18:59:06.067323+0000","last_peered":"2024-09-20T18:59:06.067323+0000","last_clean":"2024-09-20T18:59:06.067323+0000","last_became_active":"2024-09-20T18:49:51.966715+0000","last_became_peered":"2024-09-20T18:49:51.966715+0000","last_unstale":"2024-09-20T18:59:06.067323+0000","last_undegraded":"2024-09-20T18:59:06.067323+0000","last_fullsized":"2024-09-20T18:59:06.067323+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:21.861836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:21.861836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:51.627461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":432,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-20T18:59:05.079043+0000","last_change":"2024-09-20T18:55:15.252839+0000","last_active":"2024-09-20T18:59:05.079043+0000","last_peered":"2024-09-20T18:59:05.079043+0000","last_clean":"2024-09-20T18:59:05.079043+0000","last_became_active":"2024-09-20T18:49:45.944703+0000","last_became_peered":"2024-09-20T18:49:45.944703+0000","last_unstale":"2024-09-20T18:59:05.079043+0000","last_undegraded":"2024-09-20T18:59:05.079043+0000","last_fullsized":"2024-09-20T18:59:05.079043+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:15.252782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:15.252782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:15.451958+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":470,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.752469+0000","last_change":"2024-09-20T18:58:33.660957+0000","last_active":"2024-09-20T18:59:10.752469+0000","last_peered":"2024-09-20T18:59:10.752469+0000","last_clean":"2024-09-20T18:59:10.752469+0000","last_became_active":"2024-09-20T18:51:23.813399+0000","last_became_peered":"2024-09-20T18:51:23.813399+0000","last_unstale":"2024-09-20T18:59:10.752469+0000","last_undegraded":"2024-09-20T18:59:10.752469+0000","last_fullsized":"2024-09-20T18:59:10.752469+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-20T18:58:33.660531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:33.660531+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:38.572897+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":437,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.761483+0000","last_change":"2024-09-20T18:58:32.526595+0000","last_active":"2024-09-20T18:59:10.761483+0000","last_peered":"2024-09-20T18:59:10.761483+0000","last_clean":"2024-09-20T18:59:10.761483+0000","last_became_active":"2024-09-20T18:49:49.959696+0000","last_became_peered":"2024-09-20T18:49:49.959696+0000","last_unstale":"2024-09-20T18:59:10.761483+0000","last_undegraded":"2024-09-20T18:59:10.761483+0000","last_fullsized":"2024-09-20T18:59:10.761483+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.526527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.526527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:36.077135+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":436,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-20T18:59:05.077106+0000","last_change":"2024-09-20T18:58:26.465305+0000","last_active":"2024-09-20T18:59:05.077106+0000","last_peered":"2024-09-20T18:59:05.077106+0000","last_clean":"2024-09-20T18:59:05.077106+0000","last_became_active":"2024-09-20T18:49:51.970663+0000","last_became_peered":"2024-09-20T18:49:51.970663+0000","last_unstale":"2024-09-20T18:59:05.077106+0000","last_undegraded":"2024-09-20T18:59:05.077106+0000","last_fullsized":"2024-09-20T18:59:05.077106+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:26.465264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:26.465264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:51.798999+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":449,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.753919+0000","last_change":"2024-09-20T18:59:05.079781+0000","last_active":"2024-09-20T18:59:10.753919+0000","last_peered":"2024-09-20T18:59:10.753919+0000","last_clean":"2024-09-20T18:59:10.753919+0000","last_became_active":"2024-09-20T18:59:05.079278+0000","last_became_peered":"2024-09-20T18:59:05.079278+0000","last_unstale":"2024-09-20T18:59:10.753919+0000","last_undegraded":"2024-09-20T18:59:10.753919+0000","last_fullsized":"2024-09-20T18:59:10.753919+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:00.906652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:00.906652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:12.887584+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":443,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.087780+0000","last_change":"2024-09-20T18:58:34.996936+0000","last_active":"2024-09-20T18:59:06.087780+0000","last_peered":"2024-09-20T18:59:06.087780+0000","last_clean":"2024-09-20T18:59:06.087780+0000","last_became_active":"2024-09-20T18:49:45.950480+0000","last_became_peered":"2024-09-20T18:49:45.950480+0000","last_unstale":"2024-09-20T18:59:06.087780+0000","last_undegraded":"2024-09-20T18:59:06.087780+0000","last_fullsized":"2024-09-20T18:59:06.087780+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:34.996893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:34.996893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:50.843422+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":466,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.065836+0000","last_change":"2024-09-20T18:58:28.253192+0000","last_active":"2024-09-20T18:59:06.065836+0000","last_peered":"2024-09-20T18:59:06.065836+0000","last_clean":"2024-09-20T18:59:06.065836+0000","last_became_active":"2024-09-20T18:49:47.950939+0000","last_became_peered":"2024-09-20T18:49:47.950939+0000","last_unstale":"2024-09-20T18:59:06.065836+0000","last_undegraded":"2024-09-20T18:59:06.065836+0000","last_fullsized":"2024-09-20T18:59:06.065836+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:58:28.253141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:28.253141+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:55.772150+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":426,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.067884+0000","last_change":"2024-09-20T18:57:14.941518+0000","last_active":"2024-09-20T18:59:06.067884+0000","last_peered":"2024-09-20T18:59:06.067884+0000","last_clean":"2024-09-20T18:59:06.067884+0000","last_became_active":"2024-09-20T18:49:49.958518+0000","last_became_peered":"2024-09-20T18:49:49.958518+0000","last_unstale":"2024-09-20T18:59:06.067884+0000","last_undegraded":"2024-09-20T18:59:06.067884+0000","last_fullsized":"2024-09-20T18:59:06.067884+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:14.941458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:14.941458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:43.066178+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":445,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.071251+0000","last_change":"2024-09-20T18:59:05.084868+0000","last_active":"2024-09-20T18:59:06.071251+0000","last_peered":"2024-09-20T18:59:06.071251+0000","last_clean":"2024-09-20T18:59:06.071251+0000","last_became_active":"2024-09-20T18:59:05.084496+0000","last_became_peered":"2024-09-20T18:59:05.084496+0000","last_unstale":"2024-09-20T18:59:06.071251+0000","last_undegraded":"2024-09-20T18:59:06.071251+0000","last_fullsized":"2024-09-20T18:59:06.071251+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:35.125451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:35.125451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:41.218529+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"59'1","reported_seq":460,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.753703+0000","last_change":"2024-09-20T18:59:03.397298+0000","last_active":"2024-09-20T18:59:10.753703+0000","last_peered":"2024-09-20T18:59:10.753703+0000","last_clean":"2024-09-20T18:59:10.753703+0000","last_became_active":"2024-09-20T18:49:45.940100+0000","last_became_peered":"2024-09-20T18:49:45.940100+0000","last_unstale":"2024-09-20T18:59:10.753703+0000","last_undegraded":"2024-09-20T18:59:10.753703+0000","last_fullsized":"2024-09-20T18:59:10.753703+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:59:03.397225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:59:03.397225+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:17.466210+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":445,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.070867+0000","last_change":"2024-09-20T18:57:09.819080+0000","last_active":"2024-09-20T18:59:06.070867+0000","last_peered":"2024-09-20T18:59:06.070867+0000","last_clean":"2024-09-20T18:59:06.070867+0000","last_became_active":"2024-09-20T18:49:47.950909+0000","last_became_peered":"2024-09-20T18:49:47.950909+0000","last_unstale":"2024-09-20T18:59:06.070867+0000","last_undegraded":"2024-09-20T18:59:06.070867+0000","last_fullsized":"2024-09-20T18:59:06.070867+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:57:09.818999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:09.818999+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:51.024316+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":438,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.752953+0000","last_change":"2024-09-20T18:54:44.239273+0000","last_active":"2024-09-20T18:59:10.752953+0000","last_peered":"2024-09-20T18:59:10.752953+0000","last_clean":"2024-09-20T18:59:10.752953+0000","last_became_active":"2024-09-20T18:49:49.959102+0000","last_became_peered":"2024-09-20T18:49:49.959102+0000","last_unstale":"2024-09-20T18:59:10.752953+0000","last_undegraded":"2024-09-20T18:59:10.752953+0000","last_fullsized":"2024-09-20T18:59:10.752953+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:44.239226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.239226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:17.660601+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":434,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.761948+0000","last_change":"2024-09-20T18:58:25.501434+0000","last_active":"2024-09-20T18:59:10.761948+0000","last_peered":"2024-09-20T18:59:10.761948+0000","last_clean":"2024-09-20T18:59:10.761948+0000","last_became_active":"2024-09-20T18:49:51.970058+0000","last_became_peered":"2024-09-20T18:49:51.970058+0000","last_unstale":"2024-09-20T18:59:10.761948+0000","last_undegraded":"2024-09-20T18:59:10.761948+0000","last_fullsized":"2024-09-20T18:59:10.761948+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T18:58:25.501384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:25.501384+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:36.383692+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":435,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.067654+0000","last_change":"2024-09-20T18:57:18.032140+0000","last_active":"2024-09-20T18:59:06.067654+0000","last_peered":"2024-09-20T18:59:06.067654+0000","last_clean":"2024-09-20T18:59:06.067654+0000","last_became_active":"2024-09-20T18:49:45.933078+0000","last_became_peered":"2024-09-20T18:49:45.933078+0000","last_unstale":"2024-09-20T18:59:06.067654+0000","last_undegraded":"2024-09-20T18:59:06.067654+0000","last_fullsized":"2024-09-20T18:59:06.067654+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:57:18.032067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:18.032067+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:47.054122+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":463,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.067536+0000","last_change":"2024-09-20T18:59:02.024076+0000","last_active":"2024-09-20T18:59:06.067536+0000","last_peered":"2024-09-20T18:59:06.067536+0000","last_clean":"2024-09-20T18:59:06.067536+0000","last_became_active":"2024-09-20T18:49:48.500809+0000","last_became_peered":"2024-09-20T18:49:48.500809+0000","last_unstale":"2024-09-20T18:59:06.067536+0000","last_undegraded":"2024-09-20T18:59:06.067536+0000","last_fullsized":"2024-09-20T18:59:06.067536+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:59:02.023996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:59:02.023996+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:06.789889+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"262'10","reported_seq":21,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.774412+0000","last_change":"2024-09-20T18:59:05.105124+0000","last_active":"2024-09-20T18:59:10.774412+0000","last_peered":"2024-09-20T18:59:10.774412+0000","last_clean":"2024-09-20T18:59:10.774412+0000","last_became_active":"2024-09-20T18:59:05.089298+0000","last_became_peered":"2024-09-20T18:59:05.089298+0000","last_unstale":"2024-09-20T18:59:10.774412+0000","last_undegraded":"2024-09-20T18:59:10.774412+0000","last_fullsized":"2024-09-20T18:59:10.774412+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'9","last_scrub_stamp":"2024-09-20T18:57:48.853302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:48.853302+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2","version":"262'3149","reported_seq":3192,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.622792+0000","last_change":"2024-09-20T18:58:33.827583+0000","last_active":"2024-09-20T18:59:06.622792+0000","last_peered":"2024-09-20T18:59:06.622792+0000","last_clean":"2024-09-20T18:59:06.622792+0000","last_became_active":"2024-09-20T18:58:33.827098+0000","last_became_peered":"2024-09-20T18:58:33.827098+0000","last_unstale":"2024-09-20T18:59:06.622792+0000","last_undegraded":"2024-09-20T18:59:06.622792+0000","last_fullsized":"2024-09-20T18:59:06.622792+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":3149,"log_dups_size":0,"ondisk_log_size":3149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:41.122270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":206372864,"num_objects":3149,"num_object_clones":0,"num_object_copies":9447,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3149,"num_write_kb":201536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":450,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.066078+0000","last_change":"2024-09-20T18:55:18.581290+0000","last_active":"2024-09-20T18:59:06.066078+0000","last_peered":"2024-09-20T18:59:06.066078+0000","last_clean":"2024-09-20T18:59:06.066078+0000","last_became_active":"2024-09-20T18:49:58.992302+0000","last_became_peered":"2024-09-20T18:49:58.992302+0000","last_unstale":"2024-09-20T18:59:06.066078+0000","last_undegraded":"2024-09-20T18:59:06.066078+0000","last_fullsized":"2024-09-20T18:59:06.066078+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:55:18.581200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:18.581200+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:42.176183+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":435,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.071202+0000","last_change":"2024-09-20T18:54:54.288199+0000","last_active":"2024-09-20T18:59:06.071202+0000","last_peered":"2024-09-20T18:59:06.071202+0000","last_clean":"2024-09-20T18:59:06.071202+0000","last_became_active":"2024-09-20T18:49:45.936104+0000","last_became_peered":"2024-09-20T18:49:45.936104+0000","last_unstale":"2024-09-20T18:59:06.071202+0000","last_undegraded":"2024-09-20T18:59:06.071202+0000","last_fullsized":"2024-09-20T18:59:06.071202+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:54.288117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:54.288117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:21.124624+0000","scrub_duration":46,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":555,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.753996+0000","last_change":"2024-09-20T18:54:14.292660+0000","last_active":"2024-09-20T18:59:10.753996+0000","last_peered":"2024-09-20T18:59:10.753996+0000","last_clean":"2024-09-20T18:59:10.753996+0000","last_became_active":"2024-09-20T18:49:49.960697+0000","last_became_peered":"2024-09-20T18:49:49.960697+0000","last_unstale":"2024-09-20T18:59:10.753996+0000","last_undegraded":"2024-09-20T18:59:10.753996+0000","last_fullsized":"2024-09-20T18:59:10.753996+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:54:14.292610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:14.292610+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:10.776813+0000","scrub_duration":34,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":429,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-20T18:59:05.080070+0000","last_change":"2024-09-20T18:58:31.493658+0000","last_active":"2024-09-20T18:59:05.080070+0000","last_peered":"2024-09-20T18:59:05.080070+0000","last_clean":"2024-09-20T18:59:05.080070+0000","last_became_active":"2024-09-20T18:49:51.971399+0000","last_became_peered":"2024-09-20T18:49:51.971399+0000","last_unstale":"2024-09-20T18:59:05.080070+0000","last_undegraded":"2024-09-20T18:59:05.080070+0000","last_fullsized":"2024-09-20T18:59:05.080070+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:31.493595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:31.493595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:54.153071+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"263'3453","reported_seq":3498,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.903412+0000","last_change":"2024-09-20T18:58:33.820158+0000","last_active":"2024-09-20T18:59:10.903412+0000","last_peered":"2024-09-20T18:59:10.903412+0000","last_clean":"2024-09-20T18:59:10.903412+0000","last_became_active":"2024-09-20T18:58:33.819886+0000","last_became_peered":"2024-09-20T18:58:33.819886+0000","last_unstale":"2024-09-20T18:59:10.903412+0000","last_undegraded":"2024-09-20T18:59:10.903412+0000","last_fullsized":"2024-09-20T18:59:10.903412+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":3453,"log_dups_size":0,"ondisk_log_size":3453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:02.637424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":226295808,"num_objects":3453,"num_object_clones":0,"num_object_copies":10359,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3453,"num_write_kb":220992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":471,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.070382+0000","last_change":"2024-09-20T18:58:08.025426+0000","last_active":"2024-09-20T18:59:06.070382+0000","last_peered":"2024-09-20T18:59:06.070382+0000","last_clean":"2024-09-20T18:59:06.070382+0000","last_became_active":"2024-09-20T18:49:48.499189+0000","last_became_peered":"2024-09-20T18:49:48.499189+0000","last_unstale":"2024-09-20T18:59:06.070382+0000","last_undegraded":"2024-09-20T18:59:06.070382+0000","last_fullsized":"2024-09-20T18:59:06.070382+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-20T18:58:08.025202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:08.025202+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:16.521840+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":435,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.077002+0000","last_change":"2024-09-20T18:58:25.159724+0000","last_active":"2024-09-20T18:59:06.077002+0000","last_peered":"2024-09-20T18:59:06.077002+0000","last_clean":"2024-09-20T18:59:06.077002+0000","last_became_active":"2024-09-20T18:49:45.940544+0000","last_became_peered":"2024-09-20T18:49:45.940544+0000","last_unstale":"2024-09-20T18:59:06.077002+0000","last_undegraded":"2024-09-20T18:59:06.077002+0000","last_fullsized":"2024-09-20T18:59:06.077002+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:25.159673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:25.159673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:40.807883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":439,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.752652+0000","last_change":"2024-09-20T18:54:51.229251+0000","last_active":"2024-09-20T18:59:10.752652+0000","last_peered":"2024-09-20T18:59:10.752652+0000","last_clean":"2024-09-20T18:59:10.752652+0000","last_became_active":"2024-09-20T18:49:49.963073+0000","last_became_peered":"2024-09-20T18:49:49.963073+0000","last_unstale":"2024-09-20T18:59:10.752652+0000","last_undegraded":"2024-09-20T18:59:10.752652+0000","last_fullsized":"2024-09-20T18:59:10.752652+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.229201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.229201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:20.666517+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":443,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.753527+0000","last_change":"2024-09-20T18:58:27.704583+0000","last_active":"2024-09-20T18:59:10.753527+0000","last_peered":"2024-09-20T18:59:10.753527+0000","last_clean":"2024-09-20T18:59:10.753527+0000","last_became_active":"2024-09-20T18:49:51.968793+0000","last_became_peered":"2024-09-20T18:49:51.968793+0000","last_unstale":"2024-09-20T18:59:10.753527+0000","last_undegraded":"2024-09-20T18:59:10.753527+0000","last_fullsized":"2024-09-20T18:59:10.753527+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:27.704520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:27.704520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:37.550976+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.c","version":"262'3251","reported_seq":3294,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.644068+0000","last_change":"2024-09-20T18:58:33.845752+0000","last_active":"2024-09-20T18:59:06.644068+0000","last_peered":"2024-09-20T18:59:06.644068+0000","last_clean":"2024-09-20T18:59:06.644068+0000","last_became_active":"2024-09-20T18:58:33.845463+0000","last_became_peered":"2024-09-20T18:58:33.845463+0000","last_unstale":"2024-09-20T18:59:06.644068+0000","last_undegraded":"2024-09-20T18:59:06.644068+0000","last_fullsized":"2024-09-20T18:59:06.644068+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":3251,"log_dups_size":0,"ondisk_log_size":3251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:58.514651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213057536,"num_objects":3251,"num_object_clones":0,"num_object_copies":9753,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3251,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3251,"num_write_kb":208064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":518,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.752724+0000","last_change":"2024-09-20T18:59:05.105526+0000","last_active":"2024-09-20T18:59:10.752724+0000","last_peered":"2024-09-20T18:59:10.752724+0000","last_clean":"2024-09-20T18:59:10.752724+0000","last_became_active":"2024-09-20T18:59:05.074532+0000","last_became_peered":"2024-09-20T18:59:05.074532+0000","last_unstale":"2024-09-20T18:59:10.752724+0000","last_undegraded":"2024-09-20T18:59:10.752724+0000","last_fullsized":"2024-09-20T18:59:10.752724+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-20T18:54:57.394893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.394893+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":437,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.067788+0000","last_change":"2024-09-20T18:58:29.939909+0000","last_active":"2024-09-20T18:59:06.067788+0000","last_peered":"2024-09-20T18:59:06.067788+0000","last_clean":"2024-09-20T18:59:06.067788+0000","last_became_active":"2024-09-20T18:49:45.940469+0000","last_became_peered":"2024-09-20T18:49:45.940469+0000","last_unstale":"2024-09-20T18:59:06.067788+0000","last_undegraded":"2024-09-20T18:59:06.067788+0000","last_fullsized":"2024-09-20T18:59:06.067788+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:58:29.939813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:29.939813+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:46.132321+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"66'8","reported_seq":564,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.070231+0000","last_change":"2024-09-20T18:58:23.114419+0000","last_active":"2024-09-20T18:59:06.070231+0000","last_peered":"2024-09-20T18:59:06.070231+0000","last_clean":"2024-09-20T18:59:06.070231+0000","last_became_active":"2024-09-20T18:49:49.956133+0000","last_became_peered":"2024-09-20T18:49:49.956133+0000","last_unstale":"2024-09-20T18:59:06.070231+0000","last_undegraded":"2024-09-20T18:59:06.070231+0000","last_fullsized":"2024-09-20T18:59:06.070231+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:58:23.114371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:23.114371+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:27.894374+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":448,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.761611+0000","last_change":"2024-09-20T18:59:05.082117+0000","last_active":"2024-09-20T18:59:10.761611+0000","last_peered":"2024-09-20T18:59:10.761611+0000","last_clean":"2024-09-20T18:59:10.761611+0000","last_became_active":"2024-09-20T18:59:05.081814+0000","last_became_peered":"2024-09-20T18:59:05.081814+0000","last_unstale":"2024-09-20T18:59:10.761611+0000","last_undegraded":"2024-09-20T18:59:10.761611+0000","last_fullsized":"2024-09-20T18:59:10.761611+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:29.530001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:29.530001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:29.664505+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"260'3094","reported_seq":3135,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-20T18:59:05.083252+0000","last_change":"2024-09-20T18:58:33.826834+0000","last_active":"2024-09-20T18:59:05.083252+0000","last_peered":"2024-09-20T18:59:05.083252+0000","last_clean":"2024-09-20T18:59:05.083252+0000","last_became_active":"2024-09-20T18:58:33.826568+0000","last_became_peered":"2024-09-20T18:58:33.826568+0000","last_unstale":"2024-09-20T18:59:05.083252+0000","last_undegraded":"2024-09-20T18:59:05.083252+0000","last_fullsized":"2024-09-20T18:59:05.083252+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":3094,"log_dups_size":0,"ondisk_log_size":3094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:50.766101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":202768384,"num_objects":3094,"num_object_clones":0,"num_object_copies":9282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3094,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3094,"num_write_kb":198016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":470,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.073425+0000","last_change":"2024-09-20T18:59:06.064691+0000","last_active":"2024-09-20T18:59:06.073425+0000","last_peered":"2024-09-20T18:59:06.073425+0000","last_clean":"2024-09-20T18:59:06.073425+0000","last_became_active":"2024-09-20T18:49:47.949626+0000","last_became_peered":"2024-09-20T18:49:47.949626+0000","last_unstale":"2024-09-20T18:59:06.073425+0000","last_undegraded":"2024-09-20T18:59:06.073425+0000","last_fullsized":"2024-09-20T18:59:06.073425+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:59:06.064599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:59:06.064599+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:33.522592+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":449,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-20T18:59:05.081315+0000","last_change":"2024-09-20T18:58:32.528966+0000","last_active":"2024-09-20T18:59:05.081315+0000","last_peered":"2024-09-20T18:59:05.081315+0000","last_clean":"2024-09-20T18:59:05.081315+0000","last_became_active":"2024-09-20T18:49:45.942496+0000","last_became_peered":"2024-09-20T18:49:45.942496+0000","last_unstale":"2024-09-20T18:59:05.081315+0000","last_undegraded":"2024-09-20T18:59:05.081315+0000","last_fullsized":"2024-09-20T18:59:05.081315+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.528310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.528310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:37.417253+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"262'10","reported_seq":579,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.087621+0000","last_change":"2024-09-20T18:59:05.098010+0000","last_active":"2024-09-20T18:59:06.087621+0000","last_peered":"2024-09-20T18:59:06.087621+0000","last_clean":"2024-09-20T18:59:06.087621+0000","last_became_active":"2024-09-20T18:59:05.080183+0000","last_became_peered":"2024-09-20T18:59:05.080183+0000","last_unstale":"2024-09-20T18:59:06.087621+0000","last_undegraded":"2024-09-20T18:59:06.087621+0000","last_fullsized":"2024-09-20T18:59:06.087621+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'9","last_scrub_stamp":"2024-09-20T18:59:00.055110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:59:00.055110+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:07.456134+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":416,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.067464+0000","last_change":"2024-09-20T18:58:26.948165+0000","last_active":"2024-09-20T18:59:06.067464+0000","last_peered":"2024-09-20T18:59:06.067464+0000","last_clean":"2024-09-20T18:59:06.067464+0000","last_became_active":"2024-09-20T18:49:51.964780+0000","last_became_peered":"2024-09-20T18:49:51.964780+0000","last_unstale":"2024-09-20T18:59:06.067464+0000","last_undegraded":"2024-09-20T18:59:06.067464+0000","last_fullsized":"2024-09-20T18:59:06.067464+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:26.948102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:26.948102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:28.716180+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"263'3541","reported_seq":3586,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.890318+0000","last_change":"2024-09-20T18:58:33.845804+0000","last_active":"2024-09-20T18:59:10.890318+0000","last_peered":"2024-09-20T18:59:10.890318+0000","last_clean":"2024-09-20T18:59:10.890318+0000","last_became_active":"2024-09-20T18:58:33.845493+0000","last_became_peered":"2024-09-20T18:58:33.845493+0000","last_unstale":"2024-09-20T18:59:10.890318+0000","last_undegraded":"2024-09-20T18:59:10.890318+0000","last_fullsized":"2024-09-20T18:59:10.890318+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":3541,"log_dups_size":0,"ondisk_log_size":3541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:59.386071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":232062976,"num_objects":3541,"num_object_clones":0,"num_object_copies":10623,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3541,"num_write_kb":226624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":468,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.087673+0000","last_change":"2024-09-20T18:54:14.923895+0000","last_active":"2024-09-20T18:59:06.087673+0000","last_peered":"2024-09-20T18:59:06.087673+0000","last_clean":"2024-09-20T18:59:06.087673+0000","last_became_active":"2024-09-20T18:49:48.497736+0000","last_became_peered":"2024-09-20T18:49:48.497736+0000","last_unstale":"2024-09-20T18:59:06.087673+0000","last_undegraded":"2024-09-20T18:59:06.087673+0000","last_fullsized":"2024-09-20T18:59:06.087673+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-20T18:54:14.923682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:14.923682+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:39.022337+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":427,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.068915+0000","last_change":"2024-09-20T18:57:09.940634+0000","last_active":"2024-09-20T18:59:06.068915+0000","last_peered":"2024-09-20T18:59:06.068915+0000","last_clean":"2024-09-20T18:59:06.068915+0000","last_became_active":"2024-09-20T18:49:45.942253+0000","last_became_peered":"2024-09-20T18:49:45.942253+0000","last_unstale":"2024-09-20T18:59:06.068915+0000","last_undegraded":"2024-09-20T18:59:06.068915+0000","last_fullsized":"2024-09-20T18:59:06.068915+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:09.940517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:09.940517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:19.931681+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":16,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.773674+0000","last_change":"2024-09-20T18:59:05.085948+0000","last_active":"2024-09-20T18:59:10.773674+0000","last_peered":"2024-09-20T18:59:10.773674+0000","last_clean":"2024-09-20T18:59:10.773674+0000","last_became_active":"2024-09-20T18:59:05.085636+0000","last_became_peered":"2024-09-20T18:59:05.085636+0000","last_unstale":"2024-09-20T18:59:10.773674+0000","last_undegraded":"2024-09-20T18:59:10.773674+0000","last_fullsized":"2024-09-20T18:59:10.773674+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:50.610871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:50.610871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":453,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.066932+0000","last_change":"2024-09-20T18:59:05.085472+0000","last_active":"2024-09-20T18:59:06.066932+0000","last_peered":"2024-09-20T18:59:06.066932+0000","last_clean":"2024-09-20T18:59:06.066932+0000","last_became_active":"2024-09-20T18:59:05.085131+0000","last_became_peered":"2024-09-20T18:59:05.085131+0000","last_unstale":"2024-09-20T18:59:06.066932+0000","last_undegraded":"2024-09-20T18:59:06.066932+0000","last_fullsized":"2024-09-20T18:59:06.066932+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:58.973176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:58.973176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:19.423898+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"262'3200","reported_seq":3243,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:07.129033+0000","last_change":"2024-09-20T18:58:33.822011+0000","last_active":"2024-09-20T18:59:07.129033+0000","last_peered":"2024-09-20T18:59:07.129033+0000","last_clean":"2024-09-20T18:59:07.129033+0000","last_became_active":"2024-09-20T18:58:33.820506+0000","last_became_peered":"2024-09-20T18:58:33.820506+0000","last_unstale":"2024-09-20T18:59:07.129033+0000","last_undegraded":"2024-09-20T18:59:07.129033+0000","last_fullsized":"2024-09-20T18:59:07.129033+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":3200,"log_dups_size":0,"ondisk_log_size":3200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:55.828171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":209715200,"num_objects":3200,"num_object_clones":0,"num_object_copies":9600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3200,"num_write_kb":204800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"66'28","reported_seq":600,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.068815+0000","last_change":"2024-09-20T18:58:07.018449+0000","last_active":"2024-09-20T18:59:06.068815+0000","last_peered":"2024-09-20T18:59:06.068815+0000","last_clean":"2024-09-20T18:59:06.068815+0000","last_became_active":"2024-09-20T18:49:47.955335+0000","last_became_peered":"2024-09-20T18:49:47.955335+0000","last_unstale":"2024-09-20T18:59:06.068815+0000","last_undegraded":"2024-09-20T18:59:06.068815+0000","last_fullsized":"2024-09-20T18:59:06.068815+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'28","last_scrub_stamp":"2024-09-20T18:58:07.018340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:07.018340+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:28.256912+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":445,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.752810+0000","last_change":"2024-09-20T18:59:05.082509+0000","last_active":"2024-09-20T18:59:10.752810+0000","last_peered":"2024-09-20T18:59:10.752810+0000","last_clean":"2024-09-20T18:59:10.752810+0000","last_became_active":"2024-09-20T18:59:05.081468+0000","last_became_peered":"2024-09-20T18:59:05.081468+0000","last_unstale":"2024-09-20T18:59:10.752810+0000","last_undegraded":"2024-09-20T18:59:10.752810+0000","last_fullsized":"2024-09-20T18:59:10.752810+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:21.807703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:21.807703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":481,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-20T18:59:05.108420+0000","last_change":"2024-09-20T18:59:05.108332+0000","last_active":"2024-09-20T18:59:05.108420+0000","last_peered":"2024-09-20T18:59:05.108420+0000","last_clean":"2024-09-20T18:59:05.108420+0000","last_became_active":"2024-09-20T18:59:05.086786+0000","last_became_peered":"2024-09-20T18:59:05.086786+0000","last_unstale":"2024-09-20T18:59:05.108420+0000","last_undegraded":"2024-09-20T18:59:05.108420+0000","last_fullsized":"2024-09-20T18:59:05.108420+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-20T18:54:51.227531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.227531+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0012781159999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":444,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.067761+0000","last_change":"2024-09-20T18:58:59.099799+0000","last_active":"2024-09-20T18:59:06.067761+0000","last_peered":"2024-09-20T18:59:06.067761+0000","last_clean":"2024-09-20T18:59:06.067761+0000","last_became_active":"2024-09-20T18:49:49.953971+0000","last_became_peered":"2024-09-20T18:49:49.953971+0000","last_unstale":"2024-09-20T18:59:06.067761+0000","last_undegraded":"2024-09-20T18:59:06.067761+0000","last_fullsized":"2024-09-20T18:59:06.067761+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:59.099736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:59.099736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:28.205403+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":415,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.070796+0000","last_change":"2024-09-20T18:57:12.744116+0000","last_active":"2024-09-20T18:59:06.070796+0000","last_peered":"2024-09-20T18:59:06.070796+0000","last_clean":"2024-09-20T18:59:06.070796+0000","last_became_active":"2024-09-20T18:49:51.969400+0000","last_became_peered":"2024-09-20T18:49:51.969400+0000","last_unstale":"2024-09-20T18:59:06.070796+0000","last_undegraded":"2024-09-20T18:59:06.070796+0000","last_fullsized":"2024-09-20T18:59:06.070796+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:12.744040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:12.744040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:54.130091+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"260'3153","reported_seq":3194,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-20T18:59:05.078355+0000","last_change":"2024-09-20T18:58:33.829042+0000","last_active":"2024-09-20T18:59:05.078355+0000","last_peered":"2024-09-20T18:59:05.078355+0000","last_clean":"2024-09-20T18:59:05.078355+0000","last_became_active":"2024-09-20T18:58:33.828787+0000","last_became_peered":"2024-09-20T18:58:33.828787+0000","last_unstale":"2024-09-20T18:59:05.078355+0000","last_undegraded":"2024-09-20T18:59:05.078355+0000","last_fullsized":"2024-09-20T18:59:05.078355+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":3153,"log_dups_size":0,"ondisk_log_size":3153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:57.211364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":206635008,"num_objects":3153,"num_object_clones":0,"num_object_copies":9459,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3153,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3153,"num_write_kb":201792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":474,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.069014+0000","last_change":"2024-09-20T18:58:06.079117+0000","last_active":"2024-09-20T18:59:06.069014+0000","last_peered":"2024-09-20T18:59:06.069014+0000","last_clean":"2024-09-20T18:59:06.069014+0000","last_became_active":"2024-09-20T18:49:47.950904+0000","last_became_peered":"2024-09-20T18:49:47.950904+0000","last_unstale":"2024-09-20T18:59:06.069014+0000","last_undegraded":"2024-09-20T18:59:06.069014+0000","last_fullsized":"2024-09-20T18:59:06.069014+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-20T18:58:06.078352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:06.078352+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:26.574178+0000","scrub_duration":59,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":465,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.152269+0000","last_change":"2024-09-20T18:59:05.080117+0000","last_active":"2024-09-20T18:59:06.152269+0000","last_peered":"2024-09-20T18:59:06.152269+0000","last_clean":"2024-09-20T18:59:06.152269+0000","last_became_active":"2024-09-20T18:59:05.079789+0000","last_became_peered":"2024-09-20T18:59:05.079789+0000","last_unstale":"2024-09-20T18:59:06.152269+0000","last_undegraded":"2024-09-20T18:59:06.152269+0000","last_fullsized":"2024-09-20T18:59:06.152269+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:28.987458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:28.987458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:38.276955+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":447,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.065064+0000","last_change":"2024-09-20T18:58:24.229997+0000","last_active":"2024-09-20T18:59:06.065064+0000","last_peered":"2024-09-20T18:59:06.065064+0000","last_clean":"2024-09-20T18:59:06.065064+0000","last_became_active":"2024-09-20T18:49:44.937135+0000","last_became_peered":"2024-09-20T18:49:44.937135+0000","last_unstale":"2024-09-20T18:59:06.065064+0000","last_undegraded":"2024-09-20T18:59:06.065064+0000","last_fullsized":"2024-09-20T18:59:06.065064+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:58:24.229938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:58:24.229938+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:47.181712+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00095558800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":422,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.066114+0000","last_change":"2024-09-20T18:51:59.796450+0000","last_active":"2024-09-20T18:59:06.066114+0000","last_peered":"2024-09-20T18:59:06.066114+0000","last_clean":"2024-09-20T18:59:06.066114+0000","last_became_active":"2024-09-20T18:49:49.954424+0000","last_became_peered":"2024-09-20T18:49:49.954424+0000","last_unstale":"2024-09-20T18:59:06.066114+0000","last_undegraded":"2024-09-20T18:59:06.066114+0000","last_fullsized":"2024-09-20T18:59:06.066114+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:59.796383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:59.796383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:21.220437+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":435,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.763058+0000","last_change":"2024-09-20T18:59:05.082433+0000","last_active":"2024-09-20T18:59:10.763058+0000","last_peered":"2024-09-20T18:59:10.763058+0000","last_clean":"2024-09-20T18:59:10.763058+0000","last_became_active":"2024-09-20T18:59:05.081459+0000","last_became_peered":"2024-09-20T18:59:05.081459+0000","last_unstale":"2024-09-20T18:59:10.763058+0000","last_undegraded":"2024-09-20T18:59:10.763058+0000","last_fullsized":"2024-09-20T18:59:10.763058+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:39.814259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:39.814259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"262'3117","reported_seq":3238,"reported_epoch":262,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-20T18:59:07.152453+0000","last_change":"2024-09-20T18:59:06.260187+0000","last_active":"2024-09-20T18:59:07.152453+0000","last_peered":"2024-09-20T18:59:07.152453+0000","last_clean":"2024-09-20T18:59:04.032824+0000","last_became_active":"2024-09-20T18:59:06.207214+0000","last_became_peered":"2024-09-20T18:59:06.207214+0000","last_unstale":"2024-09-20T18:59:07.152453+0000","last_undegraded":"2024-09-20T18:59:07.152453+0000","last_fullsized":"2024-09-20T18:59:06.068079+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":3117,"log_dups_size":0,"ondisk_log_size":3117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":204275712,"num_objects":3117,"num_object_clones":0,"num_object_copies":9351,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3013,"num_objects_unfound":0,"num_objects_dirty":3117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3117,"num_write_kb":199488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":2490368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,5],"avail_no_missing":["0","2","5"],"object_location_counts":[{"shards":"0,2,5","objects":3013},{"shards":"0,5","objects":104}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":474,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.067619+0000","last_change":"2024-09-20T18:59:06.022021+0000","last_active":"2024-09-20T18:59:06.067619+0000","last_peered":"2024-09-20T18:59:06.067619+0000","last_clean":"2024-09-20T18:59:06.067619+0000","last_became_active":"2024-09-20T18:49:47.949230+0000","last_became_peered":"2024-09-20T18:49:47.949230+0000","last_unstale":"2024-09-20T18:59:06.067619+0000","last_undegraded":"2024-09-20T18:59:06.067619+0000","last_fullsized":"2024-09-20T18:59:06.067619+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:59:06.021962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:59:06.021962+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:12.002280+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":444,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.067403+0000","last_change":"2024-09-20T18:54:27.905856+0000","last_active":"2024-09-20T18:59:06.067403+0000","last_peered":"2024-09-20T18:59:06.067403+0000","last_clean":"2024-09-20T18:59:06.067403+0000","last_became_active":"2024-09-20T18:49:45.939197+0000","last_became_peered":"2024-09-20T18:49:45.939197+0000","last_unstale":"2024-09-20T18:59:06.067403+0000","last_undegraded":"2024-09-20T18:59:06.067403+0000","last_fullsized":"2024-09-20T18:59:06.067403+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:27.905803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:27.905803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:43.998998+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":1016,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.761644+0000","last_change":"2024-09-20T18:54:26.359887+0000","last_active":"2024-09-20T18:59:10.761644+0000","last_peered":"2024-09-20T18:59:10.761644+0000","last_clean":"2024-09-20T18:59:10.761644+0000","last_became_active":"2024-09-20T18:49:44.936819+0000","last_became_peered":"2024-09-20T18:49:44.936819+0000","last_unstale":"2024-09-20T18:59:10.761644+0000","last_undegraded":"2024-09-20T18:59:10.761644+0000","last_fullsized":"2024-09-20T18:59:10.761644+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-20T18:54:26.359807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:54:26.359807+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:42.503306+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0010996199999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":559,"num_read_kb":554,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":433,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-20T18:59:05.083882+0000","last_change":"2024-09-20T18:57:54.362085+0000","last_active":"2024-09-20T18:59:05.083882+0000","last_peered":"2024-09-20T18:59:05.083882+0000","last_clean":"2024-09-20T18:59:05.083882+0000","last_became_active":"2024-09-20T18:49:49.961066+0000","last_became_peered":"2024-09-20T18:49:49.961066+0000","last_unstale":"2024-09-20T18:59:05.083882+0000","last_undegraded":"2024-09-20T18:59:05.083882+0000","last_fullsized":"2024-09-20T18:59:05.083882+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:54.362038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:54.362038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:08.560302+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":439,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.068875+0000","last_change":"2024-09-20T18:58:08.019154+0000","last_active":"2024-09-20T18:59:06.068875+0000","last_peered":"2024-09-20T18:59:06.068875+0000","last_clean":"2024-09-20T18:59:06.068875+0000","last_became_active":"2024-09-20T18:49:51.970343+0000","last_became_peered":"2024-09-20T18:49:51.970343+0000","last_unstale":"2024-09-20T18:59:06.068875+0000","last_undegraded":"2024-09-20T18:59:06.068875+0000","last_fullsized":"2024-09-20T18:59:06.068875+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:08.019092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:08.019092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:32.791415+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"262'3193","reported_seq":3236,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.603527+0000","last_change":"2024-09-20T18:58:33.820307+0000","last_active":"2024-09-20T18:59:06.603527+0000","last_peered":"2024-09-20T18:59:06.603527+0000","last_clean":"2024-09-20T18:59:06.603527+0000","last_became_active":"2024-09-20T18:58:33.820027+0000","last_became_peered":"2024-09-20T18:58:33.820027+0000","last_unstale":"2024-09-20T18:59:06.603527+0000","last_undegraded":"2024-09-20T18:59:06.603527+0000","last_fullsized":"2024-09-20T18:59:06.603527+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":3193,"log_dups_size":0,"ondisk_log_size":3193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:41.996589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":209256448,"num_objects":3193,"num_object_clones":0,"num_object_copies":9579,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3193,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3193,"num_write_kb":204352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"262'18","reported_seq":35,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.773972+0000","last_change":"2024-09-20T18:59:05.102446+0000","last_active":"2024-09-20T18:59:10.773972+0000","last_peered":"2024-09-20T18:59:10.773972+0000","last_clean":"2024-09-20T18:59:10.773972+0000","last_became_active":"2024-09-20T18:59:05.088069+0000","last_became_peered":"2024-09-20T18:59:05.088069+0000","last_unstale":"2024-09-20T18:59:10.773972+0000","last_undegraded":"2024-09-20T18:59:10.773972+0000","last_fullsized":"2024-09-20T18:59:10.773972+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'17","last_scrub_stamp":"2024-09-20T18:57:22.816154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:22.816154+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":24,"num_read_kb":14,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":457,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.071685+0000","last_change":"2024-09-20T18:59:05.079328+0000","last_active":"2024-09-20T18:59:06.071685+0000","last_peered":"2024-09-20T18:59:06.071685+0000","last_clean":"2024-09-20T18:59:06.071685+0000","last_became_active":"2024-09-20T18:59:05.079111+0000","last_became_peered":"2024-09-20T18:59:05.079111+0000","last_unstale":"2024-09-20T18:59:06.071685+0000","last_undegraded":"2024-09-20T18:59:06.071685+0000","last_fullsized":"2024-09-20T18:59:06.071685+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.100625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.100625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:41.970563+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":513,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.761860+0000","last_change":"2024-09-20T18:59:05.108551+0000","last_active":"2024-09-20T18:59:10.761860+0000","last_peered":"2024-09-20T18:59:10.761860+0000","last_clean":"2024-09-20T18:59:10.761860+0000","last_became_active":"2024-09-20T18:59:05.086103+0000","last_became_peered":"2024-09-20T18:59:05.086103+0000","last_unstale":"2024-09-20T18:59:10.761860+0000","last_undegraded":"2024-09-20T18:59:10.761860+0000","last_fullsized":"2024-09-20T18:59:10.761860+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T18:55:13.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T18:55:13.696607+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":430,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.762108+0000","last_change":"2024-09-20T18:58:23.545370+0000","last_active":"2024-09-20T18:59:10.762108+0000","last_peered":"2024-09-20T18:59:10.762108+0000","last_clean":"2024-09-20T18:59:10.762108+0000","last_became_active":"2024-09-20T18:49:49.956128+0000","last_became_peered":"2024-09-20T18:49:49.956128+0000","last_unstale":"2024-09-20T18:59:10.762108+0000","last_undegraded":"2024-09-20T18:59:10.762108+0000","last_fullsized":"2024-09-20T18:59:10.762108+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:23.545321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:23.545321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:34.942242+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":437,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-20T18:59:05.083612+0000","last_change":"2024-09-20T18:58:03.523127+0000","last_active":"2024-09-20T18:59:05.083612+0000","last_peered":"2024-09-20T18:59:05.083612+0000","last_clean":"2024-09-20T18:59:05.083612+0000","last_became_active":"2024-09-20T18:49:51.971332+0000","last_became_peered":"2024-09-20T18:49:51.971332+0000","last_unstale":"2024-09-20T18:59:05.083612+0000","last_undegraded":"2024-09-20T18:59:05.083612+0000","last_fullsized":"2024-09-20T18:59:05.083612+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:03.523078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:03.523078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:14.059180+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"263'3532","reported_seq":3577,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.890702+0000","last_change":"2024-09-20T18:58:33.825100+0000","last_active":"2024-09-20T18:59:10.890702+0000","last_peered":"2024-09-20T18:59:10.890702+0000","last_clean":"2024-09-20T18:59:10.890702+0000","last_became_active":"2024-09-20T18:58:33.824751+0000","last_became_peered":"2024-09-20T18:58:33.824751+0000","last_unstale":"2024-09-20T18:59:10.890702+0000","last_undegraded":"2024-09-20T18:59:10.890702+0000","last_fullsized":"2024-09-20T18:59:10.890702+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":3532,"log_dups_size":0,"ondisk_log_size":3532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:33.144864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":231473152,"num_objects":3532,"num_object_clones":0,"num_object_copies":10596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3532,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3532,"num_write_kb":226048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":492,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.754027+0000","last_change":"2024-09-20T18:59:05.115678+0000","last_active":"2024-09-20T18:59:10.754027+0000","last_peered":"2024-09-20T18:59:10.754027+0000","last_clean":"2024-09-20T18:59:10.754027+0000","last_became_active":"2024-09-20T18:59:05.079753+0000","last_became_peered":"2024-09-20T18:59:05.079753+0000","last_unstale":"2024-09-20T18:59:10.754027+0000","last_undegraded":"2024-09-20T18:59:10.754027+0000","last_fullsized":"2024-09-20T18:59:10.754027+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:58:24.616580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:24.616580+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:36.854453+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":439,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.753112+0000","last_change":"2024-09-20T18:58:32.694078+0000","last_active":"2024-09-20T18:59:10.753112+0000","last_peered":"2024-09-20T18:59:10.753112+0000","last_clean":"2024-09-20T18:59:10.753112+0000","last_became_active":"2024-09-20T18:49:45.944579+0000","last_became_peered":"2024-09-20T18:49:45.944579+0000","last_unstale":"2024-09-20T18:59:10.753112+0000","last_undegraded":"2024-09-20T18:59:10.753112+0000","last_fullsized":"2024-09-20T18:59:10.753112+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.693993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.693993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:48.788982+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":447,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-20T18:59:05.090935+0000","last_change":"2024-09-20T18:59:05.090855+0000","last_active":"2024-09-20T18:59:05.090935+0000","last_peered":"2024-09-20T18:59:05.090935+0000","last_clean":"2024-09-20T18:59:05.090935+0000","last_became_active":"2024-09-20T18:59:05.090504+0000","last_became_peered":"2024-09-20T18:59:05.090504+0000","last_unstale":"2024-09-20T18:59:05.090935+0000","last_undegraded":"2024-09-20T18:59:05.090935+0000","last_fullsized":"2024-09-20T18:59:05.090935+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:11.401777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:11.401777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":445,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.070819+0000","last_change":"2024-09-20T18:59:05.080255+0000","last_active":"2024-09-20T18:59:06.070819+0000","last_peered":"2024-09-20T18:59:06.070819+0000","last_clean":"2024-09-20T18:59:06.070819+0000","last_became_active":"2024-09-20T18:59:05.079897+0000","last_became_peered":"2024-09-20T18:59:05.079897+0000","last_unstale":"2024-09-20T18:59:06.070819+0000","last_undegraded":"2024-09-20T18:59:06.070819+0000","last_fullsized":"2024-09-20T18:59:06.070819+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:01.083156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:59:01.083156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:15.111438+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.0","version":"262'3297","reported_seq":3361,"reported_epoch":262,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-20T18:59:06.431291+0000","last_change":"2024-09-20T18:59:06.215289+0000","last_active":"2024-09-20T18:59:06.431291+0000","last_peered":"2024-09-20T18:59:06.431291+0000","last_clean":"2024-09-20T18:59:04.061108+0000","last_became_active":"2024-09-20T18:59:06.197951+0000","last_became_peered":"2024-09-20T18:59:06.197951+0000","last_unstale":"2024-09-20T18:59:06.431291+0000","last_undegraded":"2024-09-20T18:59:06.431291+0000","last_fullsized":"2024-09-20T18:59:06.066740+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":3297,"log_dups_size":0,"ondisk_log_size":3297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":216072192,"num_objects":3297,"num_object_clones":0,"num_object_copies":9891,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3262,"num_objects_unfound":0,"num_objects_dirty":3297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3297,"num_write_kb":211008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":524288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,2],"avail_no_missing":["1","0","2"],"object_location_counts":[{"shards":"0,1,2","objects":3262},{"shards":"1,2","objects":35}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":471,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.753322+0000","last_change":"2024-09-20T18:58:36.716146+0000","last_active":"2024-09-20T18:59:10.753322+0000","last_peered":"2024-09-20T18:59:10.753322+0000","last_clean":"2024-09-20T18:59:10.753322+0000","last_became_active":"2024-09-20T18:49:47.947636+0000","last_became_peered":"2024-09-20T18:49:47.947636+0000","last_unstale":"2024-09-20T18:59:10.753322+0000","last_undegraded":"2024-09-20T18:59:10.753322+0000","last_fullsized":"2024-09-20T18:59:10.753322+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-20T18:58:36.716079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:36.716079+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:36.836323+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.773804+0000","last_change":"2024-09-20T18:59:05.081802+0000","last_active":"2024-09-20T18:59:10.773804+0000","last_peered":"2024-09-20T18:59:10.773804+0000","last_clean":"2024-09-20T18:59:10.773804+0000","last_became_active":"2024-09-20T18:59:05.080829+0000","last_became_peered":"2024-09-20T18:59:05.080829+0000","last_unstale":"2024-09-20T18:59:10.773804+0000","last_undegraded":"2024-09-20T18:59:10.773804+0000","last_fullsized":"2024-09-20T18:59:10.773804+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:25.649733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:25.649733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:33.813140+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":435,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.068547+0000","last_change":"2024-09-20T18:58:36.100033+0000","last_active":"2024-09-20T18:59:06.068547+0000","last_peered":"2024-09-20T18:59:06.068547+0000","last_clean":"2024-09-20T18:59:06.068547+0000","last_became_active":"2024-09-20T18:49:49.959160+0000","last_became_peered":"2024-09-20T18:49:49.959160+0000","last_unstale":"2024-09-20T18:59:06.068547+0000","last_undegraded":"2024-09-20T18:59:06.068547+0000","last_fullsized":"2024-09-20T18:59:06.068547+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:36.099985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:36.099985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:56.689064+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":434,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.066232+0000","last_change":"2024-09-20T18:58:25.227679+0000","last_active":"2024-09-20T18:59:06.066232+0000","last_peered":"2024-09-20T18:59:06.066232+0000","last_clean":"2024-09-20T18:59:06.066232+0000","last_became_active":"2024-09-20T18:49:51.970108+0000","last_became_peered":"2024-09-20T18:49:51.970108+0000","last_unstale":"2024-09-20T18:59:06.066232+0000","last_undegraded":"2024-09-20T18:59:06.066232+0000","last_fullsized":"2024-09-20T18:59:06.066232+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:25.227630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:25.227630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:27.041924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1","version":"262'3224","reported_seq":3287,"reported_epoch":262,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-20T18:59:06.636741+0000","last_change":"2024-09-20T18:59:06.244451+0000","last_active":"2024-09-20T18:59:06.636741+0000","last_peered":"2024-09-20T18:59:06.636741+0000","last_clean":"2024-09-20T18:59:04.063362+0000","last_became_active":"2024-09-20T18:59:06.208908+0000","last_became_peered":"2024-09-20T18:59:06.208908+0000","last_unstale":"2024-09-20T18:59:06.636741+0000","last_undegraded":"2024-09-20T18:59:06.636741+0000","last_fullsized":"2024-09-20T18:59:06.070513+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":3224,"log_dups_size":0,"ondisk_log_size":3224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":211288064,"num_objects":3224,"num_object_clones":0,"num_object_copies":9672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3170,"num_objects_unfound":0,"num_objects_dirty":3224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3224,"num_write_kb":206336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":524288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,7],"avail_no_missing":["3","4","7"],"object_location_counts":[{"shards":"3,4,7","objects":3170},{"shards":"3,7","objects":54}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":496,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.753049+0000","last_change":"2024-09-20T18:59:10.364668+0000","last_active":"2024-09-20T18:59:10.753049+0000","last_peered":"2024-09-20T18:59:10.753049+0000","last_clean":"2024-09-20T18:59:10.753049+0000","last_became_active":"2024-09-20T18:59:05.073564+0000","last_became_peered":"2024-09-20T18:59:05.073564+0000","last_unstale":"2024-09-20T18:59:10.753049+0000","last_undegraded":"2024-09-20T18:59:10.753049+0000","last_fullsized":"2024-09-20T18:59:10.753049+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:59:10.364583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:59:10.364583+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:23.352032+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":443,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.072271+0000","last_change":"2024-09-20T18:58:31.126134+0000","last_active":"2024-09-20T18:59:06.072271+0000","last_peered":"2024-09-20T18:59:06.072271+0000","last_clean":"2024-09-20T18:59:06.072271+0000","last_became_active":"2024-09-20T18:49:45.937264+0000","last_became_peered":"2024-09-20T18:49:45.937264+0000","last_unstale":"2024-09-20T18:59:06.072271+0000","last_undegraded":"2024-09-20T18:59:06.072271+0000","last_fullsized":"2024-09-20T18:59:06.072271+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:31.126087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:31.126087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:45.329561+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":559,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.065664+0000","last_change":"2024-09-20T18:55:17.549660+0000","last_active":"2024-09-20T18:59:06.065664+0000","last_peered":"2024-09-20T18:59:06.065664+0000","last_clean":"2024-09-20T18:59:06.065664+0000","last_became_active":"2024-09-20T18:49:49.957005+0000","last_became_peered":"2024-09-20T18:49:49.957005+0000","last_unstale":"2024-09-20T18:59:06.065664+0000","last_undegraded":"2024-09-20T18:59:06.065664+0000","last_fullsized":"2024-09-20T18:59:06.065664+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:17.549598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:17.549598+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:44.147487+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":435,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.753844+0000","last_change":"2024-09-20T18:58:28.688247+0000","last_active":"2024-09-20T18:59:10.753844+0000","last_peered":"2024-09-20T18:59:10.753844+0000","last_clean":"2024-09-20T18:59:10.753844+0000","last_became_active":"2024-09-20T18:49:51.969355+0000","last_became_peered":"2024-09-20T18:49:51.969355+0000","last_unstale":"2024-09-20T18:59:10.753844+0000","last_undegraded":"2024-09-20T18:59:10.753844+0000","last_fullsized":"2024-09-20T18:59:10.753844+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:28.688195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:28.688195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:31.796237+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"260'3091","reported_seq":3132,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-20T18:59:05.077367+0000","last_change":"2024-09-20T18:58:33.826744+0000","last_active":"2024-09-20T18:59:05.077367+0000","last_peered":"2024-09-20T18:59:05.077367+0000","last_clean":"2024-09-20T18:59:05.077367+0000","last_became_active":"2024-09-20T18:58:33.826480+0000","last_became_peered":"2024-09-20T18:58:33.826480+0000","last_unstale":"2024-09-20T18:59:05.077367+0000","last_undegraded":"2024-09-20T18:59:05.077367+0000","last_fullsized":"2024-09-20T18:59:05.077367+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":3091,"log_dups_size":0,"ondisk_log_size":3091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:44.033875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":202571776,"num_objects":3091,"num_object_clones":0,"num_object_copies":9273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3091,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3091,"num_write_kb":197824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":456,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.087638+0000","last_change":"2024-09-20T18:54:54.897991+0000","last_active":"2024-09-20T18:59:06.087638+0000","last_peered":"2024-09-20T18:59:06.087638+0000","last_clean":"2024-09-20T18:59:06.087638+0000","last_became_active":"2024-09-20T18:49:47.946008+0000","last_became_peered":"2024-09-20T18:49:47.946008+0000","last_unstale":"2024-09-20T18:59:06.087638+0000","last_undegraded":"2024-09-20T18:59:06.087638+0000","last_fullsized":"2024-09-20T18:59:06.087638+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:54.897933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:54.897933+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:52.009917+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":463,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-20T18:59:05.090432+0000","last_change":"2024-09-20T18:59:05.090348+0000","last_active":"2024-09-20T18:59:05.090432+0000","last_peered":"2024-09-20T18:59:05.090432+0000","last_clean":"2024-09-20T18:59:05.090432+0000","last_became_active":"2024-09-20T18:59:05.089886+0000","last_became_peered":"2024-09-20T18:59:05.089886+0000","last_unstale":"2024-09-20T18:59:05.090432+0000","last_undegraded":"2024-09-20T18:59:05.090432+0000","last_fullsized":"2024-09-20T18:59:05.090432+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:56.405349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:56.405349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:22.122798+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":438,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.065052+0000","last_change":"2024-09-20T18:54:43.379898+0000","last_active":"2024-09-20T18:59:06.065052+0000","last_peered":"2024-09-20T18:59:06.065052+0000","last_clean":"2024-09-20T18:59:06.065052+0000","last_became_active":"2024-09-20T18:49:49.956767+0000","last_became_peered":"2024-09-20T18:49:49.956767+0000","last_unstale":"2024-09-20T18:59:06.065052+0000","last_undegraded":"2024-09-20T18:59:06.065052+0000","last_fullsized":"2024-09-20T18:59:06.065052+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:43.379706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:43.379706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:38.219215+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":424,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.067315+0000","last_change":"2024-09-20T18:54:24.976627+0000","last_active":"2024-09-20T18:59:06.067315+0000","last_peered":"2024-09-20T18:59:06.067315+0000","last_clean":"2024-09-20T18:59:06.067315+0000","last_became_active":"2024-09-20T18:49:51.964098+0000","last_became_peered":"2024-09-20T18:49:51.964098+0000","last_unstale":"2024-09-20T18:59:06.067315+0000","last_undegraded":"2024-09-20T18:59:06.067315+0000","last_fullsized":"2024-09-20T18:59:06.067315+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:24.976584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:24.976584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:44.950022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"260'3261","reported_seq":3302,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-20T18:59:05.083513+0000","last_change":"2024-09-20T18:58:33.824658+0000","last_active":"2024-09-20T18:59:05.083513+0000","last_peered":"2024-09-20T18:59:05.083513+0000","last_clean":"2024-09-20T18:59:05.083513+0000","last_became_active":"2024-09-20T18:58:33.824342+0000","last_became_peered":"2024-09-20T18:58:33.824342+0000","last_unstale":"2024-09-20T18:59:05.083513+0000","last_undegraded":"2024-09-20T18:59:05.083513+0000","last_fullsized":"2024-09-20T18:59:05.083513+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":3261,"log_dups_size":0,"ondisk_log_size":3261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:49.056054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213712896,"num_objects":3261,"num_object_clones":0,"num_object_copies":9783,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3261,"num_write_kb":208704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":36,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.773577+0000","last_change":"2024-09-20T18:59:05.107086+0000","last_active":"2024-09-20T18:59:10.773577+0000","last_peered":"2024-09-20T18:59:10.773577+0000","last_clean":"2024-09-20T18:59:10.773577+0000","last_became_active":"2024-09-20T18:59:05.087391+0000","last_became_peered":"2024-09-20T18:59:05.087391+0000","last_unstale":"2024-09-20T18:59:10.773577+0000","last_undegraded":"2024-09-20T18:59:10.773577+0000","last_fullsized":"2024-09-20T18:59:10.773577+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-20T18:57:22.033475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:22.033475+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":31,"num_read_kb":20,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":460,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.763422+0000","last_change":"2024-09-20T18:59:05.087282+0000","last_active":"2024-09-20T18:59:10.763422+0000","last_peered":"2024-09-20T18:59:10.763422+0000","last_clean":"2024-09-20T18:59:10.763422+0000","last_became_active":"2024-09-20T18:59:05.086670+0000","last_became_peered":"2024-09-20T18:59:05.086670+0000","last_unstale":"2024-09-20T18:59:10.763422+0000","last_undegraded":"2024-09-20T18:59:10.763422+0000","last_fullsized":"2024-09-20T18:59:10.763422+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:44.501192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.501192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":438,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.065371+0000","last_change":"2024-09-20T18:55:13.577772+0000","last_active":"2024-09-20T18:59:06.065371+0000","last_peered":"2024-09-20T18:59:06.065371+0000","last_clean":"2024-09-20T18:59:06.065371+0000","last_became_active":"2024-09-20T18:49:49.957558+0000","last_became_peered":"2024-09-20T18:49:49.957558+0000","last_unstale":"2024-09-20T18:59:06.065371+0000","last_undegraded":"2024-09-20T18:59:06.065371+0000","last_fullsized":"2024-09-20T18:59:06.065371+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:13.577699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:13.577699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:43.163602+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":434,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.761742+0000","last_change":"2024-09-20T18:53:47.370152+0000","last_active":"2024-09-20T18:59:10.761742+0000","last_peered":"2024-09-20T18:59:10.761742+0000","last_clean":"2024-09-20T18:59:10.761742+0000","last_became_active":"2024-09-20T18:49:51.970798+0000","last_became_peered":"2024-09-20T18:49:51.970798+0000","last_unstale":"2024-09-20T18:59:10.761742+0000","last_undegraded":"2024-09-20T18:59:10.761742+0000","last_fullsized":"2024-09-20T18:59:10.761742+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:47.370108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:47.370108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:38.509312+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.4","version":"262'3101","reported_seq":3144,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.423034+0000","last_change":"2024-09-20T18:58:33.846071+0000","last_active":"2024-09-20T18:59:06.423034+0000","last_peered":"2024-09-20T18:59:06.423034+0000","last_clean":"2024-09-20T18:59:06.423034+0000","last_became_active":"2024-09-20T18:58:33.845736+0000","last_became_peered":"2024-09-20T18:58:33.845736+0000","last_unstale":"2024-09-20T18:59:06.423034+0000","last_undegraded":"2024-09-20T18:59:06.423034+0000","last_fullsized":"2024-09-20T18:59:06.423034+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":3101,"log_dups_size":0,"ondisk_log_size":3101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:36.696830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":203227136,"num_objects":3101,"num_object_clones":0,"num_object_copies":9303,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3101,"num_write_kb":198464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":458,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.752913+0000","last_change":"2024-09-20T18:53:52.898818+0000","last_active":"2024-09-20T18:59:10.752913+0000","last_peered":"2024-09-20T18:59:10.752913+0000","last_clean":"2024-09-20T18:59:10.752913+0000","last_became_active":"2024-09-20T18:49:47.949116+0000","last_became_peered":"2024-09-20T18:49:47.949116+0000","last_unstale":"2024-09-20T18:59:10.752913+0000","last_undegraded":"2024-09-20T18:59:10.752913+0000","last_fullsized":"2024-09-20T18:59:10.752913+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:53:52.898760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:52.898760+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:48.557733+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":446,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.762090+0000","last_change":"2024-09-20T18:54:52.650798+0000","last_active":"2024-09-20T18:59:10.762090+0000","last_peered":"2024-09-20T18:59:10.762090+0000","last_clean":"2024-09-20T18:59:10.762090+0000","last_became_active":"2024-09-20T18:49:45.944858+0000","last_became_peered":"2024-09-20T18:49:45.944858+0000","last_unstale":"2024-09-20T18:59:10.762090+0000","last_undegraded":"2024-09-20T18:59:10.762090+0000","last_fullsized":"2024-09-20T18:59:10.762090+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:52.650756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:52.650756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:50.597183+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":438,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.064976+0000","last_change":"2024-09-20T18:58:33.180357+0000","last_active":"2024-09-20T18:59:06.064976+0000","last_peered":"2024-09-20T18:59:06.064976+0000","last_clean":"2024-09-20T18:59:06.064976+0000","last_became_active":"2024-09-20T18:49:49.954944+0000","last_became_peered":"2024-09-20T18:49:49.954944+0000","last_unstale":"2024-09-20T18:59:06.064976+0000","last_undegraded":"2024-09-20T18:59:06.064976+0000","last_fullsized":"2024-09-20T18:59:06.064976+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:33.180309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:33.180309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:34.621967+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":431,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.071997+0000","last_change":"2024-09-20T18:58:24.162921+0000","last_active":"2024-09-20T18:59:06.071997+0000","last_peered":"2024-09-20T18:59:06.071997+0000","last_clean":"2024-09-20T18:59:06.071997+0000","last_became_active":"2024-09-20T18:49:51.972970+0000","last_became_peered":"2024-09-20T18:49:51.972970+0000","last_unstale":"2024-09-20T18:59:06.071997+0000","last_undegraded":"2024-09-20T18:59:06.071997+0000","last_fullsized":"2024-09-20T18:59:06.071997+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:24.162879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:24.162879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:46.703799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"262'3245","reported_seq":3287,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:07.163801+0000","last_change":"2024-09-20T18:58:33.821228+0000","last_active":"2024-09-20T18:59:07.163801+0000","last_peered":"2024-09-20T18:59:07.163801+0000","last_clean":"2024-09-20T18:59:07.163801+0000","last_became_active":"2024-09-20T18:58:33.819949+0000","last_became_peered":"2024-09-20T18:58:33.819949+0000","last_unstale":"2024-09-20T18:59:07.163801+0000","last_undegraded":"2024-09-20T18:59:07.163801+0000","last_fullsized":"2024-09-20T18:59:07.163801+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":3245,"log_dups_size":0,"ondisk_log_size":3245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:38.576354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":212664320,"num_objects":3245,"num_object_clones":0,"num_object_copies":9735,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3245,"num_write_kb":207680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":508,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-20T18:59:05.108534+0000","last_change":"2024-09-20T18:59:05.108455+0000","last_active":"2024-09-20T18:59:05.108534+0000","last_peered":"2024-09-20T18:59:05.108534+0000","last_clean":"2024-09-20T18:59:05.108534+0000","last_became_active":"2024-09-20T18:59:05.083726+0000","last_became_peered":"2024-09-20T18:59:05.083726+0000","last_unstale":"2024-09-20T18:59:05.108534+0000","last_undegraded":"2024-09-20T18:59:05.108534+0000","last_fullsized":"2024-09-20T18:59:05.108534+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:58:30.522129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:30.522129+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:45.758905+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":474,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.762020+0000","last_change":"2024-09-20T18:58:57.563127+0000","last_active":"2024-09-20T18:59:10.762020+0000","last_peered":"2024-09-20T18:59:10.762020+0000","last_clean":"2024-09-20T18:59:10.762020+0000","last_became_active":"2024-09-20T18:49:45.944256+0000","last_became_peered":"2024-09-20T18:49:45.944256+0000","last_unstale":"2024-09-20T18:59:10.762020+0000","last_undegraded":"2024-09-20T18:59:10.762020+0000","last_fullsized":"2024-09-20T18:59:10.762020+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-20T18:58:57.563058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:57.563058+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:20.547993+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"262'10","reported_seq":573,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.763147+0000","last_change":"2024-09-20T18:59:05.111302+0000","last_active":"2024-09-20T18:59:10.763147+0000","last_peered":"2024-09-20T18:59:10.763147+0000","last_clean":"2024-09-20T18:59:10.763147+0000","last_became_active":"2024-09-20T18:59:05.090811+0000","last_became_peered":"2024-09-20T18:59:05.090811+0000","last_unstale":"2024-09-20T18:59:10.763147+0000","last_undegraded":"2024-09-20T18:59:10.763147+0000","last_fullsized":"2024-09-20T18:59:10.763147+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'9","last_scrub_stamp":"2024-09-20T18:58:27.521051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:27.521051+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:30.505818+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":443,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-20T18:59:05.087414+0000","last_change":"2024-09-20T18:59:05.087336+0000","last_active":"2024-09-20T18:59:05.087414+0000","last_peered":"2024-09-20T18:59:05.087414+0000","last_clean":"2024-09-20T18:59:05.087414+0000","last_became_active":"2024-09-20T18:59:05.087005+0000","last_became_peered":"2024-09-20T18:59:05.087005+0000","last_unstale":"2024-09-20T18:59:05.087414+0000","last_undegraded":"2024-09-20T18:59:05.087414+0000","last_fullsized":"2024-09-20T18:59:05.087414+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:05.523359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:05.523359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:32.510495+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.773963+0000","last_change":"2024-09-20T18:59:05.085844+0000","last_active":"2024-09-20T18:59:10.773963+0000","last_peered":"2024-09-20T18:59:10.773963+0000","last_clean":"2024-09-20T18:59:10.773963+0000","last_became_active":"2024-09-20T18:59:05.085427+0000","last_became_peered":"2024-09-20T18:59:05.085427+0000","last_unstale":"2024-09-20T18:59:10.773963+0000","last_undegraded":"2024-09-20T18:59:10.773963+0000","last_fullsized":"2024-09-20T18:59:10.773963+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:26.721735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:26.721735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":447,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.076600+0000","last_change":"2024-09-20T18:58:09.063555+0000","last_active":"2024-09-20T18:59:06.076600+0000","last_peered":"2024-09-20T18:59:06.076600+0000","last_clean":"2024-09-20T18:59:06.076600+0000","last_became_active":"2024-09-20T18:49:48.500662+0000","last_became_peered":"2024-09-20T18:49:48.500662+0000","last_unstale":"2024-09-20T18:59:06.076600+0000","last_undegraded":"2024-09-20T18:59:06.076600+0000","last_fullsized":"2024-09-20T18:59:06.076600+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-20T18:58:09.063481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:09.063481+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:21.861420+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":432,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-20T18:59:05.077594+0000","last_change":"2024-09-20T18:58:28.460106+0000","last_active":"2024-09-20T18:59:05.077594+0000","last_peered":"2024-09-20T18:59:05.077594+0000","last_clean":"2024-09-20T18:59:05.077594+0000","last_became_active":"2024-09-20T18:49:49.956749+0000","last_became_peered":"2024-09-20T18:49:49.956749+0000","last_unstale":"2024-09-20T18:59:05.077594+0000","last_undegraded":"2024-09-20T18:59:05.077594+0000","last_fullsized":"2024-09-20T18:59:05.077594+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:28.460054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:28.460054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:47.283266+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":447,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.762969+0000","last_change":"2024-09-20T18:59:05.082737+0000","last_active":"2024-09-20T18:59:10.762969+0000","last_peered":"2024-09-20T18:59:10.762969+0000","last_clean":"2024-09-20T18:59:10.762969+0000","last_became_active":"2024-09-20T18:59:05.082018+0000","last_became_peered":"2024-09-20T18:59:05.082018+0000","last_unstale":"2024-09-20T18:59:10.762969+0000","last_undegraded":"2024-09-20T18:59:10.762969+0000","last_fullsized":"2024-09-20T18:59:10.762969+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.637739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.637739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":447,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.067701+0000","last_change":"2024-09-20T18:54:44.830872+0000","last_active":"2024-09-20T18:59:06.067701+0000","last_peered":"2024-09-20T18:59:06.067701+0000","last_clean":"2024-09-20T18:59:06.067701+0000","last_became_active":"2024-09-20T18:49:48.500803+0000","last_became_peered":"2024-09-20T18:49:48.500803+0000","last_unstale":"2024-09-20T18:59:06.067701+0000","last_undegraded":"2024-09-20T18:59:06.067701+0000","last_fullsized":"2024-09-20T18:59:06.067701+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:44.830790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.830790+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:46.954072+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":460,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.763424+0000","last_change":"2024-09-20T18:59:05.091318+0000","last_active":"2024-09-20T18:59:10.763424+0000","last_peered":"2024-09-20T18:59:10.763424+0000","last_clean":"2024-09-20T18:59:10.763424+0000","last_became_active":"2024-09-20T18:59:05.090805+0000","last_became_peered":"2024-09-20T18:59:05.090805+0000","last_unstale":"2024-09-20T18:59:10.763424+0000","last_undegraded":"2024-09-20T18:59:10.763424+0000","last_fullsized":"2024-09-20T18:59:10.763424+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:59.589891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:59.589891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:14.124488+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":434,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.072314+0000","last_change":"2024-09-20T18:57:46.898929+0000","last_active":"2024-09-20T18:59:06.072314+0000","last_peered":"2024-09-20T18:59:06.072314+0000","last_clean":"2024-09-20T18:59:06.072314+0000","last_became_active":"2024-09-20T18:49:49.961947+0000","last_became_peered":"2024-09-20T18:49:49.961947+0000","last_unstale":"2024-09-20T18:59:06.072314+0000","last_undegraded":"2024-09-20T18:59:06.072314+0000","last_fullsized":"2024-09-20T18:59:06.072314+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:46.898862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:46.898862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:06.640666+0000","scrub_duration":38,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"66'1","reported_seq":442,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.065460+0000","last_change":"2024-09-20T18:58:30.171382+0000","last_active":"2024-09-20T18:59:06.065460+0000","last_peered":"2024-09-20T18:59:06.065460+0000","last_clean":"2024-09-20T18:59:06.065460+0000","last_became_active":"2024-09-20T18:49:51.965121+0000","last_became_peered":"2024-09-20T18:49:51.965121+0000","last_unstale":"2024-09-20T18:59:06.065460+0000","last_undegraded":"2024-09-20T18:59:06.065460+0000","last_fullsized":"2024-09-20T18:59:06.065460+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T18:58:30.171329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:30.171329+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:33.520988+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":444,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-20T18:59:05.080646+0000","last_change":"2024-09-20T18:57:59.430353+0000","last_active":"2024-09-20T18:59:05.080646+0000","last_peered":"2024-09-20T18:59:05.080646+0000","last_clean":"2024-09-20T18:59:05.080646+0000","last_became_active":"2024-09-20T18:49:48.501011+0000","last_became_peered":"2024-09-20T18:49:48.501011+0000","last_unstale":"2024-09-20T18:59:05.080646+0000","last_undegraded":"2024-09-20T18:59:05.080646+0000","last_fullsized":"2024-09-20T18:59:05.080646+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:57:59.430294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:59.430294+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:21.333603+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":443,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.067517+0000","last_change":"2024-09-20T18:55:27.028184+0000","last_active":"2024-09-20T18:59:06.067517+0000","last_peered":"2024-09-20T18:59:06.067517+0000","last_clean":"2024-09-20T18:59:06.067517+0000","last_became_active":"2024-09-20T18:49:45.938511+0000","last_became_peered":"2024-09-20T18:49:45.938511+0000","last_unstale":"2024-09-20T18:59:06.067517+0000","last_undegraded":"2024-09-20T18:59:06.067517+0000","last_fullsized":"2024-09-20T18:59:06.067517+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:27.028126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:27.028126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":555,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.071030+0000","last_change":"2024-09-20T18:55:27.284156+0000","last_active":"2024-09-20T18:59:06.071030+0000","last_peered":"2024-09-20T18:59:06.071030+0000","last_clean":"2024-09-20T18:59:06.071030+0000","last_became_active":"2024-09-20T18:49:49.960550+0000","last_became_peered":"2024-09-20T18:49:49.960550+0000","last_unstale":"2024-09-20T18:59:06.071030+0000","last_undegraded":"2024-09-20T18:59:06.071030+0000","last_fullsized":"2024-09-20T18:59:06.071030+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:27.284086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:27.284086+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:45.094499+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":434,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.753669+0000","last_change":"2024-09-20T18:55:12.348079+0000","last_active":"2024-09-20T18:59:10.753669+0000","last_peered":"2024-09-20T18:59:10.753669+0000","last_clean":"2024-09-20T18:59:10.753669+0000","last_became_active":"2024-09-20T18:49:51.966254+0000","last_became_peered":"2024-09-20T18:49:51.966254+0000","last_unstale":"2024-09-20T18:59:10.753669+0000","last_undegraded":"2024-09-20T18:59:10.753669+0000","last_fullsized":"2024-09-20T18:59:10.753669+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:12.347993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:12.347993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:52.541239+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":464,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.071922+0000","last_change":"2024-09-20T18:57:20.701592+0000","last_active":"2024-09-20T18:59:06.071922+0000","last_peered":"2024-09-20T18:59:06.071922+0000","last_clean":"2024-09-20T18:59:06.071922+0000","last_became_active":"2024-09-20T18:49:58.997494+0000","last_became_peered":"2024-09-20T18:49:58.997494+0000","last_unstale":"2024-09-20T18:59:06.071922+0000","last_undegraded":"2024-09-20T18:59:06.071922+0000","last_fullsized":"2024-09-20T18:59:06.071922+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:57:20.701543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:20.701543+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":446,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.762414+0000","last_change":"2024-09-20T18:54:57.632458+0000","last_active":"2024-09-20T18:59:10.762414+0000","last_peered":"2024-09-20T18:59:10.762414+0000","last_clean":"2024-09-20T18:59:10.762414+0000","last_became_active":"2024-09-20T18:49:45.949492+0000","last_became_peered":"2024-09-20T18:49:45.949492+0000","last_unstale":"2024-09-20T18:59:10.762414+0000","last_undegraded":"2024-09-20T18:59:10.762414+0000","last_fullsized":"2024-09-20T18:59:10.762414+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.632380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.632380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:44.496241+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":552,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-20T18:59:05.080222+0000","last_change":"2024-09-20T18:55:28.302353+0000","last_active":"2024-09-20T18:59:05.080222+0000","last_peered":"2024-09-20T18:59:05.080222+0000","last_clean":"2024-09-20T18:59:05.080222+0000","last_became_active":"2024-09-20T18:49:49.962243+0000","last_became_peered":"2024-09-20T18:49:49.962243+0000","last_unstale":"2024-09-20T18:59:05.080222+0000","last_undegraded":"2024-09-20T18:59:05.080222+0000","last_fullsized":"2024-09-20T18:59:05.080222+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:28.302303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:28.302303+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:17.501674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":439,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.067304+0000","last_change":"2024-09-20T18:58:55.232948+0000","last_active":"2024-09-20T18:59:06.067304+0000","last_peered":"2024-09-20T18:59:06.067304+0000","last_clean":"2024-09-20T18:59:06.067304+0000","last_became_active":"2024-09-20T18:49:51.969093+0000","last_became_peered":"2024-09-20T18:49:51.969093+0000","last_unstale":"2024-09-20T18:59:06.067304+0000","last_undegraded":"2024-09-20T18:59:06.067304+0000","last_fullsized":"2024-09-20T18:59:06.067304+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:55.232883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:55.232883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:56.795382+0000","scrub_duration":59,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":500,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.752622+0000","last_change":"2024-09-20T18:59:05.117294+0000","last_active":"2024-09-20T18:59:10.752622+0000","last_peered":"2024-09-20T18:59:10.752622+0000","last_clean":"2024-09-20T18:59:10.752622+0000","last_became_active":"2024-09-20T18:59:05.081917+0000","last_became_peered":"2024-09-20T18:59:05.081917+0000","last_unstale":"2024-09-20T18:59:10.752622+0000","last_undegraded":"2024-09-20T18:59:10.752622+0000","last_fullsized":"2024-09-20T18:59:10.752622+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:57:50.899514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:50.899514+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":453,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.753286+0000","last_change":"2024-09-20T18:59:06.310100+0000","last_active":"2024-09-20T18:59:10.753286+0000","last_peered":"2024-09-20T18:59:10.753286+0000","last_clean":"2024-09-20T18:59:10.753286+0000","last_became_active":"2024-09-20T18:59:05.073087+0000","last_became_peered":"2024-09-20T18:59:05.073087+0000","last_unstale":"2024-09-20T18:59:10.753286+0000","last_undegraded":"2024-09-20T18:59:10.753286+0000","last_fullsized":"2024-09-20T18:59:10.753286+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:06.310027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:59:06.310027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:35.049125+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":426,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.073278+0000","last_change":"2024-09-20T18:57:11.986605+0000","last_active":"2024-09-20T18:59:06.073278+0000","last_peered":"2024-09-20T18:59:06.073278+0000","last_clean":"2024-09-20T18:59:06.073278+0000","last_became_active":"2024-09-20T18:49:49.957067+0000","last_became_peered":"2024-09-20T18:49:49.957067+0000","last_unstale":"2024-09-20T18:59:06.073278+0000","last_undegraded":"2024-09-20T18:59:06.073278+0000","last_fullsized":"2024-09-20T18:59:06.073278+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:11.986528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:11.986528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:41.091844+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":422,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.070242+0000","last_change":"2024-09-20T18:57:16.789412+0000","last_active":"2024-09-20T18:59:06.070242+0000","last_peered":"2024-09-20T18:59:06.070242+0000","last_clean":"2024-09-20T18:59:06.070242+0000","last_became_active":"2024-09-20T18:49:51.966915+0000","last_became_peered":"2024-09-20T18:49:51.966915+0000","last_unstale":"2024-09-20T18:59:06.070242+0000","last_undegraded":"2024-09-20T18:59:06.070242+0000","last_fullsized":"2024-09-20T18:59:06.070242+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:16.789095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:16.789095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":496,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.073522+0000","last_change":"2024-09-20T18:59:05.103863+0000","last_active":"2024-09-20T18:59:06.073522+0000","last_peered":"2024-09-20T18:59:06.073522+0000","last_clean":"2024-09-20T18:59:06.073522+0000","last_became_active":"2024-09-20T18:59:05.085230+0000","last_became_peered":"2024-09-20T18:59:05.085230+0000","last_unstale":"2024-09-20T18:59:06.073522+0000","last_undegraded":"2024-09-20T18:59:06.073522+0000","last_fullsized":"2024-09-20T18:59:06.073522+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:58:28.035450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:28.035450+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:44.274290+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":437,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.763308+0000","last_change":"2024-09-20T18:54:42.487465+0000","last_active":"2024-09-20T18:59:10.763308+0000","last_peered":"2024-09-20T18:59:10.763308+0000","last_clean":"2024-09-20T18:59:10.763308+0000","last_became_active":"2024-09-20T18:49:45.944357+0000","last_became_peered":"2024-09-20T18:49:45.944357+0000","last_unstale":"2024-09-20T18:59:10.763308+0000","last_undegraded":"2024-09-20T18:59:10.763308+0000","last_fullsized":"2024-09-20T18:59:10.763308+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:42.487403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:42.487403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:51.577354+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":434,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.072073+0000","last_change":"2024-09-20T18:57:57.915090+0000","last_active":"2024-09-20T18:59:06.072073+0000","last_peered":"2024-09-20T18:59:06.072073+0000","last_clean":"2024-09-20T18:59:06.072073+0000","last_became_active":"2024-09-20T18:49:49.962523+0000","last_became_peered":"2024-09-20T18:49:49.962523+0000","last_unstale":"2024-09-20T18:59:06.072073+0000","last_undegraded":"2024-09-20T18:59:06.072073+0000","last_fullsized":"2024-09-20T18:59:06.072073+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:57.914913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:57.914913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":431,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.067180+0000","last_change":"2024-09-20T18:54:26.882441+0000","last_active":"2024-09-20T18:59:06.067180+0000","last_peered":"2024-09-20T18:59:06.067180+0000","last_clean":"2024-09-20T18:59:06.067180+0000","last_became_active":"2024-09-20T18:49:51.967313+0000","last_became_peered":"2024-09-20T18:49:51.967313+0000","last_unstale":"2024-09-20T18:59:06.067180+0000","last_undegraded":"2024-09-20T18:59:06.067180+0000","last_fullsized":"2024-09-20T18:59:06.067180+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:26.882375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:26.882375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:40.010268+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":496,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.070757+0000","last_change":"2024-09-20T18:59:05.096414+0000","last_active":"2024-09-20T18:59:06.070757+0000","last_peered":"2024-09-20T18:59:06.070757+0000","last_clean":"2024-09-20T18:59:06.070757+0000","last_became_active":"2024-09-20T18:59:05.075390+0000","last_became_peered":"2024-09-20T18:59:05.075390+0000","last_unstale":"2024-09-20T18:59:06.070757+0000","last_undegraded":"2024-09-20T18:59:06.070757+0000","last_fullsized":"2024-09-20T18:59:06.070757+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:57:54.862329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:54.862329+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":448,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-09-20T18:59:05.077657+0000","last_change":"2024-09-20T18:58:06.479419+0000","last_active":"2024-09-20T18:59:05.077657+0000","last_peered":"2024-09-20T18:59:05.077657+0000","last_clean":"2024-09-20T18:59:05.077657+0000","last_became_active":"2024-09-20T18:49:45.942105+0000","last_became_peered":"2024-09-20T18:49:45.942105+0000","last_unstale":"2024-09-20T18:59:05.077657+0000","last_undegraded":"2024-09-20T18:59:05.077657+0000","last_fullsized":"2024-09-20T18:59:05.077657+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:06.479364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:06.479364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:26.337228+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":452,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.761352+0000","last_change":"2024-09-20T18:59:05.081655+0000","last_active":"2024-09-20T18:59:10.761352+0000","last_peered":"2024-09-20T18:59:10.761352+0000","last_clean":"2024-09-20T18:59:10.761352+0000","last_became_active":"2024-09-20T18:59:05.081320+0000","last_became_peered":"2024-09-20T18:59:05.081320+0000","last_unstale":"2024-09-20T18:59:10.761352+0000","last_undegraded":"2024-09-20T18:59:10.761352+0000","last_fullsized":"2024-09-20T18:59:10.761352+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:46.329547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:46.329547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":436,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.072198+0000","last_change":"2024-09-20T18:59:05.081584+0000","last_active":"2024-09-20T18:59:06.072198+0000","last_peered":"2024-09-20T18:59:06.072198+0000","last_clean":"2024-09-20T18:59:06.072198+0000","last_became_active":"2024-09-20T18:59:05.081179+0000","last_became_peered":"2024-09-20T18:59:05.081179+0000","last_unstale":"2024-09-20T18:59:06.072198+0000","last_undegraded":"2024-09-20T18:59:06.072198+0000","last_fullsized":"2024-09-20T18:59:06.072198+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:28.165712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:28.165712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:49.800757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":471,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.070332+0000","last_change":"2024-09-20T18:59:05.098374+0000","last_active":"2024-09-20T18:59:06.070332+0000","last_peered":"2024-09-20T18:59:06.070332+0000","last_clean":"2024-09-20T18:59:06.070332+0000","last_became_active":"2024-09-20T18:59:05.076365+0000","last_became_peered":"2024-09-20T18:59:05.076365+0000","last_unstale":"2024-09-20T18:59:06.070332+0000","last_undegraded":"2024-09-20T18:59:06.070332+0000","last_fullsized":"2024-09-20T18:59:06.070332+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-20T18:55:20.282193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:20.282193+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":443,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.068025+0000","last_change":"2024-09-20T18:54:55.900487+0000","last_active":"2024-09-20T18:59:06.068025+0000","last_peered":"2024-09-20T18:59:06.068025+0000","last_clean":"2024-09-20T18:59:06.068025+0000","last_became_active":"2024-09-20T18:49:45.940988+0000","last_became_peered":"2024-09-20T18:49:45.940988+0000","last_unstale":"2024-09-20T18:59:06.068025+0000","last_undegraded":"2024-09-20T18:59:06.068025+0000","last_fullsized":"2024-09-20T18:59:06.068025+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:55.899611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:55.899611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:50.986844+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":16,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.773860+0000","last_change":"2024-09-20T18:59:05.086874+0000","last_active":"2024-09-20T18:59:10.773860+0000","last_peered":"2024-09-20T18:59:10.773860+0000","last_clean":"2024-09-20T18:59:10.773860+0000","last_became_active":"2024-09-20T18:59:05.086570+0000","last_became_peered":"2024-09-20T18:59:05.086570+0000","last_unstale":"2024-09-20T18:59:10.773860+0000","last_undegraded":"2024-09-20T18:59:10.773860+0000","last_fullsized":"2024-09-20T18:59:10.773860+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:52.929828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:52.929828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":425,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.761718+0000","last_change":"2024-09-20T18:58:22.558181+0000","last_active":"2024-09-20T18:59:10.761718+0000","last_peered":"2024-09-20T18:59:10.761718+0000","last_clean":"2024-09-20T18:59:10.761718+0000","last_became_active":"2024-09-20T18:49:51.968662+0000","last_became_peered":"2024-09-20T18:49:51.968662+0000","last_unstale":"2024-09-20T18:59:10.761718+0000","last_undegraded":"2024-09-20T18:59:10.761718+0000","last_fullsized":"2024-09-20T18:59:10.761718+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:22.558131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:22.558131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:46.688905+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":414,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.067752+0000","last_change":"2024-09-20T18:57:10.970843+0000","last_active":"2024-09-20T18:59:06.067752+0000","last_peered":"2024-09-20T18:59:06.067752+0000","last_clean":"2024-09-20T18:59:06.067752+0000","last_became_active":"2024-09-20T18:49:51.968216+0000","last_became_peered":"2024-09-20T18:49:51.968216+0000","last_unstale":"2024-09-20T18:59:06.067752+0000","last_undegraded":"2024-09-20T18:59:06.067752+0000","last_fullsized":"2024-09-20T18:59:06.067752+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:10.970779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:10.970779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:46.067511+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":442,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.070622+0000","last_change":"2024-09-20T18:58:36.077005+0000","last_active":"2024-09-20T18:59:06.070622+0000","last_peered":"2024-09-20T18:59:06.070622+0000","last_clean":"2024-09-20T18:59:06.070622+0000","last_became_active":"2024-09-20T18:49:45.937992+0000","last_became_peered":"2024-09-20T18:49:45.937992+0000","last_unstale":"2024-09-20T18:59:06.070622+0000","last_undegraded":"2024-09-20T18:59:06.070622+0000","last_fullsized":"2024-09-20T18:59:06.070622+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:36.076937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:36.076937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:38.647424+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":28,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-20T18:59:10.774502+0000","last_change":"2024-09-20T18:59:05.116749+0000","last_active":"2024-09-20T18:59:10.774502+0000","last_peered":"2024-09-20T18:59:10.774502+0000","last_clean":"2024-09-20T18:59:10.774502+0000","last_became_active":"2024-09-20T18:59:05.089294+0000","last_became_peered":"2024-09-20T18:59:05.089294+0000","last_unstale":"2024-09-20T18:59:10.774502+0000","last_undegraded":"2024-09-20T18:59:10.774502+0000","last_fullsized":"2024-09-20T18:59:10.774502+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:54:44.719676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.719676+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":12,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":428,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-20T18:59:06.067454+0000","last_change":"2024-09-20T18:58:27.904636+0000","last_active":"2024-09-20T18:59:06.067454+0000","last_peered":"2024-09-20T18:59:06.067454+0000","last_clean":"2024-09-20T18:59:06.067454+0000","last_became_active":"2024-09-20T18:49:49.957582+0000","last_became_peered":"2024-09-20T18:49:49.957582+0000","last_unstale":"2024-09-20T18:59:06.067454+0000","last_undegraded":"2024-09-20T18:59:06.067454+0000","last_fullsized":"2024-09-20T18:59:06.067454+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:27.904592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:27.904592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:35.767269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":3401449472,"num_objects":51902,"num_object_clones":0,"num_object_copies":155706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":9445,"num_objects_unfound":0,"num_objects_dirty":51902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":51902,"num_write_kb":3321728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":3538944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":10336206848,"data_stored":10336206848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51902,"ondisk_log_size":51902,"up":48,"acting":45,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":695,"num_read_kb":452,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":2244,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":395,"ondisk_log_size":395,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":559,"num_read_kb":554,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266805,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2333752,"kb_used_data":2013800,"kb_used_omap":1,"kb_used_meta":319934,"kb_avail":91411400,"statfs":{"total":95995035648,"available":93605273600,"internally_reserved":0,"allocated":2062131200,"data_stored":2061497496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":327612874},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528444,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":892260,"kb_used_data":708372,"kb_used_omap":1,"kb_used_meta":183870,"kb_avail":92852892,"statfs":{"total":95995035648,"available":95081361408,"internally_reserved":0,"allocated":725372928,"data_stored":724759042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":188283334},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757376,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2129376,"kb_used_data":1809416,"kb_used_omap":1,"kb_used_meta":319934,"kb_avail":91615776,"statfs":{"total":95995035648,"available":93814554624,"internally_reserved":0,"allocated":1852841984,"data_stored":1852233266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":327612869},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986310,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1154244,"kb_used_data":866228,"kb_used_omap":1,"kb_used_meta":287998,"kb_avail":92590908,"statfs":{"total":95995035648,"available":94813089792,"internally_reserved":0,"allocated":887017472,"data_stored":886371885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":294910405},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182540,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2008124,"kb_used_data":1620132,"kb_used_omap":1,"kb_used_meta":387966,"kb_avail":91737028,"statfs":{"total":95995035648,"available":93938716672,"internally_reserved":0,"allocated":1659015168,"data_stored":1658377874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":397277639},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,2,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411473,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1236268,"kb_used_data":803864,"kb_used_omap":1,"kb_used_meta":432382,"kb_avail":92508884,"statfs":{"total":95995035648,"available":94729097216,"internally_reserved":0,"allocated":823156736,"data_stored":822539148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":442759621},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574999,"num_pgs":73,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2157072,"kb_used_data":1814908,"kb_used_omap":1,"kb_used_meta":342142,"kb_avail":91588080,"statfs":{"total":95995035648,"available":93786193920,"internally_reserved":0,"allocated":1858465792,"data_stored":1857805589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":350353861},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738525,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1396092,"kb_used_data":1026152,"kb_used_omap":1,"kb_used_meta":369918,"kb_avail":92349060,"statfs":{"total":95995035648,"available":94565437440,"internally_reserved":0,"allocated":1050779648,"data_stored":1050145640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":378796485},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1053163520,"data_stored":1053163520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1866727424,"data_stored":1866727424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":822870016,"data_stored":822870016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1676869632,"data_stored":1676869632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":893255680,"data_stored":893255680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1851523072,"data_stored":1851523072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":96600064,"data_stored":96600064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2075197440,"data_stored":2075197440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T18:59:15.834 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph pg force-recovery 9.b 9.0 2024-09-20T18:59:15.860 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:15.917 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:15.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:15.919 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:16.020 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:59:16.040 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:16.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:16.041 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:59:16.116 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T18:59:16.142 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:16.168 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:16.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:16.169 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:16.251 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 36 16 61684 61668 107.046 69.125 0.0243513 0.00933577 2024-09-20T18:59:16.269 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:16.313 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:16.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:16.315 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:16.415 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:16.475 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:16.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:16.477 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:16.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:16 smithi142 ceph-mon[32508]: pgmap v463: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 3.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 639 B/s rd, 49 MiB/s wr, 778 op/s; 8955/169626 objects misplaced (5.279%); 4.1 MiB/s, 66 objects/s recovering 2024-09-20T18:59:16.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:16 smithi142 ceph-mon[32508]: osdmap e265: 8 total, 8 up, 8 in 2024-09-20T18:59:16.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:16 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:59:16.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:16 smithi083 ceph-mon[36260]: pgmap v463: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 3.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 639 B/s rd, 49 MiB/s wr, 778 op/s; 8955/169626 objects misplaced (5.279%); 4.1 MiB/s, 66 objects/s recovering 2024-09-20T18:59:16.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:16 smithi083 ceph-mon[36260]: osdmap e265: 8 total, 8 up, 8 in 2024-09-20T18:59:16.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:16 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:59:16.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:16 smithi083 ceph-mon[29421]: pgmap v463: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 3.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 639 B/s rd, 49 MiB/s wr, 778 op/s; 8955/169626 objects misplaced (5.279%); 4.1 MiB/s, 66 objects/s recovering 2024-09-20T18:59:16.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:16 smithi083 ceph-mon[29421]: osdmap e265: 8 total, 8 up, 8 in 2024-09-20T18:59:16.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:16 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:59:16.578 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:59:16.603 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:16.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:16.604 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:59:16.705 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:16.739 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:16.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:16.741 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:16.842 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:59:16.874 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:16.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:16.875 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:59:16.976 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:59:17.014 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:17.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:17.015 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:59:17.116 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:17.150 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:17.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:17.151 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:17.251 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 37 16 62717 62701 105.898 64.5625 0.00760595 0.00943724 2024-09-20T18:59:17.253 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:17.266 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:59:16 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:59:17.294 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:17.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:17.295 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:17.395 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:17.416 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:17.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:17.417 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:17.518 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:17.560 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:17.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:17.561 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:17.662 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:17.695 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:17.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:17.696 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:17.797 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:59:17.829 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:17.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:17.830 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:59:17.931 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:17.959 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:17.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:17.960 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:18.061 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:59:18.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:18.105 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:18.105 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:59:18.205 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:59:18.235 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:18.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:18.236 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:59:18.251 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 38 16 63741 63725 104.795 64 0.0114493 0.00953649 2024-09-20T18:59:18.337 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:59:18.372 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:18.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:18.373 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:59:18.473 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:18.494 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:18.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:18.495 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:18.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:18 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:59:18.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:18 smithi142 ceph-mon[32508]: pgmap v465: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 3.7 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.6 KiB/s rd, 99 MiB/s wr, 1.59k op/s; 8633/182967 objects misplaced (4.718%); 8.1 MiB/s, 130 objects/s recovering 2024-09-20T18:59:18.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:18 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:59:18.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:18 smithi083 ceph-mon[29421]: pgmap v465: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 3.7 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.6 KiB/s rd, 99 MiB/s wr, 1.59k op/s; 8633/182967 objects misplaced (4.718%); 8.1 MiB/s, 130 objects/s recovering 2024-09-20T18:59:18.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:18 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:59:18.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:18 smithi083 ceph-mon[36260]: pgmap v465: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 3.7 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.6 KiB/s rd, 99 MiB/s wr, 1.59k op/s; 8633/182967 objects misplaced (4.718%); 8.1 MiB/s, 130 objects/s recovering 2024-09-20T18:59:18.596 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:18.623 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:18.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:18.624 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:18.724 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:59:18.755 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:18.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:18.756 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:59:18.857 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:59:18.890 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:18.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:18.891 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:59:18.992 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:59:19.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:19.023 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:19.023 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:59:19.124 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:19.145 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:19.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:19.146 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:19.246 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:59:19.251 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 39 16 64814 64798 103.827 67.0625 0.0072766 0.0096258 2024-09-20T18:59:19.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:19.282 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:19.283 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:59:19.383 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:59:19.406 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:19.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:19.407 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:59:19.508 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:59:19.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:19.543 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:19.543 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:59:19.644 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:19.677 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:19.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:19.678 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:19.779 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:19.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:19.810 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:19.811 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:19.914 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:59:19.953 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:19.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:19.954 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:59:20.054 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:20.090 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:20.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:20.091 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:20.192 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:59:20.241 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:20.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:20.242 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:59:20.252 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:2024-09-20T18:59:20.242865+0000 min lat: 0.00133037 max lat: 2.15034 avg lat: 0.0097049 2024-09-20T18:59:20.252 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-20T18:59:20.252 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 40 16 65939 65923 102.989 70.3125 0.0308168 0.0097049 2024-09-20T18:59:20.282 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_historic_ops 2024-09-20T18:59:20.343 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:20.374 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:20.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:20.375 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:20.476 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:20.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:20.507 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:20.507 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:20.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:20 smithi142 ceph-mon[32508]: pgmap v466: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 3.8 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 639 B/s rd, 78 MiB/s wr, 1.25k op/s; 8445/186276 objects misplaced (4.534%); 7.8 MiB/s, 124 objects/s recovering 2024-09-20T18:59:20.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:20 smithi083 ceph-mon[29421]: pgmap v466: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 3.8 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 639 B/s rd, 78 MiB/s wr, 1.25k op/s; 8445/186276 objects misplaced (4.534%); 7.8 MiB/s, 124 objects/s recovering 2024-09-20T18:59:20.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:20 smithi083 ceph-mon[36260]: pgmap v466: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 3.8 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 639 B/s rd, 78 MiB/s wr, 1.25k op/s; 8445/186276 objects misplaced (4.534%); 7.8 MiB/s, 124 objects/s recovering 2024-09-20T18:59:20.608 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:59:20.657 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:20.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:20.658 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:59:20.759 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:59:20.790 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:20.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:20.791 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:59:20.892 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:59:20.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:20.928 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:20.928 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:59:21.029 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:59:21.047 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:59:20 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:59:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:59:21.076 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:21.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:21.077 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:59:21.178 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:21.208 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:21.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:21.209 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:21.252 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 41 16 66971 66955 102.05 64.5 0.00701869 0.00979126 2024-09-20T18:59:21.309 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:21.345 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:21.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:21.346 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:21.447 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:59:21.480 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:21.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:21.481 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:59:21.582 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:59:21.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:21.622 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:21.622 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:59:21.723 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:59:21.749 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:21.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:21.750 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:59:21.851 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:21.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:21.883 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:21.883 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:21.984 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:22.005 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:22.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:22.007 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:22.016 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:21 smithi142 ceph-mon[32508]: pgmap v467: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 3.8 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 51 MiB/s wr, 815 op/s; 8445/189174 objects misplaced (4.464%); 4.0 MiB/s, 63 objects/s recovering 2024-09-20T18:59:22.047 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:21 smithi083 ceph-mon[29421]: pgmap v467: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 3.8 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 51 MiB/s wr, 815 op/s; 8445/189174 objects misplaced (4.464%); 4.0 MiB/s, 63 objects/s recovering 2024-09-20T18:59:22.047 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:21 smithi083 ceph-mon[36260]: pgmap v467: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 3.8 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 51 MiB/s wr, 815 op/s; 8445/189174 objects misplaced (4.464%); 4.0 MiB/s, 63 objects/s recovering 2024-09-20T18:59:22.107 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:22.133 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:22.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:22.134 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:22.234 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:22.252 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 42 16 67873 67857 100.963 56.375 0.013944 0.00989815 2024-09-20T18:59:22.266 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:22.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:22.267 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:22.367 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:22.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:22.393 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:22.394 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:22.494 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:59:22.537 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:22.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:22.538 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:59:22.638 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:59:22.673 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:22.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:22.674 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:59:22.775 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:59:22.799 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:22.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:22.800 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:59:22.901 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:22.936 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:22.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:22.937 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:23.038 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:23.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:23.077 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:23.077 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:23.178 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:23.215 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:23.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:23.216 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:23.252 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 43 16 68837 68821 100.016 60.25 0.0371747 0.0099925 2024-09-20T18:59:23.316 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:23.361 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:23.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:23.362 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:23.463 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:23.498 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:23.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:23.499 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:23.600 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:23.632 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:23.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:23.633 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:23.734 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:23.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:23.771 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:23.771 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:23.872 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:59:23.941 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:59:23.951 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:59:23.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:23.952 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:23.952 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:59:23.961 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:59:24.053 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:24.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:24.099 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:24.099 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:24.200 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:24.242 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:24.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:24.243 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:24.252 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 44 16 69909 69893 99.2652 67 0.0274664 0.010069 2024-09-20T18:59:24.345 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:24.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:24.377 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:24.378 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:24.399 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:24 smithi083 ceph-mon[29421]: pgmap v468: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 4.1 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.0 KiB/s rd, 69 MiB/s wr, 1.10k op/s; 7947/202041 objects misplaced (3.933%); 6.4 MiB/s, 102 objects/s recovering 2024-09-20T18:59:24.400 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:24 smithi083 ceph-mon[36260]: pgmap v468: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 4.1 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.0 KiB/s rd, 69 MiB/s wr, 1.10k op/s; 7947/202041 objects misplaced (3.933%); 6.4 MiB/s, 102 objects/s recovering 2024-09-20T18:59:24.478 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:24.509 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:24.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:24.510 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:24.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:24 smithi142 ceph-mon[32508]: pgmap v468: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 4.1 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.0 KiB/s rd, 69 MiB/s wr, 1.10k op/s; 7947/202041 objects misplaced (3.933%); 6.4 MiB/s, 102 objects/s recovering 2024-09-20T18:59:24.611 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:24.657 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:24.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:24.658 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:24.759 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:59:24.786 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:24.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:24.787 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:59:24.888 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:59:24.920 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:24.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:24.921 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:59:25.021 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:25.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:25.064 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:25.064 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:25.164 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:59:25.204 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:25.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:25.205 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:59:25.252 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 45 15 70945 70930 98.4995 64.8125 0.017143 0.010147 2024-09-20T18:59:25.306 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:59:25.334 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:25.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:25.335 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:59:25.436 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:59:25.474 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:25.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:25.476 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:59:25.576 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:25.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:25.612 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:25.612 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:25.713 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:25.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:25.757 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:25.758 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:25.824 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T18:59:25.833 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T18:59:25.833 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-20T18:59:25.833 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-20T18:59:25.833 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-20T18:59:25.834 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.834 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:25327 7.3 7:cbe7b718:::benchmark_data_smithi083_70643_object25326:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:59:25.834 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.263028+0000", 2024-09-20T18:59:25.834 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 497.712799233, 2024-09-20T18:59:25.834 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.1801133, 2024-09-20T18:59:25.834 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:59:25.834 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:59:25.834 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:25.834 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:59:25.834 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:59:25.835 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:59:25.835 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25327 2024-09-20T18:59:25.835 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.835 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:59:25.835 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.835 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:59:25.835 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263028+0000", 2024-09-20T18:59:25.835 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:25.835 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.835 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.835 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:59:25.835 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263028+0000", 2024-09-20T18:59:25.835 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:25.835 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.836 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.836 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:59:25.836 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263029+0000", 2024-09-20T18:59:25.836 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1200000000000001e-06 2024-09-20T18:59:25.836 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.836 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.836 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:59:25.836 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263050+0000", 2024-09-20T18:59:25.836 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1019999999999999e-05 2024-09-20T18:59:25.836 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.836 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.836 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:59:25.836 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263060+0000", 2024-09-20T18:59:25.836 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0049e-05 2024-09-20T18:59:25.837 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.837 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.837 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:59:25.837 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263069+0000", 2024-09-20T18:59:25.837 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8999999999999995e-06 2024-09-20T18:59:25.837 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.837 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.837 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:59:25.837 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263770+0000", 2024-09-20T18:59:25.837 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00070160100000000005 2024-09-20T18:59:25.837 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.837 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.837 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:59:25.837 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263876+0000", 2024-09-20T18:59:25.838 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010538400000000001 2024-09-20T18:59:25.838 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.838 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.838 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,4", 2024-09-20T18:59:25.838 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263991+0000", 2024-09-20T18:59:25.838 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115151 2024-09-20T18:59:25.838 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.838 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.838 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:59:25.838 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.265911+0000", 2024-09-20T18:59:25.838 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0019197319999999999 2024-09-20T18:59:25.838 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.838 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.838 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:25.839 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.267644+0000", 2024-09-20T18:59:25.839 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0017335600000000001 2024-09-20T18:59:25.839 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.839 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.839 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:25.839 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443038+0000", 2024-09-20T18:59:25.839 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17539418100000001 2024-09-20T18:59:25.839 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.839 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.839 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:59:25.839 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443095+0000", 2024-09-20T18:59:25.839 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6972999999999998e-05 2024-09-20T18:59:25.839 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.839 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.840 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:59:25.840 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443141+0000", 2024-09-20T18:59:25.840 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5629e-05 2024-09-20T18:59:25.840 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:25.840 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:25.840 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:25.840 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.840 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.840 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:25329 7.3 7:c0dcc395:::benchmark_data_smithi083_70643_object25328:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:59:25.840 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.263576+0000", 2024-09-20T18:59:25.840 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 497.71225104299998, 2024-09-20T18:59:25.840 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17968018699999999, 2024-09-20T18:59:25.841 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:59:25.841 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:59:25.841 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:25.841 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:59:25.841 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:59:25.841 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:59:25.841 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25329 2024-09-20T18:59:25.841 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.841 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:59:25.841 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.841 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:59:25.841 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263576+0000", 2024-09-20T18:59:25.841 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:25.841 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.842 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.842 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:59:25.842 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263576+0000", 2024-09-20T18:59:25.842 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:25.842 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.842 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.842 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:59:25.842 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263577+0000", 2024-09-20T18:59:25.842 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.3799999999999996e-07 2024-09-20T18:59:25.842 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.843 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.843 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:59:25.843 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263599+0000", 2024-09-20T18:59:25.843 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2333000000000001e-05 2024-09-20T18:59:25.843 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.843 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.843 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:59:25.843 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263608+0000", 2024-09-20T18:59:25.843 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5690000000000005e-06 2024-09-20T18:59:25.843 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.843 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.843 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:59:25.843 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.263614+0000", 2024-09-20T18:59:25.843 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6449999999999999e-06 2024-09-20T18:59:25.844 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.844 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.844 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:59:25.844 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.264856+0000", 2024-09-20T18:59:25.844 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001241486 2024-09-20T18:59:25.844 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.844 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.844 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:59:25.844 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.264959+0000", 2024-09-20T18:59:25.844 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000103037 2024-09-20T18:59:25.844 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.844 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.844 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,4", 2024-09-20T18:59:25.844 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.265077+0000", 2024-09-20T18:59:25.844 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000118163 2024-09-20T18:59:25.845 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.845 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.845 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:59:25.845 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266326+0000", 2024-09-20T18:59:25.845 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001248929 2024-09-20T18:59:25.845 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.845 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.845 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:25.845 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.267696+0000", 2024-09-20T18:59:25.845 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001369924 2024-09-20T18:59:25.845 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.845 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.845 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:25.845 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443180+0000", 2024-09-20T18:59:25.845 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17548422 2024-09-20T18:59:25.846 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.846 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.846 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:59:25.846 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443222+0000", 2024-09-20T18:59:25.846 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2426999999999999e-05 2024-09-20T18:59:25.846 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.846 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.846 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:59:25.846 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443256+0000", 2024-09-20T18:59:25.846 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3615999999999998e-05 2024-09-20T18:59:25.846 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:25.846 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:25.846 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:25.846 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.846 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.847 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:25332 7.2 7:4a99327a:::benchmark_data_smithi083_70643_object25331:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:59:25.847 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.266138+0000", 2024-09-20T18:59:25.847 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 497.70968857899999, 2024-09-20T18:59:25.847 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17949907200000001, 2024-09-20T18:59:25.847 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:59:25.847 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:59:25.847 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:25.847 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:59:25.847 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:59:25.847 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:59:25.847 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25332 2024-09-20T18:59:25.847 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.848 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:59:25.848 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.848 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:59:25.848 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266138+0000", 2024-09-20T18:59:25.848 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:25.848 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.848 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.848 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:59:25.848 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266138+0000", 2024-09-20T18:59:25.848 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:25.848 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.848 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.848 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:59:25.849 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266140+0000", 2024-09-20T18:59:25.849 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1480000000000001e-06 2024-09-20T18:59:25.849 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.849 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.849 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:59:25.849 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266160+0000", 2024-09-20T18:59:25.849 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0908999999999999e-05 2024-09-20T18:59:25.849 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.849 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.849 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:59:25.849 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266170+0000", 2024-09-20T18:59:25.849 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1390000000000001e-06 2024-09-20T18:59:25.849 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.849 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.850 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:59:25.850 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266176+0000", 2024-09-20T18:59:25.850 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3740000000000003e-06 2024-09-20T18:59:25.850 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.850 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.850 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:59:25.850 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266234+0000", 2024-09-20T18:59:25.850 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8084999999999997e-05 2024-09-20T18:59:25.850 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.850 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.850 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:59:25.850 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266342+0000", 2024-09-20T18:59:25.850 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000108448 2024-09-20T18:59:25.850 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.851 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.851 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T18:59:25.851 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266470+0000", 2024-09-20T18:59:25.851 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012756700000000001 2024-09-20T18:59:25.851 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.851 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.851 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:25.851 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444956+0000", 2024-09-20T18:59:25.851 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.178485641 2024-09-20T18:59:25.851 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.851 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.851 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:25.851 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445026+0000", 2024-09-20T18:59:25.851 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0778999999999994e-05 2024-09-20T18:59:25.852 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.852 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.852 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:59:25.852 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445522+0000", 2024-09-20T18:59:25.852 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00049521500000000002 2024-09-20T18:59:25.852 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.852 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.852 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:59:25.852 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445598+0000", 2024-09-20T18:59:25.852 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5993000000000002e-05 2024-09-20T18:59:25.852 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.852 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.852 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:59:25.852 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445637+0000", 2024-09-20T18:59:25.852 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9774000000000002e-05 2024-09-20T18:59:25.853 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:25.853 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:25.853 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:25.853 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.853 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.853 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:25333 7.2 7:4c00f64e:::benchmark_data_smithi083_70643_object25332:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:59:25.853 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.266192+0000", 2024-09-20T18:59:25.853 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 497.709634714, 2024-09-20T18:59:25.853 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17952103699999999, 2024-09-20T18:59:25.853 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:59:25.853 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:59:25.853 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:25.853 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:59:25.853 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:59:25.854 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:59:25.854 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25333 2024-09-20T18:59:25.854 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.854 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:59:25.854 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.854 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:59:25.854 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266192+0000", 2024-09-20T18:59:25.854 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:25.854 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.854 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.854 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:59:25.854 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266192+0000", 2024-09-20T18:59:25.854 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:25.855 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.855 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.855 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:59:25.855 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266193+0000", 2024-09-20T18:59:25.855 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0999999999999998e-07 2024-09-20T18:59:25.855 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.855 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.855 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:59:25.855 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266211+0000", 2024-09-20T18:59:25.855 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8587e-05 2024-09-20T18:59:25.855 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.855 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.855 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:59:25.855 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266218+0000", 2024-09-20T18:59:25.856 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7340000000000002e-06 2024-09-20T18:59:25.856 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.856 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.856 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:59:25.856 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.266224+0000", 2024-09-20T18:59:25.856 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6590000000000004e-06 2024-09-20T18:59:25.856 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.856 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.856 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:59:25.856 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268185+0000", 2024-09-20T18:59:25.856 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0019615240000000001 2024-09-20T18:59:25.856 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.856 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.856 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:59:25.857 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268326+0000", 2024-09-20T18:59:25.857 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014032399999999999 2024-09-20T18:59:25.857 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.857 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.857 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T18:59:25.857 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.268450+0000", 2024-09-20T18:59:25.857 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012426599999999999 2024-09-20T18:59:25.857 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.857 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.857 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:25.857 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444991+0000", 2024-09-20T18:59:25.857 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17654108199999999 2024-09-20T18:59:25.858 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.858 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.858 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:25.858 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445507+0000", 2024-09-20T18:59:25.858 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00051583599999999999 2024-09-20T18:59:25.858 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.858 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.858 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:59:25.858 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445645+0000", 2024-09-20T18:59:25.858 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000137959 2024-09-20T18:59:25.858 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.858 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.859 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:59:25.859 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445682+0000", 2024-09-20T18:59:25.859 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7455e-05 2024-09-20T18:59:25.859 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.859 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.859 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:59:25.859 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445713+0000", 2024-09-20T18:59:25.859 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1000999999999997e-05 2024-09-20T18:59:25.859 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:25.859 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:25.859 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:25.859 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.859 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.859 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:25336 7.2 7:4be582b4:::benchmark_data_smithi083_70643_object25335:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:59:25.860 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.267773+0000", 2024-09-20T18:59:25.860 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 497.70805386000001, 2024-09-20T18:59:25.860 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.18517899500000001, 2024-09-20T18:59:25.860 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:59:25.860 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:59:25.860 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:25.860 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:59:25.860 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:59:25.860 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:59:25.860 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25336 2024-09-20T18:59:25.860 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.860 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:59:25.860 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.860 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:59:25.861 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.267773+0000", 2024-09-20T18:59:25.861 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:25.861 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.861 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.861 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:59:25.861 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.267773+0000", 2024-09-20T18:59:25.861 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:25.861 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.861 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.861 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:59:25.861 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.267774+0000", 2024-09-20T18:59:25.862 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.13e-06 2024-09-20T18:59:25.862 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.862 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.862 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:59:25.862 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.267797+0000", 2024-09-20T18:59:25.862 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2852000000000001e-05 2024-09-20T18:59:25.862 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.862 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.862 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:59:25.862 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.267806+0000", 2024-09-20T18:59:25.862 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4519999999999997e-06 2024-09-20T18:59:25.863 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.863 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.863 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:59:25.863 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.267812+0000", 2024-09-20T18:59:25.863 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3489999999999997e-06 2024-09-20T18:59:25.863 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.863 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.863 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:59:25.863 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.270828+0000", 2024-09-20T18:59:25.863 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0030158020000000002 2024-09-20T18:59:25.863 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.863 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.863 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:59:25.864 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443376+0000", 2024-09-20T18:59:25.864 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17254844 2024-09-20T18:59:25.864 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.864 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.864 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T18:59:25.864 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443495+0000", 2024-09-20T18:59:25.864 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000119338 2024-09-20T18:59:25.864 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.864 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.864 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:59:25.864 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.446411+0000", 2024-09-20T18:59:25.864 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0029155869999999999 2024-09-20T18:59:25.864 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.864 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.865 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:25.865 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.452802+0000", 2024-09-20T18:59:25.865 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063913169999999997 2024-09-20T18:59:25.865 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.865 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.865 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:25.865 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.452870+0000", 2024-09-20T18:59:25.865 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7824999999999995e-05 2024-09-20T18:59:25.865 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.865 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.865 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:59:25.865 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.452915+0000", 2024-09-20T18:59:25.865 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4815999999999999e-05 2024-09-20T18:59:25.866 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.866 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.866 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:59:25.866 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.452952+0000", 2024-09-20T18:59:25.866 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7086999999999999e-05 2024-09-20T18:59:25.866 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:25.866 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:25.866 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:25.866 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.866 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.866 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:25337 7.f e93/83 7:fd2b3e49:::benchmark_data_smithi083_70643_object25336:head v 93'1586, mlcod=93'1584)", 2024-09-20T18:59:25.866 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.269129+0000", 2024-09-20T18:59:25.866 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 497.70669754800002, 2024-09-20T18:59:25.866 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.174986899, 2024-09-20T18:59:25.867 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:59:25.867 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:59:25.867 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:25.867 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:59:25.867 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.867 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:59:25.867 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269129+0000", 2024-09-20T18:59:25.867 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:25.867 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.867 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.867 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:59:25.867 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269129+0000", 2024-09-20T18:59:25.867 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:25.867 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.868 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.868 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:59:25.868 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269130+0000", 2024-09-20T18:59:25.868 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8000000000000001e-07 2024-09-20T18:59:25.868 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.868 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.868 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:59:25.868 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269151+0000", 2024-09-20T18:59:25.868 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1593000000000001e-05 2024-09-20T18:59:25.868 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.868 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.868 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:59:25.868 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269158+0000", 2024-09-20T18:59:25.868 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1069999999999999e-06 2024-09-20T18:59:25.869 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.869 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.869 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:59:25.869 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269163+0000", 2024-09-20T18:59:25.869 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7029999999999997e-06 2024-09-20T18:59:25.869 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.869 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.869 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:59:25.869 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269349+0000", 2024-09-20T18:59:25.869 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000185331 2024-09-20T18:59:25.869 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.869 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.869 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:59:25.869 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269365+0000", 2024-09-20T18:59:25.869 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6424000000000001e-05 2024-09-20T18:59:25.870 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.870 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.870 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:59:25.870 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271621+0000", 2024-09-20T18:59:25.870 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0022557200000000001 2024-09-20T18:59:25.870 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.870 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.870 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:59:25.870 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444116+0000", 2024-09-20T18:59:25.870 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17249564100000001 2024-09-20T18:59:25.870 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:25.870 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:25.870 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:25.870 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.870 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.871 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:25341 7.a 7:5bb50a52:::benchmark_data_smithi083_70643_object25340:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:59:25.871 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.269273+0000", 2024-09-20T18:59:25.871 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 497.706553477, 2024-09-20T18:59:25.871 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.18597625000000001, 2024-09-20T18:59:25.871 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:59:25.871 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:59:25.871 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:25.871 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:59:25.871 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T18:59:25.871 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:59:25.871 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 25341 2024-09-20T18:59:25.871 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.871 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:59:25.871 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.872 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:59:25.872 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269273+0000", 2024-09-20T18:59:25.872 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:25.872 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.872 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.872 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:59:25.872 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269273+0000", 2024-09-20T18:59:25.872 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:25.872 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.872 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.872 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:59:25.872 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269274+0000", 2024-09-20T18:59:25.872 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0139999999999999e-06 2024-09-20T18:59:25.872 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.872 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.873 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:59:25.873 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269293+0000", 2024-09-20T18:59:25.873 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8499e-05 2024-09-20T18:59:25.873 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.873 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.873 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:59:25.873 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269301+0000", 2024-09-20T18:59:25.873 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8429999999999993e-06 2024-09-20T18:59:25.873 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.873 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.873 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:59:25.873 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.269306+0000", 2024-09-20T18:59:25.873 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2809999999999999e-06 2024-09-20T18:59:25.873 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.873 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.874 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:59:25.874 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445057+0000", 2024-09-20T18:59:25.874 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17575099 2024-09-20T18:59:25.874 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.874 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.874 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:59:25.874 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445166+0000", 2024-09-20T18:59:25.875 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000109335 2024-09-20T18:59:25.875 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.875 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.875 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,5", 2024-09-20T18:59:25.876 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.445293+0000", 2024-09-20T18:59:25.876 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000126995 2024-09-20T18:59:25.876 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.876 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.876 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:25.876 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.452767+0000", 2024-09-20T18:59:25.876 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0074736009999999999 2024-09-20T18:59:25.876 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.876 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.876 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:25.876 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.452832+0000", 2024-09-20T18:59:25.876 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4817e-05 2024-09-20T18:59:25.876 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.876 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.877 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:59:25.877 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.455135+0000", 2024-09-20T18:59:25.877 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0023034459999999998 2024-09-20T18:59:25.877 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.877 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.877 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:59:25.877 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.455211+0000", 2024-09-20T18:59:25.877 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5896999999999995e-05 2024-09-20T18:59:25.877 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.877 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.877 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:59:25.877 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.455250+0000", 2024-09-20T18:59:25.877 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8532000000000003e-05 2024-09-20T18:59:25.877 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:25.877 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:25.878 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:25.878 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.878 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.878 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:25343 7.6 e93/83 7:61f2dd3b:::benchmark_data_smithi083_70643_object25342:head v 93'1583, mlcod=93'1579)", 2024-09-20T18:59:25.878 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:07.271493+0000", 2024-09-20T18:59:25.878 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 497.70433427299997, 2024-09-20T18:59:25.878 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.17491087999999999, 2024-09-20T18:59:25.878 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:59:25.878 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:59:25.878 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:25.878 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:59:25.878 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.878 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:59:25.879 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271493+0000", 2024-09-20T18:59:25.879 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:25.879 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.879 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.879 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:59:25.879 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271493+0000", 2024-09-20T18:59:25.879 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:25.879 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.879 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.879 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:59:25.879 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271493+0000", 2024-09-20T18:59:25.879 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8400000000000004e-07 2024-09-20T18:59:25.879 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.880 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.880 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:59:25.880 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.271513+0000", 2024-09-20T18:59:25.880 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0126999999999999e-05 2024-09-20T18:59:25.880 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.880 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.880 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:59:25.880 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443960+0000", 2024-09-20T18:59:25.880 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.172446983 2024-09-20T18:59:25.880 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.880 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.880 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:59:25.880 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.443970+0000", 2024-09-20T18:59:25.880 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5119999999999997e-06 2024-09-20T18:59:25.880 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.880 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.881 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:59:25.881 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444547+0000", 2024-09-20T18:59:25.881 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00057698700000000005 2024-09-20T18:59:25.881 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.881 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.881 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:59:25.881 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.444578+0000", 2024-09-20T18:59:25.881 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0956e-05 2024-09-20T18:59:25.881 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.881 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.881 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:59:25.881 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.446361+0000", 2024-09-20T18:59:25.881 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001783473 2024-09-20T18:59:25.881 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.882 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.882 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:59:25.882 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:07.446404+0000", 2024-09-20T18:59:25.882 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2157999999999999e-05 2024-09-20T18:59:25.882 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:25.882 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:25.882 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:25.882 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.882 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.882 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.24955.0:218155 8.7 e233/213 8:ebcbd19d:::benchmark_data_smithi083_97422_object102064:head v 233'13685, mlcod=233'13682)", 2024-09-20T18:59:25.882 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:58:01.869876+0000", 2024-09-20T18:59:25.882 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 83.105950777999993, 2024-09-20T18:59:25.882 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.218638782, 2024-09-20T18:59:25.882 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:59:25.883 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:59:25.883 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:25.883 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:59:25.883 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.883 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:59:25.883 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.869876+0000", 2024-09-20T18:59:25.883 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:25.883 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.883 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.883 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:59:25.883 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.869876+0000", 2024-09-20T18:59:25.883 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:25.883 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.883 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.883 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:59:25.884 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.869877+0000", 2024-09-20T18:59:25.884 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9999999999999996e-07 2024-09-20T18:59:25.884 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.884 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.884 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:59:25.884 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.869883+0000", 2024-09-20T18:59:25.884 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7819999999999999e-06 2024-09-20T18:59:25.884 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.884 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.884 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:59:25.884 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.869891+0000", 2024-09-20T18:59:25.884 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6289999999999999e-06 2024-09-20T18:59:25.884 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.884 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.885 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:59:25.885 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.869897+0000", 2024-09-20T18:59:25.885 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.049e-06 2024-09-20T18:59:25.885 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.885 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.885 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:59:25.885 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.874096+0000", 2024-09-20T18:59:25.885 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0041987489999999999 2024-09-20T18:59:25.885 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.885 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.885 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:59:25.885 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.874117+0000", 2024-09-20T18:59:25.885 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0361999999999999e-05 2024-09-20T18:59:25.885 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.885 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.886 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:59:25.886 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.088464+0000", 2024-09-20T18:59:25.886 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.21434697999999999 2024-09-20T18:59:25.886 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.886 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.886 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:59:25.886 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.088515+0000", 2024-09-20T18:59:25.886 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1431000000000003e-05 2024-09-20T18:59:25.886 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:25.886 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:25.886 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:25.886 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.886 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.887 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:218167 8.f 8:f8df719b:::benchmark_data_smithi083_97422_object102076:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-20T18:59:25.887 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:58:01.876042+0000", 2024-09-20T18:59:25.887 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 83.099784485000001, 2024-09-20T18:59:25.887 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.212910085, 2024-09-20T18:59:25.887 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:59:25.887 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:59:25.887 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:25.887 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:59:25.887 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:59:25.887 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:59:25.887 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 218167 2024-09-20T18:59:25.887 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.887 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:59:25.887 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.888 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:59:25.888 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.876042+0000", 2024-09-20T18:59:25.888 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:25.888 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.888 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.888 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:59:25.888 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.876042+0000", 2024-09-20T18:59:25.888 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:25.888 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.888 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.888 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:59:25.888 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.876044+0000", 2024-09-20T18:59:25.888 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1340000000000001e-06 2024-09-20T18:59:25.888 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.889 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.889 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:59:25.889 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.876047+0000", 2024-09-20T18:59:25.889 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9969999999999999e-06 2024-09-20T18:59:25.889 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.889 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.889 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:59:25.889 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.876054+0000", 2024-09-20T18:59:25.889 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1620000000000002e-06 2024-09-20T18:59:25.889 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.889 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.889 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:59:25.889 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.876059+0000", 2024-09-20T18:59:25.889 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0610000000000003e-06 2024-09-20T18:59:25.889 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.889 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.890 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:59:25.890 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.876376+0000", 2024-09-20T18:59:25.890 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00031752499999999998 2024-09-20T18:59:25.890 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.890 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.890 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:59:25.890 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.876544+0000", 2024-09-20T18:59:25.890 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000167732 2024-09-20T18:59:25.890 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.890 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.890 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T18:59:25.890 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.876643+0000", 2024-09-20T18:59:25.891 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9339000000000006e-05 2024-09-20T18:59:25.891 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.891 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.891 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:25.891 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.885374+0000", 2024-09-20T18:59:25.891 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0087305390000000007 2024-09-20T18:59:25.891 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.891 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.891 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:25.891 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.081897+0000", 2024-09-20T18:59:25.891 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.19652308099999999 2024-09-20T18:59:25.891 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.891 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.891 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:59:25.892 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.088624+0000", 2024-09-20T18:59:25.892 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0067268320000000003 2024-09-20T18:59:25.892 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.892 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.892 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:59:25.892 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.088808+0000", 2024-09-20T18:59:25.892 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018438999999999999 2024-09-20T18:59:25.892 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.892 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.892 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:59:25.892 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.088953+0000", 2024-09-20T18:59:25.892 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014429300000000001 2024-09-20T18:59:25.892 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:25.892 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:25.893 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:25.893 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.893 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.893 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:218168 8.f 8:fa798818:::benchmark_data_smithi083_97422_object102077:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-20T18:59:25.893 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:58:01.879290+0000", 2024-09-20T18:59:25.893 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 83.096537283000004, 2024-09-20T18:59:25.893 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.209845379, 2024-09-20T18:59:25.893 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:59:25.893 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:59:25.893 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:25.893 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:59:25.893 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:59:25.893 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:59:25.893 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 218168 2024-09-20T18:59:25.893 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.894 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:59:25.894 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.894 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:59:25.894 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.879290+0000", 2024-09-20T18:59:25.894 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:25.894 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.894 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.894 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:59:25.894 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.879290+0000", 2024-09-20T18:59:25.894 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:25.894 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.894 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.894 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:59:25.894 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.879291+0000", 2024-09-20T18:59:25.895 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4890000000000001e-06 2024-09-20T18:59:25.895 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.895 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.895 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:59:25.895 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.879296+0000", 2024-09-20T18:59:25.895 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9479999999999999e-06 2024-09-20T18:59:25.895 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.895 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.895 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:59:25.895 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.879306+0000", 2024-09-20T18:59:25.895 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6749999999999997e-06 2024-09-20T18:59:25.895 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.895 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.896 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:59:25.896 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.879313+0000", 2024-09-20T18:59:25.896 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1570000000000003e-06 2024-09-20T18:59:25.896 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.896 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.896 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:59:25.896 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.879650+0000", 2024-09-20T18:59:25.896 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00033674399999999999 2024-09-20T18:59:25.896 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.896 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.896 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:59:25.896 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.879933+0000", 2024-09-20T18:59:25.896 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00028346099999999999 2024-09-20T18:59:25.896 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.897 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.897 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T18:59:25.897 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.880034+0000", 2024-09-20T18:59:25.897 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000100523 2024-09-20T18:59:25.897 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.897 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.897 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:25.897 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.086134+0000", 2024-09-20T18:59:25.897 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.20610060399999999 2024-09-20T18:59:25.897 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.897 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.897 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:25.897 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.088528+0000", 2024-09-20T18:59:25.897 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0023938850000000001 2024-09-20T18:59:25.897 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.898 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.898 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:59:25.898 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.089043+0000", 2024-09-20T18:59:25.898 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000514921 2024-09-20T18:59:25.898 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.898 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.898 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:59:25.898 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.089099+0000", 2024-09-20T18:59:25.898 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5773000000000003e-05 2024-09-20T18:59:25.898 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.898 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.898 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:59:25.898 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.089135+0000", 2024-09-20T18:59:25.898 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6198999999999997e-05 2024-09-20T18:59:25.899 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:25.899 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:25.899 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:25.899 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.899 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.899 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:218173 8.f 8:fc626e9a:::benchmark_data_smithi083_97422_object102082:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-20T18:59:25.899 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:58:01.879894+0000", 2024-09-20T18:59:25.899 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 83.095932465000004, 2024-09-20T18:59:25.899 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.22879112400000001, 2024-09-20T18:59:25.899 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:59:25.899 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:59:25.899 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:25.899 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:59:25.899 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T18:59:25.900 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:59:25.900 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 218173 2024-09-20T18:59:25.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.900 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:59:25.900 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.900 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:59:25.900 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.879894+0000", 2024-09-20T18:59:25.900 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:25.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.900 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.900 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:59:25.900 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.879894+0000", 2024-09-20T18:59:25.900 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:25.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.901 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.901 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:59:25.901 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.879895+0000", 2024-09-20T18:59:25.901 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3600000000000003e-07 2024-09-20T18:59:25.901 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.901 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.901 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:59:25.901 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.879899+0000", 2024-09-20T18:59:25.901 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3500000000000001e-06 2024-09-20T18:59:25.901 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.901 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.901 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:59:25.901 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.879907+0000", 2024-09-20T18:59:25.901 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.943e-06 2024-09-20T18:59:25.901 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.902 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.902 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:59:25.902 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.879913+0000", 2024-09-20T18:59:25.902 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3910000000000003e-06 2024-09-20T18:59:25.902 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.902 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.902 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:59:25.902 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.095128+0000", 2024-09-20T18:59:25.902 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.21521499199999999 2024-09-20T18:59:25.902 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.902 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.902 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:59:25.902 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.095243+0000", 2024-09-20T18:59:25.902 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011465599999999999 2024-09-20T18:59:25.903 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.903 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.903 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T18:59:25.903 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.095357+0000", 2024-09-20T18:59:25.903 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000114868 2024-09-20T18:59:25.903 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.903 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.903 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:25.903 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.104639+0000", 2024-09-20T18:59:25.903 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0092811200000000003 2024-09-20T18:59:25.903 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.903 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.903 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:59:25.903 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.105329+0000", 2024-09-20T18:59:25.904 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000690195 2024-09-20T18:59:25.904 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.904 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.904 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:25.904 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.108593+0000", 2024-09-20T18:59:25.904 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003263967 2024-09-20T18:59:25.904 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.904 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.904 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:59:25.904 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.108646+0000", 2024-09-20T18:59:25.904 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3495999999999997e-05 2024-09-20T18:59:25.904 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.904 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.904 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:59:25.904 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.108686+0000", 2024-09-20T18:59:25.905 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9409999999999997e-05 2024-09-20T18:59:25.905 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:25.905 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:25.905 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:25.905 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.905 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.905 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.24955.0:218167 8.f e233/160 ondisk, result = 0)", 2024-09-20T18:59:25.905 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:58:01.881107+0000", 2024-09-20T18:59:25.905 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 83.094720039999999, 2024-09-20T18:59:25.905 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.20784897499999999, 2024-09-20T18:59:25.905 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:59:25.905 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:59:25.905 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:59:25.906 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:59:25.906 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.906 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:59:25.906 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.881107+0000", 2024-09-20T18:59:25.906 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:25.906 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.906 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.906 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:59:25.906 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.881107+0000", 2024-09-20T18:59:25.906 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:25.906 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.906 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.906 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:59:25.906 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.881108+0000", 2024-09-20T18:59:25.907 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7400000000000002e-07 2024-09-20T18:59:25.907 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.907 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.907 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:59:25.907 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.881112+0000", 2024-09-20T18:59:25.907 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5260000000000004e-06 2024-09-20T18:59:25.907 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.907 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.907 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:59:25.907 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.881121+0000", 2024-09-20T18:59:25.907 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0469999999999999e-06 2024-09-20T18:59:25.907 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.907 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.907 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:59:25.908 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.881127+0000", 2024-09-20T18:59:25.908 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9970000000000004e-06 2024-09-20T18:59:25.908 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.908 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.908 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:59:25.908 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.081884+0000", 2024-09-20T18:59:25.908 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.200756394 2024-09-20T18:59:25.908 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.908 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.908 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:59:25.908 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.081891+0000", 2024-09-20T18:59:25.908 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.6109999999999996e-06 2024-09-20T18:59:25.908 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.908 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.908 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:59:25.909 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.088956+0000", 2024-09-20T18:59:25.909 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0070646260000000001 2024-09-20T18:59:25.909 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:25.909 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:25.909 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:25.909 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.909 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.909 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.24955.0:218171 8.7 e233/213 8:e3fe301b:::benchmark_data_smithi083_97422_object102080:head v 233'13686, mlcod=233'13682)", 2024-09-20T18:59:25.909 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:58:01.882984+0000", 2024-09-20T18:59:25.909 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 83.092842650999998, 2024-09-20T18:59:25.909 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.21396309799999999, 2024-09-20T18:59:25.909 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:59:25.909 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:59:25.910 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:25.910 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:59:25.910 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.910 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:59:25.910 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.882984+0000", 2024-09-20T18:59:25.910 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:25.910 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.910 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.910 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:59:25.910 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.882984+0000", 2024-09-20T18:59:25.910 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:25.910 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.910 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.910 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:59:25.911 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.882985+0000", 2024-09-20T18:59:25.911 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5799999999999999e-07 2024-09-20T18:59:25.911 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.911 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.911 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:59:25.911 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.882990+0000", 2024-09-20T18:59:25.911 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0919999999999997e-06 2024-09-20T18:59:25.911 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.911 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.911 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:59:25.911 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.882997+0000", 2024-09-20T18:59:25.911 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9269999999999999e-06 2024-09-20T18:59:25.911 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.911 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.912 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:59:25.912 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.883003+0000", 2024-09-20T18:59:25.912 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2319999999999998e-06 2024-09-20T18:59:25.912 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.912 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.912 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:59:25.912 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.080830+0000", 2024-09-20T18:59:25.912 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.197826429 2024-09-20T18:59:25.912 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.912 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.912 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:59:25.912 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.080862+0000", 2024-09-20T18:59:25.912 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2858999999999998e-05 2024-09-20T18:59:25.912 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.912 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.912 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:59:25.913 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.096909+0000", 2024-09-20T18:59:25.913 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.016046662 2024-09-20T18:59:25.913 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.913 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.913 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:59:25.913 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.096947+0000", 2024-09-20T18:59:25.913 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8238999999999998e-05 2024-09-20T18:59:25.913 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:25.913 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:25.913 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:25.913 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.913 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.913 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.24955.0:218179 8.7 e233/213 8:e44c0f9a:::benchmark_data_smithi083_97422_object102088:head v 233'13687, mlcod=233'13683)", 2024-09-20T18:59:25.913 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:58:01.885843+0000", 2024-09-20T18:59:25.913 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 83.089984426000001, 2024-09-20T18:59:25.914 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.211128274, 2024-09-20T18:59:25.914 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:59:25.914 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:59:25.914 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:25.914 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:59:25.914 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.914 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:59:25.914 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.885843+0000", 2024-09-20T18:59:25.914 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:25.914 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.914 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.914 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:59:25.914 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.885843+0000", 2024-09-20T18:59:25.914 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:25.914 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.915 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.915 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:59:25.915 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.885843+0000", 2024-09-20T18:59:25.915 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5999999999999991e-07 2024-09-20T18:59:25.915 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.915 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.915 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:59:25.915 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.885858+0000", 2024-09-20T18:59:25.915 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4286e-05 2024-09-20T18:59:25.915 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.915 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.915 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:59:25.915 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.080663+0000", 2024-09-20T18:59:25.915 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.19480504600000001 2024-09-20T18:59:25.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.916 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.916 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:59:25.916 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.080676+0000", 2024-09-20T18:59:25.916 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3417999999999999e-05 2024-09-20T18:59:25.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.916 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.916 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:59:25.916 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.083322+0000", 2024-09-20T18:59:25.916 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0026456880000000002 2024-09-20T18:59:25.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.916 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.916 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:59:25.916 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.083341+0000", 2024-09-20T18:59:25.916 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9539999999999999e-05 2024-09-20T18:59:25.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.916 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.917 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:59:25.917 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.096951+0000", 2024-09-20T18:59:25.917 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.013609677000000001 2024-09-20T18:59:25.917 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.917 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.917 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:59:25.917 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.096971+0000", 2024-09-20T18:59:25.917 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9658999999999999e-05 2024-09-20T18:59:25.917 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:25.917 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:25.917 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:25.917 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.917 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.917 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.24955.0:218168 8.f e233/160 ondisk, result = 0)", 2024-09-20T18:59:25.918 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:58:01.888899+0000", 2024-09-20T18:59:25.918 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 83.086927969000001, 2024-09-20T18:59:25.918 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.19963995100000001, 2024-09-20T18:59:25.918 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:59:25.918 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:59:25.918 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T18:59:25.918 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:59:25.918 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.918 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:59:25.918 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.888899+0000", 2024-09-20T18:59:25.918 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:25.918 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.918 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.919 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:59:25.919 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.888899+0000", 2024-09-20T18:59:25.919 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:25.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.919 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.919 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:59:25.919 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.888900+0000", 2024-09-20T18:59:25.919 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0400000000000005e-07 2024-09-20T18:59:25.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.919 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.919 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:59:25.919 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.888906+0000", 2024-09-20T18:59:25.919 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.784e-06 2024-09-20T18:59:25.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.920 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.920 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:59:25.920 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.080754+0000", 2024-09-20T18:59:25.920 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.191848466 2024-09-20T18:59:25.920 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.920 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.920 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:59:25.920 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.080764+0000", 2024-09-20T18:59:25.920 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5359999999999998e-06 2024-09-20T18:59:25.920 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.920 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.920 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:59:25.920 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.088464+0000", 2024-09-20T18:59:25.920 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0077003829999999999 2024-09-20T18:59:25.920 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.921 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.921 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:59:25.921 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.088499+0000", 2024-09-20T18:59:25.921 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.485e-05 2024-09-20T18:59:25.921 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.921 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.921 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:59:25.921 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.088539+0000", 2024-09-20T18:59:25.921 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0027999999999997e-05 2024-09-20T18:59:25.921 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:25.921 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:25.921 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:25.921 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.921 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.922 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:50423 9.1 9:8787b2bd:::benchmark_data_smithi083_124552_object50422:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-20T18:59:25.922 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:59:04.093296+0000", 2024-09-20T18:59:25.922 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 20.882531205999999, 2024-09-20T18:59:25.922 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.136410235, 2024-09-20T18:59:25.922 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:59:25.922 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:59:25.922 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:25.922 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:59:25.922 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T18:59:25.922 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T18:59:25.922 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 50423 2024-09-20T18:59:25.922 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.922 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:59:25.922 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.923 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:59:25.923 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.093296+0000", 2024-09-20T18:59:25.923 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:25.923 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.923 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.923 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:59:25.923 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.093296+0000", 2024-09-20T18:59:25.923 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:25.923 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.923 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.923 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:59:25.923 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.093297+0000", 2024-09-20T18:59:25.923 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.37e-06 2024-09-20T18:59:25.923 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.924 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.924 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:59:25.924 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.093316+0000", 2024-09-20T18:59:25.924 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8660000000000001e-05 2024-09-20T18:59:25.924 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.924 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.924 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:59:25.924 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.093342+0000", 2024-09-20T18:59:25.924 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6103999999999999e-05 2024-09-20T18:59:25.924 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.924 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.924 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:59:25.924 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.093347+0000", 2024-09-20T18:59:25.924 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0490000000000003e-06 2024-09-20T18:59:25.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.925 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.925 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:59:25.925 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.093482+0000", 2024-09-20T18:59:25.925 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000135449 2024-09-20T18:59:25.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.925 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.925 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:59:25.925 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.093490+0000", 2024-09-20T18:59:25.925 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0469999999999994e-06 2024-09-20T18:59:25.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.925 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.925 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:59:25.925 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.067142+0000", 2024-09-20T18:59:25.925 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97365191799999995 2024-09-20T18:59:25.926 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.926 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.926 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:59:25.926 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.067153+0000", 2024-09-20T18:59:25.926 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0559e-05 2024-09-20T18:59:25.926 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.926 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.926 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:59:25.926 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.215644+0000", 2024-09-20T18:59:25.926 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.148490633 2024-09-20T18:59:25.926 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.926 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.926 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:59:25.926 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.215772+0000", 2024-09-20T18:59:25.927 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012840799999999999 2024-09-20T18:59:25.927 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.927 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.927 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 6,7", 2024-09-20T18:59:25.927 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.215927+0000", 2024-09-20T18:59:25.927 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000155315 2024-09-20T18:59:25.927 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.927 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.927 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:25.927 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.223137+0000", 2024-09-20T18:59:25.927 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0072102049999999999 2024-09-20T18:59:25.927 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.927 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.927 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:25.928 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.223182+0000", 2024-09-20T18:59:25.928 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4158e-05 2024-09-20T18:59:25.928 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.928 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.928 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:59:25.928 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.227203+0000", 2024-09-20T18:59:25.928 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0040211530000000004 2024-09-20T18:59:25.928 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.928 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.928 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:59:25.928 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.227258+0000", 2024-09-20T18:59:25.928 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5050999999999999e-05 2024-09-20T18:59:25.928 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.928 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.929 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:59:25.929 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.229706+0000", 2024-09-20T18:59:25.929 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0024481559999999999 2024-09-20T18:59:25.929 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:25.929 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:25.929 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:25.929 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.929 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.929 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:50474 9.1 9:8f4be195:::benchmark_data_smithi083_124552_object50473:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-20T18:59:25.929 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:59:04.131531+0000", 2024-09-20T18:59:25.929 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 20.844296288999999, 2024-09-20T18:59:25.929 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.101374029, 2024-09-20T18:59:25.929 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:59:25.929 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:59:25.930 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:25.930 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:59:25.930 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T18:59:25.930 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T18:59:25.930 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 50474 2024-09-20T18:59:25.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.930 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:59:25.930 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.930 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:59:25.930 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.131531+0000", 2024-09-20T18:59:25.930 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:25.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.930 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.930 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:59:25.931 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.131531+0000", 2024-09-20T18:59:25.931 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:25.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.931 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.931 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:59:25.931 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.131532+0000", 2024-09-20T18:59:25.931 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5489999999999999e-06 2024-09-20T18:59:25.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.931 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.931 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:59:25.931 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.131552+0000", 2024-09-20T18:59:25.931 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9630999999999999e-05 2024-09-20T18:59:25.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.931 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.931 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:59:25.932 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.131568+0000", 2024-09-20T18:59:25.932 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6606999999999999e-05 2024-09-20T18:59:25.932 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.932 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.932 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:59:25.932 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.131577+0000", 2024-09-20T18:59:25.932 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7879999999999995e-06 2024-09-20T18:59:25.932 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.932 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.932 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:59:25.932 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.131644+0000", 2024-09-20T18:59:25.932 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7180000000000004e-05 2024-09-20T18:59:25.932 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.933 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.933 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:59:25.933 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.131654+0000", 2024-09-20T18:59:25.933 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2390000000000008e-06 2024-09-20T18:59:25.933 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.933 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.933 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:59:25.933 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.067311+0000", 2024-09-20T18:59:25.933 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.935657501 2024-09-20T18:59:25.933 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.933 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.933 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:59:25.933 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.067320+0000", 2024-09-20T18:59:25.933 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9779999999999994e-06 2024-09-20T18:59:25.933 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.934 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.934 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:59:25.934 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.220228+0000", 2024-09-20T18:59:25.934 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1529083090000001 2024-09-20T18:59:25.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.934 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.934 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:59:25.934 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.220362+0000", 2024-09-20T18:59:25.934 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013401 2024-09-20T18:59:25.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.934 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.934 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 6,7", 2024-09-20T18:59:25.934 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.220523+0000", 2024-09-20T18:59:25.934 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000160247 2024-09-20T18:59:25.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.935 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:59:25.935 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.229718+0000", 2024-09-20T18:59:25.935 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0091949860000000005 2024-09-20T18:59:25.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.935 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:25.935 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.229797+0000", 2024-09-20T18:59:25.935 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9443999999999994e-05 2024-09-20T18:59:25.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.935 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:25.935 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.230036+0000", 2024-09-20T18:59:25.935 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000239072 2024-09-20T18:59:25.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.936 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.936 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:59:25.936 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.230111+0000", 2024-09-20T18:59:25.936 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4411000000000003e-05 2024-09-20T18:59:25.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.936 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.936 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:59:25.936 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.232905+0000", 2024-09-20T18:59:25.936 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0027940769999999998 2024-09-20T18:59:25.936 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:25.936 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:25.936 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:25.937 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.937 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.937 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:50506 9.1 9:80ae6680:::benchmark_data_smithi083_124552_object50505:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-20T18:59:25.937 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:59:04.155840+0000", 2024-09-20T18:59:25.937 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 20.819986486000001, 2024-09-20T18:59:25.937 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.082518146, 2024-09-20T18:59:25.937 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:59:25.937 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:59:25.937 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:25.937 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:59:25.937 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T18:59:25.937 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T18:59:25.937 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 50506 2024-09-20T18:59:25.937 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.938 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:59:25.938 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.938 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:59:25.938 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.155840+0000", 2024-09-20T18:59:25.938 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:25.938 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.938 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.938 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:59:25.938 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.155840+0000", 2024-09-20T18:59:25.938 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:25.938 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.938 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.938 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:59:25.938 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.155842+0000", 2024-09-20T18:59:25.939 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1090000000000001e-06 2024-09-20T18:59:25.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.939 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.939 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:59:25.939 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.155869+0000", 2024-09-20T18:59:25.939 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7801999999999999e-05 2024-09-20T18:59:25.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.939 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.939 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:59:25.939 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.155880+0000", 2024-09-20T18:59:25.939 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1059e-05 2024-09-20T18:59:25.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.939 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.939 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:59:25.940 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.155888+0000", 2024-09-20T18:59:25.940 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7559999999999999e-06 2024-09-20T18:59:25.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.940 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.940 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:59:25.940 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.155955+0000", 2024-09-20T18:59:25.940 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7022999999999998e-05 2024-09-20T18:59:25.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.940 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.940 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:59:25.940 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.155965+0000", 2024-09-20T18:59:25.940 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3710000000000006e-06 2024-09-20T18:59:25.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.941 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.941 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:59:25.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.067356+0000", 2024-09-20T18:59:25.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.91139177299999996 2024-09-20T18:59:25.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.941 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.941 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:59:25.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.067364+0000", 2024-09-20T18:59:25.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.6000000000000001e-06 2024-09-20T18:59:25.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.941 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.941 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:59:25.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.223316+0000", 2024-09-20T18:59:25.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1559515810000001 2024-09-20T18:59:25.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.942 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.942 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:59:25.942 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.223488+0000", 2024-09-20T18:59:25.942 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017242399999999999 2024-09-20T18:59:25.942 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.942 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.942 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 6,7", 2024-09-20T18:59:25.942 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.223628+0000", 2024-09-20T18:59:25.942 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013994100000000001 2024-09-20T18:59:25.942 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.942 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.942 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:25.942 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.229891+0000", 2024-09-20T18:59:25.942 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0062633239999999998 2024-09-20T18:59:25.942 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.943 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.943 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:59:25.943 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.229911+0000", 2024-09-20T18:59:25.943 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9910000000000001e-05 2024-09-20T18:59:25.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.943 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.943 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:25.943 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.235269+0000", 2024-09-20T18:59:25.943 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0053577429999999999 2024-09-20T18:59:25.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.943 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.943 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:59:25.943 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.235358+0000", 2024-09-20T18:59:25.943 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9536999999999993e-05 2024-09-20T18:59:25.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.944 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.944 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:59:25.944 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.238359+0000", 2024-09-20T18:59:25.944 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003000193 2024-09-20T18:59:25.944 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:25.944 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:25.944 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:25.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.944 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.944 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:50517 9.1 9:8bbc6b9c:::benchmark_data_smithi083_124552_object50516:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-20T18:59:25.944 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:59:04.167833+0000", 2024-09-20T18:59:25.944 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 20.807994049000001, 2024-09-20T18:59:25.944 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0735188230000001, 2024-09-20T18:59:25.944 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T18:59:25.945 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T18:59:25.945 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:25.945 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T18:59:25.945 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T18:59:25.945 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T18:59:25.945 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 50517 2024-09-20T18:59:25.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.945 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T18:59:25.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.945 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T18:59:25.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.167833+0000", 2024-09-20T18:59:25.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:25.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.945 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T18:59:25.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.167833+0000", 2024-09-20T18:59:25.946 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T18:59:25.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.946 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T18:59:25.946 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.167835+0000", 2024-09-20T18:59:25.946 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.821e-06 2024-09-20T18:59:25.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.946 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T18:59:25.946 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.167864+0000", 2024-09-20T18:59:25.946 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9714e-05 2024-09-20T18:59:25.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.946 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T18:59:25.946 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.167883+0000", 2024-09-20T18:59:25.947 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8788999999999999e-05 2024-09-20T18:59:25.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.947 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.947 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T18:59:25.947 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.167891+0000", 2024-09-20T18:59:25.947 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.1669999999999999e-06 2024-09-20T18:59:25.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.947 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.947 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:59:25.947 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.167959+0000", 2024-09-20T18:59:25.947 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7779000000000003e-05 2024-09-20T18:59:25.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.947 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.947 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:59:25.947 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.167968+0000", 2024-09-20T18:59:25.948 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9630000000000004e-06 2024-09-20T18:59:25.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.948 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:59:25.948 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.068252+0000", 2024-09-20T18:59:25.948 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.90028381000000002 2024-09-20T18:59:25.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.948 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T18:59:25.948 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.068262+0000", 2024-09-20T18:59:25.948 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0278000000000001e-05 2024-09-20T18:59:25.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.948 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T18:59:25.948 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.225517+0000", 2024-09-20T18:59:25.949 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.157254279 2024-09-20T18:59:25.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.949 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T18:59:25.949 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.225656+0000", 2024-09-20T18:59:25.949 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013903900000000001 2024-09-20T18:59:25.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.949 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 6,7", 2024-09-20T18:59:25.949 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.225821+0000", 2024-09-20T18:59:25.949 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016562099999999999 2024-09-20T18:59:25.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.949 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T18:59:25.949 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.229921+0000", 2024-09-20T18:59:25.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.004099742 2024-09-20T18:59:25.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:25.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.229977+0000", 2024-09-20T18:59:25.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6523000000000001e-05 2024-09-20T18:59:25.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:25.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.238409+0000", 2024-09-20T18:59:25.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0084314219999999992 2024-09-20T18:59:25.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T18:59:25.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.238463+0000", 2024-09-20T18:59:25.951 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4308000000000002e-05 2024-09-20T18:59:25.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:25.951 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:25.951 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T18:59:25.951 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.241352+0000", 2024-09-20T18:59:25.951 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0028885680000000002 2024-09-20T18:59:25.951 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:25.951 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:25.951 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:25.951 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:25.951 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:25.951 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T18:59:25.952 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:25.987 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:25.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:25.988 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:26.089 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:26.116 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:26.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:26.117 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:26.218 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:26.249 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:26.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:26.251 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:26.252 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 46 16 71963 71947 97.7398 63.5625 0.048912 0.010226 2024-09-20T18:59:26.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:26 smithi083 ceph-mon[29421]: pgmap v469: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 4.1 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1023 B/s rd, 67 MiB/s wr, 1.08k op/s; 7947/202041 objects misplaced (3.933%); 6.3 MiB/s, 100 objects/s recovering 2024-09-20T18:59:26.297 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:26 smithi083 ceph-mon[36260]: pgmap v469: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 4.1 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1023 B/s rd, 67 MiB/s wr, 1.08k op/s; 7947/202041 objects misplaced (3.933%); 6.3 MiB/s, 100 objects/s recovering 2024-09-20T18:59:26.351 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:59:26.387 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:26.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:26.388 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:59:26.488 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:26.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:26 smithi142 ceph-mon[32508]: pgmap v469: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 4.1 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1023 B/s rd, 67 MiB/s wr, 1.08k op/s; 7947/202041 objects misplaced (3.933%); 6.3 MiB/s, 100 objects/s recovering 2024-09-20T18:59:26.526 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:26.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:26.527 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:26.628 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:26.659 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:26.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:26.660 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:26.761 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:26.788 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:26.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:26.789 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:26.890 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:26.919 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:26.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:26.920 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:27.020 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:27.063 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:27.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:27.064 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:27.165 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:27.198 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:27.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:27.199 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:27.252 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 47 15 72913 72898 96.9248 59.4375 0.00668269 0.0103117 2024-09-20T18:59:27.267 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:59:26 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:59:27.300 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:59:27.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:27.325 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:27.325 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:59:27.426 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:27.464 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:27.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:27.465 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:27.566 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:27.600 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:27.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:27.601 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:27.702 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:59:27.742 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:27.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:27.743 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:59:27.844 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:59:27.882 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:27.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:27.883 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:59:27.984 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:59:28.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:28.017 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:28.017 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:59:28.118 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:59:28.162 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:28.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:28.163 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:59:28.252 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 48 16 73961 73945 96.2687 65.4375 0.010699 0.0103823 2024-09-20T18:59:28.264 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:28.303 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:28.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:28.304 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:28.405 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:28.438 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:28.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:28.439 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:28.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:28 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:59:28.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:28 smithi142 ceph-mon[32508]: pgmap v470: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 4.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.3 KiB/s rd, 83 MiB/s wr, 1.32k op/s; 7615/214275 objects misplaced (3.554%); 7.5 MiB/s, 119 objects/s recovering 2024-09-20T18:59:28.540 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:59:28.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:28 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:59:28.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:28 smithi083 ceph-mon[29421]: pgmap v470: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 4.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.3 KiB/s rd, 83 MiB/s wr, 1.32k op/s; 7615/214275 objects misplaced (3.554%); 7.5 MiB/s, 119 objects/s recovering 2024-09-20T18:59:28.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:28 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:59:28.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:28 smithi083 ceph-mon[36260]: pgmap v470: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 4.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.3 KiB/s rd, 83 MiB/s wr, 1.32k op/s; 7615/214275 objects misplaced (3.554%); 7.5 MiB/s, 119 objects/s recovering 2024-09-20T18:59:28.562 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:28.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:28.563 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:59:28.664 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:28.701 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:28.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:28.702 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:28.802 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:59:28.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:28.827 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:28.828 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:59:28.928 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:59:28.963 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:28.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:28.964 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:59:29.065 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:59:29.102 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:29.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:29.103 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:59:29.203 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:59:29.242 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:29.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:29.243 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:59:29.252 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 49 16 74973 74957 95.5948 63.25 0.0113826 0.0104553 2024-09-20T18:59:29.344 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:59:29.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:29.381 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:29.381 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:59:29.482 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:29.521 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:29.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:29.522 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:29.622 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:29.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:29.653 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:29.654 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:29.754 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:29.788 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:29.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:29.789 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:29.889 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:29.921 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:29.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:29.922 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:30.023 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:30.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:30.063 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:30.063 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:30.164 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:30.193 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:30.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:30.194 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:30.252 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 50 16 76044 76028 95.0216 66.9375 0.00885451 0.0105175 2024-09-20T18:59:30.294 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:30.322 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:30.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:30.323 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:30.424 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:30.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:30.463 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:30.463 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:30.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:30 smithi142 ceph-mon[32508]: pgmap v471: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 4.4 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 852 B/s rd, 59 MiB/s wr, 949 op/s; 7370/217128 objects misplaced (3.394%); 6.6 MiB/s, 105 objects/s recovering 2024-09-20T18:59:30.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:30 smithi083 ceph-mon[29421]: pgmap v471: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 4.4 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 852 B/s rd, 59 MiB/s wr, 949 op/s; 7370/217128 objects misplaced (3.394%); 6.6 MiB/s, 105 objects/s recovering 2024-09-20T18:59:30.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:30 smithi083 ceph-mon[36260]: pgmap v471: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 4.4 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 852 B/s rd, 59 MiB/s wr, 949 op/s; 7370/217128 objects misplaced (3.394%); 6.6 MiB/s, 105 objects/s recovering 2024-09-20T18:59:30.564 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:30.594 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:30.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:30.595 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:30.696 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:59:30.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:30.717 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:30.718 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:59:30.818 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:59:30.855 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:30.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:30.857 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:59:30.957 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:31.004 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:31.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:31.005 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:31.046 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:59:30 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:59:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:59:31.106 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:31.138 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:31.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:31.139 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:31.240 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:59:31.255 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 51 16 76950 76934 94.2652 56.625 0.00632014 0.0106028 2024-09-20T18:59:31.272 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:31.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:31.273 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:59:31.374 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:31.395 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:31.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:31.398 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:31.499 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:59:31.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:31 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:59:31.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:31 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:59:31.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:31 smithi142 ceph-mon[32508]: osdmap e266: 8 total, 8 up, 8 in 2024-09-20T18:59:31.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:31 smithi142 ceph-mon[32508]: osdmap e267: 8 total, 8 up, 8 in 2024-09-20T18:59:31.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:31 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:59:31.534 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:31.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:31.535 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:59:31.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:31 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:59:31.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:31 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:59:31.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:31 smithi083 ceph-mon[29421]: osdmap e266: 8 total, 8 up, 8 in 2024-09-20T18:59:31.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:31 smithi083 ceph-mon[29421]: osdmap e267: 8 total, 8 up, 8 in 2024-09-20T18:59:31.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:31 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:59:31.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:31 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:59:31.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:31 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:59:31.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:31 smithi083 ceph-mon[36260]: osdmap e266: 8 total, 8 up, 8 in 2024-09-20T18:59:31.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:31 smithi083 ceph-mon[36260]: osdmap e267: 8 total, 8 up, 8 in 2024-09-20T18:59:31.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:31 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:59:31.636 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:59:31.673 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:31.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:31.675 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:59:31.776 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:31.797 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:31.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:31.798 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:31.899 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:31.937 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:31.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:31.938 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:32.039 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:32.069 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:32.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:32.070 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:32.171 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:32.219 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:32.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:32.220 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:32.255 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 52 15 77737 77722 93.3995 49.25 0.0115363 0.0107016 2024-09-20T18:59:32.321 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:59:32.359 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:32.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:32.360 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:59:32.460 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:59:32.495 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:32.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:32.496 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:59:32.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:32 smithi142 ceph-mon[32508]: pgmap v474: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 4.5 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 47 MiB/s wr, 749 op/s; 7370/220008 objects misplaced (3.350%); 4.5 MiB/s, 72 objects/s recovering 2024-09-20T18:59:32.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:32 smithi083 ceph-mon[29421]: pgmap v474: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 4.5 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 47 MiB/s wr, 749 op/s; 7370/220008 objects misplaced (3.350%); 4.5 MiB/s, 72 objects/s recovering 2024-09-20T18:59:32.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:32 smithi083 ceph-mon[36260]: pgmap v474: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 4.5 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 47 MiB/s wr, 749 op/s; 7370/220008 objects misplaced (3.350%); 4.5 MiB/s, 72 objects/s recovering 2024-09-20T18:59:32.597 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:32.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:32.636 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:32.636 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:32.737 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:32.771 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:32.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:32.772 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:32.872 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:59:32.913 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:32.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:32.914 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:59:33.015 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:59:33.050 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:33.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:33.051 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:59:33.152 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:59:33.188 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:33.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:33.189 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:59:33.254 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 53 16 78794 78778 92.8825 66 0.0111162 0.0107612 2024-09-20T18:59:33.289 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:59:33.314 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:33.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:33.315 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:59:33.416 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:33.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:33.451 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:33.452 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:33.552 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:33.588 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:33.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:33.589 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:33.689 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:33.724 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:33.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:33.725 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:33.825 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:59:33.868 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:33.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:33.869 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:59:33.970 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:34.002 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:34.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:34.003 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:34.104 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:59:34.138 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:34.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:34.139 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:59:34.240 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:34.255 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 54 16 79869 79853 92.4066 67.1875 0.0137865 0.0108156 2024-09-20T18:59:34.272 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:34.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:34.273 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:34.373 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:59:34.397 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:34.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:34.398 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:59:34.499 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:59:34.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:34 smithi142 ceph-mon[32508]: pgmap v475: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 4.7 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 78 MiB/s wr, 1.25k op/s; 6763/231951 objects misplaced (2.916%); 9.2 MiB/s, 147 objects/s recovering 2024-09-20T18:59:34.530 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:34.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:34.531 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:59:34.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:34 smithi083 ceph-mon[29421]: pgmap v475: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 4.7 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 78 MiB/s wr, 1.25k op/s; 6763/231951 objects misplaced (2.916%); 9.2 MiB/s, 147 objects/s recovering 2024-09-20T18:59:34.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:34 smithi083 ceph-mon[36260]: pgmap v475: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 4.7 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 78 MiB/s wr, 1.25k op/s; 6763/231951 objects misplaced (2.916%); 9.2 MiB/s, 147 objects/s recovering 2024-09-20T18:59:34.631 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:59:34.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:34.669 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:34.669 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:59:34.770 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:59:34.801 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:34.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:34.802 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:59:34.903 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:59:34.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:34.944 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:34.944 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:59:35.045 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:59:35.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:35.083 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:35.083 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:59:35.184 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:35.223 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:35.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:35.224 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:35.255 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 55 15 80812 80797 91.7992 59 0.0245506 0.0108883 2024-09-20T18:59:35.325 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:35.359 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:35.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:35.360 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:35.461 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:35.497 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:35.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:35.498 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:35.599 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:35.632 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:35.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:35.633 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:35.734 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:59:35.767 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:35.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:35.768 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:59:35.869 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:59:35.919 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:35.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:35.920 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:59:35.953 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-20T18:59:36.020 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:36.070 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:36.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:36.071 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:36.172 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:59:36.179 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-20T18:59:36.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:36.231 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:36.231 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:59:36.256 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 56 16 81796 81780 91.257 61.4375 0.022388 0.0109516 2024-09-20T18:59:36.266 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:36 smithi142 ceph-mon[32508]: pgmap v476: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 4.7 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 639 B/s rd, 46 MiB/s wr, 736 op/s; 6763/231951 objects misplaced (2.916%); 6.7 MiB/s, 106 objects/s recovering 2024-09-20T18:59:36.266 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:36 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3401692989' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:59:36.266 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:36 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:59:36.296 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:36 smithi083 ceph-mon[36260]: pgmap v476: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 4.7 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 639 B/s rd, 46 MiB/s wr, 736 op/s; 6763/231951 objects misplaced (2.916%); 6.7 MiB/s, 106 objects/s recovering 2024-09-20T18:59:36.297 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:36 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3401692989' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:59:36.297 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:36 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:59:36.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:36 smithi083 ceph-mon[29421]: pgmap v476: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 4.7 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 639 B/s rd, 46 MiB/s wr, 736 op/s; 6763/231951 objects misplaced (2.916%); 6.7 MiB/s, 106 objects/s recovering 2024-09-20T18:59:36.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:36 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3401692989' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:59:36.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:36 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:59:36.325 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:59:36.332 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:36.377 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:36.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:36.378 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:36.445 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:59:36.479 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:36.524 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:36.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:36.525 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:36.567 INFO:teuthology.orchestra.run.smithi083.stderr:instructing pg(s) [9.b] on osd.0 to force-recovery; instructing pg(s) [9.0] on osd.1 to force-recovery; 2024-09-20T18:59:36.567 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-09-20T18:59:36.626 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:36.666 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:36.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:36.667 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:36.768 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:59:36.807 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:36.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:36.808 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:59:36.909 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:59:36.954 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:36.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:36.955 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:59:36.969 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T18:59:36.969 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-09-20T18:59:36.969 INFO:teuthology.orchestra.run.smithi142.stdout: "num_ops": 0 2024-09-20T18:59:36.969 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T18:59:37.056 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:59:37.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:37.089 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:37.089 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:59:37.157 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:59:36 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:59:37.191 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:59:37.232 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:37.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:37.233 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:59:37.255 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 57 16 82746 82730 90.6976 59.375 0.0210047 0.0110183 2024-09-20T18:59:37.333 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:37.379 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:37.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:37.380 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:37.480 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:59:37.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:37 smithi142 ceph-mon[32508]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:59:37.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:37 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T18:59:37.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:37 smithi142 ceph-mon[32508]: osdmap e268: 8 total, 8 up, 8 in 2024-09-20T18:59:37.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:37 smithi142 ceph-mon[32508]: from='client.15654 -' entity='client.admin' cmd=[{"prefix": "pg force-recovery", "pgid": ["9.b", "9.0"], "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:59:37.516 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:37.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:37.517 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:59:37.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:37 smithi083 ceph-mon[29421]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:59:37.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:37 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T18:59:37.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:37 smithi083 ceph-mon[29421]: osdmap e268: 8 total, 8 up, 8 in 2024-09-20T18:59:37.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:37 smithi083 ceph-mon[29421]: from='client.15654 -' entity='client.admin' cmd=[{"prefix": "pg force-recovery", "pgid": ["9.b", "9.0"], "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:59:37.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:37 smithi083 ceph-mon[36260]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:59:37.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:37 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T18:59:37.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:37 smithi083 ceph-mon[36260]: osdmap e268: 8 total, 8 up, 8 in 2024-09-20T18:59:37.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:37 smithi083 ceph-mon[36260]: from='client.15654 -' entity='client.admin' cmd=[{"prefix": "pg force-recovery", "pgid": ["9.b", "9.0"], "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T18:59:37.618 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:37.653 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:37.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:37.654 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:37.755 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:59:37.804 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:37.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:37.805 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:59:37.808 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_blocked_ops 2024-09-20T18:59:37.906 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:37.943 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:37.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:37.944 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:38.044 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:59:38.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:38.070 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:38.071 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:59:38.171 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:38.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:38 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:59:38.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:38 smithi142 ceph-mon[32508]: pgmap v478: 148 pgs: 1 active+recovering+forced_recovery+undersized+remapped, 2 active+recovering+undersized+remapped, 145 active+clean; 5.0 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.5 KiB/s rd, 81 MiB/s wr, 1.30k op/s; 6383/243996 objects misplaced (2.616%); 9.0 MiB/s, 143 objects/s recovering 2024-09-20T18:59:38.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:38.219 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:38.220 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:38.246 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:59:38.255 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 58 16 83695 83679 90.1564 59.3125 0.00945631 0.0110864 2024-09-20T18:59:38.320 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:38.359 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:38.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:38.360 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:38.461 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:59:38.498 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:38.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:38.499 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:59:38.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:38 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:59:38.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:38 smithi083 ceph-mon[29421]: pgmap v478: 148 pgs: 1 active+recovering+forced_recovery+undersized+remapped, 2 active+recovering+undersized+remapped, 145 active+clean; 5.0 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.5 KiB/s rd, 81 MiB/s wr, 1.30k op/s; 6383/243996 objects misplaced (2.616%); 9.0 MiB/s, 143 objects/s recovering 2024-09-20T18:59:38.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:38 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:59:38.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:38 smithi083 ceph-mon[36260]: pgmap v478: 148 pgs: 1 active+recovering+forced_recovery+undersized+remapped, 2 active+recovering+undersized+remapped, 145 active+clean; 5.0 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.5 KiB/s rd, 81 MiB/s wr, 1.30k op/s; 6383/243996 objects misplaced (2.616%); 9.0 MiB/s, 143 objects/s recovering 2024-09-20T18:59:38.600 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:38.623 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:38.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:38.624 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:38.725 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:38.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:38.773 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:38.773 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:38.874 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:38.915 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:38.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:38.916 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:39.017 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:39.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:39.047 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:39.047 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:39.054 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T18:59:39.054 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-09-20T18:59:39.054 INFO:teuthology.orchestra.run.smithi142.stdout: "complaint_time": 30, 2024-09-20T18:59:39.054 INFO:teuthology.orchestra.run.smithi142.stdout: "num_blocked_ops": 0 2024-09-20T18:59:39.054 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T18:59:39.148 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:39.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:39.190 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:39.190 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:39.255 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 59 16 84626 84610 89.6146 58.1875 0.0219684 0.0111534 2024-09-20T18:59:39.290 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:39.326 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:39.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:39.327 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:39.428 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:39.470 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:39.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:39.471 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:39.572 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:39.604 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:39.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:39.605 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:39.706 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:39.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:39.736 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:39.736 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:39.800 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T18:59:39.804 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_historic_ops 2024-09-20T18:59:39.837 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:39.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:39.887 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:39.887 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:39.988 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:59:40.022 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:40.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:40.024 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:59:40.124 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:40.177 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:40.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:40.178 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:40.220 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:59:40.255 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:2024-09-20T18:59:40.246626+0000 min lat: 0.00133037 max lat: 2.15034 avg lat: 0.0111993 2024-09-20T18:59:40.255 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-20T18:59:40.255 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 60 16 85711 85695 89.2511 67.8125 0.00818469 0.0111993 2024-09-20T18:59:40.279 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:59:40.316 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:40.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:40.317 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:59:40.350 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:40 smithi142 ceph-mon[32508]: pgmap v479: 148 pgs: 2 active+recovering+forced_recovery+undersized+remapped, 1 active+recovering+undersized+remapped, 145 active+clean; 5.0 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 75 MiB/s wr, 1.20k op/s; 6129/246729 objects misplaced (2.484%); 9.4 MiB/s, 150 objects/s recovering 2024-09-20T18:59:40.418 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:40.497 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:40.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:40.498 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:40.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:40 smithi083 ceph-mon[29421]: pgmap v479: 148 pgs: 2 active+recovering+forced_recovery+undersized+remapped, 1 active+recovering+undersized+remapped, 145 active+clean; 5.0 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 75 MiB/s wr, 1.20k op/s; 6129/246729 objects misplaced (2.484%); 9.4 MiB/s, 150 objects/s recovering 2024-09-20T18:59:40.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:40 smithi083 ceph-mon[36260]: pgmap v479: 148 pgs: 2 active+recovering+forced_recovery+undersized+remapped, 1 active+recovering+undersized+remapped, 145 active+clean; 5.0 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 75 MiB/s wr, 1.20k op/s; 6129/246729 objects misplaced (2.484%); 9.4 MiB/s, 150 objects/s recovering 2024-09-20T18:59:40.600 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:40.638 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:40.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:40.639 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:40.740 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:59:40.789 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:40.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:40.790 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:59:40.856 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T18:59:40.856 INFO:teuthology.orchestra.run.smithi142.stdout: "size": 20, 2024-09-20T18:59:40.856 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 600, 2024-09-20T18:59:40.857 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-09-20T18:59:40.857 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.857 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:26463 7.1 7:878a0722:::benchmark_data_smithi083_70643_object26462:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:59:40.857 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:07.994856+0000", 2024-09-20T18:59:40.857 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 512.84969249400001, 2024-09-20T18:59:40.857 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13761967, 2024-09-20T18:59:40.857 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:59:40.857 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:59:40.857 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:40.857 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:59:40.857 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:59:40.857 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:59:40.857 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 26463 2024-09-20T18:59:40.857 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.858 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:59:40.858 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.858 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:59:40.858 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.994856+0000", 2024-09-20T18:59:40.858 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:40.858 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.858 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.858 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:59:40.858 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.994856+0000", 2024-09-20T18:59:40.858 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:40.858 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.858 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.859 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:59:40.859 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.994858+0000", 2024-09-20T18:59:40.859 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.649e-06 2024-09-20T18:59:40.859 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.859 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.859 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:59:40.859 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.994924+0000", 2024-09-20T18:59:40.859 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.6484999999999998e-05 2024-09-20T18:59:40.859 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.859 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.859 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:59:40.859 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.994937+0000", 2024-09-20T18:59:40.859 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3084e-05 2024-09-20T18:59:40.859 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.860 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.860 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:59:40.860 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.994945+0000", 2024-09-20T18:59:40.860 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.4889999999999998e-06 2024-09-20T18:59:40.860 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.860 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.860 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:59:40.860 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.994984+0000", 2024-09-20T18:59:40.860 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9270999999999997e-05 2024-09-20T18:59:40.860 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.860 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.861 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:59:40.861 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.995073+0000", 2024-09-20T18:59:40.861 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.8747999999999998e-05 2024-09-20T18:59:40.861 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.861 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.861 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,6", 2024-09-20T18:59:40.861 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.995161+0000", 2024-09-20T18:59:40.861 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.7538000000000001e-05 2024-09-20T18:59:40.861 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.861 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.861 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:59:40.862 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.996461+0000", 2024-09-20T18:59:40.862 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0013008060000000001 2024-09-20T18:59:40.862 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.862 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.862 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:40.862 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.996788+0000", 2024-09-20T18:59:40.862 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00032643100000000002 2024-09-20T18:59:40.862 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.862 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.862 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:40.862 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.132406+0000", 2024-09-20T18:59:40.862 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13561836799999999 2024-09-20T18:59:40.862 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.862 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.863 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:59:40.863 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.132445+0000", 2024-09-20T18:59:40.863 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.8640999999999998e-05 2024-09-20T18:59:40.863 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.863 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.863 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:59:40.863 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.132476+0000", 2024-09-20T18:59:40.863 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.116e-05 2024-09-20T18:59:40.863 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:40.863 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:59:40.863 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:40.863 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.863 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.864 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:26470 7.1 7:85a6a71e:::benchmark_data_smithi083_70643_object26469:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:59:40.864 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:07.997258+0000", 2024-09-20T18:59:40.864 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 512.84729115100004, 2024-09-20T18:59:40.864 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13539621399999999, 2024-09-20T18:59:40.864 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:59:40.864 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:59:40.864 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:40.864 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:59:40.864 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:59:40.864 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:59:40.864 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 26470 2024-09-20T18:59:40.864 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.864 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:59:40.864 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.865 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:59:40.865 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.997258+0000", 2024-09-20T18:59:40.865 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:40.865 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.865 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.865 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:59:40.865 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.997258+0000", 2024-09-20T18:59:40.865 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:40.865 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.865 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.865 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:59:40.865 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.997259+0000", 2024-09-20T18:59:40.866 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6810000000000001e-06 2024-09-20T18:59:40.866 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.866 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.866 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:59:40.866 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.997317+0000", 2024-09-20T18:59:40.866 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8174000000000002e-05 2024-09-20T18:59:40.866 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.866 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.866 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:59:40.866 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.997332+0000", 2024-09-20T18:59:40.866 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4687e-05 2024-09-20T18:59:40.866 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.866 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.866 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:59:40.867 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.997340+0000", 2024-09-20T18:59:40.867 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.4089999999999997e-06 2024-09-20T18:59:40.867 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.867 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.867 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:59:40.867 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.997395+0000", 2024-09-20T18:59:40.867 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5318000000000001e-05 2024-09-20T18:59:40.867 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.867 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.867 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:59:40.867 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.997488+0000", 2024-09-20T18:59:40.867 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.2930000000000006e-05 2024-09-20T18:59:40.867 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.867 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.868 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,6", 2024-09-20T18:59:40.868 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.997587+0000", 2024-09-20T18:59:40.868 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.9167999999999996e-05 2024-09-20T18:59:40.868 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.868 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.868 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:59:40.868 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.998680+0000", 2024-09-20T18:59:40.868 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001092759 2024-09-20T18:59:40.868 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.868 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.868 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:40.868 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.001637+0000", 2024-09-20T18:59:40.868 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.002956991 2024-09-20T18:59:40.868 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.869 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.869 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:40.869 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.132549+0000", 2024-09-20T18:59:40.869 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13091224700000001 2024-09-20T18:59:40.869 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.869 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.869 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:59:40.869 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.132609+0000", 2024-09-20T18:59:40.869 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0123e-05 2024-09-20T18:59:40.869 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.869 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.869 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:59:40.869 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.132654+0000", 2024-09-20T18:59:40.870 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.4727000000000001e-05 2024-09-20T18:59:40.870 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:40.870 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:59:40.870 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:40.870 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.870 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.870 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:27565 7.1 7:8e95a048:::benchmark_data_smithi083_70643_object27564:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T18:59:40.870 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:08.738835+0000", 2024-09-20T18:59:40.870 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 512.10571410900002, 2024-09-20T18:59:40.870 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.103678123, 2024-09-20T18:59:40.870 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:59:40.870 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:59:40.870 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:40.871 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:59:40.871 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T18:59:40.871 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T18:59:40.872 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 27565 2024-09-20T18:59:40.872 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.872 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:59:40.872 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.872 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:59:40.872 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.738835+0000", 2024-09-20T18:59:40.872 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:40.872 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.872 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.872 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:59:40.872 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.738835+0000", 2024-09-20T18:59:40.872 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:40.872 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.873 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.873 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:59:40.873 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.738836+0000", 2024-09-20T18:59:40.873 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.451e-06 2024-09-20T18:59:40.873 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.873 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.873 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:59:40.873 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.738941+0000", 2024-09-20T18:59:40.873 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00010476000000000001 2024-09-20T18:59:40.873 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.873 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.873 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:59:40.873 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.738949+0000", 2024-09-20T18:59:40.873 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.1869999999999997e-06 2024-09-20T18:59:40.874 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.874 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.874 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:59:40.874 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.738955+0000", 2024-09-20T18:59:40.874 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.908e-06 2024-09-20T18:59:40.874 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.874 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.874 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:59:40.874 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.739267+0000", 2024-09-20T18:59:40.874 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00031249899999999999 2024-09-20T18:59:40.874 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.874 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.874 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:59:40.875 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.739392+0000", 2024-09-20T18:59:40.875 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012450699999999999 2024-09-20T18:59:40.875 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.875 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.875 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,6", 2024-09-20T18:59:40.875 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.739517+0000", 2024-09-20T18:59:40.875 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012527400000000001 2024-09-20T18:59:40.875 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.875 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.875 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:59:40.875 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.740572+0000", 2024-09-20T18:59:40.875 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001054452 2024-09-20T18:59:40.875 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.875 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.875 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:40.876 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.741601+0000", 2024-09-20T18:59:40.876 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0010288160000000001 2024-09-20T18:59:40.876 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.876 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.876 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:40.876 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.842412+0000", 2024-09-20T18:59:40.876 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100811759 2024-09-20T18:59:40.876 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.876 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.876 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:59:40.876 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.842469+0000", 2024-09-20T18:59:40.876 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.7169000000000001e-05 2024-09-20T18:59:40.876 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.876 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.876 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:59:40.877 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:08.842513+0000", 2024-09-20T18:59:40.877 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3340999999999997e-05 2024-09-20T18:59:40.877 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:40.877 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:59:40.877 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:40.877 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.877 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.877 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.24955.0:20800 8.2 8:432f30b8:::benchmark_data_smithi083_97422_object20799:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e166)", 2024-09-20T18:59:40.877 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:54:47.519031+0000", 2024-09-20T18:59:40.877 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 293.32551755600002, 2024-09-20T18:59:40.877 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10456202000000001, 2024-09-20T18:59:40.877 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:59:40.877 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:59:40.877 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:40.877 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:59:40.878 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.24955", 2024-09-20T18:59:40.878 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:59:40.878 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 20800 2024-09-20T18:59:40.878 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.878 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:59:40.878 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.878 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:59:40.878 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:47.519031+0000", 2024-09-20T18:59:40.878 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:40.878 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.878 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.878 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:59:40.878 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:47.519031+0000", 2024-09-20T18:59:40.878 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:40.878 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.879 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.879 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:59:40.879 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:47.519038+0000", 2024-09-20T18:59:40.879 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.9419999999999998e-06 2024-09-20T18:59:40.879 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.879 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.879 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:59:40.879 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:47.519078+0000", 2024-09-20T18:59:40.879 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9749000000000002e-05 2024-09-20T18:59:40.879 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.879 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.879 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:59:40.879 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:47.519096+0000", 2024-09-20T18:59:40.879 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.844e-05 2024-09-20T18:59:40.879 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.880 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.880 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:59:40.880 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:47.519106+0000", 2024-09-20T18:59:40.880 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.3179999999999996e-06 2024-09-20T18:59:40.880 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.880 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.880 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:59:40.880 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:47.519166+0000", 2024-09-20T18:59:40.880 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.054e-05 2024-09-20T18:59:40.880 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.880 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.880 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:59:40.880 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:47.519300+0000", 2024-09-20T18:59:40.881 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013389399999999999 2024-09-20T18:59:40.881 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.881 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.881 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:59:40.881 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:47.519448+0000", 2024-09-20T18:59:40.881 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00014763499999999999 2024-09-20T18:59:40.881 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.881 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.881 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:59:40.881 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:47.520929+0000", 2024-09-20T18:59:40.881 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001480931 2024-09-20T18:59:40.881 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.881 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.881 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:40.882 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:47.521905+0000", 2024-09-20T18:59:40.882 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00097665300000000007 2024-09-20T18:59:40.882 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.882 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.882 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:40.882 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:47.623479+0000", 2024-09-20T18:59:40.882 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.101573859 2024-09-20T18:59:40.882 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.882 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.882 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:59:40.882 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:47.623542+0000", 2024-09-20T18:59:40.882 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2996999999999993e-05 2024-09-20T18:59:40.882 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.882 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.883 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:59:40.883 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:54:47.623593+0000", 2024-09-20T18:59:40.883 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.1062e-05 2024-09-20T18:59:40.883 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:40.883 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:59:40.883 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:40.883 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.883 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.883 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.24955.0:99043 8.5 8:a3384a22:::benchmark_data_smithi083_97422_object99042:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e189)", 2024-09-20T18:59:40.883 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:55:49.495752+0000", 2024-09-20T18:59:40.883 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 231.34879723500001, 2024-09-20T18:59:40.883 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.104598647, 2024-09-20T18:59:40.883 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:59:40.883 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:59:40.883 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:40.884 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:59:40.884 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.24955", 2024-09-20T18:59:40.884 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:59:40.884 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 99043 2024-09-20T18:59:40.884 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.884 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:59:40.884 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.884 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:59:40.884 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:49.495752+0000", 2024-09-20T18:59:40.884 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:40.884 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.884 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.884 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:59:40.884 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:49.495752+0000", 2024-09-20T18:59:40.884 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:40.885 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.885 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.885 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:59:40.885 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:49.495753+0000", 2024-09-20T18:59:40.885 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4100000000000001e-06 2024-09-20T18:59:40.885 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.885 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.885 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:59:40.885 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:49.495779+0000", 2024-09-20T18:59:40.885 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.6149e-05 2024-09-20T18:59:40.885 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.885 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.885 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:59:40.886 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:49.495788+0000", 2024-09-20T18:59:40.886 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.5320000000000006e-06 2024-09-20T18:59:40.886 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.886 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.886 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:59:40.886 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:49.495794+0000", 2024-09-20T18:59:40.886 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0140000000000004e-06 2024-09-20T18:59:40.886 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.886 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.886 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:59:40.886 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:49.501736+0000", 2024-09-20T18:59:40.886 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0059419399999999997 2024-09-20T18:59:40.886 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.886 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.887 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:59:40.887 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:49.501859+0000", 2024-09-20T18:59:40.887 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012343099999999999 2024-09-20T18:59:40.887 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.887 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.887 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,7", 2024-09-20T18:59:40.887 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:49.501971+0000", 2024-09-20T18:59:40.887 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000112006 2024-09-20T18:59:40.887 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.887 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.887 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:59:40.887 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:49.504082+0000", 2024-09-20T18:59:40.887 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0021111189999999998 2024-09-20T18:59:40.887 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.888 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.888 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:40.888 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:49.504367+0000", 2024-09-20T18:59:40.888 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00028447800000000001 2024-09-20T18:59:40.888 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.888 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.888 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:40.888 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:49.598292+0000", 2024-09-20T18:59:40.888 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.093925566000000002 2024-09-20T18:59:40.888 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.888 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.888 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:59:40.888 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:49.598354+0000", 2024-09-20T18:59:40.888 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2025000000000003e-05 2024-09-20T18:59:40.888 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.889 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.889 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:59:40.889 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:49.600350+0000", 2024-09-20T18:59:40.889 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0019959769999999999 2024-09-20T18:59:40.889 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:40.889 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:59:40.889 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:40.889 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.889 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.889 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.24955.0:101471 8.5 8:a0258bcb:::benchmark_data_smithi083_97422_object101470:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e190)", 2024-09-20T18:59:40.889 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:55:52.379230+0000", 2024-09-20T18:59:40.889 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 228.46531887200001, 2024-09-20T18:59:40.890 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.15025427499999999, 2024-09-20T18:59:40.890 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:59:40.890 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:59:40.890 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:40.890 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:59:40.890 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.24955", 2024-09-20T18:59:40.890 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:59:40.890 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 101471 2024-09-20T18:59:40.890 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.890 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:59:40.890 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.890 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:59:40.890 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.379230+0000", 2024-09-20T18:59:40.891 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:40.891 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.891 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.891 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:59:40.891 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.379230+0000", 2024-09-20T18:59:40.891 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:40.891 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.891 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.891 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:59:40.891 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.379232+0000", 2024-09-20T18:59:40.891 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1600000000000001e-06 2024-09-20T18:59:40.891 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.891 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.891 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:59:40.891 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.379275+0000", 2024-09-20T18:59:40.891 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3075999999999999e-05 2024-09-20T18:59:40.892 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.892 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.892 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:59:40.892 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.379289+0000", 2024-09-20T18:59:40.892 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4336e-05 2024-09-20T18:59:40.892 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.892 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.892 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:59:40.892 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.379302+0000", 2024-09-20T18:59:40.892 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2565000000000001e-05 2024-09-20T18:59:40.892 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.892 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.892 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:59:40.892 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.379364+0000", 2024-09-20T18:59:40.892 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2192000000000004e-05 2024-09-20T18:59:40.893 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.893 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.893 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:59:40.893 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.379508+0000", 2024-09-20T18:59:40.893 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000143974 2024-09-20T18:59:40.893 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.893 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.893 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,7", 2024-09-20T18:59:40.893 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.379687+0000", 2024-09-20T18:59:40.893 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00017867800000000001 2024-09-20T18:59:40.893 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.893 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.893 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:59:40.893 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.381844+0000", 2024-09-20T18:59:40.893 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0021569969999999999 2024-09-20T18:59:40.894 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.894 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.894 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:40.894 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.394790+0000", 2024-09-20T18:59:40.894 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.012945778999999999 2024-09-20T18:59:40.894 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.894 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.894 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:40.894 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.527006+0000", 2024-09-20T18:59:40.894 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13221633499999999 2024-09-20T18:59:40.894 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.894 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.894 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:59:40.894 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.527073+0000", 2024-09-20T18:59:40.894 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.6665e-05 2024-09-20T18:59:40.895 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.895 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.895 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:59:40.895 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.529484+0000", 2024-09-20T18:59:40.895 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0024115180000000001 2024-09-20T18:59:40.895 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:40.895 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:59:40.895 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:40.895 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.895 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.895 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.24955.0:101490 8.5 8:a93ae8b2:::benchmark_data_smithi083_97422_object101489:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e190)", 2024-09-20T18:59:40.895 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:55:52.406070+0000", 2024-09-20T18:59:40.895 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 228.438479016, 2024-09-20T18:59:40.895 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.12551009399999999, 2024-09-20T18:59:40.895 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:59:40.896 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:59:40.896 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:40.896 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:59:40.896 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.24955", 2024-09-20T18:59:40.896 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:59:40.896 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 101490 2024-09-20T18:59:40.896 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.896 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:59:40.896 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.896 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:59:40.896 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.406070+0000", 2024-09-20T18:59:40.896 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:40.896 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.896 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.896 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:59:40.897 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.406070+0000", 2024-09-20T18:59:40.897 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:40.897 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.897 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.897 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:59:40.897 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.406071+0000", 2024-09-20T18:59:40.897 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.637e-06 2024-09-20T18:59:40.897 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.897 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.897 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:59:40.897 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.406116+0000", 2024-09-20T18:59:40.897 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.4690000000000001e-05 2024-09-20T18:59:40.897 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.897 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.897 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:59:40.897 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.406129+0000", 2024-09-20T18:59:40.898 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2771e-05 2024-09-20T18:59:40.898 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.898 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.898 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:59:40.898 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.406138+0000", 2024-09-20T18:59:40.898 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.7940000000000008e-06 2024-09-20T18:59:40.898 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.898 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.898 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:59:40.898 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.406814+0000", 2024-09-20T18:59:40.898 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00067630399999999997 2024-09-20T18:59:40.898 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.898 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.898 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:59:40.898 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.406969+0000", 2024-09-20T18:59:40.899 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000155045 2024-09-20T18:59:40.899 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.899 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.899 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,7", 2024-09-20T18:59:40.899 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.407159+0000", 2024-09-20T18:59:40.899 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00019038 2024-09-20T18:59:40.899 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.899 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.899 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:40.899 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.415551+0000", 2024-09-20T18:59:40.899 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0083918759999999995 2024-09-20T18:59:40.899 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.899 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.900 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:59:40.900 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.415572+0000", 2024-09-20T18:59:40.900 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1038000000000001e-05 2024-09-20T18:59:40.900 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.900 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.900 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:40.900 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.529549+0000", 2024-09-20T18:59:40.900 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.113976494 2024-09-20T18:59:40.900 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.900 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.900 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:59:40.900 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.529613+0000", 2024-09-20T18:59:40.900 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.3967999999999995e-05 2024-09-20T18:59:40.900 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.901 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.901 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:59:40.901 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.531580+0000", 2024-09-20T18:59:40.901 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0019670970000000001 2024-09-20T18:59:40.901 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:40.901 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:59:40.901 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:40.901 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.901 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.901 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.24955.0:153674 8.5 8:a73bace0:::benchmark_data_smithi083_97422_object37583:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e212)", 2024-09-20T18:59:40.901 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:56:55.029317+0000", 2024-09-20T18:59:40.901 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 165.81523206, 2024-09-20T18:59:40.901 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.115623772, 2024-09-20T18:59:40.901 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:59:40.901 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:59:40.902 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:40.902 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:59:40.902 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.24955", 2024-09-20T18:59:40.902 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:59:40.902 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 153674 2024-09-20T18:59:40.902 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.902 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:59:40.902 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.902 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:59:40.902 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.029317+0000", 2024-09-20T18:59:40.902 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:40.902 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.902 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.902 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:59:40.903 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.029317+0000", 2024-09-20T18:59:40.903 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:40.903 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.903 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.903 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:59:40.903 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.029318+0000", 2024-09-20T18:59:40.903 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2300000000000001e-06 2024-09-20T18:59:40.903 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.903 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.903 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:59:40.903 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.029323+0000", 2024-09-20T18:59:40.903 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7890000000000002e-06 2024-09-20T18:59:40.903 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.903 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.904 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:59:40.904 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.029334+0000", 2024-09-20T18:59:40.904 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0890000000000001e-05 2024-09-20T18:59:40.904 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.904 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.904 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:59:40.904 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.029341+0000", 2024-09-20T18:59:40.904 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.2300000000000002e-06 2024-09-20T18:59:40.904 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.904 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.904 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:59:40.904 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.030115+0000", 2024-09-20T18:59:40.904 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00077372499999999996 2024-09-20T18:59:40.904 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.904 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.905 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:59:40.905 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.030242+0000", 2024-09-20T18:59:40.905 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012694900000000001 2024-09-20T18:59:40.905 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.905 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.905 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,7", 2024-09-20T18:59:40.905 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.030375+0000", 2024-09-20T18:59:40.905 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013332799999999999 2024-09-20T18:59:40.905 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.905 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.905 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:40.905 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.037210+0000", 2024-09-20T18:59:40.905 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.006835608 2024-09-20T18:59:40.905 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.906 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.906 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:59:40.906 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.037253+0000", 2024-09-20T18:59:40.906 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.2688000000000002e-05 2024-09-20T18:59:40.906 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.906 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.906 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:40.906 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.143712+0000", 2024-09-20T18:59:40.906 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10645900599999999 2024-09-20T18:59:40.906 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.906 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.906 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:59:40.906 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.143756+0000", 2024-09-20T18:59:40.906 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3423000000000001e-05 2024-09-20T18:59:40.907 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.907 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.907 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:59:40.907 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.144940+0000", 2024-09-20T18:59:40.907 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0011849059999999999 2024-09-20T18:59:40.907 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:40.907 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:59:40.907 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:40.907 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.907 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.907 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.24955.0:153681 8.2 8:49e1e3cd:::benchmark_data_smithi083_97422_object37590:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e212)", 2024-09-20T18:59:40.907 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:56:55.032508+0000", 2024-09-20T18:59:40.907 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 165.812040946, 2024-09-20T18:59:40.908 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.11130313999999999, 2024-09-20T18:59:40.908 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:59:40.908 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:59:40.908 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:40.908 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:59:40.908 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.24955", 2024-09-20T18:59:40.908 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:59:40.908 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 153681 2024-09-20T18:59:40.908 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.908 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:59:40.908 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.908 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:59:40.908 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.032508+0000", 2024-09-20T18:59:40.908 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:40.908 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.908 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.909 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:59:40.909 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.032508+0000", 2024-09-20T18:59:40.909 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:40.909 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.909 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.909 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:59:40.909 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.032508+0000", 2024-09-20T18:59:40.909 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.99e-07 2024-09-20T18:59:40.909 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.909 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.909 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:59:40.909 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.032511+0000", 2024-09-20T18:59:40.909 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9790000000000002e-06 2024-09-20T18:59:40.909 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.909 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.909 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:59:40.909 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.032518+0000", 2024-09-20T18:59:40.910 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.195e-06 2024-09-20T18:59:40.910 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.910 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.910 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:59:40.910 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.032523+0000", 2024-09-20T18:59:40.910 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4949999999999999e-06 2024-09-20T18:59:40.910 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.910 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.910 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:59:40.910 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.035207+0000", 2024-09-20T18:59:40.910 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0026842369999999999 2024-09-20T18:59:40.910 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.910 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.910 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:59:40.911 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.035344+0000", 2024-09-20T18:59:40.911 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013628699999999999 2024-09-20T18:59:40.911 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.911 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.911 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:59:40.911 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.035465+0000", 2024-09-20T18:59:40.911 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000121826 2024-09-20T18:59:40.911 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.911 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.911 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:59:40.911 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.037871+0000", 2024-09-20T18:59:40.911 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0024054749999999998 2024-09-20T18:59:40.911 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.911 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.911 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:40.912 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.037952+0000", 2024-09-20T18:59:40.912 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.1516999999999999e-05 2024-09-20T18:59:40.912 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.912 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.912 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:40.912 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.143738+0000", 2024-09-20T18:59:40.912 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10578591800000001 2024-09-20T18:59:40.912 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.912 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.912 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:59:40.912 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.143776+0000", 2024-09-20T18:59:40.912 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.8077000000000002e-05 2024-09-20T18:59:40.912 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.912 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.912 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:59:40.912 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.143811+0000", 2024-09-20T18:59:40.913 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4535000000000001e-05 2024-09-20T18:59:40.913 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:40.913 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:59:40.913 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:40.913 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.913 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.913 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.24955.0:153690 8.2 8:41fc6567:::benchmark_data_smithi083_97422_object37599:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e212)", 2024-09-20T18:59:40.913 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:56:55.046063+0000", 2024-09-20T18:59:40.913 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 165.79848591000001, 2024-09-20T18:59:40.913 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10965862799999999, 2024-09-20T18:59:40.913 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:59:40.913 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:59:40.913 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:40.913 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:59:40.913 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.24955", 2024-09-20T18:59:40.914 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:59:40.914 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 153690 2024-09-20T18:59:40.914 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.914 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:59:40.914 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.914 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:59:40.914 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.046063+0000", 2024-09-20T18:59:40.914 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:40.914 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.914 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.914 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:59:40.914 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.046063+0000", 2024-09-20T18:59:40.914 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:40.914 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.914 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.914 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:59:40.915 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.046064+0000", 2024-09-20T18:59:40.915 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.8700000000000004e-07 2024-09-20T18:59:40.915 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.915 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.915 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:59:40.915 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.046068+0000", 2024-09-20T18:59:40.915 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.2409999999999997e-06 2024-09-20T18:59:40.915 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.915 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.915 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:59:40.915 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.046076+0000", 2024-09-20T18:59:40.915 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.32e-06 2024-09-20T18:59:40.915 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.915 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.916 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:59:40.916 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.046081+0000", 2024-09-20T18:59:40.916 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.6099999999999999e-06 2024-09-20T18:59:40.916 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.916 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.916 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:59:40.916 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.046122+0000", 2024-09-20T18:59:40.916 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0899999999999998e-05 2024-09-20T18:59:40.916 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.916 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.916 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:59:40.916 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.046205+0000", 2024-09-20T18:59:40.916 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.3238999999999994e-05 2024-09-20T18:59:40.916 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.916 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.917 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:59:40.917 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.046281+0000", 2024-09-20T18:59:40.917 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5574999999999994e-05 2024-09-20T18:59:40.917 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.917 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.917 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:59:40.917 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.050403+0000", 2024-09-20T18:59:40.917 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.004122222 2024-09-20T18:59:40.917 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.917 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.917 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:40.917 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.050457+0000", 2024-09-20T18:59:40.917 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.41e-05 2024-09-20T18:59:40.917 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.918 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.918 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:40.918 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.155616+0000", 2024-09-20T18:59:40.918 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.105158548 2024-09-20T18:59:40.918 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.918 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.918 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:59:40.918 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.155673+0000", 2024-09-20T18:59:40.918 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.783e-05 2024-09-20T18:59:40.918 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.918 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.918 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:59:40.918 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.155721+0000", 2024-09-20T18:59:40.918 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8056000000000002e-05 2024-09-20T18:59:40.918 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:40.919 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:59:40.919 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:40.919 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.919 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.919 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.24955.0:153692 8.2 8:4b1fbe78:::benchmark_data_smithi083_97422_object37601:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e212)", 2024-09-20T18:59:40.919 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:56:55.046091+0000", 2024-09-20T18:59:40.919 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 165.79845742399999, 2024-09-20T18:59:40.919 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10979633, 2024-09-20T18:59:40.919 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:59:40.919 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:59:40.919 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:40.919 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:59:40.919 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.24955", 2024-09-20T18:59:40.919 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:59:40.919 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 153692 2024-09-20T18:59:40.920 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.920 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:59:40.920 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.920 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:59:40.920 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.046091+0000", 2024-09-20T18:59:40.920 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:40.920 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.920 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.920 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:59:40.920 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.046091+0000", 2024-09-20T18:59:40.920 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:40.920 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.920 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.920 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:59:40.921 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.046092+0000", 2024-09-20T18:59:40.921 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.51e-07 2024-09-20T18:59:40.921 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.921 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.921 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:59:40.921 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.046094+0000", 2024-09-20T18:59:40.921 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0779999999999998e-06 2024-09-20T18:59:40.921 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.921 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.921 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:59:40.921 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.046100+0000", 2024-09-20T18:59:40.921 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9939999999999997e-06 2024-09-20T18:59:40.921 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.921 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.922 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:59:40.922 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.046103+0000", 2024-09-20T18:59:40.922 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.1860000000000001e-06 2024-09-20T18:59:40.922 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.922 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.922 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:59:40.922 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.048138+0000", 2024-09-20T18:59:40.922 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0020351330000000002 2024-09-20T18:59:40.922 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.922 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.922 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:59:40.922 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.048238+0000", 2024-09-20T18:59:40.922 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000100129 2024-09-20T18:59:40.922 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.922 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.923 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:59:40.923 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.048350+0000", 2024-09-20T18:59:40.923 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000111575 2024-09-20T18:59:40.923 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.923 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.923 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:59:40.923 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.050518+0000", 2024-09-20T18:59:40.923 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0021680509999999998 2024-09-20T18:59:40.923 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.923 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.923 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:40.923 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.050759+0000", 2024-09-20T18:59:40.923 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000240685 2024-09-20T18:59:40.923 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.924 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.924 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:40.924 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.155790+0000", 2024-09-20T18:59:40.924 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.105031161 2024-09-20T18:59:40.924 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:59:40.925 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.925 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.926 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:59:40.926 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.155846+0000", 2024-09-20T18:59:40.926 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5971999999999999e-05 2024-09-20T18:59:40.926 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.926 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.926 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:59:40.926 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:55.155888+0000", 2024-09-20T18:59:40.926 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.1915000000000001e-05 2024-09-20T18:59:40.926 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:40.926 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:59:40.926 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:40.926 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.926 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.927 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.24955.0:155226 8.5 8:a088136e:::benchmark_data_smithi083_97422_object39135:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e212)", 2024-09-20T18:59:40.927 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:56:56.880020+0000", 2024-09-20T18:59:40.927 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 163.96452920999999, 2024-09-20T18:59:40.927 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.107864349, 2024-09-20T18:59:40.927 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:59:40.927 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:59:40.927 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:40.927 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:59:40.927 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.24955", 2024-09-20T18:59:40.927 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:59:40.927 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 155226 2024-09-20T18:59:40.927 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.927 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:59:40.927 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.927 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:59:40.927 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:56.880020+0000", 2024-09-20T18:59:40.928 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:40.928 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.928 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.928 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:59:40.928 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:56.880020+0000", 2024-09-20T18:59:40.928 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:40.928 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.928 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.928 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:59:40.928 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:56.880021+0000", 2024-09-20T18:59:40.928 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5829999999999999e-06 2024-09-20T18:59:40.928 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.928 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.928 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:59:40.928 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:56.880030+0000", 2024-09-20T18:59:40.928 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.0580000000000004e-06 2024-09-20T18:59:40.929 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.929 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.929 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:59:40.929 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:56.880053+0000", 2024-09-20T18:59:40.929 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3133e-05 2024-09-20T18:59:40.929 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.929 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.929 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:59:40.929 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:56.880061+0000", 2024-09-20T18:59:40.929 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.4050000000000003e-06 2024-09-20T18:59:40.929 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.929 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.929 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:59:40.929 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:56.880170+0000", 2024-09-20T18:59:40.929 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000109473 2024-09-20T18:59:40.929 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.930 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.930 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:59:40.930 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:56.880299+0000", 2024-09-20T18:59:40.930 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012856599999999999 2024-09-20T18:59:40.930 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.930 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.930 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,7", 2024-09-20T18:59:40.930 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:56.880454+0000", 2024-09-20T18:59:40.930 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000155019 2024-09-20T18:59:40.930 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.930 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.930 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:59:40.930 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:56.882678+0000", 2024-09-20T18:59:40.930 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0022238380000000001 2024-09-20T18:59:40.930 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.931 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.931 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:40.931 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:56.883246+0000", 2024-09-20T18:59:40.931 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00056799499999999996 2024-09-20T18:59:40.931 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.931 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.931 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:40.931 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:56.986776+0000", 2024-09-20T18:59:40.931 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.103530395 2024-09-20T18:59:40.931 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.931 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.931 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:59:40.931 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:56.986836+0000", 2024-09-20T18:59:40.931 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0299e-05 2024-09-20T18:59:40.931 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.932 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.932 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:59:40.932 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:56:56.987884+0000", 2024-09-20T18:59:40.932 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001047585 2024-09-20T18:59:40.932 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:40.932 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:59:40.932 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:40.932 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.932 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.932 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.24955.0:216247 8.2 8:437badeb:::benchmark_data_smithi083_97422_object100156:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-20T18:59:40.932 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:57:59.598310+0000", 2024-09-20T18:59:40.932 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 101.24623885, 2024-09-20T18:59:40.932 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10748500699999999, 2024-09-20T18:59:40.933 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:59:40.933 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:59:40.933 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:40.933 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:59:40.933 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.24955", 2024-09-20T18:59:40.933 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:59:40.933 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 216247 2024-09-20T18:59:40.933 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.933 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:59:40.933 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.933 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:59:40.933 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:59.598310+0000", 2024-09-20T18:59:40.933 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:40.933 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.933 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.934 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:59:40.934 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:59.598310+0000", 2024-09-20T18:59:40.934 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:40.934 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.934 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.934 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:59:40.934 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:59.598311+0000", 2024-09-20T18:59:40.934 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.6000000000000003e-07 2024-09-20T18:59:40.934 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.934 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.934 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:59:40.934 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:59.598314+0000", 2024-09-20T18:59:40.934 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.224e-06 2024-09-20T18:59:40.935 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.935 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.935 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:59:40.935 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:59.598321+0000", 2024-09-20T18:59:40.935 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.9659999999999999e-06 2024-09-20T18:59:40.935 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.935 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.935 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:59:40.935 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:59.598325+0000", 2024-09-20T18:59:40.935 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9609999999999999e-06 2024-09-20T18:59:40.935 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.935 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.935 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:59:40.935 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:59.598366+0000", 2024-09-20T18:59:40.936 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.138e-05 2024-09-20T18:59:40.936 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.936 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.936 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:59:40.936 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:59.598473+0000", 2024-09-20T18:59:40.936 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000106981 2024-09-20T18:59:40.936 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.936 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.936 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:59:40.936 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:59.598588+0000", 2024-09-20T18:59:40.936 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000114636 2024-09-20T18:59:40.936 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.936 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.936 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:59:40.937 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:59.600244+0000", 2024-09-20T18:59:40.937 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00165617 2024-09-20T18:59:40.937 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.937 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.937 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:40.937 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:59.600707+0000", 2024-09-20T18:59:40.937 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000462616 2024-09-20T18:59:40.937 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.937 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.937 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:40.937 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:59.705543+0000", 2024-09-20T18:59:40.937 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.104835895 2024-09-20T18:59:40.937 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.937 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.937 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:59:40.937 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:59.705694+0000", 2024-09-20T18:59:40.938 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000151891 2024-09-20T18:59:40.938 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.938 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.938 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:59:40.938 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:59.705795+0000", 2024-09-20T18:59:40.938 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00010052699999999999 2024-09-20T18:59:40.938 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:40.938 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:59:40.938 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:40.938 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.938 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.938 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.24955.0:216258 8.5 8:a7216738:::benchmark_data_smithi083_97422_object100167:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-20T18:59:40.938 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:57:59.608016+0000", 2024-09-20T18:59:40.938 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 101.236532424, 2024-09-20T18:59:40.938 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.105039931, 2024-09-20T18:59:40.939 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:59:40.939 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:59:40.939 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:40.939 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:59:40.939 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.24955", 2024-09-20T18:59:40.939 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:59:40.939 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 216258 2024-09-20T18:59:40.939 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.939 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:59:40.939 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.939 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:59:40.939 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:59.608016+0000", 2024-09-20T18:59:40.939 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:40.939 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.939 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.940 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:59:40.940 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:59.608016+0000", 2024-09-20T18:59:40.940 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:40.940 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.940 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.940 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:59:40.940 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:59.608018+0000", 2024-09-20T18:59:40.940 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7549999999999999e-06 2024-09-20T18:59:40.940 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.940 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.940 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:59:40.940 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:59.608024+0000", 2024-09-20T18:59:40.940 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5339999999999999e-06 2024-09-20T18:59:40.940 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.940 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.940 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:59:40.941 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:59.608047+0000", 2024-09-20T18:59:40.941 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3708999999999999e-05 2024-09-20T18:59:40.941 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.941 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.941 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:59:40.941 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:59.608055+0000", 2024-09-20T18:59:40.941 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.729e-06 2024-09-20T18:59:40.941 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.941 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.941 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:59:40.941 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:59.608112+0000", 2024-09-20T18:59:40.941 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6749000000000002e-05 2024-09-20T18:59:40.941 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.941 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.941 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:59:40.942 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:59.608222+0000", 2024-09-20T18:59:40.942 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000110135 2024-09-20T18:59:40.942 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.942 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.942 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,7", 2024-09-20T18:59:40.942 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:59.608329+0000", 2024-09-20T18:59:40.942 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000107114 2024-09-20T18:59:40.942 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.942 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.942 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:59:40.942 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:59.611764+0000", 2024-09-20T18:59:40.942 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0034345539999999998 2024-09-20T18:59:40.942 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.942 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.943 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:40.943 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:59.611804+0000", 2024-09-20T18:59:40.943 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0068999999999999e-05 2024-09-20T18:59:40.943 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.943 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.943 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:40.943 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:59.712966+0000", 2024-09-20T18:59:40.943 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.101162236 2024-09-20T18:59:40.943 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.943 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.943 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:59:40.943 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:59.713006+0000", 2024-09-20T18:59:40.943 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9628000000000001e-05 2024-09-20T18:59:40.943 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.944 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.944 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:59:40.944 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:59.713056+0000", 2024-09-20T18:59:40.944 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0719e-05 2024-09-20T18:59:40.944 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:40.944 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:59:40.944 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:40.944 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.944 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.944 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.24955.0:216259 8.5 8:a089bad9:::benchmark_data_smithi083_97422_object100168:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-20T18:59:40.944 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:57:59.609987+0000", 2024-09-20T18:59:40.944 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 101.234562021, 2024-09-20T18:59:40.944 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10322257899999999, 2024-09-20T18:59:40.944 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:59:40.945 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:59:40.945 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:40.945 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:59:40.945 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.24955", 2024-09-20T18:59:40.945 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:59:40.945 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 216259 2024-09-20T18:59:40.945 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.945 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:59:40.945 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.945 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:59:40.945 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:59.609987+0000", 2024-09-20T18:59:40.945 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:40.945 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.945 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.946 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:59:40.946 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:59.609987+0000", 2024-09-20T18:59:40.946 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:40.946 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.946 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.946 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:59:40.946 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:59.609988+0000", 2024-09-20T18:59:40.946 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.8000000000000004e-07 2024-09-20T18:59:40.946 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.946 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.946 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:59:40.946 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:59.609991+0000", 2024-09-20T18:59:40.946 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.8249999999999998e-06 2024-09-20T18:59:40.947 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.947 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.947 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:59:40.947 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:59.610002+0000", 2024-09-20T18:59:40.947 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0101e-05 2024-09-20T18:59:40.947 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.947 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.947 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:59:40.947 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:59.610007+0000", 2024-09-20T18:59:40.947 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4260000000000002e-06 2024-09-20T18:59:40.947 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.947 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.947 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:59:40.948 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:59.610418+0000", 2024-09-20T18:59:40.948 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000411426 2024-09-20T18:59:40.948 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.948 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.948 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:59:40.948 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:59.610512+0000", 2024-09-20T18:59:40.948 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.3259999999999998e-05 2024-09-20T18:59:40.948 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.948 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.948 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,7", 2024-09-20T18:59:40.948 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:59.610583+0000", 2024-09-20T18:59:40.948 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.0926999999999994e-05 2024-09-20T18:59:40.948 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.948 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.949 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:59:40.949 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:59.611885+0000", 2024-09-20T18:59:40.949 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0013026089999999999 2024-09-20T18:59:40.949 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.949 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.949 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:40.949 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:59.612083+0000", 2024-09-20T18:59:40.949 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00019735299999999999 2024-09-20T18:59:40.949 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.949 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.949 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:40.949 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:59.713117+0000", 2024-09-20T18:59:40.949 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.101034292 2024-09-20T18:59:40.949 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.950 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.950 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:59:40.950 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:59.713169+0000", 2024-09-20T18:59:40.950 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.2170999999999999e-05 2024-09-20T18:59:40.950 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.950 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.950 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:59:40.950 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:57:59.713209+0000", 2024-09-20T18:59:40.950 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0309000000000003e-05 2024-09-20T18:59:40.950 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:40.950 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:59:40.950 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:40.950 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.951 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.951 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.24955.0:217585 8.2 8:4293c90a:::benchmark_data_smithi083_97422_object101494:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-20T18:59:40.951 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:58:01.171935+0000", 2024-09-20T18:59:40.951 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 99.672613708, 2024-09-20T18:59:40.951 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.15055682200000001, 2024-09-20T18:59:40.951 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:59:40.951 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:59:40.951 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:40.951 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:59:40.951 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.24955", 2024-09-20T18:59:40.951 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:59:40.951 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 217585 2024-09-20T18:59:40.951 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.952 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:59:40.952 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.952 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:59:40.952 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:01.171935+0000", 2024-09-20T18:59:40.952 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:40.952 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.952 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.952 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:59:40.952 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:01.171935+0000", 2024-09-20T18:59:40.952 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:40.952 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.952 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.952 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:59:40.953 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:01.171936+0000", 2024-09-20T18:59:40.953 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.102e-06 2024-09-20T18:59:40.953 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.953 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.953 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:59:40.953 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:01.171941+0000", 2024-09-20T18:59:40.953 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.493e-06 2024-09-20T18:59:40.953 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.953 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.953 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:59:40.953 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:01.171953+0000", 2024-09-20T18:59:40.953 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2074e-05 2024-09-20T18:59:40.953 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.954 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.954 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:59:40.954 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:01.171960+0000", 2024-09-20T18:59:40.954 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.8950000000000001e-06 2024-09-20T18:59:40.954 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.954 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.954 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:59:40.954 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:01.172015+0000", 2024-09-20T18:59:40.954 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5000000000000002e-05 2024-09-20T18:59:40.954 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.954 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.954 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:59:40.954 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:01.172120+0000", 2024-09-20T18:59:40.954 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000105515 2024-09-20T18:59:40.955 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.955 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.955 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:59:40.955 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:01.172227+0000", 2024-09-20T18:59:40.955 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000106648 2024-09-20T18:59:40.955 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.955 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.955 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:59:40.955 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:01.174594+0000", 2024-09-20T18:59:40.955 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0023676970000000002 2024-09-20T18:59:40.955 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.955 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.955 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:40.956 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:01.175040+0000", 2024-09-20T18:59:40.956 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00044583300000000003 2024-09-20T18:59:40.956 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.956 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.956 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:40.956 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:01.322400+0000", 2024-09-20T18:59:40.956 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.14736007000000001 2024-09-20T18:59:40.956 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.956 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.956 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:59:40.956 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:01.322449+0000", 2024-09-20T18:59:40.956 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8979e-05 2024-09-20T18:59:40.956 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.956 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.956 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:59:40.957 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:01.322492+0000", 2024-09-20T18:59:40.957 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.2515999999999997e-05 2024-09-20T18:59:40.957 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:40.957 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:59:40.957 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:40.957 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.957 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.957 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.24955.0:218183 8.2 8:4b7a1d73:::benchmark_data_smithi083_97422_object102092:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-20T18:59:40.957 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:58:01.889376+0000", 2024-09-20T18:59:40.957 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 98.955173259999995, 2024-09-20T18:59:40.957 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.19228917000000001, 2024-09-20T18:59:40.957 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:59:40.957 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:59:40.957 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:40.958 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:59:40.958 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.24955", 2024-09-20T18:59:40.958 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:59:40.958 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 218183 2024-09-20T18:59:40.958 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.958 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:59:40.958 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.958 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:59:40.958 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:01.889376+0000", 2024-09-20T18:59:40.958 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:40.958 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.958 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.958 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:59:40.958 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:01.889376+0000", 2024-09-20T18:59:40.959 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:40.959 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.959 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.959 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:59:40.959 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:01.889377+0000", 2024-09-20T18:59:40.959 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0869999999999999e-06 2024-09-20T18:59:40.959 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.959 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.959 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:59:40.959 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:01.889381+0000", 2024-09-20T18:59:40.959 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.6809999999999997e-06 2024-09-20T18:59:40.959 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.959 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.959 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:59:40.959 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:01.889391+0000", 2024-09-20T18:59:40.959 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0162e-05 2024-09-20T18:59:40.960 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.960 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.960 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:59:40.960 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:01.889398+0000", 2024-09-20T18:59:40.960 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2870000000000001e-06 2024-09-20T18:59:40.960 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.960 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.960 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:59:40.960 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:01.889442+0000", 2024-09-20T18:59:40.960 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.4379000000000003e-05 2024-09-20T18:59:40.960 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.960 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.960 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:59:40.960 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:01.889563+0000", 2024-09-20T18:59:40.960 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000120568 2024-09-20T18:59:40.960 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.961 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.961 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,5", 2024-09-20T18:59:40.961 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:01.889676+0000", 2024-09-20T18:59:40.961 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000113721 2024-09-20T18:59:40.961 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.961 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.961 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:40.961 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:01.891822+0000", 2024-09-20T18:59:40.961 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0021460279999999999 2024-09-20T18:59:40.961 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.961 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.961 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:59:40.961 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:01.891828+0000", 2024-09-20T18:59:40.961 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9780000000000002e-06 2024-09-20T18:59:40.962 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.962 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.962 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:40.962 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:02.081564+0000", 2024-09-20T18:59:40.962 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.189735352 2024-09-20T18:59:40.962 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.962 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.962 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:59:40.962 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:02.081618+0000", 2024-09-20T18:59:40.962 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4413999999999998e-05 2024-09-20T18:59:40.962 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.962 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.962 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:59:40.962 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:02.081665+0000", 2024-09-20T18:59:40.962 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.6513e-05 2024-09-20T18:59:40.962 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:40.963 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:59:40.963 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:40.963 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.963 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.963 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.24955.0:218297 8.5 8:a149d94e:::benchmark_data_smithi083_97422_object102206:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-20T18:59:40.963 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:58:02.262171+0000", 2024-09-20T18:59:40.963 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 98.582377457999996, 2024-09-20T18:59:40.963 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.15427866600000001, 2024-09-20T18:59:40.963 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:59:40.963 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:59:40.963 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:40.963 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:59:40.963 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.24955", 2024-09-20T18:59:40.963 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:59:40.963 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 218297 2024-09-20T18:59:40.963 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.964 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:59:40.964 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.964 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:59:40.964 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:02.262171+0000", 2024-09-20T18:59:40.964 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:40.964 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.964 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.964 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:59:40.964 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:02.262171+0000", 2024-09-20T18:59:40.964 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:40.964 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.964 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.964 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:59:40.964 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:02.262173+0000", 2024-09-20T18:59:40.964 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.485e-06 2024-09-20T18:59:40.965 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.965 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.965 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:59:40.965 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:02.262178+0000", 2024-09-20T18:59:40.965 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.9119999999999997e-06 2024-09-20T18:59:40.965 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.965 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.965 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:59:40.965 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:02.262189+0000", 2024-09-20T18:59:40.965 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1243000000000001e-05 2024-09-20T18:59:40.965 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.965 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.965 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:59:40.965 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:02.262195+0000", 2024-09-20T18:59:40.965 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9610000000000002e-06 2024-09-20T18:59:40.966 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.966 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.966 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:59:40.966 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:02.262256+0000", 2024-09-20T18:59:40.966 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.1288e-05 2024-09-20T18:59:40.966 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.966 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.966 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:59:40.966 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:02.262375+0000", 2024-09-20T18:59:40.966 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011862300000000001 2024-09-20T18:59:40.966 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.966 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.966 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,7", 2024-09-20T18:59:40.966 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:02.262479+0000", 2024-09-20T18:59:40.967 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000104441 2024-09-20T18:59:40.967 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.967 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.967 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:59:40.967 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:02.266405+0000", 2024-09-20T18:59:40.967 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0039256339999999999 2024-09-20T18:59:40.967 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.967 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.967 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:40.967 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:02.268262+0000", 2024-09-20T18:59:40.967 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0018568790000000001 2024-09-20T18:59:40.967 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.967 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.968 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:40.968 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:02.416348+0000", 2024-09-20T18:59:40.968 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.14808643399999999 2024-09-20T18:59:40.968 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.968 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.968 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:59:40.968 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:02.416400+0000", 2024-09-20T18:59:40.968 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.2290000000000002e-05 2024-09-20T18:59:40.968 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.968 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.968 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:59:40.968 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:02.416450+0000", 2024-09-20T18:59:40.968 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.9475999999999999e-05 2024-09-20T18:59:40.968 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:40.969 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:59:40.969 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:40.969 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.969 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.969 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.24955.0:218300 8.5 8:a3e450c0:::benchmark_data_smithi083_97422_object102209:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-20T18:59:40.969 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:58:02.263192+0000", 2024-09-20T18:59:40.969 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 98.581356845000002, 2024-09-20T18:59:40.969 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.153362473, 2024-09-20T18:59:40.969 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:59:40.969 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:59:40.969 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:40.969 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:59:40.969 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.24955", 2024-09-20T18:59:40.969 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:59:40.969 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 218300 2024-09-20T18:59:40.970 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.970 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:59:40.970 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.970 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:59:40.970 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:02.263192+0000", 2024-09-20T18:59:40.970 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:40.970 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.970 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.970 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:59:40.970 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:02.263192+0000", 2024-09-20T18:59:40.970 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:40.970 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.970 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.970 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:59:40.970 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:02.263193+0000", 2024-09-20T18:59:40.971 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0249999999999999e-06 2024-09-20T18:59:40.971 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.971 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.971 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:59:40.971 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:02.263197+0000", 2024-09-20T18:59:40.971 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9049999999999999e-06 2024-09-20T18:59:40.971 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.971 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.971 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:59:40.971 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:02.263207+0000", 2024-09-20T18:59:40.971 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.7769999999999997e-06 2024-09-20T18:59:40.971 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.971 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.971 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:59:40.971 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:02.263213+0000", 2024-09-20T18:59:40.972 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2829999999999998e-06 2024-09-20T18:59:40.972 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.972 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.972 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:59:40.972 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:02.264401+0000", 2024-09-20T18:59:40.972 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0011878819999999999 2024-09-20T18:59:40.972 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.972 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.972 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:59:40.972 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:02.264508+0000", 2024-09-20T18:59:40.972 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000107642 2024-09-20T18:59:40.972 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.972 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.972 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,7", 2024-09-20T18:59:40.972 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:02.264594+0000", 2024-09-20T18:59:40.973 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.5499000000000003e-05 2024-09-20T18:59:40.973 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.973 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.973 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:59:40.973 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:02.268228+0000", 2024-09-20T18:59:40.973 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0036342319999999998 2024-09-20T18:59:40.973 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.973 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.973 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:40.973 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:02.268294+0000", 2024-09-20T18:59:40.973 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5653000000000003e-05 2024-09-20T18:59:40.973 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.973 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.974 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:40.974 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:02.416488+0000", 2024-09-20T18:59:40.974 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.148194247 2024-09-20T18:59:40.974 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.974 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.974 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:59:40.974 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:02.416520+0000", 2024-09-20T18:59:40.974 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.1528e-05 2024-09-20T18:59:40.974 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.974 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.974 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:59:40.974 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:02.416554+0000", 2024-09-20T18:59:40.974 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4799999999999999e-05 2024-09-20T18:59:40.975 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:40.975 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:59:40.975 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:40.975 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.975 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.975 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.24955.0:218303 8.5 8:af8cf81e:::benchmark_data_smithi083_97422_object102212:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-20T18:59:40.975 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:58:02.265905+0000", 2024-09-20T18:59:40.975 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 98.578644143000005, 2024-09-20T18:59:40.975 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.15222290099999999, 2024-09-20T18:59:40.975 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:59:40.975 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:59:40.975 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T18:59:40.975 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:59:40.975 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.24955", 2024-09-20T18:59:40.976 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T18:59:40.976 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 218303 2024-09-20T18:59:40.976 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.976 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:59:40.976 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.976 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:59:40.976 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:02.265905+0000", 2024-09-20T18:59:40.976 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:40.976 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.976 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.976 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:59:40.976 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:02.265905+0000", 2024-09-20T18:59:40.976 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:40.976 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.976 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.977 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:59:40.977 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:02.265906+0000", 2024-09-20T18:59:40.977 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.333e-06 2024-09-20T18:59:40.977 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.977 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.977 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:59:40.977 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:02.265911+0000", 2024-09-20T18:59:40.977 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0490000000000003e-06 2024-09-20T18:59:40.977 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.977 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.977 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:59:40.977 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:02.265925+0000", 2024-09-20T18:59:40.977 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3757999999999999e-05 2024-09-20T18:59:40.978 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.978 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.978 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:59:40.978 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:02.265933+0000", 2024-09-20T18:59:40.978 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.8229999999999995e-06 2024-09-20T18:59:40.978 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.978 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.978 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:59:40.978 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:02.266437+0000", 2024-09-20T18:59:40.978 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00050453399999999995 2024-09-20T18:59:40.978 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.978 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.978 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:59:40.978 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:02.266558+0000", 2024-09-20T18:59:40.979 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000120754 2024-09-20T18:59:40.979 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.979 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.979 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,7", 2024-09-20T18:59:40.979 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:02.266679+0000", 2024-09-20T18:59:40.979 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012160899999999999 2024-09-20T18:59:40.979 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.979 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.979 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:59:40.979 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:02.268305+0000", 2024-09-20T18:59:40.979 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0016251519999999999 2024-09-20T18:59:40.979 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.979 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.980 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:40.980 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:02.268321+0000", 2024-09-20T18:59:40.980 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6722999999999999e-05 2024-09-20T18:59:40.980 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.980 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.980 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:40.980 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:02.418030+0000", 2024-09-20T18:59:40.980 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.14970910100000001 2024-09-20T18:59:40.980 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.980 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.980 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T18:59:40.980 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:02.418090+0000", 2024-09-20T18:59:40.980 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.914e-05 2024-09-20T18:59:40.980 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:40.980 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:40.981 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T18:59:40.981 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:58:02.418128+0000", 2024-09-20T18:59:40.981 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.7925e-05 2024-09-20T18:59:40.981 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:40.981 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:59:40.981 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:40.981 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:40.981 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:59:40.981 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T18:59:41.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:41.014 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:41.014 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:59:41.046 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:59:40 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:59:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:59:41.115 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:59:41.150 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:41.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:41.151 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:59:41.252 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:41.260 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 61 16 86649 86633 88.7484 58.625 0.0106474 0.0112634 2024-09-20T18:59:41.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:41.309 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:41.309 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:41.410 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:59:41.444 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:41.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:41.445 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:59:41.546 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:59:41.585 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:41.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:41.586 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:59:41.687 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:41.718 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:41.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:41.719 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:41.819 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:59:41.853 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:41.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:41.854 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:59:41.955 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:41.990 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-20T18:59:41.993 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:41.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:41.994 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:42.046 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:41 smithi083 ceph-mon[29421]: osdmap e269: 8 total, 8 up, 8 in 2024-09-20T18:59:42.046 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:41 smithi083 ceph-mon[29421]: pgmap v481: 148 pgs: 2 active+recovering+forced_recovery+undersized+remapped, 1 active+recovering+undersized+remapped, 145 active+clean; 5.1 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 46 MiB/s wr, 729 op/s; 6129/249432 objects misplaced (2.457%); 5.0 MiB/s, 79 objects/s recovering 2024-09-20T18:59:42.046 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:41 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3432122910' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:59:42.047 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:41 smithi083 ceph-mon[36260]: osdmap e269: 8 total, 8 up, 8 in 2024-09-20T18:59:42.047 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:41 smithi083 ceph-mon[36260]: pgmap v481: 148 pgs: 2 active+recovering+forced_recovery+undersized+remapped, 1 active+recovering+undersized+remapped, 145 active+clean; 5.1 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 46 MiB/s wr, 729 op/s; 6129/249432 objects misplaced (2.457%); 5.0 MiB/s, 79 objects/s recovering 2024-09-20T18:59:42.047 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:41 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3432122910' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:59:42.096 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:59:42.119 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:42.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:42.120 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:59:42.156 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:59:42.156 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:59:42.156 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:59:42.156 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:59:42.156 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:59:42.156 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:59:42.156 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:59:42.156 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:59:42.156 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:59:42.157 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:59:42.157 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:59:42.157 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:59:42.157 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:59:42.157 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:59:42.157 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:59:42.157 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:59:42.221 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:42.258 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 62 16 87513 87497 88.1879 54 0.0157395 0.0113352 2024-09-20T18:59:42.267 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:41 smithi142 ceph-mon[32508]: osdmap e269: 8 total, 8 up, 8 in 2024-09-20T18:59:42.267 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:41 smithi142 ceph-mon[32508]: pgmap v481: 148 pgs: 2 active+recovering+forced_recovery+undersized+remapped, 1 active+recovering+undersized+remapped, 145 active+clean; 5.1 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 46 MiB/s wr, 729 op/s; 6129/249432 objects misplaced (2.457%); 5.0 MiB/s, 79 objects/s recovering 2024-09-20T18:59:42.267 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:41 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3432122910' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:59:42.276 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:42.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:42.277 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:42.334 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:59:42.378 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:59:42.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:42.424 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:42.424 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:59:42.525 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:59:42.550 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:42.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:42.551 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:59:42.652 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:42.683 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:42.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:42.685 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:42.785 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:42.825 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:42.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:42.826 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:42.927 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:42.941 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T18:59:42.942 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:59:42.961 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:42.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:42.962 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:43.046 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:42 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:59:43.047 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:42 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:59:43.047 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:42 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:59:43.047 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:42 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:59:43.063 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:43.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:43.097 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:43.097 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:43.198 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:43.222 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:43.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:43.223 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:43.256 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 63 16 88492 88476 87.7593 61.1875 0.0179188 0.0113884 2024-09-20T18:59:43.266 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:42 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:59:43.267 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:42 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:59:43.324 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:59:43.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:43.361 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:43.361 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:59:43.462 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:43.493 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:43.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:43.494 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:43.595 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:59:43.618 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_blocked_ops 2024-09-20T18:59:43.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:43.635 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:43.635 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:59:43.736 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:59:43.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:43.783 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:43.783 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:59:43.884 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:59:43.920 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:43.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:43.921 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:59:44.022 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:44.034 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:59:44.051 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:44.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:44.052 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:44.153 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:44.166 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:43 smithi142 ceph-mon[32508]: osdmap e270: 8 total, 8 up, 8 in 2024-09-20T18:59:44.166 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:43 smithi142 ceph-mon[32508]: pgmap v483: 148 pgs: 1 active+recovering+undersized+remapped, 2 active+recovering+forced_recovery+undersized+remapped, 145 active+clean; 5.3 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 748 B/s rd, 70 MiB/s wr, 1.11k op/s; 5474/261273 objects misplaced (2.095%); 12 MiB/s, 188 objects/s recovering 2024-09-20T18:59:44.189 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:44.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:44.190 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:44.256 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 64 15 89480 89465 87.3538 61.8125 0.016155 0.0114426 2024-09-20T18:59:44.292 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:59:44.296 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:43 smithi083 ceph-mon[29421]: osdmap e270: 8 total, 8 up, 8 in 2024-09-20T18:59:44.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:43 smithi083 ceph-mon[29421]: pgmap v483: 148 pgs: 1 active+recovering+undersized+remapped, 2 active+recovering+forced_recovery+undersized+remapped, 145 active+clean; 5.3 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 748 B/s rd, 70 MiB/s wr, 1.11k op/s; 5474/261273 objects misplaced (2.095%); 12 MiB/s, 188 objects/s recovering 2024-09-20T18:59:44.297 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:43 smithi083 ceph-mon[36260]: osdmap e270: 8 total, 8 up, 8 in 2024-09-20T18:59:44.297 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:43 smithi083 ceph-mon[36260]: pgmap v483: 148 pgs: 1 active+recovering+undersized+remapped, 2 active+recovering+forced_recovery+undersized+remapped, 145 active+clean; 5.3 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 748 B/s rd, 70 MiB/s wr, 1.11k op/s; 5474/261273 objects misplaced (2.095%); 12 MiB/s, 188 objects/s recovering 2024-09-20T18:59:44.324 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:44.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:44.325 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:59:44.426 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:59:44.457 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:44.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:44.458 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:59:44.558 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:59:44.592 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:44.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:44.593 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:59:44.658 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T18:59:44.658 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:59:44.694 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:59:44.732 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:44.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:44.733 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:59:44.834 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:44.870 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:44.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:44.871 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:44.972 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:45.010 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:45.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:45.011 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:45.018 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 7, 6] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-20T18:59:45.019 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T18:59:45.019 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-20T18:59:45.019 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd dump -f json-pretty 2024-09-20T18:59:45.112 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:59:45.163 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:45.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:45.164 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:59:45.256 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 65 16 90513 90497 87.0021 64.5 0.0239481 0.0114891 2024-09-20T18:59:45.264 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:45.283 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:59:45.289 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:45.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:45.290 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:45.375 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_historic_ops 2024-09-20T18:59:45.392 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:59:45.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:45.430 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:45.430 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:59:45.531 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:45.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:45.560 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:45.560 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:45.661 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:59:45.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:45.698 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:45.698 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:59:45.746 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:59:45.800 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:45.844 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:45.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:45.846 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:45.946 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:45.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:45.996 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:45.996 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:46.097 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:46.145 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:46.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:46.146 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:46.210 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:46 smithi142 ceph-mon[32508]: pgmap v484: 148 pgs: 1 active+recovering+undersized+remapped, 2 active+recovering+forced_recovery+undersized+remapped, 145 active+clean; 5.3 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 639 B/s rd, 45 MiB/s wr, 720 op/s; 5474/261273 objects misplaced (2.095%); 7.1 MiB/s, 113 objects/s recovering 2024-09-20T18:59:46.210 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:46 smithi142 ceph-mon[32508]: osdmap e271: 8 total, 8 up, 8 in 2024-09-20T18:59:46.210 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:46 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:59:46.247 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:59:46.256 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 66 16 91429 91413 86.5513 57.25 0.0176744 0.0115484 2024-09-20T18:59:46.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:46.287 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:46.287 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:59:46.360 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T18:59:46.360 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:59:46.389 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:59:46.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:46.423 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:46.423 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:59:46.524 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:59:46.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:46 smithi083 ceph-mon[29421]: pgmap v484: 148 pgs: 1 active+recovering+undersized+remapped, 2 active+recovering+forced_recovery+undersized+remapped, 145 active+clean; 5.3 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 639 B/s rd, 45 MiB/s wr, 720 op/s; 5474/261273 objects misplaced (2.095%); 7.1 MiB/s, 113 objects/s recovering 2024-09-20T18:59:46.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:46 smithi083 ceph-mon[29421]: osdmap e271: 8 total, 8 up, 8 in 2024-09-20T18:59:46.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:46 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:59:46.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:46 smithi083 ceph-mon[36260]: pgmap v484: 148 pgs: 1 active+recovering+undersized+remapped, 2 active+recovering+forced_recovery+undersized+remapped, 145 active+clean; 5.3 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 639 B/s rd, 45 MiB/s wr, 720 op/s; 5474/261273 objects misplaced (2.095%); 7.1 MiB/s, 113 objects/s recovering 2024-09-20T18:59:46.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:46 smithi083 ceph-mon[36260]: osdmap e271: 8 total, 8 up, 8 in 2024-09-20T18:59:46.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:46 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T18:59:46.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:46.567 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:46.567 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:59:46.668 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:59:46.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:46.703 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:46.703 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:59:46.804 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:59:46.838 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:46.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:46.839 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:59:46.940 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:46.993 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:46.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:46.994 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:47.069 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:59:47.095 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:47.130 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:47.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:47.132 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:47.183 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:59:46 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:59:47.185 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-20T18:59:47.233 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:47.256 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 67 16 92341 92325 86.1102 57 0.0407974 0.0116077 2024-09-20T18:59:47.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:47.266 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:47.266 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:47.322 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T18:59:47.366 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:59:47.413 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:47.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:47.414 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:59:47.515 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:47.558 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:47.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:47.559 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:47.594 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:59:47.660 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:47.745 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:47.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:47.746 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:47.847 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:59:47.885 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:47.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:47.886 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:59:47.987 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:59:48.020 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:48.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:48.021 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:59:48.086 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-20T18:59:48.122 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:48.163 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:48.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:48.164 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:48.256 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 68 16 93243 93227 85.6728 56.375 0.014372 0.0116675 2024-09-20T18:59:48.265 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:48.266 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:48 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:59:48.266 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:48 smithi142 ceph-mon[32508]: pgmap v486: 148 pgs: 1 active+recovering+undersized+remapped, 2 active+recovering+forced_recovery+undersized+remapped, 145 active+clean; 5.5 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.6 KiB/s rd, 88 MiB/s wr, 1.40k op/s; 5018/272949 objects misplaced (1.838%); 11 MiB/s, 177 objects/s recovering 2024-09-20T18:59:48.266 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:48 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:59:48.266 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:48 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1547441792' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:59:48.266 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:48 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:59:48.311 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:48.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:48.313 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:48.414 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:59:48.478 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:48.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:48.479 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:59:48.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:48 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:59:48.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:48 smithi083 ceph-mon[29421]: pgmap v486: 148 pgs: 1 active+recovering+undersized+remapped, 2 active+recovering+forced_recovery+undersized+remapped, 145 active+clean; 5.5 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.6 KiB/s rd, 88 MiB/s wr, 1.40k op/s; 5018/272949 objects misplaced (1.838%); 11 MiB/s, 177 objects/s recovering 2024-09-20T18:59:48.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:48 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:59:48.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:48 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1547441792' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:59:48.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:48 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:59:48.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:48 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:59:48.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:48 smithi083 ceph-mon[36260]: pgmap v486: 148 pgs: 1 active+recovering+undersized+remapped, 2 active+recovering+forced_recovery+undersized+remapped, 145 active+clean; 5.5 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.6 KiB/s rd, 88 MiB/s wr, 1.40k op/s; 5018/272949 objects misplaced (1.838%); 11 MiB/s, 177 objects/s recovering 2024-09-20T18:59:48.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:48 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T18:59:48.548 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:48 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1547441792' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:59:48.548 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:48 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T18:59:48.580 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:59:48.595 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T18:59:48.595 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:59:48.615 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:48.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:48.616 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:59:48.681 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:59:48.681 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T18:59:48.681 INFO:teuthology.orchestra.run.smithi083.stdout: "epoch": 272, 2024-09-20T18:59:48.681 INFO:teuthology.orchestra.run.smithi083.stdout: "fsid": "1ed5a50a-7780-11ef-baf5-efdc52797490", 2024-09-20T18:59:48.681 INFO:teuthology.orchestra.run.smithi083.stdout: "created": "2024-09-20T18:44:10.399800+0000", 2024-09-20T18:59:48.681 INFO:teuthology.orchestra.run.smithi083.stdout: "modified": "2024-09-20T18:59:48.061224+0000", 2024-09-20T18:59:48.681 INFO:teuthology.orchestra.run.smithi083.stdout: "last_up_change": "2024-09-20T18:49:34.836377+0000", 2024-09-20T18:59:48.681 INFO:teuthology.orchestra.run.smithi083.stdout: "last_in_change": "2024-09-20T18:59:04.044029+0000", 2024-09-20T18:59:48.682 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-20T18:59:48.682 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_num": 5804032, 2024-09-20T18:59:48.682 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_set": [ 2024-09-20T18:59:48.682 INFO:teuthology.orchestra.run.smithi083.stdout: "nodeep-scrub", 2024-09-20T18:59:48.682 INFO:teuthology.orchestra.run.smithi083.stdout: "pglog_hardlimit", 2024-09-20T18:59:48.682 INFO:teuthology.orchestra.run.smithi083.stdout: "purged_snapdirs", 2024-09-20T18:59:48.682 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_deletes", 2024-09-20T18:59:48.682 INFO:teuthology.orchestra.run.smithi083.stdout: "sortbitwise" 2024-09-20T18:59:48.683 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T18:59:48.683 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_version": 18, 2024-09-20T18:59:48.683 INFO:teuthology.orchestra.run.smithi083.stdout: "full_ratio": 0.94999998807907104, 2024-09-20T18:59:48.683 INFO:teuthology.orchestra.run.smithi083.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-20T18:59:48.683 INFO:teuthology.orchestra.run.smithi083.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-20T18:59:48.683 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_snapshot": "", 2024-09-20T18:59:48.683 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_max": 9, 2024-09-20T18:59:48.683 INFO:teuthology.orchestra.run.smithi083.stdout: "max_osd": 8, 2024-09-20T18:59:48.683 INFO:teuthology.orchestra.run.smithi083.stdout: "require_min_compat_client": "luminous", 2024-09-20T18:59:48.683 INFO:teuthology.orchestra.run.smithi083.stdout: "min_compat_client": "luminous", 2024-09-20T18:59:48.683 INFO:teuthology.orchestra.run.smithi083.stdout: "require_osd_release": "squid", 2024-09-20T18:59:48.683 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_crimson": false, 2024-09-20T18:59:48.683 INFO:teuthology.orchestra.run.smithi083.stdout: "pools": [ 2024-09-20T18:59:48.683 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.684 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 1, 2024-09-20T18:59:48.684 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": ".mgr", 2024-09-20T18:59:48.684 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-20T18:47:15.285795+0000", 2024-09-20T18:59:48.684 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-20T18:59:48.684 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool", 2024-09-20T18:59:48.684 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-20T18:59:48.684 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 3, 2024-09-20T18:59:48.684 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 2, 2024-09-20T18:59:48.684 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-20T18:59:48.684 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-20T18:59:48.685 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-20T18:59:48.685 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T18:59:48.685 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T18:59:48.685 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-20T18:59:48.685 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-20T18:59:48.685 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-20T18:59:48.685 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 1, 2024-09-20T18:59:48.685 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 1, 2024-09-20T18:59:48.685 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 1, 2024-09-20T18:59:48.685 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 1, 2024-09-20T18:59:48.685 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 1, 2024-09-20T18:59:48.685 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-20T18:59:48.686 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-20T18:59:48.686 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-20T18:59:48.686 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-20T18:59:48.686 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-20T18:59:48.686 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-20T18:59:48.686 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-20T18:59:48.686 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.686 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "254", 2024-09-20T18:59:48.686 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-20T18:59:48.686 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T18:59:48.686 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T18:59:48.686 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-20T18:59:48.686 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-20T18:59:48.687 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 0, 2024-09-20T18:59:48.687 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 0, 2024-09-20T18:59:48.687 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-20T18:59:48.687 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-20T18:59:48.687 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-20T18:59:48.687 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-20T18:59:48.687 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-20T18:59:48.687 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-20T18:59:48.687 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-20T18:59:48.687 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-20T18:59:48.687 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-20T18:59:48.687 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-20T18:59:48.687 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-20T18:59:48.687 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T18:59:48.688 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T18:59:48.688 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T18:59:48.688 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-20T18:59:48.688 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-20T18:59:48.688 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-20T18:59:48.688 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-20T18:59:48.688 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-20T18:59:48.688 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.688 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-20T18:59:48.688 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-20T18:59:48.689 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-20T18:59:48.689 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-20T18:59:48.689 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-20T18:59:48.689 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T18:59:48.689 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-20T18:59:48.689 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-20T18:59:48.689 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-20T18:59:48.689 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-20T18:59:48.689 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-20T18:59:48.689 INFO:teuthology.orchestra.run.smithi083.stdout: "options": { 2024-09-20T18:59:48.689 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_max": 32, 2024-09-20T18:59:48.689 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_min": 1 2024-09-20T18:59:48.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.690 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-20T18:59:48.690 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": {} 2024-09-20T18:59:48.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.690 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-20T18:59:48.690 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-20T18:59:48.690 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 7.8899998664855957, 2024-09-20T18:59:48.690 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 7.8899998664855957, 2024-09-20T18:59:48.690 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 0.37999999523162842, 2024-09-20T18:59:48.690 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 3, 2024-09-20T18:59:48.690 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 3, 2024-09-20T18:59:48.690 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 1, 2024-09-20T18:59:48.690 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 1, 2024-09-20T18:59:48.691 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 1 2024-09-20T18:59:48.691 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:48.691 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.691 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.691 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 2, 2024-09-20T18:59:48.691 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": "datapool", 2024-09-20T18:59:48.691 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-20T18:49:43.452984+0000", 2024-09-20T18:59:48.691 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 8193, 2024-09-20T18:59:48.691 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-20T18:59:48.691 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-20T18:59:48.691 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 3, 2024-09-20T18:59:48.691 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 2, 2024-09-20T18:59:48.691 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-20T18:59:48.691 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-20T18:59:48.692 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-20T18:59:48.692 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T18:59:48.692 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T18:59:48.692 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-20T18:59:48.692 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-20T18:59:48.692 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-20T18:59:48.692 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 3, 2024-09-20T18:59:48.692 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 3, 2024-09-20T18:59:48.692 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 3, 2024-09-20T18:59:48.692 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 3, 2024-09-20T18:59:48.692 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 3, 2024-09-20T18:59:48.692 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-20T18:59:48.692 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-20T18:59:48.692 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-20T18:59:48.693 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-20T18:59:48.693 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-20T18:59:48.693 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-20T18:59:48.693 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-20T18:59:48.693 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.693 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "61", 2024-09-20T18:59:48.693 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-20T18:59:48.693 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T18:59:48.693 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T18:59:48.693 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-20T18:59:48.693 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-20T18:59:48.693 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 2, 2024-09-20T18:59:48.693 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 61, 2024-09-20T18:59:48.693 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-20T18:59:48.693 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-20T18:59:48.694 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-20T18:59:48.694 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-20T18:59:48.694 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-20T18:59:48.694 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-20T18:59:48.694 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-20T18:59:48.694 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-20T18:59:48.694 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-20T18:59:48.694 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-20T18:59:48.694 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-20T18:59:48.694 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T18:59:48.694 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T18:59:48.694 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T18:59:48.694 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-20T18:59:48.695 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-20T18:59:48.695 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-20T18:59:48.695 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-20T18:59:48.695 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-20T18:59:48.695 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.695 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-20T18:59:48.695 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-20T18:59:48.695 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-20T18:59:48.695 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-20T18:59:48.695 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-20T18:59:48.695 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T18:59:48.695 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-20T18:59:48.695 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-20T18:59:48.695 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-20T18:59:48.696 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-20T18:59:48.696 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-20T18:59:48.696 INFO:teuthology.orchestra.run.smithi083.stdout: "options": {}, 2024-09-20T18:59:48.696 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-20T18:59:48.696 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd": {} 2024-09-20T18:59:48.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.696 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-20T18:59:48.696 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-20T18:59:48.696 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 2.6500000953674316, 2024-09-20T18:59:48.696 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 2.6500000953674316, 2024-09-20T18:59:48.696 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 0.87999999523162842, 2024-09-20T18:59:48.696 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 2.3299999237060547, 2024-09-20T18:59:48.696 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 2.3299999237060547, 2024-09-20T18:59:48.696 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 1, 2024-09-20T18:59:48.697 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 1, 2024-09-20T18:59:48.697 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 1 2024-09-20T18:59:48.697 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:48.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.697 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.697 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 3, 2024-09-20T18:59:48.697 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": ".rgw.root", 2024-09-20T18:59:48.697 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-20T18:49:44.217011+0000", 2024-09-20T18:59:48.697 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-20T18:59:48.697 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool", 2024-09-20T18:59:48.697 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-20T18:59:48.697 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 3, 2024-09-20T18:59:48.697 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 2, 2024-09-20T18:59:48.697 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-20T18:59:48.698 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-20T18:59:48.698 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-20T18:59:48.698 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T18:59:48.698 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T18:59:48.698 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-20T18:59:48.698 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-20T18:59:48.698 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-20T18:59:48.698 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 32, 2024-09-20T18:59:48.698 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 32, 2024-09-20T18:59:48.698 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 32, 2024-09-20T18:59:48.698 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 32, 2024-09-20T18:59:48.698 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 32, 2024-09-20T18:59:48.699 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-20T18:59:48.699 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-20T18:59:48.699 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-20T18:59:48.699 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-20T18:59:48.700 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-20T18:59:48.700 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-20T18:59:48.700 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-20T18:59:48.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.700 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "60", 2024-09-20T18:59:48.700 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-20T18:59:48.700 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T18:59:48.700 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T18:59:48.700 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-20T18:59:48.700 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-20T18:59:48.700 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 0, 2024-09-20T18:59:48.701 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 0, 2024-09-20T18:59:48.701 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-20T18:59:48.701 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-20T18:59:48.701 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-20T18:59:48.701 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-20T18:59:48.701 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-20T18:59:48.701 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-20T18:59:48.701 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-20T18:59:48.701 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-20T18:59:48.701 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-20T18:59:48.701 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-20T18:59:48.701 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-20T18:59:48.701 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T18:59:48.701 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T18:59:48.702 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T18:59:48.702 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-20T18:59:48.702 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-20T18:59:48.702 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-20T18:59:48.702 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-20T18:59:48.702 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-20T18:59:48.702 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.702 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-20T18:59:48.702 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-20T18:59:48.702 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-20T18:59:48.702 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-20T18:59:48.702 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-20T18:59:48.702 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T18:59:48.702 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-20T18:59:48.703 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-20T18:59:48.703 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-20T18:59:48.703 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-20T18:59:48.703 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-20T18:59:48.703 INFO:teuthology.orchestra.run.smithi083.stdout: "options": {}, 2024-09-20T18:59:48.703 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-20T18:59:48.703 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw": {} 2024-09-20T18:59:48.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.703 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-20T18:59:48.703 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-20T18:59:48.703 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 1.5, 2024-09-20T18:59:48.703 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 1.5, 2024-09-20T18:59:48.703 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 1, 2024-09-20T18:59:48.704 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 1.5, 2024-09-20T18:59:48.704 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 1.5, 2024-09-20T18:59:48.704 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 1, 2024-09-20T18:59:48.704 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 1, 2024-09-20T18:59:48.704 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 1 2024-09-20T18:59:48.704 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:48.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.704 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.704 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 4, 2024-09-20T18:59:48.704 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": "default.rgw.log", 2024-09-20T18:59:48.704 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-20T18:49:46.007275+0000", 2024-09-20T18:59:48.704 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-20T18:59:48.704 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool", 2024-09-20T18:59:48.704 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-20T18:59:48.705 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 3, 2024-09-20T18:59:48.705 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 2, 2024-09-20T18:59:48.705 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-20T18:59:48.705 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-20T18:59:48.705 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-20T18:59:48.705 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T18:59:48.705 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T18:59:48.705 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-20T18:59:48.705 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-20T18:59:48.705 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-20T18:59:48.705 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 32, 2024-09-20T18:59:48.705 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 32, 2024-09-20T18:59:48.705 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 32, 2024-09-20T18:59:48.705 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 32, 2024-09-20T18:59:48.706 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 32, 2024-09-20T18:59:48.706 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-20T18:59:48.706 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-20T18:59:48.706 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-20T18:59:48.706 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-20T18:59:48.706 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-20T18:59:48.706 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-20T18:59:48.706 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-20T18:59:48.706 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.706 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "62", 2024-09-20T18:59:48.706 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-20T18:59:48.706 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T18:59:48.706 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T18:59:48.707 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-20T18:59:48.707 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-20T18:59:48.707 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 0, 2024-09-20T18:59:48.707 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 0, 2024-09-20T18:59:48.707 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-20T18:59:48.707 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-20T18:59:48.707 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-20T18:59:48.707 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-20T18:59:48.707 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-20T18:59:48.707 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-20T18:59:48.707 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-20T18:59:48.708 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-20T18:59:48.708 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-20T18:59:48.708 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-20T18:59:48.708 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-20T18:59:48.708 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T18:59:48.708 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T18:59:48.708 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T18:59:48.708 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-20T18:59:48.708 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-20T18:59:48.708 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-20T18:59:48.708 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-20T18:59:48.708 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-20T18:59:48.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.708 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-20T18:59:48.709 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-20T18:59:48.709 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-20T18:59:48.709 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-20T18:59:48.709 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-20T18:59:48.709 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T18:59:48.709 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-20T18:59:48.709 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-20T18:59:48.709 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-20T18:59:48.709 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-20T18:59:48.709 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-20T18:59:48.709 INFO:teuthology.orchestra.run.smithi083.stdout: "options": {}, 2024-09-20T18:59:48.709 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-20T18:59:48.709 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw": {} 2024-09-20T18:59:48.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.710 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-20T18:59:48.710 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-20T18:59:48.710 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 2.25, 2024-09-20T18:59:48.710 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 2.25, 2024-09-20T18:59:48.710 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 1, 2024-09-20T18:59:48.710 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 2.25, 2024-09-20T18:59:48.710 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 2.25, 2024-09-20T18:59:48.710 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 1, 2024-09-20T18:59:48.710 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 1, 2024-09-20T18:59:48.710 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 1 2024-09-20T18:59:48.711 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:48.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.711 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.711 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 5, 2024-09-20T18:59:48.711 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": "default.rgw.control", 2024-09-20T18:59:48.711 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-20T18:49:47.961258+0000", 2024-09-20T18:59:48.711 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-20T18:59:48.711 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool", 2024-09-20T18:59:48.711 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-20T18:59:48.711 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 3, 2024-09-20T18:59:48.712 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 2, 2024-09-20T18:59:48.712 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-20T18:59:48.712 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-20T18:59:48.712 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-20T18:59:48.712 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T18:59:48.712 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T18:59:48.712 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-20T18:59:48.712 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-20T18:59:48.712 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-20T18:59:48.712 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 32, 2024-09-20T18:59:48.712 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 32, 2024-09-20T18:59:48.712 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 32, 2024-09-20T18:59:48.713 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 32, 2024-09-20T18:59:48.713 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 32, 2024-09-20T18:59:48.713 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-20T18:59:48.713 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-20T18:59:48.713 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-20T18:59:48.713 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-20T18:59:48.713 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-20T18:59:48.713 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-20T18:59:48.713 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-20T18:59:48.714 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.714 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "64", 2024-09-20T18:59:48.714 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-20T18:59:48.714 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T18:59:48.714 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T18:59:48.714 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-20T18:59:48.714 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-20T18:59:48.714 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 0, 2024-09-20T18:59:48.714 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 0, 2024-09-20T18:59:48.714 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-20T18:59:48.714 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-20T18:59:48.714 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-20T18:59:48.714 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-20T18:59:48.714 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-20T18:59:48.715 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-20T18:59:48.715 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-20T18:59:48.715 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-20T18:59:48.715 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-20T18:59:48.715 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-20T18:59:48.715 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-20T18:59:48.715 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T18:59:48.715 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T18:59:48.715 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T18:59:48.715 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-20T18:59:48.715 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-20T18:59:48.715 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-20T18:59:48.715 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-20T18:59:48.716 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-20T18:59:48.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.716 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-20T18:59:48.716 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-20T18:59:48.716 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-20T18:59:48.716 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-20T18:59:48.716 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-20T18:59:48.716 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T18:59:48.716 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-20T18:59:48.716 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-20T18:59:48.716 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-20T18:59:48.716 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-20T18:59:48.717 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-20T18:59:48.717 INFO:teuthology.orchestra.run.smithi083.stdout: "options": {}, 2024-09-20T18:59:48.717 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-20T18:59:48.717 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw": {} 2024-09-20T18:59:48.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.717 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-20T18:59:48.717 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-20T18:59:48.717 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 1.25, 2024-09-20T18:59:48.717 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 1.25, 2024-09-20T18:59:48.717 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 1, 2024-09-20T18:59:48.717 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 1.25, 2024-09-20T18:59:48.717 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 1.25, 2024-09-20T18:59:48.717 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 1, 2024-09-20T18:59:48.717 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 1, 2024-09-20T18:59:48.718 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 1 2024-09-20T18:59:48.718 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:48.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.718 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.718 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 6, 2024-09-20T18:59:48.718 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": "default.rgw.meta", 2024-09-20T18:59:48.718 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-20T18:49:50.011276+0000", 2024-09-20T18:59:48.718 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-20T18:59:48.718 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool", 2024-09-20T18:59:48.718 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-20T18:59:48.718 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 3, 2024-09-20T18:59:48.718 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 2, 2024-09-20T18:59:48.718 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-20T18:59:48.719 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-20T18:59:48.719 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-20T18:59:48.719 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T18:59:48.719 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T18:59:48.719 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-20T18:59:48.719 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-20T18:59:48.719 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-20T18:59:48.719 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 32, 2024-09-20T18:59:48.719 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 32, 2024-09-20T18:59:48.719 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 32, 2024-09-20T18:59:48.719 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 32, 2024-09-20T18:59:48.719 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 32, 2024-09-20T18:59:48.719 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-20T18:59:48.719 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-20T18:59:48.720 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-20T18:59:48.720 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-20T18:59:48.720 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-20T18:59:48.720 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-20T18:59:48.720 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-20T18:59:48.720 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.720 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "66", 2024-09-20T18:59:48.720 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-20T18:59:48.720 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T18:59:48.720 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T18:59:48.720 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-20T18:59:48.720 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-20T18:59:48.720 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 0, 2024-09-20T18:59:48.721 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 0, 2024-09-20T18:59:48.721 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-20T18:59:48.721 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-20T18:59:48.721 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-20T18:59:48.721 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-20T18:59:48.721 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-20T18:59:48.721 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-20T18:59:48.721 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-20T18:59:48.721 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-20T18:59:48.721 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-20T18:59:48.722 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-20T18:59:48.722 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-20T18:59:48.722 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T18:59:48.722 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T18:59:48.722 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T18:59:48.722 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-20T18:59:48.722 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-20T18:59:48.722 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-20T18:59:48.722 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-20T18:59:48.722 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-20T18:59:48.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.722 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-20T18:59:48.722 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-20T18:59:48.723 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-20T18:59:48.723 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-20T18:59:48.723 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-20T18:59:48.723 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T18:59:48.723 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-20T18:59:48.723 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-20T18:59:48.723 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-20T18:59:48.723 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-20T18:59:48.723 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-20T18:59:48.723 INFO:teuthology.orchestra.run.smithi083.stdout: "options": { 2024-09-20T18:59:48.723 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_bias": 4 2024-09-20T18:59:48.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.723 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-20T18:59:48.723 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw": {} 2024-09-20T18:59:48.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.724 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-20T18:59:48.724 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-20T18:59:48.724 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 1.75, 2024-09-20T18:59:48.724 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 1.75, 2024-09-20T18:59:48.724 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 1, 2024-09-20T18:59:48.724 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 1.75, 2024-09-20T18:59:48.724 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 1.75, 2024-09-20T18:59:48.724 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 1, 2024-09-20T18:59:48.724 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 1, 2024-09-20T18:59:48.724 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 1 2024-09-20T18:59:48.724 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:48.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.725 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.725 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 9, 2024-09-20T18:59:48.725 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": "unique_pool_2", 2024-09-20T18:59:48.725 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-20T18:58:32.024544+0000", 2024-09-20T18:59:48.725 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-20T18:59:48.725 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool", 2024-09-20T18:59:48.725 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-20T18:59:48.725 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 3, 2024-09-20T18:59:48.725 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 2, 2024-09-20T18:59:48.725 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-20T18:59:48.725 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-20T18:59:48.725 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-20T18:59:48.725 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T18:59:48.726 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T18:59:48.726 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-20T18:59:48.726 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-20T18:59:48.726 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-20T18:59:48.726 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 16, 2024-09-20T18:59:48.726 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 16, 2024-09-20T18:59:48.726 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 16, 2024-09-20T18:59:48.726 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 16, 2024-09-20T18:59:48.726 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 16, 2024-09-20T18:59:48.726 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-20T18:59:48.726 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-20T18:59:48.726 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-20T18:59:48.726 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-20T18:59:48.727 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-20T18:59:48.727 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-20T18:59:48.727 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-20T18:59:48.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.727 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "250", 2024-09-20T18:59:48.727 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-20T18:59:48.727 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T18:59:48.727 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T18:59:48.727 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-20T18:59:48.727 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-20T18:59:48.727 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 0, 2024-09-20T18:59:48.727 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 0, 2024-09-20T18:59:48.727 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-20T18:59:48.728 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-20T18:59:48.728 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-20T18:59:48.728 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-20T18:59:48.728 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-20T18:59:48.728 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-20T18:59:48.728 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-20T18:59:48.728 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-20T18:59:48.728 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-20T18:59:48.728 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-20T18:59:48.728 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-20T18:59:48.728 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T18:59:48.728 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T18:59:48.728 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T18:59:48.728 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-20T18:59:48.729 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-20T18:59:48.729 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-20T18:59:48.729 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-20T18:59:48.729 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-20T18:59:48.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.729 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-20T18:59:48.729 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-20T18:59:48.729 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-20T18:59:48.729 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-20T18:59:48.729 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-20T18:59:48.729 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T18:59:48.729 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-20T18:59:48.729 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-20T18:59:48.729 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-20T18:59:48.730 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-20T18:59:48.730 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-20T18:59:48.730 INFO:teuthology.orchestra.run.smithi083.stdout: "options": {}, 2024-09-20T18:59:48.730 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-20T18:59:48.730 INFO:teuthology.orchestra.run.smithi083.stdout: "rados": {} 2024-09-20T18:59:48.730 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.730 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-20T18:59:48.730 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-20T18:59:48.730 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 2, 2024-09-20T18:59:48.730 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 2, 2024-09-20T18:59:48.730 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 1, 2024-09-20T18:59:48.730 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 2, 2024-09-20T18:59:48.730 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 2, 2024-09-20T18:59:48.730 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 1, 2024-09-20T18:59:48.731 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 1, 2024-09-20T18:59:48.731 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 1 2024-09-20T18:59:48.731 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:48.731 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:48.731 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T18:59:48.731 INFO:teuthology.orchestra.run.smithi083.stdout: "osds": [ 2024-09-20T18:59:48.731 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.731 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 0, 2024-09-20T18:59:48.731 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "c714e1ec-f0c3-4054-9792-61b84e53eaa8", 2024-09-20T18:59:48.731 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T18:59:48.732 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T18:59:48.732 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T18:59:48.732 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T18:59:48.732 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T18:59:48.732 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T18:59:48.732 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 8, 2024-09-20T18:59:48.732 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 261, 2024-09-20T18:59:48.732 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T18:59:48.732 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T18:59:48.732 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T18:59:48.732 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:59:48.732 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.732 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:59:48.732 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6801", 2024-09-20T18:59:48.733 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2574084049 2024-09-20T18:59:48.733 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:48.733 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:48.733 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.733 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T18:59:48.733 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:59:48.733 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.733 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:59:48.733 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6802", 2024-09-20T18:59:48.733 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2574084049 2024-09-20T18:59:48.733 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:48.733 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:48.733 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.733 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T18:59:48.734 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:59:48.734 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.734 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:59:48.734 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6804", 2024-09-20T18:59:48.734 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2574084049 2024-09-20T18:59:48.734 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:48.734 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:48.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.734 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T18:59:48.734 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:59:48.734 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.734 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:59:48.734 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6803", 2024-09-20T18:59:48.734 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2574084049 2024-09-20T18:59:48.735 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:48.735 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:48.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.735 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.83:6801/2574084049", 2024-09-20T18:59:48.735 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.83:6802/2574084049", 2024-09-20T18:59:48.735 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.83:6804/2574084049", 2024-09-20T18:59:48.735 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.83:6803/2574084049", 2024-09-20T18:59:48.735 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T18:59:48.735 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T18:59:48.735 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T18:59:48.735 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:48.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.736 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.736 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 1, 2024-09-20T18:59:48.736 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "9dc0f2a8-bafa-4a42-88f5-0301dd58f78d", 2024-09-20T18:59:48.736 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T18:59:48.736 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T18:59:48.736 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T18:59:48.736 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T18:59:48.736 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:59:48.737 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T18:59:48.737 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T18:59:48.737 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 13, 2024-09-20T18:59:48.738 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 261, 2024-09-20T18:59:48.738 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T18:59:48.738 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T18:59:48.738 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T18:59:48.738 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:59:48.738 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.738 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:59:48.738 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6805", 2024-09-20T18:59:48.738 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 4213285798 2024-09-20T18:59:48.738 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:48.738 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:48.738 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.738 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T18:59:48.739 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:59:48.739 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.739 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:59:48.739 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6806", 2024-09-20T18:59:48.739 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 4213285798 2024-09-20T18:59:48.739 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:48.739 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:48.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.739 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T18:59:48.739 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:59:48.739 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.739 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:59:48.739 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6808", 2024-09-20T18:59:48.739 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 4213285798 2024-09-20T18:59:48.740 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:48.740 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:48.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.740 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T18:59:48.740 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:59:48.740 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.740 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:59:48.740 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6807", 2024-09-20T18:59:48.740 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 4213285798 2024-09-20T18:59:48.740 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:48.740 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:48.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.740 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.83:6805/4213285798", 2024-09-20T18:59:48.740 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.83:6806/4213285798", 2024-09-20T18:59:48.740 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.83:6808/4213285798", 2024-09-20T18:59:48.741 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.83:6807/4213285798", 2024-09-20T18:59:48.741 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T18:59:48.741 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T18:59:48.741 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T18:59:48.741 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:48.741 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.741 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.741 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 2, 2024-09-20T18:59:48.741 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "448ab692-ec41-44dc-b2d5-dbefd255b3fd", 2024-09-20T18:59:48.741 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T18:59:48.741 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T18:59:48.742 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T18:59:48.742 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T18:59:48.742 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T18:59:48.742 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T18:59:48.742 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 18, 2024-09-20T18:59:48.742 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 83, 2024-09-20T18:59:48.742 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T18:59:48.742 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T18:59:48.742 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T18:59:48.742 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:59:48.742 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.742 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:59:48.743 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6809", 2024-09-20T18:59:48.743 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 805947571 2024-09-20T18:59:48.743 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:48.743 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:48.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.743 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T18:59:48.743 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:59:48.743 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.743 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:59:48.743 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6810", 2024-09-20T18:59:48.743 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 805947571 2024-09-20T18:59:48.743 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:48.743 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:48.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.744 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T18:59:48.744 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:59:48.744 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.744 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:59:48.744 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6812", 2024-09-20T18:59:48.744 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 805947571 2024-09-20T18:59:48.744 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:48.744 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:48.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.744 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T18:59:48.744 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:59:48.745 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.745 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:59:48.745 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6811", 2024-09-20T18:59:48.745 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 805947571 2024-09-20T18:59:48.745 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:48.745 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:48.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.745 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.83:6809/805947571", 2024-09-20T18:59:48.745 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.83:6810/805947571", 2024-09-20T18:59:48.745 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.83:6812/805947571", 2024-09-20T18:59:48.745 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.83:6811/805947571", 2024-09-20T18:59:48.745 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T18:59:48.745 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T18:59:48.745 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T18:59:48.746 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:48.746 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.746 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.746 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 3, 2024-09-20T18:59:48.746 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "7cda5e50-e22e-4d4a-823b-53c8f5061193", 2024-09-20T18:59:48.746 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T18:59:48.746 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T18:59:48.746 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T18:59:48.746 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T18:59:48.746 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T18:59:48.746 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T18:59:48.746 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 25, 2024-09-20T18:59:48.746 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 261, 2024-09-20T18:59:48.746 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T18:59:48.747 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T18:59:48.747 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T18:59:48.747 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:59:48.747 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.747 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:59:48.747 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6813", 2024-09-20T18:59:48.747 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 987237318 2024-09-20T18:59:48.747 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:48.747 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:48.747 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.747 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T18:59:48.747 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:59:48.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.748 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:59:48.748 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6814", 2024-09-20T18:59:48.748 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 987237318 2024-09-20T18:59:48.748 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:48.748 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:48.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.748 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T18:59:48.748 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:59:48.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.748 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:59:48.748 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6816", 2024-09-20T18:59:48.748 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 987237318 2024-09-20T18:59:48.748 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:48.749 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:48.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.749 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T18:59:48.749 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:59:48.749 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.749 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:59:48.749 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6815", 2024-09-20T18:59:48.749 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 987237318 2024-09-20T18:59:48.749 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:48.749 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:48.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.749 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.83:6813/987237318", 2024-09-20T18:59:48.749 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.83:6814/987237318", 2024-09-20T18:59:48.749 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.83:6816/987237318", 2024-09-20T18:59:48.750 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.83:6815/987237318", 2024-09-20T18:59:48.750 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T18:59:48.750 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T18:59:48.750 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T18:59:48.750 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:48.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.750 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.750 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 4, 2024-09-20T18:59:48.750 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "3d62654d-d31d-4310-9934-49e694dd17d9", 2024-09-20T18:59:48.750 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T18:59:48.750 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T18:59:48.750 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T18:59:48.750 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T18:59:48.751 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T18:59:48.751 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T18:59:48.751 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 31, 2024-09-20T18:59:48.751 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 260, 2024-09-20T18:59:48.751 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T18:59:48.751 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T18:59:48.751 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T18:59:48.751 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:59:48.751 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.751 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:59:48.751 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6800", 2024-09-20T18:59:48.751 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2096684065 2024-09-20T18:59:48.751 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:48.751 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:48.752 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.752 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T18:59:48.752 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:59:48.752 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.752 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:59:48.752 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6801", 2024-09-20T18:59:48.752 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2096684065 2024-09-20T18:59:48.752 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:48.752 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:48.752 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.753 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T18:59:48.753 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:59:48.753 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.753 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:59:48.753 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6803", 2024-09-20T18:59:48.753 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2096684065 2024-09-20T18:59:48.753 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:48.753 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:48.753 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.753 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T18:59:48.753 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:59:48.753 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.753 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:59:48.753 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6802", 2024-09-20T18:59:48.753 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2096684065 2024-09-20T18:59:48.754 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:48.754 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:48.754 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.754 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.142:6800/2096684065", 2024-09-20T18:59:48.754 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.142:6801/2096684065", 2024-09-20T18:59:48.754 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.142:6803/2096684065", 2024-09-20T18:59:48.754 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.142:6802/2096684065", 2024-09-20T18:59:48.754 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T18:59:48.754 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T18:59:48.754 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T18:59:48.754 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:48.754 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.754 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.755 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 5, 2024-09-20T18:59:48.755 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "b3e4e21b-639f-4d25-8bbf-ea8922a9222a", 2024-09-20T18:59:48.755 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T18:59:48.755 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T18:59:48.755 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T18:59:48.755 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T18:59:48.755 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T18:59:48.755 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T18:59:48.755 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 38, 2024-09-20T18:59:48.755 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 260, 2024-09-20T18:59:48.755 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T18:59:48.755 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T18:59:48.755 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T18:59:48.755 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:59:48.756 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.756 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:59:48.756 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6804", 2024-09-20T18:59:48.756 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2500466067 2024-09-20T18:59:48.756 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:48.756 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:48.756 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.756 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T18:59:48.756 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:59:48.756 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.756 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:59:48.756 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6805", 2024-09-20T18:59:48.756 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2500466067 2024-09-20T18:59:48.756 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:48.756 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:48.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.757 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T18:59:48.757 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:59:48.757 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.757 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:59:48.757 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6807", 2024-09-20T18:59:48.757 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2500466067 2024-09-20T18:59:48.757 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:48.757 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:48.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.757 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T18:59:48.757 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:59:48.757 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.758 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:59:48.758 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6806", 2024-09-20T18:59:48.758 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2500466067 2024-09-20T18:59:48.758 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:48.758 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:48.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.758 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.142:6804/2500466067", 2024-09-20T18:59:48.758 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.142:6805/2500466067", 2024-09-20T18:59:48.758 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.142:6807/2500466067", 2024-09-20T18:59:48.758 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.142:6806/2500466067", 2024-09-20T18:59:48.758 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T18:59:48.758 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T18:59:48.758 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T18:59:48.758 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:48.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.759 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.759 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 6, 2024-09-20T18:59:48.759 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "c5db7107-ccb4-4ae2-9e63-f444897942e3", 2024-09-20T18:59:48.759 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T18:59:48.759 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T18:59:48.759 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T18:59:48.759 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T18:59:48.759 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T18:59:48.759 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T18:59:48.759 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 45, 2024-09-20T18:59:48.759 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 260, 2024-09-20T18:59:48.759 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T18:59:48.759 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T18:59:48.760 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T18:59:48.760 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:59:48.760 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.760 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:59:48.760 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6808", 2024-09-20T18:59:48.760 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1688186226 2024-09-20T18:59:48.760 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:48.760 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:48.760 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.760 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T18:59:48.760 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:59:48.760 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.760 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:59:48.760 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6809", 2024-09-20T18:59:48.761 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1688186226 2024-09-20T18:59:48.761 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:48.761 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:48.761 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.761 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T18:59:48.761 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:59:48.761 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.761 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:59:48.761 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6811", 2024-09-20T18:59:48.761 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1688186226 2024-09-20T18:59:48.761 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:48.761 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:48.761 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.761 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T18:59:48.762 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:59:48.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.762 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:59:48.762 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6810", 2024-09-20T18:59:48.762 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1688186226 2024-09-20T18:59:48.762 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:48.762 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:48.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.762 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.142:6808/1688186226", 2024-09-20T18:59:48.762 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.142:6809/1688186226", 2024-09-20T18:59:48.763 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.142:6811/1688186226", 2024-09-20T18:59:48.763 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.142:6810/1688186226", 2024-09-20T18:59:48.763 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T18:59:48.763 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T18:59:48.763 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T18:59:48.763 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:48.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.763 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.763 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 7, 2024-09-20T18:59:48.763 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "07d16806-e2c8-4acb-b284-d75701e61622", 2024-09-20T18:59:48.763 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T18:59:48.763 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T18:59:48.763 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T18:59:48.763 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T18:59:48.764 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T18:59:48.764 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T18:59:48.764 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 53, 2024-09-20T18:59:48.764 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 260, 2024-09-20T18:59:48.764 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T18:59:48.764 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T18:59:48.764 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T18:59:48.764 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:59:48.764 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.764 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:59:48.764 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6812", 2024-09-20T18:59:48.764 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3679770984 2024-09-20T18:59:48.764 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:48.764 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:48.764 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.765 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T18:59:48.765 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:59:48.765 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.765 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:59:48.765 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6813", 2024-09-20T18:59:48.765 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3679770984 2024-09-20T18:59:48.765 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:48.765 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:48.765 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.765 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T18:59:48.765 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:59:48.765 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.765 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:59:48.765 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6815", 2024-09-20T18:59:48.766 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3679770984 2024-09-20T18:59:48.766 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:48.766 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:48.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.766 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T18:59:48.766 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T18:59:48.766 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.766 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T18:59:48.766 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6814", 2024-09-20T18:59:48.766 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3679770984 2024-09-20T18:59:48.766 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:48.766 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:48.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.767 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.142:6812/3679770984", 2024-09-20T18:59:48.767 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.142:6813/3679770984", 2024-09-20T18:59:48.767 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.142:6815/3679770984", 2024-09-20T18:59:48.767 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.142:6814/3679770984", 2024-09-20T18:59:48.767 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T18:59:48.767 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T18:59:48.767 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T18:59:48.767 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:48.767 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:48.767 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T18:59:48.767 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_xinfo": [ 2024-09-20T18:59:48.767 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.767 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 0, 2024-09-20T18:59:48.767 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T18:59:48.767 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T18:59:48.768 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T18:59:48.768 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T18:59:48.768 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T18:59:48.768 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T18:59:42.629168+0000", 2024-09-20T18:59:48.768 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T18:59:48.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.768 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.768 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 1, 2024-09-20T18:59:48.768 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T18:59:48.768 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T18:59:48.768 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T18:59:48.769 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T18:59:48.769 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T18:59:48.769 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T18:59:03.261985+0000", 2024-09-20T18:59:48.769 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T18:59:48.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.769 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.769 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 2, 2024-09-20T18:59:48.769 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T18:59:48.769 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T18:59:48.769 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T18:59:48.769 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T18:59:48.769 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T18:59:48.769 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T18:59:29.779151+0000", 2024-09-20T18:59:48.769 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T18:59:48.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.770 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.770 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 3, 2024-09-20T18:59:48.770 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T18:59:48.770 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T18:59:48.770 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T18:59:48.770 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T18:59:48.770 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T18:59:48.770 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T18:58:52.089701+0000", 2024-09-20T18:59:48.770 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T18:59:48.770 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.770 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.770 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 4, 2024-09-20T18:59:48.770 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T18:59:48.770 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T18:59:48.771 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T18:59:48.771 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T18:59:48.771 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T18:59:48.771 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T18:58:53.934974+0000", 2024-09-20T18:59:48.771 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T18:59:48.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.771 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 5, 2024-09-20T18:59:48.771 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T18:59:48.771 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T18:59:48.771 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T18:59:48.771 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T18:59:48.771 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T18:59:48.772 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T18:59:03.859777+0000", 2024-09-20T18:59:48.772 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T18:59:48.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.772 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.772 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 6, 2024-09-20T18:59:48.772 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T18:59:48.772 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T18:59:48.772 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T18:59:48.772 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T18:59:48.772 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T18:59:48.772 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T18:58:54.618326+0000", 2024-09-20T18:59:48.772 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T18:59:48.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.773 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.773 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 7, 2024-09-20T18:59:48.773 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T18:59:48.773 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T18:59:48.773 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T18:59:48.773 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T18:59:48.773 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T18:59:48.773 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T18:58:55.226519+0000", 2024-09-20T18:59:48.774 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T18:59:48.774 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:48.774 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T18:59:48.774 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_upmap": [], 2024-09-20T18:59:48.774 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_upmap_items": [ 2024-09-20T18:59:48.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.774 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "4.f", 2024-09-20T18:59:48.774 INFO:teuthology.orchestra.run.smithi083.stdout: "mappings": [ 2024-09-20T18:59:48.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.774 INFO:teuthology.orchestra.run.smithi083.stdout: "from": 1, 2024-09-20T18:59:48.774 INFO:teuthology.orchestra.run.smithi083.stdout: "to": 2 2024-09-20T18:59:48.774 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:48.774 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:48.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.775 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "4.14", 2024-09-20T18:59:48.775 INFO:teuthology.orchestra.run.smithi083.stdout: "mappings": [ 2024-09-20T18:59:48.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.775 INFO:teuthology.orchestra.run.smithi083.stdout: "from": 1, 2024-09-20T18:59:48.775 INFO:teuthology.orchestra.run.smithi083.stdout: "to": 5 2024-09-20T18:59:48.775 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:48.775 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:48.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.775 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "4.18", 2024-09-20T18:59:48.775 INFO:teuthology.orchestra.run.smithi083.stdout: "mappings": [ 2024-09-20T18:59:48.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.775 INFO:teuthology.orchestra.run.smithi083.stdout: "from": 1, 2024-09-20T18:59:48.776 INFO:teuthology.orchestra.run.smithi083.stdout: "to": 2 2024-09-20T18:59:48.776 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:48.776 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:48.776 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:48.776 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T18:59:48.776 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_upmap_primaries": [], 2024-09-20T18:59:48.776 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_temp": [ 2024-09-20T18:59:48.776 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.776 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "9.0", 2024-09-20T18:59:48.776 INFO:teuthology.orchestra.run.smithi083.stdout: "osds": [ 2024-09-20T18:59:48.776 INFO:teuthology.orchestra.run.smithi083.stdout: 1, 2024-09-20T18:59:48.776 INFO:teuthology.orchestra.run.smithi083.stdout: 2 2024-09-20T18:59:48.776 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:48.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.776 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.777 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "9.1", 2024-09-20T18:59:48.777 INFO:teuthology.orchestra.run.smithi083.stdout: "osds": [ 2024-09-20T18:59:48.777 INFO:teuthology.orchestra.run.smithi083.stdout: 3, 2024-09-20T18:59:48.777 INFO:teuthology.orchestra.run.smithi083.stdout: 7 2024-09-20T18:59:48.777 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:48.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.777 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T18:59:48.777 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "9.b", 2024-09-20T18:59:48.777 INFO:teuthology.orchestra.run.smithi083.stdout: "osds": [ 2024-09-20T18:59:48.777 INFO:teuthology.orchestra.run.smithi083.stdout: 0, 2024-09-20T18:59:48.777 INFO:teuthology.orchestra.run.smithi083.stdout: 5 2024-09-20T18:59:48.777 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T18:59:48.777 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:48.777 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T18:59:48.777 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_temp": [], 2024-09-20T18:59:48.778 INFO:teuthology.orchestra.run.smithi083.stdout: "blocklist": { 2024-09-20T18:59:48.778 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/3474253603": "2024-09-21T18:44:32.434180+0000", 2024-09-20T18:59:48.778 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:6800/3525856829": "2024-09-21T18:44:32.434180+0000", 2024-09-20T18:59:48.778 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/220746284": "2024-09-21T18:44:32.434180+0000", 2024-09-20T18:59:48.778 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/3875445393": "2024-09-21T18:44:56.985620+0000", 2024-09-20T18:59:48.778 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/3009092807": "2024-09-21T18:44:56.985620+0000", 2024-09-20T18:59:48.778 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/4267132131": "2024-09-21T18:44:56.985620+0000", 2024-09-20T18:59:48.778 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/3684382972": "2024-09-21T18:50:14.729526+0000", 2024-09-20T18:59:48.778 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/3132933178": "2024-09-21T18:50:14.729526+0000", 2024-09-20T18:59:48.778 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:6800/1007846146": "2024-09-21T18:50:14.729526+0000", 2024-09-20T18:59:48.778 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:6800/2985596191": "2024-09-21T18:44:56.985620+0000", 2024-09-20T18:59:48.778 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/2410768464": "2024-09-21T18:50:14.729526+0000", 2024-09-20T18:59:48.778 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/1868694097": "2024-09-21T18:50:14.729526+0000", 2024-09-20T18:59:48.779 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/3777331122": "2024-09-21T18:44:32.434180+0000", 2024-09-20T18:59:48.779 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/2161728215": "2024-09-21T18:50:14.729526+0000" 2024-09-20T18:59:48.779 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.779 INFO:teuthology.orchestra.run.smithi083.stdout: "range_blocklist": {}, 2024-09-20T18:59:48.779 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profiles": { 2024-09-20T18:59:48.779 INFO:teuthology.orchestra.run.smithi083.stdout: "default": { 2024-09-20T18:59:48.779 INFO:teuthology.orchestra.run.smithi083.stdout: "crush-failure-domain": "osd", 2024-09-20T18:59:48.779 INFO:teuthology.orchestra.run.smithi083.stdout: "k": "2", 2024-09-20T18:59:48.779 INFO:teuthology.orchestra.run.smithi083.stdout: "m": "1", 2024-09-20T18:59:48.779 INFO:teuthology.orchestra.run.smithi083.stdout: "plugin": "jerasure", 2024-09-20T18:59:48.779 INFO:teuthology.orchestra.run.smithi083.stdout: "technique": "reed_sol_van" 2024-09-20T18:59:48.779 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:48.779 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T18:59:48.779 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps_queue": [], 2024-09-20T18:59:48.779 INFO:teuthology.orchestra.run.smithi083.stdout: "new_removed_snaps": [], 2024-09-20T18:59:48.780 INFO:teuthology.orchestra.run.smithi083.stdout: "new_purged_snaps": [], 2024-09-20T18:59:48.780 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_node_flags": {}, 2024-09-20T18:59:48.780 INFO:teuthology.orchestra.run.smithi083.stdout: "device_class_flags": {}, 2024-09-20T18:59:48.780 INFO:teuthology.orchestra.run.smithi083.stdout: "stretch_mode": { 2024-09-20T18:59:48.780 INFO:teuthology.orchestra.run.smithi083.stdout: "stretch_mode_enabled": false, 2024-09-20T18:59:48.780 INFO:teuthology.orchestra.run.smithi083.stdout: "stretch_bucket_count": 0, 2024-09-20T18:59:48.780 INFO:teuthology.orchestra.run.smithi083.stdout: "degraded_stretch_mode": 0, 2024-09-20T18:59:48.780 INFO:teuthology.orchestra.run.smithi083.stdout: "recovering_stretch_mode": 0, 2024-09-20T18:59:48.780 INFO:teuthology.orchestra.run.smithi083.stdout: "stretch_mode_bucket": 0 2024-09-20T18:59:48.780 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T18:59:48.780 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T18:59:48.813 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:48.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:48.815 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:59:48.916 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:48.952 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:48.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:48.954 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:49.055 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:49.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:49.087 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:49.087 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:49.188 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:59:49.224 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:49.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:49.225 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:59:49.256 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 69 16 94260 94244 85.3523 63.5625 0.00859341 0.0117112 2024-09-20T18:59:49.326 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:49.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:49.367 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:49.367 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:49.468 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:49.502 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:49.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:49.503 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:49.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:49 smithi142 ceph-mon[32508]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:59:49.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:49 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T18:59:49.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:49 smithi142 ceph-mon[32508]: osdmap e272: 8 total, 8 up, 8 in 2024-09-20T18:59:49.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:49 smithi142 ceph-mon[32508]: 6.e scrub starts 2024-09-20T18:59:49.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:49 smithi142 ceph-mon[32508]: 6.e scrub ok 2024-09-20T18:59:49.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:49 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1381037776' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-20T18:59:49.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:49 smithi083 ceph-mon[29421]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:59:49.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:49 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T18:59:49.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:49 smithi083 ceph-mon[29421]: osdmap e272: 8 total, 8 up, 8 in 2024-09-20T18:59:49.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:49 smithi083 ceph-mon[29421]: 6.e scrub starts 2024-09-20T18:59:49.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:49 smithi083 ceph-mon[29421]: 6.e scrub ok 2024-09-20T18:59:49.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:49 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1381037776' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-20T18:59:49.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:49 smithi083 ceph-mon[36260]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T18:59:49.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:49 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T18:59:49.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:49 smithi083 ceph-mon[36260]: osdmap e272: 8 total, 8 up, 8 in 2024-09-20T18:59:49.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:49 smithi083 ceph-mon[36260]: 6.e scrub starts 2024-09-20T18:59:49.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:49 smithi083 ceph-mon[36260]: 6.e scrub ok 2024-09-20T18:59:49.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:49 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1381037776' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-20T18:59:49.605 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:49.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:49.638 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:49.638 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:49.663 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_blocked_ops 2024-09-20T18:59:49.740 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:49.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:49.776 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:49.776 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:49.877 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:59:49.908 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:49.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:49.909 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:59:49.942 INFO:tasks.thrashosds.thrasher:j is {'epoch': 272, 'fsid': '1ed5a50a-7780-11ef-baf5-efdc52797490', 'created': '2024-09-20T18:44:10.399800+0000', 'modified': '2024-09-20T18:59:48.061224+0000', 'last_up_change': '2024-09-20T18:49:34.836377+0000', 'last_in_change': '2024-09-20T18:59:04.044029+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 18, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 9, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-20T18:47:15.285795+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '254', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'datapool', 'create_time': '2024-09-20T18:49:43.452984+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 61, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.6500000953674316, 'score_stable': 2.6500000953674316, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 2.3299999237060547, 'raw_score_stable': 2.3299999237060547, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': '.rgw.root', 'create_time': '2024-09-20T18:49:44.217011+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-20T18:49:46.007275+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-20T18:49:47.961258+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-20T18:49:50.011276+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'unique_pool_2', 'create_time': '2024-09-20T18:58:32.024544+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '250', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': 'c714e1ec-f0c3-4054-9792-61b84e53eaa8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 261, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6801', 'nonce': 2574084049}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6802', 'nonce': 2574084049}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6804', 'nonce': 2574084049}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6803', 'nonce': 2574084049}]}, 'public_addr': '172.21.15.83:6801/2574084049', 'cluster_addr': '172.21.15.83:6802/2574084049', 'heartbeat_back_addr': '172.21.15.83:6804/2574084049', 'heartbeat_front_addr': '172.21.15.83:6803/2574084049', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '9dc0f2a8-bafa-4a42-88f5-0301dd58f78d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 261, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6805', 'nonce': 4213285798}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6806', 'nonce': 4213285798}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6808', 'nonce': 4213285798}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6807', 'nonce': 4213285798}]}, 'public_addr': '172.21.15.83:6805/4213285798', 'cluster_addr': '172.21.15.83:6806/4213285798', 'heartbeat_back_addr': '172.21.15.83:6808/4213285798', 'heartbeat_front_addr': '172.21.15.83:6807/4213285798', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '448ab692-ec41-44dc-b2d5-dbefd255b3fd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 83, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6809', 'nonce': 805947571}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6810', 'nonce': 805947571}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6812', 'nonce': 805947571}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6811', 'nonce': 805947571}]}, 'public_addr': '172.21.15.83:6809/805947571', 'cluster_addr': '172.21.15.83:6810/805947571', 'heartbeat_back_addr': '172.21.15.83:6812/805947571', 'heartbeat_front_addr': '172.21.15.83:6811/805947571', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '7cda5e50-e22e-4d4a-823b-53c8f5061193', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 25, 'up_thru': 261, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6813', 'nonce': 987237318}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6814', 'nonce': 987237318}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6816', 'nonce': 987237318}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6815', 'nonce': 987237318}]}, 'public_addr': '172.21.15.83:6813/987237318', 'cluster_addr': '172.21.15.83:6814/987237318', 'heartbeat_back_addr': '172.21.15.83:6816/987237318', 'heartbeat_front_addr': '172.21.15.83:6815/987237318', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '3d62654d-d31d-4310-9934-49e694dd17d9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 31, 'up_thru': 260, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6800', 'nonce': 2096684065}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6801', 'nonce': 2096684065}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6803', 'nonce': 2096684065}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6802', 'nonce': 2096684065}]}, 'public_addr': '172.21.15.142:6800/2096684065', 'cluster_addr': '172.21.15.142:6801/2096684065', 'heartbeat_back_addr': '172.21.15.142:6803/2096684065', 'heartbeat_front_addr': '172.21.15.142:6802/2096684065', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'b3e4e21b-639f-4d25-8bbf-ea8922a9222a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 38, 'up_thru': 260, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6804', 'nonce': 2500466067}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6805', 'nonce': 2500466067}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6807', 'nonce': 2500466067}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6806', 'nonce': 2500466067}]}, 'public_addr': '172.21.15.142:6804/2500466067', 'cluster_addr': '172.21.15.142:6805/2500466067', 'heartbeat_back_addr': '172.21.15.142:6807/2500466067', 'heartbeat_front_addr': '172.21.15.142:6806/2500466067', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'c5db7107-ccb4-4ae2-9e63-f444897942e3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 45, 'up_thru': 260, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6808', 'nonce': 1688186226}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6809', 'nonce': 1688186226}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6811', 'nonce': 1688186226}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6810', 'nonce': 1688186226}]}, 'public_addr': '172.21.15.142:6808/1688186226', 'cluster_addr': '172.21.15.142:6809/1688186226', 'heartbeat_back_addr': '172.21.15.142:6811/1688186226', 'heartbeat_front_addr': '172.21.15.142:6810/1688186226', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '07d16806-e2c8-4acb-b284-d75701e61622', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 53, 'up_thru': 260, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6812', 'nonce': 3679770984}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6813', 'nonce': 3679770984}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6815', 'nonce': 3679770984}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6814', 'nonce': 3679770984}]}, 'public_addr': '172.21.15.142:6812/3679770984', 'cluster_addr': '172.21.15.142:6813/3679770984', 'heartbeat_back_addr': '172.21.15.142:6815/3679770984', 'heartbeat_front_addr': '172.21.15.142:6814/3679770984', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T18:59:42.629168+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T18:59:03.261985+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T18:59:29.779151+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T18:58:52.089701+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T18:58:53.934974+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T18:59:03.859777+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T18:58:54.618326+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T18:58:55.226519+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '4.f', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.14', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.18', 'mappings': [{'from': 1, 'to': 2}]}], 'pg_upmap_primaries': [], 'pg_temp': [{'pgid': '9.0', 'osds': [1, 2]}, {'pgid': '9.1', 'osds': [3, 7]}, {'pgid': '9.b', 'osds': [0, 5]}], 'primary_temp': [], 'blocklist': {'172.21.15.83:0/3474253603': '2024-09-21T18:44:32.434180+0000', '172.21.15.83:6800/3525856829': '2024-09-21T18:44:32.434180+0000', '172.21.15.83:0/220746284': '2024-09-21T18:44:32.434180+0000', '172.21.15.83:0/3875445393': '2024-09-21T18:44:56.985620+0000', '172.21.15.83:0/3009092807': '2024-09-21T18:44:56.985620+0000', '172.21.15.83:0/4267132131': '2024-09-21T18:44:56.985620+0000', '172.21.15.83:0/3684382972': '2024-09-21T18:50:14.729526+0000', '172.21.15.83:0/3132933178': '2024-09-21T18:50:14.729526+0000', '172.21.15.83:6800/1007846146': '2024-09-21T18:50:14.729526+0000', '172.21.15.83:6800/2985596191': '2024-09-21T18:44:56.985620+0000', '172.21.15.83:0/2410768464': '2024-09-21T18:50:14.729526+0000', '172.21.15.83:0/1868694097': '2024-09-21T18:50:14.729526+0000', '172.21.15.83:0/3777331122': '2024-09-21T18:44:32.434180+0000', '172.21.15.83:0/2161728215': '2024-09-21T18:50:14.729526+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-20T18:59:49.943 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T18:59:50.010 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:59:50.074 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:59:50.077 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:59:50.082 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:50.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:50.083 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:59:50.184 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:50.205 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:50 smithi083 ceph-mon[29421]: pgmap v488: 148 pgs: 1 active+recovering+undersized+remapped, 2 active+recovering+forced_recovery+undersized+remapped, 145 active+clean; 5.6 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 824 B/s rd, 48 MiB/s wr, 773 op/s; 4726/275667 objects misplaced (1.714%); 7.5 MiB/s, 120 objects/s recovering 2024-09-20T18:59:50.206 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:50 smithi083 ceph-mon[36260]: pgmap v488: 148 pgs: 1 active+recovering+undersized+remapped, 2 active+recovering+forced_recovery+undersized+remapped, 145 active+clean; 5.6 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 824 B/s rd, 48 MiB/s wr, 773 op/s; 4726/275667 objects misplaced (1.714%); 7.5 MiB/s, 120 objects/s recovering 2024-09-20T18:59:50.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:50.236 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:50.236 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:50.256 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 70 16 95200 95184 84.972 58.75 0.00892989 0.0117627 2024-09-20T18:59:50.336 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:50.366 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:50 smithi142 ceph-mon[32508]: pgmap v488: 148 pgs: 1 active+recovering+undersized+remapped, 2 active+recovering+forced_recovery+undersized+remapped, 145 active+clean; 5.6 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 824 B/s rd, 48 MiB/s wr, 773 op/s; 4726/275667 objects misplaced (1.714%); 7.5 MiB/s, 120 objects/s recovering 2024-09-20T18:59:50.386 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:50.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:50.387 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:50.487 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:50.519 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:50.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:50.520 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:50.620 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:59:50.621 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T18:59:50.622 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:59:50.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:50.656 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:50.656 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:59:50.756 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:59:50.803 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:50.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:50.804 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:59:50.905 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:50.936 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:50.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:50.937 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:51.038 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:51.046 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 18:59:50 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:18:59:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T18:59:51.075 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:51.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:51.076 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:51.177 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:51.212 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:51.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:51.213 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:51.256 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 71 15 95985 95970 84.4671 49.125 0.0164428 0.011834 2024-09-20T18:59:51.314 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:59:51.348 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:51.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:51.349 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:59:51.385 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:51 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:59:51.385 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:51 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:59:51.385 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:51 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:59:51.385 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:51 smithi142 ceph-mon[32508]: osdmap e273: 8 total, 8 up, 8 in 2024-09-20T18:59:51.386 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_historic_ops 2024-09-20T18:59:51.449 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:51.485 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:51.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:51.486 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:51.526 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:51 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:59:51.526 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:51 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:59:51.526 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:51 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:59:51.526 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:51 smithi083 ceph-mon[29421]: osdmap e273: 8 total, 8 up, 8 in 2024-09-20T18:59:51.526 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:51 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T18:59:51.526 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:51 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T18:59:51.526 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:51 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T18:59:51.526 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:51 smithi083 ceph-mon[36260]: osdmap e273: 8 total, 8 up, 8 in 2024-09-20T18:59:51.587 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:59:51.637 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:51.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:51.639 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:59:51.733 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:59:51.740 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:51.782 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:51.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:51.783 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:51.826 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:59:51.883 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:59:51.928 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:51.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:51.929 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:59:52.030 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:52.063 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:52.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:52.064 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:52.165 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:59:52.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:52.199 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:52.199 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:59:52.257 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 72 16 96987 96971 84.1628 62.5625 0.0129637 0.0118771 2024-09-20T18:59:52.266 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:52 smithi142 ceph-mon[32508]: pgmap v490: 148 pgs: 1 active+recovering+undersized+remapped, 2 active+recovering+forced_recovery+undersized+remapped, 145 active+clean; 5.6 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.7 KiB/s rd, 59 MiB/s wr, 945 op/s; 4726/278274 objects misplaced (1.698%); 7.8 MiB/s, 124 objects/s recovering 2024-09-20T18:59:52.300 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:52.339 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:52.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:52.341 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:52.441 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:52.453 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T18:59:52.453 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T18:59:52.483 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:52.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:52.485 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:52.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:52 smithi083 ceph-mon[29421]: pgmap v490: 148 pgs: 1 active+recovering+undersized+remapped, 2 active+recovering+forced_recovery+undersized+remapped, 145 active+clean; 5.6 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.7 KiB/s rd, 59 MiB/s wr, 945 op/s; 4726/278274 objects misplaced (1.698%); 7.8 MiB/s, 124 objects/s recovering 2024-09-20T18:59:52.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:52 smithi083 ceph-mon[36260]: pgmap v490: 148 pgs: 1 active+recovering+undersized+remapped, 2 active+recovering+forced_recovery+undersized+remapped, 145 active+clean; 5.6 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.7 KiB/s rd, 59 MiB/s wr, 945 op/s; 4726/278274 objects misplaced (1.698%); 7.8 MiB/s, 124 objects/s recovering 2024-09-20T18:59:52.585 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:52.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:52.618 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:52.618 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:52.719 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:52.752 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:52.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:52.753 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:52.854 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:59:52.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:52.888 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:52.888 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:59:52.989 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:53.023 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:53.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:53.024 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:53.125 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:59:53.164 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:53.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:53.165 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:59:53.200 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-20T18:59:53.256 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 73 16 98072 98056 83.9388 67.8125 0.00415623 0.0119046 2024-09-20T18:59:53.266 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:59:53.300 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:53.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:53.301 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:59:53.403 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:53.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:53.475 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:53.475 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:53.576 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:53.635 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:53.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:53.636 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:53.639 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T18:59:53.653 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:59:53.737 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:53.764 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:53.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:53.765 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:53.866 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:59:53.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:53.952 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:53.953 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:59:54.054 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:59:54.085 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:54.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:54.086 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:59:54.187 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:54.225 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:54.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:54.226 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:54.257 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 74 16 98963 98947 83.557 55.6875 0.0126284 0.0119636 2024-09-20T18:59:54.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:54 smithi083 ceph-mon[29421]: pgmap v491: 148 pgs: 1 active+recovering+undersized+remapped, 2 active+recovering+forced_recovery+undersized+remapped, 145 active+clean; 5.9 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.4 KiB/s rd, 81 MiB/s wr, 1.29k op/s; 3943/289338 objects misplaced (1.363%); 13 MiB/s, 211 objects/s recovering 2024-09-20T18:59:54.297 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:54 smithi083 ceph-mon[36260]: pgmap v491: 148 pgs: 1 active+recovering+undersized+remapped, 2 active+recovering+forced_recovery+undersized+remapped, 145 active+clean; 5.9 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.4 KiB/s rd, 81 MiB/s wr, 1.29k op/s; 3943/289338 objects misplaced (1.363%); 13 MiB/s, 211 objects/s recovering 2024-09-20T18:59:54.312 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T18:59:54.314 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-20T18:59:54.327 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:59:54.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:54.356 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:54.356 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:59:54.458 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:54.489 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:54.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:54.490 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:54.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:54 smithi142 ceph-mon[32508]: pgmap v491: 148 pgs: 1 active+recovering+undersized+remapped, 2 active+recovering+forced_recovery+undersized+remapped, 145 active+clean; 5.9 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.4 KiB/s rd, 81 MiB/s wr, 1.29k op/s; 3943/289338 objects misplaced (1.363%); 13 MiB/s, 211 objects/s recovering 2024-09-20T18:59:54.590 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:59:54.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:54.614 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:54.614 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:59:54.716 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:59:54.740 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:54.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:54.741 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:59:54.842 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:54.881 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:54.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:54.882 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:54.983 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:55.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:55.024 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:55.024 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:55.125 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:59:55.156 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:55.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:55.158 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:59:55.257 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 75 16 99814 99798 83.152 53.1875 0.00850388 0.0120109 2024-09-20T18:59:55.258 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:55.279 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:55.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:55.280 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:55.381 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:55.392 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:55 smithi083 ceph-mon[29421]: from='client.15666 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:59:55.392 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:55 smithi083 ceph-mon[36260]: from='client.15666 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:59:55.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:55.417 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:55.417 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:55.453 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T18:59:55.454 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T18:59:55.454 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T18:59:55.454 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T18:59:55.454 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T18:59:55.454 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T18:59:55.455 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T18:59:55.455 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T18:59:55.455 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T18:59:55.455 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T18:59:55.455 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T18:59:55.455 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T18:59:55.457 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T18:59:55.458 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T18:59:55.468 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T18:59:55.468 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T18:59:55.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:55 smithi142 ceph-mon[32508]: from='client.15666 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T18:59:55.517 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:59:55.540 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:55.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:55.541 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:59:55.642 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:59:55.679 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:55.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:55.680 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:59:55.781 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:55.819 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:55.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:55.820 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:55.921 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:55.961 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:55.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:55.962 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:56.063 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:56.084 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:56.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:56.084 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:56.185 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:56.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:56.221 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:56.221 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:56.257 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 76 16 100675 100659 82.7659 53.8125 0.0227083 0.0120771 2024-09-20T18:59:56.322 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:59:56.358 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:56.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:56.359 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:59:56.459 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:59:56.500 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:56.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:56.501 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:59:56.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:56 smithi142 ceph-mon[32508]: pgmap v492: 148 pgs: 1 active+recovering+undersized+remapped, 2 active+recovering+forced_recovery+undersized+remapped, 145 active+clean; 5.9 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 639 B/s rd, 43 MiB/s wr, 683 op/s; 3943/289338 objects misplaced (1.363%); 8.4 MiB/s, 134 objects/s recovering 2024-09-20T18:59:56.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:56 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T18:59:56.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:56 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T18:59:56.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:56 smithi142 ceph-mon[32508]: osdmap e274: 8 total, 8 up, 8 in 2024-09-20T18:59:56.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:56 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1653715797' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:59:56.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:56 smithi142 ceph-mon[32508]: 4.9 scrub starts 2024-09-20T18:59:56.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:56 smithi142 ceph-mon[32508]: 4.9 scrub ok 2024-09-20T18:59:56.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:56 smithi142 ceph-mon[32508]: osdmap e275: 8 total, 8 up, 8 in 2024-09-20T18:59:56.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:56 smithi083 ceph-mon[29421]: pgmap v492: 148 pgs: 1 active+recovering+undersized+remapped, 2 active+recovering+forced_recovery+undersized+remapped, 145 active+clean; 5.9 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 639 B/s rd, 43 MiB/s wr, 683 op/s; 3943/289338 objects misplaced (1.363%); 8.4 MiB/s, 134 objects/s recovering 2024-09-20T18:59:56.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:56 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T18:59:56.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:56 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T18:59:56.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:56 smithi083 ceph-mon[29421]: osdmap e274: 8 total, 8 up, 8 in 2024-09-20T18:59:56.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:56 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1653715797' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:59:56.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:56 smithi083 ceph-mon[29421]: 4.9 scrub starts 2024-09-20T18:59:56.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:56 smithi083 ceph-mon[29421]: 4.9 scrub ok 2024-09-20T18:59:56.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:56 smithi083 ceph-mon[29421]: osdmap e275: 8 total, 8 up, 8 in 2024-09-20T18:59:56.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:56 smithi083 ceph-mon[36260]: pgmap v492: 148 pgs: 1 active+recovering+undersized+remapped, 2 active+recovering+forced_recovery+undersized+remapped, 145 active+clean; 5.9 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 639 B/s rd, 43 MiB/s wr, 683 op/s; 3943/289338 objects misplaced (1.363%); 8.4 MiB/s, 134 objects/s recovering 2024-09-20T18:59:56.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:56 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T18:59:56.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:56 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T18:59:56.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:56 smithi083 ceph-mon[36260]: osdmap e274: 8 total, 8 up, 8 in 2024-09-20T18:59:56.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:56 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1653715797' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T18:59:56.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:56 smithi083 ceph-mon[36260]: 4.9 scrub starts 2024-09-20T18:59:56.548 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:56 smithi083 ceph-mon[36260]: 4.9 scrub ok 2024-09-20T18:59:56.548 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:56 smithi083 ceph-mon[36260]: osdmap e275: 8 total, 8 up, 8 in 2024-09-20T18:59:56.602 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:56.643 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:56.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:56.644 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:56.744 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:59:56.772 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:56.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:56.773 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:59:56.874 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:56.912 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:56.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:56.913 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:57.013 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:57.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:57.056 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:57.056 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:57.157 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:59:57.195 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:57.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:57.196 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:59:57.257 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 77 16 101628 101612 82.4646 59.5625 0.00888824 0.0121139 2024-09-20T18:59:57.266 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 18:59:56 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T18:59:57.298 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:59:57.333 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:57.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:57.334 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:59:57.435 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:57.473 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:57.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:57.474 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:57.575 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:57.597 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:57.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:57.598 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:57.627 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:59:57.699 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:57.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:57.731 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:57.731 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:57.832 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:57.871 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:57.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:57.872 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:57.973 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:58.010 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:58.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:58.011 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:58.087 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":491,"stamp":"2024-09-20T18:59:53.011513+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6308239899,"num_objects":96446,"num_object_clones":0,"num_object_copies":289338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3943,"num_objects_unfound":0,"num_objects_dirty":96446,"num_whiteouts":0,"num_read":1444,"num_read_kb":1301,"num_write":96796,"num_write_kb":6161481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5710,"num_bytes_recovered":365892274,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82807,"ondisk_log_size":82807,"up":444,"acting":441,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":451,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":21892136,"kb_used_data":18996576,"kb_used_omap":12,"kb_used_meta":2895411,"kb_avail":728069080,"statfs":{"total":767960285184,"available":745542737920,"internally_reserved":0,"allocated":19452493824,"data_stored":19446981286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":2964901424},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":613089280,"num_objects":9355,"num_object_clones":0,"num_object_copies":28065,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-1531,"num_objects_unfound":0,"num_objects_dirty":9355,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":9355,"num_write_kb":598720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1531,"num_bytes_recovered":100335616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.245700"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":466,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.792854+0000","last_change":"2024-09-20T18:55:28.037494+0000","last_active":"2024-09-20T18:59:50.792854+0000","last_peered":"2024-09-20T18:59:50.792854+0000","last_clean":"2024-09-20T18:59:50.792854+0000","last_became_active":"2024-09-20T18:49:45.943201+0000","last_became_peered":"2024-09-20T18:49:45.943201+0000","last_unstale":"2024-09-20T18:59:50.792854+0000","last_undegraded":"2024-09-20T18:59:50.792854+0000","last_fullsized":"2024-09-20T18:59:50.792854+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:28.037449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:28.037449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":530,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.766763+0000","last_change":"2024-09-20T18:59:05.094771+0000","last_active":"2024-09-20T18:59:50.766763+0000","last_peered":"2024-09-20T18:59:50.766763+0000","last_clean":"2024-09-20T18:59:50.766763+0000","last_became_active":"2024-09-20T18:59:05.076159+0000","last_became_peered":"2024-09-20T18:59:05.076159+0000","last_unstale":"2024-09-20T18:59:50.766763+0000","last_undegraded":"2024-09-20T18:59:50.766763+0000","last_fullsized":"2024-09-20T18:59:50.766763+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:50.231620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:50.231620+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:25.223399+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":465,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.790003+0000","last_change":"2024-09-20T18:57:49.016078+0000","last_active":"2024-09-20T18:59:50.790003+0000","last_peered":"2024-09-20T18:59:50.790003+0000","last_clean":"2024-09-20T18:59:50.790003+0000","last_became_active":"2024-09-20T18:49:49.959514+0000","last_became_peered":"2024-09-20T18:49:49.959514+0000","last_unstale":"2024-09-20T18:59:50.790003+0000","last_undegraded":"2024-09-20T18:59:50.790003+0000","last_fullsized":"2024-09-20T18:59:50.790003+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:49.016006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:49.016006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":462,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.766868+0000","last_change":"2024-09-20T18:58:02.084435+0000","last_active":"2024-09-20T18:59:50.766868+0000","last_peered":"2024-09-20T18:59:50.766868+0000","last_clean":"2024-09-20T18:59:50.766868+0000","last_became_active":"2024-09-20T18:49:51.969421+0000","last_became_peered":"2024-09-20T18:49:51.969421+0000","last_unstale":"2024-09-20T18:59:50.766868+0000","last_undegraded":"2024-09-20T18:59:50.766868+0000","last_fullsized":"2024-09-20T18:59:50.766868+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:02.084375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:02.084375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:43.288290+0000","scrub_duration":197,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":467,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.807675+0000","last_change":"2024-09-20T18:59:05.081431+0000","last_active":"2024-09-20T18:59:50.807675+0000","last_peered":"2024-09-20T18:59:50.807675+0000","last_clean":"2024-09-20T18:59:50.807675+0000","last_became_active":"2024-09-20T18:59:05.081081+0000","last_became_peered":"2024-09-20T18:59:05.081081+0000","last_unstale":"2024-09-20T18:59:50.807675+0000","last_undegraded":"2024-09-20T18:59:50.807675+0000","last_fullsized":"2024-09-20T18:59:50.807675+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.216188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.216188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":480,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.802310+0000","last_change":"2024-09-20T18:59:05.073093+0000","last_active":"2024-09-20T18:59:50.802310+0000","last_peered":"2024-09-20T18:59:50.802310+0000","last_clean":"2024-09-20T18:59:50.802310+0000","last_became_active":"2024-09-20T18:59:05.072675+0000","last_became_peered":"2024-09-20T18:59:05.072675+0000","last_unstale":"2024-09-20T18:59:50.802310+0000","last_undegraded":"2024-09-20T18:59:50.802310+0000","last_fullsized":"2024-09-20T18:59:50.802310+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:03.055929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:59:03.055929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:26.275543+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":486,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.813005+0000","last_change":"2024-09-20T18:58:57.097877+0000","last_active":"2024-09-20T18:59:50.813005+0000","last_peered":"2024-09-20T18:59:50.813005+0000","last_clean":"2024-09-20T18:59:50.813005+0000","last_became_active":"2024-09-20T18:49:47.952150+0000","last_became_peered":"2024-09-20T18:49:47.952150+0000","last_unstale":"2024-09-20T18:59:50.813005+0000","last_undegraded":"2024-09-20T18:59:50.813005+0000","last_fullsized":"2024-09-20T18:59:50.813005+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:58:57.097766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:57.097766+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:06.978089+0000","scrub_duration":38,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":473,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.766737+0000","last_change":"2024-09-20T18:59:05.077076+0000","last_active":"2024-09-20T18:59:50.766737+0000","last_peered":"2024-09-20T18:59:50.766737+0000","last_clean":"2024-09-20T18:59:50.766737+0000","last_became_active":"2024-09-20T18:59:05.076607+0000","last_became_peered":"2024-09-20T18:59:05.076607+0000","last_unstale":"2024-09-20T18:59:50.766737+0000","last_undegraded":"2024-09-20T18:59:50.766737+0000","last_fullsized":"2024-09-20T18:59:50.766737+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:06.846018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:06.846018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:45.340583+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"59'1","reported_seq":509,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.800172+0000","last_change":"2024-09-20T18:59:05.111316+0000","last_active":"2024-09-20T18:59:50.800172+0000","last_peered":"2024-09-20T18:59:50.800172+0000","last_clean":"2024-09-20T18:59:50.800172+0000","last_became_active":"2024-09-20T18:59:05.090056+0000","last_became_peered":"2024-09-20T18:59:05.090056+0000","last_unstale":"2024-09-20T18:59:50.800172+0000","last_undegraded":"2024-09-20T18:59:50.800172+0000","last_fullsized":"2024-09-20T18:59:50.800172+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:58:09.472360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:09.472360+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":487,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.767222+0000","last_change":"2024-09-20T18:58:29.671959+0000","last_active":"2024-09-20T18:59:50.767222+0000","last_peered":"2024-09-20T18:59:50.767222+0000","last_clean":"2024-09-20T18:59:50.767222+0000","last_became_active":"2024-09-20T18:49:47.947448+0000","last_became_peered":"2024-09-20T18:49:47.947448+0000","last_unstale":"2024-09-20T18:59:50.767222+0000","last_undegraded":"2024-09-20T18:59:50.767222+0000","last_fullsized":"2024-09-20T18:59:50.767222+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:58:29.671887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:29.671887+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:55.089337+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":465,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.799577+0000","last_change":"2024-09-20T18:58:25.505818+0000","last_active":"2024-09-20T18:59:50.799577+0000","last_peered":"2024-09-20T18:59:50.799577+0000","last_clean":"2024-09-20T18:59:50.799577+0000","last_became_active":"2024-09-20T18:49:49.961188+0000","last_became_peered":"2024-09-20T18:49:49.961188+0000","last_unstale":"2024-09-20T18:59:50.799577+0000","last_undegraded":"2024-09-20T18:59:50.799577+0000","last_fullsized":"2024-09-20T18:59:50.799577+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:25.505757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:25.505757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":45,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":438,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.791900+0000","last_change":"2024-09-20T18:58:21.861893+0000","last_active":"2024-09-20T18:59:50.791900+0000","last_peered":"2024-09-20T18:59:50.791900+0000","last_clean":"2024-09-20T18:59:50.791900+0000","last_became_active":"2024-09-20T18:49:51.966715+0000","last_became_peered":"2024-09-20T18:49:51.966715+0000","last_unstale":"2024-09-20T18:59:50.791900+0000","last_undegraded":"2024-09-20T18:59:50.791900+0000","last_fullsized":"2024-09-20T18:59:50.791900+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:21.861836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:21.861836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:51.627461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":457,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.799704+0000","last_change":"2024-09-20T18:55:15.252839+0000","last_active":"2024-09-20T18:59:50.799704+0000","last_peered":"2024-09-20T18:59:50.799704+0000","last_clean":"2024-09-20T18:59:50.799704+0000","last_became_active":"2024-09-20T18:49:45.944703+0000","last_became_peered":"2024-09-20T18:49:45.944703+0000","last_unstale":"2024-09-20T18:59:50.799704+0000","last_undegraded":"2024-09-20T18:59:50.799704+0000","last_fullsized":"2024-09-20T18:59:50.799704+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:15.252782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:15.252782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":491,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.766972+0000","last_change":"2024-09-20T18:58:33.660957+0000","last_active":"2024-09-20T18:59:50.766972+0000","last_peered":"2024-09-20T18:59:50.766972+0000","last_clean":"2024-09-20T18:59:50.766972+0000","last_became_active":"2024-09-20T18:51:23.813399+0000","last_became_peered":"2024-09-20T18:51:23.813399+0000","last_unstale":"2024-09-20T18:59:50.766972+0000","last_undegraded":"2024-09-20T18:59:50.766972+0000","last_fullsized":"2024-09-20T18:59:50.766972+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-20T18:58:33.660531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:33.660531+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":458,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.789189+0000","last_change":"2024-09-20T18:58:32.526595+0000","last_active":"2024-09-20T18:59:50.789189+0000","last_peered":"2024-09-20T18:59:50.789189+0000","last_clean":"2024-09-20T18:59:50.789189+0000","last_became_active":"2024-09-20T18:49:49.959696+0000","last_became_peered":"2024-09-20T18:49:49.959696+0000","last_unstale":"2024-09-20T18:59:50.789189+0000","last_undegraded":"2024-09-20T18:59:50.789189+0000","last_fullsized":"2024-09-20T18:59:50.789189+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.526527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.526527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":461,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.799394+0000","last_change":"2024-09-20T18:58:26.465305+0000","last_active":"2024-09-20T18:59:50.799394+0000","last_peered":"2024-09-20T18:59:50.799394+0000","last_clean":"2024-09-20T18:59:50.799394+0000","last_became_active":"2024-09-20T18:49:51.970663+0000","last_became_peered":"2024-09-20T18:49:51.970663+0000","last_unstale":"2024-09-20T18:59:50.799394+0000","last_undegraded":"2024-09-20T18:59:50.799394+0000","last_fullsized":"2024-09-20T18:59:50.799394+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:26.465264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:26.465264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:51.798999+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":469,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.767497+0000","last_change":"2024-09-20T18:59:05.079781+0000","last_active":"2024-09-20T18:59:50.767497+0000","last_peered":"2024-09-20T18:59:50.767497+0000","last_clean":"2024-09-20T18:59:50.767497+0000","last_became_active":"2024-09-20T18:59:05.079278+0000","last_became_peered":"2024-09-20T18:59:05.079278+0000","last_unstale":"2024-09-20T18:59:50.767497+0000","last_undegraded":"2024-09-20T18:59:50.767497+0000","last_fullsized":"2024-09-20T18:59:50.767497+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:00.906652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:00.906652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:33.272916+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":466,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.795139+0000","last_change":"2024-09-20T18:58:34.996936+0000","last_active":"2024-09-20T18:59:50.795139+0000","last_peered":"2024-09-20T18:59:50.795139+0000","last_clean":"2024-09-20T18:59:50.795139+0000","last_became_active":"2024-09-20T18:49:45.950480+0000","last_became_peered":"2024-09-20T18:49:45.950480+0000","last_unstale":"2024-09-20T18:59:50.795139+0000","last_undegraded":"2024-09-20T18:59:50.795139+0000","last_fullsized":"2024-09-20T18:59:50.795139+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:34.996893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:34.996893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":489,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.781934+0000","last_change":"2024-09-20T18:58:28.253192+0000","last_active":"2024-09-20T18:59:50.781934+0000","last_peered":"2024-09-20T18:59:50.781934+0000","last_clean":"2024-09-20T18:59:50.781934+0000","last_became_active":"2024-09-20T18:49:47.950939+0000","last_became_peered":"2024-09-20T18:49:47.950939+0000","last_unstale":"2024-09-20T18:59:50.781934+0000","last_undegraded":"2024-09-20T18:59:50.781934+0000","last_fullsized":"2024-09-20T18:59:50.781934+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:58:28.253141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:28.253141+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:55.772150+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":449,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.811423+0000","last_change":"2024-09-20T18:57:14.941518+0000","last_active":"2024-09-20T18:59:50.811423+0000","last_peered":"2024-09-20T18:59:50.811423+0000","last_clean":"2024-09-20T18:59:50.811423+0000","last_became_active":"2024-09-20T18:49:49.958518+0000","last_became_peered":"2024-09-20T18:49:49.958518+0000","last_unstale":"2024-09-20T18:59:50.811423+0000","last_undegraded":"2024-09-20T18:59:50.811423+0000","last_fullsized":"2024-09-20T18:59:50.811423+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:14.941458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:14.941458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":467,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.798344+0000","last_change":"2024-09-20T18:59:05.084868+0000","last_active":"2024-09-20T18:59:50.798344+0000","last_peered":"2024-09-20T18:59:50.798344+0000","last_clean":"2024-09-20T18:59:50.798344+0000","last_became_active":"2024-09-20T18:59:05.084496+0000","last_became_peered":"2024-09-20T18:59:05.084496+0000","last_unstale":"2024-09-20T18:59:50.798344+0000","last_undegraded":"2024-09-20T18:59:50.798344+0000","last_fullsized":"2024-09-20T18:59:50.798344+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:35.125451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:35.125451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"59'1","reported_seq":481,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.767162+0000","last_change":"2024-09-20T18:59:03.397298+0000","last_active":"2024-09-20T18:59:50.767162+0000","last_peered":"2024-09-20T18:59:50.767162+0000","last_clean":"2024-09-20T18:59:50.767162+0000","last_became_active":"2024-09-20T18:49:45.940100+0000","last_became_peered":"2024-09-20T18:49:45.940100+0000","last_unstale":"2024-09-20T18:59:50.767162+0000","last_undegraded":"2024-09-20T18:59:50.767162+0000","last_fullsized":"2024-09-20T18:59:50.767162+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:59:03.397225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:59:03.397225+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:17.466210+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":468,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.800181+0000","last_change":"2024-09-20T18:57:09.819080+0000","last_active":"2024-09-20T18:59:50.800181+0000","last_peered":"2024-09-20T18:59:50.800181+0000","last_clean":"2024-09-20T18:59:50.800181+0000","last_became_active":"2024-09-20T18:49:47.950909+0000","last_became_peered":"2024-09-20T18:49:47.950909+0000","last_unstale":"2024-09-20T18:59:50.800181+0000","last_undegraded":"2024-09-20T18:59:50.800181+0000","last_fullsized":"2024-09-20T18:59:50.800181+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:57:09.818999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:09.818999+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":459,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.767272+0000","last_change":"2024-09-20T18:54:44.239273+0000","last_active":"2024-09-20T18:59:50.767272+0000","last_peered":"2024-09-20T18:59:50.767272+0000","last_clean":"2024-09-20T18:59:50.767272+0000","last_became_active":"2024-09-20T18:49:49.959102+0000","last_became_peered":"2024-09-20T18:49:49.959102+0000","last_unstale":"2024-09-20T18:59:50.767272+0000","last_undegraded":"2024-09-20T18:59:50.767272+0000","last_fullsized":"2024-09-20T18:59:50.767272+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:44.239226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.239226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:18.193129+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":454,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.789994+0000","last_change":"2024-09-20T18:58:25.501434+0000","last_active":"2024-09-20T18:59:50.789994+0000","last_peered":"2024-09-20T18:59:50.789994+0000","last_clean":"2024-09-20T18:59:50.789994+0000","last_became_active":"2024-09-20T18:49:51.970058+0000","last_became_peered":"2024-09-20T18:49:51.970058+0000","last_unstale":"2024-09-20T18:59:50.789994+0000","last_undegraded":"2024-09-20T18:59:50.789994+0000","last_fullsized":"2024-09-20T18:59:50.789994+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T18:58:25.501384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:25.501384+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":458,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.790346+0000","last_change":"2024-09-20T18:57:18.032140+0000","last_active":"2024-09-20T18:59:50.790346+0000","last_peered":"2024-09-20T18:59:50.790346+0000","last_clean":"2024-09-20T18:59:50.790346+0000","last_became_active":"2024-09-20T18:49:45.933078+0000","last_became_peered":"2024-09-20T18:49:45.933078+0000","last_unstale":"2024-09-20T18:59:50.790346+0000","last_undegraded":"2024-09-20T18:59:50.790346+0000","last_fullsized":"2024-09-20T18:59:50.790346+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:57:18.032067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:18.032067+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":486,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.792349+0000","last_change":"2024-09-20T18:59:02.024076+0000","last_active":"2024-09-20T18:59:50.792349+0000","last_peered":"2024-09-20T18:59:50.792349+0000","last_clean":"2024-09-20T18:59:50.792349+0000","last_became_active":"2024-09-20T18:49:48.500809+0000","last_became_peered":"2024-09-20T18:49:48.500809+0000","last_unstale":"2024-09-20T18:59:50.792349+0000","last_undegraded":"2024-09-20T18:59:50.792349+0000","last_fullsized":"2024-09-20T18:59:50.792349+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:59:02.023996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:59:02.023996+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:06.789889+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"262'10","reported_seq":49,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.789674+0000","last_change":"2024-09-20T18:59:05.105124+0000","last_active":"2024-09-20T18:59:50.789674+0000","last_peered":"2024-09-20T18:59:50.789674+0000","last_clean":"2024-09-20T18:59:50.789674+0000","last_became_active":"2024-09-20T18:59:05.089298+0000","last_became_peered":"2024-09-20T18:59:05.089298+0000","last_unstale":"2024-09-20T18:59:50.789674+0000","last_undegraded":"2024-09-20T18:59:50.789674+0000","last_fullsized":"2024-09-20T18:59:50.789674+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'9","last_scrub_stamp":"2024-09-20T18:57:48.853302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:48.853302+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2","version":"273'5964","reported_seq":6029,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:51.874101+0000","last_change":"2024-09-20T18:58:33.827583+0000","last_active":"2024-09-20T18:59:51.874101+0000","last_peered":"2024-09-20T18:59:51.874101+0000","last_clean":"2024-09-20T18:59:51.874101+0000","last_became_active":"2024-09-20T18:58:33.827098+0000","last_became_peered":"2024-09-20T18:58:33.827098+0000","last_unstale":"2024-09-20T18:59:51.874101+0000","last_undegraded":"2024-09-20T18:59:51.874101+0000","last_fullsized":"2024-09-20T18:59:51.874101+0000","mapping_epoch":246,"log_start":"253'1200","ondisk_log_start":"253'1200","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":4764,"log_dups_size":0,"ondisk_log_size":4764,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":390856704,"num_objects":5964,"num_object_clones":0,"num_object_copies":17892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5964,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5964,"num_write_kb":381696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":472,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.782042+0000","last_change":"2024-09-20T18:55:18.581290+0000","last_active":"2024-09-20T18:59:50.782042+0000","last_peered":"2024-09-20T18:59:50.782042+0000","last_clean":"2024-09-20T18:59:50.782042+0000","last_became_active":"2024-09-20T18:49:58.992302+0000","last_became_peered":"2024-09-20T18:49:58.992302+0000","last_unstale":"2024-09-20T18:59:50.782042+0000","last_undegraded":"2024-09-20T18:59:50.782042+0000","last_fullsized":"2024-09-20T18:59:50.782042+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:55:18.581200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:18.581200+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":457,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.800405+0000","last_change":"2024-09-20T18:54:54.288199+0000","last_active":"2024-09-20T18:59:50.800405+0000","last_peered":"2024-09-20T18:59:50.800405+0000","last_clean":"2024-09-20T18:59:50.800405+0000","last_became_active":"2024-09-20T18:49:45.936104+0000","last_became_peered":"2024-09-20T18:49:45.936104+0000","last_unstale":"2024-09-20T18:59:50.800405+0000","last_undegraded":"2024-09-20T18:59:50.800405+0000","last_fullsized":"2024-09-20T18:59:50.800405+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:54.288117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:54.288117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":46,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":583,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.767597+0000","last_change":"2024-09-20T18:54:14.292660+0000","last_active":"2024-09-20T18:59:50.767597+0000","last_peered":"2024-09-20T18:59:50.767597+0000","last_clean":"2024-09-20T18:59:50.767597+0000","last_became_active":"2024-09-20T18:49:49.960697+0000","last_became_peered":"2024-09-20T18:49:49.960697+0000","last_unstale":"2024-09-20T18:59:50.767597+0000","last_undegraded":"2024-09-20T18:59:50.767597+0000","last_fullsized":"2024-09-20T18:59:50.767597+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:54:14.292610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:14.292610+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:13.283791+0000","scrub_duration":34,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":453,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.799709+0000","last_change":"2024-09-20T18:58:31.493658+0000","last_active":"2024-09-20T18:59:50.799709+0000","last_peered":"2024-09-20T18:59:50.799709+0000","last_clean":"2024-09-20T18:59:50.799709+0000","last_became_active":"2024-09-20T18:49:51.971399+0000","last_became_peered":"2024-09-20T18:49:51.971399+0000","last_unstale":"2024-09-20T18:59:50.799709+0000","last_undegraded":"2024-09-20T18:59:50.799709+0000","last_fullsized":"2024-09-20T18:59:50.799709+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:31.493595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:31.493595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:54.153071+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"273'5997","reported_seq":6061,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.912860+0000","last_change":"2024-09-20T18:58:33.820158+0000","last_active":"2024-09-20T18:59:50.912860+0000","last_peered":"2024-09-20T18:59:50.912860+0000","last_clean":"2024-09-20T18:59:50.912860+0000","last_became_active":"2024-09-20T18:58:33.819886+0000","last_became_peered":"2024-09-20T18:58:33.819886+0000","last_unstale":"2024-09-20T18:59:50.912860+0000","last_undegraded":"2024-09-20T18:59:50.912860+0000","last_fullsized":"2024-09-20T18:59:50.912860+0000","mapping_epoch":246,"log_start":"253'1100","ondisk_log_start":"253'1100","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":4897,"log_dups_size":0,"ondisk_log_size":4897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:02.637424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393019392,"num_objects":5997,"num_object_clones":0,"num_object_copies":17991,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5997,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5997,"num_write_kb":383808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":494,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.797927+0000","last_change":"2024-09-20T18:58:08.025426+0000","last_active":"2024-09-20T18:59:50.797927+0000","last_peered":"2024-09-20T18:59:50.797927+0000","last_clean":"2024-09-20T18:59:50.797927+0000","last_became_active":"2024-09-20T18:49:48.499189+0000","last_became_peered":"2024-09-20T18:49:48.499189+0000","last_unstale":"2024-09-20T18:59:50.797927+0000","last_undegraded":"2024-09-20T18:59:50.797927+0000","last_fullsized":"2024-09-20T18:59:50.797927+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-20T18:58:08.025202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:08.025202+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":457,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.798703+0000","last_change":"2024-09-20T18:58:25.159724+0000","last_active":"2024-09-20T18:59:50.798703+0000","last_peered":"2024-09-20T18:59:50.798703+0000","last_clean":"2024-09-20T18:59:50.798703+0000","last_became_active":"2024-09-20T18:49:45.940544+0000","last_became_peered":"2024-09-20T18:49:45.940544+0000","last_unstale":"2024-09-20T18:59:50.798703+0000","last_undegraded":"2024-09-20T18:59:50.798703+0000","last_fullsized":"2024-09-20T18:59:50.798703+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:25.159673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:25.159673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":459,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.766804+0000","last_change":"2024-09-20T18:54:51.229251+0000","last_active":"2024-09-20T18:59:50.766804+0000","last_peered":"2024-09-20T18:59:50.766804+0000","last_clean":"2024-09-20T18:59:50.766804+0000","last_became_active":"2024-09-20T18:49:49.963073+0000","last_became_peered":"2024-09-20T18:49:49.963073+0000","last_unstale":"2024-09-20T18:59:50.766804+0000","last_undegraded":"2024-09-20T18:59:50.766804+0000","last_fullsized":"2024-09-20T18:59:50.766804+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.229201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.229201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:21.230603+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":463,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.767076+0000","last_change":"2024-09-20T18:58:27.704583+0000","last_active":"2024-09-20T18:59:50.767076+0000","last_peered":"2024-09-20T18:59:50.767076+0000","last_clean":"2024-09-20T18:59:50.767076+0000","last_became_active":"2024-09-20T18:49:51.968793+0000","last_became_peered":"2024-09-20T18:49:51.968793+0000","last_unstale":"2024-09-20T18:59:50.767076+0000","last_undegraded":"2024-09-20T18:59:50.767076+0000","last_fullsized":"2024-09-20T18:59:50.767076+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:27.704520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:27.704520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.c","version":"273'5994","reported_seq":6059,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:51.858098+0000","last_change":"2024-09-20T18:58:33.845752+0000","last_active":"2024-09-20T18:59:51.858098+0000","last_peered":"2024-09-20T18:59:51.858098+0000","last_clean":"2024-09-20T18:59:51.858098+0000","last_became_active":"2024-09-20T18:58:33.845463+0000","last_became_peered":"2024-09-20T18:58:33.845463+0000","last_unstale":"2024-09-20T18:59:51.858098+0000","last_undegraded":"2024-09-20T18:59:51.858098+0000","last_fullsized":"2024-09-20T18:59:51.858098+0000","mapping_epoch":246,"log_start":"253'1300","ondisk_log_start":"253'1300","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":4694,"log_dups_size":0,"ondisk_log_size":4694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:58.514651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":392822784,"num_objects":5994,"num_object_clones":0,"num_object_copies":17982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5994,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5994,"num_write_kb":383616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":538,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.766898+0000","last_change":"2024-09-20T18:59:05.105526+0000","last_active":"2024-09-20T18:59:50.766898+0000","last_peered":"2024-09-20T18:59:50.766898+0000","last_clean":"2024-09-20T18:59:50.766898+0000","last_became_active":"2024-09-20T18:59:05.074532+0000","last_became_peered":"2024-09-20T18:59:05.074532+0000","last_unstale":"2024-09-20T18:59:50.766898+0000","last_undegraded":"2024-09-20T18:59:50.766898+0000","last_fullsized":"2024-09-20T18:59:50.766898+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-20T18:54:57.394893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.394893+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:23.216727+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":459,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.792566+0000","last_change":"2024-09-20T18:58:29.939909+0000","last_active":"2024-09-20T18:59:50.792566+0000","last_peered":"2024-09-20T18:59:50.792566+0000","last_clean":"2024-09-20T18:59:50.792566+0000","last_became_active":"2024-09-20T18:49:45.940469+0000","last_became_peered":"2024-09-20T18:49:45.940469+0000","last_unstale":"2024-09-20T18:59:50.792566+0000","last_undegraded":"2024-09-20T18:59:50.792566+0000","last_fullsized":"2024-09-20T18:59:50.792566+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:58:29.939813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:29.939813+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"66'8","reported_seq":595,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.797820+0000","last_change":"2024-09-20T18:58:23.114419+0000","last_active":"2024-09-20T18:59:50.797820+0000","last_peered":"2024-09-20T18:59:50.797820+0000","last_clean":"2024-09-20T18:59:50.797820+0000","last_became_active":"2024-09-20T18:49:49.956133+0000","last_became_peered":"2024-09-20T18:49:49.956133+0000","last_unstale":"2024-09-20T18:59:50.797820+0000","last_undegraded":"2024-09-20T18:59:50.797820+0000","last_fullsized":"2024-09-20T18:59:50.797820+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:58:23.114371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:23.114371+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":468,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.789466+0000","last_change":"2024-09-20T18:59:05.082117+0000","last_active":"2024-09-20T18:59:50.789466+0000","last_peered":"2024-09-20T18:59:50.789466+0000","last_clean":"2024-09-20T18:59:50.789466+0000","last_became_active":"2024-09-20T18:59:05.081814+0000","last_became_peered":"2024-09-20T18:59:05.081814+0000","last_unstale":"2024-09-20T18:59:50.789466+0000","last_undegraded":"2024-09-20T18:59:50.789466+0000","last_fullsized":"2024-09-20T18:59:50.789466+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:29.530001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:29.530001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"273'5950","reported_seq":6015,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:51.079360+0000","last_change":"2024-09-20T18:58:33.826834+0000","last_active":"2024-09-20T18:59:51.079360+0000","last_peered":"2024-09-20T18:59:51.079360+0000","last_clean":"2024-09-20T18:59:51.079360+0000","last_became_active":"2024-09-20T18:58:33.826568+0000","last_became_peered":"2024-09-20T18:58:33.826568+0000","last_unstale":"2024-09-20T18:59:51.079360+0000","last_undegraded":"2024-09-20T18:59:51.079360+0000","last_fullsized":"2024-09-20T18:59:51.079360+0000","mapping_epoch":246,"log_start":"252'600","ondisk_log_start":"252'600","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":5350,"log_dups_size":0,"ondisk_log_size":5350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":389939200,"num_objects":5950,"num_object_clones":0,"num_object_copies":17850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5950,"num_write_kb":380800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":492,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.793043+0000","last_change":"2024-09-20T18:59:06.064691+0000","last_active":"2024-09-20T18:59:50.793043+0000","last_peered":"2024-09-20T18:59:50.793043+0000","last_clean":"2024-09-20T18:59:50.793043+0000","last_became_active":"2024-09-20T18:49:47.949626+0000","last_became_peered":"2024-09-20T18:49:47.949626+0000","last_unstale":"2024-09-20T18:59:50.793043+0000","last_undegraded":"2024-09-20T18:59:50.793043+0000","last_fullsized":"2024-09-20T18:59:50.793043+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:59:06.064599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:59:06.064599+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:33.522592+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":473,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.800189+0000","last_change":"2024-09-20T18:58:32.528966+0000","last_active":"2024-09-20T18:59:50.800189+0000","last_peered":"2024-09-20T18:59:50.800189+0000","last_clean":"2024-09-20T18:59:50.800189+0000","last_became_active":"2024-09-20T18:49:45.942496+0000","last_became_peered":"2024-09-20T18:49:45.942496+0000","last_unstale":"2024-09-20T18:59:50.800189+0000","last_undegraded":"2024-09-20T18:59:50.800189+0000","last_fullsized":"2024-09-20T18:59:50.800189+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.528310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.528310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"262'10","reported_seq":610,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.793624+0000","last_change":"2024-09-20T18:59:05.098010+0000","last_active":"2024-09-20T18:59:50.793624+0000","last_peered":"2024-09-20T18:59:50.793624+0000","last_clean":"2024-09-20T18:59:50.793624+0000","last_became_active":"2024-09-20T18:59:05.080183+0000","last_became_peered":"2024-09-20T18:59:05.080183+0000","last_unstale":"2024-09-20T18:59:50.793624+0000","last_undegraded":"2024-09-20T18:59:50.793624+0000","last_fullsized":"2024-09-20T18:59:50.793624+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'9","last_scrub_stamp":"2024-09-20T18:59:00.055110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:59:00.055110+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:07.456134+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":438,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.792040+0000","last_change":"2024-09-20T18:58:26.948165+0000","last_active":"2024-09-20T18:59:50.792040+0000","last_peered":"2024-09-20T18:59:50.792040+0000","last_clean":"2024-09-20T18:59:50.792040+0000","last_became_active":"2024-09-20T18:49:51.964780+0000","last_became_peered":"2024-09-20T18:49:51.964780+0000","last_unstale":"2024-09-20T18:59:50.792040+0000","last_undegraded":"2024-09-20T18:59:50.792040+0000","last_fullsized":"2024-09-20T18:59:50.792040+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:26.948102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:26.948102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"273'6069","reported_seq":6134,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.902266+0000","last_change":"2024-09-20T18:58:33.845804+0000","last_active":"2024-09-20T18:59:50.902266+0000","last_peered":"2024-09-20T18:59:50.902266+0000","last_clean":"2024-09-20T18:59:50.902266+0000","last_became_active":"2024-09-20T18:58:33.845493+0000","last_became_peered":"2024-09-20T18:58:33.845493+0000","last_unstale":"2024-09-20T18:59:50.902266+0000","last_undegraded":"2024-09-20T18:59:50.902266+0000","last_fullsized":"2024-09-20T18:59:50.902266+0000","mapping_epoch":246,"log_start":"253'1200","ondisk_log_start":"253'1200","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":4869,"log_dups_size":0,"ondisk_log_size":4869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:59.386071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":397737984,"num_objects":6069,"num_object_clones":0,"num_object_copies":18207,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6069,"num_write_kb":388416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":490,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.794908+0000","last_change":"2024-09-20T18:54:14.923895+0000","last_active":"2024-09-20T18:59:50.794908+0000","last_peered":"2024-09-20T18:59:50.794908+0000","last_clean":"2024-09-20T18:59:50.794908+0000","last_became_active":"2024-09-20T18:49:48.497736+0000","last_became_peered":"2024-09-20T18:49:48.497736+0000","last_unstale":"2024-09-20T18:59:50.794908+0000","last_undegraded":"2024-09-20T18:59:50.794908+0000","last_fullsized":"2024-09-20T18:59:50.794908+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-20T18:54:14.923682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:14.923682+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":449,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.812121+0000","last_change":"2024-09-20T18:57:09.940634+0000","last_active":"2024-09-20T18:59:50.812121+0000","last_peered":"2024-09-20T18:59:50.812121+0000","last_clean":"2024-09-20T18:59:50.812121+0000","last_became_active":"2024-09-20T18:49:45.942253+0000","last_became_peered":"2024-09-20T18:49:45.942253+0000","last_unstale":"2024-09-20T18:59:50.812121+0000","last_undegraded":"2024-09-20T18:59:50.812121+0000","last_fullsized":"2024-09-20T18:59:50.812121+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:09.940517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:09.940517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":36,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.788783+0000","last_change":"2024-09-20T18:59:05.085948+0000","last_active":"2024-09-20T18:59:50.788783+0000","last_peered":"2024-09-20T18:59:50.788783+0000","last_clean":"2024-09-20T18:59:50.788783+0000","last_became_active":"2024-09-20T18:59:05.085636+0000","last_became_peered":"2024-09-20T18:59:05.085636+0000","last_unstale":"2024-09-20T18:59:50.788783+0000","last_undegraded":"2024-09-20T18:59:50.788783+0000","last_fullsized":"2024-09-20T18:59:50.788783+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:50.610871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:50.610871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":475,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.789792+0000","last_change":"2024-09-20T18:59:05.085472+0000","last_active":"2024-09-20T18:59:50.789792+0000","last_peered":"2024-09-20T18:59:50.789792+0000","last_clean":"2024-09-20T18:59:50.789792+0000","last_became_active":"2024-09-20T18:59:05.085131+0000","last_became_peered":"2024-09-20T18:59:05.085131+0000","last_unstale":"2024-09-20T18:59:50.789792+0000","last_undegraded":"2024-09-20T18:59:50.789792+0000","last_fullsized":"2024-09-20T18:59:50.789792+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:58.973176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:58.973176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"273'6131","reported_seq":6196,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:52.195969+0000","last_change":"2024-09-20T18:58:33.822011+0000","last_active":"2024-09-20T18:59:52.195969+0000","last_peered":"2024-09-20T18:59:52.195969+0000","last_clean":"2024-09-20T18:59:52.195969+0000","last_became_active":"2024-09-20T18:58:33.820506+0000","last_became_peered":"2024-09-20T18:58:33.820506+0000","last_unstale":"2024-09-20T18:59:52.195969+0000","last_undegraded":"2024-09-20T18:59:52.195969+0000","last_fullsized":"2024-09-20T18:59:52.195969+0000","mapping_epoch":246,"log_start":"251'200","ondisk_log_start":"251'200","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":5931,"log_dups_size":0,"ondisk_log_size":5931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:55.828171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":401801216,"num_objects":6131,"num_object_clones":0,"num_object_copies":18393,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6131,"num_write_kb":392384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"66'28","reported_seq":631,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.812026+0000","last_change":"2024-09-20T18:58:07.018449+0000","last_active":"2024-09-20T18:59:50.812026+0000","last_peered":"2024-09-20T18:59:50.812026+0000","last_clean":"2024-09-20T18:59:50.812026+0000","last_became_active":"2024-09-20T18:49:47.955335+0000","last_became_peered":"2024-09-20T18:49:47.955335+0000","last_unstale":"2024-09-20T18:59:50.812026+0000","last_undegraded":"2024-09-20T18:59:50.812026+0000","last_fullsized":"2024-09-20T18:59:50.812026+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'28","last_scrub_stamp":"2024-09-20T18:58:07.018340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:07.018340+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":465,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.767121+0000","last_change":"2024-09-20T18:59:05.082509+0000","last_active":"2024-09-20T18:59:50.767121+0000","last_peered":"2024-09-20T18:59:50.767121+0000","last_clean":"2024-09-20T18:59:50.767121+0000","last_became_active":"2024-09-20T18:59:05.081468+0000","last_became_peered":"2024-09-20T18:59:05.081468+0000","last_unstale":"2024-09-20T18:59:50.767121+0000","last_undegraded":"2024-09-20T18:59:50.767121+0000","last_fullsized":"2024-09-20T18:59:50.767121+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:21.807703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:21.807703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:26.213224+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":505,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.799606+0000","last_change":"2024-09-20T18:59:05.108332+0000","last_active":"2024-09-20T18:59:50.799606+0000","last_peered":"2024-09-20T18:59:50.799606+0000","last_clean":"2024-09-20T18:59:50.799606+0000","last_became_active":"2024-09-20T18:59:05.086786+0000","last_became_peered":"2024-09-20T18:59:05.086786+0000","last_unstale":"2024-09-20T18:59:50.799606+0000","last_undegraded":"2024-09-20T18:59:50.799606+0000","last_fullsized":"2024-09-20T18:59:50.799606+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-20T18:54:51.227531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.227531+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0012781159999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":466,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.801961+0000","last_change":"2024-09-20T18:58:59.099799+0000","last_active":"2024-09-20T18:59:50.801961+0000","last_peered":"2024-09-20T18:59:50.801961+0000","last_clean":"2024-09-20T18:59:50.801961+0000","last_became_active":"2024-09-20T18:49:49.953971+0000","last_became_peered":"2024-09-20T18:49:49.953971+0000","last_unstale":"2024-09-20T18:59:50.801961+0000","last_undegraded":"2024-09-20T18:59:50.801961+0000","last_fullsized":"2024-09-20T18:59:50.801961+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:59.099736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:59.099736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:28.205403+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":437,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.801992+0000","last_change":"2024-09-20T18:57:12.744116+0000","last_active":"2024-09-20T18:59:50.801992+0000","last_peered":"2024-09-20T18:59:50.801992+0000","last_clean":"2024-09-20T18:59:50.801992+0000","last_became_active":"2024-09-20T18:49:51.969400+0000","last_became_peered":"2024-09-20T18:49:51.969400+0000","last_unstale":"2024-09-20T18:59:50.801992+0000","last_undegraded":"2024-09-20T18:59:50.801992+0000","last_fullsized":"2024-09-20T18:59:50.801992+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:12.744040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:12.744040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:54.130091+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"273'6062","reported_seq":6126,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:51.066463+0000","last_change":"2024-09-20T18:58:33.829042+0000","last_active":"2024-09-20T18:59:51.066463+0000","last_peered":"2024-09-20T18:59:51.066463+0000","last_clean":"2024-09-20T18:59:51.066463+0000","last_became_active":"2024-09-20T18:58:33.828787+0000","last_became_peered":"2024-09-20T18:58:33.828787+0000","last_unstale":"2024-09-20T18:59:51.066463+0000","last_undegraded":"2024-09-20T18:59:51.066463+0000","last_fullsized":"2024-09-20T18:59:51.066463+0000","mapping_epoch":246,"log_start":"252'700","ondisk_log_start":"252'700","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":5362,"log_dups_size":0,"ondisk_log_size":5362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:57.211364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":397279232,"num_objects":6062,"num_object_clones":0,"num_object_copies":18186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6062,"num_write_kb":387968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":496,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.811790+0000","last_change":"2024-09-20T18:58:06.079117+0000","last_active":"2024-09-20T18:59:50.811790+0000","last_peered":"2024-09-20T18:59:50.811790+0000","last_clean":"2024-09-20T18:59:50.811790+0000","last_became_active":"2024-09-20T18:49:47.950904+0000","last_became_peered":"2024-09-20T18:49:47.950904+0000","last_unstale":"2024-09-20T18:59:50.811790+0000","last_undegraded":"2024-09-20T18:59:50.811790+0000","last_fullsized":"2024-09-20T18:59:50.811790+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-20T18:58:06.078352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:06.078352+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":59,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":488,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.791959+0000","last_change":"2024-09-20T18:59:05.080117+0000","last_active":"2024-09-20T18:59:50.791959+0000","last_peered":"2024-09-20T18:59:50.791959+0000","last_clean":"2024-09-20T18:59:50.791959+0000","last_became_active":"2024-09-20T18:59:05.079789+0000","last_became_peered":"2024-09-20T18:59:05.079789+0000","last_unstale":"2024-09-20T18:59:50.791959+0000","last_undegraded":"2024-09-20T18:59:50.791959+0000","last_fullsized":"2024-09-20T18:59:50.791959+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:28.987458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:28.987458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":469,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.783068+0000","last_change":"2024-09-20T18:58:24.229997+0000","last_active":"2024-09-20T18:59:50.783068+0000","last_peered":"2024-09-20T18:59:50.783068+0000","last_clean":"2024-09-20T18:59:50.783068+0000","last_became_active":"2024-09-20T18:49:44.937135+0000","last_became_peered":"2024-09-20T18:49:44.937135+0000","last_unstale":"2024-09-20T18:59:50.783068+0000","last_undegraded":"2024-09-20T18:59:50.783068+0000","last_fullsized":"2024-09-20T18:59:50.783068+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:58:24.229938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:58:24.229938+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00095558800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":444,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.786750+0000","last_change":"2024-09-20T18:51:59.796450+0000","last_active":"2024-09-20T18:59:50.786750+0000","last_peered":"2024-09-20T18:59:50.786750+0000","last_clean":"2024-09-20T18:59:50.786750+0000","last_became_active":"2024-09-20T18:49:49.954424+0000","last_became_peered":"2024-09-20T18:49:49.954424+0000","last_unstale":"2024-09-20T18:59:50.786750+0000","last_undegraded":"2024-09-20T18:59:50.786750+0000","last_fullsized":"2024-09-20T18:59:50.786750+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:59.796383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:59.796383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":455,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.790886+0000","last_change":"2024-09-20T18:59:05.082433+0000","last_active":"2024-09-20T18:59:50.790886+0000","last_peered":"2024-09-20T18:59:50.790886+0000","last_clean":"2024-09-20T18:59:50.790886+0000","last_became_active":"2024-09-20T18:59:05.081459+0000","last_became_peered":"2024-09-20T18:59:05.081459+0000","last_unstale":"2024-09-20T18:59:50.790886+0000","last_undegraded":"2024-09-20T18:59:50.790886+0000","last_fullsized":"2024-09-20T18:59:50.790886+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:39.814259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:39.814259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"273'5891","reported_seq":10425,"reported_epoch":273,"state":"active+recovering+forced_recovery+undersized+remapped","last_fresh":"2024-09-20T18:59:52.213468+0000","last_change":"2024-09-20T18:59:36.561934+0000","last_active":"2024-09-20T18:59:52.213468+0000","last_peered":"2024-09-20T18:59:52.213468+0000","last_clean":"2024-09-20T18:59:04.032824+0000","last_became_active":"2024-09-20T18:59:06.207214+0000","last_became_peered":"2024-09-20T18:59:06.207214+0000","last_unstale":"2024-09-20T18:59:52.213468+0000","last_undegraded":"2024-09-20T18:59:52.213468+0000","last_fullsized":"2024-09-20T18:59:06.068079+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":5891,"log_dups_size":0,"ondisk_log_size":5891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":386072576,"num_objects":5891,"num_object_clones":0,"num_object_copies":17673,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":819,"num_objects_unfound":0,"num_objects_dirty":5891,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5891,"num_write_kb":377024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2232,"num_bytes_recovered":146276352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,5],"avail_no_missing":["0","2","5"],"object_location_counts":[{"shards":"0,2,5","objects":819},{"shards":"0,5","objects":5072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":496,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.792407+0000","last_change":"2024-09-20T18:59:06.022021+0000","last_active":"2024-09-20T18:59:50.792407+0000","last_peered":"2024-09-20T18:59:50.792407+0000","last_clean":"2024-09-20T18:59:50.792407+0000","last_became_active":"2024-09-20T18:49:47.949230+0000","last_became_peered":"2024-09-20T18:49:47.949230+0000","last_unstale":"2024-09-20T18:59:50.792407+0000","last_undegraded":"2024-09-20T18:59:50.792407+0000","last_fullsized":"2024-09-20T18:59:50.792407+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:59:06.021962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:59:06.021962+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:12.002280+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":466,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.793280+0000","last_change":"2024-09-20T18:54:27.905856+0000","last_active":"2024-09-20T18:59:50.793280+0000","last_peered":"2024-09-20T18:59:50.793280+0000","last_clean":"2024-09-20T18:59:50.793280+0000","last_became_active":"2024-09-20T18:49:45.939197+0000","last_became_peered":"2024-09-20T18:49:45.939197+0000","last_unstale":"2024-09-20T18:59:50.793280+0000","last_undegraded":"2024-09-20T18:59:50.793280+0000","last_fullsized":"2024-09-20T18:59:50.793280+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:27.905803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:27.905803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":1077,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.788407+0000","last_change":"2024-09-20T18:54:26.359887+0000","last_active":"2024-09-20T18:59:50.788407+0000","last_peered":"2024-09-20T18:59:50.788407+0000","last_clean":"2024-09-20T18:59:50.788407+0000","last_became_active":"2024-09-20T18:49:44.936819+0000","last_became_peered":"2024-09-20T18:49:44.936819+0000","last_unstale":"2024-09-20T18:59:50.788407+0000","last_undegraded":"2024-09-20T18:59:50.788407+0000","last_fullsized":"2024-09-20T18:59:50.788407+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-20T18:54:26.359807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:54:26.359807+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0010996199999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":599,"num_read_kb":594,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":457,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.800226+0000","last_change":"2024-09-20T18:57:54.362085+0000","last_active":"2024-09-20T18:59:50.800226+0000","last_peered":"2024-09-20T18:59:50.800226+0000","last_clean":"2024-09-20T18:59:50.800226+0000","last_became_active":"2024-09-20T18:49:49.961066+0000","last_became_peered":"2024-09-20T18:49:49.961066+0000","last_unstale":"2024-09-20T18:59:50.800226+0000","last_undegraded":"2024-09-20T18:59:50.800226+0000","last_fullsized":"2024-09-20T18:59:50.800226+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:54.362038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:54.362038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":461,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.811614+0000","last_change":"2024-09-20T18:58:08.019154+0000","last_active":"2024-09-20T18:59:50.811614+0000","last_peered":"2024-09-20T18:59:50.811614+0000","last_clean":"2024-09-20T18:59:50.811614+0000","last_became_active":"2024-09-20T18:49:51.970343+0000","last_became_peered":"2024-09-20T18:49:51.970343+0000","last_unstale":"2024-09-20T18:59:50.811614+0000","last_undegraded":"2024-09-20T18:59:50.811614+0000","last_fullsized":"2024-09-20T18:59:50.811614+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:08.019092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:08.019092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"273'5889","reported_seq":5954,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:51.870393+0000","last_change":"2024-09-20T18:58:33.820307+0000","last_active":"2024-09-20T18:59:51.870393+0000","last_peered":"2024-09-20T18:59:51.870393+0000","last_clean":"2024-09-20T18:59:51.870393+0000","last_became_active":"2024-09-20T18:58:33.820027+0000","last_became_peered":"2024-09-20T18:58:33.820027+0000","last_unstale":"2024-09-20T18:59:51.870393+0000","last_undegraded":"2024-09-20T18:59:51.870393+0000","last_fullsized":"2024-09-20T18:59:51.870393+0000","mapping_epoch":246,"log_start":"253'1200","ondisk_log_start":"253'1200","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":4689,"log_dups_size":0,"ondisk_log_size":4689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":385941504,"num_objects":5889,"num_object_clones":0,"num_object_copies":17667,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5889,"num_write_kb":376896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"262'18","reported_seq":63,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.789447+0000","last_change":"2024-09-20T18:59:05.102446+0000","last_active":"2024-09-20T18:59:50.789447+0000","last_peered":"2024-09-20T18:59:50.789447+0000","last_clean":"2024-09-20T18:59:50.789447+0000","last_became_active":"2024-09-20T18:59:05.088069+0000","last_became_peered":"2024-09-20T18:59:05.088069+0000","last_unstale":"2024-09-20T18:59:50.789447+0000","last_undegraded":"2024-09-20T18:59:50.789447+0000","last_fullsized":"2024-09-20T18:59:50.789447+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'17","last_scrub_stamp":"2024-09-20T18:57:22.816154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:22.816154+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":24,"num_read_kb":14,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":479,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.798318+0000","last_change":"2024-09-20T18:59:05.079328+0000","last_active":"2024-09-20T18:59:50.798318+0000","last_peered":"2024-09-20T18:59:50.798318+0000","last_clean":"2024-09-20T18:59:50.798318+0000","last_became_active":"2024-09-20T18:59:05.079111+0000","last_became_peered":"2024-09-20T18:59:05.079111+0000","last_unstale":"2024-09-20T18:59:50.798318+0000","last_undegraded":"2024-09-20T18:59:50.798318+0000","last_fullsized":"2024-09-20T18:59:50.798318+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.100625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.100625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":534,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.788594+0000","last_change":"2024-09-20T18:59:05.108551+0000","last_active":"2024-09-20T18:59:50.788594+0000","last_peered":"2024-09-20T18:59:50.788594+0000","last_clean":"2024-09-20T18:59:50.788594+0000","last_became_active":"2024-09-20T18:59:05.086103+0000","last_became_peered":"2024-09-20T18:59:05.086103+0000","last_unstale":"2024-09-20T18:59:50.788594+0000","last_undegraded":"2024-09-20T18:59:50.788594+0000","last_fullsized":"2024-09-20T18:59:50.788594+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T18:55:13.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T18:55:13.696607+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":450,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.790156+0000","last_change":"2024-09-20T18:58:23.545370+0000","last_active":"2024-09-20T18:59:50.790156+0000","last_peered":"2024-09-20T18:59:50.790156+0000","last_clean":"2024-09-20T18:59:50.790156+0000","last_became_active":"2024-09-20T18:49:49.956128+0000","last_became_peered":"2024-09-20T18:49:49.956128+0000","last_unstale":"2024-09-20T18:59:50.790156+0000","last_undegraded":"2024-09-20T18:59:50.790156+0000","last_fullsized":"2024-09-20T18:59:50.790156+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:23.545321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:23.545321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":461,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.800104+0000","last_change":"2024-09-20T18:58:03.523127+0000","last_active":"2024-09-20T18:59:50.800104+0000","last_peered":"2024-09-20T18:59:50.800104+0000","last_clean":"2024-09-20T18:59:50.800104+0000","last_became_active":"2024-09-20T18:49:51.971332+0000","last_became_peered":"2024-09-20T18:49:51.971332+0000","last_unstale":"2024-09-20T18:59:50.800104+0000","last_undegraded":"2024-09-20T18:59:50.800104+0000","last_fullsized":"2024-09-20T18:59:50.800104+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:03.523078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:03.523078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"273'6002","reported_seq":6067,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.863763+0000","last_change":"2024-09-20T18:58:33.825100+0000","last_active":"2024-09-20T18:59:50.863763+0000","last_peered":"2024-09-20T18:59:50.863763+0000","last_clean":"2024-09-20T18:59:50.863763+0000","last_became_active":"2024-09-20T18:58:33.824751+0000","last_became_peered":"2024-09-20T18:58:33.824751+0000","last_unstale":"2024-09-20T18:59:50.863763+0000","last_undegraded":"2024-09-20T18:59:50.863763+0000","last_fullsized":"2024-09-20T18:59:50.863763+0000","mapping_epoch":246,"log_start":"253'1100","ondisk_log_start":"253'1100","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":4902,"log_dups_size":0,"ondisk_log_size":4902,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393347072,"num_objects":6002,"num_object_clones":0,"num_object_copies":18006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6002,"num_write_kb":384128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":512,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.767646+0000","last_change":"2024-09-20T18:59:05.115678+0000","last_active":"2024-09-20T18:59:50.767646+0000","last_peered":"2024-09-20T18:59:50.767646+0000","last_clean":"2024-09-20T18:59:50.767646+0000","last_became_active":"2024-09-20T18:59:05.079753+0000","last_became_peered":"2024-09-20T18:59:05.079753+0000","last_unstale":"2024-09-20T18:59:50.767646+0000","last_undegraded":"2024-09-20T18:59:50.767646+0000","last_fullsized":"2024-09-20T18:59:50.767646+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:58:24.616580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:24.616580+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":459,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.767090+0000","last_change":"2024-09-20T18:58:32.694078+0000","last_active":"2024-09-20T18:59:50.767090+0000","last_peered":"2024-09-20T18:59:50.767090+0000","last_clean":"2024-09-20T18:59:50.767090+0000","last_became_active":"2024-09-20T18:49:45.944579+0000","last_became_peered":"2024-09-20T18:49:45.944579+0000","last_unstale":"2024-09-20T18:59:50.767090+0000","last_undegraded":"2024-09-20T18:59:50.767090+0000","last_fullsized":"2024-09-20T18:59:50.767090+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.693993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.693993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":471,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.799997+0000","last_change":"2024-09-20T18:59:05.090855+0000","last_active":"2024-09-20T18:59:50.799997+0000","last_peered":"2024-09-20T18:59:50.799997+0000","last_clean":"2024-09-20T18:59:50.799997+0000","last_became_active":"2024-09-20T18:59:05.090504+0000","last_became_peered":"2024-09-20T18:59:05.090504+0000","last_unstale":"2024-09-20T18:59:50.799997+0000","last_undegraded":"2024-09-20T18:59:50.799997+0000","last_fullsized":"2024-09-20T18:59:50.799997+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:11.401777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:11.401777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":467,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.808201+0000","last_change":"2024-09-20T18:59:05.080255+0000","last_active":"2024-09-20T18:59:50.808201+0000","last_peered":"2024-09-20T18:59:50.808201+0000","last_clean":"2024-09-20T18:59:50.808201+0000","last_became_active":"2024-09-20T18:59:05.079897+0000","last_became_peered":"2024-09-20T18:59:05.079897+0000","last_unstale":"2024-09-20T18:59:50.808201+0000","last_undegraded":"2024-09-20T18:59:50.808201+0000","last_fullsized":"2024-09-20T18:59:50.808201+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:01.083156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:59:01.083156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:15.111438+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.0","version":"273'6219","reported_seq":9570,"reported_epoch":273,"state":"active+recovering+forced_recovery+undersized+remapped","last_fresh":"2024-09-20T18:59:51.647976+0000","last_change":"2024-09-20T18:59:36.558017+0000","last_active":"2024-09-20T18:59:51.647976+0000","last_peered":"2024-09-20T18:59:51.647976+0000","last_clean":"2024-09-20T18:59:04.061108+0000","last_became_active":"2024-09-20T18:59:06.197951+0000","last_became_peered":"2024-09-20T18:59:06.197951+0000","last_unstale":"2024-09-20T18:59:51.647976+0000","last_undegraded":"2024-09-20T18:59:51.647976+0000","last_fullsized":"2024-09-20T18:59:06.066740+0000","mapping_epoch":261,"log_start":"253'1400","ondisk_log_start":"253'1400","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":4819,"log_dups_size":0,"ondisk_log_size":4819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":407568384,"num_objects":6219,"num_object_clones":0,"num_object_copies":18657,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1629,"num_objects_unfound":0,"num_objects_dirty":6219,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6219,"num_write_kb":398016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1641,"num_bytes_recovered":107544576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,2],"avail_no_missing":["1","0","2"],"object_location_counts":[{"shards":"0,1,2","objects":1629},{"shards":"1,2","objects":4590}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":491,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.767525+0000","last_change":"2024-09-20T18:58:36.716146+0000","last_active":"2024-09-20T18:59:50.767525+0000","last_peered":"2024-09-20T18:59:50.767525+0000","last_clean":"2024-09-20T18:59:50.767525+0000","last_became_active":"2024-09-20T18:49:47.947636+0000","last_became_peered":"2024-09-20T18:49:47.947636+0000","last_unstale":"2024-09-20T18:59:50.767525+0000","last_undegraded":"2024-09-20T18:59:50.767525+0000","last_fullsized":"2024-09-20T18:59:50.767525+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-20T18:58:36.716079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:36.716079+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":36,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.788919+0000","last_change":"2024-09-20T18:59:05.081802+0000","last_active":"2024-09-20T18:59:50.788919+0000","last_peered":"2024-09-20T18:59:50.788919+0000","last_clean":"2024-09-20T18:59:50.788919+0000","last_became_active":"2024-09-20T18:59:05.080829+0000","last_became_peered":"2024-09-20T18:59:05.080829+0000","last_unstale":"2024-09-20T18:59:50.788919+0000","last_undegraded":"2024-09-20T18:59:50.788919+0000","last_fullsized":"2024-09-20T18:59:50.788919+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:25.649733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:25.649733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":457,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.811712+0000","last_change":"2024-09-20T18:58:36.100033+0000","last_active":"2024-09-20T18:59:50.811712+0000","last_peered":"2024-09-20T18:59:50.811712+0000","last_clean":"2024-09-20T18:59:50.811712+0000","last_became_active":"2024-09-20T18:49:49.959160+0000","last_became_peered":"2024-09-20T18:49:49.959160+0000","last_unstale":"2024-09-20T18:59:50.811712+0000","last_undegraded":"2024-09-20T18:59:50.811712+0000","last_fullsized":"2024-09-20T18:59:50.811712+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:36.099985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:36.099985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:56.689064+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":456,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.782171+0000","last_change":"2024-09-20T18:58:25.227679+0000","last_active":"2024-09-20T18:59:50.782171+0000","last_peered":"2024-09-20T18:59:50.782171+0000","last_clean":"2024-09-20T18:59:50.782171+0000","last_became_active":"2024-09-20T18:49:51.970108+0000","last_became_peered":"2024-09-20T18:49:51.970108+0000","last_unstale":"2024-09-20T18:59:50.782171+0000","last_undegraded":"2024-09-20T18:59:50.782171+0000","last_fullsized":"2024-09-20T18:59:50.782171+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:25.227630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:25.227630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1","version":"273'6042","reported_seq":9477,"reported_epoch":273,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-20T18:59:51.877187+0000","last_change":"2024-09-20T18:59:06.244451+0000","last_active":"2024-09-20T18:59:51.877187+0000","last_peered":"2024-09-20T18:59:51.877187+0000","last_clean":"2024-09-20T18:59:04.063362+0000","last_became_active":"2024-09-20T18:59:06.208908+0000","last_became_peered":"2024-09-20T18:59:06.208908+0000","last_unstale":"2024-09-20T18:59:51.877187+0000","last_undegraded":"2024-09-20T18:59:51.877187+0000","last_fullsized":"2024-09-20T18:59:06.070513+0000","mapping_epoch":261,"log_start":"253'1300","ondisk_log_start":"253'1300","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":4742,"log_dups_size":0,"ondisk_log_size":4742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":395968512,"num_objects":6042,"num_object_clones":0,"num_object_copies":18126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1495,"num_objects_unfound":0,"num_objects_dirty":6042,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6042,"num_write_kb":386688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1683,"num_bytes_recovered":110297088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,7],"avail_no_missing":["3","4","7"],"object_location_counts":[{"shards":"3,4,7","objects":1495},{"shards":"3,7","objects":4547}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":516,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.767256+0000","last_change":"2024-09-20T18:59:10.364668+0000","last_active":"2024-09-20T18:59:50.767256+0000","last_peered":"2024-09-20T18:59:50.767256+0000","last_clean":"2024-09-20T18:59:50.767256+0000","last_became_active":"2024-09-20T18:59:05.073564+0000","last_became_peered":"2024-09-20T18:59:05.073564+0000","last_unstale":"2024-09-20T18:59:50.767256+0000","last_undegraded":"2024-09-20T18:59:50.767256+0000","last_fullsized":"2024-09-20T18:59:50.767256+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:59:10.364583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:59:10.364583+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:23.352032+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":465,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.808049+0000","last_change":"2024-09-20T18:58:31.126134+0000","last_active":"2024-09-20T18:59:50.808049+0000","last_peered":"2024-09-20T18:59:50.808049+0000","last_clean":"2024-09-20T18:59:50.808049+0000","last_became_active":"2024-09-20T18:49:45.937264+0000","last_became_peered":"2024-09-20T18:49:45.937264+0000","last_unstale":"2024-09-20T18:59:50.808049+0000","last_undegraded":"2024-09-20T18:59:50.808049+0000","last_fullsized":"2024-09-20T18:59:50.808049+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:31.126087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:31.126087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":590,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.783219+0000","last_change":"2024-09-20T18:55:17.549660+0000","last_active":"2024-09-20T18:59:50.783219+0000","last_peered":"2024-09-20T18:59:50.783219+0000","last_clean":"2024-09-20T18:59:50.783219+0000","last_became_active":"2024-09-20T18:49:49.957005+0000","last_became_peered":"2024-09-20T18:49:49.957005+0000","last_unstale":"2024-09-20T18:59:50.783219+0000","last_undegraded":"2024-09-20T18:59:50.783219+0000","last_fullsized":"2024-09-20T18:59:50.783219+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:17.549598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:17.549598+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":463,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.767390+0000","last_change":"2024-09-20T18:59:48.398618+0000","last_active":"2024-09-20T18:59:50.767390+0000","last_peered":"2024-09-20T18:59:50.767390+0000","last_clean":"2024-09-20T18:59:50.767390+0000","last_became_active":"2024-09-20T18:49:51.969355+0000","last_became_peered":"2024-09-20T18:49:51.969355+0000","last_unstale":"2024-09-20T18:59:50.767390+0000","last_undegraded":"2024-09-20T18:59:50.767390+0000","last_fullsized":"2024-09-20T18:59:50.767390+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:48.398542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:59:48.398542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:04.009303+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"273'5975","reported_seq":6040,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:51.085807+0000","last_change":"2024-09-20T18:58:33.826744+0000","last_active":"2024-09-20T18:59:51.085807+0000","last_peered":"2024-09-20T18:59:51.085807+0000","last_clean":"2024-09-20T18:59:51.085807+0000","last_became_active":"2024-09-20T18:58:33.826480+0000","last_became_peered":"2024-09-20T18:58:33.826480+0000","last_unstale":"2024-09-20T18:59:51.085807+0000","last_undegraded":"2024-09-20T18:59:51.085807+0000","last_fullsized":"2024-09-20T18:59:51.085807+0000","mapping_epoch":246,"log_start":"252'700","ondisk_log_start":"252'700","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":5275,"log_dups_size":0,"ondisk_log_size":5275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":391577600,"num_objects":5975,"num_object_clones":0,"num_object_copies":17925,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5975,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5975,"num_write_kb":382400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":478,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.794804+0000","last_change":"2024-09-20T18:54:54.897991+0000","last_active":"2024-09-20T18:59:50.794804+0000","last_peered":"2024-09-20T18:59:50.794804+0000","last_clean":"2024-09-20T18:59:50.794804+0000","last_became_active":"2024-09-20T18:49:47.946008+0000","last_became_peered":"2024-09-20T18:49:47.946008+0000","last_unstale":"2024-09-20T18:59:50.794804+0000","last_undegraded":"2024-09-20T18:59:50.794804+0000","last_fullsized":"2024-09-20T18:59:50.794804+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:54.897933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:54.897933+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:52.009917+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":487,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.799789+0000","last_change":"2024-09-20T18:59:05.090348+0000","last_active":"2024-09-20T18:59:50.799789+0000","last_peered":"2024-09-20T18:59:50.799789+0000","last_clean":"2024-09-20T18:59:50.799789+0000","last_became_active":"2024-09-20T18:59:05.089886+0000","last_became_peered":"2024-09-20T18:59:05.089886+0000","last_unstale":"2024-09-20T18:59:50.799789+0000","last_undegraded":"2024-09-20T18:59:50.799789+0000","last_fullsized":"2024-09-20T18:59:50.799789+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:56.405349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:56.405349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":460,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.783879+0000","last_change":"2024-09-20T18:54:43.379898+0000","last_active":"2024-09-20T18:59:50.783879+0000","last_peered":"2024-09-20T18:59:50.783879+0000","last_clean":"2024-09-20T18:59:50.783879+0000","last_became_active":"2024-09-20T18:49:49.956767+0000","last_became_peered":"2024-09-20T18:49:49.956767+0000","last_unstale":"2024-09-20T18:59:50.783879+0000","last_undegraded":"2024-09-20T18:59:50.783879+0000","last_fullsized":"2024-09-20T18:59:50.783879+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:43.379706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:43.379706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":446,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.793207+0000","last_change":"2024-09-20T18:54:24.976627+0000","last_active":"2024-09-20T18:59:50.793207+0000","last_peered":"2024-09-20T18:59:50.793207+0000","last_clean":"2024-09-20T18:59:50.793207+0000","last_became_active":"2024-09-20T18:49:51.964098+0000","last_became_peered":"2024-09-20T18:49:51.964098+0000","last_unstale":"2024-09-20T18:59:50.793207+0000","last_undegraded":"2024-09-20T18:59:50.793207+0000","last_fullsized":"2024-09-20T18:59:50.793207+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:24.976584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:24.976584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"273'6059","reported_seq":6124,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:51.061934+0000","last_change":"2024-09-20T18:58:33.824658+0000","last_active":"2024-09-20T18:59:51.061934+0000","last_peered":"2024-09-20T18:59:51.061934+0000","last_clean":"2024-09-20T18:59:51.061934+0000","last_became_active":"2024-09-20T18:58:33.824342+0000","last_became_peered":"2024-09-20T18:58:33.824342+0000","last_unstale":"2024-09-20T18:59:51.061934+0000","last_undegraded":"2024-09-20T18:59:51.061934+0000","last_fullsized":"2024-09-20T18:59:51.061934+0000","mapping_epoch":246,"log_start":"252'700","ondisk_log_start":"252'700","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":5359,"log_dups_size":0,"ondisk_log_size":5359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":397082624,"num_objects":6059,"num_object_clones":0,"num_object_copies":18177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6059,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6059,"num_write_kb":387776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":56,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.788686+0000","last_change":"2024-09-20T18:59:05.107086+0000","last_active":"2024-09-20T18:59:50.788686+0000","last_peered":"2024-09-20T18:59:50.788686+0000","last_clean":"2024-09-20T18:59:50.788686+0000","last_became_active":"2024-09-20T18:59:05.087391+0000","last_became_peered":"2024-09-20T18:59:05.087391+0000","last_unstale":"2024-09-20T18:59:50.788686+0000","last_undegraded":"2024-09-20T18:59:50.788686+0000","last_fullsized":"2024-09-20T18:59:50.788686+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-20T18:57:22.033475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:22.033475+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":31,"num_read_kb":20,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":480,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.791182+0000","last_change":"2024-09-20T18:59:05.087282+0000","last_active":"2024-09-20T18:59:50.791182+0000","last_peered":"2024-09-20T18:59:50.791182+0000","last_clean":"2024-09-20T18:59:50.791182+0000","last_became_active":"2024-09-20T18:59:05.086670+0000","last_became_peered":"2024-09-20T18:59:05.086670+0000","last_unstale":"2024-09-20T18:59:50.791182+0000","last_undegraded":"2024-09-20T18:59:50.791182+0000","last_fullsized":"2024-09-20T18:59:50.791182+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:44.501192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.501192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":460,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.783593+0000","last_change":"2024-09-20T18:55:13.577772+0000","last_active":"2024-09-20T18:59:50.783593+0000","last_peered":"2024-09-20T18:59:50.783593+0000","last_clean":"2024-09-20T18:59:50.783593+0000","last_became_active":"2024-09-20T18:49:49.957558+0000","last_became_peered":"2024-09-20T18:49:49.957558+0000","last_unstale":"2024-09-20T18:59:50.783593+0000","last_undegraded":"2024-09-20T18:59:50.783593+0000","last_fullsized":"2024-09-20T18:59:50.783593+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:13.577699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:13.577699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":454,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.788500+0000","last_change":"2024-09-20T18:53:47.370152+0000","last_active":"2024-09-20T18:59:50.788500+0000","last_peered":"2024-09-20T18:59:50.788500+0000","last_clean":"2024-09-20T18:59:50.788500+0000","last_became_active":"2024-09-20T18:49:51.970798+0000","last_became_peered":"2024-09-20T18:49:51.970798+0000","last_unstale":"2024-09-20T18:59:50.788500+0000","last_undegraded":"2024-09-20T18:59:50.788500+0000","last_fullsized":"2024-09-20T18:59:50.788500+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:47.370108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:47.370108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.4","version":"273'5890","reported_seq":5955,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:51.631378+0000","last_change":"2024-09-20T18:58:33.846071+0000","last_active":"2024-09-20T18:59:51.631378+0000","last_peered":"2024-09-20T18:59:51.631378+0000","last_clean":"2024-09-20T18:59:51.631378+0000","last_became_active":"2024-09-20T18:58:33.845736+0000","last_became_peered":"2024-09-20T18:58:33.845736+0000","last_unstale":"2024-09-20T18:59:51.631378+0000","last_undegraded":"2024-09-20T18:59:51.631378+0000","last_fullsized":"2024-09-20T18:59:51.631378+0000","mapping_epoch":246,"log_start":"253'1100","ondisk_log_start":"253'1100","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":4790,"log_dups_size":0,"ondisk_log_size":4790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":386007040,"num_objects":5890,"num_object_clones":0,"num_object_copies":17670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5890,"num_write_kb":376960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":478,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.767011+0000","last_change":"2024-09-20T18:53:52.898818+0000","last_active":"2024-09-20T18:59:50.767011+0000","last_peered":"2024-09-20T18:59:50.767011+0000","last_clean":"2024-09-20T18:59:50.767011+0000","last_became_active":"2024-09-20T18:49:47.949116+0000","last_became_peered":"2024-09-20T18:49:47.949116+0000","last_unstale":"2024-09-20T18:59:50.767011+0000","last_undegraded":"2024-09-20T18:59:50.767011+0000","last_fullsized":"2024-09-20T18:59:50.767011+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:53:52.898760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:53:52.898760+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":466,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.790811+0000","last_change":"2024-09-20T18:54:52.650798+0000","last_active":"2024-09-20T18:59:50.790811+0000","last_peered":"2024-09-20T18:59:50.790811+0000","last_clean":"2024-09-20T18:59:50.790811+0000","last_became_active":"2024-09-20T18:49:45.944858+0000","last_became_peered":"2024-09-20T18:49:45.944858+0000","last_unstale":"2024-09-20T18:59:50.790811+0000","last_undegraded":"2024-09-20T18:59:50.790811+0000","last_fullsized":"2024-09-20T18:59:50.790811+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:52.650756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:52.650756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":460,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.783418+0000","last_change":"2024-09-20T18:58:33.180357+0000","last_active":"2024-09-20T18:59:50.783418+0000","last_peered":"2024-09-20T18:59:50.783418+0000","last_clean":"2024-09-20T18:59:50.783418+0000","last_became_active":"2024-09-20T18:49:49.954944+0000","last_became_peered":"2024-09-20T18:49:49.954944+0000","last_unstale":"2024-09-20T18:59:50.783418+0000","last_undegraded":"2024-09-20T18:59:50.783418+0000","last_fullsized":"2024-09-20T18:59:50.783418+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:33.180309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:33.180309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":453,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.807724+0000","last_change":"2024-09-20T18:58:24.162921+0000","last_active":"2024-09-20T18:59:50.807724+0000","last_peered":"2024-09-20T18:59:50.807724+0000","last_clean":"2024-09-20T18:59:50.807724+0000","last_became_active":"2024-09-20T18:49:51.972970+0000","last_became_peered":"2024-09-20T18:49:51.972970+0000","last_unstale":"2024-09-20T18:59:50.807724+0000","last_undegraded":"2024-09-20T18:59:50.807724+0000","last_fullsized":"2024-09-20T18:59:50.807724+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:24.162879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:24.162879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"273'6113","reported_seq":6178,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:52.160942+0000","last_change":"2024-09-20T18:58:33.821228+0000","last_active":"2024-09-20T18:59:52.160942+0000","last_peered":"2024-09-20T18:59:52.160942+0000","last_clean":"2024-09-20T18:59:52.160942+0000","last_became_active":"2024-09-20T18:58:33.819949+0000","last_became_peered":"2024-09-20T18:58:33.819949+0000","last_unstale":"2024-09-20T18:59:52.160942+0000","last_undegraded":"2024-09-20T18:59:52.160942+0000","last_fullsized":"2024-09-20T18:59:52.160942+0000","mapping_epoch":246,"log_start":"251'200","ondisk_log_start":"251'200","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":5913,"log_dups_size":0,"ondisk_log_size":5913,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":400621568,"num_objects":6113,"num_object_clones":0,"num_object_copies":18339,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6113,"num_write_kb":391232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":532,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.799302+0000","last_change":"2024-09-20T18:59:05.108455+0000","last_active":"2024-09-20T18:59:50.799302+0000","last_peered":"2024-09-20T18:59:50.799302+0000","last_clean":"2024-09-20T18:59:50.799302+0000","last_became_active":"2024-09-20T18:59:05.083726+0000","last_became_peered":"2024-09-20T18:59:05.083726+0000","last_unstale":"2024-09-20T18:59:50.799302+0000","last_undegraded":"2024-09-20T18:59:50.799302+0000","last_fullsized":"2024-09-20T18:59:50.799302+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:58:30.522129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:30.522129+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":494,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.789841+0000","last_change":"2024-09-20T18:58:57.563127+0000","last_active":"2024-09-20T18:59:50.789841+0000","last_peered":"2024-09-20T18:59:50.789841+0000","last_clean":"2024-09-20T18:59:50.789841+0000","last_became_active":"2024-09-20T18:49:45.944256+0000","last_became_peered":"2024-09-20T18:49:45.944256+0000","last_unstale":"2024-09-20T18:59:50.789841+0000","last_undegraded":"2024-09-20T18:59:50.789841+0000","last_fullsized":"2024-09-20T18:59:50.789841+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-20T18:58:57.563058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:57.563058+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:20.547993+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"262'10","reported_seq":601,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.789157+0000","last_change":"2024-09-20T18:59:05.111302+0000","last_active":"2024-09-20T18:59:50.789157+0000","last_peered":"2024-09-20T18:59:50.789157+0000","last_clean":"2024-09-20T18:59:50.789157+0000","last_became_active":"2024-09-20T18:59:05.090811+0000","last_became_peered":"2024-09-20T18:59:05.090811+0000","last_unstale":"2024-09-20T18:59:50.789157+0000","last_undegraded":"2024-09-20T18:59:50.789157+0000","last_fullsized":"2024-09-20T18:59:50.789157+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'9","last_scrub_stamp":"2024-09-20T18:58:27.521051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:27.521051+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":467,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.799413+0000","last_change":"2024-09-20T18:59:05.087336+0000","last_active":"2024-09-20T18:59:50.799413+0000","last_peered":"2024-09-20T18:59:50.799413+0000","last_clean":"2024-09-20T18:59:50.799413+0000","last_became_active":"2024-09-20T18:59:05.087005+0000","last_became_peered":"2024-09-20T18:59:05.087005+0000","last_unstale":"2024-09-20T18:59:50.799413+0000","last_undegraded":"2024-09-20T18:59:50.799413+0000","last_fullsized":"2024-09-20T18:59:50.799413+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:05.523359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:05.523359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":36,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.788429+0000","last_change":"2024-09-20T18:59:05.085844+0000","last_active":"2024-09-20T18:59:50.788429+0000","last_peered":"2024-09-20T18:59:50.788429+0000","last_clean":"2024-09-20T18:59:50.788429+0000","last_became_active":"2024-09-20T18:59:05.085427+0000","last_became_peered":"2024-09-20T18:59:05.085427+0000","last_unstale":"2024-09-20T18:59:50.788429+0000","last_undegraded":"2024-09-20T18:59:50.788429+0000","last_fullsized":"2024-09-20T18:59:50.788429+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:26.721735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:26.721735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":470,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.798581+0000","last_change":"2024-09-20T18:58:09.063555+0000","last_active":"2024-09-20T18:59:50.798581+0000","last_peered":"2024-09-20T18:59:50.798581+0000","last_clean":"2024-09-20T18:59:50.798581+0000","last_became_active":"2024-09-20T18:49:48.500662+0000","last_became_peered":"2024-09-20T18:49:48.500662+0000","last_unstale":"2024-09-20T18:59:50.798581+0000","last_undegraded":"2024-09-20T18:59:50.798581+0000","last_fullsized":"2024-09-20T18:59:50.798581+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-20T18:58:09.063481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:09.063481+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":457,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.799943+0000","last_change":"2024-09-20T18:58:28.460106+0000","last_active":"2024-09-20T18:59:50.799943+0000","last_peered":"2024-09-20T18:59:50.799943+0000","last_clean":"2024-09-20T18:59:50.799943+0000","last_became_active":"2024-09-20T18:49:49.956749+0000","last_became_peered":"2024-09-20T18:49:49.956749+0000","last_unstale":"2024-09-20T18:59:50.799943+0000","last_undegraded":"2024-09-20T18:59:50.799943+0000","last_fullsized":"2024-09-20T18:59:50.799943+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:28.460054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:28.460054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":468,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.790788+0000","last_change":"2024-09-20T18:59:05.082737+0000","last_active":"2024-09-20T18:59:50.790788+0000","last_peered":"2024-09-20T18:59:50.790788+0000","last_clean":"2024-09-20T18:59:50.790788+0000","last_became_active":"2024-09-20T18:59:05.082018+0000","last_became_peered":"2024-09-20T18:59:05.082018+0000","last_unstale":"2024-09-20T18:59:50.790788+0000","last_undegraded":"2024-09-20T18:59:50.790788+0000","last_fullsized":"2024-09-20T18:59:50.790788+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.637739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.637739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":470,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.792489+0000","last_change":"2024-09-20T18:54:44.830872+0000","last_active":"2024-09-20T18:59:50.792489+0000","last_peered":"2024-09-20T18:59:50.792489+0000","last_clean":"2024-09-20T18:59:50.792489+0000","last_became_active":"2024-09-20T18:49:48.500803+0000","last_became_peered":"2024-09-20T18:49:48.500803+0000","last_unstale":"2024-09-20T18:59:50.792489+0000","last_undegraded":"2024-09-20T18:59:50.792489+0000","last_fullsized":"2024-09-20T18:59:50.792489+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:44.830790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.830790+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":480,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.790461+0000","last_change":"2024-09-20T18:59:05.091318+0000","last_active":"2024-09-20T18:59:50.790461+0000","last_peered":"2024-09-20T18:59:50.790461+0000","last_clean":"2024-09-20T18:59:50.790461+0000","last_became_active":"2024-09-20T18:59:05.090805+0000","last_became_peered":"2024-09-20T18:59:05.090805+0000","last_unstale":"2024-09-20T18:59:50.790461+0000","last_undegraded":"2024-09-20T18:59:50.790461+0000","last_fullsized":"2024-09-20T18:59:50.790461+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:59.589891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:59.589891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:14.124488+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":457,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.798418+0000","last_change":"2024-09-20T18:57:46.898929+0000","last_active":"2024-09-20T18:59:50.798418+0000","last_peered":"2024-09-20T18:59:50.798418+0000","last_clean":"2024-09-20T18:59:50.798418+0000","last_became_active":"2024-09-20T18:49:49.961947+0000","last_became_peered":"2024-09-20T18:49:49.961947+0000","last_unstale":"2024-09-20T18:59:50.798418+0000","last_undegraded":"2024-09-20T18:59:50.798418+0000","last_fullsized":"2024-09-20T18:59:50.798418+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:46.898862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:46.898862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":38,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"66'1","reported_seq":465,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.781521+0000","last_change":"2024-09-20T18:58:30.171382+0000","last_active":"2024-09-20T18:59:50.781521+0000","last_peered":"2024-09-20T18:59:50.781521+0000","last_clean":"2024-09-20T18:59:50.781521+0000","last_became_active":"2024-09-20T18:49:51.965121+0000","last_became_peered":"2024-09-20T18:49:51.965121+0000","last_unstale":"2024-09-20T18:59:50.781521+0000","last_undegraded":"2024-09-20T18:59:50.781521+0000","last_fullsized":"2024-09-20T18:59:50.781521+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T18:58:30.171329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:30.171329+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":469,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.800092+0000","last_change":"2024-09-20T18:57:59.430353+0000","last_active":"2024-09-20T18:59:50.800092+0000","last_peered":"2024-09-20T18:59:50.800092+0000","last_clean":"2024-09-20T18:59:50.800092+0000","last_became_active":"2024-09-20T18:49:48.501011+0000","last_became_peered":"2024-09-20T18:49:48.501011+0000","last_unstale":"2024-09-20T18:59:50.800092+0000","last_undegraded":"2024-09-20T18:59:50.800092+0000","last_fullsized":"2024-09-20T18:59:50.800092+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:57:59.430294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:59.430294+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":466,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.791882+0000","last_change":"2024-09-20T18:55:27.028184+0000","last_active":"2024-09-20T18:59:50.791882+0000","last_peered":"2024-09-20T18:59:50.791882+0000","last_clean":"2024-09-20T18:59:50.791882+0000","last_became_active":"2024-09-20T18:49:45.938511+0000","last_became_peered":"2024-09-20T18:49:45.938511+0000","last_unstale":"2024-09-20T18:59:50.791882+0000","last_undegraded":"2024-09-20T18:59:50.791882+0000","last_fullsized":"2024-09-20T18:59:50.791882+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:27.028126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:27.028126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":587,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.808421+0000","last_change":"2024-09-20T18:55:27.284156+0000","last_active":"2024-09-20T18:59:50.808421+0000","last_peered":"2024-09-20T18:59:50.808421+0000","last_clean":"2024-09-20T18:59:50.808421+0000","last_became_active":"2024-09-20T18:49:49.960550+0000","last_became_peered":"2024-09-20T18:49:49.960550+0000","last_unstale":"2024-09-20T18:59:50.808421+0000","last_undegraded":"2024-09-20T18:59:50.808421+0000","last_fullsized":"2024-09-20T18:59:50.808421+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:27.284086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:27.284086+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":455,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.767622+0000","last_change":"2024-09-20T18:55:12.348079+0000","last_active":"2024-09-20T18:59:50.767622+0000","last_peered":"2024-09-20T18:59:50.767622+0000","last_clean":"2024-09-20T18:59:50.767622+0000","last_became_active":"2024-09-20T18:49:51.966254+0000","last_became_peered":"2024-09-20T18:49:51.966254+0000","last_unstale":"2024-09-20T18:59:50.767622+0000","last_undegraded":"2024-09-20T18:59:50.767622+0000","last_fullsized":"2024-09-20T18:59:50.767622+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:12.347993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:55:12.347993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:52.541239+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":487,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.808606+0000","last_change":"2024-09-20T18:57:20.701592+0000","last_active":"2024-09-20T18:59:50.808606+0000","last_peered":"2024-09-20T18:59:50.808606+0000","last_clean":"2024-09-20T18:59:50.808606+0000","last_became_active":"2024-09-20T18:49:58.997494+0000","last_became_peered":"2024-09-20T18:49:58.997494+0000","last_unstale":"2024-09-20T18:59:50.808606+0000","last_undegraded":"2024-09-20T18:59:50.808606+0000","last_fullsized":"2024-09-20T18:59:50.808606+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:57:20.701543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:20.701543+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":466,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.790167+0000","last_change":"2024-09-20T18:54:57.632458+0000","last_active":"2024-09-20T18:59:50.790167+0000","last_peered":"2024-09-20T18:59:50.790167+0000","last_clean":"2024-09-20T18:59:50.790167+0000","last_became_active":"2024-09-20T18:49:45.949492+0000","last_became_peered":"2024-09-20T18:49:45.949492+0000","last_unstale":"2024-09-20T18:59:50.790167+0000","last_undegraded":"2024-09-20T18:59:50.790167+0000","last_fullsized":"2024-09-20T18:59:50.790167+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.632380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.632380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":586,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.799806+0000","last_change":"2024-09-20T18:55:28.302353+0000","last_active":"2024-09-20T18:59:50.799806+0000","last_peered":"2024-09-20T18:59:50.799806+0000","last_clean":"2024-09-20T18:59:50.799806+0000","last_became_active":"2024-09-20T18:49:49.962243+0000","last_became_peered":"2024-09-20T18:49:49.962243+0000","last_unstale":"2024-09-20T18:59:50.799806+0000","last_undegraded":"2024-09-20T18:59:50.799806+0000","last_fullsized":"2024-09-20T18:59:50.799806+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:28.302303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:28.302303+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":462,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.792060+0000","last_change":"2024-09-20T18:58:55.232948+0000","last_active":"2024-09-20T18:59:50.792060+0000","last_peered":"2024-09-20T18:59:50.792060+0000","last_clean":"2024-09-20T18:59:50.792060+0000","last_became_active":"2024-09-20T18:49:51.969093+0000","last_became_peered":"2024-09-20T18:49:51.969093+0000","last_unstale":"2024-09-20T18:59:50.792060+0000","last_undegraded":"2024-09-20T18:59:50.792060+0000","last_fullsized":"2024-09-20T18:59:50.792060+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:55.232883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:55.232883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:56.795382+0000","scrub_duration":59,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":520,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.766988+0000","last_change":"2024-09-20T18:59:05.117294+0000","last_active":"2024-09-20T18:59:50.766988+0000","last_peered":"2024-09-20T18:59:50.766988+0000","last_clean":"2024-09-20T18:59:50.766988+0000","last_became_active":"2024-09-20T18:59:05.081917+0000","last_became_peered":"2024-09-20T18:59:05.081917+0000","last_unstale":"2024-09-20T18:59:50.766988+0000","last_undegraded":"2024-09-20T18:59:50.766988+0000","last_fullsized":"2024-09-20T18:59:50.766988+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:57:50.899514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:50.899514+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:28.208671+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":473,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.767559+0000","last_change":"2024-09-20T18:59:06.310100+0000","last_active":"2024-09-20T18:59:50.767559+0000","last_peered":"2024-09-20T18:59:50.767559+0000","last_clean":"2024-09-20T18:59:50.767559+0000","last_became_active":"2024-09-20T18:59:05.073087+0000","last_became_peered":"2024-09-20T18:59:05.073087+0000","last_unstale":"2024-09-20T18:59:50.767559+0000","last_undegraded":"2024-09-20T18:59:50.767559+0000","last_fullsized":"2024-09-20T18:59:50.767559+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:06.310027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:59:06.310027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:35.049125+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":449,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.792904+0000","last_change":"2024-09-20T18:57:11.986605+0000","last_active":"2024-09-20T18:59:50.792904+0000","last_peered":"2024-09-20T18:59:50.792904+0000","last_clean":"2024-09-20T18:59:50.792904+0000","last_became_active":"2024-09-20T18:49:49.957067+0000","last_became_peered":"2024-09-20T18:49:49.957067+0000","last_unstale":"2024-09-20T18:59:50.792904+0000","last_undegraded":"2024-09-20T18:59:50.792904+0000","last_fullsized":"2024-09-20T18:59:50.792904+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:11.986528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:11.986528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":445,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.806969+0000","last_change":"2024-09-20T18:57:16.789412+0000","last_active":"2024-09-20T18:59:50.806969+0000","last_peered":"2024-09-20T18:59:50.806969+0000","last_clean":"2024-09-20T18:59:50.806969+0000","last_became_active":"2024-09-20T18:49:51.966915+0000","last_became_peered":"2024-09-20T18:49:51.966915+0000","last_unstale":"2024-09-20T18:59:50.806969+0000","last_undegraded":"2024-09-20T18:59:50.806969+0000","last_fullsized":"2024-09-20T18:59:50.806969+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:16.789095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:16.789095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":519,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.793131+0000","last_change":"2024-09-20T18:59:05.103863+0000","last_active":"2024-09-20T18:59:50.793131+0000","last_peered":"2024-09-20T18:59:50.793131+0000","last_clean":"2024-09-20T18:59:50.793131+0000","last_became_active":"2024-09-20T18:59:05.085230+0000","last_became_peered":"2024-09-20T18:59:05.085230+0000","last_unstale":"2024-09-20T18:59:50.793131+0000","last_undegraded":"2024-09-20T18:59:50.793131+0000","last_fullsized":"2024-09-20T18:59:50.793131+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:58:28.035450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:28.035450+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":458,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.791116+0000","last_change":"2024-09-20T18:54:42.487465+0000","last_active":"2024-09-20T18:59:50.791116+0000","last_peered":"2024-09-20T18:59:50.791116+0000","last_clean":"2024-09-20T18:59:50.791116+0000","last_became_active":"2024-09-20T18:49:45.944357+0000","last_became_peered":"2024-09-20T18:49:45.944357+0000","last_unstale":"2024-09-20T18:59:50.791116+0000","last_undegraded":"2024-09-20T18:59:50.791116+0000","last_fullsized":"2024-09-20T18:59:50.791116+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:42.487403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:42.487403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:51.577354+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":457,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.807814+0000","last_change":"2024-09-20T18:57:57.915090+0000","last_active":"2024-09-20T18:59:50.807814+0000","last_peered":"2024-09-20T18:59:50.807814+0000","last_clean":"2024-09-20T18:59:50.807814+0000","last_became_active":"2024-09-20T18:49:49.962523+0000","last_became_peered":"2024-09-20T18:49:49.962523+0000","last_unstale":"2024-09-20T18:59:50.807814+0000","last_undegraded":"2024-09-20T18:59:50.807814+0000","last_fullsized":"2024-09-20T18:59:50.807814+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:57.914913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:57.914913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":454,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.791744+0000","last_change":"2024-09-20T18:54:26.882441+0000","last_active":"2024-09-20T18:59:50.791744+0000","last_peered":"2024-09-20T18:59:50.791744+0000","last_clean":"2024-09-20T18:59:50.791744+0000","last_became_active":"2024-09-20T18:49:51.967313+0000","last_became_peered":"2024-09-20T18:49:51.967313+0000","last_unstale":"2024-09-20T18:59:50.791744+0000","last_undegraded":"2024-09-20T18:59:50.791744+0000","last_fullsized":"2024-09-20T18:59:50.791744+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:26.882375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:26.882375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":518,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.812656+0000","last_change":"2024-09-20T18:59:05.096414+0000","last_active":"2024-09-20T18:59:50.812656+0000","last_peered":"2024-09-20T18:59:50.812656+0000","last_clean":"2024-09-20T18:59:50.812656+0000","last_became_active":"2024-09-20T18:59:05.075390+0000","last_became_peered":"2024-09-20T18:59:05.075390+0000","last_unstale":"2024-09-20T18:59:50.812656+0000","last_undegraded":"2024-09-20T18:59:50.812656+0000","last_fullsized":"2024-09-20T18:59:50.812656+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:57:54.862329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:54.862329+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":473,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.799943+0000","last_change":"2024-09-20T18:58:06.479419+0000","last_active":"2024-09-20T18:59:50.799943+0000","last_peered":"2024-09-20T18:59:50.799943+0000","last_clean":"2024-09-20T18:59:50.799943+0000","last_became_active":"2024-09-20T18:49:45.942105+0000","last_became_peered":"2024-09-20T18:49:45.942105+0000","last_unstale":"2024-09-20T18:59:50.799943+0000","last_undegraded":"2024-09-20T18:59:50.799943+0000","last_fullsized":"2024-09-20T18:59:50.799943+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:06.479364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:06.479364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":472,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.788146+0000","last_change":"2024-09-20T18:59:05.081655+0000","last_active":"2024-09-20T18:59:50.788146+0000","last_peered":"2024-09-20T18:59:50.788146+0000","last_clean":"2024-09-20T18:59:50.788146+0000","last_became_active":"2024-09-20T18:59:05.081320+0000","last_became_peered":"2024-09-20T18:59:05.081320+0000","last_unstale":"2024-09-20T18:59:50.788146+0000","last_undegraded":"2024-09-20T18:59:50.788146+0000","last_fullsized":"2024-09-20T18:59:50.788146+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:46.329547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:46.329547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":459,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.807959+0000","last_change":"2024-09-20T18:59:05.081584+0000","last_active":"2024-09-20T18:59:50.807959+0000","last_peered":"2024-09-20T18:59:50.807959+0000","last_clean":"2024-09-20T18:59:50.807959+0000","last_became_active":"2024-09-20T18:59:05.081179+0000","last_became_peered":"2024-09-20T18:59:05.081179+0000","last_unstale":"2024-09-20T18:59:50.807959+0000","last_undegraded":"2024-09-20T18:59:50.807959+0000","last_fullsized":"2024-09-20T18:59:50.807959+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:28.165712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:28.165712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":493,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.797880+0000","last_change":"2024-09-20T18:59:05.098374+0000","last_active":"2024-09-20T18:59:50.797880+0000","last_peered":"2024-09-20T18:59:50.797880+0000","last_clean":"2024-09-20T18:59:50.797880+0000","last_became_active":"2024-09-20T18:59:05.076365+0000","last_became_peered":"2024-09-20T18:59:05.076365+0000","last_unstale":"2024-09-20T18:59:50.797880+0000","last_undegraded":"2024-09-20T18:59:50.797880+0000","last_fullsized":"2024-09-20T18:59:50.797880+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-20T18:55:20.282193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:20.282193+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":466,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.792931+0000","last_change":"2024-09-20T18:54:55.900487+0000","last_active":"2024-09-20T18:59:50.792931+0000","last_peered":"2024-09-20T18:59:50.792931+0000","last_clean":"2024-09-20T18:59:50.792931+0000","last_became_active":"2024-09-20T18:49:45.940988+0000","last_became_peered":"2024-09-20T18:49:45.940988+0000","last_unstale":"2024-09-20T18:59:50.792931+0000","last_undegraded":"2024-09-20T18:59:50.792931+0000","last_fullsized":"2024-09-20T18:59:50.792931+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:55.899611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:55.899611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:50.986844+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":36,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.790825+0000","last_change":"2024-09-20T18:59:05.086874+0000","last_active":"2024-09-20T18:59:50.790825+0000","last_peered":"2024-09-20T18:59:50.790825+0000","last_clean":"2024-09-20T18:59:50.790825+0000","last_became_active":"2024-09-20T18:59:05.086570+0000","last_became_peered":"2024-09-20T18:59:05.086570+0000","last_unstale":"2024-09-20T18:59:50.790825+0000","last_undegraded":"2024-09-20T18:59:50.790825+0000","last_fullsized":"2024-09-20T18:59:50.790825+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:52.929828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:52.929828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":446,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.789059+0000","last_change":"2024-09-20T18:58:22.558181+0000","last_active":"2024-09-20T18:59:50.789059+0000","last_peered":"2024-09-20T18:59:50.789059+0000","last_clean":"2024-09-20T18:59:50.789059+0000","last_became_active":"2024-09-20T18:49:51.968662+0000","last_became_peered":"2024-09-20T18:49:51.968662+0000","last_unstale":"2024-09-20T18:59:50.789059+0000","last_undegraded":"2024-09-20T18:59:50.789059+0000","last_fullsized":"2024-09-20T18:59:50.789059+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:22.558131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:22.558131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":436,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.811043+0000","last_change":"2024-09-20T18:57:10.970843+0000","last_active":"2024-09-20T18:59:50.811043+0000","last_peered":"2024-09-20T18:59:50.811043+0000","last_clean":"2024-09-20T18:59:50.811043+0000","last_became_active":"2024-09-20T18:49:51.968216+0000","last_became_peered":"2024-09-20T18:49:51.968216+0000","last_unstale":"2024-09-20T18:59:50.811043+0000","last_undegraded":"2024-09-20T18:59:50.811043+0000","last_fullsized":"2024-09-20T18:59:50.811043+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:10.970779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:10.970779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":465,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.800336+0000","last_change":"2024-09-20T18:58:36.077005+0000","last_active":"2024-09-20T18:59:50.800336+0000","last_peered":"2024-09-20T18:59:50.800336+0000","last_clean":"2024-09-20T18:59:50.800336+0000","last_became_active":"2024-09-20T18:49:45.937992+0000","last_became_peered":"2024-09-20T18:49:45.937992+0000","last_unstale":"2024-09-20T18:59:50.800336+0000","last_undegraded":"2024-09-20T18:59:50.800336+0000","last_fullsized":"2024-09-20T18:59:50.800336+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:36.076937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:36.076937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":48,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.789761+0000","last_change":"2024-09-20T18:59:05.116749+0000","last_active":"2024-09-20T18:59:50.789761+0000","last_peered":"2024-09-20T18:59:50.789761+0000","last_clean":"2024-09-20T18:59:50.789761+0000","last_became_active":"2024-09-20T18:59:05.089294+0000","last_became_peered":"2024-09-20T18:59:05.089294+0000","last_unstale":"2024-09-20T18:59:50.789761+0000","last_undegraded":"2024-09-20T18:59:50.789761+0000","last_fullsized":"2024-09-20T18:59:50.789761+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:54:44.719676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.719676+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":12,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":451,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-20T18:59:50.792256+0000","last_change":"2024-09-20T18:58:27.904636+0000","last_active":"2024-09-20T18:59:50.792256+0000","last_peered":"2024-09-20T18:59:50.792256+0000","last_clean":"2024-09-20T18:59:50.792256+0000","last_became_active":"2024-09-20T18:49:49.957582+0000","last_became_peered":"2024-09-20T18:49:49.957582+0000","last_unstale":"2024-09-20T18:59:50.792256+0000","last_undegraded":"2024-09-20T18:59:50.792256+0000","last_fullsized":"2024-09-20T18:59:50.792256+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:27.904592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:27.904592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":6307643392,"num_objects":96247,"num_object_clones":0,"num_object_copies":288741,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3943,"num_objects_unfound":0,"num_objects_dirty":96247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96247,"num_write_kb":6159808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5556,"num_bytes_recovered":364118016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":19253166080,"data_stored":19253166080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82247,"ondisk_log_size":82247,"up":48,"acting":45,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":695,"num_read_kb":452,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":2244,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":395,"ondisk_log_size":395,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":599,"num_read_kb":594,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266813,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3799664,"kb_used_data":3461664,"kb_used_omap":1,"kb_used_meta":337982,"kb_avail":89945488,"statfs":{"total":95995035648,"available":92104179712,"internally_reserved":0,"allocated":3544743936,"data_stored":3544056562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":346094026},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528452,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1357596,"kb_used_data":1141772,"kb_used_omap":1,"kb_used_meta":215806,"kb_avail":92387556,"statfs":{"total":95995035648,"available":94604857344,"internally_reserved":0,"allocated":1169174528,"data_stored":1168506972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":220985798},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757385,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3793372,"kb_used_data":3429004,"kb_used_omap":1,"kb_used_meta":364350,"kb_avail":89951780,"statfs":{"total":95995035648,"available":92110622720,"internally_reserved":0,"allocated":3511300096,"data_stored":3510632355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":373094853},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986318,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1626428,"kb_used_data":1342572,"kb_used_omap":1,"kb_used_meta":283838,"kb_avail":92118724,"statfs":{"total":95995035648,"available":94329573376,"internally_reserved":0,"allocated":1374793728,"data_stored":1374094471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":290650565},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182549,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3540040,"kb_used_data":3053492,"kb_used_omap":1,"kb_used_meta":486526,"kb_avail":90205112,"statfs":{"total":95995035648,"available":92370034688,"internally_reserved":0,"allocated":3126775808,"data_stored":3126074114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":498203079},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411482,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1779236,"kb_used_data":1346832,"kb_used_omap":1,"kb_used_meta":432382,"kb_avail":91965916,"statfs":{"total":95995035648,"available":94173097984,"internally_reserved":0,"allocated":1379155968,"data_stored":1378484710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":442759621},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575008,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3854236,"kb_used_data":3475980,"kb_used_omap":1,"kb_used_meta":378238,"kb_avail":89890916,"statfs":{"total":95995035648,"available":92048297984,"internally_reserved":0,"allocated":3559403520,"data_stored":3558678917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":387316165},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738534,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2141564,"kb_used_data":1745260,"kb_used_omap":1,"kb_used_meta":396286,"kb_avail":91603588,"statfs":{"total":95995035648,"available":93802074112,"internally_reserved":0,"allocated":1787146240,"data_stored":1786453185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":405797317},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1790443520,"data_stored":1790443520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3562733568,"data_stored":3562733568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1387200512,"data_stored":1387200512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3145269248,"data_stored":3145269248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1378418688,"data_stored":1378418688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3529179136,"data_stored":3529179136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":908263424,"data_stored":908263424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3551657984,"data_stored":3551657984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T18:59:58.093 INFO:tasks.thrashosds.thrasher:Setting 3.c to [2, 7, 6] 2024-09-20T18:59:58.093 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.c', '2', '7', '6'] 2024-09-20T18:59:58.093 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd pg-upmap 3.c 2 7 6 2024-09-20T18:59:58.111 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:59:58.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:58.184 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:58.184 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:59:58.255 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T18:59:58.255 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-09-20T18:59:58.255 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.255 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:102038 9.3 9:c6d1b336:::benchmark_data_smithi083_124552_object102037:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e275)", 2024-09-20T18:59:58.255 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:57.885903+0000", 2024-09-20T18:59:58.255 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.0022245699999999999, 2024-09-20T18:59:58.255 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0022558639999999998, 2024-09-20T18:59:58.255 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:59:58.255 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:59:58.255 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "waiting for sub ops", 2024-09-20T18:59:58.255 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:59:58.256 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T18:59:58.256 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T18:59:58.256 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 102038 2024-09-20T18:59:58.256 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.256 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:59:58.256 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.256 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:59:58.256 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.885903+0000", 2024-09-20T18:59:58.256 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:58.256 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.256 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.256 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:59:58.256 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.885903+0000", 2024-09-20T18:59:58.256 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:58.257 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.257 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.257 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:59:58.257 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.885904+0000", 2024-09-20T18:59:58.257 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.438e-06 2024-09-20T18:59:58.257 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.257 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.257 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:59:58.257 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.885950+0000", 2024-09-20T18:59:58.257 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.5945999999999997e-05 2024-09-20T18:59:58.257 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.257 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.257 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:59:58.257 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.885960+0000", 2024-09-20T18:59:58.257 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0032e-05 2024-09-20T18:59:58.257 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.258 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.258 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:59:58.258 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.885967+0000", 2024-09-20T18:59:58.258 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5729999999999996e-06 2024-09-20T18:59:58.258 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.258 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.258 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:59:58.258 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.886048+0000", 2024-09-20T18:59:58.258 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.0704999999999994e-05 2024-09-20T18:59:58.258 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.258 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.258 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:59:58.258 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.886183+0000", 2024-09-20T18:59:58.258 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000135477 2024-09-20T18:59:58.259 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.259 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.259 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,3", 2024-09-20T18:59:58.259 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.886320+0000", 2024-09-20T18:59:58.259 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013645899999999999 2024-09-20T18:59:58.259 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:58.259 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:59:58.259 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:58.259 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.259 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.259 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:102019 9.d 9:b6161e25:::benchmark_data_smithi083_124552_object102018:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e275)", 2024-09-20T18:59:58.259 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:57.869216+0000", 2024-09-20T18:59:58.259 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.018911462, 2024-09-20T18:59:58.260 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.019011986000000002, 2024-09-20T18:59:58.260 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:59:58.260 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:59:58.260 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "waiting for sub ops", 2024-09-20T18:59:58.260 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:59:58.260 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T18:59:58.260 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T18:59:58.260 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 102019 2024-09-20T18:59:58.260 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.260 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:59:58.260 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.260 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:59:58.260 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.869216+0000", 2024-09-20T18:59:58.260 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:58.260 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.261 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.261 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:59:58.261 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.869216+0000", 2024-09-20T18:59:58.261 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:58.261 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.261 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.261 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:59:58.261 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.869218+0000", 2024-09-20T18:59:58.261 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.671e-06 2024-09-20T18:59:58.261 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.261 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.261 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:59:58.261 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.869251+0000", 2024-09-20T18:59:58.261 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.3611000000000001e-05 2024-09-20T18:59:58.261 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.262 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.262 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:59:58.262 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.869267+0000", 2024-09-20T18:59:58.262 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5897e-05 2024-09-20T18:59:58.262 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.262 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.262 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:59:58.262 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.869275+0000", 2024-09-20T18:59:58.262 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.1529999999999997e-06 2024-09-20T18:59:58.262 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.262 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.262 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:59:58.262 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.870091+0000", 2024-09-20T18:59:58.262 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00081585499999999999 2024-09-20T18:59:58.263 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.263 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.263 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:59:58.263 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.870282+0000", 2024-09-20T18:59:58.263 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00019113800000000001 2024-09-20T18:59:58.263 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.263 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.263 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,5", 2024-09-20T18:59:58.263 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.870779+0000", 2024-09-20T18:59:58.263 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00049704999999999997 2024-09-20T18:59:58.263 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.263 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.263 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:58.263 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.877908+0000", 2024-09-20T18:59:58.263 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0071287750000000004 2024-09-20T18:59:58.264 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.264 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.264 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T18:59:58.264 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.880132+0000", 2024-09-20T18:59:58.264 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0022240910000000001 2024-09-20T18:59:58.264 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.264 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.264 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:58.264 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.885571+0000", 2024-09-20T18:59:58.264 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0054387120000000001 2024-09-20T18:59:58.264 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:58.264 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:59:58.264 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:58.264 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.264 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.265 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:102011 9.7 e275/246 9:eb7598da:::benchmark_data_smithi083_124552_object102010:head v 275'6485, mlcod=275'6482)", 2024-09-20T18:59:58.265 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:57.870784+0000", 2024-09-20T18:59:58.265 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.017343758000000001, 2024-09-20T18:59:58.265 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.017513984999999999, 2024-09-20T18:59:58.265 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:59:58.265 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:59:58.265 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-09-20T18:59:58.265 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:59:58.265 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.265 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:59:58.265 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.870784+0000", 2024-09-20T18:59:58.265 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:58.265 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.265 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.265 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:59:58.265 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.870784+0000", 2024-09-20T18:59:58.266 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:58.266 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.266 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.266 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:59:58.266 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.870784+0000", 2024-09-20T18:59:58.266 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.8299999999999996e-07 2024-09-20T18:59:58.266 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.266 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.266 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:59:58.266 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.870807+0000", 2024-09-20T18:59:58.266 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2878e-05 2024-09-20T18:59:58.266 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.266 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.266 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:59:58.267 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.870826+0000", 2024-09-20T18:59:58.267 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8930999999999999e-05 2024-09-20T18:59:58.267 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.267 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.267 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:59:58.267 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.870833+0000", 2024-09-20T18:59:58.267 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.6760000000000001e-06 2024-09-20T18:59:58.267 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.267 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.267 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:59:58.267 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.876900+0000", 2024-09-20T18:59:58.267 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0060674350000000004 2024-09-20T18:59:58.267 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.267 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.267 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:59:58.268 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.876921+0000", 2024-09-20T18:59:58.268 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0890000000000002e-05 2024-09-20T18:59:58.268 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:58.268 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:59:58.268 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:58.268 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.268 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.268 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:102016 9.7 e275/246 9:e244fb7d:::benchmark_data_smithi083_124552_object102015:head v 275'6486, mlcod=275'6483)", 2024-09-20T18:59:58.268 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:57.874546+0000", 2024-09-20T18:59:58.268 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.013581632999999999, 2024-09-20T18:59:58.268 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.013799171000000001, 2024-09-20T18:59:58.268 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:59:58.268 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:59:58.268 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-09-20T18:59:58.269 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:59:58.269 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.269 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:59:58.269 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.874546+0000", 2024-09-20T18:59:58.269 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:58.269 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.269 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.269 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:59:58.269 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.874546+0000", 2024-09-20T18:59:58.269 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:58.269 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.269 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.269 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:59:58.269 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.874547+0000", 2024-09-20T18:59:58.269 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1689999999999999e-06 2024-09-20T18:59:58.269 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.270 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.270 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:59:58.270 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.874577+0000", 2024-09-20T18:59:58.270 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9719000000000001e-05 2024-09-20T18:59:58.270 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.270 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.270 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:59:58.270 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.874590+0000", 2024-09-20T18:59:58.270 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3365e-05 2024-09-20T18:59:58.270 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.270 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.270 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:59:58.270 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.874598+0000", 2024-09-20T18:59:58.270 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.9100000000000005e-06 2024-09-20T18:59:58.271 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.271 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.271 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:59:58.271 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.879594+0000", 2024-09-20T18:59:58.271 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0049964060000000001 2024-09-20T18:59:58.271 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.271 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.271 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:59:58.271 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.879616+0000", 2024-09-20T18:59:58.271 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1846999999999999e-05 2024-09-20T18:59:58.271 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:58.271 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:59:58.271 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:58.271 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.271 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.272 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:102028 9.e 9:7828fe8b:::benchmark_data_smithi083_124552_object102027:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e275)", 2024-09-20T18:59:58.272 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:57.879182+0000", 2024-09-20T18:59:58.272 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.0089450910000000005, 2024-09-20T18:59:58.272 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0092118259999999993, 2024-09-20T18:59:58.272 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:59:58.272 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:59:58.272 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "waiting for sub ops", 2024-09-20T18:59:58.272 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T18:59:58.272 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T18:59:58.272 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T18:59:58.272 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 102028 2024-09-20T18:59:58.272 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.272 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:59:58.272 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.272 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:59:58.273 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.879182+0000", 2024-09-20T18:59:58.273 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:58.273 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.273 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.273 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:59:58.273 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.879182+0000", 2024-09-20T18:59:58.273 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:58.273 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.273 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.273 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:59:58.273 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.879184+0000", 2024-09-20T18:59:58.273 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4440000000000001e-06 2024-09-20T18:59:58.273 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.273 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.273 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:59:58.274 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.879219+0000", 2024-09-20T18:59:58.274 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.5488000000000003e-05 2024-09-20T18:59:58.274 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.274 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.274 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:59:58.274 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.879231+0000", 2024-09-20T18:59:58.274 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1602e-05 2024-09-20T18:59:58.274 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.274 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.274 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:59:58.274 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.879239+0000", 2024-09-20T18:59:58.274 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.8180000000000004e-06 2024-09-20T18:59:58.274 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.274 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.275 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:59:58.275 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.880539+0000", 2024-09-20T18:59:58.275 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00129996 2024-09-20T18:59:58.275 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.275 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.275 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:59:58.275 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.880660+0000", 2024-09-20T18:59:58.275 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012167099999999999 2024-09-20T18:59:58.275 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.275 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.275 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,4", 2024-09-20T18:59:58.275 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.880797+0000", 2024-09-20T18:59:58.275 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013696100000000001 2024-09-20T18:59:58.275 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.276 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.276 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:58.276 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.884651+0000", 2024-09-20T18:59:58.276 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.003853216 2024-09-20T18:59:58.276 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.276 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.276 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T18:59:58.276 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.884806+0000", 2024-09-20T18:59:58.276 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015555500000000001 2024-09-20T18:59:58.276 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:58.276 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:59:58.276 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:58.276 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.276 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.277 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:102029 9.5 e275/246 9:ae963f12:::benchmark_data_smithi083_124552_object102028:head v 275'6418, mlcod=275'6415)", 2024-09-20T18:59:58.277 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:57.881349+0000", 2024-09-20T18:59:58.277 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.006778839, 2024-09-20T18:59:58.277 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0071155130000000004, 2024-09-20T18:59:58.277 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:59:58.277 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:59:58.277 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-09-20T18:59:58.277 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:59:58.277 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.277 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:59:58.277 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.881349+0000", 2024-09-20T18:59:58.277 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:58.277 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.277 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.277 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:59:58.277 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.881349+0000", 2024-09-20T18:59:58.277 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:58.278 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.278 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.278 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:59:58.278 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.881349+0000", 2024-09-20T18:59:58.278 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.8800000000000002e-07 2024-09-20T18:59:58.278 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.278 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.278 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:59:58.278 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.881384+0000", 2024-09-20T18:59:58.278 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4798000000000001e-05 2024-09-20T18:59:58.278 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.278 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.278 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:59:58.278 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.881398+0000", 2024-09-20T18:59:58.278 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3393999999999999e-05 2024-09-20T18:59:58.279 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.279 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.279 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:59:58.279 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.881404+0000", 2024-09-20T18:59:58.279 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.8730000000000001e-06 2024-09-20T18:59:58.279 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.279 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.279 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:59:58.279 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.881653+0000", 2024-09-20T18:59:58.279 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00024874700000000001 2024-09-20T18:59:58.279 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.279 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.279 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:59:58.279 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.881670+0000", 2024-09-20T18:59:58.279 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6671e-05 2024-09-20T18:59:58.279 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:58.280 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:59:58.280 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:58.280 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.280 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.280 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:102030 9.7 e275/246 9:ec65e65e:::benchmark_data_smithi083_124552_object102029:head v 275'6487, mlcod=275'6483)", 2024-09-20T18:59:58.280 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:57.884935+0000", 2024-09-20T18:59:58.280 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.0031924560000000002, 2024-09-20T18:59:58.280 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.003574842, 2024-09-20T18:59:58.280 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:59:58.280 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:59:58.280 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-09-20T18:59:58.280 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:59:58.280 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.280 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:59:58.280 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.884935+0000", 2024-09-20T18:59:58.280 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:58.281 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.281 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.281 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:59:58.281 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.884935+0000", 2024-09-20T18:59:58.281 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:58.281 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.281 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.281 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:59:58.281 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.884935+0000", 2024-09-20T18:59:58.281 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.8700000000000002e-07 2024-09-20T18:59:58.281 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.281 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.281 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:59:58.281 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.884959+0000", 2024-09-20T18:59:58.281 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.4204000000000001e-05 2024-09-20T18:59:58.281 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.282 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.282 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:59:58.282 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.884965+0000", 2024-09-20T18:59:58.282 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5690000000000004e-06 2024-09-20T18:59:58.282 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.282 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.282 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:59:58.282 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.884969+0000", 2024-09-20T18:59:58.282 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.357e-06 2024-09-20T18:59:58.282 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.282 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.282 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:59:58.282 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.887074+0000", 2024-09-20T18:59:58.282 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.002104294 2024-09-20T18:59:58.282 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.283 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.283 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:59:58.283 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.887091+0000", 2024-09-20T18:59:58.283 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7484000000000001e-05 2024-09-20T18:59:58.283 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:58.283 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:59:58.283 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:58.283 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.283 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.283 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop_reply(client.25484.0:102019 9.d e275/246 ondisk, result = 0)", 2024-09-20T18:59:58.283 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:57.885014+0000", 2024-09-20T18:59:58.283 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.0031130160000000001, 2024-09-20T18:59:58.283 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0035421789999999999, 2024-09-20T18:59:58.283 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:59:58.283 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:59:58.284 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-09-20T18:59:58.284 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:59:58.284 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.284 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:59:58.284 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.885014+0000", 2024-09-20T18:59:58.284 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:58.284 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.284 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.284 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:59:58.284 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.885014+0000", 2024-09-20T18:59:58.284 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:58.284 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.284 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.284 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:59:58.284 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.885015+0000", 2024-09-20T18:59:58.284 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9799999999999999e-07 2024-09-20T18:59:58.285 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.285 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.285 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:59:58.285 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.885017+0000", 2024-09-20T18:59:58.285 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.165e-06 2024-09-20T18:59:58.285 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.285 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.285 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:59:58.285 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.885022+0000", 2024-09-20T18:59:58.285 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.434e-06 2024-09-20T18:59:58.285 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.285 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.285 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:59:58.285 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.885027+0000", 2024-09-20T18:59:58.285 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.4259999999999996e-06 2024-09-20T18:59:58.286 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.286 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.286 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:59:58.286 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.885533+0000", 2024-09-20T18:59:58.286 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00050628399999999997 2024-09-20T18:59:58.286 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.286 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.286 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:59:58.286 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.885554+0000", 2024-09-20T18:59:58.286 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.073e-05 2024-09-20T18:59:58.286 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:58.286 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:59:58.286 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:58.286 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.286 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.287 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:102033 9.c e275/246 9:3ed27a2c:::benchmark_data_smithi083_124552_object102032:head v 275'6331, mlcod=275'6329)", 2024-09-20T18:59:58.287 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:57.885678+0000", 2024-09-20T18:59:58.287 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.0024494230000000001, 2024-09-20T18:59:58.287 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0029236269999999998, 2024-09-20T18:59:58.287 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T18:59:58.287 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T18:59:58.287 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-09-20T18:59:58.287 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T18:59:58.287 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.287 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T18:59:58.287 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.885678+0000", 2024-09-20T18:59:58.287 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:58.287 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.287 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.287 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T18:59:58.288 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.885678+0000", 2024-09-20T18:59:58.288 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T18:59:58.288 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.288 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.288 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T18:59:58.288 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.885679+0000", 2024-09-20T18:59:58.288 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.3399999999999998e-07 2024-09-20T18:59:58.288 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.288 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.288 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T18:59:58.288 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.885707+0000", 2024-09-20T18:59:58.288 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.8262000000000001e-05 2024-09-20T18:59:58.288 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.288 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.288 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T18:59:58.288 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.885725+0000", 2024-09-20T18:59:58.289 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7638e-05 2024-09-20T18:59:58.289 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.289 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.289 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T18:59:58.289 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.885732+0000", 2024-09-20T18:59:58.289 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.3660000000000003e-06 2024-09-20T18:59:58.289 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.289 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.289 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T18:59:58.289 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.886146+0000", 2024-09-20T18:59:58.289 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00041362400000000002 2024-09-20T18:59:58.289 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T18:59:58.289 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T18:59:58.289 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T18:59:58.289 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.886163+0000", 2024-09-20T18:59:58.289 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7604999999999998e-05 2024-09-20T18:59:58.290 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:58.290 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T18:59:58.290 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:58.290 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T18:59:58.290 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-09-20T18:59:58.290 INFO:teuthology.orchestra.run.smithi142.stdout: "num_ops": 9 2024-09-20T18:59:58.290 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T18:59:58.290 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:59:58.291 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 78 16 102384 102368 82.013 47.25 0.00751937 0.0121886 2024-09-20T18:59:58.296 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:58 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:59:58.296 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:58 smithi083 ceph-mon[29421]: pgmap v495: 148 pgs: 1 active+recovering+undersized+remapped, 2 active+recovering+forced_recovery+undersized+remapped, 145 active+clean; 6.1 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 60 KiB/s rd, 84 MiB/s wr, 1.45k op/s; 3448/300966 objects misplaced (1.146%); 13 MiB/s, 204 objects/s recovering 2024-09-20T18:59:58.296 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:58 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:59:58.296 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:58 smithi083 ceph-mon[36260]: pgmap v495: 148 pgs: 1 active+recovering+undersized+remapped, 2 active+recovering+forced_recovery+undersized+remapped, 145 active+clean; 6.1 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 60 KiB/s rd, 84 MiB/s wr, 1.45k op/s; 3448/300966 objects misplaced (1.146%); 13 MiB/s, 204 objects/s recovering 2024-09-20T18:59:58.322 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:58.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:58.323 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:59:58.423 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T18:59:58.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:58.462 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:58.463 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T18:59:58.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:58 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T18:59:58.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:58 smithi142 ceph-mon[32508]: pgmap v495: 148 pgs: 1 active+recovering+undersized+remapped, 2 active+recovering+forced_recovery+undersized+remapped, 145 active+clean; 6.1 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 60 KiB/s rd, 84 MiB/s wr, 1.45k op/s; 3448/300966 objects misplaced (1.146%); 13 MiB/s, 204 objects/s recovering 2024-09-20T18:59:58.563 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:58.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:58.601 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:58.601 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:58.702 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T18:59:58.765 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:58.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:58.768 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T18:59:58.811 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T18:59:58.869 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:58.899 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:58.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:58.901 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:59.001 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T18:59:59.035 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:59.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:59.036 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T18:59:59.137 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:59.151 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-20T18:59:59.179 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_blocked_ops 2024-09-20T18:59:59.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:59.181 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:59.182 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T18:59:59.257 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 79 16 103377 103361 81.7605 62.0625 0.0128786 0.0122259 2024-09-20T18:59:59.283 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T18:59:59.296 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:59 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/314735193' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:59:59.296 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 18:59:59 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:59:59.296 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:59 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/314735193' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:59:59.296 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 18:59:59 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:59:59.319 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:59.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:59.320 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T18:59:59.421 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T18:59:59.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:59.477 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:59.477 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T18:59:59.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:59 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/314735193' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:59:59.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 18:59:59 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T18:59:59.558 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T18:59:59.579 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T18:59:59.617 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T18:59:59.624 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:59.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:59.625 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T18:59:59.726 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T18:59:59.763 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T18:59:59.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:59.763 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T18:59:59.864 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T18:59:59.898 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T18:59:59.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T18:59:59.899 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:00:00.000 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:00:00.040 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:00.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:00.041 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:00:00.141 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:00.164 INFO:teuthology.orchestra.run.smithi083.stderr:set 3.c pg_upmap mapping to [2,7,6] 2024-09-20T19:00:00.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:00.174 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:00.174 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:00.219 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T19:00:00.219 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-09-20T19:00:00.219 INFO:teuthology.orchestra.run.smithi142.stdout: "complaint_time": 30, 2024-09-20T19:00:00.219 INFO:teuthology.orchestra.run.smithi142.stdout: "num_blocked_ops": 0 2024-09-20T19:00:00.220 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T19:00:00.257 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:2024-09-20T19:00:00.249024+0000 min lat: 0.00133037 max lat: 2.15034 avg lat: 0.0122636 2024-09-20T19:00:00.257 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-20T19:00:00.257 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 80 16 104363 104347 81.5087 61.625 0.0173208 0.0122636 2024-09-20T19:00:00.275 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:00:00.296 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:00 smithi083 ceph-mon[29421]: pgmap v496: 148 pgs: 1 active+recovering+undersized+remapped, 2 active+recovering+forced_recovery+undersized+remapped, 145 active+clean; 6.2 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 54 KiB/s rd, 66 MiB/s wr, 1.14k op/s; 3052/303552 objects misplaced (1.005%); 13 MiB/s, 209 objects/s recovering 2024-09-20T19:00:00.296 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:00 smithi083 ceph-mon[29421]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T19:00:00.296 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:00 smithi083 ceph-mon[29421]: Cluster is now healthy 2024-09-20T19:00:00.296 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:00 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:00:00.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:00 smithi083 ceph-mon[29421]: osdmap e276: 8 total, 8 up, 8 in 2024-09-20T19:00:00.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:00 smithi083 ceph-mon[29421]: 6.17 scrub starts 2024-09-20T19:00:00.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:00 smithi083 ceph-mon[29421]: 6.17 scrub ok 2024-09-20T19:00:00.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:00 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2224809861' entity='client.admin' cmd={"prefix": "osd pg-upmap", "pgid": "3.c", "id": [2, 7, 6]} : dispatch 2024-09-20T19:00:00.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:00 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pg-upmap", "pgid": "3.c", "id": [2, 7, 6]} : dispatch 2024-09-20T19:00:00.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:00 smithi083 ceph-mon[29421]: overall HEALTH_OK 2024-09-20T19:00:00.297 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:00 smithi083 ceph-mon[36260]: pgmap v496: 148 pgs: 1 active+recovering+undersized+remapped, 2 active+recovering+forced_recovery+undersized+remapped, 145 active+clean; 6.2 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 54 KiB/s rd, 66 MiB/s wr, 1.14k op/s; 3052/303552 objects misplaced (1.005%); 13 MiB/s, 209 objects/s recovering 2024-09-20T19:00:00.297 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:00 smithi083 ceph-mon[36260]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T19:00:00.297 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:00 smithi083 ceph-mon[36260]: Cluster is now healthy 2024-09-20T19:00:00.297 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:00 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:00:00.297 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:00 smithi083 ceph-mon[36260]: osdmap e276: 8 total, 8 up, 8 in 2024-09-20T19:00:00.297 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:00 smithi083 ceph-mon[36260]: 6.17 scrub starts 2024-09-20T19:00:00.297 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:00 smithi083 ceph-mon[36260]: 6.17 scrub ok 2024-09-20T19:00:00.298 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:00 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2224809861' entity='client.admin' cmd={"prefix": "osd pg-upmap", "pgid": "3.c", "id": [2, 7, 6]} : dispatch 2024-09-20T19:00:00.298 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:00 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pg-upmap", "pgid": "3.c", "id": [2, 7, 6]} : dispatch 2024-09-20T19:00:00.298 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:00 smithi083 ceph-mon[36260]: overall HEALTH_OK 2024-09-20T19:00:00.309 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:00.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:00.311 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:00:00.411 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:00.441 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:00.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:00.443 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:00.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:00 smithi142 ceph-mon[32508]: pgmap v496: 148 pgs: 1 active+recovering+undersized+remapped, 2 active+recovering+forced_recovery+undersized+remapped, 145 active+clean; 6.2 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 54 KiB/s rd, 66 MiB/s wr, 1.14k op/s; 3052/303552 objects misplaced (1.005%); 13 MiB/s, 209 objects/s recovering 2024-09-20T19:00:00.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:00 smithi142 ceph-mon[32508]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T19:00:00.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:00 smithi142 ceph-mon[32508]: Cluster is now healthy 2024-09-20T19:00:00.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:00 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:00:00.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:00 smithi142 ceph-mon[32508]: osdmap e276: 8 total, 8 up, 8 in 2024-09-20T19:00:00.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:00 smithi142 ceph-mon[32508]: 6.17 scrub starts 2024-09-20T19:00:00.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:00 smithi142 ceph-mon[32508]: 6.17 scrub ok 2024-09-20T19:00:00.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:00 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2224809861' entity='client.admin' cmd={"prefix": "osd pg-upmap", "pgid": "3.c", "id": [2, 7, 6]} : dispatch 2024-09-20T19:00:00.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:00 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pg-upmap", "pgid": "3.c", "id": [2, 7, 6]} : dispatch 2024-09-20T19:00:00.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:00 smithi142 ceph-mon[32508]: overall HEALTH_OK 2024-09-20T19:00:00.543 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:00:00.580 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:00.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:00.581 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:00:00.681 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:00:00.720 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:00.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:00.721 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:00:00.796 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:00:00 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:00:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:00:00.822 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:00.855 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:00.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:00.856 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:00.956 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:01.001 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:01.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:01.001 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:01.049 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_historic_ops 2024-09-20T19:00:01.102 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:01.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:01.152 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:01.152 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:01.253 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:01.257 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 81 16 105467 105451 81.3542 69 0.00801181 0.0122865 2024-09-20T19:00:01.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:01.294 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:01.294 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:01.395 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:01.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:01.442 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:01.442 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:01.475 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:00:01.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:01 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "3.c", "id": [2, 7, 6]}]': finished 2024-09-20T19:00:01.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:01 smithi142 ceph-mon[32508]: osdmap e277: 8 total, 8 up, 8 in 2024-09-20T19:00:01.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:01 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:00:01.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:01 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:00:01.543 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:00:01.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:01 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "3.c", "id": [2, 7, 6]}]': finished 2024-09-20T19:00:01.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:01 smithi083 ceph-mon[29421]: osdmap e277: 8 total, 8 up, 8 in 2024-09-20T19:00:01.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:01 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:00:01.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:01 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:00:01.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:01 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "3.c", "id": [2, 7, 6]}]': finished 2024-09-20T19:00:01.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:01 smithi083 ceph-mon[36260]: osdmap e277: 8 total, 8 up, 8 in 2024-09-20T19:00:01.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:01 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:00:01.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:01 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:00:01.585 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:01.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:01.587 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:00:01.687 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:01.749 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:01.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:01.750 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:01.850 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:00:01.894 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:01.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:01.895 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:00:01.996 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:02.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:02.058 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:02.058 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:02.067 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:00:02.069 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T19:00:02.069 INFO:teuthology.orchestra.run.smithi142.stdout: "size": 20, 2024-09-20T19:00:02.069 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 600, 2024-09-20T19:00:02.069 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-09-20T19:00:02.069 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.069 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:25342 7.5 7:ae6f0d57:::benchmark_data_smithi083_70643_object25341:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T19:00:02.069 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:51:07.269377+0000", 2024-09-20T19:00:02.069 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 534.78794696499995, 2024-09-20T19:00:02.070 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.17285128899999999, 2024-09-20T19:00:02.070 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:00:02.070 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:00:02.070 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:02.070 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:00:02.070 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T19:00:02.070 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T19:00:02.070 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 25342 2024-09-20T19:00:02.070 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.070 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:00:02.070 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.070 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:00:02.070 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.269377+0000", 2024-09-20T19:00:02.070 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:00:02.071 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.071 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.071 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:00:02.071 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.269377+0000", 2024-09-20T19:00:02.071 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:00:02.071 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.071 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.071 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:00:02.071 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.269378+0000", 2024-09-20T19:00:02.071 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.615e-06 2024-09-20T19:00:02.071 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.071 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.071 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:00:02.071 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.269401+0000", 2024-09-20T19:00:02.071 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3159e-05 2024-09-20T19:00:02.072 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.072 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.072 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:00:02.072 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.269417+0000", 2024-09-20T19:00:02.072 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5760000000000002e-05 2024-09-20T19:00:02.072 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.072 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.072 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:00:02.072 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.269425+0000", 2024-09-20T19:00:02.072 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.6190000000000002e-06 2024-09-20T19:00:02.072 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.072 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.072 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:00:02.072 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.269487+0000", 2024-09-20T19:00:02.072 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2053999999999999e-05 2024-09-20T19:00:02.073 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.073 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.073 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:00:02.073 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.269633+0000", 2024-09-20T19:00:02.073 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00014581899999999999 2024-09-20T19:00:02.073 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.073 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.073 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,2", 2024-09-20T19:00:02.073 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.269777+0000", 2024-09-20T19:00:02.073 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00014390599999999999 2024-09-20T19:00:02.073 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.073 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.073 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:00:02.073 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.270815+0000", 2024-09-20T19:00:02.073 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0010380400000000001 2024-09-20T19:00:02.074 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.074 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.074 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:02.074 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.272683+0000", 2024-09-20T19:00:02.074 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0018686810000000001 2024-09-20T19:00:02.074 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.074 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.074 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:02.074 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.442115+0000", 2024-09-20T19:00:02.074 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.16943219800000001 2024-09-20T19:00:02.074 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.074 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.074 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:00:02.074 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.442185+0000", 2024-09-20T19:00:02.074 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.9690999999999994e-05 2024-09-20T19:00:02.075 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.075 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.075 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:00:02.075 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:51:07.442228+0000", 2024-09-20T19:00:02.075 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.2747000000000003e-05 2024-09-20T19:00:02.075 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:00:02.075 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:00:02.075 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:00:02.075 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.075 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.075 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:139222 7.5 7:a5a82ccf:::benchmark_data_smithi083_70643_object16420:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-20T19:00:02.075 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:42.806570+0000", 2024-09-20T19:00:02.075 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 439.25075403800003, 2024-09-20T19:00:02.075 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.12715717200000001, 2024-09-20T19:00:02.075 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:00:02.076 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:00:02.076 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:02.076 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:00:02.076 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T19:00:02.076 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T19:00:02.076 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 139222 2024-09-20T19:00:02.076 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.076 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:00:02.076 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.076 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:00:02.076 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806570+0000", 2024-09-20T19:00:02.076 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:00:02.076 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.076 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.076 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:00:02.077 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806570+0000", 2024-09-20T19:00:02.077 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:00:02.077 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.077 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.077 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:00:02.077 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806571+0000", 2024-09-20T19:00:02.077 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.621e-06 2024-09-20T19:00:02.077 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.077 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.077 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:00:02.077 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806576+0000", 2024-09-20T19:00:02.077 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3449999999999999e-06 2024-09-20T19:00:02.077 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.077 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.077 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:00:02.078 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806587+0000", 2024-09-20T19:00:02.078 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1375e-05 2024-09-20T19:00:02.078 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.078 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.078 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:00:02.078 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806595+0000", 2024-09-20T19:00:02.078 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.7449999999999995e-06 2024-09-20T19:00:02.078 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.078 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.078 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:00:02.078 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806658+0000", 2024-09-20T19:00:02.078 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.3601000000000003e-05 2024-09-20T19:00:02.078 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.078 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.078 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:00:02.079 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806734+0000", 2024-09-20T19:00:02.079 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5519000000000007e-05 2024-09-20T19:00:02.079 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.079 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.079 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,2", 2024-09-20T19:00:02.079 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806809+0000", 2024-09-20T19:00:02.079 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5587000000000007e-05 2024-09-20T19:00:02.079 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.079 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.079 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:00:02.079 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.810258+0000", 2024-09-20T19:00:02.079 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0034485219999999999 2024-09-20T19:00:02.079 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.079 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.080 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:02.080 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.810281+0000", 2024-09-20T19:00:02.080 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2999000000000001e-05 2024-09-20T19:00:02.080 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.080 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.080 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:02.080 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.933655+0000", 2024-09-20T19:00:02.080 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.12337419299999999 2024-09-20T19:00:02.080 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.080 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.080 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:00:02.080 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.933685+0000", 2024-09-20T19:00:02.080 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.0219e-05 2024-09-20T19:00:02.080 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.080 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.081 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:00:02.081 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.933727+0000", 2024-09-20T19:00:02.081 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.1446000000000003e-05 2024-09-20T19:00:02.081 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:00:02.081 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:00:02.081 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:00:02.081 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.081 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.081 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:139223 7.5 7:ae1b4edf:::benchmark_data_smithi083_70643_object16421:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-20T19:00:02.081 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:42.806736+0000", 2024-09-20T19:00:02.081 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 439.25058732799999, 2024-09-20T19:00:02.081 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.127118553, 2024-09-20T19:00:02.081 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:00:02.081 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:00:02.082 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:02.082 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:00:02.082 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T19:00:02.082 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T19:00:02.082 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 139223 2024-09-20T19:00:02.082 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.082 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:00:02.083 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.083 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:00:02.083 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806736+0000", 2024-09-20T19:00:02.083 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:00:02.083 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.083 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.083 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:00:02.083 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806736+0000", 2024-09-20T19:00:02.083 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:00:02.083 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.083 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.083 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:00:02.083 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806737+0000", 2024-09-20T19:00:02.083 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0529999999999999e-06 2024-09-20T19:00:02.084 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.084 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.084 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:00:02.084 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806742+0000", 2024-09-20T19:00:02.084 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3749999999999996e-06 2024-09-20T19:00:02.084 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.084 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.084 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:00:02.084 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806752+0000", 2024-09-20T19:00:02.084 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0730999999999999e-05 2024-09-20T19:00:02.084 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.084 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.084 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:00:02.084 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806760+0000", 2024-09-20T19:00:02.085 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.4569999999999999e-06 2024-09-20T19:00:02.085 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.085 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.085 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:00:02.085 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.808698+0000", 2024-09-20T19:00:02.085 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0019381380000000001 2024-09-20T19:00:02.085 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.085 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.085 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:00:02.085 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.808770+0000", 2024-09-20T19:00:02.085 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.1746000000000001e-05 2024-09-20T19:00:02.085 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.085 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.085 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,2", 2024-09-20T19:00:02.085 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.808835+0000", 2024-09-20T19:00:02.086 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4832999999999994e-05 2024-09-20T19:00:02.086 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.086 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.086 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:02.086 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.810390+0000", 2024-09-20T19:00:02.086 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001555463 2024-09-20T19:00:02.086 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.086 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.086 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:00:02.086 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.810400+0000", 2024-09-20T19:00:02.086 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.8239999999999995e-06 2024-09-20T19:00:02.086 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.086 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.086 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:02.086 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.933775+0000", 2024-09-20T19:00:02.087 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.12337550799999999 2024-09-20T19:00:02.087 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.087 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.087 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:00:02.087 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.933814+0000", 2024-09-20T19:00:02.087 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.8874000000000001e-05 2024-09-20T19:00:02.087 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.087 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.087 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:00:02.087 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.933855+0000", 2024-09-20T19:00:02.087 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0550999999999999e-05 2024-09-20T19:00:02.087 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:00:02.087 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:00:02.087 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:00:02.088 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.088 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.088 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.24955.0:101455 8.8 e190/160 8:1ef316d4:::benchmark_data_smithi083_97422_object101454:head v 190'6357, mlcod=190'6355)", 2024-09-20T19:00:02.088 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:55:52.357548+0000", 2024-09-20T19:00:02.088 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 249.699775335, 2024-09-20T19:00:02.088 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.15217040600000001, 2024-09-20T19:00:02.088 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:00:02.088 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:00:02.088 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:02.088 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:00:02.088 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.088 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:00:02.088 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.357548+0000", 2024-09-20T19:00:02.088 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:00:02.089 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.089 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.089 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:00:02.089 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.357548+0000", 2024-09-20T19:00:02.089 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:00:02.089 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.089 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.089 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:00:02.089 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.357549+0000", 2024-09-20T19:00:02.089 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.0800000000000004e-07 2024-09-20T19:00:02.089 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.089 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.089 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:00:02.089 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.357594+0000", 2024-09-20T19:00:02.089 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.5052999999999997e-05 2024-09-20T19:00:02.090 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.090 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.090 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:00:02.090 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.357603+0000", 2024-09-20T19:00:02.090 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.0999999999999993e-06 2024-09-20T19:00:02.090 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.090 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.090 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:00:02.090 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.357610+0000", 2024-09-20T19:00:02.090 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5459999999999997e-06 2024-09-20T19:00:02.090 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.090 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.090 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:00:02.090 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.357656+0000", 2024-09-20T19:00:02.090 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.6118999999999997e-05 2024-09-20T19:00:02.091 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.091 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.091 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:00:02.091 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.357668+0000", 2024-09-20T19:00:02.091 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2367e-05 2024-09-20T19:00:02.091 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.091 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.091 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:00:02.091 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.360245+0000", 2024-09-20T19:00:02.091 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0025771190000000001 2024-09-20T19:00:02.091 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.091 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.091 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:00:02.091 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.509719+0000", 2024-09-20T19:00:02.091 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.14947329400000001 2024-09-20T19:00:02.092 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:00:02.092 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:00:02.092 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:00:02.092 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.092 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.092 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.24955.0:101453 8.6 e190/182 8:61900b1b:::benchmark_data_smithi083_97422_object101452:head v 190'6308, mlcod=0'0)", 2024-09-20T19:00:02.092 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:55:52.360511+0000", 2024-09-20T19:00:02.092 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 249.69681227500001, 2024-09-20T19:00:02.092 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.15620442300000001, 2024-09-20T19:00:02.092 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:00:02.092 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:00:02.092 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:02.092 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:00:02.092 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.093 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:00:02.093 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.360511+0000", 2024-09-20T19:00:02.093 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:00:02.093 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.093 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.093 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:00:02.093 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.360511+0000", 2024-09-20T19:00:02.093 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:00:02.093 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.093 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.093 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:00:02.093 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.360512+0000", 2024-09-20T19:00:02.093 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.0800000000000003e-07 2024-09-20T19:00:02.093 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.093 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.094 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:00:02.094 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.360571+0000", 2024-09-20T19:00:02.094 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8850000000000001e-05 2024-09-20T19:00:02.094 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.094 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.094 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:00:02.094 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.509692+0000", 2024-09-20T19:00:02.094 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.14912044099999999 2024-09-20T19:00:02.094 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.094 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.094 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:00:02.094 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.509707+0000", 2024-09-20T19:00:02.094 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5568000000000001e-05 2024-09-20T19:00:02.094 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.095 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.095 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:00:02.095 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.509813+0000", 2024-09-20T19:00:02.095 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00010567400000000001 2024-09-20T19:00:02.095 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.095 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.095 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:00:02.095 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.509836+0000", 2024-09-20T19:00:02.095 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2853999999999998e-05 2024-09-20T19:00:02.095 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.095 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.095 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:00:02.095 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.516559+0000", 2024-09-20T19:00:02.095 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.006723807 2024-09-20T19:00:02.096 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.096 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.096 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:00:02.096 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.516716+0000", 2024-09-20T19:00:02.096 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015632100000000001 2024-09-20T19:00:02.096 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:00:02.096 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:00:02.096 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:00:02.096 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.096 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.096 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.24955.0:101471 8.5 e190/182 8:a0258bcb:::benchmark_data_smithi083_97422_object101470:head v 190'6528, mlcod=0'0)", 2024-09-20T19:00:02.096 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:55:52.379819+0000", 2024-09-20T19:00:02.096 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 249.67750469500001, 2024-09-20T19:00:02.096 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13654260100000001, 2024-09-20T19:00:02.097 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:00:02.097 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:00:02.097 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:02.097 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:00:02.097 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.097 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:00:02.097 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.379819+0000", 2024-09-20T19:00:02.097 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:00:02.097 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.097 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.097 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:00:02.097 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.379819+0000", 2024-09-20T19:00:02.097 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:00:02.097 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.098 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.098 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:00:02.098 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.379820+0000", 2024-09-20T19:00:02.098 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5519999999999999e-06 2024-09-20T19:00:02.098 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.098 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.098 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:00:02.098 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.379854+0000", 2024-09-20T19:00:02.098 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.3157000000000001e-05 2024-09-20T19:00:02.098 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.098 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.098 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:00:02.098 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.509694+0000", 2024-09-20T19:00:02.098 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.12983992999999999 2024-09-20T19:00:02.098 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.099 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.099 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:00:02.099 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.509706+0000", 2024-09-20T19:00:02.099 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2965e-05 2024-09-20T19:00:02.099 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.099 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.099 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:00:02.099 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.509768+0000", 2024-09-20T19:00:02.099 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.1013e-05 2024-09-20T19:00:02.099 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.099 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.099 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:00:02.099 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.509797+0000", 2024-09-20T19:00:02.099 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9916999999999998e-05 2024-09-20T19:00:02.100 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.100 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.100 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:00:02.100 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.516311+0000", 2024-09-20T19:00:02.100 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0065133980000000001 2024-09-20T19:00:02.100 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.100 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.100 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:00:02.100 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.516361+0000", 2024-09-20T19:00:02.100 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0668999999999999e-05 2024-09-20T19:00:02.100 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:00:02.100 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:00:02.100 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:00:02.100 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.100 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.100 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:72379 9.d 9:bae8f567:::benchmark_data_smithi083_124552_object72378:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e265)", 2024-09-20T19:00:02.101 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:26.629075+0000", 2024-09-20T19:00:02.101 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 35.428248584999999, 2024-09-20T19:00:02.101 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.133017778, 2024-09-20T19:00:02.101 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:00:02.101 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:00:02.101 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:02.101 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:00:02.101 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:00:02.101 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:00:02.101 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 72379 2024-09-20T19:00:02.101 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.101 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:00:02.101 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.101 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:00:02.102 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.629075+0000", 2024-09-20T19:00:02.102 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:00:02.102 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.102 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.102 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:00:02.102 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.629075+0000", 2024-09-20T19:00:02.102 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:00:02.102 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.102 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.102 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:00:02.102 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.629077+0000", 2024-09-20T19:00:02.102 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.753e-06 2024-09-20T19:00:02.102 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.102 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.102 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:00:02.103 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.629117+0000", 2024-09-20T19:00:02.103 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0463000000000002e-05 2024-09-20T19:00:02.103 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.103 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.103 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:00:02.103 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.629128+0000", 2024-09-20T19:00:02.103 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1252e-05 2024-09-20T19:00:02.103 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.103 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.103 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:00:02.103 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.629136+0000", 2024-09-20T19:00:02.103 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5850000000000002e-06 2024-09-20T19:00:02.103 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.103 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.103 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:00:02.104 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.629297+0000", 2024-09-20T19:00:02.104 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00016093100000000001 2024-09-20T19:00:02.104 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.104 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.104 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:00:02.104 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.629447+0000", 2024-09-20T19:00:02.104 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000149717 2024-09-20T19:00:02.104 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.104 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.104 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,5", 2024-09-20T19:00:02.104 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.629595+0000", 2024-09-20T19:00:02.104 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000148436 2024-09-20T19:00:02.104 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.104 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.104 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:02.105 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.638400+0000", 2024-09-20T19:00:02.105 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0088051139999999993 2024-09-20T19:00:02.105 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.105 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.105 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:00:02.105 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.638412+0000", 2024-09-20T19:00:02.105 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1804000000000001e-05 2024-09-20T19:00:02.105 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.105 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.105 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:02.105 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.761963+0000", 2024-09-20T19:00:02.105 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.123550901 2024-09-20T19:00:02.105 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.105 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.105 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:00:02.106 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.762026+0000", 2024-09-20T19:00:02.106 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.3350999999999997e-05 2024-09-20T19:00:02.106 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.106 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.106 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:00:02.106 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.762093+0000", 2024-09-20T19:00:02.106 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.6470999999999994e-05 2024-09-20T19:00:02.106 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:00:02.106 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:00:02.106 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:00:02.106 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.106 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.106 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:72382 9.d 9:b4fdabfe:::benchmark_data_smithi083_124552_object72381:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e265)", 2024-09-20T19:00:02.106 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:26.629907+0000", 2024-09-20T19:00:02.106 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 35.427417019000004, 2024-09-20T19:00:02.107 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13337285700000001, 2024-09-20T19:00:02.107 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:00:02.107 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:00:02.107 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:02.107 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:00:02.107 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:00:02.107 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:00:02.107 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 72382 2024-09-20T19:00:02.107 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.107 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:00:02.107 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.107 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:00:02.107 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.629907+0000", 2024-09-20T19:00:02.107 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:00:02.108 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.108 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.108 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:00:02.108 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.629907+0000", 2024-09-20T19:00:02.108 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:00:02.108 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.108 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.108 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:00:02.108 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.629908+0000", 2024-09-20T19:00:02.108 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.206e-06 2024-09-20T19:00:02.108 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.108 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.108 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:00:02.108 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.629934+0000", 2024-09-20T19:00:02.108 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.6463000000000001e-05 2024-09-20T19:00:02.109 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.109 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.109 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:00:02.109 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.629946+0000", 2024-09-20T19:00:02.109 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1579e-05 2024-09-20T19:00:02.110 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.110 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.110 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:00:02.110 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.629953+0000", 2024-09-20T19:00:02.110 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.1810000000000004e-06 2024-09-20T19:00:02.110 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.110 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.110 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:00:02.110 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.632103+0000", 2024-09-20T19:00:02.110 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.002150365 2024-09-20T19:00:02.110 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.110 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.110 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:00:02.110 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.632219+0000", 2024-09-20T19:00:02.111 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000115746 2024-09-20T19:00:02.111 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.111 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.111 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,5", 2024-09-20T19:00:02.111 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.632337+0000", 2024-09-20T19:00:02.111 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011818499999999999 2024-09-20T19:00:02.111 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.111 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.111 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:02.111 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.638442+0000", 2024-09-20T19:00:02.111 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0061042379999999997 2024-09-20T19:00:02.111 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.111 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.111 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:00:02.112 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.638466+0000", 2024-09-20T19:00:02.112 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.4266000000000001e-05 2024-09-20T19:00:02.112 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.112 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.112 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:02.112 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.763189+0000", 2024-09-20T19:00:02.112 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.124723375 2024-09-20T19:00:02.112 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.112 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.112 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:00:02.112 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.763240+0000", 2024-09-20T19:00:02.112 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0708999999999998e-05 2024-09-20T19:00:02.112 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.112 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.112 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:00:02.113 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.763279+0000", 2024-09-20T19:00:02.113 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9544e-05 2024-09-20T19:00:02.113 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:00:02.113 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:00:02.113 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:00:02.113 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.113 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.113 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:72386 9.3 9:c310ff33:::benchmark_data_smithi083_124552_object72385:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e265)", 2024-09-20T19:00:02.113 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:26.634291+0000", 2024-09-20T19:00:02.113 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 35.423032132000003, 2024-09-20T19:00:02.113 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.133404362, 2024-09-20T19:00:02.113 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:00:02.113 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:00:02.113 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:02.113 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:00:02.114 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:00:02.114 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:00:02.114 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 72386 2024-09-20T19:00:02.114 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.114 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:00:02.114 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.114 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:00:02.114 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.634291+0000", 2024-09-20T19:00:02.114 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:00:02.114 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.114 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.114 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:00:02.114 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.634291+0000", 2024-09-20T19:00:02.114 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:00:02.114 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.115 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.115 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:00:02.115 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.634293+0000", 2024-09-20T19:00:02.115 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4810000000000001e-06 2024-09-20T19:00:02.115 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.115 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.115 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:00:02.115 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.634352+0000", 2024-09-20T19:00:02.115 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9089000000000003e-05 2024-09-20T19:00:02.115 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.115 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.115 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:00:02.115 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.634363+0000", 2024-09-20T19:00:02.115 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1100999999999999e-05 2024-09-20T19:00:02.115 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.116 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.116 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:00:02.116 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.634371+0000", 2024-09-20T19:00:02.116 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.0490000000000004e-06 2024-09-20T19:00:02.116 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.116 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.116 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:00:02.116 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.634434+0000", 2024-09-20T19:00:02.116 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2587000000000002e-05 2024-09-20T19:00:02.116 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.116 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.116 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:00:02.116 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.634569+0000", 2024-09-20T19:00:02.116 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013524699999999999 2024-09-20T19:00:02.116 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.117 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.117 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,3", 2024-09-20T19:00:02.117 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.634714+0000", 2024-09-20T19:00:02.117 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00014523699999999999 2024-09-20T19:00:02.117 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.117 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.117 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:00:02.117 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.639963+0000", 2024-09-20T19:00:02.117 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0052484699999999999 2024-09-20T19:00:02.117 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.117 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.117 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:02.117 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.640003+0000", 2024-09-20T19:00:02.117 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0185999999999998e-05 2024-09-20T19:00:02.117 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.118 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.118 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:02.118 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.767600+0000", 2024-09-20T19:00:02.118 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.127597345 2024-09-20T19:00:02.118 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.118 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.118 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:00:02.118 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.767655+0000", 2024-09-20T19:00:02.118 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5003999999999997e-05 2024-09-20T19:00:02.118 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.118 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.118 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:00:02.118 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.767696+0000", 2024-09-20T19:00:02.118 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0565999999999998e-05 2024-09-20T19:00:02.119 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:00:02.119 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:00:02.119 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:00:02.119 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.119 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.119 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:72387 9.e 9:76fd9fac:::benchmark_data_smithi083_124552_object72386:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e265)", 2024-09-20T19:00:02.119 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:26.635171+0000", 2024-09-20T19:00:02.119 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 35.422152601999997, 2024-09-20T19:00:02.119 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.12958456300000001, 2024-09-20T19:00:02.119 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:00:02.119 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:00:02.119 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:02.119 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:00:02.119 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:00:02.120 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:00:02.120 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 72387 2024-09-20T19:00:02.120 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.120 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:00:02.120 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.120 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:00:02.120 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.635171+0000", 2024-09-20T19:00:02.120 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:00:02.120 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.120 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.120 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:00:02.120 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.635171+0000", 2024-09-20T19:00:02.120 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:00:02.120 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.120 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.121 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:00:02.121 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.635172+0000", 2024-09-20T19:00:02.121 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.296e-06 2024-09-20T19:00:02.121 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.121 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.121 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:00:02.121 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.635231+0000", 2024-09-20T19:00:02.121 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9156000000000001e-05 2024-09-20T19:00:02.121 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.121 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.121 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:00:02.121 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.635247+0000", 2024-09-20T19:00:02.121 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5506999999999999e-05 2024-09-20T19:00:02.121 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.121 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.122 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:00:02.122 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.635254+0000", 2024-09-20T19:00:02.122 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.4250000000000001e-06 2024-09-20T19:00:02.122 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.122 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.122 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:00:02.122 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.635351+0000", 2024-09-20T19:00:02.122 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.6688999999999995e-05 2024-09-20T19:00:02.122 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.122 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.122 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:00:02.122 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.635469+0000", 2024-09-20T19:00:02.122 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000118397 2024-09-20T19:00:02.123 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.123 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.123 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,4", 2024-09-20T19:00:02.123 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.635588+0000", 2024-09-20T19:00:02.123 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000118047 2024-09-20T19:00:02.123 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.123 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.123 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:00:02.123 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.639082+0000", 2024-09-20T19:00:02.123 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.003494128 2024-09-20T19:00:02.123 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.123 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.123 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:02.123 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.639796+0000", 2024-09-20T19:00:02.123 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00071393900000000002 2024-09-20T19:00:02.124 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.124 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.124 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:02.124 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.764649+0000", 2024-09-20T19:00:02.124 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.124853776 2024-09-20T19:00:02.124 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.124 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.124 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:00:02.124 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.764711+0000", 2024-09-20T19:00:02.124 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.1246000000000003e-05 2024-09-20T19:00:02.124 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.124 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.124 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:00:02.124 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.764756+0000", 2024-09-20T19:00:02.125 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.4956999999999997e-05 2024-09-20T19:00:02.125 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:00:02.125 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:00:02.125 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:00:02.125 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.125 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.125 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:72388 9.3 9:c69fcb53:::benchmark_data_smithi083_124552_object72387:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e265)", 2024-09-20T19:00:02.125 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:26.635542+0000", 2024-09-20T19:00:02.125 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 35.421782008999998, 2024-09-20T19:00:02.125 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.132329168, 2024-09-20T19:00:02.125 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:00:02.125 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:00:02.125 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:02.125 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:00:02.125 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:00:02.126 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:00:02.126 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 72388 2024-09-20T19:00:02.126 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.126 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:00:02.126 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.126 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:00:02.126 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.635542+0000", 2024-09-20T19:00:02.126 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:00:02.126 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.126 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.126 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:00:02.126 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.635542+0000", 2024-09-20T19:00:02.126 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:00:02.126 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.126 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.127 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:00:02.127 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.635542+0000", 2024-09-20T19:00:02.127 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.7899999999999998e-07 2024-09-20T19:00:02.127 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.127 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.127 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:00:02.127 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.635617+0000", 2024-09-20T19:00:02.127 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.4859999999999998e-05 2024-09-20T19:00:02.127 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.127 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.127 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:00:02.127 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.635625+0000", 2024-09-20T19:00:02.127 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5979999999999999e-06 2024-09-20T19:00:02.127 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.127 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.128 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:00:02.128 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.635631+0000", 2024-09-20T19:00:02.128 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8599999999999998e-06 2024-09-20T19:00:02.128 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.128 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.128 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:00:02.128 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.637372+0000", 2024-09-20T19:00:02.128 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001741352 2024-09-20T19:00:02.128 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.128 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.128 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:00:02.128 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.637487+0000", 2024-09-20T19:00:02.128 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000115296 2024-09-20T19:00:02.128 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.128 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.129 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,3", 2024-09-20T19:00:02.129 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.637613+0000", 2024-09-20T19:00:02.129 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012625200000000001 2024-09-20T19:00:02.129 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.129 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.129 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:00:02.129 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.640092+0000", 2024-09-20T19:00:02.129 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.002478975 2024-09-20T19:00:02.129 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.129 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.129 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:02.129 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.640178+0000", 2024-09-20T19:00:02.129 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.5749999999999997e-05 2024-09-20T19:00:02.129 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.130 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.130 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:02.130 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.767762+0000", 2024-09-20T19:00:02.130 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.127583327 2024-09-20T19:00:02.130 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.130 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.130 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:00:02.130 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.767818+0000", 2024-09-20T19:00:02.130 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6589999999999999e-05 2024-09-20T19:00:02.130 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.130 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.130 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:00:02.130 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.767871+0000", 2024-09-20T19:00:02.130 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.2629e-05 2024-09-20T19:00:02.130 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:00:02.131 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:00:02.131 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:00:02.131 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.131 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.131 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:72392 9.e 9:7b127873:::benchmark_data_smithi083_124552_object72391:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e265)", 2024-09-20T19:00:02.131 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:26.643135+0000", 2024-09-20T19:00:02.131 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 35.414188799000001, 2024-09-20T19:00:02.131 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.121729718, 2024-09-20T19:00:02.131 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:00:02.131 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:00:02.131 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:02.131 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:00:02.131 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:00:02.131 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:00:02.132 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 72392 2024-09-20T19:00:02.132 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.132 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:00:02.132 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.132 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:00:02.132 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.643135+0000", 2024-09-20T19:00:02.132 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:00:02.132 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.132 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.132 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:00:02.132 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.643135+0000", 2024-09-20T19:00:02.132 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:00:02.132 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.132 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.133 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:00:02.133 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.643145+0000", 2024-09-20T19:00:02.133 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.06e-05 2024-09-20T19:00:02.133 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.133 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.133 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:00:02.133 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.643178+0000", 2024-09-20T19:00:02.133 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.2843999999999999e-05 2024-09-20T19:00:02.133 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.133 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.133 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:00:02.133 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.643191+0000", 2024-09-20T19:00:02.133 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2669e-05 2024-09-20T19:00:02.133 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.133 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.134 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:00:02.134 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.643199+0000", 2024-09-20T19:00:02.134 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.6459999999999993e-06 2024-09-20T19:00:02.134 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.134 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.134 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:00:02.134 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.643288+0000", 2024-09-20T19:00:02.134 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.9757000000000001e-05 2024-09-20T19:00:02.134 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.134 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.134 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:00:02.134 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.643406+0000", 2024-09-20T19:00:02.134 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011794399999999999 2024-09-20T19:00:02.134 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.135 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.135 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,4", 2024-09-20T19:00:02.135 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.643499+0000", 2024-09-20T19:00:02.135 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.3202999999999995e-05 2024-09-20T19:00:02.135 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.135 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.135 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:00:02.135 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.645839+0000", 2024-09-20T19:00:02.135 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0023393089999999999 2024-09-20T19:00:02.135 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.135 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.135 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:02.135 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.646251+0000", 2024-09-20T19:00:02.135 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000411808 2024-09-20T19:00:02.135 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.136 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.136 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:02.136 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.764797+0000", 2024-09-20T19:00:02.136 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.118546278 2024-09-20T19:00:02.136 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.136 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.136 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:00:02.136 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.764829+0000", 2024-09-20T19:00:02.136 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.2339999999999999e-05 2024-09-20T19:00:02.136 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.136 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.136 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:00:02.136 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.764865+0000", 2024-09-20T19:00:02.136 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.5320000000000001e-05 2024-09-20T19:00:02.136 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:00:02.137 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:00:02.137 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:00:02.137 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.137 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.137 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:72396 9.3 9:c1995335:::benchmark_data_smithi083_124552_object72395:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e265)", 2024-09-20T19:00:02.137 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:26.646793+0000", 2024-09-20T19:00:02.137 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 35.410530113, 2024-09-20T19:00:02.137 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.12121157, 2024-09-20T19:00:02.137 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:00:02.137 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:00:02.137 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:02.137 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:00:02.137 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:00:02.137 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:00:02.137 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 72396 2024-09-20T19:00:02.137 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.138 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:00:02.138 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.138 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:00:02.138 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.646793+0000", 2024-09-20T19:00:02.138 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:00:02.138 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.138 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.138 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:00:02.138 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.646793+0000", 2024-09-20T19:00:02.138 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:00:02.138 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.138 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.138 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:00:02.138 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.646795+0000", 2024-09-20T19:00:02.139 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.265e-06 2024-09-20T19:00:02.139 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.139 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.139 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:00:02.139 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.646851+0000", 2024-09-20T19:00:02.139 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6369000000000002e-05 2024-09-20T19:00:02.139 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.139 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.139 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:00:02.139 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.646860+0000", 2024-09-20T19:00:02.139 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.5180000000000004e-06 2024-09-20T19:00:02.139 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.139 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.139 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:00:02.139 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.646864+0000", 2024-09-20T19:00:02.140 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.527e-06 2024-09-20T19:00:02.140 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.140 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.140 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:00:02.140 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.646913+0000", 2024-09-20T19:00:02.140 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.9141000000000003e-05 2024-09-20T19:00:02.140 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.140 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.140 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:00:02.140 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.647010+0000", 2024-09-20T19:00:02.140 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.7188999999999994e-05 2024-09-20T19:00:02.140 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.140 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.140 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,3", 2024-09-20T19:00:02.141 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.647114+0000", 2024-09-20T19:00:02.141 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000103145 2024-09-20T19:00:02.141 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.141 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.141 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:00:02.141 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.648976+0000", 2024-09-20T19:00:02.141 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001861927 2024-09-20T19:00:02.141 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.141 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.141 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:02.141 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.649632+0000", 2024-09-20T19:00:02.141 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00065602900000000003 2024-09-20T19:00:02.141 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.141 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.142 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:02.142 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.767917+0000", 2024-09-20T19:00:02.142 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.11828525099999999 2024-09-20T19:00:02.142 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.142 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.142 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:00:02.142 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.767970+0000", 2024-09-20T19:00:02.142 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3115000000000002e-05 2024-09-20T19:00:02.142 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.142 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.142 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:00:02.142 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.768005+0000", 2024-09-20T19:00:02.142 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.5094e-05 2024-09-20T19:00:02.142 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:00:02.143 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:00:02.143 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:00:02.143 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.143 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.143 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:77043 9.d 9:b7b318c0:::benchmark_data_smithi083_124552_object77042:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e267)", 2024-09-20T19:00:02.143 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:31.323324+0000", 2024-09-20T19:00:02.143 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 30.733999656000002, 2024-09-20T19:00:02.143 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.127625708, 2024-09-20T19:00:02.143 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:00:02.143 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:00:02.143 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:02.143 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:00:02.143 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:00:02.143 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:00:02.143 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 77043 2024-09-20T19:00:02.144 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.144 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:00:02.144 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.144 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:00:02.144 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:31.323324+0000", 2024-09-20T19:00:02.144 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:00:02.144 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.144 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.144 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:00:02.144 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:31.323324+0000", 2024-09-20T19:00:02.144 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:00:02.144 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.144 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.144 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:00:02.145 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:31.323325+0000", 2024-09-20T19:00:02.145 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9599999999999999e-07 2024-09-20T19:00:02.145 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.145 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.145 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:00:02.145 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:31.323385+0000", 2024-09-20T19:00:02.145 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0080000000000001e-05 2024-09-20T19:00:02.145 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.145 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.145 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:00:02.145 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:31.323391+0000", 2024-09-20T19:00:02.145 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.6329999999999999e-06 2024-09-20T19:00:02.145 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.145 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.145 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:00:02.146 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:31.323397+0000", 2024-09-20T19:00:02.146 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8259999999999998e-06 2024-09-20T19:00:02.146 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.146 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.146 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:00:02.146 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:31.326616+0000", 2024-09-20T19:00:02.146 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0032193080000000002 2024-09-20T19:00:02.146 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.146 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.146 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:00:02.146 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:31.326728+0000", 2024-09-20T19:00:02.146 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000111733 2024-09-20T19:00:02.146 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.146 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.147 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,5", 2024-09-20T19:00:02.147 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:31.326844+0000", 2024-09-20T19:00:02.147 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011613299999999999 2024-09-20T19:00:02.147 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.147 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.147 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:00:02.147 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:31.332778+0000", 2024-09-20T19:00:02.147 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0059334399999999999 2024-09-20T19:00:02.147 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.147 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.147 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:02.147 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:31.335596+0000", 2024-09-20T19:00:02.147 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0028181030000000002 2024-09-20T19:00:02.148 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.148 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.148 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:02.148 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:31.450842+0000", 2024-09-20T19:00:02.148 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.11524606900000001 2024-09-20T19:00:02.148 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.148 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.148 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:00:02.148 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:31.450902+0000", 2024-09-20T19:00:02.149 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0513999999999997e-05 2024-09-20T19:00:02.149 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.149 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.149 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:00:02.149 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:31.450950+0000", 2024-09-20T19:00:02.149 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7273e-05 2024-09-20T19:00:02.149 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:00:02.149 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:00:02.149 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:00:02.149 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.149 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.150 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:77047 9.d 9:b5f6c8b0:::benchmark_data_smithi083_124552_object77046:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e267)", 2024-09-20T19:00:02.150 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:31.326395+0000", 2024-09-20T19:00:02.150 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 30.730928387999999, 2024-09-20T19:00:02.150 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.12732231699999999, 2024-09-20T19:00:02.150 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:00:02.150 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:00:02.150 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:02.150 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:00:02.150 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:00:02.150 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:00:02.150 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 77047 2024-09-20T19:00:02.150 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.150 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:00:02.150 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.151 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:00:02.151 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:31.326395+0000", 2024-09-20T19:00:02.151 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:00:02.151 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.151 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.151 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:00:02.151 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:31.326395+0000", 2024-09-20T19:00:02.151 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:00:02.151 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.151 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.151 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:00:02.151 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:31.326397+0000", 2024-09-20T19:00:02.151 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5889999999999999e-06 2024-09-20T19:00:02.151 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.152 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.152 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:00:02.152 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:31.326471+0000", 2024-09-20T19:00:02.152 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.4289999999999995e-05 2024-09-20T19:00:02.152 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.152 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.152 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:00:02.152 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:31.326482+0000", 2024-09-20T19:00:02.152 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1178e-05 2024-09-20T19:00:02.152 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.152 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.152 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:00:02.152 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:31.326492+0000", 2024-09-20T19:00:02.152 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.499e-06 2024-09-20T19:00:02.153 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.153 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.153 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:00:02.153 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:31.329732+0000", 2024-09-20T19:00:02.153 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0032397509999999999 2024-09-20T19:00:02.153 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.153 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.153 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:00:02.153 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:31.329858+0000", 2024-09-20T19:00:02.153 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012643999999999999 2024-09-20T19:00:02.153 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.153 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.153 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,5", 2024-09-20T19:00:02.153 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:31.329992+0000", 2024-09-20T19:00:02.153 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013422799999999999 2024-09-20T19:00:02.154 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.154 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.154 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:00:02.154 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:31.332969+0000", 2024-09-20T19:00:02.154 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0029772879999999998 2024-09-20T19:00:02.154 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.154 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.154 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:02.154 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:31.335868+0000", 2024-09-20T19:00:02.154 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0028984050000000002 2024-09-20T19:00:02.154 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.154 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.154 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:02.154 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:31.453639+0000", 2024-09-20T19:00:02.154 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.117770694 2024-09-20T19:00:02.155 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.155 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.155 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:00:02.155 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:31.453674+0000", 2024-09-20T19:00:02.155 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.5043999999999999e-05 2024-09-20T19:00:02.155 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.155 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.155 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:00:02.155 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:31.453718+0000", 2024-09-20T19:00:02.155 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3911e-05 2024-09-20T19:00:02.155 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:00:02.155 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:00:02.155 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:00:02.155 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.156 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.156 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:77054 9.d 9:b5c9f22f:::benchmark_data_smithi083_124552_object77053:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e267)", 2024-09-20T19:00:02.156 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:31.333180+0000", 2024-09-20T19:00:02.156 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 30.724143983000001, 2024-09-20T19:00:02.156 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.120657519, 2024-09-20T19:00:02.156 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:00:02.156 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:00:02.156 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:02.156 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:00:02.156 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:00:02.156 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:00:02.156 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 77054 2024-09-20T19:00:02.156 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.156 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:00:02.156 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.157 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:00:02.157 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:31.333180+0000", 2024-09-20T19:00:02.157 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:00:02.157 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.157 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.157 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:00:02.157 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:31.333180+0000", 2024-09-20T19:00:02.157 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:00:02.157 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.157 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.157 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:00:02.157 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:31.333181+0000", 2024-09-20T19:00:02.157 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.389e-06 2024-09-20T19:00:02.157 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.157 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.158 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:00:02.158 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:31.333253+0000", 2024-09-20T19:00:02.158 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.1522999999999999e-05 2024-09-20T19:00:02.158 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.158 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.158 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:00:02.158 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:31.333264+0000", 2024-09-20T19:00:02.158 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1803e-05 2024-09-20T19:00:02.158 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.158 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.158 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:00:02.158 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:31.333272+0000", 2024-09-20T19:00:02.158 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.8930000000000005e-06 2024-09-20T19:00:02.158 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.158 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.159 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:00:02.159 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:31.333327+0000", 2024-09-20T19:00:02.159 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5163e-05 2024-09-20T19:00:02.159 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.159 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.159 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:00:02.159 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:31.333471+0000", 2024-09-20T19:00:02.159 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00014391799999999999 2024-09-20T19:00:02.159 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.159 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.159 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,5", 2024-09-20T19:00:02.159 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:31.333616+0000", 2024-09-20T19:00:02.159 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000144559 2024-09-20T19:00:02.159 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.160 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.160 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:00:02.160 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:31.335708+0000", 2024-09-20T19:00:02.160 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0020918320000000001 2024-09-20T19:00:02.160 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.160 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.160 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:02.160 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:31.335899+0000", 2024-09-20T19:00:02.160 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000191077 2024-09-20T19:00:02.160 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.160 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.160 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:02.160 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:31.453757+0000", 2024-09-20T19:00:02.160 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.11785815600000001 2024-09-20T19:00:02.160 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.161 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.161 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:00:02.161 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:31.453799+0000", 2024-09-20T19:00:02.161 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.1619000000000003e-05 2024-09-20T19:00:02.161 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.161 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.161 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:00:02.161 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:31.453837+0000", 2024-09-20T19:00:02.161 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.8587000000000002e-05 2024-09-20T19:00:02.161 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:00:02.161 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:00:02.161 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:00:02.161 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.161 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.161 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:84060 9.e 9:73c1c2b3:::benchmark_data_smithi083_124552_object84059:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e268)", 2024-09-20T19:00:02.162 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:38.588091+0000", 2024-09-20T19:00:02.162 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 23.469232892000001, 2024-09-20T19:00:02.162 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13688319800000001, 2024-09-20T19:00:02.162 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:00:02.162 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:00:02.162 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:02.162 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:00:02.162 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:00:02.162 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:00:02.162 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 84060 2024-09-20T19:00:02.162 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.162 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:00:02.163 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.163 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:00:02.163 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.588091+0000", 2024-09-20T19:00:02.163 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:00:02.163 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.163 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.163 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:00:02.163 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.588091+0000", 2024-09-20T19:00:02.163 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:00:02.163 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.163 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.163 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:00:02.163 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.588092+0000", 2024-09-20T19:00:02.163 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.694e-06 2024-09-20T19:00:02.163 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.164 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.164 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:00:02.164 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.588232+0000", 2024-09-20T19:00:02.164 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013927899999999999 2024-09-20T19:00:02.164 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.164 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.164 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:00:02.164 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.588247+0000", 2024-09-20T19:00:02.164 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5670000000000001e-05 2024-09-20T19:00:02.164 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.164 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.164 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:00:02.164 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.588255+0000", 2024-09-20T19:00:02.164 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.0630000000000006e-06 2024-09-20T19:00:02.165 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.165 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.165 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:00:02.165 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.588320+0000", 2024-09-20T19:00:02.165 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5049999999999996e-05 2024-09-20T19:00:02.165 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.165 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.165 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:00:02.165 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.588440+0000", 2024-09-20T19:00:02.165 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011967 2024-09-20T19:00:02.165 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.165 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.165 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,4", 2024-09-20T19:00:02.165 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.588596+0000", 2024-09-20T19:00:02.165 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015572200000000001 2024-09-20T19:00:02.166 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.166 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.166 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:00:02.166 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.591386+0000", 2024-09-20T19:00:02.166 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.002789962 2024-09-20T19:00:02.166 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.166 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.166 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:02.166 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.591731+0000", 2024-09-20T19:00:02.166 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00034524700000000002 2024-09-20T19:00:02.166 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.166 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.166 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:02.167 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.724872+0000", 2024-09-20T19:00:02.167 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13314100300000001 2024-09-20T19:00:02.167 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.167 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.167 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:00:02.167 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.724930+0000", 2024-09-20T19:00:02.167 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.7472000000000001e-05 2024-09-20T19:00:02.167 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.167 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.167 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:00:02.167 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.724974+0000", 2024-09-20T19:00:02.167 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.4366000000000002e-05 2024-09-20T19:00:02.167 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:00:02.167 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:00:02.168 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:00:02.168 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.168 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.168 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:84067 9.3 9:cfb01319:::benchmark_data_smithi083_124552_object84066:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e268)", 2024-09-20T19:00:02.168 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:38.595983+0000", 2024-09-20T19:00:02.168 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 23.461340261, 2024-09-20T19:00:02.168 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.12985870299999999, 2024-09-20T19:00:02.168 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:00:02.168 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:00:02.168 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:02.168 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:00:02.168 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:00:02.168 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:00:02.168 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 84067 2024-09-20T19:00:02.168 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.168 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:00:02.169 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.169 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:00:02.169 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.595983+0000", 2024-09-20T19:00:02.169 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:00:02.169 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.169 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.169 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:00:02.169 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.595983+0000", 2024-09-20T19:00:02.169 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:00:02.169 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.169 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.169 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:00:02.169 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.595985+0000", 2024-09-20T19:00:02.169 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9800000000000001e-06 2024-09-20T19:00:02.169 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.169 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.170 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:00:02.170 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.596024+0000", 2024-09-20T19:00:02.170 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.8640000000000003e-05 2024-09-20T19:00:02.170 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.170 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.170 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:00:02.170 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.596045+0000", 2024-09-20T19:00:02.170 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0707e-05 2024-09-20T19:00:02.170 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.170 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.170 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:00:02.170 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.596053+0000", 2024-09-20T19:00:02.170 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.3319999999999992e-06 2024-09-20T19:00:02.170 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.171 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.171 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:00:02.171 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.596111+0000", 2024-09-20T19:00:02.171 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.7803999999999997e-05 2024-09-20T19:00:02.171 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.171 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.171 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:00:02.171 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.596263+0000", 2024-09-20T19:00:02.171 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000151784 2024-09-20T19:00:02.171 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.171 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.171 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,3", 2024-09-20T19:00:02.171 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.596417+0000", 2024-09-20T19:00:02.171 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015399399999999999 2024-09-20T19:00:02.171 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.171 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.172 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:00:02.172 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.599163+0000", 2024-09-20T19:00:02.172 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.002746727 2024-09-20T19:00:02.172 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.172 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.172 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:02.172 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.722887+0000", 2024-09-20T19:00:02.172 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.12372408999999999 2024-09-20T19:00:02.172 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.172 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.172 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:02.172 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.725618+0000", 2024-09-20T19:00:02.172 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0027303980000000002 2024-09-20T19:00:02.172 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.172 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.173 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:00:02.173 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.725797+0000", 2024-09-20T19:00:02.173 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00017908400000000001 2024-09-20T19:00:02.173 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.173 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.173 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:00:02.173 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.725842+0000", 2024-09-20T19:00:02.173 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.5163000000000001e-05 2024-09-20T19:00:02.173 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:00:02.173 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:00:02.173 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:00:02.173 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.173 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.173 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:101137 9.e 9:705a8f57:::benchmark_data_smithi083_124552_object101136:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e275)", 2024-09-20T19:00:02.174 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:56.658485+0000", 2024-09-20T19:00:02.174 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 5.3988386620000002, 2024-09-20T19:00:02.174 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.122965137, 2024-09-20T19:00:02.174 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:00:02.174 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:00:02.174 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:02.174 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:00:02.174 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:00:02.174 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:00:02.174 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 101137 2024-09-20T19:00:02.174 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.174 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:00:02.174 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.174 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:00:02.174 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:56.658485+0000", 2024-09-20T19:00:02.175 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:00:02.175 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.175 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.175 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:00:02.175 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:56.658485+0000", 2024-09-20T19:00:02.175 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:00:02.175 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.175 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.175 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:00:02.175 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:56.658486+0000", 2024-09-20T19:00:02.175 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0279999999999999e-06 2024-09-20T19:00:02.175 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.175 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.175 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:00:02.175 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:56.658560+0000", 2024-09-20T19:00:02.176 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.3980000000000007e-05 2024-09-20T19:00:02.176 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.176 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.176 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:00:02.176 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:56.658570+0000", 2024-09-20T19:00:02.176 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0461e-05 2024-09-20T19:00:02.176 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.176 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.176 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:00:02.176 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:56.658577+0000", 2024-09-20T19:00:02.176 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.0489999999999998e-06 2024-09-20T19:00:02.176 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.176 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.176 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:00:02.177 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:56.765026+0000", 2024-09-20T19:00:02.177 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10644901599999999 2024-09-20T19:00:02.177 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.177 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.177 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:00:02.177 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:56.765164+0000", 2024-09-20T19:00:02.177 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013742499999999999 2024-09-20T19:00:02.177 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.177 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.177 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,4", 2024-09-20T19:00:02.177 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:56.765313+0000", 2024-09-20T19:00:02.177 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00014904 2024-09-20T19:00:02.177 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.177 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.177 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:02.178 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:56.770699+0000", 2024-09-20T19:00:02.178 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0053863640000000003 2024-09-20T19:00:02.178 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.178 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.178 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:02.178 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:56.775837+0000", 2024-09-20T19:00:02.178 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0051380430000000001 2024-09-20T19:00:02.178 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.178 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.178 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:00:02.178 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:56.781336+0000", 2024-09-20T19:00:02.178 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0054987120000000002 2024-09-20T19:00:02.178 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.178 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.178 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:00:02.179 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:56.781383+0000", 2024-09-20T19:00:02.179 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7407999999999997e-05 2024-09-20T19:00:02.179 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.179 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.179 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:00:02.179 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:56.781450+0000", 2024-09-20T19:00:02.179 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.6611000000000003e-05 2024-09-20T19:00:02.179 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:00:02.179 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:00:02.179 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:00:02.179 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.179 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.179 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:101846 9.d 9:bb37c834:::benchmark_data_smithi083_124552_object101845:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e275)", 2024-09-20T19:00:02.179 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:57.497943+0000", 2024-09-20T19:00:02.179 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 4.5593804110000002, 2024-09-20T19:00:02.180 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.22483731800000001, 2024-09-20T19:00:02.180 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:00:02.180 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:00:02.180 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:02.180 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:00:02.180 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:00:02.180 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:00:02.180 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 101846 2024-09-20T19:00:02.180 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.180 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:00:02.180 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.180 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:00:02.180 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.497943+0000", 2024-09-20T19:00:02.180 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:00:02.180 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.181 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.181 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:00:02.181 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.497943+0000", 2024-09-20T19:00:02.181 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:00:02.181 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.181 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.181 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:00:02.181 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.497945+0000", 2024-09-20T19:00:02.181 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6029999999999999e-06 2024-09-20T19:00:02.181 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.181 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.181 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:00:02.181 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.497989+0000", 2024-09-20T19:00:02.181 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3755999999999999e-05 2024-09-20T19:00:02.181 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.182 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.182 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:00:02.182 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.498001+0000", 2024-09-20T19:00:02.182 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2615e-05 2024-09-20T19:00:02.182 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.182 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.182 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:00:02.182 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.498010+0000", 2024-09-20T19:00:02.182 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.3599999999999996e-06 2024-09-20T19:00:02.182 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.182 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.182 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:00:02.182 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.498066+0000", 2024-09-20T19:00:02.182 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6923999999999999e-05 2024-09-20T19:00:02.183 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.183 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.183 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:00:02.183 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.498167+0000", 2024-09-20T19:00:02.183 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000100721 2024-09-20T19:00:02.183 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.183 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.183 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,5", 2024-09-20T19:00:02.183 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.498268+0000", 2024-09-20T19:00:02.183 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000100488 2024-09-20T19:00:02.183 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.183 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.183 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:00:02.183 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.500395+0000", 2024-09-20T19:00:02.183 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0021269259999999999 2024-09-20T19:00:02.184 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.184 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.184 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:02.184 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.500711+0000", 2024-09-20T19:00:02.184 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00031641299999999998 2024-09-20T19:00:02.184 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.184 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.184 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:02.184 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.722681+0000", 2024-09-20T19:00:02.184 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.22197028999999999 2024-09-20T19:00:02.184 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.184 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.184 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:00:02.184 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.722735+0000", 2024-09-20T19:00:02.184 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3879999999999999e-05 2024-09-20T19:00:02.185 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:00:02.185 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:00:02.185 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:00:02.185 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.722780+0000", 2024-09-20T19:00:02.185 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.5342000000000001e-05 2024-09-20T19:00:02.185 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:00:02.185 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:00:02.185 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:00:02.185 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:00:02.185 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:00:02.185 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T19:00:02.185 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:00:02.220 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:02 smithi083 ceph-mon[29421]: pgmap v499: 148 pgs: 1 active+recovering+undersized+remapped, 2 active+recovering+forced_recovery+undersized+remapped, 145 active+clean; 6.2 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 75 KiB/s rd, 60 MiB/s wr, 1.08k op/s; 3052/306255 objects misplaced (0.997%); 9.5 MiB/s, 151 objects/s recovering 2024-09-20T19:00:02.220 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:02 smithi083 ceph-mon[29421]: osdmap e278: 8 total, 8 up, 8 in 2024-09-20T19:00:02.220 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:02 smithi083 ceph-mon[36260]: pgmap v499: 148 pgs: 1 active+recovering+undersized+remapped, 2 active+recovering+forced_recovery+undersized+remapped, 145 active+clean; 6.2 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 75 KiB/s rd, 60 MiB/s wr, 1.08k op/s; 3052/306255 objects misplaced (0.997%); 9.5 MiB/s, 151 objects/s recovering 2024-09-20T19:00:02.220 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:02 smithi083 ceph-mon[36260]: osdmap e278: 8 total, 8 up, 8 in 2024-09-20T19:00:02.230 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:02.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:02.231 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:00:02.257 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 82 16 106539 106523 81.1791 67 0.0229116 0.0123126 2024-09-20T19:00:02.332 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:02.362 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:02.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:02.363 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:02.464 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:02.489 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:02.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:02.490 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:02.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:02 smithi142 ceph-mon[32508]: pgmap v499: 148 pgs: 1 active+recovering+undersized+remapped, 2 active+recovering+forced_recovery+undersized+remapped, 145 active+clean; 6.2 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 75 KiB/s rd, 60 MiB/s wr, 1.08k op/s; 3052/306255 objects misplaced (0.997%); 9.5 MiB/s, 151 objects/s recovering 2024-09-20T19:00:02.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:02 smithi142 ceph-mon[32508]: osdmap e278: 8 total, 8 up, 8 in 2024-09-20T19:00:02.591 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:02.603 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:00:02.603 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:00:02.613 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:02.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:02.614 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:02.622 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:00:02.623 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:00:02.641 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:00:02.641 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:00:02.658 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:00:02.659 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:00:02.678 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:00:02.679 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:00:02.698 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:00:02.698 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:00:02.715 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:02.717 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:00:02.718 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:00:02.738 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:02.738 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:00:02.738 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:00:02.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:02.739 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:02.840 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:02.863 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:02.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:02.864 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:02.964 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:02.986 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:02.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:02.987 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:03.088 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:00:03.110 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:03.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:03.111 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:00:03.211 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:00:03.236 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:03.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:03.237 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:00:03.257 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 83 16 106737 106721 80.3502 12.375 0.00572739 0.012376 2024-09-20T19:00:03.304 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T19:00:03.338 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:03.372 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:03.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:03.373 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:03.474 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:00:03.497 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:03.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:03.498 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:00:03.513 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:03 smithi083 ceph-mon[29421]: 5.f scrub starts 2024-09-20T19:00:03.513 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:03 smithi083 ceph-mon[29421]: 5.f scrub ok 2024-09-20T19:00:03.513 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:03 smithi083 ceph-mon[29421]: osdmap e279: 8 total, 8 up, 8 in 2024-09-20T19:00:03.513 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:03 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2012072828' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:00:03.513 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:03 smithi083 ceph-mon[36260]: 5.f scrub starts 2024-09-20T19:00:03.513 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:03 smithi083 ceph-mon[36260]: 5.f scrub ok 2024-09-20T19:00:03.514 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:03 smithi083 ceph-mon[36260]: osdmap e279: 8 total, 8 up, 8 in 2024-09-20T19:00:03.514 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:03 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2012072828' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:00:03.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:03 smithi142 ceph-mon[32508]: 5.f scrub starts 2024-09-20T19:00:03.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:03 smithi142 ceph-mon[32508]: 5.f scrub ok 2024-09-20T19:00:03.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:03 smithi142 ceph-mon[32508]: osdmap e279: 8 total, 8 up, 8 in 2024-09-20T19:00:03.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:03 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2012072828' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:00:03.598 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:03.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:03.626 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:03.626 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:03.727 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:03.746 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:00:03.771 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:03.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:03.775 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:03.875 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:03.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:03.911 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:03.911 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:04.012 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:00:04.044 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:04.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:04.045 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:00:04.091 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-20T19:00:04.146 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:00:04.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:04.198 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:04.198 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:00:04.257 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 84 16 107783 107767 80.1716 65.375 0.0140411 0.0124685 2024-09-20T19:00:04.299 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:04.327 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:04.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:04.328 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:04.428 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:00:04.458 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:04.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:04.459 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:00:04.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:04 smithi142 ceph-mon[32508]: 2.2 scrub starts 2024-09-20T19:00:04.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:04 smithi142 ceph-mon[32508]: 2.2 scrub ok 2024-09-20T19:00:04.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:04 smithi142 ceph-mon[32508]: pgmap v502: 148 pgs: 1 peering, 1 active+recovering+undersized+remapped, 1 active+recovering+forced_recovery+undersized+remapped, 145 active+clean; 6.5 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 75 MiB/s wr, 1.20k op/s; 1952/317982 objects misplaced (0.614%); 17 MiB/s, 274 objects/s recovering 2024-09-20T19:00:04.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:04 smithi142 ceph-mon[32508]: osdmap e280: 8 total, 8 up, 8 in 2024-09-20T19:00:04.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:04 smithi142 ceph-mon[32508]: 4.d scrub starts 2024-09-20T19:00:04.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:04 smithi142 ceph-mon[32508]: 4.d scrub ok 2024-09-20T19:00:04.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:04 smithi083 ceph-mon[29421]: 2.2 scrub starts 2024-09-20T19:00:04.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:04 smithi083 ceph-mon[29421]: 2.2 scrub ok 2024-09-20T19:00:04.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:04 smithi083 ceph-mon[29421]: pgmap v502: 148 pgs: 1 peering, 1 active+recovering+undersized+remapped, 1 active+recovering+forced_recovery+undersized+remapped, 145 active+clean; 6.5 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 75 MiB/s wr, 1.20k op/s; 1952/317982 objects misplaced (0.614%); 17 MiB/s, 274 objects/s recovering 2024-09-20T19:00:04.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:04 smithi083 ceph-mon[29421]: osdmap e280: 8 total, 8 up, 8 in 2024-09-20T19:00:04.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:04 smithi083 ceph-mon[29421]: 4.d scrub starts 2024-09-20T19:00:04.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:04 smithi083 ceph-mon[29421]: 4.d scrub ok 2024-09-20T19:00:04.548 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:04 smithi083 ceph-mon[36260]: 2.2 scrub starts 2024-09-20T19:00:04.548 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:04 smithi083 ceph-mon[36260]: 2.2 scrub ok 2024-09-20T19:00:04.548 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:04 smithi083 ceph-mon[36260]: pgmap v502: 148 pgs: 1 peering, 1 active+recovering+undersized+remapped, 1 active+recovering+forced_recovery+undersized+remapped, 145 active+clean; 6.5 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 75 MiB/s wr, 1.20k op/s; 1952/317982 objects misplaced (0.614%); 17 MiB/s, 274 objects/s recovering 2024-09-20T19:00:04.548 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:04 smithi083 ceph-mon[36260]: osdmap e280: 8 total, 8 up, 8 in 2024-09-20T19:00:04.548 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:04 smithi083 ceph-mon[36260]: 4.d scrub starts 2024-09-20T19:00:04.548 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:04 smithi083 ceph-mon[36260]: 4.d scrub ok 2024-09-20T19:00:04.561 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:04.564 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:00:04.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:04.618 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:04.618 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:04.719 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:04.762 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:04.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:04.763 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:04.864 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:04.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:04.901 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:04.901 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:05.002 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:05.033 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:05.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:05.034 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:05.134 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:05.170 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:05.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:05.171 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:05.181 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T19:00:05.181 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:00:05.230 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-20T19:00:05.258 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 85 16 108831 108815 79.9989 65.5 0.0172714 0.0124944 2024-09-20T19:00:05.272 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:05.296 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:05 smithi083 ceph-mon[29421]: 3.1c scrub starts 2024-09-20T19:00:05.296 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:05 smithi083 ceph-mon[29421]: 3.1c scrub ok 2024-09-20T19:00:05.296 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:05 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/299844105' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:00:05.296 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:05 smithi083 ceph-mon[36260]: 3.1c scrub starts 2024-09-20T19:00:05.296 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:05 smithi083 ceph-mon[36260]: 3.1c scrub ok 2024-09-20T19:00:05.296 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:05 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/299844105' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:00:05.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:05.306 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:05.306 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:05.407 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:00:05.443 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:05.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:05.444 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:00:05.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:05 smithi142 ceph-mon[32508]: 3.1c scrub starts 2024-09-20T19:00:05.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:05 smithi142 ceph-mon[32508]: 3.1c scrub ok 2024-09-20T19:00:05.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:05 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/299844105' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:00:05.545 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:05.560 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T19:00:05.578 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:05.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:05.579 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:05.680 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:00:05.701 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:05.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:05.702 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:00:05.803 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:05.832 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:05.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:05.835 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:05.935 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:05.969 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:05.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:05.970 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:06.071 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:06.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:06.108 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:06.108 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:06.209 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:06.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:06.241 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:06.241 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:06.258 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 86 16 109935 109919 79.871 69 0.0073628 0.0125146 2024-09-20T19:00:06.341 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:06.375 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:06.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:06.376 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:06.477 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:00:06.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:06 smithi142 ceph-mon[32508]: 4.3 scrub starts 2024-09-20T19:00:06.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:06 smithi142 ceph-mon[32508]: 4.3 scrub ok 2024-09-20T19:00:06.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:06 smithi142 ceph-mon[32508]: pgmap v504: 148 pgs: 1 peering, 1 active+recovering+undersized+remapped, 1 active+recovering+forced_recovery+undersized+remapped, 145 active+clean; 6.5 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.0 KiB/s rd, 62 MiB/s wr, 989 op/s; 1952/317982 objects misplaced (0.614%); 14 MiB/s, 226 objects/s recovering 2024-09-20T19:00:06.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:06 smithi142 ceph-mon[32508]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:00:06.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:06 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/299844105' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T19:00:06.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:06 smithi142 ceph-mon[32508]: osdmap e281: 8 total, 8 up, 8 in 2024-09-20T19:00:06.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:06.518 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:06.518 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:00:06.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:06 smithi083 ceph-mon[29421]: 4.3 scrub starts 2024-09-20T19:00:06.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:06 smithi083 ceph-mon[29421]: 4.3 scrub ok 2024-09-20T19:00:06.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:06 smithi083 ceph-mon[29421]: pgmap v504: 148 pgs: 1 peering, 1 active+recovering+undersized+remapped, 1 active+recovering+forced_recovery+undersized+remapped, 145 active+clean; 6.5 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.0 KiB/s rd, 62 MiB/s wr, 989 op/s; 1952/317982 objects misplaced (0.614%); 14 MiB/s, 226 objects/s recovering 2024-09-20T19:00:06.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:06 smithi083 ceph-mon[29421]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:00:06.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:06 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/299844105' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T19:00:06.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:06 smithi083 ceph-mon[29421]: osdmap e281: 8 total, 8 up, 8 in 2024-09-20T19:00:06.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:06 smithi083 ceph-mon[36260]: 4.3 scrub starts 2024-09-20T19:00:06.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:06 smithi083 ceph-mon[36260]: 4.3 scrub ok 2024-09-20T19:00:06.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:06 smithi083 ceph-mon[36260]: pgmap v504: 148 pgs: 1 peering, 1 active+recovering+undersized+remapped, 1 active+recovering+forced_recovery+undersized+remapped, 145 active+clean; 6.5 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.0 KiB/s rd, 62 MiB/s wr, 989 op/s; 1952/317982 objects misplaced (0.614%); 14 MiB/s, 226 objects/s recovering 2024-09-20T19:00:06.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:06 smithi083 ceph-mon[36260]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:00:06.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:06 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/299844105' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T19:00:06.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:06 smithi083 ceph-mon[36260]: osdmap e281: 8 total, 8 up, 8 in 2024-09-20T19:00:06.619 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:00:06.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:06.651 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:06.652 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:00:06.752 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:06.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:06.795 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:06.795 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:06.897 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:06.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:06.931 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:06.932 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:07.018 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 7, 6] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-20T19:00:07.018 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T19:00:07.019 INFO:tasks.thrashosds.thrasher:Shrinking pool unique_pool_2 2024-09-20T19:00:07.019 INFO:tasks.cephadm.ceph_manager.ceph:contract_pool unique_pool_2 by 10 min 8 2024-09-20T19:00:07.019 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T19:00:07.032 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:07.063 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:07.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:07.064 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:07.165 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:00:07.195 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:07.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:07.196 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:00:07.258 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 87 16 111127 111111 79.8091 74.5 0.00743821 0.0125218 2024-09-20T19:00:07.266 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:00:06 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:00:07.266 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:07 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:00:07.266 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:07 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:00:07.266 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:07 smithi142 ceph-mon[32508]: osdmap e282: 8 total, 8 up, 8 in 2024-09-20T19:00:07.297 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:00:07.318 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:07.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:07.319 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:00:07.347 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:07 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:00:07.347 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:07 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:00:07.347 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:07 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:00:07.419 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:07.458 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:07.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:07.459 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:07.560 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:07.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:07.581 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:07.582 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:07.682 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:07.723 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:07.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:07.724 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:07.796 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:07 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:00:07.796 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:07 smithi083 ceph-mon[29421]: osdmap e282: 8 total, 8 up, 8 in 2024-09-20T19:00:07.796 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:07 smithi083 ceph-mon[36260]: osdmap e282: 8 total, 8 up, 8 in 2024-09-20T19:00:07.826 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:07.856 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:07.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:07.857 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:07.958 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:07.992 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:07.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:07.993 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:08.094 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:00:08.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:08.126 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:08.127 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:00:08.227 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:08.258 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 88 16 112105 112089 79.5968 61.125 0.00619929 0.0125585 2024-09-20T19:00:08.266 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:08.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:08.267 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:08.368 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:00:08.389 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:08.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:08.390 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:00:08.491 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:08.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:08 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:00:08.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:08 smithi142 ceph-mon[32508]: pgmap v507: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+forced_recovery+undersized+remapped, 1 peering, 145 active+clean; 6.7 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.0 KiB/s rd, 60 MiB/s wr, 953 op/s; 851/329031 objects misplaced (0.259%); 20 MiB/s, 315 objects/s recovering 2024-09-20T19:00:08.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:08 smithi142 ceph-mon[32508]: Health check failed: Degraded data redundancy: 2 pgs undersized (PG_DEGRADED) 2024-09-20T19:00:08.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:08.524 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:08.524 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:08.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:08 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:00:08.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:08 smithi083 ceph-mon[29421]: pgmap v507: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+forced_recovery+undersized+remapped, 1 peering, 145 active+clean; 6.7 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.0 KiB/s rd, 60 MiB/s wr, 953 op/s; 851/329031 objects misplaced (0.259%); 20 MiB/s, 315 objects/s recovering 2024-09-20T19:00:08.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:08 smithi083 ceph-mon[29421]: Health check failed: Degraded data redundancy: 2 pgs undersized (PG_DEGRADED) 2024-09-20T19:00:08.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:08 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:00:08.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:08 smithi083 ceph-mon[36260]: pgmap v507: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+forced_recovery+undersized+remapped, 1 peering, 145 active+clean; 6.7 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.0 KiB/s rd, 60 MiB/s wr, 953 op/s; 851/329031 objects misplaced (0.259%); 20 MiB/s, 315 objects/s recovering 2024-09-20T19:00:08.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:08 smithi083 ceph-mon[36260]: Health check failed: Degraded data redundancy: 2 pgs undersized (PG_DEGRADED) 2024-09-20T19:00:08.626 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:00:08.648 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:08.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:08.649 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:00:08.749 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:00:08.785 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:08.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:08.786 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:00:08.887 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:00:08.909 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:08.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:08.910 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:00:09.011 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:00:09.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:09.050 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:09.050 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:00:09.151 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:09.183 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:09.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:09.184 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:09.258 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 89 16 113107 113091 79.406 62.625 0.0055662 0.0125886 2024-09-20T19:00:09.285 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:00:09.317 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:09.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:09.318 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:00:09.418 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:09.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:09.452 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:09.452 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:09.553 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:09.578 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:09.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:09.580 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:09.680 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:09.714 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:09.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:09.717 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:09.818 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:09.839 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:09.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:09.840 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:09.941 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:09.972 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:09.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:09.973 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:10.074 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:10.122 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:10.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:10.123 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:10.224 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:10.258 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 90 1 114144 114143 79.2542 65.75 0.0183883 0.0126129 2024-09-20T19:00:10.259 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:10.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:10.260 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:10.296 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:10 smithi083 ceph-mon[29421]: pgmap v508: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+forced_recovery+undersized+remapped, 146 active+clean; 6.7 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 852 B/s rd, 47 MiB/s wr, 757 op/s; 851/331620 objects misplaced (0.257%); 11 MiB/s, 183 objects/s recovering 2024-09-20T19:00:10.296 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:10 smithi083 ceph-mon[36260]: pgmap v508: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+forced_recovery+undersized+remapped, 146 active+clean; 6.7 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 852 B/s rd, 47 MiB/s wr, 757 op/s; 851/331620 objects misplaced (0.257%); 11 MiB/s, 183 objects/s recovering 2024-09-20T19:00:10.361 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:00:10.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:10.383 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:10.383 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:00:10.484 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:10.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:10.506 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:10.506 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:10.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:10 smithi142 ceph-mon[32508]: pgmap v508: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+forced_recovery+undersized+remapped, 146 active+clean; 6.7 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 852 B/s rd, 47 MiB/s wr, 757 op/s; 851/331620 objects misplaced (0.257%); 11 MiB/s, 183 objects/s recovering 2024-09-20T19:00:10.607 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:10.629 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:10.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:10.630 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:10.731 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:10.752 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:10.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:10.753 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:10.854 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:10.875 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:10.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:10.876 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:10.977 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:11.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:11.000 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:11.000 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:11.046 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:00:10 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:00:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:00:11.101 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:00:11.123 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:11.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:11.124 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:00:11.151 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_blocked_ops 2024-09-20T19:00:11.225 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:11.258 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 91 1 114144 114143 78.3834 0 - 0.0126129 2024-09-20T19:00:11.262 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:11.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:11.264 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:11.365 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:11.386 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:11.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:11.387 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:11.488 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:11.509 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:11.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:11.511 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:11.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:11 smithi142 ceph-mon[32508]: osdmap e283: 8 total, 8 up, 8 in 2024-09-20T19:00:11.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:11 smithi083 ceph-mon[29421]: osdmap e283: 8 total, 8 up, 8 in 2024-09-20T19:00:11.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:11 smithi083 ceph-mon[36260]: osdmap e283: 8 total, 8 up, 8 in 2024-09-20T19:00:11.616 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:11.637 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:11.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:11.638 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:11.711 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:00:11.739 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:11.763 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:11.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:11.764 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:11.865 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:11.886 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:11.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:11.888 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:11.988 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:12.010 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:12.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:12.011 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:12.112 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:00:12.133 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:12.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:12.134 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:00:12.154 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:00:12.154 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:00:12.235 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:12.259 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Total time run: 91.015 2024-09-20T19:00:12.260 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Total writes made: 114144 2024-09-20T19:00:12.260 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Write size: 65536 2024-09-20T19:00:12.260 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Object size: 65536 2024-09-20T19:00:12.260 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Bandwidth (MB/sec): 78.3827 2024-09-20T19:00:12.260 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Stddev Bandwidth: 38.8412 2024-09-20T19:00:12.260 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Max bandwidth (MB/sec): 205.688 2024-09-20T19:00:12.260 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Min bandwidth (MB/sec): 0 2024-09-20T19:00:12.260 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Average IOPS: 1254 2024-09-20T19:00:12.260 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Stddev IOPS: 621.46 2024-09-20T19:00:12.260 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Max IOPS: 3291 2024-09-20T19:00:12.260 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Min IOPS: 0 2024-09-20T19:00:12.260 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Average Latency(s): 0.0126217 2024-09-20T19:00:12.260 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Stddev Latency(s): 0.0296034 2024-09-20T19:00:12.260 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Max latency(s): 2.15034 2024-09-20T19:00:12.261 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Min latency(s): 0.00133037 2024-09-20T19:00:12.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:12.261 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:12.262 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:12.290 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Cleaning up (deleting benchmark objects) 2024-09-20T19:00:12.363 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:12.392 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:12.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:12.393 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:12.412 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T19:00:12.414 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-20T19:00:12.494 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:00:12.498 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:12 smithi083 ceph-mon[36260]: pgmap v510: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+forced_recovery+undersized+remapped, 146 active+clean; 6.8 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 1.7 KiB/s rd, 56 MiB/s wr, 899 op/s; 851/334146 objects misplaced (0.255%); 11 MiB/s, 183 objects/s recovering 2024-09-20T19:00:12.498 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:12 smithi083 ceph-mon[36260]: osdmap e284: 8 total, 8 up, 8 in 2024-09-20T19:00:12.498 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:12 smithi083 ceph-mon[29421]: pgmap v510: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+forced_recovery+undersized+remapped, 146 active+clean; 6.8 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 1.7 KiB/s rd, 56 MiB/s wr, 899 op/s; 851/334146 objects misplaced (0.255%); 11 MiB/s, 183 objects/s recovering 2024-09-20T19:00:12.498 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:12 smithi083 ceph-mon[29421]: osdmap e284: 8 total, 8 up, 8 in 2024-09-20T19:00:12.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:12 smithi142 ceph-mon[32508]: pgmap v510: 148 pgs: 1 active+recovering+undersized+remapped, 1 active+recovering+forced_recovery+undersized+remapped, 146 active+clean; 6.8 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 1.7 KiB/s rd, 56 MiB/s wr, 899 op/s; 851/334146 objects misplaced (0.255%); 11 MiB/s, 183 objects/s recovering 2024-09-20T19:00:12.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:12 smithi142 ceph-mon[32508]: osdmap e284: 8 total, 8 up, 8 in 2024-09-20T19:00:12.521 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:12.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:12.522 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:00:12.623 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:12.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:12.657 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:12.657 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:12.758 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:00:12.794 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:12.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:12.795 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:00:12.896 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:12.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:12.930 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:12.930 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:13.031 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:00:13.065 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:13.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:13.066 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:00:13.166 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:13.201 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:13.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:13.202 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:13.302 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:13.341 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:13.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:13.342 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:13.443 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:00:13.475 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":512,"stamp":"2024-09-20T19:00:12.247917+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":7481137691,"num_objects":114343,"num_object_clones":0,"num_object_copies":343029,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":114343,"num_whiteouts":0,"num_read":1893,"num_read_kb":1750,"num_write":114979,"num_write_kb":7306889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9653,"num_bytes_recovered":624300722,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83575,"ondisk_log_size":83575,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":448,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":25458076,"kb_used_data":22444476,"kb_used_omap":12,"kb_used_meta":3013427,"kb_avail":724503140,"statfs":{"total":767960285184,"available":741891215360,"internally_reserved":0,"allocated":22983143424,"data_stored":22980032807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":3085749808},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":381419520,"num_objects":5820,"num_object_clones":0,"num_object_copies":17460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-1439,"num_objects_unfound":0,"num_objects_dirty":5820,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5820,"num_write_kb":372480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1439,"num_bytes_recovered":94306304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.469293"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":488,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.241333+0000","last_change":"2024-09-20T18:55:28.037494+0000","last_active":"2024-09-20T19:00:11.241333+0000","last_peered":"2024-09-20T19:00:11.241333+0000","last_clean":"2024-09-20T19:00:11.241333+0000","last_became_active":"2024-09-20T18:49:45.943201+0000","last_became_peered":"2024-09-20T18:49:45.943201+0000","last_unstale":"2024-09-20T19:00:11.241333+0000","last_undegraded":"2024-09-20T19:00:11.241333+0000","last_fullsized":"2024-09-20T19:00:11.241333+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:28.037449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:28.037449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"273'15","reported_seq":559,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.232193+0000","last_change":"2024-09-20T18:59:05.094771+0000","last_active":"2024-09-20T19:00:10.232193+0000","last_peered":"2024-09-20T19:00:10.232193+0000","last_clean":"2024-09-20T19:00:10.232193+0000","last_became_active":"2024-09-20T18:59:05.076159+0000","last_became_peered":"2024-09-20T18:59:05.076159+0000","last_unstale":"2024-09-20T19:00:10.232193+0000","last_undegraded":"2024-09-20T19:00:10.232193+0000","last_fullsized":"2024-09-20T19:00:10.232193+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:50.231620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:50.231620+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:25.223399+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":487,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.236113+0000","last_change":"2024-09-20T18:57:49.016078+0000","last_active":"2024-09-20T19:00:11.236113+0000","last_peered":"2024-09-20T19:00:11.236113+0000","last_clean":"2024-09-20T19:00:11.236113+0000","last_became_active":"2024-09-20T18:49:49.959514+0000","last_became_peered":"2024-09-20T18:49:49.959514+0000","last_unstale":"2024-09-20T19:00:11.236113+0000","last_undegraded":"2024-09-20T19:00:11.236113+0000","last_fullsized":"2024-09-20T19:00:11.236113+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:49.016006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:49.016006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":483,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.232317+0000","last_change":"2024-09-20T18:58:02.084435+0000","last_active":"2024-09-20T19:00:10.232317+0000","last_peered":"2024-09-20T19:00:10.232317+0000","last_clean":"2024-09-20T19:00:10.232317+0000","last_became_active":"2024-09-20T18:49:51.969421+0000","last_became_peered":"2024-09-20T18:49:51.969421+0000","last_unstale":"2024-09-20T19:00:10.232317+0000","last_undegraded":"2024-09-20T19:00:10.232317+0000","last_fullsized":"2024-09-20T19:00:10.232317+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:02.084375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:02.084375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:43.288290+0000","scrub_duration":197,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":489,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.237378+0000","last_change":"2024-09-20T18:59:05.081431+0000","last_active":"2024-09-20T19:00:11.237378+0000","last_peered":"2024-09-20T19:00:11.237378+0000","last_clean":"2024-09-20T19:00:11.237378+0000","last_became_active":"2024-09-20T18:59:05.081081+0000","last_became_peered":"2024-09-20T18:59:05.081081+0000","last_unstale":"2024-09-20T19:00:11.237378+0000","last_undegraded":"2024-09-20T19:00:11.237378+0000","last_fullsized":"2024-09-20T19:00:11.237378+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.216188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.216188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":502,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.238400+0000","last_change":"2024-09-20T18:59:05.073093+0000","last_active":"2024-09-20T19:00:11.238400+0000","last_peered":"2024-09-20T19:00:11.238400+0000","last_clean":"2024-09-20T19:00:11.238400+0000","last_became_active":"2024-09-20T18:59:05.072675+0000","last_became_peered":"2024-09-20T18:59:05.072675+0000","last_unstale":"2024-09-20T19:00:11.238400+0000","last_undegraded":"2024-09-20T19:00:11.238400+0000","last_fullsized":"2024-09-20T19:00:11.238400+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:03.055929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:59:03.055929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:26.275543+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"274'27","reported_seq":526,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.239617+0000","last_change":"2024-09-20T18:58:57.097877+0000","last_active":"2024-09-20T19:00:11.239617+0000","last_peered":"2024-09-20T19:00:11.239617+0000","last_clean":"2024-09-20T19:00:11.239617+0000","last_became_active":"2024-09-20T18:49:47.952150+0000","last_became_peered":"2024-09-20T18:49:47.952150+0000","last_unstale":"2024-09-20T19:00:11.239617+0000","last_undegraded":"2024-09-20T19:00:11.239617+0000","last_fullsized":"2024-09-20T19:00:11.239617+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:58:57.097766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:57.097766+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":38,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":493,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.232107+0000","last_change":"2024-09-20T18:59:05.077076+0000","last_active":"2024-09-20T19:00:10.232107+0000","last_peered":"2024-09-20T19:00:10.232107+0000","last_clean":"2024-09-20T19:00:10.232107+0000","last_became_active":"2024-09-20T18:59:05.076607+0000","last_became_peered":"2024-09-20T18:59:05.076607+0000","last_unstale":"2024-09-20T19:00:10.232107+0000","last_undegraded":"2024-09-20T19:00:10.232107+0000","last_fullsized":"2024-09-20T19:00:10.232107+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:06.846018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:06.846018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:45.340583+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"59'1","reported_seq":529,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.238312+0000","last_change":"2024-09-20T18:59:05.111316+0000","last_active":"2024-09-20T19:00:10.238312+0000","last_peered":"2024-09-20T19:00:10.238312+0000","last_clean":"2024-09-20T19:00:10.238312+0000","last_became_active":"2024-09-20T18:59:05.090056+0000","last_became_peered":"2024-09-20T18:59:05.090056+0000","last_unstale":"2024-09-20T19:00:10.238312+0000","last_undegraded":"2024-09-20T19:00:10.238312+0000","last_fullsized":"2024-09-20T19:00:10.238312+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:58:09.472360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:09.472360+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"273'15","reported_seq":516,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.232628+0000","last_change":"2024-09-20T18:58:29.671959+0000","last_active":"2024-09-20T19:00:10.232628+0000","last_peered":"2024-09-20T19:00:10.232628+0000","last_clean":"2024-09-20T19:00:10.232628+0000","last_became_active":"2024-09-20T18:49:47.947448+0000","last_became_peered":"2024-09-20T18:49:47.947448+0000","last_unstale":"2024-09-20T19:00:10.232628+0000","last_undegraded":"2024-09-20T19:00:10.232628+0000","last_fullsized":"2024-09-20T19:00:10.232628+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:58:29.671887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:29.671887+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":485,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.237588+0000","last_change":"2024-09-20T18:58:25.505818+0000","last_active":"2024-09-20T19:00:10.237588+0000","last_peered":"2024-09-20T19:00:10.237588+0000","last_clean":"2024-09-20T19:00:10.237588+0000","last_became_active":"2024-09-20T18:49:49.961188+0000","last_became_peered":"2024-09-20T18:49:49.961188+0000","last_unstale":"2024-09-20T19:00:10.237588+0000","last_undegraded":"2024-09-20T19:00:10.237588+0000","last_fullsized":"2024-09-20T19:00:10.237588+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:25.505757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:25.505757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":45,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":460,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.240583+0000","last_change":"2024-09-20T18:58:21.861893+0000","last_active":"2024-09-20T19:00:11.240583+0000","last_peered":"2024-09-20T19:00:11.240583+0000","last_clean":"2024-09-20T19:00:11.240583+0000","last_became_active":"2024-09-20T18:49:51.966715+0000","last_became_peered":"2024-09-20T18:49:51.966715+0000","last_unstale":"2024-09-20T19:00:11.240583+0000","last_undegraded":"2024-09-20T19:00:11.240583+0000","last_fullsized":"2024-09-20T19:00:11.240583+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:21.861836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:21.861836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":485,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.237190+0000","last_change":"2024-09-20T19:00:03.432784+0000","last_active":"2024-09-20T19:00:10.237190+0000","last_peered":"2024-09-20T19:00:10.237190+0000","last_clean":"2024-09-20T19:00:10.237190+0000","last_became_active":"2024-09-20T18:49:45.944703+0000","last_became_peered":"2024-09-20T18:49:45.944703+0000","last_unstale":"2024-09-20T19:00:10.237190+0000","last_undegraded":"2024-09-20T19:00:10.237190+0000","last_fullsized":"2024-09-20T19:00:10.237190+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:00:03.432711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:00:03.432711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:04.816004+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"273'7","reported_seq":514,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.232485+0000","last_change":"2024-09-20T18:58:33.660957+0000","last_active":"2024-09-20T19:00:10.232485+0000","last_peered":"2024-09-20T19:00:10.232485+0000","last_clean":"2024-09-20T19:00:10.232485+0000","last_became_active":"2024-09-20T18:51:23.813399+0000","last_became_peered":"2024-09-20T18:51:23.813399+0000","last_unstale":"2024-09-20T19:00:10.232485+0000","last_undegraded":"2024-09-20T19:00:10.232485+0000","last_fullsized":"2024-09-20T19:00:10.232485+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-20T18:58:33.660531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:33.660531+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":478,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.240599+0000","last_change":"2024-09-20T18:58:32.526595+0000","last_active":"2024-09-20T19:00:10.240599+0000","last_peered":"2024-09-20T19:00:10.240599+0000","last_clean":"2024-09-20T19:00:10.240599+0000","last_became_active":"2024-09-20T18:49:49.959696+0000","last_became_peered":"2024-09-20T18:49:49.959696+0000","last_unstale":"2024-09-20T19:00:10.240599+0000","last_undegraded":"2024-09-20T19:00:10.240599+0000","last_fullsized":"2024-09-20T19:00:10.240599+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.526527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.526527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":481,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.237087+0000","last_change":"2024-09-20T18:58:26.465305+0000","last_active":"2024-09-20T19:00:10.237087+0000","last_peered":"2024-09-20T19:00:10.237087+0000","last_clean":"2024-09-20T19:00:10.237087+0000","last_became_active":"2024-09-20T18:49:51.970663+0000","last_became_peered":"2024-09-20T18:49:51.970663+0000","last_unstale":"2024-09-20T19:00:10.237087+0000","last_undegraded":"2024-09-20T19:00:10.237087+0000","last_fullsized":"2024-09-20T19:00:10.237087+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:26.465264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:26.465264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":490,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.233390+0000","last_change":"2024-09-20T18:59:05.079781+0000","last_active":"2024-09-20T19:00:10.233390+0000","last_peered":"2024-09-20T19:00:10.233390+0000","last_clean":"2024-09-20T19:00:10.233390+0000","last_became_active":"2024-09-20T18:59:05.079278+0000","last_became_peered":"2024-09-20T18:59:05.079278+0000","last_unstale":"2024-09-20T19:00:10.233390+0000","last_undegraded":"2024-09-20T19:00:10.233390+0000","last_fullsized":"2024-09-20T19:00:10.233390+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:00.906652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:00.906652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:33.272916+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":488,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.241323+0000","last_change":"2024-09-20T18:58:34.996936+0000","last_active":"2024-09-20T19:00:11.241323+0000","last_peered":"2024-09-20T19:00:11.241323+0000","last_clean":"2024-09-20T19:00:11.241323+0000","last_became_active":"2024-09-20T18:49:45.950480+0000","last_became_peered":"2024-09-20T18:49:45.950480+0000","last_unstale":"2024-09-20T19:00:11.241323+0000","last_undegraded":"2024-09-20T19:00:11.241323+0000","last_fullsized":"2024-09-20T19:00:11.241323+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:34.996893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:34.996893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"273'27","reported_seq":529,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.234421+0000","last_change":"2024-09-20T18:58:28.253192+0000","last_active":"2024-09-20T19:00:11.234421+0000","last_peered":"2024-09-20T19:00:11.234421+0000","last_clean":"2024-09-20T19:00:11.234421+0000","last_became_active":"2024-09-20T18:49:47.950939+0000","last_became_peered":"2024-09-20T18:49:47.950939+0000","last_unstale":"2024-09-20T19:00:11.234421+0000","last_undegraded":"2024-09-20T19:00:11.234421+0000","last_fullsized":"2024-09-20T19:00:11.234421+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:58:28.253141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:28.253141+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":471,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.238128+0000","last_change":"2024-09-20T18:57:14.941518+0000","last_active":"2024-09-20T19:00:11.238128+0000","last_peered":"2024-09-20T19:00:11.238128+0000","last_clean":"2024-09-20T19:00:11.238128+0000","last_became_active":"2024-09-20T18:49:49.958518+0000","last_became_peered":"2024-09-20T18:49:49.958518+0000","last_unstale":"2024-09-20T19:00:11.238128+0000","last_undegraded":"2024-09-20T19:00:11.238128+0000","last_fullsized":"2024-09-20T19:00:11.238128+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:14.941458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:14.941458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":489,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.239394+0000","last_change":"2024-09-20T18:59:05.084868+0000","last_active":"2024-09-20T19:00:11.239394+0000","last_peered":"2024-09-20T19:00:11.239394+0000","last_clean":"2024-09-20T19:00:11.239394+0000","last_became_active":"2024-09-20T18:59:05.084496+0000","last_became_peered":"2024-09-20T18:59:05.084496+0000","last_unstale":"2024-09-20T19:00:11.239394+0000","last_undegraded":"2024-09-20T19:00:11.239394+0000","last_fullsized":"2024-09-20T19:00:11.239394+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:35.125451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:35.125451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"59'1","reported_seq":501,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.232722+0000","last_change":"2024-09-20T18:59:03.397298+0000","last_active":"2024-09-20T19:00:10.232722+0000","last_peered":"2024-09-20T19:00:10.232722+0000","last_clean":"2024-09-20T19:00:10.232722+0000","last_became_active":"2024-09-20T18:49:45.940100+0000","last_became_peered":"2024-09-20T18:49:45.940100+0000","last_unstale":"2024-09-20T19:00:10.232722+0000","last_undegraded":"2024-09-20T19:00:10.232722+0000","last_fullsized":"2024-09-20T19:00:10.232722+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:59:03.397225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:59:03.397225+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:17.466210+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"274'18","reported_seq":499,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.238602+0000","last_change":"2024-09-20T18:57:09.819080+0000","last_active":"2024-09-20T19:00:11.238602+0000","last_peered":"2024-09-20T19:00:11.238602+0000","last_clean":"2024-09-20T19:00:11.238602+0000","last_became_active":"2024-09-20T18:49:47.950909+0000","last_became_peered":"2024-09-20T18:49:47.950909+0000","last_unstale":"2024-09-20T19:00:11.238602+0000","last_undegraded":"2024-09-20T19:00:11.238602+0000","last_fullsized":"2024-09-20T19:00:11.238602+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:57:09.818999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:09.818999+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":479,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.234714+0000","last_change":"2024-09-20T18:54:44.239273+0000","last_active":"2024-09-20T19:00:10.234714+0000","last_peered":"2024-09-20T19:00:10.234714+0000","last_clean":"2024-09-20T19:00:10.234714+0000","last_became_active":"2024-09-20T18:49:49.959102+0000","last_became_peered":"2024-09-20T18:49:49.959102+0000","last_unstale":"2024-09-20T19:00:10.234714+0000","last_undegraded":"2024-09-20T19:00:10.234714+0000","last_fullsized":"2024-09-20T19:00:10.234714+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:44.239226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.239226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:18.193129+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":475,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.240697+0000","last_change":"2024-09-20T18:58:25.501434+0000","last_active":"2024-09-20T19:00:10.240697+0000","last_peered":"2024-09-20T19:00:10.240697+0000","last_clean":"2024-09-20T19:00:10.240697+0000","last_became_active":"2024-09-20T18:49:51.970058+0000","last_became_peered":"2024-09-20T18:49:51.970058+0000","last_unstale":"2024-09-20T19:00:10.240697+0000","last_undegraded":"2024-09-20T19:00:10.240697+0000","last_fullsized":"2024-09-20T19:00:10.240697+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T18:58:25.501384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:25.501384+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":480,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.236230+0000","last_change":"2024-09-20T18:57:18.032140+0000","last_active":"2024-09-20T19:00:11.236230+0000","last_peered":"2024-09-20T19:00:11.236230+0000","last_clean":"2024-09-20T19:00:11.236230+0000","last_became_active":"2024-09-20T18:49:45.933078+0000","last_became_peered":"2024-09-20T18:49:45.933078+0000","last_unstale":"2024-09-20T19:00:11.236230+0000","last_undegraded":"2024-09-20T19:00:11.236230+0000","last_fullsized":"2024-09-20T19:00:11.236230+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:57:18.032067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:18.032067+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"274'20","reported_seq":523,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.241758+0000","last_change":"2024-09-20T18:59:02.024076+0000","last_active":"2024-09-20T19:00:11.241758+0000","last_peered":"2024-09-20T19:00:11.241758+0000","last_clean":"2024-09-20T19:00:11.241758+0000","last_became_active":"2024-09-20T18:49:48.500809+0000","last_became_peered":"2024-09-20T18:49:48.500809+0000","last_unstale":"2024-09-20T19:00:11.241758+0000","last_undegraded":"2024-09-20T19:00:11.241758+0000","last_fullsized":"2024-09-20T19:00:11.241758+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:59:02.023996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:59:02.023996+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"262'10","reported_seq":73,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.246614+0000","last_change":"2024-09-20T18:59:05.105124+0000","last_active":"2024-09-20T19:00:10.246614+0000","last_peered":"2024-09-20T19:00:10.246614+0000","last_clean":"2024-09-20T19:00:10.246614+0000","last_became_active":"2024-09-20T18:59:05.089298+0000","last_became_peered":"2024-09-20T18:59:05.089298+0000","last_unstale":"2024-09-20T19:00:10.246614+0000","last_undegraded":"2024-09-20T19:00:10.246614+0000","last_fullsized":"2024-09-20T19:00:10.246614+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'9","last_scrub_stamp":"2024-09-20T18:57:48.853302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:48.853302+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2","version":"282'7039","reported_seq":7126,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.237123+0000","last_change":"2024-09-20T18:58:33.827583+0000","last_active":"2024-09-20T19:00:11.237123+0000","last_peered":"2024-09-20T19:00:11.237123+0000","last_clean":"2024-09-20T19:00:11.237123+0000","last_became_active":"2024-09-20T18:58:33.827098+0000","last_became_peered":"2024-09-20T18:58:33.827098+0000","last_unstale":"2024-09-20T19:00:11.237123+0000","last_undegraded":"2024-09-20T19:00:11.237123+0000","last_fullsized":"2024-09-20T19:00:11.237123+0000","mapping_epoch":246,"log_start":"257'2200","ondisk_log_start":"257'2200","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":4839,"log_dups_size":0,"ondisk_log_size":4839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":461307904,"num_objects":7039,"num_object_clones":0,"num_object_copies":21117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7039,"num_write_kb":450496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"274'27","reported_seq":512,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.234926+0000","last_change":"2024-09-20T18:55:18.581290+0000","last_active":"2024-09-20T19:00:11.234926+0000","last_peered":"2024-09-20T19:00:11.234926+0000","last_clean":"2024-09-20T19:00:11.234926+0000","last_became_active":"2024-09-20T18:49:58.992302+0000","last_became_peered":"2024-09-20T18:49:58.992302+0000","last_unstale":"2024-09-20T19:00:11.234926+0000","last_undegraded":"2024-09-20T19:00:11.234926+0000","last_fullsized":"2024-09-20T19:00:11.234926+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:55:18.581200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:18.581200+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":479,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.237598+0000","last_change":"2024-09-20T18:54:54.288199+0000","last_active":"2024-09-20T19:00:11.237598+0000","last_peered":"2024-09-20T19:00:11.237598+0000","last_clean":"2024-09-20T19:00:11.237598+0000","last_became_active":"2024-09-20T18:49:45.936104+0000","last_became_peered":"2024-09-20T18:49:45.936104+0000","last_unstale":"2024-09-20T19:00:11.237598+0000","last_undegraded":"2024-09-20T19:00:11.237598+0000","last_fullsized":"2024-09-20T19:00:11.237598+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:54.288117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:54.288117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":46,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":607,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.233471+0000","last_change":"2024-09-20T18:54:14.292660+0000","last_active":"2024-09-20T19:00:10.233471+0000","last_peered":"2024-09-20T19:00:10.233471+0000","last_clean":"2024-09-20T19:00:10.233471+0000","last_became_active":"2024-09-20T18:49:49.960697+0000","last_became_peered":"2024-09-20T18:49:49.960697+0000","last_unstale":"2024-09-20T19:00:10.233471+0000","last_undegraded":"2024-09-20T19:00:10.233471+0000","last_fullsized":"2024-09-20T19:00:10.233471+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:54:14.292610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:14.292610+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:13.283791+0000","scrub_duration":34,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":473,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.237867+0000","last_change":"2024-09-20T18:58:31.493658+0000","last_active":"2024-09-20T19:00:10.237867+0000","last_peered":"2024-09-20T19:00:10.237867+0000","last_clean":"2024-09-20T19:00:10.237867+0000","last_became_active":"2024-09-20T18:49:51.971399+0000","last_became_peered":"2024-09-20T18:49:51.971399+0000","last_unstale":"2024-09-20T19:00:10.237867+0000","last_undegraded":"2024-09-20T19:00:10.237867+0000","last_fullsized":"2024-09-20T19:00:10.237867+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:31.493595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:31.493595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"282'7152","reported_seq":7237,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.241904+0000","last_change":"2024-09-20T18:58:33.820158+0000","last_active":"2024-09-20T19:00:10.241904+0000","last_peered":"2024-09-20T19:00:10.241904+0000","last_clean":"2024-09-20T19:00:10.241904+0000","last_became_active":"2024-09-20T18:58:33.819886+0000","last_became_peered":"2024-09-20T18:58:33.819886+0000","last_unstale":"2024-09-20T19:00:10.241904+0000","last_undegraded":"2024-09-20T19:00:10.241904+0000","last_fullsized":"2024-09-20T19:00:10.241904+0000","mapping_epoch":246,"log_start":"258'2356","ondisk_log_start":"258'2356","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":4796,"log_dups_size":0,"ondisk_log_size":4796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":468713472,"num_objects":7152,"num_object_clones":0,"num_object_copies":21456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7152,"num_write_kb":457728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"273'30","reported_seq":534,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.237211+0000","last_change":"2024-09-20T18:58:08.025426+0000","last_active":"2024-09-20T19:00:11.237211+0000","last_peered":"2024-09-20T19:00:11.237211+0000","last_clean":"2024-09-20T19:00:11.237211+0000","last_became_active":"2024-09-20T18:49:48.499189+0000","last_became_peered":"2024-09-20T18:49:48.499189+0000","last_unstale":"2024-09-20T19:00:11.237211+0000","last_undegraded":"2024-09-20T19:00:11.237211+0000","last_fullsized":"2024-09-20T19:00:11.237211+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-20T18:58:08.025202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:08.025202+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":479,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.239821+0000","last_change":"2024-09-20T18:58:25.159724+0000","last_active":"2024-09-20T19:00:11.239821+0000","last_peered":"2024-09-20T19:00:11.239821+0000","last_clean":"2024-09-20T19:00:11.239821+0000","last_became_active":"2024-09-20T18:49:45.940544+0000","last_became_peered":"2024-09-20T18:49:45.940544+0000","last_unstale":"2024-09-20T19:00:11.239821+0000","last_undegraded":"2024-09-20T19:00:11.239821+0000","last_fullsized":"2024-09-20T19:00:11.239821+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:25.159673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:25.159673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":479,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.233644+0000","last_change":"2024-09-20T18:54:51.229251+0000","last_active":"2024-09-20T19:00:10.233644+0000","last_peered":"2024-09-20T19:00:10.233644+0000","last_clean":"2024-09-20T19:00:10.233644+0000","last_became_active":"2024-09-20T18:49:49.963073+0000","last_became_peered":"2024-09-20T18:49:49.963073+0000","last_unstale":"2024-09-20T19:00:10.233644+0000","last_undegraded":"2024-09-20T19:00:10.233644+0000","last_fullsized":"2024-09-20T19:00:10.233644+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.229201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.229201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:21.230603+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":483,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.232533+0000","last_change":"2024-09-20T18:58:27.704583+0000","last_active":"2024-09-20T19:00:10.232533+0000","last_peered":"2024-09-20T19:00:10.232533+0000","last_clean":"2024-09-20T19:00:10.232533+0000","last_became_active":"2024-09-20T18:49:51.968793+0000","last_became_peered":"2024-09-20T18:49:51.968793+0000","last_unstale":"2024-09-20T19:00:10.232533+0000","last_undegraded":"2024-09-20T19:00:10.232533+0000","last_fullsized":"2024-09-20T19:00:10.232533+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:27.704520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:27.704520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.c","version":"282'7072","reported_seq":7159,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.237576+0000","last_change":"2024-09-20T18:58:33.845752+0000","last_active":"2024-09-20T19:00:11.237576+0000","last_peered":"2024-09-20T19:00:11.237576+0000","last_clean":"2024-09-20T19:00:11.237576+0000","last_became_active":"2024-09-20T18:58:33.845463+0000","last_became_peered":"2024-09-20T18:58:33.845463+0000","last_unstale":"2024-09-20T19:00:11.237576+0000","last_undegraded":"2024-09-20T19:00:11.237576+0000","last_fullsized":"2024-09-20T19:00:11.237576+0000","mapping_epoch":246,"log_start":"257'2300","ondisk_log_start":"257'2300","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":4772,"log_dups_size":0,"ondisk_log_size":4772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":463470592,"num_objects":7072,"num_object_clones":0,"num_object_copies":21216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7072,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7072,"num_write_kb":452608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"274'28","reported_seq":579,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.233731+0000","last_change":"2024-09-20T18:59:05.105526+0000","last_active":"2024-09-20T19:00:10.233731+0000","last_peered":"2024-09-20T19:00:10.233731+0000","last_clean":"2024-09-20T19:00:10.233731+0000","last_became_active":"2024-09-20T18:59:05.074532+0000","last_became_peered":"2024-09-20T18:59:05.074532+0000","last_unstale":"2024-09-20T19:00:10.233731+0000","last_undegraded":"2024-09-20T19:00:10.233731+0000","last_fullsized":"2024-09-20T19:00:10.233731+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-20T18:54:57.394893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.394893+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:23.216727+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":481,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.241962+0000","last_change":"2024-09-20T18:58:29.939909+0000","last_active":"2024-09-20T19:00:11.241962+0000","last_peered":"2024-09-20T19:00:11.241962+0000","last_clean":"2024-09-20T19:00:11.241962+0000","last_became_active":"2024-09-20T18:49:45.940469+0000","last_became_peered":"2024-09-20T18:49:45.940469+0000","last_unstale":"2024-09-20T19:00:11.241962+0000","last_undegraded":"2024-09-20T19:00:11.241962+0000","last_fullsized":"2024-09-20T19:00:11.241962+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:58:29.939813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:29.939813+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"66'8","reported_seq":622,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.236866+0000","last_change":"2024-09-20T18:58:23.114419+0000","last_active":"2024-09-20T19:00:11.236866+0000","last_peered":"2024-09-20T19:00:11.236866+0000","last_clean":"2024-09-20T19:00:11.236866+0000","last_became_active":"2024-09-20T18:49:49.956133+0000","last_became_peered":"2024-09-20T18:49:49.956133+0000","last_unstale":"2024-09-20T19:00:11.236866+0000","last_undegraded":"2024-09-20T19:00:11.236866+0000","last_fullsized":"2024-09-20T19:00:11.236866+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:58:23.114371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:23.114371+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":488,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.241135+0000","last_change":"2024-09-20T18:59:05.082117+0000","last_active":"2024-09-20T19:00:10.241135+0000","last_peered":"2024-09-20T19:00:10.241135+0000","last_clean":"2024-09-20T19:00:10.241135+0000","last_became_active":"2024-09-20T18:59:05.081814+0000","last_became_peered":"2024-09-20T18:59:05.081814+0000","last_unstale":"2024-09-20T19:00:10.241135+0000","last_undegraded":"2024-09-20T19:00:10.241135+0000","last_fullsized":"2024-09-20T19:00:10.241135+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:29.530001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:29.530001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"282'7066","reported_seq":7151,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.237838+0000","last_change":"2024-09-20T18:58:33.826834+0000","last_active":"2024-09-20T19:00:10.237838+0000","last_peered":"2024-09-20T19:00:10.237838+0000","last_clean":"2024-09-20T19:00:10.237838+0000","last_became_active":"2024-09-20T18:58:33.826568+0000","last_became_peered":"2024-09-20T18:58:33.826568+0000","last_unstale":"2024-09-20T19:00:10.237838+0000","last_undegraded":"2024-09-20T19:00:10.237838+0000","last_fullsized":"2024-09-20T19:00:10.237838+0000","mapping_epoch":246,"log_start":"255'1700","ondisk_log_start":"255'1700","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":5366,"log_dups_size":0,"ondisk_log_size":5366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":463077376,"num_objects":7066,"num_object_clones":0,"num_object_copies":21198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7066,"num_write_kb":452224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"274'20","reported_seq":529,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.235725+0000","last_change":"2024-09-20T18:59:06.064691+0000","last_active":"2024-09-20T19:00:11.235725+0000","last_peered":"2024-09-20T19:00:11.235725+0000","last_clean":"2024-09-20T19:00:11.235725+0000","last_became_active":"2024-09-20T18:49:47.949626+0000","last_became_peered":"2024-09-20T18:49:47.949626+0000","last_unstale":"2024-09-20T19:00:11.235725+0000","last_undegraded":"2024-09-20T19:00:11.235725+0000","last_fullsized":"2024-09-20T19:00:11.235725+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:59:06.064599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:59:06.064599+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:33.522592+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":493,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.238407+0000","last_change":"2024-09-20T18:58:32.528966+0000","last_active":"2024-09-20T19:00:10.238407+0000","last_peered":"2024-09-20T19:00:10.238407+0000","last_clean":"2024-09-20T19:00:10.238407+0000","last_became_active":"2024-09-20T18:49:45.942496+0000","last_became_peered":"2024-09-20T18:49:45.942496+0000","last_unstale":"2024-09-20T19:00:10.238407+0000","last_undegraded":"2024-09-20T19:00:10.238407+0000","last_fullsized":"2024-09-20T19:00:10.238407+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.528310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.528310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"262'10","reported_seq":636,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.241094+0000","last_change":"2024-09-20T18:59:05.098010+0000","last_active":"2024-09-20T19:00:11.241094+0000","last_peered":"2024-09-20T19:00:11.241094+0000","last_clean":"2024-09-20T19:00:11.241094+0000","last_became_active":"2024-09-20T18:59:05.080183+0000","last_became_peered":"2024-09-20T18:59:05.080183+0000","last_unstale":"2024-09-20T19:00:11.241094+0000","last_undegraded":"2024-09-20T19:00:11.241094+0000","last_fullsized":"2024-09-20T19:00:11.241094+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'9","last_scrub_stamp":"2024-09-20T18:59:00.055110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:59:00.055110+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":461,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.240668+0000","last_change":"2024-09-20T18:58:26.948165+0000","last_active":"2024-09-20T19:00:11.240668+0000","last_peered":"2024-09-20T19:00:11.240668+0000","last_clean":"2024-09-20T19:00:11.240668+0000","last_became_active":"2024-09-20T18:49:51.964780+0000","last_became_peered":"2024-09-20T18:49:51.964780+0000","last_unstale":"2024-09-20T19:00:11.240668+0000","last_undegraded":"2024-09-20T19:00:11.240668+0000","last_fullsized":"2024-09-20T19:00:11.240668+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:26.948102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:26.948102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"282'7199","reported_seq":7284,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.242483+0000","last_change":"2024-09-20T18:58:33.845804+0000","last_active":"2024-09-20T19:00:10.242483+0000","last_peered":"2024-09-20T19:00:10.242483+0000","last_clean":"2024-09-20T19:00:10.242483+0000","last_became_active":"2024-09-20T18:58:33.845493+0000","last_became_peered":"2024-09-20T18:58:33.845493+0000","last_unstale":"2024-09-20T19:00:10.242483+0000","last_undegraded":"2024-09-20T19:00:10.242483+0000","last_fullsized":"2024-09-20T19:00:10.242483+0000","mapping_epoch":246,"log_start":"258'2407","ondisk_log_start":"258'2407","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":4792,"log_dups_size":0,"ondisk_log_size":4792,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":471793664,"num_objects":7199,"num_object_clones":0,"num_object_copies":21597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7199,"num_write_kb":460736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"274'29","reported_seq":535,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.241209+0000","last_change":"2024-09-20T19:00:04.810907+0000","last_active":"2024-09-20T19:00:11.241209+0000","last_peered":"2024-09-20T19:00:11.241209+0000","last_clean":"2024-09-20T19:00:11.241209+0000","last_became_active":"2024-09-20T18:49:48.497736+0000","last_became_peered":"2024-09-20T18:49:48.497736+0000","last_unstale":"2024-09-20T19:00:11.241209+0000","last_undegraded":"2024-09-20T19:00:11.241209+0000","last_fullsized":"2024-09-20T19:00:11.241209+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'29","last_scrub_stamp":"2024-09-20T19:00:04.810831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:00:04.810831+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:28.033769+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":471,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.237261+0000","last_change":"2024-09-20T18:57:09.940634+0000","last_active":"2024-09-20T19:00:11.237261+0000","last_peered":"2024-09-20T19:00:11.237261+0000","last_clean":"2024-09-20T19:00:11.237261+0000","last_became_active":"2024-09-20T18:49:45.942253+0000","last_became_peered":"2024-09-20T18:49:45.942253+0000","last_unstale":"2024-09-20T19:00:11.237261+0000","last_undegraded":"2024-09-20T19:00:11.237261+0000","last_fullsized":"2024-09-20T19:00:11.237261+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:09.940517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:09.940517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":56,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.246060+0000","last_change":"2024-09-20T18:59:05.085948+0000","last_active":"2024-09-20T19:00:10.246060+0000","last_peered":"2024-09-20T19:00:10.246060+0000","last_clean":"2024-09-20T19:00:10.246060+0000","last_became_active":"2024-09-20T18:59:05.085636+0000","last_became_peered":"2024-09-20T18:59:05.085636+0000","last_unstale":"2024-09-20T19:00:10.246060+0000","last_undegraded":"2024-09-20T19:00:10.246060+0000","last_fullsized":"2024-09-20T19:00:10.246060+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:50.610871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:50.610871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":497,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.236020+0000","last_change":"2024-09-20T18:59:05.085472+0000","last_active":"2024-09-20T19:00:11.236020+0000","last_peered":"2024-09-20T19:00:11.236020+0000","last_clean":"2024-09-20T19:00:11.236020+0000","last_became_active":"2024-09-20T18:59:05.085131+0000","last_became_peered":"2024-09-20T18:59:05.085131+0000","last_unstale":"2024-09-20T19:00:11.236020+0000","last_undegraded":"2024-09-20T19:00:11.236020+0000","last_fullsized":"2024-09-20T19:00:11.236020+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:58.973176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:58.973176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"282'7205","reported_seq":7292,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.240572+0000","last_change":"2024-09-20T18:58:33.822011+0000","last_active":"2024-09-20T19:00:11.240572+0000","last_peered":"2024-09-20T19:00:11.240572+0000","last_clean":"2024-09-20T19:00:11.240572+0000","last_became_active":"2024-09-20T18:58:33.820506+0000","last_became_peered":"2024-09-20T18:58:33.820506+0000","last_unstale":"2024-09-20T19:00:11.240572+0000","last_undegraded":"2024-09-20T19:00:11.240572+0000","last_fullsized":"2024-09-20T19:00:11.240572+0000","mapping_epoch":246,"log_start":"253'1300","ondisk_log_start":"253'1300","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":5905,"log_dups_size":0,"ondisk_log_size":5905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":472186880,"num_objects":7205,"num_object_clones":0,"num_object_copies":21615,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7205,"num_write_kb":461120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"274'40","reported_seq":675,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.237131+0000","last_change":"2024-09-20T18:58:07.018449+0000","last_active":"2024-09-20T19:00:11.237131+0000","last_peered":"2024-09-20T19:00:11.237131+0000","last_clean":"2024-09-20T19:00:11.237131+0000","last_became_active":"2024-09-20T18:49:47.955335+0000","last_became_peered":"2024-09-20T18:49:47.955335+0000","last_unstale":"2024-09-20T19:00:11.237131+0000","last_undegraded":"2024-09-20T19:00:11.237131+0000","last_fullsized":"2024-09-20T19:00:11.237131+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'28","last_scrub_stamp":"2024-09-20T18:58:07.018340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:07.018340+0000","objects_scrubbed":10,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":486,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.233220+0000","last_change":"2024-09-20T18:59:05.082509+0000","last_active":"2024-09-20T19:00:10.233220+0000","last_peered":"2024-09-20T19:00:10.233220+0000","last_clean":"2024-09-20T19:00:10.233220+0000","last_became_active":"2024-09-20T18:59:05.081468+0000","last_became_peered":"2024-09-20T18:59:05.081468+0000","last_unstale":"2024-09-20T19:00:10.233220+0000","last_undegraded":"2024-09-20T19:00:10.233220+0000","last_fullsized":"2024-09-20T19:00:10.233220+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:21.807703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:21.807703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:26.213224+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":534,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.237745+0000","last_change":"2024-09-20T19:00:02.437598+0000","last_active":"2024-09-20T19:00:10.237745+0000","last_peered":"2024-09-20T19:00:10.237745+0000","last_clean":"2024-09-20T19:00:10.237745+0000","last_became_active":"2024-09-20T18:59:05.086786+0000","last_became_peered":"2024-09-20T18:59:05.086786+0000","last_unstale":"2024-09-20T19:00:10.237745+0000","last_undegraded":"2024-09-20T19:00:10.237745+0000","last_fullsized":"2024-09-20T19:00:10.237745+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-20T19:00:02.437550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:00:02.437550+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:09.312576+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0012781159999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":488,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.241712+0000","last_change":"2024-09-20T18:58:59.099799+0000","last_active":"2024-09-20T19:00:11.241712+0000","last_peered":"2024-09-20T19:00:11.241712+0000","last_clean":"2024-09-20T19:00:11.241712+0000","last_became_active":"2024-09-20T18:49:49.953971+0000","last_became_peered":"2024-09-20T18:49:49.953971+0000","last_unstale":"2024-09-20T19:00:11.241712+0000","last_undegraded":"2024-09-20T19:00:11.241712+0000","last_fullsized":"2024-09-20T19:00:11.241712+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:59.099736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:59.099736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:28.205403+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":459,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.238147+0000","last_change":"2024-09-20T18:57:12.744116+0000","last_active":"2024-09-20T19:00:11.238147+0000","last_peered":"2024-09-20T19:00:11.238147+0000","last_clean":"2024-09-20T19:00:11.238147+0000","last_became_active":"2024-09-20T18:49:51.969400+0000","last_became_peered":"2024-09-20T18:49:51.969400+0000","last_unstale":"2024-09-20T19:00:11.238147+0000","last_undegraded":"2024-09-20T19:00:11.238147+0000","last_fullsized":"2024-09-20T19:00:11.238147+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:12.744040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:12.744040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"282'7213","reported_seq":7298,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.242093+0000","last_change":"2024-09-20T18:58:33.829042+0000","last_active":"2024-09-20T19:00:10.242093+0000","last_peered":"2024-09-20T19:00:10.242093+0000","last_clean":"2024-09-20T19:00:10.242093+0000","last_became_active":"2024-09-20T18:58:33.828787+0000","last_became_peered":"2024-09-20T18:58:33.828787+0000","last_unstale":"2024-09-20T19:00:10.242093+0000","last_undegraded":"2024-09-20T19:00:10.242093+0000","last_fullsized":"2024-09-20T19:00:10.242093+0000","mapping_epoch":246,"log_start":"255'1800","ondisk_log_start":"255'1800","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":5413,"log_dups_size":0,"ondisk_log_size":5413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":472711168,"num_objects":7213,"num_object_clones":0,"num_object_copies":21639,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7213,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7213,"num_write_kb":461632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"273'17","reported_seq":524,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.240113+0000","last_change":"2024-09-20T18:58:06.079117+0000","last_active":"2024-09-20T19:00:11.240113+0000","last_peered":"2024-09-20T19:00:11.240113+0000","last_clean":"2024-09-20T19:00:11.240113+0000","last_became_active":"2024-09-20T18:49:47.950904+0000","last_became_peered":"2024-09-20T18:49:47.950904+0000","last_unstale":"2024-09-20T19:00:11.240113+0000","last_undegraded":"2024-09-20T19:00:11.240113+0000","last_fullsized":"2024-09-20T19:00:11.240113+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-20T18:58:06.078352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:06.078352+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":59,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":36,"num_read_kb":25,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":510,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.238451+0000","last_change":"2024-09-20T18:59:05.080117+0000","last_active":"2024-09-20T19:00:11.238451+0000","last_peered":"2024-09-20T19:00:11.238451+0000","last_clean":"2024-09-20T19:00:11.238451+0000","last_became_active":"2024-09-20T18:59:05.079789+0000","last_became_peered":"2024-09-20T18:59:05.079789+0000","last_unstale":"2024-09-20T19:00:11.238451+0000","last_undegraded":"2024-09-20T19:00:11.238451+0000","last_fullsized":"2024-09-20T19:00:11.238451+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:28.987458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:28.987458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":492,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.234073+0000","last_change":"2024-09-20T18:58:24.229997+0000","last_active":"2024-09-20T19:00:11.234073+0000","last_peered":"2024-09-20T19:00:11.234073+0000","last_clean":"2024-09-20T19:00:11.234073+0000","last_became_active":"2024-09-20T18:49:44.937135+0000","last_became_peered":"2024-09-20T18:49:44.937135+0000","last_unstale":"2024-09-20T19:00:11.234073+0000","last_undegraded":"2024-09-20T19:00:11.234073+0000","last_fullsized":"2024-09-20T19:00:11.234073+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:58:24.229938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:58:24.229938+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00095558800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":466,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.234741+0000","last_change":"2024-09-20T18:51:59.796450+0000","last_active":"2024-09-20T19:00:11.234741+0000","last_peered":"2024-09-20T19:00:11.234741+0000","last_clean":"2024-09-20T19:00:11.234741+0000","last_became_active":"2024-09-20T18:49:49.954424+0000","last_became_peered":"2024-09-20T18:49:49.954424+0000","last_unstale":"2024-09-20T19:00:11.234741+0000","last_undegraded":"2024-09-20T19:00:11.234741+0000","last_fullsized":"2024-09-20T19:00:11.234741+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:59.796383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:59.796383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":475,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.241809+0000","last_change":"2024-09-20T18:59:05.082433+0000","last_active":"2024-09-20T19:00:10.241809+0000","last_peered":"2024-09-20T19:00:10.241809+0000","last_clean":"2024-09-20T19:00:10.241809+0000","last_became_active":"2024-09-20T18:59:05.081459+0000","last_became_peered":"2024-09-20T18:59:05.081459+0000","last_unstale":"2024-09-20T19:00:10.241809+0000","last_undegraded":"2024-09-20T19:00:10.241809+0000","last_fullsized":"2024-09-20T19:00:10.241809+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:39.814259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:39.814259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"282'6962","reported_seq":13164,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.241121+0000","last_change":"2024-09-20T19:00:03.218867+0000","last_active":"2024-09-20T19:00:11.241121+0000","last_peered":"2024-09-20T19:00:11.241121+0000","last_clean":"2024-09-20T19:00:11.241121+0000","last_became_active":"2024-09-20T19:00:03.209632+0000","last_became_peered":"2024-09-20T19:00:03.209632+0000","last_unstale":"2024-09-20T19:00:11.241121+0000","last_undegraded":"2024-09-20T19:00:11.241121+0000","last_fullsized":"2024-09-20T19:00:11.241121+0000","mapping_epoch":279,"log_start":"253'1000","ondisk_log_start":"253'1000","created":246,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":5962,"log_dups_size":0,"ondisk_log_size":5962,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":456261632,"num_objects":6962,"num_object_clones":0,"num_object_copies":20886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6962,"num_write_kb":445568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3051,"num_bytes_recovered":199950336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"273'24","reported_seq":536,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.241870+0000","last_change":"2024-09-20T18:59:06.022021+0000","last_active":"2024-09-20T19:00:11.241870+0000","last_peered":"2024-09-20T19:00:11.241870+0000","last_clean":"2024-09-20T19:00:11.241870+0000","last_became_active":"2024-09-20T18:49:47.949230+0000","last_became_peered":"2024-09-20T18:49:47.949230+0000","last_unstale":"2024-09-20T19:00:11.241870+0000","last_undegraded":"2024-09-20T19:00:11.241870+0000","last_fullsized":"2024-09-20T19:00:11.241870+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:59:06.021962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:59:06.021962+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:12.002280+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":488,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.240724+0000","last_change":"2024-09-20T18:54:27.905856+0000","last_active":"2024-09-20T19:00:11.240724+0000","last_peered":"2024-09-20T19:00:11.240724+0000","last_clean":"2024-09-20T19:00:11.240724+0000","last_became_active":"2024-09-20T18:49:45.939197+0000","last_became_peered":"2024-09-20T18:49:45.939197+0000","last_unstale":"2024-09-20T19:00:11.240724+0000","last_undegraded":"2024-09-20T19:00:11.240724+0000","last_fullsized":"2024-09-20T19:00:11.240724+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:27.905803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:27.905803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:09.859516+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":1117,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.240428+0000","last_change":"2024-09-20T18:54:26.359887+0000","last_active":"2024-09-20T19:00:10.240428+0000","last_peered":"2024-09-20T19:00:10.240428+0000","last_clean":"2024-09-20T19:00:10.240428+0000","last_became_active":"2024-09-20T18:49:44.936819+0000","last_became_peered":"2024-09-20T18:49:44.936819+0000","last_unstale":"2024-09-20T19:00:10.240428+0000","last_undegraded":"2024-09-20T19:00:10.240428+0000","last_fullsized":"2024-09-20T19:00:10.240428+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-20T18:54:26.359807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:54:26.359807+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0010996199999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":619,"num_read_kb":614,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":477,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.237916+0000","last_change":"2024-09-20T18:57:54.362085+0000","last_active":"2024-09-20T19:00:10.237916+0000","last_peered":"2024-09-20T19:00:10.237916+0000","last_clean":"2024-09-20T19:00:10.237916+0000","last_became_active":"2024-09-20T18:49:49.961066+0000","last_became_peered":"2024-09-20T18:49:49.961066+0000","last_unstale":"2024-09-20T19:00:10.237916+0000","last_undegraded":"2024-09-20T19:00:10.237916+0000","last_fullsized":"2024-09-20T19:00:10.237916+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:54.362038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:54.362038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":483,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.235375+0000","last_change":"2024-09-20T18:58:08.019154+0000","last_active":"2024-09-20T19:00:11.235375+0000","last_peered":"2024-09-20T19:00:11.235375+0000","last_clean":"2024-09-20T19:00:11.235375+0000","last_became_active":"2024-09-20T18:49:51.970343+0000","last_became_peered":"2024-09-20T18:49:51.970343+0000","last_unstale":"2024-09-20T19:00:11.235375+0000","last_undegraded":"2024-09-20T19:00:11.235375+0000","last_fullsized":"2024-09-20T19:00:11.235375+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:08.019092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:08.019092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"282'6947","reported_seq":7034,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.237371+0000","last_change":"2024-09-20T18:58:33.820307+0000","last_active":"2024-09-20T19:00:11.237371+0000","last_peered":"2024-09-20T19:00:11.237371+0000","last_clean":"2024-09-20T19:00:11.237371+0000","last_became_active":"2024-09-20T18:58:33.820027+0000","last_became_peered":"2024-09-20T18:58:33.820027+0000","last_unstale":"2024-09-20T19:00:11.237371+0000","last_undegraded":"2024-09-20T19:00:11.237371+0000","last_fullsized":"2024-09-20T19:00:11.237371+0000","mapping_epoch":246,"log_start":"256'2100","ondisk_log_start":"256'2100","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":4847,"log_dups_size":0,"ondisk_log_size":4847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":455278592,"num_objects":6947,"num_object_clones":0,"num_object_copies":20841,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6947,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6947,"num_write_kb":444608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"274'30","reported_seq":105,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.246758+0000","last_change":"2024-09-20T18:59:05.102446+0000","last_active":"2024-09-20T19:00:10.246758+0000","last_peered":"2024-09-20T19:00:10.246758+0000","last_clean":"2024-09-20T19:00:10.246758+0000","last_became_active":"2024-09-20T18:59:05.088069+0000","last_became_peered":"2024-09-20T18:59:05.088069+0000","last_unstale":"2024-09-20T19:00:10.246758+0000","last_undegraded":"2024-09-20T19:00:10.246758+0000","last_fullsized":"2024-09-20T19:00:10.246758+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'17","last_scrub_stamp":"2024-09-20T18:57:22.816154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:22.816154+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":501,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.238520+0000","last_change":"2024-09-20T18:59:05.079328+0000","last_active":"2024-09-20T19:00:11.238520+0000","last_peered":"2024-09-20T19:00:11.238520+0000","last_clean":"2024-09-20T19:00:11.238520+0000","last_became_active":"2024-09-20T18:59:05.079111+0000","last_became_peered":"2024-09-20T18:59:05.079111+0000","last_unstale":"2024-09-20T19:00:11.238520+0000","last_undegraded":"2024-09-20T19:00:11.238520+0000","last_fullsized":"2024-09-20T19:00:11.238520+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.100625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.100625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":554,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.240531+0000","last_change":"2024-09-20T18:59:05.108551+0000","last_active":"2024-09-20T19:00:10.240531+0000","last_peered":"2024-09-20T19:00:10.240531+0000","last_clean":"2024-09-20T19:00:10.240531+0000","last_became_active":"2024-09-20T18:59:05.086103+0000","last_became_peered":"2024-09-20T18:59:05.086103+0000","last_unstale":"2024-09-20T19:00:10.240531+0000","last_undegraded":"2024-09-20T19:00:10.240531+0000","last_fullsized":"2024-09-20T19:00:10.240531+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T18:55:13.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T18:55:13.696607+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":470,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.241569+0000","last_change":"2024-09-20T18:58:23.545370+0000","last_active":"2024-09-20T19:00:10.241569+0000","last_peered":"2024-09-20T19:00:10.241569+0000","last_clean":"2024-09-20T19:00:10.241569+0000","last_became_active":"2024-09-20T18:49:49.956128+0000","last_became_peered":"2024-09-20T18:49:49.956128+0000","last_unstale":"2024-09-20T19:00:10.241569+0000","last_undegraded":"2024-09-20T19:00:10.241569+0000","last_fullsized":"2024-09-20T19:00:10.241569+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:23.545321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:23.545321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":481,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.237705+0000","last_change":"2024-09-20T18:58:03.523127+0000","last_active":"2024-09-20T19:00:10.237705+0000","last_peered":"2024-09-20T19:00:10.237705+0000","last_clean":"2024-09-20T19:00:10.237705+0000","last_became_active":"2024-09-20T18:49:51.971332+0000","last_became_peered":"2024-09-20T18:49:51.971332+0000","last_unstale":"2024-09-20T19:00:10.237705+0000","last_undegraded":"2024-09-20T19:00:10.237705+0000","last_fullsized":"2024-09-20T19:00:10.237705+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:03.523078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:03.523078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"282'7182","reported_seq":7267,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.241084+0000","last_change":"2024-09-20T18:58:33.825100+0000","last_active":"2024-09-20T19:00:10.241084+0000","last_peered":"2024-09-20T19:00:10.241084+0000","last_clean":"2024-09-20T19:00:10.241084+0000","last_became_active":"2024-09-20T18:58:33.824751+0000","last_became_peered":"2024-09-20T18:58:33.824751+0000","last_unstale":"2024-09-20T19:00:10.241084+0000","last_undegraded":"2024-09-20T19:00:10.241084+0000","last_fullsized":"2024-09-20T19:00:10.241084+0000","mapping_epoch":246,"log_start":"258'2352","ondisk_log_start":"258'2352","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":4830,"log_dups_size":0,"ondisk_log_size":4830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470679552,"num_objects":7182,"num_object_clones":0,"num_object_copies":21546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7182,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7182,"num_write_kb":459648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"274'19","reported_seq":544,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.233522+0000","last_change":"2024-09-20T18:59:05.115678+0000","last_active":"2024-09-20T19:00:10.233522+0000","last_peered":"2024-09-20T19:00:10.233522+0000","last_clean":"2024-09-20T19:00:10.233522+0000","last_became_active":"2024-09-20T18:59:05.079753+0000","last_became_peered":"2024-09-20T18:59:05.079753+0000","last_unstale":"2024-09-20T19:00:10.233522+0000","last_undegraded":"2024-09-20T19:00:10.233522+0000","last_fullsized":"2024-09-20T19:00:10.233522+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:58:24.616580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:24.616580+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":479,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.235794+0000","last_change":"2024-09-20T18:58:32.694078+0000","last_active":"2024-09-20T19:00:10.235794+0000","last_peered":"2024-09-20T19:00:10.235794+0000","last_clean":"2024-09-20T19:00:10.235794+0000","last_became_active":"2024-09-20T18:49:45.944579+0000","last_became_peered":"2024-09-20T18:49:45.944579+0000","last_unstale":"2024-09-20T19:00:10.235794+0000","last_undegraded":"2024-09-20T19:00:10.235794+0000","last_fullsized":"2024-09-20T19:00:10.235794+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.693993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.693993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":499,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.237454+0000","last_change":"2024-09-20T19:00:01.457399+0000","last_active":"2024-09-20T19:00:10.237454+0000","last_peered":"2024-09-20T19:00:10.237454+0000","last_clean":"2024-09-20T19:00:10.237454+0000","last_became_active":"2024-09-20T18:59:05.090504+0000","last_became_peered":"2024-09-20T18:59:05.090504+0000","last_unstale":"2024-09-20T19:00:10.237454+0000","last_undegraded":"2024-09-20T19:00:10.237454+0000","last_fullsized":"2024-09-20T19:00:10.237454+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:00:01.457330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:00:01.457330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:21.889350+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":489,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.238601+0000","last_change":"2024-09-20T18:59:05.080255+0000","last_active":"2024-09-20T19:00:11.238601+0000","last_peered":"2024-09-20T19:00:11.238601+0000","last_clean":"2024-09-20T19:00:11.238601+0000","last_became_active":"2024-09-20T18:59:05.079897+0000","last_became_peered":"2024-09-20T18:59:05.079897+0000","last_unstale":"2024-09-20T19:00:11.238601+0000","last_undegraded":"2024-09-20T19:00:11.238601+0000","last_fullsized":"2024-09-20T19:00:11.238601+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:01.083156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:59:01.083156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:15.111438+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.0","version":"282'7383","reported_seq":14016,"reported_epoch":283,"state":"peering","last_fresh":"2024-09-20T19:00:11.124959+0000","last_change":"2024-09-20T19:00:11.238009+0000","last_active":"2024-09-20T19:00:11.238009+0000","last_peered":"2024-09-20T19:00:11.124959+0000","last_clean":"2024-09-20T18:59:04.061108+0000","last_became_active":"2024-09-20T18:59:06.197951+0000","last_became_peered":"2024-09-20T18:59:06.197951+0000","last_unstale":"2024-09-20T19:00:11.124959+0000","last_undegraded":"2024-09-20T19:00:11.124959+0000","last_fullsized":"2024-09-20T18:59:06.066740+0000","mapping_epoch":284,"log_start":"258'2600","ondisk_log_start":"258'2600","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":4783,"log_dups_size":0,"ondisk_log_size":4783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483852288,"num_objects":7383,"num_object_clones":0,"num_object_copies":22149,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7383,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7383,"num_write_kb":472512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3270,"num_bytes_recovered":214302720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2,6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"274'31","reported_seq":540,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.233396+0000","last_change":"2024-09-20T19:00:03.510074+0000","last_active":"2024-09-20T19:00:10.233396+0000","last_peered":"2024-09-20T19:00:10.233396+0000","last_clean":"2024-09-20T19:00:10.233396+0000","last_became_active":"2024-09-20T18:49:47.947636+0000","last_became_peered":"2024-09-20T18:49:47.947636+0000","last_unstale":"2024-09-20T19:00:10.233396+0000","last_undegraded":"2024-09-20T19:00:10.233396+0000","last_fullsized":"2024-09-20T19:00:10.233396+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'31","last_scrub_stamp":"2024-09-20T19:00:03.509980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:00:03.509980+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:07.241575+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":56,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.246142+0000","last_change":"2024-09-20T18:59:05.081802+0000","last_active":"2024-09-20T19:00:10.246142+0000","last_peered":"2024-09-20T19:00:10.246142+0000","last_clean":"2024-09-20T19:00:10.246142+0000","last_became_active":"2024-09-20T18:59:05.080829+0000","last_became_peered":"2024-09-20T18:59:05.080829+0000","last_unstale":"2024-09-20T19:00:10.246142+0000","last_undegraded":"2024-09-20T19:00:10.246142+0000","last_fullsized":"2024-09-20T19:00:10.246142+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:25.649733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:25.649733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":479,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.235485+0000","last_change":"2024-09-20T18:58:36.100033+0000","last_active":"2024-09-20T19:00:11.235485+0000","last_peered":"2024-09-20T19:00:11.235485+0000","last_clean":"2024-09-20T19:00:11.235485+0000","last_became_active":"2024-09-20T18:49:49.959160+0000","last_became_peered":"2024-09-20T18:49:49.959160+0000","last_unstale":"2024-09-20T19:00:11.235485+0000","last_undegraded":"2024-09-20T19:00:11.235485+0000","last_fullsized":"2024-09-20T19:00:11.235485+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:36.099985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:36.099985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":478,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.235028+0000","last_change":"2024-09-20T18:58:25.227679+0000","last_active":"2024-09-20T19:00:11.235028+0000","last_peered":"2024-09-20T19:00:11.235028+0000","last_clean":"2024-09-20T19:00:11.235028+0000","last_became_active":"2024-09-20T18:49:51.970108+0000","last_became_peered":"2024-09-20T18:49:51.970108+0000","last_unstale":"2024-09-20T19:00:11.235028+0000","last_undegraded":"2024-09-20T19:00:11.235028+0000","last_fullsized":"2024-09-20T19:00:11.235028+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:25.227630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:25.227630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1","version":"284'7150","reported_seq":13605,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.251789+0000","last_change":"2024-09-20T19:00:11.251453+0000","last_active":"2024-09-20T19:00:11.251789+0000","last_peered":"2024-09-20T19:00:11.251789+0000","last_clean":"2024-09-20T19:00:11.251789+0000","last_became_active":"2024-09-20T19:00:11.244922+0000","last_became_peered":"2024-09-20T19:00:11.244922+0000","last_unstale":"2024-09-20T19:00:11.251789+0000","last_undegraded":"2024-09-20T19:00:11.251789+0000","last_fullsized":"2024-09-20T19:00:11.251789+0000","mapping_epoch":283,"log_start":"258'2300","ondisk_log_start":"258'2300","created":246,"last_epoch_clean":284,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":4850,"log_dups_size":0,"ondisk_log_size":4850,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":468582400,"num_objects":7150,"num_object_clones":0,"num_object_copies":21450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7150,"num_write_kb":457600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3178,"num_bytes_recovered":208273408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"273'20","reported_seq":551,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.234793+0000","last_change":"2024-09-20T18:59:10.364668+0000","last_active":"2024-09-20T19:00:10.234793+0000","last_peered":"2024-09-20T19:00:10.234793+0000","last_clean":"2024-09-20T19:00:10.234793+0000","last_became_active":"2024-09-20T18:59:05.073564+0000","last_became_peered":"2024-09-20T18:59:05.073564+0000","last_unstale":"2024-09-20T19:00:10.234793+0000","last_undegraded":"2024-09-20T19:00:10.234793+0000","last_fullsized":"2024-09-20T19:00:10.234793+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:59:10.364583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:59:10.364583+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:23.352032+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":487,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.237881+0000","last_change":"2024-09-20T18:58:31.126134+0000","last_active":"2024-09-20T19:00:11.237881+0000","last_peered":"2024-09-20T19:00:11.237881+0000","last_clean":"2024-09-20T19:00:11.237881+0000","last_became_active":"2024-09-20T18:49:45.937264+0000","last_became_peered":"2024-09-20T18:49:45.937264+0000","last_unstale":"2024-09-20T19:00:11.237881+0000","last_undegraded":"2024-09-20T19:00:11.237881+0000","last_fullsized":"2024-09-20T19:00:11.237881+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:31.126087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:31.126087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":616,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.234268+0000","last_change":"2024-09-20T18:55:17.549660+0000","last_active":"2024-09-20T19:00:11.234268+0000","last_peered":"2024-09-20T19:00:11.234268+0000","last_clean":"2024-09-20T19:00:11.234268+0000","last_became_active":"2024-09-20T18:49:49.957005+0000","last_became_peered":"2024-09-20T18:49:49.957005+0000","last_unstale":"2024-09-20T19:00:11.234268+0000","last_undegraded":"2024-09-20T19:00:11.234268+0000","last_fullsized":"2024-09-20T19:00:11.234268+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:17.549598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:17.549598+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":483,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.233307+0000","last_change":"2024-09-20T18:59:48.398618+0000","last_active":"2024-09-20T19:00:10.233307+0000","last_peered":"2024-09-20T19:00:10.233307+0000","last_clean":"2024-09-20T19:00:10.233307+0000","last_became_active":"2024-09-20T18:49:51.969355+0000","last_became_peered":"2024-09-20T18:49:51.969355+0000","last_unstale":"2024-09-20T19:00:10.233307+0000","last_undegraded":"2024-09-20T19:00:10.233307+0000","last_fullsized":"2024-09-20T19:00:10.233307+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:48.398542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:59:48.398542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:04.009303+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"282'7099","reported_seq":7184,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.238486+0000","last_change":"2024-09-20T18:58:33.826744+0000","last_active":"2024-09-20T19:00:10.238486+0000","last_peered":"2024-09-20T19:00:10.238486+0000","last_clean":"2024-09-20T19:00:10.238486+0000","last_became_active":"2024-09-20T18:58:33.826480+0000","last_became_peered":"2024-09-20T18:58:33.826480+0000","last_unstale":"2024-09-20T19:00:10.238486+0000","last_undegraded":"2024-09-20T19:00:10.238486+0000","last_fullsized":"2024-09-20T19:00:10.238486+0000","mapping_epoch":246,"log_start":"255'1700","ondisk_log_start":"255'1700","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":5399,"log_dups_size":0,"ondisk_log_size":5399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":465240064,"num_objects":7099,"num_object_clones":0,"num_object_copies":21297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7099,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7099,"num_write_kb":454336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"274'15","reported_seq":509,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.241185+0000","last_change":"2024-09-20T18:54:54.897991+0000","last_active":"2024-09-20T19:00:11.241185+0000","last_peered":"2024-09-20T19:00:11.241185+0000","last_clean":"2024-09-20T19:00:11.241185+0000","last_became_active":"2024-09-20T18:49:47.946008+0000","last_became_peered":"2024-09-20T18:49:47.946008+0000","last_unstale":"2024-09-20T19:00:11.241185+0000","last_undegraded":"2024-09-20T19:00:11.241185+0000","last_fullsized":"2024-09-20T19:00:11.241185+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:54.897933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:54.897933+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.233942+0000","last_change":"2024-09-20T19:00:01.191220+0000","last_active":"2024-09-20T19:00:11.233942+0000","last_peered":"2024-09-20T19:00:11.233942+0000","last_clean":"2024-09-20T19:00:11.233942+0000","last_became_active":"2024-09-20T19:00:01.190875+0000","last_became_peered":"2024-09-20T19:00:01.190875+0000","last_unstale":"2024-09-20T19:00:11.233942+0000","last_undegraded":"2024-09-20T19:00:11.233942+0000","last_fullsized":"2024-09-20T19:00:11.233942+0000","mapping_epoch":277,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":278,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:56.405349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:56.405349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":482,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.234642+0000","last_change":"2024-09-20T18:54:43.379898+0000","last_active":"2024-09-20T19:00:11.234642+0000","last_peered":"2024-09-20T19:00:11.234642+0000","last_clean":"2024-09-20T19:00:11.234642+0000","last_became_active":"2024-09-20T18:49:49.956767+0000","last_became_peered":"2024-09-20T18:49:49.956767+0000","last_unstale":"2024-09-20T19:00:11.234642+0000","last_undegraded":"2024-09-20T19:00:11.234642+0000","last_fullsized":"2024-09-20T19:00:11.234642+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:43.379706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:43.379706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":468,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.240707+0000","last_change":"2024-09-20T18:54:24.976627+0000","last_active":"2024-09-20T19:00:11.240707+0000","last_peered":"2024-09-20T19:00:11.240707+0000","last_clean":"2024-09-20T19:00:11.240707+0000","last_became_active":"2024-09-20T18:49:51.964098+0000","last_became_peered":"2024-09-20T18:49:51.964098+0000","last_unstale":"2024-09-20T19:00:11.240707+0000","last_undegraded":"2024-09-20T19:00:11.240707+0000","last_fullsized":"2024-09-20T19:00:11.240707+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:24.976584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:24.976584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:10.886988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"282'7255","reported_seq":7340,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.237592+0000","last_change":"2024-09-20T18:58:33.824658+0000","last_active":"2024-09-20T19:00:10.237592+0000","last_peered":"2024-09-20T19:00:10.237592+0000","last_clean":"2024-09-20T19:00:10.237592+0000","last_became_active":"2024-09-20T18:58:33.824342+0000","last_became_peered":"2024-09-20T18:58:33.824342+0000","last_unstale":"2024-09-20T19:00:10.237592+0000","last_undegraded":"2024-09-20T19:00:10.237592+0000","last_fullsized":"2024-09-20T19:00:10.237592+0000","mapping_epoch":246,"log_start":"255'1800","ondisk_log_start":"255'1800","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":5455,"log_dups_size":0,"ondisk_log_size":5455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":475463680,"num_objects":7255,"num_object_clones":0,"num_object_copies":21765,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7255,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7255,"num_write_kb":464320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"274'35","reported_seq":100,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.246007+0000","last_change":"2024-09-20T18:59:05.107086+0000","last_active":"2024-09-20T19:00:10.246007+0000","last_peered":"2024-09-20T19:00:10.246007+0000","last_clean":"2024-09-20T19:00:10.246007+0000","last_became_active":"2024-09-20T18:59:05.087391+0000","last_became_peered":"2024-09-20T18:59:05.087391+0000","last_unstale":"2024-09-20T19:00:10.246007+0000","last_undegraded":"2024-09-20T19:00:10.246007+0000","last_fullsized":"2024-09-20T19:00:10.246007+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-20T18:57:22.033475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:22.033475+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":500,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.242636+0000","last_change":"2024-09-20T18:59:05.087282+0000","last_active":"2024-09-20T19:00:10.242636+0000","last_peered":"2024-09-20T19:00:10.242636+0000","last_clean":"2024-09-20T19:00:10.242636+0000","last_became_active":"2024-09-20T18:59:05.086670+0000","last_became_peered":"2024-09-20T18:59:05.086670+0000","last_unstale":"2024-09-20T19:00:10.242636+0000","last_undegraded":"2024-09-20T19:00:10.242636+0000","last_fullsized":"2024-09-20T19:00:10.242636+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:44.501192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.501192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":482,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.234270+0000","last_change":"2024-09-20T18:55:13.577772+0000","last_active":"2024-09-20T19:00:11.234270+0000","last_peered":"2024-09-20T19:00:11.234270+0000","last_clean":"2024-09-20T19:00:11.234270+0000","last_became_active":"2024-09-20T18:49:49.957558+0000","last_became_peered":"2024-09-20T18:49:49.957558+0000","last_unstale":"2024-09-20T19:00:11.234270+0000","last_undegraded":"2024-09-20T19:00:11.234270+0000","last_fullsized":"2024-09-20T19:00:11.234270+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:13.577699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:13.577699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":474,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.240468+0000","last_change":"2024-09-20T18:53:47.370152+0000","last_active":"2024-09-20T19:00:10.240468+0000","last_peered":"2024-09-20T19:00:10.240468+0000","last_clean":"2024-09-20T19:00:10.240468+0000","last_became_active":"2024-09-20T18:49:51.970798+0000","last_became_peered":"2024-09-20T18:49:51.970798+0000","last_unstale":"2024-09-20T19:00:10.240468+0000","last_undegraded":"2024-09-20T19:00:10.240468+0000","last_fullsized":"2024-09-20T19:00:10.240468+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:47.370108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:47.370108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.4","version":"282'7066","reported_seq":7153,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.237043+0000","last_change":"2024-09-20T18:58:33.846071+0000","last_active":"2024-09-20T19:00:11.237043+0000","last_peered":"2024-09-20T19:00:11.237043+0000","last_clean":"2024-09-20T19:00:11.237043+0000","last_became_active":"2024-09-20T18:58:33.845736+0000","last_became_peered":"2024-09-20T18:58:33.845736+0000","last_unstale":"2024-09-20T19:00:11.237043+0000","last_undegraded":"2024-09-20T19:00:11.237043+0000","last_fullsized":"2024-09-20T19:00:11.237043+0000","mapping_epoch":246,"log_start":"258'2300","ondisk_log_start":"258'2300","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":4766,"log_dups_size":0,"ondisk_log_size":4766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":463077376,"num_objects":7066,"num_object_clones":0,"num_object_copies":21198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7066,"num_write_kb":452224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"273'18","reported_seq":515,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.235915+0000","last_change":"2024-09-20T18:59:55.490161+0000","last_active":"2024-09-20T19:00:10.235915+0000","last_peered":"2024-09-20T19:00:10.235915+0000","last_clean":"2024-09-20T19:00:10.235915+0000","last_became_active":"2024-09-20T18:49:47.949116+0000","last_became_peered":"2024-09-20T18:49:47.949116+0000","last_unstale":"2024-09-20T19:00:10.235915+0000","last_undegraded":"2024-09-20T19:00:10.235915+0000","last_fullsized":"2024-09-20T19:00:10.235915+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'18","last_scrub_stamp":"2024-09-20T18:59:55.490086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:59:55.490086+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:15.287945+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":486,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.242773+0000","last_change":"2024-09-20T18:54:52.650798+0000","last_active":"2024-09-20T19:00:10.242773+0000","last_peered":"2024-09-20T19:00:10.242773+0000","last_clean":"2024-09-20T19:00:10.242773+0000","last_became_active":"2024-09-20T18:49:45.944858+0000","last_became_peered":"2024-09-20T18:49:45.944858+0000","last_unstale":"2024-09-20T19:00:10.242773+0000","last_undegraded":"2024-09-20T19:00:10.242773+0000","last_fullsized":"2024-09-20T19:00:10.242773+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:52.650756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:52.650756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":482,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.234537+0000","last_change":"2024-09-20T18:58:33.180357+0000","last_active":"2024-09-20T19:00:11.234537+0000","last_peered":"2024-09-20T19:00:11.234537+0000","last_clean":"2024-09-20T19:00:11.234537+0000","last_became_active":"2024-09-20T18:49:49.954944+0000","last_became_peered":"2024-09-20T18:49:49.954944+0000","last_unstale":"2024-09-20T19:00:11.234537+0000","last_undegraded":"2024-09-20T19:00:11.234537+0000","last_fullsized":"2024-09-20T19:00:11.234537+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:33.180309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:33.180309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":475,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.237567+0000","last_change":"2024-09-20T18:58:24.162921+0000","last_active":"2024-09-20T19:00:11.237567+0000","last_peered":"2024-09-20T19:00:11.237567+0000","last_clean":"2024-09-20T19:00:11.237567+0000","last_became_active":"2024-09-20T18:49:51.972970+0000","last_became_peered":"2024-09-20T18:49:51.972970+0000","last_unstale":"2024-09-20T19:00:11.237567+0000","last_undegraded":"2024-09-20T19:00:11.237567+0000","last_fullsized":"2024-09-20T19:00:11.237567+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:24.162879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:24.162879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"282'7154","reported_seq":7241,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.241456+0000","last_change":"2024-09-20T18:58:33.821228+0000","last_active":"2024-09-20T19:00:11.241456+0000","last_peered":"2024-09-20T19:00:11.241456+0000","last_clean":"2024-09-20T19:00:11.241456+0000","last_became_active":"2024-09-20T18:58:33.819949+0000","last_became_peered":"2024-09-20T18:58:33.819949+0000","last_unstale":"2024-09-20T19:00:11.241456+0000","last_undegraded":"2024-09-20T19:00:11.241456+0000","last_fullsized":"2024-09-20T19:00:11.241456+0000","mapping_epoch":246,"log_start":"253'1200","ondisk_log_start":"253'1200","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":5954,"log_dups_size":0,"ondisk_log_size":5954,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":468844544,"num_objects":7154,"num_object_clones":0,"num_object_copies":21462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7154,"num_write_kb":457856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"273'27","reported_seq":570,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.238672+0000","last_change":"2024-09-20T18:59:05.108455+0000","last_active":"2024-09-20T19:00:10.238672+0000","last_peered":"2024-09-20T19:00:10.238672+0000","last_clean":"2024-09-20T19:00:10.238672+0000","last_became_active":"2024-09-20T18:59:05.083726+0000","last_became_peered":"2024-09-20T18:59:05.083726+0000","last_unstale":"2024-09-20T19:00:10.238672+0000","last_undegraded":"2024-09-20T19:00:10.238672+0000","last_fullsized":"2024-09-20T19:00:10.238672+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:58:30.522129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:30.522129+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":514,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.242890+0000","last_change":"2024-09-20T18:58:57.563127+0000","last_active":"2024-09-20T19:00:10.242890+0000","last_peered":"2024-09-20T19:00:10.242890+0000","last_clean":"2024-09-20T19:00:10.242890+0000","last_became_active":"2024-09-20T18:49:45.944256+0000","last_became_peered":"2024-09-20T18:49:45.944256+0000","last_unstale":"2024-09-20T19:00:10.242890+0000","last_undegraded":"2024-09-20T19:00:10.242890+0000","last_fullsized":"2024-09-20T19:00:10.242890+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-20T18:58:57.563058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:57.563058+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:20.547993+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"262'10","reported_seq":625,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.241967+0000","last_change":"2024-09-20T18:59:05.111302+0000","last_active":"2024-09-20T19:00:10.241967+0000","last_peered":"2024-09-20T19:00:10.241967+0000","last_clean":"2024-09-20T19:00:10.241967+0000","last_became_active":"2024-09-20T18:59:05.090811+0000","last_became_peered":"2024-09-20T18:59:05.090811+0000","last_unstale":"2024-09-20T19:00:10.241967+0000","last_undegraded":"2024-09-20T19:00:10.241967+0000","last_fullsized":"2024-09-20T19:00:10.241967+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'9","last_scrub_stamp":"2024-09-20T18:58:27.521051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:27.521051+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":487,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.237389+0000","last_change":"2024-09-20T18:59:05.087336+0000","last_active":"2024-09-20T19:00:10.237389+0000","last_peered":"2024-09-20T19:00:10.237389+0000","last_clean":"2024-09-20T19:00:10.237389+0000","last_became_active":"2024-09-20T18:59:05.087005+0000","last_became_peered":"2024-09-20T18:59:05.087005+0000","last_unstale":"2024-09-20T19:00:10.237389+0000","last_undegraded":"2024-09-20T19:00:10.237389+0000","last_fullsized":"2024-09-20T19:00:10.237389+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:05.523359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:05.523359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":56,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.247480+0000","last_change":"2024-09-20T18:59:05.085844+0000","last_active":"2024-09-20T19:00:10.247480+0000","last_peered":"2024-09-20T19:00:10.247480+0000","last_clean":"2024-09-20T19:00:10.247480+0000","last_became_active":"2024-09-20T18:59:05.085427+0000","last_became_peered":"2024-09-20T18:59:05.085427+0000","last_unstale":"2024-09-20T19:00:10.247480+0000","last_undegraded":"2024-09-20T19:00:10.247480+0000","last_fullsized":"2024-09-20T19:00:10.247480+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:26.721735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:26.721735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"273'12","reported_seq":501,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.239690+0000","last_change":"2024-09-20T18:58:09.063555+0000","last_active":"2024-09-20T19:00:11.239690+0000","last_peered":"2024-09-20T19:00:11.239690+0000","last_clean":"2024-09-20T19:00:11.239690+0000","last_became_active":"2024-09-20T18:49:48.500662+0000","last_became_peered":"2024-09-20T18:49:48.500662+0000","last_unstale":"2024-09-20T19:00:11.239690+0000","last_undegraded":"2024-09-20T19:00:11.239690+0000","last_fullsized":"2024-09-20T19:00:11.239690+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-20T18:58:09.063481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:09.063481+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":477,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.238678+0000","last_change":"2024-09-20T18:58:28.460106+0000","last_active":"2024-09-20T19:00:10.238678+0000","last_peered":"2024-09-20T19:00:10.238678+0000","last_clean":"2024-09-20T19:00:10.238678+0000","last_became_active":"2024-09-20T18:49:49.956749+0000","last_became_peered":"2024-09-20T18:49:49.956749+0000","last_unstale":"2024-09-20T19:00:10.238678+0000","last_undegraded":"2024-09-20T19:00:10.238678+0000","last_fullsized":"2024-09-20T19:00:10.238678+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:28.460054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:28.460054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":488,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.241708+0000","last_change":"2024-09-20T18:59:05.082737+0000","last_active":"2024-09-20T19:00:10.241708+0000","last_peered":"2024-09-20T19:00:10.241708+0000","last_clean":"2024-09-20T19:00:10.241708+0000","last_became_active":"2024-09-20T18:59:05.082018+0000","last_became_peered":"2024-09-20T18:59:05.082018+0000","last_unstale":"2024-09-20T19:00:10.241708+0000","last_undegraded":"2024-09-20T19:00:10.241708+0000","last_fullsized":"2024-09-20T19:00:10.241708+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.637739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.637739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"273'15","reported_seq":501,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.241882+0000","last_change":"2024-09-20T18:54:44.830872+0000","last_active":"2024-09-20T19:00:11.241882+0000","last_peered":"2024-09-20T19:00:11.241882+0000","last_clean":"2024-09-20T19:00:11.241882+0000","last_became_active":"2024-09-20T18:49:48.500803+0000","last_became_peered":"2024-09-20T18:49:48.500803+0000","last_unstale":"2024-09-20T19:00:11.241882+0000","last_undegraded":"2024-09-20T19:00:11.241882+0000","last_fullsized":"2024-09-20T19:00:11.241882+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:44.830790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.830790+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":500,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.243020+0000","last_change":"2024-09-20T18:59:05.091318+0000","last_active":"2024-09-20T19:00:10.243020+0000","last_peered":"2024-09-20T19:00:10.243020+0000","last_clean":"2024-09-20T19:00:10.243020+0000","last_became_active":"2024-09-20T18:59:05.090805+0000","last_became_peered":"2024-09-20T18:59:05.090805+0000","last_unstale":"2024-09-20T19:00:10.243020+0000","last_undegraded":"2024-09-20T19:00:10.243020+0000","last_fullsized":"2024-09-20T19:00:10.243020+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:59.589891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:59.589891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:14.124488+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":479,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.239486+0000","last_change":"2024-09-20T18:57:46.898929+0000","last_active":"2024-09-20T19:00:11.239486+0000","last_peered":"2024-09-20T19:00:11.239486+0000","last_clean":"2024-09-20T19:00:11.239486+0000","last_became_active":"2024-09-20T18:49:49.961947+0000","last_became_peered":"2024-09-20T18:49:49.961947+0000","last_unstale":"2024-09-20T19:00:11.239486+0000","last_undegraded":"2024-09-20T19:00:11.239486+0000","last_fullsized":"2024-09-20T19:00:11.239486+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:46.898862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:46.898862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":38,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"66'1","reported_seq":487,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.234040+0000","last_change":"2024-09-20T18:58:30.171382+0000","last_active":"2024-09-20T19:00:11.234040+0000","last_peered":"2024-09-20T19:00:11.234040+0000","last_clean":"2024-09-20T19:00:11.234040+0000","last_became_active":"2024-09-20T18:49:51.965121+0000","last_became_peered":"2024-09-20T18:49:51.965121+0000","last_unstale":"2024-09-20T19:00:11.234040+0000","last_undegraded":"2024-09-20T19:00:11.234040+0000","last_fullsized":"2024-09-20T19:00:11.234040+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T18:58:30.171329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:30.171329+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"274'15","reported_seq":498,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.238218+0000","last_change":"2024-09-20T18:57:59.430353+0000","last_active":"2024-09-20T19:00:10.238218+0000","last_peered":"2024-09-20T19:00:10.238218+0000","last_clean":"2024-09-20T19:00:10.238218+0000","last_became_active":"2024-09-20T18:49:48.501011+0000","last_became_peered":"2024-09-20T18:49:48.501011+0000","last_unstale":"2024-09-20T19:00:10.238218+0000","last_undegraded":"2024-09-20T19:00:10.238218+0000","last_fullsized":"2024-09-20T19:00:10.238218+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:57:59.430294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:59.430294+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":488,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.241039+0000","last_change":"2024-09-20T18:55:27.028184+0000","last_active":"2024-09-20T19:00:11.241039+0000","last_peered":"2024-09-20T19:00:11.241039+0000","last_clean":"2024-09-20T19:00:11.241039+0000","last_became_active":"2024-09-20T18:49:45.938511+0000","last_became_peered":"2024-09-20T18:49:45.938511+0000","last_unstale":"2024-09-20T19:00:11.241039+0000","last_undegraded":"2024-09-20T19:00:11.241039+0000","last_fullsized":"2024-09-20T19:00:11.241039+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:27.028126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:27.028126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":613,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.238738+0000","last_change":"2024-09-20T18:55:27.284156+0000","last_active":"2024-09-20T19:00:11.238738+0000","last_peered":"2024-09-20T19:00:11.238738+0000","last_clean":"2024-09-20T19:00:11.238738+0000","last_became_active":"2024-09-20T18:49:49.960550+0000","last_became_peered":"2024-09-20T18:49:49.960550+0000","last_unstale":"2024-09-20T19:00:11.238738+0000","last_undegraded":"2024-09-20T19:00:11.238738+0000","last_fullsized":"2024-09-20T19:00:11.238738+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:27.284086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:27.284086+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":483,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.233305+0000","last_change":"2024-09-20T18:59:59.497880+0000","last_active":"2024-09-20T19:00:10.233305+0000","last_peered":"2024-09-20T19:00:10.233305+0000","last_clean":"2024-09-20T19:00:10.233305+0000","last_became_active":"2024-09-20T18:49:51.966254+0000","last_became_peered":"2024-09-20T18:49:51.966254+0000","last_unstale":"2024-09-20T19:00:10.233305+0000","last_undegraded":"2024-09-20T19:00:10.233305+0000","last_fullsized":"2024-09-20T19:00:10.233305+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:59.497810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:59:59.497810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:04.610550+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"274'20","reported_seq":524,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.239051+0000","last_change":"2024-09-20T18:57:20.701592+0000","last_active":"2024-09-20T19:00:11.239051+0000","last_peered":"2024-09-20T19:00:11.239051+0000","last_clean":"2024-09-20T19:00:11.239051+0000","last_became_active":"2024-09-20T18:49:58.997494+0000","last_became_peered":"2024-09-20T18:49:58.997494+0000","last_unstale":"2024-09-20T19:00:11.239051+0000","last_undegraded":"2024-09-20T19:00:11.239051+0000","last_fullsized":"2024-09-20T19:00:11.239051+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:57:20.701543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:20.701543+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":486,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.241354+0000","last_change":"2024-09-20T18:54:57.632458+0000","last_active":"2024-09-20T19:00:10.241354+0000","last_peered":"2024-09-20T19:00:10.241354+0000","last_clean":"2024-09-20T19:00:10.241354+0000","last_became_active":"2024-09-20T18:49:45.949492+0000","last_became_peered":"2024-09-20T18:49:45.949492+0000","last_unstale":"2024-09-20T19:00:10.241354+0000","last_undegraded":"2024-09-20T19:00:10.241354+0000","last_fullsized":"2024-09-20T19:00:10.241354+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.632380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.632380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":610,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.237995+0000","last_change":"2024-09-20T18:55:28.302353+0000","last_active":"2024-09-20T19:00:10.237995+0000","last_peered":"2024-09-20T19:00:10.237995+0000","last_clean":"2024-09-20T19:00:10.237995+0000","last_became_active":"2024-09-20T18:49:49.962243+0000","last_became_peered":"2024-09-20T18:49:49.962243+0000","last_unstale":"2024-09-20T19:00:10.237995+0000","last_undegraded":"2024-09-20T19:00:10.237995+0000","last_fullsized":"2024-09-20T19:00:10.237995+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:28.302303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:28.302303+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:09.385599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":484,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.241496+0000","last_change":"2024-09-20T18:58:55.232948+0000","last_active":"2024-09-20T19:00:11.241496+0000","last_peered":"2024-09-20T19:00:11.241496+0000","last_clean":"2024-09-20T19:00:11.241496+0000","last_became_active":"2024-09-20T18:49:51.969093+0000","last_became_peered":"2024-09-20T18:49:51.969093+0000","last_unstale":"2024-09-20T19:00:11.241496+0000","last_undegraded":"2024-09-20T19:00:11.241496+0000","last_fullsized":"2024-09-20T19:00:11.241496+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:55.232883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:55.232883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":59,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"273'19","reported_seq":552,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.232578+0000","last_change":"2024-09-20T18:59:05.117294+0000","last_active":"2024-09-20T19:00:10.232578+0000","last_peered":"2024-09-20T19:00:10.232578+0000","last_clean":"2024-09-20T19:00:10.232578+0000","last_became_active":"2024-09-20T18:59:05.081917+0000","last_became_peered":"2024-09-20T18:59:05.081917+0000","last_unstale":"2024-09-20T19:00:10.232578+0000","last_undegraded":"2024-09-20T19:00:10.232578+0000","last_fullsized":"2024-09-20T19:00:10.232578+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:57:50.899514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:50.899514+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:28.208671+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":493,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.235024+0000","last_change":"2024-09-20T18:59:06.310100+0000","last_active":"2024-09-20T19:00:10.235024+0000","last_peered":"2024-09-20T19:00:10.235024+0000","last_clean":"2024-09-20T19:00:10.235024+0000","last_became_active":"2024-09-20T18:59:05.073087+0000","last_became_peered":"2024-09-20T18:59:05.073087+0000","last_unstale":"2024-09-20T19:00:10.235024+0000","last_undegraded":"2024-09-20T19:00:10.235024+0000","last_fullsized":"2024-09-20T19:00:10.235024+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:06.310027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:59:06.310027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:35.049125+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":471,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.235590+0000","last_change":"2024-09-20T18:57:11.986605+0000","last_active":"2024-09-20T19:00:11.235590+0000","last_peered":"2024-09-20T19:00:11.235590+0000","last_clean":"2024-09-20T19:00:11.235590+0000","last_became_active":"2024-09-20T18:49:49.957067+0000","last_became_peered":"2024-09-20T18:49:49.957067+0000","last_unstale":"2024-09-20T19:00:11.235590+0000","last_undegraded":"2024-09-20T19:00:11.235590+0000","last_fullsized":"2024-09-20T19:00:11.235590+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:11.986528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:11.986528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":467,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.237159+0000","last_change":"2024-09-20T18:57:16.789412+0000","last_active":"2024-09-20T19:00:11.237159+0000","last_peered":"2024-09-20T19:00:11.237159+0000","last_clean":"2024-09-20T19:00:11.237159+0000","last_became_active":"2024-09-20T18:49:51.966915+0000","last_became_peered":"2024-09-20T18:49:51.966915+0000","last_unstale":"2024-09-20T19:00:11.237159+0000","last_undegraded":"2024-09-20T19:00:11.237159+0000","last_fullsized":"2024-09-20T19:00:11.237159+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:16.789095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:16.789095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"273'15","reported_seq":550,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.235809+0000","last_change":"2024-09-20T18:59:05.103863+0000","last_active":"2024-09-20T19:00:11.235809+0000","last_peered":"2024-09-20T19:00:11.235809+0000","last_clean":"2024-09-20T19:00:11.235809+0000","last_became_active":"2024-09-20T18:59:05.085230+0000","last_became_peered":"2024-09-20T18:59:05.085230+0000","last_unstale":"2024-09-20T19:00:11.235809+0000","last_undegraded":"2024-09-20T19:00:11.235809+0000","last_fullsized":"2024-09-20T19:00:11.235809+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:58:28.035450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:28.035450+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":478,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.242456+0000","last_change":"2024-09-20T18:54:42.487465+0000","last_active":"2024-09-20T19:00:10.242456+0000","last_peered":"2024-09-20T19:00:10.242456+0000","last_clean":"2024-09-20T19:00:10.242456+0000","last_became_active":"2024-09-20T18:49:45.944357+0000","last_became_peered":"2024-09-20T18:49:45.944357+0000","last_unstale":"2024-09-20T19:00:10.242456+0000","last_undegraded":"2024-09-20T19:00:10.242456+0000","last_fullsized":"2024-09-20T19:00:10.242456+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:42.487403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:42.487403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":479,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.237651+0000","last_change":"2024-09-20T18:57:57.915090+0000","last_active":"2024-09-20T19:00:11.237651+0000","last_peered":"2024-09-20T19:00:11.237651+0000","last_clean":"2024-09-20T19:00:11.237651+0000","last_became_active":"2024-09-20T18:49:49.962523+0000","last_became_peered":"2024-09-20T18:49:49.962523+0000","last_unstale":"2024-09-20T19:00:11.237651+0000","last_undegraded":"2024-09-20T19:00:11.237651+0000","last_fullsized":"2024-09-20T19:00:11.237651+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:57.914913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:57.914913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":476,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.240484+0000","last_change":"2024-09-20T18:54:26.882441+0000","last_active":"2024-09-20T19:00:11.240484+0000","last_peered":"2024-09-20T19:00:11.240484+0000","last_clean":"2024-09-20T19:00:11.240484+0000","last_became_active":"2024-09-20T18:49:51.967313+0000","last_became_peered":"2024-09-20T18:49:51.967313+0000","last_unstale":"2024-09-20T19:00:11.240484+0000","last_undegraded":"2024-09-20T19:00:11.240484+0000","last_fullsized":"2024-09-20T19:00:11.240484+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:26.882375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:26.882375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:06.821534+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"274'19","reported_seq":552,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.239353+0000","last_change":"2024-09-20T18:59:05.096414+0000","last_active":"2024-09-20T19:00:11.239353+0000","last_peered":"2024-09-20T19:00:11.239353+0000","last_clean":"2024-09-20T19:00:11.239353+0000","last_became_active":"2024-09-20T18:59:05.075390+0000","last_became_peered":"2024-09-20T18:59:05.075390+0000","last_unstale":"2024-09-20T19:00:11.239353+0000","last_undegraded":"2024-09-20T19:00:11.239353+0000","last_fullsized":"2024-09-20T19:00:11.239353+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:57:54.862329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:54.862329+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":493,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.238748+0000","last_change":"2024-09-20T18:58:06.479419+0000","last_active":"2024-09-20T19:00:10.238748+0000","last_peered":"2024-09-20T19:00:10.238748+0000","last_clean":"2024-09-20T19:00:10.238748+0000","last_became_active":"2024-09-20T18:49:45.942105+0000","last_became_peered":"2024-09-20T18:49:45.942105+0000","last_unstale":"2024-09-20T19:00:10.238748+0000","last_undegraded":"2024-09-20T19:00:10.238748+0000","last_fullsized":"2024-09-20T19:00:10.238748+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:06.479364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:06.479364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":492,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.240186+0000","last_change":"2024-09-20T18:59:05.081655+0000","last_active":"2024-09-20T19:00:10.240186+0000","last_peered":"2024-09-20T19:00:10.240186+0000","last_clean":"2024-09-20T19:00:10.240186+0000","last_became_active":"2024-09-20T18:59:05.081320+0000","last_became_peered":"2024-09-20T18:59:05.081320+0000","last_unstale":"2024-09-20T19:00:10.240186+0000","last_undegraded":"2024-09-20T19:00:10.240186+0000","last_fullsized":"2024-09-20T19:00:10.240186+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:46.329547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:46.329547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":481,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.237788+0000","last_change":"2024-09-20T18:59:05.081584+0000","last_active":"2024-09-20T19:00:11.237788+0000","last_peered":"2024-09-20T19:00:11.237788+0000","last_clean":"2024-09-20T19:00:11.237788+0000","last_became_active":"2024-09-20T18:59:05.081179+0000","last_became_peered":"2024-09-20T18:59:05.081179+0000","last_unstale":"2024-09-20T19:00:11.237788+0000","last_undegraded":"2024-09-20T19:00:11.237788+0000","last_fullsized":"2024-09-20T19:00:11.237788+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:28.165712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:28.165712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"273'8","reported_seq":521,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.237107+0000","last_change":"2024-09-20T18:59:05.098374+0000","last_active":"2024-09-20T19:00:11.237107+0000","last_peered":"2024-09-20T19:00:11.237107+0000","last_clean":"2024-09-20T19:00:11.237107+0000","last_became_active":"2024-09-20T18:59:05.076365+0000","last_became_peered":"2024-09-20T18:59:05.076365+0000","last_unstale":"2024-09-20T19:00:11.237107+0000","last_undegraded":"2024-09-20T19:00:11.237107+0000","last_fullsized":"2024-09-20T19:00:11.237107+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-20T18:55:20.282193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:20.282193+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":488,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.241409+0000","last_change":"2024-09-20T18:54:55.900487+0000","last_active":"2024-09-20T19:00:11.241409+0000","last_peered":"2024-09-20T19:00:11.241409+0000","last_clean":"2024-09-20T19:00:11.241409+0000","last_became_active":"2024-09-20T18:49:45.940988+0000","last_became_peered":"2024-09-20T18:49:45.940988+0000","last_unstale":"2024-09-20T19:00:11.241409+0000","last_undegraded":"2024-09-20T19:00:11.241409+0000","last_fullsized":"2024-09-20T19:00:11.241409+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:55.899611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:55.899611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":56,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.246793+0000","last_change":"2024-09-20T18:59:05.086874+0000","last_active":"2024-09-20T19:00:10.246793+0000","last_peered":"2024-09-20T19:00:10.246793+0000","last_clean":"2024-09-20T19:00:10.246793+0000","last_became_active":"2024-09-20T18:59:05.086570+0000","last_became_peered":"2024-09-20T18:59:05.086570+0000","last_unstale":"2024-09-20T19:00:10.246793+0000","last_undegraded":"2024-09-20T19:00:10.246793+0000","last_fullsized":"2024-09-20T19:00:10.246793+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:52.929828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:52.929828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":466,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.240513+0000","last_change":"2024-09-20T18:58:22.558181+0000","last_active":"2024-09-20T19:00:10.240513+0000","last_peered":"2024-09-20T19:00:10.240513+0000","last_clean":"2024-09-20T19:00:10.240513+0000","last_became_active":"2024-09-20T18:49:51.968662+0000","last_became_peered":"2024-09-20T18:49:51.968662+0000","last_unstale":"2024-09-20T19:00:10.240513+0000","last_undegraded":"2024-09-20T19:00:10.240513+0000","last_fullsized":"2024-09-20T19:00:10.240513+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:22.558131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:22.558131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":459,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.237837+0000","last_change":"2024-09-20T18:57:10.970843+0000","last_active":"2024-09-20T19:00:11.237837+0000","last_peered":"2024-09-20T19:00:11.237837+0000","last_clean":"2024-09-20T19:00:11.237837+0000","last_became_active":"2024-09-20T18:49:51.968216+0000","last_became_peered":"2024-09-20T18:49:51.968216+0000","last_unstale":"2024-09-20T19:00:11.237837+0000","last_undegraded":"2024-09-20T19:00:11.237837+0000","last_fullsized":"2024-09-20T19:00:11.237837+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:10.970779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:10.970779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":487,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.237517+0000","last_change":"2024-09-20T18:58:36.077005+0000","last_active":"2024-09-20T19:00:11.237517+0000","last_peered":"2024-09-20T19:00:11.237517+0000","last_clean":"2024-09-20T19:00:11.237517+0000","last_became_active":"2024-09-20T18:49:45.937992+0000","last_became_peered":"2024-09-20T18:49:45.937992+0000","last_unstale":"2024-09-20T19:00:11.237517+0000","last_undegraded":"2024-09-20T19:00:11.237517+0000","last_fullsized":"2024-09-20T19:00:11.237517+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:36.076937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:36.076937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"274'19","reported_seq":80,"reported_epoch":283,"state":"active+clean","last_fresh":"2024-09-20T19:00:10.246622+0000","last_change":"2024-09-20T18:59:05.116749+0000","last_active":"2024-09-20T19:00:10.246622+0000","last_peered":"2024-09-20T19:00:10.246622+0000","last_clean":"2024-09-20T19:00:10.246622+0000","last_became_active":"2024-09-20T18:59:05.089294+0000","last_became_peered":"2024-09-20T18:59:05.089294+0000","last_unstale":"2024-09-20T19:00:10.246622+0000","last_undegraded":"2024-09-20T19:00:10.246622+0000","last_fullsized":"2024-09-20T19:00:10.246622+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:54:44.719676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.719676+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":473,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-20T19:00:11.241642+0000","last_change":"2024-09-20T18:58:27.904636+0000","last_active":"2024-09-20T19:00:11.241642+0000","last_peered":"2024-09-20T19:00:11.241642+0000","last_clean":"2024-09-20T19:00:11.241642+0000","last_became_active":"2024-09-20T18:49:49.957582+0000","last_became_peered":"2024-09-20T18:49:49.957582+0000","last_unstale":"2024-09-20T19:00:11.241642+0000","last_undegraded":"2024-09-20T19:00:11.241642+0000","last_fullsized":"2024-09-20T19:00:11.241642+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:27.904592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:27.904592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":7480541184,"num_objects":114144,"num_object_clones":0,"num_object_copies":342432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":114144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114144,"num_write_kb":7305216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9499,"num_bytes_recovered":622526464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":22858760192,"data_stored":22858760192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82729,"ondisk_log_size":82729,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1124,"num_read_kb":881,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":2244,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":681,"ondisk_log_size":681,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":619,"num_read_kb":614,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266817,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4482116,"kb_used_data":4121908,"kb_used_omap":1,"kb_used_meta":360190,"kb_avail":89263036,"statfs":{"total":95995035648,"available":91405348864,"internally_reserved":0,"allocated":4220833792,"data_stored":4220450020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":368835018},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528456,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1786032,"kb_used_data":1498016,"kb_used_omap":1,"kb_used_meta":287998,"kb_avail":91959120,"statfs":{"total":95995035648,"available":94166138880,"internally_reserved":0,"allocated":1533968384,"data_stored":1533604430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":294910406},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757389,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4470080,"kb_used_data":4100140,"kb_used_omap":1,"kb_used_meta":369918,"kb_avail":89275072,"statfs":{"total":95995035648,"available":91417673728,"internally_reserved":0,"allocated":4198543360,"data_stored":4198173848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":378796485},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986322,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1849440,"kb_used_data":1569740,"kb_used_omap":1,"kb_used_meta":279678,"kb_avail":91895712,"statfs":{"total":95995035648,"available":94101209088,"internally_reserved":0,"allocated":1607413760,"data_stored":1607012732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":286390725},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182553,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4121068,"kb_used_data":3634516,"kb_used_omap":1,"kb_used_meta":486526,"kb_avail":89624084,"statfs":{"total":95995035648,"available":91775062016,"internally_reserved":0,"allocated":3721744384,"data_stored":3721340902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":498203079},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411486,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1806340,"kb_used_data":1378096,"kb_used_omap":1,"kb_used_meta":428222,"kb_avail":91938812,"statfs":{"total":95995035648,"available":94145343488,"internally_reserved":0,"allocated":1411170304,"data_stored":1410797275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":438499781},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575012,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4522804,"kb_used_data":4122336,"kb_used_omap":1,"kb_used_meta":400446,"kb_avail":89222348,"statfs":{"total":95995035648,"available":91363684352,"internally_reserved":0,"allocated":4221272064,"data_stored":4220851067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":410057157},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738538,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2420196,"kb_used_data":2019724,"kb_used_omap":1,"kb_used_meta":400446,"kb_avail":91324956,"statfs":{"total":95995035648,"available":93516754944,"internally_reserved":0,"allocated":2068197376,"data_stored":2067802533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":410057157},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2066808832,"data_stored":2066808832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4220452864,"data_stored":4220452864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1410400256,"data_stored":1410400256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3720937472,"data_stored":3720937472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1606615040,"data_stored":1606615040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4197777408,"data_stored":4197777408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1403387904,"data_stored":1403387904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4232380416,"data_stored":4232380416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T19:00:13.481 INFO:tasks.cephadm.ceph_manager.ceph:would drop below min_pgs, proj 6, currently 16 2024-09-20T19:00:13.499 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:13.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:13.499 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:00:13.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:13 smithi142 ceph-mon[32508]: osdmap e285: 8 total, 8 up, 8 in 2024-09-20T19:00:13.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:13 smithi142 ceph-mon[32508]: from='client.25589 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:00:13.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:13 smithi083 ceph-mon[29421]: osdmap e285: 8 total, 8 up, 8 in 2024-09-20T19:00:13.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:13 smithi083 ceph-mon[29421]: from='client.25589 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:00:13.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:13 smithi083 ceph-mon[36260]: osdmap e285: 8 total, 8 up, 8 in 2024-09-20T19:00:13.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:13 smithi083 ceph-mon[36260]: from='client.25589 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:00:13.601 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:13.629 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:13.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:13.630 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:13.650 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T19:00:13.731 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:13.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:13.785 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:13.785 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:13.886 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:13.934 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:13.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:13.935 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:14.036 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:00:14.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:14.080 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:14.081 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:00:14.181 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:00:14.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:14.212 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:14.212 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:00:14.313 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:00:14.344 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:14.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:14.345 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:00:14.446 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:14.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:14.480 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:14.481 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:14.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:14 smithi083 ceph-mon[29421]: pgmap v513: 148 pgs: 1 peering, 147 active+clean; 7.0 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 853 B/s rd, 49 MiB/s wr, 778 op/s; 8.9 MiB/s, 141 objects/s recovering 2024-09-20T19:00:14.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:14 smithi083 ceph-mon[29421]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2 pgs undersized) 2024-09-20T19:00:14.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:14 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/4179900005' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:00:14.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:14 smithi083 ceph-mon[36260]: pgmap v513: 148 pgs: 1 peering, 147 active+clean; 7.0 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 853 B/s rd, 49 MiB/s wr, 778 op/s; 8.9 MiB/s, 141 objects/s recovering 2024-09-20T19:00:14.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:14 smithi083 ceph-mon[36260]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2 pgs undersized) 2024-09-20T19:00:14.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:14 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/4179900005' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:00:14.582 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:14.621 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:14.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:14.622 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:14.723 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:14.760 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:14.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:14.761 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:14.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:14 smithi142 ceph-mon[32508]: pgmap v513: 148 pgs: 1 peering, 147 active+clean; 7.0 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 853 B/s rd, 49 MiB/s wr, 778 op/s; 8.9 MiB/s, 141 objects/s recovering 2024-09-20T19:00:14.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:14 smithi142 ceph-mon[32508]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2 pgs undersized) 2024-09-20T19:00:14.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:14 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/4179900005' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:00:14.805 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:00:14.806 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:00:14.806 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:00:14.806 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:00:14.806 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:00:14.806 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:00:14.806 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:00:14.806 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:00:14.806 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:00:14.806 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:00:14.807 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:00:14.807 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:00:14.807 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:00:14.807 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:00:14.807 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:00:14.807 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:00:14.862 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:14.893 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:14.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:14.894 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:14.994 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:00:15.023 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:15.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:15.024 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:00:15.125 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:15.163 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:15.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:15.164 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:15.265 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:15.298 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:15.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:15.299 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:15.400 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:15.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:15.436 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:15.436 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:15.537 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:00:15.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:15 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:00:15.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:15 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:00:15.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:15 smithi083 ceph-mon[29421]: osdmap e286: 8 total, 8 up, 8 in 2024-09-20T19:00:15.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:15 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:00:15.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:15 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:00:15.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:15 smithi083 ceph-mon[36260]: osdmap e286: 8 total, 8 up, 8 in 2024-09-20T19:00:15.578 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:15.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:15.579 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:00:15.632 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T19:00:15.632 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:00:15.680 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:15.713 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:15.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:15.714 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:15.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:15 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:00:15.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:15 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:00:15.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:15 smithi142 ceph-mon[32508]: osdmap e286: 8 total, 8 up, 8 in 2024-09-20T19:00:15.815 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:00:15.851 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:15.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:15.852 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:00:15.953 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:15.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:15.981 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:15.982 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:16.082 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:16.103 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:16.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:16.104 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:16.205 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:00:16.227 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:16.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:16.228 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:00:16.329 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:16.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:16.353 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:16.353 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:16.454 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:16.487 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:16.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:16.488 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:16.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:16 smithi083 ceph-mon[29421]: pgmap v515: 148 pgs: 1 peering, 147 active+clean; 7.0 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.0 KiB/s rd, 50 MiB/s wr, 794 op/s; 11 MiB/s, 177 objects/s recovering 2024-09-20T19:00:16.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:16 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:00:16.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:16 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:00:16.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:16 smithi083 ceph-mon[36260]: pgmap v515: 148 pgs: 1 peering, 147 active+clean; 7.0 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.0 KiB/s rd, 50 MiB/s wr, 794 op/s; 11 MiB/s, 177 objects/s recovering 2024-09-20T19:00:16.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:16 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:00:16.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:16 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:00:16.590 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:16.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:16.623 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:16.623 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:16.724 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:16.756 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:16.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:16.757 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:16.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:16 smithi142 ceph-mon[32508]: pgmap v515: 148 pgs: 1 peering, 147 active+clean; 7.0 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.0 KiB/s rd, 50 MiB/s wr, 794 op/s; 11 MiB/s, 177 objects/s recovering 2024-09-20T19:00:16.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:16 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:00:16.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:16 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:00:16.859 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:16.880 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:16.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:16.881 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:16.981 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:17.011 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:17.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:17.012 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:17.113 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:00:17.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:17.146 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:17.146 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:00:17.247 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:00:17.266 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:00:16 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:00:17.282 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:17.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:17.284 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:00:17.384 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:00:17.416 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:17.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:17.417 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:00:17.517 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:17.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:17.559 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:17.559 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:17.580 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:17 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:00:17.580 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:17 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:00:17.580 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:17 smithi083 ceph-mon[29421]: osdmap e287: 8 total, 8 up, 8 in 2024-09-20T19:00:17.581 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:17 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:00:17.581 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:17 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:00:17.581 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:17 smithi083 ceph-mon[36260]: osdmap e287: 8 total, 8 up, 8 in 2024-09-20T19:00:17.582 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_historic_ops 2024-09-20T19:00:17.661 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:00:17.689 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:17.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:17.690 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:00:17.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:17 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:00:17.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:17 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:00:17.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:17 smithi142 ceph-mon[32508]: osdmap e287: 8 total, 8 up, 8 in 2024-09-20T19:00:17.791 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:17.839 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:17.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:17.841 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:17.942 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:17.972 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:17.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:17.973 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:18.074 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:18.081 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:00:18.149 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:18.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:18.150 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:18.251 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:00:18.259 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:00:18.282 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:18.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:18.283 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:00:18.384 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:18.414 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:18.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:18.415 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:18.482 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 7, 6] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-20T19:00:18.483 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T19:00:18.483 INFO:tasks.thrashosds.thrasher:fixing pg num pool unique_pool_2 2024-09-20T19:00:18.483 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T19:00:18.516 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:00:18.549 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:18.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:18.550 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:00:18.651 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:00:18.652 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:18 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:00:18.652 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:18 smithi083 ceph-mon[29421]: pgmap v517: 148 pgs: 148 active+clean; 6.7 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 1.0 KiB/s rd, 20 MiB/s wr, 1.05k op/s; 9.2 MiB/s, 146 objects/s recovering 2024-09-20T19:00:18.652 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:18 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:00:18.652 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:18 smithi083 ceph-mon[36260]: pgmap v517: 148 pgs: 148 active+clean; 6.7 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 1.0 KiB/s rd, 20 MiB/s wr, 1.05k op/s; 9.2 MiB/s, 146 objects/s recovering 2024-09-20T19:00:18.679 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:18.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:18.680 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:00:18.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:18 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:00:18.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:18 smithi142 ceph-mon[32508]: pgmap v517: 148 pgs: 148 active+clean; 6.7 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 1.0 KiB/s rd, 20 MiB/s wr, 1.05k op/s; 9.2 MiB/s, 146 objects/s recovering 2024-09-20T19:00:18.781 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:00:18.808 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:18.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:18.809 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:00:18.909 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:18.959 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:18.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:18.960 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:19.061 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:19.105 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:19.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:19.106 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:19.207 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:00:19.227 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T19:00:19.227 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-20T19:00:19.227 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-20T19:00:19.227 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-20T19:00:19.227 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.228 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:50415 9.b 9:d7a6ac9f:::benchmark_data_smithi083_124552_object50414:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-20T19:00:19.228 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:59:04.085102+0000", 2024-09-20T19:00:19.228 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 75.117221248000007, 2024-09-20T19:00:19.228 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1350377269999998, 2024-09-20T19:00:19.228 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:19.228 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:19.228 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:19.228 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:19.228 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:00:19.228 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:00:19.228 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 50415 2024-09-20T19:00:19.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.229 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:19.229 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.229 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:19.229 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.085102+0000", 2024-09-20T19:00:19.229 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:19.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.229 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.229 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:19.229 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.085102+0000", 2024-09-20T19:00:19.229 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:19.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.229 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.229 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:19.229 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.085104+0000", 2024-09-20T19:00:19.230 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4139999999999999e-06 2024-09-20T19:00:19.230 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.230 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.230 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:19.230 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.085126+0000", 2024-09-20T19:00:19.230 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2751e-05 2024-09-20T19:00:19.230 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.230 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.230 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:19.230 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.085144+0000", 2024-09-20T19:00:19.230 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7601e-05 2024-09-20T19:00:19.230 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.231 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.231 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:19.231 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.085154+0000", 2024-09-20T19:00:19.231 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6970000000000005e-06 2024-09-20T19:00:19.231 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.231 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.231 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:19.231 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.085213+0000", 2024-09-20T19:00:19.231 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8947999999999999e-05 2024-09-20T19:00:19.231 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.231 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.231 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:00:19.231 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.085222+0000", 2024-09-20T19:00:19.231 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3440000000000007e-06 2024-09-20T19:00:19.232 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.232 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.232 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:19.232 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.067321+0000", 2024-09-20T19:00:19.232 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.98209903600000004 2024-09-20T19:00:19.232 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.232 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.232 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:00:19.232 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.067332+0000", 2024-09-20T19:00:19.232 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0516000000000001e-05 2024-09-20T19:00:19.232 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.232 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.232 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:19.232 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.211443+0000", 2024-09-20T19:00:19.233 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.144111458 2024-09-20T19:00:19.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.233 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.233 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:19.233 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.211610+0000", 2024-09-20T19:00:19.233 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016707199999999999 2024-09-20T19:00:19.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.233 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.233 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,6", 2024-09-20T19:00:19.233 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.211766+0000", 2024-09-20T19:00:19.233 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015551200000000001 2024-09-20T19:00:19.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.233 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.234 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:19.234 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.217784+0000", 2024-09-20T19:00:19.234 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0060187909999999999 2024-09-20T19:00:19.234 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.234 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.234 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:19.234 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.217871+0000", 2024-09-20T19:00:19.234 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6176999999999998e-05 2024-09-20T19:00:19.234 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.234 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.234 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:00:19.234 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.217887+0000", 2024-09-20T19:00:19.234 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6509999999999999e-05 2024-09-20T19:00:19.234 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.235 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.235 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:19.235 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.217950+0000", 2024-09-20T19:00:19.235 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3081000000000001e-05 2024-09-20T19:00:19.235 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.235 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.235 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:19.235 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.220140+0000", 2024-09-20T19:00:19.235 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002189819 2024-09-20T19:00:19.235 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:19.236 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:19.236 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:19.236 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.236 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.236 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:50433 9.b 9:dbe14904:::benchmark_data_smithi083_124552_object50432:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-20T19:00:19.236 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:59:04.100787+0000", 2024-09-20T19:00:19.236 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 75.101536517, 2024-09-20T19:00:19.236 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1280354180000001, 2024-09-20T19:00:19.236 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:19.236 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:19.236 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:19.236 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:19.236 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:00:19.237 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:00:19.237 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 50433 2024-09-20T19:00:19.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.237 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:19.237 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.237 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:19.237 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.100787+0000", 2024-09-20T19:00:19.237 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:19.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.237 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.237 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:19.237 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.100787+0000", 2024-09-20T19:00:19.237 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:19.238 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.238 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.238 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:19.238 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.100788+0000", 2024-09-20T19:00:19.238 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2729999999999999e-06 2024-09-20T19:00:19.238 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.238 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.238 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:19.238 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.100811+0000", 2024-09-20T19:00:19.238 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2371999999999999e-05 2024-09-20T19:00:19.238 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.238 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.238 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:19.239 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.100825+0000", 2024-09-20T19:00:19.239 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4202000000000001e-05 2024-09-20T19:00:19.239 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.239 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.239 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:19.239 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.100833+0000", 2024-09-20T19:00:19.239 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9189999999999999e-06 2024-09-20T19:00:19.239 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.239 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.239 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:19.239 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.100935+0000", 2024-09-20T19:00:19.239 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000102286 2024-09-20T19:00:19.239 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.239 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.240 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:00:19.240 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.100946+0000", 2024-09-20T19:00:19.240 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0509999999999999e-05 2024-09-20T19:00:19.240 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.240 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.240 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:19.240 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.067356+0000", 2024-09-20T19:00:19.240 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.96641018300000003 2024-09-20T19:00:19.240 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.240 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.240 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:00:19.240 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.067363+0000", 2024-09-20T19:00:19.240 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0650000000000001e-06 2024-09-20T19:00:19.240 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.241 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.241 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:19.241 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.215420+0000", 2024-09-20T19:00:19.241 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.148057509 2024-09-20T19:00:19.241 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.241 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.241 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:19.241 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.215543+0000", 2024-09-20T19:00:19.241 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000122686 2024-09-20T19:00:19.241 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.241 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.241 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,6", 2024-09-20T19:00:19.241 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.215720+0000", 2024-09-20T19:00:19.241 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017689 2024-09-20T19:00:19.241 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.242 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.242 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:19.242 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.222383+0000", 2024-09-20T19:00:19.242 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0066632219999999999 2024-09-20T19:00:19.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.242 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.242 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:19.242 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.222429+0000", 2024-09-20T19:00:19.242 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6381999999999997e-05 2024-09-20T19:00:19.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.242 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.243 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:00:19.243 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.226551+0000", 2024-09-20T19:00:19.243 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0041210480000000004 2024-09-20T19:00:19.243 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.243 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.243 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:19.243 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.226609+0000", 2024-09-20T19:00:19.243 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8109000000000002e-05 2024-09-20T19:00:19.243 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.243 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.243 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:19.244 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.228822+0000", 2024-09-20T19:00:19.244 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0022137620000000002 2024-09-20T19:00:19.244 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:19.244 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:19.244 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:19.244 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.244 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.244 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:50435 9.b 9:d070b511:::benchmark_data_smithi083_124552_object50434:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-20T19:00:19.244 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:59:04.101133+0000", 2024-09-20T19:00:19.244 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 75.101190747999993, 2024-09-20T19:00:19.244 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1303714540000001, 2024-09-20T19:00:19.244 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:19.244 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:19.244 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:19.245 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:19.245 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:00:19.245 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:00:19.245 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 50435 2024-09-20T19:00:19.245 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.245 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:19.245 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.245 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:19.245 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.101133+0000", 2024-09-20T19:00:19.245 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:19.245 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.245 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.246 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:19.246 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.101133+0000", 2024-09-20T19:00:19.246 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:19.246 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.246 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.246 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:19.246 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.101134+0000", 2024-09-20T19:00:19.246 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0669999999999999e-06 2024-09-20T19:00:19.246 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.246 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.246 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:19.246 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.101155+0000", 2024-09-20T19:00:19.246 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1066000000000001e-05 2024-09-20T19:00:19.247 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.247 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.247 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:19.247 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.101169+0000", 2024-09-20T19:00:19.247 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4117999999999999e-05 2024-09-20T19:00:19.247 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.247 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.247 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:19.247 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.101177+0000", 2024-09-20T19:00:19.247 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8059999999999995e-06 2024-09-20T19:00:19.247 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.247 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.247 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:19.248 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.101230+0000", 2024-09-20T19:00:19.248 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3031000000000001e-05 2024-09-20T19:00:19.248 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.248 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.248 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:00:19.248 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.101238+0000", 2024-09-20T19:00:19.248 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.49e-06 2024-09-20T19:00:19.248 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.248 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.248 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:19.248 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.067387+0000", 2024-09-20T19:00:19.248 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.966148377 2024-09-20T19:00:19.248 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.248 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.249 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:00:19.249 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.067394+0000", 2024-09-20T19:00:19.249 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2930000000000003e-06 2024-09-20T19:00:19.249 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.249 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.249 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:19.249 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.220287+0000", 2024-09-20T19:00:19.249 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1528933450000001 2024-09-20T19:00:19.249 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.249 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.249 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:19.249 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.220436+0000", 2024-09-20T19:00:19.249 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000149161 2024-09-20T19:00:19.249 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.250 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.250 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,6", 2024-09-20T19:00:19.250 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.220614+0000", 2024-09-20T19:00:19.250 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017738500000000001 2024-09-20T19:00:19.250 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.250 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.250 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:19.250 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.228898+0000", 2024-09-20T19:00:19.250 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0082836590000000005 2024-09-20T19:00:19.250 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.250 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.251 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:00:19.251 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.229000+0000", 2024-09-20T19:00:19.251 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000102329 2024-09-20T19:00:19.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.251 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.251 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:19.251 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.229164+0000", 2024-09-20T19:00:19.251 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016400799999999999 2024-09-20T19:00:19.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.251 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.251 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:19.251 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.229223+0000", 2024-09-20T19:00:19.251 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9064999999999998e-05 2024-09-20T19:00:19.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.252 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.252 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:19.252 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.231504+0000", 2024-09-20T19:00:19.252 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0022812539999999999 2024-09-20T19:00:19.252 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:19.252 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:19.252 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:19.252 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.252 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.253 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:50453 9.b 9:d2c9d3c8:::benchmark_data_smithi083_124552_object50452:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-20T19:00:19.253 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:59:04.116554+0000", 2024-09-20T19:00:19.253 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 75.085769111000005, 2024-09-20T19:00:19.253 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1174190890000002, 2024-09-20T19:00:19.253 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:19.253 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:19.253 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:19.253 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:19.253 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:00:19.253 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:00:19.253 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 50453 2024-09-20T19:00:19.253 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.253 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:19.253 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.253 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:19.253 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.116554+0000", 2024-09-20T19:00:19.254 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:19.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.254 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.254 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:19.254 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.116554+0000", 2024-09-20T19:00:19.254 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:19.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.254 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.254 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:19.254 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.116556+0000", 2024-09-20T19:00:19.254 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.609e-06 2024-09-20T19:00:19.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.254 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.254 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:19.255 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.116581+0000", 2024-09-20T19:00:19.255 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5381e-05 2024-09-20T19:00:19.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.255 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.255 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:19.255 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.116599+0000", 2024-09-20T19:00:19.255 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7138000000000001e-05 2024-09-20T19:00:19.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.255 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.255 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:19.255 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.116608+0000", 2024-09-20T19:00:19.256 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0769999999999997e-06 2024-09-20T19:00:19.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.256 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.256 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:19.256 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.119281+0000", 2024-09-20T19:00:19.256 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0026735550000000002 2024-09-20T19:00:19.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.256 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.256 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:00:19.256 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.119291+0000", 2024-09-20T19:00:19.256 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0304e-05 2024-09-20T19:00:19.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.256 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.257 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:19.257 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.067417+0000", 2024-09-20T19:00:19.257 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.94812586899999995 2024-09-20T19:00:19.257 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.257 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.257 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:00:19.257 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.067424+0000", 2024-09-20T19:00:19.257 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1579999999999999e-06 2024-09-20T19:00:19.257 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.257 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.257 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:19.257 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.222467+0000", 2024-09-20T19:00:19.257 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1550422280000001 2024-09-20T19:00:19.257 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.258 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.258 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:19.258 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.222590+0000", 2024-09-20T19:00:19.258 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012347700000000001 2024-09-20T19:00:19.258 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.258 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.258 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,6", 2024-09-20T19:00:19.258 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.222784+0000", 2024-09-20T19:00:19.258 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000194129 2024-09-20T19:00:19.258 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.258 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.258 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:00:19.258 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.229006+0000", 2024-09-20T19:00:19.258 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0062213340000000002 2024-09-20T19:00:19.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.259 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:19.259 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.229066+0000", 2024-09-20T19:00:19.259 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0715999999999999e-05 2024-09-20T19:00:19.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.259 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:19.259 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.231572+0000", 2024-09-20T19:00:19.259 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0025054209999999999 2024-09-20T19:00:19.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.259 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:19.259 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.231635+0000", 2024-09-20T19:00:19.259 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3614000000000005e-05 2024-09-20T19:00:19.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.260 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.260 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:19.260 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.233973+0000", 2024-09-20T19:00:19.260 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0023380789999999999 2024-09-20T19:00:19.260 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:19.260 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:19.260 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:19.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.260 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.260 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:50521 9.b 9:d0532340:::benchmark_data_smithi083_124552_object50520:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-20T19:00:19.260 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:59:04.171737+0000", 2024-09-20T19:00:19.260 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 75.030586798000002, 2024-09-20T19:00:19.260 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0649375430000001, 2024-09-20T19:00:19.260 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:19.260 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:19.261 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:19.261 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:19.261 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:00:19.261 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:00:19.261 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 50521 2024-09-20T19:00:19.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.261 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:19.261 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.261 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:19.261 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.171737+0000", 2024-09-20T19:00:19.261 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:19.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.261 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.262 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:19.262 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.171737+0000", 2024-09-20T19:00:19.262 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:19.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.262 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.262 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:19.262 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.171739+0000", 2024-09-20T19:00:19.262 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9649999999999998e-06 2024-09-20T19:00:19.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.262 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.262 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:19.262 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.171765+0000", 2024-09-20T19:00:19.262 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6614e-05 2024-09-20T19:00:19.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.262 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.263 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:19.263 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.171782+0000", 2024-09-20T19:00:19.263 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7142999999999998e-05 2024-09-20T19:00:19.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.263 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.263 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:19.263 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.171792+0000", 2024-09-20T19:00:19.263 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5820000000000008e-06 2024-09-20T19:00:19.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.263 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.263 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:19.263 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.171880+0000", 2024-09-20T19:00:19.263 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8052000000000003e-05 2024-09-20T19:00:19.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.263 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.264 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:00:19.264 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.171889+0000", 2024-09-20T19:00:19.264 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7260000000000008e-06 2024-09-20T19:00:19.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.264 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:19.264 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.067447+0000", 2024-09-20T19:00:19.264 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.89555867300000003 2024-09-20T19:00:19.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.264 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:00:19.264 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.067455+0000", 2024-09-20T19:00:19.264 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2819999999999999e-06 2024-09-20T19:00:19.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.265 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:19.265 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.224669+0000", 2024-09-20T19:00:19.265 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1572144769999999 2024-09-20T19:00:19.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.265 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:19.265 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.224787+0000", 2024-09-20T19:00:19.265 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011791200000000001 2024-09-20T19:00:19.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.265 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,6", 2024-09-20T19:00:19.265 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.224899+0000", 2024-09-20T19:00:19.265 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000111538 2024-09-20T19:00:19.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.266 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.266 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:00:19.266 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.229020+0000", 2024-09-20T19:00:19.266 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0041215449999999999 2024-09-20T19:00:19.266 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.266 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.266 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:19.266 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.229111+0000", 2024-09-20T19:00:19.266 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0562999999999993e-05 2024-09-20T19:00:19.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.267 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.267 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:19.267 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.234196+0000", 2024-09-20T19:00:19.267 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0050855149999999997 2024-09-20T19:00:19.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.267 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.267 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:19.267 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.234362+0000", 2024-09-20T19:00:19.267 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000165656 2024-09-20T19:00:19.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.267 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.268 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:19.268 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.236674+0000", 2024-09-20T19:00:19.268 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0023123000000000002 2024-09-20T19:00:19.268 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:19.268 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:19.268 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:19.268 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.268 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.269 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:50523 9.b 9:d03f098e:::benchmark_data_smithi083_124552_object50522:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-20T19:00:19.269 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:59:04.176222+0000", 2024-09-20T19:00:19.269 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 75.026101166000004, 2024-09-20T19:00:19.269 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.086244008, 2024-09-20T19:00:19.269 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:19.269 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:19.269 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:19.270 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:19.270 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:00:19.270 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:00:19.270 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 50523 2024-09-20T19:00:19.270 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.270 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:19.270 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.271 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:19.271 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.176222+0000", 2024-09-20T19:00:19.271 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:19.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.271 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.271 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:19.272 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.176222+0000", 2024-09-20T19:00:19.272 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:19.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.272 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.273 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:00:19.274 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:19.274 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.176224+0000", 2024-09-20T19:00:19.274 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0540000000000002e-06 2024-09-20T19:00:19.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.274 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.274 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:19.275 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.176251+0000", 2024-09-20T19:00:19.275 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6821e-05 2024-09-20T19:00:19.278 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.278 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.278 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:19.278 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.176268+0000", 2024-09-20T19:00:19.278 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.685e-05 2024-09-20T19:00:19.279 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.279 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.279 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:19.279 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.176277+0000", 2024-09-20T19:00:19.279 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6289999999999999e-06 2024-09-20T19:00:19.279 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.280 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.280 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:19.280 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.176339+0000", 2024-09-20T19:00:19.280 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2719999999999996e-05 2024-09-20T19:00:19.280 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.280 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.281 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:00:19.281 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.176348+0000", 2024-09-20T19:00:19.281 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4449999999999996e-06 2024-09-20T19:00:19.281 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.281 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.281 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:19.281 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.068528+0000", 2024-09-20T19:00:19.282 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.89218012300000005 2024-09-20T19:00:19.282 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.282 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.282 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:00:19.282 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.068537+0000", 2024-09-20T19:00:19.282 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5820000000000002e-06 2024-09-20T19:00:19.283 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.283 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.283 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:19.283 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.237005+0000", 2024-09-20T19:00:19.283 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.16846866 2024-09-20T19:00:19.283 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.284 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.284 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:19.284 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.237161+0000", 2024-09-20T19:00:19.284 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000155895 2024-09-20T19:00:19.284 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.284 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.285 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,6", 2024-09-20T19:00:19.285 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.237395+0000", 2024-09-20T19:00:19.285 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00023377200000000001 2024-09-20T19:00:19.285 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.285 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.285 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:19.285 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.244490+0000", 2024-09-20T19:00:19.286 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0070951770000000003 2024-09-20T19:00:19.286 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.286 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.286 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:19.286 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.244542+0000", 2024-09-20T19:00:19.286 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2303999999999999e-05 2024-09-20T19:00:19.287 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.287 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.287 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:00:19.287 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.260004+0000", 2024-09-20T19:00:19.287 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.015462073 2024-09-20T19:00:19.287 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.287 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.288 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:19.288 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.260186+0000", 2024-09-20T19:00:19.288 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000181416 2024-09-20T19:00:19.288 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.288 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.288 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:19.288 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.262466+0000", 2024-09-20T19:00:19.288 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0022804869999999999 2024-09-20T19:00:19.289 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:19.289 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:19.289 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:19.289 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.289 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.289 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:101817 9.b 9:de269700:::benchmark_data_smithi083_124552_object101816:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e275)", 2024-09-20T19:00:19.289 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:59:57.454759+0000", 2024-09-20T19:00:19.289 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 21.747564132000001, 2024-09-20T19:00:19.289 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.25020478899999998, 2024-09-20T19:00:19.289 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:19.289 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:19.289 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:19.290 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:19.290 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:00:19.290 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:00:19.290 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 101817 2024-09-20T19:00:19.290 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.290 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:19.290 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.290 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:19.290 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:57.454759+0000", 2024-09-20T19:00:19.290 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:19.290 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.290 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.290 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:19.291 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:57.454759+0000", 2024-09-20T19:00:19.291 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:19.291 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.291 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.291 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:19.291 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:57.454761+0000", 2024-09-20T19:00:19.291 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4699999999999999e-06 2024-09-20T19:00:19.291 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.291 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.291 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:19.291 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:57.454784+0000", 2024-09-20T19:00:19.291 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3095000000000002e-05 2024-09-20T19:00:19.291 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.291 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.292 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:19.292 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:57.454795+0000", 2024-09-20T19:00:19.292 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1243999999999999e-05 2024-09-20T19:00:19.292 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.292 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.292 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:19.292 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:57.454804+0000", 2024-09-20T19:00:19.292 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4880000000000007e-06 2024-09-20T19:00:19.292 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.292 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.292 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:19.292 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:57.454897+0000", 2024-09-20T19:00:19.292 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3088e-05 2024-09-20T19:00:19.292 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.292 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.293 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:19.293 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:57.455197+0000", 2024-09-20T19:00:19.293 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00030024799999999998 2024-09-20T19:00:19.293 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.293 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.293 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,6", 2024-09-20T19:00:19.293 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:57.455938+0000", 2024-09-20T19:00:19.293 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00074127399999999999 2024-09-20T19:00:19.293 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.293 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.293 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:19.293 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:57.462846+0000", 2024-09-20T19:00:19.293 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0069075999999999999 2024-09-20T19:00:19.293 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.294 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.294 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:00:19.294 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:57.465819+0000", 2024-09-20T19:00:19.294 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0029731580000000001 2024-09-20T19:00:19.294 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.294 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.294 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:19.294 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:57.704440+0000", 2024-09-20T19:00:19.294 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.23862056400000001 2024-09-20T19:00:19.294 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.294 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.294 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:19.294 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:57.704496+0000", 2024-09-20T19:00:19.295 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6178000000000003e-05 2024-09-20T19:00:19.295 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.295 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.295 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:19.295 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:57.704964+0000", 2024-09-20T19:00:19.295 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00046838199999999999 2024-09-20T19:00:19.295 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:19.295 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:19.295 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:19.295 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.295 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.295 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:101835 9.b 9:d30a326b:::benchmark_data_smithi083_124552_object101834:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e275)", 2024-09-20T19:00:19.295 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:59:57.480175+0000", 2024-09-20T19:00:19.296 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 21.722148848, 2024-09-20T19:00:19.296 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.23282166900000001, 2024-09-20T19:00:19.296 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:19.296 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:19.296 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:19.296 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:19.296 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:00:19.296 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:00:19.296 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 101835 2024-09-20T19:00:19.296 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.296 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:19.296 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.296 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:19.296 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:57.480175+0000", 2024-09-20T19:00:19.297 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:19.297 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.297 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.297 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:19.297 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:57.480175+0000", 2024-09-20T19:00:19.297 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:19.297 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.297 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.297 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:19.297 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:57.480176+0000", 2024-09-20T19:00:19.297 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8109999999999999e-06 2024-09-20T19:00:19.297 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.297 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.297 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:19.297 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:57.480204+0000", 2024-09-20T19:00:19.297 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7058000000000001e-05 2024-09-20T19:00:19.298 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.298 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.298 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:19.298 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:57.480216+0000", 2024-09-20T19:00:19.298 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2340999999999999e-05 2024-09-20T19:00:19.298 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.298 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.298 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:19.298 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:57.480228+0000", 2024-09-20T19:00:19.298 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2092e-05 2024-09-20T19:00:19.298 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.298 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.298 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:19.298 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:57.480759+0000", 2024-09-20T19:00:19.299 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00053096600000000001 2024-09-20T19:00:19.299 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.299 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.299 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:19.299 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:57.481301+0000", 2024-09-20T19:00:19.299 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00054161200000000002 2024-09-20T19:00:19.299 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.299 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.299 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,6", 2024-09-20T19:00:19.299 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:57.481481+0000", 2024-09-20T19:00:19.299 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018094500000000001 2024-09-20T19:00:19.299 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.299 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.300 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:00:19.300 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:57.486115+0000", 2024-09-20T19:00:19.300 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0046331819999999996 2024-09-20T19:00:19.300 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.300 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.300 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:19.300 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:57.486434+0000", 2024-09-20T19:00:19.300 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00031927099999999999 2024-09-20T19:00:19.300 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.300 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.300 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:19.300 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:57.712527+0000", 2024-09-20T19:00:19.301 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.22609331399999999 2024-09-20T19:00:19.301 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.301 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.301 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:19.301 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:57.712576+0000", 2024-09-20T19:00:19.301 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8659000000000003e-05 2024-09-20T19:00:19.301 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.301 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.301 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:19.301 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:57.712996+0000", 2024-09-20T19:00:19.301 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00042041799999999999 2024-09-20T19:00:19.302 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:19.302 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:19.302 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:19.302 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.302 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.302 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:118916 9.9 9:97f4c141:::benchmark_data_smithi083_124552_object4769:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e286)", 2024-09-20T19:00:19.302 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:16.029517+0000", 2024-09-20T19:00:19.302 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.1728063569999998, 2024-09-20T19:00:19.302 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.25553782000000003, 2024-09-20T19:00:19.302 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:19.302 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:19.302 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:19.302 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:19.303 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:00:19.303 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:00:19.303 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 118916 2024-09-20T19:00:19.303 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.303 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:19.303 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.303 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:19.303 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029517+0000", 2024-09-20T19:00:19.303 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:19.303 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.303 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.303 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:19.303 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029517+0000", 2024-09-20T19:00:19.303 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:19.304 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.304 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.304 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:19.304 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029518+0000", 2024-09-20T19:00:19.304 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.333e-06 2024-09-20T19:00:19.304 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.304 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.304 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:19.304 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029523+0000", 2024-09-20T19:00:19.304 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8509999999999998e-06 2024-09-20T19:00:19.305 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.305 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.305 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:19.305 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029532+0000", 2024-09-20T19:00:19.305 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1439999999999992e-06 2024-09-20T19:00:19.305 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.305 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.305 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:19.305 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029538+0000", 2024-09-20T19:00:19.305 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9000000000000003e-06 2024-09-20T19:00:19.305 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.305 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.305 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:19.306 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029748+0000", 2024-09-20T19:00:19.306 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000209624 2024-09-20T19:00:19.306 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.306 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.306 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:19.306 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029873+0000", 2024-09-20T19:00:19.306 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012514399999999999 2024-09-20T19:00:19.306 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.306 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.306 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,7", 2024-09-20T19:00:19.306 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029989+0000", 2024-09-20T19:00:19.306 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000116252 2024-09-20T19:00:19.306 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.306 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.307 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:19.307 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.038690+0000", 2024-09-20T19:00:19.307 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0087003329999999993 2024-09-20T19:00:19.307 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.307 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.307 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:00:19.307 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.038701+0000", 2024-09-20T19:00:19.307 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0978000000000001e-05 2024-09-20T19:00:19.307 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.307 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.307 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:19.307 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.284970+0000", 2024-09-20T19:00:19.307 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.24626904599999999 2024-09-20T19:00:19.308 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.308 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.308 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:19.308 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.285017+0000", 2024-09-20T19:00:19.308 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7345999999999997e-05 2024-09-20T19:00:19.308 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.308 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.308 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:19.308 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.285055+0000", 2024-09-20T19:00:19.308 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7868999999999999e-05 2024-09-20T19:00:19.308 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:19.308 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:19.308 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:19.309 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.309 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.309 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:118921 9.b 9:df718a79:::benchmark_data_smithi083_124552_object4774:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e286)", 2024-09-20T19:00:19.309 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:16.029557+0000", 2024-09-20T19:00:19.309 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.1727662219999999, 2024-09-20T19:00:19.309 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.24206374, 2024-09-20T19:00:19.309 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:19.309 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:19.309 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:19.309 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:19.309 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:00:19.309 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:00:19.309 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 118921 2024-09-20T19:00:19.309 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.309 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:19.310 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.310 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:19.310 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029557+0000", 2024-09-20T19:00:19.310 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:19.310 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.310 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.310 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:19.310 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029557+0000", 2024-09-20T19:00:19.310 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:19.310 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.311 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.311 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:19.311 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029558+0000", 2024-09-20T19:00:19.311 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2399999999999999e-07 2024-09-20T19:00:19.311 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.311 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.311 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:19.311 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029560+0000", 2024-09-20T19:00:19.311 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3099999999999999e-06 2024-09-20T19:00:19.311 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:19.312 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:19.313 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:00:19.313 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.314 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:19.314 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029565+0000", 2024-09-20T19:00:19.314 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1580000000000004e-06 2024-09-20T19:00:19.314 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.314 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.314 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:19.315 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029570+0000", 2024-09-20T19:00:19.315 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6920000000000001e-06 2024-09-20T19:00:19.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.315 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.315 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:19.315 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029971+0000", 2024-09-20T19:00:19.315 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000401059 2024-09-20T19:00:19.316 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.316 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.316 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:19.316 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.030068+0000", 2024-09-20T19:00:19.316 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7409000000000002e-05 2024-09-20T19:00:19.316 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.316 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.317 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,6", 2024-09-20T19:00:19.317 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.030167+0000", 2024-09-20T19:00:19.317 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8799000000000001e-05 2024-09-20T19:00:19.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.317 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.317 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:00:19.318 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.038645+0000", 2024-09-20T19:00:19.318 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0084776480000000008 2024-09-20T19:00:19.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.318 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.318 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:19.318 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.038694+0000", 2024-09-20T19:00:19.318 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9453999999999998e-05 2024-09-20T19:00:19.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.319 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.319 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:19.319 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.271539+0000", 2024-09-20T19:00:19.319 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.23284454700000001 2024-09-20T19:00:19.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.320 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.320 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:19.320 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.271583+0000", 2024-09-20T19:00:19.320 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4202000000000001e-05 2024-09-20T19:00:19.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.320 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.320 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:19.321 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.271621+0000", 2024-09-20T19:00:19.321 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8037999999999997e-05 2024-09-20T19:00:19.321 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:19.321 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:19.321 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:19.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.322 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.322 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:118933 9.b 9:d5175723:::benchmark_data_smithi083_124552_object4786:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e286)", 2024-09-20T19:00:19.322 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:16.029579+0000", 2024-09-20T19:00:19.322 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.1727445790000002, 2024-09-20T19:00:19.322 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.242201376, 2024-09-20T19:00:19.323 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:19.323 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:19.323 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:19.323 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:19.323 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:00:19.323 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:00:19.324 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 118933 2024-09-20T19:00:19.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.324 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:19.324 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.324 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:19.324 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029579+0000", 2024-09-20T19:00:19.325 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:19.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.325 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.325 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:19.325 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029579+0000", 2024-09-20T19:00:19.325 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:19.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.326 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.326 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:19.326 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029579+0000", 2024-09-20T19:00:19.326 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6800000000000001e-07 2024-09-20T19:00:19.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.326 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.327 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:19.327 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029581+0000", 2024-09-20T19:00:19.327 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7829999999999999e-06 2024-09-20T19:00:19.327 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.327 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.327 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:19.327 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029586+0000", 2024-09-20T19:00:19.328 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6079999999999998e-06 2024-09-20T19:00:19.328 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.328 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.328 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:19.328 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029590+0000", 2024-09-20T19:00:19.328 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3810000000000001e-06 2024-09-20T19:00:19.328 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.329 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.329 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:19.329 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.032837+0000", 2024-09-20T19:00:19.329 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0032474190000000001 2024-09-20T19:00:19.329 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.329 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.329 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:19.329 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.032956+0000", 2024-09-20T19:00:19.329 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0001182 2024-09-20T19:00:19.329 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.330 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.330 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,6", 2024-09-20T19:00:19.330 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.033054+0000", 2024-09-20T19:00:19.330 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8815999999999997e-05 2024-09-20T19:00:19.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.330 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.330 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:00:19.330 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.038651+0000", 2024-09-20T19:00:19.330 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0055967020000000003 2024-09-20T19:00:19.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.330 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.330 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:19.330 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.271500+0000", 2024-09-20T19:00:19.331 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.23284850600000001 2024-09-20T19:00:19.331 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.331 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.331 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:19.331 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.271691+0000", 2024-09-20T19:00:19.331 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000191032 2024-09-20T19:00:19.331 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.331 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.331 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:19.331 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.271746+0000", 2024-09-20T19:00:19.331 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5411999999999997e-05 2024-09-20T19:00:19.331 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.331 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.331 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:19.331 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.271780+0000", 2024-09-20T19:00:19.332 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4149000000000001e-05 2024-09-20T19:00:19.332 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:19.332 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:19.332 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:19.332 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.332 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.332 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:118941 9.9 9:9eaa040b:::benchmark_data_smithi083_124552_object4794:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e286)", 2024-09-20T19:00:19.332 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:16.029595+0000", 2024-09-20T19:00:19.332 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.17272803, 2024-09-20T19:00:19.332 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.25558177100000001, 2024-09-20T19:00:19.332 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:19.332 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:19.332 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:19.332 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:19.333 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:00:19.333 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:00:19.333 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 118941 2024-09-20T19:00:19.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.333 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:19.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.333 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:19.333 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029595+0000", 2024-09-20T19:00:19.333 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:19.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.333 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:19.333 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029595+0000", 2024-09-20T19:00:19.333 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:19.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.334 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.334 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:19.334 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029596+0000", 2024-09-20T19:00:19.334 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.15e-07 2024-09-20T19:00:19.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.334 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.334 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:19.334 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029597+0000", 2024-09-20T19:00:19.334 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7099999999999999e-06 2024-09-20T19:00:19.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.334 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.334 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:19.334 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029602+0000", 2024-09-20T19:00:19.335 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2570000000000001e-06 2024-09-20T19:00:19.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.335 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.335 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:19.335 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029606+0000", 2024-09-20T19:00:19.335 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.206e-06 2024-09-20T19:00:19.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.335 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.335 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:19.335 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.032741+0000", 2024-09-20T19:00:19.335 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0031351249999999999 2024-09-20T19:00:19.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.335 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.335 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:19.336 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.032865+0000", 2024-09-20T19:00:19.336 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012376600000000001 2024-09-20T19:00:19.336 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.336 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.336 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,7", 2024-09-20T19:00:19.336 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.032973+0000", 2024-09-20T19:00:19.336 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000108337 2024-09-20T19:00:19.336 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.337 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.337 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:00:19.337 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.038705+0000", 2024-09-20T19:00:19.337 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00573187 2024-09-20T19:00:19.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.337 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.337 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:19.337 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.284780+0000", 2024-09-20T19:00:19.337 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.24607500199999999 2024-09-20T19:00:19.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.337 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.337 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:19.337 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.285090+0000", 2024-09-20T19:00:19.338 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00030969500000000002 2024-09-20T19:00:19.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.338 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.338 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:19.338 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.285144+0000", 2024-09-20T19:00:19.338 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4666999999999997e-05 2024-09-20T19:00:19.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.338 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.338 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:19.338 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.285177+0000", 2024-09-20T19:00:19.338 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2821000000000003e-05 2024-09-20T19:00:19.338 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:19.338 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:19.338 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:19.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.339 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.339 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:118943 9.9 9:9ccbab0c:::benchmark_data_smithi083_124552_object4796:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e286)", 2024-09-20T19:00:19.339 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:16.029611+0000", 2024-09-20T19:00:19.339 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.1727128599999999, 2024-09-20T19:00:19.339 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.25568317800000001, 2024-09-20T19:00:19.339 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:19.339 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:19.339 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:19.339 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:19.339 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:00:19.339 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:00:19.339 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 118943 2024-09-20T19:00:19.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.340 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:19.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.340 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:19.340 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029611+0000", 2024-09-20T19:00:19.340 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:19.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.340 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:19.340 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029611+0000", 2024-09-20T19:00:19.340 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:19.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.340 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:19.340 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029611+0000", 2024-09-20T19:00:19.340 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9700000000000003e-07 2024-09-20T19:00:19.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.341 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.341 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:19.341 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029613+0000", 2024-09-20T19:00:19.341 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6130000000000001e-06 2024-09-20T19:00:19.341 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.341 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.341 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:19.341 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029617+0000", 2024-09-20T19:00:19.341 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0670000000000002e-06 2024-09-20T19:00:19.341 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.341 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.341 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:19.341 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029631+0000", 2024-09-20T19:00:19.341 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4413000000000001e-05 2024-09-20T19:00:19.341 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.341 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.342 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:19.342 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.035755+0000", 2024-09-20T19:00:19.342 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0061240690000000002 2024-09-20T19:00:19.342 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.342 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.342 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:19.342 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.035860+0000", 2024-09-20T19:00:19.342 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000104755 2024-09-20T19:00:19.342 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.342 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.342 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,7", 2024-09-20T19:00:19.342 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.035972+0000", 2024-09-20T19:00:19.343 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000111956 2024-09-20T19:00:19.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.343 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.343 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:19.343 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.284840+0000", 2024-09-20T19:00:19.343 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.24886845499999999 2024-09-20T19:00:19.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.343 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.343 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:00:19.343 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.284862+0000", 2024-09-20T19:00:19.343 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1807999999999999e-05 2024-09-20T19:00:19.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.343 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.343 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:19.343 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.285222+0000", 2024-09-20T19:00:19.343 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00035969300000000001 2024-09-20T19:00:19.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.344 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.344 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:19.344 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.285263+0000", 2024-09-20T19:00:19.344 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0782999999999999e-05 2024-09-20T19:00:19.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.344 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.344 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:19.344 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.285294+0000", 2024-09-20T19:00:19.344 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1269000000000002e-05 2024-09-20T19:00:19.344 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:19.344 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:19.344 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:19.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.344 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.344 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:118948 9.9 9:9435db55:::benchmark_data_smithi083_124552_object4801:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e286)", 2024-09-20T19:00:19.345 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:16.029637+0000", 2024-09-20T19:00:19.345 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.1726866290000002, 2024-09-20T19:00:19.345 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.25576086199999998, 2024-09-20T19:00:19.345 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:19.345 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:19.345 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:19.345 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:19.345 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:00:19.345 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:00:19.345 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 118948 2024-09-20T19:00:19.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.346 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:19.346 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.346 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:19.346 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029637+0000", 2024-09-20T19:00:19.346 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:19.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.346 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.346 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:19.346 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029637+0000", 2024-09-20T19:00:19.346 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:19.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.346 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.346 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:19.347 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029637+0000", 2024-09-20T19:00:19.347 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7500000000000001e-07 2024-09-20T19:00:19.347 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.347 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.347 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:19.347 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029639+0000", 2024-09-20T19:00:19.347 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.939e-06 2024-09-20T19:00:19.347 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.347 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.347 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:19.347 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029644+0000", 2024-09-20T19:00:19.347 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6609999999999999e-06 2024-09-20T19:00:19.347 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.347 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.348 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:19.348 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029648+0000", 2024-09-20T19:00:19.348 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2520000000000001e-06 2024-09-20T19:00:19.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.348 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.348 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:19.348 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.038756+0000", 2024-09-20T19:00:19.348 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0091078630000000008 2024-09-20T19:00:19.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.348 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.348 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:19.348 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.038873+0000", 2024-09-20T19:00:19.348 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000116883 2024-09-20T19:00:19.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.348 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.348 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,7", 2024-09-20T19:00:19.349 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.039256+0000", 2024-09-20T19:00:19.349 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000383015 2024-09-20T19:00:19.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.349 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.349 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:00:19.349 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.284868+0000", 2024-09-20T19:00:19.349 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.245612208 2024-09-20T19:00:19.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.349 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.349 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:19.349 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.284914+0000", 2024-09-20T19:00:19.349 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6406999999999998e-05 2024-09-20T19:00:19.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.349 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.350 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:19.350 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.285328+0000", 2024-09-20T19:00:19.350 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00041334299999999999 2024-09-20T19:00:19.350 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.350 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.350 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:19.350 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.285368+0000", 2024-09-20T19:00:19.350 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0559999999999998e-05 2024-09-20T19:00:19.350 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.350 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.350 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:19.350 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.285398+0000", 2024-09-20T19:00:19.350 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9356000000000002e-05 2024-09-20T19:00:19.351 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:19.351 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:19.351 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:19.351 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.351 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.351 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:118955 9.b 9:db990ba2:::benchmark_data_smithi083_124552_object4808:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e286)", 2024-09-20T19:00:19.351 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:16.029653+0000", 2024-09-20T19:00:19.351 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.172670943, 2024-09-20T19:00:19.351 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.245852342, 2024-09-20T19:00:19.351 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:19.351 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:19.352 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:19.352 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:19.352 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:00:19.352 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:00:19.352 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 118955 2024-09-20T19:00:19.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.352 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:19.352 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.352 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:19.352 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029653+0000", 2024-09-20T19:00:19.352 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:19.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.352 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.353 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:19.353 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029653+0000", 2024-09-20T19:00:19.353 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:19.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.353 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.353 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:19.353 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029653+0000", 2024-09-20T19:00:19.353 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3000000000000002e-07 2024-09-20T19:00:19.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.353 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.353 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:19.353 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029655+0000", 2024-09-20T19:00:19.353 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6670000000000001e-06 2024-09-20T19:00:19.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.354 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.354 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:19.354 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029659+0000", 2024-09-20T19:00:19.354 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.16e-06 2024-09-20T19:00:19.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.354 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.354 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:19.354 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029663+0000", 2024-09-20T19:00:19.354 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9910000000000004e-06 2024-09-20T19:00:19.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.354 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.354 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:19.354 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.035750+0000", 2024-09-20T19:00:19.355 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0060874689999999999 2024-09-20T19:00:19.355 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.355 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.355 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:19.355 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.035860+0000", 2024-09-20T19:00:19.355 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011000100000000001 2024-09-20T19:00:19.355 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.355 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.355 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,6", 2024-09-20T19:00:19.355 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.035972+0000", 2024-09-20T19:00:19.355 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000111637 2024-09-20T19:00:19.355 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.356 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.356 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:00:19.356 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.038802+0000", 2024-09-20T19:00:19.356 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0028307010000000001 2024-09-20T19:00:19.356 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.356 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.356 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:19.356 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.271655+0000", 2024-09-20T19:00:19.356 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.23285298500000001 2024-09-20T19:00:19.356 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.356 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.356 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:19.356 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.275420+0000", 2024-09-20T19:00:19.357 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003764421 2024-09-20T19:00:19.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.357 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.357 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:19.357 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.275465+0000", 2024-09-20T19:00:19.357 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5157999999999997e-05 2024-09-20T19:00:19.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.357 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.357 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:19.357 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.275505+0000", 2024-09-20T19:00:19.357 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9821999999999999e-05 2024-09-20T19:00:19.357 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:19.357 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:19.357 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:19.358 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.358 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.358 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:118962 9.9 9:99b4eb3b:::benchmark_data_smithi083_124552_object4815:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e286)", 2024-09-20T19:00:19.358 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:16.029692+0000", 2024-09-20T19:00:19.358 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.1726313340000001, 2024-09-20T19:00:19.358 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.25597717599999997, 2024-09-20T19:00:19.358 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:19.358 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:19.358 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:19.358 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:19.358 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:00:19.358 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:00:19.358 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 118962 2024-09-20T19:00:19.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.359 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:19.359 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.359 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:19.359 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029692+0000", 2024-09-20T19:00:19.359 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:19.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.359 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.359 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:19.359 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029692+0000", 2024-09-20T19:00:19.359 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:19.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.360 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.360 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:19.360 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029692+0000", 2024-09-20T19:00:19.360 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2399999999999999e-07 2024-09-20T19:00:19.360 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.360 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.360 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:19.360 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029695+0000", 2024-09-20T19:00:19.360 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.153e-06 2024-09-20T19:00:19.360 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.360 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.360 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:19.360 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029700+0000", 2024-09-20T19:00:19.361 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.22e-06 2024-09-20T19:00:19.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.361 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.361 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:19.361 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.029704+0000", 2024-09-20T19:00:19.361 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5310000000000003e-06 2024-09-20T19:00:19.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.361 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.361 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:19.361 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.271544+0000", 2024-09-20T19:00:19.361 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.241839254 2024-09-20T19:00:19.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.362 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.362 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:19.362 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.271665+0000", 2024-09-20T19:00:19.362 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121128 2024-09-20T19:00:19.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.362 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.362 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,7", 2024-09-20T19:00:19.362 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.271929+0000", 2024-09-20T19:00:19.362 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00026400600000000001 2024-09-20T19:00:19.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.362 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.362 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:00:19.363 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.284878+0000", 2024-09-20T19:00:19.363 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.012949153 2024-09-20T19:00:19.363 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.363 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.363 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:19.363 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.285544+0000", 2024-09-20T19:00:19.363 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00066605500000000003 2024-09-20T19:00:19.363 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.363 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.363 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:19.363 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.285586+0000", 2024-09-20T19:00:19.363 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2302999999999998e-05 2024-09-20T19:00:19.363 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.363 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.364 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:19.364 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.285638+0000", 2024-09-20T19:00:19.364 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2210000000000003e-05 2024-09-20T19:00:19.364 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.364 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.364 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:19.364 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.285669+0000", 2024-09-20T19:00:19.364 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0839000000000001e-05 2024-09-20T19:00:19.364 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:19.364 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:19.365 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:19.365 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.365 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.365 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:118968 9.9 9:97a6479c:::benchmark_data_smithi083_124552_object4821:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e286)", 2024-09-20T19:00:19.365 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:16.034397+0000", 2024-09-20T19:00:19.365 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.167926547, 2024-09-20T19:00:19.366 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.27485015099999999, 2024-09-20T19:00:19.366 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:19.366 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:19.366 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:19.366 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:19.366 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:00:19.366 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:00:19.366 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 118968 2024-09-20T19:00:19.366 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.366 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:19.366 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.366 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:19.366 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.034397+0000", 2024-09-20T19:00:19.366 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:19.367 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.367 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.367 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:19.367 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.034397+0000", 2024-09-20T19:00:19.367 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:19.367 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.367 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.367 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:19.367 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.034398+0000", 2024-09-20T19:00:19.367 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.316e-06 2024-09-20T19:00:19.367 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.367 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.367 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:19.367 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.034402+0000", 2024-09-20T19:00:19.368 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0199999999999996e-06 2024-09-20T19:00:19.368 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.368 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.368 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:19.368 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.034411+0000", 2024-09-20T19:00:19.368 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9139999999999997e-06 2024-09-20T19:00:19.368 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.368 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.368 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:19.368 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.034418+0000", 2024-09-20T19:00:19.368 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9140000000000002e-06 2024-09-20T19:00:19.368 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.368 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.368 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:19.369 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.278430+0000", 2024-09-20T19:00:19.369 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.24401189500000001 2024-09-20T19:00:19.369 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.369 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.369 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:19.369 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.278545+0000", 2024-09-20T19:00:19.369 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000114801 2024-09-20T19:00:19.369 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.369 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.369 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,7", 2024-09-20T19:00:19.369 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.278660+0000", 2024-09-20T19:00:19.369 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000114837 2024-09-20T19:00:19.369 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.369 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.370 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:19.370 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.285713+0000", 2024-09-20T19:00:19.370 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0070530310000000004 2024-09-20T19:00:19.370 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.370 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.370 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:19.370 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.285744+0000", 2024-09-20T19:00:19.370 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0908000000000003e-05 2024-09-20T19:00:19.370 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.370 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.370 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:00:19.370 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.309115+0000", 2024-09-20T19:00:19.370 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.023371831999999999 2024-09-20T19:00:19.370 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.371 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.371 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:19.371 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.309204+0000", 2024-09-20T19:00:19.371 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8317000000000002e-05 2024-09-20T19:00:19.371 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.371 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.371 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:19.371 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.309247+0000", 2024-09-20T19:00:19.371 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3365999999999998e-05 2024-09-20T19:00:19.371 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:19.371 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:19.371 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:19.371 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.371 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.372 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:118969 9.5 9:aaa29e57:::benchmark_data_smithi083_124552_object4822:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e286)", 2024-09-20T19:00:19.372 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:16.034435+0000", 2024-09-20T19:00:19.372 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.167888262, 2024-09-20T19:00:19.372 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.23710205200000001, 2024-09-20T19:00:19.372 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:19.372 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:19.372 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:19.372 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:19.372 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:00:19.372 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:00:19.372 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 118969 2024-09-20T19:00:19.372 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.372 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:19.372 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.372 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:19.373 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.034435+0000", 2024-09-20T19:00:19.373 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:19.373 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.373 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.373 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:19.373 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.034435+0000", 2024-09-20T19:00:19.373 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:19.373 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.373 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.373 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:19.373 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.034436+0000", 2024-09-20T19:00:19.373 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7e-07 2024-09-20T19:00:19.373 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.373 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.374 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:19.374 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.034439+0000", 2024-09-20T19:00:19.374 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.033e-06 2024-09-20T19:00:19.374 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.374 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.374 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:19.374 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.034445+0000", 2024-09-20T19:00:19.374 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1140000000000002e-06 2024-09-20T19:00:19.374 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.374 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.374 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:19.374 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.034450+0000", 2024-09-20T19:00:19.375 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2610000000000001e-06 2024-09-20T19:00:19.375 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.375 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.375 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:19.375 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.034497+0000", 2024-09-20T19:00:19.375 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6579000000000002e-05 2024-09-20T19:00:19.375 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.375 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.375 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:19.375 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.034577+0000", 2024-09-20T19:00:19.375 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0185000000000006e-05 2024-09-20T19:00:19.375 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.375 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.375 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,7", 2024-09-20T19:00:19.376 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.034639+0000", 2024-09-20T19:00:19.376 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2329000000000006e-05 2024-09-20T19:00:19.376 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.376 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.376 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:00:19.376 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.037011+0000", 2024-09-20T19:00:19.376 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0023720109999999998 2024-09-20T19:00:19.376 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.376 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.376 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:19.376 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.041364+0000", 2024-09-20T19:00:19.376 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0043530060000000004 2024-09-20T19:00:19.376 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.376 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.377 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:19.377 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.041889+0000", 2024-09-20T19:00:19.377 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00052468899999999995 2024-09-20T19:00:19.377 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.377 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.377 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:19.377 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.271491+0000", 2024-09-20T19:00:19.377 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.229601787 2024-09-20T19:00:19.377 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.377 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.377 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:19.377 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.271537+0000", 2024-09-20T19:00:19.377 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6588000000000002e-05 2024-09-20T19:00:19.377 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:19.378 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:19.378 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:19.378 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.378 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.378 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:118970 9.9 9:94c7d343:::benchmark_data_smithi083_124552_object4823:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e286)", 2024-09-20T19:00:19.378 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:16.038092+0000", 2024-09-20T19:00:19.378 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.1642310419999999, 2024-09-20T19:00:19.378 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.271405116, 2024-09-20T19:00:19.378 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:19.378 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:19.378 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:19.378 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:19.378 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:00:19.378 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:00:19.379 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 118970 2024-09-20T19:00:19.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.379 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:19.379 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.379 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:19.379 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.038092+0000", 2024-09-20T19:00:19.379 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:19.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.379 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.379 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:19.379 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.038092+0000", 2024-09-20T19:00:19.379 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:19.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.379 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.380 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:19.380 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.038094+0000", 2024-09-20T19:00:19.380 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.601e-06 2024-09-20T19:00:19.380 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.380 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.380 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:19.380 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.038099+0000", 2024-09-20T19:00:19.380 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3360000000000002e-06 2024-09-20T19:00:19.380 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.380 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.380 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:19.380 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.038108+0000", 2024-09-20T19:00:19.380 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9819999999999997e-06 2024-09-20T19:00:19.380 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.380 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.381 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:19.381 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.038114+0000", 2024-09-20T19:00:19.381 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9229999999999999e-06 2024-09-20T19:00:19.381 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.381 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.381 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:19.381 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.285770+0000", 2024-09-20T19:00:19.381 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.24765589599999999 2024-09-20T19:00:19.381 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.381 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.381 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:19.381 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.285880+0000", 2024-09-20T19:00:19.382 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000109381 2024-09-20T19:00:19.382 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.382 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.382 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,7", 2024-09-20T19:00:19.382 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.285984+0000", 2024-09-20T19:00:19.382 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010437699999999999 2024-09-20T19:00:19.382 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.382 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.382 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:19.382 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.309100+0000", 2024-09-20T19:00:19.382 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.023115693999999999 2024-09-20T19:00:19.382 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.382 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.383 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:00:19.383 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.309258+0000", 2024-09-20T19:00:19.383 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015836800000000001 2024-09-20T19:00:19.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.383 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.383 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:19.383 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.309381+0000", 2024-09-20T19:00:19.383 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012310500000000001 2024-09-20T19:00:19.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.383 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.383 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:19.383 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.309462+0000", 2024-09-20T19:00:19.383 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0748999999999995e-05 2024-09-20T19:00:19.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.384 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.384 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:19.384 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.309498+0000", 2024-09-20T19:00:19.384 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5704000000000002e-05 2024-09-20T19:00:19.384 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:19.384 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:19.384 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:19.384 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.384 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.384 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:118976 9.b 9:d3ced886:::benchmark_data_smithi083_124552_object4829:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e286)", 2024-09-20T19:00:19.384 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:16.044070+0000", 2024-09-20T19:00:19.384 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.1582532639999998, 2024-09-20T19:00:19.384 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.24002256599999999, 2024-09-20T19:00:19.384 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:19.385 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:19.385 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:19.385 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:19.385 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:00:19.385 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:00:19.385 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 118976 2024-09-20T19:00:19.385 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.385 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:19.385 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.385 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:19.385 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.044070+0000", 2024-09-20T19:00:19.385 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:19.385 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.386 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.386 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:19.386 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.044070+0000", 2024-09-20T19:00:19.386 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:19.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.386 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.386 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:19.386 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.044072+0000", 2024-09-20T19:00:19.386 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.494e-06 2024-09-20T19:00:19.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.386 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.386 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:19.386 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.044077+0000", 2024-09-20T19:00:19.386 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7080000000000002e-06 2024-09-20T19:00:19.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.387 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.387 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:19.387 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.271420+0000", 2024-09-20T19:00:19.387 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.227342657 2024-09-20T19:00:19.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.387 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.387 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:19.387 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.271436+0000", 2024-09-20T19:00:19.387 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5716999999999999e-05 2024-09-20T19:00:19.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.387 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.387 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:19.387 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.276066+0000", 2024-09-20T19:00:19.388 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.004630482 2024-09-20T19:00:19.388 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.388 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.388 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:19.388 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.276213+0000", 2024-09-20T19:00:19.388 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000147198 2024-09-20T19:00:19.388 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.388 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.388 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,6", 2024-09-20T19:00:19.388 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.276317+0000", 2024-09-20T19:00:19.388 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000103175 2024-09-20T19:00:19.388 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.388 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.388 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:00:19.388 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.283815+0000", 2024-09-20T19:00:19.389 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.007497849 2024-09-20T19:00:19.389 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.389 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.389 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:19.389 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.283951+0000", 2024-09-20T19:00:19.389 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013610099999999999 2024-09-20T19:00:19.389 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.389 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.389 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:19.389 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.284003+0000", 2024-09-20T19:00:19.389 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2753999999999997e-05 2024-09-20T19:00:19.389 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.389 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.389 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:19.389 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.284045+0000", 2024-09-20T19:00:19.390 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1471999999999999e-05 2024-09-20T19:00:19.390 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:19.390 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:19.390 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:19.390 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.284093+0000", 2024-09-20T19:00:19.390 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7958999999999999e-05 2024-09-20T19:00:19.390 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:19.390 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:19.390 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:19.390 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:19.390 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:19.390 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T19:00:19.414 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:00:19.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:19.448 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:19.448 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:00:19.549 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:00:19.580 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:19.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:19.581 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:00:19.669 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:19 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:00:19.669 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:19 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:00:19.669 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:19 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:00:19.670 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:19 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:00:19.670 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:19 smithi083 ceph-mon[29421]: osdmap e288: 8 total, 8 up, 8 in 2024-09-20T19:00:19.670 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:19 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:00:19.670 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:19 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:00:19.670 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:19 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:00:19.670 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:19 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:00:19.671 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:19 smithi083 ceph-mon[36260]: osdmap e288: 8 total, 8 up, 8 in 2024-09-20T19:00:19.682 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:19.727 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:19.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:19.731 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:19.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:19 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:00:19.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:19 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:00:19.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:19 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:00:19.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:19 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:00:19.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:19 smithi142 ceph-mon[32508]: osdmap e288: 8 total, 8 up, 8 in 2024-09-20T19:00:19.831 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:00:19.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:19.862 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:19.862 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:00:19.963 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:19.996 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:19.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:19.997 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:20.060 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T19:00:20.098 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:20.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:20.145 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:20.145 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:20.246 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:20.287 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:20.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:20.288 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:20.334 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-20T19:00:20.388 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:00:20.416 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:20.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:20.417 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:00:20.517 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:20.543 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:20.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:20.544 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:20.644 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:20.691 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:20.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:20.692 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:20.730 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:20 smithi083 ceph-mon[29421]: pgmap v519: 148 pgs: 148 active+clean; 6.6 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1023 B/s rd, 170 B/s wr, 876 op/s 2024-09-20T19:00:20.730 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:20 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/839734156' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:00:20.731 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:20 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:00:20.731 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:20 smithi083 ceph-mon[29421]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:00:20.731 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:20 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T19:00:20.731 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:20 smithi083 ceph-mon[29421]: osdmap e289: 8 total, 8 up, 8 in 2024-09-20T19:00:20.731 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:20 smithi083 ceph-mon[36260]: pgmap v519: 148 pgs: 148 active+clean; 6.6 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1023 B/s rd, 170 B/s wr, 876 op/s 2024-09-20T19:00:20.731 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:20 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/839734156' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:00:20.731 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:20 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:00:20.731 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:20 smithi083 ceph-mon[36260]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:00:20.731 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:20 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T19:00:20.731 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:20 smithi083 ceph-mon[36260]: osdmap e289: 8 total, 8 up, 8 in 2024-09-20T19:00:20.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:20 smithi142 ceph-mon[32508]: pgmap v519: 148 pgs: 148 active+clean; 6.6 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1023 B/s rd, 170 B/s wr, 876 op/s 2024-09-20T19:00:20.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:20 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/839734156' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:00:20.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:20 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:00:20.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:20 smithi142 ceph-mon[32508]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:00:20.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:20 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T19:00:20.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:20 smithi142 ceph-mon[32508]: osdmap e289: 8 total, 8 up, 8 in 2024-09-20T19:00:20.792 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:20.815 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T19:00:20.819 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-20T19:00:20.833 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:20.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:20.834 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:20.936 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:20.969 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:20.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:20.970 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:21.046 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:00:20 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:00:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:00:21.071 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:21.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:21.093 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:21.093 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:21.194 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:21.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:21.215 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:21.216 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:21.316 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:21.352 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:21.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:21.353 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:21.454 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:21.489 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:21.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:21.490 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:21.591 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:21.623 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:21.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:21.624 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:21.724 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:21.757 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:21.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:21.758 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:21.859 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:00:21.892 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:21.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:21.893 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:00:21.994 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:00:22.017 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:21 smithi142 ceph-mon[32508]: from='client.15714 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:00:22.017 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:21 smithi142 ceph-mon[32508]: pgmap v521: 148 pgs: 148 active+clean; 6.6 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.8 KiB/s rd, 170 B/s wr, 1.06k op/s 2024-09-20T19:00:22.030 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:22.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:22.032 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:00:22.132 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:00:22.163 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:22.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:22.164 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:00:22.265 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:00:22.296 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:21 smithi083 ceph-mon[29421]: from='client.15714 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:00:22.296 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:21 smithi083 ceph-mon[29421]: pgmap v521: 148 pgs: 148 active+clean; 6.6 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.8 KiB/s rd, 170 B/s wr, 1.06k op/s 2024-09-20T19:00:22.296 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:21 smithi083 ceph-mon[36260]: from='client.15714 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:00:22.296 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:21 smithi083 ceph-mon[36260]: pgmap v521: 148 pgs: 148 active+clean; 6.6 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.8 KiB/s rd, 170 B/s wr, 1.06k op/s 2024-09-20T19:00:22.300 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:22.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:22.301 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:00:22.402 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:00:22.438 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:22.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:22.439 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:00:22.540 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:00:22.574 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:22.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:22.576 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:00:22.676 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:22.712 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:22.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:22.713 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:22.813 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:22.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:22.845 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:22.846 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:22.946 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:00:22.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:22.986 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:22.986 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:00:23.087 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:23.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:23.120 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:23.120 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:23.138 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-20T19:00:23.221 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:00:23.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:23.282 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:23.282 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:00:23.383 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:23.435 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:23.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:23.437 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:23.537 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:23.572 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:23.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:23.573 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:23.584 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:00:23.632 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":520,"stamp":"2024-09-20T19:00:20.327676+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":7136877107,"num_objects":109091,"num_object_clones":0,"num_object_copies":327273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109091,"num_whiteouts":0,"num_read":1899,"num_read_kb":1756,"num_write":120233,"num_write_kb":7306890,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9653,"num_bytes_recovered":624300722,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83929,"ondisk_log_size":83929,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":447,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":24373652,"kb_used_data":21372352,"kb_used_omap":12,"kb_used_meta":3000947,"kb_avail":725587564,"statfs":{"total":767960285184,"available":743001665536,"internally_reserved":0,"allocated":21885288448,"data_stored":21882025311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":3072970288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,2,3,6,2,0,0,5,8],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-344260584,"num_objects":-5252,"num_object_clones":0,"num_object_copies":-15756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5252,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":5254,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.055886"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":495,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.307802+0000","last_change":"2024-09-20T18:55:28.037494+0000","last_active":"2024-09-20T19:00:16.307802+0000","last_peered":"2024-09-20T19:00:16.307802+0000","last_clean":"2024-09-20T19:00:16.307802+0000","last_became_active":"2024-09-20T18:49:45.943201+0000","last_became_peered":"2024-09-20T18:49:45.943201+0000","last_unstale":"2024-09-20T19:00:16.307802+0000","last_undegraded":"2024-09-20T19:00:16.307802+0000","last_fullsized":"2024-09-20T19:00:16.307802+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:28.037449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:28.037449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"273'15","reported_seq":565,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.270045+0000","last_change":"2024-09-20T18:59:05.094771+0000","last_active":"2024-09-20T19:00:14.270045+0000","last_peered":"2024-09-20T19:00:14.270045+0000","last_clean":"2024-09-20T19:00:14.270045+0000","last_became_active":"2024-09-20T18:59:05.076159+0000","last_became_peered":"2024-09-20T18:59:05.076159+0000","last_unstale":"2024-09-20T19:00:14.270045+0000","last_undegraded":"2024-09-20T19:00:14.270045+0000","last_fullsized":"2024-09-20T19:00:14.270045+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:50.231620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:50.231620+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:25.223399+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":493,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.296975+0000","last_change":"2024-09-20T18:57:49.016078+0000","last_active":"2024-09-20T19:00:16.296975+0000","last_peered":"2024-09-20T19:00:16.296975+0000","last_clean":"2024-09-20T19:00:16.296975+0000","last_became_active":"2024-09-20T18:49:49.959514+0000","last_became_peered":"2024-09-20T18:49:49.959514+0000","last_unstale":"2024-09-20T19:00:16.296975+0000","last_undegraded":"2024-09-20T19:00:16.296975+0000","last_fullsized":"2024-09-20T19:00:16.296975+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:49.016006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:49.016006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":489,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.270536+0000","last_change":"2024-09-20T18:58:02.084435+0000","last_active":"2024-09-20T19:00:14.270536+0000","last_peered":"2024-09-20T19:00:14.270536+0000","last_clean":"2024-09-20T19:00:14.270536+0000","last_became_active":"2024-09-20T18:49:51.969421+0000","last_became_peered":"2024-09-20T18:49:51.969421+0000","last_unstale":"2024-09-20T19:00:14.270536+0000","last_undegraded":"2024-09-20T19:00:14.270536+0000","last_fullsized":"2024-09-20T19:00:14.270536+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:02.084375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:02.084375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:43.288290+0000","scrub_duration":197,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":495,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.302536+0000","last_change":"2024-09-20T18:59:05.081431+0000","last_active":"2024-09-20T19:00:16.302536+0000","last_peered":"2024-09-20T19:00:16.302536+0000","last_clean":"2024-09-20T19:00:16.302536+0000","last_became_active":"2024-09-20T18:59:05.081081+0000","last_became_peered":"2024-09-20T18:59:05.081081+0000","last_unstale":"2024-09-20T19:00:16.302536+0000","last_undegraded":"2024-09-20T19:00:16.302536+0000","last_fullsized":"2024-09-20T19:00:16.302536+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.216188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.216188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":508,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.302614+0000","last_change":"2024-09-20T18:59:05.073093+0000","last_active":"2024-09-20T19:00:16.302614+0000","last_peered":"2024-09-20T19:00:16.302614+0000","last_clean":"2024-09-20T19:00:16.302614+0000","last_became_active":"2024-09-20T18:59:05.072675+0000","last_became_peered":"2024-09-20T18:59:05.072675+0000","last_unstale":"2024-09-20T19:00:16.302614+0000","last_undegraded":"2024-09-20T19:00:16.302614+0000","last_fullsized":"2024-09-20T19:00:16.302614+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:03.055929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:59:03.055929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:26.275543+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"274'27","reported_seq":532,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.302384+0000","last_change":"2024-09-20T18:58:57.097877+0000","last_active":"2024-09-20T19:00:16.302384+0000","last_peered":"2024-09-20T19:00:16.302384+0000","last_clean":"2024-09-20T19:00:16.302384+0000","last_became_active":"2024-09-20T18:49:47.952150+0000","last_became_peered":"2024-09-20T18:49:47.952150+0000","last_unstale":"2024-09-20T19:00:16.302384+0000","last_undegraded":"2024-09-20T19:00:16.302384+0000","last_fullsized":"2024-09-20T19:00:16.302384+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:58:57.097766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:57.097766+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":38,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":499,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.269952+0000","last_change":"2024-09-20T18:59:05.077076+0000","last_active":"2024-09-20T19:00:14.269952+0000","last_peered":"2024-09-20T19:00:14.269952+0000","last_clean":"2024-09-20T19:00:14.269952+0000","last_became_active":"2024-09-20T18:59:05.076607+0000","last_became_peered":"2024-09-20T18:59:05.076607+0000","last_unstale":"2024-09-20T19:00:14.269952+0000","last_undegraded":"2024-09-20T19:00:14.269952+0000","last_fullsized":"2024-09-20T19:00:14.269952+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:06.846018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:06.846018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:45.340583+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"59'1","reported_seq":535,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.279766+0000","last_change":"2024-09-20T18:59:05.111316+0000","last_active":"2024-09-20T19:00:14.279766+0000","last_peered":"2024-09-20T19:00:14.279766+0000","last_clean":"2024-09-20T19:00:14.279766+0000","last_became_active":"2024-09-20T18:59:05.090056+0000","last_became_peered":"2024-09-20T18:59:05.090056+0000","last_unstale":"2024-09-20T19:00:14.279766+0000","last_undegraded":"2024-09-20T19:00:14.279766+0000","last_fullsized":"2024-09-20T19:00:14.279766+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:58:09.472360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:09.472360+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"273'15","reported_seq":522,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.270870+0000","last_change":"2024-09-20T18:58:29.671959+0000","last_active":"2024-09-20T19:00:14.270870+0000","last_peered":"2024-09-20T19:00:14.270870+0000","last_clean":"2024-09-20T19:00:14.270870+0000","last_became_active":"2024-09-20T18:49:47.947448+0000","last_became_peered":"2024-09-20T18:49:47.947448+0000","last_unstale":"2024-09-20T19:00:14.270870+0000","last_undegraded":"2024-09-20T19:00:14.270870+0000","last_fullsized":"2024-09-20T19:00:14.270870+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:58:29.671887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:29.671887+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":491,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.279354+0000","last_change":"2024-09-20T18:58:25.505818+0000","last_active":"2024-09-20T19:00:14.279354+0000","last_peered":"2024-09-20T19:00:14.279354+0000","last_clean":"2024-09-20T19:00:14.279354+0000","last_became_active":"2024-09-20T18:49:49.961188+0000","last_became_peered":"2024-09-20T18:49:49.961188+0000","last_unstale":"2024-09-20T19:00:14.279354+0000","last_undegraded":"2024-09-20T19:00:14.279354+0000","last_fullsized":"2024-09-20T19:00:14.279354+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:25.505757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:25.505757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":45,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":466,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.308908+0000","last_change":"2024-09-20T18:58:21.861893+0000","last_active":"2024-09-20T19:00:16.308908+0000","last_peered":"2024-09-20T19:00:16.308908+0000","last_clean":"2024-09-20T19:00:16.308908+0000","last_became_active":"2024-09-20T18:49:51.966715+0000","last_became_peered":"2024-09-20T18:49:51.966715+0000","last_unstale":"2024-09-20T19:00:16.308908+0000","last_undegraded":"2024-09-20T19:00:16.308908+0000","last_fullsized":"2024-09-20T19:00:16.308908+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:21.861836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:21.861836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":491,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.279553+0000","last_change":"2024-09-20T19:00:03.432784+0000","last_active":"2024-09-20T19:00:14.279553+0000","last_peered":"2024-09-20T19:00:14.279553+0000","last_clean":"2024-09-20T19:00:14.279553+0000","last_became_active":"2024-09-20T18:49:45.944703+0000","last_became_peered":"2024-09-20T18:49:45.944703+0000","last_unstale":"2024-09-20T19:00:14.279553+0000","last_undegraded":"2024-09-20T19:00:14.279553+0000","last_fullsized":"2024-09-20T19:00:14.279553+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:00:03.432711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:00:03.432711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:04.816004+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"273'7","reported_seq":520,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.271096+0000","last_change":"2024-09-20T18:58:33.660957+0000","last_active":"2024-09-20T19:00:14.271096+0000","last_peered":"2024-09-20T19:00:14.271096+0000","last_clean":"2024-09-20T19:00:14.271096+0000","last_became_active":"2024-09-20T18:51:23.813399+0000","last_became_peered":"2024-09-20T18:51:23.813399+0000","last_unstale":"2024-09-20T19:00:14.271096+0000","last_undegraded":"2024-09-20T19:00:14.271096+0000","last_fullsized":"2024-09-20T19:00:14.271096+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-20T18:58:33.660531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:33.660531+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":484,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.276338+0000","last_change":"2024-09-20T18:58:32.526595+0000","last_active":"2024-09-20T19:00:14.276338+0000","last_peered":"2024-09-20T19:00:14.276338+0000","last_clean":"2024-09-20T19:00:14.276338+0000","last_became_active":"2024-09-20T18:49:49.959696+0000","last_became_peered":"2024-09-20T18:49:49.959696+0000","last_unstale":"2024-09-20T19:00:14.276338+0000","last_undegraded":"2024-09-20T19:00:14.276338+0000","last_fullsized":"2024-09-20T19:00:14.276338+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.526527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.526527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":487,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.279247+0000","last_change":"2024-09-20T18:58:26.465305+0000","last_active":"2024-09-20T19:00:14.279247+0000","last_peered":"2024-09-20T19:00:14.279247+0000","last_clean":"2024-09-20T19:00:14.279247+0000","last_became_active":"2024-09-20T18:49:51.970663+0000","last_became_peered":"2024-09-20T18:49:51.970663+0000","last_unstale":"2024-09-20T19:00:14.279247+0000","last_undegraded":"2024-09-20T19:00:14.279247+0000","last_fullsized":"2024-09-20T19:00:14.279247+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:26.465264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:26.465264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":496,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.271524+0000","last_change":"2024-09-20T18:59:05.079781+0000","last_active":"2024-09-20T19:00:14.271524+0000","last_peered":"2024-09-20T19:00:14.271524+0000","last_clean":"2024-09-20T19:00:14.271524+0000","last_became_active":"2024-09-20T18:59:05.079278+0000","last_became_peered":"2024-09-20T18:59:05.079278+0000","last_unstale":"2024-09-20T19:00:14.271524+0000","last_undegraded":"2024-09-20T19:00:14.271524+0000","last_fullsized":"2024-09-20T19:00:14.271524+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:00.906652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:00.906652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:33.272916+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":494,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.309345+0000","last_change":"2024-09-20T18:58:34.996936+0000","last_active":"2024-09-20T19:00:16.309345+0000","last_peered":"2024-09-20T19:00:16.309345+0000","last_clean":"2024-09-20T19:00:16.309345+0000","last_became_active":"2024-09-20T18:49:45.950480+0000","last_became_peered":"2024-09-20T18:49:45.950480+0000","last_unstale":"2024-09-20T19:00:16.309345+0000","last_undegraded":"2024-09-20T19:00:16.309345+0000","last_fullsized":"2024-09-20T19:00:16.309345+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:34.996893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:34.996893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"273'27","reported_seq":535,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.297386+0000","last_change":"2024-09-20T18:58:28.253192+0000","last_active":"2024-09-20T19:00:16.297386+0000","last_peered":"2024-09-20T19:00:16.297386+0000","last_clean":"2024-09-20T19:00:16.297386+0000","last_became_active":"2024-09-20T18:49:47.950939+0000","last_became_peered":"2024-09-20T18:49:47.950939+0000","last_unstale":"2024-09-20T19:00:16.297386+0000","last_undegraded":"2024-09-20T19:00:16.297386+0000","last_fullsized":"2024-09-20T19:00:16.297386+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:58:28.253141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:28.253141+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":477,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.297974+0000","last_change":"2024-09-20T18:57:14.941518+0000","last_active":"2024-09-20T19:00:16.297974+0000","last_peered":"2024-09-20T19:00:16.297974+0000","last_clean":"2024-09-20T19:00:16.297974+0000","last_became_active":"2024-09-20T18:49:49.958518+0000","last_became_peered":"2024-09-20T18:49:49.958518+0000","last_unstale":"2024-09-20T19:00:16.297974+0000","last_undegraded":"2024-09-20T19:00:16.297974+0000","last_fullsized":"2024-09-20T19:00:16.297974+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:14.941458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:14.941458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":496,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.303996+0000","last_change":"2024-09-20T18:59:05.084868+0000","last_active":"2024-09-20T19:00:16.303996+0000","last_peered":"2024-09-20T19:00:16.303996+0000","last_clean":"2024-09-20T19:00:16.303996+0000","last_became_active":"2024-09-20T18:59:05.084496+0000","last_became_peered":"2024-09-20T18:59:05.084496+0000","last_unstale":"2024-09-20T19:00:16.303996+0000","last_undegraded":"2024-09-20T19:00:16.303996+0000","last_fullsized":"2024-09-20T19:00:16.303996+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:35.125451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:35.125451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"59'1","reported_seq":507,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.270946+0000","last_change":"2024-09-20T18:59:03.397298+0000","last_active":"2024-09-20T19:00:14.270946+0000","last_peered":"2024-09-20T19:00:14.270946+0000","last_clean":"2024-09-20T19:00:14.270946+0000","last_became_active":"2024-09-20T18:49:45.940100+0000","last_became_peered":"2024-09-20T18:49:45.940100+0000","last_unstale":"2024-09-20T19:00:14.270946+0000","last_undegraded":"2024-09-20T19:00:14.270946+0000","last_fullsized":"2024-09-20T19:00:14.270946+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:59:03.397225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:59:03.397225+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:17.466210+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"274'18","reported_seq":505,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.302391+0000","last_change":"2024-09-20T18:57:09.819080+0000","last_active":"2024-09-20T19:00:16.302391+0000","last_peered":"2024-09-20T19:00:16.302391+0000","last_clean":"2024-09-20T19:00:16.302391+0000","last_became_active":"2024-09-20T18:49:47.950909+0000","last_became_peered":"2024-09-20T18:49:47.950909+0000","last_unstale":"2024-09-20T19:00:16.302391+0000","last_undegraded":"2024-09-20T19:00:16.302391+0000","last_fullsized":"2024-09-20T19:00:16.302391+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:57:09.818999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:09.818999+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":485,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.272027+0000","last_change":"2024-09-20T18:54:44.239273+0000","last_active":"2024-09-20T19:00:14.272027+0000","last_peered":"2024-09-20T19:00:14.272027+0000","last_clean":"2024-09-20T19:00:14.272027+0000","last_became_active":"2024-09-20T18:49:49.959102+0000","last_became_peered":"2024-09-20T18:49:49.959102+0000","last_unstale":"2024-09-20T19:00:14.272027+0000","last_undegraded":"2024-09-20T19:00:14.272027+0000","last_fullsized":"2024-09-20T19:00:14.272027+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:44.239226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.239226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:18.193129+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":481,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.276280+0000","last_change":"2024-09-20T18:58:25.501434+0000","last_active":"2024-09-20T19:00:14.276280+0000","last_peered":"2024-09-20T19:00:14.276280+0000","last_clean":"2024-09-20T19:00:14.276280+0000","last_became_active":"2024-09-20T18:49:51.970058+0000","last_became_peered":"2024-09-20T18:49:51.970058+0000","last_unstale":"2024-09-20T19:00:14.276280+0000","last_undegraded":"2024-09-20T19:00:14.276280+0000","last_fullsized":"2024-09-20T19:00:14.276280+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T18:58:25.501384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:25.501384+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":486,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.297119+0000","last_change":"2024-09-20T18:57:18.032140+0000","last_active":"2024-09-20T19:00:16.297119+0000","last_peered":"2024-09-20T19:00:16.297119+0000","last_clean":"2024-09-20T19:00:16.297119+0000","last_became_active":"2024-09-20T18:49:45.933078+0000","last_became_peered":"2024-09-20T18:49:45.933078+0000","last_unstale":"2024-09-20T19:00:16.297119+0000","last_undegraded":"2024-09-20T19:00:16.297119+0000","last_fullsized":"2024-09-20T19:00:16.297119+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:57:18.032067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:18.032067+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"274'20","reported_seq":529,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.307989+0000","last_change":"2024-09-20T18:59:02.024076+0000","last_active":"2024-09-20T19:00:16.307989+0000","last_peered":"2024-09-20T19:00:16.307989+0000","last_clean":"2024-09-20T19:00:16.307989+0000","last_became_active":"2024-09-20T18:49:48.500809+0000","last_became_peered":"2024-09-20T18:49:48.500809+0000","last_unstale":"2024-09-20T19:00:16.307989+0000","last_undegraded":"2024-09-20T19:00:16.307989+0000","last_fullsized":"2024-09-20T19:00:16.307989+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:59:02.023996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:59:02.023996+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"262'10","reported_seq":80,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.282245+0000","last_change":"2024-09-20T18:59:05.105124+0000","last_active":"2024-09-20T19:00:14.282245+0000","last_peered":"2024-09-20T19:00:14.282245+0000","last_clean":"2024-09-20T19:00:14.282245+0000","last_became_active":"2024-09-20T18:59:05.089298+0000","last_became_peered":"2024-09-20T18:59:05.089298+0000","last_unstale":"2024-09-20T19:00:14.282245+0000","last_undegraded":"2024-09-20T19:00:14.282245+0000","last_fullsized":"2024-09-20T19:00:14.282245+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'9","last_scrub_stamp":"2024-09-20T18:57:48.853302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:48.853302+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2","version":"287'7398","reported_seq":7491,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.826511+0000","last_change":"2024-09-20T18:58:33.827583+0000","last_active":"2024-09-20T19:00:16.826511+0000","last_peered":"2024-09-20T19:00:16.826511+0000","last_clean":"2024-09-20T19:00:16.826511+0000","last_became_active":"2024-09-20T18:58:33.827098+0000","last_became_peered":"2024-09-20T18:58:33.827098+0000","last_unstale":"2024-09-20T19:00:16.826511+0000","last_undegraded":"2024-09-20T19:00:16.826511+0000","last_fullsized":"2024-09-20T19:00:16.826511+0000","mapping_epoch":246,"log_start":"258'2600","ondisk_log_start":"258'2600","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":4798,"log_dups_size":0,"ondisk_log_size":4798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":437780480,"num_objects":6680,"num_object_clones":0,"num_object_copies":20040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6680,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7398,"num_write_kb":450496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"274'27","reported_seq":518,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.297704+0000","last_change":"2024-09-20T18:55:18.581290+0000","last_active":"2024-09-20T19:00:16.297704+0000","last_peered":"2024-09-20T19:00:16.297704+0000","last_clean":"2024-09-20T19:00:16.297704+0000","last_became_active":"2024-09-20T18:49:58.992302+0000","last_became_peered":"2024-09-20T18:49:58.992302+0000","last_unstale":"2024-09-20T19:00:16.297704+0000","last_undegraded":"2024-09-20T19:00:16.297704+0000","last_fullsized":"2024-09-20T19:00:16.297704+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:55:18.581200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:18.581200+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":485,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.305274+0000","last_change":"2024-09-20T18:54:54.288199+0000","last_active":"2024-09-20T19:00:16.305274+0000","last_peered":"2024-09-20T19:00:16.305274+0000","last_clean":"2024-09-20T19:00:16.305274+0000","last_became_active":"2024-09-20T18:49:45.936104+0000","last_became_peered":"2024-09-20T18:49:45.936104+0000","last_unstale":"2024-09-20T19:00:16.305274+0000","last_undegraded":"2024-09-20T19:00:16.305274+0000","last_fullsized":"2024-09-20T19:00:16.305274+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:54.288117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:54.288117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":46,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":614,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.271616+0000","last_change":"2024-09-20T18:54:14.292660+0000","last_active":"2024-09-20T19:00:14.271616+0000","last_peered":"2024-09-20T19:00:14.271616+0000","last_clean":"2024-09-20T19:00:14.271616+0000","last_became_active":"2024-09-20T18:49:49.960697+0000","last_became_peered":"2024-09-20T18:49:49.960697+0000","last_unstale":"2024-09-20T19:00:14.271616+0000","last_undegraded":"2024-09-20T19:00:14.271616+0000","last_fullsized":"2024-09-20T19:00:14.271616+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:54:14.292610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:14.292610+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":34,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":479,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.279520+0000","last_change":"2024-09-20T18:58:31.493658+0000","last_active":"2024-09-20T19:00:14.279520+0000","last_peered":"2024-09-20T19:00:14.279520+0000","last_clean":"2024-09-20T19:00:14.279520+0000","last_became_active":"2024-09-20T18:49:51.971399+0000","last_became_peered":"2024-09-20T18:49:51.971399+0000","last_unstale":"2024-09-20T19:00:14.279520+0000","last_undegraded":"2024-09-20T19:00:14.279520+0000","last_fullsized":"2024-09-20T19:00:14.279520+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:31.493595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:31.493595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"286'7470","reported_seq":7562,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:15.931228+0000","last_change":"2024-09-20T18:58:33.820158+0000","last_active":"2024-09-20T19:00:15.931228+0000","last_peered":"2024-09-20T19:00:15.931228+0000","last_clean":"2024-09-20T19:00:15.931228+0000","last_became_active":"2024-09-20T18:58:33.819886+0000","last_became_peered":"2024-09-20T18:58:33.819886+0000","last_unstale":"2024-09-20T19:00:15.931228+0000","last_undegraded":"2024-09-20T19:00:15.931228+0000","last_fullsized":"2024-09-20T19:00:15.931228+0000","mapping_epoch":246,"log_start":"258'2656","ondisk_log_start":"258'2656","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":4814,"log_dups_size":0,"ondisk_log_size":4814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":447938584,"num_objects":6836,"num_object_clones":0,"num_object_copies":20508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6836,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7470,"num_write_kb":457729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"273'30","reported_seq":540,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.302669+0000","last_change":"2024-09-20T18:58:08.025426+0000","last_active":"2024-09-20T19:00:16.302669+0000","last_peered":"2024-09-20T19:00:16.302669+0000","last_clean":"2024-09-20T19:00:16.302669+0000","last_became_active":"2024-09-20T18:49:48.499189+0000","last_became_peered":"2024-09-20T18:49:48.499189+0000","last_unstale":"2024-09-20T19:00:16.302669+0000","last_undegraded":"2024-09-20T19:00:16.302669+0000","last_fullsized":"2024-09-20T19:00:16.302669+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-20T18:58:08.025202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:08.025202+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":485,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.304274+0000","last_change":"2024-09-20T18:58:25.159724+0000","last_active":"2024-09-20T19:00:16.304274+0000","last_peered":"2024-09-20T19:00:16.304274+0000","last_clean":"2024-09-20T19:00:16.304274+0000","last_became_active":"2024-09-20T18:49:45.940544+0000","last_became_peered":"2024-09-20T18:49:45.940544+0000","last_unstale":"2024-09-20T19:00:16.304274+0000","last_undegraded":"2024-09-20T19:00:16.304274+0000","last_fullsized":"2024-09-20T19:00:16.304274+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:25.159673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:25.159673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":486,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.272074+0000","last_change":"2024-09-20T18:54:51.229251+0000","last_active":"2024-09-20T19:00:14.272074+0000","last_peered":"2024-09-20T19:00:14.272074+0000","last_clean":"2024-09-20T19:00:14.272074+0000","last_became_active":"2024-09-20T18:49:49.963073+0000","last_became_peered":"2024-09-20T18:49:49.963073+0000","last_unstale":"2024-09-20T19:00:14.272074+0000","last_undegraded":"2024-09-20T19:00:14.272074+0000","last_fullsized":"2024-09-20T19:00:14.272074+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.229201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.229201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:21.230603+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":489,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.270797+0000","last_change":"2024-09-20T18:58:27.704583+0000","last_active":"2024-09-20T19:00:14.270797+0000","last_peered":"2024-09-20T19:00:14.270797+0000","last_clean":"2024-09-20T19:00:14.270797+0000","last_became_active":"2024-09-20T18:49:51.968793+0000","last_became_peered":"2024-09-20T18:49:51.968793+0000","last_unstale":"2024-09-20T19:00:14.270797+0000","last_undegraded":"2024-09-20T19:00:14.270797+0000","last_fullsized":"2024-09-20T19:00:14.270797+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:27.704520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:27.704520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:12.539542+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.c","version":"287'7439","reported_seq":7532,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.846668+0000","last_change":"2024-09-20T18:58:33.845752+0000","last_active":"2024-09-20T19:00:16.846668+0000","last_peered":"2024-09-20T19:00:16.846668+0000","last_clean":"2024-09-20T19:00:16.846668+0000","last_became_active":"2024-09-20T18:58:33.845463+0000","last_became_peered":"2024-09-20T18:58:33.845463+0000","last_unstale":"2024-09-20T19:00:16.846668+0000","last_undegraded":"2024-09-20T19:00:16.846668+0000","last_fullsized":"2024-09-20T19:00:16.846668+0000","mapping_epoch":246,"log_start":"258'2600","ondisk_log_start":"258'2600","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":4839,"log_dups_size":0,"ondisk_log_size":4839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":439418880,"num_objects":6705,"num_object_clones":0,"num_object_copies":20115,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7439,"num_write_kb":452608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"274'28","reported_seq":586,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.272160+0000","last_change":"2024-09-20T18:59:05.105526+0000","last_active":"2024-09-20T19:00:14.272160+0000","last_peered":"2024-09-20T19:00:14.272160+0000","last_clean":"2024-09-20T19:00:14.272160+0000","last_became_active":"2024-09-20T18:59:05.074532+0000","last_became_peered":"2024-09-20T18:59:05.074532+0000","last_unstale":"2024-09-20T19:00:14.272160+0000","last_undegraded":"2024-09-20T19:00:14.272160+0000","last_fullsized":"2024-09-20T19:00:14.272160+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-20T18:54:57.394893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.394893+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:23.216727+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":487,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.308212+0000","last_change":"2024-09-20T18:58:29.939909+0000","last_active":"2024-09-20T19:00:16.308212+0000","last_peered":"2024-09-20T19:00:16.308212+0000","last_clean":"2024-09-20T19:00:16.308212+0000","last_became_active":"2024-09-20T18:49:45.940469+0000","last_became_peered":"2024-09-20T18:49:45.940469+0000","last_unstale":"2024-09-20T19:00:16.308212+0000","last_undegraded":"2024-09-20T19:00:16.308212+0000","last_fullsized":"2024-09-20T19:00:16.308212+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:58:29.939813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:29.939813+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"66'8","reported_seq":629,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.302114+0000","last_change":"2024-09-20T18:58:23.114419+0000","last_active":"2024-09-20T19:00:16.302114+0000","last_peered":"2024-09-20T19:00:16.302114+0000","last_clean":"2024-09-20T19:00:16.302114+0000","last_became_active":"2024-09-20T18:49:49.956133+0000","last_became_peered":"2024-09-20T18:49:49.956133+0000","last_unstale":"2024-09-20T19:00:16.302114+0000","last_undegraded":"2024-09-20T19:00:16.302114+0000","last_fullsized":"2024-09-20T19:00:16.302114+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:58:23.114371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:23.114371+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":494,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.276223+0000","last_change":"2024-09-20T18:59:05.082117+0000","last_active":"2024-09-20T19:00:14.276223+0000","last_peered":"2024-09-20T19:00:14.276223+0000","last_clean":"2024-09-20T19:00:14.276223+0000","last_became_active":"2024-09-20T18:59:05.081814+0000","last_became_peered":"2024-09-20T18:59:05.081814+0000","last_unstale":"2024-09-20T19:00:14.276223+0000","last_undegraded":"2024-09-20T19:00:14.276223+0000","last_fullsized":"2024-09-20T19:00:14.276223+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:29.530001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:29.530001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"286'7356","reported_seq":7447,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.041602+0000","last_change":"2024-09-20T18:58:33.826834+0000","last_active":"2024-09-20T19:00:16.041602+0000","last_peered":"2024-09-20T19:00:16.041602+0000","last_clean":"2024-09-20T19:00:16.041602+0000","last_became_active":"2024-09-20T18:58:33.826568+0000","last_became_peered":"2024-09-20T18:58:33.826568+0000","last_unstale":"2024-09-20T19:00:16.041602+0000","last_undegraded":"2024-09-20T19:00:16.041602+0000","last_fullsized":"2024-09-20T19:00:16.041602+0000","mapping_epoch":246,"log_start":"255'1900","ondisk_log_start":"255'1900","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":5456,"log_dups_size":0,"ondisk_log_size":5456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":444071936,"num_objects":6776,"num_object_clones":0,"num_object_copies":20328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6776,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7356,"num_write_kb":452224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"274'20","reported_seq":536,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.299580+0000","last_change":"2024-09-20T18:59:06.064691+0000","last_active":"2024-09-20T19:00:16.299580+0000","last_peered":"2024-09-20T19:00:16.299580+0000","last_clean":"2024-09-20T19:00:16.299580+0000","last_became_active":"2024-09-20T18:49:47.949626+0000","last_became_peered":"2024-09-20T18:49:47.949626+0000","last_unstale":"2024-09-20T19:00:16.299580+0000","last_undegraded":"2024-09-20T19:00:16.299580+0000","last_fullsized":"2024-09-20T19:00:16.299580+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:59:06.064599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:59:06.064599+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:33.522592+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":500,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.279853+0000","last_change":"2024-09-20T18:58:32.528966+0000","last_active":"2024-09-20T19:00:14.279853+0000","last_peered":"2024-09-20T19:00:14.279853+0000","last_clean":"2024-09-20T19:00:14.279853+0000","last_became_active":"2024-09-20T18:49:45.942496+0000","last_became_peered":"2024-09-20T18:49:45.942496+0000","last_unstale":"2024-09-20T19:00:14.279853+0000","last_undegraded":"2024-09-20T19:00:14.279853+0000","last_fullsized":"2024-09-20T19:00:14.279853+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.528310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.528310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"262'10","reported_seq":644,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.308786+0000","last_change":"2024-09-20T18:59:05.098010+0000","last_active":"2024-09-20T19:00:16.308786+0000","last_peered":"2024-09-20T19:00:16.308786+0000","last_clean":"2024-09-20T19:00:16.308786+0000","last_became_active":"2024-09-20T18:59:05.080183+0000","last_became_peered":"2024-09-20T18:59:05.080183+0000","last_unstale":"2024-09-20T19:00:16.308786+0000","last_undegraded":"2024-09-20T19:00:16.308786+0000","last_fullsized":"2024-09-20T19:00:16.308786+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'9","last_scrub_stamp":"2024-09-20T18:59:00.055110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:59:00.055110+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":467,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.309053+0000","last_change":"2024-09-20T18:58:26.948165+0000","last_active":"2024-09-20T19:00:16.309053+0000","last_peered":"2024-09-20T19:00:16.309053+0000","last_clean":"2024-09-20T19:00:16.309053+0000","last_became_active":"2024-09-20T18:49:51.964780+0000","last_became_peered":"2024-09-20T18:49:51.964780+0000","last_unstale":"2024-09-20T19:00:16.309053+0000","last_undegraded":"2024-09-20T19:00:16.309053+0000","last_fullsized":"2024-09-20T19:00:16.309053+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:26.948102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:26.948102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"286'7492","reported_seq":7583,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:15.940044+0000","last_change":"2024-09-20T18:58:33.845804+0000","last_active":"2024-09-20T19:00:15.940044+0000","last_peered":"2024-09-20T19:00:15.940044+0000","last_clean":"2024-09-20T19:00:15.940044+0000","last_became_active":"2024-09-20T18:58:33.845493+0000","last_became_peered":"2024-09-20T18:58:33.845493+0000","last_unstale":"2024-09-20T19:00:15.940044+0000","last_undegraded":"2024-09-20T19:00:15.940044+0000","last_fullsized":"2024-09-20T19:00:15.940044+0000","mapping_epoch":246,"log_start":"258'2707","ondisk_log_start":"258'2707","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":4785,"log_dups_size":0,"ondisk_log_size":4785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":452591616,"num_objects":6906,"num_object_clones":0,"num_object_copies":20718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7492,"num_write_kb":460736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"274'29","reported_seq":542,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.309140+0000","last_change":"2024-09-20T19:00:04.810907+0000","last_active":"2024-09-20T19:00:16.309140+0000","last_peered":"2024-09-20T19:00:16.309140+0000","last_clean":"2024-09-20T19:00:16.309140+0000","last_became_active":"2024-09-20T18:49:48.497736+0000","last_became_peered":"2024-09-20T18:49:48.497736+0000","last_unstale":"2024-09-20T19:00:16.309140+0000","last_undegraded":"2024-09-20T19:00:16.309140+0000","last_fullsized":"2024-09-20T19:00:16.309140+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'29","last_scrub_stamp":"2024-09-20T19:00:04.810831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:00:04.810831+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:28.033769+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":478,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.300239+0000","last_change":"2024-09-20T18:57:09.940634+0000","last_active":"2024-09-20T19:00:16.300239+0000","last_peered":"2024-09-20T19:00:16.300239+0000","last_clean":"2024-09-20T19:00:16.300239+0000","last_became_active":"2024-09-20T18:49:45.942253+0000","last_became_peered":"2024-09-20T18:49:45.942253+0000","last_unstale":"2024-09-20T19:00:16.300239+0000","last_undegraded":"2024-09-20T19:00:16.300239+0000","last_fullsized":"2024-09-20T19:00:16.300239+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:09.940517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:09.940517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:12.687941+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":62,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.281329+0000","last_change":"2024-09-20T18:59:05.085948+0000","last_active":"2024-09-20T19:00:14.281329+0000","last_peered":"2024-09-20T19:00:14.281329+0000","last_clean":"2024-09-20T19:00:14.281329+0000","last_became_active":"2024-09-20T18:59:05.085636+0000","last_became_peered":"2024-09-20T18:59:05.085636+0000","last_unstale":"2024-09-20T19:00:14.281329+0000","last_undegraded":"2024-09-20T19:00:14.281329+0000","last_fullsized":"2024-09-20T19:00:14.281329+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:50.610871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:50.610871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:13.135963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":504,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.296926+0000","last_change":"2024-09-20T18:59:05.085472+0000","last_active":"2024-09-20T19:00:16.296926+0000","last_peered":"2024-09-20T19:00:16.296926+0000","last_clean":"2024-09-20T19:00:16.296926+0000","last_became_active":"2024-09-20T18:59:05.085131+0000","last_became_peered":"2024-09-20T18:59:05.085131+0000","last_unstale":"2024-09-20T19:00:16.296926+0000","last_undegraded":"2024-09-20T19:00:16.296926+0000","last_fullsized":"2024-09-20T19:00:16.296926+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:58.973176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:58.973176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"287'7561","reported_seq":7654,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:17.194855+0000","last_change":"2024-09-20T18:58:33.822011+0000","last_active":"2024-09-20T19:00:17.194855+0000","last_peered":"2024-09-20T19:00:17.194855+0000","last_clean":"2024-09-20T19:00:17.194855+0000","last_became_active":"2024-09-20T18:58:33.820506+0000","last_became_peered":"2024-09-20T18:58:33.820506+0000","last_unstale":"2024-09-20T19:00:17.194855+0000","last_undegraded":"2024-09-20T19:00:17.194855+0000","last_fullsized":"2024-09-20T19:00:17.194855+0000","mapping_epoch":246,"log_start":"254'1500","ondisk_log_start":"254'1500","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":6061,"log_dups_size":0,"ondisk_log_size":6061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":448856064,"num_objects":6849,"num_object_clones":0,"num_object_copies":20547,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6849,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7561,"num_write_kb":461120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"274'40","reported_seq":683,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.300105+0000","last_change":"2024-09-20T18:58:07.018449+0000","last_active":"2024-09-20T19:00:16.300105+0000","last_peered":"2024-09-20T19:00:16.300105+0000","last_clean":"2024-09-20T19:00:16.300105+0000","last_became_active":"2024-09-20T18:49:47.955335+0000","last_became_peered":"2024-09-20T18:49:47.955335+0000","last_unstale":"2024-09-20T19:00:16.300105+0000","last_undegraded":"2024-09-20T19:00:16.300105+0000","last_fullsized":"2024-09-20T19:00:16.300105+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'28","last_scrub_stamp":"2024-09-20T18:58:07.018340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:07.018340+0000","objects_scrubbed":10,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":492,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.271404+0000","last_change":"2024-09-20T18:59:05.082509+0000","last_active":"2024-09-20T19:00:14.271404+0000","last_peered":"2024-09-20T19:00:14.271404+0000","last_clean":"2024-09-20T19:00:14.271404+0000","last_became_active":"2024-09-20T18:59:05.081468+0000","last_became_peered":"2024-09-20T18:59:05.081468+0000","last_unstale":"2024-09-20T19:00:14.271404+0000","last_undegraded":"2024-09-20T19:00:14.271404+0000","last_fullsized":"2024-09-20T19:00:14.271404+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:21.807703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:21.807703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:26.213224+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":540,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.279484+0000","last_change":"2024-09-20T19:00:02.437598+0000","last_active":"2024-09-20T19:00:14.279484+0000","last_peered":"2024-09-20T19:00:14.279484+0000","last_clean":"2024-09-20T19:00:14.279484+0000","last_became_active":"2024-09-20T18:59:05.086786+0000","last_became_peered":"2024-09-20T18:59:05.086786+0000","last_unstale":"2024-09-20T19:00:14.279484+0000","last_undegraded":"2024-09-20T19:00:14.279484+0000","last_fullsized":"2024-09-20T19:00:14.279484+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-20T19:00:02.437550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:00:02.437550+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:09.312576+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0012781159999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":495,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.308534+0000","last_change":"2024-09-20T18:58:59.099799+0000","last_active":"2024-09-20T19:00:16.308534+0000","last_peered":"2024-09-20T19:00:16.308534+0000","last_clean":"2024-09-20T19:00:16.308534+0000","last_became_active":"2024-09-20T18:49:49.953971+0000","last_became_peered":"2024-09-20T18:49:49.953971+0000","last_unstale":"2024-09-20T19:00:16.308534+0000","last_undegraded":"2024-09-20T19:00:16.308534+0000","last_fullsized":"2024-09-20T19:00:16.308534+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:59.099736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:59.099736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:28.205403+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":465,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.302387+0000","last_change":"2024-09-20T18:57:12.744116+0000","last_active":"2024-09-20T19:00:16.302387+0000","last_peered":"2024-09-20T19:00:16.302387+0000","last_clean":"2024-09-20T19:00:16.302387+0000","last_became_active":"2024-09-20T18:49:51.969400+0000","last_became_peered":"2024-09-20T18:49:51.969400+0000","last_unstale":"2024-09-20T19:00:16.302387+0000","last_undegraded":"2024-09-20T19:00:16.302387+0000","last_fullsized":"2024-09-20T19:00:16.302387+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:12.744040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:12.744040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"286'7503","reported_seq":7594,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.049317+0000","last_change":"2024-09-20T18:58:33.829042+0000","last_active":"2024-09-20T19:00:16.049317+0000","last_peered":"2024-09-20T19:00:16.049317+0000","last_clean":"2024-09-20T19:00:16.049317+0000","last_became_active":"2024-09-20T18:58:33.828787+0000","last_became_peered":"2024-09-20T18:58:33.828787+0000","last_unstale":"2024-09-20T19:00:16.049317+0000","last_undegraded":"2024-09-20T19:00:16.049317+0000","last_fullsized":"2024-09-20T19:00:16.049317+0000","mapping_epoch":246,"log_start":"256'2100","ondisk_log_start":"256'2100","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":5403,"log_dups_size":0,"ondisk_log_size":5403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":453705728,"num_objects":6923,"num_object_clones":0,"num_object_copies":20769,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7503,"num_write_kb":461632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"273'17","reported_seq":530,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.300533+0000","last_change":"2024-09-20T18:58:06.079117+0000","last_active":"2024-09-20T19:00:16.300533+0000","last_peered":"2024-09-20T19:00:16.300533+0000","last_clean":"2024-09-20T19:00:16.300533+0000","last_became_active":"2024-09-20T18:49:47.950904+0000","last_became_peered":"2024-09-20T18:49:47.950904+0000","last_unstale":"2024-09-20T19:00:16.300533+0000","last_undegraded":"2024-09-20T19:00:16.300533+0000","last_fullsized":"2024-09-20T19:00:16.300533+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-20T18:58:06.078352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:06.078352+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":59,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":36,"num_read_kb":25,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":516,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.298683+0000","last_change":"2024-09-20T18:59:05.080117+0000","last_active":"2024-09-20T19:00:16.298683+0000","last_peered":"2024-09-20T19:00:16.298683+0000","last_clean":"2024-09-20T19:00:16.298683+0000","last_became_active":"2024-09-20T18:59:05.079789+0000","last_became_peered":"2024-09-20T18:59:05.079789+0000","last_unstale":"2024-09-20T19:00:16.298683+0000","last_undegraded":"2024-09-20T19:00:16.298683+0000","last_fullsized":"2024-09-20T19:00:16.298683+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:28.987458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:28.987458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":498,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.296405+0000","last_change":"2024-09-20T18:58:24.229997+0000","last_active":"2024-09-20T19:00:16.296405+0000","last_peered":"2024-09-20T19:00:16.296405+0000","last_clean":"2024-09-20T19:00:16.296405+0000","last_became_active":"2024-09-20T18:49:44.937135+0000","last_became_peered":"2024-09-20T18:49:44.937135+0000","last_unstale":"2024-09-20T19:00:16.296405+0000","last_undegraded":"2024-09-20T19:00:16.296405+0000","last_fullsized":"2024-09-20T19:00:16.296405+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:58:24.229938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:58:24.229938+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00095558800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":472,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.297464+0000","last_change":"2024-09-20T18:51:59.796450+0000","last_active":"2024-09-20T19:00:16.297464+0000","last_peered":"2024-09-20T19:00:16.297464+0000","last_clean":"2024-09-20T19:00:16.297464+0000","last_became_active":"2024-09-20T18:49:49.954424+0000","last_became_peered":"2024-09-20T18:49:49.954424+0000","last_unstale":"2024-09-20T19:00:16.297464+0000","last_undegraded":"2024-09-20T19:00:16.297464+0000","last_fullsized":"2024-09-20T19:00:16.297464+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:51:59.796383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:51:59.796383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:15.500000+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":481,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.276184+0000","last_change":"2024-09-20T18:59:05.082433+0000","last_active":"2024-09-20T19:00:14.276184+0000","last_peered":"2024-09-20T19:00:14.276184+0000","last_clean":"2024-09-20T19:00:14.276184+0000","last_became_active":"2024-09-20T18:59:05.081459+0000","last_became_peered":"2024-09-20T18:59:05.081459+0000","last_unstale":"2024-09-20T19:00:14.276184+0000","last_undegraded":"2024-09-20T19:00:14.276184+0000","last_fullsized":"2024-09-20T19:00:14.276184+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:52:39.814259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:52:39.814259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:11.944156+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"287'7314","reported_seq":13522,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:17.221505+0000","last_change":"2024-09-20T19:00:03.218867+0000","last_active":"2024-09-20T19:00:17.221505+0000","last_peered":"2024-09-20T19:00:17.221505+0000","last_clean":"2024-09-20T19:00:17.221505+0000","last_became_active":"2024-09-20T19:00:03.209632+0000","last_became_peered":"2024-09-20T19:00:03.209632+0000","last_unstale":"2024-09-20T19:00:17.221505+0000","last_undegraded":"2024-09-20T19:00:17.221505+0000","last_fullsized":"2024-09-20T19:00:17.221505+0000","mapping_epoch":279,"log_start":"253'1300","ondisk_log_start":"253'1300","created":246,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":6014,"log_dups_size":0,"ondisk_log_size":6014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":433192960,"num_objects":6610,"num_object_clones":0,"num_object_copies":19830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6610,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7314,"num_write_kb":445568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3051,"num_bytes_recovered":199950336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"273'24","reported_seq":542,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.308064+0000","last_change":"2024-09-20T18:59:06.022021+0000","last_active":"2024-09-20T19:00:16.308064+0000","last_peered":"2024-09-20T19:00:16.308064+0000","last_clean":"2024-09-20T19:00:16.308064+0000","last_became_active":"2024-09-20T18:49:47.949230+0000","last_became_peered":"2024-09-20T18:49:47.949230+0000","last_unstale":"2024-09-20T19:00:16.308064+0000","last_undegraded":"2024-09-20T19:00:16.308064+0000","last_fullsized":"2024-09-20T19:00:16.308064+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:59:06.021962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:59:06.021962+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":495,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.310211+0000","last_change":"2024-09-20T18:54:27.905856+0000","last_active":"2024-09-20T19:00:16.310211+0000","last_peered":"2024-09-20T19:00:16.310211+0000","last_clean":"2024-09-20T19:00:16.310211+0000","last_became_active":"2024-09-20T18:49:45.939197+0000","last_became_peered":"2024-09-20T18:49:45.939197+0000","last_unstale":"2024-09-20T19:00:16.310211+0000","last_undegraded":"2024-09-20T19:00:16.310211+0000","last_fullsized":"2024-09-20T19:00:16.310211+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:27.905803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:27.905803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:09.859516+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":1128,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.275942+0000","last_change":"2024-09-20T18:54:26.359887+0000","last_active":"2024-09-20T19:00:14.275942+0000","last_peered":"2024-09-20T19:00:14.275942+0000","last_clean":"2024-09-20T19:00:14.275942+0000","last_became_active":"2024-09-20T18:49:44.936819+0000","last_became_peered":"2024-09-20T18:49:44.936819+0000","last_unstale":"2024-09-20T19:00:14.275942+0000","last_undegraded":"2024-09-20T19:00:14.275942+0000","last_fullsized":"2024-09-20T19:00:14.275942+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-20T18:54:26.359807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T18:54:26.359807+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0010996199999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":624,"num_read_kb":619,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":483,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.280191+0000","last_change":"2024-09-20T18:57:54.362085+0000","last_active":"2024-09-20T19:00:14.280191+0000","last_peered":"2024-09-20T19:00:14.280191+0000","last_clean":"2024-09-20T19:00:14.280191+0000","last_became_active":"2024-09-20T18:49:49.961066+0000","last_became_peered":"2024-09-20T18:49:49.961066+0000","last_unstale":"2024-09-20T19:00:14.280191+0000","last_undegraded":"2024-09-20T19:00:14.280191+0000","last_fullsized":"2024-09-20T19:00:14.280191+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:54.362038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:54.362038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:11.325643+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":489,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.299696+0000","last_change":"2024-09-20T18:58:08.019154+0000","last_active":"2024-09-20T19:00:16.299696+0000","last_peered":"2024-09-20T19:00:16.299696+0000","last_clean":"2024-09-20T19:00:16.299696+0000","last_became_active":"2024-09-20T18:49:51.970343+0000","last_became_peered":"2024-09-20T18:49:51.970343+0000","last_unstale":"2024-09-20T19:00:16.299696+0000","last_undegraded":"2024-09-20T19:00:16.299696+0000","last_fullsized":"2024-09-20T19:00:16.299696+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:08.019092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:08.019092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"287'7289","reported_seq":7382,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.847637+0000","last_change":"2024-09-20T18:58:33.820307+0000","last_active":"2024-09-20T19:00:16.847637+0000","last_peered":"2024-09-20T19:00:16.847637+0000","last_clean":"2024-09-20T19:00:16.847637+0000","last_became_active":"2024-09-20T18:58:33.820027+0000","last_became_peered":"2024-09-20T18:58:33.820027+0000","last_unstale":"2024-09-20T19:00:16.847637+0000","last_undegraded":"2024-09-20T19:00:16.847637+0000","last_fullsized":"2024-09-20T19:00:16.847637+0000","mapping_epoch":246,"log_start":"258'2500","ondisk_log_start":"258'2500","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":4789,"log_dups_size":0,"ondisk_log_size":4789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":432865280,"num_objects":6605,"num_object_clones":0,"num_object_copies":19815,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6605,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7289,"num_write_kb":444608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"274'30","reported_seq":112,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.281794+0000","last_change":"2024-09-20T18:59:05.102446+0000","last_active":"2024-09-20T19:00:14.281794+0000","last_peered":"2024-09-20T19:00:14.281794+0000","last_clean":"2024-09-20T19:00:14.281794+0000","last_became_active":"2024-09-20T18:59:05.088069+0000","last_became_peered":"2024-09-20T18:59:05.088069+0000","last_unstale":"2024-09-20T19:00:14.281794+0000","last_undegraded":"2024-09-20T19:00:14.281794+0000","last_fullsized":"2024-09-20T19:00:14.281794+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'17","last_scrub_stamp":"2024-09-20T18:57:22.816154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:22.816154+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":508,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.302325+0000","last_change":"2024-09-20T18:59:05.079328+0000","last_active":"2024-09-20T19:00:16.302325+0000","last_peered":"2024-09-20T19:00:16.302325+0000","last_clean":"2024-09-20T19:00:16.302325+0000","last_became_active":"2024-09-20T18:59:05.079111+0000","last_became_peered":"2024-09-20T18:59:05.079111+0000","last_unstale":"2024-09-20T19:00:16.302325+0000","last_undegraded":"2024-09-20T19:00:16.302325+0000","last_fullsized":"2024-09-20T19:00:16.302325+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.100625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.100625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":560,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.275991+0000","last_change":"2024-09-20T18:59:05.108551+0000","last_active":"2024-09-20T19:00:14.275991+0000","last_peered":"2024-09-20T19:00:14.275991+0000","last_clean":"2024-09-20T19:00:14.275991+0000","last_became_active":"2024-09-20T18:59:05.086103+0000","last_became_peered":"2024-09-20T18:59:05.086103+0000","last_unstale":"2024-09-20T19:00:14.275991+0000","last_undegraded":"2024-09-20T19:00:14.275991+0000","last_fullsized":"2024-09-20T19:00:14.275991+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T18:55:13.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T18:55:13.696607+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":477,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.276807+0000","last_change":"2024-09-20T18:58:23.545370+0000","last_active":"2024-09-20T19:00:14.276807+0000","last_peered":"2024-09-20T19:00:14.276807+0000","last_clean":"2024-09-20T19:00:14.276807+0000","last_became_active":"2024-09-20T18:49:49.956128+0000","last_became_peered":"2024-09-20T18:49:49.956128+0000","last_unstale":"2024-09-20T19:00:14.276807+0000","last_undegraded":"2024-09-20T19:00:14.276807+0000","last_fullsized":"2024-09-20T19:00:14.276807+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:23.545321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:23.545321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":487,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.279968+0000","last_change":"2024-09-20T18:58:03.523127+0000","last_active":"2024-09-20T19:00:14.279968+0000","last_peered":"2024-09-20T19:00:14.279968+0000","last_clean":"2024-09-20T19:00:14.279968+0000","last_became_active":"2024-09-20T18:49:51.971332+0000","last_became_peered":"2024-09-20T18:49:51.971332+0000","last_unstale":"2024-09-20T19:00:14.279968+0000","last_undegraded":"2024-09-20T19:00:14.279968+0000","last_fullsized":"2024-09-20T19:00:14.279968+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:03.523078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:03.523078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"286'7482","reported_seq":7573,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:15.876373+0000","last_change":"2024-09-20T18:58:33.825100+0000","last_active":"2024-09-20T19:00:15.876373+0000","last_peered":"2024-09-20T19:00:15.876373+0000","last_clean":"2024-09-20T19:00:15.876373+0000","last_became_active":"2024-09-20T18:58:33.824751+0000","last_became_peered":"2024-09-20T18:58:33.824751+0000","last_unstale":"2024-09-20T19:00:15.876373+0000","last_undegraded":"2024-09-20T19:00:15.876373+0000","last_fullsized":"2024-09-20T19:00:15.876373+0000","mapping_epoch":246,"log_start":"258'2652","ondisk_log_start":"258'2652","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":4830,"log_dups_size":0,"ondisk_log_size":4830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":451018752,"num_objects":6882,"num_object_clones":0,"num_object_copies":20646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7482,"num_write_kb":459648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"274'19","reported_seq":550,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.271715+0000","last_change":"2024-09-20T18:59:05.115678+0000","last_active":"2024-09-20T19:00:14.271715+0000","last_peered":"2024-09-20T19:00:14.271715+0000","last_clean":"2024-09-20T19:00:14.271715+0000","last_became_active":"2024-09-20T18:59:05.079753+0000","last_became_peered":"2024-09-20T18:59:05.079753+0000","last_unstale":"2024-09-20T19:00:14.271715+0000","last_undegraded":"2024-09-20T19:00:14.271715+0000","last_fullsized":"2024-09-20T19:00:14.271715+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:58:24.616580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:24.616580+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:10.560525+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":485,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.272445+0000","last_change":"2024-09-20T18:58:32.694078+0000","last_active":"2024-09-20T19:00:14.272445+0000","last_peered":"2024-09-20T19:00:14.272445+0000","last_clean":"2024-09-20T19:00:14.272445+0000","last_became_active":"2024-09-20T18:49:45.944579+0000","last_became_peered":"2024-09-20T18:49:45.944579+0000","last_unstale":"2024-09-20T19:00:14.272445+0000","last_undegraded":"2024-09-20T19:00:14.272445+0000","last_fullsized":"2024-09-20T19:00:14.272445+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.693993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.693993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":505,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.279887+0000","last_change":"2024-09-20T19:00:01.457399+0000","last_active":"2024-09-20T19:00:14.279887+0000","last_peered":"2024-09-20T19:00:14.279887+0000","last_clean":"2024-09-20T19:00:14.279887+0000","last_became_active":"2024-09-20T18:59:05.090504+0000","last_became_peered":"2024-09-20T18:59:05.090504+0000","last_unstale":"2024-09-20T19:00:14.279887+0000","last_undegraded":"2024-09-20T19:00:14.279887+0000","last_fullsized":"2024-09-20T19:00:14.279887+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:00:01.457330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:00:01.457330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:21.889350+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":495,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.306328+0000","last_change":"2024-09-20T18:59:05.080255+0000","last_active":"2024-09-20T19:00:16.306328+0000","last_peered":"2024-09-20T19:00:16.306328+0000","last_clean":"2024-09-20T19:00:16.306328+0000","last_became_active":"2024-09-20T18:59:05.079897+0000","last_became_peered":"2024-09-20T18:59:05.079897+0000","last_unstale":"2024-09-20T19:00:16.306328+0000","last_undegraded":"2024-09-20T19:00:16.306328+0000","last_fullsized":"2024-09-20T19:00:16.306328+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:01.083156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:59:01.083156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.0","version":"287'7716","reported_seq":14363,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.685310+0000","last_change":"2024-09-20T19:00:12.260854+0000","last_active":"2024-09-20T19:00:16.685310+0000","last_peered":"2024-09-20T19:00:16.685310+0000","last_clean":"2024-09-20T19:00:16.685310+0000","last_became_active":"2024-09-20T19:00:12.257557+0000","last_became_peered":"2024-09-20T19:00:12.257557+0000","last_unstale":"2024-09-20T19:00:16.685310+0000","last_undegraded":"2024-09-20T19:00:16.685310+0000","last_fullsized":"2024-09-20T19:00:16.685310+0000","mapping_epoch":284,"log_start":"258'2900","ondisk_log_start":"258'2900","created":246,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":4816,"log_dups_size":0,"ondisk_log_size":4816,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":462028800,"num_objects":7050,"num_object_clones":0,"num_object_copies":21150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7716,"num_write_kb":472512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3270,"num_bytes_recovered":214302720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"274'31","reported_seq":546,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.271829+0000","last_change":"2024-09-20T19:00:03.510074+0000","last_active":"2024-09-20T19:00:14.271829+0000","last_peered":"2024-09-20T19:00:14.271829+0000","last_clean":"2024-09-20T19:00:14.271829+0000","last_became_active":"2024-09-20T18:49:47.947636+0000","last_became_peered":"2024-09-20T18:49:47.947636+0000","last_unstale":"2024-09-20T19:00:14.271829+0000","last_undegraded":"2024-09-20T19:00:14.271829+0000","last_fullsized":"2024-09-20T19:00:14.271829+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'31","last_scrub_stamp":"2024-09-20T19:00:03.509980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:00:03.509980+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:07.241575+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":62,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.281453+0000","last_change":"2024-09-20T18:59:05.081802+0000","last_active":"2024-09-20T19:00:14.281453+0000","last_peered":"2024-09-20T19:00:14.281453+0000","last_clean":"2024-09-20T19:00:14.281453+0000","last_became_active":"2024-09-20T18:59:05.080829+0000","last_became_peered":"2024-09-20T18:59:05.080829+0000","last_unstale":"2024-09-20T19:00:14.281453+0000","last_undegraded":"2024-09-20T19:00:14.281453+0000","last_fullsized":"2024-09-20T19:00:14.281453+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:25.649733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:25.649733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":485,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.299782+0000","last_change":"2024-09-20T18:58:36.100033+0000","last_active":"2024-09-20T19:00:16.299782+0000","last_peered":"2024-09-20T19:00:16.299782+0000","last_clean":"2024-09-20T19:00:16.299782+0000","last_became_active":"2024-09-20T18:49:49.959160+0000","last_became_peered":"2024-09-20T18:49:49.959160+0000","last_unstale":"2024-09-20T19:00:16.299782+0000","last_undegraded":"2024-09-20T19:00:16.299782+0000","last_fullsized":"2024-09-20T19:00:16.299782+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:36.099985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:36.099985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":484,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.297824+0000","last_change":"2024-09-20T18:58:25.227679+0000","last_active":"2024-09-20T19:00:16.297824+0000","last_peered":"2024-09-20T19:00:16.297824+0000","last_clean":"2024-09-20T19:00:16.297824+0000","last_became_active":"2024-09-20T18:49:51.970108+0000","last_became_peered":"2024-09-20T18:49:51.970108+0000","last_unstale":"2024-09-20T19:00:16.297824+0000","last_undegraded":"2024-09-20T19:00:16.297824+0000","last_fullsized":"2024-09-20T19:00:16.297824+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:25.227630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:25.227630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1","version":"287'7511","reported_seq":13972,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.827967+0000","last_change":"2024-09-20T19:00:11.251453+0000","last_active":"2024-09-20T19:00:16.827967+0000","last_peered":"2024-09-20T19:00:16.827967+0000","last_clean":"2024-09-20T19:00:16.827967+0000","last_became_active":"2024-09-20T19:00:11.244922+0000","last_became_peered":"2024-09-20T19:00:11.244922+0000","last_unstale":"2024-09-20T19:00:16.827967+0000","last_undegraded":"2024-09-20T19:00:16.827967+0000","last_fullsized":"2024-09-20T19:00:16.827967+0000","mapping_epoch":283,"log_start":"258'2700","ondisk_log_start":"258'2700","created":246,"last_epoch_clean":284,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":4811,"log_dups_size":0,"ondisk_log_size":4811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":444923904,"num_objects":6789,"num_object_clones":0,"num_object_copies":20367,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6789,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7511,"num_write_kb":457600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3178,"num_bytes_recovered":208273408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"273'20","reported_seq":557,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.272111+0000","last_change":"2024-09-20T18:59:10.364668+0000","last_active":"2024-09-20T19:00:14.272111+0000","last_peered":"2024-09-20T19:00:14.272111+0000","last_clean":"2024-09-20T19:00:14.272111+0000","last_became_active":"2024-09-20T18:59:05.073564+0000","last_became_peered":"2024-09-20T18:59:05.073564+0000","last_unstale":"2024-09-20T19:00:14.272111+0000","last_undegraded":"2024-09-20T19:00:14.272111+0000","last_fullsized":"2024-09-20T19:00:14.272111+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:59:10.364583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:59:10.364583+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:23.352032+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":493,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.302864+0000","last_change":"2024-09-20T18:58:31.126134+0000","last_active":"2024-09-20T19:00:16.302864+0000","last_peered":"2024-09-20T19:00:16.302864+0000","last_clean":"2024-09-20T19:00:16.302864+0000","last_became_active":"2024-09-20T18:49:45.937264+0000","last_became_peered":"2024-09-20T18:49:45.937264+0000","last_unstale":"2024-09-20T19:00:16.302864+0000","last_undegraded":"2024-09-20T19:00:16.302864+0000","last_fullsized":"2024-09-20T19:00:16.302864+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:31.126087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:31.126087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":623,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.297520+0000","last_change":"2024-09-20T18:55:17.549660+0000","last_active":"2024-09-20T19:00:16.297520+0000","last_peered":"2024-09-20T19:00:16.297520+0000","last_clean":"2024-09-20T19:00:16.297520+0000","last_became_active":"2024-09-20T18:49:49.957005+0000","last_became_peered":"2024-09-20T18:49:49.957005+0000","last_unstale":"2024-09-20T19:00:16.297520+0000","last_undegraded":"2024-09-20T19:00:16.297520+0000","last_fullsized":"2024-09-20T19:00:16.297520+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:17.549598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:17.549598+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":489,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.271426+0000","last_change":"2024-09-20T18:59:48.398618+0000","last_active":"2024-09-20T19:00:14.271426+0000","last_peered":"2024-09-20T19:00:14.271426+0000","last_clean":"2024-09-20T19:00:14.271426+0000","last_became_active":"2024-09-20T18:49:51.969355+0000","last_became_peered":"2024-09-20T18:49:51.969355+0000","last_unstale":"2024-09-20T19:00:14.271426+0000","last_undegraded":"2024-09-20T19:00:14.271426+0000","last_fullsized":"2024-09-20T19:00:14.271426+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:48.398542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:59:48.398542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:04.009303+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"286'7404","reported_seq":7495,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.043797+0000","last_change":"2024-09-20T18:58:33.826744+0000","last_active":"2024-09-20T19:00:16.043797+0000","last_peered":"2024-09-20T19:00:16.043797+0000","last_clean":"2024-09-20T19:00:16.043797+0000","last_became_active":"2024-09-20T18:58:33.826480+0000","last_became_peered":"2024-09-20T18:58:33.826480+0000","last_unstale":"2024-09-20T19:00:16.043797+0000","last_undegraded":"2024-09-20T19:00:16.043797+0000","last_fullsized":"2024-09-20T19:00:16.043797+0000","mapping_epoch":246,"log_start":"256'2000","ondisk_log_start":"256'2000","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":5404,"log_dups_size":0,"ondisk_log_size":5404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":445251584,"num_objects":6794,"num_object_clones":0,"num_object_copies":20382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6794,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7404,"num_write_kb":454336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"274'15","reported_seq":515,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.308843+0000","last_change":"2024-09-20T18:54:54.897991+0000","last_active":"2024-09-20T19:00:16.308843+0000","last_peered":"2024-09-20T19:00:16.308843+0000","last_clean":"2024-09-20T19:00:16.308843+0000","last_became_active":"2024-09-20T18:49:47.946008+0000","last_became_peered":"2024-09-20T18:49:47.946008+0000","last_unstale":"2024-09-20T19:00:16.308843+0000","last_undegraded":"2024-09-20T19:00:16.308843+0000","last_fullsized":"2024-09-20T19:00:16.308843+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:54.897933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:54.897933+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:13.815921+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":30,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.297002+0000","last_change":"2024-09-20T19:00:01.191220+0000","last_active":"2024-09-20T19:00:16.297002+0000","last_peered":"2024-09-20T19:00:16.297002+0000","last_clean":"2024-09-20T19:00:16.297002+0000","last_became_active":"2024-09-20T19:00:01.190875+0000","last_became_peered":"2024-09-20T19:00:01.190875+0000","last_unstale":"2024-09-20T19:00:16.297002+0000","last_undegraded":"2024-09-20T19:00:16.297002+0000","last_fullsized":"2024-09-20T19:00:16.297002+0000","mapping_epoch":277,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":278,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:56.405349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:56.405349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":488,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.296609+0000","last_change":"2024-09-20T18:54:43.379898+0000","last_active":"2024-09-20T19:00:16.296609+0000","last_peered":"2024-09-20T19:00:16.296609+0000","last_clean":"2024-09-20T19:00:16.296609+0000","last_became_active":"2024-09-20T18:49:49.956767+0000","last_became_peered":"2024-09-20T18:49:49.956767+0000","last_unstale":"2024-09-20T19:00:16.296609+0000","last_undegraded":"2024-09-20T19:00:16.296609+0000","last_fullsized":"2024-09-20T19:00:16.296609+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:43.379706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:43.379706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":474,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.310108+0000","last_change":"2024-09-20T18:54:24.976627+0000","last_active":"2024-09-20T19:00:16.310108+0000","last_peered":"2024-09-20T19:00:16.310108+0000","last_clean":"2024-09-20T19:00:16.310108+0000","last_became_active":"2024-09-20T18:49:51.964098+0000","last_became_peered":"2024-09-20T18:49:51.964098+0000","last_unstale":"2024-09-20T19:00:16.310108+0000","last_undegraded":"2024-09-20T19:00:16.310108+0000","last_fullsized":"2024-09-20T19:00:16.310108+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:24.976584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:24.976584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:10.886988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"286'7552","reported_seq":7643,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.041284+0000","last_change":"2024-09-20T18:58:33.824658+0000","last_active":"2024-09-20T19:00:16.041284+0000","last_peered":"2024-09-20T19:00:16.041284+0000","last_clean":"2024-09-20T19:00:16.041284+0000","last_became_active":"2024-09-20T18:58:33.824342+0000","last_became_peered":"2024-09-20T18:58:33.824342+0000","last_unstale":"2024-09-20T19:00:16.041284+0000","last_undegraded":"2024-09-20T19:00:16.041284+0000","last_fullsized":"2024-09-20T19:00:16.041284+0000","mapping_epoch":246,"log_start":"256'2100","ondisk_log_start":"256'2100","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":5452,"log_dups_size":0,"ondisk_log_size":5452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":455999488,"num_objects":6958,"num_object_clones":0,"num_object_copies":20874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6958,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7552,"num_write_kb":464320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"274'35","reported_seq":106,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.281243+0000","last_change":"2024-09-20T18:59:05.107086+0000","last_active":"2024-09-20T19:00:14.281243+0000","last_peered":"2024-09-20T19:00:14.281243+0000","last_clean":"2024-09-20T19:00:14.281243+0000","last_became_active":"2024-09-20T18:59:05.087391+0000","last_became_peered":"2024-09-20T18:59:05.087391+0000","last_unstale":"2024-09-20T19:00:14.281243+0000","last_undegraded":"2024-09-20T19:00:14.281243+0000","last_fullsized":"2024-09-20T19:00:14.281243+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-20T18:57:22.033475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:22.033475+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":506,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.276671+0000","last_change":"2024-09-20T18:59:05.087282+0000","last_active":"2024-09-20T19:00:14.276671+0000","last_peered":"2024-09-20T19:00:14.276671+0000","last_clean":"2024-09-20T19:00:14.276671+0000","last_became_active":"2024-09-20T18:59:05.086670+0000","last_became_peered":"2024-09-20T18:59:05.086670+0000","last_unstale":"2024-09-20T19:00:14.276671+0000","last_undegraded":"2024-09-20T19:00:14.276671+0000","last_fullsized":"2024-09-20T19:00:14.276671+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:44.501192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.501192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":488,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.296866+0000","last_change":"2024-09-20T18:55:13.577772+0000","last_active":"2024-09-20T19:00:16.296866+0000","last_peered":"2024-09-20T19:00:16.296866+0000","last_clean":"2024-09-20T19:00:16.296866+0000","last_became_active":"2024-09-20T18:49:49.957558+0000","last_became_peered":"2024-09-20T18:49:49.957558+0000","last_unstale":"2024-09-20T19:00:16.296866+0000","last_undegraded":"2024-09-20T19:00:16.296866+0000","last_fullsized":"2024-09-20T19:00:16.296866+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:13.577699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:13.577699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":480,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.275933+0000","last_change":"2024-09-20T18:53:47.370152+0000","last_active":"2024-09-20T19:00:14.275933+0000","last_peered":"2024-09-20T19:00:14.275933+0000","last_clean":"2024-09-20T19:00:14.275933+0000","last_became_active":"2024-09-20T18:49:51.970798+0000","last_became_peered":"2024-09-20T18:49:51.970798+0000","last_unstale":"2024-09-20T19:00:14.275933+0000","last_undegraded":"2024-09-20T19:00:14.275933+0000","last_fullsized":"2024-09-20T19:00:14.275933+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:47.370108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:47.370108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.4","version":"287'7391","reported_seq":7484,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.706663+0000","last_change":"2024-09-20T18:58:33.846071+0000","last_active":"2024-09-20T19:00:16.706663+0000","last_peered":"2024-09-20T19:00:16.706663+0000","last_clean":"2024-09-20T19:00:16.706663+0000","last_became_active":"2024-09-20T18:58:33.845736+0000","last_became_peered":"2024-09-20T18:58:33.845736+0000","last_unstale":"2024-09-20T19:00:16.706663+0000","last_undegraded":"2024-09-20T19:00:16.706663+0000","last_fullsized":"2024-09-20T19:00:16.706663+0000","mapping_epoch":246,"log_start":"258'2600","ondisk_log_start":"258'2600","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":4791,"log_dups_size":0,"ondisk_log_size":4791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":441778176,"num_objects":6741,"num_object_clones":0,"num_object_copies":20223,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7391,"num_write_kb":452224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"273'18","reported_seq":521,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.272298+0000","last_change":"2024-09-20T18:59:55.490161+0000","last_active":"2024-09-20T19:00:14.272298+0000","last_peered":"2024-09-20T19:00:14.272298+0000","last_clean":"2024-09-20T19:00:14.272298+0000","last_became_active":"2024-09-20T18:49:47.949116+0000","last_became_peered":"2024-09-20T18:49:47.949116+0000","last_unstale":"2024-09-20T19:00:14.272298+0000","last_undegraded":"2024-09-20T19:00:14.272298+0000","last_fullsized":"2024-09-20T19:00:14.272298+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'18","last_scrub_stamp":"2024-09-20T18:59:55.490086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:59:55.490086+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:15.287945+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":492,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.276648+0000","last_change":"2024-09-20T18:54:52.650798+0000","last_active":"2024-09-20T19:00:14.276648+0000","last_peered":"2024-09-20T19:00:14.276648+0000","last_clean":"2024-09-20T19:00:14.276648+0000","last_became_active":"2024-09-20T18:49:45.944858+0000","last_became_peered":"2024-09-20T18:49:45.944858+0000","last_unstale":"2024-09-20T19:00:14.276648+0000","last_undegraded":"2024-09-20T19:00:14.276648+0000","last_fullsized":"2024-09-20T19:00:14.276648+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:52.650756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:52.650756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":488,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.296208+0000","last_change":"2024-09-20T18:58:33.180357+0000","last_active":"2024-09-20T19:00:16.296208+0000","last_peered":"2024-09-20T19:00:16.296208+0000","last_clean":"2024-09-20T19:00:16.296208+0000","last_became_active":"2024-09-20T18:49:49.954944+0000","last_became_peered":"2024-09-20T18:49:49.954944+0000","last_unstale":"2024-09-20T19:00:16.296208+0000","last_undegraded":"2024-09-20T19:00:16.296208+0000","last_fullsized":"2024-09-20T19:00:16.296208+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:33.180309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:33.180309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":481,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.302615+0000","last_change":"2024-09-20T18:58:24.162921+0000","last_active":"2024-09-20T19:00:16.302615+0000","last_peered":"2024-09-20T19:00:16.302615+0000","last_clean":"2024-09-20T19:00:16.302615+0000","last_became_active":"2024-09-20T18:49:51.972970+0000","last_became_peered":"2024-09-20T18:49:51.972970+0000","last_unstale":"2024-09-20T19:00:16.302615+0000","last_undegraded":"2024-09-20T19:00:16.302615+0000","last_fullsized":"2024-09-20T19:00:16.302615+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:24.162879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:24.162879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"287'7520","reported_seq":7612,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:17.203278+0000","last_change":"2024-09-20T18:58:33.821228+0000","last_active":"2024-09-20T19:00:17.203278+0000","last_peered":"2024-09-20T19:00:17.203278+0000","last_clean":"2024-09-20T19:00:17.203278+0000","last_became_active":"2024-09-20T18:58:33.819949+0000","last_became_peered":"2024-09-20T18:58:33.819949+0000","last_unstale":"2024-09-20T19:00:17.203278+0000","last_undegraded":"2024-09-20T19:00:17.203278+0000","last_fullsized":"2024-09-20T19:00:17.203278+0000","mapping_epoch":246,"log_start":"254'1500","ondisk_log_start":"254'1500","created":246,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:58:32.801969+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.801969+0000","objects_scrubbed":0,"log_size":6020,"log_dups_size":0,"ondisk_log_size":6020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":444858368,"num_objects":6788,"num_object_clones":0,"num_object_copies":20364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6788,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7520,"num_write_kb":457856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"273'27","reported_seq":576,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.279193+0000","last_change":"2024-09-20T18:59:05.108455+0000","last_active":"2024-09-20T19:00:14.279193+0000","last_peered":"2024-09-20T19:00:14.279193+0000","last_clean":"2024-09-20T19:00:14.279193+0000","last_became_active":"2024-09-20T18:59:05.083726+0000","last_became_peered":"2024-09-20T18:59:05.083726+0000","last_unstale":"2024-09-20T19:00:14.279193+0000","last_undegraded":"2024-09-20T19:00:14.279193+0000","last_fullsized":"2024-09-20T19:00:14.279193+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-20T18:58:30.522129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:30.522129+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":520,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.276704+0000","last_change":"2024-09-20T18:58:57.563127+0000","last_active":"2024-09-20T19:00:14.276704+0000","last_peered":"2024-09-20T19:00:14.276704+0000","last_clean":"2024-09-20T19:00:14.276704+0000","last_became_active":"2024-09-20T18:49:45.944256+0000","last_became_peered":"2024-09-20T18:49:45.944256+0000","last_unstale":"2024-09-20T19:00:14.276704+0000","last_undegraded":"2024-09-20T19:00:14.276704+0000","last_fullsized":"2024-09-20T19:00:14.276704+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-20T18:58:57.563058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:57.563058+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:20.547993+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"262'10","reported_seq":632,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.276335+0000","last_change":"2024-09-20T18:59:05.111302+0000","last_active":"2024-09-20T19:00:14.276335+0000","last_peered":"2024-09-20T19:00:14.276335+0000","last_clean":"2024-09-20T19:00:14.276335+0000","last_became_active":"2024-09-20T18:59:05.090811+0000","last_became_peered":"2024-09-20T18:59:05.090811+0000","last_unstale":"2024-09-20T19:00:14.276335+0000","last_undegraded":"2024-09-20T19:00:14.276335+0000","last_fullsized":"2024-09-20T19:00:14.276335+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'9","last_scrub_stamp":"2024-09-20T18:58:27.521051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:27.521051+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":493,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.279292+0000","last_change":"2024-09-20T18:59:05.087336+0000","last_active":"2024-09-20T19:00:14.279292+0000","last_peered":"2024-09-20T19:00:14.279292+0000","last_clean":"2024-09-20T19:00:14.279292+0000","last_became_active":"2024-09-20T18:59:05.087005+0000","last_became_peered":"2024-09-20T18:59:05.087005+0000","last_unstale":"2024-09-20T19:00:14.279292+0000","last_undegraded":"2024-09-20T19:00:14.279292+0000","last_fullsized":"2024-09-20T19:00:14.279292+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:05.523359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:05.523359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":62,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.281202+0000","last_change":"2024-09-20T18:59:05.085844+0000","last_active":"2024-09-20T19:00:14.281202+0000","last_peered":"2024-09-20T19:00:14.281202+0000","last_clean":"2024-09-20T19:00:14.281202+0000","last_became_active":"2024-09-20T18:59:05.085427+0000","last_became_peered":"2024-09-20T18:59:05.085427+0000","last_unstale":"2024-09-20T19:00:14.281202+0000","last_undegraded":"2024-09-20T19:00:14.281202+0000","last_fullsized":"2024-09-20T19:00:14.281202+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:26.721735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:26.721735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"273'12","reported_seq":507,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.304176+0000","last_change":"2024-09-20T18:58:09.063555+0000","last_active":"2024-09-20T19:00:16.304176+0000","last_peered":"2024-09-20T19:00:16.304176+0000","last_clean":"2024-09-20T19:00:16.304176+0000","last_became_active":"2024-09-20T18:49:48.500662+0000","last_became_peered":"2024-09-20T18:49:48.500662+0000","last_unstale":"2024-09-20T19:00:16.304176+0000","last_undegraded":"2024-09-20T19:00:16.304176+0000","last_fullsized":"2024-09-20T19:00:16.304176+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-20T18:58:09.063481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:09.063481+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":483,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.279752+0000","last_change":"2024-09-20T18:58:28.460106+0000","last_active":"2024-09-20T19:00:14.279752+0000","last_peered":"2024-09-20T19:00:14.279752+0000","last_clean":"2024-09-20T19:00:14.279752+0000","last_became_active":"2024-09-20T18:49:49.956749+0000","last_became_peered":"2024-09-20T18:49:49.956749+0000","last_unstale":"2024-09-20T19:00:14.279752+0000","last_undegraded":"2024-09-20T19:00:14.279752+0000","last_fullsized":"2024-09-20T19:00:14.279752+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:28.460054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:28.460054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":494,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.276092+0000","last_change":"2024-09-20T18:59:05.082737+0000","last_active":"2024-09-20T19:00:14.276092+0000","last_peered":"2024-09-20T19:00:14.276092+0000","last_clean":"2024-09-20T19:00:14.276092+0000","last_became_active":"2024-09-20T18:59:05.082018+0000","last_became_peered":"2024-09-20T18:59:05.082018+0000","last_unstale":"2024-09-20T19:00:14.276092+0000","last_undegraded":"2024-09-20T19:00:14.276092+0000","last_fullsized":"2024-09-20T19:00:14.276092+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.637739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.637739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"273'15","reported_seq":507,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.308138+0000","last_change":"2024-09-20T18:54:44.830872+0000","last_active":"2024-09-20T19:00:16.308138+0000","last_peered":"2024-09-20T19:00:16.308138+0000","last_clean":"2024-09-20T19:00:16.308138+0000","last_became_active":"2024-09-20T18:49:48.500803+0000","last_became_peered":"2024-09-20T18:49:48.500803+0000","last_unstale":"2024-09-20T19:00:16.308138+0000","last_undegraded":"2024-09-20T19:00:16.308138+0000","last_fullsized":"2024-09-20T19:00:16.308138+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:44.830790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.830790+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:11.859332+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":506,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.277166+0000","last_change":"2024-09-20T18:59:05.091318+0000","last_active":"2024-09-20T19:00:14.277166+0000","last_peered":"2024-09-20T19:00:14.277166+0000","last_clean":"2024-09-20T19:00:14.277166+0000","last_became_active":"2024-09-20T18:59:05.090805+0000","last_became_peered":"2024-09-20T18:59:05.090805+0000","last_unstale":"2024-09-20T19:00:14.277166+0000","last_undegraded":"2024-09-20T19:00:14.277166+0000","last_fullsized":"2024-09-20T19:00:14.277166+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:59.589891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:59.589891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":485,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.304041+0000","last_change":"2024-09-20T18:57:46.898929+0000","last_active":"2024-09-20T19:00:16.304041+0000","last_peered":"2024-09-20T19:00:16.304041+0000","last_clean":"2024-09-20T19:00:16.304041+0000","last_became_active":"2024-09-20T18:49:49.961947+0000","last_became_peered":"2024-09-20T18:49:49.961947+0000","last_unstale":"2024-09-20T19:00:16.304041+0000","last_undegraded":"2024-09-20T19:00:16.304041+0000","last_fullsized":"2024-09-20T19:00:16.304041+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:46.898862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:46.898862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":38,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"66'1","reported_seq":493,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.297075+0000","last_change":"2024-09-20T18:58:30.171382+0000","last_active":"2024-09-20T19:00:16.297075+0000","last_peered":"2024-09-20T19:00:16.297075+0000","last_clean":"2024-09-20T19:00:16.297075+0000","last_became_active":"2024-09-20T18:49:51.965121+0000","last_became_peered":"2024-09-20T18:49:51.965121+0000","last_unstale":"2024-09-20T19:00:16.297075+0000","last_undegraded":"2024-09-20T19:00:16.297075+0000","last_fullsized":"2024-09-20T19:00:16.297075+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T18:58:30.171329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:30.171329+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"274'15","reported_seq":504,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.279729+0000","last_change":"2024-09-20T18:57:59.430353+0000","last_active":"2024-09-20T19:00:14.279729+0000","last_peered":"2024-09-20T19:00:14.279729+0000","last_clean":"2024-09-20T19:00:14.279729+0000","last_became_active":"2024-09-20T18:49:48.501011+0000","last_became_peered":"2024-09-20T18:49:48.501011+0000","last_unstale":"2024-09-20T19:00:14.279729+0000","last_undegraded":"2024-09-20T19:00:14.279729+0000","last_fullsized":"2024-09-20T19:00:14.279729+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:57:59.430294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:59.430294+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":494,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.307673+0000","last_change":"2024-09-20T18:55:27.028184+0000","last_active":"2024-09-20T19:00:16.307673+0000","last_peered":"2024-09-20T19:00:16.307673+0000","last_clean":"2024-09-20T19:00:16.307673+0000","last_became_active":"2024-09-20T18:49:45.938511+0000","last_became_peered":"2024-09-20T18:49:45.938511+0000","last_unstale":"2024-09-20T19:00:16.307673+0000","last_undegraded":"2024-09-20T19:00:16.307673+0000","last_fullsized":"2024-09-20T19:00:16.307673+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:55:27.028126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:55:27.028126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":620,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.306450+0000","last_change":"2024-09-20T18:55:27.284156+0000","last_active":"2024-09-20T19:00:16.306450+0000","last_peered":"2024-09-20T19:00:16.306450+0000","last_clean":"2024-09-20T19:00:16.306450+0000","last_became_active":"2024-09-20T18:49:49.960550+0000","last_became_peered":"2024-09-20T18:49:49.960550+0000","last_unstale":"2024-09-20T19:00:16.306450+0000","last_undegraded":"2024-09-20T19:00:16.306450+0000","last_fullsized":"2024-09-20T19:00:16.306450+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:27.284086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:27.284086+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":489,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.272096+0000","last_change":"2024-09-20T18:59:59.497880+0000","last_active":"2024-09-20T19:00:14.272096+0000","last_peered":"2024-09-20T19:00:14.272096+0000","last_clean":"2024-09-20T19:00:14.272096+0000","last_became_active":"2024-09-20T18:49:51.966254+0000","last_became_peered":"2024-09-20T18:49:51.966254+0000","last_unstale":"2024-09-20T19:00:14.272096+0000","last_undegraded":"2024-09-20T19:00:14.272096+0000","last_fullsized":"2024-09-20T19:00:14.272096+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:59.497810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:59:59.497810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:04.610550+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"274'20","reported_seq":530,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.308770+0000","last_change":"2024-09-20T18:57:20.701592+0000","last_active":"2024-09-20T19:00:16.308770+0000","last_peered":"2024-09-20T19:00:16.308770+0000","last_clean":"2024-09-20T19:00:16.308770+0000","last_became_active":"2024-09-20T18:49:58.997494+0000","last_became_peered":"2024-09-20T18:49:58.997494+0000","last_unstale":"2024-09-20T19:00:16.308770+0000","last_undegraded":"2024-09-20T19:00:16.308770+0000","last_fullsized":"2024-09-20T19:00:16.308770+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:57:20.701543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:20.701543+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":492,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.276526+0000","last_change":"2024-09-20T18:54:57.632458+0000","last_active":"2024-09-20T19:00:14.276526+0000","last_peered":"2024-09-20T19:00:14.276526+0000","last_clean":"2024-09-20T19:00:14.276526+0000","last_became_active":"2024-09-20T18:49:45.949492+0000","last_became_peered":"2024-09-20T18:49:45.949492+0000","last_unstale":"2024-09-20T19:00:14.276526+0000","last_undegraded":"2024-09-20T19:00:14.276526+0000","last_fullsized":"2024-09-20T19:00:14.276526+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.632380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.632380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":617,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.279614+0000","last_change":"2024-09-20T18:55:28.302353+0000","last_active":"2024-09-20T19:00:14.279614+0000","last_peered":"2024-09-20T19:00:14.279614+0000","last_clean":"2024-09-20T19:00:14.279614+0000","last_became_active":"2024-09-20T18:49:49.962243+0000","last_became_peered":"2024-09-20T18:49:49.962243+0000","last_unstale":"2024-09-20T19:00:14.279614+0000","last_undegraded":"2024-09-20T19:00:14.279614+0000","last_fullsized":"2024-09-20T19:00:14.279614+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:55:28.302303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:55:28.302303+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:09.385599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":490,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.307779+0000","last_change":"2024-09-20T18:58:55.232948+0000","last_active":"2024-09-20T19:00:16.307779+0000","last_peered":"2024-09-20T19:00:16.307779+0000","last_clean":"2024-09-20T19:00:16.307779+0000","last_became_active":"2024-09-20T18:49:51.969093+0000","last_became_peered":"2024-09-20T18:49:51.969093+0000","last_unstale":"2024-09-20T19:00:16.307779+0000","last_undegraded":"2024-09-20T19:00:16.307779+0000","last_fullsized":"2024-09-20T19:00:16.307779+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:55.232883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:55.232883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":59,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"273'19","reported_seq":558,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.271175+0000","last_change":"2024-09-20T18:59:05.117294+0000","last_active":"2024-09-20T19:00:14.271175+0000","last_peered":"2024-09-20T19:00:14.271175+0000","last_clean":"2024-09-20T19:00:14.271175+0000","last_became_active":"2024-09-20T18:59:05.081917+0000","last_became_peered":"2024-09-20T18:59:05.081917+0000","last_unstale":"2024-09-20T19:00:14.271175+0000","last_undegraded":"2024-09-20T19:00:14.271175+0000","last_fullsized":"2024-09-20T19:00:14.271175+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:57:50.899514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:50.899514+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:28.208671+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":499,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.272363+0000","last_change":"2024-09-20T18:59:06.310100+0000","last_active":"2024-09-20T19:00:14.272363+0000","last_peered":"2024-09-20T19:00:14.272363+0000","last_clean":"2024-09-20T19:00:14.272363+0000","last_became_active":"2024-09-20T18:59:05.073087+0000","last_became_peered":"2024-09-20T18:59:05.073087+0000","last_unstale":"2024-09-20T19:00:14.272363+0000","last_undegraded":"2024-09-20T19:00:14.272363+0000","last_fullsized":"2024-09-20T19:00:14.272363+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:06.310027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:59:06.310027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:35.049125+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":477,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.299303+0000","last_change":"2024-09-20T18:57:11.986605+0000","last_active":"2024-09-20T19:00:16.299303+0000","last_peered":"2024-09-20T19:00:16.299303+0000","last_clean":"2024-09-20T19:00:16.299303+0000","last_became_active":"2024-09-20T18:49:49.957067+0000","last_became_peered":"2024-09-20T18:49:49.957067+0000","last_unstale":"2024-09-20T19:00:16.299303+0000","last_undegraded":"2024-09-20T19:00:16.299303+0000","last_fullsized":"2024-09-20T19:00:16.299303+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:11.986528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:11.986528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:15.695036+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":473,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.301997+0000","last_change":"2024-09-20T18:57:16.789412+0000","last_active":"2024-09-20T19:00:16.301997+0000","last_peered":"2024-09-20T19:00:16.301997+0000","last_clean":"2024-09-20T19:00:16.301997+0000","last_became_active":"2024-09-20T18:49:51.966915+0000","last_became_peered":"2024-09-20T18:49:51.966915+0000","last_unstale":"2024-09-20T19:00:16.301997+0000","last_undegraded":"2024-09-20T19:00:16.301997+0000","last_fullsized":"2024-09-20T19:00:16.301997+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:16.789095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:16.789095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"273'15","reported_seq":556,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.299624+0000","last_change":"2024-09-20T18:59:05.103863+0000","last_active":"2024-09-20T19:00:16.299624+0000","last_peered":"2024-09-20T19:00:16.299624+0000","last_clean":"2024-09-20T19:00:16.299624+0000","last_became_active":"2024-09-20T18:59:05.085230+0000","last_became_peered":"2024-09-20T18:59:05.085230+0000","last_unstale":"2024-09-20T19:00:16.299624+0000","last_undegraded":"2024-09-20T19:00:16.299624+0000","last_fullsized":"2024-09-20T19:00:16.299624+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:58:28.035450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:28.035450+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":484,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.276571+0000","last_change":"2024-09-20T18:54:42.487465+0000","last_active":"2024-09-20T19:00:14.276571+0000","last_peered":"2024-09-20T19:00:14.276571+0000","last_clean":"2024-09-20T19:00:14.276571+0000","last_became_active":"2024-09-20T18:49:45.944357+0000","last_became_peered":"2024-09-20T18:49:45.944357+0000","last_unstale":"2024-09-20T19:00:14.276571+0000","last_undegraded":"2024-09-20T19:00:14.276571+0000","last_fullsized":"2024-09-20T19:00:14.276571+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:42.487403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:42.487403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":485,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.302651+0000","last_change":"2024-09-20T18:57:57.915090+0000","last_active":"2024-09-20T19:00:16.302651+0000","last_peered":"2024-09-20T19:00:16.302651+0000","last_clean":"2024-09-20T19:00:16.302651+0000","last_became_active":"2024-09-20T18:49:49.962523+0000","last_became_peered":"2024-09-20T18:49:49.962523+0000","last_unstale":"2024-09-20T19:00:16.302651+0000","last_undegraded":"2024-09-20T19:00:16.302651+0000","last_fullsized":"2024-09-20T19:00:16.302651+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:57.914913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:57.914913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":482,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.308715+0000","last_change":"2024-09-20T18:54:26.882441+0000","last_active":"2024-09-20T19:00:16.308715+0000","last_peered":"2024-09-20T19:00:16.308715+0000","last_clean":"2024-09-20T19:00:16.308715+0000","last_became_active":"2024-09-20T18:49:51.967313+0000","last_became_peered":"2024-09-20T18:49:51.967313+0000","last_unstale":"2024-09-20T19:00:16.308715+0000","last_undegraded":"2024-09-20T19:00:16.308715+0000","last_fullsized":"2024-09-20T19:00:16.308715+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:26.882375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:54:26.882375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:06.821534+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"274'19","reported_seq":558,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.302111+0000","last_change":"2024-09-20T18:59:05.096414+0000","last_active":"2024-09-20T19:00:16.302111+0000","last_peered":"2024-09-20T19:00:16.302111+0000","last_clean":"2024-09-20T19:00:16.302111+0000","last_became_active":"2024-09-20T18:59:05.075390+0000","last_became_peered":"2024-09-20T18:59:05.075390+0000","last_unstale":"2024-09-20T19:00:16.302111+0000","last_undegraded":"2024-09-20T19:00:16.302111+0000","last_fullsized":"2024-09-20T19:00:16.302111+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:57:54.862329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:54.862329+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":499,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.279777+0000","last_change":"2024-09-20T18:58:06.479419+0000","last_active":"2024-09-20T19:00:14.279777+0000","last_peered":"2024-09-20T19:00:14.279777+0000","last_clean":"2024-09-20T19:00:14.279777+0000","last_became_active":"2024-09-20T18:49:45.942105+0000","last_became_peered":"2024-09-20T18:49:45.942105+0000","last_unstale":"2024-09-20T19:00:14.279777+0000","last_undegraded":"2024-09-20T19:00:14.279777+0000","last_fullsized":"2024-09-20T19:00:14.279777+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:06.479364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:06.479364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":498,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.275715+0000","last_change":"2024-09-20T18:59:05.081655+0000","last_active":"2024-09-20T19:00:14.275715+0000","last_peered":"2024-09-20T19:00:14.275715+0000","last_clean":"2024-09-20T19:00:14.275715+0000","last_became_active":"2024-09-20T18:59:05.081320+0000","last_became_peered":"2024-09-20T18:59:05.081320+0000","last_unstale":"2024-09-20T19:00:14.275715+0000","last_undegraded":"2024-09-20T19:00:14.275715+0000","last_fullsized":"2024-09-20T19:00:14.275715+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:46.329547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:53:46.329547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":487,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.302782+0000","last_change":"2024-09-20T18:59:05.081584+0000","last_active":"2024-09-20T19:00:16.302782+0000","last_peered":"2024-09-20T19:00:16.302782+0000","last_clean":"2024-09-20T19:00:16.302782+0000","last_became_active":"2024-09-20T18:59:05.081179+0000","last_became_peered":"2024-09-20T18:59:05.081179+0000","last_unstale":"2024-09-20T19:00:16.302782+0000","last_undegraded":"2024-09-20T19:00:16.302782+0000","last_fullsized":"2024-09-20T19:00:16.302782+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:28.165712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:28.165712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"273'8","reported_seq":527,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.302575+0000","last_change":"2024-09-20T18:59:05.098374+0000","last_active":"2024-09-20T19:00:16.302575+0000","last_peered":"2024-09-20T19:00:16.302575+0000","last_clean":"2024-09-20T19:00:16.302575+0000","last_became_active":"2024-09-20T18:59:05.076365+0000","last_became_peered":"2024-09-20T18:59:05.076365+0000","last_unstale":"2024-09-20T19:00:16.302575+0000","last_undegraded":"2024-09-20T19:00:16.302575+0000","last_fullsized":"2024-09-20T19:00:16.302575+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-20T18:55:20.282193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:55:20.282193+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":494,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.307845+0000","last_change":"2024-09-20T18:54:55.900487+0000","last_active":"2024-09-20T19:00:16.307845+0000","last_peered":"2024-09-20T19:00:16.307845+0000","last_clean":"2024-09-20T19:00:16.307845+0000","last_became_active":"2024-09-20T18:49:45.940988+0000","last_became_peered":"2024-09-20T18:49:45.940988+0000","last_unstale":"2024-09-20T19:00:16.307845+0000","last_undegraded":"2024-09-20T19:00:16.307845+0000","last_fullsized":"2024-09-20T19:00:16.307845+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:55.899611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:55.899611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:12.863475+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":62,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.283845+0000","last_change":"2024-09-20T18:59:05.086874+0000","last_active":"2024-09-20T19:00:14.283845+0000","last_peered":"2024-09-20T19:00:14.283845+0000","last_clean":"2024-09-20T19:00:14.283845+0000","last_became_active":"2024-09-20T18:59:05.086570+0000","last_became_peered":"2024-09-20T18:59:05.086570+0000","last_unstale":"2024-09-20T19:00:14.283845+0000","last_undegraded":"2024-09-20T19:00:14.283845+0000","last_fullsized":"2024-09-20T19:00:14.283845+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:52.929828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:52.929828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":472,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.276239+0000","last_change":"2024-09-20T18:58:22.558181+0000","last_active":"2024-09-20T19:00:14.276239+0000","last_peered":"2024-09-20T19:00:14.276239+0000","last_clean":"2024-09-20T19:00:14.276239+0000","last_became_active":"2024-09-20T18:49:51.968662+0000","last_became_peered":"2024-09-20T18:49:51.968662+0000","last_unstale":"2024-09-20T19:00:14.276239+0000","last_undegraded":"2024-09-20T19:00:14.276239+0000","last_fullsized":"2024-09-20T19:00:14.276239+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:22.558131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:22.558131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":465,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.297739+0000","last_change":"2024-09-20T18:57:10.970843+0000","last_active":"2024-09-20T19:00:16.297739+0000","last_peered":"2024-09-20T19:00:16.297739+0000","last_clean":"2024-09-20T19:00:16.297739+0000","last_became_active":"2024-09-20T18:49:51.968216+0000","last_became_peered":"2024-09-20T18:49:51.968216+0000","last_unstale":"2024-09-20T19:00:16.297739+0000","last_undegraded":"2024-09-20T19:00:16.297739+0000","last_fullsized":"2024-09-20T19:00:16.297739+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:10.970779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:10.970779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":493,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.305195+0000","last_change":"2024-09-20T18:58:36.077005+0000","last_active":"2024-09-20T19:00:16.305195+0000","last_peered":"2024-09-20T19:00:16.305195+0000","last_clean":"2024-09-20T19:00:16.305195+0000","last_became_active":"2024-09-20T18:49:45.937992+0000","last_became_peered":"2024-09-20T18:49:45.937992+0000","last_unstale":"2024-09-20T19:00:16.305195+0000","last_undegraded":"2024-09-20T19:00:16.305195+0000","last_fullsized":"2024-09-20T19:00:16.305195+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:36.076937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:36.076937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"274'19","reported_seq":86,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-20T19:00:14.282333+0000","last_change":"2024-09-20T18:59:05.116749+0000","last_active":"2024-09-20T19:00:14.282333+0000","last_peered":"2024-09-20T19:00:14.282333+0000","last_clean":"2024-09-20T19:00:14.282333+0000","last_became_active":"2024-09-20T18:59:05.089294+0000","last_became_peered":"2024-09-20T18:59:05.089294+0000","last_unstale":"2024-09-20T19:00:14.282333+0000","last_undegraded":"2024-09-20T19:00:14.282333+0000","last_fullsized":"2024-09-20T19:00:14.282333+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:54:44.719676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:44.719676+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:14.182838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":479,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-20T19:00:16.307920+0000","last_change":"2024-09-20T18:58:27.904636+0000","last_active":"2024-09-20T19:00:16.307920+0000","last_peered":"2024-09-20T19:00:16.307920+0000","last_clean":"2024-09-20T19:00:16.307920+0000","last_became_active":"2024-09-20T18:49:49.957582+0000","last_became_peered":"2024-09-20T18:49:49.957582+0000","last_unstale":"2024-09-20T19:00:16.307920+0000","last_undegraded":"2024-09-20T19:00:16.307920+0000","last_fullsized":"2024-09-20T19:00:16.307920+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:27.904592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:27.904592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":7136280600,"num_objects":108892,"num_object_clones":0,"num_object_copies":326676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108892,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":119398,"num_write_kb":7305217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9499,"num_bytes_recovered":622526464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":21632397312,"data_stored":21632385096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83083,"ondisk_log_size":83083,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1124,"num_read_kb":881,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":2244,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":681,"ondisk_log_size":681,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":624,"num_read_kb":619,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266818,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4317432,"kb_used_data":3961384,"kb_used_omap":1,"kb_used_meta":356030,"kb_avail":89427720,"statfs":{"total":95995035648,"available":91573985280,"internally_reserved":0,"allocated":4056457216,"data_stored":4056047801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":364575178},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528457,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1747812,"kb_used_data":1455632,"kb_used_omap":1,"kb_used_meta":292158,"kb_avail":91997340,"statfs":{"total":95995035648,"available":94205276160,"internally_reserved":0,"allocated":1490567168,"data_stored":1490181643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":299170246},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757390,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4332456,"kb_used_data":3954196,"kb_used_omap":1,"kb_used_meta":378238,"kb_avail":89412696,"statfs":{"total":95995035648,"available":91558600704,"internally_reserved":0,"allocated":4049096704,"data_stored":4048706922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":387316165},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986323,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1793220,"kb_used_data":1509360,"kb_used_omap":1,"kb_used_meta":283838,"kb_avail":91951932,"statfs":{"total":95995035648,"available":94158778368,"internally_reserved":0,"allocated":1545584640,"data_stored":1545167414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":290650565},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182554,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3954188,"kb_used_data":3475960,"kb_used_omap":1,"kb_used_meta":478206,"kb_avail":89790964,"statfs":{"total":95995035648,"available":91945947136,"internally_reserved":0,"allocated":3559383040,"data_stored":3558963373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":489683399},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411487,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1760612,"kb_used_data":1328148,"kb_used_omap":1,"kb_used_meta":432382,"kb_avail":91984540,"statfs":{"total":95995035648,"available":94192168960,"internally_reserved":0,"allocated":1360023552,"data_stored":1359634325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":442759621},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575013,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4356888,"kb_used_data":3964616,"kb_used_omap":1,"kb_used_meta":392126,"kb_avail":89388264,"statfs":{"total":95995035648,"available":91533582336,"internally_reserved":0,"allocated":4059766784,"data_stored":4059325517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":401537477},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738539,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2111044,"kb_used_data":1723056,"kb_used_omap":1,"kb_used_meta":387966,"kb_avail":91634108,"statfs":{"total":95995035648,"available":93833326592,"internally_reserved":0,"allocated":1764409344,"data_stored":1763998316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":397277637},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,5,8],"upper_bound":512},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1757937664,"data_stored":1757937664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4023717888,"data_stored":4023713816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1349779456,"data_stored":1349779456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3538878464,"data_stored":3538878464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1537343488,"data_stored":1537343488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4021882880,"data_stored":4021878808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1348599808,"data_stored":1348599808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4054257664,"data_stored":4054253592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T19:00:23.639 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd pool set unique_pool_2 pgp_num 16 2024-09-20T19:00:23.674 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:23.705 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:23.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:23.707 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:23.807 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:00:23.839 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:23.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:23.840 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:00:23.941 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:24.002 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:24.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:24.003 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:24.104 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:24.133 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:24.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:24.134 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:24.151 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:24 smithi083 ceph-mon[29421]: pgmap v522: 148 pgs: 148 active+clean; 6.3 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 760 B/s rd, 0 B/s wr, 1.33k op/s 2024-09-20T19:00:24.152 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:24 smithi083 ceph-mon[36260]: pgmap v522: 148 pgs: 148 active+clean; 6.3 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 760 B/s rd, 0 B/s wr, 1.33k op/s 2024-09-20T19:00:24.165 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:00:24.224 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:00:24.224 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:00:24.234 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:00:24.265 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:00:24.265 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:00:24.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:24.290 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:24.291 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:00:24.299 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:00:24.300 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:00:24.332 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:00:24.332 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:00:24.365 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:00:24.365 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:00:24.369 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T19:00:24.391 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:24.396 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:00:24.396 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:00:24.419 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:24.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:24.420 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:24.429 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:00:24.429 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:00:24.462 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:00:24.462 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:00:24.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:24 smithi142 ceph-mon[32508]: pgmap v522: 148 pgs: 148 active+clean; 6.3 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 760 B/s rd, 0 B/s wr, 1.33k op/s 2024-09-20T19:00:24.522 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:24.549 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:24.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:24.550 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:24.650 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:24.679 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:24.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:24.680 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:24.781 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:24.821 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:24.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:24.822 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:24.922 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:24.959 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:24.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:24.960 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:25.061 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:25.091 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:25.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:25.092 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:25.192 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:25.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:25.230 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:25.230 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:25.330 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:25.367 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:25.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:25.368 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:25.399 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:25 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1277908631' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:00:25.399 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:25 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1277908631' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:00:25.469 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:00:25.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:25.503 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:25.504 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:00:25.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:25 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1277908631' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:00:25.539 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:00:25.559 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T19:00:25.559 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:00:25.604 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:25.642 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:25.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:25.643 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:25.744 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:00:25.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:25.778 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:25.779 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:00:25.779 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:00:25.879 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:00:25.911 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:25.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:25.912 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:00:26.013 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:26.061 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:26.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:26.062 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:26.162 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:00:26.203 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:26.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:26.204 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:00:26.305 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:00:26.336 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:26.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:26.337 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:00:26.438 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:26.472 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:26.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:26.473 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:26.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:26 smithi142 ceph-mon[32508]: pgmap v523: 148 pgs: 148 active+clean; 6.3 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 854 op/s 2024-09-20T19:00:26.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:26 smithi083 ceph-mon[29421]: pgmap v523: 148 pgs: 148 active+clean; 6.3 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 854 op/s 2024-09-20T19:00:26.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:26 smithi083 ceph-mon[36260]: pgmap v523: 148 pgs: 148 active+clean; 6.3 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 854 op/s 2024-09-20T19:00:26.574 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:26.610 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:26.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:26.611 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:26.712 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:26.743 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:26.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:26.744 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:26.844 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:26.877 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:26.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:26.879 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:26.979 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:27.015 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:27.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:27.017 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:27.118 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:27.134 INFO:teuthology.orchestra.run.smithi083.stderr:set pool 9 pgp_num to 16 2024-09-20T19:00:27.160 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:27.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:27.161 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:27.189 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T19:00:27.262 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:27.266 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:00:26 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:00:27.266 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:27 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3369372437' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"} : dispatch 2024-09-20T19:00:27.266 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:27 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"} : dispatch 2024-09-20T19:00:27.312 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:27.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:27.313 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:27.414 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:00:27.449 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:27.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:27.450 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:00:27.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:27 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3369372437' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"} : dispatch 2024-09-20T19:00:27.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:27 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"} : dispatch 2024-09-20T19:00:27.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:27 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3369372437' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"} : dispatch 2024-09-20T19:00:27.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:27 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"} : dispatch 2024-09-20T19:00:27.551 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:27.581 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:27.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:27.582 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:27.683 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:27.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:27.714 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:27.714 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:27.814 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:00:27.851 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:27.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:27.852 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:00:27.953 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:27.985 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:27.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:27.986 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:28.087 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:28.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:28.122 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:28.123 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:28.223 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:28.255 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:28.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:28.256 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:28.288 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:28 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:00:28.288 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:28 smithi083 ceph-mon[29421]: pgmap v524: 148 pgs: 148 active+clean; 6.0 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.27k op/s 2024-09-20T19:00:28.288 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:28 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"}]': finished 2024-09-20T19:00:28.288 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:28 smithi083 ceph-mon[29421]: osdmap e290: 8 total, 8 up, 8 in 2024-09-20T19:00:28.289 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:28 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:00:28.289 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:28 smithi083 ceph-mon[36260]: pgmap v524: 148 pgs: 148 active+clean; 6.0 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.27k op/s 2024-09-20T19:00:28.289 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:28 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"}]': finished 2024-09-20T19:00:28.289 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:28 smithi083 ceph-mon[36260]: osdmap e290: 8 total, 8 up, 8 in 2024-09-20T19:00:28.356 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:00:28.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:28.405 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:28.406 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:00:28.506 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:00:28.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:28 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:00:28.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:28 smithi142 ceph-mon[32508]: pgmap v524: 148 pgs: 148 active+clean; 6.0 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.27k op/s 2024-09-20T19:00:28.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:28 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"}]': finished 2024-09-20T19:00:28.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:28 smithi142 ceph-mon[32508]: osdmap e290: 8 total, 8 up, 8 in 2024-09-20T19:00:28.542 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:28.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:28.543 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:00:28.644 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:28.679 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:28.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:28.684 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:28.785 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:28.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:28.818 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:28.818 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:28.919 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:00:28.955 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:28.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:28.956 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:00:29.057 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:00:29.087 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:29.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:29.088 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:00:29.136 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-20T19:00:29.188 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:29.219 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:29.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:29.220 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:29.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:29 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1792662787' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:00:29.296 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:29 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:00:29.296 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:29 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1792662787' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:00:29.296 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:29 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:00:29.320 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:29.357 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:29.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:29.358 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:29.459 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:29.466 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_blocked_ops 2024-09-20T19:00:29.492 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:29.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:29.493 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:29.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:29 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1792662787' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:00:29.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:29 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:00:29.594 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:00:29.640 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:29.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:29.641 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:00:29.742 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:29.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:29.791 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:29.791 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:29.893 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:29.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:29.929 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:29.929 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:30.030 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:30.062 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:30.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:30.063 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:30.164 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:00:30.205 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:00:30.232 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:30.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:30.235 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:00:30.277 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:00:30.336 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:30.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:30.372 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:30.372 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:30.408 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:30 smithi083 ceph-mon[29421]: pgmap v526: 148 pgs: 148 active+clean; 5.9 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.41k op/s 2024-09-20T19:00:30.408 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:30 smithi083 ceph-mon[29421]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:00:30.408 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:30 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T19:00:30.408 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:30 smithi083 ceph-mon[29421]: osdmap e291: 8 total, 8 up, 8 in 2024-09-20T19:00:30.408 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:30 smithi083 ceph-mon[36260]: pgmap v526: 148 pgs: 148 active+clean; 5.9 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.41k op/s 2024-09-20T19:00:30.408 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:30 smithi083 ceph-mon[36260]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:00:30.408 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:30 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T19:00:30.408 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:30 smithi083 ceph-mon[36260]: osdmap e291: 8 total, 8 up, 8 in 2024-09-20T19:00:30.473 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:30.509 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:30.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:30.510 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:30.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:30 smithi142 ceph-mon[32508]: pgmap v526: 148 pgs: 148 active+clean; 5.9 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.41k op/s 2024-09-20T19:00:30.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:30 smithi142 ceph-mon[32508]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:00:30.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:30 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T19:00:30.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:30 smithi142 ceph-mon[32508]: osdmap e291: 8 total, 8 up, 8 in 2024-09-20T19:00:30.611 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:00:30.647 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:30.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:30.648 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:00:30.749 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:30.784 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:30.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:30.785 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:30.886 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:30.921 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:30.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:30.922 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:30.932 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:00:30 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:00:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:00:31.022 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:31.057 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:31.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:31.058 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:31.116 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:00:31.116 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:00:31.149 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:00:31.149 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:00:31.159 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:31.188 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:00:31.188 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:00:31.188 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:31 smithi083 ceph-mon[29421]: 3.6 scrub starts 2024-09-20T19:00:31.188 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:31 smithi083 ceph-mon[29421]: 3.6 scrub ok 2024-09-20T19:00:31.188 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:31 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:00:31.188 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:31 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2283805646' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:00:31.189 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:31 smithi083 ceph-mon[36260]: 3.6 scrub starts 2024-09-20T19:00:31.189 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:31 smithi083 ceph-mon[36260]: 3.6 scrub ok 2024-09-20T19:00:31.189 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:31 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:00:31.189 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:31 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2283805646' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:00:31.190 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:31.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:31.191 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:31.226 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:00:31.226 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:00:31.265 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:00:31.266 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:00:31.292 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:00:31.298 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:00:31.298 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:00:31.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:31.323 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:31.324 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:00:31.335 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:00:31.335 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:00:31.342 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T19:00:31.342 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-20T19:00:31.342 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-20T19:00:31.342 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-20T19:00:31.342 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T19:00:31.373 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:00:31.374 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:00:31.424 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:00:31.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:31.461 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:31.462 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:00:31.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:31 smithi142 ceph-mon[32508]: 3.6 scrub starts 2024-09-20T19:00:31.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:31 smithi142 ceph-mon[32508]: 3.6 scrub ok 2024-09-20T19:00:31.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:31 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:00:31.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:31 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2283805646' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:00:31.562 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:31.599 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:31.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:31.600 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:31.701 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:00:31.736 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:31.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:31.737 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:00:31.838 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:00:31.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:31.875 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:31.876 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:00:31.976 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:32.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:32.012 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:32.012 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:32.113 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:32.147 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:32.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:32.148 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:32.197 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T19:00:32.248 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:00:32.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:32 smithi083 ceph-mon[29421]: pgmap v528: 148 pgs: 148 active+clean; 5.8 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 973 op/s 2024-09-20T19:00:32.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:32 smithi083 ceph-mon[29421]: 5.d scrub starts 2024-09-20T19:00:32.296 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:32 smithi083 ceph-mon[29421]: 5.d scrub ok 2024-09-20T19:00:32.296 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:32 smithi083 ceph-mon[29421]: 3.3 scrub starts 2024-09-20T19:00:32.296 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:32 smithi083 ceph-mon[29421]: 3.3 scrub ok 2024-09-20T19:00:32.296 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:32 smithi083 ceph-mon[36260]: pgmap v528: 148 pgs: 148 active+clean; 5.8 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 973 op/s 2024-09-20T19:00:32.296 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:32 smithi083 ceph-mon[36260]: 5.d scrub starts 2024-09-20T19:00:32.296 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:32 smithi083 ceph-mon[36260]: 5.d scrub ok 2024-09-20T19:00:32.296 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:32 smithi083 ceph-mon[36260]: 3.3 scrub starts 2024-09-20T19:00:32.296 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:32 smithi083 ceph-mon[36260]: 3.3 scrub ok 2024-09-20T19:00:32.305 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:32.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:32.306 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:00:32.407 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:32.443 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:32.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:32.444 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:32.469 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_historic_ops 2024-09-20T19:00:32.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:32 smithi142 ceph-mon[32508]: pgmap v528: 148 pgs: 148 active+clean; 5.8 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 973 op/s 2024-09-20T19:00:32.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:32 smithi142 ceph-mon[32508]: 5.d scrub starts 2024-09-20T19:00:32.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:32 smithi142 ceph-mon[32508]: 5.d scrub ok 2024-09-20T19:00:32.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:32 smithi142 ceph-mon[32508]: 3.3 scrub starts 2024-09-20T19:00:32.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:32 smithi142 ceph-mon[32508]: 3.3 scrub ok 2024-09-20T19:00:32.545 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:00:32.576 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:32.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:32.577 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:00:32.678 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:32.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:32.716 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:32.717 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:32.817 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:32.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:32.868 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:32.869 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:32.970 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:00:32.998 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:32.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:32.999 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:00:33.017 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:00:33.100 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:33.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:33.142 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:33.142 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:33.243 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:33.263 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:00:33.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:33.282 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:33.282 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:33.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:33 smithi083 ceph-mon[29421]: 5.1f scrub starts 2024-09-20T19:00:33.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:33 smithi083 ceph-mon[29421]: 5.1f scrub ok 2024-09-20T19:00:33.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:33 smithi083 ceph-mon[29421]: 3.8 scrub starts 2024-09-20T19:00:33.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:33 smithi083 ceph-mon[29421]: 3.8 scrub ok 2024-09-20T19:00:33.297 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:33 smithi083 ceph-mon[36260]: 5.1f scrub starts 2024-09-20T19:00:33.298 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:33 smithi083 ceph-mon[36260]: 5.1f scrub ok 2024-09-20T19:00:33.298 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:33 smithi083 ceph-mon[36260]: 3.8 scrub starts 2024-09-20T19:00:33.298 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:33 smithi083 ceph-mon[36260]: 3.8 scrub ok 2024-09-20T19:00:33.382 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:33.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:33.418 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:33.419 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:33.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:33 smithi142 ceph-mon[32508]: 5.1f scrub starts 2024-09-20T19:00:33.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:33 smithi142 ceph-mon[32508]: 5.1f scrub ok 2024-09-20T19:00:33.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:33 smithi142 ceph-mon[32508]: 3.8 scrub starts 2024-09-20T19:00:33.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:33 smithi142 ceph-mon[32508]: 3.8 scrub ok 2024-09-20T19:00:33.520 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:33.557 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:33.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:33.558 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:33.658 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:33.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:33.692 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:33.692 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:33.793 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:33.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:33.825 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:33.825 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:33.926 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:33.963 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:33.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:33.964 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:34.065 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:34.066 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 7, 6] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-20T19:00:34.066 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T19:00:34.067 INFO:tasks.thrashosds.thrasher:Killing osd 4, live_osds are [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-20T19:00:34.067 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-20T19:00:34.067 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@osd.4 2024-09-20T19:00:34.105 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:34.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:34.106 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:34.207 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:34.237 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:34.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:34.238 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:34.339 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:34.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:34.376 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:34.376 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:34.477 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:34.502 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:34 smithi142 ceph-mon[32508]: pgmap v529: 148 pgs: 148 active+clean; 5.5 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.64k op/s 2024-09-20T19:00:34.503 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:00:34 smithi142 systemd[1]: Stopping Ceph osd.4 for 1ed5a50a-7780-11ef-baf5-efdc52797490... 2024-09-20T19:00:34.506 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:34.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:34.508 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:34.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:34 smithi083 ceph-mon[29421]: pgmap v529: 148 pgs: 148 active+clean; 5.5 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.64k op/s 2024-09-20T19:00:34.508 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:34 smithi083 ceph-mon[36260]: pgmap v529: 148 pgs: 148 active+clean; 5.5 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.64k op/s 2024-09-20T19:00:34.609 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:34.643 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:34.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:34.645 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:34.746 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:34.766 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:00:34 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4[36598]: 2024-09-20T19:00:34.497+0000 7f936b479640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-20T19:00:34.766 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:00:34 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4[36598]: 2024-09-20T19:00:34.497+0000 7f936b479640 -1 osd.4 291 *** Got signal Terminated *** 2024-09-20T19:00:34.766 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:00:34 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4[36598]: 2024-09-20T19:00:34.497+0000 7f936b479640 -1 osd.4 291 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-20T19:00:34.773 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:34.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:34.774 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:34.875 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:34.908 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:34.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:34.909 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:34.957 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T19:00:35.010 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:35.035 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:35.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:35.036 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:35.137 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:35.196 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:35.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:35.198 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:35.270 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:35 smithi083 ceph-mon[29421]: 4.10 scrub starts 2024-09-20T19:00:35.270 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:35 smithi083 ceph-mon[29421]: 4.10 scrub ok 2024-09-20T19:00:35.270 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:35 smithi083 ceph-mon[29421]: osd.4 marked itself down and dead 2024-09-20T19:00:35.270 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:35 smithi083 ceph-mon[36260]: 4.10 scrub starts 2024-09-20T19:00:35.270 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:35 smithi083 ceph-mon[36260]: 4.10 scrub ok 2024-09-20T19:00:35.270 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:35 smithi083 ceph-mon[36260]: osd.4 marked itself down and dead 2024-09-20T19:00:35.299 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:00:35.312 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T19:00:35.312 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-20T19:00:35.312 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-20T19:00:35.312 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-20T19:00:35.312 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.313 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:128076 8.3 8:ca202ecf:::benchmark_data_smithi083_97422_object11985:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e203)", 2024-09-20T19:00:35.313 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:24.802138+0000", 2024-09-20T19:00:35.313 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 250.499775914, 2024-09-20T19:00:35.313 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.90051086899999999, 2024-09-20T19:00:35.313 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:35.313 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:35.313 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:35.313 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:35.313 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T19:00:35.313 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T19:00:35.313 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 128076 2024-09-20T19:00:35.313 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.313 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:35.314 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.314 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:35.314 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.802138+0000", 2024-09-20T19:00:35.314 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:35.314 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.314 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.314 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:35.314 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.802138+0000", 2024-09-20T19:00:35.314 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:35.314 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.314 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.314 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:35.314 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.802139+0000", 2024-09-20T19:00:35.315 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8199999999999999e-07 2024-09-20T19:00:35.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.315 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.315 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:35.315 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.802143+0000", 2024-09-20T19:00:35.315 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0500000000000002e-06 2024-09-20T19:00:35.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.315 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.315 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:35.315 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.802151+0000", 2024-09-20T19:00:35.315 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8860000000000004e-06 2024-09-20T19:00:35.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.316 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.316 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:35.316 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.802157+0000", 2024-09-20T19:00:35.316 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.105e-06 2024-09-20T19:00:35.316 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.316 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.316 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.316 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.802209+0000", 2024-09-20T19:00:35.316 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1229e-05 2024-09-20T19:00:35.316 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.316 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.316 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:00:35.316 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.802215+0000", 2024-09-20T19:00:35.316 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8700000000000003e-06 2024-09-20T19:00:35.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.317 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.317 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.317 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.694932+0000", 2024-09-20T19:00:35.317 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.89271625700000001 2024-09-20T19:00:35.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.317 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.317 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:35.317 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.695043+0000", 2024-09-20T19:00:35.317 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000111789 2024-09-20T19:00:35.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.317 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.317 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,5", 2024-09-20T19:00:35.318 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.695170+0000", 2024-09-20T19:00:35.318 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000126255 2024-09-20T19:00:35.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.318 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.318 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:35.318 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.700200+0000", 2024-09-20T19:00:35.318 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0050300939999999997 2024-09-20T19:00:35.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.318 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.318 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:35.318 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.700261+0000", 2024-09-20T19:00:35.318 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1340000000000006e-05 2024-09-20T19:00:35.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.318 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.319 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:00:35.319 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.702559+0000", 2024-09-20T19:00:35.319 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002297707 2024-09-20T19:00:35.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.319 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.319 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:35.319 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.702606+0000", 2024-09-20T19:00:35.319 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7037999999999999e-05 2024-09-20T19:00:35.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.319 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.319 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:35.319 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.702649+0000", 2024-09-20T19:00:35.319 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3467000000000002e-05 2024-09-20T19:00:35.320 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:35.320 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:35.320 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:35.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.320 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.320 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:128090 8.3 8:c10076ee:::benchmark_data_smithi083_97422_object11999:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e203)", 2024-09-20T19:00:35.320 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:24.815492+0000", 2024-09-20T19:00:35.320 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 250.48642266600001, 2024-09-20T19:00:35.320 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88975575799999995, 2024-09-20T19:00:35.320 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:35.320 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:35.320 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:35.320 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:35.320 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T19:00:35.321 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T19:00:35.321 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 128090 2024-09-20T19:00:35.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.321 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:35.321 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.321 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:35.321 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.815492+0000", 2024-09-20T19:00:35.321 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:35.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.321 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.321 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:35.321 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.815492+0000", 2024-09-20T19:00:35.321 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:35.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.322 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.322 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:35.322 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.815493+0000", 2024-09-20T19:00:35.322 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2920000000000001e-06 2024-09-20T19:00:35.322 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.322 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.322 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:35.322 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.815497+0000", 2024-09-20T19:00:35.322 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8349999999999997e-06 2024-09-20T19:00:35.322 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.322 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.322 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:35.322 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.815505+0000", 2024-09-20T19:00:35.323 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2830000000000001e-06 2024-09-20T19:00:35.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.323 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.323 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:35.323 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.815512+0000", 2024-09-20T19:00:35.323 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8499999999999996e-06 2024-09-20T19:00:35.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.323 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.323 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.323 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.815561+0000", 2024-09-20T19:00:35.323 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8566000000000002e-05 2024-09-20T19:00:35.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.323 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.323 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:00:35.324 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:24.815568+0000", 2024-09-20T19:00:35.324 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.746e-06 2024-09-20T19:00:35.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.324 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.324 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.324 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.697779+0000", 2024-09-20T19:00:35.324 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88221116799999999 2024-09-20T19:00:35.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.324 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.324 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:35.324 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.697904+0000", 2024-09-20T19:00:35.324 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012453900000000001 2024-09-20T19:00:35.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.325 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.325 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,5", 2024-09-20T19:00:35.325 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.698026+0000", 2024-09-20T19:00:35.325 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121783 2024-09-20T19:00:35.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.325 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.325 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:35.325 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.705094+0000", 2024-09-20T19:00:35.325 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.007068146 2024-09-20T19:00:35.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.325 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.325 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:35.325 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.705140+0000", 2024-09-20T19:00:35.325 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6332000000000003e-05 2024-09-20T19:00:35.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.326 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.326 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:00:35.326 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.705154+0000", 2024-09-20T19:00:35.326 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3516000000000001e-05 2024-09-20T19:00:35.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.326 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.326 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:35.326 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.705203+0000", 2024-09-20T19:00:35.326 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.918e-05 2024-09-20T19:00:35.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.326 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.326 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:35.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:35.328 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:25.705247+0000", 2024-09-20T19:00:35.328 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4521999999999998e-05 2024-09-20T19:00:35.328 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:35.328 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:35.328 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:35.328 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.328 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.328 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157429 8.7 8:ee8a5b93:::benchmark_data_smithi083_97422_object41338:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e212)", 2024-09-20T19:00:35.328 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.048178+0000", 2024-09-20T19:00:35.328 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 216.253736301, 2024-09-20T19:00:35.328 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.034527355, 2024-09-20T19:00:35.328 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:35.329 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:35.329 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-20T19:00:35.329 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:35.329 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T19:00:35.329 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T19:00:35.329 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157429 2024-09-20T19:00:35.329 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.329 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:35.329 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.329 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:35.329 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.048178+0000", 2024-09-20T19:00:35.329 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:35.329 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.330 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.330 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:35.330 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.048178+0000", 2024-09-20T19:00:35.330 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:35.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.330 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.330 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:35.330 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.048180+0000", 2024-09-20T19:00:35.330 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.618e-06 2024-09-20T19:00:35.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.330 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.330 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:35.330 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.048187+0000", 2024-09-20T19:00:35.331 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8240000000000002e-06 2024-09-20T19:00:35.331 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.331 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.331 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:35.331 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.048205+0000", 2024-09-20T19:00:35.331 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8717000000000001e-05 2024-09-20T19:00:35.331 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.331 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.331 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:35.331 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.048215+0000", 2024-09-20T19:00:35.331 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.022e-05 2024-09-20T19:00:35.331 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.331 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.332 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.332 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.048294+0000", 2024-09-20T19:00:35.332 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8088000000000002e-05 2024-09-20T19:00:35.332 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.332 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.332 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:35.332 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.048409+0000", 2024-09-20T19:00:35.332 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115146 2024-09-20T19:00:35.332 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.332 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.332 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3,6", 2024-09-20T19:00:35.332 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.048517+0000", 2024-09-20T19:00:35.332 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010874099999999999 2024-09-20T19:00:35.332 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.333 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:35.333 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.059282+0000", 2024-09-20T19:00:35.333 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010764078 2024-09-20T19:00:35.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.333 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:00:35.333 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.059297+0000", 2024-09-20T19:00:35.333 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5773e-05 2024-09-20T19:00:35.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.333 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:35.333 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.059343+0000", 2024-09-20T19:00:35.334 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5816000000000003e-05 2024-09-20T19:00:35.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.334 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.334 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.334 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.070249+0000", 2024-09-20T19:00:35.334 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010906381999999999 2024-09-20T19:00:35.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.334 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.334 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:00:35.334 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.070260+0000", 2024-09-20T19:00:35.334 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0404000000000001e-05 2024-09-20T19:00:35.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.334 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.334 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.335 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.082592+0000", 2024-09-20T19:00:35.335 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0123322640000001 2024-09-20T19:00:35.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.335 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.335 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:35.335 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.082705+0000", 2024-09-20T19:00:35.335 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011328399999999999 2024-09-20T19:00:35.335 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:35.335 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:35.335 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:35.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.335 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.335 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157435 8.7 8:e8101dbf:::benchmark_data_smithi083_97422_object41344:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e212)", 2024-09-20T19:00:35.335 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.053885+0000", 2024-09-20T19:00:35.336 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 216.24802907099999, 2024-09-20T19:00:35.336 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0294991419999999, 2024-09-20T19:00:35.336 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:35.336 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:35.336 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-20T19:00:35.336 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:35.336 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T19:00:35.336 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T19:00:35.336 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157435 2024-09-20T19:00:35.336 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.336 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:35.336 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.336 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:35.336 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053885+0000", 2024-09-20T19:00:35.336 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:35.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.337 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.337 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:35.337 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053885+0000", 2024-09-20T19:00:35.337 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:35.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.337 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.337 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:35.337 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053887+0000", 2024-09-20T19:00:35.337 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.68e-06 2024-09-20T19:00:35.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.337 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.337 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:35.337 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053894+0000", 2024-09-20T19:00:35.337 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5939999999999999e-06 2024-09-20T19:00:35.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.338 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.338 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:35.338 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053907+0000", 2024-09-20T19:00:35.338 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3599e-05 2024-09-20T19:00:35.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.338 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.338 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:35.338 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053916+0000", 2024-09-20T19:00:35.338 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2029999999999998e-06 2024-09-20T19:00:35.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.338 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.338 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.338 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.054004+0000", 2024-09-20T19:00:35.338 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8026000000000001e-05 2024-09-20T19:00:35.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.339 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.339 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:35.339 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.054123+0000", 2024-09-20T19:00:35.339 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000119046 2024-09-20T19:00:35.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.339 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.339 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3,6", 2024-09-20T19:00:35.339 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.056711+0000", 2024-09-20T19:00:35.339 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0025878049999999999 2024-09-20T19:00:35.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.339 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.339 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:00:35.340 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.059354+0000", 2024-09-20T19:00:35.340 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0026431969999999999 2024-09-20T19:00:35.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.340 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.340 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.070287+0000", 2024-09-20T19:00:35.340 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010932261 2024-09-20T19:00:35.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.340 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:00:35.340 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.070294+0000", 2024-09-20T19:00:35.340 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0029999999999997e-06 2024-09-20T19:00:35.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.340 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.341 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.083330+0000", 2024-09-20T19:00:35.341 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.013036592 2024-09-20T19:00:35.341 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.341 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.341 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:35.341 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.083384+0000", 2024-09-20T19:00:35.341 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4135999999999998e-05 2024-09-20T19:00:35.341 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:35.341 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:35.341 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:35.341 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.341 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.341 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157436 8.7 8:e2843ea0:::benchmark_data_smithi083_97422_object41345:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e212)", 2024-09-20T19:00:35.341 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.053939+0000", 2024-09-20T19:00:35.342 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 216.24797554, 2024-09-20T19:00:35.342 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.029517123, 2024-09-20T19:00:35.342 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:35.342 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:35.342 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-20T19:00:35.342 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:35.342 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T19:00:35.342 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T19:00:35.342 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157436 2024-09-20T19:00:35.342 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.342 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:35.342 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.342 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:35.343 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053939+0000", 2024-09-20T19:00:35.343 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:35.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.343 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.343 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:35.343 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053939+0000", 2024-09-20T19:00:35.343 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:35.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.343 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.343 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:35.343 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053939+0000", 2024-09-20T19:00:35.343 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9100000000000004e-07 2024-09-20T19:00:35.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.344 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.344 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:35.344 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053942+0000", 2024-09-20T19:00:35.344 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7980000000000001e-06 2024-09-20T19:00:35.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.344 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.344 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:35.344 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053949+0000", 2024-09-20T19:00:35.344 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8580000000000002e-06 2024-09-20T19:00:35.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.344 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.344 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:35.344 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053955+0000", 2024-09-20T19:00:35.344 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1449999999999996e-06 2024-09-20T19:00:35.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.345 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.345 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.345 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.060057+0000", 2024-09-20T19:00:35.345 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0061021729999999998 2024-09-20T19:00:35.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.345 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.345 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:35.345 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.060177+0000", 2024-09-20T19:00:35.345 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000120013 2024-09-20T19:00:35.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.345 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.345 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3,6", 2024-09-20T19:00:35.345 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.060294+0000", 2024-09-20T19:00:35.345 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000117056 2024-09-20T19:00:35.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.346 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.346 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.346 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.070317+0000", 2024-09-20T19:00:35.346 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010022778 2024-09-20T19:00:35.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.346 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.346 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:00:35.346 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.070324+0000", 2024-09-20T19:00:35.346 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4300000000000003e-06 2024-09-20T19:00:35.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.346 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.346 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.347 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.083413+0000", 2024-09-20T19:00:35.347 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0130890960000001 2024-09-20T19:00:35.347 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.347 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.347 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:35.347 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.083456+0000", 2024-09-20T19:00:35.347 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3185000000000001e-05 2024-09-20T19:00:35.347 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:35.347 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:35.347 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:35.347 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.347 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.347 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157449 8.7 8:e0d43d9e:::benchmark_data_smithi083_97422_object41358:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e213)", 2024-09-20T19:00:35.348 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.075612+0000", 2024-09-20T19:00:35.348 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 216.226302691, 2024-09-20T19:00:35.348 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.016308008, 2024-09-20T19:00:35.348 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:35.348 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:35.348 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:35.348 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:35.348 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T19:00:35.348 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T19:00:35.348 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157449 2024-09-20T19:00:35.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.348 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:35.348 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.349 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:35.349 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.075612+0000", 2024-09-20T19:00:35.349 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:35.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.349 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.349 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:35.349 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.075612+0000", 2024-09-20T19:00:35.349 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:35.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.349 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.349 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:35.349 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.075613+0000", 2024-09-20T19:00:35.349 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1570000000000001e-06 2024-09-20T19:00:35.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.350 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.350 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:35.350 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.075618+0000", 2024-09-20T19:00:35.350 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7899999999999999e-06 2024-09-20T19:00:35.350 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.350 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.350 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:35.350 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.075627+0000", 2024-09-20T19:00:35.350 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2569999999999996e-06 2024-09-20T19:00:35.350 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.350 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.350 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:35.350 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.075662+0000", 2024-09-20T19:00:35.351 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5439999999999999e-05 2024-09-20T19:00:35.351 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.351 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.351 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.351 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.078479+0000", 2024-09-20T19:00:35.351 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0028168910000000002 2024-09-20T19:00:35.351 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.351 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.351 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:00:35.351 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.078488+0000", 2024-09-20T19:00:35.351 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4069999999999993e-06 2024-09-20T19:00:35.351 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.351 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.351 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.351 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.083476+0000", 2024-09-20T19:00:35.352 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.004987893 2024-09-20T19:00:35.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.352 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.352 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:35.352 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.083567+0000", 2024-09-20T19:00:35.352 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1598e-05 2024-09-20T19:00:35.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.352 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.352 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3", 2024-09-20T19:00:35.352 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.083675+0000", 2024-09-20T19:00:35.352 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000107701 2024-09-20T19:00:35.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.352 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.353 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:35.353 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.091482+0000", 2024-09-20T19:00:35.353 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0078073639999999998 2024-09-20T19:00:35.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.353 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.353 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:00:35.353 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.091668+0000", 2024-09-20T19:00:35.353 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018606100000000001 2024-09-20T19:00:35.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.353 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.353 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:35.353 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.091718+0000", 2024-09-20T19:00:35.353 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9830000000000002e-05 2024-09-20T19:00:35.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.353 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.354 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:35.354 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.091878+0000", 2024-09-20T19:00:35.354 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000160166 2024-09-20T19:00:35.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.354 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.354 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:35.354 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.091920+0000", 2024-09-20T19:00:35.354 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1452999999999998e-05 2024-09-20T19:00:35.354 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:35.354 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:35.354 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:35.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.354 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.354 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157451 8.7 8:ea555b5d:::benchmark_data_smithi083_97422_object41360:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e213)", 2024-09-20T19:00:35.355 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.080725+0000", 2024-09-20T19:00:35.355 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 216.221189724, 2024-09-20T19:00:35.355 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0181950740000001, 2024-09-20T19:00:35.355 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:35.355 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:35.355 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:35.355 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:35.355 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T19:00:35.355 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T19:00:35.355 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157451 2024-09-20T19:00:35.355 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.355 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:35.355 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.355 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:35.356 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.080725+0000", 2024-09-20T19:00:35.356 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:35.356 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.356 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.356 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:35.356 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.080725+0000", 2024-09-20T19:00:35.356 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:35.356 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.356 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.356 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:35.356 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.080726+0000", 2024-09-20T19:00:35.356 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3459999999999999e-06 2024-09-20T19:00:35.356 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.356 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.356 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:35.357 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.080731+0000", 2024-09-20T19:00:35.357 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3619999999999997e-06 2024-09-20T19:00:35.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.357 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.357 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:35.357 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.080742+0000", 2024-09-20T19:00:35.357 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0302000000000001e-05 2024-09-20T19:00:35.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.357 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.357 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:35.357 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.080757+0000", 2024-09-20T19:00:35.357 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4955e-05 2024-09-20T19:00:35.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.358 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.358 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.358 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.080819+0000", 2024-09-20T19:00:35.358 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2310999999999993e-05 2024-09-20T19:00:35.358 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.358 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.358 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:00:35.358 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.080826+0000", 2024-09-20T19:00:35.358 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1069999999999999e-06 2024-09-20T19:00:35.358 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.358 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.358 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.358 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.085855+0000", 2024-09-20T19:00:35.358 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0050288679999999 2024-09-20T19:00:35.358 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.359 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.359 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:35.359 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.085947+0000", 2024-09-20T19:00:35.359 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1911999999999997e-05 2024-09-20T19:00:35.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.359 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.359 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3", 2024-09-20T19:00:35.359 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.086042+0000", 2024-09-20T19:00:35.359 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5234000000000003e-05 2024-09-20T19:00:35.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.359 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.359 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:00:35.359 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.091673+0000", 2024-09-20T19:00:35.359 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0056306079999999996 2024-09-20T19:00:35.360 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.360 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.360 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:35.360 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.098793+0000", 2024-09-20T19:00:35.360 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.007120637 2024-09-20T19:00:35.360 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.360 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.360 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:35.360 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.098828+0000", 2024-09-20T19:00:35.360 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4405e-05 2024-09-20T19:00:35.360 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.360 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.360 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:35.361 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.098882+0000", 2024-09-20T19:00:35.361 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4481000000000002e-05 2024-09-20T19:00:35.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.361 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.361 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:35.361 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.098920+0000", 2024-09-20T19:00:35.361 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7546000000000003e-05 2024-09-20T19:00:35.361 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:35.361 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:35.361 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:35.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.361 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.361 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157481 8.7 8:ebfa0664:::benchmark_data_smithi083_97422_object41390:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e213)", 2024-09-20T19:00:35.361 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.116650+0000", 2024-09-20T19:00:35.362 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 216.185264211, 2024-09-20T19:00:35.362 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.98244276600000002, 2024-09-20T19:00:35.362 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:35.362 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:35.362 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:35.362 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:35.362 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T19:00:35.362 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T19:00:35.362 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157481 2024-09-20T19:00:35.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.362 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:35.362 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.362 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:35.362 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.116650+0000", 2024-09-20T19:00:35.363 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:35.363 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.363 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.363 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:35.363 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.116650+0000", 2024-09-20T19:00:35.363 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:35.363 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.363 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.363 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:35.363 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.116652+0000", 2024-09-20T19:00:35.363 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5680000000000001e-06 2024-09-20T19:00:35.363 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.363 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.363 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:35.364 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.116658+0000", 2024-09-20T19:00:35.364 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4309999999999999e-06 2024-09-20T19:00:35.364 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.364 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.364 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:35.364 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.116671+0000", 2024-09-20T19:00:35.364 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2384e-05 2024-09-20T19:00:35.364 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.364 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.364 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:35.364 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.116678+0000", 2024-09-20T19:00:35.364 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8690000000000004e-06 2024-09-20T19:00:35.364 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.364 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.365 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.365 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.116734+0000", 2024-09-20T19:00:35.365 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6041000000000001e-05 2024-09-20T19:00:35.365 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.365 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.365 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:00:35.365 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.116742+0000", 2024-09-20T19:00:35.365 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8189999999999992e-06 2024-09-20T19:00:35.365 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.365 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.365 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.365 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.088569+0000", 2024-09-20T19:00:35.365 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97182719100000003 2024-09-20T19:00:35.365 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.366 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.366 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:35.366 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.088666+0000", 2024-09-20T19:00:35.366 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6897000000000004e-05 2024-09-20T19:00:35.366 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.366 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.366 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3", 2024-09-20T19:00:35.366 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.088770+0000", 2024-09-20T19:00:35.366 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010320299999999999 2024-09-20T19:00:35.366 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.366 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.366 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:00:35.366 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.091677+0000", 2024-09-20T19:00:35.367 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0029076900000000001 2024-09-20T19:00:35.367 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.367 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.367 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:35.367 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.098961+0000", 2024-09-20T19:00:35.367 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0072835080000000002 2024-09-20T19:00:35.367 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.367 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.367 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:35.367 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.099000+0000", 2024-09-20T19:00:35.367 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.892e-05 2024-09-20T19:00:35.367 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.367 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.367 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:35.368 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.099055+0000", 2024-09-20T19:00:35.368 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5554999999999999e-05 2024-09-20T19:00:35.368 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.368 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.368 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:35.368 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.099093+0000", 2024-09-20T19:00:35.368 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.769e-05 2024-09-20T19:00:35.368 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:35.368 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:35.368 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:35.368 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.368 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.368 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157496 8.7 8:efb55f87:::benchmark_data_smithi083_97422_object41405:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e213)", 2024-09-20T19:00:35.368 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.130402+0000", 2024-09-20T19:00:35.368 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 216.17151249299999, 2024-09-20T19:00:35.369 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.99286820799999997, 2024-09-20T19:00:35.369 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:35.369 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:35.369 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:35.369 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:35.369 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T19:00:35.369 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:35.369 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:00:35.371 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T19:00:35.371 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157496 2024-09-20T19:00:35.371 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.371 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:35.371 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.371 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:35.371 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.130402+0000", 2024-09-20T19:00:35.371 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:35.371 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.372 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.372 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:35.372 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.130402+0000", 2024-09-20T19:00:35.372 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:35.372 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.372 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.372 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:35.372 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.130404+0000", 2024-09-20T19:00:35.372 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.748e-06 2024-09-20T19:00:35.372 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.372 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.372 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:35.372 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.130410+0000", 2024-09-20T19:00:35.373 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2720000000000003e-06 2024-09-20T19:00:35.373 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.373 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.373 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:35.373 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.130426+0000", 2024-09-20T19:00:35.373 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6557000000000001e-05 2024-09-20T19:00:35.373 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.373 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.373 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:35.373 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.130436+0000", 2024-09-20T19:00:35.373 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4320000000000005e-06 2024-09-20T19:00:35.373 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.373 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.373 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.373 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.130499+0000", 2024-09-20T19:00:35.374 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3149000000000001e-05 2024-09-20T19:00:35.374 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.374 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.374 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:00:35.374 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.130508+0000", 2024-09-20T19:00:35.374 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1619999999999997e-06 2024-09-20T19:00:35.374 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.374 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.374 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.374 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.091952+0000", 2024-09-20T19:00:35.374 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.96144401999999995 2024-09-20T19:00:35.374 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.374 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.374 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:35.375 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.092057+0000", 2024-09-20T19:00:35.375 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000104268 2024-09-20T19:00:35.375 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.375 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.375 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3", 2024-09-20T19:00:35.375 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.092163+0000", 2024-09-20T19:00:35.375 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106545 2024-09-20T19:00:35.375 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.375 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.375 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:35.375 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.099133+0000", 2024-09-20T19:00:35.375 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0069702990000000001 2024-09-20T19:00:35.375 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.375 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.375 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:35.376 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.104449+0000", 2024-09-20T19:00:35.376 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0053152920000000001 2024-09-20T19:00:35.376 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.376 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.376 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:00:35.376 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.123091+0000", 2024-09-20T19:00:35.376 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.018642757999999999 2024-09-20T19:00:35.376 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.376 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.376 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:35.376 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.123225+0000", 2024-09-20T19:00:35.376 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000133881 2024-09-20T19:00:35.376 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.376 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.376 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:35.376 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.123270+0000", 2024-09-20T19:00:35.377 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4824999999999999e-05 2024-09-20T19:00:35.377 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:35.377 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:35.377 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:35.377 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.377 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.377 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157528 8.7 8:e4768593:::benchmark_data_smithi083_97422_object41437:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e213)", 2024-09-20T19:00:35.377 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.165392+0000", 2024-09-20T19:00:35.377 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 216.136522195, 2024-09-20T19:00:35.377 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.958042846, 2024-09-20T19:00:35.377 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:35.377 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:35.377 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:35.377 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:35.378 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T19:00:35.378 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T19:00:35.378 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157528 2024-09-20T19:00:35.378 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.378 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:35.378 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.378 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:35.378 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.165392+0000", 2024-09-20T19:00:35.378 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:35.378 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.378 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.378 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:35.378 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.165392+0000", 2024-09-20T19:00:35.379 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:35.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.379 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.379 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:35.379 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.165394+0000", 2024-09-20T19:00:35.379 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.77e-06 2024-09-20T19:00:35.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.379 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.379 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:35.379 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.165400+0000", 2024-09-20T19:00:35.379 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2099999999999998e-06 2024-09-20T19:00:35.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.379 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.379 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:35.380 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.165419+0000", 2024-09-20T19:00:35.380 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9046999999999999e-05 2024-09-20T19:00:35.380 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.380 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.380 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:35.380 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.165428+0000", 2024-09-20T19:00:35.380 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1030000000000008e-06 2024-09-20T19:00:35.380 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.380 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.380 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.380 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.166013+0000", 2024-09-20T19:00:35.380 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000584868 2024-09-20T19:00:35.380 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.380 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.381 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:00:35.381 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.166022+0000", 2024-09-20T19:00:35.381 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7090000000000008e-06 2024-09-20T19:00:35.381 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.381 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.381 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.381 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.095256+0000", 2024-09-20T19:00:35.381 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.92923407700000005 2024-09-20T19:00:35.381 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.381 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.381 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:35.381 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.095349+0000", 2024-09-20T19:00:35.381 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2724000000000002e-05 2024-09-20T19:00:35.381 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.381 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.382 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3", 2024-09-20T19:00:35.382 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.095688+0000", 2024-09-20T19:00:35.382 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000339379 2024-09-20T19:00:35.382 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.382 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.382 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:35.382 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.104489+0000", 2024-09-20T19:00:35.382 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0088013359999999999 2024-09-20T19:00:35.382 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.382 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.382 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:35.382 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.106738+0000", 2024-09-20T19:00:35.382 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0022487029999999999 2024-09-20T19:00:35.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.383 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.383 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:00:35.383 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.123280+0000", 2024-09-20T19:00:35.383 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.016541613 2024-09-20T19:00:35.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.383 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.383 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:35.383 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.123398+0000", 2024-09-20T19:00:35.383 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011812299999999999 2024-09-20T19:00:35.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.383 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.383 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:35.383 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.123435+0000", 2024-09-20T19:00:35.384 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7184000000000002e-05 2024-09-20T19:00:35.384 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:35.384 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:35.384 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:35.384 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.384 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.384 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157535 8.7 8:ed2a8870:::benchmark_data_smithi083_97422_object41444:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e213)", 2024-09-20T19:00:35.384 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.173838+0000", 2024-09-20T19:00:35.384 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 216.128076583, 2024-09-20T19:00:35.384 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.95335319799999996, 2024-09-20T19:00:35.384 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:35.384 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:35.384 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:35.385 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:35.385 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T19:00:35.385 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T19:00:35.385 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157535 2024-09-20T19:00:35.385 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.385 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:35.385 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.385 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:35.385 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.173838+0000", 2024-09-20T19:00:35.385 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:35.385 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.385 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.385 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:35.385 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.173838+0000", 2024-09-20T19:00:35.386 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:35.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.386 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.386 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:35.386 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.173839+0000", 2024-09-20T19:00:35.386 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5340000000000001e-06 2024-09-20T19:00:35.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.386 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.386 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:35.386 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.173846+0000", 2024-09-20T19:00:35.386 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2910000000000004e-06 2024-09-20T19:00:35.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.386 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.386 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:35.387 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.173867+0000", 2024-09-20T19:00:35.387 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0928e-05 2024-09-20T19:00:35.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.387 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.387 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:35.387 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.173874+0000", 2024-09-20T19:00:35.387 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.6990000000000003e-06 2024-09-20T19:00:35.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.387 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.387 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.387 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.173935+0000", 2024-09-20T19:00:35.387 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0899000000000001e-05 2024-09-20T19:00:35.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.388 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.388 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:00:35.388 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.173944+0000", 2024-09-20T19:00:35.388 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5369999999999997e-06 2024-09-20T19:00:35.388 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.388 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.388 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.388 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.099161+0000", 2024-09-20T19:00:35.388 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.92521724299999997 2024-09-20T19:00:35.388 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.388 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.388 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:35.388 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.099277+0000", 2024-09-20T19:00:35.389 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115689 2024-09-20T19:00:35.389 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.389 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.389 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3", 2024-09-20T19:00:35.389 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.099390+0000", 2024-09-20T19:00:35.389 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011385199999999999 2024-09-20T19:00:35.389 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.389 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.389 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:35.389 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.106780+0000", 2024-09-20T19:00:35.389 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.007389367 2024-09-20T19:00:35.389 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.389 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.389 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:35.390 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.114134+0000", 2024-09-20T19:00:35.390 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0073541429999999996 2024-09-20T19:00:35.390 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.390 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.390 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:00:35.390 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.123441+0000", 2024-09-20T19:00:35.390 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0093066620000000003 2024-09-20T19:00:35.390 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.390 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.390 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:35.390 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.127133+0000", 2024-09-20T19:00:35.390 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0036927879999999998 2024-09-20T19:00:35.390 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.391 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.391 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:35.391 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.127191+0000", 2024-09-20T19:00:35.391 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7565999999999997e-05 2024-09-20T19:00:35.391 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:35.391 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:35.391 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:35.391 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.391 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.391 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157545 8.7 8:e3edf173:::benchmark_data_smithi083_97422_object41454:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e213)", 2024-09-20T19:00:35.391 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.188692+0000", 2024-09-20T19:00:35.391 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 216.11322197800001, 2024-09-20T19:00:35.391 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.93865532500000004, 2024-09-20T19:00:35.391 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:35.391 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:35.392 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:35.392 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:35.392 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T19:00:35.392 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T19:00:35.392 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157545 2024-09-20T19:00:35.392 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.392 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:35.392 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.392 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:35.392 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.188692+0000", 2024-09-20T19:00:35.392 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:35.392 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.392 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.392 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:35.393 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.188692+0000", 2024-09-20T19:00:35.393 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:35.393 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.393 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.393 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:35.393 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.188694+0000", 2024-09-20T19:00:35.393 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.869e-06 2024-09-20T19:00:35.393 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.393 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.393 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:35.393 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.188701+0000", 2024-09-20T19:00:35.393 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9829999999999999e-06 2024-09-20T19:00:35.393 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.394 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.394 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:35.394 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.188718+0000", 2024-09-20T19:00:35.394 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6750999999999999e-05 2024-09-20T19:00:35.394 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.394 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.394 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:35.394 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.188727+0000", 2024-09-20T19:00:35.394 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6449999999999994e-06 2024-09-20T19:00:35.394 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.394 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.394 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.394 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.188800+0000", 2024-09-20T19:00:35.395 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3022999999999995e-05 2024-09-20T19:00:35.395 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.395 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.395 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:00:35.395 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.188810+0000", 2024-09-20T19:00:35.395 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0019e-05 2024-09-20T19:00:35.395 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.395 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.395 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.395 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.101615+0000", 2024-09-20T19:00:35.395 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.91280537699999997 2024-09-20T19:00:35.395 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.395 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.395 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:35.396 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.101713+0000", 2024-09-20T19:00:35.396 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7870999999999999e-05 2024-09-20T19:00:35.396 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.396 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.396 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3", 2024-09-20T19:00:35.396 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.101823+0000", 2024-09-20T19:00:35.396 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010982 2024-09-20T19:00:35.396 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.396 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.396 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:35.396 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.106819+0000", 2024-09-20T19:00:35.396 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00499592 2024-09-20T19:00:35.396 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.396 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.397 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:35.397 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.114173+0000", 2024-09-20T19:00:35.397 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0073544980000000001 2024-09-20T19:00:35.397 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.397 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.397 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:00:35.397 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.127205+0000", 2024-09-20T19:00:35.397 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.013031839 2024-09-20T19:00:35.397 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.397 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.397 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:35.397 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.127309+0000", 2024-09-20T19:00:35.397 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010435899999999999 2024-09-20T19:00:35.397 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.397 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.398 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:35.398 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.127348+0000", 2024-09-20T19:00:35.398 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.8350999999999999e-05 2024-09-20T19:00:35.398 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:35.398 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:35.398 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:35.398 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.398 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.398 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157575 8.7 8:e03ab630:::benchmark_data_smithi083_97422_object41484:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e213)", 2024-09-20T19:00:35.398 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.236943+0000", 2024-09-20T19:00:35.398 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 216.064971361, 2024-09-20T19:00:35.398 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.89067574599999999, 2024-09-20T19:00:35.398 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:35.398 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:35.399 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:35.399 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:35.399 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T19:00:35.399 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T19:00:35.399 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157575 2024-09-20T19:00:35.399 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.399 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:35.399 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.399 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:35.399 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.236943+0000", 2024-09-20T19:00:35.399 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:35.399 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.399 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.400 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:35.400 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.236943+0000", 2024-09-20T19:00:35.400 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:35.400 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.400 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.400 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:35.400 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.236945+0000", 2024-09-20T19:00:35.400 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.584e-06 2024-09-20T19:00:35.400 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.400 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.400 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:35.400 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.236949+0000", 2024-09-20T19:00:35.400 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.673e-06 2024-09-20T19:00:35.400 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.401 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.401 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:35.401 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.236963+0000", 2024-09-20T19:00:35.401 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4209000000000001e-05 2024-09-20T19:00:35.401 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.401 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.401 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:35.401 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.236970+0000", 2024-09-20T19:00:35.401 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0029999999999997e-06 2024-09-20T19:00:35.401 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.401 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.401 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.401 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.237078+0000", 2024-09-20T19:00:35.401 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000107026 2024-09-20T19:00:35.401 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.402 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.402 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:00:35.402 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.237085+0000", 2024-09-20T19:00:35.402 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4669999999999998e-06 2024-09-20T19:00:35.402 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.402 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.402 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.402 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.106857+0000", 2024-09-20T19:00:35.402 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.86977158899999996 2024-09-20T19:00:35.402 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.402 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.402 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:35.403 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.106967+0000", 2024-09-20T19:00:35.403 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011026700000000001 2024-09-20T19:00:35.403 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.403 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.403 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3", 2024-09-20T19:00:35.403 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.107090+0000", 2024-09-20T19:00:35.403 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012295399999999999 2024-09-20T19:00:35.403 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.403 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.403 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:00:35.403 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.127396+0000", 2024-09-20T19:00:35.403 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.020306026000000001 2024-09-20T19:00:35.403 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.403 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.404 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:35.404 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.127459+0000", 2024-09-20T19:00:35.404 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3322000000000001e-05 2024-09-20T19:00:35.404 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.404 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.404 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:35.404 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.127496+0000", 2024-09-20T19:00:35.404 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7156999999999997e-05 2024-09-20T19:00:35.404 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.404 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.404 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:35.404 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.127581+0000", 2024-09-20T19:00:35.404 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4679999999999996e-05 2024-09-20T19:00:35.405 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.405 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.405 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:35.405 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.127619+0000", 2024-09-20T19:00:35.405 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7789e-05 2024-09-20T19:00:35.405 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:35.405 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:35.405 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:35.405 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.405 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.405 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157577 8.7 8:ed15ec18:::benchmark_data_smithi083_97422_object41486:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e213)", 2024-09-20T19:00:35.405 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.239083+0000", 2024-09-20T19:00:35.405 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 216.06283119400001, 2024-09-20T19:00:35.406 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.88873236799999999, 2024-09-20T19:00:35.406 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:35.406 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:35.406 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:35.406 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:35.406 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T19:00:35.406 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T19:00:35.406 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157577 2024-09-20T19:00:35.406 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.406 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:35.406 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.406 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:35.406 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.239083+0000", 2024-09-20T19:00:35.407 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:35.407 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.407 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.407 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:35.407 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.239083+0000", 2024-09-20T19:00:35.407 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:35.407 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.407 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.407 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:35.407 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.239084+0000", 2024-09-20T19:00:35.407 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5599999999999995e-07 2024-09-20T19:00:35.407 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.407 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.407 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:35.408 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.239087+0000", 2024-09-20T19:00:35.408 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8990000000000001e-06 2024-09-20T19:00:35.408 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.408 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.408 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:35.408 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.239094+0000", 2024-09-20T19:00:35.408 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5859999999999999e-06 2024-09-20T19:00:35.408 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.408 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.408 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:35.408 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.239100+0000", 2024-09-20T19:00:35.408 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7490000000000004e-06 2024-09-20T19:00:35.408 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.409 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.409 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.409 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.239149+0000", 2024-09-20T19:00:35.409 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9202000000000001e-05 2024-09-20T19:00:35.409 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.409 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.409 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:00:35.409 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.239157+0000", 2024-09-20T19:00:35.409 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.276e-06 2024-09-20T19:00:35.409 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.409 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.409 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.409 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.109327+0000", 2024-09-20T19:00:35.409 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.87016914899999998 2024-09-20T19:00:35.410 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.410 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.410 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:35.410 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.109435+0000", 2024-09-20T19:00:35.410 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010798099999999999 2024-09-20T19:00:35.410 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.410 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.410 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3", 2024-09-20T19:00:35.410 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.109546+0000", 2024-09-20T19:00:35.410 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011176299999999999 2024-09-20T19:00:35.410 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.410 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.410 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:00:35.410 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.127401+0000", 2024-09-20T19:00:35.411 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.017854716999999999 2024-09-20T19:00:35.411 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.411 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.411 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:35.411 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.127661+0000", 2024-09-20T19:00:35.411 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00026038800000000001 2024-09-20T19:00:35.411 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.411 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.411 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:35.411 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.127697+0000", 2024-09-20T19:00:35.411 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5666e-05 2024-09-20T19:00:35.411 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.411 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.411 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:35.412 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.127779+0000", 2024-09-20T19:00:35.412 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.1860000000000006e-05 2024-09-20T19:00:35.412 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.412 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.412 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:35.412 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.127816+0000", 2024-09-20T19:00:35.412 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6575999999999997e-05 2024-09-20T19:00:35.412 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:35.412 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:35.412 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:35.412 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.412 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.412 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:50392 9.0 9:015f775a:::benchmark_data_smithi083_124552_object50391:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e259)", 2024-09-20T19:00:35.413 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:59:04.061560+0000", 2024-09-20T19:00:35.413 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 91.240354287000002, 2024-09-20T19:00:35.413 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1500109090000001, 2024-09-20T19:00:35.413 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:35.413 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:35.413 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-20T19:00:35.413 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:35.413 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:00:35.413 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:00:35.413 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 50392 2024-09-20T19:00:35.413 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.413 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:35.413 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.414 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:35.414 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.061560+0000", 2024-09-20T19:00:35.414 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:35.414 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.414 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.414 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:35.414 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.061560+0000", 2024-09-20T19:00:35.414 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:35.414 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.414 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.414 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:35.414 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.061561+0000", 2024-09-20T19:00:35.414 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.252e-06 2024-09-20T19:00:35.414 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.415 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.415 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:35.415 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.061585+0000", 2024-09-20T19:00:35.415 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3275e-05 2024-09-20T19:00:35.415 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.415 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.415 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:35.415 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.061596+0000", 2024-09-20T19:00:35.415 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0963e-05 2024-09-20T19:00:35.415 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.415 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.415 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:35.415 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.061604+0000", 2024-09-20T19:00:35.415 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2549999999999997e-06 2024-09-20T19:00:35.415 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.416 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.416 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.416 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.063037+0000", 2024-09-20T19:00:35.416 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001432699 2024-09-20T19:00:35.416 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.416 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.416 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:35.416 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.063196+0000", 2024-09-20T19:00:35.416 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015964199999999999 2024-09-20T19:00:35.416 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.416 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.416 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,2", 2024-09-20T19:00:35.416 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.063344+0000", 2024-09-20T19:00:35.416 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014738699999999999 2024-09-20T19:00:35.416 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.417 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.417 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:00:35.417 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.068011+0000", 2024-09-20T19:00:35.417 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0046678240000000001 2024-09-20T19:00:35.417 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.417 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.417 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.417 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.098231+0000", 2024-09-20T19:00:35.417 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.030219817999999999 2024-09-20T19:00:35.417 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.417 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.417 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:00:35.417 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.098244+0000", 2024-09-20T19:00:35.417 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2381000000000001e-05 2024-09-20T19:00:35.417 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.418 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.418 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.418 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.074475+0000", 2024-09-20T19:00:35.418 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97623147099999996 2024-09-20T19:00:35.418 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.418 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.418 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:00:35.418 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.074562+0000", 2024-09-20T19:00:35.418 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7317000000000004e-05 2024-09-20T19:00:35.418 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.418 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.418 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.418 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.201219+0000", 2024-09-20T19:00:35.418 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1266569 2024-09-20T19:00:35.419 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.419 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.419 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for degraded object", 2024-09-20T19:00:35.419 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.201984+0000", 2024-09-20T19:00:35.419 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00076430200000000002 2024-09-20T19:00:35.419 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.419 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.419 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.419 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.211479+0000", 2024-09-20T19:00:35.419 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0094952230000000006 2024-09-20T19:00:35.419 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.419 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.419 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:35.419 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.211571+0000", 2024-09-20T19:00:35.419 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2200000000000005e-05 2024-09-20T19:00:35.419 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:35.420 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:35.420 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:35.420 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.420 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.420 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:50400 9.0 9:0091def0:::benchmark_data_smithi083_124552_object50399:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-20T19:00:35.420 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:59:04.066245+0000", 2024-09-20T19:00:35.420 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 91.235669587000004, 2024-09-20T19:00:35.420 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1471234040000002, 2024-09-20T19:00:35.420 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:35.420 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:35.420 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:35.420 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:35.420 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:00:35.420 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:00:35.420 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 50400 2024-09-20T19:00:35.421 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.421 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:35.421 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.421 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:35.421 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.066245+0000", 2024-09-20T19:00:35.421 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:35.421 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.421 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.421 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:35.421 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.066245+0000", 2024-09-20T19:00:35.421 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:35.421 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.421 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.421 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:35.422 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.066247+0000", 2024-09-20T19:00:35.422 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.73e-06 2024-09-20T19:00:35.422 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.422 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.422 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:35.422 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.066271+0000", 2024-09-20T19:00:35.422 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4793000000000001e-05 2024-09-20T19:00:35.422 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.422 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.422 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:35.422 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.066286+0000", 2024-09-20T19:00:35.422 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4515000000000001e-05 2024-09-20T19:00:35.422 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.422 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.422 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:35.423 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.066295+0000", 2024-09-20T19:00:35.423 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7360000000000007e-06 2024-09-20T19:00:35.423 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.423 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.423 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.423 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.067890+0000", 2024-09-20T19:00:35.423 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001595791 2024-09-20T19:00:35.423 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.423 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.423 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op must wait for map", 2024-09-20T19:00:35.423 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.067896+0000", 2024-09-20T19:00:35.423 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.446e-06 2024-09-20T19:00:35.423 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.423 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.423 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.423 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.099590+0000", 2024-09-20T19:00:35.424 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.031694211999999999 2024-09-20T19:00:35.424 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.424 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.424 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:00:35.424 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.099600+0000", 2024-09-20T19:00:35.424 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.91e-06 2024-09-20T19:00:35.424 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.424 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.424 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.424 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.075887+0000", 2024-09-20T19:00:35.424 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.976287401 2024-09-20T19:00:35.424 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.424 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.424 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:00:35.425 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.075898+0000", 2024-09-20T19:00:35.425 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0294e-05 2024-09-20T19:00:35.425 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.425 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.425 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.425 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.203092+0000", 2024-09-20T19:00:35.425 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1271948649999999 2024-09-20T19:00:35.425 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.425 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.425 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:35.425 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.203244+0000", 2024-09-20T19:00:35.425 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015116500000000001 2024-09-20T19:00:35.425 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.425 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.425 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,6", 2024-09-20T19:00:35.425 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.203418+0000", 2024-09-20T19:00:35.426 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017446200000000001 2024-09-20T19:00:35.426 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.426 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.426 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:00:35.426 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.208270+0000", 2024-09-20T19:00:35.426 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0048516799999999997 2024-09-20T19:00:35.426 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.426 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.426 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:35.426 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.208385+0000", 2024-09-20T19:00:35.426 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000114954 2024-09-20T19:00:35.426 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.426 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.426 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:35.427 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.211663+0000", 2024-09-20T19:00:35.427 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0032777679999999999 2024-09-20T19:00:35.427 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.427 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.427 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:35.427 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.211716+0000", 2024-09-20T19:00:35.427 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3748000000000001e-05 2024-09-20T19:00:35.427 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.427 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.427 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:35.427 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.213368+0000", 2024-09-20T19:00:35.427 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0016519340000000001 2024-09-20T19:00:35.427 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:35.427 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:35.428 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:35.428 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.428 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.428 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:50401 9.0 9:08f1b92c:::benchmark_data_smithi083_124552_object50400:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-20T19:00:35.428 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:59:04.067487+0000", 2024-09-20T19:00:35.428 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 91.234427636999996, 2024-09-20T19:00:35.428 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.149403511, 2024-09-20T19:00:35.428 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:35.428 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:35.428 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:35.428 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:35.428 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:00:35.428 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:00:35.428 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 50401 2024-09-20T19:00:35.428 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.429 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:35.429 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.429 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:35.429 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.067487+0000", 2024-09-20T19:00:35.429 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:35.429 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.429 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.429 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:35.429 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.067487+0000", 2024-09-20T19:00:35.429 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:35.429 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.429 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.429 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:35.429 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.067488+0000", 2024-09-20T19:00:35.429 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4670000000000001e-06 2024-09-20T19:00:35.430 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.430 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.430 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:35.430 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.067512+0000", 2024-09-20T19:00:35.430 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3561999999999999e-05 2024-09-20T19:00:35.430 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.430 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.430 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:35.430 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.067523+0000", 2024-09-20T19:00:35.430 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.148e-05 2024-09-20T19:00:35.430 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.430 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.430 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:35.430 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.067531+0000", 2024-09-20T19:00:35.431 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2400000000000001e-06 2024-09-20T19:00:35.431 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.431 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.431 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.431 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.068001+0000", 2024-09-20T19:00:35.431 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00047044099999999999 2024-09-20T19:00:35.431 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.431 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.431 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting_for_map not empty", 2024-09-20T19:00:35.431 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.068005+0000", 2024-09-20T19:00:35.431 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3490000000000002e-06 2024-09-20T19:00:35.431 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.431 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.431 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.431 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.099629+0000", 2024-09-20T19:00:35.432 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.031623658999999998 2024-09-20T19:00:35.432 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.432 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.432 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:00:35.432 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.099637+0000", 2024-09-20T19:00:35.432 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5750000000000003e-06 2024-09-20T19:00:35.432 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.432 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.432 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.432 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.075922+0000", 2024-09-20T19:00:35.432 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97628498500000005 2024-09-20T19:00:35.432 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.432 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.432 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:00:35.433 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.075929+0000", 2024-09-20T19:00:35.433 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2389999999999996e-06 2024-09-20T19:00:35.433 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.433 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.433 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.433 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.204955+0000", 2024-09-20T19:00:35.433 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.129026353 2024-09-20T19:00:35.433 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.433 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.433 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:35.433 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.205053+0000", 2024-09-20T19:00:35.433 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7659999999999997e-05 2024-09-20T19:00:35.433 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.433 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.433 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,6", 2024-09-20T19:00:35.434 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.205157+0000", 2024-09-20T19:00:35.434 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000104237 2024-09-20T19:00:35.434 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.434 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.434 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:00:35.434 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.208403+0000", 2024-09-20T19:00:35.434 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0032460050000000002 2024-09-20T19:00:35.434 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.434 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.434 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:35.434 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.213804+0000", 2024-09-20T19:00:35.434 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0054012280000000001 2024-09-20T19:00:35.434 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.434 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.435 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:35.435 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.215227+0000", 2024-09-20T19:00:35.435 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014227039999999999 2024-09-20T19:00:35.435 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.435 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.435 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:35.435 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.215288+0000", 2024-09-20T19:00:35.435 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0730999999999998e-05 2024-09-20T19:00:35.435 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.435 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.435 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:35.435 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.216890+0000", 2024-09-20T19:00:35.435 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001602596 2024-09-20T19:00:35.435 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:35.435 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:35.436 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:35.436 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.436 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.436 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:50420 9.0 9:04fafae8:::benchmark_data_smithi083_124552_object50419:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-20T19:00:35.436 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:59:04.091011+0000", 2024-09-20T19:00:35.436 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 91.210903399000003, 2024-09-20T19:00:35.436 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.143183112, 2024-09-20T19:00:35.436 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:35.436 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:35.436 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:35.436 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:35.436 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:00:35.436 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:00:35.436 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 50420 2024-09-20T19:00:35.436 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.437 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:35.437 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.437 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:35.437 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.091011+0000", 2024-09-20T19:00:35.437 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:35.437 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.437 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.437 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:35.437 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.091011+0000", 2024-09-20T19:00:35.437 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:35.437 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.437 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.437 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:35.437 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.091013+0000", 2024-09-20T19:00:35.437 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6440000000000001e-06 2024-09-20T19:00:35.438 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.438 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.438 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:35.438 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.091069+0000", 2024-09-20T19:00:35.438 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6008000000000003e-05 2024-09-20T19:00:35.438 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.438 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.438 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:35.438 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.091080+0000", 2024-09-20T19:00:35.438 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1449e-05 2024-09-20T19:00:35.438 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.438 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.438 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:35.438 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.091087+0000", 2024-09-20T19:00:35.439 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3769999999999998e-06 2024-09-20T19:00:35.439 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.439 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.439 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.439 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.099660+0000", 2024-09-20T19:00:35.439 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0085721310000000002 2024-09-20T19:00:35.439 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.439 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.439 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:00:35.439 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.099667+0000", 2024-09-20T19:00:35.439 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9439999999999999e-06 2024-09-20T19:00:35.439 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.439 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.439 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.440 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.075951+0000", 2024-09-20T19:00:35.440 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97628466400000002 2024-09-20T19:00:35.440 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.440 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.440 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:00:35.440 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.075958+0000", 2024-09-20T19:00:35.440 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1019999999999999e-06 2024-09-20T19:00:35.440 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.440 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.440 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.440 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.216952+0000", 2024-09-20T19:00:35.440 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1409938340000001 2024-09-20T19:00:35.440 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.440 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.440 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:35.440 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.217094+0000", 2024-09-20T19:00:35.441 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014154900000000001 2024-09-20T19:00:35.441 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.441 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.441 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,6", 2024-09-20T19:00:35.441 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.217258+0000", 2024-09-20T19:00:35.441 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016462399999999999 2024-09-20T19:00:35.441 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.441 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.441 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:35.441 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.223577+0000", 2024-09-20T19:00:35.441 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063191740000000003 2024-09-20T19:00:35.441 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.441 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.441 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:00:35.441 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.223593+0000", 2024-09-20T19:00:35.442 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5588999999999999e-05 2024-09-20T19:00:35.442 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.442 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.442 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:35.442 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.230738+0000", 2024-09-20T19:00:35.442 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0071453879999999999 2024-09-20T19:00:35.442 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.442 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.442 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:35.442 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.230856+0000", 2024-09-20T19:00:35.442 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000117713 2024-09-20T19:00:35.442 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.442 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.442 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:35.443 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.234194+0000", 2024-09-20T19:00:35.443 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0033379220000000001 2024-09-20T19:00:35.443 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:35.443 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:35.443 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:35.443 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.443 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.443 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:50490 9.0 9:061fbcaa:::benchmark_data_smithi083_124552_object50489:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-20T19:00:35.443 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:59:04.144098+0000", 2024-09-20T19:00:35.443 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 91.157816796999995, 2024-09-20T19:00:35.443 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.093487444, 2024-09-20T19:00:35.443 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:35.443 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:35.443 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:35.443 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:35.444 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:00:35.444 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:00:35.444 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 50490 2024-09-20T19:00:35.444 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.444 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:35.444 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.444 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:35.444 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.144098+0000", 2024-09-20T19:00:35.444 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:35.444 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.444 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.444 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:35.444 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.144098+0000", 2024-09-20T19:00:35.444 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:35.444 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.444 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.445 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:35.445 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.144099+0000", 2024-09-20T19:00:35.445 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.412e-06 2024-09-20T19:00:35.445 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.445 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.445 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:35.445 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.144124+0000", 2024-09-20T19:00:35.445 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4788000000000001e-05 2024-09-20T19:00:35.445 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.445 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.445 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:35.445 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.144138+0000", 2024-09-20T19:00:35.445 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3735e-05 2024-09-20T19:00:35.445 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.445 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.445 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:35.446 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.144146+0000", 2024-09-20T19:00:35.446 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.1689999999999992e-06 2024-09-20T19:00:35.446 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.446 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.446 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.446 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.144218+0000", 2024-09-20T19:00:35.446 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1945000000000003e-05 2024-09-20T19:00:35.446 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.446 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.446 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:00:35.446 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.144227+0000", 2024-09-20T19:00:35.446 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6090000000000007e-06 2024-09-20T19:00:35.446 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.446 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.446 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.447 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.075982+0000", 2024-09-20T19:00:35.447 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.93175522499999996 2024-09-20T19:00:35.447 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.447 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.447 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:00:35.447 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.075990+0000", 2024-09-20T19:00:35.447 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4789999999999999e-06 2024-09-20T19:00:35.447 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.447 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.447 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.447 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.218974+0000", 2024-09-20T19:00:35.447 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1429841510000001 2024-09-20T19:00:35.447 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.447 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.448 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:35.448 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.219119+0000", 2024-09-20T19:00:35.448 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014451399999999999 2024-09-20T19:00:35.448 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.448 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.448 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,6", 2024-09-20T19:00:35.448 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.219402+0000", 2024-09-20T19:00:35.448 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00028311699999999999 2024-09-20T19:00:35.448 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.448 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.448 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:35.448 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.223642+0000", 2024-09-20T19:00:35.448 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0042407119999999998 2024-09-20T19:00:35.448 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.449 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.449 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:00:35.449 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.230612+0000", 2024-09-20T19:00:35.449 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0069695499999999997 2024-09-20T19:00:35.449 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.449 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.449 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:35.449 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.234261+0000", 2024-09-20T19:00:35.449 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0036485459999999999 2024-09-20T19:00:35.449 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.449 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.449 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:35.449 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.234386+0000", 2024-09-20T19:00:35.449 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012551199999999999 2024-09-20T19:00:35.449 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.449 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.450 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:35.450 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.237585+0000", 2024-09-20T19:00:35.450 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0031989800000000001 2024-09-20T19:00:35.450 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:35.450 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:35.450 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:35.450 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.450 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.450 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:50503 9.0 9:0e691601:::benchmark_data_smithi083_124552_object50502:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-20T19:00:35.450 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:59:04.153655+0000", 2024-09-20T19:00:35.450 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 91.148259402999997, 2024-09-20T19:00:35.450 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0864925420000002, 2024-09-20T19:00:35.450 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:35.450 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:35.451 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:35.451 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:35.451 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:00:35.451 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:00:35.451 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 50503 2024-09-20T19:00:35.451 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.451 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:35.451 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.451 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:35.451 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.153655+0000", 2024-09-20T19:00:35.451 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:35.451 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.451 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.451 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:35.451 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.153655+0000", 2024-09-20T19:00:35.452 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:35.452 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.452 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.452 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:35.452 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.153656+0000", 2024-09-20T19:00:35.452 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2099999999999996e-07 2024-09-20T19:00:35.452 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.452 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.452 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:35.452 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.153675+0000", 2024-09-20T19:00:35.452 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9873000000000001e-05 2024-09-20T19:00:35.452 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.452 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.452 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:35.452 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.153683+0000", 2024-09-20T19:00:35.452 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5469999999999999e-06 2024-09-20T19:00:35.453 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.453 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.453 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:35.453 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.153689+0000", 2024-09-20T19:00:35.453 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1129999999999997e-06 2024-09-20T19:00:35.453 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.453 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.453 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.453 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.153774+0000", 2024-09-20T19:00:35.453 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4806000000000001e-05 2024-09-20T19:00:35.453 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.453 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.453 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:00:35.453 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.153783+0000", 2024-09-20T19:00:35.454 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8890000000000008e-06 2024-09-20T19:00:35.454 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.454 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.454 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.454 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.076016+0000", 2024-09-20T19:00:35.454 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.92223312199999996 2024-09-20T19:00:35.454 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.454 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.454 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:00:35.454 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.076025+0000", 2024-09-20T19:00:35.454 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0480000000000004e-06 2024-09-20T19:00:35.454 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.454 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.455 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:35.455 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.221175+0000", 2024-09-20T19:00:35.455 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1451503329999999 2024-09-20T19:00:35.455 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.455 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.455 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:35.455 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.221314+0000", 2024-09-20T19:00:35.455 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013887599999999999 2024-09-20T19:00:35.455 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.455 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.455 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,6", 2024-09-20T19:00:35.455 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.221482+0000", 2024-09-20T19:00:35.455 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016769800000000001 2024-09-20T19:00:35.455 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.455 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.456 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:35.456 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.230668+0000", 2024-09-20T19:00:35.456 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0091861119999999997 2024-09-20T19:00:35.456 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.456 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.456 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:00:35.456 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.234213+0000", 2024-09-20T19:00:35.456 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0035446620000000001 2024-09-20T19:00:35.456 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.456 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.456 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:35.456 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.237649+0000", 2024-09-20T19:00:35.456 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003436555 2024-09-20T19:00:35.456 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.457 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.457 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:35.457 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.237809+0000", 2024-09-20T19:00:35.457 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015965099999999999 2024-09-20T19:00:35.457 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:35.457 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:35.457 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:35.457 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.240148+0000", 2024-09-20T19:00:35.457 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0023386359999999998 2024-09-20T19:00:35.457 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:35.457 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:35.457 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:35.457 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:35.457 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:35.458 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T19:00:35.470 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:35.492 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:35.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:35.493 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:35.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:35 smithi142 ceph-mon[32508]: 4.10 scrub starts 2024-09-20T19:00:35.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:35 smithi142 ceph-mon[32508]: 4.10 scrub ok 2024-09-20T19:00:35.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:35 smithi142 ceph-mon[32508]: osd.4 marked itself down and dead 2024-09-20T19:00:35.593 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:00:35.615 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:35.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:35.616 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:00:35.716 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:35.737 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:35.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:35.738 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:35.839 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:35.860 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:35.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:35.861 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:35.961 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:35.984 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:35.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:35.985 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:36.046 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:00:35 smithi142 podman[107841]: 2024-09-20 19:00:35.758356966 +0000 UTC m=+1.402448124 container died 8eeeecc584da503ab66ddfd70f60bb7b35173bed75554cd78a2c42ed0ba147b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2) 2024-09-20T19:00:36.086 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:36.108 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:36.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:36.109 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:36.210 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:00:36.237 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:36.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:36.239 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:00:36.339 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:36.388 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:36.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:36.389 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:36.489 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:00:36.515 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:36 smithi083 ceph-mon[29421]: 5.14 scrub starts 2024-09-20T19:00:36.515 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:36 smithi083 ceph-mon[29421]: 5.14 scrub ok 2024-09-20T19:00:36.515 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:36 smithi083 ceph-mon[29421]: 4.4 scrub starts 2024-09-20T19:00:36.515 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:36 smithi083 ceph-mon[29421]: 4.4 scrub ok 2024-09-20T19:00:36.515 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:36 smithi083 ceph-mon[29421]: 3.e scrub starts 2024-09-20T19:00:36.515 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:36 smithi083 ceph-mon[29421]: 3.e scrub ok 2024-09-20T19:00:36.515 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:36 smithi083 ceph-mon[29421]: pgmap v530: 148 pgs: 148 active+clean; 5.5 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 980 op/s 2024-09-20T19:00:36.515 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:36 smithi083 ceph-mon[29421]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-20T19:00:36.515 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:36 smithi083 ceph-mon[29421]: osdmap e292: 8 total, 7 up, 8 in 2024-09-20T19:00:36.516 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:00:36 smithi142 podman[107841]: 2024-09-20 19:00:36.203856386 +0000 UTC m=+1.847947533 container cleanup 8eeeecc584da503ab66ddfd70f60bb7b35173bed75554cd78a2c42ed0ba147b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-20T19:00:36.516 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:00:36 smithi142 bash[107841]: ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4 2024-09-20T19:00:36.516 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:36 smithi083 ceph-mon[36260]: 5.14 scrub starts 2024-09-20T19:00:36.516 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:36 smithi083 ceph-mon[36260]: 5.14 scrub ok 2024-09-20T19:00:36.516 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:36 smithi083 ceph-mon[36260]: 4.4 scrub starts 2024-09-20T19:00:36.516 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:36 smithi083 ceph-mon[36260]: 4.4 scrub ok 2024-09-20T19:00:36.516 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:36 smithi083 ceph-mon[36260]: 3.e scrub starts 2024-09-20T19:00:36.516 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:36 smithi083 ceph-mon[36260]: 3.e scrub ok 2024-09-20T19:00:36.516 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:36 smithi083 ceph-mon[36260]: pgmap v530: 148 pgs: 148 active+clean; 5.5 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 980 op/s 2024-09-20T19:00:36.517 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:36 smithi083 ceph-mon[36260]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-20T19:00:36.517 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:36 smithi083 ceph-mon[36260]: osdmap e292: 8 total, 7 up, 8 in 2024-09-20T19:00:36.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:36 smithi142 ceph-mon[32508]: 5.14 scrub starts 2024-09-20T19:00:36.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:36 smithi142 ceph-mon[32508]: 5.14 scrub ok 2024-09-20T19:00:36.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:36 smithi142 ceph-mon[32508]: 4.4 scrub starts 2024-09-20T19:00:36.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:36 smithi142 ceph-mon[32508]: 4.4 scrub ok 2024-09-20T19:00:36.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:36 smithi142 ceph-mon[32508]: 3.e scrub starts 2024-09-20T19:00:36.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:36 smithi142 ceph-mon[32508]: 3.e scrub ok 2024-09-20T19:00:36.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:36 smithi142 ceph-mon[32508]: pgmap v530: 148 pgs: 148 active+clean; 5.5 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 980 op/s 2024-09-20T19:00:36.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:36 smithi142 ceph-mon[32508]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-20T19:00:36.517 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:36 smithi142 ceph-mon[32508]: osdmap e292: 8 total, 7 up, 8 in 2024-09-20T19:00:36.519 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:36.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:36.520 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:00:36.621 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:36.655 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:36.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:36.656 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:36.757 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:36.794 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:36.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:36.795 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:36.857 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-20T19:00:36.870 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:00:36.896 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:36.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:36.973 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:36.974 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:37.074 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:00:37.102 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:37.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:37.103 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:00:37.204 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:37.214 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-20T19:00:37.230 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:37.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:37.231 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:37.265 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:00:36 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:00:37.266 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:37 smithi142 ceph-mon[32508]: osdmap e293: 8 total, 7 up, 8 in 2024-09-20T19:00:37.266 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:37 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3463487899' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:00:37.331 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:37.353 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:37.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:37.354 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:37.432 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:37 smithi083 ceph-mon[29421]: osdmap e293: 8 total, 7 up, 8 in 2024-09-20T19:00:37.432 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:37 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3463487899' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:00:37.433 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:37 smithi083 ceph-mon[36260]: osdmap e293: 8 total, 7 up, 8 in 2024-09-20T19:00:37.433 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:37 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3463487899' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:00:37.455 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:00:37.475 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:00:37.475 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:00:37.482 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:37.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:37.484 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:00:37.504 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:00:37.504 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:00:37.535 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:00:37.535 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:00:37.569 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:00:37.569 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:00:37.570 INFO:teuthology.orchestra.run.smithi083.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-20T19:00:37.584 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:37.607 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:00:37.607 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:00:37.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:37.617 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:37.618 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:37.641 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:00:37.641 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:00:37.676 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:00:37.676 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:00:37.718 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:37.750 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:37.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:37.751 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:37.852 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:00:37.882 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:37.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:37.883 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:00:37.984 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:38.015 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:00:37 smithi142 podman[107942]: 2024-09-20 19:00:37.706256069 +0000 UTC m=+1.942474825 container remove 8eeeecc584da503ab66ddfd70f60bb7b35173bed75554cd78a2c42ed0ba147b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-20T19:00:38.017 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:38.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:38.019 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:38.119 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:38.158 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:38.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:38.159 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:38.259 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:38.299 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:38.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:38.300 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:38.401 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:38.432 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:38.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:38.433 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:38.473 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:00:38 smithi142 podman[108189]: 2024-09-20 19:00:38.200421574 +0000 UTC m=+0.026688815 image pull 441f9a3fe51528ef653b21f42e9d884db6124a0c7d482e865382bdfdb9ec979c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2 2024-09-20T19:00:38.473 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:00:38 smithi142 podman[108189]: 2024-09-20 19:00:38.341209769 +0000 UTC m=+0.167477042 container create 7db7c1c98a1a83ee91c7cebbaf2110d4e2d0ade2ed0b365c3e4151af877f8b8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-20T19:00:38.474 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:38 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:00:38.474 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:38 smithi142 ceph-mon[32508]: pgmap v533: 148 pgs: 13 active+undersized, 28 peering, 1 stale+active+clean, 14 active+undersized+degraded, 92 active+clean; 5.2 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 0 B/s wr, 1.39k op/s; 10429/257499 objects degraded (4.050%) 2024-09-20T19:00:38.474 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:38 smithi142 ceph-mon[32508]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T19:00:38.474 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:38 smithi142 ceph-mon[32508]: Health check failed: Reduced data availability: 4 pgs inactive, 5 pgs peering (PG_AVAILABILITY) 2024-09-20T19:00:38.474 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:38 smithi142 ceph-mon[32508]: Health check failed: Degraded data redundancy: 10429/257499 objects degraded (4.050%), 14 pgs degraded (PG_DEGRADED) 2024-09-20T19:00:38.474 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:38 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3463487899' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:00:38.474 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:38 smithi142 ceph-mon[32508]: osdmap e294: 8 total, 7 up, 8 in 2024-09-20T19:00:38.474 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:38 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2986492672' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:00:38.534 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:00:38.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:38 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:00:38.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:38 smithi083 ceph-mon[29421]: pgmap v533: 148 pgs: 13 active+undersized, 28 peering, 1 stale+active+clean, 14 active+undersized+degraded, 92 active+clean; 5.2 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 0 B/s wr, 1.39k op/s; 10429/257499 objects degraded (4.050%) 2024-09-20T19:00:38.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:38 smithi083 ceph-mon[29421]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T19:00:38.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:38 smithi083 ceph-mon[29421]: Health check failed: Reduced data availability: 4 pgs inactive, 5 pgs peering (PG_AVAILABILITY) 2024-09-20T19:00:38.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:38 smithi083 ceph-mon[29421]: Health check failed: Degraded data redundancy: 10429/257499 objects degraded (4.050%), 14 pgs degraded (PG_DEGRADED) 2024-09-20T19:00:38.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:38 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3463487899' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:00:38.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:38 smithi083 ceph-mon[29421]: osdmap e294: 8 total, 7 up, 8 in 2024-09-20T19:00:38.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:38 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2986492672' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:00:38.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:38 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:00:38.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:38 smithi083 ceph-mon[36260]: pgmap v533: 148 pgs: 13 active+undersized, 28 peering, 1 stale+active+clean, 14 active+undersized+degraded, 92 active+clean; 5.2 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 0 B/s wr, 1.39k op/s; 10429/257499 objects degraded (4.050%) 2024-09-20T19:00:38.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:38 smithi083 ceph-mon[36260]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T19:00:38.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:38 smithi083 ceph-mon[36260]: Health check failed: Reduced data availability: 4 pgs inactive, 5 pgs peering (PG_AVAILABILITY) 2024-09-20T19:00:38.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:38 smithi083 ceph-mon[36260]: Health check failed: Degraded data redundancy: 10429/257499 objects degraded (4.050%), 14 pgs degraded (PG_DEGRADED) 2024-09-20T19:00:38.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:38 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3463487899' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:00:38.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:38 smithi083 ceph-mon[36260]: osdmap e294: 8 total, 7 up, 8 in 2024-09-20T19:00:38.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:38 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2986492672' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:00:38.569 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:38.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:38.570 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:00:38.670 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:38.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:38.705 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:38.705 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:38.766 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:00:38 smithi142 podman[108189]: 2024-09-20 19:00:38.60247067 +0000 UTC m=+0.428737924 container init 7db7c1c98a1a83ee91c7cebbaf2110d4e2d0ade2ed0b365c3e4151af877f8b8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-20T19:00:38.766 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:00:38 smithi142 podman[108189]: 2024-09-20 19:00:38.60783322 +0000 UTC m=+0.434100460 container start 7db7c1c98a1a83ee91c7cebbaf2110d4e2d0ade2ed0b365c3e4151af877f8b8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-20T19:00:38.767 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:00:38 smithi142 podman[108189]: 2024-09-20 19:00:38.68570412 +0000 UTC m=+0.511971358 container attach 7db7c1c98a1a83ee91c7cebbaf2110d4e2d0ade2ed0b365c3e4151af877f8b8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-20T19:00:38.806 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:38.839 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:38.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:38.840 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:38.940 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:38.974 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:38.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:38.975 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:39.076 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:39.108 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:39.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:39.109 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:39.210 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:39.244 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:39.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:39.245 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:39.265 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:00:38 smithi142 podman[108189]: 2024-09-20 19:00:38.880825931 +0000 UTC m=+0.707093167 container died 7db7c1c98a1a83ee91c7cebbaf2110d4e2d0ade2ed0b365c3e4151af877f8b8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-20T19:00:39.346 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:00:39.380 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:39.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:39.381 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:00:39.444 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:00:39.482 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:39.518 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:39.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:39.519 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:39.580 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@osd.4.service' 2024-09-20T19:00:39.620 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:39.654 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:39.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:39.655 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:39.688 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-20T19:00:39.688 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-20T19:00:39.689 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.4 2024-09-20T19:00:39.689 INFO:tasks.thrashosds.thrasher:remote for osd 4 is ubuntu@smithi142.front.sepia.ceph.com 2024-09-20T19:00:39.689 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell -n osd.4 --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-4 --journal-path /var/lib/ceph/osd/ceph-4/journal --op list-pgs 2024-09-20T19:00:39.756 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:00:39.806 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:39.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:39.807 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:00:39.908 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:39.945 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-20T19:00:39.945 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-20T19:00:39.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:39.950 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:39.950 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:40.051 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:40.057 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/osd.4/config 2024-09-20T19:00:40.090 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:40.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:40.091 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:40.192 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:40.227 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:40.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:40.228 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:40.329 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:40.343 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:40 smithi142 ceph-mon[32508]: pgmap v535: 148 pgs: 17 active+undersized, 28 peering, 16 active+undersized+degraded, 87 active+clean; 5.2 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 0 B/s wr, 883 op/s; 10434/254682 objects degraded (4.097%) 2024-09-20T19:00:40.369 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:40.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:40.370 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:40.408 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:40 smithi083 ceph-mon[29421]: pgmap v535: 148 pgs: 17 active+undersized, 28 peering, 16 active+undersized+degraded, 87 active+clean; 5.2 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 0 B/s wr, 883 op/s; 10434/254682 objects degraded (4.097%) 2024-09-20T19:00:40.409 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:40 smithi083 ceph-mon[36260]: pgmap v535: 148 pgs: 17 active+undersized, 28 peering, 16 active+undersized+degraded, 87 active+clean; 5.2 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 0 B/s wr, 883 op/s; 10434/254682 objects degraded (4.097%) 2024-09-20T19:00:40.411 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T19:00:40.447 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T19:00:40.447 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:00:40.470 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:40.526 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:40.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:40.527 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:40.627 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:40.671 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:40.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:40.672 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:40.773 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:40.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:40.806 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:40.806 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:40.907 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:00:40.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:40.936 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:40.936 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:00:41.037 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:41.045 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:00:40 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:00:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:00:41.070 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:41.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:41.072 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:41.173 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:41.194 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_blocked_ops 2024-09-20T19:00:41.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:41.199 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:41.199 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:41.300 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:41.306 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:00:41.345 INFO:teuthology.orchestra.run.smithi142.stderr:Creating an OSD daemon form without an OSD FSID value 2024-09-20T19:00:41.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:41.373 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:41.373 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:41.474 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:41.520 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:41.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:41.521 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:41.622 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:41.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:41.664 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:41.664 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:41.765 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:41.804 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:41.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:41.805 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:41.879 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:41 smithi142 ceph-mon[32508]: pgmap v536: 148 pgs: 30 active+undersized, 7 peering, 24 active+undersized+degraded, 87 active+clean; 5.1 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 0 B/s wr, 1.06k op/s; 15466/251433 objects degraded (6.151%) 2024-09-20T19:00:41.905 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:41.947 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T19:00:41.948 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:41.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:41.949 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:41.969 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:00:42.045 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:41 smithi083 ceph-mon[29421]: pgmap v536: 148 pgs: 30 active+undersized, 7 peering, 24 active+undersized+degraded, 87 active+clean; 5.1 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 0 B/s wr, 1.06k op/s; 15466/251433 objects degraded (6.151%) 2024-09-20T19:00:42.046 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:41 smithi083 ceph-mon[36260]: pgmap v536: 148 pgs: 30 active+undersized, 7 peering, 24 active+undersized+degraded, 87 active+clean; 5.1 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 0 B/s wr, 1.06k op/s; 15466/251433 objects degraded (6.151%) 2024-09-20T19:00:42.050 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:00:42.100 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:42.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:42.101 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:00:42.202 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:00:42.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:42.241 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:42.241 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:00:42.342 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:42.377 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:42.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:42.378 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:42.479 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:42.510 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:42.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:42.511 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:42.612 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:42.643 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:42.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:42.644 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:42.745 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:42.781 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:42.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:42.782 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:42.818 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-20T19:00:42.883 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:42.918 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:42.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:42.919 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:42.999 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T19:00:43.000 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:00:43.020 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:00:43.059 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:43.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:43.061 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:00:43.161 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:00:43.203 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:43.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:43.204 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:00:43.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:42 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/4142070775' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:00:43.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:42 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:00:43.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:42 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/4142070775' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:00:43.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:42 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:00:43.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:42 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/4142070775' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:00:43.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:42 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:00:43.305 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:43.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:43.346 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:43.347 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:43.447 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:43.491 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:43.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:43.492 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:43.593 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:00:43.628 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:43.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:43.629 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:00:43.730 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:43.770 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:43.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:43.771 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:43.824 INFO:teuthology.orchestra.run.smithi142.stdout:9.e 2024-09-20T19:00:43.824 INFO:teuthology.orchestra.run.smithi142.stdout:9.c 2024-09-20T19:00:43.824 INFO:teuthology.orchestra.run.smithi142.stdout:9.4 2024-09-20T19:00:43.824 INFO:teuthology.orchestra.run.smithi142.stdout:5.1 2024-09-20T19:00:43.824 INFO:teuthology.orchestra.run.smithi142.stdout:4.5 2024-09-20T19:00:43.824 INFO:teuthology.orchestra.run.smithi142.stdout:4.2 2024-09-20T19:00:43.824 INFO:teuthology.orchestra.run.smithi142.stdout:4.d 2024-09-20T19:00:43.824 INFO:teuthology.orchestra.run.smithi142.stdout:4.1d 2024-09-20T19:00:43.824 INFO:teuthology.orchestra.run.smithi142.stdout:4.1 2024-09-20T19:00:43.824 INFO:teuthology.orchestra.run.smithi142.stdout:3.15 2024-09-20T19:00:43.824 INFO:teuthology.orchestra.run.smithi142.stdout:4.13 2024-09-20T19:00:43.824 INFO:teuthology.orchestra.run.smithi142.stdout:4.b 2024-09-20T19:00:43.825 INFO:teuthology.orchestra.run.smithi142.stdout:4.c 2024-09-20T19:00:43.825 INFO:teuthology.orchestra.run.smithi142.stdout:3.1a 2024-09-20T19:00:43.825 INFO:teuthology.orchestra.run.smithi142.stdout:4.18 2024-09-20T19:00:43.825 INFO:teuthology.orchestra.run.smithi142.stdout:3.f 2024-09-20T19:00:43.825 INFO:teuthology.orchestra.run.smithi142.stdout:3.e 2024-09-20T19:00:43.825 INFO:teuthology.orchestra.run.smithi142.stdout:3.3 2024-09-20T19:00:43.825 INFO:teuthology.orchestra.run.smithi142.stdout:3.11 2024-09-20T19:00:43.825 INFO:teuthology.orchestra.run.smithi142.stdout:3.13 2024-09-20T19:00:43.825 INFO:teuthology.orchestra.run.smithi142.stdout:5.18 2024-09-20T19:00:43.825 INFO:teuthology.orchestra.run.smithi142.stdout:4.1b 2024-09-20T19:00:43.825 INFO:teuthology.orchestra.run.smithi142.stdout:3.a 2024-09-20T19:00:43.825 INFO:teuthology.orchestra.run.smithi142.stdout:3.14 2024-09-20T19:00:43.825 INFO:teuthology.orchestra.run.smithi142.stdout:6.e 2024-09-20T19:00:43.825 INFO:teuthology.orchestra.run.smithi142.stdout:3.1b 2024-09-20T19:00:43.825 INFO:teuthology.orchestra.run.smithi142.stdout:3.6 2024-09-20T19:00:43.826 INFO:teuthology.orchestra.run.smithi142.stdout:3.b 2024-09-20T19:00:43.826 INFO:teuthology.orchestra.run.smithi142.stdout:6.14 2024-09-20T19:00:43.826 INFO:teuthology.orchestra.run.smithi142.stdout:6.6 2024-09-20T19:00:43.826 INFO:teuthology.orchestra.run.smithi142.stdout:3.1d 2024-09-20T19:00:43.826 INFO:teuthology.orchestra.run.smithi142.stdout:3.19 2024-09-20T19:00:43.826 INFO:teuthology.orchestra.run.smithi142.stdout:3.1 2024-09-20T19:00:43.826 INFO:teuthology.orchestra.run.smithi142.stdout:5.a 2024-09-20T19:00:43.826 INFO:teuthology.orchestra.run.smithi142.stdout:4.9 2024-09-20T19:00:43.826 INFO:teuthology.orchestra.run.smithi142.stdout:3.9 2024-09-20T19:00:43.826 INFO:teuthology.orchestra.run.smithi142.stdout:5.0 2024-09-20T19:00:43.826 INFO:teuthology.orchestra.run.smithi142.stdout:4.e 2024-09-20T19:00:43.826 INFO:teuthology.orchestra.run.smithi142.stdout:4.f 2024-09-20T19:00:43.826 INFO:teuthology.orchestra.run.smithi142.stdout:5.10 2024-09-20T19:00:43.826 INFO:teuthology.orchestra.run.smithi142.stdout:5.11 2024-09-20T19:00:43.827 INFO:teuthology.orchestra.run.smithi142.stdout:5.1a 2024-09-20T19:00:43.827 INFO:teuthology.orchestra.run.smithi142.stdout:6.1d 2024-09-20T19:00:43.827 INFO:teuthology.orchestra.run.smithi142.stdout:5.1c 2024-09-20T19:00:43.827 INFO:teuthology.orchestra.run.smithi142.stdout:5.1f 2024-09-20T19:00:43.827 INFO:teuthology.orchestra.run.smithi142.stdout:5.5 2024-09-20T19:00:43.827 INFO:teuthology.orchestra.run.smithi142.stdout:3.1c 2024-09-20T19:00:43.827 INFO:teuthology.orchestra.run.smithi142.stdout:5.9 2024-09-20T19:00:43.827 INFO:teuthology.orchestra.run.smithi142.stdout:5.c 2024-09-20T19:00:43.827 INFO:teuthology.orchestra.run.smithi142.stdout:6.17 2024-09-20T19:00:43.827 INFO:teuthology.orchestra.run.smithi142.stdout:5.e 2024-09-20T19:00:43.827 INFO:teuthology.orchestra.run.smithi142.stdout:5.f 2024-09-20T19:00:43.827 INFO:teuthology.orchestra.run.smithi142.stdout:6.1e 2024-09-20T19:00:43.827 INFO:teuthology.orchestra.run.smithi142.stdout:6.15 2024-09-20T19:00:43.827 INFO:teuthology.orchestra.run.smithi142.stdout:6.1a 2024-09-20T19:00:43.828 INFO:teuthology.orchestra.run.smithi142.stdout:6.2 2024-09-20T19:00:43.828 INFO:teuthology.orchestra.run.smithi142.stdout:4.1a 2024-09-20T19:00:43.828 INFO:teuthology.orchestra.run.smithi142.stdout:6.7 2024-09-20T19:00:43.828 INFO:teuthology.orchestra.run.smithi142.stdout:6.f 2024-09-20T19:00:43.828 INFO:teuthology.orchestra.run.smithi142.stdout:6.12 2024-09-20T19:00:43.828 INFO:teuthology.orchestra.run.smithi142.stdout:5.1d 2024-09-20T19:00:43.828 INFO:teuthology.orchestra.run.smithi142.stdout:9.1 2024-09-20T19:00:43.872 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:43.905 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:43.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:43.906 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:44.006 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:44.032 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:43 smithi083 ceph-mon[29421]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:00:44.032 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:43 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T19:00:44.033 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:43 smithi083 ceph-mon[29421]: osdmap e295: 8 total, 7 up, 8 in 2024-09-20T19:00:44.033 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:43 smithi083 ceph-mon[29421]: pgmap v538: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 4.8 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 0 B/s wr, 1.33k op/s; 14706/236118 objects degraded (6.228%) 2024-09-20T19:00:44.034 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:43 smithi083 ceph-mon[36260]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:00:44.034 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:43 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T19:00:44.034 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:43 smithi083 ceph-mon[36260]: osdmap e295: 8 total, 7 up, 8 in 2024-09-20T19:00:44.034 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:43 smithi083 ceph-mon[36260]: pgmap v538: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 4.8 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 0 B/s wr, 1.33k op/s; 14706/236118 objects degraded (6.228%) 2024-09-20T19:00:44.037 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:44.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:44.038 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:44.139 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:44.169 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:44.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:44.170 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:44.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:43 smithi142 ceph-mon[32508]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:00:44.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:43 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T19:00:44.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:43 smithi142 ceph-mon[32508]: osdmap e295: 8 total, 7 up, 8 in 2024-09-20T19:00:44.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:43 smithi142 ceph-mon[32508]: pgmap v538: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 4.8 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 0 B/s wr, 1.33k op/s; 14706/236118 objects degraded (6.228%) 2024-09-20T19:00:44.272 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:00:44.308 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:44.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:44.309 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:00:44.400 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_historic_ops 2024-09-20T19:00:44.410 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:00:44.438 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:00:44.491 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:44.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:44.492 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:00:44.593 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:00:44.624 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:44.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:44.625 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:00:44.726 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:44.757 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:44.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:44.757 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:44.858 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:44.895 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:44.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:44.896 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:44.923 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:44 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 14706/236118 objects degraded (6.228%), 27 pgs degraded (PG_DEGRADED) 2024-09-20T19:00:44.924 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:44 smithi083 ceph-mon[29421]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 2 pgs peering) 2024-09-20T19:00:44.924 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:44 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:00:44.924 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:44 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:00:44.924 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:44 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 14706/236118 objects degraded (6.228%), 27 pgs degraded (PG_DEGRADED) 2024-09-20T19:00:44.924 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:44 smithi083 ceph-mon[36260]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 2 pgs peering) 2024-09-20T19:00:44.924 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:44 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:00:44.924 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:44 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:00:44.997 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:45.015 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:44 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 14706/236118 objects degraded (6.228%), 27 pgs degraded (PG_DEGRADED) 2024-09-20T19:00:45.015 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:44 smithi142 ceph-mon[32508]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 2 pgs peering) 2024-09-20T19:00:45.015 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:44 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:00:45.015 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:44 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:00:45.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:45.040 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:45.040 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:45.077 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell -n osd.4 --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-4 --journal-path /var/lib/ceph/osd/ceph-4/journal --op export --pgid 5.1f --file /var/log/ceph/exp.5.1f.4 2024-09-20T19:00:45.099 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:00:45.108 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:00:45.108 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:00:45.140 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:00:45.145 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:00:45.145 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:00:45.179 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:00:45.180 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:00:45.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:45.181 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:45.181 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:00:45.212 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:00:45.212 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:00:45.213 INFO:teuthology.orchestra.run.smithi083.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-20T19:00:45.242 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:00:45.243 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:00:45.275 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:00:45.276 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:00:45.282 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:45.312 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:00:45.312 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:00:45.327 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:45.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:45.329 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:45.366 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/osd.4/config 2024-09-20T19:00:45.429 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:45.480 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:45.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:45.481 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:45.582 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:00:45.617 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:45.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:45.617 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:00:45.718 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:45.749 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:45.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:45.750 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:45.784 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T19:00:45.851 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:45.894 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T19:00:45.894 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-20T19:00:45.894 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-20T19:00:45.894 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-20T19:00:45.894 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.895 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:27549 7.e 7:77436293:::benchmark_data_smithi083_70643_object27548:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T19:00:45.895 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:08.731657+0000", 2024-09-20T19:00:45.895 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 577.15064219099997, 2024-09-20T19:00:45.895 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.113586293, 2024-09-20T19:00:45.895 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:45.895 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:45.895 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:45.895 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:45.895 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T19:00:45.895 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T19:00:45.895 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 27549 2024-09-20T19:00:45.895 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.896 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:45.896 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.896 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:45.896 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.731657+0000", 2024-09-20T19:00:45.896 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:45.896 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.896 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.896 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:45.896 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.731657+0000", 2024-09-20T19:00:45.896 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:45.896 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.896 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.897 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:45.897 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.731658+0000", 2024-09-20T19:00:45.897 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1400000000000001e-06 2024-09-20T19:00:45.897 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.897 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.897 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:45.897 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.731686+0000", 2024-09-20T19:00:45.897 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7840000000000001e-05 2024-09-20T19:00:45.897 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.898 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.898 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:45.898 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.731697+0000", 2024-09-20T19:00:45.898 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0774999999999999e-05 2024-09-20T19:00:45.898 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.898 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.898 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:45.898 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.731704+0000", 2024-09-20T19:00:45.898 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1740000000000003e-06 2024-09-20T19:00:45.898 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.898 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.899 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:45.899 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.731757+0000", 2024-09-20T19:00:45.899 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3214000000000002e-05 2024-09-20T19:00:45.899 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.899 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.899 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:45.899 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.731876+0000", 2024-09-20T19:00:45.899 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000118829 2024-09-20T19:00:45.899 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.899 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.899 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T19:00:45.899 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.732005+0000", 2024-09-20T19:00:45.900 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000128937 2024-09-20T19:00:45.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.900 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.900 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:00:45.900 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733315+0000", 2024-09-20T19:00:45.900 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001309943 2024-09-20T19:00:45.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.900 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.900 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:45.900 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843149+0000", 2024-09-20T19:00:45.900 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.109834472 2024-09-20T19:00:45.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.901 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.901 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:45.901 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843225+0000", 2024-09-20T19:00:45.901 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5304999999999998e-05 2024-09-20T19:00:45.901 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.901 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.901 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:45.901 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.844447+0000", 2024-09-20T19:00:45.901 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0012227239999999999 2024-09-20T19:00:45.901 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.901 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.902 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:45.902 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.845243+0000", 2024-09-20T19:00:45.902 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00079593999999999997 2024-09-20T19:00:45.902 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:45.902 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:45.902 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:45.902 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.902 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.902 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:27551 7.8 7:1bc4112c:::benchmark_data_smithi083_70643_object27550:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T19:00:45.902 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:08.733037+0000", 2024-09-20T19:00:45.902 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 577.14926240199998, 2024-09-20T19:00:45.903 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.110337717, 2024-09-20T19:00:45.903 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:45.903 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:45.903 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:45.903 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:45.903 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T19:00:45.903 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T19:00:45.903 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 27551 2024-09-20T19:00:45.903 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.903 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:45.903 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.904 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:45.904 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733037+0000", 2024-09-20T19:00:45.904 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:45.904 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.904 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.904 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:45.904 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733037+0000", 2024-09-20T19:00:45.904 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:45.904 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.904 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.904 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:45.904 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733038+0000", 2024-09-20T19:00:45.905 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.497e-06 2024-09-20T19:00:45.905 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.905 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.905 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:45.905 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733062+0000", 2024-09-20T19:00:45.905 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3498000000000001e-05 2024-09-20T19:00:45.905 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.905 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.905 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:45.905 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733072+0000", 2024-09-20T19:00:45.905 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0173000000000001e-05 2024-09-20T19:00:45.905 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.906 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.906 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:45.906 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733079+0000", 2024-09-20T19:00:45.906 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5370000000000003e-06 2024-09-20T19:00:45.906 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.906 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.906 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:45.906 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733187+0000", 2024-09-20T19:00:45.906 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010888 2024-09-20T19:00:45.906 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.906 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.907 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:45.907 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733330+0000", 2024-09-20T19:00:45.907 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000142206 2024-09-20T19:00:45.907 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.907 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.907 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4,7", 2024-09-20T19:00:45.907 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733549+0000", 2024-09-20T19:00:45.907 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0002198 2024-09-20T19:00:45.907 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.907 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.907 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:00:45.908 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.735541+0000", 2024-09-20T19:00:45.908 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0019915129999999999 2024-09-20T19:00:45.908 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.908 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.908 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:45.908 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.842311+0000", 2024-09-20T19:00:45.908 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.106769612 2024-09-20T19:00:45.908 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.908 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.908 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:45.908 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.842372+0000", 2024-09-20T19:00:45.909 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0968000000000003e-05 2024-09-20T19:00:45.909 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.909 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.909 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:45.909 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843315+0000", 2024-09-20T19:00:45.909 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00094376300000000002 2024-09-20T19:00:45.909 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.909 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.909 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:45.909 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843375+0000", 2024-09-20T19:00:45.909 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.927e-05 2024-09-20T19:00:45.909 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:45.910 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:45.910 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:45.910 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.910 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.910 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:27555 7.e 7:73709da2:::benchmark_data_smithi083_70643_object27554:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T19:00:45.910 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:08.733695+0000", 2024-09-20T19:00:45.910 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 577.14860427899998, 2024-09-20T19:00:45.910 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.11168107300000001, 2024-09-20T19:00:45.910 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:45.910 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:45.910 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:45.911 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:45.911 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T19:00:45.911 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T19:00:45.911 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 27555 2024-09-20T19:00:45.911 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.911 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:45.911 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.911 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:45.911 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733695+0000", 2024-09-20T19:00:45.911 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:45.911 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.911 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.912 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:45.912 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733695+0000", 2024-09-20T19:00:45.912 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:45.912 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.912 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.912 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:45.912 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733696+0000", 2024-09-20T19:00:45.912 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0200000000000001e-07 2024-09-20T19:00:45.912 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.912 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.912 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:45.912 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733718+0000", 2024-09-20T19:00:45.913 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2226999999999999e-05 2024-09-20T19:00:45.913 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.913 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.913 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:45.913 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733727+0000", 2024-09-20T19:00:45.913 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6049999999999998e-06 2024-09-20T19:00:45.913 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.913 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.913 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:45.913 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733734+0000", 2024-09-20T19:00:45.913 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0639999999999996e-06 2024-09-20T19:00:45.913 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.914 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.914 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:45.914 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733789+0000", 2024-09-20T19:00:45.914 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5087999999999998e-05 2024-09-20T19:00:45.914 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.914 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.914 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:45.914 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733920+0000", 2024-09-20T19:00:45.914 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013146800000000001 2024-09-20T19:00:45.914 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.914 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.915 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T19:00:45.915 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.733995+0000", 2024-09-20T19:00:45.915 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4664000000000002e-05 2024-09-20T19:00:45.915 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.915 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.915 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:00:45.915 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.735538+0000", 2024-09-20T19:00:45.915 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001542671 2024-09-20T19:00:45.915 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.915 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.915 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:45.915 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.843187+0000", 2024-09-20T19:00:45.916 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.107649921 2024-09-20T19:00:45.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.916 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.916 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:45.916 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.845296+0000", 2024-09-20T19:00:45.916 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0021081920000000001 2024-09-20T19:00:45.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.916 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.916 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:45.916 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.845341+0000", 2024-09-20T19:00:45.916 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5141e-05 2024-09-20T19:00:45.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.917 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.917 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:45.917 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.845376+0000", 2024-09-20T19:00:45.917 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.523e-05 2024-09-20T19:00:45.917 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:45.917 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:45.917 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:45.917 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.917 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.917 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:27560 7.e 7:7d7d0b94:::benchmark_data_smithi083_70643_object27559:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T19:00:45.917 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:51:08.736128+0000", 2024-09-20T19:00:45.918 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 577.14617167999995, 2024-09-20T19:00:45.918 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.112670376, 2024-09-20T19:00:45.918 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:45.918 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:45.918 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:45.918 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:45.918 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T19:00:45.918 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T19:00:45.918 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 27560 2024-09-20T19:00:45.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.919 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:45.919 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.919 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:45.919 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736128+0000", 2024-09-20T19:00:45.919 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:45.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.919 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.919 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:45.919 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736128+0000", 2024-09-20T19:00:45.920 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:45.920 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.920 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.920 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:45.920 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736129+0000", 2024-09-20T19:00:45.920 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.251e-06 2024-09-20T19:00:45.920 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.920 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.920 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:45.920 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736150+0000", 2024-09-20T19:00:45.920 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1450999999999998e-05 2024-09-20T19:00:45.920 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.920 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.921 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:45.921 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736161+0000", 2024-09-20T19:00:45.921 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0682e-05 2024-09-20T19:00:45.921 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.921 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.921 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:45.921 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736168+0000", 2024-09-20T19:00:45.921 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8549999999999996e-06 2024-09-20T19:00:45.921 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.921 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.921 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:45.921 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736218+0000", 2024-09-20T19:00:45.921 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0497000000000001e-05 2024-09-20T19:00:45.922 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.922 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.922 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:45.922 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.736327+0000", 2024-09-20T19:00:45.922 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010837599999999999 2024-09-20T19:00:45.922 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.922 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.922 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T19:00:45.922 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.841982+0000", 2024-09-20T19:00:45.922 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.105654846 2024-09-20T19:00:45.922 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.922 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.923 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:00:45.923 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.845386+0000", 2024-09-20T19:00:45.923 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0034042719999999999 2024-09-20T19:00:45.923 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.923 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.923 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:45.923 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.845423+0000", 2024-09-20T19:00:45.923 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7160000000000003e-05 2024-09-20T19:00:45.923 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.923 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.923 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:45.923 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.848702+0000", 2024-09-20T19:00:45.924 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0032792099999999999 2024-09-20T19:00:45.924 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.924 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.924 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:45.924 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.848758+0000", 2024-09-20T19:00:45.924 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.537e-05 2024-09-20T19:00:45.924 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.924 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.924 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:45.924 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:51:08.848798+0000", 2024-09-20T19:00:45.924 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0405999999999999e-05 2024-09-20T19:00:45.924 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:45.924 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:45.925 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:45.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.925 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.925 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:117083 7.e 7:7035ec2c:::benchmark_data_smithi083_70643_object117082:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-20T19:00:45.925 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:52:21.789905+0000", 2024-09-20T19:00:45.925 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 504.09239438100002, 2024-09-20T19:00:45.925 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.1098601, 2024-09-20T19:00:45.925 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:45.925 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:45.925 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:45.925 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:45.925 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T19:00:45.926 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T19:00:45.926 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 117083 2024-09-20T19:00:45.926 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.926 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:45.926 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.926 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:45.926 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.789905+0000", 2024-09-20T19:00:45.926 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:45.926 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.926 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.926 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:45.926 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.789905+0000", 2024-09-20T19:00:45.926 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:45.927 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.927 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.927 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:45.927 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.789906+0000", 2024-09-20T19:00:45.927 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4109999999999999e-06 2024-09-20T19:00:45.927 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.927 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.928 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:45.928 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.789929+0000", 2024-09-20T19:00:45.928 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3212000000000001e-05 2024-09-20T19:00:45.928 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.928 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.928 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:45.928 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.789941+0000", 2024-09-20T19:00:45.928 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1103e-05 2024-09-20T19:00:45.928 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.928 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.928 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:45.928 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.789948+0000", 2024-09-20T19:00:45.929 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.968e-06 2024-09-20T19:00:45.929 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.929 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.929 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:45.929 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.790013+0000", 2024-09-20T19:00:45.929 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5518000000000005e-05 2024-09-20T19:00:45.929 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.929 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.929 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:45.929 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.790130+0000", 2024-09-20T19:00:45.929 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000116906 2024-09-20T19:00:45.929 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.929 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.929 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T19:00:45.930 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.790270+0000", 2024-09-20T19:00:45.930 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013975700000000001 2024-09-20T19:00:45.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.930 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.930 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:00:45.930 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.793553+0000", 2024-09-20T19:00:45.930 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003283589 2024-09-20T19:00:45.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.930 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.930 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:45.930 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.799223+0000", 2024-09-20T19:00:45.930 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0056701340000000003 2024-09-20T19:00:45.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.931 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.931 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:45.931 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.899580+0000", 2024-09-20T19:00:45.931 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.100356374 2024-09-20T19:00:45.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.931 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.931 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:45.931 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.899639+0000", 2024-09-20T19:00:45.931 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8990000000000003e-05 2024-09-20T19:00:45.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.931 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.931 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:45.932 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:52:21.899765+0000", 2024-09-20T19:00:45.932 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000126138 2024-09-20T19:00:45.932 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:45.932 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:45.932 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:45.932 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.932 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.932 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:187757 7.e 7:70705a0e:::benchmark_data_smithi083_70643_object64955:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-20T19:00:45.932 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:53:29.029730+0000", 2024-09-20T19:00:45.932 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 436.85256874200002, 2024-09-20T19:00:45.932 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.137882645, 2024-09-20T19:00:45.933 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:45.933 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:45.933 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:45.933 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:45.933 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T19:00:45.933 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T19:00:45.933 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 187757 2024-09-20T19:00:45.933 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.933 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:45.933 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.933 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:45.933 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.029730+0000", 2024-09-20T19:00:45.934 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:45.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.934 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.934 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:45.934 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.029730+0000", 2024-09-20T19:00:45.934 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:45.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.934 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.934 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:45.934 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.029732+0000", 2024-09-20T19:00:45.934 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.415e-06 2024-09-20T19:00:45.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.935 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:45.935 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.029737+0000", 2024-09-20T19:00:45.935 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0239999999999997e-06 2024-09-20T19:00:45.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.935 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:45.935 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.029748+0000", 2024-09-20T19:00:45.935 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0920999999999999e-05 2024-09-20T19:00:45.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.935 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:45.936 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.029755+0000", 2024-09-20T19:00:45.936 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1430000000000001e-06 2024-09-20T19:00:45.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.936 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.936 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:45.936 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.030815+0000", 2024-09-20T19:00:45.936 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0010603590000000001 2024-09-20T19:00:45.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.936 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.936 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:45.936 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.030930+0000", 2024-09-20T19:00:45.936 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000114171 2024-09-20T19:00:45.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.937 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.937 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T19:00:45.937 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.031080+0000", 2024-09-20T19:00:45.937 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015042000000000001 2024-09-20T19:00:45.937 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.937 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.937 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:00:45.937 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.036072+0000", 2024-09-20T19:00:45.937 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049921159999999996 2024-09-20T19:00:45.937 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.937 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.937 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:45.938 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.037324+0000", 2024-09-20T19:00:45.938 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0012515930000000001 2024-09-20T19:00:45.938 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.938 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.938 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:45.938 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045902+0000", 2024-09-20T19:00:45.938 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0085779580000000001 2024-09-20T19:00:45.938 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.938 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.938 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:45.938 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.167564+0000", 2024-09-20T19:00:45.938 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.121662143 2024-09-20T19:00:45.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.939 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.939 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:45.939 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.167613+0000", 2024-09-20T19:00:45.939 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9382000000000003e-05 2024-09-20T19:00:45.939 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:45.939 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:45.939 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:45.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.939 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.939 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:187765 7.e 7:7e1cec20:::benchmark_data_smithi083_70643_object64963:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-20T19:00:45.939 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:53:29.040056+0000", 2024-09-20T19:00:45.940 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 436.84224365599999, 2024-09-20T19:00:45.940 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.132718637, 2024-09-20T19:00:45.940 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:45.940 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:45.940 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:45.940 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:45.940 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T19:00:45.940 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T19:00:45.940 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 187765 2024-09-20T19:00:45.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.940 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:45.941 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.941 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:45.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.040056+0000", 2024-09-20T19:00:45.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:45.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.941 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.941 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:45.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.040056+0000", 2024-09-20T19:00:45.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:45.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.941 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.941 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:45.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.040057+0000", 2024-09-20T19:00:45.942 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.528e-06 2024-09-20T19:00:45.942 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.942 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.942 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:45.942 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.040063+0000", 2024-09-20T19:00:45.942 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8170000000000004e-06 2024-09-20T19:00:45.942 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.942 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.942 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:45.942 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.040074+0000", 2024-09-20T19:00:45.942 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1253000000000001e-05 2024-09-20T19:00:45.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.943 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.943 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:45.943 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.040081+0000", 2024-09-20T19:00:45.943 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1389999999999998e-06 2024-09-20T19:00:45.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:45.944 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:45.945 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:45.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.945 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:45.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.040134+0000", 2024-09-20T19:00:45.946 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2549000000000001e-05 2024-09-20T19:00:45.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.946 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:45.946 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.040246+0000", 2024-09-20T19:00:45.946 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000112271 2024-09-20T19:00:45.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.946 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T19:00:45.946 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.040354+0000", 2024-09-20T19:00:45.946 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010791 2024-09-20T19:00:45.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.947 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.947 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:45.947 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045867+0000", 2024-09-20T19:00:45.947 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0055125449999999998 2024-09-20T19:00:45.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.947 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.947 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:00:45.947 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.172613+0000", 2024-09-20T19:00:45.947 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.12674679599999999 2024-09-20T19:00:45.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.947 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.947 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:45.948 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.172691+0000", 2024-09-20T19:00:45.948 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7255000000000005e-05 2024-09-20T19:00:45.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.948 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:45.948 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.172735+0000", 2024-09-20T19:00:45.948 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4069000000000001e-05 2024-09-20T19:00:45.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.948 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:45.948 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.172774+0000", 2024-09-20T19:00:45.948 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9505000000000002e-05 2024-09-20T19:00:45.949 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:45.949 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:45.949 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:45.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.949 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:187763 7.3 e141/83 7:c71c27bb:::benchmark_data_smithi083_70643_object64961:head v 141'11587, mlcod=141'11584)", 2024-09-20T19:00:45.949 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:53:29.043149+0000", 2024-09-20T19:00:45.949 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 436.83914997199997, 2024-09-20T19:00:45.949 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.125479285, 2024-09-20T19:00:45.949 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:45.949 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:45.949 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:45.950 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:45.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:45.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043149+0000", 2024-09-20T19:00:45.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:45.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:45.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043149+0000", 2024-09-20T19:00:45.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:45.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.951 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.951 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:45.951 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043150+0000", 2024-09-20T19:00:45.951 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7800000000000001e-07 2024-09-20T19:00:45.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.951 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.951 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:45.951 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043154+0000", 2024-09-20T19:00:45.951 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6480000000000002e-06 2024-09-20T19:00:45.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.951 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.951 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:45.952 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043162+0000", 2024-09-20T19:00:45.952 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1810000000000004e-06 2024-09-20T19:00:45.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.952 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.952 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:45.952 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043167+0000", 2024-09-20T19:00:45.952 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4500000000000003e-06 2024-09-20T19:00:45.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.952 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.952 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:45.952 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043227+0000", 2024-09-20T19:00:45.953 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9697999999999997e-05 2024-09-20T19:00:45.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.953 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.953 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:45.953 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043243+0000", 2024-09-20T19:00:45.953 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6461999999999999e-05 2024-09-20T19:00:45.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.953 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.953 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:45.953 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.168513+0000", 2024-09-20T19:00:45.953 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.125269624 2024-09-20T19:00:45.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:45.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.168629+0000", 2024-09-20T19:00:45.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011564400000000001 2024-09-20T19:00:45.954 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:45.954 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:45.954 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:45.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.954 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:187773 7.e 7:7168b707:::benchmark_data_smithi083_70643_object64971:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-20T19:00:45.954 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:53:29.043782+0000", 2024-09-20T19:00:45.955 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 436.838517244, 2024-09-20T19:00:45.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.14065076300000001, 2024-09-20T19:00:45.955 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:45.955 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:45.955 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:45.955 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:45.955 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T19:00:45.955 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T19:00:45.955 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 187773 2024-09-20T19:00:45.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.955 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:45.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:45.956 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043782+0000", 2024-09-20T19:00:45.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:45.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.956 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:45.956 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043782+0000", 2024-09-20T19:00:45.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:45.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.956 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:45.956 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043784+0000", 2024-09-20T19:00:45.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5260000000000001e-06 2024-09-20T19:00:45.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.957 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.957 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:45.957 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043788+0000", 2024-09-20T19:00:45.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.481e-06 2024-09-20T19:00:45.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.957 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.957 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:45.957 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043798+0000", 2024-09-20T19:00:45.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0393e-05 2024-09-20T19:00:45.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.958 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:45.958 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043805+0000", 2024-09-20T19:00:45.958 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5030000000000002e-06 2024-09-20T19:00:45.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.958 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:45.958 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.167654+0000", 2024-09-20T19:00:45.958 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.123848659 2024-09-20T19:00:45.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.958 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:45.959 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.167881+0000", 2024-09-20T19:00:45.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000227698 2024-09-20T19:00:45.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.959 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.959 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T19:00:45.959 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.167981+0000", 2024-09-20T19:00:45.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9611999999999994e-05 2024-09-20T19:00:45.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.959 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.959 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:45.959 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.172817+0000", 2024-09-20T19:00:45.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0048360829999999997 2024-09-20T19:00:45.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.960 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:00:45.960 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.172828+0000", 2024-09-20T19:00:45.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1280999999999999e-05 2024-09-20T19:00:45.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.960 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:45.960 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.184218+0000", 2024-09-20T19:00:45.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011389882 2024-09-20T19:00:45.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.961 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:45.961 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.184387+0000", 2024-09-20T19:00:45.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000168992 2024-09-20T19:00:45.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.961 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:45.961 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.184433+0000", 2024-09-20T19:00:45.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5652999999999998e-05 2024-09-20T19:00:45.961 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:45.961 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:45.961 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:45.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.962 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.962 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:187757 7.e e141/83 ondisk, result = 0)", 2024-09-20T19:00:45.962 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:53:29.045594+0000", 2024-09-20T19:00:45.962 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 436.83670529, 2024-09-20T19:00:45.962 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.12203122600000001, 2024-09-20T19:00:45.962 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:45.962 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:45.962 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T19:00:45.962 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:45.962 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.962 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:45.963 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045594+0000", 2024-09-20T19:00:45.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:45.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:45.963 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045594+0000", 2024-09-20T19:00:45.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:45.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:45.963 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045595+0000", 2024-09-20T19:00:45.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6499999999999998e-07 2024-09-20T19:00:45.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.964 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.964 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:45.964 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045599+0000", 2024-09-20T19:00:45.964 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6160000000000004e-06 2024-09-20T19:00:45.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.964 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.964 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:45.964 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045609+0000", 2024-09-20T19:00:45.964 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2269999999999999e-06 2024-09-20T19:00:45.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.964 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.965 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:45.965 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045615+0000", 2024-09-20T19:00:45.965 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1349999999999997e-06 2024-09-20T19:00:45.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.965 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.965 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:45.965 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045892+0000", 2024-09-20T19:00:45.965 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000276724 2024-09-20T19:00:45.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.965 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.965 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:45.965 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045897+0000", 2024-09-20T19:00:45.965 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2970000000000003e-06 2024-09-20T19:00:45.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.966 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.966 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:45.966 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.167625+0000", 2024-09-20T19:00:45.966 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.12172836200000001 2024-09-20T19:00:45.966 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:45.966 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:45.966 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:45.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.966 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.966 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:187772 7.a e141/83 7:5948c6a4:::benchmark_data_smithi083_70643_object64970:head v 141'11671, mlcod=141'11669)", 2024-09-20T19:00:45.966 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:53:29.045936+0000", 2024-09-20T19:00:45.967 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 436.83636359399998, 2024-09-20T19:00:45.967 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.12458983799999999, 2024-09-20T19:00:45.967 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:45.967 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:45.967 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:45.967 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:45.967 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.967 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:45.967 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045936+0000", 2024-09-20T19:00:45.967 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:45.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.967 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.968 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:45.968 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045936+0000", 2024-09-20T19:00:45.968 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:45.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.968 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.968 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:45.968 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045936+0000", 2024-09-20T19:00:45.968 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2900000000000003e-07 2024-09-20T19:00:45.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.968 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.968 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:45.968 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045942+0000", 2024-09-20T19:00:45.968 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.66e-06 2024-09-20T19:00:45.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.969 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:45.969 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.165480+0000", 2024-09-20T19:00:45.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.119537582 2024-09-20T19:00:45.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.969 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:45.969 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.165496+0000", 2024-09-20T19:00:45.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6249999999999999e-05 2024-09-20T19:00:45.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.970 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:45.970 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.165558+0000", 2024-09-20T19:00:45.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2120999999999997e-05 2024-09-20T19:00:45.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.970 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:45.970 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.165582+0000", 2024-09-20T19:00:45.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4409e-05 2024-09-20T19:00:45.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.970 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:45.971 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.170489+0000", 2024-09-20T19:00:45.971 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049066860000000004 2024-09-20T19:00:45.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.971 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.971 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:45.971 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.170525+0000", 2024-09-20T19:00:45.971 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6400999999999999e-05 2024-09-20T19:00:45.971 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:45.971 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:45.971 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:45.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.971 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.972 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:187774 7.e 7:778445cb:::benchmark_data_smithi083_70643_object64972:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-20T19:00:45.972 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:53:29.046524+0000", 2024-09-20T19:00:45.972 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 436.83577516899999, 2024-09-20T19:00:45.972 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.15650777099999999, 2024-09-20T19:00:45.972 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:45.972 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:45.972 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:45.972 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:45.972 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T19:00:45.972 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T19:00:45.972 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 187774 2024-09-20T19:00:45.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.973 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:45.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.973 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:45.973 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.046524+0000", 2024-09-20T19:00:45.973 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:45.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.973 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:45.973 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.046524+0000", 2024-09-20T19:00:45.973 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:45.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.974 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:45.974 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.046526+0000", 2024-09-20T19:00:45.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.494e-06 2024-09-20T19:00:45.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.974 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:45.974 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.046531+0000", 2024-09-20T19:00:45.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4840000000000003e-06 2024-09-20T19:00:45.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.975 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.975 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:45.975 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.165656+0000", 2024-09-20T19:00:45.975 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.11912532100000001 2024-09-20T19:00:45.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.975 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.975 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:45.975 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.165666+0000", 2024-09-20T19:00:45.975 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1789999999999997e-06 2024-09-20T19:00:45.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.975 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.975 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:45.976 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.173061+0000", 2024-09-20T19:00:45.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0073953969999999997 2024-09-20T19:00:45.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.976 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:45.976 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.173154+0000", 2024-09-20T19:00:45.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3113000000000001e-05 2024-09-20T19:00:45.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.976 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T19:00:45.976 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.173250+0000", 2024-09-20T19:00:45.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5493000000000002e-05 2024-09-20T19:00:45.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:45.977 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.180360+0000", 2024-09-20T19:00:45.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0071109220000000004 2024-09-20T19:00:45.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:00:45.977 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.188345+0000", 2024-09-20T19:00:45.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0079849840000000005 2024-09-20T19:00:45.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.978 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:45.978 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.202813+0000", 2024-09-20T19:00:45.978 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.014467647 2024-09-20T19:00:45.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.978 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:45.978 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.202985+0000", 2024-09-20T19:00:45.978 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017191999999999999 2024-09-20T19:00:45.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.978 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:45.979 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.203032+0000", 2024-09-20T19:00:45.979 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6817000000000003e-05 2024-09-20T19:00:45.979 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:45.993 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:45.993 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:45.994 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.994 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.994 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:187765 7.e e141/83 ondisk, result = 0)", 2024-09-20T19:00:45.994 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:53:29.046987+0000", 2024-09-20T19:00:45.994 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 436.83531217000001, 2024-09-20T19:00:45.994 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.12579221700000001, 2024-09-20T19:00:45.994 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:45.994 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:45.994 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T19:00:45.994 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:45.994 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.994 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:45.995 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.046987+0000", 2024-09-20T19:00:45.995 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:45.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.995 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.995 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:45.995 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.046987+0000", 2024-09-20T19:00:45.995 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:45.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.995 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.995 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:45.995 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.046988+0000", 2024-09-20T19:00:45.995 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6499999999999999e-07 2024-09-20T19:00:45.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.996 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.996 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:45.996 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.046992+0000", 2024-09-20T19:00:45.996 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9650000000000002e-06 2024-09-20T19:00:45.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.996 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.996 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:45.996 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.165597+0000", 2024-09-20T19:00:45.996 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.118605087 2024-09-20T19:00:45.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.997 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.997 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:45.997 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.165603+0000", 2024-09-20T19:00:45.997 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0100000000000001e-06 2024-09-20T19:00:45.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.997 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.997 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:45.997 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.172679+0000", 2024-09-20T19:00:45.997 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0070759810000000003 2024-09-20T19:00:45.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.997 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.997 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:45.998 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.172686+0000", 2024-09-20T19:00:45.998 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9959999999999996e-06 2024-09-20T19:00:45.998 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.998 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.998 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:45.998 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.172779+0000", 2024-09-20T19:00:45.998 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3512999999999997e-05 2024-09-20T19:00:45.998 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:45.998 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:45.998 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:45.998 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:45.999 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.999 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.24955.0:218218 8.6 e233/182 8:6660910c:::benchmark_data_smithi083_97422_object102127:head v 233'13570, mlcod=210'8949)", 2024-09-20T19:00:45.999 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:58:02.122025+0000", 2024-09-20T19:00:45.999 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 163.76027402400001, 2024-09-20T19:00:45.999 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.112690573, 2024-09-20T19:00:45.999 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:45.999 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:45.999 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:45.999 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:45.999 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:45.999 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:46.000 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.122025+0000", 2024-09-20T19:00:46.000 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:46.000 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.000 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.000 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:46.000 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.122025+0000", 2024-09-20T19:00:46.000 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:46.000 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.000 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.000 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:46.000 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.122026+0000", 2024-09-20T19:00:46.001 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2399999999999996e-07 2024-09-20T19:00:46.001 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.001 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.001 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:46.001 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.122033+0000", 2024-09-20T19:00:46.001 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.562e-06 2024-09-20T19:00:46.001 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.001 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.001 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:46.001 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.214763+0000", 2024-09-20T19:00:46.001 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.092730564000000001 2024-09-20T19:00:46.001 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.002 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.002 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:46.002 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.214772+0000", 2024-09-20T19:00:46.002 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7979999999999995e-06 2024-09-20T19:00:46.002 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.002 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.002 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:46.002 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.214856+0000", 2024-09-20T19:00:46.002 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4146000000000004e-05 2024-09-20T19:00:46.002 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.002 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.003 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:46.003 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.214878+0000", 2024-09-20T19:00:46.003 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1722e-05 2024-09-20T19:00:46.003 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.003 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.003 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:46.003 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.234612+0000", 2024-09-20T19:00:46.003 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019734267 2024-09-20T19:00:46.003 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.003 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.003 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:46.003 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.234716+0000", 2024-09-20T19:00:46.004 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010359 2024-09-20T19:00:46.004 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:46.004 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:46.004 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:46.004 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.004 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.004 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.25484.0:73676 9.2 e265/246 9:4d342ee9:::benchmark_data_smithi083_124552_object73675:head v 265'4551, mlcod=265'4548)", 2024-09-20T19:00:46.004 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:59:27.940611+0000", 2024-09-20T19:00:46.004 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 77.941688329000002, 2024-09-20T19:00:46.004 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.10976248500000001, 2024-09-20T19:00:46.004 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:46.004 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:46.005 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:46.005 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:46.005 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.005 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:46.005 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:27.940611+0000", 2024-09-20T19:00:46.005 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:46.005 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.005 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.005 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:46.005 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:27.940611+0000", 2024-09-20T19:00:46.005 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:46.006 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.006 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.006 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:46.006 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:27.940612+0000", 2024-09-20T19:00:46.006 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.271e-06 2024-09-20T19:00:46.006 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.006 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.006 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:46.006 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:27.940637+0000", 2024-09-20T19:00:46.006 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5219999999999999e-05 2024-09-20T19:00:46.006 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.006 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.007 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:46.007 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:28.036375+0000", 2024-09-20T19:00:46.007 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.095737696999999997 2024-09-20T19:00:46.007 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.007 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.007 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:46.007 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:28.036388+0000", 2024-09-20T19:00:46.007 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3001e-05 2024-09-20T19:00:46.007 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.007 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.007 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:46.007 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:28.038717+0000", 2024-09-20T19:00:46.007 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0023287429999999999 2024-09-20T19:00:46.008 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.008 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.008 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:46.008 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:28.038864+0000", 2024-09-20T19:00:46.008 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014680800000000001 2024-09-20T19:00:46.008 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.008 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.008 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:46.008 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:28.050214+0000", 2024-09-20T19:00:46.008 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011350813 2024-09-20T19:00:46.008 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.008 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.008 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:46.009 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:28.050373+0000", 2024-09-20T19:00:46.009 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000158932 2024-09-20T19:00:46.009 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:46.009 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:46.009 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:46.009 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.009 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.009 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.25484.0:124578 9.2 e289/246 9:422abb8a:::benchmark_data_smithi083_124552_object10431:head v 289'7686, mlcod=289'7682)", 2024-09-20T19:00:46.009 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:20.951824+0000", 2024-09-20T19:00:46.010 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 24.930475481999999, 2024-09-20T19:00:46.010 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.32891357100000002, 2024-09-20T19:00:46.010 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:46.010 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:46.011 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:46.011 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:46.011 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.011 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:46.011 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.951824+0000", 2024-09-20T19:00:46.011 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:46.011 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.011 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.011 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:46.011 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.951824+0000", 2024-09-20T19:00:46.011 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:46.011 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.011 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.012 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:46.012 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.951825+0000", 2024-09-20T19:00:46.012 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0470000000000001e-06 2024-09-20T19:00:46.012 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.012 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.012 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:46.012 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.951831+0000", 2024-09-20T19:00:46.012 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2999999999999998e-06 2024-09-20T19:00:46.012 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.012 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.012 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:46.013 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.951841+0000", 2024-09-20T19:00:46.013 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0234999999999999e-05 2024-09-20T19:00:46.013 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.013 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.013 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:46.013 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.951854+0000", 2024-09-20T19:00:46.013 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.287e-05 2024-09-20T19:00:46.013 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.013 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.013 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:46.013 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.951926+0000", 2024-09-20T19:00:46.013 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1932000000000002e-05 2024-09-20T19:00:46.014 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.014 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.014 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:46.014 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.951943+0000", 2024-09-20T19:00:46.014 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7033000000000001e-05 2024-09-20T19:00:46.014 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.014 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.014 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:46.014 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.280700+0000", 2024-09-20T19:00:46.014 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.328756979 2024-09-20T19:00:46.014 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.015 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.015 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:46.015 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.280737+0000", 2024-09-20T19:00:46.015 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7175000000000002e-05 2024-09-20T19:00:46.015 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:46.015 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:46.015 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:46.015 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.015 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.015 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.25484.0:124584 9.0 e289/284 9:06443903:::benchmark_data_smithi083_124552_object10437:head v 289'8049, mlcod=289'8046)", 2024-09-20T19:00:46.015 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:20.953013+0000", 2024-09-20T19:00:46.015 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 24.929286169000001, 2024-09-20T19:00:46.016 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.323892601, 2024-09-20T19:00:46.016 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:46.016 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:46.016 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:46.016 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:46.016 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.016 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:46.016 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.953013+0000", 2024-09-20T19:00:46.016 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:46.016 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.017 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.017 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:46.017 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.953013+0000", 2024-09-20T19:00:46.017 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:46.017 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.017 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.017 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:46.017 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.953014+0000", 2024-09-20T19:00:46.017 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0999999999999997e-07 2024-09-20T19:00:46.017 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.017 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.018 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:46.018 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.953020+0000", 2024-09-20T19:00:46.018 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8390000000000003e-06 2024-09-20T19:00:46.018 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.018 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.018 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:46.018 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.953030+0000", 2024-09-20T19:00:46.018 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8530000000000004e-06 2024-09-20T19:00:46.018 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.018 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.018 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:46.019 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.953036+0000", 2024-09-20T19:00:46.019 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1510000000000001e-06 2024-09-20T19:00:46.019 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.019 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.019 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:46.019 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.953099+0000", 2024-09-20T19:00:46.019 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2959000000000005e-05 2024-09-20T19:00:46.019 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.019 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.019 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:46.019 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.953117+0000", 2024-09-20T19:00:46.020 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8559999999999998e-05 2024-09-20T19:00:46.020 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.020 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.020 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:46.020 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.959129+0000", 2024-09-20T19:00:46.020 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0060115500000000001 2024-09-20T19:00:46.020 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.020 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.020 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:46.020 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.276906+0000", 2024-09-20T19:00:46.020 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.31777687900000001 2024-09-20T19:00:46.021 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:46.021 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:46.021 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:46.021 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.021 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.021 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.25484.0:124590 9.6 e289/246 9:6ce021c9:::benchmark_data_smithi083_124552_object10443:head v 289'7744, mlcod=289'7742)", 2024-09-20T19:00:46.021 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:20.956000+0000", 2024-09-20T19:00:46.021 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 24.926298929000001, 2024-09-20T19:00:46.021 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.32402082500000001, 2024-09-20T19:00:46.021 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:46.021 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:46.021 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:46.022 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:46.022 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.022 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:46.022 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.956000+0000", 2024-09-20T19:00:46.022 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:46.022 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.022 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.022 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:46.022 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.956000+0000", 2024-09-20T19:00:46.022 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:46.022 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.022 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.023 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:46.023 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.956001+0000", 2024-09-20T19:00:46.023 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9600000000000008e-07 2024-09-20T19:00:46.023 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.023 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.023 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:46.023 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.956007+0000", 2024-09-20T19:00:46.023 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3199999999999999e-06 2024-09-20T19:00:46.023 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.023 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.023 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:46.023 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.956014+0000", 2024-09-20T19:00:46.023 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7880000000000007e-06 2024-09-20T19:00:46.024 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.024 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.024 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:46.024 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.956019+0000", 2024-09-20T19:00:46.024 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5050000000000001e-06 2024-09-20T19:00:46.024 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.024 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.024 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:46.024 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.956069+0000", 2024-09-20T19:00:46.024 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0467000000000003e-05 2024-09-20T19:00:46.024 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.024 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.025 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:46.025 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.956089+0000", 2024-09-20T19:00:46.025 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.948e-05 2024-09-20T19:00:46.025 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.025 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.025 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:46.025 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.279983+0000", 2024-09-20T19:00:46.025 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.32389452099999999 2024-09-20T19:00:46.025 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.025 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.025 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:46.025 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.280021+0000", 2024-09-20T19:00:46.025 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7747999999999998e-05 2024-09-20T19:00:46.026 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:46.026 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:46.026 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:46.026 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.026 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.026 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.25484.0:124594 9.6 e289/246 9:67fbc174:::benchmark_data_smithi083_124552_object10447:head v 289'7745, mlcod=289'7742)", 2024-09-20T19:00:46.026 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:20.960552+0000", 2024-09-20T19:00:46.026 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 24.921747088, 2024-09-20T19:00:46.026 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.32917961699999998, 2024-09-20T19:00:46.026 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:46.026 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:46.026 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:46.027 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:46.027 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.027 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:46.027 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.960552+0000", 2024-09-20T19:00:46.027 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:46.027 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.027 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.027 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:46.027 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.960552+0000", 2024-09-20T19:00:46.027 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:46.027 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.027 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.028 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:46.028 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.960553+0000", 2024-09-20T19:00:46.028 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0610000000000001e-06 2024-09-20T19:00:46.028 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.028 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.028 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:46.028 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.960562+0000", 2024-09-20T19:00:46.028 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0580000000000004e-06 2024-09-20T19:00:46.028 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.028 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.028 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:46.028 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.276862+0000", 2024-09-20T19:00:46.029 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.31629938200000002 2024-09-20T19:00:46.029 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.029 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.029 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:46.029 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.276872+0000", 2024-09-20T19:00:46.029 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0614e-05 2024-09-20T19:00:46.029 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.029 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.029 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:46.029 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.276943+0000", 2024-09-20T19:00:46.029 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0933999999999995e-05 2024-09-20T19:00:46.029 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.029 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.030 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:46.030 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.276962+0000", 2024-09-20T19:00:46.030 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9293999999999999e-05 2024-09-20T19:00:46.030 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.030 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.030 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:46.030 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.289671+0000", 2024-09-20T19:00:46.030 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.012708129 2024-09-20T19:00:46.030 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.030 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.030 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:46.030 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.289732+0000", 2024-09-20T19:00:46.031 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1144999999999998e-05 2024-09-20T19:00:46.031 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:46.031 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:46.031 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:46.031 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.031 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.031 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.25484.0:124596 9.2 e289/246 9:42d01fa2:::benchmark_data_smithi083_124552_object10449:head v 289'7687, mlcod=289'7684)", 2024-09-20T19:00:46.031 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:20.963075+0000", 2024-09-20T19:00:46.031 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 24.919224332999999, 2024-09-20T19:00:46.031 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.32681833599999999, 2024-09-20T19:00:46.031 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:46.032 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:46.032 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:00:46.032 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:46.032 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.032 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:46.032 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.963075+0000", 2024-09-20T19:00:46.032 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:46.032 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.032 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.032 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:46.032 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.963075+0000", 2024-09-20T19:00:46.032 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:46.032 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.033 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.033 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:46.033 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.963076+0000", 2024-09-20T19:00:46.033 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6600000000000005e-07 2024-09-20T19:00:46.033 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.033 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.033 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:46.033 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.963083+0000", 2024-09-20T19:00:46.033 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9140000000000002e-06 2024-09-20T19:00:46.033 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.033 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.033 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:46.034 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.276845+0000", 2024-09-20T19:00:46.034 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.31376241599999999 2024-09-20T19:00:46.034 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.034 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.034 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:46.034 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.276871+0000", 2024-09-20T19:00:46.034 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6123999999999999e-05 2024-09-20T19:00:46.034 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.034 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.034 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:46.034 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.276934+0000", 2024-09-20T19:00:46.034 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3147999999999999e-05 2024-09-20T19:00:46.035 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.035 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.035 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:46.035 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.276960+0000", 2024-09-20T19:00:46.035 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5939e-05 2024-09-20T19:00:46.035 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.035 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.035 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:00:46.035 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.289643+0000", 2024-09-20T19:00:46.035 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.012682236 2024-09-20T19:00:46.035 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:46.035 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:46.036 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:00:46.036 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.289893+0000", 2024-09-20T19:00:46.036 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00025069300000000002 2024-09-20T19:00:46.036 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:46.036 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:46.036 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:46.036 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:46.036 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:46.036 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T19:00:46.045 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:45 smithi083 ceph-mon[29421]: osdmap e296: 8 total, 7 up, 8 in 2024-09-20T19:00:46.045 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:45 smithi083 ceph-mon[29421]: pgmap v540: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 4.8 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 654 B/s rd, 0 B/s wr, 911 op/s; 14706/236118 objects degraded (6.228%) 2024-09-20T19:00:46.046 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:45 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/474639697' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:00:46.046 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:45 smithi083 ceph-mon[36260]: osdmap e296: 8 total, 7 up, 8 in 2024-09-20T19:00:46.046 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:45 smithi083 ceph-mon[36260]: pgmap v540: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 4.8 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 654 B/s rd, 0 B/s wr, 911 op/s; 14706/236118 objects degraded (6.228%) 2024-09-20T19:00:46.046 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:45 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/474639697' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:00:46.046 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:00:46.075 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:46.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:46.077 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:00:46.177 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:46.225 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:46.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:46.226 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:46.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:45 smithi142 ceph-mon[32508]: osdmap e296: 8 total, 7 up, 8 in 2024-09-20T19:00:46.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:45 smithi142 ceph-mon[32508]: pgmap v540: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 4.8 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 654 B/s rd, 0 B/s wr, 911 op/s; 14706/236118 objects degraded (6.228%) 2024-09-20T19:00:46.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:45 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/474639697' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:00:46.327 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:46.360 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:46.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:46.361 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:46.462 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:46.499 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:46.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:46.500 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:46.600 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:46.635 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:46.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:46.636 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:46.641 INFO:teuthology.orchestra.run.smithi142.stderr:Creating an OSD daemon form without an OSD FSID value 2024-09-20T19:00:46.737 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:46.772 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:46.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:46.773 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:46.874 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:46.913 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:46.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:46.914 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:47.014 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:47.040 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:47.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:47.041 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:47.142 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:47.175 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:47.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:47.176 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:47.265 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:00:46 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:00:47.266 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:46 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:00:47.266 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:46 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:00:47.277 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:47.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:46 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:00:47.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:46 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:00:47.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:46 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:00:47.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:46 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:00:47.314 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:47.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:47.315 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:47.416 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:47.451 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:47.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:47.452 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:47.553 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:47.585 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:47.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:47.586 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:47.687 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:00:47.722 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:47.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:47.723 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:00:47.824 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:47.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:47.860 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:47.860 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:47.961 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:47.996 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:47.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:47.998 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:48.098 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:48.124 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:48.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:48.125 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:48.226 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:48.261 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:48.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:48.262 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:48.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:47 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:00:48.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:47 smithi142 ceph-mon[32508]: pgmap v541: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 4.5 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.43k op/s; 13482/220290 objects degraded (6.120%) 2024-09-20T19:00:48.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:47 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:00:48.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:47 smithi083 ceph-mon[29421]: pgmap v541: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 4.5 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.43k op/s; 13482/220290 objects degraded (6.120%) 2024-09-20T19:00:48.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:47 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:00:48.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:47 smithi083 ceph-mon[36260]: pgmap v541: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 4.5 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.43k op/s; 13482/220290 objects degraded (6.120%) 2024-09-20T19:00:48.363 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:48.389 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:48.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:48.390 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:48.491 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:48.512 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:48.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:48.513 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:48.614 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:48.645 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:48.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:48.646 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:48.747 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:48.785 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:48.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:48.786 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:48.887 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:48.919 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:48.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:48.920 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:49.020 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:49.055 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:49.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:49.056 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:49.078 INFO:teuthology.orchestra.run.smithi142.stderr:Exporting 5.1f info 5.1f( v 262'10 (0'0,262'10] local-lis/les=260/261 n=1 ec=62/62 lis/c=260/260 les/c/f=261/261/0 sis=260) 2024-09-20T19:00:49.080 INFO:teuthology.orchestra.run.smithi142.stderr:Read #5:f95f44c2:::notify.0:head# 2024-09-20T19:00:49.080 INFO:teuthology.orchestra.run.smithi142.stderr:Export successful 2024-09-20T19:00:49.157 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:49.197 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:49.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:49.198 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:49.299 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:49.337 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:49.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:49.338 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:49.378 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:49 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 13482/216561 objects degraded (6.225%), 27 pgs degraded (PG_DEGRADED) 2024-09-20T19:00:49.439 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:49.466 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:49.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:49.467 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:49.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:49 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 13482/216561 objects degraded (6.225%), 27 pgs degraded (PG_DEGRADED) 2024-09-20T19:00:49.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:49 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 13482/216561 objects degraded (6.225%), 27 pgs degraded (PG_DEGRADED) 2024-09-20T19:00:49.584 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:49.621 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:49.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:49.622 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:49.723 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:49.759 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:49.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:49.760 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:49.861 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:49.900 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:49.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:49.901 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:50.002 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:00:50.035 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:50.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:50.036 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:00:50.049 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell -n osd.4 --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-4 --journal-path /var/lib/ceph/osd/ceph-4/journal --force --op remove --pgid 5.1f 2024-09-20T19:00:50.137 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:00:50.174 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:50.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:50.175 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:00:50.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:50 smithi142 ceph-mon[32508]: pgmap v542: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 4.4 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.45k op/s; 13482/216561 objects degraded (6.225%) 2024-09-20T19:00:50.276 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:50.316 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:50.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:50.317 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:50.381 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/osd.4/config 2024-09-20T19:00:50.418 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:00:50.462 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:50.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:50.463 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:00:50.501 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:50 smithi083 ceph-mon[29421]: pgmap v542: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 4.4 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.45k op/s; 13482/216561 objects degraded (6.225%) 2024-09-20T19:00:50.509 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:50 smithi083 ceph-mon[36260]: pgmap v542: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 4.4 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.45k op/s; 13482/216561 objects degraded (6.225%) 2024-09-20T19:00:50.552 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-20T19:00:50.552 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-20T19:00:50.564 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:50.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:50.595 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:50.595 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:50.696 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:50.723 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:50.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:50.723 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:50.795 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:00:50 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:00:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:00:50.824 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:50.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:50.861 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:50.861 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:50.961 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:51.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:51.011 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:51.012 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:51.112 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:51.150 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:51.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:51.151 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:51.251 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:51.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:51 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T19:00:51.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:51 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T19:00:51.302 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:51.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:51.303 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:51.373 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-20T19:00:51.405 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:00:51.437 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:51.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:51.438 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:00:51.488 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:00:51.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:51 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T19:00:51.515 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 19:00:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=cleanup t=2024-09-20T19:00:51.216807512Z level=info msg="Completed cleanup jobs" duration=2.818452ms 2024-09-20T19:00:51.515 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 19:00:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=sqlstore.transactions t=2024-09-20T19:00:51.222133535Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-20T19:00:51.516 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 19:00:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=plugins.update.checker t=2024-09-20T19:00:51.370426274Z level=info msg="Update check succeeded" duration=64.695605ms 2024-09-20T19:00:51.539 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:51.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:51.608 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:51.608 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:51.709 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:51.741 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:51.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:51.742 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:51.818 INFO:teuthology.orchestra.run.smithi142.stderr:Creating an OSD daemon form without an OSD FSID value 2024-09-20T19:00:51.843 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:51.893 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:51.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:51.895 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:51.996 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:52.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:52.032 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:52.032 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:52.133 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:00:52.155 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:00:52.186 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:52.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:52.187 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:00:52.288 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:52.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:52 smithi083 ceph-mon[29421]: pgmap v543: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 4.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 947 op/s; 13032/212724 objects degraded (6.126%) 2024-09-20T19:00:52.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:52 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-20T19:00:52.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:52 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-20T19:00:52.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:52 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-20T19:00:52.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:52 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-20T19:00:52.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:52 smithi083 ceph-mon[29421]: Adjusting osd_memory_target on smithi142 to 3248M 2024-09-20T19:00:52.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:52 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:00:52.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:52 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-20T19:00:52.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:52 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-20T19:00:52.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:52 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-20T19:00:52.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:52 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-20T19:00:52.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:52 smithi083 ceph-mon[29421]: Adjusting osd_memory_target on smithi083 to 3248M 2024-09-20T19:00:52.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:52 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:00:52.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:52 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T19:00:52.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:52 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T19:00:52.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:52 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:00:52.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:52 smithi083 ceph-mon[36260]: pgmap v543: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 4.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 947 op/s; 13032/212724 objects degraded (6.126%) 2024-09-20T19:00:52.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:52 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-20T19:00:52.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:52 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-20T19:00:52.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:52 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-20T19:00:52.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:52 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-20T19:00:52.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:52 smithi083 ceph-mon[36260]: Adjusting osd_memory_target on smithi142 to 3248M 2024-09-20T19:00:52.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:52 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:00:52.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:52 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-20T19:00:52.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:52 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-20T19:00:52.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:52 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-20T19:00:52.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:52 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-20T19:00:52.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:52 smithi083 ceph-mon[36260]: Adjusting osd_memory_target on smithi083 to 3248M 2024-09-20T19:00:52.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:52 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:00:52.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:52 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T19:00:52.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:52 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T19:00:52.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:52 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:00:52.317 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:52.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:52.318 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:52.350 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:52 smithi142 ceph-mon[32508]: pgmap v543: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 4.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 947 op/s; 13032/212724 objects degraded (6.126%) 2024-09-20T19:00:52.350 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:52 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-20T19:00:52.350 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:52 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-20T19:00:52.350 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:52 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-20T19:00:52.350 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:52 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-20T19:00:52.350 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:52 smithi142 ceph-mon[32508]: Adjusting osd_memory_target on smithi142 to 3248M 2024-09-20T19:00:52.350 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:52 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:00:52.350 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:52 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-20T19:00:52.350 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:52 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-20T19:00:52.350 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:52 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-20T19:00:52.351 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:52 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-20T19:00:52.351 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:52 smithi142 ceph-mon[32508]: Adjusting osd_memory_target on smithi083 to 3248M 2024-09-20T19:00:52.351 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:52 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:00:52.351 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:52 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T19:00:52.351 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:52 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T19:00:52.351 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:52 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:00:52.418 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:00:52.449 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:52.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:52.450 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:00:52.550 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:52.553 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T19:00:52.588 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:52.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:52.589 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:52.690 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:52.751 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:52.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:52.752 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:52.768 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T19:00:52.768 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-20T19:00:52.768 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.768 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.25484.0:164188 9.1 e296/283)", 2024-09-20T19:00:52.768 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:52.749013+0000", 2024-09-20T19:00:52.768 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0076343160000000004, 2024-09-20T19:00:52.769 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0076563769999999998, 2024-09-20T19:00:52.769 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:52.769 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:52.769 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "queued for pg", 2024-09-20T19:00:52.769 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:52.769 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.769 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:52.769 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.749013+0000", 2024-09-20T19:00:52.769 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:52.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.769 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.769 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:52.769 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.749013+0000", 2024-09-20T19:00:52.770 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:52.770 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.770 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.770 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:52.770 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.749014+0000", 2024-09-20T19:00:52.770 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6599999999999996e-07 2024-09-20T19:00:52.770 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.770 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.770 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:52.770 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.749018+0000", 2024-09-20T19:00:52.770 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0409999999999999e-06 2024-09-20T19:00:52.770 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.771 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:52.771 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.749025+0000", 2024-09-20T19:00:52.771 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.6310000000000003e-06 2024-09-20T19:00:52.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.771 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:52.771 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.749031+0000", 2024-09-20T19:00:52.771 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5659999999999997e-06 2024-09-20T19:00:52.771 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:52.771 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:52.771 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:52.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.772 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.25484.0:164194 9.3 e296/246 9:cd2f28a4:::benchmark_data_smithi083_124552_object50047:head v 296'10337, mlcod=296'10333)", 2024-09-20T19:00:52.772 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:52.752236+0000", 2024-09-20T19:00:52.772 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0044119390000000001, 2024-09-20T19:00:52.772 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0044775270000000002, 2024-09-20T19:00:52.772 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:52.772 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:52.772 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T19:00:52.772 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:52.772 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.772 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:52.772 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.752236+0000", 2024-09-20T19:00:52.772 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:52.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.772 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.773 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:52.773 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.752236+0000", 2024-09-20T19:00:52.773 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:52.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.773 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.773 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:52.773 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.752237+0000", 2024-09-20T19:00:52.773 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0699999999999999e-06 2024-09-20T19:00:52.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.773 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.773 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:52.773 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.752244+0000", 2024-09-20T19:00:52.773 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.323e-06 2024-09-20T19:00:52.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.774 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:52.774 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.752256+0000", 2024-09-20T19:00:52.774 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1697e-05 2024-09-20T19:00:52.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.774 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:52.774 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.752263+0000", 2024-09-20T19:00:52.774 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5329999999999997e-06 2024-09-20T19:00:52.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.774 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:52.774 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.752397+0000", 2024-09-20T19:00:52.775 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013386600000000001 2024-09-20T19:00:52.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.775 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:52.775 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.752423+0000", 2024-09-20T19:00:52.775 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5976e-05 2024-09-20T19:00:52.775 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:52.775 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:52.775 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:52.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.775 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.25484.0:164193 9.4 e296/292 9:280ae61f:::benchmark_data_smithi083_124552_object50046:head v 296'10123, mlcod=296'10121)", 2024-09-20T19:00:52.775 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:52.752303+0000", 2024-09-20T19:00:52.775 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0043445890000000003, 2024-09-20T19:00:52.776 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0044573449999999997, 2024-09-20T19:00:52.776 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:52.776 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:52.776 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T19:00:52.776 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:52.776 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.776 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:52.776 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.752303+0000", 2024-09-20T19:00:52.776 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:52.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.776 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.776 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:52.776 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.752303+0000", 2024-09-20T19:00:52.776 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:52.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.777 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.777 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:52.777 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.752304+0000", 2024-09-20T19:00:52.777 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7300000000000005e-07 2024-09-20T19:00:52.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.777 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.777 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:52.777 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.752310+0000", 2024-09-20T19:00:52.777 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7050000000000002e-06 2024-09-20T19:00:52.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.777 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.777 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:52.777 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.752320+0000", 2024-09-20T19:00:52.777 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8910000000000007e-06 2024-09-20T19:00:52.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.778 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.778 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:52.778 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.752327+0000", 2024-09-20T19:00:52.778 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.353e-06 2024-09-20T19:00:52.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.778 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.778 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:52.778 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.752511+0000", 2024-09-20T19:00:52.778 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018423499999999999 2024-09-20T19:00:52.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.778 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.778 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:52.779 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.752531+0000", 2024-09-20T19:00:52.779 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9871999999999999e-05 2024-09-20T19:00:52.779 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:52.779 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:52.779 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:52.779 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.779 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.779 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.25484.0:164196 9.3 e296/246 9:c6e7c543:::benchmark_data_smithi083_124552_object50049:head v 296'10338, mlcod=296'10335)", 2024-09-20T19:00:52.779 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:52.752323+0000", 2024-09-20T19:00:52.779 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0043240700000000002, 2024-09-20T19:00:52.779 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0044801850000000002, 2024-09-20T19:00:52.779 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:52.779 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:52.779 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T19:00:52.780 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:52.780 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.780 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:52.780 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.752323+0000", 2024-09-20T19:00:52.780 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:52.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.780 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.780 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:52.780 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.752323+0000", 2024-09-20T19:00:52.780 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:52.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.780 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.780 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:52.780 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.752324+0000", 2024-09-20T19:00:52.781 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3900000000000005e-07 2024-09-20T19:00:52.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.781 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:52.781 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.752328+0000", 2024-09-20T19:00:52.781 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.766e-06 2024-09-20T19:00:52.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.781 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:52.781 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.752333+0000", 2024-09-20T19:00:52.781 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7590000000000003e-06 2024-09-20T19:00:52.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.781 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:52.781 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.752338+0000", 2024-09-20T19:00:52.782 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8180000000000003e-06 2024-09-20T19:00:52.782 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.782 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.782 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:52.782 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.755981+0000", 2024-09-20T19:00:52.782 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0036430619999999999 2024-09-20T19:00:52.782 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.782 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.782 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:52.782 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.756088+0000", 2024-09-20T19:00:52.782 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106943 2024-09-20T19:00:52.782 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:52.782 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:52.783 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:52.783 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.783 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.783 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:164195 9.8 9:18a58985:::benchmark_data_smithi083_124552_object50048:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e296)", 2024-09-20T19:00:52.783 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:52.752384+0000", 2024-09-20T19:00:52.783 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0042635939999999999, 2024-09-20T19:00:52.783 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0044767920000000003, 2024-09-20T19:00:52.783 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:52.783 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:52.783 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "waiting for sub ops", 2024-09-20T19:00:52.783 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:52.783 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:00:52.783 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:00:52.783 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 164195 2024-09-20T19:00:52.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.784 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:52.784 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.784 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:52.784 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.752384+0000", 2024-09-20T19:00:52.784 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:52.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.784 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.784 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:52.784 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.752384+0000", 2024-09-20T19:00:52.784 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:52.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.784 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.784 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:52.784 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.752385+0000", 2024-09-20T19:00:52.785 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3969999999999999e-06 2024-09-20T19:00:52.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.785 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:52.785 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.752390+0000", 2024-09-20T19:00:52.785 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0150000000000003e-06 2024-09-20T19:00:52.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.785 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:52.785 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.752400+0000", 2024-09-20T19:00:52.785 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2140000000000002e-06 2024-09-20T19:00:52.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.785 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:52.786 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.752406+0000", 2024-09-20T19:00:52.786 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0970000000000002e-06 2024-09-20T19:00:52.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.786 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.786 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:52.786 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.754037+0000", 2024-09-20T19:00:52.786 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001631616 2024-09-20T19:00:52.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.786 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.786 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:52.786 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.754139+0000", 2024-09-20T19:00:52.786 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010177499999999999 2024-09-20T19:00:52.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.787 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.787 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T19:00:52.787 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.754238+0000", 2024-09-20T19:00:52.787 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8906999999999993e-05 2024-09-20T19:00:52.787 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:52.787 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:52.787 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:52.787 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.787 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.787 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.25484.0:164197 9.7 e296/246 9:e0da3a39:::benchmark_data_smithi083_124552_object50050:head v 296'10497, mlcod=296'10494)", 2024-09-20T19:00:52.787 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:52.752438+0000", 2024-09-20T19:00:52.787 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0042097849999999997, 2024-09-20T19:00:52.787 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0044797650000000001, 2024-09-20T19:00:52.787 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:52.787 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:52.788 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T19:00:52.788 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:52.788 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.788 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:52.788 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.752438+0000", 2024-09-20T19:00:52.788 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:52.788 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.788 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.788 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:52.788 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.752438+0000", 2024-09-20T19:00:52.788 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:52.788 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.788 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.788 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:52.789 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.752438+0000", 2024-09-20T19:00:52.789 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0100000000000005e-07 2024-09-20T19:00:52.789 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.789 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.789 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:52.789 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.752442+0000", 2024-09-20T19:00:52.789 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9550000000000002e-06 2024-09-20T19:00:52.789 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.789 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.789 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:52.789 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.752448+0000", 2024-09-20T19:00:52.789 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.677e-06 2024-09-20T19:00:52.789 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.789 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.790 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:52.790 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.752452+0000", 2024-09-20T19:00:52.790 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4739999999999999e-06 2024-09-20T19:00:52.790 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.790 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.790 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:52.790 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.754572+0000", 2024-09-20T19:00:52.790 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0021191809999999999 2024-09-20T19:00:52.790 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.790 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.790 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:52.790 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.754588+0000", 2024-09-20T19:00:52.790 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6897000000000001e-05 2024-09-20T19:00:52.790 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:52.791 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:52.791 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:52.791 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.791 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.791 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:164200 9.2 9:46c40f3d:::benchmark_data_smithi083_124552_object50053:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e296)", 2024-09-20T19:00:52.791 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:52.756205+0000", 2024-09-20T19:00:52.791 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.00044204900000000002, 2024-09-20T19:00:52.791 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00075717899999999997, 2024-09-20T19:00:52.791 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:52.791 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:52.791 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "waiting for sub ops", 2024-09-20T19:00:52.791 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:52.791 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:00:52.792 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:00:52.792 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 164200 2024-09-20T19:00:52.792 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.792 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:52.792 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.792 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:52.792 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.756205+0000", 2024-09-20T19:00:52.792 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:52.792 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.792 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.792 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:52.792 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.756205+0000", 2024-09-20T19:00:52.792 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:52.792 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.793 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.793 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:52.793 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.756207+0000", 2024-09-20T19:00:52.793 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3960000000000001e-06 2024-09-20T19:00:52.793 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.793 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.793 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:52.793 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.756212+0000", 2024-09-20T19:00:52.793 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9300000000000002e-06 2024-09-20T19:00:52.793 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.793 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.793 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:52.793 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.756223+0000", 2024-09-20T19:00:52.793 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1338000000000001e-05 2024-09-20T19:00:52.794 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.794 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.794 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:52.794 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.756231+0000", 2024-09-20T19:00:52.794 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4220000000000003e-06 2024-09-20T19:00:52.794 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.794 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.794 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:52.794 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.756291+0000", 2024-09-20T19:00:52.794 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0510000000000002e-05 2024-09-20T19:00:52.794 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.794 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.794 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:52.794 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.756409+0000", 2024-09-20T19:00:52.795 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000118268 2024-09-20T19:00:52.795 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.795 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.795 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:00:52.795 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.756514+0000", 2024-09-20T19:00:52.795 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000104579 2024-09-20T19:00:52.795 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:52.795 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:52.795 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:52.795 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.795 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.795 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.25484.0:164189 9.f e296/246 9:f49d1ca7:::benchmark_data_smithi083_124552_object50042:head v 296'10139, mlcod=296'10137)", 2024-09-20T19:00:52.795 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:52.745228+0000", 2024-09-20T19:00:52.795 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.011419334999999999, 2024-09-20T19:00:52.795 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011809152, 2024-09-20T19:00:52.796 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:52.796 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:52.796 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T19:00:52.796 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:52.796 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.796 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:52.796 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.745228+0000", 2024-09-20T19:00:52.796 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:52.796 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.796 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.796 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:52.796 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.745228+0000", 2024-09-20T19:00:52.796 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:52.796 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.797 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.797 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:52.797 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.745229+0000", 2024-09-20T19:00:52.797 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6700000000000002e-07 2024-09-20T19:00:52.797 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.797 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.797 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:52.797 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.745233+0000", 2024-09-20T19:00:52.797 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2069999999999997e-06 2024-09-20T19:00:52.797 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.797 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.797 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:52.797 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.745249+0000", 2024-09-20T19:00:52.797 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5851000000000001e-05 2024-09-20T19:00:52.798 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.798 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.798 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:52.798 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.745256+0000", 2024-09-20T19:00:52.798 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8700000000000003e-06 2024-09-20T19:00:52.798 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.798 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.798 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:52.798 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.745457+0000", 2024-09-20T19:00:52.798 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000201089 2024-09-20T19:00:52.798 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.798 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.798 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:52.798 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.745477+0000", 2024-09-20T19:00:52.799 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9641999999999999e-05 2024-09-20T19:00:52.799 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:52.799 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:52.799 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:52.799 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.799 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.799 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:164188 9.1 9:8123d6e8:::benchmark_data_smithi083_124552_object50041:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e296)", 2024-09-20T19:00:52.799 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:52.745333+0000", 2024-09-20T19:00:52.799 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.011314879, 2024-09-20T19:00:52.799 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011752659, 2024-09-20T19:00:52.799 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:00:52.799 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:00:52.799 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "waiting for sub ops", 2024-09-20T19:00:52.799 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:00:52.800 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:00:52.800 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:00:52.800 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 164188 2024-09-20T19:00:52.800 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.800 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:00:52.800 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.800 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:00:52.800 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.745333+0000", 2024-09-20T19:00:52.800 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:52.800 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.800 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.800 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:00:52.800 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.745333+0000", 2024-09-20T19:00:52.800 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:00:52.800 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.801 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.801 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:00:52.801 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.745334+0000", 2024-09-20T19:00:52.801 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.339e-06 2024-09-20T19:00:52.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.801 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.801 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:00:52.801 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.745337+0000", 2024-09-20T19:00:52.801 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.529e-06 2024-09-20T19:00:52.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.801 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.801 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:00:52.801 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.745345+0000", 2024-09-20T19:00:52.801 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7929999999999998e-06 2024-09-20T19:00:52.802 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.802 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.802 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:00:52.802 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.745351+0000", 2024-09-20T19:00:52.802 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2989999999999996e-06 2024-09-20T19:00:52.802 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.802 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.802 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:00:52.802 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.746412+0000", 2024-09-20T19:00:52.803 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0010610490000000001 2024-09-20T19:00:52.803 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.803 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.803 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:00:52.803 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.746518+0000", 2024-09-20T19:00:52.803 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106548 2024-09-20T19:00:52.803 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.803 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.803 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 6,7", 2024-09-20T19:00:52.803 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.746620+0000", 2024-09-20T19:00:52.803 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010176800000000001 2024-09-20T19:00:52.803 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:00:52.803 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:00:52.803 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:00:52.804 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:52.754531+0000", 2024-09-20T19:00:52.804 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0079111549999999996 2024-09-20T19:00:52.804 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:52.804 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:00:52.804 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:52.804 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:00:52.804 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T19:00:52.804 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 9 2024-09-20T19:00:52.804 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T19:00:52.853 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:52.879 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:52.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:52.880 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:52.981 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:53.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:53.015 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:53.015 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:53.116 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:53.159 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:53.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:53.160 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:53.188 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-20T19:00:53.260 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:53.292 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:53.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:53.293 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:53.393 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:53.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:53.432 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:53.432 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:53.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:53 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1494740328' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:00:53.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:53 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:00:53.533 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:53.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:53 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1494740328' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:00:53.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:53 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:00:53.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:53 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1494740328' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:00:53.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:53 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:00:53.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:53.558 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:53.558 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:53.659 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:00:53.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:53.694 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:53.694 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:00:53.795 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:53.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:53.826 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:53.826 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:53.927 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:00:53.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:53.949 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:53.949 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:00:54.050 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:54.070 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:54.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:54.071 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:54.173 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:54.194 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:54.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:54.195 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:54.296 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:54.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:54.330 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:54.331 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:54.431 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:54.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:54.477 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:54.477 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:54.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:54 smithi142 ceph-mon[32508]: pgmap v544: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 4.0 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.31k op/s; 12232/196752 objects degraded (6.217%) 2024-09-20T19:00:54.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:54 smithi142 ceph-mon[32508]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:00:54.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:54 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T19:00:54.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:54 smithi142 ceph-mon[32508]: osdmap e297: 8 total, 7 up, 8 in 2024-09-20T19:00:54.531 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:54 smithi083 ceph-mon[29421]: pgmap v544: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 4.0 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.31k op/s; 12232/196752 objects degraded (6.217%) 2024-09-20T19:00:54.531 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:54 smithi083 ceph-mon[29421]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:00:54.531 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:54 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T19:00:54.531 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:54 smithi083 ceph-mon[29421]: osdmap e297: 8 total, 7 up, 8 in 2024-09-20T19:00:54.531 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:54 smithi083 ceph-mon[36260]: pgmap v544: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 4.0 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.31k op/s; 12232/196752 objects degraded (6.217%) 2024-09-20T19:00:54.532 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:54 smithi083 ceph-mon[36260]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:00:54.532 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:54 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T19:00:54.532 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:54 smithi083 ceph-mon[36260]: osdmap e297: 8 total, 7 up, 8 in 2024-09-20T19:00:54.578 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:54.615 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:54.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:54.616 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:54.717 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:00:54.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:54.751 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:54.751 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:00:54.852 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:00:54.884 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:54.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:54.886 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:00:54.986 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:55.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:55.024 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:55.024 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:55.125 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:55.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:55.163 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:55.164 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:55.265 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:55.299 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:55.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:55.300 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:55.401 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:55.431 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:55.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:55.432 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:55.533 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:55.567 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:55.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:55.568 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:55.669 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:00:55.705 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:55.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:55.706 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:00:55.807 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:55.825 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_blocked_ops 2024-09-20T19:00:55.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:55.853 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:55.853 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:55.890 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:00:55.953 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:55.989 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:55.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:55.990 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:56.090 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:56.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:56.123 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:56.123 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:56.143 INFO:teuthology.orchestra.run.smithi142.stdout: marking collection for removal 2024-09-20T19:00:56.143 INFO:teuthology.orchestra.run.smithi142.stdout:setting '_remove' omap key 2024-09-20T19:00:56.144 INFO:teuthology.orchestra.run.smithi142.stdout:finish_remove_pgs 5.1f_head removing 5.1f 2024-09-20T19:00:56.145 INFO:teuthology.orchestra.run.smithi142.stdout:Remove successful 2024-09-20T19:00:56.224 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:56.249 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:56.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:56.250 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:56.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:56 smithi083 ceph-mon[29421]: pgmap v546: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 4.0 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.31k op/s; 12232/196752 objects degraded (6.217%) 2024-09-20T19:00:56.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:56 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 12232/196752 objects degraded (6.217%), 27 pgs degraded (PG_DEGRADED) 2024-09-20T19:00:56.296 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:56 smithi083 ceph-mon[36260]: pgmap v546: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 4.0 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.31k op/s; 12232/196752 objects degraded (6.217%) 2024-09-20T19:00:56.296 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:56 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 12232/196752 objects degraded (6.217%), 27 pgs degraded (PG_DEGRADED) 2024-09-20T19:00:56.350 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:00:56.379 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:56.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:56.379 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:00:56.480 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:56.515 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:56.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:56 smithi142 ceph-mon[32508]: pgmap v546: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 4.0 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.31k op/s; 12232/196752 objects degraded (6.217%) 2024-09-20T19:00:56.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:56 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 12232/196752 objects degraded (6.217%), 27 pgs degraded (PG_DEGRADED) 2024-09-20T19:00:56.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:56.516 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:56.606 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:00:56.606 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:00:56.618 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:56.650 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:00:56.650 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:00:56.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:56.659 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:56.660 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:56.684 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:00:56.684 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:00:56.693 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:00:56.725 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:00:56.725 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:00:56.725 INFO:teuthology.orchestra.run.smithi083.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-20T19:00:56.757 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:00:56.757 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:00:56.761 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:00:56.795 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:00:56.795 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:00:56.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:56.810 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:56.810 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:00:56.836 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:00:56.836 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:00:56.911 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:56.949 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:56.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:56.950 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:57.051 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:57.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:57.085 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:57.086 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:57.186 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:57.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:57.231 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:57.231 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:57.254 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T19:00:57.265 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:00:57 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:00:57.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:57 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2644320317' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:00:57.332 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:57.352 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell -n osd.4 --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-4 --journal-path /var/lib/ceph/osd/ceph-4/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.5.1f.4 2024-09-20T19:00:57.365 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:57.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:57.366 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:57.468 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:00:57.509 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:57.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:57.510 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:00:57.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:57 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2644320317' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:00:57.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:57 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2644320317' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:00:57.611 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:57.651 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:57.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:57.652 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:57.656 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/osd.4/config 2024-09-20T19:00:57.752 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:57.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:57.786 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:57.786 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:57.887 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:57.925 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:57.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:57.926 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:58.027 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:00:58.063 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:58.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:58.064 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:00:58.164 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:58.192 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:58.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:58.193 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:58.294 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:58.339 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:58.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:58.340 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:58.440 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:00:58.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:58.476 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:58.476 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:00:58.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:58 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:00:58.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:58 smithi142 ceph-mon[32508]: pgmap v547: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 3.7 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.28k op/s; 11162/181956 objects degraded (6.134%) 2024-09-20T19:00:58.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:58 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:00:58.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:58 smithi083 ceph-mon[29421]: pgmap v547: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 3.7 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.28k op/s; 11162/181956 objects degraded (6.134%) 2024-09-20T19:00:58.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:58 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:00:58.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:58 smithi083 ceph-mon[36260]: pgmap v547: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 3.7 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.28k op/s; 11162/181956 objects degraded (6.134%) 2024-09-20T19:00:58.578 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:58.612 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:58.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:58.613 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:58.714 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:00:58.742 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:58.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:58.743 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:00:58.844 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:58.865 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:58.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:58.866 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:58.967 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:00:59.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:59.001 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:59.001 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:00:59.016 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-20T19:00:59.017 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-20T19:00:59.102 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:59.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:59.141 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:59.141 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:59.242 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:59.287 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T19:00:59.287 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:00:59.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:59.289 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:59.289 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:59.390 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:00:59.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:59.432 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:59.433 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:00:59.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:59 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:00:59.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:59 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:00:59.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:00:59 smithi142 ceph-mon[32508]: osdmap e298: 8 total, 7 up, 8 in 2024-09-20T19:00:59.534 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:59.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:59 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:00:59.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:59 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:00:59.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:00:59 smithi083 ceph-mon[29421]: osdmap e298: 8 total, 7 up, 8 in 2024-09-20T19:00:59.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:59 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:00:59.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:59 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:00:59.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:00:59 smithi083 ceph-mon[36260]: osdmap e298: 8 total, 7 up, 8 in 2024-09-20T19:00:59.569 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:59.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:59.570 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:59.670 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:00:59.703 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:59.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:59.704 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:00:59.805 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:00:59.835 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:00:59.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:59.836 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:00:59.936 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:00:59.960 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:00:59.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:00:59.983 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:00:59.983 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:00:59.987 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_historic_ops 2024-09-20T19:01:00.084 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:00.111 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:00.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:00.112 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:00.213 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:00.242 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:00.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:00.243 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:00.343 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:00.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:00.384 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:00.385 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:00.485 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:00.512 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:00 smithi083 ceph-mon[29421]: pgmap v549: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 3.6 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.43k op/s; 11162/178458 objects degraded (6.255%) 2024-09-20T19:01:00.513 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:00 smithi083 ceph-mon[36260]: pgmap v549: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 3.6 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.43k op/s; 11162/178458 objects degraded (6.255%) 2024-09-20T19:01:00.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:00 smithi142 ceph-mon[32508]: pgmap v549: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 3.6 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.43k op/s; 11162/178458 objects degraded (6.255%) 2024-09-20T19:01:00.518 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:00.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:00.519 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:00.618 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:01:00.620 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:00.640 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:00.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:00.641 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:00.742 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:00.781 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:00.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:00.782 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:00.795 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:01:00 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:01:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:01:00.883 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:00.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:00.918 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:00.918 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:01.018 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T19:01:01.018 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:01.050 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:01.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:01.051 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:01.152 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:01.190 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:01.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:01.191 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:01.290 INFO:teuthology.orchestra.run.smithi142.stderr:Creating an OSD daemon form without an OSD FSID value 2024-09-20T19:01:01.292 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:01.329 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:01.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:01.330 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:01.430 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:01.460 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:01.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:01.461 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:01.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:01 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 11162/178458 objects degraded (6.255%), 27 pgs degraded (PG_DEGRADED) 2024-09-20T19:01:01.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:01 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3028489128' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:01:01.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:01 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:01:01.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:01 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:01:01.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:01 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 11162/178458 objects degraded (6.255%), 27 pgs degraded (PG_DEGRADED) 2024-09-20T19:01:01.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:01 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3028489128' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:01:01.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:01 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:01:01.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:01 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:01:01.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:01 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 11162/178458 objects degraded (6.255%), 27 pgs degraded (PG_DEGRADED) 2024-09-20T19:01:01.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:01 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3028489128' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:01:01.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:01 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:01:01.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:01 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:01:01.562 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:01.591 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:01.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:01.592 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:01.692 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:01.735 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:01.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:01.736 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:01.816 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-20T19:01:01.836 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:01.873 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:01.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:01.874 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:01.975 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:02.008 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:02.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:02.009 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:02.110 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:02.140 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:02.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:02.142 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:02.186 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T19:01:02.187 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:01:02.242 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:02.278 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:02.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:02.279 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:02.379 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:02.410 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:02.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:02.411 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:02.512 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:02.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:02 smithi142 ceph-mon[32508]: pgmap v550: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 3.5 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 909 op/s; 10768/174948 objects degraded (6.155%) 2024-09-20T19:01:02.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:02 smithi142 ceph-mon[32508]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:01:02.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:02 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T19:01:02.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:02 smithi142 ceph-mon[32508]: osdmap e299: 8 total, 7 up, 8 in 2024-09-20T19:01:02.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:02 smithi083 ceph-mon[29421]: pgmap v550: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 3.5 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 909 op/s; 10768/174948 objects degraded (6.155%) 2024-09-20T19:01:02.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:02 smithi083 ceph-mon[29421]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:01:02.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:02 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T19:01:02.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:02 smithi083 ceph-mon[29421]: osdmap e299: 8 total, 7 up, 8 in 2024-09-20T19:01:02.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:02 smithi083 ceph-mon[36260]: pgmap v550: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 3.5 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 909 op/s; 10768/174948 objects degraded (6.155%) 2024-09-20T19:01:02.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:02 smithi083 ceph-mon[36260]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:01:02.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:02 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T19:01:02.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:02 smithi083 ceph-mon[36260]: osdmap e299: 8 total, 7 up, 8 in 2024-09-20T19:01:02.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:02.550 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:02.550 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:02.651 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:02.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:02.687 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:02.687 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:02.712 INFO:teuthology.orchestra.run.smithi142.stdout:get_pg_num_history pg_num_history pg_num_history(e291 pg_nums {1={19=1},2={57=3},3={58=32},4={60=32},5={62=32},6={64=32},8={160=16},9={246=16}} deleted_pools 241,8) 2024-09-20T19:01:02.713 INFO:teuthology.orchestra.run.smithi142.stdout:Importing pgid 5.1f 2024-09-20T19:01:02.713 INFO:teuthology.orchestra.run.smithi142.stdout:Write #5:f95f44c2:::notify.0:head# 2024-09-20T19:01:02.713 INFO:teuthology.orchestra.run.smithi142.stdout:snapset 0=[]:{} 2024-09-20T19:01:02.713 INFO:teuthology.orchestra.run.smithi142.stdout:write_pg epoch 291 info 5.1f( v 262'10 (0'0,262'10] local-lis/les=260/261 n=1 ec=62/62 lis/c=260/260 les/c/f=261/261/0 sis=260) 2024-09-20T19:01:02.716 INFO:teuthology.orchestra.run.smithi142.stdout:Import successful 2024-09-20T19:01:02.788 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:02.818 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:02.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:02.819 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:02.919 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:02.959 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:02.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:02.960 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:03.009 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-20T19:01:03.060 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:03.097 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:03.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:03.138 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:03.238 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:03.273 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:03.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:03.274 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:03.375 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:03.411 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:03.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:03.412 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:03.513 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:03.553 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:03.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:03.554 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:03.655 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:03.689 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:01:03.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:03.695 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:03.696 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:03.796 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:03.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:03.836 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:03.836 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:03.937 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:03.979 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/exp.5.1f.4 2024-09-20T19:01:03.980 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:03.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:03.981 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:04.019 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:01:04.081 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:04.114 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:04.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:04.115 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:04.216 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:04.258 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:04.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:04.259 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:04.333 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:01:04.333 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:01:04.360 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:01:04.360 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:01:04.360 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:04.391 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:01:04.391 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:01:04.392 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:04.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:04.393 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:04.429 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:04 smithi083 ceph-mon[29421]: 9.9 scrub starts 2024-09-20T19:01:04.429 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:04 smithi083 ceph-mon[29421]: pgmap v552: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 3.2 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.54k op/s; 10014/159813 objects degraded (6.266%) 2024-09-20T19:01:04.429 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:04 smithi083 ceph-mon[29421]: 9.9 scrub ok 2024-09-20T19:01:04.429 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:04 smithi083 ceph-mon[36260]: 9.9 scrub starts 2024-09-20T19:01:04.429 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:04 smithi083 ceph-mon[36260]: pgmap v552: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 3.2 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.54k op/s; 10014/159813 objects degraded (6.266%) 2024-09-20T19:01:04.429 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:04 smithi083 ceph-mon[36260]: 9.9 scrub ok 2024-09-20T19:01:04.430 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:01:04.430 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:01:04.431 INFO:teuthology.orchestra.run.smithi083.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-20T19:01:04.460 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:01:04.460 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:01:04.495 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:04.496 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:01:04.497 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:01:04.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:04 smithi142 ceph-mon[32508]: 9.9 scrub starts 2024-09-20T19:01:04.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:04 smithi142 ceph-mon[32508]: pgmap v552: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 3.2 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.54k op/s; 10014/159813 objects degraded (6.266%) 2024-09-20T19:01:04.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:04 smithi142 ceph-mon[32508]: 9.9 scrub ok 2024-09-20T19:01:04.531 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:01:04.531 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:01:04.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:04.534 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:04.534 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:04.635 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:04.666 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:04.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:04.667 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:04.768 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:04.802 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:04.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:04.803 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:04.845 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T19:01:04.845 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-09-20T19:01:04.846 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.846 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:179019 9.f 9:f477f863:::benchmark_data_smithi083_124552_object64872:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e299)", 2024-09-20T19:01:04.846 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:04.566257+0000", 2024-09-20T19:01:04.846 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.017693591000000002, 2024-09-20T19:01:04.846 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.017705054000000001, 2024-09-20T19:01:04.846 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:04.846 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:04.846 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:04.846 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:01:04.846 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:01:04.846 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:01:04.847 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 179019 2024-09-20T19:01:04.847 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.847 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:04.847 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.847 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:04.847 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.566257+0000", 2024-09-20T19:01:04.847 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:04.847 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.847 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.847 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:04.847 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.566257+0000", 2024-09-20T19:01:04.847 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:04.847 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.848 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.848 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:04.848 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.566257+0000", 2024-09-20T19:01:04.848 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0999999999999999e-07 2024-09-20T19:01:04.848 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.848 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.848 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:04.848 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.566259+0000", 2024-09-20T19:01:04.848 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2589999999999999e-06 2024-09-20T19:01:04.848 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.848 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.848 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:04.848 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.566265+0000", 2024-09-20T19:01:04.848 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8510000000000004e-06 2024-09-20T19:01:04.849 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.849 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.849 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:04.849 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.566270+0000", 2024-09-20T19:01:04.849 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.2299999999999999e-06 2024-09-20T19:01:04.849 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.849 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.849 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:04.849 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.569833+0000", 2024-09-20T19:01:04.849 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.003562071 2024-09-20T19:01:04.849 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.849 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.849 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:04.850 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.569933+0000", 2024-09-20T19:01:04.850 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000100892 2024-09-20T19:01:04.850 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.850 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.850 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,3", 2024-09-20T19:01:04.850 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.570070+0000", 2024-09-20T19:01:04.850 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013654199999999999 2024-09-20T19:01:04.850 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.850 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.850 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:04.850 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.576849+0000", 2024-09-20T19:01:04.850 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0067786799999999996 2024-09-20T19:01:04.851 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.851 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.851 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:01:04.851 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.576914+0000", 2024-09-20T19:01:04.851 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5518000000000005e-05 2024-09-20T19:01:04.851 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.851 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.851 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:04.851 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.583824+0000", 2024-09-20T19:01:04.851 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0069097439999999998 2024-09-20T19:01:04.851 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.851 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.851 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:04.852 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.583865+0000", 2024-09-20T19:01:04.852 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.1037e-05 2024-09-20T19:01:04.852 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.852 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.852 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:04.852 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.583962+0000", 2024-09-20T19:01:04.852 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.6719999999999996e-05 2024-09-20T19:01:04.852 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:04.852 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:04.852 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:04.852 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.852 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.852 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:179023 9.f 9:fbd4c248:::benchmark_data_smithi083_124552_object64876:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e299)", 2024-09-20T19:01:04.853 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:04.566616+0000", 2024-09-20T19:01:04.853 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.017334334999999999, 2024-09-20T19:01:04.853 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.017459776999999999, 2024-09-20T19:01:04.853 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:04.853 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:04.853 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "waiting for sub ops", 2024-09-20T19:01:04.853 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:01:04.853 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:01:04.853 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:01:04.853 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 179023 2024-09-20T19:01:04.853 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.853 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:04.853 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.854 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:04.854 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.566616+0000", 2024-09-20T19:01:04.854 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:04.854 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.854 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.854 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:04.854 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.566616+0000", 2024-09-20T19:01:04.854 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:04.854 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.854 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.854 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:04.854 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.566616+0000", 2024-09-20T19:01:04.854 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.03e-07 2024-09-20T19:01:04.854 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.855 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.855 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:04.855 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.566619+0000", 2024-09-20T19:01:04.855 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2649999999999999e-06 2024-09-20T19:01:04.855 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.855 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.855 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:04.855 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.566625+0000", 2024-09-20T19:01:04.855 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.3709999999999996e-06 2024-09-20T19:01:04.855 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.855 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.855 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:04.855 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.566637+0000", 2024-09-20T19:01:04.855 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.167e-05 2024-09-20T19:01:04.856 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.856 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.856 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:04.856 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.572137+0000", 2024-09-20T19:01:04.856 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0055008920000000003 2024-09-20T19:01:04.856 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.856 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.856 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:04.856 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.572248+0000", 2024-09-20T19:01:04.856 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000110097 2024-09-20T19:01:04.856 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.856 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.856 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,3", 2024-09-20T19:01:04.856 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.572364+0000", 2024-09-20T19:01:04.857 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000116468 2024-09-20T19:01:04.857 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.857 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.857 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:04.857 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.576891+0000", 2024-09-20T19:01:04.857 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0045273070000000004 2024-09-20T19:01:04.857 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.857 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.857 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:01:04.857 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.576919+0000", 2024-09-20T19:01:04.857 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.7511000000000001e-05 2024-09-20T19:01:04.857 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:04.857 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:04.857 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:04.858 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.858 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.858 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:179024 9.f 9:f245013d:::benchmark_data_smithi083_124552_object64877:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e299)", 2024-09-20T19:01:04.858 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:04.566651+0000", 2024-09-20T19:01:04.858 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.017298800999999999, 2024-09-20T19:01:04.858 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.017494328, 2024-09-20T19:01:04.858 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:04.858 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:04.858 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "waiting for sub ops", 2024-09-20T19:01:04.858 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:01:04.858 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:01:04.858 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:01:04.858 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 179024 2024-09-20T19:01:04.859 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.859 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:04.859 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.859 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:04.859 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.566651+0000", 2024-09-20T19:01:04.859 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:04.859 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.859 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.859 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:04.859 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.566651+0000", 2024-09-20T19:01:04.859 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:04.859 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.859 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.860 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:04.860 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.566652+0000", 2024-09-20T19:01:04.860 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8100000000000003e-07 2024-09-20T19:01:04.860 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.860 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.860 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:04.860 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.566654+0000", 2024-09-20T19:01:04.860 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.5620000000000002e-06 2024-09-20T19:01:04.860 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.860 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.860 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:04.860 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.566662+0000", 2024-09-20T19:01:04.860 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.9959999999999993e-06 2024-09-20T19:01:04.860 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.861 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.861 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:04.861 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.566668+0000", 2024-09-20T19:01:04.861 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6749999999999999e-06 2024-09-20T19:01:04.861 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.861 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.861 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:04.861 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.574754+0000", 2024-09-20T19:01:04.861 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0080863640000000004 2024-09-20T19:01:04.861 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.861 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.861 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:04.862 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.574856+0000", 2024-09-20T19:01:04.862 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000101693 2024-09-20T19:01:04.862 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.862 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.862 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,3", 2024-09-20T19:01:04.862 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.574967+0000", 2024-09-20T19:01:04.862 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000110794 2024-09-20T19:01:04.862 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.862 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.862 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:01:04.862 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.576927+0000", 2024-09-20T19:01:04.862 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0019596969999999998 2024-09-20T19:01:04.862 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.863 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.863 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:04.863 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.581146+0000", 2024-09-20T19:01:04.863 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0042197930000000003 2024-09-20T19:01:04.863 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:04.863 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:04.863 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:04.863 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.863 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.863 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:179036 9.6 9:6d09e84c:::benchmark_data_smithi083_124552_object64889:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e299)", 2024-09-20T19:01:04.863 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:04.580107+0000", 2024-09-20T19:01:04.863 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.0038431699999999999, 2024-09-20T19:01:04.864 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0041052240000000002, 2024-09-20T19:01:04.864 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:04.864 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:04.864 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "waiting for sub ops", 2024-09-20T19:01:04.864 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:01:04.864 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:01:04.864 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:01:04.864 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 179036 2024-09-20T19:01:04.864 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.864 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:04.864 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.864 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:04.864 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.580107+0000", 2024-09-20T19:01:04.865 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:04.865 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.865 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.865 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:04.865 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.580107+0000", 2024-09-20T19:01:04.865 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:04.865 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.865 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.865 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:04.865 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.580108+0000", 2024-09-20T19:01:04.865 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.145e-06 2024-09-20T19:01:04.865 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.865 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.865 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:04.866 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.580112+0000", 2024-09-20T19:01:04.866 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.2969999999999997e-06 2024-09-20T19:01:04.866 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.866 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.866 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:04.866 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.580121+0000", 2024-09-20T19:01:04.866 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.5080000000000005e-06 2024-09-20T19:01:04.866 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.866 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.866 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:04.866 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.580127+0000", 2024-09-20T19:01:04.866 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0299999999999999e-06 2024-09-20T19:01:04.866 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.866 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.867 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:04.867 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.580193+0000", 2024-09-20T19:01:04.867 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5727000000000003e-05 2024-09-20T19:01:04.867 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.867 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.867 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:04.867 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.580273+0000", 2024-09-20T19:01:04.867 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.0409999999999998e-05 2024-09-20T19:01:04.867 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.867 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.867 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:01:04.867 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.580379+0000", 2024-09-20T19:01:04.867 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00010582 2024-09-20T19:01:04.867 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:04.868 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:04.868 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:04.868 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.868 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.868 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:179037 9.7 9:edca6a1f:::benchmark_data_smithi083_124552_object64890:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e299)", 2024-09-20T19:01:04.868 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:04.580260+0000", 2024-09-20T19:01:04.868 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.0036902810000000001, 2024-09-20T19:01:04.868 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0040112289999999998, 2024-09-20T19:01:04.868 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:04.868 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:04.868 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "waiting for sub ops", 2024-09-20T19:01:04.868 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:01:04.868 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:01:04.868 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:01:04.868 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 179037 2024-09-20T19:01:04.869 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.869 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:04.869 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.869 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:04.869 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.580260+0000", 2024-09-20T19:01:04.869 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:04.869 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.869 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.869 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:04.869 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.580260+0000", 2024-09-20T19:01:04.869 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:04.869 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.869 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.869 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:04.870 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.580261+0000", 2024-09-20T19:01:04.870 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.018e-06 2024-09-20T19:01:04.870 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.870 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.870 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:04.870 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.580267+0000", 2024-09-20T19:01:04.870 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4699999999999999e-06 2024-09-20T19:01:04.870 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.870 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.870 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:04.870 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.580278+0000", 2024-09-20T19:01:04.870 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0553e-05 2024-09-20T19:01:04.870 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.870 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.871 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:04.871 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.580285+0000", 2024-09-20T19:01:04.871 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.9020000000000002e-06 2024-09-20T19:01:04.871 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.871 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.871 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:04.871 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.580992+0000", 2024-09-20T19:01:04.871 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00070752800000000002 2024-09-20T19:01:04.871 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.871 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.871 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:04.871 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.581100+0000", 2024-09-20T19:01:04.871 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000107799 2024-09-20T19:01:04.871 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.871 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.872 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,7", 2024-09-20T19:01:04.872 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.581203+0000", 2024-09-20T19:01:04.872 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000102332 2024-09-20T19:01:04.872 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:04.872 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:04.872 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:04.872 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.872 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.872 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:179038 9.b e299/279 9:d9092f43:::benchmark_data_smithi083_124552_object64891:head v 299'10910, mlcod=299'10908)", 2024-09-20T19:01:04.872 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:04.582652+0000", 2024-09-20T19:01:04.872 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.001297784, 2024-09-20T19:01:04.872 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001691686, 2024-09-20T19:01:04.872 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:04.872 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:04.872 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-09-20T19:01:04.873 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:04.873 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.873 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:04.873 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.582652+0000", 2024-09-20T19:01:04.873 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:04.873 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.873 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.873 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:04.873 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.582652+0000", 2024-09-20T19:01:04.873 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:04.873 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.873 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.873 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:04.873 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.582654+0000", 2024-09-20T19:01:04.873 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1999999999999999e-06 2024-09-20T19:01:04.873 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.874 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.874 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:04.874 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.582659+0000", 2024-09-20T19:01:04.874 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0839999999999999e-06 2024-09-20T19:01:04.874 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.874 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.874 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:04.874 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.582668+0000", 2024-09-20T19:01:04.874 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.9309999999999997e-06 2024-09-20T19:01:04.874 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.874 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.874 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:04.874 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.582674+0000", 2024-09-20T19:01:04.874 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5479999999999998e-06 2024-09-20T19:01:04.875 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.875 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.875 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:04.875 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.583305+0000", 2024-09-20T19:01:04.875 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00063053600000000001 2024-09-20T19:01:04.875 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.875 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.875 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:04.875 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.583322+0000", 2024-09-20T19:01:04.875 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7339000000000001e-05 2024-09-20T19:01:04.875 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:04.875 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:04.875 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:04.875 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.876 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.876 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:179041 9.f 9.9c5aab8f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e299)", 2024-09-20T19:01:04.876 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:04.583727+0000", 2024-09-20T19:01:04.876 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.00022277399999999999, 2024-09-20T19:01:04.876 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00067191600000000005, 2024-09-20T19:01:04.876 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:04.876 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:04.876 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-09-20T19:01:04.876 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:01:04.876 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:01:04.876 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:01:04.876 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 179041 2024-09-20T19:01:04.876 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.877 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:04.877 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.877 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:04.877 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.583727+0000", 2024-09-20T19:01:04.877 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:04.877 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.877 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.877 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:04.877 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.583727+0000", 2024-09-20T19:01:04.877 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:04.877 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.877 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.877 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:04.877 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.583729+0000", 2024-09-20T19:01:04.877 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.883e-06 2024-09-20T19:01:04.878 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.878 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.878 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:04.878 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.583734+0000", 2024-09-20T19:01:04.878 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.1780000000000002e-06 2024-09-20T19:01:04.878 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.878 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.878 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:04.878 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.583744+0000", 2024-09-20T19:01:04.878 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.0880000000000001e-06 2024-09-20T19:01:04.878 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.878 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.878 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:04.879 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.583751+0000", 2024-09-20T19:01:04.879 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.6310000000000003e-06 2024-09-20T19:01:04.879 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.879 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.879 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:04.879 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.584378+0000", 2024-09-20T19:01:04.879 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00062668800000000003 2024-09-20T19:01:04.879 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:04.879 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:04.879 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:04.879 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.879 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.879 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:179042 9.f 9.e1ea599f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e299)", 2024-09-20T19:01:04.879 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:04.583844+0000", 2024-09-20T19:01:04.879 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.00010623, 2024-09-20T19:01:04.880 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00060762100000000003, 2024-09-20T19:01:04.880 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:04.880 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:04.880 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "queued for pg", 2024-09-20T19:01:04.880 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:01:04.880 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:01:04.880 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:01:04.880 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 179042 2024-09-20T19:01:04.880 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.880 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:04.880 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.880 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:04.880 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.583844+0000", 2024-09-20T19:01:04.880 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:04.880 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.881 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.881 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:04.881 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.583844+0000", 2024-09-20T19:01:04.881 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:04.881 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.881 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.881 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:04.881 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.583845+0000", 2024-09-20T19:01:04.881 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0219999999999999e-06 2024-09-20T19:01:04.881 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.881 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.881 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:04.881 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.583851+0000", 2024-09-20T19:01:04.881 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2790000000000004e-06 2024-09-20T19:01:04.881 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.881 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.882 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:04.882 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.583861+0000", 2024-09-20T19:01:04.882 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.5410000000000006e-06 2024-09-20T19:01:04.882 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.882 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.882 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:04.882 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.583868+0000", 2024-09-20T19:01:04.882 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.934e-06 2024-09-20T19:01:04.882 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:04.882 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:04.882 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:04.882 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.882 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.882 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop_reply(client.25484.0:179036 9.6 e299/246 ondisk, result = 0)", 2024-09-20T19:01:04.883 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:04.583888+0000", 2024-09-20T19:01:04.883 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 6.2440000000000005e-05, 2024-09-20T19:01:04.883 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00060723999999999999, 2024-09-20T19:01:04.883 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:04.883 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:04.883 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-09-20T19:01:04.883 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:04.883 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.883 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:04.883 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.583888+0000", 2024-09-20T19:01:04.883 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:04.883 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.883 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.883 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:04.883 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.583888+0000", 2024-09-20T19:01:04.884 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:04.884 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.884 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.884 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:04.884 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.583888+0000", 2024-09-20T19:01:04.884 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2300000000000001e-07 2024-09-20T19:01:04.884 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.884 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.884 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:04.884 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.583893+0000", 2024-09-20T19:01:04.884 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.172e-06 2024-09-20T19:01:04.884 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.884 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.884 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:04.884 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.583899+0000", 2024-09-20T19:01:04.884 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.8050000000000001e-06 2024-09-20T19:01:04.885 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.885 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.885 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:04.885 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.583905+0000", 2024-09-20T19:01:04.885 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9889999999999997e-06 2024-09-20T19:01:04.885 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.885 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.885 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:04.885 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.584478+0000", 2024-09-20T19:01:04.885 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00057254500000000004 2024-09-20T19:01:04.885 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.885 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.885 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:04.885 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.584485+0000", 2024-09-20T19:01:04.885 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.469e-06 2024-09-20T19:01:04.886 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:04.886 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:04.886 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:04.886 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.886 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.886 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:179043 9.a 9:5ecde9ae:::benchmark_data_smithi083_124552_object64896:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e299)", 2024-09-20T19:01:04.886 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:04.583981+0000", 2024-09-20T19:01:04.886 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 4294967295.999969, 2024-09-20T19:01:04.886 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00056819100000000001, 2024-09-20T19:01:04.886 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:04.886 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:04.886 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "waiting for sub ops", 2024-09-20T19:01:04.886 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:01:04.886 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:01:04.887 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:01:04.887 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 179043 2024-09-20T19:01:04.887 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.887 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:04.887 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.887 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:04.887 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.583981+0000", 2024-09-20T19:01:04.887 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:04.887 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.887 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.887 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:04.887 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.583981+0000", 2024-09-20T19:01:04.887 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:04.887 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.887 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.888 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:04.888 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.583982+0000", 2024-09-20T19:01:04.888 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.7299999999999996e-07 2024-09-20T19:01:04.888 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.888 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.888 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:04.888 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.583985+0000", 2024-09-20T19:01:04.888 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.1310000000000002e-06 2024-09-20T19:01:04.888 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.888 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.888 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:04.888 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.583992+0000", 2024-09-20T19:01:04.888 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.9999999999999999e-06 2024-09-20T19:01:04.889 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.889 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.889 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:04.889 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.583997+0000", 2024-09-20T19:01:04.889 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3650000000000003e-06 2024-09-20T19:01:04.889 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.889 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.889 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:04.889 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.584054+0000", 2024-09-20T19:01:04.889 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.7201999999999999e-05 2024-09-20T19:01:04.889 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.889 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.889 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:04.890 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.584179+0000", 2024-09-20T19:01:04.890 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012508899999999999 2024-09-20T19:01:04.890 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:04.890 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:04.890 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,7", 2024-09-20T19:01:04.890 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:04.584289+0000", 2024-09-20T19:01:04.890 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000109305 2024-09-20T19:01:04.890 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:04.890 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:04.890 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:04.890 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:04.890 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-09-20T19:01:04.890 INFO:teuthology.orchestra.run.smithi142.stdout: "num_ops": 10 2024-09-20T19:01:04.890 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T19:01:04.904 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:04.934 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:04.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:04.935 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:05.036 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:05.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:05.070 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:05.071 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:05.171 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:05.206 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:05.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:05.207 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:05.308 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:05.343 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:05.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:05.344 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:05.393 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-20T19:01:05.393 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-20T19:01:05.445 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:05.481 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:05.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:05.482 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:05.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:05 smithi142 ceph-mon[32508]: 5.13 scrub starts 2024-09-20T19:01:05.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:05 smithi142 ceph-mon[32508]: 5.13 scrub ok 2024-09-20T19:01:05.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:05 smithi142 ceph-mon[32508]: 6.16 scrub starts 2024-09-20T19:01:05.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:05 smithi142 ceph-mon[32508]: 6.16 scrub ok 2024-09-20T19:01:05.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:05 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1953240575' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:01:05.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:05 smithi083 ceph-mon[29421]: 5.13 scrub starts 2024-09-20T19:01:05.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:05 smithi083 ceph-mon[29421]: 5.13 scrub ok 2024-09-20T19:01:05.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:05 smithi083 ceph-mon[29421]: 6.16 scrub starts 2024-09-20T19:01:05.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:05 smithi083 ceph-mon[29421]: 6.16 scrub ok 2024-09-20T19:01:05.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:05 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1953240575' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:01:05.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:05 smithi083 ceph-mon[36260]: 5.13 scrub starts 2024-09-20T19:01:05.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:05 smithi083 ceph-mon[36260]: 5.13 scrub ok 2024-09-20T19:01:05.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:05 smithi083 ceph-mon[36260]: 6.16 scrub starts 2024-09-20T19:01:05.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:05 smithi083 ceph-mon[36260]: 6.16 scrub ok 2024-09-20T19:01:05.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:05 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1953240575' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:01:05.583 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:05.609 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T19:01:05.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:05.616 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:05.616 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:05.659 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_blocked_ops 2024-09-20T19:01:05.717 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:05.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:05.784 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:05.785 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:05.885 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:05.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:05.922 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:05.923 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:06.024 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:06.045 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:01:06.066 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:06.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:06.068 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:06.100 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:01:06.169 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:06.200 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:06.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:06.201 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:06.302 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:06.335 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:06.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:06.336 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:06.436 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:06.468 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:06.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:06.469 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:06.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:06 smithi142 ceph-mon[32508]: pgmap v553: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 3.2 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 922 op/s; 10014/159813 objects degraded (6.266%) 2024-09-20T19:01:06.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:06 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 10014/159813 objects degraded (6.266%), 27 pgs degraded (PG_DEGRADED) 2024-09-20T19:01:06.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:06 smithi083 ceph-mon[29421]: pgmap v553: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 3.2 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 922 op/s; 10014/159813 objects degraded (6.266%) 2024-09-20T19:01:06.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:06 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 10014/159813 objects degraded (6.266%), 27 pgs degraded (PG_DEGRADED) 2024-09-20T19:01:06.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:06 smithi083 ceph-mon[36260]: pgmap v553: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 3.2 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 922 op/s; 10014/159813 objects degraded (6.266%) 2024-09-20T19:01:06.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:06 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 10014/159813 objects degraded (6.266%), 27 pgs degraded (PG_DEGRADED) 2024-09-20T19:01:06.570 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:06.603 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:06.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:06.604 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:06.705 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:06.723 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T19:01:06.723 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-09-20T19:01:06.723 INFO:teuthology.orchestra.run.smithi142.stdout: "complaint_time": 30, 2024-09-20T19:01:06.723 INFO:teuthology.orchestra.run.smithi142.stdout: "num_blocked_ops": 0 2024-09-20T19:01:06.723 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T19:01:06.748 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:06.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:06.749 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:06.850 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:06.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:06.875 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:06.875 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:06.976 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:07.015 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:07.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:07.016 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:07.117 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:07.151 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:07.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:07.152 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:07.253 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:07.293 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:07.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:07.295 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:07.353 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:01:07 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:01:07.354 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:07 smithi142 ceph-mon[32508]: 4.11 scrub starts 2024-09-20T19:01:07.354 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:07 smithi142 ceph-mon[32508]: 4.11 scrub ok 2024-09-20T19:01:07.354 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:07 smithi142 ceph-mon[32508]: 3.5 scrub starts 2024-09-20T19:01:07.354 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:07 smithi142 ceph-mon[32508]: 3.5 scrub ok 2024-09-20T19:01:07.354 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:07 smithi142 ceph-mon[32508]: 4.1e scrub starts 2024-09-20T19:01:07.354 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:07 smithi142 ceph-mon[32508]: 4.1e scrub ok 2024-09-20T19:01:07.395 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T19:01:07.395 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:07.401 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_historic_ops 2024-09-20T19:01:07.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:07.429 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:07.429 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:07.530 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:07.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:07 smithi083 ceph-mon[29421]: 4.11 scrub starts 2024-09-20T19:01:07.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:07 smithi083 ceph-mon[29421]: 4.11 scrub ok 2024-09-20T19:01:07.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:07 smithi083 ceph-mon[29421]: 3.5 scrub starts 2024-09-20T19:01:07.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:07 smithi083 ceph-mon[29421]: 3.5 scrub ok 2024-09-20T19:01:07.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:07 smithi083 ceph-mon[29421]: 4.1e scrub starts 2024-09-20T19:01:07.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:07 smithi083 ceph-mon[29421]: 4.1e scrub ok 2024-09-20T19:01:07.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:07 smithi083 ceph-mon[36260]: 4.11 scrub starts 2024-09-20T19:01:07.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:07 smithi083 ceph-mon[36260]: 4.11 scrub ok 2024-09-20T19:01:07.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:07 smithi083 ceph-mon[36260]: 3.5 scrub starts 2024-09-20T19:01:07.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:07 smithi083 ceph-mon[36260]: 3.5 scrub ok 2024-09-20T19:01:07.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:07 smithi083 ceph-mon[36260]: 4.1e scrub starts 2024-09-20T19:01:07.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:07 smithi083 ceph-mon[36260]: 4.1e scrub ok 2024-09-20T19:01:07.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:07.572 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:07.572 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:07.674 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:07.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:07.721 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:07.721 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:07.766 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:01:07.822 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:07.858 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:07.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:07.860 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:07.961 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:07.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:07.987 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:07.988 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:08.088 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:08.134 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:08.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:08.135 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:08.236 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:08.269 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:08.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:08.270 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:08.319 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T19:01:08.319 INFO:teuthology.orchestra.run.smithi142.stdout: "size": 20, 2024-09-20T19:01:08.319 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 600, 2024-09-20T19:01:08.319 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-09-20T19:01:08.319 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.319 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:101813 9.6 9:6a97dc4d:::benchmark_data_smithi083_124552_object101812:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e275)", 2024-09-20T19:01:08.319 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:57.454719+0000", 2024-09-20T19:01:08.319 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 70.852647641999994, 2024-09-20T19:01:08.319 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.245643423, 2024-09-20T19:01:08.319 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:08.320 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:08.320 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:08.320 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:01:08.320 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:01:08.320 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:01:08.320 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 101813 2024-09-20T19:01:08.320 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.320 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:08.320 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.320 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:08.320 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.454719+0000", 2024-09-20T19:01:08.320 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:08.320 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.320 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.321 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:08.321 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.454719+0000", 2024-09-20T19:01:08.321 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:08.321 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.321 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.321 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:08.321 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.454721+0000", 2024-09-20T19:01:08.321 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.092e-06 2024-09-20T19:01:08.321 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.321 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.321 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:08.321 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.454821+0000", 2024-09-20T19:01:08.321 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.9264000000000004e-05 2024-09-20T19:01:08.321 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.322 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.322 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:08.322 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.454844+0000", 2024-09-20T19:01:08.322 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3275e-05 2024-09-20T19:01:08.322 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.322 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.322 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:08.322 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.454851+0000", 2024-09-20T19:01:08.322 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.2150000000000004e-06 2024-09-20T19:01:08.322 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.322 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.322 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:08.322 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.454928+0000", 2024-09-20T19:01:08.323 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.7286999999999994e-05 2024-09-20T19:01:08.323 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.323 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.323 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:08.323 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.475113+0000", 2024-09-20T19:01:08.323 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.020184166999999999 2024-09-20T19:01:08.323 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.323 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.323 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:01:08.323 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.475254+0000", 2024-09-20T19:01:08.323 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00014129 2024-09-20T19:01:08.323 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.323 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.323 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:08.324 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.700227+0000", 2024-09-20T19:01:08.324 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.22497298499999999 2024-09-20T19:01:08.324 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.324 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.324 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:01:08.324 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.700238+0000", 2024-09-20T19:01:08.324 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1204e-05 2024-09-20T19:01:08.324 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.324 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.324 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:08.324 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.700277+0000", 2024-09-20T19:01:08.324 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.8742999999999998e-05 2024-09-20T19:01:08.324 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.325 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.325 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:08.325 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.700326+0000", 2024-09-20T19:01:08.325 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.9224000000000002e-05 2024-09-20T19:01:08.325 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.325 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.325 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:08.325 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.700363+0000", 2024-09-20T19:01:08.325 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6677000000000001e-05 2024-09-20T19:01:08.325 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:08.325 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:08.325 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:08.325 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.325 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.326 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:101814 9.a 9:5f144506:::benchmark_data_smithi083_124552_object101813:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e275)", 2024-09-20T19:01:08.326 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:57.454874+0000", 2024-09-20T19:01:08.326 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 70.852493034999995, 2024-09-20T19:01:08.326 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.24509635499999999, 2024-09-20T19:01:08.326 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:08.326 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:08.326 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:08.326 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:01:08.326 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:01:08.326 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:01:08.326 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 101814 2024-09-20T19:01:08.326 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.326 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:08.326 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.327 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:08.327 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.454874+0000", 2024-09-20T19:01:08.327 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:08.327 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.327 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.327 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:08.327 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.454874+0000", 2024-09-20T19:01:08.327 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:08.327 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.327 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.327 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:08.327 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.454875+0000", 2024-09-20T19:01:08.327 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.23e-07 2024-09-20T19:01:08.327 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.328 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.328 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:08.328 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.454904+0000", 2024-09-20T19:01:08.328 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9767000000000001e-05 2024-09-20T19:01:08.328 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.328 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.328 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:08.328 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.454911+0000", 2024-09-20T19:01:08.328 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2809999999999997e-06 2024-09-20T19:01:08.328 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.328 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.328 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:08.328 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.454916+0000", 2024-09-20T19:01:08.328 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3009999999999997e-06 2024-09-20T19:01:08.329 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.329 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.329 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:08.329 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.454965+0000", 2024-09-20T19:01:08.329 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8677000000000002e-05 2024-09-20T19:01:08.329 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.329 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.329 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:08.329 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.475123+0000", 2024-09-20T19:01:08.329 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.020158206000000001 2024-09-20T19:01:08.329 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.329 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.329 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,7", 2024-09-20T19:01:08.329 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.475266+0000", 2024-09-20T19:01:08.329 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000142989 2024-09-20T19:01:08.330 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.330 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.330 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:01:08.330 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.479206+0000", 2024-09-20T19:01:08.330 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.003939784 2024-09-20T19:01:08.330 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.330 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.330 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:08.330 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.484059+0000", 2024-09-20T19:01:08.330 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0048534149999999998 2024-09-20T19:01:08.330 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.330 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.330 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:08.331 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.699859+0000", 2024-09-20T19:01:08.331 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.21579965300000001 2024-09-20T19:01:08.331 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.331 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.331 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:08.331 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.699923+0000", 2024-09-20T19:01:08.331 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4431999999999996e-05 2024-09-20T19:01:08.331 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.331 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.331 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:08.331 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.699970+0000", 2024-09-20T19:01:08.331 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7126999999999998e-05 2024-09-20T19:01:08.331 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:08.331 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:08.331 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:08.332 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.332 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.332 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:101818 9.6 9:6be85cb6:::benchmark_data_smithi083_124552_object101817:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e275)", 2024-09-20T19:01:08.332 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:57.455092+0000", 2024-09-20T19:01:08.332 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 70.852275398000003, 2024-09-20T19:01:08.332 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.24541227099999999, 2024-09-20T19:01:08.332 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:08.332 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:08.332 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:08.332 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:01:08.332 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:01:08.332 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:01:08.332 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 101818 2024-09-20T19:01:08.333 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.333 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:08.333 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.333 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:08.333 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.455092+0000", 2024-09-20T19:01:08.333 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:08.333 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.333 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.333 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:08.333 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.455092+0000", 2024-09-20T19:01:08.333 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:08.333 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.333 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.333 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:08.333 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.455099+0000", 2024-09-20T19:01:08.333 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.6890000000000004e-06 2024-09-20T19:01:08.334 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.334 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.334 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:08.334 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.455154+0000", 2024-09-20T19:01:08.334 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4653e-05 2024-09-20T19:01:08.334 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.334 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.334 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:08.334 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.475426+0000", 2024-09-20T19:01:08.334 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.020272367999999999 2024-09-20T19:01:08.334 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.334 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.334 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:08.334 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.475435+0000", 2024-09-20T19:01:08.335 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.3240000000000003e-06 2024-09-20T19:01:08.335 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.335 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.335 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:08.335 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.478230+0000", 2024-09-20T19:01:08.335 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0027948949999999999 2024-09-20T19:01:08.335 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.335 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.335 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:08.335 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.478341+0000", 2024-09-20T19:01:08.335 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011116999999999999 2024-09-20T19:01:08.335 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.335 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.335 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:01:08.336 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.478466+0000", 2024-09-20T19:01:08.336 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000124838 2024-09-20T19:01:08.336 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.336 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.336 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:01:08.336 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.700243+0000", 2024-09-20T19:01:08.336 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.221777312 2024-09-20T19:01:08.336 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.336 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.336 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:08.336 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.700398+0000", 2024-09-20T19:01:08.336 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015468799999999999 2024-09-20T19:01:08.336 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.337 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.337 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:08.337 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.700428+0000", 2024-09-20T19:01:08.337 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.0961999999999999e-05 2024-09-20T19:01:08.337 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.337 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.337 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:08.337 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.700472+0000", 2024-09-20T19:01:08.337 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3846e-05 2024-09-20T19:01:08.337 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.337 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.337 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:08.337 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.700504+0000", 2024-09-20T19:01:08.337 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.1526000000000003e-05 2024-09-20T19:01:08.338 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:08.338 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:08.338 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:08.338 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.338 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.338 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:101817 9.b e275/261 9:de269700:::benchmark_data_smithi083_124552_object101816:head v 275'6183, mlcod=258'2642)", 2024-09-20T19:01:08.338 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:57.456401+0000", 2024-09-20T19:01:08.338 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 70.850966378999999, 2024-09-20T19:01:08.338 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.245359931, 2024-09-20T19:01:08.338 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:08.338 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:08.338 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:08.338 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:08.339 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.339 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:08.339 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.456401+0000", 2024-09-20T19:01:08.339 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:08.339 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.339 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.339 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:08.339 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.456401+0000", 2024-09-20T19:01:08.339 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:08.339 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.339 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.339 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:08.339 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.456402+0000", 2024-09-20T19:01:08.339 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.7000000000000003e-07 2024-09-20T19:01:08.340 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.340 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.340 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:08.340 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.456438+0000", 2024-09-20T19:01:08.340 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6287e-05 2024-09-20T19:01:08.340 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.340 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.340 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:08.340 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.475329+0000", 2024-09-20T19:01:08.340 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.018891086000000001 2024-09-20T19:01:08.340 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.340 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.340 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:08.340 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.475339+0000", 2024-09-20T19:01:08.340 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0219e-05 2024-09-20T19:01:08.341 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.341 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.341 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:08.341 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.475554+0000", 2024-09-20T19:01:08.341 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00021468600000000001 2024-09-20T19:01:08.341 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.341 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.341 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:08.341 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.475579+0000", 2024-09-20T19:01:08.341 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.4984e-05 2024-09-20T19:01:08.341 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.341 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.341 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:08.341 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.487176+0000", 2024-09-20T19:01:08.342 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.011597624000000001 2024-09-20T19:01:08.342 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.342 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.342 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:08.342 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.701761+0000", 2024-09-20T19:01:08.342 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.21458407500000001 2024-09-20T19:01:08.342 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:08.342 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:08.342 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:08.342 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.342 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.342 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:101830 9.a 9:5a1a6026:::benchmark_data_smithi083_124552_object101829:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e275)", 2024-09-20T19:01:08.342 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:57.476577+0000", 2024-09-20T19:01:08.342 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 70.830789641999999, 2024-09-20T19:01:08.343 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.22357284799999999, 2024-09-20T19:01:08.343 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:08.343 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:08.343 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:08.343 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:01:08.343 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:01:08.343 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:01:08.343 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 101830 2024-09-20T19:01:08.343 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.343 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:08.343 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.343 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:08.343 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.476577+0000", 2024-09-20T19:01:08.343 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:08.343 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.344 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.344 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:08.344 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.476577+0000", 2024-09-20T19:01:08.344 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:08.344 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.344 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.344 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:08.344 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.476578+0000", 2024-09-20T19:01:08.344 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.1500000000000003e-07 2024-09-20T19:01:08.344 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.344 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.344 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:08.344 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.476632+0000", 2024-09-20T19:01:08.344 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4184999999999997e-05 2024-09-20T19:01:08.344 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.345 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.345 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:08.345 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.476642+0000", 2024-09-20T19:01:08.345 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.2799999999999992e-06 2024-09-20T19:01:08.345 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.345 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.345 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:08.345 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.476648+0000", 2024-09-20T19:01:08.345 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.1550000000000004e-06 2024-09-20T19:01:08.345 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.345 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.345 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:08.345 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.480583+0000", 2024-09-20T19:01:08.345 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0039356019999999999 2024-09-20T19:01:08.346 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.346 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.346 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:08.346 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.480699+0000", 2024-09-20T19:01:08.346 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000115049 2024-09-20T19:01:08.347 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.347 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.347 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,7", 2024-09-20T19:01:08.347 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.480831+0000", 2024-09-20T19:01:08.347 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013262500000000001 2024-09-20T19:01:08.347 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.347 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.347 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:01:08.347 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.484073+0000", 2024-09-20T19:01:08.347 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0032413989999999998 2024-09-20T19:01:08.347 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.347 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.347 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:08.347 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.700013+0000", 2024-09-20T19:01:08.348 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.21594089299999999 2024-09-20T19:01:08.348 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.348 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.348 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:08.348 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.700057+0000", 2024-09-20T19:01:08.348 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3720000000000002e-05 2024-09-20T19:01:08.348 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.348 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.348 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:08.348 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.700106+0000", 2024-09-20T19:01:08.348 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.9323999999999997e-05 2024-09-20T19:01:08.348 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.348 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.348 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:08.348 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.700150+0000", 2024-09-20T19:01:08.349 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3701000000000001e-05 2024-09-20T19:01:08.349 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:08.349 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:08.349 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:08.349 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.349 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.349 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:101831 9.6 9:6f8af8ea:::benchmark_data_smithi083_124552_object101830:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e275)", 2024-09-20T19:01:08.349 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:57.476708+0000", 2024-09-20T19:01:08.349 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 70.830658584000005, 2024-09-20T19:01:08.349 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.22393711499999999, 2024-09-20T19:01:08.349 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:08.349 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:08.349 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:08.349 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:01:08.349 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:01:08.350 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:01:08.350 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 101831 2024-09-20T19:01:08.350 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.350 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:08.350 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.350 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:08.350 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.476708+0000", 2024-09-20T19:01:08.350 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:08.350 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.350 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.350 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:08.350 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.476708+0000", 2024-09-20T19:01:08.350 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:08.350 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.351 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.351 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:08.351 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.476709+0000", 2024-09-20T19:01:08.351 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9500000000000002e-07 2024-09-20T19:01:08.351 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.351 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.351 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:08.351 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.476770+0000", 2024-09-20T19:01:08.351 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.1088999999999997e-05 2024-09-20T19:01:08.351 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.351 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.351 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:08.351 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.476777+0000", 2024-09-20T19:01:08.351 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.2649999999999999e-06 2024-09-20T19:01:08.352 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.352 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.352 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:08.352 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.476783+0000", 2024-09-20T19:01:08.352 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.468e-06 2024-09-20T19:01:08.352 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.352 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.352 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:08.352 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.484647+0000", 2024-09-20T19:01:08.352 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0078640659999999994 2024-09-20T19:01:08.352 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.352 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.352 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:08.352 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.484776+0000", 2024-09-20T19:01:08.353 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012880300000000001 2024-09-20T19:01:08.353 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.353 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.353 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:01:08.353 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.484916+0000", 2024-09-20T19:01:08.353 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000140686 2024-09-20T19:01:08.353 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.353 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.353 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:01:08.353 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.700247+0000", 2024-09-20T19:01:08.353 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.21533068699999999 2024-09-20T19:01:08.353 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.353 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.354 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:08.354 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.700538+0000", 2024-09-20T19:01:08.354 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00029095500000000002 2024-09-20T19:01:08.354 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.354 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.354 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:08.354 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.700574+0000", 2024-09-20T19:01:08.354 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6436000000000001e-05 2024-09-20T19:01:08.354 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.354 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.354 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:08.354 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.700617+0000", 2024-09-20T19:01:08.354 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.2565999999999999e-05 2024-09-20T19:01:08.355 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.355 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.355 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:08.355 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.700646+0000", 2024-09-20T19:01:08.355 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.8498999999999999e-05 2024-09-20T19:01:08.355 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:08.355 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:08.355 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:08.355 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.355 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.355 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:101828 9.5 e275/246 9:a60cd413:::benchmark_data_smithi083_124552_object101827:head v 275'6402, mlcod=275'6400)", 2024-09-20T19:01:08.355 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:57.479501+0000", 2024-09-20T19:01:08.355 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 70.827865852000002, 2024-09-20T19:01:08.355 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.22378414999999999, 2024-09-20T19:01:08.355 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:08.356 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:08.356 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:08.356 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:08.356 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.356 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:08.356 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.479501+0000", 2024-09-20T19:01:08.356 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:08.356 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.356 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.356 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:08.356 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.479501+0000", 2024-09-20T19:01:08.356 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:08.356 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.356 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.356 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:08.357 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.479502+0000", 2024-09-20T19:01:08.357 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.4399999999999999e-07 2024-09-20T19:01:08.357 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.357 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.357 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:08.357 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.479540+0000", 2024-09-20T19:01:08.357 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.8306999999999998e-05 2024-09-20T19:01:08.357 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.357 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.357 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:08.357 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.479549+0000", 2024-09-20T19:01:08.357 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.3769999999999996e-06 2024-09-20T19:01:08.357 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.357 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.357 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:08.358 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.479555+0000", 2024-09-20T19:01:08.358 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.127e-06 2024-09-20T19:01:08.358 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.358 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.358 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:08.358 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.480389+0000", 2024-09-20T19:01:08.358 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00083417500000000004 2024-09-20T19:01:08.358 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.358 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.358 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:08.358 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.480407+0000", 2024-09-20T19:01:08.358 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7719000000000001e-05 2024-09-20T19:01:08.359 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.359 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.359 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:08.359 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.703240+0000", 2024-09-20T19:01:08.359 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.22283377800000001 2024-09-20T19:01:08.359 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.359 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.359 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:08.359 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.703285+0000", 2024-09-20T19:01:08.359 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.4823000000000001e-05 2024-09-20T19:01:08.359 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:08.359 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:08.359 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:08.359 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.360 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.360 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop_reply(client.25484.0:101813 9.6 e275/246 ondisk, result = 0)", 2024-09-20T19:01:08.360 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:57.480492+0000", 2024-09-20T19:01:08.360 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 70.826874603999997, 2024-09-20T19:01:08.360 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.21975540099999999, 2024-09-20T19:01:08.360 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:08.360 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:08.360 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-09-20T19:01:08.360 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:08.360 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.360 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:08.360 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.480492+0000", 2024-09-20T19:01:08.360 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:08.360 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.360 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.361 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:08.361 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.480492+0000", 2024-09-20T19:01:08.361 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:08.361 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.361 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.361 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:08.361 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.480493+0000", 2024-09-20T19:01:08.361 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.9599999999999998e-07 2024-09-20T19:01:08.361 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.361 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.361 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:08.361 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.480498+0000", 2024-09-20T19:01:08.361 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3029999999999999e-06 2024-09-20T19:01:08.361 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.361 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.362 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:08.362 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.480509+0000", 2024-09-20T19:01:08.362 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0217999999999999e-05 2024-09-20T19:01:08.362 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.362 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.362 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:08.362 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.480515+0000", 2024-09-20T19:01:08.362 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.6409999999999996e-06 2024-09-20T19:01:08.362 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.362 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.362 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:08.362 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.700214+0000", 2024-09-20T19:01:08.362 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.21969873200000001 2024-09-20T19:01:08.362 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.363 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.363 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:08.363 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.700221+0000", 2024-09-20T19:01:08.363 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.9580000000000001e-06 2024-09-20T19:01:08.363 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.363 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.363 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:08.363 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.700248+0000", 2024-09-20T19:01:08.363 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.6752999999999999e-05 2024-09-20T19:01:08.363 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:08.363 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:08.363 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:08.363 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.363 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.363 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:101835 9.b e275/261 9:d30a326b:::benchmark_data_smithi083_124552_object101834:head v 275'6184, mlcod=258'2642)", 2024-09-20T19:01:08.364 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:57.481572+0000", 2024-09-20T19:01:08.364 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 70.825794583999993, 2024-09-20T19:01:08.364 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.23071034300000001, 2024-09-20T19:01:08.364 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:08.364 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:08.364 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:08.364 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:08.364 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.364 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:08.364 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.481572+0000", 2024-09-20T19:01:08.364 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:08.364 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.364 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.364 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:08.364 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.481572+0000", 2024-09-20T19:01:08.365 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:08.365 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.365 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.365 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:08.365 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.481573+0000", 2024-09-20T19:01:08.365 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.2200000000000003e-07 2024-09-20T19:01:08.365 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.365 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.365 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:08.365 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.481613+0000", 2024-09-20T19:01:08.365 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9548999999999997e-05 2024-09-20T19:01:08.365 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.365 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.365 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:08.366 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.481887+0000", 2024-09-20T19:01:08.366 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00027467600000000001 2024-09-20T19:01:08.366 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.366 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.366 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:08.366 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.481894+0000", 2024-09-20T19:01:08.366 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.6470000000000001e-06 2024-09-20T19:01:08.366 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.366 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.366 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:08.366 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.701801+0000", 2024-09-20T19:01:08.366 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.219906938 2024-09-20T19:01:08.366 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.366 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.367 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:08.367 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.701824+0000", 2024-09-20T19:01:08.367 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3054e-05 2024-09-20T19:01:08.367 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.367 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.367 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:08.367 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.712234+0000", 2024-09-20T19:01:08.367 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.010409865000000001 2024-09-20T19:01:08.367 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.367 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.367 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:08.367 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.712283+0000", 2024-09-20T19:01:08.367 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8792000000000003e-05 2024-09-20T19:01:08.368 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:08.368 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:08.368 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:08.368 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.368 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.368 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:101829 9.5 e275/246 9:a228037d:::benchmark_data_smithi083_124552_object101828:head v 275'6403, mlcod=275'6400)", 2024-09-20T19:01:08.368 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:57.481919+0000", 2024-09-20T19:01:08.368 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 70.825448010000002, 2024-09-20T19:01:08.368 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.22140067799999999, 2024-09-20T19:01:08.368 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:08.368 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:08.368 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:08.368 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:08.368 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.368 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:08.369 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.481919+0000", 2024-09-20T19:01:08.369 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:08.369 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.369 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.369 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:08.369 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.481919+0000", 2024-09-20T19:01:08.369 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:08.369 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.369 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.369 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:08.369 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.481919+0000", 2024-09-20T19:01:08.369 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.5299999999999999e-07 2024-09-20T19:01:08.369 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.369 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.370 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:08.370 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.481946+0000", 2024-09-20T19:01:08.370 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.6596000000000001e-05 2024-09-20T19:01:08.370 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.370 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.370 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:08.370 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.481952+0000", 2024-09-20T19:01:08.370 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9850000000000003e-06 2024-09-20T19:01:08.370 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.370 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.370 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:08.370 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.481956+0000", 2024-09-20T19:01:08.370 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3989999999999997e-06 2024-09-20T19:01:08.370 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.371 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.371 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:08.371 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.483448+0000", 2024-09-20T19:01:08.371 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001491764 2024-09-20T19:01:08.371 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.371 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.371 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:08.371 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.483465+0000", 2024-09-20T19:01:08.371 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6520000000000001e-05 2024-09-20T19:01:08.371 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.371 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.371 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:08.371 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.703291+0000", 2024-09-20T19:01:08.371 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.21982632099999999 2024-09-20T19:01:08.371 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.372 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.372 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:08.372 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.703320+0000", 2024-09-20T19:01:08.372 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.864e-05 2024-09-20T19:01:08.372 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:08.372 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:08.372 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:08.372 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.372 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.372 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:101836 9.6 9:6bfb3d0b:::benchmark_data_smithi083_124552_object101835:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e275)", 2024-09-20T19:01:08.372 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:57.484341+0000", 2024-09-20T19:01:08.372 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 70.823026212000002, 2024-09-20T19:01:08.372 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.22717348100000001, 2024-09-20T19:01:08.373 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:08.373 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:08.373 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:08.373 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:01:08.373 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:01:08.373 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:01:08.373 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 101836 2024-09-20T19:01:08.373 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.373 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:08.373 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.373 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:08.373 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.484341+0000", 2024-09-20T19:01:08.373 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:08.373 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.374 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.374 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:08.374 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.484341+0000", 2024-09-20T19:01:08.374 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:08.374 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.374 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.374 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:08.374 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.484343+0000", 2024-09-20T19:01:08.374 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.728e-06 2024-09-20T19:01:08.374 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.374 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.374 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:08.374 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.484401+0000", 2024-09-20T19:01:08.374 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8594999999999997e-05 2024-09-20T19:01:08.375 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.375 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.375 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:08.375 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.484412+0000", 2024-09-20T19:01:08.375 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1192000000000001e-05 2024-09-20T19:01:08.375 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.375 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.375 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:08.375 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.484419+0000", 2024-09-20T19:01:08.375 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.1029999999999996e-06 2024-09-20T19:01:08.375 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.375 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.375 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:08.375 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.700675+0000", 2024-09-20T19:01:08.376 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.216255221 2024-09-20T19:01:08.376 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.376 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.376 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:08.376 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.700800+0000", 2024-09-20T19:01:08.376 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012584599999999999 2024-09-20T19:01:08.376 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.376 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.376 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:01:08.376 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.700921+0000", 2024-09-20T19:01:08.376 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000120919 2024-09-20T19:01:08.376 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.376 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.376 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:08.377 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.706663+0000", 2024-09-20T19:01:08.377 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0057415230000000001 2024-09-20T19:01:08.377 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.377 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.377 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:01:08.377 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.708002+0000", 2024-09-20T19:01:08.377 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0013395659999999999 2024-09-20T19:01:08.377 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.377 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.377 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:08.377 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.711427+0000", 2024-09-20T19:01:08.377 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0034246900000000002 2024-09-20T19:01:08.377 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.377 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.378 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:08.378 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.711478+0000", 2024-09-20T19:01:08.378 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0569000000000003e-05 2024-09-20T19:01:08.378 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.378 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.378 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:08.378 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.711514+0000", 2024-09-20T19:01:08.378 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6529000000000002e-05 2024-09-20T19:01:08.378 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:08.378 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:08.378 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:08.378 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.378 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.379 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:101837 9.6 9:60550345:::benchmark_data_smithi083_124552_object101836:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e275)", 2024-09-20T19:01:08.379 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:57.484434+0000", 2024-09-20T19:01:08.379 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 70.822933473999996, 2024-09-20T19:01:08.379 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.233120257, 2024-09-20T19:01:08.379 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:08.379 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:08.379 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:08.379 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:01:08.379 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:01:08.379 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:01:08.379 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 101837 2024-09-20T19:01:08.379 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.379 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:08.379 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.380 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:08.380 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.484434+0000", 2024-09-20T19:01:08.380 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:08.380 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.380 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.380 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:08.380 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.484434+0000", 2024-09-20T19:01:08.380 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:08.380 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.380 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.380 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:08.380 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.484434+0000", 2024-09-20T19:01:08.380 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7700000000000005e-07 2024-09-20T19:01:08.380 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.380 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.380 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:08.381 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.484482+0000", 2024-09-20T19:01:08.381 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8192999999999998e-05 2024-09-20T19:01:08.381 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.381 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.381 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:08.381 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.484488+0000", 2024-09-20T19:01:08.381 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.28e-06 2024-09-20T19:01:08.381 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.381 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.381 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:08.381 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.484494+0000", 2024-09-20T19:01:08.381 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.1970000000000004e-06 2024-09-20T19:01:08.381 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.381 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.381 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:08.381 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.703639+0000", 2024-09-20T19:01:08.382 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.219145598 2024-09-20T19:01:08.382 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.382 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.382 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:08.382 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.703764+0000", 2024-09-20T19:01:08.382 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000125019 2024-09-20T19:01:08.382 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.382 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.382 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:01:08.382 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.703910+0000", 2024-09-20T19:01:08.382 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00014603500000000001 2024-09-20T19:01:08.382 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.383 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.383 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:08.383 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.711391+0000", 2024-09-20T19:01:08.383 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0074806459999999996 2024-09-20T19:01:08.383 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.383 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.383 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:08.383 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.711551+0000", 2024-09-20T19:01:08.383 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015964899999999999 2024-09-20T19:01:08.383 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.383 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.383 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:01:08.383 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.717356+0000", 2024-09-20T19:01:08.383 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0058050159999999996 2024-09-20T19:01:08.384 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.384 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.384 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:08.384 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.717514+0000", 2024-09-20T19:01:08.384 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015859000000000001 2024-09-20T19:01:08.384 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.384 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.384 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:08.384 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.717554+0000", 2024-09-20T19:01:08.384 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9557000000000001e-05 2024-09-20T19:01:08.384 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:08.384 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:08.384 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:08.384 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.384 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.384 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:101840 9.7 9:e11054f3:::benchmark_data_smithi083_124552_object101839:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e275)", 2024-09-20T19:01:08.385 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:57.485731+0000", 2024-09-20T19:01:08.385 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 70.821636033000004, 2024-09-20T19:01:08.385 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.22556601300000001, 2024-09-20T19:01:08.385 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:08.385 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:08.385 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:08.385 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:01:08.385 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:01:08.385 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:01:08.385 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 101840 2024-09-20T19:01:08.385 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.385 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:08.385 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.386 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:08.386 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.485731+0000", 2024-09-20T19:01:08.386 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:08.386 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.386 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.386 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:08.386 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.485731+0000", 2024-09-20T19:01:08.386 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:08.386 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.386 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.386 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:08.386 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.485733+0000", 2024-09-20T19:01:08.386 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.75e-06 2024-09-20T19:01:08.386 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.387 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.387 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:08.387 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.485785+0000", 2024-09-20T19:01:08.387 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:08.388 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.1983e-05 2024-09-20T19:01:08.388 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.388 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.388 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:08.389 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.485795+0000", 2024-09-20T19:01:08.389 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.03e-05 2024-09-20T19:01:08.389 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.389 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.389 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:08.389 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.485802+0000", 2024-09-20T19:01:08.389 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.1339999999999998e-06 2024-09-20T19:01:08.389 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.389 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.389 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:08.389 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.486942+0000", 2024-09-20T19:01:08.389 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001139859 2024-09-20T19:01:08.389 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.389 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.390 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:08.390 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.699792+0000", 2024-09-20T19:01:08.390 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.21284998499999999 2024-09-20T19:01:08.390 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.390 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.390 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,7", 2024-09-20T19:01:08.390 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.700390+0000", 2024-09-20T19:01:08.390 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00059842300000000005 2024-09-20T19:01:08.390 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.390 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.390 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:08.390 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.704059+0000", 2024-09-20T19:01:08.390 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0036681230000000001 2024-09-20T19:01:08.390 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.391 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.391 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:01:08.391 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.704078+0000", 2024-09-20T19:01:08.391 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9623000000000002e-05 2024-09-20T19:01:08.391 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.391 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.391 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:08.391 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.711149+0000", 2024-09-20T19:01:08.391 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0070711860000000001 2024-09-20T19:01:08.391 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.391 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.391 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:08.391 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.711248+0000", 2024-09-20T19:01:08.391 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.8987000000000006e-05 2024-09-20T19:01:08.392 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.392 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.392 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:08.392 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.711297+0000", 2024-09-20T19:01:08.392 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8659999999999998e-05 2024-09-20T19:01:08.392 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:08.392 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:08.392 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:08.392 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.392 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.392 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:124590 9.6 9:6ce021c9:::benchmark_data_smithi083_124552_object10443:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e289)", 2024-09-20T19:01:08.392 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:00:20.955292+0000", 2024-09-20T19:01:08.392 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 47.352074580999997, 2024-09-20T19:01:08.392 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.324912592, 2024-09-20T19:01:08.393 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:08.393 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:08.393 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:08.393 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:01:08.393 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:01:08.393 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:01:08.393 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 124590 2024-09-20T19:01:08.393 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.393 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:08.393 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.393 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:08.393 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.955292+0000", 2024-09-20T19:01:08.393 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:08.393 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.393 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.394 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:08.394 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.955292+0000", 2024-09-20T19:01:08.394 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:08.394 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.394 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.394 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:08.394 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.955293+0000", 2024-09-20T19:01:08.394 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.8899999999999998e-07 2024-09-20T19:01:08.394 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.394 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.394 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:08.394 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.955297+0000", 2024-09-20T19:01:08.394 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6370000000000001e-06 2024-09-20T19:01:08.395 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.395 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.395 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:08.395 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.955309+0000", 2024-09-20T19:01:08.395 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.205e-05 2024-09-20T19:01:08.395 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.395 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.395 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:08.395 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.955316+0000", 2024-09-20T19:01:08.395 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.7789999999999998e-06 2024-09-20T19:01:08.395 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.395 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.395 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:08.396 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.955398+0000", 2024-09-20T19:01:08.396 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.1893999999999999e-05 2024-09-20T19:01:08.396 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.396 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.396 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:08.396 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.955512+0000", 2024-09-20T19:01:08.396 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000114036 2024-09-20T19:01:08.396 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.396 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.396 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:01:08.396 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.955606+0000", 2024-09-20T19:01:08.396 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.4199999999999999e-05 2024-09-20T19:01:08.396 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.396 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.397 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:01:08.397 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.958828+0000", 2024-09-20T19:01:08.397 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.003221912 2024-09-20T19:01:08.397 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.397 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.397 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:08.397 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.963128+0000", 2024-09-20T19:01:08.397 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.004300379 2024-09-20T19:01:08.397 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.397 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.397 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:08.397 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:21.280133+0000", 2024-09-20T19:01:08.397 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.317005388 2024-09-20T19:01:08.397 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.398 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.398 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:08.398 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:21.280172+0000", 2024-09-20T19:01:08.398 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9014000000000003e-05 2024-09-20T19:01:08.398 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.398 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.398 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:08.398 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:21.280205+0000", 2024-09-20T19:01:08.398 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.2514000000000001e-05 2024-09-20T19:01:08.398 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:08.398 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:08.398 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:08.398 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.398 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.398 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:124594 9.6 9:67fbc174:::benchmark_data_smithi083_124552_object10447:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e289)", 2024-09-20T19:01:08.399 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:00:20.960066+0000", 2024-09-20T19:01:08.399 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 47.347300844000003, 2024-09-20T19:01:08.399 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.32985013699999999, 2024-09-20T19:01:08.399 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:08.399 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:08.399 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:08.399 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:01:08.399 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:01:08.399 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:01:08.399 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 124594 2024-09-20T19:01:08.399 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.399 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:08.399 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.399 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:08.400 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.960066+0000", 2024-09-20T19:01:08.400 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:08.400 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.400 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.400 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:08.400 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.960066+0000", 2024-09-20T19:01:08.400 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:08.400 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.400 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.400 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:08.400 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.960068+0000", 2024-09-20T19:01:08.400 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.514e-06 2024-09-20T19:01:08.400 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.400 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.400 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:08.401 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.960073+0000", 2024-09-20T19:01:08.401 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3040000000000004e-06 2024-09-20T19:01:08.401 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.401 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.401 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:08.401 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.960082+0000", 2024-09-20T19:01:08.401 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.4590000000000004e-06 2024-09-20T19:01:08.401 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.401 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.401 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:08.401 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.960089+0000", 2024-09-20T19:01:08.401 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.3419999999999996e-06 2024-09-20T19:01:08.401 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.402 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.402 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:08.402 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.960147+0000", 2024-09-20T19:01:08.402 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8289999999999999e-05 2024-09-20T19:01:08.402 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.402 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.402 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:08.402 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.960255+0000", 2024-09-20T19:01:08.402 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000107666 2024-09-20T19:01:08.402 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.402 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.402 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:01:08.402 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.960363+0000", 2024-09-20T19:01:08.403 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000108186 2024-09-20T19:01:08.403 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.403 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.403 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:01:08.403 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.963253+0000", 2024-09-20T19:01:08.403 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0028899799999999999 2024-09-20T19:01:08.403 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.403 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.403 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:08.403 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.966302+0000", 2024-09-20T19:01:08.403 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.003049169 2024-09-20T19:01:08.403 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.403 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.403 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:08.404 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:21.289803+0000", 2024-09-20T19:01:08.404 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.32350114299999999 2024-09-20T19:01:08.404 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.404 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.404 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:08.404 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:21.289861+0000", 2024-09-20T19:01:08.404 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8034e-05 2024-09-20T19:01:08.404 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.404 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.404 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:08.404 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:21.289916+0000", 2024-09-20T19:01:08.404 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5050000000000003e-05 2024-09-20T19:01:08.404 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:08.405 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:08.405 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:08.405 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.405 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.405 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:124626 9.6 9:6930ec60:::benchmark_data_smithi083_124552_object10479:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e289)", 2024-09-20T19:01:08.405 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:00:20.983128+0000", 2024-09-20T19:01:08.405 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 47.324238903999998, 2024-09-20T19:01:08.405 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30694355299999998, 2024-09-20T19:01:08.405 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:08.405 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:08.405 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:08.405 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:01:08.405 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:01:08.405 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:01:08.405 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 124626 2024-09-20T19:01:08.406 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.406 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:08.406 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.406 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:08.406 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.983128+0000", 2024-09-20T19:01:08.406 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:08.406 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.406 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.406 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:08.406 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.983128+0000", 2024-09-20T19:01:08.406 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:08.406 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.406 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.406 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:08.407 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.983130+0000", 2024-09-20T19:01:08.407 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.426e-06 2024-09-20T19:01:08.407 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.407 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.407 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:08.407 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.983134+0000", 2024-09-20T19:01:08.407 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.4309999999999996e-06 2024-09-20T19:01:08.407 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.407 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.407 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:08.407 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.983144+0000", 2024-09-20T19:01:08.407 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0023e-05 2024-09-20T19:01:08.407 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.408 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.408 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:08.408 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.983149+0000", 2024-09-20T19:01:08.408 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.9949999999999996e-06 2024-09-20T19:01:08.408 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.408 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.408 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:08.408 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.983201+0000", 2024-09-20T19:01:08.408 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.2312999999999999e-05 2024-09-20T19:01:08.408 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.408 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.408 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:08.408 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.983303+0000", 2024-09-20T19:01:08.408 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000101433 2024-09-20T19:01:08.409 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.409 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.409 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:01:08.409 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.983418+0000", 2024-09-20T19:01:08.409 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000114851 2024-09-20T19:01:08.409 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.409 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.409 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:01:08.409 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.987483+0000", 2024-09-20T19:01:08.409 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0040654990000000002 2024-09-20T19:01:08.409 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.409 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.409 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:08.409 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.989475+0000", 2024-09-20T19:01:08.410 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001992204 2024-09-20T19:01:08.410 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.410 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.410 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:08.410 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:21.289975+0000", 2024-09-20T19:01:08.410 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30049981100000001 2024-09-20T19:01:08.410 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.410 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.410 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:08.410 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:21.290025+0000", 2024-09-20T19:01:08.410 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0009000000000002e-05 2024-09-20T19:01:08.410 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.410 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.411 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:08.411 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:21.290072+0000", 2024-09-20T19:01:08.411 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.6557999999999997e-05 2024-09-20T19:01:08.411 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:08.411 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:08.411 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:08.411 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.411 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.411 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:124628 9.6 9:6a933d55:::benchmark_data_smithi083_124552_object10481:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e289)", 2024-09-20T19:01:08.411 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:00:20.983388+0000", 2024-09-20T19:01:08.411 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 47.323979430000001, 2024-09-20T19:01:08.411 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30682658800000001, 2024-09-20T19:01:08.411 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:08.411 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:08.412 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:08.412 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:01:08.412 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:01:08.412 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:01:08.412 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 124628 2024-09-20T19:01:08.412 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.412 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:08.412 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.412 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:08.412 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.983388+0000", 2024-09-20T19:01:08.412 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:08.412 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.412 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.412 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:08.413 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.983388+0000", 2024-09-20T19:01:08.413 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:08.413 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.413 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.413 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:08.413 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.983388+0000", 2024-09-20T19:01:08.413 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.9699999999999995e-07 2024-09-20T19:01:08.413 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.413 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.413 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:08.413 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.983394+0000", 2024-09-20T19:01:08.413 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0229999999999998e-06 2024-09-20T19:01:08.413 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.413 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.413 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:08.414 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.983402+0000", 2024-09-20T19:01:08.414 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.1079999999999993e-06 2024-09-20T19:01:08.414 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.414 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.414 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:08.414 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.983409+0000", 2024-09-20T19:01:08.414 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.6789999999999999e-06 2024-09-20T19:01:08.414 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.414 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.414 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:08.414 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.985457+0000", 2024-09-20T19:01:08.414 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.002048322 2024-09-20T19:01:08.414 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.414 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.415 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:08.415 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.985569+0000", 2024-09-20T19:01:08.415 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011113899999999999 2024-09-20T19:01:08.415 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.415 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.415 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:01:08.415 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.985636+0000", 2024-09-20T19:01:08.415 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.7665999999999999e-05 2024-09-20T19:01:08.415 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.415 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.415 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:01:08.415 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.989267+0000", 2024-09-20T19:01:08.415 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0036308569999999999 2024-09-20T19:01:08.415 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.416 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.416 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:08.416 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.989524+0000", 2024-09-20T19:01:08.416 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00025713199999999998 2024-09-20T19:01:08.416 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.416 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.416 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:08.416 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:21.290125+0000", 2024-09-20T19:01:08.416 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30060116100000001 2024-09-20T19:01:08.416 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.416 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.416 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:08.416 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:21.290176+0000", 2024-09-20T19:01:08.416 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0964000000000002e-05 2024-09-20T19:01:08.417 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.417 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.417 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:08.417 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:21.290214+0000", 2024-09-20T19:01:08.417 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.7740000000000001e-05 2024-09-20T19:01:08.417 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:08.417 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:08.417 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:08.417 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.417 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.417 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:124629 9.6 9:6a9ad403:::benchmark_data_smithi083_124552_object10482:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e289)", 2024-09-20T19:01:08.417 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:00:20.986104+0000", 2024-09-20T19:01:08.417 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 47.321262849999997, 2024-09-20T19:01:08.418 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30423246900000001, 2024-09-20T19:01:08.418 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:08.418 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:08.418 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:08.418 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:01:08.418 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:01:08.418 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:01:08.418 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 124629 2024-09-20T19:01:08.418 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.418 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:08.418 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.418 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:08.418 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.986104+0000", 2024-09-20T19:01:08.418 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:08.419 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.419 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.419 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:08.419 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.986104+0000", 2024-09-20T19:01:08.419 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:08.419 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.419 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.419 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:08.419 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.986105+0000", 2024-09-20T19:01:08.419 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.077e-06 2024-09-20T19:01:08.419 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.419 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.419 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:08.419 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.986108+0000", 2024-09-20T19:01:08.419 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.2100000000000002e-06 2024-09-20T19:01:08.420 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.420 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.420 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:08.420 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.986117+0000", 2024-09-20T19:01:08.420 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.037e-06 2024-09-20T19:01:08.420 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.420 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.420 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:08.420 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.986125+0000", 2024-09-20T19:01:08.420 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.3320000000000003e-06 2024-09-20T19:01:08.420 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.420 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.420 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:08.420 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.987508+0000", 2024-09-20T19:01:08.421 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0013835099999999999 2024-09-20T19:01:08.421 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.421 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.421 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:08.421 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.987586+0000", 2024-09-20T19:01:08.421 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.7923999999999995e-05 2024-09-20T19:01:08.421 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.421 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.421 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:01:08.421 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.987672+0000", 2024-09-20T19:01:08.421 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.6081999999999993e-05 2024-09-20T19:01:08.421 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.421 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.421 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:01:08.422 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.989531+0000", 2024-09-20T19:01:08.422 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0018587300000000001 2024-09-20T19:01:08.422 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.422 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.422 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:08.422 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.989591+0000", 2024-09-20T19:01:08.422 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0047000000000003e-05 2024-09-20T19:01:08.422 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.422 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.422 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:08.422 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:21.290255+0000", 2024-09-20T19:01:08.422 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30066345700000002 2024-09-20T19:01:08.422 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.422 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.423 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:08.423 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:21.290302+0000", 2024-09-20T19:01:08.423 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7559999999999999e-05 2024-09-20T19:01:08.423 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.423 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.423 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:08.423 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:21.290337+0000", 2024-09-20T19:01:08.423 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4502999999999998e-05 2024-09-20T19:01:08.423 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:08.423 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:08.423 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:08.423 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.423 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.423 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:124634 9.6 9:6166b273:::benchmark_data_smithi083_124552_object10487:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e289)", 2024-09-20T19:01:08.424 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:00:20.993979+0000", 2024-09-20T19:01:08.424 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 47.313388357999997, 2024-09-20T19:01:08.424 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.297307777, 2024-09-20T19:01:08.424 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:08.424 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:08.424 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:08.424 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:01:08.424 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:01:08.424 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:01:08.424 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 124634 2024-09-20T19:01:08.424 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.424 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:08.424 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.424 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:08.424 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.993979+0000", 2024-09-20T19:01:08.425 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:08.425 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.425 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.425 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:08.425 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.993979+0000", 2024-09-20T19:01:08.425 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:08.425 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.425 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.425 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:08.425 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.993979+0000", 2024-09-20T19:01:08.425 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5799999999999998e-07 2024-09-20T19:01:08.425 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.425 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.425 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:08.426 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.993983+0000", 2024-09-20T19:01:08.426 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4599999999999999e-06 2024-09-20T19:01:08.426 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.426 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.426 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:08.426 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.993993+0000", 2024-09-20T19:01:08.426 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0081000000000001e-05 2024-09-20T19:01:08.426 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.426 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.426 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:08.426 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.993999+0000", 2024-09-20T19:01:08.426 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.772e-06 2024-09-20T19:01:08.426 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.426 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.427 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:08.427 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.994054+0000", 2024-09-20T19:01:08.427 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5692000000000001e-05 2024-09-20T19:01:08.427 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.427 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.427 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:08.427 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.994124+0000", 2024-09-20T19:01:08.427 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.9999999999999994e-05 2024-09-20T19:01:08.427 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.427 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.427 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:01:08.427 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.994189+0000", 2024-09-20T19:01:08.427 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4448000000000004e-05 2024-09-20T19:01:08.427 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.427 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.428 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:01:08.428 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.996228+0000", 2024-09-20T19:01:08.428 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0020387280000000001 2024-09-20T19:01:08.428 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.428 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.428 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:08.428 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.996462+0000", 2024-09-20T19:01:08.428 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00023460199999999999 2024-09-20T19:01:08.428 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.428 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.428 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:08.428 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:21.291206+0000", 2024-09-20T19:01:08.428 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.29474367899999998 2024-09-20T19:01:08.428 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.429 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.429 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:08.429 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:21.291253+0000", 2024-09-20T19:01:08.429 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.6903000000000001e-05 2024-09-20T19:01:08.429 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.429 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.429 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:08.429 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:21.291286+0000", 2024-09-20T19:01:08.429 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.3654e-05 2024-09-20T19:01:08.429 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:08.429 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:08.429 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:08.429 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.429 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.429 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14460.0:2699 5.e 5:7759931f:::notify.3:head [watch reconnect cookie 94056120170112 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e292)", 2024-09-20T19:01:08.430 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:00:35.180070+0000", 2024-09-20T19:01:08.430 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 33.127297042000002, 2024-09-20T19:01:08.430 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0139281419999999, 2024-09-20T19:01:08.430 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:08.430 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:08.430 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-09-20T19:01:08.430 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:01:08.430 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14460", 2024-09-20T19:01:08.430 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1555339426", 2024-09-20T19:01:08.430 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 2699 2024-09-20T19:01:08.430 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.430 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:08.430 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.430 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:08.431 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.180070+0000", 2024-09-20T19:01:08.431 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:08.431 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.431 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.431 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:08.431 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.180070+0000", 2024-09-20T19:01:08.431 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:08.431 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.431 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.431 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:08.431 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.180071+0000", 2024-09-20T19:01:08.431 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.093e-06 2024-09-20T19:01:08.431 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.431 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.432 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:08.432 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.180077+0000", 2024-09-20T19:01:08.432 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5079999999999996e-06 2024-09-20T19:01:08.432 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.432 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.432 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:08.432 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.180086+0000", 2024-09-20T19:01:08.432 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.9499999999999996e-06 2024-09-20T19:01:08.432 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.432 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.432 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:08.432 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.180094+0000", 2024-09-20T19:01:08.432 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5780000000000001e-06 2024-09-20T19:01:08.432 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.432 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.433 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:08.433 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.180159+0000", 2024-09-20T19:01:08.433 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5139000000000001e-05 2024-09-20T19:01:08.433 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.433 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.433 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op must wait for map", 2024-09-20T19:01:08.433 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.180165+0000", 2024-09-20T19:01:08.433 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.7629999999999997e-06 2024-09-20T19:01:08.433 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.433 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.433 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:08.433 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.182672+0000", 2024-09-20T19:01:08.433 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0025072710000000002 2024-09-20T19:01:08.433 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.434 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.434 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:01:08.434 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.182681+0000", 2024-09-20T19:01:08.434 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.4780000000000008e-06 2024-09-20T19:01:08.434 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.434 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.434 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:08.434 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.191896+0000", 2024-09-20T19:01:08.434 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.009215755 2024-09-20T19:01:08.434 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.434 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.434 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:08.434 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.191989+0000", 2024-09-20T19:01:08.434 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.2868000000000006e-05 2024-09-20T19:01:08.434 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:08.435 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:08.435 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:08.435 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.193998+0000", 2024-09-20T19:01:08.435 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0020087389999999998 2024-09-20T19:01:08.435 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:08.435 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:08.435 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:08.435 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:08.435 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:08.435 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T19:01:08.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:08.437 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:08.437 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:08.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:08 smithi142 ceph-mon[32508]: 5.17 scrub starts 2024-09-20T19:01:08.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:08 smithi142 ceph-mon[32508]: 5.17 scrub ok 2024-09-20T19:01:08.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:08 smithi142 ceph-mon[32508]: 9.7 scrub starts 2024-09-20T19:01:08.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:08 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:01:08.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:08 smithi142 ceph-mon[32508]: pgmap v554: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 3.0 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.12k op/s; 9229/148680 objects degraded (6.207%) 2024-09-20T19:01:08.529 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:08 smithi083 ceph-mon[29421]: 5.17 scrub starts 2024-09-20T19:01:08.529 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:08 smithi083 ceph-mon[29421]: 5.17 scrub ok 2024-09-20T19:01:08.529 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:08 smithi083 ceph-mon[29421]: 9.7 scrub starts 2024-09-20T19:01:08.530 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:08 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:01:08.530 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:08 smithi083 ceph-mon[29421]: pgmap v554: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 3.0 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.12k op/s; 9229/148680 objects degraded (6.207%) 2024-09-20T19:01:08.530 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:08 smithi083 ceph-mon[36260]: 5.17 scrub starts 2024-09-20T19:01:08.530 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:08 smithi083 ceph-mon[36260]: 5.17 scrub ok 2024-09-20T19:01:08.530 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:08 smithi083 ceph-mon[36260]: 9.7 scrub starts 2024-09-20T19:01:08.530 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:08 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:01:08.530 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:08 smithi083 ceph-mon[36260]: pgmap v554: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 3.0 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.12k op/s; 9229/148680 objects degraded (6.207%) 2024-09-20T19:01:08.538 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:08.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:08.594 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:08.594 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:08.694 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:08.715 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:08.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:08.716 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:08.817 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:08.855 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:08.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:08.856 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:08.957 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:08.993 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:08.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:08.994 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:09.025 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 7, 6] out_osds: [] dead_osds: [4] live_osds: [0, 1, 2, 3, 5, 6, 7] 2024-09-20T19:01:09.026 INFO:tasks.thrashosds.thrasher:Reviving osd 4 2024-09-20T19:01:09.026 INFO:tasks.cephadm.osd.4:Restarting osd.4 (starting--it wasn't running)... 2024-09-20T19:01:09.026 DEBUG:teuthology.orchestra.run.smithi142:osd.4> sudo journalctl -f -n 0 -u ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@osd.4.service 2024-09-20T19:01:09.029 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl start ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@osd.4 2024-09-20T19:01:09.095 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:09.157 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:09.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:09.159 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:09.225 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:01:09.260 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:09.265 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:01:09 smithi142 systemd[1]: Starting Ceph osd.4 for 1ed5a50a-7780-11ef-baf5-efdc52797490... 2024-09-20T19:01:09.285 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:09 smithi083 ceph-mon[36260]: 9.7 scrub ok 2024-09-20T19:01:09.292 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:09.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:09.293 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:09.380 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-20T19:01:09.394 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:09.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:09.429 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:09.429 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:09.530 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:09.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:09 smithi083 ceph-mon[29421]: 9.7 scrub ok 2024-09-20T19:01:09.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:09.588 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:09.588 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:09.638 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:09 smithi142 ceph-mon[32508]: 9.7 scrub ok 2024-09-20T19:01:09.689 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:09.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:09.739 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:09.739 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:09.830 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:01:09.831 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:01:09.841 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:09.855 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:01:09.856 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:01:09.865 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:09.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:09.867 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:09.878 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:01:09.878 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:01:09.900 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:01:09.900 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:01:09.901 INFO:teuthology.orchestra.run.smithi083.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-20T19:01:09.928 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:01:09.929 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:01:09.960 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:01:09.960 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:01:09.961 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:01:09 smithi142 podman[111870]: 2024-09-20 19:01:09.535990612 +0000 UTC m=+0.043405926 image pull 441f9a3fe51528ef653b21f42e9d884db6124a0c7d482e865382bdfdb9ec979c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2 2024-09-20T19:01:09.961 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:01:09 smithi142 podman[111870]: 2024-09-20 19:01:09.670245809 +0000 UTC m=+0.177661104 container create 34f99b59c580d78e1fbeb8ba2f9235250615659bac3724274adfab2ab0482e1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4-activate, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-20T19:01:09.968 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:09.988 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:01:09.988 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:01:10.004 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:10.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:10.005 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:10.106 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:10.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:10.146 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:10.147 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:10.176 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:01:10.220 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:01:09 smithi142 podman[111870]: 2024-09-20 19:01:09.95377747 +0000 UTC m=+0.461192776 container init 34f99b59c580d78e1fbeb8ba2f9235250615659bac3724274adfab2ab0482e1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4-activate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-20T19:01:10.220 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:01:09 smithi142 podman[111870]: 2024-09-20 19:01:09.965803365 +0000 UTC m=+0.473218664 container start 34f99b59c580d78e1fbeb8ba2f9235250615659bac3724274adfab2ab0482e1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4-activate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True) 2024-09-20T19:01:10.220 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:01:10 smithi142 podman[111870]: 2024-09-20 19:01:10.078641957 +0000 UTC m=+0.586057251 container attach 34f99b59c580d78e1fbeb8ba2f9235250615659bac3724274adfab2ab0482e1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4-activate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-20-1028-88deadb) 2024-09-20T19:01:10.220 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:01:10 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4-activate[111925]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-20T19:01:10.220 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:01:10 smithi142 bash[111870]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-20T19:01:10.220 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:01:10 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4-activate[111925]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-20T19:01:10.220 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:01:10 smithi142 bash[111870]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-20T19:01:10.247 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:10.291 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:10.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:10.292 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:10.293 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-20T19:01:10.393 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:10.426 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:10.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:10.427 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:10.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:10 smithi142 ceph-mon[32508]: 4.8 scrub starts 2024-09-20T19:01:10.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:10 smithi142 ceph-mon[32508]: 4.8 scrub ok 2024-09-20T19:01:10.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:10 smithi142 ceph-mon[32508]: pgmap v555: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 2.8 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.29k op/s; 8818/139854 objects degraded (6.305%) 2024-09-20T19:01:10.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:10 smithi142 ceph-mon[32508]: 9.3 scrub starts 2024-09-20T19:01:10.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:10 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1291406435' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:01:10.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:10 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:01:10.516 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:10 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/4026610331' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:01:10.528 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:10.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:10 smithi083 ceph-mon[29421]: 4.8 scrub starts 2024-09-20T19:01:10.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:10 smithi083 ceph-mon[29421]: 4.8 scrub ok 2024-09-20T19:01:10.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:10 smithi083 ceph-mon[29421]: pgmap v555: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 2.8 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.29k op/s; 8818/139854 objects degraded (6.305%) 2024-09-20T19:01:10.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:10 smithi083 ceph-mon[29421]: 9.3 scrub starts 2024-09-20T19:01:10.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:10 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1291406435' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:01:10.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:10 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:01:10.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:10 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/4026610331' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:01:10.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:10 smithi083 ceph-mon[36260]: 4.8 scrub starts 2024-09-20T19:01:10.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:10 smithi083 ceph-mon[36260]: 4.8 scrub ok 2024-09-20T19:01:10.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:10 smithi083 ceph-mon[36260]: pgmap v555: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 2.8 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.29k op/s; 8818/139854 objects degraded (6.305%) 2024-09-20T19:01:10.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:10 smithi083 ceph-mon[36260]: 9.3 scrub starts 2024-09-20T19:01:10.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:10 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1291406435' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:01:10.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:10 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:01:10.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:10 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/4026610331' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:01:10.561 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:10.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:10.562 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:10.662 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:10.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:10.687 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:10.687 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:10.778 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-20T19:01:10.779 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-20T19:01:10.788 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:10.814 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:10.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:10.815 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:10.916 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:10.951 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:10.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:10.952 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:11.045 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:01:10 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:01:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:01:11.053 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:11.090 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:11.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:11.091 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:11.191 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:11.220 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:11.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:11.221 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:11.322 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:11.352 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:11.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:11.353 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:11.454 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:11.483 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:11.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:11.484 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:11.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:11 smithi142 ceph-mon[32508]: 9.3 scrub ok 2024-09-20T19:01:11.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:11 smithi142 ceph-mon[32508]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T19:01:11.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:11 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:01:11.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:11 smithi142 ceph-mon[32508]: osdmap e300: 8 total, 7 up, 8 in 2024-09-20T19:01:11.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:11 smithi142 ceph-mon[32508]: 4.0 scrub starts 2024-09-20T19:01:11.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:11 smithi142 ceph-mon[32508]: 4.0 scrub ok 2024-09-20T19:01:11.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:11 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 8818/139854 objects degraded (6.305%), 27 pgs degraded (PG_DEGRADED) 2024-09-20T19:01:11.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:11 smithi083 ceph-mon[29421]: 9.3 scrub ok 2024-09-20T19:01:11.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:11 smithi083 ceph-mon[29421]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T19:01:11.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:11 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:01:11.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:11 smithi083 ceph-mon[29421]: osdmap e300: 8 total, 7 up, 8 in 2024-09-20T19:01:11.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:11 smithi083 ceph-mon[29421]: 4.0 scrub starts 2024-09-20T19:01:11.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:11 smithi083 ceph-mon[29421]: 4.0 scrub ok 2024-09-20T19:01:11.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:11 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 8818/139854 objects degraded (6.305%), 27 pgs degraded (PG_DEGRADED) 2024-09-20T19:01:11.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:11 smithi083 ceph-mon[36260]: 9.3 scrub ok 2024-09-20T19:01:11.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:11 smithi083 ceph-mon[36260]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T19:01:11.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:11 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:01:11.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:11 smithi083 ceph-mon[36260]: osdmap e300: 8 total, 7 up, 8 in 2024-09-20T19:01:11.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:11 smithi083 ceph-mon[36260]: 4.0 scrub starts 2024-09-20T19:01:11.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:11 smithi083 ceph-mon[36260]: 4.0 scrub ok 2024-09-20T19:01:11.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:11 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 8818/139854 objects degraded (6.305%), 27 pgs degraded (PG_DEGRADED) 2024-09-20T19:01:11.584 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:11.625 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:11.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:11.626 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:11.727 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:11.761 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:11.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:11.762 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:11.862 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:11.892 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:11.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:11.893 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:11.994 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:12.027 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:12.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:12.028 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:12.129 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:12.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:12.156 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:12.156 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:12.257 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:12.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:12.291 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:12.291 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:12.392 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:12.408 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:12 smithi142 ceph-mon[32508]: 5.15 scrub starts 2024-09-20T19:01:12.408 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:12 smithi142 ceph-mon[32508]: 5.15 scrub ok 2024-09-20T19:01:12.408 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:12 smithi142 ceph-mon[32508]: pgmap v557: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 2.8 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 553 B/s rd, 0 B/s wr, 1.01k op/s; 8818/139854 objects degraded (6.305%) 2024-09-20T19:01:12.418 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:12.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:12.419 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:12.456 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T19:01:12.457 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:01:12.520 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:12.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:12.554 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:12.554 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:12.655 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:12.689 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:12.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:12.690 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:12.780 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T19:01:12.791 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:12.795 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:12 smithi083 ceph-mon[29421]: 5.15 scrub starts 2024-09-20T19:01:12.795 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:12 smithi083 ceph-mon[29421]: 5.15 scrub ok 2024-09-20T19:01:12.795 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:12 smithi083 ceph-mon[29421]: pgmap v557: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 2.8 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 553 B/s rd, 0 B/s wr, 1.01k op/s; 8818/139854 objects degraded (6.305%) 2024-09-20T19:01:12.795 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:12 smithi083 ceph-mon[36260]: 5.15 scrub starts 2024-09-20T19:01:12.795 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:12 smithi083 ceph-mon[36260]: 5.15 scrub ok 2024-09-20T19:01:12.795 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:12 smithi083 ceph-mon[36260]: pgmap v557: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 2.8 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 553 B/s rd, 0 B/s wr, 1.01k op/s; 8818/139854 objects degraded (6.305%) 2024-09-20T19:01:12.833 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:12.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:12.834 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:12.934 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:12.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:12.979 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:12.980 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:13.080 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:13.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:13.133 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:13.134 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:13.174 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_blocked_ops 2024-09-20T19:01:13.214 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:01:13.235 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:13.266 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:01:12 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4-activate[111925]: --> Failed to activate via raw: did not find any matching OSD to activate 2024-09-20T19:01:13.266 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:01:12 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4-activate[111925]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-20T19:01:13.266 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:01:12 smithi142 bash[111870]: --> Failed to activate via raw: did not find any matching OSD to activate 2024-09-20T19:01:13.266 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:01:12 smithi142 bash[111870]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-20T19:01:13.266 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:01:12 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4-activate[111925]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-20T19:01:13.266 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:01:12 smithi142 bash[111870]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-20T19:01:13.266 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:01:13 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4-activate[111925]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-20T19:01:13.266 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:01:13 smithi142 bash[111870]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-20T19:01:13.266 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:01:13 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4-activate[111925]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_4 --path /var/lib/ceph/osd/ceph-4 --no-mon-config 2024-09-20T19:01:13.266 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:01:13 smithi142 bash[111870]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_4 --path /var/lib/ceph/osd/ceph-4 --no-mon-config 2024-09-20T19:01:13.271 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:13.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:13.272 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:13.372 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:13.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:13.418 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:13.418 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:13.519 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:13.564 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:13.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:13.565 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:13.576 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:13 smithi083 ceph-mon[29421]: 6.10 scrub starts 2024-09-20T19:01:13.576 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:13 smithi083 ceph-mon[29421]: 6.10 scrub ok 2024-09-20T19:01:13.576 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:13 smithi083 ceph-mon[29421]: 4.17 scrub starts 2024-09-20T19:01:13.576 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:13 smithi083 ceph-mon[29421]: 4.17 scrub ok 2024-09-20T19:01:13.576 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:13 smithi083 ceph-mon[36260]: 6.10 scrub starts 2024-09-20T19:01:13.576 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:13 smithi083 ceph-mon[36260]: 6.10 scrub ok 2024-09-20T19:01:13.576 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:13 smithi083 ceph-mon[36260]: 4.17 scrub starts 2024-09-20T19:01:13.577 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:13 smithi083 ceph-mon[36260]: 4.17 scrub ok 2024-09-20T19:01:13.636 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T19:01:13.666 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:13.698 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:13.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:13.699 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:13.754 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:01:13 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4-activate[111925]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-4/block 2024-09-20T19:01:13.754 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:01:13 smithi142 bash[111870]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-4/block 2024-09-20T19:01:13.754 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:01:13 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4-activate[111925]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-4/block 2024-09-20T19:01:13.754 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:01:13 smithi142 bash[111870]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-4/block 2024-09-20T19:01:13.754 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:01:13 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4-activate[111925]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-09-20T19:01:13.754 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:01:13 smithi142 bash[111870]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-09-20T19:01:13.754 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:01:13 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4-activate[111925]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-20T19:01:13.754 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:01:13 smithi142 bash[111870]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-20T19:01:13.755 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:01:13 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4-activate[111925]: --> ceph-volume lvm activate successful for osd ID: 4 2024-09-20T19:01:13.755 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:01:13 smithi142 bash[111870]: --> ceph-volume lvm activate successful for osd ID: 4 2024-09-20T19:01:13.755 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:01:13 smithi142 conmon[111925]: conmon 34f99b59c580d78e1fbe : Failed to open cgroups file: /sys/fs/cgroup/machine.slice/libpod-34f99b59c580d78e1fbeb8ba2f9235250615659bac3724274adfab2ab0482e1f.scope/container/memory.events 2024-09-20T19:01:13.755 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:01:13 smithi142 podman[112377]: 2024-09-20 19:01:13.461220743 +0000 UTC m=+0.032142071 container died 34f99b59c580d78e1fbeb8ba2f9235250615659bac3724274adfab2ab0482e1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4-activate, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-20T19:01:13.755 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:13 smithi142 ceph-mon[32508]: 6.10 scrub starts 2024-09-20T19:01:13.755 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:13 smithi142 ceph-mon[32508]: 6.10 scrub ok 2024-09-20T19:01:13.755 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:13 smithi142 ceph-mon[32508]: 4.17 scrub starts 2024-09-20T19:01:13.755 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:13 smithi142 ceph-mon[32508]: 4.17 scrub ok 2024-09-20T19:01:13.800 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:13.830 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:13.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:13.831 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:13.931 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:13.954 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:13.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:13.955 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:14.056 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:14.078 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:14.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:14.079 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:14.180 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:14.214 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:14.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:14.215 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:14.220 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:01:14.316 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:14.347 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:14.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:14.348 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:14.356 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:01:14 smithi142 podman[112377]: 2024-09-20 19:01:14.060423503 +0000 UTC m=+0.631344861 container remove 34f99b59c580d78e1fbeb8ba2f9235250615659bac3724274adfab2ab0482e1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4-activate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-20T19:01:14.356 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:14 smithi142 ceph-mon[32508]: 5.7 scrub starts 2024-09-20T19:01:14.356 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:14 smithi142 ceph-mon[32508]: 5.7 scrub ok 2024-09-20T19:01:14.356 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:14 smithi142 ceph-mon[32508]: pgmap v558: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 2.5 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.27k op/s; 7681/121809 objects degraded (6.306%) 2024-09-20T19:01:14.449 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:14.485 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:14.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:14.485 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:14.586 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:14.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:14.619 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:14.619 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:14.630 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:01:14 smithi142 podman[112499]: 2024-09-20 19:01:14.256268675 +0000 UTC m=+0.031439786 image pull 441f9a3fe51528ef653b21f42e9d884db6124a0c7d482e865382bdfdb9ec979c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2 2024-09-20T19:01:14.630 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:01:14 smithi142 podman[112499]: 2024-09-20 19:01:14.395223298 +0000 UTC m=+0.170394385 container create 0e1bf4a72b09a3ebdb20ce95b1bf69f073f5863223b2bfded919824418a56a7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-20T19:01:14.721 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:14.755 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:14.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:14.756 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:14.795 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:14 smithi083 ceph-mon[29421]: 5.7 scrub starts 2024-09-20T19:01:14.795 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:14 smithi083 ceph-mon[29421]: 5.7 scrub ok 2024-09-20T19:01:14.795 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:14 smithi083 ceph-mon[29421]: pgmap v558: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 2.5 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.27k op/s; 7681/121809 objects degraded (6.306%) 2024-09-20T19:01:14.795 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:14 smithi083 ceph-mon[36260]: 5.7 scrub starts 2024-09-20T19:01:14.795 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:14 smithi083 ceph-mon[36260]: 5.7 scrub ok 2024-09-20T19:01:14.795 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:14 smithi083 ceph-mon[36260]: pgmap v558: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 2.5 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.27k op/s; 7681/121809 objects degraded (6.306%) 2024-09-20T19:01:14.857 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:14.893 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:14.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:14.894 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:14.994 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:15.033 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:15.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:15.034 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:15.135 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:15.171 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:15.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:15.172 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:15.224 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:01:15 smithi142 podman[112499]: 2024-09-20 19:01:15.092699412 +0000 UTC m=+0.867870510 container init 0e1bf4a72b09a3ebdb20ce95b1bf69f073f5863223b2bfded919824418a56a7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True) 2024-09-20T19:01:15.224 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:01:15 smithi142 podman[112499]: 2024-09-20 19:01:15.098459032 +0000 UTC m=+0.873630120 container start 0e1bf4a72b09a3ebdb20ce95b1bf69f073f5863223b2bfded919824418a56a7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4, ceph=True, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-20T19:01:15.233 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-20T19:01:15.273 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:15.314 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:15.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:15.315 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:15.416 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:15.439 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:15.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:15.440 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:15.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:15 smithi142 ceph-mon[32508]: 5.2 scrub starts 2024-09-20T19:01:15.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:15 smithi142 ceph-mon[32508]: 5.2 scrub ok 2024-09-20T19:01:15.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:15 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2829895599' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:01:15.511 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T19:01:15.511 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-09-20T19:01:15.511 INFO:teuthology.orchestra.run.smithi142.stdout: "complaint_time": 30, 2024-09-20T19:01:15.511 INFO:teuthology.orchestra.run.smithi142.stdout: "num_blocked_ops": 0 2024-09-20T19:01:15.511 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T19:01:15.511 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:01:15 smithi142 bash[112499]: 0e1bf4a72b09a3ebdb20ce95b1bf69f073f5863223b2bfded919824418a56a7f 2024-09-20T19:01:15.512 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:01:15 smithi142 systemd[1]: Started Ceph osd.4 for 1ed5a50a-7780-11ef-baf5-efdc52797490. 2024-09-20T19:01:15.540 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:15.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:15 smithi083 ceph-mon[29421]: 5.2 scrub starts 2024-09-20T19:01:15.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:15 smithi083 ceph-mon[29421]: 5.2 scrub ok 2024-09-20T19:01:15.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:15 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2829895599' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:01:15.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:15 smithi083 ceph-mon[36260]: 5.2 scrub starts 2024-09-20T19:01:15.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:15 smithi083 ceph-mon[36260]: 5.2 scrub ok 2024-09-20T19:01:15.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:15 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2829895599' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:01:15.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:15.593 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:15.594 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:15.602 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:01:15.602 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:01:15.603 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:01:15.603 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:01:15.603 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:01:15.603 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:01:15.603 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:01:15.603 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:01:15.603 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:01:15.603 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:01:15.603 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:01:15.603 INFO:teuthology.orchestra.run.smithi083.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-20T19:01:15.603 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:01:15.603 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:01:15.604 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:01:15.694 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:15.724 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:15.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:15.725 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:15.826 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:15.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:15.871 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:15.871 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:15.972 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:16.007 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:16.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:16.008 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:16.109 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:16.141 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:16.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:16.143 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:16.224 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_historic_ops 2024-09-20T19:01:16.243 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:16.250 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:01:16.274 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:16.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:16.275 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:16.376 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:16.408 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:16.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:16.409 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:16.424 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:01:16 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4[112539]: 2024-09-20T19:01:16.223+0000 7f7b5c931640 -1 Falling back to public interface 2024-09-20T19:01:16.424 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:16 smithi142 ceph-mon[32508]: 5.16 scrub starts 2024-09-20T19:01:16.424 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:16 smithi142 ceph-mon[32508]: 5.16 scrub ok 2024-09-20T19:01:16.424 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:16 smithi142 ceph-mon[32508]: 6.9 scrub starts 2024-09-20T19:01:16.424 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:16 smithi142 ceph-mon[32508]: 6.9 scrub ok 2024-09-20T19:01:16.425 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:16 smithi142 ceph-mon[32508]: pgmap v559: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 2.5 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.27k op/s; 7681/121809 objects degraded (6.306%) 2024-09-20T19:01:16.425 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:16 smithi142 ceph-mon[32508]: 6.5 scrub starts 2024-09-20T19:01:16.425 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:16 smithi142 ceph-mon[32508]: 6.5 scrub ok 2024-09-20T19:01:16.425 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:16 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 7681/121809 objects degraded (6.306%), 27 pgs degraded (PG_DEGRADED) 2024-09-20T19:01:16.425 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:16 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:01:16.425 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:16 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:01:16.484 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-20T19:01:16.484 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-20T19:01:16.488 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:01:16.511 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:16.565 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:16.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:16.566 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:16.621 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:16 smithi083 ceph-mon[36260]: 5.16 scrub starts 2024-09-20T19:01:16.622 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:16 smithi083 ceph-mon[36260]: 5.16 scrub ok 2024-09-20T19:01:16.622 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:16 smithi083 ceph-mon[36260]: 6.9 scrub starts 2024-09-20T19:01:16.622 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:16 smithi083 ceph-mon[36260]: 6.9 scrub ok 2024-09-20T19:01:16.622 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:16 smithi083 ceph-mon[36260]: pgmap v559: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 2.5 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.27k op/s; 7681/121809 objects degraded (6.306%) 2024-09-20T19:01:16.622 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:16 smithi083 ceph-mon[36260]: 6.5 scrub starts 2024-09-20T19:01:16.622 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:16 smithi083 ceph-mon[36260]: 6.5 scrub ok 2024-09-20T19:01:16.622 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:16 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 7681/121809 objects degraded (6.306%), 27 pgs degraded (PG_DEGRADED) 2024-09-20T19:01:16.622 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:16 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:01:16.622 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:16 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:01:16.622 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:16 smithi083 ceph-mon[29421]: 5.16 scrub starts 2024-09-20T19:01:16.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:16 smithi083 ceph-mon[29421]: 5.16 scrub ok 2024-09-20T19:01:16.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:16 smithi083 ceph-mon[29421]: 6.9 scrub starts 2024-09-20T19:01:16.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:16 smithi083 ceph-mon[29421]: 6.9 scrub ok 2024-09-20T19:01:16.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:16 smithi083 ceph-mon[29421]: pgmap v559: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 2.5 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.27k op/s; 7681/121809 objects degraded (6.306%) 2024-09-20T19:01:16.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:16 smithi083 ceph-mon[29421]: 6.5 scrub starts 2024-09-20T19:01:16.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:16 smithi083 ceph-mon[29421]: 6.5 scrub ok 2024-09-20T19:01:16.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:16 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 7681/121809 objects degraded (6.306%), 27 pgs degraded (PG_DEGRADED) 2024-09-20T19:01:16.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:16 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:01:16.623 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:16 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:01:16.667 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:16.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:16.704 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:16.704 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:16.805 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:16.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:16.843 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:16.844 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:16.945 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:16.978 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:16.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:16.979 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:17.080 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:17.124 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:17.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:17.125 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:17.145 INFO:teuthology.orchestra.run.smithi142.stderr:no valid command found; 10 closest matches: 2024-09-20T19:01:17.145 INFO:teuthology.orchestra.run.smithi142.stderr:0 2024-09-20T19:01:17.145 INFO:teuthology.orchestra.run.smithi142.stderr:1 2024-09-20T19:01:17.145 INFO:teuthology.orchestra.run.smithi142.stderr:2 2024-09-20T19:01:17.145 INFO:teuthology.orchestra.run.smithi142.stderr:abort 2024-09-20T19:01:17.145 INFO:teuthology.orchestra.run.smithi142.stderr:assert 2024-09-20T19:01:17.145 INFO:teuthology.orchestra.run.smithi142.stderr:config diff 2024-09-20T19:01:17.145 INFO:teuthology.orchestra.run.smithi142.stderr:config diff get 2024-09-20T19:01:17.145 INFO:teuthology.orchestra.run.smithi142.stderr:config get 2024-09-20T19:01:17.145 INFO:teuthology.orchestra.run.smithi142.stderr:config help [] 2024-09-20T19:01:17.145 INFO:teuthology.orchestra.run.smithi142.stderr:config set ... 2024-09-20T19:01:17.145 INFO:teuthology.orchestra.run.smithi142.stderr:admin_socket: invalid command 2024-09-20T19:01:17.146 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:01:17 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:01:17.226 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:17.254 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:17.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:17.255 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:17.355 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:17.395 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:17.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:17.396 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:17.496 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:17.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:17 smithi142 ceph-mon[32508]: 4.1f scrub starts 2024-09-20T19:01:17.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:17 smithi142 ceph-mon[32508]: 4.1f scrub ok 2024-09-20T19:01:17.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:17 smithi142 ceph-mon[32508]: 4.16 scrub starts 2024-09-20T19:01:17.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:17 smithi142 ceph-mon[32508]: 4.16 scrub ok 2024-09-20T19:01:17.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:17 smithi142 ceph-mon[32508]: 9.d scrub starts 2024-09-20T19:01:17.529 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:17.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:17.530 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:17.631 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:17.663 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:17.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:17.664 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:17.765 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:17.798 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:17 smithi083 ceph-mon[29421]: 4.1f scrub starts 2024-09-20T19:01:17.798 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:17 smithi083 ceph-mon[29421]: 4.1f scrub ok 2024-09-20T19:01:17.798 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:17 smithi083 ceph-mon[29421]: 4.16 scrub starts 2024-09-20T19:01:17.799 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:17 smithi083 ceph-mon[29421]: 4.16 scrub ok 2024-09-20T19:01:17.799 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:17 smithi083 ceph-mon[29421]: 9.d scrub starts 2024-09-20T19:01:17.799 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:17 smithi083 ceph-mon[36260]: 4.1f scrub starts 2024-09-20T19:01:17.799 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:17 smithi083 ceph-mon[36260]: 4.1f scrub ok 2024-09-20T19:01:17.799 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:17 smithi083 ceph-mon[36260]: 4.16 scrub starts 2024-09-20T19:01:17.799 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:17 smithi083 ceph-mon[36260]: 4.16 scrub ok 2024-09-20T19:01:17.799 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:17 smithi083 ceph-mon[36260]: 9.d scrub starts 2024-09-20T19:01:17.800 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:17.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:17.801 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:17.902 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:17.931 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-20T19:01:17.931 INFO:tasks.cephadm.ceph_manager.ceph:waiting on admin_socket for osd-4, ['dump_ops_in_flight'] 2024-09-20T19:01:17.935 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:17.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:17.936 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:17.978 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:01:18.036 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:18.069 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:18.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:18.070 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:18.171 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:18.199 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:18.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:18.200 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:18.301 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:18.335 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:18.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:18.336 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:18.370 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-20T19:01:18.421 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:18 smithi142 ceph-mon[32508]: 3.10 scrub starts 2024-09-20T19:01:18.421 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:18 smithi142 ceph-mon[32508]: 3.10 scrub ok 2024-09-20T19:01:18.422 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:18 smithi142 ceph-mon[32508]: 3.4 scrub starts 2024-09-20T19:01:18.422 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:18 smithi142 ceph-mon[32508]: 3.4 scrub ok 2024-09-20T19:01:18.422 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:18 smithi142 ceph-mon[32508]: 9.d scrub ok 2024-09-20T19:01:18.422 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:18 smithi142 ceph-mon[32508]: 3.17 scrub starts 2024-09-20T19:01:18.422 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:18 smithi142 ceph-mon[32508]: 3.17 scrub ok 2024-09-20T19:01:18.422 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:18 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:01:18.422 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:18 smithi142 ceph-mon[32508]: pgmap v560: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 2.3 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.24k op/s; 6880/111435 objects degraded (6.174%) 2024-09-20T19:01:18.422 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:18 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:01:18.422 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:18 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:01:18.422 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:18 smithi142 ceph-mon[32508]: osdmap e301: 8 total, 7 up, 8 in 2024-09-20T19:01:18.422 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:18 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/387430631' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:01:18.422 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:18 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:01:18.437 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:18.474 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:18.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:18.475 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:18.485 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T19:01:18.576 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:18.582 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T19:01:18.582 INFO:teuthology.orchestra.run.smithi142.stdout: "size": 20, 2024-09-20T19:01:18.582 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 600, 2024-09-20T19:01:18.582 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-09-20T19:01:18.582 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.582 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:120182 7.d 7:b9824c9e:::benchmark_data_smithi083_70643_object120181:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T19:01:18.582 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:24.962568+0000", 2024-09-20T19:01:18.582 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 533.60833339999999, 2024-09-20T19:01:18.582 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.116782992, 2024-09-20T19:01:18.582 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:18.582 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:18.582 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:18.583 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:01:18.583 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T19:01:18.583 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T19:01:18.583 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 120182 2024-09-20T19:01:18.583 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.583 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:18.583 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.583 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:18.583 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962568+0000", 2024-09-20T19:01:18.583 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:18.583 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.583 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.583 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:18.583 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962568+0000", 2024-09-20T19:01:18.583 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:18.584 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.584 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.584 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:18.584 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962569+0000", 2024-09-20T19:01:18.584 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4160000000000001e-06 2024-09-20T19:01:18.584 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.584 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.584 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:18.584 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962623+0000", 2024-09-20T19:01:18.584 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4024000000000003e-05 2024-09-20T19:01:18.584 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.584 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.584 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:18.584 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962636+0000", 2024-09-20T19:01:18.585 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2149e-05 2024-09-20T19:01:18.585 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.585 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.585 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:18.585 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962643+0000", 2024-09-20T19:01:18.585 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.554e-06 2024-09-20T19:01:18.585 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.585 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.585 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:18.585 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962684+0000", 2024-09-20T19:01:18.585 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.1139e-05 2024-09-20T19:01:18.585 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.585 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.585 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:18.585 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962772+0000", 2024-09-20T19:01:18.586 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.7750999999999994e-05 2024-09-20T19:01:18.586 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.586 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.586 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,1", 2024-09-20T19:01:18.586 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962859+0000", 2024-09-20T19:01:18.586 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.6434999999999995e-05 2024-09-20T19:01:18.586 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.586 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.586 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:01:18.586 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.966204+0000", 2024-09-20T19:01:18.586 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0033459029999999999 2024-09-20T19:01:18.586 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.586 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.586 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:18.586 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.970405+0000", 2024-09-20T19:01:18.587 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0042008729999999999 2024-09-20T19:01:18.587 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.587 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.587 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:18.587 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.079209+0000", 2024-09-20T19:01:18.587 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10880400899999999 2024-09-20T19:01:18.587 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.587 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.587 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:18.587 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.079303+0000", 2024-09-20T19:01:18.587 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.4022000000000002e-05 2024-09-20T19:01:18.587 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.587 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.587 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:18.588 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.079351+0000", 2024-09-20T19:01:18.588 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7716999999999997e-05 2024-09-20T19:01:18.588 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:18.588 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:18.588 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:18.588 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.588 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.588 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:120184 7.c 7:3c34b9f0:::benchmark_data_smithi083_70643_object120183:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T19:01:18.588 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:24.962667+0000", 2024-09-20T19:01:18.588 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 533.60823457599997, 2024-09-20T19:01:18.588 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.113312679, 2024-09-20T19:01:18.588 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:18.588 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:18.588 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:18.588 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:01:18.589 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T19:01:18.589 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T19:01:18.589 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 120184 2024-09-20T19:01:18.589 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.589 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:18.589 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.589 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:18.589 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962667+0000", 2024-09-20T19:01:18.589 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:18.589 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.589 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.589 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:18.589 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962667+0000", 2024-09-20T19:01:18.589 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:18.590 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.590 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.590 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:18.590 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962667+0000", 2024-09-20T19:01:18.590 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8400000000000004e-07 2024-09-20T19:01:18.590 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.590 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.590 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:18.590 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962702+0000", 2024-09-20T19:01:18.590 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4734000000000003e-05 2024-09-20T19:01:18.590 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.590 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.590 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:18.590 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962710+0000", 2024-09-20T19:01:18.590 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.0900000000000005e-06 2024-09-20T19:01:18.591 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.591 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.591 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:18.591 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962716+0000", 2024-09-20T19:01:18.591 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.1820000000000003e-06 2024-09-20T19:01:18.591 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.591 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.591 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:18.591 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962771+0000", 2024-09-20T19:01:18.591 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4447000000000002e-05 2024-09-20T19:01:18.591 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.591 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.591 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:18.591 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.962883+0000", 2024-09-20T19:01:18.592 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000111996 2024-09-20T19:01:18.592 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.592 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.592 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,7", 2024-09-20T19:01:18.592 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.963021+0000", 2024-09-20T19:01:18.592 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013835799999999999 2024-09-20T19:01:18.592 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.592 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.592 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:18.592 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.966241+0000", 2024-09-20T19:01:18.592 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0032201299999999999 2024-09-20T19:01:18.592 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.592 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.592 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:01:18.592 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:24.966257+0000", 2024-09-20T19:01:18.592 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5325e-05 2024-09-20T19:01:18.593 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.593 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.593 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:18.593 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.075873+0000", 2024-09-20T19:01:18.593 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.109616093 2024-09-20T19:01:18.593 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.593 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.593 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:18.593 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.075932+0000", 2024-09-20T19:01:18.593 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9487999999999997e-05 2024-09-20T19:01:18.593 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.593 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.593 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:18.593 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:25.075980+0000", 2024-09-20T19:01:18.593 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7252000000000001e-05 2024-09-20T19:01:18.594 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:18.594 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:18.594 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:18.594 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.594 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.594 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:139218 7.d 7:bef6f051:::benchmark_data_smithi083_70643_object16416:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-20T19:01:18.594 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:42.802191+0000", 2024-09-20T19:01:18.594 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 515.76871059300004, 2024-09-20T19:01:18.594 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.12718874399999999, 2024-09-20T19:01:18.594 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:18.594 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:18.594 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:18.594 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:01:18.595 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T19:01:18.595 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T19:01:18.595 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 139218 2024-09-20T19:01:18.595 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.595 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:18.596 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.596 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:18.596 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.802191+0000", 2024-09-20T19:01:18.596 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:18.596 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.596 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.596 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:18.596 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.802191+0000", 2024-09-20T19:01:18.596 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:18.596 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.596 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.596 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:18.596 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.802192+0000", 2024-09-20T19:01:18.596 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0759999999999999e-06 2024-09-20T19:01:18.597 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.597 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.597 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:18.597 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.802196+0000", 2024-09-20T19:01:18.597 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.7309999999999999e-06 2024-09-20T19:01:18.597 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.597 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.597 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:18.597 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.802203+0000", 2024-09-20T19:01:18.597 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.4599999999999997e-06 2024-09-20T19:01:18.597 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.597 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.597 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:18.597 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.802209+0000", 2024-09-20T19:01:18.597 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5550000000000002e-06 2024-09-20T19:01:18.598 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.598 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.598 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:18.598 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.802250+0000", 2024-09-20T19:01:18.598 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.1502999999999999e-05 2024-09-20T19:01:18.598 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.598 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.598 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:18.598 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.802322+0000", 2024-09-20T19:01:18.598 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.1723000000000004e-05 2024-09-20T19:01:18.598 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.598 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.598 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,1", 2024-09-20T19:01:18.598 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.802390+0000", 2024-09-20T19:01:18.599 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.8120000000000005e-05 2024-09-20T19:01:18.599 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.599 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.599 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:18.599 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.807367+0000", 2024-09-20T19:01:18.599 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0049773769999999998 2024-09-20T19:01:18.599 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.599 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.599 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:01:18.599 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.807382+0000", 2024-09-20T19:01:18.599 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4667e-05 2024-09-20T19:01:18.599 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.599 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.599 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:18.600 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.929326+0000", 2024-09-20T19:01:18.600 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.121944356 2024-09-20T19:01:18.600 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.600 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.600 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:18.600 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.929356+0000", 2024-09-20T19:01:18.600 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9995e-05 2024-09-20T19:01:18.600 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.600 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.600 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:18.600 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.929380+0000", 2024-09-20T19:01:18.600 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3181e-05 2024-09-20T19:01:18.600 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:18.600 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:18.601 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:18.601 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.601 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.601 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:139220 7.d 7:b6a17747:::benchmark_data_smithi083_70643_object16418:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-20T19:01:18.601 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:42.803402+0000", 2024-09-20T19:01:18.601 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 515.767499402, 2024-09-20T19:01:18.601 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.129377032, 2024-09-20T19:01:18.601 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:18.601 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:18.601 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:18.601 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:01:18.601 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T19:01:18.601 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T19:01:18.601 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 139220 2024-09-20T19:01:18.601 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.602 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:18.602 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.602 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:18.602 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.803402+0000", 2024-09-20T19:01:18.602 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:18.602 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.602 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.602 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:18.602 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.803402+0000", 2024-09-20T19:01:18.602 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:18.602 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.602 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.602 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:18.602 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.803403+0000", 2024-09-20T19:01:18.602 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.097e-06 2024-09-20T19:01:18.603 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.603 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.603 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:18.603 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.803408+0000", 2024-09-20T19:01:18.603 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.561e-06 2024-09-20T19:01:18.603 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.603 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.603 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:18.603 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.803419+0000", 2024-09-20T19:01:18.603 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1591e-05 2024-09-20T19:01:18.603 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.603 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.603 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:18.603 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.803426+0000", 2024-09-20T19:01:18.603 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.9160000000000004e-06 2024-09-20T19:01:18.604 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.604 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.604 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:18.604 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.805298+0000", 2024-09-20T19:01:18.604 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0018719780000000001 2024-09-20T19:01:18.604 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.604 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.604 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:18.604 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.805418+0000", 2024-09-20T19:01:18.604 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000119739 2024-09-20T19:01:18.604 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.604 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.604 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,1", 2024-09-20T19:01:18.604 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.805543+0000", 2024-09-20T19:01:18.604 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012481699999999999 2024-09-20T19:01:18.605 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.605 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.605 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:01:18.605 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.807413+0000", 2024-09-20T19:01:18.605 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001870678 2024-09-20T19:01:18.605 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.605 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.605 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:18.605 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.808024+0000", 2024-09-20T19:01:18.605 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00061090700000000005 2024-09-20T19:01:18.605 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.605 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.605 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:18.605 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.932673+0000", 2024-09-20T19:01:18.605 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.12464900299999999 2024-09-20T19:01:18.606 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.606 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.606 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:18.606 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.932739+0000", 2024-09-20T19:01:18.606 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5979e-05 2024-09-20T19:01:18.606 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.606 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.606 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:18.606 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.932779+0000", 2024-09-20T19:01:18.606 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9765999999999998e-05 2024-09-20T19:01:18.606 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:18.606 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:18.606 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:18.606 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.607 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.607 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14808.0:139224 7.7 7:e3ad72f8:::benchmark_data_smithi083_70643_object16422:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-20T19:01:18.607 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:52:42.806758+0000", 2024-09-20T19:01:18.607 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 515.76414339600001, 2024-09-20T19:01:18.607 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.121962579, 2024-09-20T19:01:18.607 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:18.607 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:18.607 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:18.607 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:01:18.607 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14808", 2024-09-20T19:01:18.607 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T19:01:18.607 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 139224 2024-09-20T19:01:18.607 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.607 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:18.607 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.608 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:18.608 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806758+0000", 2024-09-20T19:01:18.608 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:18.608 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.608 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.608 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:18.608 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806758+0000", 2024-09-20T19:01:18.608 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:18.608 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.608 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.608 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:18.608 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806759+0000", 2024-09-20T19:01:18.608 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.065e-06 2024-09-20T19:01:18.608 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.608 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.609 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:18.609 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806764+0000", 2024-09-20T19:01:18.609 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8010000000000003e-06 2024-09-20T19:01:18.609 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.609 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.609 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:18.609 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806777+0000", 2024-09-20T19:01:18.609 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3292000000000001e-05 2024-09-20T19:01:18.609 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.609 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.609 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:18.609 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806785+0000", 2024-09-20T19:01:18.609 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.9959999999999993e-06 2024-09-20T19:01:18.609 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.610 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.610 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:18.610 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806840+0000", 2024-09-20T19:01:18.610 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4308000000000002e-05 2024-09-20T19:01:18.610 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.610 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.610 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:18.610 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.806948+0000", 2024-09-20T19:01:18.610 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00010843799999999999 2024-09-20T19:01:18.610 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.610 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.610 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,3", 2024-09-20T19:01:18.610 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.807066+0000", 2024-09-20T19:01:18.610 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000117766 2024-09-20T19:01:18.611 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.611 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.611 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:01:18.611 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.809368+0000", 2024-09-20T19:01:18.611 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0023018130000000002 2024-09-20T19:01:18.611 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.611 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.611 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:18.611 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.809785+0000", 2024-09-20T19:01:18.611 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00041736699999999998 2024-09-20T19:01:18.611 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.611 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.611 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:18.611 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.928629+0000", 2024-09-20T19:01:18.611 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.118843746 2024-09-20T19:01:18.612 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.612 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.612 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:18.612 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.928687+0000", 2024-09-20T19:01:18.612 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8418999999999998e-05 2024-09-20T19:01:18.612 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.612 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.612 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:18.612 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:52:42.928721+0000", 2024-09-20T19:01:18.612 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.3568000000000001e-05 2024-09-20T19:01:18.612 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:18.612 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:18.612 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:18.612 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.612 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.613 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14460.0:2700 5.1f 5:f95f44c2:::notify.0:head [watch reconnect cookie 94056120166656 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e292)", 2024-09-20T19:01:18.613 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:00:35.177950+0000", 2024-09-20T19:01:18.613 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 43.392951066999998, 2024-09-20T19:01:18.613 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0123814630000001, 2024-09-20T19:01:18.613 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:18.613 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:18.613 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-09-20T19:01:18.613 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:01:18.613 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14460", 2024-09-20T19:01:18.613 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1555339426", 2024-09-20T19:01:18.613 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 2700 2024-09-20T19:01:18.613 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.613 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:18.613 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.614 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:18.614 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.177950+0000", 2024-09-20T19:01:18.614 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:18.614 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.614 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.614 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:18.614 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.177950+0000", 2024-09-20T19:01:18.614 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:18.614 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.614 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.614 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:18.614 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.177952+0000", 2024-09-20T19:01:18.614 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8199999999999999e-06 2024-09-20T19:01:18.614 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.615 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.615 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:18.615 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.177958+0000", 2024-09-20T19:01:18.615 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8900000000000004e-06 2024-09-20T19:01:18.615 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.615 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.615 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:18.615 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.177973+0000", 2024-09-20T19:01:18.615 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4907e-05 2024-09-20T19:01:18.615 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.615 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.615 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:18.615 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.177980+0000", 2024-09-20T19:01:18.615 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.888e-06 2024-09-20T19:01:18.615 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.616 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.616 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:18.616 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.178048+0000", 2024-09-20T19:01:18.616 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.7621999999999997e-05 2024-09-20T19:01:18.616 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.616 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.616 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op must wait for map", 2024-09-20T19:01:18.616 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.178054+0000", 2024-09-20T19:01:18.616 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5379999999999999e-06 2024-09-20T19:01:18.616 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.616 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.616 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:18.616 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.187513+0000", 2024-09-20T19:01:18.616 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0094585039999999995 2024-09-20T19:01:18.617 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.617 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.617 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:01:18.617 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.187522+0000", 2024-09-20T19:01:18.617 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.2499999999999995e-06 2024-09-20T19:01:18.617 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.617 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.617 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:18.617 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.187878+0000", 2024-09-20T19:01:18.617 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0003558610000001 2024-09-20T19:01:18.617 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.617 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.617 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:18.617 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.188026+0000", 2024-09-20T19:01:18.617 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000148844 2024-09-20T19:01:18.618 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.618 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.618 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:18.618 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.190332+0000", 2024-09-20T19:01:18.618 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.002305339 2024-09-20T19:01:18.618 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:18.618 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:18.618 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:18.618 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.618 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.618 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14460.0:2701 4.5 4:a60f9269:lc::lua_package_allowlist:head [watch reconnect cookie 94056125358080 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e292)", 2024-09-20T19:01:18.618 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:00:35.177999+0000", 2024-09-20T19:01:18.618 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 43.392902321000001, 2024-09-20T19:01:18.618 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.012127011, 2024-09-20T19:01:18.619 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:18.619 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:18.619 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-09-20T19:01:18.619 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:01:18.619 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14460", 2024-09-20T19:01:18.619 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1555339426", 2024-09-20T19:01:18.619 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 2701 2024-09-20T19:01:18.619 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.619 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:18.619 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.619 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:18.619 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.177999+0000", 2024-09-20T19:01:18.619 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:18.619 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.620 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.620 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:18.620 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.177999+0000", 2024-09-20T19:01:18.620 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:18.620 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.620 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.620 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:18.620 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.178000+0000", 2024-09-20T19:01:18.620 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.32e-07 2024-09-20T19:01:18.620 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.620 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.620 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:18.620 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.178002+0000", 2024-09-20T19:01:18.620 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.5019999999999999e-06 2024-09-20T19:01:18.620 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.621 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.621 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:18.621 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.178007+0000", 2024-09-20T19:01:18.621 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.989e-06 2024-09-20T19:01:18.621 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.621 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.621 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:18.621 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.178012+0000", 2024-09-20T19:01:18.621 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4399999999999996e-06 2024-09-20T19:01:18.621 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.621 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.621 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:18.621 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.178072+0000", 2024-09-20T19:01:18.621 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9533000000000001e-05 2024-09-20T19:01:18.621 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.622 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.622 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op must wait for map", 2024-09-20T19:01:18.622 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.178077+0000", 2024-09-20T19:01:18.622 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3299999999999998e-06 2024-09-20T19:01:18.622 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.622 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.622 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:18.622 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.188535+0000", 2024-09-20T19:01:18.622 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.010457582999999999 2024-09-20T19:01:18.622 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.622 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.622 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:01:18.622 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.188543+0000", 2024-09-20T19:01:18.623 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5750000000000003e-06 2024-09-20T19:01:18.623 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.623 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.623 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:18.623 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.187930+0000", 2024-09-20T19:01:18.623 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.99938751299999995 2024-09-20T19:01:18.623 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.623 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.623 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:18.623 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.188063+0000", 2024-09-20T19:01:18.623 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000133003 2024-09-20T19:01:18.623 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.623 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.623 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:18.623 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.190126+0000", 2024-09-20T19:01:18.624 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0020631109999999999 2024-09-20T19:01:18.624 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:18.624 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:18.624 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:18.624 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.624 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.624 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:184478 9.b e299/279 9:df5bc8f0:::benchmark_data_smithi083_124552_object70331:head v 299'11243, mlcod=299'11241)", 2024-09-20T19:01:18.624 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:08.841811+0000", 2024-09-20T19:01:18.624 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 9.7290906289999999, 2024-09-20T19:01:18.624 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.015213545, 2024-09-20T19:01:18.624 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:18.624 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:18.624 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:18.624 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:18.624 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.625 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:18.625 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:08.841811+0000", 2024-09-20T19:01:18.625 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:18.625 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.625 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.625 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:18.625 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:08.841811+0000", 2024-09-20T19:01:18.625 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:18.625 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.625 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.625 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:18.625 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:08.841812+0000", 2024-09-20T19:01:18.625 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0219999999999999e-06 2024-09-20T19:01:18.625 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.625 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.626 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:18.626 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:08.841818+0000", 2024-09-20T19:01:18.626 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0889999999999996e-06 2024-09-20T19:01:18.626 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.626 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.626 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:18.626 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:08.841831+0000", 2024-09-20T19:01:18.626 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3276e-05 2024-09-20T19:01:18.626 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.626 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.626 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:18.626 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:08.841838+0000", 2024-09-20T19:01:18.626 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.331e-06 2024-09-20T19:01:18.627 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.627 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.627 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:18.627 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:08.843783+0000", 2024-09-20T19:01:18.627 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00194594 2024-09-20T19:01:18.627 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.627 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.627 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:18.627 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:08.843805+0000", 2024-09-20T19:01:18.627 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1659e-05 2024-09-20T19:01:18.627 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.627 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.627 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:18.627 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:08.856813+0000", 2024-09-20T19:01:18.627 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.013007863 2024-09-20T19:01:18.628 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.628 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.628 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:18.628 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:08.857024+0000", 2024-09-20T19:01:18.628 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00021136500000000001 2024-09-20T19:01:18.628 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:18.628 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:18.628 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:18.628 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.628 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.628 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:185100 9.0 e299/284 9:0df8bb3a:::benchmark_data_smithi083_124552_object70953:head v 299'11948, mlcod=299'11945)", 2024-09-20T19:01:18.628 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:09.348659+0000", 2024-09-20T19:01:18.628 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 9.2222419969999994, 2024-09-20T19:01:18.628 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.016931277000000002, 2024-09-20T19:01:18.628 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:18.629 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:18.629 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:18.629 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:18.629 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.629 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:18.629 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.348659+0000", 2024-09-20T19:01:18.629 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:18.629 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.629 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.629 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:18.629 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.348659+0000", 2024-09-20T19:01:18.629 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:18.629 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.629 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.629 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:18.630 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.348661+0000", 2024-09-20T19:01:18.630 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1510000000000001e-06 2024-09-20T19:01:18.630 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.630 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.630 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:18.630 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.348668+0000", 2024-09-20T19:01:18.630 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.4580000000000004e-06 2024-09-20T19:01:18.630 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.630 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.630 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:18.630 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.348682+0000", 2024-09-20T19:01:18.630 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3634e-05 2024-09-20T19:01:18.630 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.630 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.631 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:18.631 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.348689+0000", 2024-09-20T19:01:18.631 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.2309999999999999e-06 2024-09-20T19:01:18.631 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.631 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.631 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:18.631 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.350318+0000", 2024-09-20T19:01:18.631 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0016291420000000001 2024-09-20T19:01:18.631 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.631 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.631 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:18.631 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.350336+0000", 2024-09-20T19:01:18.631 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7583000000000001e-05 2024-09-20T19:01:18.631 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.631 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.632 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:18.632 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.365388+0000", 2024-09-20T19:01:18.632 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.015052289 2024-09-20T19:01:18.632 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.632 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.632 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:18.632 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.365591+0000", 2024-09-20T19:01:18.632 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00020278900000000001 2024-09-20T19:01:18.632 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:18.632 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:18.632 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:18.632 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.632 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.632 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:185150 9.b e299/279 9:ddf705d1:::benchmark_data_smithi083_124552_object71003:head v 299'11278, mlcod=299'11274)", 2024-09-20T19:01:18.632 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:09.404104+0000", 2024-09-20T19:01:18.633 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 9.1667978340000005, 2024-09-20T19:01:18.633 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.018813211999999999, 2024-09-20T19:01:18.633 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:18.633 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:18.633 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:18.633 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:18.633 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.633 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:18.633 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.404104+0000", 2024-09-20T19:01:18.633 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:18.633 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.633 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.633 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:18.633 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.404104+0000", 2024-09-20T19:01:18.634 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:18.634 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.634 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.634 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:18.634 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.404105+0000", 2024-09-20T19:01:18.634 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3149999999999999e-06 2024-09-20T19:01:18.634 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.634 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.634 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:18.634 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.404113+0000", 2024-09-20T19:01:18.634 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.3799999999999994e-06 2024-09-20T19:01:18.634 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.634 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.634 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:18.634 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.404124+0000", 2024-09-20T19:01:18.635 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0393e-05 2024-09-20T19:01:18.635 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.635 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.635 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:18.635 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.404130+0000", 2024-09-20T19:01:18.635 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5740000000000001e-06 2024-09-20T19:01:18.635 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.635 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.635 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:18.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:18.637 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.407418+0000", 2024-09-20T19:01:18.637 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0032876149999999998 2024-09-20T19:01:18.637 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.637 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.637 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:18.637 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.407462+0000", 2024-09-20T19:01:18.637 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3977999999999998e-05 2024-09-20T19:01:18.637 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.637 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.637 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:18.637 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.422785+0000", 2024-09-20T19:01:18.637 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.015322682000000001 2024-09-20T19:01:18.637 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.638 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.638 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:18.638 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.422917+0000", 2024-09-20T19:01:18.638 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000132275 2024-09-20T19:01:18.638 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:18.638 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:18.638 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:18.638 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.638 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.638 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:185151 9.b e299/279 9:da69dab2:::benchmark_data_smithi083_124552_object71004:head v 299'11279, mlcod=299'11274)", 2024-09-20T19:01:18.638 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:09.407532+0000", 2024-09-20T19:01:18.638 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 9.1633694840000004, 2024-09-20T19:01:18.638 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.015413344000000001, 2024-09-20T19:01:18.638 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:18.639 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:18.639 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:18.639 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:18.639 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.639 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:18.639 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.407532+0000", 2024-09-20T19:01:18.639 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:18.639 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.639 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.639 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:18.639 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.407532+0000", 2024-09-20T19:01:18.639 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:18.639 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.639 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.640 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:18.640 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.407533+0000", 2024-09-20T19:01:18.640 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2559999999999999e-06 2024-09-20T19:01:18.640 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.640 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.640 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:18.640 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.407542+0000", 2024-09-20T19:01:18.640 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.6840000000000002e-06 2024-09-20T19:01:18.640 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.640 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.640 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:18.640 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.407553+0000", 2024-09-20T19:01:18.640 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1362000000000001e-05 2024-09-20T19:01:18.640 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.640 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.641 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:18.641 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.407560+0000", 2024-09-20T19:01:18.641 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.472e-06 2024-09-20T19:01:18.641 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.641 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.641 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:18.641 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.411257+0000", 2024-09-20T19:01:18.641 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0036971030000000002 2024-09-20T19:01:18.641 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.641 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.641 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:18.641 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.411312+0000", 2024-09-20T19:01:18.641 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4957999999999998e-05 2024-09-20T19:01:18.641 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.642 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.642 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:18.642 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.422921+0000", 2024-09-20T19:01:18.642 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.011608904999999999 2024-09-20T19:01:18.642 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.642 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.642 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:18.642 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.422945+0000", 2024-09-20T19:01:18.642 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.4604e-05 2024-09-20T19:01:18.642 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:18.642 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:18.642 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:18.642 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.642 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.642 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:185170 9.0 e299/284 9:0ff106bc:::benchmark_data_smithi083_124552_object71023:head v 299'11955, mlcod=299'11952)", 2024-09-20T19:01:18.643 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:09.425731+0000", 2024-09-20T19:01:18.643 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 9.1451706510000008, 2024-09-20T19:01:18.643 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.019365067, 2024-09-20T19:01:18.643 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:18.643 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:18.643 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:18.643 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:18.643 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.643 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:18.643 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.425731+0000", 2024-09-20T19:01:18.643 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:18.643 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.643 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.643 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:18.643 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.425731+0000", 2024-09-20T19:01:18.644 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:18.644 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.644 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.644 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:18.644 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.425732+0000", 2024-09-20T19:01:18.644 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1540000000000001e-06 2024-09-20T19:01:18.644 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.644 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.644 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:18.644 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.425738+0000", 2024-09-20T19:01:18.644 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9139999999999996e-06 2024-09-20T19:01:18.644 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.644 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.644 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:18.645 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.425753+0000", 2024-09-20T19:01:18.645 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5608e-05 2024-09-20T19:01:18.645 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.645 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.645 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:18.645 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.425761+0000", 2024-09-20T19:01:18.645 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5970000000000003e-06 2024-09-20T19:01:18.645 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.645 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.645 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:18.645 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.429540+0000", 2024-09-20T19:01:18.645 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0037784649999999999 2024-09-20T19:01:18.645 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.645 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.645 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:18.646 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.429587+0000", 2024-09-20T19:01:18.646 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7375e-05 2024-09-20T19:01:18.646 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.646 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.646 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:18.646 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.445042+0000", 2024-09-20T19:01:18.646 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.015455071000000001 2024-09-20T19:01:18.646 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.646 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.646 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:18.646 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.445096+0000", 2024-09-20T19:01:18.646 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3882999999999999e-05 2024-09-20T19:01:18.646 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:18.646 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:18.646 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:18.647 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.647 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.647 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:185178 9.0 e299/284 9:02151d09:::benchmark_data_smithi083_124552_object71031:head v 299'11957, mlcod=299'11953)", 2024-09-20T19:01:18.647 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:09.432080+0000", 2024-09-20T19:01:18.647 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 9.1388217609999991, 2024-09-20T19:01:18.647 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.017739181, 2024-09-20T19:01:18.647 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:18.647 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:18.647 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:18.647 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:18.647 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.647 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:18.647 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.432080+0000", 2024-09-20T19:01:18.647 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:18.648 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.648 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.648 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:18.648 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.432080+0000", 2024-09-20T19:01:18.648 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:18.648 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.648 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.648 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:18.648 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.432080+0000", 2024-09-20T19:01:18.648 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.8999999999999998e-07 2024-09-20T19:01:18.648 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.648 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.648 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:18.648 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.432084+0000", 2024-09-20T19:01:18.648 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9879999999999998e-06 2024-09-20T19:01:18.649 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.649 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.649 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:18.649 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.432090+0000", 2024-09-20T19:01:18.649 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5659999999999997e-06 2024-09-20T19:01:18.649 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.649 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.649 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:18.649 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.432094+0000", 2024-09-20T19:01:18.649 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.6800000000000001e-06 2024-09-20T19:01:18.649 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.649 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.649 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:18.649 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.435495+0000", 2024-09-20T19:01:18.649 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0034005239999999998 2024-09-20T19:01:18.649 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.649 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.650 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:18.650 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.435521+0000", 2024-09-20T19:01:18.650 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.6438999999999999e-05 2024-09-20T19:01:18.650 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.650 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.650 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:18.650 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.449636+0000", 2024-09-20T19:01:18.650 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.014114484 2024-09-20T19:01:18.650 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.650 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.650 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:18.650 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.449819+0000", 2024-09-20T19:01:18.650 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00018321 2024-09-20T19:01:18.650 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:18.651 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:18.651 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:18.651 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.651 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.651 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:185548 9.1 e299/283 9:8041797c:::benchmark_data_smithi083_124552_object71401:head v 299'11651, mlcod=299'11649)", 2024-09-20T19:01:18.651 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:09.782122+0000", 2024-09-20T19:01:18.651 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 8.7887789670000007, 2024-09-20T19:01:18.651 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.015493623, 2024-09-20T19:01:18.651 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:18.651 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:18.651 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:18.651 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:18.651 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.651 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:18.652 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.782122+0000", 2024-09-20T19:01:18.652 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:18.652 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.652 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.652 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:18.652 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.782122+0000", 2024-09-20T19:01:18.652 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:18.652 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.652 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.652 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:18.652 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.782123+0000", 2024-09-20T19:01:18.652 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.7499999999999999e-07 2024-09-20T19:01:18.652 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.652 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.652 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:18.653 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.782130+0000", 2024-09-20T19:01:18.653 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.9199999999999998e-06 2024-09-20T19:01:18.653 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.653 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.653 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:18.653 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.782143+0000", 2024-09-20T19:01:18.653 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2679e-05 2024-09-20T19:01:18.653 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.653 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.653 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:18.653 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.782150+0000", 2024-09-20T19:01:18.653 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.6599999999999998e-06 2024-09-20T19:01:18.653 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.653 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.653 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:18.654 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.782337+0000", 2024-09-20T19:01:18.654 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00018797099999999999 2024-09-20T19:01:18.654 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.654 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.654 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:18.654 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.782382+0000", 2024-09-20T19:01:18.654 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.4316000000000001e-05 2024-09-20T19:01:18.654 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.654 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.654 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:18.654 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.797448+0000", 2024-09-20T19:01:18.654 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.01506656 2024-09-20T19:01:18.654 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.654 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.655 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:18.655 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:09.797616+0000", 2024-09-20T19:01:18.655 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00016774199999999999 2024-09-20T19:01:18.655 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:18.655 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:18.655 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:18.655 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.655 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.655 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:192385 9.b e300/279 9:d94d42a0:::benchmark_data_smithi083_124552_object78238:head v 300'11720, mlcod=300'11717)", 2024-09-20T19:01:18.655 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:15.494042+0000", 2024-09-20T19:01:18.655 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 3.0768591449999998, 2024-09-20T19:01:18.655 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.017375912, 2024-09-20T19:01:18.655 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:18.655 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:18.656 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:18.656 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:18.656 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.656 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:18.656 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:15.494042+0000", 2024-09-20T19:01:18.656 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:18.656 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.656 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.656 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:18.656 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:15.494042+0000", 2024-09-20T19:01:18.656 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:18.656 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.656 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.656 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:18.656 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:15.494043+0000", 2024-09-20T19:01:18.657 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.6200000000000006e-07 2024-09-20T19:01:18.657 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.657 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.657 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:18.657 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:15.494049+0000", 2024-09-20T19:01:18.657 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9309999999999996e-06 2024-09-20T19:01:18.657 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.657 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.657 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:18.657 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:15.494065+0000", 2024-09-20T19:01:18.657 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5588000000000001e-05 2024-09-20T19:01:18.657 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.657 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.657 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:18.657 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:15.494072+0000", 2024-09-20T19:01:18.658 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5610000000000001e-06 2024-09-20T19:01:18.658 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.658 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.658 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:18.658 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:15.494993+0000", 2024-09-20T19:01:18.658 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00092069299999999999 2024-09-20T19:01:18.658 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.658 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.658 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:18.658 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:15.495011+0000", 2024-09-20T19:01:18.658 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7872000000000001e-05 2024-09-20T19:01:18.658 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.658 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.658 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:18.659 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:15.511375+0000", 2024-09-20T19:01:18.659 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.016364337999999999 2024-09-20T19:01:18.659 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.659 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.659 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:18.659 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:15.511418+0000", 2024-09-20T19:01:18.659 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.2966999999999997e-05 2024-09-20T19:01:18.659 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:18.659 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:18.659 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:18.659 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.659 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.659 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:193609 9.0 e300/284 9:016e441b:::benchmark_data_smithi083_124552_object79462:head v 300'12456, mlcod=300'12454)", 2024-09-20T19:01:18.659 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:16.409440+0000", 2024-09-20T19:01:18.659 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 2.1614613770000002, 2024-09-20T19:01:18.660 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.018953964, 2024-09-20T19:01:18.660 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:18.660 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:18.660 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:18.660 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:18.660 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.660 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:18.660 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:16.409440+0000", 2024-09-20T19:01:18.660 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:18.660 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.660 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.660 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:18.660 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:16.409440+0000", 2024-09-20T19:01:18.660 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:18.661 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.661 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.661 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:18.661 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:16.409441+0000", 2024-09-20T19:01:18.661 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.5199999999999995e-07 2024-09-20T19:01:18.661 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.661 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.661 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:18.661 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:16.409448+0000", 2024-09-20T19:01:18.661 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.0509999999999999e-06 2024-09-20T19:01:18.661 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.661 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.661 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:18.661 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:16.409463+0000", 2024-09-20T19:01:18.661 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5373999999999999e-05 2024-09-20T19:01:18.662 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.662 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.662 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:18.662 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:16.409471+0000", 2024-09-20T19:01:18.662 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.2880000000000004e-06 2024-09-20T19:01:18.662 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.662 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.662 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:18.662 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:16.409594+0000", 2024-09-20T19:01:18.662 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000123609 2024-09-20T19:01:18.662 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.662 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.662 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:18.663 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:16.409649+0000", 2024-09-20T19:01:18.663 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4530999999999997e-05 2024-09-20T19:01:18.663 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.663 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.663 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:18.663 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:16.428352+0000", 2024-09-20T19:01:18.663 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.018703218000000001 2024-09-20T19:01:18.663 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.663 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.663 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:18.663 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:16.428394+0000", 2024-09-20T19:01:18.663 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.1940999999999997e-05 2024-09-20T19:01:18.663 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:18.663 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:18.664 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:18.664 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.664 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.664 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:193614 9.0 e300/284 9:011f6d03:::benchmark_data_smithi083_124552_object79467:head v 300'12457, mlcod=300'12454)", 2024-09-20T19:01:18.664 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:16.409688+0000", 2024-09-20T19:01:18.664 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 2.1612132920000002, 2024-09-20T19:01:18.664 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.018735066000000002, 2024-09-20T19:01:18.664 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:18.664 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:18.664 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:18.664 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:18.664 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.664 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:18.664 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:16.409688+0000", 2024-09-20T19:01:18.665 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:18.665 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.665 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.665 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:18.665 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:16.409688+0000", 2024-09-20T19:01:18.665 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:18.665 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.665 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.665 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:18.665 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:16.409689+0000", 2024-09-20T19:01:18.665 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4700000000000001e-07 2024-09-20T19:01:18.665 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.665 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.665 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:18.665 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:16.409694+0000", 2024-09-20T19:01:18.666 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.023e-06 2024-09-20T19:01:18.666 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.666 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.666 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:18.666 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:16.409713+0000", 2024-09-20T19:01:18.666 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9100999999999999e-05 2024-09-20T19:01:18.666 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.666 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.666 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:18.666 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:16.409719+0000", 2024-09-20T19:01:18.666 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4670000000000001e-06 2024-09-20T19:01:18.666 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.666 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.666 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:18.666 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:16.413074+0000", 2024-09-20T19:01:18.667 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00335484 2024-09-20T19:01:18.667 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.667 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.667 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:18.667 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:16.413135+0000", 2024-09-20T19:01:18.667 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0795000000000003e-05 2024-09-20T19:01:18.667 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.667 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.667 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:18.667 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:16.428397+0000", 2024-09-20T19:01:18.667 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.015261844 2024-09-20T19:01:18.667 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.667 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.667 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:18.667 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:16.428423+0000", 2024-09-20T19:01:18.667 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.6349000000000001e-05 2024-09-20T19:01:18.668 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:18.668 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:18.668 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:18.668 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.668 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.668 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:193616 9.0 e300/284 9:0b263a69:::benchmark_data_smithi083_124552_object79469:head v 300'12458, mlcod=300'12454)", 2024-09-20T19:01:18.668 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:16.412946+0000", 2024-09-20T19:01:18.668 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 2.157955372, 2024-09-20T19:01:18.668 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.015506229, 2024-09-20T19:01:18.668 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:18.668 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:18.668 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:18.668 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:18.668 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.668 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:18.669 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:16.412946+0000", 2024-09-20T19:01:18.669 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:18.669 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.669 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.669 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:18.669 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:16.412946+0000", 2024-09-20T19:01:18.669 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:18.669 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.669 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.669 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:18.669 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:16.412947+0000", 2024-09-20T19:01:18.669 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.47e-07 2024-09-20T19:01:18.669 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.669 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.670 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:18.670 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:16.412953+0000", 2024-09-20T19:01:18.670 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.1029999999999998e-06 2024-09-20T19:01:18.670 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.670 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.670 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:18.670 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:16.412980+0000", 2024-09-20T19:01:18.670 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.7423000000000002e-05 2024-09-20T19:01:18.670 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.670 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.670 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:18.670 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:16.412988+0000", 2024-09-20T19:01:18.670 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.515e-06 2024-09-20T19:01:18.670 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.671 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.671 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:18.671 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:16.416273+0000", 2024-09-20T19:01:18.671 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0032848669999999999 2024-09-20T19:01:18.671 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.671 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.671 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:18.671 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:16.416336+0000", 2024-09-20T19:01:18.671 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2705000000000004e-05 2024-09-20T19:01:18.671 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.671 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.671 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:18.671 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:16.428427+0000", 2024-09-20T19:01:18.671 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.012091421999999999 2024-09-20T19:01:18.671 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.672 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.672 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:18.672 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:16.428452+0000", 2024-09-20T19:01:18.672 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.5347e-05 2024-09-20T19:01:18.672 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:18.672 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:18.672 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:18.672 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.672 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.672 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:193744 9.b e300/279 9:d1a8cdd7:::benchmark_data_smithi083_124552_object79597:head v 300'11809, mlcod=300'11805)", 2024-09-20T19:01:18.672 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:16.525039+0000", 2024-09-20T19:01:18.672 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 2.0458626799999999, 2024-09-20T19:01:18.672 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.015615172, 2024-09-20T19:01:18.672 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:18.673 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:18.673 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:18.673 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:18.673 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.673 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:18.673 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:16.525039+0000", 2024-09-20T19:01:18.673 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:18.673 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.673 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.673 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:18.673 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:16.525039+0000", 2024-09-20T19:01:18.673 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:18.673 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.673 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.674 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:18.674 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:16.525040+0000", 2024-09-20T19:01:18.674 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.063e-06 2024-09-20T19:01:18.674 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.674 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.674 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:18.674 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:16.525047+0000", 2024-09-20T19:01:18.674 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.1969999999999999e-06 2024-09-20T19:01:18.674 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.674 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.674 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:18.674 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:16.525069+0000", 2024-09-20T19:01:18.674 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1532999999999999e-05 2024-09-20T19:01:18.674 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.675 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.675 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:18.675 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:16.525076+0000", 2024-09-20T19:01:18.675 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.7100000000000007e-06 2024-09-20T19:01:18.675 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.675 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.675 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:18.675 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:16.530854+0000", 2024-09-20T19:01:18.675 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0057775170000000002 2024-09-20T19:01:18.675 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.675 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.675 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:18.675 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:16.530931+0000", 2024-09-20T19:01:18.676 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.7521999999999994e-05 2024-09-20T19:01:18.676 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.676 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.676 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:18.676 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:16.540600+0000", 2024-09-20T19:01:18.676 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0096682079999999993 2024-09-20T19:01:18.676 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.676 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.676 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:18.676 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:16.540654+0000", 2024-09-20T19:01:18.676 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4422000000000002e-05 2024-09-20T19:01:18.676 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:18.676 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:18.676 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:18.676 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.677 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.677 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:193746 9.1 e300/283 9:89dd25d5:::benchmark_data_smithi083_124552_object79599:head v 300'12165, mlcod=300'12161)", 2024-09-20T19:01:18.677 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:16.526422+0000", 2024-09-20T19:01:18.677 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 2.044479575, 2024-09-20T19:01:18.677 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.015594708000000001, 2024-09-20T19:01:18.677 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:18.677 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:18.677 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:18.677 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:18.677 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.677 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:18.677 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:16.526422+0000", 2024-09-20T19:01:18.677 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:18.677 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.678 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.678 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:18.678 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:16.526422+0000", 2024-09-20T19:01:18.678 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:18.678 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.678 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.678 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:18.678 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:16.526423+0000", 2024-09-20T19:01:18.678 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.9999999999999996e-07 2024-09-20T19:01:18.678 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.678 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.678 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:18.678 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:16.526429+0000", 2024-09-20T19:01:18.678 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.3629999999999999e-06 2024-09-20T19:01:18.679 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.679 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.679 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:18.679 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:16.526443+0000", 2024-09-20T19:01:18.679 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3721e-05 2024-09-20T19:01:18.679 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.679 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.679 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:18.679 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:16.526449+0000", 2024-09-20T19:01:18.679 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.3230000000000003e-06 2024-09-20T19:01:18.679 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.679 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.679 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:18.679 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:16.528756+0000", 2024-09-20T19:01:18.680 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0023064069999999999 2024-09-20T19:01:18.680 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.680 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.680 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:18.680 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:16.528805+0000", 2024-09-20T19:01:18.680 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.9325e-05 2024-09-20T19:01:18.680 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.680 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.680 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:18.680 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:16.541984+0000", 2024-09-20T19:01:18.680 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.013179461999999999 2024-09-20T19:01:18.680 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:18.680 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:18.680 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:18.680 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:16.542017+0000", 2024-09-20T19:01:18.681 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.2206999999999998e-05 2024-09-20T19:01:18.681 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:18.681 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:18.681 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:18.681 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:18.681 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:18.681 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T19:01:18.681 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:18.682 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:18.753 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:18 smithi083 ceph-mon[29421]: 3.10 scrub starts 2024-09-20T19:01:18.753 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:18 smithi083 ceph-mon[29421]: 3.10 scrub ok 2024-09-20T19:01:18.753 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:18 smithi083 ceph-mon[29421]: 3.4 scrub starts 2024-09-20T19:01:18.753 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:18 smithi083 ceph-mon[29421]: 3.4 scrub ok 2024-09-20T19:01:18.753 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:18 smithi083 ceph-mon[29421]: 9.d scrub ok 2024-09-20T19:01:18.753 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:18 smithi083 ceph-mon[29421]: 3.17 scrub starts 2024-09-20T19:01:18.753 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:18 smithi083 ceph-mon[29421]: 3.17 scrub ok 2024-09-20T19:01:18.753 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:18 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:01:18.753 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:18 smithi083 ceph-mon[29421]: pgmap v560: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 2.3 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.24k op/s; 6880/111435 objects degraded (6.174%) 2024-09-20T19:01:18.753 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:18 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:01:18.753 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:18 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:01:18.753 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:18 smithi083 ceph-mon[29421]: osdmap e301: 8 total, 7 up, 8 in 2024-09-20T19:01:18.754 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:18 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/387430631' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:01:18.754 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:18 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:01:18.754 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:18 smithi083 ceph-mon[36260]: 3.10 scrub starts 2024-09-20T19:01:18.754 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:18 smithi083 ceph-mon[36260]: 3.10 scrub ok 2024-09-20T19:01:18.754 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:18 smithi083 ceph-mon[36260]: 3.4 scrub starts 2024-09-20T19:01:18.754 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:18 smithi083 ceph-mon[36260]: 3.4 scrub ok 2024-09-20T19:01:18.754 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:18 smithi083 ceph-mon[36260]: 9.d scrub ok 2024-09-20T19:01:18.754 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:18 smithi083 ceph-mon[36260]: 3.17 scrub starts 2024-09-20T19:01:18.754 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:18 smithi083 ceph-mon[36260]: 3.17 scrub ok 2024-09-20T19:01:18.754 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:18 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:01:18.754 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:18 smithi083 ceph-mon[36260]: pgmap v560: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 2.3 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.24k op/s; 6880/111435 objects degraded (6.174%) 2024-09-20T19:01:18.754 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:18 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:01:18.755 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:18 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:01:18.755 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:18 smithi083 ceph-mon[36260]: osdmap e301: 8 total, 7 up, 8 in 2024-09-20T19:01:18.755 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:18 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/387430631' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:01:18.755 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:18 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:01:18.782 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:18.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:18.827 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:18.827 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:18.928 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:18.967 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:18.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:18.968 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:19.069 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:19.106 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:19.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:19.107 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:19.208 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:19.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:19.241 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:19.241 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:19.342 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:19.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:19.404 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:19.404 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:19.484 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:01:19.505 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:19.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:19.538 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:19.538 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:19.616 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:19 smithi083 ceph-mon[29421]: 9.a scrub starts 2024-09-20T19:01:19.616 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:19 smithi083 ceph-mon[29421]: 5.12 scrub starts 2024-09-20T19:01:19.616 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:19 smithi083 ceph-mon[29421]: 5.12 scrub ok 2024-09-20T19:01:19.616 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:19 smithi083 ceph-mon[29421]: 5.6 scrub starts 2024-09-20T19:01:19.616 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:19 smithi083 ceph-mon[29421]: 5.6 scrub ok 2024-09-20T19:01:19.616 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:19 smithi083 ceph-mon[29421]: 9.a scrub ok 2024-09-20T19:01:19.616 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:19 smithi083 ceph-mon[29421]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:01:19.616 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:19 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T19:01:19.616 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:19 smithi083 ceph-mon[29421]: osdmap e302: 8 total, 7 up, 8 in 2024-09-20T19:01:19.617 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:19 smithi083 ceph-mon[36260]: 9.a scrub starts 2024-09-20T19:01:19.617 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:19 smithi083 ceph-mon[36260]: 5.12 scrub starts 2024-09-20T19:01:19.617 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:19 smithi083 ceph-mon[36260]: 5.12 scrub ok 2024-09-20T19:01:19.617 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:19 smithi083 ceph-mon[36260]: 5.6 scrub starts 2024-09-20T19:01:19.617 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:19 smithi083 ceph-mon[36260]: 5.6 scrub ok 2024-09-20T19:01:19.617 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:19 smithi083 ceph-mon[36260]: 9.a scrub ok 2024-09-20T19:01:19.617 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:19 smithi083 ceph-mon[36260]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:01:19.617 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:19 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T19:01:19.617 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:19 smithi083 ceph-mon[36260]: osdmap e302: 8 total, 7 up, 8 in 2024-09-20T19:01:19.629 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:19 smithi142 ceph-mon[32508]: 9.a scrub starts 2024-09-20T19:01:19.629 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:19 smithi142 ceph-mon[32508]: 5.12 scrub starts 2024-09-20T19:01:19.629 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:19 smithi142 ceph-mon[32508]: 5.12 scrub ok 2024-09-20T19:01:19.630 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:19 smithi142 ceph-mon[32508]: 5.6 scrub starts 2024-09-20T19:01:19.630 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:19 smithi142 ceph-mon[32508]: 5.6 scrub ok 2024-09-20T19:01:19.630 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:19 smithi142 ceph-mon[32508]: 9.a scrub ok 2024-09-20T19:01:19.630 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:19 smithi142 ceph-mon[32508]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:01:19.630 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:19 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T19:01:19.630 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:19 smithi142 ceph-mon[32508]: osdmap e302: 8 total, 7 up, 8 in 2024-09-20T19:01:19.631 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-20T19:01:19.639 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:19.728 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:19.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:19.729 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:19.829 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:19.862 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:19.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:19.863 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:19.963 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:20.011 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:20.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:20.012 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:20.048 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:01:20.113 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:20.115 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:01:20.115 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:01:20.143 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:20.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:20.144 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:20.154 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:01:20.154 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:01:20.188 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:01:20.188 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:01:20.227 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:01:20.227 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:01:20.227 INFO:teuthology.orchestra.run.smithi083.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-20T19:01:20.244 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:20.265 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:01:20.265 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:01:20.266 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:01:19 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4[112539]: 2024-09-20T19:01:19.980+0000 7f7b5c931640 -1 osd.4 291 log_to_monitors true 2024-09-20T19:01:20.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:20.283 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:20.284 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:20.301 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:01:20.301 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:01:20.332 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:01:20.332 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:01:20.384 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:20.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:20.427 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:20.428 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:20.528 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:20.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:20 smithi083 ceph-mon[29421]: pgmap v563: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 2.1 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.51k op/s; 6495/103662 objects degraded (6.266%) 2024-09-20T19:01:20.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:20 smithi083 ceph-mon[29421]: from='osd.4 v2:172.21.15.142:6800/3322919402' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-20T19:01:20.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:20 smithi083 ceph-mon[29421]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-20T19:01:20.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:20 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/967432650' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:01:20.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:20 smithi083 ceph-mon[36260]: pgmap v563: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 2.1 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.51k op/s; 6495/103662 objects degraded (6.266%) 2024-09-20T19:01:20.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:20 smithi083 ceph-mon[36260]: from='osd.4 v2:172.21.15.142:6800/3322919402' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-20T19:01:20.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:20 smithi083 ceph-mon[36260]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-20T19:01:20.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:20 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/967432650' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:01:20.551 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:20.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:20.553 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:20.654 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:20.678 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:20.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:20.679 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:20.683 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:20 smithi142 ceph-mon[32508]: pgmap v563: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 2.1 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.51k op/s; 6495/103662 objects degraded (6.266%) 2024-09-20T19:01:20.683 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:20 smithi142 ceph-mon[32508]: from='osd.4 v2:172.21.15.142:6800/3322919402' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-20T19:01:20.684 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:20 smithi142 ceph-mon[32508]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-20T19:01:20.684 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:20 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/967432650' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:01:20.781 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:20.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:20.810 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:20.810 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:20.811 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T19:01:20.811 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:01:20.911 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:20.943 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:20.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:20.944 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:21.015 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:01:20 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4[112539]: 2024-09-20T19:01:20.877+0000 7f7b506d3640 -1 osd.4 291 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-20T19:01:21.045 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:21.045 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:01:20 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:01:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:01:21.078 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:21.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:21.079 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:21.180 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:21.218 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:21.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:21.219 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:21.319 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:21.352 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:21.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:21.353 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:21.436 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T19:01:21.454 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:21.471 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:21 smithi083 ceph-mon[36260]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-20T19:01:21.471 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:21 smithi083 ceph-mon[36260]: from='osd.4 v2:172.21.15.142:6800/3322919402' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-20T19:01:21.471 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:21 smithi083 ceph-mon[36260]: osdmap e303: 8 total, 7 up, 8 in 2024-09-20T19:01:21.471 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:21 smithi083 ceph-mon[36260]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-20T19:01:21.471 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:21 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 6495/103662 objects degraded (6.266%), 27 pgs degraded (PG_DEGRADED) 2024-09-20T19:01:21.471 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:21 smithi083 ceph-mon[36260]: from='osd.4 ' entity='osd.4' 2024-09-20T19:01:21.471 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:21 smithi083 ceph-mon[29421]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-20T19:01:21.472 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:21 smithi083 ceph-mon[29421]: from='osd.4 v2:172.21.15.142:6800/3322919402' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-20T19:01:21.472 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:21 smithi083 ceph-mon[29421]: osdmap e303: 8 total, 7 up, 8 in 2024-09-20T19:01:21.472 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:21 smithi083 ceph-mon[29421]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-20T19:01:21.472 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:21 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 6495/103662 objects degraded (6.266%), 27 pgs degraded (PG_DEGRADED) 2024-09-20T19:01:21.472 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:21 smithi083 ceph-mon[29421]: from='osd.4 ' entity='osd.4' 2024-09-20T19:01:21.481 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:21.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:21.482 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:21.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:21 smithi142 ceph-mon[32508]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-20T19:01:21.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:21 smithi142 ceph-mon[32508]: from='osd.4 v2:172.21.15.142:6800/3322919402' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-20T19:01:21.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:21 smithi142 ceph-mon[32508]: osdmap e303: 8 total, 7 up, 8 in 2024-09-20T19:01:21.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:21 smithi142 ceph-mon[32508]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-09-20T19:01:21.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:21 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 6495/103662 objects degraded (6.266%), 27 pgs degraded (PG_DEGRADED) 2024-09-20T19:01:21.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:21 smithi142 ceph-mon[32508]: from='osd.4 ' entity='osd.4' 2024-09-20T19:01:21.535 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_blocked_ops 2024-09-20T19:01:21.580 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-20T19:01:21.580 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-20T19:01:21.582 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:21.656 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:21.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:21.657 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:21.758 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:21.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:21.796 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:21.796 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:21.856 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:01:21.896 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:21.919 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:21.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:21.920 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:21.948 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:01:22.020 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:22.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:22.044 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:22.044 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:22.145 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:22.167 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:22.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:22.168 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:22.268 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:22.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:22.292 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:22.293 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:22.393 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:22.416 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:22.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:22.417 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:22.519 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:22.540 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:22.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:22.541 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:22.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:22 smithi083 ceph-mon[36260]: pgmap v565: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 2.1 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.01k op/s; 6495/103662 objects degraded (6.266%) 2024-09-20T19:01:22.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:22 smithi083 ceph-mon[36260]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-20T19:01:22.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:22 smithi083 ceph-mon[36260]: osd.4 v2:172.21.15.142:6800/3322919402 boot 2024-09-20T19:01:22.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:22 smithi083 ceph-mon[36260]: osdmap e304: 8 total, 8 up, 8 in 2024-09-20T19:01:22.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:22 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-20T19:01:22.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:22 smithi083 ceph-mon[29421]: pgmap v565: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 2.1 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.01k op/s; 6495/103662 objects degraded (6.266%) 2024-09-20T19:01:22.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:22 smithi083 ceph-mon[29421]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-20T19:01:22.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:22 smithi083 ceph-mon[29421]: osd.4 v2:172.21.15.142:6800/3322919402 boot 2024-09-20T19:01:22.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:22 smithi083 ceph-mon[29421]: osdmap e304: 8 total, 8 up, 8 in 2024-09-20T19:01:22.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:22 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-20T19:01:22.642 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:22.664 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:22.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:22.665 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:22.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:22 smithi142 ceph-mon[32508]: pgmap v565: 148 pgs: 34 active+undersized, 27 active+undersized+degraded, 87 active+clean; 2.1 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.01k op/s; 6495/103662 objects degraded (6.266%) 2024-09-20T19:01:22.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:22 smithi142 ceph-mon[32508]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-20T19:01:22.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:22 smithi142 ceph-mon[32508]: osd.4 v2:172.21.15.142:6800/3322919402 boot 2024-09-20T19:01:22.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:22 smithi142 ceph-mon[32508]: osdmap e304: 8 total, 8 up, 8 in 2024-09-20T19:01:22.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:22 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-20T19:01:22.765 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:22.787 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:22.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:22.788 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:22.834 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-20T19:01:22.888 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:22.909 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:22.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:22.909 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:22.932 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-20T19:01:22.961 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T19:01:22.962 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:01:23.010 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:23.031 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:23.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:23.032 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:23.134 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:23.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:23.157 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:23.157 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:23.258 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:23.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:23.282 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:23.282 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:23.382 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:23.405 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:23.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:23.406 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:23.507 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:23.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:23.532 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:23.532 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:23.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:23 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:01:23.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:23 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:01:23.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:23 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1175886666' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:01:23.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:23 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:01:23.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:23 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T19:01:23.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:23 smithi083 ceph-mon[29421]: osdmap e305: 8 total, 8 up, 8 in 2024-09-20T19:01:23.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:23 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:01:23.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:23 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:01:23.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:23 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1175886666' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:01:23.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:23 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:01:23.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:23 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T19:01:23.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:23 smithi083 ceph-mon[36260]: osdmap e305: 8 total, 8 up, 8 in 2024-09-20T19:01:23.581 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T19:01:23.633 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:23.654 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:23.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:23.655 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:23.755 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:23.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:23 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:01:23.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:23 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:01:23.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:23 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1175886666' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:01:23.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:23 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:01:23.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:23 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T19:01:23.767 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:23 smithi142 ceph-mon[32508]: osdmap e305: 8 total, 8 up, 8 in 2024-09-20T19:01:23.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:23.778 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:23.778 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:23.842 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:01:23.879 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:23.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:23.901 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:23.901 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:24.002 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:24.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:24.025 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:24.025 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:24.126 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:24.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:24.150 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:24.151 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:24.252 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:24.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:24.290 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:24.290 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:24.391 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:24.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:24.428 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:24.428 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:24.471 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:01:24.472 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:01:24.500 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:01:24.501 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:01:24.529 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:24.532 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:01:24.532 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:01:24.563 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:24.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:24.565 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:24.565 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:24 smithi083 ceph-mon[29421]: pgmap v568: 148 pgs: 10 peering, 29 active+undersized, 22 active+undersized+degraded, 87 active+clean; 1.7 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.28k op/s; 3633/85626 objects degraded (4.243%) 2024-09-20T19:01:24.565 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:24 smithi083 ceph-mon[29421]: osdmap e306: 8 total, 8 up, 8 in 2024-09-20T19:01:24.565 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:24 smithi083 ceph-mon[36260]: pgmap v568: 148 pgs: 10 peering, 29 active+undersized, 22 active+undersized+degraded, 87 active+clean; 1.7 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.28k op/s; 3633/85626 objects degraded (4.243%) 2024-09-20T19:01:24.565 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:24 smithi083 ceph-mon[36260]: osdmap e306: 8 total, 8 up, 8 in 2024-09-20T19:01:24.566 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:01:24.566 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:01:24.595 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:01:24.595 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:01:24.629 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:01:24.629 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:01:24.658 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:01:24.658 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:01:24.665 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:24.693 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:01:24.693 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:01:24.701 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:24.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:24.702 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:24.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:24 smithi142 ceph-mon[32508]: pgmap v568: 148 pgs: 10 peering, 29 active+undersized, 22 active+undersized+degraded, 87 active+clean; 1.7 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.28k op/s; 3633/85626 objects degraded (4.243%) 2024-09-20T19:01:24.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:24 smithi142 ceph-mon[32508]: osdmap e306: 8 total, 8 up, 8 in 2024-09-20T19:01:24.803 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:24.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:24.839 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:24.839 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:24.940 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:24.981 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:24.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:24.982 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:25.083 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:25.098 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_historic_ops 2024-09-20T19:01:25.130 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:25.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:25.132 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:25.185 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:01:25.233 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:25.262 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:25.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:25.263 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:25.364 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:25.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:25.414 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:25.414 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:25.515 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:25.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:25.563 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:25.563 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:25.627 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T19:01:25.634 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:25 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1489646005' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:01:25.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:25 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1489646005' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:01:25.657 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:25 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1489646005' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:01:25.664 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:25.718 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:25.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:25.719 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:25.819 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:25.856 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:25.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:25.857 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:25.958 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:25.995 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T19:01:25.995 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-09-20T19:01:25.995 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:25.995 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:202076 9.c e308/305 9:35bee498:::benchmark_data_smithi083_124552_object87929:head v 308'12555, mlcod=293'9029)", 2024-09-20T19:01:25.995 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:25.973575+0000", 2024-09-20T19:01:25.995 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.0061066669999999997, 2024-09-20T19:01:25.995 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0061364940000000001, 2024-09-20T19:01:25.995 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:25.996 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:25.996 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-09-20T19:01:25.996 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:25.996 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:25.996 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:25.996 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.973575+0000", 2024-09-20T19:01:25.996 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:25.996 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:25.996 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:25.996 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:25.996 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.973575+0000", 2024-09-20T19:01:25.996 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:25.996 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:25.996 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:25.996 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:25.997 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.973577+0000", 2024-09-20T19:01:25.997 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.339e-06 2024-09-20T19:01:25.997 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:25.997 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:25.997 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:25.997 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.973584+0000", 2024-09-20T19:01:25.997 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.7009999999999996e-06 2024-09-20T19:01:25.997 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:25.997 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:25.997 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:25.997 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.973597+0000", 2024-09-20T19:01:25.997 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2357999999999999e-05 2024-09-20T19:01:25.997 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:25.997 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:25.998 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:25.998 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.973605+0000", 2024-09-20T19:01:25.998 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.8030000000000003e-06 2024-09-20T19:01:25.998 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:25.998 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:25.998 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:25.998 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.977546+0000", 2024-09-20T19:01:25.998 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0039402079999999997 2024-09-20T19:01:25.998 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:25.998 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:25.998 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:25.998 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.977567+0000", 2024-09-20T19:01:25.998 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1807e-05 2024-09-20T19:01:25.998 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:25.999 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:25.999 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:25.999 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:25.999 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:25.999 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:202031 9.4 e308/305 9:239c9fd6:::benchmark_data_smithi083_124552_object87884:head v 308'12455, mlcod=293'8878)", 2024-09-20T19:01:25.999 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:25.931089+0000", 2024-09-20T19:01:25.999 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.048592409000000003, 2024-09-20T19:01:25.999 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.048676428000000001, 2024-09-20T19:01:25.999 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:25.999 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:25.999 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-09-20T19:01:25.999 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:25.999 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:25.999 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:26.000 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.931089+0000", 2024-09-20T19:01:26.000 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:26.000 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:26.000 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.000 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:26.000 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.931089+0000", 2024-09-20T19:01:26.000 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:26.000 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:26.000 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.000 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:26.000 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.931090+0000", 2024-09-20T19:01:26.000 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.0499999999999999e-07 2024-09-20T19:01:26.000 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:26.000 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.001 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:26.001 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.931095+0000", 2024-09-20T19:01:26.001 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8380000000000001e-06 2024-09-20T19:01:26.001 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:26.001 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.001 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:26.001 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.931100+0000", 2024-09-20T19:01:26.001 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3109999999999996e-06 2024-09-20T19:01:26.001 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:26.001 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.001 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:26.001 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.931104+0000", 2024-09-20T19:01:26.001 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.216e-06 2024-09-20T19:01:26.002 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:26.002 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.002 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:26.002 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.931586+0000", 2024-09-20T19:01:26.002 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000481528 2024-09-20T19:01:26.002 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:26.002 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.002 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:26.002 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.931633+0000", 2024-09-20T19:01:26.002 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7525000000000003e-05 2024-09-20T19:01:26.002 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:26.002 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:26.002 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:26.002 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:26.003 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.003 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:202032 9.4 e308/305 9:2d783284:::benchmark_data_smithi083_124552_object87885:head v 308'12456, mlcod=293'8878)", 2024-09-20T19:01:26.003 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:25.931110+0000", 2024-09-20T19:01:26.003 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.048571430999999998, 2024-09-20T19:01:26.003 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.048703335, 2024-09-20T19:01:26.003 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:26.003 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:26.003 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-09-20T19:01:26.003 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:26.003 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.003 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:26.003 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.931110+0000", 2024-09-20T19:01:26.003 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:26.003 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:26.004 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.004 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:26.004 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.931110+0000", 2024-09-20T19:01:26.004 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:26.004 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:26.004 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.004 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:26.004 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.931111+0000", 2024-09-20T19:01:26.004 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6500000000000001e-07 2024-09-20T19:01:26.004 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:26.004 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.004 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:26.004 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.931114+0000", 2024-09-20T19:01:26.005 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9679999999999998e-06 2024-09-20T19:01:26.005 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:26.005 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.005 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:26.005 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.931118+0000", 2024-09-20T19:01:26.005 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0600000000000001e-06 2024-09-20T19:01:26.005 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:26.005 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.005 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:26.005 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.931121+0000", 2024-09-20T19:01:26.005 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.473e-06 2024-09-20T19:01:26.005 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:26.005 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.005 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:26.005 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.940478+0000", 2024-09-20T19:01:26.005 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0093569529999999995 2024-09-20T19:01:26.006 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:26.006 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.006 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:26.006 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.940509+0000", 2024-09-20T19:01:26.006 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.0880999999999998e-05 2024-09-20T19:01:26.006 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:26.006 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:26.006 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:26.006 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:26.006 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.006 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:202052 9.4 e308/305 9:28f81496:::benchmark_data_smithi083_124552_object87905:head v 308'12457, mlcod=293'8878)", 2024-09-20T19:01:26.006 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:25.938366+0000", 2024-09-20T19:01:26.006 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.041316063, 2024-09-20T19:01:26.006 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.041494977000000002, 2024-09-20T19:01:26.007 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:26.007 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:26.007 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-09-20T19:01:26.007 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:26.007 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.007 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:26.007 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.938366+0000", 2024-09-20T19:01:26.007 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:26.007 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:26.007 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.007 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:26.007 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.938366+0000", 2024-09-20T19:01:26.007 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:26.007 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:26.008 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.008 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:26.008 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.938367+0000", 2024-09-20T19:01:26.008 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.0200000000000001e-07 2024-09-20T19:01:26.008 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:26.008 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.008 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:26.008 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.938373+0000", 2024-09-20T19:01:26.008 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0279999999999997e-06 2024-09-20T19:01:26.008 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:26.008 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.008 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:26.008 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.938384+0000", 2024-09-20T19:01:26.008 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1629e-05 2024-09-20T19:01:26.008 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:26.009 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.009 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:26.009 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.938391+0000", 2024-09-20T19:01:26.009 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.6830000000000002e-06 2024-09-20T19:01:26.009 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:26.009 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.009 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:26.009 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.957361+0000", 2024-09-20T19:01:26.009 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.018969667999999999 2024-09-20T19:01:26.009 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:26.009 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.009 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:26.009 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.957385+0000", 2024-09-20T19:01:26.009 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.4853000000000001e-05 2024-09-20T19:01:26.009 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:26.010 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:26.010 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:26.010 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:26.010 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.010 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:202046 9.c e308/305 9:32a55499:::benchmark_data_smithi083_124552_object87899:head v 308'12553, mlcod=293'9027)", 2024-09-20T19:01:26.010 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:25.944941+0000", 2024-09-20T19:01:26.010 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.034741356000000001, 2024-09-20T19:01:26.010 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.034966799999999999, 2024-09-20T19:01:26.010 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:26.010 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:26.010 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-09-20T19:01:26.010 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:26.010 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.010 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:26.010 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.944941+0000", 2024-09-20T19:01:26.011 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:26.011 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:26.011 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.011 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:26.011 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.944941+0000", 2024-09-20T19:01:26.011 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:26.011 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:26.011 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.011 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:26.011 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.944941+0000", 2024-09-20T19:01:26.011 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4200000000000002e-07 2024-09-20T19:01:26.011 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:26.011 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.011 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:26.011 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.944947+0000", 2024-09-20T19:01:26.012 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.1090000000000003e-06 2024-09-20T19:01:26.012 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:26.012 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.012 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:26.012 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.944953+0000", 2024-09-20T19:01:26.012 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6629999999999998e-06 2024-09-20T19:01:26.012 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:26.012 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.012 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:26.012 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.944958+0000", 2024-09-20T19:01:26.012 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8779999999999997e-06 2024-09-20T19:01:26.012 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:26.012 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.012 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:26.012 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.957385+0000", 2024-09-20T19:01:26.012 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.012427168000000001 2024-09-20T19:01:26.013 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:26.013 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.013 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:26.013 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.957400+0000", 2024-09-20T19:01:26.013 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4839e-05 2024-09-20T19:01:26.013 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:26.013 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:26.013 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:26.013 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:26.013 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.013 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:202049 9.c e308/305 9:30eca193:::benchmark_data_smithi083_124552_object87902:head v 308'12554, mlcod=293'9029)", 2024-09-20T19:01:26.013 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:25.945175+0000", 2024-09-20T19:01:26.013 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.034506587999999998, 2024-09-20T19:01:26.013 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.034779761999999999, 2024-09-20T19:01:26.013 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:26.014 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:26.014 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-09-20T19:01:26.014 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:26.014 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.014 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:26.014 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.945175+0000", 2024-09-20T19:01:26.014 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:26.014 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:26.014 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.014 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:26.014 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.945175+0000", 2024-09-20T19:01:26.014 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:26.014 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:26.014 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.015 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:26.015 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.945176+0000", 2024-09-20T19:01:26.015 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.7399999999999991e-07 2024-09-20T19:01:26.015 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:26.015 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.015 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:26.015 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.945183+0000", 2024-09-20T19:01:26.015 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.3899999999999998e-06 2024-09-20T19:01:26.015 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:26.015 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.015 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:26.015 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.945197+0000", 2024-09-20T19:01:26.015 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4579999999999999e-05 2024-09-20T19:01:26.015 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:26.015 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.015 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:26.016 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.945204+0000", 2024-09-20T19:01:26.016 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.0090000000000001e-06 2024-09-20T19:01:26.016 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:26.016 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.016 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:26.016 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.970024+0000", 2024-09-20T19:01:26.016 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.024819325999999999 2024-09-20T19:01:26.016 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:26.016 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.016 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:26.016 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.970053+0000", 2024-09-20T19:01:26.016 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9111999999999998e-05 2024-09-20T19:01:26.016 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:26.016 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:26.016 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:26.016 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:26.017 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.017 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:202072 9.4 e308/305 9:23d98c62:::benchmark_data_smithi083_124552_object87925:head v 308'12458, mlcod=293'8878)", 2024-09-20T19:01:26.017 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:25.963519+0000", 2024-09-20T19:01:26.017 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.016163311999999999, 2024-09-20T19:01:26.017 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.016480221, 2024-09-20T19:01:26.017 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:26.017 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:26.017 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-09-20T19:01:26.017 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:26.017 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.017 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:26.017 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.963519+0000", 2024-09-20T19:01:26.017 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:26.017 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:26.017 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.017 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:26.018 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.963519+0000", 2024-09-20T19:01:26.018 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:26.018 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:26.018 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.018 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:26.018 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.963520+0000", 2024-09-20T19:01:26.018 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.114e-06 2024-09-20T19:01:26.018 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:26.018 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.018 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:26.018 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.963527+0000", 2024-09-20T19:01:26.018 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.345e-06 2024-09-20T19:01:26.018 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:26.018 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.019 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:26.019 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.963542+0000", 2024-09-20T19:01:26.019 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4969e-05 2024-09-20T19:01:26.019 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:26.019 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.019 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:26.019 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.963549+0000", 2024-09-20T19:01:26.019 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4930000000000003e-06 2024-09-20T19:01:26.019 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:26.019 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.019 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:26.019 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.975309+0000", 2024-09-20T19:01:26.019 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.011760804 2024-09-20T19:01:26.019 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:26.020 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:26.020 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:26.020 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.975334+0000", 2024-09-20T19:01:26.020 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.5071000000000001e-05 2024-09-20T19:01:26.020 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:26.020 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:26.020 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:26.020 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:26.020 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-09-20T19:01:26.020 INFO:teuthology.orchestra.run.smithi142.stdout: "num_ops": 7 2024-09-20T19:01:26.020 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T19:01:26.021 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:01:26.021 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:26.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:26.023 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:26.046 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:01:26.123 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:26.162 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:26.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:26.163 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:26.263 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:26.304 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:26.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:26.305 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:26.406 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:26.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:26.444 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:26.444 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:26.529 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:26 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:01:26.529 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:26 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:01:26.530 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:26 smithi083 ceph-mon[29421]: pgmap v570: 148 pgs: 10 peering, 29 active+undersized, 22 active+undersized+degraded, 87 active+clean; 1.7 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.28k op/s; 3633/85626 objects degraded (4.243%) 2024-09-20T19:01:26.530 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:26 smithi083 ceph-mon[29421]: osdmap e307: 8 total, 8 up, 8 in 2024-09-20T19:01:26.530 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:26 smithi083 ceph-mon[29421]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:01:26.530 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:26 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 3633/85626 objects degraded (4.243%), 22 pgs degraded (PG_DEGRADED) 2024-09-20T19:01:26.530 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:26 smithi083 ceph-mon[29421]: osdmap e308: 8 total, 8 up, 8 in 2024-09-20T19:01:26.530 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:26 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:01:26.530 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:26 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:01:26.530 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:26 smithi083 ceph-mon[36260]: pgmap v570: 148 pgs: 10 peering, 29 active+undersized, 22 active+undersized+degraded, 87 active+clean; 1.7 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.28k op/s; 3633/85626 objects degraded (4.243%) 2024-09-20T19:01:26.530 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:26 smithi083 ceph-mon[36260]: osdmap e307: 8 total, 8 up, 8 in 2024-09-20T19:01:26.530 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:26 smithi083 ceph-mon[36260]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:01:26.531 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:26 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 3633/85626 objects degraded (4.243%), 22 pgs degraded (PG_DEGRADED) 2024-09-20T19:01:26.531 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:26 smithi083 ceph-mon[36260]: osdmap e308: 8 total, 8 up, 8 in 2024-09-20T19:01:26.545 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:26.580 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:26.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:26.581 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:26.682 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:26.723 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:26.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:26.724 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:26.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:26 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:01:26.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:26 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:01:26.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:26 smithi142 ceph-mon[32508]: pgmap v570: 148 pgs: 10 peering, 29 active+undersized, 22 active+undersized+degraded, 87 active+clean; 1.7 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.28k op/s; 3633/85626 objects degraded (4.243%) 2024-09-20T19:01:26.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:26 smithi142 ceph-mon[32508]: osdmap e307: 8 total, 8 up, 8 in 2024-09-20T19:01:26.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:26 smithi142 ceph-mon[32508]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:01:26.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:26 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 3633/85626 objects degraded (4.243%), 22 pgs degraded (PG_DEGRADED) 2024-09-20T19:01:26.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:26 smithi142 ceph-mon[32508]: osdmap e308: 8 total, 8 up, 8 in 2024-09-20T19:01:26.825 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:26.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:26.857 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:26.857 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:26.958 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:27.000 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:27.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:27.001 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd reweight 0 1 2024-09-20T19:01:27.002 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:27.102 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:27.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:27.150 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:27.150 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:27.218 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:01:27 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:01:27.251 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:27.287 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:27.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:27.288 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:27.388 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:27.420 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:01:27.421 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-20T19:01:27.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:27.437 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:27.437 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:27.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:27 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/227621328' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:01:27.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:27 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:01:27.538 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:01:27.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:27.565 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:27.566 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:01:27.667 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:27.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:27.708 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:27.708 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:27.785 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:27 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/227621328' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:01:27.786 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:27 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:01:27.786 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:27 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/227621328' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:01:27.786 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:27 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:01:27.808 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:27.834 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T19:01:27.834 INFO:teuthology.orchestra.run.smithi142.stdout: "size": 20, 2024-09-20T19:01:27.834 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 600, 2024-09-20T19:01:27.834 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-09-20T19:01:27.835 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.835 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.24955.0:101455 8.8 e190/160 8:1ef316d4:::benchmark_data_smithi083_97422_object101454:head v 190'6357, mlcod=190'6355)", 2024-09-20T19:01:27.835 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:55:52.357548+0000", 2024-09-20T19:01:27.835 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 335.20369161399998, 2024-09-20T19:01:27.835 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.15217040600000001, 2024-09-20T19:01:27.836 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:27.836 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:27.836 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:27.836 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:27.836 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.836 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:27.837 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.357548+0000", 2024-09-20T19:01:27.837 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:27.837 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.837 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.837 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:27.837 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.357548+0000", 2024-09-20T19:01:27.837 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:27.838 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.838 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.838 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:27.838 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.357549+0000", 2024-09-20T19:01:27.838 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.0800000000000004e-07 2024-09-20T19:01:27.838 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.838 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.839 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:27.839 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.357594+0000", 2024-09-20T19:01:27.839 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.5052999999999997e-05 2024-09-20T19:01:27.839 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.839 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.839 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:27.840 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.357603+0000", 2024-09-20T19:01:27.840 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.0999999999999993e-06 2024-09-20T19:01:27.840 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.840 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.840 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:27.840 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.357610+0000", 2024-09-20T19:01:27.840 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5459999999999997e-06 2024-09-20T19:01:27.840 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.840 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.840 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:27.841 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.357656+0000", 2024-09-20T19:01:27.841 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.6118999999999997e-05 2024-09-20T19:01:27.841 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.841 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.841 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:27.841 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.357668+0000", 2024-09-20T19:01:27.841 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2367e-05 2024-09-20T19:01:27.841 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.841 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.841 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:27.841 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.360245+0000", 2024-09-20T19:01:27.841 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0025771190000000001 2024-09-20T19:01:27.841 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.841 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.841 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:27.842 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.509719+0000", 2024-09-20T19:01:27.842 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.14947329400000001 2024-09-20T19:01:27.842 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:27.842 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:27.842 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:27.842 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.842 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.842 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.24955.0:101453 8.6 e190/182 8:61900b1b:::benchmark_data_smithi083_97422_object101452:head v 190'6308, mlcod=0'0)", 2024-09-20T19:01:27.842 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:55:52.360511+0000", 2024-09-20T19:01:27.842 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 335.20072855400002, 2024-09-20T19:01:27.842 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.15620442300000001, 2024-09-20T19:01:27.842 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:27.842 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:27.843 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:27.843 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:27.843 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.843 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:27.843 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.360511+0000", 2024-09-20T19:01:27.843 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:27.843 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.843 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.843 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:27.843 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.360511+0000", 2024-09-20T19:01:27.843 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:27.843 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.843 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.843 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:27.844 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.360512+0000", 2024-09-20T19:01:27.844 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.0800000000000003e-07 2024-09-20T19:01:27.844 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.844 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.844 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:27.844 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.360571+0000", 2024-09-20T19:01:27.844 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8850000000000001e-05 2024-09-20T19:01:27.844 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.844 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.844 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:27.844 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.509692+0000", 2024-09-20T19:01:27.844 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.14912044099999999 2024-09-20T19:01:27.844 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.844 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.844 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:27.845 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.509707+0000", 2024-09-20T19:01:27.845 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5568000000000001e-05 2024-09-20T19:01:27.845 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.845 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.845 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:27.845 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.509813+0000", 2024-09-20T19:01:27.845 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00010567400000000001 2024-09-20T19:01:27.845 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.845 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.845 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:27.845 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.509836+0000", 2024-09-20T19:01:27.845 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2853999999999998e-05 2024-09-20T19:01:27.845 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.845 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.846 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:27.846 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.516559+0000", 2024-09-20T19:01:27.846 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.006723807 2024-09-20T19:01:27.846 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.846 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.846 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:27.846 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.516716+0000", 2024-09-20T19:01:27.846 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015632100000000001 2024-09-20T19:01:27.846 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:27.846 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:27.846 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:27.846 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.846 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.846 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.24955.0:101471 8.5 e190/182 8:a0258bcb:::benchmark_data_smithi083_97422_object101470:head v 190'6528, mlcod=0'0)", 2024-09-20T19:01:27.847 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:55:52.379819+0000", 2024-09-20T19:01:27.847 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 335.18142097399999, 2024-09-20T19:01:27.847 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13654260100000001, 2024-09-20T19:01:27.847 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:27.847 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:27.847 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:27.847 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:27.847 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.847 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:27.847 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.379819+0000", 2024-09-20T19:01:27.847 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:27.847 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.847 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.848 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:27.848 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.379819+0000", 2024-09-20T19:01:27.848 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:27.848 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.848 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.848 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:27.848 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.379820+0000", 2024-09-20T19:01:27.848 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5519999999999999e-06 2024-09-20T19:01:27.848 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.848 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.848 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:27.848 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.379854+0000", 2024-09-20T19:01:27.848 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.3157000000000001e-05 2024-09-20T19:01:27.848 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.848 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.848 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:27.849 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.509694+0000", 2024-09-20T19:01:27.849 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.12983992999999999 2024-09-20T19:01:27.849 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.849 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.849 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:27.849 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.509706+0000", 2024-09-20T19:01:27.849 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2965e-05 2024-09-20T19:01:27.849 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.849 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.849 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:27.849 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.509768+0000", 2024-09-20T19:01:27.849 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.1013e-05 2024-09-20T19:01:27.849 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.849 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.849 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:27.849 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.509797+0000", 2024-09-20T19:01:27.850 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9916999999999998e-05 2024-09-20T19:01:27.850 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.850 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.850 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:27.850 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.516311+0000", 2024-09-20T19:01:27.850 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0065133980000000001 2024-09-20T19:01:27.850 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.850 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.850 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:27.850 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.516361+0000", 2024-09-20T19:01:27.850 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0668999999999999e-05 2024-09-20T19:01:27.850 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:27.850 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:27.850 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:27.851 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.851 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.851 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:72379 9.d 9:bae8f567:::benchmark_data_smithi083_124552_object72378:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e265)", 2024-09-20T19:01:27.851 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:26.629075+0000", 2024-09-20T19:01:27.851 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 120.932164864, 2024-09-20T19:01:27.851 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.133017778, 2024-09-20T19:01:27.851 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:27.851 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:27.851 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:27.851 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:01:27.851 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:01:27.851 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:01:27.851 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 72379 2024-09-20T19:01:27.851 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.852 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:27.852 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.852 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:27.852 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.629075+0000", 2024-09-20T19:01:27.852 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:27.852 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.852 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.852 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:27.852 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.629075+0000", 2024-09-20T19:01:27.852 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:27.852 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.852 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.852 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:27.852 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.629077+0000", 2024-09-20T19:01:27.853 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.753e-06 2024-09-20T19:01:27.853 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.853 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.853 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:27.853 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.629117+0000", 2024-09-20T19:01:27.853 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0463000000000002e-05 2024-09-20T19:01:27.853 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.853 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.853 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:27.853 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.629128+0000", 2024-09-20T19:01:27.853 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1252e-05 2024-09-20T19:01:27.853 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.853 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.853 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:27.853 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.629136+0000", 2024-09-20T19:01:27.854 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5850000000000002e-06 2024-09-20T19:01:27.854 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.854 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.854 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:27.854 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.629297+0000", 2024-09-20T19:01:27.854 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00016093100000000001 2024-09-20T19:01:27.854 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.854 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.854 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:27.854 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.629447+0000", 2024-09-20T19:01:27.854 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000149717 2024-09-20T19:01:27.854 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.854 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.854 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,5", 2024-09-20T19:01:27.855 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.629595+0000", 2024-09-20T19:01:27.855 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000148436 2024-09-20T19:01:27.855 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.855 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.855 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:27.855 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.638400+0000", 2024-09-20T19:01:27.855 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0088051139999999993 2024-09-20T19:01:27.855 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.855 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.855 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:01:27.855 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.638412+0000", 2024-09-20T19:01:27.855 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1804000000000001e-05 2024-09-20T19:01:27.855 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.856 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.856 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:27.856 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.761963+0000", 2024-09-20T19:01:27.856 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.123550901 2024-09-20T19:01:27.856 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.856 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.856 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:27.856 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.762026+0000", 2024-09-20T19:01:27.856 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.3350999999999997e-05 2024-09-20T19:01:27.856 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.856 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.856 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:27.856 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.762093+0000", 2024-09-20T19:01:27.856 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.6470999999999994e-05 2024-09-20T19:01:27.857 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:27.857 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:27.857 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:27.857 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.857 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.857 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:72382 9.d 9:b4fdabfe:::benchmark_data_smithi083_124552_object72381:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e265)", 2024-09-20T19:01:27.857 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:26.629907+0000", 2024-09-20T19:01:27.857 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 120.931333298, 2024-09-20T19:01:27.857 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13337285700000001, 2024-09-20T19:01:27.857 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:27.857 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:27.857 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:27.857 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:01:27.857 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:01:27.858 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:01:27.858 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 72382 2024-09-20T19:01:27.858 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.858 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:27.858 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.858 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:27.858 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.629907+0000", 2024-09-20T19:01:27.858 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:27.858 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.858 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.858 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:27.858 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.629907+0000", 2024-09-20T19:01:27.858 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:27.858 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.859 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.859 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:27.859 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.629908+0000", 2024-09-20T19:01:27.859 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.206e-06 2024-09-20T19:01:27.859 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.859 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.859 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:27.859 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.629934+0000", 2024-09-20T19:01:27.859 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.6463000000000001e-05 2024-09-20T19:01:27.859 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.859 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.859 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:27.859 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.629946+0000", 2024-09-20T19:01:27.859 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1579e-05 2024-09-20T19:01:27.859 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.860 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.860 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:27.860 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.629953+0000", 2024-09-20T19:01:27.860 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.1810000000000004e-06 2024-09-20T19:01:27.860 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.860 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.860 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:27.860 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.632103+0000", 2024-09-20T19:01:27.860 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.002150365 2024-09-20T19:01:27.860 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.860 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.860 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:27.860 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.632219+0000", 2024-09-20T19:01:27.860 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000115746 2024-09-20T19:01:27.861 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.861 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.861 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,5", 2024-09-20T19:01:27.861 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.632337+0000", 2024-09-20T19:01:27.861 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011818499999999999 2024-09-20T19:01:27.861 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.861 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.861 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:27.861 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.638442+0000", 2024-09-20T19:01:27.861 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0061042379999999997 2024-09-20T19:01:27.861 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.861 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.861 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:01:27.861 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.638466+0000", 2024-09-20T19:01:27.862 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.4266000000000001e-05 2024-09-20T19:01:27.862 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.862 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.862 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:27.862 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.763189+0000", 2024-09-20T19:01:27.862 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.124723375 2024-09-20T19:01:27.862 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.862 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.862 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:27.862 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.763240+0000", 2024-09-20T19:01:27.862 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0708999999999998e-05 2024-09-20T19:01:27.862 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.862 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.862 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:27.863 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.763279+0000", 2024-09-20T19:01:27.863 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9544e-05 2024-09-20T19:01:27.863 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:27.863 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:27.863 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:27.863 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.863 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.863 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:72386 9.3 9:c310ff33:::benchmark_data_smithi083_124552_object72385:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e265)", 2024-09-20T19:01:27.863 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:26.634291+0000", 2024-09-20T19:01:27.863 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 120.926948411, 2024-09-20T19:01:27.863 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.133404362, 2024-09-20T19:01:27.863 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:27.863 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:27.863 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:27.864 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:01:27.864 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:01:27.864 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:01:27.864 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 72386 2024-09-20T19:01:27.864 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.864 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:27.864 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.864 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:27.864 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.634291+0000", 2024-09-20T19:01:27.864 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:27.864 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.864 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.864 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:27.864 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.634291+0000", 2024-09-20T19:01:27.865 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:27.865 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.865 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.865 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:27.865 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.634293+0000", 2024-09-20T19:01:27.865 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4810000000000001e-06 2024-09-20T19:01:27.865 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.865 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.865 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:27.865 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.634352+0000", 2024-09-20T19:01:27.865 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9089000000000003e-05 2024-09-20T19:01:27.865 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.865 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.865 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:27.865 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.634363+0000", 2024-09-20T19:01:27.866 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1100999999999999e-05 2024-09-20T19:01:27.866 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.866 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.866 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:27.866 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.634371+0000", 2024-09-20T19:01:27.866 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.0490000000000004e-06 2024-09-20T19:01:27.866 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.866 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.866 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:27.866 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.634434+0000", 2024-09-20T19:01:27.866 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2587000000000002e-05 2024-09-20T19:01:27.866 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.866 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.867 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:27.867 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.634569+0000", 2024-09-20T19:01:27.867 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013524699999999999 2024-09-20T19:01:27.867 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.867 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.867 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,3", 2024-09-20T19:01:27.867 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.634714+0000", 2024-09-20T19:01:27.867 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00014523699999999999 2024-09-20T19:01:27.867 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.867 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.867 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:01:27.867 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.639963+0000", 2024-09-20T19:01:27.867 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0052484699999999999 2024-09-20T19:01:27.867 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.868 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.868 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:27.868 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.640003+0000", 2024-09-20T19:01:27.868 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0185999999999998e-05 2024-09-20T19:01:27.868 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.868 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.868 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:27.868 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.767600+0000", 2024-09-20T19:01:27.868 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.127597345 2024-09-20T19:01:27.868 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.868 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.868 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:27.868 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.767655+0000", 2024-09-20T19:01:27.868 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5003999999999997e-05 2024-09-20T19:01:27.869 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.869 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.869 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:27.869 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.767696+0000", 2024-09-20T19:01:27.869 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0565999999999998e-05 2024-09-20T19:01:27.869 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:27.869 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:27.869 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:27.869 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.869 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.869 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:72388 9.3 9:c69fcb53:::benchmark_data_smithi083_124552_object72387:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e265)", 2024-09-20T19:01:27.869 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:26.635542+0000", 2024-09-20T19:01:27.870 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 120.92569828800001, 2024-09-20T19:01:27.870 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.132329168, 2024-09-20T19:01:27.870 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:27.870 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:27.870 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:27.870 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:01:27.870 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:01:27.870 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:01:27.870 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 72388 2024-09-20T19:01:27.870 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.871 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:27.871 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.871 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:27.871 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.635542+0000", 2024-09-20T19:01:27.871 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:27.871 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.871 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.871 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:27.872 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.635542+0000", 2024-09-20T19:01:27.872 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:27.872 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.872 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.872 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:27.872 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.635542+0000", 2024-09-20T19:01:27.872 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.7899999999999998e-07 2024-09-20T19:01:27.873 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.873 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.873 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:27.873 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.635617+0000", 2024-09-20T19:01:27.873 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.4859999999999998e-05 2024-09-20T19:01:27.873 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.874 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.874 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:27.874 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.635625+0000", 2024-09-20T19:01:27.874 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5979999999999999e-06 2024-09-20T19:01:27.874 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.874 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.875 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:27.875 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.635631+0000", 2024-09-20T19:01:27.875 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8599999999999998e-06 2024-09-20T19:01:27.875 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.875 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.875 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:27.876 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.637372+0000", 2024-09-20T19:01:27.876 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001741352 2024-09-20T19:01:27.876 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.876 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.876 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:27.876 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.637487+0000", 2024-09-20T19:01:27.877 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000115296 2024-09-20T19:01:27.877 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.877 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.877 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,3", 2024-09-20T19:01:27.877 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.637613+0000", 2024-09-20T19:01:27.877 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012625200000000001 2024-09-20T19:01:27.877 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.878 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.878 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:01:27.878 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.640092+0000", 2024-09-20T19:01:27.878 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.002478975 2024-09-20T19:01:27.878 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.879 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.879 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:27.879 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.640178+0000", 2024-09-20T19:01:27.879 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.5749999999999997e-05 2024-09-20T19:01:27.879 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.879 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.880 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:27.880 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.767762+0000", 2024-09-20T19:01:27.880 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.127583327 2024-09-20T19:01:27.880 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.880 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.880 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:27.881 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.767818+0000", 2024-09-20T19:01:27.881 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6589999999999999e-05 2024-09-20T19:01:27.881 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.881 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.881 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:27.881 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.767871+0000", 2024-09-20T19:01:27.882 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.2629e-05 2024-09-20T19:01:27.882 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:27.882 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:27.882 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:27.882 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.882 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.883 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:84060 9.e 9:73c1c2b3:::benchmark_data_smithi083_124552_object84059:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e268)", 2024-09-20T19:01:27.883 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:38.588091+0000", 2024-09-20T19:01:27.883 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 108.973149171, 2024-09-20T19:01:27.883 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13688319800000001, 2024-09-20T19:01:27.883 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:27.883 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:27.884 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:27.884 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:01:27.884 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:01:27.884 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:01:27.884 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 84060 2024-09-20T19:01:27.884 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.885 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:27.885 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.885 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:27.885 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.588091+0000", 2024-09-20T19:01:27.885 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:27.885 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.885 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.886 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:27.886 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.588091+0000", 2024-09-20T19:01:27.886 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:27.886 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.886 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.886 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:27.887 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.588092+0000", 2024-09-20T19:01:27.887 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.694e-06 2024-09-20T19:01:27.887 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.887 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.887 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:27.887 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.588232+0000", 2024-09-20T19:01:27.887 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013927899999999999 2024-09-20T19:01:27.888 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.888 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.888 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:27.888 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.588247+0000", 2024-09-20T19:01:27.888 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5670000000000001e-05 2024-09-20T19:01:27.888 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.889 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.889 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:27.889 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.588255+0000", 2024-09-20T19:01:27.889 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.0630000000000006e-06 2024-09-20T19:01:27.889 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.889 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.889 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:27.890 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.588320+0000", 2024-09-20T19:01:27.890 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5049999999999996e-05 2024-09-20T19:01:27.890 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.890 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.890 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:27.890 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.588440+0000", 2024-09-20T19:01:27.891 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011967 2024-09-20T19:01:27.891 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.891 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.891 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,4", 2024-09-20T19:01:27.891 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.588596+0000", 2024-09-20T19:01:27.891 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015572200000000001 2024-09-20T19:01:27.892 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.892 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.892 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:01:27.892 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.591386+0000", 2024-09-20T19:01:27.892 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.002789962 2024-09-20T19:01:27.892 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.893 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.893 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:27.893 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.591731+0000", 2024-09-20T19:01:27.893 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00034524700000000002 2024-09-20T19:01:27.893 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.893 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.893 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:27.894 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.724872+0000", 2024-09-20T19:01:27.894 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13314100300000001 2024-09-20T19:01:27.894 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.894 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.894 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:27.894 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.724930+0000", 2024-09-20T19:01:27.895 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.7472000000000001e-05 2024-09-20T19:01:27.895 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.895 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.895 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:27.895 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.724974+0000", 2024-09-20T19:01:27.895 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.4366000000000002e-05 2024-09-20T19:01:27.895 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:27.895 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:27.896 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:27.896 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.896 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.896 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:84067 9.3 9:cfb01319:::benchmark_data_smithi083_124552_object84066:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e268)", 2024-09-20T19:01:27.896 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:38.595983+0000", 2024-09-20T19:01:27.896 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 108.96525654, 2024-09-20T19:01:27.896 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.12985870299999999, 2024-09-20T19:01:27.897 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:27.897 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:27.897 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:27.897 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:01:27.897 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:01:27.897 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:01:27.897 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 84067 2024-09-20T19:01:27.897 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.897 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:27.897 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.897 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:27.897 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.595983+0000", 2024-09-20T19:01:27.897 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:27.898 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.898 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.898 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:27.898 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.595983+0000", 2024-09-20T19:01:27.898 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:27.898 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.898 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.898 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:27.898 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.595985+0000", 2024-09-20T19:01:27.898 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9800000000000001e-06 2024-09-20T19:01:27.898 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.898 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.899 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:27.899 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.596024+0000", 2024-09-20T19:01:27.899 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.8640000000000003e-05 2024-09-20T19:01:27.899 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.899 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.899 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:27.899 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.596045+0000", 2024-09-20T19:01:27.899 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0707e-05 2024-09-20T19:01:27.899 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.899 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.899 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:27.899 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.596053+0000", 2024-09-20T19:01:27.899 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.3319999999999992e-06 2024-09-20T19:01:27.899 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.899 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.900 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:27.900 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.596111+0000", 2024-09-20T19:01:27.900 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.7803999999999997e-05 2024-09-20T19:01:27.900 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.900 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.900 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:27.900 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.596263+0000", 2024-09-20T19:01:27.900 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000151784 2024-09-20T19:01:27.900 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.900 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.900 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,3", 2024-09-20T19:01:27.900 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.596417+0000", 2024-09-20T19:01:27.900 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015399399999999999 2024-09-20T19:01:27.900 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.900 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.901 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:01:27.901 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.599163+0000", 2024-09-20T19:01:27.901 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.002746727 2024-09-20T19:01:27.901 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.901 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.901 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:27.901 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.722887+0000", 2024-09-20T19:01:27.901 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.12372408999999999 2024-09-20T19:01:27.901 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.901 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.901 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:27.901 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.725618+0000", 2024-09-20T19:01:27.901 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0027303980000000002 2024-09-20T19:01:27.901 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.901 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.902 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:27.902 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.725797+0000", 2024-09-20T19:01:27.902 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00017908400000000001 2024-09-20T19:01:27.902 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.902 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.902 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:27.902 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.725842+0000", 2024-09-20T19:01:27.902 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.5163000000000001e-05 2024-09-20T19:01:27.902 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:27.902 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:27.902 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:27.902 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.902 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.903 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:101846 9.d 9:bb37c834:::benchmark_data_smithi083_124552_object101845:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e275)", 2024-09-20T19:01:27.903 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:57.497943+0000", 2024-09-20T19:01:27.903 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 90.063296690000001, 2024-09-20T19:01:27.903 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.22483731800000001, 2024-09-20T19:01:27.903 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:27.903 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:27.903 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:27.903 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:01:27.903 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:01:27.903 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:01:27.903 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 101846 2024-09-20T19:01:27.903 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.903 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:27.904 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.904 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:27.904 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.497943+0000", 2024-09-20T19:01:27.904 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:27.904 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.904 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.904 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:27.904 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.497943+0000", 2024-09-20T19:01:27.904 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:27.904 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.904 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.904 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:27.904 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.497945+0000", 2024-09-20T19:01:27.904 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6029999999999999e-06 2024-09-20T19:01:27.904 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.905 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.905 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:27.905 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.497989+0000", 2024-09-20T19:01:27.905 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3755999999999999e-05 2024-09-20T19:01:27.905 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.905 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.905 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:27.905 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.498001+0000", 2024-09-20T19:01:27.905 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2615e-05 2024-09-20T19:01:27.905 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.905 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.905 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:27.905 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.498010+0000", 2024-09-20T19:01:27.905 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.3599999999999996e-06 2024-09-20T19:01:27.905 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.906 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.906 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:27.906 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.498066+0000", 2024-09-20T19:01:27.906 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6923999999999999e-05 2024-09-20T19:01:27.906 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.906 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.906 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:27.906 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.498167+0000", 2024-09-20T19:01:27.906 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000100721 2024-09-20T19:01:27.906 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.906 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.906 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,5", 2024-09-20T19:01:27.906 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.498268+0000", 2024-09-20T19:01:27.907 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000100488 2024-09-20T19:01:27.907 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.907 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.907 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:01:27.907 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.500395+0000", 2024-09-20T19:01:27.907 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0021269259999999999 2024-09-20T19:01:27.907 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.907 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.907 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:27.907 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.500711+0000", 2024-09-20T19:01:27.907 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00031641299999999998 2024-09-20T19:01:27.907 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.907 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.907 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:27.908 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.722681+0000", 2024-09-20T19:01:27.908 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.22197028999999999 2024-09-20T19:01:27.908 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.908 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.908 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:27.908 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.722735+0000", 2024-09-20T19:01:27.908 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3879999999999999e-05 2024-09-20T19:01:27.908 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.908 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.908 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:27.908 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.722780+0000", 2024-09-20T19:01:27.908 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.5342000000000001e-05 2024-09-20T19:01:27.908 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:27.908 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:27.909 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:27.909 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.909 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.909 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:119689 9.d 9:bd867cd9:::benchmark_data_smithi083_124552_object5542:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e287)", 2024-09-20T19:01:27.909 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:00:16.841487+0000", 2024-09-20T19:01:27.909 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 70.719752451000005, 2024-09-20T19:01:27.909 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13893808099999999, 2024-09-20T19:01:27.909 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:27.909 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:27.909 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:27.909 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:01:27.909 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:01:27.909 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:01:27.909 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 119689 2024-09-20T19:01:27.909 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.910 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:27.910 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.910 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:27.910 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841487+0000", 2024-09-20T19:01:27.910 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:27.910 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.910 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.910 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:27.910 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841487+0000", 2024-09-20T19:01:27.910 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:27.910 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.910 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.910 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:27.910 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841489+0000", 2024-09-20T19:01:27.911 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2309999999999999e-06 2024-09-20T19:01:27.911 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.911 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.911 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:27.911 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841493+0000", 2024-09-20T19:01:27.911 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.374e-06 2024-09-20T19:01:27.911 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.911 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.911 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:27.911 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841502+0000", 2024-09-20T19:01:27.911 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.3649999999999993e-06 2024-09-20T19:01:27.911 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.912 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.912 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:27.912 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841509+0000", 2024-09-20T19:01:27.912 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.0999999999999998e-06 2024-09-20T19:01:27.912 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.912 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.912 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:27.912 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841561+0000", 2024-09-20T19:01:27.912 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.1968000000000001e-05 2024-09-20T19:01:27.912 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.912 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.912 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:27.912 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841684+0000", 2024-09-20T19:01:27.912 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012302 2024-09-20T19:01:27.913 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.913 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.913 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,5", 2024-09-20T19:01:27.913 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841798+0000", 2024-09-20T19:01:27.913 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011330799999999999 2024-09-20T19:01:27.913 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.913 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.913 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:27.913 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.845340+0000", 2024-09-20T19:01:27.913 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0035421129999999999 2024-09-20T19:01:27.913 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.913 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.913 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:01:27.914 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.849468+0000", 2024-09-20T19:01:27.914 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0041283489999999999 2024-09-20T19:01:27.914 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.914 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.914 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:27.914 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.980284+0000", 2024-09-20T19:01:27.914 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13081616800000001 2024-09-20T19:01:27.914 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.914 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.914 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:27.915 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.980351+0000", 2024-09-20T19:01:27.915 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.6928999999999996e-05 2024-09-20T19:01:27.915 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.915 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.915 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:27.915 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.980425+0000", 2024-09-20T19:01:27.915 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.4155999999999999e-05 2024-09-20T19:01:27.915 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:27.915 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:27.915 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:27.915 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.915 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.915 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:119690 9.3 9:cd5c9a6c:::benchmark_data_smithi083_124552_object5543:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e287)", 2024-09-20T19:01:27.916 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:00:16.841539+0000", 2024-09-20T19:01:27.916 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 70.719701240999996, 2024-09-20T19:01:27.916 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.138853324, 2024-09-20T19:01:27.916 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:27.916 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:27.916 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:27.916 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:01:27.916 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:01:27.916 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:01:27.916 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 119690 2024-09-20T19:01:27.916 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.916 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:27.916 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.916 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:27.917 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841539+0000", 2024-09-20T19:01:27.917 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:27.917 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.917 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.917 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:27.917 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841539+0000", 2024-09-20T19:01:27.917 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:27.917 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.917 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.917 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:27.917 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841539+0000", 2024-09-20T19:01:27.917 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4499999999999997e-07 2024-09-20T19:01:27.917 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.917 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.918 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:27.918 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841542+0000", 2024-09-20T19:01:27.918 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.128e-06 2024-09-20T19:01:27.918 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.918 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.918 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:27.918 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841548+0000", 2024-09-20T19:01:27.918 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8220000000000003e-06 2024-09-20T19:01:27.918 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.918 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.918 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:27.918 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841553+0000", 2024-09-20T19:01:27.918 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.967e-06 2024-09-20T19:01:27.919 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.919 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.919 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:27.919 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841603+0000", 2024-09-20T19:01:27.919 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.9636999999999999e-05 2024-09-20T19:01:27.919 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.919 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.919 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:27.919 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841706+0000", 2024-09-20T19:01:27.919 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000103609 2024-09-20T19:01:27.919 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.919 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.919 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,3", 2024-09-20T19:01:27.919 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841803+0000", 2024-09-20T19:01:27.919 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.6217000000000004e-05 2024-09-20T19:01:27.920 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.920 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.920 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:27.920 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.845333+0000", 2024-09-20T19:01:27.920 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0035301540000000002 2024-09-20T19:01:27.920 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.920 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.920 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:01:27.920 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.846476+0000", 2024-09-20T19:01:27.920 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0011436619999999999 2024-09-20T19:01:27.920 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.920 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.920 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:27.920 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.980293+0000", 2024-09-20T19:01:27.921 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13381654500000001 2024-09-20T19:01:27.921 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.921 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.921 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:27.921 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.980328+0000", 2024-09-20T19:01:27.921 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.5255e-05 2024-09-20T19:01:27.921 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.921 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.921 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:27.921 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.980392+0000", 2024-09-20T19:01:27.921 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.3782999999999996e-05 2024-09-20T19:01:27.921 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:27.921 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:27.921 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:27.921 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.922 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.922 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:119697 9.d 9:b589807c:::benchmark_data_smithi083_124552_object5550:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e287)", 2024-09-20T19:01:27.922 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:00:16.846849+0000", 2024-09-20T19:01:27.922 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 70.714391284000001, 2024-09-20T19:01:27.922 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13367747499999999, 2024-09-20T19:01:27.922 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:27.922 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:27.922 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:27.922 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:01:27.922 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:01:27.922 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:01:27.922 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 119697 2024-09-20T19:01:27.922 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.923 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:27.923 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.923 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:27.923 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.846849+0000", 2024-09-20T19:01:27.923 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:27.923 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.923 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.923 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:27.923 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.846849+0000", 2024-09-20T19:01:27.923 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:27.923 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.923 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.923 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:27.924 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.846850+0000", 2024-09-20T19:01:27.924 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5829999999999999e-06 2024-09-20T19:01:27.924 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.924 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.924 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:27.924 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.846856+0000", 2024-09-20T19:01:27.924 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6570000000000002e-06 2024-09-20T19:01:27.924 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.924 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.924 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:27.924 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.846867+0000", 2024-09-20T19:01:27.924 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1321999999999999e-05 2024-09-20T19:01:27.924 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.924 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.924 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:27.925 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.846875+0000", 2024-09-20T19:01:27.925 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.9270000000000005e-06 2024-09-20T19:01:27.925 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.925 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.925 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:27.925 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.846926+0000", 2024-09-20T19:01:27.925 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0759e-05 2024-09-20T19:01:27.925 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.925 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.925 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:27.925 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.847050+0000", 2024-09-20T19:01:27.925 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012434300000000001 2024-09-20T19:01:27.925 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.925 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.925 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,5", 2024-09-20T19:01:27.925 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.847172+0000", 2024-09-20T19:01:27.926 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000121586 2024-09-20T19:01:27.926 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.926 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.926 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:01:27.926 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.849552+0000", 2024-09-20T19:01:27.926 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0023799810000000002 2024-09-20T19:01:27.926 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.926 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.926 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:27.926 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.849792+0000", 2024-09-20T19:01:27.926 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000239965 2024-09-20T19:01:27.926 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.926 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.926 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:27.927 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.980470+0000", 2024-09-20T19:01:27.927 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13067801600000001 2024-09-20T19:01:27.927 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.927 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.927 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:27.927 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.980505+0000", 2024-09-20T19:01:27.927 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.5627000000000003e-05 2024-09-20T19:01:27.927 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.927 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.927 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:27.927 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.980526+0000", 2024-09-20T19:01:27.927 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0709000000000001e-05 2024-09-20T19:01:27.927 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:27.927 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:27.927 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:27.927 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.928 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.928 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14460.0:2702 5.9 5:93e5b521:::notify.7:head [watch reconnect cookie 94056120174720 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e292)", 2024-09-20T19:01:27.928 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:00:35.178267+0000", 2024-09-20T19:01:27.928 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 52.382972674000001, 2024-09-20T19:01:27.928 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.011898462, 2024-09-20T19:01:27.928 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:27.928 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:27.928 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-09-20T19:01:27.928 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:01:27.928 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14460", 2024-09-20T19:01:27.928 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1555339426", 2024-09-20T19:01:27.928 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 2702 2024-09-20T19:01:27.928 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.928 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:27.929 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.929 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:27.929 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.178267+0000", 2024-09-20T19:01:27.929 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:27.929 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.929 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.929 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:27.929 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.178267+0000", 2024-09-20T19:01:27.929 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:27.929 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.929 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.929 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:27.929 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.178269+0000", 2024-09-20T19:01:27.929 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.703e-06 2024-09-20T19:01:27.930 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.930 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.930 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:27.930 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.178274+0000", 2024-09-20T19:01:27.930 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3909999999999997e-06 2024-09-20T19:01:27.930 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.930 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.930 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:27.930 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.178292+0000", 2024-09-20T19:01:27.930 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7601e-05 2024-09-20T19:01:27.930 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.930 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.930 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:27.930 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.178301+0000", 2024-09-20T19:01:27.931 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.6950000000000006e-06 2024-09-20T19:01:27.931 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.931 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.931 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:27.931 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.178364+0000", 2024-09-20T19:01:27.931 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.3304000000000002e-05 2024-09-20T19:01:27.931 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.931 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.931 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op must wait for map", 2024-09-20T19:01:27.931 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.178370+0000", 2024-09-20T19:01:27.931 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2979999999999997e-06 2024-09-20T19:01:27.931 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.931 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.931 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:27.932 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.191158+0000", 2024-09-20T19:01:27.932 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.012787822000000001 2024-09-20T19:01:27.932 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.932 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.932 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:01:27.932 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.191166+0000", 2024-09-20T19:01:27.932 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.9959999999999993e-06 2024-09-20T19:01:27.932 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.932 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.932 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:27.932 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.188716+0000", 2024-09-20T19:01:27.932 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.997550198 2024-09-20T19:01:27.932 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.932 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.933 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:27.933 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.188848+0000", 2024-09-20T19:01:27.933 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013202400000000001 2024-09-20T19:01:27.933 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.933 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.933 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:27.933 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.190166+0000", 2024-09-20T19:01:27.933 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0013174300000000001 2024-09-20T19:01:27.933 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:27.933 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:27.933 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:27.933 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.933 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.933 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:142953 9.e 9:785c93fe:::benchmark_data_smithi083_124552_object28806:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e292)", 2024-09-20T19:01:27.934 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:00:35.227055+0000", 2024-09-20T19:01:27.934 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 52.334184993000001, 2024-09-20T19:01:27.934 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.985725662, 2024-09-20T19:01:27.934 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:27.934 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:27.934 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:27.934 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:01:27.934 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:01:27.934 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:01:27.934 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 142953 2024-09-20T19:01:27.934 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.934 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:27.934 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.935 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:27.935 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.227055+0000", 2024-09-20T19:01:27.935 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:27.935 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.935 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.935 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:27.935 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.227055+0000", 2024-09-20T19:01:27.935 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:27.935 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.935 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.935 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:27.935 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.227056+0000", 2024-09-20T19:01:27.935 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1170000000000001e-06 2024-09-20T19:01:27.936 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.936 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.936 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:27.936 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.227060+0000", 2024-09-20T19:01:27.936 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.1320000000000004e-06 2024-09-20T19:01:27.936 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.936 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.936 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:27.936 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.227070+0000", 2024-09-20T19:01:27.936 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.8179999999999999e-06 2024-09-20T19:01:27.936 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.936 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.936 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:27.936 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.227075+0000", 2024-09-20T19:01:27.937 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5289999999999999e-06 2024-09-20T19:01:27.937 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.937 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.937 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:27.937 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.227131+0000", 2024-09-20T19:01:27.937 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5482999999999997e-05 2024-09-20T19:01:27.937 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.937 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.937 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:01:27.937 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.227140+0000", 2024-09-20T19:01:27.937 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.4159999999999993e-06 2024-09-20T19:01:27.937 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.937 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.937 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:27.937 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.205610+0000", 2024-09-20T19:01:27.938 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.97846971500000002 2024-09-20T19:01:27.938 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.938 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.938 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:27.938 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.205744+0000", 2024-09-20T19:01:27.938 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013422099999999999 2024-09-20T19:01:27.938 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.938 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.938 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1", 2024-09-20T19:01:27.938 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.205878+0000", 2024-09-20T19:01:27.938 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000134108 2024-09-20T19:01:27.938 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.938 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.938 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:27.939 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.212569+0000", 2024-09-20T19:01:27.939 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0066901629999999998 2024-09-20T19:01:27.939 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.939 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.939 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:01:27.939 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.212643+0000", 2024-09-20T19:01:27.939 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.4120000000000002e-05 2024-09-20T19:01:27.939 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.939 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.939 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:27.939 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.212692+0000", 2024-09-20T19:01:27.939 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.922e-05 2024-09-20T19:01:27.939 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.939 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.940 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:27.940 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.212781+0000", 2024-09-20T19:01:27.940 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.8620000000000002e-05 2024-09-20T19:01:27.940 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:27.940 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:27.940 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:27.940 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.940 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.940 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:142961 9.e 9:70c9c31e:::benchmark_data_smithi083_124552_object28814:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e292)", 2024-09-20T19:01:27.940 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:00:35.233023+0000", 2024-09-20T19:01:27.940 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 52.328216558000001, 2024-09-20T19:01:27.940 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.98462836899999995, 2024-09-20T19:01:27.940 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:27.940 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:27.940 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:27.940 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:01:27.941 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:01:27.941 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:01:27.941 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 142961 2024-09-20T19:01:27.941 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.941 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:27.941 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.941 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:27.941 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.233023+0000", 2024-09-20T19:01:27.941 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:27.941 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.941 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.941 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:27.941 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.233023+0000", 2024-09-20T19:01:27.941 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:27.941 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.941 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.942 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:27.942 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.233025+0000", 2024-09-20T19:01:27.942 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.252e-06 2024-09-20T19:01:27.942 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.942 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.942 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:27.942 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.233029+0000", 2024-09-20T19:01:27.942 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.442e-06 2024-09-20T19:01:27.942 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.942 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.942 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:27.942 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.233048+0000", 2024-09-20T19:01:27.942 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9096000000000001e-05 2024-09-20T19:01:27.942 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.942 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.943 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:27.943 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.233055+0000", 2024-09-20T19:01:27.943 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.0729999999999999e-06 2024-09-20T19:01:27.943 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.943 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.943 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:27.943 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.233109+0000", 2024-09-20T19:01:27.943 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4049999999999999e-05 2024-09-20T19:01:27.943 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.943 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.943 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:01:27.943 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.233117+0000", 2024-09-20T19:01:27.943 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.379e-06 2024-09-20T19:01:27.944 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.944 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.944 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:27.944 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.208136+0000", 2024-09-20T19:01:27.944 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.97501899599999997 2024-09-20T19:01:27.944 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.944 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.944 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:27.944 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.208257+0000", 2024-09-20T19:01:27.944 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012119 2024-09-20T19:01:27.944 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.944 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.944 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1", 2024-09-20T19:01:27.944 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.208383+0000", 2024-09-20T19:01:27.944 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000126484 2024-09-20T19:01:27.945 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.945 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.945 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:27.945 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.212629+0000", 2024-09-20T19:01:27.945 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0042453329999999996 2024-09-20T19:01:27.945 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.945 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.945 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:01:27.945 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.217490+0000", 2024-09-20T19:01:27.945 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0048616470000000002 2024-09-20T19:01:27.945 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.945 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.945 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:27.945 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.217551+0000", 2024-09-20T19:01:27.945 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0797000000000001e-05 2024-09-20T19:01:27.946 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.946 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.946 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:27.946 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.217652+0000", 2024-09-20T19:01:27.946 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00010063 2024-09-20T19:01:27.946 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:27.946 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:27.946 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:27.946 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.946 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.946 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:142969 9.e 9:74a6f6aa:::benchmark_data_smithi083_124552_object28822:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e292)", 2024-09-20T19:01:27.946 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:00:35.238908+0000", 2024-09-20T19:01:27.946 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 52.322331935000001, 2024-09-20T19:01:27.947 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.98019590400000001, 2024-09-20T19:01:27.947 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:27.947 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:27.947 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:27.947 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:01:27.947 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:01:27.947 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:01:27.947 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 142969 2024-09-20T19:01:27.947 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.947 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:27.947 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.947 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:27.947 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.238908+0000", 2024-09-20T19:01:27.947 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:27.948 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.948 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.948 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:27.948 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.238908+0000", 2024-09-20T19:01:27.948 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:27.948 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.948 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.948 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:27.948 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.238909+0000", 2024-09-20T19:01:27.948 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.7800000000000001e-07 2024-09-20T19:01:27.948 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.948 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.948 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:27.948 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.238912+0000", 2024-09-20T19:01:27.949 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.3919999999999999e-06 2024-09-20T19:01:27.949 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.949 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.949 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:27.949 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.238919+0000", 2024-09-20T19:01:27.949 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.0679999999999999e-06 2024-09-20T19:01:27.949 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.949 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.949 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:27.949 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.238925+0000", 2024-09-20T19:01:27.949 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.1369999999999999e-06 2024-09-20T19:01:27.949 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.949 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.949 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:27.950 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.238981+0000", 2024-09-20T19:01:27.950 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6067999999999999e-05 2024-09-20T19:01:27.950 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.950 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.950 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:01:27.950 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.238990+0000", 2024-09-20T19:01:27.950 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.2600000000000005e-06 2024-09-20T19:01:27.950 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.950 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.950 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:27.950 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.210313+0000", 2024-09-20T19:01:27.950 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.97132380500000004 2024-09-20T19:01:27.950 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.950 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.951 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:27.951 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.210428+0000", 2024-09-20T19:01:27.951 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000114369 2024-09-20T19:01:27.951 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.951 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.951 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1", 2024-09-20T19:01:27.951 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.210549+0000", 2024-09-20T19:01:27.951 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000121558 2024-09-20T19:01:27.951 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.951 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.951 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:01:27.951 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.217664+0000", 2024-09-20T19:01:27.951 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0071143090000000001 2024-09-20T19:01:27.952 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.952 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.952 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:27.952 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.218937+0000", 2024-09-20T19:01:27.952 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0012730599999999999 2024-09-20T19:01:27.952 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.952 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.952 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:27.952 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.218991+0000", 2024-09-20T19:01:27.952 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4722999999999998e-05 2024-09-20T19:01:27.952 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.952 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.952 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:27.952 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.219104+0000", 2024-09-20T19:01:27.952 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000112477 2024-09-20T19:01:27.953 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:27.953 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:27.953 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:27.953 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.953 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.953 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:142981 9.e 9:7a27425f:::benchmark_data_smithi083_124552_object28834:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e292)", 2024-09-20T19:01:27.953 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:00:35.254008+0000", 2024-09-20T19:01:27.953 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 52.307231444000003, 2024-09-20T19:01:27.953 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.966262548, 2024-09-20T19:01:27.953 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:27.953 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:27.953 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:27.953 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:01:27.954 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:01:27.954 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:01:27.954 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 142981 2024-09-20T19:01:27.954 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.954 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:27.954 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.954 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:27.954 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.254008+0000", 2024-09-20T19:01:27.954 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:27.954 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.954 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.954 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:27.954 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.254008+0000", 2024-09-20T19:01:27.954 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:27.955 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.955 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.955 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:27.955 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.254010+0000", 2024-09-20T19:01:27.955 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.443e-06 2024-09-20T19:01:27.955 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.955 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.955 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:27.955 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.254014+0000", 2024-09-20T19:01:27.955 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9260000000000002e-06 2024-09-20T19:01:27.955 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.955 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.955 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:27.955 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.254024+0000", 2024-09-20T19:01:27.956 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.9839999999999996e-06 2024-09-20T19:01:27.956 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.956 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.956 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:27.956 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.254029+0000", 2024-09-20T19:01:27.956 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6869999999999999e-06 2024-09-20T19:01:27.956 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.956 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.956 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:27.956 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.254089+0000", 2024-09-20T19:01:27.956 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9435000000000003e-05 2024-09-20T19:01:27.956 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.956 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.956 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:01:27.957 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.254094+0000", 2024-09-20T19:01:27.957 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4399999999999996e-06 2024-09-20T19:01:27.957 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.957 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.957 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:27.957 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.212819+0000", 2024-09-20T19:01:27.957 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.95872477300000003 2024-09-20T19:01:27.957 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.957 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.957 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:27.957 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.212953+0000", 2024-09-20T19:01:27.957 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013434699999999999 2024-09-20T19:01:27.957 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.958 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.958 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1", 2024-09-20T19:01:27.958 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.213056+0000", 2024-09-20T19:01:27.958 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000102675 2024-09-20T19:01:27.958 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.958 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.958 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:01:27.958 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.217670+0000", 2024-09-20T19:01:27.958 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0046140879999999997 2024-09-20T19:01:27.958 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.958 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.958 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:27.958 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.219977+0000", 2024-09-20T19:01:27.959 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0023067230000000001 2024-09-20T19:01:27.959 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.959 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.959 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:27.959 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.220160+0000", 2024-09-20T19:01:27.959 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00018349800000000001 2024-09-20T19:01:27.959 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.959 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.959 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:27.959 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.220271+0000", 2024-09-20T19:01:27.959 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000110529 2024-09-20T19:01:27.960 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:27.960 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:27.960 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:27.960 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.960 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.960 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:142991 9.e 9:75f432de:::benchmark_data_smithi083_124552_object28844:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e292)", 2024-09-20T19:01:27.960 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:00:35.271404+0000", 2024-09-20T19:01:27.960 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 52.289836082999997, 2024-09-20T19:01:27.960 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.94906818900000001, 2024-09-20T19:01:27.960 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:27.960 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:27.960 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:27.960 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:01:27.960 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:01:27.960 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:01:27.960 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 142991 2024-09-20T19:01:27.961 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.961 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:27.961 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.961 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:27.961 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.271404+0000", 2024-09-20T19:01:27.961 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:27.961 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.961 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.961 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:27.961 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.271404+0000", 2024-09-20T19:01:27.961 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:27.961 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.961 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.961 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:27.961 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.271405+0000", 2024-09-20T19:01:27.962 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.569999999999999e-07 2024-09-20T19:01:27.962 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.962 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.962 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:27.962 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.271408+0000", 2024-09-20T19:01:27.962 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.647e-06 2024-09-20T19:01:27.962 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.962 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.962 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:27.962 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.271418+0000", 2024-09-20T19:01:27.962 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.6360000000000006e-06 2024-09-20T19:01:27.962 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.962 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.962 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:27.962 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.271424+0000", 2024-09-20T19:01:27.963 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6080000000000003e-06 2024-09-20T19:01:27.963 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.963 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.963 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:27.963 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.271468+0000", 2024-09-20T19:01:27.963 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3958999999999997e-05 2024-09-20T19:01:27.963 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.963 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.963 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:01:27.963 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.271473+0000", 2024-09-20T19:01:27.963 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0710000000000002e-06 2024-09-20T19:01:27.963 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.963 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.963 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:27.964 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.214908+0000", 2024-09-20T19:01:27.964 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.94343566999999995 2024-09-20T19:01:27.964 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.964 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.964 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:27.964 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.215018+0000", 2024-09-20T19:01:27.964 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000109641 2024-09-20T19:01:27.964 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.964 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.964 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1", 2024-09-20T19:01:27.964 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.215110+0000", 2024-09-20T19:01:27.964 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.2324000000000006e-05 2024-09-20T19:01:27.964 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.964 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.964 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:01:27.965 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.220290+0000", 2024-09-20T19:01:27.965 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0051801019999999998 2024-09-20T19:01:27.965 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.965 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.965 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:27.965 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.220324+0000", 2024-09-20T19:01:27.965 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4020000000000003e-05 2024-09-20T19:01:27.965 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.965 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.965 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:27.965 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.220389+0000", 2024-09-20T19:01:27.965 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5048999999999993e-05 2024-09-20T19:01:27.965 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.965 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.965 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:27.965 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.220472+0000", 2024-09-20T19:01:27.966 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.2504999999999997e-05 2024-09-20T19:01:27.966 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:27.966 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:27.966 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:27.966 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.966 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.966 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:184349 9.3 9:c8c5782c:::benchmark_data_smithi083_124552_object70202:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e299)", 2024-09-20T19:01:27.966 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:08.628496+0000", 2024-09-20T19:01:27.966 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 18.932744155999998, 2024-09-20T19:01:27.966 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.129854478, 2024-09-20T19:01:27.966 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:27.966 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:27.966 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:27.966 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:01:27.967 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:01:27.967 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:01:27.967 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 184349 2024-09-20T19:01:27.967 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.967 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:27.967 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.967 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:27.967 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:08.628496+0000", 2024-09-20T19:01:27.967 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:27.967 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.967 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.967 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:27.967 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:08.628496+0000", 2024-09-20T19:01:27.968 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:27.968 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.968 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.968 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:27.968 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:08.628497+0000", 2024-09-20T19:01:27.968 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2669999999999999e-06 2024-09-20T19:01:27.968 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.968 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.968 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:27.968 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:08.628502+0000", 2024-09-20T19:01:27.968 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7219999999999999e-06 2024-09-20T19:01:27.968 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.968 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.968 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:27.969 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:08.628510+0000", 2024-09-20T19:01:27.969 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.5620000000000004e-06 2024-09-20T19:01:27.969 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.969 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.969 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:27.969 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:08.628518+0000", 2024-09-20T19:01:27.969 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.3520000000000001e-06 2024-09-20T19:01:27.969 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.969 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.969 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:27.969 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:08.628610+0000", 2024-09-20T19:01:27.969 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.2121999999999997e-05 2024-09-20T19:01:27.969 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.969 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.970 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:27.970 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:08.628812+0000", 2024-09-20T19:01:27.970 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00020209899999999999 2024-09-20T19:01:27.970 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.970 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.970 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,3", 2024-09-20T19:01:27.970 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:08.628951+0000", 2024-09-20T19:01:27.970 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000138706 2024-09-20T19:01:27.970 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.970 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.970 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:27.970 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:08.632592+0000", 2024-09-20T19:01:27.970 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0036414559999999999 2024-09-20T19:01:27.971 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.971 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.971 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:01:27.971 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:08.632640+0000", 2024-09-20T19:01:27.971 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7613e-05 2024-09-20T19:01:27.971 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.971 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.971 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:01:27.971 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:08.758209+0000", 2024-09-20T19:01:27.971 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.12556927600000001 2024-09-20T19:01:27.971 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.971 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.971 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:27.971 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:08.758303+0000", 2024-09-20T19:01:27.972 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.4536000000000004e-05 2024-09-20T19:01:27.972 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:27.972 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:27.972 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:27.972 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:08.758350+0000", 2024-09-20T19:01:27.972 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.6767000000000001e-05 2024-09-20T19:01:27.972 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:27.972 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:27.972 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:27.972 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:27.972 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:27.972 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T19:01:27.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:27.974 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:27.975 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:27.991 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T19:01:28.076 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:28.111 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:28.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:28.112 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:28.212 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:01:28.248 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:28.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:28.249 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:01:28.350 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:28.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:28.387 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:28.387 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:28.488 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:28.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:28 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:01:28.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:28 smithi142 ceph-mon[32508]: pgmap v573: 148 pgs: 2 active+recovering+undersized+remapped, 10 peering, 136 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 367 op/s; 0 B/s, 43 objects/s recovering 2024-09-20T19:01:28.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:28 smithi142 ceph-mon[32508]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3633/85626 objects degraded (4.243%), 22 pgs degraded) 2024-09-20T19:01:28.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:28 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T19:01:28.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:28 smithi142 ceph-mon[32508]: osdmap e309: 8 total, 8 up, 8 in 2024-09-20T19:01:28.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:28 smithi142 ceph-mon[32508]: 4.b scrub starts 2024-09-20T19:01:28.527 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:28.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:28.528 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:28.630 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:28.656 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:28.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:28.657 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:28.758 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:28.795 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:28 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:01:28.796 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:28 smithi083 ceph-mon[29421]: pgmap v573: 148 pgs: 2 active+recovering+undersized+remapped, 10 peering, 136 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 367 op/s; 0 B/s, 43 objects/s recovering 2024-09-20T19:01:28.796 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:28 smithi083 ceph-mon[29421]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3633/85626 objects degraded (4.243%), 22 pgs degraded) 2024-09-20T19:01:28.796 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:28 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T19:01:28.796 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:28 smithi083 ceph-mon[29421]: osdmap e309: 8 total, 8 up, 8 in 2024-09-20T19:01:28.796 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:28 smithi083 ceph-mon[29421]: 4.b scrub starts 2024-09-20T19:01:28.796 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:28 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:01:28.796 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:28 smithi083 ceph-mon[36260]: pgmap v573: 148 pgs: 2 active+recovering+undersized+remapped, 10 peering, 136 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 367 op/s; 0 B/s, 43 objects/s recovering 2024-09-20T19:01:28.796 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:28 smithi083 ceph-mon[36260]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3633/85626 objects degraded (4.243%), 22 pgs degraded) 2024-09-20T19:01:28.796 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:28 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T19:01:28.796 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:28 smithi083 ceph-mon[36260]: osdmap e309: 8 total, 8 up, 8 in 2024-09-20T19:01:28.796 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:28 smithi083 ceph-mon[36260]: 4.b scrub starts 2024-09-20T19:01:28.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:28.799 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:28.800 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:28.900 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:28.935 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:28.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:28.936 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:29.037 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:29.064 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:29.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:29.065 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:29.082 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-20T19:01:29.166 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:29.203 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:29.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:29.204 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:29.305 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:01:29.343 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:29.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:29.345 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:01:29.445 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:29.468 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:29.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:29.469 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:29.499 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:01:29.570 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:01:29.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:29.617 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:29.617 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:01:29.718 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:29.749 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:29.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:29.750 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:29.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:29 smithi142 ceph-mon[32508]: 4.b scrub ok 2024-09-20T19:01:29.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:29 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:01:29.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:29 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:01:29.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:29 smithi142 ceph-mon[32508]: osdmap e310: 8 total, 8 up, 8 in 2024-09-20T19:01:29.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:29 smithi142 ceph-mon[32508]: 3.1d scrub starts 2024-09-20T19:01:29.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:29 smithi142 ceph-mon[32508]: 3.1d scrub ok 2024-09-20T19:01:29.795 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:29 smithi083 ceph-mon[29421]: 4.b scrub ok 2024-09-20T19:01:29.795 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:29 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:01:29.795 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:29 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:01:29.795 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:29 smithi083 ceph-mon[29421]: osdmap e310: 8 total, 8 up, 8 in 2024-09-20T19:01:29.795 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:29 smithi083 ceph-mon[29421]: 3.1d scrub starts 2024-09-20T19:01:29.795 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:29 smithi083 ceph-mon[29421]: 3.1d scrub ok 2024-09-20T19:01:29.795 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:29 smithi083 ceph-mon[36260]: 4.b scrub ok 2024-09-20T19:01:29.795 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:29 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:01:29.795 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:29 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:01:29.795 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:29 smithi083 ceph-mon[36260]: osdmap e310: 8 total, 8 up, 8 in 2024-09-20T19:01:29.795 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:29 smithi083 ceph-mon[36260]: 3.1d scrub starts 2024-09-20T19:01:29.796 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:29 smithi083 ceph-mon[36260]: 3.1d scrub ok 2024-09-20T19:01:29.851 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:29.886 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:29.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:29.888 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:29.989 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:30.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:30.013 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:30.013 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:30.113 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T19:01:30.113 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-09-20T19:01:30.113 INFO:teuthology.orchestra.run.smithi142.stdout: "num_ops": 0 2024-09-20T19:01:30.114 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T19:01:30.114 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:30.161 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:30.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:30.162 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:30.262 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:01:30.304 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:30.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:30.305 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:01:30.406 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:30.431 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:30.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:30.432 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:30.533 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:30.570 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:30.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:30.571 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:30.672 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:30.709 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:30.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:30.710 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:30.728 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:30 smithi083 ceph-mon[29421]: pgmap v576: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 1.6 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 715 op/s; 0 B/s, 72 objects/s recovering 2024-09-20T19:01:30.728 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:30 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:01:30.729 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:30 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:01:30.729 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:30 smithi083 ceph-mon[36260]: pgmap v576: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 1.6 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 715 op/s; 0 B/s, 72 objects/s recovering 2024-09-20T19:01:30.729 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:30 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:01:30.729 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:30 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:01:30.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:30 smithi142 ceph-mon[32508]: pgmap v576: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 1.6 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 715 op/s; 0 B/s, 72 objects/s recovering 2024-09-20T19:01:30.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:30 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:01:30.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:30 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:01:30.811 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:30.849 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:30.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:30.850 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:30.949 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_blocked_ops 2024-09-20T19:01:30.951 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:30.982 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:30.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:30.983 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:31.045 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:01:30 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:01:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:01:31.086 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:31.113 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:31.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:31.114 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:31.215 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:31.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:31.271 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:31.271 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:31.365 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:01:31.372 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:31.396 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:31.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:31.397 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:31.498 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:31.540 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:31.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:31.542 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:31.642 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:31.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:31.677 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:31.677 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:31.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:31 smithi142 ceph-mon[32508]: osdmap e311: 8 total, 8 up, 8 in 2024-09-20T19:01:31.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:31 smithi142 ceph-mon[32508]: 6.7 scrub starts 2024-09-20T19:01:31.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:31 smithi142 ceph-mon[32508]: 6.7 scrub ok 2024-09-20T19:01:31.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:31 smithi142 ceph-mon[32508]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:01:31.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:31 smithi142 ceph-mon[32508]: osdmap e312: 8 total, 8 up, 8 in 2024-09-20T19:01:31.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:31 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:01:31.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:31 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:01:31.778 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:31.795 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:31 smithi083 ceph-mon[29421]: osdmap e311: 8 total, 8 up, 8 in 2024-09-20T19:01:31.795 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:31 smithi083 ceph-mon[29421]: 6.7 scrub starts 2024-09-20T19:01:31.795 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:31 smithi083 ceph-mon[29421]: 6.7 scrub ok 2024-09-20T19:01:31.795 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:31 smithi083 ceph-mon[29421]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:01:31.795 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:31 smithi083 ceph-mon[29421]: osdmap e312: 8 total, 8 up, 8 in 2024-09-20T19:01:31.795 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:31 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:01:31.795 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:31 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:01:31.795 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:31 smithi083 ceph-mon[36260]: osdmap e311: 8 total, 8 up, 8 in 2024-09-20T19:01:31.796 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:31 smithi083 ceph-mon[36260]: 6.7 scrub starts 2024-09-20T19:01:31.796 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:31 smithi083 ceph-mon[36260]: 6.7 scrub ok 2024-09-20T19:01:31.796 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:31 smithi083 ceph-mon[36260]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:01:31.796 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:31 smithi083 ceph-mon[36260]: osdmap e312: 8 total, 8 up, 8 in 2024-09-20T19:01:31.796 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:31 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:01:31.796 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:31 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:01:31.814 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:31.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:31.815 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:31.916 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:31.953 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:31.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:31.954 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:32.006 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T19:01:32.006 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-09-20T19:01:32.006 INFO:teuthology.orchestra.run.smithi142.stdout: "complaint_time": 30, 2024-09-20T19:01:32.006 INFO:teuthology.orchestra.run.smithi142.stdout: "num_blocked_ops": 0 2024-09-20T19:01:32.006 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T19:01:32.055 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:32.091 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:32.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:32.092 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:32.193 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:32.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:32.214 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:32.214 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:32.315 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:32.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:32.347 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:32.347 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:32.448 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:32.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:32.483 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:32.483 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:32.583 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:32.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:32.614 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:32.614 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:32.714 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:32.752 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:32.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:32.753 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:32.764 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:32 smithi142 ceph-mon[32508]: 5.b scrub starts 2024-09-20T19:01:32.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:32 smithi142 ceph-mon[32508]: 5.b scrub ok 2024-09-20T19:01:32.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:32 smithi142 ceph-mon[32508]: 3.12 scrub starts 2024-09-20T19:01:32.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:32 smithi142 ceph-mon[32508]: 3.12 scrub ok 2024-09-20T19:01:32.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:32 smithi142 ceph-mon[32508]: pgmap v579: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 1.6 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 0 B/s wr, 325 op/s; 0 B/s, 26 objects/s recovering 2024-09-20T19:01:32.795 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:32 smithi083 ceph-mon[29421]: 5.b scrub starts 2024-09-20T19:01:32.795 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:32 smithi083 ceph-mon[29421]: 5.b scrub ok 2024-09-20T19:01:32.795 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:32 smithi083 ceph-mon[29421]: 3.12 scrub starts 2024-09-20T19:01:32.795 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:32 smithi083 ceph-mon[29421]: 3.12 scrub ok 2024-09-20T19:01:32.795 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:32 smithi083 ceph-mon[29421]: pgmap v579: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 1.6 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 0 B/s wr, 325 op/s; 0 B/s, 26 objects/s recovering 2024-09-20T19:01:32.795 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:32 smithi083 ceph-mon[36260]: 5.b scrub starts 2024-09-20T19:01:32.796 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:32 smithi083 ceph-mon[36260]: 5.b scrub ok 2024-09-20T19:01:32.796 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:32 smithi083 ceph-mon[36260]: 3.12 scrub starts 2024-09-20T19:01:32.796 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:32 smithi083 ceph-mon[36260]: 3.12 scrub ok 2024-09-20T19:01:32.796 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:32 smithi083 ceph-mon[36260]: pgmap v579: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 1.6 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 0 B/s wr, 325 op/s; 0 B/s, 26 objects/s recovering 2024-09-20T19:01:32.853 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:32.892 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:32.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:32.893 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:32.994 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:33.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:33.020 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:33.020 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:33.112 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_historic_ops 2024-09-20T19:01:33.121 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:33.197 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:33.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:33.198 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:33.299 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:33.324 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:33.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:33.325 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:33.426 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:33.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:33.468 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:33.468 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:33.489 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:01:33.569 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:33.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:33.604 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:33.604 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:33.705 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:33.741 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:33.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:33.742 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:33.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:33 smithi142 ceph-mon[32508]: 6.f scrub starts 2024-09-20T19:01:33.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:33 smithi142 ceph-mon[32508]: 6.f scrub ok 2024-09-20T19:01:33.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:33 smithi142 ceph-mon[32508]: 3.1f scrub starts 2024-09-20T19:01:33.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:33 smithi142 ceph-mon[32508]: 3.1f scrub ok 2024-09-20T19:01:33.794 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:33 smithi083 ceph-mon[29421]: 6.f scrub starts 2024-09-20T19:01:33.795 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:33 smithi083 ceph-mon[29421]: 6.f scrub ok 2024-09-20T19:01:33.795 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:33 smithi083 ceph-mon[29421]: 3.1f scrub starts 2024-09-20T19:01:33.795 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:33 smithi083 ceph-mon[29421]: 3.1f scrub ok 2024-09-20T19:01:33.795 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:33 smithi083 ceph-mon[36260]: 6.f scrub starts 2024-09-20T19:01:33.795 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:33 smithi083 ceph-mon[36260]: 6.f scrub ok 2024-09-20T19:01:33.795 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:33 smithi083 ceph-mon[36260]: 3.1f scrub starts 2024-09-20T19:01:33.795 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:33 smithi083 ceph-mon[36260]: 3.1f scrub ok 2024-09-20T19:01:33.843 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:33.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:33.939 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:33.939 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:34.040 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:34.063 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:34.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:34.064 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:34.165 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:34.194 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:34.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:34.195 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:34.296 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:34.330 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:34.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:34.331 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:34.432 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:34.455 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:34.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:34.456 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:34.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:34 smithi083 ceph-mon[29421]: pgmap v580: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 1.3 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 905 B/s rd, 0 B/s wr, 873 op/s; 0 B/s, 124 objects/s recovering 2024-09-20T19:01:34.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:34 smithi083 ceph-mon[36260]: pgmap v580: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 1.3 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 905 B/s rd, 0 B/s wr, 873 op/s; 0 B/s, 124 objects/s recovering 2024-09-20T19:01:34.557 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:34.585 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:34.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:34.586 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:34.687 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:34.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:34.709 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:34.709 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:34.764 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:34 smithi142 ceph-mon[32508]: pgmap v580: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 1.3 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 905 B/s rd, 0 B/s wr, 873 op/s; 0 B/s, 124 objects/s recovering 2024-09-20T19:01:34.810 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:34.849 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:34.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:34.850 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:34.950 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:34.988 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:34.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:34.989 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:35.089 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:01:35.110 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:35.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:35.112 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:01:35.212 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:01:35.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:35.248 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:35.248 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:01:35.348 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:35.381 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:35.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:35.382 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:35.483 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:35.516 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:35.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:35.517 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:35.618 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:35.649 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:35.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:35.651 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:35.751 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:35.790 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:35.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:35.791 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:35.892 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:35.921 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:35.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:35.922 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:36.023 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:36.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:36.045 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:36.045 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:36.146 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:36.180 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:36.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:36.181 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:36.282 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:36.307 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:36.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:36.308 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:36.408 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:36.439 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:36.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:36.440 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:36.541 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:36.576 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:36.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:36.577 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:36.678 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:36.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:36.710 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:36.710 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:36.764 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:36 smithi142 ceph-mon[32508]: pgmap v581: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 1.3 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 769 B/s rd, 0 B/s wr, 742 op/s; 0 B/s, 105 objects/s recovering 2024-09-20T19:01:36.795 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:36 smithi083 ceph-mon[29421]: pgmap v581: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 1.3 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 769 B/s rd, 0 B/s wr, 742 op/s; 0 B/s, 105 objects/s recovering 2024-09-20T19:01:36.795 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:36 smithi083 ceph-mon[36260]: pgmap v581: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 1.3 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 769 B/s rd, 0 B/s wr, 742 op/s; 0 B/s, 105 objects/s recovering 2024-09-20T19:01:36.810 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:01:36.845 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:36.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:36.846 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:01:36.947 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:36.984 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:36.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:36.985 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:37.085 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:37.114 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:37.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:37.115 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:37.216 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:37.247 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:37.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:37.248 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:37.264 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:01:37 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:01:37.349 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:37.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:37.389 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:37.389 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:37.490 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:37.527 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:37.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:37.528 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:37.629 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:37.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:37.652 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:37.652 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:37.753 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:37.783 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:37.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:37.784 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:37.794 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:37 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:01:37.795 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:37 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:01:37.795 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:37 smithi083 ceph-mon[29421]: 4.15 scrub starts 2024-09-20T19:01:37.795 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:37 smithi083 ceph-mon[29421]: 4.15 scrub ok 2024-09-20T19:01:37.795 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:37 smithi083 ceph-mon[29421]: osdmap e313: 8 total, 8 up, 8 in 2024-09-20T19:01:37.795 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:37 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:01:37.795 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:37 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:01:37.795 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:37 smithi083 ceph-mon[36260]: 4.15 scrub starts 2024-09-20T19:01:37.795 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:37 smithi083 ceph-mon[36260]: 4.15 scrub ok 2024-09-20T19:01:37.795 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:37 smithi083 ceph-mon[36260]: osdmap e313: 8 total, 8 up, 8 in 2024-09-20T19:01:37.885 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:37.917 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:37.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:37.919 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:38.014 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:37 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:01:38.015 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:37 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:01:38.015 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:37 smithi142 ceph-mon[32508]: 4.15 scrub starts 2024-09-20T19:01:38.015 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:37 smithi142 ceph-mon[32508]: 4.15 scrub ok 2024-09-20T19:01:38.015 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:37 smithi142 ceph-mon[32508]: osdmap e313: 8 total, 8 up, 8 in 2024-09-20T19:01:38.016 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T19:01:38.017 INFO:teuthology.orchestra.run.smithi142.stdout: "size": 20, 2024-09-20T19:01:38.017 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 600, 2024-09-20T19:01:38.017 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-09-20T19:01:38.017 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.017 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:200388 9.4 e306/305 9:25aa6e64:::benchmark_data_smithi083_124552_object86241:head v 306'12361, mlcod=291'8814)", 2024-09-20T19:01:38.017 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:24.232287+0000", 2024-09-20T19:01:38.017 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 12.906430557, 2024-09-20T19:01:38.017 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.05522167, 2024-09-20T19:01:38.017 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:38.017 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:38.017 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:38.017 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:38.018 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.018 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:38.018 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.232287+0000", 2024-09-20T19:01:38.018 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:38.018 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.018 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.018 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:38.018 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.232287+0000", 2024-09-20T19:01:38.018 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:38.018 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.018 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.018 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:38.019 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.232288+0000", 2024-09-20T19:01:38.019 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.3500000000000005e-07 2024-09-20T19:01:38.019 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.019 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.019 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:38.019 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.232295+0000", 2024-09-20T19:01:38.019 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.0910000000000004e-06 2024-09-20T19:01:38.019 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.019 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.019 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:38.020 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.232308+0000", 2024-09-20T19:01:38.020 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3393e-05 2024-09-20T19:01:38.020 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.020 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.020 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:38.020 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.232316+0000", 2024-09-20T19:01:38.020 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.3749999999999997e-06 2024-09-20T19:01:38.020 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.020 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.020 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:38.020 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.234966+0000", 2024-09-20T19:01:38.020 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0026504179999999999 2024-09-20T19:01:38.020 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.021 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.021 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:38.021 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.234991+0000", 2024-09-20T19:01:38.021 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.4981999999999999e-05 2024-09-20T19:01:38.021 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.021 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.021 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:38.021 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.287458+0000", 2024-09-20T19:01:38.021 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.052466984000000001 2024-09-20T19:01:38.021 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.022 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.022 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:38.022 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.287508+0000", 2024-09-20T19:01:38.022 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0491999999999997e-05 2024-09-20T19:01:38.022 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:38.022 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:38.022 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:38.022 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.022 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.022 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:200395 9.4 e306/305 9:204c3c35:::benchmark_data_smithi083_124552_object86248:head v 306'12362, mlcod=291'8814)", 2024-09-20T19:01:38.022 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:24.240400+0000", 2024-09-20T19:01:38.022 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 12.898316957, 2024-09-20T19:01:38.022 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.048786438000000001, 2024-09-20T19:01:38.023 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:38.023 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:38.023 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:38.023 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:38.023 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.023 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:38.023 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.240400+0000", 2024-09-20T19:01:38.023 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:38.023 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.023 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.023 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:38.023 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.240400+0000", 2024-09-20T19:01:38.023 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:38.024 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.024 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.024 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:38.024 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.240401+0000", 2024-09-20T19:01:38.024 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.9199999999999999e-07 2024-09-20T19:01:38.024 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.024 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.024 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:38.024 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.240407+0000", 2024-09-20T19:01:38.024 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3820000000000003e-06 2024-09-20T19:01:38.024 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.024 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.024 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:38.025 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.240420+0000", 2024-09-20T19:01:38.025 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.309e-05 2024-09-20T19:01:38.025 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.025 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.025 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:38.025 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.240426+0000", 2024-09-20T19:01:38.025 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2570000000000004e-06 2024-09-20T19:01:38.025 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.025 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.025 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:38.025 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.241642+0000", 2024-09-20T19:01:38.025 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0012156809999999999 2024-09-20T19:01:38.025 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.025 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.026 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:38.026 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.241662+0000", 2024-09-20T19:01:38.026 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0100000000000001e-05 2024-09-20T19:01:38.026 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.026 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.026 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:38.026 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.289139+0000", 2024-09-20T19:01:38.026 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.047477600000000002 2024-09-20T19:01:38.026 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.026 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.026 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:38.026 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.289187+0000", 2024-09-20T19:01:38.026 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7336000000000002e-05 2024-09-20T19:01:38.026 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:38.026 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:38.027 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:38.027 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.027 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.027 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:200396 9.4 e306/305 9:25503433:::benchmark_data_smithi083_124552_object86249:head v 306'12363, mlcod=291'8814)", 2024-09-20T19:01:38.027 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:24.243538+0000", 2024-09-20T19:01:38.027 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 12.895179071999999, 2024-09-20T19:01:38.027 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.049450610999999998, 2024-09-20T19:01:38.027 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:38.027 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:38.027 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:38.027 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:38.027 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.027 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:38.027 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.243538+0000", 2024-09-20T19:01:38.028 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:38.028 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.028 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.028 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:38.028 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.243538+0000", 2024-09-20T19:01:38.028 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:38.028 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.028 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.028 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:38.028 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.243539+0000", 2024-09-20T19:01:38.028 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.8500000000000001e-07 2024-09-20T19:01:38.028 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.028 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.028 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:38.028 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.243545+0000", 2024-09-20T19:01:38.028 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8579999999999996e-06 2024-09-20T19:01:38.029 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.029 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.029 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:38.029 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.243555+0000", 2024-09-20T19:01:38.029 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0057e-05 2024-09-20T19:01:38.029 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.029 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.029 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:38.029 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.243562+0000", 2024-09-20T19:01:38.029 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.6590000000000007e-06 2024-09-20T19:01:38.029 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.029 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.029 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:38.029 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.271297+0000", 2024-09-20T19:01:38.029 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.027734435000000002 2024-09-20T19:01:38.029 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.030 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.030 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:38.030 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.271318+0000", 2024-09-20T19:01:38.030 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0710999999999999e-05 2024-09-20T19:01:38.030 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.030 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.030 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:38.030 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.292915+0000", 2024-09-20T19:01:38.030 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.021597794999999999 2024-09-20T19:01:38.030 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.030 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.030 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:38.030 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.292989+0000", 2024-09-20T19:01:38.030 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.3411000000000006e-05 2024-09-20T19:01:38.030 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:38.031 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:38.031 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:38.031 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.031 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.031 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:200582 9.c e306/305 9:3c5407e3:::benchmark_data_smithi083_124552_object86435:head v 306'12461, mlcod=293'8972)", 2024-09-20T19:01:38.031 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:24.429295+0000", 2024-09-20T19:01:38.031 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 12.709422044, 2024-09-20T19:01:38.031 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.053539096000000001, 2024-09-20T19:01:38.031 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:38.031 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:38.031 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:38.032 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:38.032 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.032 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:38.032 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.429295+0000", 2024-09-20T19:01:38.032 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:38.032 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.032 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.032 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:38.032 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.429295+0000", 2024-09-20T19:01:38.032 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:38.032 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.032 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.032 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:38.032 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.429296+0000", 2024-09-20T19:01:38.032 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.1100000000000002e-07 2024-09-20T19:01:38.032 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.033 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.033 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:38.033 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.429300+0000", 2024-09-20T19:01:38.033 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3640000000000001e-06 2024-09-20T19:01:38.033 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.033 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.033 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:38.033 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.429305+0000", 2024-09-20T19:01:38.033 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3469999999999998e-06 2024-09-20T19:01:38.033 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.033 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.033 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:38.033 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.429310+0000", 2024-09-20T19:01:38.033 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.4279999999999998e-06 2024-09-20T19:01:38.034 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.034 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.034 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:38.034 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.429945+0000", 2024-09-20T19:01:38.034 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00063519 2024-09-20T19:01:38.034 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.034 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.034 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:38.034 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.429999+0000", 2024-09-20T19:01:38.034 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4035e-05 2024-09-20T19:01:38.034 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.034 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.035 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:38.035 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.482748+0000", 2024-09-20T19:01:38.035 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.052749404999999999 2024-09-20T19:01:38.035 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.035 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.035 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:38.035 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.482834+0000", 2024-09-20T19:01:38.035 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.6015999999999998e-05 2024-09-20T19:01:38.035 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:38.035 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:38.035 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:38.036 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.036 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.036 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:200580 9.4 e306/305 9:24e07e98:::benchmark_data_smithi083_124552_object86433:head v 306'12374, mlcod=293'8824)", 2024-09-20T19:01:38.036 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:24.431194+0000", 2024-09-20T19:01:38.036 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 12.707523318, 2024-09-20T19:01:38.036 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.051699992, 2024-09-20T19:01:38.036 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:38.036 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:38.036 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:38.036 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:38.036 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.036 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:38.036 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.431194+0000", 2024-09-20T19:01:38.036 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:38.037 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.037 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.037 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:38.037 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.431194+0000", 2024-09-20T19:01:38.037 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:38.037 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.037 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.037 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:38.037 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.431195+0000", 2024-09-20T19:01:38.037 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.0299999999999998e-07 2024-09-20T19:01:38.037 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.038 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.038 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:38.038 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.431200+0000", 2024-09-20T19:01:38.038 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.2979999999999999e-06 2024-09-20T19:01:38.038 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.038 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.038 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:38.038 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.431213+0000", 2024-09-20T19:01:38.038 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3281e-05 2024-09-20T19:01:38.038 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.038 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.039 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:38.039 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.431221+0000", 2024-09-20T19:01:38.039 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.1119999999999996e-06 2024-09-20T19:01:38.039 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.039 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.039 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:38.039 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.431385+0000", 2024-09-20T19:01:38.039 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00016385299999999999 2024-09-20T19:01:38.039 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.039 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.039 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:38.039 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.431430+0000", 2024-09-20T19:01:38.039 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.5222999999999997e-05 2024-09-20T19:01:38.039 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.040 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.040 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:38.040 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.482841+0000", 2024-09-20T19:01:38.040 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.051410628999999999 2024-09-20T19:01:38.040 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.040 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.040 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:38.040 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.482894+0000", 2024-09-20T19:01:38.040 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.2893000000000003e-05 2024-09-20T19:01:38.040 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:38.040 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:38.040 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:38.040 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.040 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.040 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:200591 9.c e306/305 9:3873cf45:::benchmark_data_smithi083_124552_object86444:head v 306'12462, mlcod=293'8972)", 2024-09-20T19:01:38.041 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:24.432966+0000", 2024-09-20T19:01:38.041 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 12.705751692, 2024-09-20T19:01:38.041 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.050042291000000003, 2024-09-20T19:01:38.041 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:38.041 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:38.041 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:38.041 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:38.041 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.041 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:38.041 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.432966+0000", 2024-09-20T19:01:38.041 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:38.041 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.041 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.042 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:38.042 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.432966+0000", 2024-09-20T19:01:38.042 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:38.042 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.042 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.042 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:38.042 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.432967+0000", 2024-09-20T19:01:38.042 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1230000000000001e-06 2024-09-20T19:01:38.042 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.042 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.042 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:38.042 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.432974+0000", 2024-09-20T19:01:38.042 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.9770000000000003e-06 2024-09-20T19:01:38.043 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.043 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.043 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:38.043 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.432989+0000", 2024-09-20T19:01:38.043 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5016e-05 2024-09-20T19:01:38.043 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.043 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.043 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:38.043 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.432996+0000", 2024-09-20T19:01:38.043 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5739999999999998e-06 2024-09-20T19:01:38.043 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.043 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.043 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:38.043 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.437464+0000", 2024-09-20T19:01:38.043 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0044675720000000004 2024-09-20T19:01:38.044 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.044 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.044 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:38.044 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.437513+0000", 2024-09-20T19:01:38.044 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.9295999999999997e-05 2024-09-20T19:01:38.044 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.044 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.044 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:38.044 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.482986+0000", 2024-09-20T19:01:38.044 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.045472882999999999 2024-09-20T19:01:38.044 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.044 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.044 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:38.044 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:24.483008+0000", 2024-09-20T19:01:38.045 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1849999999999999e-05 2024-09-20T19:01:38.045 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:38.045 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:38.045 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:38.045 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.045 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.045 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:201675 9.c e307/305 9:388dbe21:::benchmark_data_smithi083_124552_object87528:head v 307'12523, mlcod=293'9017)", 2024-09-20T19:01:38.045 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:25.551094+0000", 2024-09-20T19:01:38.045 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 11.587623795000001, 2024-09-20T19:01:38.045 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.050565898999999997, 2024-09-20T19:01:38.045 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:38.045 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:38.046 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:38.046 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:38.046 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.046 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:38.046 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.551094+0000", 2024-09-20T19:01:38.046 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:38.046 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.046 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.046 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:38.046 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.551094+0000", 2024-09-20T19:01:38.046 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:38.046 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.047 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.047 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:38.047 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.551094+0000", 2024-09-20T19:01:38.047 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.37e-07 2024-09-20T19:01:38.047 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.047 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.047 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:38.047 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.551100+0000", 2024-09-20T19:01:38.047 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2659999999999998e-06 2024-09-20T19:01:38.047 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.047 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.047 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:38.047 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.551111+0000", 2024-09-20T19:01:38.047 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0667e-05 2024-09-20T19:01:38.048 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.048 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.048 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:38.048 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.551118+0000", 2024-09-20T19:01:38.048 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.866e-06 2024-09-20T19:01:38.048 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.048 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.048 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:38.048 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.555841+0000", 2024-09-20T19:01:38.048 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0047226109999999998 2024-09-20T19:01:38.049 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.049 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.049 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:38.049 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.555860+0000", 2024-09-20T19:01:38.049 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9811000000000001e-05 2024-09-20T19:01:38.049 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.049 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.049 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:38.049 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.601619+0000", 2024-09-20T19:01:38.049 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.045758777 2024-09-20T19:01:38.049 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.049 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.049 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:38.050 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.601659+0000", 2024-09-20T19:01:38.050 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0263999999999999e-05 2024-09-20T19:01:38.050 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:38.050 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:38.050 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:38.050 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.050 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.050 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:201679 9.4 e307/305 9:2ddf9a69:::benchmark_data_smithi083_124552_object87532:head v 307'12435, mlcod=293'8865)", 2024-09-20T19:01:38.050 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:25.551837+0000", 2024-09-20T19:01:38.050 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 11.586880537000001, 2024-09-20T19:01:38.050 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.050608184000000001, 2024-09-20T19:01:38.050 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:38.050 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:38.051 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:38.051 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:38.051 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.051 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:38.051 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.551837+0000", 2024-09-20T19:01:38.051 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:38.051 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.051 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.051 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:38.051 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.551837+0000", 2024-09-20T19:01:38.051 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:38.051 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.052 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.052 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:38.052 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.551838+0000", 2024-09-20T19:01:38.052 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.2099999999999995e-07 2024-09-20T19:01:38.052 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.052 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.052 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:38.052 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.551846+0000", 2024-09-20T19:01:38.052 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.7839999999999992e-06 2024-09-20T19:01:38.052 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.052 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.052 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:38.052 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.551856+0000", 2024-09-20T19:01:38.053 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.8479999999999996e-06 2024-09-20T19:01:38.053 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.053 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.053 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:38.053 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.551863+0000", 2024-09-20T19:01:38.053 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4829999999999996e-06 2024-09-20T19:01:38.053 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.053 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.053 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:38.053 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.560194+0000", 2024-09-20T19:01:38.053 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0083316439999999992 2024-09-20T19:01:38.054 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.054 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.054 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:38.054 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.560218+0000", 2024-09-20T19:01:38.054 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3404000000000002e-05 2024-09-20T19:01:38.054 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.054 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.054 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:38.054 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.602402+0000", 2024-09-20T19:01:38.054 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.042184028999999998 2024-09-20T19:01:38.055 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.055 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.055 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:38.055 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.602445+0000", 2024-09-20T19:01:38.055 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3170999999999997e-05 2024-09-20T19:01:38.055 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:38.055 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:38.055 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:38.055 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.055 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.055 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:202031 9.4 e308/305 9:239c9fd6:::benchmark_data_smithi083_124552_object87884:head v 308'12455, mlcod=293'8878)", 2024-09-20T19:01:38.055 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:25.931089+0000", 2024-09-20T19:01:38.055 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 11.207627817000001, 2024-09-20T19:01:38.056 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.058741660000000001, 2024-09-20T19:01:38.056 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:38.056 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:38.056 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:38.056 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:38.056 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.056 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:38.056 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.931089+0000", 2024-09-20T19:01:38.056 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:38.056 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.056 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.056 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:38.056 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.931089+0000", 2024-09-20T19:01:38.056 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:38.057 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.057 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.057 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:38.057 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.931090+0000", 2024-09-20T19:01:38.057 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.0499999999999999e-07 2024-09-20T19:01:38.057 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.057 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.057 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:38.057 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.931095+0000", 2024-09-20T19:01:38.057 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8380000000000001e-06 2024-09-20T19:01:38.057 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.057 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.057 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:38.058 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.931100+0000", 2024-09-20T19:01:38.058 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3109999999999996e-06 2024-09-20T19:01:38.058 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.058 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.058 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:38.058 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.931104+0000", 2024-09-20T19:01:38.058 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.216e-06 2024-09-20T19:01:38.058 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.058 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.058 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:38.059 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.931586+0000", 2024-09-20T19:01:38.059 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000481528 2024-09-20T19:01:38.059 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.059 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.059 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:38.059 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.931633+0000", 2024-09-20T19:01:38.059 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7525000000000003e-05 2024-09-20T19:01:38.059 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.059 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.059 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:38.059 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.989672+0000", 2024-09-20T19:01:38.059 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.058038516999999998 2024-09-20T19:01:38.059 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.059 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.060 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:38.060 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.989831+0000", 2024-09-20T19:01:38.060 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015941999999999999 2024-09-20T19:01:38.060 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:38.060 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:38.060 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:38.060 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.060 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.060 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:202032 9.4 e308/305 9:2d783284:::benchmark_data_smithi083_124552_object87885:head v 308'12456, mlcod=293'8878)", 2024-09-20T19:01:38.060 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:25.931110+0000", 2024-09-20T19:01:38.060 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 11.207606839, 2024-09-20T19:01:38.060 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.058850316999999999, 2024-09-20T19:01:38.061 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:38.061 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:38.061 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:38.061 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:38.061 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.061 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:38.061 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.931110+0000", 2024-09-20T19:01:38.061 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:38.061 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.061 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.061 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:38.061 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.931110+0000", 2024-09-20T19:01:38.061 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:38.062 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.062 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.062 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:38.062 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.931111+0000", 2024-09-20T19:01:38.062 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6500000000000001e-07 2024-09-20T19:01:38.062 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.062 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.062 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:38.062 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.931114+0000", 2024-09-20T19:01:38.062 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9679999999999998e-06 2024-09-20T19:01:38.062 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.062 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.062 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:38.063 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.931118+0000", 2024-09-20T19:01:38.063 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0600000000000001e-06 2024-09-20T19:01:38.063 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.063 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.063 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:38.063 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.931121+0000", 2024-09-20T19:01:38.063 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.473e-06 2024-09-20T19:01:38.063 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.063 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.063 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:38.063 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.940478+0000", 2024-09-20T19:01:38.063 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0093569529999999995 2024-09-20T19:01:38.063 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.063 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.064 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:38.064 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.940509+0000", 2024-09-20T19:01:38.064 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.0880999999999998e-05 2024-09-20T19:01:38.064 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.064 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.064 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:38.064 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.989836+0000", 2024-09-20T19:01:38.064 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.049327109000000001 2024-09-20T19:01:38.064 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.064 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.064 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:38.064 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.989961+0000", 2024-09-20T19:01:38.064 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012470800000000001 2024-09-20T19:01:38.065 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:38.065 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:38.065 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:38.065 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.065 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.065 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:202052 9.4 e308/305 9:28f81496:::benchmark_data_smithi083_124552_object87905:head v 308'12457, mlcod=293'8878)", 2024-09-20T19:01:38.065 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:25.938366+0000", 2024-09-20T19:01:38.065 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 11.200351470999999, 2024-09-20T19:01:38.065 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.056471383, 2024-09-20T19:01:38.065 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:38.065 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:38.065 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:38.065 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:38.065 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.066 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:38.066 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.938366+0000", 2024-09-20T19:01:38.066 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:38.066 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.066 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.066 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:38.066 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.938366+0000", 2024-09-20T19:01:38.066 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:38.066 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.066 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.066 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:38.066 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.938367+0000", 2024-09-20T19:01:38.066 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.0200000000000001e-07 2024-09-20T19:01:38.066 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.067 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.067 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:38.067 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.938373+0000", 2024-09-20T19:01:38.067 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0279999999999997e-06 2024-09-20T19:01:38.067 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.067 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.067 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:38.067 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.938384+0000", 2024-09-20T19:01:38.067 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1629e-05 2024-09-20T19:01:38.067 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.067 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.067 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:38.067 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.938391+0000", 2024-09-20T19:01:38.067 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.6830000000000002e-06 2024-09-20T19:01:38.067 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.068 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.068 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:38.068 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.957361+0000", 2024-09-20T19:01:38.068 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.018969667999999999 2024-09-20T19:01:38.068 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.068 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.068 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:38.068 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.957385+0000", 2024-09-20T19:01:38.068 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.4853000000000001e-05 2024-09-20T19:01:38.068 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.068 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.068 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:38.068 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.994663+0000", 2024-09-20T19:01:38.069 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.037277976999999997 2024-09-20T19:01:38.069 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.069 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.069 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:38.069 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.994837+0000", 2024-09-20T19:01:38.069 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000173843 2024-09-20T19:01:38.069 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:38.069 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:38.069 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:38.069 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.069 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.070 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:202046 9.c e308/305 9:32a55499:::benchmark_data_smithi083_124552_object87899:head v 308'12553, mlcod=293'9027)", 2024-09-20T19:01:38.070 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:25.944941+0000", 2024-09-20T19:01:38.070 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 11.193776764000001, 2024-09-20T19:01:38.070 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.050464578000000003, 2024-09-20T19:01:38.070 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:38.070 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:38.070 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:38.070 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:38.070 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.070 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:38.070 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.944941+0000", 2024-09-20T19:01:38.070 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:38.070 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.071 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.071 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:38.071 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.944941+0000", 2024-09-20T19:01:38.071 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:38.071 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.071 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.071 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:38.071 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.944941+0000", 2024-09-20T19:01:38.071 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4200000000000002e-07 2024-09-20T19:01:38.071 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.071 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.071 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:38.071 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.944947+0000", 2024-09-20T19:01:38.072 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.1090000000000003e-06 2024-09-20T19:01:38.072 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.072 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.072 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:38.072 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.944953+0000", 2024-09-20T19:01:38.072 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6629999999999998e-06 2024-09-20T19:01:38.072 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.072 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.072 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:38.072 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.944958+0000", 2024-09-20T19:01:38.072 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8779999999999997e-06 2024-09-20T19:01:38.072 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.073 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.073 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:38.073 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.957385+0000", 2024-09-20T19:01:38.073 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.012427168000000001 2024-09-20T19:01:38.073 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.073 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.073 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:38.073 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.957400+0000", 2024-09-20T19:01:38.073 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4839e-05 2024-09-20T19:01:38.073 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.073 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.073 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:38.073 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.994840+0000", 2024-09-20T19:01:38.074 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.037440809999999998 2024-09-20T19:01:38.074 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.074 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.074 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:38.074 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.995405+0000", 2024-09-20T19:01:38.074 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00056476900000000001 2024-09-20T19:01:38.074 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:38.074 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:38.074 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:38.074 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.074 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.075 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:202049 9.c e308/305 9:30eca193:::benchmark_data_smithi083_124552_object87902:head v 308'12554, mlcod=293'9029)", 2024-09-20T19:01:38.075 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:25.945175+0000", 2024-09-20T19:01:38.075 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 11.193541996, 2024-09-20T19:01:38.075 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.050432099000000001, 2024-09-20T19:01:38.075 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:38.075 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:38.075 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:38.075 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:38.075 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.075 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:38.075 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.945175+0000", 2024-09-20T19:01:38.075 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:38.076 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.076 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.076 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:38.077 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.945175+0000", 2024-09-20T19:01:38.077 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:38.077 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.077 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.077 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:38.077 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.945176+0000", 2024-09-20T19:01:38.077 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.7399999999999991e-07 2024-09-20T19:01:38.077 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.077 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.077 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:38.077 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.945183+0000", 2024-09-20T19:01:38.077 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.3899999999999998e-06 2024-09-20T19:01:38.078 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.078 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.078 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:38.078 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.945197+0000", 2024-09-20T19:01:38.078 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4579999999999999e-05 2024-09-20T19:01:38.078 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.078 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.078 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:38.078 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.945204+0000", 2024-09-20T19:01:38.078 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.0090000000000001e-06 2024-09-20T19:01:38.079 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.079 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.079 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:38.079 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.970024+0000", 2024-09-20T19:01:38.079 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.024819325999999999 2024-09-20T19:01:38.079 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.079 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.079 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:38.079 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.970053+0000", 2024-09-20T19:01:38.079 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9111999999999998e-05 2024-09-20T19:01:38.079 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.079 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.080 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:38.080 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.995484+0000", 2024-09-20T19:01:38.080 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.025431174000000001 2024-09-20T19:01:38.080 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.080 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.080 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:38.080 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:25.995607+0000", 2024-09-20T19:01:38.080 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012353399999999999 2024-09-20T19:01:38.080 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:38.080 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:38.080 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:38.080 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.080 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.081 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:204729 9.c e310/305 9:3d6ff9d0:::benchmark_data_smithi083_124552_object90582:head v 310'12720, mlcod=294'9127)", 2024-09-20T19:01:38.081 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:28.487330+0000", 2024-09-20T19:01:38.081 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 8.6513870389999994, 2024-09-20T19:01:38.081 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.047888370999999999, 2024-09-20T19:01:38.081 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:38.081 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:38.081 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:38.081 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:38.081 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.081 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:38.081 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:28.487330+0000", 2024-09-20T19:01:38.081 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:38.081 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.081 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.082 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:38.082 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:28.487330+0000", 2024-09-20T19:01:38.082 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:38.082 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.082 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.082 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:38.082 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:28.487332+0000", 2024-09-20T19:01:38.082 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3260000000000001e-06 2024-09-20T19:01:38.082 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.082 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.082 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:38.082 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:28.487339+0000", 2024-09-20T19:01:38.082 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.3730000000000004e-06 2024-09-20T19:01:38.083 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.083 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.083 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:38.083 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:28.487353+0000", 2024-09-20T19:01:38.083 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4453e-05 2024-09-20T19:01:38.083 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.083 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.083 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:38.083 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:28.487362+0000", 2024-09-20T19:01:38.083 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.8040000000000007e-06 2024-09-20T19:01:38.083 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.083 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.083 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:38.083 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:28.494852+0000", 2024-09-20T19:01:38.084 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0074901910000000002 2024-09-20T19:01:38.084 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.084 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.084 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:38.084 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:28.494911+0000", 2024-09-20T19:01:38.084 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8201000000000001e-05 2024-09-20T19:01:38.084 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.084 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.084 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:38.084 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:28.535171+0000", 2024-09-20T19:01:38.084 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.040260337 2024-09-20T19:01:38.084 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.084 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.084 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:38.085 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:28.535219+0000", 2024-09-20T19:01:38.085 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7685999999999997e-05 2024-09-20T19:01:38.085 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:38.085 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:38.085 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:38.085 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.085 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.085 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:204720 9.4 e310/305 9:2e606f98:::benchmark_data_smithi083_124552_object90573:head v 310'12619, mlcod=294'8977)", 2024-09-20T19:01:38.085 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:28.487402+0000", 2024-09-20T19:01:38.085 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 8.6513148619999996, 2024-09-20T19:01:38.085 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.048010074, 2024-09-20T19:01:38.085 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:38.085 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:38.085 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:38.086 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:38.086 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.086 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:38.086 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:28.487402+0000", 2024-09-20T19:01:38.086 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:38.086 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.086 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.086 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:38.086 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:28.487402+0000", 2024-09-20T19:01:38.086 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:38.086 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.086 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.087 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:38.087 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:28.487403+0000", 2024-09-20T19:01:38.087 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.7700000000000004e-07 2024-09-20T19:01:38.087 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.087 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.087 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:38.087 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:28.487408+0000", 2024-09-20T19:01:38.087 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.5589999999999999e-06 2024-09-20T19:01:38.087 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.087 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.087 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:38.087 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:28.487415+0000", 2024-09-20T19:01:38.087 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.8210000000000002e-06 2024-09-20T19:01:38.088 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.088 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.088 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:38.088 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:28.487421+0000", 2024-09-20T19:01:38.088 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5999999999999997e-06 2024-09-20T19:01:38.088 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.088 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.088 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:38.088 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:28.501860+0000", 2024-09-20T19:01:38.088 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.014438806 2024-09-20T19:01:38.088 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.088 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.088 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:38.088 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:28.501889+0000", 2024-09-20T19:01:38.089 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9319000000000002e-05 2024-09-20T19:01:38.089 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.089 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.089 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:38.089 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:28.535294+0000", 2024-09-20T19:01:38.089 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.033404859000000002 2024-09-20T19:01:38.089 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.089 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.089 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:38.089 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:28.535413+0000", 2024-09-20T19:01:38.089 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000118533 2024-09-20T19:01:38.089 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:38.089 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:38.089 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:38.089 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.090 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.090 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:205840 9.e e310/305 9:73b4079c:::benchmark_data_smithi083_124552_object91693:head v 310'13025, mlcod=294'9311)", 2024-09-20T19:01:38.090 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:29.652944+0000", 2024-09-20T19:01:38.090 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 7.4857735720000003, 2024-09-20T19:01:38.090 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.050784032, 2024-09-20T19:01:38.090 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:38.090 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:38.090 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:38.090 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:38.090 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.090 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:38.090 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:29.652944+0000", 2024-09-20T19:01:38.091 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:38.091 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.091 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.091 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:38.091 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:29.652944+0000", 2024-09-20T19:01:38.091 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:38.091 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.091 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.091 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:38.091 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:29.652945+0000", 2024-09-20T19:01:38.091 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0640000000000001e-06 2024-09-20T19:01:38.091 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.091 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.091 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:38.092 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:29.652952+0000", 2024-09-20T19:01:38.092 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5440000000000001e-06 2024-09-20T19:01:38.092 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.092 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.092 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:38.092 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:29.652965+0000", 2024-09-20T19:01:38.092 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2975e-05 2024-09-20T19:01:38.092 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.092 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.092 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:38.092 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:29.652972+0000", 2024-09-20T19:01:38.092 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.1169999999999998e-06 2024-09-20T19:01:38.093 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.093 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.093 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:38.093 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:29.662779+0000", 2024-09-20T19:01:38.093 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0098065349999999999 2024-09-20T19:01:38.093 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.093 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.093 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:38.093 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:29.662815+0000", 2024-09-20T19:01:38.093 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.5651999999999997e-05 2024-09-20T19:01:38.093 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.093 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.093 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:38.094 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:29.703680+0000", 2024-09-20T19:01:38.094 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.040865085000000002 2024-09-20T19:01:38.094 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.094 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.094 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:38.094 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:29.703728+0000", 2024-09-20T19:01:38.094 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8059999999999997e-05 2024-09-20T19:01:38.094 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:38.094 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:38.094 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:38.094 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.094 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.094 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:206280 9.c e310/305 9:3ec7aab0:::benchmark_data_smithi083_124552_object92133:head v 310'12812, mlcod=294'9178)", 2024-09-20T19:01:38.094 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:30.125559+0000", 2024-09-20T19:01:38.095 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 7.0131588049999998, 2024-09-20T19:01:38.095 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.050721490000000001, 2024-09-20T19:01:38.095 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:38.095 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:38.095 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:38.095 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:38.095 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.095 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:38.095 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:30.125559+0000", 2024-09-20T19:01:38.095 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:38.095 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.095 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.095 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:38.095 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:30.125559+0000", 2024-09-20T19:01:38.096 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:38.096 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.096 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.096 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:38.096 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:30.125560+0000", 2024-09-20T19:01:38.096 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.221e-06 2024-09-20T19:01:38.096 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.096 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.096 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:38.096 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:30.125568+0000", 2024-09-20T19:01:38.096 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.0080000000000002e-06 2024-09-20T19:01:38.096 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.096 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.096 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:38.097 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:30.125583+0000", 2024-09-20T19:01:38.097 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5579000000000001e-05 2024-09-20T19:01:38.097 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.097 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.097 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:38.097 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:30.125591+0000", 2024-09-20T19:01:38.097 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.678e-06 2024-09-20T19:01:38.097 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.097 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.097 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:38.097 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:30.125652+0000", 2024-09-20T19:01:38.097 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0779999999999997e-05 2024-09-20T19:01:38.097 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.097 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.098 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:38.098 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:30.125678+0000", 2024-09-20T19:01:38.098 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.6602e-05 2024-09-20T19:01:38.098 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.098 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.098 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:38.098 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:30.176238+0000", 2024-09-20T19:01:38.098 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.050559525000000001 2024-09-20T19:01:38.098 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.098 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.098 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:38.098 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:30.176280+0000", 2024-09-20T19:01:38.099 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.2097e-05 2024-09-20T19:01:38.099 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:38.099 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:38.099 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:38.099 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.099 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.099 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:209399 9.4 e312/305 9:29c8cb01:::benchmark_data_smithi083_124552_object95252:head v 312'12885, mlcod=294'9171)", 2024-09-20T19:01:38.099 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:33.420376+0000", 2024-09-20T19:01:38.099 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 3.7183413879999998, 2024-09-20T19:01:38.099 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.050676156, 2024-09-20T19:01:38.099 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:38.099 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:38.099 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:38.099 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:38.100 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.100 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:38.100 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:33.420376+0000", 2024-09-20T19:01:38.100 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:38.100 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.100 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.100 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:38.100 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:33.420376+0000", 2024-09-20T19:01:38.100 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:38.100 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.100 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.100 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:38.100 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:33.420377+0000", 2024-09-20T19:01:38.100 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.181e-06 2024-09-20T19:01:38.101 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.101 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.101 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:38.101 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:33.420385+0000", 2024-09-20T19:01:38.101 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.8280000000000003e-06 2024-09-20T19:01:38.101 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.101 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.101 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:38.101 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:33.420400+0000", 2024-09-20T19:01:38.101 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.471e-05 2024-09-20T19:01:38.101 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.101 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.101 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:38.101 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:33.420408+0000", 2024-09-20T19:01:38.102 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.7150000000000004e-06 2024-09-20T19:01:38.102 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.102 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.102 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:38.102 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:33.434178+0000", 2024-09-20T19:01:38.102 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.013769727000000001 2024-09-20T19:01:38.102 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.102 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.102 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:38.102 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:33.434343+0000", 2024-09-20T19:01:38.102 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000165296 2024-09-20T19:01:38.102 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.103 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.103 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:38.103 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:33.470974+0000", 2024-09-20T19:01:38.103 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.036630176 2024-09-20T19:01:38.103 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.103 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.103 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:38.103 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:33.471052+0000", 2024-09-20T19:01:38.103 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.8522999999999993e-05 2024-09-20T19:01:38.103 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:38.103 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:38.103 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:38.103 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.103 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.104 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:209611 9.c e312/305 9:37409d47:::benchmark_data_smithi083_124552_object95464:head v 312'12985, mlcod=294'9309)", 2024-09-20T19:01:38.104 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:33.633230+0000", 2024-09-20T19:01:38.104 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 3.5054875069999998, 2024-09-20T19:01:38.104 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.050076178999999998, 2024-09-20T19:01:38.104 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:38.104 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:38.104 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:38.104 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:38.104 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.104 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:38.104 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:33.633230+0000", 2024-09-20T19:01:38.104 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:38.104 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.104 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.105 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:38.105 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:33.633230+0000", 2024-09-20T19:01:38.105 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:38.105 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.105 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.105 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:38.105 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:33.633230+0000", 2024-09-20T19:01:38.106 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.7599999999999997e-07 2024-09-20T19:01:38.106 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.106 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.106 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:38.106 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:33.633235+0000", 2024-09-20T19:01:38.106 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.1840000000000001e-06 2024-09-20T19:01:38.106 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.106 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.106 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:38.106 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:33.633241+0000", 2024-09-20T19:01:38.106 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.9310000000000002e-06 2024-09-20T19:01:38.106 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.106 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.106 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:38.107 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:33.633245+0000", 2024-09-20T19:01:38.107 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.7670000000000001e-06 2024-09-20T19:01:38.107 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.107 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.107 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:38.107 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:33.643014+0000", 2024-09-20T19:01:38.107 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0097687050000000008 2024-09-20T19:01:38.107 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.107 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.107 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:38.107 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:33.643051+0000", 2024-09-20T19:01:38.107 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6708999999999997e-05 2024-09-20T19:01:38.107 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.108 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.108 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:38.108 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:33.683163+0000", 2024-09-20T19:01:38.108 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.040112150999999999 2024-09-20T19:01:38.108 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.108 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.108 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:38.108 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:33.683306+0000", 2024-09-20T19:01:38.108 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000143156 2024-09-20T19:01:38.108 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:38.108 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:38.108 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:38.108 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.108 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.109 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:209628 9.c e312/305 9:3e9e97a3:::benchmark_data_smithi083_124552_object95481:head v 312'12988, mlcod=294'9309)", 2024-09-20T19:01:38.109 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:33.645443+0000", 2024-09-20T19:01:38.109 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 3.4932746020000001, 2024-09-20T19:01:38.109 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.047779399, 2024-09-20T19:01:38.109 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:01:38.109 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:01:38.109 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:01:38.109 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:01:38.109 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.109 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:01:38.109 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:33.645443+0000", 2024-09-20T19:01:38.109 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:38.110 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.110 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.110 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:01:38.110 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:33.645443+0000", 2024-09-20T19:01:38.110 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:01:38.110 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.110 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.110 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:01:38.110 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:33.645443+0000", 2024-09-20T19:01:38.110 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.6400000000000001e-07 2024-09-20T19:01:38.110 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.111 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.111 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:01:38.111 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:33.645450+0000", 2024-09-20T19:01:38.111 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4389999999999997e-06 2024-09-20T19:01:38.111 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.111 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.111 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:01:38.111 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:33.645471+0000", 2024-09-20T19:01:38.111 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0692000000000001e-05 2024-09-20T19:01:38.111 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.111 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.111 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:01:38.111 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:33.645478+0000", 2024-09-20T19:01:38.111 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.3849999999999996e-06 2024-09-20T19:01:38.112 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.112 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.112 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:01:38.112 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:33.657829+0000", 2024-09-20T19:01:38.112 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.012350754 2024-09-20T19:01:38.112 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.112 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.112 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:01:38.112 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:33.657860+0000", 2024-09-20T19:01:38.112 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.1313000000000003e-05 2024-09-20T19:01:38.112 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.112 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.112 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:01:38.112 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:33.693190+0000", 2024-09-20T19:01:38.113 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.035330237 2024-09-20T19:01:38.113 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:01:38.113 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:01:38.113 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:01:38.113 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:33.693222+0000", 2024-09-20T19:01:38.113 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.1814999999999999e-05 2024-09-20T19:01:38.113 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:38.113 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:38.113 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:38.113 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:01:38.113 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:01:38.113 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T19:01:38.114 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:38.145 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:38.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:38.146 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:38.247 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:38.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:38.280 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:38.280 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:38.382 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:38.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:38.419 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:38.419 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:38.520 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:38.560 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.0 to 1 (10000) 2024-09-20T19:01:38.560 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:38.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:38.561 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:38.662 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:38.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:38.697 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:38.697 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:38.795 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:38 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:01:38.795 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:38 smithi083 ceph-mon[29421]: pgmap v583: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 1.1 GiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.15k op/s; 0 B/s, 194 objects/s recovering 2024-09-20T19:01:38.795 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:38 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2345685179' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-20T19:01:38.795 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:38 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-20T19:01:38.796 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:38 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:01:38.796 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:38 smithi083 ceph-mon[36260]: pgmap v583: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 1.1 GiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.15k op/s; 0 B/s, 194 objects/s recovering 2024-09-20T19:01:38.796 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:38 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2345685179' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-20T19:01:38.796 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:38 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-20T19:01:38.799 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:38.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:38.838 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:38.838 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:38.939 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:38.976 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:38.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:38.977 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:39.015 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:38 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:01:39.015 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:38 smithi142 ceph-mon[32508]: pgmap v583: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 1.1 GiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.15k op/s; 0 B/s, 194 objects/s recovering 2024-09-20T19:01:39.015 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:38 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2345685179' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-20T19:01:39.015 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:38 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-20T19:01:39.078 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:39.117 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:39.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:39.118 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:39.219 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:39.248 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:39.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:39.249 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:39.350 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:39.381 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:39.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:39.382 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:39.393 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd reweight 1 1 2024-09-20T19:01:39.420 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:01:39.483 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:39.526 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:39.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:39.527 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:39.600 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:39 smithi083 ceph-mon[29421]: 6.0 scrub starts 2024-09-20T19:01:39.600 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:39 smithi083 ceph-mon[29421]: 6.0 scrub ok 2024-09-20T19:01:39.600 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:39 smithi083 ceph-mon[29421]: 5.f scrub starts 2024-09-20T19:01:39.600 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:39 smithi083 ceph-mon[29421]: 5.f scrub ok 2024-09-20T19:01:39.600 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:39 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-20T19:01:39.600 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:39 smithi083 ceph-mon[29421]: osdmap e314: 8 total, 8 up, 8 in 2024-09-20T19:01:39.600 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:39 smithi083 ceph-mon[36260]: 6.0 scrub starts 2024-09-20T19:01:39.600 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:39 smithi083 ceph-mon[36260]: 6.0 scrub ok 2024-09-20T19:01:39.600 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:39 smithi083 ceph-mon[36260]: 5.f scrub starts 2024-09-20T19:01:39.600 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:39 smithi083 ceph-mon[36260]: 5.f scrub ok 2024-09-20T19:01:39.601 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:39 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-20T19:01:39.601 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:39 smithi083 ceph-mon[36260]: osdmap e314: 8 total, 8 up, 8 in 2024-09-20T19:01:39.628 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:39.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:39.669 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:39.669 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:39.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:39 smithi142 ceph-mon[32508]: 6.0 scrub starts 2024-09-20T19:01:39.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:39 smithi142 ceph-mon[32508]: 6.0 scrub ok 2024-09-20T19:01:39.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:39 smithi142 ceph-mon[32508]: 5.f scrub starts 2024-09-20T19:01:39.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:39 smithi142 ceph-mon[32508]: 5.f scrub ok 2024-09-20T19:01:39.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:39 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-20T19:01:39.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:39 smithi142 ceph-mon[32508]: osdmap e314: 8 total, 8 up, 8 in 2024-09-20T19:01:39.769 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:39.808 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:39.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:39.809 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:39.910 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:39.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:39.945 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:39.945 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:40.046 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:40.084 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:40.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:40.085 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:40.112 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:01:40.112 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:01:40.148 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:01:40.148 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:01:40.183 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:01:40.183 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:01:40.186 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:40.194 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:01:40.230 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:01:40.230 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:01:40.232 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:40.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:40.233 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:40.250 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-20T19:01:40.269 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:01:40.269 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:01:40.309 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:01:40.310 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:01:40.334 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:01:40.343 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:01:40.343 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:01:40.367 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:40.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:40.368 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:01:40.375 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:01:40.376 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:01:40.469 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:01:40.499 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:40.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:40.500 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:01:40.600 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:40.633 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:40.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:40.634 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:40.638 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T19:01:40.735 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:40.785 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:40.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:40.786 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:40.795 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:40 smithi083 ceph-mon[29421]: pgmap v585: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 1005 MiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.18k op/s; 0 B/s, 186 objects/s recovering 2024-09-20T19:01:40.795 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:40 smithi083 ceph-mon[29421]: 2.2 scrub starts 2024-09-20T19:01:40.795 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:40 smithi083 ceph-mon[29421]: 2.2 scrub ok 2024-09-20T19:01:40.795 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:40 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/47836817' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:01:40.795 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:01:40 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:01:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:01:40.795 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:40 smithi083 ceph-mon[36260]: pgmap v585: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 1005 MiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.18k op/s; 0 B/s, 186 objects/s recovering 2024-09-20T19:01:40.795 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:40 smithi083 ceph-mon[36260]: 2.2 scrub starts 2024-09-20T19:01:40.796 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:40 smithi083 ceph-mon[36260]: 2.2 scrub ok 2024-09-20T19:01:40.796 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:40 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/47836817' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:01:40.887 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:40.921 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:40 smithi142 ceph-mon[32508]: pgmap v585: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 1005 MiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.18k op/s; 0 B/s, 186 objects/s recovering 2024-09-20T19:01:40.921 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:40 smithi142 ceph-mon[32508]: 2.2 scrub starts 2024-09-20T19:01:40.921 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:40 smithi142 ceph-mon[32508]: 2.2 scrub ok 2024-09-20T19:01:40.922 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:40 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/47836817' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:01:40.943 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:40.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:40.944 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:41.045 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:41.089 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:41.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:41.090 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:41.190 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:41.217 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:41.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:41.218 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:41.319 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:41.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:41.354 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:41.354 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:41.455 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:41.482 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:41.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:41.484 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:41.584 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:41.612 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:41.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:41.613 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:41.714 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:01:41.752 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:41.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:41.753 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:01:41.854 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:41.890 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:41.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:41.891 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:41.991 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:42.032 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:42.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:42.033 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:42.134 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:42.167 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:42.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:42.168 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:42.183 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:41 smithi083 ceph-mon[29421]: 4.a scrub starts 2024-09-20T19:01:42.183 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:41 smithi083 ceph-mon[29421]: 4.a scrub ok 2024-09-20T19:01:42.184 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:41 smithi083 ceph-mon[29421]: 5.1e scrub starts 2024-09-20T19:01:42.184 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:41 smithi083 ceph-mon[29421]: 5.1e scrub ok 2024-09-20T19:01:42.184 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:41 smithi083 ceph-mon[29421]: pgmap v586: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 1005 MiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 560 op/s; 0 B/s, 98 objects/s recovering 2024-09-20T19:01:42.184 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:41 smithi083 ceph-mon[36260]: 4.a scrub starts 2024-09-20T19:01:42.184 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:41 smithi083 ceph-mon[36260]: 4.a scrub ok 2024-09-20T19:01:42.184 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:41 smithi083 ceph-mon[36260]: 5.1e scrub starts 2024-09-20T19:01:42.184 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:41 smithi083 ceph-mon[36260]: 5.1e scrub ok 2024-09-20T19:01:42.184 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:41 smithi083 ceph-mon[36260]: pgmap v586: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 1005 MiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 560 op/s; 0 B/s, 98 objects/s recovering 2024-09-20T19:01:42.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:41 smithi142 ceph-mon[32508]: 4.a scrub starts 2024-09-20T19:01:42.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:41 smithi142 ceph-mon[32508]: 4.a scrub ok 2024-09-20T19:01:42.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:41 smithi142 ceph-mon[32508]: 5.1e scrub starts 2024-09-20T19:01:42.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:41 smithi142 ceph-mon[32508]: 5.1e scrub ok 2024-09-20T19:01:42.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:41 smithi142 ceph-mon[32508]: pgmap v586: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 1005 MiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 560 op/s; 0 B/s, 98 objects/s recovering 2024-09-20T19:01:42.269 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:42.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:42.309 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:42.310 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:42.379 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:01:42.410 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:42.477 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:42.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:42.478 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:42.579 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:42.617 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:42.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:42.618 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:42.710 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:01:42.719 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:42.754 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:42.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:42.755 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:42.848 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.1 to 1 (10000) 2024-09-20T19:01:42.855 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:42.896 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:42.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:42.897 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:42.998 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:01:43.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:43.035 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:43.035 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:01:43.136 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:01:43.171 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:43.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:43.172 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:01:43.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:42 smithi142 ceph-mon[32508]: 5.11 scrub starts 2024-09-20T19:01:43.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:42 smithi142 ceph-mon[32508]: 5.11 scrub ok 2024-09-20T19:01:43.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:42 smithi142 ceph-mon[32508]: 9.b scrub starts 2024-09-20T19:01:43.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:42 smithi142 ceph-mon[32508]: 9.b scrub ok 2024-09-20T19:01:43.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:42 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3037776445' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-20T19:01:43.273 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:43.278 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:42 smithi083 ceph-mon[29421]: 5.11 scrub starts 2024-09-20T19:01:43.278 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:42 smithi083 ceph-mon[29421]: 5.11 scrub ok 2024-09-20T19:01:43.278 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:42 smithi083 ceph-mon[29421]: 9.b scrub starts 2024-09-20T19:01:43.278 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:42 smithi083 ceph-mon[29421]: 9.b scrub ok 2024-09-20T19:01:43.278 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:42 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3037776445' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-20T19:01:43.279 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:42 smithi083 ceph-mon[36260]: 5.11 scrub starts 2024-09-20T19:01:43.279 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:42 smithi083 ceph-mon[36260]: 5.11 scrub ok 2024-09-20T19:01:43.279 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:42 smithi083 ceph-mon[36260]: 9.b scrub starts 2024-09-20T19:01:43.279 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:42 smithi083 ceph-mon[36260]: 9.b scrub ok 2024-09-20T19:01:43.279 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:42 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3037776445' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-20T19:01:43.314 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:43.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:43.315 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:43.416 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:43.446 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:43.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:43.447 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:43.548 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:43.587 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:43.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:43.591 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:43.692 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:43.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:43.728 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:43.728 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:43.774 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd reweight 2 1 2024-09-20T19:01:43.828 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:43.880 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:43.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:43.881 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:43.904 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-20T19:01:43.938 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T19:01:43.982 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:44.002 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:43 smithi083 ceph-mon[36260]: 3.a scrub starts 2024-09-20T19:01:44.002 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:43 smithi083 ceph-mon[36260]: 3.a scrub ok 2024-09-20T19:01:44.002 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:43 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3037776445' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-20T19:01:44.002 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:43 smithi083 ceph-mon[36260]: osdmap e315: 8 total, 8 up, 8 in 2024-09-20T19:01:44.003 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:43 smithi083 ceph-mon[36260]: pgmap v588: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 702 MiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 784 B/s rd, 0 B/s wr, 1.12k op/s; 0 B/s, 191 objects/s recovering 2024-09-20T19:01:44.003 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:43 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3644469769' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:01:44.003 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:43 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:01:44.003 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:43 smithi083 ceph-mon[29421]: 3.a scrub starts 2024-09-20T19:01:44.003 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:43 smithi083 ceph-mon[29421]: 3.a scrub ok 2024-09-20T19:01:44.003 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:43 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3037776445' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-20T19:01:44.003 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:43 smithi083 ceph-mon[29421]: osdmap e315: 8 total, 8 up, 8 in 2024-09-20T19:01:44.003 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:43 smithi083 ceph-mon[29421]: pgmap v588: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 702 MiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 784 B/s rd, 0 B/s wr, 1.12k op/s; 0 B/s, 191 objects/s recovering 2024-09-20T19:01:44.003 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:43 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3644469769' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:01:44.003 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:43 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:01:44.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:44.042 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:44.042 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:44.143 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:44.180 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:44.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:44.181 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:44.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:43 smithi142 ceph-mon[32508]: 3.a scrub starts 2024-09-20T19:01:44.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:43 smithi142 ceph-mon[32508]: 3.a scrub ok 2024-09-20T19:01:44.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:43 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3037776445' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-20T19:01:44.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:43 smithi142 ceph-mon[32508]: osdmap e315: 8 total, 8 up, 8 in 2024-09-20T19:01:44.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:43 smithi142 ceph-mon[32508]: pgmap v588: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 702 MiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 784 B/s rd, 0 B/s wr, 1.12k op/s; 0 B/s, 191 objects/s recovering 2024-09-20T19:01:44.266 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:43 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3644469769' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:01:44.266 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:43 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:01:44.282 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:44.319 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:44.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:44.320 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:44.421 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:01:44.450 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:44.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:44.452 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:01:44.552 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:44.557 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T19:01:44.557 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:01:44.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:44.612 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:44.612 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:44.713 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:44.752 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:44.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:44.753 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:44.853 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:44.880 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:44.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:44.881 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:44.982 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:45.019 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:45.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:45.022 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:45.044 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:44 smithi083 ceph-mon[29421]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T19:01:45.045 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:44 smithi083 ceph-mon[29421]: Cluster is now healthy 2024-09-20T19:01:45.045 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:44 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:01:45.045 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:44 smithi083 ceph-mon[29421]: osdmap e316: 8 total, 8 up, 8 in 2024-09-20T19:01:45.045 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:44 smithi083 ceph-mon[36260]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T19:01:45.045 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:44 smithi083 ceph-mon[36260]: Cluster is now healthy 2024-09-20T19:01:45.045 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:44 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:01:45.045 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:44 smithi083 ceph-mon[36260]: osdmap e316: 8 total, 8 up, 8 in 2024-09-20T19:01:45.123 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:45.156 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:45.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:45.157 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:45.258 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:45.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:44 smithi142 ceph-mon[32508]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T19:01:45.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:44 smithi142 ceph-mon[32508]: Cluster is now healthy 2024-09-20T19:01:45.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:44 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:01:45.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:44 smithi142 ceph-mon[32508]: osdmap e316: 8 total, 8 up, 8 in 2024-09-20T19:01:45.301 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:45.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:45.302 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:45.325 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:01:45.404 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:45.432 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:45.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:45.434 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:45.535 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:01:45.573 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:45.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:45.574 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:01:45.675 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:45.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:45.708 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:45.708 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:45.809 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:45.842 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:45.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:45.843 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:45.944 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:01:45.981 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:45.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:45.982 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:01:46.083 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:01:46.115 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:46.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:46.116 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:01:46.217 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:01:46.240 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:46.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:46.241 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:01:46.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:45 smithi142 ceph-mon[32508]: 5.1f scrub starts 2024-09-20T19:01:46.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:45 smithi142 ceph-mon[32508]: 5.1f scrub ok 2024-09-20T19:01:46.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:45 smithi142 ceph-mon[32508]: 3.1b scrub starts 2024-09-20T19:01:46.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:45 smithi142 ceph-mon[32508]: 3.1b scrub ok 2024-09-20T19:01:46.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:45 smithi142 ceph-mon[32508]: pgmap v590: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 702 MiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 785 B/s rd, 0 B/s wr, 744 op/s; 0 B/s, 123 objects/s recovering 2024-09-20T19:01:46.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:45 smithi083 ceph-mon[29421]: 5.1f scrub starts 2024-09-20T19:01:46.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:45 smithi083 ceph-mon[29421]: 5.1f scrub ok 2024-09-20T19:01:46.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:45 smithi083 ceph-mon[29421]: 3.1b scrub starts 2024-09-20T19:01:46.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:45 smithi083 ceph-mon[29421]: 3.1b scrub ok 2024-09-20T19:01:46.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:45 smithi083 ceph-mon[29421]: pgmap v590: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 702 MiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 785 B/s rd, 0 B/s wr, 744 op/s; 0 B/s, 123 objects/s recovering 2024-09-20T19:01:46.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:45 smithi083 ceph-mon[36260]: 5.1f scrub starts 2024-09-20T19:01:46.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:45 smithi083 ceph-mon[36260]: 5.1f scrub ok 2024-09-20T19:01:46.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:45 smithi083 ceph-mon[36260]: 3.1b scrub starts 2024-09-20T19:01:46.296 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:45 smithi083 ceph-mon[36260]: 3.1b scrub ok 2024-09-20T19:01:46.296 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:45 smithi083 ceph-mon[36260]: pgmap v590: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 702 MiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 785 B/s rd, 0 B/s wr, 744 op/s; 0 B/s, 123 objects/s recovering 2024-09-20T19:01:46.342 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:46.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:46.384 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:46.384 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:46.485 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:46.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:46.520 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:46.520 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:46.621 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:46.656 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:46.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:46.657 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:46.758 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:46.789 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:46.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:46.790 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:46.891 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:46.924 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:46.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:46.926 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:47.026 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:47.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:47.059 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:47.059 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:47.160 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:47.193 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:47.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:47.194 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:47.294 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:47.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:47 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:01:47.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:47 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:01:47.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:47 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:01:47.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:47 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:01:47.323 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T19:01:47.328 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:47.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:47.329 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:47.430 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:47.471 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:47.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:47.472 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:47.514 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:01:47 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:01:47.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:47 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:01:47.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:47 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:01:47.572 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:47.618 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:47.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:47.619 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:47.720 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:01:47.748 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:47.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:47.749 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:01:47.850 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:47.887 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:47.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:47.888 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:47.989 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:48.024 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:48.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:48.025 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:48.126 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:48.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:48.158 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:48.159 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:48.259 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:48.297 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:48.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:48.298 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:48.399 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:48.433 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:48.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:48.434 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:48.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:48 smithi142 ceph-mon[32508]: pgmap v591: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 511 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 988 op/s; 0 B/s, 178 objects/s recovering 2024-09-20T19:01:48.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:48 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:01:48.535 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:01:48.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:48 smithi083 ceph-mon[29421]: pgmap v591: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 511 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 988 op/s; 0 B/s, 178 objects/s recovering 2024-09-20T19:01:48.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:48 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:01:48.544 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:48 smithi083 ceph-mon[36260]: pgmap v591: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 511 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 988 op/s; 0 B/s, 178 objects/s recovering 2024-09-20T19:01:48.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:48 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:01:48.568 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:48.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:48.569 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:01:48.670 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:48.705 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:48.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:48.706 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:48.807 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:48.830 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:48.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:48.831 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:48.932 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:48.966 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:48.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:48.968 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:49.068 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:49.108 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:49.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:49.109 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:49.210 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:01:49.240 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:49.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:49.241 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:01:49.342 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:49.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:49.377 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:49.377 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:49.479 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:49.512 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:49.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:49.513 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:49.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:49 smithi142 ceph-mon[32508]: 3.16 scrub starts 2024-09-20T19:01:49.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:49 smithi142 ceph-mon[32508]: 3.16 scrub ok 2024-09-20T19:01:49.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:49 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:01:49.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:49 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:01:49.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:49 smithi083 ceph-mon[29421]: 3.16 scrub starts 2024-09-20T19:01:49.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:49 smithi083 ceph-mon[29421]: 3.16 scrub ok 2024-09-20T19:01:49.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:49 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:01:49.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:49 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:01:49.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:49 smithi083 ceph-mon[36260]: 3.16 scrub starts 2024-09-20T19:01:49.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:49 smithi083 ceph-mon[36260]: 3.16 scrub ok 2024-09-20T19:01:49.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:49 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:01:49.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:49 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:01:49.615 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:49.643 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:49.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:49.644 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:49.745 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:49.770 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:49.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:49.771 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:49.872 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:49.909 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:49.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:49.910 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:50.011 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:50.032 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:50.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:50.033 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:50.134 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:50.168 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:50.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:50.169 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:50.270 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:50.299 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:50.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:50.300 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:50.401 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:50.440 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:50.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:50.441 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:50.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:50 smithi142 ceph-mon[32508]: 3.1c scrub starts 2024-09-20T19:01:50.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:50 smithi142 ceph-mon[32508]: 3.1c scrub ok 2024-09-20T19:01:50.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:50 smithi142 ceph-mon[32508]: pgmap v592: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 370 MiB data, 4.7 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.27k op/s; 0 B/s, 204 objects/s recovering 2024-09-20T19:01:50.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:50 smithi142 ceph-mon[32508]: osdmap e317: 8 total, 8 up, 8 in 2024-09-20T19:01:50.542 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:50.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:50 smithi083 ceph-mon[29421]: 3.1c scrub starts 2024-09-20T19:01:50.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:50 smithi083 ceph-mon[29421]: 3.1c scrub ok 2024-09-20T19:01:50.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:50 smithi083 ceph-mon[29421]: pgmap v592: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 370 MiB data, 4.7 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.27k op/s; 0 B/s, 204 objects/s recovering 2024-09-20T19:01:50.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:50 smithi083 ceph-mon[29421]: osdmap e317: 8 total, 8 up, 8 in 2024-09-20T19:01:50.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:50 smithi083 ceph-mon[36260]: 3.1c scrub starts 2024-09-20T19:01:50.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:50 smithi083 ceph-mon[36260]: 3.1c scrub ok 2024-09-20T19:01:50.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:50 smithi083 ceph-mon[36260]: pgmap v592: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 370 MiB data, 4.7 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.27k op/s; 0 B/s, 204 objects/s recovering 2024-09-20T19:01:50.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:50 smithi083 ceph-mon[36260]: osdmap e317: 8 total, 8 up, 8 in 2024-09-20T19:01:50.577 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:50.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:50.577 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:50.678 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:50.701 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:50.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:50.702 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:50.803 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:50.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:50.853 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:50.853 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:50.954 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:50.985 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:50.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:50.986 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:51.044 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:01:50 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:01:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:01:51.087 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:51.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:51.121 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:51.121 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:51.222 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:51.260 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:51.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:51.261 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:51.362 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:51.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:51.401 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:51.401 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:51.502 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:01:51.538 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:51.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:51.539 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:01:51.640 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:51.680 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:51.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:51.681 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:51.782 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:51.825 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:51.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:51.826 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:51.926 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:01:51.962 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:51.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:51.963 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:01:52.014 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:51 smithi142 ceph-mon[32508]: osdmap e318: 8 total, 8 up, 8 in 2024-09-20T19:01:52.015 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:51 smithi142 ceph-mon[32508]: pgmap v595: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 370 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 712 B/s rd, 0 B/s wr, 740 op/s; 0 B/s, 115 objects/s recovering 2024-09-20T19:01:52.064 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:52.094 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:52.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:52.095 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:52.196 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:52.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:52.229 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:52.230 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:52.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:51 smithi083 ceph-mon[29421]: osdmap e318: 8 total, 8 up, 8 in 2024-09-20T19:01:52.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:51 smithi083 ceph-mon[29421]: pgmap v595: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 370 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 712 B/s rd, 0 B/s wr, 740 op/s; 0 B/s, 115 objects/s recovering 2024-09-20T19:01:52.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:51 smithi083 ceph-mon[36260]: osdmap e318: 8 total, 8 up, 8 in 2024-09-20T19:01:52.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:51 smithi083 ceph-mon[36260]: pgmap v595: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 370 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 712 B/s rd, 0 B/s wr, 740 op/s; 0 B/s, 115 objects/s recovering 2024-09-20T19:01:52.307 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:01:52.331 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:52.370 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_blocked_ops 2024-09-20T19:01:52.383 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:52.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:52.384 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:52.388 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Removed 114144 objects 2024-09-20T19:01:52.388 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Clean up completed and total clean up time :100.1 2024-09-20T19:01:52.485 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:52.510 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:52.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:52.511 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:52.612 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:01:52.636 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:52.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:52.637 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:01:52.738 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:52.760 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:52.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:52.760 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:52.861 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:52.884 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:52.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:52.885 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:52.986 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:53.008 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:53.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:53.009 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:53.110 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:53.133 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:53.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:53.134 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:53.235 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:53.258 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:53.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:53.259 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:53.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:52 smithi142 ceph-mon[32508]: 5.1b scrub starts 2024-09-20T19:01:53.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:52 smithi142 ceph-mon[32508]: 5.1b scrub ok 2024-09-20T19:01:53.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:52 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T19:01:53.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:52 smithi142 ceph-mon[32508]: 3.c scrub starts 2024-09-20T19:01:53.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:52 smithi142 ceph-mon[32508]: 3.c scrub ok 2024-09-20T19:01:53.294 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:52 smithi083 ceph-mon[29421]: 5.1b scrub starts 2024-09-20T19:01:53.294 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:52 smithi083 ceph-mon[29421]: 5.1b scrub ok 2024-09-20T19:01:53.294 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:52 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T19:01:53.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:52 smithi083 ceph-mon[29421]: 3.c scrub starts 2024-09-20T19:01:53.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:52 smithi083 ceph-mon[29421]: 3.c scrub ok 2024-09-20T19:01:53.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:52 smithi083 ceph-mon[36260]: 5.1b scrub starts 2024-09-20T19:01:53.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:52 smithi083 ceph-mon[36260]: 5.1b scrub ok 2024-09-20T19:01:53.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:52 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T19:01:53.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:52 smithi083 ceph-mon[36260]: 3.c scrub starts 2024-09-20T19:01:53.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:52 smithi083 ceph-mon[36260]: 3.c scrub ok 2024-09-20T19:01:53.360 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:53.384 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:53.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:53.384 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:53.485 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:53.507 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:53.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:53.508 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:53.608 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:53.630 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:53.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:53.632 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:53.733 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:53.757 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:53.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:53.758 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:53.859 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:53.881 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:53.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:53.882 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:53.983 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:54.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:54.004 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:54.004 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:54.105 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:54.127 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:54.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:54.128 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:54.229 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:54.251 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:54.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:54.252 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:54.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:53 smithi142 ceph-mon[32508]: pgmap v596: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 42 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.32k op/s; 0 B/s, 226 objects/s recovering 2024-09-20T19:01:54.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:53 smithi142 ceph-mon[32508]: 5.8 scrub starts 2024-09-20T19:01:54.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:53 smithi142 ceph-mon[32508]: 5.8 scrub ok 2024-09-20T19:01:54.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:53 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T19:01:54.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:53 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T19:01:54.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:53 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:01:54.294 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:53 smithi083 ceph-mon[29421]: pgmap v596: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 42 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.32k op/s; 0 B/s, 226 objects/s recovering 2024-09-20T19:01:54.294 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:53 smithi083 ceph-mon[29421]: 5.8 scrub starts 2024-09-20T19:01:54.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:53 smithi083 ceph-mon[29421]: 5.8 scrub ok 2024-09-20T19:01:54.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:53 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T19:01:54.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:53 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T19:01:54.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:53 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:01:54.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:53 smithi083 ceph-mon[36260]: pgmap v596: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 42 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.32k op/s; 0 B/s, 226 objects/s recovering 2024-09-20T19:01:54.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:53 smithi083 ceph-mon[36260]: 5.8 scrub starts 2024-09-20T19:01:54.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:53 smithi083 ceph-mon[36260]: 5.8 scrub ok 2024-09-20T19:01:54.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:53 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T19:01:54.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:53 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T19:01:54.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:53 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:01:54.353 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:54.374 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:54.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:54.375 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:54.476 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:54.499 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:54.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:54.500 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:54.601 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:54.623 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:54.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:54.624 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:54.725 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:54.748 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:54.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:54.749 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:54.850 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:54.874 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:54.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:54.876 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:54.976 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:55.000 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:55.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:55.001 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:55.101 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:55.124 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:55.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:55.125 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:55.226 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:55.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:55.250 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:55.250 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:55.351 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:55.372 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:55.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:55.373 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:55.430 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_2 2024-09-20T19:01:55.430 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd pool rm unique_pool_2 unique_pool_2 --yes-i-really-really-mean-it 2024-09-20T19:01:55.475 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:55.497 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:55.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:55.498 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:55.599 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:55.626 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:55.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:55.627 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:55.727 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:55.750 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:55.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:55.751 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:55.852 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:55.874 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:55.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:55.875 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:55.976 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:01:55.998 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:55.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:55.999 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:01:56.100 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:01:56.122 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:56.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:56.124 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:01:56.225 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:56.247 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:56.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:56.248 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:56.349 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:56.370 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:56.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:56.371 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:56.472 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:01:56.493 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:56.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:56.494 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:01:56.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:56 smithi142 ceph-mon[32508]: pgmap v597: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 42 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 937 op/s; 0 B/s, 149 objects/s recovering 2024-09-20T19:01:56.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:56 smithi083 ceph-mon[29421]: pgmap v597: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 42 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 937 op/s; 0 B/s, 149 objects/s recovering 2024-09-20T19:01:56.544 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:56 smithi083 ceph-mon[36260]: pgmap v597: 148 pgs: 3 active+recovering+undersized+remapped, 145 active+clean; 42 MiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 937 op/s; 0 B/s, 149 objects/s recovering 2024-09-20T19:01:56.595 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:56.617 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:56.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:56.617 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:56.718 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:01:56.739 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:56.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:56.740 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:01:56.841 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:56.863 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:56.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:56.864 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:56.965 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:56.990 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:56.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:56.991 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:57.005 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:01:57.092 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:57.114 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:57.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:57.115 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:57.216 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:57.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:57.239 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:57.239 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:57.340 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:01:57.363 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:57.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:57.364 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:01:57.464 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:57.486 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:57.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:57.487 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:57.514 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:01:57 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:01:57.588 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:57.609 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:57.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:57.610 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:57.711 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:01:57.732 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:57.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:57.733 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:01:57.834 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:57.840 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.2 to 1 (10000) 2024-09-20T19:01:57.862 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:57.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:57.863 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:57.964 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:57.985 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:57.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:57.985 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:58.086 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:58.108 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:58.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:58.109 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:58.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:57 smithi083 ceph-mon[29421]: 6.a scrub starts 2024-09-20T19:01:58.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:57 smithi083 ceph-mon[29421]: 6.a scrub ok 2024-09-20T19:01:58.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:57 smithi083 ceph-mon[29421]: osdmap e319: 8 total, 8 up, 8 in 2024-09-20T19:01:58.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:57 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2825946887' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-20T19:01:58.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:57 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-20T19:01:58.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:57 smithi083 ceph-mon[29421]: pgmap v599: 148 pgs: 1 active+undersized+remapped, 2 active+recovering+undersized+remapped, 145 active+clean; 3.9 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 735 op/s; 0 B/s, 431 objects/s recovering 2024-09-20T19:01:58.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:57 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:01:58.123 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:57 smithi083 ceph-mon[36260]: 6.a scrub starts 2024-09-20T19:01:58.123 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:57 smithi083 ceph-mon[36260]: 6.a scrub ok 2024-09-20T19:01:58.123 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:57 smithi083 ceph-mon[36260]: osdmap e319: 8 total, 8 up, 8 in 2024-09-20T19:01:58.123 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:57 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2825946887' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-20T19:01:58.123 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:57 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-20T19:01:58.124 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:57 smithi083 ceph-mon[36260]: pgmap v599: 148 pgs: 1 active+undersized+remapped, 2 active+recovering+undersized+remapped, 145 active+clean; 3.9 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 735 op/s; 0 B/s, 431 objects/s recovering 2024-09-20T19:01:58.124 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:57 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:01:58.209 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:58.230 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:58.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:58.231 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:58.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:57 smithi142 ceph-mon[32508]: 6.a scrub starts 2024-09-20T19:01:58.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:57 smithi142 ceph-mon[32508]: 6.a scrub ok 2024-09-20T19:01:58.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:57 smithi142 ceph-mon[32508]: osdmap e319: 8 total, 8 up, 8 in 2024-09-20T19:01:58.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:57 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2825946887' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-20T19:01:58.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:57 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-20T19:01:58.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:57 smithi142 ceph-mon[32508]: pgmap v599: 148 pgs: 1 active+undersized+remapped, 2 active+recovering+undersized+remapped, 145 active+clean; 3.9 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 735 op/s; 0 B/s, 431 objects/s recovering 2024-09-20T19:01:58.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:57 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:01:58.332 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:58.355 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:58.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:58.356 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:58.457 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:01:58.478 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:58.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:58.479 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:01:58.580 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:58.601 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:58.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:58.602 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:58.702 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:01:58.724 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:58.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:58.725 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:01:58.825 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:58.849 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:58.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:58.850 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:58.950 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:58.972 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:58.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:58.973 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:59.074 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:59.099 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:59.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:59.100 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:59.111 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:58 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-20T19:01:59.111 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:58 smithi083 ceph-mon[29421]: osdmap e320: 8 total, 8 up, 8 in 2024-09-20T19:01:59.111 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:58 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-20T19:01:59.111 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:58 smithi083 ceph-mon[36260]: osdmap e320: 8 total, 8 up, 8 in 2024-09-20T19:01:59.201 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:59.223 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:59.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:59.224 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:59.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:58 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-20T19:01:59.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:58 smithi142 ceph-mon[32508]: osdmap e320: 8 total, 8 up, 8 in 2024-09-20T19:01:59.325 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:01:59.346 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:59.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:59.347 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:01:59.448 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:01:59.469 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:01:59.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:59.470 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:01:59.570 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:01:59.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:59.595 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:59.595 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:01:59.695 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:59.717 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:59.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:59.718 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:01:59.819 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:01:59.841 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:59.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:59.842 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:01:59.943 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:01:59.965 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:01:59.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:01:59.966 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:00.066 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:00.088 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:00.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:00.089 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:00.190 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:00.212 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:00.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:00.213 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:00.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:59 smithi142 ceph-mon[32508]: 2.1 scrub starts 2024-09-20T19:02:00.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:59 smithi142 ceph-mon[32508]: 2.1 scrub ok 2024-09-20T19:02:00.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:01:59 smithi142 ceph-mon[32508]: pgmap v601: 148 pgs: 1 active+undersized+remapped, 2 active+recovering+undersized+remapped, 145 active+clean; 583 KiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 740 op/s; 0 B/s, 543 objects/s recovering 2024-09-20T19:02:00.294 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:59 smithi083 ceph-mon[29421]: 2.1 scrub starts 2024-09-20T19:02:00.294 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:59 smithi083 ceph-mon[29421]: 2.1 scrub ok 2024-09-20T19:02:00.294 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:01:59 smithi083 ceph-mon[29421]: pgmap v601: 148 pgs: 1 active+undersized+remapped, 2 active+recovering+undersized+remapped, 145 active+clean; 583 KiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 740 op/s; 0 B/s, 543 objects/s recovering 2024-09-20T19:02:00.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:59 smithi083 ceph-mon[36260]: 2.1 scrub starts 2024-09-20T19:02:00.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:59 smithi083 ceph-mon[36260]: 2.1 scrub ok 2024-09-20T19:02:00.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:01:59 smithi083 ceph-mon[36260]: pgmap v601: 148 pgs: 1 active+undersized+remapped, 2 active+recovering+undersized+remapped, 145 active+clean; 583 KiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 740 op/s; 0 B/s, 543 objects/s recovering 2024-09-20T19:02:00.314 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:02:00.336 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:00.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:00.337 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:02:00.437 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:02:00.460 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:00.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:00.461 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:02:00.561 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:00.585 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:00.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:00.586 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:00.687 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:00.710 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:00.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:00.711 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:00.811 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:00.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:00.837 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:00.837 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:00.938 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:00.960 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:00.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:00.961 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:01.044 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:00 smithi083 ceph-mon[29421]: 4.1c scrub starts 2024-09-20T19:02:01.044 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:00 smithi083 ceph-mon[29421]: 4.1c scrub ok 2024-09-20T19:02:01.045 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:00 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2217815265' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:02:01.045 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:02:00 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:02:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:02:01.045 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:00 smithi083 ceph-mon[36260]: 4.1c scrub starts 2024-09-20T19:02:01.045 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:00 smithi083 ceph-mon[36260]: 4.1c scrub ok 2024-09-20T19:02:01.045 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:00 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2217815265' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:02:01.062 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:01.084 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:01.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:01.085 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:01.185 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:01.207 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:01.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:01.208 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:01.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:00 smithi142 ceph-mon[32508]: 4.1c scrub starts 2024-09-20T19:02:01.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:00 smithi142 ceph-mon[32508]: 4.1c scrub ok 2024-09-20T19:02:01.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:00 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2217815265' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:02:01.308 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:02:01.330 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:01.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:01.331 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:02:01.386 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:02:01.386 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:02:01.387 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:02:01.387 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:02:01.387 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:02:01.387 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:02:01.387 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:02:01.387 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:02:01.387 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:02:01.387 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:02:01.387 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:02:01.387 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:02:01.388 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:02:01.388 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:02:01.388 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:02:01.388 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:02:01.431 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:01.452 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:01.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:01.453 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:01.554 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:01.575 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:01.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:01.576 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:01.677 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:01.697 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:01.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:01.698 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:01.799 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:01.820 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:01.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:01.821 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:01.921 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:01.943 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:01.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:01.944 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:02.044 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:02.066 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:02.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:02.067 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:02.167 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:02.188 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:02.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:02.189 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:02.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:01 smithi142 ceph-mon[32508]: pgmap v602: 148 pgs: 1 active+undersized+remapped, 2 active+recovering+undersized+remapped, 145 active+clean; 583 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 83 op/s; 0 B/s, 419 objects/s recovering 2024-09-20T19:02:02.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:01 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:02:02.290 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:02.294 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:01 smithi083 ceph-mon[29421]: pgmap v602: 148 pgs: 1 active+undersized+remapped, 2 active+recovering+undersized+remapped, 145 active+clean; 583 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 83 op/s; 0 B/s, 419 objects/s recovering 2024-09-20T19:02:02.294 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:01 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:02:02.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:01 smithi083 ceph-mon[36260]: pgmap v602: 148 pgs: 1 active+undersized+remapped, 2 active+recovering+undersized+remapped, 145 active+clean; 583 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 83 op/s; 0 B/s, 419 objects/s recovering 2024-09-20T19:02:02.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:01 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:02:02.311 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:02.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:02.312 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:02.413 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:02:02.434 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:02.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:02.435 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:02:02.536 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:02.557 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:02.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:02.558 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:02.659 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:02.684 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:02.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:02.685 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:02.786 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:02.808 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:02.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:02.809 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:02.910 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:02.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:02.933 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:02.933 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:03.033 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:03.056 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:03.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:03.057 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:03.158 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:03.182 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:03.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:03.183 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:03.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:02 smithi142 ceph-mon[32508]: 6.14 scrub starts 2024-09-20T19:02:03.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:02 smithi142 ceph-mon[32508]: 6.14 scrub ok 2024-09-20T19:02:03.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:02 smithi142 ceph-mon[32508]: osdmap e321: 8 total, 8 up, 8 in 2024-09-20T19:02:03.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:02 smithi142 ceph-mon[32508]: 6.1b scrub starts 2024-09-20T19:02:03.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:02 smithi142 ceph-mon[32508]: 6.1b scrub ok 2024-09-20T19:02:03.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:02 smithi142 ceph-mon[32508]: 2.0 scrub starts 2024-09-20T19:02:03.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:02 smithi142 ceph-mon[32508]: 2.0 scrub ok 2024-09-20T19:02:03.284 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:03.294 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:02 smithi083 ceph-mon[29421]: 6.14 scrub starts 2024-09-20T19:02:03.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:02 smithi083 ceph-mon[29421]: 6.14 scrub ok 2024-09-20T19:02:03.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:02 smithi083 ceph-mon[29421]: osdmap e321: 8 total, 8 up, 8 in 2024-09-20T19:02:03.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:02 smithi083 ceph-mon[29421]: 6.1b scrub starts 2024-09-20T19:02:03.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:02 smithi083 ceph-mon[29421]: 6.1b scrub ok 2024-09-20T19:02:03.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:02 smithi083 ceph-mon[29421]: 2.0 scrub starts 2024-09-20T19:02:03.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:02 smithi083 ceph-mon[29421]: 2.0 scrub ok 2024-09-20T19:02:03.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:02 smithi083 ceph-mon[36260]: 6.14 scrub starts 2024-09-20T19:02:03.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:02 smithi083 ceph-mon[36260]: 6.14 scrub ok 2024-09-20T19:02:03.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:02 smithi083 ceph-mon[36260]: osdmap e321: 8 total, 8 up, 8 in 2024-09-20T19:02:03.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:02 smithi083 ceph-mon[36260]: 6.1b scrub starts 2024-09-20T19:02:03.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:02 smithi083 ceph-mon[36260]: 6.1b scrub ok 2024-09-20T19:02:03.296 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:02 smithi083 ceph-mon[36260]: 2.0 scrub starts 2024-09-20T19:02:03.296 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:02 smithi083 ceph-mon[36260]: 2.0 scrub ok 2024-09-20T19:02:03.305 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:03.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:03.306 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:03.407 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:03.432 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:03.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:03.433 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:03.534 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:03.556 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:03.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:03.557 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:03.657 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:03.685 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:03.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:03.686 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:03.787 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:03.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:03.811 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:03.811 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:03.912 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:03.937 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:03.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:03.939 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:04.039 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:04.044 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:03 smithi083 ceph-mon[29421]: 9.5 scrub starts 2024-09-20T19:02:04.044 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:03 smithi083 ceph-mon[29421]: 9.5 scrub ok 2024-09-20T19:02:04.045 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:03 smithi083 ceph-mon[29421]: 5.1c scrub starts 2024-09-20T19:02:04.045 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:03 smithi083 ceph-mon[29421]: 5.1c scrub ok 2024-09-20T19:02:04.045 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:03 smithi083 ceph-mon[29421]: 5.d scrub starts 2024-09-20T19:02:04.045 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:03 smithi083 ceph-mon[29421]: 5.d scrub ok 2024-09-20T19:02:04.045 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:03 smithi083 ceph-mon[29421]: osdmap e322: 8 total, 8 up, 8 in 2024-09-20T19:02:04.045 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:03 smithi083 ceph-mon[29421]: pgmap v605: 148 pgs: 2 peering, 146 active+clean; 583 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 9 op/s; 0 B/s, 635 objects/s recovering 2024-09-20T19:02:04.045 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:03 smithi083 ceph-mon[29421]: 6.15 scrub starts 2024-09-20T19:02:04.046 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:03 smithi083 ceph-mon[29421]: 6.15 scrub ok 2024-09-20T19:02:04.046 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:03 smithi083 ceph-mon[36260]: 9.5 scrub starts 2024-09-20T19:02:04.046 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:03 smithi083 ceph-mon[36260]: 9.5 scrub ok 2024-09-20T19:02:04.046 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:03 smithi083 ceph-mon[36260]: 5.1c scrub starts 2024-09-20T19:02:04.046 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:03 smithi083 ceph-mon[36260]: 5.1c scrub ok 2024-09-20T19:02:04.047 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:03 smithi083 ceph-mon[36260]: 5.d scrub starts 2024-09-20T19:02:04.047 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:03 smithi083 ceph-mon[36260]: 5.d scrub ok 2024-09-20T19:02:04.047 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:03 smithi083 ceph-mon[36260]: osdmap e322: 8 total, 8 up, 8 in 2024-09-20T19:02:04.047 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:03 smithi083 ceph-mon[36260]: pgmap v605: 148 pgs: 2 peering, 146 active+clean; 583 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 9 op/s; 0 B/s, 635 objects/s recovering 2024-09-20T19:02:04.047 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:03 smithi083 ceph-mon[36260]: 6.15 scrub starts 2024-09-20T19:02:04.047 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:03 smithi083 ceph-mon[36260]: 6.15 scrub ok 2024-09-20T19:02:04.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:04.066 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:04.067 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:04.167 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:04.192 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:04.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:04.193 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:04.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:03 smithi142 ceph-mon[32508]: 9.5 scrub starts 2024-09-20T19:02:04.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:03 smithi142 ceph-mon[32508]: 9.5 scrub ok 2024-09-20T19:02:04.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:03 smithi142 ceph-mon[32508]: 5.1c scrub starts 2024-09-20T19:02:04.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:03 smithi142 ceph-mon[32508]: 5.1c scrub ok 2024-09-20T19:02:04.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:03 smithi142 ceph-mon[32508]: 5.d scrub starts 2024-09-20T19:02:04.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:03 smithi142 ceph-mon[32508]: 5.d scrub ok 2024-09-20T19:02:04.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:03 smithi142 ceph-mon[32508]: osdmap e322: 8 total, 8 up, 8 in 2024-09-20T19:02:04.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:03 smithi142 ceph-mon[32508]: pgmap v605: 148 pgs: 2 peering, 146 active+clean; 583 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 9 op/s; 0 B/s, 635 objects/s recovering 2024-09-20T19:02:04.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:03 smithi142 ceph-mon[32508]: 6.15 scrub starts 2024-09-20T19:02:04.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:03 smithi142 ceph-mon[32508]: 6.15 scrub ok 2024-09-20T19:02:04.294 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:04.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:04.321 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:04.321 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:04.422 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:04.447 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:04.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:04.448 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:04.549 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:04.777 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:02:04.779 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd reweight 3 1 2024-09-20T19:02:04.804 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:04.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:04.805 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:04.906 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:02:04.923 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:04 smithi083 ceph-mon[29421]: 6.18 scrub starts 2024-09-20T19:02:04.923 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:04 smithi083 ceph-mon[29421]: 6.18 scrub ok 2024-09-20T19:02:04.923 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:04 smithi083 ceph-mon[29421]: 9.6 scrub starts 2024-09-20T19:02:04.923 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:04 smithi083 ceph-mon[29421]: 9.6 scrub ok 2024-09-20T19:02:04.923 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:04 smithi083 ceph-mon[29421]: 6.6 scrub starts 2024-09-20T19:02:04.923 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:04 smithi083 ceph-mon[29421]: 6.6 scrub ok 2024-09-20T19:02:04.923 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:04 smithi083 ceph-mon[36260]: 6.18 scrub starts 2024-09-20T19:02:04.923 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:04 smithi083 ceph-mon[36260]: 6.18 scrub ok 2024-09-20T19:02:04.924 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:04 smithi083 ceph-mon[36260]: 9.6 scrub starts 2024-09-20T19:02:04.924 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:04 smithi083 ceph-mon[36260]: 9.6 scrub ok 2024-09-20T19:02:04.924 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:04 smithi083 ceph-mon[36260]: 6.6 scrub starts 2024-09-20T19:02:04.924 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:04 smithi083 ceph-mon[36260]: 6.6 scrub ok 2024-09-20T19:02:04.931 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:04.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:04.932 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:02:05.033 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:02:05.057 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:05.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:05.058 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:02:05.159 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:05.180 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:05.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:05.181 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:05.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:04 smithi142 ceph-mon[32508]: 6.18 scrub starts 2024-09-20T19:02:05.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:04 smithi142 ceph-mon[32508]: 6.18 scrub ok 2024-09-20T19:02:05.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:04 smithi142 ceph-mon[32508]: 9.6 scrub starts 2024-09-20T19:02:05.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:04 smithi142 ceph-mon[32508]: 9.6 scrub ok 2024-09-20T19:02:05.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:04 smithi142 ceph-mon[32508]: 6.6 scrub starts 2024-09-20T19:02:05.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:04 smithi142 ceph-mon[32508]: 6.6 scrub ok 2024-09-20T19:02:05.282 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:05.304 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:05.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:05.305 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:05.405 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:05.428 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:05.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:05.430 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:05.530 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:05.551 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:05.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:05.552 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:05.597 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:02:05.653 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:02:05.679 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:05.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:05.680 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:02:05.781 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:02:05.807 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:05.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:05.808 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:02:05.873 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'unique_pool_2' removed 2024-09-20T19:02:05.880 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-20T19:02:05.909 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:02:05.939 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:05.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:05.940 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:02:06.041 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:06.063 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:06.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:06.064 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:06.121 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:05 smithi083 ceph-mon[29421]: pgmap v606: 148 pgs: 2 peering, 146 active+clean; 583 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 706 B/s rd, 0 op/s; 0 B/s, 401 objects/s recovering 2024-09-20T19:02:06.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:05 smithi083 ceph-mon[29421]: 3.d scrub starts 2024-09-20T19:02:06.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:05 smithi083 ceph-mon[29421]: 3.d scrub ok 2024-09-20T19:02:06.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:05 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1506088618' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:02:06.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:05 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:02:06.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:05 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/409571859' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true} : dispatch 2024-09-20T19:02:06.122 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:05 smithi083 ceph-mon[36260]: pgmap v606: 148 pgs: 2 peering, 146 active+clean; 583 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 706 B/s rd, 0 op/s; 0 B/s, 401 objects/s recovering 2024-09-20T19:02:06.122 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:05 smithi083 ceph-mon[36260]: 3.d scrub starts 2024-09-20T19:02:06.123 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:05 smithi083 ceph-mon[36260]: 3.d scrub ok 2024-09-20T19:02:06.123 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:05 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1506088618' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:02:06.123 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:05 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:02:06.123 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:05 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/409571859' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true} : dispatch 2024-09-20T19:02:06.165 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:02:06.186 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:06.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:06.187 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:02:06.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:05 smithi142 ceph-mon[32508]: pgmap v606: 148 pgs: 2 peering, 146 active+clean; 583 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 706 B/s rd, 0 op/s; 0 B/s, 401 objects/s recovering 2024-09-20T19:02:06.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:05 smithi142 ceph-mon[32508]: 3.d scrub starts 2024-09-20T19:02:06.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:05 smithi142 ceph-mon[32508]: 3.d scrub ok 2024-09-20T19:02:06.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:05 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1506088618' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:02:06.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:05 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:02:06.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:05 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/409571859' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true} : dispatch 2024-09-20T19:02:06.289 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:06.310 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:06.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:06.311 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:06.412 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:06.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:06.434 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:06.435 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:06.535 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:02:06.559 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:06.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:06.560 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:02:06.660 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:06.682 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:06.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:06.683 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:06.784 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:06.807 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:06.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:06.808 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:06.908 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:02:06.930 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:06.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:06.931 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:02:07.032 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:07.044 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:06 smithi083 ceph-mon[29421]: 5.3 scrub starts 2024-09-20T19:02:07.044 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:06 smithi083 ceph-mon[29421]: 5.3 scrub ok 2024-09-20T19:02:07.044 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:06 smithi083 ceph-mon[29421]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:02:07.044 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:06 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T19:02:07.044 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:06 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/409571859' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]': finished 2024-09-20T19:02:07.044 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:06 smithi083 ceph-mon[29421]: osdmap e323: 8 total, 8 up, 8 in 2024-09-20T19:02:07.044 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:06 smithi083 ceph-mon[29421]: 5.1d scrub starts 2024-09-20T19:02:07.044 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:06 smithi083 ceph-mon[29421]: 5.1d scrub ok 2024-09-20T19:02:07.045 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:06 smithi083 ceph-mon[36260]: 5.3 scrub starts 2024-09-20T19:02:07.045 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:06 smithi083 ceph-mon[36260]: 5.3 scrub ok 2024-09-20T19:02:07.045 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:06 smithi083 ceph-mon[36260]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:02:07.045 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:06 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T19:02:07.045 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:06 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/409571859' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]': finished 2024-09-20T19:02:07.045 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:06 smithi083 ceph-mon[36260]: osdmap e323: 8 total, 8 up, 8 in 2024-09-20T19:02:07.045 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:06 smithi083 ceph-mon[36260]: 5.1d scrub starts 2024-09-20T19:02:07.045 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:06 smithi083 ceph-mon[36260]: 5.1d scrub ok 2024-09-20T19:02:07.054 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:07.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:07.055 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:07.155 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:07.180 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:07.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:07.181 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:07.264 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:02:07 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:02:07.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:06 smithi142 ceph-mon[32508]: 5.3 scrub starts 2024-09-20T19:02:07.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:06 smithi142 ceph-mon[32508]: 5.3 scrub ok 2024-09-20T19:02:07.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:06 smithi142 ceph-mon[32508]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:02:07.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:06 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T19:02:07.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:06 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/409571859' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]': finished 2024-09-20T19:02:07.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:06 smithi142 ceph-mon[32508]: osdmap e323: 8 total, 8 up, 8 in 2024-09-20T19:02:07.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:06 smithi142 ceph-mon[32508]: 5.1d scrub starts 2024-09-20T19:02:07.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:06 smithi142 ceph-mon[32508]: 5.1d scrub ok 2024-09-20T19:02:07.283 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:07.304 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:07.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:07.305 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:07.406 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:07.428 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:07.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:07.429 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:07.530 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:02:07.552 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:07.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:07.553 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:02:07.562 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-20T19:02:07.563 INFO:tasks.radosbench:Beginning radosbench... 2024-09-20T19:02:07.563 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_3 2024-09-20T19:02:07.563 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd pool create unique_pool_3 16 2024-09-20T19:02:07.569 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:02:07.654 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:02:07.675 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:07.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:07.676 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:02:07.777 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:07.798 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:07.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:07.800 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:07.900 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:07.939 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:07.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:07.940 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:08.040 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:08.062 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:08.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:08.063 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:08.164 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:08.186 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:08.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:08.187 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:08.198 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:07 smithi083 ceph-mon[29421]: pgmap v608: 132 pgs: 132 active+clean; 583 KiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-20T19:02:08.199 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:07 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:02:08.199 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:07 smithi083 ceph-mon[36260]: pgmap v608: 132 pgs: 132 active+clean; 583 KiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-20T19:02:08.199 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:07 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:02:08.199 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T19:02:08.199 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-20T19:02:08.199 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-20T19:02:08.199 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-20T19:02:08.199 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T19:02:08.204 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T19:02:08.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:07 smithi142 ceph-mon[32508]: pgmap v608: 132 pgs: 132 active+clean; 583 KiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-20T19:02:08.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:07 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:02:08.288 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:08.309 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:08.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:08.310 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:08.410 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:08.432 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:08.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:08.433 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:08.534 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:08.556 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:08.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:08.557 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:08.658 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:08.680 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:08.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:08.681 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:08.782 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:02:08.808 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:08.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:08.809 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:02:08.856 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_historic_ops 2024-09-20T19:02:08.909 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:08.931 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:08.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:08.932 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:09.032 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:02:09.055 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:09.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:09.056 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:02:09.157 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:09.180 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:09.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:09.181 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:09.187 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T19:02:09.281 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:09.303 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:09.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:09.304 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:09.306 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:02:09.353 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:02:09.405 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:02:09.427 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:09.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:09.428 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:02:09.529 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:02:09.551 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:09.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:09.551 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:02:09.652 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:09.673 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:09.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:09.674 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:09.775 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:09.796 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:09.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:09.797 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:09.898 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:09.920 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:09.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:09.922 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:10.027 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:10.049 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:10.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:10.050 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:10.135 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.3 to 1 (10000) 2024-09-20T19:02:10.151 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:10.175 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:10.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:10.176 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:10.220 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:02:10.221 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:02:10.238 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:02:10.238 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:02:10.254 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:02:10.255 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:02:10.255 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:10 smithi083 ceph-mon[29421]: pgmap v609: 132 pgs: 132 active+clean; 583 KiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 566 B/s rd, 0 op/s 2024-09-20T19:02:10.255 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:10 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/96281879' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-20T19:02:10.255 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:10 smithi083 ceph-mon[36260]: pgmap v609: 132 pgs: 132 active+clean; 583 KiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 566 B/s rd, 0 op/s 2024-09-20T19:02:10.255 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:10 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/96281879' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-20T19:02:10.270 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:02:10.270 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:02:10.277 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:10.287 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:02:10.287 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:02:10.298 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:10.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:10.299 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:10.304 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:02:10.304 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:02:10.321 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:02:10.321 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:02:10.337 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:02:10.337 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:02:10.399 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:10.421 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:10.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:10.422 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:10.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:10 smithi142 ceph-mon[32508]: pgmap v609: 132 pgs: 132 active+clean; 583 KiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 566 B/s rd, 0 op/s 2024-09-20T19:02:10.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:10 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/96281879' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-20T19:02:10.523 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:02:10.544 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:10.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:10.545 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:02:10.646 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:10.670 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:10.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:10.671 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:10.772 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:10.793 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:10.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:10.794 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:10.826 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'unique_pool_3' created 2024-09-20T19:02:10.844 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:02:10 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:02:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:02:10.896 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:10.916 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:10.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:10.917 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:11.018 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:02:11.031 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd reweight 4 1 2024-09-20T19:02:11.040 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:11.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:11.041 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:02:11.142 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:11.164 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:11.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:11.165 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:11.265 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:02:11.287 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:11.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:11.288 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:02:11.389 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:11.410 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:11.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:11.411 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:11.512 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:11.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:11 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/96281879' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-20T19:02:11.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:11 smithi142 ceph-mon[32508]: osdmap e324: 8 total, 8 up, 8 in 2024-09-20T19:02:11.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:11 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3704279694' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16} : dispatch 2024-09-20T19:02:11.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:11 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16} : dispatch 2024-09-20T19:02:11.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:11 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3014913190' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:02:11.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:11 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]': finished 2024-09-20T19:02:11.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:11 smithi142 ceph-mon[32508]: osdmap e325: 8 total, 8 up, 8 in 2024-09-20T19:02:11.534 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:11.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:11.535 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:11.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:11 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/96281879' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-20T19:02:11.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:11 smithi083 ceph-mon[29421]: osdmap e324: 8 total, 8 up, 8 in 2024-09-20T19:02:11.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:11 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3704279694' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16} : dispatch 2024-09-20T19:02:11.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:11 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16} : dispatch 2024-09-20T19:02:11.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:11 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3014913190' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:02:11.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:11 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]': finished 2024-09-20T19:02:11.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:11 smithi083 ceph-mon[29421]: osdmap e325: 8 total, 8 up, 8 in 2024-09-20T19:02:11.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:11 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/96281879' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-20T19:02:11.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:11 smithi083 ceph-mon[36260]: osdmap e324: 8 total, 8 up, 8 in 2024-09-20T19:02:11.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:11 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3704279694' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16} : dispatch 2024-09-20T19:02:11.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:11 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16} : dispatch 2024-09-20T19:02:11.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:11 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3014913190' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:02:11.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:11 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]': finished 2024-09-20T19:02:11.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:11 smithi083 ceph-mon[36260]: osdmap e325: 8 total, 8 up, 8 in 2024-09-20T19:02:11.636 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:11.658 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:11.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:11.659 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:11.760 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:11.781 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:11.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:11.782 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:11.882 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:02:11.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:11.909 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:11.909 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:02:11.992 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:02:12.009 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:02:12.030 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:12.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:12.031 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:02:12.132 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:12.154 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:12.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:12.155 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:12.256 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:12.263 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:12 smithi083 ceph-mon[29421]: pgmap v612: 148 pgs: 6 creating+peering, 10 unknown, 132 active+clean; 583 KiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-20T19:02:12.263 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:12 smithi083 ceph-mon[29421]: osdmap e326: 8 total, 8 up, 8 in 2024-09-20T19:02:12.264 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:12 smithi083 ceph-mon[29421]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-20T19:02:12.264 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:12 smithi083 ceph-mon[36260]: pgmap v612: 148 pgs: 6 creating+peering, 10 unknown, 132 active+clean; 583 KiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-20T19:02:12.264 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:12 smithi083 ceph-mon[36260]: osdmap e326: 8 total, 8 up, 8 in 2024-09-20T19:02:12.264 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:12 smithi083 ceph-mon[36260]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-20T19:02:12.277 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:12.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:12.278 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:12.379 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:02:12.400 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:12.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:12.401 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:02:12.498 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T19:02:12.498 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:02:12.502 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:12.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:12 smithi142 ceph-mon[32508]: pgmap v612: 148 pgs: 6 creating+peering, 10 unknown, 132 active+clean; 583 KiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-20T19:02:12.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:12 smithi142 ceph-mon[32508]: osdmap e326: 8 total, 8 up, 8 in 2024-09-20T19:02:12.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:12 smithi142 ceph-mon[32508]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-20T19:02:12.524 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:12.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:12.525 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:12.625 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:02:12.648 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:12.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:12.649 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:02:12.749 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:02:12.771 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:12.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:12.772 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:02:12.873 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:12.896 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:12.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:12.897 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:12.998 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:13.020 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:13.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:13.021 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:13.100 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:02:13.122 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:02:13.128 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd pool application enable unique_pool_3 rados --yes-i-really-mean-it || true 2024-09-20T19:02:13.128 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:02:13.145 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:13.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:13.146 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:02:13.246 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:13.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:13.268 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:13.269 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:13.369 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:13.394 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:13.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:13.395 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:13.496 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:13.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:13.518 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:13.518 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:13.619 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:13.643 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:13.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:13.644 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:13.745 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:13.766 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:13.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:13.767 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:13.868 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:13.889 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:13.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:13.890 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:13.970 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T19:02:13.991 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:13.992 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:13 smithi083 ceph-mon[29421]: osdmap e327: 8 total, 8 up, 8 in 2024-09-20T19:02:13.992 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:13 smithi083 ceph-mon[29421]: pgmap v615: 148 pgs: 11 creating+peering, 2 unknown, 135 active+clean; 583 KiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T19:02:13.993 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:13 smithi083 ceph-mon[36260]: osdmap e327: 8 total, 8 up, 8 in 2024-09-20T19:02:13.993 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:13 smithi083 ceph-mon[36260]: pgmap v615: 148 pgs: 11 creating+peering, 2 unknown, 135 active+clean; 583 KiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T19:02:14.015 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:14.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:14.016 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:14.117 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:02:14.140 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:14.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:14.141 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:02:14.241 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:14.263 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:14.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:14.264 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:14.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:13 smithi142 ceph-mon[32508]: osdmap e327: 8 total, 8 up, 8 in 2024-09-20T19:02:14.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:13 smithi142 ceph-mon[32508]: pgmap v615: 148 pgs: 11 creating+peering, 2 unknown, 135 active+clean; 583 KiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T19:02:14.365 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:14.374 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-20T19:02:14.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:14.388 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:14.388 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:14.489 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:14.513 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:14.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:14.514 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:14.614 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:14.619 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:02:14.641 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:14.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:14.643 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:14.743 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:14.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:14.768 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:14.768 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:14.849 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.4 to 1 (10000) 2024-09-20T19:02:14.849 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-20T19:02:14.870 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:14.892 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:14.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:14.893 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:14.994 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:02:15.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:15.021 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:15.022 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:02:15.044 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:14 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1718704811' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:02:15.045 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:14 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:02:15.045 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:14 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2057658307' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-20T19:02:15.045 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:14 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-20T19:02:15.045 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:14 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1718704811' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:02:15.045 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:14 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:02:15.046 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:14 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2057658307' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-20T19:02:15.046 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:14 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-20T19:02:15.122 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:02:15.148 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:15.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:15.149 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:02:15.250 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:15.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:14 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1718704811' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:02:15.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:14 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:02:15.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:14 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2057658307' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-20T19:02:15.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:14 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-20T19:02:15.271 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:15.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:15.272 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:15.372 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:02:15.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:15.399 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:15.400 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:02:15.500 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:15.525 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:15.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:15.526 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:15.617 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd reweight 5 1 2024-09-20T19:02:15.627 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:02:15.652 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:15.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:15.653 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:02:15.754 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:02:15.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:15.781 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:15.781 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:02:15.837 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:02:15.841 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:02:15.882 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:15.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:15.903 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:15.903 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:16.004 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:16.031 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:16.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:16.032 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:16.132 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:16.159 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:16.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:16.160 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:16.261 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:16.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:15 smithi142 ceph-mon[32508]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:02:16.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:15 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T19:02:16.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:15 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-20T19:02:16.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:15 smithi142 ceph-mon[32508]: osdmap e328: 8 total, 8 up, 8 in 2024-09-20T19:02:16.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:15 smithi142 ceph-mon[32508]: pgmap v617: 148 pgs: 11 creating+peering, 2 unknown, 135 active+clean; 583 KiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T19:02:16.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:16.284 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:16.284 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:16.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:15 smithi083 ceph-mon[36260]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:02:16.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:15 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T19:02:16.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:15 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-20T19:02:16.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:15 smithi083 ceph-mon[36260]: osdmap e328: 8 total, 8 up, 8 in 2024-09-20T19:02:16.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:15 smithi083 ceph-mon[36260]: pgmap v617: 148 pgs: 11 creating+peering, 2 unknown, 135 active+clean; 583 KiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T19:02:16.294 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:15 smithi083 ceph-mon[29421]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:02:16.294 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:15 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T19:02:16.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:15 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-20T19:02:16.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:15 smithi083 ceph-mon[29421]: osdmap e328: 8 total, 8 up, 8 in 2024-09-20T19:02:16.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:15 smithi083 ceph-mon[29421]: pgmap v617: 148 pgs: 11 creating+peering, 2 unknown, 135 active+clean; 583 KiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T19:02:16.384 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:16.411 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:16.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:16.412 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:16.513 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:16.533 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:16.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:16.534 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:16.635 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:16.656 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:16.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:16.657 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:16.758 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:02:16.784 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:16.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:16.785 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:02:16.886 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:16.906 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:16.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:16.907 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:17.008 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:02:17.033 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:17.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:17.034 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:02:17.120 INFO:teuthology.orchestra.run.smithi083.stderr:enabled application 'rados' on pool 'unique_pool_3' 2024-09-20T19:02:17.135 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:17.157 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:17.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:17.158 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:17.259 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:17.285 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:17.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:17.286 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:17.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:17 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:02:17.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:17 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:02:17.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:17 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3476326918' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-20T19:02:17.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:17 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-20T19:02:17.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:17 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:02:17.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:17 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:02:17.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:17 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3476326918' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-20T19:02:17.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:17 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-20T19:02:17.306 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T19:02:17.306 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:02:17.387 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:17.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:17.409 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:17.410 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:17.510 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:17.514 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:02:17 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:02:17.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:17 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:02:17.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:17 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:02:17.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:17 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3476326918' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-20T19:02:17.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:17 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-20T19:02:17.537 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:17.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:17.538 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:17.639 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:17.660 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:17.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:17.661 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:17.762 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:17.783 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:17.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:17.784 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:17.885 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:17.906 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:17.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:17.907 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:18.001 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_blocked_ops 2024-09-20T19:02:18.008 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:18.065 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:18.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:18.066 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:18.166 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:18.188 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:18.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:18.190 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:18.290 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:18.294 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:18 smithi083 ceph-mon[29421]: pgmap v618: 148 pgs: 148 active+clean; 583 KiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-20T19:02:18.294 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:18 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:02:18.294 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:18 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-20T19:02:18.294 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:18 smithi083 ceph-mon[29421]: osdmap e329: 8 total, 8 up, 8 in 2024-09-20T19:02:18.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:18 smithi083 ceph-mon[36260]: pgmap v618: 148 pgs: 148 active+clean; 583 KiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-20T19:02:18.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:18 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:02:18.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:18 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-20T19:02:18.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:18 smithi083 ceph-mon[36260]: osdmap e329: 8 total, 8 up, 8 in 2024-09-20T19:02:18.312 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:18.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:18.313 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:18.414 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:18.442 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:18.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:18.443 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:18.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:18 smithi142 ceph-mon[32508]: pgmap v618: 148 pgs: 148 active+clean; 583 KiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-20T19:02:18.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:18 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:02:18.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:18 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-20T19:02:18.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:18 smithi142 ceph-mon[32508]: osdmap e329: 8 total, 8 up, 8 in 2024-09-20T19:02:18.544 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:18.565 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:18.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:18.566 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:18.650 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T19:02:18.667 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:02:18.694 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:18.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:18.695 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:02:18.796 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:18.821 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:18.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:18.822 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:18.923 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:02:18.946 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:18.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:18.947 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:02:18.971 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:02:18.971 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:02:18.971 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:02:18.971 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:02:18.991 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:02:18.991 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:02:19.014 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:02:19.015 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:02:19.015 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:02:19.020 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:02:19.034 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:02:19.035 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:02:19.035 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:02:19.048 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:19.055 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:02:19.055 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:02:19.071 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:19.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:19.072 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:19.072 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:02:19.072 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:02:19.089 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:02:19.090 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:02:19.172 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:19.196 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:19.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:19.197 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:19.265 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:19 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/4026153935' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:02:19.265 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:19 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/4026153935' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:02:19.298 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:19.319 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:19.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:19.320 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:19.421 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:19.446 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:19.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:19.447 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:19.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:19 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/4026153935' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:02:19.549 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:19.571 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:19.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:19.572 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:19.673 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:02:19.680 DEBUG:teuthology.orchestra.run.smithi083:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_3 bench 90 write' 2024-09-20T19:02:19.685 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-20T19:02:19.686 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-20T19:02:19.700 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:19.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:19.701 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:02:19.802 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:19.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:19.829 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:19.829 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:19.865 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:hints = 1 2024-09-20T19:02:19.865 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-20T19:02:19.865 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Object prefix: benchmark_data_smithi083_148100 2024-09-20T19:02:19.865 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-20T19:02:19.865 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 0 0 0 0 0 0 - 0 2024-09-20T19:02:19.930 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:19.971 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:19.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:19.972 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:20.072 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:20.105 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:20.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:20.107 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:20.208 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:02:20.242 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:20.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:20.243 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:02:20.344 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:20.373 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:20.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:20.374 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:20.377 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:20 smithi083 ceph-mon[29421]: pgmap v620: 148 pgs: 148 active+clean; 583 KiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 819 B/s rd, 0 op/s 2024-09-20T19:02:20.377 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:20 smithi083 ceph-mon[29421]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-20T19:02:20.377 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:20 smithi083 ceph-mon[36260]: pgmap v620: 148 pgs: 148 active+clean; 583 KiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 819 B/s rd, 0 op/s 2024-09-20T19:02:20.377 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:20 smithi083 ceph-mon[36260]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-20T19:02:20.475 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:20.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:20 smithi142 ceph-mon[32508]: pgmap v620: 148 pgs: 148 active+clean; 583 KiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 819 B/s rd, 0 op/s 2024-09-20T19:02:20.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:20 smithi142 ceph-mon[32508]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-20T19:02:20.516 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:20.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:20.518 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:20.577 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T19:02:20.577 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-20T19:02:20.577 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-20T19:02:20.578 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-20T19:02:20.578 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T19:02:20.618 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:20.654 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:20.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:20.655 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:20.756 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:02:20.790 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:20.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:20.791 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:02:20.831 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.5 to 1 (10000) 2024-09-20T19:02:20.865 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 1 16 3440 3424 213.987 214 0.00748679 0.0046225 2024-09-20T19:02:20.892 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:20.927 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:20.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:20.928 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:21.029 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:21.044 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:02:20 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:02:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:02:21.066 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:21.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:21.067 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:21.168 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:21.204 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:21.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:21.205 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:21.306 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:21.341 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:21.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:21.342 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:21.410 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:21 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/4045736533' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-20T19:02:21.410 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:21 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-20T19:02:21.410 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:21 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-20T19:02:21.410 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:21 smithi083 ceph-mon[29421]: osdmap e330: 8 total, 8 up, 8 in 2024-09-20T19:02:21.410 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:21 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3443231953' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:02:21.410 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:21 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:02:21.411 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:21 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/4045736533' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-20T19:02:21.411 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:21 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-20T19:02:21.411 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:21 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-20T19:02:21.411 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:21 smithi083 ceph-mon[36260]: osdmap e330: 8 total, 8 up, 8 in 2024-09-20T19:02:21.411 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:21 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3443231953' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:02:21.411 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:21 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:02:21.443 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:21.470 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_historic_ops 2024-09-20T19:02:21.481 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:21.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:21.482 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:21.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:21 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/4045736533' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-20T19:02:21.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:21 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-20T19:02:21.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:21 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-20T19:02:21.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:21 smithi142 ceph-mon[32508]: osdmap e330: 8 total, 8 up, 8 in 2024-09-20T19:02:21.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:21 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3443231953' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:02:21.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:21 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:02:21.583 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:21.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:21.623 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:21.624 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:21.724 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:21.759 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:21.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:21.760 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:21.860 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-20T19:02:21.861 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:21.865 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 2 15 6915 6900 215.596 217.25 0.00383554 0.00462632 2024-09-20T19:02:21.909 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:21.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:21.910 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:22.011 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:22.046 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:22.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:22.047 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:22.148 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:22.152 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd reweight 7 1 2024-09-20T19:02:22.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:22.182 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:22.182 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:22.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:22 smithi142 ceph-mon[32508]: pgmap v622: 148 pgs: 148 active+clean; 106 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 820 B/s rd, 17 MiB/s wr, 271 op/s 2024-09-20T19:02:22.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:22 smithi142 ceph-mon[32508]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:02:22.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:22 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T19:02:22.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:22 smithi142 ceph-mon[32508]: osdmap e331: 8 total, 8 up, 8 in 2024-09-20T19:02:22.283 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:22.321 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:22.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:22.322 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:22.423 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:22.430 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:22 smithi083 ceph-mon[36260]: pgmap v622: 148 pgs: 148 active+clean; 106 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 820 B/s rd, 17 MiB/s wr, 271 op/s 2024-09-20T19:02:22.430 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:22 smithi083 ceph-mon[36260]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:02:22.431 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:22 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T19:02:22.431 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:22 smithi083 ceph-mon[36260]: osdmap e331: 8 total, 8 up, 8 in 2024-09-20T19:02:22.431 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:22 smithi083 ceph-mon[29421]: pgmap v622: 148 pgs: 148 active+clean; 106 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 820 B/s rd, 17 MiB/s wr, 271 op/s 2024-09-20T19:02:22.431 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:22 smithi083 ceph-mon[29421]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:02:22.431 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:22 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T19:02:22.431 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:22 smithi083 ceph-mon[29421]: osdmap e331: 8 total, 8 up, 8 in 2024-09-20T19:02:22.455 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:22.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:22.456 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:22.557 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:02:22.590 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:22.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:22.591 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:02:22.692 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:22.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:22.727 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:22.728 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:22.829 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:22.865 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:22.866 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 3 16 10178 10162 211.683 203.875 0.00324795 0.00471647 2024-09-20T19:02:22.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:22.866 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:22.967 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:23.004 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:23.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:23.005 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:23.106 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:23.140 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:23.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:23.141 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:23.242 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:23.273 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:23.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:23.274 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:23.323 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:02:23.362 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:02:23.375 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:23.404 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:23.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:23.405 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:23.506 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:23.541 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:23.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:23.542 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:23.643 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:23.676 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:23.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:23.677 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:23.778 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:23.813 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:23.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:23.814 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:23.865 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 4 15 13210 13195 206.149 189.562 0.00484789 0.00484539 2024-09-20T19:02:23.915 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:23.948 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:23.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:23.949 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:24.049 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:24.088 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:24.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:24.089 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:24.176 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T19:02:24.176 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-20T19:02:24.176 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-20T19:02:24.177 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-20T19:02:24.177 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.177 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157429 8.7 8:ee8a5b93:::benchmark_data_smithi083_97422_object41338:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e212)", 2024-09-20T19:02:24.177 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.048178+0000", 2024-09-20T19:02:24.177 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 325.118257535, 2024-09-20T19:02:24.177 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.034527355, 2024-09-20T19:02:24.177 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:24.177 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:24.177 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-20T19:02:24.177 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:02:24.177 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T19:02:24.177 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T19:02:24.178 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157429 2024-09-20T19:02:24.178 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.178 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:24.178 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.178 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:24.178 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.048178+0000", 2024-09-20T19:02:24.178 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:24.178 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.178 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.178 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:24.178 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.048178+0000", 2024-09-20T19:02:24.178 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:24.179 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.179 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.179 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:24.179 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.048180+0000", 2024-09-20T19:02:24.179 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.618e-06 2024-09-20T19:02:24.179 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.179 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.179 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:24.179 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.048187+0000", 2024-09-20T19:02:24.179 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8240000000000002e-06 2024-09-20T19:02:24.179 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.179 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.179 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:24.180 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.048205+0000", 2024-09-20T19:02:24.180 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8717000000000001e-05 2024-09-20T19:02:24.180 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.180 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.180 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:24.180 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.048215+0000", 2024-09-20T19:02:24.180 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.022e-05 2024-09-20T19:02:24.180 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.180 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.180 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.180 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.048294+0000", 2024-09-20T19:02:24.180 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8088000000000002e-05 2024-09-20T19:02:24.180 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.181 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.181 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:24.181 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.048409+0000", 2024-09-20T19:02:24.181 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115146 2024-09-20T19:02:24.181 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.181 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.181 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3,6", 2024-09-20T19:02:24.181 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.048517+0000", 2024-09-20T19:02:24.181 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010874099999999999 2024-09-20T19:02:24.181 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.181 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.181 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:24.182 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.059282+0000", 2024-09-20T19:02:24.182 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010764078 2024-09-20T19:02:24.182 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.182 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.182 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:02:24.182 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.059297+0000", 2024-09-20T19:02:24.182 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5773e-05 2024-09-20T19:02:24.182 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.182 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.182 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:24.182 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.059343+0000", 2024-09-20T19:02:24.182 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5816000000000003e-05 2024-09-20T19:02:24.183 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.183 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.183 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.183 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.070249+0000", 2024-09-20T19:02:24.183 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010906381999999999 2024-09-20T19:02:24.183 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.183 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.183 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:02:24.183 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.070260+0000", 2024-09-20T19:02:24.183 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0404000000000001e-05 2024-09-20T19:02:24.183 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.186 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.186 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.186 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.082592+0000", 2024-09-20T19:02:24.186 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0123322640000001 2024-09-20T19:02:24.186 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.186 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.186 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:24.186 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.082705+0000", 2024-09-20T19:02:24.186 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011328399999999999 2024-09-20T19:02:24.186 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:24.186 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:24.187 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:24.187 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.187 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.187 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157435 8.7 8:e8101dbf:::benchmark_data_smithi083_97422_object41344:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e212)", 2024-09-20T19:02:24.187 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.053885+0000", 2024-09-20T19:02:24.187 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 325.11255030500001, 2024-09-20T19:02:24.187 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0294991419999999, 2024-09-20T19:02:24.187 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:24.187 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:24.187 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-20T19:02:24.187 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:02:24.187 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T19:02:24.187 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T19:02:24.187 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157435 2024-09-20T19:02:24.188 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.188 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:24.188 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.188 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:24.188 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053885+0000", 2024-09-20T19:02:24.188 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:24.188 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.188 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.188 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:24.188 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053885+0000", 2024-09-20T19:02:24.188 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:24.188 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.188 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.188 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:24.189 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053887+0000", 2024-09-20T19:02:24.189 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.68e-06 2024-09-20T19:02:24.189 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.189 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.189 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:24.189 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053894+0000", 2024-09-20T19:02:24.189 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5939999999999999e-06 2024-09-20T19:02:24.189 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.189 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.189 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:24.189 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053907+0000", 2024-09-20T19:02:24.189 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3599e-05 2024-09-20T19:02:24.190 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.190 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.190 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:24.190 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053916+0000", 2024-09-20T19:02:24.190 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2029999999999998e-06 2024-09-20T19:02:24.190 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.190 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.190 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.190 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.054004+0000", 2024-09-20T19:02:24.190 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8026000000000001e-05 2024-09-20T19:02:24.191 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.191 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.191 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:24.191 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.054123+0000", 2024-09-20T19:02:24.191 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000119046 2024-09-20T19:02:24.191 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.191 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.191 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3,6", 2024-09-20T19:02:24.191 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.056711+0000", 2024-09-20T19:02:24.191 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0025878049999999999 2024-09-20T19:02:24.191 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.191 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.191 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:02:24.191 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.059354+0000", 2024-09-20T19:02:24.192 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0026431969999999999 2024-09-20T19:02:24.192 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.192 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.192 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.192 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.070287+0000", 2024-09-20T19:02:24.192 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010932261 2024-09-20T19:02:24.192 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.192 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.192 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:02:24.192 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.070294+0000", 2024-09-20T19:02:24.192 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0029999999999997e-06 2024-09-20T19:02:24.192 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.192 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.192 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.193 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.083330+0000", 2024-09-20T19:02:24.193 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.013036592 2024-09-20T19:02:24.193 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.193 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.193 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:24.193 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.083384+0000", 2024-09-20T19:02:24.193 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4135999999999998e-05 2024-09-20T19:02:24.193 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:24.193 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:24.193 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:24.193 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.193 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.193 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157436 8.7 8:e2843ea0:::benchmark_data_smithi083_97422_object41345:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e212)", 2024-09-20T19:02:24.193 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.053939+0000", 2024-09-20T19:02:24.194 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 325.11249677400002, 2024-09-20T19:02:24.194 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:24.195 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.029517123, 2024-09-20T19:02:24.195 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:24.195 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:24.195 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-20T19:02:24.195 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:02:24.195 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T19:02:24.195 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T19:02:24.195 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157436 2024-09-20T19:02:24.195 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.195 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:24.196 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.196 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:24.196 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053939+0000", 2024-09-20T19:02:24.196 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:24.196 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.196 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.196 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:24.196 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053939+0000", 2024-09-20T19:02:24.196 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:24.196 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.196 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.196 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:24.196 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053939+0000", 2024-09-20T19:02:24.196 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9100000000000004e-07 2024-09-20T19:02:24.197 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.197 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.197 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:24.197 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053942+0000", 2024-09-20T19:02:24.197 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7980000000000001e-06 2024-09-20T19:02:24.197 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.197 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.197 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:24.197 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053949+0000", 2024-09-20T19:02:24.197 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8580000000000002e-06 2024-09-20T19:02:24.197 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.197 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.197 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:24.197 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053955+0000", 2024-09-20T19:02:24.198 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1449999999999996e-06 2024-09-20T19:02:24.198 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.198 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.198 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.198 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.060057+0000", 2024-09-20T19:02:24.198 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0061021729999999998 2024-09-20T19:02:24.198 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.198 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.198 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:24.198 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.060177+0000", 2024-09-20T19:02:24.198 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000120013 2024-09-20T19:02:24.198 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.198 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.199 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3,6", 2024-09-20T19:02:24.199 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.060294+0000", 2024-09-20T19:02:24.199 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000117056 2024-09-20T19:02:24.199 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.199 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.199 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.199 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.070317+0000", 2024-09-20T19:02:24.199 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010022778 2024-09-20T19:02:24.199 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.199 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.199 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:02:24.199 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.070324+0000", 2024-09-20T19:02:24.199 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4300000000000003e-06 2024-09-20T19:02:24.200 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.200 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.200 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.200 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.083413+0000", 2024-09-20T19:02:24.200 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0130890960000001 2024-09-20T19:02:24.200 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.200 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.200 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:24.200 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.083456+0000", 2024-09-20T19:02:24.200 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3185000000000001e-05 2024-09-20T19:02:24.200 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:24.200 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:24.200 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:24.200 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.200 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.201 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157449 8.7 8:e0d43d9e:::benchmark_data_smithi083_97422_object41358:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e213)", 2024-09-20T19:02:24.201 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.075612+0000", 2024-09-20T19:02:24.201 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 325.090823925, 2024-09-20T19:02:24.201 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.016308008, 2024-09-20T19:02:24.201 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:24.201 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:24.201 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:02:24.201 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:02:24.201 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T19:02:24.201 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T19:02:24.201 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157449 2024-09-20T19:02:24.201 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.201 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:24.202 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.202 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:24.202 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.075612+0000", 2024-09-20T19:02:24.202 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:24.202 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.202 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.202 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:24.202 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.075612+0000", 2024-09-20T19:02:24.202 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:24.202 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.202 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.202 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:24.202 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.075613+0000", 2024-09-20T19:02:24.202 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1570000000000001e-06 2024-09-20T19:02:24.203 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.203 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.203 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:24.203 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.075618+0000", 2024-09-20T19:02:24.203 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7899999999999999e-06 2024-09-20T19:02:24.203 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.203 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.203 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:24.203 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.075627+0000", 2024-09-20T19:02:24.203 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2569999999999996e-06 2024-09-20T19:02:24.203 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.203 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.203 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:24.203 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.075662+0000", 2024-09-20T19:02:24.204 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5439999999999999e-05 2024-09-20T19:02:24.204 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.204 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.204 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.204 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.078479+0000", 2024-09-20T19:02:24.204 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0028168910000000002 2024-09-20T19:02:24.204 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.204 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.204 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:02:24.204 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.078488+0000", 2024-09-20T19:02:24.204 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4069999999999993e-06 2024-09-20T19:02:24.204 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.204 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.204 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.205 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.083476+0000", 2024-09-20T19:02:24.205 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.004987893 2024-09-20T19:02:24.205 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.205 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.205 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:24.205 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.083567+0000", 2024-09-20T19:02:24.205 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1598e-05 2024-09-20T19:02:24.205 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.205 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.205 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3", 2024-09-20T19:02:24.205 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.083675+0000", 2024-09-20T19:02:24.205 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000107701 2024-09-20T19:02:24.205 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.205 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.206 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:24.206 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.091482+0000", 2024-09-20T19:02:24.206 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0078073639999999998 2024-09-20T19:02:24.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.206 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.206 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:02:24.206 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.091668+0000", 2024-09-20T19:02:24.206 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018606100000000001 2024-09-20T19:02:24.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.206 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.206 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:24.206 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.091718+0000", 2024-09-20T19:02:24.206 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9830000000000002e-05 2024-09-20T19:02:24.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.207 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.207 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:02:24.207 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.091878+0000", 2024-09-20T19:02:24.207 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000160166 2024-09-20T19:02:24.207 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.207 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.207 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:24.207 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.091920+0000", 2024-09-20T19:02:24.207 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1452999999999998e-05 2024-09-20T19:02:24.207 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:24.207 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:24.207 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:24.207 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.207 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.208 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157451 8.7 8:ea555b5d:::benchmark_data_smithi083_97422_object41360:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e213)", 2024-09-20T19:02:24.208 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.080725+0000", 2024-09-20T19:02:24.208 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 325.08571095799999, 2024-09-20T19:02:24.208 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0181950740000001, 2024-09-20T19:02:24.208 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:24.208 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:24.208 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:02:24.208 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:02:24.208 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T19:02:24.208 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T19:02:24.208 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157451 2024-09-20T19:02:24.208 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.208 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:24.208 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.209 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:24.209 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.080725+0000", 2024-09-20T19:02:24.209 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:24.209 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.209 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.209 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:24.209 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.080725+0000", 2024-09-20T19:02:24.209 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:24.209 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.209 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.209 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:24.209 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.080726+0000", 2024-09-20T19:02:24.209 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3459999999999999e-06 2024-09-20T19:02:24.209 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.210 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.210 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:24.210 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.080731+0000", 2024-09-20T19:02:24.210 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3619999999999997e-06 2024-09-20T19:02:24.210 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.210 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.210 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:24.210 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.080742+0000", 2024-09-20T19:02:24.210 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0302000000000001e-05 2024-09-20T19:02:24.210 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.210 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.210 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:24.210 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.080757+0000", 2024-09-20T19:02:24.210 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4955e-05 2024-09-20T19:02:24.211 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.211 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.211 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.211 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.080819+0000", 2024-09-20T19:02:24.211 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2310999999999993e-05 2024-09-20T19:02:24.211 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.211 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.211 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:02:24.211 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.080826+0000", 2024-09-20T19:02:24.211 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1069999999999999e-06 2024-09-20T19:02:24.211 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.211 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.211 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.211 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.085855+0000", 2024-09-20T19:02:24.212 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0050288679999999 2024-09-20T19:02:24.212 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.212 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.212 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:24.212 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.085947+0000", 2024-09-20T19:02:24.212 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1911999999999997e-05 2024-09-20T19:02:24.212 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.212 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.212 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3", 2024-09-20T19:02:24.212 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.086042+0000", 2024-09-20T19:02:24.212 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5234000000000003e-05 2024-09-20T19:02:24.212 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.212 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.213 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:02:24.213 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.091673+0000", 2024-09-20T19:02:24.213 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0056306079999999996 2024-09-20T19:02:24.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.213 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.213 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:24.213 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.098793+0000", 2024-09-20T19:02:24.213 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.007120637 2024-09-20T19:02:24.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.213 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.213 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:24.213 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.098828+0000", 2024-09-20T19:02:24.213 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4405e-05 2024-09-20T19:02:24.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.214 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.214 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:02:24.214 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.098882+0000", 2024-09-20T19:02:24.214 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4481000000000002e-05 2024-09-20T19:02:24.214 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.214 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.214 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:24.214 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.098920+0000", 2024-09-20T19:02:24.214 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7546000000000003e-05 2024-09-20T19:02:24.214 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:24.214 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:24.214 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:24.215 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.215 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.215 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157481 8.7 8:ebfa0664:::benchmark_data_smithi083_97422_object41390:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e213)", 2024-09-20T19:02:24.215 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.116650+0000", 2024-09-20T19:02:24.215 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 325.049785445, 2024-09-20T19:02:24.215 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.98244276600000002, 2024-09-20T19:02:24.215 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:24.215 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:24.215 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:02:24.215 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:02:24.215 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T19:02:24.215 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T19:02:24.215 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157481 2024-09-20T19:02:24.215 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.216 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:24.216 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.216 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:24.216 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.116650+0000", 2024-09-20T19:02:24.216 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:24.216 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.216 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.216 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:24.216 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.116650+0000", 2024-09-20T19:02:24.216 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:24.216 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.216 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.216 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:24.216 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.116652+0000", 2024-09-20T19:02:24.216 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5680000000000001e-06 2024-09-20T19:02:24.217 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.217 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.217 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:24.217 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.116658+0000", 2024-09-20T19:02:24.217 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4309999999999999e-06 2024-09-20T19:02:24.217 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.217 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.217 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:24.217 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.116671+0000", 2024-09-20T19:02:24.217 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2384e-05 2024-09-20T19:02:24.217 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.217 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.217 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:24.217 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.116678+0000", 2024-09-20T19:02:24.218 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8690000000000004e-06 2024-09-20T19:02:24.218 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.218 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.218 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.218 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.116734+0000", 2024-09-20T19:02:24.218 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6041000000000001e-05 2024-09-20T19:02:24.218 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.218 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.218 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:02:24.218 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.116742+0000", 2024-09-20T19:02:24.218 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8189999999999992e-06 2024-09-20T19:02:24.218 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.218 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.218 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.219 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.088569+0000", 2024-09-20T19:02:24.219 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97182719100000003 2024-09-20T19:02:24.219 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.219 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.219 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:24.219 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.088666+0000", 2024-09-20T19:02:24.219 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6897000000000004e-05 2024-09-20T19:02:24.219 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.219 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.219 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3", 2024-09-20T19:02:24.219 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.088770+0000", 2024-09-20T19:02:24.219 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010320299999999999 2024-09-20T19:02:24.219 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.219 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.219 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:02:24.220 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.091677+0000", 2024-09-20T19:02:24.220 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0029076900000000001 2024-09-20T19:02:24.220 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.220 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.220 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:24.220 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.098961+0000", 2024-09-20T19:02:24.220 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0072835080000000002 2024-09-20T19:02:24.220 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.220 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.220 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:24.220 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.099000+0000", 2024-09-20T19:02:24.220 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.892e-05 2024-09-20T19:02:24.220 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.220 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.221 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:02:24.221 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.099055+0000", 2024-09-20T19:02:24.221 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5554999999999999e-05 2024-09-20T19:02:24.221 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.221 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.221 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:24.221 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.099093+0000", 2024-09-20T19:02:24.221 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.769e-05 2024-09-20T19:02:24.221 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:24.221 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:24.221 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:24.221 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.221 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.221 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157496 8.7 8:efb55f87:::benchmark_data_smithi083_97422_object41405:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e213)", 2024-09-20T19:02:24.221 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.130402+0000", 2024-09-20T19:02:24.222 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 325.03603372700002, 2024-09-20T19:02:24.222 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.99286820799999997, 2024-09-20T19:02:24.222 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:24.222 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:24.222 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:02:24.222 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:02:24.222 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T19:02:24.222 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T19:02:24.222 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157496 2024-09-20T19:02:24.222 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.222 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:24.222 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.222 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:24.223 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.130402+0000", 2024-09-20T19:02:24.223 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:24.223 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.223 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.223 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:24.223 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.130402+0000", 2024-09-20T19:02:24.223 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:24.223 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.223 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.223 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:24.223 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.130404+0000", 2024-09-20T19:02:24.223 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.748e-06 2024-09-20T19:02:24.223 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.223 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.224 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:24.224 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.130410+0000", 2024-09-20T19:02:24.224 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2720000000000003e-06 2024-09-20T19:02:24.224 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.224 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.224 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:24.224 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.130426+0000", 2024-09-20T19:02:24.224 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6557000000000001e-05 2024-09-20T19:02:24.224 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.224 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.224 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:24.224 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.130436+0000", 2024-09-20T19:02:24.224 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4320000000000005e-06 2024-09-20T19:02:24.224 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.225 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.225 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.225 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.130499+0000", 2024-09-20T19:02:24.225 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3149000000000001e-05 2024-09-20T19:02:24.225 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.225 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.225 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:02:24.225 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.130508+0000", 2024-09-20T19:02:24.225 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1619999999999997e-06 2024-09-20T19:02:24.225 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.225 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.225 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.225 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.091952+0000", 2024-09-20T19:02:24.226 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.96144401999999995 2024-09-20T19:02:24.226 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.226 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.226 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:24.226 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.092057+0000", 2024-09-20T19:02:24.226 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000104268 2024-09-20T19:02:24.226 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.226 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.226 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3", 2024-09-20T19:02:24.226 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.092163+0000", 2024-09-20T19:02:24.226 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106545 2024-09-20T19:02:24.226 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.226 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.226 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:24.226 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.099133+0000", 2024-09-20T19:02:24.227 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0069702990000000001 2024-09-20T19:02:24.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.227 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.227 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:24.227 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.104449+0000", 2024-09-20T19:02:24.227 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0053152920000000001 2024-09-20T19:02:24.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.227 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.227 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:02:24.227 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.123091+0000", 2024-09-20T19:02:24.227 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.018642757999999999 2024-09-20T19:02:24.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.227 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.228 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:02:24.228 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.123225+0000", 2024-09-20T19:02:24.228 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000133881 2024-09-20T19:02:24.228 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.228 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.228 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:24.228 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.123270+0000", 2024-09-20T19:02:24.228 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4824999999999999e-05 2024-09-20T19:02:24.228 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:24.228 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:24.228 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:24.228 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.228 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.228 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157528 8.7 8:e4768593:::benchmark_data_smithi083_97422_object41437:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e213)", 2024-09-20T19:02:24.229 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.165392+0000", 2024-09-20T19:02:24.229 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 325.00104342899999, 2024-09-20T19:02:24.229 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.958042846, 2024-09-20T19:02:24.229 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:24.229 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:24.229 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:02:24.229 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:02:24.229 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T19:02:24.229 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T19:02:24.229 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157528 2024-09-20T19:02:24.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.229 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:24.229 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.230 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:24.230 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.165392+0000", 2024-09-20T19:02:24.230 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:24.230 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.230 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.230 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:24.230 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.165392+0000", 2024-09-20T19:02:24.230 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:24.230 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.230 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.230 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:24.230 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.165394+0000", 2024-09-20T19:02:24.230 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.77e-06 2024-09-20T19:02:24.230 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.230 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.231 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:24.231 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.165400+0000", 2024-09-20T19:02:24.231 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2099999999999998e-06 2024-09-20T19:02:24.231 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.231 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.231 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:24.231 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.165419+0000", 2024-09-20T19:02:24.231 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9046999999999999e-05 2024-09-20T19:02:24.231 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.231 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.231 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:24.231 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.165428+0000", 2024-09-20T19:02:24.231 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1030000000000008e-06 2024-09-20T19:02:24.232 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.232 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.232 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.232 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.166013+0000", 2024-09-20T19:02:24.232 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000584868 2024-09-20T19:02:24.232 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.232 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.232 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:02:24.232 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.166022+0000", 2024-09-20T19:02:24.232 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7090000000000008e-06 2024-09-20T19:02:24.232 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.232 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.232 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.233 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.095256+0000", 2024-09-20T19:02:24.233 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.92923407700000005 2024-09-20T19:02:24.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.233 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.233 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:24.233 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.095349+0000", 2024-09-20T19:02:24.233 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2724000000000002e-05 2024-09-20T19:02:24.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.233 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.233 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3", 2024-09-20T19:02:24.233 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.095688+0000", 2024-09-20T19:02:24.233 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000339379 2024-09-20T19:02:24.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.234 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.234 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:24.234 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.104489+0000", 2024-09-20T19:02:24.234 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0088013359999999999 2024-09-20T19:02:24.234 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.234 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.234 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:24.234 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.106738+0000", 2024-09-20T19:02:24.234 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0022487029999999999 2024-09-20T19:02:24.234 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.234 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.234 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:02:24.234 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.123280+0000", 2024-09-20T19:02:24.234 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.016541613 2024-09-20T19:02:24.235 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.235 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.235 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:02:24.235 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.123398+0000", 2024-09-20T19:02:24.235 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011812299999999999 2024-09-20T19:02:24.235 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.235 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.235 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:24.235 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.123435+0000", 2024-09-20T19:02:24.235 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7184000000000002e-05 2024-09-20T19:02:24.235 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:24.235 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:24.235 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:24.236 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.236 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.236 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:50392 9.0 9:015f775a:::benchmark_data_smithi083_124552_object50391:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e259)", 2024-09-20T19:02:24.236 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:59:04.061560+0000", 2024-09-20T19:02:24.236 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 200.104875521, 2024-09-20T19:02:24.236 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1500109090000001, 2024-09-20T19:02:24.236 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:24.236 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:24.236 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-20T19:02:24.236 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:02:24.237 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:02:24.237 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:02:24.237 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 50392 2024-09-20T19:02:24.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.238 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:24.238 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.238 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:24.238 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.061560+0000", 2024-09-20T19:02:24.238 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:24.238 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.238 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.238 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:24.238 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.061560+0000", 2024-09-20T19:02:24.238 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:24.238 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.238 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.238 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:24.239 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.061561+0000", 2024-09-20T19:02:24.239 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.252e-06 2024-09-20T19:02:24.239 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.239 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.239 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:24.239 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.061585+0000", 2024-09-20T19:02:24.239 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3275e-05 2024-09-20T19:02:24.239 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.239 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.239 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:24.239 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.061596+0000", 2024-09-20T19:02:24.239 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0963e-05 2024-09-20T19:02:24.239 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.239 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.240 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:24.240 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.061604+0000", 2024-09-20T19:02:24.240 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2549999999999997e-06 2024-09-20T19:02:24.240 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.240 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.240 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.240 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.063037+0000", 2024-09-20T19:02:24.240 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001432699 2024-09-20T19:02:24.240 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.240 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.240 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:24.240 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.063196+0000", 2024-09-20T19:02:24.240 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015964199999999999 2024-09-20T19:02:24.240 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.241 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.241 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,2", 2024-09-20T19:02:24.241 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.063344+0000", 2024-09-20T19:02:24.241 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014738699999999999 2024-09-20T19:02:24.241 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.241 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.241 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:02:24.241 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.068011+0000", 2024-09-20T19:02:24.241 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0046678240000000001 2024-09-20T19:02:24.241 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.241 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.241 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.241 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.098231+0000", 2024-09-20T19:02:24.241 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.030219817999999999 2024-09-20T19:02:24.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.242 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.242 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:02:24.242 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.098244+0000", 2024-09-20T19:02:24.242 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2381000000000001e-05 2024-09-20T19:02:24.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.242 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.242 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.242 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.074475+0000", 2024-09-20T19:02:24.242 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97623147099999996 2024-09-20T19:02:24.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.242 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.242 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:02:24.242 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.074562+0000", 2024-09-20T19:02:24.242 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7317000000000004e-05 2024-09-20T19:02:24.243 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.243 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.243 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.243 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.201219+0000", 2024-09-20T19:02:24.243 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1266569 2024-09-20T19:02:24.243 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.243 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.243 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for degraded object", 2024-09-20T19:02:24.243 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.201984+0000", 2024-09-20T19:02:24.243 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00076430200000000002 2024-09-20T19:02:24.243 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.243 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.243 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.243 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.211479+0000", 2024-09-20T19:02:24.244 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0094952230000000006 2024-09-20T19:02:24.244 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.244 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.244 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:24.244 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.211571+0000", 2024-09-20T19:02:24.244 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2200000000000005e-05 2024-09-20T19:02:24.244 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:24.244 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:24.244 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:24.244 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.244 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.244 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:50400 9.0 9:0091def0:::benchmark_data_smithi083_124552_object50399:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-20T19:02:24.244 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:59:04.066245+0000", 2024-09-20T19:02:24.244 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 200.10019082100001, 2024-09-20T19:02:24.245 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1471234040000002, 2024-09-20T19:02:24.245 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:24.245 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:24.245 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:02:24.245 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:02:24.245 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:02:24.245 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:02:24.245 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 50400 2024-09-20T19:02:24.245 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.245 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:24.245 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.245 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:24.245 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.066245+0000", 2024-09-20T19:02:24.245 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:24.246 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.246 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.246 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:24.246 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.066245+0000", 2024-09-20T19:02:24.246 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:24.246 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.246 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.246 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:24.246 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.066247+0000", 2024-09-20T19:02:24.246 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.73e-06 2024-09-20T19:02:24.246 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.246 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.246 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:24.246 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.066271+0000", 2024-09-20T19:02:24.246 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4793000000000001e-05 2024-09-20T19:02:24.247 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.247 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.247 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:24.247 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.066286+0000", 2024-09-20T19:02:24.247 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4515000000000001e-05 2024-09-20T19:02:24.247 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.247 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.247 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:24.247 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.066295+0000", 2024-09-20T19:02:24.247 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7360000000000007e-06 2024-09-20T19:02:24.247 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.247 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.247 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.247 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.067890+0000", 2024-09-20T19:02:24.248 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001595791 2024-09-20T19:02:24.248 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.248 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.248 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op must wait for map", 2024-09-20T19:02:24.248 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.067896+0000", 2024-09-20T19:02:24.248 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.446e-06 2024-09-20T19:02:24.248 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.248 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.248 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.248 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.099590+0000", 2024-09-20T19:02:24.248 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.031694211999999999 2024-09-20T19:02:24.248 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.248 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.248 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:02:24.249 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.099600+0000", 2024-09-20T19:02:24.249 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.91e-06 2024-09-20T19:02:24.249 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.249 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.249 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.249 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.075887+0000", 2024-09-20T19:02:24.249 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.976287401 2024-09-20T19:02:24.249 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.249 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.249 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:02:24.249 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.075898+0000", 2024-09-20T19:02:24.249 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0294e-05 2024-09-20T19:02:24.249 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.249 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.249 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.249 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.203092+0000", 2024-09-20T19:02:24.250 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1271948649999999 2024-09-20T19:02:24.250 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.250 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.250 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:24.250 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.203244+0000", 2024-09-20T19:02:24.250 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015116500000000001 2024-09-20T19:02:24.250 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.250 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.250 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,6", 2024-09-20T19:02:24.250 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.203418+0000", 2024-09-20T19:02:24.250 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017446200000000001 2024-09-20T19:02:24.250 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.250 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.250 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:02:24.250 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.208270+0000", 2024-09-20T19:02:24.251 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0048516799999999997 2024-09-20T19:02:24.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.251 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.251 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:24.251 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.208385+0000", 2024-09-20T19:02:24.251 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000114954 2024-09-20T19:02:24.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.251 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.251 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:24.251 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.211663+0000", 2024-09-20T19:02:24.251 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0032777679999999999 2024-09-20T19:02:24.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.251 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.252 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:02:24.252 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.211716+0000", 2024-09-20T19:02:24.252 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3748000000000001e-05 2024-09-20T19:02:24.252 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.252 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.252 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:24.252 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.213368+0000", 2024-09-20T19:02:24.252 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0016519340000000001 2024-09-20T19:02:24.252 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:24.252 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:24.252 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:24.252 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.252 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.253 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:50401 9.0 9:08f1b92c:::benchmark_data_smithi083_124552_object50400:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-20T19:02:24.253 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:59:04.067487+0000", 2024-09-20T19:02:24.253 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 200.098948871, 2024-09-20T19:02:24.253 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.149403511, 2024-09-20T19:02:24.253 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:24.253 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:24.253 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:02:24.253 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:02:24.253 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:02:24.253 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:02:24.253 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 50401 2024-09-20T19:02:24.253 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.253 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:24.253 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.254 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:24.254 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.067487+0000", 2024-09-20T19:02:24.254 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:24.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.254 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.254 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:24.254 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.067487+0000", 2024-09-20T19:02:24.254 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:24.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.254 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.254 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:24.254 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.067488+0000", 2024-09-20T19:02:24.254 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4670000000000001e-06 2024-09-20T19:02:24.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.255 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.255 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:24.255 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.067512+0000", 2024-09-20T19:02:24.255 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3561999999999999e-05 2024-09-20T19:02:24.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.255 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.255 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:24.255 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.067523+0000", 2024-09-20T19:02:24.255 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.148e-05 2024-09-20T19:02:24.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.255 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.255 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:24.255 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.067531+0000", 2024-09-20T19:02:24.255 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2400000000000001e-06 2024-09-20T19:02:24.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.256 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.256 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.256 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.068001+0000", 2024-09-20T19:02:24.256 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00047044099999999999 2024-09-20T19:02:24.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.256 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.256 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting_for_map not empty", 2024-09-20T19:02:24.256 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.068005+0000", 2024-09-20T19:02:24.256 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3490000000000002e-06 2024-09-20T19:02:24.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.256 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.256 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.257 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.099629+0000", 2024-09-20T19:02:24.257 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.031623658999999998 2024-09-20T19:02:24.257 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.257 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.257 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:02:24.257 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.099637+0000", 2024-09-20T19:02:24.257 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5750000000000003e-06 2024-09-20T19:02:24.257 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.257 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.257 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.257 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.075922+0000", 2024-09-20T19:02:24.257 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97628498500000005 2024-09-20T19:02:24.257 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.257 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.258 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:02:24.258 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.075929+0000", 2024-09-20T19:02:24.258 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2389999999999996e-06 2024-09-20T19:02:24.258 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.258 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.258 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.258 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.204955+0000", 2024-09-20T19:02:24.258 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.129026353 2024-09-20T19:02:24.258 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.258 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.258 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:24.258 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.205053+0000", 2024-09-20T19:02:24.258 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7659999999999997e-05 2024-09-20T19:02:24.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.259 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,6", 2024-09-20T19:02:24.259 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.205157+0000", 2024-09-20T19:02:24.259 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000104237 2024-09-20T19:02:24.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.259 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:02:24.259 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.208403+0000", 2024-09-20T19:02:24.259 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0032460050000000002 2024-09-20T19:02:24.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.259 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:24.259 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.213804+0000", 2024-09-20T19:02:24.260 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0054012280000000001 2024-09-20T19:02:24.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.260 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.260 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:24.260 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.215227+0000", 2024-09-20T19:02:24.260 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014227039999999999 2024-09-20T19:02:24.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.260 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.260 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:02:24.260 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.215288+0000", 2024-09-20T19:02:24.260 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0730999999999998e-05 2024-09-20T19:02:24.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.260 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.260 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:24.261 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.216890+0000", 2024-09-20T19:02:24.261 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001602596 2024-09-20T19:02:24.261 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:24.261 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:24.261 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:24.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.261 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.261 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:50420 9.0 9:04fafae8:::benchmark_data_smithi083_124552_object50419:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-20T19:02:24.261 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:59:04.091011+0000", 2024-09-20T19:02:24.261 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 200.07542463300001, 2024-09-20T19:02:24.261 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.143183112, 2024-09-20T19:02:24.261 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:24.261 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:24.261 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:02:24.261 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:02:24.262 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:02:24.262 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:02:24.262 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 50420 2024-09-20T19:02:24.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.262 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:24.262 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.262 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:24.262 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.091011+0000", 2024-09-20T19:02:24.262 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:24.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.262 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.262 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:24.262 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.091011+0000", 2024-09-20T19:02:24.262 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:24.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.263 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.263 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:24.263 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.091013+0000", 2024-09-20T19:02:24.263 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6440000000000001e-06 2024-09-20T19:02:24.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.263 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.263 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:24.263 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.091069+0000", 2024-09-20T19:02:24.263 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6008000000000003e-05 2024-09-20T19:02:24.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.263 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.263 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:24.263 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.091080+0000", 2024-09-20T19:02:24.263 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1449e-05 2024-09-20T19:02:24.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.264 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:24.264 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.091087+0000", 2024-09-20T19:02:24.264 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3769999999999998e-06 2024-09-20T19:02:24.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.264 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.264 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.099660+0000", 2024-09-20T19:02:24.264 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0085721310000000002 2024-09-20T19:02:24.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.265 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:02:24.265 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.099667+0000", 2024-09-20T19:02:24.265 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9439999999999999e-06 2024-09-20T19:02:24.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.265 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.265 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.075951+0000", 2024-09-20T19:02:24.265 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97628466400000002 2024-09-20T19:02:24.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.265 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:02:24.265 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.075958+0000", 2024-09-20T19:02:24.265 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1019999999999999e-06 2024-09-20T19:02:24.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.266 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.266 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.216952+0000", 2024-09-20T19:02:24.266 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1409938340000001 2024-09-20T19:02:24.266 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.266 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.266 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:24.266 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.217094+0000", 2024-09-20T19:02:24.266 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014154900000000001 2024-09-20T19:02:24.266 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.266 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.266 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,6", 2024-09-20T19:02:24.266 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.217258+0000", 2024-09-20T19:02:24.266 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016462399999999999 2024-09-20T19:02:24.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.267 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.267 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:24.267 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.223577+0000", 2024-09-20T19:02:24.267 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063191740000000003 2024-09-20T19:02:24.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.267 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.267 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:02:24.267 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.223593+0000", 2024-09-20T19:02:24.267 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5588999999999999e-05 2024-09-20T19:02:24.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.267 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.267 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:24.268 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.230738+0000", 2024-09-20T19:02:24.268 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0071453879999999999 2024-09-20T19:02:24.268 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.268 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.268 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:02:24.268 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.230856+0000", 2024-09-20T19:02:24.268 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000117713 2024-09-20T19:02:24.268 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.268 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.268 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:24.268 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.234194+0000", 2024-09-20T19:02:24.268 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0033379220000000001 2024-09-20T19:02:24.268 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:24.268 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:24.269 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:24.269 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.269 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.269 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:50490 9.0 9:061fbcaa:::benchmark_data_smithi083_124552_object50489:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-20T19:02:24.269 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:59:04.144098+0000", 2024-09-20T19:02:24.269 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 200.022338031, 2024-09-20T19:02:24.269 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.093487444, 2024-09-20T19:02:24.269 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:24.269 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:24.269 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:02:24.269 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:02:24.269 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:02:24.269 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:02:24.269 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 50490 2024-09-20T19:02:24.270 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.270 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:24.270 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.270 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:24.270 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.144098+0000", 2024-09-20T19:02:24.270 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:24.270 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.270 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.270 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:24.270 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.144098+0000", 2024-09-20T19:02:24.270 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:24.270 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.270 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.270 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:24.271 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.144099+0000", 2024-09-20T19:02:24.271 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.412e-06 2024-09-20T19:02:24.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.271 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.271 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:24.271 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.144124+0000", 2024-09-20T19:02:24.271 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4788000000000001e-05 2024-09-20T19:02:24.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.271 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.271 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:24.271 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.144138+0000", 2024-09-20T19:02:24.271 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3735e-05 2024-09-20T19:02:24.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.271 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.271 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:24.272 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.144146+0000", 2024-09-20T19:02:24.272 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.1689999999999992e-06 2024-09-20T19:02:24.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.272 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.272 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.272 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.144218+0000", 2024-09-20T19:02:24.272 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1945000000000003e-05 2024-09-20T19:02:24.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.272 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.272 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:02:24.272 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.144227+0000", 2024-09-20T19:02:24.272 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6090000000000007e-06 2024-09-20T19:02:24.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.272 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.273 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.273 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.075982+0000", 2024-09-20T19:02:24.273 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.93175522499999996 2024-09-20T19:02:24.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.273 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.273 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:02:24.273 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.075990+0000", 2024-09-20T19:02:24.273 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4789999999999999e-06 2024-09-20T19:02:24.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.273 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.273 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.273 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.218974+0000", 2024-09-20T19:02:24.273 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1429841510000001 2024-09-20T19:02:24.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.274 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.274 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:24.274 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.219119+0000", 2024-09-20T19:02:24.274 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014451399999999999 2024-09-20T19:02:24.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.274 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.274 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,6", 2024-09-20T19:02:24.274 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.219402+0000", 2024-09-20T19:02:24.274 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00028311699999999999 2024-09-20T19:02:24.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.274 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.274 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:24.274 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.223642+0000", 2024-09-20T19:02:24.274 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0042407119999999998 2024-09-20T19:02:24.275 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.275 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.275 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:02:24.275 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.230612+0000", 2024-09-20T19:02:24.275 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0069695499999999997 2024-09-20T19:02:24.275 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.275 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.275 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:24.275 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.234261+0000", 2024-09-20T19:02:24.275 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0036485459999999999 2024-09-20T19:02:24.275 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.275 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.275 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:02:24.275 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.234386+0000", 2024-09-20T19:02:24.275 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012551199999999999 2024-09-20T19:02:24.276 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.276 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.276 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:24.276 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.237585+0000", 2024-09-20T19:02:24.276 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0031989800000000001 2024-09-20T19:02:24.276 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:24.276 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:24.276 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:24.276 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.276 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.276 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:50503 9.0 9:0e691601:::benchmark_data_smithi083_124552_object50502:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-20T19:02:24.276 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:59:04.153655+0000", 2024-09-20T19:02:24.276 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 200.01278063699999, 2024-09-20T19:02:24.277 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0864925420000002, 2024-09-20T19:02:24.277 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:24.277 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:24.277 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:02:24.277 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:02:24.277 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:02:24.277 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:02:24.277 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 50503 2024-09-20T19:02:24.277 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.277 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:24.277 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.277 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:24.277 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.153655+0000", 2024-09-20T19:02:24.277 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:24.278 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.279 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.279 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:24.279 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.153655+0000", 2024-09-20T19:02:24.279 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:24.279 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.279 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.279 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:24.279 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.153656+0000", 2024-09-20T19:02:24.279 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2099999999999996e-07 2024-09-20T19:02:24.279 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.280 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.280 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:24.280 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.153675+0000", 2024-09-20T19:02:24.280 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9873000000000001e-05 2024-09-20T19:02:24.280 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.280 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.280 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:24.280 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.153683+0000", 2024-09-20T19:02:24.280 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5469999999999999e-06 2024-09-20T19:02:24.280 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.280 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.280 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:24.280 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.153689+0000", 2024-09-20T19:02:24.280 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1129999999999997e-06 2024-09-20T19:02:24.280 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.281 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.281 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.281 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.153774+0000", 2024-09-20T19:02:24.281 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4806000000000001e-05 2024-09-20T19:02:24.281 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.281 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.281 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:02:24.281 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.153783+0000", 2024-09-20T19:02:24.281 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8890000000000008e-06 2024-09-20T19:02:24.281 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.281 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.281 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.281 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.076016+0000", 2024-09-20T19:02:24.282 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.92223312199999996 2024-09-20T19:02:24.282 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.282 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.282 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:02:24.282 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.076025+0000", 2024-09-20T19:02:24.282 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0480000000000004e-06 2024-09-20T19:02:24.282 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.282 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.282 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.282 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.221175+0000", 2024-09-20T19:02:24.282 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1451503329999999 2024-09-20T19:02:24.282 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.282 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.282 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:24.283 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.221314+0000", 2024-09-20T19:02:24.283 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013887599999999999 2024-09-20T19:02:24.283 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.283 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.283 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,6", 2024-09-20T19:02:24.283 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.221482+0000", 2024-09-20T19:02:24.283 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016769800000000001 2024-09-20T19:02:24.283 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.283 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.283 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:24.283 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.230668+0000", 2024-09-20T19:02:24.283 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0091861119999999997 2024-09-20T19:02:24.283 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.284 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.284 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:02:24.284 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.234213+0000", 2024-09-20T19:02:24.284 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0035446620000000001 2024-09-20T19:02:24.284 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.284 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.284 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:24.284 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.237649+0000", 2024-09-20T19:02:24.284 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003436555 2024-09-20T19:02:24.284 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.284 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.284 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:02:24.284 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.237809+0000", 2024-09-20T19:02:24.284 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015965099999999999 2024-09-20T19:02:24.285 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.285 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.285 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:24.285 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.240148+0000", 2024-09-20T19:02:24.285 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0023386359999999998 2024-09-20T19:02:24.285 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:24.285 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:24.285 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:24.285 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.285 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.285 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:142920 9.4 9:275bd396:::benchmark_data_smithi083_124552_object28773:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e291)", 2024-09-20T19:02:24.285 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:35.173575+0000", 2024-09-20T19:02:24.285 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 108.99286083299999, 2024-09-20T19:02:24.286 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0532428149999999, 2024-09-20T19:02:24.286 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:24.286 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:24.286 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-20T19:02:24.286 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:02:24.286 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:02:24.286 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:02:24.286 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 142920 2024-09-20T19:02:24.286 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.286 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:24.286 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.286 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:24.286 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.173575+0000", 2024-09-20T19:02:24.287 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:24.287 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.287 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.287 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:24.287 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.173575+0000", 2024-09-20T19:02:24.287 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:24.287 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.287 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.287 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:24.287 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.173576+0000", 2024-09-20T19:02:24.287 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6899999999999999e-06 2024-09-20T19:02:24.287 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.287 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.287 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:24.288 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.173583+0000", 2024-09-20T19:02:24.288 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0889999999999996e-06 2024-09-20T19:02:24.288 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.288 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.288 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:24.288 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.173598+0000", 2024-09-20T19:02:24.288 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5917e-05 2024-09-20T19:02:24.288 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.288 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.288 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:24.288 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.173606+0000", 2024-09-20T19:02:24.288 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.362e-06 2024-09-20T19:02:24.288 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.288 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.288 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.289 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.173664+0000", 2024-09-20T19:02:24.289 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7725e-05 2024-09-20T19:02:24.289 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.289 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.289 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:24.289 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.173770+0000", 2024-09-20T19:02:24.289 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106901 2024-09-20T19:02:24.289 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.289 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.289 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3,4", 2024-09-20T19:02:24.289 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.173894+0000", 2024-09-20T19:02:24.289 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000123139 2024-09-20T19:02:24.289 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.289 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.290 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:24.290 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.181240+0000", 2024-09-20T19:02:24.290 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0073461539999999997 2024-09-20T19:02:24.290 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.290 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.290 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:02:24.290 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.181260+0000", 2024-09-20T19:02:24.290 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9987e-05 2024-09-20T19:02:24.290 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.290 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.290 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.290 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.200929+0000", 2024-09-20T19:02:24.290 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019669494999999999 2024-09-20T19:02:24.290 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.291 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.291 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:02:24.291 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.200940+0000", 2024-09-20T19:02:24.291 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0864e-05 2024-09-20T19:02:24.291 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.291 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.291 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.291 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.226746+0000", 2024-09-20T19:02:24.291 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.025806225 2024-09-20T19:02:24.291 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.291 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.291 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:24.291 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.226818+0000", 2024-09-20T19:02:24.291 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1266999999999994e-05 2024-09-20T19:02:24.292 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:24.292 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:24.292 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:24.292 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.292 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.292 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:142948 9.4 9:23ac88ff:::benchmark_data_smithi083_124552_object28801:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e292)", 2024-09-20T19:02:24.292 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:35.220440+0000", 2024-09-20T19:02:24.292 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 108.94599596899999, 2024-09-20T19:02:24.292 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0177205739999999, 2024-09-20T19:02:24.292 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:24.292 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:24.292 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:02:24.292 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:02:24.292 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:02:24.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:24 smithi083 ceph-mon[29421]: 6.d scrub starts 2024-09-20T19:02:24.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:24 smithi083 ceph-mon[29421]: 6.d scrub ok 2024-09-20T19:02:24.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:24 smithi083 ceph-mon[29421]: 3.7 scrub starts 2024-09-20T19:02:24.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:24 smithi083 ceph-mon[29421]: 3.7 scrub ok 2024-09-20T19:02:24.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:24 smithi083 ceph-mon[29421]: 5.e scrub starts 2024-09-20T19:02:24.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:24 smithi083 ceph-mon[29421]: 5.e scrub ok 2024-09-20T19:02:24.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:24 smithi083 ceph-mon[29421]: pgmap v624: 148 pgs: 148 active+clean; 296 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 853 B/s rd, 49 MiB/s wr, 787 op/s 2024-09-20T19:02:24.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:24 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:02:24.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:24 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:02:24.294 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:24 smithi083 ceph-mon[29421]: 5.18 scrub starts 2024-09-20T19:02:24.294 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:24 smithi083 ceph-mon[29421]: 5.18 scrub ok 2024-09-20T19:02:24.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:24 smithi083 ceph-mon[36260]: 6.d scrub starts 2024-09-20T19:02:24.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:24 smithi083 ceph-mon[36260]: 6.d scrub ok 2024-09-20T19:02:24.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:24 smithi083 ceph-mon[36260]: 3.7 scrub starts 2024-09-20T19:02:24.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:24 smithi083 ceph-mon[36260]: 3.7 scrub ok 2024-09-20T19:02:24.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:24 smithi083 ceph-mon[36260]: 5.e scrub starts 2024-09-20T19:02:24.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:24 smithi083 ceph-mon[36260]: 5.e scrub ok 2024-09-20T19:02:24.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:24 smithi083 ceph-mon[36260]: pgmap v624: 148 pgs: 148 active+clean; 296 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 853 B/s rd, 49 MiB/s wr, 787 op/s 2024-09-20T19:02:24.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:24 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:02:24.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:24 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:02:24.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:24 smithi083 ceph-mon[36260]: 5.18 scrub starts 2024-09-20T19:02:24.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:24 smithi083 ceph-mon[36260]: 5.18 scrub ok 2024-09-20T19:02:24.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:24.296 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:02:24.296 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 142948 2024-09-20T19:02:24.296 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.296 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:24.296 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.296 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:24.296 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.220440+0000", 2024-09-20T19:02:24.296 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:24.296 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.296 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.297 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:24.297 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.220440+0000", 2024-09-20T19:02:24.297 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:24.297 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.297 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.297 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:24.297 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.220441+0000", 2024-09-20T19:02:24.297 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1999999999999998e-07 2024-09-20T19:02:24.297 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.297 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.297 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:24.297 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.220445+0000", 2024-09-20T19:02:24.297 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1219999999999997e-06 2024-09-20T19:02:24.297 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.298 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.298 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:24.298 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.220453+0000", 2024-09-20T19:02:24.298 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5359999999999992e-06 2024-09-20T19:02:24.298 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.298 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.298 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:24.298 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.220459+0000", 2024-09-20T19:02:24.298 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0929999999999999e-06 2024-09-20T19:02:24.298 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.298 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.298 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.298 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.220510+0000", 2024-09-20T19:02:24.299 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0853000000000003e-05 2024-09-20T19:02:24.299 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.299 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.299 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:02:24.299 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.220518+0000", 2024-09-20T19:02:24.299 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4039999999999998e-06 2024-09-20T19:02:24.299 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.299 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.299 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.299 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.227191+0000", 2024-09-20T19:02:24.299 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0066732009999999 2024-09-20T19:02:24.299 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.299 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.299 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:24.299 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.227287+0000", 2024-09-20T19:02:24.300 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6204000000000003e-05 2024-09-20T19:02:24.300 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.300 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.300 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-20T19:02:24.300 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.227405+0000", 2024-09-20T19:02:24.300 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000118386 2024-09-20T19:02:24.300 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.300 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.300 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:24.300 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.237785+0000", 2024-09-20T19:02:24.300 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010379578 2024-09-20T19:02:24.300 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.300 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.300 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:02:24.301 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.238035+0000", 2024-09-20T19:02:24.301 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00024981 2024-09-20T19:02:24.301 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.301 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.301 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:02:24.301 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.238080+0000", 2024-09-20T19:02:24.301 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5015000000000002e-05 2024-09-20T19:02:24.301 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.301 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.301 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:24.301 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.238160+0000", 2024-09-20T19:02:24.301 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0451999999999995e-05 2024-09-20T19:02:24.301 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:24.301 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:24.301 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:24.302 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.302 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.302 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:142954 9.4 9:221e2aa5:::benchmark_data_smithi083_124552_object28807:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e292)", 2024-09-20T19:02:24.302 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:35.228555+0000", 2024-09-20T19:02:24.302 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 108.93788097300001, 2024-09-20T19:02:24.302 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0097211580000001, 2024-09-20T19:02:24.302 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:24.302 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:24.302 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:02:24.302 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:02:24.302 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:02:24.302 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:02:24.302 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 142954 2024-09-20T19:02:24.303 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.303 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:24.303 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.303 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:24.303 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.228555+0000", 2024-09-20T19:02:24.303 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:24.303 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.303 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.303 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:24.303 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.228555+0000", 2024-09-20T19:02:24.303 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:24.303 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.303 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.304 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:24.304 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.228556+0000", 2024-09-20T19:02:24.304 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.533e-06 2024-09-20T19:02:24.304 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.304 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.304 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:24.304 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.228563+0000", 2024-09-20T19:02:24.304 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6619999999999999e-06 2024-09-20T19:02:24.304 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.304 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.304 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:24.304 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.228587+0000", 2024-09-20T19:02:24.304 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.444e-05 2024-09-20T19:02:24.304 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.304 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.305 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:24.305 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.228596+0000", 2024-09-20T19:02:24.305 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6060000000000003e-06 2024-09-20T19:02:24.305 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.305 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.305 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.305 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.228659+0000", 2024-09-20T19:02:24.305 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2677999999999999e-05 2024-09-20T19:02:24.305 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.305 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.305 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:02:24.305 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.228668+0000", 2024-09-20T19:02:24.305 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2220000000000008e-06 2024-09-20T19:02:24.305 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.306 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.306 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.306 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.229869+0000", 2024-09-20T19:02:24.306 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.00120093 2024-09-20T19:02:24.306 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.306 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.306 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:24.306 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.229964+0000", 2024-09-20T19:02:24.306 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5072e-05 2024-09-20T19:02:24.306 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.306 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.306 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-20T19:02:24.306 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.230066+0000", 2024-09-20T19:02:24.306 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000102558 2024-09-20T19:02:24.306 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.307 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.307 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:24.307 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.237878+0000", 2024-09-20T19:02:24.307 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0078119929999999997 2024-09-20T19:02:24.307 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.307 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.307 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:02:24.307 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.238170+0000", 2024-09-20T19:02:24.307 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00029176400000000002 2024-09-20T19:02:24.307 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.307 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.307 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:02:24.307 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.238208+0000", 2024-09-20T19:02:24.307 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7512999999999998e-05 2024-09-20T19:02:24.308 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.308 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.308 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:24.308 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.238276+0000", 2024-09-20T19:02:24.308 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8187000000000003e-05 2024-09-20T19:02:24.308 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:24.308 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:24.308 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:24.308 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.308 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.308 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:142972 9.4 9:2d2af5ad:::benchmark_data_smithi083_124552_object28825:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e292)", 2024-09-20T19:02:24.308 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:35.245464+0000", 2024-09-20T19:02:24.308 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 108.920971298, 2024-09-20T19:02:24.308 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.99292613500000004, 2024-09-20T19:02:24.308 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:24.309 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:24.309 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:02:24.309 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:02:24.309 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:02:24.309 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:02:24.309 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 142972 2024-09-20T19:02:24.309 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.309 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:24.309 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.309 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:24.309 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.245464+0000", 2024-09-20T19:02:24.309 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:24.309 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.309 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.310 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:24.310 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.245464+0000", 2024-09-20T19:02:24.310 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:24.310 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.310 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.310 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:24.310 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.245466+0000", 2024-09-20T19:02:24.310 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.869e-06 2024-09-20T19:02:24.310 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.310 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.310 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:24.310 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.245473+0000", 2024-09-20T19:02:24.310 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.776e-06 2024-09-20T19:02:24.310 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.311 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.311 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:24.311 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.245485+0000", 2024-09-20T19:02:24.311 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.202e-05 2024-09-20T19:02:24.311 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.311 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.311 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:24.311 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.245494+0000", 2024-09-20T19:02:24.311 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0920000000000004e-06 2024-09-20T19:02:24.311 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.311 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.311 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.311 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.245551+0000", 2024-09-20T19:02:24.311 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7138000000000001e-05 2024-09-20T19:02:24.312 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.312 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.312 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:02:24.312 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.245559+0000", 2024-09-20T19:02:24.312 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8509999999999999e-06 2024-09-20T19:02:24.312 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.312 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.312 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.312 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.232633+0000", 2024-09-20T19:02:24.312 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.98707439200000002 2024-09-20T19:02:24.312 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.312 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.312 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:24.312 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.232734+0000", 2024-09-20T19:02:24.313 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000100424 2024-09-20T19:02:24.313 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.313 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.313 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-20T19:02:24.313 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.232845+0000", 2024-09-20T19:02:24.313 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000110645 2024-09-20T19:02:24.313 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.313 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.313 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:24.313 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.238021+0000", 2024-09-20T19:02:24.313 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0051761410000000004 2024-09-20T19:02:24.313 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.313 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.314 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:02:24.314 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.238283+0000", 2024-09-20T19:02:24.314 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000262339 2024-09-20T19:02:24.314 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.314 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.314 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:02:24.314 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.238322+0000", 2024-09-20T19:02:24.314 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9017000000000003e-05 2024-09-20T19:02:24.314 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.314 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.314 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:24.314 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.238390+0000", 2024-09-20T19:02:24.314 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8430999999999996e-05 2024-09-20T19:02:24.314 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:24.314 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:24.315 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:24.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.315 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.315 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:142980 9.4 9:2b9b75e3:::benchmark_data_smithi083_124552_object28833:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e292)", 2024-09-20T19:02:24.315 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:35.254040+0000", 2024-09-20T19:02:24.315 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 108.91239515300001, 2024-09-20T19:02:24.315 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.99718591300000003, 2024-09-20T19:02:24.315 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:24.315 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:24.315 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:02:24.315 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:02:24.315 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:02:24.315 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:02:24.316 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 142980 2024-09-20T19:02:24.316 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.316 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:24.316 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.316 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:24.316 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.254040+0000", 2024-09-20T19:02:24.316 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:24.316 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.316 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.316 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:24.316 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.254040+0000", 2024-09-20T19:02:24.316 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:24.316 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.316 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.317 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:24.317 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.254043+0000", 2024-09-20T19:02:24.317 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1100000000000001e-06 2024-09-20T19:02:24.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.317 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.317 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:24.317 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.254049+0000", 2024-09-20T19:02:24.317 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2949999999999999e-06 2024-09-20T19:02:24.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.317 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.317 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:24.317 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.254067+0000", 2024-09-20T19:02:24.317 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7912999999999999e-05 2024-09-20T19:02:24.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.318 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.318 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:24.318 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.254075+0000", 2024-09-20T19:02:24.318 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4889999999999995e-06 2024-09-20T19:02:24.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.318 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.318 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.318 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.254143+0000", 2024-09-20T19:02:24.318 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8182999999999994e-05 2024-09-20T19:02:24.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.318 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.318 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:02:24.318 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.254154+0000", 2024-09-20T19:02:24.319 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0227000000000001e-05 2024-09-20T19:02:24.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.319 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.319 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.319 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.235381+0000", 2024-09-20T19:02:24.319 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.98122688400000002 2024-09-20T19:02:24.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.319 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.319 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:24.319 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.235478+0000", 2024-09-20T19:02:24.319 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7344999999999997e-05 2024-09-20T19:02:24.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.319 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.319 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-20T19:02:24.319 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.235580+0000", 2024-09-20T19:02:24.320 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010220700000000001 2024-09-20T19:02:24.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.320 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.320 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:02:24.320 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.250981+0000", 2024-09-20T19:02:24.320 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.015400713999999999 2024-09-20T19:02:24.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.320 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.320 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:24.320 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.251017+0000", 2024-09-20T19:02:24.320 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6649000000000001e-05 2024-09-20T19:02:24.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.320 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.320 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:02:24.320 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.251114+0000", 2024-09-20T19:02:24.321 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6287999999999997e-05 2024-09-20T19:02:24.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.321 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.321 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:24.321 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.251226+0000", 2024-09-20T19:02:24.321 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011260900000000001 2024-09-20T19:02:24.321 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:24.321 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:24.321 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:24.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.321 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.321 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:142984 9.4 9:2e403320:::benchmark_data_smithi083_124552_object28837:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e292)", 2024-09-20T19:02:24.321 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:35.256799+0000", 2024-09-20T19:02:24.321 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 108.909636474, 2024-09-20T19:02:24.322 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.99530053100000004, 2024-09-20T19:02:24.322 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:24.322 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:24.322 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:02:24.322 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:02:24.322 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:02:24.322 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:02:24.322 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 142984 2024-09-20T19:02:24.322 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.322 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:24.322 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.322 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:24.322 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.256799+0000", 2024-09-20T19:02:24.322 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:24.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.323 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.323 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:24.323 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.256799+0000", 2024-09-20T19:02:24.323 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:24.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.323 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.323 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:24.323 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.256801+0000", 2024-09-20T19:02:24.323 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9199999999999998e-06 2024-09-20T19:02:24.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.323 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.323 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:24.323 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.256808+0000", 2024-09-20T19:02:24.324 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.601e-06 2024-09-20T19:02:24.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.324 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.324 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:24.324 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.256825+0000", 2024-09-20T19:02:24.324 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7189000000000001e-05 2024-09-20T19:02:24.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.324 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.324 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:24.324 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.256834+0000", 2024-09-20T19:02:24.324 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2669999999999995e-06 2024-09-20T19:02:24.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.324 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.324 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.325 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.256916+0000", 2024-09-20T19:02:24.325 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2342000000000005e-05 2024-09-20T19:02:24.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.325 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.325 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:02:24.325 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.256926+0000", 2024-09-20T19:02:24.325 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6220000000000004e-06 2024-09-20T19:02:24.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.325 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.325 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:24.325 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.238429+0000", 2024-09-20T19:02:24.325 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.981503126 2024-09-20T19:02:24.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.325 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.326 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:24.326 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.238613+0000", 2024-09-20T19:02:24.326 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018369299999999999 2024-09-20T19:02:24.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.326 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.326 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-20T19:02:24.326 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.238721+0000", 2024-09-20T19:02:24.326 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000108472 2024-09-20T19:02:24.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.326 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.326 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:02:24.326 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.251237+0000", 2024-09-20T19:02:24.326 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.012515584 2024-09-20T19:02:24.327 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.327 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.327 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:24.327 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.251956+0000", 2024-09-20T19:02:24.327 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00071889299999999996 2024-09-20T19:02:24.327 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.327 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.327 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:02:24.327 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.252024+0000", 2024-09-20T19:02:24.327 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8477000000000002e-05 2024-09-20T19:02:24.327 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:24.327 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:24.327 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:24.327 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.252100+0000", 2024-09-20T19:02:24.327 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5345000000000005e-05 2024-09-20T19:02:24.328 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:24.328 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:24.328 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:24.328 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:24.328 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:24.328 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T19:02:24.328 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:24.328 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:24.429 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:02:24.465 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:24.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:24.466 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:02:24.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:24 smithi142 ceph-mon[32508]: 6.d scrub starts 2024-09-20T19:02:24.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:24 smithi142 ceph-mon[32508]: 6.d scrub ok 2024-09-20T19:02:24.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:24 smithi142 ceph-mon[32508]: 3.7 scrub starts 2024-09-20T19:02:24.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:24 smithi142 ceph-mon[32508]: 3.7 scrub ok 2024-09-20T19:02:24.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:24 smithi142 ceph-mon[32508]: 5.e scrub starts 2024-09-20T19:02:24.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:24 smithi142 ceph-mon[32508]: 5.e scrub ok 2024-09-20T19:02:24.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:24 smithi142 ceph-mon[32508]: pgmap v624: 148 pgs: 148 active+clean; 296 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 853 B/s rd, 49 MiB/s wr, 787 op/s 2024-09-20T19:02:24.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:24 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:02:24.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:24 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:02:24.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:24 smithi142 ceph-mon[32508]: 5.18 scrub starts 2024-09-20T19:02:24.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:24 smithi142 ceph-mon[32508]: 5.18 scrub ok 2024-09-20T19:02:24.567 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:24.600 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:24.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:24.601 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:24.702 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:24.735 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:24.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:24.736 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:24.810 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T19:02:24.837 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:02:24.866 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 5 16 15969 15953 199.389 172.375 0.00555274 0.00500753 2024-09-20T19:02:24.873 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:24.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:24.874 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:02:24.975 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:25.011 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:25.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:25.012 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:25.113 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:25.142 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:25.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:25.143 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:25.163 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.7 to 1 (10000) 2024-09-20T19:02:25.225 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T19:02:25.243 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:02:25.275 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:25.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:25.276 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:02:25.377 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:25.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:25.415 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:25.415 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:25.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:25 smithi142 ceph-mon[32508]: 6.1 scrub starts 2024-09-20T19:02:25.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:25 smithi142 ceph-mon[32508]: 6.1 scrub ok 2024-09-20T19:02:25.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:25 smithi142 ceph-mon[32508]: osdmap e332: 8 total, 8 up, 8 in 2024-09-20T19:02:25.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:25 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1848881938' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-20T19:02:25.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:25 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-20T19:02:25.516 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:25.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:25 smithi083 ceph-mon[29421]: 6.1 scrub starts 2024-09-20T19:02:25.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:25 smithi083 ceph-mon[29421]: 6.1 scrub ok 2024-09-20T19:02:25.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:25 smithi083 ceph-mon[29421]: osdmap e332: 8 total, 8 up, 8 in 2024-09-20T19:02:25.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:25 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1848881938' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-20T19:02:25.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:25 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-20T19:02:25.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:25 smithi083 ceph-mon[36260]: 6.1 scrub starts 2024-09-20T19:02:25.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:25 smithi083 ceph-mon[36260]: 6.1 scrub ok 2024-09-20T19:02:25.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:25 smithi083 ceph-mon[36260]: osdmap e332: 8 total, 8 up, 8 in 2024-09-20T19:02:25.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:25 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1848881938' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-20T19:02:25.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:25 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-20T19:02:25.557 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:25.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:25.558 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:25.659 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:25.691 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:25.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:25.692 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:25.697 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-20T19:02:25.793 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:25.838 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:25.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:25.839 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:25.865 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 6 16 18611 18595 193.676 165.125 0.00905722 0.00515327 2024-09-20T19:02:25.940 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:25.973 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:25.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:25.974 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:26.075 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:26.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:26.116 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:26.116 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:26.217 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:26.252 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:26.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:26.253 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:26.353 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:26.386 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:26.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:26.387 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:26.488 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:26.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:26 smithi142 ceph-mon[32508]: 4.7 scrub starts 2024-09-20T19:02:26.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:26 smithi142 ceph-mon[32508]: 4.7 scrub ok 2024-09-20T19:02:26.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:26 smithi142 ceph-mon[32508]: 5.a scrub starts 2024-09-20T19:02:26.518 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:26 smithi142 ceph-mon[32508]: 5.a scrub ok 2024-09-20T19:02:26.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:26 smithi142 ceph-mon[32508]: 3.1 scrub starts 2024-09-20T19:02:26.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:26 smithi142 ceph-mon[32508]: 3.1 scrub ok 2024-09-20T19:02:26.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:26 smithi142 ceph-mon[32508]: pgmap v626: 148 pgs: 148 active+clean; 296 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 853 B/s rd, 49 MiB/s wr, 787 op/s 2024-09-20T19:02:26.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:26 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-20T19:02:26.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:26 smithi142 ceph-mon[32508]: osdmap e333: 8 total, 8 up, 8 in 2024-09-20T19:02:26.519 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:26 smithi142 ceph-mon[32508]: osdmap e334: 8 total, 8 up, 8 in 2024-09-20T19:02:26.523 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:26.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:26.524 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:26.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:26 smithi083 ceph-mon[29421]: 4.7 scrub starts 2024-09-20T19:02:26.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:26 smithi083 ceph-mon[29421]: 4.7 scrub ok 2024-09-20T19:02:26.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:26 smithi083 ceph-mon[29421]: 5.a scrub starts 2024-09-20T19:02:26.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:26 smithi083 ceph-mon[29421]: 5.a scrub ok 2024-09-20T19:02:26.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:26 smithi083 ceph-mon[29421]: 3.1 scrub starts 2024-09-20T19:02:26.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:26 smithi083 ceph-mon[29421]: 3.1 scrub ok 2024-09-20T19:02:26.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:26 smithi083 ceph-mon[29421]: pgmap v626: 148 pgs: 148 active+clean; 296 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 853 B/s rd, 49 MiB/s wr, 787 op/s 2024-09-20T19:02:26.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:26 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-20T19:02:26.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:26 smithi083 ceph-mon[29421]: osdmap e333: 8 total, 8 up, 8 in 2024-09-20T19:02:26.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:26 smithi083 ceph-mon[29421]: osdmap e334: 8 total, 8 up, 8 in 2024-09-20T19:02:26.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:26 smithi083 ceph-mon[36260]: 4.7 scrub starts 2024-09-20T19:02:26.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:26 smithi083 ceph-mon[36260]: 4.7 scrub ok 2024-09-20T19:02:26.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:26 smithi083 ceph-mon[36260]: 5.a scrub starts 2024-09-20T19:02:26.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:26 smithi083 ceph-mon[36260]: 5.a scrub ok 2024-09-20T19:02:26.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:26 smithi083 ceph-mon[36260]: 3.1 scrub starts 2024-09-20T19:02:26.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:26 smithi083 ceph-mon[36260]: 3.1 scrub ok 2024-09-20T19:02:26.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:26 smithi083 ceph-mon[36260]: pgmap v626: 148 pgs: 148 active+clean; 296 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 853 B/s rd, 49 MiB/s wr, 787 op/s 2024-09-20T19:02:26.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:26 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-20T19:02:26.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:26 smithi083 ceph-mon[36260]: osdmap e333: 8 total, 8 up, 8 in 2024-09-20T19:02:26.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:26 smithi083 ceph-mon[36260]: osdmap e334: 8 total, 8 up, 8 in 2024-09-20T19:02:26.625 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:02:26.658 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:26.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:26.659 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:02:26.760 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:26.796 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:26.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:26.797 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:26.866 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 7 16 21174 21158 188.89 160.188 0.00481591 0.00528806 2024-09-20T19:02:26.898 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:26.931 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:26.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:26.932 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:27.025 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd reweight 6 1 2024-09-20T19:02:27.033 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:27.071 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:02:27.081 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:02:27.110 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:02:27.151 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:27.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:27.152 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:27.217 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:27 smithi083 ceph-mon[36260]: 4.5 scrub starts 2024-09-20T19:02:27.217 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:27 smithi083 ceph-mon[36260]: 4.5 scrub ok 2024-09-20T19:02:27.217 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:27 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:02:27.217 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:27 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:02:27.218 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:27 smithi083 ceph-mon[29421]: 4.5 scrub starts 2024-09-20T19:02:27.218 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:27 smithi083 ceph-mon[29421]: 4.5 scrub ok 2024-09-20T19:02:27.218 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:27 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:02:27.218 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:27 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:02:27.253 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:27.293 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:27.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:27.294 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:27.395 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:02:27.430 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:27.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:27.431 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:02:27.514 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:02:27 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:02:27.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:27 smithi142 ceph-mon[32508]: 4.5 scrub starts 2024-09-20T19:02:27.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:27 smithi142 ceph-mon[32508]: 4.5 scrub ok 2024-09-20T19:02:27.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:27 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:02:27.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:27 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:02:27.533 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:27.571 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:27.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:27.573 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:27.674 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:27.700 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:27.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:27.701 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:27.802 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:27.838 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:27.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:27.839 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:27.885 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 8 16 23315 23299 182.004 133.812 0.0038965 0.00548759 2024-09-20T19:02:27.939 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:27.973 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:27.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:27.974 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:28.074 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:02:28.108 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:28.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:28.109 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:02:28.209 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:28.245 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:28.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:28.247 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:28.347 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:28.382 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:28.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:28.384 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:28.484 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:28.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:28 smithi142 ceph-mon[32508]: 6.4 scrub starts 2024-09-20T19:02:28.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:28 smithi142 ceph-mon[32508]: 6.4 scrub ok 2024-09-20T19:02:28.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:28 smithi142 ceph-mon[32508]: pgmap v629: 148 pgs: 148 active+clean; 1.1 GiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 977 B/s rd, 174 MiB/s wr, 2.79k op/s 2024-09-20T19:02:28.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:28 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:02:28.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:28 smithi142 ceph-mon[32508]: osdmap e335: 8 total, 8 up, 8 in 2024-09-20T19:02:28.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:28 smithi142 ceph-mon[32508]: 5.10 scrub starts 2024-09-20T19:02:28.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:28 smithi142 ceph-mon[32508]: 5.10 scrub ok 2024-09-20T19:02:28.520 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:28.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:28.521 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:28.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:28 smithi083 ceph-mon[29421]: 6.4 scrub starts 2024-09-20T19:02:28.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:28 smithi083 ceph-mon[29421]: 6.4 scrub ok 2024-09-20T19:02:28.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:28 smithi083 ceph-mon[29421]: pgmap v629: 148 pgs: 148 active+clean; 1.1 GiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 977 B/s rd, 174 MiB/s wr, 2.79k op/s 2024-09-20T19:02:28.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:28 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:02:28.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:28 smithi083 ceph-mon[29421]: osdmap e335: 8 total, 8 up, 8 in 2024-09-20T19:02:28.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:28 smithi083 ceph-mon[29421]: 5.10 scrub starts 2024-09-20T19:02:28.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:28 smithi083 ceph-mon[29421]: 5.10 scrub ok 2024-09-20T19:02:28.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:28 smithi083 ceph-mon[36260]: 6.4 scrub starts 2024-09-20T19:02:28.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:28 smithi083 ceph-mon[36260]: 6.4 scrub ok 2024-09-20T19:02:28.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:28 smithi083 ceph-mon[36260]: pgmap v629: 148 pgs: 148 active+clean; 1.1 GiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 977 B/s rd, 174 MiB/s wr, 2.79k op/s 2024-09-20T19:02:28.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:28 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:02:28.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:28 smithi083 ceph-mon[36260]: osdmap e335: 8 total, 8 up, 8 in 2024-09-20T19:02:28.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:28 smithi083 ceph-mon[36260]: 5.10 scrub starts 2024-09-20T19:02:28.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:28 smithi083 ceph-mon[36260]: 5.10 scrub ok 2024-09-20T19:02:28.622 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:02:28.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:28.643 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:28.643 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:02:28.744 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:28.781 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:28.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:28.782 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:28.866 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 9 16 25632 25616 177.87 144.812 0.00872004 0.0056166 2024-09-20T19:02:28.883 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:28.920 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:28.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:28.921 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:29.022 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:02:29.056 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:29.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:29.057 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:02:29.158 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:29.193 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:29.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:29.194 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:29.203 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-20T19:02:29.295 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:29.329 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:29.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:29.330 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:29.431 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:29.464 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:29.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:29.465 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:29.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:29 smithi142 ceph-mon[32508]: 5.2 scrub starts 2024-09-20T19:02:29.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:29 smithi142 ceph-mon[32508]: 5.2 scrub ok 2024-09-20T19:02:29.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:29 smithi142 ceph-mon[32508]: 5.7 scrub starts 2024-09-20T19:02:29.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:29 smithi142 ceph-mon[32508]: 5.7 scrub ok 2024-09-20T19:02:29.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:29 smithi142 ceph-mon[32508]: 4.f scrub starts 2024-09-20T19:02:29.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:29 smithi142 ceph-mon[32508]: 4.f scrub ok 2024-09-20T19:02:29.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:29 smithi142 ceph-mon[32508]: 6.1c scrub starts 2024-09-20T19:02:29.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:29 smithi142 ceph-mon[32508]: 6.1c scrub ok 2024-09-20T19:02:29.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:29 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2507111170' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:02:29.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:29 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:02:29.533 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:29 smithi083 ceph-mon[29421]: 5.2 scrub starts 2024-09-20T19:02:29.533 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:29 smithi083 ceph-mon[29421]: 5.2 scrub ok 2024-09-20T19:02:29.533 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:29 smithi083 ceph-mon[29421]: 5.7 scrub starts 2024-09-20T19:02:29.533 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:29 smithi083 ceph-mon[29421]: 5.7 scrub ok 2024-09-20T19:02:29.533 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:29 smithi083 ceph-mon[29421]: 4.f scrub starts 2024-09-20T19:02:29.533 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:29 smithi083 ceph-mon[29421]: 4.f scrub ok 2024-09-20T19:02:29.533 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:29 smithi083 ceph-mon[29421]: 6.1c scrub starts 2024-09-20T19:02:29.533 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:29 smithi083 ceph-mon[29421]: 6.1c scrub ok 2024-09-20T19:02:29.533 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:29 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2507111170' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:02:29.533 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:29 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:02:29.534 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:29 smithi083 ceph-mon[36260]: 5.2 scrub starts 2024-09-20T19:02:29.534 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:29 smithi083 ceph-mon[36260]: 5.2 scrub ok 2024-09-20T19:02:29.534 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:29 smithi083 ceph-mon[36260]: 5.7 scrub starts 2024-09-20T19:02:29.534 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:29 smithi083 ceph-mon[36260]: 5.7 scrub ok 2024-09-20T19:02:29.534 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:29 smithi083 ceph-mon[36260]: 4.f scrub starts 2024-09-20T19:02:29.534 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:29 smithi083 ceph-mon[36260]: 4.f scrub ok 2024-09-20T19:02:29.534 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:29 smithi083 ceph-mon[36260]: 6.1c scrub starts 2024-09-20T19:02:29.534 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:29 smithi083 ceph-mon[36260]: 6.1c scrub ok 2024-09-20T19:02:29.534 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:29 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2507111170' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:02:29.534 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:29 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:02:29.567 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:29.588 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:29.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:29.589 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:29.690 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:02:29.723 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:29.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:29.724 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:02:29.825 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:02:29.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:29.863 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:29.864 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:02:29.866 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 10 15 27991 27976 174.832 147.5 0.0060144 0.00571431 2024-09-20T19:02:29.964 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:02:29.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:29.999 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:30.000 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:02:30.100 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:30.135 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:30.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:30.136 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:30.237 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:30.272 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:30.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:30.273 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:30.374 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:30.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:30.410 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:30.410 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:30.511 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:30.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:30 smithi142 ceph-mon[32508]: 6.1f scrub starts 2024-09-20T19:02:30.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:30 smithi142 ceph-mon[32508]: 6.1f scrub ok 2024-09-20T19:02:30.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:30 smithi142 ceph-mon[32508]: pgmap v631: 148 pgs: 148 active+clean; 1.2 GiB data, 6.7 GiB used, 709 GiB / 715 GiB avail; 1.0 KiB/s rd, 187 MiB/s wr, 2.98k op/s 2024-09-20T19:02:30.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:30 smithi142 ceph-mon[32508]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T19:02:30.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:30 smithi142 ceph-mon[32508]: Cluster is now healthy 2024-09-20T19:02:30.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:30 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:02:30.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:30 smithi142 ceph-mon[32508]: osdmap e336: 8 total, 8 up, 8 in 2024-09-20T19:02:30.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:30 smithi142 ceph-mon[32508]: 6.12 scrub starts 2024-09-20T19:02:30.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:30 smithi142 ceph-mon[32508]: 6.12 scrub ok 2024-09-20T19:02:30.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:30 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:02:30.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:30 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:02:30.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:30 smithi083 ceph-mon[29421]: 6.1f scrub starts 2024-09-20T19:02:30.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:30 smithi083 ceph-mon[29421]: 6.1f scrub ok 2024-09-20T19:02:30.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:30 smithi083 ceph-mon[29421]: pgmap v631: 148 pgs: 148 active+clean; 1.2 GiB data, 6.7 GiB used, 709 GiB / 715 GiB avail; 1.0 KiB/s rd, 187 MiB/s wr, 2.98k op/s 2024-09-20T19:02:30.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:30 smithi083 ceph-mon[29421]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T19:02:30.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:30 smithi083 ceph-mon[29421]: Cluster is now healthy 2024-09-20T19:02:30.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:30 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:02:30.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:30 smithi083 ceph-mon[29421]: osdmap e336: 8 total, 8 up, 8 in 2024-09-20T19:02:30.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:30 smithi083 ceph-mon[29421]: 6.12 scrub starts 2024-09-20T19:02:30.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:30 smithi083 ceph-mon[29421]: 6.12 scrub ok 2024-09-20T19:02:30.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:30 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:02:30.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:30 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:02:30.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:30 smithi083 ceph-mon[36260]: 6.1f scrub starts 2024-09-20T19:02:30.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:30 smithi083 ceph-mon[36260]: 6.1f scrub ok 2024-09-20T19:02:30.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:30 smithi083 ceph-mon[36260]: pgmap v631: 148 pgs: 148 active+clean; 1.2 GiB data, 6.7 GiB used, 709 GiB / 715 GiB avail; 1.0 KiB/s rd, 187 MiB/s wr, 2.98k op/s 2024-09-20T19:02:30.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:30 smithi083 ceph-mon[36260]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T19:02:30.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:30 smithi083 ceph-mon[36260]: Cluster is now healthy 2024-09-20T19:02:30.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:30 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:02:30.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:30 smithi083 ceph-mon[36260]: osdmap e336: 8 total, 8 up, 8 in 2024-09-20T19:02:30.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:30 smithi083 ceph-mon[36260]: 6.12 scrub starts 2024-09-20T19:02:30.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:30 smithi083 ceph-mon[36260]: 6.12 scrub ok 2024-09-20T19:02:30.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:30 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:02:30.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:30 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:02:30.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:30.547 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:30.547 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:30.648 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:02:30.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:30.679 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:30.679 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:02:30.780 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:30.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:30.822 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:30.822 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:30.866 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 11 16 30141 30125 171.147 134.312 0.00530354 0.00583663 2024-09-20T19:02:30.909 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:02:30 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:02:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:02:30.923 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:30.962 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:30.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:30.963 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:31.064 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:02:31.084 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:31.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:31.085 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:02:31.186 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:31.221 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:31.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:31.223 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:31.323 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:31.344 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:31.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:31.345 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:31.446 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:31.477 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:31.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:31.478 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:31.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:31 smithi142 ceph-mon[32508]: 3.0 scrub starts 2024-09-20T19:02:31.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:31 smithi142 ceph-mon[32508]: 3.0 scrub ok 2024-09-20T19:02:31.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:31 smithi142 ceph-mon[32508]: osdmap e337: 8 total, 8 up, 8 in 2024-09-20T19:02:31.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:31 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3324771675' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:02:31.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:31 smithi142 ceph-mon[32508]: osdmap e338: 8 total, 8 up, 8 in 2024-09-20T19:02:31.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:31 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:02:31.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:31 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:02:31.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:31 smithi083 ceph-mon[29421]: 3.0 scrub starts 2024-09-20T19:02:31.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:31 smithi083 ceph-mon[29421]: 3.0 scrub ok 2024-09-20T19:02:31.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:31 smithi083 ceph-mon[29421]: osdmap e337: 8 total, 8 up, 8 in 2024-09-20T19:02:31.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:31 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3324771675' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:02:31.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:31 smithi083 ceph-mon[29421]: osdmap e338: 8 total, 8 up, 8 in 2024-09-20T19:02:31.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:31 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:02:31.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:31 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:02:31.544 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:31 smithi083 ceph-mon[36260]: 3.0 scrub starts 2024-09-20T19:02:31.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:31 smithi083 ceph-mon[36260]: 3.0 scrub ok 2024-09-20T19:02:31.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:31 smithi083 ceph-mon[36260]: osdmap e337: 8 total, 8 up, 8 in 2024-09-20T19:02:31.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:31 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3324771675' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:02:31.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:31 smithi083 ceph-mon[36260]: osdmap e338: 8 total, 8 up, 8 in 2024-09-20T19:02:31.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:31 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:02:31.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:31 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:02:31.579 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:02:31.614 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:31.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:31.615 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:02:31.716 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:31.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:31.751 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:31.751 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:31.852 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:31.867 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 12 15 31839 31824 165.731 106.188 0.00356917 0.00602861 2024-09-20T19:02:31.889 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:31.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:31.890 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:31.991 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:32.018 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:32.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:32.020 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:32.058 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:02:32.058 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:02:32.059 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:02:32.059 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:02:32.059 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:02:32.059 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:02:32.059 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:02:32.059 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:02:32.059 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:02:32.059 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:02:32.059 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:02:32.059 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:02:32.059 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:02:32.060 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:02:32.060 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:02:32.060 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:02:32.120 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:32.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:32.142 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:32.142 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:32.243 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:32.279 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:32.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:32.279 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:32.381 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:32.403 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:32.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:32.404 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:32.505 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:32.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:32 smithi142 ceph-mon[32508]: 3.5 scrub starts 2024-09-20T19:02:32.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:32 smithi142 ceph-mon[32508]: 3.5 scrub ok 2024-09-20T19:02:32.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:32 smithi142 ceph-mon[32508]: 4.12 scrub starts 2024-09-20T19:02:32.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:32 smithi142 ceph-mon[32508]: 4.12 scrub ok 2024-09-20T19:02:32.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:32 smithi142 ceph-mon[32508]: pgmap v635: 148 pgs: 148 active+clean; 1.5 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 117 MiB/s wr, 1.87k op/s 2024-09-20T19:02:32.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:32 smithi142 ceph-mon[32508]: 6.9 scrub starts 2024-09-20T19:02:32.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:32 smithi142 ceph-mon[32508]: 6.9 scrub ok 2024-09-20T19:02:32.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:32.539 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:32.539 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:32.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:32 smithi083 ceph-mon[29421]: 3.5 scrub starts 2024-09-20T19:02:32.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:32 smithi083 ceph-mon[29421]: 3.5 scrub ok 2024-09-20T19:02:32.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:32 smithi083 ceph-mon[29421]: 4.12 scrub starts 2024-09-20T19:02:32.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:32 smithi083 ceph-mon[29421]: 4.12 scrub ok 2024-09-20T19:02:32.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:32 smithi083 ceph-mon[29421]: pgmap v635: 148 pgs: 148 active+clean; 1.5 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 117 MiB/s wr, 1.87k op/s 2024-09-20T19:02:32.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:32 smithi083 ceph-mon[29421]: 6.9 scrub starts 2024-09-20T19:02:32.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:32 smithi083 ceph-mon[29421]: 6.9 scrub ok 2024-09-20T19:02:32.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:32 smithi083 ceph-mon[36260]: 3.5 scrub starts 2024-09-20T19:02:32.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:32 smithi083 ceph-mon[36260]: 3.5 scrub ok 2024-09-20T19:02:32.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:32 smithi083 ceph-mon[36260]: 4.12 scrub starts 2024-09-20T19:02:32.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:32 smithi083 ceph-mon[36260]: 4.12 scrub ok 2024-09-20T19:02:32.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:32 smithi083 ceph-mon[36260]: pgmap v635: 148 pgs: 148 active+clean; 1.5 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 117 MiB/s wr, 1.87k op/s 2024-09-20T19:02:32.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:32 smithi083 ceph-mon[36260]: 6.9 scrub starts 2024-09-20T19:02:32.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:32 smithi083 ceph-mon[36260]: 6.9 scrub ok 2024-09-20T19:02:32.640 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:32.673 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:32.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:32.674 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:32.775 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:32.797 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:32.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:32.799 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:32.866 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 13 16 33790 33774 162.357 121.875 0.00499231 0.00615438 2024-09-20T19:02:32.900 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:32.935 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:32.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:32.936 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:33.037 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:33.066 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:33.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:33.067 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:33.168 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:33.202 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:33.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:33.203 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:33.304 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:33.338 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:33.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:33.339 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:33.440 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:33.473 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:33.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:33.474 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:33.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:33 smithi142 ceph-mon[32508]: 3.2 scrub starts 2024-09-20T19:02:33.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:33 smithi142 ceph-mon[32508]: 3.2 scrub ok 2024-09-20T19:02:33.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:33 smithi083 ceph-mon[29421]: 3.2 scrub starts 2024-09-20T19:02:33.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:33 smithi083 ceph-mon[29421]: 3.2 scrub ok 2024-09-20T19:02:33.544 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:33 smithi083 ceph-mon[36260]: 3.2 scrub starts 2024-09-20T19:02:33.544 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:33 smithi083 ceph-mon[36260]: 3.2 scrub ok 2024-09-20T19:02:33.575 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:33.608 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:33.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:33.609 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:33.710 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:33.745 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:33.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:33.746 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:33.847 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:02:33.867 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 14 16 35767 35751 159.585 123.562 0.00198 0.00623333 2024-09-20T19:02:33.868 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:33.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:33.870 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:02:33.970 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:02:34.004 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:34.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:34.005 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:02:34.106 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:34.140 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:34.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:34.142 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:34.242 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:34.279 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:34.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:34.280 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:34.380 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:34.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:34.420 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:34.421 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:34.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:34 smithi142 ceph-mon[32508]: 5.c scrub starts 2024-09-20T19:02:34.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:34 smithi142 ceph-mon[32508]: 5.c scrub ok 2024-09-20T19:02:34.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:34 smithi142 ceph-mon[32508]: pgmap v636: 148 pgs: 148 active+clean; 1.9 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 867 B/s rd, 138 MiB/s wr, 2.20k op/s 2024-09-20T19:02:34.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:34 smithi142 ceph-mon[32508]: 5.1a scrub starts 2024-09-20T19:02:34.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:34 smithi142 ceph-mon[32508]: 5.1a scrub ok 2024-09-20T19:02:34.522 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:34.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:34 smithi083 ceph-mon[29421]: 5.c scrub starts 2024-09-20T19:02:34.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:34 smithi083 ceph-mon[29421]: 5.c scrub ok 2024-09-20T19:02:34.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:34 smithi083 ceph-mon[29421]: pgmap v636: 148 pgs: 148 active+clean; 1.9 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 867 B/s rd, 138 MiB/s wr, 2.20k op/s 2024-09-20T19:02:34.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:34 smithi083 ceph-mon[29421]: 5.1a scrub starts 2024-09-20T19:02:34.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:34 smithi083 ceph-mon[29421]: 5.1a scrub ok 2024-09-20T19:02:34.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:34 smithi083 ceph-mon[36260]: 5.c scrub starts 2024-09-20T19:02:34.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:34 smithi083 ceph-mon[36260]: 5.c scrub ok 2024-09-20T19:02:34.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:34 smithi083 ceph-mon[36260]: pgmap v636: 148 pgs: 148 active+clean; 1.9 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 867 B/s rd, 138 MiB/s wr, 2.20k op/s 2024-09-20T19:02:34.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:34 smithi083 ceph-mon[36260]: 5.1a scrub starts 2024-09-20T19:02:34.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:34 smithi083 ceph-mon[36260]: 5.1a scrub ok 2024-09-20T19:02:34.559 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:34.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:34.560 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:34.661 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:02:34.695 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:34.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:34.696 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:02:34.796 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:34.831 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:34.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:34.832 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:34.866 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 15 16 37846 37830 157.608 129.938 0.00560466 0.0063399 2024-09-20T19:02:34.933 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:34.968 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:34.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:34.969 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:35.070 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:02:35.100 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:35.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:35.101 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:02:35.202 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:02:35.235 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:35.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:35.236 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:02:35.252 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T19:02:35.408 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:02:35.408 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:35.451 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:35.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:35.452 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:35.553 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:35.588 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:35.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:35.589 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:35.690 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:02:35.723 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:35.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:35.724 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:02:35.825 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:35.862 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:35.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:35.863 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:35.866 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 16 15 39936 39921 155.925 130.688 0.00873633 0.00640801 2024-09-20T19:02:35.964 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:36.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:36.000 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:36.001 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:36.101 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:36.141 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:36.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:36.142 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:36.159 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_blocked_ops 2024-09-20T19:02:36.243 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:02:36.275 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:36.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:36.276 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:02:36.377 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:36.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:36.415 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:36.415 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:36.480 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:36 smithi083 ceph-mon[29421]: 3.b scrub starts 2024-09-20T19:02:36.480 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:36 smithi083 ceph-mon[29421]: 3.b scrub ok 2024-09-20T19:02:36.480 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:36 smithi083 ceph-mon[29421]: pgmap v637: 148 pgs: 148 active+clean; 1.9 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 853 B/s rd, 118 MiB/s wr, 1.89k op/s 2024-09-20T19:02:36.481 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:36 smithi083 ceph-mon[29421]: 4.1e scrub starts 2024-09-20T19:02:36.481 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:36 smithi083 ceph-mon[29421]: 4.1e scrub ok 2024-09-20T19:02:36.481 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:36 smithi083 ceph-mon[36260]: 3.b scrub starts 2024-09-20T19:02:36.481 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:36 smithi083 ceph-mon[36260]: 3.b scrub ok 2024-09-20T19:02:36.481 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:36 smithi083 ceph-mon[36260]: pgmap v637: 148 pgs: 148 active+clean; 1.9 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 853 B/s rd, 118 MiB/s wr, 1.89k op/s 2024-09-20T19:02:36.481 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:36 smithi083 ceph-mon[36260]: 4.1e scrub starts 2024-09-20T19:02:36.481 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:36 smithi083 ceph-mon[36260]: 4.1e scrub ok 2024-09-20T19:02:36.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:36 smithi142 ceph-mon[32508]: 3.b scrub starts 2024-09-20T19:02:36.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:36 smithi142 ceph-mon[32508]: 3.b scrub ok 2024-09-20T19:02:36.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:36 smithi142 ceph-mon[32508]: pgmap v637: 148 pgs: 148 active+clean; 1.9 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 853 B/s rd, 118 MiB/s wr, 1.89k op/s 2024-09-20T19:02:36.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:36 smithi142 ceph-mon[32508]: 4.1e scrub starts 2024-09-20T19:02:36.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:36 smithi142 ceph-mon[32508]: 4.1e scrub ok 2024-09-20T19:02:36.515 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:36.553 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:36.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:36.554 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:36.654 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:02:36.689 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:36.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:36.690 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:02:36.791 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:36.828 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:36.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:36.829 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:36.867 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 17 16 41923 41907 154.05 124.125 0.0109049 0.00648512 2024-09-20T19:02:36.929 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:36.947 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:02:36.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:36.977 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:36.977 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:36.980 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:02:37.078 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:37.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:37.115 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:37.115 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:37.216 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:02:37.250 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:37.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:37.251 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:02:37.352 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:37.370 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:37 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:02:37.371 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:37 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:02:37.371 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:37 smithi083 ceph-mon[29421]: 4.2 scrub starts 2024-09-20T19:02:37.371 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:37 smithi083 ceph-mon[29421]: 4.2 scrub ok 2024-09-20T19:02:37.371 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:37 smithi083 ceph-mon[29421]: 6.b scrub starts 2024-09-20T19:02:37.371 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:37 smithi083 ceph-mon[29421]: 6.b scrub ok 2024-09-20T19:02:37.371 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:37 smithi083 ceph-mon[29421]: osdmap e339: 8 total, 8 up, 8 in 2024-09-20T19:02:37.371 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:37 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:02:37.371 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:37 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:02:37.371 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:37 smithi083 ceph-mon[36260]: 4.2 scrub starts 2024-09-20T19:02:37.371 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:37 smithi083 ceph-mon[36260]: 4.2 scrub ok 2024-09-20T19:02:37.372 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:37 smithi083 ceph-mon[36260]: 6.b scrub starts 2024-09-20T19:02:37.372 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:37 smithi083 ceph-mon[36260]: 6.b scrub ok 2024-09-20T19:02:37.372 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:37 smithi083 ceph-mon[36260]: osdmap e339: 8 total, 8 up, 8 in 2024-09-20T19:02:37.410 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:37.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:37.411 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:37.486 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T19:02:37.512 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:37.514 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:02:37 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:02:37.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:37 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:02:37.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:37 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:02:37.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:37 smithi142 ceph-mon[32508]: 4.2 scrub starts 2024-09-20T19:02:37.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:37 smithi142 ceph-mon[32508]: 4.2 scrub ok 2024-09-20T19:02:37.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:37 smithi142 ceph-mon[32508]: 6.b scrub starts 2024-09-20T19:02:37.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:37 smithi142 ceph-mon[32508]: 6.b scrub ok 2024-09-20T19:02:37.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:37 smithi142 ceph-mon[32508]: osdmap e339: 8 total, 8 up, 8 in 2024-09-20T19:02:37.617 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:37.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:37.618 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:37.719 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:02:37.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:37.754 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:37.754 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:02:37.856 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:37.868 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 18 16 43728 43712 151.749 112.812 0.00464707 0.00658395 2024-09-20T19:02:37.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:37.899 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:37.900 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:38.000 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:38.046 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:38.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:38.048 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:38.055 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T19:02:38.056 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:02:38.148 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:38.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:38.171 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:38.172 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:38.263 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.6 to 1 (10000) 2024-09-20T19:02:38.272 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:38.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:38.307 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:38.308 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:38.408 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:38.442 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:38.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:38.443 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:38.543 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:38.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:38 smithi083 ceph-mon[29421]: 4.4 scrub starts 2024-09-20T19:02:38.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:38 smithi083 ceph-mon[29421]: 4.4 scrub ok 2024-09-20T19:02:38.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:38 smithi083 ceph-mon[29421]: pgmap v639: 148 pgs: 148 active+clean; 2.4 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.5 KiB/s rd, 182 MiB/s wr, 2.91k op/s 2024-09-20T19:02:38.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:38 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:02:38.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:38 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3576312187' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-20T19:02:38.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:38 smithi083 ceph-mon[36260]: 4.4 scrub starts 2024-09-20T19:02:38.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:38 smithi083 ceph-mon[36260]: 4.4 scrub ok 2024-09-20T19:02:38.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:38 smithi083 ceph-mon[36260]: pgmap v639: 148 pgs: 148 active+clean; 2.4 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.5 KiB/s rd, 182 MiB/s wr, 2.91k op/s 2024-09-20T19:02:38.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:38 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:02:38.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:38 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3576312187' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-20T19:02:38.578 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:38.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:38.579 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:38.680 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:38.701 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:38.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:38.703 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:38.764 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:38 smithi142 ceph-mon[32508]: 4.4 scrub starts 2024-09-20T19:02:38.764 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:38 smithi142 ceph-mon[32508]: 4.4 scrub ok 2024-09-20T19:02:38.764 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:38 smithi142 ceph-mon[32508]: pgmap v639: 148 pgs: 148 active+clean; 2.4 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.5 KiB/s rd, 182 MiB/s wr, 2.91k op/s 2024-09-20T19:02:38.764 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:38 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:02:38.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:38 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3576312187' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-20T19:02:38.803 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:38.835 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:38.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:38.837 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:38.860 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T19:02:38.868 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 19 16 45189 45173 148.568 91.3125 0.00905997 0.00672609 2024-09-20T19:02:38.938 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:38.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:38.978 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:38.978 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:39.079 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:39.114 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:39.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:39.115 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:39.216 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:39.253 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:39.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:39.254 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:39.355 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:39.388 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:39.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:39.389 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:39.489 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:39.528 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:39.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:39.529 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:39.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:39 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3576312187' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-20T19:02:39.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:39 smithi083 ceph-mon[29421]: osdmap e340: 8 total, 8 up, 8 in 2024-09-20T19:02:39.544 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:39 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3576312187' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-20T19:02:39.544 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:39 smithi083 ceph-mon[36260]: osdmap e340: 8 total, 8 up, 8 in 2024-09-20T19:02:39.630 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:39.651 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_historic_ops 2024-09-20T19:02:39.670 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:39.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:39.671 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:39.764 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:39 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3576312187' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-20T19:02:39.764 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:39 smithi142 ceph-mon[32508]: osdmap e340: 8 total, 8 up, 8 in 2024-09-20T19:02:39.771 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:39.811 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:39.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:39.812 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:39.870 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:2024-09-20T19:02:39.863875+0000 min lat: 0.00125834 max lat: 0.148182 avg lat: 0.00680061 2024-09-20T19:02:39.870 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-20T19:02:39.870 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 20 16 47045 47029 146.93 116 0.0168967 0.00680061 2024-09-20T19:02:39.913 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:39.960 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:39.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:39.962 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:40.062 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:40.100 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:40.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:40.101 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:40.202 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:02:40.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:40.242 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:40.242 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:02:40.343 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:40.399 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-09-20T19:02:40.399 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T19:02:40.408 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:40.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:40.410 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:40.413 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:02:40.510 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:02:40.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:40.543 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:40.543 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:02:40.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:40 smithi083 ceph-mon[29421]: 5.12 scrub starts 2024-09-20T19:02:40.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:40 smithi083 ceph-mon[29421]: 5.12 scrub ok 2024-09-20T19:02:40.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:40 smithi083 ceph-mon[29421]: 6.13 scrub starts 2024-09-20T19:02:40.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:40 smithi083 ceph-mon[29421]: 6.13 scrub ok 2024-09-20T19:02:40.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:40 smithi083 ceph-mon[29421]: pgmap v641: 148 pgs: 148 active+clean; 2.5 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 121 MiB/s wr, 1.93k op/s 2024-09-20T19:02:40.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:40 smithi083 ceph-mon[29421]: 3.17 scrub starts 2024-09-20T19:02:40.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:40 smithi083 ceph-mon[29421]: 3.17 scrub ok 2024-09-20T19:02:40.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:40 smithi083 ceph-mon[36260]: 5.12 scrub starts 2024-09-20T19:02:40.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:40 smithi083 ceph-mon[36260]: 5.12 scrub ok 2024-09-20T19:02:40.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:40 smithi083 ceph-mon[36260]: 6.13 scrub starts 2024-09-20T19:02:40.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:40 smithi083 ceph-mon[36260]: 6.13 scrub ok 2024-09-20T19:02:40.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:40 smithi083 ceph-mon[36260]: pgmap v641: 148 pgs: 148 active+clean; 2.5 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 121 MiB/s wr, 1.93k op/s 2024-09-20T19:02:40.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:40 smithi083 ceph-mon[36260]: 3.17 scrub starts 2024-09-20T19:02:40.547 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:40 smithi083 ceph-mon[36260]: 3.17 scrub ok 2024-09-20T19:02:40.644 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:40.665 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:40 smithi142 ceph-mon[32508]: 5.12 scrub starts 2024-09-20T19:02:40.665 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:40 smithi142 ceph-mon[32508]: 5.12 scrub ok 2024-09-20T19:02:40.665 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:40 smithi142 ceph-mon[32508]: 6.13 scrub starts 2024-09-20T19:02:40.665 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:40 smithi142 ceph-mon[32508]: 6.13 scrub ok 2024-09-20T19:02:40.665 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:40 smithi142 ceph-mon[32508]: pgmap v641: 148 pgs: 148 active+clean; 2.5 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 121 MiB/s wr, 1.93k op/s 2024-09-20T19:02:40.665 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:40 smithi142 ceph-mon[32508]: 3.17 scrub starts 2024-09-20T19:02:40.665 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:40 smithi142 ceph-mon[32508]: 3.17 scrub ok 2024-09-20T19:02:40.678 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:40.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:40.679 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:40.732 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:02:40.780 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:02:40.810 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:40.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:40.811 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:02:40.870 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 21 16 48724 48708 144.93 104.938 0.0123571 0.00689441 2024-09-20T19:02:40.871 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:02:40 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:02:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:02:40.912 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:40.960 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:40.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:40.961 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:41.061 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:41.085 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:02:41.086 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:02:41.110 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:41.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:41.111 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:41.119 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:02:41.120 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:02:41.154 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:02:41.154 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:02:41.184 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:02:41.185 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:02:41.212 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:41.217 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:02:41.217 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:02:41.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:41.247 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:41.247 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:41.250 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:02:41.250 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:02:41.303 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:02:41.304 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:02:41.337 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:02:41.338 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:02:41.348 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:41.385 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:41.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:41.386 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:41.426 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:41 smithi083 ceph-mon[29421]: 4.8 scrub starts 2024-09-20T19:02:41.427 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:41 smithi083 ceph-mon[29421]: 4.8 scrub ok 2024-09-20T19:02:41.427 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:41 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1295333547' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:02:41.427 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:41 smithi083 ceph-mon[36260]: 4.8 scrub starts 2024-09-20T19:02:41.427 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:41 smithi083 ceph-mon[36260]: 4.8 scrub ok 2024-09-20T19:02:41.427 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:41 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1295333547' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:02:41.487 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:41.521 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:41.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:41.522 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:41.622 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:41.655 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:41.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:41.656 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:41.757 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:41.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:41 smithi142 ceph-mon[32508]: 4.8 scrub starts 2024-09-20T19:02:41.764 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:41 smithi142 ceph-mon[32508]: 4.8 scrub ok 2024-09-20T19:02:41.764 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:41 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1295333547' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:02:41.774 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:02:41.783 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:02:41.788 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:41.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:41.789 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:41.870 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 22 16 50303 50287 142.823 98.6875 0.0121586 0.00699669 2024-09-20T19:02:41.889 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:41.932 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:41.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:41.933 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:42.033 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:42.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:42.069 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:42.069 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:42.170 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:42.207 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:42.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:42.208 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:42.309 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:02:42.345 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:42.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:42.346 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:02:42.447 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:42.485 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:42.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:42.486 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:42.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:42 smithi083 ceph-mon[29421]: 4.19 scrub starts 2024-09-20T19:02:42.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:42 smithi083 ceph-mon[29421]: 4.19 scrub ok 2024-09-20T19:02:42.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:42 smithi083 ceph-mon[29421]: pgmap v642: 148 pgs: 148 active+clean; 2.8 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 639 B/s rd, 112 MiB/s wr, 1.78k op/s 2024-09-20T19:02:42.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:42 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3191614776' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:02:42.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:42 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:02:42.544 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:42 smithi083 ceph-mon[36260]: 4.19 scrub starts 2024-09-20T19:02:42.544 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:42 smithi083 ceph-mon[36260]: 4.19 scrub ok 2024-09-20T19:02:42.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:42 smithi083 ceph-mon[36260]: pgmap v642: 148 pgs: 148 active+clean; 2.8 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 639 B/s rd, 112 MiB/s wr, 1.78k op/s 2024-09-20T19:02:42.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:42 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3191614776' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:02:42.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:42 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:02:42.587 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:42.626 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:42.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:42.627 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:42.728 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:02:42.760 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:42.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:42.761 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:02:42.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:42 smithi142 ceph-mon[32508]: 4.19 scrub starts 2024-09-20T19:02:42.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:42 smithi142 ceph-mon[32508]: 4.19 scrub ok 2024-09-20T19:02:42.764 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:42 smithi142 ceph-mon[32508]: pgmap v642: 148 pgs: 148 active+clean; 2.8 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 639 B/s rd, 112 MiB/s wr, 1.78k op/s 2024-09-20T19:02:42.764 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:42 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3191614776' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:02:42.764 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:42 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:02:42.862 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:02:42.873 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 23 16 51967 51951 141.124 104 0.00351933 0.00705933 2024-09-20T19:02:42.873 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-20T19:02:42.884 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:42.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:42.885 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:02:42.986 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:02:43.014 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:43.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:43.015 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:02:43.109 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T19:02:43.109 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-20T19:02:43.109 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-20T19:02:43.109 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-20T19:02:43.109 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.109 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:187757 7.e 7:70705a0e:::benchmark_data_smithi083_70643_object64955:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-20T19:02:43.109 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:53:29.029730+0000", 2024-09-20T19:02:43.109 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 554.06740414299998, 2024-09-20T19:02:43.109 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.137882645, 2024-09-20T19:02:43.110 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:43.110 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:43.110 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:02:43.110 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:02:43.110 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T19:02:43.110 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T19:02:43.110 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 187757 2024-09-20T19:02:43.110 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.110 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:43.110 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.110 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:43.110 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.029730+0000", 2024-09-20T19:02:43.110 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:43.111 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.111 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.111 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:43.111 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.029730+0000", 2024-09-20T19:02:43.111 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:43.111 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.111 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.111 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:43.111 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.029732+0000", 2024-09-20T19:02:43.111 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.415e-06 2024-09-20T19:02:43.111 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.111 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.111 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:43.112 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.029737+0000", 2024-09-20T19:02:43.112 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0239999999999997e-06 2024-09-20T19:02:43.112 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.112 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.112 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:43.112 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.029748+0000", 2024-09-20T19:02:43.112 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0920999999999999e-05 2024-09-20T19:02:43.112 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.112 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.112 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:43.112 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.029755+0000", 2024-09-20T19:02:43.112 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1430000000000001e-06 2024-09-20T19:02:43.112 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.112 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.113 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:43.113 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.030815+0000", 2024-09-20T19:02:43.113 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0010603590000000001 2024-09-20T19:02:43.113 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.113 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.113 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:43.113 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.030930+0000", 2024-09-20T19:02:43.113 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000114171 2024-09-20T19:02:43.113 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.113 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.113 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T19:02:43.113 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.031080+0000", 2024-09-20T19:02:43.113 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015042000000000001 2024-09-20T19:02:43.114 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.114 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.114 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:02:43.114 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.036072+0000", 2024-09-20T19:02:43.114 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049921159999999996 2024-09-20T19:02:43.114 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.114 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.114 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:43.114 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.037324+0000", 2024-09-20T19:02:43.114 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0012515930000000001 2024-09-20T19:02:43.114 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.114 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.114 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:43.114 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045902+0000", 2024-09-20T19:02:43.115 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0085779580000000001 2024-09-20T19:02:43.115 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.115 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.115 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:02:43.115 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.167564+0000", 2024-09-20T19:02:43.115 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.121662143 2024-09-20T19:02:43.115 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.115 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.115 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:43.115 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.167613+0000", 2024-09-20T19:02:43.115 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9382000000000003e-05 2024-09-20T19:02:43.115 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:43.115 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:43.115 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:43.116 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.116 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.116 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:187765 7.e 7:7e1cec20:::benchmark_data_smithi083_70643_object64963:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-20T19:02:43.116 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:53:29.040056+0000", 2024-09-20T19:02:43.116 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 554.05707905700001, 2024-09-20T19:02:43.116 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.132718637, 2024-09-20T19:02:43.116 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:43.116 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:43.116 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:02:43.116 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:02:43.117 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T19:02:43.117 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T19:02:43.117 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 187765 2024-09-20T19:02:43.117 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.117 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:43.117 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.117 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:43.117 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.040056+0000", 2024-09-20T19:02:43.117 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:43.117 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.117 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.117 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:43.117 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.040056+0000", 2024-09-20T19:02:43.117 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:43.118 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.118 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.118 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:43.118 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.040057+0000", 2024-09-20T19:02:43.118 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.528e-06 2024-09-20T19:02:43.118 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.118 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.118 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:43.118 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.040063+0000", 2024-09-20T19:02:43.118 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8170000000000004e-06 2024-09-20T19:02:43.118 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.118 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.118 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:43.118 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.040074+0000", 2024-09-20T19:02:43.119 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1253000000000001e-05 2024-09-20T19:02:43.119 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.119 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.119 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:43.119 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.040081+0000", 2024-09-20T19:02:43.119 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1389999999999998e-06 2024-09-20T19:02:43.119 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.119 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.119 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:43.119 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.040134+0000", 2024-09-20T19:02:43.119 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2549000000000001e-05 2024-09-20T19:02:43.119 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.119 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.120 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:43.120 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.040246+0000", 2024-09-20T19:02:43.120 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000112271 2024-09-20T19:02:43.120 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.120 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.120 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T19:02:43.120 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.040354+0000", 2024-09-20T19:02:43.120 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010791 2024-09-20T19:02:43.120 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.120 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.120 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:43.120 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045867+0000", 2024-09-20T19:02:43.120 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0055125449999999998 2024-09-20T19:02:43.120 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.121 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.121 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:02:43.121 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.172613+0000", 2024-09-20T19:02:43.121 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.12674679599999999 2024-09-20T19:02:43.121 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.121 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.121 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:43.121 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.172691+0000", 2024-09-20T19:02:43.121 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7255000000000005e-05 2024-09-20T19:02:43.121 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.122 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.122 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:02:43.122 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.172735+0000", 2024-09-20T19:02:43.122 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4069000000000001e-05 2024-09-20T19:02:43.122 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.122 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.122 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:43.122 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.172774+0000", 2024-09-20T19:02:43.122 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9505000000000002e-05 2024-09-20T19:02:43.122 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:43.122 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:43.122 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:43.122 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.123 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.123 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:187763 7.3 e141/83 7:c71c27bb:::benchmark_data_smithi083_70643_object64961:head v 141'11587, mlcod=141'11584)", 2024-09-20T19:02:43.123 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:53:29.043149+0000", 2024-09-20T19:02:43.123 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 554.05398537300005, 2024-09-20T19:02:43.123 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.125479285, 2024-09-20T19:02:43.123 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:43.123 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:43.123 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:02:43.123 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:43.123 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.123 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:43.123 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043149+0000", 2024-09-20T19:02:43.123 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:43.123 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.123 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.124 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:43.124 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043149+0000", 2024-09-20T19:02:43.124 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:43.124 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.124 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.124 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:43.124 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043150+0000", 2024-09-20T19:02:43.124 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7800000000000001e-07 2024-09-20T19:02:43.124 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.124 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.124 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:43.124 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043154+0000", 2024-09-20T19:02:43.124 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6480000000000002e-06 2024-09-20T19:02:43.124 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.125 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.125 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:43.125 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043162+0000", 2024-09-20T19:02:43.125 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1810000000000004e-06 2024-09-20T19:02:43.125 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.125 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.125 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:43.125 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043167+0000", 2024-09-20T19:02:43.125 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4500000000000003e-06 2024-09-20T19:02:43.125 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.125 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.125 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:43.125 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043227+0000", 2024-09-20T19:02:43.125 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9697999999999997e-05 2024-09-20T19:02:43.126 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.126 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.126 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:43.126 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043243+0000", 2024-09-20T19:02:43.126 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6461999999999999e-05 2024-09-20T19:02:43.126 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.126 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.126 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:02:43.126 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.168513+0000", 2024-09-20T19:02:43.126 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.125269624 2024-09-20T19:02:43.126 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.126 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.126 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:43.126 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.168629+0000", 2024-09-20T19:02:43.127 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011564400000000001 2024-09-20T19:02:43.127 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:43.127 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:43.127 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:43.127 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.127 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.127 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:187773 7.e 7:7168b707:::benchmark_data_smithi083_70643_object64971:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-20T19:02:43.127 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:53:29.043782+0000", 2024-09-20T19:02:43.127 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 554.05335264500002, 2024-09-20T19:02:43.127 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.14065076300000001, 2024-09-20T19:02:43.127 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:43.127 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:43.127 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:02:43.127 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:02:43.128 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T19:02:43.128 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T19:02:43.128 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 187773 2024-09-20T19:02:43.128 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.128 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:43.128 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.128 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:43.128 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043782+0000", 2024-09-20T19:02:43.128 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:43.128 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.128 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.128 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:43.128 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043782+0000", 2024-09-20T19:02:43.128 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:43.128 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.129 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.129 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:43.129 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043784+0000", 2024-09-20T19:02:43.129 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5260000000000001e-06 2024-09-20T19:02:43.129 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.129 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.129 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:43.129 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043788+0000", 2024-09-20T19:02:43.129 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.481e-06 2024-09-20T19:02:43.129 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.129 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.129 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:43.129 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043798+0000", 2024-09-20T19:02:43.130 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0393e-05 2024-09-20T19:02:43.130 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.130 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.130 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:43.130 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.043805+0000", 2024-09-20T19:02:43.130 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5030000000000002e-06 2024-09-20T19:02:43.130 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.130 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.130 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:43.130 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.167654+0000", 2024-09-20T19:02:43.130 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.123848659 2024-09-20T19:02:43.130 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.130 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.130 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:43.130 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.167881+0000", 2024-09-20T19:02:43.131 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000227698 2024-09-20T19:02:43.131 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.131 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.131 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T19:02:43.131 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.167981+0000", 2024-09-20T19:02:43.131 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9611999999999994e-05 2024-09-20T19:02:43.131 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.131 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.131 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:43.131 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.172817+0000", 2024-09-20T19:02:43.131 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0048360829999999997 2024-09-20T19:02:43.131 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.131 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.131 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:02:43.132 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.172828+0000", 2024-09-20T19:02:43.132 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1280999999999999e-05 2024-09-20T19:02:43.132 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.132 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.132 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:43.132 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.184218+0000", 2024-09-20T19:02:43.132 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011389882 2024-09-20T19:02:43.132 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.132 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.132 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:02:43.132 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.184387+0000", 2024-09-20T19:02:43.132 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000168992 2024-09-20T19:02:43.132 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.132 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.132 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:43.133 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.184433+0000", 2024-09-20T19:02:43.133 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5652999999999998e-05 2024-09-20T19:02:43.133 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:43.133 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:43.133 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:43.133 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.133 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.133 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:187757 7.e e141/83 ondisk, result = 0)", 2024-09-20T19:02:43.133 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:53:29.045594+0000", 2024-09-20T19:02:43.133 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 554.05154069100001, 2024-09-20T19:02:43.133 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.12203122600000001, 2024-09-20T19:02:43.133 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:43.133 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:43.133 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T19:02:43.134 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:43.134 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.134 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:43.134 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045594+0000", 2024-09-20T19:02:43.134 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:43.134 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.134 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.134 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:43.134 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045594+0000", 2024-09-20T19:02:43.134 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:43.134 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.134 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.134 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:43.134 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045595+0000", 2024-09-20T19:02:43.135 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6499999999999998e-07 2024-09-20T19:02:43.135 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.135 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.135 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:43.135 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045599+0000", 2024-09-20T19:02:43.135 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6160000000000004e-06 2024-09-20T19:02:43.135 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.135 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.135 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:43.135 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045609+0000", 2024-09-20T19:02:43.135 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2269999999999999e-06 2024-09-20T19:02:43.135 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.135 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.135 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:43.135 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045615+0000", 2024-09-20T19:02:43.136 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1349999999999997e-06 2024-09-20T19:02:43.136 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.136 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.136 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:43.136 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045892+0000", 2024-09-20T19:02:43.136 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000276724 2024-09-20T19:02:43.136 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.136 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.136 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:43.136 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045897+0000", 2024-09-20T19:02:43.136 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2970000000000003e-06 2024-09-20T19:02:43.136 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.136 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.137 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:43.137 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.167625+0000", 2024-09-20T19:02:43.137 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.12172836200000001 2024-09-20T19:02:43.137 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:43.137 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:43.137 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:43.137 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.137 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.137 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.14808.0:187772 7.a e141/83 7:5948c6a4:::benchmark_data_smithi083_70643_object64970:head v 141'11671, mlcod=141'11669)", 2024-09-20T19:02:43.137 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:43.139 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:53:29.045936+0000", 2024-09-20T19:02:43.139 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 554.05119899500005, 2024-09-20T19:02:43.139 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.12458983799999999, 2024-09-20T19:02:43.139 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:43.139 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:43.139 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:02:43.139 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:43.139 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.139 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:43.139 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045936+0000", 2024-09-20T19:02:43.139 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:43.139 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.140 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.140 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:43.140 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045936+0000", 2024-09-20T19:02:43.140 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:43.140 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.140 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.140 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:43.140 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045936+0000", 2024-09-20T19:02:43.140 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2900000000000003e-07 2024-09-20T19:02:43.140 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.140 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.140 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:43.140 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.045942+0000", 2024-09-20T19:02:43.140 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.66e-06 2024-09-20T19:02:43.141 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.141 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.141 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:43.141 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.165480+0000", 2024-09-20T19:02:43.141 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.119537582 2024-09-20T19:02:43.141 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.141 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.141 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:43.141 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.165496+0000", 2024-09-20T19:02:43.141 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6249999999999999e-05 2024-09-20T19:02:43.141 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.141 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.141 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:43.141 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.165558+0000", 2024-09-20T19:02:43.141 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2120999999999997e-05 2024-09-20T19:02:43.142 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.142 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.142 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:43.142 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.165582+0000", 2024-09-20T19:02:43.142 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4409e-05 2024-09-20T19:02:43.142 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.142 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.142 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:02:43.142 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.170489+0000", 2024-09-20T19:02:43.142 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0049066860000000004 2024-09-20T19:02:43.142 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.142 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.142 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:43.142 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.170525+0000", 2024-09-20T19:02:43.142 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6400999999999999e-05 2024-09-20T19:02:43.143 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:43.143 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:43.143 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:43.143 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.143 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.143 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14808.0:187774 7.e 7:778445cb:::benchmark_data_smithi083_70643_object64972:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-20T19:02:43.143 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:53:29.046524+0000", 2024-09-20T19:02:43.143 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 554.05061057, 2024-09-20T19:02:43.143 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.15650777099999999, 2024-09-20T19:02:43.143 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:43.143 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:43.143 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:02:43.143 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:02:43.144 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14808", 2024-09-20T19:02:43.144 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1779458981", 2024-09-20T19:02:43.144 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 187774 2024-09-20T19:02:43.144 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.144 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:43.144 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.144 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:43.144 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.046524+0000", 2024-09-20T19:02:43.144 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:43.144 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.144 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.144 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:43.144 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.046524+0000", 2024-09-20T19:02:43.144 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:43.144 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.145 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.145 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:43.145 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.046526+0000", 2024-09-20T19:02:43.145 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.494e-06 2024-09-20T19:02:43.145 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.145 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.145 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:43.145 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.046531+0000", 2024-09-20T19:02:43.145 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4840000000000003e-06 2024-09-20T19:02:43.145 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.145 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.145 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:43.145 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.165656+0000", 2024-09-20T19:02:43.146 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.11912532100000001 2024-09-20T19:02:43.146 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.146 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.146 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:43.146 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.165666+0000", 2024-09-20T19:02:43.146 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1789999999999997e-06 2024-09-20T19:02:43.146 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.146 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.146 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:43.146 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.173061+0000", 2024-09-20T19:02:43.146 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0073953969999999997 2024-09-20T19:02:43.146 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.146 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.146 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:43.147 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.173154+0000", 2024-09-20T19:02:43.147 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3113000000000001e-05 2024-09-20T19:02:43.147 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.147 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.147 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T19:02:43.147 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.173250+0000", 2024-09-20T19:02:43.147 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5493000000000002e-05 2024-09-20T19:02:43.147 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.147 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.147 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:43.147 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.180360+0000", 2024-09-20T19:02:43.147 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0071109220000000004 2024-09-20T19:02:43.147 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.147 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.148 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:02:43.148 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.188345+0000", 2024-09-20T19:02:43.148 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0079849840000000005 2024-09-20T19:02:43.148 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.148 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.148 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:43.148 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.202813+0000", 2024-09-20T19:02:43.148 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.014467647 2024-09-20T19:02:43.148 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.148 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.148 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:02:43.148 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.202985+0000", 2024-09-20T19:02:43.148 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017191999999999999 2024-09-20T19:02:43.148 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.149 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.149 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:43.149 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.203032+0000", 2024-09-20T19:02:43.149 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6817000000000003e-05 2024-09-20T19:02:43.149 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:43.149 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:43.149 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:43.149 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.149 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.149 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.14808.0:187765 7.e e141/83 ondisk, result = 0)", 2024-09-20T19:02:43.149 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:53:29.046987+0000", 2024-09-20T19:02:43.149 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 554.05014757100002, 2024-09-20T19:02:43.149 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.12579221700000001, 2024-09-20T19:02:43.149 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:43.149 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:43.150 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T19:02:43.150 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:43.150 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.150 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:43.150 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.046987+0000", 2024-09-20T19:02:43.150 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:43.150 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.150 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.150 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:43.150 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.046987+0000", 2024-09-20T19:02:43.150 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:43.150 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.150 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.150 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:43.151 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.046988+0000", 2024-09-20T19:02:43.151 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6499999999999999e-07 2024-09-20T19:02:43.151 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.151 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.151 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:43.151 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.046992+0000", 2024-09-20T19:02:43.151 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9650000000000002e-06 2024-09-20T19:02:43.151 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.151 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.151 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:43.151 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.165597+0000", 2024-09-20T19:02:43.151 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.118605087 2024-09-20T19:02:43.151 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.152 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.152 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:43.152 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.165603+0000", 2024-09-20T19:02:43.152 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0100000000000001e-06 2024-09-20T19:02:43.152 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.152 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.152 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:43.152 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.172679+0000", 2024-09-20T19:02:43.152 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0070759810000000003 2024-09-20T19:02:43.152 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.152 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.152 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:43.152 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.172686+0000", 2024-09-20T19:02:43.152 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9959999999999996e-06 2024-09-20T19:02:43.153 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.153 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.153 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:43.153 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:53:29.172779+0000", 2024-09-20T19:02:43.153 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3512999999999997e-05 2024-09-20T19:02:43.153 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:43.153 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:43.153 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:43.153 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.153 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.153 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.24955.0:218218 8.6 e233/182 8:6660910c:::benchmark_data_smithi083_97422_object102127:head v 233'13570, mlcod=210'8949)", 2024-09-20T19:02:43.153 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:58:02.122025+0000", 2024-09-20T19:02:43.153 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 280.97510942500003, 2024-09-20T19:02:43.153 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.112690573, 2024-09-20T19:02:43.154 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:43.154 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:43.154 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:02:43.154 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:43.154 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.154 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:43.154 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.122025+0000", 2024-09-20T19:02:43.154 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:43.154 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.154 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.154 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:43.154 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.122025+0000", 2024-09-20T19:02:43.154 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:43.155 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.155 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.155 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:43.155 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.122026+0000", 2024-09-20T19:02:43.155 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2399999999999996e-07 2024-09-20T19:02:43.155 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.155 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.155 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:43.155 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.122033+0000", 2024-09-20T19:02:43.155 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.562e-06 2024-09-20T19:02:43.155 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.155 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.155 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:43.155 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.214763+0000", 2024-09-20T19:02:43.156 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.092730564000000001 2024-09-20T19:02:43.156 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.156 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.156 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:43.156 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.214772+0000", 2024-09-20T19:02:43.156 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7979999999999995e-06 2024-09-20T19:02:43.156 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.156 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.156 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:43.156 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.214856+0000", 2024-09-20T19:02:43.156 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4146000000000004e-05 2024-09-20T19:02:43.156 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.156 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.157 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:43.157 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.214878+0000", 2024-09-20T19:02:43.157 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1722e-05 2024-09-20T19:02:43.157 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.157 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.157 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:02:43.157 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.234612+0000", 2024-09-20T19:02:43.157 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019734267 2024-09-20T19:02:43.157 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.157 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.157 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:43.157 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.234716+0000", 2024-09-20T19:02:43.157 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010359 2024-09-20T19:02:43.158 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:43.158 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:43.158 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:43.158 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.158 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.158 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.25484.0:73676 9.2 e265/246 9:4d342ee9:::benchmark_data_smithi083_124552_object73675:head v 265'4551, mlcod=265'4548)", 2024-09-20T19:02:43.158 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:59:27.940611+0000", 2024-09-20T19:02:43.158 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 195.15652373, 2024-09-20T19:02:43.158 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.10976248500000001, 2024-09-20T19:02:43.158 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:43.158 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:43.158 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:02:43.158 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:43.158 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.159 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:43.159 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:27.940611+0000", 2024-09-20T19:02:43.159 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:43.159 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.159 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.159 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:43.159 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:27.940611+0000", 2024-09-20T19:02:43.159 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:43.159 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.159 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.159 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:43.159 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:27.940612+0000", 2024-09-20T19:02:43.159 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.271e-06 2024-09-20T19:02:43.159 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.160 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.160 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:43.160 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:27.940637+0000", 2024-09-20T19:02:43.160 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5219999999999999e-05 2024-09-20T19:02:43.160 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.160 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.160 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:43.160 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:28.036375+0000", 2024-09-20T19:02:43.160 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.095737696999999997 2024-09-20T19:02:43.160 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.160 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.160 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:43.160 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:28.036388+0000", 2024-09-20T19:02:43.160 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3001e-05 2024-09-20T19:02:43.161 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.161 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.161 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:43.161 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:28.038717+0000", 2024-09-20T19:02:43.161 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0023287429999999999 2024-09-20T19:02:43.161 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.161 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.161 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:43.161 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:28.038864+0000", 2024-09-20T19:02:43.161 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014680800000000001 2024-09-20T19:02:43.161 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.161 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.161 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:02:43.161 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:28.050214+0000", 2024-09-20T19:02:43.161 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011350813 2024-09-20T19:02:43.162 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.162 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.162 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:43.162 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:28.050373+0000", 2024-09-20T19:02:43.162 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000158932 2024-09-20T19:02:43.162 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:43.162 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:43.162 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:43.162 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.162 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.162 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.25484.0:124578 9.2 e289/246 9:422abb8a:::benchmark_data_smithi083_124552_object10431:head v 289'7686, mlcod=289'7682)", 2024-09-20T19:02:43.162 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:20.951824+0000", 2024-09-20T19:02:43.162 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 142.14531088300001, 2024-09-20T19:02:43.162 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.32891357100000002, 2024-09-20T19:02:43.162 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:43.163 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:43.163 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:02:43.163 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:43.163 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.163 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:43.163 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.951824+0000", 2024-09-20T19:02:43.163 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:43.163 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.163 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.163 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:43.163 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.951824+0000", 2024-09-20T19:02:43.163 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:43.163 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.163 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.164 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:43.164 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.951825+0000", 2024-09-20T19:02:43.164 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0470000000000001e-06 2024-09-20T19:02:43.164 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.164 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.164 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:43.164 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.951831+0000", 2024-09-20T19:02:43.164 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2999999999999998e-06 2024-09-20T19:02:43.164 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.164 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.164 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:43.164 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.951841+0000", 2024-09-20T19:02:43.164 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0234999999999999e-05 2024-09-20T19:02:43.164 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.165 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.165 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:43.165 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.951854+0000", 2024-09-20T19:02:43.165 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.287e-05 2024-09-20T19:02:43.165 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.165 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.165 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:43.165 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.951926+0000", 2024-09-20T19:02:43.165 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1932000000000002e-05 2024-09-20T19:02:43.165 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.165 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.165 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:43.165 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.951943+0000", 2024-09-20T19:02:43.165 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7033000000000001e-05 2024-09-20T19:02:43.165 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.166 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.166 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:02:43.166 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.280700+0000", 2024-09-20T19:02:43.166 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.328756979 2024-09-20T19:02:43.166 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.166 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.166 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:43.166 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.280737+0000", 2024-09-20T19:02:43.166 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7175000000000002e-05 2024-09-20T19:02:43.166 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:43.166 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:43.166 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:43.166 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.167 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.168 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.25484.0:124584 9.0 e289/284 9:06443903:::benchmark_data_smithi083_124552_object10437:head v 289'8049, mlcod=289'8046)", 2024-09-20T19:02:43.168 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:20.953013+0000", 2024-09-20T19:02:43.168 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 142.14412157000001, 2024-09-20T19:02:43.168 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.323892601, 2024-09-20T19:02:43.168 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:43.168 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:43.169 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:02:43.169 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:43.169 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.169 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:43.169 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.953013+0000", 2024-09-20T19:02:43.169 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:43.169 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.169 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.169 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:43.169 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.953013+0000", 2024-09-20T19:02:43.170 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:43.170 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.170 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.170 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:43.170 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.953014+0000", 2024-09-20T19:02:43.170 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0999999999999997e-07 2024-09-20T19:02:43.170 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.170 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.170 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:43.170 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.953020+0000", 2024-09-20T19:02:43.170 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8390000000000003e-06 2024-09-20T19:02:43.170 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.170 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.171 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:43.171 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.953030+0000", 2024-09-20T19:02:43.171 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8530000000000004e-06 2024-09-20T19:02:43.171 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.171 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.171 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:43.171 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.953036+0000", 2024-09-20T19:02:43.171 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1510000000000001e-06 2024-09-20T19:02:43.171 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.171 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.171 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:43.171 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.953099+0000", 2024-09-20T19:02:43.171 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2959000000000005e-05 2024-09-20T19:02:43.171 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.172 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.172 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:43.172 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.953117+0000", 2024-09-20T19:02:43.172 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8559999999999998e-05 2024-09-20T19:02:43.172 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.172 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.172 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:02:43.172 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.959129+0000", 2024-09-20T19:02:43.172 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0060115500000000001 2024-09-20T19:02:43.172 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.172 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.172 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:43.172 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.276906+0000", 2024-09-20T19:02:43.172 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.31777687900000001 2024-09-20T19:02:43.173 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:43.173 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:43.173 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:43.173 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.173 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.173 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.25484.0:124590 9.6 e289/246 9:6ce021c9:::benchmark_data_smithi083_124552_object10443:head v 289'7744, mlcod=289'7742)", 2024-09-20T19:02:43.173 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:20.956000+0000", 2024-09-20T19:02:43.173 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 142.14113433, 2024-09-20T19:02:43.173 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.32402082500000001, 2024-09-20T19:02:43.173 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:43.173 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:43.173 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:02:43.173 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:43.173 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.173 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:43.174 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.956000+0000", 2024-09-20T19:02:43.174 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:43.174 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.174 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.174 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:43.174 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.956000+0000", 2024-09-20T19:02:43.174 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:43.174 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.174 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.174 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:43.174 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.956001+0000", 2024-09-20T19:02:43.174 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9600000000000008e-07 2024-09-20T19:02:43.174 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.174 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.175 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:43.175 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.956007+0000", 2024-09-20T19:02:43.175 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3199999999999999e-06 2024-09-20T19:02:43.175 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.175 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.175 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:43.175 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.956014+0000", 2024-09-20T19:02:43.175 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7880000000000007e-06 2024-09-20T19:02:43.175 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.175 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.175 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:43.175 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.956019+0000", 2024-09-20T19:02:43.175 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5050000000000001e-06 2024-09-20T19:02:43.175 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.176 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.176 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:43.176 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.956069+0000", 2024-09-20T19:02:43.176 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0467000000000003e-05 2024-09-20T19:02:43.176 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.176 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.176 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:43.176 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.956089+0000", 2024-09-20T19:02:43.176 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.948e-05 2024-09-20T19:02:43.176 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.176 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.176 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:02:43.176 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.279983+0000", 2024-09-20T19:02:43.176 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.32389452099999999 2024-09-20T19:02:43.177 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.177 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.177 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:43.177 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.280021+0000", 2024-09-20T19:02:43.177 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7747999999999998e-05 2024-09-20T19:02:43.177 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:43.177 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:43.177 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:43.177 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.177 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.177 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.25484.0:124594 9.6 e289/246 9:67fbc174:::benchmark_data_smithi083_124552_object10447:head v 289'7745, mlcod=289'7742)", 2024-09-20T19:02:43.177 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:20.960552+0000", 2024-09-20T19:02:43.177 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 142.13658248900001, 2024-09-20T19:02:43.177 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.32917961699999998, 2024-09-20T19:02:43.178 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:43.178 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:43.178 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:02:43.178 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:43.178 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.178 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:43.178 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.960552+0000", 2024-09-20T19:02:43.178 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:43.178 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.178 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.178 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:43.178 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.960552+0000", 2024-09-20T19:02:43.178 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:43.178 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.179 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.179 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:43.179 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.960553+0000", 2024-09-20T19:02:43.179 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0610000000000001e-06 2024-09-20T19:02:43.179 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.179 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.179 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:43.179 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.960562+0000", 2024-09-20T19:02:43.179 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0580000000000004e-06 2024-09-20T19:02:43.179 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.179 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.179 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:43.179 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.276862+0000", 2024-09-20T19:02:43.179 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.31629938200000002 2024-09-20T19:02:43.180 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.180 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.180 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:43.180 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.276872+0000", 2024-09-20T19:02:43.180 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0614e-05 2024-09-20T19:02:43.180 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.180 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.180 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:43.180 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.276943+0000", 2024-09-20T19:02:43.180 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0933999999999995e-05 2024-09-20T19:02:43.180 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.180 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.180 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:43.180 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.276962+0000", 2024-09-20T19:02:43.180 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9293999999999999e-05 2024-09-20T19:02:43.181 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.181 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.181 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:02:43.181 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.289671+0000", 2024-09-20T19:02:43.181 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.012708129 2024-09-20T19:02:43.181 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.181 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.181 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:43.181 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.289732+0000", 2024-09-20T19:02:43.181 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1144999999999998e-05 2024-09-20T19:02:43.181 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:43.181 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:43.181 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:43.181 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.182 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.182 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.25484.0:124596 9.2 e289/246 9:42d01fa2:::benchmark_data_smithi083_124552_object10449:head v 289'7687, mlcod=289'7684)", 2024-09-20T19:02:43.182 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:20.963075+0000", 2024-09-20T19:02:43.182 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 142.134059734, 2024-09-20T19:02:43.182 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.32681833599999999, 2024-09-20T19:02:43.182 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:43.182 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:43.182 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:02:43.182 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:43.182 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.182 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:43.182 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.963075+0000", 2024-09-20T19:02:43.182 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:43.182 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.183 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.183 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:43.183 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.963075+0000", 2024-09-20T19:02:43.183 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:43.183 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.183 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.183 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:43.183 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.963076+0000", 2024-09-20T19:02:43.183 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6600000000000005e-07 2024-09-20T19:02:43.183 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.183 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.183 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:43.183 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.963083+0000", 2024-09-20T19:02:43.183 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9140000000000002e-06 2024-09-20T19:02:43.184 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.184 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.184 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:43.184 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.276845+0000", 2024-09-20T19:02:43.184 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.31376241599999999 2024-09-20T19:02:43.184 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.184 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.184 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:43.184 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.276871+0000", 2024-09-20T19:02:43.184 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6123999999999999e-05 2024-09-20T19:02:43.184 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.184 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.184 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:43.184 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.276934+0000", 2024-09-20T19:02:43.185 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3147999999999999e-05 2024-09-20T19:02:43.185 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.185 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.185 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:43.185 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.276960+0000", 2024-09-20T19:02:43.185 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5939e-05 2024-09-20T19:02:43.185 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.185 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.185 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:02:43.185 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.289643+0000", 2024-09-20T19:02:43.185 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.012682236 2024-09-20T19:02:43.185 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.185 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.185 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:43.185 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.289893+0000", 2024-09-20T19:02:43.186 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00025069300000000002 2024-09-20T19:02:43.186 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:43.186 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:43.186 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:43.186 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.186 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.186 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.25484.0:210426 9.2 e312/246 9:4c6f61c8:::benchmark_data_smithi083_124552_object96279:head v 312'12986, mlcod=312'12983)", 2024-09-20T19:02:43.186 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:01:34.555445+0000", 2024-09-20T19:02:43.186 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 68.541689547999994, 2024-09-20T19:02:43.186 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.196926772, 2024-09-20T19:02:43.186 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:43.186 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:43.186 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:02:43.186 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:43.187 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.187 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:43.187 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:01:34.555445+0000", 2024-09-20T19:02:43.187 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:43.187 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.187 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.187 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:43.187 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:01:34.555445+0000", 2024-09-20T19:02:43.187 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:43.187 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.187 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.187 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:43.187 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:01:34.555446+0000", 2024-09-20T19:02:43.187 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.023e-06 2024-09-20T19:02:43.188 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.188 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.188 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:43.188 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:01:34.555453+0000", 2024-09-20T19:02:43.188 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4910000000000002e-06 2024-09-20T19:02:43.188 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.188 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.188 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:43.188 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:01:34.741524+0000", 2024-09-20T19:02:43.188 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.18607137300000001 2024-09-20T19:02:43.188 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.188 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.188 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:43.188 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:01:34.741534+0000", 2024-09-20T19:02:43.188 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0219e-05 2024-09-20T19:02:43.189 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.189 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.189 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:43.189 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:01:34.742048+0000", 2024-09-20T19:02:43.189 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000514267 2024-09-20T19:02:43.189 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.189 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.189 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:43.189 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:01:34.742068+0000", 2024-09-20T19:02:43.189 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9916999999999999e-05 2024-09-20T19:02:43.189 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.189 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.189 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:02:43.189 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:01:34.752173+0000", 2024-09-20T19:02:43.189 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.01010463 2024-09-20T19:02:43.190 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.190 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.190 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:43.190 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:01:34.752372+0000", 2024-09-20T19:02:43.190 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019885200000000001 2024-09-20T19:02:43.190 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:43.190 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:43.190 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:43.190 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.190 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.191 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25775.0:51921 10.3 10:c83df5a9:::benchmark_data_smithi083_148100_object51920:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e340)", 2024-09-20T19:02:43.191 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:02:42.777109+0000", 2024-09-20T19:02:43.191 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.32002594200000001, 2024-09-20T19:02:43.191 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.118903313, 2024-09-20T19:02:43.191 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:43.191 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:43.191 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:02:43.191 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:02:43.191 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25775", 2024-09-20T19:02:43.191 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:02:43.191 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 51921 2024-09-20T19:02:43.191 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.191 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:43.191 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.191 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:43.192 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.777109+0000", 2024-09-20T19:02:43.192 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:43.192 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.192 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.192 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:43.192 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.777109+0000", 2024-09-20T19:02:43.192 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:43.192 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.192 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.192 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:43.192 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.777110+0000", 2024-09-20T19:02:43.192 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4300000000000001e-07 2024-09-20T19:02:43.192 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.192 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.193 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:43.193 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.777133+0000", 2024-09-20T19:02:43.193 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3312999999999999e-05 2024-09-20T19:02:43.193 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.193 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.193 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:43.193 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.777143+0000", 2024-09-20T19:02:43.193 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0111e-05 2024-09-20T19:02:43.193 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.193 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.193 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:43.193 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.777150+0000", 2024-09-20T19:02:43.193 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7889999999999997e-06 2024-09-20T19:02:43.193 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.194 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.194 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:43.194 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.777197+0000", 2024-09-20T19:02:43.194 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7525000000000003e-05 2024-09-20T19:02:43.194 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.194 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.194 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:43.194 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.777304+0000", 2024-09-20T19:02:43.194 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106548 2024-09-20T19:02:43.194 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.194 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.194 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3,5", 2024-09-20T19:02:43.194 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.777571+0000", 2024-09-20T19:02:43.194 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00026748100000000002 2024-09-20T19:02:43.195 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.195 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.195 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:43.195 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.780123+0000", 2024-09-20T19:02:43.195 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002551813 2024-09-20T19:02:43.195 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.195 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.195 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:02:43.195 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.781319+0000", 2024-09-20T19:02:43.195 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001195619 2024-09-20T19:02:43.195 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.195 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.195 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:43.195 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.895858+0000", 2024-09-20T19:02:43.195 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.114538906 2024-09-20T19:02:43.196 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.196 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.196 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:02:43.196 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.895962+0000", 2024-09-20T19:02:43.196 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000104503 2024-09-20T19:02:43.196 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.196 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.196 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:43.196 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.896012+0000", 2024-09-20T19:02:43.196 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9762000000000002e-05 2024-09-20T19:02:43.196 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:43.196 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:43.196 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:43.196 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.197 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.197 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25775.0:51934 10.3 10:c859bea5:::benchmark_data_smithi083_148100_object51933:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e340)", 2024-09-20T19:02:43.197 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:02:42.785246+0000", 2024-09-20T19:02:43.197 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.31188823599999999, 2024-09-20T19:02:43.197 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.11136808500000001, 2024-09-20T19:02:43.197 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:43.197 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:43.197 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:02:43.197 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:02:43.197 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25775", 2024-09-20T19:02:43.197 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:02:43.197 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 51934 2024-09-20T19:02:43.197 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.197 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:43.198 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.198 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:43.198 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.785246+0000", 2024-09-20T19:02:43.198 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:43.198 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.198 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.198 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:43.198 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.785246+0000", 2024-09-20T19:02:43.198 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:43.198 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.198 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.198 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:43.198 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.785248+0000", 2024-09-20T19:02:43.198 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1200000000000001e-06 2024-09-20T19:02:43.199 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.199 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.199 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:43.199 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.785268+0000", 2024-09-20T19:02:43.199 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0937e-05 2024-09-20T19:02:43.199 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.199 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.199 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:43.199 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.785283+0000", 2024-09-20T19:02:43.199 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4209999999999999e-05 2024-09-20T19:02:43.199 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.199 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.199 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:43.199 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.785290+0000", 2024-09-20T19:02:43.199 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9750000000000001e-06 2024-09-20T19:02:43.200 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.200 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.200 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:43.200 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.785342+0000", 2024-09-20T19:02:43.200 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2646999999999999e-05 2024-09-20T19:02:43.200 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.200 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.200 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:43.200 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.785460+0000", 2024-09-20T19:02:43.200 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000117747 2024-09-20T19:02:43.200 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.200 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.200 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3,5", 2024-09-20T19:02:43.200 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.785577+0000", 2024-09-20T19:02:43.201 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000116709 2024-09-20T19:02:43.201 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.201 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.201 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:02:43.201 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.788761+0000", 2024-09-20T19:02:43.201 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003184391 2024-09-20T19:02:43.201 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.201 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.201 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:43.201 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.788846+0000", 2024-09-20T19:02:43.201 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4473000000000003e-05 2024-09-20T19:02:43.201 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.201 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.201 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:43.202 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.896097+0000", 2024-09-20T19:02:43.202 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.107251207 2024-09-20T19:02:43.202 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.202 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.202 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:02:43.202 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.896577+0000", 2024-09-20T19:02:43.202 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00048043000000000001 2024-09-20T19:02:43.202 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.202 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.202 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:43.202 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.896614+0000", 2024-09-20T19:02:43.202 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7239e-05 2024-09-20T19:02:43.202 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:43.202 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:43.203 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:43.203 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.203 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.203 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25775.0:51949 10.3 10:c7df5d6a:::benchmark_data_smithi083_148100_object51948:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e340)", 2024-09-20T19:02:43.203 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:02:42.795193+0000", 2024-09-20T19:02:43.203 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.301941562, 2024-09-20T19:02:43.203 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.101751296, 2024-09-20T19:02:43.203 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:43.203 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:43.203 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:02:43.203 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:02:43.203 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25775", 2024-09-20T19:02:43.203 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:02:43.203 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 51949 2024-09-20T19:02:43.204 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.204 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:43.204 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.204 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:43.204 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.795193+0000", 2024-09-20T19:02:43.204 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:43.204 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.204 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.204 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:43.204 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.795193+0000", 2024-09-20T19:02:43.204 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:43.204 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.204 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.204 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:43.205 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.795195+0000", 2024-09-20T19:02:43.205 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.531e-06 2024-09-20T19:02:43.205 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.205 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.205 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:43.205 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.795215+0000", 2024-09-20T19:02:43.205 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0599999999999999e-05 2024-09-20T19:02:43.205 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.205 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.205 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:43.205 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.795229+0000", 2024-09-20T19:02:43.205 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3372e-05 2024-09-20T19:02:43.205 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.205 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.206 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:43.206 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.795236+0000", 2024-09-20T19:02:43.206 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.956e-06 2024-09-20T19:02:43.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.206 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.206 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:43.206 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.795294+0000", 2024-09-20T19:02:43.206 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8372000000000003e-05 2024-09-20T19:02:43.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.206 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.206 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:43.206 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.795398+0000", 2024-09-20T19:02:43.206 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000104336 2024-09-20T19:02:43.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.207 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.207 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3,5", 2024-09-20T19:02:43.207 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.795491+0000", 2024-09-20T19:02:43.207 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2470000000000001e-05 2024-09-20T19:02:43.207 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.207 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.207 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:02:43.207 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.797323+0000", 2024-09-20T19:02:43.207 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0018327370000000001 2024-09-20T19:02:43.207 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.207 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.207 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:43.207 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.799501+0000", 2024-09-20T19:02:43.207 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0021777469999999998 2024-09-20T19:02:43.207 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.208 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.208 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:43.208 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.896655+0000", 2024-09-20T19:02:43.208 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.097154307999999995 2024-09-20T19:02:43.208 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.208 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.208 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:02:43.208 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.896811+0000", 2024-09-20T19:02:43.208 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015532400000000001 2024-09-20T19:02:43.208 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.208 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.208 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:43.208 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.896944+0000", 2024-09-20T19:02:43.208 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013354299999999999 2024-09-20T19:02:43.209 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:43.209 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:43.209 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:43.209 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.209 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.209 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25775.0:51965 10.3 10:cc522185:::benchmark_data_smithi083_148100_object51964:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e340)", 2024-09-20T19:02:43.210 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:02:42.807773+0000", 2024-09-20T19:02:43.210 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.28936203199999999, 2024-09-20T19:02:43.210 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.091849228000000005, 2024-09-20T19:02:43.210 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:43.210 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:43.210 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:02:43.210 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:02:43.210 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25775", 2024-09-20T19:02:43.210 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:02:43.210 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 51965 2024-09-20T19:02:43.210 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.211 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:43.211 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.211 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:43.211 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.807773+0000", 2024-09-20T19:02:43.211 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:43.211 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.211 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.211 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:43.211 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.807773+0000", 2024-09-20T19:02:43.211 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:43.211 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.211 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.211 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:43.211 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.807774+0000", 2024-09-20T19:02:43.211 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2419999999999999e-06 2024-09-20T19:02:43.212 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.212 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.212 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:43.212 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.807795+0000", 2024-09-20T19:02:43.212 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1321000000000001e-05 2024-09-20T19:02:43.212 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.212 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.212 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:43.212 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.807815+0000", 2024-09-20T19:02:43.212 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9828999999999999e-05 2024-09-20T19:02:43.212 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.212 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.212 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:43.212 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.807821+0000", 2024-09-20T19:02:43.213 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3999999999999997e-06 2024-09-20T19:02:43.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.213 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.213 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:43.213 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.807893+0000", 2024-09-20T19:02:43.213 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1534999999999999e-05 2024-09-20T19:02:43.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.213 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.213 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:43.213 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.808052+0000", 2024-09-20T19:02:43.213 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000159419 2024-09-20T19:02:43.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.213 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.213 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3,5", 2024-09-20T19:02:43.213 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.808205+0000", 2024-09-20T19:02:43.214 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015260100000000001 2024-09-20T19:02:43.214 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.214 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.214 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:43.214 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.810294+0000", 2024-09-20T19:02:43.214 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0020888840000000001 2024-09-20T19:02:43.214 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.214 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.214 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:02:43.214 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.810350+0000", 2024-09-20T19:02:43.214 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5797999999999997e-05 2024-09-20T19:02:43.214 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.214 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.214 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:43.215 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.899271+0000", 2024-09-20T19:02:43.215 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.088921375999999996 2024-09-20T19:02:43.215 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.215 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.215 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:02:43.215 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.899581+0000", 2024-09-20T19:02:43.215 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00030987000000000001 2024-09-20T19:02:43.215 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:43.215 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:43.215 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:43.215 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.899622+0000", 2024-09-20T19:02:43.215 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0952999999999999e-05 2024-09-20T19:02:43.215 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:43.215 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:43.216 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:43.216 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:43.216 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:43.216 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T19:02:43.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:43.217 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:43.217 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:43.317 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:02:43.352 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:43.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:43.353 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:02:43.454 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:43.493 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:43.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:43.494 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:43.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:43 smithi083 ceph-mon[29421]: 3.4 scrub starts 2024-09-20T19:02:43.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:43 smithi083 ceph-mon[29421]: 3.4 scrub ok 2024-09-20T19:02:43.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:43 smithi083 ceph-mon[29421]: 4.1a scrub starts 2024-09-20T19:02:43.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:43 smithi083 ceph-mon[29421]: 4.1a scrub ok 2024-09-20T19:02:43.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:43 smithi083 ceph-mon[29421]: 5.b scrub starts 2024-09-20T19:02:43.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:43 smithi083 ceph-mon[29421]: 5.b scrub ok 2024-09-20T19:02:43.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:43 smithi083 ceph-mon[29421]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:02:43.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:43 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T19:02:43.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:43 smithi083 ceph-mon[29421]: osdmap e341: 8 total, 8 up, 8 in 2024-09-20T19:02:43.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:43 smithi083 ceph-mon[36260]: 3.4 scrub starts 2024-09-20T19:02:43.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:43 smithi083 ceph-mon[36260]: 3.4 scrub ok 2024-09-20T19:02:43.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:43 smithi083 ceph-mon[36260]: 4.1a scrub starts 2024-09-20T19:02:43.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:43 smithi083 ceph-mon[36260]: 4.1a scrub ok 2024-09-20T19:02:43.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:43 smithi083 ceph-mon[36260]: 5.b scrub starts 2024-09-20T19:02:43.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:43 smithi083 ceph-mon[36260]: 5.b scrub ok 2024-09-20T19:02:43.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:43 smithi083 ceph-mon[36260]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:02:43.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:43 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T19:02:43.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:43 smithi083 ceph-mon[36260]: osdmap e341: 8 total, 8 up, 8 in 2024-09-20T19:02:43.595 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:02:43.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:43.632 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:43.632 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:02:43.733 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:43.764 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:43 smithi142 ceph-mon[32508]: 3.4 scrub starts 2024-09-20T19:02:43.764 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:43 smithi142 ceph-mon[32508]: 3.4 scrub ok 2024-09-20T19:02:43.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:43 smithi142 ceph-mon[32508]: 4.1a scrub starts 2024-09-20T19:02:43.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:43 smithi142 ceph-mon[32508]: 4.1a scrub ok 2024-09-20T19:02:43.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:43 smithi142 ceph-mon[32508]: 5.b scrub starts 2024-09-20T19:02:43.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:43 smithi142 ceph-mon[32508]: 5.b scrub ok 2024-09-20T19:02:43.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:43 smithi142 ceph-mon[32508]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:02:43.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:43 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T19:02:43.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:43 smithi142 ceph-mon[32508]: osdmap e341: 8 total, 8 up, 8 in 2024-09-20T19:02:43.769 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:43.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:43.770 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:43.870 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:43.874 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 24 16 53644 53628 139.611 104.812 0.0152527 0.00715814 2024-09-20T19:02:43.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:43.909 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:43.909 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:44.010 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:44.031 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:44.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:44.032 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:44.133 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:02:44.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:44.175 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:44.175 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:02:44.276 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:44.308 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:44.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:44.309 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:44.409 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:44.426 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T19:02:44.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:44.447 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:44.447 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:44.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:44 smithi083 ceph-mon[29421]: pgmap v644: 148 pgs: 148 active+clean; 3.0 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 748 B/s rd, 119 MiB/s wr, 1.91k op/s 2024-09-20T19:02:44.544 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:44 smithi083 ceph-mon[36260]: pgmap v644: 148 pgs: 148 active+clean; 3.0 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 748 B/s rd, 119 MiB/s wr, 1.91k op/s 2024-09-20T19:02:44.548 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:02:44.584 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:44.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:44.585 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:02:44.685 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:44.730 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:44.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:44.731 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:44.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:44 smithi142 ceph-mon[32508]: pgmap v644: 148 pgs: 148 active+clean; 3.0 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 748 B/s rd, 119 MiB/s wr, 1.91k op/s 2024-09-20T19:02:44.832 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:44.873 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 25 16 55247 55231 138.034 100.188 0.00658495 0.0072398 2024-09-20T19:02:44.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:44.874 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:44.874 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:44.975 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:02:45.003 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:45.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:45.004 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:02:45.104 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:45.142 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:45.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:45.143 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:45.244 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:45.275 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:45.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:45.277 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:45.377 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:45.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:45.411 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:45.412 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:45.512 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:45.546 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:45.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:45.547 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:45.648 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:45.686 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:45.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:45.687 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:45.787 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:02:45.823 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:45.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:45.824 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:02:45.873 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 26 16 56847 56831 136.571 100 0.00645432 0.00731738 2024-09-20T19:02:45.925 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:45.962 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:45.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:45.964 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:46.064 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:46.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:46.087 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:46.087 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:46.188 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:46.222 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:46.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:46.223 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:46.324 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:02:46.360 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:46.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:46.361 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:02:46.462 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:46.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:46.499 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:46.499 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:46.600 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:02:46.635 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:46.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:46.636 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:02:46.736 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:46.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:46.763 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:46.763 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:46.764 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:46 smithi142 ceph-mon[32508]: pgmap v645: 148 pgs: 148 active+clean; 3.0 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 639 B/s rd, 77 MiB/s wr, 1.23k op/s 2024-09-20T19:02:46.764 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:46 smithi142 ceph-mon[32508]: osdmap e342: 8 total, 8 up, 8 in 2024-09-20T19:02:46.764 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:46 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:02:46.764 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:46 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:02:46.794 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:46 smithi083 ceph-mon[29421]: pgmap v645: 148 pgs: 148 active+clean; 3.0 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 639 B/s rd, 77 MiB/s wr, 1.23k op/s 2024-09-20T19:02:46.794 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:46 smithi083 ceph-mon[29421]: osdmap e342: 8 total, 8 up, 8 in 2024-09-20T19:02:46.794 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:46 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:02:46.794 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:46 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:02:46.795 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:46 smithi083 ceph-mon[36260]: pgmap v645: 148 pgs: 148 active+clean; 3.0 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 639 B/s rd, 77 MiB/s wr, 1.23k op/s 2024-09-20T19:02:46.795 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:46 smithi083 ceph-mon[36260]: osdmap e342: 8 total, 8 up, 8 in 2024-09-20T19:02:46.795 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:46 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:02:46.795 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:46 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:02:46.864 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:46.874 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 27 16 58382 58366 135.063 95.9375 0.0067845 0.00739924 2024-09-20T19:02:46.899 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:46.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:46.900 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:47.001 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:47.036 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:47.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:47.037 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:47.138 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:47.175 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:47.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:47.176 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:47.255 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-20T19:02:47.276 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:47.360 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:47.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:47.361 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:47.462 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:02:47.492 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:47.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:47.493 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:02:47.514 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:02:47 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:02:47.594 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:02:47.626 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:47.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:47.627 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:02:47.728 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:02:47.760 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:47.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:47.761 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:02:47.861 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:47.873 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 28 16 59890 59874 133.605 94.25 0.0138778 0.0074799 2024-09-20T19:02:47.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:47.905 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:47.905 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:47.952 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T19:02:47.954 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-20T19:02:48.006 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:02:48.040 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:48.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:48.041 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:02:48.142 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:48.178 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:48.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:48.179 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:48.280 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:02:48.315 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:48.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:48.316 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:02:48.416 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:02:48.452 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:48.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:48.453 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:02:48.554 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:48.591 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:48.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:48.592 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:48.693 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:02:48.697 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:48 smithi083 ceph-mon[29421]: pgmap v647: 148 pgs: 148 active+clean; 3.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 123 MiB/s wr, 1.97k op/s 2024-09-20T19:02:48.697 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:48 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:02:48.697 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:48 smithi083 ceph-mon[36260]: pgmap v647: 148 pgs: 148 active+clean; 3.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 123 MiB/s wr, 1.97k op/s 2024-09-20T19:02:48.698 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:48 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:02:48.710 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:02:48.723 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:48.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:48.724 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:02:48.758 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":647,"stamp":"2024-09-20T19:02:47.083582+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3700890139,"num_objects":56661,"num_object_clones":0,"num_object_copies":169983,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56661,"num_whiteouts":0,"num_read":2032,"num_read_kb":1889,"num_write":57297,"num_write_kb":3615241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":1774258,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57318,"ondisk_log_size":57318,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":14009188,"kb_used_data":10506332,"kb_used_omap":16,"kb_used_meta":3502703,"kb_avail":735952028,"statfs":{"total":767960285184,"available":753614876672,"internally_reserved":0,"allocated":10758483968,"data_stored":10754275161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16790,"internal_metadata":3586768490},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,5,7,8,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1030684672,"num_objects":15727,"num_object_clones":0,"num_object_copies":47181,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15727,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":15727,"num_write_kb":1006528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002030"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":610,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-20T19:02:38.263845+0000","last_change":"2024-09-20T19:01:31.817967+0000","last_active":"2024-09-20T19:02:38.263845+0000","last_peered":"2024-09-20T19:02:38.263845+0000","last_clean":"2024-09-20T19:02:38.263845+0000","last_became_active":"2024-09-20T18:49:45.943201+0000","last_became_peered":"2024-09-20T18:49:45.943201+0000","last_unstale":"2024-09-20T19:02:38.263845+0000","last_undegraded":"2024-09-20T19:02:38.263845+0000","last_fullsized":"2024-09-20T19:02:38.263845+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:31.817909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:31.817909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:02:43.907273+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"273'15","reported_seq":625,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-09-20T19:02:42.865340+0000","last_change":"2024-09-20T19:01:22.840670+0000","last_active":"2024-09-20T19:02:42.865340+0000","last_peered":"2024-09-20T19:02:42.865340+0000","last_clean":"2024-09-20T19:02:42.865340+0000","last_became_active":"2024-09-20T19:01:22.838745+0000","last_became_peered":"2024-09-20T19:01:22.838745+0000","last_unstale":"2024-09-20T19:02:42.865340+0000","last_undegraded":"2024-09-20T19:02:42.865340+0000","last_fullsized":"2024-09-20T19:02:42.865340+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:50.231620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:50.231620+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:09.095233+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":604,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.834259+0000","last_change":"2024-09-20T18:57:49.016078+0000","last_active":"2024-09-20T19:02:45.834259+0000","last_peered":"2024-09-20T19:02:45.834259+0000","last_clean":"2024-09-20T19:02:45.834259+0000","last_became_active":"2024-09-20T18:49:49.959514+0000","last_became_peered":"2024-09-20T18:49:49.959514+0000","last_unstale":"2024-09-20T19:02:45.834259+0000","last_undegraded":"2024-09-20T19:02:45.834259+0000","last_fullsized":"2024-09-20T19:02:45.834259+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:49.016006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:49.016006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:10.916493+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":551,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-09-20T19:02:42.866120+0000","last_change":"2024-09-20T19:01:22.841355+0000","last_active":"2024-09-20T19:02:42.866120+0000","last_peered":"2024-09-20T19:02:42.866120+0000","last_clean":"2024-09-20T19:02:42.866120+0000","last_became_active":"2024-09-20T19:01:22.838827+0000","last_became_peered":"2024-09-20T19:01:22.838827+0000","last_unstale":"2024-09-20T19:02:42.866120+0000","last_undegraded":"2024-09-20T19:02:42.866120+0000","last_fullsized":"2024-09-20T19:02:42.866120+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:02.084375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:02.084375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:20.990491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":610,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-20T19:02:38.269829+0000","last_change":"2024-09-20T19:02:02.441633+0000","last_active":"2024-09-20T19:02:38.269829+0000","last_peered":"2024-09-20T19:02:38.269829+0000","last_clean":"2024-09-20T19:02:38.269829+0000","last_became_active":"2024-09-20T18:59:05.081081+0000","last_became_peered":"2024-09-20T18:59:05.081081+0000","last_unstale":"2024-09-20T19:02:38.269829+0000","last_undegraded":"2024-09-20T19:02:38.269829+0000","last_fullsized":"2024-09-20T19:02:38.269829+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:02.441577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:02.441577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:28.158812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":615,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-20T19:02:38.269726+0000","last_change":"2024-09-20T18:59:05.073093+0000","last_active":"2024-09-20T19:02:38.269726+0000","last_peered":"2024-09-20T19:02:38.269726+0000","last_clean":"2024-09-20T19:02:38.269726+0000","last_became_active":"2024-09-20T18:59:05.072675+0000","last_became_peered":"2024-09-20T18:59:05.072675+0000","last_unstale":"2024-09-20T19:02:38.269726+0000","last_undegraded":"2024-09-20T19:02:38.269726+0000","last_fullsized":"2024-09-20T19:02:38.269726+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:03.055929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:59:03.055929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"274'27","reported_seq":647,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-20T19:02:40.899215+0000","last_change":"2024-09-20T19:02:40.899215+0000","last_active":"2024-09-20T19:02:40.899215+0000","last_peered":"2024-09-20T19:02:40.899215+0000","last_clean":"2024-09-20T19:02:40.899215+0000","last_became_active":"2024-09-20T18:49:47.952150+0000","last_became_peered":"2024-09-20T18:49:47.952150+0000","last_unstale":"2024-09-20T19:02:40.899215+0000","last_undegraded":"2024-09-20T19:02:40.899215+0000","last_fullsized":"2024-09-20T19:02:40.899215+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'27","last_scrub_stamp":"2024-09-20T19:02:40.899139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:40.899139+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:02.856822+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":542,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-09-20T19:02:42.865293+0000","last_change":"2024-09-20T19:02:23.982879+0000","last_active":"2024-09-20T19:02:42.865293+0000","last_peered":"2024-09-20T19:02:42.865293+0000","last_clean":"2024-09-20T19:02:42.865293+0000","last_became_active":"2024-09-20T19:01:22.839075+0000","last_became_peered":"2024-09-20T19:01:22.839075+0000","last_unstale":"2024-09-20T19:02:42.865293+0000","last_undegraded":"2024-09-20T19:02:42.865293+0000","last_fullsized":"2024-09-20T19:02:42.865293+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:23.982813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:23.982813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:51.507141+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"59'1","reported_seq":667,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.835414+0000","last_change":"2024-09-20T19:01:28.592723+0000","last_active":"2024-09-20T19:02:45.835414+0000","last_peered":"2024-09-20T19:02:45.835414+0000","last_clean":"2024-09-20T19:02:45.835414+0000","last_became_active":"2024-09-20T19:01:22.847016+0000","last_became_peered":"2024-09-20T19:01:22.847016+0000","last_unstale":"2024-09-20T19:02:45.835414+0000","last_undegraded":"2024-09-20T19:02:45.835414+0000","last_fullsized":"2024-09-20T19:02:45.835414+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:01:28.592653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:28.592653+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:43.784234+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"273'15","reported_seq":590,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-09-20T19:02:42.865960+0000","last_change":"2024-09-20T19:02:42.114419+0000","last_active":"2024-09-20T19:02:42.865960+0000","last_peered":"2024-09-20T19:02:42.865960+0000","last_clean":"2024-09-20T19:02:42.865960+0000","last_became_active":"2024-09-20T19:01:22.844899+0000","last_became_peered":"2024-09-20T19:01:22.844899+0000","last_unstale":"2024-09-20T19:02:42.865960+0000","last_undegraded":"2024-09-20T19:02:42.865960+0000","last_fullsized":"2024-09-20T19:02:42.865960+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:02:42.114331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:42.114331+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:44.431155+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":612,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.835084+0000","last_change":"2024-09-20T19:01:51.564693+0000","last_active":"2024-09-20T19:02:45.835084+0000","last_peered":"2024-09-20T19:02:45.835084+0000","last_clean":"2024-09-20T19:02:45.835084+0000","last_became_active":"2024-09-20T18:49:49.961188+0000","last_became_peered":"2024-09-20T18:49:49.961188+0000","last_unstale":"2024-09-20T19:02:45.835084+0000","last_undegraded":"2024-09-20T19:02:45.835084+0000","last_fullsized":"2024-09-20T19:02:45.835084+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:51.564624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:51.564624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:01.653667+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":581,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-20T19:02:38.268405+0000","last_change":"2024-09-20T19:02:03.029635+0000","last_active":"2024-09-20T19:02:38.268405+0000","last_peered":"2024-09-20T19:02:38.268405+0000","last_clean":"2024-09-20T19:02:38.268405+0000","last_became_active":"2024-09-20T18:49:51.966715+0000","last_became_peered":"2024-09-20T18:49:51.966715+0000","last_unstale":"2024-09-20T19:02:38.268405+0000","last_undegraded":"2024-09-20T19:02:38.268405+0000","last_fullsized":"2024-09-20T19:02:38.268405+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:03.029585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:03.029585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:07.806574+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":623,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.835268+0000","last_change":"2024-09-20T19:01:48.520088+0000","last_active":"2024-09-20T19:02:45.835268+0000","last_peered":"2024-09-20T19:02:45.835268+0000","last_clean":"2024-09-20T19:02:45.835268+0000","last_became_active":"2024-09-20T19:01:22.846527+0000","last_became_peered":"2024-09-20T19:01:22.846527+0000","last_unstale":"2024-09-20T19:02:45.835268+0000","last_undegraded":"2024-09-20T19:02:45.835268+0000","last_fullsized":"2024-09-20T19:02:45.835268+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:48.520017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:48.520017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:02:49.230646+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"273'7","reported_seq":580,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-09-20T19:02:42.865477+0000","last_change":"2024-09-20T19:01:22.844317+0000","last_active":"2024-09-20T19:02:42.865477+0000","last_peered":"2024-09-20T19:02:42.865477+0000","last_clean":"2024-09-20T19:02:42.865477+0000","last_became_active":"2024-09-20T19:01:22.844013+0000","last_became_peered":"2024-09-20T19:01:22.844013+0000","last_unstale":"2024-09-20T19:02:42.865477+0000","last_undegraded":"2024-09-20T19:02:42.865477+0000","last_fullsized":"2024-09-20T19:02:42.865477+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-20T18:58:33.660531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:33.660531+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":616,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.831700+0000","last_change":"2024-09-20T19:02:33.203392+0000","last_active":"2024-09-20T19:02:45.831700+0000","last_peered":"2024-09-20T19:02:45.831700+0000","last_clean":"2024-09-20T19:02:45.831700+0000","last_became_active":"2024-09-20T19:01:22.830605+0000","last_became_peered":"2024-09-20T19:01:22.830605+0000","last_unstale":"2024-09-20T19:02:45.831700+0000","last_undegraded":"2024-09-20T19:02:45.831700+0000","last_fullsized":"2024-09-20T19:02:45.831700+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:33.203312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:33.203312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:35.713757+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":600,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.834899+0000","last_change":"2024-09-20T18:58:26.465305+0000","last_active":"2024-09-20T19:02:45.834899+0000","last_peered":"2024-09-20T19:02:45.834899+0000","last_clean":"2024-09-20T19:02:45.834899+0000","last_became_active":"2024-09-20T18:49:51.970663+0000","last_became_peered":"2024-09-20T18:49:51.970663+0000","last_unstale":"2024-09-20T19:02:45.834899+0000","last_undegraded":"2024-09-20T19:02:45.834899+0000","last_fullsized":"2024-09-20T19:02:45.834899+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:26.465264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:26.465264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:18.787658+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":531,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-09-20T19:02:42.866029+0000","last_change":"2024-09-20T19:01:22.841173+0000","last_active":"2024-09-20T19:02:42.866029+0000","last_peered":"2024-09-20T19:02:42.866029+0000","last_clean":"2024-09-20T19:02:42.866029+0000","last_became_active":"2024-09-20T19:01:22.838149+0000","last_became_peered":"2024-09-20T19:01:22.838149+0000","last_unstale":"2024-09-20T19:02:42.866029+0000","last_undegraded":"2024-09-20T19:02:42.866029+0000","last_fullsized":"2024-09-20T19:02:42.866029+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:00.906652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:00.906652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:19.046861+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":620,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-20T19:02:38.262560+0000","last_change":"2024-09-20T19:01:44.998192+0000","last_active":"2024-09-20T19:02:38.262560+0000","last_peered":"2024-09-20T19:02:38.262560+0000","last_clean":"2024-09-20T19:02:38.262560+0000","last_became_active":"2024-09-20T19:01:22.845773+0000","last_became_peered":"2024-09-20T19:01:22.845773+0000","last_unstale":"2024-09-20T19:02:38.262560+0000","last_undegraded":"2024-09-20T19:02:38.262560+0000","last_fullsized":"2024-09-20T19:02:38.262560+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:44.998147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:44.998147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:02:50.507993+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"273'27","reported_seq":654,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.838129+0000","last_change":"2024-09-20T19:01:59.787246+0000","last_active":"2024-09-20T19:02:45.838129+0000","last_peered":"2024-09-20T19:02:45.838129+0000","last_clean":"2024-09-20T19:02:45.838129+0000","last_became_active":"2024-09-20T18:49:47.950939+0000","last_became_peered":"2024-09-20T18:49:47.950939+0000","last_unstale":"2024-09-20T19:02:45.838129+0000","last_undegraded":"2024-09-20T19:02:45.838129+0000","last_fullsized":"2024-09-20T19:02:45.838129+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'27","last_scrub_stamp":"2024-09-20T19:01:59.787179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:59.787179+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:02.865513+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":607,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.834665+0000","last_change":"2024-09-20T19:02:05.889054+0000","last_active":"2024-09-20T19:02:45.834665+0000","last_peered":"2024-09-20T19:02:45.834665+0000","last_clean":"2024-09-20T19:02:45.834665+0000","last_became_active":"2024-09-20T19:01:22.834771+0000","last_became_peered":"2024-09-20T19:01:22.834771+0000","last_unstale":"2024-09-20T19:02:45.834665+0000","last_undegraded":"2024-09-20T19:02:45.834665+0000","last_fullsized":"2024-09-20T19:02:45.834665+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:05.888974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:05.888974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:34.411336+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":611,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-20T19:02:38.271634+0000","last_change":"2024-09-20T19:02:28.934597+0000","last_active":"2024-09-20T19:02:38.271634+0000","last_peered":"2024-09-20T19:02:38.271634+0000","last_clean":"2024-09-20T19:02:38.271634+0000","last_became_active":"2024-09-20T18:59:05.084496+0000","last_became_peered":"2024-09-20T18:59:05.084496+0000","last_unstale":"2024-09-20T19:02:38.271634+0000","last_undegraded":"2024-09-20T19:02:38.271634+0000","last_fullsized":"2024-09-20T19:02:38.271634+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:28.934264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:28.934264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:43.415620+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"59'1","reported_seq":537,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-09-20T19:02:42.866230+0000","last_change":"2024-09-20T19:01:22.840962+0000","last_active":"2024-09-20T19:02:42.866230+0000","last_peered":"2024-09-20T19:02:42.866230+0000","last_clean":"2024-09-20T19:02:42.866230+0000","last_became_active":"2024-09-20T19:01:22.840591+0000","last_became_peered":"2024-09-20T19:01:22.840591+0000","last_unstale":"2024-09-20T19:02:42.866230+0000","last_undegraded":"2024-09-20T19:02:42.866230+0000","last_fullsized":"2024-09-20T19:02:42.866230+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:59:03.397225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:59:03.397225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"274'18","reported_seq":623,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-20T19:02:38.272193+0000","last_change":"2024-09-20T19:01:22.837011+0000","last_active":"2024-09-20T19:02:38.272193+0000","last_peered":"2024-09-20T19:02:38.272193+0000","last_clean":"2024-09-20T19:02:38.272193+0000","last_became_active":"2024-09-20T19:01:22.836111+0000","last_became_peered":"2024-09-20T19:01:22.836111+0000","last_unstale":"2024-09-20T19:02:38.272193+0000","last_undegraded":"2024-09-20T19:02:38.272193+0000","last_fullsized":"2024-09-20T19:02:38.272193+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:57:09.818999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:09.818999+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:07.600886+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":586,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-09-20T19:02:42.865629+0000","last_change":"2024-09-20T19:02:02.094517+0000","last_active":"2024-09-20T19:02:42.865629+0000","last_peered":"2024-09-20T19:02:42.865629+0000","last_clean":"2024-09-20T19:02:42.865629+0000","last_became_active":"2024-09-20T19:01:22.848532+0000","last_became_peered":"2024-09-20T19:01:22.848532+0000","last_unstale":"2024-09-20T19:02:42.865629+0000","last_undegraded":"2024-09-20T19:02:42.865629+0000","last_fullsized":"2024-09-20T19:02:42.865629+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:02.094473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:02.094473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:17.459163+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":602,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.832648+0000","last_change":"2024-09-20T19:02:28.294390+0000","last_active":"2024-09-20T19:02:45.832648+0000","last_peered":"2024-09-20T19:02:45.832648+0000","last_clean":"2024-09-20T19:02:45.832648+0000","last_became_active":"2024-09-20T18:49:51.970058+0000","last_became_peered":"2024-09-20T18:49:51.970058+0000","last_unstale":"2024-09-20T19:02:45.832648+0000","last_undegraded":"2024-09-20T19:02:45.832648+0000","last_fullsized":"2024-09-20T19:02:45.832648+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T19:02:28.294292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:28.294292+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:41.507259+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":608,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.834476+0000","last_change":"2024-09-20T19:01:22.837474+0000","last_active":"2024-09-20T19:02:45.834476+0000","last_peered":"2024-09-20T19:02:45.834476+0000","last_clean":"2024-09-20T19:02:45.834476+0000","last_became_active":"2024-09-20T19:01:22.837139+0000","last_became_peered":"2024-09-20T19:01:22.837139+0000","last_unstale":"2024-09-20T19:02:45.834476+0000","last_undegraded":"2024-09-20T19:02:45.834476+0000","last_fullsized":"2024-09-20T19:02:45.834476+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:57:18.032067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:18.032067+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:09.880584+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"274'20","reported_seq":652,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-20T19:02:38.262784+0000","last_change":"2024-09-20T19:02:35.097053+0000","last_active":"2024-09-20T19:02:38.262784+0000","last_peered":"2024-09-20T19:02:38.262784+0000","last_clean":"2024-09-20T19:02:38.262784+0000","last_became_active":"2024-09-20T18:49:48.500809+0000","last_became_peered":"2024-09-20T18:49:48.500809+0000","last_unstale":"2024-09-20T19:02:38.262784+0000","last_undegraded":"2024-09-20T19:02:38.262784+0000","last_fullsized":"2024-09-20T19:02:38.262784+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'20","last_scrub_stamp":"2024-09-20T19:02:35.096979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:35.096979+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:51.795986+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"308'12","reported_seq":252,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.832302+0000","last_change":"2024-09-20T19:01:44.541242+0000","last_active":"2024-09-20T19:02:45.832302+0000","last_peered":"2024-09-20T19:02:45.832302+0000","last_clean":"2024-09-20T19:02:45.832302+0000","last_became_active":"2024-09-20T19:01:22.845596+0000","last_became_peered":"2024-09-20T19:01:22.845596+0000","last_unstale":"2024-09-20T19:02:45.832302+0000","last_undegraded":"2024-09-20T19:02:45.832302+0000","last_fullsized":"2024-09-20T19:02:45.832302+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"308'12","last_scrub_stamp":"2024-09-20T19:01:44.541156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:44.541156+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:11.696073+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"341'3414","reported_seq":3455,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.448523+0000","last_change":"2024-09-20T19:02:11.829261+0000","last_active":"2024-09-20T19:02:45.448523+0000","last_peered":"2024-09-20T19:02:45.448523+0000","last_clean":"2024-09-20T19:02:45.448523+0000","last_became_active":"2024-09-20T19:02:11.828743+0000","last_became_peered":"2024-09-20T19:02:11.828743+0000","last_unstale":"2024-09-20T19:02:45.448523+0000","last_undegraded":"2024-09-20T19:02:45.448523+0000","last_fullsized":"2024-09-20T19:02:45.448523+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":3414,"log_dups_size":0,"ondisk_log_size":3414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:26.017450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":223739904,"num_objects":3414,"num_object_clones":0,"num_object_copies":10242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3414,"num_write_kb":218496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"274'27","reported_seq":648,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.843387+0000","last_change":"2024-09-20T19:02:27.897213+0000","last_active":"2024-09-20T19:02:45.843387+0000","last_peered":"2024-09-20T19:02:45.843387+0000","last_clean":"2024-09-20T19:02:45.843387+0000","last_became_active":"2024-09-20T19:01:22.833840+0000","last_became_peered":"2024-09-20T19:01:22.833840+0000","last_unstale":"2024-09-20T19:02:45.843387+0000","last_undegraded":"2024-09-20T19:02:45.843387+0000","last_fullsized":"2024-09-20T19:02:45.843387+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'27","last_scrub_stamp":"2024-09-20T19:02:27.897138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:27.897138+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:40.557830+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":600,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-20T19:02:38.268215+0000","last_change":"2024-09-20T19:00:31.512527+0000","last_active":"2024-09-20T19:02:38.268215+0000","last_peered":"2024-09-20T19:02:38.268215+0000","last_clean":"2024-09-20T19:02:38.268215+0000","last_became_active":"2024-09-20T18:49:45.936104+0000","last_became_peered":"2024-09-20T18:49:45.936104+0000","last_unstale":"2024-09-20T19:02:38.268215+0000","last_undegraded":"2024-09-20T19:02:38.268215+0000","last_fullsized":"2024-09-20T19:02:38.268215+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:00:31.512472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:00:31.512472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"308'10","reported_seq":707,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-09-20T19:02:42.866126+0000","last_change":"2024-09-20T19:02:23.017699+0000","last_active":"2024-09-20T19:02:42.866126+0000","last_peered":"2024-09-20T19:02:42.866126+0000","last_clean":"2024-09-20T19:02:42.866126+0000","last_became_active":"2024-09-20T19:01:22.845603+0000","last_became_peered":"2024-09-20T19:01:22.845603+0000","last_unstale":"2024-09-20T19:02:42.866126+0000","last_undegraded":"2024-09-20T19:02:42.866126+0000","last_fullsized":"2024-09-20T19:02:42.866126+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"308'10","last_scrub_stamp":"2024-09-20T19:02:23.017626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:23.017626+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:51.273397+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":600,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.835089+0000","last_change":"2024-09-20T19:02:22.736879+0000","last_active":"2024-09-20T19:02:45.835089+0000","last_peered":"2024-09-20T19:02:45.835089+0000","last_clean":"2024-09-20T19:02:45.835089+0000","last_became_active":"2024-09-20T18:49:51.971399+0000","last_became_peered":"2024-09-20T18:49:51.971399+0000","last_unstale":"2024-09-20T19:02:45.835089+0000","last_undegraded":"2024-09-20T19:02:45.835089+0000","last_fullsized":"2024-09-20T19:02:45.835089+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:22.736825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:22.736825+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:51.591260+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.e","version":"341'3557","reported_seq":3600,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.457806+0000","last_change":"2024-09-20T19:02:11.824951+0000","last_active":"2024-09-20T19:02:45.457806+0000","last_peered":"2024-09-20T19:02:45.457806+0000","last_clean":"2024-09-20T19:02:45.457806+0000","last_became_active":"2024-09-20T19:02:11.824670+0000","last_became_peered":"2024-09-20T19:02:11.824670+0000","last_unstale":"2024-09-20T19:02:45.457806+0000","last_undegraded":"2024-09-20T19:02:45.457806+0000","last_fullsized":"2024-09-20T19:02:45.457806+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":3557,"log_dups_size":0,"ondisk_log_size":3557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:16.256458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":233111552,"num_objects":3557,"num_object_clones":0,"num_object_copies":10671,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3557,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3557,"num_write_kb":227648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"273'30","reported_seq":655,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-20T19:02:38.268007+0000","last_change":"2024-09-20T19:01:10.381171+0000","last_active":"2024-09-20T19:02:38.268007+0000","last_peered":"2024-09-20T19:02:38.268007+0000","last_clean":"2024-09-20T19:02:38.268007+0000","last_became_active":"2024-09-20T18:49:48.499189+0000","last_became_peered":"2024-09-20T18:49:48.499189+0000","last_unstale":"2024-09-20T19:02:38.268007+0000","last_undegraded":"2024-09-20T19:02:38.268007+0000","last_fullsized":"2024-09-20T19:02:38.268007+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'30","last_scrub_stamp":"2024-09-20T19:01:10.381094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:10.381094+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":600,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-20T19:02:38.272670+0000","last_change":"2024-09-20T19:02:22.806587+0000","last_active":"2024-09-20T19:02:38.272670+0000","last_peered":"2024-09-20T19:02:38.272670+0000","last_clean":"2024-09-20T19:02:38.272670+0000","last_became_active":"2024-09-20T18:49:45.940544+0000","last_became_peered":"2024-09-20T18:49:45.940544+0000","last_unstale":"2024-09-20T19:02:38.272670+0000","last_undegraded":"2024-09-20T19:02:38.272670+0000","last_fullsized":"2024-09-20T19:02:38.272670+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:22.806509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:22.806509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:39.275044+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":581,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-09-20T19:02:42.865802+0000","last_change":"2024-09-20T19:01:22.835462+0000","last_active":"2024-09-20T19:02:42.865802+0000","last_peered":"2024-09-20T19:02:42.865802+0000","last_clean":"2024-09-20T19:02:42.865802+0000","last_became_active":"2024-09-20T19:01:22.833592+0000","last_became_peered":"2024-09-20T19:01:22.833592+0000","last_unstale":"2024-09-20T19:02:42.865802+0000","last_undegraded":"2024-09-20T19:02:42.865802+0000","last_fullsized":"2024-09-20T19:02:42.865802+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.229201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.229201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:18.032847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":572,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-09-20T19:02:42.866055+0000","last_change":"2024-09-20T19:01:22.830850+0000","last_active":"2024-09-20T19:02:42.866055+0000","last_peered":"2024-09-20T19:02:42.866055+0000","last_clean":"2024-09-20T19:02:42.866055+0000","last_became_active":"2024-09-20T19:01:22.830369+0000","last_became_peered":"2024-09-20T19:01:22.830369+0000","last_unstale":"2024-09-20T19:02:42.866055+0000","last_undegraded":"2024-09-20T19:02:42.866055+0000","last_fullsized":"2024-09-20T19:02:42.866055+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:27.704520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:27.704520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"342'3571","reported_seq":3615,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.998516+0000","last_change":"2024-09-20T19:02:11.828279+0000","last_active":"2024-09-20T19:02:45.998516+0000","last_peered":"2024-09-20T19:02:45.998516+0000","last_clean":"2024-09-20T19:02:45.998516+0000","last_became_active":"2024-09-20T19:02:11.827468+0000","last_became_peered":"2024-09-20T19:02:11.827468+0000","last_unstale":"2024-09-20T19:02:45.998516+0000","last_undegraded":"2024-09-20T19:02:45.998516+0000","last_fullsized":"2024-09-20T19:02:45.998516+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":3571,"log_dups_size":0,"ondisk_log_size":3571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:26.126856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":234029056,"num_objects":3571,"num_object_clones":0,"num_object_copies":10713,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3571,"num_write_kb":228544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"274'28","reported_seq":647,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-09-20T19:02:42.865705+0000","last_change":"2024-09-20T19:01:22.848177+0000","last_active":"2024-09-20T19:02:42.865705+0000","last_peered":"2024-09-20T19:02:42.865705+0000","last_clean":"2024-09-20T19:02:42.865705+0000","last_became_active":"2024-09-20T19:01:22.847843+0000","last_became_peered":"2024-09-20T19:01:22.847843+0000","last_unstale":"2024-09-20T19:02:42.865705+0000","last_undegraded":"2024-09-20T19:02:42.865705+0000","last_fullsized":"2024-09-20T19:02:42.865705+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-20T18:54:57.394893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.394893+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:06.093887+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":41,"num_read_kb":34,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":613,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-20T19:02:38.263083+0000","last_change":"2024-09-20T19:01:22.852962+0000","last_active":"2024-09-20T19:02:38.263083+0000","last_peered":"2024-09-20T19:02:38.263083+0000","last_clean":"2024-09-20T19:02:38.263083+0000","last_became_active":"2024-09-20T19:01:22.852594+0000","last_became_peered":"2024-09-20T19:01:22.852594+0000","last_unstale":"2024-09-20T19:02:38.263083+0000","last_undegraded":"2024-09-20T19:02:38.263083+0000","last_fullsized":"2024-09-20T19:02:38.263083+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:00:29.890681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:00:29.890681+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:19.042344+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"308'10","reported_seq":778,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-20T19:02:38.267870+0000","last_change":"2024-09-20T19:01:22.834166+0000","last_active":"2024-09-20T19:02:38.267870+0000","last_peered":"2024-09-20T19:02:38.267870+0000","last_clean":"2024-09-20T19:02:38.267870+0000","last_became_active":"2024-09-20T19:01:22.833774+0000","last_became_peered":"2024-09-20T19:01:22.833774+0000","last_unstale":"2024-09-20T19:02:38.267870+0000","last_undegraded":"2024-09-20T19:02:38.267870+0000","last_fullsized":"2024-09-20T19:02:38.267870+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:58:23.114371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:23.114371+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:18.678273+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":608,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.831942+0000","last_change":"2024-09-20T18:59:05.082117+0000","last_active":"2024-09-20T19:02:45.831942+0000","last_peered":"2024-09-20T19:02:45.831942+0000","last_clean":"2024-09-20T19:02:45.831942+0000","last_became_active":"2024-09-20T18:59:05.081814+0000","last_became_peered":"2024-09-20T18:59:05.081814+0000","last_unstale":"2024-09-20T19:02:45.831942+0000","last_undegraded":"2024-09-20T19:02:45.831942+0000","last_fullsized":"2024-09-20T19:02:45.831942+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:29.530001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:29.530001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:11.272988+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"342'3470","reported_seq":3515,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:46.151036+0000","last_change":"2024-09-20T19:02:11.830056+0000","last_active":"2024-09-20T19:02:46.151036+0000","last_peered":"2024-09-20T19:02:46.151036+0000","last_clean":"2024-09-20T19:02:46.151036+0000","last_became_active":"2024-09-20T19:02:11.829446+0000","last_became_peered":"2024-09-20T19:02:11.829446+0000","last_unstale":"2024-09-20T19:02:46.151036+0000","last_undegraded":"2024-09-20T19:02:46.151036+0000","last_fullsized":"2024-09-20T19:02:46.151036+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":3470,"log_dups_size":0,"ondisk_log_size":3470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:28.206813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":227409920,"num_objects":3470,"num_object_clones":0,"num_object_copies":10410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3470,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3470,"num_write_kb":222080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"274'20","reported_seq":666,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.835260+0000","last_change":"2024-09-20T19:02:35.931157+0000","last_active":"2024-09-20T19:02:45.835260+0000","last_peered":"2024-09-20T19:02:45.835260+0000","last_clean":"2024-09-20T19:02:45.835260+0000","last_became_active":"2024-09-20T19:01:22.834995+0000","last_became_peered":"2024-09-20T19:01:22.834995+0000","last_unstale":"2024-09-20T19:02:45.835260+0000","last_undegraded":"2024-09-20T19:02:45.835260+0000","last_fullsized":"2024-09-20T19:02:45.835260+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'20","last_scrub_stamp":"2024-09-20T19:02:35.930910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:35.930910+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:40.603359+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":629,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.835506+0000","last_change":"2024-09-20T19:02:30.686939+0000","last_active":"2024-09-20T19:02:45.835506+0000","last_peered":"2024-09-20T19:02:45.835506+0000","last_clean":"2024-09-20T19:02:45.835506+0000","last_became_active":"2024-09-20T18:49:45.942496+0000","last_became_peered":"2024-09-20T18:49:45.942496+0000","last_unstale":"2024-09-20T19:02:45.835506+0000","last_undegraded":"2024-09-20T19:02:45.835506+0000","last_fullsized":"2024-09-20T19:02:45.835506+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:30.686872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:30.686872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:51.214827+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"262'10","reported_seq":788,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-20T19:02:38.261608+0000","last_change":"2024-09-20T19:02:04.994109+0000","last_active":"2024-09-20T19:02:38.261608+0000","last_peered":"2024-09-20T19:02:38.261608+0000","last_clean":"2024-09-20T19:02:38.261608+0000","last_became_active":"2024-09-20T18:59:05.080183+0000","last_became_peered":"2024-09-20T18:59:05.080183+0000","last_unstale":"2024-09-20T19:02:38.261608+0000","last_undegraded":"2024-09-20T19:02:38.261608+0000","last_fullsized":"2024-09-20T19:02:38.261608+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'10","last_scrub_stamp":"2024-09-20T19:02:04.994047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:04.994047+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:34.082210+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":582,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-20T19:02:38.268542+0000","last_change":"2024-09-20T19:01:37.900410+0000","last_active":"2024-09-20T19:02:38.268542+0000","last_peered":"2024-09-20T19:02:38.268542+0000","last_clean":"2024-09-20T19:02:38.268542+0000","last_became_active":"2024-09-20T18:49:51.964780+0000","last_became_peered":"2024-09-20T18:49:51.964780+0000","last_unstale":"2024-09-20T19:02:38.268542+0000","last_undegraded":"2024-09-20T19:02:38.268542+0000","last_fullsized":"2024-09-20T19:02:38.268542+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:37.900328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:37.900328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:02:56.513120+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.d","version":"342'3710","reported_seq":3754,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.994599+0000","last_change":"2024-09-20T19:02:11.829599+0000","last_active":"2024-09-20T19:02:45.994599+0000","last_peered":"2024-09-20T19:02:45.994599+0000","last_clean":"2024-09-20T19:02:45.994599+0000","last_became_active":"2024-09-20T19:02:11.829309+0000","last_became_peered":"2024-09-20T19:02:11.829309+0000","last_unstale":"2024-09-20T19:02:45.994599+0000","last_undegraded":"2024-09-20T19:02:45.994599+0000","last_fullsized":"2024-09-20T19:02:45.994599+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":3710,"log_dups_size":0,"ondisk_log_size":3710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:26.741575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243138560,"num_objects":3710,"num_object_clones":0,"num_object_copies":11130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3710,"num_write_kb":237440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"274'29","reported_seq":649,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-20T19:02:38.262367+0000","last_change":"2024-09-20T19:00:04.810907+0000","last_active":"2024-09-20T19:02:38.262367+0000","last_peered":"2024-09-20T19:02:38.262367+0000","last_clean":"2024-09-20T19:02:38.262367+0000","last_became_active":"2024-09-20T18:49:48.497736+0000","last_became_peered":"2024-09-20T18:49:48.497736+0000","last_unstale":"2024-09-20T19:02:38.262367+0000","last_undegraded":"2024-09-20T19:02:38.262367+0000","last_fullsized":"2024-09-20T19:02:38.262367+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'29","last_scrub_stamp":"2024-09-20T19:00:04.810831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:00:04.810831+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:18.029237+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":605,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.833958+0000","last_change":"2024-09-20T19:02:41.859231+0000","last_active":"2024-09-20T19:02:45.833958+0000","last_peered":"2024-09-20T19:02:45.833958+0000","last_clean":"2024-09-20T19:02:45.833958+0000","last_became_active":"2024-09-20T18:49:45.942253+0000","last_became_peered":"2024-09-20T18:49:45.942253+0000","last_unstale":"2024-09-20T19:02:45.833958+0000","last_undegraded":"2024-09-20T19:02:45.833958+0000","last_fullsized":"2024-09-20T19:02:45.833958+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:41.859168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:41.859168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:03.925342+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":191,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.832977+0000","last_change":"2024-09-20T19:02:27.639446+0000","last_active":"2024-09-20T19:02:45.832977+0000","last_peered":"2024-09-20T19:02:45.832977+0000","last_clean":"2024-09-20T19:02:45.832977+0000","last_became_active":"2024-09-20T18:59:05.085636+0000","last_became_peered":"2024-09-20T18:59:05.085636+0000","last_unstale":"2024-09-20T19:02:45.832977+0000","last_undegraded":"2024-09-20T19:02:45.832977+0000","last_fullsized":"2024-09-20T19:02:45.832977+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:27.639384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:27.639384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:29.637433+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":623,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.834306+0000","last_change":"2024-09-20T19:02:23.886386+0000","last_active":"2024-09-20T19:02:45.834306+0000","last_peered":"2024-09-20T19:02:45.834306+0000","last_clean":"2024-09-20T19:02:45.834306+0000","last_became_active":"2024-09-20T18:59:05.085131+0000","last_became_peered":"2024-09-20T18:59:05.085131+0000","last_unstale":"2024-09-20T19:02:45.834306+0000","last_undegraded":"2024-09-20T19:02:45.834306+0000","last_fullsized":"2024-09-20T19:02:45.834306+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:23.886308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:23.886308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:42.873874+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"342'3478","reported_seq":3523,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.998998+0000","last_change":"2024-09-20T19:02:11.827946+0000","last_active":"2024-09-20T19:02:45.998998+0000","last_peered":"2024-09-20T19:02:45.998998+0000","last_clean":"2024-09-20T19:02:45.998998+0000","last_became_active":"2024-09-20T19:02:11.827305+0000","last_became_peered":"2024-09-20T19:02:11.827305+0000","last_unstale":"2024-09-20T19:02:45.998998+0000","last_undegraded":"2024-09-20T19:02:45.998998+0000","last_fullsized":"2024-09-20T19:02:45.998998+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":3478,"log_dups_size":0,"ondisk_log_size":3478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:14.570559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":227934208,"num_objects":3478,"num_object_clones":0,"num_object_copies":10434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3478,"num_write_kb":222592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"274'40","reported_seq":840,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.833904+0000","last_change":"2024-09-20T19:02:36.901021+0000","last_active":"2024-09-20T19:02:45.833904+0000","last_peered":"2024-09-20T19:02:45.833904+0000","last_clean":"2024-09-20T19:02:45.833904+0000","last_became_active":"2024-09-20T18:49:47.955335+0000","last_became_peered":"2024-09-20T18:49:47.955335+0000","last_unstale":"2024-09-20T19:02:45.833904+0000","last_undegraded":"2024-09-20T19:02:45.833904+0000","last_fullsized":"2024-09-20T19:02:45.833904+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'40","last_scrub_stamp":"2024-09-20T19:02:36.900916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:36.900916+0000","objects_scrubbed":10,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:42.732206+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":594,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-09-20T19:02:42.865711+0000","last_change":"2024-09-20T19:01:22.844253+0000","last_active":"2024-09-20T19:02:42.865711+0000","last_peered":"2024-09-20T19:02:42.865711+0000","last_clean":"2024-09-20T19:02:42.865711+0000","last_became_active":"2024-09-20T19:01:22.843966+0000","last_became_peered":"2024-09-20T19:01:22.843966+0000","last_unstale":"2024-09-20T19:02:42.865711+0000","last_undegraded":"2024-09-20T19:02:42.865711+0000","last_fullsized":"2024-09-20T19:02:42.865711+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:00:31.630361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:00:31.630361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":661,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.835904+0000","last_change":"2024-09-20T19:01:39.517101+0000","last_active":"2024-09-20T19:02:45.835904+0000","last_peered":"2024-09-20T19:02:45.835904+0000","last_clean":"2024-09-20T19:02:45.835904+0000","last_became_active":"2024-09-20T18:59:05.086786+0000","last_became_peered":"2024-09-20T18:59:05.086786+0000","last_unstale":"2024-09-20T19:02:45.835904+0000","last_undegraded":"2024-09-20T19:02:45.835904+0000","last_fullsized":"2024-09-20T19:02:45.835904+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-20T19:01:39.517017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:01:39.517017+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:08.168757+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.0012781159999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":613,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-20T19:02:38.263100+0000","last_change":"2024-09-20T19:01:22.852983+0000","last_active":"2024-09-20T19:02:38.263100+0000","last_peered":"2024-09-20T19:02:38.263100+0000","last_clean":"2024-09-20T19:02:38.263100+0000","last_became_active":"2024-09-20T19:01:22.852495+0000","last_became_peered":"2024-09-20T19:01:22.852495+0000","last_unstale":"2024-09-20T19:02:38.263100+0000","last_undegraded":"2024-09-20T19:02:38.263100+0000","last_fullsized":"2024-09-20T19:02:38.263100+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:59.099736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:59.099736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:13.023185+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":591,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-20T19:02:38.269453+0000","last_change":"2024-09-20T19:02:04.526741+0000","last_active":"2024-09-20T19:02:38.269453+0000","last_peered":"2024-09-20T19:02:38.269453+0000","last_clean":"2024-09-20T19:02:38.269453+0000","last_became_active":"2024-09-20T19:01:22.837728+0000","last_became_peered":"2024-09-20T19:01:22.837728+0000","last_unstale":"2024-09-20T19:02:38.269453+0000","last_undegraded":"2024-09-20T19:02:38.269453+0000","last_fullsized":"2024-09-20T19:02:38.269453+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:04.526683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:04.526683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:22.012800+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"342'3627","reported_seq":3672,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:46.291903+0000","last_change":"2024-09-20T19:02:11.825132+0000","last_active":"2024-09-20T19:02:46.291903+0000","last_peered":"2024-09-20T19:02:46.291903+0000","last_clean":"2024-09-20T19:02:46.291903+0000","last_became_active":"2024-09-20T19:02:11.824846+0000","last_became_peered":"2024-09-20T19:02:11.824846+0000","last_unstale":"2024-09-20T19:02:46.291903+0000","last_undegraded":"2024-09-20T19:02:46.291903+0000","last_fullsized":"2024-09-20T19:02:46.291903+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":3627,"log_dups_size":0,"ondisk_log_size":3627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:12.639014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":237699072,"num_objects":3627,"num_object_clones":0,"num_object_copies":10881,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3627,"num_write_kb":232128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"273'17","reported_seq":649,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.834701+0000","last_change":"2024-09-20T19:02:24.853473+0000","last_active":"2024-09-20T19:02:45.834701+0000","last_peered":"2024-09-20T19:02:45.834701+0000","last_clean":"2024-09-20T19:02:45.834701+0000","last_became_active":"2024-09-20T18:49:47.950904+0000","last_became_peered":"2024-09-20T18:49:47.950904+0000","last_unstale":"2024-09-20T19:02:45.834701+0000","last_undegraded":"2024-09-20T19:02:45.834701+0000","last_fullsized":"2024-09-20T19:02:45.834701+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'17","last_scrub_stamp":"2024-09-20T19:02:24.853374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:24.853374+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:46.813829+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":36,"num_read_kb":25,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":635,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.834003+0000","last_change":"2024-09-20T19:02:29.840761+0000","last_active":"2024-09-20T19:02:45.834003+0000","last_peered":"2024-09-20T19:02:45.834003+0000","last_clean":"2024-09-20T19:02:45.834003+0000","last_became_active":"2024-09-20T18:59:05.079789+0000","last_became_peered":"2024-09-20T18:59:05.079789+0000","last_unstale":"2024-09-20T19:02:45.834003+0000","last_undegraded":"2024-09-20T19:02:45.834003+0000","last_fullsized":"2024-09-20T19:02:45.834003+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:29.840601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:29.840601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:39.311309+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":617,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.837626+0000","last_change":"2024-09-20T19:01:58.789304+0000","last_active":"2024-09-20T19:02:45.837626+0000","last_peered":"2024-09-20T19:02:45.837626+0000","last_clean":"2024-09-20T19:02:45.837626+0000","last_became_active":"2024-09-20T18:49:44.937135+0000","last_became_peered":"2024-09-20T18:49:44.937135+0000","last_unstale":"2024-09-20T19:02:45.837626+0000","last_undegraded":"2024-09-20T19:02:45.837626+0000","last_fullsized":"2024-09-20T19:02:45.837626+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:01:58.789251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:01:58.789251+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:25.250777+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00095558800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":592,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.843389+0000","last_change":"2024-09-20T19:01:17.747452+0000","last_active":"2024-09-20T19:02:45.843389+0000","last_peered":"2024-09-20T19:02:45.843389+0000","last_clean":"2024-09-20T19:02:45.843389+0000","last_became_active":"2024-09-20T18:49:49.954424+0000","last_became_peered":"2024-09-20T18:49:49.954424+0000","last_unstale":"2024-09-20T19:02:45.843389+0000","last_undegraded":"2024-09-20T19:02:45.843389+0000","last_fullsized":"2024-09-20T19:02:45.843389+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:17.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:17.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":603,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.832509+0000","last_change":"2024-09-20T19:01:15.203591+0000","last_active":"2024-09-20T19:02:45.832509+0000","last_peered":"2024-09-20T19:02:45.832509+0000","last_clean":"2024-09-20T19:02:45.832509+0000","last_became_active":"2024-09-20T18:59:05.081459+0000","last_became_peered":"2024-09-20T18:59:05.081459+0000","last_unstale":"2024-09-20T19:02:45.832509+0000","last_undegraded":"2024-09-20T19:02:45.832509+0000","last_fullsized":"2024-09-20T19:02:45.832509+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:15.203529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:15.203529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"340'3124","reported_seq":3165,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-20T19:02:42.333315+0000","last_change":"2024-09-20T19:02:11.828477+0000","last_active":"2024-09-20T19:02:42.333315+0000","last_peered":"2024-09-20T19:02:42.333315+0000","last_clean":"2024-09-20T19:02:42.333315+0000","last_became_active":"2024-09-20T19:02:11.827996+0000","last_became_peered":"2024-09-20T19:02:11.827996+0000","last_unstale":"2024-09-20T19:02:42.333315+0000","last_undegraded":"2024-09-20T19:02:42.333315+0000","last_fullsized":"2024-09-20T19:02:42.333315+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":3124,"log_dups_size":0,"ondisk_log_size":3124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:30.922698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":204734464,"num_objects":3124,"num_object_clones":0,"num_object_copies":9372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3124,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3124,"num_write_kb":199936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"273'24","reported_seq":649,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-20T19:02:38.262900+0000","last_change":"2024-09-20T18:59:06.022021+0000","last_active":"2024-09-20T19:02:38.262900+0000","last_peered":"2024-09-20T19:02:38.262900+0000","last_clean":"2024-09-20T19:02:38.262900+0000","last_became_active":"2024-09-20T18:49:47.949230+0000","last_became_peered":"2024-09-20T18:49:47.949230+0000","last_unstale":"2024-09-20T19:02:38.262900+0000","last_undegraded":"2024-09-20T19:02:38.262900+0000","last_fullsized":"2024-09-20T19:02:38.262900+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:59:06.021962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:59:06.021962+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:11.985717+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":621,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-20T19:02:38.262064+0000","last_change":"2024-09-20T19:02:24.984120+0000","last_active":"2024-09-20T19:02:38.262064+0000","last_peered":"2024-09-20T19:02:38.262064+0000","last_clean":"2024-09-20T19:02:38.262064+0000","last_became_active":"2024-09-20T19:01:22.845659+0000","last_became_peered":"2024-09-20T19:01:22.845659+0000","last_unstale":"2024-09-20T19:02:38.262064+0000","last_undegraded":"2024-09-20T19:02:38.262064+0000","last_fullsized":"2024-09-20T19:02:38.262064+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:24.984022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:24.984022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:27.154123+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":1399,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.831729+0000","last_change":"2024-09-20T19:02:02.494344+0000","last_active":"2024-09-20T19:02:45.831729+0000","last_peered":"2024-09-20T19:02:45.831729+0000","last_clean":"2024-09-20T19:02:45.831729+0000","last_became_active":"2024-09-20T18:49:44.936819+0000","last_became_peered":"2024-09-20T18:49:44.936819+0000","last_unstale":"2024-09-20T19:02:45.831729+0000","last_undegraded":"2024-09-20T19:02:45.831729+0000","last_fullsized":"2024-09-20T19:02:45.831729+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-20T19:02:02.494291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:02:02.494291+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:09.952437+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0010996199999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":774,"num_read_kb":769,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":612,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.835909+0000","last_change":"2024-09-20T19:02:27.785566+0000","last_active":"2024-09-20T19:02:45.835909+0000","last_peered":"2024-09-20T19:02:45.835909+0000","last_clean":"2024-09-20T19:02:45.835909+0000","last_became_active":"2024-09-20T18:49:49.961066+0000","last_became_peered":"2024-09-20T18:49:49.961066+0000","last_unstale":"2024-09-20T19:02:45.835909+0000","last_undegraded":"2024-09-20T19:02:45.835909+0000","last_fullsized":"2024-09-20T19:02:45.835909+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:27.785524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:27.785524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:48.498566+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":609,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.833433+0000","last_change":"2024-09-20T19:02:26.861253+0000","last_active":"2024-09-20T19:02:45.833433+0000","last_peered":"2024-09-20T19:02:45.833433+0000","last_clean":"2024-09-20T19:02:45.833433+0000","last_became_active":"2024-09-20T18:49:51.970343+0000","last_became_peered":"2024-09-20T18:49:51.970343+0000","last_unstale":"2024-09-20T19:02:45.833433+0000","last_undegraded":"2024-09-20T19:02:45.833433+0000","last_fullsized":"2024-09-20T19:02:45.833433+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:26.861185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:26.861185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:38.453435+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"342'3585","reported_seq":3630,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.988667+0000","last_change":"2024-09-20T19:02:11.828065+0000","last_active":"2024-09-20T19:02:45.988667+0000","last_peered":"2024-09-20T19:02:45.988667+0000","last_clean":"2024-09-20T19:02:45.988667+0000","last_became_active":"2024-09-20T19:02:11.827628+0000","last_became_peered":"2024-09-20T19:02:11.827628+0000","last_unstale":"2024-09-20T19:02:45.988667+0000","last_undegraded":"2024-09-20T19:02:45.988667+0000","last_fullsized":"2024-09-20T19:02:45.988667+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":3585,"log_dups_size":0,"ondisk_log_size":3585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:21.892715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":234946560,"num_objects":3585,"num_object_clones":0,"num_object_copies":10755,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3585,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3585,"num_write_kb":229440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"308'32","reported_seq":276,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.832426+0000","last_change":"2024-09-20T19:02:25.699418+0000","last_active":"2024-09-20T19:02:45.832426+0000","last_peered":"2024-09-20T19:02:45.832426+0000","last_clean":"2024-09-20T19:02:45.832426+0000","last_became_active":"2024-09-20T19:01:22.846100+0000","last_became_peered":"2024-09-20T19:01:22.846100+0000","last_unstale":"2024-09-20T19:02:45.832426+0000","last_undegraded":"2024-09-20T19:02:45.832426+0000","last_fullsized":"2024-09-20T19:02:45.832426+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"308'32","last_scrub_stamp":"2024-09-20T19:02:25.699337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:25.699337+0000","objects_scrubbed":8,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:28.007253+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":623,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-20T19:02:38.269114+0000","last_change":"2024-09-20T19:02:31.965132+0000","last_active":"2024-09-20T19:02:38.269114+0000","last_peered":"2024-09-20T19:02:38.269114+0000","last_clean":"2024-09-20T19:02:38.269114+0000","last_became_active":"2024-09-20T18:59:05.079111+0000","last_became_peered":"2024-09-20T18:59:05.079111+0000","last_unstale":"2024-09-20T19:02:38.269114+0000","last_undegraded":"2024-09-20T19:02:38.269114+0000","last_fullsized":"2024-09-20T19:02:38.269114+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:31.965066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:31.965066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:50.572885+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":673,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.831894+0000","last_change":"2024-09-20T18:59:05.108551+0000","last_active":"2024-09-20T19:02:45.831894+0000","last_peered":"2024-09-20T19:02:45.831894+0000","last_clean":"2024-09-20T19:02:45.831894+0000","last_became_active":"2024-09-20T18:59:05.086103+0000","last_became_peered":"2024-09-20T18:59:05.086103+0000","last_unstale":"2024-09-20T19:02:45.831894+0000","last_undegraded":"2024-09-20T19:02:45.831894+0000","last_fullsized":"2024-09-20T19:02:45.831894+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T18:55:13.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T18:55:13.696607+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:10.266543+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":590,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.832818+0000","last_change":"2024-09-20T18:58:23.545370+0000","last_active":"2024-09-20T19:02:45.832818+0000","last_peered":"2024-09-20T19:02:45.832818+0000","last_clean":"2024-09-20T19:02:45.832818+0000","last_became_active":"2024-09-20T18:49:49.956128+0000","last_became_peered":"2024-09-20T18:49:49.956128+0000","last_unstale":"2024-09-20T19:02:45.832818+0000","last_undegraded":"2024-09-20T19:02:45.832818+0000","last_fullsized":"2024-09-20T19:02:45.832818+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:23.545321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:23.545321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:12.296749+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":619,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.835733+0000","last_change":"2024-09-20T19:01:30.557430+0000","last_active":"2024-09-20T19:02:45.835733+0000","last_peered":"2024-09-20T19:02:45.835733+0000","last_clean":"2024-09-20T19:02:45.835733+0000","last_became_active":"2024-09-20T19:01:22.847692+0000","last_became_peered":"2024-09-20T19:01:22.847692+0000","last_unstale":"2024-09-20T19:02:45.835733+0000","last_undegraded":"2024-09-20T19:02:45.835733+0000","last_fullsized":"2024-09-20T19:02:45.835733+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:30.557368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:30.557368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:02:45.868996+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.0","version":"342'3615","reported_seq":3660,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.999125+0000","last_change":"2024-09-20T19:02:11.825018+0000","last_active":"2024-09-20T19:02:45.999125+0000","last_peered":"2024-09-20T19:02:45.999125+0000","last_clean":"2024-09-20T19:02:45.999125+0000","last_became_active":"2024-09-20T19:02:11.824703+0000","last_became_peered":"2024-09-20T19:02:11.824703+0000","last_unstale":"2024-09-20T19:02:45.999125+0000","last_undegraded":"2024-09-20T19:02:45.999125+0000","last_fullsized":"2024-09-20T19:02:45.999125+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":3615,"log_dups_size":0,"ondisk_log_size":3615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:38.061938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":236912640,"num_objects":3615,"num_object_clones":0,"num_object_copies":10845,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3615,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3615,"num_write_kb":231360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"274'19","reported_seq":612,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-09-20T19:02:42.866226+0000","last_change":"2024-09-20T19:01:22.843592+0000","last_active":"2024-09-20T19:02:42.866226+0000","last_peered":"2024-09-20T19:02:42.866226+0000","last_clean":"2024-09-20T19:02:42.866226+0000","last_became_active":"2024-09-20T19:01:22.843143+0000","last_became_peered":"2024-09-20T19:01:22.843143+0000","last_unstale":"2024-09-20T19:02:42.866226+0000","last_undegraded":"2024-09-20T19:02:42.866226+0000","last_fullsized":"2024-09-20T19:02:42.866226+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:58:24.616580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:24.616580+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":572,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-09-20T19:02:42.865382+0000","last_change":"2024-09-20T19:01:22.837818+0000","last_active":"2024-09-20T19:02:42.865382+0000","last_peered":"2024-09-20T19:02:42.865382+0000","last_clean":"2024-09-20T19:02:42.865382+0000","last_became_active":"2024-09-20T19:01:22.836557+0000","last_became_peered":"2024-09-20T19:01:22.836557+0000","last_unstale":"2024-09-20T19:02:42.865382+0000","last_undegraded":"2024-09-20T19:02:42.865382+0000","last_fullsized":"2024-09-20T19:02:42.865382+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.693993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.693993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":637,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.836091+0000","last_change":"2024-09-20T19:01:38.463472+0000","last_active":"2024-09-20T19:02:45.836091+0000","last_peered":"2024-09-20T19:02:45.836091+0000","last_clean":"2024-09-20T19:02:45.836091+0000","last_became_active":"2024-09-20T19:01:22.831488+0000","last_became_peered":"2024-09-20T19:01:22.831488+0000","last_unstale":"2024-09-20T19:02:45.836091+0000","last_undegraded":"2024-09-20T19:02:45.836091+0000","last_fullsized":"2024-09-20T19:02:45.836091+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:38.463404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:38.463404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:02:51.812777+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":602,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-20T19:02:38.270212+0000","last_change":"2024-09-20T18:59:05.080255+0000","last_active":"2024-09-20T19:02:38.270212+0000","last_peered":"2024-09-20T19:02:38.270212+0000","last_clean":"2024-09-20T19:02:38.270212+0000","last_became_active":"2024-09-20T18:59:05.079897+0000","last_became_peered":"2024-09-20T18:59:05.079897+0000","last_unstale":"2024-09-20T19:02:38.270212+0000","last_undegraded":"2024-09-20T19:02:38.270212+0000","last_fullsized":"2024-09-20T19:02:38.270212+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:01.083156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:59:01.083156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"342'3640","reported_seq":3685,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:46.379979+0000","last_change":"2024-09-20T19:02:11.825263+0000","last_active":"2024-09-20T19:02:46.379979+0000","last_peered":"2024-09-20T19:02:46.379979+0000","last_clean":"2024-09-20T19:02:46.379979+0000","last_became_active":"2024-09-20T19:02:11.825011+0000","last_became_peered":"2024-09-20T19:02:11.825011+0000","last_unstale":"2024-09-20T19:02:46.379979+0000","last_undegraded":"2024-09-20T19:02:46.379979+0000","last_fullsized":"2024-09-20T19:02:46.379979+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":3640,"log_dups_size":0,"ondisk_log_size":3640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:12.092479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":238551040,"num_objects":3640,"num_object_clones":0,"num_object_copies":10920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3640,"num_write_kb":232960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"274'31","reported_seq":619,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-09-20T19:02:42.866098+0000","last_change":"2024-09-20T19:01:22.838956+0000","last_active":"2024-09-20T19:02:42.866098+0000","last_peered":"2024-09-20T19:02:42.866098+0000","last_clean":"2024-09-20T19:02:42.866098+0000","last_became_active":"2024-09-20T19:01:22.838577+0000","last_became_peered":"2024-09-20T19:01:22.838577+0000","last_unstale":"2024-09-20T19:02:42.866098+0000","last_undegraded":"2024-09-20T19:02:42.866098+0000","last_fullsized":"2024-09-20T19:02:42.866098+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'31","last_scrub_stamp":"2024-09-20T19:00:03.509980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:00:03.509980+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":49,"num_read_kb":39,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":194,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.833134+0000","last_change":"2024-09-20T19:01:42.479567+0000","last_active":"2024-09-20T19:02:45.833134+0000","last_peered":"2024-09-20T19:02:45.833134+0000","last_clean":"2024-09-20T19:02:45.833134+0000","last_became_active":"2024-09-20T19:01:22.845553+0000","last_became_peered":"2024-09-20T19:01:22.845553+0000","last_unstale":"2024-09-20T19:02:45.833134+0000","last_undegraded":"2024-09-20T19:02:45.833134+0000","last_fullsized":"2024-09-20T19:02:45.833134+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:42.479489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:42.479489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:02:53.833459+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":615,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.833597+0000","last_change":"2024-09-20T19:02:32.856930+0000","last_active":"2024-09-20T19:02:45.833597+0000","last_peered":"2024-09-20T19:02:45.833597+0000","last_clean":"2024-09-20T19:02:45.833597+0000","last_became_active":"2024-09-20T19:01:22.837146+0000","last_became_peered":"2024-09-20T19:01:22.837146+0000","last_unstale":"2024-09-20T19:02:45.833597+0000","last_undegraded":"2024-09-20T19:02:45.833597+0000","last_fullsized":"2024-09-20T19:02:45.833597+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:32.856856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:32.856856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:59.715742+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":614,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.843604+0000","last_change":"2024-09-20T19:01:31.687258+0000","last_active":"2024-09-20T19:02:45.843604+0000","last_peered":"2024-09-20T19:02:45.843604+0000","last_clean":"2024-09-20T19:02:45.843604+0000","last_became_active":"2024-09-20T19:01:22.833913+0000","last_became_peered":"2024-09-20T19:01:22.833913+0000","last_unstale":"2024-09-20T19:02:45.843604+0000","last_undegraded":"2024-09-20T19:02:45.843604+0000","last_fullsized":"2024-09-20T19:02:45.843604+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:31.687188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:31.687188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:44.818347+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2","version":"342'3695","reported_seq":3739,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:46.904657+0000","last_change":"2024-09-20T19:02:11.829938+0000","last_active":"2024-09-20T19:02:46.904657+0000","last_peered":"2024-09-20T19:02:46.904657+0000","last_clean":"2024-09-20T19:02:46.904657+0000","last_became_active":"2024-09-20T19:02:11.829572+0000","last_became_peered":"2024-09-20T19:02:11.829572+0000","last_unstale":"2024-09-20T19:02:46.904657+0000","last_undegraded":"2024-09-20T19:02:46.904657+0000","last_fullsized":"2024-09-20T19:02:46.904657+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":3695,"log_dups_size":0,"ondisk_log_size":3695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:34.876687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":242155520,"num_objects":3695,"num_object_clones":0,"num_object_copies":11085,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3695,"num_write_kb":236480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"273'20","reported_seq":617,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-09-20T19:02:42.865715+0000","last_change":"2024-09-20T19:01:22.850846+0000","last_active":"2024-09-20T19:02:42.865715+0000","last_peered":"2024-09-20T19:02:42.865715+0000","last_clean":"2024-09-20T19:02:42.865715+0000","last_became_active":"2024-09-20T19:01:22.848851+0000","last_became_peered":"2024-09-20T19:01:22.848851+0000","last_unstale":"2024-09-20T19:02:42.865715+0000","last_undegraded":"2024-09-20T19:02:42.865715+0000","last_fullsized":"2024-09-20T19:02:42.865715+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:59:10.364583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:59:10.364583+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":619,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-20T19:02:38.270335+0000","last_change":"2024-09-20T19:02:34.986113+0000","last_active":"2024-09-20T19:02:38.270335+0000","last_peered":"2024-09-20T19:02:38.270335+0000","last_clean":"2024-09-20T19:02:38.270335+0000","last_became_active":"2024-09-20T19:01:22.835966+0000","last_became_peered":"2024-09-20T19:01:22.835966+0000","last_unstale":"2024-09-20T19:02:38.270335+0000","last_undegraded":"2024-09-20T19:02:38.270335+0000","last_fullsized":"2024-09-20T19:02:38.270335+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:34.985163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:34.985163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:44.611937+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":780,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.839390+0000","last_change":"2024-09-20T19:02:02.813926+0000","last_active":"2024-09-20T19:02:45.839390+0000","last_peered":"2024-09-20T19:02:45.839390+0000","last_clean":"2024-09-20T19:02:45.839390+0000","last_became_active":"2024-09-20T18:49:49.957005+0000","last_became_peered":"2024-09-20T18:49:49.957005+0000","last_unstale":"2024-09-20T19:02:45.839390+0000","last_undegraded":"2024-09-20T19:02:45.839390+0000","last_fullsized":"2024-09-20T19:02:45.839390+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T19:02:02.813865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:02.813865+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:31.557168+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":546,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-09-20T19:02:42.866335+0000","last_change":"2024-09-20T19:01:22.843904+0000","last_active":"2024-09-20T19:02:42.866335+0000","last_peered":"2024-09-20T19:02:42.866335+0000","last_clean":"2024-09-20T19:02:42.866335+0000","last_became_active":"2024-09-20T19:01:22.843167+0000","last_became_peered":"2024-09-20T19:01:22.843167+0000","last_unstale":"2024-09-20T19:02:42.866335+0000","last_undegraded":"2024-09-20T19:02:42.866335+0000","last_fullsized":"2024-09-20T19:02:42.866335+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:48.398542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:59:48.398542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.5","version":"342'3563","reported_seq":3608,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:46.359928+0000","last_change":"2024-09-20T19:02:11.828434+0000","last_active":"2024-09-20T19:02:46.359928+0000","last_peered":"2024-09-20T19:02:46.359928+0000","last_clean":"2024-09-20T19:02:46.359928+0000","last_became_active":"2024-09-20T19:02:11.828162+0000","last_became_peered":"2024-09-20T19:02:11.828162+0000","last_unstale":"2024-09-20T19:02:46.359928+0000","last_undegraded":"2024-09-20T19:02:46.359928+0000","last_fullsized":"2024-09-20T19:02:46.359928+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":3563,"log_dups_size":0,"ondisk_log_size":3563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:13.184255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":233504768,"num_objects":3563,"num_object_clones":0,"num_object_copies":10689,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3563,"num_write_kb":228032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"274'15","reported_seq":641,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-20T19:02:38.261659+0000","last_change":"2024-09-20T19:01:27.802153+0000","last_active":"2024-09-20T19:02:38.261659+0000","last_peered":"2024-09-20T19:02:38.261659+0000","last_clean":"2024-09-20T19:02:38.261659+0000","last_became_active":"2024-09-20T19:01:22.845683+0000","last_became_peered":"2024-09-20T19:01:22.845683+0000","last_unstale":"2024-09-20T19:02:38.261659+0000","last_undegraded":"2024-09-20T19:02:38.261659+0000","last_fullsized":"2024-09-20T19:02:38.261659+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'15","last_scrub_stamp":"2024-09-20T19:01:27.802100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:27.802100+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:02:51.153972+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":149,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.837694+0000","last_change":"2024-09-20T19:01:52.716074+0000","last_active":"2024-09-20T19:02:45.837694+0000","last_peered":"2024-09-20T19:02:45.837694+0000","last_clean":"2024-09-20T19:02:45.837694+0000","last_became_active":"2024-09-20T19:00:01.190875+0000","last_became_peered":"2024-09-20T19:00:01.190875+0000","last_unstale":"2024-09-20T19:02:45.837694+0000","last_undegraded":"2024-09-20T19:02:45.837694+0000","last_fullsized":"2024-09-20T19:02:45.837694+0000","mapping_epoch":277,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":278,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:52.716025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:52.716025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:08.654757+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":618,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.837489+0000","last_change":"2024-09-20T19:02:24.939055+0000","last_active":"2024-09-20T19:02:45.837489+0000","last_peered":"2024-09-20T19:02:45.837489+0000","last_clean":"2024-09-20T19:02:45.837489+0000","last_became_active":"2024-09-20T19:01:22.835347+0000","last_became_peered":"2024-09-20T19:01:22.835347+0000","last_unstale":"2024-09-20T19:02:45.837489+0000","last_undegraded":"2024-09-20T19:02:45.837489+0000","last_fullsized":"2024-09-20T19:02:45.837489+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:24.938980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:24.938980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:52.810786+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":597,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-20T19:02:38.261558+0000","last_change":"2024-09-20T19:02:31.120809+0000","last_active":"2024-09-20T19:02:38.261558+0000","last_peered":"2024-09-20T19:02:38.261558+0000","last_clean":"2024-09-20T19:02:38.261558+0000","last_became_active":"2024-09-20T18:49:51.964098+0000","last_became_peered":"2024-09-20T18:49:51.964098+0000","last_unstale":"2024-09-20T19:02:38.261558+0000","last_undegraded":"2024-09-20T19:02:38.261558+0000","last_fullsized":"2024-09-20T19:02:38.261558+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:31.120623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:31.120623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:41.294006+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"340'3186","reported_seq":3227,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-20T19:02:42.337269+0000","last_change":"2024-09-20T19:02:11.829578+0000","last_active":"2024-09-20T19:02:42.337269+0000","last_peered":"2024-09-20T19:02:42.337269+0000","last_clean":"2024-09-20T19:02:42.337269+0000","last_became_active":"2024-09-20T19:02:11.828890+0000","last_became_peered":"2024-09-20T19:02:11.828890+0000","last_unstale":"2024-09-20T19:02:42.337269+0000","last_undegraded":"2024-09-20T19:02:42.337269+0000","last_fullsized":"2024-09-20T19:02:42.337269+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":3186,"log_dups_size":0,"ondisk_log_size":3186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:19.809730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":208797696,"num_objects":3186,"num_object_clones":0,"num_object_copies":9558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3186,"num_write_kb":203904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"274'35","reported_seq":227,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.832889+0000","last_change":"2024-09-20T19:01:40.477072+0000","last_active":"2024-09-20T19:02:45.832889+0000","last_peered":"2024-09-20T19:02:45.832889+0000","last_clean":"2024-09-20T19:02:45.832889+0000","last_became_active":"2024-09-20T18:59:05.087391+0000","last_became_peered":"2024-09-20T18:59:05.087391+0000","last_unstale":"2024-09-20T19:02:45.832889+0000","last_undegraded":"2024-09-20T19:02:45.832889+0000","last_fullsized":"2024-09-20T19:02:45.832889+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'35","last_scrub_stamp":"2024-09-20T19:01:40.476968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:40.476968+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:09.580262+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":627,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.832839+0000","last_change":"2024-09-20T19:02:05.440745+0000","last_active":"2024-09-20T19:02:45.832839+0000","last_peered":"2024-09-20T19:02:45.832839+0000","last_clean":"2024-09-20T19:02:45.832839+0000","last_became_active":"2024-09-20T18:59:05.086670+0000","last_became_peered":"2024-09-20T18:59:05.086670+0000","last_unstale":"2024-09-20T19:02:45.832839+0000","last_undegraded":"2024-09-20T19:02:45.832839+0000","last_fullsized":"2024-09-20T19:02:45.832839+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:05.440677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:05.440677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:22.708355+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":615,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.838163+0000","last_change":"2024-09-20T19:02:42.745448+0000","last_active":"2024-09-20T19:02:45.838163+0000","last_peered":"2024-09-20T19:02:45.838163+0000","last_clean":"2024-09-20T19:02:45.838163+0000","last_became_active":"2024-09-20T18:49:49.957558+0000","last_became_peered":"2024-09-20T18:49:49.957558+0000","last_unstale":"2024-09-20T19:02:45.838163+0000","last_undegraded":"2024-09-20T19:02:45.838163+0000","last_fullsized":"2024-09-20T19:02:45.838163+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:42.745372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:42.745372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:49.093207+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":593,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.831816+0000","last_change":"2024-09-20T18:53:47.370152+0000","last_active":"2024-09-20T19:02:45.831816+0000","last_peered":"2024-09-20T19:02:45.831816+0000","last_clean":"2024-09-20T19:02:45.831816+0000","last_became_active":"2024-09-20T18:49:51.970798+0000","last_became_peered":"2024-09-20T18:49:51.970798+0000","last_unstale":"2024-09-20T19:02:45.831816+0000","last_undegraded":"2024-09-20T19:02:45.831816+0000","last_fullsized":"2024-09-20T19:02:45.831816+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:47.370108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:47.370108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:19.239137+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.7","version":"342'3567","reported_seq":3611,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.984715+0000","last_change":"2024-09-20T19:02:11.827681+0000","last_active":"2024-09-20T19:02:45.984715+0000","last_peered":"2024-09-20T19:02:45.984715+0000","last_clean":"2024-09-20T19:02:45.984715+0000","last_became_active":"2024-09-20T19:02:11.827360+0000","last_became_peered":"2024-09-20T19:02:11.827360+0000","last_unstale":"2024-09-20T19:02:45.984715+0000","last_undegraded":"2024-09-20T19:02:45.984715+0000","last_fullsized":"2024-09-20T19:02:45.984715+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":3567,"log_dups_size":0,"ondisk_log_size":3567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:31.729709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":233766912,"num_objects":3567,"num_object_clones":0,"num_object_copies":10701,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3567,"num_write_kb":228288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"273'18","reported_seq":582,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-09-20T19:02:42.865474+0000","last_change":"2024-09-20T19:01:22.847461+0000","last_active":"2024-09-20T19:02:42.865474+0000","last_peered":"2024-09-20T19:02:42.865474+0000","last_clean":"2024-09-20T19:02:42.865474+0000","last_became_active":"2024-09-20T19:01:22.847140+0000","last_became_peered":"2024-09-20T19:01:22.847140+0000","last_unstale":"2024-09-20T19:02:42.865474+0000","last_undegraded":"2024-09-20T19:02:42.865474+0000","last_fullsized":"2024-09-20T19:02:42.865474+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'18","last_scrub_stamp":"2024-09-20T18:59:55.490086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:59:55.490086+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":624,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.833554+0000","last_change":"2024-09-20T19:01:22.844920+0000","last_active":"2024-09-20T19:02:45.833554+0000","last_peered":"2024-09-20T19:02:45.833554+0000","last_clean":"2024-09-20T19:02:45.833554+0000","last_became_active":"2024-09-20T19:01:22.844608+0000","last_became_peered":"2024-09-20T19:01:22.844608+0000","last_unstale":"2024-09-20T19:02:45.833554+0000","last_undegraded":"2024-09-20T19:02:45.833554+0000","last_fullsized":"2024-09-20T19:02:45.833554+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:00:34.940850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:00:34.940850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":607,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.837229+0000","last_change":"2024-09-20T19:01:53.734774+0000","last_active":"2024-09-20T19:02:45.837229+0000","last_peered":"2024-09-20T19:02:45.837229+0000","last_clean":"2024-09-20T19:02:45.837229+0000","last_became_active":"2024-09-20T18:49:49.954944+0000","last_became_peered":"2024-09-20T18:49:49.954944+0000","last_unstale":"2024-09-20T19:02:45.837229+0000","last_undegraded":"2024-09-20T19:02:45.837229+0000","last_fullsized":"2024-09-20T19:02:45.837229+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:53.734726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:53.734726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:21.517887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":596,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-20T19:02:38.269931+0000","last_change":"2024-09-20T19:02:35.968178+0000","last_active":"2024-09-20T19:02:38.269931+0000","last_peered":"2024-09-20T19:02:38.269931+0000","last_clean":"2024-09-20T19:02:38.269931+0000","last_became_active":"2024-09-20T18:49:51.972970+0000","last_became_peered":"2024-09-20T18:49:51.972970+0000","last_unstale":"2024-09-20T19:02:38.269931+0000","last_undegraded":"2024-09-20T19:02:38.269931+0000","last_fullsized":"2024-09-20T19:02:38.269931+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:35.968052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:35.968052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:53.629726+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.6","version":"342'3660","reported_seq":3705,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:46.334156+0000","last_change":"2024-09-20T19:02:11.829337+0000","last_active":"2024-09-20T19:02:46.334156+0000","last_peered":"2024-09-20T19:02:46.334156+0000","last_clean":"2024-09-20T19:02:46.334156+0000","last_became_active":"2024-09-20T19:02:11.829014+0000","last_became_peered":"2024-09-20T19:02:11.829014+0000","last_unstale":"2024-09-20T19:02:46.334156+0000","last_undegraded":"2024-09-20T19:02:46.334156+0000","last_fullsized":"2024-09-20T19:02:46.334156+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":3660,"log_dups_size":0,"ondisk_log_size":3660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:29.577085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239861760,"num_objects":3660,"num_object_clones":0,"num_object_copies":10980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3660,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3660,"num_write_kb":234240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"273'27","reported_seq":706,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.834750+0000","last_change":"2024-09-20T19:02:39.709344+0000","last_active":"2024-09-20T19:02:45.834750+0000","last_peered":"2024-09-20T19:02:45.834750+0000","last_clean":"2024-09-20T19:02:45.834750+0000","last_became_active":"2024-09-20T18:59:05.083726+0000","last_became_peered":"2024-09-20T18:59:05.083726+0000","last_unstale":"2024-09-20T19:02:45.834750+0000","last_undegraded":"2024-09-20T19:02:45.834750+0000","last_fullsized":"2024-09-20T19:02:45.834750+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'27","last_scrub_stamp":"2024-09-20T19:02:39.709269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:39.709269+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:44.315182+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":644,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.833244+0000","last_change":"2024-09-20T19:01:22.830278+0000","last_active":"2024-09-20T19:02:45.833244+0000","last_peered":"2024-09-20T19:02:45.833244+0000","last_clean":"2024-09-20T19:02:45.833244+0000","last_became_active":"2024-09-20T19:01:22.829928+0000","last_became_peered":"2024-09-20T19:01:22.829928+0000","last_unstale":"2024-09-20T19:02:45.833244+0000","last_undegraded":"2024-09-20T19:02:45.833244+0000","last_fullsized":"2024-09-20T19:02:45.833244+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-20T18:58:57.563058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:57.563058+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:45.184378+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"308'12","reported_seq":789,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.833705+0000","last_change":"2024-09-20T19:01:22.833874+0000","last_active":"2024-09-20T19:02:45.833705+0000","last_peered":"2024-09-20T19:02:45.833705+0000","last_clean":"2024-09-20T19:02:45.833705+0000","last_became_active":"2024-09-20T19:01:22.832710+0000","last_became_peered":"2024-09-20T19:01:22.832710+0000","last_unstale":"2024-09-20T19:02:45.833705+0000","last_undegraded":"2024-09-20T19:02:45.833705+0000","last_fullsized":"2024-09-20T19:02:45.833705+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'9","last_scrub_stamp":"2024-09-20T18:58:27.521051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:27.521051+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:21.248521+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":614,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.835573+0000","last_change":"2024-09-20T19:01:56.634132+0000","last_active":"2024-09-20T19:02:45.835573+0000","last_peered":"2024-09-20T19:02:45.835573+0000","last_clean":"2024-09-20T19:02:45.835573+0000","last_became_active":"2024-09-20T18:59:05.087005+0000","last_became_peered":"2024-09-20T18:59:05.087005+0000","last_unstale":"2024-09-20T19:02:45.835573+0000","last_undegraded":"2024-09-20T19:02:45.835573+0000","last_fullsized":"2024-09-20T19:02:45.835573+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:56.634089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:56.634089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:05.408503+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":183,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.831415+0000","last_change":"2024-09-20T19:01:16.542398+0000","last_active":"2024-09-20T19:02:45.831415+0000","last_peered":"2024-09-20T19:02:45.831415+0000","last_clean":"2024-09-20T19:02:45.831415+0000","last_became_active":"2024-09-20T18:59:05.085427+0000","last_became_peered":"2024-09-20T18:59:05.085427+0000","last_unstale":"2024-09-20T19:02:45.831415+0000","last_undegraded":"2024-09-20T19:02:45.831415+0000","last_fullsized":"2024-09-20T19:02:45.831415+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:16.542335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:16.542335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"273'12","reported_seq":622,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-20T19:02:38.272540+0000","last_change":"2024-09-20T19:01:12.385223+0000","last_active":"2024-09-20T19:02:38.272540+0000","last_peered":"2024-09-20T19:02:38.272540+0000","last_clean":"2024-09-20T19:02:38.272540+0000","last_became_active":"2024-09-20T18:49:48.500662+0000","last_became_peered":"2024-09-20T18:49:48.500662+0000","last_unstale":"2024-09-20T19:02:38.272540+0000","last_undegraded":"2024-09-20T19:02:38.272540+0000","last_fullsized":"2024-09-20T19:02:38.272540+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'12","last_scrub_stamp":"2024-09-20T19:01:12.385152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:12.385152+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":604,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.835823+0000","last_change":"2024-09-20T19:01:14.466096+0000","last_active":"2024-09-20T19:02:45.835823+0000","last_peered":"2024-09-20T19:02:45.835823+0000","last_clean":"2024-09-20T19:02:45.835823+0000","last_became_active":"2024-09-20T18:49:49.956749+0000","last_became_peered":"2024-09-20T18:49:49.956749+0000","last_unstale":"2024-09-20T19:02:45.835823+0000","last_undegraded":"2024-09-20T19:02:45.835823+0000","last_fullsized":"2024-09-20T19:02:45.835823+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:14.466042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:14.466042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:15.787734+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":626,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.832421+0000","last_change":"2024-09-20T19:02:03.509954+0000","last_active":"2024-09-20T19:02:45.832421+0000","last_peered":"2024-09-20T19:02:45.832421+0000","last_clean":"2024-09-20T19:02:45.832421+0000","last_became_active":"2024-09-20T19:01:22.830233+0000","last_became_peered":"2024-09-20T19:01:22.830233+0000","last_unstale":"2024-09-20T19:02:45.832421+0000","last_undegraded":"2024-09-20T19:02:45.832421+0000","last_fullsized":"2024-09-20T19:02:45.832421+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:03.509903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:03.509903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:11.556992+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"273'15","reported_seq":622,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-20T19:02:38.262992+0000","last_change":"2024-09-20T19:01:15.875779+0000","last_active":"2024-09-20T19:02:38.262992+0000","last_peered":"2024-09-20T19:02:38.262992+0000","last_clean":"2024-09-20T19:02:38.262992+0000","last_became_active":"2024-09-20T18:49:48.500803+0000","last_became_peered":"2024-09-20T18:49:48.500803+0000","last_unstale":"2024-09-20T19:02:38.262992+0000","last_undegraded":"2024-09-20T19:02:38.262992+0000","last_fullsized":"2024-09-20T19:02:38.262992+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:01:15.875693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:15.875693+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:02:42.612024+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":630,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.833877+0000","last_change":"2024-09-20T19:01:22.834162+0000","last_active":"2024-09-20T19:02:45.833877+0000","last_peered":"2024-09-20T19:02:45.833877+0000","last_clean":"2024-09-20T19:02:45.833877+0000","last_became_active":"2024-09-20T19:01:22.831615+0000","last_became_peered":"2024-09-20T19:01:22.831615+0000","last_unstale":"2024-09-20T19:02:45.833877+0000","last_undegraded":"2024-09-20T19:02:45.833877+0000","last_fullsized":"2024-09-20T19:02:45.833877+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:59.589891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:59.589891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:43.164641+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":600,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-20T19:02:38.271826+0000","last_change":"2024-09-20T19:01:06.482291+0000","last_active":"2024-09-20T19:02:38.271826+0000","last_peered":"2024-09-20T19:02:38.271826+0000","last_clean":"2024-09-20T19:02:38.271826+0000","last_became_active":"2024-09-20T18:49:49.961947+0000","last_became_peered":"2024-09-20T18:49:49.961947+0000","last_unstale":"2024-09-20T19:02:38.271826+0000","last_undegraded":"2024-09-20T19:02:38.271826+0000","last_fullsized":"2024-09-20T19:02:38.271826+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:06.482212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:06.482212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"66'1","reported_seq":623,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.837785+0000","last_change":"2024-09-20T19:02:01.843182+0000","last_active":"2024-09-20T19:02:45.837785+0000","last_peered":"2024-09-20T19:02:45.837785+0000","last_clean":"2024-09-20T19:02:45.837785+0000","last_became_active":"2024-09-20T19:01:22.837187+0000","last_became_peered":"2024-09-20T19:01:22.837187+0000","last_unstale":"2024-09-20T19:02:45.837785+0000","last_undegraded":"2024-09-20T19:02:45.837785+0000","last_fullsized":"2024-09-20T19:02:45.837785+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T19:02:01.843111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:01.843111+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:06.311104+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"274'15","reported_seq":625,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.835407+0000","last_change":"2024-09-20T19:01:36.532486+0000","last_active":"2024-09-20T19:02:45.835407+0000","last_peered":"2024-09-20T19:02:45.835407+0000","last_clean":"2024-09-20T19:02:45.835407+0000","last_became_active":"2024-09-20T18:49:48.501011+0000","last_became_peered":"2024-09-20T18:49:48.501011+0000","last_unstale":"2024-09-20T19:02:45.835407+0000","last_undegraded":"2024-09-20T19:02:45.835407+0000","last_fullsized":"2024-09-20T19:02:45.835407+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'15","last_scrub_stamp":"2024-09-20T19:01:36.532419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:36.532419+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:02:57.030996+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":609,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-20T19:02:38.266426+0000","last_change":"2024-09-20T19:01:30.864866+0000","last_active":"2024-09-20T19:02:38.266426+0000","last_peered":"2024-09-20T19:02:38.266426+0000","last_clean":"2024-09-20T19:02:38.266426+0000","last_became_active":"2024-09-20T18:49:45.938511+0000","last_became_peered":"2024-09-20T18:49:45.938511+0000","last_unstale":"2024-09-20T19:02:38.266426+0000","last_undegraded":"2024-09-20T19:02:38.266426+0000","last_fullsized":"2024-09-20T19:02:38.266426+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:30.864794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:30.864794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:02:51.939240+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":764,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-20T19:02:38.270387+0000","last_change":"2024-09-20T19:00:34.485855+0000","last_active":"2024-09-20T19:02:38.270387+0000","last_peered":"2024-09-20T19:02:38.270387+0000","last_clean":"2024-09-20T19:02:38.270387+0000","last_became_active":"2024-09-20T18:49:49.960550+0000","last_became_peered":"2024-09-20T18:49:49.960550+0000","last_unstale":"2024-09-20T19:02:38.270387+0000","last_undegraded":"2024-09-20T19:02:38.270387+0000","last_fullsized":"2024-09-20T19:02:38.270387+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T19:00:34.485624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:00:34.485624+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":554,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-09-20T19:02:42.865890+0000","last_change":"2024-09-20T19:01:22.848313+0000","last_active":"2024-09-20T19:02:42.865890+0000","last_peered":"2024-09-20T19:02:42.865890+0000","last_clean":"2024-09-20T19:02:42.865890+0000","last_became_active":"2024-09-20T19:01:22.848017+0000","last_became_peered":"2024-09-20T19:01:22.848017+0000","last_unstale":"2024-09-20T19:02:42.865890+0000","last_undegraded":"2024-09-20T19:02:42.865890+0000","last_fullsized":"2024-09-20T19:02:42.865890+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:59.497810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:59:59.497810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"274'20","reported_seq":637,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-20T19:02:38.270525+0000","last_change":"2024-09-20T18:57:20.701592+0000","last_active":"2024-09-20T19:02:38.270525+0000","last_peered":"2024-09-20T19:02:38.270525+0000","last_clean":"2024-09-20T19:02:38.270525+0000","last_became_active":"2024-09-20T18:49:58.997494+0000","last_became_peered":"2024-09-20T18:49:58.997494+0000","last_unstale":"2024-09-20T19:02:38.270525+0000","last_undegraded":"2024-09-20T19:02:38.270525+0000","last_fullsized":"2024-09-20T19:02:38.270525+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:57:20.701543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:20.701543+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:12.702800+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":616,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.832225+0000","last_change":"2024-09-20T19:01:22.833495+0000","last_active":"2024-09-20T19:02:45.832225+0000","last_peered":"2024-09-20T19:02:45.832225+0000","last_clean":"2024-09-20T19:02:45.832225+0000","last_became_active":"2024-09-20T19:01:22.831291+0000","last_became_peered":"2024-09-20T19:01:22.831291+0000","last_unstale":"2024-09-20T19:02:45.832225+0000","last_undegraded":"2024-09-20T19:02:45.832225+0000","last_fullsized":"2024-09-20T19:02:45.832225+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.632380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.632380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:07.354816+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":769,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.835271+0000","last_change":"2024-09-20T19:01:10.601374+0000","last_active":"2024-09-20T19:02:45.835271+0000","last_peered":"2024-09-20T19:02:45.835271+0000","last_clean":"2024-09-20T19:02:45.835271+0000","last_became_active":"2024-09-20T18:49:49.962243+0000","last_became_peered":"2024-09-20T18:49:49.962243+0000","last_unstale":"2024-09-20T19:02:45.835271+0000","last_undegraded":"2024-09-20T19:02:45.835271+0000","last_fullsized":"2024-09-20T19:02:45.835271+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T19:01:10.601300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:10.601300+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":605,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-20T19:02:38.261774+0000","last_change":"2024-09-20T19:01:03.821977+0000","last_active":"2024-09-20T19:02:38.261774+0000","last_peered":"2024-09-20T19:02:38.261774+0000","last_clean":"2024-09-20T19:02:38.261774+0000","last_became_active":"2024-09-20T18:49:51.969093+0000","last_became_peered":"2024-09-20T18:49:51.969093+0000","last_unstale":"2024-09-20T19:02:38.261774+0000","last_undegraded":"2024-09-20T19:02:38.261774+0000","last_fullsized":"2024-09-20T19:02:38.261774+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:03.821908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:03.821908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"273'19","reported_seq":618,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-09-20T19:02:42.865632+0000","last_change":"2024-09-20T19:01:22.841541+0000","last_active":"2024-09-20T19:02:42.865632+0000","last_peered":"2024-09-20T19:02:42.865632+0000","last_clean":"2024-09-20T19:02:42.865632+0000","last_became_active":"2024-09-20T19:01:22.841252+0000","last_became_peered":"2024-09-20T19:01:22.841252+0000","last_unstale":"2024-09-20T19:02:42.865632+0000","last_undegraded":"2024-09-20T19:02:42.865632+0000","last_fullsized":"2024-09-20T19:02:42.865632+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:57:50.899514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:50.899514+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:20.035402+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":536,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-09-20T19:02:42.865688+0000","last_change":"2024-09-20T19:01:22.838793+0000","last_active":"2024-09-20T19:02:42.865688+0000","last_peered":"2024-09-20T19:02:42.865688+0000","last_clean":"2024-09-20T19:02:42.865688+0000","last_became_active":"2024-09-20T19:01:22.838429+0000","last_became_peered":"2024-09-20T19:01:22.838429+0000","last_unstale":"2024-09-20T19:02:42.865688+0000","last_undegraded":"2024-09-20T19:02:42.865688+0000","last_fullsized":"2024-09-20T19:02:42.865688+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:06.310027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:59:06.310027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":604,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.835118+0000","last_change":"2024-09-20T19:02:38.906352+0000","last_active":"2024-09-20T19:02:45.835118+0000","last_peered":"2024-09-20T19:02:45.835118+0000","last_clean":"2024-09-20T19:02:45.835118+0000","last_became_active":"2024-09-20T18:49:49.957067+0000","last_became_peered":"2024-09-20T18:49:49.957067+0000","last_unstale":"2024-09-20T19:02:45.835118+0000","last_undegraded":"2024-09-20T19:02:45.835118+0000","last_fullsized":"2024-09-20T19:02:45.835118+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:38.906161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:38.906161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:41.340426+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":580,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-20T19:02:38.268354+0000","last_change":"2024-09-20T18:57:16.789412+0000","last_active":"2024-09-20T19:02:38.268354+0000","last_peered":"2024-09-20T19:02:38.268354+0000","last_clean":"2024-09-20T19:02:38.268354+0000","last_became_active":"2024-09-20T18:49:51.966915+0000","last_became_peered":"2024-09-20T18:49:51.966915+0000","last_unstale":"2024-09-20T19:02:38.268354+0000","last_undegraded":"2024-09-20T19:02:38.268354+0000","last_fullsized":"2024-09-20T19:02:38.268354+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:16.789095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:16.789095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:17.675828+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"273'15","reported_seq":675,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.835347+0000","last_change":"2024-09-20T19:02:30.816061+0000","last_active":"2024-09-20T19:02:45.835347+0000","last_peered":"2024-09-20T19:02:45.835347+0000","last_clean":"2024-09-20T19:02:45.835347+0000","last_became_active":"2024-09-20T18:59:05.085230+0000","last_became_peered":"2024-09-20T18:59:05.085230+0000","last_unstale":"2024-09-20T19:02:45.835347+0000","last_undegraded":"2024-09-20T19:02:45.835347+0000","last_fullsized":"2024-09-20T19:02:45.835347+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:02:30.815747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:30.815747+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:44.961926+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":608,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.832696+0000","last_change":"2024-09-20T19:01:22.830951+0000","last_active":"2024-09-20T19:02:45.832696+0000","last_peered":"2024-09-20T19:02:45.832696+0000","last_clean":"2024-09-20T19:02:45.832696+0000","last_became_active":"2024-09-20T19:01:22.830642+0000","last_became_peered":"2024-09-20T19:01:22.830642+0000","last_unstale":"2024-09-20T19:02:45.832696+0000","last_undegraded":"2024-09-20T19:02:45.832696+0000","last_fullsized":"2024-09-20T19:02:45.832696+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:42.487403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:42.487403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:06.397840+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":600,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-20T19:02:38.270077+0000","last_change":"2024-09-20T19:01:03.507072+0000","last_active":"2024-09-20T19:02:38.270077+0000","last_peered":"2024-09-20T19:02:38.270077+0000","last_clean":"2024-09-20T19:02:38.270077+0000","last_became_active":"2024-09-20T18:49:49.962523+0000","last_became_peered":"2024-09-20T18:49:49.962523+0000","last_unstale":"2024-09-20T19:02:38.270077+0000","last_undegraded":"2024-09-20T19:02:38.270077+0000","last_fullsized":"2024-09-20T19:02:38.270077+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:03.507016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:03.507016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":597,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-20T19:02:38.268225+0000","last_change":"2024-09-20T19:01:11.847792+0000","last_active":"2024-09-20T19:02:38.268225+0000","last_peered":"2024-09-20T19:02:38.268225+0000","last_clean":"2024-09-20T19:02:38.268225+0000","last_became_active":"2024-09-20T18:49:51.967313+0000","last_became_peered":"2024-09-20T18:49:51.967313+0000","last_unstale":"2024-09-20T19:02:38.268225+0000","last_undegraded":"2024-09-20T19:02:38.268225+0000","last_fullsized":"2024-09-20T19:02:38.268225+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:11.847716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:11.847716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:21.028973+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"274'19","reported_seq":673,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-20T19:02:38.268477+0000","last_change":"2024-09-20T19:01:05.495603+0000","last_active":"2024-09-20T19:02:38.268477+0000","last_peered":"2024-09-20T19:02:38.268477+0000","last_clean":"2024-09-20T19:02:38.268477+0000","last_became_active":"2024-09-20T18:59:05.075390+0000","last_became_peered":"2024-09-20T18:59:05.075390+0000","last_unstale":"2024-09-20T19:02:38.268477+0000","last_undegraded":"2024-09-20T19:02:38.268477+0000","last_fullsized":"2024-09-20T19:02:38.268477+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'19","last_scrub_stamp":"2024-09-20T19:01:05.495478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:05.495478+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":620,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.835905+0000","last_change":"2024-09-20T19:01:47.521858+0000","last_active":"2024-09-20T19:02:45.835905+0000","last_peered":"2024-09-20T19:02:45.835905+0000","last_clean":"2024-09-20T19:02:45.835905+0000","last_became_active":"2024-09-20T18:49:45.942105+0000","last_became_peered":"2024-09-20T18:49:45.942105+0000","last_unstale":"2024-09-20T19:02:45.835905+0000","last_undegraded":"2024-09-20T19:02:45.835905+0000","last_fullsized":"2024-09-20T19:02:45.835905+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:47.521789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:47.521789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:09.440449+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":630,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.831266+0000","last_change":"2024-09-20T19:02:27.267300+0000","last_active":"2024-09-20T19:02:45.831266+0000","last_peered":"2024-09-20T19:02:45.831266+0000","last_clean":"2024-09-20T19:02:45.831266+0000","last_became_active":"2024-09-20T19:01:22.827211+0000","last_became_peered":"2024-09-20T19:01:22.827211+0000","last_unstale":"2024-09-20T19:02:45.831266+0000","last_undegraded":"2024-09-20T19:02:45.831266+0000","last_fullsized":"2024-09-20T19:02:45.831266+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:27.267254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:27.267254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:51.446089+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":602,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-20T19:02:38.950217+0000","last_change":"2024-09-20T19:02:38.950217+0000","last_active":"2024-09-20T19:02:38.950217+0000","last_peered":"2024-09-20T19:02:38.950217+0000","last_clean":"2024-09-20T19:02:38.950217+0000","last_became_active":"2024-09-20T18:59:05.081179+0000","last_became_peered":"2024-09-20T18:59:05.081179+0000","last_unstale":"2024-09-20T19:02:38.950217+0000","last_undegraded":"2024-09-20T19:02:38.950217+0000","last_fullsized":"2024-09-20T19:02:38.950217+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:38.950017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:38.950017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:02.988187+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"273'8","reported_seq":642,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-20T19:02:38.267917+0000","last_change":"2024-09-20T19:00:33.504967+0000","last_active":"2024-09-20T19:02:38.267917+0000","last_peered":"2024-09-20T19:02:38.267917+0000","last_clean":"2024-09-20T19:02:38.267917+0000","last_became_active":"2024-09-20T18:59:05.076365+0000","last_became_peered":"2024-09-20T18:59:05.076365+0000","last_unstale":"2024-09-20T19:02:38.267917+0000","last_undegraded":"2024-09-20T19:02:38.267917+0000","last_fullsized":"2024-09-20T19:02:38.267917+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'8","last_scrub_stamp":"2024-09-20T19:00:33.504891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:00:33.504891+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":617,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-20T19:02:39.184249+0000","last_change":"2024-09-20T19:02:39.184249+0000","last_active":"2024-09-20T19:02:39.184249+0000","last_peered":"2024-09-20T19:02:39.184249+0000","last_clean":"2024-09-20T19:02:39.184249+0000","last_became_active":"2024-09-20T18:49:45.940988+0000","last_became_peered":"2024-09-20T18:49:45.940988+0000","last_unstale":"2024-09-20T19:02:39.184249+0000","last_undegraded":"2024-09-20T19:02:39.184249+0000","last_fullsized":"2024-09-20T19:02:39.184249+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:39.184150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:39.184150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:54.134855+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":194,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.831612+0000","last_change":"2024-09-20T19:01:41.497193+0000","last_active":"2024-09-20T19:02:45.831612+0000","last_peered":"2024-09-20T19:02:45.831612+0000","last_clean":"2024-09-20T19:02:45.831612+0000","last_became_active":"2024-09-20T19:01:22.845873+0000","last_became_peered":"2024-09-20T19:01:22.845873+0000","last_unstale":"2024-09-20T19:02:45.831612+0000","last_undegraded":"2024-09-20T19:02:45.831612+0000","last_fullsized":"2024-09-20T19:02:45.831612+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:41.497115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:41.497115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":604,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.831611+0000","last_change":"2024-09-20T19:02:29.277994+0000","last_active":"2024-09-20T19:02:45.831611+0000","last_peered":"2024-09-20T19:02:45.831611+0000","last_clean":"2024-09-20T19:02:45.831611+0000","last_became_active":"2024-09-20T19:01:22.829961+0000","last_became_peered":"2024-09-20T19:01:22.829961+0000","last_unstale":"2024-09-20T19:02:45.831611+0000","last_undegraded":"2024-09-20T19:02:45.831611+0000","last_fullsized":"2024-09-20T19:02:45.831611+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:29.277916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:29.277916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:32.717744+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":587,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.833587+0000","last_change":"2024-09-20T19:01:22.834863+0000","last_active":"2024-09-20T19:02:45.833587+0000","last_peered":"2024-09-20T19:02:45.833587+0000","last_clean":"2024-09-20T19:02:45.833587+0000","last_became_active":"2024-09-20T19:01:22.834453+0000","last_became_peered":"2024-09-20T19:01:22.834453+0000","last_unstale":"2024-09-20T19:02:45.833587+0000","last_undegraded":"2024-09-20T19:02:45.833587+0000","last_fullsized":"2024-09-20T19:02:45.833587+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:10.970779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:10.970779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:07.916601+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":600,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-20T19:02:38.268146+0000","last_change":"2024-09-20T18:58:36.077005+0000","last_active":"2024-09-20T19:02:38.268146+0000","last_peered":"2024-09-20T19:02:38.268146+0000","last_clean":"2024-09-20T19:02:38.268146+0000","last_became_active":"2024-09-20T18:49:45.937992+0000","last_became_peered":"2024-09-20T18:49:45.937992+0000","last_unstale":"2024-09-20T19:02:38.268146+0000","last_undegraded":"2024-09-20T19:02:38.268146+0000","last_fullsized":"2024-09-20T19:02:38.268146+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:36.076937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:36.076937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:21.762264+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"274'19","reported_seq":207,"reported_epoch":342,"state":"active+clean","last_fresh":"2024-09-20T19:02:45.832398+0000","last_change":"2024-09-20T19:01:15.544760+0000","last_active":"2024-09-20T19:02:45.832398+0000","last_peered":"2024-09-20T19:02:45.832398+0000","last_clean":"2024-09-20T19:02:45.832398+0000","last_became_active":"2024-09-20T18:59:05.089294+0000","last_became_peered":"2024-09-20T18:59:05.089294+0000","last_unstale":"2024-09-20T19:02:45.832398+0000","last_undegraded":"2024-09-20T19:02:45.832398+0000","last_fullsized":"2024-09-20T19:02:45.832398+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'19","last_scrub_stamp":"2024-09-20T19:01:15.544663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:15.544663+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:17.714021+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":594,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-09-20T19:02:38.262702+0000","last_change":"2024-09-20T19:01:40.961449+0000","last_active":"2024-09-20T19:02:38.262702+0000","last_peered":"2024-09-20T19:02:38.262702+0000","last_clean":"2024-09-20T19:02:38.262702+0000","last_became_active":"2024-09-20T18:49:49.957582+0000","last_became_peered":"2024-09-20T18:49:49.957582+0000","last_unstale":"2024-09-20T19:02:38.262702+0000","last_undegraded":"2024-09-20T19:02:38.262702+0000","last_fullsized":"2024-09-20T19:02:38.262702+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:40.961363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:40.961363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:02:49.088980+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":3700293632,"num_objects":56462,"num_object_clones":0,"num_object_copies":169386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":56462,"num_write_kb":3613568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":10869604352,"data_stored":10869604352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":56462,"ondisk_log_size":56462,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1185,"internal_metadata":0},"log_size":78,"ondisk_log_size":78,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1115,"num_read_kb":872,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":2244,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":683,"ondisk_log_size":683,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":774,"num_read_kb":769,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266848,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1301196,"kb_used_data":877116,"kb_used_omap":1,"kb_used_meta":424062,"kb_avail":92443956,"statfs":{"total":95995035648,"available":94662610944,"internally_reserved":0,"allocated":898166784,"data_stored":897640360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":434239946},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528487,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1931320,"kb_used_data":1597480,"kb_used_omap":1,"kb_used_meta":333822,"kb_avail":91813832,"statfs":{"total":95995035648,"available":94017363968,"internally_reserved":0,"allocated":1635819520,"data_stored":1635312914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":341834182},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,4,4,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757420,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1642168,"kb_used_data":1137576,"kb_used_omap":1,"kb_used_meta":504574,"kb_avail":92102984,"statfs":{"total":95995035648,"available":94313455616,"internally_reserved":0,"allocated":1164877824,"data_stored":1164371033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":516684229},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":304,"seq":1305670058002,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1928600,"kb_used_data":1789960,"kb_used_omap":5,"kb_used_meta":138618,"kb_avail":91816552,"statfs":{"total":95995035648,"available":94020149248,"internally_reserved":0,"allocated":1832919040,"data_stored":1832380733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5633,"internal_metadata":141945343},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,1,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182583,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1580744,"kb_used_data":1008120,"kb_used_omap":1,"kb_used_meta":572606,"kb_avail":92164408,"statfs":{"total":95995035648,"available":94376353792,"internally_reserved":0,"allocated":1032314880,"data_stored":1031784944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":586348999},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411517,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1865420,"kb_used_data":1364984,"kb_used_omap":1,"kb_used_meta":500414,"kb_avail":91879732,"statfs":{"total":95995035648,"available":94084845568,"internally_reserved":0,"allocated":1397743616,"data_stored":1397227877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":512424389},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575043,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1222972,"kb_used_data":708648,"kb_used_omap":1,"kb_used_meta":514302,"kb_avail":92522180,"statfs":{"total":95995035648,"available":94742712320,"internally_reserved":0,"allocated":725655552,"data_stored":725091845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":526645701},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738568,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2536768,"kb_used_data":2022448,"kb_used_omap":1,"kb_used_meta":514302,"kb_avail":91208384,"statfs":{"total":95995035648,"available":93397385216,"internally_reserved":0,"allocated":2070986752,"data_stored":2070465455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":526645701},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1185,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2087911424,"data_stored":2087911424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":726728704,"data_stored":726728704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1413939200,"data_stored":1413939200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1042153472,"data_stored":1042153472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1849032704,"data_stored":1849032704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1182138368,"data_stored":1182138368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1647968256,"data_stored":1647968256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":919732224,"data_stored":919732224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T19:02:48.765 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T19:02:48.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:48 smithi142 ceph-mon[32508]: pgmap v647: 148 pgs: 148 active+clean; 3.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 123 MiB/s wr, 1.97k op/s 2024-09-20T19:02:48.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:48 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:02:48.825 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:48.858 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:48.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:48.859 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:48.875 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 29 16 61457 61441 132.375 97.9375 0.00697521 0.00754907 2024-09-20T19:02:48.962 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:02:48.994 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:48.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:48.996 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:02:49.097 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:49.142 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:49.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:49.143 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:49.244 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:49.296 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:49.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:49.297 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:49.351 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:02:49.380 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:02:49.380 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:02:49.398 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:49.409 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:02:49.419 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:02:49.419 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:02:49.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:49.441 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:49.441 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:49.457 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:02:49.457 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:02:49.492 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:02:49.492 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:02:49.526 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:02:49.527 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:02:49.542 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:02:49.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:49 smithi083 ceph-mon[29421]: from='client.25817 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:02:49.544 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:49 smithi083 ceph-mon[36260]: from='client.25817 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:02:49.561 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:02:49.561 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:02:49.578 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:49.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:49.579 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:02:49.599 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:02:49.600 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:02:49.630 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:02:49.630 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:02:49.680 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:49.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:49.721 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:49.722 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:49.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:49 smithi142 ceph-mon[32508]: from='client.25817 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:02:49.822 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:49.843 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T19:02:49.855 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:49.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:49.856 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:49.875 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 30 16 62865 62849 130.896 88 0.00776536 0.00763458 2024-09-20T19:02:49.957 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:49.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:49.995 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:49.996 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:50.096 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:50.129 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:50.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:50.130 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:50.231 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:50.259 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:50.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:50.260 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:50.361 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:50.394 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:50.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:50.396 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:50.496 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:50.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:50.536 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:50.536 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:50.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:50 smithi083 ceph-mon[29421]: pgmap v648: 148 pgs: 148 active+clean; 3.5 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 96 MiB/s wr, 1.54k op/s 2024-09-20T19:02:50.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:50 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2231734201' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:02:50.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:50 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:02:50.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:50 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:02:50.544 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:50 smithi083 ceph-mon[36260]: pgmap v648: 148 pgs: 148 active+clean; 3.5 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 96 MiB/s wr, 1.54k op/s 2024-09-20T19:02:50.544 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:50 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2231734201' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:02:50.544 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:50 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:02:50.544 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:50 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:02:50.637 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:50.669 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:50.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:50.670 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:50.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:50 smithi142 ceph-mon[32508]: pgmap v648: 148 pgs: 148 active+clean; 3.5 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 96 MiB/s wr, 1.54k op/s 2024-09-20T19:02:50.764 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:50 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2231734201' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:02:50.764 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:50 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:02:50.764 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:50 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:02:50.771 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:50.807 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:50.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:50.808 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:50.874 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 31 16 64075 64059 129.113 75.625 0.00500229 0.00774047 2024-09-20T19:02:50.909 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:50.932 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:50.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:50.933 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:51.018 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:02:50 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:02:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:02:51.035 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:51.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:51.064 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:51.065 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:51.165 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:51.202 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:51.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:51.203 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:51.304 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:51.337 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:51.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:51.338 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:51.439 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:51.473 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:51.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:51.474 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:51.575 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:51.611 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:51.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:51.612 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:51.712 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:51.742 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:51.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:51.744 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:51.764 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:51 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:02:51.764 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:51 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:02:51.764 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:51 smithi142 ceph-mon[32508]: osdmap e343: 8 total, 8 up, 8 in 2024-09-20T19:02:51.764 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:51 smithi142 ceph-mon[32508]: osdmap e344: 8 total, 8 up, 8 in 2024-09-20T19:02:51.794 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:51 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:02:51.794 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:51 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:02:51.794 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:51 smithi083 ceph-mon[29421]: osdmap e343: 8 total, 8 up, 8 in 2024-09-20T19:02:51.794 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:51 smithi083 ceph-mon[29421]: osdmap e344: 8 total, 8 up, 8 in 2024-09-20T19:02:51.795 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:51 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:02:51.795 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:51 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:02:51.795 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:51 smithi083 ceph-mon[36260]: osdmap e343: 8 total, 8 up, 8 in 2024-09-20T19:02:51.795 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:51 smithi083 ceph-mon[36260]: osdmap e344: 8 total, 8 up, 8 in 2024-09-20T19:02:51.845 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:51.874 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 32 16 65360 65344 127.588 80.3125 0.00461563 0.00783322 2024-09-20T19:02:51.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:51.879 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:51.880 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:51.981 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:52.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:52.019 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:52.019 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:52.120 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:52.155 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:52.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:52.156 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:52.257 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:02:52.295 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:52.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:52.296 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:02:52.397 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:52.431 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:52.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:52.432 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:52.533 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:52.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:52.570 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:52.570 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:52.571 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T19:02:52.571 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:02:52.671 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:52.709 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:52.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:52.710 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:52.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:52 smithi142 ceph-mon[32508]: pgmap v651: 148 pgs: 148 active+clean; 3.7 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 852 B/s rd, 122 MiB/s wr, 1.95k op/s 2024-09-20T19:02:52.794 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:52 smithi083 ceph-mon[29421]: pgmap v651: 148 pgs: 148 active+clean; 3.7 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 852 B/s rd, 122 MiB/s wr, 1.95k op/s 2024-09-20T19:02:52.794 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:52 smithi083 ceph-mon[36260]: pgmap v651: 148 pgs: 148 active+clean; 3.7 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 852 B/s rd, 122 MiB/s wr, 1.95k op/s 2024-09-20T19:02:52.811 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:52.846 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:52.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:52.847 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:52.874 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 33 16 66925 66909 126.686 97.8125 0.00702793 0.0078888 2024-09-20T19:02:52.948 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:52.985 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:52.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:52.986 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:53.087 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:53.129 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:53.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:53.130 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:53.231 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:02:53.265 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:53.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:53.266 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:02:53.367 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:53.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:53.423 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:53.423 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:53.448 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:02:53.456 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_blocked_ops 2024-09-20T19:02:53.524 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:53.543 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T19:02:53.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:53 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:02:53.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:53 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:02:53.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:53 smithi083 ceph-mon[29421]: osdmap e345: 8 total, 8 up, 8 in 2024-09-20T19:02:53.548 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:53 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:02:53.548 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:53 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:02:53.548 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:53 smithi083 ceph-mon[36260]: osdmap e345: 8 total, 8 up, 8 in 2024-09-20T19:02:53.554 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-20T19:02:53.588 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:53.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:53.589 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:53.690 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:02:53.727 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:53.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:53.728 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:02:53.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:53 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:02:53.764 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:53 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:02:53.764 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:53 smithi142 ceph-mon[32508]: osdmap e345: 8 total, 8 up, 8 in 2024-09-20T19:02:53.829 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:53.875 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 34 16 68240 68224 125.373 82.1875 0.0173797 0.00797051 2024-09-20T19:02:53.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:53.879 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:53.879 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:53.980 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:54.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:54.015 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:54.015 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:54.116 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:54.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:54.163 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:54.163 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:54.264 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:54.320 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:54.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:54.321 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:54.422 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:54.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:54.465 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:54.465 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:54.485 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T19:02:54.566 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:54.608 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:54.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:54.609 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:54.643 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:54 smithi083 ceph-mon[29421]: pgmap v653: 148 pgs: 148 active+clean; 3.9 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 852 B/s rd, 84 MiB/s wr, 1.35k op/s 2024-09-20T19:02:54.643 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:54 smithi083 ceph-mon[29421]: from='client.25826 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:02:54.644 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:54 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T19:02:54.644 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:54 smithi083 ceph-mon[36260]: pgmap v653: 148 pgs: 148 active+clean; 3.9 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 852 B/s rd, 84 MiB/s wr, 1.35k op/s 2024-09-20T19:02:54.644 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:54 smithi083 ceph-mon[36260]: from='client.25826 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:02:54.644 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:54 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T19:02:54.710 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:54.748 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:54.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:54.750 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:54.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:54 smithi142 ceph-mon[32508]: pgmap v653: 148 pgs: 148 active+clean; 3.9 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 852 B/s rd, 84 MiB/s wr, 1.35k op/s 2024-09-20T19:02:54.764 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:54 smithi142 ceph-mon[32508]: from='client.25826 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:02:54.764 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:54 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T19:02:54.850 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:54.876 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 35 16 69500 69484 124.037 78.75 0.00977805 0.0080569 2024-09-20T19:02:54.892 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:54.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:54.893 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:54.994 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:55.033 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":653,"stamp":"2024-09-20T19:02:53.086119+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4230486555,"num_objects":64742,"num_object_clones":0,"num_object_copies":194226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64742,"num_whiteouts":0,"num_read":2037,"num_read_kb":1894,"num_write":65378,"num_write_kb":4132425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":1774258,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65399,"ondisk_log_size":65399,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":15963600,"kb_used_data":12363508,"kb_used_omap":16,"kb_used_meta":3599855,"kb_avail":733997616,"statfs":{"total":767960285184,"available":751613558784,"internally_reserved":0,"allocated":12660232192,"data_stored":12655926227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16790,"internal_metadata":3686252138},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":529596416,"num_objects":8081,"num_object_clones":0,"num_object_copies":24243,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8081,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":8081,"num_write_kb":517184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002536"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":618,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.833538+0000","last_change":"2024-09-20T19:01:31.817967+0000","last_active":"2024-09-20T19:02:50.833538+0000","last_peered":"2024-09-20T19:02:50.833538+0000","last_clean":"2024-09-20T19:02:50.833538+0000","last_became_active":"2024-09-20T18:49:45.943201+0000","last_became_peered":"2024-09-20T18:49:45.943201+0000","last_unstale":"2024-09-20T19:02:50.833538+0000","last_undegraded":"2024-09-20T19:02:50.833538+0000","last_fullsized":"2024-09-20T19:02:50.833538+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:31.817909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:31.817909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"273'15","reported_seq":629,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.328208+0000","last_change":"2024-09-20T19:01:22.840670+0000","last_active":"2024-09-20T19:02:50.328208+0000","last_peered":"2024-09-20T19:02:50.328208+0000","last_clean":"2024-09-20T19:02:50.328208+0000","last_became_active":"2024-09-20T19:01:22.838745+0000","last_became_peered":"2024-09-20T19:01:22.838745+0000","last_unstale":"2024-09-20T19:02:50.328208+0000","last_undegraded":"2024-09-20T19:02:50.328208+0000","last_fullsized":"2024-09-20T19:02:50.328208+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:50.231620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:50.231620+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:09.095233+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":608,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.841214+0000","last_change":"2024-09-20T18:57:49.016078+0000","last_active":"2024-09-20T19:02:50.841214+0000","last_peered":"2024-09-20T19:02:50.841214+0000","last_clean":"2024-09-20T19:02:50.841214+0000","last_became_active":"2024-09-20T18:49:49.959514+0000","last_became_peered":"2024-09-20T18:49:49.959514+0000","last_unstale":"2024-09-20T19:02:50.841214+0000","last_undegraded":"2024-09-20T19:02:50.841214+0000","last_fullsized":"2024-09-20T19:02:50.841214+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:49.016006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:49.016006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:10.916493+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":555,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.329102+0000","last_change":"2024-09-20T19:01:22.841355+0000","last_active":"2024-09-20T19:02:50.329102+0000","last_peered":"2024-09-20T19:02:50.329102+0000","last_clean":"2024-09-20T19:02:50.329102+0000","last_became_active":"2024-09-20T19:01:22.838827+0000","last_became_peered":"2024-09-20T19:01:22.838827+0000","last_unstale":"2024-09-20T19:02:50.329102+0000","last_undegraded":"2024-09-20T19:02:50.329102+0000","last_fullsized":"2024-09-20T19:02:50.329102+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:02.084375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:02.084375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:20.990491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":618,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.838185+0000","last_change":"2024-09-20T19:02:02.441633+0000","last_active":"2024-09-20T19:02:50.838185+0000","last_peered":"2024-09-20T19:02:50.838185+0000","last_clean":"2024-09-20T19:02:50.838185+0000","last_became_active":"2024-09-20T18:59:05.081081+0000","last_became_peered":"2024-09-20T18:59:05.081081+0000","last_unstale":"2024-09-20T19:02:50.838185+0000","last_undegraded":"2024-09-20T19:02:50.838185+0000","last_fullsized":"2024-09-20T19:02:50.838185+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:02.441577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:02.441577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:28.158812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":623,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.838926+0000","last_change":"2024-09-20T18:59:05.073093+0000","last_active":"2024-09-20T19:02:50.838926+0000","last_peered":"2024-09-20T19:02:50.838926+0000","last_clean":"2024-09-20T19:02:50.838926+0000","last_became_active":"2024-09-20T18:59:05.072675+0000","last_became_peered":"2024-09-20T18:59:05.072675+0000","last_unstale":"2024-09-20T19:02:50.838926+0000","last_undegraded":"2024-09-20T19:02:50.838926+0000","last_fullsized":"2024-09-20T19:02:50.838926+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:03.055929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:59:03.055929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:45.957147+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"274'27","reported_seq":655,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.840686+0000","last_change":"2024-09-20T19:02:40.899215+0000","last_active":"2024-09-20T19:02:50.840686+0000","last_peered":"2024-09-20T19:02:50.840686+0000","last_clean":"2024-09-20T19:02:50.840686+0000","last_became_active":"2024-09-20T18:49:47.952150+0000","last_became_peered":"2024-09-20T18:49:47.952150+0000","last_unstale":"2024-09-20T19:02:50.840686+0000","last_undegraded":"2024-09-20T19:02:50.840686+0000","last_fullsized":"2024-09-20T19:02:50.840686+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'27","last_scrub_stamp":"2024-09-20T19:02:40.899139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:40.899139+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:02.856822+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":546,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.328126+0000","last_change":"2024-09-20T19:02:23.982879+0000","last_active":"2024-09-20T19:02:50.328126+0000","last_peered":"2024-09-20T19:02:50.328126+0000","last_clean":"2024-09-20T19:02:50.328126+0000","last_became_active":"2024-09-20T19:01:22.839075+0000","last_became_peered":"2024-09-20T19:01:22.839075+0000","last_unstale":"2024-09-20T19:02:50.328126+0000","last_undegraded":"2024-09-20T19:02:50.328126+0000","last_fullsized":"2024-09-20T19:02:50.328126+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:23.982813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:23.982813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:51.507141+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"59'1","reported_seq":671,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.839911+0000","last_change":"2024-09-20T19:01:28.592723+0000","last_active":"2024-09-20T19:02:50.839911+0000","last_peered":"2024-09-20T19:02:50.839911+0000","last_clean":"2024-09-20T19:02:50.839911+0000","last_became_active":"2024-09-20T19:01:22.847016+0000","last_became_peered":"2024-09-20T19:01:22.847016+0000","last_unstale":"2024-09-20T19:02:50.839911+0000","last_undegraded":"2024-09-20T19:02:50.839911+0000","last_fullsized":"2024-09-20T19:02:50.839911+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:01:28.592653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:28.592653+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:43.784234+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"273'15","reported_seq":594,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.328944+0000","last_change":"2024-09-20T19:02:42.114419+0000","last_active":"2024-09-20T19:02:50.328944+0000","last_peered":"2024-09-20T19:02:50.328944+0000","last_clean":"2024-09-20T19:02:50.328944+0000","last_became_active":"2024-09-20T19:01:22.844899+0000","last_became_peered":"2024-09-20T19:01:22.844899+0000","last_unstale":"2024-09-20T19:02:50.328944+0000","last_undegraded":"2024-09-20T19:02:50.328944+0000","last_fullsized":"2024-09-20T19:02:50.328944+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:02:42.114331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:42.114331+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:44.431155+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":616,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.841375+0000","last_change":"2024-09-20T19:01:51.564693+0000","last_active":"2024-09-20T19:02:50.841375+0000","last_peered":"2024-09-20T19:02:50.841375+0000","last_clean":"2024-09-20T19:02:50.841375+0000","last_became_active":"2024-09-20T18:49:49.961188+0000","last_became_peered":"2024-09-20T18:49:49.961188+0000","last_unstale":"2024-09-20T19:02:50.841375+0000","last_undegraded":"2024-09-20T19:02:50.841375+0000","last_fullsized":"2024-09-20T19:02:50.841375+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:51.564624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:51.564624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:01.653667+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":589,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.834185+0000","last_change":"2024-09-20T19:02:03.029635+0000","last_active":"2024-09-20T19:02:50.834185+0000","last_peered":"2024-09-20T19:02:50.834185+0000","last_clean":"2024-09-20T19:02:50.834185+0000","last_became_active":"2024-09-20T18:49:51.966715+0000","last_became_peered":"2024-09-20T18:49:51.966715+0000","last_unstale":"2024-09-20T19:02:50.834185+0000","last_undegraded":"2024-09-20T19:02:50.834185+0000","last_fullsized":"2024-09-20T19:02:50.834185+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:03.029585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:03.029585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:07.806574+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":627,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.839323+0000","last_change":"2024-09-20T19:01:48.520088+0000","last_active":"2024-09-20T19:02:50.839323+0000","last_peered":"2024-09-20T19:02:50.839323+0000","last_clean":"2024-09-20T19:02:50.839323+0000","last_became_active":"2024-09-20T19:01:22.846527+0000","last_became_peered":"2024-09-20T19:01:22.846527+0000","last_unstale":"2024-09-20T19:02:50.839323+0000","last_undegraded":"2024-09-20T19:02:50.839323+0000","last_fullsized":"2024-09-20T19:02:50.839323+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:48.520017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:48.520017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:49.827940+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"273'7","reported_seq":584,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.328163+0000","last_change":"2024-09-20T19:01:22.844317+0000","last_active":"2024-09-20T19:02:50.328163+0000","last_peered":"2024-09-20T19:02:50.328163+0000","last_clean":"2024-09-20T19:02:50.328163+0000","last_became_active":"2024-09-20T19:01:22.844013+0000","last_became_peered":"2024-09-20T19:01:22.844013+0000","last_unstale":"2024-09-20T19:02:50.328163+0000","last_undegraded":"2024-09-20T19:02:50.328163+0000","last_fullsized":"2024-09-20T19:02:50.328163+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-20T18:58:33.660531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:33.660531+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":620,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.840118+0000","last_change":"2024-09-20T19:02:33.203392+0000","last_active":"2024-09-20T19:02:50.840118+0000","last_peered":"2024-09-20T19:02:50.840118+0000","last_clean":"2024-09-20T19:02:50.840118+0000","last_became_active":"2024-09-20T19:01:22.830605+0000","last_became_peered":"2024-09-20T19:01:22.830605+0000","last_unstale":"2024-09-20T19:02:50.840118+0000","last_undegraded":"2024-09-20T19:02:50.840118+0000","last_fullsized":"2024-09-20T19:02:50.840118+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:33.203312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:33.203312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:35.713757+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":604,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.838976+0000","last_change":"2024-09-20T18:58:26.465305+0000","last_active":"2024-09-20T19:02:50.838976+0000","last_peered":"2024-09-20T19:02:50.838976+0000","last_clean":"2024-09-20T19:02:50.838976+0000","last_became_active":"2024-09-20T18:49:51.970663+0000","last_became_peered":"2024-09-20T18:49:51.970663+0000","last_unstale":"2024-09-20T19:02:50.838976+0000","last_undegraded":"2024-09-20T19:02:50.838976+0000","last_fullsized":"2024-09-20T19:02:50.838976+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:26.465264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:26.465264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:18.787658+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":535,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.330513+0000","last_change":"2024-09-20T19:01:22.841173+0000","last_active":"2024-09-20T19:02:50.330513+0000","last_peered":"2024-09-20T19:02:50.330513+0000","last_clean":"2024-09-20T19:02:50.330513+0000","last_became_active":"2024-09-20T19:01:22.838149+0000","last_became_peered":"2024-09-20T19:01:22.838149+0000","last_unstale":"2024-09-20T19:02:50.330513+0000","last_undegraded":"2024-09-20T19:02:50.330513+0000","last_fullsized":"2024-09-20T19:02:50.330513+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:00.906652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:00.906652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:19.046861+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":628,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.837526+0000","last_change":"2024-09-20T19:01:44.998192+0000","last_active":"2024-09-20T19:02:50.837526+0000","last_peered":"2024-09-20T19:02:50.837526+0000","last_clean":"2024-09-20T19:02:50.837526+0000","last_became_active":"2024-09-20T19:01:22.845773+0000","last_became_peered":"2024-09-20T19:01:22.845773+0000","last_unstale":"2024-09-20T19:02:50.837526+0000","last_undegraded":"2024-09-20T19:02:50.837526+0000","last_fullsized":"2024-09-20T19:02:50.837526+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:44.998147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:44.998147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"273'27","reported_seq":658,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.838211+0000","last_change":"2024-09-20T19:01:59.787246+0000","last_active":"2024-09-20T19:02:50.838211+0000","last_peered":"2024-09-20T19:02:50.838211+0000","last_clean":"2024-09-20T19:02:50.838211+0000","last_became_active":"2024-09-20T18:49:47.950939+0000","last_became_peered":"2024-09-20T18:49:47.950939+0000","last_unstale":"2024-09-20T19:02:50.838211+0000","last_undegraded":"2024-09-20T19:02:50.838211+0000","last_fullsized":"2024-09-20T19:02:50.838211+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'27","last_scrub_stamp":"2024-09-20T19:01:59.787179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:59.787179+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:02.865513+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":611,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.840419+0000","last_change":"2024-09-20T19:02:05.889054+0000","last_active":"2024-09-20T19:02:50.840419+0000","last_peered":"2024-09-20T19:02:50.840419+0000","last_clean":"2024-09-20T19:02:50.840419+0000","last_became_active":"2024-09-20T19:01:22.834771+0000","last_became_peered":"2024-09-20T19:01:22.834771+0000","last_unstale":"2024-09-20T19:02:50.840419+0000","last_undegraded":"2024-09-20T19:02:50.840419+0000","last_fullsized":"2024-09-20T19:02:50.840419+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:05.888974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:05.888974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:34.411336+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":619,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.840136+0000","last_change":"2024-09-20T19:02:28.934597+0000","last_active":"2024-09-20T19:02:50.840136+0000","last_peered":"2024-09-20T19:02:50.840136+0000","last_clean":"2024-09-20T19:02:50.840136+0000","last_became_active":"2024-09-20T18:59:05.084496+0000","last_became_peered":"2024-09-20T18:59:05.084496+0000","last_unstale":"2024-09-20T19:02:50.840136+0000","last_undegraded":"2024-09-20T19:02:50.840136+0000","last_fullsized":"2024-09-20T19:02:50.840136+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:28.934264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:28.934264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:43.415620+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"59'1","reported_seq":541,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.329202+0000","last_change":"2024-09-20T19:01:22.840962+0000","last_active":"2024-09-20T19:02:50.329202+0000","last_peered":"2024-09-20T19:02:50.329202+0000","last_clean":"2024-09-20T19:02:50.329202+0000","last_became_active":"2024-09-20T19:01:22.840591+0000","last_became_peered":"2024-09-20T19:01:22.840591+0000","last_unstale":"2024-09-20T19:02:50.329202+0000","last_undegraded":"2024-09-20T19:02:50.329202+0000","last_fullsized":"2024-09-20T19:02:50.329202+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:59:03.397225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:59:03.397225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"274'18","reported_seq":631,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.838109+0000","last_change":"2024-09-20T19:01:22.837011+0000","last_active":"2024-09-20T19:02:50.838109+0000","last_peered":"2024-09-20T19:02:50.838109+0000","last_clean":"2024-09-20T19:02:50.838109+0000","last_became_active":"2024-09-20T19:01:22.836111+0000","last_became_peered":"2024-09-20T19:01:22.836111+0000","last_unstale":"2024-09-20T19:02:50.838109+0000","last_undegraded":"2024-09-20T19:02:50.838109+0000","last_fullsized":"2024-09-20T19:02:50.838109+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:57:09.818999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:09.818999+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:07.600886+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":590,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.332066+0000","last_change":"2024-09-20T19:02:02.094517+0000","last_active":"2024-09-20T19:02:50.332066+0000","last_peered":"2024-09-20T19:02:50.332066+0000","last_clean":"2024-09-20T19:02:50.332066+0000","last_became_active":"2024-09-20T19:01:22.848532+0000","last_became_peered":"2024-09-20T19:01:22.848532+0000","last_unstale":"2024-09-20T19:02:50.332066+0000","last_undegraded":"2024-09-20T19:02:50.332066+0000","last_fullsized":"2024-09-20T19:02:50.332066+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:02.094473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:02.094473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:17.459163+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":606,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.841599+0000","last_change":"2024-09-20T19:02:28.294390+0000","last_active":"2024-09-20T19:02:50.841599+0000","last_peered":"2024-09-20T19:02:50.841599+0000","last_clean":"2024-09-20T19:02:50.841599+0000","last_became_active":"2024-09-20T18:49:51.970058+0000","last_became_peered":"2024-09-20T18:49:51.970058+0000","last_unstale":"2024-09-20T19:02:50.841599+0000","last_undegraded":"2024-09-20T19:02:50.841599+0000","last_fullsized":"2024-09-20T19:02:50.841599+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T19:02:28.294292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:28.294292+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:41.507259+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":612,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.840383+0000","last_change":"2024-09-20T19:01:22.837474+0000","last_active":"2024-09-20T19:02:50.840383+0000","last_peered":"2024-09-20T19:02:50.840383+0000","last_clean":"2024-09-20T19:02:50.840383+0000","last_became_active":"2024-09-20T19:01:22.837139+0000","last_became_peered":"2024-09-20T19:01:22.837139+0000","last_unstale":"2024-09-20T19:02:50.840383+0000","last_undegraded":"2024-09-20T19:02:50.840383+0000","last_fullsized":"2024-09-20T19:02:50.840383+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:57:18.032067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:18.032067+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:09.880584+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"274'20","reported_seq":660,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.834962+0000","last_change":"2024-09-20T19:02:35.097053+0000","last_active":"2024-09-20T19:02:50.834962+0000","last_peered":"2024-09-20T19:02:50.834962+0000","last_clean":"2024-09-20T19:02:50.834962+0000","last_became_active":"2024-09-20T18:49:48.500809+0000","last_became_peered":"2024-09-20T18:49:48.500809+0000","last_unstale":"2024-09-20T19:02:50.834962+0000","last_undegraded":"2024-09-20T19:02:50.834962+0000","last_fullsized":"2024-09-20T19:02:50.834962+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'20","last_scrub_stamp":"2024-09-20T19:02:35.096979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:35.096979+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:51.795986+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"308'12","reported_seq":257,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.837868+0000","last_change":"2024-09-20T19:01:44.541242+0000","last_active":"2024-09-20T19:02:50.837868+0000","last_peered":"2024-09-20T19:02:50.837868+0000","last_clean":"2024-09-20T19:02:50.837868+0000","last_became_active":"2024-09-20T19:01:22.845596+0000","last_became_peered":"2024-09-20T19:01:22.845596+0000","last_unstale":"2024-09-20T19:02:50.837868+0000","last_undegraded":"2024-09-20T19:02:50.837868+0000","last_fullsized":"2024-09-20T19:02:50.837868+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"308'12","last_scrub_stamp":"2024-09-20T19:01:44.541156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:44.541156+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:11.696073+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"343'3851","reported_seq":3898,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.464192+0000","last_change":"2024-09-20T19:02:11.829261+0000","last_active":"2024-09-20T19:02:50.464192+0000","last_peered":"2024-09-20T19:02:50.464192+0000","last_clean":"2024-09-20T19:02:50.464192+0000","last_became_active":"2024-09-20T19:02:11.828743+0000","last_became_peered":"2024-09-20T19:02:11.828743+0000","last_unstale":"2024-09-20T19:02:50.464192+0000","last_undegraded":"2024-09-20T19:02:50.464192+0000","last_fullsized":"2024-09-20T19:02:50.464192+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":3851,"log_dups_size":0,"ondisk_log_size":3851,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:26.017450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":252379136,"num_objects":3851,"num_object_clones":0,"num_object_copies":11553,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3851,"num_write_kb":246464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"274'27","reported_seq":652,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.835960+0000","last_change":"2024-09-20T19:02:27.897213+0000","last_active":"2024-09-20T19:02:50.835960+0000","last_peered":"2024-09-20T19:02:50.835960+0000","last_clean":"2024-09-20T19:02:50.835960+0000","last_became_active":"2024-09-20T19:01:22.833840+0000","last_became_peered":"2024-09-20T19:01:22.833840+0000","last_unstale":"2024-09-20T19:02:50.835960+0000","last_undegraded":"2024-09-20T19:02:50.835960+0000","last_fullsized":"2024-09-20T19:02:50.835960+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'27","last_scrub_stamp":"2024-09-20T19:02:27.897138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:27.897138+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:40.557830+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":609,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.839056+0000","last_change":"2024-09-20T19:00:31.512527+0000","last_active":"2024-09-20T19:02:50.839056+0000","last_peered":"2024-09-20T19:02:50.839056+0000","last_clean":"2024-09-20T19:02:50.839056+0000","last_became_active":"2024-09-20T18:49:45.936104+0000","last_became_peered":"2024-09-20T18:49:45.936104+0000","last_unstale":"2024-09-20T19:02:50.839056+0000","last_undegraded":"2024-09-20T19:02:50.839056+0000","last_fullsized":"2024-09-20T19:02:50.839056+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:00:31.512472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:00:31.512472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:48.924236+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"308'10","reported_seq":712,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.330601+0000","last_change":"2024-09-20T19:02:23.017699+0000","last_active":"2024-09-20T19:02:50.330601+0000","last_peered":"2024-09-20T19:02:50.330601+0000","last_clean":"2024-09-20T19:02:50.330601+0000","last_became_active":"2024-09-20T19:01:22.845603+0000","last_became_peered":"2024-09-20T19:01:22.845603+0000","last_unstale":"2024-09-20T19:02:50.330601+0000","last_undegraded":"2024-09-20T19:02:50.330601+0000","last_fullsized":"2024-09-20T19:02:50.330601+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"308'10","last_scrub_stamp":"2024-09-20T19:02:23.017626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:23.017626+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:51.273397+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":604,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.839494+0000","last_change":"2024-09-20T19:02:22.736879+0000","last_active":"2024-09-20T19:02:50.839494+0000","last_peered":"2024-09-20T19:02:50.839494+0000","last_clean":"2024-09-20T19:02:50.839494+0000","last_became_active":"2024-09-20T18:49:51.971399+0000","last_became_peered":"2024-09-20T18:49:51.971399+0000","last_unstale":"2024-09-20T19:02:50.839494+0000","last_undegraded":"2024-09-20T19:02:50.839494+0000","last_fullsized":"2024-09-20T19:02:50.839494+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:22.736825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:22.736825+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:51.591260+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.e","version":"343'4068","reported_seq":4115,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.472795+0000","last_change":"2024-09-20T19:02:11.824951+0000","last_active":"2024-09-20T19:02:50.472795+0000","last_peered":"2024-09-20T19:02:50.472795+0000","last_clean":"2024-09-20T19:02:50.472795+0000","last_became_active":"2024-09-20T19:02:11.824670+0000","last_became_peered":"2024-09-20T19:02:11.824670+0000","last_unstale":"2024-09-20T19:02:50.472795+0000","last_undegraded":"2024-09-20T19:02:50.472795+0000","last_fullsized":"2024-09-20T19:02:50.472795+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4068,"log_dups_size":0,"ondisk_log_size":4068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:16.256458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":266600448,"num_objects":4068,"num_object_clones":0,"num_object_copies":12204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4068,"num_write_kb":260352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"273'30","reported_seq":663,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.838823+0000","last_change":"2024-09-20T19:01:10.381171+0000","last_active":"2024-09-20T19:02:50.838823+0000","last_peered":"2024-09-20T19:02:50.838823+0000","last_clean":"2024-09-20T19:02:50.838823+0000","last_became_active":"2024-09-20T18:49:48.499189+0000","last_became_peered":"2024-09-20T18:49:48.499189+0000","last_unstale":"2024-09-20T19:02:50.838823+0000","last_undegraded":"2024-09-20T19:02:50.838823+0000","last_fullsized":"2024-09-20T19:02:50.838823+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'30","last_scrub_stamp":"2024-09-20T19:01:10.381094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:10.381094+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":609,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.840535+0000","last_change":"2024-09-20T19:02:22.806587+0000","last_active":"2024-09-20T19:02:50.840535+0000","last_peered":"2024-09-20T19:02:50.840535+0000","last_clean":"2024-09-20T19:02:50.840535+0000","last_became_active":"2024-09-20T18:49:45.940544+0000","last_became_peered":"2024-09-20T18:49:45.940544+0000","last_unstale":"2024-09-20T19:02:50.840535+0000","last_undegraded":"2024-09-20T19:02:50.840535+0000","last_fullsized":"2024-09-20T19:02:50.840535+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:22.806509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:22.806509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:39.275044+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":585,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.328579+0000","last_change":"2024-09-20T19:01:22.835462+0000","last_active":"2024-09-20T19:02:50.328579+0000","last_peered":"2024-09-20T19:02:50.328579+0000","last_clean":"2024-09-20T19:02:50.328579+0000","last_became_active":"2024-09-20T19:01:22.833592+0000","last_became_peered":"2024-09-20T19:01:22.833592+0000","last_unstale":"2024-09-20T19:02:50.328579+0000","last_undegraded":"2024-09-20T19:02:50.328579+0000","last_fullsized":"2024-09-20T19:02:50.328579+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.229201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.229201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:18.032847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":576,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.328998+0000","last_change":"2024-09-20T19:01:22.830850+0000","last_active":"2024-09-20T19:02:50.328998+0000","last_peered":"2024-09-20T19:02:50.328998+0000","last_clean":"2024-09-20T19:02:50.328998+0000","last_became_active":"2024-09-20T19:01:22.830369+0000","last_became_peered":"2024-09-20T19:01:22.830369+0000","last_unstale":"2024-09-20T19:02:50.328998+0000","last_undegraded":"2024-09-20T19:02:50.328998+0000","last_fullsized":"2024-09-20T19:02:50.328998+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:27.704520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:27.704520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"344'3993","reported_seq":4042,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.993682+0000","last_change":"2024-09-20T19:02:11.828279+0000","last_active":"2024-09-20T19:02:50.993682+0000","last_peered":"2024-09-20T19:02:50.993682+0000","last_clean":"2024-09-20T19:02:50.993682+0000","last_became_active":"2024-09-20T19:02:11.827468+0000","last_became_peered":"2024-09-20T19:02:11.827468+0000","last_unstale":"2024-09-20T19:02:50.993682+0000","last_undegraded":"2024-09-20T19:02:50.993682+0000","last_fullsized":"2024-09-20T19:02:50.993682+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":3993,"log_dups_size":0,"ondisk_log_size":3993,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:26.126856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":261685248,"num_objects":3993,"num_object_clones":0,"num_object_copies":11979,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3993,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3993,"num_write_kb":255552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"274'28","reported_seq":651,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.328497+0000","last_change":"2024-09-20T19:01:22.848177+0000","last_active":"2024-09-20T19:02:50.328497+0000","last_peered":"2024-09-20T19:02:50.328497+0000","last_clean":"2024-09-20T19:02:50.328497+0000","last_became_active":"2024-09-20T19:01:22.847843+0000","last_became_peered":"2024-09-20T19:01:22.847843+0000","last_unstale":"2024-09-20T19:02:50.328497+0000","last_undegraded":"2024-09-20T19:02:50.328497+0000","last_fullsized":"2024-09-20T19:02:50.328497+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-20T18:54:57.394893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.394893+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:06.093887+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":41,"num_read_kb":34,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":622,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.835301+0000","last_change":"2024-09-20T19:01:22.852962+0000","last_active":"2024-09-20T19:02:50.835301+0000","last_peered":"2024-09-20T19:02:50.835301+0000","last_clean":"2024-09-20T19:02:50.835301+0000","last_became_active":"2024-09-20T19:01:22.852594+0000","last_became_peered":"2024-09-20T19:01:22.852594+0000","last_unstale":"2024-09-20T19:02:50.835301+0000","last_undegraded":"2024-09-20T19:02:50.835301+0000","last_fullsized":"2024-09-20T19:02:50.835301+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:00:29.890681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:00:29.890681+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:19.042344+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"308'10","reported_seq":788,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.838675+0000","last_change":"2024-09-20T19:01:22.834166+0000","last_active":"2024-09-20T19:02:50.838675+0000","last_peered":"2024-09-20T19:02:50.838675+0000","last_clean":"2024-09-20T19:02:50.838675+0000","last_became_active":"2024-09-20T19:01:22.833774+0000","last_became_peered":"2024-09-20T19:01:22.833774+0000","last_unstale":"2024-09-20T19:02:50.838675+0000","last_undegraded":"2024-09-20T19:02:50.838675+0000","last_fullsized":"2024-09-20T19:02:50.838675+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:58:23.114371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:23.114371+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:18.678273+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":612,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.840523+0000","last_change":"2024-09-20T18:59:05.082117+0000","last_active":"2024-09-20T19:02:50.840523+0000","last_peered":"2024-09-20T19:02:50.840523+0000","last_clean":"2024-09-20T19:02:50.840523+0000","last_became_active":"2024-09-20T18:59:05.081814+0000","last_became_peered":"2024-09-20T18:59:05.081814+0000","last_unstale":"2024-09-20T19:02:50.840523+0000","last_undegraded":"2024-09-20T19:02:50.840523+0000","last_fullsized":"2024-09-20T19:02:50.840523+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:29.530001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:29.530001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:11.272988+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"344'3880","reported_seq":3929,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:51.080025+0000","last_change":"2024-09-20T19:02:11.830056+0000","last_active":"2024-09-20T19:02:51.080025+0000","last_peered":"2024-09-20T19:02:51.080025+0000","last_clean":"2024-09-20T19:02:51.080025+0000","last_became_active":"2024-09-20T19:02:11.829446+0000","last_became_peered":"2024-09-20T19:02:11.829446+0000","last_unstale":"2024-09-20T19:02:51.080025+0000","last_undegraded":"2024-09-20T19:02:51.080025+0000","last_fullsized":"2024-09-20T19:02:51.080025+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":3880,"log_dups_size":0,"ondisk_log_size":3880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:28.206813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":254279680,"num_objects":3880,"num_object_clones":0,"num_object_copies":11640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3880,"num_write_kb":248320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"274'20","reported_seq":670,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.840544+0000","last_change":"2024-09-20T19:02:35.931157+0000","last_active":"2024-09-20T19:02:50.840544+0000","last_peered":"2024-09-20T19:02:50.840544+0000","last_clean":"2024-09-20T19:02:50.840544+0000","last_became_active":"2024-09-20T19:01:22.834995+0000","last_became_peered":"2024-09-20T19:01:22.834995+0000","last_unstale":"2024-09-20T19:02:50.840544+0000","last_undegraded":"2024-09-20T19:02:50.840544+0000","last_fullsized":"2024-09-20T19:02:50.840544+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'20","last_scrub_stamp":"2024-09-20T19:02:35.930910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:35.930910+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:40.603359+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":633,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.840001+0000","last_change":"2024-09-20T19:02:30.686939+0000","last_active":"2024-09-20T19:02:50.840001+0000","last_peered":"2024-09-20T19:02:50.840001+0000","last_clean":"2024-09-20T19:02:50.840001+0000","last_became_active":"2024-09-20T18:49:45.942496+0000","last_became_peered":"2024-09-20T18:49:45.942496+0000","last_unstale":"2024-09-20T19:02:50.840001+0000","last_undegraded":"2024-09-20T19:02:50.840001+0000","last_fullsized":"2024-09-20T19:02:50.840001+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:30.686872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:30.686872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:51.214827+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"262'10","reported_seq":798,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.837224+0000","last_change":"2024-09-20T19:02:04.994109+0000","last_active":"2024-09-20T19:02:50.837224+0000","last_peered":"2024-09-20T19:02:50.837224+0000","last_clean":"2024-09-20T19:02:50.837224+0000","last_became_active":"2024-09-20T18:59:05.080183+0000","last_became_peered":"2024-09-20T18:59:05.080183+0000","last_unstale":"2024-09-20T19:02:50.837224+0000","last_undegraded":"2024-09-20T19:02:50.837224+0000","last_fullsized":"2024-09-20T19:02:50.837224+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'10","last_scrub_stamp":"2024-09-20T19:02:04.994047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:04.994047+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:34.082210+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":590,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.834301+0000","last_change":"2024-09-20T19:01:37.900410+0000","last_active":"2024-09-20T19:02:50.834301+0000","last_peered":"2024-09-20T19:02:50.834301+0000","last_clean":"2024-09-20T19:02:50.834301+0000","last_became_active":"2024-09-20T18:49:51.964780+0000","last_became_peered":"2024-09-20T18:49:51.964780+0000","last_unstale":"2024-09-20T19:02:50.834301+0000","last_undegraded":"2024-09-20T19:02:50.834301+0000","last_fullsized":"2024-09-20T19:02:50.834301+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:37.900328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:37.900328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:02:56.513120+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.d","version":"344'4181","reported_seq":4228,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.993704+0000","last_change":"2024-09-20T19:02:11.829599+0000","last_active":"2024-09-20T19:02:50.993704+0000","last_peered":"2024-09-20T19:02:50.993704+0000","last_clean":"2024-09-20T19:02:50.993704+0000","last_became_active":"2024-09-20T19:02:11.829309+0000","last_became_peered":"2024-09-20T19:02:11.829309+0000","last_unstale":"2024-09-20T19:02:50.993704+0000","last_undegraded":"2024-09-20T19:02:50.993704+0000","last_fullsized":"2024-09-20T19:02:50.993704+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4181,"log_dups_size":0,"ondisk_log_size":4181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:26.741575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":274006016,"num_objects":4181,"num_object_clones":0,"num_object_copies":12543,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4181,"num_write_kb":267584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"274'29","reported_seq":657,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.837352+0000","last_change":"2024-09-20T19:00:04.810907+0000","last_active":"2024-09-20T19:02:50.837352+0000","last_peered":"2024-09-20T19:02:50.837352+0000","last_clean":"2024-09-20T19:02:50.837352+0000","last_became_active":"2024-09-20T18:49:48.497736+0000","last_became_peered":"2024-09-20T18:49:48.497736+0000","last_unstale":"2024-09-20T19:02:50.837352+0000","last_undegraded":"2024-09-20T19:02:50.837352+0000","last_fullsized":"2024-09-20T19:02:50.837352+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'29","last_scrub_stamp":"2024-09-20T19:00:04.810831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:00:04.810831+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:18.029237+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":609,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.842129+0000","last_change":"2024-09-20T19:02:41.859231+0000","last_active":"2024-09-20T19:02:50.842129+0000","last_peered":"2024-09-20T19:02:50.842129+0000","last_clean":"2024-09-20T19:02:50.842129+0000","last_became_active":"2024-09-20T18:49:45.942253+0000","last_became_peered":"2024-09-20T18:49:45.942253+0000","last_unstale":"2024-09-20T19:02:50.842129+0000","last_undegraded":"2024-09-20T19:02:50.842129+0000","last_fullsized":"2024-09-20T19:02:50.842129+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:41.859168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:41.859168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:03.925342+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":195,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.836896+0000","last_change":"2024-09-20T19:02:27.639446+0000","last_active":"2024-09-20T19:02:50.836896+0000","last_peered":"2024-09-20T19:02:50.836896+0000","last_clean":"2024-09-20T19:02:50.836896+0000","last_became_active":"2024-09-20T18:59:05.085636+0000","last_became_peered":"2024-09-20T18:59:05.085636+0000","last_unstale":"2024-09-20T19:02:50.836896+0000","last_undegraded":"2024-09-20T19:02:50.836896+0000","last_fullsized":"2024-09-20T19:02:50.836896+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:27.639384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:27.639384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:29.637433+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":627,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.841121+0000","last_change":"2024-09-20T19:02:23.886386+0000","last_active":"2024-09-20T19:02:50.841121+0000","last_peered":"2024-09-20T19:02:50.841121+0000","last_clean":"2024-09-20T19:02:50.841121+0000","last_became_active":"2024-09-20T18:59:05.085131+0000","last_became_peered":"2024-09-20T18:59:05.085131+0000","last_unstale":"2024-09-20T19:02:50.841121+0000","last_undegraded":"2024-09-20T19:02:50.841121+0000","last_fullsized":"2024-09-20T19:02:50.841121+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:23.886308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:23.886308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:42.873874+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"344'3918","reported_seq":3962,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:51.003436+0000","last_change":"2024-09-20T19:02:11.827946+0000","last_active":"2024-09-20T19:02:51.003436+0000","last_peered":"2024-09-20T19:02:51.003436+0000","last_clean":"2024-09-20T19:02:51.003436+0000","last_became_active":"2024-09-20T19:02:11.827305+0000","last_became_peered":"2024-09-20T19:02:11.827305+0000","last_unstale":"2024-09-20T19:02:51.003436+0000","last_undegraded":"2024-09-20T19:02:51.003436+0000","last_fullsized":"2024-09-20T19:02:51.003436+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":3918,"log_dups_size":0,"ondisk_log_size":3918,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:14.570559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":256770048,"num_objects":3918,"num_object_clones":0,"num_object_copies":11754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3918,"num_write_kb":250752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"274'40","reported_seq":845,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.842072+0000","last_change":"2024-09-20T19:02:36.901021+0000","last_active":"2024-09-20T19:02:50.842072+0000","last_peered":"2024-09-20T19:02:50.842072+0000","last_clean":"2024-09-20T19:02:50.842072+0000","last_became_active":"2024-09-20T18:49:47.955335+0000","last_became_peered":"2024-09-20T18:49:47.955335+0000","last_unstale":"2024-09-20T19:02:50.842072+0000","last_undegraded":"2024-09-20T19:02:50.842072+0000","last_fullsized":"2024-09-20T19:02:50.842072+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'40","last_scrub_stamp":"2024-09-20T19:02:36.900916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:36.900916+0000","objects_scrubbed":10,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:42.732206+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":598,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.328290+0000","last_change":"2024-09-20T19:01:22.844253+0000","last_active":"2024-09-20T19:02:50.328290+0000","last_peered":"2024-09-20T19:02:50.328290+0000","last_clean":"2024-09-20T19:02:50.328290+0000","last_became_active":"2024-09-20T19:01:22.843966+0000","last_became_peered":"2024-09-20T19:01:22.843966+0000","last_unstale":"2024-09-20T19:02:50.328290+0000","last_undegraded":"2024-09-20T19:02:50.328290+0000","last_fullsized":"2024-09-20T19:02:50.328290+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:00:31.630361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:00:31.630361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":665,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.839462+0000","last_change":"2024-09-20T19:01:39.517101+0000","last_active":"2024-09-20T19:02:50.839462+0000","last_peered":"2024-09-20T19:02:50.839462+0000","last_clean":"2024-09-20T19:02:50.839462+0000","last_became_active":"2024-09-20T18:59:05.086786+0000","last_became_peered":"2024-09-20T18:59:05.086786+0000","last_unstale":"2024-09-20T19:02:50.839462+0000","last_undegraded":"2024-09-20T19:02:50.839462+0000","last_fullsized":"2024-09-20T19:02:50.839462+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-20T19:01:39.517017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:01:39.517017+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:08.168757+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.0012781159999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":621,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.833969+0000","last_change":"2024-09-20T19:01:22.852983+0000","last_active":"2024-09-20T19:02:50.833969+0000","last_peered":"2024-09-20T19:02:50.833969+0000","last_clean":"2024-09-20T19:02:50.833969+0000","last_became_active":"2024-09-20T19:01:22.852495+0000","last_became_peered":"2024-09-20T19:01:22.852495+0000","last_unstale":"2024-09-20T19:02:50.833969+0000","last_undegraded":"2024-09-20T19:02:50.833969+0000","last_fullsized":"2024-09-20T19:02:50.833969+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:59.099736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:59.099736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:13.023185+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":600,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.838713+0000","last_change":"2024-09-20T19:02:04.526741+0000","last_active":"2024-09-20T19:02:50.838713+0000","last_peered":"2024-09-20T19:02:50.838713+0000","last_clean":"2024-09-20T19:02:50.838713+0000","last_became_active":"2024-09-20T19:01:22.837728+0000","last_became_peered":"2024-09-20T19:01:22.837728+0000","last_unstale":"2024-09-20T19:02:50.838713+0000","last_undegraded":"2024-09-20T19:02:50.838713+0000","last_fullsized":"2024-09-20T19:02:50.838713+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:04.526683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:04.526683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:22.012800+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"344'4096","reported_seq":4145,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:51.349906+0000","last_change":"2024-09-20T19:02:11.825132+0000","last_active":"2024-09-20T19:02:51.349906+0000","last_peered":"2024-09-20T19:02:51.349906+0000","last_clean":"2024-09-20T19:02:51.349906+0000","last_became_active":"2024-09-20T19:02:11.824846+0000","last_became_peered":"2024-09-20T19:02:11.824846+0000","last_unstale":"2024-09-20T19:02:51.349906+0000","last_undegraded":"2024-09-20T19:02:51.349906+0000","last_fullsized":"2024-09-20T19:02:51.349906+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4096,"log_dups_size":0,"ondisk_log_size":4096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:12.639014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":268435456,"num_objects":4096,"num_object_clones":0,"num_object_copies":12288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4096,"num_write_kb":262144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"273'17","reported_seq":654,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.842661+0000","last_change":"2024-09-20T19:02:24.853473+0000","last_active":"2024-09-20T19:02:50.842661+0000","last_peered":"2024-09-20T19:02:50.842661+0000","last_clean":"2024-09-20T19:02:50.842661+0000","last_became_active":"2024-09-20T18:49:47.950904+0000","last_became_peered":"2024-09-20T18:49:47.950904+0000","last_unstale":"2024-09-20T19:02:50.842661+0000","last_undegraded":"2024-09-20T19:02:50.842661+0000","last_fullsized":"2024-09-20T19:02:50.842661+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'17","last_scrub_stamp":"2024-09-20T19:02:24.853374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:24.853374+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:46.813829+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":36,"num_read_kb":25,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":639,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.841735+0000","last_change":"2024-09-20T19:02:29.840761+0000","last_active":"2024-09-20T19:02:50.841735+0000","last_peered":"2024-09-20T19:02:50.841735+0000","last_clean":"2024-09-20T19:02:50.841735+0000","last_became_active":"2024-09-20T18:59:05.079789+0000","last_became_peered":"2024-09-20T18:59:05.079789+0000","last_unstale":"2024-09-20T19:02:50.841735+0000","last_undegraded":"2024-09-20T19:02:50.841735+0000","last_fullsized":"2024-09-20T19:02:50.841735+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:29.840601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:29.840601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:39.311309+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":621,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.836109+0000","last_change":"2024-09-20T19:01:58.789304+0000","last_active":"2024-09-20T19:02:50.836109+0000","last_peered":"2024-09-20T19:02:50.836109+0000","last_clean":"2024-09-20T19:02:50.836109+0000","last_became_active":"2024-09-20T18:49:44.937135+0000","last_became_peered":"2024-09-20T18:49:44.937135+0000","last_unstale":"2024-09-20T19:02:50.836109+0000","last_undegraded":"2024-09-20T19:02:50.836109+0000","last_fullsized":"2024-09-20T19:02:50.836109+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:01:58.789251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:01:58.789251+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:25.250777+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00095558800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":596,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.845879+0000","last_change":"2024-09-20T19:01:17.747452+0000","last_active":"2024-09-20T19:02:50.845879+0000","last_peered":"2024-09-20T19:02:50.845879+0000","last_clean":"2024-09-20T19:02:50.845879+0000","last_became_active":"2024-09-20T18:49:49.954424+0000","last_became_peered":"2024-09-20T18:49:49.954424+0000","last_unstale":"2024-09-20T19:02:50.845879+0000","last_undegraded":"2024-09-20T19:02:50.845879+0000","last_fullsized":"2024-09-20T19:02:50.845879+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:17.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:17.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:49.870946+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":607,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.841710+0000","last_change":"2024-09-20T19:01:15.203591+0000","last_active":"2024-09-20T19:02:50.841710+0000","last_peered":"2024-09-20T19:02:50.841710+0000","last_clean":"2024-09-20T19:02:50.841710+0000","last_became_active":"2024-09-20T18:59:05.081459+0000","last_became_peered":"2024-09-20T18:59:05.081459+0000","last_unstale":"2024-09-20T19:02:50.841710+0000","last_undegraded":"2024-09-20T19:02:50.841710+0000","last_fullsized":"2024-09-20T19:02:50.841710+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:15.203529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:15.203529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"344'4038","reported_seq":4087,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:52.331447+0000","last_change":"2024-09-20T19:02:11.828477+0000","last_active":"2024-09-20T19:02:52.331447+0000","last_peered":"2024-09-20T19:02:52.331447+0000","last_clean":"2024-09-20T19:02:52.331447+0000","last_became_active":"2024-09-20T19:02:11.827996+0000","last_became_peered":"2024-09-20T19:02:11.827996+0000","last_unstale":"2024-09-20T19:02:52.331447+0000","last_undegraded":"2024-09-20T19:02:52.331447+0000","last_fullsized":"2024-09-20T19:02:52.331447+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4038,"log_dups_size":0,"ondisk_log_size":4038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:30.922698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":264634368,"num_objects":4038,"num_object_clones":0,"num_object_copies":12114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4038,"num_write_kb":258432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"273'24","reported_seq":658,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.835106+0000","last_change":"2024-09-20T18:59:06.022021+0000","last_active":"2024-09-20T19:02:50.835106+0000","last_peered":"2024-09-20T19:02:50.835106+0000","last_clean":"2024-09-20T19:02:50.835106+0000","last_became_active":"2024-09-20T18:49:47.949230+0000","last_became_peered":"2024-09-20T18:49:47.949230+0000","last_unstale":"2024-09-20T19:02:50.835106+0000","last_undegraded":"2024-09-20T19:02:50.835106+0000","last_fullsized":"2024-09-20T19:02:50.835106+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:59:06.021962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:59:06.021962+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:11.985717+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":629,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.834472+0000","last_change":"2024-09-20T19:02:24.984120+0000","last_active":"2024-09-20T19:02:50.834472+0000","last_peered":"2024-09-20T19:02:50.834472+0000","last_clean":"2024-09-20T19:02:50.834472+0000","last_became_active":"2024-09-20T19:01:22.845659+0000","last_became_peered":"2024-09-20T19:01:22.845659+0000","last_unstale":"2024-09-20T19:02:50.834472+0000","last_undegraded":"2024-09-20T19:02:50.834472+0000","last_fullsized":"2024-09-20T19:02:50.834472+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:24.984022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:24.984022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:27.154123+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":1408,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.840694+0000","last_change":"2024-09-20T19:02:02.494344+0000","last_active":"2024-09-20T19:02:50.840694+0000","last_peered":"2024-09-20T19:02:50.840694+0000","last_clean":"2024-09-20T19:02:50.840694+0000","last_became_active":"2024-09-20T18:49:44.936819+0000","last_became_peered":"2024-09-20T18:49:44.936819+0000","last_unstale":"2024-09-20T19:02:50.840694+0000","last_undegraded":"2024-09-20T19:02:50.840694+0000","last_fullsized":"2024-09-20T19:02:50.840694+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-20T19:02:02.494291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:02:02.494291+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:09.952437+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0010996199999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":779,"num_read_kb":774,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":617,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.842518+0000","last_change":"2024-09-20T19:02:27.785566+0000","last_active":"2024-09-20T19:02:50.842518+0000","last_peered":"2024-09-20T19:02:50.842518+0000","last_clean":"2024-09-20T19:02:50.842518+0000","last_became_active":"2024-09-20T18:49:49.961066+0000","last_became_peered":"2024-09-20T18:49:49.961066+0000","last_unstale":"2024-09-20T19:02:50.842518+0000","last_undegraded":"2024-09-20T19:02:50.842518+0000","last_fullsized":"2024-09-20T19:02:50.842518+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:27.785524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:27.785524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:48.498566+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":613,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.841559+0000","last_change":"2024-09-20T19:02:26.861253+0000","last_active":"2024-09-20T19:02:50.841559+0000","last_peered":"2024-09-20T19:02:50.841559+0000","last_clean":"2024-09-20T19:02:50.841559+0000","last_became_active":"2024-09-20T18:49:51.970343+0000","last_became_peered":"2024-09-20T18:49:51.970343+0000","last_unstale":"2024-09-20T19:02:50.841559+0000","last_undegraded":"2024-09-20T19:02:50.841559+0000","last_fullsized":"2024-09-20T19:02:50.841559+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:26.861185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:26.861185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:38.453435+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"344'4055","reported_seq":4104,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.978262+0000","last_change":"2024-09-20T19:02:11.828065+0000","last_active":"2024-09-20T19:02:50.978262+0000","last_peered":"2024-09-20T19:02:50.978262+0000","last_clean":"2024-09-20T19:02:50.978262+0000","last_became_active":"2024-09-20T19:02:11.827628+0000","last_became_peered":"2024-09-20T19:02:11.827628+0000","last_unstale":"2024-09-20T19:02:50.978262+0000","last_undegraded":"2024-09-20T19:02:50.978262+0000","last_fullsized":"2024-09-20T19:02:50.978262+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4055,"log_dups_size":0,"ondisk_log_size":4055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:21.892715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":265748480,"num_objects":4055,"num_object_clones":0,"num_object_copies":12165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4055,"num_write_kb":259520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"308'32","reported_seq":281,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.837669+0000","last_change":"2024-09-20T19:02:25.699418+0000","last_active":"2024-09-20T19:02:50.837669+0000","last_peered":"2024-09-20T19:02:50.837669+0000","last_clean":"2024-09-20T19:02:50.837669+0000","last_became_active":"2024-09-20T19:01:22.846100+0000","last_became_peered":"2024-09-20T19:01:22.846100+0000","last_unstale":"2024-09-20T19:02:50.837669+0000","last_undegraded":"2024-09-20T19:02:50.837669+0000","last_fullsized":"2024-09-20T19:02:50.837669+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"308'32","last_scrub_stamp":"2024-09-20T19:02:25.699337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:25.699337+0000","objects_scrubbed":8,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:28.007253+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":631,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.838457+0000","last_change":"2024-09-20T19:02:31.965132+0000","last_active":"2024-09-20T19:02:50.838457+0000","last_peered":"2024-09-20T19:02:50.838457+0000","last_clean":"2024-09-20T19:02:50.838457+0000","last_became_active":"2024-09-20T18:59:05.079111+0000","last_became_peered":"2024-09-20T18:59:05.079111+0000","last_unstale":"2024-09-20T19:02:50.838457+0000","last_undegraded":"2024-09-20T19:02:50.838457+0000","last_fullsized":"2024-09-20T19:02:50.838457+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:31.965066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:31.965066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:50.572885+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":677,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.840789+0000","last_change":"2024-09-20T18:59:05.108551+0000","last_active":"2024-09-20T19:02:50.840789+0000","last_peered":"2024-09-20T19:02:50.840789+0000","last_clean":"2024-09-20T19:02:50.840789+0000","last_became_active":"2024-09-20T18:59:05.086103+0000","last_became_peered":"2024-09-20T18:59:05.086103+0000","last_unstale":"2024-09-20T19:02:50.840789+0000","last_undegraded":"2024-09-20T19:02:50.840789+0000","last_fullsized":"2024-09-20T19:02:50.840789+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T18:55:13.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T18:55:13.696607+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:10.266543+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":594,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.841756+0000","last_change":"2024-09-20T18:58:23.545370+0000","last_active":"2024-09-20T19:02:50.841756+0000","last_peered":"2024-09-20T19:02:50.841756+0000","last_clean":"2024-09-20T19:02:50.841756+0000","last_became_active":"2024-09-20T18:49:49.956128+0000","last_became_peered":"2024-09-20T18:49:49.956128+0000","last_unstale":"2024-09-20T19:02:50.841756+0000","last_undegraded":"2024-09-20T19:02:50.841756+0000","last_fullsized":"2024-09-20T19:02:50.841756+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:23.545321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:23.545321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:12.296749+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":624,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.842362+0000","last_change":"2024-09-20T19:01:30.557430+0000","last_active":"2024-09-20T19:02:50.842362+0000","last_peered":"2024-09-20T19:02:50.842362+0000","last_clean":"2024-09-20T19:02:50.842362+0000","last_became_active":"2024-09-20T19:01:22.847692+0000","last_became_peered":"2024-09-20T19:01:22.847692+0000","last_unstale":"2024-09-20T19:02:50.842362+0000","last_undegraded":"2024-09-20T19:02:50.842362+0000","last_fullsized":"2024-09-20T19:02:50.842362+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:30.557368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:30.557368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:47.782922+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.0","version":"344'4058","reported_seq":4107,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:51.007578+0000","last_change":"2024-09-20T19:02:11.825018+0000","last_active":"2024-09-20T19:02:51.007578+0000","last_peered":"2024-09-20T19:02:51.007578+0000","last_clean":"2024-09-20T19:02:51.007578+0000","last_became_active":"2024-09-20T19:02:11.824703+0000","last_became_peered":"2024-09-20T19:02:11.824703+0000","last_unstale":"2024-09-20T19:02:51.007578+0000","last_undegraded":"2024-09-20T19:02:51.007578+0000","last_fullsized":"2024-09-20T19:02:51.007578+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4058,"log_dups_size":0,"ondisk_log_size":4058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:38.061938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":265945088,"num_objects":4058,"num_object_clones":0,"num_object_copies":12174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4058,"num_write_kb":259712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"274'19","reported_seq":616,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.330620+0000","last_change":"2024-09-20T19:01:22.843592+0000","last_active":"2024-09-20T19:02:50.330620+0000","last_peered":"2024-09-20T19:02:50.330620+0000","last_clean":"2024-09-20T19:02:50.330620+0000","last_became_active":"2024-09-20T19:01:22.843143+0000","last_became_peered":"2024-09-20T19:01:22.843143+0000","last_unstale":"2024-09-20T19:02:50.330620+0000","last_undegraded":"2024-09-20T19:02:50.330620+0000","last_fullsized":"2024-09-20T19:02:50.330620+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:58:24.616580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:24.616580+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":576,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.328166+0000","last_change":"2024-09-20T19:01:22.837818+0000","last_active":"2024-09-20T19:02:50.328166+0000","last_peered":"2024-09-20T19:02:50.328166+0000","last_clean":"2024-09-20T19:02:50.328166+0000","last_became_active":"2024-09-20T19:01:22.836557+0000","last_became_peered":"2024-09-20T19:01:22.836557+0000","last_unstale":"2024-09-20T19:02:50.328166+0000","last_undegraded":"2024-09-20T19:02:50.328166+0000","last_fullsized":"2024-09-20T19:02:50.328166+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.693993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.693993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:48.100558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":641,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.842202+0000","last_change":"2024-09-20T19:01:38.463472+0000","last_active":"2024-09-20T19:02:50.842202+0000","last_peered":"2024-09-20T19:02:50.842202+0000","last_clean":"2024-09-20T19:02:50.842202+0000","last_became_active":"2024-09-20T19:01:22.831488+0000","last_became_peered":"2024-09-20T19:01:22.831488+0000","last_unstale":"2024-09-20T19:02:50.842202+0000","last_undegraded":"2024-09-20T19:02:50.842202+0000","last_fullsized":"2024-09-20T19:02:50.842202+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:38.463404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:38.463404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:02:51.812777+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":610,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.838801+0000","last_change":"2024-09-20T18:59:05.080255+0000","last_active":"2024-09-20T19:02:50.838801+0000","last_peered":"2024-09-20T19:02:50.838801+0000","last_clean":"2024-09-20T19:02:50.838801+0000","last_became_active":"2024-09-20T18:59:05.079897+0000","last_became_peered":"2024-09-20T18:59:05.079897+0000","last_unstale":"2024-09-20T19:02:50.838801+0000","last_undegraded":"2024-09-20T19:02:50.838801+0000","last_fullsized":"2024-09-20T19:02:50.838801+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:01.083156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:59:01.083156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:42.880064+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"344'4076","reported_seq":4125,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:51.385769+0000","last_change":"2024-09-20T19:02:11.825263+0000","last_active":"2024-09-20T19:02:51.385769+0000","last_peered":"2024-09-20T19:02:51.385769+0000","last_clean":"2024-09-20T19:02:51.385769+0000","last_became_active":"2024-09-20T19:02:11.825011+0000","last_became_peered":"2024-09-20T19:02:11.825011+0000","last_unstale":"2024-09-20T19:02:51.385769+0000","last_undegraded":"2024-09-20T19:02:51.385769+0000","last_fullsized":"2024-09-20T19:02:51.385769+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4076,"log_dups_size":0,"ondisk_log_size":4076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:12.092479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":267124736,"num_objects":4076,"num_object_clones":0,"num_object_copies":12228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4076,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4076,"num_write_kb":260864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"274'31","reported_seq":623,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.330396+0000","last_change":"2024-09-20T19:01:22.838956+0000","last_active":"2024-09-20T19:02:50.330396+0000","last_peered":"2024-09-20T19:02:50.330396+0000","last_clean":"2024-09-20T19:02:50.330396+0000","last_became_active":"2024-09-20T19:01:22.838577+0000","last_became_peered":"2024-09-20T19:01:22.838577+0000","last_unstale":"2024-09-20T19:02:50.330396+0000","last_undegraded":"2024-09-20T19:02:50.330396+0000","last_fullsized":"2024-09-20T19:02:50.330396+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'31","last_scrub_stamp":"2024-09-20T19:00:03.509980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:00:03.509980+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":49,"num_read_kb":39,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":198,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.837057+0000","last_change":"2024-09-20T19:01:42.479567+0000","last_active":"2024-09-20T19:02:50.837057+0000","last_peered":"2024-09-20T19:02:50.837057+0000","last_clean":"2024-09-20T19:02:50.837057+0000","last_became_active":"2024-09-20T19:01:22.845553+0000","last_became_peered":"2024-09-20T19:01:22.845553+0000","last_unstale":"2024-09-20T19:02:50.837057+0000","last_undegraded":"2024-09-20T19:02:50.837057+0000","last_fullsized":"2024-09-20T19:02:50.837057+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:42.479489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:42.479489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:02:53.833459+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":619,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.841882+0000","last_change":"2024-09-20T19:02:32.856930+0000","last_active":"2024-09-20T19:02:50.841882+0000","last_peered":"2024-09-20T19:02:50.841882+0000","last_clean":"2024-09-20T19:02:50.841882+0000","last_became_active":"2024-09-20T19:01:22.837146+0000","last_became_peered":"2024-09-20T19:01:22.837146+0000","last_unstale":"2024-09-20T19:02:50.841882+0000","last_undegraded":"2024-09-20T19:02:50.841882+0000","last_fullsized":"2024-09-20T19:02:50.841882+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:32.856856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:32.856856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:59.715742+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":618,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.836316+0000","last_change":"2024-09-20T19:01:31.687258+0000","last_active":"2024-09-20T19:02:50.836316+0000","last_peered":"2024-09-20T19:02:50.836316+0000","last_clean":"2024-09-20T19:02:50.836316+0000","last_became_active":"2024-09-20T19:01:22.833913+0000","last_became_peered":"2024-09-20T19:01:22.833913+0000","last_unstale":"2024-09-20T19:02:50.836316+0000","last_undegraded":"2024-09-20T19:02:50.836316+0000","last_fullsized":"2024-09-20T19:02:50.836316+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:31.687188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:31.687188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:44.818347+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2","version":"344'4127","reported_seq":4176,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:51.898417+0000","last_change":"2024-09-20T19:02:11.829938+0000","last_active":"2024-09-20T19:02:51.898417+0000","last_peered":"2024-09-20T19:02:51.898417+0000","last_clean":"2024-09-20T19:02:51.898417+0000","last_became_active":"2024-09-20T19:02:11.829572+0000","last_became_peered":"2024-09-20T19:02:11.829572+0000","last_unstale":"2024-09-20T19:02:51.898417+0000","last_undegraded":"2024-09-20T19:02:51.898417+0000","last_fullsized":"2024-09-20T19:02:51.898417+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4127,"log_dups_size":0,"ondisk_log_size":4127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:34.876687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":270467072,"num_objects":4127,"num_object_clones":0,"num_object_copies":12381,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4127,"num_write_kb":264128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"273'20","reported_seq":621,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.332236+0000","last_change":"2024-09-20T19:01:22.850846+0000","last_active":"2024-09-20T19:02:50.332236+0000","last_peered":"2024-09-20T19:02:50.332236+0000","last_clean":"2024-09-20T19:02:50.332236+0000","last_became_active":"2024-09-20T19:01:22.848851+0000","last_became_peered":"2024-09-20T19:01:22.848851+0000","last_unstale":"2024-09-20T19:02:50.332236+0000","last_undegraded":"2024-09-20T19:02:50.332236+0000","last_fullsized":"2024-09-20T19:02:50.332236+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:59:10.364583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:59:10.364583+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":627,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.838043+0000","last_change":"2024-09-20T19:02:34.986113+0000","last_active":"2024-09-20T19:02:50.838043+0000","last_peered":"2024-09-20T19:02:50.838043+0000","last_clean":"2024-09-20T19:02:50.838043+0000","last_became_active":"2024-09-20T19:01:22.835966+0000","last_became_peered":"2024-09-20T19:01:22.835966+0000","last_unstale":"2024-09-20T19:02:50.838043+0000","last_undegraded":"2024-09-20T19:02:50.838043+0000","last_fullsized":"2024-09-20T19:02:50.838043+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:34.985163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:34.985163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:44.611937+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":785,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.836063+0000","last_change":"2024-09-20T19:02:02.813926+0000","last_active":"2024-09-20T19:02:50.836063+0000","last_peered":"2024-09-20T19:02:50.836063+0000","last_clean":"2024-09-20T19:02:50.836063+0000","last_became_active":"2024-09-20T18:49:49.957005+0000","last_became_peered":"2024-09-20T18:49:49.957005+0000","last_unstale":"2024-09-20T19:02:50.836063+0000","last_undegraded":"2024-09-20T19:02:50.836063+0000","last_fullsized":"2024-09-20T19:02:50.836063+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T19:02:02.813865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:02.813865+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:31.557168+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":550,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.330705+0000","last_change":"2024-09-20T19:01:22.843904+0000","last_active":"2024-09-20T19:02:50.330705+0000","last_peered":"2024-09-20T19:02:50.330705+0000","last_clean":"2024-09-20T19:02:50.330705+0000","last_became_active":"2024-09-20T19:01:22.843167+0000","last_became_peered":"2024-09-20T19:01:22.843167+0000","last_unstale":"2024-09-20T19:02:50.330705+0000","last_undegraded":"2024-09-20T19:02:50.330705+0000","last_fullsized":"2024-09-20T19:02:50.330705+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:48.398542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:59:48.398542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.5","version":"344'3977","reported_seq":4026,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:51.376878+0000","last_change":"2024-09-20T19:02:11.828434+0000","last_active":"2024-09-20T19:02:51.376878+0000","last_peered":"2024-09-20T19:02:51.376878+0000","last_clean":"2024-09-20T19:02:51.376878+0000","last_became_active":"2024-09-20T19:02:11.828162+0000","last_became_peered":"2024-09-20T19:02:11.828162+0000","last_unstale":"2024-09-20T19:02:51.376878+0000","last_undegraded":"2024-09-20T19:02:51.376878+0000","last_fullsized":"2024-09-20T19:02:51.376878+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":3977,"log_dups_size":0,"ondisk_log_size":3977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:13.184255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":260636672,"num_objects":3977,"num_object_clones":0,"num_object_copies":11931,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3977,"num_write_kb":254528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"274'15","reported_seq":649,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.837274+0000","last_change":"2024-09-20T19:01:27.802153+0000","last_active":"2024-09-20T19:02:50.837274+0000","last_peered":"2024-09-20T19:02:50.837274+0000","last_clean":"2024-09-20T19:02:50.837274+0000","last_became_active":"2024-09-20T19:01:22.845683+0000","last_became_peered":"2024-09-20T19:01:22.845683+0000","last_unstale":"2024-09-20T19:02:50.837274+0000","last_undegraded":"2024-09-20T19:02:50.837274+0000","last_fullsized":"2024-09-20T19:02:50.837274+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'15","last_scrub_stamp":"2024-09-20T19:01:27.802100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:27.802100+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:02:51.153972+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":153,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.837726+0000","last_change":"2024-09-20T19:01:52.716074+0000","last_active":"2024-09-20T19:02:50.837726+0000","last_peered":"2024-09-20T19:02:50.837726+0000","last_clean":"2024-09-20T19:02:50.837726+0000","last_became_active":"2024-09-20T19:00:01.190875+0000","last_became_peered":"2024-09-20T19:00:01.190875+0000","last_unstale":"2024-09-20T19:02:50.837726+0000","last_undegraded":"2024-09-20T19:02:50.837726+0000","last_fullsized":"2024-09-20T19:02:50.837726+0000","mapping_epoch":277,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":278,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:52.716025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:52.716025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:08.654757+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":623,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.836098+0000","last_change":"2024-09-20T19:02:24.939055+0000","last_active":"2024-09-20T19:02:50.836098+0000","last_peered":"2024-09-20T19:02:50.836098+0000","last_clean":"2024-09-20T19:02:50.836098+0000","last_became_active":"2024-09-20T19:01:22.835347+0000","last_became_peered":"2024-09-20T19:01:22.835347+0000","last_unstale":"2024-09-20T19:02:50.836098+0000","last_undegraded":"2024-09-20T19:02:50.836098+0000","last_fullsized":"2024-09-20T19:02:50.836098+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:24.938980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:24.938980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:52.810786+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":605,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.834384+0000","last_change":"2024-09-20T19:02:31.120809+0000","last_active":"2024-09-20T19:02:50.834384+0000","last_peered":"2024-09-20T19:02:50.834384+0000","last_clean":"2024-09-20T19:02:50.834384+0000","last_became_active":"2024-09-20T18:49:51.964098+0000","last_became_peered":"2024-09-20T18:49:51.964098+0000","last_unstale":"2024-09-20T19:02:50.834384+0000","last_undegraded":"2024-09-20T19:02:50.834384+0000","last_fullsized":"2024-09-20T19:02:50.834384+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:31.120623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:31.120623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:41.294006+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"344'4092","reported_seq":4141,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:52.347505+0000","last_change":"2024-09-20T19:02:11.829578+0000","last_active":"2024-09-20T19:02:52.347505+0000","last_peered":"2024-09-20T19:02:52.347505+0000","last_clean":"2024-09-20T19:02:52.347505+0000","last_became_active":"2024-09-20T19:02:11.828890+0000","last_became_peered":"2024-09-20T19:02:11.828890+0000","last_unstale":"2024-09-20T19:02:52.347505+0000","last_undegraded":"2024-09-20T19:02:52.347505+0000","last_fullsized":"2024-09-20T19:02:52.347505+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4092,"log_dups_size":0,"ondisk_log_size":4092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:19.809730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":268173312,"num_objects":4092,"num_object_clones":0,"num_object_copies":12276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4092,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4092,"num_write_kb":261888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"274'35","reported_seq":231,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.836957+0000","last_change":"2024-09-20T19:01:40.477072+0000","last_active":"2024-09-20T19:02:50.836957+0000","last_peered":"2024-09-20T19:02:50.836957+0000","last_clean":"2024-09-20T19:02:50.836957+0000","last_became_active":"2024-09-20T18:59:05.087391+0000","last_became_peered":"2024-09-20T18:59:05.087391+0000","last_unstale":"2024-09-20T19:02:50.836957+0000","last_undegraded":"2024-09-20T19:02:50.836957+0000","last_fullsized":"2024-09-20T19:02:50.836957+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'35","last_scrub_stamp":"2024-09-20T19:01:40.476968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:40.476968+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:09.580262+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":631,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.841980+0000","last_change":"2024-09-20T19:02:05.440745+0000","last_active":"2024-09-20T19:02:50.841980+0000","last_peered":"2024-09-20T19:02:50.841980+0000","last_clean":"2024-09-20T19:02:50.841980+0000","last_became_active":"2024-09-20T18:59:05.086670+0000","last_became_peered":"2024-09-20T18:59:05.086670+0000","last_unstale":"2024-09-20T19:02:50.841980+0000","last_undegraded":"2024-09-20T19:02:50.841980+0000","last_fullsized":"2024-09-20T19:02:50.841980+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:05.440677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:05.440677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:22.708355+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":619,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.836067+0000","last_change":"2024-09-20T19:02:42.745448+0000","last_active":"2024-09-20T19:02:50.836067+0000","last_peered":"2024-09-20T19:02:50.836067+0000","last_clean":"2024-09-20T19:02:50.836067+0000","last_became_active":"2024-09-20T18:49:49.957558+0000","last_became_peered":"2024-09-20T18:49:49.957558+0000","last_unstale":"2024-09-20T19:02:50.836067+0000","last_undegraded":"2024-09-20T19:02:50.836067+0000","last_fullsized":"2024-09-20T19:02:50.836067+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:42.745372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:42.745372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:49.093207+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":597,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.840715+0000","last_change":"2024-09-20T18:53:47.370152+0000","last_active":"2024-09-20T19:02:50.840715+0000","last_peered":"2024-09-20T19:02:50.840715+0000","last_clean":"2024-09-20T19:02:50.840715+0000","last_became_active":"2024-09-20T18:49:51.970798+0000","last_became_peered":"2024-09-20T18:49:51.970798+0000","last_unstale":"2024-09-20T19:02:50.840715+0000","last_undegraded":"2024-09-20T19:02:50.840715+0000","last_fullsized":"2024-09-20T19:02:50.840715+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:47.370108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:47.370108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:19.239137+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.7","version":"344'4026","reported_seq":4075,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.998159+0000","last_change":"2024-09-20T19:02:11.827681+0000","last_active":"2024-09-20T19:02:50.998159+0000","last_peered":"2024-09-20T19:02:50.998159+0000","last_clean":"2024-09-20T19:02:50.998159+0000","last_became_active":"2024-09-20T19:02:11.827360+0000","last_became_peered":"2024-09-20T19:02:11.827360+0000","last_unstale":"2024-09-20T19:02:50.998159+0000","last_undegraded":"2024-09-20T19:02:50.998159+0000","last_fullsized":"2024-09-20T19:02:50.998159+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4026,"log_dups_size":0,"ondisk_log_size":4026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:31.729709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":263847936,"num_objects":4026,"num_object_clones":0,"num_object_copies":12078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4026,"num_write_kb":257664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"273'18","reported_seq":586,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.328311+0000","last_change":"2024-09-20T19:01:22.847461+0000","last_active":"2024-09-20T19:02:50.328311+0000","last_peered":"2024-09-20T19:02:50.328311+0000","last_clean":"2024-09-20T19:02:50.328311+0000","last_became_active":"2024-09-20T19:01:22.847140+0000","last_became_peered":"2024-09-20T19:01:22.847140+0000","last_unstale":"2024-09-20T19:02:50.328311+0000","last_undegraded":"2024-09-20T19:02:50.328311+0000","last_fullsized":"2024-09-20T19:02:50.328311+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'18","last_scrub_stamp":"2024-09-20T18:59:55.490086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:59:55.490086+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":628,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.842099+0000","last_change":"2024-09-20T19:01:22.844920+0000","last_active":"2024-09-20T19:02:50.842099+0000","last_peered":"2024-09-20T19:02:50.842099+0000","last_clean":"2024-09-20T19:02:50.842099+0000","last_became_active":"2024-09-20T19:01:22.844608+0000","last_became_peered":"2024-09-20T19:01:22.844608+0000","last_unstale":"2024-09-20T19:02:50.842099+0000","last_undegraded":"2024-09-20T19:02:50.842099+0000","last_fullsized":"2024-09-20T19:02:50.842099+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:00:34.940850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:00:34.940850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:47.134749+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":612,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.835838+0000","last_change":"2024-09-20T19:01:53.734774+0000","last_active":"2024-09-20T19:02:50.835838+0000","last_peered":"2024-09-20T19:02:50.835838+0000","last_clean":"2024-09-20T19:02:50.835838+0000","last_became_active":"2024-09-20T18:49:49.954944+0000","last_became_peered":"2024-09-20T18:49:49.954944+0000","last_unstale":"2024-09-20T19:02:50.835838+0000","last_undegraded":"2024-09-20T19:02:50.835838+0000","last_fullsized":"2024-09-20T19:02:50.835838+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:53.734726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:53.734726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:21.517887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":604,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.838275+0000","last_change":"2024-09-20T19:02:35.968178+0000","last_active":"2024-09-20T19:02:50.838275+0000","last_peered":"2024-09-20T19:02:50.838275+0000","last_clean":"2024-09-20T19:02:50.838275+0000","last_became_active":"2024-09-20T18:49:51.972970+0000","last_became_peered":"2024-09-20T18:49:51.972970+0000","last_unstale":"2024-09-20T19:02:50.838275+0000","last_undegraded":"2024-09-20T19:02:50.838275+0000","last_fullsized":"2024-09-20T19:02:50.838275+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:35.968052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:35.968052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:53.629726+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.6","version":"344'4107","reported_seq":4156,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:51.342237+0000","last_change":"2024-09-20T19:02:11.829337+0000","last_active":"2024-09-20T19:02:51.342237+0000","last_peered":"2024-09-20T19:02:51.342237+0000","last_clean":"2024-09-20T19:02:51.342237+0000","last_became_active":"2024-09-20T19:02:11.829014+0000","last_became_peered":"2024-09-20T19:02:11.829014+0000","last_unstale":"2024-09-20T19:02:51.342237+0000","last_undegraded":"2024-09-20T19:02:51.342237+0000","last_fullsized":"2024-09-20T19:02:51.342237+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4107,"log_dups_size":0,"ondisk_log_size":4107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:29.577085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":269156352,"num_objects":4107,"num_object_clones":0,"num_object_copies":12321,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4107,"num_write_kb":262848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"273'27","reported_seq":710,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.838735+0000","last_change":"2024-09-20T19:02:39.709344+0000","last_active":"2024-09-20T19:02:50.838735+0000","last_peered":"2024-09-20T19:02:50.838735+0000","last_clean":"2024-09-20T19:02:50.838735+0000","last_became_active":"2024-09-20T18:59:05.083726+0000","last_became_peered":"2024-09-20T18:59:05.083726+0000","last_unstale":"2024-09-20T19:02:50.838735+0000","last_undegraded":"2024-09-20T19:02:50.838735+0000","last_fullsized":"2024-09-20T19:02:50.838735+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'27","last_scrub_stamp":"2024-09-20T19:02:39.709269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:39.709269+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:44.315182+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":648,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.840119+0000","last_change":"2024-09-20T19:01:22.830278+0000","last_active":"2024-09-20T19:02:50.840119+0000","last_peered":"2024-09-20T19:02:50.840119+0000","last_clean":"2024-09-20T19:02:50.840119+0000","last_became_active":"2024-09-20T19:01:22.829928+0000","last_became_peered":"2024-09-20T19:01:22.829928+0000","last_unstale":"2024-09-20T19:02:50.840119+0000","last_undegraded":"2024-09-20T19:02:50.840119+0000","last_fullsized":"2024-09-20T19:02:50.840119+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-20T18:58:57.563058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:57.563058+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:45.184378+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"308'12","reported_seq":795,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:51.154507+0000","last_change":"2024-09-20T19:01:22.833874+0000","last_active":"2024-09-20T19:02:51.154507+0000","last_peered":"2024-09-20T19:02:51.154507+0000","last_clean":"2024-09-20T19:02:51.154507+0000","last_became_active":"2024-09-20T19:01:22.832710+0000","last_became_peered":"2024-09-20T19:01:22.832710+0000","last_unstale":"2024-09-20T19:02:51.154507+0000","last_undegraded":"2024-09-20T19:02:51.154507+0000","last_fullsized":"2024-09-20T19:02:51.154507+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'9","last_scrub_stamp":"2024-09-20T18:58:27.521051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:27.521051+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:21.248521+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":618,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.839028+0000","last_change":"2024-09-20T19:01:56.634132+0000","last_active":"2024-09-20T19:02:50.839028+0000","last_peered":"2024-09-20T19:02:50.839028+0000","last_clean":"2024-09-20T19:02:50.839028+0000","last_became_active":"2024-09-20T18:59:05.087005+0000","last_became_peered":"2024-09-20T18:59:05.087005+0000","last_unstale":"2024-09-20T19:02:50.839028+0000","last_undegraded":"2024-09-20T19:02:50.839028+0000","last_fullsized":"2024-09-20T19:02:50.839028+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:56.634089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:56.634089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:05.408503+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":187,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.836663+0000","last_change":"2024-09-20T19:01:16.542398+0000","last_active":"2024-09-20T19:02:50.836663+0000","last_peered":"2024-09-20T19:02:50.836663+0000","last_clean":"2024-09-20T19:02:50.836663+0000","last_became_active":"2024-09-20T18:59:05.085427+0000","last_became_peered":"2024-09-20T18:59:05.085427+0000","last_unstale":"2024-09-20T19:02:50.836663+0000","last_undegraded":"2024-09-20T19:02:50.836663+0000","last_fullsized":"2024-09-20T19:02:50.836663+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:16.542335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:16.542335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:47.734953+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"273'12","reported_seq":630,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.840433+0000","last_change":"2024-09-20T19:01:12.385223+0000","last_active":"2024-09-20T19:02:50.840433+0000","last_peered":"2024-09-20T19:02:50.840433+0000","last_clean":"2024-09-20T19:02:50.840433+0000","last_became_active":"2024-09-20T18:49:48.500662+0000","last_became_peered":"2024-09-20T18:49:48.500662+0000","last_unstale":"2024-09-20T19:02:50.840433+0000","last_undegraded":"2024-09-20T19:02:50.840433+0000","last_fullsized":"2024-09-20T19:02:50.840433+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'12","last_scrub_stamp":"2024-09-20T19:01:12.385152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:12.385152+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":608,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.840291+0000","last_change":"2024-09-20T19:01:14.466096+0000","last_active":"2024-09-20T19:02:50.840291+0000","last_peered":"2024-09-20T19:02:50.840291+0000","last_clean":"2024-09-20T19:02:50.840291+0000","last_became_active":"2024-09-20T18:49:49.956749+0000","last_became_peered":"2024-09-20T18:49:49.956749+0000","last_unstale":"2024-09-20T19:02:50.840291+0000","last_undegraded":"2024-09-20T19:02:50.840291+0000","last_fullsized":"2024-09-20T19:02:50.840291+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:14.466042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:14.466042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:15.787734+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":630,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.840397+0000","last_change":"2024-09-20T19:02:03.509954+0000","last_active":"2024-09-20T19:02:50.840397+0000","last_peered":"2024-09-20T19:02:50.840397+0000","last_clean":"2024-09-20T19:02:50.840397+0000","last_became_active":"2024-09-20T19:01:22.830233+0000","last_became_peered":"2024-09-20T19:01:22.830233+0000","last_unstale":"2024-09-20T19:02:50.840397+0000","last_undegraded":"2024-09-20T19:02:50.840397+0000","last_fullsized":"2024-09-20T19:02:50.840397+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:03.509903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:03.509903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:11.556992+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"273'15","reported_seq":630,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.835155+0000","last_change":"2024-09-20T19:01:15.875779+0000","last_active":"2024-09-20T19:02:50.835155+0000","last_peered":"2024-09-20T19:02:50.835155+0000","last_clean":"2024-09-20T19:02:50.835155+0000","last_became_active":"2024-09-20T18:49:48.500803+0000","last_became_peered":"2024-09-20T18:49:48.500803+0000","last_unstale":"2024-09-20T19:02:50.835155+0000","last_undegraded":"2024-09-20T19:02:50.835155+0000","last_fullsized":"2024-09-20T19:02:50.835155+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:01:15.875693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:15.875693+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:49.127502+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":634,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.840687+0000","last_change":"2024-09-20T19:01:22.834162+0000","last_active":"2024-09-20T19:02:50.840687+0000","last_peered":"2024-09-20T19:02:50.840687+0000","last_clean":"2024-09-20T19:02:50.840687+0000","last_became_active":"2024-09-20T19:01:22.831615+0000","last_became_peered":"2024-09-20T19:01:22.831615+0000","last_unstale":"2024-09-20T19:02:50.840687+0000","last_undegraded":"2024-09-20T19:02:50.840687+0000","last_fullsized":"2024-09-20T19:02:50.840687+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:59.589891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:59.589891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:43.164641+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":608,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.840357+0000","last_change":"2024-09-20T19:01:06.482291+0000","last_active":"2024-09-20T19:02:50.840357+0000","last_peered":"2024-09-20T19:02:50.840357+0000","last_clean":"2024-09-20T19:02:50.840357+0000","last_became_active":"2024-09-20T18:49:49.961947+0000","last_became_peered":"2024-09-20T18:49:49.961947+0000","last_unstale":"2024-09-20T19:02:50.840357+0000","last_undegraded":"2024-09-20T19:02:50.840357+0000","last_fullsized":"2024-09-20T19:02:50.840357+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:06.482212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:06.482212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"66'1","reported_seq":627,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.837816+0000","last_change":"2024-09-20T19:02:01.843182+0000","last_active":"2024-09-20T19:02:50.837816+0000","last_peered":"2024-09-20T19:02:50.837816+0000","last_clean":"2024-09-20T19:02:50.837816+0000","last_became_active":"2024-09-20T19:01:22.837187+0000","last_became_peered":"2024-09-20T19:01:22.837187+0000","last_unstale":"2024-09-20T19:02:50.837816+0000","last_undegraded":"2024-09-20T19:02:50.837816+0000","last_fullsized":"2024-09-20T19:02:50.837816+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T19:02:01.843111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:01.843111+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:06.311104+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"274'15","reported_seq":629,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.839822+0000","last_change":"2024-09-20T19:01:36.532486+0000","last_active":"2024-09-20T19:02:50.839822+0000","last_peered":"2024-09-20T19:02:50.839822+0000","last_clean":"2024-09-20T19:02:50.839822+0000","last_became_active":"2024-09-20T18:49:48.501011+0000","last_became_peered":"2024-09-20T18:49:48.501011+0000","last_unstale":"2024-09-20T19:02:50.839822+0000","last_undegraded":"2024-09-20T19:02:50.839822+0000","last_fullsized":"2024-09-20T19:02:50.839822+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'15","last_scrub_stamp":"2024-09-20T19:01:36.532419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:36.532419+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:02:57.030996+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":617,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.835019+0000","last_change":"2024-09-20T19:01:30.864866+0000","last_active":"2024-09-20T19:02:50.835019+0000","last_peered":"2024-09-20T19:02:50.835019+0000","last_clean":"2024-09-20T19:02:50.835019+0000","last_became_active":"2024-09-20T18:49:45.938511+0000","last_became_peered":"2024-09-20T18:49:45.938511+0000","last_unstale":"2024-09-20T19:02:50.835019+0000","last_undegraded":"2024-09-20T19:02:50.835019+0000","last_fullsized":"2024-09-20T19:02:50.835019+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:30.864794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:30.864794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:02:51.939240+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":774,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.838990+0000","last_change":"2024-09-20T19:00:34.485855+0000","last_active":"2024-09-20T19:02:50.838990+0000","last_peered":"2024-09-20T19:02:50.838990+0000","last_clean":"2024-09-20T19:02:50.838990+0000","last_became_active":"2024-09-20T18:49:49.960550+0000","last_became_peered":"2024-09-20T18:49:49.960550+0000","last_unstale":"2024-09-20T19:02:50.838990+0000","last_undegraded":"2024-09-20T19:02:50.838990+0000","last_fullsized":"2024-09-20T19:02:50.838990+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T19:00:34.485624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:00:34.485624+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":558,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.330648+0000","last_change":"2024-09-20T19:01:22.848313+0000","last_active":"2024-09-20T19:02:50.330648+0000","last_peered":"2024-09-20T19:02:50.330648+0000","last_clean":"2024-09-20T19:02:50.330648+0000","last_became_active":"2024-09-20T19:01:22.848017+0000","last_became_peered":"2024-09-20T19:01:22.848017+0000","last_unstale":"2024-09-20T19:02:50.330648+0000","last_undegraded":"2024-09-20T19:02:50.330648+0000","last_fullsized":"2024-09-20T19:02:50.330648+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:59.497810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:59:59.497810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"274'20","reported_seq":645,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.839357+0000","last_change":"2024-09-20T18:57:20.701592+0000","last_active":"2024-09-20T19:02:50.839357+0000","last_peered":"2024-09-20T19:02:50.839357+0000","last_clean":"2024-09-20T19:02:50.839357+0000","last_became_active":"2024-09-20T18:49:58.997494+0000","last_became_peered":"2024-09-20T18:49:58.997494+0000","last_unstale":"2024-09-20T19:02:50.839357+0000","last_undegraded":"2024-09-20T19:02:50.839357+0000","last_fullsized":"2024-09-20T19:02:50.839357+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:57:20.701543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:20.701543+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:12.702800+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":620,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.840840+0000","last_change":"2024-09-20T19:01:22.833495+0000","last_active":"2024-09-20T19:02:50.840840+0000","last_peered":"2024-09-20T19:02:50.840840+0000","last_clean":"2024-09-20T19:02:50.840840+0000","last_became_active":"2024-09-20T19:01:22.831291+0000","last_became_peered":"2024-09-20T19:01:22.831291+0000","last_unstale":"2024-09-20T19:02:50.840840+0000","last_undegraded":"2024-09-20T19:02:50.840840+0000","last_fullsized":"2024-09-20T19:02:50.840840+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.632380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.632380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:07.354816+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":774,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.839674+0000","last_change":"2024-09-20T19:01:10.601374+0000","last_active":"2024-09-20T19:02:50.839674+0000","last_peered":"2024-09-20T19:02:50.839674+0000","last_clean":"2024-09-20T19:02:50.839674+0000","last_became_active":"2024-09-20T18:49:49.962243+0000","last_became_peered":"2024-09-20T18:49:49.962243+0000","last_unstale":"2024-09-20T19:02:50.839674+0000","last_undegraded":"2024-09-20T19:02:50.839674+0000","last_fullsized":"2024-09-20T19:02:50.839674+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T19:01:10.601300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:10.601300+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:46.785830+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":613,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.834735+0000","last_change":"2024-09-20T19:01:03.821977+0000","last_active":"2024-09-20T19:02:50.834735+0000","last_peered":"2024-09-20T19:02:50.834735+0000","last_clean":"2024-09-20T19:02:50.834735+0000","last_became_active":"2024-09-20T18:49:51.969093+0000","last_became_peered":"2024-09-20T18:49:51.969093+0000","last_unstale":"2024-09-20T19:02:50.834735+0000","last_undegraded":"2024-09-20T19:02:50.834735+0000","last_fullsized":"2024-09-20T19:02:50.834735+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:03.821908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:03.821908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:47.087429+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"273'19","reported_seq":622,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.328415+0000","last_change":"2024-09-20T19:01:22.841541+0000","last_active":"2024-09-20T19:02:50.328415+0000","last_peered":"2024-09-20T19:02:50.328415+0000","last_clean":"2024-09-20T19:02:50.328415+0000","last_became_active":"2024-09-20T19:01:22.841252+0000","last_became_peered":"2024-09-20T19:01:22.841252+0000","last_unstale":"2024-09-20T19:02:50.328415+0000","last_undegraded":"2024-09-20T19:02:50.328415+0000","last_fullsized":"2024-09-20T19:02:50.328415+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:57:50.899514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:50.899514+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:20.035402+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":540,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.332154+0000","last_change":"2024-09-20T19:01:22.838793+0000","last_active":"2024-09-20T19:02:50.332154+0000","last_peered":"2024-09-20T19:02:50.332154+0000","last_clean":"2024-09-20T19:02:50.332154+0000","last_became_active":"2024-09-20T19:01:22.838429+0000","last_became_peered":"2024-09-20T19:01:22.838429+0000","last_unstale":"2024-09-20T19:02:50.332154+0000","last_undegraded":"2024-09-20T19:02:50.332154+0000","last_fullsized":"2024-09-20T19:02:50.332154+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:06.310027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:59:06.310027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":608,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.841795+0000","last_change":"2024-09-20T19:02:38.906352+0000","last_active":"2024-09-20T19:02:50.841795+0000","last_peered":"2024-09-20T19:02:50.841795+0000","last_clean":"2024-09-20T19:02:50.841795+0000","last_became_active":"2024-09-20T18:49:49.957067+0000","last_became_peered":"2024-09-20T18:49:49.957067+0000","last_unstale":"2024-09-20T19:02:50.841795+0000","last_undegraded":"2024-09-20T19:02:50.841795+0000","last_fullsized":"2024-09-20T19:02:50.841795+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:38.906161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:38.906161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:41.340426+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":588,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.839684+0000","last_change":"2024-09-20T18:57:16.789412+0000","last_active":"2024-09-20T19:02:50.839684+0000","last_peered":"2024-09-20T19:02:50.839684+0000","last_clean":"2024-09-20T19:02:50.839684+0000","last_became_active":"2024-09-20T18:49:51.966915+0000","last_became_peered":"2024-09-20T18:49:51.966915+0000","last_unstale":"2024-09-20T19:02:50.839684+0000","last_undegraded":"2024-09-20T19:02:50.839684+0000","last_fullsized":"2024-09-20T19:02:50.839684+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:16.789095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:16.789095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:17.675828+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"273'15","reported_seq":679,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.841963+0000","last_change":"2024-09-20T19:02:30.816061+0000","last_active":"2024-09-20T19:02:50.841963+0000","last_peered":"2024-09-20T19:02:50.841963+0000","last_clean":"2024-09-20T19:02:50.841963+0000","last_became_active":"2024-09-20T18:59:05.085230+0000","last_became_peered":"2024-09-20T18:59:05.085230+0000","last_unstale":"2024-09-20T19:02:50.841963+0000","last_undegraded":"2024-09-20T19:02:50.841963+0000","last_fullsized":"2024-09-20T19:02:50.841963+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:02:30.815747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:30.815747+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:44.961926+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":612,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.840123+0000","last_change":"2024-09-20T19:01:22.830951+0000","last_active":"2024-09-20T19:02:50.840123+0000","last_peered":"2024-09-20T19:02:50.840123+0000","last_clean":"2024-09-20T19:02:50.840123+0000","last_became_active":"2024-09-20T19:01:22.830642+0000","last_became_peered":"2024-09-20T19:01:22.830642+0000","last_unstale":"2024-09-20T19:02:50.840123+0000","last_undegraded":"2024-09-20T19:02:50.840123+0000","last_fullsized":"2024-09-20T19:02:50.840123+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:42.487403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:42.487403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:06.397840+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":608,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.840224+0000","last_change":"2024-09-20T19:01:03.507072+0000","last_active":"2024-09-20T19:02:50.840224+0000","last_peered":"2024-09-20T19:02:50.840224+0000","last_clean":"2024-09-20T19:02:50.840224+0000","last_became_active":"2024-09-20T18:49:49.962523+0000","last_became_peered":"2024-09-20T18:49:49.962523+0000","last_unstale":"2024-09-20T19:02:50.840224+0000","last_undegraded":"2024-09-20T19:02:50.840224+0000","last_fullsized":"2024-09-20T19:02:50.840224+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:03.507016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:03.507016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":605,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.834068+0000","last_change":"2024-09-20T19:01:11.847792+0000","last_active":"2024-09-20T19:02:50.834068+0000","last_peered":"2024-09-20T19:02:50.834068+0000","last_clean":"2024-09-20T19:02:50.834068+0000","last_became_active":"2024-09-20T18:49:51.967313+0000","last_became_peered":"2024-09-20T18:49:51.967313+0000","last_unstale":"2024-09-20T19:02:50.834068+0000","last_undegraded":"2024-09-20T19:02:50.834068+0000","last_fullsized":"2024-09-20T19:02:50.834068+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:11.847716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:11.847716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:21.028973+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"274'19","reported_seq":681,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.839794+0000","last_change":"2024-09-20T19:01:05.495603+0000","last_active":"2024-09-20T19:02:50.839794+0000","last_peered":"2024-09-20T19:02:50.839794+0000","last_clean":"2024-09-20T19:02:50.839794+0000","last_became_active":"2024-09-20T18:59:05.075390+0000","last_became_peered":"2024-09-20T18:59:05.075390+0000","last_unstale":"2024-09-20T19:02:50.839794+0000","last_undegraded":"2024-09-20T19:02:50.839794+0000","last_fullsized":"2024-09-20T19:02:50.839794+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'19","last_scrub_stamp":"2024-09-20T19:01:05.495478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:05.495478+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":624,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.840375+0000","last_change":"2024-09-20T19:01:47.521858+0000","last_active":"2024-09-20T19:02:50.840375+0000","last_peered":"2024-09-20T19:02:50.840375+0000","last_clean":"2024-09-20T19:02:50.840375+0000","last_became_active":"2024-09-20T18:49:45.942105+0000","last_became_peered":"2024-09-20T18:49:45.942105+0000","last_unstale":"2024-09-20T19:02:50.840375+0000","last_undegraded":"2024-09-20T19:02:50.840375+0000","last_fullsized":"2024-09-20T19:02:50.840375+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:47.521789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:47.521789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:09.440449+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":634,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.839736+0000","last_change":"2024-09-20T19:02:27.267300+0000","last_active":"2024-09-20T19:02:50.839736+0000","last_peered":"2024-09-20T19:02:50.839736+0000","last_clean":"2024-09-20T19:02:50.839736+0000","last_became_active":"2024-09-20T19:01:22.827211+0000","last_became_peered":"2024-09-20T19:01:22.827211+0000","last_unstale":"2024-09-20T19:02:50.839736+0000","last_undegraded":"2024-09-20T19:02:50.839736+0000","last_fullsized":"2024-09-20T19:02:50.839736+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:27.267254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:27.267254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:51.446089+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":610,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.840364+0000","last_change":"2024-09-20T19:02:38.950217+0000","last_active":"2024-09-20T19:02:50.840364+0000","last_peered":"2024-09-20T19:02:50.840364+0000","last_clean":"2024-09-20T19:02:50.840364+0000","last_became_active":"2024-09-20T18:59:05.081179+0000","last_became_peered":"2024-09-20T18:59:05.081179+0000","last_unstale":"2024-09-20T19:02:50.840364+0000","last_undegraded":"2024-09-20T19:02:50.840364+0000","last_fullsized":"2024-09-20T19:02:50.840364+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:38.950017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:38.950017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:02.988187+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"273'8","reported_seq":650,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.838738+0000","last_change":"2024-09-20T19:00:33.504967+0000","last_active":"2024-09-20T19:02:50.838738+0000","last_peered":"2024-09-20T19:02:50.838738+0000","last_clean":"2024-09-20T19:02:50.838738+0000","last_became_active":"2024-09-20T18:59:05.076365+0000","last_became_peered":"2024-09-20T18:59:05.076365+0000","last_unstale":"2024-09-20T19:02:50.838738+0000","last_undegraded":"2024-09-20T19:02:50.838738+0000","last_fullsized":"2024-09-20T19:02:50.838738+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'8","last_scrub_stamp":"2024-09-20T19:00:33.504891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:00:33.504891+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":625,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.833625+0000","last_change":"2024-09-20T19:02:39.184249+0000","last_active":"2024-09-20T19:02:50.833625+0000","last_peered":"2024-09-20T19:02:50.833625+0000","last_clean":"2024-09-20T19:02:50.833625+0000","last_became_active":"2024-09-20T18:49:45.940988+0000","last_became_peered":"2024-09-20T18:49:45.940988+0000","last_unstale":"2024-09-20T19:02:50.833625+0000","last_undegraded":"2024-09-20T19:02:50.833625+0000","last_fullsized":"2024-09-20T19:02:50.833625+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:39.184150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:39.184150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:54.134855+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":198,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.836808+0000","last_change":"2024-09-20T19:01:41.497193+0000","last_active":"2024-09-20T19:02:50.836808+0000","last_peered":"2024-09-20T19:02:50.836808+0000","last_clean":"2024-09-20T19:02:50.836808+0000","last_became_active":"2024-09-20T19:01:22.845873+0000","last_became_peered":"2024-09-20T19:01:22.845873+0000","last_unstale":"2024-09-20T19:02:50.836808+0000","last_undegraded":"2024-09-20T19:02:50.836808+0000","last_fullsized":"2024-09-20T19:02:50.836808+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:41.497115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:41.497115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":608,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.840271+0000","last_change":"2024-09-20T19:02:29.277994+0000","last_active":"2024-09-20T19:02:50.840271+0000","last_peered":"2024-09-20T19:02:50.840271+0000","last_clean":"2024-09-20T19:02:50.840271+0000","last_became_active":"2024-09-20T19:01:22.829961+0000","last_became_peered":"2024-09-20T19:01:22.829961+0000","last_unstale":"2024-09-20T19:02:50.840271+0000","last_undegraded":"2024-09-20T19:02:50.840271+0000","last_fullsized":"2024-09-20T19:02:50.840271+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:29.277916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:29.277916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:32.717744+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":591,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.840521+0000","last_change":"2024-09-20T19:01:22.834863+0000","last_active":"2024-09-20T19:02:50.840521+0000","last_peered":"2024-09-20T19:02:50.840521+0000","last_clean":"2024-09-20T19:02:50.840521+0000","last_became_active":"2024-09-20T19:01:22.834453+0000","last_became_peered":"2024-09-20T19:01:22.834453+0000","last_unstale":"2024-09-20T19:02:50.840521+0000","last_undegraded":"2024-09-20T19:02:50.840521+0000","last_fullsized":"2024-09-20T19:02:50.840521+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:10.970779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:10.970779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:07.916601+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":608,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.838962+0000","last_change":"2024-09-20T18:58:36.077005+0000","last_active":"2024-09-20T19:02:50.838962+0000","last_peered":"2024-09-20T19:02:50.838962+0000","last_clean":"2024-09-20T19:02:50.838962+0000","last_became_active":"2024-09-20T18:49:45.937992+0000","last_became_peered":"2024-09-20T18:49:45.937992+0000","last_unstale":"2024-09-20T19:02:50.838962+0000","last_undegraded":"2024-09-20T19:02:50.838962+0000","last_fullsized":"2024-09-20T19:02:50.838962+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:36.076937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:36.076937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:21.762264+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"274'19","reported_seq":211,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.837966+0000","last_change":"2024-09-20T19:01:15.544760+0000","last_active":"2024-09-20T19:02:50.837966+0000","last_peered":"2024-09-20T19:02:50.837966+0000","last_clean":"2024-09-20T19:02:50.837966+0000","last_became_active":"2024-09-20T18:59:05.089294+0000","last_became_peered":"2024-09-20T18:59:05.089294+0000","last_unstale":"2024-09-20T19:02:50.837966+0000","last_undegraded":"2024-09-20T19:02:50.837966+0000","last_fullsized":"2024-09-20T19:02:50.837966+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'19","last_scrub_stamp":"2024-09-20T19:01:15.544663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:15.544663+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:17.714021+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":602,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-20T19:02:50.834882+0000","last_change":"2024-09-20T19:01:40.961449+0000","last_active":"2024-09-20T19:02:50.834882+0000","last_peered":"2024-09-20T19:02:50.834882+0000","last_clean":"2024-09-20T19:02:50.834882+0000","last_became_active":"2024-09-20T18:49:49.957582+0000","last_became_peered":"2024-09-20T18:49:49.957582+0000","last_unstale":"2024-09-20T19:02:50.834882+0000","last_undegraded":"2024-09-20T19:02:50.834882+0000","last_fullsized":"2024-09-20T19:02:50.834882+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:40.961363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:40.961363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":4229890048,"num_objects":64543,"num_object_clones":0,"num_object_copies":193629,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64543,"num_write_kb":4130752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12750815232,"data_stored":12750815232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64543,"ondisk_log_size":64543,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1185,"internal_metadata":0},"log_size":78,"ondisk_log_size":78,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1115,"num_read_kb":872,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":2244,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":683,"ondisk_log_size":683,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":779,"num_read_kb":774,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266849,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1443032,"kb_used_data":992584,"kb_used_omap":1,"kb_used_meta":450430,"kb_avail":92302120,"statfs":{"total":95995035648,"available":94517370880,"internally_reserved":0,"allocated":1016406016,"data_stored":1015874161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":461240778},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528488,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2170708,"kb_used_data":1806272,"kb_used_omap":1,"kb_used_meta":364350,"kb_avail":91574444,"statfs":{"total":95995035648,"available":93772230656,"internally_reserved":0,"allocated":1849622528,"data_stored":1849105160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":373094854},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757421,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1783060,"kb_used_data":1286784,"kb_used_omap":1,"kb_used_meta":496254,"kb_avail":91962092,"statfs":{"total":95995035648,"available":94169182208,"internally_reserved":0,"allocated":1317666816,"data_stored":1317149263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":508164549},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":304,"seq":1305670058003,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2169316,"kb_used_data":2030676,"kb_used_omap":5,"kb_used_meta":138618,"kb_avail":91575836,"statfs":{"total":95995035648,"available":93773656064,"internally_reserved":0,"allocated":2079412224,"data_stored":2078868486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5633,"internal_metadata":141945343},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182585,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1877304,"kb_used_data":1304680,"kb_used_omap":1,"kb_used_meta":572606,"kb_avail":91867848,"statfs":{"total":95995035648,"available":94072676352,"internally_reserved":0,"allocated":1335992320,"data_stored":1335440760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":586348999},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411518,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2047080,"kb_used_data":1538320,"kb_used_omap":1,"kb_used_meta":508734,"kb_avail":91698072,"statfs":{"total":95995035648,"available":93898825728,"internally_reserved":0,"allocated":1575239680,"data_stored":1574713179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":520944069},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575044,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1309144,"kb_used_data":790656,"kb_used_omap":1,"kb_used_meta":518462,"kb_avail":92436008,"statfs":{"total":95995035648,"available":94654472192,"internally_reserved":0,"allocated":809631744,"data_stored":809057275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":530905541},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738570,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3163956,"kb_used_data":2613536,"kb_used_omap":1,"kb_used_meta":550398,"kb_avail":90581196,"statfs":{"total":95995035648,"available":92755144704,"internally_reserved":0,"allocated":2676260864,"data_stored":2675717943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":563608005},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1185,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2693267456,"data_stored":2693267456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":810811392,"data_stored":810811392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1590427648,"data_stored":1590427648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1345585152,"data_stored":1345585152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2090205184,"data_stored":2090205184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1329332224,"data_stored":1329332224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1857814528,"data_stored":1857814528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1033371648,"data_stored":1033371648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T19:02:55.041 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-09-20T19:02:55.041 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T19:02:55.041 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-09-20T19:02:55.041 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-20T19:02:55.041 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-20T19:02:55.041 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.7 config set filestore_inject_stall 3 2024-09-20T19:02:55.044 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:55.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:55.045 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:55.146 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:55.216 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:02:55.224 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:02:55.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:55.234 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:55.234 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:55.335 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:55.357 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-20T19:02:55.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:55.380 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:55.380 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:55.433 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:02:55.481 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:55.520 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:55.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:55.521 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:55.622 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:55.628 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:55 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1948522918' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:02:55.628 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:55 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T19:02:55.628 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:55 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T19:02:55.628 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:55 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:02:55.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:55.667 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:55.667 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:55.768 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:55.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:55 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1948522918' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:02:55.794 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:55 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T19:02:55.794 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:55 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T19:02:55.794 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:55 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:02:55.794 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:55 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1948522918' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:02:55.794 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:55 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T19:02:55.794 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:55 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T19:02:55.794 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:55 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:02:55.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:55.806 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:55.806 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:55.876 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 36 16 70683 70667 122.646 73.9375 0.00733686 0.00814865 2024-09-20T19:02:55.907 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:55.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:55.948 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:55.948 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:56.049 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:02:56.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:56.090 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:56.090 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:02:56.127 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T19:02:56.127 INFO:teuthology.orchestra.run.smithi142.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-20T19:02:56.127 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T19:02:56.191 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:56.225 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:56.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:56.228 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:56.330 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:02:56.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:56.367 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:56.368 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:02:56.470 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:56.497 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T19:02:56.498 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:02:56.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:56.506 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:56.506 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:56.607 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:56.633 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:56 smithi142 ceph-mon[32508]: pgmap v654: 148 pgs: 148 active+clean; 3.9 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 853 B/s rd, 74 MiB/s wr, 1.19k op/s 2024-09-20T19:02:56.633 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:56 smithi142 ceph-mon[32508]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:02:56.633 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:56 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1948522918' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T19:02:56.633 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:56 smithi142 ceph-mon[32508]: osdmap e346: 8 total, 8 up, 8 in 2024-09-20T19:02:56.646 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:56.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:56.647 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:56.748 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:02:56.784 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:56.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:56.785 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:02:56.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:56 smithi083 ceph-mon[29421]: pgmap v654: 148 pgs: 148 active+clean; 3.9 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 853 B/s rd, 74 MiB/s wr, 1.19k op/s 2024-09-20T19:02:56.794 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:56 smithi083 ceph-mon[29421]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:02:56.794 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:56 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1948522918' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T19:02:56.794 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:56 smithi083 ceph-mon[29421]: osdmap e346: 8 total, 8 up, 8 in 2024-09-20T19:02:56.794 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:56 smithi083 ceph-mon[36260]: pgmap v654: 148 pgs: 148 active+clean; 3.9 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 853 B/s rd, 74 MiB/s wr, 1.19k op/s 2024-09-20T19:02:56.794 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:56 smithi083 ceph-mon[36260]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:02:56.794 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:56 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1948522918' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T19:02:56.794 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:56 smithi083 ceph-mon[36260]: osdmap e346: 8 total, 8 up, 8 in 2024-09-20T19:02:56.876 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 37 16 72059 72043 121.655 86 0.00470779 0.00821492 2024-09-20T19:02:56.886 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:02:56.928 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:56.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:56.929 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:02:57.029 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:57.062 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:57.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:57.063 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:57.164 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:02:57.195 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:57.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:57.196 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:02:57.224 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_historic_ops 2024-09-20T19:02:57.263 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:02:57 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:02:57.297 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:02:57.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:57.346 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:57.347 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:02:57.447 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:02:57.480 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:57.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:57.481 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:02:57.582 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:57.617 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:57.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:57.618 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:57.719 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:57.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:57.768 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:57.768 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:57.833 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:02:57.833 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:02:57.844 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:02:57.868 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:02:57.870 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:02:57.870 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:02:57.877 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 38 16 73365 73349 120.601 81.625 0.0229084 0.00828645 2024-09-20T19:02:57.909 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:02:57.909 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:02:57.916 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:57.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:57.917 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:02:57.950 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:02:57.950 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:02:57.987 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:02:57.988 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:02:58.018 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:58.023 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:02:58.023 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:02:58.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:58.052 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:58.052 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:58.058 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:02:58.058 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:02:58.090 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:02:58.090 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:02:58.153 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:58.176 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:58.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:58.178 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:58.279 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:02:58.312 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:58.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:58.313 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:02:58.414 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:58.448 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T19:02:58.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:58.450 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:58.450 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:58.551 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:58.581 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:58.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:58.582 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:58.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:58 smithi083 ceph-mon[29421]: pgmap v656: 148 pgs: 148 active+clean; 4.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.6 KiB/s rd, 120 MiB/s wr, 1.93k op/s 2024-09-20T19:02:58.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:58 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:02:58.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:02:58 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2139012394' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:02:58.655 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:58 smithi083 ceph-mon[36260]: pgmap v656: 148 pgs: 148 active+clean; 4.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.6 KiB/s rd, 120 MiB/s wr, 1.93k op/s 2024-09-20T19:02:58.655 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:58 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:02:58.655 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:02:58 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2139012394' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:02:58.656 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T19:02:58.656 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-20T19:02:58.656 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-20T19:02:58.656 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-20T19:02:58.656 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.656 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:218173 8.f 8:fc626e9a:::benchmark_data_smithi083_97422_object102082:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-20T19:02:58.656 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:58:01.879894+0000", 2024-09-20T19:02:58.656 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 296.76736467699999, 2024-09-20T19:02:58.656 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.22879112400000001, 2024-09-20T19:02:58.657 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:58.657 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:58.657 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:02:58.657 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:02:58.657 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T19:02:58.657 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T19:02:58.657 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 218173 2024-09-20T19:02:58.657 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.657 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:58.657 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.657 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:58.657 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.879894+0000", 2024-09-20T19:02:58.657 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:58.657 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.658 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.658 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:58.658 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.879894+0000", 2024-09-20T19:02:58.658 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:58.658 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.658 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.658 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:58.658 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.879895+0000", 2024-09-20T19:02:58.658 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3600000000000003e-07 2024-09-20T19:02:58.658 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.658 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.658 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:58.659 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.879899+0000", 2024-09-20T19:02:58.659 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3500000000000001e-06 2024-09-20T19:02:58.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.659 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.659 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:58.659 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.879907+0000", 2024-09-20T19:02:58.659 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.943e-06 2024-09-20T19:02:58.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.659 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.659 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:58.659 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.879913+0000", 2024-09-20T19:02:58.659 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3910000000000003e-06 2024-09-20T19:02:58.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.659 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.660 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:58.660 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.095128+0000", 2024-09-20T19:02:58.660 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.21521499199999999 2024-09-20T19:02:58.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.660 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.660 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:58.660 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.095243+0000", 2024-09-20T19:02:58.660 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011465599999999999 2024-09-20T19:02:58.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.660 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.660 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:02:58.660 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.095357+0000", 2024-09-20T19:02:58.660 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000114868 2024-09-20T19:02:58.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.660 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.661 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:58.661 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.104639+0000", 2024-09-20T19:02:58.661 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0092811200000000003 2024-09-20T19:02:58.661 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.661 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.661 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:02:58.661 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.105329+0000", 2024-09-20T19:02:58.661 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000690195 2024-09-20T19:02:58.661 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.661 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.661 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:58.661 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.108593+0000", 2024-09-20T19:02:58.661 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003263967 2024-09-20T19:02:58.661 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.662 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.662 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:02:58.662 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.108646+0000", 2024-09-20T19:02:58.662 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3495999999999997e-05 2024-09-20T19:02:58.662 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.662 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.662 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:58.662 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.108686+0000", 2024-09-20T19:02:58.662 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9409999999999997e-05 2024-09-20T19:02:58.662 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:58.662 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:58.662 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:58.662 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.662 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.663 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:50423 9.1 9:8787b2bd:::benchmark_data_smithi083_124552_object50422:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-20T19:02:58.663 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:59:04.093296+0000", 2024-09-20T19:02:58.663 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 234.553963418, 2024-09-20T19:02:58.663 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.136410235, 2024-09-20T19:02:58.663 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:58.663 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:58.663 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:02:58.663 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:02:58.663 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:02:58.663 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:02:58.663 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 50423 2024-09-20T19:02:58.663 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.663 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:58.663 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.664 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:58.664 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.093296+0000", 2024-09-20T19:02:58.664 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:58.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.664 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.664 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:58.664 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.093296+0000", 2024-09-20T19:02:58.664 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:58.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.665 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.665 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:58.665 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.093297+0000", 2024-09-20T19:02:58.665 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.37e-06 2024-09-20T19:02:58.665 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.665 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.665 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:58.665 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.093316+0000", 2024-09-20T19:02:58.665 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8660000000000001e-05 2024-09-20T19:02:58.665 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.665 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.665 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:58.666 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.093342+0000", 2024-09-20T19:02:58.666 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6103999999999999e-05 2024-09-20T19:02:58.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.666 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.666 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:58.666 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.093347+0000", 2024-09-20T19:02:58.666 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0490000000000003e-06 2024-09-20T19:02:58.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.666 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.666 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:58.666 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.093482+0000", 2024-09-20T19:02:58.666 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000135449 2024-09-20T19:02:58.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.667 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.667 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:02:58.667 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.093490+0000", 2024-09-20T19:02:58.667 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0469999999999994e-06 2024-09-20T19:02:58.667 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.667 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.667 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:58.667 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.067142+0000", 2024-09-20T19:02:58.667 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97365191799999995 2024-09-20T19:02:58.667 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.667 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.667 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:02:58.667 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.067153+0000", 2024-09-20T19:02:58.668 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0559e-05 2024-09-20T19:02:58.668 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.668 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.668 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:58.668 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.215644+0000", 2024-09-20T19:02:58.668 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.148490633 2024-09-20T19:02:58.668 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.668 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.668 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:58.668 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.215772+0000", 2024-09-20T19:02:58.668 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012840799999999999 2024-09-20T19:02:58.668 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.668 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.668 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 6,7", 2024-09-20T19:02:58.668 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.215927+0000", 2024-09-20T19:02:58.669 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000155315 2024-09-20T19:02:58.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.669 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.669 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:58.669 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.223137+0000", 2024-09-20T19:02:58.669 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0072102049999999999 2024-09-20T19:02:58.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.669 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.669 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:58.669 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.223182+0000", 2024-09-20T19:02:58.669 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4158e-05 2024-09-20T19:02:58.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.669 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.669 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:02:58.669 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.227203+0000", 2024-09-20T19:02:58.670 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0040211530000000004 2024-09-20T19:02:58.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.670 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.670 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:02:58.670 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.227258+0000", 2024-09-20T19:02:58.670 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5050999999999999e-05 2024-09-20T19:02:58.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.670 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.670 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:58.670 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.229706+0000", 2024-09-20T19:02:58.670 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0024481559999999999 2024-09-20T19:02:58.670 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:58.670 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:58.671 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:58.671 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.671 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.671 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:50474 9.1 9:8f4be195:::benchmark_data_smithi083_124552_object50473:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-20T19:02:58.671 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:59:04.131531+0000", 2024-09-20T19:02:58.671 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 234.51572850100001, 2024-09-20T19:02:58.671 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.101374029, 2024-09-20T19:02:58.671 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:58.671 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:58.671 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:02:58.671 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:02:58.671 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:02:58.671 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:02:58.671 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 50474 2024-09-20T19:02:58.672 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.672 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:58.672 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.672 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:58.672 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.131531+0000", 2024-09-20T19:02:58.672 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:58.672 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.672 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.672 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:58.672 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.131531+0000", 2024-09-20T19:02:58.672 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:58.672 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.672 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.673 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:58.673 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.131532+0000", 2024-09-20T19:02:58.673 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5489999999999999e-06 2024-09-20T19:02:58.673 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.673 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.673 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:58.673 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.131552+0000", 2024-09-20T19:02:58.673 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9630999999999999e-05 2024-09-20T19:02:58.673 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.673 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.673 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:58.673 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.131568+0000", 2024-09-20T19:02:58.673 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6606999999999999e-05 2024-09-20T19:02:58.673 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.674 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.674 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:58.674 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.131577+0000", 2024-09-20T19:02:58.674 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7879999999999995e-06 2024-09-20T19:02:58.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.674 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.674 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:58.674 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.131644+0000", 2024-09-20T19:02:58.674 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7180000000000004e-05 2024-09-20T19:02:58.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.674 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.674 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:02:58.674 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.131654+0000", 2024-09-20T19:02:58.674 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2390000000000008e-06 2024-09-20T19:02:58.675 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.675 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.675 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:58.675 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.067311+0000", 2024-09-20T19:02:58.675 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.935657501 2024-09-20T19:02:58.675 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.675 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.675 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:02:58.675 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.067320+0000", 2024-09-20T19:02:58.675 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9779999999999994e-06 2024-09-20T19:02:58.675 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.675 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.675 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:58.676 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.220228+0000", 2024-09-20T19:02:58.676 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1529083090000001 2024-09-20T19:02:58.676 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.676 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.676 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:58.676 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.220362+0000", 2024-09-20T19:02:58.676 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013401 2024-09-20T19:02:58.676 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.676 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.676 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 6,7", 2024-09-20T19:02:58.676 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.220523+0000", 2024-09-20T19:02:58.676 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000160247 2024-09-20T19:02:58.676 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.676 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.677 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:02:58.677 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.229718+0000", 2024-09-20T19:02:58.677 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0091949860000000005 2024-09-20T19:02:58.677 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.677 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.677 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:58.677 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.229797+0000", 2024-09-20T19:02:58.677 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9443999999999994e-05 2024-09-20T19:02:58.677 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.677 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.677 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:58.677 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.230036+0000", 2024-09-20T19:02:58.677 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000239072 2024-09-20T19:02:58.677 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.678 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.678 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:02:58.678 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.230111+0000", 2024-09-20T19:02:58.678 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4411000000000003e-05 2024-09-20T19:02:58.678 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.678 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.678 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:58.678 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.232905+0000", 2024-09-20T19:02:58.678 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0027940769999999998 2024-09-20T19:02:58.678 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:58.678 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:58.679 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:58.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.679 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.679 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:50506 9.1 9:80ae6680:::benchmark_data_smithi083_124552_object50505:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-20T19:02:58.679 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:59:04.155840+0000", 2024-09-20T19:02:58.679 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 234.49141869799999, 2024-09-20T19:02:58.679 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.082518146, 2024-09-20T19:02:58.679 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:58.679 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:58.679 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:02:58.679 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:02:58.679 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:02:58.679 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:02:58.680 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 50506 2024-09-20T19:02:58.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.680 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:58.680 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.680 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:58.680 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.155840+0000", 2024-09-20T19:02:58.680 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:58.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.680 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.680 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:58.680 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.155840+0000", 2024-09-20T19:02:58.680 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:58.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.680 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.680 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:58.681 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.155842+0000", 2024-09-20T19:02:58.681 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1090000000000001e-06 2024-09-20T19:02:58.681 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.681 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.681 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:58.681 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.155869+0000", 2024-09-20T19:02:58.681 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7801999999999999e-05 2024-09-20T19:02:58.681 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.681 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.681 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:58.681 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.155880+0000", 2024-09-20T19:02:58.681 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1059e-05 2024-09-20T19:02:58.681 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.681 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.682 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:58.682 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.155888+0000", 2024-09-20T19:02:58.682 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7559999999999999e-06 2024-09-20T19:02:58.682 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.682 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.682 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:58.682 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.155955+0000", 2024-09-20T19:02:58.682 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7022999999999998e-05 2024-09-20T19:02:58.682 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.682 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.682 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:02:58.682 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.155965+0000", 2024-09-20T19:02:58.682 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3710000000000006e-06 2024-09-20T19:02:58.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.683 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.683 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:58.683 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.067356+0000", 2024-09-20T19:02:58.683 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.91139177299999996 2024-09-20T19:02:58.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.683 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.683 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:02:58.683 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.067364+0000", 2024-09-20T19:02:58.683 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.6000000000000001e-06 2024-09-20T19:02:58.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.683 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.683 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:58.684 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.223316+0000", 2024-09-20T19:02:58.684 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1559515810000001 2024-09-20T19:02:58.684 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.684 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.684 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:58.684 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.223488+0000", 2024-09-20T19:02:58.684 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017242399999999999 2024-09-20T19:02:58.684 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:02:58.686 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.686 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.686 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 6,7", 2024-09-20T19:02:58.686 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.223628+0000", 2024-09-20T19:02:58.686 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013994100000000001 2024-09-20T19:02:58.686 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.686 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.687 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:58.687 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.229891+0000", 2024-09-20T19:02:58.687 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0062633239999999998 2024-09-20T19:02:58.687 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.687 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.687 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:02:58.687 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.229911+0000", 2024-09-20T19:02:58.687 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9910000000000001e-05 2024-09-20T19:02:58.687 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.687 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.687 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:58.687 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.235269+0000", 2024-09-20T19:02:58.687 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0053577429999999999 2024-09-20T19:02:58.687 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.688 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.688 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:02:58.688 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.235358+0000", 2024-09-20T19:02:58.688 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9536999999999993e-05 2024-09-20T19:02:58.688 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.688 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.688 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:58.688 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.238359+0000", 2024-09-20T19:02:58.688 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003000193 2024-09-20T19:02:58.688 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:58.689 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:58.689 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:58.689 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.689 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.689 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:50517 9.1 9:8bbc6b9c:::benchmark_data_smithi083_124552_object50516:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-20T19:02:58.689 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:59:04.167833+0000", 2024-09-20T19:02:58.690 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 234.47942626099999, 2024-09-20T19:02:58.690 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0735188230000001, 2024-09-20T19:02:58.690 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:58.690 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:58.690 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:02:58.690 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:02:58.690 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:02:58.691 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:02:58.691 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 50517 2024-09-20T19:02:58.691 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.691 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:58.691 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.691 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:58.691 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.167833+0000", 2024-09-20T19:02:58.692 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:58.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.692 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.692 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:58.692 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.167833+0000", 2024-09-20T19:02:58.692 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:58.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.692 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.693 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:58.693 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.167835+0000", 2024-09-20T19:02:58.693 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.821e-06 2024-09-20T19:02:58.693 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.693 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.693 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:58.693 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.167864+0000", 2024-09-20T19:02:58.694 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9714e-05 2024-09-20T19:02:58.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.694 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.694 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:58.694 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.167883+0000", 2024-09-20T19:02:58.694 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8788999999999999e-05 2024-09-20T19:02:58.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.695 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.695 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:58.695 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.167891+0000", 2024-09-20T19:02:58.695 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.1669999999999999e-06 2024-09-20T19:02:58.695 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.695 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.696 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:58.696 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.167959+0000", 2024-09-20T19:02:58.696 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7779000000000003e-05 2024-09-20T19:02:58.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.696 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.696 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:02:58.696 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.167968+0000", 2024-09-20T19:02:58.696 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9630000000000004e-06 2024-09-20T19:02:58.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.697 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.697 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:58.697 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.068252+0000", 2024-09-20T19:02:58.697 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.90028381000000002 2024-09-20T19:02:58.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.697 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.697 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:02:58.697 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.068262+0000", 2024-09-20T19:02:58.697 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0278000000000001e-05 2024-09-20T19:02:58.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.697 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.697 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:58.697 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.225517+0000", 2024-09-20T19:02:58.698 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.157254279 2024-09-20T19:02:58.698 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.698 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.698 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:58.698 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.225656+0000", 2024-09-20T19:02:58.698 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013903900000000001 2024-09-20T19:02:58.698 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.698 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.698 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 6,7", 2024-09-20T19:02:58.698 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.225821+0000", 2024-09-20T19:02:58.698 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016562099999999999 2024-09-20T19:02:58.698 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.698 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.699 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:02:58.699 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.229921+0000", 2024-09-20T19:02:58.699 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.004099742 2024-09-20T19:02:58.699 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.699 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.699 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:58.699 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.229977+0000", 2024-09-20T19:02:58.699 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6523000000000001e-05 2024-09-20T19:02:58.699 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.699 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.699 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:58.699 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.238409+0000", 2024-09-20T19:02:58.699 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0084314219999999992 2024-09-20T19:02:58.699 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.700 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.700 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:02:58.700 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.238463+0000", 2024-09-20T19:02:58.700 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4308000000000002e-05 2024-09-20T19:02:58.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.700 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.700 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:58.700 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.241352+0000", 2024-09-20T19:02:58.700 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0028885680000000002 2024-09-20T19:02:58.700 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:58.700 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:58.700 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:58.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.701 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.701 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:118977 9.8 9:18fffd38:::benchmark_data_smithi083_124552_object4830:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e286)", 2024-09-20T19:02:58.701 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:16.045469+0000", 2024-09-20T19:02:58.701 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 162.601790628, 2024-09-20T19:02:58.701 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.233874096, 2024-09-20T19:02:58.701 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:58.701 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:58.701 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:02:58.701 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:02:58.701 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:02:58.701 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:02:58.701 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 118977 2024-09-20T19:02:58.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.701 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:58.702 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.702 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:58.702 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.045469+0000", 2024-09-20T19:02:58.702 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:58.702 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.702 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.702 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:58.702 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.045469+0000", 2024-09-20T19:02:58.702 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:58.702 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.702 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.702 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:58.702 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.045470+0000", 2024-09-20T19:02:58.702 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.359e-06 2024-09-20T19:02:58.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.703 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.703 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:58.703 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.045474+0000", 2024-09-20T19:02:58.703 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0609999999999997e-06 2024-09-20T19:02:58.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.703 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.703 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:58.703 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.045484+0000", 2024-09-20T19:02:58.703 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0332999999999999e-05 2024-09-20T19:02:58.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.703 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.703 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:58.704 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.045489+0000", 2024-09-20T19:02:58.704 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1869999999999996e-06 2024-09-20T19:02:58.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.704 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.704 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:58.704 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.045547+0000", 2024-09-20T19:02:58.704 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7046999999999998e-05 2024-09-20T19:02:58.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.704 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.704 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:58.704 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.045681+0000", 2024-09-20T19:02:58.704 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013437200000000001 2024-09-20T19:02:58.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.705 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.705 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T19:02:58.705 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.045767+0000", 2024-09-20T19:02:58.705 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6248000000000005e-05 2024-09-20T19:02:58.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.705 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.705 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:02:58.705 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.047811+0000", 2024-09-20T19:02:58.705 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0020440440000000001 2024-09-20T19:02:58.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.705 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.705 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:58.705 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.048095+0000", 2024-09-20T19:02:58.705 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00028369700000000002 2024-09-20T19:02:58.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.706 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.706 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:58.706 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.279270+0000", 2024-09-20T19:02:58.706 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.23117499899999999 2024-09-20T19:02:58.706 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.706 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.706 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:02:58.706 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.279311+0000", 2024-09-20T19:02:58.706 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0840000000000002e-05 2024-09-20T19:02:58.706 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.706 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.706 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:58.706 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.279343+0000", 2024-09-20T19:02:58.707 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1909000000000002e-05 2024-09-20T19:02:58.707 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:58.707 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:58.707 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:58.707 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.707 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.707 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:118986 9.8 9:12412698:::benchmark_data_smithi083_124552_object4839:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e286)", 2024-09-20T19:02:58.707 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:16.058237+0000", 2024-09-20T19:02:58.707 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 162.58902242799999, 2024-09-20T19:02:58.707 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.221293453, 2024-09-20T19:02:58.707 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:58.707 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:58.707 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:02:58.707 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:02:58.708 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:02:58.708 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:02:58.708 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 118986 2024-09-20T19:02:58.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.708 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:58.708 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.708 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:58.708 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.058237+0000", 2024-09-20T19:02:58.708 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:58.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.708 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.708 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:58.708 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.058237+0000", 2024-09-20T19:02:58.708 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:58.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.709 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.709 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:58.709 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.058238+0000", 2024-09-20T19:02:58.709 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1799999999999999e-06 2024-09-20T19:02:58.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.709 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.709 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:58.709 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.058242+0000", 2024-09-20T19:02:58.709 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3880000000000002e-06 2024-09-20T19:02:58.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.709 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.709 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:58.709 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.058251+0000", 2024-09-20T19:02:58.709 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0420000000000008e-06 2024-09-20T19:02:58.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.710 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.710 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:58.710 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.058256+0000", 2024-09-20T19:02:58.710 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8960000000000002e-06 2024-09-20T19:02:58.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.710 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.710 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:58.710 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.058309+0000", 2024-09-20T19:02:58.710 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2902999999999998e-05 2024-09-20T19:02:58.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.710 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.710 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:58.710 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.058410+0000", 2024-09-20T19:02:58.711 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010079200000000001 2024-09-20T19:02:58.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.711 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.711 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T19:02:58.711 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.058476+0000", 2024-09-20T19:02:58.711 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6050000000000006e-05 2024-09-20T19:02:58.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.711 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.711 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:02:58.711 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.062309+0000", 2024-09-20T19:02:58.711 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003832701 2024-09-20T19:02:58.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.711 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.712 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:58.712 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.062336+0000", 2024-09-20T19:02:58.712 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7426000000000001e-05 2024-09-20T19:02:58.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.712 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.712 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:58.712 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.279409+0000", 2024-09-20T19:02:58.712 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.21707326399999999 2024-09-20T19:02:58.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.712 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.712 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:02:58.712 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.279490+0000", 2024-09-20T19:02:58.712 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0500999999999994e-05 2024-09-20T19:02:58.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.713 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.713 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:58.713 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.279530+0000", 2024-09-20T19:02:58.713 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0309999999999999e-05 2024-09-20T19:02:58.713 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:58.713 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:58.713 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:58.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.713 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.713 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:124578 9.2 9:422abb8a:::benchmark_data_smithi083_124552_object10431:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e289)", 2024-09-20T19:02:58.713 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:20.946829+0000", 2024-09-20T19:02:58.713 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 157.70043028200001, 2024-09-20T19:02:58.713 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.33414796600000002, 2024-09-20T19:02:58.713 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:58.714 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:58.714 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:02:58.714 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:02:58.714 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:02:58.714 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:02:58.714 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 124578 2024-09-20T19:02:58.714 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.714 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:58.714 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.714 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:58.714 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.946829+0000", 2024-09-20T19:02:58.714 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:58.714 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.715 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.715 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:58.715 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.946829+0000", 2024-09-20T19:02:58.715 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:58.715 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.715 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.715 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:58.715 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.946830+0000", 2024-09-20T19:02:58.715 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2299999999999999e-07 2024-09-20T19:02:58.715 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.715 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.715 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:58.715 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.946834+0000", 2024-09-20T19:02:58.715 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1189999999999999e-06 2024-09-20T19:02:58.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.716 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.716 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:58.716 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.946854+0000", 2024-09-20T19:02:58.716 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9754000000000001e-05 2024-09-20T19:02:58.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.716 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.716 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:58.716 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.946861+0000", 2024-09-20T19:02:58.716 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7719999999999994e-06 2024-09-20T19:02:58.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.716 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.716 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:58.716 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.951540+0000", 2024-09-20T19:02:58.716 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.004678354 2024-09-20T19:02:58.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.717 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.717 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:58.717 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.951640+0000", 2024-09-20T19:02:58.717 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000100055 2024-09-20T19:02:58.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.717 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.717 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:02:58.717 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.951741+0000", 2024-09-20T19:02:58.718 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010086600000000001 2024-09-20T19:02:58.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.718 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.718 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:02:58.718 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.954721+0000", 2024-09-20T19:02:58.718 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0029804609999999998 2024-09-20T19:02:58.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.718 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.718 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:58.718 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.956254+0000", 2024-09-20T19:02:58.718 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001532683 2024-09-20T19:02:58.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.719 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.719 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:58.719 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.280892+0000", 2024-09-20T19:02:58.719 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.32463842100000001 2024-09-20T19:02:58.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.719 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.719 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:02:58.719 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.280935+0000", 2024-09-20T19:02:58.719 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2592999999999997e-05 2024-09-20T19:02:58.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.719 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.719 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:58.719 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.280977+0000", 2024-09-20T19:02:58.719 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1965000000000002e-05 2024-09-20T19:02:58.720 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:58.720 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:58.720 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:58.720 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.720 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.720 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:124596 9.2 9:42d01fa2:::benchmark_data_smithi083_124552_object10449:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e289)", 2024-09-20T19:02:58.720 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:20.962657+0000", 2024-09-20T19:02:58.720 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 157.68460213399999, 2024-09-20T19:02:58.720 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.32748537700000002, 2024-09-20T19:02:58.720 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:58.720 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:58.720 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:02:58.720 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:02:58.721 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:02:58.721 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:02:58.721 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 124596 2024-09-20T19:02:58.721 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.721 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:58.721 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.721 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:58.721 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.962657+0000", 2024-09-20T19:02:58.721 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:58.721 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.721 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.721 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:58.721 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.962657+0000", 2024-09-20T19:02:58.721 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:58.721 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.722 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.722 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:58.722 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.962658+0000", 2024-09-20T19:02:58.722 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4020000000000001e-06 2024-09-20T19:02:58.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.722 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.722 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:58.722 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.962663+0000", 2024-09-20T19:02:58.722 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0590000000000002e-06 2024-09-20T19:02:58.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.722 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.722 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:58.723 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.962678+0000", 2024-09-20T19:02:58.723 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4469e-05 2024-09-20T19:02:58.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.723 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.723 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:58.723 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.962685+0000", 2024-09-20T19:02:58.723 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3549999999999999e-06 2024-09-20T19:02:58.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.723 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.723 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:58.723 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.962732+0000", 2024-09-20T19:02:58.723 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6502999999999998e-05 2024-09-20T19:02:58.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.723 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.724 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:58.724 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.962831+0000", 2024-09-20T19:02:58.724 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9489999999999998e-05 2024-09-20T19:02:58.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.724 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.724 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:02:58.724 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.962910+0000", 2024-09-20T19:02:58.724 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8527000000000002e-05 2024-09-20T19:02:58.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.724 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.724 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:02:58.724 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.973808+0000", 2024-09-20T19:02:58.724 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010898134 2024-09-20T19:02:58.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.724 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.725 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:58.725 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.973889+0000", 2024-09-20T19:02:58.725 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0641000000000003e-05 2024-09-20T19:02:58.725 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.725 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.725 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:58.725 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.290050+0000", 2024-09-20T19:02:58.725 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.31616166899999998 2024-09-20T19:02:58.725 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.725 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.725 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:02:58.725 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.290100+0000", 2024-09-20T19:02:58.725 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0179999999999997e-05 2024-09-20T19:02:58.725 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.726 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.726 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:58.726 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.290142+0000", 2024-09-20T19:02:58.726 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1947999999999999e-05 2024-09-20T19:02:58.726 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:58.726 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:58.726 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:58.726 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.726 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.726 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:124603 9.2 9:4d3bc5dd:::benchmark_data_smithi083_124552_object10456:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e289)", 2024-09-20T19:02:58.726 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:20.964654+0000", 2024-09-20T19:02:58.726 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 157.68260485299999, 2024-09-20T19:02:58.727 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.32571787499999999, 2024-09-20T19:02:58.727 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:58.727 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:58.727 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:02:58.727 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:02:58.727 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:02:58.727 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:02:58.727 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 124603 2024-09-20T19:02:58.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.727 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:58.727 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.727 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:58.727 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.964654+0000", 2024-09-20T19:02:58.728 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:58.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.728 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.728 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:58.728 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.964654+0000", 2024-09-20T19:02:58.728 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:58.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.729 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.729 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:58.729 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.964655+0000", 2024-09-20T19:02:58.729 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6099999999999999e-07 2024-09-20T19:02:58.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.729 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.730 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:58.730 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.964698+0000", 2024-09-20T19:02:58.730 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2778999999999998e-05 2024-09-20T19:02:58.730 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.730 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.730 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:58.731 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.964707+0000", 2024-09-20T19:02:58.731 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0869999999999996e-06 2024-09-20T19:02:58.731 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.731 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.731 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:58.731 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.964713+0000", 2024-09-20T19:02:58.731 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3339999999999998e-06 2024-09-20T19:02:58.731 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.731 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.731 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:58.731 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.965135+0000", 2024-09-20T19:02:58.731 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00042182599999999998 2024-09-20T19:02:58.731 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.731 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.732 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:58.732 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.965229+0000", 2024-09-20T19:02:58.732 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4297999999999997e-05 2024-09-20T19:02:58.732 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.732 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.732 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:02:58.732 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.965326+0000", 2024-09-20T19:02:58.732 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6902000000000002e-05 2024-09-20T19:02:58.732 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.732 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.732 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:02:58.732 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.973814+0000", 2024-09-20T19:02:58.732 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0084871779999999997 2024-09-20T19:02:58.732 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.733 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.733 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:58.733 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.973930+0000", 2024-09-20T19:02:58.733 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011664599999999999 2024-09-20T19:02:58.733 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.733 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.733 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:58.733 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.290288+0000", 2024-09-20T19:02:58.733 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.31635755799999998 2024-09-20T19:02:58.733 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.733 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.733 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:02:58.733 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.290337+0000", 2024-09-20T19:02:58.733 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8948e-05 2024-09-20T19:02:58.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.734 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.734 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:58.734 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.290372+0000", 2024-09-20T19:02:58.734 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5457999999999998e-05 2024-09-20T19:02:58.734 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:58.734 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:58.734 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:58.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.734 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.734 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:124604 9.2 9:40e573dc:::benchmark_data_smithi083_124552_object10457:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e289)", 2024-09-20T19:02:58.734 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:20.964729+0000", 2024-09-20T19:02:58.734 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 157.68253032800001, 2024-09-20T19:02:58.735 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.32575654300000001, 2024-09-20T19:02:58.735 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:58.735 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:58.735 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:02:58.735 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:02:58.735 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:02:58.735 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:02:58.735 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 124604 2024-09-20T19:02:58.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.735 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:58.735 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.735 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:58.735 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.964729+0000", 2024-09-20T19:02:58.735 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:58.736 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.736 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.736 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:58.736 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.964729+0000", 2024-09-20T19:02:58.736 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:58.736 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.736 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.736 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:58.736 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.964729+0000", 2024-09-20T19:02:58.736 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6599999999999996e-07 2024-09-20T19:02:58.736 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.736 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.736 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:58.736 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.964732+0000", 2024-09-20T19:02:58.736 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9679999999999998e-06 2024-09-20T19:02:58.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.737 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.737 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:58.737 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.964738+0000", 2024-09-20T19:02:58.737 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6910000000000002e-06 2024-09-20T19:02:58.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.737 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.737 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:58.737 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.964744+0000", 2024-09-20T19:02:58.737 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5899999999999998e-06 2024-09-20T19:02:58.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.737 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.737 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:58.737 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.967815+0000", 2024-09-20T19:02:58.738 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0030715209999999998 2024-09-20T19:02:58.738 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.738 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.738 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:58.738 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.967939+0000", 2024-09-20T19:02:58.738 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012392100000000001 2024-09-20T19:02:58.738 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.738 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.738 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:02:58.738 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.968041+0000", 2024-09-20T19:02:58.738 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000102166 2024-09-20T19:02:58.738 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.739 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.739 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:58.739 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.973986+0000", 2024-09-20T19:02:58.739 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059442610000000002 2024-09-20T19:02:58.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.739 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.739 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:02:58.739 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.973997+0000", 2024-09-20T19:02:58.739 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1313e-05 2024-09-20T19:02:58.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.739 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.739 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:58.739 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.290407+0000", 2024-09-20T19:02:58.739 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.31641063699999999 2024-09-20T19:02:58.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.740 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.740 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:02:58.740 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.290452+0000", 2024-09-20T19:02:58.740 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4753999999999999e-05 2024-09-20T19:02:58.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.740 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.740 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:58.740 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.290485+0000", 2024-09-20T19:02:58.740 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3154999999999997e-05 2024-09-20T19:02:58.740 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:58.740 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:58.740 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:58.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.740 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.740 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:124610 9.2 9:4e613fc5:::benchmark_data_smithi083_124552_object10463:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e289)", 2024-09-20T19:02:58.741 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:20.970344+0000", 2024-09-20T19:02:58.741 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 157.676915108, 2024-09-20T19:02:58.741 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.32025382099999999, 2024-09-20T19:02:58.741 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:58.741 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:58.741 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:02:58.741 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:02:58.741 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:02:58.741 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:02:58.741 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 124610 2024-09-20T19:02:58.741 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.741 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:58.741 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.741 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:58.741 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.970344+0000", 2024-09-20T19:02:58.742 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:58.742 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.742 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.742 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:58.742 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.970344+0000", 2024-09-20T19:02:58.742 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:58.742 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.742 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.742 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:58.742 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.970345+0000", 2024-09-20T19:02:58.742 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5000000000000003e-07 2024-09-20T19:02:58.742 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.742 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.742 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:58.743 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.970347+0000", 2024-09-20T19:02:58.743 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6769999999999999e-06 2024-09-20T19:02:58.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.743 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.743 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:58.743 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.970354+0000", 2024-09-20T19:02:58.743 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6100000000000002e-06 2024-09-20T19:02:58.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.743 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.743 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:58.743 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.970359+0000", 2024-09-20T19:02:58.743 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5990000000000001e-06 2024-09-20T19:02:58.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.743 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.743 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:58.744 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.971408+0000", 2024-09-20T19:02:58.744 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0010481220000000001 2024-09-20T19:02:58.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.744 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.744 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:58.744 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.971499+0000", 2024-09-20T19:02:58.744 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1843999999999997e-05 2024-09-20T19:02:58.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.744 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.744 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:02:58.744 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.971597+0000", 2024-09-20T19:02:58.744 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8011999999999996e-05 2024-09-20T19:02:58.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.744 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.744 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:02:58.744 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.974098+0000", 2024-09-20T19:02:58.745 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00250016 2024-09-20T19:02:58.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.745 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.745 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:58.745 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.978935+0000", 2024-09-20T19:02:58.745 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.004836974 2024-09-20T19:02:58.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.745 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.745 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:58.745 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.290520+0000", 2024-09-20T19:02:58.745 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.31158548699999999 2024-09-20T19:02:58.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.745 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.745 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:02:58.746 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.290566+0000", 2024-09-20T19:02:58.746 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5519000000000002e-05 2024-09-20T19:02:58.746 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.746 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.746 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:58.746 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.290598+0000", 2024-09-20T19:02:58.746 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2267000000000001e-05 2024-09-20T19:02:58.746 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:58.746 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:58.746 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:58.746 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.746 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.746 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:124618 9.2 9:4c60c683:::benchmark_data_smithi083_124552_object10471:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e289)", 2024-09-20T19:02:58.746 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:20.974705+0000", 2024-09-20T19:02:58.747 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 157.67255380899999, 2024-09-20T19:02:58.747 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.31613734999999998, 2024-09-20T19:02:58.747 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:58.747 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:58.747 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:02:58.747 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:02:58.747 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:02:58.747 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:02:58.747 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 124618 2024-09-20T19:02:58.747 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.747 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:58.747 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.747 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:58.747 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.974705+0000", 2024-09-20T19:02:58.748 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:58.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.748 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:58.748 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.974705+0000", 2024-09-20T19:02:58.748 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:58.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.748 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:58.748 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.974707+0000", 2024-09-20T19:02:58.748 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4109999999999999e-06 2024-09-20T19:02:58.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.748 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.748 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:58.748 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.974711+0000", 2024-09-20T19:02:58.748 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5229999999999997e-06 2024-09-20T19:02:58.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.749 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.749 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:58.749 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.974720+0000", 2024-09-20T19:02:58.749 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.3829999999999992e-06 2024-09-20T19:02:58.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.749 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.749 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:58.749 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.974726+0000", 2024-09-20T19:02:58.749 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7039999999999997e-06 2024-09-20T19:02:58.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.749 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.749 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:58.749 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.974777+0000", 2024-09-20T19:02:58.749 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0785999999999998e-05 2024-09-20T19:02:58.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.750 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.750 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:58.750 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.974886+0000", 2024-09-20T19:02:58.750 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010864199999999999 2024-09-20T19:02:58.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.750 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.750 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:02:58.750 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.974983+0000", 2024-09-20T19:02:58.750 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7293000000000005e-05 2024-09-20T19:02:58.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.750 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.750 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:02:58.750 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.977705+0000", 2024-09-20T19:02:58.751 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0027215339999999998 2024-09-20T19:02:58.751 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.751 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.751 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:58.751 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.979236+0000", 2024-09-20T19:02:58.751 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0015310079999999999 2024-09-20T19:02:58.751 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.751 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.751 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:58.751 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.290703+0000", 2024-09-20T19:02:58.751 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.311467775 2024-09-20T19:02:58.752 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.752 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.752 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:02:58.752 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.290802+0000", 2024-09-20T19:02:58.752 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8282999999999994e-05 2024-09-20T19:02:58.752 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.752 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.752 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:58.752 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.290843+0000", 2024-09-20T19:02:58.752 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1007999999999998e-05 2024-09-20T19:02:58.752 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:58.752 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:58.752 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:58.753 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.753 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.753 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:124636 9.2 9:45855acf:::benchmark_data_smithi083_124552_object10489:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e289)", 2024-09-20T19:02:58.753 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:20.997591+0000", 2024-09-20T19:02:58.753 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 157.649668214, 2024-09-20T19:02:58.753 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.29766285199999998, 2024-09-20T19:02:58.753 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:58.753 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:58.753 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:02:58.754 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:02:58.754 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:02:58.754 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:02:58.754 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 124636 2024-09-20T19:02:58.754 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.754 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:58.755 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.755 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:58.755 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.997591+0000", 2024-09-20T19:02:58.755 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:58.755 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.755 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.755 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:58.756 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.997591+0000", 2024-09-20T19:02:58.756 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:58.756 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.756 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.756 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:58.756 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.997592+0000", 2024-09-20T19:02:58.756 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.6000000000000003e-07 2024-09-20T19:02:58.756 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.757 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.757 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:58.757 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.997595+0000", 2024-09-20T19:02:58.757 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6449999999999998e-06 2024-09-20T19:02:58.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.757 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.757 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:58.757 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.997604+0000", 2024-09-20T19:02:58.757 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2090000000000005e-06 2024-09-20T19:02:58.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.758 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.758 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:58.758 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.997608+0000", 2024-09-20T19:02:58.758 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8960000000000002e-06 2024-09-20T19:02:58.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.758 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.758 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:58.759 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.997646+0000", 2024-09-20T19:02:58.759 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7184000000000002e-05 2024-09-20T19:02:58.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.759 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.759 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:58.759 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.997718+0000", 2024-09-20T19:02:58.759 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1884000000000005e-05 2024-09-20T19:02:58.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.759 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.759 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:02:58.760 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.997784+0000", 2024-09-20T19:02:58.760 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6018000000000004e-05 2024-09-20T19:02:58.760 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.760 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.760 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:02:58.760 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.999669+0000", 2024-09-20T19:02:58.760 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001885954 2024-09-20T19:02:58.760 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.760 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.760 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:58.760 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.999896+0000", 2024-09-20T19:02:58.760 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022635999999999999 2024-09-20T19:02:58.760 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.760 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.760 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:58.761 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.294934+0000", 2024-09-20T19:02:58.761 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.29503834600000001 2024-09-20T19:02:58.761 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.761 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.761 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:02:58.761 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.295205+0000", 2024-09-20T19:02:58.761 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00027075200000000003 2024-09-20T19:02:58.761 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.761 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.761 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:58.761 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.295254+0000", 2024-09-20T19:02:58.761 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8844000000000002e-05 2024-09-20T19:02:58.761 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:58.761 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:58.762 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:58.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.762 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:142913 9.c 9:3d798970:::benchmark_data_smithi083_124552_object28766:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e291)", 2024-09-20T19:02:58.762 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:35.168460+0000", 2024-09-20T19:02:58.762 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 143.47879899500001, 2024-09-20T19:02:58.762 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.040925943, 2024-09-20T19:02:58.762 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:58.762 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:58.762 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-20T19:02:58.762 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:02:58.762 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:02:58.762 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:02:58.762 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 142913 2024-09-20T19:02:58.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.763 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:58.763 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.763 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:58.763 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.168460+0000", 2024-09-20T19:02:58.763 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:58.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.763 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.763 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:58.763 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.168460+0000", 2024-09-20T19:02:58.763 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:58.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.763 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.763 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:58.764 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.168462+0000", 2024-09-20T19:02:58.764 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9520000000000001e-06 2024-09-20T19:02:58.764 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.764 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.764 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:58.764 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.168469+0000", 2024-09-20T19:02:58.764 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6309999999999997e-06 2024-09-20T19:02:58.764 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.764 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.764 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:58.764 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.168485+0000", 2024-09-20T19:02:58.764 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5798e-05 2024-09-20T19:02:58.764 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.764 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.765 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:58.765 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.168492+0000", 2024-09-20T19:02:58.765 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9179999999999994e-06 2024-09-20T19:02:58.765 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.765 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.765 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:58.765 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.168833+0000", 2024-09-20T19:02:58.765 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00034100200000000003 2024-09-20T19:02:58.765 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.765 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.765 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:58.765 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.169144+0000", 2024-09-20T19:02:58.765 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00031083499999999997 2024-09-20T19:02:58.765 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.766 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.766 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4,7", 2024-09-20T19:02:58.766 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.169258+0000", 2024-09-20T19:02:58.766 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011411 2024-09-20T19:02:58.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.766 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.766 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:02:58.766 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.172245+0000", 2024-09-20T19:02:58.766 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002987048 2024-09-20T19:02:58.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.766 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.766 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:58.766 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.203777+0000", 2024-09-20T19:02:58.767 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.031531525999999997 2024-09-20T19:02:58.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.767 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.767 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:58.767 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.213031+0000", 2024-09-20T19:02:58.767 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0092539930000000003 2024-09-20T19:02:58.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.767 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.767 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:02:58.767 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.213039+0000", 2024-09-20T19:02:58.767 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4090000000000003e-06 2024-09-20T19:02:58.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.767 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.767 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:58.768 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.209326+0000", 2024-09-20T19:02:58.768 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.99628679899999995 2024-09-20T19:02:58.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.768 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.768 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:58.768 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.209386+0000", 2024-09-20T19:02:58.768 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9922e-05 2024-09-20T19:02:58.768 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:58.768 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:58.768 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:58.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.768 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.768 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:142917 9.c 9:32f2df36:::benchmark_data_smithi083_124552_object28770:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e291)", 2024-09-20T19:02:58.768 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:35.172691+0000", 2024-09-20T19:02:58.769 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 143.47456862600001, 2024-09-20T19:02:58.769 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0372109220000001, 2024-09-20T19:02:58.769 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:58.769 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:58.769 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-20T19:02:58.769 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:02:58.769 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:02:58.769 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:02:58.769 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 142917 2024-09-20T19:02:58.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.769 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:58.769 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.769 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:58.769 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.172691+0000", 2024-09-20T19:02:58.769 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:58.770 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.770 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.770 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:58.770 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.172691+0000", 2024-09-20T19:02:58.770 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:58.770 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.770 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.770 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:58.770 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.172692+0000", 2024-09-20T19:02:58.770 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.95e-06 2024-09-20T19:02:58.770 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.770 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.770 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:58.771 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.172701+0000", 2024-09-20T19:02:58.771 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0630000000000006e-06 2024-09-20T19:02:58.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.771 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:58.771 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.172714+0000", 2024-09-20T19:02:58.771 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3303e-05 2024-09-20T19:02:58.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.771 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:58.771 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.172723+0000", 2024-09-20T19:02:58.771 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6160000000000008e-06 2024-09-20T19:02:58.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.772 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:58.772 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.172857+0000", 2024-09-20T19:02:58.772 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000133829 2024-09-20T19:02:58.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.772 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.772 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:58.772 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.172975+0000", 2024-09-20T19:02:58.772 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000117224 2024-09-20T19:02:58.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.772 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.772 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4,7", 2024-09-20T19:02:58.772 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.173088+0000", 2024-09-20T19:02:58.772 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000113551 2024-09-20T19:02:58.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.773 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.773 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:02:58.773 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.202597+0000", 2024-09-20T19:02:58.773 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.029508818999999999 2024-09-20T19:02:58.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.773 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.773 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:58.773 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.203835+0000", 2024-09-20T19:02:58.773 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001238444 2024-09-20T19:02:58.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.773 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.773 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:58.773 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.215654+0000", 2024-09-20T19:02:58.773 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011818403999999999 2024-09-20T19:02:58.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.774 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:02:58.774 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.215699+0000", 2024-09-20T19:02:58.774 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5084999999999999e-05 2024-09-20T19:02:58.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.774 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:58.774 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.209843+0000", 2024-09-20T19:02:58.774 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.99414379500000005 2024-09-20T19:02:58.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.774 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:58.774 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.209901+0000", 2024-09-20T19:02:58.774 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8838999999999997e-05 2024-09-20T19:02:58.775 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:58.775 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:58.775 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:58.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.775 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:142919 9.c 9:3aa96e52:::benchmark_data_smithi083_124552_object28772:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e291)", 2024-09-20T19:02:58.775 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:35.175226+0000", 2024-09-20T19:02:58.775 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 143.47203338099999, 2024-09-20T19:02:58.775 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.044694204, 2024-09-20T19:02:58.775 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:58.775 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:58.775 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:02:58.775 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:02:58.775 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:02:58.776 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:02:58.776 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 142919 2024-09-20T19:02:58.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.776 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:58.776 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.776 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:58.776 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.175226+0000", 2024-09-20T19:02:58.776 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:58.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.776 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.776 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:58.776 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.175226+0000", 2024-09-20T19:02:58.776 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:58.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.776 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.777 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:58.777 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.175228+0000", 2024-09-20T19:02:58.777 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.88e-06 2024-09-20T19:02:58.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.777 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.777 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:58.777 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.175235+0000", 2024-09-20T19:02:58.777 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8419999999999999e-06 2024-09-20T19:02:58.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.778 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.779 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:58.779 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.175253+0000", 2024-09-20T19:02:58.779 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8995e-05 2024-09-20T19:02:58.779 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.779 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.779 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:58.779 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.175262+0000", 2024-09-20T19:02:58.779 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.884e-06 2024-09-20T19:02:58.779 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.779 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.779 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:58.780 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.215752+0000", 2024-09-20T19:02:58.780 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.040489565999999998 2024-09-20T19:02:58.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.780 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.780 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:02:58.780 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.215759+0000", 2024-09-20T19:02:58.780 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1380000000000001e-06 2024-09-20T19:02:58.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.780 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.780 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:58.780 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.209925+0000", 2024-09-20T19:02:58.781 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.99416557900000002 2024-09-20T19:02:58.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.781 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:58.781 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.210025+0000", 2024-09-20T19:02:58.781 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010050700000000001 2024-09-20T19:02:58.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.781 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 7", 2024-09-20T19:02:58.781 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.210147+0000", 2024-09-20T19:02:58.781 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000122309 2024-09-20T19:02:58.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.781 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:58.781 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.219740+0000", 2024-09-20T19:02:58.782 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0095921180000000002 2024-09-20T19:02:58.782 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.782 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.782 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:02:58.782 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.219768+0000", 2024-09-20T19:02:58.782 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8391e-05 2024-09-20T19:02:58.782 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.782 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.782 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:02:58.782 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.219815+0000", 2024-09-20T19:02:58.782 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7197999999999998e-05 2024-09-20T19:02:58.782 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.782 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.783 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:58.783 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.219920+0000", 2024-09-20T19:02:58.783 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010479700000000001 2024-09-20T19:02:58.783 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:58.783 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:58.783 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:58.783 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.783 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.783 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14460.0:2698 5.0 5:05bf5b68:::notify.1:head [watch reconnect cookie 94056120167808 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e292)", 2024-09-20T19:02:58.783 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:35.178092+0000", 2024-09-20T19:02:58.783 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 143.46916729700001, 2024-09-20T19:02:58.783 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.021665381, 2024-09-20T19:02:58.783 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:58.783 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:58.784 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T19:02:58.784 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:02:58.784 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14460", 2024-09-20T19:02:58.784 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1555339426", 2024-09-20T19:02:58.784 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2698 2024-09-20T19:02:58.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.784 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:58.784 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.784 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:58.784 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.178092+0000", 2024-09-20T19:02:58.784 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:58.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.784 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.784 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:58.784 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.178092+0000", 2024-09-20T19:02:58.785 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:58.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.785 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:58.785 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.178093+0000", 2024-09-20T19:02:58.785 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2359999999999999e-06 2024-09-20T19:02:58.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.785 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:58.785 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.178098+0000", 2024-09-20T19:02:58.785 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1410000000000004e-06 2024-09-20T19:02:58.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.785 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:58.785 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.178113+0000", 2024-09-20T19:02:58.786 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4262e-05 2024-09-20T19:02:58.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.786 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.786 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:58.786 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.178120+0000", 2024-09-20T19:02:58.786 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.6210000000000004e-06 2024-09-20T19:02:58.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.786 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.786 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:58.786 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.180022+0000", 2024-09-20T19:02:58.786 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001901668 2024-09-20T19:02:58.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.786 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.786 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op must wait for map", 2024-09-20T19:02:58.787 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.180028+0000", 2024-09-20T19:02:58.787 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9359999999999996e-06 2024-09-20T19:02:58.787 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.787 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.787 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:58.787 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.195134+0000", 2024-09-20T19:02:58.787 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.015105831 2024-09-20T19:02:58.787 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.787 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.787 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:02:58.787 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.195142+0000", 2024-09-20T19:02:58.787 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8659999999999995e-06 2024-09-20T19:02:58.787 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.787 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.788 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:58.788 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.192667+0000", 2024-09-20T19:02:58.788 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.99752498700000003 2024-09-20T19:02:58.788 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.788 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.788 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:58.788 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.192972+0000", 2024-09-20T19:02:58.788 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000304677 2024-09-20T19:02:58.788 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.788 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.788 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:58.788 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.199757+0000", 2024-09-20T19:02:58.788 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0067851559999999997 2024-09-20T19:02:58.788 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:58.788 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:58.789 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:58.789 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.789 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.789 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:142931 9.c 9:3971d803:::benchmark_data_smithi083_124552_object28784:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e292)", 2024-09-20T19:02:58.789 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:35.206532+0000", 2024-09-20T19:02:58.789 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 143.44072762799999, 2024-09-20T19:02:58.789 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.016374055, 2024-09-20T19:02:58.789 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:58.789 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:58.789 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:02:58.789 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:02:58.789 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:02:58.789 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:02:58.789 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 142931 2024-09-20T19:02:58.789 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.790 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:58.790 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.790 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:58.790 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.206532+0000", 2024-09-20T19:02:58.790 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:58.790 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.790 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.790 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:58.790 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.206532+0000", 2024-09-20T19:02:58.790 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:58.790 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.790 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.790 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:58.790 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.206532+0000", 2024-09-20T19:02:58.791 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9500000000000001e-07 2024-09-20T19:02:58.791 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.791 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.791 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:58.791 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.206537+0000", 2024-09-20T19:02:58.791 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5600000000000004e-06 2024-09-20T19:02:58.791 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.791 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.791 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:58.791 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.206546+0000", 2024-09-20T19:02:58.791 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9479999999999997e-06 2024-09-20T19:02:58.791 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.791 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.791 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:58.791 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.206553+0000", 2024-09-20T19:02:58.792 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0119999999999999e-06 2024-09-20T19:02:58.792 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.792 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.792 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:58.792 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.215793+0000", 2024-09-20T19:02:58.792 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0092399519999999992 2024-09-20T19:02:58.792 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.792 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.792 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:02:58.792 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.215799+0000", 2024-09-20T19:02:58.792 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7389999999999996e-06 2024-09-20T19:02:58.792 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.792 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.792 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:58.793 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.212804+0000", 2024-09-20T19:02:58.793 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.99700544700000004 2024-09-20T19:02:58.793 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.793 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.793 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:58.793 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.213100+0000", 2024-09-20T19:02:58.793 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00029572499999999999 2024-09-20T19:02:58.793 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.793 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.793 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 7", 2024-09-20T19:02:58.793 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.213492+0000", 2024-09-20T19:02:58.793 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000391939 2024-09-20T19:02:58.793 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.793 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.793 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:02:58.794 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.219928+0000", 2024-09-20T19:02:58.794 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0064358100000000001 2024-09-20T19:02:58.794 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.794 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.794 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:58.794 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.222687+0000", 2024-09-20T19:02:58.794 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0027591949999999999 2024-09-20T19:02:58.794 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.794 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.794 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:02:58.794 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.222797+0000", 2024-09-20T19:02:58.794 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000110713 2024-09-20T19:02:58.795 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.795 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.795 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:58.795 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.222906+0000", 2024-09-20T19:02:58.795 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010812 2024-09-20T19:02:58.795 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:58.795 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:58.795 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:58.795 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.795 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.795 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:142996 9.c 9:3cdc78e2:::benchmark_data_smithi083_124552_object28849:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e292)", 2024-09-20T19:02:58.795 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:35.284730+0000", 2024-09-20T19:02:58.795 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 143.362529459, 2024-09-20T19:02:58.795 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.93861088699999995, 2024-09-20T19:02:58.796 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:02:58.796 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:02:58.796 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:02:58.796 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:02:58.796 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:02:58.796 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:02:58.796 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 142996 2024-09-20T19:02:58.796 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.796 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:02:58.796 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.796 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:02:58.796 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.284730+0000", 2024-09-20T19:02:58.796 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:58.796 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.796 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.797 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:02:58.797 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.284730+0000", 2024-09-20T19:02:58.797 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:02:58.797 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.797 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.797 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:02:58.797 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.284731+0000", 2024-09-20T19:02:58.797 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.376e-06 2024-09-20T19:02:58.797 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.797 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.797 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:02:58.797 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.284736+0000", 2024-09-20T19:02:58.797 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1680000000000003e-06 2024-09-20T19:02:58.797 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.797 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.798 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:02:58.798 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.284747+0000", 2024-09-20T19:02:58.798 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1077000000000001e-05 2024-09-20T19:02:58.798 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.798 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.798 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:02:58.798 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.284753+0000", 2024-09-20T19:02:58.798 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.874e-06 2024-09-20T19:02:58.798 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.798 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.798 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:58.798 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.284810+0000", 2024-09-20T19:02:58.798 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6573000000000002e-05 2024-09-20T19:02:58.798 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.799 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.799 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:02:58.799 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.284816+0000", 2024-09-20T19:02:58.799 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4840000000000001e-06 2024-09-20T19:02:58.799 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.799 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.799 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:02:58.799 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.216111+0000", 2024-09-20T19:02:58.799 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.93129438899999994 2024-09-20T19:02:58.799 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.799 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.799 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:02:58.799 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.216258+0000", 2024-09-20T19:02:58.799 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014760700000000001 2024-09-20T19:02:58.799 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.800 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.800 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 7", 2024-09-20T19:02:58.800 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.217033+0000", 2024-09-20T19:02:58.800 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00077462600000000005 2024-09-20T19:02:58.800 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.800 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.800 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:02:58.800 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.223055+0000", 2024-09-20T19:02:58.800 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0060222510000000002 2024-09-20T19:02:58.800 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.800 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.800 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:02:58.800 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.223067+0000", 2024-09-20T19:02:58.800 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2087e-05 2024-09-20T19:02:58.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.801 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.801 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:02:58.801 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.223265+0000", 2024-09-20T19:02:58.801 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019747800000000001 2024-09-20T19:02:58.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:02:58.801 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:02:58.801 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:02:58.801 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.223341+0000", 2024-09-20T19:02:58.801 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5896999999999995e-05 2024-09-20T19:02:58.801 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:58.801 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:58.801 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:58.801 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:02:58.801 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:02:58.802 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T19:02:58.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:58.803 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:58.803 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:02:58.803 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:58 smithi142 ceph-mon[32508]: pgmap v656: 148 pgs: 148 active+clean; 4.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.6 KiB/s rd, 120 MiB/s wr, 1.93k op/s 2024-09-20T19:02:58.803 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:58 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:02:58.803 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:02:58 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2139012394' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:02:58.877 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 39 16 74467 74451 119.275 68.875 0.0100399 0.00837901 2024-09-20T19:02:58.904 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:58.927 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:58.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:58.928 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:59.029 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:59.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:59.064 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:59.064 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:59.165 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:59.197 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:59.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:59.198 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:59.299 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:59.319 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:59.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:59.320 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:59.421 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:02:59.448 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:59.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:59.450 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:02:59.550 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:02:59.578 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:59.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:59.580 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:02:59.680 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:02:59.720 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:02:59.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:59.721 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:02:59.822 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:59.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:59.861 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:59.861 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:02:59.877 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:2024-09-20T19:02:59.871481+0000 min lat: 0.00125834 max lat: 0.208094 avg lat: 0.00849071 2024-09-20T19:02:59.877 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-20T19:02:59.877 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 40 16 75374 75358 117.71 56.6875 0.00947566 0.00849071 2024-09-20T19:02:59.963 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:02:59.993 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:02:59.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:02:59.994 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:00.095 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:00.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:00.135 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:00.135 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:00.236 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:03:00.278 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:00.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:00.279 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:03:00.380 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:00.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:00.415 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:00.415 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:00.515 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:00.546 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:00.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:00.547 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:00.648 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:00.679 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:00.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:00.680 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:00.728 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:00 smithi083 ceph-mon[29421]: pgmap v657: 148 pgs: 148 active+clean; 4.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 79 MiB/s wr, 1.27k op/s 2024-09-20T19:03:00.728 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:00 smithi083 ceph-mon[36260]: pgmap v657: 148 pgs: 148 active+clean; 4.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 79 MiB/s wr, 1.27k op/s 2024-09-20T19:03:00.764 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:00 smithi142 ceph-mon[32508]: pgmap v657: 148 pgs: 148 active+clean; 4.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 79 MiB/s wr, 1.27k op/s 2024-09-20T19:03:00.781 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:00.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:00.817 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:00.817 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:00.877 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 41 16 76625 76609 116.746 78.1875 0.02636 0.0085587 2024-09-20T19:03:00.918 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:00.953 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:00.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:00.955 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:01.044 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:03:00 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:03:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:03:01.056 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:01.090 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:01.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:01.091 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:01.192 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:01.230 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:01.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:01.231 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:01.332 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:01.368 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:01.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:01.370 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:01.470 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:01.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:01.503 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:01.504 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:01.604 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:01.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:01.665 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:01.665 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:01.766 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:01.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:01.801 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:01.802 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:01.877 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 42 16 77954 77938 115.944 83.0625 0.00959322 0.00861994 2024-09-20T19:03:01.903 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:01.938 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:01.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:01.939 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:02.040 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:02.043 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:01 smithi083 ceph-mon[29421]: pgmap v658: 148 pgs: 148 active+clean; 4.5 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 620 B/s rd, 73 MiB/s wr, 1.17k op/s 2024-09-20T19:03:02.043 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:01 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:03:02.044 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:01 smithi083 ceph-mon[36260]: pgmap v658: 148 pgs: 148 active+clean; 4.5 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 620 B/s rd, 73 MiB/s wr, 1.17k op/s 2024-09-20T19:03:02.044 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:01 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:03:02.070 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 7, 6] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 5, 6, 7, 4] 2024-09-20T19:03:02.070 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T19:03:02.071 INFO:tasks.thrashosds.thrasher:Shrinking pool .mgr 2024-09-20T19:03:02.071 INFO:tasks.cephadm.ceph_manager.ceph:contract_pool .mgr by 10 min 8 2024-09-20T19:03:02.071 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T19:03:02.075 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:02.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:02.076 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:02.129 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:03:02.145 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-20T19:03:02.177 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:02.184 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:01 smithi142 ceph-mon[32508]: pgmap v658: 148 pgs: 148 active+clean; 4.5 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 620 B/s rd, 73 MiB/s wr, 1.17k op/s 2024-09-20T19:03:02.184 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:01 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:03:02.233 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:02.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:02.234 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:02.334 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:02.377 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:02.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:02.378 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:02.479 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:02.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:02.531 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:02.531 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:02.560 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:03:02.632 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:02.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:02.678 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:02.678 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:02.779 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:02.791 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:03:02.833 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:02.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:02.834 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:02.877 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 43 15 79143 79128 114.977 74.375 0.0156575 0.00869193 2024-09-20T19:03:02.936 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:02.962 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:02.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:02.963 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:03.064 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:03.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:03.119 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:03.119 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:03.220 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:03.252 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:03.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:03.253 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:03.268 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T19:03:03.354 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:03.386 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:03.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:03.387 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:03.488 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:03:03.521 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:03.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:03.523 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:03:03.624 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:03.641 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T19:03:03.642 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-09-20T19:03:03.642 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:03.642 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:79476 10.9 10:9660a774:::benchmark_data_smithi083_148100_object79475:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e347)", 2024-09-20T19:03:03.642 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:03:03.173869+0000", 2024-09-20T19:03:03.642 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.018205150999999999, 2024-09-20T19:03:03.642 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.018236835, 2024-09-20T19:03:03.642 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:03:03.642 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:03:03.642 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "waiting for sub ops", 2024-09-20T19:03:03.642 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:03:03.642 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:03:03.643 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:03:03.643 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 79476 2024-09-20T19:03:03.643 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:03.643 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:03:03.643 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:03.643 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:03:03.643 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.173869+0000", 2024-09-20T19:03:03.643 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:03.643 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:03.643 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:03.643 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:03:03.643 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.173869+0000", 2024-09-20T19:03:03.644 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:03.644 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:03.644 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:03.644 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:03:03.644 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.173870+0000", 2024-09-20T19:03:03.644 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.9400000000000004e-07 2024-09-20T19:03:03.644 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:03.644 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:03.644 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:03:03.644 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.173916+0000", 2024-09-20T19:03:03.644 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.5911000000000001e-05 2024-09-20T19:03:03.644 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:03.644 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:03.645 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:03:03.645 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.173930+0000", 2024-09-20T19:03:03.645 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4039999999999999e-05 2024-09-20T19:03:03.645 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:03.645 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:03.645 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:03:03.645 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.173937+0000", 2024-09-20T19:03:03.645 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.4499999999999998e-06 2024-09-20T19:03:03.645 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:03.645 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:03.645 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:03.645 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.176329+0000", 2024-09-20T19:03:03.645 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0023924419999999998 2024-09-20T19:03:03.645 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:03.645 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:03.645 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:03:03.646 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.176464+0000", 2024-09-20T19:03:03.646 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000134523 2024-09-20T19:03:03.646 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:03.646 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:03.646 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 2,7", 2024-09-20T19:03:03.646 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.176603+0000", 2024-09-20T19:03:03.646 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013916800000000001 2024-09-20T19:03:03.646 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:03.646 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:03.646 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:03:03.646 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.180166+0000", 2024-09-20T19:03:03.646 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0035625959999999999 2024-09-20T19:03:03.646 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:03.646 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:03.647 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:03:03.647 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.180383+0000", 2024-09-20T19:03:03.647 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000217693 2024-09-20T19:03:03.647 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:03.647 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:03:03.647 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:03.647 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:03.647 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:03.647 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:79485 10.9 10:976d269f:::benchmark_data_smithi083_148100_object79484:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e347)", 2024-09-20T19:03:03.647 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:03:03.180925+0000", 2024-09-20T19:03:03.647 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.011148988, 2024-09-20T19:03:03.647 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.011235149, 2024-09-20T19:03:03.647 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:03:03.647 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:03:03.648 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "waiting for sub ops", 2024-09-20T19:03:03.648 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:03:03.648 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:03:03.648 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:03:03.648 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 79485 2024-09-20T19:03:03.648 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:03.648 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:03:03.648 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:03.648 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:03:03.648 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.180925+0000", 2024-09-20T19:03:03.648 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:03.648 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:03.648 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:03.648 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:03:03.648 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.180925+0000", 2024-09-20T19:03:03.649 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:03.649 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:03.649 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:03.649 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:03:03.649 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.180926+0000", 2024-09-20T19:03:03.649 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.6500000000000008e-07 2024-09-20T19:03:03.649 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:03.649 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:03.649 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:03:03.649 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.180979+0000", 2024-09-20T19:03:03.649 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3149999999999998e-05 2024-09-20T19:03:03.649 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:03.649 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:03.649 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:03:03.650 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.180992+0000", 2024-09-20T19:03:03.650 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3423e-05 2024-09-20T19:03:03.650 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:03.650 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:03.650 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:03:03.650 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.180999+0000", 2024-09-20T19:03:03.650 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.9990000000000002e-06 2024-09-20T19:03:03.650 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:03.650 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:03.650 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:03.650 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.181050+0000", 2024-09-20T19:03:03.650 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0356999999999999e-05 2024-09-20T19:03:03.650 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:03.651 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:03.651 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:03:03.651 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.181160+0000", 2024-09-20T19:03:03.651 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000110103 2024-09-20T19:03:03.651 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:03.651 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:03.651 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 2,7", 2024-09-20T19:03:03.651 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.181278+0000", 2024-09-20T19:03:03.651 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000117985 2024-09-20T19:03:03.651 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:03.651 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:03.651 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:03:03.651 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.184885+0000", 2024-09-20T19:03:03.652 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.003606677 2024-09-20T19:03:03.652 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:03.652 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:03.652 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:03:03.652 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.187406+0000", 2024-09-20T19:03:03.652 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.002521368 2024-09-20T19:03:03.652 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:03.652 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:03:03.652 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:03.652 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:03.652 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-09-20T19:03:03.652 INFO:teuthology.orchestra.run.smithi142.stdout: "num_ops": 2 2024-09-20T19:03:03.652 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T19:03:03.653 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:03.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:03.654 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:03.719 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:03:03.755 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:03.772 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T19:03:03.775 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-20T19:03:03.808 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:03.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:03.809 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:03.875 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-20T19:03:03.877 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 44 16 80181 80165 113.837 64.8125 0.0131286 0.00877904 2024-09-20T19:03:03.910 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:03.942 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:03.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:03.943 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:04.013 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:03 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:03:04.014 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:03 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:03:04.014 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:03 smithi142 ceph-mon[32508]: osdmap e347: 8 total, 8 up, 8 in 2024-09-20T19:03:04.014 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:03 smithi142 ceph-mon[32508]: pgmap v660: 148 pgs: 148 active+clean; 4.7 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 99 MiB/s wr, 1.58k op/s 2024-09-20T19:03:04.014 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:03 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1155246174' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:03:04.014 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:03 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:03:04.014 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:03 smithi142 ceph-mon[32508]: from='client.15975 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:03:04.043 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:03 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:03:04.044 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:03 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:03:04.044 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:03 smithi083 ceph-mon[29421]: osdmap e347: 8 total, 8 up, 8 in 2024-09-20T19:03:04.044 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:03 smithi083 ceph-mon[29421]: pgmap v660: 148 pgs: 148 active+clean; 4.7 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 99 MiB/s wr, 1.58k op/s 2024-09-20T19:03:04.044 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:03 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1155246174' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:03:04.044 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:03 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:03:04.044 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:03 smithi083 ceph-mon[29421]: from='client.15975 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:03:04.045 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:03 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:03:04.045 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:03 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:03:04.045 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:03 smithi083 ceph-mon[36260]: osdmap e347: 8 total, 8 up, 8 in 2024-09-20T19:03:04.045 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:03 smithi083 ceph-mon[36260]: pgmap v660: 148 pgs: 148 active+clean; 4.7 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 99 MiB/s wr, 1.58k op/s 2024-09-20T19:03:04.045 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:03 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1155246174' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:03:04.045 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:03 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:03:04.045 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:03 smithi083 ceph-mon[36260]: from='client.15975 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:03:04.045 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:04.075 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:04.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:04.076 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:04.176 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:04.197 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:04.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:04.199 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:04.299 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:04.333 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:04.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:04.334 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:04.435 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:04.472 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:04.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:04.473 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:04.573 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:04.593 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:04.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:04.594 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:04.695 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:04.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:04.726 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:04.727 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:04.827 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:04.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:04.863 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:04.863 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:04.878 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 45 16 81226 81210 112.759 65.3125 0.0115496 0.00886335 2024-09-20T19:03:04.964 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:04.995 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:04.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:04.996 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:05.097 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:05.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:05.132 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:05.132 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:05.165 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:04 smithi083 ceph-mon[29421]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:03:05.165 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:04 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T19:03:05.165 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:04 smithi083 ceph-mon[29421]: osdmap e348: 8 total, 8 up, 8 in 2024-09-20T19:03:05.166 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:04 smithi083 ceph-mon[36260]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:03:05.166 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:04 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T19:03:05.166 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:04 smithi083 ceph-mon[36260]: osdmap e348: 8 total, 8 up, 8 in 2024-09-20T19:03:05.173 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:03:05.173 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:03:05.206 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:03:05.206 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:03:05.233 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:05.242 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:03:05.242 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:03:05.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:04 smithi142 ceph-mon[32508]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:03:05.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:04 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T19:03:05.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:04 smithi142 ceph-mon[32508]: osdmap e348: 8 total, 8 up, 8 in 2024-09-20T19:03:05.271 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:05.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:05.272 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:05.273 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:03:05.273 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:03:05.306 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:03:05.306 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:03:05.339 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:03:05.339 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:03:05.372 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:03:05.372 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:03:05.373 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:05.413 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:03:05.413 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:03:05.416 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:05.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:05.417 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:05.518 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:05.534 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":660,"stamp":"2024-09-20T19:03:03.091911+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5059254811,"num_objects":77388,"num_object_clones":0,"num_object_copies":232164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77388,"num_whiteouts":0,"num_read":2047,"num_read_kb":1904,"num_write":78024,"num_write_kb":4941769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":1774258,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77745,"ondisk_log_size":77745,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":18487484,"kb_used_data":14782012,"kb_used_omap":16,"kb_used_meta":3705327,"kb_avail":731473732,"statfs":{"total":767960285184,"available":749029101568,"internally_reserved":0,"allocated":15136780288,"data_stored":15132355441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16790,"internal_metadata":3794255466},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,3,3,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":828768256,"num_objects":12646,"num_object_clones":0,"num_object_copies":37938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12646,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":12646,"num_write_kb":809344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.005367"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":622,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.359142+0000","last_change":"2024-09-20T19:01:31.817967+0000","last_active":"2024-09-20T19:02:55.359142+0000","last_peered":"2024-09-20T19:02:55.359142+0000","last_clean":"2024-09-20T19:02:55.359142+0000","last_became_active":"2024-09-20T18:49:45.943201+0000","last_became_peered":"2024-09-20T18:49:45.943201+0000","last_unstale":"2024-09-20T19:02:55.359142+0000","last_undegraded":"2024-09-20T19:02:55.359142+0000","last_fullsized":"2024-09-20T19:02:55.359142+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:31.817909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:31.817909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:53.060041+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"273'15","reported_seq":635,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.363432+0000","last_change":"2024-09-20T19:01:22.840670+0000","last_active":"2024-09-20T19:02:55.363432+0000","last_peered":"2024-09-20T19:02:55.363432+0000","last_clean":"2024-09-20T19:02:55.363432+0000","last_became_active":"2024-09-20T19:01:22.838745+0000","last_became_peered":"2024-09-20T19:01:22.838745+0000","last_unstale":"2024-09-20T19:02:55.363432+0000","last_undegraded":"2024-09-20T19:02:55.363432+0000","last_fullsized":"2024-09-20T19:02:55.363432+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-20T18:54:50.231620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:50.231620+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:09.095233+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":612,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.363067+0000","last_change":"2024-09-20T18:57:49.016078+0000","last_active":"2024-09-20T19:02:55.363067+0000","last_peered":"2024-09-20T19:02:55.363067+0000","last_clean":"2024-09-20T19:02:55.363067+0000","last_became_active":"2024-09-20T18:49:49.959514+0000","last_became_peered":"2024-09-20T18:49:49.959514+0000","last_unstale":"2024-09-20T19:02:55.363067+0000","last_undegraded":"2024-09-20T19:02:55.363067+0000","last_fullsized":"2024-09-20T19:02:55.363067+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:49.016006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:49.016006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:10.916493+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":561,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.364915+0000","last_change":"2024-09-20T19:01:22.841355+0000","last_active":"2024-09-20T19:02:55.364915+0000","last_peered":"2024-09-20T19:02:55.364915+0000","last_clean":"2024-09-20T19:02:55.364915+0000","last_became_active":"2024-09-20T19:01:22.838827+0000","last_became_peered":"2024-09-20T19:01:22.838827+0000","last_unstale":"2024-09-20T19:02:55.364915+0000","last_undegraded":"2024-09-20T19:02:55.364915+0000","last_fullsized":"2024-09-20T19:02:55.364915+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:02.084375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:02.084375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:20.990491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":622,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.366221+0000","last_change":"2024-09-20T19:02:02.441633+0000","last_active":"2024-09-20T19:02:55.366221+0000","last_peered":"2024-09-20T19:02:55.366221+0000","last_clean":"2024-09-20T19:02:55.366221+0000","last_became_active":"2024-09-20T18:59:05.081081+0000","last_became_peered":"2024-09-20T18:59:05.081081+0000","last_unstale":"2024-09-20T19:02:55.366221+0000","last_undegraded":"2024-09-20T19:02:55.366221+0000","last_fullsized":"2024-09-20T19:02:55.366221+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:02.441577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:02.441577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:28.158812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":627,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.364176+0000","last_change":"2024-09-20T18:59:05.073093+0000","last_active":"2024-09-20T19:02:55.364176+0000","last_peered":"2024-09-20T19:02:55.364176+0000","last_clean":"2024-09-20T19:02:55.364176+0000","last_became_active":"2024-09-20T18:59:05.072675+0000","last_became_peered":"2024-09-20T18:59:05.072675+0000","last_unstale":"2024-09-20T19:02:55.364176+0000","last_undegraded":"2024-09-20T19:02:55.364176+0000","last_fullsized":"2024-09-20T19:02:55.364176+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:03.055929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:59:03.055929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:45.957147+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"274'27","reported_seq":659,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.363700+0000","last_change":"2024-09-20T19:02:40.899215+0000","last_active":"2024-09-20T19:02:55.363700+0000","last_peered":"2024-09-20T19:02:55.363700+0000","last_clean":"2024-09-20T19:02:55.363700+0000","last_became_active":"2024-09-20T18:49:47.952150+0000","last_became_peered":"2024-09-20T18:49:47.952150+0000","last_unstale":"2024-09-20T19:02:55.363700+0000","last_undegraded":"2024-09-20T19:02:55.363700+0000","last_fullsized":"2024-09-20T19:02:55.363700+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'27","last_scrub_stamp":"2024-09-20T19:02:40.899139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:40.899139+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:02.856822+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":552,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.363353+0000","last_change":"2024-09-20T19:02:23.982879+0000","last_active":"2024-09-20T19:02:55.363353+0000","last_peered":"2024-09-20T19:02:55.363353+0000","last_clean":"2024-09-20T19:02:55.363353+0000","last_became_active":"2024-09-20T19:01:22.839075+0000","last_became_peered":"2024-09-20T19:01:22.839075+0000","last_unstale":"2024-09-20T19:02:55.363353+0000","last_undegraded":"2024-09-20T19:02:55.363353+0000","last_fullsized":"2024-09-20T19:02:55.363353+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:23.982813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:23.982813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:51.507141+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"59'1","reported_seq":675,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.363188+0000","last_change":"2024-09-20T19:01:28.592723+0000","last_active":"2024-09-20T19:02:55.363188+0000","last_peered":"2024-09-20T19:02:55.363188+0000","last_clean":"2024-09-20T19:02:55.363188+0000","last_became_active":"2024-09-20T19:01:22.847016+0000","last_became_peered":"2024-09-20T19:01:22.847016+0000","last_unstale":"2024-09-20T19:02:55.363188+0000","last_undegraded":"2024-09-20T19:02:55.363188+0000","last_fullsized":"2024-09-20T19:02:55.363188+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:01:28.592653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:28.592653+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:43.784234+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"273'15","reported_seq":600,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.364222+0000","last_change":"2024-09-20T19:02:42.114419+0000","last_active":"2024-09-20T19:02:55.364222+0000","last_peered":"2024-09-20T19:02:55.364222+0000","last_clean":"2024-09-20T19:02:55.364222+0000","last_became_active":"2024-09-20T19:01:22.844899+0000","last_became_peered":"2024-09-20T19:01:22.844899+0000","last_unstale":"2024-09-20T19:02:55.364222+0000","last_undegraded":"2024-09-20T19:02:55.364222+0000","last_fullsized":"2024-09-20T19:02:55.364222+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:02:42.114331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:42.114331+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:44.431155+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":620,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.362190+0000","last_change":"2024-09-20T19:01:51.564693+0000","last_active":"2024-09-20T19:02:55.362190+0000","last_peered":"2024-09-20T19:02:55.362190+0000","last_clean":"2024-09-20T19:02:55.362190+0000","last_became_active":"2024-09-20T18:49:49.961188+0000","last_became_peered":"2024-09-20T18:49:49.961188+0000","last_unstale":"2024-09-20T19:02:55.362190+0000","last_undegraded":"2024-09-20T19:02:55.362190+0000","last_fullsized":"2024-09-20T19:02:55.362190+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:51.564624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:51.564624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:01.653667+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":593,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.359902+0000","last_change":"2024-09-20T19:02:03.029635+0000","last_active":"2024-09-20T19:02:55.359902+0000","last_peered":"2024-09-20T19:02:55.359902+0000","last_clean":"2024-09-20T19:02:55.359902+0000","last_became_active":"2024-09-20T18:49:51.966715+0000","last_became_peered":"2024-09-20T18:49:51.966715+0000","last_unstale":"2024-09-20T19:02:55.359902+0000","last_undegraded":"2024-09-20T19:02:55.359902+0000","last_fullsized":"2024-09-20T19:02:55.359902+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:03.029585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:03.029585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:07.806574+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":631,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.362631+0000","last_change":"2024-09-20T19:01:48.520088+0000","last_active":"2024-09-20T19:02:55.362631+0000","last_peered":"2024-09-20T19:02:55.362631+0000","last_clean":"2024-09-20T19:02:55.362631+0000","last_became_active":"2024-09-20T19:01:22.846527+0000","last_became_peered":"2024-09-20T19:01:22.846527+0000","last_unstale":"2024-09-20T19:02:55.362631+0000","last_undegraded":"2024-09-20T19:02:55.362631+0000","last_fullsized":"2024-09-20T19:02:55.362631+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:48.520017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:48.520017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:49.827940+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"273'7","reported_seq":590,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.361968+0000","last_change":"2024-09-20T19:01:22.844317+0000","last_active":"2024-09-20T19:02:55.361968+0000","last_peered":"2024-09-20T19:02:55.361968+0000","last_clean":"2024-09-20T19:02:55.361968+0000","last_became_active":"2024-09-20T19:01:22.844013+0000","last_became_peered":"2024-09-20T19:01:22.844013+0000","last_unstale":"2024-09-20T19:02:55.361968+0000","last_undegraded":"2024-09-20T19:02:55.361968+0000","last_fullsized":"2024-09-20T19:02:55.361968+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-20T18:58:33.660531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:33.660531+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:54.196090+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":624,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.363746+0000","last_change":"2024-09-20T19:02:33.203392+0000","last_active":"2024-09-20T19:02:55.363746+0000","last_peered":"2024-09-20T19:02:55.363746+0000","last_clean":"2024-09-20T19:02:55.363746+0000","last_became_active":"2024-09-20T19:01:22.830605+0000","last_became_peered":"2024-09-20T19:01:22.830605+0000","last_unstale":"2024-09-20T19:02:55.363746+0000","last_undegraded":"2024-09-20T19:02:55.363746+0000","last_fullsized":"2024-09-20T19:02:55.363746+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:33.203312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:33.203312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:35.713757+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":608,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.363315+0000","last_change":"2024-09-20T18:58:26.465305+0000","last_active":"2024-09-20T19:02:55.363315+0000","last_peered":"2024-09-20T19:02:55.363315+0000","last_clean":"2024-09-20T19:02:55.363315+0000","last_became_active":"2024-09-20T18:49:51.970663+0000","last_became_peered":"2024-09-20T18:49:51.970663+0000","last_unstale":"2024-09-20T19:02:55.363315+0000","last_undegraded":"2024-09-20T19:02:55.363315+0000","last_fullsized":"2024-09-20T19:02:55.363315+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:26.465264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:26.465264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:18.787658+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":541,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.363429+0000","last_change":"2024-09-20T19:01:22.841173+0000","last_active":"2024-09-20T19:02:55.363429+0000","last_peered":"2024-09-20T19:02:55.363429+0000","last_clean":"2024-09-20T19:02:55.363429+0000","last_became_active":"2024-09-20T19:01:22.838149+0000","last_became_peered":"2024-09-20T19:01:22.838149+0000","last_unstale":"2024-09-20T19:02:55.363429+0000","last_undegraded":"2024-09-20T19:02:55.363429+0000","last_fullsized":"2024-09-20T19:02:55.363429+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:00.906652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:00.906652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:19.046861+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":632,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.360203+0000","last_change":"2024-09-20T19:01:44.998192+0000","last_active":"2024-09-20T19:02:55.360203+0000","last_peered":"2024-09-20T19:02:55.360203+0000","last_clean":"2024-09-20T19:02:55.360203+0000","last_became_active":"2024-09-20T19:01:22.845773+0000","last_became_peered":"2024-09-20T19:01:22.845773+0000","last_unstale":"2024-09-20T19:02:55.360203+0000","last_undegraded":"2024-09-20T19:02:55.360203+0000","last_fullsized":"2024-09-20T19:02:55.360203+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:44.998147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:44.998147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"273'27","reported_seq":662,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.369259+0000","last_change":"2024-09-20T19:01:59.787246+0000","last_active":"2024-09-20T19:02:55.369259+0000","last_peered":"2024-09-20T19:02:55.369259+0000","last_clean":"2024-09-20T19:02:55.369259+0000","last_became_active":"2024-09-20T18:49:47.950939+0000","last_became_peered":"2024-09-20T18:49:47.950939+0000","last_unstale":"2024-09-20T19:02:55.369259+0000","last_undegraded":"2024-09-20T19:02:55.369259+0000","last_fullsized":"2024-09-20T19:02:55.369259+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'27","last_scrub_stamp":"2024-09-20T19:01:59.787179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:59.787179+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:02.865513+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":615,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.364500+0000","last_change":"2024-09-20T19:02:05.889054+0000","last_active":"2024-09-20T19:02:55.364500+0000","last_peered":"2024-09-20T19:02:55.364500+0000","last_clean":"2024-09-20T19:02:55.364500+0000","last_became_active":"2024-09-20T19:01:22.834771+0000","last_became_peered":"2024-09-20T19:01:22.834771+0000","last_unstale":"2024-09-20T19:02:55.364500+0000","last_undegraded":"2024-09-20T19:02:55.364500+0000","last_fullsized":"2024-09-20T19:02:55.364500+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:05.888974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:05.888974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:34.411336+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":623,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.363871+0000","last_change":"2024-09-20T19:02:28.934597+0000","last_active":"2024-09-20T19:02:55.363871+0000","last_peered":"2024-09-20T19:02:55.363871+0000","last_clean":"2024-09-20T19:02:55.363871+0000","last_became_active":"2024-09-20T18:59:05.084496+0000","last_became_peered":"2024-09-20T18:59:05.084496+0000","last_unstale":"2024-09-20T19:02:55.363871+0000","last_undegraded":"2024-09-20T19:02:55.363871+0000","last_fullsized":"2024-09-20T19:02:55.363871+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:28.934264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:28.934264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:43.415620+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"59'1","reported_seq":547,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.365720+0000","last_change":"2024-09-20T19:01:22.840962+0000","last_active":"2024-09-20T19:02:55.365720+0000","last_peered":"2024-09-20T19:02:55.365720+0000","last_clean":"2024-09-20T19:02:55.365720+0000","last_became_active":"2024-09-20T19:01:22.840591+0000","last_became_peered":"2024-09-20T19:01:22.840591+0000","last_unstale":"2024-09-20T19:02:55.365720+0000","last_undegraded":"2024-09-20T19:02:55.365720+0000","last_fullsized":"2024-09-20T19:02:55.365720+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:59:03.397225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:59:03.397225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"274'18","reported_seq":635,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.367983+0000","last_change":"2024-09-20T19:01:22.837011+0000","last_active":"2024-09-20T19:02:55.367983+0000","last_peered":"2024-09-20T19:02:55.367983+0000","last_clean":"2024-09-20T19:02:55.367983+0000","last_became_active":"2024-09-20T19:01:22.836111+0000","last_became_peered":"2024-09-20T19:01:22.836111+0000","last_unstale":"2024-09-20T19:02:55.367983+0000","last_undegraded":"2024-09-20T19:02:55.367983+0000","last_fullsized":"2024-09-20T19:02:55.367983+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:57:09.818999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:09.818999+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:07.600886+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":596,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.362570+0000","last_change":"2024-09-20T19:02:02.094517+0000","last_active":"2024-09-20T19:02:55.362570+0000","last_peered":"2024-09-20T19:02:55.362570+0000","last_clean":"2024-09-20T19:02:55.362570+0000","last_became_active":"2024-09-20T19:01:22.848532+0000","last_became_peered":"2024-09-20T19:01:22.848532+0000","last_unstale":"2024-09-20T19:02:55.362570+0000","last_undegraded":"2024-09-20T19:02:55.362570+0000","last_fullsized":"2024-09-20T19:02:55.362570+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:02.094473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:02.094473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:17.459163+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":610,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.364359+0000","last_change":"2024-09-20T19:02:28.294390+0000","last_active":"2024-09-20T19:02:55.364359+0000","last_peered":"2024-09-20T19:02:55.364359+0000","last_clean":"2024-09-20T19:02:55.364359+0000","last_became_active":"2024-09-20T18:49:51.970058+0000","last_became_peered":"2024-09-20T18:49:51.970058+0000","last_unstale":"2024-09-20T19:02:55.364359+0000","last_undegraded":"2024-09-20T19:02:55.364359+0000","last_fullsized":"2024-09-20T19:02:55.364359+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T19:02:28.294292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:28.294292+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:41.507259+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":616,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.363257+0000","last_change":"2024-09-20T19:01:22.837474+0000","last_active":"2024-09-20T19:02:55.363257+0000","last_peered":"2024-09-20T19:02:55.363257+0000","last_clean":"2024-09-20T19:02:55.363257+0000","last_became_active":"2024-09-20T19:01:22.837139+0000","last_became_peered":"2024-09-20T19:01:22.837139+0000","last_unstale":"2024-09-20T19:02:55.363257+0000","last_undegraded":"2024-09-20T19:02:55.363257+0000","last_fullsized":"2024-09-20T19:02:55.363257+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:57:18.032067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:18.032067+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:09.880584+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"274'20","reported_seq":664,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.359459+0000","last_change":"2024-09-20T19:02:35.097053+0000","last_active":"2024-09-20T19:02:55.359459+0000","last_peered":"2024-09-20T19:02:55.359459+0000","last_clean":"2024-09-20T19:02:55.359459+0000","last_became_active":"2024-09-20T18:49:48.500809+0000","last_became_peered":"2024-09-20T18:49:48.500809+0000","last_unstale":"2024-09-20T19:02:55.359459+0000","last_undegraded":"2024-09-20T19:02:55.359459+0000","last_fullsized":"2024-09-20T19:02:55.359459+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'20","last_scrub_stamp":"2024-09-20T19:02:35.096979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:35.096979+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:51.795986+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"308'12","reported_seq":263,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.360476+0000","last_change":"2024-09-20T19:01:44.541242+0000","last_active":"2024-09-20T19:02:55.360476+0000","last_peered":"2024-09-20T19:02:55.360476+0000","last_clean":"2024-09-20T19:02:55.360476+0000","last_became_active":"2024-09-20T19:01:22.845596+0000","last_became_peered":"2024-09-20T19:01:22.845596+0000","last_unstale":"2024-09-20T19:02:55.360476+0000","last_undegraded":"2024-09-20T19:02:55.360476+0000","last_fullsized":"2024-09-20T19:02:55.360476+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"308'12","last_scrub_stamp":"2024-09-20T19:01:44.541156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:44.541156+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:11.696073+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"346'4625","reported_seq":4677,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:03:00.502027+0000","last_change":"2024-09-20T19:02:11.829261+0000","last_active":"2024-09-20T19:03:00.502027+0000","last_peered":"2024-09-20T19:03:00.502027+0000","last_clean":"2024-09-20T19:03:00.502027+0000","last_became_active":"2024-09-20T19:02:11.828743+0000","last_became_peered":"2024-09-20T19:02:11.828743+0000","last_unstale":"2024-09-20T19:03:00.502027+0000","last_undegraded":"2024-09-20T19:03:00.502027+0000","last_fullsized":"2024-09-20T19:03:00.502027+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4625,"log_dups_size":0,"ondisk_log_size":4625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:26.017450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":303104000,"num_objects":4625,"num_object_clones":0,"num_object_copies":13875,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4625,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4625,"num_write_kb":296000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"274'27","reported_seq":656,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.370042+0000","last_change":"2024-09-20T19:02:27.897213+0000","last_active":"2024-09-20T19:02:55.370042+0000","last_peered":"2024-09-20T19:02:55.370042+0000","last_clean":"2024-09-20T19:02:55.370042+0000","last_became_active":"2024-09-20T19:01:22.833840+0000","last_became_peered":"2024-09-20T19:01:22.833840+0000","last_unstale":"2024-09-20T19:02:55.370042+0000","last_undegraded":"2024-09-20T19:02:55.370042+0000","last_fullsized":"2024-09-20T19:02:55.370042+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'27","last_scrub_stamp":"2024-09-20T19:02:27.897138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:27.897138+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:40.557830+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":613,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.363114+0000","last_change":"2024-09-20T19:00:31.512527+0000","last_active":"2024-09-20T19:02:55.363114+0000","last_peered":"2024-09-20T19:02:55.363114+0000","last_clean":"2024-09-20T19:02:55.363114+0000","last_became_active":"2024-09-20T18:49:45.936104+0000","last_became_peered":"2024-09-20T18:49:45.936104+0000","last_unstale":"2024-09-20T19:02:55.363114+0000","last_undegraded":"2024-09-20T19:02:55.363114+0000","last_fullsized":"2024-09-20T19:02:55.363114+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:00:31.512472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:00:31.512472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:48.924236+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"308'10","reported_seq":720,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.364342+0000","last_change":"2024-09-20T19:02:23.017699+0000","last_active":"2024-09-20T19:02:55.364342+0000","last_peered":"2024-09-20T19:02:55.364342+0000","last_clean":"2024-09-20T19:02:55.364342+0000","last_became_active":"2024-09-20T19:01:22.845603+0000","last_became_peered":"2024-09-20T19:01:22.845603+0000","last_unstale":"2024-09-20T19:02:55.364342+0000","last_undegraded":"2024-09-20T19:02:55.364342+0000","last_fullsized":"2024-09-20T19:02:55.364342+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"308'10","last_scrub_stamp":"2024-09-20T19:02:23.017626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:23.017626+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:51.273397+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":608,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.362770+0000","last_change":"2024-09-20T19:02:22.736879+0000","last_active":"2024-09-20T19:02:55.362770+0000","last_peered":"2024-09-20T19:02:55.362770+0000","last_clean":"2024-09-20T19:02:55.362770+0000","last_became_active":"2024-09-20T18:49:51.971399+0000","last_became_peered":"2024-09-20T18:49:51.971399+0000","last_unstale":"2024-09-20T19:02:55.362770+0000","last_undegraded":"2024-09-20T19:02:55.362770+0000","last_fullsized":"2024-09-20T19:02:55.362770+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:22.736825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:22.736825+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:51.591260+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.e","version":"346'4850","reported_seq":4903,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:03:00.488659+0000","last_change":"2024-09-20T19:02:11.824951+0000","last_active":"2024-09-20T19:03:00.488659+0000","last_peered":"2024-09-20T19:03:00.488659+0000","last_clean":"2024-09-20T19:03:00.488659+0000","last_became_active":"2024-09-20T19:02:11.824670+0000","last_became_peered":"2024-09-20T19:02:11.824670+0000","last_unstale":"2024-09-20T19:03:00.488659+0000","last_undegraded":"2024-09-20T19:03:00.488659+0000","last_fullsized":"2024-09-20T19:03:00.488659+0000","mapping_epoch":325,"log_start":"330'300","ondisk_log_start":"330'300","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4550,"log_dups_size":0,"ondisk_log_size":4550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:16.256458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":317849600,"num_objects":4850,"num_object_clones":0,"num_object_copies":14550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4850,"num_write_kb":310400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"273'30","reported_seq":667,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.364267+0000","last_change":"2024-09-20T19:01:10.381171+0000","last_active":"2024-09-20T19:02:55.364267+0000","last_peered":"2024-09-20T19:02:55.364267+0000","last_clean":"2024-09-20T19:02:55.364267+0000","last_became_active":"2024-09-20T18:49:48.499189+0000","last_became_peered":"2024-09-20T18:49:48.499189+0000","last_unstale":"2024-09-20T19:02:55.364267+0000","last_undegraded":"2024-09-20T19:02:55.364267+0000","last_fullsized":"2024-09-20T19:02:55.364267+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'30","last_scrub_stamp":"2024-09-20T19:01:10.381094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:10.381094+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":613,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.364315+0000","last_change":"2024-09-20T19:02:22.806587+0000","last_active":"2024-09-20T19:02:55.364315+0000","last_peered":"2024-09-20T19:02:55.364315+0000","last_clean":"2024-09-20T19:02:55.364315+0000","last_became_active":"2024-09-20T18:49:45.940544+0000","last_became_peered":"2024-09-20T18:49:45.940544+0000","last_unstale":"2024-09-20T19:02:55.364315+0000","last_undegraded":"2024-09-20T19:02:55.364315+0000","last_fullsized":"2024-09-20T19:02:55.364315+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:22.806509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:22.806509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:39.275044+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":591,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.362468+0000","last_change":"2024-09-20T19:01:22.835462+0000","last_active":"2024-09-20T19:02:55.362468+0000","last_peered":"2024-09-20T19:02:55.362468+0000","last_clean":"2024-09-20T19:02:55.362468+0000","last_became_active":"2024-09-20T19:01:22.833592+0000","last_became_peered":"2024-09-20T19:01:22.833592+0000","last_unstale":"2024-09-20T19:02:55.362468+0000","last_undegraded":"2024-09-20T19:02:55.362468+0000","last_fullsized":"2024-09-20T19:02:55.362468+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.229201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.229201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:18.032847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":582,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.364317+0000","last_change":"2024-09-20T19:01:22.830850+0000","last_active":"2024-09-20T19:02:55.364317+0000","last_peered":"2024-09-20T19:02:55.364317+0000","last_clean":"2024-09-20T19:02:55.364317+0000","last_became_active":"2024-09-20T19:01:22.830369+0000","last_became_peered":"2024-09-20T19:01:22.830369+0000","last_unstale":"2024-09-20T19:02:55.364317+0000","last_undegraded":"2024-09-20T19:02:55.364317+0000","last_fullsized":"2024-09-20T19:02:55.364317+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:27.704520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:27.704520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"346'4753","reported_seq":4806,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:03:01.015287+0000","last_change":"2024-09-20T19:02:11.828279+0000","last_active":"2024-09-20T19:03:01.015287+0000","last_peered":"2024-09-20T19:03:01.015287+0000","last_clean":"2024-09-20T19:03:01.015287+0000","last_became_active":"2024-09-20T19:02:11.827468+0000","last_became_peered":"2024-09-20T19:02:11.827468+0000","last_unstale":"2024-09-20T19:03:01.015287+0000","last_undegraded":"2024-09-20T19:03:01.015287+0000","last_fullsized":"2024-09-20T19:03:01.015287+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4753,"log_dups_size":0,"ondisk_log_size":4753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:26.126856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":311492608,"num_objects":4753,"num_object_clones":0,"num_object_copies":14259,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4753,"num_write_kb":304192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"274'28","reported_seq":657,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.362200+0000","last_change":"2024-09-20T19:01:22.848177+0000","last_active":"2024-09-20T19:02:55.362200+0000","last_peered":"2024-09-20T19:02:55.362200+0000","last_clean":"2024-09-20T19:02:55.362200+0000","last_became_active":"2024-09-20T19:01:22.847843+0000","last_became_peered":"2024-09-20T19:01:22.847843+0000","last_unstale":"2024-09-20T19:02:55.362200+0000","last_undegraded":"2024-09-20T19:02:55.362200+0000","last_fullsized":"2024-09-20T19:02:55.362200+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-20T18:54:57.394893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.394893+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:06.093887+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":41,"num_read_kb":34,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":626,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.359576+0000","last_change":"2024-09-20T19:01:22.852962+0000","last_active":"2024-09-20T19:02:55.359576+0000","last_peered":"2024-09-20T19:02:55.359576+0000","last_clean":"2024-09-20T19:02:55.359576+0000","last_became_active":"2024-09-20T19:01:22.852594+0000","last_became_peered":"2024-09-20T19:01:22.852594+0000","last_unstale":"2024-09-20T19:02:55.359576+0000","last_undegraded":"2024-09-20T19:02:55.359576+0000","last_fullsized":"2024-09-20T19:02:55.359576+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:00:29.890681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:00:29.890681+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:19.042344+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"308'10","reported_seq":794,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:56.156635+0000","last_change":"2024-09-20T19:01:22.834166+0000","last_active":"2024-09-20T19:02:56.156635+0000","last_peered":"2024-09-20T19:02:56.156635+0000","last_clean":"2024-09-20T19:02:56.156635+0000","last_became_active":"2024-09-20T19:01:22.833774+0000","last_became_peered":"2024-09-20T19:01:22.833774+0000","last_unstale":"2024-09-20T19:02:56.156635+0000","last_undegraded":"2024-09-20T19:02:56.156635+0000","last_fullsized":"2024-09-20T19:02:56.156635+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:58:23.114371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:23.114371+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:18.678273+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":616,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.363906+0000","last_change":"2024-09-20T18:59:05.082117+0000","last_active":"2024-09-20T19:02:55.363906+0000","last_peered":"2024-09-20T19:02:55.363906+0000","last_clean":"2024-09-20T19:02:55.363906+0000","last_became_active":"2024-09-20T18:59:05.081814+0000","last_became_peered":"2024-09-20T18:59:05.081814+0000","last_unstale":"2024-09-20T19:02:55.363906+0000","last_undegraded":"2024-09-20T19:02:55.363906+0000","last_fullsized":"2024-09-20T19:02:55.363906+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:29.530001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:29.530001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:11.272988+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"346'4726","reported_seq":4778,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:03:01.163598+0000","last_change":"2024-09-20T19:02:11.830056+0000","last_active":"2024-09-20T19:03:01.163598+0000","last_peered":"2024-09-20T19:03:01.163598+0000","last_clean":"2024-09-20T19:03:01.163598+0000","last_became_active":"2024-09-20T19:02:11.829446+0000","last_became_peered":"2024-09-20T19:02:11.829446+0000","last_unstale":"2024-09-20T19:03:01.163598+0000","last_undegraded":"2024-09-20T19:03:01.163598+0000","last_fullsized":"2024-09-20T19:03:01.163598+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4726,"log_dups_size":0,"ondisk_log_size":4726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:28.206813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":309723136,"num_objects":4726,"num_object_clones":0,"num_object_copies":14178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4726,"num_write_kb":302464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"274'20","reported_seq":674,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.365625+0000","last_change":"2024-09-20T19:02:35.931157+0000","last_active":"2024-09-20T19:02:55.365625+0000","last_peered":"2024-09-20T19:02:55.365625+0000","last_clean":"2024-09-20T19:02:55.365625+0000","last_became_active":"2024-09-20T19:01:22.834995+0000","last_became_peered":"2024-09-20T19:01:22.834995+0000","last_unstale":"2024-09-20T19:02:55.365625+0000","last_undegraded":"2024-09-20T19:02:55.365625+0000","last_fullsized":"2024-09-20T19:02:55.365625+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'20","last_scrub_stamp":"2024-09-20T19:02:35.930910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:35.930910+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:40.603359+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":637,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.363275+0000","last_change":"2024-09-20T19:02:30.686939+0000","last_active":"2024-09-20T19:02:55.363275+0000","last_peered":"2024-09-20T19:02:55.363275+0000","last_clean":"2024-09-20T19:02:55.363275+0000","last_became_active":"2024-09-20T18:49:45.942496+0000","last_became_peered":"2024-09-20T18:49:45.942496+0000","last_unstale":"2024-09-20T19:02:55.363275+0000","last_undegraded":"2024-09-20T19:02:55.363275+0000","last_fullsized":"2024-09-20T19:02:55.363275+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:30.686872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:30.686872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:51.214827+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"262'10","reported_seq":804,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:56.155552+0000","last_change":"2024-09-20T19:02:04.994109+0000","last_active":"2024-09-20T19:02:56.155552+0000","last_peered":"2024-09-20T19:02:56.155552+0000","last_clean":"2024-09-20T19:02:56.155552+0000","last_became_active":"2024-09-20T18:59:05.080183+0000","last_became_peered":"2024-09-20T18:59:05.080183+0000","last_unstale":"2024-09-20T19:02:56.155552+0000","last_undegraded":"2024-09-20T19:02:56.155552+0000","last_fullsized":"2024-09-20T19:02:56.155552+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'10","last_scrub_stamp":"2024-09-20T19:02:04.994047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:04.994047+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:34.082210+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":594,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.360057+0000","last_change":"2024-09-20T19:01:37.900410+0000","last_active":"2024-09-20T19:02:55.360057+0000","last_peered":"2024-09-20T19:02:55.360057+0000","last_clean":"2024-09-20T19:02:55.360057+0000","last_became_active":"2024-09-20T18:49:51.964780+0000","last_became_peered":"2024-09-20T18:49:51.964780+0000","last_unstale":"2024-09-20T19:02:55.360057+0000","last_undegraded":"2024-09-20T19:02:55.360057+0000","last_fullsized":"2024-09-20T19:02:55.360057+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:37.900328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:37.900328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:02:56.513120+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.d","version":"346'4973","reported_seq":5026,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:03:00.968499+0000","last_change":"2024-09-20T19:02:11.829599+0000","last_active":"2024-09-20T19:03:00.968499+0000","last_peered":"2024-09-20T19:03:00.968499+0000","last_clean":"2024-09-20T19:03:00.968499+0000","last_became_active":"2024-09-20T19:02:11.829309+0000","last_became_peered":"2024-09-20T19:02:11.829309+0000","last_unstale":"2024-09-20T19:03:00.968499+0000","last_undegraded":"2024-09-20T19:03:00.968499+0000","last_fullsized":"2024-09-20T19:03:00.968499+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4973,"log_dups_size":0,"ondisk_log_size":4973,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:26.741575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":325910528,"num_objects":4973,"num_object_clones":0,"num_object_copies":14919,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4973,"num_write_kb":318272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"274'29","reported_seq":661,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.360029+0000","last_change":"2024-09-20T19:00:04.810907+0000","last_active":"2024-09-20T19:02:55.360029+0000","last_peered":"2024-09-20T19:02:55.360029+0000","last_clean":"2024-09-20T19:02:55.360029+0000","last_became_active":"2024-09-20T18:49:48.497736+0000","last_became_peered":"2024-09-20T18:49:48.497736+0000","last_unstale":"2024-09-20T19:02:55.360029+0000","last_undegraded":"2024-09-20T19:02:55.360029+0000","last_fullsized":"2024-09-20T19:02:55.360029+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'29","last_scrub_stamp":"2024-09-20T19:00:04.810831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:00:04.810831+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:18.029237+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":613,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.364101+0000","last_change":"2024-09-20T19:02:41.859231+0000","last_active":"2024-09-20T19:02:55.364101+0000","last_peered":"2024-09-20T19:02:55.364101+0000","last_clean":"2024-09-20T19:02:55.364101+0000","last_became_active":"2024-09-20T18:49:45.942253+0000","last_became_peered":"2024-09-20T18:49:45.942253+0000","last_unstale":"2024-09-20T19:02:55.364101+0000","last_undegraded":"2024-09-20T19:02:55.364101+0000","last_fullsized":"2024-09-20T19:02:55.364101+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:41.859168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:41.859168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:03.925342+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":199,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.359156+0000","last_change":"2024-09-20T19:02:27.639446+0000","last_active":"2024-09-20T19:02:55.359156+0000","last_peered":"2024-09-20T19:02:55.359156+0000","last_clean":"2024-09-20T19:02:55.359156+0000","last_became_active":"2024-09-20T18:59:05.085636+0000","last_became_peered":"2024-09-20T18:59:05.085636+0000","last_unstale":"2024-09-20T19:02:55.359156+0000","last_undegraded":"2024-09-20T19:02:55.359156+0000","last_fullsized":"2024-09-20T19:02:55.359156+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:27.639384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:27.639384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:29.637433+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":631,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.362977+0000","last_change":"2024-09-20T19:02:23.886386+0000","last_active":"2024-09-20T19:02:55.362977+0000","last_peered":"2024-09-20T19:02:55.362977+0000","last_clean":"2024-09-20T19:02:55.362977+0000","last_became_active":"2024-09-20T18:59:05.085131+0000","last_became_peered":"2024-09-20T18:59:05.085131+0000","last_unstale":"2024-09-20T19:02:55.362977+0000","last_undegraded":"2024-09-20T19:02:55.362977+0000","last_fullsized":"2024-09-20T19:02:55.362977+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:23.886308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:23.886308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:42.873874+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"346'4708","reported_seq":4761,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:03:00.993480+0000","last_change":"2024-09-20T19:02:11.827946+0000","last_active":"2024-09-20T19:03:00.993480+0000","last_peered":"2024-09-20T19:03:00.993480+0000","last_clean":"2024-09-20T19:03:00.993480+0000","last_became_active":"2024-09-20T19:02:11.827305+0000","last_became_peered":"2024-09-20T19:02:11.827305+0000","last_unstale":"2024-09-20T19:03:00.993480+0000","last_undegraded":"2024-09-20T19:03:00.993480+0000","last_fullsized":"2024-09-20T19:03:00.993480+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4708,"log_dups_size":0,"ondisk_log_size":4708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:14.570559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":308543488,"num_objects":4708,"num_object_clones":0,"num_object_copies":14124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4708,"num_write_kb":301312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"274'40","reported_seq":851,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:56.155243+0000","last_change":"2024-09-20T19:02:36.901021+0000","last_active":"2024-09-20T19:02:56.155243+0000","last_peered":"2024-09-20T19:02:56.155243+0000","last_clean":"2024-09-20T19:02:56.155243+0000","last_became_active":"2024-09-20T18:49:47.955335+0000","last_became_peered":"2024-09-20T18:49:47.955335+0000","last_unstale":"2024-09-20T19:02:56.155243+0000","last_undegraded":"2024-09-20T19:02:56.155243+0000","last_fullsized":"2024-09-20T19:02:56.155243+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'40","last_scrub_stamp":"2024-09-20T19:02:36.900916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:36.900916+0000","objects_scrubbed":10,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:42.732206+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":604,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.362165+0000","last_change":"2024-09-20T19:01:22.844253+0000","last_active":"2024-09-20T19:02:55.362165+0000","last_peered":"2024-09-20T19:02:55.362165+0000","last_clean":"2024-09-20T19:02:55.362165+0000","last_became_active":"2024-09-20T19:01:22.843966+0000","last_became_peered":"2024-09-20T19:01:22.843966+0000","last_unstale":"2024-09-20T19:02:55.362165+0000","last_undegraded":"2024-09-20T19:02:55.362165+0000","last_fullsized":"2024-09-20T19:02:55.362165+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:00:31.630361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:00:31.630361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":669,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.364026+0000","last_change":"2024-09-20T19:01:39.517101+0000","last_active":"2024-09-20T19:02:55.364026+0000","last_peered":"2024-09-20T19:02:55.364026+0000","last_clean":"2024-09-20T19:02:55.364026+0000","last_became_active":"2024-09-20T18:59:05.086786+0000","last_became_peered":"2024-09-20T18:59:05.086786+0000","last_unstale":"2024-09-20T19:02:55.364026+0000","last_undegraded":"2024-09-20T19:02:55.364026+0000","last_fullsized":"2024-09-20T19:02:55.364026+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-20T19:01:39.517017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:01:39.517017+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:08.168757+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.0012781159999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":625,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.361690+0000","last_change":"2024-09-20T19:01:22.852983+0000","last_active":"2024-09-20T19:02:55.361690+0000","last_peered":"2024-09-20T19:02:55.361690+0000","last_clean":"2024-09-20T19:02:55.361690+0000","last_became_active":"2024-09-20T19:01:22.852495+0000","last_became_peered":"2024-09-20T19:01:22.852495+0000","last_unstale":"2024-09-20T19:02:55.361690+0000","last_undegraded":"2024-09-20T19:02:55.361690+0000","last_fullsized":"2024-09-20T19:02:55.361690+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:59.099736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:59.099736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:13.023185+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":604,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.363924+0000","last_change":"2024-09-20T19:02:04.526741+0000","last_active":"2024-09-20T19:02:55.363924+0000","last_peered":"2024-09-20T19:02:55.363924+0000","last_clean":"2024-09-20T19:02:55.363924+0000","last_became_active":"2024-09-20T19:01:22.837728+0000","last_became_peered":"2024-09-20T19:01:22.837728+0000","last_unstale":"2024-09-20T19:02:55.363924+0000","last_undegraded":"2024-09-20T19:02:55.363924+0000","last_fullsized":"2024-09-20T19:02:55.363924+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:04.526683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:04.526683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:22.012800+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"346'4860","reported_seq":4913,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:03:01.348886+0000","last_change":"2024-09-20T19:02:11.825132+0000","last_active":"2024-09-20T19:03:01.348886+0000","last_peered":"2024-09-20T19:03:01.348886+0000","last_clean":"2024-09-20T19:03:01.348886+0000","last_became_active":"2024-09-20T19:02:11.824846+0000","last_became_peered":"2024-09-20T19:02:11.824846+0000","last_unstale":"2024-09-20T19:03:01.348886+0000","last_undegraded":"2024-09-20T19:03:01.348886+0000","last_fullsized":"2024-09-20T19:03:01.348886+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4860,"log_dups_size":0,"ondisk_log_size":4860,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:12.639014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":318504960,"num_objects":4860,"num_object_clones":0,"num_object_copies":14580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4860,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4860,"num_write_kb":311040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"273'17","reported_seq":658,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.365217+0000","last_change":"2024-09-20T19:02:24.853473+0000","last_active":"2024-09-20T19:02:55.365217+0000","last_peered":"2024-09-20T19:02:55.365217+0000","last_clean":"2024-09-20T19:02:55.365217+0000","last_became_active":"2024-09-20T18:49:47.950904+0000","last_became_peered":"2024-09-20T18:49:47.950904+0000","last_unstale":"2024-09-20T19:02:55.365217+0000","last_undegraded":"2024-09-20T19:02:55.365217+0000","last_fullsized":"2024-09-20T19:02:55.365217+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'17","last_scrub_stamp":"2024-09-20T19:02:24.853374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:24.853374+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:46.813829+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":36,"num_read_kb":25,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":643,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.364232+0000","last_change":"2024-09-20T19:02:29.840761+0000","last_active":"2024-09-20T19:02:55.364232+0000","last_peered":"2024-09-20T19:02:55.364232+0000","last_clean":"2024-09-20T19:02:55.364232+0000","last_became_active":"2024-09-20T18:59:05.079789+0000","last_became_peered":"2024-09-20T18:59:05.079789+0000","last_unstale":"2024-09-20T19:02:55.364232+0000","last_undegraded":"2024-09-20T19:02:55.364232+0000","last_fullsized":"2024-09-20T19:02:55.364232+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:29.840601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:29.840601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:39.311309+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":625,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.372539+0000","last_change":"2024-09-20T19:01:58.789304+0000","last_active":"2024-09-20T19:02:55.372539+0000","last_peered":"2024-09-20T19:02:55.372539+0000","last_clean":"2024-09-20T19:02:55.372539+0000","last_became_active":"2024-09-20T18:49:44.937135+0000","last_became_peered":"2024-09-20T18:49:44.937135+0000","last_unstale":"2024-09-20T19:02:55.372539+0000","last_undegraded":"2024-09-20T19:02:55.372539+0000","last_fullsized":"2024-09-20T19:02:55.372539+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:01:58.789251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:01:58.789251+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:25.250777+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00095558800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":600,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.369819+0000","last_change":"2024-09-20T19:01:17.747452+0000","last_active":"2024-09-20T19:02:55.369819+0000","last_peered":"2024-09-20T19:02:55.369819+0000","last_clean":"2024-09-20T19:02:55.369819+0000","last_became_active":"2024-09-20T18:49:49.954424+0000","last_became_peered":"2024-09-20T18:49:49.954424+0000","last_unstale":"2024-09-20T19:02:55.369819+0000","last_undegraded":"2024-09-20T19:02:55.369819+0000","last_fullsized":"2024-09-20T19:02:55.369819+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:17.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:17.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:49.870946+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":611,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.364478+0000","last_change":"2024-09-20T19:01:15.203591+0000","last_active":"2024-09-20T19:02:55.364478+0000","last_peered":"2024-09-20T19:02:55.364478+0000","last_clean":"2024-09-20T19:02:55.364478+0000","last_became_active":"2024-09-20T18:59:05.081459+0000","last_became_peered":"2024-09-20T18:59:05.081459+0000","last_unstale":"2024-09-20T19:02:55.364478+0000","last_undegraded":"2024-09-20T19:02:55.364478+0000","last_fullsized":"2024-09-20T19:02:55.364478+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:15.203529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:15.203529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:52.197162+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"346'4804","reported_seq":4857,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:03:02.366316+0000","last_change":"2024-09-20T19:02:11.828477+0000","last_active":"2024-09-20T19:03:02.366316+0000","last_peered":"2024-09-20T19:03:02.366316+0000","last_clean":"2024-09-20T19:03:02.366316+0000","last_became_active":"2024-09-20T19:02:11.827996+0000","last_became_peered":"2024-09-20T19:02:11.827996+0000","last_unstale":"2024-09-20T19:03:02.366316+0000","last_undegraded":"2024-09-20T19:03:02.366316+0000","last_fullsized":"2024-09-20T19:03:02.366316+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4804,"log_dups_size":0,"ondisk_log_size":4804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:30.922698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":314834944,"num_objects":4804,"num_object_clones":0,"num_object_copies":14412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4804,"num_write_kb":307456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"273'24","reported_seq":662,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.359478+0000","last_change":"2024-09-20T18:59:06.022021+0000","last_active":"2024-09-20T19:02:55.359478+0000","last_peered":"2024-09-20T19:02:55.359478+0000","last_clean":"2024-09-20T19:02:55.359478+0000","last_became_active":"2024-09-20T18:49:47.949230+0000","last_became_peered":"2024-09-20T18:49:47.949230+0000","last_unstale":"2024-09-20T19:02:55.359478+0000","last_undegraded":"2024-09-20T19:02:55.359478+0000","last_fullsized":"2024-09-20T19:02:55.359478+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:59:06.021962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:59:06.021962+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:11.985717+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":633,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.360986+0000","last_change":"2024-09-20T19:02:24.984120+0000","last_active":"2024-09-20T19:02:55.360986+0000","last_peered":"2024-09-20T19:02:55.360986+0000","last_clean":"2024-09-20T19:02:55.360986+0000","last_became_active":"2024-09-20T19:01:22.845659+0000","last_became_peered":"2024-09-20T19:01:22.845659+0000","last_unstale":"2024-09-20T19:02:55.360986+0000","last_undegraded":"2024-09-20T19:02:55.360986+0000","last_fullsized":"2024-09-20T19:02:55.360986+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:24.984022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:24.984022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:27.154123+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":1422,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:59.450421+0000","last_change":"2024-09-20T19:02:02.494344+0000","last_active":"2024-09-20T19:02:59.450421+0000","last_peered":"2024-09-20T19:02:59.450421+0000","last_clean":"2024-09-20T19:02:59.450421+0000","last_became_active":"2024-09-20T18:49:44.936819+0000","last_became_peered":"2024-09-20T18:49:44.936819+0000","last_unstale":"2024-09-20T19:02:59.450421+0000","last_undegraded":"2024-09-20T19:02:59.450421+0000","last_fullsized":"2024-09-20T19:02:59.450421+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-20T19:02:02.494291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:02:02.494291+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:09.952437+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0010996199999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":789,"num_read_kb":784,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":621,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.364254+0000","last_change":"2024-09-20T19:02:27.785566+0000","last_active":"2024-09-20T19:02:55.364254+0000","last_peered":"2024-09-20T19:02:55.364254+0000","last_clean":"2024-09-20T19:02:55.364254+0000","last_became_active":"2024-09-20T18:49:49.961066+0000","last_became_peered":"2024-09-20T18:49:49.961066+0000","last_unstale":"2024-09-20T19:02:55.364254+0000","last_undegraded":"2024-09-20T19:02:55.364254+0000","last_fullsized":"2024-09-20T19:02:55.364254+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:27.785524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:27.785524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:48.498566+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":617,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.363733+0000","last_change":"2024-09-20T19:02:26.861253+0000","last_active":"2024-09-20T19:02:55.363733+0000","last_peered":"2024-09-20T19:02:55.363733+0000","last_clean":"2024-09-20T19:02:55.363733+0000","last_became_active":"2024-09-20T18:49:51.970343+0000","last_became_peered":"2024-09-20T18:49:51.970343+0000","last_unstale":"2024-09-20T19:02:55.363733+0000","last_undegraded":"2024-09-20T19:02:55.363733+0000","last_fullsized":"2024-09-20T19:02:55.363733+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:26.861185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:26.861185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:38.453435+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"346'4826","reported_seq":4879,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:03:01.014836+0000","last_change":"2024-09-20T19:02:11.828065+0000","last_active":"2024-09-20T19:03:01.014836+0000","last_peered":"2024-09-20T19:03:01.014836+0000","last_clean":"2024-09-20T19:03:01.014836+0000","last_became_active":"2024-09-20T19:02:11.827628+0000","last_became_peered":"2024-09-20T19:02:11.827628+0000","last_unstale":"2024-09-20T19:03:01.014836+0000","last_undegraded":"2024-09-20T19:03:01.014836+0000","last_fullsized":"2024-09-20T19:03:01.014836+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4826,"log_dups_size":0,"ondisk_log_size":4826,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:21.892715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":316276736,"num_objects":4826,"num_object_clones":0,"num_object_copies":14478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4826,"num_write_kb":308864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"308'32","reported_seq":287,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.359516+0000","last_change":"2024-09-20T19:02:25.699418+0000","last_active":"2024-09-20T19:02:55.359516+0000","last_peered":"2024-09-20T19:02:55.359516+0000","last_clean":"2024-09-20T19:02:55.359516+0000","last_became_active":"2024-09-20T19:01:22.846100+0000","last_became_peered":"2024-09-20T19:01:22.846100+0000","last_unstale":"2024-09-20T19:02:55.359516+0000","last_undegraded":"2024-09-20T19:02:55.359516+0000","last_fullsized":"2024-09-20T19:02:55.359516+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"308'32","last_scrub_stamp":"2024-09-20T19:02:25.699337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:25.699337+0000","objects_scrubbed":8,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:28.007253+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":635,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.364283+0000","last_change":"2024-09-20T19:02:31.965132+0000","last_active":"2024-09-20T19:02:55.364283+0000","last_peered":"2024-09-20T19:02:55.364283+0000","last_clean":"2024-09-20T19:02:55.364283+0000","last_became_active":"2024-09-20T18:59:05.079111+0000","last_became_peered":"2024-09-20T18:59:05.079111+0000","last_unstale":"2024-09-20T19:02:55.364283+0000","last_undegraded":"2024-09-20T19:02:55.364283+0000","last_fullsized":"2024-09-20T19:02:55.364283+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:31.965066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:31.965066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:50.572885+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":681,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.365028+0000","last_change":"2024-09-20T18:59:05.108551+0000","last_active":"2024-09-20T19:02:55.365028+0000","last_peered":"2024-09-20T19:02:55.365028+0000","last_clean":"2024-09-20T19:02:55.365028+0000","last_became_active":"2024-09-20T18:59:05.086103+0000","last_became_peered":"2024-09-20T18:59:05.086103+0000","last_unstale":"2024-09-20T19:02:55.365028+0000","last_undegraded":"2024-09-20T19:02:55.365028+0000","last_fullsized":"2024-09-20T19:02:55.365028+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T18:55:13.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T18:55:13.696607+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:10.266543+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":598,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.365261+0000","last_change":"2024-09-20T18:58:23.545370+0000","last_active":"2024-09-20T19:02:55.365261+0000","last_peered":"2024-09-20T19:02:55.365261+0000","last_clean":"2024-09-20T19:02:55.365261+0000","last_became_active":"2024-09-20T18:49:49.956128+0000","last_became_peered":"2024-09-20T18:49:49.956128+0000","last_unstale":"2024-09-20T19:02:55.365261+0000","last_undegraded":"2024-09-20T19:02:55.365261+0000","last_fullsized":"2024-09-20T19:02:55.365261+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:23.545321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:23.545321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:12.296749+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":628,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.364087+0000","last_change":"2024-09-20T19:01:30.557430+0000","last_active":"2024-09-20T19:02:55.364087+0000","last_peered":"2024-09-20T19:02:55.364087+0000","last_clean":"2024-09-20T19:02:55.364087+0000","last_became_active":"2024-09-20T19:01:22.847692+0000","last_became_peered":"2024-09-20T19:01:22.847692+0000","last_unstale":"2024-09-20T19:02:55.364087+0000","last_undegraded":"2024-09-20T19:02:55.364087+0000","last_fullsized":"2024-09-20T19:02:55.364087+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:30.557368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:30.557368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:47.782922+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.0","version":"346'4815","reported_seq":4867,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:03:01.027687+0000","last_change":"2024-09-20T19:02:11.825018+0000","last_active":"2024-09-20T19:03:01.027687+0000","last_peered":"2024-09-20T19:03:01.027687+0000","last_clean":"2024-09-20T19:03:01.027687+0000","last_became_active":"2024-09-20T19:02:11.824703+0000","last_became_peered":"2024-09-20T19:02:11.824703+0000","last_unstale":"2024-09-20T19:03:01.027687+0000","last_undegraded":"2024-09-20T19:03:01.027687+0000","last_fullsized":"2024-09-20T19:03:01.027687+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4815,"log_dups_size":0,"ondisk_log_size":4815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:38.061938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":315555840,"num_objects":4815,"num_object_clones":0,"num_object_copies":14445,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4815,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4815,"num_write_kb":308160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"274'19","reported_seq":622,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.363591+0000","last_change":"2024-09-20T19:01:22.843592+0000","last_active":"2024-09-20T19:02:55.363591+0000","last_peered":"2024-09-20T19:02:55.363591+0000","last_clean":"2024-09-20T19:02:55.363591+0000","last_became_active":"2024-09-20T19:01:22.843143+0000","last_became_peered":"2024-09-20T19:01:22.843143+0000","last_unstale":"2024-09-20T19:02:55.363591+0000","last_undegraded":"2024-09-20T19:02:55.363591+0000","last_fullsized":"2024-09-20T19:02:55.363591+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:58:24.616580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:24.616580+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:53.172764+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":582,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.361830+0000","last_change":"2024-09-20T19:01:22.837818+0000","last_active":"2024-09-20T19:02:55.361830+0000","last_peered":"2024-09-20T19:02:55.361830+0000","last_clean":"2024-09-20T19:02:55.361830+0000","last_became_active":"2024-09-20T19:01:22.836557+0000","last_became_peered":"2024-09-20T19:01:22.836557+0000","last_unstale":"2024-09-20T19:02:55.361830+0000","last_undegraded":"2024-09-20T19:02:55.361830+0000","last_fullsized":"2024-09-20T19:02:55.361830+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.693993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.693993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:48.100558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":645,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.363895+0000","last_change":"2024-09-20T19:01:38.463472+0000","last_active":"2024-09-20T19:02:55.363895+0000","last_peered":"2024-09-20T19:02:55.363895+0000","last_clean":"2024-09-20T19:02:55.363895+0000","last_became_active":"2024-09-20T19:01:22.831488+0000","last_became_peered":"2024-09-20T19:01:22.831488+0000","last_unstale":"2024-09-20T19:02:55.363895+0000","last_undegraded":"2024-09-20T19:02:55.363895+0000","last_fullsized":"2024-09-20T19:02:55.363895+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:38.463404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:38.463404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:51.830473+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":614,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.362581+0000","last_change":"2024-09-20T18:59:05.080255+0000","last_active":"2024-09-20T19:02:55.362581+0000","last_peered":"2024-09-20T19:02:55.362581+0000","last_clean":"2024-09-20T19:02:55.362581+0000","last_became_active":"2024-09-20T18:59:05.079897+0000","last_became_peered":"2024-09-20T18:59:05.079897+0000","last_unstale":"2024-09-20T19:02:55.362581+0000","last_undegraded":"2024-09-20T19:02:55.362581+0000","last_fullsized":"2024-09-20T19:02:55.362581+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:01.083156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:59:01.083156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:42.880064+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"346'4919","reported_seq":4972,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:03:01.382703+0000","last_change":"2024-09-20T19:02:11.825263+0000","last_active":"2024-09-20T19:03:01.382703+0000","last_peered":"2024-09-20T19:03:01.382703+0000","last_clean":"2024-09-20T19:03:01.382703+0000","last_became_active":"2024-09-20T19:02:11.825011+0000","last_became_peered":"2024-09-20T19:02:11.825011+0000","last_unstale":"2024-09-20T19:03:01.382703+0000","last_undegraded":"2024-09-20T19:03:01.382703+0000","last_fullsized":"2024-09-20T19:03:01.382703+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4919,"log_dups_size":0,"ondisk_log_size":4919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:12.092479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":322371584,"num_objects":4919,"num_object_clones":0,"num_object_copies":14757,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4919,"num_write_kb":314816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"274'31","reported_seq":629,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.366112+0000","last_change":"2024-09-20T19:01:22.838956+0000","last_active":"2024-09-20T19:02:55.366112+0000","last_peered":"2024-09-20T19:02:55.366112+0000","last_clean":"2024-09-20T19:02:55.366112+0000","last_became_active":"2024-09-20T19:01:22.838577+0000","last_became_peered":"2024-09-20T19:01:22.838577+0000","last_unstale":"2024-09-20T19:02:55.366112+0000","last_undegraded":"2024-09-20T19:02:55.366112+0000","last_fullsized":"2024-09-20T19:02:55.366112+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'31","last_scrub_stamp":"2024-09-20T19:00:03.509980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:00:03.509980+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":49,"num_read_kb":39,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":202,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.359296+0000","last_change":"2024-09-20T19:01:42.479567+0000","last_active":"2024-09-20T19:02:55.359296+0000","last_peered":"2024-09-20T19:02:55.359296+0000","last_clean":"2024-09-20T19:02:55.359296+0000","last_became_active":"2024-09-20T19:01:22.845553+0000","last_became_peered":"2024-09-20T19:01:22.845553+0000","last_unstale":"2024-09-20T19:02:55.359296+0000","last_undegraded":"2024-09-20T19:02:55.359296+0000","last_fullsized":"2024-09-20T19:02:55.359296+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:42.479489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:42.479489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:54.774853+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":623,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.363829+0000","last_change":"2024-09-20T19:02:32.856930+0000","last_active":"2024-09-20T19:02:55.363829+0000","last_peered":"2024-09-20T19:02:55.363829+0000","last_clean":"2024-09-20T19:02:55.363829+0000","last_became_active":"2024-09-20T19:01:22.837146+0000","last_became_peered":"2024-09-20T19:01:22.837146+0000","last_unstale":"2024-09-20T19:02:55.363829+0000","last_undegraded":"2024-09-20T19:02:55.363829+0000","last_fullsized":"2024-09-20T19:02:55.363829+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:32.856856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:32.856856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:59.715742+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":622,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.370389+0000","last_change":"2024-09-20T19:01:31.687258+0000","last_active":"2024-09-20T19:02:55.370389+0000","last_peered":"2024-09-20T19:02:55.370389+0000","last_clean":"2024-09-20T19:02:55.370389+0000","last_became_active":"2024-09-20T19:01:22.833913+0000","last_became_peered":"2024-09-20T19:01:22.833913+0000","last_unstale":"2024-09-20T19:02:55.370389+0000","last_undegraded":"2024-09-20T19:02:55.370389+0000","last_fullsized":"2024-09-20T19:02:55.370389+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:31.687188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:31.687188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:44.818347+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2","version":"346'4921","reported_seq":4974,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:03:01.907088+0000","last_change":"2024-09-20T19:02:11.829938+0000","last_active":"2024-09-20T19:03:01.907088+0000","last_peered":"2024-09-20T19:03:01.907088+0000","last_clean":"2024-09-20T19:03:01.907088+0000","last_became_active":"2024-09-20T19:02:11.829572+0000","last_became_peered":"2024-09-20T19:02:11.829572+0000","last_unstale":"2024-09-20T19:03:01.907088+0000","last_undegraded":"2024-09-20T19:03:01.907088+0000","last_fullsized":"2024-09-20T19:03:01.907088+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4921,"log_dups_size":0,"ondisk_log_size":4921,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:34.876687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":322502656,"num_objects":4921,"num_object_clones":0,"num_object_copies":14763,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4921,"num_write_kb":314944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"273'20","reported_seq":627,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.365297+0000","last_change":"2024-09-20T19:01:22.850846+0000","last_active":"2024-09-20T19:02:55.365297+0000","last_peered":"2024-09-20T19:02:55.365297+0000","last_clean":"2024-09-20T19:02:55.365297+0000","last_became_active":"2024-09-20T19:01:22.848851+0000","last_became_peered":"2024-09-20T19:01:22.848851+0000","last_unstale":"2024-09-20T19:02:55.365297+0000","last_undegraded":"2024-09-20T19:02:55.365297+0000","last_fullsized":"2024-09-20T19:02:55.365297+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:59:10.364583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:59:10.364583+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":631,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.366955+0000","last_change":"2024-09-20T19:02:34.986113+0000","last_active":"2024-09-20T19:02:55.366955+0000","last_peered":"2024-09-20T19:02:55.366955+0000","last_clean":"2024-09-20T19:02:55.366955+0000","last_became_active":"2024-09-20T19:01:22.835966+0000","last_became_peered":"2024-09-20T19:01:22.835966+0000","last_unstale":"2024-09-20T19:02:55.366955+0000","last_undegraded":"2024-09-20T19:02:55.366955+0000","last_fullsized":"2024-09-20T19:02:55.366955+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:34.985163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:34.985163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:44.611937+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":791,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:56.155829+0000","last_change":"2024-09-20T19:02:02.813926+0000","last_active":"2024-09-20T19:02:56.155829+0000","last_peered":"2024-09-20T19:02:56.155829+0000","last_clean":"2024-09-20T19:02:56.155829+0000","last_became_active":"2024-09-20T18:49:49.957005+0000","last_became_peered":"2024-09-20T18:49:49.957005+0000","last_unstale":"2024-09-20T19:02:56.155829+0000","last_undegraded":"2024-09-20T19:02:56.155829+0000","last_fullsized":"2024-09-20T19:02:56.155829+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T19:02:02.813865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:02.813865+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:31.557168+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":556,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.363795+0000","last_change":"2024-09-20T19:01:22.843904+0000","last_active":"2024-09-20T19:02:55.363795+0000","last_peered":"2024-09-20T19:02:55.363795+0000","last_clean":"2024-09-20T19:02:55.363795+0000","last_became_active":"2024-09-20T19:01:22.843167+0000","last_became_peered":"2024-09-20T19:01:22.843167+0000","last_unstale":"2024-09-20T19:02:55.363795+0000","last_undegraded":"2024-09-20T19:02:55.363795+0000","last_fullsized":"2024-09-20T19:02:55.363795+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:48.398542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:59:48.398542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.5","version":"346'4777","reported_seq":4830,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:03:01.319052+0000","last_change":"2024-09-20T19:02:11.828434+0000","last_active":"2024-09-20T19:03:01.319052+0000","last_peered":"2024-09-20T19:03:01.319052+0000","last_clean":"2024-09-20T19:03:01.319052+0000","last_became_active":"2024-09-20T19:02:11.828162+0000","last_became_peered":"2024-09-20T19:02:11.828162+0000","last_unstale":"2024-09-20T19:03:01.319052+0000","last_undegraded":"2024-09-20T19:03:01.319052+0000","last_fullsized":"2024-09-20T19:03:01.319052+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4777,"log_dups_size":0,"ondisk_log_size":4777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:13.184255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":313065472,"num_objects":4777,"num_object_clones":0,"num_object_copies":14331,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4777,"num_write_kb":305728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"274'15","reported_seq":653,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.359935+0000","last_change":"2024-09-20T19:01:27.802153+0000","last_active":"2024-09-20T19:02:55.359935+0000","last_peered":"2024-09-20T19:02:55.359935+0000","last_clean":"2024-09-20T19:02:55.359935+0000","last_became_active":"2024-09-20T19:01:22.845683+0000","last_became_peered":"2024-09-20T19:01:22.845683+0000","last_unstale":"2024-09-20T19:02:55.359935+0000","last_undegraded":"2024-09-20T19:02:55.359935+0000","last_fullsized":"2024-09-20T19:02:55.359935+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'15","last_scrub_stamp":"2024-09-20T19:01:27.802100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:27.802100+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":157,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.368579+0000","last_change":"2024-09-20T19:01:52.716074+0000","last_active":"2024-09-20T19:02:55.368579+0000","last_peered":"2024-09-20T19:02:55.368579+0000","last_clean":"2024-09-20T19:02:55.368579+0000","last_became_active":"2024-09-20T19:00:01.190875+0000","last_became_peered":"2024-09-20T19:00:01.190875+0000","last_unstale":"2024-09-20T19:02:55.368579+0000","last_undegraded":"2024-09-20T19:02:55.368579+0000","last_fullsized":"2024-09-20T19:02:55.368579+0000","mapping_epoch":277,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":278,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:52.716025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:52.716025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:08.654757+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":627,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.370827+0000","last_change":"2024-09-20T19:02:24.939055+0000","last_active":"2024-09-20T19:02:55.370827+0000","last_peered":"2024-09-20T19:02:55.370827+0000","last_clean":"2024-09-20T19:02:55.370827+0000","last_became_active":"2024-09-20T19:01:22.835347+0000","last_became_peered":"2024-09-20T19:01:22.835347+0000","last_unstale":"2024-09-20T19:02:55.370827+0000","last_undegraded":"2024-09-20T19:02:55.370827+0000","last_fullsized":"2024-09-20T19:02:55.370827+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:24.938980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:24.938980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:52.810786+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":610,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.360934+0000","last_change":"2024-09-20T19:02:31.120809+0000","last_active":"2024-09-20T19:02:55.360934+0000","last_peered":"2024-09-20T19:02:55.360934+0000","last_clean":"2024-09-20T19:02:55.360934+0000","last_became_active":"2024-09-20T18:49:51.964098+0000","last_became_peered":"2024-09-20T18:49:51.964098+0000","last_unstale":"2024-09-20T19:02:55.360934+0000","last_undegraded":"2024-09-20T19:02:55.360934+0000","last_fullsized":"2024-09-20T19:02:55.360934+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:31.120623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:31.120623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:41.294006+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"346'4902","reported_seq":4955,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:03:02.373507+0000","last_change":"2024-09-20T19:02:11.829578+0000","last_active":"2024-09-20T19:03:02.373507+0000","last_peered":"2024-09-20T19:03:02.373507+0000","last_clean":"2024-09-20T19:03:02.373507+0000","last_became_active":"2024-09-20T19:02:11.828890+0000","last_became_peered":"2024-09-20T19:02:11.828890+0000","last_unstale":"2024-09-20T19:03:02.373507+0000","last_undegraded":"2024-09-20T19:03:02.373507+0000","last_fullsized":"2024-09-20T19:03:02.373507+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4902,"log_dups_size":0,"ondisk_log_size":4902,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:19.809730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":321257472,"num_objects":4902,"num_object_clones":0,"num_object_copies":14706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4902,"num_write_kb":313728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"274'35","reported_seq":235,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.359057+0000","last_change":"2024-09-20T19:01:40.477072+0000","last_active":"2024-09-20T19:02:55.359057+0000","last_peered":"2024-09-20T19:02:55.359057+0000","last_clean":"2024-09-20T19:02:55.359057+0000","last_became_active":"2024-09-20T18:59:05.087391+0000","last_became_peered":"2024-09-20T18:59:05.087391+0000","last_unstale":"2024-09-20T19:02:55.359057+0000","last_undegraded":"2024-09-20T19:02:55.359057+0000","last_fullsized":"2024-09-20T19:02:55.359057+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'35","last_scrub_stamp":"2024-09-20T19:01:40.476968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:40.476968+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:09.580262+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":635,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.364875+0000","last_change":"2024-09-20T19:02:05.440745+0000","last_active":"2024-09-20T19:02:55.364875+0000","last_peered":"2024-09-20T19:02:55.364875+0000","last_clean":"2024-09-20T19:02:55.364875+0000","last_became_active":"2024-09-20T18:59:05.086670+0000","last_became_peered":"2024-09-20T18:59:05.086670+0000","last_unstale":"2024-09-20T19:02:55.364875+0000","last_undegraded":"2024-09-20T19:02:55.364875+0000","last_fullsized":"2024-09-20T19:02:55.364875+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:05.440677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:05.440677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:22.708355+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":623,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.368887+0000","last_change":"2024-09-20T19:02:42.745448+0000","last_active":"2024-09-20T19:02:55.368887+0000","last_peered":"2024-09-20T19:02:55.368887+0000","last_clean":"2024-09-20T19:02:55.368887+0000","last_became_active":"2024-09-20T18:49:49.957558+0000","last_became_peered":"2024-09-20T18:49:49.957558+0000","last_unstale":"2024-09-20T19:02:55.368887+0000","last_undegraded":"2024-09-20T19:02:55.368887+0000","last_fullsized":"2024-09-20T19:02:55.368887+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:42.745372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:42.745372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:49.093207+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":602,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.364977+0000","last_change":"2024-09-20T18:53:47.370152+0000","last_active":"2024-09-20T19:02:55.364977+0000","last_peered":"2024-09-20T19:02:55.364977+0000","last_clean":"2024-09-20T19:02:55.364977+0000","last_became_active":"2024-09-20T18:49:51.970798+0000","last_became_peered":"2024-09-20T18:49:51.970798+0000","last_unstale":"2024-09-20T19:02:55.364977+0000","last_undegraded":"2024-09-20T19:02:55.364977+0000","last_fullsized":"2024-09-20T19:02:55.364977+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:47.370108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:47.370108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:19.239137+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.7","version":"346'4832","reported_seq":4885,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:03:01.015475+0000","last_change":"2024-09-20T19:02:11.827681+0000","last_active":"2024-09-20T19:03:01.015475+0000","last_peered":"2024-09-20T19:03:01.015475+0000","last_clean":"2024-09-20T19:03:01.015475+0000","last_became_active":"2024-09-20T19:02:11.827360+0000","last_became_peered":"2024-09-20T19:02:11.827360+0000","last_unstale":"2024-09-20T19:03:01.015475+0000","last_undegraded":"2024-09-20T19:03:01.015475+0000","last_fullsized":"2024-09-20T19:03:01.015475+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4832,"log_dups_size":0,"ondisk_log_size":4832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:31.729709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":316669952,"num_objects":4832,"num_object_clones":0,"num_object_copies":14496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4832,"num_write_kb":309248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"273'18","reported_seq":592,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.362025+0000","last_change":"2024-09-20T19:01:22.847461+0000","last_active":"2024-09-20T19:02:55.362025+0000","last_peered":"2024-09-20T19:02:55.362025+0000","last_clean":"2024-09-20T19:02:55.362025+0000","last_became_active":"2024-09-20T19:01:22.847140+0000","last_became_peered":"2024-09-20T19:01:22.847140+0000","last_unstale":"2024-09-20T19:02:55.362025+0000","last_undegraded":"2024-09-20T19:02:55.362025+0000","last_fullsized":"2024-09-20T19:02:55.362025+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'18","last_scrub_stamp":"2024-09-20T18:59:55.490086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:59:55.490086+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":632,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.365450+0000","last_change":"2024-09-20T19:01:22.844920+0000","last_active":"2024-09-20T19:02:55.365450+0000","last_peered":"2024-09-20T19:02:55.365450+0000","last_clean":"2024-09-20T19:02:55.365450+0000","last_became_active":"2024-09-20T19:01:22.844608+0000","last_became_peered":"2024-09-20T19:01:22.844608+0000","last_unstale":"2024-09-20T19:02:55.365450+0000","last_undegraded":"2024-09-20T19:02:55.365450+0000","last_fullsized":"2024-09-20T19:02:55.365450+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:00:34.940850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:00:34.940850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:47.134749+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":616,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.368671+0000","last_change":"2024-09-20T19:01:53.734774+0000","last_active":"2024-09-20T19:02:55.368671+0000","last_peered":"2024-09-20T19:02:55.368671+0000","last_clean":"2024-09-20T19:02:55.368671+0000","last_became_active":"2024-09-20T18:49:49.954944+0000","last_became_peered":"2024-09-20T18:49:49.954944+0000","last_unstale":"2024-09-20T19:02:55.368671+0000","last_undegraded":"2024-09-20T19:02:55.368671+0000","last_fullsized":"2024-09-20T19:02:55.368671+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:53.734726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:53.734726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:21.517887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":608,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.366426+0000","last_change":"2024-09-20T19:02:35.968178+0000","last_active":"2024-09-20T19:02:55.366426+0000","last_peered":"2024-09-20T19:02:55.366426+0000","last_clean":"2024-09-20T19:02:55.366426+0000","last_became_active":"2024-09-20T18:49:51.972970+0000","last_became_peered":"2024-09-20T18:49:51.972970+0000","last_unstale":"2024-09-20T19:02:55.366426+0000","last_undegraded":"2024-09-20T19:02:55.366426+0000","last_fullsized":"2024-09-20T19:02:55.366426+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:35.968052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:35.968052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:53.629726+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.6","version":"346'4898","reported_seq":4951,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:03:01.359956+0000","last_change":"2024-09-20T19:02:11.829337+0000","last_active":"2024-09-20T19:03:01.359956+0000","last_peered":"2024-09-20T19:03:01.359956+0000","last_clean":"2024-09-20T19:03:01.359956+0000","last_became_active":"2024-09-20T19:02:11.829014+0000","last_became_peered":"2024-09-20T19:02:11.829014+0000","last_unstale":"2024-09-20T19:03:01.359956+0000","last_undegraded":"2024-09-20T19:03:01.359956+0000","last_fullsized":"2024-09-20T19:03:01.359956+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4898,"log_dups_size":0,"ondisk_log_size":4898,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:29.577085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":320995328,"num_objects":4898,"num_object_clones":0,"num_object_copies":14694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4898,"num_write_kb":313472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"273'27","reported_seq":714,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.362205+0000","last_change":"2024-09-20T19:02:39.709344+0000","last_active":"2024-09-20T19:02:55.362205+0000","last_peered":"2024-09-20T19:02:55.362205+0000","last_clean":"2024-09-20T19:02:55.362205+0000","last_became_active":"2024-09-20T18:59:05.083726+0000","last_became_peered":"2024-09-20T18:59:05.083726+0000","last_unstale":"2024-09-20T19:02:55.362205+0000","last_undegraded":"2024-09-20T19:02:55.362205+0000","last_fullsized":"2024-09-20T19:02:55.362205+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'27","last_scrub_stamp":"2024-09-20T19:02:39.709269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:39.709269+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:44.315182+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":652,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.364670+0000","last_change":"2024-09-20T19:01:22.830278+0000","last_active":"2024-09-20T19:02:55.364670+0000","last_peered":"2024-09-20T19:02:55.364670+0000","last_clean":"2024-09-20T19:02:55.364670+0000","last_became_active":"2024-09-20T19:01:22.829928+0000","last_became_peered":"2024-09-20T19:01:22.829928+0000","last_unstale":"2024-09-20T19:02:55.364670+0000","last_undegraded":"2024-09-20T19:02:55.364670+0000","last_fullsized":"2024-09-20T19:02:55.364670+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-20T18:58:57.563058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:57.563058+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:45.184378+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"308'12","reported_seq":801,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:56.160255+0000","last_change":"2024-09-20T19:01:22.833874+0000","last_active":"2024-09-20T19:02:56.160255+0000","last_peered":"2024-09-20T19:02:56.160255+0000","last_clean":"2024-09-20T19:02:56.160255+0000","last_became_active":"2024-09-20T19:01:22.832710+0000","last_became_peered":"2024-09-20T19:01:22.832710+0000","last_unstale":"2024-09-20T19:02:56.160255+0000","last_undegraded":"2024-09-20T19:02:56.160255+0000","last_fullsized":"2024-09-20T19:02:56.160255+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'9","last_scrub_stamp":"2024-09-20T18:58:27.521051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:27.521051+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:21.248521+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":623,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.363722+0000","last_change":"2024-09-20T19:01:56.634132+0000","last_active":"2024-09-20T19:02:55.363722+0000","last_peered":"2024-09-20T19:02:55.363722+0000","last_clean":"2024-09-20T19:02:55.363722+0000","last_became_active":"2024-09-20T18:59:05.087005+0000","last_became_peered":"2024-09-20T18:59:05.087005+0000","last_unstale":"2024-09-20T19:02:55.363722+0000","last_undegraded":"2024-09-20T19:02:55.363722+0000","last_fullsized":"2024-09-20T19:02:55.363722+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:56.634089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:56.634089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:05.408503+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":191,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.357929+0000","last_change":"2024-09-20T19:01:16.542398+0000","last_active":"2024-09-20T19:02:55.357929+0000","last_peered":"2024-09-20T19:02:55.357929+0000","last_clean":"2024-09-20T19:02:55.357929+0000","last_became_active":"2024-09-20T18:59:05.085427+0000","last_became_peered":"2024-09-20T18:59:05.085427+0000","last_unstale":"2024-09-20T19:02:55.357929+0000","last_undegraded":"2024-09-20T19:02:55.357929+0000","last_fullsized":"2024-09-20T19:02:55.357929+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:16.542335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:16.542335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:47.734953+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"273'12","reported_seq":634,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.364162+0000","last_change":"2024-09-20T19:01:12.385223+0000","last_active":"2024-09-20T19:02:55.364162+0000","last_peered":"2024-09-20T19:02:55.364162+0000","last_clean":"2024-09-20T19:02:55.364162+0000","last_became_active":"2024-09-20T18:49:48.500662+0000","last_became_peered":"2024-09-20T18:49:48.500662+0000","last_unstale":"2024-09-20T19:02:55.364162+0000","last_undegraded":"2024-09-20T19:02:55.364162+0000","last_fullsized":"2024-09-20T19:02:55.364162+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'12","last_scrub_stamp":"2024-09-20T19:01:12.385152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:12.385152+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":612,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.362534+0000","last_change":"2024-09-20T19:01:14.466096+0000","last_active":"2024-09-20T19:02:55.362534+0000","last_peered":"2024-09-20T19:02:55.362534+0000","last_clean":"2024-09-20T19:02:55.362534+0000","last_became_active":"2024-09-20T18:49:49.956749+0000","last_became_peered":"2024-09-20T18:49:49.956749+0000","last_unstale":"2024-09-20T19:02:55.362534+0000","last_undegraded":"2024-09-20T19:02:55.362534+0000","last_fullsized":"2024-09-20T19:02:55.362534+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:14.466042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:14.466042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:15.787734+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":634,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.364372+0000","last_change":"2024-09-20T19:02:03.509954+0000","last_active":"2024-09-20T19:02:55.364372+0000","last_peered":"2024-09-20T19:02:55.364372+0000","last_clean":"2024-09-20T19:02:55.364372+0000","last_became_active":"2024-09-20T19:01:22.830233+0000","last_became_peered":"2024-09-20T19:01:22.830233+0000","last_unstale":"2024-09-20T19:02:55.364372+0000","last_undegraded":"2024-09-20T19:02:55.364372+0000","last_fullsized":"2024-09-20T19:02:55.364372+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:03.509903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:03.509903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:11.556992+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"273'15","reported_seq":634,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.359550+0000","last_change":"2024-09-20T19:01:15.875779+0000","last_active":"2024-09-20T19:02:55.359550+0000","last_peered":"2024-09-20T19:02:55.359550+0000","last_clean":"2024-09-20T19:02:55.359550+0000","last_became_active":"2024-09-20T18:49:48.500803+0000","last_became_peered":"2024-09-20T18:49:48.500803+0000","last_unstale":"2024-09-20T19:02:55.359550+0000","last_undegraded":"2024-09-20T19:02:55.359550+0000","last_fullsized":"2024-09-20T19:02:55.359550+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:01:15.875693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:15.875693+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:49.127502+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":638,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.365061+0000","last_change":"2024-09-20T19:01:22.834162+0000","last_active":"2024-09-20T19:02:55.365061+0000","last_peered":"2024-09-20T19:02:55.365061+0000","last_clean":"2024-09-20T19:02:55.365061+0000","last_became_active":"2024-09-20T19:01:22.831615+0000","last_became_peered":"2024-09-20T19:01:22.831615+0000","last_unstale":"2024-09-20T19:02:55.365061+0000","last_undegraded":"2024-09-20T19:02:55.365061+0000","last_fullsized":"2024-09-20T19:02:55.365061+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:59.589891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:59.589891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:43.164641+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":612,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.364021+0000","last_change":"2024-09-20T19:01:06.482291+0000","last_active":"2024-09-20T19:02:55.364021+0000","last_peered":"2024-09-20T19:02:55.364021+0000","last_clean":"2024-09-20T19:02:55.364021+0000","last_became_active":"2024-09-20T18:49:49.961947+0000","last_became_peered":"2024-09-20T18:49:49.961947+0000","last_unstale":"2024-09-20T19:02:55.364021+0000","last_undegraded":"2024-09-20T19:02:55.364021+0000","last_fullsized":"2024-09-20T19:02:55.364021+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:06.482212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:06.482212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"66'1","reported_seq":631,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.368680+0000","last_change":"2024-09-20T19:02:01.843182+0000","last_active":"2024-09-20T19:02:55.368680+0000","last_peered":"2024-09-20T19:02:55.368680+0000","last_clean":"2024-09-20T19:02:55.368680+0000","last_became_active":"2024-09-20T19:01:22.837187+0000","last_became_peered":"2024-09-20T19:01:22.837187+0000","last_unstale":"2024-09-20T19:02:55.368680+0000","last_undegraded":"2024-09-20T19:02:55.368680+0000","last_fullsized":"2024-09-20T19:02:55.368680+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T19:02:01.843111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:01.843111+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:06.311104+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"274'15","reported_seq":633,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.363105+0000","last_change":"2024-09-20T19:01:36.532486+0000","last_active":"2024-09-20T19:02:55.363105+0000","last_peered":"2024-09-20T19:02:55.363105+0000","last_clean":"2024-09-20T19:02:55.363105+0000","last_became_active":"2024-09-20T18:49:48.501011+0000","last_became_peered":"2024-09-20T18:49:48.501011+0000","last_unstale":"2024-09-20T19:02:55.363105+0000","last_undegraded":"2024-09-20T19:02:55.363105+0000","last_fullsized":"2024-09-20T19:02:55.363105+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'15","last_scrub_stamp":"2024-09-20T19:01:36.532419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:36.532419+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:02:57.030996+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":621,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.368725+0000","last_change":"2024-09-20T19:01:30.864866+0000","last_active":"2024-09-20T19:02:55.368725+0000","last_peered":"2024-09-20T19:02:55.368725+0000","last_clean":"2024-09-20T19:02:55.368725+0000","last_became_active":"2024-09-20T18:49:45.938511+0000","last_became_peered":"2024-09-20T18:49:45.938511+0000","last_unstale":"2024-09-20T19:02:55.368725+0000","last_undegraded":"2024-09-20T19:02:55.368725+0000","last_fullsized":"2024-09-20T19:02:55.368725+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:30.864794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:30.864794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":780,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:56.157154+0000","last_change":"2024-09-20T19:00:34.485855+0000","last_active":"2024-09-20T19:02:56.157154+0000","last_peered":"2024-09-20T19:02:56.157154+0000","last_clean":"2024-09-20T19:02:56.157154+0000","last_became_active":"2024-09-20T18:49:49.960550+0000","last_became_peered":"2024-09-20T18:49:49.960550+0000","last_unstale":"2024-09-20T19:02:56.157154+0000","last_undegraded":"2024-09-20T19:02:56.157154+0000","last_fullsized":"2024-09-20T19:02:56.157154+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T19:00:34.485624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:00:34.485624+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:54.909280+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":564,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.362585+0000","last_change":"2024-09-20T19:01:22.848313+0000","last_active":"2024-09-20T19:02:55.362585+0000","last_peered":"2024-09-20T19:02:55.362585+0000","last_clean":"2024-09-20T19:02:55.362585+0000","last_became_active":"2024-09-20T19:01:22.848017+0000","last_became_peered":"2024-09-20T19:01:22.848017+0000","last_unstale":"2024-09-20T19:02:55.362585+0000","last_undegraded":"2024-09-20T19:02:55.362585+0000","last_fullsized":"2024-09-20T19:02:55.362585+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:59.497810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:59:59.497810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"274'20","reported_seq":649,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.362939+0000","last_change":"2024-09-20T18:57:20.701592+0000","last_active":"2024-09-20T19:02:55.362939+0000","last_peered":"2024-09-20T19:02:55.362939+0000","last_clean":"2024-09-20T19:02:55.362939+0000","last_became_active":"2024-09-20T18:49:58.997494+0000","last_became_peered":"2024-09-20T18:49:58.997494+0000","last_unstale":"2024-09-20T19:02:55.362939+0000","last_undegraded":"2024-09-20T19:02:55.362939+0000","last_fullsized":"2024-09-20T19:02:55.362939+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:57:20.701543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:20.701543+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:12.702800+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":624,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.364220+0000","last_change":"2024-09-20T19:01:22.833495+0000","last_active":"2024-09-20T19:02:55.364220+0000","last_peered":"2024-09-20T19:02:55.364220+0000","last_clean":"2024-09-20T19:02:55.364220+0000","last_became_active":"2024-09-20T19:01:22.831291+0000","last_became_peered":"2024-09-20T19:01:22.831291+0000","last_unstale":"2024-09-20T19:02:55.364220+0000","last_undegraded":"2024-09-20T19:02:55.364220+0000","last_fullsized":"2024-09-20T19:02:55.364220+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.632380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.632380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:07.354816+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":780,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:56.155221+0000","last_change":"2024-09-20T19:01:10.601374+0000","last_active":"2024-09-20T19:02:56.155221+0000","last_peered":"2024-09-20T19:02:56.155221+0000","last_clean":"2024-09-20T19:02:56.155221+0000","last_became_active":"2024-09-20T18:49:49.962243+0000","last_became_peered":"2024-09-20T18:49:49.962243+0000","last_unstale":"2024-09-20T19:02:56.155221+0000","last_undegraded":"2024-09-20T19:02:56.155221+0000","last_fullsized":"2024-09-20T19:02:56.155221+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T19:01:10.601300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:10.601300+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:46.785830+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":617,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.359250+0000","last_change":"2024-09-20T19:01:03.821977+0000","last_active":"2024-09-20T19:02:55.359250+0000","last_peered":"2024-09-20T19:02:55.359250+0000","last_clean":"2024-09-20T19:02:55.359250+0000","last_became_active":"2024-09-20T18:49:51.969093+0000","last_became_peered":"2024-09-20T18:49:51.969093+0000","last_unstale":"2024-09-20T19:02:55.359250+0000","last_undegraded":"2024-09-20T19:02:55.359250+0000","last_fullsized":"2024-09-20T19:02:55.359250+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:03.821908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:03.821908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:47.087429+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"273'19","reported_seq":628,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.362309+0000","last_change":"2024-09-20T19:01:22.841541+0000","last_active":"2024-09-20T19:02:55.362309+0000","last_peered":"2024-09-20T19:02:55.362309+0000","last_clean":"2024-09-20T19:02:55.362309+0000","last_became_active":"2024-09-20T19:01:22.841252+0000","last_became_peered":"2024-09-20T19:01:22.841252+0000","last_unstale":"2024-09-20T19:02:55.362309+0000","last_undegraded":"2024-09-20T19:02:55.362309+0000","last_fullsized":"2024-09-20T19:02:55.362309+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:57:50.899514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:50.899514+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:20.035402+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":546,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.362658+0000","last_change":"2024-09-20T19:01:22.838793+0000","last_active":"2024-09-20T19:02:55.362658+0000","last_peered":"2024-09-20T19:02:55.362658+0000","last_clean":"2024-09-20T19:02:55.362658+0000","last_became_active":"2024-09-20T19:01:22.838429+0000","last_became_peered":"2024-09-20T19:01:22.838429+0000","last_unstale":"2024-09-20T19:02:55.362658+0000","last_undegraded":"2024-09-20T19:02:55.362658+0000","last_fullsized":"2024-09-20T19:02:55.362658+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:06.310027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:59:06.310027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":612,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.365484+0000","last_change":"2024-09-20T19:02:38.906352+0000","last_active":"2024-09-20T19:02:55.365484+0000","last_peered":"2024-09-20T19:02:55.365484+0000","last_clean":"2024-09-20T19:02:55.365484+0000","last_became_active":"2024-09-20T18:49:49.957067+0000","last_became_peered":"2024-09-20T18:49:49.957067+0000","last_unstale":"2024-09-20T19:02:55.365484+0000","last_undegraded":"2024-09-20T19:02:55.365484+0000","last_fullsized":"2024-09-20T19:02:55.365484+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:38.906161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:38.906161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:41.340426+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":592,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.363237+0000","last_change":"2024-09-20T18:57:16.789412+0000","last_active":"2024-09-20T19:02:55.363237+0000","last_peered":"2024-09-20T19:02:55.363237+0000","last_clean":"2024-09-20T19:02:55.363237+0000","last_became_active":"2024-09-20T18:49:51.966915+0000","last_became_peered":"2024-09-20T18:49:51.966915+0000","last_unstale":"2024-09-20T19:02:55.363237+0000","last_undegraded":"2024-09-20T19:02:55.363237+0000","last_fullsized":"2024-09-20T19:02:55.363237+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:16.789095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:16.789095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:17.675828+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"273'15","reported_seq":683,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.365768+0000","last_change":"2024-09-20T19:02:30.816061+0000","last_active":"2024-09-20T19:02:55.365768+0000","last_peered":"2024-09-20T19:02:55.365768+0000","last_clean":"2024-09-20T19:02:55.365768+0000","last_became_active":"2024-09-20T18:59:05.085230+0000","last_became_peered":"2024-09-20T18:59:05.085230+0000","last_unstale":"2024-09-20T19:02:55.365768+0000","last_undegraded":"2024-09-20T19:02:55.365768+0000","last_fullsized":"2024-09-20T19:02:55.365768+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:02:30.815747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:30.815747+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:44.961926+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":616,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.364698+0000","last_change":"2024-09-20T19:01:22.830951+0000","last_active":"2024-09-20T19:02:55.364698+0000","last_peered":"2024-09-20T19:02:55.364698+0000","last_clean":"2024-09-20T19:02:55.364698+0000","last_became_active":"2024-09-20T19:01:22.830642+0000","last_became_peered":"2024-09-20T19:01:22.830642+0000","last_unstale":"2024-09-20T19:02:55.364698+0000","last_undegraded":"2024-09-20T19:02:55.364698+0000","last_fullsized":"2024-09-20T19:02:55.364698+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:42.487403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:42.487403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:06.397840+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":612,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.366587+0000","last_change":"2024-09-20T19:01:03.507072+0000","last_active":"2024-09-20T19:02:55.366587+0000","last_peered":"2024-09-20T19:02:55.366587+0000","last_clean":"2024-09-20T19:02:55.366587+0000","last_became_active":"2024-09-20T18:49:49.962523+0000","last_became_peered":"2024-09-20T18:49:49.962523+0000","last_unstale":"2024-09-20T19:02:55.366587+0000","last_undegraded":"2024-09-20T19:02:55.366587+0000","last_fullsized":"2024-09-20T19:02:55.366587+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:03.507016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:03.507016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":609,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.359720+0000","last_change":"2024-09-20T19:01:11.847792+0000","last_active":"2024-09-20T19:02:55.359720+0000","last_peered":"2024-09-20T19:02:55.359720+0000","last_clean":"2024-09-20T19:02:55.359720+0000","last_became_active":"2024-09-20T18:49:51.967313+0000","last_became_peered":"2024-09-20T18:49:51.967313+0000","last_unstale":"2024-09-20T19:02:55.359720+0000","last_undegraded":"2024-09-20T19:02:55.359720+0000","last_fullsized":"2024-09-20T19:02:55.359720+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:11.847716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:11.847716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:21.028973+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"274'19","reported_seq":685,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.363397+0000","last_change":"2024-09-20T19:01:05.495603+0000","last_active":"2024-09-20T19:02:55.363397+0000","last_peered":"2024-09-20T19:02:55.363397+0000","last_clean":"2024-09-20T19:02:55.363397+0000","last_became_active":"2024-09-20T18:59:05.075390+0000","last_became_peered":"2024-09-20T18:59:05.075390+0000","last_unstale":"2024-09-20T19:02:55.363397+0000","last_undegraded":"2024-09-20T19:02:55.363397+0000","last_fullsized":"2024-09-20T19:02:55.363397+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'19","last_scrub_stamp":"2024-09-20T19:01:05.495478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:05.495478+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":628,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.362619+0000","last_change":"2024-09-20T19:01:47.521858+0000","last_active":"2024-09-20T19:02:55.362619+0000","last_peered":"2024-09-20T19:02:55.362619+0000","last_clean":"2024-09-20T19:02:55.362619+0000","last_became_active":"2024-09-20T18:49:45.942105+0000","last_became_peered":"2024-09-20T18:49:45.942105+0000","last_unstale":"2024-09-20T19:02:55.362619+0000","last_undegraded":"2024-09-20T19:02:55.362619+0000","last_fullsized":"2024-09-20T19:02:55.362619+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:47.521789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:47.521789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:09.440449+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":638,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.363963+0000","last_change":"2024-09-20T19:02:27.267300+0000","last_active":"2024-09-20T19:02:55.363963+0000","last_peered":"2024-09-20T19:02:55.363963+0000","last_clean":"2024-09-20T19:02:55.363963+0000","last_became_active":"2024-09-20T19:01:22.827211+0000","last_became_peered":"2024-09-20T19:01:22.827211+0000","last_unstale":"2024-09-20T19:02:55.363963+0000","last_undegraded":"2024-09-20T19:02:55.363963+0000","last_fullsized":"2024-09-20T19:02:55.363963+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:27.267254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:27.267254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:51.446089+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":614,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.367879+0000","last_change":"2024-09-20T19:02:38.950217+0000","last_active":"2024-09-20T19:02:55.367879+0000","last_peered":"2024-09-20T19:02:55.367879+0000","last_clean":"2024-09-20T19:02:55.367879+0000","last_became_active":"2024-09-20T18:59:05.081179+0000","last_became_peered":"2024-09-20T18:59:05.081179+0000","last_unstale":"2024-09-20T19:02:55.367879+0000","last_undegraded":"2024-09-20T19:02:55.367879+0000","last_fullsized":"2024-09-20T19:02:55.367879+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:38.950017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:38.950017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:02.988187+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"273'8","reported_seq":654,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.362767+0000","last_change":"2024-09-20T19:00:33.504967+0000","last_active":"2024-09-20T19:02:55.362767+0000","last_peered":"2024-09-20T19:02:55.362767+0000","last_clean":"2024-09-20T19:02:55.362767+0000","last_became_active":"2024-09-20T18:59:05.076365+0000","last_became_peered":"2024-09-20T18:59:05.076365+0000","last_unstale":"2024-09-20T19:02:55.362767+0000","last_undegraded":"2024-09-20T19:02:55.362767+0000","last_fullsized":"2024-09-20T19:02:55.362767+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'8","last_scrub_stamp":"2024-09-20T19:00:33.504891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:00:33.504891+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:50.863517+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":629,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.359238+0000","last_change":"2024-09-20T19:02:39.184249+0000","last_active":"2024-09-20T19:02:55.359238+0000","last_peered":"2024-09-20T19:02:55.359238+0000","last_clean":"2024-09-20T19:02:55.359238+0000","last_became_active":"2024-09-20T18:49:45.940988+0000","last_became_peered":"2024-09-20T18:49:45.940988+0000","last_unstale":"2024-09-20T19:02:55.359238+0000","last_undegraded":"2024-09-20T19:02:55.359238+0000","last_fullsized":"2024-09-20T19:02:55.359238+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:39.184150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:39.184150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:54.134855+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":202,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.357502+0000","last_change":"2024-09-20T19:01:41.497193+0000","last_active":"2024-09-20T19:02:55.357502+0000","last_peered":"2024-09-20T19:02:55.357502+0000","last_clean":"2024-09-20T19:02:55.357502+0000","last_became_active":"2024-09-20T19:01:22.845873+0000","last_became_peered":"2024-09-20T19:01:22.845873+0000","last_unstale":"2024-09-20T19:02:55.357502+0000","last_undegraded":"2024-09-20T19:02:55.357502+0000","last_fullsized":"2024-09-20T19:02:55.357502+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:41.497115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:41.497115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:53.771888+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":612,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.363643+0000","last_change":"2024-09-20T19:02:29.277994+0000","last_active":"2024-09-20T19:02:55.363643+0000","last_peered":"2024-09-20T19:02:55.363643+0000","last_clean":"2024-09-20T19:02:55.363643+0000","last_became_active":"2024-09-20T19:01:22.829961+0000","last_became_peered":"2024-09-20T19:01:22.829961+0000","last_unstale":"2024-09-20T19:02:55.363643+0000","last_undegraded":"2024-09-20T19:02:55.363643+0000","last_fullsized":"2024-09-20T19:02:55.363643+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:29.277916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:29.277916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:32.717744+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":595,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.364192+0000","last_change":"2024-09-20T19:01:22.834863+0000","last_active":"2024-09-20T19:02:55.364192+0000","last_peered":"2024-09-20T19:02:55.364192+0000","last_clean":"2024-09-20T19:02:55.364192+0000","last_became_active":"2024-09-20T19:01:22.834453+0000","last_became_peered":"2024-09-20T19:01:22.834453+0000","last_unstale":"2024-09-20T19:02:55.364192+0000","last_undegraded":"2024-09-20T19:02:55.364192+0000","last_fullsized":"2024-09-20T19:02:55.364192+0000","mapping_epoch":304,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:10.970779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:10.970779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:07.916601+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":612,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.363043+0000","last_change":"2024-09-20T18:58:36.077005+0000","last_active":"2024-09-20T19:02:55.363043+0000","last_peered":"2024-09-20T19:02:55.363043+0000","last_clean":"2024-09-20T19:02:55.363043+0000","last_became_active":"2024-09-20T18:49:45.937992+0000","last_became_peered":"2024-09-20T18:49:45.937992+0000","last_unstale":"2024-09-20T19:02:55.363043+0000","last_undegraded":"2024-09-20T19:02:55.363043+0000","last_fullsized":"2024-09-20T19:02:55.363043+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:36.076937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:36.076937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:21.762264+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"274'19","reported_seq":215,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.360525+0000","last_change":"2024-09-20T19:01:15.544760+0000","last_active":"2024-09-20T19:02:55.360525+0000","last_peered":"2024-09-20T19:02:55.360525+0000","last_clean":"2024-09-20T19:02:55.360525+0000","last_became_active":"2024-09-20T18:59:05.089294+0000","last_became_peered":"2024-09-20T18:59:05.089294+0000","last_unstale":"2024-09-20T19:02:55.360525+0000","last_undegraded":"2024-09-20T19:02:55.360525+0000","last_fullsized":"2024-09-20T19:02:55.360525+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'19","last_scrub_stamp":"2024-09-20T19:01:15.544663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:15.544663+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:17.714021+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":606,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-20T19:02:55.359390+0000","last_change":"2024-09-20T19:01:40.961449+0000","last_active":"2024-09-20T19:02:55.359390+0000","last_peered":"2024-09-20T19:02:55.359390+0000","last_clean":"2024-09-20T19:02:55.359390+0000","last_became_active":"2024-09-20T18:49:49.957582+0000","last_became_peered":"2024-09-20T18:49:49.957582+0000","last_unstale":"2024-09-20T19:02:55.359390+0000","last_undegraded":"2024-09-20T19:02:55.359390+0000","last_fullsized":"2024-09-20T19:02:55.359390+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:40.961363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:40.961363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:54.068818+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":5058658304,"num_objects":77189,"num_object_clones":0,"num_object_copies":231567,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":77189,"num_write_kb":4940096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":15225520128,"data_stored":15225520128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76889,"ondisk_log_size":76889,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1185,"internal_metadata":0},"log_size":78,"ondisk_log_size":78,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1115,"num_read_kb":872,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":2244,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":683,"ondisk_log_size":683,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":789,"num_read_kb":784,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266851,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1661768,"kb_used_data":1197432,"kb_used_omap":1,"kb_used_meta":464318,"kb_avail":92083384,"statfs":{"total":95995035648,"available":94293385216,"internally_reserved":0,"allocated":1226170368,"data_stored":1225616939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":475462090},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528490,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2517556,"kb_used_data":2149028,"kb_used_omap":1,"kb_used_meta":368510,"kb_avail":91227596,"statfs":{"total":95995035648,"available":93417058304,"internally_reserved":0,"allocated":2200604672,"data_stored":2200071061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":377354694},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757423,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2050292,"kb_used_data":1541540,"kb_used_omap":1,"kb_used_meta":508734,"kb_avail":91694860,"statfs":{"total":95995035648,"available":93895536640,"internally_reserved":0,"allocated":1578536960,"data_stored":1578003164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":520944069},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":304,"seq":1305670058005,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2579348,"kb_used_data":2436548,"kb_used_omap":5,"kb_used_meta":142778,"kb_avail":91165804,"statfs":{"total":95995035648,"available":93353783296,"internally_reserved":0,"allocated":2495025152,"data_stored":2494459840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5633,"internal_metadata":146205183},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182587,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2155856,"kb_used_data":1565184,"kb_used_omap":1,"kb_used_meta":590654,"kb_avail":91589296,"statfs":{"total":95995035648,"available":93787439104,"internally_reserved":0,"allocated":1602748416,"data_stored":1602186044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":604830151},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411520,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2376888,"kb_used_data":1832040,"kb_used_omap":1,"kb_used_meta":544830,"kb_avail":91368264,"statfs":{"total":95995035648,"available":93561102336,"internally_reserved":0,"allocated":1876008960,"data_stored":1875471647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":557906373},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575046,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1468008,"kb_used_data":941208,"kb_used_omap":1,"kb_used_meta":526782,"kb_avail":92277144,"statfs":{"total":95995035648,"available":94491795456,"internally_reserved":0,"allocated":963796992,"data_stored":963211711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":539425221},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738572,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3677768,"kb_used_data":3119032,"kb_used_omap":1,"kb_used_meta":558718,"kb_avail":90067384,"statfs":{"total":95995035648,"available":92229001216,"internally_reserved":0,"allocated":3193888768,"data_stored":3193335035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":572127685},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1185,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3203989504,"data_stored":3203989504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":963903488,"data_stored":963903488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1898774528,"data_stored":1898774528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1607860224,"data_stored":1607860224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2503671808,"data_stored":2503671808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1590624256,"data_stored":1590624256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2215182336,"data_stored":2215182336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1241513984,"data_stored":1241513984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T19:03:05.540 INFO:tasks.cephadm.ceph_manager.ceph:would drop below min_pgs, proj -9, currently 1 2024-09-20T19:03:05.553 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:05.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:05.554 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:05.655 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:05.688 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_blocked_ops 2024-09-20T19:03:05.703 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:05.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:05.704 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:05.805 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:05.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:05.840 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:05.840 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:05.877 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 46 16 82496 82480 112.033 79.375 0.0150096 0.00892058 2024-09-20T19:03:05.941 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:05.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:06.080 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:06.081 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:06.081 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:05 smithi083 ceph-mon[29421]: 4.c scrub starts 2024-09-20T19:03:06.081 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:05 smithi083 ceph-mon[29421]: 4.c scrub ok 2024-09-20T19:03:06.081 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:05 smithi083 ceph-mon[29421]: pgmap v662: 148 pgs: 148 active+clean; 4.7 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 639 B/s rd, 53 MiB/s wr, 847 op/s 2024-09-20T19:03:06.081 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:05 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/4184880784' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:03:06.081 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:05 smithi083 ceph-mon[36260]: 4.c scrub starts 2024-09-20T19:03:06.082 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:05 smithi083 ceph-mon[36260]: 4.c scrub ok 2024-09-20T19:03:06.082 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:05 smithi083 ceph-mon[36260]: pgmap v662: 148 pgs: 148 active+clean; 4.7 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 639 B/s rd, 53 MiB/s wr, 847 op/s 2024-09-20T19:03:06.082 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:05 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/4184880784' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:03:06.082 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:05 smithi142 ceph-mon[32508]: 4.c scrub starts 2024-09-20T19:03:06.082 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:05 smithi142 ceph-mon[32508]: 4.c scrub ok 2024-09-20T19:03:06.082 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:05 smithi142 ceph-mon[32508]: pgmap v662: 148 pgs: 148 active+clean; 4.7 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 639 B/s rd, 53 MiB/s wr, 847 op/s 2024-09-20T19:03:06.082 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:05 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/4184880784' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:03:06.141 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:03:06.182 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:06.204 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:06.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:06.205 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:06.305 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:06.337 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:06.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:06.338 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:06.439 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:06.482 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:06.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:06.483 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:06.567 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T19:03:06.584 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:06.660 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:06.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:06.662 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:06.763 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:06.799 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:06.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:06.800 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:06.878 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 47 16 83707 83691 111.258 75.6875 0.0124107 0.00898364 2024-09-20T19:03:06.901 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:06.930 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:06 smithi142 ceph-mon[32508]: 4.0 scrub starts 2024-09-20T19:03:06.930 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:06 smithi142 ceph-mon[32508]: 4.0 scrub ok 2024-09-20T19:03:06.948 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:06.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:06.949 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:06.985 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:03:07.050 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:07.089 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:07.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:07.089 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:07.097 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T19:03:07.098 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:03:07.146 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:06 smithi083 ceph-mon[29421]: 4.0 scrub starts 2024-09-20T19:03:07.146 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:06 smithi083 ceph-mon[29421]: 4.0 scrub ok 2024-09-20T19:03:07.146 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:06 smithi083 ceph-mon[36260]: 4.0 scrub starts 2024-09-20T19:03:07.146 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:06 smithi083 ceph-mon[36260]: 4.0 scrub ok 2024-09-20T19:03:07.191 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:07.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:07.235 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:07.235 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:07.264 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:03:07 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:03:07.336 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:07.371 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:07.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:07.372 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:07.473 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:07.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:07.519 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:07.519 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:07.620 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:07.654 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:07.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:07.656 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:07.756 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:07.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:07.791 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:07.791 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:07.839 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_historic_ops 2024-09-20T19:03:07.878 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 48 16 84926 84910 110.528 76.1875 0.0114735 0.00904225 2024-09-20T19:03:07.893 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:07.929 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:07.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:07.930 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:08.014 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:07 smithi142 ceph-mon[32508]: 4.1 scrub starts 2024-09-20T19:03:08.014 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:07 smithi142 ceph-mon[32508]: 4.1 scrub ok 2024-09-20T19:03:08.014 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:07 smithi142 ceph-mon[32508]: 6.a scrub starts 2024-09-20T19:03:08.014 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:07 smithi142 ceph-mon[32508]: 6.a scrub ok 2024-09-20T19:03:08.014 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:07 smithi142 ceph-mon[32508]: pgmap v663: 148 pgs: 148 active+clean; 5.0 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 86 MiB/s wr, 1.38k op/s 2024-09-20T19:03:08.014 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:07 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:03:08.014 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:07 smithi142 ceph-mon[32508]: 3.15 scrub starts 2024-09-20T19:03:08.014 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:07 smithi142 ceph-mon[32508]: 3.15 scrub ok 2024-09-20T19:03:08.031 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:08.075 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:08.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:08.076 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:08.177 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:03:08.191 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T19:03:08.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:08.215 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:08.215 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:03:08.219 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:07 smithi083 ceph-mon[29421]: 4.1 scrub starts 2024-09-20T19:03:08.219 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:07 smithi083 ceph-mon[29421]: 4.1 scrub ok 2024-09-20T19:03:08.219 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:07 smithi083 ceph-mon[29421]: 6.a scrub starts 2024-09-20T19:03:08.219 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:07 smithi083 ceph-mon[29421]: 6.a scrub ok 2024-09-20T19:03:08.219 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:07 smithi083 ceph-mon[29421]: pgmap v663: 148 pgs: 148 active+clean; 5.0 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 86 MiB/s wr, 1.38k op/s 2024-09-20T19:03:08.219 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:07 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:03:08.219 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:07 smithi083 ceph-mon[29421]: 3.15 scrub starts 2024-09-20T19:03:08.219 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:07 smithi083 ceph-mon[29421]: 3.15 scrub ok 2024-09-20T19:03:08.219 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:07 smithi083 ceph-mon[36260]: 4.1 scrub starts 2024-09-20T19:03:08.219 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:07 smithi083 ceph-mon[36260]: 4.1 scrub ok 2024-09-20T19:03:08.220 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:07 smithi083 ceph-mon[36260]: 6.a scrub starts 2024-09-20T19:03:08.220 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:07 smithi083 ceph-mon[36260]: 6.a scrub ok 2024-09-20T19:03:08.220 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:07 smithi083 ceph-mon[36260]: pgmap v663: 148 pgs: 148 active+clean; 5.0 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 86 MiB/s wr, 1.38k op/s 2024-09-20T19:03:08.220 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:07 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:03:08.220 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:07 smithi083 ceph-mon[36260]: 3.15 scrub starts 2024-09-20T19:03:08.220 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:07 smithi083 ceph-mon[36260]: 3.15 scrub ok 2024-09-20T19:03:08.283 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:03:08.316 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:08.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:08.357 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:08.357 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:08.458 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:08.493 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:08.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:08.494 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:08.595 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:03:08.633 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:08.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:08.634 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:03:08.639 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:03:08.735 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:08.776 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:08.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:08.777 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:08.878 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:08.879 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 49 16 86082 86066 109.747 72.25 0.00632161 0.00910773 2024-09-20T19:03:08.904 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-20T19:03:08.915 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:08.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:08.916 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:08.925 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T19:03:08.925 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:03:09.017 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:09.043 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:08 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3518867692' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:03:09.043 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:08 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:03:09.044 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:08 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3518867692' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:03:09.044 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:08 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:03:09.045 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:09.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:09.046 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:09.147 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:09.168 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:09.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:09.169 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:09.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:08 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3518867692' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:03:09.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:08 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:03:09.270 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:09.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:09.306 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:09.306 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:09.406 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:09.450 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:09.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:09.451 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:09.551 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:09.581 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:09.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:09.582 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:09.683 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:09.716 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:09.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:09.717 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:09.818 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:09.847 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:09.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:09.848 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:09.878 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 50 16 86924 86908 108.604 52.625 0.0105499 0.00920302 2024-09-20T19:03:09.949 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:09.981 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:09.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:09.983 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:10.084 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:10.111 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:10.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:10.112 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:10.166 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:09 smithi142 ceph-mon[32508]: 3.1a scrub starts 2024-09-20T19:03:10.166 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:09 smithi142 ceph-mon[32508]: 3.1a scrub ok 2024-09-20T19:03:10.166 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:09 smithi142 ceph-mon[32508]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T19:03:10.167 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:09 smithi142 ceph-mon[32508]: Cluster is now healthy 2024-09-20T19:03:10.167 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:09 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:03:10.167 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:09 smithi142 ceph-mon[32508]: osdmap e349: 8 total, 8 up, 8 in 2024-09-20T19:03:10.167 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:09 smithi142 ceph-mon[32508]: pgmap v665: 148 pgs: 148 active+clean; 5.1 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 820 B/s rd, 58 MiB/s wr, 926 op/s 2024-09-20T19:03:10.167 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:09 smithi142 ceph-mon[32508]: 4.1c scrub starts 2024-09-20T19:03:10.167 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:09 smithi142 ceph-mon[32508]: 4.1c scrub ok 2024-09-20T19:03:10.213 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:10.254 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:10.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:10.255 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:10.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:09 smithi083 ceph-mon[29421]: 3.1a scrub starts 2024-09-20T19:03:10.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:09 smithi083 ceph-mon[29421]: 3.1a scrub ok 2024-09-20T19:03:10.294 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:09 smithi083 ceph-mon[29421]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T19:03:10.294 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:09 smithi083 ceph-mon[29421]: Cluster is now healthy 2024-09-20T19:03:10.294 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:09 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:03:10.294 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:09 smithi083 ceph-mon[29421]: osdmap e349: 8 total, 8 up, 8 in 2024-09-20T19:03:10.294 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:09 smithi083 ceph-mon[29421]: pgmap v665: 148 pgs: 148 active+clean; 5.1 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 820 B/s rd, 58 MiB/s wr, 926 op/s 2024-09-20T19:03:10.294 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:09 smithi083 ceph-mon[29421]: 4.1c scrub starts 2024-09-20T19:03:10.294 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:09 smithi083 ceph-mon[29421]: 4.1c scrub ok 2024-09-20T19:03:10.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:09 smithi083 ceph-mon[36260]: 3.1a scrub starts 2024-09-20T19:03:10.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:09 smithi083 ceph-mon[36260]: 3.1a scrub ok 2024-09-20T19:03:10.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:09 smithi083 ceph-mon[36260]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T19:03:10.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:09 smithi083 ceph-mon[36260]: Cluster is now healthy 2024-09-20T19:03:10.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:09 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:03:10.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:09 smithi083 ceph-mon[36260]: osdmap e349: 8 total, 8 up, 8 in 2024-09-20T19:03:10.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:09 smithi083 ceph-mon[36260]: pgmap v665: 148 pgs: 148 active+clean; 5.1 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 820 B/s rd, 58 MiB/s wr, 926 op/s 2024-09-20T19:03:10.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:09 smithi083 ceph-mon[36260]: 4.1c scrub starts 2024-09-20T19:03:10.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:09 smithi083 ceph-mon[36260]: 4.1c scrub ok 2024-09-20T19:03:10.355 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:10.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:10.393 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:10.393 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:10.494 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:10.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:10.531 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:10.531 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:10.540 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 7, 6] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 5, 6, 7, 4] 2024-09-20T19:03:10.541 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T19:03:10.541 INFO:tasks.thrashosds.thrasher:Removing osd 4, in_osds are: [0, 1, 2, 3, 4, 5, 7, 6] 2024-09-20T19:03:10.541 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd out 4 2024-09-20T19:03:10.632 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:10.680 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:10.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:10.681 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:10.781 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:10.815 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:10.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:10.815 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:10.879 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 51 16 88106 88090 107.923 73.875 0.040148 0.00925934 2024-09-20T19:03:10.906 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:03:10 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:03:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:03:10.916 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:10.952 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:10.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:10.953 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:10.983 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-20T19:03:11.054 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:03:11.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:11.099 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:11.099 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:03:11.172 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:10 smithi083 ceph-mon[29421]: 4.a scrub starts 2024-09-20T19:03:11.172 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:10 smithi083 ceph-mon[29421]: 4.a scrub ok 2024-09-20T19:03:11.172 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:10 smithi083 ceph-mon[29421]: 6.18 scrub starts 2024-09-20T19:03:11.172 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:10 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/4035849111' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:03:11.173 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:10 smithi083 ceph-mon[36260]: 4.a scrub starts 2024-09-20T19:03:11.173 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:10 smithi083 ceph-mon[36260]: 4.a scrub ok 2024-09-20T19:03:11.173 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:10 smithi083 ceph-mon[36260]: 6.18 scrub starts 2024-09-20T19:03:11.173 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:10 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/4035849111' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:03:11.199 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:11.236 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:11.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:11.237 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:11.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:10 smithi142 ceph-mon[32508]: 4.a scrub starts 2024-09-20T19:03:11.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:10 smithi142 ceph-mon[32508]: 4.a scrub ok 2024-09-20T19:03:11.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:10 smithi142 ceph-mon[32508]: 6.18 scrub starts 2024-09-20T19:03:11.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:10 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/4035849111' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:03:11.338 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:11.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:11.386 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:11.386 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:11.463 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:03:11.487 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:11.505 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:03:11.505 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:03:11.505 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:03:11.505 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:03:11.505 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:03:11.505 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:03:11.505 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:03:11.505 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:03:11.506 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:03:11.506 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:03:11.506 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:03:11.506 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:03:11.506 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:03:11.506 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:03:11.506 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:03:11.506 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:03:11.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:11.539 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:11.540 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:11.640 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:11.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:11.673 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:11.673 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:11.774 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:03:11.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:11.814 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:11.814 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:03:11.879 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 52 16 89286 89270 107.265 73.75 0.00968396 0.00931745 2024-09-20T19:03:11.915 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:11.943 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:11.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:11.944 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:12.045 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:12.077 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T19:03:12.077 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-09-20T19:03:12.077 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:12.077 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:89506 10.d 10:bbfc8966:::benchmark_data_smithi083_148100_object89505:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e349)", 2024-09-20T19:03:12.077 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:03:12.058669+0000", 2024-09-20T19:03:12.078 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.010982466, 2024-09-20T19:03:12.078 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.011019899, 2024-09-20T19:03:12.078 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:03:12.078 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:03:12.078 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "waiting for sub ops", 2024-09-20T19:03:12.078 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:03:12.078 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:03:12.078 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:03:12.078 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 89506 2024-09-20T19:03:12.078 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:12.078 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:03:12.079 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:12.079 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:03:12.079 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:12.058669+0000", 2024-09-20T19:03:12.079 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:12.079 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:12.079 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:12.079 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:03:12.079 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:12.058669+0000", 2024-09-20T19:03:12.079 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:12.079 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:12.079 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:12.079 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:03:12.079 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:12.058671+0000", 2024-09-20T19:03:12.079 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.61e-06 2024-09-20T19:03:12.080 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:12.080 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:12.080 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:03:12.080 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:12.058703+0000", 2024-09-20T19:03:12.080 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.1792999999999999e-05 2024-09-20T19:03:12.080 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:12.080 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:12.080 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:03:12.080 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:12.058712+0000", 2024-09-20T19:03:12.080 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.7669999999999992e-06 2024-09-20T19:03:12.080 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:12.080 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:12.080 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:03:12.081 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:12.058718+0000", 2024-09-20T19:03:12.081 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.1489999999999999e-06 2024-09-20T19:03:12.081 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:12.081 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:12.081 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:12.081 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:12.058770+0000", 2024-09-20T19:03:12.081 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.2645999999999997e-05 2024-09-20T19:03:12.081 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:12.081 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:12.081 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:03:12.081 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:12.058898+0000", 2024-09-20T19:03:12.081 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000127973 2024-09-20T19:03:12.081 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:12.081 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:12.082 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,4", 2024-09-20T19:03:12.082 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:12.059020+0000", 2024-09-20T19:03:12.082 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000122059 2024-09-20T19:03:12.082 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:12.082 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:12.082 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:03:12.082 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:12.062410+0000", 2024-09-20T19:03:12.082 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0033896239999999999 2024-09-20T19:03:12.082 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:12.082 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:12.082 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:03:12.082 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:12.066945+0000", 2024-09-20T19:03:12.082 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0045350980000000004 2024-09-20T19:03:12.082 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:12.083 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:03:12.083 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:12.083 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:12.083 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:12.083 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:89515 10.d 10:b825fa90:::benchmark_data_smithi083_148100_object89514:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e349)", 2024-09-20T19:03:12.083 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:03:12.061910+0000", 2024-09-20T19:03:12.083 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.0077414520000000002, 2024-09-20T19:03:12.083 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0078685600000000001, 2024-09-20T19:03:12.083 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:03:12.083 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:03:12.083 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "waiting for sub ops", 2024-09-20T19:03:12.083 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:03:12.083 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:03:12.084 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:03:12.084 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 89515 2024-09-20T19:03:12.084 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:12.084 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:03:12.084 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:12.084 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:03:12.084 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:12.061910+0000", 2024-09-20T19:03:12.084 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:12.084 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:12.084 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:12.084 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:03:12.084 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:12.061910+0000", 2024-09-20T19:03:12.084 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:12.084 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:12.084 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:12.085 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:03:12.085 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:12.061912+0000", 2024-09-20T19:03:12.085 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4410000000000001e-06 2024-09-20T19:03:12.085 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:12.085 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:12.085 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:03:12.085 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:12.062493+0000", 2024-09-20T19:03:12.085 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00058069999999999997 2024-09-20T19:03:12.085 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:12.085 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:12.085 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:03:12.085 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:12.062504+0000", 2024-09-20T19:03:12.085 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1095e-05 2024-09-20T19:03:12.085 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:12.085 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:12.086 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:03:12.086 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:12.062511+0000", 2024-09-20T19:03:12.086 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.2429999999999999e-06 2024-09-20T19:03:12.086 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:12.086 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:12.086 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:12.086 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:12.062582+0000", 2024-09-20T19:03:12.086 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.1050999999999994e-05 2024-09-20T19:03:12.086 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:12.086 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:12.086 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:03:12.086 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:12.062695+0000", 2024-09-20T19:03:12.086 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011311699999999999 2024-09-20T19:03:12.087 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:12.087 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:12.087 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,4", 2024-09-20T19:03:12.087 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:12.062830+0000", 2024-09-20T19:03:12.087 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013541399999999999 2024-09-20T19:03:12.087 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:12.087 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:12.087 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:03:12.087 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:12.066982+0000", 2024-09-20T19:03:12.087 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0041516230000000001 2024-09-20T19:03:12.087 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:12.087 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:12.087 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:03:12.087 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:12.067429+0000", 2024-09-20T19:03:12.087 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00044662899999999998 2024-09-20T19:03:12.088 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:12.088 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:03:12.088 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:12.088 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:12.088 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:12.088 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:89517 10.0 10:0a250315:::benchmark_data_smithi083_148100_object89516:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e349)", 2024-09-20T19:03:12.088 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:03:12.062861+0000", 2024-09-20T19:03:12.088 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.0067904389999999997, 2024-09-20T19:03:12.088 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0069970659999999997, 2024-09-20T19:03:12.088 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:03:12.088 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:03:12.088 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "waiting for sub ops", 2024-09-20T19:03:12.088 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:03:12.088 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:03:12.088 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:03:12.089 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 89517 2024-09-20T19:03:12.089 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:12.089 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:03:12.089 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:12.089 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:03:12.089 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:12.062861+0000", 2024-09-20T19:03:12.089 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:12.089 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:12.089 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:12.089 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:03:12.089 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:12.062861+0000", 2024-09-20T19:03:12.089 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:12.089 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:12.089 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:12.089 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:03:12.090 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:12.062863+0000", 2024-09-20T19:03:12.090 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1319999999999999e-06 2024-09-20T19:03:12.090 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:12.090 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:12.090 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:03:12.090 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:12.062890+0000", 2024-09-20T19:03:12.090 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.7945000000000001e-05 2024-09-20T19:03:12.090 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:12.090 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:12.090 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:03:12.090 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:12.062902+0000", 2024-09-20T19:03:12.090 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1134999999999999e-05 2024-09-20T19:03:12.090 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:12.090 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:12.091 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:03:12.091 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:12.062909+0000", 2024-09-20T19:03:12.091 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.9959999999999996e-06 2024-09-20T19:03:12.091 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:12.091 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:12.091 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:12.091 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:12.062965+0000", 2024-09-20T19:03:12.091 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6359e-05 2024-09-20T19:03:12.091 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:12.092 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:12.092 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:03:12.092 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:12.063076+0000", 2024-09-20T19:03:12.092 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000110819 2024-09-20T19:03:12.092 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:12.092 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:12.092 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,4", 2024-09-20T19:03:12.092 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:12.063192+0000", 2024-09-20T19:03:12.092 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000116345 2024-09-20T19:03:12.092 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:12.092 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:12.092 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:03:12.092 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:12.067207+0000", 2024-09-20T19:03:12.092 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0040145800000000002 2024-09-20T19:03:12.093 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:12.093 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:12.093 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:03:12.093 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:12.067217+0000", 2024-09-20T19:03:12.093 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0757000000000001e-05 2024-09-20T19:03:12.093 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:12.093 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:03:12.093 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:12.093 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:12.093 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:12.093 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25775.0:89513 10.e e349/325 10:792c1d54:::benchmark_data_smithi083_148100_object89512:head v 349'5663, mlcod=349'5660)", 2024-09-20T19:03:12.093 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:03:12.068100+0000", 2024-09-20T19:03:12.093 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.0015519119999999999, 2024-09-20T19:03:12.093 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001839367, 2024-09-20T19:03:12.093 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:03:12.094 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:03:12.094 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-09-20T19:03:12.094 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:03:12.094 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:12.094 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:03:12.094 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:12.068100+0000", 2024-09-20T19:03:12.094 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:12.094 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:12.094 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:12.094 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:03:12.094 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:12.068100+0000", 2024-09-20T19:03:12.094 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:12.094 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:12.095 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:12.095 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:03:12.095 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:12.068101+0000", 2024-09-20T19:03:12.095 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.4300000000000002e-07 2024-09-20T19:03:12.095 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:12.095 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:12.095 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:03:12.095 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:12.068153+0000", 2024-09-20T19:03:12.095 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.2234999999999997e-05 2024-09-20T19:03:12.095 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:12.095 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:12.095 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:03:12.095 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:12.068173+0000", 2024-09-20T19:03:12.095 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9621000000000001e-05 2024-09-20T19:03:12.095 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:12.096 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:12.096 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:03:12.096 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:12.068180+0000", 2024-09-20T19:03:12.096 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.0419999999999997e-06 2024-09-20T19:03:12.096 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:12.096 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:12.096 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:12.096 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:12.068240+0000", 2024-09-20T19:03:12.096 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0325000000000003e-05 2024-09-20T19:03:12.096 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:12.096 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:12.096 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:03:12.096 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:12.068258+0000", 2024-09-20T19:03:12.096 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7648000000000001e-05 2024-09-20T19:03:12.097 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:12.097 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:03:12.097 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:12.097 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:12.097 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-09-20T19:03:12.097 INFO:teuthology.orchestra.run.smithi142.stdout: "num_ops": 4 2024-09-20T19:03:12.097 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T19:03:12.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:12.098 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:12.099 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:12.200 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:03:12.233 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:12.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:12.234 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:03:12.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:11 smithi142 ceph-mon[32508]: 6.18 scrub ok 2024-09-20T19:03:12.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:11 smithi142 ceph-mon[32508]: 4.18 scrub starts 2024-09-20T19:03:12.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:11 smithi142 ceph-mon[32508]: 4.18 scrub ok 2024-09-20T19:03:12.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:11 smithi142 ceph-mon[32508]: pgmap v666: 148 pgs: 148 active+clean; 5.2 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 639 B/s rd, 66 MiB/s wr, 1.06k op/s 2024-09-20T19:03:12.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:11 smithi083 ceph-mon[29421]: 6.18 scrub ok 2024-09-20T19:03:12.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:11 smithi083 ceph-mon[29421]: 4.18 scrub starts 2024-09-20T19:03:12.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:11 smithi083 ceph-mon[29421]: 4.18 scrub ok 2024-09-20T19:03:12.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:11 smithi083 ceph-mon[29421]: pgmap v666: 148 pgs: 148 active+clean; 5.2 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 639 B/s rd, 66 MiB/s wr, 1.06k op/s 2024-09-20T19:03:12.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:11 smithi083 ceph-mon[36260]: 6.18 scrub ok 2024-09-20T19:03:12.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:11 smithi083 ceph-mon[36260]: 4.18 scrub starts 2024-09-20T19:03:12.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:11 smithi083 ceph-mon[36260]: 4.18 scrub ok 2024-09-20T19:03:12.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:11 smithi083 ceph-mon[36260]: pgmap v666: 148 pgs: 148 active+clean; 5.2 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 639 B/s rd, 66 MiB/s wr, 1.06k op/s 2024-09-20T19:03:12.335 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:12.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:12.371 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:12.371 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:12.454 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:03:12.472 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:12.504 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:12.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:12.505 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:12.606 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:12.638 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:12.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:12.639 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:12.740 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:12.771 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:12.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:12.772 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:12.873 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:03:12.879 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 53 16 90521 90505 106.698 77.1875 0.016998 0.00936792 2024-09-20T19:03:12.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:12.910 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:12.910 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:03:12.923 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_blocked_ops 2024-09-20T19:03:13.010 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:12 smithi083 ceph-mon[29421]: 6.e scrub starts 2024-09-20T19:03:13.010 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:12 smithi083 ceph-mon[29421]: 6.e scrub ok 2024-09-20T19:03:13.010 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:12 smithi083 ceph-mon[29421]: 2.2 scrub starts 2024-09-20T19:03:13.010 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:12 smithi083 ceph-mon[29421]: 2.2 scrub ok 2024-09-20T19:03:13.010 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:12 smithi083 ceph-mon[29421]: 6.14 scrub starts 2024-09-20T19:03:13.010 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:12 smithi083 ceph-mon[29421]: 6.14 scrub ok 2024-09-20T19:03:13.011 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:13.011 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:12 smithi083 ceph-mon[36260]: 6.e scrub starts 2024-09-20T19:03:13.011 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:12 smithi083 ceph-mon[36260]: 6.e scrub ok 2024-09-20T19:03:13.011 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:12 smithi083 ceph-mon[36260]: 2.2 scrub starts 2024-09-20T19:03:13.011 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:12 smithi083 ceph-mon[36260]: 2.2 scrub ok 2024-09-20T19:03:13.011 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:12 smithi083 ceph-mon[36260]: 6.14 scrub starts 2024-09-20T19:03:13.011 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:12 smithi083 ceph-mon[36260]: 6.14 scrub ok 2024-09-20T19:03:13.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:13.054 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:13.054 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:13.155 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:13.191 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:13.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:13.192 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:13.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:12 smithi142 ceph-mon[32508]: 6.e scrub starts 2024-09-20T19:03:13.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:12 smithi142 ceph-mon[32508]: 6.e scrub ok 2024-09-20T19:03:13.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:12 smithi142 ceph-mon[32508]: 2.2 scrub starts 2024-09-20T19:03:13.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:12 smithi142 ceph-mon[32508]: 2.2 scrub ok 2024-09-20T19:03:13.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:12 smithi142 ceph-mon[32508]: 6.14 scrub starts 2024-09-20T19:03:13.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:12 smithi142 ceph-mon[32508]: 6.14 scrub ok 2024-09-20T19:03:13.293 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:13.331 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:13.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:13.332 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:13.398 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:03:13.436 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:13.472 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:13.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:13.473 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:13.574 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:13.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:13.612 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:13.612 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:13.713 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:13.744 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:13.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:13.745 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:13.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:13 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mon-a[29417]: 2024-09-20T19:03:13.404+0000 7f6c091c6640 -1 mon.a@0(leader).osd e349 definitely_dead 0 2024-09-20T19:03:13.846 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:03:13.872 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T19:03:13.879 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 54 16 91713 91697 106.102 74.5 0.0200368 0.00941967 2024-09-20T19:03:13.893 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:13.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:13.894 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:03:13.943 INFO:teuthology.orchestra.run.smithi083.stderr:marked out osd.4. 2024-09-20T19:03:13.995 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:14.015 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:14.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:14.016 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:14.020 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T19:03:14.020 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-09-20T19:03:14.020 INFO:teuthology.orchestra.run.smithi142.stdout: "complaint_time": 30, 2024-09-20T19:03:14.020 INFO:teuthology.orchestra.run.smithi142.stdout: "num_blocked_ops": 0 2024-09-20T19:03:14.020 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T19:03:14.117 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:14.139 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:14.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:14.140 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:14.241 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:14.262 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:14.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:14.263 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:14.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:13 smithi142 ceph-mon[32508]: 4.d scrub starts 2024-09-20T19:03:14.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:13 smithi142 ceph-mon[32508]: 4.d scrub ok 2024-09-20T19:03:14.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:13 smithi142 ceph-mon[32508]: pgmap v667: 148 pgs: 148 active+clean; 5.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.1 KiB/s rd, 77 MiB/s wr, 1.24k op/s 2024-09-20T19:03:14.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:13 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2515491601' entity='client.admin' cmd={"prefix": "osd out", "ids": ["4"]} : dispatch 2024-09-20T19:03:14.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:13 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd out", "ids": ["4"]} : dispatch 2024-09-20T19:03:14.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:13 smithi142 ceph-mon[32508]: Client client.admin marked osd.4 out, while it was still marked up 2024-09-20T19:03:14.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:13 smithi142 ceph-mon[32508]: 3.c scrub starts 2024-09-20T19:03:14.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:13 smithi142 ceph-mon[32508]: 3.c scrub ok 2024-09-20T19:03:14.294 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:13 smithi083 ceph-mon[29421]: 4.d scrub starts 2024-09-20T19:03:14.294 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:13 smithi083 ceph-mon[29421]: 4.d scrub ok 2024-09-20T19:03:14.294 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:13 smithi083 ceph-mon[29421]: pgmap v667: 148 pgs: 148 active+clean; 5.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.1 KiB/s rd, 77 MiB/s wr, 1.24k op/s 2024-09-20T19:03:14.294 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:13 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2515491601' entity='client.admin' cmd={"prefix": "osd out", "ids": ["4"]} : dispatch 2024-09-20T19:03:14.294 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:13 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd out", "ids": ["4"]} : dispatch 2024-09-20T19:03:14.294 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:13 smithi083 ceph-mon[29421]: Client client.admin marked osd.4 out, while it was still marked up 2024-09-20T19:03:14.294 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:13 smithi083 ceph-mon[29421]: 3.c scrub starts 2024-09-20T19:03:14.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:13 smithi083 ceph-mon[29421]: 3.c scrub ok 2024-09-20T19:03:14.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:13 smithi083 ceph-mon[36260]: 4.d scrub starts 2024-09-20T19:03:14.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:13 smithi083 ceph-mon[36260]: 4.d scrub ok 2024-09-20T19:03:14.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:13 smithi083 ceph-mon[36260]: pgmap v667: 148 pgs: 148 active+clean; 5.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.1 KiB/s rd, 77 MiB/s wr, 1.24k op/s 2024-09-20T19:03:14.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:13 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2515491601' entity='client.admin' cmd={"prefix": "osd out", "ids": ["4"]} : dispatch 2024-09-20T19:03:14.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:13 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd out", "ids": ["4"]} : dispatch 2024-09-20T19:03:14.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:13 smithi083 ceph-mon[36260]: Client client.admin marked osd.4 out, while it was still marked up 2024-09-20T19:03:14.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:13 smithi083 ceph-mon[36260]: 3.c scrub starts 2024-09-20T19:03:14.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:13 smithi083 ceph-mon[36260]: 3.c scrub ok 2024-09-20T19:03:14.364 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:14.385 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:14.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:14.386 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:14.468 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T19:03:14.487 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:14.532 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:14.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:14.533 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:14.634 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:14.656 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:14.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:14.657 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:14.758 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:14.780 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:14.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:14.781 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:14.805 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_historic_ops 2024-09-20T19:03:14.820 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:03:14.879 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 55 16 91786 91770 104.256 4.5625 0.00356977 0.0094256 2024-09-20T19:03:14.881 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:14.903 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:14.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:14.904 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:15.004 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:15.013 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:14 smithi142 ceph-mon[32508]: 4.9 scrub starts 2024-09-20T19:03:15.013 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:14 smithi142 ceph-mon[32508]: 4.9 scrub ok 2024-09-20T19:03:15.013 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:14 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["4"]}]': finished 2024-09-20T19:03:15.013 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:14 smithi142 ceph-mon[32508]: osdmap e350: 8 total, 8 up, 7 in 2024-09-20T19:03:15.014 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:14 smithi142 ceph-mon[32508]: 1.0 scrub starts 2024-09-20T19:03:15.014 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:14 smithi142 ceph-mon[32508]: 1.0 scrub ok 2024-09-20T19:03:15.027 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:15.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:15.028 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:15.043 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:14 smithi083 ceph-mon[29421]: 4.9 scrub starts 2024-09-20T19:03:15.043 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:14 smithi083 ceph-mon[29421]: 4.9 scrub ok 2024-09-20T19:03:15.043 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:14 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["4"]}]': finished 2024-09-20T19:03:15.043 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:14 smithi083 ceph-mon[29421]: osdmap e350: 8 total, 8 up, 7 in 2024-09-20T19:03:15.044 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:14 smithi083 ceph-mon[29421]: 1.0 scrub starts 2024-09-20T19:03:15.044 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:14 smithi083 ceph-mon[29421]: 1.0 scrub ok 2024-09-20T19:03:15.044 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:14 smithi083 ceph-mon[36260]: 4.9 scrub starts 2024-09-20T19:03:15.044 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:14 smithi083 ceph-mon[36260]: 4.9 scrub ok 2024-09-20T19:03:15.044 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:14 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["4"]}]': finished 2024-09-20T19:03:15.044 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:14 smithi083 ceph-mon[36260]: osdmap e350: 8 total, 8 up, 7 in 2024-09-20T19:03:15.044 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:14 smithi083 ceph-mon[36260]: 1.0 scrub starts 2024-09-20T19:03:15.044 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:14 smithi083 ceph-mon[36260]: 1.0 scrub ok 2024-09-20T19:03:15.129 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:15.139 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:03:15.154 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:15.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:15.155 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:15.256 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:15.279 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:15.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:15.280 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:15.370 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:03:15.370 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:03:15.382 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:03:15.384 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:03:15.384 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:03:15.402 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:03:15.402 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:03:15.404 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:15.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:15.405 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:03:15.421 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:03:15.421 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:03:15.421 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:03:15.440 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:03:15.440 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:03:15.464 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:03:15.464 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:03:15.481 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:03:15.481 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:03:15.497 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:03:15.498 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:03:15.505 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:15.526 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:15.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:15.527 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:15.628 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:15.649 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:15.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:15.650 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:15.707 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T19:03:15.708 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:03:15.751 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:15.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:15.773 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:15.773 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:15.874 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:15.879 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 56 16 91786 91770 102.394 0 - 0.0094256 2024-09-20T19:03:15.896 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:15.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:15.897 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:15.998 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:16.032 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:16.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:16.033 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:16.134 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:16.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:16.166 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:16.166 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:16.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:15 smithi142 ceph-mon[32508]: 10.a scrub starts 2024-09-20T19:03:16.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:15 smithi142 ceph-mon[32508]: osdmap e351: 8 total, 8 up, 7 in 2024-09-20T19:03:16.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:15 smithi142 ceph-mon[32508]: pgmap v670: 148 pgs: 148 active+clean; 5.4 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 825 B/s rd, 57 MiB/s wr, 909 op/s 2024-09-20T19:03:16.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:15 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [3, 5]} : dispatch 2024-09-20T19:03:16.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:15 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1e", "id": [3, 2]} : dispatch 2024-09-20T19:03:16.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:15 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.0", "id": [0, 6]} : dispatch 2024-09-20T19:03:16.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:15 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/790101256' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:03:16.266 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:15 smithi083 ceph-mon[29421]: 10.a scrub starts 2024-09-20T19:03:16.266 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:15 smithi083 ceph-mon[29421]: osdmap e351: 8 total, 8 up, 7 in 2024-09-20T19:03:16.266 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:15 smithi083 ceph-mon[29421]: pgmap v670: 148 pgs: 148 active+clean; 5.4 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 825 B/s rd, 57 MiB/s wr, 909 op/s 2024-09-20T19:03:16.266 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:15 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [3, 5]} : dispatch 2024-09-20T19:03:16.266 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:15 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1e", "id": [3, 2]} : dispatch 2024-09-20T19:03:16.266 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:15 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.0", "id": [0, 6]} : dispatch 2024-09-20T19:03:16.266 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:15 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/790101256' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:03:16.266 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:15 smithi083 ceph-mon[36260]: 10.a scrub starts 2024-09-20T19:03:16.267 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:15 smithi083 ceph-mon[36260]: osdmap e351: 8 total, 8 up, 7 in 2024-09-20T19:03:16.267 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:15 smithi083 ceph-mon[36260]: pgmap v670: 148 pgs: 148 active+clean; 5.4 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 825 B/s rd, 57 MiB/s wr, 909 op/s 2024-09-20T19:03:16.267 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:15 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [3, 5]} : dispatch 2024-09-20T19:03:16.267 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:15 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1e", "id": [3, 2]} : dispatch 2024-09-20T19:03:16.267 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:15 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.0", "id": [0, 6]} : dispatch 2024-09-20T19:03:16.267 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:15 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/790101256' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:03:16.267 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:16.295 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:16.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:16.296 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:16.396 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:16.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:16.432 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:16.432 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:16.517 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-20T19:03:16.533 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:16.565 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:16.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:16.566 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:16.667 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:16.696 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:16.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:16.697 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:16.798 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:16.846 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:16.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:16.847 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:16.879 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 57 16 92286 92270 101.146 15.625 0.00650761 0.00987994 2024-09-20T19:03:16.948 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:16.974 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:03:16.986 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:16.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:16.987 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:17.087 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:17.125 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:17.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:17.127 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:17.149 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:16 smithi083 ceph-mon[36260]: 5.5 scrub starts 2024-09-20T19:03:17.149 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:16 smithi083 ceph-mon[36260]: 5.5 scrub ok 2024-09-20T19:03:17.149 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:16 smithi083 ceph-mon[36260]: 10.a scrub ok 2024-09-20T19:03:17.149 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:16 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [3, 5]}]': finished 2024-09-20T19:03:17.149 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:16 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1e", "id": [3, 2]}]': finished 2024-09-20T19:03:17.149 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:16 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.0", "id": [0, 6]}]': finished 2024-09-20T19:03:17.149 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:16 smithi083 ceph-mon[36260]: 10.2 starting backfill to osd.3 from (0'0,0'0] MAX to 349'5729 2024-09-20T19:03:17.149 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:16 smithi083 ceph-mon[36260]: 10.c starting backfill to osd.3 from (0'0,0'0] MAX to 349'5670 2024-09-20T19:03:17.149 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:16 smithi083 ceph-mon[36260]: 10.1 starting backfill to osd.1 from (0'0,0'0] MAX to 349'5648 2024-09-20T19:03:17.149 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:16 smithi083 ceph-mon[36260]: osdmap e352: 8 total, 8 up, 7 in 2024-09-20T19:03:17.149 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:16 smithi083 ceph-mon[36260]: 10.4 starting backfill to osd.1 from (0'0,0'0] MAX to 349'5722 2024-09-20T19:03:17.150 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:16 smithi083 ceph-mon[36260]: 10.e starting backfill to osd.5 from (0'0,0'0] MAX to 349'5818 2024-09-20T19:03:17.150 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:16 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:03:17.150 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:16 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:03:17.150 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:16 smithi083 ceph-mon[29421]: 5.5 scrub starts 2024-09-20T19:03:17.150 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:16 smithi083 ceph-mon[29421]: 5.5 scrub ok 2024-09-20T19:03:17.150 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:16 smithi083 ceph-mon[29421]: 10.a scrub ok 2024-09-20T19:03:17.150 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:16 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [3, 5]}]': finished 2024-09-20T19:03:17.150 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:16 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1e", "id": [3, 2]}]': finished 2024-09-20T19:03:17.150 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:16 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.0", "id": [0, 6]}]': finished 2024-09-20T19:03:17.150 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:16 smithi083 ceph-mon[29421]: 10.2 starting backfill to osd.3 from (0'0,0'0] MAX to 349'5729 2024-09-20T19:03:17.150 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:16 smithi083 ceph-mon[29421]: 10.c starting backfill to osd.3 from (0'0,0'0] MAX to 349'5670 2024-09-20T19:03:17.150 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:16 smithi083 ceph-mon[29421]: 10.1 starting backfill to osd.1 from (0'0,0'0] MAX to 349'5648 2024-09-20T19:03:17.151 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:16 smithi083 ceph-mon[29421]: osdmap e352: 8 total, 8 up, 7 in 2024-09-20T19:03:17.151 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:16 smithi083 ceph-mon[29421]: 10.4 starting backfill to osd.1 from (0'0,0'0] MAX to 349'5722 2024-09-20T19:03:17.151 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:16 smithi083 ceph-mon[29421]: 10.e starting backfill to osd.5 from (0'0,0'0] MAX to 349'5818 2024-09-20T19:03:17.151 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:16 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:03:17.151 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:16 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:03:17.227 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:17.263 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:17.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:17.264 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:17.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:16 smithi142 ceph-mon[32508]: 5.5 scrub starts 2024-09-20T19:03:17.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:16 smithi142 ceph-mon[32508]: 5.5 scrub ok 2024-09-20T19:03:17.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:16 smithi142 ceph-mon[32508]: 10.a scrub ok 2024-09-20T19:03:17.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:16 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [3, 5]}]': finished 2024-09-20T19:03:17.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:16 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1e", "id": [3, 2]}]': finished 2024-09-20T19:03:17.264 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:16 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.0", "id": [0, 6]}]': finished 2024-09-20T19:03:17.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:16 smithi142 ceph-mon[32508]: 10.2 starting backfill to osd.3 from (0'0,0'0] MAX to 349'5729 2024-09-20T19:03:17.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:16 smithi142 ceph-mon[32508]: 10.c starting backfill to osd.3 from (0'0,0'0] MAX to 349'5670 2024-09-20T19:03:17.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:16 smithi142 ceph-mon[32508]: 10.1 starting backfill to osd.1 from (0'0,0'0] MAX to 349'5648 2024-09-20T19:03:17.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:16 smithi142 ceph-mon[32508]: osdmap e352: 8 total, 8 up, 7 in 2024-09-20T19:03:17.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:16 smithi142 ceph-mon[32508]: 10.4 starting backfill to osd.1 from (0'0,0'0] MAX to 349'5722 2024-09-20T19:03:17.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:16 smithi142 ceph-mon[32508]: 10.e starting backfill to osd.5 from (0'0,0'0] MAX to 349'5818 2024-09-20T19:03:17.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:16 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:03:17.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:16 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:03:17.265 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:03:17 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:03:17.365 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:17.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:17.403 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:17.403 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:17.504 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:17.540 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:17.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:17.541 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:17.584 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T19:03:17.584 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:03:17.642 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:17.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:17.678 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:17.678 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:17.779 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:17.812 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:17.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:17.813 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:17.879 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 58 16 93039 93023 100.214 47.0625 0.00796181 0.00997171 2024-09-20T19:03:17.914 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:17.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:17.959 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:17.959 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:17.981 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-20T19:03:18.060 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:18.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:18.096 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:18.096 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:18.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:17 smithi142 ceph-mon[32508]: osdmap e353: 8 total, 8 up, 7 in 2024-09-20T19:03:18.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:17 smithi142 ceph-mon[32508]: pgmap v673: 148 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 activating+undersized+degraded+remapped, 1 peering, 2 active+remapped+backfill_toofull, 140 active+clean; 5.6 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 47 MiB/s wr, 755 op/s; 11654/275463 objects degraded (4.231%); 28701/275463 objects misplaced (10.419%); 161 KiB/s, 18 objects/s recovering 2024-09-20T19:03:18.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:17 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:03:18.144 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:17 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1028060837' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:03:18.197 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:18.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:18.228 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:18.228 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:18.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:17 smithi083 ceph-mon[29421]: osdmap e353: 8 total, 8 up, 7 in 2024-09-20T19:03:18.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:17 smithi083 ceph-mon[29421]: pgmap v673: 148 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 activating+undersized+degraded+remapped, 1 peering, 2 active+remapped+backfill_toofull, 140 active+clean; 5.6 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 47 MiB/s wr, 755 op/s; 11654/275463 objects degraded (4.231%); 28701/275463 objects misplaced (10.419%); 161 KiB/s, 18 objects/s recovering 2024-09-20T19:03:18.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:17 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:03:18.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:17 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1028060837' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:03:18.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:17 smithi083 ceph-mon[36260]: osdmap e353: 8 total, 8 up, 7 in 2024-09-20T19:03:18.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:17 smithi083 ceph-mon[36260]: pgmap v673: 148 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 activating+undersized+degraded+remapped, 1 peering, 2 active+remapped+backfill_toofull, 140 active+clean; 5.6 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 47 MiB/s wr, 755 op/s; 11654/275463 objects degraded (4.231%); 28701/275463 objects misplaced (10.419%); 161 KiB/s, 18 objects/s recovering 2024-09-20T19:03:18.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:17 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:03:18.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:17 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1028060837' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:03:18.329 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:18.371 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:18.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:18.372 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:18.418 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_blocked_ops 2024-09-20T19:03:18.473 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:18.538 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:18.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:18.539 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:18.640 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:18.676 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:18.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:18.677 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:18.778 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:03:18.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:18.824 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:18.824 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:03:18.879 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 59 16 93864 93848 99.3896 51.5625 0.00404735 0.0100539 2024-09-20T19:03:18.882 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:03:18.897 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T19:03:18.925 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:18.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:18.983 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:18.983 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:19.043 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:18 smithi083 ceph-mon[29421]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T19:03:19.043 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:18 smithi083 ceph-mon[29421]: Health check failed: Degraded data redundancy: 11654/275463 objects degraded (4.231%), 2 pgs degraded (PG_DEGRADED) 2024-09-20T19:03:19.043 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:18 smithi083 ceph-mon[29421]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:03:19.043 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:18 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1028060837' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T19:03:19.043 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:18 smithi083 ceph-mon[29421]: osdmap e354: 8 total, 8 up, 7 in 2024-09-20T19:03:19.044 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:18 smithi083 ceph-mon[36260]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T19:03:19.044 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:18 smithi083 ceph-mon[36260]: Health check failed: Degraded data redundancy: 11654/275463 objects degraded (4.231%), 2 pgs degraded (PG_DEGRADED) 2024-09-20T19:03:19.044 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:18 smithi083 ceph-mon[36260]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:03:19.044 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:18 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1028060837' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T19:03:19.044 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:18 smithi083 ceph-mon[36260]: osdmap e354: 8 total, 8 up, 7 in 2024-09-20T19:03:19.083 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:03:19.126 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:19.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:19.127 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:03:19.227 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:19.266 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:18 smithi142 ceph-mon[32508]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T19:03:19.266 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:18 smithi142 ceph-mon[32508]: Health check failed: Degraded data redundancy: 11654/275463 objects degraded (4.231%), 2 pgs degraded (PG_DEGRADED) 2024-09-20T19:03:19.266 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:18 smithi142 ceph-mon[32508]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:03:19.266 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:18 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1028060837' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T19:03:19.266 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:18 smithi142 ceph-mon[32508]: osdmap e354: 8 total, 8 up, 7 in 2024-09-20T19:03:19.269 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:19.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:19.270 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:19.316 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:03:19.372 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:19.395 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:19.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:19.396 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:19.496 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:19.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:19.539 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:19.540 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:19.640 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:19.676 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T19:03:19.676 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:03:19.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:19.683 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:19.683 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:19.779 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 7, 6] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 5, 6, 7, 4] 2024-09-20T19:03:19.780 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T19:03:19.780 INFO:tasks.thrashosds.thrasher:inject_pause on osd.2 2024-09-20T19:03:19.780 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-20T19:03:19.780 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-20T19:03:19.780 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.2 config set filestore_inject_stall 3 2024-09-20T19:03:19.784 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:03:19.832 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:19.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:19.833 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:03:19.880 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:2024-09-20T19:03:19.874334+0000 min lat: 0.00125834 max lat: 2.39565 avg lat: 0.0101421 2024-09-20T19:03:19.880 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-20T19:03:19.880 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 60 16 94645 94629 98.5467 48.8125 0.00952354 0.0101421 2024-09-20T19:03:19.933 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:19.954 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:03:19.954 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:03:19.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:19.956 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:19.956 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:19.972 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:03:19.972 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:03:19.990 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:03:19.991 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:03:20.006 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:03:20.007 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:03:20.024 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:03:20.024 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:03:20.025 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:19 smithi083 ceph-mon[29421]: pgmap v675: 148 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 activating+undersized+degraded+remapped, 1 peering, 3 active+remapped+backfill_toofull, 139 active+clean; 5.6 GiB data, 17 GiB used, 608 GiB / 626 GiB avail; 1.2 KiB/s rd, 50 MiB/s wr, 804 op/s; 11654/276462 objects degraded (4.215%); 34478/276462 objects misplaced (12.471%); 154 KiB/s, 24 objects/s recovering 2024-09-20T19:03:20.025 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:19 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3722292660' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:03:20.025 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:19 smithi083 ceph-mon[36260]: pgmap v675: 148 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 activating+undersized+degraded+remapped, 1 peering, 3 active+remapped+backfill_toofull, 139 active+clean; 5.6 GiB data, 17 GiB used, 608 GiB / 626 GiB avail; 1.2 KiB/s rd, 50 MiB/s wr, 804 op/s; 11654/276462 objects degraded (4.215%); 34478/276462 objects misplaced (12.471%); 154 KiB/s, 24 objects/s recovering 2024-09-20T19:03:20.025 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:19 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3722292660' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:03:20.056 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:03:20.056 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:03:20.057 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:20.092 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:03:20.093 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:03:20.105 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:20.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:20.106 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:20.127 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:03:20.127 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:03:20.208 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:20.240 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:20.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:20.241 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:20.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:19 smithi142 ceph-mon[32508]: pgmap v675: 148 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 activating+undersized+degraded+remapped, 1 peering, 3 active+remapped+backfill_toofull, 139 active+clean; 5.6 GiB data, 17 GiB used, 608 GiB / 626 GiB avail; 1.2 KiB/s rd, 50 MiB/s wr, 804 op/s; 11654/276462 objects degraded (4.215%); 34478/276462 objects misplaced (12.471%); 154 KiB/s, 24 objects/s recovering 2024-09-20T19:03:20.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:20 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3722292660' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:03:20.341 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:20.366 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:20.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:20.367 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:20.468 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:20.507 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:20.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:20.509 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:20.609 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:20.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:20.643 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:20.643 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:20.744 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:03:20.785 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:20.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:20.786 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:03:20.793 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:03:20 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:03:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:03:20.795 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T19:03:20.880 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 61 16 95347 95331 97.6505 43.875 0.0164442 0.0102353 2024-09-20T19:03:20.886 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:03:20.921 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:20.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:20.922 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:03:21.021 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:03:21.023 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:21.080 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:21.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:21.081 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:21.182 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:21.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:21.218 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:21.218 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:21.318 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:21.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:21.355 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:21.355 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:21.456 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:21.490 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:21.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:21.491 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:21.592 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:21.625 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:21.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:21.626 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:21.727 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:21.763 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:21.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:21.764 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:21.865 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:21.880 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 62 16 96088 96072 96.8225 46.3125 0.0232027 0.0103213 2024-09-20T19:03:21.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:21.913 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:21.913 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:22.014 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:22.043 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:21 smithi083 ceph-mon[29421]: pgmap v676: 148 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 activating+undersized+degraded+remapped, 1 peering, 3 active+remapped+backfill_toofull, 139 active+clean; 5.6 GiB data, 17 GiB used, 608 GiB / 626 GiB avail; 853 B/s rd, 35 MiB/s wr, 559 op/s; 11654/276462 objects degraded (4.215%); 34478/276462 objects misplaced (12.471%); 107 KiB/s, 16 objects/s recovering 2024-09-20T19:03:22.043 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:21 smithi083 ceph-mon[36260]: pgmap v676: 148 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 activating+undersized+degraded+remapped, 1 peering, 3 active+remapped+backfill_toofull, 139 active+clean; 5.6 GiB data, 17 GiB used, 608 GiB / 626 GiB avail; 853 B/s rd, 35 MiB/s wr, 559 op/s; 11654/276462 objects degraded (4.215%); 34478/276462 objects misplaced (12.471%); 107 KiB/s, 16 objects/s recovering 2024-09-20T19:03:22.045 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:22.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:22.046 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:22.147 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:22.176 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:22.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:22.177 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:22.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:21 smithi142 ceph-mon[32508]: pgmap v676: 148 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 activating+undersized+degraded+remapped, 1 peering, 3 active+remapped+backfill_toofull, 139 active+clean; 5.6 GiB data, 17 GiB used, 608 GiB / 626 GiB avail; 853 B/s rd, 35 MiB/s wr, 559 op/s; 11654/276462 objects degraded (4.215%); 34478/276462 objects misplaced (12.471%); 107 KiB/s, 16 objects/s recovering 2024-09-20T19:03:22.279 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:22.310 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:22.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:22.311 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:22.411 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:03:22.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:22.446 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:22.446 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:03:22.547 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:22.568 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:22.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:22.569 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:22.669 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:22.703 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:22.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:22.704 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:22.805 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:22.826 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:22.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:22.828 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:22.880 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 63 16 96755 96739 95.9474 41.6875 0.00487894 0.0104149 2024-09-20T19:03:22.917 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T19:03:22.918 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-20T19:03:22.918 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T19:03:22.928 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:22.972 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:22.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:22.973 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:22.978 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T19:03:23.073 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:23.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:23.112 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:23.113 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:23.213 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:23.239 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:23.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:23.240 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:23.341 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:23.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:23.368 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:23.368 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:23.469 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:23.496 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:23.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:23.497 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:23.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:23 smithi142 ceph-mon[32508]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 11654/276462 objects degraded (4.215%), 2 pgs degraded) 2024-09-20T19:03:23.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:23 smithi083 ceph-mon[29421]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 11654/276462 objects degraded (4.215%), 2 pgs degraded) 2024-09-20T19:03:23.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:23 smithi083 ceph-mon[36260]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 11654/276462 objects degraded (4.215%), 2 pgs degraded) 2024-09-20T19:03:23.598 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:23.638 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:23.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:23.640 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:23.693 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:03:23.740 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:23.787 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:23.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:23.788 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:23.880 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 64 16 97513 97497 95.1885 47.375 0.00767258 0.0104974 2024-09-20T19:03:23.888 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:23.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:23.929 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:23.929 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:24.029 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:03:24.062 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:24.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:24.063 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:03:24.164 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:24.207 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:24.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:24.208 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:24.309 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:24.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:24.336 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:24.337 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:24.437 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:03:24.478 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:24.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:24.479 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:03:24.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:24 smithi142 ceph-mon[32508]: pgmap v677: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 5.8 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.4 KiB/s rd, 62 MiB/s wr, 995 op/s; 45420/287766 objects misplaced (15.784%); 17 MiB/s, 278 objects/s recovering 2024-09-20T19:03:24.525 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:24 smithi083 ceph-mon[29421]: pgmap v677: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 5.8 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.4 KiB/s rd, 62 MiB/s wr, 995 op/s; 45420/287766 objects misplaced (15.784%); 17 MiB/s, 278 objects/s recovering 2024-09-20T19:03:24.526 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:24 smithi083 ceph-mon[36260]: pgmap v677: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 5.8 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.4 KiB/s rd, 62 MiB/s wr, 995 op/s; 45420/287766 objects misplaced (15.784%); 17 MiB/s, 278 objects/s recovering 2024-09-20T19:03:24.580 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:24.615 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:24.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:24.616 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:24.717 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:24.746 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:24.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:24.748 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:24.848 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:24.880 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:03:24.880 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 65 16 98199 98183 94.3838 42.875 0.00922115 0.0105873 2024-09-20T19:03:24.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:24.883 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:24.883 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:24.984 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:03:25.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:25.020 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:25.020 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:03:25.121 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:25.156 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:25.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:25.157 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:25.233 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-20T19:03:25.258 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:25.293 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:25.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:25.294 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:25.394 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:25.424 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:25.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:25.425 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:25.433 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:25 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/228586182' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:03:25.433 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:25 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:03:25.433 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:25 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/228586182' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:03:25.433 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:25 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:03:25.453 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:03:25.453 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:03:25.487 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:03:25.488 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:03:25.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:25 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/228586182' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:03:25.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:25 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:03:25.523 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:03:25.523 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:03:25.526 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:25.551 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:03:25.551 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:03:25.555 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:25.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:25.557 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:25.568 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:03:25.568 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:03:25.584 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:03:25.584 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:03:25.632 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:03:25.632 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:03:25.658 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:25.666 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:03:25.666 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:03:25.693 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:25.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:25.694 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:25.795 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:25.824 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:25.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:25.825 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:25.880 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 66 16 98905 98889 93.6221 44.125 0.00559087 0.0106729 2024-09-20T19:03:25.926 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:25.962 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:25.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:25.963 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:26.064 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:26.094 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:26.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:26.095 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:26.196 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:26.217 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:26.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:26.218 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:26.319 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:26.366 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:26.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:26.367 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:26.467 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:26.494 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:26.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:26.495 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:26.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:26 smithi142 ceph-mon[32508]: pgmap v678: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 5.8 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 628 B/s rd, 31 MiB/s wr, 503 op/s; 45420/287766 objects misplaced (15.784%); 14 MiB/s, 235 objects/s recovering 2024-09-20T19:03:26.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:26 smithi142 ceph-mon[32508]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:03:26.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:26 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T19:03:26.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:26 smithi142 ceph-mon[32508]: osdmap e355: 8 total, 8 up, 7 in 2024-09-20T19:03:26.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:26 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1446020339' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:03:26.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:26 smithi142 ceph-mon[32508]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T19:03:26.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:26 smithi083 ceph-mon[29421]: pgmap v678: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 5.8 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 628 B/s rd, 31 MiB/s wr, 503 op/s; 45420/287766 objects misplaced (15.784%); 14 MiB/s, 235 objects/s recovering 2024-09-20T19:03:26.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:26 smithi083 ceph-mon[29421]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:03:26.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:26 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T19:03:26.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:26 smithi083 ceph-mon[29421]: osdmap e355: 8 total, 8 up, 7 in 2024-09-20T19:03:26.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:26 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1446020339' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:03:26.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:26 smithi083 ceph-mon[29421]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T19:03:26.544 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:26 smithi083 ceph-mon[36260]: pgmap v678: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 5.8 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 628 B/s rd, 31 MiB/s wr, 503 op/s; 45420/287766 objects misplaced (15.784%); 14 MiB/s, 235 objects/s recovering 2024-09-20T19:03:26.544 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:26 smithi083 ceph-mon[36260]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:03:26.544 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:26 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T19:03:26.544 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:26 smithi083 ceph-mon[36260]: osdmap e355: 8 total, 8 up, 7 in 2024-09-20T19:03:26.544 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:26 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1446020339' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:03:26.544 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:26 smithi083 ceph-mon[36260]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T19:03:26.596 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:26.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:26.627 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:26.627 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:26.728 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:26.733 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_historic_ops 2024-09-20T19:03:26.757 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:26.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:26.758 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:26.859 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:26.880 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 67 16 99677 99661 92.9447 48.25 0.0128243 0.0107514 2024-09-20T19:03:26.894 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:26.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:26.895 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:26.996 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:27.040 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:27.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:27.041 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:27.142 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:27.195 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:03:27.199 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:27.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:27.200 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:27.300 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:03:27.343 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:27.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:27.344 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:03:27.445 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:27.479 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:27.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:27.480 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:27.513 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:03:27 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:03:27.581 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:27.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:27.605 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:27.605 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:27.706 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:27.727 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:27.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:27.728 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:27.829 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:03:27.864 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:27.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:27.865 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:03:27.880 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 68 16 100376 100360 92.2204 43.6875 0.0119041 0.0108376 2024-09-20T19:03:27.966 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:28.003 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:28.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:28.005 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:28.105 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:28.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:28.136 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:28.136 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:28.199 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T19:03:28.236 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:28.265 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:28.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:28.266 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:28.367 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:28.406 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:28.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:28.407 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:28.508 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:28.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:28 smithi142 ceph-mon[32508]: pgmap v680: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 6.0 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1.1 KiB/s rd, 48 MiB/s wr, 761 op/s; 44258/297276 objects misplaced (14.888%); 26 MiB/s, 423 objects/s recovering 2024-09-20T19:03:28.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:28 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:03:28.537 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:03:28.539 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:28.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:28.540 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:28.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:28 smithi083 ceph-mon[29421]: pgmap v680: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 6.0 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1.1 KiB/s rd, 48 MiB/s wr, 761 op/s; 44258/297276 objects misplaced (14.888%); 26 MiB/s, 423 objects/s recovering 2024-09-20T19:03:28.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:28 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:03:28.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:28 smithi083 ceph-mon[36260]: pgmap v680: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 6.0 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1.1 KiB/s rd, 48 MiB/s wr, 761 op/s; 44258/297276 objects misplaced (14.888%); 26 MiB/s, 423 objects/s recovering 2024-09-20T19:03:28.544 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:28 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:03:28.641 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:03:28.669 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:28.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:28.670 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:03:28.771 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:28.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:28.819 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:28.819 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:28.881 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 69 16 100974 100958 91.4257 37.375 0.00428698 0.0109308 2024-09-20T19:03:28.919 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:28.952 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:28.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:28.953 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:29.054 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:29.092 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T19:03:29.093 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:29.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:29.094 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:29.195 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:29.232 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:29.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:29.233 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:29.286 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 7, 6] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 5, 6, 7, 4] 2024-09-20T19:03:29.287 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T19:03:29.287 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-20T19:03:29.287 INFO:tasks.thrashosds.thrasher:Setting osd 0 primary_affinity to 1.000000 2024-09-20T19:03:29.287 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd primary-affinity 0 1 2024-09-20T19:03:29.334 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:29.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:29.406 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:29.406 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:29.507 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:29.530 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:03:29.541 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:29.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:29.542 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:29.643 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:29.676 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:29.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:29.677 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:29.778 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:29.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:29.828 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:29.828 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:29.881 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 70 16 101699 101683 90.7669 45.3125 0.0232468 0.0110092 2024-09-20T19:03:29.929 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:29.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:29.960 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:29.960 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:30.061 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:30.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:30.099 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:30.100 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:30.116 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:03:30.116 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:03:30.149 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:03:30.149 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:03:30.160 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:03:30.183 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:03:30.183 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:03:30.201 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:30.215 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:03:30.215 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:03:30.234 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:30.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:30.235 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:30.242 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:03:30.243 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:03:30.261 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:03:30.261 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:03:30.304 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-20T19:03:30.335 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:30.341 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:03:30.341 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:03:30.368 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:30.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:30.369 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:30.381 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:03:30.381 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:03:30.470 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:30.480 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:30 smithi083 ceph-mon[29421]: pgmap v681: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 6.1 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1023 B/s rd, 46 MiB/s wr, 735 op/s; 44448/298503 objects misplaced (14.890%); 24 MiB/s, 386 objects/s recovering 2024-09-20T19:03:30.481 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:30 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1194004184' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:03:30.481 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:30 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/302252273' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:03:30.481 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:30 smithi083 ceph-mon[36260]: pgmap v681: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 6.1 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1023 B/s rd, 46 MiB/s wr, 735 op/s; 44448/298503 objects misplaced (14.890%); 24 MiB/s, 386 objects/s recovering 2024-09-20T19:03:30.481 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:30 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1194004184' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:03:30.481 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:30 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/302252273' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:03:30.503 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:30.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:30.504 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:30.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:30 smithi142 ceph-mon[32508]: pgmap v681: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 6.1 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1023 B/s rd, 46 MiB/s wr, 735 op/s; 44448/298503 objects misplaced (14.890%); 24 MiB/s, 386 objects/s recovering 2024-09-20T19:03:30.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:30 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1194004184' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:03:30.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:30 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/302252273' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:03:30.605 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:30.646 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:30.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:30.647 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:30.748 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:30.774 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:03:30 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:03:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:03:30.778 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:30.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:30.780 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:30.881 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:30.881 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 71 16 102370 102354 90.0792 41.9375 0.0160656 0.011096 2024-09-20T19:03:30.909 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:30.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:30.910 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:31.011 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:31.031 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:31.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:31.032 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:31.133 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:31.169 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:31.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:31.170 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:31.270 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:31.310 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:31.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:31.311 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:31.315 INFO:teuthology.orchestra.run.smithi083.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-20T19:03:31.413 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:31.447 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:31.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:31.448 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:31.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:31 smithi142 ceph-mon[32508]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:03:31.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:31 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1194004184' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T19:03:31.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:31 smithi142 ceph-mon[32508]: osdmap e356: 8 total, 8 up, 7 in 2024-09-20T19:03:31.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:31 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/625636045' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 0, "weight": 1.0} : dispatch 2024-09-20T19:03:31.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:31 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 0, "weight": 1.0} : dispatch 2024-09-20T19:03:31.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:31 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:03:31.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:31 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:03:31.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:31 smithi083 ceph-mon[29421]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:03:31.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:31 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1194004184' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T19:03:31.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:31 smithi083 ceph-mon[29421]: osdmap e356: 8 total, 8 up, 7 in 2024-09-20T19:03:31.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:31 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/625636045' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 0, "weight": 1.0} : dispatch 2024-09-20T19:03:31.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:31 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 0, "weight": 1.0} : dispatch 2024-09-20T19:03:31.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:31 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:03:31.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:31 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:03:31.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:31 smithi083 ceph-mon[36260]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:03:31.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:31 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1194004184' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T19:03:31.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:31 smithi083 ceph-mon[36260]: osdmap e356: 8 total, 8 up, 7 in 2024-09-20T19:03:31.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:31 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/625636045' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 0, "weight": 1.0} : dispatch 2024-09-20T19:03:31.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:31 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 0, "weight": 1.0} : dispatch 2024-09-20T19:03:31.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:31 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:03:31.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:31 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:03:31.550 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:03:31.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:31.586 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:31.586 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:03:31.687 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:03:31.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:31.725 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:31.725 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:03:31.826 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:03:31.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:31.862 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:31.862 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:03:31.881 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 72 16 103006 102990 89.3803 39.75 0.0398112 0.0111799 2024-09-20T19:03:31.963 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:03:31.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:31.995 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:31.996 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:03:32.096 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:32.148 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:32.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:32.149 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:32.250 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:32.278 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:32.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:32.279 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:32.355 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:32 smithi142 ceph-mon[32508]: pgmap v683: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 6.1 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 639 B/s rd, 28 MiB/s wr, 447 op/s; 44448/298503 objects misplaced (14.890%); 15 MiB/s, 246 objects/s recovering 2024-09-20T19:03:32.355 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:32 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]': finished 2024-09-20T19:03:32.355 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:32 smithi142 ceph-mon[32508]: osdmap e357: 8 total, 8 up, 7 in 2024-09-20T19:03:32.380 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:32.413 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:32.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:32.414 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:32.515 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:32.548 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:32.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:32.550 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:32.650 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:32.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:32.683 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:32.684 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:32.784 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:32.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:32 smithi083 ceph-mon[29421]: pgmap v683: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 6.1 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 639 B/s rd, 28 MiB/s wr, 447 op/s; 44448/298503 objects misplaced (14.890%); 15 MiB/s, 246 objects/s recovering 2024-09-20T19:03:32.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:32 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]': finished 2024-09-20T19:03:32.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:32 smithi083 ceph-mon[29421]: osdmap e357: 8 total, 8 up, 7 in 2024-09-20T19:03:32.794 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:32 smithi083 ceph-mon[36260]: pgmap v683: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 6.1 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 639 B/s rd, 28 MiB/s wr, 447 op/s; 44448/298503 objects misplaced (14.890%); 15 MiB/s, 246 objects/s recovering 2024-09-20T19:03:32.794 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:32 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]': finished 2024-09-20T19:03:32.794 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:32 smithi083 ceph-mon[36260]: osdmap e357: 8 total, 8 up, 7 in 2024-09-20T19:03:32.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:32.820 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:32.820 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:32.881 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 73 16 103764 103748 88.8048 47.375 0.00414594 0.0112535 2024-09-20T19:03:32.921 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:32.948 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:32.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:32.949 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:33.050 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:33.079 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:33.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:33.081 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:33.181 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:33.220 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:33.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:33.221 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:33.258 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T19:03:33.258 INFO:teuthology.orchestra.run.smithi142.stdout: "size": 20, 2024-09-20T19:03:33.258 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 600, 2024-09-20T19:03:33.258 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-09-20T19:03:33.258 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.258 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.24955.0:101455 8.8 e190/160 8:1ef316d4:::benchmark_data_smithi083_97422_object101454:head v 190'6357, mlcod=190'6355)", 2024-09-20T19:03:33.259 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:55:52.357548+0000", 2024-09-20T19:03:33.259 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 459.89669196699998, 2024-09-20T19:03:33.259 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.15217040600000001, 2024-09-20T19:03:33.259 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:03:33.259 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:03:33.259 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:03:33.259 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:03:33.259 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.259 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:03:33.259 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.357548+0000", 2024-09-20T19:03:33.259 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:33.259 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.259 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.260 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:03:33.260 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.357548+0000", 2024-09-20T19:03:33.260 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:33.260 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.260 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.260 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:03:33.260 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.357549+0000", 2024-09-20T19:03:33.260 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.0800000000000004e-07 2024-09-20T19:03:33.260 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.260 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.260 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:03:33.260 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.357594+0000", 2024-09-20T19:03:33.260 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.5052999999999997e-05 2024-09-20T19:03:33.260 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.261 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.261 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:03:33.261 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.357603+0000", 2024-09-20T19:03:33.261 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.0999999999999993e-06 2024-09-20T19:03:33.261 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.261 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.261 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:03:33.261 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.357610+0000", 2024-09-20T19:03:33.261 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5459999999999997e-06 2024-09-20T19:03:33.261 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.261 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.261 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:33.261 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.357656+0000", 2024-09-20T19:03:33.262 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.6118999999999997e-05 2024-09-20T19:03:33.262 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.262 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.262 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:03:33.262 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.357668+0000", 2024-09-20T19:03:33.262 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2367e-05 2024-09-20T19:03:33.262 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.262 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.262 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:03:33.262 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.360245+0000", 2024-09-20T19:03:33.262 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0025771190000000001 2024-09-20T19:03:33.262 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.263 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.263 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:03:33.263 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.509719+0000", 2024-09-20T19:03:33.263 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.14947329400000001 2024-09-20T19:03:33.263 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:33.263 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:03:33.263 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:33.263 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.263 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.263 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.24955.0:101453 8.6 e190/182 8:61900b1b:::benchmark_data_smithi083_97422_object101452:head v 190'6308, mlcod=0'0)", 2024-09-20T19:03:33.263 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:55:52.360511+0000", 2024-09-20T19:03:33.263 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 459.89372890700002, 2024-09-20T19:03:33.263 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.15620442300000001, 2024-09-20T19:03:33.263 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:03:33.264 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:03:33.264 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:03:33.264 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:03:33.264 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.264 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:03:33.264 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.360511+0000", 2024-09-20T19:03:33.264 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:33.264 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.264 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.264 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:03:33.264 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.360511+0000", 2024-09-20T19:03:33.264 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:33.264 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.265 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.265 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:03:33.265 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.360512+0000", 2024-09-20T19:03:33.265 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.0800000000000003e-07 2024-09-20T19:03:33.265 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.265 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.265 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:03:33.265 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.360571+0000", 2024-09-20T19:03:33.265 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8850000000000001e-05 2024-09-20T19:03:33.265 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.265 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.265 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:03:33.265 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.509692+0000", 2024-09-20T19:03:33.265 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.14912044099999999 2024-09-20T19:03:33.266 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.266 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.266 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:03:33.266 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.509707+0000", 2024-09-20T19:03:33.266 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5568000000000001e-05 2024-09-20T19:03:33.266 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.266 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.266 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:33.266 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.509813+0000", 2024-09-20T19:03:33.266 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00010567400000000001 2024-09-20T19:03:33.267 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.267 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.267 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:03:33.267 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.509836+0000", 2024-09-20T19:03:33.267 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2853999999999998e-05 2024-09-20T19:03:33.267 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.267 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.267 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:03:33.267 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.516559+0000", 2024-09-20T19:03:33.267 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.006723807 2024-09-20T19:03:33.267 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.267 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.267 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:03:33.267 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.516716+0000", 2024-09-20T19:03:33.268 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015632100000000001 2024-09-20T19:03:33.268 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:33.268 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:03:33.268 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:33.268 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.268 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.268 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.24955.0:101471 8.5 e190/182 8:a0258bcb:::benchmark_data_smithi083_97422_object101470:head v 190'6528, mlcod=0'0)", 2024-09-20T19:03:33.268 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:55:52.379819+0000", 2024-09-20T19:03:33.268 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 459.87442132699999, 2024-09-20T19:03:33.268 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13654260100000001, 2024-09-20T19:03:33.268 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:03:33.268 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:03:33.268 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:03:33.268 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:03:33.269 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.269 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:03:33.269 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.379819+0000", 2024-09-20T19:03:33.269 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:33.269 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.269 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.269 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:03:33.269 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.379819+0000", 2024-09-20T19:03:33.269 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:33.269 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.269 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.269 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:03:33.269 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.379820+0000", 2024-09-20T19:03:33.269 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5519999999999999e-06 2024-09-20T19:03:33.270 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.270 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.270 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:03:33.270 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.379854+0000", 2024-09-20T19:03:33.270 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.3157000000000001e-05 2024-09-20T19:03:33.270 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.270 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.270 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:03:33.270 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.509694+0000", 2024-09-20T19:03:33.270 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.12983992999999999 2024-09-20T19:03:33.270 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.270 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.270 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:03:33.270 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.509706+0000", 2024-09-20T19:03:33.271 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2965e-05 2024-09-20T19:03:33.271 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.271 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.271 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:33.271 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.509768+0000", 2024-09-20T19:03:33.271 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.1013e-05 2024-09-20T19:03:33.271 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.271 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.271 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:03:33.271 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.509797+0000", 2024-09-20T19:03:33.271 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9916999999999998e-05 2024-09-20T19:03:33.271 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.271 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.271 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:03:33.272 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.516311+0000", 2024-09-20T19:03:33.272 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0065133980000000001 2024-09-20T19:03:33.272 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.272 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.272 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:03:33.272 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:55:52.516361+0000", 2024-09-20T19:03:33.272 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0668999999999999e-05 2024-09-20T19:03:33.272 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:33.272 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:03:33.272 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:33.272 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.272 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.272 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:72379 9.d 9:bae8f567:::benchmark_data_smithi083_124552_object72378:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e265)", 2024-09-20T19:03:33.272 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:26.629075+0000", 2024-09-20T19:03:33.273 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 245.62516521699999, 2024-09-20T19:03:33.273 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.133017778, 2024-09-20T19:03:33.273 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:03:33.273 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:03:33.273 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:03:33.273 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:03:33.273 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:03:33.273 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:03:33.273 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 72379 2024-09-20T19:03:33.273 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.273 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:03:33.273 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.273 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:03:33.273 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.629075+0000", 2024-09-20T19:03:33.273 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:33.274 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.274 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.274 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:03:33.274 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.629075+0000", 2024-09-20T19:03:33.274 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:33.274 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.274 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.274 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:03:33.274 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.629077+0000", 2024-09-20T19:03:33.274 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.753e-06 2024-09-20T19:03:33.274 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.274 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.274 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:03:33.274 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.629117+0000", 2024-09-20T19:03:33.275 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0463000000000002e-05 2024-09-20T19:03:33.275 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.275 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.275 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:03:33.275 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.629128+0000", 2024-09-20T19:03:33.275 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1252e-05 2024-09-20T19:03:33.275 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.275 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.275 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:03:33.275 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.629136+0000", 2024-09-20T19:03:33.275 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5850000000000002e-06 2024-09-20T19:03:33.275 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.275 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.275 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:33.276 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.629297+0000", 2024-09-20T19:03:33.276 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00016093100000000001 2024-09-20T19:03:33.276 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.276 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.276 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:03:33.276 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.629447+0000", 2024-09-20T19:03:33.276 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000149717 2024-09-20T19:03:33.276 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.276 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.276 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,5", 2024-09-20T19:03:33.276 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.629595+0000", 2024-09-20T19:03:33.276 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000148436 2024-09-20T19:03:33.276 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.276 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.276 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:03:33.277 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.638400+0000", 2024-09-20T19:03:33.277 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0088051139999999993 2024-09-20T19:03:33.277 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.277 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.277 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:03:33.277 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.638412+0000", 2024-09-20T19:03:33.277 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1804000000000001e-05 2024-09-20T19:03:33.277 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.277 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.277 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:03:33.277 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.761963+0000", 2024-09-20T19:03:33.277 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.123550901 2024-09-20T19:03:33.277 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.278 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.278 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:03:33.278 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.762026+0000", 2024-09-20T19:03:33.278 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.3350999999999997e-05 2024-09-20T19:03:33.278 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.278 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.278 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:03:33.278 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.762093+0000", 2024-09-20T19:03:33.278 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.6470999999999994e-05 2024-09-20T19:03:33.278 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:33.278 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:03:33.278 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:33.278 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.278 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.279 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:72382 9.d 9:b4fdabfe:::benchmark_data_smithi083_124552_object72381:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e265)", 2024-09-20T19:03:33.279 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:26.629907+0000", 2024-09-20T19:03:33.279 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 245.624333651, 2024-09-20T19:03:33.279 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13337285700000001, 2024-09-20T19:03:33.279 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:03:33.279 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:03:33.279 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:03:33.279 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:03:33.279 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:03:33.279 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:03:33.279 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 72382 2024-09-20T19:03:33.279 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.279 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:03:33.279 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.280 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:03:33.280 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.629907+0000", 2024-09-20T19:03:33.280 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:33.280 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.280 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.280 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:03:33.280 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.629907+0000", 2024-09-20T19:03:33.280 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:33.280 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.280 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.280 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:03:33.280 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.629908+0000", 2024-09-20T19:03:33.280 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.206e-06 2024-09-20T19:03:33.280 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.281 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.281 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:03:33.281 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.629934+0000", 2024-09-20T19:03:33.281 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.6463000000000001e-05 2024-09-20T19:03:33.281 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.281 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.281 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:03:33.281 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.629946+0000", 2024-09-20T19:03:33.281 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1579e-05 2024-09-20T19:03:33.281 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.281 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.281 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:03:33.281 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.629953+0000", 2024-09-20T19:03:33.281 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.1810000000000004e-06 2024-09-20T19:03:33.282 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.282 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.282 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:33.282 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.632103+0000", 2024-09-20T19:03:33.282 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.002150365 2024-09-20T19:03:33.282 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.282 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.282 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:03:33.282 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.632219+0000", 2024-09-20T19:03:33.282 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000115746 2024-09-20T19:03:33.282 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.282 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.282 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,5", 2024-09-20T19:03:33.282 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.632337+0000", 2024-09-20T19:03:33.283 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011818499999999999 2024-09-20T19:03:33.283 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.283 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.283 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:03:33.283 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.638442+0000", 2024-09-20T19:03:33.283 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0061042379999999997 2024-09-20T19:03:33.283 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.283 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.283 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:03:33.283 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.638466+0000", 2024-09-20T19:03:33.283 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.4266000000000001e-05 2024-09-20T19:03:33.283 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.283 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.283 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:03:33.284 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.763189+0000", 2024-09-20T19:03:33.284 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.124723375 2024-09-20T19:03:33.284 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.284 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.284 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:03:33.284 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.763240+0000", 2024-09-20T19:03:33.284 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0708999999999998e-05 2024-09-20T19:03:33.284 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.284 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.284 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:03:33.284 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.763279+0000", 2024-09-20T19:03:33.284 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9544e-05 2024-09-20T19:03:33.284 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:33.284 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:03:33.284 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:33.285 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.285 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.285 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:72386 9.3 9:c310ff33:::benchmark_data_smithi083_124552_object72385:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e265)", 2024-09-20T19:03:33.285 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:26.634291+0000", 2024-09-20T19:03:33.285 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 245.61994876399999, 2024-09-20T19:03:33.285 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.133404362, 2024-09-20T19:03:33.285 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:03:33.285 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:03:33.285 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:03:33.285 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:03:33.285 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:03:33.285 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:03:33.285 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 72386 2024-09-20T19:03:33.285 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.286 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:03:33.286 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.286 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:03:33.286 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.634291+0000", 2024-09-20T19:03:33.286 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:33.286 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.286 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.286 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:03:33.286 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.634291+0000", 2024-09-20T19:03:33.286 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:33.286 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.286 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.286 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:03:33.286 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.634293+0000", 2024-09-20T19:03:33.287 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4810000000000001e-06 2024-09-20T19:03:33.287 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.287 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.287 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:03:33.287 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.634352+0000", 2024-09-20T19:03:33.287 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9089000000000003e-05 2024-09-20T19:03:33.287 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.287 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.287 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:03:33.287 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.634363+0000", 2024-09-20T19:03:33.287 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1100999999999999e-05 2024-09-20T19:03:33.287 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.287 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.287 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:03:33.288 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.634371+0000", 2024-09-20T19:03:33.288 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.0490000000000004e-06 2024-09-20T19:03:33.288 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.288 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.288 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:33.288 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.634434+0000", 2024-09-20T19:03:33.288 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2587000000000002e-05 2024-09-20T19:03:33.288 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.288 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.288 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:03:33.288 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.634569+0000", 2024-09-20T19:03:33.288 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013524699999999999 2024-09-20T19:03:33.288 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.288 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.289 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,3", 2024-09-20T19:03:33.289 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.634714+0000", 2024-09-20T19:03:33.289 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00014523699999999999 2024-09-20T19:03:33.289 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.289 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.289 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:03:33.289 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.639963+0000", 2024-09-20T19:03:33.289 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0052484699999999999 2024-09-20T19:03:33.289 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.289 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.289 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:03:33.289 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.640003+0000", 2024-09-20T19:03:33.289 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0185999999999998e-05 2024-09-20T19:03:33.289 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.289 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.290 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:03:33.290 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.767600+0000", 2024-09-20T19:03:33.290 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.127597345 2024-09-20T19:03:33.290 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.290 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.290 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:03:33.290 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.767655+0000", 2024-09-20T19:03:33.290 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5003999999999997e-05 2024-09-20T19:03:33.290 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.290 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.290 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:03:33.290 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.767696+0000", 2024-09-20T19:03:33.290 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0565999999999998e-05 2024-09-20T19:03:33.290 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:33.291 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:03:33.291 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:33.291 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.291 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.291 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:72388 9.3 9:c69fcb53:::benchmark_data_smithi083_124552_object72387:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e265)", 2024-09-20T19:03:33.291 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:26.635542+0000", 2024-09-20T19:03:33.291 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 245.61869864100001, 2024-09-20T19:03:33.291 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.132329168, 2024-09-20T19:03:33.291 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:03:33.291 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:03:33.291 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:03:33.291 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:03:33.291 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:03:33.291 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:03:33.292 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 72388 2024-09-20T19:03:33.292 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.292 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:03:33.292 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.292 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:03:33.292 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.635542+0000", 2024-09-20T19:03:33.292 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:33.292 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.292 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.292 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:03:33.292 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.635542+0000", 2024-09-20T19:03:33.292 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:33.292 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.292 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.293 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:03:33.293 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.635542+0000", 2024-09-20T19:03:33.293 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.7899999999999998e-07 2024-09-20T19:03:33.293 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.293 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.293 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:03:33.293 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.635617+0000", 2024-09-20T19:03:33.293 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.4859999999999998e-05 2024-09-20T19:03:33.293 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.293 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.293 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:03:33.293 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.635625+0000", 2024-09-20T19:03:33.293 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5979999999999999e-06 2024-09-20T19:03:33.294 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.294 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.294 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:03:33.294 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.635631+0000", 2024-09-20T19:03:33.294 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8599999999999998e-06 2024-09-20T19:03:33.294 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.294 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.294 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:33.294 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.637372+0000", 2024-09-20T19:03:33.294 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001741352 2024-09-20T19:03:33.294 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.294 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.294 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:03:33.294 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.637487+0000", 2024-09-20T19:03:33.295 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000115296 2024-09-20T19:03:33.295 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.295 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.295 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,3", 2024-09-20T19:03:33.295 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.637613+0000", 2024-09-20T19:03:33.295 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012625200000000001 2024-09-20T19:03:33.295 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.295 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.295 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:03:33.295 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.640092+0000", 2024-09-20T19:03:33.295 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.002478975 2024-09-20T19:03:33.295 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.295 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.295 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:03:33.296 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.640178+0000", 2024-09-20T19:03:33.296 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.5749999999999997e-05 2024-09-20T19:03:33.296 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.296 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.296 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:03:33.296 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.767762+0000", 2024-09-20T19:03:33.296 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.127583327 2024-09-20T19:03:33.296 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.296 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.296 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:03:33.296 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.767818+0000", 2024-09-20T19:03:33.296 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6589999999999999e-05 2024-09-20T19:03:33.296 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.296 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.297 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:03:33.297 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:26.767871+0000", 2024-09-20T19:03:33.297 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.2629e-05 2024-09-20T19:03:33.297 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:33.297 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:03:33.297 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:33.297 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.297 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.297 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:84060 9.e 9:73c1c2b3:::benchmark_data_smithi083_124552_object84059:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e268)", 2024-09-20T19:03:33.297 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:38.588091+0000", 2024-09-20T19:03:33.297 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 233.66614952399999, 2024-09-20T19:03:33.297 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13688319800000001, 2024-09-20T19:03:33.297 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:03:33.297 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:03:33.298 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:03:33.298 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:03:33.298 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:03:33.298 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:03:33.298 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 84060 2024-09-20T19:03:33.298 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.298 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:03:33.298 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.298 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:03:33.298 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.588091+0000", 2024-09-20T19:03:33.298 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:33.298 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.298 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.298 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:03:33.298 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.588091+0000", 2024-09-20T19:03:33.299 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:33.299 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.299 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.299 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:03:33.299 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.588092+0000", 2024-09-20T19:03:33.299 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.694e-06 2024-09-20T19:03:33.299 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.299 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.299 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:03:33.299 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.588232+0000", 2024-09-20T19:03:33.299 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013927899999999999 2024-09-20T19:03:33.299 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.299 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.299 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:03:33.300 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.588247+0000", 2024-09-20T19:03:33.300 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5670000000000001e-05 2024-09-20T19:03:33.300 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.300 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.300 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:03:33.300 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.588255+0000", 2024-09-20T19:03:33.300 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.0630000000000006e-06 2024-09-20T19:03:33.300 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.300 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.300 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:33.300 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.588320+0000", 2024-09-20T19:03:33.300 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5049999999999996e-05 2024-09-20T19:03:33.300 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.301 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.301 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:03:33.301 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.588440+0000", 2024-09-20T19:03:33.301 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011967 2024-09-20T19:03:33.301 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.301 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.301 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,4", 2024-09-20T19:03:33.301 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.588596+0000", 2024-09-20T19:03:33.301 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015572200000000001 2024-09-20T19:03:33.301 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.301 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.301 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:03:33.301 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.591386+0000", 2024-09-20T19:03:33.302 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.002789962 2024-09-20T19:03:33.302 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.302 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.302 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:03:33.302 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.591731+0000", 2024-09-20T19:03:33.302 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00034524700000000002 2024-09-20T19:03:33.302 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.302 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.302 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:03:33.302 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.724872+0000", 2024-09-20T19:03:33.302 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13314100300000001 2024-09-20T19:03:33.302 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.302 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.302 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:03:33.303 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.724930+0000", 2024-09-20T19:03:33.303 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.7472000000000001e-05 2024-09-20T19:03:33.303 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.303 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.303 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:03:33.303 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.724974+0000", 2024-09-20T19:03:33.303 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.4366000000000002e-05 2024-09-20T19:03:33.303 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:33.303 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:03:33.303 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:33.303 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.303 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.303 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:84067 9.3 9:cfb01319:::benchmark_data_smithi083_124552_object84066:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e268)", 2024-09-20T19:03:33.304 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:38.595983+0000", 2024-09-20T19:03:33.304 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 233.65825689299999, 2024-09-20T19:03:33.304 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.12985870299999999, 2024-09-20T19:03:33.304 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:03:33.304 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:03:33.304 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:03:33.304 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:03:33.304 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:03:33.304 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:03:33.304 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 84067 2024-09-20T19:03:33.304 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.304 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:03:33.304 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.304 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:03:33.305 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.595983+0000", 2024-09-20T19:03:33.305 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:33.305 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.305 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.305 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:03:33.305 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.595983+0000", 2024-09-20T19:03:33.305 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:33.305 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.305 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.305 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:03:33.305 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.595985+0000", 2024-09-20T19:03:33.305 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9800000000000001e-06 2024-09-20T19:03:33.305 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.306 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.306 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:03:33.306 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.596024+0000", 2024-09-20T19:03:33.306 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.8640000000000003e-05 2024-09-20T19:03:33.306 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.306 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.306 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:03:33.306 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.596045+0000", 2024-09-20T19:03:33.306 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0707e-05 2024-09-20T19:03:33.306 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.306 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.306 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:03:33.306 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.596053+0000", 2024-09-20T19:03:33.306 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.3319999999999992e-06 2024-09-20T19:03:33.307 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.307 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.307 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:33.307 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.596111+0000", 2024-09-20T19:03:33.307 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.7803999999999997e-05 2024-09-20T19:03:33.307 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.307 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.307 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:03:33.307 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.596263+0000", 2024-09-20T19:03:33.307 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000151784 2024-09-20T19:03:33.307 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.307 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.307 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,3", 2024-09-20T19:03:33.307 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.596417+0000", 2024-09-20T19:03:33.308 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015399399999999999 2024-09-20T19:03:33.308 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.308 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.308 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:03:33.308 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.599163+0000", 2024-09-20T19:03:33.308 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.002746727 2024-09-20T19:03:33.308 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.308 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.308 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:03:33.308 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.722887+0000", 2024-09-20T19:03:33.308 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.12372408999999999 2024-09-20T19:03:33.308 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.308 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.308 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:03:33.309 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.725618+0000", 2024-09-20T19:03:33.309 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0027303980000000002 2024-09-20T19:03:33.309 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.309 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.309 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:03:33.309 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.725797+0000", 2024-09-20T19:03:33.309 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00017908400000000001 2024-09-20T19:03:33.309 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.309 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.309 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:03:33.309 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.725842+0000", 2024-09-20T19:03:33.309 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.5163000000000001e-05 2024-09-20T19:03:33.309 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:33.310 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:03:33.310 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:33.310 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.310 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.310 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:101846 9.d 9:bb37c834:::benchmark_data_smithi083_124552_object101845:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e275)", 2024-09-20T19:03:33.310 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:57.497943+0000", 2024-09-20T19:03:33.310 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 214.75629704299999, 2024-09-20T19:03:33.310 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.22483731800000001, 2024-09-20T19:03:33.310 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:03:33.310 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:03:33.310 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:03:33.310 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:03:33.310 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:03:33.310 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:03:33.311 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 101846 2024-09-20T19:03:33.311 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.311 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:03:33.311 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.311 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:03:33.311 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.497943+0000", 2024-09-20T19:03:33.311 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:33.311 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.311 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.311 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:03:33.311 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.497943+0000", 2024-09-20T19:03:33.312 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:33.312 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.312 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.312 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:03:33.312 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.497945+0000", 2024-09-20T19:03:33.312 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6029999999999999e-06 2024-09-20T19:03:33.312 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.312 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.312 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:03:33.312 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.497989+0000", 2024-09-20T19:03:33.312 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3755999999999999e-05 2024-09-20T19:03:33.312 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.313 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.313 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:03:33.313 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.498001+0000", 2024-09-20T19:03:33.313 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2615e-05 2024-09-20T19:03:33.313 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.313 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.313 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:03:33.313 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.498010+0000", 2024-09-20T19:03:33.313 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.3599999999999996e-06 2024-09-20T19:03:33.313 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.314 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.314 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:33.314 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.498066+0000", 2024-09-20T19:03:33.314 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6923999999999999e-05 2024-09-20T19:03:33.314 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.314 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.314 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:03:33.314 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.498167+0000", 2024-09-20T19:03:33.314 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000100721 2024-09-20T19:03:33.314 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.315 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.315 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,5", 2024-09-20T19:03:33.315 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.498268+0000", 2024-09-20T19:03:33.315 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000100488 2024-09-20T19:03:33.315 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.315 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.315 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:03:33.315 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.500395+0000", 2024-09-20T19:03:33.315 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0021269259999999999 2024-09-20T19:03:33.315 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.315 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.315 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:03:33.315 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.500711+0000", 2024-09-20T19:03:33.315 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00031641299999999998 2024-09-20T19:03:33.316 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.316 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.316 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:03:33.316 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.722681+0000", 2024-09-20T19:03:33.316 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.22197028999999999 2024-09-20T19:03:33.316 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.316 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.316 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:03:33.316 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.722735+0000", 2024-09-20T19:03:33.316 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3879999999999999e-05 2024-09-20T19:03:33.316 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.316 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.316 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:03:33.316 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.722780+0000", 2024-09-20T19:03:33.317 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.5342000000000001e-05 2024-09-20T19:03:33.317 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:33.317 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:03:33.317 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:33.317 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.317 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.317 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:119689 9.d 9:bd867cd9:::benchmark_data_smithi083_124552_object5542:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e287)", 2024-09-20T19:03:33.317 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:00:16.841487+0000", 2024-09-20T19:03:33.317 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 195.41275280400001, 2024-09-20T19:03:33.317 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13893808099999999, 2024-09-20T19:03:33.317 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:03:33.317 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:03:33.317 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:03:33.318 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:03:33.318 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:03:33.318 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:03:33.318 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 119689 2024-09-20T19:03:33.318 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.318 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:03:33.318 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.318 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:03:33.318 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841487+0000", 2024-09-20T19:03:33.318 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:33.318 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.318 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.318 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:03:33.318 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841487+0000", 2024-09-20T19:03:33.319 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:33.319 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.319 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.319 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:03:33.319 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841489+0000", 2024-09-20T19:03:33.319 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2309999999999999e-06 2024-09-20T19:03:33.319 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.319 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.319 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:03:33.319 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841493+0000", 2024-09-20T19:03:33.319 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.374e-06 2024-09-20T19:03:33.319 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.319 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.319 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:03:33.320 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841502+0000", 2024-09-20T19:03:33.320 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.3649999999999993e-06 2024-09-20T19:03:33.320 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.320 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.320 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:03:33.320 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841509+0000", 2024-09-20T19:03:33.320 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.0999999999999998e-06 2024-09-20T19:03:33.320 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.320 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.320 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:33.320 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841561+0000", 2024-09-20T19:03:33.320 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.1968000000000001e-05 2024-09-20T19:03:33.320 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.320 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.320 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:03:33.321 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841684+0000", 2024-09-20T19:03:33.321 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012302 2024-09-20T19:03:33.321 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.321 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.321 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,5", 2024-09-20T19:03:33.321 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841798+0000", 2024-09-20T19:03:33.321 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011330799999999999 2024-09-20T19:03:33.321 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.321 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.321 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:03:33.321 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.845340+0000", 2024-09-20T19:03:33.321 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0035421129999999999 2024-09-20T19:03:33.321 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.321 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.322 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:03:33.322 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.849468+0000", 2024-09-20T19:03:33.322 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0041283489999999999 2024-09-20T19:03:33.322 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.322 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.322 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:03:33.322 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.980284+0000", 2024-09-20T19:03:33.322 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13081616800000001 2024-09-20T19:03:33.322 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.322 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.322 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:03:33.322 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.980351+0000", 2024-09-20T19:03:33.322 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.6928999999999996e-05 2024-09-20T19:03:33.322 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.323 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.323 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:03:33.323 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.980425+0000", 2024-09-20T19:03:33.323 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.4155999999999999e-05 2024-09-20T19:03:33.323 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:33.323 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:03:33.323 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:33.323 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.323 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.323 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:119690 9.3 9:cd5c9a6c:::benchmark_data_smithi083_124552_object5543:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e287)", 2024-09-20T19:03:33.323 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:00:16.841539+0000", 2024-09-20T19:03:33.323 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 195.412701594, 2024-09-20T19:03:33.323 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.138853324, 2024-09-20T19:03:33.323 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:03:33.324 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:03:33.324 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:03:33.324 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:03:33.324 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:03:33.324 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:03:33.324 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 119690 2024-09-20T19:03:33.324 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.324 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:03:33.324 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.324 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:03:33.324 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841539+0000", 2024-09-20T19:03:33.324 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:33.324 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.324 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.325 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:03:33.325 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841539+0000", 2024-09-20T19:03:33.325 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:33.325 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.325 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.325 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:03:33.325 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841539+0000", 2024-09-20T19:03:33.325 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4499999999999997e-07 2024-09-20T19:03:33.325 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.325 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.325 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:03:33.325 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841542+0000", 2024-09-20T19:03:33.325 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.128e-06 2024-09-20T19:03:33.325 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.326 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.326 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:03:33.326 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841548+0000", 2024-09-20T19:03:33.326 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8220000000000003e-06 2024-09-20T19:03:33.326 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.326 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.326 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:03:33.326 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841553+0000", 2024-09-20T19:03:33.326 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.967e-06 2024-09-20T19:03:33.326 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.326 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.326 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:33.326 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841603+0000", 2024-09-20T19:03:33.326 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.9636999999999999e-05 2024-09-20T19:03:33.326 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.327 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.327 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:03:33.327 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841706+0000", 2024-09-20T19:03:33.327 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000103609 2024-09-20T19:03:33.327 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.327 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.327 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,3", 2024-09-20T19:03:33.327 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841803+0000", 2024-09-20T19:03:33.327 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.6217000000000004e-05 2024-09-20T19:03:33.327 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.327 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.327 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:03:33.327 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.845333+0000", 2024-09-20T19:03:33.328 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0035301540000000002 2024-09-20T19:03:33.328 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.328 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.328 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:03:33.328 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.846476+0000", 2024-09-20T19:03:33.328 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0011436619999999999 2024-09-20T19:03:33.328 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.328 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.328 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:03:33.328 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.980293+0000", 2024-09-20T19:03:33.328 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13381654500000001 2024-09-20T19:03:33.328 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.328 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.328 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:03:33.328 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.980328+0000", 2024-09-20T19:03:33.329 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.5255e-05 2024-09-20T19:03:33.329 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.329 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.329 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:03:33.329 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.980392+0000", 2024-09-20T19:03:33.329 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.3782999999999996e-05 2024-09-20T19:03:33.329 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:33.329 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:03:33.329 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:33.329 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.329 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.329 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:119697 9.d 9:b589807c:::benchmark_data_smithi083_124552_object5550:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e287)", 2024-09-20T19:03:33.329 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:00:16.846849+0000", 2024-09-20T19:03:33.329 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 195.40739163699999, 2024-09-20T19:03:33.330 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13367747499999999, 2024-09-20T19:03:33.330 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:03:33.330 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:03:33.330 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:03:33.330 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:03:33.330 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:03:33.330 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:03:33.330 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 119697 2024-09-20T19:03:33.330 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.330 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:03:33.330 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.330 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:03:33.330 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.846849+0000", 2024-09-20T19:03:33.331 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:33.331 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.331 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.331 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:03:33.331 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.846849+0000", 2024-09-20T19:03:33.331 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:33.331 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.331 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.331 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:03:33.331 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.846850+0000", 2024-09-20T19:03:33.331 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5829999999999999e-06 2024-09-20T19:03:33.331 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.331 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.331 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:03:33.331 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.846856+0000", 2024-09-20T19:03:33.332 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6570000000000002e-06 2024-09-20T19:03:33.332 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.332 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.332 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:03:33.332 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.846867+0000", 2024-09-20T19:03:33.332 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1321999999999999e-05 2024-09-20T19:03:33.332 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.332 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.332 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:03:33.332 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.846875+0000", 2024-09-20T19:03:33.332 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.9270000000000005e-06 2024-09-20T19:03:33.332 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.332 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.332 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:33.333 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.846926+0000", 2024-09-20T19:03:33.333 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0759e-05 2024-09-20T19:03:33.333 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.333 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.333 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:03:33.333 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.847050+0000", 2024-09-20T19:03:33.333 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012434300000000001 2024-09-20T19:03:33.333 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.333 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.333 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,5", 2024-09-20T19:03:33.333 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.847172+0000", 2024-09-20T19:03:33.333 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000121586 2024-09-20T19:03:33.333 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.333 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.334 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:03:33.334 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.849552+0000", 2024-09-20T19:03:33.334 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0023799810000000002 2024-09-20T19:03:33.334 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.334 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.334 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:03:33.334 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.849792+0000", 2024-09-20T19:03:33.334 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000239965 2024-09-20T19:03:33.334 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.334 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.334 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:03:33.334 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.980470+0000", 2024-09-20T19:03:33.334 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13067801600000001 2024-09-20T19:03:33.334 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.335 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.335 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:03:33.335 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.980505+0000", 2024-09-20T19:03:33.335 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.5627000000000003e-05 2024-09-20T19:03:33.335 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.335 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.335 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:03:33.335 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.980526+0000", 2024-09-20T19:03:33.335 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0709000000000001e-05 2024-09-20T19:03:33.335 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:33.335 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:03:33.335 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:33.335 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.335 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.336 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14460.0:2702 5.9 5:93e5b521:::notify.7:head [watch reconnect cookie 94056120174720 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e292)", 2024-09-20T19:03:33.336 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:00:35.178267+0000", 2024-09-20T19:03:33.336 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 177.075973027, 2024-09-20T19:03:33.336 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.011898462, 2024-09-20T19:03:33.336 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:03:33.336 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:03:33.336 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-09-20T19:03:33.336 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:03:33.336 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14460", 2024-09-20T19:03:33.336 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1555339426", 2024-09-20T19:03:33.336 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 2702 2024-09-20T19:03:33.336 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.336 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:03:33.336 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.337 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:03:33.337 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.178267+0000", 2024-09-20T19:03:33.337 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:33.337 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.337 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.337 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:03:33.337 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.178267+0000", 2024-09-20T19:03:33.337 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:33.337 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.337 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.337 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:03:33.337 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.178269+0000", 2024-09-20T19:03:33.337 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.703e-06 2024-09-20T19:03:33.337 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.338 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.338 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:03:33.338 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.178274+0000", 2024-09-20T19:03:33.338 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3909999999999997e-06 2024-09-20T19:03:33.338 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.338 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.338 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:03:33.338 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.178292+0000", 2024-09-20T19:03:33.338 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7601e-05 2024-09-20T19:03:33.338 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.338 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.338 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:03:33.338 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.178301+0000", 2024-09-20T19:03:33.338 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.6950000000000006e-06 2024-09-20T19:03:33.339 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.339 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.339 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:33.339 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.178364+0000", 2024-09-20T19:03:33.339 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.3304000000000002e-05 2024-09-20T19:03:33.339 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.339 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.339 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op must wait for map", 2024-09-20T19:03:33.339 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.178370+0000", 2024-09-20T19:03:33.339 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2979999999999997e-06 2024-09-20T19:03:33.339 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.339 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.339 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:33.339 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.191158+0000", 2024-09-20T19:03:33.339 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.012787822000000001 2024-09-20T19:03:33.340 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.340 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.340 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:03:33.340 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.191166+0000", 2024-09-20T19:03:33.340 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.9959999999999993e-06 2024-09-20T19:03:33.340 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.340 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.340 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:33.340 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.188716+0000", 2024-09-20T19:03:33.340 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.997550198 2024-09-20T19:03:33.340 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.340 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.340 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:03:33.340 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.188848+0000", 2024-09-20T19:03:33.341 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013202400000000001 2024-09-20T19:03:33.341 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.341 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.341 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:03:33.341 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.190166+0000", 2024-09-20T19:03:33.341 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0013174300000000001 2024-09-20T19:03:33.341 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:33.341 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:03:33.341 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:33.341 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.341 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.341 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:142953 9.e 9:785c93fe:::benchmark_data_smithi083_124552_object28806:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e292)", 2024-09-20T19:03:33.341 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:00:35.227055+0000", 2024-09-20T19:03:33.341 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 177.02718534600001, 2024-09-20T19:03:33.342 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.985725662, 2024-09-20T19:03:33.342 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:03:33.342 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:03:33.342 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:03:33.342 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:03:33.342 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:03:33.342 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:03:33.342 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 142953 2024-09-20T19:03:33.342 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.342 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:03:33.342 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.342 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:03:33.342 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.227055+0000", 2024-09-20T19:03:33.343 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:33.343 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.343 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.343 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:03:33.343 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.227055+0000", 2024-09-20T19:03:33.343 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:33.343 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.343 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.343 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:03:33.343 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.227056+0000", 2024-09-20T19:03:33.343 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1170000000000001e-06 2024-09-20T19:03:33.343 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.343 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.343 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:03:33.344 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.227060+0000", 2024-09-20T19:03:33.344 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.1320000000000004e-06 2024-09-20T19:03:33.344 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.344 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.344 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:03:33.344 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.227070+0000", 2024-09-20T19:03:33.344 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.8179999999999999e-06 2024-09-20T19:03:33.344 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.344 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.344 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:03:33.344 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.227075+0000", 2024-09-20T19:03:33.344 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5289999999999999e-06 2024-09-20T19:03:33.344 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.344 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.344 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:33.345 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.227131+0000", 2024-09-20T19:03:33.345 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5482999999999997e-05 2024-09-20T19:03:33.345 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.345 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.345 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:03:33.345 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.227140+0000", 2024-09-20T19:03:33.345 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.4159999999999993e-06 2024-09-20T19:03:33.345 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.345 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.345 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:33.345 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.205610+0000", 2024-09-20T19:03:33.345 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.97846971500000002 2024-09-20T19:03:33.345 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.345 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.346 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:03:33.346 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.205744+0000", 2024-09-20T19:03:33.346 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013422099999999999 2024-09-20T19:03:33.346 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.346 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.346 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1", 2024-09-20T19:03:33.346 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.205878+0000", 2024-09-20T19:03:33.346 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000134108 2024-09-20T19:03:33.346 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.346 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.346 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:03:33.346 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.212569+0000", 2024-09-20T19:03:33.346 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0066901629999999998 2024-09-20T19:03:33.346 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.347 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.347 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:03:33.347 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.212643+0000", 2024-09-20T19:03:33.347 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.4120000000000002e-05 2024-09-20T19:03:33.347 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.347 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.347 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:03:33.347 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.212692+0000", 2024-09-20T19:03:33.347 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.922e-05 2024-09-20T19:03:33.347 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.347 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.347 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:03:33.347 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.212781+0000", 2024-09-20T19:03:33.347 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.8620000000000002e-05 2024-09-20T19:03:33.348 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:33.348 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:03:33.348 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:33.348 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.348 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.348 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:142961 9.e 9:70c9c31e:::benchmark_data_smithi083_124552_object28814:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e292)", 2024-09-20T19:03:33.348 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:00:35.233023+0000", 2024-09-20T19:03:33.348 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 177.02121691100001, 2024-09-20T19:03:33.348 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.98462836899999995, 2024-09-20T19:03:33.348 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:03:33.348 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:03:33.348 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:03:33.348 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:03:33.349 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:03:33.349 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:03:33.349 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 142961 2024-09-20T19:03:33.349 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.349 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:03:33.349 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.349 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:03:33.349 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.233023+0000", 2024-09-20T19:03:33.349 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:33.349 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.349 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.349 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:03:33.349 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.233023+0000", 2024-09-20T19:03:33.349 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:33.350 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.350 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.350 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:03:33.350 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.233025+0000", 2024-09-20T19:03:33.350 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.252e-06 2024-09-20T19:03:33.350 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.350 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.350 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:03:33.350 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.233029+0000", 2024-09-20T19:03:33.350 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.442e-06 2024-09-20T19:03:33.350 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.350 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.350 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:03:33.350 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.233048+0000", 2024-09-20T19:03:33.351 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9096000000000001e-05 2024-09-20T19:03:33.351 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.351 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.351 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:03:33.351 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.233055+0000", 2024-09-20T19:03:33.351 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.0729999999999999e-06 2024-09-20T19:03:33.351 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.351 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.351 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:33.351 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.233109+0000", 2024-09-20T19:03:33.351 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4049999999999999e-05 2024-09-20T19:03:33.351 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.351 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.351 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:03:33.352 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.233117+0000", 2024-09-20T19:03:33.352 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.379e-06 2024-09-20T19:03:33.352 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.352 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.352 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:33.352 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.208136+0000", 2024-09-20T19:03:33.352 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.97501899599999997 2024-09-20T19:03:33.352 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.352 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.352 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:03:33.352 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.208257+0000", 2024-09-20T19:03:33.352 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012119 2024-09-20T19:03:33.352 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.352 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.352 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1", 2024-09-20T19:03:33.353 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.208383+0000", 2024-09-20T19:03:33.353 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000126484 2024-09-20T19:03:33.353 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.353 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.353 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:03:33.353 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.212629+0000", 2024-09-20T19:03:33.353 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0042453329999999996 2024-09-20T19:03:33.353 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.353 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.353 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:03:33.353 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.217490+0000", 2024-09-20T19:03:33.353 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0048616470000000002 2024-09-20T19:03:33.353 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.353 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.354 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:03:33.354 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.217551+0000", 2024-09-20T19:03:33.354 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0797000000000001e-05 2024-09-20T19:03:33.354 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.354 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.354 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:03:33.354 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.217652+0000", 2024-09-20T19:03:33.354 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00010063 2024-09-20T19:03:33.354 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:33.354 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:03:33.354 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:33.354 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.354 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.355 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:142969 9.e 9:74a6f6aa:::benchmark_data_smithi083_124552_object28822:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e292)", 2024-09-20T19:03:33.355 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:00:35.238908+0000", 2024-09-20T19:03:33.355 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 177.015332288, 2024-09-20T19:03:33.355 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.98019590400000001, 2024-09-20T19:03:33.355 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:03:33.355 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:03:33.355 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:03:33.355 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:03:33.355 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:03:33.355 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:03:33.355 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 142969 2024-09-20T19:03:33.355 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.355 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:03:33.355 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.356 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:03:33.356 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.238908+0000", 2024-09-20T19:03:33.356 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:33.356 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.356 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.356 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:03:33.356 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.238908+0000", 2024-09-20T19:03:33.356 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:33.356 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.356 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.356 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:03:33.356 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.238909+0000", 2024-09-20T19:03:33.356 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.7800000000000001e-07 2024-09-20T19:03:33.356 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.357 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.357 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:03:33.357 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.238912+0000", 2024-09-20T19:03:33.357 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.3919999999999999e-06 2024-09-20T19:03:33.357 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.357 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.357 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:03:33.357 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.238919+0000", 2024-09-20T19:03:33.357 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.0679999999999999e-06 2024-09-20T19:03:33.357 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.357 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.357 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:03:33.357 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.238925+0000", 2024-09-20T19:03:33.357 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.1369999999999999e-06 2024-09-20T19:03:33.358 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.358 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.358 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:33.358 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.238981+0000", 2024-09-20T19:03:33.358 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6067999999999999e-05 2024-09-20T19:03:33.358 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.358 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.358 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:03:33.358 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.238990+0000", 2024-09-20T19:03:33.358 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.2600000000000005e-06 2024-09-20T19:03:33.358 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.358 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.358 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:33.359 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.210313+0000", 2024-09-20T19:03:33.359 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.97132380500000004 2024-09-20T19:03:33.359 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.359 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.359 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:03:33.359 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.210428+0000", 2024-09-20T19:03:33.359 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000114369 2024-09-20T19:03:33.359 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.359 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.359 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1", 2024-09-20T19:03:33.359 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.210549+0000", 2024-09-20T19:03:33.359 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000121558 2024-09-20T19:03:33.359 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.360 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.360 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:03:33.360 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.217664+0000", 2024-09-20T19:03:33.360 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0071143090000000001 2024-09-20T19:03:33.360 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.360 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.360 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:03:33.360 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.218937+0000", 2024-09-20T19:03:33.360 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0012730599999999999 2024-09-20T19:03:33.360 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.360 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.360 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:03:33.360 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.218991+0000", 2024-09-20T19:03:33.360 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4722999999999998e-05 2024-09-20T19:03:33.361 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.361 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.361 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:03:33.361 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.219104+0000", 2024-09-20T19:03:33.361 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000112477 2024-09-20T19:03:33.361 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:33.361 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:03:33.361 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:33.361 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.361 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.361 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:142981 9.e 9:7a27425f:::benchmark_data_smithi083_124552_object28834:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e292)", 2024-09-20T19:03:33.361 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:00:35.254008+0000", 2024-09-20T19:03:33.361 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 177.000231797, 2024-09-20T19:03:33.361 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.966262548, 2024-09-20T19:03:33.362 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:03:33.362 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:03:33.362 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:03:33.362 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:03:33.362 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:03:33.362 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:03:33.362 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 142981 2024-09-20T19:03:33.362 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.362 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:03:33.362 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.362 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:03:33.362 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.254008+0000", 2024-09-20T19:03:33.362 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:33.362 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.362 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.363 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:03:33.363 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.254008+0000", 2024-09-20T19:03:33.363 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:33.363 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.363 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.363 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:03:33.363 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.254010+0000", 2024-09-20T19:03:33.363 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.443e-06 2024-09-20T19:03:33.363 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.363 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.363 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:03:33.363 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.254014+0000", 2024-09-20T19:03:33.363 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9260000000000002e-06 2024-09-20T19:03:33.363 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.364 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.364 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:03:33.364 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.254024+0000", 2024-09-20T19:03:33.364 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.9839999999999996e-06 2024-09-20T19:03:33.364 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.364 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.364 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:03:33.364 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.254029+0000", 2024-09-20T19:03:33.364 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6869999999999999e-06 2024-09-20T19:03:33.364 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.364 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.364 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:33.364 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.254089+0000", 2024-09-20T19:03:33.364 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9435000000000003e-05 2024-09-20T19:03:33.365 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.365 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.365 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:03:33.365 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.254094+0000", 2024-09-20T19:03:33.365 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4399999999999996e-06 2024-09-20T19:03:33.365 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.365 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.365 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:33.365 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.212819+0000", 2024-09-20T19:03:33.365 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.95872477300000003 2024-09-20T19:03:33.365 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.365 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.365 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:03:33.365 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.212953+0000", 2024-09-20T19:03:33.365 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013434699999999999 2024-09-20T19:03:33.366 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.366 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.366 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1", 2024-09-20T19:03:33.366 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.213056+0000", 2024-09-20T19:03:33.366 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000102675 2024-09-20T19:03:33.366 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.366 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.366 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:03:33.366 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.217670+0000", 2024-09-20T19:03:33.366 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0046140879999999997 2024-09-20T19:03:33.366 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.367 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.367 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:03:33.367 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.219977+0000", 2024-09-20T19:03:33.367 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0023067230000000001 2024-09-20T19:03:33.367 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.367 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.367 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:03:33.367 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.220160+0000", 2024-09-20T19:03:33.367 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00018349800000000001 2024-09-20T19:03:33.367 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.367 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.367 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:03:33.367 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.220271+0000", 2024-09-20T19:03:33.367 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000110529 2024-09-20T19:03:33.367 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:33.368 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:03:33.368 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:33.368 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.368 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.368 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:142991 9.e 9:75f432de:::benchmark_data_smithi083_124552_object28844:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e292)", 2024-09-20T19:03:33.368 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:00:35.271404+0000", 2024-09-20T19:03:33.368 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 176.98283643600001, 2024-09-20T19:03:33.368 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.94906818900000001, 2024-09-20T19:03:33.368 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:03:33.368 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:03:33.368 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:03:33.368 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:03:33.368 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:03:33.369 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:03:33.369 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 142991 2024-09-20T19:03:33.369 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.369 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:03:33.369 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.369 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:03:33.369 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.271404+0000", 2024-09-20T19:03:33.369 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:33.369 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.369 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.369 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:03:33.369 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.271404+0000", 2024-09-20T19:03:33.369 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:33.369 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.370 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.370 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:03:33.370 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.271405+0000", 2024-09-20T19:03:33.370 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.569999999999999e-07 2024-09-20T19:03:33.370 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.370 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.370 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:03:33.370 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.271408+0000", 2024-09-20T19:03:33.370 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.647e-06 2024-09-20T19:03:33.370 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.370 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.370 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:03:33.370 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.271418+0000", 2024-09-20T19:03:33.371 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.6360000000000006e-06 2024-09-20T19:03:33.371 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.371 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.371 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:03:33.371 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.271424+0000", 2024-09-20T19:03:33.371 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6080000000000003e-06 2024-09-20T19:03:33.371 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.371 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.371 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:33.371 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.271468+0000", 2024-09-20T19:03:33.371 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3958999999999997e-05 2024-09-20T19:03:33.371 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.371 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.371 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:03:33.371 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.271473+0000", 2024-09-20T19:03:33.372 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0710000000000002e-06 2024-09-20T19:03:33.372 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.372 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.372 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:33.372 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.214908+0000", 2024-09-20T19:03:33.372 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.94343566999999995 2024-09-20T19:03:33.372 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.372 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.372 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:03:33.372 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.215018+0000", 2024-09-20T19:03:33.372 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000109641 2024-09-20T19:03:33.372 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.372 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.373 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1", 2024-09-20T19:03:33.373 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.215110+0000", 2024-09-20T19:03:33.373 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.2324000000000006e-05 2024-09-20T19:03:33.373 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.373 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.373 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:03:33.373 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.220290+0000", 2024-09-20T19:03:33.373 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0051801019999999998 2024-09-20T19:03:33.373 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.373 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.373 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:03:33.373 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.220324+0000", 2024-09-20T19:03:33.373 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4020000000000003e-05 2024-09-20T19:03:33.373 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.374 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.374 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:03:33.374 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.220389+0000", 2024-09-20T19:03:33.374 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5048999999999993e-05 2024-09-20T19:03:33.374 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.374 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.374 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:03:33.374 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.220472+0000", 2024-09-20T19:03:33.374 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.2504999999999997e-05 2024-09-20T19:03:33.374 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:33.374 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:03:33.374 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:33.374 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.375 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.375 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:184349 9.3 9:c8c5782c:::benchmark_data_smithi083_124552_object70202:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e299)", 2024-09-20T19:03:33.375 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:01:08.628496+0000", 2024-09-20T19:03:33.375 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 143.62574450899999, 2024-09-20T19:03:33.375 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.129854478, 2024-09-20T19:03:33.375 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:03:33.375 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:03:33.375 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:03:33.375 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:03:33.375 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:03:33.375 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:03:33.375 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 184349 2024-09-20T19:03:33.375 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.375 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:03:33.376 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.376 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:03:33.376 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:08.628496+0000", 2024-09-20T19:03:33.376 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:33.376 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.376 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.376 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:03:33.376 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:08.628496+0000", 2024-09-20T19:03:33.376 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:33.376 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.376 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.376 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:03:33.376 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:08.628497+0000", 2024-09-20T19:03:33.376 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2669999999999999e-06 2024-09-20T19:03:33.377 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.377 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.377 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:03:33.377 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:08.628502+0000", 2024-09-20T19:03:33.377 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7219999999999999e-06 2024-09-20T19:03:33.377 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.377 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.377 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:03:33.377 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:08.628510+0000", 2024-09-20T19:03:33.377 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.5620000000000004e-06 2024-09-20T19:03:33.377 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.377 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.377 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:03:33.377 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:08.628518+0000", 2024-09-20T19:03:33.378 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.3520000000000001e-06 2024-09-20T19:03:33.378 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.378 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.378 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:33.378 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:08.628610+0000", 2024-09-20T19:03:33.378 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.2121999999999997e-05 2024-09-20T19:03:33.378 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.378 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.378 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:03:33.378 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:08.628812+0000", 2024-09-20T19:03:33.378 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00020209899999999999 2024-09-20T19:03:33.378 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.378 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.378 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,3", 2024-09-20T19:03:33.379 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:08.628951+0000", 2024-09-20T19:03:33.379 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000138706 2024-09-20T19:03:33.379 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.379 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.379 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:03:33.379 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:08.632592+0000", 2024-09-20T19:03:33.379 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0036414559999999999 2024-09-20T19:03:33.379 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.379 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.379 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:03:33.379 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:08.632640+0000", 2024-09-20T19:03:33.379 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7613e-05 2024-09-20T19:03:33.379 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.379 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.380 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:03:33.380 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:08.758209+0000", 2024-09-20T19:03:33.380 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.12556927600000001 2024-09-20T19:03:33.380 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.380 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.380 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:03:33.380 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:08.758303+0000", 2024-09-20T19:03:33.380 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.4536000000000004e-05 2024-09-20T19:03:33.380 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:33.380 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:33.380 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:03:33.380 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:01:08.758350+0000", 2024-09-20T19:03:33.380 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.6767000000000001e-05 2024-09-20T19:03:33.380 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:33.381 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:03:33.381 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:33.381 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:33.381 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:03:33.381 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T19:03:33.381 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:33.421 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:33.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:33.422 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:33.458 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T19:03:33.523 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:33.567 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:33.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:33.568 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:33.669 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:33.703 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:33.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:33.704 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:33.805 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:33.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:33.846 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:33.847 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:33.881 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 74 16 104556 104540 88.2737 49.5 0.0499698 0.0113231 2024-09-20T19:03:33.947 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:33.984 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:33.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:33.985 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:34.086 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:03:34.103 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:03:34.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:34.125 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:34.125 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:03:34.226 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:34.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:34.259 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:34.260 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:34.360 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:34.393 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:34.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:34.394 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:34.494 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:34.533 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:34.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:34.534 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:34.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:34 smithi083 ceph-mon[29421]: pgmap v685: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 6.3 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1.3 KiB/s rd, 56 MiB/s wr, 889 op/s; 43528/308784 objects misplaced (14.097%); 30 MiB/s, 479 objects/s recovering 2024-09-20T19:03:34.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:34 smithi083 ceph-mon[36260]: pgmap v685: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 6.3 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1.3 KiB/s rd, 56 MiB/s wr, 889 op/s; 43528/308784 objects misplaced (14.097%); 30 MiB/s, 479 objects/s recovering 2024-09-20T19:03:34.634 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:34.664 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:34.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:34.665 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:34.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:34 smithi142 ceph-mon[32508]: pgmap v685: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 6.3 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1.3 KiB/s rd, 56 MiB/s wr, 889 op/s; 43528/308784 objects misplaced (14.097%); 30 MiB/s, 479 objects/s recovering 2024-09-20T19:03:34.765 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:34.803 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:34.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:34.804 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:34.881 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 75 16 105326 105310 87.7384 48.125 0.0464559 0.0113909 2024-09-20T19:03:34.905 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:34.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:34.946 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:34.946 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:35.047 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:35.091 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:35.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:35.092 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:35.193 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:35.229 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:35.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:35.230 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:35.331 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:35.332 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-20T19:03:35.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:35.371 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:35.371 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:35.472 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:03:35.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:35.505 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:35.506 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:03:35.508 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T19:03:35.606 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:35.641 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:35.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:35.643 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:35.696 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:35 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/4085274732' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:03:35.696 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:35 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:03:35.696 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:35 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/4085274732' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:03:35.696 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:35 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:03:35.744 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:35.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:35 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/4085274732' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:03:35.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:35 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:03:35.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:35.785 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:35.785 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:35.881 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 76 16 106103 106087 87.2229 48.5625 0.083351 0.0114567 2024-09-20T19:03:35.887 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:35.926 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:35.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:35.927 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:36.028 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:36.049 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:36.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:36.051 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:36.151 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:36.188 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:36.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:36.189 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:36.243 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:03:36.290 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:36.338 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:36.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:36.339 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:36.376 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:36 smithi083 ceph-mon[29421]: pgmap v686: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 6.3 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 639 B/s rd, 30 MiB/s wr, 479 op/s; 43528/308784 objects misplaced (14.097%); 14 MiB/s, 225 objects/s recovering 2024-09-20T19:03:36.376 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:36 smithi083 ceph-mon[29421]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T19:03:36.376 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:36 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:03:36.377 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:36 smithi083 ceph-mon[29421]: osdmap e358: 8 total, 8 up, 7 in 2024-09-20T19:03:36.377 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:36 smithi083 ceph-mon[36260]: pgmap v686: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 6.3 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 639 B/s rd, 30 MiB/s wr, 479 op/s; 43528/308784 objects misplaced (14.097%); 14 MiB/s, 225 objects/s recovering 2024-09-20T19:03:36.377 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:36 smithi083 ceph-mon[36260]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T19:03:36.377 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:36 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:03:36.377 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:36 smithi083 ceph-mon[36260]: osdmap e358: 8 total, 8 up, 7 in 2024-09-20T19:03:36.440 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:03:36.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:36.461 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:36.462 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:03:36.562 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:36.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:36.600 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:36.600 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:36.700 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:36.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:36 smithi142 ceph-mon[32508]: pgmap v686: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 6.3 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 639 B/s rd, 30 MiB/s wr, 479 op/s; 43528/308784 objects misplaced (14.097%); 14 MiB/s, 225 objects/s recovering 2024-09-20T19:03:36.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:36 smithi142 ceph-mon[32508]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T19:03:36.764 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:36 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:03:36.764 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:36 smithi142 ceph-mon[32508]: osdmap e358: 8 total, 8 up, 7 in 2024-09-20T19:03:36.791 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:36.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:36.792 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:36.868 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:03:36.868 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:03:36.881 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 77 16 106826 106810 86.677 45.1875 0.0171399 0.0115304 2024-09-20T19:03:36.893 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:36.895 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:03:36.896 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:03:36.916 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:36.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:36.917 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:36.917 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:03:36.918 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:03:36.937 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:03:36.937 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:03:36.976 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:03:36.976 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:03:37.009 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:03:37.010 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:03:37.018 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:37.045 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:03:37.045 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:03:37.058 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:37.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:37.059 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:37.105 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:03:37.105 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:03:37.160 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:37.197 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:37.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:37.198 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:37.298 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:37.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:37.338 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:37.338 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:37.439 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:37.472 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:37.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:37.474 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:37.532 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:03:37 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:03:37.533 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:37 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1702391672' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:03:37.574 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:37.613 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:37.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:37.614 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:37.715 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:37.750 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:37.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:37.751 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:37.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:37 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1702391672' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:03:37.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:37 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1702391672' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:03:37.853 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:37.881 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 78 16 107565 107549 86.1579 46.1875 0.0558751 0.0116011 2024-09-20T19:03:37.892 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:37.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:37.893 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:37.994 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:03:38.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:38.030 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:38.030 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:03:38.131 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:38.136 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T19:03:38.164 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:38.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:38.165 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:38.266 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:03:38.307 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:38.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:38.308 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:03:38.408 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:38.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:38.453 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:38.453 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:38.551 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:03:38.554 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:38.599 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:38.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:38.601 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:38.693 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:38 smithi083 ceph-mon[29421]: pgmap v688: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 6.5 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1.5 KiB/s rd, 62 MiB/s wr, 993 op/s; 42418/318747 objects misplaced (13.308%); 33 MiB/s, 534 objects/s recovering 2024-09-20T19:03:38.693 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:38 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:03:38.693 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:38 smithi083 ceph-mon[29421]: 10.3 scrub starts 2024-09-20T19:03:38.693 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:38 smithi083 ceph-mon[36260]: pgmap v688: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 6.5 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1.5 KiB/s rd, 62 MiB/s wr, 993 op/s; 42418/318747 objects misplaced (13.308%); 33 MiB/s, 534 objects/s recovering 2024-09-20T19:03:38.693 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:38 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:03:38.693 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:38 smithi083 ceph-mon[36260]: 10.3 scrub starts 2024-09-20T19:03:38.702 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:38.740 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:38.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:38.741 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:38.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:38 smithi142 ceph-mon[32508]: pgmap v688: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 6.5 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1.5 KiB/s rd, 62 MiB/s wr, 993 op/s; 42418/318747 objects misplaced (13.308%); 33 MiB/s, 534 objects/s recovering 2024-09-20T19:03:38.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:38 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:03:38.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:38 smithi142 ceph-mon[32508]: 10.3 scrub starts 2024-09-20T19:03:38.842 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:03:38.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:38.880 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:38.880 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:03:38.881 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 79 16 108341 108325 85.6813 48.5 0.00917664 0.0116659 2024-09-20T19:03:38.981 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:38.983 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 7, 6] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 5, 6, 7, 4] 2024-09-20T19:03:38.983 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T19:03:38.983 INFO:tasks.thrashosds.thrasher:fixing pg num pool .mgr 2024-09-20T19:03:38.984 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T19:03:39.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:39.013 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:39.014 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:39.114 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-20T19:03:39.114 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:39.175 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:39.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:39.176 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:39.277 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:39.325 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:39.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:39.326 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:39.426 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:39.447 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:03:39.459 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:39.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:39.460 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:39.576 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:39.577 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:03:39.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:39.615 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:39.615 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:39.716 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:39.759 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:39.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:39.760 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:39.861 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:39.881 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:2024-09-20T19:03:39.876503+0000 min lat: 0.00125834 max lat: 2.39565 avg lat: 0.0117342 2024-09-20T19:03:39.881 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-20T19:03:39.882 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 80 16 109067 109051 85.1774 45.375 0.0571262 0.0117342 2024-09-20T19:03:39.892 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:39.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:39.893 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:39.993 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:39.999 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T19:03:40.033 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:40.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:40.044 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:40.145 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:40.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:40.189 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:40.189 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:40.290 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:40.329 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:40.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:40.330 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:40.357 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-20T19:03:40.366 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T19:03:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-20T19:03:40.431 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:40.470 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:40.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:40.471 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:40.572 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:40.609 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:40.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:40.610 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:40.710 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:40.727 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:40 smithi083 ceph-mon[29421]: 10.3 scrub ok 2024-09-20T19:03:40.728 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:40 smithi083 ceph-mon[29421]: pgmap v689: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 6.5 GiB data, 20 GiB used, 605 GiB / 626 GiB avail; 1.2 KiB/s rd, 57 MiB/s wr, 906 op/s; 42657/320235 objects misplaced (13.321%); 28 MiB/s, 454 objects/s recovering 2024-09-20T19:03:40.728 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:40 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1315618327' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:03:40.728 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:40 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:03:40.728 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:40 smithi083 ceph-mon[36260]: 10.3 scrub ok 2024-09-20T19:03:40.728 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:40 smithi083 ceph-mon[36260]: pgmap v689: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 6.5 GiB data, 20 GiB used, 605 GiB / 626 GiB avail; 1.2 KiB/s rd, 57 MiB/s wr, 906 op/s; 42657/320235 objects misplaced (13.321%); 28 MiB/s, 454 objects/s recovering 2024-09-20T19:03:40.728 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:40 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1315618327' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:03:40.728 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:40 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:03:40.749 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:40.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:40.750 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:40.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:40 smithi142 ceph-mon[32508]: 10.3 scrub ok 2024-09-20T19:03:40.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:40 smithi142 ceph-mon[32508]: pgmap v689: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 6.5 GiB data, 20 GiB used, 605 GiB / 626 GiB avail; 1.2 KiB/s rd, 57 MiB/s wr, 906 op/s; 42657/320235 objects misplaced (13.321%); 28 MiB/s, 454 objects/s recovering 2024-09-20T19:03:40.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:40 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1315618327' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:03:40.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:40 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:03:40.851 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:03:40.882 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 81 16 109768 109752 84.6668 43.8125 0.0220113 0.0118053 2024-09-20T19:03:40.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:40.886 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:40.887 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:03:40.990 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:41.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:41.028 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:41.028 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:41.043 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:03:40 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:03:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:03:41.129 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:41.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:41.163 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:41.163 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:41.234 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T19:03:41.235 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:03:41.264 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:41.302 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:41.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:41.304 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:41.405 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:03:41.444 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:41.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:41.445 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:03:41.478 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:03:41.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:41 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:03:41.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:41 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:03:41.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:41 smithi142 ceph-mon[32508]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:03:41.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:41 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T19:03:41.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:41 smithi142 ceph-mon[32508]: from='client.16056 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:03:41.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:41 smithi142 ceph-mon[32508]: osdmap e359: 8 total, 8 up, 7 in 2024-09-20T19:03:41.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:41 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:03:41.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:41 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:03:41.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:41 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:03:41.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:41 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:03:41.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:41 smithi083 ceph-mon[36260]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:03:41.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:41 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T19:03:41.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:41 smithi083 ceph-mon[36260]: from='client.16056 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:03:41.544 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:41 smithi083 ceph-mon[36260]: osdmap e359: 8 total, 8 up, 7 in 2024-09-20T19:03:41.544 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:41 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:03:41.544 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:41 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:03:41.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:41 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:03:41.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:41 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:03:41.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:41 smithi083 ceph-mon[29421]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:03:41.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:41 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T19:03:41.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:41 smithi083 ceph-mon[29421]: from='client.16056 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:03:41.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:41 smithi083 ceph-mon[29421]: osdmap e359: 8 total, 8 up, 7 in 2024-09-20T19:03:41.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:41 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:03:41.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:41 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:03:41.546 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:41.574 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:41.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:41.575 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:41.676 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:41.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:41.710 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:41.711 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:41.812 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:41.855 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:41.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:41.856 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:41.881 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 82 16 110595 110579 84.2646 51.6875 0.00338208 0.0118597 2024-09-20T19:03:41.956 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:41.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:41.996 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:41.997 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:42.058 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":690,"stamp":"2024-09-20T19:03:40.342287+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6983195163,"num_objects":106745,"num_object_clones":0,"num_object_copies":320235,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":42657,"num_objects_unfound":0,"num_objects_dirty":106745,"num_whiteouts":0,"num_read":2082,"num_read_kb":1939,"num_write":107381,"num_write_kb":6820617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7759,"num_bytes_recovered":493559860,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79305,"ondisk_log_size":79305,"up":444,"acting":441,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":21479912,"kb_used_data":17733932,"kb_used_omap":10,"kb_used_meta":3745845,"kb_avail":634736152,"statfs":{"total":671965249536,"available":649969819648,"internally_reserved":0,"allocated":18159546368,"data_stored":18156632995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11157,"internal_metadata":3835745387},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,2,10,3,2,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":348585984,"num_objects":5319,"num_object_clones":0,"num_object_copies":15957,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-590,"num_objects_unfound":0,"num_objects_dirty":5319,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5319,"num_write_kb":340416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1949,"num_bytes_recovered":127729664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.038915"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":646,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.326355+0000","last_change":"2024-09-20T19:01:31.817967+0000","last_active":"2024-09-20T19:03:35.326355+0000","last_peered":"2024-09-20T19:03:35.326355+0000","last_clean":"2024-09-20T19:03:35.326355+0000","last_became_active":"2024-09-20T18:49:45.943201+0000","last_became_peered":"2024-09-20T18:49:45.943201+0000","last_unstale":"2024-09-20T19:03:35.326355+0000","last_undegraded":"2024-09-20T19:03:35.326355+0000","last_fullsized":"2024-09-20T19:03:35.326355+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:31.817909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:31.817909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:53.060041+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"273'15","reported_seq":603,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.340754+0000","last_change":"2024-09-20T19:03:14.973129+0000","last_active":"2024-09-20T19:03:35.340754+0000","last_peered":"2024-09-20T19:03:35.340754+0000","last_clean":"2024-09-20T19:03:35.340754+0000","last_became_active":"2024-09-20T19:03:14.945368+0000","last_became_peered":"2024-09-20T19:03:14.945368+0000","last_unstale":"2024-09-20T19:03:35.340754+0000","last_undegraded":"2024-09-20T19:03:35.340754+0000","last_fullsized":"2024-09-20T19:03:35.340754+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:03:10.214154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:10.214154+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:27.580882+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,3],"acting":[6,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":636,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.331032+0000","last_change":"2024-09-20T18:57:49.016078+0000","last_active":"2024-09-20T19:03:35.331032+0000","last_peered":"2024-09-20T19:03:35.331032+0000","last_clean":"2024-09-20T19:03:35.331032+0000","last_became_active":"2024-09-20T18:49:49.959514+0000","last_became_peered":"2024-09-20T18:49:49.959514+0000","last_unstale":"2024-09-20T19:03:35.331032+0000","last_undegraded":"2024-09-20T19:03:35.331032+0000","last_fullsized":"2024-09-20T19:03:35.331032+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:49.016006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:49.016006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":496,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.331294+0000","last_change":"2024-09-20T19:03:14.959169+0000","last_active":"2024-09-20T19:03:35.331294+0000","last_peered":"2024-09-20T19:03:35.331294+0000","last_clean":"2024-09-20T19:03:35.331294+0000","last_became_active":"2024-09-20T19:03:14.958700+0000","last_became_peered":"2024-09-20T19:03:14.958700+0000","last_unstale":"2024-09-20T19:03:35.331294+0000","last_undegraded":"2024-09-20T19:03:35.331294+0000","last_fullsized":"2024-09-20T19:03:35.331294+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:02.084375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:02.084375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":646,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.335116+0000","last_change":"2024-09-20T19:02:02.441633+0000","last_active":"2024-09-20T19:03:35.335116+0000","last_peered":"2024-09-20T19:03:35.335116+0000","last_clean":"2024-09-20T19:03:35.335116+0000","last_became_active":"2024-09-20T18:59:05.081081+0000","last_became_peered":"2024-09-20T18:59:05.081081+0000","last_unstale":"2024-09-20T19:03:35.335116+0000","last_undegraded":"2024-09-20T19:03:35.335116+0000","last_fullsized":"2024-09-20T19:03:35.335116+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:02.441577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:02.441577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":651,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.334938+0000","last_change":"2024-09-20T18:59:05.073093+0000","last_active":"2024-09-20T19:03:35.334938+0000","last_peered":"2024-09-20T19:03:35.334938+0000","last_clean":"2024-09-20T19:03:35.334938+0000","last_became_active":"2024-09-20T18:59:05.072675+0000","last_became_peered":"2024-09-20T18:59:05.072675+0000","last_unstale":"2024-09-20T19:03:35.334938+0000","last_undegraded":"2024-09-20T19:03:35.334938+0000","last_fullsized":"2024-09-20T19:03:35.334938+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:03.055929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:59:03.055929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:45.957147+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"274'27","reported_seq":683,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.334775+0000","last_change":"2024-09-20T19:02:40.899215+0000","last_active":"2024-09-20T19:03:35.334775+0000","last_peered":"2024-09-20T19:03:35.334775+0000","last_clean":"2024-09-20T19:03:35.334775+0000","last_became_active":"2024-09-20T18:49:47.952150+0000","last_became_peered":"2024-09-20T18:49:47.952150+0000","last_unstale":"2024-09-20T19:03:35.334775+0000","last_undegraded":"2024-09-20T19:03:35.334775+0000","last_fullsized":"2024-09-20T19:03:35.334775+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'27","last_scrub_stamp":"2024-09-20T19:02:40.899139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:40.899139+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:02.856822+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":494,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.340669+0000","last_change":"2024-09-20T19:03:14.950167+0000","last_active":"2024-09-20T19:03:35.340669+0000","last_peered":"2024-09-20T19:03:35.340669+0000","last_clean":"2024-09-20T19:03:35.340669+0000","last_became_active":"2024-09-20T19:03:14.949859+0000","last_became_peered":"2024-09-20T19:03:14.949859+0000","last_unstale":"2024-09-20T19:03:35.340669+0000","last_undegraded":"2024-09-20T19:03:35.340669+0000","last_fullsized":"2024-09-20T19:03:35.340669+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:23.982813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:23.982813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"59'1","reported_seq":710,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.333186+0000","last_change":"2024-09-20T19:03:14.982740+0000","last_active":"2024-09-20T19:03:35.333186+0000","last_peered":"2024-09-20T19:03:35.333186+0000","last_clean":"2024-09-20T19:03:35.333186+0000","last_became_active":"2024-09-20T19:03:14.961672+0000","last_became_peered":"2024-09-20T19:03:14.961672+0000","last_unstale":"2024-09-20T19:03:35.333186+0000","last_undegraded":"2024-09-20T19:03:35.333186+0000","last_fullsized":"2024-09-20T19:03:35.333186+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:01:28.592653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:28.592653+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1308,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"273'15","reported_seq":560,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.333733+0000","last_change":"2024-09-20T19:03:14.983993+0000","last_active":"2024-09-20T19:03:35.333733+0000","last_peered":"2024-09-20T19:03:35.333733+0000","last_clean":"2024-09-20T19:03:35.333733+0000","last_became_active":"2024-09-20T19:03:14.956526+0000","last_became_peered":"2024-09-20T19:03:14.956526+0000","last_unstale":"2024-09-20T19:03:35.333733+0000","last_undegraded":"2024-09-20T19:03:35.333733+0000","last_fullsized":"2024-09-20T19:03:35.333733+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:02:42.114331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:42.114331+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:01.019356+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":644,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.332576+0000","last_change":"2024-09-20T19:01:51.564693+0000","last_active":"2024-09-20T19:03:35.332576+0000","last_peered":"2024-09-20T19:03:35.332576+0000","last_clean":"2024-09-20T19:03:35.332576+0000","last_became_active":"2024-09-20T18:49:49.961188+0000","last_became_peered":"2024-09-20T18:49:49.961188+0000","last_unstale":"2024-09-20T19:03:35.332576+0000","last_undegraded":"2024-09-20T19:03:35.332576+0000","last_fullsized":"2024-09-20T19:03:35.332576+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:51.564624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:51.564624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:02.846143+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":625,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.325063+0000","last_change":"2024-09-20T19:03:09.963421+0000","last_active":"2024-09-20T19:03:35.325063+0000","last_peered":"2024-09-20T19:03:35.325063+0000","last_clean":"2024-09-20T19:03:35.325063+0000","last_became_active":"2024-09-20T18:49:51.966715+0000","last_became_peered":"2024-09-20T18:49:51.966715+0000","last_unstale":"2024-09-20T19:03:35.325063+0000","last_undegraded":"2024-09-20T19:03:35.325063+0000","last_fullsized":"2024-09-20T19:03:35.325063+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:09.963298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:03:09.963298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:19.165030+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":662,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.332802+0000","last_change":"2024-09-20T19:03:14.961029+0000","last_active":"2024-09-20T19:03:35.332802+0000","last_peered":"2024-09-20T19:03:35.332802+0000","last_clean":"2024-09-20T19:03:35.332802+0000","last_became_active":"2024-09-20T19:03:14.960730+0000","last_became_peered":"2024-09-20T19:03:14.960730+0000","last_unstale":"2024-09-20T19:03:35.332802+0000","last_undegraded":"2024-09-20T19:03:35.332802+0000","last_fullsized":"2024-09-20T19:03:35.332802+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:48.520017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:48.520017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"273'7","reported_seq":554,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.335417+0000","last_change":"2024-09-20T19:03:14.985188+0000","last_active":"2024-09-20T19:03:35.335417+0000","last_peered":"2024-09-20T19:03:35.335417+0000","last_clean":"2024-09-20T19:03:35.335417+0000","last_became_active":"2024-09-20T19:03:14.953258+0000","last_became_peered":"2024-09-20T19:03:14.953258+0000","last_unstale":"2024-09-20T19:03:35.335417+0000","last_undegraded":"2024-09-20T19:03:35.335417+0000","last_fullsized":"2024-09-20T19:03:35.335417+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-20T18:58:33.660531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:33.660531+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":655,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.327437+0000","last_change":"2024-09-20T19:03:14.961082+0000","last_active":"2024-09-20T19:03:35.327437+0000","last_peered":"2024-09-20T19:03:35.327437+0000","last_clean":"2024-09-20T19:03:35.327437+0000","last_became_active":"2024-09-20T19:03:14.960775+0000","last_became_peered":"2024-09-20T19:03:14.960775+0000","last_unstale":"2024-09-20T19:03:35.327437+0000","last_undegraded":"2024-09-20T19:03:35.327437+0000","last_fullsized":"2024-09-20T19:03:35.327437+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:33.203312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:33.203312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":632,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.331596+0000","last_change":"2024-09-20T18:58:26.465305+0000","last_active":"2024-09-20T19:03:35.331596+0000","last_peered":"2024-09-20T19:03:35.331596+0000","last_clean":"2024-09-20T19:03:35.331596+0000","last_became_active":"2024-09-20T18:49:51.970663+0000","last_became_peered":"2024-09-20T18:49:51.970663+0000","last_unstale":"2024-09-20T19:03:35.331596+0000","last_undegraded":"2024-09-20T19:03:35.331596+0000","last_fullsized":"2024-09-20T19:03:35.331596+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:26.465264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:26.465264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":491,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.335767+0000","last_change":"2024-09-20T19:03:14.965568+0000","last_active":"2024-09-20T19:03:35.335767+0000","last_peered":"2024-09-20T19:03:35.335767+0000","last_clean":"2024-09-20T19:03:35.335767+0000","last_became_active":"2024-09-20T19:03:14.965259+0000","last_became_peered":"2024-09-20T19:03:14.965259+0000","last_unstale":"2024-09-20T19:03:35.335767+0000","last_undegraded":"2024-09-20T19:03:35.335767+0000","last_fullsized":"2024-09-20T19:03:35.335767+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:00.906652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:00.906652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":663,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.323989+0000","last_change":"2024-09-20T19:03:14.955611+0000","last_active":"2024-09-20T19:03:35.323989+0000","last_peered":"2024-09-20T19:03:35.323989+0000","last_clean":"2024-09-20T19:03:35.323989+0000","last_became_active":"2024-09-20T19:03:14.953751+0000","last_became_peered":"2024-09-20T19:03:14.953751+0000","last_unstale":"2024-09-20T19:03:35.323989+0000","last_undegraded":"2024-09-20T19:03:35.323989+0000","last_fullsized":"2024-09-20T19:03:35.323989+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:44.998147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:44.998147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"273'27","reported_seq":694,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.324342+0000","last_change":"2024-09-20T19:03:09.456720+0000","last_active":"2024-09-20T19:03:35.324342+0000","last_peered":"2024-09-20T19:03:35.324342+0000","last_clean":"2024-09-20T19:03:35.324342+0000","last_became_active":"2024-09-20T18:49:47.950939+0000","last_became_peered":"2024-09-20T18:49:47.950939+0000","last_unstale":"2024-09-20T19:03:35.324342+0000","last_undegraded":"2024-09-20T19:03:35.324342+0000","last_fullsized":"2024-09-20T19:03:35.324342+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'27","last_scrub_stamp":"2024-09-20T19:03:09.456631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:09.456631+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:26.973189+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":646,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.331835+0000","last_change":"2024-09-20T19:03:14.956903+0000","last_active":"2024-09-20T19:03:35.331835+0000","last_peered":"2024-09-20T19:03:35.331835+0000","last_clean":"2024-09-20T19:03:35.331835+0000","last_became_active":"2024-09-20T19:03:14.956533+0000","last_became_peered":"2024-09-20T19:03:14.956533+0000","last_unstale":"2024-09-20T19:03:35.331835+0000","last_undegraded":"2024-09-20T19:03:35.331835+0000","last_fullsized":"2024-09-20T19:03:35.331835+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:05.888974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:05.888974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":647,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.333875+0000","last_change":"2024-09-20T19:02:28.934597+0000","last_active":"2024-09-20T19:03:35.333875+0000","last_peered":"2024-09-20T19:03:35.333875+0000","last_clean":"2024-09-20T19:03:35.333875+0000","last_became_active":"2024-09-20T18:59:05.084496+0000","last_became_peered":"2024-09-20T18:59:05.084496+0000","last_unstale":"2024-09-20T19:03:35.333875+0000","last_undegraded":"2024-09-20T19:03:35.333875+0000","last_fullsized":"2024-09-20T19:03:35.333875+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:28.934264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:28.934264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:43.415620+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"59'1","reported_seq":70,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.333235+0000","last_change":"2024-09-20T19:03:14.962321+0000","last_active":"2024-09-20T19:03:35.333235+0000","last_peered":"2024-09-20T19:03:35.333235+0000","last_clean":"2024-09-20T19:03:35.333235+0000","last_became_active":"2024-09-20T19:03:14.956472+0000","last_became_peered":"2024-09-20T19:03:14.956472+0000","last_unstale":"2024-09-20T19:03:35.333235+0000","last_undegraded":"2024-09-20T19:03:35.333235+0000","last_fullsized":"2024-09-20T19:03:35.333235+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:03:08.218812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:03:08.218812+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:22.426186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":993,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"274'18","reported_seq":678,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.349159+0000","last_change":"2024-09-20T19:03:14.975728+0000","last_active":"2024-09-20T19:03:35.349159+0000","last_peered":"2024-09-20T19:03:35.349159+0000","last_clean":"2024-09-20T19:03:35.349159+0000","last_became_active":"2024-09-20T19:03:14.946964+0000","last_became_peered":"2024-09-20T19:03:14.946964+0000","last_unstale":"2024-09-20T19:03:35.349159+0000","last_undegraded":"2024-09-20T19:03:35.349159+0000","last_fullsized":"2024-09-20T19:03:35.349159+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:57:09.818999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:09.818999+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":523,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.335078+0000","last_change":"2024-09-20T19:03:14.951418+0000","last_active":"2024-09-20T19:03:35.335078+0000","last_peered":"2024-09-20T19:03:35.335078+0000","last_clean":"2024-09-20T19:03:35.335078+0000","last_became_active":"2024-09-20T19:03:14.951127+0000","last_became_peered":"2024-09-20T19:03:14.951127+0000","last_unstale":"2024-09-20T19:03:35.335078+0000","last_undegraded":"2024-09-20T19:03:35.335078+0000","last_fullsized":"2024-09-20T19:03:35.335078+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:02.094473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:02.094473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":634,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.329375+0000","last_change":"2024-09-20T19:02:28.294390+0000","last_active":"2024-09-20T19:03:35.329375+0000","last_peered":"2024-09-20T19:03:35.329375+0000","last_clean":"2024-09-20T19:03:35.329375+0000","last_became_active":"2024-09-20T18:49:51.970058+0000","last_became_peered":"2024-09-20T18:49:51.970058+0000","last_unstale":"2024-09-20T19:03:35.329375+0000","last_undegraded":"2024-09-20T19:03:35.329375+0000","last_fullsized":"2024-09-20T19:03:35.329375+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T19:02:28.294292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:28.294292+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:41.507259+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":651,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.331192+0000","last_change":"2024-09-20T19:03:14.968129+0000","last_active":"2024-09-20T19:03:35.331192+0000","last_peered":"2024-09-20T19:03:35.331192+0000","last_clean":"2024-09-20T19:03:35.331192+0000","last_became_active":"2024-09-20T19:03:14.955428+0000","last_became_peered":"2024-09-20T19:03:14.955428+0000","last_unstale":"2024-09-20T19:03:35.331192+0000","last_undegraded":"2024-09-20T19:03:35.331192+0000","last_fullsized":"2024-09-20T19:03:35.331192+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:57:18.032067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:18.032067+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"274'20","reported_seq":707,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.329235+0000","last_change":"2024-09-20T19:03:16.982915+0000","last_active":"2024-09-20T19:03:35.329235+0000","last_peered":"2024-09-20T19:03:35.329235+0000","last_clean":"2024-09-20T19:03:35.329235+0000","last_became_active":"2024-09-20T19:03:16.968636+0000","last_became_peered":"2024-09-20T19:03:16.968636+0000","last_unstale":"2024-09-20T19:03:35.329235+0000","last_undegraded":"2024-09-20T19:03:35.329235+0000","last_fullsized":"2024-09-20T19:03:35.329235+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'20","last_scrub_stamp":"2024-09-20T19:02:35.096979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:35.096979+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:44.003456+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"352'13","reported_seq":307,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.339988+0000","last_change":"2024-09-20T19:03:14.978232+0000","last_active":"2024-09-20T19:03:35.339988+0000","last_peered":"2024-09-20T19:03:35.339988+0000","last_clean":"2024-09-20T19:03:35.339988+0000","last_became_active":"2024-09-20T19:03:14.949808+0000","last_became_peered":"2024-09-20T19:03:14.949808+0000","last_unstale":"2024-09-20T19:03:35.339988+0000","last_undegraded":"2024-09-20T19:03:35.339988+0000","last_fullsized":"2024-09-20T19:03:35.339988+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"308'12","last_scrub_stamp":"2024-09-20T19:01:44.541156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:44.541156+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"358'6571","reported_seq":12983,"reported_epoch":358,"state":"active+remapped+backfilling","last_fresh":"2024-09-20T19:03:36.279564+0000","last_change":"2024-09-20T19:03:17.031524+0000","last_active":"2024-09-20T19:03:36.279564+0000","last_peered":"2024-09-20T19:03:36.279564+0000","last_clean":"2024-09-20T19:03:13.895389+0000","last_became_active":"2024-09-20T19:03:16.167295+0000","last_became_peered":"2024-09-20T19:03:16.167295+0000","last_unstale":"2024-09-20T19:03:36.279564+0000","last_undegraded":"2024-09-20T19:03:36.279564+0000","last_fullsized":"2024-09-20T19:03:36.279564+0000","mapping_epoch":351,"log_start":"337'1800","ondisk_log_start":"337'1800","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4771,"log_dups_size":0,"ondisk_log_size":4771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":430637056,"num_objects":6571,"num_object_clones":0,"num_object_copies":19713,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3160,"num_objects_unfound":0,"num_objects_dirty":6571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6571,"num_write_kb":420544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3163,"num_bytes_recovered":207290368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,4],"avail_no_missing":["7","0","4"],"object_location_counts":[{"shards":"0,4,7","objects":6571}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"274'27","reported_seq":704,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.324334+0000","last_change":"2024-09-20T19:03:14.978865+0000","last_active":"2024-09-20T19:03:35.324334+0000","last_peered":"2024-09-20T19:03:35.324334+0000","last_clean":"2024-09-20T19:03:35.324334+0000","last_became_active":"2024-09-20T19:03:14.944970+0000","last_became_peered":"2024-09-20T19:03:14.944970+0000","last_unstale":"2024-09-20T19:03:35.324334+0000","last_undegraded":"2024-09-20T19:03:35.324334+0000","last_fullsized":"2024-09-20T19:03:35.324334+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'27","last_scrub_stamp":"2024-09-20T19:02:27.897138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:27.897138+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":637,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.333768+0000","last_change":"2024-09-20T19:00:31.512527+0000","last_active":"2024-09-20T19:03:35.333768+0000","last_peered":"2024-09-20T19:03:35.333768+0000","last_clean":"2024-09-20T19:03:35.333768+0000","last_became_active":"2024-09-20T18:49:45.936104+0000","last_became_peered":"2024-09-20T18:49:45.936104+0000","last_unstale":"2024-09-20T19:03:35.333768+0000","last_undegraded":"2024-09-20T19:03:35.333768+0000","last_fullsized":"2024-09-20T19:03:35.333768+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:00:31.512472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:00:31.512472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:48.924236+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"352'11","reported_seq":105,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.341484+0000","last_change":"2024-09-20T19:03:14.973138+0000","last_active":"2024-09-20T19:03:35.341484+0000","last_peered":"2024-09-20T19:03:35.341484+0000","last_clean":"2024-09-20T19:03:35.341484+0000","last_became_active":"2024-09-20T19:03:14.961478+0000","last_became_peered":"2024-09-20T19:03:14.961478+0000","last_unstale":"2024-09-20T19:03:35.341484+0000","last_undegraded":"2024-09-20T19:03:35.341484+0000","last_fullsized":"2024-09-20T19:03:35.341484+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"308'10","last_scrub_stamp":"2024-09-20T19:02:23.017626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:23.017626+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:39.117901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":633,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.332516+0000","last_change":"2024-09-20T19:02:22.736879+0000","last_active":"2024-09-20T19:03:35.332516+0000","last_peered":"2024-09-20T19:03:35.332516+0000","last_clean":"2024-09-20T19:03:35.332516+0000","last_became_active":"2024-09-20T18:49:51.971399+0000","last_became_peered":"2024-09-20T18:49:51.971399+0000","last_unstale":"2024-09-20T19:03:35.332516+0000","last_undegraded":"2024-09-20T19:03:35.332516+0000","last_fullsized":"2024-09-20T19:03:35.332516+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:22.736825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:22.736825+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:51.591260+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.e","version":"358'6696","reported_seq":6779,"reported_epoch":358,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-20T19:03:36.260265+0000","last_change":"2024-09-20T19:03:16.060689+0000","last_active":"2024-09-20T19:03:36.260265+0000","last_peered":"2024-09-20T19:03:36.260265+0000","last_clean":"2024-09-20T19:03:13.896248+0000","last_became_active":"2024-09-20T19:03:16.051298+0000","last_became_peered":"2024-09-20T19:03:16.051298+0000","last_unstale":"2024-09-20T19:03:36.260265+0000","last_undegraded":"2024-09-20T19:03:36.260265+0000","last_fullsized":"2024-09-20T19:03:36.260265+0000","mapping_epoch":351,"log_start":"336'1800","ondisk_log_start":"336'1800","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4896,"log_dups_size":0,"ondisk_log_size":4896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":438829056,"num_objects":6696,"num_object_clones":0,"num_object_copies":20088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":6696,"num_objects_unfound":0,"num_objects_dirty":6696,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6696,"num_write_kb":428544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,5],"acting":[6,2,4],"avail_no_missing":["6","2","4"],"object_location_counts":[{"shards":"2,4,6","objects":6696}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"273'30","reported_seq":699,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.333541+0000","last_change":"2024-09-20T19:03:06.038303+0000","last_active":"2024-09-20T19:03:35.333541+0000","last_peered":"2024-09-20T19:03:35.333541+0000","last_clean":"2024-09-20T19:03:35.333541+0000","last_became_active":"2024-09-20T18:49:48.499189+0000","last_became_peered":"2024-09-20T18:49:48.499189+0000","last_unstale":"2024-09-20T19:03:35.333541+0000","last_undegraded":"2024-09-20T19:03:35.333541+0000","last_fullsized":"2024-09-20T19:03:35.333541+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'30","last_scrub_stamp":"2024-09-20T19:03:06.038213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:06.038213+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:20.950961+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":637,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.336036+0000","last_change":"2024-09-20T19:02:22.806587+0000","last_active":"2024-09-20T19:03:35.336036+0000","last_peered":"2024-09-20T19:03:35.336036+0000","last_clean":"2024-09-20T19:03:35.336036+0000","last_became_active":"2024-09-20T18:49:45.940544+0000","last_became_peered":"2024-09-20T18:49:45.940544+0000","last_unstale":"2024-09-20T19:03:35.336036+0000","last_undegraded":"2024-09-20T19:03:35.336036+0000","last_fullsized":"2024-09-20T19:03:35.336036+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:22.806509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:22.806509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:39.275044+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":526,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.334474+0000","last_change":"2024-09-20T19:03:14.952042+0000","last_active":"2024-09-20T19:03:35.334474+0000","last_peered":"2024-09-20T19:03:35.334474+0000","last_clean":"2024-09-20T19:03:35.334474+0000","last_became_active":"2024-09-20T19:03:14.951719+0000","last_became_peered":"2024-09-20T19:03:14.951719+0000","last_unstale":"2024-09-20T19:03:35.334474+0000","last_undegraded":"2024-09-20T19:03:35.334474+0000","last_fullsized":"2024-09-20T19:03:35.334474+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.229201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.229201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":517,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.333635+0000","last_change":"2024-09-20T19:03:14.952656+0000","last_active":"2024-09-20T19:03:35.333635+0000","last_peered":"2024-09-20T19:03:35.333635+0000","last_clean":"2024-09-20T19:03:35.333635+0000","last_became_active":"2024-09-20T19:03:14.952336+0000","last_became_peered":"2024-09-20T19:03:14.952336+0000","last_unstale":"2024-09-20T19:03:35.333635+0000","last_undegraded":"2024-09-20T19:03:35.333635+0000","last_fullsized":"2024-09-20T19:03:35.333635+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:27.704520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:27.704520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"358'6587","reported_seq":6663,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:36.261214+0000","last_change":"2024-09-20T19:02:11.828279+0000","last_active":"2024-09-20T19:03:36.261214+0000","last_peered":"2024-09-20T19:03:36.261214+0000","last_clean":"2024-09-20T19:03:36.261214+0000","last_became_active":"2024-09-20T19:02:11.827468+0000","last_became_peered":"2024-09-20T19:02:11.827468+0000","last_unstale":"2024-09-20T19:03:36.261214+0000","last_undegraded":"2024-09-20T19:03:36.261214+0000","last_fullsized":"2024-09-20T19:03:36.261214+0000","mapping_epoch":325,"log_start":"335'1655","ondisk_log_start":"335'1655","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4932,"log_dups_size":0,"ondisk_log_size":4932,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":431685632,"num_objects":6587,"num_object_clones":0,"num_object_copies":19761,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6587,"num_write_kb":421568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"274'28","reported_seq":631,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.331609+0000","last_change":"2024-09-20T19:03:14.980995+0000","last_active":"2024-09-20T19:03:35.331609+0000","last_peered":"2024-09-20T19:03:35.331609+0000","last_clean":"2024-09-20T19:03:35.331609+0000","last_became_active":"2024-09-20T19:03:14.960136+0000","last_became_peered":"2024-09-20T19:03:14.960136+0000","last_unstale":"2024-09-20T19:03:35.331609+0000","last_undegraded":"2024-09-20T19:03:35.331609+0000","last_fullsized":"2024-09-20T19:03:35.331609+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'28","last_scrub_stamp":"2024-09-20T19:03:06.269623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:06.269623+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:24.425577+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":41,"num_read_kb":34,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":661,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.329529+0000","last_change":"2024-09-20T19:03:14.965158+0000","last_active":"2024-09-20T19:03:35.329529+0000","last_peered":"2024-09-20T19:03:35.329529+0000","last_clean":"2024-09-20T19:03:35.329529+0000","last_became_active":"2024-09-20T19:03:14.951317+0000","last_became_peered":"2024-09-20T19:03:14.951317+0000","last_unstale":"2024-09-20T19:03:35.329529+0000","last_undegraded":"2024-09-20T19:03:35.329529+0000","last_fullsized":"2024-09-20T19:03:35.329529+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:00:29.890681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:00:29.890681+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"353'12","reported_seq":848,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.333409+0000","last_change":"2024-09-20T19:03:17.027278+0000","last_active":"2024-09-20T19:03:35.333409+0000","last_peered":"2024-09-20T19:03:35.333409+0000","last_clean":"2024-09-20T19:03:35.333409+0000","last_became_active":"2024-09-20T19:03:16.989458+0000","last_became_peered":"2024-09-20T19:03:16.989458+0000","last_unstale":"2024-09-20T19:03:35.333409+0000","last_undegraded":"2024-09-20T19:03:35.333409+0000","last_fullsized":"2024-09-20T19:03:35.333409+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:58:23.114371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:23.114371+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":640,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.327598+0000","last_change":"2024-09-20T18:59:05.082117+0000","last_active":"2024-09-20T19:03:35.327598+0000","last_peered":"2024-09-20T19:03:35.327598+0000","last_clean":"2024-09-20T19:03:35.327598+0000","last_became_active":"2024-09-20T18:59:05.081814+0000","last_became_peered":"2024-09-20T18:59:05.081814+0000","last_unstale":"2024-09-20T19:03:35.327598+0000","last_undegraded":"2024-09-20T19:03:35.327598+0000","last_fullsized":"2024-09-20T19:03:35.327598+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:29.530001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:29.530001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"358'6579","reported_seq":13069,"reported_epoch":358,"state":"active+remapped+backfilling","last_fresh":"2024-09-20T19:03:36.281559+0000","last_change":"2024-09-20T19:03:16.195280+0000","last_active":"2024-09-20T19:03:36.281559+0000","last_peered":"2024-09-20T19:03:36.281559+0000","last_clean":"2024-09-20T19:03:13.927198+0000","last_became_active":"2024-09-20T19:03:16.168255+0000","last_became_peered":"2024-09-20T19:03:16.168255+0000","last_unstale":"2024-09-20T19:03:36.281559+0000","last_undegraded":"2024-09-20T19:03:36.281559+0000","last_fullsized":"2024-09-20T19:03:36.281559+0000","mapping_epoch":351,"log_start":"338'1809","ondisk_log_start":"338'1809","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4770,"log_dups_size":0,"ondisk_log_size":4770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":431161344,"num_objects":6579,"num_object_clones":0,"num_object_copies":19737,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3163,"num_objects_unfound":0,"num_objects_dirty":6579,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6579,"num_write_kb":421056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3204,"num_bytes_recovered":209977344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,3],"acting":[7,0,4],"avail_no_missing":["7","0","4"],"object_location_counts":[{"shards":"0,4,7","objects":6579}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"274'20","reported_seq":717,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.333034+0000","last_change":"2024-09-20T19:03:14.970171+0000","last_active":"2024-09-20T19:03:35.333034+0000","last_peered":"2024-09-20T19:03:35.333034+0000","last_clean":"2024-09-20T19:03:35.333034+0000","last_became_active":"2024-09-20T19:03:14.957350+0000","last_became_peered":"2024-09-20T19:03:14.957350+0000","last_unstale":"2024-09-20T19:03:35.333034+0000","last_undegraded":"2024-09-20T19:03:35.333034+0000","last_fullsized":"2024-09-20T19:03:35.333034+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'20","last_scrub_stamp":"2024-09-20T19:02:35.930910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:35.930910+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:56.035276+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":661,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.333309+0000","last_change":"2024-09-20T19:02:30.686939+0000","last_active":"2024-09-20T19:03:35.333309+0000","last_peered":"2024-09-20T19:03:35.333309+0000","last_clean":"2024-09-20T19:03:35.333309+0000","last_became_active":"2024-09-20T18:49:45.942496+0000","last_became_peered":"2024-09-20T18:49:45.942496+0000","last_unstale":"2024-09-20T19:03:35.333309+0000","last_undegraded":"2024-09-20T19:03:35.333309+0000","last_fullsized":"2024-09-20T19:03:35.333309+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:30.686872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:30.686872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:51.214827+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"262'10","reported_seq":835,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.323702+0000","last_change":"2024-09-20T19:02:04.994109+0000","last_active":"2024-09-20T19:03:35.323702+0000","last_peered":"2024-09-20T19:03:35.323702+0000","last_clean":"2024-09-20T19:03:35.323702+0000","last_became_active":"2024-09-20T18:59:05.080183+0000","last_became_peered":"2024-09-20T18:59:05.080183+0000","last_unstale":"2024-09-20T19:03:35.323702+0000","last_undegraded":"2024-09-20T19:03:35.323702+0000","last_fullsized":"2024-09-20T19:03:35.323702+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'10","last_scrub_stamp":"2024-09-20T19:02:04.994047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:04.994047+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":618,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.325197+0000","last_change":"2024-09-20T19:01:37.900410+0000","last_active":"2024-09-20T19:03:35.325197+0000","last_peered":"2024-09-20T19:03:35.325197+0000","last_clean":"2024-09-20T19:03:35.325197+0000","last_became_active":"2024-09-20T18:49:51.964780+0000","last_became_peered":"2024-09-20T18:49:51.964780+0000","last_unstale":"2024-09-20T19:03:35.325197+0000","last_undegraded":"2024-09-20T19:03:35.325197+0000","last_fullsized":"2024-09-20T19:03:35.325197+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:37.900328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:37.900328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:02.124500+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.d","version":"358'6835","reported_seq":7604,"reported_epoch":358,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-20T19:03:36.231379+0000","last_change":"2024-09-20T19:03:16.400294+0000","last_active":"2024-09-20T19:03:36.231379+0000","last_peered":"2024-09-20T19:03:36.231379+0000","last_clean":"2024-09-20T19:03:13.933264+0000","last_became_active":"2024-09-20T19:03:16.292336+0000","last_became_peered":"2024-09-20T19:03:16.292336+0000","last_unstale":"2024-09-20T19:03:36.231379+0000","last_undegraded":"2024-09-20T19:03:36.231379+0000","last_fullsized":"2024-09-20T19:03:15.946661+0000","mapping_epoch":351,"log_start":"337'1917","ondisk_log_start":"337'1917","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4918,"log_dups_size":0,"ondisk_log_size":4918,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":447938560,"num_objects":6835,"num_object_clones":0,"num_object_copies":20505,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":5571,"num_objects_unfound":0,"num_objects_dirty":6835,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6835,"num_write_kb":437440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":346,"num_bytes_recovered":22675456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0],"avail_no_missing":["6","0","4"],"object_location_counts":[{"shards":"0,4,6","objects":5571},{"shards":"0,6","objects":1264}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"274'29","reported_seq":685,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.323821+0000","last_change":"2024-09-20T19:00:04.810907+0000","last_active":"2024-09-20T19:03:35.323821+0000","last_peered":"2024-09-20T19:03:35.323821+0000","last_clean":"2024-09-20T19:03:35.323821+0000","last_became_active":"2024-09-20T18:49:48.497736+0000","last_became_peered":"2024-09-20T18:49:48.497736+0000","last_unstale":"2024-09-20T19:03:35.323821+0000","last_undegraded":"2024-09-20T19:03:35.323821+0000","last_fullsized":"2024-09-20T19:03:35.323821+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'29","last_scrub_stamp":"2024-09-20T19:00:04.810831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:00:04.810831+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":637,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.332728+0000","last_change":"2024-09-20T19:02:41.859231+0000","last_active":"2024-09-20T19:03:35.332728+0000","last_peered":"2024-09-20T19:03:35.332728+0000","last_clean":"2024-09-20T19:03:35.332728+0000","last_became_active":"2024-09-20T18:49:45.942253+0000","last_became_peered":"2024-09-20T18:49:45.942253+0000","last_unstale":"2024-09-20T19:03:35.332728+0000","last_undegraded":"2024-09-20T19:03:35.332728+0000","last_fullsized":"2024-09-20T19:03:35.332728+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:41.859168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:41.859168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:03.925342+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":224,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.334871+0000","last_change":"2024-09-20T19:02:27.639446+0000","last_active":"2024-09-20T19:03:35.334871+0000","last_peered":"2024-09-20T19:03:35.334871+0000","last_clean":"2024-09-20T19:03:35.334871+0000","last_became_active":"2024-09-20T18:59:05.085636+0000","last_became_peered":"2024-09-20T18:59:05.085636+0000","last_unstale":"2024-09-20T19:03:35.334871+0000","last_undegraded":"2024-09-20T19:03:35.334871+0000","last_fullsized":"2024-09-20T19:03:35.334871+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:27.639384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:27.639384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":655,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.331021+0000","last_change":"2024-09-20T19:02:23.886386+0000","last_active":"2024-09-20T19:03:35.331021+0000","last_peered":"2024-09-20T19:03:35.331021+0000","last_clean":"2024-09-20T19:03:35.331021+0000","last_became_active":"2024-09-20T18:59:05.085131+0000","last_became_peered":"2024-09-20T18:59:05.085131+0000","last_unstale":"2024-09-20T19:03:35.331021+0000","last_undegraded":"2024-09-20T19:03:35.331021+0000","last_fullsized":"2024-09-20T19:03:35.331021+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:23.886308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:23.886308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:42.873874+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"358'6520","reported_seq":6661,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:36.205876+0000","last_change":"2024-09-20T19:03:15.459244+0000","last_active":"2024-09-20T19:03:36.205876+0000","last_peered":"2024-09-20T19:03:36.205876+0000","last_clean":"2024-09-20T19:03:36.205876+0000","last_became_active":"2024-09-20T19:02:11.827305+0000","last_became_peered":"2024-09-20T19:02:11.827305+0000","last_unstale":"2024-09-20T19:03:36.205876+0000","last_undegraded":"2024-09-20T19:03:36.205876+0000","last_fullsized":"2024-09-20T19:03:36.205876+0000","mapping_epoch":325,"log_start":"335'1602","ondisk_log_start":"335'1602","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"350'5605","last_scrub_stamp":"2024-09-20T19:03:15.459174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:03:15.459174+0000","objects_scrubbed":5605,"log_size":4918,"log_dups_size":0,"ondisk_log_size":4918,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:35.506042+0000","scrub_duration":598,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":427294720,"num_objects":6520,"num_object_clones":0,"num_object_copies":19560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6520,"num_write_kb":417280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"353'41","reported_seq":911,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.332632+0000","last_change":"2024-09-20T19:03:17.024385+0000","last_active":"2024-09-20T19:03:35.332632+0000","last_peered":"2024-09-20T19:03:35.332632+0000","last_clean":"2024-09-20T19:03:35.332632+0000","last_became_active":"2024-09-20T19:03:16.976916+0000","last_became_peered":"2024-09-20T19:03:16.976916+0000","last_unstale":"2024-09-20T19:03:35.332632+0000","last_undegraded":"2024-09-20T19:03:35.332632+0000","last_fullsized":"2024-09-20T19:03:35.332632+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'40","last_scrub_stamp":"2024-09-20T19:02:36.900916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:36.900916+0000","objects_scrubbed":10,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:05.286209+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":539,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.324056+0000","last_change":"2024-09-20T19:03:14.951169+0000","last_active":"2024-09-20T19:03:35.324056+0000","last_peered":"2024-09-20T19:03:35.324056+0000","last_clean":"2024-09-20T19:03:35.324056+0000","last_became_active":"2024-09-20T19:03:14.950852+0000","last_became_peered":"2024-09-20T19:03:14.950852+0000","last_unstale":"2024-09-20T19:03:35.324056+0000","last_undegraded":"2024-09-20T19:03:35.324056+0000","last_fullsized":"2024-09-20T19:03:35.324056+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:00:31.630361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:00:31.630361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":701,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.332493+0000","last_change":"2024-09-20T19:03:11.910684+0000","last_active":"2024-09-20T19:03:35.332493+0000","last_peered":"2024-09-20T19:03:35.332493+0000","last_clean":"2024-09-20T19:03:35.332493+0000","last_became_active":"2024-09-20T18:59:05.086786+0000","last_became_peered":"2024-09-20T18:59:05.086786+0000","last_unstale":"2024-09-20T19:03:35.332493+0000","last_undegraded":"2024-09-20T19:03:35.332493+0000","last_fullsized":"2024-09-20T19:03:35.332493+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-20T19:03:11.909394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:03:11.909394+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:35.264669+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0.0012781159999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":664,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.324454+0000","last_change":"2024-09-20T19:03:15.034384+0000","last_active":"2024-09-20T19:03:35.324454+0000","last_peered":"2024-09-20T19:03:35.324454+0000","last_clean":"2024-09-20T19:03:35.324454+0000","last_became_active":"2024-09-20T19:03:14.948079+0000","last_became_peered":"2024-09-20T19:03:14.948079+0000","last_unstale":"2024-09-20T19:03:35.324454+0000","last_undegraded":"2024-09-20T19:03:35.324454+0000","last_fullsized":"2024-09-20T19:03:35.324454+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:15.034340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:03:15.034340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:37.241809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":635,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.334642+0000","last_change":"2024-09-20T19:03:14.949138+0000","last_active":"2024-09-20T19:03:35.334642+0000","last_peered":"2024-09-20T19:03:35.334642+0000","last_clean":"2024-09-20T19:03:35.334642+0000","last_became_active":"2024-09-20T19:03:14.948815+0000","last_became_peered":"2024-09-20T19:03:14.948815+0000","last_unstale":"2024-09-20T19:03:35.334642+0000","last_undegraded":"2024-09-20T19:03:35.334642+0000","last_fullsized":"2024-09-20T19:03:35.334642+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:04.526683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:04.526683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"358'6692","reported_seq":6768,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:36.425799+0000","last_change":"2024-09-20T19:02:11.825132+0000","last_active":"2024-09-20T19:03:36.425799+0000","last_peered":"2024-09-20T19:03:36.425799+0000","last_clean":"2024-09-20T19:03:36.425799+0000","last_became_active":"2024-09-20T19:02:11.824846+0000","last_became_peered":"2024-09-20T19:02:11.824846+0000","last_unstale":"2024-09-20T19:03:36.425799+0000","last_undegraded":"2024-09-20T19:03:36.425799+0000","last_fullsized":"2024-09-20T19:03:36.425799+0000","mapping_epoch":325,"log_start":"335'1527","ondisk_log_start":"335'1527","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":5165,"log_dups_size":0,"ondisk_log_size":5165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":438566912,"num_objects":6692,"num_object_clones":0,"num_object_copies":20076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6692,"num_write_kb":428288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"273'17","reported_seq":682,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.332373+0000","last_change":"2024-09-20T19:02:24.853473+0000","last_active":"2024-09-20T19:03:35.332373+0000","last_peered":"2024-09-20T19:03:35.332373+0000","last_clean":"2024-09-20T19:03:35.332373+0000","last_became_active":"2024-09-20T18:49:47.950904+0000","last_became_peered":"2024-09-20T18:49:47.950904+0000","last_unstale":"2024-09-20T19:03:35.332373+0000","last_undegraded":"2024-09-20T19:03:35.332373+0000","last_fullsized":"2024-09-20T19:03:35.332373+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'17","last_scrub_stamp":"2024-09-20T19:02:24.853374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:24.853374+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:46.813829+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":36,"num_read_kb":25,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":667,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.331423+0000","last_change":"2024-09-20T19:02:29.840761+0000","last_active":"2024-09-20T19:03:35.331423+0000","last_peered":"2024-09-20T19:03:35.331423+0000","last_clean":"2024-09-20T19:03:35.331423+0000","last_became_active":"2024-09-20T18:59:05.079789+0000","last_became_peered":"2024-09-20T18:59:05.079789+0000","last_unstale":"2024-09-20T19:03:35.331423+0000","last_undegraded":"2024-09-20T19:03:35.331423+0000","last_fullsized":"2024-09-20T19:03:35.331423+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:29.840601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:29.840601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:39.311309+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":649,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.325253+0000","last_change":"2024-09-20T19:01:58.789304+0000","last_active":"2024-09-20T19:03:35.325253+0000","last_peered":"2024-09-20T19:03:35.325253+0000","last_clean":"2024-09-20T19:03:35.325253+0000","last_became_active":"2024-09-20T18:49:44.937135+0000","last_became_peered":"2024-09-20T18:49:44.937135+0000","last_unstale":"2024-09-20T19:03:35.325253+0000","last_undegraded":"2024-09-20T19:03:35.325253+0000","last_fullsized":"2024-09-20T19:03:35.325253+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:01:58.789251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:01:58.789251+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00095558800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":624,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.325529+0000","last_change":"2024-09-20T19:01:17.747452+0000","last_active":"2024-09-20T19:03:35.325529+0000","last_peered":"2024-09-20T19:03:35.325529+0000","last_clean":"2024-09-20T19:03:35.325529+0000","last_became_active":"2024-09-20T18:49:49.954424+0000","last_became_peered":"2024-09-20T18:49:49.954424+0000","last_unstale":"2024-09-20T19:03:35.325529+0000","last_undegraded":"2024-09-20T19:03:35.325529+0000","last_fullsized":"2024-09-20T19:03:35.325529+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:17.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:17.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:49.870946+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":635,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.329116+0000","last_change":"2024-09-20T19:01:15.203591+0000","last_active":"2024-09-20T19:03:35.329116+0000","last_peered":"2024-09-20T19:03:35.329116+0000","last_clean":"2024-09-20T19:03:35.329116+0000","last_became_active":"2024-09-20T18:59:05.081459+0000","last_became_peered":"2024-09-20T18:59:05.081459+0000","last_unstale":"2024-09-20T19:03:35.329116+0000","last_undegraded":"2024-09-20T19:03:35.329116+0000","last_fullsized":"2024-09-20T19:03:35.329116+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:15.203529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:15.203529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:52.197162+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"358'6620","reported_seq":6697,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:37.384240+0000","last_change":"2024-09-20T19:02:11.828477+0000","last_active":"2024-09-20T19:03:37.384240+0000","last_peered":"2024-09-20T19:03:37.384240+0000","last_clean":"2024-09-20T19:03:37.384240+0000","last_became_active":"2024-09-20T19:02:11.827996+0000","last_became_peered":"2024-09-20T19:02:11.827996+0000","last_unstale":"2024-09-20T19:03:37.384240+0000","last_undegraded":"2024-09-20T19:03:37.384240+0000","last_fullsized":"2024-09-20T19:03:37.384240+0000","mapping_epoch":325,"log_start":"338'2284","ondisk_log_start":"338'2284","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4336,"log_dups_size":0,"ondisk_log_size":4336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":433848320,"num_objects":6620,"num_object_clones":0,"num_object_copies":19860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6620,"num_write_kb":423680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"273'24","reported_seq":686,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.329336+0000","last_change":"2024-09-20T18:59:06.022021+0000","last_active":"2024-09-20T19:03:35.329336+0000","last_peered":"2024-09-20T19:03:35.329336+0000","last_clean":"2024-09-20T19:03:35.329336+0000","last_became_active":"2024-09-20T18:49:47.949230+0000","last_became_peered":"2024-09-20T18:49:47.949230+0000","last_unstale":"2024-09-20T19:03:35.329336+0000","last_undegraded":"2024-09-20T19:03:35.329336+0000","last_fullsized":"2024-09-20T19:03:35.329336+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:59:06.021962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:59:06.021962+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":664,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.326095+0000","last_change":"2024-09-20T19:03:14.954738+0000","last_active":"2024-09-20T19:03:35.326095+0000","last_peered":"2024-09-20T19:03:35.326095+0000","last_clean":"2024-09-20T19:03:35.326095+0000","last_became_active":"2024-09-20T19:03:14.954260+0000","last_became_peered":"2024-09-20T19:03:14.954260+0000","last_unstale":"2024-09-20T19:03:35.326095+0000","last_undegraded":"2024-09-20T19:03:35.326095+0000","last_fullsized":"2024-09-20T19:03:35.326095+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:24.984022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:24.984022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":1481,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.327615+0000","last_change":"2024-09-20T19:02:02.494344+0000","last_active":"2024-09-20T19:03:35.327615+0000","last_peered":"2024-09-20T19:03:35.327615+0000","last_clean":"2024-09-20T19:03:35.327615+0000","last_became_active":"2024-09-20T18:49:44.936819+0000","last_became_peered":"2024-09-20T18:49:44.936819+0000","last_unstale":"2024-09-20T19:03:35.327615+0000","last_undegraded":"2024-09-20T19:03:35.327615+0000","last_fullsized":"2024-09-20T19:03:35.327615+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-20T19:02:02.494291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:02:02.494291+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0010996199999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":824,"num_read_kb":819,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":645,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.333811+0000","last_change":"2024-09-20T19:02:27.785566+0000","last_active":"2024-09-20T19:03:35.333811+0000","last_peered":"2024-09-20T19:03:35.333811+0000","last_clean":"2024-09-20T19:03:35.333811+0000","last_became_active":"2024-09-20T18:49:49.961066+0000","last_became_peered":"2024-09-20T18:49:49.961066+0000","last_unstale":"2024-09-20T19:03:35.333811+0000","last_undegraded":"2024-09-20T19:03:35.333811+0000","last_fullsized":"2024-09-20T19:03:35.333811+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:27.785524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:27.785524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:48.498566+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":641,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.332842+0000","last_change":"2024-09-20T19:02:26.861253+0000","last_active":"2024-09-20T19:03:35.332842+0000","last_peered":"2024-09-20T19:03:35.332842+0000","last_clean":"2024-09-20T19:03:35.332842+0000","last_became_active":"2024-09-20T18:49:51.970343+0000","last_became_peered":"2024-09-20T18:49:51.970343+0000","last_unstale":"2024-09-20T19:03:35.332842+0000","last_undegraded":"2024-09-20T19:03:35.332842+0000","last_fullsized":"2024-09-20T19:03:35.332842+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:26.861185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:26.861185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:38.453435+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"358'6576","reported_seq":6653,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:36.253133+0000","last_change":"2024-09-20T19:02:11.828065+0000","last_active":"2024-09-20T19:03:36.253133+0000","last_peered":"2024-09-20T19:03:36.253133+0000","last_clean":"2024-09-20T19:03:36.253133+0000","last_became_active":"2024-09-20T19:02:11.827628+0000","last_became_peered":"2024-09-20T19:02:11.827628+0000","last_unstale":"2024-09-20T19:03:36.253133+0000","last_undegraded":"2024-09-20T19:03:36.253133+0000","last_fullsized":"2024-09-20T19:03:36.253133+0000","mapping_epoch":325,"log_start":"336'1727","ondisk_log_start":"336'1727","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4849,"log_dups_size":0,"ondisk_log_size":4849,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":430964736,"num_objects":6576,"num_object_clones":0,"num_object_copies":19728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6576,"num_write_kb":420864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"352'33","reported_seq":345,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.346659+0000","last_change":"2024-09-20T19:03:14.979177+0000","last_active":"2024-09-20T19:03:35.346659+0000","last_peered":"2024-09-20T19:03:35.346659+0000","last_clean":"2024-09-20T19:03:35.346659+0000","last_became_active":"2024-09-20T19:03:14.948410+0000","last_became_peered":"2024-09-20T19:03:14.948410+0000","last_unstale":"2024-09-20T19:03:35.346659+0000","last_undegraded":"2024-09-20T19:03:35.346659+0000","last_fullsized":"2024-09-20T19:03:35.346659+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"308'32","last_scrub_stamp":"2024-09-20T19:02:25.699337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:25.699337+0000","objects_scrubbed":8,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:50.874584+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":308,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":659,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.333994+0000","last_change":"2024-09-20T19:02:31.965132+0000","last_active":"2024-09-20T19:03:35.333994+0000","last_peered":"2024-09-20T19:03:35.333994+0000","last_clean":"2024-09-20T19:03:35.333994+0000","last_became_active":"2024-09-20T18:59:05.079111+0000","last_became_peered":"2024-09-20T18:59:05.079111+0000","last_unstale":"2024-09-20T19:03:35.333994+0000","last_undegraded":"2024-09-20T19:03:35.333994+0000","last_fullsized":"2024-09-20T19:03:35.333994+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:31.965066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:31.965066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:50.572885+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":713,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.327752+0000","last_change":"2024-09-20T19:03:14.368338+0000","last_active":"2024-09-20T19:03:35.327752+0000","last_peered":"2024-09-20T19:03:35.327752+0000","last_clean":"2024-09-20T19:03:35.327752+0000","last_became_active":"2024-09-20T18:59:05.086103+0000","last_became_peered":"2024-09-20T18:59:05.086103+0000","last_unstale":"2024-09-20T19:03:35.327752+0000","last_undegraded":"2024-09-20T19:03:35.327752+0000","last_fullsized":"2024-09-20T19:03:35.327752+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:03:14.368279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:03:14.368279+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:42.830264+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":622,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.329531+0000","last_change":"2024-09-20T18:58:23.545370+0000","last_active":"2024-09-20T19:03:35.329531+0000","last_peered":"2024-09-20T19:03:35.329531+0000","last_clean":"2024-09-20T19:03:35.329531+0000","last_became_active":"2024-09-20T18:49:49.956128+0000","last_became_peered":"2024-09-20T18:49:49.956128+0000","last_unstale":"2024-09-20T19:03:35.329531+0000","last_undegraded":"2024-09-20T19:03:35.329531+0000","last_fullsized":"2024-09-20T19:03:35.329531+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:23.545321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:23.545321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":659,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.333645+0000","last_change":"2024-09-20T19:03:14.959711+0000","last_active":"2024-09-20T19:03:35.333645+0000","last_peered":"2024-09-20T19:03:35.333645+0000","last_clean":"2024-09-20T19:03:35.333645+0000","last_became_active":"2024-09-20T19:03:14.959411+0000","last_became_peered":"2024-09-20T19:03:14.959411+0000","last_unstale":"2024-09-20T19:03:35.333645+0000","last_undegraded":"2024-09-20T19:03:35.333645+0000","last_fullsized":"2024-09-20T19:03:35.333645+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:30.557368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:30.557368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.0","version":"358'6676","reported_seq":7446,"reported_epoch":358,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-20T19:03:36.238811+0000","last_change":"2024-09-20T19:03:16.155496+0000","last_active":"2024-09-20T19:03:36.238811+0000","last_peered":"2024-09-20T19:03:36.238811+0000","last_clean":"2024-09-20T19:03:13.922116+0000","last_became_active":"2024-09-20T19:03:16.128779+0000","last_became_peered":"2024-09-20T19:03:16.128779+0000","last_unstale":"2024-09-20T19:03:36.238811+0000","last_undegraded":"2024-09-20T19:03:36.238811+0000","last_fullsized":"2024-09-20T19:03:15.944421+0000","mapping_epoch":351,"log_start":"336'1739","ondisk_log_start":"336'1739","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4937,"log_dups_size":0,"ondisk_log_size":4937,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":437518336,"num_objects":6676,"num_object_clones":0,"num_object_copies":20028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":5391,"num_objects_unfound":0,"num_objects_dirty":6676,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6676,"num_write_kb":427264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":346,"num_bytes_recovered":22675456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1],"avail_no_missing":["6","1","4"],"object_location_counts":[{"shards":"1,4,6","objects":5391},{"shards":"1,6","objects":1285}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"274'19","reported_seq":592,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.335936+0000","last_change":"2024-09-20T19:03:14.982340+0000","last_active":"2024-09-20T19:03:35.335936+0000","last_peered":"2024-09-20T19:03:35.335936+0000","last_clean":"2024-09-20T19:03:35.335936+0000","last_became_active":"2024-09-20T19:03:14.948357+0000","last_became_peered":"2024-09-20T19:03:14.948357+0000","last_unstale":"2024-09-20T19:03:35.335936+0000","last_undegraded":"2024-09-20T19:03:35.335936+0000","last_fullsized":"2024-09-20T19:03:35.335936+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:58:24.616580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:24.616580+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,3],"acting":[6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":517,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.324227+0000","last_change":"2024-09-20T19:03:14.947842+0000","last_active":"2024-09-20T19:03:35.324227+0000","last_peered":"2024-09-20T19:03:35.324227+0000","last_clean":"2024-09-20T19:03:35.324227+0000","last_became_active":"2024-09-20T19:03:14.947457+0000","last_became_peered":"2024-09-20T19:03:14.947457+0000","last_unstale":"2024-09-20T19:03:35.324227+0000","last_undegraded":"2024-09-20T19:03:35.324227+0000","last_fullsized":"2024-09-20T19:03:35.324227+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.693993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.693993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:23.404776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,1],"acting":[2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":677,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.333532+0000","last_change":"2024-09-20T19:03:14.963776+0000","last_active":"2024-09-20T19:03:35.333532+0000","last_peered":"2024-09-20T19:03:35.333532+0000","last_clean":"2024-09-20T19:03:35.333532+0000","last_became_active":"2024-09-20T19:03:14.962949+0000","last_became_peered":"2024-09-20T19:03:14.962949+0000","last_unstale":"2024-09-20T19:03:35.333532+0000","last_undegraded":"2024-09-20T19:03:35.333532+0000","last_fullsized":"2024-09-20T19:03:35.333532+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:38.463404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:38.463404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":639,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.335000+0000","last_change":"2024-09-20T18:59:05.080255+0000","last_active":"2024-09-20T19:03:35.335000+0000","last_peered":"2024-09-20T19:03:35.335000+0000","last_clean":"2024-09-20T19:03:35.335000+0000","last_became_active":"2024-09-20T18:59:05.079897+0000","last_became_peered":"2024-09-20T18:59:05.079897+0000","last_unstale":"2024-09-20T19:03:35.335000+0000","last_undegraded":"2024-09-20T19:03:35.335000+0000","last_fullsized":"2024-09-20T19:03:35.335000+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:01.083156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:59:01.083156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:42.880064+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"358'6802","reported_seq":6879,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:36.402349+0000","last_change":"2024-09-20T19:02:11.825263+0000","last_active":"2024-09-20T19:03:36.402349+0000","last_peered":"2024-09-20T19:03:36.402349+0000","last_clean":"2024-09-20T19:03:36.402349+0000","last_became_active":"2024-09-20T19:02:11.825011+0000","last_became_peered":"2024-09-20T19:02:11.825011+0000","last_unstale":"2024-09-20T19:03:36.402349+0000","last_undegraded":"2024-09-20T19:03:36.402349+0000","last_fullsized":"2024-09-20T19:03:36.402349+0000","mapping_epoch":325,"log_start":"334'1228","ondisk_log_start":"334'1228","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":5574,"log_dups_size":0,"ondisk_log_size":5574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":445775872,"num_objects":6802,"num_object_clones":0,"num_object_copies":20406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6802,"num_write_kb":435328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"274'31","reported_seq":586,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.326590+0000","last_change":"2024-09-20T19:03:14.986988+0000","last_active":"2024-09-20T19:03:35.326590+0000","last_peered":"2024-09-20T19:03:35.326590+0000","last_clean":"2024-09-20T19:03:35.326590+0000","last_became_active":"2024-09-20T19:03:14.954783+0000","last_became_peered":"2024-09-20T19:03:14.954783+0000","last_unstale":"2024-09-20T19:03:35.326590+0000","last_undegraded":"2024-09-20T19:03:35.326590+0000","last_fullsized":"2024-09-20T19:03:35.326590+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'31","last_scrub_stamp":"2024-09-20T19:03:12.196851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:12.196851+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:29.260709+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":49,"num_read_kb":39,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":234,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.335002+0000","last_change":"2024-09-20T19:03:14.963804+0000","last_active":"2024-09-20T19:03:35.335002+0000","last_peered":"2024-09-20T19:03:35.335002+0000","last_clean":"2024-09-20T19:03:35.335002+0000","last_became_active":"2024-09-20T19:03:14.959291+0000","last_became_peered":"2024-09-20T19:03:14.959291+0000","last_unstale":"2024-09-20T19:03:35.335002+0000","last_undegraded":"2024-09-20T19:03:35.335002+0000","last_fullsized":"2024-09-20T19:03:35.335002+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:42.479489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:42.479489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,3],"acting":[6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":655,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.332461+0000","last_change":"2024-09-20T19:03:14.948543+0000","last_active":"2024-09-20T19:03:35.332461+0000","last_peered":"2024-09-20T19:03:35.332461+0000","last_clean":"2024-09-20T19:03:35.332461+0000","last_became_active":"2024-09-20T19:03:14.948239+0000","last_became_peered":"2024-09-20T19:03:14.948239+0000","last_unstale":"2024-09-20T19:03:35.332461+0000","last_undegraded":"2024-09-20T19:03:35.332461+0000","last_fullsized":"2024-09-20T19:03:35.332461+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:32.856856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:32.856856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:56.734943+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":654,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.324507+0000","last_change":"2024-09-20T19:03:14.944092+0000","last_active":"2024-09-20T19:03:35.324507+0000","last_peered":"2024-09-20T19:03:35.324507+0000","last_clean":"2024-09-20T19:03:35.324507+0000","last_became_active":"2024-09-20T19:03:14.943782+0000","last_became_peered":"2024-09-20T19:03:14.943782+0000","last_unstale":"2024-09-20T19:03:35.324507+0000","last_undegraded":"2024-09-20T19:03:35.324507+0000","last_fullsized":"2024-09-20T19:03:35.324507+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:31.687188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:31.687188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:26.355079+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2","version":"358'6704","reported_seq":6786,"reported_epoch":358,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-20T19:03:36.857308+0000","last_change":"2024-09-20T19:03:16.198592+0000","last_active":"2024-09-20T19:03:36.857308+0000","last_peered":"2024-09-20T19:03:36.857308+0000","last_clean":"2024-09-20T19:03:13.930949+0000","last_became_active":"2024-09-20T19:03:16.167977+0000","last_became_peered":"2024-09-20T19:03:16.167977+0000","last_unstale":"2024-09-20T19:03:36.857308+0000","last_undegraded":"2024-09-20T19:03:36.857308+0000","last_fullsized":"2024-09-20T19:03:36.857308+0000","mapping_epoch":351,"log_start":"338'2282","ondisk_log_start":"338'2282","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4422,"log_dups_size":0,"ondisk_log_size":4422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":439353344,"num_objects":6704,"num_object_clones":0,"num_object_copies":20112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":6704,"num_objects_unfound":0,"num_objects_dirty":6704,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6704,"num_write_kb":429056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,4],"avail_no_missing":["1","0","4"],"object_location_counts":[{"shards":"0,1,4","objects":6704}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"273'20","reported_seq":597,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.335347+0000","last_change":"2024-09-20T19:03:14.972618+0000","last_active":"2024-09-20T19:03:35.335347+0000","last_peered":"2024-09-20T19:03:35.335347+0000","last_clean":"2024-09-20T19:03:35.335347+0000","last_became_active":"2024-09-20T19:03:14.950217+0000","last_became_peered":"2024-09-20T19:03:14.950217+0000","last_unstale":"2024-09-20T19:03:35.335347+0000","last_undegraded":"2024-09-20T19:03:35.335347+0000","last_fullsized":"2024-09-20T19:03:35.335347+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'20","last_scrub_stamp":"2024-09-20T19:03:04.282800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:04.282800+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:21.780218+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":663,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.335727+0000","last_change":"2024-09-20T19:03:14.955780+0000","last_active":"2024-09-20T19:03:35.335727+0000","last_peered":"2024-09-20T19:03:35.335727+0000","last_clean":"2024-09-20T19:03:35.335727+0000","last_became_active":"2024-09-20T19:03:14.953433+0000","last_became_peered":"2024-09-20T19:03:14.953433+0000","last_unstale":"2024-09-20T19:03:35.335727+0000","last_undegraded":"2024-09-20T19:03:35.335727+0000","last_fullsized":"2024-09-20T19:03:35.335727+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:34.985163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:34.985163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:56.492382+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":823,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.326495+0000","last_change":"2024-09-20T19:02:02.813926+0000","last_active":"2024-09-20T19:03:35.326495+0000","last_peered":"2024-09-20T19:03:35.326495+0000","last_clean":"2024-09-20T19:03:35.326495+0000","last_became_active":"2024-09-20T18:49:49.957005+0000","last_became_peered":"2024-09-20T18:49:49.957005+0000","last_unstale":"2024-09-20T19:03:35.326495+0000","last_undegraded":"2024-09-20T19:03:35.326495+0000","last_fullsized":"2024-09-20T19:03:35.326495+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T19:02:02.813865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:02.813865+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":491,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.333903+0000","last_change":"2024-09-20T19:03:14.958483+0000","last_active":"2024-09-20T19:03:35.333903+0000","last_peered":"2024-09-20T19:03:35.333903+0000","last_clean":"2024-09-20T19:03:35.333903+0000","last_became_active":"2024-09-20T19:03:14.958200+0000","last_became_peered":"2024-09-20T19:03:14.958200+0000","last_unstale":"2024-09-20T19:03:35.333903+0000","last_undegraded":"2024-09-20T19:03:35.333903+0000","last_fullsized":"2024-09-20T19:03:35.333903+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:11.207994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:03:11.207994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:17.068313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.5","version":"358'6600","reported_seq":6677,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:36.402462+0000","last_change":"2024-09-20T19:02:11.828434+0000","last_active":"2024-09-20T19:03:36.402462+0000","last_peered":"2024-09-20T19:03:36.402462+0000","last_clean":"2024-09-20T19:03:36.402462+0000","last_became_active":"2024-09-20T19:02:11.828162+0000","last_became_peered":"2024-09-20T19:02:11.828162+0000","last_unstale":"2024-09-20T19:03:36.402462+0000","last_undegraded":"2024-09-20T19:03:36.402462+0000","last_fullsized":"2024-09-20T19:03:36.402462+0000","mapping_epoch":325,"log_start":"332'968","ondisk_log_start":"332'968","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":5632,"log_dups_size":0,"ondisk_log_size":5632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":432537600,"num_objects":6600,"num_object_clones":0,"num_object_copies":19800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6600,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6600,"num_write_kb":422400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"274'15","reported_seq":695,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.323746+0000","last_change":"2024-09-20T19:03:14.963710+0000","last_active":"2024-09-20T19:03:35.323746+0000","last_peered":"2024-09-20T19:03:35.323746+0000","last_clean":"2024-09-20T19:03:35.323746+0000","last_became_active":"2024-09-20T19:03:14.950377+0000","last_became_peered":"2024-09-20T19:03:14.950377+0000","last_unstale":"2024-09-20T19:03:35.323746+0000","last_undegraded":"2024-09-20T19:03:35.323746+0000","last_fullsized":"2024-09-20T19:03:35.323746+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'15","last_scrub_stamp":"2024-09-20T19:01:27.802100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:27.802100+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":189,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.323924+0000","last_change":"2024-09-20T19:03:13.479475+0000","last_active":"2024-09-20T19:03:35.323924+0000","last_peered":"2024-09-20T19:03:35.323924+0000","last_clean":"2024-09-20T19:03:35.323924+0000","last_became_active":"2024-09-20T19:00:01.190875+0000","last_became_peered":"2024-09-20T19:00:01.190875+0000","last_unstale":"2024-09-20T19:03:35.323924+0000","last_undegraded":"2024-09-20T19:03:35.323924+0000","last_fullsized":"2024-09-20T19:03:35.323924+0000","mapping_epoch":277,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":278,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:13.479412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:03:13.479412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:15.835045+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":658,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.323657+0000","last_change":"2024-09-20T19:03:14.942108+0000","last_active":"2024-09-20T19:03:35.323657+0000","last_peered":"2024-09-20T19:03:35.323657+0000","last_clean":"2024-09-20T19:03:35.323657+0000","last_became_active":"2024-09-20T19:03:14.941742+0000","last_became_peered":"2024-09-20T19:03:14.941742+0000","last_unstale":"2024-09-20T19:03:35.323657+0000","last_undegraded":"2024-09-20T19:03:35.323657+0000","last_fullsized":"2024-09-20T19:03:35.323657+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:24.938980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:24.938980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:50.054784+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":634,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.326004+0000","last_change":"2024-09-20T19:02:31.120809+0000","last_active":"2024-09-20T19:03:35.326004+0000","last_peered":"2024-09-20T19:03:35.326004+0000","last_clean":"2024-09-20T19:03:35.326004+0000","last_became_active":"2024-09-20T18:49:51.964098+0000","last_became_peered":"2024-09-20T18:49:51.964098+0000","last_unstale":"2024-09-20T19:03:35.326004+0000","last_undegraded":"2024-09-20T19:03:35.326004+0000","last_fullsized":"2024-09-20T19:03:35.326004+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:31.120623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:31.120623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:41.294006+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"358'6609","reported_seq":6693,"reported_epoch":358,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-20T19:03:37.413663+0000","last_change":"2024-09-20T19:03:16.178808+0000","last_active":"2024-09-20T19:03:37.413663+0000","last_peered":"2024-09-20T19:03:37.413663+0000","last_clean":"2024-09-20T19:03:13.908942+0000","last_became_active":"2024-09-20T19:03:16.166786+0000","last_became_peered":"2024-09-20T19:03:16.166786+0000","last_unstale":"2024-09-20T19:03:37.413663+0000","last_undegraded":"2024-09-20T19:03:37.413663+0000","last_fullsized":"2024-09-20T19:03:37.413663+0000","mapping_epoch":351,"log_start":"338'2297","ondisk_log_start":"338'2297","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4312,"log_dups_size":0,"ondisk_log_size":4312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":433127424,"num_objects":6609,"num_object_clones":0,"num_object_copies":19827,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":6609,"num_objects_unfound":0,"num_objects_dirty":6609,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6609,"num_write_kb":422976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,4],"avail_no_missing":["0","3","4"],"object_location_counts":[{"shards":"0,3,4","objects":6609}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"274'35","reported_seq":268,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.334784+0000","last_change":"2024-09-20T19:03:09.841596+0000","last_active":"2024-09-20T19:03:35.334784+0000","last_peered":"2024-09-20T19:03:35.334784+0000","last_clean":"2024-09-20T19:03:35.334784+0000","last_became_active":"2024-09-20T18:59:05.087391+0000","last_became_peered":"2024-09-20T18:59:05.087391+0000","last_unstale":"2024-09-20T19:03:35.334784+0000","last_undegraded":"2024-09-20T19:03:35.334784+0000","last_fullsized":"2024-09-20T19:03:35.334784+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'35","last_scrub_stamp":"2024-09-20T19:03:09.841361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:09.841361+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:33.281410+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":660,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.329043+0000","last_change":"2024-09-20T19:02:05.440745+0000","last_active":"2024-09-20T19:03:35.329043+0000","last_peered":"2024-09-20T19:03:35.329043+0000","last_clean":"2024-09-20T19:03:35.329043+0000","last_became_active":"2024-09-20T18:59:05.086670+0000","last_became_peered":"2024-09-20T18:59:05.086670+0000","last_unstale":"2024-09-20T19:03:35.329043+0000","last_undegraded":"2024-09-20T19:03:35.329043+0000","last_fullsized":"2024-09-20T19:03:35.329043+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:05.440677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:05.440677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":648,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.325907+0000","last_change":"2024-09-20T19:02:42.745448+0000","last_active":"2024-09-20T19:03:35.325907+0000","last_peered":"2024-09-20T19:03:35.325907+0000","last_clean":"2024-09-20T19:03:35.325907+0000","last_became_active":"2024-09-20T18:49:49.957558+0000","last_became_peered":"2024-09-20T18:49:49.957558+0000","last_unstale":"2024-09-20T19:03:35.325907+0000","last_undegraded":"2024-09-20T19:03:35.325907+0000","last_fullsized":"2024-09-20T19:03:35.325907+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:42.745372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:42.745372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:49.093207+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":626,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.327664+0000","last_change":"2024-09-20T18:53:47.370152+0000","last_active":"2024-09-20T19:03:35.327664+0000","last_peered":"2024-09-20T19:03:35.327664+0000","last_clean":"2024-09-20T19:03:35.327664+0000","last_became_active":"2024-09-20T18:49:51.970798+0000","last_became_peered":"2024-09-20T18:49:51.970798+0000","last_unstale":"2024-09-20T19:03:35.327664+0000","last_undegraded":"2024-09-20T19:03:35.327664+0000","last_fullsized":"2024-09-20T19:03:35.327664+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:47.370108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:47.370108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.7","version":"358'6726","reported_seq":6803,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:36.183946+0000","last_change":"2024-09-20T19:02:11.827681+0000","last_active":"2024-09-20T19:03:36.183946+0000","last_peered":"2024-09-20T19:03:36.183946+0000","last_clean":"2024-09-20T19:03:36.183946+0000","last_became_active":"2024-09-20T19:02:11.827360+0000","last_became_peered":"2024-09-20T19:02:11.827360+0000","last_unstale":"2024-09-20T19:03:36.183946+0000","last_undegraded":"2024-09-20T19:03:36.183946+0000","last_fullsized":"2024-09-20T19:03:36.183946+0000","mapping_epoch":325,"log_start":"336'1807","ondisk_log_start":"336'1807","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4919,"log_dups_size":0,"ondisk_log_size":4919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":440795136,"num_objects":6726,"num_object_clones":0,"num_object_copies":20178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6726,"num_write_kb":430464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"273'18","reported_seq":553,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.331118+0000","last_change":"2024-09-20T19:03:14.981172+0000","last_active":"2024-09-20T19:03:35.331118+0000","last_peered":"2024-09-20T19:03:35.331118+0000","last_clean":"2024-09-20T19:03:35.331118+0000","last_became_active":"2024-09-20T19:03:14.956290+0000","last_became_peered":"2024-09-20T19:03:14.956290+0000","last_unstale":"2024-09-20T19:03:35.331118+0000","last_undegraded":"2024-09-20T19:03:35.331118+0000","last_fullsized":"2024-09-20T19:03:35.331118+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'18","last_scrub_stamp":"2024-09-20T19:03:13.201592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:13.201592+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:37.912706+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":664,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.328556+0000","last_change":"2024-09-20T19:03:14.961785+0000","last_active":"2024-09-20T19:03:35.328556+0000","last_peered":"2024-09-20T19:03:35.328556+0000","last_clean":"2024-09-20T19:03:35.328556+0000","last_became_active":"2024-09-20T19:03:14.961495+0000","last_became_peered":"2024-09-20T19:03:14.961495+0000","last_unstale":"2024-09-20T19:03:35.328556+0000","last_undegraded":"2024-09-20T19:03:35.328556+0000","last_fullsized":"2024-09-20T19:03:35.328556+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:00:34.940850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:00:34.940850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":640,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.324810+0000","last_change":"2024-09-20T19:01:53.734774+0000","last_active":"2024-09-20T19:03:35.324810+0000","last_peered":"2024-09-20T19:03:35.324810+0000","last_clean":"2024-09-20T19:03:35.324810+0000","last_became_active":"2024-09-20T18:49:49.954944+0000","last_became_peered":"2024-09-20T18:49:49.954944+0000","last_unstale":"2024-09-20T19:03:35.324810+0000","last_undegraded":"2024-09-20T19:03:35.324810+0000","last_fullsized":"2024-09-20T19:03:35.324810+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:53.734726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:53.734726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":633,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.335196+0000","last_change":"2024-09-20T19:02:35.968178+0000","last_active":"2024-09-20T19:03:35.335196+0000","last_peered":"2024-09-20T19:03:35.335196+0000","last_clean":"2024-09-20T19:03:35.335196+0000","last_became_active":"2024-09-20T18:49:51.972970+0000","last_became_peered":"2024-09-20T18:49:51.972970+0000","last_unstale":"2024-09-20T19:03:35.335196+0000","last_undegraded":"2024-09-20T19:03:35.335196+0000","last_fullsized":"2024-09-20T19:03:35.335196+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:35.968052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:35.968052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:53.629726+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.6","version":"358'6753","reported_seq":7724,"reported_epoch":358,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-20T19:03:36.432068+0000","last_change":"2024-09-20T19:03:16.388415+0000","last_active":"2024-09-20T19:03:36.432068+0000","last_peered":"2024-09-20T19:03:36.432068+0000","last_clean":"2024-09-20T19:03:13.927169+0000","last_became_active":"2024-09-20T19:03:16.282080+0000","last_became_peered":"2024-09-20T19:03:16.282080+0000","last_unstale":"2024-09-20T19:03:36.432068+0000","last_undegraded":"2024-09-20T19:03:36.432068+0000","last_fullsized":"2024-09-20T19:03:15.944009+0000","mapping_epoch":351,"log_start":"336'1662","ondisk_log_start":"336'1662","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":5091,"log_dups_size":0,"ondisk_log_size":5091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":442564608,"num_objects":6753,"num_object_clones":0,"num_object_copies":20259,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":5363,"num_objects_unfound":0,"num_objects_dirty":6753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6753,"num_write_kb":432192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":445,"num_bytes_recovered":29163520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0],"avail_no_missing":["5","0","4"],"object_location_counts":[{"shards":"0,4,5","objects":5363},{"shards":"0,5","objects":1390}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"273'27","reported_seq":738,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.331295+0000","last_change":"2024-09-20T19:02:39.709344+0000","last_active":"2024-09-20T19:03:35.331295+0000","last_peered":"2024-09-20T19:03:35.331295+0000","last_clean":"2024-09-20T19:03:35.331295+0000","last_became_active":"2024-09-20T18:59:05.083726+0000","last_became_peered":"2024-09-20T18:59:05.083726+0000","last_unstale":"2024-09-20T19:03:35.331295+0000","last_undegraded":"2024-09-20T19:03:35.331295+0000","last_fullsized":"2024-09-20T19:03:35.331295+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'27","last_scrub_stamp":"2024-09-20T19:02:39.709269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:39.709269+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:44.315182+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":690,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.329482+0000","last_change":"2024-09-20T19:03:14.986178+0000","last_active":"2024-09-20T19:03:35.329482+0000","last_peered":"2024-09-20T19:03:35.329482+0000","last_clean":"2024-09-20T19:03:35.329482+0000","last_became_active":"2024-09-20T19:03:14.957738+0000","last_became_peered":"2024-09-20T19:03:14.957738+0000","last_unstale":"2024-09-20T19:03:35.329482+0000","last_undegraded":"2024-09-20T19:03:35.329482+0000","last_fullsized":"2024-09-20T19:03:35.329482+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-20T18:58:57.563058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:57.563058+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"352'13","reported_seq":843,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.328735+0000","last_change":"2024-09-20T19:03:14.970339+0000","last_active":"2024-09-20T19:03:35.328735+0000","last_peered":"2024-09-20T19:03:35.328735+0000","last_clean":"2024-09-20T19:03:35.328735+0000","last_became_active":"2024-09-20T19:03:14.948540+0000","last_became_peered":"2024-09-20T19:03:14.948540+0000","last_unstale":"2024-09-20T19:03:35.328735+0000","last_undegraded":"2024-09-20T19:03:35.328735+0000","last_fullsized":"2024-09-20T19:03:35.328735+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'9","last_scrub_stamp":"2024-09-20T18:58:27.521051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:27.521051+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":655,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.331398+0000","last_change":"2024-09-20T19:03:06.888620+0000","last_active":"2024-09-20T19:03:35.331398+0000","last_peered":"2024-09-20T19:03:35.331398+0000","last_clean":"2024-09-20T19:03:35.331398+0000","last_became_active":"2024-09-20T18:59:05.087005+0000","last_became_peered":"2024-09-20T18:59:05.087005+0000","last_unstale":"2024-09-20T19:03:35.331398+0000","last_undegraded":"2024-09-20T19:03:35.331398+0000","last_fullsized":"2024-09-20T19:03:35.331398+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:06.888563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:03:06.888563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:07.838057+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":215,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.340493+0000","last_change":"2024-09-20T19:01:16.542398+0000","last_active":"2024-09-20T19:03:35.340493+0000","last_peered":"2024-09-20T19:03:35.340493+0000","last_clean":"2024-09-20T19:03:35.340493+0000","last_became_active":"2024-09-20T18:59:05.085427+0000","last_became_peered":"2024-09-20T18:59:05.085427+0000","last_unstale":"2024-09-20T19:03:35.340493+0000","last_undegraded":"2024-09-20T19:03:35.340493+0000","last_fullsized":"2024-09-20T19:03:35.340493+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:16.542335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:16.542335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:47.734953+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"273'12","reported_seq":658,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.334176+0000","last_change":"2024-09-20T19:01:12.385223+0000","last_active":"2024-09-20T19:03:35.334176+0000","last_peered":"2024-09-20T19:03:35.334176+0000","last_clean":"2024-09-20T19:03:35.334176+0000","last_became_active":"2024-09-20T18:49:48.500662+0000","last_became_peered":"2024-09-20T18:49:48.500662+0000","last_unstale":"2024-09-20T19:03:35.334176+0000","last_undegraded":"2024-09-20T19:03:35.334176+0000","last_fullsized":"2024-09-20T19:03:35.334176+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'12","last_scrub_stamp":"2024-09-20T19:01:12.385152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:12.385152+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:57.005041+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":636,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.341587+0000","last_change":"2024-09-20T19:01:14.466096+0000","last_active":"2024-09-20T19:03:35.341587+0000","last_peered":"2024-09-20T19:03:35.341587+0000","last_clean":"2024-09-20T19:03:35.341587+0000","last_became_active":"2024-09-20T18:49:49.956749+0000","last_became_peered":"2024-09-20T18:49:49.956749+0000","last_unstale":"2024-09-20T19:03:35.341587+0000","last_undegraded":"2024-09-20T19:03:35.341587+0000","last_fullsized":"2024-09-20T19:03:35.341587+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:14.466042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:14.466042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":665,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.328737+0000","last_change":"2024-09-20T19:03:14.956967+0000","last_active":"2024-09-20T19:03:35.328737+0000","last_peered":"2024-09-20T19:03:35.328737+0000","last_clean":"2024-09-20T19:03:35.328737+0000","last_became_active":"2024-09-20T19:03:14.956216+0000","last_became_peered":"2024-09-20T19:03:14.956216+0000","last_unstale":"2024-09-20T19:03:35.328737+0000","last_undegraded":"2024-09-20T19:03:35.328737+0000","last_fullsized":"2024-09-20T19:03:35.328737+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:03.509903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:03.509903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,5],"acting":[7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"273'15","reported_seq":658,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.329438+0000","last_change":"2024-09-20T19:01:15.875779+0000","last_active":"2024-09-20T19:03:35.329438+0000","last_peered":"2024-09-20T19:03:35.329438+0000","last_clean":"2024-09-20T19:03:35.329438+0000","last_became_active":"2024-09-20T18:49:48.500803+0000","last_became_peered":"2024-09-20T18:49:48.500803+0000","last_unstale":"2024-09-20T19:03:35.329438+0000","last_undegraded":"2024-09-20T19:03:35.329438+0000","last_fullsized":"2024-09-20T19:03:35.329438+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:01:15.875693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:15.875693+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:49.127502+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":669,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.328894+0000","last_change":"2024-09-20T19:03:14.958361+0000","last_active":"2024-09-20T19:03:35.328894+0000","last_peered":"2024-09-20T19:03:35.328894+0000","last_clean":"2024-09-20T19:03:35.328894+0000","last_became_active":"2024-09-20T19:03:14.958013+0000","last_became_peered":"2024-09-20T19:03:14.958013+0000","last_unstale":"2024-09-20T19:03:35.328894+0000","last_undegraded":"2024-09-20T19:03:35.328894+0000","last_fullsized":"2024-09-20T19:03:35.328894+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:59.589891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:59.589891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":636,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.334020+0000","last_change":"2024-09-20T19:01:06.482291+0000","last_active":"2024-09-20T19:03:35.334020+0000","last_peered":"2024-09-20T19:03:35.334020+0000","last_clean":"2024-09-20T19:03:35.334020+0000","last_became_active":"2024-09-20T18:49:49.961947+0000","last_became_peered":"2024-09-20T18:49:49.961947+0000","last_unstale":"2024-09-20T19:03:35.334020+0000","last_undegraded":"2024-09-20T19:03:35.334020+0000","last_fullsized":"2024-09-20T19:03:35.334020+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:06.482212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:06.482212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:58.987808+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"66'1","reported_seq":674,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.324003+0000","last_change":"2024-09-20T19:03:14.960172+0000","last_active":"2024-09-20T19:03:35.324003+0000","last_peered":"2024-09-20T19:03:35.324003+0000","last_clean":"2024-09-20T19:03:35.324003+0000","last_became_active":"2024-09-20T19:03:14.945523+0000","last_became_peered":"2024-09-20T19:03:14.945523+0000","last_unstale":"2024-09-20T19:03:35.324003+0000","last_undegraded":"2024-09-20T19:03:35.324003+0000","last_fullsized":"2024-09-20T19:03:35.324003+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T19:03:12.499684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:03:12.499684+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:30.881550+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,0],"acting":[2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"274'15","reported_seq":657,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.333107+0000","last_change":"2024-09-20T19:01:36.532486+0000","last_active":"2024-09-20T19:03:35.333107+0000","last_peered":"2024-09-20T19:03:35.333107+0000","last_clean":"2024-09-20T19:03:35.333107+0000","last_became_active":"2024-09-20T18:49:48.501011+0000","last_became_peered":"2024-09-20T18:49:48.501011+0000","last_unstale":"2024-09-20T19:03:35.333107+0000","last_undegraded":"2024-09-20T19:03:35.333107+0000","last_fullsized":"2024-09-20T19:03:35.333107+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'15","last_scrub_stamp":"2024-09-20T19:01:36.532419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:36.532419+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:58.862030+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":645,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.326537+0000","last_change":"2024-09-20T19:01:30.864866+0000","last_active":"2024-09-20T19:03:35.326537+0000","last_peered":"2024-09-20T19:03:35.326537+0000","last_clean":"2024-09-20T19:03:35.326537+0000","last_became_active":"2024-09-20T18:49:45.938511+0000","last_became_peered":"2024-09-20T18:49:45.938511+0000","last_unstale":"2024-09-20T19:03:35.326537+0000","last_undegraded":"2024-09-20T19:03:35.326537+0000","last_fullsized":"2024-09-20T19:03:35.326537+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:30.864794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:30.864794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:01.115401+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":811,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.335181+0000","last_change":"2024-09-20T19:00:34.485855+0000","last_active":"2024-09-20T19:03:35.335181+0000","last_peered":"2024-09-20T19:03:35.335181+0000","last_clean":"2024-09-20T19:03:35.335181+0000","last_became_active":"2024-09-20T18:49:49.960550+0000","last_became_peered":"2024-09-20T18:49:49.960550+0000","last_unstale":"2024-09-20T19:03:35.335181+0000","last_undegraded":"2024-09-20T19:03:35.335181+0000","last_fullsized":"2024-09-20T19:03:35.335181+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T19:00:34.485624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:00:34.485624+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:54.909280+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":499,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.324428+0000","last_change":"2024-09-20T19:03:14.957109+0000","last_active":"2024-09-20T19:03:35.324428+0000","last_peered":"2024-09-20T19:03:35.324428+0000","last_clean":"2024-09-20T19:03:35.324428+0000","last_became_active":"2024-09-20T19:03:14.956518+0000","last_became_peered":"2024-09-20T19:03:14.956518+0000","last_unstale":"2024-09-20T19:03:35.324428+0000","last_undegraded":"2024-09-20T19:03:35.324428+0000","last_fullsized":"2024-09-20T19:03:35.324428+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:59.497810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:59:59.497810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:24.357379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"274'20","reported_seq":673,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.335271+0000","last_change":"2024-09-20T18:57:20.701592+0000","last_active":"2024-09-20T19:03:35.335271+0000","last_peered":"2024-09-20T19:03:35.335271+0000","last_clean":"2024-09-20T19:03:35.335271+0000","last_became_active":"2024-09-20T18:49:58.997494+0000","last_became_peered":"2024-09-20T18:49:58.997494+0000","last_unstale":"2024-09-20T19:03:35.335271+0000","last_undegraded":"2024-09-20T19:03:35.335271+0000","last_fullsized":"2024-09-20T19:03:35.335271+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:57:20.701543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:20.701543+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":655,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.328175+0000","last_change":"2024-09-20T19:03:14.953145+0000","last_active":"2024-09-20T19:03:35.328175+0000","last_peered":"2024-09-20T19:03:35.328175+0000","last_clean":"2024-09-20T19:03:35.328175+0000","last_became_active":"2024-09-20T19:03:14.952814+0000","last_became_peered":"2024-09-20T19:03:14.952814+0000","last_unstale":"2024-09-20T19:03:35.328175+0000","last_undegraded":"2024-09-20T19:03:35.328175+0000","last_fullsized":"2024-09-20T19:03:35.328175+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.632380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.632380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":811,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.332698+0000","last_change":"2024-09-20T19:01:10.601374+0000","last_active":"2024-09-20T19:03:35.332698+0000","last_peered":"2024-09-20T19:03:35.332698+0000","last_clean":"2024-09-20T19:03:35.332698+0000","last_became_active":"2024-09-20T18:49:49.962243+0000","last_became_peered":"2024-09-20T18:49:49.962243+0000","last_unstale":"2024-09-20T19:03:35.332698+0000","last_undegraded":"2024-09-20T19:03:35.332698+0000","last_fullsized":"2024-09-20T19:03:35.332698+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T19:01:10.601300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:10.601300+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:46.785830+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":641,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.328950+0000","last_change":"2024-09-20T19:01:03.821977+0000","last_active":"2024-09-20T19:03:35.328950+0000","last_peered":"2024-09-20T19:03:35.328950+0000","last_clean":"2024-09-20T19:03:35.328950+0000","last_became_active":"2024-09-20T18:49:51.969093+0000","last_became_peered":"2024-09-20T18:49:51.969093+0000","last_unstale":"2024-09-20T19:03:35.328950+0000","last_undegraded":"2024-09-20T19:03:35.328950+0000","last_fullsized":"2024-09-20T19:03:35.328950+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:03.821908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:03.821908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:47.087429+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"273'19","reported_seq":598,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.335004+0000","last_change":"2024-09-20T19:03:14.990890+0000","last_active":"2024-09-20T19:03:35.335004+0000","last_peered":"2024-09-20T19:03:35.335004+0000","last_clean":"2024-09-20T19:03:35.335004+0000","last_became_active":"2024-09-20T19:03:14.965320+0000","last_became_peered":"2024-09-20T19:03:14.965320+0000","last_unstale":"2024-09-20T19:03:35.335004+0000","last_undegraded":"2024-09-20T19:03:35.335004+0000","last_fullsized":"2024-09-20T19:03:35.335004+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:57:50.899514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:50.899514+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":486,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.329670+0000","last_change":"2024-09-20T19:03:14.954849+0000","last_active":"2024-09-20T19:03:35.329670+0000","last_peered":"2024-09-20T19:03:35.329670+0000","last_clean":"2024-09-20T19:03:35.329670+0000","last_became_active":"2024-09-20T19:03:14.952258+0000","last_became_peered":"2024-09-20T19:03:14.952258+0000","last_unstale":"2024-09-20T19:03:35.329670+0000","last_undegraded":"2024-09-20T19:03:35.329670+0000","last_fullsized":"2024-09-20T19:03:35.329670+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:06.310027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:59:06.310027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":636,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.332909+0000","last_change":"2024-09-20T19:02:38.906352+0000","last_active":"2024-09-20T19:03:35.332909+0000","last_peered":"2024-09-20T19:03:35.332909+0000","last_clean":"2024-09-20T19:03:35.332909+0000","last_became_active":"2024-09-20T18:49:49.957067+0000","last_became_peered":"2024-09-20T18:49:49.957067+0000","last_unstale":"2024-09-20T19:03:35.332909+0000","last_undegraded":"2024-09-20T19:03:35.332909+0000","last_fullsized":"2024-09-20T19:03:35.332909+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:38.906161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:38.906161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:41.340426+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":616,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.333946+0000","last_change":"2024-09-20T18:57:16.789412+0000","last_active":"2024-09-20T19:03:35.333946+0000","last_peered":"2024-09-20T19:03:35.333946+0000","last_clean":"2024-09-20T19:03:35.333946+0000","last_became_active":"2024-09-20T18:49:51.966915+0000","last_became_peered":"2024-09-20T18:49:51.966915+0000","last_unstale":"2024-09-20T19:03:35.333946+0000","last_undegraded":"2024-09-20T19:03:35.333946+0000","last_fullsized":"2024-09-20T19:03:35.333946+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:16.789095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:16.789095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"273'15","reported_seq":707,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.333112+0000","last_change":"2024-09-20T19:02:30.816061+0000","last_active":"2024-09-20T19:03:35.333112+0000","last_peered":"2024-09-20T19:03:35.333112+0000","last_clean":"2024-09-20T19:03:35.333112+0000","last_became_active":"2024-09-20T18:59:05.085230+0000","last_became_peered":"2024-09-20T18:59:05.085230+0000","last_unstale":"2024-09-20T19:03:35.333112+0000","last_undegraded":"2024-09-20T19:03:35.333112+0000","last_fullsized":"2024-09-20T19:03:35.333112+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:02:30.815747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:30.815747+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:44.961926+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":655,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.328916+0000","last_change":"2024-09-20T19:03:14.957280+0000","last_active":"2024-09-20T19:03:35.328916+0000","last_peered":"2024-09-20T19:03:35.328916+0000","last_clean":"2024-09-20T19:03:35.328916+0000","last_became_active":"2024-09-20T19:03:14.956668+0000","last_became_peered":"2024-09-20T19:03:14.956668+0000","last_unstale":"2024-09-20T19:03:35.328916+0000","last_undegraded":"2024-09-20T19:03:35.328916+0000","last_fullsized":"2024-09-20T19:03:35.328916+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:07.408197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:03:07.408197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:16.413210+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":636,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.335334+0000","last_change":"2024-09-20T19:01:03.507072+0000","last_active":"2024-09-20T19:03:35.335334+0000","last_peered":"2024-09-20T19:03:35.335334+0000","last_clean":"2024-09-20T19:03:35.335334+0000","last_became_active":"2024-09-20T18:49:49.962523+0000","last_became_peered":"2024-09-20T18:49:49.962523+0000","last_unstale":"2024-09-20T19:03:35.335334+0000","last_undegraded":"2024-09-20T19:03:35.335334+0000","last_fullsized":"2024-09-20T19:03:35.335334+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:03.507016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:03.507016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:55.956672+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":633,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.324525+0000","last_change":"2024-09-20T19:01:11.847792+0000","last_active":"2024-09-20T19:03:35.324525+0000","last_peered":"2024-09-20T19:03:35.324525+0000","last_clean":"2024-09-20T19:03:35.324525+0000","last_became_active":"2024-09-20T18:49:51.967313+0000","last_became_peered":"2024-09-20T18:49:51.967313+0000","last_unstale":"2024-09-20T19:03:35.324525+0000","last_undegraded":"2024-09-20T19:03:35.324525+0000","last_fullsized":"2024-09-20T19:03:35.324525+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:11.847716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:11.847716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"274'19","reported_seq":709,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.334553+0000","last_change":"2024-09-20T19:01:05.495603+0000","last_active":"2024-09-20T19:03:35.334553+0000","last_peered":"2024-09-20T19:03:35.334553+0000","last_clean":"2024-09-20T19:03:35.334553+0000","last_became_active":"2024-09-20T18:59:05.075390+0000","last_became_peered":"2024-09-20T18:59:05.075390+0000","last_unstale":"2024-09-20T19:03:35.334553+0000","last_undegraded":"2024-09-20T19:03:35.334553+0000","last_fullsized":"2024-09-20T19:03:35.334553+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'19","last_scrub_stamp":"2024-09-20T19:01:05.495478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:05.495478+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:02.014342+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":652,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.341731+0000","last_change":"2024-09-20T19:01:47.521858+0000","last_active":"2024-09-20T19:03:35.341731+0000","last_peered":"2024-09-20T19:03:35.341731+0000","last_clean":"2024-09-20T19:03:35.341731+0000","last_became_active":"2024-09-20T18:49:45.942105+0000","last_became_peered":"2024-09-20T18:49:45.942105+0000","last_unstale":"2024-09-20T19:03:35.341731+0000","last_undegraded":"2024-09-20T19:03:35.341731+0000","last_fullsized":"2024-09-20T19:03:35.341731+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:47.521789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:47.521789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":669,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.327227+0000","last_change":"2024-09-20T19:03:14.957124+0000","last_active":"2024-09-20T19:03:35.327227+0000","last_peered":"2024-09-20T19:03:35.327227+0000","last_clean":"2024-09-20T19:03:35.327227+0000","last_became_active":"2024-09-20T19:03:14.956811+0000","last_became_peered":"2024-09-20T19:03:14.956811+0000","last_unstale":"2024-09-20T19:03:35.327227+0000","last_undegraded":"2024-09-20T19:03:35.327227+0000","last_fullsized":"2024-09-20T19:03:35.327227+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:27.267254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:27.267254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:53.812193+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":638,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.335805+0000","last_change":"2024-09-20T19:02:38.950217+0000","last_active":"2024-09-20T19:03:35.335805+0000","last_peered":"2024-09-20T19:03:35.335805+0000","last_clean":"2024-09-20T19:03:35.335805+0000","last_became_active":"2024-09-20T18:59:05.081179+0000","last_became_peered":"2024-09-20T18:59:05.081179+0000","last_unstale":"2024-09-20T19:03:35.335805+0000","last_undegraded":"2024-09-20T19:03:35.335805+0000","last_fullsized":"2024-09-20T19:03:35.335805+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:38.950017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:38.950017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:02.988187+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"273'8","reported_seq":678,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.333456+0000","last_change":"2024-09-20T19:00:33.504967+0000","last_active":"2024-09-20T19:03:35.333456+0000","last_peered":"2024-09-20T19:03:35.333456+0000","last_clean":"2024-09-20T19:03:35.333456+0000","last_became_active":"2024-09-20T18:59:05.076365+0000","last_became_peered":"2024-09-20T18:59:05.076365+0000","last_unstale":"2024-09-20T19:03:35.333456+0000","last_undegraded":"2024-09-20T19:03:35.333456+0000","last_fullsized":"2024-09-20T19:03:35.333456+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'8","last_scrub_stamp":"2024-09-20T19:00:33.504891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:00:33.504891+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:50.863517+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":653,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.326538+0000","last_change":"2024-09-20T19:02:39.184249+0000","last_active":"2024-09-20T19:03:35.326538+0000","last_peered":"2024-09-20T19:03:35.326538+0000","last_clean":"2024-09-20T19:03:35.326538+0000","last_became_active":"2024-09-20T18:49:45.940988+0000","last_became_peered":"2024-09-20T18:49:45.940988+0000","last_unstale":"2024-09-20T19:03:35.326538+0000","last_undegraded":"2024-09-20T19:03:35.326538+0000","last_fullsized":"2024-09-20T19:03:35.326538+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:39.184150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:39.184150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:54.134855+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":233,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.334688+0000","last_change":"2024-09-20T19:03:14.949099+0000","last_active":"2024-09-20T19:03:35.334688+0000","last_peered":"2024-09-20T19:03:35.334688+0000","last_clean":"2024-09-20T19:03:35.334688+0000","last_became_active":"2024-09-20T19:03:14.948648+0000","last_became_peered":"2024-09-20T19:03:14.948648+0000","last_unstale":"2024-09-20T19:03:35.334688+0000","last_undegraded":"2024-09-20T19:03:35.334688+0000","last_fullsized":"2024-09-20T19:03:35.334688+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:41.497115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:41.497115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":643,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.327341+0000","last_change":"2024-09-20T19:03:14.949524+0000","last_active":"2024-09-20T19:03:35.327341+0000","last_peered":"2024-09-20T19:03:35.327341+0000","last_clean":"2024-09-20T19:03:35.327341+0000","last_became_active":"2024-09-20T19:03:14.949200+0000","last_became_peered":"2024-09-20T19:03:14.949200+0000","last_unstale":"2024-09-20T19:03:35.327341+0000","last_undegraded":"2024-09-20T19:03:35.327341+0000","last_fullsized":"2024-09-20T19:03:35.327341+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:29.277916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:29.277916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:55.810061+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":626,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.331553+0000","last_change":"2024-09-20T19:03:14.956682+0000","last_active":"2024-09-20T19:03:35.331553+0000","last_peered":"2024-09-20T19:03:35.331553+0000","last_clean":"2024-09-20T19:03:35.331553+0000","last_became_active":"2024-09-20T19:03:14.956082+0000","last_became_peered":"2024-09-20T19:03:14.956082+0000","last_unstale":"2024-09-20T19:03:35.331553+0000","last_undegraded":"2024-09-20T19:03:35.331553+0000","last_fullsized":"2024-09-20T19:03:35.331553+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:10.970779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:10.970779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":636,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.333681+0000","last_change":"2024-09-20T18:58:36.077005+0000","last_active":"2024-09-20T19:03:35.333681+0000","last_peered":"2024-09-20T19:03:35.333681+0000","last_clean":"2024-09-20T19:03:35.333681+0000","last_became_active":"2024-09-20T18:49:45.937992+0000","last_became_peered":"2024-09-20T18:49:45.937992+0000","last_unstale":"2024-09-20T19:03:35.333681+0000","last_undegraded":"2024-09-20T19:03:35.333681+0000","last_fullsized":"2024-09-20T19:03:35.333681+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:36.076937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:36.076937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"274'19","reported_seq":240,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.340063+0000","last_change":"2024-09-20T19:01:15.544760+0000","last_active":"2024-09-20T19:03:35.340063+0000","last_peered":"2024-09-20T19:03:35.340063+0000","last_clean":"2024-09-20T19:03:35.340063+0000","last_became_active":"2024-09-20T18:59:05.089294+0000","last_became_peered":"2024-09-20T18:59:05.089294+0000","last_unstale":"2024-09-20T19:03:35.340063+0000","last_undegraded":"2024-09-20T19:03:35.340063+0000","last_fullsized":"2024-09-20T19:03:35.340063+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'19","last_scrub_stamp":"2024-09-20T19:01:15.544663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:15.544663+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":630,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-20T19:03:35.329112+0000","last_change":"2024-09-20T19:01:40.961449+0000","last_active":"2024-09-20T19:03:35.329112+0000","last_peered":"2024-09-20T19:03:35.329112+0000","last_clean":"2024-09-20T19:03:35.329112+0000","last_became_active":"2024-09-20T18:49:49.957582+0000","last_became_peered":"2024-09-20T18:49:49.957582+0000","last_unstale":"2024-09-20T19:03:35.329112+0000","last_undegraded":"2024-09-20T19:03:35.329112+0000","last_fullsized":"2024-09-20T19:03:35.329112+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:40.961363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:40.961363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:54.068818+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":6982598656,"num_objects":106546,"num_object_clones":0,"num_object_copies":319638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":42657,"num_objects_unfound":0,"num_objects_dirty":106546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106546,"num_write_kb":6818944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7504,"num_bytes_recovered":491782144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":21512716288,"data_stored":21512716288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78442,"ondisk_log_size":78442,"up":48,"acting":45,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1185,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1115,"num_read_kb":872,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":4076,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":471040,"data_stored":12580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":685,"ondisk_log_size":685,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2485,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":6452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":824,"num_read_kb":819,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266858,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2283084,"kb_used_data":1764604,"kb_used_omap":1,"kb_used_meta":518462,"kb_avail":91462068,"statfs":{"total":95995035648,"available":93657157632,"internally_reserved":0,"allocated":1806954496,"data_stored":1806539457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":530905546},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528497,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3362124,"kb_used_data":2979704,"kb_used_omap":1,"kb_used_meta":382398,"kb_avail":90383028,"statfs":{"total":95995035648,"available":92552220672,"internally_reserved":0,"allocated":3051216896,"data_stored":3050805731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":391576006},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,2,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757430,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2652936,"kb_used_data":2134452,"kb_used_omap":1,"kb_used_meta":518462,"kb_avail":91092216,"statfs":{"total":95995035648,"available":93278429184,"internally_reserved":0,"allocated":2185678848,"data_stored":2185271466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":530905541},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":0,"seq":1305670058012,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182594,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3084232,"kb_used_data":2457464,"kb_used_omap":1,"kb_used_meta":626750,"kb_avail":90660920,"statfs":{"total":95995035648,"available":92836782080,"internally_reserved":0,"allocated":2516443136,"data_stored":2516031258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":641792455},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411527,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3110144,"kb_used_data":2555564,"kb_used_omap":1,"kb_used_meta":554558,"kb_avail":90635008,"statfs":{"total":95995035648,"available":92810248192,"internally_reserved":0,"allocated":2616897536,"data_stored":2616498677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":567867845},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575053,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2126256,"kb_used_data":1575840,"kb_used_omap":1,"kb_used_meta":550398,"kb_avail":91618896,"statfs":{"total":95995035648,"available":93817749504,"internally_reserved":0,"allocated":1613660160,"data_stored":1613209545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":563608005},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738579,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4861136,"kb_used_data":4266304,"kb_used_omap":1,"kb_used_meta":594814,"kb_avail":88884016,"statfs":{"total":95995035648,"available":91017232384,"internally_reserved":0,"allocated":4368695296,"data_stored":4368276861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":609089989},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1185,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4373938176,"data_stored":4373938176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1636630528,"data_stored":1636630528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2618753024,"data_stored":2618753024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2515599360,"data_stored":2515599360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":3302162432,"data_stored":3302162432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2196373504,"data_stored":2196373504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3055353856,"data_stored":3055353856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1813905408,"data_stored":1813905408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T19:03:42.064 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd pool set .mgr pgp_num 1 2024-09-20T19:03:42.097 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:42.153 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:42.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:42.154 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:42.255 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:42.297 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:42.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:42.298 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:42.399 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:03:42.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:42.431 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:42.431 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:03:42.532 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:03:42.572 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:42.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:42.573 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:03:42.610 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:42 smithi083 ceph-mon[29421]: pgmap v691: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 6.5 GiB data, 20 GiB used, 605 GiB / 626 GiB avail; 639 B/s rd, 30 MiB/s wr, 477 op/s; 42657/320235 objects misplaced (13.321%); 14 MiB/s, 229 objects/s recovering 2024-09-20T19:03:42.610 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:42 smithi083 ceph-mon[29421]: osdmap e360: 8 total, 8 up, 7 in 2024-09-20T19:03:42.611 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:42 smithi083 ceph-mon[36260]: pgmap v691: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 6.5 GiB data, 20 GiB used, 605 GiB / 626 GiB avail; 639 B/s rd, 30 MiB/s wr, 477 op/s; 42657/320235 objects misplaced (13.321%); 14 MiB/s, 229 objects/s recovering 2024-09-20T19:03:42.611 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:42 smithi083 ceph-mon[36260]: osdmap e360: 8 total, 8 up, 7 in 2024-09-20T19:03:42.656 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:03:42.657 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:03:42.673 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:42.692 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:03:42.692 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:03:42.709 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:42.710 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:03:42.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:42.711 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:42.727 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:03:42.727 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:03:42.762 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:03:42.762 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:03:42.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:42 smithi142 ceph-mon[32508]: pgmap v691: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 6.5 GiB data, 20 GiB used, 605 GiB / 626 GiB avail; 639 B/s rd, 30 MiB/s wr, 477 op/s; 42657/320235 objects misplaced (13.321%); 14 MiB/s, 229 objects/s recovering 2024-09-20T19:03:42.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:42 smithi142 ceph-mon[32508]: osdmap e360: 8 total, 8 up, 7 in 2024-09-20T19:03:42.795 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:03:42.795 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:03:42.812 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:42.831 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:03:42.831 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:03:42.862 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:42.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:42.863 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:42.872 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:03:42.873 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:03:42.882 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 83 16 111378 111362 83.8389 48.9375 0.0118203 0.011922 2024-09-20T19:03:42.902 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:03:42.903 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:03:42.964 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:42.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:42.994 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:42.994 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:42.995 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_blocked_ops 2024-09-20T19:03:43.095 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:43.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:43.125 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:43.225 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:43.234 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T19:03:43.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:43.276 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:43.276 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:43.377 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:43.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:43.443 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:43.443 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:43.454 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:03:43.523 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:43 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3074968841' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:03:43.523 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:43 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3074968841' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:03:43.544 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:43.583 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:43.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:43.584 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:43.685 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:43.723 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:43.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:43.724 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:43.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:43 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3074968841' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:03:43.825 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:43.854 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:43.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:43.856 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:43.882 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 84 16 112080 112064 83.3631 43.875 0.0270186 0.0119886 2024-09-20T19:03:43.957 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:43.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:43.987 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:43.987 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:44.076 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T19:03:44.076 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-09-20T19:03:44.076 INFO:teuthology.orchestra.run.smithi142.stdout: "complaint_time": 30, 2024-09-20T19:03:44.076 INFO:teuthology.orchestra.run.smithi142.stdout: "num_blocked_ops": 0 2024-09-20T19:03:44.076 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T19:03:44.089 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:03:44.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:44.137 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:44.137 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:03:44.238 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:44.270 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:44.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:44.271 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:44.372 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:44.390 INFO:teuthology.orchestra.run.smithi083.stderr:set pool 1 pgp_num to 1 2024-09-20T19:03:44.420 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:44.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:44.421 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:44.522 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:44.563 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:44.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:44.564 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:44.665 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:44.699 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:44.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:44.700 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:44.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:44 smithi142 ceph-mon[32508]: pgmap v693: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 6.7 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1.3 KiB/s rd, 61 MiB/s wr, 974 op/s; 41696/331515 objects misplaced (12.577%); 29 MiB/s, 471 objects/s recovering 2024-09-20T19:03:44.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:44 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/197089646' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"} : dispatch 2024-09-20T19:03:44.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:44 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"} : dispatch 2024-09-20T19:03:44.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:44 smithi083 ceph-mon[29421]: pgmap v693: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 6.7 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1.3 KiB/s rd, 61 MiB/s wr, 974 op/s; 41696/331515 objects misplaced (12.577%); 29 MiB/s, 471 objects/s recovering 2024-09-20T19:03:44.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:44 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/197089646' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"} : dispatch 2024-09-20T19:03:44.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:44 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"} : dispatch 2024-09-20T19:03:44.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:44 smithi083 ceph-mon[36260]: pgmap v693: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 6.7 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1.3 KiB/s rd, 61 MiB/s wr, 974 op/s; 41696/331515 objects misplaced (12.577%); 29 MiB/s, 471 objects/s recovering 2024-09-20T19:03:44.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:44 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/197089646' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"} : dispatch 2024-09-20T19:03:44.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:44 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"} : dispatch 2024-09-20T19:03:44.801 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:44.838 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:44.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:44.839 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:44.853 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_historic_ops 2024-09-20T19:03:44.882 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 85 16 112903 112887 82.9875 51.4375 0.0180608 0.0120432 2024-09-20T19:03:44.940 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:44.965 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:44.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:44.966 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:45.066 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:45.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:45.103 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:45.104 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:45.205 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:45.233 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:45.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:45.234 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:45.335 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:45.344 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:03:45.372 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:45.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:45.373 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:45.474 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:45.521 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:45.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:45.522 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:45.623 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:45.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:45.661 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:45.661 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:45.762 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:03:45.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:45 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"}]': finished 2024-09-20T19:03:45.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:45 smithi142 ceph-mon[32508]: osdmap e361: 8 total, 8 up, 7 in 2024-09-20T19:03:45.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:45 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"}]': finished 2024-09-20T19:03:45.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:45 smithi083 ceph-mon[29421]: osdmap e361: 8 total, 8 up, 7 in 2024-09-20T19:03:45.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:45 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"}]': finished 2024-09-20T19:03:45.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:45 smithi083 ceph-mon[36260]: osdmap e361: 8 total, 8 up, 7 in 2024-09-20T19:03:45.806 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:45.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:45.807 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:03:45.882 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 86 16 113673 113657 82.5821 48.125 0.00459618 0.012103 2024-09-20T19:03:45.908 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:03:45.944 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:45.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:45.945 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:03:46.046 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:03:46.085 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:46.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:46.086 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:03:46.187 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:46.225 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:46.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:46.226 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:46.327 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:03:46.360 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:46.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:46.361 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:03:46.462 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:46.483 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:46.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:46.484 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:46.584 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:46.626 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:46.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:46.627 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:46.728 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:46.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:46 smithi142 ceph-mon[32508]: pgmap v695: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 6.7 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 853 B/s rd, 39 MiB/s wr, 627 op/s; 41696/331515 objects misplaced (12.577%); 19 MiB/s, 306 objects/s recovering 2024-09-20T19:03:46.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:46 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:03:46.763 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:46.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:46.764 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:46.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:46 smithi083 ceph-mon[29421]: pgmap v695: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 6.7 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 853 B/s rd, 39 MiB/s wr, 627 op/s; 41696/331515 objects misplaced (12.577%); 19 MiB/s, 306 objects/s recovering 2024-09-20T19:03:46.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:46 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:03:46.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:46 smithi083 ceph-mon[36260]: pgmap v695: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 6.7 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 853 B/s rd, 39 MiB/s wr, 627 op/s; 41696/331515 objects misplaced (12.577%); 19 MiB/s, 306 objects/s recovering 2024-09-20T19:03:46.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:46 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:03:46.865 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:46.882 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 87 16 114502 114486 82.2285 51.8125 0.0107509 0.0121562 2024-09-20T19:03:46.901 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:46.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:46.902 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:47.002 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:47.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:47.044 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:47.044 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:47.145 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:03:47.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:47.183 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:47.183 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:03:47.240 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T19:03:47.240 INFO:teuthology.orchestra.run.smithi142.stdout: "size": 20, 2024-09-20T19:03:47.240 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 600, 2024-09-20T19:03:47.240 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-09-20T19:03:47.240 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.240 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:74256 10.e 10:75ae218b:::benchmark_data_smithi083_148100_object74255:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e346)", 2024-09-20T19:03:47.241 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:02:58.610076+0000", 2024-09-20T19:03:47.241 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 48.621894333, 2024-09-20T19:03:47.241 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.136052799, 2024-09-20T19:03:47.241 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:03:47.241 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:03:47.241 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:03:47.241 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:03:47.241 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:03:47.241 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:03:47.241 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 74256 2024-09-20T19:03:47.241 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.241 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:03:47.241 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.241 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:03:47.242 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.610076+0000", 2024-09-20T19:03:47.242 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:47.242 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.242 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.242 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:03:47.242 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.610076+0000", 2024-09-20T19:03:47.242 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:47.242 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.242 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.242 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:03:47.242 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.610077+0000", 2024-09-20T19:03:47.242 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4100000000000001e-06 2024-09-20T19:03:47.242 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.242 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.243 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:03:47.243 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.610116+0000", 2024-09-20T19:03:47.243 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.8541999999999998e-05 2024-09-20T19:03:47.243 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.243 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.243 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:03:47.243 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.610127+0000", 2024-09-20T19:03:47.243 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0900999999999999e-05 2024-09-20T19:03:47.243 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.243 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.243 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:03:47.243 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.610134+0000", 2024-09-20T19:03:47.243 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.029e-06 2024-09-20T19:03:47.244 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.244 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.244 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:47.244 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.610181+0000", 2024-09-20T19:03:47.244 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.6678000000000003e-05 2024-09-20T19:03:47.244 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.244 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.244 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:03:47.244 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.610255+0000", 2024-09-20T19:03:47.244 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.4810999999999999e-05 2024-09-20T19:03:47.244 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.244 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.244 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 2,6", 2024-09-20T19:03:47.244 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.610337+0000", 2024-09-20T19:03:47.245 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.1342999999999997e-05 2024-09-20T19:03:47.245 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.245 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.245 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:03:47.245 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.612445+0000", 2024-09-20T19:03:47.245 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.002108586 2024-09-20T19:03:47.245 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.245 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.245 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:03:47.245 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.612558+0000", 2024-09-20T19:03:47.245 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000112433 2024-09-20T19:03:47.245 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.245 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.246 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:03:47.246 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.745424+0000", 2024-09-20T19:03:47.246 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13286580200000001 2024-09-20T19:03:47.246 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.246 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.246 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:03:47.246 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.746089+0000", 2024-09-20T19:03:47.246 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000665399 2024-09-20T19:03:47.246 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.246 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.246 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:03:47.246 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.746129+0000", 2024-09-20T19:03:47.246 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9864999999999999e-05 2024-09-20T19:03:47.247 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:47.247 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:03:47.247 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:47.247 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.247 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.247 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:74259 10.e 10:76df373f:::benchmark_data_smithi083_148100_object74258:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e346)", 2024-09-20T19:03:47.247 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:02:58.616415+0000", 2024-09-20T19:03:47.247 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 48.615555102000002, 2024-09-20T19:03:47.247 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.133869291, 2024-09-20T19:03:47.247 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:03:47.247 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:03:47.247 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:03:47.247 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:03:47.248 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:03:47.248 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:03:47.248 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 74259 2024-09-20T19:03:47.248 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.248 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:03:47.248 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.248 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:03:47.248 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.616415+0000", 2024-09-20T19:03:47.248 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:47.248 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.248 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.248 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:03:47.248 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.616415+0000", 2024-09-20T19:03:47.248 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:47.249 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.249 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.249 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:03:47.249 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.616417+0000", 2024-09-20T19:03:47.249 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.437e-06 2024-09-20T19:03:47.249 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.249 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.249 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:03:47.249 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.616464+0000", 2024-09-20T19:03:47.249 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7648000000000002e-05 2024-09-20T19:03:47.249 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.249 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.249 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:03:47.249 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.616473+0000", 2024-09-20T19:03:47.250 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.2870000000000004e-06 2024-09-20T19:03:47.250 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.250 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.250 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:03:47.250 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.616478+0000", 2024-09-20T19:03:47.250 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.9529999999999998e-06 2024-09-20T19:03:47.250 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.250 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.250 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:47.250 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.616529+0000", 2024-09-20T19:03:47.250 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0930999999999997e-05 2024-09-20T19:03:47.250 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.250 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.250 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:03:47.250 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.742796+0000", 2024-09-20T19:03:47.251 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.12626707000000001 2024-09-20T19:03:47.251 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.251 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.251 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 2,6", 2024-09-20T19:03:47.251 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.742947+0000", 2024-09-20T19:03:47.251 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000151485 2024-09-20T19:03:47.251 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.251 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.251 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:03:47.251 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.746172+0000", 2024-09-20T19:03:47.251 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0032244579999999999 2024-09-20T19:03:47.251 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.251 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.251 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:03:47.252 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.747658+0000", 2024-09-20T19:03:47.252 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001486816 2024-09-20T19:03:47.252 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.252 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.252 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:03:47.252 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.750184+0000", 2024-09-20T19:03:47.252 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0025253770000000001 2024-09-20T19:03:47.252 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.252 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.252 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:03:47.252 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.750235+0000", 2024-09-20T19:03:47.252 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.1119000000000003e-05 2024-09-20T19:03:47.252 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.252 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.253 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:03:47.253 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.750285+0000", 2024-09-20T19:03:47.253 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.9709999999999997e-05 2024-09-20T19:03:47.253 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:47.253 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:03:47.253 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:47.253 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.253 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.253 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25775.0:74258 10.6 e346/325 10:603cb175:::benchmark_data_smithi083_148100_object74257:head v 346'4675, mlcod=346'4673)", 2024-09-20T19:03:47.253 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:02:58.616715+0000", 2024-09-20T19:03:47.253 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 48.615255636999997, 2024-09-20T19:03:47.253 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.131125189, 2024-09-20T19:03:47.253 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:03:47.253 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:03:47.254 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:03:47.254 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:03:47.254 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.254 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:03:47.254 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.616715+0000", 2024-09-20T19:03:47.254 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:47.254 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.255 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.255 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:03:47.255 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.616715+0000", 2024-09-20T19:03:47.255 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:47.255 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.255 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.255 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:03:47.255 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.616725+0000", 2024-09-20T19:03:47.255 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.8779999999999993e-06 2024-09-20T19:03:47.255 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.255 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.255 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:03:47.255 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.616759+0000", 2024-09-20T19:03:47.255 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4548000000000002e-05 2024-09-20T19:03:47.256 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.256 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.256 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:03:47.256 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.740906+0000", 2024-09-20T19:03:47.256 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.12414689499999999 2024-09-20T19:03:47.256 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.256 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.256 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:03:47.256 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.740921+0000", 2024-09-20T19:03:47.256 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5228999999999999e-05 2024-09-20T19:03:47.256 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.256 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.256 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:47.256 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.740980+0000", 2024-09-20T19:03:47.256 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8998e-05 2024-09-20T19:03:47.257 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.257 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.257 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:03:47.257 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.741008+0000", 2024-09-20T19:03:47.257 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.7543999999999999e-05 2024-09-20T19:03:47.257 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.257 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.257 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:03:47.257 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.747691+0000", 2024-09-20T19:03:47.257 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0066831299999999998 2024-09-20T19:03:47.257 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.257 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.257 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:03:47.257 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.747840+0000", 2024-09-20T19:03:47.258 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00014896700000000001 2024-09-20T19:03:47.258 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:47.258 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:03:47.258 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:47.258 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.258 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.258 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop_reply(client.25775.0:74256 10.e e346/325 ondisk, result = 0)", 2024-09-20T19:03:47.258 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:02:58.617355+0000", 2024-09-20T19:03:47.258 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 48.614615614000002, 2024-09-20T19:03:47.258 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.128781805, 2024-09-20T19:03:47.258 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:03:47.258 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:03:47.258 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-09-20T19:03:47.258 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:03:47.259 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.259 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:03:47.259 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.617355+0000", 2024-09-20T19:03:47.259 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:47.259 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.259 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.259 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:03:47.259 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.617355+0000", 2024-09-20T19:03:47.259 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:47.259 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.259 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.259 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:03:47.259 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.617356+0000", 2024-09-20T19:03:47.259 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.4700000000000001e-07 2024-09-20T19:03:47.259 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.260 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.260 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:03:47.260 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.617360+0000", 2024-09-20T19:03:47.260 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3420000000000001e-06 2024-09-20T19:03:47.260 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.260 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.260 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:03:47.260 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.740907+0000", 2024-09-20T19:03:47.260 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.123547053 2024-09-20T19:03:47.260 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.260 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.260 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:03:47.260 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.740921+0000", 2024-09-20T19:03:47.260 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.449e-05 2024-09-20T19:03:47.261 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.261 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.261 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:47.261 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.745412+0000", 2024-09-20T19:03:47.261 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0044910419999999998 2024-09-20T19:03:47.261 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.261 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.261 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:03:47.261 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.745419+0000", 2024-09-20T19:03:47.261 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.1279999999999996e-06 2024-09-20T19:03:47.261 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.261 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.261 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:03:47.261 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.746137+0000", 2024-09-20T19:03:47.261 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00071800300000000004 2024-09-20T19:03:47.262 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:47.262 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:03:47.262 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:47.262 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.262 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.262 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:74271 10.1 10:8bb3e401:::benchmark_data_smithi083_148100_object74270:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e346)", 2024-09-20T19:03:47.262 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:02:58.617997+0000", 2024-09-20T19:03:47.262 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 48.613973037999997, 2024-09-20T19:03:47.262 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.12988050000000001, 2024-09-20T19:03:47.262 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:03:47.262 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:03:47.262 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:03:47.262 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:03:47.262 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:03:47.263 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:03:47.263 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 74271 2024-09-20T19:03:47.263 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.263 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:03:47.263 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.263 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:03:47.263 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.617997+0000", 2024-09-20T19:03:47.263 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:47.263 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.263 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.263 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:03:47.263 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.617997+0000", 2024-09-20T19:03:47.263 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:47.263 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.264 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.264 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:03:47.264 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.617998+0000", 2024-09-20T19:03:47.264 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.001e-06 2024-09-20T19:03:47.264 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.264 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.264 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:03:47.264 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.618145+0000", 2024-09-20T19:03:47.264 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000146704 2024-09-20T19:03:47.264 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.264 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.264 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:03:47.264 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.740935+0000", 2024-09-20T19:03:47.264 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.122789566 2024-09-20T19:03:47.264 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.265 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.265 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:03:47.265 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.740947+0000", 2024-09-20T19:03:47.265 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2877999999999999e-05 2024-09-20T19:03:47.265 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.265 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.265 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:47.265 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.741010+0000", 2024-09-20T19:03:47.265 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2160000000000001e-05 2024-09-20T19:03:47.265 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.265 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.265 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:03:47.265 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.741152+0000", 2024-09-20T19:03:47.265 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00014213499999999999 2024-09-20T19:03:47.265 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.266 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.266 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,7", 2024-09-20T19:03:47.266 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.741245+0000", 2024-09-20T19:03:47.266 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.3341e-05 2024-09-20T19:03:47.266 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.266 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.266 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:03:47.266 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.747621+0000", 2024-09-20T19:03:47.266 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0063756469999999999 2024-09-20T19:03:47.266 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.266 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.266 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:03:47.266 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.747683+0000", 2024-09-20T19:03:47.266 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2577999999999996e-05 2024-09-20T19:03:47.266 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.267 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.267 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:03:47.267 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.747770+0000", 2024-09-20T19:03:47.267 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.6630000000000002e-05 2024-09-20T19:03:47.267 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.267 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.267 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:03:47.267 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.747836+0000", 2024-09-20T19:03:47.267 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5827000000000005e-05 2024-09-20T19:03:47.267 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.267 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.267 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:03:47.267 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:58.747878+0000", 2024-09-20T19:03:47.267 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.2033000000000002e-05 2024-09-20T19:03:47.267 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:47.268 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:03:47.268 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:47.268 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.268 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.268 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25775.0:74702 10.0 e346/325 10:0781416f:::benchmark_data_smithi083_148100_object74701:head v 346'4668, mlcod=346'4666)", 2024-09-20T19:03:47.268 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:02:59.072089+0000", 2024-09-20T19:03:47.268 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 48.159881542000001, 2024-09-20T19:03:47.268 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.075749440000000001, 2024-09-20T19:03:47.268 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:03:47.268 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:03:47.268 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:03:47.268 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:03:47.268 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.268 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:03:47.269 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:59.072089+0000", 2024-09-20T19:03:47.269 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:47.269 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.269 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.269 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:03:47.269 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:59.072089+0000", 2024-09-20T19:03:47.269 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:47.269 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.269 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.269 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:03:47.269 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:59.072089+0000", 2024-09-20T19:03:47.269 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.41e-07 2024-09-20T19:03:47.269 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.269 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.269 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:03:47.270 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:59.072114+0000", 2024-09-20T19:03:47.270 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.5228999999999998e-05 2024-09-20T19:03:47.270 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.270 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.270 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:03:47.270 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:59.072121+0000", 2024-09-20T19:03:47.270 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.1129999999999997e-06 2024-09-20T19:03:47.270 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.270 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.270 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:03:47.270 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:59.072125+0000", 2024-09-20T19:03:47.270 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.6070000000000001e-06 2024-09-20T19:03:47.270 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.270 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.271 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:47.271 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:59.141260+0000", 2024-09-20T19:03:47.271 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.069134630000000002 2024-09-20T19:03:47.271 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.271 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.271 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:03:47.271 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:59.141292+0000", 2024-09-20T19:03:47.271 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.1795000000000003e-05 2024-09-20T19:03:47.271 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.271 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.271 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:03:47.271 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:59.147798+0000", 2024-09-20T19:03:47.271 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.006506614 2024-09-20T19:03:47.271 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.272 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.272 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:03:47.272 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:59.147838+0000", 2024-09-20T19:03:47.272 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0111000000000003e-05 2024-09-20T19:03:47.272 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:47.272 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:03:47.272 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:47.272 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.272 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.272 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25775.0:74697 10.c e346/325 10:3fdcc267:::benchmark_data_smithi083_148100_object74696:head v 346'4591, mlcod=346'4587)", 2024-09-20T19:03:47.272 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:02:59.072572+0000", 2024-09-20T19:03:47.272 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 48.159397994999999, 2024-09-20T19:03:47.272 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.080686976999999993, 2024-09-20T19:03:47.272 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:03:47.272 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:03:47.273 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:03:47.273 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:03:47.273 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.273 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:03:47.273 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:59.072572+0000", 2024-09-20T19:03:47.273 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:47.273 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.273 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.273 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:03:47.273 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:59.072572+0000", 2024-09-20T19:03:47.273 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:47.273 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.273 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.273 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:03:47.273 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:59.072573+0000", 2024-09-20T19:03:47.274 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.1399999999999995e-07 2024-09-20T19:03:47.274 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.274 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.274 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:03:47.274 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:59.072596+0000", 2024-09-20T19:03:47.274 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2999000000000001e-05 2024-09-20T19:03:47.274 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.274 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.274 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:03:47.274 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:59.141207+0000", 2024-09-20T19:03:47.274 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.068610844000000004 2024-09-20T19:03:47.274 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.274 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.274 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:03:47.275 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:59.141218+0000", 2024-09-20T19:03:47.275 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0597999999999999e-05 2024-09-20T19:03:47.275 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.275 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.275 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:47.275 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:59.143857+0000", 2024-09-20T19:03:47.275 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0026396010000000001 2024-09-20T19:03:47.275 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.275 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.275 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:03:47.275 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:59.143873+0000", 2024-09-20T19:03:47.275 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5945999999999999e-05 2024-09-20T19:03:47.275 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.275 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.275 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:03:47.276 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:59.153235+0000", 2024-09-20T19:03:47.276 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0093621759999999998 2024-09-20T19:03:47.276 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.276 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.276 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:03:47.276 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:59.153259+0000", 2024-09-20T19:03:47.276 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3898999999999999e-05 2024-09-20T19:03:47.276 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:47.276 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:03:47.276 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:47.276 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.276 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.276 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25775.0:74703 10.4 e346/325 10:27d0cec6:::benchmark_data_smithi083_148100_object74702:head v 346'4626, mlcod=346'4623)", 2024-09-20T19:03:47.277 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:02:59.075592+0000", 2024-09-20T19:03:47.277 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 48.156378687999997, 2024-09-20T19:03:47.277 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.077617070999999996, 2024-09-20T19:03:47.277 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:03:47.277 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:03:47.277 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:03:47.277 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:03:47.277 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.277 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:03:47.277 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:59.075592+0000", 2024-09-20T19:03:47.277 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:47.277 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.277 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.277 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:03:47.277 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:59.075592+0000", 2024-09-20T19:03:47.278 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:47.278 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.278 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.278 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:03:47.278 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:59.075593+0000", 2024-09-20T19:03:47.278 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.6299999999999993e-07 2024-09-20T19:03:47.278 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.278 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.278 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:03:47.278 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:59.075642+0000", 2024-09-20T19:03:47.278 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.9137000000000001e-05 2024-09-20T19:03:47.278 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.278 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.278 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:03:47.279 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:59.141195+0000", 2024-09-20T19:03:47.279 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.065553009999999995 2024-09-20T19:03:47.279 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.279 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.279 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:03:47.279 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:59.141210+0000", 2024-09-20T19:03:47.279 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5258e-05 2024-09-20T19:03:47.279 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.279 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.279 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:47.279 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:59.141485+0000", 2024-09-20T19:03:47.279 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00027448499999999999 2024-09-20T19:03:47.279 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.279 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.280 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:03:47.280 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:59.141501+0000", 2024-09-20T19:03:47.280 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6705e-05 2024-09-20T19:03:47.280 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.280 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.280 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:03:47.280 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:59.153170+0000", 2024-09-20T19:03:47.280 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.011668474 2024-09-20T19:03:47.280 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.280 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.280 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:03:47.280 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:59.153209+0000", 2024-09-20T19:03:47.280 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9038999999999997e-05 2024-09-20T19:03:47.280 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:47.281 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:03:47.281 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:47.281 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.281 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.281 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:74827 10.e 10:7215f548:::benchmark_data_smithi083_148100_object74826:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e346)", 2024-09-20T19:03:47.281 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:02:59.252718+0000", 2024-09-20T19:03:47.282 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 47.979252649000003, 2024-09-20T19:03:47.282 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.178107031, 2024-09-20T19:03:47.282 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:03:47.282 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:03:47.282 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:03:47.282 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:03:47.282 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:03:47.282 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:03:47.282 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 74827 2024-09-20T19:03:47.282 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.282 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:03:47.283 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.283 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:03:47.283 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:59.252718+0000", 2024-09-20T19:03:47.283 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:47.283 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.283 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.283 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:03:47.283 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:59.252718+0000", 2024-09-20T19:03:47.283 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:47.283 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.283 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.283 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:03:47.283 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:59.252731+0000", 2024-09-20T19:03:47.284 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2826999999999999e-05 2024-09-20T19:03:47.284 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.284 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.284 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:03:47.284 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:59.252797+0000", 2024-09-20T19:03:47.284 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.6149000000000007e-05 2024-09-20T19:03:47.284 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.284 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.284 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:03:47.284 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:59.252841+0000", 2024-09-20T19:03:47.284 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.4385000000000003e-05 2024-09-20T19:03:47.284 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.284 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.285 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:03:47.285 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:59.252862+0000", 2024-09-20T19:03:47.285 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1237e-05 2024-09-20T19:03:47.285 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.285 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.285 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:47.285 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:59.252936+0000", 2024-09-20T19:03:47.285 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.3460999999999993e-05 2024-09-20T19:03:47.285 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.285 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.285 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:03:47.285 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:59.253016+0000", 2024-09-20T19:03:47.285 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.0177000000000002e-05 2024-09-20T19:03:47.285 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.286 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.286 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 2,6", 2024-09-20T19:03:47.286 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:59.253112+0000", 2024-09-20T19:03:47.286 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.5948999999999999e-05 2024-09-20T19:03:47.286 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.286 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.286 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:03:47.286 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:59.255007+0000", 2024-09-20T19:03:47.286 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001895447 2024-09-20T19:03:47.286 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.286 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.286 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:03:47.286 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:59.255497+0000", 2024-09-20T19:03:47.286 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00048975400000000001 2024-09-20T19:03:47.287 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.287 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.287 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:03:47.287 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:59.430737+0000", 2024-09-20T19:03:47.287 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.17524031800000001 2024-09-20T19:03:47.287 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.287 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.287 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:03:47.287 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:59.430786+0000", 2024-09-20T19:03:47.287 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8729000000000001e-05 2024-09-20T19:03:47.287 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.287 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.287 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:03:47.287 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:02:59.430825+0000", 2024-09-20T19:03:47.288 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.8597999999999999e-05 2024-09-20T19:03:47.288 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:47.288 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:03:47.288 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:47.288 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.288 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.288 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:80003 10.e 10:7ce083b0:::benchmark_data_smithi083_148100_object80002:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e347)", 2024-09-20T19:03:47.288 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:03:03.660554+0000", 2024-09-20T19:03:47.288 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 43.571416233000001, 2024-09-20T19:03:47.288 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.078325054000000005, 2024-09-20T19:03:47.288 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:03:47.288 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:03:47.288 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:03:47.288 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:03:47.289 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:03:47.289 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:03:47.289 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 80003 2024-09-20T19:03:47.289 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.289 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:03:47.289 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.289 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:03:47.289 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.660554+0000", 2024-09-20T19:03:47.289 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:47.289 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.289 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.289 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:03:47.289 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.660554+0000", 2024-09-20T19:03:47.289 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:47.289 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.290 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.290 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:03:47.290 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.660556+0000", 2024-09-20T19:03:47.290 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7710000000000001e-06 2024-09-20T19:03:47.290 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.290 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.290 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:03:47.290 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.660593+0000", 2024-09-20T19:03:47.290 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6850000000000001e-05 2024-09-20T19:03:47.290 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.290 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.290 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:03:47.290 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.660605+0000", 2024-09-20T19:03:47.290 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2408e-05 2024-09-20T19:03:47.291 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.291 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.291 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:03:47.291 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.660613+0000", 2024-09-20T19:03:47.291 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.588e-06 2024-09-20T19:03:47.291 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.291 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.291 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:47.291 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.660943+0000", 2024-09-20T19:03:47.291 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00032999100000000002 2024-09-20T19:03:47.291 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.291 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.291 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:03:47.291 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.661054+0000", 2024-09-20T19:03:47.291 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000111702 2024-09-20T19:03:47.292 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.292 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.292 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 2,6", 2024-09-20T19:03:47.292 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.661177+0000", 2024-09-20T19:03:47.292 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012242700000000001 2024-09-20T19:03:47.292 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.292 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.292 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:03:47.292 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.663402+0000", 2024-09-20T19:03:47.292 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0022255899999999999 2024-09-20T19:03:47.292 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.292 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.292 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:03:47.292 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.663957+0000", 2024-09-20T19:03:47.293 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00055484600000000001 2024-09-20T19:03:47.293 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.293 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.293 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:03:47.293 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.738768+0000", 2024-09-20T19:03:47.293 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.074810686000000001 2024-09-20T19:03:47.293 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.293 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.293 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:03:47.293 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.738828+0000", 2024-09-20T19:03:47.293 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9737000000000001e-05 2024-09-20T19:03:47.293 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.293 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.293 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:03:47.294 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.738879+0000", 2024-09-20T19:03:47.294 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.1458000000000001e-05 2024-09-20T19:03:47.294 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:47.294 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:03:47.294 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:47.294 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.294 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.294 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:80007 10.1 10:81e6574f:::benchmark_data_smithi083_148100_object80006:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e347)", 2024-09-20T19:03:47.294 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:03:03.665022+0000", 2024-09-20T19:03:47.294 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 43.566948056000001, 2024-09-20T19:03:47.294 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.078607963000000003, 2024-09-20T19:03:47.294 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:03:47.294 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:03:47.294 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:03:47.294 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:03:47.295 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:03:47.295 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:03:47.295 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 80007 2024-09-20T19:03:47.295 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.295 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:03:47.295 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.295 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:03:47.295 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.665022+0000", 2024-09-20T19:03:47.295 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:47.295 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.295 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.295 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:03:47.295 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.665022+0000", 2024-09-20T19:03:47.296 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:47.296 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.296 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.296 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:03:47.296 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.665023+0000", 2024-09-20T19:03:47.296 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.1500000000000005e-07 2024-09-20T19:03:47.296 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.296 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.296 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:03:47.296 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.665057+0000", 2024-09-20T19:03:47.296 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4646e-05 2024-09-20T19:03:47.296 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.296 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.296 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:03:47.297 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.665065+0000", 2024-09-20T19:03:47.297 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.9119999999999998e-06 2024-09-20T19:03:47.297 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.297 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.297 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:03:47.297 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.665072+0000", 2024-09-20T19:03:47.297 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2179999999999996e-06 2024-09-20T19:03:47.297 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.297 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.297 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:47.297 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.665123+0000", 2024-09-20T19:03:47.297 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.1631000000000001e-05 2024-09-20T19:03:47.297 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.297 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.298 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:03:47.298 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.665241+0000", 2024-09-20T19:03:47.298 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000117269 2024-09-20T19:03:47.298 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.298 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.298 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,7", 2024-09-20T19:03:47.298 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.665352+0000", 2024-09-20T19:03:47.298 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000111904 2024-09-20T19:03:47.298 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.298 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.298 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:03:47.298 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.740993+0000", 2024-09-20T19:03:47.298 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.075640913000000004 2024-09-20T19:03:47.298 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.298 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.299 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:03:47.299 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.741051+0000", 2024-09-20T19:03:47.299 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.7225000000000002e-05 2024-09-20T19:03:47.299 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.299 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.299 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:03:47.299 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.743487+0000", 2024-09-20T19:03:47.299 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0024364719999999999 2024-09-20T19:03:47.299 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.299 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.299 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:03:47.299 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.743583+0000", 2024-09-20T19:03:47.299 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.6322000000000004e-05 2024-09-20T19:03:47.300 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.300 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.300 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:03:47.300 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.743630+0000", 2024-09-20T19:03:47.300 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.6935999999999999e-05 2024-09-20T19:03:47.300 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:47.300 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:03:47.300 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:47.300 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.300 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.300 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25775.0:80009 10.2 e347/325 10:481918d6:::benchmark_data_smithi083_148100_object80008:head v 347'5035, mlcod=347'5031)", 2024-09-20T19:03:47.300 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:03:03.666556+0000", 2024-09-20T19:03:47.300 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 43.565414308000001, 2024-09-20T19:03:47.300 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.074581155999999996, 2024-09-20T19:03:47.300 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:03:47.301 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:03:47.301 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:03:47.301 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:03:47.301 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.301 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:03:47.301 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.666556+0000", 2024-09-20T19:03:47.301 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:47.301 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.301 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.301 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:03:47.301 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.666556+0000", 2024-09-20T19:03:47.301 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:47.301 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.301 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.302 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:03:47.302 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.666557+0000", 2024-09-20T19:03:47.302 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.6899999999999997e-07 2024-09-20T19:03:47.302 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.302 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.302 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:03:47.302 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.666614+0000", 2024-09-20T19:03:47.302 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.7537000000000002e-05 2024-09-20T19:03:47.302 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.302 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.302 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:03:47.302 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.738151+0000", 2024-09-20T19:03:47.303 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.071536534999999998 2024-09-20T19:03:47.303 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.303 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.303 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:03:47.303 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.738162+0000", 2024-09-20T19:03:47.303 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1644000000000001e-05 2024-09-20T19:03:47.303 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.303 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.303 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:47.303 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.738221+0000", 2024-09-20T19:03:47.303 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8984000000000003e-05 2024-09-20T19:03:47.303 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.303 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.303 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:03:47.304 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.738244+0000", 2024-09-20T19:03:47.304 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2954000000000001e-05 2024-09-20T19:03:47.304 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.304 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.304 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:03:47.304 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.740991+0000", 2024-09-20T19:03:47.304 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0027465580000000001 2024-09-20T19:03:47.304 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.304 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.304 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:03:47.304 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.741137+0000", 2024-09-20T19:03:47.304 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00014627499999999999 2024-09-20T19:03:47.304 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:47.304 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:03:47.305 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:47.305 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.305 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.305 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:80282 10.e 10:7465786a:::benchmark_data_smithi083_148100_object80281:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e348)", 2024-09-20T19:03:47.305 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:03:03.954407+0000", 2024-09-20T19:03:47.305 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 43.277562899000003, 2024-09-20T19:03:47.305 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.092382073999999995, 2024-09-20T19:03:47.305 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:03:47.305 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:03:47.305 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:03:47.305 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:03:47.305 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:03:47.305 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:03:47.305 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 80282 2024-09-20T19:03:47.305 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.305 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:03:47.306 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.306 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:03:47.306 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.954407+0000", 2024-09-20T19:03:47.306 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:47.306 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.306 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.306 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:03:47.306 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.954407+0000", 2024-09-20T19:03:47.306 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:47.306 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.306 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.306 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:03:47.306 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.954409+0000", 2024-09-20T19:03:47.306 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5379999999999999e-06 2024-09-20T19:03:47.307 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.307 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.307 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:03:47.307 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.954473+0000", 2024-09-20T19:03:47.307 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4288999999999994e-05 2024-09-20T19:03:47.307 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.307 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.307 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:03:47.307 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.954484+0000", 2024-09-20T19:03:47.307 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0587e-05 2024-09-20T19:03:47.307 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.307 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.307 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:03:47.307 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:04.042410+0000", 2024-09-20T19:03:47.308 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.087925691 2024-09-20T19:03:47.308 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.308 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.308 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:47.308 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:04.042635+0000", 2024-09-20T19:03:47.308 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00022589100000000001 2024-09-20T19:03:47.308 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.308 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.308 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:03:47.308 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:04.042781+0000", 2024-09-20T19:03:47.308 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00014575800000000001 2024-09-20T19:03:47.308 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.308 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.308 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 2,6", 2024-09-20T19:03:47.308 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:04.043133+0000", 2024-09-20T19:03:47.308 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00035183700000000003 2024-09-20T19:03:47.309 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.309 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.309 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:03:47.309 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:04.045978+0000", 2024-09-20T19:03:47.309 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0028450749999999999 2024-09-20T19:03:47.309 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.309 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.309 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:03:47.309 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:04.046361+0000", 2024-09-20T19:03:47.309 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00038306599999999999 2024-09-20T19:03:47.309 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.309 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.309 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:03:47.309 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:04.046679+0000", 2024-09-20T19:03:47.310 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00031746200000000002 2024-09-20T19:03:47.310 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.310 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.310 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:03:47.310 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:04.046752+0000", 2024-09-20T19:03:47.310 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.3398000000000004e-05 2024-09-20T19:03:47.310 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.310 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.310 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:03:47.310 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:04.046790+0000", 2024-09-20T19:03:47.310 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.7481999999999998e-05 2024-09-20T19:03:47.310 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:47.310 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:03:47.310 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:47.311 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.311 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.311 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25775.0:80285 10.4 e348/325 10:253f8f4b:::benchmark_data_smithi083_148100_object80284:head v 348'5012, mlcod=348'5010)", 2024-09-20T19:03:47.311 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:03:03.955471+0000", 2024-09-20T19:03:47.311 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 43.276498885999999, 2024-09-20T19:03:47.311 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.092793385000000006, 2024-09-20T19:03:47.311 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:03:47.311 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:03:47.311 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:03:47.311 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:03:47.311 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.311 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:03:47.311 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.955471+0000", 2024-09-20T19:03:47.311 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:47.312 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.312 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.312 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:03:47.312 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.955471+0000", 2024-09-20T19:03:47.312 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:47.312 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.312 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.312 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:03:47.312 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.955472+0000", 2024-09-20T19:03:47.312 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5499999999999997e-07 2024-09-20T19:03:47.312 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.312 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.312 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:03:47.312 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:03.955530+0000", 2024-09-20T19:03:47.313 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.7874000000000002e-05 2024-09-20T19:03:47.313 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.313 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.313 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:03:47.313 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:04.042835+0000", 2024-09-20T19:03:47.313 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.087304728999999998 2024-09-20T19:03:47.313 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.313 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.313 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:03:47.313 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:04.042845+0000", 2024-09-20T19:03:47.313 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0193999999999999e-05 2024-09-20T19:03:47.313 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.313 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.313 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:47.314 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:04.042904+0000", 2024-09-20T19:03:47.314 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.91e-05 2024-09-20T19:03:47.314 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.314 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.314 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:03:47.314 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:04.042923+0000", 2024-09-20T19:03:47.314 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9219999999999999e-05 2024-09-20T19:03:47.314 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.314 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.314 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:03:47.314 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:04.048213+0000", 2024-09-20T19:03:47.314 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0052895219999999996 2024-09-20T19:03:47.314 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.314 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.314 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:03:47.315 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:04.048265+0000", 2024-09-20T19:03:47.315 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.1990999999999997e-05 2024-09-20T19:03:47.315 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:47.315 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:03:47.315 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:47.315 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.315 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.315 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25775.0:98046 10.1 e354/351 10:8aae6118:::benchmark_data_smithi083_148100_object98045:head v 354'6051, mlcod=354'6048)", 2024-09-20T19:03:47.315 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:03:24.627235+0000", 2024-09-20T19:03:47.315 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 22.604735087000002, 2024-09-20T19:03:47.315 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.117287243, 2024-09-20T19:03:47.315 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:03:47.315 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:03:47.315 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:03:47.316 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:03:47.316 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.316 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:03:47.316 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:24.627235+0000", 2024-09-20T19:03:47.316 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:47.316 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.316 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.316 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:03:47.316 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:24.627235+0000", 2024-09-20T19:03:47.316 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:47.316 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.316 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.316 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:03:47.316 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:24.627236+0000", 2024-09-20T19:03:47.316 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.04e-06 2024-09-20T19:03:47.317 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.317 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.317 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:03:47.317 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:24.627259+0000", 2024-09-20T19:03:47.317 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2198999999999999e-05 2024-09-20T19:03:47.317 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.317 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.317 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:03:47.317 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:24.627269+0000", 2024-09-20T19:03:47.317 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0025e-05 2024-09-20T19:03:47.317 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.317 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.317 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:03:47.317 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:24.627276+0000", 2024-09-20T19:03:47.317 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.2649999999999999e-06 2024-09-20T19:03:47.318 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.318 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.318 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:47.318 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:24.627333+0000", 2024-09-20T19:03:47.318 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.7590999999999998e-05 2024-09-20T19:03:47.318 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.318 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.318 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:03:47.318 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:24.627360+0000", 2024-09-20T19:03:47.318 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.6160999999999999e-05 2024-09-20T19:03:47.318 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.318 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.318 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:03:47.318 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:24.630108+0000", 2024-09-20T19:03:47.319 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0027480740000000001 2024-09-20T19:03:47.319 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.319 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.319 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:03:47.319 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:24.744523+0000", 2024-09-20T19:03:47.319 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.11441488800000001 2024-09-20T19:03:47.319 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:47.319 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:03:47.319 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:47.319 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.319 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.319 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25775.0:98053 10.1 e354/351 10:8682d176:::benchmark_data_smithi083_148100_object98052:head v 354'6052, mlcod=354'6048)", 2024-09-20T19:03:47.319 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:03:24.631624+0000", 2024-09-20T19:03:47.319 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 22.600346565999999, 2024-09-20T19:03:47.319 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.119440316, 2024-09-20T19:03:47.320 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:03:47.320 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:03:47.320 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:03:47.320 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:03:47.320 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.320 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:03:47.320 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:24.631624+0000", 2024-09-20T19:03:47.320 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:47.320 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.320 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.320 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:03:47.320 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:24.631624+0000", 2024-09-20T19:03:47.320 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:47.320 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.320 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.321 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:03:47.321 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:24.631625+0000", 2024-09-20T19:03:47.321 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4780000000000001e-06 2024-09-20T19:03:47.321 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.321 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.321 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:03:47.321 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:24.631649+0000", 2024-09-20T19:03:47.321 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3472000000000002e-05 2024-09-20T19:03:47.321 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.321 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.321 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:03:47.321 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:24.744736+0000", 2024-09-20T19:03:47.321 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.113087558 2024-09-20T19:03:47.321 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.321 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.322 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:03:47.322 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:24.744746+0000", 2024-09-20T19:03:47.322 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.8779999999999993e-06 2024-09-20T19:03:47.322 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.322 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.322 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:47.322 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:47.323 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:24.744792+0000", 2024-09-20T19:03:47.323 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.5898e-05 2024-09-20T19:03:47.323 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.324 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.324 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:03:47.324 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:24.744812+0000", 2024-09-20T19:03:47.324 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0370999999999999e-05 2024-09-20T19:03:47.324 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.324 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.324 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:03:47.324 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:24.751004+0000", 2024-09-20T19:03:47.324 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0061919870000000004 2024-09-20T19:03:47.324 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.324 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.324 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:03:47.324 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:24.751064+0000", 2024-09-20T19:03:47.324 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9673999999999998e-05 2024-09-20T19:03:47.325 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:47.325 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:03:47.325 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:47.325 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.325 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.325 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25775.0:98054 10.2 e354/351 10:42224c39:::benchmark_data_smithi083_148100_object98053:head v 354'6140, mlcod=354'6138)", 2024-09-20T19:03:47.325 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:03:24.631684+0000", 2024-09-20T19:03:47.325 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 22.600286223000001, 2024-09-20T19:03:47.325 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.11751627100000001, 2024-09-20T19:03:47.325 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:03:47.325 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:03:47.325 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:03:47.325 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:03:47.325 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.326 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:03:47.326 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:24.631684+0000", 2024-09-20T19:03:47.326 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:47.326 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.326 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.326 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:03:47.326 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:24.631684+0000", 2024-09-20T19:03:47.326 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:47.326 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.326 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.326 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:03:47.326 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:24.631685+0000", 2024-09-20T19:03:47.326 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5799999999999998e-07 2024-09-20T19:03:47.326 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.326 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.327 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:03:47.327 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:24.631736+0000", 2024-09-20T19:03:47.327 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0695999999999997e-05 2024-09-20T19:03:47.327 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.327 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.327 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:03:47.327 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:24.744482+0000", 2024-09-20T19:03:47.327 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.112746622 2024-09-20T19:03:47.327 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.327 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.327 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:03:47.327 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:24.744497+0000", 2024-09-20T19:03:47.327 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4734e-05 2024-09-20T19:03:47.327 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.328 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.328 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:47.328 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:24.744591+0000", 2024-09-20T19:03:47.328 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.4535000000000002e-05 2024-09-20T19:03:47.328 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.328 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.328 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:03:47.328 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:24.744618+0000", 2024-09-20T19:03:47.328 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.6676e-05 2024-09-20T19:03:47.328 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.328 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.328 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:03:47.328 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:24.749152+0000", 2024-09-20T19:03:47.328 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0045336839999999996 2024-09-20T19:03:47.328 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.329 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.329 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:03:47.329 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:24.749200+0000", 2024-09-20T19:03:47.329 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8566000000000002e-05 2024-09-20T19:03:47.329 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:47.329 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:03:47.329 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:47.329 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.329 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.329 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25775.0:98061 10.e e354/351 10:72009667:::benchmark_data_smithi083_148100_object98060:head v 354'6201, mlcod=354'6199)", 2024-09-20T19:03:47.329 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:03:24.639229+0000", 2024-09-20T19:03:47.329 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 22.592741801999999, 2024-09-20T19:03:47.329 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.123853691, 2024-09-20T19:03:47.329 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:03:47.329 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:03:47.330 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:03:47.330 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:03:47.330 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.330 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:03:47.330 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:24.639229+0000", 2024-09-20T19:03:47.330 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:47.330 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.330 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.330 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:03:47.330 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:24.639229+0000", 2024-09-20T19:03:47.330 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:47.330 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.330 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.330 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:03:47.331 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:24.639230+0000", 2024-09-20T19:03:47.331 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8649999999999999e-06 2024-09-20T19:03:47.331 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.331 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.331 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:03:47.331 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:24.639257+0000", 2024-09-20T19:03:47.331 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.6250000000000001e-05 2024-09-20T19:03:47.331 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.331 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.331 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:03:47.331 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:24.745070+0000", 2024-09-20T19:03:47.331 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.105813773 2024-09-20T19:03:47.331 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.331 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.332 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:03:47.332 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:24.745076+0000", 2024-09-20T19:03:47.332 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0240000000000003e-06 2024-09-20T19:03:47.332 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.332 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.332 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:47.332 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:24.745120+0000", 2024-09-20T19:03:47.332 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3874e-05 2024-09-20T19:03:47.332 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.332 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.332 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:03:47.332 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:24.745140+0000", 2024-09-20T19:03:47.332 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9457e-05 2024-09-20T19:03:47.332 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.333 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.333 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:03:47.333 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:24.762957+0000", 2024-09-20T19:03:47.333 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.017816849999999999 2024-09-20T19:03:47.333 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.333 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.333 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:03:47.333 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:24.763082+0000", 2024-09-20T19:03:47.333 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012559800000000001 2024-09-20T19:03:47.333 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:47.333 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:03:47.333 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:47.333 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.333 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.333 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25775.0:102949 10.e e357/351 10:7c9615c3:::benchmark_data_smithi083_148100_object102948:head v 357'6494, mlcod=357'6491)", 2024-09-20T19:03:47.333 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:03:31.733261+0000", 2024-09-20T19:03:47.334 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 15.498709277, 2024-09-20T19:03:47.334 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.097336547999999995, 2024-09-20T19:03:47.334 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:03:47.334 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:03:47.334 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:03:47.334 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:03:47.334 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.334 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:03:47.334 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:31.733261+0000", 2024-09-20T19:03:47.334 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:47.334 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.334 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.334 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:03:47.334 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:31.733261+0000", 2024-09-20T19:03:47.335 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:47.335 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.335 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.335 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:03:47.335 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:31.733262+0000", 2024-09-20T19:03:47.335 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1510000000000001e-06 2024-09-20T19:03:47.335 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.335 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.335 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:03:47.335 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:31.733287+0000", 2024-09-20T19:03:47.338 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.5173000000000001e-05 2024-09-20T19:03:47.338 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.338 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.338 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:03:47.338 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:31.733298+0000", 2024-09-20T19:03:47.338 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0776e-05 2024-09-20T19:03:47.338 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.338 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.338 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:03:47.339 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:31.733306+0000", 2024-09-20T19:03:47.339 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.4839999999999998e-06 2024-09-20T19:03:47.339 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.339 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.339 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:47.339 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:31.823873+0000", 2024-09-20T19:03:47.339 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.090566836999999997 2024-09-20T19:03:47.339 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.339 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.339 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:03:47.339 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:31.823947+0000", 2024-09-20T19:03:47.339 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.4064000000000001e-05 2024-09-20T19:03:47.339 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.339 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.340 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:03:47.340 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:31.830504+0000", 2024-09-20T19:03:47.340 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0065578449999999996 2024-09-20T19:03:47.340 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.340 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.340 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:03:47.340 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:31.830598+0000", 2024-09-20T19:03:47.340 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.3218000000000001e-05 2024-09-20T19:03:47.340 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:47.340 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:03:47.340 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:47.340 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.340 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.340 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25775.0:102952 10.e e357/351 10:7e3346ef:::benchmark_data_smithi083_148100_object102951:head v 357'6495, mlcod=357'6491)", 2024-09-20T19:03:47.340 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:03:31.735229+0000", 2024-09-20T19:03:47.341 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 15.496741136000001, 2024-09-20T19:03:47.341 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.095990378000000001, 2024-09-20T19:03:47.341 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:03:47.341 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:03:47.341 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:03:47.341 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:03:47.341 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.341 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:03:47.341 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:31.735229+0000", 2024-09-20T19:03:47.341 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:47.341 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.341 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.341 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:03:47.341 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:31.735229+0000", 2024-09-20T19:03:47.341 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:03:47.342 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.342 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.342 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:03:47.342 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:31.735230+0000", 2024-09-20T19:03:47.342 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0669999999999999e-06 2024-09-20T19:03:47.342 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.342 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.342 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:03:47.342 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:31.735276+0000", 2024-09-20T19:03:47.342 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.5911999999999997e-05 2024-09-20T19:03:47.342 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.342 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.342 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:03:47.342 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:31.822356+0000", 2024-09-20T19:03:47.343 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.087079595999999995 2024-09-20T19:03:47.343 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.343 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.343 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:03:47.343 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:31.822370+0000", 2024-09-20T19:03:47.343 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4651000000000001e-05 2024-09-20T19:03:47.343 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.343 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.343 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:03:47.343 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:31.826802+0000", 2024-09-20T19:03:47.343 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0044318400000000003 2024-09-20T19:03:47.343 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.343 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.343 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:03:47.343 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:31.826826+0000", 2024-09-20T19:03:47.344 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3391999999999999e-05 2024-09-20T19:03:47.344 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.344 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.344 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:03:47.344 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:31.830862+0000", 2024-09-20T19:03:47.344 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0040363109999999999 2024-09-20T19:03:47.344 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:03:47.344 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:03:47.344 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:03:47.344 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:03:31.831220+0000", 2024-09-20T19:03:47.344 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000357609 2024-09-20T19:03:47.344 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:47.344 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:03:47.344 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:47.344 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:03:47.345 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:03:47.345 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T19:03:47.345 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:03:47 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:03:47.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:47.347 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:47.347 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:47.448 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:47.491 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:47.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:47.492 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:47.593 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:47.623 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:47.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:47.624 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:47.725 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:47.765 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:47.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:47.766 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:47.866 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:47.883 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 88 16 115267 115251 81.8369 47.8125 0.0208242 0.0122146 2024-09-20T19:03:47.905 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:47.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:47.906 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:48.007 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:48.043 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:48.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:48.044 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:48.144 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:48.184 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:48.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:48.185 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:48.285 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:48.320 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:48.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:48.321 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:48.422 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:48.448 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:48.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:48.449 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:48.546 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:03:48.550 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:48.592 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:48.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:48.593 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:48.690 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:48 smithi083 ceph-mon[29421]: pgmap v696: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 6.9 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 1.5 KiB/s rd, 67 MiB/s wr, 1.07k op/s; 40587/341964 objects misplaced (11.869%); 34 MiB/s, 537 objects/s recovering 2024-09-20T19:03:48.691 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:48 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:03:48.691 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:48 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:03:48.691 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:48 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:03:48.691 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:48 smithi083 ceph-mon[29421]: osdmap e362: 8 total, 8 up, 7 in 2024-09-20T19:03:48.691 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:48 smithi083 ceph-mon[36260]: pgmap v696: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 6.9 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 1.5 KiB/s rd, 67 MiB/s wr, 1.07k op/s; 40587/341964 objects misplaced (11.869%); 34 MiB/s, 537 objects/s recovering 2024-09-20T19:03:48.691 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:48 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:03:48.691 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:48 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:03:48.692 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:48 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:03:48.692 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:48 smithi083 ceph-mon[36260]: osdmap e362: 8 total, 8 up, 7 in 2024-09-20T19:03:48.694 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:03:48.717 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:48.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:48.718 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:03:48.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:48 smithi142 ceph-mon[32508]: pgmap v696: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 6.9 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 1.5 KiB/s rd, 67 MiB/s wr, 1.07k op/s; 40587/341964 objects misplaced (11.869%); 34 MiB/s, 537 objects/s recovering 2024-09-20T19:03:48.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:48 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:03:48.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:48 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:03:48.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:48 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:03:48.764 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:48 smithi142 ceph-mon[32508]: osdmap e362: 8 total, 8 up, 7 in 2024-09-20T19:03:48.819 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:48.854 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:48.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:48.856 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:48.883 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 89 16 116081 116065 81.489 50.875 0.0200332 0.0122659 2024-09-20T19:03:48.956 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:48.991 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:48.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:48.992 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:49.093 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:49.128 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:49.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:49.129 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:49.230 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:49.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:49.271 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:49.271 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:49.372 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:49.396 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:49.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:49.398 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:49.498 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:49.530 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:49.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:49.531 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:49.579 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T19:03:49.632 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:49.666 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:49.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:49.667 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:49.768 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:03:49.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:49.804 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:49.804 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:03:49.883 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 90 5 116859 116854 81.1314 49.3125 0.03544 0.0123192 2024-09-20T19:03:49.905 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:49.932 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:49.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:49.933 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:49.942 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:03:50.034 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:50.057 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:50.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:50.059 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:50.159 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:50.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:50.190 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:50.191 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:50.292 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:50.318 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:50.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:50.320 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:50.421 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:50.435 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-20T19:03:50.448 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:50.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:50.449 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:50.491 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:03:50.491 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:03:50.511 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:03:50.511 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:03:50.531 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:03:50.531 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:03:50.550 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:03:50.550 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:03:50.550 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:50.570 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:03:50.570 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:03:50.573 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:50.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:50.574 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:50.587 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:03:50.587 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:03:50.604 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:03:50.604 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:03:50.635 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:03:50.635 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:03:50.636 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:50 smithi083 ceph-mon[29421]: pgmap v698: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 7.0 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 661 B/s rd, 59 MiB/s wr, 938 op/s; 40817/343386 objects misplaced (11.887%); 17 MiB/s, 271 objects/s recovering 2024-09-20T19:03:50.636 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:50 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3462352510' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:03:50.636 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:50 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:03:50.636 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:50 smithi083 ceph-mon[36260]: pgmap v698: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 7.0 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 661 B/s rd, 59 MiB/s wr, 938 op/s; 40817/343386 objects misplaced (11.887%); 17 MiB/s, 271 objects/s recovering 2024-09-20T19:03:50.636 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:50 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3462352510' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:03:50.636 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:50 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:03:50.675 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:50.698 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:50.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:50.699 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:50.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:50 smithi142 ceph-mon[32508]: pgmap v698: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 7.0 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 661 B/s rd, 59 MiB/s wr, 938 op/s; 40817/343386 objects misplaced (11.887%); 17 MiB/s, 271 objects/s recovering 2024-09-20T19:03:50.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:50 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3462352510' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:03:50.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:50 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:03:50.800 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:50.828 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:50.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:50.829 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:50.883 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Total time run: 90.031 2024-09-20T19:03:50.883 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Total writes made: 116859 2024-09-20T19:03:50.883 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Write size: 65536 2024-09-20T19:03:50.883 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Object size: 65536 2024-09-20T19:03:50.883 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Bandwidth (MB/sec): 81.1241 2024-09-20T19:03:50.884 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Stddev Bandwidth: 44.2917 2024-09-20T19:03:50.884 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Max bandwidth (MB/sec): 217.25 2024-09-20T19:03:50.884 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Min bandwidth (MB/sec): 0 2024-09-20T19:03:50.884 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Average IOPS: 1297 2024-09-20T19:03:50.884 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Stddev IOPS: 708.668 2024-09-20T19:03:50.884 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Max IOPS: 3476 2024-09-20T19:03:50.884 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Min IOPS: 0 2024-09-20T19:03:50.884 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Average Latency(s): 0.0123206 2024-09-20T19:03:50.884 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Stddev Latency(s): 0.0296441 2024-09-20T19:03:50.884 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Max latency(s): 2.39565 2024-09-20T19:03:50.884 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Min latency(s): 0.00125834 2024-09-20T19:03:50.901 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Cleaning up (deleting benchmark objects) 2024-09-20T19:03:50.929 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:50.950 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:50.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:50.951 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:51.043 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:03:50 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:03:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:03:51.051 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:51.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:51.083 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:51.083 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:51.184 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:51.212 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:51.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:51.213 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:51.314 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:51.348 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:51.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:51.349 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:51.450 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:51.501 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:51.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:51.502 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:51.603 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:51.643 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:51.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:51.644 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:51.744 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:51 smithi142 ceph-mon[32508]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:03:51.744 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:51 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T19:03:51.744 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:51 smithi142 ceph-mon[32508]: osdmap e363: 8 total, 8 up, 7 in 2024-09-20T19:03:51.744 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:51 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1629170573' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:03:51.745 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:51.777 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:51.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:51.778 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:51.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:51 smithi083 ceph-mon[29421]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:03:51.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:51 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T19:03:51.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:51 smithi083 ceph-mon[29421]: osdmap e363: 8 total, 8 up, 7 in 2024-09-20T19:03:51.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:51 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1629170573' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:03:51.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:51 smithi083 ceph-mon[36260]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:03:51.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:51 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T19:03:51.794 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:51 smithi083 ceph-mon[36260]: osdmap e363: 8 total, 8 up, 7 in 2024-09-20T19:03:51.794 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:51 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1629170573' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:03:51.801 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-20T19:03:51.880 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:51.911 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:51.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:51.912 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:52.012 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:52.042 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:52.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:52.043 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:52.144 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:52.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:52.188 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:52.188 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:52.204 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:03:52.289 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:52.327 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:52.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:52.328 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:52.429 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:52.459 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:52.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:52.460 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:52.561 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:52.597 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:52.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:52.598 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:52.614 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:52 smithi083 ceph-mon[29421]: pgmap v700: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 7.0 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 760 B/s rd, 37 MiB/s wr, 588 op/s; 40817/343386 objects misplaced (11.887%); 17 MiB/s, 267 objects/s recovering 2024-09-20T19:03:52.615 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:52 smithi083 ceph-mon[36260]: pgmap v700: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 7.0 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 760 B/s rd, 37 MiB/s wr, 588 op/s; 40817/343386 objects misplaced (11.887%); 17 MiB/s, 267 objects/s recovering 2024-09-20T19:03:52.699 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:52.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:52.743 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:52.743 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:52.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:52 smithi142 ceph-mon[32508]: pgmap v700: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 7.0 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 760 B/s rd, 37 MiB/s wr, 588 op/s; 40817/343386 objects misplaced (11.887%); 17 MiB/s, 267 objects/s recovering 2024-09-20T19:03:52.844 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:03:52.880 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:52.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:52.881 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:03:52.982 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:53.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:53.024 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:53.024 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:53.118 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T19:03:53.118 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:03:53.125 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:03:53.162 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:53.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:53.164 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:03:53.265 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:53.301 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:53.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:53.302 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:53.402 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:53.435 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:53.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:53.436 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:53.492 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T19:03:53.536 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:53.573 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:53.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:53.574 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:53.575 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 7, 6] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 5, 6, 7, 4] 2024-09-20T19:03:53.575 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T19:03:53.576 INFO:tasks.thrashosds.thrasher:fixing pg num pool unique_pool_3 2024-09-20T19:03:53.576 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T19:03:53.675 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:53.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:53.724 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:53.724 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:53.825 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:53.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:53.872 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:53.873 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:53.973 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:54.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:54.008 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:54.008 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:54.037 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T19:03:54.109 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:54.146 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:54.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:54.147 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:54.247 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:03:54.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:54.283 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:54.283 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:03:54.384 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:54.418 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:54.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:54.420 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:54.520 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:54.551 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:54.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:54.552 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:54.653 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:54.690 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:54.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:54.691 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:54.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:54 smithi142 ceph-mon[32508]: pgmap v701: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 7.1 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 1.4 KiB/s rd, 51 MiB/s wr, 887 op/s; 38592/349563 objects misplaced (11.040%); 34 MiB/s, 548 objects/s recovering 2024-09-20T19:03:54.792 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:54.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:54 smithi083 ceph-mon[29421]: pgmap v701: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 7.1 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 1.4 KiB/s rd, 51 MiB/s wr, 887 op/s; 38592/349563 objects misplaced (11.040%); 34 MiB/s, 548 objects/s recovering 2024-09-20T19:03:54.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:54 smithi083 ceph-mon[36260]: pgmap v701: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 7.1 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 1.4 KiB/s rd, 51 MiB/s wr, 887 op/s; 38592/349563 objects misplaced (11.040%); 34 MiB/s, 548 objects/s recovering 2024-09-20T19:03:54.831 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:54.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:54.832 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:54.858 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_blocked_ops 2024-09-20T19:03:54.933 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:54.944 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:03:54.948 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:03:54.954 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:54.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:54.955 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:54.985 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:03:55.056 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:55.079 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:55.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:55.080 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:55.180 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:55.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:55.219 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:55.219 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:55.320 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:55.362 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:55.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:55.363 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:55.464 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:03:55.503 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:55.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:55.504 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:03:55.605 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:55.632 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:55.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:55.633 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:55.734 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:55.762 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:55.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:55.763 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:55.863 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:55.891 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:55.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:55.908 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:56.009 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:56.048 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:56.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:56.050 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:56.151 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:56.183 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:56.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:56.184 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:56.285 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:56.313 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:56.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:56.314 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:56.415 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:56.436 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:56.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:56.437 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:56.521 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:03:56.521 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:03:56.521 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:03:56.522 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:03:56.522 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:03:56.522 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:03:56.522 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:03:56.523 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:03:56.523 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:03:56.523 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:03:56.523 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:03:56.523 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:03:56.524 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:03:56.524 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:03:56.531 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:03:56.532 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:03:56.538 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:03:56.562 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:56.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:56.564 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:03:56.663 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T19:03:56.664 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:56 smithi083 ceph-mon[29421]: pgmap v702: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 7.1 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 767 B/s rd, 24 MiB/s wr, 451 op/s; 38592/349563 objects misplaced (11.040%); 20 MiB/s, 324 objects/s recovering 2024-09-20T19:03:56.664 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:56 smithi083 ceph-mon[29421]: osdmap e364: 8 total, 8 up, 7 in 2024-09-20T19:03:56.664 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:56 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T19:03:56.664 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:56 smithi083 ceph-mon[29421]: osdmap e365: 8 total, 8 up, 7 in 2024-09-20T19:03:56.664 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:56 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1774975810' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:03:56.665 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:56 smithi083 ceph-mon[36260]: pgmap v702: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 7.1 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 767 B/s rd, 24 MiB/s wr, 451 op/s; 38592/349563 objects misplaced (11.040%); 20 MiB/s, 324 objects/s recovering 2024-09-20T19:03:56.665 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:56 smithi083 ceph-mon[36260]: osdmap e364: 8 total, 8 up, 7 in 2024-09-20T19:03:56.665 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:56 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T19:03:56.665 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:56 smithi083 ceph-mon[36260]: osdmap e365: 8 total, 8 up, 7 in 2024-09-20T19:03:56.665 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:56 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1774975810' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:03:56.670 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:56.670 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-20T19:03:56.693 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:56.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:56.694 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:56.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:56 smithi142 ceph-mon[32508]: pgmap v702: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 7.1 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 767 B/s rd, 24 MiB/s wr, 451 op/s; 38592/349563 objects misplaced (11.040%); 20 MiB/s, 324 objects/s recovering 2024-09-20T19:03:56.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:56 smithi142 ceph-mon[32508]: osdmap e364: 8 total, 8 up, 7 in 2024-09-20T19:03:56.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:56 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T19:03:56.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:56 smithi142 ceph-mon[32508]: osdmap e365: 8 total, 8 up, 7 in 2024-09-20T19:03:56.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:56 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1774975810' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:03:56.795 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:56.818 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:56.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:56.819 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:56.920 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:56.942 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:56.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:56.943 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:57.043 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:57.073 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:57.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:57.074 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:57.175 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:57.214 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:57.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:57.215 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:57.316 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:57.351 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:57.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:57.352 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:57.453 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:57.490 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:57.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:57.491 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:57.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:57 smithi142 ceph-mon[32508]: from='client.16092 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:03:57.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:57 smithi142 ceph-mon[32508]: osdmap e366: 8 total, 8 up, 7 in 2024-09-20T19:03:57.513 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:03:57 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:03:57.565 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":704,"stamp":"2024-09-20T19:03:55.850464+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":7623809587,"num_objects":116521,"num_object_clones":0,"num_object_copies":349563,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":38592,"num_objects_unfound":0,"num_objects_dirty":116521,"num_whiteouts":0,"num_read":2098,"num_read_kb":1955,"num_write":118233,"num_write_kb":7480650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13993,"num_bytes_recovered":902111284,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79457,"ondisk_log_size":79457,"up":444,"acting":441,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":23748184,"kb_used_data":19901844,"kb_used_omap":10,"kb_used_meta":3845813,"kb_avail":632467880,"statfs":{"total":671965249536,"available":647647109120,"internally_reserved":0,"allocated":20379488256,"data_stored":20376747670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11157,"internal_metadata":3938112619},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,5,3,6,5,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":134873112,"num_objects":2059,"num_object_clones":0,"num_object_copies":6177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-2225,"num_objects_unfound":0,"num_objects_dirty":2059,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3135,"num_write_kb":166145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2595,"num_bytes_recovered":170065920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.739455"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":656,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.430556+0000","last_change":"2024-09-20T19:01:31.817967+0000","last_active":"2024-09-20T19:03:50.430556+0000","last_peered":"2024-09-20T19:03:50.430556+0000","last_clean":"2024-09-20T19:03:50.430556+0000","last_became_active":"2024-09-20T18:49:45.943201+0000","last_became_peered":"2024-09-20T18:49:45.943201+0000","last_unstale":"2024-09-20T19:03:50.430556+0000","last_undegraded":"2024-09-20T19:03:50.430556+0000","last_fullsized":"2024-09-20T19:03:50.430556+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:31.817909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:31.817909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:53.060041+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"273'15","reported_seq":613,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.630341+0000","last_change":"2024-09-20T19:03:14.973129+0000","last_active":"2024-09-20T19:03:50.630341+0000","last_peered":"2024-09-20T19:03:50.630341+0000","last_clean":"2024-09-20T19:03:50.630341+0000","last_became_active":"2024-09-20T19:03:14.945368+0000","last_became_peered":"2024-09-20T19:03:14.945368+0000","last_unstale":"2024-09-20T19:03:50.630341+0000","last_undegraded":"2024-09-20T19:03:50.630341+0000","last_fullsized":"2024-09-20T19:03:50.630341+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:03:10.214154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:10.214154+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:27.580882+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,3],"acting":[6,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":646,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.612703+0000","last_change":"2024-09-20T18:57:49.016078+0000","last_active":"2024-09-20T19:03:50.612703+0000","last_peered":"2024-09-20T19:03:50.612703+0000","last_clean":"2024-09-20T19:03:50.612703+0000","last_became_active":"2024-09-20T18:49:49.959514+0000","last_became_peered":"2024-09-20T18:49:49.959514+0000","last_unstale":"2024-09-20T19:03:50.612703+0000","last_undegraded":"2024-09-20T19:03:50.612703+0000","last_fullsized":"2024-09-20T19:03:50.612703+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:49.016006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:49.016006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":506,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.612912+0000","last_change":"2024-09-20T19:03:14.959169+0000","last_active":"2024-09-20T19:03:50.612912+0000","last_peered":"2024-09-20T19:03:50.612912+0000","last_clean":"2024-09-20T19:03:50.612912+0000","last_became_active":"2024-09-20T19:03:14.958700+0000","last_became_peered":"2024-09-20T19:03:14.958700+0000","last_unstale":"2024-09-20T19:03:50.612912+0000","last_undegraded":"2024-09-20T19:03:50.612912+0000","last_fullsized":"2024-09-20T19:03:50.612912+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:02.084375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:02.084375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":656,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.613304+0000","last_change":"2024-09-20T19:02:02.441633+0000","last_active":"2024-09-20T19:03:50.613304+0000","last_peered":"2024-09-20T19:03:50.613304+0000","last_clean":"2024-09-20T19:03:50.613304+0000","last_became_active":"2024-09-20T18:59:05.081081+0000","last_became_peered":"2024-09-20T18:59:05.081081+0000","last_unstale":"2024-09-20T19:03:50.613304+0000","last_undegraded":"2024-09-20T19:03:50.613304+0000","last_fullsized":"2024-09-20T19:03:50.613304+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:02.441577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:02.441577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":662,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.614266+0000","last_change":"2024-09-20T18:59:05.073093+0000","last_active":"2024-09-20T19:03:50.614266+0000","last_peered":"2024-09-20T19:03:50.614266+0000","last_clean":"2024-09-20T19:03:50.614266+0000","last_became_active":"2024-09-20T18:59:05.072675+0000","last_became_peered":"2024-09-20T18:59:05.072675+0000","last_unstale":"2024-09-20T19:03:50.614266+0000","last_undegraded":"2024-09-20T19:03:50.614266+0000","last_fullsized":"2024-09-20T19:03:50.614266+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:03.055929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:59:03.055929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"274'27","reported_seq":693,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.612765+0000","last_change":"2024-09-20T19:02:40.899215+0000","last_active":"2024-09-20T19:03:50.612765+0000","last_peered":"2024-09-20T19:03:50.612765+0000","last_clean":"2024-09-20T19:03:50.612765+0000","last_became_active":"2024-09-20T18:49:47.952150+0000","last_became_peered":"2024-09-20T18:49:47.952150+0000","last_unstale":"2024-09-20T19:03:50.612765+0000","last_undegraded":"2024-09-20T19:03:50.612765+0000","last_fullsized":"2024-09-20T19:03:50.612765+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'27","last_scrub_stamp":"2024-09-20T19:02:40.899139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:40.899139+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:02.856822+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":504,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.630261+0000","last_change":"2024-09-20T19:03:14.950167+0000","last_active":"2024-09-20T19:03:50.630261+0000","last_peered":"2024-09-20T19:03:50.630261+0000","last_clean":"2024-09-20T19:03:50.630261+0000","last_became_active":"2024-09-20T19:03:14.949859+0000","last_became_peered":"2024-09-20T19:03:14.949859+0000","last_unstale":"2024-09-20T19:03:50.630261+0000","last_undegraded":"2024-09-20T19:03:50.630261+0000","last_fullsized":"2024-09-20T19:03:50.630261+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:23.982813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:23.982813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"59'1","reported_seq":720,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.613840+0000","last_change":"2024-09-20T19:03:14.982740+0000","last_active":"2024-09-20T19:03:50.613840+0000","last_peered":"2024-09-20T19:03:50.613840+0000","last_clean":"2024-09-20T19:03:50.613840+0000","last_became_active":"2024-09-20T19:03:14.961672+0000","last_became_peered":"2024-09-20T19:03:14.961672+0000","last_unstale":"2024-09-20T19:03:50.613840+0000","last_undegraded":"2024-09-20T19:03:50.613840+0000","last_fullsized":"2024-09-20T19:03:50.613840+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:01:28.592653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:28.592653+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1308,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"273'15","reported_seq":570,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.614063+0000","last_change":"2024-09-20T19:03:14.983993+0000","last_active":"2024-09-20T19:03:50.614063+0000","last_peered":"2024-09-20T19:03:50.614063+0000","last_clean":"2024-09-20T19:03:50.614063+0000","last_became_active":"2024-09-20T19:03:14.956526+0000","last_became_peered":"2024-09-20T19:03:14.956526+0000","last_unstale":"2024-09-20T19:03:50.614063+0000","last_undegraded":"2024-09-20T19:03:50.614063+0000","last_fullsized":"2024-09-20T19:03:50.614063+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:02:42.114331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:42.114331+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:01.019356+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":654,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.613879+0000","last_change":"2024-09-20T19:01:51.564693+0000","last_active":"2024-09-20T19:03:50.613879+0000","last_peered":"2024-09-20T19:03:50.613879+0000","last_clean":"2024-09-20T19:03:50.613879+0000","last_became_active":"2024-09-20T18:49:49.961188+0000","last_became_peered":"2024-09-20T18:49:49.961188+0000","last_unstale":"2024-09-20T19:03:50.613879+0000","last_undegraded":"2024-09-20T19:03:50.613879+0000","last_fullsized":"2024-09-20T19:03:50.613879+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:51.564624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:51.564624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:02.846143+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":635,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.427897+0000","last_change":"2024-09-20T19:03:09.963421+0000","last_active":"2024-09-20T19:03:50.427897+0000","last_peered":"2024-09-20T19:03:50.427897+0000","last_clean":"2024-09-20T19:03:50.427897+0000","last_became_active":"2024-09-20T18:49:51.966715+0000","last_became_peered":"2024-09-20T18:49:51.966715+0000","last_unstale":"2024-09-20T19:03:50.427897+0000","last_undegraded":"2024-09-20T19:03:50.427897+0000","last_fullsized":"2024-09-20T19:03:50.427897+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:09.963298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:03:09.963298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:19.165030+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":672,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.613728+0000","last_change":"2024-09-20T19:03:14.961029+0000","last_active":"2024-09-20T19:03:50.613728+0000","last_peered":"2024-09-20T19:03:50.613728+0000","last_clean":"2024-09-20T19:03:50.613728+0000","last_became_active":"2024-09-20T19:03:14.960730+0000","last_became_peered":"2024-09-20T19:03:14.960730+0000","last_unstale":"2024-09-20T19:03:50.613728+0000","last_undegraded":"2024-09-20T19:03:50.613728+0000","last_fullsized":"2024-09-20T19:03:50.613728+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:48.520017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:48.520017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"273'7","reported_seq":564,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.613614+0000","last_change":"2024-09-20T19:03:14.985188+0000","last_active":"2024-09-20T19:03:50.613614+0000","last_peered":"2024-09-20T19:03:50.613614+0000","last_clean":"2024-09-20T19:03:50.613614+0000","last_became_active":"2024-09-20T19:03:14.953258+0000","last_became_peered":"2024-09-20T19:03:14.953258+0000","last_unstale":"2024-09-20T19:03:50.613614+0000","last_undegraded":"2024-09-20T19:03:50.613614+0000","last_fullsized":"2024-09-20T19:03:50.613614+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-20T18:58:33.660531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:33.660531+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":665,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.629176+0000","last_change":"2024-09-20T19:03:14.961082+0000","last_active":"2024-09-20T19:03:50.629176+0000","last_peered":"2024-09-20T19:03:50.629176+0000","last_clean":"2024-09-20T19:03:50.629176+0000","last_became_active":"2024-09-20T19:03:14.960775+0000","last_became_peered":"2024-09-20T19:03:14.960775+0000","last_unstale":"2024-09-20T19:03:50.629176+0000","last_undegraded":"2024-09-20T19:03:50.629176+0000","last_fullsized":"2024-09-20T19:03:50.629176+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:33.203312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:33.203312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":642,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.612918+0000","last_change":"2024-09-20T18:58:26.465305+0000","last_active":"2024-09-20T19:03:50.612918+0000","last_peered":"2024-09-20T19:03:50.612918+0000","last_clean":"2024-09-20T19:03:50.612918+0000","last_became_active":"2024-09-20T18:49:51.970663+0000","last_became_peered":"2024-09-20T18:49:51.970663+0000","last_unstale":"2024-09-20T19:03:50.612918+0000","last_undegraded":"2024-09-20T19:03:50.612918+0000","last_fullsized":"2024-09-20T19:03:50.612918+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:26.465264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:26.465264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":501,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.628966+0000","last_change":"2024-09-20T19:03:14.965568+0000","last_active":"2024-09-20T19:03:50.628966+0000","last_peered":"2024-09-20T19:03:50.628966+0000","last_clean":"2024-09-20T19:03:50.628966+0000","last_became_active":"2024-09-20T19:03:14.965259+0000","last_became_peered":"2024-09-20T19:03:14.965259+0000","last_unstale":"2024-09-20T19:03:50.628966+0000","last_undegraded":"2024-09-20T19:03:50.628966+0000","last_fullsized":"2024-09-20T19:03:50.628966+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:00.906652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:00.906652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":673,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.428951+0000","last_change":"2024-09-20T19:03:14.955611+0000","last_active":"2024-09-20T19:03:50.428951+0000","last_peered":"2024-09-20T19:03:50.428951+0000","last_clean":"2024-09-20T19:03:50.428951+0000","last_became_active":"2024-09-20T19:03:14.953751+0000","last_became_peered":"2024-09-20T19:03:14.953751+0000","last_unstale":"2024-09-20T19:03:50.428951+0000","last_undegraded":"2024-09-20T19:03:50.428951+0000","last_fullsized":"2024-09-20T19:03:50.428951+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:44.998147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:44.998147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"273'27","reported_seq":704,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.426671+0000","last_change":"2024-09-20T19:03:09.456720+0000","last_active":"2024-09-20T19:03:50.426671+0000","last_peered":"2024-09-20T19:03:50.426671+0000","last_clean":"2024-09-20T19:03:50.426671+0000","last_became_active":"2024-09-20T18:49:47.950939+0000","last_became_peered":"2024-09-20T18:49:47.950939+0000","last_unstale":"2024-09-20T19:03:50.426671+0000","last_undegraded":"2024-09-20T19:03:50.426671+0000","last_fullsized":"2024-09-20T19:03:50.426671+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'27","last_scrub_stamp":"2024-09-20T19:03:09.456631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:09.456631+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:26.973189+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":656,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.613391+0000","last_change":"2024-09-20T19:03:14.956903+0000","last_active":"2024-09-20T19:03:50.613391+0000","last_peered":"2024-09-20T19:03:50.613391+0000","last_clean":"2024-09-20T19:03:50.613391+0000","last_became_active":"2024-09-20T19:03:14.956533+0000","last_became_peered":"2024-09-20T19:03:14.956533+0000","last_unstale":"2024-09-20T19:03:50.613391+0000","last_undegraded":"2024-09-20T19:03:50.613391+0000","last_fullsized":"2024-09-20T19:03:50.613391+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:05.888974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:05.888974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":657,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.614222+0000","last_change":"2024-09-20T19:02:28.934597+0000","last_active":"2024-09-20T19:03:50.614222+0000","last_peered":"2024-09-20T19:03:50.614222+0000","last_clean":"2024-09-20T19:03:50.614222+0000","last_became_active":"2024-09-20T18:59:05.084496+0000","last_became_peered":"2024-09-20T18:59:05.084496+0000","last_unstale":"2024-09-20T19:03:50.614222+0000","last_undegraded":"2024-09-20T19:03:50.614222+0000","last_fullsized":"2024-09-20T19:03:50.614222+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:28.934264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:28.934264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"59'1","reported_seq":80,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.613058+0000","last_change":"2024-09-20T19:03:14.962321+0000","last_active":"2024-09-20T19:03:50.613058+0000","last_peered":"2024-09-20T19:03:50.613058+0000","last_clean":"2024-09-20T19:03:50.613058+0000","last_became_active":"2024-09-20T19:03:14.956472+0000","last_became_peered":"2024-09-20T19:03:14.956472+0000","last_unstale":"2024-09-20T19:03:50.613058+0000","last_undegraded":"2024-09-20T19:03:50.613058+0000","last_fullsized":"2024-09-20T19:03:50.613058+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:03:08.218812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:03:08.218812+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:22.426186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":993,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"274'18","reported_seq":688,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.613478+0000","last_change":"2024-09-20T19:03:14.975728+0000","last_active":"2024-09-20T19:03:50.613478+0000","last_peered":"2024-09-20T19:03:50.613478+0000","last_clean":"2024-09-20T19:03:50.613478+0000","last_became_active":"2024-09-20T19:03:14.946964+0000","last_became_peered":"2024-09-20T19:03:14.946964+0000","last_unstale":"2024-09-20T19:03:50.613478+0000","last_undegraded":"2024-09-20T19:03:50.613478+0000","last_fullsized":"2024-09-20T19:03:50.613478+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:57:09.818999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:09.818999+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":533,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.613463+0000","last_change":"2024-09-20T19:03:14.951418+0000","last_active":"2024-09-20T19:03:50.613463+0000","last_peered":"2024-09-20T19:03:50.613463+0000","last_clean":"2024-09-20T19:03:50.613463+0000","last_became_active":"2024-09-20T19:03:14.951127+0000","last_became_peered":"2024-09-20T19:03:14.951127+0000","last_unstale":"2024-09-20T19:03:50.613463+0000","last_undegraded":"2024-09-20T19:03:50.613463+0000","last_fullsized":"2024-09-20T19:03:50.613463+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:02.094473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:02.094473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":644,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.630773+0000","last_change":"2024-09-20T19:02:28.294390+0000","last_active":"2024-09-20T19:03:50.630773+0000","last_peered":"2024-09-20T19:03:50.630773+0000","last_clean":"2024-09-20T19:03:50.630773+0000","last_became_active":"2024-09-20T18:49:51.970058+0000","last_became_peered":"2024-09-20T18:49:51.970058+0000","last_unstale":"2024-09-20T19:03:50.630773+0000","last_undegraded":"2024-09-20T19:03:50.630773+0000","last_fullsized":"2024-09-20T19:03:50.630773+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T19:02:28.294292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:28.294292+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":661,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.612905+0000","last_change":"2024-09-20T19:03:14.968129+0000","last_active":"2024-09-20T19:03:50.612905+0000","last_peered":"2024-09-20T19:03:50.612905+0000","last_clean":"2024-09-20T19:03:50.612905+0000","last_became_active":"2024-09-20T19:03:14.955428+0000","last_became_peered":"2024-09-20T19:03:14.955428+0000","last_unstale":"2024-09-20T19:03:50.612905+0000","last_undegraded":"2024-09-20T19:03:50.612905+0000","last_fullsized":"2024-09-20T19:03:50.612905+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:57:18.032067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:18.032067+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"274'20","reported_seq":718,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.430321+0000","last_change":"2024-09-20T19:03:16.982915+0000","last_active":"2024-09-20T19:03:50.430321+0000","last_peered":"2024-09-20T19:03:50.430321+0000","last_clean":"2024-09-20T19:03:50.430321+0000","last_became_active":"2024-09-20T19:03:16.968636+0000","last_became_peered":"2024-09-20T19:03:16.968636+0000","last_unstale":"2024-09-20T19:03:50.430321+0000","last_undegraded":"2024-09-20T19:03:50.430321+0000","last_fullsized":"2024-09-20T19:03:50.430321+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'20","last_scrub_stamp":"2024-09-20T19:02:35.096979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:35.096979+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"352'13","reported_seq":320,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.629694+0000","last_change":"2024-09-20T19:03:14.978232+0000","last_active":"2024-09-20T19:03:50.629694+0000","last_peered":"2024-09-20T19:03:50.629694+0000","last_clean":"2024-09-20T19:03:50.629694+0000","last_became_active":"2024-09-20T19:03:14.949808+0000","last_became_peered":"2024-09-20T19:03:14.949808+0000","last_unstale":"2024-09-20T19:03:50.629694+0000","last_undegraded":"2024-09-20T19:03:50.629694+0000","last_fullsized":"2024-09-20T19:03:50.629694+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"308'12","last_scrub_stamp":"2024-09-20T19:01:44.541156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:44.541156+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"363'7246","reported_seq":18852,"reported_epoch":363,"state":"active+remapped+backfilling","last_fresh":"2024-09-20T19:03:51.319004+0000","last_change":"2024-09-20T19:03:17.031524+0000","last_active":"2024-09-20T19:03:51.319004+0000","last_peered":"2024-09-20T19:03:51.319004+0000","last_clean":"2024-09-20T19:03:13.895389+0000","last_became_active":"2024-09-20T19:03:16.167295+0000","last_became_peered":"2024-09-20T19:03:16.167295+0000","last_unstale":"2024-09-20T19:03:51.319004+0000","last_undegraded":"2024-09-20T19:03:51.319004+0000","last_fullsized":"2024-09-20T19:03:51.319004+0000","mapping_epoch":351,"log_start":"338'2400","ondisk_log_start":"338'2400","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4846,"log_dups_size":0,"ondisk_log_size":4846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":473169920,"num_objects":7220,"num_object_clones":0,"num_object_copies":21660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":787,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7246,"num_write_kb":462912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5755,"num_bytes_recovered":377159680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,4],"avail_no_missing":["7","0","4"],"object_location_counts":[{"shards":"0,4,7","objects":7220}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"274'27","reported_seq":714,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.427038+0000","last_change":"2024-09-20T19:03:14.978865+0000","last_active":"2024-09-20T19:03:50.427038+0000","last_peered":"2024-09-20T19:03:50.427038+0000","last_clean":"2024-09-20T19:03:50.427038+0000","last_became_active":"2024-09-20T19:03:14.944970+0000","last_became_peered":"2024-09-20T19:03:14.944970+0000","last_unstale":"2024-09-20T19:03:50.427038+0000","last_undegraded":"2024-09-20T19:03:50.427038+0000","last_fullsized":"2024-09-20T19:03:50.427038+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'27","last_scrub_stamp":"2024-09-20T19:02:27.897138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:27.897138+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":647,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.612729+0000","last_change":"2024-09-20T19:00:31.512527+0000","last_active":"2024-09-20T19:03:50.612729+0000","last_peered":"2024-09-20T19:03:50.612729+0000","last_clean":"2024-09-20T19:03:50.612729+0000","last_became_active":"2024-09-20T18:49:45.936104+0000","last_became_peered":"2024-09-20T18:49:45.936104+0000","last_unstale":"2024-09-20T19:03:50.612729+0000","last_undegraded":"2024-09-20T19:03:50.612729+0000","last_fullsized":"2024-09-20T19:03:50.612729+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:00:31.512472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:00:31.512472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"352'11","reported_seq":118,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.616769+0000","last_change":"2024-09-20T19:03:14.973138+0000","last_active":"2024-09-20T19:03:50.616769+0000","last_peered":"2024-09-20T19:03:50.616769+0000","last_clean":"2024-09-20T19:03:50.616769+0000","last_became_active":"2024-09-20T19:03:14.961478+0000","last_became_peered":"2024-09-20T19:03:14.961478+0000","last_unstale":"2024-09-20T19:03:50.616769+0000","last_undegraded":"2024-09-20T19:03:50.616769+0000","last_fullsized":"2024-09-20T19:03:50.616769+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"308'10","last_scrub_stamp":"2024-09-20T19:02:23.017626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:23.017626+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":643,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.613454+0000","last_change":"2024-09-20T19:02:22.736879+0000","last_active":"2024-09-20T19:03:50.613454+0000","last_peered":"2024-09-20T19:03:50.613454+0000","last_clean":"2024-09-20T19:03:50.613454+0000","last_became_active":"2024-09-20T18:49:51.971399+0000","last_became_peered":"2024-09-20T18:49:51.971399+0000","last_unstale":"2024-09-20T19:03:50.613454+0000","last_undegraded":"2024-09-20T19:03:50.613454+0000","last_fullsized":"2024-09-20T19:03:50.613454+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:22.736825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:22.736825+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:51.591260+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.e","version":"363'7435","reported_seq":7532,"reported_epoch":363,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-20T19:03:51.495997+0000","last_change":"2024-09-20T19:03:46.067883+0000","last_active":"2024-09-20T19:03:51.495997+0000","last_peered":"2024-09-20T19:03:51.495997+0000","last_clean":"2024-09-20T19:03:13.896248+0000","last_became_active":"2024-09-20T19:03:16.051298+0000","last_became_peered":"2024-09-20T19:03:16.051298+0000","last_unstale":"2024-09-20T19:03:51.495997+0000","last_undegraded":"2024-09-20T19:03:51.495997+0000","last_fullsized":"2024-09-20T19:03:51.495997+0000","mapping_epoch":351,"log_start":"338'2500","ondisk_log_start":"338'2500","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4935,"log_dups_size":0,"ondisk_log_size":4935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":482410496,"num_objects":7361,"num_object_clones":0,"num_object_copies":22083,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":7361,"num_objects_unfound":0,"num_objects_dirty":7361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7435,"num_write_kb":473472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,5],"acting":[6,2,4],"avail_no_missing":["6","2","4"],"object_location_counts":[{"shards":"2,4,6","objects":7361}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"273'30","reported_seq":710,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.612541+0000","last_change":"2024-09-20T19:03:06.038303+0000","last_active":"2024-09-20T19:03:50.612541+0000","last_peered":"2024-09-20T19:03:50.612541+0000","last_clean":"2024-09-20T19:03:50.612541+0000","last_became_active":"2024-09-20T18:49:48.499189+0000","last_became_peered":"2024-09-20T18:49:48.499189+0000","last_unstale":"2024-09-20T19:03:50.612541+0000","last_undegraded":"2024-09-20T19:03:50.612541+0000","last_fullsized":"2024-09-20T19:03:50.612541+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'30","last_scrub_stamp":"2024-09-20T19:03:06.038213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:06.038213+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:20.950961+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":647,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.614465+0000","last_change":"2024-09-20T19:02:22.806587+0000","last_active":"2024-09-20T19:03:50.614465+0000","last_peered":"2024-09-20T19:03:50.614465+0000","last_clean":"2024-09-20T19:03:50.614465+0000","last_became_active":"2024-09-20T18:49:45.940544+0000","last_became_peered":"2024-09-20T18:49:45.940544+0000","last_unstale":"2024-09-20T19:03:50.614465+0000","last_undegraded":"2024-09-20T19:03:50.614465+0000","last_fullsized":"2024-09-20T19:03:50.614465+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:22.806509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:22.806509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":536,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.612383+0000","last_change":"2024-09-20T19:03:14.952042+0000","last_active":"2024-09-20T19:03:50.612383+0000","last_peered":"2024-09-20T19:03:50.612383+0000","last_clean":"2024-09-20T19:03:50.612383+0000","last_became_active":"2024-09-20T19:03:14.951719+0000","last_became_peered":"2024-09-20T19:03:14.951719+0000","last_unstale":"2024-09-20T19:03:50.612383+0000","last_undegraded":"2024-09-20T19:03:50.612383+0000","last_fullsized":"2024-09-20T19:03:50.612383+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.229201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.229201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":527,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.614028+0000","last_change":"2024-09-20T19:03:14.952656+0000","last_active":"2024-09-20T19:03:50.614028+0000","last_peered":"2024-09-20T19:03:50.614028+0000","last_clean":"2024-09-20T19:03:50.614028+0000","last_became_active":"2024-09-20T19:03:14.952336+0000","last_became_peered":"2024-09-20T19:03:14.952336+0000","last_unstale":"2024-09-20T19:03:50.614028+0000","last_undegraded":"2024-09-20T19:03:50.614028+0000","last_fullsized":"2024-09-20T19:03:50.614028+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:27.704520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:27.704520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"363'7216","reported_seq":7303,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:51.502215+0000","last_change":"2024-09-20T19:02:11.828279+0000","last_active":"2024-09-20T19:03:51.502215+0000","last_peered":"2024-09-20T19:03:51.502215+0000","last_clean":"2024-09-20T19:03:51.502215+0000","last_became_active":"2024-09-20T19:02:11.827468+0000","last_became_peered":"2024-09-20T19:02:11.827468+0000","last_unstale":"2024-09-20T19:03:51.502215+0000","last_undegraded":"2024-09-20T19:03:51.502215+0000","last_fullsized":"2024-09-20T19:03:51.502215+0000","mapping_epoch":325,"log_start":"338'2355","ondisk_log_start":"338'2355","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4861,"log_dups_size":0,"ondisk_log_size":4861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470155264,"num_objects":7174,"num_object_clones":0,"num_object_copies":21522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7216,"num_write_kb":460480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"274'28","reported_seq":641,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.612929+0000","last_change":"2024-09-20T19:03:14.980995+0000","last_active":"2024-09-20T19:03:50.612929+0000","last_peered":"2024-09-20T19:03:50.612929+0000","last_clean":"2024-09-20T19:03:50.612929+0000","last_became_active":"2024-09-20T19:03:14.960136+0000","last_became_peered":"2024-09-20T19:03:14.960136+0000","last_unstale":"2024-09-20T19:03:50.612929+0000","last_undegraded":"2024-09-20T19:03:50.612929+0000","last_fullsized":"2024-09-20T19:03:50.612929+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'28","last_scrub_stamp":"2024-09-20T19:03:06.269623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:06.269623+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:24.425577+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":41,"num_read_kb":34,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":671,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.430408+0000","last_change":"2024-09-20T19:03:14.965158+0000","last_active":"2024-09-20T19:03:50.430408+0000","last_peered":"2024-09-20T19:03:50.430408+0000","last_clean":"2024-09-20T19:03:50.430408+0000","last_became_active":"2024-09-20T19:03:14.951317+0000","last_became_peered":"2024-09-20T19:03:14.951317+0000","last_unstale":"2024-09-20T19:03:50.430408+0000","last_undegraded":"2024-09-20T19:03:50.430408+0000","last_fullsized":"2024-09-20T19:03:50.430408+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:00:29.890681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:00:29.890681+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"353'12","reported_seq":861,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.612543+0000","last_change":"2024-09-20T19:03:17.027278+0000","last_active":"2024-09-20T19:03:50.612543+0000","last_peered":"2024-09-20T19:03:50.612543+0000","last_clean":"2024-09-20T19:03:50.612543+0000","last_became_active":"2024-09-20T19:03:16.989458+0000","last_became_peered":"2024-09-20T19:03:16.989458+0000","last_unstale":"2024-09-20T19:03:50.612543+0000","last_undegraded":"2024-09-20T19:03:50.612543+0000","last_fullsized":"2024-09-20T19:03:50.612543+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:58:23.114371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:23.114371+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":650,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.629094+0000","last_change":"2024-09-20T18:59:05.082117+0000","last_active":"2024-09-20T19:03:50.629094+0000","last_peered":"2024-09-20T19:03:50.629094+0000","last_clean":"2024-09-20T19:03:50.629094+0000","last_became_active":"2024-09-20T18:59:05.081814+0000","last_became_peered":"2024-09-20T18:59:05.081814+0000","last_unstale":"2024-09-20T19:03:50.629094+0000","last_undegraded":"2024-09-20T19:03:50.629094+0000","last_fullsized":"2024-09-20T19:03:50.629094+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:29.530001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:29.530001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"363'7244","reported_seq":19019,"reported_epoch":363,"state":"active+remapped+backfilling","last_fresh":"2024-09-20T19:03:51.322638+0000","last_change":"2024-09-20T19:03:16.195280+0000","last_active":"2024-09-20T19:03:51.322638+0000","last_peered":"2024-09-20T19:03:51.322638+0000","last_clean":"2024-09-20T19:03:13.927198+0000","last_became_active":"2024-09-20T19:03:16.168255+0000","last_became_peered":"2024-09-20T19:03:16.168255+0000","last_unstale":"2024-09-20T19:03:51.322638+0000","last_undegraded":"2024-09-20T19:03:51.322638+0000","last_fullsized":"2024-09-20T19:03:51.322638+0000","mapping_epoch":351,"log_start":"338'2409","ondisk_log_start":"338'2409","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4835,"log_dups_size":0,"ondisk_log_size":4835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":472252416,"num_objects":7206,"num_object_clones":0,"num_object_copies":21618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":725,"num_objects_unfound":0,"num_objects_dirty":7206,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7244,"num_write_kb":462400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5842,"num_bytes_recovered":382861312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,3],"acting":[7,0,4],"avail_no_missing":["7","0","4"],"object_location_counts":[{"shards":"0,4,7","objects":7206}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"274'20","reported_seq":727,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.612884+0000","last_change":"2024-09-20T19:03:14.970171+0000","last_active":"2024-09-20T19:03:50.612884+0000","last_peered":"2024-09-20T19:03:50.612884+0000","last_clean":"2024-09-20T19:03:50.612884+0000","last_became_active":"2024-09-20T19:03:14.957350+0000","last_became_peered":"2024-09-20T19:03:14.957350+0000","last_unstale":"2024-09-20T19:03:50.612884+0000","last_undegraded":"2024-09-20T19:03:50.612884+0000","last_fullsized":"2024-09-20T19:03:50.612884+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'20","last_scrub_stamp":"2024-09-20T19:02:35.930910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:35.930910+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:56.035276+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":671,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.614029+0000","last_change":"2024-09-20T19:02:30.686939+0000","last_active":"2024-09-20T19:03:50.614029+0000","last_peered":"2024-09-20T19:03:50.614029+0000","last_clean":"2024-09-20T19:03:50.614029+0000","last_became_active":"2024-09-20T18:49:45.942496+0000","last_became_peered":"2024-09-20T18:49:45.942496+0000","last_unstale":"2024-09-20T19:03:50.614029+0000","last_undegraded":"2024-09-20T19:03:50.614029+0000","last_fullsized":"2024-09-20T19:03:50.614029+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:30.686872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:30.686872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:51.214827+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"262'10","reported_seq":848,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.428325+0000","last_change":"2024-09-20T19:02:04.994109+0000","last_active":"2024-09-20T19:03:50.428325+0000","last_peered":"2024-09-20T19:03:50.428325+0000","last_clean":"2024-09-20T19:03:50.428325+0000","last_became_active":"2024-09-20T18:59:05.080183+0000","last_became_peered":"2024-09-20T18:59:05.080183+0000","last_unstale":"2024-09-20T19:03:50.428325+0000","last_undegraded":"2024-09-20T19:03:50.428325+0000","last_fullsized":"2024-09-20T19:03:50.428325+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'10","last_scrub_stamp":"2024-09-20T19:02:04.994047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:04.994047+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":628,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.428002+0000","last_change":"2024-09-20T19:01:37.900410+0000","last_active":"2024-09-20T19:03:50.428002+0000","last_peered":"2024-09-20T19:03:50.428002+0000","last_clean":"2024-09-20T19:03:50.428002+0000","last_became_active":"2024-09-20T18:49:51.964780+0000","last_became_peered":"2024-09-20T18:49:51.964780+0000","last_unstale":"2024-09-20T19:03:50.428002+0000","last_undegraded":"2024-09-20T19:03:50.428002+0000","last_fullsized":"2024-09-20T19:03:50.428002+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:37.900328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:37.900328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:02.124500+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.d","version":"363'7502","reported_seq":8917,"reported_epoch":363,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-20T19:03:51.475121+0000","last_change":"2024-09-20T19:03:16.400294+0000","last_active":"2024-09-20T19:03:51.475121+0000","last_peered":"2024-09-20T19:03:51.475121+0000","last_clean":"2024-09-20T19:03:13.933264+0000","last_became_active":"2024-09-20T19:03:16.292336+0000","last_became_peered":"2024-09-20T19:03:16.292336+0000","last_unstale":"2024-09-20T19:03:51.475121+0000","last_undegraded":"2024-09-20T19:03:51.475121+0000","last_fullsized":"2024-09-20T19:03:15.946661+0000","mapping_epoch":351,"log_start":"338'2617","ondisk_log_start":"338'2617","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4885,"log_dups_size":0,"ondisk_log_size":4885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":487915544,"num_objects":7446,"num_object_clones":0,"num_object_copies":22338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":5254,"num_objects_unfound":0,"num_objects_dirty":7446,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7502,"num_write_kb":478273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":663,"num_bytes_recovered":43450368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0],"avail_no_missing":["6","0","4"],"object_location_counts":[{"shards":"0,4,6","objects":5254},{"shards":"0,6","objects":2192}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"274'29","reported_seq":695,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.428503+0000","last_change":"2024-09-20T19:00:04.810907+0000","last_active":"2024-09-20T19:03:50.428503+0000","last_peered":"2024-09-20T19:03:50.428503+0000","last_clean":"2024-09-20T19:03:50.428503+0000","last_became_active":"2024-09-20T18:49:48.497736+0000","last_became_peered":"2024-09-20T18:49:48.497736+0000","last_unstale":"2024-09-20T19:03:50.428503+0000","last_undegraded":"2024-09-20T19:03:50.428503+0000","last_fullsized":"2024-09-20T19:03:50.428503+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'29","last_scrub_stamp":"2024-09-20T19:00:04.810831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:00:04.810831+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":647,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.613737+0000","last_change":"2024-09-20T19:02:41.859231+0000","last_active":"2024-09-20T19:03:50.613737+0000","last_peered":"2024-09-20T19:03:50.613737+0000","last_clean":"2024-09-20T19:03:50.613737+0000","last_became_active":"2024-09-20T18:49:45.942253+0000","last_became_peered":"2024-09-20T18:49:45.942253+0000","last_unstale":"2024-09-20T19:03:50.613737+0000","last_undegraded":"2024-09-20T19:03:50.613737+0000","last_fullsized":"2024-09-20T19:03:50.613737+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:41.859168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:41.859168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:03.925342+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":234,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.613387+0000","last_change":"2024-09-20T19:02:27.639446+0000","last_active":"2024-09-20T19:03:50.613387+0000","last_peered":"2024-09-20T19:03:50.613387+0000","last_clean":"2024-09-20T19:03:50.613387+0000","last_became_active":"2024-09-20T18:59:05.085636+0000","last_became_peered":"2024-09-20T18:59:05.085636+0000","last_unstale":"2024-09-20T19:03:50.613387+0000","last_undegraded":"2024-09-20T19:03:50.613387+0000","last_fullsized":"2024-09-20T19:03:50.613387+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:27.639384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:27.639384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":665,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.612607+0000","last_change":"2024-09-20T19:02:23.886386+0000","last_active":"2024-09-20T19:03:50.612607+0000","last_peered":"2024-09-20T19:03:50.612607+0000","last_clean":"2024-09-20T19:03:50.612607+0000","last_became_active":"2024-09-20T18:59:05.085131+0000","last_became_peered":"2024-09-20T18:59:05.085131+0000","last_unstale":"2024-09-20T19:03:50.612607+0000","last_undegraded":"2024-09-20T19:03:50.612607+0000","last_fullsized":"2024-09-20T19:03:50.612607+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:23.886308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:23.886308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"363'7223","reported_seq":7374,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:51.511690+0000","last_change":"2024-09-20T19:03:15.459244+0000","last_active":"2024-09-20T19:03:51.511690+0000","last_peered":"2024-09-20T19:03:51.511690+0000","last_clean":"2024-09-20T19:03:51.511690+0000","last_became_active":"2024-09-20T19:02:11.827305+0000","last_became_peered":"2024-09-20T19:02:11.827305+0000","last_unstale":"2024-09-20T19:03:51.511690+0000","last_undegraded":"2024-09-20T19:03:51.511690+0000","last_fullsized":"2024-09-20T19:03:51.511690+0000","mapping_epoch":325,"log_start":"338'2302","ondisk_log_start":"338'2302","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"350'5605","last_scrub_stamp":"2024-09-20T19:03:15.459174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:03:15.459174+0000","objects_scrubbed":5605,"log_size":4921,"log_dups_size":0,"ondisk_log_size":4921,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:35.506042+0000","scrub_duration":598,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":468123648,"num_objects":7143,"num_object_clones":0,"num_object_copies":21429,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7143,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7223,"num_write_kb":459712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"353'41","reported_seq":924,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.613123+0000","last_change":"2024-09-20T19:03:17.024385+0000","last_active":"2024-09-20T19:03:50.613123+0000","last_peered":"2024-09-20T19:03:50.613123+0000","last_clean":"2024-09-20T19:03:50.613123+0000","last_became_active":"2024-09-20T19:03:16.976916+0000","last_became_peered":"2024-09-20T19:03:16.976916+0000","last_unstale":"2024-09-20T19:03:50.613123+0000","last_undegraded":"2024-09-20T19:03:50.613123+0000","last_fullsized":"2024-09-20T19:03:50.613123+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'40","last_scrub_stamp":"2024-09-20T19:02:36.900916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:36.900916+0000","objects_scrubbed":10,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:05.286209+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":549,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.428876+0000","last_change":"2024-09-20T19:03:14.951169+0000","last_active":"2024-09-20T19:03:50.428876+0000","last_peered":"2024-09-20T19:03:50.428876+0000","last_clean":"2024-09-20T19:03:50.428876+0000","last_became_active":"2024-09-20T19:03:14.950852+0000","last_became_peered":"2024-09-20T19:03:14.950852+0000","last_unstale":"2024-09-20T19:03:50.428876+0000","last_undegraded":"2024-09-20T19:03:50.428876+0000","last_fullsized":"2024-09-20T19:03:50.428876+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:00:31.630361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:00:31.630361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":711,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.613134+0000","last_change":"2024-09-20T19:03:11.910684+0000","last_active":"2024-09-20T19:03:50.613134+0000","last_peered":"2024-09-20T19:03:50.613134+0000","last_clean":"2024-09-20T19:03:50.613134+0000","last_became_active":"2024-09-20T18:59:05.086786+0000","last_became_peered":"2024-09-20T18:59:05.086786+0000","last_unstale":"2024-09-20T19:03:50.613134+0000","last_undegraded":"2024-09-20T19:03:50.613134+0000","last_fullsized":"2024-09-20T19:03:50.613134+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-20T19:03:11.909394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:03:11.909394+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:35.264669+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0.0012781159999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":674,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.429307+0000","last_change":"2024-09-20T19:03:15.034384+0000","last_active":"2024-09-20T19:03:50.429307+0000","last_peered":"2024-09-20T19:03:50.429307+0000","last_clean":"2024-09-20T19:03:50.429307+0000","last_became_active":"2024-09-20T19:03:14.948079+0000","last_became_peered":"2024-09-20T19:03:14.948079+0000","last_unstale":"2024-09-20T19:03:50.429307+0000","last_undegraded":"2024-09-20T19:03:50.429307+0000","last_fullsized":"2024-09-20T19:03:50.429307+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:15.034340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:03:15.034340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:37.241809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":645,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.614100+0000","last_change":"2024-09-20T19:03:14.949138+0000","last_active":"2024-09-20T19:03:50.614100+0000","last_peered":"2024-09-20T19:03:50.614100+0000","last_clean":"2024-09-20T19:03:50.614100+0000","last_became_active":"2024-09-20T19:03:14.948815+0000","last_became_peered":"2024-09-20T19:03:14.948815+0000","last_unstale":"2024-09-20T19:03:50.614100+0000","last_undegraded":"2024-09-20T19:03:50.614100+0000","last_fullsized":"2024-09-20T19:03:50.614100+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:04.526683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:04.526683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"363'7393","reported_seq":7480,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:51.449447+0000","last_change":"2024-09-20T19:02:11.825132+0000","last_active":"2024-09-20T19:03:51.449447+0000","last_peered":"2024-09-20T19:03:51.449447+0000","last_clean":"2024-09-20T19:03:51.449447+0000","last_became_active":"2024-09-20T19:02:11.824846+0000","last_became_peered":"2024-09-20T19:02:11.824846+0000","last_unstale":"2024-09-20T19:03:51.449447+0000","last_undegraded":"2024-09-20T19:03:51.449447+0000","last_fullsized":"2024-09-20T19:03:51.449447+0000","mapping_epoch":325,"log_start":"338'2227","ondisk_log_start":"338'2227","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":5166,"log_dups_size":0,"ondisk_log_size":5166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":481492992,"num_objects":7347,"num_object_clones":0,"num_object_copies":22041,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7393,"num_write_kb":471680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"273'17","reported_seq":692,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.613694+0000","last_change":"2024-09-20T19:02:24.853473+0000","last_active":"2024-09-20T19:03:50.613694+0000","last_peered":"2024-09-20T19:03:50.613694+0000","last_clean":"2024-09-20T19:03:50.613694+0000","last_became_active":"2024-09-20T18:49:47.950904+0000","last_became_peered":"2024-09-20T18:49:47.950904+0000","last_unstale":"2024-09-20T19:03:50.613694+0000","last_undegraded":"2024-09-20T19:03:50.613694+0000","last_fullsized":"2024-09-20T19:03:50.613694+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'17","last_scrub_stamp":"2024-09-20T19:02:24.853374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:24.853374+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":36,"num_read_kb":25,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":678,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.612612+0000","last_change":"2024-09-20T19:02:29.840761+0000","last_active":"2024-09-20T19:03:50.612612+0000","last_peered":"2024-09-20T19:03:50.612612+0000","last_clean":"2024-09-20T19:03:50.612612+0000","last_became_active":"2024-09-20T18:59:05.079789+0000","last_became_peered":"2024-09-20T18:59:05.079789+0000","last_unstale":"2024-09-20T19:03:50.612612+0000","last_undegraded":"2024-09-20T19:03:50.612612+0000","last_fullsized":"2024-09-20T19:03:50.612612+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:29.840601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:29.840601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":659,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.426809+0000","last_change":"2024-09-20T19:01:58.789304+0000","last_active":"2024-09-20T19:03:50.426809+0000","last_peered":"2024-09-20T19:03:50.426809+0000","last_clean":"2024-09-20T19:03:50.426809+0000","last_became_active":"2024-09-20T18:49:44.937135+0000","last_became_peered":"2024-09-20T18:49:44.937135+0000","last_unstale":"2024-09-20T19:03:50.426809+0000","last_undegraded":"2024-09-20T19:03:50.426809+0000","last_fullsized":"2024-09-20T19:03:50.426809+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:01:58.789251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:01:58.789251+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:48.326383+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00095558800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":634,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.427228+0000","last_change":"2024-09-20T19:01:17.747452+0000","last_active":"2024-09-20T19:03:50.427228+0000","last_peered":"2024-09-20T19:03:50.427228+0000","last_clean":"2024-09-20T19:03:50.427228+0000","last_became_active":"2024-09-20T18:49:49.954424+0000","last_became_peered":"2024-09-20T18:49:49.954424+0000","last_unstale":"2024-09-20T19:03:50.427228+0000","last_undegraded":"2024-09-20T19:03:50.427228+0000","last_fullsized":"2024-09-20T19:03:50.427228+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:17.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:17.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":645,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.629575+0000","last_change":"2024-09-20T19:01:15.203591+0000","last_active":"2024-09-20T19:03:50.629575+0000","last_peered":"2024-09-20T19:03:50.629575+0000","last_clean":"2024-09-20T19:03:50.629575+0000","last_became_active":"2024-09-20T18:59:05.081459+0000","last_became_peered":"2024-09-20T18:59:05.081459+0000","last_unstale":"2024-09-20T19:03:50.629575+0000","last_undegraded":"2024-09-20T19:03:50.629575+0000","last_fullsized":"2024-09-20T19:03:50.629575+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:15.203529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:15.203529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:52.197162+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"363'7289","reported_seq":7376,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:52.412266+0000","last_change":"2024-09-20T19:02:11.828477+0000","last_active":"2024-09-20T19:03:52.412266+0000","last_peered":"2024-09-20T19:03:52.412266+0000","last_clean":"2024-09-20T19:03:52.412266+0000","last_became_active":"2024-09-20T19:02:11.827996+0000","last_became_peered":"2024-09-20T19:02:11.827996+0000","last_unstale":"2024-09-20T19:03:52.412266+0000","last_undegraded":"2024-09-20T19:03:52.412266+0000","last_fullsized":"2024-09-20T19:03:52.412266+0000","mapping_epoch":325,"log_start":"340'2984","ondisk_log_start":"340'2984","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4305,"log_dups_size":0,"ondisk_log_size":4305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":467730432,"num_objects":7137,"num_object_clones":0,"num_object_copies":21411,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7289,"num_write_kb":461632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"273'24","reported_seq":696,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.430310+0000","last_change":"2024-09-20T18:59:06.022021+0000","last_active":"2024-09-20T19:03:50.430310+0000","last_peered":"2024-09-20T19:03:50.430310+0000","last_clean":"2024-09-20T19:03:50.430310+0000","last_became_active":"2024-09-20T18:49:47.949230+0000","last_became_peered":"2024-09-20T18:49:47.949230+0000","last_unstale":"2024-09-20T19:03:50.430310+0000","last_undegraded":"2024-09-20T19:03:50.430310+0000","last_fullsized":"2024-09-20T19:03:50.430310+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:59:06.021962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:59:06.021962+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":674,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.428709+0000","last_change":"2024-09-20T19:03:14.954738+0000","last_active":"2024-09-20T19:03:50.428709+0000","last_peered":"2024-09-20T19:03:50.428709+0000","last_clean":"2024-09-20T19:03:50.428709+0000","last_became_active":"2024-09-20T19:03:14.954260+0000","last_became_peered":"2024-09-20T19:03:14.954260+0000","last_unstale":"2024-09-20T19:03:50.428709+0000","last_undegraded":"2024-09-20T19:03:50.428709+0000","last_fullsized":"2024-09-20T19:03:50.428709+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:24.984022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:24.984022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":1507,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.630175+0000","last_change":"2024-09-20T19:02:02.494344+0000","last_active":"2024-09-20T19:03:50.630175+0000","last_peered":"2024-09-20T19:03:50.630175+0000","last_clean":"2024-09-20T19:03:50.630175+0000","last_became_active":"2024-09-20T18:49:44.936819+0000","last_became_peered":"2024-09-20T18:49:44.936819+0000","last_unstale":"2024-09-20T19:03:50.630175+0000","last_undegraded":"2024-09-20T19:03:50.630175+0000","last_fullsized":"2024-09-20T19:03:50.630175+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-20T19:02:02.494291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:02:02.494291+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0010996199999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":839,"num_read_kb":834,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":655,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.613977+0000","last_change":"2024-09-20T19:02:27.785566+0000","last_active":"2024-09-20T19:03:50.613977+0000","last_peered":"2024-09-20T19:03:50.613977+0000","last_clean":"2024-09-20T19:03:50.613977+0000","last_became_active":"2024-09-20T18:49:49.961066+0000","last_became_peered":"2024-09-20T18:49:49.961066+0000","last_unstale":"2024-09-20T19:03:50.613977+0000","last_undegraded":"2024-09-20T19:03:50.613977+0000","last_fullsized":"2024-09-20T19:03:50.613977+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:27.785524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:27.785524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":651,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.612817+0000","last_change":"2024-09-20T19:02:26.861253+0000","last_active":"2024-09-20T19:03:50.612817+0000","last_peered":"2024-09-20T19:03:50.612817+0000","last_clean":"2024-09-20T19:03:50.612817+0000","last_became_active":"2024-09-20T18:49:51.970343+0000","last_became_peered":"2024-09-20T18:49:51.970343+0000","last_unstale":"2024-09-20T19:03:50.612817+0000","last_undegraded":"2024-09-20T19:03:50.612817+0000","last_fullsized":"2024-09-20T19:03:50.612817+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:26.861185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:26.861185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"363'7306","reported_seq":7393,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:51.495651+0000","last_change":"2024-09-20T19:02:11.828065+0000","last_active":"2024-09-20T19:03:51.495651+0000","last_peered":"2024-09-20T19:03:51.495651+0000","last_clean":"2024-09-20T19:03:51.495651+0000","last_became_active":"2024-09-20T19:02:11.827628+0000","last_became_peered":"2024-09-20T19:02:11.827628+0000","last_unstale":"2024-09-20T19:03:51.495651+0000","last_undegraded":"2024-09-20T19:03:51.495651+0000","last_fullsized":"2024-09-20T19:03:51.495651+0000","mapping_epoch":325,"log_start":"338'2427","ondisk_log_start":"338'2427","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4879,"log_dups_size":0,"ondisk_log_size":4879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":475267072,"num_objects":7252,"num_object_clones":0,"num_object_copies":21756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7252,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":465856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"352'33","reported_seq":358,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.633054+0000","last_change":"2024-09-20T19:03:14.979177+0000","last_active":"2024-09-20T19:03:50.633054+0000","last_peered":"2024-09-20T19:03:50.633054+0000","last_clean":"2024-09-20T19:03:50.633054+0000","last_became_active":"2024-09-20T19:03:14.948410+0000","last_became_peered":"2024-09-20T19:03:14.948410+0000","last_unstale":"2024-09-20T19:03:50.633054+0000","last_undegraded":"2024-09-20T19:03:50.633054+0000","last_fullsized":"2024-09-20T19:03:50.633054+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"308'32","last_scrub_stamp":"2024-09-20T19:02:25.699337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:25.699337+0000","objects_scrubbed":8,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":308,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":669,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.614156+0000","last_change":"2024-09-20T19:02:31.965132+0000","last_active":"2024-09-20T19:03:50.614156+0000","last_peered":"2024-09-20T19:03:50.614156+0000","last_clean":"2024-09-20T19:03:50.614156+0000","last_became_active":"2024-09-20T18:59:05.079111+0000","last_became_peered":"2024-09-20T18:59:05.079111+0000","last_unstale":"2024-09-20T19:03:50.614156+0000","last_undegraded":"2024-09-20T19:03:50.614156+0000","last_fullsized":"2024-09-20T19:03:50.614156+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:31.965066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:31.965066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":725,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.629958+0000","last_change":"2024-09-20T19:03:14.368338+0000","last_active":"2024-09-20T19:03:50.629958+0000","last_peered":"2024-09-20T19:03:50.629958+0000","last_clean":"2024-09-20T19:03:50.629958+0000","last_became_active":"2024-09-20T18:59:05.086103+0000","last_became_peered":"2024-09-20T18:59:05.086103+0000","last_unstale":"2024-09-20T19:03:50.629958+0000","last_undegraded":"2024-09-20T19:03:50.629958+0000","last_fullsized":"2024-09-20T19:03:50.629958+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:03:14.368279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:03:14.368279+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:32.552795+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":632,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.630851+0000","last_change":"2024-09-20T18:58:23.545370+0000","last_active":"2024-09-20T19:03:50.630851+0000","last_peered":"2024-09-20T19:03:50.630851+0000","last_clean":"2024-09-20T19:03:50.630851+0000","last_became_active":"2024-09-20T18:49:49.956128+0000","last_became_peered":"2024-09-20T18:49:49.956128+0000","last_unstale":"2024-09-20T19:03:50.630851+0000","last_undegraded":"2024-09-20T19:03:50.630851+0000","last_fullsized":"2024-09-20T19:03:50.630851+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:23.545321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:23.545321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":669,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.613967+0000","last_change":"2024-09-20T19:03:14.959711+0000","last_active":"2024-09-20T19:03:50.613967+0000","last_peered":"2024-09-20T19:03:50.613967+0000","last_clean":"2024-09-20T19:03:50.613967+0000","last_became_active":"2024-09-20T19:03:14.959411+0000","last_became_peered":"2024-09-20T19:03:14.959411+0000","last_unstale":"2024-09-20T19:03:50.613967+0000","last_undegraded":"2024-09-20T19:03:50.613967+0000","last_fullsized":"2024-09-20T19:03:50.613967+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:30.557368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:30.557368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.0","version":"363'7337","reported_seq":8769,"reported_epoch":363,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-20T19:03:51.505550+0000","last_change":"2024-09-20T19:03:16.155496+0000","last_active":"2024-09-20T19:03:51.505550+0000","last_peered":"2024-09-20T19:03:51.505550+0000","last_clean":"2024-09-20T19:03:13.922116+0000","last_became_active":"2024-09-20T19:03:16.128779+0000","last_became_peered":"2024-09-20T19:03:16.128779+0000","last_unstale":"2024-09-20T19:03:51.505550+0000","last_undegraded":"2024-09-20T19:03:51.505550+0000","last_fullsized":"2024-09-20T19:03:15.944421+0000","mapping_epoch":351,"log_start":"338'2439","ondisk_log_start":"338'2439","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4898,"log_dups_size":0,"ondisk_log_size":4898,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":477298688,"num_objects":7283,"num_object_clones":0,"num_object_copies":21849,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":5065,"num_objects_unfound":0,"num_objects_dirty":7283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7337,"num_write_kb":467840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":672,"num_bytes_recovered":44040192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1],"avail_no_missing":["6","1","4"],"object_location_counts":[{"shards":"1,4,6","objects":5065},{"shards":"1,6","objects":2218}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"274'19","reported_seq":602,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.628889+0000","last_change":"2024-09-20T19:03:14.982340+0000","last_active":"2024-09-20T19:03:50.628889+0000","last_peered":"2024-09-20T19:03:50.628889+0000","last_clean":"2024-09-20T19:03:50.628889+0000","last_became_active":"2024-09-20T19:03:14.948357+0000","last_became_peered":"2024-09-20T19:03:14.948357+0000","last_unstale":"2024-09-20T19:03:50.628889+0000","last_undegraded":"2024-09-20T19:03:50.628889+0000","last_fullsized":"2024-09-20T19:03:50.628889+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:58:24.616580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:24.616580+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,3],"acting":[6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":527,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.426311+0000","last_change":"2024-09-20T19:03:14.947842+0000","last_active":"2024-09-20T19:03:50.426311+0000","last_peered":"2024-09-20T19:03:50.426311+0000","last_clean":"2024-09-20T19:03:50.426311+0000","last_became_active":"2024-09-20T19:03:14.947457+0000","last_became_peered":"2024-09-20T19:03:14.947457+0000","last_unstale":"2024-09-20T19:03:50.426311+0000","last_undegraded":"2024-09-20T19:03:50.426311+0000","last_fullsized":"2024-09-20T19:03:50.426311+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.693993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.693993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:23.404776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,1],"acting":[2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":687,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.613679+0000","last_change":"2024-09-20T19:03:14.963776+0000","last_active":"2024-09-20T19:03:50.613679+0000","last_peered":"2024-09-20T19:03:50.613679+0000","last_clean":"2024-09-20T19:03:50.613679+0000","last_became_active":"2024-09-20T19:03:14.962949+0000","last_became_peered":"2024-09-20T19:03:14.962949+0000","last_unstale":"2024-09-20T19:03:50.613679+0000","last_undegraded":"2024-09-20T19:03:50.613679+0000","last_fullsized":"2024-09-20T19:03:50.613679+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:38.463404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:38.463404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":649,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.613378+0000","last_change":"2024-09-20T18:59:05.080255+0000","last_active":"2024-09-20T19:03:50.613378+0000","last_peered":"2024-09-20T19:03:50.613378+0000","last_clean":"2024-09-20T19:03:50.613378+0000","last_became_active":"2024-09-20T18:59:05.079897+0000","last_became_peered":"2024-09-20T18:59:05.079897+0000","last_unstale":"2024-09-20T19:03:50.613378+0000","last_undegraded":"2024-09-20T19:03:50.613378+0000","last_fullsized":"2024-09-20T19:03:50.613378+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:01.083156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:59:01.083156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"363'7469","reported_seq":7632,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:51.466895+0000","last_change":"2024-09-20T19:03:38.975534+0000","last_active":"2024-09-20T19:03:51.466895+0000","last_peered":"2024-09-20T19:03:51.466895+0000","last_clean":"2024-09-20T19:03:51.466895+0000","last_became_active":"2024-09-20T19:02:11.825011+0000","last_became_peered":"2024-09-20T19:02:11.825011+0000","last_unstale":"2024-09-20T19:03:51.466895+0000","last_undegraded":"2024-09-20T19:03:51.466895+0000","last_fullsized":"2024-09-20T19:03:51.466895+0000","mapping_epoch":325,"log_start":"336'1828","ondisk_log_start":"336'1828","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"358'6900","last_scrub_stamp":"2024-09-20T19:03:38.975306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:03:38.975306+0000","objects_scrubbed":6865,"log_size":5641,"log_dups_size":0,"ondisk_log_size":5641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:46.960940+0000","scrub_duration":1655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":485425152,"num_objects":7407,"num_object_clones":0,"num_object_copies":22221,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7469,"num_write_kb":476032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"274'31","reported_seq":596,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.426838+0000","last_change":"2024-09-20T19:03:14.986988+0000","last_active":"2024-09-20T19:03:50.426838+0000","last_peered":"2024-09-20T19:03:50.426838+0000","last_clean":"2024-09-20T19:03:50.426838+0000","last_became_active":"2024-09-20T19:03:14.954783+0000","last_became_peered":"2024-09-20T19:03:14.954783+0000","last_unstale":"2024-09-20T19:03:50.426838+0000","last_undegraded":"2024-09-20T19:03:50.426838+0000","last_fullsized":"2024-09-20T19:03:50.426838+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'31","last_scrub_stamp":"2024-09-20T19:03:12.196851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:12.196851+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:29.260709+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":49,"num_read_kb":39,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":244,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.613467+0000","last_change":"2024-09-20T19:03:14.963804+0000","last_active":"2024-09-20T19:03:50.613467+0000","last_peered":"2024-09-20T19:03:50.613467+0000","last_clean":"2024-09-20T19:03:50.613467+0000","last_became_active":"2024-09-20T19:03:14.959291+0000","last_became_peered":"2024-09-20T19:03:14.959291+0000","last_unstale":"2024-09-20T19:03:50.613467+0000","last_undegraded":"2024-09-20T19:03:50.613467+0000","last_fullsized":"2024-09-20T19:03:50.613467+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:42.479489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:42.479489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,3],"acting":[6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":665,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.612904+0000","last_change":"2024-09-20T19:03:14.948543+0000","last_active":"2024-09-20T19:03:50.612904+0000","last_peered":"2024-09-20T19:03:50.612904+0000","last_clean":"2024-09-20T19:03:50.612904+0000","last_became_active":"2024-09-20T19:03:14.948239+0000","last_became_peered":"2024-09-20T19:03:14.948239+0000","last_unstale":"2024-09-20T19:03:50.612904+0000","last_undegraded":"2024-09-20T19:03:50.612904+0000","last_fullsized":"2024-09-20T19:03:50.612904+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:32.856856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:32.856856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:56.734943+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":664,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.427136+0000","last_change":"2024-09-20T19:03:14.944092+0000","last_active":"2024-09-20T19:03:50.427136+0000","last_peered":"2024-09-20T19:03:50.427136+0000","last_clean":"2024-09-20T19:03:50.427136+0000","last_became_active":"2024-09-20T19:03:14.943782+0000","last_became_peered":"2024-09-20T19:03:14.943782+0000","last_unstale":"2024-09-20T19:03:50.427136+0000","last_undegraded":"2024-09-20T19:03:50.427136+0000","last_fullsized":"2024-09-20T19:03:50.427136+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:31.687188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:31.687188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:26.355079+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2","version":"363'7386","reported_seq":7482,"reported_epoch":363,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-20T19:03:51.910218+0000","last_change":"2024-09-20T19:03:46.202902+0000","last_active":"2024-09-20T19:03:51.910218+0000","last_peered":"2024-09-20T19:03:51.910218+0000","last_clean":"2024-09-20T19:03:13.930949+0000","last_became_active":"2024-09-20T19:03:16.167977+0000","last_became_peered":"2024-09-20T19:03:16.167977+0000","last_unstale":"2024-09-20T19:03:51.910218+0000","last_undegraded":"2024-09-20T19:03:51.910218+0000","last_fullsized":"2024-09-20T19:03:51.910218+0000","mapping_epoch":351,"log_start":"340'2982","ondisk_log_start":"340'2982","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4404,"log_dups_size":0,"ondisk_log_size":4404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":477233152,"num_objects":7282,"num_object_clones":0,"num_object_copies":21846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":7282,"num_objects_unfound":0,"num_objects_dirty":7282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7386,"num_write_kb":469376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,4],"avail_no_missing":["1","0","4"],"object_location_counts":[{"shards":"0,1,4","objects":7282}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"273'20","reported_seq":607,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.613770+0000","last_change":"2024-09-20T19:03:14.972618+0000","last_active":"2024-09-20T19:03:50.613770+0000","last_peered":"2024-09-20T19:03:50.613770+0000","last_clean":"2024-09-20T19:03:50.613770+0000","last_became_active":"2024-09-20T19:03:14.950217+0000","last_became_peered":"2024-09-20T19:03:14.950217+0000","last_unstale":"2024-09-20T19:03:50.613770+0000","last_undegraded":"2024-09-20T19:03:50.613770+0000","last_fullsized":"2024-09-20T19:03:50.613770+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'20","last_scrub_stamp":"2024-09-20T19:03:04.282800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:04.282800+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:21.780218+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":673,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.613782+0000","last_change":"2024-09-20T19:03:14.955780+0000","last_active":"2024-09-20T19:03:50.613782+0000","last_peered":"2024-09-20T19:03:50.613782+0000","last_clean":"2024-09-20T19:03:50.613782+0000","last_became_active":"2024-09-20T19:03:14.953433+0000","last_became_peered":"2024-09-20T19:03:14.953433+0000","last_unstale":"2024-09-20T19:03:50.613782+0000","last_undegraded":"2024-09-20T19:03:50.613782+0000","last_fullsized":"2024-09-20T19:03:50.613782+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:34.985163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:34.985163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:56.492382+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":836,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.426852+0000","last_change":"2024-09-20T19:02:02.813926+0000","last_active":"2024-09-20T19:03:50.426852+0000","last_peered":"2024-09-20T19:03:50.426852+0000","last_clean":"2024-09-20T19:03:50.426852+0000","last_became_active":"2024-09-20T18:49:49.957005+0000","last_became_peered":"2024-09-20T18:49:49.957005+0000","last_unstale":"2024-09-20T19:03:50.426852+0000","last_undegraded":"2024-09-20T19:03:50.426852+0000","last_fullsized":"2024-09-20T19:03:50.426852+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T19:02:02.813865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:02.813865+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":501,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.613708+0000","last_change":"2024-09-20T19:03:14.958483+0000","last_active":"2024-09-20T19:03:50.613708+0000","last_peered":"2024-09-20T19:03:50.613708+0000","last_clean":"2024-09-20T19:03:50.613708+0000","last_became_active":"2024-09-20T19:03:14.958200+0000","last_became_peered":"2024-09-20T19:03:14.958200+0000","last_unstale":"2024-09-20T19:03:50.613708+0000","last_undegraded":"2024-09-20T19:03:50.613708+0000","last_fullsized":"2024-09-20T19:03:50.613708+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:11.207994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:03:11.207994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:17.068313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.5","version":"363'7280","reported_seq":7366,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:51.470632+0000","last_change":"2024-09-20T19:02:11.828434+0000","last_active":"2024-09-20T19:03:51.470632+0000","last_peered":"2024-09-20T19:03:51.470632+0000","last_clean":"2024-09-20T19:03:51.470632+0000","last_became_active":"2024-09-20T19:02:11.828162+0000","last_became_peered":"2024-09-20T19:02:11.828162+0000","last_unstale":"2024-09-20T19:03:51.470632+0000","last_undegraded":"2024-09-20T19:03:51.470632+0000","last_fullsized":"2024-09-20T19:03:51.470632+0000","mapping_epoch":325,"log_start":"336'1668","ondisk_log_start":"336'1668","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":5612,"log_dups_size":0,"ondisk_log_size":5612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:42.284085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":474349568,"num_objects":7238,"num_object_clones":0,"num_object_copies":21714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7280,"num_write_kb":464576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"274'15","reported_seq":705,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.428418+0000","last_change":"2024-09-20T19:03:14.963710+0000","last_active":"2024-09-20T19:03:50.428418+0000","last_peered":"2024-09-20T19:03:50.428418+0000","last_clean":"2024-09-20T19:03:50.428418+0000","last_became_active":"2024-09-20T19:03:14.950377+0000","last_became_peered":"2024-09-20T19:03:14.950377+0000","last_unstale":"2024-09-20T19:03:50.428418+0000","last_undegraded":"2024-09-20T19:03:50.428418+0000","last_fullsized":"2024-09-20T19:03:50.428418+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'15","last_scrub_stamp":"2024-09-20T19:01:27.802100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:27.802100+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":199,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.426110+0000","last_change":"2024-09-20T19:03:13.479475+0000","last_active":"2024-09-20T19:03:50.426110+0000","last_peered":"2024-09-20T19:03:50.426110+0000","last_clean":"2024-09-20T19:03:50.426110+0000","last_became_active":"2024-09-20T19:00:01.190875+0000","last_became_peered":"2024-09-20T19:00:01.190875+0000","last_unstale":"2024-09-20T19:03:50.426110+0000","last_undegraded":"2024-09-20T19:03:50.426110+0000","last_fullsized":"2024-09-20T19:03:50.426110+0000","mapping_epoch":277,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":278,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:13.479412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:03:13.479412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:15.835045+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":668,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.426200+0000","last_change":"2024-09-20T19:03:14.942108+0000","last_active":"2024-09-20T19:03:50.426200+0000","last_peered":"2024-09-20T19:03:50.426200+0000","last_clean":"2024-09-20T19:03:50.426200+0000","last_became_active":"2024-09-20T19:03:14.941742+0000","last_became_peered":"2024-09-20T19:03:14.941742+0000","last_unstale":"2024-09-20T19:03:50.426200+0000","last_undegraded":"2024-09-20T19:03:50.426200+0000","last_fullsized":"2024-09-20T19:03:50.426200+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:24.938980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:24.938980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":644,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.428675+0000","last_change":"2024-09-20T19:02:31.120809+0000","last_active":"2024-09-20T19:03:50.428675+0000","last_peered":"2024-09-20T19:03:50.428675+0000","last_clean":"2024-09-20T19:03:50.428675+0000","last_became_active":"2024-09-20T18:49:51.964098+0000","last_became_peered":"2024-09-20T18:49:51.964098+0000","last_unstale":"2024-09-20T19:03:50.428675+0000","last_undegraded":"2024-09-20T19:03:50.428675+0000","last_fullsized":"2024-09-20T19:03:50.428675+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:31.120623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:31.120623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"363'7264","reported_seq":7362,"reported_epoch":363,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-20T19:03:52.422679+0000","last_change":"2024-09-20T19:03:46.179755+0000","last_active":"2024-09-20T19:03:52.422679+0000","last_peered":"2024-09-20T19:03:52.422679+0000","last_clean":"2024-09-20T19:03:13.908942+0000","last_became_active":"2024-09-20T19:03:16.166786+0000","last_became_peered":"2024-09-20T19:03:16.166786+0000","last_unstale":"2024-09-20T19:03:52.422679+0000","last_undegraded":"2024-09-20T19:03:52.422679+0000","last_fullsized":"2024-09-20T19:03:52.422679+0000","mapping_epoch":351,"log_start":"340'2897","ondisk_log_start":"340'2897","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4367,"log_dups_size":0,"ondisk_log_size":4367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":466354176,"num_objects":7116,"num_object_clones":0,"num_object_copies":21348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":7116,"num_objects_unfound":0,"num_objects_dirty":7116,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7264,"num_write_kb":460160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,4],"avail_no_missing":["0","3","4"],"object_location_counts":[{"shards":"0,3,4","objects":7116}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"274'35","reported_seq":278,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.613347+0000","last_change":"2024-09-20T19:03:09.841596+0000","last_active":"2024-09-20T19:03:50.613347+0000","last_peered":"2024-09-20T19:03:50.613347+0000","last_clean":"2024-09-20T19:03:50.613347+0000","last_became_active":"2024-09-20T18:59:05.087391+0000","last_became_peered":"2024-09-20T18:59:05.087391+0000","last_unstale":"2024-09-20T19:03:50.613347+0000","last_undegraded":"2024-09-20T19:03:50.613347+0000","last_fullsized":"2024-09-20T19:03:50.613347+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'35","last_scrub_stamp":"2024-09-20T19:03:09.841361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:09.841361+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:33.281410+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":670,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.630618+0000","last_change":"2024-09-20T19:02:05.440745+0000","last_active":"2024-09-20T19:03:50.630618+0000","last_peered":"2024-09-20T19:03:50.630618+0000","last_clean":"2024-09-20T19:03:50.630618+0000","last_became_active":"2024-09-20T18:59:05.086670+0000","last_became_peered":"2024-09-20T18:59:05.086670+0000","last_unstale":"2024-09-20T19:03:50.630618+0000","last_undegraded":"2024-09-20T19:03:50.630618+0000","last_fullsized":"2024-09-20T19:03:50.630618+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:05.440677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:05.440677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":658,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.426237+0000","last_change":"2024-09-20T19:02:42.745448+0000","last_active":"2024-09-20T19:03:50.426237+0000","last_peered":"2024-09-20T19:03:50.426237+0000","last_clean":"2024-09-20T19:03:50.426237+0000","last_became_active":"2024-09-20T18:49:49.957558+0000","last_became_peered":"2024-09-20T18:49:49.957558+0000","last_unstale":"2024-09-20T19:03:50.426237+0000","last_undegraded":"2024-09-20T19:03:50.426237+0000","last_fullsized":"2024-09-20T19:03:50.426237+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:42.745372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:42.745372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":636,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.629875+0000","last_change":"2024-09-20T18:53:47.370152+0000","last_active":"2024-09-20T19:03:50.629875+0000","last_peered":"2024-09-20T19:03:50.629875+0000","last_clean":"2024-09-20T19:03:50.629875+0000","last_became_active":"2024-09-20T18:49:51.970798+0000","last_became_peered":"2024-09-20T18:49:51.970798+0000","last_unstale":"2024-09-20T19:03:50.629875+0000","last_undegraded":"2024-09-20T19:03:50.629875+0000","last_fullsized":"2024-09-20T19:03:50.629875+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:47.370108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:47.370108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.7","version":"363'7390","reported_seq":7477,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:51.517870+0000","last_change":"2024-09-20T19:02:11.827681+0000","last_active":"2024-09-20T19:03:51.517870+0000","last_peered":"2024-09-20T19:03:51.517870+0000","last_clean":"2024-09-20T19:03:51.517870+0000","last_became_active":"2024-09-20T19:02:11.827360+0000","last_became_peered":"2024-09-20T19:02:11.827360+0000","last_unstale":"2024-09-20T19:03:51.517870+0000","last_undegraded":"2024-09-20T19:03:51.517870+0000","last_fullsized":"2024-09-20T19:03:51.517870+0000","mapping_epoch":325,"log_start":"338'2507","ondisk_log_start":"338'2507","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4883,"log_dups_size":0,"ondisk_log_size":4883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":480641024,"num_objects":7334,"num_object_clones":0,"num_object_copies":22002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7334,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7390,"num_write_kb":471168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"273'18","reported_seq":563,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.612791+0000","last_change":"2024-09-20T19:03:14.981172+0000","last_active":"2024-09-20T19:03:50.612791+0000","last_peered":"2024-09-20T19:03:50.612791+0000","last_clean":"2024-09-20T19:03:50.612791+0000","last_became_active":"2024-09-20T19:03:14.956290+0000","last_became_peered":"2024-09-20T19:03:14.956290+0000","last_unstale":"2024-09-20T19:03:50.612791+0000","last_undegraded":"2024-09-20T19:03:50.612791+0000","last_fullsized":"2024-09-20T19:03:50.612791+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'18","last_scrub_stamp":"2024-09-20T19:03:13.201592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:13.201592+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:37.912706+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":674,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.629974+0000","last_change":"2024-09-20T19:03:14.961785+0000","last_active":"2024-09-20T19:03:50.629974+0000","last_peered":"2024-09-20T19:03:50.629974+0000","last_clean":"2024-09-20T19:03:50.629974+0000","last_became_active":"2024-09-20T19:03:14.961495+0000","last_became_peered":"2024-09-20T19:03:14.961495+0000","last_unstale":"2024-09-20T19:03:50.629974+0000","last_undegraded":"2024-09-20T19:03:50.629974+0000","last_fullsized":"2024-09-20T19:03:50.629974+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:00:34.940850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:00:34.940850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":650,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.426014+0000","last_change":"2024-09-20T19:01:53.734774+0000","last_active":"2024-09-20T19:03:50.426014+0000","last_peered":"2024-09-20T19:03:50.426014+0000","last_clean":"2024-09-20T19:03:50.426014+0000","last_became_active":"2024-09-20T18:49:49.954944+0000","last_became_peered":"2024-09-20T18:49:49.954944+0000","last_unstale":"2024-09-20T19:03:50.426014+0000","last_undegraded":"2024-09-20T19:03:50.426014+0000","last_fullsized":"2024-09-20T19:03:50.426014+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:53.734726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:53.734726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:45.272008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":643,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.613392+0000","last_change":"2024-09-20T19:02:35.968178+0000","last_active":"2024-09-20T19:03:50.613392+0000","last_peered":"2024-09-20T19:03:50.613392+0000","last_clean":"2024-09-20T19:03:50.613392+0000","last_became_active":"2024-09-20T18:49:51.972970+0000","last_became_peered":"2024-09-20T18:49:51.972970+0000","last_unstale":"2024-09-20T19:03:50.613392+0000","last_undegraded":"2024-09-20T19:03:50.613392+0000","last_fullsized":"2024-09-20T19:03:50.613392+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:35.968052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:35.968052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:53.629726+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.6","version":"363'7418","reported_seq":9121,"reported_epoch":363,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-20T19:03:51.449618+0000","last_change":"2024-09-20T19:03:16.388415+0000","last_active":"2024-09-20T19:03:51.449618+0000","last_peered":"2024-09-20T19:03:51.449618+0000","last_clean":"2024-09-20T19:03:13.927169+0000","last_became_active":"2024-09-20T19:03:16.282080+0000","last_became_peered":"2024-09-20T19:03:16.282080+0000","last_unstale":"2024-09-20T19:03:51.449618+0000","last_undegraded":"2024-09-20T19:03:51.449618+0000","last_fullsized":"2024-09-20T19:03:15.944009+0000","mapping_epoch":351,"log_start":"338'2262","ondisk_log_start":"338'2262","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":5156,"log_dups_size":0,"ondisk_log_size":5156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483393536,"num_objects":7376,"num_object_clones":0,"num_object_copies":22128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":5002,"num_objects_unfound":0,"num_objects_dirty":7376,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7418,"num_write_kb":473408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":806,"num_bytes_recovered":52822016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0],"avail_no_missing":["5","0","4"],"object_location_counts":[{"shards":"0,4,5","objects":5002},{"shards":"0,5","objects":2374}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"273'27","reported_seq":748,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.612509+0000","last_change":"2024-09-20T19:02:39.709344+0000","last_active":"2024-09-20T19:03:50.612509+0000","last_peered":"2024-09-20T19:03:50.612509+0000","last_clean":"2024-09-20T19:03:50.612509+0000","last_became_active":"2024-09-20T18:59:05.083726+0000","last_became_peered":"2024-09-20T18:59:05.083726+0000","last_unstale":"2024-09-20T19:03:50.612509+0000","last_undegraded":"2024-09-20T19:03:50.612509+0000","last_fullsized":"2024-09-20T19:03:50.612509+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'27","last_scrub_stamp":"2024-09-20T19:02:39.709269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:39.709269+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":700,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.629997+0000","last_change":"2024-09-20T19:03:14.986178+0000","last_active":"2024-09-20T19:03:50.629997+0000","last_peered":"2024-09-20T19:03:50.629997+0000","last_clean":"2024-09-20T19:03:50.629997+0000","last_became_active":"2024-09-20T19:03:14.957738+0000","last_became_peered":"2024-09-20T19:03:14.957738+0000","last_unstale":"2024-09-20T19:03:50.629997+0000","last_undegraded":"2024-09-20T19:03:50.629997+0000","last_fullsized":"2024-09-20T19:03:50.629997+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-20T18:58:57.563058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:57.563058+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"352'13","reported_seq":856,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.631155+0000","last_change":"2024-09-20T19:03:14.970339+0000","last_active":"2024-09-20T19:03:50.631155+0000","last_peered":"2024-09-20T19:03:50.631155+0000","last_clean":"2024-09-20T19:03:50.631155+0000","last_became_active":"2024-09-20T19:03:14.948540+0000","last_became_peered":"2024-09-20T19:03:14.948540+0000","last_unstale":"2024-09-20T19:03:50.631155+0000","last_undegraded":"2024-09-20T19:03:50.631155+0000","last_fullsized":"2024-09-20T19:03:50.631155+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'9","last_scrub_stamp":"2024-09-20T18:58:27.521051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:27.521051+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":665,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.612941+0000","last_change":"2024-09-20T19:03:06.888620+0000","last_active":"2024-09-20T19:03:50.612941+0000","last_peered":"2024-09-20T19:03:50.612941+0000","last_clean":"2024-09-20T19:03:50.612941+0000","last_became_active":"2024-09-20T18:59:05.087005+0000","last_became_peered":"2024-09-20T18:59:05.087005+0000","last_unstale":"2024-09-20T19:03:50.612941+0000","last_undegraded":"2024-09-20T19:03:50.612941+0000","last_fullsized":"2024-09-20T19:03:50.612941+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:06.888563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:03:06.888563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:07.838057+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":226,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.630136+0000","last_change":"2024-09-20T19:01:16.542398+0000","last_active":"2024-09-20T19:03:50.630136+0000","last_peered":"2024-09-20T19:03:50.630136+0000","last_clean":"2024-09-20T19:03:50.630136+0000","last_became_active":"2024-09-20T18:59:05.085427+0000","last_became_peered":"2024-09-20T18:59:05.085427+0000","last_unstale":"2024-09-20T19:03:50.630136+0000","last_undegraded":"2024-09-20T19:03:50.630136+0000","last_fullsized":"2024-09-20T19:03:50.630136+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:16.542335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:16.542335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"273'12","reported_seq":668,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.614407+0000","last_change":"2024-09-20T19:01:12.385223+0000","last_active":"2024-09-20T19:03:50.614407+0000","last_peered":"2024-09-20T19:03:50.614407+0000","last_clean":"2024-09-20T19:03:50.614407+0000","last_became_active":"2024-09-20T18:49:48.500662+0000","last_became_peered":"2024-09-20T18:49:48.500662+0000","last_unstale":"2024-09-20T19:03:50.614407+0000","last_undegraded":"2024-09-20T19:03:50.614407+0000","last_fullsized":"2024-09-20T19:03:50.614407+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'12","last_scrub_stamp":"2024-09-20T19:01:12.385152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:12.385152+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:57.005041+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":646,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.616824+0000","last_change":"2024-09-20T19:01:14.466096+0000","last_active":"2024-09-20T19:03:50.616824+0000","last_peered":"2024-09-20T19:03:50.616824+0000","last_clean":"2024-09-20T19:03:50.616824+0000","last_became_active":"2024-09-20T18:49:49.956749+0000","last_became_peered":"2024-09-20T18:49:49.956749+0000","last_unstale":"2024-09-20T19:03:50.616824+0000","last_undegraded":"2024-09-20T19:03:50.616824+0000","last_fullsized":"2024-09-20T19:03:50.616824+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:14.466042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:14.466042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":675,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.629537+0000","last_change":"2024-09-20T19:03:14.956967+0000","last_active":"2024-09-20T19:03:50.629537+0000","last_peered":"2024-09-20T19:03:50.629537+0000","last_clean":"2024-09-20T19:03:50.629537+0000","last_became_active":"2024-09-20T19:03:14.956216+0000","last_became_peered":"2024-09-20T19:03:14.956216+0000","last_unstale":"2024-09-20T19:03:50.629537+0000","last_undegraded":"2024-09-20T19:03:50.629537+0000","last_fullsized":"2024-09-20T19:03:50.629537+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:03.509903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:03.509903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,5],"acting":[7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"273'15","reported_seq":668,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.430373+0000","last_change":"2024-09-20T19:01:15.875779+0000","last_active":"2024-09-20T19:03:50.430373+0000","last_peered":"2024-09-20T19:03:50.430373+0000","last_clean":"2024-09-20T19:03:50.430373+0000","last_became_active":"2024-09-20T18:49:48.500803+0000","last_became_peered":"2024-09-20T18:49:48.500803+0000","last_unstale":"2024-09-20T19:03:50.430373+0000","last_undegraded":"2024-09-20T19:03:50.430373+0000","last_fullsized":"2024-09-20T19:03:50.430373+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:01:15.875693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:15.875693+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":680,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.631334+0000","last_change":"2024-09-20T19:03:14.958361+0000","last_active":"2024-09-20T19:03:50.631334+0000","last_peered":"2024-09-20T19:03:50.631334+0000","last_clean":"2024-09-20T19:03:50.631334+0000","last_became_active":"2024-09-20T19:03:14.958013+0000","last_became_peered":"2024-09-20T19:03:14.958013+0000","last_unstale":"2024-09-20T19:03:50.631334+0000","last_undegraded":"2024-09-20T19:03:50.631334+0000","last_fullsized":"2024-09-20T19:03:50.631334+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:59.589891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:59.589891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":646,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.614350+0000","last_change":"2024-09-20T19:01:06.482291+0000","last_active":"2024-09-20T19:03:50.614350+0000","last_peered":"2024-09-20T19:03:50.614350+0000","last_clean":"2024-09-20T19:03:50.614350+0000","last_became_active":"2024-09-20T18:49:49.961947+0000","last_became_peered":"2024-09-20T18:49:49.961947+0000","last_unstale":"2024-09-20T19:03:50.614350+0000","last_undegraded":"2024-09-20T19:03:50.614350+0000","last_fullsized":"2024-09-20T19:03:50.614350+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:06.482212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:06.482212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:58.987808+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"66'1","reported_seq":684,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.426207+0000","last_change":"2024-09-20T19:03:14.960172+0000","last_active":"2024-09-20T19:03:50.426207+0000","last_peered":"2024-09-20T19:03:50.426207+0000","last_clean":"2024-09-20T19:03:50.426207+0000","last_became_active":"2024-09-20T19:03:14.945523+0000","last_became_peered":"2024-09-20T19:03:14.945523+0000","last_unstale":"2024-09-20T19:03:50.426207+0000","last_undegraded":"2024-09-20T19:03:50.426207+0000","last_fullsized":"2024-09-20T19:03:50.426207+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T19:03:12.499684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:03:12.499684+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:30.881550+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,0],"acting":[2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"274'15","reported_seq":667,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.614027+0000","last_change":"2024-09-20T19:01:36.532486+0000","last_active":"2024-09-20T19:03:50.614027+0000","last_peered":"2024-09-20T19:03:50.614027+0000","last_clean":"2024-09-20T19:03:50.614027+0000","last_became_active":"2024-09-20T18:49:48.501011+0000","last_became_peered":"2024-09-20T18:49:48.501011+0000","last_unstale":"2024-09-20T19:03:50.614027+0000","last_undegraded":"2024-09-20T19:03:50.614027+0000","last_fullsized":"2024-09-20T19:03:50.614027+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'15","last_scrub_stamp":"2024-09-20T19:01:36.532419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:36.532419+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:58.862030+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":656,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.428581+0000","last_change":"2024-09-20T19:01:30.864866+0000","last_active":"2024-09-20T19:03:50.428581+0000","last_peered":"2024-09-20T19:03:50.428581+0000","last_clean":"2024-09-20T19:03:50.428581+0000","last_became_active":"2024-09-20T18:49:45.938511+0000","last_became_peered":"2024-09-20T18:49:45.938511+0000","last_unstale":"2024-09-20T19:03:50.428581+0000","last_undegraded":"2024-09-20T19:03:50.428581+0000","last_fullsized":"2024-09-20T19:03:50.428581+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:30.864794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:30.864794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:01.115401+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":824,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.613579+0000","last_change":"2024-09-20T19:00:34.485855+0000","last_active":"2024-09-20T19:03:50.613579+0000","last_peered":"2024-09-20T19:03:50.613579+0000","last_clean":"2024-09-20T19:03:50.613579+0000","last_became_active":"2024-09-20T18:49:49.960550+0000","last_became_peered":"2024-09-20T18:49:49.960550+0000","last_unstale":"2024-09-20T19:03:50.613579+0000","last_undegraded":"2024-09-20T19:03:50.613579+0000","last_fullsized":"2024-09-20T19:03:50.613579+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T19:00:34.485624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:00:34.485624+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:54.909280+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":509,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.427075+0000","last_change":"2024-09-20T19:03:14.957109+0000","last_active":"2024-09-20T19:03:50.427075+0000","last_peered":"2024-09-20T19:03:50.427075+0000","last_clean":"2024-09-20T19:03:50.427075+0000","last_became_active":"2024-09-20T19:03:14.956518+0000","last_became_peered":"2024-09-20T19:03:14.956518+0000","last_unstale":"2024-09-20T19:03:50.427075+0000","last_undegraded":"2024-09-20T19:03:50.427075+0000","last_fullsized":"2024-09-20T19:03:50.427075+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:59.497810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:59:59.497810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:24.357379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"274'20","reported_seq":684,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.613721+0000","last_change":"2024-09-20T18:57:20.701592+0000","last_active":"2024-09-20T19:03:50.613721+0000","last_peered":"2024-09-20T19:03:50.613721+0000","last_clean":"2024-09-20T19:03:50.613721+0000","last_became_active":"2024-09-20T18:49:58.997494+0000","last_became_peered":"2024-09-20T18:49:58.997494+0000","last_unstale":"2024-09-20T19:03:50.613721+0000","last_undegraded":"2024-09-20T19:03:50.613721+0000","last_fullsized":"2024-09-20T19:03:50.613721+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:57:20.701543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:20.701543+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":666,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.630281+0000","last_change":"2024-09-20T19:03:14.953145+0000","last_active":"2024-09-20T19:03:50.630281+0000","last_peered":"2024-09-20T19:03:50.630281+0000","last_clean":"2024-09-20T19:03:50.630281+0000","last_became_active":"2024-09-20T19:03:14.952814+0000","last_became_peered":"2024-09-20T19:03:14.952814+0000","last_unstale":"2024-09-20T19:03:50.630281+0000","last_undegraded":"2024-09-20T19:03:50.630281+0000","last_fullsized":"2024-09-20T19:03:50.630281+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.632380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.632380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":824,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.613630+0000","last_change":"2024-09-20T19:01:10.601374+0000","last_active":"2024-09-20T19:03:50.613630+0000","last_peered":"2024-09-20T19:03:50.613630+0000","last_clean":"2024-09-20T19:03:50.613630+0000","last_became_active":"2024-09-20T18:49:49.962243+0000","last_became_peered":"2024-09-20T18:49:49.962243+0000","last_unstale":"2024-09-20T19:03:50.613630+0000","last_undegraded":"2024-09-20T19:03:50.613630+0000","last_fullsized":"2024-09-20T19:03:50.613630+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T19:01:10.601300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:10.601300+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":651,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.429144+0000","last_change":"2024-09-20T19:01:03.821977+0000","last_active":"2024-09-20T19:03:50.429144+0000","last_peered":"2024-09-20T19:03:50.429144+0000","last_clean":"2024-09-20T19:03:50.429144+0000","last_became_active":"2024-09-20T18:49:51.969093+0000","last_became_peered":"2024-09-20T18:49:51.969093+0000","last_unstale":"2024-09-20T19:03:50.429144+0000","last_undegraded":"2024-09-20T19:03:50.429144+0000","last_fullsized":"2024-09-20T19:03:50.429144+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:03.821908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:03.821908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"273'19","reported_seq":608,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.629427+0000","last_change":"2024-09-20T19:03:14.990890+0000","last_active":"2024-09-20T19:03:50.629427+0000","last_peered":"2024-09-20T19:03:50.629427+0000","last_clean":"2024-09-20T19:03:50.629427+0000","last_became_active":"2024-09-20T19:03:14.965320+0000","last_became_peered":"2024-09-20T19:03:14.965320+0000","last_unstale":"2024-09-20T19:03:50.629427+0000","last_undegraded":"2024-09-20T19:03:50.629427+0000","last_fullsized":"2024-09-20T19:03:50.629427+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:57:50.899514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:50.899514+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":496,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.630956+0000","last_change":"2024-09-20T19:03:14.954849+0000","last_active":"2024-09-20T19:03:50.630956+0000","last_peered":"2024-09-20T19:03:50.630956+0000","last_clean":"2024-09-20T19:03:50.630956+0000","last_became_active":"2024-09-20T19:03:14.952258+0000","last_became_peered":"2024-09-20T19:03:14.952258+0000","last_unstale":"2024-09-20T19:03:50.630956+0000","last_undegraded":"2024-09-20T19:03:50.630956+0000","last_fullsized":"2024-09-20T19:03:50.630956+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:06.310027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:59:06.310027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":647,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.612823+0000","last_change":"2024-09-20T19:02:38.906352+0000","last_active":"2024-09-20T19:03:50.612823+0000","last_peered":"2024-09-20T19:03:50.612823+0000","last_clean":"2024-09-20T19:03:50.612823+0000","last_became_active":"2024-09-20T18:49:49.957067+0000","last_became_peered":"2024-09-20T18:49:49.957067+0000","last_unstale":"2024-09-20T19:03:50.612823+0000","last_undegraded":"2024-09-20T19:03:50.612823+0000","last_fullsized":"2024-09-20T19:03:50.612823+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:38.906161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:38.906161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":627,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.612401+0000","last_change":"2024-09-20T18:57:16.789412+0000","last_active":"2024-09-20T19:03:50.612401+0000","last_peered":"2024-09-20T19:03:50.612401+0000","last_clean":"2024-09-20T19:03:50.612401+0000","last_became_active":"2024-09-20T18:49:51.966915+0000","last_became_peered":"2024-09-20T18:49:51.966915+0000","last_unstale":"2024-09-20T19:03:50.612401+0000","last_undegraded":"2024-09-20T19:03:50.612401+0000","last_fullsized":"2024-09-20T19:03:50.612401+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:16.789095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:16.789095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"273'15","reported_seq":718,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.612959+0000","last_change":"2024-09-20T19:02:30.816061+0000","last_active":"2024-09-20T19:03:50.612959+0000","last_peered":"2024-09-20T19:03:50.612959+0000","last_clean":"2024-09-20T19:03:50.612959+0000","last_became_active":"2024-09-20T18:59:05.085230+0000","last_became_peered":"2024-09-20T18:59:05.085230+0000","last_unstale":"2024-09-20T19:03:50.612959+0000","last_undegraded":"2024-09-20T19:03:50.612959+0000","last_fullsized":"2024-09-20T19:03:50.612959+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:02:30.815747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:30.815747+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":665,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.630489+0000","last_change":"2024-09-20T19:03:14.957280+0000","last_active":"2024-09-20T19:03:50.630489+0000","last_peered":"2024-09-20T19:03:50.630489+0000","last_clean":"2024-09-20T19:03:50.630489+0000","last_became_active":"2024-09-20T19:03:14.956668+0000","last_became_peered":"2024-09-20T19:03:14.956668+0000","last_unstale":"2024-09-20T19:03:50.630489+0000","last_undegraded":"2024-09-20T19:03:50.630489+0000","last_fullsized":"2024-09-20T19:03:50.630489+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:07.408197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:03:07.408197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:16.413210+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":646,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.613531+0000","last_change":"2024-09-20T19:01:03.507072+0000","last_active":"2024-09-20T19:03:50.613531+0000","last_peered":"2024-09-20T19:03:50.613531+0000","last_clean":"2024-09-20T19:03:50.613531+0000","last_became_active":"2024-09-20T18:49:49.962523+0000","last_became_peered":"2024-09-20T18:49:49.962523+0000","last_unstale":"2024-09-20T19:03:50.613531+0000","last_undegraded":"2024-09-20T19:03:50.613531+0000","last_fullsized":"2024-09-20T19:03:50.613531+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:03.507016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:03.507016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:55.956672+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":643,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.427817+0000","last_change":"2024-09-20T19:01:11.847792+0000","last_active":"2024-09-20T19:03:50.427817+0000","last_peered":"2024-09-20T19:03:50.427817+0000","last_clean":"2024-09-20T19:03:50.427817+0000","last_became_active":"2024-09-20T18:49:51.967313+0000","last_became_peered":"2024-09-20T18:49:51.967313+0000","last_unstale":"2024-09-20T19:03:50.427817+0000","last_undegraded":"2024-09-20T19:03:50.427817+0000","last_fullsized":"2024-09-20T19:03:50.427817+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:11.847716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:11.847716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"274'19","reported_seq":720,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.612454+0000","last_change":"2024-09-20T19:01:05.495603+0000","last_active":"2024-09-20T19:03:50.612454+0000","last_peered":"2024-09-20T19:03:50.612454+0000","last_clean":"2024-09-20T19:03:50.612454+0000","last_became_active":"2024-09-20T18:59:05.075390+0000","last_became_peered":"2024-09-20T18:59:05.075390+0000","last_unstale":"2024-09-20T19:03:50.612454+0000","last_undegraded":"2024-09-20T19:03:50.612454+0000","last_fullsized":"2024-09-20T19:03:50.612454+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'19","last_scrub_stamp":"2024-09-20T19:01:05.495478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:05.495478+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:02.014342+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":662,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.616902+0000","last_change":"2024-09-20T19:01:47.521858+0000","last_active":"2024-09-20T19:03:50.616902+0000","last_peered":"2024-09-20T19:03:50.616902+0000","last_clean":"2024-09-20T19:03:50.616902+0000","last_became_active":"2024-09-20T18:49:45.942105+0000","last_became_peered":"2024-09-20T18:49:45.942105+0000","last_unstale":"2024-09-20T19:03:50.616902+0000","last_undegraded":"2024-09-20T19:03:50.616902+0000","last_fullsized":"2024-09-20T19:03:50.616902+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:47.521789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:47.521789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":680,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.629819+0000","last_change":"2024-09-20T19:03:14.957124+0000","last_active":"2024-09-20T19:03:50.629819+0000","last_peered":"2024-09-20T19:03:50.629819+0000","last_clean":"2024-09-20T19:03:50.629819+0000","last_became_active":"2024-09-20T19:03:14.956811+0000","last_became_peered":"2024-09-20T19:03:14.956811+0000","last_unstale":"2024-09-20T19:03:50.629819+0000","last_undegraded":"2024-09-20T19:03:50.629819+0000","last_fullsized":"2024-09-20T19:03:50.629819+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:27.267254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:27.267254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:53.812193+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":648,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.613702+0000","last_change":"2024-09-20T19:02:38.950217+0000","last_active":"2024-09-20T19:03:50.613702+0000","last_peered":"2024-09-20T19:03:50.613702+0000","last_clean":"2024-09-20T19:03:50.613702+0000","last_became_active":"2024-09-20T18:59:05.081179+0000","last_became_peered":"2024-09-20T18:59:05.081179+0000","last_unstale":"2024-09-20T19:03:50.613702+0000","last_undegraded":"2024-09-20T19:03:50.613702+0000","last_fullsized":"2024-09-20T19:03:50.613702+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:38.950017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:38.950017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:02.988187+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"273'8","reported_seq":689,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.612393+0000","last_change":"2024-09-20T19:00:33.504967+0000","last_active":"2024-09-20T19:03:50.612393+0000","last_peered":"2024-09-20T19:03:50.612393+0000","last_clean":"2024-09-20T19:03:50.612393+0000","last_became_active":"2024-09-20T18:59:05.076365+0000","last_became_peered":"2024-09-20T18:59:05.076365+0000","last_unstale":"2024-09-20T19:03:50.612393+0000","last_undegraded":"2024-09-20T19:03:50.612393+0000","last_fullsized":"2024-09-20T19:03:50.612393+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'8","last_scrub_stamp":"2024-09-20T19:00:33.504891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:00:33.504891+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:50.863517+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":663,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.430578+0000","last_change":"2024-09-20T19:02:39.184249+0000","last_active":"2024-09-20T19:03:50.430578+0000","last_peered":"2024-09-20T19:03:50.430578+0000","last_clean":"2024-09-20T19:03:50.430578+0000","last_became_active":"2024-09-20T18:49:45.940988+0000","last_became_peered":"2024-09-20T18:49:45.940988+0000","last_unstale":"2024-09-20T19:03:50.430578+0000","last_undegraded":"2024-09-20T19:03:50.430578+0000","last_fullsized":"2024-09-20T19:03:50.430578+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:39.184150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:39.184150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:54.134855+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":244,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.628746+0000","last_change":"2024-09-20T19:03:14.949099+0000","last_active":"2024-09-20T19:03:50.628746+0000","last_peered":"2024-09-20T19:03:50.628746+0000","last_clean":"2024-09-20T19:03:50.628746+0000","last_became_active":"2024-09-20T19:03:14.948648+0000","last_became_peered":"2024-09-20T19:03:14.948648+0000","last_unstale":"2024-09-20T19:03:50.628746+0000","last_undegraded":"2024-09-20T19:03:50.628746+0000","last_fullsized":"2024-09-20T19:03:50.628746+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:41.497115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:41.497115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":653,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.629083+0000","last_change":"2024-09-20T19:03:14.949524+0000","last_active":"2024-09-20T19:03:50.629083+0000","last_peered":"2024-09-20T19:03:50.629083+0000","last_clean":"2024-09-20T19:03:50.629083+0000","last_became_active":"2024-09-20T19:03:14.949200+0000","last_became_peered":"2024-09-20T19:03:14.949200+0000","last_unstale":"2024-09-20T19:03:50.629083+0000","last_undegraded":"2024-09-20T19:03:50.629083+0000","last_fullsized":"2024-09-20T19:03:50.629083+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:29.277916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:29.277916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:55.810061+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":636,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.612839+0000","last_change":"2024-09-20T19:03:14.956682+0000","last_active":"2024-09-20T19:03:50.612839+0000","last_peered":"2024-09-20T19:03:50.612839+0000","last_clean":"2024-09-20T19:03:50.612839+0000","last_became_active":"2024-09-20T19:03:14.956082+0000","last_became_peered":"2024-09-20T19:03:14.956082+0000","last_unstale":"2024-09-20T19:03:50.612839+0000","last_undegraded":"2024-09-20T19:03:50.612839+0000","last_fullsized":"2024-09-20T19:03:50.612839+0000","mapping_epoch":350,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":351,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:10.970779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:10.970779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":646,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.612647+0000","last_change":"2024-09-20T18:58:36.077005+0000","last_active":"2024-09-20T19:03:50.612647+0000","last_peered":"2024-09-20T19:03:50.612647+0000","last_clean":"2024-09-20T19:03:50.612647+0000","last_became_active":"2024-09-20T18:49:45.937992+0000","last_became_peered":"2024-09-20T18:49:45.937992+0000","last_unstale":"2024-09-20T19:03:50.612647+0000","last_undegraded":"2024-09-20T19:03:50.612647+0000","last_fullsized":"2024-09-20T19:03:50.612647+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:36.076937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:36.076937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"274'19","reported_seq":250,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.629717+0000","last_change":"2024-09-20T19:01:15.544760+0000","last_active":"2024-09-20T19:03:50.629717+0000","last_peered":"2024-09-20T19:03:50.629717+0000","last_clean":"2024-09-20T19:03:50.629717+0000","last_became_active":"2024-09-20T18:59:05.089294+0000","last_became_peered":"2024-09-20T18:59:05.089294+0000","last_unstale":"2024-09-20T19:03:50.629717+0000","last_undegraded":"2024-09-20T19:03:50.629717+0000","last_fullsized":"2024-09-20T19:03:50.629717+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'19","last_scrub_stamp":"2024-09-20T19:01:15.544663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:15.544663+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":641,"reported_epoch":363,"state":"active+clean","last_fresh":"2024-09-20T19:03:50.430253+0000","last_change":"2024-09-20T19:01:40.961449+0000","last_active":"2024-09-20T19:03:50.430253+0000","last_peered":"2024-09-20T19:03:50.430253+0000","last_clean":"2024-09-20T19:03:50.430253+0000","last_became_active":"2024-09-20T18:49:49.957582+0000","last_became_peered":"2024-09-20T18:49:49.957582+0000","last_unstale":"2024-09-20T19:03:50.430253+0000","last_undegraded":"2024-09-20T19:03:50.430253+0000","last_fullsized":"2024-09-20T19:03:50.430253+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:40.961363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:40.961363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:54.068818+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":7623213080,"num_objects":116322,"num_object_clones":0,"num_object_copies":348966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":38592,"num_objects_unfound":0,"num_objects_dirty":116322,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":117398,"num_write_kb":7478977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13738,"num_bytes_recovered":900333568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":23769526272,"data_stored":23769514056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78594,"ondisk_log_size":78594,"up":48,"acting":45,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1185,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1115,"num_read_kb":872,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":4076,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":471040,"data_stored":12580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":685,"ondisk_log_size":685,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2485,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":6452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":839,"num_read_kb":834,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266861,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2516884,"kb_used_data":2002560,"kb_used_omap":1,"kb_used_meta":514302,"kb_avail":91228268,"statfs":{"total":95995035648,"available":93417746432,"internally_reserved":0,"allocated":2050621440,"data_stored":2050232830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":526645706},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528500,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3678228,"kb_used_data":3277760,"kb_used_omap":1,"kb_used_meta":400446,"kb_avail":90066924,"statfs":{"total":95995035648,"available":92228530176,"internally_reserved":0,"allocated":3356426240,"data_stored":3356037432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":410057158},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,1,5,5,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757433,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2885132,"kb_used_data":2352760,"kb_used_omap":1,"kb_used_meta":532350,"kb_avail":90860020,"statfs":{"total":95995035648,"available":93040660480,"internally_reserved":0,"allocated":2409226240,"data_stored":2408845287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":545126853},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":0,"seq":1305670058016,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182597,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3550356,"kb_used_data":2883328,"kb_used_omap":1,"kb_used_meta":667006,"kb_avail":90194796,"statfs":{"total":95995035648,"available":92359471104,"internally_reserved":0,"allocated":2952527872,"data_stored":2952138351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":683014599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411530,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3358084,"kb_used_data":2803376,"kb_used_omap":1,"kb_used_meta":554558,"kb_avail":90387068,"statfs":{"total":95995035648,"available":92556357632,"internally_reserved":0,"allocated":2870657024,"data_stored":2870284594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":567867845},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575056,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2556212,"kb_used_data":1965540,"kb_used_omap":1,"kb_used_meta":590654,"kb_avail":91188940,"statfs":{"total":95995035648,"available":93377474560,"internally_reserved":0,"allocated":2012712960,"data_stored":2012288774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":604830149},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738582,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5203288,"kb_used_data":4616520,"kb_used_omap":1,"kb_used_meta":586494,"kb_avail":88541864,"statfs":{"total":95995035648,"available":90666868736,"internally_reserved":0,"allocated":4727316480,"data_stored":4726920402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":600570309},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1185,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4718530560,"data_stored":4718526488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2006777856,"data_stored":2006777856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2864840704,"data_stored":2864840704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2941784064,"data_stored":2941779992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":3451715584,"data_stored":3451715584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2399469568,"data_stored":2399469568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3341488128,"data_stored":3341484056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2044919808,"data_stored":2044919808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T19:03:57.571 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd pool set unique_pool_3 pgp_num 16 2024-09-20T19:03:57.592 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:57.614 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:57.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:57.615 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:57.716 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:57.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:57.767 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:57.768 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:57.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:57 smithi083 ceph-mon[29421]: from='client.16092 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:03:57.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:57 smithi083 ceph-mon[29421]: osdmap e366: 8 total, 8 up, 7 in 2024-09-20T19:03:57.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:57 smithi083 ceph-mon[36260]: from='client.16092 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:03:57.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:57 smithi083 ceph-mon[36260]: osdmap e366: 8 total, 8 up, 7 in 2024-09-20T19:03:57.869 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:03:57.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:57.907 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:57.908 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:03:58.008 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:58.044 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:58.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:58.045 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:58.146 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:58.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:58.186 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:58.187 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:58.288 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:58.300 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:03:58.316 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:58.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:58.317 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:58.342 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:03:58.418 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:58.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:58.454 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:58.454 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:58.555 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:58.575 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:58.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:58.576 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:58.677 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:58.707 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:58 smithi083 ceph-mon[29421]: pgmap v706: 148 pgs: 1 peering, 3 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 141 active+clean; 6.9 GiB data, 22 GiB used, 603 GiB / 626 GiB avail; 1.8 KiB/s rd, 27 MiB/s wr, 1.06k op/s; 36239/339909 objects misplaced (10.661%); 47 MiB/s, 744 objects/s recovering 2024-09-20T19:03:58.707 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:03:58 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:03:58.708 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:58 smithi083 ceph-mon[36260]: pgmap v706: 148 pgs: 1 peering, 3 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 141 active+clean; 6.9 GiB data, 22 GiB used, 603 GiB / 626 GiB avail; 1.8 KiB/s rd, 27 MiB/s wr, 1.06k op/s; 36239/339909 objects misplaced (10.661%); 47 MiB/s, 744 objects/s recovering 2024-09-20T19:03:58.708 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:03:58 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:03:58.713 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:58.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:58.714 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:58.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:58 smithi142 ceph-mon[32508]: pgmap v706: 148 pgs: 1 peering, 3 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 141 active+clean; 6.9 GiB data, 22 GiB used, 603 GiB / 626 GiB avail; 1.8 KiB/s rd, 27 MiB/s wr, 1.06k op/s; 36239/339909 objects misplaced (10.661%); 47 MiB/s, 744 objects/s recovering 2024-09-20T19:03:58.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:03:58 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:03:58.814 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:03:58.835 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:58.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:58.836 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:03:58.937 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:03:58.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:58.965 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:58.966 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:03:59.066 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:59.094 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:59.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:59.095 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:59.196 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:03:59.221 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T19:03:59.221 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-20T19:03:59.221 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-20T19:03:59.221 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-20T19:03:59.221 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T19:03:59.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:59.238 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:59.238 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:03:59.339 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:03:59.379 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:59.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:59.380 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:03:59.481 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:59.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:59.518 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:59.518 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:03:59.618 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:59.640 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:59.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:59.641 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:59.742 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:03:59.776 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:03:59.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:59.777 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:03:59.878 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:03:59.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:03:59.912 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:03:59.912 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:00.013 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:04:00.047 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:00.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:00.049 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:04:00.105 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_historic_ops 2024-09-20T19:04:00.149 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:00.173 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:00.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:00.174 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:00.235 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T19:04:00.275 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:00.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:00.309 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:00.310 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:00.410 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:04:00.432 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:00.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:00.434 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:04:00.534 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:04:00.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:00.595 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:00.596 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:04:00.696 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:04:00.726 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:04:00.730 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:00 smithi083 ceph-mon[29421]: pgmap v707: 148 pgs: 1 peering, 3 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 141 active+clean; 6.9 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 607 op/s; 36028/338631 objects misplaced (10.639%); 20 MiB/s, 312 objects/s recovering 2024-09-20T19:04:00.730 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:00 smithi083 ceph-mon[36260]: pgmap v707: 148 pgs: 1 peering, 3 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 141 active+clean; 6.9 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 607 op/s; 36028/338631 objects misplaced (10.639%); 20 MiB/s, 312 objects/s recovering 2024-09-20T19:04:00.731 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:00.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:00.732 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:04:00.733 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:04:00 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:04:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:04:00.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:00 smithi142 ceph-mon[32508]: pgmap v707: 148 pgs: 1 peering, 3 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 141 active+clean; 6.9 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 607 op/s; 36028/338631 objects misplaced (10.639%); 20 MiB/s, 312 objects/s recovering 2024-09-20T19:04:00.833 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:00.872 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:00.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:00.873 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:00.973 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:01.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:01.026 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:01.027 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:01.118 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:04:01.127 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:01.149 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:01.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:01.150 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:01.251 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:01.289 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:01.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:01.290 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:01.391 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:01.424 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:01.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:01.425 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:01.471 INFO:teuthology.orchestra.run.smithi083.stderr:set pool 10 pgp_num to 16 2024-09-20T19:04:01.487 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-20T19:04:01.496 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:01 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3430005468' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:04:01.496 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:01 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:04:01.496 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:01 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3875143396' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pgp_num", "val": "16"} : dispatch 2024-09-20T19:04:01.526 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:01.563 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:01.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:01.564 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:01.665 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:01.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:01.703 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:01.703 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:01.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:01 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3430005468' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:04:01.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:01 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:04:01.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:01 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3875143396' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pgp_num", "val": "16"} : dispatch 2024-09-20T19:04:01.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:01 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:04:01.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:01 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:04:01.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:01 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3430005468' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:04:01.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:01 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:04:01.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:01 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3875143396' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pgp_num", "val": "16"} : dispatch 2024-09-20T19:04:01.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:01 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:04:01.804 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:01.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:01.831 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:01.831 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:01.932 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:01.967 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:01.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:01.968 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:02.069 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:02.098 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:02.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:02.099 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:02.200 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:02.240 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:02.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:02.241 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:02.342 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:04:02.364 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:02.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:02.365 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:04:02.466 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:02.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:02.504 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:02.504 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:02.605 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:02.647 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:02.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:02.648 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:02.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:02 smithi083 ceph-mon[29421]: pgmap v708: 148 pgs: 1 peering, 3 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 141 active+clean; 6.9 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 608 op/s; 36028/338631 objects misplaced (10.639%); 20 MiB/s, 312 objects/s recovering 2024-09-20T19:04:02.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:02 smithi083 ceph-mon[29421]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:04:02.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:02 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T19:04:02.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:02 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3875143396' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pgp_num", "val": "16"}]': finished 2024-09-20T19:04:02.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:02 smithi083 ceph-mon[29421]: osdmap e367: 8 total, 8 up, 7 in 2024-09-20T19:04:02.654 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:02 smithi083 ceph-mon[36260]: pgmap v708: 148 pgs: 1 peering, 3 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 141 active+clean; 6.9 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 608 op/s; 36028/338631 objects misplaced (10.639%); 20 MiB/s, 312 objects/s recovering 2024-09-20T19:04:02.654 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:02 smithi083 ceph-mon[36260]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:04:02.654 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:02 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T19:04:02.655 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:02 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3875143396' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pgp_num", "val": "16"}]': finished 2024-09-20T19:04:02.655 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:02 smithi083 ceph-mon[36260]: osdmap e367: 8 total, 8 up, 7 in 2024-09-20T19:04:02.748 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:02.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:02 smithi142 ceph-mon[32508]: pgmap v708: 148 pgs: 1 peering, 3 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 141 active+clean; 6.9 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 608 op/s; 36028/338631 objects misplaced (10.639%); 20 MiB/s, 312 objects/s recovering 2024-09-20T19:04:02.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:02 smithi142 ceph-mon[32508]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:04:02.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:02 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T19:04:02.764 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:02 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3875143396' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pgp_num", "val": "16"}]': finished 2024-09-20T19:04:02.764 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:02 smithi142 ceph-mon[32508]: osdmap e367: 8 total, 8 up, 7 in 2024-09-20T19:04:02.792 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:02.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:02.793 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:02.893 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:02.937 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:02.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:02.938 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:03.039 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:03.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:03.091 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:03.091 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:03.192 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:04:03.229 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:03.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:03.230 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:04:03.330 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:03.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:03.368 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:03.368 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:03.469 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:04:03.512 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:03.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:03.513 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:04:03.518 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:04:03.518 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:04:03.518 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:04:03.518 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:04:03.518 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:04:03.518 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:04:03.518 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:04:03.519 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:04:03.519 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:04:03.519 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:04:03.519 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:04:03.519 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:04:03.519 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:04:03.519 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:04:03.519 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:04:03.519 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:04:03.614 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:03.643 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:03.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:03.644 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:03.745 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:03.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:03 smithi142 ceph-mon[32508]: 10.5 scrub starts 2024-09-20T19:04:03.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:03 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2770349803' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:04:03.773 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:03.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:03.774 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:03.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:03 smithi083 ceph-mon[29421]: 10.5 scrub starts 2024-09-20T19:04:03.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:03 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2770349803' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:04:03.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:03 smithi083 ceph-mon[36260]: 10.5 scrub starts 2024-09-20T19:04:03.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:03 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2770349803' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:04:03.876 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:03.916 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:03.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:03.917 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:04.018 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:04:04.055 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:04.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:04.056 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:04:04.149 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T19:04:04.149 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:04:04.157 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:04:04.190 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:04.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:04.191 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:04:04.292 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:04.324 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:04.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:04.325 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:04.426 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:04.468 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:04.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:04.469 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:04.570 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:04.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:04.605 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:04.606 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:04.706 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:04.743 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:04.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:04.744 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:04.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:04 smithi142 ceph-mon[32508]: pgmap v710: 148 pgs: 3 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 142 active+clean; 6.7 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.02k op/s; 34894/327441 objects misplaced (10.657%); 19 MiB/s, 311 objects/s recovering 2024-09-20T19:04:04.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:04 smithi142 ceph-mon[32508]: 5.a scrub starts 2024-09-20T19:04:04.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:04 smithi142 ceph-mon[32508]: 5.a scrub ok 2024-09-20T19:04:04.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:04 smithi083 ceph-mon[29421]: pgmap v710: 148 pgs: 3 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 142 active+clean; 6.7 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.02k op/s; 34894/327441 objects misplaced (10.657%); 19 MiB/s, 311 objects/s recovering 2024-09-20T19:04:04.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:04 smithi083 ceph-mon[29421]: 5.a scrub starts 2024-09-20T19:04:04.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:04 smithi083 ceph-mon[29421]: 5.a scrub ok 2024-09-20T19:04:04.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:04 smithi083 ceph-mon[36260]: pgmap v710: 148 pgs: 3 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 142 active+clean; 6.7 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.02k op/s; 34894/327441 objects misplaced (10.657%); 19 MiB/s, 311 objects/s recovering 2024-09-20T19:04:04.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:04 smithi083 ceph-mon[36260]: 5.a scrub starts 2024-09-20T19:04:04.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:04 smithi083 ceph-mon[36260]: 5.a scrub ok 2024-09-20T19:04:04.845 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:04.887 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:04.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:04.888 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:04.989 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:05.029 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:05.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:05.030 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:05.130 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:04:05.163 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:05.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:05.164 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:04:05.265 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:05.297 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-20T19:04:05.314 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:05.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:05.315 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:05.416 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:05.439 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:05.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:05.440 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:05.541 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:05.569 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:05.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:05.570 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:05.670 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:05.702 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:05.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:05.703 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:05.718 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:04:05.804 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:05.842 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:05 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:04:05.842 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:05 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:04:05.842 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:05 smithi083 ceph-mon[29421]: 10.5 scrub ok 2024-09-20T19:04:05.842 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:05 smithi083 ceph-mon[29421]: osdmap e368: 8 total, 8 up, 7 in 2024-09-20T19:04:05.842 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:05 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:04:05.842 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:05 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:04:05.843 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:05 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:04:05.843 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:05 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:04:05.843 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:05 smithi083 ceph-mon[36260]: 10.5 scrub ok 2024-09-20T19:04:05.843 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:05 smithi083 ceph-mon[36260]: osdmap e368: 8 total, 8 up, 7 in 2024-09-20T19:04:05.843 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:05 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:04:05.843 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:05 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:04:05.846 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:05.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:05.848 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:05.850 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:05 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:04:05.850 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:05 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:04:05.850 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:05 smithi142 ceph-mon[32508]: 10.5 scrub ok 2024-09-20T19:04:05.851 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:05 smithi142 ceph-mon[32508]: osdmap e368: 8 total, 8 up, 7 in 2024-09-20T19:04:05.851 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:05 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:04:05.851 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:05 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:04:05.948 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:05.988 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:05.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:05.989 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:06.090 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:06.129 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:06.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:06.130 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:06.231 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:06.267 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:06.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:06.268 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:06.355 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T19:04:06.355 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:04:06.369 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:04:06.414 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:06.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:06.415 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:04:06.516 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:06.549 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:06.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:06.550 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:06.651 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:06.715 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:06.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:06.716 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:06.817 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:06.853 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:06.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:06.854 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:06.861 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T19:04:06.874 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:06 smithi083 ceph-mon[29421]: pgmap v712: 148 pgs: 3 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 142 active+clean; 6.7 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 519 op/s; 34894/327441 objects misplaced (10.657%); 3.0 MiB/s, 48 objects/s recovering 2024-09-20T19:04:06.874 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:06 smithi083 ceph-mon[29421]: osdmap e369: 8 total, 8 up, 7 in 2024-09-20T19:04:06.874 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:06 smithi083 ceph-mon[36260]: pgmap v712: 148 pgs: 3 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 142 active+clean; 6.7 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 519 op/s; 34894/327441 objects misplaced (10.657%); 3.0 MiB/s, 48 objects/s recovering 2024-09-20T19:04:06.874 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:06 smithi083 ceph-mon[36260]: osdmap e369: 8 total, 8 up, 7 in 2024-09-20T19:04:06.882 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:06 smithi142 ceph-mon[32508]: pgmap v712: 148 pgs: 3 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 142 active+clean; 6.7 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 519 op/s; 34894/327441 objects misplaced (10.657%); 3.0 MiB/s, 48 objects/s recovering 2024-09-20T19:04:06.882 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:06 smithi142 ceph-mon[32508]: osdmap e369: 8 total, 8 up, 7 in 2024-09-20T19:04:06.955 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:06.990 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:06.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:06.991 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:07.092 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:07.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:07.130 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:07.131 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:07.212 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_blocked_ops 2024-09-20T19:04:07.232 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:07.270 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:07.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:07.271 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:07.372 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:07.426 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:07.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:07.427 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:07.444 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T19:04:07.512 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:04:07 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:04:07.528 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:04:07.556 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:07.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:07.557 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:04:07.658 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:07.707 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:07.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:07.709 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:07.810 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:07.842 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:07.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:07.844 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:07.944 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:07.975 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:07.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:07.976 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:08.077 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:08.160 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:08.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:08.161 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:08.172 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:04:08.262 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:08.299 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:08.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:08.300 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:08.401 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:08.422 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:08 smithi083 ceph-mon[29421]: pgmap v714: 148 pgs: 3 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 142 active+clean; 6.4 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.28k op/s; 33972/315651 objects misplaced (10.763%); 7.6 MiB/s, 122 objects/s recovering 2024-09-20T19:04:08.422 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:08 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:04:08.422 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:08 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:04:08.422 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:08 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:04:08.423 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:08 smithi083 ceph-mon[36260]: pgmap v714: 148 pgs: 3 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 142 active+clean; 6.4 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.28k op/s; 33972/315651 objects misplaced (10.763%); 7.6 MiB/s, 122 objects/s recovering 2024-09-20T19:04:08.423 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:08 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:04:08.423 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:08 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:04:08.423 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:08 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:04:08.437 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:08.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:08.438 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:08.448 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 7, 6] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 5, 6, 7, 4] 2024-09-20T19:04:08.448 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T19:04:08.448 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-09-20T19:04:08.448 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-20T19:04:08.448 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-20T19:04:08.449 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.4 config set filestore_inject_stall 3 2024-09-20T19:04:08.483 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:08 smithi142 ceph-mon[32508]: pgmap v714: 148 pgs: 3 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 142 active+clean; 6.4 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.28k op/s; 33972/315651 objects misplaced (10.763%); 7.6 MiB/s, 122 objects/s recovering 2024-09-20T19:04:08.483 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:08 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:04:08.483 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:08 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:04:08.483 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:08 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:04:08.539 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:08.572 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:08.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:08.573 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:08.674 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:08.708 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:08.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:08.709 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:08.810 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:08.841 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:08.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:08.842 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:08.874 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:04:08.942 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:08.957 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:04:09.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:09.013 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:09.014 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:09.114 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:09.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:09.169 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:09.169 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:09.270 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:09.312 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:09.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:09.313 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:09.414 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:04:09.463 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:09.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:09.465 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:04:09.488 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T19:04:09.488 INFO:teuthology.orchestra.run.smithi142.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-20T19:04:09.489 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T19:04:09.565 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:09.594 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:09.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:09.595 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:09.696 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:09.732 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:09.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:09.733 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:09.785 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:04:09.785 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:04:09.820 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:04:09.820 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:04:09.833 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:09.858 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:04:09.858 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:04:09.864 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:09.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:09.865 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:09.903 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:04:09.903 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:04:09.944 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:04:09.945 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:04:09.966 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:04:09.991 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:09.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:09.992 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:04:09.999 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:04:09.999 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:04:10.018 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:04:10.042 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:04:10.042 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:04:10.047 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T19:04:10.047 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-20T19:04:10.047 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-20T19:04:10.047 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-20T19:04:10.048 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T19:04:10.092 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:04:10.092 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:04:10.093 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:10.130 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:10.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:10.131 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:10.161 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:09 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:04:10.161 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:09 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:04:10.161 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:09 smithi083 ceph-mon[29421]: pgmap v715: 148 pgs: 3 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 142 active+clean; 6.4 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 669 B/s rd, 0 B/s wr, 1.02k op/s; 33702/313980 objects misplaced (10.734%); 6.0 MiB/s, 95 objects/s recovering 2024-09-20T19:04:10.161 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:09 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1589982612' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:04:10.162 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:09 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:04:10.162 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:09 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:04:10.162 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:09 smithi083 ceph-mon[36260]: pgmap v715: 148 pgs: 3 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 142 active+clean; 6.4 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 669 B/s rd, 0 B/s wr, 1.02k op/s; 33702/313980 objects misplaced (10.734%); 6.0 MiB/s, 95 objects/s recovering 2024-09-20T19:04:10.162 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:09 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1589982612' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:04:10.232 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:10.262 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:10.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:09 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:04:10.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:09 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:04:10.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:09 smithi142 ceph-mon[32508]: pgmap v715: 148 pgs: 3 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 142 active+clean; 6.4 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 669 B/s rd, 0 B/s wr, 1.02k op/s; 33702/313980 objects misplaced (10.734%); 6.0 MiB/s, 95 objects/s recovering 2024-09-20T19:04:10.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:09 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1589982612' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:04:10.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:10.264 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:10.365 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:10.402 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:10.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:10.404 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:10.504 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:10.539 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:10.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:10.540 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:10.641 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:10.678 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:10.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:10.679 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:10.780 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:10.814 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:10.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:10.815 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:10.899 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:04:10 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:04:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:04:10.915 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:10.953 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:10.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:10.954 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:11.054 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:11.089 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:11.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:11.090 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:11.191 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:11.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:11.230 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:11.231 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:11.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:10 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:04:11.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:10 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:04:11.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:10 smithi142 ceph-mon[32508]: osdmap e370: 8 total, 8 up, 7 in 2024-09-20T19:04:11.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:10 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T19:04:11.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:10 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T19:04:11.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:10 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:04:11.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:10 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:04:11.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:10 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:04:11.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:10 smithi083 ceph-mon[29421]: osdmap e370: 8 total, 8 up, 7 in 2024-09-20T19:04:11.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:10 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T19:04:11.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:10 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T19:04:11.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:10 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:04:11.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:10 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:04:11.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:10 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:04:11.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:10 smithi083 ceph-mon[36260]: osdmap e370: 8 total, 8 up, 7 in 2024-09-20T19:04:11.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:10 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T19:04:11.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:10 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T19:04:11.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:10 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:04:11.331 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:04:11.362 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:11.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:11.364 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:04:11.464 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:11.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:11.511 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:11.511 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:11.612 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:11.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:11.646 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:11.646 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:11.747 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:11.784 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:11.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:11.785 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:11.886 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:11.925 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:11.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:11.927 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:12.011 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-20T19:04:12.028 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:12.075 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:12.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:12.076 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:12.096 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_historic_ops 2024-09-20T19:04:12.177 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:04:12.215 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:12.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:12.216 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:04:12.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:12 smithi142 ceph-mon[32508]: pgmap v717: 148 pgs: 3 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 142 active+clean; 6.4 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 774 B/s rd, 0 B/s wr, 679 op/s; 33702/313980 objects misplaced (10.734%); 3.3 MiB/s, 52 objects/s recovering 2024-09-20T19:04:12.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:12 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3206999364' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:04:12.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:12 smithi083 ceph-mon[29421]: pgmap v717: 148 pgs: 3 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 142 active+clean; 6.4 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 774 B/s rd, 0 B/s wr, 679 op/s; 33702/313980 objects misplaced (10.734%); 3.3 MiB/s, 52 objects/s recovering 2024-09-20T19:04:12.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:12 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3206999364' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:04:12.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:12 smithi083 ceph-mon[36260]: pgmap v717: 148 pgs: 3 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 142 active+clean; 6.4 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 774 B/s rd, 0 B/s wr, 679 op/s; 33702/313980 objects misplaced (10.734%); 3.3 MiB/s, 52 objects/s recovering 2024-09-20T19:04:12.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:12 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3206999364' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:04:12.316 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:12.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:12.359 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:12.359 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:12.460 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:12.493 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:12.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:12.494 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:12.595 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:12.627 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:12.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:12.628 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:12.729 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:12.763 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:12.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:12.764 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:12.865 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:12.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:12.898 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:12.898 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:12.999 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:13.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:13.037 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:13.038 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:13.138 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:13.171 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:13.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:13.173 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:13.273 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:13.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:13 smithi083 ceph-mon[29421]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T19:04:13.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:13 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3206999364' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:04:13.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:13 smithi083 ceph-mon[29421]: osdmap e371: 8 total, 8 up, 7 in 2024-09-20T19:04:13.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:13 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:04:13.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:13 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:04:13.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:13 smithi083 ceph-mon[36260]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T19:04:13.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:13 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3206999364' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:04:13.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:13 smithi083 ceph-mon[36260]: osdmap e371: 8 total, 8 up, 7 in 2024-09-20T19:04:13.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:13 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:04:13.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:13 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:04:13.304 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:13.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:13.305 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:13.386 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T19:04:13.406 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:04:13.479 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:13.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:13.480 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:04:13.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:13 smithi142 ceph-mon[32508]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T19:04:13.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:13 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3206999364' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:04:13.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:13 smithi142 ceph-mon[32508]: osdmap e371: 8 total, 8 up, 7 in 2024-09-20T19:04:13.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:13 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:04:13.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:13 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:04:13.581 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:13.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:13.617 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:13.617 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:13.718 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:13.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:13.749 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:13.749 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:13.850 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:13.871 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:13.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:13.872 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:13.973 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:04:14.004 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:14.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:14.005 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:04:14.106 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:14.137 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:14.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:14.139 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:14.159 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:04:14.202 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:04:14.239 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:04:14.283 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:14.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:14.284 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:04:14.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:14 smithi083 ceph-mon[29421]: osdmap e372: 8 total, 8 up, 7 in 2024-09-20T19:04:14.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:14 smithi083 ceph-mon[29421]: pgmap v720: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 142 active+clean; 6.1 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 858 op/s; 7/300204 objects degraded (0.002%); 32496/300204 objects misplaced (10.825%); 3.9 MiB/s, 61 objects/s recovering 2024-09-20T19:04:14.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:14 smithi083 ceph-mon[36260]: osdmap e372: 8 total, 8 up, 7 in 2024-09-20T19:04:14.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:14 smithi083 ceph-mon[36260]: pgmap v720: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 142 active+clean; 6.1 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 858 op/s; 7/300204 objects degraded (0.002%); 32496/300204 objects misplaced (10.825%); 3.9 MiB/s, 61 objects/s recovering 2024-09-20T19:04:14.385 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:14.418 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:14.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:14.419 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:14.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:14 smithi142 ceph-mon[32508]: osdmap e372: 8 total, 8 up, 7 in 2024-09-20T19:04:14.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:14 smithi142 ceph-mon[32508]: pgmap v720: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 142 active+clean; 6.1 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 858 op/s; 7/300204 objects degraded (0.002%); 32496/300204 objects misplaced (10.825%); 3.9 MiB/s, 61 objects/s recovering 2024-09-20T19:04:14.519 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:04:14.551 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:14.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:14.552 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:04:14.653 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:04:14.685 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:14.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:14.686 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:04:14.787 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:14.825 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:14.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:14.826 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:14.927 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:04:14.960 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:14.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:14.961 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:04:15.062 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:04:15.084 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:15.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:15.085 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:04:15.185 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:15.194 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 7, 6] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 5, 6, 7, 4] 2024-09-20T19:04:15.195 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T19:04:15.195 INFO:tasks.thrashosds.thrasher:Adding osd 4 2024-09-20T19:04:15.195 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd in 4 2024-09-20T19:04:15.215 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:15.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:15.216 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:15.317 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:15.345 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:15.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:15.346 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:15.360 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:15 smithi083 ceph-mon[29421]: Health check failed: Degraded data redundancy: 7/300204 objects degraded (0.002%), 1 pg degraded (PG_DEGRADED) 2024-09-20T19:04:15.360 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:15 smithi083 ceph-mon[36260]: Health check failed: Degraded data redundancy: 7/300204 objects degraded (0.002%), 1 pg degraded (PG_DEGRADED) 2024-09-20T19:04:15.447 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:15.495 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:15.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:15.496 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:15.512 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:15 smithi142 ceph-mon[32508]: Health check failed: Degraded data redundancy: 7/300204 objects degraded (0.002%), 1 pg degraded (PG_DEGRADED) 2024-09-20T19:04:15.597 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:15.634 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:15.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:15.635 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:15.736 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:15.773 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:15.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:15.774 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:15.875 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:15.906 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:15.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:15.907 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:16.008 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:04:16.039 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:04:16.040 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:04:16.040 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:04:16.040 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:04:16.040 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:04:16.040 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:04:16.040 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:04:16.040 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:04:16.040 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:04:16.040 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:04:16.040 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:04:16.040 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:04:16.040 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:04:16.040 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:04:16.043 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:16.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:16.044 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:04:16.069 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:04:16.069 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:04:16.080 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T19:04:16.145 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:16.185 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:16.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:16.186 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:16.286 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:16.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:16 smithi083 ceph-mon[29421]: pgmap v721: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 142 active+clean; 6.1 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 765 op/s; 7/300204 objects degraded (0.002%); 32496/300204 objects misplaced (10.825%); 3.9 MiB/s, 61 objects/s recovering 2024-09-20T19:04:16.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:16 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/596333349' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:04:16.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:16 smithi083 ceph-mon[36260]: pgmap v721: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 142 active+clean; 6.1 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 765 op/s; 7/300204 objects degraded (0.002%); 32496/300204 objects misplaced (10.825%); 3.9 MiB/s, 61 objects/s recovering 2024-09-20T19:04:16.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:16 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/596333349' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:04:16.319 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:16.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:16.320 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:16.421 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:16.465 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:16.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:16.466 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:16.512 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:16 smithi142 ceph-mon[32508]: pgmap v721: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 142 active+clean; 6.1 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 765 op/s; 7/300204 objects degraded (0.002%); 32496/300204 objects misplaced (10.825%); 3.9 MiB/s, 61 objects/s recovering 2024-09-20T19:04:16.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:16 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/596333349' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:04:16.566 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:16.602 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:16.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:16.603 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:16.704 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:16.738 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:16.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:16.739 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:16.840 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:16.894 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T19:04:16.894 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:04:16.900 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:16.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:16.901 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:17.002 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:17.037 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:17.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:17.038 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:17.138 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:04:17.177 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:17.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:17.178 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:04:17.279 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:17.304 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:17.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:17.305 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:17.405 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:17.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:17.443 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:17.443 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:17.512 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:04:17 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:04:17.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:17 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:04:17.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:17 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:04:17.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:17 smithi142 ceph-mon[32508]: 3.c scrub starts 2024-09-20T19:04:17.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:17 smithi142 ceph-mon[32508]: 3.c scrub ok 2024-09-20T19:04:17.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:17 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:04:17.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:17 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:04:17.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:17 smithi083 ceph-mon[29421]: 3.c scrub starts 2024-09-20T19:04:17.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:17 smithi083 ceph-mon[29421]: 3.c scrub ok 2024-09-20T19:04:17.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:17 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:04:17.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:17 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:04:17.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:17 smithi083 ceph-mon[36260]: 3.c scrub starts 2024-09-20T19:04:17.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:17 smithi083 ceph-mon[36260]: 3.c scrub ok 2024-09-20T19:04:17.544 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:04:17.577 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:17.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:17.578 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:04:17.679 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:04:17.715 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:17.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:17.716 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:04:17.816 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:04:17.850 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:17.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:17.851 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:04:17.952 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:17.985 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:17.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:17.986 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:18.086 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:18.125 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:18.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:18.126 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:18.151 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:04:18.168 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:04:18.180 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-20T19:04:18.227 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:18.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:18.266 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:18.266 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:18.367 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:04:18.400 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:18.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:18.401 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:04:18.466 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:18 smithi083 ceph-mon[29421]: pgmap v722: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 142 active+clean; 5.8 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.23k op/s; 191/287703 objects degraded (0.066%); 31160/287703 objects misplaced (10.831%); 8.1 MiB/s, 130 objects/s recovering 2024-09-20T19:04:18.466 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:18 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:04:18.466 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:18 smithi083 ceph-mon[29421]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T19:04:18.467 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:18 smithi083 ceph-mon[36260]: pgmap v722: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 142 active+clean; 5.8 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.23k op/s; 191/287703 objects degraded (0.066%); 31160/287703 objects misplaced (10.831%); 8.1 MiB/s, 130 objects/s recovering 2024-09-20T19:04:18.467 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:18 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:04:18.467 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:18 smithi083 ceph-mon[36260]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T19:04:18.501 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:18.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:18.531 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:18.531 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:18.632 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:18.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:18.675 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:18.675 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:18.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:18 smithi142 ceph-mon[32508]: pgmap v722: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 142 active+clean; 5.8 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.23k op/s; 191/287703 objects degraded (0.066%); 31160/287703 objects misplaced (10.831%); 8.1 MiB/s, 130 objects/s recovering 2024-09-20T19:04:18.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:18 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:04:18.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:18 smithi142 ceph-mon[32508]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T19:04:18.776 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:04:18.812 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:18.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:18.813 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:04:18.914 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:18.948 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:18.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:18.949 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:19.050 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:04:19.082 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:19.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:19.083 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:04:19.184 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:19.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:19.235 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:19.235 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:19.300 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:04:19.336 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:19.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:19.386 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:19.386 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:19.407 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:19 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:04:19.407 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:19 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:04:19.409 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:19 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:04:19.409 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:19 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:04:19.409 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:19 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mon-a[29417]: 2024-09-20T19:04:19.399+0000 7f6c091c6640 -1 mon.a@0(leader).osd e373 definitely_dead 0 2024-09-20T19:04:19.486 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:19.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:19.525 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:19.525 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:19.626 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:19.658 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:19.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:19.659 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:19.705 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T19:04:19.760 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:04:19.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:19 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:04:19.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:19 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:04:19.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:19.795 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:19.795 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:04:19.896 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:19.934 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:19.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:19.935 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:19.952 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T19:04:19.952 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:04:20.036 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:04:20.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:20.071 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:20.072 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:04:20.172 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:20.210 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:20.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:20.211 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:20.312 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:20.324 INFO:teuthology.orchestra.run.smithi083.stderr:marked in osd.4. 2024-09-20T19:04:20.326 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-20T19:04:20.356 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:20.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:20.357 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:20.458 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:20.479 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:20.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:20.480 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:20.581 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:20.602 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:20.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:20.603 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:20.704 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:20.725 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:20.725 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:20 smithi083 ceph-mon[29421]: pgmap v723: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 142 active+clean; 5.8 GiB data, 19 GiB used, 606 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.17k op/s; 191/285954 objects degraded (0.067%); 30850/285954 objects misplaced (10.788%); 7.3 MiB/s, 116 objects/s recovering 2024-09-20T19:04:20.725 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:20 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 191/285954 objects degraded (0.067%), 1 pg degraded, 3 pgs undersized (PG_DEGRADED) 2024-09-20T19:04:20.726 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:20 smithi083 ceph-mon[29421]: osdmap e373: 8 total, 8 up, 7 in 2024-09-20T19:04:20.726 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:20 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2508889802' entity='client.admin' cmd={"prefix": "osd in", "ids": ["4"]} : dispatch 2024-09-20T19:04:20.726 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:20 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1355137441' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:04:20.726 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:20 smithi083 ceph-mon[36260]: pgmap v723: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 142 active+clean; 5.8 GiB data, 19 GiB used, 606 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.17k op/s; 191/285954 objects degraded (0.067%); 30850/285954 objects misplaced (10.788%); 7.3 MiB/s, 116 objects/s recovering 2024-09-20T19:04:20.726 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:20 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 191/285954 objects degraded (0.067%), 1 pg degraded, 3 pgs undersized (PG_DEGRADED) 2024-09-20T19:04:20.726 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:20 smithi083 ceph-mon[36260]: osdmap e373: 8 total, 8 up, 7 in 2024-09-20T19:04:20.726 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:20 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2508889802' entity='client.admin' cmd={"prefix": "osd in", "ids": ["4"]} : dispatch 2024-09-20T19:04:20.726 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:20 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1355137441' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:04:20.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:20.727 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:20.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:20 smithi142 ceph-mon[32508]: pgmap v723: 148 pgs: 1 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 142 active+clean; 5.8 GiB data, 19 GiB used, 606 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.17k op/s; 191/285954 objects degraded (0.067%); 30850/285954 objects misplaced (10.788%); 7.3 MiB/s, 116 objects/s recovering 2024-09-20T19:04:20.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:20 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 191/285954 objects degraded (0.067%), 1 pg degraded, 3 pgs undersized (PG_DEGRADED) 2024-09-20T19:04:20.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:20 smithi142 ceph-mon[32508]: osdmap e373: 8 total, 8 up, 7 in 2024-09-20T19:04:20.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:20 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2508889802' entity='client.admin' cmd={"prefix": "osd in", "ids": ["4"]} : dispatch 2024-09-20T19:04:20.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:20 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1355137441' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:04:20.828 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:20.839 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_blocked_ops 2024-09-20T19:04:20.849 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:20.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:20.850 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:20.952 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:20.973 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:20.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:20.974 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:21.042 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:04:20 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:04:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:04:21.074 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:21.098 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:21.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:21.099 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:21.200 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:21.222 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:21.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:21.223 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:21.275 INFO:tasks.thrashosds.thrasher:Added osd 4 2024-09-20T19:04:21.324 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:21.348 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:21.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:21.349 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:21.450 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:21.471 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:21.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:21.472 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:21.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:21 smithi083 ceph-mon[29421]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:04:21.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:21 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2508889802' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["4"]}]': finished 2024-09-20T19:04:21.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:21 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1355137441' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T19:04:21.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:21 smithi083 ceph-mon[29421]: osdmap e374: 8 total, 8 up, 8 in 2024-09-20T19:04:21.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:21 smithi083 ceph-mon[29421]: osdmap e375: 8 total, 8 up, 8 in 2024-09-20T19:04:21.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:21 smithi083 ceph-mon[36260]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:04:21.544 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:21 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2508889802' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["4"]}]': finished 2024-09-20T19:04:21.544 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:21 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1355137441' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T19:04:21.544 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:21 smithi083 ceph-mon[36260]: osdmap e374: 8 total, 8 up, 8 in 2024-09-20T19:04:21.544 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:21 smithi083 ceph-mon[36260]: osdmap e375: 8 total, 8 up, 8 in 2024-09-20T19:04:21.573 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:21.598 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:21.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:21.599 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:21.700 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:21.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:21.727 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:21.727 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:21.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:21 smithi142 ceph-mon[32508]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:04:21.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:21 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2508889802' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["4"]}]': finished 2024-09-20T19:04:21.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:21 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1355137441' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T19:04:21.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:21 smithi142 ceph-mon[32508]: osdmap e374: 8 total, 8 up, 8 in 2024-09-20T19:04:21.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:21 smithi142 ceph-mon[32508]: osdmap e375: 8 total, 8 up, 8 in 2024-09-20T19:04:21.828 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:21.852 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:21.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:21.853 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:21.954 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:04:21.985 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:21.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:21.986 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:04:22.087 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:04:22.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:22.121 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:22.121 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:04:22.222 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:22.260 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:22.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:22.262 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:22.362 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:22.395 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:22.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:22.396 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:22.497 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:22.528 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:22.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:22.529 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:22.630 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:22.659 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:22.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:22.661 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:22.761 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:04:22.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:22 smithi142 ceph-mon[32508]: pgmap v727: 148 pgs: 1 unknown, 22 peering, 1 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 119 active+clean; 5.4 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 852 B/s rd, 0 op/s; 191/268272 objects degraded (0.071%); 30850/268272 objects misplaced (11.500%) 2024-09-20T19:04:22.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:22 smithi142 ceph-mon[32508]: Health check failed: Reduced data availability: 12 pgs inactive, 19 pgs peering (PG_AVAILABILITY) 2024-09-20T19:04:22.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:22 smithi142 ceph-mon[32508]: 10.1 starting backfill to osd.4 from (0'0,0'0] MAX to 373'8793 2024-09-20T19:04:22.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:22 smithi142 ceph-mon[32508]: osdmap e376: 8 total, 8 up, 8 in 2024-09-20T19:04:22.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:22 smithi083 ceph-mon[29421]: pgmap v727: 148 pgs: 1 unknown, 22 peering, 1 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 119 active+clean; 5.4 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 852 B/s rd, 0 op/s; 191/268272 objects degraded (0.071%); 30850/268272 objects misplaced (11.500%) 2024-09-20T19:04:22.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:22 smithi083 ceph-mon[29421]: Health check failed: Reduced data availability: 12 pgs inactive, 19 pgs peering (PG_AVAILABILITY) 2024-09-20T19:04:22.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:22 smithi083 ceph-mon[29421]: 10.1 starting backfill to osd.4 from (0'0,0'0] MAX to 373'8793 2024-09-20T19:04:22.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:22 smithi083 ceph-mon[29421]: osdmap e376: 8 total, 8 up, 8 in 2024-09-20T19:04:22.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:22 smithi083 ceph-mon[36260]: pgmap v727: 148 pgs: 1 unknown, 22 peering, 1 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 119 active+clean; 5.4 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 852 B/s rd, 0 op/s; 191/268272 objects degraded (0.071%); 30850/268272 objects misplaced (11.500%) 2024-09-20T19:04:22.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:22 smithi083 ceph-mon[36260]: Health check failed: Reduced data availability: 12 pgs inactive, 19 pgs peering (PG_AVAILABILITY) 2024-09-20T19:04:22.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:22 smithi083 ceph-mon[36260]: 10.1 starting backfill to osd.4 from (0'0,0'0] MAX to 373'8793 2024-09-20T19:04:22.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:22 smithi083 ceph-mon[36260]: osdmap e376: 8 total, 8 up, 8 in 2024-09-20T19:04:22.805 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:22.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:22.806 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:04:22.907 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:22.932 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:22.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:22.934 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:23.034 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:23.070 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:23.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:23.071 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:23.172 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:23.201 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:23.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:23.202 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:23.302 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:23.333 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:23.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:23.334 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:23.435 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:04:23.465 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:23.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:23.467 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:04:23.567 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:23.594 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:23.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:23.595 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:23.673 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:23 smithi083 ceph-mon[29421]: 10.c starting backfill to osd.4 from (0'0,0'0] MAX to 373'8699 2024-09-20T19:04:23.673 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:23 smithi083 ceph-mon[36260]: 10.c starting backfill to osd.4 from (0'0,0'0] MAX to 373'8699 2024-09-20T19:04:23.696 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:23.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:23.735 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:23.735 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:23.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:23 smithi142 ceph-mon[32508]: 10.c starting backfill to osd.4 from (0'0,0'0] MAX to 373'8699 2024-09-20T19:04:23.836 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:04:23.881 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:23.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:23.882 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:04:23.983 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:24.020 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:24.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:24.021 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:24.121 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:24.122 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:04:24.156 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:04:24.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:24.165 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:24.165 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:24.266 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:24.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:24.299 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:24.299 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:24.400 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:04:24.432 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:24.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:24.433 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:04:24.521 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:24 smithi083 ceph-mon[29421]: pgmap v729: 148 pgs: 1 remapped+peering, 1 activating, 22 peering, 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 119 active+clean; 5.6 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 797 op/s; 16169/276402 objects misplaced (5.850%); 16 MiB/s, 255 objects/s recovering 2024-09-20T19:04:24.521 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:24 smithi083 ceph-mon[29421]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 191/268272 objects degraded (0.071%), 1 pg degraded, 3 pgs undersized) 2024-09-20T19:04:24.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:24 smithi083 ceph-mon[36260]: pgmap v729: 148 pgs: 1 remapped+peering, 1 activating, 22 peering, 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 119 active+clean; 5.6 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 797 op/s; 16169/276402 objects misplaced (5.850%); 16 MiB/s, 255 objects/s recovering 2024-09-20T19:04:24.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:24 smithi083 ceph-mon[36260]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 191/268272 objects degraded (0.071%), 1 pg degraded, 3 pgs undersized) 2024-09-20T19:04:24.534 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:24.562 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:24.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:24.563 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:24.664 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:04:24.696 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:24.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:24.697 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:04:24.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:24 smithi142 ceph-mon[32508]: pgmap v729: 148 pgs: 1 remapped+peering, 1 activating, 22 peering, 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 119 active+clean; 5.6 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 797 op/s; 16169/276402 objects misplaced (5.850%); 16 MiB/s, 255 objects/s recovering 2024-09-20T19:04:24.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:24 smithi142 ceph-mon[32508]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 191/268272 objects degraded (0.071%), 1 pg degraded, 3 pgs undersized) 2024-09-20T19:04:24.798 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:24.837 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:24.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:24.838 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:24.864 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T19:04:24.864 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:04:24.939 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:24.969 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:24.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:24.970 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:25.071 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:04:25.099 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:25.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:25.100 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:04:25.201 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:25.225 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:25.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:25.226 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:25.327 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:25.362 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:25.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:25.363 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:25.464 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:25.505 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:25.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:25.507 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:25.607 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:25.640 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:25.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:25.641 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:25.742 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:25.774 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:25.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:25.775 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:25.876 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:25.905 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:25.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:25.907 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:26.010 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:26.023 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T19:04:26.046 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:26.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:26.047 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:26.148 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:26.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:26.190 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:26.190 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:26.276 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 7, 6, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 5, 6, 7, 4] 2024-09-20T19:04:26.277 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T19:04:26.277 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-09-20T19:04:26.277 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-20T19:04:26.277 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-20T19:04:26.277 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.7 config set filestore_inject_stall 3 2024-09-20T19:04:26.290 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:26.316 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:26.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:26.317 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:26.418 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:04:26.450 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:26.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:26.450 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:04:26.551 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:26.586 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:26.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:26.587 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:26.590 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:26 smithi083 ceph-mon[29421]: pgmap v730: 148 pgs: 1 remapped+peering, 1 activating, 22 peering, 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 119 active+clean; 5.6 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 879 B/s rd, 0 B/s wr, 548 op/s; 16169/276402 objects misplaced (5.850%); 11 MiB/s, 175 objects/s recovering 2024-09-20T19:04:26.590 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:26 smithi083 ceph-mon[36260]: pgmap v730: 148 pgs: 1 remapped+peering, 1 activating, 22 peering, 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 119 active+clean; 5.6 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 879 B/s rd, 0 B/s wr, 548 op/s; 16169/276402 objects misplaced (5.850%); 11 MiB/s, 175 objects/s recovering 2024-09-20T19:04:26.688 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:26.703 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:04:26.718 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:26.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:26.720 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:26.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:26 smithi142 ceph-mon[32508]: pgmap v730: 148 pgs: 1 remapped+peering, 1 activating, 22 peering, 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 119 active+clean; 5.6 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 879 B/s rd, 0 B/s wr, 548 op/s; 16169/276402 objects misplaced (5.850%); 11 MiB/s, 175 objects/s recovering 2024-09-20T19:04:26.804 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:04:26.804 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:04:26.804 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:04:26.804 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:04:26.805 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:04:26.805 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:04:26.805 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:04:26.805 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:04:26.805 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:04:26.805 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:04:26.805 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:04:26.805 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:04:26.805 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:04:26.805 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:04:26.805 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:04:26.806 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:04:26.821 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:26.847 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:26.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:26.848 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:26.949 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:26.980 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:26.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:26.981 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:27.082 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:27.122 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:27.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:27.123 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:27.224 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:27.254 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:27.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:27.255 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:27.355 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:27.381 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:27.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:27.382 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:27.483 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:27.487 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T19:04:27.487 INFO:teuthology.orchestra.run.smithi142.stdout: "success": "" 2024-09-20T19:04:27.487 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T19:04:27.488 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:27 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3385750711' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:04:27.488 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:04:27 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:04:27.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:27.527 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:27.528 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:27.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:27 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3385750711' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:04:27.542 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:27 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3385750711' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:04:27.628 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:27.680 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:27.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:27.681 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:27.782 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:27.811 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:27.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:27.812 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:27.913 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:27.949 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:27.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:27.950 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:28.038 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_historic_ops 2024-09-20T19:04:28.054 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:28.129 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:04:28.130 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:28.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:28.131 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:28.232 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:28.252 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:28.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:28.253 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:28.354 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:28.377 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:28.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:28.378 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:28.479 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:28.499 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:28.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:28.500 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:28.537 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:28 smithi083 ceph-mon[29421]: pgmap v731: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 143 active+clean; 5.4 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.01k op/s; 15148/265335 objects misplaced (5.709%); 18 MiB/s, 294 objects/s recovering 2024-09-20T19:04:28.538 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:28 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:04:28.538 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:28 smithi083 ceph-mon[29421]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T19:04:28.538 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:28 smithi083 ceph-mon[29421]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 12 pgs inactive, 19 pgs peering) 2024-09-20T19:04:28.538 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:28 smithi083 ceph-mon[36260]: pgmap v731: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 143 active+clean; 5.4 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.01k op/s; 15148/265335 objects misplaced (5.709%); 18 MiB/s, 294 objects/s recovering 2024-09-20T19:04:28.538 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:28 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:04:28.538 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:28 smithi083 ceph-mon[36260]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T19:04:28.539 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:28 smithi083 ceph-mon[36260]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 12 pgs inactive, 19 pgs peering) 2024-09-20T19:04:28.601 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:28.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:28.638 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:28.638 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:28.733 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:04:28.739 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:28.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:28 smithi142 ceph-mon[32508]: pgmap v731: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 143 active+clean; 5.4 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.01k op/s; 15148/265335 objects misplaced (5.709%); 18 MiB/s, 294 objects/s recovering 2024-09-20T19:04:28.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:28 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:04:28.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:28 smithi142 ceph-mon[32508]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T19:04:28.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:28 smithi142 ceph-mon[32508]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 12 pgs inactive, 19 pgs peering) 2024-09-20T19:04:28.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:28.790 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:28.790 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:28.891 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:28.929 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:28.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:28.930 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:29.031 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:29.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:29.077 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:29.077 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:29.178 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:29.201 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:29.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:29.202 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:29.302 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:29.313 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T19:04:29.313 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-20T19:04:29.313 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-20T19:04:29.313 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-20T19:04:29.313 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.314 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.24955.0:218218 8.6 e233/182 8:6660910c:::benchmark_data_smithi083_97422_object102127:head v 233'13570, mlcod=210'8949)", 2024-09-20T19:04:29.314 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:58:02.122025+0000", 2024-09-20T19:04:29.314 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 387.18269473399999, 2024-09-20T19:04:29.314 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.112690573, 2024-09-20T19:04:29.314 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:04:29.314 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:04:29.314 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:29.314 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:04:29.314 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.314 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:04:29.314 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.122025+0000", 2024-09-20T19:04:29.314 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:29.314 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.315 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.315 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:04:29.315 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.122025+0000", 2024-09-20T19:04:29.315 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:29.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.315 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.315 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:04:29.315 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.122026+0000", 2024-09-20T19:04:29.315 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2399999999999996e-07 2024-09-20T19:04:29.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.315 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.315 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:04:29.315 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.122033+0000", 2024-09-20T19:04:29.315 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.562e-06 2024-09-20T19:04:29.316 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.316 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.316 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:04:29.316 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.214763+0000", 2024-09-20T19:04:29.316 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.092730564000000001 2024-09-20T19:04:29.316 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.316 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.316 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:04:29.316 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.214772+0000", 2024-09-20T19:04:29.316 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7979999999999995e-06 2024-09-20T19:04:29.316 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.316 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.316 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:29.316 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.214856+0000", 2024-09-20T19:04:29.317 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4146000000000004e-05 2024-09-20T19:04:29.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.317 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.317 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:04:29.317 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.214878+0000", 2024-09-20T19:04:29.317 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1722e-05 2024-09-20T19:04:29.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.317 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.317 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:04:29.317 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.234612+0000", 2024-09-20T19:04:29.317 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019734267 2024-09-20T19:04:29.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.317 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.317 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:04:29.317 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.234716+0000", 2024-09-20T19:04:29.318 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010359 2024-09-20T19:04:29.318 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:29.318 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:29.318 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:29.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.318 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.318 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.25484.0:73676 9.2 e265/246 9:4d342ee9:::benchmark_data_smithi083_124552_object73675:head v 265'4551, mlcod=265'4548)", 2024-09-20T19:04:29.318 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:59:27.940611+0000", 2024-09-20T19:04:29.318 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 301.36410903900003, 2024-09-20T19:04:29.318 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.10976248500000001, 2024-09-20T19:04:29.318 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:04:29.318 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:04:29.318 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:29.318 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:04:29.318 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.319 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:04:29.319 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:27.940611+0000", 2024-09-20T19:04:29.319 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:29.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.319 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.319 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:04:29.319 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:27.940611+0000", 2024-09-20T19:04:29.319 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:29.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.319 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.319 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:04:29.319 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:27.940612+0000", 2024-09-20T19:04:29.319 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.271e-06 2024-09-20T19:04:29.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.320 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.320 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:04:29.320 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:27.940637+0000", 2024-09-20T19:04:29.320 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5219999999999999e-05 2024-09-20T19:04:29.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.320 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.320 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:04:29.320 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:28.036375+0000", 2024-09-20T19:04:29.320 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.095737696999999997 2024-09-20T19:04:29.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.320 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.320 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:04:29.320 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:28.036388+0000", 2024-09-20T19:04:29.320 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3001e-05 2024-09-20T19:04:29.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.321 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.321 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:29.321 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:28.038717+0000", 2024-09-20T19:04:29.321 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0023287429999999999 2024-09-20T19:04:29.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.321 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.321 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:04:29.321 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:28.038864+0000", 2024-09-20T19:04:29.321 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014680800000000001 2024-09-20T19:04:29.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.321 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.321 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:04:29.322 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:28.050214+0000", 2024-09-20T19:04:29.322 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011350813 2024-09-20T19:04:29.322 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.322 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.322 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:04:29.322 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:28.050373+0000", 2024-09-20T19:04:29.322 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000158932 2024-09-20T19:04:29.322 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:29.322 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:29.322 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:29.322 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.322 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.322 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.25484.0:124578 9.2 e289/246 9:422abb8a:::benchmark_data_smithi083_124552_object10431:head v 289'7686, mlcod=289'7682)", 2024-09-20T19:04:29.322 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:20.951824+0000", 2024-09-20T19:04:29.323 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 248.352896192, 2024-09-20T19:04:29.323 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.32891357100000002, 2024-09-20T19:04:29.323 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:04:29.323 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:04:29.323 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:29.323 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:04:29.323 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.323 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:04:29.323 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.951824+0000", 2024-09-20T19:04:29.323 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:29.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.323 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.323 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:04:29.323 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.951824+0000", 2024-09-20T19:04:29.323 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:29.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.324 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.324 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:04:29.324 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.951825+0000", 2024-09-20T19:04:29.324 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0470000000000001e-06 2024-09-20T19:04:29.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.324 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.324 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:04:29.324 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.951831+0000", 2024-09-20T19:04:29.324 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2999999999999998e-06 2024-09-20T19:04:29.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.324 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.324 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:04:29.324 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.951841+0000", 2024-09-20T19:04:29.324 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0234999999999999e-05 2024-09-20T19:04:29.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.325 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.325 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:04:29.325 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.951854+0000", 2024-09-20T19:04:29.325 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.287e-05 2024-09-20T19:04:29.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.325 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.325 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:29.325 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.951926+0000", 2024-09-20T19:04:29.325 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1932000000000002e-05 2024-09-20T19:04:29.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.325 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.325 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:04:29.325 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.951943+0000", 2024-09-20T19:04:29.325 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7033000000000001e-05 2024-09-20T19:04:29.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.326 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.326 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:04:29.326 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.280700+0000", 2024-09-20T19:04:29.326 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.328756979 2024-09-20T19:04:29.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.326 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.326 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:04:29.326 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.280737+0000", 2024-09-20T19:04:29.326 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7175000000000002e-05 2024-09-20T19:04:29.326 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:29.326 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:29.326 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:29.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.327 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.327 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.25484.0:124584 9.0 e289/284 9:06443903:::benchmark_data_smithi083_124552_object10437:head v 289'8049, mlcod=289'8046)", 2024-09-20T19:04:29.327 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:20.953013+0000", 2024-09-20T19:04:29.327 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 248.35170687900001, 2024-09-20T19:04:29.327 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.323892601, 2024-09-20T19:04:29.327 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:04:29.327 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:04:29.327 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:29.327 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:04:29.327 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.327 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:04:29.327 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.953013+0000", 2024-09-20T19:04:29.327 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:29.327 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.328 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.328 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:04:29.328 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.953013+0000", 2024-09-20T19:04:29.328 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:29.328 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.328 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.328 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:04:29.328 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.953014+0000", 2024-09-20T19:04:29.328 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0999999999999997e-07 2024-09-20T19:04:29.328 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.328 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.328 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:04:29.328 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.953020+0000", 2024-09-20T19:04:29.328 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8390000000000003e-06 2024-09-20T19:04:29.328 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.329 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.329 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:04:29.329 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.953030+0000", 2024-09-20T19:04:29.329 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8530000000000004e-06 2024-09-20T19:04:29.329 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.329 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.329 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:04:29.329 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.953036+0000", 2024-09-20T19:04:29.329 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1510000000000001e-06 2024-09-20T19:04:29.329 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.329 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.329 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:29.329 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.953099+0000", 2024-09-20T19:04:29.329 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2959000000000005e-05 2024-09-20T19:04:29.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.330 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.330 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:04:29.330 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.953117+0000", 2024-09-20T19:04:29.330 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8559999999999998e-05 2024-09-20T19:04:29.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.330 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.330 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:04:29.330 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.959129+0000", 2024-09-20T19:04:29.330 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0060115500000000001 2024-09-20T19:04:29.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.330 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.330 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:04:29.330 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.276906+0000", 2024-09-20T19:04:29.331 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.31777687900000001 2024-09-20T19:04:29.331 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:29.331 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:29.331 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:29.331 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.331 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.331 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.25484.0:124590 9.6 e289/246 9:6ce021c9:::benchmark_data_smithi083_124552_object10443:head v 289'7744, mlcod=289'7742)", 2024-09-20T19:04:29.331 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:20.956000+0000", 2024-09-20T19:04:29.331 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 248.348719639, 2024-09-20T19:04:29.331 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.32402082500000001, 2024-09-20T19:04:29.331 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:04:29.332 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:04:29.332 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:29.332 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:04:29.332 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.332 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:04:29.332 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.956000+0000", 2024-09-20T19:04:29.332 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:29.332 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.332 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.332 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:04:29.332 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.956000+0000", 2024-09-20T19:04:29.332 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:29.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.333 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:04:29.333 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.956001+0000", 2024-09-20T19:04:29.333 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9600000000000008e-07 2024-09-20T19:04:29.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.333 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:04:29.333 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.956007+0000", 2024-09-20T19:04:29.333 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3199999999999999e-06 2024-09-20T19:04:29.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.333 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:04:29.333 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.956014+0000", 2024-09-20T19:04:29.334 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7880000000000007e-06 2024-09-20T19:04:29.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.334 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.334 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:04:29.334 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.956019+0000", 2024-09-20T19:04:29.334 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5050000000000001e-06 2024-09-20T19:04:29.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.334 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.334 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:29.334 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.956069+0000", 2024-09-20T19:04:29.335 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0467000000000003e-05 2024-09-20T19:04:29.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.335 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.335 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:04:29.335 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.956089+0000", 2024-09-20T19:04:29.335 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.948e-05 2024-09-20T19:04:29.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.335 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.335 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:04:29.335 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.279983+0000", 2024-09-20T19:04:29.335 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.32389452099999999 2024-09-20T19:04:29.336 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.336 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.336 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:04:29.336 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.280021+0000", 2024-09-20T19:04:29.336 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7747999999999998e-05 2024-09-20T19:04:29.336 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:29.336 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:29.336 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:29.336 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.336 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.336 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.25484.0:124594 9.6 e289/246 9:67fbc174:::benchmark_data_smithi083_124552_object10447:head v 289'7745, mlcod=289'7742)", 2024-09-20T19:04:29.336 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:20.960552+0000", 2024-09-20T19:04:29.337 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 248.344167798, 2024-09-20T19:04:29.337 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.32917961699999998, 2024-09-20T19:04:29.337 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:04:29.337 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:04:29.337 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:29.337 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:04:29.337 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.337 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:04:29.337 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.960552+0000", 2024-09-20T19:04:29.337 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:29.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.337 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.338 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:04:29.338 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.960552+0000", 2024-09-20T19:04:29.338 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:29.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.338 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.338 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:04:29.338 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.960553+0000", 2024-09-20T19:04:29.338 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0610000000000001e-06 2024-09-20T19:04:29.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.338 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.338 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:04:29.338 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.960562+0000", 2024-09-20T19:04:29.339 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0580000000000004e-06 2024-09-20T19:04:29.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.339 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.339 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:04:29.339 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.276862+0000", 2024-09-20T19:04:29.339 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.31629938200000002 2024-09-20T19:04:29.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.339 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.339 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:04:29.339 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.276872+0000", 2024-09-20T19:04:29.340 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0614e-05 2024-09-20T19:04:29.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.340 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:29.340 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.276943+0000", 2024-09-20T19:04:29.340 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0933999999999995e-05 2024-09-20T19:04:29.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.340 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:04:29.340 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.276962+0000", 2024-09-20T19:04:29.340 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9293999999999999e-05 2024-09-20T19:04:29.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.341 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.341 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:04:29.341 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.289671+0000", 2024-09-20T19:04:29.341 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.012708129 2024-09-20T19:04:29.341 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.341 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.341 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:04:29.341 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.289732+0000", 2024-09-20T19:04:29.341 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1144999999999998e-05 2024-09-20T19:04:29.341 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:29.341 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:29.342 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:29.342 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.342 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.342 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.25484.0:124596 9.2 e289/246 9:42d01fa2:::benchmark_data_smithi083_124552_object10449:head v 289'7687, mlcod=289'7684)", 2024-09-20T19:04:29.342 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:20.963075+0000", 2024-09-20T19:04:29.342 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 248.341645043, 2024-09-20T19:04:29.342 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.32681833599999999, 2024-09-20T19:04:29.342 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:04:29.342 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:04:29.342 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:29.342 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:04:29.342 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.343 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:04:29.343 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.963075+0000", 2024-09-20T19:04:29.343 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:29.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.344 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.344 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:04:29.344 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.963075+0000", 2024-09-20T19:04:29.344 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:29.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.344 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.344 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:04:29.344 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.963076+0000", 2024-09-20T19:04:29.344 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6600000000000005e-07 2024-09-20T19:04:29.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.345 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.345 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:04:29.345 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.963083+0000", 2024-09-20T19:04:29.345 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9140000000000002e-06 2024-09-20T19:04:29.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.345 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.345 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:04:29.345 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.276845+0000", 2024-09-20T19:04:29.345 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.31376241599999999 2024-09-20T19:04:29.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.345 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.345 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:04:29.346 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.276871+0000", 2024-09-20T19:04:29.346 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6123999999999999e-05 2024-09-20T19:04:29.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.346 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.346 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:29.346 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.276934+0000", 2024-09-20T19:04:29.346 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3147999999999999e-05 2024-09-20T19:04:29.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.346 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.347 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:04:29.347 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.276960+0000", 2024-09-20T19:04:29.347 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5939e-05 2024-09-20T19:04:29.347 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.347 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.347 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:04:29.347 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.289643+0000", 2024-09-20T19:04:29.347 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.012682236 2024-09-20T19:04:29.347 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.347 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.347 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:04:29.347 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.289893+0000", 2024-09-20T19:04:29.347 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00025069300000000002 2024-09-20T19:04:29.348 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:29.348 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:29.348 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:29.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.348 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.348 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.25484.0:210426 9.2 e312/246 9:4c6f61c8:::benchmark_data_smithi083_124552_object96279:head v 312'12986, mlcod=312'12983)", 2024-09-20T19:04:29.348 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:01:34.555445+0000", 2024-09-20T19:04:29.348 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 174.74927485699999, 2024-09-20T19:04:29.348 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.196926772, 2024-09-20T19:04:29.348 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:04:29.348 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:04:29.348 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:29.349 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:04:29.349 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.349 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:04:29.349 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:01:34.555445+0000", 2024-09-20T19:04:29.349 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:29.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.349 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.349 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:04:29.349 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:01:34.555445+0000", 2024-09-20T19:04:29.349 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:29.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.349 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.349 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:04:29.349 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:01:34.555446+0000", 2024-09-20T19:04:29.350 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.023e-06 2024-09-20T19:04:29.350 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.350 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.350 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:04:29.350 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:01:34.555453+0000", 2024-09-20T19:04:29.350 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4910000000000002e-06 2024-09-20T19:04:29.350 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.350 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.350 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:04:29.350 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:01:34.741524+0000", 2024-09-20T19:04:29.350 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.18607137300000001 2024-09-20T19:04:29.350 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.351 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.351 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:04:29.351 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:01:34.741534+0000", 2024-09-20T19:04:29.351 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0219e-05 2024-09-20T19:04:29.351 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.351 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.351 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:29.351 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:01:34.742048+0000", 2024-09-20T19:04:29.351 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000514267 2024-09-20T19:04:29.351 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.351 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.352 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:04:29.352 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:01:34.742068+0000", 2024-09-20T19:04:29.352 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9916999999999999e-05 2024-09-20T19:04:29.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.352 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.352 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:04:29.352 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:01:34.752173+0000", 2024-09-20T19:04:29.352 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.01010463 2024-09-20T19:04:29.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.352 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.352 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:04:29.352 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:01:34.752372+0000", 2024-09-20T19:04:29.352 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019885200000000001 2024-09-20T19:04:29.353 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:29.353 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:29.353 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:29.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.353 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.353 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25775.0:51921 10.3 10:c83df5a9:::benchmark_data_smithi083_148100_object51920:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e340)", 2024-09-20T19:04:29.353 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:02:42.777109+0000", 2024-09-20T19:04:29.353 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 106.527611251, 2024-09-20T19:04:29.353 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.118903313, 2024-09-20T19:04:29.353 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:04:29.353 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:04:29.353 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:29.353 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:04:29.354 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25775", 2024-09-20T19:04:29.354 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:04:29.354 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 51921 2024-09-20T19:04:29.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.354 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:04:29.354 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.354 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:04:29.354 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.777109+0000", 2024-09-20T19:04:29.354 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:29.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.354 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.355 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:04:29.355 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.777109+0000", 2024-09-20T19:04:29.355 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:29.355 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.355 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.355 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:04:29.355 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.777110+0000", 2024-09-20T19:04:29.355 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4300000000000001e-07 2024-09-20T19:04:29.355 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.355 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.355 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:04:29.355 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.777133+0000", 2024-09-20T19:04:29.356 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3312999999999999e-05 2024-09-20T19:04:29.356 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.356 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.356 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:04:29.356 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.777143+0000", 2024-09-20T19:04:29.356 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0111e-05 2024-09-20T19:04:29.356 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.356 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.356 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:04:29.356 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.777150+0000", 2024-09-20T19:04:29.356 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7889999999999997e-06 2024-09-20T19:04:29.356 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.356 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.357 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:29.357 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.777197+0000", 2024-09-20T19:04:29.357 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7525000000000003e-05 2024-09-20T19:04:29.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.357 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.357 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:04:29.357 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.777304+0000", 2024-09-20T19:04:29.357 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106548 2024-09-20T19:04:29.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.357 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.357 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3,5", 2024-09-20T19:04:29.358 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.777571+0000", 2024-09-20T19:04:29.358 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00026748100000000002 2024-09-20T19:04:29.358 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.358 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.358 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:29.358 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.780123+0000", 2024-09-20T19:04:29.358 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002551813 2024-09-20T19:04:29.358 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.358 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.358 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:04:29.358 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.781319+0000", 2024-09-20T19:04:29.358 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001195619 2024-09-20T19:04:29.358 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.359 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.359 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:29.359 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.895858+0000", 2024-09-20T19:04:29.359 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.114538906 2024-09-20T19:04:29.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.359 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.359 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:04:29.359 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.895962+0000", 2024-09-20T19:04:29.359 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000104503 2024-09-20T19:04:29.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.359 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.359 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:04:29.359 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.896012+0000", 2024-09-20T19:04:29.360 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9762000000000002e-05 2024-09-20T19:04:29.360 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:29.360 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:29.360 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:29.360 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.360 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.360 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25775.0:51934 10.3 10:c859bea5:::benchmark_data_smithi083_148100_object51933:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e340)", 2024-09-20T19:04:29.360 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:02:42.785246+0000", 2024-09-20T19:04:29.360 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 106.519473545, 2024-09-20T19:04:29.360 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.11136808500000001, 2024-09-20T19:04:29.360 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:04:29.360 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:04:29.361 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:29.361 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:04:29.361 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25775", 2024-09-20T19:04:29.361 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:04:29.361 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 51934 2024-09-20T19:04:29.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.361 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:04:29.361 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.361 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:04:29.361 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.785246+0000", 2024-09-20T19:04:29.361 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:29.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.362 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.362 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:04:29.362 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.785246+0000", 2024-09-20T19:04:29.362 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:29.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.362 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.362 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:04:29.362 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.785248+0000", 2024-09-20T19:04:29.362 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1200000000000001e-06 2024-09-20T19:04:29.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.362 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.362 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:04:29.363 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.785268+0000", 2024-09-20T19:04:29.363 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0937e-05 2024-09-20T19:04:29.363 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.363 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.363 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:04:29.363 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.785283+0000", 2024-09-20T19:04:29.363 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4209999999999999e-05 2024-09-20T19:04:29.363 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.363 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.363 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:04:29.363 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.785290+0000", 2024-09-20T19:04:29.363 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9750000000000001e-06 2024-09-20T19:04:29.364 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.364 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.364 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:29.364 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.785342+0000", 2024-09-20T19:04:29.364 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2646999999999999e-05 2024-09-20T19:04:29.364 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.364 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.364 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:04:29.364 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.785460+0000", 2024-09-20T19:04:29.364 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000117747 2024-09-20T19:04:29.364 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.364 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.365 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3,5", 2024-09-20T19:04:29.365 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.785577+0000", 2024-09-20T19:04:29.365 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000116709 2024-09-20T19:04:29.365 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.365 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.365 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:04:29.365 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.788761+0000", 2024-09-20T19:04:29.365 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003184391 2024-09-20T19:04:29.365 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.365 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.365 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:29.365 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.788846+0000", 2024-09-20T19:04:29.366 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4473000000000003e-05 2024-09-20T19:04:29.366 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.366 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.366 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:29.366 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.896097+0000", 2024-09-20T19:04:29.366 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.107251207 2024-09-20T19:04:29.366 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.366 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.366 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:04:29.366 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.896577+0000", 2024-09-20T19:04:29.366 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00048043000000000001 2024-09-20T19:04:29.366 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.367 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.367 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:04:29.367 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.896614+0000", 2024-09-20T19:04:29.367 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7239e-05 2024-09-20T19:04:29.367 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:29.367 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:29.367 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:29.367 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.367 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.367 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25775.0:51949 10.3 10:c7df5d6a:::benchmark_data_smithi083_148100_object51948:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e340)", 2024-09-20T19:04:29.367 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:02:42.795193+0000", 2024-09-20T19:04:29.368 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 106.50952687100001, 2024-09-20T19:04:29.368 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.101751296, 2024-09-20T19:04:29.368 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:04:29.368 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:04:29.368 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:29.368 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:04:29.368 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25775", 2024-09-20T19:04:29.368 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:04:29.368 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 51949 2024-09-20T19:04:29.368 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.368 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:04:29.368 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.368 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:04:29.369 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.795193+0000", 2024-09-20T19:04:29.369 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:29.369 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.369 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.369 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:04:29.369 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.795193+0000", 2024-09-20T19:04:29.369 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:29.369 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.369 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.369 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:04:29.369 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.795195+0000", 2024-09-20T19:04:29.369 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.531e-06 2024-09-20T19:04:29.370 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.370 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.370 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:04:29.370 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.795215+0000", 2024-09-20T19:04:29.370 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0599999999999999e-05 2024-09-20T19:04:29.370 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.370 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.370 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:04:29.370 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.795229+0000", 2024-09-20T19:04:29.370 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3372e-05 2024-09-20T19:04:29.370 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.371 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.371 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:04:29.371 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.795236+0000", 2024-09-20T19:04:29.371 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.956e-06 2024-09-20T19:04:29.371 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.371 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.371 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:29.371 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.795294+0000", 2024-09-20T19:04:29.371 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8372000000000003e-05 2024-09-20T19:04:29.371 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.371 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.371 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:04:29.372 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.795398+0000", 2024-09-20T19:04:29.372 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000104336 2024-09-20T19:04:29.372 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.372 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.372 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3,5", 2024-09-20T19:04:29.372 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.795491+0000", 2024-09-20T19:04:29.372 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2470000000000001e-05 2024-09-20T19:04:29.372 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.372 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.372 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:04:29.372 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.797323+0000", 2024-09-20T19:04:29.372 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0018327370000000001 2024-09-20T19:04:29.372 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.373 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.373 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:29.373 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.799501+0000", 2024-09-20T19:04:29.373 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0021777469999999998 2024-09-20T19:04:29.373 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.373 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.373 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:29.373 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.896655+0000", 2024-09-20T19:04:29.373 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.097154307999999995 2024-09-20T19:04:29.373 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.373 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.373 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:04:29.374 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.896811+0000", 2024-09-20T19:04:29.374 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015532400000000001 2024-09-20T19:04:29.374 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.374 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.374 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:04:29.374 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.896944+0000", 2024-09-20T19:04:29.374 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013354299999999999 2024-09-20T19:04:29.374 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:29.374 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:29.374 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:29.374 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.374 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.374 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25775.0:80354 10.3 10:c6711421:::benchmark_data_smithi083_148100_object80353:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e348)", 2024-09-20T19:04:29.375 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:03:04.110702+0000", 2024-09-20T19:04:29.375 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 85.194017697999996, 2024-09-20T19:04:29.375 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.095704600000000001, 2024-09-20T19:04:29.375 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:04:29.375 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:04:29.375 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:29.375 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:04:29.375 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25775", 2024-09-20T19:04:29.375 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:04:29.375 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 80354 2024-09-20T19:04:29.375 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.376 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:04:29.376 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.376 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:04:29.376 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:04.110702+0000", 2024-09-20T19:04:29.376 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:29.376 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.376 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.376 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:04:29.376 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:04.110702+0000", 2024-09-20T19:04:29.376 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:29.376 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.376 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.377 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:04:29.377 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:04.110704+0000", 2024-09-20T19:04:29.377 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5740000000000001e-06 2024-09-20T19:04:29.377 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.377 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.377 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:04:29.377 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:04.110727+0000", 2024-09-20T19:04:29.377 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3176999999999998e-05 2024-09-20T19:04:29.377 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.377 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.377 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:04:29.377 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:04.110739+0000", 2024-09-20T19:04:29.378 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1649e-05 2024-09-20T19:04:29.378 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.378 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.378 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:04:29.378 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:04.110746+0000", 2024-09-20T19:04:29.378 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.6769999999999995e-06 2024-09-20T19:04:29.378 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.378 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.378 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:29.378 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:04.110854+0000", 2024-09-20T19:04:29.378 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000107212 2024-09-20T19:04:29.378 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.379 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.379 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:04:29.379 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:04.110986+0000", 2024-09-20T19:04:29.379 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000132733 2024-09-20T19:04:29.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.379 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.379 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3,5", 2024-09-20T19:04:29.379 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:04.111123+0000", 2024-09-20T19:04:29.379 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013673799999999999 2024-09-20T19:04:29.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.379 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.379 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:29.380 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:04.117328+0000", 2024-09-20T19:04:29.380 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0062047170000000002 2024-09-20T19:04:29.380 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.380 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.380 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:04:29.380 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:04.127446+0000", 2024-09-20T19:04:29.380 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010118784 2024-09-20T19:04:29.380 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.380 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.380 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:29.380 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:04.206306+0000", 2024-09-20T19:04:29.380 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.078859329000000006 2024-09-20T19:04:29.381 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.381 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.381 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:04:29.381 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:04.206360+0000", 2024-09-20T19:04:29.381 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4117999999999999e-05 2024-09-20T19:04:29.381 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.381 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.381 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:04:29.381 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:04.206407+0000", 2024-09-20T19:04:29.381 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6891999999999998e-05 2024-09-20T19:04:29.381 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:29.381 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:29.382 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:29.382 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.382 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.382 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25775.0:114185 10.3 10:c6f1f176:::benchmark_data_smithi083_148100_object114184:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e361)", 2024-09-20T19:04:29.382 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:03:46.438931+0000", 2024-09-20T19:04:29.382 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 42.865788690999999, 2024-09-20T19:04:29.382 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.066868371999999995, 2024-09-20T19:04:29.382 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:04:29.382 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:04:29.382 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:29.382 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:04:29.383 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25775", 2024-09-20T19:04:29.383 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:04:29.383 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 114185 2024-09-20T19:04:29.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.383 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:04:29.383 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.383 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:04:29.383 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:46.438931+0000", 2024-09-20T19:04:29.383 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:29.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.383 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.383 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:04:29.384 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:46.438931+0000", 2024-09-20T19:04:29.384 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:29.384 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.384 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.384 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:04:29.384 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:46.438932+0000", 2024-09-20T19:04:29.384 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2529999999999999e-06 2024-09-20T19:04:29.384 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.384 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.384 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:04:29.384 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:46.438953+0000", 2024-09-20T19:04:29.385 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0364e-05 2024-09-20T19:04:29.385 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.385 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.385 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:04:29.385 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:46.438965+0000", 2024-09-20T19:04:29.385 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2261e-05 2024-09-20T19:04:29.385 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.385 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.385 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:04:29.385 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:46.438974+0000", 2024-09-20T19:04:29.385 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4980000000000006e-06 2024-09-20T19:04:29.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.386 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.386 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:29.386 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:46.439037+0000", 2024-09-20T19:04:29.386 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3570000000000003e-05 2024-09-20T19:04:29.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.386 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.386 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:04:29.386 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:46.439174+0000", 2024-09-20T19:04:29.386 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013725200000000001 2024-09-20T19:04:29.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.387 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.387 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3,5", 2024-09-20T19:04:29.387 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:46.439339+0000", 2024-09-20T19:04:29.387 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000164346 2024-09-20T19:04:29.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.387 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.387 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:29.387 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:46.499629+0000", 2024-09-20T19:04:29.387 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.060290245999999999 2024-09-20T19:04:29.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.387 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.387 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:29.387 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:46.499807+0000", 2024-09-20T19:04:29.388 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017799499999999999 2024-09-20T19:04:29.388 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.388 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.388 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:04:29.388 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:46.505585+0000", 2024-09-20T19:04:29.388 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0057782679999999996 2024-09-20T19:04:29.388 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.388 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.388 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:04:29.388 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:46.505751+0000", 2024-09-20T19:04:29.388 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016573 2024-09-20T19:04:29.388 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.388 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.389 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:04:29.389 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:46.505800+0000", 2024-09-20T19:04:29.389 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8588999999999998e-05 2024-09-20T19:04:29.389 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:29.389 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:29.389 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:29.389 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.389 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.389 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.25775.0:114741 10.8 e361/325 10:1a525e15:::benchmark_data_smithi083_148100_object114740:head v 361'7078, mlcod=361'7072)", 2024-09-20T19:04:29.389 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:03:47.220020+0000", 2024-09-20T19:04:29.389 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 42.084699956999998, 2024-09-20T19:04:29.389 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.11739464400000001, 2024-09-20T19:04:29.390 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:04:29.390 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:04:29.390 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:29.390 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:04:29.390 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.390 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:04:29.390 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.220020+0000", 2024-09-20T19:04:29.390 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:29.390 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.390 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.390 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:04:29.391 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.220020+0000", 2024-09-20T19:04:29.391 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:29.391 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.391 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.391 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:04:29.391 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.220021+0000", 2024-09-20T19:04:29.391 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9400000000000005e-07 2024-09-20T19:04:29.391 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.391 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.391 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:04:29.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:29.392 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:29.393 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:29.393 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-20T19:04:29.394 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.220043+0000", 2024-09-20T19:04:29.394 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1846999999999999e-05 2024-09-20T19:04:29.394 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.394 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.395 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:04:29.395 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.220051+0000", 2024-09-20T19:04:29.395 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9020000000000005e-06 2024-09-20T19:04:29.395 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.395 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.396 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:04:29.396 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.220058+0000", 2024-09-20T19:04:29.396 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5350000000000001e-06 2024-09-20T19:04:29.396 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.396 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.396 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:29.396 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.220102+0000", 2024-09-20T19:04:29.396 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3554999999999999e-05 2024-09-20T19:04:29.396 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.396 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.397 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:04:29.397 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.220118+0000", 2024-09-20T19:04:29.397 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6895999999999999e-05 2024-09-20T19:04:29.397 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.397 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.397 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:04:29.397 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.222643+0000", 2024-09-20T19:04:29.397 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0025243190000000001 2024-09-20T19:04:29.397 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.397 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.397 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:04:29.398 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.337415+0000", 2024-09-20T19:04:29.398 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.114771896 2024-09-20T19:04:29.398 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:29.398 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:29.398 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:29.398 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.398 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.398 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25775.0:114742 10.5 10:a8d0d8fc:::benchmark_data_smithi083_148100_object114741:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e361)", 2024-09-20T19:04:29.398 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:03:47.222659+0000", 2024-09-20T19:04:29.398 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 42.082060661, 2024-09-20T19:04:29.398 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.13039831599999999, 2024-09-20T19:04:29.399 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:04:29.399 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:04:29.399 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:29.399 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:04:29.399 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25775", 2024-09-20T19:04:29.399 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:04:29.399 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 114742 2024-09-20T19:04:29.399 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.399 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:04:29.399 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.400 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:04:29.400 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.222659+0000", 2024-09-20T19:04:29.400 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:29.400 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.400 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.400 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:04:29.400 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.222659+0000", 2024-09-20T19:04:29.400 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:29.400 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.400 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.400 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:04:29.400 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.222661+0000", 2024-09-20T19:04:29.401 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5510000000000001e-06 2024-09-20T19:04:29.401 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.401 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.401 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:04:29.401 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.222684+0000", 2024-09-20T19:04:29.401 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2759999999999999e-05 2024-09-20T19:04:29.401 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.401 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.401 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:04:29.401 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.337385+0000", 2024-09-20T19:04:29.401 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.114700989 2024-09-20T19:04:29.401 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.402 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.402 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:04:29.402 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.337402+0000", 2024-09-20T19:04:29.402 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7537000000000002e-05 2024-09-20T19:04:29.402 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.402 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.402 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:29.402 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.337469+0000", 2024-09-20T19:04:29.402 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6610000000000001e-05 2024-09-20T19:04:29.402 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.402 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.403 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:04:29.403 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.337616+0000", 2024-09-20T19:04:29.403 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000147274 2024-09-20T19:04:29.403 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.403 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.403 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,3", 2024-09-20T19:04:29.403 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.337807+0000", 2024-09-20T19:04:29.403 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019082800000000001 2024-09-20T19:04:29.403 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.403 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.403 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:04:29.403 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.340737+0000", 2024-09-20T19:04:29.403 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0029298409999999999 2024-09-20T19:04:29.404 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.404 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.404 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:29.404 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.347617+0000", 2024-09-20T19:04:29.404 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0068798410000000003 2024-09-20T19:04:29.404 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.404 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.404 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:29.404 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.352900+0000", 2024-09-20T19:04:29.404 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.005283444 2024-09-20T19:04:29.404 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.404 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.405 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:04:29.405 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.352958+0000", 2024-09-20T19:04:29.405 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8208999999999998e-05 2024-09-20T19:04:29.405 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.405 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.405 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:04:29.405 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.353058+0000", 2024-09-20T19:04:29.405 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9432000000000006e-05 2024-09-20T19:04:29.405 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:29.405 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:29.405 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:29.405 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.406 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.406 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.25775.0:114753 10.9 e361/325 10:9b143418:::benchmark_data_smithi083_148100_object114752:head v 361'7224, mlcod=361'7222)", 2024-09-20T19:04:29.406 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:03:47.234595+0000", 2024-09-20T19:04:29.406 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 42.070124956000001, 2024-09-20T19:04:29.406 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.122020085, 2024-09-20T19:04:29.406 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:04:29.406 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:04:29.406 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:29.406 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:04:29.406 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.406 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:04:29.406 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.234595+0000", 2024-09-20T19:04:29.407 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:29.407 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.407 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.407 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:04:29.407 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.234595+0000", 2024-09-20T19:04:29.407 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:29.407 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.407 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.407 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:04:29.407 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.234596+0000", 2024-09-20T19:04:29.407 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1999999999999999e-06 2024-09-20T19:04:29.407 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.407 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.408 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:04:29.408 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.234667+0000", 2024-09-20T19:04:29.408 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1254999999999995e-05 2024-09-20T19:04:29.408 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.408 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.408 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:04:29.408 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.337440+0000", 2024-09-20T19:04:29.408 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.10277243599999999 2024-09-20T19:04:29.408 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.408 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.408 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:04:29.408 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.337447+0000", 2024-09-20T19:04:29.408 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2099999999999996e-06 2024-09-20T19:04:29.409 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.409 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.409 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:29.409 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.337509+0000", 2024-09-20T19:04:29.409 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2257999999999999e-05 2024-09-20T19:04:29.409 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.409 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.409 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:04:29.409 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.337531+0000", 2024-09-20T19:04:29.409 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1491000000000001e-05 2024-09-20T19:04:29.409 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.409 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.409 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:04:29.410 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.356376+0000", 2024-09-20T19:04:29.410 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.018845641 2024-09-20T19:04:29.410 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.410 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.410 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:04:29.410 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.356615+0000", 2024-09-20T19:04:29.410 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00023859400000000001 2024-09-20T19:04:29.410 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:29.410 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:29.411 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:29.411 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.411 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.411 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25775.0:135924 10.3 10:ce75d974:::benchmark_data_smithi083_148100_object19062:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e372)", 2024-09-20T19:04:29.411 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:04:13.816026+0000", 2024-09-20T19:04:29.411 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.488694132999999, 2024-09-20T19:04:29.411 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.076325722999999998, 2024-09-20T19:04:29.411 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:04:29.411 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:04:29.411 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:29.411 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:04:29.411 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25775", 2024-09-20T19:04:29.412 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:04:29.412 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 135924 2024-09-20T19:04:29.412 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.412 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:04:29.412 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.412 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:04:29.412 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:13.816026+0000", 2024-09-20T19:04:29.412 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:29.412 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.412 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.412 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:04:29.413 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:13.816026+0000", 2024-09-20T19:04:29.413 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:29.413 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.413 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.413 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:04:29.413 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:13.816028+0000", 2024-09-20T19:04:29.413 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.846e-06 2024-09-20T19:04:29.413 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.413 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.413 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:04:29.414 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:13.816034+0000", 2024-09-20T19:04:29.414 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.302e-06 2024-09-20T19:04:29.414 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.414 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.414 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:04:29.414 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:13.816047+0000", 2024-09-20T19:04:29.414 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.313e-05 2024-09-20T19:04:29.414 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.414 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.414 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:04:29.414 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:13.816055+0000", 2024-09-20T19:04:29.415 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.6580000000000002e-06 2024-09-20T19:04:29.415 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.415 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.415 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:29.415 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:13.816117+0000", 2024-09-20T19:04:29.415 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1836999999999998e-05 2024-09-20T19:04:29.415 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.415 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.415 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:04:29.415 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:13.816249+0000", 2024-09-20T19:04:29.415 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013240600000000001 2024-09-20T19:04:29.416 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.416 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.416 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3,5", 2024-09-20T19:04:29.416 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:13.816819+0000", 2024-09-20T19:04:29.416 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00056974800000000002 2024-09-20T19:04:29.416 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.416 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.416 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:04:29.416 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:13.822999+0000", 2024-09-20T19:04:29.416 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0061798240000000004 2024-09-20T19:04:29.416 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.417 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.417 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:29.417 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:13.823116+0000", 2024-09-20T19:04:29.417 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000117555 2024-09-20T19:04:29.417 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.417 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.417 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:29.417 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:13.892260+0000", 2024-09-20T19:04:29.417 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.069144137999999994 2024-09-20T19:04:29.417 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.417 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.418 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:04:29.418 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:13.892308+0000", 2024-09-20T19:04:29.418 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7293999999999998e-05 2024-09-20T19:04:29.418 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.418 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.418 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:04:29.418 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:13.892352+0000", 2024-09-20T19:04:29.418 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3985e-05 2024-09-20T19:04:29.418 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:29.418 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:29.418 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:29.419 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.419 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.419 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25775.0:135928 10.3 10:ce0bf5c6:::benchmark_data_smithi083_148100_object19066:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e372)", 2024-09-20T19:04:29.419 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:04:13.817874+0000", 2024-09-20T19:04:29.419 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 15.486845569, 2024-09-20T19:04:29.419 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.074643641999999996, 2024-09-20T19:04:29.419 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:04:29.419 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:04:29.419 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:29.419 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:04:29.420 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25775", 2024-09-20T19:04:29.420 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:04:29.420 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 135928 2024-09-20T19:04:29.420 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.420 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:04:29.420 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.420 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:04:29.420 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:13.817874+0000", 2024-09-20T19:04:29.420 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:29.420 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.420 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.420 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:04:29.421 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:13.817874+0000", 2024-09-20T19:04:29.421 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:29.421 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.421 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.421 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:04:29.421 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:13.817876+0000", 2024-09-20T19:04:29.421 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.449e-06 2024-09-20T19:04:29.421 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.421 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.421 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:04:29.421 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:13.817881+0000", 2024-09-20T19:04:29.421 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1919999999999996e-06 2024-09-20T19:04:29.421 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.422 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.422 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:04:29.422 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:13.817897+0000", 2024-09-20T19:04:29.422 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5775000000000001e-05 2024-09-20T19:04:29.422 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.422 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.422 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:04:29.422 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:13.817904+0000", 2024-09-20T19:04:29.422 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3379999999999999e-06 2024-09-20T19:04:29.422 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.422 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.423 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:29.423 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:13.819843+0000", 2024-09-20T19:04:29.423 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001938768 2024-09-20T19:04:29.423 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.423 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.423 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:04:29.423 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:13.819981+0000", 2024-09-20T19:04:29.423 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013851799999999999 2024-09-20T19:04:29.423 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.423 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.423 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3,5", 2024-09-20T19:04:29.423 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:13.820096+0000", 2024-09-20T19:04:29.423 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000114707 2024-09-20T19:04:29.424 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.424 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.424 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:04:29.424 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:13.823209+0000", 2024-09-20T19:04:29.424 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0031126029999999998 2024-09-20T19:04:29.424 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.424 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.424 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:29.424 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:13.824364+0000", 2024-09-20T19:04:29.424 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001155655 2024-09-20T19:04:29.424 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.424 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.424 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:29.424 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:13.892416+0000", 2024-09-20T19:04:29.425 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.068051985999999995 2024-09-20T19:04:29.425 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.425 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.425 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:04:29.425 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:13.892475+0000", 2024-09-20T19:04:29.425 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8356000000000002e-05 2024-09-20T19:04:29.425 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.425 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.425 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:04:29.425 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:13.892518+0000", 2024-09-20T19:04:29.425 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3294999999999998e-05 2024-09-20T19:04:29.425 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:29.425 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:29.425 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:29.426 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.426 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.426 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25775.0:137067 10.3 10:c293e5c4:::benchmark_data_smithi083_148100_object20205:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e372)", 2024-09-20T19:04:29.426 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:04:15.028647+0000", 2024-09-20T19:04:29.426 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 14.276072852, 2024-09-20T19:04:29.426 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.067387450000000002, 2024-09-20T19:04:29.426 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:04:29.426 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:04:29.426 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:29.426 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:04:29.426 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25775", 2024-09-20T19:04:29.427 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:04:29.427 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 137067 2024-09-20T19:04:29.427 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.427 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:04:29.427 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.427 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:04:29.427 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:15.028647+0000", 2024-09-20T19:04:29.427 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:29.427 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.427 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.427 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:04:29.427 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:15.028647+0000", 2024-09-20T19:04:29.428 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:29.428 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.428 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.428 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:04:29.428 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:15.028649+0000", 2024-09-20T19:04:29.428 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9439999999999999e-06 2024-09-20T19:04:29.428 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.428 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.428 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:04:29.428 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:15.028655+0000", 2024-09-20T19:04:29.428 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0290000000000002e-06 2024-09-20T19:04:29.429 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.429 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.429 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:04:29.429 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:15.028667+0000", 2024-09-20T19:04:29.429 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1899000000000001e-05 2024-09-20T19:04:29.429 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.429 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.429 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:04:29.429 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:15.028674+0000", 2024-09-20T19:04:29.429 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1790000000000002e-06 2024-09-20T19:04:29.429 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.429 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.430 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:29.430 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:15.028735+0000", 2024-09-20T19:04:29.430 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1111999999999994e-05 2024-09-20T19:04:29.430 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.430 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.430 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:04:29.430 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:15.028861+0000", 2024-09-20T19:04:29.430 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012560300000000001 2024-09-20T19:04:29.430 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.430 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.430 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3,5", 2024-09-20T19:04:29.430 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:15.028985+0000", 2024-09-20T19:04:29.431 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000124189 2024-09-20T19:04:29.431 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.431 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.431 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:29.431 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:15.032950+0000", 2024-09-20T19:04:29.431 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0039649949999999998 2024-09-20T19:04:29.431 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.431 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.431 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:04:29.431 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:15.032974+0000", 2024-09-20T19:04:29.431 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4117e-05 2024-09-20T19:04:29.432 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.432 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.432 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:29.432 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:15.095955+0000", 2024-09-20T19:04:29.432 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.062980807999999999 2024-09-20T19:04:29.432 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.432 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.432 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:04:29.432 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:15.095987+0000", 2024-09-20T19:04:29.432 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2240000000000003e-05 2024-09-20T19:04:29.432 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.432 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.432 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:04:29.433 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:15.096035+0000", 2024-09-20T19:04:29.433 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7335e-05 2024-09-20T19:04:29.433 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:29.433 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:29.433 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:29.433 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.433 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.433 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25775.0:137074 10.5 10:a1b0a7e0:::benchmark_data_smithi083_148100_object20212:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e372)", 2024-09-20T19:04:29.433 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:04:15.033813+0000", 2024-09-20T19:04:29.433 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 14.270907151999999, 2024-09-20T19:04:29.433 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.071077408999999994, 2024-09-20T19:04:29.433 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:04:29.434 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:04:29.434 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:29.434 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:04:29.434 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25775", 2024-09-20T19:04:29.434 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:04:29.434 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 137074 2024-09-20T19:04:29.434 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.434 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:04:29.434 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.434 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:04:29.434 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:15.033813+0000", 2024-09-20T19:04:29.434 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:29.435 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.435 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.435 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:04:29.435 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:15.033813+0000", 2024-09-20T19:04:29.435 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:29.435 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.435 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.435 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:04:29.435 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:15.033814+0000", 2024-09-20T19:04:29.435 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3540000000000001e-06 2024-09-20T19:04:29.435 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.435 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.436 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:04:29.436 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:15.033819+0000", 2024-09-20T19:04:29.436 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2109999999999997e-06 2024-09-20T19:04:29.436 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.436 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.436 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:04:29.436 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:15.033830+0000", 2024-09-20T19:04:29.436 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0356e-05 2024-09-20T19:04:29.436 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.436 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.436 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:04:29.436 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:15.033836+0000", 2024-09-20T19:04:29.436 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7900000000000002e-06 2024-09-20T19:04:29.437 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.437 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.437 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:29.437 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:15.033933+0000", 2024-09-20T19:04:29.437 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6399999999999999e-05 2024-09-20T19:04:29.437 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.437 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.437 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:04:29.437 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:15.095947+0000", 2024-09-20T19:04:29.437 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.062013802 2024-09-20T19:04:29.437 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.437 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.438 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,3", 2024-09-20T19:04:29.438 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:15.096074+0000", 2024-09-20T19:04:29.438 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012756599999999999 2024-09-20T19:04:29.438 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.438 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.438 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:29.438 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:15.103822+0000", 2024-09-20T19:04:29.438 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.007747973 2024-09-20T19:04:29.438 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.438 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.438 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:04:29.438 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:15.103937+0000", 2024-09-20T19:04:29.439 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011495800000000001 2024-09-20T19:04:29.439 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.439 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.439 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:29.439 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:15.104255+0000", 2024-09-20T19:04:29.439 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00031771399999999997 2024-09-20T19:04:29.439 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.439 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.439 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:04:29.439 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:15.104859+0000", 2024-09-20T19:04:29.439 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00060418699999999995 2024-09-20T19:04:29.439 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:29.440 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:29.440 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:04:29.440 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:04:15.104890+0000", 2024-09-20T19:04:29.440 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1097999999999999e-05 2024-09-20T19:04:29.440 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:29.440 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:29.440 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:29.440 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:29.440 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:29.440 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T19:04:29.441 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:29 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/46001148' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:04:29.494 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:29.533 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:29.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:29.534 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:29.542 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:29 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/46001148' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:04:29.634 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T19:04:29.634 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:04:29.659 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:29.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:29.660 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:04:29.761 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:04:29.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:29 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/46001148' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:04:29.802 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:29.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:29.803 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:04:29.904 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:29.937 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:29.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:29.938 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:30.038 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:04:30.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:30.071 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:30.071 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:04:30.172 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:30.220 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:30.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:30.221 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:30.322 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:30.355 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:30.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:30.356 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:30.457 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:30.481 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:30.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:30.482 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:30.512 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:30 smithi142 ceph-mon[32508]: pgmap v732: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 143 active+clean; 5.3 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 55 MiB/s wr, 1.97k op/s; 14885/262845 objects misplaced (5.663%); 64 MiB/s, 1.04k objects/s recovering 2024-09-20T19:04:30.512 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:30 smithi142 ceph-mon[32508]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:04:30.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:30 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/46001148' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T19:04:30.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:30 smithi142 ceph-mon[32508]: osdmap e377: 8 total, 8 up, 8 in 2024-09-20T19:04:30.527 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:30 smithi083 ceph-mon[29421]: pgmap v732: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 143 active+clean; 5.3 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 55 MiB/s wr, 1.97k op/s; 14885/262845 objects misplaced (5.663%); 64 MiB/s, 1.04k objects/s recovering 2024-09-20T19:04:30.527 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:30 smithi083 ceph-mon[29421]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:04:30.527 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:30 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/46001148' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T19:04:30.527 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:30 smithi083 ceph-mon[29421]: osdmap e377: 8 total, 8 up, 8 in 2024-09-20T19:04:30.527 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:30 smithi083 ceph-mon[36260]: pgmap v732: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 143 active+clean; 5.3 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 55 MiB/s wr, 1.97k op/s; 14885/262845 objects misplaced (5.663%); 64 MiB/s, 1.04k objects/s recovering 2024-09-20T19:04:30.527 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:30 smithi083 ceph-mon[36260]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:04:30.527 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:30 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/46001148' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T19:04:30.527 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:30 smithi083 ceph-mon[36260]: osdmap e377: 8 total, 8 up, 8 in 2024-09-20T19:04:30.583 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:30.614 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:30.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:30.615 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:30.716 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:30.752 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:30.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:30.753 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:30.792 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:04:30 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:04:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:04:30.853 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:30.881 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:30.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:30.883 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:30.983 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:30.994 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:04:31.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:31.016 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:31.017 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:31.017 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-20T19:04:31.118 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:31.159 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:31.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:31.161 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:31.261 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:31.300 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:31.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:31.301 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:31.402 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:04:31.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:31.428 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:31.428 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:04:31.529 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:31.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:31.565 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:31.566 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:31.605 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:04:31.605 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:04:31.636 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:04:31.637 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:04:31.666 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:04:31.669 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:04:31.670 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:04:31.691 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:04:31.701 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:31.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:31.702 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:04:31.705 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:04:31.705 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:04:31.735 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:04:31.736 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:04:31.773 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:04:31.773 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:04:31.803 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:31.807 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:04:31.807 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:04:31.834 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:04:31.834 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:04:31.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:31.848 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:31.848 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:31.949 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:04:31.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:31.984 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:31.985 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:04:32.086 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:32.114 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:32.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:32.115 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:32.216 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:32.248 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:32.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:32.249 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:32.350 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:32.385 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:32.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:32.386 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:32.486 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:32.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:32.523 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:32.523 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:32.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:32 smithi083 ceph-mon[29421]: pgmap v734: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 143 active+clean; 5.3 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 553 B/s rd, 49 MiB/s wr, 1.53k op/s; 14885/262089 objects misplaced (5.679%); 54 MiB/s, 877 objects/s recovering 2024-09-20T19:04:32.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:32 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:04:32.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:32 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:04:32.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:32 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2185478506' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:04:32.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:32 smithi083 ceph-mon[36260]: pgmap v734: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 143 active+clean; 5.3 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 553 B/s rd, 49 MiB/s wr, 1.53k op/s; 14885/262089 objects misplaced (5.679%); 54 MiB/s, 877 objects/s recovering 2024-09-20T19:04:32.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:32 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:04:32.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:32 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:04:32.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:32 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2185478506' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:04:32.578 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T19:04:32.624 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:32.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:32.658 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:32.658 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:32.759 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:04:32.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:32 smithi142 ceph-mon[32508]: pgmap v734: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 143 active+clean; 5.3 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 553 B/s rd, 49 MiB/s wr, 1.53k op/s; 14885/262089 objects misplaced (5.679%); 54 MiB/s, 877 objects/s recovering 2024-09-20T19:04:32.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:32 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:04:32.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:32 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:04:32.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:32 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2185478506' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:04:32.796 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:32.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:32.797 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:04:32.872 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T19:04:32.873 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:04:32.898 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:32.936 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:32.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:32.937 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:33.037 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:33.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:33.065 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:33.065 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:33.165 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:33.198 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:33.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:33.199 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:33.300 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:04:33.330 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:33.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:33.331 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:04:33.432 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:33.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:33.475 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:33.475 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:33.576 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:33.608 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:33.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:33.609 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:33.709 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:33.740 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:33.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:33.740 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:33.819 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_blocked_ops 2024-09-20T19:04:33.842 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:33.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:33.905 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:33.906 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:34.007 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:04:34.044 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:34.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:34.045 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:04:34.146 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:34.174 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:34.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:34.175 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:34.276 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:04:34.331 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:04:34.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:34.332 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:34.332 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:04:34.433 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:34.460 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:34.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:34.461 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:34.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:34 smithi083 ceph-mon[29421]: pgmap v735: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 143 active+clean; 5.1 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 921 B/s rd, 0 B/s wr, 883 op/s; 13860/249924 objects misplaced (5.546%); 8.9 MiB/s, 147 objects/s recovering 2024-09-20T19:04:34.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:34 smithi083 ceph-mon[36260]: pgmap v735: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 143 active+clean; 5.1 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 921 B/s rd, 0 B/s wr, 883 op/s; 13860/249924 objects misplaced (5.546%); 8.9 MiB/s, 147 objects/s recovering 2024-09-20T19:04:34.562 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:04:34.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:34.595 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:34.595 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:04:34.696 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:34.729 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:34.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:34.730 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:34.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:34 smithi142 ceph-mon[32508]: pgmap v735: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 143 active+clean; 5.1 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 921 B/s rd, 0 B/s wr, 883 op/s; 13860/249924 objects misplaced (5.546%); 8.9 MiB/s, 147 objects/s recovering 2024-09-20T19:04:34.831 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:34.867 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:34.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:34.868 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:34.900 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:04:34.968 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:04:35.001 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:35.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:35.002 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:04:35.103 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:35.138 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:35.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:35.139 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:35.211 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 7, 6, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 5, 6, 7, 4] 2024-09-20T19:04:35.211 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T19:04:35.212 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-20T19:04:35.212 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd dump -f json-pretty 2024-09-20T19:04:35.240 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:35.272 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:35.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:35.273 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:35.316 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-20T19:04:35.374 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:35.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:35.410 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:35.410 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:35.511 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:35.520 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:35 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3388387004' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:04:35.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:35 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3388387004' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:04:35.548 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:35.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:35.549 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:35.649 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:35.686 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:35.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:35.687 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:35.707 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T19:04:35.707 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:04:35.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:35 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3388387004' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:04:35.788 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:35.823 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:35.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:35.824 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:35.925 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:35.952 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:35.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:35.954 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:36.054 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:36.087 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:36.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:36.088 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:36.189 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:36.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:36.223 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:36.224 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:36.306 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T19:04:36.324 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:36.383 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:36.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:36.384 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:36.485 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:36.494 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_historic_ops 2024-09-20T19:04:36.534 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:36.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:36.535 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:36.579 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:36 smithi083 ceph-mon[29421]: pgmap v736: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 143 active+clean; 5.1 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 921 B/s rd, 0 B/s wr, 883 op/s; 13860/249924 objects misplaced (5.546%); 8.9 MiB/s, 147 objects/s recovering 2024-09-20T19:04:36.579 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:36 smithi083 ceph-mon[29421]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:04:36.579 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:36 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3388387004' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T19:04:36.579 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:36 smithi083 ceph-mon[29421]: osdmap e378: 8 total, 8 up, 8 in 2024-09-20T19:04:36.579 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:36 smithi083 ceph-mon[29421]: 6.c scrub starts 2024-09-20T19:04:36.579 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:36 smithi083 ceph-mon[29421]: 5.4 scrub starts 2024-09-20T19:04:36.579 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:36 smithi083 ceph-mon[29421]: 6.c scrub ok 2024-09-20T19:04:36.579 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:36 smithi083 ceph-mon[29421]: 5.4 scrub ok 2024-09-20T19:04:36.580 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:36 smithi083 ceph-mon[36260]: pgmap v736: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 143 active+clean; 5.1 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 921 B/s rd, 0 B/s wr, 883 op/s; 13860/249924 objects misplaced (5.546%); 8.9 MiB/s, 147 objects/s recovering 2024-09-20T19:04:36.580 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:36 smithi083 ceph-mon[36260]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:04:36.580 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:36 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3388387004' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T19:04:36.580 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:36 smithi083 ceph-mon[36260]: osdmap e378: 8 total, 8 up, 8 in 2024-09-20T19:04:36.580 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:36 smithi083 ceph-mon[36260]: 6.c scrub starts 2024-09-20T19:04:36.580 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:36 smithi083 ceph-mon[36260]: 5.4 scrub starts 2024-09-20T19:04:36.580 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:36 smithi083 ceph-mon[36260]: 6.c scrub ok 2024-09-20T19:04:36.580 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:36 smithi083 ceph-mon[36260]: 5.4 scrub ok 2024-09-20T19:04:36.636 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:04:36.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:36.681 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:36.681 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:04:36.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:36 smithi142 ceph-mon[32508]: pgmap v736: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 143 active+clean; 5.1 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 921 B/s rd, 0 B/s wr, 883 op/s; 13860/249924 objects misplaced (5.546%); 8.9 MiB/s, 147 objects/s recovering 2024-09-20T19:04:36.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:36 smithi142 ceph-mon[32508]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:04:36.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:36 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3388387004' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T19:04:36.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:36 smithi142 ceph-mon[32508]: osdmap e378: 8 total, 8 up, 8 in 2024-09-20T19:04:36.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:36 smithi142 ceph-mon[32508]: 6.c scrub starts 2024-09-20T19:04:36.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:36 smithi142 ceph-mon[32508]: 5.4 scrub starts 2024-09-20T19:04:36.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:36 smithi142 ceph-mon[32508]: 6.c scrub ok 2024-09-20T19:04:36.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:36 smithi142 ceph-mon[32508]: 5.4 scrub ok 2024-09-20T19:04:36.782 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:36.819 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:36.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:36.820 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:36.920 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:36.962 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:36.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:36.963 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:37.064 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:37.088 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:37.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:37.089 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:37.190 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:37.231 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:37.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:37.232 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:37.333 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:37.359 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:37.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:37.360 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:37.460 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:37.501 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:37.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:37.503 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:37.603 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:37.632 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:37.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:37.633 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:37.734 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:37.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:37.756 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:37.756 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:37.762 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:04:37 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:04:37.857 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:37.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:37.895 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:37.895 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:37.996 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:38.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:38.029 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:38.029 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:38.130 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:04:38.167 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:38.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:38.168 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:04:38.218 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:04:38.242 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:04:38.264 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:04:38.268 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:38.304 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:38.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:38.305 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:38.375 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:38 smithi083 ceph-mon[29421]: pgmap v738: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 143 active+clean; 4.8 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.03k op/s; 13042/238038 objects misplaced (5.479%); 7.9 MiB/s, 126 objects/s recovering 2024-09-20T19:04:38.376 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:38 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:04:38.376 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:38 smithi083 ceph-mon[29421]: 3.1e scrub starts 2024-09-20T19:04:38.376 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:38 smithi083 ceph-mon[29421]: 3.1e scrub ok 2024-09-20T19:04:38.376 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:38 smithi083 ceph-mon[36260]: pgmap v738: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 143 active+clean; 4.8 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.03k op/s; 13042/238038 objects misplaced (5.479%); 7.9 MiB/s, 126 objects/s recovering 2024-09-20T19:04:38.376 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:38 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:04:38.376 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:38 smithi083 ceph-mon[36260]: 3.1e scrub starts 2024-09-20T19:04:38.376 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:38 smithi083 ceph-mon[36260]: 3.1e scrub ok 2024-09-20T19:04:38.406 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:38.447 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:38.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:38.448 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:38.549 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:38.586 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:38.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:38.587 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:38.688 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:38.723 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:38.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:38.724 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:38.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:38 smithi142 ceph-mon[32508]: pgmap v738: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 143 active+clean; 4.8 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.03k op/s; 13042/238038 objects misplaced (5.479%); 7.9 MiB/s, 126 objects/s recovering 2024-09-20T19:04:38.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:38 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:04:38.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:38 smithi142 ceph-mon[32508]: 3.1e scrub starts 2024-09-20T19:04:38.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:38 smithi142 ceph-mon[32508]: 3.1e scrub ok 2024-09-20T19:04:38.825 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:04:38.864 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:38.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:38.865 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:04:38.966 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:39.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:39.005 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:39.005 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:39.106 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:04:39.141 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:39.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:39.142 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:04:39.243 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:04:39.277 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:39.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:39.278 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:04:39.324 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:04:39.324 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:04:39.367 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:04:39.367 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:04:39.378 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:39.402 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:04:39.402 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:04:39.411 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:39.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:39.412 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:39.425 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T19:04:39.425 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T19:04:39.426 INFO:teuthology.orchestra.run.smithi083.stdout: "epoch": 378, 2024-09-20T19:04:39.426 INFO:teuthology.orchestra.run.smithi083.stdout: "fsid": "1ed5a50a-7780-11ef-baf5-efdc52797490", 2024-09-20T19:04:39.426 INFO:teuthology.orchestra.run.smithi083.stdout: "created": "2024-09-20T18:44:10.399800+0000", 2024-09-20T19:04:39.426 INFO:teuthology.orchestra.run.smithi083.stdout: "modified": "2024-09-20T19:04:35.289192+0000", 2024-09-20T19:04:39.426 INFO:teuthology.orchestra.run.smithi083.stdout: "last_up_change": "2024-09-20T19:01:21.809732+0000", 2024-09-20T19:04:39.426 INFO:teuthology.orchestra.run.smithi083.stdout: "last_in_change": "2024-09-20T19:04:20.299488+0000", 2024-09-20T19:04:39.426 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-20T19:04:39.426 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_num": 5804032, 2024-09-20T19:04:39.426 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_set": [ 2024-09-20T19:04:39.426 INFO:teuthology.orchestra.run.smithi083.stdout: "nodeep-scrub", 2024-09-20T19:04:39.426 INFO:teuthology.orchestra.run.smithi083.stdout: "pglog_hardlimit", 2024-09-20T19:04:39.426 INFO:teuthology.orchestra.run.smithi083.stdout: "purged_snapdirs", 2024-09-20T19:04:39.426 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_deletes", 2024-09-20T19:04:39.427 INFO:teuthology.orchestra.run.smithi083.stdout: "sortbitwise" 2024-09-20T19:04:39.427 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T19:04:39.427 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_version": 19, 2024-09-20T19:04:39.427 INFO:teuthology.orchestra.run.smithi083.stdout: "full_ratio": 0.94999998807907104, 2024-09-20T19:04:39.427 INFO:teuthology.orchestra.run.smithi083.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-20T19:04:39.427 INFO:teuthology.orchestra.run.smithi083.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-20T19:04:39.427 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_snapshot": "", 2024-09-20T19:04:39.427 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_max": 10, 2024-09-20T19:04:39.427 INFO:teuthology.orchestra.run.smithi083.stdout: "max_osd": 8, 2024-09-20T19:04:39.427 INFO:teuthology.orchestra.run.smithi083.stdout: "require_min_compat_client": "luminous", 2024-09-20T19:04:39.427 INFO:teuthology.orchestra.run.smithi083.stdout: "min_compat_client": "luminous", 2024-09-20T19:04:39.427 INFO:teuthology.orchestra.run.smithi083.stdout: "require_osd_release": "squid", 2024-09-20T19:04:39.427 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_crimson": false, 2024-09-20T19:04:39.427 INFO:teuthology.orchestra.run.smithi083.stdout: "pools": [ 2024-09-20T19:04:39.428 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.428 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 1, 2024-09-20T19:04:39.428 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": ".mgr", 2024-09-20T19:04:39.428 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-20T18:47:15.285795+0000", 2024-09-20T19:04:39.428 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-20T19:04:39.428 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool", 2024-09-20T19:04:39.428 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-20T19:04:39.428 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 3, 2024-09-20T19:04:39.428 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 2, 2024-09-20T19:04:39.428 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-20T19:04:39.428 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-20T19:04:39.428 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-20T19:04:39.428 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T19:04:39.428 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T19:04:39.429 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-20T19:04:39.429 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-20T19:04:39.429 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-20T19:04:39.429 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 1, 2024-09-20T19:04:39.429 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 1, 2024-09-20T19:04:39.429 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 1, 2024-09-20T19:04:39.429 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 1, 2024-09-20T19:04:39.429 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 1, 2024-09-20T19:04:39.429 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-20T19:04:39.429 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-20T19:04:39.429 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-20T19:04:39.429 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-20T19:04:39.429 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-20T19:04:39.429 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-20T19:04:39.430 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-20T19:04:39.430 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.430 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "361", 2024-09-20T19:04:39.430 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-20T19:04:39.430 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T19:04:39.430 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T19:04:39.430 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-20T19:04:39.430 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-20T19:04:39.430 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 0, 2024-09-20T19:04:39.430 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 0, 2024-09-20T19:04:39.430 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-20T19:04:39.430 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-20T19:04:39.430 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-20T19:04:39.430 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-20T19:04:39.431 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-20T19:04:39.431 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-20T19:04:39.431 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-20T19:04:39.431 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-20T19:04:39.431 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-20T19:04:39.431 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-20T19:04:39.431 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-20T19:04:39.431 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T19:04:39.431 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T19:04:39.431 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T19:04:39.431 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-20T19:04:39.431 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-20T19:04:39.431 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-20T19:04:39.431 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-20T19:04:39.432 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-20T19:04:39.432 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.432 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-20T19:04:39.432 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-20T19:04:39.432 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-20T19:04:39.432 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-20T19:04:39.432 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-20T19:04:39.432 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T19:04:39.432 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-20T19:04:39.432 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-20T19:04:39.432 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-20T19:04:39.432 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-20T19:04:39.432 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-20T19:04:39.433 INFO:teuthology.orchestra.run.smithi083.stdout: "options": { 2024-09-20T19:04:39.433 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_max": 32, 2024-09-20T19:04:39.433 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_min": 1 2024-09-20T19:04:39.433 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.433 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-20T19:04:39.433 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": {} 2024-09-20T19:04:39.433 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.433 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-20T19:04:39.433 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-20T19:04:39.433 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 7.8899998664855957, 2024-09-20T19:04:39.433 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 7.8899998664855957, 2024-09-20T19:04:39.433 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 0.37999999523162842, 2024-09-20T19:04:39.433 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 3, 2024-09-20T19:04:39.433 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 3, 2024-09-20T19:04:39.434 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 1, 2024-09-20T19:04:39.434 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 1, 2024-09-20T19:04:39.434 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 1 2024-09-20T19:04:39.434 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.434 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.434 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.434 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 2, 2024-09-20T19:04:39.434 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": "datapool", 2024-09-20T19:04:39.434 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-20T18:49:43.452984+0000", 2024-09-20T19:04:39.434 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 8193, 2024-09-20T19:04:39.434 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-20T19:04:39.434 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-20T19:04:39.434 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 3, 2024-09-20T19:04:39.434 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 2, 2024-09-20T19:04:39.435 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-20T19:04:39.435 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-20T19:04:39.435 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-20T19:04:39.435 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T19:04:39.435 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T19:04:39.435 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-20T19:04:39.435 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-20T19:04:39.435 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-20T19:04:39.435 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 3, 2024-09-20T19:04:39.435 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 3, 2024-09-20T19:04:39.435 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 3, 2024-09-20T19:04:39.435 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 3, 2024-09-20T19:04:39.436 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 3, 2024-09-20T19:04:39.436 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-20T19:04:39.436 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-20T19:04:39.436 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-20T19:04:39.436 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-20T19:04:39.436 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-20T19:04:39.436 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-20T19:04:39.436 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-20T19:04:39.436 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.436 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "61", 2024-09-20T19:04:39.436 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-20T19:04:39.436 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T19:04:39.436 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T19:04:39.436 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-20T19:04:39.437 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-20T19:04:39.437 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 2, 2024-09-20T19:04:39.437 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 61, 2024-09-20T19:04:39.437 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-20T19:04:39.437 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-20T19:04:39.437 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-20T19:04:39.437 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-20T19:04:39.437 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-20T19:04:39.437 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-20T19:04:39.437 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-20T19:04:39.437 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-20T19:04:39.437 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-20T19:04:39.437 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-20T19:04:39.438 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-20T19:04:39.438 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T19:04:39.438 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T19:04:39.438 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T19:04:39.438 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-20T19:04:39.438 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-20T19:04:39.438 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-20T19:04:39.438 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-20T19:04:39.438 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-20T19:04:39.438 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.438 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-20T19:04:39.438 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-20T19:04:39.438 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-20T19:04:39.438 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-20T19:04:39.439 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-20T19:04:39.439 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T19:04:39.439 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-20T19:04:39.439 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-20T19:04:39.439 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-20T19:04:39.439 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-20T19:04:39.439 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-20T19:04:39.439 INFO:teuthology.orchestra.run.smithi083.stdout: "options": {}, 2024-09-20T19:04:39.439 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-20T19:04:39.439 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd": {} 2024-09-20T19:04:39.439 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.439 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-20T19:04:39.439 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-20T19:04:39.439 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 2.6500000953674316, 2024-09-20T19:04:39.440 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 2.6500000953674316, 2024-09-20T19:04:39.440 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 0.87999999523162842, 2024-09-20T19:04:39.440 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 2.3299999237060547, 2024-09-20T19:04:39.440 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 2.3299999237060547, 2024-09-20T19:04:39.440 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 1, 2024-09-20T19:04:39.440 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 1, 2024-09-20T19:04:39.440 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 1 2024-09-20T19:04:39.440 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.440 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.440 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.440 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 3, 2024-09-20T19:04:39.440 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": ".rgw.root", 2024-09-20T19:04:39.440 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-20T18:49:44.217011+0000", 2024-09-20T19:04:39.440 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-20T19:04:39.441 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool", 2024-09-20T19:04:39.441 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-20T19:04:39.441 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 3, 2024-09-20T19:04:39.441 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 2, 2024-09-20T19:04:39.441 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-20T19:04:39.441 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-20T19:04:39.441 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-20T19:04:39.441 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T19:04:39.441 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T19:04:39.441 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-20T19:04:39.441 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-20T19:04:39.441 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-20T19:04:39.441 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 32, 2024-09-20T19:04:39.441 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 32, 2024-09-20T19:04:39.441 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 32, 2024-09-20T19:04:39.442 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 32, 2024-09-20T19:04:39.442 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 32, 2024-09-20T19:04:39.442 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-20T19:04:39.442 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-20T19:04:39.442 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-20T19:04:39.442 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-20T19:04:39.442 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-20T19:04:39.442 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-20T19:04:39.442 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-20T19:04:39.442 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.442 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "60", 2024-09-20T19:04:39.442 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-20T19:04:39.442 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T19:04:39.442 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T19:04:39.443 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-20T19:04:39.443 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-20T19:04:39.443 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 0, 2024-09-20T19:04:39.443 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 0, 2024-09-20T19:04:39.443 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-20T19:04:39.443 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-20T19:04:39.443 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-20T19:04:39.443 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-20T19:04:39.443 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-20T19:04:39.443 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-20T19:04:39.443 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-20T19:04:39.443 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-20T19:04:39.444 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-20T19:04:39.444 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-20T19:04:39.444 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-20T19:04:39.444 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T19:04:39.444 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T19:04:39.444 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T19:04:39.444 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-20T19:04:39.444 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-20T19:04:39.444 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-20T19:04:39.444 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-20T19:04:39.444 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-20T19:04:39.444 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.444 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-20T19:04:39.444 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-20T19:04:39.445 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-20T19:04:39.445 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-20T19:04:39.445 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-20T19:04:39.445 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T19:04:39.445 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-20T19:04:39.445 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-20T19:04:39.445 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-20T19:04:39.445 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-20T19:04:39.445 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-20T19:04:39.445 INFO:teuthology.orchestra.run.smithi083.stdout: "options": {}, 2024-09-20T19:04:39.445 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-20T19:04:39.445 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw": {} 2024-09-20T19:04:39.445 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.445 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-20T19:04:39.446 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-20T19:04:39.446 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 1.5, 2024-09-20T19:04:39.446 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 1.5, 2024-09-20T19:04:39.446 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 1, 2024-09-20T19:04:39.446 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 1.5, 2024-09-20T19:04:39.446 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 1.5, 2024-09-20T19:04:39.446 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 1, 2024-09-20T19:04:39.446 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 1, 2024-09-20T19:04:39.446 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 1 2024-09-20T19:04:39.446 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.446 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.446 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.446 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 4, 2024-09-20T19:04:39.446 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": "default.rgw.log", 2024-09-20T19:04:39.447 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-20T18:49:46.007275+0000", 2024-09-20T19:04:39.447 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-20T19:04:39.447 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool", 2024-09-20T19:04:39.447 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-20T19:04:39.447 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 3, 2024-09-20T19:04:39.447 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 2, 2024-09-20T19:04:39.447 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-20T19:04:39.447 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-20T19:04:39.447 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-20T19:04:39.447 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T19:04:39.447 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T19:04:39.447 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-20T19:04:39.447 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-20T19:04:39.448 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-20T19:04:39.448 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 32, 2024-09-20T19:04:39.448 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 32, 2024-09-20T19:04:39.448 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 32, 2024-09-20T19:04:39.448 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 32, 2024-09-20T19:04:39.448 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 32, 2024-09-20T19:04:39.448 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-20T19:04:39.448 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-20T19:04:39.448 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-20T19:04:39.448 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-20T19:04:39.448 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-20T19:04:39.448 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-20T19:04:39.448 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-20T19:04:39.448 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.449 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "62", 2024-09-20T19:04:39.449 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-20T19:04:39.449 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T19:04:39.449 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T19:04:39.449 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-20T19:04:39.449 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-20T19:04:39.449 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 0, 2024-09-20T19:04:39.449 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 0, 2024-09-20T19:04:39.449 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-20T19:04:39.449 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-20T19:04:39.449 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-20T19:04:39.449 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-20T19:04:39.449 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-20T19:04:39.449 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-20T19:04:39.449 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-20T19:04:39.450 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-20T19:04:39.450 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-20T19:04:39.450 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-20T19:04:39.450 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-20T19:04:39.450 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T19:04:39.450 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T19:04:39.450 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T19:04:39.450 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-20T19:04:39.450 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-20T19:04:39.450 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-20T19:04:39.450 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-20T19:04:39.450 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-20T19:04:39.450 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.451 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-20T19:04:39.451 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-20T19:04:39.451 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-20T19:04:39.451 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-20T19:04:39.451 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-20T19:04:39.451 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T19:04:39.451 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-20T19:04:39.451 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-20T19:04:39.451 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-20T19:04:39.451 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-20T19:04:39.451 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-20T19:04:39.451 INFO:teuthology.orchestra.run.smithi083.stdout: "options": {}, 2024-09-20T19:04:39.451 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-20T19:04:39.451 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw": {} 2024-09-20T19:04:39.452 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.452 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-20T19:04:39.452 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-20T19:04:39.452 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 2.25, 2024-09-20T19:04:39.452 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 2.25, 2024-09-20T19:04:39.452 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 1, 2024-09-20T19:04:39.452 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 2.25, 2024-09-20T19:04:39.452 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 2.25, 2024-09-20T19:04:39.452 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 1, 2024-09-20T19:04:39.452 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 1, 2024-09-20T19:04:39.452 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 1 2024-09-20T19:04:39.452 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.452 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.452 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.453 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 5, 2024-09-20T19:04:39.453 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": "default.rgw.control", 2024-09-20T19:04:39.453 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-20T18:49:47.961258+0000", 2024-09-20T19:04:39.453 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-20T19:04:39.453 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool", 2024-09-20T19:04:39.453 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-20T19:04:39.453 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 3, 2024-09-20T19:04:39.453 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 2, 2024-09-20T19:04:39.453 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-20T19:04:39.453 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-20T19:04:39.453 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-20T19:04:39.453 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T19:04:39.453 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T19:04:39.453 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-20T19:04:39.454 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-20T19:04:39.454 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-20T19:04:39.454 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 32, 2024-09-20T19:04:39.454 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 32, 2024-09-20T19:04:39.454 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 32, 2024-09-20T19:04:39.454 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 32, 2024-09-20T19:04:39.454 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 32, 2024-09-20T19:04:39.454 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-20T19:04:39.454 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-20T19:04:39.454 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-20T19:04:39.454 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-20T19:04:39.454 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-20T19:04:39.454 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-20T19:04:39.454 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-20T19:04:39.454 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.455 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "64", 2024-09-20T19:04:39.455 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-20T19:04:39.455 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T19:04:39.455 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T19:04:39.455 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-20T19:04:39.455 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-20T19:04:39.455 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 0, 2024-09-20T19:04:39.455 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 0, 2024-09-20T19:04:39.455 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-20T19:04:39.455 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-20T19:04:39.455 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-20T19:04:39.455 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-20T19:04:39.455 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-20T19:04:39.455 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-20T19:04:39.456 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-20T19:04:39.456 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-20T19:04:39.456 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-20T19:04:39.456 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-20T19:04:39.456 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-20T19:04:39.456 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T19:04:39.456 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T19:04:39.456 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T19:04:39.456 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-20T19:04:39.456 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-20T19:04:39.456 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-20T19:04:39.456 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-20T19:04:39.456 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-20T19:04:39.456 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.456 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-20T19:04:39.457 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-20T19:04:39.457 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-20T19:04:39.457 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-20T19:04:39.457 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-20T19:04:39.457 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T19:04:39.457 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-20T19:04:39.457 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-20T19:04:39.457 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-20T19:04:39.457 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-20T19:04:39.457 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-20T19:04:39.457 INFO:teuthology.orchestra.run.smithi083.stdout: "options": {}, 2024-09-20T19:04:39.457 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-20T19:04:39.457 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw": {} 2024-09-20T19:04:39.457 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.458 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-20T19:04:39.458 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-20T19:04:39.458 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 1.25, 2024-09-20T19:04:39.458 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 1.25, 2024-09-20T19:04:39.458 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 1, 2024-09-20T19:04:39.458 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 1.25, 2024-09-20T19:04:39.458 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 1.25, 2024-09-20T19:04:39.458 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 1, 2024-09-20T19:04:39.458 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 1, 2024-09-20T19:04:39.458 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 1 2024-09-20T19:04:39.458 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.459 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.459 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.459 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 6, 2024-09-20T19:04:39.459 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": "default.rgw.meta", 2024-09-20T19:04:39.459 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-20T18:49:50.011276+0000", 2024-09-20T19:04:39.459 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-20T19:04:39.459 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool", 2024-09-20T19:04:39.459 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-20T19:04:39.459 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 3, 2024-09-20T19:04:39.459 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 2, 2024-09-20T19:04:39.459 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-20T19:04:39.460 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-20T19:04:39.460 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-20T19:04:39.460 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T19:04:39.460 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T19:04:39.460 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-20T19:04:39.460 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-20T19:04:39.460 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-20T19:04:39.460 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 32, 2024-09-20T19:04:39.460 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 32, 2024-09-20T19:04:39.460 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 32, 2024-09-20T19:04:39.460 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 32, 2024-09-20T19:04:39.460 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 32, 2024-09-20T19:04:39.460 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-20T19:04:39.460 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-20T19:04:39.461 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-20T19:04:39.461 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-20T19:04:39.461 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-20T19:04:39.461 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-20T19:04:39.461 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-20T19:04:39.461 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.461 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "66", 2024-09-20T19:04:39.461 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-20T19:04:39.461 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T19:04:39.461 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T19:04:39.461 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-20T19:04:39.461 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-20T19:04:39.461 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 0, 2024-09-20T19:04:39.461 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 0, 2024-09-20T19:04:39.462 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-20T19:04:39.462 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-20T19:04:39.462 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-20T19:04:39.462 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-20T19:04:39.462 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-20T19:04:39.462 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-20T19:04:39.462 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-20T19:04:39.462 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-20T19:04:39.462 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-20T19:04:39.462 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-20T19:04:39.462 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-20T19:04:39.462 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T19:04:39.462 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T19:04:39.462 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T19:04:39.463 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-20T19:04:39.463 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-20T19:04:39.463 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-20T19:04:39.463 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-20T19:04:39.463 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-20T19:04:39.463 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.463 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-20T19:04:39.463 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-20T19:04:39.463 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-20T19:04:39.463 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-20T19:04:39.463 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-20T19:04:39.463 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T19:04:39.463 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-20T19:04:39.463 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-20T19:04:39.464 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-20T19:04:39.464 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-20T19:04:39.464 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-20T19:04:39.464 INFO:teuthology.orchestra.run.smithi083.stdout: "options": { 2024-09-20T19:04:39.464 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_bias": 4 2024-09-20T19:04:39.464 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.464 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-20T19:04:39.464 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw": {} 2024-09-20T19:04:39.464 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.464 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-20T19:04:39.464 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-20T19:04:39.464 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 1.75, 2024-09-20T19:04:39.464 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 1.75, 2024-09-20T19:04:39.464 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 1, 2024-09-20T19:04:39.464 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 1.75, 2024-09-20T19:04:39.465 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 1.75, 2024-09-20T19:04:39.465 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 1, 2024-09-20T19:04:39.465 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 1, 2024-09-20T19:04:39.465 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 1 2024-09-20T19:04:39.465 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.465 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.465 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.465 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 10, 2024-09-20T19:04:39.465 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": "unique_pool_3", 2024-09-20T19:04:39.465 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-20T19:02:10.165662+0000", 2024-09-20T19:04:39.465 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-20T19:04:39.465 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool", 2024-09-20T19:04:39.465 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-20T19:04:39.465 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 3, 2024-09-20T19:04:39.465 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 2, 2024-09-20T19:04:39.466 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-20T19:04:39.466 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-20T19:04:39.466 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-20T19:04:39.466 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T19:04:39.466 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T19:04:39.466 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-20T19:04:39.466 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-20T19:04:39.466 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-20T19:04:39.466 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 16, 2024-09-20T19:04:39.466 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 16, 2024-09-20T19:04:39.466 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 16, 2024-09-20T19:04:39.466 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 16, 2024-09-20T19:04:39.466 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 16, 2024-09-20T19:04:39.466 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-20T19:04:39.467 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-20T19:04:39.467 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-20T19:04:39.467 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-20T19:04:39.467 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-20T19:04:39.467 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-20T19:04:39.467 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-20T19:04:39.467 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.467 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "367", 2024-09-20T19:04:39.467 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-20T19:04:39.467 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T19:04:39.467 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T19:04:39.467 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-20T19:04:39.467 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-20T19:04:39.467 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 0, 2024-09-20T19:04:39.468 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 0, 2024-09-20T19:04:39.468 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-20T19:04:39.468 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-20T19:04:39.468 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-20T19:04:39.468 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-20T19:04:39.468 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-20T19:04:39.468 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-20T19:04:39.468 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-20T19:04:39.468 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-20T19:04:39.468 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-20T19:04:39.468 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-20T19:04:39.468 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-20T19:04:39.468 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T19:04:39.468 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T19:04:39.469 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T19:04:39.469 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-20T19:04:39.469 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-20T19:04:39.469 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-20T19:04:39.469 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-20T19:04:39.469 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-20T19:04:39.469 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.469 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-20T19:04:39.469 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-20T19:04:39.469 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-20T19:04:39.469 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-20T19:04:39.469 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-20T19:04:39.469 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T19:04:39.469 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-20T19:04:39.469 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-20T19:04:39.470 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-20T19:04:39.470 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-20T19:04:39.470 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-20T19:04:39.470 INFO:teuthology.orchestra.run.smithi083.stdout: "options": {}, 2024-09-20T19:04:39.470 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-20T19:04:39.470 INFO:teuthology.orchestra.run.smithi083.stdout: "rados": {} 2024-09-20T19:04:39.470 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.470 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-20T19:04:39.470 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-20T19:04:39.470 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 3, 2024-09-20T19:04:39.470 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 3, 2024-09-20T19:04:39.470 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 1, 2024-09-20T19:04:39.470 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 3, 2024-09-20T19:04:39.471 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 3, 2024-09-20T19:04:39.471 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 1, 2024-09-20T19:04:39.471 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 1, 2024-09-20T19:04:39.471 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 1 2024-09-20T19:04:39.471 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.471 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.471 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T19:04:39.471 INFO:teuthology.orchestra.run.smithi083.stdout: "osds": [ 2024-09-20T19:04:39.471 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.471 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 0, 2024-09-20T19:04:39.471 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "c714e1ec-f0c3-4054-9792-61b84e53eaa8", 2024-09-20T19:04:39.471 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T19:04:39.471 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T19:04:39.471 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T19:04:39.471 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T19:04:39.472 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T19:04:39.472 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T19:04:39.472 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 8, 2024-09-20T19:04:39.472 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 375, 2024-09-20T19:04:39.472 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T19:04:39.472 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T19:04:39.472 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T19:04:39.472 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:04:39.472 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.472 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:04:39.472 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6801", 2024-09-20T19:04:39.472 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2574084049 2024-09-20T19:04:39.472 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.472 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.473 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.473 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T19:04:39.473 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:04:39.473 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.473 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:04:39.473 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6802", 2024-09-20T19:04:39.473 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2574084049 2024-09-20T19:04:39.473 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.473 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.473 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.473 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T19:04:39.473 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:04:39.473 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.473 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:04:39.474 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6804", 2024-09-20T19:04:39.474 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2574084049 2024-09-20T19:04:39.474 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.474 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.474 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.474 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T19:04:39.474 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:04:39.474 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.474 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:04:39.474 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6803", 2024-09-20T19:04:39.474 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2574084049 2024-09-20T19:04:39.474 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.474 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.474 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.475 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.83:6801/2574084049", 2024-09-20T19:04:39.475 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.83:6802/2574084049", 2024-09-20T19:04:39.475 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.83:6804/2574084049", 2024-09-20T19:04:39.475 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.83:6803/2574084049", 2024-09-20T19:04:39.475 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T19:04:39.475 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T19:04:39.475 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T19:04:39.475 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.475 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.475 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.475 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 1, 2024-09-20T19:04:39.475 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "9dc0f2a8-bafa-4a42-88f5-0301dd58f78d", 2024-09-20T19:04:39.475 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T19:04:39.475 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T19:04:39.476 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T19:04:39.476 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T19:04:39.476 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:04:39.476 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:04:39.477 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T19:04:39.477 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T19:04:39.477 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 13, 2024-09-20T19:04:39.477 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 375, 2024-09-20T19:04:39.477 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T19:04:39.477 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T19:04:39.477 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T19:04:39.477 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:04:39.477 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.478 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:04:39.478 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6805", 2024-09-20T19:04:39.478 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 4213285798 2024-09-20T19:04:39.478 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.478 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.478 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.478 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T19:04:39.478 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:04:39.478 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.478 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:04:39.479 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6806", 2024-09-20T19:04:39.479 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 4213285798 2024-09-20T19:04:39.479 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.479 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.479 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.479 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T19:04:39.479 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:04:39.479 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.479 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:04:39.479 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6808", 2024-09-20T19:04:39.479 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 4213285798 2024-09-20T19:04:39.479 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.479 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.479 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.480 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T19:04:39.480 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:04:39.480 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.480 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:04:39.480 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6807", 2024-09-20T19:04:39.480 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 4213285798 2024-09-20T19:04:39.480 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.480 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.480 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.480 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.83:6805/4213285798", 2024-09-20T19:04:39.480 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.83:6806/4213285798", 2024-09-20T19:04:39.480 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.83:6808/4213285798", 2024-09-20T19:04:39.480 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.83:6807/4213285798", 2024-09-20T19:04:39.480 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T19:04:39.481 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T19:04:39.481 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T19:04:39.481 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.481 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.481 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.481 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 2, 2024-09-20T19:04:39.481 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "448ab692-ec41-44dc-b2d5-dbefd255b3fd", 2024-09-20T19:04:39.481 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T19:04:39.481 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T19:04:39.481 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T19:04:39.481 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T19:04:39.481 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T19:04:39.481 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T19:04:39.481 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 18, 2024-09-20T19:04:39.482 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 374, 2024-09-20T19:04:39.482 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T19:04:39.482 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T19:04:39.482 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T19:04:39.482 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:04:39.482 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.482 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:04:39.482 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6809", 2024-09-20T19:04:39.482 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 805947571 2024-09-20T19:04:39.482 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.482 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.482 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.482 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T19:04:39.483 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:04:39.483 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.483 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:04:39.483 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6810", 2024-09-20T19:04:39.483 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 805947571 2024-09-20T19:04:39.483 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.483 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.483 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.483 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T19:04:39.483 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:04:39.483 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.483 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:04:39.483 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6812", 2024-09-20T19:04:39.483 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 805947571 2024-09-20T19:04:39.484 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.484 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.484 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.484 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T19:04:39.484 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:04:39.484 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.484 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:04:39.484 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6811", 2024-09-20T19:04:39.484 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 805947571 2024-09-20T19:04:39.484 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.484 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.484 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.484 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.83:6809/805947571", 2024-09-20T19:04:39.484 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.83:6810/805947571", 2024-09-20T19:04:39.484 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.83:6812/805947571", 2024-09-20T19:04:39.485 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.83:6811/805947571", 2024-09-20T19:04:39.485 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T19:04:39.485 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T19:04:39.485 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T19:04:39.485 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.485 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.485 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.485 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 3, 2024-09-20T19:04:39.485 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "7cda5e50-e22e-4d4a-823b-53c8f5061193", 2024-09-20T19:04:39.485 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T19:04:39.485 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T19:04:39.485 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T19:04:39.485 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T19:04:39.485 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T19:04:39.486 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T19:04:39.486 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 25, 2024-09-20T19:04:39.486 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 374, 2024-09-20T19:04:39.486 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T19:04:39.486 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T19:04:39.486 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T19:04:39.486 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:04:39.486 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.486 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:04:39.486 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6813", 2024-09-20T19:04:39.486 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 987237318 2024-09-20T19:04:39.486 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.486 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.487 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.487 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T19:04:39.487 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:04:39.487 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.487 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:04:39.487 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6814", 2024-09-20T19:04:39.487 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 987237318 2024-09-20T19:04:39.487 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.487 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.487 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.487 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T19:04:39.487 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:04:39.487 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.488 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:04:39.488 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6816", 2024-09-20T19:04:39.488 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 987237318 2024-09-20T19:04:39.488 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.488 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.488 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.488 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T19:04:39.488 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:04:39.488 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.488 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:04:39.488 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6815", 2024-09-20T19:04:39.488 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 987237318 2024-09-20T19:04:39.488 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.489 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.489 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.489 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.83:6813/987237318", 2024-09-20T19:04:39.489 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.83:6814/987237318", 2024-09-20T19:04:39.489 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.83:6816/987237318", 2024-09-20T19:04:39.489 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.83:6815/987237318", 2024-09-20T19:04:39.489 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T19:04:39.489 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T19:04:39.489 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T19:04:39.489 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.489 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.489 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.489 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 4, 2024-09-20T19:04:39.489 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "3d62654d-d31d-4310-9934-49e694dd17d9", 2024-09-20T19:04:39.490 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T19:04:39.490 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T19:04:39.490 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T19:04:39.490 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T19:04:39.490 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 31, 2024-09-20T19:04:39.490 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 291, 2024-09-20T19:04:39.490 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 304, 2024-09-20T19:04:39.490 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 375, 2024-09-20T19:04:39.490 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 292, 2024-09-20T19:04:39.490 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T19:04:39.491 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T19:04:39.491 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:04:39.491 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.491 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:04:39.491 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6800", 2024-09-20T19:04:39.491 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3322919402 2024-09-20T19:04:39.491 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.491 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.491 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.491 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T19:04:39.491 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:04:39.491 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.491 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:04:39.491 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6801", 2024-09-20T19:04:39.492 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3322919402 2024-09-20T19:04:39.492 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.492 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.492 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.492 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T19:04:39.492 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:04:39.492 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.492 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:04:39.492 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6803", 2024-09-20T19:04:39.492 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3322919402 2024-09-20T19:04:39.492 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.492 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.492 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.492 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T19:04:39.493 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:04:39.493 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.493 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:04:39.493 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6802", 2024-09-20T19:04:39.493 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3322919402 2024-09-20T19:04:39.493 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.493 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.493 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.493 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.142:6800/3322919402", 2024-09-20T19:04:39.493 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.142:6801/3322919402", 2024-09-20T19:04:39.493 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.142:6803/3322919402", 2024-09-20T19:04:39.493 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.142:6802/3322919402", 2024-09-20T19:04:39.493 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T19:04:39.493 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T19:04:39.494 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T19:04:39.494 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.494 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.494 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.494 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 5, 2024-09-20T19:04:39.494 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "b3e4e21b-639f-4d25-8bbf-ea8922a9222a", 2024-09-20T19:04:39.494 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T19:04:39.494 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T19:04:39.494 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T19:04:39.494 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T19:04:39.494 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T19:04:39.494 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T19:04:39.494 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 38, 2024-09-20T19:04:39.495 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 374, 2024-09-20T19:04:39.495 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T19:04:39.495 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T19:04:39.495 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T19:04:39.495 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:04:39.495 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.495 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:04:39.495 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6804", 2024-09-20T19:04:39.495 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2500466067 2024-09-20T19:04:39.495 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.495 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.495 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.495 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T19:04:39.495 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:04:39.496 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.496 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:04:39.496 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6805", 2024-09-20T19:04:39.496 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2500466067 2024-09-20T19:04:39.496 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.496 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.496 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.496 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T19:04:39.496 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:04:39.496 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.496 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:04:39.496 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6807", 2024-09-20T19:04:39.496 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2500466067 2024-09-20T19:04:39.496 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.497 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.497 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.497 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T19:04:39.497 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:04:39.497 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.497 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:04:39.497 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6806", 2024-09-20T19:04:39.497 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2500466067 2024-09-20T19:04:39.497 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.497 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.497 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.497 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.142:6804/2500466067", 2024-09-20T19:04:39.497 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.142:6805/2500466067", 2024-09-20T19:04:39.497 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.142:6807/2500466067", 2024-09-20T19:04:39.498 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.142:6806/2500466067", 2024-09-20T19:04:39.498 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T19:04:39.498 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T19:04:39.498 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T19:04:39.498 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.498 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.498 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.498 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 6, 2024-09-20T19:04:39.498 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "c5db7107-ccb4-4ae2-9e63-f444897942e3", 2024-09-20T19:04:39.498 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T19:04:39.498 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T19:04:39.498 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T19:04:39.498 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T19:04:39.498 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T19:04:39.498 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T19:04:39.499 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 45, 2024-09-20T19:04:39.499 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 374, 2024-09-20T19:04:39.499 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T19:04:39.499 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T19:04:39.499 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T19:04:39.499 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:04:39.499 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.499 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:04:39.499 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6808", 2024-09-20T19:04:39.499 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1688186226 2024-09-20T19:04:39.499 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.499 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.499 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.499 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T19:04:39.500 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:04:39.500 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.500 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:04:39.500 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6809", 2024-09-20T19:04:39.500 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1688186226 2024-09-20T19:04:39.500 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.500 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.500 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.500 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T19:04:39.500 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:04:39.500 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.500 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:04:39.500 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6811", 2024-09-20T19:04:39.500 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1688186226 2024-09-20T19:04:39.501 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.501 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.501 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.501 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T19:04:39.501 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:04:39.501 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.501 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:04:39.501 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6810", 2024-09-20T19:04:39.501 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1688186226 2024-09-20T19:04:39.501 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.501 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.501 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.501 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.142:6808/1688186226", 2024-09-20T19:04:39.501 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.142:6809/1688186226", 2024-09-20T19:04:39.502 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.142:6811/1688186226", 2024-09-20T19:04:39.502 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.142:6810/1688186226", 2024-09-20T19:04:39.502 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T19:04:39.502 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T19:04:39.502 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T19:04:39.502 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.502 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.502 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.502 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 7, 2024-09-20T19:04:39.502 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "07d16806-e2c8-4acb-b284-d75701e61622", 2024-09-20T19:04:39.502 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T19:04:39.502 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T19:04:39.502 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T19:04:39.502 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T19:04:39.506 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T19:04:39.506 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T19:04:39.506 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 53, 2024-09-20T19:04:39.506 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 375, 2024-09-20T19:04:39.506 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T19:04:39.506 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T19:04:39.506 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T19:04:39.506 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:04:39.506 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.506 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:04:39.506 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6812", 2024-09-20T19:04:39.506 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3679770984 2024-09-20T19:04:39.506 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.507 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.507 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.507 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T19:04:39.507 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:04:39.507 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.507 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:04:39.507 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6813", 2024-09-20T19:04:39.507 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3679770984 2024-09-20T19:04:39.507 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.507 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.507 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.507 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T19:04:39.507 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:04:39.507 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.507 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:04:39.508 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6815", 2024-09-20T19:04:39.508 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3679770984 2024-09-20T19:04:39.508 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.508 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.508 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.508 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T19:04:39.508 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:04:39.508 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.508 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:04:39.508 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6814", 2024-09-20T19:04:39.508 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3679770984 2024-09-20T19:04:39.508 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.508 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.508 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.509 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.142:6812/3679770984", 2024-09-20T19:04:39.509 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.142:6813/3679770984", 2024-09-20T19:04:39.509 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.142:6815/3679770984", 2024-09-20T19:04:39.509 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.142:6814/3679770984", 2024-09-20T19:04:39.509 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T19:04:39.509 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T19:04:39.509 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T19:04:39.509 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.509 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.509 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T19:04:39.509 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_xinfo": [ 2024-09-20T19:04:39.509 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.509 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 0, 2024-09-20T19:04:39.509 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T19:04:39.509 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T19:04:39.510 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T19:04:39.510 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T19:04:39.510 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T19:04:39.510 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T19:04:09.758659+0000", 2024-09-20T19:04:39.510 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T19:04:39.510 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.510 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.510 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 1, 2024-09-20T19:04:39.510 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T19:04:39.510 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T19:04:39.510 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T19:04:39.510 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T19:04:39.510 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T19:04:39.511 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T19:03:47.301035+0000", 2024-09-20T19:04:39.511 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T19:04:39.511 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.511 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.511 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 2, 2024-09-20T19:04:39.511 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T19:04:39.511 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T19:04:39.511 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T19:04:39.511 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T19:04:39.511 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T19:04:39.511 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T19:03:40.518081+0000", 2024-09-20T19:04:39.511 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T19:04:39.511 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.511 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.511 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 3, 2024-09-20T19:04:39.512 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T19:04:39.512 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T19:04:39.512 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T19:04:39.512 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T19:04:39.512 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T19:04:39.512 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T19:04:18.454375+0000", 2024-09-20T19:04:39.512 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T19:04:39.512 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.512 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.512 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 4, 2024-09-20T19:04:39.512 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T19:04:39.512 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T19:04:39.512 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T19:04:39.512 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T19:04:39.512 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T19:04:39.512 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T19:04:04.763014+0000", 2024-09-20T19:04:39.513 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 291 2024-09-20T19:04:39.513 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.513 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.513 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 5, 2024-09-20T19:04:39.513 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T19:04:39.513 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T19:04:39.513 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T19:04:39.513 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T19:04:39.513 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T19:04:39.513 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T19:03:40.295266+0000", 2024-09-20T19:04:39.513 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T19:04:39.513 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.513 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.513 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 6, 2024-09-20T19:04:39.513 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T19:04:39.514 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T19:04:39.514 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T19:04:39.514 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T19:04:39.514 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T19:04:39.514 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T19:04:03.749517+0000", 2024-09-20T19:04:39.514 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T19:04:39.514 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.514 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.514 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 7, 2024-09-20T19:04:39.514 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T19:04:39.514 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T19:04:39.514 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T19:04:39.514 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T19:04:39.514 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T19:04:39.514 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T19:04:12.175096+0000", 2024-09-20T19:04:39.515 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T19:04:39.515 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.515 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T19:04:39.515 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_upmap": [ 2024-09-20T19:04:39.515 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.515 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "3.c", 2024-09-20T19:04:39.515 INFO:teuthology.orchestra.run.smithi083.stdout: "osds": [ 2024-09-20T19:04:39.515 INFO:teuthology.orchestra.run.smithi083.stdout: 2, 2024-09-20T19:04:39.515 INFO:teuthology.orchestra.run.smithi083.stdout: 7, 2024-09-20T19:04:39.515 INFO:teuthology.orchestra.run.smithi083.stdout: 6 2024-09-20T19:04:39.515 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.515 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.515 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T19:04:39.515 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_upmap_items": [ 2024-09-20T19:04:39.515 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.516 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "4.4", 2024-09-20T19:04:39.516 INFO:teuthology.orchestra.run.smithi083.stdout: "mappings": [ 2024-09-20T19:04:39.516 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.516 INFO:teuthology.orchestra.run.smithi083.stdout: "from": 3, 2024-09-20T19:04:39.516 INFO:teuthology.orchestra.run.smithi083.stdout: "to": 5 2024-09-20T19:04:39.516 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.516 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.516 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.516 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.516 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "4.f", 2024-09-20T19:04:39.516 INFO:teuthology.orchestra.run.smithi083.stdout: "mappings": [ 2024-09-20T19:04:39.516 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.516 INFO:teuthology.orchestra.run.smithi083.stdout: "from": 1, 2024-09-20T19:04:39.516 INFO:teuthology.orchestra.run.smithi083.stdout: "to": 2 2024-09-20T19:04:39.517 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.517 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.517 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.517 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.517 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "4.14", 2024-09-20T19:04:39.517 INFO:teuthology.orchestra.run.smithi083.stdout: "mappings": [ 2024-09-20T19:04:39.517 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.517 INFO:teuthology.orchestra.run.smithi083.stdout: "from": 1, 2024-09-20T19:04:39.517 INFO:teuthology.orchestra.run.smithi083.stdout: "to": 5 2024-09-20T19:04:39.517 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.517 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.517 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.517 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.517 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "4.18", 2024-09-20T19:04:39.518 INFO:teuthology.orchestra.run.smithi083.stdout: "mappings": [ 2024-09-20T19:04:39.518 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.518 INFO:teuthology.orchestra.run.smithi083.stdout: "from": 1, 2024-09-20T19:04:39.518 INFO:teuthology.orchestra.run.smithi083.stdout: "to": 2 2024-09-20T19:04:39.518 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.518 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.518 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.518 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.518 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "4.1e", 2024-09-20T19:04:39.518 INFO:teuthology.orchestra.run.smithi083.stdout: "mappings": [ 2024-09-20T19:04:39.518 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.518 INFO:teuthology.orchestra.run.smithi083.stdout: "from": 3, 2024-09-20T19:04:39.518 INFO:teuthology.orchestra.run.smithi083.stdout: "to": 2 2024-09-20T19:04:39.518 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.519 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.519 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.519 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T19:04:39.519 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_upmap_primaries": [], 2024-09-20T19:04:39.519 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_temp": [ 2024-09-20T19:04:39.519 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.519 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "10.0", 2024-09-20T19:04:39.519 INFO:teuthology.orchestra.run.smithi083.stdout: "osds": [ 2024-09-20T19:04:39.519 INFO:teuthology.orchestra.run.smithi083.stdout: 6, 2024-09-20T19:04:39.519 INFO:teuthology.orchestra.run.smithi083.stdout: 1 2024-09-20T19:04:39.519 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.519 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.519 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.519 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "10.1", 2024-09-20T19:04:39.520 INFO:teuthology.orchestra.run.smithi083.stdout: "osds": [ 2024-09-20T19:04:39.520 INFO:teuthology.orchestra.run.smithi083.stdout: 0, 2024-09-20T19:04:39.520 INFO:teuthology.orchestra.run.smithi083.stdout: 7, 2024-09-20T19:04:39.520 INFO:teuthology.orchestra.run.smithi083.stdout: 1 2024-09-20T19:04:39.520 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.520 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.520 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.520 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "10.6", 2024-09-20T19:04:39.520 INFO:teuthology.orchestra.run.smithi083.stdout: "osds": [ 2024-09-20T19:04:39.520 INFO:teuthology.orchestra.run.smithi083.stdout: 5, 2024-09-20T19:04:39.520 INFO:teuthology.orchestra.run.smithi083.stdout: 0 2024-09-20T19:04:39.520 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.520 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.520 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.521 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "10.c", 2024-09-20T19:04:39.521 INFO:teuthology.orchestra.run.smithi083.stdout: "osds": [ 2024-09-20T19:04:39.521 INFO:teuthology.orchestra.run.smithi083.stdout: 7, 2024-09-20T19:04:39.521 INFO:teuthology.orchestra.run.smithi083.stdout: 0, 2024-09-20T19:04:39.521 INFO:teuthology.orchestra.run.smithi083.stdout: 3 2024-09-20T19:04:39.521 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.521 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.521 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:39.521 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "10.d", 2024-09-20T19:04:39.521 INFO:teuthology.orchestra.run.smithi083.stdout: "osds": [ 2024-09-20T19:04:39.521 INFO:teuthology.orchestra.run.smithi083.stdout: 6, 2024-09-20T19:04:39.521 INFO:teuthology.orchestra.run.smithi083.stdout: 0 2024-09-20T19:04:39.521 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:39.521 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.522 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T19:04:39.522 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_temp": [], 2024-09-20T19:04:39.522 INFO:teuthology.orchestra.run.smithi083.stdout: "blocklist": { 2024-09-20T19:04:39.522 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/3474253603": "2024-09-21T18:44:32.434180+0000", 2024-09-20T19:04:39.522 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:6800/3525856829": "2024-09-21T18:44:32.434180+0000", 2024-09-20T19:04:39.522 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/220746284": "2024-09-21T18:44:32.434180+0000", 2024-09-20T19:04:39.522 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/3875445393": "2024-09-21T18:44:56.985620+0000", 2024-09-20T19:04:39.522 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/3009092807": "2024-09-21T18:44:56.985620+0000", 2024-09-20T19:04:39.522 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/4267132131": "2024-09-21T18:44:56.985620+0000", 2024-09-20T19:04:39.522 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/3684382972": "2024-09-21T18:50:14.729526+0000", 2024-09-20T19:04:39.522 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/3132933178": "2024-09-21T18:50:14.729526+0000", 2024-09-20T19:04:39.523 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:6800/1007846146": "2024-09-21T18:50:14.729526+0000", 2024-09-20T19:04:39.523 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:6800/2985596191": "2024-09-21T18:44:56.985620+0000", 2024-09-20T19:04:39.523 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/2410768464": "2024-09-21T18:50:14.729526+0000", 2024-09-20T19:04:39.523 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/1868694097": "2024-09-21T18:50:14.729526+0000", 2024-09-20T19:04:39.523 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/3777331122": "2024-09-21T18:44:32.434180+0000", 2024-09-20T19:04:39.523 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/2161728215": "2024-09-21T18:50:14.729526+0000" 2024-09-20T19:04:39.523 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.523 INFO:teuthology.orchestra.run.smithi083.stdout: "range_blocklist": {}, 2024-09-20T19:04:39.523 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profiles": { 2024-09-20T19:04:39.523 INFO:teuthology.orchestra.run.smithi083.stdout: "default": { 2024-09-20T19:04:39.523 INFO:teuthology.orchestra.run.smithi083.stdout: "crush-failure-domain": "osd", 2024-09-20T19:04:39.523 INFO:teuthology.orchestra.run.smithi083.stdout: "k": "2", 2024-09-20T19:04:39.523 INFO:teuthology.orchestra.run.smithi083.stdout: "m": "1", 2024-09-20T19:04:39.524 INFO:teuthology.orchestra.run.smithi083.stdout: "plugin": "jerasure", 2024-09-20T19:04:39.524 INFO:teuthology.orchestra.run.smithi083.stdout: "technique": "reed_sol_van" 2024-09-20T19:04:39.524 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.524 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:39.524 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps_queue": [], 2024-09-20T19:04:39.524 INFO:teuthology.orchestra.run.smithi083.stdout: "new_removed_snaps": [], 2024-09-20T19:04:39.524 INFO:teuthology.orchestra.run.smithi083.stdout: "new_purged_snaps": [], 2024-09-20T19:04:39.524 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_node_flags": {}, 2024-09-20T19:04:39.524 INFO:teuthology.orchestra.run.smithi083.stdout: "device_class_flags": {}, 2024-09-20T19:04:39.524 INFO:teuthology.orchestra.run.smithi083.stdout: "stretch_mode": { 2024-09-20T19:04:39.524 INFO:teuthology.orchestra.run.smithi083.stdout: "stretch_mode_enabled": false, 2024-09-20T19:04:39.524 INFO:teuthology.orchestra.run.smithi083.stdout: "stretch_bucket_count": 0, 2024-09-20T19:04:39.524 INFO:teuthology.orchestra.run.smithi083.stdout: "degraded_stretch_mode": 0, 2024-09-20T19:04:39.524 INFO:teuthology.orchestra.run.smithi083.stdout: "recovering_stretch_mode": 0, 2024-09-20T19:04:39.524 INFO:teuthology.orchestra.run.smithi083.stdout: "stretch_mode_bucket": 0 2024-09-20T19:04:39.525 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:39.525 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T19:04:39.525 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:39.526 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:39 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/185523849' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:04:39.526 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:39 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/185523849' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:04:39.526 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:04:39.527 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:04:39.528 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:04:39.528 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:04:39.558 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:39.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:39.559 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:39.561 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:04:39.562 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:04:39.622 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:04:39.622 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:04:39.660 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:39.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:39.693 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:39.693 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:39.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:39 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/185523849' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:04:39.794 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:04:39.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:39.828 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:39.828 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:04:39.929 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:39.960 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:39.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:39.961 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:40.062 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:40.095 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:40.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:40.096 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:40.148 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T19:04:40.197 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:40.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:40.234 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:40.234 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:40.334 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:40.375 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:40.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:40.376 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:40.477 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:40.514 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:40.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:40.515 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:40.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:40 smithi083 ceph-mon[29421]: pgmap v739: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 143 active+clean; 4.8 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 941 B/s rd, 0 B/s wr, 933 op/s; 12760/235473 objects misplaced (5.419%); 6.4 MiB/s, 103 objects/s recovering 2024-09-20T19:04:40.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:40 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/4019635092' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-20T19:04:40.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:40 smithi083 ceph-mon[29421]: 3.8 scrub starts 2024-09-20T19:04:40.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:40 smithi083 ceph-mon[29421]: 3.8 scrub ok 2024-09-20T19:04:40.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:40 smithi083 ceph-mon[29421]: 5.9 scrub starts 2024-09-20T19:04:40.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:40 smithi083 ceph-mon[29421]: 5.9 scrub ok 2024-09-20T19:04:40.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:40 smithi083 ceph-mon[36260]: pgmap v739: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 143 active+clean; 4.8 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 941 B/s rd, 0 B/s wr, 933 op/s; 12760/235473 objects misplaced (5.419%); 6.4 MiB/s, 103 objects/s recovering 2024-09-20T19:04:40.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:40 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/4019635092' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-20T19:04:40.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:40 smithi083 ceph-mon[36260]: 3.8 scrub starts 2024-09-20T19:04:40.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:40 smithi083 ceph-mon[36260]: 3.8 scrub ok 2024-09-20T19:04:40.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:40 smithi083 ceph-mon[36260]: 5.9 scrub starts 2024-09-20T19:04:40.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:40 smithi083 ceph-mon[36260]: 5.9 scrub ok 2024-09-20T19:04:40.616 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:40.643 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:40.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:40.644 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:40.745 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:40.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:40 smithi142 ceph-mon[32508]: pgmap v739: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 143 active+clean; 4.8 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 941 B/s rd, 0 B/s wr, 933 op/s; 12760/235473 objects misplaced (5.419%); 6.4 MiB/s, 103 objects/s recovering 2024-09-20T19:04:40.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:40 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/4019635092' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-20T19:04:40.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:40 smithi142 ceph-mon[32508]: 3.8 scrub starts 2024-09-20T19:04:40.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:40 smithi142 ceph-mon[32508]: 3.8 scrub ok 2024-09-20T19:04:40.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:40 smithi142 ceph-mon[32508]: 5.9 scrub starts 2024-09-20T19:04:40.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:40 smithi142 ceph-mon[32508]: 5.9 scrub ok 2024-09-20T19:04:40.780 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:40.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:40.781 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:40.825 INFO:tasks.thrashosds.thrasher:j is {'epoch': 378, 'fsid': '1ed5a50a-7780-11ef-baf5-efdc52797490', 'created': '2024-09-20T18:44:10.399800+0000', 'modified': '2024-09-20T19:04:35.289192+0000', 'last_up_change': '2024-09-20T19:01:21.809732+0000', 'last_in_change': '2024-09-20T19:04:20.299488+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 19, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 10, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-20T18:47:15.285795+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '361', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'datapool', 'create_time': '2024-09-20T18:49:43.452984+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 61, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.6500000953674316, 'score_stable': 2.6500000953674316, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 2.3299999237060547, 'raw_score_stable': 2.3299999237060547, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': '.rgw.root', 'create_time': '2024-09-20T18:49:44.217011+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-20T18:49:46.007275+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-20T18:49:47.961258+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-20T18:49:50.011276+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'unique_pool_3', 'create_time': '2024-09-20T19:02:10.165662+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '367', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3, 'score_stable': 3, 'optimal_score': 1, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': 'c714e1ec-f0c3-4054-9792-61b84e53eaa8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 375, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6801', 'nonce': 2574084049}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6802', 'nonce': 2574084049}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6804', 'nonce': 2574084049}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6803', 'nonce': 2574084049}]}, 'public_addr': '172.21.15.83:6801/2574084049', 'cluster_addr': '172.21.15.83:6802/2574084049', 'heartbeat_back_addr': '172.21.15.83:6804/2574084049', 'heartbeat_front_addr': '172.21.15.83:6803/2574084049', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '9dc0f2a8-bafa-4a42-88f5-0301dd58f78d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 375, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6805', 'nonce': 4213285798}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6806', 'nonce': 4213285798}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6808', 'nonce': 4213285798}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6807', 'nonce': 4213285798}]}, 'public_addr': '172.21.15.83:6805/4213285798', 'cluster_addr': '172.21.15.83:6806/4213285798', 'heartbeat_back_addr': '172.21.15.83:6808/4213285798', 'heartbeat_front_addr': '172.21.15.83:6807/4213285798', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '448ab692-ec41-44dc-b2d5-dbefd255b3fd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 374, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6809', 'nonce': 805947571}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6810', 'nonce': 805947571}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6812', 'nonce': 805947571}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6811', 'nonce': 805947571}]}, 'public_addr': '172.21.15.83:6809/805947571', 'cluster_addr': '172.21.15.83:6810/805947571', 'heartbeat_back_addr': '172.21.15.83:6812/805947571', 'heartbeat_front_addr': '172.21.15.83:6811/805947571', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '7cda5e50-e22e-4d4a-823b-53c8f5061193', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 25, 'up_thru': 374, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6813', 'nonce': 987237318}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6814', 'nonce': 987237318}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6816', 'nonce': 987237318}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6815', 'nonce': 987237318}]}, 'public_addr': '172.21.15.83:6813/987237318', 'cluster_addr': '172.21.15.83:6814/987237318', 'heartbeat_back_addr': '172.21.15.83:6816/987237318', 'heartbeat_front_addr': '172.21.15.83:6815/987237318', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '3d62654d-d31d-4310-9934-49e694dd17d9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 31, 'last_clean_end': 291, 'up_from': 304, 'up_thru': 375, 'down_at': 292, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6800', 'nonce': 3322919402}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6801', 'nonce': 3322919402}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6803', 'nonce': 3322919402}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6802', 'nonce': 3322919402}]}, 'public_addr': '172.21.15.142:6800/3322919402', 'cluster_addr': '172.21.15.142:6801/3322919402', 'heartbeat_back_addr': '172.21.15.142:6803/3322919402', 'heartbeat_front_addr': '172.21.15.142:6802/3322919402', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'b3e4e21b-639f-4d25-8bbf-ea8922a9222a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 38, 'up_thru': 374, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6804', 'nonce': 2500466067}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6805', 'nonce': 2500466067}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6807', 'nonce': 2500466067}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6806', 'nonce': 2500466067}]}, 'public_addr': '172.21.15.142:6804/2500466067', 'cluster_addr': '172.21.15.142:6805/2500466067', 'heartbeat_back_addr': '172.21.15.142:6807/2500466067', 'heartbeat_front_addr': '172.21.15.142:6806/2500466067', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'c5db7107-ccb4-4ae2-9e63-f444897942e3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 45, 'up_thru': 374, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6808', 'nonce': 1688186226}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6809', 'nonce': 1688186226}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6811', 'nonce': 1688186226}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6810', 'nonce': 1688186226}]}, 'public_addr': '172.21.15.142:6808/1688186226', 'cluster_addr': '172.21.15.142:6809/1688186226', 'heartbeat_back_addr': '172.21.15.142:6811/1688186226', 'heartbeat_front_addr': '172.21.15.142:6810/1688186226', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '07d16806-e2c8-4acb-b284-d75701e61622', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 53, 'up_thru': 375, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6812', 'nonce': 3679770984}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6813', 'nonce': 3679770984}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6815', 'nonce': 3679770984}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6814', 'nonce': 3679770984}]}, 'public_addr': '172.21.15.142:6812/3679770984', 'cluster_addr': '172.21.15.142:6813/3679770984', 'heartbeat_back_addr': '172.21.15.142:6815/3679770984', 'heartbeat_front_addr': '172.21.15.142:6814/3679770984', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T19:04:09.758659+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T19:03:47.301035+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T19:03:40.518081+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T19:04:18.454375+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T19:04:04.763014+0000', 'dead_epoch': 291}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T19:03:40.295266+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T19:04:03.749517+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T19:04:12.175096+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.c', 'osds': [2, 7, 6]}], 'pg_upmap_items': [{'pgid': '4.4', 'mappings': [{'from': 3, 'to': 5}]}, {'pgid': '4.f', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.14', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.18', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.1e', 'mappings': [{'from': 3, 'to': 2}]}], 'pg_upmap_primaries': [], 'pg_temp': [{'pgid': '10.0', 'osds': [6, 1]}, {'pgid': '10.1', 'osds': [0, 7, 1]}, {'pgid': '10.6', 'osds': [5, 0]}, {'pgid': '10.c', 'osds': [7, 0, 3]}, {'pgid': '10.d', 'osds': [6, 0]}], 'primary_temp': [], 'blocklist': {'172.21.15.83:0/3474253603': '2024-09-21T18:44:32.434180+0000', '172.21.15.83:6800/3525856829': '2024-09-21T18:44:32.434180+0000', '172.21.15.83:0/220746284': '2024-09-21T18:44:32.434180+0000', '172.21.15.83:0/3875445393': '2024-09-21T18:44:56.985620+0000', '172.21.15.83:0/3009092807': '2024-09-21T18:44:56.985620+0000', '172.21.15.83:0/4267132131': '2024-09-21T18:44:56.985620+0000', '172.21.15.83:0/3684382972': '2024-09-21T18:50:14.729526+0000', '172.21.15.83:0/3132933178': '2024-09-21T18:50:14.729526+0000', '172.21.15.83:6800/1007846146': '2024-09-21T18:50:14.729526+0000', '172.21.15.83:6800/2985596191': '2024-09-21T18:44:56.985620+0000', '172.21.15.83:0/2410768464': '2024-09-21T18:50:14.729526+0000', '172.21.15.83:0/1868694097': '2024-09-21T18:50:14.729526+0000', '172.21.15.83:0/3777331122': '2024-09-21T18:44:32.434180+0000', '172.21.15.83:0/2161728215': '2024-09-21T18:50:14.729526+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-20T19:04:40.825 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T19:04:40.882 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:40.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:40.937 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:40.937 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:40.983 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:04:40 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:04:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:04:41.039 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:41.061 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T19:04:41.061 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-20T19:04:41.061 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-20T19:04:41.061 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-20T19:04:41.061 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.062 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:218173 8.f 8:fc626e9a:::benchmark_data_smithi083_97422_object102082:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-20T19:04:41.062 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:58:01.879894+0000", 2024-09-20T19:04:41.062 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 399.172338911, 2024-09-20T19:04:41.062 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.22879112400000001, 2024-09-20T19:04:41.062 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:04:41.062 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:04:41.062 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:41.062 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:04:41.062 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T19:04:41.062 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T19:04:41.062 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 218173 2024-09-20T19:04:41.062 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.062 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:04:41.063 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.063 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:04:41.063 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.879894+0000", 2024-09-20T19:04:41.063 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:41.063 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.063 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.063 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:04:41.063 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.879894+0000", 2024-09-20T19:04:41.063 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:41.063 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.063 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.063 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:04:41.063 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.879895+0000", 2024-09-20T19:04:41.063 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3600000000000003e-07 2024-09-20T19:04:41.064 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.064 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.064 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:04:41.064 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.879899+0000", 2024-09-20T19:04:41.064 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3500000000000001e-06 2024-09-20T19:04:41.064 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.064 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.064 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:04:41.064 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.879907+0000", 2024-09-20T19:04:41.064 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.943e-06 2024-09-20T19:04:41.064 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.064 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.064 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:04:41.064 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.879913+0000", 2024-09-20T19:04:41.064 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3910000000000003e-06 2024-09-20T19:04:41.065 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.065 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.065 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:41.065 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.095128+0000", 2024-09-20T19:04:41.065 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.21521499199999999 2024-09-20T19:04:41.065 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.065 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.065 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:04:41.065 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.095243+0000", 2024-09-20T19:04:41.065 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011465599999999999 2024-09-20T19:04:41.065 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.065 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.065 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:04:41.065 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.095357+0000", 2024-09-20T19:04:41.066 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000114868 2024-09-20T19:04:41.066 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.066 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.066 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:41.066 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.104639+0000", 2024-09-20T19:04:41.066 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0092811200000000003 2024-09-20T19:04:41.066 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.066 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.066 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:04:41.066 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.105329+0000", 2024-09-20T19:04:41.066 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000690195 2024-09-20T19:04:41.066 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.066 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.066 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:41.067 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.108593+0000", 2024-09-20T19:04:41.067 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003263967 2024-09-20T19:04:41.067 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.067 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.067 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:04:41.067 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.108646+0000", 2024-09-20T19:04:41.067 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3495999999999997e-05 2024-09-20T19:04:41.067 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.067 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.067 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:04:41.067 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.108686+0000", 2024-09-20T19:04:41.067 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9409999999999997e-05 2024-09-20T19:04:41.067 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:41.067 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:41.067 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:41.068 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.068 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.068 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:50423 9.1 9:8787b2bd:::benchmark_data_smithi083_124552_object50422:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-20T19:04:41.068 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:59:04.093296+0000", 2024-09-20T19:04:41.068 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 336.95893765199997, 2024-09-20T19:04:41.068 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.136410235, 2024-09-20T19:04:41.068 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:04:41.068 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:04:41.068 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:41.068 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:04:41.068 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:04:41.068 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:04:41.068 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 50423 2024-09-20T19:04:41.068 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.069 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:04:41.069 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.069 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:04:41.069 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.093296+0000", 2024-09-20T19:04:41.069 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:41.069 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.069 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.069 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:04:41.069 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.093296+0000", 2024-09-20T19:04:41.069 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:41.069 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.069 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.069 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:04:41.069 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.093297+0000", 2024-09-20T19:04:41.070 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.37e-06 2024-09-20T19:04:41.070 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.070 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.070 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:04:41.070 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.093316+0000", 2024-09-20T19:04:41.070 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8660000000000001e-05 2024-09-20T19:04:41.070 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.070 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.070 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:04:41.070 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.093342+0000", 2024-09-20T19:04:41.070 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6103999999999999e-05 2024-09-20T19:04:41.070 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.070 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.070 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:04:41.070 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.093347+0000", 2024-09-20T19:04:41.071 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0490000000000003e-06 2024-09-20T19:04:41.071 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.071 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.071 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:41.071 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.093482+0000", 2024-09-20T19:04:41.071 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000135449 2024-09-20T19:04:41.071 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.071 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.071 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:04:41.071 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.093490+0000", 2024-09-20T19:04:41.071 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0469999999999994e-06 2024-09-20T19:04:41.071 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.071 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.071 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:41.072 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.067142+0000", 2024-09-20T19:04:41.072 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97365191799999995 2024-09-20T19:04:41.072 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.072 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.072 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:04:41.072 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.067153+0000", 2024-09-20T19:04:41.072 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0559e-05 2024-09-20T19:04:41.072 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.072 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.072 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:41.072 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.215644+0000", 2024-09-20T19:04:41.072 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.148490633 2024-09-20T19:04:41.072 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.072 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.072 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:04:41.073 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.215772+0000", 2024-09-20T19:04:41.073 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012840799999999999 2024-09-20T19:04:41.073 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.073 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.073 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 6,7", 2024-09-20T19:04:41.073 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.215927+0000", 2024-09-20T19:04:41.073 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000155315 2024-09-20T19:04:41.073 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.073 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.073 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:41.073 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.223137+0000", 2024-09-20T19:04:41.073 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0072102049999999999 2024-09-20T19:04:41.073 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.074 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.074 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:41.074 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.223182+0000", 2024-09-20T19:04:41.074 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4158e-05 2024-09-20T19:04:41.074 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.074 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.074 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:04:41.074 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.227203+0000", 2024-09-20T19:04:41.074 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0040211530000000004 2024-09-20T19:04:41.074 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.074 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.074 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:04:41.074 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.227258+0000", 2024-09-20T19:04:41.074 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5050999999999999e-05 2024-09-20T19:04:41.075 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.075 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.075 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:04:41.075 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.229706+0000", 2024-09-20T19:04:41.075 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0024481559999999999 2024-09-20T19:04:41.075 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:41.075 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:41.075 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:41.075 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.075 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.075 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:50474 9.1 9:8f4be195:::benchmark_data_smithi083_124552_object50473:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-20T19:04:41.075 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:59:04.131531+0000", 2024-09-20T19:04:41.075 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 336.92070273500002, 2024-09-20T19:04:41.075 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.101374029, 2024-09-20T19:04:41.076 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:04:41.076 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:04:41.076 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:41.076 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:04:41.076 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:04:41.076 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:04:41.076 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 50474 2024-09-20T19:04:41.076 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.076 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:04:41.076 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.076 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:04:41.076 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.131531+0000", 2024-09-20T19:04:41.076 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:41.077 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.077 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.077 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:04:41.077 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.131531+0000", 2024-09-20T19:04:41.077 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:41.077 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.077 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.077 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:04:41.077 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.131532+0000", 2024-09-20T19:04:41.077 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5489999999999999e-06 2024-09-20T19:04:41.077 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.077 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.077 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:04:41.077 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.131552+0000", 2024-09-20T19:04:41.078 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9630999999999999e-05 2024-09-20T19:04:41.078 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.078 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.078 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:04:41.078 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.131568+0000", 2024-09-20T19:04:41.078 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6606999999999999e-05 2024-09-20T19:04:41.078 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.078 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.078 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:04:41.078 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.131577+0000", 2024-09-20T19:04:41.078 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7879999999999995e-06 2024-09-20T19:04:41.078 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.078 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.078 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:41.079 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.131644+0000", 2024-09-20T19:04:41.079 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7180000000000004e-05 2024-09-20T19:04:41.079 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.079 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.079 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:04:41.079 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.131654+0000", 2024-09-20T19:04:41.079 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2390000000000008e-06 2024-09-20T19:04:41.079 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.079 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.079 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:41.079 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.067311+0000", 2024-09-20T19:04:41.079 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.935657501 2024-09-20T19:04:41.079 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.079 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.080 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:04:41.080 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.067320+0000", 2024-09-20T19:04:41.080 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9779999999999994e-06 2024-09-20T19:04:41.080 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.080 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.080 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:41.080 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.220228+0000", 2024-09-20T19:04:41.080 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1529083090000001 2024-09-20T19:04:41.080 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.080 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.080 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:04:41.080 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.220362+0000", 2024-09-20T19:04:41.080 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013401 2024-09-20T19:04:41.080 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.080 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.081 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 6,7", 2024-09-20T19:04:41.081 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.220523+0000", 2024-09-20T19:04:41.081 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000160247 2024-09-20T19:04:41.081 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.081 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.081 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:04:41.081 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.229718+0000", 2024-09-20T19:04:41.081 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0091949860000000005 2024-09-20T19:04:41.081 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.081 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.081 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:41.081 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.229797+0000", 2024-09-20T19:04:41.081 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9443999999999994e-05 2024-09-20T19:04:41.081 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.082 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.082 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:41.082 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.230036+0000", 2024-09-20T19:04:41.082 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000239072 2024-09-20T19:04:41.082 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.082 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.082 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:04:41.082 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.230111+0000", 2024-09-20T19:04:41.082 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4411000000000003e-05 2024-09-20T19:04:41.082 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.082 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.082 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:04:41.082 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.232905+0000", 2024-09-20T19:04:41.082 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0027940769999999998 2024-09-20T19:04:41.083 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:41.083 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:41.083 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:41.083 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.083 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.083 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:50506 9.1 9:80ae6680:::benchmark_data_smithi083_124552_object50505:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-20T19:04:41.083 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:59:04.155840+0000", 2024-09-20T19:04:41.083 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 336.89639293200003, 2024-09-20T19:04:41.083 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.082518146, 2024-09-20T19:04:41.083 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:04:41.083 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:04:41.083 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:41.083 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:04:41.083 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:04:41.083 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:04:41.084 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 50506 2024-09-20T19:04:41.084 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.084 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:04:41.084 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.084 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:04:41.084 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.155840+0000", 2024-09-20T19:04:41.084 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:41.084 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.084 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.084 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:04:41.084 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.155840+0000", 2024-09-20T19:04:41.084 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:41.084 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.084 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.084 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:04:41.085 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.155842+0000", 2024-09-20T19:04:41.085 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1090000000000001e-06 2024-09-20T19:04:41.085 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.085 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.085 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:04:41.085 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.155869+0000", 2024-09-20T19:04:41.085 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7801999999999999e-05 2024-09-20T19:04:41.085 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.085 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.085 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:04:41.085 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.155880+0000", 2024-09-20T19:04:41.085 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1059e-05 2024-09-20T19:04:41.085 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.085 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.085 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:04:41.086 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.155888+0000", 2024-09-20T19:04:41.086 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7559999999999999e-06 2024-09-20T19:04:41.086 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.086 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.086 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:41.086 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.155955+0000", 2024-09-20T19:04:41.086 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7022999999999998e-05 2024-09-20T19:04:41.086 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.086 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.086 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:04:41.086 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.155965+0000", 2024-09-20T19:04:41.086 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3710000000000006e-06 2024-09-20T19:04:41.086 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.086 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.087 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:41.087 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.067356+0000", 2024-09-20T19:04:41.087 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.91139177299999996 2024-09-20T19:04:41.087 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.087 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.087 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:04:41.087 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.067364+0000", 2024-09-20T19:04:41.087 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.6000000000000001e-06 2024-09-20T19:04:41.087 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.087 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.087 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:41.087 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.223316+0000", 2024-09-20T19:04:41.087 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1559515810000001 2024-09-20T19:04:41.087 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.088 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.088 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:04:41.088 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.223488+0000", 2024-09-20T19:04:41.088 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017242399999999999 2024-09-20T19:04:41.089 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.089 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.089 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 6,7", 2024-09-20T19:04:41.089 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.223628+0000", 2024-09-20T19:04:41.089 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013994100000000001 2024-09-20T19:04:41.089 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.089 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.089 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:41.089 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.229891+0000", 2024-09-20T19:04:41.089 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0062633239999999998 2024-09-20T19:04:41.089 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.090 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.090 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:04:41.090 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.229911+0000", 2024-09-20T19:04:41.090 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9910000000000001e-05 2024-09-20T19:04:41.090 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.090 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.090 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:41.090 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.235269+0000", 2024-09-20T19:04:41.090 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0053577429999999999 2024-09-20T19:04:41.090 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.090 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.090 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:04:41.090 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.235358+0000", 2024-09-20T19:04:41.090 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9536999999999993e-05 2024-09-20T19:04:41.091 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.091 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.091 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:04:41.091 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.238359+0000", 2024-09-20T19:04:41.091 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003000193 2024-09-20T19:04:41.091 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:41.091 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:41.091 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:41.091 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.091 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.091 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:50517 9.1 9:8bbc6b9c:::benchmark_data_smithi083_124552_object50516:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-20T19:04:41.091 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:59:04.167833+0000", 2024-09-20T19:04:41.091 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 336.88440049500002, 2024-09-20T19:04:41.092 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0735188230000001, 2024-09-20T19:04:41.092 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:04:41.092 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:04:41.092 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:41.092 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:04:41.092 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:04:41.092 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:04:41.092 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 50517 2024-09-20T19:04:41.092 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.092 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:04:41.092 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.092 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:04:41.092 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.167833+0000", 2024-09-20T19:04:41.092 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:41.093 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.093 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.093 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:04:41.093 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.167833+0000", 2024-09-20T19:04:41.093 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:41.093 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.093 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.093 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:04:41.093 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.167835+0000", 2024-09-20T19:04:41.093 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.821e-06 2024-09-20T19:04:41.093 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.093 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.093 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:04:41.093 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.167864+0000", 2024-09-20T19:04:41.094 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9714e-05 2024-09-20T19:04:41.094 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.094 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.094 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:04:41.094 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.167883+0000", 2024-09-20T19:04:41.094 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8788999999999999e-05 2024-09-20T19:04:41.094 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.094 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.094 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:04:41.094 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.167891+0000", 2024-09-20T19:04:41.094 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.1669999999999999e-06 2024-09-20T19:04:41.094 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.094 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.095 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:41.095 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.167959+0000", 2024-09-20T19:04:41.095 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7779000000000003e-05 2024-09-20T19:04:41.095 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.095 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.095 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:04:41.095 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.167968+0000", 2024-09-20T19:04:41.095 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9630000000000004e-06 2024-09-20T19:04:41.095 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.095 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.095 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:41.095 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.068252+0000", 2024-09-20T19:04:41.095 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.90028381000000002 2024-09-20T19:04:41.095 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.096 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.096 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:04:41.096 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.068262+0000", 2024-09-20T19:04:41.096 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0278000000000001e-05 2024-09-20T19:04:41.096 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.096 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.096 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:41.096 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.225517+0000", 2024-09-20T19:04:41.096 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.157254279 2024-09-20T19:04:41.096 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.096 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.096 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:04:41.096 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.225656+0000", 2024-09-20T19:04:41.096 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013903900000000001 2024-09-20T19:04:41.096 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.097 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.097 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 6,7", 2024-09-20T19:04:41.097 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.225821+0000", 2024-09-20T19:04:41.097 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016562099999999999 2024-09-20T19:04:41.097 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.097 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.097 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:04:41.097 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.229921+0000", 2024-09-20T19:04:41.097 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.004099742 2024-09-20T19:04:41.097 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.097 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.097 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:41.097 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.229977+0000", 2024-09-20T19:04:41.097 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6523000000000001e-05 2024-09-20T19:04:41.097 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.098 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.098 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:41.098 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.238409+0000", 2024-09-20T19:04:41.098 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0084314219999999992 2024-09-20T19:04:41.098 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.098 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.098 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:04:41.098 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.238463+0000", 2024-09-20T19:04:41.098 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4308000000000002e-05 2024-09-20T19:04:41.098 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.098 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.098 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:04:41.098 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.241352+0000", 2024-09-20T19:04:41.099 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0028885680000000002 2024-09-20T19:04:41.099 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:41.099 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:41.099 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:41.099 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.099 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.099 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:118977 9.8 9:18fffd38:::benchmark_data_smithi083_124552_object4830:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e286)", 2024-09-20T19:04:41.099 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:16.045469+0000", 2024-09-20T19:04:41.099 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 265.00676486200001, 2024-09-20T19:04:41.099 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.233874096, 2024-09-20T19:04:41.099 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:04:41.099 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:04:41.099 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:41.099 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:04:41.100 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:04:41.100 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:04:41.100 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 118977 2024-09-20T19:04:41.100 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.100 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:04:41.100 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.100 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:04:41.100 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.045469+0000", 2024-09-20T19:04:41.100 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:41.100 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.100 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.100 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:04:41.100 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.045469+0000", 2024-09-20T19:04:41.100 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:41.100 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.101 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.101 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:04:41.101 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.045470+0000", 2024-09-20T19:04:41.101 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.359e-06 2024-09-20T19:04:41.101 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.101 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.101 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:04:41.101 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.045474+0000", 2024-09-20T19:04:41.101 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0609999999999997e-06 2024-09-20T19:04:41.101 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.101 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.101 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:04:41.101 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.045484+0000", 2024-09-20T19:04:41.101 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0332999999999999e-05 2024-09-20T19:04:41.102 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.102 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.102 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:04:41.102 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.045489+0000", 2024-09-20T19:04:41.102 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1869999999999996e-06 2024-09-20T19:04:41.102 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.102 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.102 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:41.102 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.045547+0000", 2024-09-20T19:04:41.102 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7046999999999998e-05 2024-09-20T19:04:41.102 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.102 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.102 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:04:41.102 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.045681+0000", 2024-09-20T19:04:41.103 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013437200000000001 2024-09-20T19:04:41.103 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.103 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.103 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T19:04:41.103 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.045767+0000", 2024-09-20T19:04:41.103 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6248000000000005e-05 2024-09-20T19:04:41.103 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.103 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.103 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:04:41.103 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.047811+0000", 2024-09-20T19:04:41.103 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0020440440000000001 2024-09-20T19:04:41.103 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.103 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.103 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:41.104 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.048095+0000", 2024-09-20T19:04:41.104 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00028369700000000002 2024-09-20T19:04:41.104 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.104 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.104 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:41.104 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.279270+0000", 2024-09-20T19:04:41.104 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.23117499899999999 2024-09-20T19:04:41.104 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.104 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.104 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:04:41.104 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.279311+0000", 2024-09-20T19:04:41.104 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0840000000000002e-05 2024-09-20T19:04:41.104 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.104 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.105 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:04:41.105 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.279343+0000", 2024-09-20T19:04:41.105 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1909000000000002e-05 2024-09-20T19:04:41.105 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:41.105 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:41.105 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:41.105 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.105 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.105 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:118986 9.8 9:12412698:::benchmark_data_smithi083_124552_object4839:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e286)", 2024-09-20T19:04:41.105 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:16.058237+0000", 2024-09-20T19:04:41.105 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 264.99399666199997, 2024-09-20T19:04:41.105 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.221293453, 2024-09-20T19:04:41.105 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:04:41.105 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:04:41.106 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:41.106 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:04:41.106 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:04:41.106 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:04:41.106 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 118986 2024-09-20T19:04:41.106 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.106 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:04:41.106 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.106 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:04:41.106 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.058237+0000", 2024-09-20T19:04:41.106 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:41.106 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.106 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.107 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:04:41.107 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.058237+0000", 2024-09-20T19:04:41.107 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:41.107 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.107 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.107 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:04:41.107 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.058238+0000", 2024-09-20T19:04:41.107 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1799999999999999e-06 2024-09-20T19:04:41.107 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.107 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.107 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:04:41.107 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.058242+0000", 2024-09-20T19:04:41.107 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3880000000000002e-06 2024-09-20T19:04:41.108 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.108 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.108 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:04:41.108 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.058251+0000", 2024-09-20T19:04:41.108 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0420000000000008e-06 2024-09-20T19:04:41.108 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.108 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.108 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:04:41.108 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.058256+0000", 2024-09-20T19:04:41.108 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8960000000000002e-06 2024-09-20T19:04:41.108 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.108 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.108 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:41.108 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.058309+0000", 2024-09-20T19:04:41.108 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2902999999999998e-05 2024-09-20T19:04:41.109 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.109 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.109 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:04:41.109 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.058410+0000", 2024-09-20T19:04:41.109 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010079200000000001 2024-09-20T19:04:41.109 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.109 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.109 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T19:04:41.109 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.058476+0000", 2024-09-20T19:04:41.109 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6050000000000006e-05 2024-09-20T19:04:41.109 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.109 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.109 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:04:41.109 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.062309+0000", 2024-09-20T19:04:41.110 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003832701 2024-09-20T19:04:41.110 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.110 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.110 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:41.110 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.062336+0000", 2024-09-20T19:04:41.110 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7426000000000001e-05 2024-09-20T19:04:41.110 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.110 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.110 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:41.110 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.279409+0000", 2024-09-20T19:04:41.110 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.21707326399999999 2024-09-20T19:04:41.110 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.110 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.110 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:04:41.111 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.279490+0000", 2024-09-20T19:04:41.111 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0500999999999994e-05 2024-09-20T19:04:41.111 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.111 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.111 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:04:41.111 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.279530+0000", 2024-09-20T19:04:41.111 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0309999999999999e-05 2024-09-20T19:04:41.111 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:41.111 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:41.111 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:41.111 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.111 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.111 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:124578 9.2 9:422abb8a:::benchmark_data_smithi083_124552_object10431:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e289)", 2024-09-20T19:04:41.111 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:20.946829+0000", 2024-09-20T19:04:41.112 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 260.10540451600002, 2024-09-20T19:04:41.112 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.33414796600000002, 2024-09-20T19:04:41.112 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:04:41.112 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:04:41.112 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:41.112 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:04:41.112 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:04:41.112 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:04:41.112 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 124578 2024-09-20T19:04:41.112 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.112 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:04:41.112 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.112 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:04:41.112 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.946829+0000", 2024-09-20T19:04:41.112 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:41.113 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.113 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.113 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:04:41.113 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.946829+0000", 2024-09-20T19:04:41.113 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:41.113 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.113 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.113 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:04:41.113 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.946830+0000", 2024-09-20T19:04:41.113 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2299999999999999e-07 2024-09-20T19:04:41.113 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.113 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.113 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:04:41.113 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.946834+0000", 2024-09-20T19:04:41.114 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1189999999999999e-06 2024-09-20T19:04:41.114 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.114 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.114 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:04:41.114 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.946854+0000", 2024-09-20T19:04:41.114 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9754000000000001e-05 2024-09-20T19:04:41.114 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.114 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.114 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:04:41.114 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.946861+0000", 2024-09-20T19:04:41.114 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7719999999999994e-06 2024-09-20T19:04:41.114 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.114 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.114 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:41.114 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.951540+0000", 2024-09-20T19:04:41.115 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.004678354 2024-09-20T19:04:41.115 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.115 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.115 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:04:41.115 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.951640+0000", 2024-09-20T19:04:41.115 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000100055 2024-09-20T19:04:41.115 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.115 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.115 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:04:41.115 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.951741+0000", 2024-09-20T19:04:41.115 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010086600000000001 2024-09-20T19:04:41.115 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.115 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.115 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:04:41.116 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.954721+0000", 2024-09-20T19:04:41.116 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0029804609999999998 2024-09-20T19:04:41.116 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.116 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.116 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:41.116 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.956254+0000", 2024-09-20T19:04:41.116 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001532683 2024-09-20T19:04:41.116 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.116 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.116 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:41.116 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.280892+0000", 2024-09-20T19:04:41.116 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.32463842100000001 2024-09-20T19:04:41.116 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.116 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.117 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:04:41.117 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.280935+0000", 2024-09-20T19:04:41.117 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2592999999999997e-05 2024-09-20T19:04:41.117 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.117 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.117 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:04:41.117 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.280977+0000", 2024-09-20T19:04:41.117 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1965000000000002e-05 2024-09-20T19:04:41.117 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:41.117 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:41.117 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:41.117 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.117 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.117 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:124596 9.2 9:42d01fa2:::benchmark_data_smithi083_124552_object10449:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e289)", 2024-09-20T19:04:41.117 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:20.962657+0000", 2024-09-20T19:04:41.118 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 260.089576368, 2024-09-20T19:04:41.118 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.32748537700000002, 2024-09-20T19:04:41.118 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:04:41.118 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:04:41.118 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:41.118 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:04:41.118 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:04:41.118 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:04:41.118 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 124596 2024-09-20T19:04:41.118 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.118 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:04:41.118 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.118 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:04:41.118 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.962657+0000", 2024-09-20T19:04:41.119 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:41.119 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.119 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.119 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:04:41.119 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.962657+0000", 2024-09-20T19:04:41.119 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:41.119 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.119 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.119 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:04:41.119 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.962658+0000", 2024-09-20T19:04:41.119 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4020000000000001e-06 2024-09-20T19:04:41.119 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.119 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.119 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:04:41.120 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.962663+0000", 2024-09-20T19:04:41.120 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0590000000000002e-06 2024-09-20T19:04:41.120 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.120 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.120 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:04:41.120 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.962678+0000", 2024-09-20T19:04:41.120 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4469e-05 2024-09-20T19:04:41.120 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.120 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.120 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:04:41.120 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.962685+0000", 2024-09-20T19:04:41.120 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3549999999999999e-06 2024-09-20T19:04:41.120 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.120 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.121 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:41.121 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.962732+0000", 2024-09-20T19:04:41.121 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6502999999999998e-05 2024-09-20T19:04:41.121 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.121 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.121 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:04:41.121 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.962831+0000", 2024-09-20T19:04:41.121 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9489999999999998e-05 2024-09-20T19:04:41.121 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.121 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.121 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:04:41.121 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.962910+0000", 2024-09-20T19:04:41.121 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8527000000000002e-05 2024-09-20T19:04:41.121 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.122 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.122 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:04:41.122 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.973808+0000", 2024-09-20T19:04:41.122 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010898134 2024-09-20T19:04:41.122 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.122 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.122 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:41.122 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.973889+0000", 2024-09-20T19:04:41.122 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0641000000000003e-05 2024-09-20T19:04:41.122 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.122 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.122 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:41.122 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.290050+0000", 2024-09-20T19:04:41.122 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.31616166899999998 2024-09-20T19:04:41.123 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.123 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.123 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:04:41.123 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.290100+0000", 2024-09-20T19:04:41.123 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0179999999999997e-05 2024-09-20T19:04:41.123 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.123 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.123 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:04:41.123 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.290142+0000", 2024-09-20T19:04:41.123 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1947999999999999e-05 2024-09-20T19:04:41.123 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:41.123 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:41.123 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:41.123 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.124 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.124 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:124603 9.2 9:4d3bc5dd:::benchmark_data_smithi083_124552_object10456:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e289)", 2024-09-20T19:04:41.124 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:20.964654+0000", 2024-09-20T19:04:41.124 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 260.08757908699999, 2024-09-20T19:04:41.124 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.32571787499999999, 2024-09-20T19:04:41.124 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:04:41.124 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:04:41.124 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:41.124 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:04:41.124 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:04:41.124 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:04:41.124 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 124603 2024-09-20T19:04:41.124 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.124 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:04:41.125 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.125 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:04:41.125 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.964654+0000", 2024-09-20T19:04:41.125 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:41.125 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.125 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.125 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:04:41.125 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.964654+0000", 2024-09-20T19:04:41.125 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:41.125 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.125 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.125 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:04:41.125 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.964655+0000", 2024-09-20T19:04:41.126 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6099999999999999e-07 2024-09-20T19:04:41.126 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.126 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.126 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:04:41.126 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.964698+0000", 2024-09-20T19:04:41.126 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2778999999999998e-05 2024-09-20T19:04:41.126 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.126 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.126 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:04:41.126 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.964707+0000", 2024-09-20T19:04:41.126 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0869999999999996e-06 2024-09-20T19:04:41.126 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.127 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.127 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:04:41.127 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.964713+0000", 2024-09-20T19:04:41.127 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3339999999999998e-06 2024-09-20T19:04:41.127 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.127 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.127 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:41.127 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.965135+0000", 2024-09-20T19:04:41.127 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00042182599999999998 2024-09-20T19:04:41.127 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.127 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.127 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:04:41.127 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.965229+0000", 2024-09-20T19:04:41.127 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4297999999999997e-05 2024-09-20T19:04:41.128 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.128 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.128 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:04:41.128 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.965326+0000", 2024-09-20T19:04:41.128 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6902000000000002e-05 2024-09-20T19:04:41.128 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.128 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.128 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:04:41.128 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.973814+0000", 2024-09-20T19:04:41.128 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0084871779999999997 2024-09-20T19:04:41.128 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.128 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.128 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:41.128 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.973930+0000", 2024-09-20T19:04:41.129 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011664599999999999 2024-09-20T19:04:41.129 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.129 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.129 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:41.129 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.290288+0000", 2024-09-20T19:04:41.129 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.31635755799999998 2024-09-20T19:04:41.129 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.129 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.129 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:04:41.129 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.290337+0000", 2024-09-20T19:04:41.129 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8948e-05 2024-09-20T19:04:41.129 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:41.130 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:41.131 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:41.131 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.131 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:04:41.132 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.290372+0000", 2024-09-20T19:04:41.132 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5457999999999998e-05 2024-09-20T19:04:41.132 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:41.132 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:41.132 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:41.132 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.132 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.132 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:124604 9.2 9:40e573dc:::benchmark_data_smithi083_124552_object10457:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e289)", 2024-09-20T19:04:41.132 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:20.964729+0000", 2024-09-20T19:04:41.132 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 260.08750456199999, 2024-09-20T19:04:41.132 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.32575654300000001, 2024-09-20T19:04:41.132 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:04:41.132 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:04:41.132 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:41.133 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:04:41.133 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:04:41.133 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:04:41.133 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 124604 2024-09-20T19:04:41.133 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.133 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:04:41.133 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.133 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:04:41.133 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.964729+0000", 2024-09-20T19:04:41.133 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:41.133 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.133 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.133 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:04:41.133 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.964729+0000", 2024-09-20T19:04:41.134 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:41.134 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.134 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.134 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:04:41.134 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.964729+0000", 2024-09-20T19:04:41.134 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6599999999999996e-07 2024-09-20T19:04:41.134 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.134 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.134 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:04:41.134 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.964732+0000", 2024-09-20T19:04:41.134 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9679999999999998e-06 2024-09-20T19:04:41.134 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.134 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.134 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:04:41.135 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.964738+0000", 2024-09-20T19:04:41.135 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6910000000000002e-06 2024-09-20T19:04:41.135 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.135 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.135 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:04:41.135 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.964744+0000", 2024-09-20T19:04:41.135 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5899999999999998e-06 2024-09-20T19:04:41.135 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.135 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.135 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:41.135 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.967815+0000", 2024-09-20T19:04:41.135 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0030715209999999998 2024-09-20T19:04:41.135 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.135 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.135 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:04:41.136 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.967939+0000", 2024-09-20T19:04:41.136 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012392100000000001 2024-09-20T19:04:41.136 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.136 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.136 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:04:41.136 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.968041+0000", 2024-09-20T19:04:41.136 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000102166 2024-09-20T19:04:41.136 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.136 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.136 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:41.136 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.973986+0000", 2024-09-20T19:04:41.136 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059442610000000002 2024-09-20T19:04:41.136 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.136 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.137 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:04:41.137 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.973997+0000", 2024-09-20T19:04:41.137 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1313e-05 2024-09-20T19:04:41.137 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.137 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.137 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:41.137 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.290407+0000", 2024-09-20T19:04:41.137 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.31641063699999999 2024-09-20T19:04:41.137 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.137 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.137 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:04:41.137 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.290452+0000", 2024-09-20T19:04:41.137 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4753999999999999e-05 2024-09-20T19:04:41.137 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.138 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.138 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:04:41.138 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.290485+0000", 2024-09-20T19:04:41.138 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3154999999999997e-05 2024-09-20T19:04:41.138 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:41.138 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:41.138 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:41.138 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.138 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.138 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:124610 9.2 9:4e613fc5:::benchmark_data_smithi083_124552_object10463:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e289)", 2024-09-20T19:04:41.138 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:20.970344+0000", 2024-09-20T19:04:41.138 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 260.08188934200001, 2024-09-20T19:04:41.138 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.32025382099999999, 2024-09-20T19:04:41.138 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:04:41.139 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:04:41.139 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:41.139 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:04:41.139 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:04:41.139 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:04:41.139 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 124610 2024-09-20T19:04:41.139 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.139 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:04:41.139 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.139 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:04:41.140 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.970344+0000", 2024-09-20T19:04:41.140 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:41.140 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.140 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.140 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:04:41.140 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.970344+0000", 2024-09-20T19:04:41.140 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:41.140 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.140 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.140 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:04:41.140 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.970345+0000", 2024-09-20T19:04:41.140 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5000000000000003e-07 2024-09-20T19:04:41.140 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.140 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.140 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:04:41.141 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.970347+0000", 2024-09-20T19:04:41.141 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6769999999999999e-06 2024-09-20T19:04:41.141 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.141 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.141 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:04:41.141 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.970354+0000", 2024-09-20T19:04:41.141 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6100000000000002e-06 2024-09-20T19:04:41.141 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.141 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.141 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:04:41.141 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.970359+0000", 2024-09-20T19:04:41.141 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5990000000000001e-06 2024-09-20T19:04:41.141 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.141 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.142 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:41.142 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.971408+0000", 2024-09-20T19:04:41.142 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0010481220000000001 2024-09-20T19:04:41.142 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.142 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.142 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:04:41.142 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.971499+0000", 2024-09-20T19:04:41.142 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1843999999999997e-05 2024-09-20T19:04:41.142 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.142 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.142 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:04:41.142 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.971597+0000", 2024-09-20T19:04:41.142 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8011999999999996e-05 2024-09-20T19:04:41.142 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.142 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.143 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:04:41.143 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.974098+0000", 2024-09-20T19:04:41.143 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00250016 2024-09-20T19:04:41.143 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.143 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.143 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:41.143 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.978935+0000", 2024-09-20T19:04:41.143 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.004836974 2024-09-20T19:04:41.143 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.143 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.143 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:41.143 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.290520+0000", 2024-09-20T19:04:41.143 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.31158548699999999 2024-09-20T19:04:41.143 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.144 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.144 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:04:41.144 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.290566+0000", 2024-09-20T19:04:41.144 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5519000000000002e-05 2024-09-20T19:04:41.144 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.144 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.144 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:04:41.144 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.290598+0000", 2024-09-20T19:04:41.144 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2267000000000001e-05 2024-09-20T19:04:41.144 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:41.144 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:41.144 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:41.144 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.144 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.144 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:124618 9.2 9:4c60c683:::benchmark_data_smithi083_124552_object10471:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e289)", 2024-09-20T19:04:41.145 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:20.974705+0000", 2024-09-20T19:04:41.145 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 260.07752804299997, 2024-09-20T19:04:41.145 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.31613734999999998, 2024-09-20T19:04:41.145 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:04:41.145 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:04:41.145 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:41.145 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:04:41.145 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:04:41.145 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:04:41.145 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 124618 2024-09-20T19:04:41.145 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.145 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:04:41.145 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.145 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:04:41.146 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.974705+0000", 2024-09-20T19:04:41.146 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:41.146 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.146 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.146 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:04:41.146 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.974705+0000", 2024-09-20T19:04:41.146 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:41.146 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.146 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.146 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:04:41.146 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.974707+0000", 2024-09-20T19:04:41.146 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4109999999999999e-06 2024-09-20T19:04:41.146 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.147 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.147 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:04:41.147 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.974711+0000", 2024-09-20T19:04:41.147 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5229999999999997e-06 2024-09-20T19:04:41.147 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.147 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.147 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:04:41.147 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.974720+0000", 2024-09-20T19:04:41.147 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.3829999999999992e-06 2024-09-20T19:04:41.147 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.147 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.147 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:04:41.147 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.974726+0000", 2024-09-20T19:04:41.147 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7039999999999997e-06 2024-09-20T19:04:41.148 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.148 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.148 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:41.148 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.974777+0000", 2024-09-20T19:04:41.148 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0785999999999998e-05 2024-09-20T19:04:41.148 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.148 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.148 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:04:41.148 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.974886+0000", 2024-09-20T19:04:41.148 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010864199999999999 2024-09-20T19:04:41.148 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.148 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.148 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:04:41.148 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.974983+0000", 2024-09-20T19:04:41.148 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7293000000000005e-05 2024-09-20T19:04:41.149 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.149 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.149 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:04:41.149 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.977705+0000", 2024-09-20T19:04:41.149 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0027215339999999998 2024-09-20T19:04:41.149 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.149 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.149 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:41.149 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.979236+0000", 2024-09-20T19:04:41.149 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0015310079999999999 2024-09-20T19:04:41.149 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.149 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.149 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:41.149 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.290703+0000", 2024-09-20T19:04:41.149 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.311467775 2024-09-20T19:04:41.149 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.150 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.150 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:04:41.150 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.290802+0000", 2024-09-20T19:04:41.150 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8282999999999994e-05 2024-09-20T19:04:41.150 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.150 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.150 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:04:41.150 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.290843+0000", 2024-09-20T19:04:41.150 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1007999999999998e-05 2024-09-20T19:04:41.150 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:41.150 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:41.150 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:41.150 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.150 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.150 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:124636 9.2 9:45855acf:::benchmark_data_smithi083_124552_object10489:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e289)", 2024-09-20T19:04:41.151 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:20.997591+0000", 2024-09-20T19:04:41.151 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 260.05464244799998, 2024-09-20T19:04:41.151 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.29766285199999998, 2024-09-20T19:04:41.151 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:04:41.151 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:04:41.151 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:41.151 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:04:41.151 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:04:41.151 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:04:41.151 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 124636 2024-09-20T19:04:41.151 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.151 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:04:41.151 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.151 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:04:41.152 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.997591+0000", 2024-09-20T19:04:41.152 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:41.152 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.152 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.152 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:04:41.152 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.997591+0000", 2024-09-20T19:04:41.152 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:41.152 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.152 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.152 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:04:41.152 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.997592+0000", 2024-09-20T19:04:41.152 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.6000000000000003e-07 2024-09-20T19:04:41.152 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.152 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.152 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:04:41.153 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.997595+0000", 2024-09-20T19:04:41.153 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6449999999999998e-06 2024-09-20T19:04:41.153 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.153 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.153 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:04:41.153 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.997604+0000", 2024-09-20T19:04:41.153 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2090000000000005e-06 2024-09-20T19:04:41.153 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.153 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.153 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:04:41.153 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.997608+0000", 2024-09-20T19:04:41.153 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8960000000000002e-06 2024-09-20T19:04:41.153 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.154 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.154 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:41.154 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.997646+0000", 2024-09-20T19:04:41.154 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7184000000000002e-05 2024-09-20T19:04:41.154 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.154 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.154 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:04:41.154 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.997718+0000", 2024-09-20T19:04:41.154 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1884000000000005e-05 2024-09-20T19:04:41.154 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.154 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.154 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:04:41.154 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.997784+0000", 2024-09-20T19:04:41.154 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6018000000000004e-05 2024-09-20T19:04:41.155 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.155 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.155 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:04:41.155 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.999669+0000", 2024-09-20T19:04:41.155 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001885954 2024-09-20T19:04:41.155 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.155 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.155 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:41.155 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.999896+0000", 2024-09-20T19:04:41.155 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022635999999999999 2024-09-20T19:04:41.155 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.155 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.155 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:41.155 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.294934+0000", 2024-09-20T19:04:41.156 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.29503834600000001 2024-09-20T19:04:41.156 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.156 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.156 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:04:41.156 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.295205+0000", 2024-09-20T19:04:41.156 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00027075200000000003 2024-09-20T19:04:41.156 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.156 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.156 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:04:41.156 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.295254+0000", 2024-09-20T19:04:41.156 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8844000000000002e-05 2024-09-20T19:04:41.156 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:41.156 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:41.156 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:41.156 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.157 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.157 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:142913 9.c 9:3d798970:::benchmark_data_smithi083_124552_object28766:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e291)", 2024-09-20T19:04:41.157 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:35.168460+0000", 2024-09-20T19:04:41.157 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 245.88377322900001, 2024-09-20T19:04:41.157 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.040925943, 2024-09-20T19:04:41.157 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:04:41.157 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:04:41.157 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-20T19:04:41.157 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:04:41.157 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:04:41.157 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:04:41.157 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 142913 2024-09-20T19:04:41.157 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.157 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:04:41.158 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.158 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:04:41.158 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.168460+0000", 2024-09-20T19:04:41.158 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:41.158 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.158 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.158 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:04:41.158 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.168460+0000", 2024-09-20T19:04:41.158 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:41.158 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.158 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.158 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:04:41.158 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.168462+0000", 2024-09-20T19:04:41.158 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9520000000000001e-06 2024-09-20T19:04:41.159 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.159 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.159 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:04:41.159 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.168469+0000", 2024-09-20T19:04:41.159 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6309999999999997e-06 2024-09-20T19:04:41.159 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.159 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.159 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:04:41.159 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.168485+0000", 2024-09-20T19:04:41.159 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5798e-05 2024-09-20T19:04:41.159 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.159 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.159 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:04:41.159 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.168492+0000", 2024-09-20T19:04:41.160 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9179999999999994e-06 2024-09-20T19:04:41.160 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.160 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.160 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:41.160 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.168833+0000", 2024-09-20T19:04:41.160 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00034100200000000003 2024-09-20T19:04:41.160 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.160 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.160 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:04:41.160 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.169144+0000", 2024-09-20T19:04:41.160 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00031083499999999997 2024-09-20T19:04:41.160 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.160 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.160 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4,7", 2024-09-20T19:04:41.161 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.169258+0000", 2024-09-20T19:04:41.161 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011411 2024-09-20T19:04:41.161 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.161 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.161 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:04:41.161 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.172245+0000", 2024-09-20T19:04:41.161 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002987048 2024-09-20T19:04:41.161 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.161 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.161 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:41.161 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.203777+0000", 2024-09-20T19:04:41.161 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.031531525999999997 2024-09-20T19:04:41.161 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.161 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.162 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:41.162 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.213031+0000", 2024-09-20T19:04:41.162 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0092539930000000003 2024-09-20T19:04:41.162 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.162 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.162 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:04:41.162 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.213039+0000", 2024-09-20T19:04:41.162 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4090000000000003e-06 2024-09-20T19:04:41.162 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.162 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.162 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:41.162 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.209326+0000", 2024-09-20T19:04:41.162 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.99628679899999995 2024-09-20T19:04:41.162 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.163 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.163 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:04:41.163 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.209386+0000", 2024-09-20T19:04:41.163 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9922e-05 2024-09-20T19:04:41.163 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:41.163 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:41.163 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:41.163 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.163 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.163 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:142917 9.c 9:32f2df36:::benchmark_data_smithi083_124552_object28770:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e291)", 2024-09-20T19:04:41.163 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:35.172691+0000", 2024-09-20T19:04:41.163 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 245.87954285999999, 2024-09-20T19:04:41.163 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0372109220000001, 2024-09-20T19:04:41.163 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:04:41.163 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:04:41.164 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-20T19:04:41.164 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:04:41.164 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:04:41.164 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:04:41.164 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 142917 2024-09-20T19:04:41.164 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.164 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:04:41.164 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.164 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:04:41.164 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.172691+0000", 2024-09-20T19:04:41.164 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:41.164 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.164 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.164 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:04:41.165 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.172691+0000", 2024-09-20T19:04:41.165 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:41.165 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.165 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.165 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:04:41.165 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.172692+0000", 2024-09-20T19:04:41.165 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.95e-06 2024-09-20T19:04:41.165 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.165 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.165 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:04:41.165 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.172701+0000", 2024-09-20T19:04:41.165 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0630000000000006e-06 2024-09-20T19:04:41.165 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.165 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.166 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:04:41.166 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.172714+0000", 2024-09-20T19:04:41.166 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3303e-05 2024-09-20T19:04:41.166 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.166 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.166 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:04:41.166 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.172723+0000", 2024-09-20T19:04:41.166 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6160000000000008e-06 2024-09-20T19:04:41.166 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.166 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.166 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:41.166 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.172857+0000", 2024-09-20T19:04:41.166 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000133829 2024-09-20T19:04:41.166 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.167 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.167 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:04:41.167 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.172975+0000", 2024-09-20T19:04:41.167 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000117224 2024-09-20T19:04:41.167 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.167 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.167 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4,7", 2024-09-20T19:04:41.167 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.173088+0000", 2024-09-20T19:04:41.167 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000113551 2024-09-20T19:04:41.167 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.167 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.167 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:04:41.167 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.202597+0000", 2024-09-20T19:04:41.167 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.029508818999999999 2024-09-20T19:04:41.167 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.168 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.168 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:41.168 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.203835+0000", 2024-09-20T19:04:41.168 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001238444 2024-09-20T19:04:41.168 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.168 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.168 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:41.168 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.215654+0000", 2024-09-20T19:04:41.168 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011818403999999999 2024-09-20T19:04:41.168 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.168 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.168 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:04:41.168 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.215699+0000", 2024-09-20T19:04:41.168 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5084999999999999e-05 2024-09-20T19:04:41.168 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.169 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.169 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:41.169 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.209843+0000", 2024-09-20T19:04:41.169 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.99414379500000005 2024-09-20T19:04:41.169 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.169 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.169 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:04:41.169 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.209901+0000", 2024-09-20T19:04:41.169 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8838999999999997e-05 2024-09-20T19:04:41.169 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:41.169 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:41.169 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:41.169 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.169 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.170 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:142919 9.c 9:3aa96e52:::benchmark_data_smithi083_124552_object28772:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e291)", 2024-09-20T19:04:41.170 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:35.175226+0000", 2024-09-20T19:04:41.170 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 245.877007615, 2024-09-20T19:04:41.170 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.044694204, 2024-09-20T19:04:41.170 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:04:41.170 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:04:41.170 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:41.170 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:04:41.170 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:04:41.170 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:04:41.170 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 142919 2024-09-20T19:04:41.170 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.170 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:04:41.170 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.171 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:04:41.171 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.175226+0000", 2024-09-20T19:04:41.171 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:41.171 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.171 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.171 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:04:41.171 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.175226+0000", 2024-09-20T19:04:41.171 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:41.171 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.171 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.171 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:04:41.171 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.175228+0000", 2024-09-20T19:04:41.171 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.88e-06 2024-09-20T19:04:41.171 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.172 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.172 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:04:41.172 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.175235+0000", 2024-09-20T19:04:41.172 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8419999999999999e-06 2024-09-20T19:04:41.172 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.172 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.172 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:04:41.172 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.175253+0000", 2024-09-20T19:04:41.172 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8995e-05 2024-09-20T19:04:41.172 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.172 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.172 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:04:41.172 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.175262+0000", 2024-09-20T19:04:41.173 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.884e-06 2024-09-20T19:04:41.173 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.173 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.173 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:41.173 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.215752+0000", 2024-09-20T19:04:41.173 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.040489565999999998 2024-09-20T19:04:41.173 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.173 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.173 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:04:41.173 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.215759+0000", 2024-09-20T19:04:41.173 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1380000000000001e-06 2024-09-20T19:04:41.173 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.173 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.173 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:41.173 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.209925+0000", 2024-09-20T19:04:41.174 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.99416557900000002 2024-09-20T19:04:41.174 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.174 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.174 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:04:41.174 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.210025+0000", 2024-09-20T19:04:41.174 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010050700000000001 2024-09-20T19:04:41.174 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.174 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.174 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 7", 2024-09-20T19:04:41.174 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.210147+0000", 2024-09-20T19:04:41.174 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000122309 2024-09-20T19:04:41.174 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.174 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.175 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:41.175 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.219740+0000", 2024-09-20T19:04:41.175 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0095921180000000002 2024-09-20T19:04:41.175 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.175 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.175 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:04:41.175 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.219768+0000", 2024-09-20T19:04:41.175 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8391e-05 2024-09-20T19:04:41.175 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.175 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.175 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:04:41.175 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.219815+0000", 2024-09-20T19:04:41.175 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7197999999999998e-05 2024-09-20T19:04:41.175 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.176 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.176 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:04:41.176 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.219920+0000", 2024-09-20T19:04:41.176 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010479700000000001 2024-09-20T19:04:41.176 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:41.176 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:41.176 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:41.176 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.176 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.176 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14460.0:2698 5.0 5:05bf5b68:::notify.1:head [watch reconnect cookie 94056120167808 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e292)", 2024-09-20T19:04:41.176 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:35.178092+0000", 2024-09-20T19:04:41.176 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 245.87414153099999, 2024-09-20T19:04:41.176 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.021665381, 2024-09-20T19:04:41.176 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:04:41.177 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:04:41.177 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T19:04:41.177 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:04:41.177 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14460", 2024-09-20T19:04:41.177 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1555339426", 2024-09-20T19:04:41.177 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2698 2024-09-20T19:04:41.177 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.177 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:04:41.177 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.177 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:04:41.177 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.178092+0000", 2024-09-20T19:04:41.177 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:41.177 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.177 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.178 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:04:41.178 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.178092+0000", 2024-09-20T19:04:41.178 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:41.178 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.178 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.178 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:04:41.178 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.178093+0000", 2024-09-20T19:04:41.178 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2359999999999999e-06 2024-09-20T19:04:41.178 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.178 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.178 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:04:41.178 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.178098+0000", 2024-09-20T19:04:41.178 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1410000000000004e-06 2024-09-20T19:04:41.178 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.179 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.179 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:04:41.179 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.178113+0000", 2024-09-20T19:04:41.179 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4262e-05 2024-09-20T19:04:41.179 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.179 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.179 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:04:41.179 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.178120+0000", 2024-09-20T19:04:41.179 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.6210000000000004e-06 2024-09-20T19:04:41.179 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.179 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.179 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:41.179 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.180022+0000", 2024-09-20T19:04:41.180 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001901668 2024-09-20T19:04:41.180 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.180 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.180 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op must wait for map", 2024-09-20T19:04:41.180 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.180028+0000", 2024-09-20T19:04:41.180 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9359999999999996e-06 2024-09-20T19:04:41.180 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.180 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.180 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:41.180 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.195134+0000", 2024-09-20T19:04:41.180 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.015105831 2024-09-20T19:04:41.180 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.180 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.180 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:04:41.180 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.195142+0000", 2024-09-20T19:04:41.181 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8659999999999995e-06 2024-09-20T19:04:41.181 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.181 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.181 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:41.181 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.192667+0000", 2024-09-20T19:04:41.181 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.99752498700000003 2024-09-20T19:04:41.181 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.181 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.181 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:04:41.181 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.192972+0000", 2024-09-20T19:04:41.181 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000304677 2024-09-20T19:04:41.181 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.181 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.181 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:04:41.182 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.199757+0000", 2024-09-20T19:04:41.182 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0067851559999999997 2024-09-20T19:04:41.182 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:41.182 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:41.182 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:41.182 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.182 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.182 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:142931 9.c 9:3971d803:::benchmark_data_smithi083_124552_object28784:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e292)", 2024-09-20T19:04:41.182 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:35.206532+0000", 2024-09-20T19:04:41.182 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 245.845701862, 2024-09-20T19:04:41.182 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.016374055, 2024-09-20T19:04:41.182 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:04:41.182 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:04:41.182 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:41.183 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:04:41.183 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:04:41.183 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:04:41.183 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 142931 2024-09-20T19:04:41.183 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.183 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:04:41.183 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.183 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:04:41.183 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.206532+0000", 2024-09-20T19:04:41.183 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:41.183 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.183 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.183 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:04:41.183 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.206532+0000", 2024-09-20T19:04:41.183 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:41.184 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.184 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.184 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:04:41.184 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.206532+0000", 2024-09-20T19:04:41.184 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9500000000000001e-07 2024-09-20T19:04:41.184 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.184 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.184 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:04:41.184 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.206537+0000", 2024-09-20T19:04:41.184 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5600000000000004e-06 2024-09-20T19:04:41.184 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.184 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.184 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:04:41.184 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.206546+0000", 2024-09-20T19:04:41.185 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9479999999999997e-06 2024-09-20T19:04:41.185 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.185 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.185 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:04:41.185 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.206553+0000", 2024-09-20T19:04:41.185 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0119999999999999e-06 2024-09-20T19:04:41.185 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.185 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.185 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:41.185 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.215793+0000", 2024-09-20T19:04:41.185 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0092399519999999992 2024-09-20T19:04:41.185 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.185 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.185 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:04:41.186 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.215799+0000", 2024-09-20T19:04:41.186 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7389999999999996e-06 2024-09-20T19:04:41.186 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.186 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.186 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:41.186 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.212804+0000", 2024-09-20T19:04:41.186 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.99700544700000004 2024-09-20T19:04:41.186 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.187 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.187 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:04:41.187 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.213100+0000", 2024-09-20T19:04:41.187 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00029572499999999999 2024-09-20T19:04:41.187 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.187 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.187 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 7", 2024-09-20T19:04:41.187 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.213492+0000", 2024-09-20T19:04:41.187 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000391939 2024-09-20T19:04:41.187 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.187 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.187 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:04:41.187 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.219928+0000", 2024-09-20T19:04:41.187 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0064358100000000001 2024-09-20T19:04:41.188 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.188 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.188 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:41.188 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.222687+0000", 2024-09-20T19:04:41.188 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0027591949999999999 2024-09-20T19:04:41.188 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.188 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.188 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:04:41.188 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.222797+0000", 2024-09-20T19:04:41.188 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000110713 2024-09-20T19:04:41.188 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.188 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.188 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:04:41.188 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.222906+0000", 2024-09-20T19:04:41.188 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010812 2024-09-20T19:04:41.188 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:41.189 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:41.189 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:41.189 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.189 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.189 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:142996 9.c 9:3cdc78e2:::benchmark_data_smithi083_124552_object28849:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e292)", 2024-09-20T19:04:41.189 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:35.284730+0000", 2024-09-20T19:04:41.189 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 245.76750369300001, 2024-09-20T19:04:41.189 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.93861088699999995, 2024-09-20T19:04:41.189 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:04:41.189 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:04:41.189 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:41.189 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:04:41.189 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:04:41.189 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:04:41.189 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 142996 2024-09-20T19:04:41.190 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.190 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:04:41.190 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.190 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:04:41.190 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.284730+0000", 2024-09-20T19:04:41.190 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:41.190 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.190 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.190 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:04:41.190 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.284730+0000", 2024-09-20T19:04:41.190 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:04:41.190 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.190 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.190 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:04:41.190 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.284731+0000", 2024-09-20T19:04:41.190 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.376e-06 2024-09-20T19:04:41.191 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.191 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.191 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:04:41.191 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.284736+0000", 2024-09-20T19:04:41.191 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1680000000000003e-06 2024-09-20T19:04:41.191 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.191 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.191 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:04:41.191 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.284747+0000", 2024-09-20T19:04:41.191 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1077000000000001e-05 2024-09-20T19:04:41.191 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.191 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.191 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:04:41.191 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.284753+0000", 2024-09-20T19:04:41.192 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.874e-06 2024-09-20T19:04:41.192 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.192 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.192 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:41.192 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.284810+0000", 2024-09-20T19:04:41.192 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6573000000000002e-05 2024-09-20T19:04:41.192 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.192 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.192 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:04:41.192 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.284816+0000", 2024-09-20T19:04:41.192 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4840000000000001e-06 2024-09-20T19:04:41.192 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.192 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.192 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:04:41.192 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.216111+0000", 2024-09-20T19:04:41.193 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.93129438899999994 2024-09-20T19:04:41.193 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.193 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.193 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:04:41.193 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.216258+0000", 2024-09-20T19:04:41.193 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014760700000000001 2024-09-20T19:04:41.193 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.193 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.193 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 7", 2024-09-20T19:04:41.193 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.217033+0000", 2024-09-20T19:04:41.193 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00077462600000000005 2024-09-20T19:04:41.193 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.193 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.193 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:41.193 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.223055+0000", 2024-09-20T19:04:41.194 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0060222510000000002 2024-09-20T19:04:41.194 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.194 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.194 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:04:41.194 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.223067+0000", 2024-09-20T19:04:41.194 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2087e-05 2024-09-20T19:04:41.194 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.194 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.194 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:04:41.194 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.223265+0000", 2024-09-20T19:04:41.194 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019747800000000001 2024-09-20T19:04:41.194 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:04:41.194 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:04:41.194 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:04:41.195 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.223341+0000", 2024-09-20T19:04:41.195 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5896999999999995e-05 2024-09-20T19:04:41.195 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:41.195 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:41.195 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:41.195 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:04:41.195 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:04:41.195 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T19:04:41.231 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:41.271 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:41.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:41.272 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:41.373 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:41.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:41.411 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:41.411 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:41.512 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:41.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:41.556 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:41.557 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:41.658 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:41.685 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:41.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:41.686 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:41.772 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:41 smithi142 ceph-mon[32508]: 4.10 scrub starts 2024-09-20T19:04:41.772 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:41 smithi142 ceph-mon[32508]: 4.10 scrub ok 2024-09-20T19:04:41.787 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:41.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:41 smithi083 ceph-mon[29421]: 4.10 scrub starts 2024-09-20T19:04:41.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:41 smithi083 ceph-mon[29421]: 4.10 scrub ok 2024-09-20T19:04:41.792 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:41 smithi083 ceph-mon[36260]: 4.10 scrub starts 2024-09-20T19:04:41.792 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:41 smithi083 ceph-mon[36260]: 4.10 scrub ok 2024-09-20T19:04:41.819 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:41.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:41.820 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:41.921 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:41.955 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:41.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:41.956 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:42.057 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:42.101 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:42.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:42.102 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:42.203 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:42.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:42.238 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:42.239 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:42.339 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:42.375 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:42.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:42.377 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:42.478 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:04:42.511 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:42.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:42.513 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:04:42.614 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:42.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:42.652 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:42.652 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:42.753 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:42.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:42 smithi142 ceph-mon[32508]: pgmap v740: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 143 active+clean; 4.8 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 921 B/s rd, 0 B/s wr, 917 op/s; 12760/234591 objects misplaced (5.439%); 6.3 MiB/s, 100 objects/s recovering 2024-09-20T19:04:42.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:42 smithi142 ceph-mon[32508]: 5.14 scrub starts 2024-09-20T19:04:42.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:42 smithi142 ceph-mon[32508]: 5.14 scrub ok 2024-09-20T19:04:42.783 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:42.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:42.784 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:42.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:42 smithi083 ceph-mon[29421]: pgmap v740: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 143 active+clean; 4.8 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 921 B/s rd, 0 B/s wr, 917 op/s; 12760/234591 objects misplaced (5.439%); 6.3 MiB/s, 100 objects/s recovering 2024-09-20T19:04:42.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:42 smithi083 ceph-mon[29421]: 5.14 scrub starts 2024-09-20T19:04:42.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:42 smithi083 ceph-mon[29421]: 5.14 scrub ok 2024-09-20T19:04:42.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:42 smithi083 ceph-mon[36260]: pgmap v740: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 143 active+clean; 4.8 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 921 B/s rd, 0 B/s wr, 917 op/s; 12760/234591 objects misplaced (5.439%); 6.3 MiB/s, 100 objects/s recovering 2024-09-20T19:04:42.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:42 smithi083 ceph-mon[36260]: 5.14 scrub starts 2024-09-20T19:04:42.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:42 smithi083 ceph-mon[36260]: 5.14 scrub ok 2024-09-20T19:04:42.885 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:42.891 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-20T19:04:42.919 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:04:42.921 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:42.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:42.923 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:42.943 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:04:43.023 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:43.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:43.070 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:43.070 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:43.171 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:43.206 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:43.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:43.207 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:43.308 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:43.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:43.340 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:43.340 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:43.427 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:04:43.441 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:43.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:43.467 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:43.468 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:43.481 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:43 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:04:43.481 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:43 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:04:43.482 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:43 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:04:43.482 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:43 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:04:43.539 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T19:04:43.568 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:43.620 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:43.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:43.621 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:43.722 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:04:43.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:43.761 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:43.762 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:04:43.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:43 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:04:43.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:43 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:04:43.863 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:43.896 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:43.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:43.897 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:43.998 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:04:44.052 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:44.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:44.053 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:04:44.064 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:04:44.100 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T19:04:44.104 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-20T19:04:44.153 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:44.205 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:44.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:44.206 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:44.307 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:44.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:44.343 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:44.343 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:44.444 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:04:44.473 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:44.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:44.474 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:04:44.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:44 smithi083 ceph-mon[29421]: pgmap v741: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 143 active+clean; 4.5 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 936 op/s; 11650/221838 objects misplaced (5.252%); 6.9 MiB/s, 110 objects/s recovering 2024-09-20T19:04:44.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:44 smithi083 ceph-mon[29421]: osdmap e379: 8 total, 8 up, 8 in 2024-09-20T19:04:44.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:44 smithi083 ceph-mon[36260]: pgmap v741: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 143 active+clean; 4.5 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 936 op/s; 11650/221838 objects misplaced (5.252%); 6.9 MiB/s, 110 objects/s recovering 2024-09-20T19:04:44.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:44 smithi083 ceph-mon[36260]: osdmap e379: 8 total, 8 up, 8 in 2024-09-20T19:04:44.575 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:04:44.615 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:44.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:44.615 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:04:44.716 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:44.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:44.742 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:44.742 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:44.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:44 smithi142 ceph-mon[32508]: pgmap v741: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 143 active+clean; 4.5 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 936 op/s; 11650/221838 objects misplaced (5.252%); 6.9 MiB/s, 110 objects/s recovering 2024-09-20T19:04:44.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:44 smithi142 ceph-mon[32508]: osdmap e379: 8 total, 8 up, 8 in 2024-09-20T19:04:44.844 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:44.897 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:44.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:44.898 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:44.999 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:04:45.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:45.043 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:45.043 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:04:45.050 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":742,"stamp":"2024-09-20T19:04:43.336762+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4833614387,"num_objects":73946,"num_object_clones":0,"num_object_copies":221838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":11650,"num_objects_unfound":0,"num_objects_dirty":73946,"num_whiteouts":0,"num_read":2147,"num_read_kb":2004,"num_write":160808,"num_write_kb":7480650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21136,"num_bytes_recovered":1367090651,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92709,"ondisk_log_size":92709,"up":444,"acting":441,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":449,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":18952392,"kb_used_data":14711048,"kb_used_omap":16,"kb_used_meta":4241199,"kb_avail":731008824,"statfs":{"total":767960285184,"available":748553035776,"internally_reserved":0,"allocated":15064113152,"data_stored":15060928129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16790,"internal_metadata":4342988394},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,6,5,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-613548032,"num_objects":-9362,"num_object_clones":0,"num_object_copies":-28086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-2210,"num_objects_unfound":0,"num_objects_dirty":-9362,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":9362,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1109,"num_bytes_recovered":72679424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.207355"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":687,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.319994+0000","last_change":"2024-09-20T19:01:31.817967+0000","last_active":"2024-09-20T19:04:35.319994+0000","last_peered":"2024-09-20T19:04:35.319994+0000","last_clean":"2024-09-20T19:04:35.319994+0000","last_became_active":"2024-09-20T18:49:45.943201+0000","last_became_peered":"2024-09-20T18:49:45.943201+0000","last_unstale":"2024-09-20T19:04:35.319994+0000","last_undegraded":"2024-09-20T19:04:35.319994+0000","last_fullsized":"2024-09-20T19:04:35.319994+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:31.817909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:31.817909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"273'15","reported_seq":28,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.313081+0000","last_change":"2024-09-20T19:04:21.040978+0000","last_active":"2024-09-20T19:04:35.313081+0000","last_peered":"2024-09-20T19:04:35.313081+0000","last_clean":"2024-09-20T19:04:35.313081+0000","last_became_active":"2024-09-20T19:04:20.925853+0000","last_became_peered":"2024-09-20T19:04:20.925853+0000","last_unstale":"2024-09-20T19:04:35.313081+0000","last_undegraded":"2024-09-20T19:04:35.313081+0000","last_fullsized":"2024-09-20T19:04:35.313081+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:03:10.214154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:10.214154+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":677,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.322355+0000","last_change":"2024-09-20T18:57:49.016078+0000","last_active":"2024-09-20T19:04:35.322355+0000","last_peered":"2024-09-20T19:04:35.322355+0000","last_clean":"2024-09-20T19:04:35.322355+0000","last_became_active":"2024-09-20T18:49:49.959514+0000","last_became_peered":"2024-09-20T18:49:49.959514+0000","last_unstale":"2024-09-20T19:04:35.322355+0000","last_undegraded":"2024-09-20T19:04:35.322355+0000","last_fullsized":"2024-09-20T19:04:35.322355+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:49.016006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:49.016006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":18,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.314030+0000","last_change":"2024-09-20T19:04:20.881918+0000","last_active":"2024-09-20T19:04:35.314030+0000","last_peered":"2024-09-20T19:04:35.314030+0000","last_clean":"2024-09-20T19:04:35.314030+0000","last_became_active":"2024-09-20T19:04:20.880905+0000","last_became_peered":"2024-09-20T19:04:20.880905+0000","last_unstale":"2024-09-20T19:04:35.314030+0000","last_undegraded":"2024-09-20T19:04:35.314030+0000","last_fullsized":"2024-09-20T19:04:35.314030+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:02.084375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:02.084375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":686,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.328585+0000","last_change":"2024-09-20T19:02:02.441633+0000","last_active":"2024-09-20T19:04:35.328585+0000","last_peered":"2024-09-20T19:04:35.328585+0000","last_clean":"2024-09-20T19:04:35.328585+0000","last_became_active":"2024-09-20T18:59:05.081081+0000","last_became_peered":"2024-09-20T18:59:05.081081+0000","last_unstale":"2024-09-20T19:04:35.328585+0000","last_undegraded":"2024-09-20T19:04:35.328585+0000","last_fullsized":"2024-09-20T19:04:35.328585+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:02.441577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:02.441577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":700,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:37.540824+0000","last_change":"2024-09-20T19:04:37.540824+0000","last_active":"2024-09-20T19:04:37.540824+0000","last_peered":"2024-09-20T19:04:37.540824+0000","last_clean":"2024-09-20T19:04:37.540824+0000","last_became_active":"2024-09-20T18:59:05.072675+0000","last_became_peered":"2024-09-20T18:59:05.072675+0000","last_unstale":"2024-09-20T19:04:37.540824+0000","last_undegraded":"2024-09-20T19:04:37.540824+0000","last_fullsized":"2024-09-20T19:04:37.540824+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:37.540760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:04:37.540760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:55.234054+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"274'27","reported_seq":724,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.329511+0000","last_change":"2024-09-20T19:02:40.899215+0000","last_active":"2024-09-20T19:04:35.329511+0000","last_peered":"2024-09-20T19:04:35.329511+0000","last_clean":"2024-09-20T19:04:35.329511+0000","last_became_active":"2024-09-20T18:49:47.952150+0000","last_became_peered":"2024-09-20T18:49:47.952150+0000","last_unstale":"2024-09-20T19:04:35.329511+0000","last_undegraded":"2024-09-20T19:04:35.329511+0000","last_fullsized":"2024-09-20T19:04:35.329511+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'27","last_scrub_stamp":"2024-09-20T19:02:40.899139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:40.899139+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":18,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.313134+0000","last_change":"2024-09-20T19:04:20.926234+0000","last_active":"2024-09-20T19:04:35.313134+0000","last_peered":"2024-09-20T19:04:35.313134+0000","last_clean":"2024-09-20T19:04:35.313134+0000","last_became_active":"2024-09-20T19:04:20.925394+0000","last_became_peered":"2024-09-20T19:04:20.925394+0000","last_unstale":"2024-09-20T19:04:35.313134+0000","last_undegraded":"2024-09-20T19:04:35.313134+0000","last_fullsized":"2024-09-20T19:04:35.313134+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:23.982813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:23.982813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"59'1","reported_seq":761,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.327097+0000","last_change":"2024-09-20T19:04:20.963496+0000","last_active":"2024-09-20T19:04:35.327097+0000","last_peered":"2024-09-20T19:04:35.327097+0000","last_clean":"2024-09-20T19:04:35.327097+0000","last_became_active":"2024-09-20T19:04:20.914145+0000","last_became_peered":"2024-09-20T19:04:20.914145+0000","last_unstale":"2024-09-20T19:04:35.327097+0000","last_undegraded":"2024-09-20T19:04:35.327097+0000","last_fullsized":"2024-09-20T19:04:35.327097+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:01:28.592653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:28.592653+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"273'15","reported_seq":28,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.314117+0000","last_change":"2024-09-20T19:04:20.984223+0000","last_active":"2024-09-20T19:04:35.314117+0000","last_peered":"2024-09-20T19:04:35.314117+0000","last_clean":"2024-09-20T19:04:35.314117+0000","last_became_active":"2024-09-20T19:04:20.898235+0000","last_became_peered":"2024-09-20T19:04:20.898235+0000","last_unstale":"2024-09-20T19:04:35.314117+0000","last_undegraded":"2024-09-20T19:04:35.314117+0000","last_fullsized":"2024-09-20T19:04:35.314117+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:02:42.114331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:42.114331+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":684,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.327524+0000","last_change":"2024-09-20T19:01:51.564693+0000","last_active":"2024-09-20T19:04:35.327524+0000","last_peered":"2024-09-20T19:04:35.327524+0000","last_clean":"2024-09-20T19:04:35.327524+0000","last_became_active":"2024-09-20T18:49:49.961188+0000","last_became_peered":"2024-09-20T18:49:49.961188+0000","last_unstale":"2024-09-20T19:04:35.327524+0000","last_undegraded":"2024-09-20T19:04:35.327524+0000","last_fullsized":"2024-09-20T19:04:35.327524+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:51.564624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:51.564624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":666,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.319437+0000","last_change":"2024-09-20T19:03:09.963421+0000","last_active":"2024-09-20T19:04:35.319437+0000","last_peered":"2024-09-20T19:04:35.319437+0000","last_clean":"2024-09-20T19:04:35.319437+0000","last_became_active":"2024-09-20T18:49:51.966715+0000","last_became_peered":"2024-09-20T18:49:51.966715+0000","last_unstale":"2024-09-20T19:04:35.319437+0000","last_undegraded":"2024-09-20T19:04:35.319437+0000","last_fullsized":"2024-09-20T19:04:35.319437+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:09.963298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:03:09.963298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":709,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.326620+0000","last_change":"2024-09-20T19:04:20.879227+0000","last_active":"2024-09-20T19:04:35.326620+0000","last_peered":"2024-09-20T19:04:35.326620+0000","last_clean":"2024-09-20T19:04:35.326620+0000","last_became_active":"2024-09-20T19:04:20.878905+0000","last_became_peered":"2024-09-20T19:04:20.878905+0000","last_unstale":"2024-09-20T19:04:35.326620+0000","last_undegraded":"2024-09-20T19:04:35.326620+0000","last_fullsized":"2024-09-20T19:04:35.326620+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:48.520017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:48.520017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"273'7","reported_seq":24,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.313466+0000","last_change":"2024-09-20T19:04:20.907713+0000","last_active":"2024-09-20T19:04:35.313466+0000","last_peered":"2024-09-20T19:04:35.313466+0000","last_clean":"2024-09-20T19:04:35.313466+0000","last_became_active":"2024-09-20T19:04:20.888572+0000","last_became_peered":"2024-09-20T19:04:20.888572+0000","last_unstale":"2024-09-20T19:04:35.313466+0000","last_undegraded":"2024-09-20T19:04:35.313466+0000","last_fullsized":"2024-09-20T19:04:35.313466+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-20T18:58:33.660531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:33.660531+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":702,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.315736+0000","last_change":"2024-09-20T19:04:20.885873+0000","last_active":"2024-09-20T19:04:35.315736+0000","last_peered":"2024-09-20T19:04:35.315736+0000","last_clean":"2024-09-20T19:04:35.315736+0000","last_became_active":"2024-09-20T19:04:20.885417+0000","last_became_peered":"2024-09-20T19:04:20.885417+0000","last_unstale":"2024-09-20T19:04:35.315736+0000","last_undegraded":"2024-09-20T19:04:35.315736+0000","last_fullsized":"2024-09-20T19:04:35.315736+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:33.203312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:33.203312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":673,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.326113+0000","last_change":"2024-09-20T18:58:26.465305+0000","last_active":"2024-09-20T19:04:35.326113+0000","last_peered":"2024-09-20T19:04:35.326113+0000","last_clean":"2024-09-20T19:04:35.326113+0000","last_became_active":"2024-09-20T18:49:51.970663+0000","last_became_peered":"2024-09-20T18:49:51.970663+0000","last_unstale":"2024-09-20T19:04:35.326113+0000","last_undegraded":"2024-09-20T19:04:35.326113+0000","last_fullsized":"2024-09-20T19:04:35.326113+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:26.465264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:26.465264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:20.487829+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":18,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.314535+0000","last_change":"2024-09-20T19:04:20.907531+0000","last_active":"2024-09-20T19:04:35.314535+0000","last_peered":"2024-09-20T19:04:35.314535+0000","last_clean":"2024-09-20T19:04:35.314535+0000","last_became_active":"2024-09-20T19:04:20.906109+0000","last_became_peered":"2024-09-20T19:04:20.906109+0000","last_unstale":"2024-09-20T19:04:35.314535+0000","last_undegraded":"2024-09-20T19:04:35.314535+0000","last_fullsized":"2024-09-20T19:04:35.314535+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:00.906652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:00.906652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":710,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.319643+0000","last_change":"2024-09-20T19:04:20.907057+0000","last_active":"2024-09-20T19:04:35.319643+0000","last_peered":"2024-09-20T19:04:35.319643+0000","last_clean":"2024-09-20T19:04:35.319643+0000","last_became_active":"2024-09-20T19:04:20.906691+0000","last_became_peered":"2024-09-20T19:04:20.906691+0000","last_unstale":"2024-09-20T19:04:35.319643+0000","last_undegraded":"2024-09-20T19:04:35.319643+0000","last_fullsized":"2024-09-20T19:04:35.319643+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:44.998147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:44.998147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"273'27","reported_seq":734,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.337830+0000","last_change":"2024-09-20T19:03:09.456720+0000","last_active":"2024-09-20T19:04:35.337830+0000","last_peered":"2024-09-20T19:04:35.337830+0000","last_clean":"2024-09-20T19:04:35.337830+0000","last_became_active":"2024-09-20T18:49:47.950939+0000","last_became_peered":"2024-09-20T18:49:47.950939+0000","last_unstale":"2024-09-20T19:04:35.337830+0000","last_undegraded":"2024-09-20T19:04:35.337830+0000","last_fullsized":"2024-09-20T19:04:35.337830+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'27","last_scrub_stamp":"2024-09-20T19:03:09.456631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:09.456631+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:32.393537+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":693,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.321490+0000","last_change":"2024-09-20T19:04:20.890659+0000","last_active":"2024-09-20T19:04:35.321490+0000","last_peered":"2024-09-20T19:04:35.321490+0000","last_clean":"2024-09-20T19:04:35.321490+0000","last_became_active":"2024-09-20T19:04:20.890370+0000","last_became_peered":"2024-09-20T19:04:20.890370+0000","last_unstale":"2024-09-20T19:04:35.321490+0000","last_undegraded":"2024-09-20T19:04:35.321490+0000","last_fullsized":"2024-09-20T19:04:35.321490+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:05.888974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:05.888974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":688,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.332722+0000","last_change":"2024-09-20T19:02:28.934597+0000","last_active":"2024-09-20T19:04:35.332722+0000","last_peered":"2024-09-20T19:04:35.332722+0000","last_clean":"2024-09-20T19:04:35.332722+0000","last_became_active":"2024-09-20T18:59:05.084496+0000","last_became_peered":"2024-09-20T18:59:05.084496+0000","last_unstale":"2024-09-20T19:04:35.332722+0000","last_undegraded":"2024-09-20T19:04:35.332722+0000","last_fullsized":"2024-09-20T19:04:35.332722+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:28.934264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:28.934264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"59'1","reported_seq":22,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.313873+0000","last_change":"2024-09-20T19:04:20.905233+0000","last_active":"2024-09-20T19:04:35.313873+0000","last_peered":"2024-09-20T19:04:35.313873+0000","last_clean":"2024-09-20T19:04:35.313873+0000","last_became_active":"2024-09-20T19:04:20.887062+0000","last_became_peered":"2024-09-20T19:04:20.887062+0000","last_unstale":"2024-09-20T19:04:35.313873+0000","last_undegraded":"2024-09-20T19:04:35.313873+0000","last_fullsized":"2024-09-20T19:04:35.313873+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:03:08.218812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:03:08.218812+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":993,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"274'18","reported_seq":737,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.334231+0000","last_change":"2024-09-20T19:04:20.925690+0000","last_active":"2024-09-20T19:04:35.334231+0000","last_peered":"2024-09-20T19:04:35.334231+0000","last_clean":"2024-09-20T19:04:35.334231+0000","last_became_active":"2024-09-20T19:04:20.896343+0000","last_became_peered":"2024-09-20T19:04:20.896343+0000","last_unstale":"2024-09-20T19:04:35.334231+0000","last_undegraded":"2024-09-20T19:04:35.334231+0000","last_fullsized":"2024-09-20T19:04:35.334231+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:57:09.818999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:09.818999+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:26.562059+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":18,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.313838+0000","last_change":"2024-09-20T19:04:20.889786+0000","last_active":"2024-09-20T19:04:35.313838+0000","last_peered":"2024-09-20T19:04:35.313838+0000","last_clean":"2024-09-20T19:04:35.313838+0000","last_became_active":"2024-09-20T19:04:20.889430+0000","last_became_peered":"2024-09-20T19:04:20.889430+0000","last_unstale":"2024-09-20T19:04:35.313838+0000","last_undegraded":"2024-09-20T19:04:35.313838+0000","last_fullsized":"2024-09-20T19:04:35.313838+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:02.094473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:02.094473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":674,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.318218+0000","last_change":"2024-09-20T19:02:28.294390+0000","last_active":"2024-09-20T19:04:35.318218+0000","last_peered":"2024-09-20T19:04:35.318218+0000","last_clean":"2024-09-20T19:04:35.318218+0000","last_became_active":"2024-09-20T18:49:51.970058+0000","last_became_peered":"2024-09-20T18:49:51.970058+0000","last_unstale":"2024-09-20T19:04:35.318218+0000","last_undegraded":"2024-09-20T19:04:35.318218+0000","last_fullsized":"2024-09-20T19:04:35.318218+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T19:02:28.294292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:28.294292+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":703,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.322472+0000","last_change":"2024-09-20T19:04:20.902126+0000","last_active":"2024-09-20T19:04:35.322472+0000","last_peered":"2024-09-20T19:04:35.322472+0000","last_clean":"2024-09-20T19:04:35.322472+0000","last_became_active":"2024-09-20T19:04:20.881466+0000","last_became_peered":"2024-09-20T19:04:20.881466+0000","last_unstale":"2024-09-20T19:04:35.322472+0000","last_undegraded":"2024-09-20T19:04:35.322472+0000","last_fullsized":"2024-09-20T19:04:35.322472+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T18:57:18.032067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:57:18.032067+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"274'20","reported_seq":748,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.327228+0000","last_change":"2024-09-20T19:03:16.982915+0000","last_active":"2024-09-20T19:04:35.327228+0000","last_peered":"2024-09-20T19:04:35.327228+0000","last_clean":"2024-09-20T19:04:35.327228+0000","last_became_active":"2024-09-20T19:03:16.968636+0000","last_became_peered":"2024-09-20T19:03:16.968636+0000","last_unstale":"2024-09-20T19:04:35.327228+0000","last_undegraded":"2024-09-20T19:04:35.327228+0000","last_fullsized":"2024-09-20T19:04:35.327228+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'20","last_scrub_stamp":"2024-09-20T19:02:35.096979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:35.096979+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"375'14","reported_seq":371,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:36.183111+0000","last_change":"2024-09-20T19:04:20.956210+0000","last_active":"2024-09-20T19:04:36.183111+0000","last_peered":"2024-09-20T19:04:36.183111+0000","last_clean":"2024-09-20T19:04:36.183111+0000","last_became_active":"2024-09-20T19:04:20.900782+0000","last_became_peered":"2024-09-20T19:04:20.900782+0000","last_unstale":"2024-09-20T19:04:36.183111+0000","last_undegraded":"2024-09-20T19:04:36.183111+0000","last_fullsized":"2024-09-20T19:04:36.183111+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"308'12","last_scrub_stamp":"2024-09-20T19:01:44.541156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:44.541156+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"378'9907","reported_seq":23117,"reported_epoch":378,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-20T19:04:42.482091+0000","last_change":"2024-09-20T19:04:22.073177+0000","last_active":"2024-09-20T19:04:42.482091+0000","last_peered":"2024-09-20T19:04:42.482091+0000","last_clean":"2024-09-20T19:04:20.251202+0000","last_became_active":"2024-09-20T19:04:22.043451+0000","last_became_peered":"2024-09-20T19:04:22.043451+0000","last_unstale":"2024-09-20T19:04:42.482091+0000","last_undegraded":"2024-09-20T19:04:42.482091+0000","last_fullsized":"2024-09-20T19:04:42.482091+0000","mapping_epoch":375,"log_start":"346'4500","ondisk_log_start":"346'4500","created":325,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":5407,"log_dups_size":0,"ondisk_log_size":5407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":298778624,"num_objects":4559,"num_object_clones":0,"num_object_copies":13677,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4559,"num_objects_unfound":0,"num_objects_dirty":4559,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9907,"num_write_kb":462912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6534,"num_bytes_recovered":428212224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[0,7,1],"avail_no_missing":["0","1","7"],"object_location_counts":[{"shards":"0,1,7","objects":4559}],"blocked_by":[],"up_primary":4,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.f","version":"274'27","reported_seq":767,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.340198+0000","last_change":"2024-09-20T19:04:20.920811+0000","last_active":"2024-09-20T19:04:35.340198+0000","last_peered":"2024-09-20T19:04:35.340198+0000","last_clean":"2024-09-20T19:04:35.340198+0000","last_became_active":"2024-09-20T19:04:20.897049+0000","last_became_peered":"2024-09-20T19:04:20.897049+0000","last_unstale":"2024-09-20T19:04:35.340198+0000","last_undegraded":"2024-09-20T19:04:35.340198+0000","last_fullsized":"2024-09-20T19:04:35.340198+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'27","last_scrub_stamp":"2024-09-20T19:02:27.897138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:27.897138+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:27.432437+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":685,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:39.537914+0000","last_change":"2024-09-20T19:04:39.537914+0000","last_active":"2024-09-20T19:04:39.537914+0000","last_peered":"2024-09-20T19:04:39.537914+0000","last_clean":"2024-09-20T19:04:39.537914+0000","last_became_active":"2024-09-20T18:49:45.936104+0000","last_became_peered":"2024-09-20T18:49:45.936104+0000","last_unstale":"2024-09-20T19:04:39.537914+0000","last_undegraded":"2024-09-20T19:04:39.537914+0000","last_fullsized":"2024-09-20T19:04:39.537914+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:39.537818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:04:39.537818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:44.147654+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"375'12","reported_seq":26,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.314673+0000","last_change":"2024-09-20T19:04:21.030980+0000","last_active":"2024-09-20T19:04:35.314673+0000","last_peered":"2024-09-20T19:04:35.314673+0000","last_clean":"2024-09-20T19:04:35.314673+0000","last_became_active":"2024-09-20T19:04:20.907228+0000","last_became_peered":"2024-09-20T19:04:20.907228+0000","last_unstale":"2024-09-20T19:04:35.314673+0000","last_undegraded":"2024-09-20T19:04:35.314673+0000","last_fullsized":"2024-09-20T19:04:35.314673+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"308'10","last_scrub_stamp":"2024-09-20T19:02:23.017626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:23.017626+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":673,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.326704+0000","last_change":"2024-09-20T19:02:22.736879+0000","last_active":"2024-09-20T19:04:35.326704+0000","last_peered":"2024-09-20T19:04:35.326704+0000","last_clean":"2024-09-20T19:04:35.326704+0000","last_became_active":"2024-09-20T18:49:51.971399+0000","last_became_peered":"2024-09-20T18:49:51.971399+0000","last_unstale":"2024-09-20T19:04:35.326704+0000","last_undegraded":"2024-09-20T19:04:35.326704+0000","last_fullsized":"2024-09-20T19:04:35.326704+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:22.736825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:22.736825+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.e","version":"378'10064","reported_seq":11104,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:40.614130+0000","last_change":"2024-09-20T19:04:21.970634+0000","last_active":"2024-09-20T19:04:40.614130+0000","last_peered":"2024-09-20T19:04:40.614130+0000","last_clean":"2024-09-20T19:04:40.614130+0000","last_became_active":"2024-09-20T19:04:21.956145+0000","last_became_peered":"2024-09-20T19:04:21.956145+0000","last_unstale":"2024-09-20T19:04:40.614130+0000","last_undegraded":"2024-09-20T19:04:40.614130+0000","last_fullsized":"2024-09-20T19:04:40.614130+0000","mapping_epoch":375,"log_start":"348'5428","ondisk_log_start":"348'5428","created":325,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4636,"log_dups_size":0,"ondisk_log_size":4636,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":310116352,"num_objects":4732,"num_object_clones":0,"num_object_copies":14196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10064,"num_write_kb":473472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":453,"num_bytes_recovered":29687808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"273'30","reported_seq":740,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.328787+0000","last_change":"2024-09-20T19:03:06.038303+0000","last_active":"2024-09-20T19:04:35.328787+0000","last_peered":"2024-09-20T19:04:35.328787+0000","last_clean":"2024-09-20T19:04:35.328787+0000","last_became_active":"2024-09-20T18:49:48.499189+0000","last_became_peered":"2024-09-20T18:49:48.499189+0000","last_unstale":"2024-09-20T19:04:35.328787+0000","last_undegraded":"2024-09-20T19:04:35.328787+0000","last_fullsized":"2024-09-20T19:04:35.328787+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'30","last_scrub_stamp":"2024-09-20T19:03:06.038213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:06.038213+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":677,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.333317+0000","last_change":"2024-09-20T19:02:22.806587+0000","last_active":"2024-09-20T19:04:35.333317+0000","last_peered":"2024-09-20T19:04:35.333317+0000","last_clean":"2024-09-20T19:04:35.333317+0000","last_became_active":"2024-09-20T18:49:45.940544+0000","last_became_peered":"2024-09-20T18:49:45.940544+0000","last_unstale":"2024-09-20T19:04:35.333317+0000","last_undegraded":"2024-09-20T19:04:35.333317+0000","last_fullsized":"2024-09-20T19:04:35.333317+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:22.806509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:22.806509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":18,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.313511+0000","last_change":"2024-09-20T19:04:20.889280+0000","last_active":"2024-09-20T19:04:35.313511+0000","last_peered":"2024-09-20T19:04:35.313511+0000","last_clean":"2024-09-20T19:04:35.313511+0000","last_became_active":"2024-09-20T19:04:20.888422+0000","last_became_peered":"2024-09-20T19:04:20.888422+0000","last_unstale":"2024-09-20T19:04:35.313511+0000","last_undegraded":"2024-09-20T19:04:35.313511+0000","last_fullsized":"2024-09-20T19:04:35.313511+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.229201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.229201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":18,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.314124+0000","last_change":"2024-09-20T19:04:20.890045+0000","last_active":"2024-09-20T19:04:35.314124+0000","last_peered":"2024-09-20T19:04:35.314124+0000","last_clean":"2024-09-20T19:04:35.314124+0000","last_became_active":"2024-09-20T19:04:20.889418+0000","last_became_peered":"2024-09-20T19:04:20.889418+0000","last_unstale":"2024-09-20T19:04:35.314124+0000","last_undegraded":"2024-09-20T19:04:35.314124+0000","last_fullsized":"2024-09-20T19:04:35.314124+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:27.704520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:27.704520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"378'9889","reported_seq":10008,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:41.741748+0000","last_change":"2024-09-20T19:02:11.828279+0000","last_active":"2024-09-20T19:04:41.741748+0000","last_peered":"2024-09-20T19:04:41.741748+0000","last_clean":"2024-09-20T19:04:41.741748+0000","last_became_active":"2024-09-20T19:02:11.827468+0000","last_became_peered":"2024-09-20T19:02:11.827468+0000","last_unstale":"2024-09-20T19:04:41.741748+0000","last_undegraded":"2024-09-20T19:04:41.741748+0000","last_fullsized":"2024-09-20T19:04:41.741748+0000","mapping_epoch":325,"log_start":"342'3855","ondisk_log_start":"342'3855","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":6034,"log_dups_size":0,"ondisk_log_size":6034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":294977536,"num_objects":4501,"num_object_clones":0,"num_object_copies":13503,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9889,"num_write_kb":460480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"274'28","reported_seq":34,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.313422+0000","last_change":"2024-09-20T19:04:20.976682+0000","last_active":"2024-09-20T19:04:35.313422+0000","last_peered":"2024-09-20T19:04:35.313422+0000","last_clean":"2024-09-20T19:04:35.313422+0000","last_became_active":"2024-09-20T19:04:20.897444+0000","last_became_peered":"2024-09-20T19:04:20.897444+0000","last_unstale":"2024-09-20T19:04:35.313422+0000","last_undegraded":"2024-09-20T19:04:35.313422+0000","last_fullsized":"2024-09-20T19:04:35.313422+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'28","last_scrub_stamp":"2024-09-20T19:03:06.269623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:06.269623+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":41,"num_read_kb":34,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":712,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.327487+0000","last_change":"2024-09-20T19:04:21.031015+0000","last_active":"2024-09-20T19:04:35.327487+0000","last_peered":"2024-09-20T19:04:35.327487+0000","last_clean":"2024-09-20T19:04:35.327487+0000","last_became_active":"2024-09-20T19:04:20.906790+0000","last_became_peered":"2024-09-20T19:04:20.906790+0000","last_unstale":"2024-09-20T19:04:35.327487+0000","last_undegraded":"2024-09-20T19:04:35.327487+0000","last_fullsized":"2024-09-20T19:04:35.327487+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:00:29.890681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:00:29.890681+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"375'13","reported_seq":913,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:36.176858+0000","last_change":"2024-09-20T19:04:20.916872+0000","last_active":"2024-09-20T19:04:36.176858+0000","last_peered":"2024-09-20T19:04:36.176858+0000","last_clean":"2024-09-20T19:04:36.176858+0000","last_became_active":"2024-09-20T19:04:20.896424+0000","last_became_peered":"2024-09-20T19:04:20.896424+0000","last_unstale":"2024-09-20T19:04:36.176858+0000","last_undegraded":"2024-09-20T19:04:36.176858+0000","last_fullsized":"2024-09-20T19:04:36.176858+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:58:23.114371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:23.114371+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:33.557657+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":681,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.316080+0000","last_change":"2024-09-20T18:59:05.082117+0000","last_active":"2024-09-20T19:04:35.316080+0000","last_peered":"2024-09-20T19:04:35.316080+0000","last_clean":"2024-09-20T19:04:35.316080+0000","last_became_active":"2024-09-20T18:59:05.081814+0000","last_became_peered":"2024-09-20T18:59:05.081814+0000","last_unstale":"2024-09-20T19:04:35.316080+0000","last_undegraded":"2024-09-20T19:04:35.316080+0000","last_fullsized":"2024-09-20T19:04:35.316080+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:29.530001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:29.530001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:34.493661+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"378'9786","reported_seq":23027,"reported_epoch":378,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-20T19:04:41.347128+0000","last_change":"2024-09-20T19:04:21.993116+0000","last_active":"2024-09-20T19:04:41.347128+0000","last_peered":"2024-09-20T19:04:41.347128+0000","last_clean":"2024-09-20T19:04:20.314777+0000","last_became_active":"2024-09-20T19:04:21.965998+0000","last_became_peered":"2024-09-20T19:04:21.965998+0000","last_unstale":"2024-09-20T19:04:41.347128+0000","last_undegraded":"2024-09-20T19:04:41.347128+0000","last_fullsized":"2024-09-20T19:04:41.347128+0000","mapping_epoch":375,"log_start":"346'4609","ondisk_log_start":"346'4609","created":325,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":5177,"log_dups_size":0,"ondisk_log_size":5177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":305659904,"num_objects":4664,"num_object_clones":0,"num_object_copies":13992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4664,"num_objects_unfound":0,"num_objects_dirty":4664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9786,"num_write_kb":462400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6550,"num_bytes_recovered":429260800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,0,3],"avail_no_missing":["7","0","3"],"object_location_counts":[{"shards":"0,3,7","objects":4664}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"274'20","reported_seq":777,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.329184+0000","last_change":"2024-09-20T19:04:20.917366+0000","last_active":"2024-09-20T19:04:35.329184+0000","last_peered":"2024-09-20T19:04:35.329184+0000","last_clean":"2024-09-20T19:04:35.329184+0000","last_became_active":"2024-09-20T19:04:20.881612+0000","last_became_peered":"2024-09-20T19:04:20.881612+0000","last_unstale":"2024-09-20T19:04:35.329184+0000","last_undegraded":"2024-09-20T19:04:35.329184+0000","last_fullsized":"2024-09-20T19:04:35.329184+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'20","last_scrub_stamp":"2024-09-20T19:02:35.930910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:35.930910+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":702,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.327169+0000","last_change":"2024-09-20T19:02:30.686939+0000","last_active":"2024-09-20T19:04:35.327169+0000","last_peered":"2024-09-20T19:04:35.327169+0000","last_clean":"2024-09-20T19:04:35.327169+0000","last_became_active":"2024-09-20T18:49:45.942496+0000","last_became_peered":"2024-09-20T18:49:45.942496+0000","last_unstale":"2024-09-20T19:04:35.327169+0000","last_undegraded":"2024-09-20T19:04:35.327169+0000","last_fullsized":"2024-09-20T19:04:35.327169+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:30.686872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:30.686872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"262'10","reported_seq":889,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:36.177036+0000","last_change":"2024-09-20T19:02:04.994109+0000","last_active":"2024-09-20T19:04:36.177036+0000","last_peered":"2024-09-20T19:04:36.177036+0000","last_clean":"2024-09-20T19:04:36.177036+0000","last_became_active":"2024-09-20T18:59:05.080183+0000","last_became_peered":"2024-09-20T18:59:05.080183+0000","last_unstale":"2024-09-20T19:04:36.177036+0000","last_undegraded":"2024-09-20T19:04:36.177036+0000","last_fullsized":"2024-09-20T19:04:36.177036+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'10","last_scrub_stamp":"2024-09-20T19:02:04.994047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:04.994047+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":659,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.320260+0000","last_change":"2024-09-20T19:01:37.900410+0000","last_active":"2024-09-20T19:04:35.320260+0000","last_peered":"2024-09-20T19:04:35.320260+0000","last_clean":"2024-09-20T19:04:35.320260+0000","last_became_active":"2024-09-20T18:49:51.964780+0000","last_became_peered":"2024-09-20T18:49:51.964780+0000","last_unstale":"2024-09-20T19:04:35.320260+0000","last_undegraded":"2024-09-20T19:04:35.320260+0000","last_fullsized":"2024-09-20T19:04:35.320260+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:37.900328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:37.900328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.d","version":"378'10290","reported_seq":14594,"reported_epoch":378,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-20T19:04:41.753723+0000","last_change":"2024-09-20T19:04:21.081624+0000","last_active":"2024-09-20T19:04:41.753723+0000","last_peered":"2024-09-20T19:04:41.753723+0000","last_clean":"2024-09-20T19:03:13.933264+0000","last_became_active":"2024-09-20T19:04:21.049054+0000","last_became_peered":"2024-09-20T19:04:21.049054+0000","last_unstale":"2024-09-20T19:04:41.753723+0000","last_undegraded":"2024-09-20T19:04:41.753723+0000","last_fullsized":"2024-09-20T19:04:20.871004+0000","mapping_epoch":374,"log_start":"344'4217","ondisk_log_start":"344'4217","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":6073,"log_dups_size":0,"ondisk_log_size":6073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":305201176,"num_objects":4658,"num_object_clones":0,"num_object_copies":13974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":797,"num_objects_unfound":0,"num_objects_dirty":4658,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10290,"num_write_kb":478273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2090,"num_bytes_recovered":136904704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,0],"avail_no_missing":["6","0"],"object_location_counts":[{"shards":"0,3,6","objects":797},{"shards":"0,6","objects":3861}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"274'29","reported_seq":726,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.319476+0000","last_change":"2024-09-20T19:00:04.810907+0000","last_active":"2024-09-20T19:04:35.319476+0000","last_peered":"2024-09-20T19:04:35.319476+0000","last_clean":"2024-09-20T19:04:35.319476+0000","last_became_active":"2024-09-20T18:49:48.497736+0000","last_became_peered":"2024-09-20T18:49:48.497736+0000","last_unstale":"2024-09-20T19:04:35.319476+0000","last_undegraded":"2024-09-20T19:04:35.319476+0000","last_fullsized":"2024-09-20T19:04:35.319476+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'29","last_scrub_stamp":"2024-09-20T19:00:04.810831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:00:04.810831+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":678,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.321460+0000","last_change":"2024-09-20T19:02:41.859231+0000","last_active":"2024-09-20T19:04:35.321460+0000","last_peered":"2024-09-20T19:04:35.321460+0000","last_clean":"2024-09-20T19:04:35.321460+0000","last_became_active":"2024-09-20T18:49:45.942253+0000","last_became_peered":"2024-09-20T18:49:45.942253+0000","last_unstale":"2024-09-20T19:04:35.321460+0000","last_undegraded":"2024-09-20T19:04:35.321460+0000","last_fullsized":"2024-09-20T19:04:35.321460+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:41.859168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:41.859168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":264,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.318944+0000","last_change":"2024-09-20T19:02:27.639446+0000","last_active":"2024-09-20T19:04:35.318944+0000","last_peered":"2024-09-20T19:04:35.318944+0000","last_clean":"2024-09-20T19:04:35.318944+0000","last_became_active":"2024-09-20T18:59:05.085636+0000","last_became_peered":"2024-09-20T18:59:05.085636+0000","last_unstale":"2024-09-20T19:04:35.318944+0000","last_undegraded":"2024-09-20T19:04:35.318944+0000","last_fullsized":"2024-09-20T19:04:35.318944+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:27.639384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:27.639384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":696,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.322284+0000","last_change":"2024-09-20T19:02:23.886386+0000","last_active":"2024-09-20T19:04:35.322284+0000","last_peered":"2024-09-20T19:04:35.322284+0000","last_clean":"2024-09-20T19:04:35.322284+0000","last_became_active":"2024-09-20T18:59:05.085131+0000","last_became_peered":"2024-09-20T18:59:05.085131+0000","last_unstale":"2024-09-20T19:04:35.322284+0000","last_undegraded":"2024-09-20T19:04:35.322284+0000","last_fullsized":"2024-09-20T19:04:35.322284+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:23.886308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:23.886308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"378'9796","reported_seq":9979,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:41.706059+0000","last_change":"2024-09-20T19:03:15.459244+0000","last_active":"2024-09-20T19:04:41.706059+0000","last_peered":"2024-09-20T19:04:41.706059+0000","last_clean":"2024-09-20T19:04:41.706059+0000","last_became_active":"2024-09-20T19:02:11.827305+0000","last_became_peered":"2024-09-20T19:02:11.827305+0000","last_unstale":"2024-09-20T19:04:41.706059+0000","last_undegraded":"2024-09-20T19:04:41.706059+0000","last_fullsized":"2024-09-20T19:04:41.706059+0000","mapping_epoch":325,"log_start":"342'3802","ondisk_log_start":"342'3802","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"350'5605","last_scrub_stamp":"2024-09-20T19:03:15.459174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:03:15.459174+0000","objects_scrubbed":5605,"log_size":5994,"log_dups_size":0,"ondisk_log_size":5994,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":598,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":299499520,"num_objects":4570,"num_object_clones":0,"num_object_copies":13710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9796,"num_write_kb":459712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"353'41","reported_seq":965,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:36.176399+0000","last_change":"2024-09-20T19:03:17.024385+0000","last_active":"2024-09-20T19:04:36.176399+0000","last_peered":"2024-09-20T19:04:36.176399+0000","last_clean":"2024-09-20T19:04:36.176399+0000","last_became_active":"2024-09-20T19:03:16.976916+0000","last_became_peered":"2024-09-20T19:03:16.976916+0000","last_unstale":"2024-09-20T19:04:36.176399+0000","last_undegraded":"2024-09-20T19:04:36.176399+0000","last_fullsized":"2024-09-20T19:04:36.176399+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'40","last_scrub_stamp":"2024-09-20T19:02:36.900916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:36.900916+0000","objects_scrubbed":10,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.313682+0000","last_change":"2024-09-20T19:04:20.899630+0000","last_active":"2024-09-20T19:04:35.313682+0000","last_peered":"2024-09-20T19:04:35.313682+0000","last_clean":"2024-09-20T19:04:35.313682+0000","last_became_active":"2024-09-20T19:04:20.899352+0000","last_became_peered":"2024-09-20T19:04:20.899352+0000","last_unstale":"2024-09-20T19:04:35.313682+0000","last_undegraded":"2024-09-20T19:04:35.313682+0000","last_fullsized":"2024-09-20T19:04:35.313682+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:00:31.630361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:00:31.630361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":742,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.326593+0000","last_change":"2024-09-20T19:03:11.910684+0000","last_active":"2024-09-20T19:04:35.326593+0000","last_peered":"2024-09-20T19:04:35.326593+0000","last_clean":"2024-09-20T19:04:35.326593+0000","last_became_active":"2024-09-20T18:59:05.086786+0000","last_became_peered":"2024-09-20T18:59:05.086786+0000","last_unstale":"2024-09-20T19:04:35.326593+0000","last_undegraded":"2024-09-20T19:04:35.326593+0000","last_fullsized":"2024-09-20T19:04:35.326593+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-20T19:03:11.909394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:03:11.909394+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0.0012781159999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":712,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.324502+0000","last_change":"2024-09-20T19:04:20.907534+0000","last_active":"2024-09-20T19:04:35.324502+0000","last_peered":"2024-09-20T19:04:35.324502+0000","last_clean":"2024-09-20T19:04:35.324502+0000","last_became_active":"2024-09-20T19:04:20.907038+0000","last_became_peered":"2024-09-20T19:04:20.907038+0000","last_unstale":"2024-09-20T19:04:35.324502+0000","last_undegraded":"2024-09-20T19:04:35.324502+0000","last_fullsized":"2024-09-20T19:04:35.324502+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:15.034340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:03:15.034340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":682,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.332773+0000","last_change":"2024-09-20T19:04:20.896616+0000","last_active":"2024-09-20T19:04:35.332773+0000","last_peered":"2024-09-20T19:04:35.332773+0000","last_clean":"2024-09-20T19:04:35.332773+0000","last_became_active":"2024-09-20T19:04:20.896281+0000","last_became_peered":"2024-09-20T19:04:20.896281+0000","last_unstale":"2024-09-20T19:04:35.332773+0000","last_undegraded":"2024-09-20T19:04:35.332773+0000","last_fullsized":"2024-09-20T19:04:35.332773+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:04.526683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:04.526683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"378'10057","reported_seq":10176,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:41.524576+0000","last_change":"2024-09-20T19:02:11.825132+0000","last_active":"2024-09-20T19:04:41.524576+0000","last_peered":"2024-09-20T19:04:41.524576+0000","last_clean":"2024-09-20T19:04:41.524576+0000","last_became_active":"2024-09-20T19:02:11.824846+0000","last_became_peered":"2024-09-20T19:02:11.824846+0000","last_unstale":"2024-09-20T19:04:41.524576+0000","last_undegraded":"2024-09-20T19:04:41.524576+0000","last_fullsized":"2024-09-20T19:04:41.524576+0000","mapping_epoch":325,"log_start":"345'4327","ondisk_log_start":"345'4327","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":5730,"log_dups_size":0,"ondisk_log_size":5730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":306905088,"num_objects":4683,"num_object_clones":0,"num_object_copies":14049,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4683,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10057,"num_write_kb":471680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"273'17","reported_seq":722,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.322184+0000","last_change":"2024-09-20T19:02:24.853473+0000","last_active":"2024-09-20T19:04:35.322184+0000","last_peered":"2024-09-20T19:04:35.322184+0000","last_clean":"2024-09-20T19:04:35.322184+0000","last_became_active":"2024-09-20T18:49:47.950904+0000","last_became_peered":"2024-09-20T18:49:47.950904+0000","last_unstale":"2024-09-20T19:04:35.322184+0000","last_undegraded":"2024-09-20T19:04:35.322184+0000","last_fullsized":"2024-09-20T19:04:35.322184+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'17","last_scrub_stamp":"2024-09-20T19:02:24.853374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:24.853374+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":36,"num_read_kb":25,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":708,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.340252+0000","last_change":"2024-09-20T19:02:29.840761+0000","last_active":"2024-09-20T19:04:35.340252+0000","last_peered":"2024-09-20T19:04:35.340252+0000","last_clean":"2024-09-20T19:04:35.340252+0000","last_became_active":"2024-09-20T18:59:05.079789+0000","last_became_peered":"2024-09-20T18:59:05.079789+0000","last_unstale":"2024-09-20T19:04:35.340252+0000","last_undegraded":"2024-09-20T19:04:35.340252+0000","last_fullsized":"2024-09-20T19:04:35.340252+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:29.840601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:29.840601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":690,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.337591+0000","last_change":"2024-09-20T19:01:58.789304+0000","last_active":"2024-09-20T19:04:35.337591+0000","last_peered":"2024-09-20T19:04:35.337591+0000","last_clean":"2024-09-20T19:04:35.337591+0000","last_became_active":"2024-09-20T18:49:44.937135+0000","last_became_peered":"2024-09-20T18:49:44.937135+0000","last_unstale":"2024-09-20T19:04:35.337591+0000","last_undegraded":"2024-09-20T19:04:35.337591+0000","last_fullsized":"2024-09-20T19:04:35.337591+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:01:58.789251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:01:58.789251+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:48.326383+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00095558800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":664,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.337926+0000","last_change":"2024-09-20T19:01:17.747452+0000","last_active":"2024-09-20T19:04:35.337926+0000","last_peered":"2024-09-20T19:04:35.337926+0000","last_clean":"2024-09-20T19:04:35.337926+0000","last_became_active":"2024-09-20T18:49:49.954424+0000","last_became_peered":"2024-09-20T18:49:49.954424+0000","last_unstale":"2024-09-20T19:04:35.337926+0000","last_undegraded":"2024-09-20T19:04:35.337926+0000","last_fullsized":"2024-09-20T19:04:35.337926+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:17.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:17.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:52.288740+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":676,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.316057+0000","last_change":"2024-09-20T19:01:15.203591+0000","last_active":"2024-09-20T19:04:35.316057+0000","last_peered":"2024-09-20T19:04:35.316057+0000","last_clean":"2024-09-20T19:04:35.316057+0000","last_became_active":"2024-09-20T18:59:05.081459+0000","last_became_peered":"2024-09-20T18:59:05.081459+0000","last_unstale":"2024-09-20T19:04:35.316057+0000","last_undegraded":"2024-09-20T19:04:35.316057+0000","last_fullsized":"2024-09-20T19:04:35.316057+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:15.203529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:15.203529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"378'9881","reported_seq":10000,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:42.457206+0000","last_change":"2024-09-20T19:02:11.828477+0000","last_active":"2024-09-20T19:04:42.457206+0000","last_peered":"2024-09-20T19:04:42.457206+0000","last_clean":"2024-09-20T19:04:42.457206+0000","last_became_active":"2024-09-20T19:02:11.827996+0000","last_became_peered":"2024-09-20T19:02:11.827996+0000","last_unstale":"2024-09-20T19:04:42.457206+0000","last_undegraded":"2024-09-20T19:04:42.457206+0000","last_fullsized":"2024-09-20T19:04:42.457206+0000","mapping_epoch":325,"log_start":"346'4484","ondisk_log_start":"346'4484","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":5397,"log_dups_size":0,"ondisk_log_size":5397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":297861120,"num_objects":4545,"num_object_clones":0,"num_object_copies":13635,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9881,"num_write_kb":461632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"273'24","reported_seq":726,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.327319+0000","last_change":"2024-09-20T18:59:06.022021+0000","last_active":"2024-09-20T19:04:35.327319+0000","last_peered":"2024-09-20T19:04:35.327319+0000","last_clean":"2024-09-20T19:04:35.327319+0000","last_became_active":"2024-09-20T18:49:47.949230+0000","last_became_peered":"2024-09-20T18:49:47.949230+0000","last_unstale":"2024-09-20T19:04:35.327319+0000","last_undegraded":"2024-09-20T19:04:35.327319+0000","last_fullsized":"2024-09-20T19:04:35.327319+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:59:06.021962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:59:06.021962+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":712,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.320211+0000","last_change":"2024-09-20T19:04:20.907250+0000","last_active":"2024-09-20T19:04:35.320211+0000","last_peered":"2024-09-20T19:04:35.320211+0000","last_clean":"2024-09-20T19:04:35.320211+0000","last_became_active":"2024-09-20T19:04:20.906969+0000","last_became_peered":"2024-09-20T19:04:20.906969+0000","last_unstale":"2024-09-20T19:04:35.320211+0000","last_undegraded":"2024-09-20T19:04:35.320211+0000","last_fullsized":"2024-09-20T19:04:35.320211+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:24.984022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:24.984022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":1586,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:39.425719+0000","last_change":"2024-09-20T19:02:02.494344+0000","last_active":"2024-09-20T19:04:39.425719+0000","last_peered":"2024-09-20T19:04:39.425719+0000","last_clean":"2024-09-20T19:04:39.425719+0000","last_became_active":"2024-09-20T18:49:44.936819+0000","last_became_peered":"2024-09-20T18:49:44.936819+0000","last_unstale":"2024-09-20T19:04:39.425719+0000","last_undegraded":"2024-09-20T19:04:39.425719+0000","last_fullsized":"2024-09-20T19:04:39.425719+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-20T19:02:02.494291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:02:02.494291+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0010996199999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":888,"num_read_kb":883,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":685,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.328197+0000","last_change":"2024-09-20T19:02:27.785566+0000","last_active":"2024-09-20T19:04:35.328197+0000","last_peered":"2024-09-20T19:04:35.328197+0000","last_clean":"2024-09-20T19:04:35.328197+0000","last_became_active":"2024-09-20T18:49:49.961066+0000","last_became_peered":"2024-09-20T18:49:49.961066+0000","last_unstale":"2024-09-20T19:04:35.328197+0000","last_undegraded":"2024-09-20T19:04:35.328197+0000","last_fullsized":"2024-09-20T19:04:35.328197+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:27.785524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:27.785524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":682,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.321146+0000","last_change":"2024-09-20T19:02:26.861253+0000","last_active":"2024-09-20T19:04:35.321146+0000","last_peered":"2024-09-20T19:04:35.321146+0000","last_clean":"2024-09-20T19:04:35.321146+0000","last_became_active":"2024-09-20T18:49:51.970343+0000","last_became_peered":"2024-09-20T18:49:51.970343+0000","last_unstale":"2024-09-20T19:04:35.321146+0000","last_undegraded":"2024-09-20T19:04:35.321146+0000","last_fullsized":"2024-09-20T19:04:35.321146+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:26.861185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:26.861185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"378'9958","reported_seq":10077,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:41.749258+0000","last_change":"2024-09-20T19:02:11.828065+0000","last_active":"2024-09-20T19:04:41.749258+0000","last_peered":"2024-09-20T19:04:41.749258+0000","last_clean":"2024-09-20T19:04:41.749258+0000","last_became_active":"2024-09-20T19:02:11.827628+0000","last_became_peered":"2024-09-20T19:02:11.827628+0000","last_unstale":"2024-09-20T19:04:41.749258+0000","last_undegraded":"2024-09-20T19:04:41.749258+0000","last_fullsized":"2024-09-20T19:04:41.749258+0000","mapping_epoch":325,"log_start":"342'3927","ondisk_log_start":"342'3927","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":6031,"log_dups_size":0,"ondisk_log_size":6031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":301465600,"num_objects":4600,"num_object_clones":0,"num_object_copies":13800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4600,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9958,"num_write_kb":465856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"375'34","reported_seq":423,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:36.183150+0000","last_change":"2024-09-20T19:04:20.988428+0000","last_active":"2024-09-20T19:04:36.183150+0000","last_peered":"2024-09-20T19:04:36.183150+0000","last_clean":"2024-09-20T19:04:36.183150+0000","last_became_active":"2024-09-20T19:04:20.928727+0000","last_became_peered":"2024-09-20T19:04:20.928727+0000","last_unstale":"2024-09-20T19:04:36.183150+0000","last_undegraded":"2024-09-20T19:04:36.183150+0000","last_fullsized":"2024-09-20T19:04:36.183150+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"308'32","last_scrub_stamp":"2024-09-20T19:02:25.699337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:25.699337+0000","objects_scrubbed":8,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":462,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":700,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.329181+0000","last_change":"2024-09-20T19:02:31.965132+0000","last_active":"2024-09-20T19:04:35.329181+0000","last_peered":"2024-09-20T19:04:35.329181+0000","last_clean":"2024-09-20T19:04:35.329181+0000","last_became_active":"2024-09-20T18:59:05.079111+0000","last_became_peered":"2024-09-20T18:59:05.079111+0000","last_unstale":"2024-09-20T19:04:35.329181+0000","last_undegraded":"2024-09-20T19:04:35.329181+0000","last_fullsized":"2024-09-20T19:04:35.329181+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:31.965066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:31.965066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":755,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.316888+0000","last_change":"2024-09-20T19:03:14.368338+0000","last_active":"2024-09-20T19:04:35.316888+0000","last_peered":"2024-09-20T19:04:35.316888+0000","last_clean":"2024-09-20T19:04:35.316888+0000","last_became_active":"2024-09-20T18:59:05.086103+0000","last_became_peered":"2024-09-20T18:59:05.086103+0000","last_unstale":"2024-09-20T19:04:35.316888+0000","last_undegraded":"2024-09-20T19:04:35.316888+0000","last_fullsized":"2024-09-20T19:04:35.316888+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:03:14.368279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:03:14.368279+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":671,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.533709+0000","last_change":"2024-09-20T19:04:35.533709+0000","last_active":"2024-09-20T19:04:35.533709+0000","last_peered":"2024-09-20T19:04:35.533709+0000","last_clean":"2024-09-20T19:04:35.533709+0000","last_became_active":"2024-09-20T18:49:49.956128+0000","last_became_peered":"2024-09-20T18:49:49.956128+0000","last_unstale":"2024-09-20T19:04:35.533709+0000","last_undegraded":"2024-09-20T19:04:35.533709+0000","last_fullsized":"2024-09-20T19:04:35.533709+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:35.533633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:35.533633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:59.293116+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":706,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.328003+0000","last_change":"2024-09-20T19:04:20.899776+0000","last_active":"2024-09-20T19:04:35.328003+0000","last_peered":"2024-09-20T19:04:35.328003+0000","last_clean":"2024-09-20T19:04:35.328003+0000","last_became_active":"2024-09-20T19:04:20.899321+0000","last_became_peered":"2024-09-20T19:04:20.899321+0000","last_unstale":"2024-09-20T19:04:35.328003+0000","last_undegraded":"2024-09-20T19:04:35.328003+0000","last_fullsized":"2024-09-20T19:04:35.328003+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:30.557368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:30.557368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.0","version":"378'10042","reported_seq":14654,"reported_epoch":378,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-20T19:04:41.743869+0000","last_change":"2024-09-20T19:04:21.081332+0000","last_active":"2024-09-20T19:04:41.743869+0000","last_peered":"2024-09-20T19:04:41.743869+0000","last_clean":"2024-09-20T19:03:13.922116+0000","last_became_active":"2024-09-20T19:04:21.039020+0000","last_became_peered":"2024-09-20T19:04:21.039020+0000","last_unstale":"2024-09-20T19:04:41.743869+0000","last_undegraded":"2024-09-20T19:04:41.743869+0000","last_fullsized":"2024-09-20T19:04:20.870176+0000","mapping_epoch":374,"log_start":"343'4039","ondisk_log_start":"343'4039","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":6003,"log_dups_size":0,"ondisk_log_size":6003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":300023808,"num_objects":4578,"num_object_clones":0,"num_object_copies":13734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":821,"num_objects_unfound":0,"num_objects_dirty":4578,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10042,"num_write_kb":467840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2243,"num_bytes_recovered":146931712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1],"avail_no_missing":["6","1"],"object_location_counts":[{"shards":"1,6","objects":3757},{"shards":"1,6,7","objects":821}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"274'19","reported_seq":30,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.314450+0000","last_change":"2024-09-20T19:04:21.040511+0000","last_active":"2024-09-20T19:04:35.314450+0000","last_peered":"2024-09-20T19:04:35.314450+0000","last_clean":"2024-09-20T19:04:35.314450+0000","last_became_active":"2024-09-20T19:04:20.897175+0000","last_became_peered":"2024-09-20T19:04:20.897175+0000","last_unstale":"2024-09-20T19:04:35.314450+0000","last_undegraded":"2024-09-20T19:04:35.314450+0000","last_fullsized":"2024-09-20T19:04:35.314450+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:58:24.616580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:24.616580+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.313592+0000","last_change":"2024-09-20T19:04:20.887455+0000","last_active":"2024-09-20T19:04:35.313592+0000","last_peered":"2024-09-20T19:04:35.313592+0000","last_clean":"2024-09-20T19:04:35.313592+0000","last_became_active":"2024-09-20T19:04:20.887102+0000","last_became_peered":"2024-09-20T19:04:20.887102+0000","last_unstale":"2024-09-20T19:04:35.313592+0000","last_undegraded":"2024-09-20T19:04:35.313592+0000","last_fullsized":"2024-09-20T19:04:35.313592+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.693993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.693993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":724,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.327869+0000","last_change":"2024-09-20T19:04:20.899830+0000","last_active":"2024-09-20T19:04:35.327869+0000","last_peered":"2024-09-20T19:04:35.327869+0000","last_clean":"2024-09-20T19:04:35.327869+0000","last_became_active":"2024-09-20T19:04:20.899331+0000","last_became_peered":"2024-09-20T19:04:20.899331+0000","last_unstale":"2024-09-20T19:04:35.327869+0000","last_undegraded":"2024-09-20T19:04:35.327869+0000","last_fullsized":"2024-09-20T19:04:35.327869+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:38.463404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:38.463404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":687,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.529672+0000","last_change":"2024-09-20T19:04:35.529672+0000","last_active":"2024-09-20T19:04:35.529672+0000","last_peered":"2024-09-20T19:04:35.529672+0000","last_clean":"2024-09-20T19:04:35.529672+0000","last_became_active":"2024-09-20T18:59:05.079897+0000","last_became_peered":"2024-09-20T18:59:05.079897+0000","last_unstale":"2024-09-20T19:04:35.529672+0000","last_undegraded":"2024-09-20T19:04:35.529672+0000","last_fullsized":"2024-09-20T19:04:35.529672+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:35.529584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:04:35.529584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:57.166916+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"378'10150","reported_seq":10345,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:41.463497+0000","last_change":"2024-09-20T19:03:38.975534+0000","last_active":"2024-09-20T19:04:41.463497+0000","last_peered":"2024-09-20T19:04:41.463497+0000","last_clean":"2024-09-20T19:04:41.463497+0000","last_became_active":"2024-09-20T19:02:11.825011+0000","last_became_peered":"2024-09-20T19:02:11.825011+0000","last_unstale":"2024-09-20T19:04:41.463497+0000","last_undegraded":"2024-09-20T19:04:41.463497+0000","last_fullsized":"2024-09-20T19:04:41.463497+0000","mapping_epoch":325,"log_start":"342'3628","ondisk_log_start":"342'3628","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"358'6900","last_scrub_stamp":"2024-09-20T19:03:38.975306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:03:38.975306+0000","objects_scrubbed":6865,"log_size":6522,"log_dups_size":0,"ondisk_log_size":6522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:54.464015+0000","scrub_duration":1655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":309723136,"num_objects":4726,"num_object_clones":0,"num_object_copies":14178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10150,"num_write_kb":476032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"274'31","reported_seq":36,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.314747+0000","last_change":"2024-09-20T19:04:20.949305+0000","last_active":"2024-09-20T19:04:35.314747+0000","last_peered":"2024-09-20T19:04:35.314747+0000","last_clean":"2024-09-20T19:04:35.314747+0000","last_became_active":"2024-09-20T19:04:20.883306+0000","last_became_peered":"2024-09-20T19:04:20.883306+0000","last_unstale":"2024-09-20T19:04:35.314747+0000","last_undegraded":"2024-09-20T19:04:35.314747+0000","last_fullsized":"2024-09-20T19:04:35.314747+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'31","last_scrub_stamp":"2024-09-20T19:03:12.196851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:12.196851+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":49,"num_read_kb":39,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":281,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.319021+0000","last_change":"2024-09-20T19:04:20.899712+0000","last_active":"2024-09-20T19:04:35.319021+0000","last_peered":"2024-09-20T19:04:35.319021+0000","last_clean":"2024-09-20T19:04:35.319021+0000","last_became_active":"2024-09-20T19:04:20.899415+0000","last_became_peered":"2024-09-20T19:04:20.899415+0000","last_unstale":"2024-09-20T19:04:35.319021+0000","last_undegraded":"2024-09-20T19:04:35.319021+0000","last_fullsized":"2024-09-20T19:04:35.319021+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:42.479489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:42.479489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":702,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.321558+0000","last_change":"2024-09-20T19:04:20.892646+0000","last_active":"2024-09-20T19:04:35.321558+0000","last_peered":"2024-09-20T19:04:35.321558+0000","last_clean":"2024-09-20T19:04:35.321558+0000","last_became_active":"2024-09-20T19:04:20.891772+0000","last_became_peered":"2024-09-20T19:04:20.891772+0000","last_unstale":"2024-09-20T19:04:35.321558+0000","last_undegraded":"2024-09-20T19:04:35.321558+0000","last_fullsized":"2024-09-20T19:04:35.321558+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:32.856856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:32.856856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":701,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.340502+0000","last_change":"2024-09-20T19:04:20.897937+0000","last_active":"2024-09-20T19:04:35.340502+0000","last_peered":"2024-09-20T19:04:35.340502+0000","last_clean":"2024-09-20T19:04:35.340502+0000","last_became_active":"2024-09-20T19:04:20.897569+0000","last_became_peered":"2024-09-20T19:04:20.897569+0000","last_unstale":"2024-09-20T19:04:35.340502+0000","last_undegraded":"2024-09-20T19:04:35.340502+0000","last_fullsized":"2024-09-20T19:04:35.340502+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:31.687188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:31.687188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:24.385920+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2","version":"378'10104","reported_seq":11570,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:41.931086+0000","last_change":"2024-09-20T19:04:21.974196+0000","last_active":"2024-09-20T19:04:41.931086+0000","last_peered":"2024-09-20T19:04:41.931086+0000","last_clean":"2024-09-20T19:04:41.931086+0000","last_became_active":"2024-09-20T19:04:21.956633+0000","last_became_peered":"2024-09-20T19:04:21.956633+0000","last_unstale":"2024-09-20T19:04:41.931086+0000","last_undegraded":"2024-09-20T19:04:41.931086+0000","last_fullsized":"2024-09-20T19:04:41.931086+0000","mapping_epoch":375,"log_start":"346'4982","ondisk_log_start":"346'4982","created":325,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":5122,"log_dups_size":0,"ondisk_log_size":5122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":299106304,"num_objects":4564,"num_object_clones":0,"num_object_copies":13692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4564,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10104,"num_write_kb":469376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":665,"num_bytes_recovered":43581440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"273'20","reported_seq":30,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.314043+0000","last_change":"2024-09-20T19:04:20.945143+0000","last_active":"2024-09-20T19:04:35.314043+0000","last_peered":"2024-09-20T19:04:35.314043+0000","last_clean":"2024-09-20T19:04:35.314043+0000","last_became_active":"2024-09-20T19:04:20.896483+0000","last_became_peered":"2024-09-20T19:04:20.896483+0000","last_unstale":"2024-09-20T19:04:35.314043+0000","last_undegraded":"2024-09-20T19:04:35.314043+0000","last_fullsized":"2024-09-20T19:04:35.314043+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'20","last_scrub_stamp":"2024-09-20T19:03:04.282800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:04.282800+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":710,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.333654+0000","last_change":"2024-09-20T19:04:20.893263+0000","last_active":"2024-09-20T19:04:35.333654+0000","last_peered":"2024-09-20T19:04:35.333654+0000","last_clean":"2024-09-20T19:04:35.333654+0000","last_became_active":"2024-09-20T19:04:20.892951+0000","last_became_peered":"2024-09-20T19:04:20.892951+0000","last_unstale":"2024-09-20T19:04:35.333654+0000","last_undegraded":"2024-09-20T19:04:35.333654+0000","last_fullsized":"2024-09-20T19:04:35.333654+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:34.985163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:34.985163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":876,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:36.177564+0000","last_change":"2024-09-20T19:02:02.813926+0000","last_active":"2024-09-20T19:04:36.177564+0000","last_peered":"2024-09-20T19:04:36.177564+0000","last_clean":"2024-09-20T19:04:36.177564+0000","last_became_active":"2024-09-20T18:49:49.957005+0000","last_became_peered":"2024-09-20T18:49:49.957005+0000","last_unstale":"2024-09-20T19:04:36.177564+0000","last_undegraded":"2024-09-20T19:04:36.177564+0000","last_fullsized":"2024-09-20T19:04:36.177564+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T19:02:02.813865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:02.813865+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:59.309926+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":18,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.314507+0000","last_change":"2024-09-20T19:04:20.885294+0000","last_active":"2024-09-20T19:04:35.314507+0000","last_peered":"2024-09-20T19:04:35.314507+0000","last_clean":"2024-09-20T19:04:35.314507+0000","last_became_active":"2024-09-20T19:04:20.884994+0000","last_became_peered":"2024-09-20T19:04:20.884994+0000","last_unstale":"2024-09-20T19:04:35.314507+0000","last_undegraded":"2024-09-20T19:04:35.314507+0000","last_fullsized":"2024-09-20T19:04:35.314507+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:11.207994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:03:11.207994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.5","version":"378'9926","reported_seq":10120,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:41.441241+0000","last_change":"2024-09-20T19:04:04.390108+0000","last_active":"2024-09-20T19:04:41.441241+0000","last_peered":"2024-09-20T19:04:41.441241+0000","last_clean":"2024-09-20T19:04:41.441241+0000","last_became_active":"2024-09-20T19:02:11.828162+0000","last_became_peered":"2024-09-20T19:02:11.828162+0000","last_unstale":"2024-09-20T19:04:41.441241+0000","last_undegraded":"2024-09-20T19:04:41.441241+0000","last_fullsized":"2024-09-20T19:04:41.441241+0000","mapping_epoch":325,"log_start":"341'3368","ondisk_log_start":"341'3368","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"367'7875","last_scrub_stamp":"2024-09-20T19:04:04.389838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:04:04.389838+0000","objects_scrubbed":6698,"log_size":6558,"log_dups_size":0,"ondisk_log_size":6558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:27.588286+0000","scrub_duration":2035,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":300941312,"num_objects":4592,"num_object_clones":0,"num_object_copies":13776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9926,"num_write_kb":464576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"274'15","reported_seq":752,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.319374+0000","last_change":"2024-09-20T19:04:20.978525+0000","last_active":"2024-09-20T19:04:35.319374+0000","last_peered":"2024-09-20T19:04:35.319374+0000","last_clean":"2024-09-20T19:04:35.319374+0000","last_became_active":"2024-09-20T19:04:20.906834+0000","last_became_peered":"2024-09-20T19:04:20.906834+0000","last_unstale":"2024-09-20T19:04:35.319374+0000","last_undegraded":"2024-09-20T19:04:35.319374+0000","last_fullsized":"2024-09-20T19:04:35.319374+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'15","last_scrub_stamp":"2024-09-20T19:01:27.802100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:27.802100+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":238,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.337483+0000","last_change":"2024-09-20T19:04:16.384877+0000","last_active":"2024-09-20T19:04:35.337483+0000","last_peered":"2024-09-20T19:04:35.337483+0000","last_clean":"2024-09-20T19:04:35.337483+0000","last_became_active":"2024-09-20T19:00:01.190875+0000","last_became_peered":"2024-09-20T19:00:01.190875+0000","last_unstale":"2024-09-20T19:04:35.337483+0000","last_undegraded":"2024-09-20T19:04:35.337483+0000","last_fullsized":"2024-09-20T19:04:35.337483+0000","mapping_epoch":277,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":278,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:16.384809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:04:16.384809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:44.880751+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":713,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.338113+0000","last_change":"2024-09-20T19:04:20.889614+0000","last_active":"2024-09-20T19:04:35.338113+0000","last_peered":"2024-09-20T19:04:35.338113+0000","last_clean":"2024-09-20T19:04:35.338113+0000","last_became_active":"2024-09-20T19:04:20.889273+0000","last_became_peered":"2024-09-20T19:04:20.889273+0000","last_unstale":"2024-09-20T19:04:35.338113+0000","last_undegraded":"2024-09-20T19:04:35.338113+0000","last_fullsized":"2024-09-20T19:04:35.338113+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:03.389611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:03.389611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:24.199614+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":674,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.319051+0000","last_change":"2024-09-20T19:02:31.120809+0000","last_active":"2024-09-20T19:04:35.319051+0000","last_peered":"2024-09-20T19:04:35.319051+0000","last_clean":"2024-09-20T19:04:35.319051+0000","last_became_active":"2024-09-20T18:49:51.964098+0000","last_became_peered":"2024-09-20T18:49:51.964098+0000","last_unstale":"2024-09-20T19:04:35.319051+0000","last_undegraded":"2024-09-20T19:04:35.319051+0000","last_fullsized":"2024-09-20T19:04:35.319051+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:31.120623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:31.120623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"378'9908","reported_seq":10047,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:42.498133+0000","last_change":"2024-09-20T19:04:21.973100+0000","last_active":"2024-09-20T19:04:42.498133+0000","last_peered":"2024-09-20T19:04:42.498133+0000","last_clean":"2024-09-20T19:04:42.498133+0000","last_became_active":"2024-09-20T19:04:21.960863+0000","last_became_peered":"2024-09-20T19:04:21.960863+0000","last_unstale":"2024-09-20T19:04:42.498133+0000","last_undegraded":"2024-09-20T19:04:42.498133+0000","last_fullsized":"2024-09-20T19:04:42.498133+0000","mapping_epoch":375,"log_start":"346'4497","ondisk_log_start":"346'4497","created":325,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":5411,"log_dups_size":0,"ondisk_log_size":5411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":293076992,"num_objects":4472,"num_object_clones":0,"num_object_copies":13416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9908,"num_write_kb":460160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"274'35","reported_seq":308,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.318848+0000","last_change":"2024-09-20T19:03:09.841596+0000","last_active":"2024-09-20T19:04:35.318848+0000","last_peered":"2024-09-20T19:04:35.318848+0000","last_clean":"2024-09-20T19:04:35.318848+0000","last_became_active":"2024-09-20T18:59:05.087391+0000","last_became_peered":"2024-09-20T18:59:05.087391+0000","last_unstale":"2024-09-20T19:04:35.318848+0000","last_undegraded":"2024-09-20T19:04:35.318848+0000","last_fullsized":"2024-09-20T19:04:35.318848+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'35","last_scrub_stamp":"2024-09-20T19:03:09.841361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:09.841361+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":700,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.316383+0000","last_change":"2024-09-20T19:02:05.440745+0000","last_active":"2024-09-20T19:04:35.316383+0000","last_peered":"2024-09-20T19:04:35.316383+0000","last_clean":"2024-09-20T19:04:35.316383+0000","last_became_active":"2024-09-20T18:59:05.086670+0000","last_became_peered":"2024-09-20T18:59:05.086670+0000","last_unstale":"2024-09-20T19:04:35.316383+0000","last_undegraded":"2024-09-20T19:04:35.316383+0000","last_fullsized":"2024-09-20T19:04:35.316383+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:05.440677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:05.440677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":688,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.338509+0000","last_change":"2024-09-20T19:02:42.745448+0000","last_active":"2024-09-20T19:04:35.338509+0000","last_peered":"2024-09-20T19:04:35.338509+0000","last_clean":"2024-09-20T19:04:35.338509+0000","last_became_active":"2024-09-20T18:49:49.957558+0000","last_became_peered":"2024-09-20T18:49:49.957558+0000","last_unstale":"2024-09-20T19:04:35.338509+0000","last_undegraded":"2024-09-20T19:04:35.338509+0000","last_fullsized":"2024-09-20T19:04:35.338509+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:42.745372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:42.745372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:01.321904+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":666,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.316167+0000","last_change":"2024-09-20T18:53:47.370152+0000","last_active":"2024-09-20T19:04:35.316167+0000","last_peered":"2024-09-20T19:04:35.316167+0000","last_clean":"2024-09-20T19:04:35.316167+0000","last_became_active":"2024-09-20T18:49:51.970798+0000","last_became_peered":"2024-09-20T18:49:51.970798+0000","last_unstale":"2024-09-20T19:04:35.316167+0000","last_undegraded":"2024-09-20T19:04:35.316167+0000","last_fullsized":"2024-09-20T19:04:35.316167+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:47.370108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:47.370108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:22.697944+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.7","version":"378'10099","reported_seq":10218,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:41.710667+0000","last_change":"2024-09-20T19:02:11.827681+0000","last_active":"2024-09-20T19:04:41.710667+0000","last_peered":"2024-09-20T19:04:41.710667+0000","last_clean":"2024-09-20T19:04:41.710667+0000","last_became_active":"2024-09-20T19:02:11.827360+0000","last_became_peered":"2024-09-20T19:02:11.827360+0000","last_unstale":"2024-09-20T19:04:41.710667+0000","last_undegraded":"2024-09-20T19:04:41.710667+0000","last_fullsized":"2024-09-20T19:04:41.710667+0000","mapping_epoch":325,"log_start":"344'4107","ondisk_log_start":"344'4107","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":5992,"log_dups_size":0,"ondisk_log_size":5992,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":303104000,"num_objects":4625,"num_object_clones":0,"num_object_copies":13875,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4625,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10099,"num_write_kb":471168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"273'18","reported_seq":30,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.313690+0000","last_change":"2024-09-20T19:04:20.936465+0000","last_active":"2024-09-20T19:04:35.313690+0000","last_peered":"2024-09-20T19:04:35.313690+0000","last_clean":"2024-09-20T19:04:35.313690+0000","last_became_active":"2024-09-20T19:04:20.888905+0000","last_became_peered":"2024-09-20T19:04:20.888905+0000","last_unstale":"2024-09-20T19:04:35.313690+0000","last_undegraded":"2024-09-20T19:04:35.313690+0000","last_fullsized":"2024-09-20T19:04:35.313690+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'18","last_scrub_stamp":"2024-09-20T19:03:13.201592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:13.201592+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":711,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.316735+0000","last_change":"2024-09-20T19:04:20.892754+0000","last_active":"2024-09-20T19:04:35.316735+0000","last_peered":"2024-09-20T19:04:35.316735+0000","last_clean":"2024-09-20T19:04:35.316735+0000","last_became_active":"2024-09-20T19:04:20.892429+0000","last_became_peered":"2024-09-20T19:04:20.892429+0000","last_unstale":"2024-09-20T19:04:35.316735+0000","last_undegraded":"2024-09-20T19:04:35.316735+0000","last_fullsized":"2024-09-20T19:04:35.316735+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:00:34.940850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:00:34.940850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":680,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.338330+0000","last_change":"2024-09-20T19:01:53.734774+0000","last_active":"2024-09-20T19:04:35.338330+0000","last_peered":"2024-09-20T19:04:35.338330+0000","last_clean":"2024-09-20T19:04:35.338330+0000","last_became_active":"2024-09-20T18:49:49.954944+0000","last_became_peered":"2024-09-20T18:49:49.954944+0000","last_unstale":"2024-09-20T19:04:35.338330+0000","last_undegraded":"2024-09-20T19:04:35.338330+0000","last_fullsized":"2024-09-20T19:04:35.338330+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:53.734726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:53.734726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:45.272008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":673,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.328688+0000","last_change":"2024-09-20T19:02:35.968178+0000","last_active":"2024-09-20T19:04:35.328688+0000","last_peered":"2024-09-20T19:04:35.328688+0000","last_clean":"2024-09-20T19:04:35.328688+0000","last_became_active":"2024-09-20T18:49:51.972970+0000","last_became_peered":"2024-09-20T18:49:51.972970+0000","last_unstale":"2024-09-20T19:04:35.328688+0000","last_undegraded":"2024-09-20T19:04:35.328688+0000","last_fullsized":"2024-09-20T19:04:35.328688+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:35.968052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:35.968052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.6","version":"378'10116","reported_seq":14847,"reported_epoch":378,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-20T19:04:41.538401+0000","last_change":"2024-09-20T19:04:21.075147+0000","last_active":"2024-09-20T19:04:41.538401+0000","last_peered":"2024-09-20T19:04:41.538401+0000","last_clean":"2024-09-20T19:03:13.927169+0000","last_became_active":"2024-09-20T19:04:21.034487+0000","last_became_peered":"2024-09-20T19:04:21.034487+0000","last_unstale":"2024-09-20T19:04:41.538401+0000","last_undegraded":"2024-09-20T19:04:41.538401+0000","last_fullsized":"2024-09-20T19:04:20.868185+0000","mapping_epoch":374,"log_start":"345'4362","ondisk_log_start":"345'4362","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":5754,"log_dups_size":0,"ondisk_log_size":5754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":306577408,"num_objects":4678,"num_object_clones":0,"num_object_copies":14034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":809,"num_objects_unfound":0,"num_objects_dirty":4678,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10116,"num_write_kb":473408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2303,"num_bytes_recovered":150732800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0],"avail_no_missing":["5","0"],"object_location_counts":[{"shards":"0,1,5","objects":809},{"shards":"0,5","objects":3869}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"273'27","reported_seq":778,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.325949+0000","last_change":"2024-09-20T19:02:39.709344+0000","last_active":"2024-09-20T19:04:35.325949+0000","last_peered":"2024-09-20T19:04:35.325949+0000","last_clean":"2024-09-20T19:04:35.325949+0000","last_became_active":"2024-09-20T18:59:05.083726+0000","last_became_peered":"2024-09-20T18:59:05.083726+0000","last_unstale":"2024-09-20T19:04:35.325949+0000","last_undegraded":"2024-09-20T19:04:35.325949+0000","last_fullsized":"2024-09-20T19:04:35.325949+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'27","last_scrub_stamp":"2024-09-20T19:02:39.709269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:39.709269+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":743,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.316735+0000","last_change":"2024-09-20T19:04:20.905276+0000","last_active":"2024-09-20T19:04:35.316735+0000","last_peered":"2024-09-20T19:04:35.316735+0000","last_clean":"2024-09-20T19:04:35.316735+0000","last_became_active":"2024-09-20T19:04:20.884854+0000","last_became_peered":"2024-09-20T19:04:20.884854+0000","last_unstale":"2024-09-20T19:04:35.316735+0000","last_undegraded":"2024-09-20T19:04:35.316735+0000","last_fullsized":"2024-09-20T19:04:35.316735+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-20T18:58:57.563058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:57.563058+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"375'14","reported_seq":915,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:39.564211+0000","last_change":"2024-09-20T19:04:39.564069+0000","last_active":"2024-09-20T19:04:39.564211+0000","last_peered":"2024-09-20T19:04:39.564211+0000","last_clean":"2024-09-20T19:04:39.564211+0000","last_became_active":"2024-09-20T19:04:20.892956+0000","last_became_peered":"2024-09-20T19:04:20.892956+0000","last_unstale":"2024-09-20T19:04:39.564211+0000","last_undegraded":"2024-09-20T19:04:39.564211+0000","last_fullsized":"2024-09-20T19:04:39.564211+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"375'14","last_scrub_stamp":"2024-09-20T19:04:39.563917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:39.563917+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:49.606578+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":695,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.326231+0000","last_change":"2024-09-20T19:03:06.888620+0000","last_active":"2024-09-20T19:04:35.326231+0000","last_peered":"2024-09-20T19:04:35.326231+0000","last_clean":"2024-09-20T19:04:35.326231+0000","last_became_active":"2024-09-20T18:59:05.087005+0000","last_became_peered":"2024-09-20T18:59:05.087005+0000","last_unstale":"2024-09-20T19:04:35.326231+0000","last_undegraded":"2024-09-20T19:04:35.326231+0000","last_fullsized":"2024-09-20T19:04:35.326231+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:06.888563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:03:06.888563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":256,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.323877+0000","last_change":"2024-09-20T19:01:16.542398+0000","last_active":"2024-09-20T19:04:35.323877+0000","last_peered":"2024-09-20T19:04:35.323877+0000","last_clean":"2024-09-20T19:04:35.323877+0000","last_became_active":"2024-09-20T18:59:05.085427+0000","last_became_peered":"2024-09-20T18:59:05.085427+0000","last_unstale":"2024-09-20T19:04:35.323877+0000","last_undegraded":"2024-09-20T19:04:35.323877+0000","last_fullsized":"2024-09-20T19:04:35.323877+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:16.542335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:16.542335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"273'12","reported_seq":699,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.333217+0000","last_change":"2024-09-20T19:01:12.385223+0000","last_active":"2024-09-20T19:04:35.333217+0000","last_peered":"2024-09-20T19:04:35.333217+0000","last_clean":"2024-09-20T19:04:35.333217+0000","last_became_active":"2024-09-20T18:49:48.500662+0000","last_became_peered":"2024-09-20T18:49:48.500662+0000","last_unstale":"2024-09-20T19:04:35.333217+0000","last_undegraded":"2024-09-20T19:04:35.333217+0000","last_fullsized":"2024-09-20T19:04:35.333217+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'12","last_scrub_stamp":"2024-09-20T19:01:12.385152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:12.385152+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":677,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.327528+0000","last_change":"2024-09-20T19:01:14.466096+0000","last_active":"2024-09-20T19:04:35.327528+0000","last_peered":"2024-09-20T19:04:35.327528+0000","last_clean":"2024-09-20T19:04:35.327528+0000","last_became_active":"2024-09-20T18:49:49.956749+0000","last_became_peered":"2024-09-20T18:49:49.956749+0000","last_unstale":"2024-09-20T19:04:35.327528+0000","last_undegraded":"2024-09-20T19:04:35.327528+0000","last_fullsized":"2024-09-20T19:04:35.327528+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:14.466042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:14.466042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":713,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.315964+0000","last_change":"2024-09-20T19:04:20.914426+0000","last_active":"2024-09-20T19:04:35.315964+0000","last_peered":"2024-09-20T19:04:35.315964+0000","last_clean":"2024-09-20T19:04:35.315964+0000","last_became_active":"2024-09-20T19:04:20.914115+0000","last_became_peered":"2024-09-20T19:04:20.914115+0000","last_unstale":"2024-09-20T19:04:35.315964+0000","last_undegraded":"2024-09-20T19:04:35.315964+0000","last_fullsized":"2024-09-20T19:04:35.315964+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:03.509903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:03.509903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"273'15","reported_seq":699,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.327410+0000","last_change":"2024-09-20T19:01:15.875779+0000","last_active":"2024-09-20T19:04:35.327410+0000","last_peered":"2024-09-20T19:04:35.327410+0000","last_clean":"2024-09-20T19:04:35.327410+0000","last_became_active":"2024-09-20T18:49:48.500803+0000","last_became_peered":"2024-09-20T18:49:48.500803+0000","last_unstale":"2024-09-20T19:04:35.327410+0000","last_undegraded":"2024-09-20T19:04:35.327410+0000","last_fullsized":"2024-09-20T19:04:35.327410+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:01:15.875693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:15.875693+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":717,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.317072+0000","last_change":"2024-09-20T19:04:20.895173+0000","last_active":"2024-09-20T19:04:35.317072+0000","last_peered":"2024-09-20T19:04:35.317072+0000","last_clean":"2024-09-20T19:04:35.317072+0000","last_became_active":"2024-09-20T19:04:20.893250+0000","last_became_peered":"2024-09-20T19:04:20.893250+0000","last_unstale":"2024-09-20T19:04:35.317072+0000","last_undegraded":"2024-09-20T19:04:35.317072+0000","last_fullsized":"2024-09-20T19:04:35.317072+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:59.589891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:59.589891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":677,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.333043+0000","last_change":"2024-09-20T19:01:06.482291+0000","last_active":"2024-09-20T19:04:35.333043+0000","last_peered":"2024-09-20T19:04:35.333043+0000","last_clean":"2024-09-20T19:04:35.333043+0000","last_became_active":"2024-09-20T18:49:49.961947+0000","last_became_peered":"2024-09-20T18:49:49.961947+0000","last_unstale":"2024-09-20T19:04:35.333043+0000","last_undegraded":"2024-09-20T19:04:35.333043+0000","last_fullsized":"2024-09-20T19:04:35.333043+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:06.482212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:06.482212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"66'1","reported_seq":726,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.337578+0000","last_change":"2024-09-20T19:04:20.913449+0000","last_active":"2024-09-20T19:04:35.337578+0000","last_peered":"2024-09-20T19:04:35.337578+0000","last_clean":"2024-09-20T19:04:35.337578+0000","last_became_active":"2024-09-20T19:04:20.892236+0000","last_became_peered":"2024-09-20T19:04:20.892236+0000","last_unstale":"2024-09-20T19:04:35.337578+0000","last_undegraded":"2024-09-20T19:04:35.337578+0000","last_fullsized":"2024-09-20T19:04:35.337578+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T19:03:12.499684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:03:12.499684+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:28.429226+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"274'15","reported_seq":698,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.327014+0000","last_change":"2024-09-20T19:01:36.532486+0000","last_active":"2024-09-20T19:04:35.327014+0000","last_peered":"2024-09-20T19:04:35.327014+0000","last_clean":"2024-09-20T19:04:35.327014+0000","last_became_active":"2024-09-20T18:49:48.501011+0000","last_became_peered":"2024-09-20T18:49:48.501011+0000","last_unstale":"2024-09-20T19:04:35.327014+0000","last_undegraded":"2024-09-20T19:04:35.327014+0000","last_fullsized":"2024-09-20T19:04:35.327014+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'15","last_scrub_stamp":"2024-09-20T19:01:36.532419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:36.532419+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":686,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.324268+0000","last_change":"2024-09-20T19:01:30.864866+0000","last_active":"2024-09-20T19:04:35.324268+0000","last_peered":"2024-09-20T19:04:35.324268+0000","last_clean":"2024-09-20T19:04:35.324268+0000","last_became_active":"2024-09-20T18:49:45.938511+0000","last_became_peered":"2024-09-20T18:49:45.938511+0000","last_unstale":"2024-09-20T19:04:35.324268+0000","last_undegraded":"2024-09-20T19:04:35.324268+0000","last_fullsized":"2024-09-20T19:04:35.324268+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:30.864794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:30.864794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":873,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:41.601208+0000","last_change":"2024-09-20T19:04:41.601208+0000","last_active":"2024-09-20T19:04:41.601208+0000","last_peered":"2024-09-20T19:04:41.601208+0000","last_clean":"2024-09-20T19:04:41.601208+0000","last_became_active":"2024-09-20T18:49:49.960550+0000","last_became_peered":"2024-09-20T18:49:49.960550+0000","last_unstale":"2024-09-20T19:04:41.601208+0000","last_undegraded":"2024-09-20T19:04:41.601208+0000","last_fullsized":"2024-09-20T19:04:41.601208+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T19:04:41.601073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:41.601073+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:03.890101+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":18,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.313941+0000","last_change":"2024-09-20T19:04:20.884397+0000","last_active":"2024-09-20T19:04:35.313941+0000","last_peered":"2024-09-20T19:04:35.313941+0000","last_clean":"2024-09-20T19:04:35.313941+0000","last_became_active":"2024-09-20T19:04:20.884100+0000","last_became_peered":"2024-09-20T19:04:20.884100+0000","last_unstale":"2024-09-20T19:04:35.313941+0000","last_undegraded":"2024-09-20T19:04:35.313941+0000","last_fullsized":"2024-09-20T19:04:35.313941+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:59.497810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:59:59.497810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"274'20","reported_seq":714,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.334519+0000","last_change":"2024-09-20T18:57:20.701592+0000","last_active":"2024-09-20T19:04:35.334519+0000","last_peered":"2024-09-20T19:04:35.334519+0000","last_clean":"2024-09-20T19:04:35.334519+0000","last_became_active":"2024-09-20T18:49:58.997494+0000","last_became_peered":"2024-09-20T18:49:58.997494+0000","last_unstale":"2024-09-20T19:04:35.334519+0000","last_undegraded":"2024-09-20T19:04:35.334519+0000","last_fullsized":"2024-09-20T19:04:35.334519+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:57:20.701543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:20.701543+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:20.547444+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":703,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.316361+0000","last_change":"2024-09-20T19:04:20.884940+0000","last_active":"2024-09-20T19:04:35.316361+0000","last_peered":"2024-09-20T19:04:35.316361+0000","last_clean":"2024-09-20T19:04:35.316361+0000","last_became_active":"2024-09-20T19:04:20.884582+0000","last_became_peered":"2024-09-20T19:04:20.884582+0000","last_unstale":"2024-09-20T19:04:35.316361+0000","last_undegraded":"2024-09-20T19:04:35.316361+0000","last_fullsized":"2024-09-20T19:04:35.316361+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.632380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.632380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:32.500437+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":865,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:36.178176+0000","last_change":"2024-09-20T19:01:10.601374+0000","last_active":"2024-09-20T19:04:36.178176+0000","last_peered":"2024-09-20T19:04:36.178176+0000","last_clean":"2024-09-20T19:04:36.178176+0000","last_became_active":"2024-09-20T18:49:49.962243+0000","last_became_peered":"2024-09-20T18:49:49.962243+0000","last_unstale":"2024-09-20T19:04:36.178176+0000","last_undegraded":"2024-09-20T19:04:36.178176+0000","last_fullsized":"2024-09-20T19:04:36.178176+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T19:01:10.601300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:10.601300+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":682,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.327016+0000","last_change":"2024-09-20T19:01:03.821977+0000","last_active":"2024-09-20T19:04:35.327016+0000","last_peered":"2024-09-20T19:04:35.327016+0000","last_clean":"2024-09-20T19:04:35.327016+0000","last_became_active":"2024-09-20T18:49:51.969093+0000","last_became_peered":"2024-09-20T18:49:51.969093+0000","last_unstale":"2024-09-20T19:04:35.327016+0000","last_undegraded":"2024-09-20T19:04:35.327016+0000","last_fullsized":"2024-09-20T19:04:35.327016+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:03.821908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:03.821908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"273'19","reported_seq":30,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.313581+0000","last_change":"2024-09-20T19:04:20.970137+0000","last_active":"2024-09-20T19:04:35.313581+0000","last_peered":"2024-09-20T19:04:35.313581+0000","last_clean":"2024-09-20T19:04:35.313581+0000","last_became_active":"2024-09-20T19:04:20.897330+0000","last_became_peered":"2024-09-20T19:04:20.897330+0000","last_unstale":"2024-09-20T19:04:35.313581+0000","last_undegraded":"2024-09-20T19:04:35.313581+0000","last_fullsized":"2024-09-20T19:04:35.313581+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:57:50.899514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:50.899514+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.313760+0000","last_change":"2024-09-20T19:04:20.896732+0000","last_active":"2024-09-20T19:04:35.313760+0000","last_peered":"2024-09-20T19:04:35.313760+0000","last_clean":"2024-09-20T19:04:35.313760+0000","last_became_active":"2024-09-20T19:04:20.896007+0000","last_became_peered":"2024-09-20T19:04:20.896007+0000","last_unstale":"2024-09-20T19:04:35.313760+0000","last_undegraded":"2024-09-20T19:04:35.313760+0000","last_fullsized":"2024-09-20T19:04:35.313760+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:06.310027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:59:06.310027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":677,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.329062+0000","last_change":"2024-09-20T19:02:38.906352+0000","last_active":"2024-09-20T19:04:35.329062+0000","last_peered":"2024-09-20T19:04:35.329062+0000","last_clean":"2024-09-20T19:04:35.329062+0000","last_became_active":"2024-09-20T18:49:49.957067+0000","last_became_peered":"2024-09-20T18:49:49.957067+0000","last_unstale":"2024-09-20T19:04:35.329062+0000","last_undegraded":"2024-09-20T19:04:35.329062+0000","last_fullsized":"2024-09-20T19:04:35.329062+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:38.906161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:38.906161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":657,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.329154+0000","last_change":"2024-09-20T18:57:16.789412+0000","last_active":"2024-09-20T19:04:35.329154+0000","last_peered":"2024-09-20T19:04:35.329154+0000","last_clean":"2024-09-20T19:04:35.329154+0000","last_became_active":"2024-09-20T18:49:51.966915+0000","last_became_peered":"2024-09-20T18:49:51.966915+0000","last_unstale":"2024-09-20T19:04:35.329154+0000","last_undegraded":"2024-09-20T19:04:35.329154+0000","last_fullsized":"2024-09-20T19:04:35.329154+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:16.789095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:16.789095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:27.583586+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"273'15","reported_seq":748,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.329262+0000","last_change":"2024-09-20T19:02:30.816061+0000","last_active":"2024-09-20T19:04:35.329262+0000","last_peered":"2024-09-20T19:04:35.329262+0000","last_clean":"2024-09-20T19:04:35.329262+0000","last_became_active":"2024-09-20T18:59:05.085230+0000","last_became_peered":"2024-09-20T18:59:05.085230+0000","last_unstale":"2024-09-20T19:04:35.329262+0000","last_undegraded":"2024-09-20T19:04:35.329262+0000","last_fullsized":"2024-09-20T19:04:35.329262+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:02:30.815747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:30.815747+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":703,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.316250+0000","last_change":"2024-09-20T19:04:20.885127+0000","last_active":"2024-09-20T19:04:35.316250+0000","last_peered":"2024-09-20T19:04:35.316250+0000","last_clean":"2024-09-20T19:04:35.316250+0000","last_became_active":"2024-09-20T19:04:20.884660+0000","last_became_peered":"2024-09-20T19:04:20.884660+0000","last_unstale":"2024-09-20T19:04:35.316250+0000","last_undegraded":"2024-09-20T19:04:35.316250+0000","last_fullsized":"2024-09-20T19:04:35.316250+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:07.408197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:03:07.408197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":677,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.333463+0000","last_change":"2024-09-20T19:01:03.507072+0000","last_active":"2024-09-20T19:04:35.333463+0000","last_peered":"2024-09-20T19:04:35.333463+0000","last_clean":"2024-09-20T19:04:35.333463+0000","last_became_active":"2024-09-20T18:49:49.962523+0000","last_became_peered":"2024-09-20T18:49:49.962523+0000","last_unstale":"2024-09-20T19:04:35.333463+0000","last_undegraded":"2024-09-20T19:04:35.333463+0000","last_fullsized":"2024-09-20T19:04:35.333463+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:03.507016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:03.507016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":674,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.319311+0000","last_change":"2024-09-20T19:01:11.847792+0000","last_active":"2024-09-20T19:04:35.319311+0000","last_peered":"2024-09-20T19:04:35.319311+0000","last_clean":"2024-09-20T19:04:35.319311+0000","last_became_active":"2024-09-20T18:49:51.967313+0000","last_became_peered":"2024-09-20T18:49:51.967313+0000","last_unstale":"2024-09-20T19:04:35.319311+0000","last_undegraded":"2024-09-20T19:04:35.319311+0000","last_fullsized":"2024-09-20T19:04:35.319311+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:11.847716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:11.847716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"274'19","reported_seq":750,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.329283+0000","last_change":"2024-09-20T19:01:05.495603+0000","last_active":"2024-09-20T19:04:35.329283+0000","last_peered":"2024-09-20T19:04:35.329283+0000","last_clean":"2024-09-20T19:04:35.329283+0000","last_became_active":"2024-09-20T18:59:05.075390+0000","last_became_peered":"2024-09-20T18:59:05.075390+0000","last_unstale":"2024-09-20T19:04:35.329283+0000","last_undegraded":"2024-09-20T19:04:35.329283+0000","last_fullsized":"2024-09-20T19:04:35.329283+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'19","last_scrub_stamp":"2024-09-20T19:01:05.495478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:05.495478+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":693,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.327613+0000","last_change":"2024-09-20T19:01:47.521858+0000","last_active":"2024-09-20T19:04:35.327613+0000","last_peered":"2024-09-20T19:04:35.327613+0000","last_clean":"2024-09-20T19:04:35.327613+0000","last_became_active":"2024-09-20T18:49:45.942105+0000","last_became_peered":"2024-09-20T18:49:45.942105+0000","last_unstale":"2024-09-20T19:04:35.327613+0000","last_undegraded":"2024-09-20T19:04:35.327613+0000","last_fullsized":"2024-09-20T19:04:35.327613+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:47.521789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:47.521789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":717,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.315759+0000","last_change":"2024-09-20T19:04:20.912120+0000","last_active":"2024-09-20T19:04:35.315759+0000","last_peered":"2024-09-20T19:04:35.315759+0000","last_clean":"2024-09-20T19:04:35.315759+0000","last_became_active":"2024-09-20T19:04:20.911292+0000","last_became_peered":"2024-09-20T19:04:20.911292+0000","last_unstale":"2024-09-20T19:04:35.315759+0000","last_undegraded":"2024-09-20T19:04:35.315759+0000","last_fullsized":"2024-09-20T19:04:35.315759+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:27.267254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:27.267254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":679,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.333552+0000","last_change":"2024-09-20T19:02:38.950217+0000","last_active":"2024-09-20T19:04:35.333552+0000","last_peered":"2024-09-20T19:04:35.333552+0000","last_clean":"2024-09-20T19:04:35.333552+0000","last_became_active":"2024-09-20T18:59:05.081179+0000","last_became_peered":"2024-09-20T18:59:05.081179+0000","last_unstale":"2024-09-20T19:04:35.333552+0000","last_undegraded":"2024-09-20T19:04:35.333552+0000","last_fullsized":"2024-09-20T19:04:35.333552+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:38.950017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:38.950017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"273'8","reported_seq":727,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:40.559947+0000","last_change":"2024-09-20T19:04:40.559947+0000","last_active":"2024-09-20T19:04:40.559947+0000","last_peered":"2024-09-20T19:04:40.559947+0000","last_clean":"2024-09-20T19:04:40.559947+0000","last_became_active":"2024-09-20T18:59:05.076365+0000","last_became_peered":"2024-09-20T18:59:05.076365+0000","last_unstale":"2024-09-20T19:04:40.559947+0000","last_undegraded":"2024-09-20T19:04:40.559947+0000","last_fullsized":"2024-09-20T19:04:40.559947+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'8","last_scrub_stamp":"2024-09-20T19:04:40.559877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:04:40.559877+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:53.505030+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":694,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.320065+0000","last_change":"2024-09-20T19:02:39.184249+0000","last_active":"2024-09-20T19:04:35.320065+0000","last_peered":"2024-09-20T19:04:35.320065+0000","last_clean":"2024-09-20T19:04:35.320065+0000","last_became_active":"2024-09-20T18:49:45.940988+0000","last_became_peered":"2024-09-20T18:49:45.940988+0000","last_unstale":"2024-09-20T19:04:35.320065+0000","last_undegraded":"2024-09-20T19:04:35.320065+0000","last_fullsized":"2024-09-20T19:04:35.320065+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:39.184150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:39.184150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":281,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.318457+0000","last_change":"2024-09-20T19:04:20.900567+0000","last_active":"2024-09-20T19:04:35.318457+0000","last_peered":"2024-09-20T19:04:35.318457+0000","last_clean":"2024-09-20T19:04:35.318457+0000","last_became_active":"2024-09-20T19:04:20.900214+0000","last_became_peered":"2024-09-20T19:04:20.900214+0000","last_unstale":"2024-09-20T19:04:35.318457+0000","last_undegraded":"2024-09-20T19:04:35.318457+0000","last_fullsized":"2024-09-20T19:04:35.318457+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:41.497115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:41.497115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":691,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.315699+0000","last_change":"2024-09-20T19:04:20.886208+0000","last_active":"2024-09-20T19:04:35.315699+0000","last_peered":"2024-09-20T19:04:35.315699+0000","last_clean":"2024-09-20T19:04:35.315699+0000","last_became_active":"2024-09-20T19:04:20.885211+0000","last_became_peered":"2024-09-20T19:04:20.885211+0000","last_unstale":"2024-09-20T19:04:35.315699+0000","last_undegraded":"2024-09-20T19:04:35.315699+0000","last_fullsized":"2024-09-20T19:04:35.315699+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:29.277916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:29.277916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":673,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.321263+0000","last_change":"2024-09-20T19:04:20.890533+0000","last_active":"2024-09-20T19:04:35.321263+0000","last_peered":"2024-09-20T19:04:35.321263+0000","last_clean":"2024-09-20T19:04:35.321263+0000","last_became_active":"2024-09-20T19:04:20.890101+0000","last_became_peered":"2024-09-20T19:04:20.890101+0000","last_unstale":"2024-09-20T19:04:35.321263+0000","last_undegraded":"2024-09-20T19:04:35.321263+0000","last_fullsized":"2024-09-20T19:04:35.321263+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:10.970779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:10.970779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":677,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.328933+0000","last_change":"2024-09-20T18:58:36.077005+0000","last_active":"2024-09-20T19:04:35.328933+0000","last_peered":"2024-09-20T19:04:35.328933+0000","last_clean":"2024-09-20T19:04:35.328933+0000","last_became_active":"2024-09-20T18:49:45.937992+0000","last_became_peered":"2024-09-20T18:49:45.937992+0000","last_unstale":"2024-09-20T19:04:35.328933+0000","last_undegraded":"2024-09-20T19:04:35.328933+0000","last_fullsized":"2024-09-20T19:04:35.328933+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:36.076937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:36.076937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:34.560802+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"274'19","reported_seq":280,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.319721+0000","last_change":"2024-09-20T19:01:15.544760+0000","last_active":"2024-09-20T19:04:35.319721+0000","last_peered":"2024-09-20T19:04:35.319721+0000","last_clean":"2024-09-20T19:04:35.319721+0000","last_became_active":"2024-09-20T18:59:05.089294+0000","last_became_peered":"2024-09-20T18:59:05.089294+0000","last_unstale":"2024-09-20T19:04:35.319721+0000","last_undegraded":"2024-09-20T19:04:35.319721+0000","last_fullsized":"2024-09-20T19:04:35.319721+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'19","last_scrub_stamp":"2024-09-20T19:01:15.544663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:15.544663+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":671,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T19:04:35.327156+0000","last_change":"2024-09-20T19:01:40.961449+0000","last_active":"2024-09-20T19:04:35.327156+0000","last_peered":"2024-09-20T19:04:35.327156+0000","last_clean":"2024-09-20T19:04:35.327156+0000","last_became_active":"2024-09-20T18:49:49.957582+0000","last_became_peered":"2024-09-20T18:49:49.957582+0000","last_unstale":"2024-09-20T19:04:35.327156+0000","last_undegraded":"2024-09-20T19:04:35.327156+0000","last_fullsized":"2024-09-20T19:04:35.327156+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:40.961363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:40.961363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":4833017880,"num_objects":73747,"num_object_clones":0,"num_object_copies":221241,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":11650,"num_objects_unfound":0,"num_objects_dirty":73747,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":159973,"num_write_kb":7478977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20838,"num_bytes_recovered":1365311488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":14949101568,"data_stored":14949089352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91841,"ondisk_log_size":91841,"up":48,"acting":45,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1185,"internal_metadata":0},"log_size":87,"ondisk_log_size":87,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1115,"num_read_kb":872,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":4890,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":686,"ondisk_log_size":686,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":3105,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":888,"num_read_kb":883,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266871,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1854456,"kb_used_data":1304040,"kb_used_omap":1,"kb_used_meta":550398,"kb_avail":91890696,"statfs":{"total":95995035648,"available":94096072704,"internally_reserved":0,"allocated":1335336960,"data_stored":1334940036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":563608010},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528510,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2482280,"kb_used_data":2077656,"kb_used_omap":1,"kb_used_meta":404606,"kb_avail":91262872,"statfs":{"total":95995035648,"available":93453180928,"internally_reserved":0,"allocated":2127519744,"data_stored":2127138566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":414316998},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757443,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2038900,"kb_used_data":1494052,"kb_used_omap":1,"kb_used_meta":544830,"kb_avail":91706252,"statfs":{"total":95995035648,"available":93907202048,"internally_reserved":0,"allocated":1529909248,"data_stored":1529515931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":557906373},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":304,"seq":1305670058025,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2170756,"kb_used_data":1951604,"kb_used_omap":5,"kb_used_meta":219130,"kb_avail":91574396,"statfs":{"total":95995035648,"available":93772181504,"internally_reserved":0,"allocated":1998442496,"data_stored":1998033689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5633,"internal_metadata":224389631},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182607,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2557812,"kb_used_data":1868580,"kb_used_omap":1,"kb_used_meta":689214,"kb_avail":91187340,"statfs":{"total":95995035648,"available":93375836160,"internally_reserved":0,"allocated":1913425920,"data_stored":1913032007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":705755591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411540,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2383720,"kb_used_data":1802776,"kb_used_omap":1,"kb_used_meta":580926,"kb_avail":91361432,"statfs":{"total":95995035648,"available":93554106368,"internally_reserved":0,"allocated":1846042624,"data_stored":1845661816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":594868677},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575066,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1905176,"kb_used_data":1288136,"kb_used_omap":1,"kb_used_meta":617022,"kb_avail":91839976,"statfs":{"total":95995035648,"available":94044135424,"internally_reserved":0,"allocated":1319051264,"data_stored":1318622488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":631830981},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738592,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3559292,"kb_used_data":2924204,"kb_used_omap":1,"kb_used_meta":635070,"kb_avail":90185860,"statfs":{"total":95995035648,"available":92350320640,"internally_reserved":0,"allocated":2994384896,"data_stored":2993983596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":650312133},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1185,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2985037824,"data_stored":2985033752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1304166400,"data_stored":1304166400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1821048832,"data_stored":1821048832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1896943616,"data_stored":1896939544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1983971328,"data_stored":1983971328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1527185408,"data_stored":1527185408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2109411328,"data_stored":2109407256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1321336832,"data_stored":1321336832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T19:04:45.058 INFO:tasks.thrashosds.thrasher:Setting 4.2 to [5, 7, 4] 2024-09-20T19:04:45.058 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '4.2', '5', '7', '4'] 2024-09-20T19:04:45.058 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd pg-upmap 4.2 5 7 4 2024-09-20T19:04:45.144 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:45.165 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:45.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:45.166 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:45.266 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:45.305 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:45.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:45.306 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:45.351 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:04:45.352 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:04:45.357 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-20T19:04:45.393 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:04:45.393 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:04:45.407 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:04:45.431 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:04:45.433 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:04:45.451 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:45.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:45.452 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:04:45.457 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:04:45.458 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:04:45.485 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:04:45.485 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:04:45.515 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:04:45.515 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:04:45.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:45 smithi083 ceph-mon[29421]: from='client.26012 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:04:45.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:45 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2327597725' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:04:45.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:45 smithi083 ceph-mon[29421]: 5.13 scrub starts 2024-09-20T19:04:45.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:45 smithi083 ceph-mon[29421]: 5.13 scrub ok 2024-09-20T19:04:45.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:45 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1212995838' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:04:45.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:45 smithi083 ceph-mon[36260]: from='client.26012 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:04:45.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:45 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2327597725' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:04:45.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:45 smithi083 ceph-mon[36260]: 5.13 scrub starts 2024-09-20T19:04:45.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:45 smithi083 ceph-mon[36260]: 5.13 scrub ok 2024-09-20T19:04:45.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:45 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1212995838' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:04:45.553 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:45.556 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:04:45.557 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:04:45.582 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:04:45.583 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:04:45.591 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:45.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:45.592 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:45.693 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:45.733 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:45.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:45.734 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:45.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:45 smithi142 ceph-mon[32508]: from='client.26012 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:04:45.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:45 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2327597725' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:04:45.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:45 smithi142 ceph-mon[32508]: 5.13 scrub starts 2024-09-20T19:04:45.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:45 smithi142 ceph-mon[32508]: 5.13 scrub ok 2024-09-20T19:04:45.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:45 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1212995838' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:04:45.835 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:45.874 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:45.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:45.875 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:45.976 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:46.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:46.021 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:46.021 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:46.122 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:46.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:46.153 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:46.154 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:46.170 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T19:04:46.170 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:04:46.255 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:04:46.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:46.287 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:46.287 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:04:46.388 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:04:46.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:46.423 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:46.423 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:04:46.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:46 smithi142 ceph-mon[32508]: pgmap v743: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 143 active+clean; 4.5 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 954 op/s; 11650/221838 objects misplaced (5.252%); 7.1 MiB/s, 112 objects/s recovering 2024-09-20T19:04:46.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:46 smithi142 ceph-mon[32508]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T19:04:46.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:46 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2327597725' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:04:46.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:46 smithi142 ceph-mon[32508]: osdmap e380: 8 total, 8 up, 8 in 2024-09-20T19:04:46.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:46 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:04:46.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:46 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:04:46.525 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:46.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:46 smithi083 ceph-mon[29421]: pgmap v743: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 143 active+clean; 4.5 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 954 op/s; 11650/221838 objects misplaced (5.252%); 7.1 MiB/s, 112 objects/s recovering 2024-09-20T19:04:46.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:46 smithi083 ceph-mon[29421]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T19:04:46.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:46 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2327597725' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:04:46.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:46 smithi083 ceph-mon[29421]: osdmap e380: 8 total, 8 up, 8 in 2024-09-20T19:04:46.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:46 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:04:46.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:46 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:04:46.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:46 smithi083 ceph-mon[36260]: pgmap v743: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 143 active+clean; 4.5 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 954 op/s; 11650/221838 objects misplaced (5.252%); 7.1 MiB/s, 112 objects/s recovering 2024-09-20T19:04:46.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:46 smithi083 ceph-mon[36260]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T19:04:46.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:46 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2327597725' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:04:46.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:46 smithi083 ceph-mon[36260]: osdmap e380: 8 total, 8 up, 8 in 2024-09-20T19:04:46.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:46 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:04:46.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:46 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:04:46.555 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:46.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:46.556 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:46.657 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:46.691 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:46.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:46.692 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:46.792 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:04:46.793 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:46.822 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:46.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:46.823 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:46.924 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:46.957 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:46.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:46.958 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:47.057 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_blocked_ops 2024-09-20T19:04:47.059 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:47.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:47.109 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:47.110 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:47.210 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:47.257 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:47.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:47.258 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:47.359 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:47.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:47.400 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:47.400 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:47.502 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:47.541 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:04:47.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:47 smithi083 ceph-mon[29421]: 3.f scrub starts 2024-09-20T19:04:47.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:47 smithi083 ceph-mon[29421]: 3.f scrub ok 2024-09-20T19:04:47.542 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:47 smithi083 ceph-mon[36260]: 3.f scrub starts 2024-09-20T19:04:47.542 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:47 smithi083 ceph-mon[36260]: 3.f scrub ok 2024-09-20T19:04:47.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:47.553 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:47.553 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:47.654 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:04:47.689 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:04:47 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:04:47.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:47 smithi142 ceph-mon[32508]: 3.f scrub starts 2024-09-20T19:04:47.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:47 smithi142 ceph-mon[32508]: 3.f scrub ok 2024-09-20T19:04:47.694 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:47.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:47.695 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:04:47.796 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:47.826 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:47.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:47.827 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:47.928 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:47.948 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:47.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:47.949 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:48.050 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:48.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:48.087 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:48.088 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:48.188 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:48.231 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:48.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:48.232 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:48.332 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T19:04:48.333 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:48.371 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:48.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:48.372 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:48.391 INFO:teuthology.orchestra.run.smithi083.stderr:set 4.2 pg_upmap mapping to [5,7,4] 2024-09-20T19:04:48.473 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:48.502 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:48.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:48.504 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:48.604 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:48.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:48.644 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:48.644 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:48.699 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:48 smithi142 ceph-mon[32508]: pgmap v745: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 143 active+clean; 4.3 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.02k op/s; 10744/210933 objects misplaced (5.094%); 9.4 MiB/s, 150 objects/s recovering 2024-09-20T19:04:48.700 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:48 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:04:48.700 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:48 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3252576039' entity='client.admin' cmd={"prefix": "osd pg-upmap", "pgid": "4.2", "id": [5, 7, 4]} : dispatch 2024-09-20T19:04:48.700 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:48 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pg-upmap", "pgid": "4.2", "id": [5, 7, 4]} : dispatch 2024-09-20T19:04:48.700 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:48 smithi142 ceph-mon[32508]: 3.11 scrub starts 2024-09-20T19:04:48.700 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:48 smithi142 ceph-mon[32508]: 3.11 scrub ok 2024-09-20T19:04:48.745 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:48.768 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:48 smithi083 ceph-mon[29421]: pgmap v745: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 143 active+clean; 4.3 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.02k op/s; 10744/210933 objects misplaced (5.094%); 9.4 MiB/s, 150 objects/s recovering 2024-09-20T19:04:48.768 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:48 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:04:48.768 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:48 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3252576039' entity='client.admin' cmd={"prefix": "osd pg-upmap", "pgid": "4.2", "id": [5, 7, 4]} : dispatch 2024-09-20T19:04:48.768 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:48 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pg-upmap", "pgid": "4.2", "id": [5, 7, 4]} : dispatch 2024-09-20T19:04:48.768 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:48 smithi083 ceph-mon[29421]: 3.11 scrub starts 2024-09-20T19:04:48.768 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:48 smithi083 ceph-mon[29421]: 3.11 scrub ok 2024-09-20T19:04:48.769 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:48 smithi083 ceph-mon[36260]: pgmap v745: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 143 active+clean; 4.3 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.02k op/s; 10744/210933 objects misplaced (5.094%); 9.4 MiB/s, 150 objects/s recovering 2024-09-20T19:04:48.769 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:48 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:04:48.769 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:48 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3252576039' entity='client.admin' cmd={"prefix": "osd pg-upmap", "pgid": "4.2", "id": [5, 7, 4]} : dispatch 2024-09-20T19:04:48.769 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:48 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pg-upmap", "pgid": "4.2", "id": [5, 7, 4]} : dispatch 2024-09-20T19:04:48.769 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:48 smithi083 ceph-mon[36260]: 3.11 scrub starts 2024-09-20T19:04:48.769 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:48 smithi083 ceph-mon[36260]: 3.11 scrub ok 2024-09-20T19:04:48.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:48.787 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:48.788 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:48.835 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T19:04:48.888 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:48.931 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:48.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:48.932 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:49.033 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:49.073 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:49.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:49.074 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:49.175 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:49.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:49.226 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:49.227 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:49.268 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:04:49.323 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:04:49.327 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:49.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:49.371 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:49.372 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:49.397 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:49 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "4.2", "id": [5, 7, 4]}]': finished 2024-09-20T19:04:49.397 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:49 smithi083 ceph-mon[29421]: osdmap e381: 8 total, 8 up, 8 in 2024-09-20T19:04:49.397 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:49 smithi083 ceph-mon[29421]: 5.8 scrub starts 2024-09-20T19:04:49.398 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:49 smithi083 ceph-mon[29421]: 5.8 scrub ok 2024-09-20T19:04:49.398 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:49 smithi083 ceph-mon[29421]: 4.17 scrub starts 2024-09-20T19:04:49.398 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:49 smithi083 ceph-mon[29421]: 4.17 scrub ok 2024-09-20T19:04:49.472 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:49.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:49.512 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:49.512 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:49.613 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:49.645 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:49.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:49.646 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:49.684 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:49 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "4.2", "id": [5, 7, 4]}]': finished 2024-09-20T19:04:49.684 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:49 smithi083 ceph-mon[36260]: osdmap e381: 8 total, 8 up, 8 in 2024-09-20T19:04:49.684 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:49 smithi083 ceph-mon[36260]: 5.8 scrub starts 2024-09-20T19:04:49.684 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:49 smithi083 ceph-mon[36260]: 5.8 scrub ok 2024-09-20T19:04:49.684 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:49 smithi083 ceph-mon[36260]: 4.17 scrub starts 2024-09-20T19:04:49.684 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:49 smithi083 ceph-mon[36260]: 4.17 scrub ok 2024-09-20T19:04:49.747 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:49.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:49 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "4.2", "id": [5, 7, 4]}]': finished 2024-09-20T19:04:49.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:49 smithi142 ceph-mon[32508]: osdmap e381: 8 total, 8 up, 8 in 2024-09-20T19:04:49.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:49 smithi142 ceph-mon[32508]: 5.8 scrub starts 2024-09-20T19:04:49.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:49 smithi142 ceph-mon[32508]: 5.8 scrub ok 2024-09-20T19:04:49.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:49 smithi142 ceph-mon[32508]: 4.17 scrub starts 2024-09-20T19:04:49.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:49 smithi142 ceph-mon[32508]: 4.17 scrub ok 2024-09-20T19:04:49.775 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:49.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:49.776 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:49.877 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:49.898 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:49.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:49.899 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:49.999 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:50.031 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:50.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:50.032 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:50.133 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:04:50.180 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:50.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:50.181 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:04:50.282 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:50.305 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:50.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:50.306 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:50.331 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:04:50.331 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:04:50.368 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:04:50.369 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:04:50.406 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:04:50.406 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:04:50.407 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:04:50.444 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:04:50.444 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:04:50.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:50.450 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:50.450 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:04:50.479 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:04:50.479 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:04:50.515 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:04:50.515 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:04:50.551 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:50.577 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:50 smithi083 ceph-mon[29421]: pgmap v747: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 143 active+clean; 4.2 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 743 op/s; 10477/208458 objects misplaced (5.026%); 6.4 MiB/s, 102 objects/s recovering 2024-09-20T19:04:50.577 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:50 smithi083 ceph-mon[29421]: osdmap e382: 8 total, 8 up, 8 in 2024-09-20T19:04:50.577 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:50 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3468246029' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:04:50.578 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:04:50.578 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:04:50.578 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:50 smithi083 ceph-mon[36260]: pgmap v747: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 143 active+clean; 4.2 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 743 op/s; 10477/208458 objects misplaced (5.026%); 6.4 MiB/s, 102 objects/s recovering 2024-09-20T19:04:50.578 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:50 smithi083 ceph-mon[36260]: osdmap e382: 8 total, 8 up, 8 in 2024-09-20T19:04:50.578 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:50 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3468246029' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:04:50.581 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:50.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:50.582 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:50.611 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:04:50.611 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:04:50.683 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:50.718 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:50.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:50.719 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:50.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:50 smithi142 ceph-mon[32508]: pgmap v747: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 143 active+clean; 4.2 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 743 op/s; 10477/208458 objects misplaced (5.026%); 6.4 MiB/s, 102 objects/s recovering 2024-09-20T19:04:50.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:50 smithi142 ceph-mon[32508]: osdmap e382: 8 total, 8 up, 8 in 2024-09-20T19:04:50.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:50 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3468246029' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:04:50.820 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:50.851 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:50.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:50.852 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:50.953 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:50.984 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:04:50 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:04:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:04:50.985 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:50.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:50.987 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:51.088 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:04:51.093 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T19:04:51.093 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:04:51.126 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:51.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:51.127 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:04:51.228 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:51.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:51.258 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:51.258 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:51.359 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:51.389 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:51.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:51.390 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:51.411 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-20T19:04:51.491 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:51.526 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:51.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:51.527 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:51.628 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:04:51.662 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:51.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:51.663 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:04:51.727 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:51 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1545899131' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:04:51.727 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:51 smithi083 ceph-mon[29421]: 5.17 scrub starts 2024-09-20T19:04:51.727 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:51 smithi083 ceph-mon[29421]: 5.17 scrub ok 2024-09-20T19:04:51.727 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:51 smithi083 ceph-mon[29421]: 3.e scrub starts 2024-09-20T19:04:51.727 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:51 smithi083 ceph-mon[29421]: 3.e scrub ok 2024-09-20T19:04:51.727 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:51 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1545899131' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:04:51.727 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:51 smithi083 ceph-mon[36260]: 5.17 scrub starts 2024-09-20T19:04:51.727 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:51 smithi083 ceph-mon[36260]: 5.17 scrub ok 2024-09-20T19:04:51.727 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:51 smithi083 ceph-mon[36260]: 3.e scrub starts 2024-09-20T19:04:51.727 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:51 smithi083 ceph-mon[36260]: 3.e scrub ok 2024-09-20T19:04:51.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:51 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1545899131' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:04:51.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:51 smithi142 ceph-mon[32508]: 5.17 scrub starts 2024-09-20T19:04:51.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:51 smithi142 ceph-mon[32508]: 5.17 scrub ok 2024-09-20T19:04:51.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:51 smithi142 ceph-mon[32508]: 3.e scrub starts 2024-09-20T19:04:51.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:51 smithi142 ceph-mon[32508]: 3.e scrub ok 2024-09-20T19:04:51.764 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:04:51.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:51.799 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:51.799 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:04:51.900 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:51.932 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:51.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:51.933 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:52.034 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:52.071 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:52.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:52.072 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:52.172 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:52.207 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:52.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:52.208 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:52.309 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:52.337 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:52.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:52.338 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:52.439 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:52.472 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:52.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:52.473 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:52.574 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:52.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:52.612 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:52.613 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:52.713 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:04:52.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:52.757 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:52.757 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:04:52.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:52 smithi142 ceph-mon[32508]: pgmap v749: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 143 active+clean; 4.2 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 791 op/s; 10477/207597 objects misplaced (5.047%); 6.4 MiB/s, 102 objects/s recovering 2024-09-20T19:04:52.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:52 smithi142 ceph-mon[32508]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:04:52.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:52 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1545899131' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T19:04:52.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:52 smithi142 ceph-mon[32508]: osdmap e383: 8 total, 8 up, 8 in 2024-09-20T19:04:52.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:52 smithi083 ceph-mon[29421]: pgmap v749: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 143 active+clean; 4.2 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 791 op/s; 10477/207597 objects misplaced (5.047%); 6.4 MiB/s, 102 objects/s recovering 2024-09-20T19:04:52.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:52 smithi083 ceph-mon[29421]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:04:52.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:52 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1545899131' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T19:04:52.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:52 smithi083 ceph-mon[29421]: osdmap e383: 8 total, 8 up, 8 in 2024-09-20T19:04:52.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:52 smithi083 ceph-mon[36260]: pgmap v749: 148 pgs: 3 active+recovering+undersized+remapped, 2 active+remapped+backfill_toofull, 143 active+clean; 4.2 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 791 op/s; 10477/207597 objects misplaced (5.047%); 6.4 MiB/s, 102 objects/s recovering 2024-09-20T19:04:52.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:52 smithi083 ceph-mon[36260]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:04:52.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:52 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1545899131' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T19:04:52.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:52 smithi083 ceph-mon[36260]: osdmap e383: 8 total, 8 up, 8 in 2024-09-20T19:04:52.858 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:52.891 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:52.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:52.892 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:52.973 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_historic_ops 2024-09-20T19:04:52.993 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:53.024 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:53.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:53.025 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:53.125 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:53.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:53.165 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:53.166 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:53.266 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:53.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:53.315 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:53.315 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:53.343 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T19:04:53.416 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:53.473 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:53.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:53.474 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:53.475 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:04:53.575 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:53.613 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:53.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:53.614 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:53.715 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:04:53.746 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:04:53.758 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:53.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:53.759 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:04:53.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:53 smithi142 ceph-mon[32508]: Health check failed: Degraded data redundancy: 5/195459 objects degraded (0.003%), 1 pg degraded (PG_DEGRADED) 2024-09-20T19:04:53.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:53 smithi083 ceph-mon[29421]: Health check failed: Degraded data redundancy: 5/195459 objects degraded (0.003%), 1 pg degraded (PG_DEGRADED) 2024-09-20T19:04:53.792 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:53 smithi083 ceph-mon[36260]: Health check failed: Degraded data redundancy: 5/195459 objects degraded (0.003%), 1 pg degraded (PG_DEGRADED) 2024-09-20T19:04:53.860 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:53.890 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:53.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:53.891 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:53.992 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:04:54.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:54.034 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:54.034 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:04:54.135 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:54.140 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T19:04:54.178 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:54.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:54.179 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:54.187 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 7, 6, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 5, 6, 7, 4] 2024-09-20T19:04:54.188 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd reweight 0 1 2024-09-20T19:04:54.280 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:54.313 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:54.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:54.314 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:54.381 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T19:04:54.381 INFO:teuthology.orchestra.run.smithi142.stdout: "size": 20, 2024-09-20T19:04:54.381 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 600, 2024-09-20T19:04:54.382 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-09-20T19:04:54.382 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.382 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:101813 9.6 9:6a97dc4d:::benchmark_data_smithi083_124552_object101812:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e275)", 2024-09-20T19:04:54.382 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:57.454719+0000", 2024-09-20T19:04:54.382 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 296.64402387600001, 2024-09-20T19:04:54.382 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.245643423, 2024-09-20T19:04:54.382 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:04:54.382 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:04:54.382 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:54.382 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:04:54.382 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:04:54.382 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:04:54.382 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 101813 2024-09-20T19:04:54.382 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.383 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:04:54.383 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.383 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:04:54.383 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.454719+0000", 2024-09-20T19:04:54.383 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:04:54.383 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.383 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.383 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:04:54.383 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.454719+0000", 2024-09-20T19:04:54.383 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:04:54.383 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.383 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.383 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:04:54.383 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.454721+0000", 2024-09-20T19:04:54.384 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.092e-06 2024-09-20T19:04:54.384 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.384 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.384 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:04:54.384 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.454821+0000", 2024-09-20T19:04:54.384 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.9264000000000004e-05 2024-09-20T19:04:54.384 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.384 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.384 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:04:54.384 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.454844+0000", 2024-09-20T19:04:54.384 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3275e-05 2024-09-20T19:04:54.384 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.384 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.384 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:04:54.385 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.454851+0000", 2024-09-20T19:04:54.385 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.2150000000000004e-06 2024-09-20T19:04:54.385 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.385 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.385 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:04:54.385 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.454928+0000", 2024-09-20T19:04:54.385 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.7286999999999994e-05 2024-09-20T19:04:54.385 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.385 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.385 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:04:54.385 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.475113+0000", 2024-09-20T19:04:54.385 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.020184166999999999 2024-09-20T19:04:54.385 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.385 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.386 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:04:54.386 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.475254+0000", 2024-09-20T19:04:54.386 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00014129 2024-09-20T19:04:54.386 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.386 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.386 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:54.386 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.700227+0000", 2024-09-20T19:04:54.386 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.22497298499999999 2024-09-20T19:04:54.386 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.386 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.386 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:04:54.386 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.700238+0000", 2024-09-20T19:04:54.386 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1204e-05 2024-09-20T19:04:54.386 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.387 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.387 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:54.387 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.700277+0000", 2024-09-20T19:04:54.387 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.8742999999999998e-05 2024-09-20T19:04:54.387 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.387 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.387 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:04:54.387 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.700326+0000", 2024-09-20T19:04:54.387 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.9224000000000002e-05 2024-09-20T19:04:54.387 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.387 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.387 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:04:54.387 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.700363+0000", 2024-09-20T19:04:54.388 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6677000000000001e-05 2024-09-20T19:04:54.388 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:04:54.388 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:04:54.388 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:04:54.388 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.388 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.388 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:101814 9.a 9:5f144506:::benchmark_data_smithi083_124552_object101813:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e275)", 2024-09-20T19:04:54.388 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:57.454874+0000", 2024-09-20T19:04:54.388 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 296.64386926899999, 2024-09-20T19:04:54.388 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.24509635499999999, 2024-09-20T19:04:54.388 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:04:54.388 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:04:54.388 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:54.388 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:04:54.388 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:04:54.389 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:04:54.389 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 101814 2024-09-20T19:04:54.389 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.389 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:04:54.389 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.389 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:04:54.389 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.454874+0000", 2024-09-20T19:04:54.389 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:04:54.389 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.389 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.389 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:04:54.389 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.454874+0000", 2024-09-20T19:04:54.389 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:04:54.389 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.390 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.390 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:04:54.390 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.454875+0000", 2024-09-20T19:04:54.390 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.23e-07 2024-09-20T19:04:54.390 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.390 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.390 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:04:54.390 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.454904+0000", 2024-09-20T19:04:54.390 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9767000000000001e-05 2024-09-20T19:04:54.390 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.390 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.390 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:04:54.390 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.454911+0000", 2024-09-20T19:04:54.391 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2809999999999997e-06 2024-09-20T19:04:54.391 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.391 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.391 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:04:54.391 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.454916+0000", 2024-09-20T19:04:54.391 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3009999999999997e-06 2024-09-20T19:04:54.391 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.391 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.391 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:04:54.391 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.454965+0000", 2024-09-20T19:04:54.391 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8677000000000002e-05 2024-09-20T19:04:54.391 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.391 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.391 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:04:54.391 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.475123+0000", 2024-09-20T19:04:54.392 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.020158206000000001 2024-09-20T19:04:54.392 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.392 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.392 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,7", 2024-09-20T19:04:54.392 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.475266+0000", 2024-09-20T19:04:54.392 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000142989 2024-09-20T19:04:54.392 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.392 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.392 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:04:54.392 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.479206+0000", 2024-09-20T19:04:54.392 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.003939784 2024-09-20T19:04:54.392 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.392 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.392 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:54.393 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.484059+0000", 2024-09-20T19:04:54.393 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0048534149999999998 2024-09-20T19:04:54.393 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.393 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.393 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:54.393 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.699859+0000", 2024-09-20T19:04:54.393 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.21579965300000001 2024-09-20T19:04:54.393 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.393 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.393 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:04:54.393 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.699923+0000", 2024-09-20T19:04:54.393 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4431999999999996e-05 2024-09-20T19:04:54.393 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.393 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.393 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:04:54.394 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.699970+0000", 2024-09-20T19:04:54.394 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7126999999999998e-05 2024-09-20T19:04:54.394 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:04:54.394 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:04:54.394 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:04:54.394 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.394 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.394 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:101818 9.6 9:6be85cb6:::benchmark_data_smithi083_124552_object101817:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e275)", 2024-09-20T19:04:54.394 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:57.455092+0000", 2024-09-20T19:04:54.394 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 296.643651632, 2024-09-20T19:04:54.394 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.24541227099999999, 2024-09-20T19:04:54.394 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:04:54.394 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:04:54.394 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:54.395 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:04:54.395 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:04:54.395 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:04:54.395 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 101818 2024-09-20T19:04:54.395 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.395 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:04:54.395 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.395 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:04:54.395 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.455092+0000", 2024-09-20T19:04:54.395 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:04:54.395 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.395 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.395 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:04:54.396 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.455092+0000", 2024-09-20T19:04:54.396 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:04:54.396 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.396 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.396 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:04:54.396 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.455099+0000", 2024-09-20T19:04:54.396 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.6890000000000004e-06 2024-09-20T19:04:54.396 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.396 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.396 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:04:54.396 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.455154+0000", 2024-09-20T19:04:54.396 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4653e-05 2024-09-20T19:04:54.396 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.396 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.397 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:04:54.397 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.475426+0000", 2024-09-20T19:04:54.397 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.020272367999999999 2024-09-20T19:04:54.397 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.397 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.397 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:04:54.397 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.475435+0000", 2024-09-20T19:04:54.397 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.3240000000000003e-06 2024-09-20T19:04:54.397 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.397 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.397 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:04:54.397 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.478230+0000", 2024-09-20T19:04:54.397 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0027948949999999999 2024-09-20T19:04:54.397 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.398 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.398 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:04:54.398 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.478341+0000", 2024-09-20T19:04:54.398 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011116999999999999 2024-09-20T19:04:54.398 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.398 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.398 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:04:54.398 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.478466+0000", 2024-09-20T19:04:54.398 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000124838 2024-09-20T19:04:54.398 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.398 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.398 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:04:54.398 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.700243+0000", 2024-09-20T19:04:54.398 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.221777312 2024-09-20T19:04:54.399 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.399 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.399 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:54.399 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.700398+0000", 2024-09-20T19:04:54.399 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015468799999999999 2024-09-20T19:04:54.399 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.399 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.399 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:54.399 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.700428+0000", 2024-09-20T19:04:54.399 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.0961999999999999e-05 2024-09-20T19:04:54.399 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.399 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.399 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:04:54.399 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.700472+0000", 2024-09-20T19:04:54.399 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3846e-05 2024-09-20T19:04:54.400 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.400 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.400 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:04:54.400 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.700504+0000", 2024-09-20T19:04:54.400 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.1526000000000003e-05 2024-09-20T19:04:54.400 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:04:54.400 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:04:54.400 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:04:54.400 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.400 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.400 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:101817 9.b e275/261 9:de269700:::benchmark_data_smithi083_124552_object101816:head v 275'6183, mlcod=258'2642)", 2024-09-20T19:04:54.400 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:57.456401+0000", 2024-09-20T19:04:54.400 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 296.64234261299998, 2024-09-20T19:04:54.400 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.245359931, 2024-09-20T19:04:54.401 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:04:54.401 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:04:54.401 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:54.401 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:04:54.401 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.401 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:04:54.401 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.456401+0000", 2024-09-20T19:04:54.401 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:04:54.401 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.401 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.401 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:04:54.401 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.456401+0000", 2024-09-20T19:04:54.401 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:04:54.401 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.402 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.402 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:04:54.402 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.456402+0000", 2024-09-20T19:04:54.402 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.7000000000000003e-07 2024-09-20T19:04:54.402 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.402 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.402 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:04:54.402 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.456438+0000", 2024-09-20T19:04:54.402 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6287e-05 2024-09-20T19:04:54.402 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.402 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.402 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:04:54.402 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.475329+0000", 2024-09-20T19:04:54.402 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.018891086000000001 2024-09-20T19:04:54.403 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.403 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.403 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:04:54.403 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.475339+0000", 2024-09-20T19:04:54.403 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0219e-05 2024-09-20T19:04:54.403 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.403 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.403 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:04:54.403 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.475554+0000", 2024-09-20T19:04:54.403 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00021468600000000001 2024-09-20T19:04:54.403 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.403 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.403 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:04:54.403 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.475579+0000", 2024-09-20T19:04:54.403 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.4984e-05 2024-09-20T19:04:54.404 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.404 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.404 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:04:54.404 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.487176+0000", 2024-09-20T19:04:54.404 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.011597624000000001 2024-09-20T19:04:54.404 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.404 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.404 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:04:54.404 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.701761+0000", 2024-09-20T19:04:54.404 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.21458407500000001 2024-09-20T19:04:54.404 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:04:54.404 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:04:54.404 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:04:54.404 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.404 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.405 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25484.0:101835 9.b e275/261 9:d30a326b:::benchmark_data_smithi083_124552_object101834:head v 275'6184, mlcod=258'2642)", 2024-09-20T19:04:54.405 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:57.481572+0000", 2024-09-20T19:04:54.405 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 296.61717081799998, 2024-09-20T19:04:54.405 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.23071034300000001, 2024-09-20T19:04:54.405 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:04:54.405 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:04:54.405 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:54.405 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:04:54.405 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.405 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:04:54.405 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.481572+0000", 2024-09-20T19:04:54.405 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:04:54.405 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.405 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.406 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:04:54.406 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.481572+0000", 2024-09-20T19:04:54.406 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:04:54.406 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.406 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.406 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:04:54.406 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.481573+0000", 2024-09-20T19:04:54.406 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.2200000000000003e-07 2024-09-20T19:04:54.406 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.406 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.406 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:04:54.406 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.481613+0000", 2024-09-20T19:04:54.406 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9548999999999997e-05 2024-09-20T19:04:54.406 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.407 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.407 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:04:54.407 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.481887+0000", 2024-09-20T19:04:54.407 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00027467600000000001 2024-09-20T19:04:54.407 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.407 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.407 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:04:54.407 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.481894+0000", 2024-09-20T19:04:54.407 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.6470000000000001e-06 2024-09-20T19:04:54.407 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.407 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.407 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:04:54.407 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.701801+0000", 2024-09-20T19:04:54.407 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.219906938 2024-09-20T19:04:54.407 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.408 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.408 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:04:54.408 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.701824+0000", 2024-09-20T19:04:54.408 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3054e-05 2024-09-20T19:04:54.408 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.408 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.408 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:04:54.408 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.712234+0000", 2024-09-20T19:04:54.408 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.010409865000000001 2024-09-20T19:04:54.408 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.408 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.408 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:04:54.408 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.712283+0000", 2024-09-20T19:04:54.409 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8792000000000003e-05 2024-09-20T19:04:54.409 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:04:54.409 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:04:54.409 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:04:54.409 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.409 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.409 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:101836 9.6 9:6bfb3d0b:::benchmark_data_smithi083_124552_object101835:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e275)", 2024-09-20T19:04:54.409 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:57.484341+0000", 2024-09-20T19:04:54.409 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 296.61440244599999, 2024-09-20T19:04:54.409 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.22717348100000001, 2024-09-20T19:04:54.409 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:04:54.409 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:04:54.409 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:54.409 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:04:54.409 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:04:54.410 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:04:54.410 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 101836 2024-09-20T19:04:54.410 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.410 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:04:54.410 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.410 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:04:54.410 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.484341+0000", 2024-09-20T19:04:54.410 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:04:54.410 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.410 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.410 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:04:54.410 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.484341+0000", 2024-09-20T19:04:54.410 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:04:54.411 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.411 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.411 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:04:54.411 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.484343+0000", 2024-09-20T19:04:54.411 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.728e-06 2024-09-20T19:04:54.411 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.411 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.411 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:04:54.411 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.484401+0000", 2024-09-20T19:04:54.411 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8594999999999997e-05 2024-09-20T19:04:54.411 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.411 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.411 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:04:54.411 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.484412+0000", 2024-09-20T19:04:54.411 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1192000000000001e-05 2024-09-20T19:04:54.412 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.412 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.412 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:04:54.412 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.484419+0000", 2024-09-20T19:04:54.412 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.1029999999999996e-06 2024-09-20T19:04:54.412 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.412 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.412 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:04:54.412 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.700675+0000", 2024-09-20T19:04:54.412 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.216255221 2024-09-20T19:04:54.412 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.412 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.412 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:04:54.412 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.700800+0000", 2024-09-20T19:04:54.413 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012584599999999999 2024-09-20T19:04:54.413 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.413 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.413 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:04:54.413 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.700921+0000", 2024-09-20T19:04:54.413 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000120919 2024-09-20T19:04:54.413 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.413 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.413 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:54.413 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.706663+0000", 2024-09-20T19:04:54.413 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0057415230000000001 2024-09-20T19:04:54.413 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.413 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.413 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:04:54.414 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.708002+0000", 2024-09-20T19:04:54.414 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0013395659999999999 2024-09-20T19:04:54.414 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.414 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.414 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:54.414 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.711427+0000", 2024-09-20T19:04:54.414 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0034246900000000002 2024-09-20T19:04:54.414 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.414 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.414 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:04:54.414 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.711478+0000", 2024-09-20T19:04:54.414 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0569000000000003e-05 2024-09-20T19:04:54.414 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.415 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.415 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:04:54.415 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.711514+0000", 2024-09-20T19:04:54.415 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6529000000000002e-05 2024-09-20T19:04:54.415 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:04:54.415 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:04:54.415 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:04:54.415 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.415 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.415 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:101837 9.6 9:60550345:::benchmark_data_smithi083_124552_object101836:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e275)", 2024-09-20T19:04:54.415 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:57.484434+0000", 2024-09-20T19:04:54.415 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 296.61430970800001, 2024-09-20T19:04:54.415 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.233120257, 2024-09-20T19:04:54.415 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:04:54.416 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:04:54.416 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:54.416 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:04:54.416 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:04:54.416 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:04:54.416 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 101837 2024-09-20T19:04:54.416 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.416 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:04:54.416 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.416 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:04:54.416 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.484434+0000", 2024-09-20T19:04:54.416 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:04:54.416 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.416 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.417 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:04:54.417 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.484434+0000", 2024-09-20T19:04:54.417 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:04:54.417 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.417 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.417 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:04:54.417 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.484434+0000", 2024-09-20T19:04:54.417 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7700000000000005e-07 2024-09-20T19:04:54.417 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.417 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.417 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:04:54.417 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.484482+0000", 2024-09-20T19:04:54.417 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8192999999999998e-05 2024-09-20T19:04:54.417 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.417 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.418 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:04:54.418 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.484488+0000", 2024-09-20T19:04:54.418 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.28e-06 2024-09-20T19:04:54.418 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.418 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.418 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:04:54.418 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.484494+0000", 2024-09-20T19:04:54.418 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.1970000000000004e-06 2024-09-20T19:04:54.418 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.418 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.418 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:04:54.418 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.703639+0000", 2024-09-20T19:04:54.418 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.219145598 2024-09-20T19:04:54.418 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.419 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.419 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:04:54.419 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.703764+0000", 2024-09-20T19:04:54.419 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000125019 2024-09-20T19:04:54.419 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.419 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.419 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:04:54.419 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.703910+0000", 2024-09-20T19:04:54.419 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00014603500000000001 2024-09-20T19:04:54.419 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.419 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.419 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:54.419 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.711391+0000", 2024-09-20T19:04:54.419 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0074806459999999996 2024-09-20T19:04:54.420 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.420 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.420 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:54.420 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.711551+0000", 2024-09-20T19:04:54.420 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015964899999999999 2024-09-20T19:04:54.420 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.420 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.420 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:04:54.420 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.717356+0000", 2024-09-20T19:04:54.420 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0058050159999999996 2024-09-20T19:04:54.420 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.421 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.421 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:04:54.421 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.717514+0000", 2024-09-20T19:04:54.421 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015859000000000001 2024-09-20T19:04:54.421 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.421 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.421 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:04:54.421 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.717554+0000", 2024-09-20T19:04:54.421 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9557000000000001e-05 2024-09-20T19:04:54.421 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:04:54.421 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:04:54.421 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:04:54.421 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.421 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.422 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:101840 9.7 9:e11054f3:::benchmark_data_smithi083_124552_object101839:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e275)", 2024-09-20T19:04:54.422 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:57.485731+0000", 2024-09-20T19:04:54.422 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 296.61301226699999, 2024-09-20T19:04:54.422 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.22556601300000001, 2024-09-20T19:04:54.422 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:04:54.422 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:04:54.422 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:54.422 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:04:54.422 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:04:54.422 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:04:54.422 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 101840 2024-09-20T19:04:54.422 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.422 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:04:54.423 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.423 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:04:54.423 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.485731+0000", 2024-09-20T19:04:54.423 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:04:54.423 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.423 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.423 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:04:54.423 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.485731+0000", 2024-09-20T19:04:54.423 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:04:54.423 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.423 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.423 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:04:54.423 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.485733+0000", 2024-09-20T19:04:54.423 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.75e-06 2024-09-20T19:04:54.424 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.424 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.424 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:04:54.424 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.485785+0000", 2024-09-20T19:04:54.424 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.1983e-05 2024-09-20T19:04:54.424 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.424 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.424 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:04:54.424 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.485795+0000", 2024-09-20T19:04:54.424 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.03e-05 2024-09-20T19:04:54.424 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.424 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.424 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:04:54.424 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.485802+0000", 2024-09-20T19:04:54.424 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.1339999999999998e-06 2024-09-20T19:04:54.425 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.425 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.425 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:04:54.425 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.486942+0000", 2024-09-20T19:04:54.425 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001139859 2024-09-20T19:04:54.425 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.425 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.425 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:04:54.425 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.699792+0000", 2024-09-20T19:04:54.425 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.21284998499999999 2024-09-20T19:04:54.425 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.425 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.425 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,7", 2024-09-20T19:04:54.425 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.700390+0000", 2024-09-20T19:04:54.426 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00059842300000000005 2024-09-20T19:04:54.426 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.426 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.426 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:54.426 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.704059+0000", 2024-09-20T19:04:54.426 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0036681230000000001 2024-09-20T19:04:54.426 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.426 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.426 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:04:54.426 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.704078+0000", 2024-09-20T19:04:54.426 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9623000000000002e-05 2024-09-20T19:04:54.426 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.426 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.426 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:54.427 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.711149+0000", 2024-09-20T19:04:54.427 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0070711860000000001 2024-09-20T19:04:54.427 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.427 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.427 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:04:54.427 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.711248+0000", 2024-09-20T19:04:54.427 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.8987000000000006e-05 2024-09-20T19:04:54.427 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.427 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.427 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:04:54.427 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.711297+0000", 2024-09-20T19:04:54.427 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8659999999999998e-05 2024-09-20T19:04:54.427 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:04:54.427 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:04:54.428 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:04:54.428 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.428 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.428 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:124590 9.6 9:6ce021c9:::benchmark_data_smithi083_124552_object10443:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e289)", 2024-09-20T19:04:54.428 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:00:20.955292+0000", 2024-09-20T19:04:54.428 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 273.14345081499999, 2024-09-20T19:04:54.428 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.324912592, 2024-09-20T19:04:54.428 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:04:54.428 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:04:54.428 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:54.428 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:04:54.428 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:04:54.428 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:04:54.428 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 124590 2024-09-20T19:04:54.429 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.429 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:04:54.429 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.429 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:04:54.429 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.955292+0000", 2024-09-20T19:04:54.429 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:04:54.429 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.429 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.429 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:04:54.429 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.955292+0000", 2024-09-20T19:04:54.429 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:04:54.429 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.429 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.429 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:04:54.429 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.955293+0000", 2024-09-20T19:04:54.430 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.8899999999999998e-07 2024-09-20T19:04:54.430 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.430 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.430 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:04:54.430 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.955297+0000", 2024-09-20T19:04:54.430 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6370000000000001e-06 2024-09-20T19:04:54.430 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.430 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.430 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:04:54.430 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.955309+0000", 2024-09-20T19:04:54.430 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.205e-05 2024-09-20T19:04:54.430 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.430 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.431 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:04:54.431 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.955316+0000", 2024-09-20T19:04:54.431 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.7789999999999998e-06 2024-09-20T19:04:54.431 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.431 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.431 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:04:54.431 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.955398+0000", 2024-09-20T19:04:54.431 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.1893999999999999e-05 2024-09-20T19:04:54.431 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.431 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.431 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:04:54.431 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.955512+0000", 2024-09-20T19:04:54.431 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000114036 2024-09-20T19:04:54.431 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.431 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.432 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:04:54.432 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.955606+0000", 2024-09-20T19:04:54.432 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.4199999999999999e-05 2024-09-20T19:04:54.432 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.432 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.432 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:04:54.432 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.958828+0000", 2024-09-20T19:04:54.432 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.003221912 2024-09-20T19:04:54.432 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.432 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.432 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:54.432 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.963128+0000", 2024-09-20T19:04:54.432 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.004300379 2024-09-20T19:04:54.432 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.433 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.433 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:54.433 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:21.280133+0000", 2024-09-20T19:04:54.433 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.317005388 2024-09-20T19:04:54.433 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.433 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.433 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:04:54.433 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:21.280172+0000", 2024-09-20T19:04:54.433 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9014000000000003e-05 2024-09-20T19:04:54.433 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.433 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.433 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:04:54.433 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:21.280205+0000", 2024-09-20T19:04:54.433 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.2514000000000001e-05 2024-09-20T19:04:54.433 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:04:54.434 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:04:54.434 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:04:54.434 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.434 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.434 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:124594 9.6 9:67fbc174:::benchmark_data_smithi083_124552_object10447:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e289)", 2024-09-20T19:04:54.434 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:00:20.960066+0000", 2024-09-20T19:04:54.434 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 273.138677078, 2024-09-20T19:04:54.434 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.32985013699999999, 2024-09-20T19:04:54.434 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:04:54.434 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:04:54.434 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:54.434 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:04:54.434 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:04:54.434 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:04:54.435 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 124594 2024-09-20T19:04:54.435 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.435 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:04:54.435 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.435 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:04:54.435 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.960066+0000", 2024-09-20T19:04:54.435 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:04:54.435 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.435 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.435 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:04:54.435 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.960066+0000", 2024-09-20T19:04:54.435 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:04:54.435 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.435 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.436 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:04:54.436 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.960068+0000", 2024-09-20T19:04:54.436 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.514e-06 2024-09-20T19:04:54.436 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.436 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.436 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:04:54.436 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.960073+0000", 2024-09-20T19:04:54.436 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3040000000000004e-06 2024-09-20T19:04:54.436 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.436 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.436 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:04:54.436 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.960082+0000", 2024-09-20T19:04:54.436 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.4590000000000004e-06 2024-09-20T19:04:54.436 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.437 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.437 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:04:54.437 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.960089+0000", 2024-09-20T19:04:54.437 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.3419999999999996e-06 2024-09-20T19:04:54.437 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.437 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.437 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:04:54.437 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.960147+0000", 2024-09-20T19:04:54.437 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8289999999999999e-05 2024-09-20T19:04:54.437 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.437 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.437 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:04:54.437 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.960255+0000", 2024-09-20T19:04:54.437 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000107666 2024-09-20T19:04:54.437 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.438 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.438 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:04:54.438 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.960363+0000", 2024-09-20T19:04:54.438 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000108186 2024-09-20T19:04:54.438 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.438 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.438 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:04:54.438 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.963253+0000", 2024-09-20T19:04:54.438 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0028899799999999999 2024-09-20T19:04:54.438 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.438 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.438 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:54.438 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.966302+0000", 2024-09-20T19:04:54.439 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.003049169 2024-09-20T19:04:54.439 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.439 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.439 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:54.439 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:21.289803+0000", 2024-09-20T19:04:54.439 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.32350114299999999 2024-09-20T19:04:54.439 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.439 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.439 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:04:54.439 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:21.289861+0000", 2024-09-20T19:04:54.439 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8034e-05 2024-09-20T19:04:54.439 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.439 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.439 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:04:54.439 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:21.289916+0000", 2024-09-20T19:04:54.440 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5050000000000003e-05 2024-09-20T19:04:54.440 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:04:54.440 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:04:54.440 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:04:54.440 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.440 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.440 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:124626 9.6 9:6930ec60:::benchmark_data_smithi083_124552_object10479:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e289)", 2024-09-20T19:04:54.440 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:00:20.983128+0000", 2024-09-20T19:04:54.440 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 273.11561513800001, 2024-09-20T19:04:54.440 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30694355299999998, 2024-09-20T19:04:54.440 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:04:54.440 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:04:54.440 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:54.440 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:04:54.441 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:04:54.441 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:04:54.441 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 124626 2024-09-20T19:04:54.441 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.441 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:04:54.441 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.441 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:04:54.441 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.983128+0000", 2024-09-20T19:04:54.441 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:04:54.441 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.441 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.441 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:04:54.441 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.983128+0000", 2024-09-20T19:04:54.441 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:04:54.442 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.442 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.442 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:04:54.442 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.983130+0000", 2024-09-20T19:04:54.442 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.426e-06 2024-09-20T19:04:54.442 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.442 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.442 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:04:54.442 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.983134+0000", 2024-09-20T19:04:54.442 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.4309999999999996e-06 2024-09-20T19:04:54.442 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.442 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.442 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:04:54.442 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.983144+0000", 2024-09-20T19:04:54.443 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0023e-05 2024-09-20T19:04:54.443 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.443 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.443 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:04:54.443 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.983149+0000", 2024-09-20T19:04:54.443 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.9949999999999996e-06 2024-09-20T19:04:54.443 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.443 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.443 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:04:54.443 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.983201+0000", 2024-09-20T19:04:54.443 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.2312999999999999e-05 2024-09-20T19:04:54.443 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.443 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.443 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:04:54.444 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.983303+0000", 2024-09-20T19:04:54.444 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000101433 2024-09-20T19:04:54.444 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.444 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.444 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:04:54.444 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.983418+0000", 2024-09-20T19:04:54.444 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000114851 2024-09-20T19:04:54.444 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.444 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.444 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:04:54.444 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.987483+0000", 2024-09-20T19:04:54.444 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0040654990000000002 2024-09-20T19:04:54.444 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.444 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.444 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:54.445 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.989475+0000", 2024-09-20T19:04:54.445 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001992204 2024-09-20T19:04:54.445 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.445 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.445 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:54.445 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:21.289975+0000", 2024-09-20T19:04:54.445 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30049981100000001 2024-09-20T19:04:54.445 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.445 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.445 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:04:54.445 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:21.290025+0000", 2024-09-20T19:04:54.445 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0009000000000002e-05 2024-09-20T19:04:54.445 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.445 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.446 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:04:54.446 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:21.290072+0000", 2024-09-20T19:04:54.446 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.6557999999999997e-05 2024-09-20T19:04:54.446 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:04:54.446 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:04:54.446 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:04:54.446 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.446 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.446 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:124628 9.6 9:6a933d55:::benchmark_data_smithi083_124552_object10481:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e289)", 2024-09-20T19:04:54.446 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:00:20.983388+0000", 2024-09-20T19:04:54.446 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 273.11535566399999, 2024-09-20T19:04:54.446 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30682658800000001, 2024-09-20T19:04:54.446 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:04:54.447 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:04:54.447 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:54.447 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:04:54.447 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:04:54.447 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:04:54.447 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 124628 2024-09-20T19:04:54.447 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.447 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:04:54.447 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.447 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:04:54.447 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.983388+0000", 2024-09-20T19:04:54.447 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:04:54.447 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.447 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.447 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:04:54.448 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.983388+0000", 2024-09-20T19:04:54.448 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:04:54.448 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.448 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.448 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:04:54.448 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.983388+0000", 2024-09-20T19:04:54.448 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.9699999999999995e-07 2024-09-20T19:04:54.448 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.448 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.448 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:04:54.448 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.983394+0000", 2024-09-20T19:04:54.448 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0229999999999998e-06 2024-09-20T19:04:54.448 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.448 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.449 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:04:54.449 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.983402+0000", 2024-09-20T19:04:54.449 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.1079999999999993e-06 2024-09-20T19:04:54.449 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.449 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.449 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:04:54.449 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.983409+0000", 2024-09-20T19:04:54.449 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.6789999999999999e-06 2024-09-20T19:04:54.449 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.449 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.449 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:04:54.449 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.985457+0000", 2024-09-20T19:04:54.449 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.002048322 2024-09-20T19:04:54.449 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.450 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.450 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:04:54.450 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.985569+0000", 2024-09-20T19:04:54.450 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011113899999999999 2024-09-20T19:04:54.450 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.450 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.450 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:04:54.450 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.985636+0000", 2024-09-20T19:04:54.450 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.7665999999999999e-05 2024-09-20T19:04:54.450 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.450 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.450 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:04:54.450 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.989267+0000", 2024-09-20T19:04:54.451 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0036308569999999999 2024-09-20T19:04:54.451 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.451 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.451 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:54.451 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.989524+0000", 2024-09-20T19:04:54.451 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00025713199999999998 2024-09-20T19:04:54.451 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.451 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.451 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:54.451 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:21.290125+0000", 2024-09-20T19:04:54.451 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30060116100000001 2024-09-20T19:04:54.451 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.451 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.451 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:04:54.454 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:21.290176+0000", 2024-09-20T19:04:54.454 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0964000000000002e-05 2024-09-20T19:04:54.454 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.454 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.454 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:04:54.454 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:21.290214+0000", 2024-09-20T19:04:54.454 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.7740000000000001e-05 2024-09-20T19:04:54.454 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:04:54.454 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:04:54.454 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:04:54.455 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.455 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.455 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:124629 9.6 9:6a9ad403:::benchmark_data_smithi083_124552_object10482:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e289)", 2024-09-20T19:04:54.455 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:00:20.986104+0000", 2024-09-20T19:04:54.455 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 273.11263908400002, 2024-09-20T19:04:54.455 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30423246900000001, 2024-09-20T19:04:54.455 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:04:54.455 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:04:54.455 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:54.455 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:04:54.455 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:04:54.455 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:04:54.455 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 124629 2024-09-20T19:04:54.455 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.456 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:04:54.456 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.456 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:04:54.456 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.986104+0000", 2024-09-20T19:04:54.456 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:04:54.456 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.456 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.456 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:04:54.456 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.986104+0000", 2024-09-20T19:04:54.456 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:04:54.456 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.457 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.457 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:04:54.457 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.986105+0000", 2024-09-20T19:04:54.457 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.077e-06 2024-09-20T19:04:54.457 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.457 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.457 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:04:54.457 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.986108+0000", 2024-09-20T19:04:54.457 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.2100000000000002e-06 2024-09-20T19:04:54.457 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.457 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.457 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:04:54.457 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.986117+0000", 2024-09-20T19:04:54.457 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.037e-06 2024-09-20T19:04:54.457 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.458 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.458 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:04:54.458 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.986125+0000", 2024-09-20T19:04:54.458 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.3320000000000003e-06 2024-09-20T19:04:54.458 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.458 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.458 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:04:54.458 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.987508+0000", 2024-09-20T19:04:54.458 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0013835099999999999 2024-09-20T19:04:54.458 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.458 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.458 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:04:54.458 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.987586+0000", 2024-09-20T19:04:54.458 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.7923999999999995e-05 2024-09-20T19:04:54.459 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.459 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.459 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:04:54.459 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.987672+0000", 2024-09-20T19:04:54.459 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.6081999999999993e-05 2024-09-20T19:04:54.459 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.459 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.459 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:04:54.459 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.989531+0000", 2024-09-20T19:04:54.459 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0018587300000000001 2024-09-20T19:04:54.459 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.459 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.459 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:54.459 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.989591+0000", 2024-09-20T19:04:54.460 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0047000000000003e-05 2024-09-20T19:04:54.460 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.460 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.460 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:54.460 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:21.290255+0000", 2024-09-20T19:04:54.460 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30066345700000002 2024-09-20T19:04:54.460 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.460 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.460 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:04:54.460 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:21.290302+0000", 2024-09-20T19:04:54.460 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7559999999999999e-05 2024-09-20T19:04:54.460 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.460 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.460 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:04:54.461 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:21.290337+0000", 2024-09-20T19:04:54.461 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4502999999999998e-05 2024-09-20T19:04:54.461 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:04:54.461 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:04:54.461 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:04:54.461 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.461 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.461 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:124634 9.6 9:6166b273:::benchmark_data_smithi083_124552_object10487:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e289)", 2024-09-20T19:04:54.461 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:00:20.993979+0000", 2024-09-20T19:04:54.461 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 273.10476459199998, 2024-09-20T19:04:54.461 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.297307777, 2024-09-20T19:04:54.461 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:04:54.461 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:04:54.461 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:54.461 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:04:54.462 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:04:54.462 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:04:54.462 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 124634 2024-09-20T19:04:54.462 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.462 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:04:54.462 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.462 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:04:54.462 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.993979+0000", 2024-09-20T19:04:54.462 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:04:54.462 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.462 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.462 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:04:54.462 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.993979+0000", 2024-09-20T19:04:54.463 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:04:54.463 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.463 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.463 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:04:54.463 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.993979+0000", 2024-09-20T19:04:54.463 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5799999999999998e-07 2024-09-20T19:04:54.463 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.463 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.463 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:04:54.463 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.993983+0000", 2024-09-20T19:04:54.463 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4599999999999999e-06 2024-09-20T19:04:54.463 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.463 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.463 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:04:54.464 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.993993+0000", 2024-09-20T19:04:54.464 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0081000000000001e-05 2024-09-20T19:04:54.464 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.464 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.464 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:04:54.464 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.993999+0000", 2024-09-20T19:04:54.464 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.772e-06 2024-09-20T19:04:54.464 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.464 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.464 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:04:54.464 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.994054+0000", 2024-09-20T19:04:54.464 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5692000000000001e-05 2024-09-20T19:04:54.464 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.464 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.465 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:04:54.465 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.994124+0000", 2024-09-20T19:04:54.465 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.9999999999999994e-05 2024-09-20T19:04:54.465 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.465 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.465 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:04:54.465 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.994189+0000", 2024-09-20T19:04:54.465 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4448000000000004e-05 2024-09-20T19:04:54.465 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.465 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.465 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:04:54.465 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.996228+0000", 2024-09-20T19:04:54.465 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0020387280000000001 2024-09-20T19:04:54.465 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.466 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.466 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:54.466 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.996462+0000", 2024-09-20T19:04:54.466 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00023460199999999999 2024-09-20T19:04:54.466 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.466 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.466 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:54.466 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:21.291206+0000", 2024-09-20T19:04:54.466 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.29474367899999998 2024-09-20T19:04:54.466 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.466 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.466 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:04:54.467 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:21.291253+0000", 2024-09-20T19:04:54.467 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.6903000000000001e-05 2024-09-20T19:04:54.467 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.467 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.467 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:04:54.467 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:21.291286+0000", 2024-09-20T19:04:54.467 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.3654e-05 2024-09-20T19:04:54.467 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:04:54.467 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:04:54.467 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:04:54.467 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.468 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.468 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14460.0:2699 5.e 5:7759931f:::notify.3:head [watch reconnect cookie 94056120170112 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e292)", 2024-09-20T19:04:54.468 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:00:35.180070+0000", 2024-09-20T19:04:54.468 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 258.91867327599999, 2024-09-20T19:04:54.468 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0139281419999999, 2024-09-20T19:04:54.468 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:04:54.468 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:04:54.468 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-09-20T19:04:54.468 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:04:54.468 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14460", 2024-09-20T19:04:54.468 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1555339426", 2024-09-20T19:04:54.468 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 2699 2024-09-20T19:04:54.468 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.468 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:04:54.469 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.469 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:04:54.469 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.180070+0000", 2024-09-20T19:04:54.469 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:04:54.469 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.469 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.469 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:04:54.469 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.180070+0000", 2024-09-20T19:04:54.469 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:04:54.469 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.469 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.469 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:04:54.469 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.180071+0000", 2024-09-20T19:04:54.469 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.093e-06 2024-09-20T19:04:54.469 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.470 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.470 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:04:54.470 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.180077+0000", 2024-09-20T19:04:54.470 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5079999999999996e-06 2024-09-20T19:04:54.470 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.470 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.470 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:04:54.470 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.180086+0000", 2024-09-20T19:04:54.470 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.9499999999999996e-06 2024-09-20T19:04:54.470 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.470 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.470 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:04:54.470 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.180094+0000", 2024-09-20T19:04:54.471 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5780000000000001e-06 2024-09-20T19:04:54.471 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.471 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.471 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:04:54.471 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.180159+0000", 2024-09-20T19:04:54.471 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5139000000000001e-05 2024-09-20T19:04:54.471 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.471 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.471 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op must wait for map", 2024-09-20T19:04:54.471 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.180165+0000", 2024-09-20T19:04:54.471 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.7629999999999997e-06 2024-09-20T19:04:54.471 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.471 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.471 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:04:54.471 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.182672+0000", 2024-09-20T19:04:54.472 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0025072710000000002 2024-09-20T19:04:54.472 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.472 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.472 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:04:54.472 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.182681+0000", 2024-09-20T19:04:54.472 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.4780000000000008e-06 2024-09-20T19:04:54.472 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.472 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.472 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:04:54.472 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.191896+0000", 2024-09-20T19:04:54.472 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.009215755 2024-09-20T19:04:54.472 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.472 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.472 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:04:54.473 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.191989+0000", 2024-09-20T19:04:54.473 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.2868000000000006e-05 2024-09-20T19:04:54.473 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.473 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.473 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:04:54.473 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.193998+0000", 2024-09-20T19:04:54.473 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0020087389999999998 2024-09-20T19:04:54.473 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:04:54.473 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:04:54.473 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:04:54.473 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.473 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.473 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:147626 10.9 10:98e2f0b0:::benchmark_data_smithi083_148100_object30764:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e376)", 2024-09-20T19:04:54.473 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:04:28.118535+0000", 2024-09-20T19:04:54.473 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 25.980208054999999, 2024-09-20T19:04:54.474 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.40251810599999999, 2024-09-20T19:04:54.474 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:04:54.474 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:04:54.474 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:54.474 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:04:54.474 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:04:54.474 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:04:54.474 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 147626 2024-09-20T19:04:54.474 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.474 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:04:54.474 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.474 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:04:54.474 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.118535+0000", 2024-09-20T19:04:54.474 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:04:54.475 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.475 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.475 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:04:54.475 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.118535+0000", 2024-09-20T19:04:54.475 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:04:54.475 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.475 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.475 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:04:54.475 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.118537+0000", 2024-09-20T19:04:54.475 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7910000000000001e-06 2024-09-20T19:04:54.475 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.475 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.475 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:04:54.475 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.118542+0000", 2024-09-20T19:04:54.476 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4049999999999999e-06 2024-09-20T19:04:54.476 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.476 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.476 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:04:54.476 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.118556+0000", 2024-09-20T19:04:54.476 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3988999999999999e-05 2024-09-20T19:04:54.476 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.476 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.476 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:04:54.476 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.118564+0000", 2024-09-20T19:04:54.476 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.0979999999999994e-06 2024-09-20T19:04:54.476 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.476 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.476 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:04:54.477 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.118622+0000", 2024-09-20T19:04:54.477 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.7796e-05 2024-09-20T19:04:54.477 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.477 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.477 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:04:54.477 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.118757+0000", 2024-09-20T19:04:54.477 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013468599999999999 2024-09-20T19:04:54.477 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.477 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.477 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 2,7", 2024-09-20T19:04:54.477 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.118877+0000", 2024-09-20T19:04:54.477 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000120103 2024-09-20T19:04:54.477 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.477 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.477 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:04:54.478 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.121979+0000", 2024-09-20T19:04:54.478 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0031019569999999998 2024-09-20T19:04:54.478 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.478 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.478 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:54.478 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.122589+0000", 2024-09-20T19:04:54.478 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00061044199999999999 2024-09-20T19:04:54.478 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.478 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.478 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:54.478 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.520939+0000", 2024-09-20T19:04:54.478 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.39834966599999999 2024-09-20T19:04:54.478 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.478 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.479 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:04:54.479 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.520995+0000", 2024-09-20T19:04:54.479 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5405999999999998e-05 2024-09-20T19:04:54.479 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.479 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.479 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:04:54.479 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.521053+0000", 2024-09-20T19:04:54.479 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8767000000000002e-05 2024-09-20T19:04:54.479 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:04:54.479 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:04:54.479 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:04:54.479 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.479 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.480 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:147635 10.9 10:95751c44:::benchmark_data_smithi083_148100_object30773:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e376)", 2024-09-20T19:04:54.480 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:04:28.131221+0000", 2024-09-20T19:04:54.480 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 25.967522455000001, 2024-09-20T19:04:54.480 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.398357515, 2024-09-20T19:04:54.480 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:04:54.480 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:04:54.480 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:54.480 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:04:54.480 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:04:54.480 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:04:54.480 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 147635 2024-09-20T19:04:54.480 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.480 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:04:54.480 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.481 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:04:54.481 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.131221+0000", 2024-09-20T19:04:54.481 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:04:54.481 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.481 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.481 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:04:54.481 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.131221+0000", 2024-09-20T19:04:54.481 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:04:54.481 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.481 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.481 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:04:54.481 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.131222+0000", 2024-09-20T19:04:54.481 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.575e-06 2024-09-20T19:04:54.481 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.482 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.482 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:04:54.482 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.131229+0000", 2024-09-20T19:04:54.482 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2740000000000004e-06 2024-09-20T19:04:54.482 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.482 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.482 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:04:54.482 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.131244+0000", 2024-09-20T19:04:54.482 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5362e-05 2024-09-20T19:04:54.482 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.482 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.482 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:04:54.482 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.131252+0000", 2024-09-20T19:04:54.482 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.3040000000000005e-06 2024-09-20T19:04:54.483 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.483 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.483 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:04:54.483 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.131312+0000", 2024-09-20T19:04:54.483 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9703000000000001e-05 2024-09-20T19:04:54.483 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.483 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.483 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:04:54.483 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.131421+0000", 2024-09-20T19:04:54.483 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00010873400000000001 2024-09-20T19:04:54.483 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.483 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.483 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 2,7", 2024-09-20T19:04:54.483 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.131541+0000", 2024-09-20T19:04:54.484 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000120625 2024-09-20T19:04:54.484 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.484 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.484 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:04:54.484 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.134390+0000", 2024-09-20T19:04:54.484 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0028481959999999999 2024-09-20T19:04:54.484 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.484 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.484 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:54.484 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.135079+0000", 2024-09-20T19:04:54.484 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00068968800000000004 2024-09-20T19:04:54.484 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.484 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.484 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:54.485 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.529488+0000", 2024-09-20T19:04:54.485 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.39440882500000002 2024-09-20T19:04:54.485 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.485 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.485 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:04:54.485 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.529539+0000", 2024-09-20T19:04:54.485 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.1038000000000002e-05 2024-09-20T19:04:54.485 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.485 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.485 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:04:54.485 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.529578+0000", 2024-09-20T19:04:54.485 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9190999999999998e-05 2024-09-20T19:04:54.485 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:04:54.485 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:04:54.485 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:04:54.486 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.486 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.486 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:147639 10.9 10:96d58120:::benchmark_data_smithi083_148100_object30777:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e376)", 2024-09-20T19:04:54.486 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:04:28.138921+0000", 2024-09-20T19:04:54.486 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 25.959822412000001, 2024-09-20T19:04:54.486 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.39076443700000002, 2024-09-20T19:04:54.486 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:04:54.486 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:04:54.486 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:54.486 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:04:54.486 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:04:54.486 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:04:54.486 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 147639 2024-09-20T19:04:54.486 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.487 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:04:54.487 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.487 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:04:54.487 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.138921+0000", 2024-09-20T19:04:54.487 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:04:54.487 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.487 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.487 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:04:54.487 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.138921+0000", 2024-09-20T19:04:54.487 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:04:54.487 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.487 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.487 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:04:54.487 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.138922+0000", 2024-09-20T19:04:54.488 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.598e-06 2024-09-20T19:04:54.488 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.488 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.488 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:04:54.488 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.138929+0000", 2024-09-20T19:04:54.488 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2890000000000003e-06 2024-09-20T19:04:54.488 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.488 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.488 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:04:54.488 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.138940+0000", 2024-09-20T19:04:54.488 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1292e-05 2024-09-20T19:04:54.488 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.488 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.488 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:04:54.488 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.138948+0000", 2024-09-20T19:04:54.489 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.7190000000000001e-06 2024-09-20T19:04:54.489 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.489 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.489 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:04:54.489 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.139013+0000", 2024-09-20T19:04:54.489 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5686999999999996e-05 2024-09-20T19:04:54.489 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.489 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.489 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:04:54.489 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.139133+0000", 2024-09-20T19:04:54.489 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000119709 2024-09-20T19:04:54.489 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.489 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.489 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 2,7", 2024-09-20T19:04:54.490 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.139232+0000", 2024-09-20T19:04:54.490 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.9275000000000001e-05 2024-09-20T19:04:54.490 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.490 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.490 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:04:54.490 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.142295+0000", 2024-09-20T19:04:54.490 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0030624210000000001 2024-09-20T19:04:54.490 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.490 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.490 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:54.490 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.142766+0000", 2024-09-20T19:04:54.490 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00047071800000000002 2024-09-20T19:04:54.490 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.490 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.491 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:54.491 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.529610+0000", 2024-09-20T19:04:54.491 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.38684496499999999 2024-09-20T19:04:54.491 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.491 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.491 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:04:54.491 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.529654+0000", 2024-09-20T19:04:54.491 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3988e-05 2024-09-20T19:04:54.491 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.491 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.491 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:04:54.491 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.529685+0000", 2024-09-20T19:04:54.491 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.0775999999999998e-05 2024-09-20T19:04:54.491 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:04:54.492 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:04:54.492 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:04:54.492 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.492 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.492 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:147641 10.9 10:957d4f1f:::benchmark_data_smithi083_148100_object30779:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e376)", 2024-09-20T19:04:54.492 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:04:28.142745+0000", 2024-09-20T19:04:54.492 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 25.955998645000001, 2024-09-20T19:04:54.492 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.38705916899999998, 2024-09-20T19:04:54.492 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:04:54.492 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:04:54.492 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:54.492 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:04:54.492 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:04:54.492 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:04:54.493 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 147641 2024-09-20T19:04:54.493 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.493 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:04:54.493 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.493 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:04:54.493 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.142745+0000", 2024-09-20T19:04:54.493 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:04:54.493 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.493 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.493 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:04:54.493 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.142745+0000", 2024-09-20T19:04:54.493 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:04:54.493 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.493 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.493 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:04:54.494 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.142746+0000", 2024-09-20T19:04:54.494 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7680000000000001e-06 2024-09-20T19:04:54.494 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.494 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.494 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:04:54.494 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.142752+0000", 2024-09-20T19:04:54.494 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.7019999999999998e-06 2024-09-20T19:04:54.494 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.494 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.494 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:04:54.494 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.142761+0000", 2024-09-20T19:04:54.494 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.4190000000000008e-06 2024-09-20T19:04:54.494 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.494 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.495 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:04:54.495 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.142768+0000", 2024-09-20T19:04:54.495 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.8009999999999998e-06 2024-09-20T19:04:54.495 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.495 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.495 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:04:54.495 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.142829+0000", 2024-09-20T19:04:54.495 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0537e-05 2024-09-20T19:04:54.495 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.495 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.495 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:04:54.495 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.142951+0000", 2024-09-20T19:04:54.495 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000122619 2024-09-20T19:04:54.495 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.496 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.496 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 2,7", 2024-09-20T19:04:54.496 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.143050+0000", 2024-09-20T19:04:54.496 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.8121999999999993e-05 2024-09-20T19:04:54.496 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.496 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.496 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:54.496 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.145907+0000", 2024-09-20T19:04:54.496 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0028578789999999998 2024-09-20T19:04:54.496 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.496 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.496 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:04:54.496 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.145926+0000", 2024-09-20T19:04:54.496 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8272000000000001e-05 2024-09-20T19:04:54.497 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.497 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.497 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:54.497 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.529726+0000", 2024-09-20T19:04:54.497 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.38380014200000001 2024-09-20T19:04:54.497 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.497 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.497 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:04:54.497 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.529774+0000", 2024-09-20T19:04:54.497 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8279999999999999e-05 2024-09-20T19:04:54.497 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.497 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.497 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:04:54.497 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.529804+0000", 2024-09-20T19:04:54.497 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9628000000000002e-05 2024-09-20T19:04:54.498 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:04:54.498 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:04:54.498 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:04:54.498 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.498 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.498 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:147647 10.9 10:988ad65f:::benchmark_data_smithi083_148100_object30785:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e376)", 2024-09-20T19:04:54.498 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:04:28.151581+0000", 2024-09-20T19:04:54.498 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 25.947161939000001, 2024-09-20T19:04:54.498 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.37836669499999998, 2024-09-20T19:04:54.498 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:04:54.498 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:04:54.498 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:04:54.498 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:04:54.499 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:04:54.499 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:04:54.499 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 147647 2024-09-20T19:04:54.499 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.499 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:04:54.499 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.499 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:04:54.499 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.151581+0000", 2024-09-20T19:04:54.499 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:04:54.499 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.499 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.499 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:04:54.499 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.151581+0000", 2024-09-20T19:04:54.499 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:04:54.499 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.500 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.500 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:04:54.500 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.151584+0000", 2024-09-20T19:04:54.500 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.5349999999999999e-06 2024-09-20T19:04:54.500 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.500 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.500 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:04:54.500 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.151590+0000", 2024-09-20T19:04:54.500 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9560000000000002e-06 2024-09-20T19:04:54.500 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.500 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.500 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:04:54.500 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.151605+0000", 2024-09-20T19:04:54.500 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5553000000000002e-05 2024-09-20T19:04:54.501 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.501 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.501 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:04:54.501 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.151615+0000", 2024-09-20T19:04:54.501 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.6160000000000008e-06 2024-09-20T19:04:54.501 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.501 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.501 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:04:54.501 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.151678+0000", 2024-09-20T19:04:54.501 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2650000000000005e-05 2024-09-20T19:04:54.501 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.501 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.501 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:04:54.501 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.151816+0000", 2024-09-20T19:04:54.501 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013797399999999999 2024-09-20T19:04:54.502 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.502 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.502 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 2,7", 2024-09-20T19:04:54.502 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.151936+0000", 2024-09-20T19:04:54.502 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012091699999999999 2024-09-20T19:04:54.502 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.502 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.502 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:54.502 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.154743+0000", 2024-09-20T19:04:54.502 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0028064359999999998 2024-09-20T19:04:54.502 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.502 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.502 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:04:54.502 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.154877+0000", 2024-09-20T19:04:54.503 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000133893 2024-09-20T19:04:54.503 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.503 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.503 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:04:54.503 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.529857+0000", 2024-09-20T19:04:54.503 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.37498010799999998 2024-09-20T19:04:54.503 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.503 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.503 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:04:54.503 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.529911+0000", 2024-09-20T19:04:54.503 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3785000000000001e-05 2024-09-20T19:04:54.503 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:04:54.503 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:04:54.503 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:04:54.504 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.529948+0000", 2024-09-20T19:04:54.504 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.7271999999999998e-05 2024-09-20T19:04:54.504 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:04:54.504 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:04:54.504 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:04:54.504 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:04:54.504 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:04:54.504 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T19:04:54.504 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:54.553 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:04:54.553 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:04:54.568 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:54.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:54.569 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:54.586 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:04:54.586 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:04:54.631 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:04:54.632 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:04:54.664 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:04:54.669 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:04:54.669 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:04:54.670 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:54.683 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:04:54.701 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:54.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:54.702 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:54.711 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:04:54.711 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:04:54.752 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:54 smithi083 ceph-mon[29421]: pgmap v751: 148 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 142 active+clean; 4.0 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 682 B/s rd, 0 B/s wr, 860 op/s; 5/195459 objects degraded (0.003%); 9351/195459 objects misplaced (4.784%); 6.2 MiB/s, 97 objects/s recovering 2024-09-20T19:04:54.753 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:04:54.753 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:04:54.753 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:54 smithi083 ceph-mon[36260]: pgmap v751: 148 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 142 active+clean; 4.0 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 682 B/s rd, 0 B/s wr, 860 op/s; 5/195459 objects degraded (0.003%); 9351/195459 objects misplaced (4.784%); 6.2 MiB/s, 97 objects/s recovering 2024-09-20T19:04:54.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:54 smithi142 ceph-mon[32508]: pgmap v751: 148 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 142 active+clean; 4.0 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 682 B/s rd, 0 B/s wr, 860 op/s; 5/195459 objects degraded (0.003%); 9351/195459 objects misplaced (4.784%); 6.2 MiB/s, 97 objects/s recovering 2024-09-20T19:04:54.788 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:04:54.789 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:04:54.803 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:54.819 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:04:54.820 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:04:54.843 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:54.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:54.844 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:54.945 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:54.987 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:54.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:54.988 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:55.088 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:55.122 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:55.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:55.123 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:55.224 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:04:55.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:55.270 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:55.270 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:04:55.371 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:55.407 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:55.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:55.408 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:55.509 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:55.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:55 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1781517887' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:04:55.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:55 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1781517887' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:04:55.543 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:55.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:55.545 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:55.646 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:55.678 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:55.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:55.682 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:55.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:55 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1781517887' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:04:55.783 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:04:55.809 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:55.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:55.810 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:04:55.911 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:04:55.932 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:55.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:55.934 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:04:56.034 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:56.067 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:56.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:56.068 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:56.169 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:04:56.189 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:56.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:56.190 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:04:56.291 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:56.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:56.325 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:56.325 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:56.426 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:56.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:56.462 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:56.462 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:56.563 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:56.599 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:56.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:56.600 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:56.701 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:56.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:56.723 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:56.723 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:56.729 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:56 smithi083 ceph-mon[29421]: pgmap v752: 148 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 142 active+clean; 4.0 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 606 B/s rd, 0 B/s wr, 641 op/s; 5/195459 objects degraded (0.003%); 9351/195459 objects misplaced (4.784%); 5.5 MiB/s, 86 objects/s recovering 2024-09-20T19:04:56.729 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:56 smithi083 ceph-mon[36260]: pgmap v752: 148 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 142 active+clean; 4.0 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 606 B/s rd, 0 B/s wr, 641 op/s; 5/195459 objects degraded (0.003%); 9351/195459 objects misplaced (4.784%); 5.5 MiB/s, 86 objects/s recovering 2024-09-20T19:04:56.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:56 smithi142 ceph-mon[32508]: pgmap v752: 148 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 142 active+clean; 4.0 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 606 B/s rd, 0 B/s wr, 641 op/s; 5/195459 objects degraded (0.003%); 9351/195459 objects misplaced (4.784%); 5.5 MiB/s, 86 objects/s recovering 2024-09-20T19:04:56.824 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:56.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:56.863 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:56.863 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:56.964 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:04:57.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:57.009 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:57.009 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:04:57.110 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:57.144 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:57.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:57.146 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:57.246 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:04:57.280 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:57.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:57.281 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:04:57.382 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:57.418 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:57.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:57.419 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:57.483 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-20T19:04:57.485 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.0 to 1 (10000) 2024-09-20T19:04:57.520 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:57.548 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:57.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:57.549 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:57.649 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:57.693 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:57.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:57.694 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:57.762 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:04:57 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:04:57.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:57 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/64420873' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-20T19:04:57.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:57 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-20T19:04:57.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:57 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1007298484' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:04:57.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:57 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:04:57.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:57 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/64420873' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-20T19:04:57.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:57 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-20T19:04:57.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:57 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1007298484' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:04:57.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:57 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:04:57.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:57 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/64420873' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-20T19:04:57.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:57 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-20T19:04:57.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:57 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1007298484' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:04:57.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:57 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:04:57.795 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:57.836 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:57.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:57.837 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:57.938 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:04:57.967 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:57.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:57.968 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:04:58.069 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:58.107 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:58.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:58.108 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:58.209 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:04:58.234 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:58.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:58.235 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:04:58.336 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:04:58.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:58.368 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:58.368 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:04:58.469 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:04:58.480 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:58 smithi083 ceph-mon[36260]: pgmap v753: 148 pgs: 1 active+recovery_wait+degraded, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 142 active+clean; 3.7 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.04k op/s; 5/183540 objects degraded (0.003%); 8369/183540 objects misplaced (4.560%); 9.9 MiB/s, 157 objects/s recovering 2024-09-20T19:04:58.480 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:58 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:04:58.480 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:58 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:04:58.480 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:58 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:04:58.480 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:58 smithi083 ceph-mon[36260]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:04:58.480 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:58 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-20T19:04:58.480 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:58 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T19:04:58.501 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:58.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:58.502 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:04:58.603 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:58.636 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:58.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:58.637 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:58.738 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:04:58.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:58 smithi142 ceph-mon[32508]: pgmap v753: 148 pgs: 1 active+recovery_wait+degraded, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 142 active+clean; 3.7 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.04k op/s; 5/183540 objects degraded (0.003%); 8369/183540 objects misplaced (4.560%); 9.9 MiB/s, 157 objects/s recovering 2024-09-20T19:04:58.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:58 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:04:58.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:58 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:04:58.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:58 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:04:58.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:58 smithi142 ceph-mon[32508]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:04:58.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:58 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-20T19:04:58.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:58 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T19:04:58.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:58 smithi142 ceph-mon[32508]: osdmap e384: 8 total, 8 up, 8 in 2024-09-20T19:04:58.775 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T19:04:58.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:58.777 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:58.777 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:04:58.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:58 smithi083 ceph-mon[29421]: pgmap v753: 148 pgs: 1 active+recovery_wait+degraded, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 142 active+clean; 3.7 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.04k op/s; 5/183540 objects degraded (0.003%); 8369/183540 objects misplaced (4.560%); 9.9 MiB/s, 157 objects/s recovering 2024-09-20T19:04:58.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:58 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:04:58.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:58 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:04:58.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:58 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:04:58.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:58 smithi083 ceph-mon[29421]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:04:58.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:58 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-20T19:04:58.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:58 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T19:04:58.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:58 smithi083 ceph-mon[29421]: osdmap e384: 8 total, 8 up, 8 in 2024-09-20T19:04:58.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:58 smithi083 ceph-mon[36260]: osdmap e384: 8 total, 8 up, 8 in 2024-09-20T19:04:58.878 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:58.899 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd reweight 1 1 2024-09-20T19:04:58.907 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:58.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:58.908 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:59.009 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:04:59.044 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:59.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:59.045 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:04:59.146 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:59.188 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:59.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:59.189 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:59.218 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:04:59.290 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:04:59.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:59.322 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:59.322 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:04:59.420 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-20T19:04:59.422 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:59.453 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:59.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:59.454 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:59.554 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:04:59.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:59.604 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:59.604 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:04:59.640 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:04:59 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 5/181005 objects degraded (0.003%), 1 pg degraded (PG_DEGRADED) 2024-09-20T19:04:59.641 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:04:59 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 5/181005 objects degraded (0.003%), 1 pg degraded (PG_DEGRADED) 2024-09-20T19:04:59.704 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:04:59.749 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:04:59.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:59.750 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:04:59.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:04:59 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 5/181005 objects degraded (0.003%), 1 pg degraded (PG_DEGRADED) 2024-09-20T19:04:59.791 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:04:59.791 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:04:59.819 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:04:59.819 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:04:59.824 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:04:59.850 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:04:59.860 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:04:59.861 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:04:59.875 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:04:59.892 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:04:59.892 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:04:59.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:04:59.896 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:04:59.896 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:04:59.935 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:04:59.935 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:04:59.967 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:04:59.967 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:04:59.998 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:00.005 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:05:00.005 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:05:00.037 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:00.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:00.038 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:00.040 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:05:00.040 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:05:00.138 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:00.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:00.184 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:00.184 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:00.194 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T19:05:00.284 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:00.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:00.322 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:00.323 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:00.423 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:00.454 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:00.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:00.455 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:00.515 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T19:05:00.516 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-09-20T19:05:00.516 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.516 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:177147 10.0 10:0adf2218:::benchmark_data_smithi083_148100_object60285:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e384)", 2024-09-20T19:05:00.516 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:00.487281+0000", 2024-09-20T19:05:00.516 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.021365037999999999, 2024-09-20T19:05:00.516 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.021395384999999999, 2024-09-20T19:05:00.516 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:05:00.517 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:05:00.517 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "waiting for sub ops", 2024-09-20T19:05:00.517 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:05:00.517 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:05:00.517 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:05:00.517 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 177147 2024-09-20T19:05:00.518 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.518 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:05:00.518 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.518 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:05:00.518 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.487281+0000", 2024-09-20T19:05:00.518 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:05:00.519 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.519 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.519 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:05:00.519 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.487281+0000", 2024-09-20T19:05:00.519 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:05:00.519 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.520 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.520 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:05:00.520 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.487281+0000", 2024-09-20T19:05:00.520 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3099999999999998e-07 2024-09-20T19:05:00.520 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.520 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.520 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:05:00.521 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.487284+0000", 2024-09-20T19:05:00.521 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3609999999999999e-06 2024-09-20T19:05:00.521 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.521 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.521 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:05:00.521 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.487290+0000", 2024-09-20T19:05:00.522 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.02e-06 2024-09-20T19:05:00.522 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.523 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.523 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:05:00.523 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.487295+0000", 2024-09-20T19:05:00.523 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3480000000000003e-06 2024-09-20T19:05:00.523 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.523 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.523 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:05:00.523 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.491210+0000", 2024-09-20T19:05:00.523 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0039152550000000003 2024-09-20T19:05:00.523 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.523 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.523 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:05:00.524 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.491320+0000", 2024-09-20T19:05:00.524 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000109937 2024-09-20T19:05:00.524 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.524 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.524 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,4", 2024-09-20T19:05:00.524 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.491431+0000", 2024-09-20T19:05:00.524 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000111233 2024-09-20T19:05:00.524 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.524 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.524 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:05:00.524 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.501137+0000", 2024-09-20T19:05:00.524 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0097052000000000006 2024-09-20T19:05:00.524 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.524 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.525 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:05:00.525 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.504013+0000", 2024-09-20T19:05:00.525 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0028765179999999998 2024-09-20T19:05:00.525 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:05:00.525 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:05:00.525 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:05:00.525 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.525 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.525 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:177154 10.0 10:0823c0c1:::benchmark_data_smithi083_148100_object60292:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e384)", 2024-09-20T19:05:00.525 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:00.488040+0000", 2024-09-20T19:05:00.525 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.020606158999999999, 2024-09-20T19:05:00.525 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.020719909000000002, 2024-09-20T19:05:00.525 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:05:00.525 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:05:00.526 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "waiting for sub ops", 2024-09-20T19:05:00.526 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:05:00.526 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:05:00.526 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:05:00.526 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 177154 2024-09-20T19:05:00.526 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.526 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:05:00.526 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.526 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:05:00.526 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.488040+0000", 2024-09-20T19:05:00.526 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:05:00.526 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.526 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.526 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:05:00.527 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.488040+0000", 2024-09-20T19:05:00.527 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:05:00.527 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.527 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.527 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:05:00.527 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.488040+0000", 2024-09-20T19:05:00.527 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.1199999999999998e-07 2024-09-20T19:05:00.527 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.527 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.527 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:05:00.527 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.488043+0000", 2024-09-20T19:05:00.527 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.7800000000000001e-06 2024-09-20T19:05:00.527 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.528 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.528 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:05:00.528 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.488048+0000", 2024-09-20T19:05:00.528 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6699999999999999e-06 2024-09-20T19:05:00.528 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.528 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.528 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:05:00.528 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.488053+0000", 2024-09-20T19:05:00.528 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8899999999999998e-06 2024-09-20T19:05:00.528 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.528 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.528 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:05:00.528 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.498092+0000", 2024-09-20T19:05:00.528 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.010038951000000001 2024-09-20T19:05:00.529 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.529 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.529 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:05:00.529 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.498329+0000", 2024-09-20T19:05:00.529 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00023709499999999999 2024-09-20T19:05:00.529 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.529 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.529 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,4", 2024-09-20T19:05:00.529 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.498482+0000", 2024-09-20T19:05:00.529 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015269200000000001 2024-09-20T19:05:00.529 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.529 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.529 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:05:00.529 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.504064+0000", 2024-09-20T19:05:00.529 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0055815839999999997 2024-09-20T19:05:00.530 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:05:00.530 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:05:00.530 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:05:00.530 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.530 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.530 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:177155 10.0 10:09970aa4:::benchmark_data_smithi083_148100_object60293:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e384)", 2024-09-20T19:05:00.530 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:00.488066+0000", 2024-09-20T19:05:00.530 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.020580040000000001, 2024-09-20T19:05:00.530 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.020762468999999999, 2024-09-20T19:05:00.530 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:05:00.531 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:05:00.531 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "waiting for sub ops", 2024-09-20T19:05:00.531 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:05:00.531 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:05:00.531 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:05:00.531 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 177155 2024-09-20T19:05:00.531 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.531 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:05:00.531 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.531 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:05:00.531 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.488066+0000", 2024-09-20T19:05:00.531 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:05:00.532 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.532 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.532 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:05:00.532 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.488066+0000", 2024-09-20T19:05:00.532 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:05:00.532 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.532 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.532 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:05:00.532 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.488066+0000", 2024-09-20T19:05:00.532 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.2500000000000001e-07 2024-09-20T19:05:00.532 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.532 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.532 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:05:00.532 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.488068+0000", 2024-09-20T19:05:00.532 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0999999999999998e-06 2024-09-20T19:05:00.533 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.533 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.533 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:05:00.533 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.488076+0000", 2024-09-20T19:05:00.533 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.4660000000000002e-06 2024-09-20T19:05:00.533 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.533 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.533 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:05:00.533 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.488080+0000", 2024-09-20T19:05:00.533 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3880000000000002e-06 2024-09-20T19:05:00.533 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.533 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.533 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:05:00.533 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.501194+0000", 2024-09-20T19:05:00.533 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.013113553999999999 2024-09-20T19:05:00.534 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.534 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.534 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:05:00.534 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.501436+0000", 2024-09-20T19:05:00.534 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000242787 2024-09-20T19:05:00.534 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.534 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.534 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,4", 2024-09-20T19:05:00.534 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.501602+0000", 2024-09-20T19:05:00.534 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000165324 2024-09-20T19:05:00.534 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.534 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.534 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:05:00.535 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.504121+0000", 2024-09-20T19:05:00.535 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0025193080000000001 2024-09-20T19:05:00.535 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:05:00.535 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:05:00.535 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:05:00.535 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.535 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.535 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:177163 10.a 10:524de8fb:::benchmark_data_smithi083_148100_object60301:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e384)", 2024-09-20T19:05:00.535 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:00.504051+0000", 2024-09-20T19:05:00.535 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.0045948880000000001, 2024-09-20T19:05:00.535 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0048471809999999999, 2024-09-20T19:05:00.535 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:05:00.535 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:05:00.535 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "waiting for sub ops", 2024-09-20T19:05:00.536 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:05:00.536 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:05:00.536 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:05:00.536 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 177163 2024-09-20T19:05:00.536 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.536 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:05:00.536 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.536 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:05:00.536 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.504051+0000", 2024-09-20T19:05:00.536 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:05:00.536 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.536 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.536 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:05:00.536 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.504051+0000", 2024-09-20T19:05:00.537 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:05:00.537 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.537 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.537 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:05:00.537 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.504052+0000", 2024-09-20T19:05:00.537 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.519e-06 2024-09-20T19:05:00.537 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.537 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.537 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:05:00.537 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.504059+0000", 2024-09-20T19:05:00.537 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.1319999999999999e-06 2024-09-20T19:05:00.537 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.537 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.537 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:05:00.537 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.504075+0000", 2024-09-20T19:05:00.538 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6889e-05 2024-09-20T19:05:00.538 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.538 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.538 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:05:00.538 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.504084+0000", 2024-09-20T19:05:00.538 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.106e-06 2024-09-20T19:05:00.538 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.538 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.538 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:05:00.538 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.504280+0000", 2024-09-20T19:05:00.538 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000196761 2024-09-20T19:05:00.538 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.538 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.539 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:05:00.539 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.504374+0000", 2024-09-20T19:05:00.539 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.3694000000000001e-05 2024-09-20T19:05:00.539 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.539 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.539 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,5", 2024-09-20T19:05:00.539 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.504467+0000", 2024-09-20T19:05:00.539 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.3280000000000001e-05 2024-09-20T19:05:00.539 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.539 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.539 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:05:00.539 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.507892+0000", 2024-09-20T19:05:00.539 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0034251720000000002 2024-09-20T19:05:00.539 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:05:00.540 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:05:00.540 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:05:00.540 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.540 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.540 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:177166 10.7 10:ea00d5bd:::benchmark_data_smithi083_148100_object60304:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e384)", 2024-09-20T19:05:00.540 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:00.504103+0000", 2024-09-20T19:05:00.540 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.0045422960000000004, 2024-09-20T19:05:00.540 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0048644430000000004, 2024-09-20T19:05:00.540 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:05:00.540 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:05:00.540 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "waiting for sub ops", 2024-09-20T19:05:00.540 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:05:00.540 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:05:00.540 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:05:00.541 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 177166 2024-09-20T19:05:00.541 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.541 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:05:00.541 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.541 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:05:00.541 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.504103+0000", 2024-09-20T19:05:00.541 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:05:00.541 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.541 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.541 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:05:00.541 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.504103+0000", 2024-09-20T19:05:00.541 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:05:00.541 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.541 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.542 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:05:00.542 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.504104+0000", 2024-09-20T19:05:00.542 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4899999999999995e-07 2024-09-20T19:05:00.542 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.542 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.542 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:05:00.542 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.504108+0000", 2024-09-20T19:05:00.542 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4249999999999998e-06 2024-09-20T19:05:00.542 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.542 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.542 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:05:00.542 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.504114+0000", 2024-09-20T19:05:00.542 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.9530000000000002e-06 2024-09-20T19:05:00.542 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.543 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.543 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:05:00.543 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.504120+0000", 2024-09-20T19:05:00.543 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9279999999999998e-06 2024-09-20T19:05:00.543 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.543 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.543 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:05:00.543 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.504226+0000", 2024-09-20T19:05:00.543 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000105982 2024-09-20T19:05:00.543 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.543 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.543 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:05:00.543 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.504350+0000", 2024-09-20T19:05:00.543 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012351500000000001 2024-09-20T19:05:00.544 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.544 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.544 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 3,5", 2024-09-20T19:05:00.544 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.504463+0000", 2024-09-20T19:05:00.544 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000113349 2024-09-20T19:05:00.544 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:05:00.544 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:05:00.544 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:05:00.544 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.544 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.544 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:177168 10.f 10:f57618c3:::benchmark_data_smithi083_148100_object60306:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e384)", 2024-09-20T19:05:00.544 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:00.504134+0000", 2024-09-20T19:05:00.544 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.0045119890000000001, 2024-09-20T19:05:00.544 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0048923650000000001, 2024-09-20T19:05:00.544 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:05:00.545 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:05:00.545 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "waiting for sub ops", 2024-09-20T19:05:00.545 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:05:00.545 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:05:00.545 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:05:00.545 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 177168 2024-09-20T19:05:00.545 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.545 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:05:00.545 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.545 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:05:00.545 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.504134+0000", 2024-09-20T19:05:00.545 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:05:00.545 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.546 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.546 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:05:00.546 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.504134+0000", 2024-09-20T19:05:00.546 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:05:00.546 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.546 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.546 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:05:00.546 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.504134+0000", 2024-09-20T19:05:00.546 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.9900000000000001e-07 2024-09-20T19:05:00.546 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.547 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.547 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:05:00.547 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.504137+0000", 2024-09-20T19:05:00.547 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.7369999999999998e-06 2024-09-20T19:05:00.547 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.547 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.547 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:05:00.547 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.504143+0000", 2024-09-20T19:05:00.547 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8810000000000001e-06 2024-09-20T19:05:00.547 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.547 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.547 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:05:00.547 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.504148+0000", 2024-09-20T19:05:00.548 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.1290000000000004e-06 2024-09-20T19:05:00.548 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.548 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.548 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:05:00.548 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.504777+0000", 2024-09-20T19:05:00.548 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00062872600000000002 2024-09-20T19:05:00.548 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.548 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.548 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:05:00.548 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.504873+0000", 2024-09-20T19:05:00.548 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.6155000000000003e-05 2024-09-20T19:05:00.548 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.548 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.548 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 2,3", 2024-09-20T19:05:00.549 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.504971+0000", 2024-09-20T19:05:00.549 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.7980999999999996e-05 2024-09-20T19:05:00.549 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:05:00.549 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:05:00.549 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:05:00.549 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.549 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.549 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:177171 10.d 10:b28d7bda:::benchmark_data_smithi083_148100_object60309:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e384)", 2024-09-20T19:05:00.549 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:00.504155+0000", 2024-09-20T19:05:00.549 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.0044911159999999999, 2024-09-20T19:05:00.549 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0049374060000000001, 2024-09-20T19:05:00.549 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:05:00.549 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:05:00.549 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "waiting for sub ops", 2024-09-20T19:05:00.549 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:05:00.550 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:05:00.550 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:05:00.550 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 177171 2024-09-20T19:05:00.550 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.550 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:05:00.550 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.550 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:05:00.550 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.504155+0000", 2024-09-20T19:05:00.550 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:05:00.550 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.550 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.550 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:05:00.550 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.504155+0000", 2024-09-20T19:05:00.550 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:05:00.551 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.551 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.551 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:05:00.551 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.504155+0000", 2024-09-20T19:05:00.551 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.3299999999999998e-07 2024-09-20T19:05:00.551 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.551 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.551 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:05:00.551 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.504158+0000", 2024-09-20T19:05:00.551 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.8650000000000001e-06 2024-09-20T19:05:00.551 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.551 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.551 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:05:00.551 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.504164+0000", 2024-09-20T19:05:00.552 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8300000000000001e-06 2024-09-20T19:05:00.552 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.552 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.552 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:05:00.552 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.504169+0000", 2024-09-20T19:05:00.552 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.9259999999999999e-06 2024-09-20T19:05:00.552 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.552 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.552 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:05:00.552 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.504712+0000", 2024-09-20T19:05:00.552 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00054328199999999997 2024-09-20T19:05:00.552 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.552 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.552 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:05:00.553 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.504836+0000", 2024-09-20T19:05:00.553 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012461199999999999 2024-09-20T19:05:00.553 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:00.553 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:00.553 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,4", 2024-09-20T19:05:00.553 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:00.504971+0000", 2024-09-20T19:05:00.553 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013447700000000001 2024-09-20T19:05:00.553 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:05:00.553 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:05:00.553 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:05:00.553 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:05:00.553 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-09-20T19:05:00.553 INFO:teuthology.orchestra.run.smithi142.stdout: "num_ops": 7 2024-09-20T19:05:00.553 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T19:05:00.556 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:05:00.595 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:00.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:00.596 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:05:00.697 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:00.732 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:00.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:00.733 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:00.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:00 smithi142 ceph-mon[32508]: pgmap v755: 148 pgs: 1 active+recovery_wait+degraded, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 142 active+clean; 3.7 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.11k op/s; 5/181005 objects degraded (0.003%); 8093/181005 objects misplaced (4.471%); 9.9 MiB/s, 157 objects/s recovering 2024-09-20T19:05:00.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:00 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1965927970' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:05:00.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:00 smithi083 ceph-mon[29421]: pgmap v755: 148 pgs: 1 active+recovery_wait+degraded, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 142 active+clean; 3.7 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.11k op/s; 5/181005 objects degraded (0.003%); 8093/181005 objects misplaced (4.471%); 9.9 MiB/s, 157 objects/s recovering 2024-09-20T19:05:00.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:00 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1965927970' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:05:00.792 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:05:00 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:05:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:05:00.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:00 smithi083 ceph-mon[36260]: pgmap v755: 148 pgs: 1 active+recovery_wait+degraded, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 142 active+clean; 3.7 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.11k op/s; 5/181005 objects degraded (0.003%); 8093/181005 objects misplaced (4.471%); 9.9 MiB/s, 157 objects/s recovering 2024-09-20T19:05:00.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:00 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1965927970' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:05:00.834 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:00.871 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:00.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:00.872 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:00.973 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:01.013 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:01.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:01.014 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:01.115 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:01.149 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:01.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:01.150 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:01.251 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:01.282 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:01.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:01.283 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:01.383 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:01.412 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:01.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:01.413 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:01.496 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_blocked_ops 2024-09-20T19:05:01.514 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:01.545 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:01.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:01.546 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:01.646 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:05:01.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:01.682 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:01.682 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:05:01.783 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:01.834 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:01.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:01.835 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:01.901 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.1 to 1 (10000) 2024-09-20T19:05:01.936 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:01.969 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:01.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:01.970 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:01.978 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:05:02.071 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:02.094 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:02.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:02.095 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:02.196 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:02.227 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:02.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:02.228 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:02.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:01 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3989831255' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-20T19:05:02.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:01 smithi142 ceph-mon[32508]: pgmap v756: 148 pgs: 1 active+recovery_wait+degraded, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 142 active+clean; 3.7 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 420 B/s rd, 0 B/s wr, 916 op/s; 5/180066 objects degraded (0.003%); 8093/180066 objects misplaced (4.494%); 8.2 MiB/s, 129 objects/s recovering 2024-09-20T19:05:02.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:01 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:05:02.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:01 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:05:02.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:01 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3989831255' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-20T19:05:02.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:01 smithi083 ceph-mon[29421]: pgmap v756: 148 pgs: 1 active+recovery_wait+degraded, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 142 active+clean; 3.7 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 420 B/s rd, 0 B/s wr, 916 op/s; 5/180066 objects degraded (0.003%); 8093/180066 objects misplaced (4.494%); 8.2 MiB/s, 129 objects/s recovering 2024-09-20T19:05:02.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:01 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:05:02.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:01 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:05:02.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:01 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3989831255' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-20T19:05:02.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:01 smithi083 ceph-mon[36260]: pgmap v756: 148 pgs: 1 active+recovery_wait+degraded, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 142 active+clean; 3.7 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 420 B/s rd, 0 B/s wr, 916 op/s; 5/180066 objects degraded (0.003%); 8093/180066 objects misplaced (4.494%); 8.2 MiB/s, 129 objects/s recovering 2024-09-20T19:05:02.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:01 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:05:02.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:01 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:05:02.329 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:02.364 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:02.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:02.365 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:02.466 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:05:02.509 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:02.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:02.511 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:05:02.611 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:02.643 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:02.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:02.644 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:02.745 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:02.779 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:02.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:02.780 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:02.881 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:02.903 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:02.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:02.904 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:03.005 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:03.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:03.039 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:03.039 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:03.140 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:03.174 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:03.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:03.175 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:03.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:02 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3989831255' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-20T19:05:03.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:02 smithi142 ceph-mon[32508]: osdmap e385: 8 total, 8 up, 8 in 2024-09-20T19:05:03.276 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:03.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:02 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3989831255' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-20T19:05:03.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:02 smithi083 ceph-mon[29421]: osdmap e385: 8 total, 8 up, 8 in 2024-09-20T19:05:03.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:02 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3989831255' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-20T19:05:03.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:02 smithi083 ceph-mon[36260]: osdmap e385: 8 total, 8 up, 8 in 2024-09-20T19:05:03.305 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:03.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:03.306 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:03.407 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:03.441 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:03.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:03.442 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:03.543 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:03.564 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:03.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:03.565 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:03.665 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:03.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:03.704 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:03.704 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:03.805 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:03.838 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:03.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:03.839 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:03.940 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:05:03.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:03.977 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:03.978 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:05:04.078 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:04.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:04.106 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:04.106 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:04.207 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:04.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:04.243 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:04.243 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:04.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:03 smithi142 ceph-mon[32508]: pgmap v758: 148 pgs: 1 active+recovery_wait+degraded, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 142 active+clean; 3.4 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.20k op/s; 5/166776 objects degraded (0.003%); 7028/166776 objects misplaced (4.214%); 9.5 MiB/s, 188 objects/s recovering 2024-09-20T19:05:04.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:03 smithi083 ceph-mon[29421]: pgmap v758: 148 pgs: 1 active+recovery_wait+degraded, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 142 active+clean; 3.4 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.20k op/s; 5/166776 objects degraded (0.003%); 7028/166776 objects misplaced (4.214%); 9.5 MiB/s, 188 objects/s recovering 2024-09-20T19:05:04.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:03 smithi083 ceph-mon[36260]: pgmap v758: 148 pgs: 1 active+recovery_wait+degraded, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 142 active+clean; 3.4 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.20k op/s; 5/166776 objects degraded (0.003%); 7028/166776 objects misplaced (4.214%); 9.5 MiB/s, 188 objects/s recovering 2024-09-20T19:05:04.344 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:04.376 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:04.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:04.377 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:04.478 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:04.506 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:04.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:04.507 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:04.608 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:04.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:04.645 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:04.646 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:04.747 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:05:04.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:04.771 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:04.771 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:05:04.872 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:04.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:04.911 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:04.911 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:05.012 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:05.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:05.055 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:05.055 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:05.156 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:05.159 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T19:05:05.159 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-09-20T19:05:05.159 INFO:teuthology.orchestra.run.smithi142.stdout: "complaint_time": 30, 2024-09-20T19:05:05.159 INFO:teuthology.orchestra.run.smithi142.stdout: "num_blocked_ops": 0 2024-09-20T19:05:05.160 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T19:05:05.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:05.202 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:05.202 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:05.203 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:04 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:05:05.203 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:04 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:05:05.204 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:04 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:05:05.204 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:04 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:05:05.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:04 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:05:05.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:04 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:05:05.302 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:05.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:05.339 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:05.339 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:05.440 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:05.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:05.483 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:05.484 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:05.584 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:05.622 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:05.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:05.623 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:05.724 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:05.751 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd reweight 2 1 2024-09-20T19:05:05.761 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:05.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:05.762 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:05.780 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:05:05.863 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:05.885 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:05.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:05.886 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:05.973 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:05 smithi083 ceph-mon[29421]: osdmap e386: 8 total, 8 up, 8 in 2024-09-20T19:05:05.973 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:05 smithi083 ceph-mon[29421]: pgmap v760: 148 pgs: 1 active+recovery_wait+degraded, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 142 active+clean; 3.4 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 668 B/s rd, 0 B/s wr, 729 op/s; 5/166776 objects degraded (0.003%); 7028/166776 objects misplaced (4.214%); 4.4 MiB/s, 108 objects/s recovering 2024-09-20T19:05:05.973 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:05 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 5/166776 objects degraded (0.003%), 1 pg degraded (PG_DEGRADED) 2024-09-20T19:05:05.974 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:05 smithi083 ceph-mon[36260]: osdmap e386: 8 total, 8 up, 8 in 2024-09-20T19:05:05.974 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:05 smithi083 ceph-mon[36260]: pgmap v760: 148 pgs: 1 active+recovery_wait+degraded, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 142 active+clean; 3.4 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 668 B/s rd, 0 B/s wr, 729 op/s; 5/166776 objects degraded (0.003%); 7028/166776 objects misplaced (4.214%); 4.4 MiB/s, 108 objects/s recovering 2024-09-20T19:05:05.974 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:05 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 5/166776 objects degraded (0.003%), 1 pg degraded (PG_DEGRADED) 2024-09-20T19:05:05.986 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:06.030 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:06.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:06.031 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:06.132 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:06.170 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:06.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:06.171 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:06.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:05 smithi142 ceph-mon[32508]: osdmap e386: 8 total, 8 up, 8 in 2024-09-20T19:05:06.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:05 smithi142 ceph-mon[32508]: pgmap v760: 148 pgs: 1 active+recovery_wait+degraded, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 142 active+clean; 3.4 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 668 B/s rd, 0 B/s wr, 729 op/s; 5/166776 objects degraded (0.003%); 7028/166776 objects misplaced (4.214%); 4.4 MiB/s, 108 objects/s recovering 2024-09-20T19:05:06.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:05 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 5/166776 objects degraded (0.003%), 1 pg degraded (PG_DEGRADED) 2024-09-20T19:05:06.273 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:06.317 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:06.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:06.319 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:06.405 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:05:06.420 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:06.458 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:06.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:06.460 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:06.560 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:06.584 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:06.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:06.585 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:06.686 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:06.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:06.745 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:06.745 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:06.846 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:06.889 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:06.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:06.890 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:06.945 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T19:05:06.968 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-20T19:05:06.990 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:07.024 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:07.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:07.025 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:07.126 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:07.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:07.182 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:07.182 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:07.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:06 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1849977468' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:05:07.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:06 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:05:07.283 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:05:07.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:06 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1849977468' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:05:07.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:06 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:05:07.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:06 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1849977468' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:05:07.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:06 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:05:07.321 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:07.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:07.322 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:05:07.423 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:05:07.463 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:07.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:07.464 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:05:07.564 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:05:07.603 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:07.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:07.604 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:05:07.705 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:05:07.727 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:07.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:07.728 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:05:07.762 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:05:07 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:05:07.829 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:07.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:07.864 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:07.864 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:07.962 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.2 to 1 (10000) 2024-09-20T19:05:07.965 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:07.999 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:07.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:08.000 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:08.100 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:08.140 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:08.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:08.141 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:08.242 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:08.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:07 smithi142 ceph-mon[32508]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:05:08.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:07 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T19:05:08.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:07 smithi142 ceph-mon[32508]: osdmap e387: 8 total, 8 up, 8 in 2024-09-20T19:05:08.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:07 smithi142 ceph-mon[32508]: pgmap v762: 148 pgs: 1 active+recovery_wait+degraded, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 142 active+clean; 3.2 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.32k op/s; 5/156237 objects degraded (0.003%); 6752/156237 objects misplaced (4.322%); 5.6 MiB/s, 290 objects/s recovering 2024-09-20T19:05:08.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:07 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:05:08.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:07 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/766919334' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-20T19:05:08.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:07 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-20T19:05:08.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:07 smithi142 ceph-mon[32508]: 3.2 scrub starts 2024-09-20T19:05:08.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:07 smithi142 ceph-mon[32508]: 3.2 scrub ok 2024-09-20T19:05:08.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:08.288 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:08.288 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:08.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:07 smithi083 ceph-mon[29421]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:05:08.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:07 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T19:05:08.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:07 smithi083 ceph-mon[29421]: osdmap e387: 8 total, 8 up, 8 in 2024-09-20T19:05:08.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:07 smithi083 ceph-mon[29421]: pgmap v762: 148 pgs: 1 active+recovery_wait+degraded, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 142 active+clean; 3.2 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.32k op/s; 5/156237 objects degraded (0.003%); 6752/156237 objects misplaced (4.322%); 5.6 MiB/s, 290 objects/s recovering 2024-09-20T19:05:08.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:07 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:05:08.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:07 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/766919334' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-20T19:05:08.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:07 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-20T19:05:08.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:07 smithi083 ceph-mon[29421]: 3.2 scrub starts 2024-09-20T19:05:08.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:07 smithi083 ceph-mon[29421]: 3.2 scrub ok 2024-09-20T19:05:08.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:07 smithi083 ceph-mon[36260]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:05:08.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:07 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T19:05:08.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:07 smithi083 ceph-mon[36260]: osdmap e387: 8 total, 8 up, 8 in 2024-09-20T19:05:08.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:07 smithi083 ceph-mon[36260]: pgmap v762: 148 pgs: 1 active+recovery_wait+degraded, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 142 active+clean; 3.2 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.32k op/s; 5/156237 objects degraded (0.003%); 6752/156237 objects misplaced (4.322%); 5.6 MiB/s, 290 objects/s recovering 2024-09-20T19:05:08.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:07 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:05:08.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:07 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/766919334' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-20T19:05:08.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:07 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-20T19:05:08.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:07 smithi083 ceph-mon[36260]: 3.2 scrub starts 2024-09-20T19:05:08.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:07 smithi083 ceph-mon[36260]: 3.2 scrub ok 2024-09-20T19:05:08.390 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:08.417 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:08.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:08.418 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:08.519 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:08.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:08.556 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:08.556 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:08.657 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:08.679 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd reweight 3 1 2024-09-20T19:05:08.700 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:08.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:08.701 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:08.733 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:05:08.802 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:08.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:08.855 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:08.855 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:08.873 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_historic_ops 2024-09-20T19:05:08.956 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:08.997 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:08.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:08.998 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:09.099 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:09.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:09.226 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:09.227 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:09.227 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:08 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-20T19:05:09.227 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:08 smithi083 ceph-mon[29421]: osdmap e388: 8 total, 8 up, 8 in 2024-09-20T19:05:09.227 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:08 smithi083 ceph-mon[29421]: 5.d scrub starts 2024-09-20T19:05:09.227 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:08 smithi083 ceph-mon[29421]: 5.d scrub ok 2024-09-20T19:05:09.228 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:08 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-20T19:05:09.228 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:08 smithi083 ceph-mon[36260]: osdmap e388: 8 total, 8 up, 8 in 2024-09-20T19:05:09.228 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:08 smithi083 ceph-mon[36260]: 5.d scrub starts 2024-09-20T19:05:09.228 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:08 smithi083 ceph-mon[36260]: 5.d scrub ok 2024-09-20T19:05:09.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:08 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-20T19:05:09.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:08 smithi142 ceph-mon[32508]: osdmap e388: 8 total, 8 up, 8 in 2024-09-20T19:05:09.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:08 smithi142 ceph-mon[32508]: 5.d scrub starts 2024-09-20T19:05:09.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:08 smithi142 ceph-mon[32508]: 5.d scrub ok 2024-09-20T19:05:09.327 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:09.341 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:05:09.363 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:09.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:09.364 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:09.376 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:05:09.377 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:05:09.406 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:05:09.406 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:05:09.439 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:05:09.439 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:05:09.465 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:09.466 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:05:09.478 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:05:09.478 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:05:09.510 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:09.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:09.512 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:09.512 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:05:09.513 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:05:09.544 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:05:09.544 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:05:09.580 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:05:09.580 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:05:09.612 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:09.614 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:05:09.614 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:05:09.660 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:09.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:09.661 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:09.762 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:05:09.805 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:09.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:09.806 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:05:09.907 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:09.932 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T19:05:09.941 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:09.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:09.942 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:10.043 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:10.085 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:10.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:10.086 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:10.151 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T19:05:10.152 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:05:10.187 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:05:10.224 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:10.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:10.225 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:05:10.240 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:09 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:05:10.240 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:09 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:05:10.240 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:09 smithi083 ceph-mon[29421]: osdmap e389: 8 total, 8 up, 8 in 2024-09-20T19:05:10.240 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:09 smithi083 ceph-mon[29421]: pgmap v765: 148 pgs: 1 active+recovery_wait+degraded, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 142 active+clean; 3.1 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.03k op/s; 5/153756 objects degraded (0.003%); 6473/153756 objects misplaced (4.210%); 0 B/s, 216 objects/s recovering 2024-09-20T19:05:10.240 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:09 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/144496453' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:05:10.240 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:09 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:05:10.240 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:09 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:05:10.240 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:09 smithi083 ceph-mon[36260]: osdmap e389: 8 total, 8 up, 8 in 2024-09-20T19:05:10.240 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:09 smithi083 ceph-mon[36260]: pgmap v765: 148 pgs: 1 active+recovery_wait+degraded, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 142 active+clean; 3.1 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.03k op/s; 5/153756 objects degraded (0.003%); 6473/153756 objects misplaced (4.210%); 0 B/s, 216 objects/s recovering 2024-09-20T19:05:10.241 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:09 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/144496453' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:05:10.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:09 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:05:10.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:09 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:05:10.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:09 smithi142 ceph-mon[32508]: osdmap e389: 8 total, 8 up, 8 in 2024-09-20T19:05:10.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:09 smithi142 ceph-mon[32508]: pgmap v765: 148 pgs: 1 active+recovery_wait+degraded, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 142 active+clean; 3.1 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.03k op/s; 5/153756 objects degraded (0.003%); 6473/153756 objects misplaced (4.210%); 0 B/s, 216 objects/s recovering 2024-09-20T19:05:10.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:09 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/144496453' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:05:10.326 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:10.364 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:10.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:10.365 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:10.466 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:10.501 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:10.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:10.502 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:10.603 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:10.634 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:10.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:10.635 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:10.736 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:05:10.775 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:10.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:10.776 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:05:10.792 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:05:10 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:05:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:05:10.864 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-20T19:05:10.877 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:10.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:10.958 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:10.958 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:11.015 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.3 to 1 (10000) 2024-09-20T19:05:11.060 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:11.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:11.143 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:11.143 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:11.244 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:11.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:10 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T19:05:11.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:10 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 5/153756 objects degraded (0.003%), 1 pg degraded (PG_DEGRADED) 2024-09-20T19:05:11.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:10 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3382335321' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-20T19:05:11.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:10 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-20T19:05:11.272 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:11.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:11.273 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:11.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:10 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T19:05:11.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:10 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 5/153756 objects degraded (0.003%), 1 pg degraded (PG_DEGRADED) 2024-09-20T19:05:11.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:10 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3382335321' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-20T19:05:11.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:10 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-20T19:05:11.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:10 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T19:05:11.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:10 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 5/153756 objects degraded (0.003%), 1 pg degraded (PG_DEGRADED) 2024-09-20T19:05:11.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:10 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3382335321' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-20T19:05:11.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:10 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-20T19:05:11.320 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:05:11.374 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:11.413 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:11.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:11.414 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:11.515 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:11.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:11.556 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:11.556 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:11.656 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:11.699 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:11.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:11.700 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:11.801 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:11.847 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:11.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:11.848 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:11.949 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:11.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:11.983 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:11.984 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:12.084 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:12.106 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:12.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:12.107 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:12.208 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:12.248 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:12.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:12.249 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:12.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:12 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:05:12.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:12 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:05:12.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:12 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-20T19:05:12.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:12 smithi142 ceph-mon[32508]: osdmap e390: 8 total, 8 up, 8 in 2024-09-20T19:05:12.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:12 smithi142 ceph-mon[32508]: pgmap v767: 148 pgs: 1 active+recovery_wait+degraded, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 142 active+clean; 3.1 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 0 B/s wr, 329 op/s; 5/152895 objects degraded (0.003%); 6473/152895 objects misplaced (4.234%) 2024-09-20T19:05:12.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:12 smithi142 ceph-mon[32508]: 5.b scrub starts 2024-09-20T19:05:12.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:12 smithi142 ceph-mon[32508]: 5.b scrub ok 2024-09-20T19:05:12.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:12 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T19:05:12.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:12 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T19:05:12.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:12 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:05:12.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:12 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:05:12.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:12 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:05:12.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:12 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-20T19:05:12.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:12 smithi083 ceph-mon[29421]: osdmap e390: 8 total, 8 up, 8 in 2024-09-20T19:05:12.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:12 smithi083 ceph-mon[29421]: pgmap v767: 148 pgs: 1 active+recovery_wait+degraded, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 142 active+clean; 3.1 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 0 B/s wr, 329 op/s; 5/152895 objects degraded (0.003%); 6473/152895 objects misplaced (4.234%) 2024-09-20T19:05:12.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:12 smithi083 ceph-mon[29421]: 5.b scrub starts 2024-09-20T19:05:12.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:12 smithi083 ceph-mon[29421]: 5.b scrub ok 2024-09-20T19:05:12.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:12 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T19:05:12.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:12 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T19:05:12.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:12 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:05:12.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:12 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:05:12.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:12 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:05:12.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:12 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-20T19:05:12.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:12 smithi083 ceph-mon[36260]: osdmap e390: 8 total, 8 up, 8 in 2024-09-20T19:05:12.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:12 smithi083 ceph-mon[36260]: pgmap v767: 148 pgs: 1 active+recovery_wait+degraded, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 142 active+clean; 3.1 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 0 B/s wr, 329 op/s; 5/152895 objects degraded (0.003%); 6473/152895 objects misplaced (4.234%) 2024-09-20T19:05:12.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:12 smithi083 ceph-mon[36260]: 5.b scrub starts 2024-09-20T19:05:12.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:12 smithi083 ceph-mon[36260]: 5.b scrub ok 2024-09-20T19:05:12.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:12 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T19:05:12.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:12 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T19:05:12.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:12 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:05:12.349 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:12.395 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T19:05:12.396 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:05:12.398 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd reweight 5 1 2024-09-20T19:05:12.420 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:12.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:12.421 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:12.457 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:05:12.523 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:12.558 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:12.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:12.559 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:12.660 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:12.703 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:12.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:12.704 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:12.805 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:12.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:12.836 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:12.836 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:12.937 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:12.977 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:12.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:12.979 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:13.079 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:13.129 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:13.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:13.130 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:13.133 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:05:13.141 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_blocked_ops 2024-09-20T19:05:13.231 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:13.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:13.281 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:13.281 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:13.382 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:13.427 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:13.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:13.428 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:13.528 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:13.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:13.592 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:13.592 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:13.642 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:05:13.693 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:13.731 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:13.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:13.732 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:13.740 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T19:05:13.833 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:13.873 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:13.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:13.874 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:13.974 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:14.008 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:14.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:14.009 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:14.110 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:14.158 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:14.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:14.159 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:14.214 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:05:14.257 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T19:05:14.257 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:05:14.260 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:14.284 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.5 to 1 (10000) 2024-09-20T19:05:14.288 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-20T19:05:14.314 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:14.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:14.316 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:14.416 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:14.457 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:14.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:14.458 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:14.504 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:14 smithi142 ceph-mon[32508]: pgmap v768: 148 pgs: 1 active+recovery_wait+degraded, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 142 active+clean; 2.8 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 931 op/s; 5/139482 objects degraded (0.004%); 5852/139482 objects misplaced (4.196%); 0 B/s, 181 objects/s recovering 2024-09-20T19:05:14.504 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:14 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:05:14.504 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:14 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:05:14.505 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:14 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3187282131' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:05:14.505 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:14 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:05:14.505 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:14 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/849731315' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-20T19:05:14.505 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:14 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-20T19:05:14.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:14 smithi083 ceph-mon[29421]: pgmap v768: 148 pgs: 1 active+recovery_wait+degraded, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 142 active+clean; 2.8 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 931 op/s; 5/139482 objects degraded (0.004%); 5852/139482 objects misplaced (4.196%); 0 B/s, 181 objects/s recovering 2024-09-20T19:05:14.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:14 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:05:14.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:14 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:05:14.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:14 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3187282131' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:05:14.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:14 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:05:14.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:14 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/849731315' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-20T19:05:14.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:14 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-20T19:05:14.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:14 smithi083 ceph-mon[36260]: pgmap v768: 148 pgs: 1 active+recovery_wait+degraded, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 142 active+clean; 2.8 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 931 op/s; 5/139482 objects degraded (0.004%); 5852/139482 objects misplaced (4.196%); 0 B/s, 181 objects/s recovering 2024-09-20T19:05:14.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:14 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:05:14.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:14 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:05:14.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:14 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3187282131' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:05:14.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:14 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:05:14.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:14 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/849731315' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-20T19:05:14.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:14 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-20T19:05:14.559 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:14.597 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:14.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:14.598 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:14.698 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:14.728 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:14.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:14.729 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:14.829 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:14.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:14.851 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:14.851 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:14.953 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:05:14.984 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_historic_ops 2024-09-20T19:05:15.002 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:15.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:15.004 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:05:15.105 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:15.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:15.142 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:15.142 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:15.243 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:15.276 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:15.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:15.277 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:15.378 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:15.428 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:15.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:15.429 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:15.436 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:05:15.512 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:15 smithi142 ceph-mon[32508]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T19:05:15.512 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:15 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:05:15.512 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:15 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-20T19:05:15.512 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:15 smithi142 ceph-mon[32508]: osdmap e391: 8 total, 8 up, 8 in 2024-09-20T19:05:15.512 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:15 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:05:15.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:15 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:05:15.530 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:15.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:15 smithi083 ceph-mon[29421]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T19:05:15.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:15 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:05:15.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:15 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-20T19:05:15.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:15 smithi083 ceph-mon[29421]: osdmap e391: 8 total, 8 up, 8 in 2024-09-20T19:05:15.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:15 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:05:15.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:15 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:05:15.542 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:15 smithi083 ceph-mon[36260]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T19:05:15.542 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:15 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:05:15.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:15 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-20T19:05:15.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:15 smithi083 ceph-mon[36260]: osdmap e391: 8 total, 8 up, 8 in 2024-09-20T19:05:15.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:15 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:05:15.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:15 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:05:15.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:15.564 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:15.564 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:15.665 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:15.709 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:15.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:15.710 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:15.811 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:15.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:15.845 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:15.845 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:15.946 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:15.985 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:15.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:15.986 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:16.065 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T19:05:16.066 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:05:16.087 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:16.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:16.127 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:16.128 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:16.181 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd reweight 7 1 2024-09-20T19:05:16.229 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:16.264 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:16.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:16.265 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:16.366 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:16.385 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:16 smithi083 ceph-mon[36260]: pgmap v770: 148 pgs: 1 active+recovery_wait+degraded, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 142 active+clean; 2.8 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 829 B/s rd, 0 B/s wr, 771 op/s; 5/139482 objects degraded (0.004%); 5852/139482 objects misplaced (4.196%); 0 B/s, 176 objects/s recovering 2024-09-20T19:05:16.385 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:16 smithi083 ceph-mon[36260]: osdmap e392: 8 total, 8 up, 8 in 2024-09-20T19:05:16.385 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:16 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 5/139482 objects degraded (0.004%), 1 pg degraded (PG_DEGRADED) 2024-09-20T19:05:16.386 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:16 smithi083 ceph-mon[29421]: pgmap v770: 148 pgs: 1 active+recovery_wait+degraded, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 142 active+clean; 2.8 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 829 B/s rd, 0 B/s wr, 771 op/s; 5/139482 objects degraded (0.004%); 5852/139482 objects misplaced (4.196%); 0 B/s, 176 objects/s recovering 2024-09-20T19:05:16.386 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:16 smithi083 ceph-mon[29421]: osdmap e392: 8 total, 8 up, 8 in 2024-09-20T19:05:16.386 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:16 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 5/139482 objects degraded (0.004%), 1 pg degraded (PG_DEGRADED) 2024-09-20T19:05:16.439 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:16.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:16.440 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:16.541 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:16.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:16.591 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:16.591 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:16.692 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:16.706 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:16 smithi142 ceph-mon[32508]: pgmap v770: 148 pgs: 1 active+recovery_wait+degraded, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 142 active+clean; 2.8 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 829 B/s rd, 0 B/s wr, 771 op/s; 5/139482 objects degraded (0.004%); 5852/139482 objects misplaced (4.196%); 0 B/s, 176 objects/s recovering 2024-09-20T19:05:16.706 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:16 smithi142 ceph-mon[32508]: osdmap e392: 8 total, 8 up, 8 in 2024-09-20T19:05:16.706 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:16 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 5/139482 objects degraded (0.004%), 1 pg degraded (PG_DEGRADED) 2024-09-20T19:05:16.724 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:16.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:16.725 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:16.760 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-20T19:05:16.826 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:16.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:16.888 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:16.889 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:16.989 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:17.032 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:17.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:17.033 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:17.134 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:17.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:17.181 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:17.181 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:17.231 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:05:17.283 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:17.319 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:17.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:17.320 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:17.402 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:05:17.402 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:05:17.402 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:05:17.402 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:05:17.402 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:05:17.402 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:05:17.402 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:05:17.402 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:05:17.403 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:05:17.403 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:05:17.403 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:05:17.403 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:05:17.403 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:05:17.403 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:05:17.403 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:05:17.404 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:05:17.421 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:05:17.455 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:17.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:17.456 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:05:17.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:17 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:05:17.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:17 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:05:17.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:17 smithi083 ceph-mon[29421]: 10.3 scrub starts 2024-09-20T19:05:17.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:17 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/4281847289' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:05:17.542 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:17 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:05:17.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:17 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:05:17.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:17 smithi083 ceph-mon[36260]: 10.3 scrub starts 2024-09-20T19:05:17.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:17 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/4281847289' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:05:17.556 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:17.593 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:17.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:17.594 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:17.694 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:17.715 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:17.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:17.716 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:17.762 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:05:17 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:05:17.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:17 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:05:17.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:17 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:05:17.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:17 smithi142 ceph-mon[32508]: 10.3 scrub starts 2024-09-20T19:05:17.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:17 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/4281847289' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:05:17.817 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:17.849 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:17.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:17.850 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:17.951 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:17.988 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:17.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:17.989 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:18.090 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:18.111 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:05:18.129 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:18.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:18.129 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:18.230 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:18.264 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:18.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:18.265 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:18.366 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:18.406 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:18.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:18.407 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:18.507 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:18.543 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:18.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:18.544 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:18.579 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:18 smithi083 ceph-mon[29421]: pgmap v772: 148 pgs: 1 active+clean+scrubbing, 1 active+recovery_wait+degraded, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 141 active+clean; 2.6 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1.35k op/s; 5/127905 objects degraded (0.004%); 5518/127905 objects misplaced (4.314%); 0 B/s, 382 objects/s recovering 2024-09-20T19:05:18.579 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:18 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:05:18.579 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:18 smithi083 ceph-mon[29421]: 10.3 scrub ok 2024-09-20T19:05:18.579 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:18 smithi083 ceph-mon[36260]: pgmap v772: 148 pgs: 1 active+clean+scrubbing, 1 active+recovery_wait+degraded, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 141 active+clean; 2.6 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1.35k op/s; 5/127905 objects degraded (0.004%); 5518/127905 objects misplaced (4.314%); 0 B/s, 382 objects/s recovering 2024-09-20T19:05:18.579 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:18 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:05:18.579 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:18 smithi083 ceph-mon[36260]: 10.3 scrub ok 2024-09-20T19:05:18.645 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:05:18.686 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:18.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:18.687 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:05:18.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:18 smithi142 ceph-mon[32508]: pgmap v772: 148 pgs: 1 active+clean+scrubbing, 1 active+recovery_wait+degraded, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 141 active+clean; 2.6 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1.35k op/s; 5/127905 objects degraded (0.004%); 5518/127905 objects misplaced (4.314%); 0 B/s, 382 objects/s recovering 2024-09-20T19:05:18.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:18 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:05:18.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:18 smithi142 ceph-mon[32508]: 10.3 scrub ok 2024-09-20T19:05:18.788 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:18.808 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:18.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:18.809 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:18.910 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:18.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:18.943 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:18.943 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:19.044 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:19.076 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:19.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:19.077 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:19.178 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:05:19.212 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:19.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:19.213 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:05:19.279 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T19:05:19.314 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:19.352 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:19.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:19.353 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:19.383 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.7 to 1 (10000) 2024-09-20T19:05:19.453 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:19.514 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:19.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:19.519 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:19.620 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:05:19.654 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:19.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:19.655 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:05:19.735 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:19 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/589554198' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-20T19:05:19.735 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:19 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/589554198' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-20T19:05:19.756 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:19.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:19 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/589554198' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-20T19:05:19.797 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:19.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:19.798 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:19.899 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:19.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:19.924 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:19.924 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:20.025 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:20.059 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:20.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:20.060 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:20.073 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T19:05:20.161 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:20.183 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:20.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:20.184 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:20.195 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd reweight 6 1 2024-09-20T19:05:20.240 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:05:20.284 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:20.319 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:20.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:20.320 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:20.420 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:20.478 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:20.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:20.479 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:20.580 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:20.615 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:20.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:20.616 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:20.670 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:20 smithi142 ceph-mon[32508]: pgmap v773: 148 pgs: 1 active+clean+scrubbing, 1 active+recovery_wait+degraded, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 141 active+clean; 2.5 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.16k op/s; 5/125169 objects degraded (0.004%); 5224/125169 objects misplaced (4.174%); 0 B/s, 294 objects/s recovering 2024-09-20T19:05:20.670 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:20 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/589554198' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-20T19:05:20.670 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:20 smithi142 ceph-mon[32508]: osdmap e393: 8 total, 8 up, 8 in 2024-09-20T19:05:20.717 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:20.726 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:20 smithi083 ceph-mon[29421]: pgmap v773: 148 pgs: 1 active+clean+scrubbing, 1 active+recovery_wait+degraded, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 141 active+clean; 2.5 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.16k op/s; 5/125169 objects degraded (0.004%); 5224/125169 objects misplaced (4.174%); 0 B/s, 294 objects/s recovering 2024-09-20T19:05:20.727 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:20 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/589554198' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-20T19:05:20.727 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:20 smithi083 ceph-mon[29421]: osdmap e393: 8 total, 8 up, 8 in 2024-09-20T19:05:20.727 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:20 smithi083 ceph-mon[36260]: pgmap v773: 148 pgs: 1 active+clean+scrubbing, 1 active+recovery_wait+degraded, 2 active+remapped+backfill_wait+backfill_toofull, 3 active+recovering+undersized+remapped, 141 active+clean; 2.5 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.16k op/s; 5/125169 objects degraded (0.004%); 5224/125169 objects misplaced (4.174%); 0 B/s, 294 objects/s recovering 2024-09-20T19:05:20.727 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:20 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/589554198' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-20T19:05:20.727 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:20 smithi083 ceph-mon[36260]: osdmap e393: 8 total, 8 up, 8 in 2024-09-20T19:05:20.740 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:20.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:20.741 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:20.842 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:20.875 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:20.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:20.876 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:20.977 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:21.010 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:21.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:21.011 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:21.042 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:05:20 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:05:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:05:21.111 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:21.133 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:21.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:21.135 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:21.235 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:21.261 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:21.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:21.262 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:21.363 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:21.385 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:21.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:21.387 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:21.487 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:21.509 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:21.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:21.510 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:21.597 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T19:05:21.597 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-09-20T19:05:21.597 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:21.597 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25775.0:196094 10.4 e393/375 10:2e700ad1:::benchmark_data_smithi083_148100_object79232:head v 393'12132, mlcod=393'12129)", 2024-09-20T19:05:21.597 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:20.819811+0000", 2024-09-20T19:05:21.597 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.0025741060000000001, 2024-09-20T19:05:21.598 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0025958629999999999, 2024-09-20T19:05:21.598 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:05:21.598 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:05:21.598 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-09-20T19:05:21.598 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:05:21.598 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:21.598 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:05:21.598 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.819811+0000", 2024-09-20T19:05:21.598 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:05:21.598 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:21.598 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:21.598 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:05:21.599 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.819811+0000", 2024-09-20T19:05:21.599 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:05:21.599 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:21.599 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:21.599 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:05:21.599 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.819812+0000", 2024-09-20T19:05:21.599 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.7700000000000004e-07 2024-09-20T19:05:21.599 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:21.599 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:21.599 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:05:21.599 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.819820+0000", 2024-09-20T19:05:21.599 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.1079999999999993e-06 2024-09-20T19:05:21.599 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:21.599 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:21.600 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:05:21.600 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.819831+0000", 2024-09-20T19:05:21.600 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1208e-05 2024-09-20T19:05:21.600 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:21.600 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:21.600 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:05:21.600 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.819838+0000", 2024-09-20T19:05:21.600 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.1419999999999998e-06 2024-09-20T19:05:21.600 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:21.600 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:21.600 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:05:21.600 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.819893+0000", 2024-09-20T19:05:21.600 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5173000000000002e-05 2024-09-20T19:05:21.600 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:21.601 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:21.601 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:05:21.601 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.819910+0000", 2024-09-20T19:05:21.601 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7668000000000001e-05 2024-09-20T19:05:21.601 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:05:21.601 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:05:21.601 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:05:21.601 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:21.601 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:21.601 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25775.0:196103 10.2 e393/375 10:41558950:::benchmark_data_smithi083_148100_object79241:head v 393'12327, mlcod=393'12325)", 2024-09-20T19:05:21.601 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:20.821413+0000", 2024-09-20T19:05:21.601 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.00097223800000000005, 2024-09-20T19:05:21.601 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0010509740000000001, 2024-09-20T19:05:21.601 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:05:21.602 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:05:21.602 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-09-20T19:05:21.602 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:05:21.602 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:21.602 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:05:21.602 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.821413+0000", 2024-09-20T19:05:21.602 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:05:21.602 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:21.602 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:21.602 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:05:21.602 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.821413+0000", 2024-09-20T19:05:21.602 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:05:21.602 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:21.602 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:21.603 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:05:21.603 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.821414+0000", 2024-09-20T19:05:21.603 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.5199999999999995e-07 2024-09-20T19:05:21.603 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:21.603 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:21.603 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:05:21.603 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.821421+0000", 2024-09-20T19:05:21.603 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.7719999999999997e-06 2024-09-20T19:05:21.603 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:21.603 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:21.603 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:05:21.603 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.821433+0000", 2024-09-20T19:05:21.603 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2680000000000001e-05 2024-09-20T19:05:21.603 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:21.604 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:21.604 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:05:21.604 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.821441+0000", 2024-09-20T19:05:21.604 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.2200000000000003e-06 2024-09-20T19:05:21.604 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:21.604 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:21.604 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:05:21.604 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.821509+0000", 2024-09-20T19:05:21.604 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.8205000000000002e-05 2024-09-20T19:05:21.604 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:21.604 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:21.604 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:05:21.604 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.821529+0000", 2024-09-20T19:05:21.604 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0242999999999999e-05 2024-09-20T19:05:21.604 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:05:21.605 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:05:21.605 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:05:21.605 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:21.605 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:21.605 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDPGPush(10.1 393/375 [PushOp(10:8016da6f:::benchmark_data_smithi083_148100_object85814:head, version: 348'5239, data_included: [0~65536], data_size: 65536, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(10:8016da6f:::benchmark_data_smithi083_148100_object85814:head@348'5239, size: 65536, copy_subset: [(0, 65536)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 65536, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-20T19:05:21.605 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:20.822379+0000", 2024-09-20T19:05:21.605 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 6.2689999999999996e-06, 2024-09-20T19:05:21.605 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015568600000000001, 2024-09-20T19:05:21.605 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:05:21.605 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:05:21.605 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "queued for pg", 2024-09-20T19:05:21.605 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:05:21.605 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:21.605 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:05:21.606 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.822379+0000", 2024-09-20T19:05:21.606 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:05:21.606 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:21.606 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:21.606 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:05:21.606 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.822379+0000", 2024-09-20T19:05:21.606 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:05:21.606 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:21.606 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:21.606 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:05:21.606 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.822380+0000", 2024-09-20T19:05:21.606 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.0200000000000001e-07 2024-09-20T19:05:21.606 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:21.606 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:21.607 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:05:21.607 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.822425+0000", 2024-09-20T19:05:21.607 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.5386999999999998e-05 2024-09-20T19:05:21.607 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:21.607 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:21.607 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:05:21.607 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.822446+0000", 2024-09-20T19:05:21.607 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0347999999999999e-05 2024-09-20T19:05:21.607 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:21.607 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:21.607 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:05:21.607 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.822461+0000", 2024-09-20T19:05:21.607 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5410999999999999e-05 2024-09-20T19:05:21.607 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:05:21.608 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:05:21.608 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:05:21.608 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:21.608 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:21.608 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "pg_backfill(progress 10.1 e 393/393 lb 10:8015ea0f:::benchmark_data_smithi083_148100_object113874:head)", 2024-09-20T19:05:21.608 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:20.822489+0000", 2024-09-20T19:05:21.608 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 4294967295.999896, 2024-09-20T19:05:21.608 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.2338999999999999e-05, 2024-09-20T19:05:21.608 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:05:21.608 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:05:21.608 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "queued for pg", 2024-09-20T19:05:21.608 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:05:21.608 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:21.608 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:05:21.608 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.822489+0000", 2024-09-20T19:05:21.609 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:05:21.609 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:21.609 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:21.609 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:05:21.609 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.822489+0000", 2024-09-20T19:05:21.609 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:05:21.609 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:21.609 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:21.609 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:05:21.609 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.822490+0000", 2024-09-20T19:05:21.609 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.8700000000000002e-07 2024-09-20T19:05:21.609 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:21.609 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:21.609 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:05:21.609 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.822497+0000", 2024-09-20T19:05:21.610 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.1729999999999998e-06 2024-09-20T19:05:21.610 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:21.610 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:21.610 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:05:21.610 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.822503+0000", 2024-09-20T19:05:21.610 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4350000000000002e-06 2024-09-20T19:05:21.610 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:21.610 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:21.610 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:05:21.610 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.822509+0000", 2024-09-20T19:05:21.610 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.113e-06 2024-09-20T19:05:21.610 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:05:21.610 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:05:21.610 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:05:21.610 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:21.611 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:21.611 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25775.0:196105 10.1 e393/375)", 2024-09-20T19:05:21.611 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:20.822517+0000", 2024-09-20T19:05:21.611 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 4294967295.9998689, 2024-09-20T19:05:21.611 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.1215e-05, 2024-09-20T19:05:21.611 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:05:21.611 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:05:21.611 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "queued for pg", 2024-09-20T19:05:21.611 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:05:21.611 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:21.611 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:05:21.611 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.822517+0000", 2024-09-20T19:05:21.611 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:05:21.611 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:21.611 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:21.612 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:05:21.612 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.822517+0000", 2024-09-20T19:05:21.612 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:05:21.612 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:21.612 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:21.612 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:05:21.612 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.822517+0000", 2024-09-20T19:05:21.612 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1500000000000001e-07 2024-09-20T19:05:21.612 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:21.612 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:21.612 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:05:21.612 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.822520+0000", 2024-09-20T19:05:21.612 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.1949999999999999e-06 2024-09-20T19:05:21.612 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:21.613 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:21.613 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:05:21.613 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.822525+0000", 2024-09-20T19:05:21.613 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.5499999999999996e-06 2024-09-20T19:05:21.613 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:05:21.613 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:05:21.613 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:05:21.613 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.822528+0000", 2024-09-20T19:05:21.613 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.8589999999999998e-06 2024-09-20T19:05:21.613 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:05:21.613 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:05:21.613 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:05:21.613 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:05:21.613 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-09-20T19:05:21.613 INFO:teuthology.orchestra.run.smithi142.stdout: "num_ops": 5 2024-09-20T19:05:21.614 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T19:05:21.614 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:21.637 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:21.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:21.638 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:21.739 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:21.749 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:21 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 5/125169 objects degraded (0.004%), 1 pg degraded (PG_DEGRADED) 2024-09-20T19:05:21.749 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:21 smithi083 ceph-mon[29421]: osdmap e394: 8 total, 8 up, 8 in 2024-09-20T19:05:21.750 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:21 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 5/125169 objects degraded (0.004%), 1 pg degraded (PG_DEGRADED) 2024-09-20T19:05:21.750 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:21 smithi083 ceph-mon[36260]: osdmap e394: 8 total, 8 up, 8 in 2024-09-20T19:05:21.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:21 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 5/125169 objects degraded (0.004%), 1 pg degraded (PG_DEGRADED) 2024-09-20T19:05:21.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:21 smithi142 ceph-mon[32508]: osdmap e394: 8 total, 8 up, 8 in 2024-09-20T19:05:21.766 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:21.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:21.767 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:21.867 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:21.889 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:21.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:21.890 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:21.990 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:22.023 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:22.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:22.024 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:22.124 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:05:22.151 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:22.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:22.152 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:05:22.192 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:05:22.192 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:05:22.253 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:22.276 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:22.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:22.277 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:22.378 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:22.400 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:22.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:22.401 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:22.502 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:22.526 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:22.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:22.527 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:22.612 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:22 smithi083 ceph-mon[29421]: pgmap v776: 148 pgs: 1 active+undersized+remapped, 1 active+clean+scrubbing, 2 active+remapped+backfill_wait+backfill_toofull, 2 active+recovering+undersized+remapped, 142 active+clean; 2.5 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 946 op/s; 5224/122448 objects misplaced (4.266%); 0 B/s, 280 objects/s recovering 2024-09-20T19:05:22.612 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:22 smithi083 ceph-mon[29421]: osdmap e395: 8 total, 8 up, 8 in 2024-09-20T19:05:22.612 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:22 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/355709954' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:05:22.613 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:22 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:05:22.613 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:22 smithi083 ceph-mon[36260]: pgmap v776: 148 pgs: 1 active+undersized+remapped, 1 active+clean+scrubbing, 2 active+remapped+backfill_wait+backfill_toofull, 2 active+recovering+undersized+remapped, 142 active+clean; 2.5 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 946 op/s; 5224/122448 objects misplaced (4.266%); 0 B/s, 280 objects/s recovering 2024-09-20T19:05:22.613 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:22 smithi083 ceph-mon[36260]: osdmap e395: 8 total, 8 up, 8 in 2024-09-20T19:05:22.613 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:22 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/355709954' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:05:22.613 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:22 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:05:22.628 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:22.652 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:22.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:22.653 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:22.754 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:22.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:22 smithi142 ceph-mon[32508]: pgmap v776: 148 pgs: 1 active+undersized+remapped, 1 active+clean+scrubbing, 2 active+remapped+backfill_wait+backfill_toofull, 2 active+recovering+undersized+remapped, 142 active+clean; 2.5 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 946 op/s; 5224/122448 objects misplaced (4.266%); 0 B/s, 280 objects/s recovering 2024-09-20T19:05:22.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:22 smithi142 ceph-mon[32508]: osdmap e395: 8 total, 8 up, 8 in 2024-09-20T19:05:22.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:22 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/355709954' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:05:22.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:22 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:05:22.777 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:22.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:22.778 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:22.879 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:22.901 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:22.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:22.902 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:22.912 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-20T19:05:22.939 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:05:22.939 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:05:22.959 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:05:22.959 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:05:22.975 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:05:22.975 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:05:22.991 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:05:22.991 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:05:23.003 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:23.017 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:05:23.017 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:05:23.030 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:23.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:23.031 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:23.037 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:05:23.037 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:05:23.063 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:05:23.064 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:05:23.095 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:05:23.095 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:05:23.131 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:23.177 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:23.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:23.178 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:23.278 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:23.300 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:23.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:23.301 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:23.402 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:23.441 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:23.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:23.442 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:23.543 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:05:23.564 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:23.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:23.565 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:05:23.666 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:23.690 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:23 smithi083 ceph-mon[29421]: 6.19 scrub starts 2024-09-20T19:05:23.690 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:23 smithi083 ceph-mon[29421]: 6.19 scrub ok 2024-09-20T19:05:23.691 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:23 smithi083 ceph-mon[29421]: 3.19 scrub starts 2024-09-20T19:05:23.691 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:23 smithi083 ceph-mon[29421]: 3.19 scrub ok 2024-09-20T19:05:23.691 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:23 smithi083 ceph-mon[29421]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:05:23.691 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:23 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T19:05:23.691 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:23 smithi083 ceph-mon[29421]: osdmap e396: 8 total, 8 up, 8 in 2024-09-20T19:05:23.691 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:23 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/4048254668' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:05:23.691 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:23 smithi083 ceph-mon[36260]: 6.19 scrub starts 2024-09-20T19:05:23.691 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:23 smithi083 ceph-mon[36260]: 6.19 scrub ok 2024-09-20T19:05:23.691 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:23 smithi083 ceph-mon[36260]: 3.19 scrub starts 2024-09-20T19:05:23.692 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:23 smithi083 ceph-mon[36260]: 3.19 scrub ok 2024-09-20T19:05:23.692 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:23 smithi083 ceph-mon[36260]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:05:23.692 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:23 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T19:05:23.692 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:23 smithi083 ceph-mon[36260]: osdmap e396: 8 total, 8 up, 8 in 2024-09-20T19:05:23.692 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:23 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/4048254668' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:05:23.697 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:23.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:23.698 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:23.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:23 smithi142 ceph-mon[32508]: 6.19 scrub starts 2024-09-20T19:05:23.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:23 smithi142 ceph-mon[32508]: 6.19 scrub ok 2024-09-20T19:05:23.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:23 smithi142 ceph-mon[32508]: 3.19 scrub starts 2024-09-20T19:05:23.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:23 smithi142 ceph-mon[32508]: 3.19 scrub ok 2024-09-20T19:05:23.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:23 smithi142 ceph-mon[32508]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:05:23.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:23 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T19:05:23.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:23 smithi142 ceph-mon[32508]: osdmap e396: 8 total, 8 up, 8 in 2024-09-20T19:05:23.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:23 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/4048254668' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:05:23.798 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:05:23.828 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:23.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:23.829 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:05:23.930 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:23.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:23.961 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:23.961 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:24.062 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:24.084 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:24.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:24.085 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:24.185 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:24.206 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:24.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:24.207 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:24.307 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:24.342 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:24.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:24.343 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:24.397 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.6 to 1 (10000) 2024-09-20T19:05:24.444 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:24.465 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:24.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:24.466 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:24.567 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:24.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:24.601 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:24.601 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:24.681 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:24 smithi083 ceph-mon[29421]: 5.16 scrub starts 2024-09-20T19:05:24.682 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:24 smithi083 ceph-mon[29421]: 5.16 scrub ok 2024-09-20T19:05:24.682 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:24 smithi083 ceph-mon[29421]: pgmap v779: 148 pgs: 2 active+remapped+backfilling, 2 active+undersized+remapped, 1 peering, 143 active+clean; 2.3 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.02k op/s; 3806/112881 objects misplaced (3.372%); 15 MiB/s, 532 objects/s recovering 2024-09-20T19:05:24.682 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:24 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1763645671' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-20T19:05:24.682 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:24 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-20T19:05:24.682 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:24 smithi083 ceph-mon[29421]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull) 2024-09-20T19:05:24.682 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:24 smithi083 ceph-mon[36260]: 5.16 scrub starts 2024-09-20T19:05:24.682 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:24 smithi083 ceph-mon[36260]: 5.16 scrub ok 2024-09-20T19:05:24.682 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:24 smithi083 ceph-mon[36260]: pgmap v779: 148 pgs: 2 active+remapped+backfilling, 2 active+undersized+remapped, 1 peering, 143 active+clean; 2.3 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.02k op/s; 3806/112881 objects misplaced (3.372%); 15 MiB/s, 532 objects/s recovering 2024-09-20T19:05:24.682 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:24 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1763645671' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-20T19:05:24.682 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:24 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-20T19:05:24.683 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:24 smithi083 ceph-mon[36260]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull) 2024-09-20T19:05:24.702 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:05:24.738 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:24.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:24.739 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:05:24.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:24 smithi142 ceph-mon[32508]: 5.16 scrub starts 2024-09-20T19:05:24.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:24 smithi142 ceph-mon[32508]: 5.16 scrub ok 2024-09-20T19:05:24.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:24 smithi142 ceph-mon[32508]: pgmap v779: 148 pgs: 2 active+remapped+backfilling, 2 active+undersized+remapped, 1 peering, 143 active+clean; 2.3 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.02k op/s; 3806/112881 objects misplaced (3.372%); 15 MiB/s, 532 objects/s recovering 2024-09-20T19:05:24.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:24 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1763645671' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-20T19:05:24.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:24 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-20T19:05:24.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:24 smithi142 ceph-mon[32508]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull) 2024-09-20T19:05:24.840 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:24.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:24.878 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:24.878 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:24.979 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:25.017 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:25.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:25.019 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:25.119 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:25.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:25.158 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:25.158 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:25.259 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:25.294 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:25.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:25.295 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:25.396 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:05:25.430 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:25.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:25.431 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:05:25.532 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:25.559 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd reweight 4 1 2024-09-20T19:05:25.572 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:25.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:25.573 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:25.674 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:25.714 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:25.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:25.715 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:25.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:25 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-20T19:05:25.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:25 smithi142 ceph-mon[32508]: osdmap e397: 8 total, 8 up, 8 in 2024-09-20T19:05:25.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:25 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-20T19:05:25.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:25 smithi083 ceph-mon[29421]: osdmap e397: 8 total, 8 up, 8 in 2024-09-20T19:05:25.792 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:25 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-20T19:05:25.792 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:25 smithi083 ceph-mon[36260]: osdmap e397: 8 total, 8 up, 8 in 2024-09-20T19:05:25.816 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:25.860 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_blocked_ops 2024-09-20T19:05:25.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:25.861 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:25.861 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:25.962 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:26.001 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:26.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:26.002 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:26.015 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:05:26.103 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:26.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:26.141 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:26.142 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:26.160 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T19:05:26.242 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:26.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:26.286 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:26.286 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:26.289 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:05:26.387 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:26.416 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:26.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:26.417 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:26.518 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:26.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:26.556 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:26.557 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:26.657 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:26.695 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:26.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:26.696 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:26.732 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T19:05:26.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:26 smithi142 ceph-mon[32508]: pgmap v781: 148 pgs: 2 active+remapped+backfilling, 2 active+undersized+remapped, 1 peering, 143 active+clean; 2.3 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 749 op/s; 3806/112881 objects misplaced (3.372%); 14 MiB/s, 401 objects/s recovering 2024-09-20T19:05:26.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:26 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 2 pgs undersized (PG_DEGRADED) 2024-09-20T19:05:26.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:26 smithi142 ceph-mon[32508]: osdmap e398: 8 total, 8 up, 8 in 2024-09-20T19:05:26.766 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:26 smithi083 ceph-mon[29421]: pgmap v781: 148 pgs: 2 active+remapped+backfilling, 2 active+undersized+remapped, 1 peering, 143 active+clean; 2.3 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 749 op/s; 3806/112881 objects misplaced (3.372%); 14 MiB/s, 401 objects/s recovering 2024-09-20T19:05:26.766 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:26 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 2 pgs undersized (PG_DEGRADED) 2024-09-20T19:05:26.766 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:26 smithi083 ceph-mon[29421]: osdmap e398: 8 total, 8 up, 8 in 2024-09-20T19:05:26.767 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:26 smithi083 ceph-mon[36260]: pgmap v781: 148 pgs: 2 active+remapped+backfilling, 2 active+undersized+remapped, 1 peering, 143 active+clean; 2.3 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 749 op/s; 3806/112881 objects misplaced (3.372%); 14 MiB/s, 401 objects/s recovering 2024-09-20T19:05:26.767 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:26 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 2 pgs undersized (PG_DEGRADED) 2024-09-20T19:05:26.767 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:26 smithi083 ceph-mon[36260]: osdmap e398: 8 total, 8 up, 8 in 2024-09-20T19:05:26.797 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:26.834 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:26.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:26.835 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:26.936 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:26.969 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:26.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:26.970 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:27.071 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:27.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:27.107 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:27.107 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:27.207 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:05:27.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:27.242 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:27.242 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:05:27.315 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T19:05:27.315 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:05:27.343 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:27.378 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:27.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:27.379 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:27.464 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:05:27 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:05:27.480 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:27.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:27.514 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:27.514 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:27.615 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:27.639 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:27.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:27.640 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:27.741 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:27.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:27 smithi142 ceph-mon[32508]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2 pgs undersized) 2024-09-20T19:05:27.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:27.786 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:27.786 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:27.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:27 smithi083 ceph-mon[29421]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2 pgs undersized) 2024-09-20T19:05:27.792 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:27 smithi083 ceph-mon[36260]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2 pgs undersized) 2024-09-20T19:05:27.888 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:27.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:27.926 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:27.926 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:28.027 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:28.061 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_historic_ops 2024-09-20T19:05:28.066 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:28.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:28.068 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:28.169 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:28.205 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:28.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:28.206 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:28.307 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:05:28.350 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:28.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:28.352 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:05:28.452 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:28.468 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:05:28.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:28.495 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:28.496 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:28.532 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.4 to 1 (10000) 2024-09-20T19:05:28.596 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:05:28.638 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:28.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:28.639 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:05:28.740 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:28.756 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:28 smithi142 ceph-mon[32508]: pgmap v783: 148 pgs: 2 active+remapped+backfilling, 146 active+clean; 2.1 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 975 B/s rd, 0 B/s wr, 1.01k op/s; 2457/104220 objects misplaced (2.358%); 24 MiB/s, 524 objects/s recovering 2024-09-20T19:05:28.756 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:28 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:05:28.756 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:28 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3985742493' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-20T19:05:28.756 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:28 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-20T19:05:28.772 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:28.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:28.773 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:28.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:28 smithi083 ceph-mon[29421]: pgmap v783: 148 pgs: 2 active+remapped+backfilling, 146 active+clean; 2.1 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 975 B/s rd, 0 B/s wr, 1.01k op/s; 2457/104220 objects misplaced (2.358%); 24 MiB/s, 524 objects/s recovering 2024-09-20T19:05:28.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:28 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:05:28.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:28 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3985742493' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-20T19:05:28.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:28 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-20T19:05:28.792 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:28 smithi083 ceph-mon[36260]: pgmap v783: 148 pgs: 2 active+remapped+backfilling, 146 active+clean; 2.1 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 975 B/s rd, 0 B/s wr, 1.01k op/s; 2457/104220 objects misplaced (2.358%); 24 MiB/s, 524 objects/s recovering 2024-09-20T19:05:28.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:28 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:05:28.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:28 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3985742493' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-20T19:05:28.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:28 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-20T19:05:28.873 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:28.910 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:28.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:28.911 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:29.012 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:29.060 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:29.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:29.061 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:29.086 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T19:05:29.086 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:05:29.162 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:29.192 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:29.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:29.193 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:29.294 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:29.338 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:29.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:29.339 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:29.440 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:29.477 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:29.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:29.478 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:29.579 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:29.616 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:29.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:29.617 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:29.685 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:29 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-20T19:05:29.685 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:29 smithi083 ceph-mon[29421]: osdmap e399: 8 total, 8 up, 8 in 2024-09-20T19:05:29.685 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:29 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-20T19:05:29.685 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:29 smithi083 ceph-mon[36260]: osdmap e399: 8 total, 8 up, 8 in 2024-09-20T19:05:29.718 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:29.728 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-09-20T19:05:29.729 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T19:05:29.748 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:29.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:29.749 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:29.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:29 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-20T19:05:29.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:29 smithi142 ceph-mon[32508]: osdmap e399: 8 total, 8 up, 8 in 2024-09-20T19:05:29.850 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:05:29.852 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-20T19:05:29.853 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:05:29.878 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:29.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:29.879 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:05:29.883 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:05:29.980 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:30.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:30.047 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:30.048 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:30.148 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:30.208 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:30.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:30.209 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:30.310 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:30.341 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:30.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:30.342 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:30.442 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:30.465 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:30.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:30.466 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:30.566 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:30.595 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:30.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:30.596 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:30.697 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:05:30.719 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:30.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:30.720 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:05:30.778 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:30 smithi083 ceph-mon[29421]: pgmap v785: 148 pgs: 2 active+remapped+backfilling, 146 active+clean; 2.1 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 615 op/s; 1548/101805 objects misplaced (1.521%); 21 MiB/s, 338 objects/s recovering 2024-09-20T19:05:30.779 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:30 smithi083 ceph-mon[36260]: pgmap v785: 148 pgs: 2 active+remapped+backfilling, 146 active+clean; 2.1 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 615 op/s; 1548/101805 objects misplaced (1.521%); 21 MiB/s, 338 objects/s recovering 2024-09-20T19:05:30.779 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:05:30 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:05:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:05:30.821 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:30.843 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:30.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:30.844 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:30.944 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:30.970 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:30.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:30.971 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:31.012 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:30 smithi142 ceph-mon[32508]: pgmap v785: 148 pgs: 2 active+remapped+backfilling, 146 active+clean; 2.1 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 615 op/s; 1548/101805 objects misplaced (1.521%); 21 MiB/s, 338 objects/s recovering 2024-09-20T19:05:31.072 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:31.120 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:31.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:31.121 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:31.222 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:31.254 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:31.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:31.255 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:31.334 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:05:31.334 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:05:31.334 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:05:31.334 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:05:31.334 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:05:31.335 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:05:31.335 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:05:31.335 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:05:31.353 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:05:31.353 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:05:31.356 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:31.384 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:31.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:31.385 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:31.404 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:05:31.404 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:05:31.454 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:05:31.454 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:05:31.484 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:05:31.485 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:31.500 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:05:31.500 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:05:31.512 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:05:31.543 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:31.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:31.544 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:31.645 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:31.680 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:31.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:31.681 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:31.762 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:31 smithi083 ceph-mon[29421]: osdmap e400: 8 total, 8 up, 8 in 2024-09-20T19:05:31.762 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:31 smithi083 ceph-mon[29421]: osdmap e401: 8 total, 8 up, 8 in 2024-09-20T19:05:31.762 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:31 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2059482277' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:05:31.762 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:31 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:05:31.762 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:31 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:05:31.763 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:31 smithi083 ceph-mon[36260]: osdmap e400: 8 total, 8 up, 8 in 2024-09-20T19:05:31.764 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:31 smithi083 ceph-mon[36260]: osdmap e401: 8 total, 8 up, 8 in 2024-09-20T19:05:31.764 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:31 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2059482277' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:05:31.764 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:31 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:05:31.764 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:31 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:05:31.782 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:31.819 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:31.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:31.820 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:31.921 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:31.956 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:31.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:31.957 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:32.012 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:31 smithi142 ceph-mon[32508]: osdmap e400: 8 total, 8 up, 8 in 2024-09-20T19:05:32.012 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:31 smithi142 ceph-mon[32508]: osdmap e401: 8 total, 8 up, 8 in 2024-09-20T19:05:32.012 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:31 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2059482277' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:05:32.012 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:31 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:05:32.012 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:31 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:05:32.058 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:32.093 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:32.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:32.094 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:32.194 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:32.229 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:32.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:32.230 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:32.330 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:32.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:32.365 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:32.365 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:32.466 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:32.504 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:32.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:32.505 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:32.536 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-20T19:05:32.558 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:32 smithi083 ceph-mon[29421]: pgmap v788: 148 pgs: 2 active+remapped+backfilling, 146 active+clean; 2.0 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 973 B/s rd, 0 B/s wr, 773 op/s; 1548/98988 objects misplaced (1.564%); 24 MiB/s, 385 objects/s recovering 2024-09-20T19:05:32.558 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:32 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/595794337' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:05:32.558 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:32 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:05:32.558 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:32 smithi083 ceph-mon[36260]: pgmap v788: 148 pgs: 2 active+remapped+backfilling, 146 active+clean; 2.0 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 973 B/s rd, 0 B/s wr, 773 op/s; 1548/98988 objects misplaced (1.564%); 24 MiB/s, 385 objects/s recovering 2024-09-20T19:05:32.558 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:32 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/595794337' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:05:32.558 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:32 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:05:32.606 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:32.634 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:32.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:32.635 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:32.736 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:05:32.756 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:32.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:32.757 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:05:32.858 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:32.880 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:32.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:32.880 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:32.981 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:33.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:33.004 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:33.005 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:33.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:32 smithi142 ceph-mon[32508]: pgmap v788: 148 pgs: 2 active+remapped+backfilling, 146 active+clean; 2.0 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 973 B/s rd, 0 B/s wr, 773 op/s; 1548/98988 objects misplaced (1.564%); 24 MiB/s, 385 objects/s recovering 2024-09-20T19:05:33.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:32 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/595794337' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:05:33.012 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:32 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:05:33.105 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:33.126 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:33.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:33.127 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:33.150 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T19:05:33.152 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-20T19:05:33.227 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:33.268 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:33.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:33.269 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:33.369 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:33.396 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:33.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:33.397 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:33.497 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:33.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:33.520 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:33.520 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:33.621 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:33.654 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:33.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:33.655 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:33.756 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:33.765 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:33 smithi083 ceph-mon[29421]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:05:33.765 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:33 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T19:05:33.765 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:33 smithi083 ceph-mon[29421]: osdmap e402: 8 total, 8 up, 8 in 2024-09-20T19:05:33.765 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:33 smithi083 ceph-mon[29421]: osdmap e403: 8 total, 8 up, 8 in 2024-09-20T19:05:33.766 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:33 smithi083 ceph-mon[36260]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:05:33.766 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:33 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T19:05:33.766 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:33 smithi083 ceph-mon[36260]: osdmap e402: 8 total, 8 up, 8 in 2024-09-20T19:05:33.766 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:33 smithi083 ceph-mon[36260]: osdmap e403: 8 total, 8 up, 8 in 2024-09-20T19:05:33.778 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:33.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:33.779 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:33.798 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":789,"stamp":"2024-09-20T19:05:32.524215+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1958877747,"num_objects":30081,"num_object_clones":0,"num_object_copies":90243,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":758,"num_objects_unfound":0,"num_objects_dirty":30081,"num_whiteouts":0,"num_read":2196,"num_read_kb":2053,"num_write":204673,"num_write_kb":7480650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32016,"num_bytes_recovered":1770071539,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92928,"ondisk_log_size":92928,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":447,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":10534288,"kb_used_data":6151080,"kb_used_omap":16,"kb_used_meta":4382831,"kb_avail":739426928,"statfs":{"total":767960285184,"available":757173174272,"internally_reserved":0,"allocated":6298705920,"data_stored":6295086062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16790,"internal_metadata":4488019562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-305332224,"num_objects":-4659,"num_object_clones":0,"num_object_copies":-13977,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-1699,"num_objects_unfound":0,"num_objects_dirty":-4659,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4659,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1505,"num_bytes_recovered":98631680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.376657"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":727,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-20T19:05:25.906026+0000","last_change":"2024-09-20T19:01:31.817967+0000","last_active":"2024-09-20T19:05:25.906026+0000","last_peered":"2024-09-20T19:05:25.906026+0000","last_clean":"2024-09-20T19:05:25.906026+0000","last_became_active":"2024-09-20T18:49:45.943201+0000","last_became_peered":"2024-09-20T18:49:45.943201+0000","last_unstale":"2024-09-20T19:05:25.906026+0000","last_undegraded":"2024-09-20T19:05:25.906026+0000","last_fullsized":"2024-09-20T19:05:25.906026+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:31.817909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:31.817909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"273'15","reported_seq":72,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.497740+0000","last_change":"2024-09-20T19:04:21.040978+0000","last_active":"2024-09-20T19:05:30.497740+0000","last_peered":"2024-09-20T19:05:30.497740+0000","last_clean":"2024-09-20T19:05:30.497740+0000","last_became_active":"2024-09-20T19:04:20.925853+0000","last_became_peered":"2024-09-20T19:04:20.925853+0000","last_unstale":"2024-09-20T19:05:30.497740+0000","last_undegraded":"2024-09-20T19:05:30.497740+0000","last_fullsized":"2024-09-20T19:05:30.497740+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:03:10.214154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:10.214154+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":723,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.914988+0000","last_change":"2024-09-20T18:57:49.016078+0000","last_active":"2024-09-20T19:05:30.914988+0000","last_peered":"2024-09-20T19:05:30.914988+0000","last_clean":"2024-09-20T19:05:30.914988+0000","last_became_active":"2024-09-20T18:49:49.959514+0000","last_became_peered":"2024-09-20T18:49:49.959514+0000","last_unstale":"2024-09-20T19:05:30.914988+0000","last_undegraded":"2024-09-20T19:05:30.914988+0000","last_fullsized":"2024-09-20T19:05:30.914988+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:49.016006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:49.016006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:25.184023+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":62,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.498621+0000","last_change":"2024-09-20T19:04:20.881918+0000","last_active":"2024-09-20T19:05:30.498621+0000","last_peered":"2024-09-20T19:05:30.498621+0000","last_clean":"2024-09-20T19:05:30.498621+0000","last_became_active":"2024-09-20T19:04:20.880905+0000","last_became_peered":"2024-09-20T19:04:20.880905+0000","last_unstale":"2024-09-20T19:05:30.498621+0000","last_undegraded":"2024-09-20T19:05:30.498621+0000","last_fullsized":"2024-09-20T19:05:30.498621+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:02.084375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:02.084375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":731,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.505984+0000","last_change":"2024-09-20T19:02:02.441633+0000","last_active":"2024-09-20T19:05:30.505984+0000","last_peered":"2024-09-20T19:05:30.505984+0000","last_clean":"2024-09-20T19:05:30.505984+0000","last_became_active":"2024-09-20T18:59:05.081081+0000","last_became_peered":"2024-09-20T18:59:05.081081+0000","last_unstale":"2024-09-20T19:05:30.505984+0000","last_undegraded":"2024-09-20T19:05:30.505984+0000","last_fullsized":"2024-09-20T19:05:30.505984+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:02.441577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:02.441577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:58.610226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":744,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.510511+0000","last_change":"2024-09-20T19:04:37.540824+0000","last_active":"2024-09-20T19:05:30.510511+0000","last_peered":"2024-09-20T19:05:30.510511+0000","last_clean":"2024-09-20T19:05:30.510511+0000","last_became_active":"2024-09-20T18:59:05.072675+0000","last_became_peered":"2024-09-20T18:59:05.072675+0000","last_unstale":"2024-09-20T19:05:30.510511+0000","last_undegraded":"2024-09-20T19:05:30.510511+0000","last_fullsized":"2024-09-20T19:05:30.510511+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:37.540760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:04:37.540760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:55.234054+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"274'27","reported_seq":768,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.506256+0000","last_change":"2024-09-20T19:02:40.899215+0000","last_active":"2024-09-20T19:05:30.506256+0000","last_peered":"2024-09-20T19:05:30.506256+0000","last_clean":"2024-09-20T19:05:30.506256+0000","last_became_active":"2024-09-20T18:49:47.952150+0000","last_became_peered":"2024-09-20T18:49:47.952150+0000","last_unstale":"2024-09-20T19:05:30.506256+0000","last_undegraded":"2024-09-20T19:05:30.506256+0000","last_fullsized":"2024-09-20T19:05:30.506256+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'27","last_scrub_stamp":"2024-09-20T19:02:40.899139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:40.899139+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":62,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.497912+0000","last_change":"2024-09-20T19:04:20.926234+0000","last_active":"2024-09-20T19:05:30.497912+0000","last_peered":"2024-09-20T19:05:30.497912+0000","last_clean":"2024-09-20T19:05:30.497912+0000","last_became_active":"2024-09-20T19:04:20.925394+0000","last_became_peered":"2024-09-20T19:04:20.925394+0000","last_unstale":"2024-09-20T19:05:30.497912+0000","last_undegraded":"2024-09-20T19:05:30.497912+0000","last_fullsized":"2024-09-20T19:05:30.497912+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:23.982813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:23.982813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"59'1","reported_seq":806,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.509627+0000","last_change":"2024-09-20T19:04:20.963496+0000","last_active":"2024-09-20T19:05:30.509627+0000","last_peered":"2024-09-20T19:05:30.509627+0000","last_clean":"2024-09-20T19:05:30.509627+0000","last_became_active":"2024-09-20T19:04:20.914145+0000","last_became_peered":"2024-09-20T19:04:20.914145+0000","last_unstale":"2024-09-20T19:05:30.509627+0000","last_undegraded":"2024-09-20T19:05:30.509627+0000","last_fullsized":"2024-09-20T19:05:30.509627+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:01:28.592653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:28.592653+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:29.742562+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"273'15","reported_seq":72,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.498657+0000","last_change":"2024-09-20T19:04:20.984223+0000","last_active":"2024-09-20T19:05:30.498657+0000","last_peered":"2024-09-20T19:05:30.498657+0000","last_clean":"2024-09-20T19:05:30.498657+0000","last_became_active":"2024-09-20T19:04:20.898235+0000","last_became_peered":"2024-09-20T19:04:20.898235+0000","last_unstale":"2024-09-20T19:05:30.498657+0000","last_undegraded":"2024-09-20T19:05:30.498657+0000","last_fullsized":"2024-09-20T19:05:30.498657+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:02:42.114331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:42.114331+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":729,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.510003+0000","last_change":"2024-09-20T19:01:51.564693+0000","last_active":"2024-09-20T19:05:30.510003+0000","last_peered":"2024-09-20T19:05:30.510003+0000","last_clean":"2024-09-20T19:05:30.510003+0000","last_became_active":"2024-09-20T18:49:49.961188+0000","last_became_peered":"2024-09-20T18:49:49.961188+0000","last_unstale":"2024-09-20T19:05:30.510003+0000","last_undegraded":"2024-09-20T19:05:30.510003+0000","last_fullsized":"2024-09-20T19:05:30.510003+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:51.564624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:51.564624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":706,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-20T19:05:25.904985+0000","last_change":"2024-09-20T19:03:09.963421+0000","last_active":"2024-09-20T19:05:25.904985+0000","last_peered":"2024-09-20T19:05:25.904985+0000","last_clean":"2024-09-20T19:05:25.904985+0000","last_became_active":"2024-09-20T18:49:51.966715+0000","last_became_peered":"2024-09-20T18:49:51.966715+0000","last_unstale":"2024-09-20T19:05:25.904985+0000","last_undegraded":"2024-09-20T19:05:25.904985+0000","last_fullsized":"2024-09-20T19:05:25.904985+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:09.963298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:03:09.963298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":754,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.509069+0000","last_change":"2024-09-20T19:04:20.879227+0000","last_active":"2024-09-20T19:05:30.509069+0000","last_peered":"2024-09-20T19:05:30.509069+0000","last_clean":"2024-09-20T19:05:30.509069+0000","last_became_active":"2024-09-20T19:04:20.878905+0000","last_became_peered":"2024-09-20T19:04:20.878905+0000","last_unstale":"2024-09-20T19:05:30.509069+0000","last_undegraded":"2024-09-20T19:05:30.509069+0000","last_fullsized":"2024-09-20T19:05:30.509069+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:48.520017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:48.520017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"273'7","reported_seq":68,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.498280+0000","last_change":"2024-09-20T19:04:20.907713+0000","last_active":"2024-09-20T19:05:30.498280+0000","last_peered":"2024-09-20T19:05:30.498280+0000","last_clean":"2024-09-20T19:05:30.498280+0000","last_became_active":"2024-09-20T19:04:20.888572+0000","last_became_peered":"2024-09-20T19:04:20.888572+0000","last_unstale":"2024-09-20T19:05:30.498280+0000","last_undegraded":"2024-09-20T19:05:30.498280+0000","last_fullsized":"2024-09-20T19:05:30.498280+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-20T18:58:33.660531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:33.660531+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":749,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.888727+0000","last_change":"2024-09-20T19:04:20.885873+0000","last_active":"2024-09-20T19:05:30.888727+0000","last_peered":"2024-09-20T19:05:30.888727+0000","last_clean":"2024-09-20T19:05:30.888727+0000","last_became_active":"2024-09-20T19:04:20.885417+0000","last_became_peered":"2024-09-20T19:04:20.885417+0000","last_unstale":"2024-09-20T19:05:30.888727+0000","last_undegraded":"2024-09-20T19:05:30.888727+0000","last_fullsized":"2024-09-20T19:05:30.888727+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:33.203312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:33.203312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":725,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.508469+0000","last_change":"2024-09-20T19:05:21.625378+0000","last_active":"2024-09-20T19:05:30.508469+0000","last_peered":"2024-09-20T19:05:30.508469+0000","last_clean":"2024-09-20T19:05:30.508469+0000","last_became_active":"2024-09-20T18:49:51.970663+0000","last_became_peered":"2024-09-20T18:49:51.970663+0000","last_unstale":"2024-09-20T19:05:30.508469+0000","last_undegraded":"2024-09-20T19:05:30.508469+0000","last_fullsized":"2024-09-20T19:05:30.508469+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:21.625311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:05:21.625311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:25.835298+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":62,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.499919+0000","last_change":"2024-09-20T19:04:20.907531+0000","last_active":"2024-09-20T19:05:30.499919+0000","last_peered":"2024-09-20T19:05:30.499919+0000","last_clean":"2024-09-20T19:05:30.499919+0000","last_became_active":"2024-09-20T19:04:20.906109+0000","last_became_peered":"2024-09-20T19:04:20.906109+0000","last_unstale":"2024-09-20T19:05:30.499919+0000","last_undegraded":"2024-09-20T19:05:30.499919+0000","last_fullsized":"2024-09-20T19:05:30.499919+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:00.906652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:00.906652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":751,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-20T19:05:25.910192+0000","last_change":"2024-09-20T19:04:20.907057+0000","last_active":"2024-09-20T19:05:25.910192+0000","last_peered":"2024-09-20T19:05:25.910192+0000","last_clean":"2024-09-20T19:05:25.910192+0000","last_became_active":"2024-09-20T19:04:20.906691+0000","last_became_peered":"2024-09-20T19:04:20.906691+0000","last_unstale":"2024-09-20T19:05:25.910192+0000","last_undegraded":"2024-09-20T19:05:25.910192+0000","last_fullsized":"2024-09-20T19:05:25.910192+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:44.998147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:44.998147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"273'27","reported_seq":781,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.921190+0000","last_change":"2024-09-20T19:03:09.456720+0000","last_active":"2024-09-20T19:05:30.921190+0000","last_peered":"2024-09-20T19:05:30.921190+0000","last_clean":"2024-09-20T19:05:30.921190+0000","last_became_active":"2024-09-20T18:49:47.950939+0000","last_became_peered":"2024-09-20T18:49:47.950939+0000","last_unstale":"2024-09-20T19:05:30.921190+0000","last_undegraded":"2024-09-20T19:05:30.921190+0000","last_fullsized":"2024-09-20T19:05:30.921190+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'27","last_scrub_stamp":"2024-09-20T19:03:09.456631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:09.456631+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:32.393537+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":740,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.916412+0000","last_change":"2024-09-20T19:04:20.890659+0000","last_active":"2024-09-20T19:05:30.916412+0000","last_peered":"2024-09-20T19:05:30.916412+0000","last_clean":"2024-09-20T19:05:30.916412+0000","last_became_active":"2024-09-20T19:04:20.890370+0000","last_became_peered":"2024-09-20T19:04:20.890370+0000","last_unstale":"2024-09-20T19:05:30.916412+0000","last_undegraded":"2024-09-20T19:05:30.916412+0000","last_fullsized":"2024-09-20T19:05:30.916412+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:05.888974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:05.888974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:26.204424+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":732,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.511431+0000","last_change":"2024-09-20T19:02:28.934597+0000","last_active":"2024-09-20T19:05:30.511431+0000","last_peered":"2024-09-20T19:05:30.511431+0000","last_clean":"2024-09-20T19:05:30.511431+0000","last_became_active":"2024-09-20T18:59:05.084496+0000","last_became_peered":"2024-09-20T18:59:05.084496+0000","last_unstale":"2024-09-20T19:05:30.511431+0000","last_undegraded":"2024-09-20T19:05:30.511431+0000","last_fullsized":"2024-09-20T19:05:30.511431+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:28.934264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:28.934264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:05.678324+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"59'1","reported_seq":66,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.498526+0000","last_change":"2024-09-20T19:04:20.905233+0000","last_active":"2024-09-20T19:05:30.498526+0000","last_peered":"2024-09-20T19:05:30.498526+0000","last_clean":"2024-09-20T19:05:30.498526+0000","last_became_active":"2024-09-20T19:04:20.887062+0000","last_became_peered":"2024-09-20T19:04:20.887062+0000","last_unstale":"2024-09-20T19:05:30.498526+0000","last_undegraded":"2024-09-20T19:05:30.498526+0000","last_fullsized":"2024-09-20T19:05:30.498526+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:03:08.218812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:03:08.218812+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":993,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"274'18","reported_seq":782,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.517472+0000","last_change":"2024-09-20T19:04:20.925690+0000","last_active":"2024-09-20T19:05:30.517472+0000","last_peered":"2024-09-20T19:05:30.517472+0000","last_clean":"2024-09-20T19:05:30.517472+0000","last_became_active":"2024-09-20T19:04:20.896343+0000","last_became_peered":"2024-09-20T19:04:20.896343+0000","last_unstale":"2024-09-20T19:05:30.517472+0000","last_undegraded":"2024-09-20T19:05:30.517472+0000","last_fullsized":"2024-09-20T19:05:30.517472+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:57:09.818999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:09.818999+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:29.636168+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":62,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.499292+0000","last_change":"2024-09-20T19:04:20.889786+0000","last_active":"2024-09-20T19:05:30.499292+0000","last_peered":"2024-09-20T19:05:30.499292+0000","last_clean":"2024-09-20T19:05:30.499292+0000","last_became_active":"2024-09-20T19:04:20.889430+0000","last_became_peered":"2024-09-20T19:04:20.889430+0000","last_unstale":"2024-09-20T19:05:30.499292+0000","last_undegraded":"2024-09-20T19:05:30.499292+0000","last_fullsized":"2024-09-20T19:05:30.499292+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:02.094473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:02.094473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":721,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.889063+0000","last_change":"2024-09-20T19:02:28.294390+0000","last_active":"2024-09-20T19:05:30.889063+0000","last_peered":"2024-09-20T19:05:30.889063+0000","last_clean":"2024-09-20T19:05:30.889063+0000","last_became_active":"2024-09-20T18:49:51.970058+0000","last_became_peered":"2024-09-20T18:49:51.970058+0000","last_unstale":"2024-09-20T19:05:30.889063+0000","last_undegraded":"2024-09-20T19:05:30.889063+0000","last_fullsized":"2024-09-20T19:05:30.889063+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T19:02:28.294292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:28.294292+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":757,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.915127+0000","last_change":"2024-09-20T19:05:22.202134+0000","last_active":"2024-09-20T19:05:30.915127+0000","last_peered":"2024-09-20T19:05:30.915127+0000","last_clean":"2024-09-20T19:05:30.915127+0000","last_became_active":"2024-09-20T19:04:20.881466+0000","last_became_peered":"2024-09-20T19:04:20.881466+0000","last_unstale":"2024-09-20T19:05:30.915127+0000","last_undegraded":"2024-09-20T19:05:30.915127+0000","last_fullsized":"2024-09-20T19:05:30.915127+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:05:22.201836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:22.201836+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:45.062354+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"274'20","reported_seq":788,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-20T19:05:25.905929+0000","last_change":"2024-09-20T19:03:16.982915+0000","last_active":"2024-09-20T19:05:25.905929+0000","last_peered":"2024-09-20T19:05:25.905929+0000","last_clean":"2024-09-20T19:05:25.905929+0000","last_became_active":"2024-09-20T19:03:16.968636+0000","last_became_peered":"2024-09-20T19:03:16.968636+0000","last_unstale":"2024-09-20T19:05:25.905929+0000","last_undegraded":"2024-09-20T19:05:25.905929+0000","last_fullsized":"2024-09-20T19:05:25.905929+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'20","last_scrub_stamp":"2024-09-20T19:02:35.096979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:35.096979+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"375'14","reported_seq":428,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.934774+0000","last_change":"2024-09-20T19:04:20.956210+0000","last_active":"2024-09-20T19:05:30.934774+0000","last_peered":"2024-09-20T19:05:30.934774+0000","last_clean":"2024-09-20T19:05:30.934774+0000","last_became_active":"2024-09-20T19:04:20.900782+0000","last_became_peered":"2024-09-20T19:04:20.900782+0000","last_unstale":"2024-09-20T19:05:30.934774+0000","last_undegraded":"2024-09-20T19:05:30.934774+0000","last_fullsized":"2024-09-20T19:05:30.934774+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"308'12","last_scrub_stamp":"2024-09-20T19:01:44.541156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:44.541156+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"398'12369","reported_seq":28570,"reported_epoch":398,"state":"active+remapped+backfilling","last_fresh":"2024-09-20T19:05:27.567991+0000","last_change":"2024-09-20T19:05:20.635200+0000","last_active":"2024-09-20T19:05:27.567991+0000","last_peered":"2024-09-20T19:05:27.567991+0000","last_clean":"2024-09-20T19:04:20.251202+0000","last_became_active":"2024-09-20T19:04:22.043451+0000","last_became_peered":"2024-09-20T19:04:22.043451+0000","last_unstale":"2024-09-20T19:05:27.567991+0000","last_undegraded":"2024-09-20T19:05:27.567991+0000","last_fullsized":"2024-09-20T19:05:27.567991+0000","mapping_epoch":375,"log_start":"361'7000","ondisk_log_start":"361'7000","created":325,"last_epoch_clean":366,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":5369,"log_dups_size":0,"ondisk_log_size":5369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":137428992,"num_objects":2097,"num_object_clones":0,"num_object_copies":6291,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":758,"num_objects_unfound":0,"num_objects_dirty":2097,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12369,"num_write_kb":462912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8008,"num_bytes_recovered":524812288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[0,7,1],"avail_no_missing":["0","1","7"],"object_location_counts":[{"shards":"0,1,7","objects":2097}],"blocked_by":[],"up_primary":4,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.f","version":"274'27","reported_seq":813,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.921767+0000","last_change":"2024-09-20T19:04:20.920811+0000","last_active":"2024-09-20T19:05:30.921767+0000","last_peered":"2024-09-20T19:05:30.921767+0000","last_clean":"2024-09-20T19:05:30.921767+0000","last_became_active":"2024-09-20T19:04:20.897049+0000","last_became_peered":"2024-09-20T19:04:20.897049+0000","last_unstale":"2024-09-20T19:05:30.921767+0000","last_undegraded":"2024-09-20T19:05:30.921767+0000","last_fullsized":"2024-09-20T19:05:30.921767+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'27","last_scrub_stamp":"2024-09-20T19:02:27.897138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:27.897138+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":730,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.506173+0000","last_change":"2024-09-20T19:04:39.537914+0000","last_active":"2024-09-20T19:05:30.506173+0000","last_peered":"2024-09-20T19:05:30.506173+0000","last_clean":"2024-09-20T19:05:30.506173+0000","last_became_active":"2024-09-20T18:49:45.936104+0000","last_became_peered":"2024-09-20T18:49:45.936104+0000","last_unstale":"2024-09-20T19:05:30.506173+0000","last_undegraded":"2024-09-20T19:05:30.506173+0000","last_fullsized":"2024-09-20T19:05:30.506173+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:39.537818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:04:39.537818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:44.147654+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"375'12","reported_seq":80,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.500010+0000","last_change":"2024-09-20T19:04:21.030980+0000","last_active":"2024-09-20T19:05:30.500010+0000","last_peered":"2024-09-20T19:05:30.500010+0000","last_clean":"2024-09-20T19:05:30.500010+0000","last_became_active":"2024-09-20T19:04:20.907228+0000","last_became_peered":"2024-09-20T19:04:20.907228+0000","last_unstale":"2024-09-20T19:05:30.500010+0000","last_undegraded":"2024-09-20T19:05:30.500010+0000","last_fullsized":"2024-09-20T19:05:30.500010+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"308'10","last_scrub_stamp":"2024-09-20T19:02:23.017626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:23.017626+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":717,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.509218+0000","last_change":"2024-09-20T19:02:22.736879+0000","last_active":"2024-09-20T19:05:30.509218+0000","last_peered":"2024-09-20T19:05:30.509218+0000","last_clean":"2024-09-20T19:05:30.509218+0000","last_became_active":"2024-09-20T18:49:51.971399+0000","last_became_peered":"2024-09-20T18:49:51.971399+0000","last_unstale":"2024-09-20T19:05:30.509218+0000","last_undegraded":"2024-09-20T19:05:30.509218+0000","last_fullsized":"2024-09-20T19:05:30.509218+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:22.736825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:22.736825+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.e","version":"400'12900","reported_seq":13984,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.657636+0000","last_change":"2024-09-20T19:04:21.970634+0000","last_active":"2024-09-20T19:05:30.657636+0000","last_peered":"2024-09-20T19:05:30.657636+0000","last_clean":"2024-09-20T19:05:30.657636+0000","last_became_active":"2024-09-20T19:04:21.956145+0000","last_became_peered":"2024-09-20T19:04:21.956145+0000","last_unstale":"2024-09-20T19:05:30.657636+0000","last_undegraded":"2024-09-20T19:05:30.657636+0000","last_fullsized":"2024-09-20T19:05:30.657636+0000","mapping_epoch":375,"log_start":"369'8328","ondisk_log_start":"369'8328","created":325,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4572,"log_dups_size":0,"ondisk_log_size":4572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":124256256,"num_objects":1896,"num_object_clones":0,"num_object_copies":5688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12900,"num_write_kb":473472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":453,"num_bytes_recovered":29687808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"273'30","reported_seq":784,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.505927+0000","last_change":"2024-09-20T19:03:06.038303+0000","last_active":"2024-09-20T19:05:30.505927+0000","last_peered":"2024-09-20T19:05:30.505927+0000","last_clean":"2024-09-20T19:05:30.505927+0000","last_became_active":"2024-09-20T18:49:48.499189+0000","last_became_peered":"2024-09-20T18:49:48.499189+0000","last_unstale":"2024-09-20T19:05:30.505927+0000","last_undegraded":"2024-09-20T19:05:30.505927+0000","last_fullsized":"2024-09-20T19:05:30.505927+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'30","last_scrub_stamp":"2024-09-20T19:03:06.038213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:06.038213+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":722,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.511937+0000","last_change":"2024-09-20T19:02:22.806587+0000","last_active":"2024-09-20T19:05:30.511937+0000","last_peered":"2024-09-20T19:05:30.511937+0000","last_clean":"2024-09-20T19:05:30.511937+0000","last_became_active":"2024-09-20T18:49:45.940544+0000","last_became_peered":"2024-09-20T18:49:45.940544+0000","last_unstale":"2024-09-20T19:05:30.511937+0000","last_undegraded":"2024-09-20T19:05:30.511937+0000","last_fullsized":"2024-09-20T19:05:30.511937+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:22.806509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:22.806509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:02.674388+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":62,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.497965+0000","last_change":"2024-09-20T19:04:20.889280+0000","last_active":"2024-09-20T19:05:30.497965+0000","last_peered":"2024-09-20T19:05:30.497965+0000","last_clean":"2024-09-20T19:05:30.497965+0000","last_became_active":"2024-09-20T19:04:20.888422+0000","last_became_peered":"2024-09-20T19:04:20.888422+0000","last_unstale":"2024-09-20T19:05:30.497965+0000","last_undegraded":"2024-09-20T19:05:30.497965+0000","last_fullsized":"2024-09-20T19:05:30.497965+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.229201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.229201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":62,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.498704+0000","last_change":"2024-09-20T19:04:20.890045+0000","last_active":"2024-09-20T19:05:30.498704+0000","last_peered":"2024-09-20T19:05:30.498704+0000","last_clean":"2024-09-20T19:05:30.498704+0000","last_became_active":"2024-09-20T19:04:20.889418+0000","last_became_peered":"2024-09-20T19:04:20.889418+0000","last_unstale":"2024-09-20T19:05:30.498704+0000","last_undegraded":"2024-09-20T19:05:30.498704+0000","last_fullsized":"2024-09-20T19:05:30.498704+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:27.704520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:27.704520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"401'12611","reported_seq":12776,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:31.857215+0000","last_change":"2024-09-20T19:02:11.828279+0000","last_active":"2024-09-20T19:05:31.857215+0000","last_peered":"2024-09-20T19:05:31.857215+0000","last_clean":"2024-09-20T19:05:31.857215+0000","last_became_active":"2024-09-20T19:02:11.827468+0000","last_became_peered":"2024-09-20T19:02:11.827468+0000","last_unstale":"2024-09-20T19:05:31.857215+0000","last_undegraded":"2024-09-20T19:05:31.857215+0000","last_fullsized":"2024-09-20T19:05:31.857215+0000","mapping_epoch":325,"log_start":"358'6555","ondisk_log_start":"358'6555","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":6056,"log_dups_size":0,"ondisk_log_size":6056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":116588544,"num_objects":1779,"num_object_clones":0,"num_object_copies":5337,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12611,"num_write_kb":460480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"274'28","reported_seq":78,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.497871+0000","last_change":"2024-09-20T19:04:20.976682+0000","last_active":"2024-09-20T19:05:30.497871+0000","last_peered":"2024-09-20T19:05:30.497871+0000","last_clean":"2024-09-20T19:05:30.497871+0000","last_became_active":"2024-09-20T19:04:20.897444+0000","last_became_peered":"2024-09-20T19:04:20.897444+0000","last_unstale":"2024-09-20T19:05:30.497871+0000","last_undegraded":"2024-09-20T19:05:30.497871+0000","last_fullsized":"2024-09-20T19:05:30.497871+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'28","last_scrub_stamp":"2024-09-20T19:03:06.269623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:06.269623+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":41,"num_read_kb":34,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":753,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-20T19:05:25.906240+0000","last_change":"2024-09-20T19:04:21.031015+0000","last_active":"2024-09-20T19:05:25.906240+0000","last_peered":"2024-09-20T19:05:25.906240+0000","last_clean":"2024-09-20T19:05:25.906240+0000","last_became_active":"2024-09-20T19:04:20.906790+0000","last_became_peered":"2024-09-20T19:04:20.906790+0000","last_unstale":"2024-09-20T19:05:25.906240+0000","last_undegraded":"2024-09-20T19:05:25.906240+0000","last_fullsized":"2024-09-20T19:05:25.906240+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:00:29.890681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:00:29.890681+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"375'13","reported_seq":966,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.505737+0000","last_change":"2024-09-20T19:04:20.916872+0000","last_active":"2024-09-20T19:05:30.505737+0000","last_peered":"2024-09-20T19:05:30.505737+0000","last_clean":"2024-09-20T19:05:30.505737+0000","last_became_active":"2024-09-20T19:04:20.896424+0000","last_became_peered":"2024-09-20T19:04:20.896424+0000","last_unstale":"2024-09-20T19:05:30.505737+0000","last_undegraded":"2024-09-20T19:05:30.505737+0000","last_fullsized":"2024-09-20T19:05:30.505737+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:58:23.114371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:23.114371+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:33.557657+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":727,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.888874+0000","last_change":"2024-09-20T18:59:05.082117+0000","last_active":"2024-09-20T19:05:30.888874+0000","last_peered":"2024-09-20T19:05:30.888874+0000","last_clean":"2024-09-20T19:05:30.888874+0000","last_became_active":"2024-09-20T18:59:05.081814+0000","last_became_peered":"2024-09-20T18:59:05.081814+0000","last_unstale":"2024-09-20T19:05:30.888874+0000","last_undegraded":"2024-09-20T19:05:30.888874+0000","last_fullsized":"2024-09-20T19:05:30.888874+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:29.530001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:29.530001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:34.493661+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"401'12605","reported_seq":30402,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:31.312314+0000","last_change":"2024-09-20T19:05:30.912897+0000","last_active":"2024-09-20T19:05:31.312314+0000","last_peered":"2024-09-20T19:05:31.312314+0000","last_clean":"2024-09-20T19:05:31.312314+0000","last_became_active":"2024-09-20T19:05:30.889717+0000","last_became_peered":"2024-09-20T19:05:30.889717+0000","last_unstale":"2024-09-20T19:05:31.312314+0000","last_undegraded":"2024-09-20T19:05:31.312314+0000","last_fullsized":"2024-09-20T19:05:31.312314+0000","mapping_epoch":400,"log_start":"365'7455","ondisk_log_start":"365'7455","created":325,"last_epoch_clean":401,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":5150,"log_dups_size":0,"ondisk_log_size":5150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":120913920,"num_objects":1845,"num_object_clones":0,"num_object_copies":5535,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12605,"num_write_kb":462400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8799,"num_bytes_recovered":576651264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"274'20","reported_seq":568,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.508916+0000","last_change":"2024-09-20T19:05:20.497269+0000","last_active":"2024-09-20T19:05:30.508916+0000","last_peered":"2024-09-20T19:05:30.508916+0000","last_clean":"2024-09-20T19:05:30.508916+0000","last_became_active":"2024-09-20T19:04:49.419140+0000","last_became_peered":"2024-09-20T19:04:49.419140+0000","last_unstale":"2024-09-20T19:05:30.508916+0000","last_undegraded":"2024-09-20T19:05:30.508916+0000","last_fullsized":"2024-09-20T19:05:30.508916+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'20","last_scrub_stamp":"2024-09-20T19:02:35.930910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:35.930910+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":746,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.509713+0000","last_change":"2024-09-20T19:02:30.686939+0000","last_active":"2024-09-20T19:05:30.509713+0000","last_peered":"2024-09-20T19:05:30.509713+0000","last_clean":"2024-09-20T19:05:30.509713+0000","last_became_active":"2024-09-20T18:49:45.942496+0000","last_became_peered":"2024-09-20T18:49:45.942496+0000","last_unstale":"2024-09-20T19:05:30.509713+0000","last_undegraded":"2024-09-20T19:05:30.509713+0000","last_fullsized":"2024-09-20T19:05:30.509713+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:30.686872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:30.686872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"262'10","reported_seq":938,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-20T19:05:25.909906+0000","last_change":"2024-09-20T19:02:04.994109+0000","last_active":"2024-09-20T19:05:25.909906+0000","last_peered":"2024-09-20T19:05:25.909906+0000","last_clean":"2024-09-20T19:05:25.909906+0000","last_became_active":"2024-09-20T18:59:05.080183+0000","last_became_peered":"2024-09-20T18:59:05.080183+0000","last_unstale":"2024-09-20T19:05:25.909906+0000","last_undegraded":"2024-09-20T19:05:25.909906+0000","last_fullsized":"2024-09-20T19:05:25.909906+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'10","last_scrub_stamp":"2024-09-20T19:02:04.994047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:04.994047+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":699,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-20T19:05:25.905140+0000","last_change":"2024-09-20T19:01:37.900410+0000","last_active":"2024-09-20T19:05:25.905140+0000","last_peered":"2024-09-20T19:05:25.905140+0000","last_clean":"2024-09-20T19:05:25.905140+0000","last_became_active":"2024-09-20T18:49:51.964780+0000","last_became_peered":"2024-09-20T18:49:51.964780+0000","last_unstale":"2024-09-20T19:05:25.905140+0000","last_undegraded":"2024-09-20T19:05:25.905140+0000","last_fullsized":"2024-09-20T19:05:25.905140+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:37.900328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:37.900328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.d","version":"401'13136","reported_seq":22391,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:31.903636+0000","last_change":"2024-09-20T19:05:23.028287+0000","last_active":"2024-09-20T19:05:31.903636+0000","last_peered":"2024-09-20T19:05:31.903636+0000","last_clean":"2024-09-20T19:05:31.903636+0000","last_became_active":"2024-09-20T19:05:22.993748+0000","last_became_peered":"2024-09-20T19:05:22.993748+0000","last_unstale":"2024-09-20T19:05:31.903636+0000","last_undegraded":"2024-09-20T19:05:31.903636+0000","last_fullsized":"2024-09-20T19:05:31.903636+0000","mapping_epoch":395,"log_start":"360'7117","ondisk_log_start":"360'7117","created":325,"last_epoch_clean":396,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":6019,"log_dups_size":0,"ondisk_log_size":6019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118685720,"num_objects":1812,"num_object_clones":0,"num_object_copies":5436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1812,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13136,"num_write_kb":478273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4538,"num_bytes_recovered":189071384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"274'29","reported_seq":766,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-20T19:05:25.910032+0000","last_change":"2024-09-20T19:00:04.810907+0000","last_active":"2024-09-20T19:05:25.910032+0000","last_peered":"2024-09-20T19:05:25.910032+0000","last_clean":"2024-09-20T19:05:25.910032+0000","last_became_active":"2024-09-20T18:49:48.497736+0000","last_became_peered":"2024-09-20T18:49:48.497736+0000","last_unstale":"2024-09-20T19:05:25.910032+0000","last_undegraded":"2024-09-20T19:05:25.910032+0000","last_fullsized":"2024-09-20T19:05:25.910032+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'29","last_scrub_stamp":"2024-09-20T19:00:04.810831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:00:04.810831+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":724,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.915739+0000","last_change":"2024-09-20T19:02:41.859231+0000","last_active":"2024-09-20T19:05:30.915739+0000","last_peered":"2024-09-20T19:05:30.915739+0000","last_clean":"2024-09-20T19:05:30.915739+0000","last_became_active":"2024-09-20T18:49:45.942253+0000","last_became_peered":"2024-09-20T18:49:45.942253+0000","last_unstale":"2024-09-20T19:05:30.915739+0000","last_undegraded":"2024-09-20T19:05:30.915739+0000","last_fullsized":"2024-09-20T19:05:30.915739+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:41.859168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:41.859168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":311,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.933863+0000","last_change":"2024-09-20T19:02:27.639446+0000","last_active":"2024-09-20T19:05:30.933863+0000","last_peered":"2024-09-20T19:05:30.933863+0000","last_clean":"2024-09-20T19:05:30.933863+0000","last_became_active":"2024-09-20T18:59:05.085636+0000","last_became_peered":"2024-09-20T18:59:05.085636+0000","last_unstale":"2024-09-20T19:05:30.933863+0000","last_undegraded":"2024-09-20T19:05:30.933863+0000","last_fullsized":"2024-09-20T19:05:30.933863+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:27.639384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:27.639384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":742,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.914897+0000","last_change":"2024-09-20T19:02:23.886386+0000","last_active":"2024-09-20T19:05:30.914897+0000","last_peered":"2024-09-20T19:05:30.914897+0000","last_clean":"2024-09-20T19:05:30.914897+0000","last_became_active":"2024-09-20T18:59:05.085131+0000","last_became_peered":"2024-09-20T18:59:05.085131+0000","last_unstale":"2024-09-20T19:05:30.914897+0000","last_undegraded":"2024-09-20T19:05:30.914897+0000","last_fullsized":"2024-09-20T19:05:30.914897+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:23.886308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:23.886308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"401'12555","reported_seq":12784,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:31.894413+0000","last_change":"2024-09-20T19:03:15.459244+0000","last_active":"2024-09-20T19:05:31.894413+0000","last_peered":"2024-09-20T19:05:31.894413+0000","last_clean":"2024-09-20T19:05:31.894413+0000","last_became_active":"2024-09-20T19:02:11.827305+0000","last_became_peered":"2024-09-20T19:02:11.827305+0000","last_unstale":"2024-09-20T19:05:31.894413+0000","last_undegraded":"2024-09-20T19:05:31.894413+0000","last_fullsized":"2024-09-20T19:05:31.894413+0000","mapping_epoch":325,"log_start":"358'6502","ondisk_log_start":"358'6502","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"350'5605","last_scrub_stamp":"2024-09-20T19:03:15.459174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:03:15.459174+0000","objects_scrubbed":5605,"log_size":6053,"log_dups_size":0,"ondisk_log_size":6053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":598,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118685696,"num_objects":1811,"num_object_clones":0,"num_object_copies":5433,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12555,"num_write_kb":459712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"353'41","reported_seq":1021,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.915644+0000","last_change":"2024-09-20T19:03:17.024385+0000","last_active":"2024-09-20T19:05:30.915644+0000","last_peered":"2024-09-20T19:05:30.915644+0000","last_clean":"2024-09-20T19:05:30.915644+0000","last_became_active":"2024-09-20T19:03:16.976916+0000","last_became_peered":"2024-09-20T19:03:16.976916+0000","last_unstale":"2024-09-20T19:05:30.915644+0000","last_undegraded":"2024-09-20T19:05:30.915644+0000","last_fullsized":"2024-09-20T19:05:30.915644+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'40","last_scrub_stamp":"2024-09-20T19:02:36.900916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:36.900916+0000","objects_scrubbed":10,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":62,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.498362+0000","last_change":"2024-09-20T19:04:20.899630+0000","last_active":"2024-09-20T19:05:30.498362+0000","last_peered":"2024-09-20T19:05:30.498362+0000","last_clean":"2024-09-20T19:05:30.498362+0000","last_became_active":"2024-09-20T19:04:20.899352+0000","last_became_peered":"2024-09-20T19:04:20.899352+0000","last_unstale":"2024-09-20T19:05:30.498362+0000","last_undegraded":"2024-09-20T19:05:30.498362+0000","last_fullsized":"2024-09-20T19:05:30.498362+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:00:31.630361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:00:31.630361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":786,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.509129+0000","last_change":"2024-09-20T19:03:11.910684+0000","last_active":"2024-09-20T19:05:30.509129+0000","last_peered":"2024-09-20T19:05:30.509129+0000","last_clean":"2024-09-20T19:05:30.509129+0000","last_became_active":"2024-09-20T18:59:05.086786+0000","last_became_peered":"2024-09-20T18:59:05.086786+0000","last_unstale":"2024-09-20T19:05:30.509129+0000","last_undegraded":"2024-09-20T19:05:30.509129+0000","last_fullsized":"2024-09-20T19:05:30.509129+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-20T19:03:11.909394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:03:11.909394+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0.0012781159999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":752,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-20T19:05:25.912722+0000","last_change":"2024-09-20T19:04:20.907534+0000","last_active":"2024-09-20T19:05:25.912722+0000","last_peered":"2024-09-20T19:05:25.912722+0000","last_clean":"2024-09-20T19:05:25.912722+0000","last_became_active":"2024-09-20T19:04:20.907038+0000","last_became_peered":"2024-09-20T19:04:20.907038+0000","last_unstale":"2024-09-20T19:05:25.912722+0000","last_undegraded":"2024-09-20T19:05:25.912722+0000","last_fullsized":"2024-09-20T19:05:25.912722+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:15.034340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:03:15.034340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":727,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.510178+0000","last_change":"2024-09-20T19:04:20.896616+0000","last_active":"2024-09-20T19:05:30.510178+0000","last_peered":"2024-09-20T19:05:30.510178+0000","last_clean":"2024-09-20T19:05:30.510178+0000","last_became_active":"2024-09-20T19:04:20.896281+0000","last_became_peered":"2024-09-20T19:04:20.896281+0000","last_unstale":"2024-09-20T19:05:30.510178+0000","last_undegraded":"2024-09-20T19:05:30.510178+0000","last_fullsized":"2024-09-20T19:05:30.510178+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:04.526683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:04.526683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:00.655405+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"400'12867","reported_seq":13030,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.651172+0000","last_change":"2024-09-20T19:02:11.825132+0000","last_active":"2024-09-20T19:05:30.651172+0000","last_peered":"2024-09-20T19:05:30.651172+0000","last_clean":"2024-09-20T19:05:30.651172+0000","last_became_active":"2024-09-20T19:02:11.824846+0000","last_became_peered":"2024-09-20T19:02:11.824846+0000","last_unstale":"2024-09-20T19:05:30.651172+0000","last_undegraded":"2024-09-20T19:05:30.651172+0000","last_fullsized":"2024-09-20T19:05:30.651172+0000","mapping_epoch":325,"log_start":"361'7127","ondisk_log_start":"361'7127","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":5740,"log_dups_size":0,"ondisk_log_size":5740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":122748928,"num_objects":1873,"num_object_clones":0,"num_object_copies":5619,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12867,"num_write_kb":471680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"273'17","reported_seq":769,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.916219+0000","last_change":"2024-09-20T19:02:24.853473+0000","last_active":"2024-09-20T19:05:30.916219+0000","last_peered":"2024-09-20T19:05:30.916219+0000","last_clean":"2024-09-20T19:05:30.916219+0000","last_became_active":"2024-09-20T18:49:47.950904+0000","last_became_peered":"2024-09-20T18:49:47.950904+0000","last_unstale":"2024-09-20T19:05:30.916219+0000","last_undegraded":"2024-09-20T19:05:30.916219+0000","last_fullsized":"2024-09-20T19:05:30.916219+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'17","last_scrub_stamp":"2024-09-20T19:02:24.853374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:24.853374+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":36,"num_read_kb":25,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":754,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.915144+0000","last_change":"2024-09-20T19:02:29.840761+0000","last_active":"2024-09-20T19:05:30.915144+0000","last_peered":"2024-09-20T19:05:30.915144+0000","last_clean":"2024-09-20T19:05:30.915144+0000","last_became_active":"2024-09-20T18:59:05.079789+0000","last_became_peered":"2024-09-20T18:59:05.079789+0000","last_unstale":"2024-09-20T19:05:30.915144+0000","last_undegraded":"2024-09-20T19:05:30.915144+0000","last_fullsized":"2024-09-20T19:05:30.915144+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:29.840601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:29.840601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":736,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.920958+0000","last_change":"2024-09-20T19:01:58.789304+0000","last_active":"2024-09-20T19:05:30.920958+0000","last_peered":"2024-09-20T19:05:30.920958+0000","last_clean":"2024-09-20T19:05:30.920958+0000","last_became_active":"2024-09-20T18:49:44.937135+0000","last_became_peered":"2024-09-20T18:49:44.937135+0000","last_unstale":"2024-09-20T19:05:30.920958+0000","last_undegraded":"2024-09-20T19:05:30.920958+0000","last_fullsized":"2024-09-20T19:05:30.920958+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:01:58.789251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:01:58.789251+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:05.405452+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00095558800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":711,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.921507+0000","last_change":"2024-09-20T19:01:17.747452+0000","last_active":"2024-09-20T19:05:30.921507+0000","last_peered":"2024-09-20T19:05:30.921507+0000","last_clean":"2024-09-20T19:05:30.921507+0000","last_became_active":"2024-09-20T18:49:49.954424+0000","last_became_peered":"2024-09-20T18:49:49.954424+0000","last_unstale":"2024-09-20T19:05:30.921507+0000","last_undegraded":"2024-09-20T19:05:30.921507+0000","last_fullsized":"2024-09-20T19:05:30.921507+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:17.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:17.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:58.423341+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":722,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.889165+0000","last_change":"2024-09-20T19:01:15.203591+0000","last_active":"2024-09-20T19:05:30.889165+0000","last_peered":"2024-09-20T19:05:30.889165+0000","last_clean":"2024-09-20T19:05:30.889165+0000","last_became_active":"2024-09-20T18:59:05.081459+0000","last_became_peered":"2024-09-20T18:59:05.081459+0000","last_unstale":"2024-09-20T19:05:30.889165+0000","last_undegraded":"2024-09-20T19:05:30.889165+0000","last_fullsized":"2024-09-20T19:05:30.889165+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:15.203529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:15.203529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"398'12338","reported_seq":12497,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-20T19:05:27.368289+0000","last_change":"2024-09-20T19:02:11.828477+0000","last_active":"2024-09-20T19:05:27.368289+0000","last_peered":"2024-09-20T19:05:27.368289+0000","last_clean":"2024-09-20T19:05:27.368289+0000","last_became_active":"2024-09-20T19:02:11.827996+0000","last_became_peered":"2024-09-20T19:02:11.827996+0000","last_unstale":"2024-09-20T19:05:27.368289+0000","last_undegraded":"2024-09-20T19:05:27.368289+0000","last_fullsized":"2024-09-20T19:05:27.368289+0000","mapping_epoch":325,"log_start":"360'6884","ondisk_log_start":"360'6884","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":5454,"log_dups_size":0,"ondisk_log_size":5454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":136839168,"num_objects":2088,"num_object_clones":0,"num_object_copies":6264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2088,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12338,"num_write_kb":461632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"273'24","reported_seq":767,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-20T19:05:25.906033+0000","last_change":"2024-09-20T18:59:06.022021+0000","last_active":"2024-09-20T19:05:25.906033+0000","last_peered":"2024-09-20T19:05:25.906033+0000","last_clean":"2024-09-20T19:05:25.906033+0000","last_became_active":"2024-09-20T18:49:47.949230+0000","last_became_peered":"2024-09-20T18:49:47.949230+0000","last_unstale":"2024-09-20T19:05:25.906033+0000","last_undegraded":"2024-09-20T19:05:25.906033+0000","last_fullsized":"2024-09-20T19:05:25.906033+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:59:06.021962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:59:06.021962+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":752,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-20T19:05:25.905318+0000","last_change":"2024-09-20T19:04:20.907250+0000","last_active":"2024-09-20T19:05:25.905318+0000","last_peered":"2024-09-20T19:05:25.905318+0000","last_clean":"2024-09-20T19:05:25.905318+0000","last_became_active":"2024-09-20T19:04:20.906969+0000","last_became_peered":"2024-09-20T19:04:20.906969+0000","last_unstale":"2024-09-20T19:05:25.905318+0000","last_undegraded":"2024-09-20T19:05:25.905318+0000","last_fullsized":"2024-09-20T19:05:25.905318+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:24.984022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:24.984022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":1682,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.888355+0000","last_change":"2024-09-20T19:02:02.494344+0000","last_active":"2024-09-20T19:05:30.888355+0000","last_peered":"2024-09-20T19:05:30.888355+0000","last_clean":"2024-09-20T19:05:30.888355+0000","last_became_active":"2024-09-20T18:49:44.936819+0000","last_became_peered":"2024-09-20T18:49:44.936819+0000","last_unstale":"2024-09-20T19:05:30.888355+0000","last_undegraded":"2024-09-20T19:05:30.888355+0000","last_fullsized":"2024-09-20T19:05:30.888355+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-20T19:02:02.494291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:02:02.494291+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0010996199999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":938,"num_read_kb":933,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":730,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.509952+0000","last_change":"2024-09-20T19:02:27.785566+0000","last_active":"2024-09-20T19:05:30.509952+0000","last_peered":"2024-09-20T19:05:30.509952+0000","last_clean":"2024-09-20T19:05:30.509952+0000","last_became_active":"2024-09-20T18:49:49.961066+0000","last_became_peered":"2024-09-20T18:49:49.961066+0000","last_unstale":"2024-09-20T19:05:30.509952+0000","last_undegraded":"2024-09-20T19:05:30.509952+0000","last_fullsized":"2024-09-20T19:05:30.509952+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:27.785524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:27.785524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":728,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.915376+0000","last_change":"2024-09-20T19:02:26.861253+0000","last_active":"2024-09-20T19:05:30.915376+0000","last_peered":"2024-09-20T19:05:30.915376+0000","last_clean":"2024-09-20T19:05:30.915376+0000","last_became_active":"2024-09-20T18:49:51.970343+0000","last_became_peered":"2024-09-20T18:49:51.970343+0000","last_unstale":"2024-09-20T19:05:30.915376+0000","last_undegraded":"2024-09-20T19:05:30.915376+0000","last_fullsized":"2024-09-20T19:05:30.915376+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:26.861185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:26.861185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"401'12789","reported_seq":12954,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:31.904574+0000","last_change":"2024-09-20T19:02:11.828065+0000","last_active":"2024-09-20T19:05:31.904574+0000","last_peered":"2024-09-20T19:05:31.904574+0000","last_clean":"2024-09-20T19:05:31.904574+0000","last_became_active":"2024-09-20T19:02:11.827628+0000","last_became_peered":"2024-09-20T19:02:11.827628+0000","last_unstale":"2024-09-20T19:05:31.904574+0000","last_undegraded":"2024-09-20T19:05:31.904574+0000","last_fullsized":"2024-09-20T19:05:31.904574+0000","mapping_epoch":325,"log_start":"358'6727","ondisk_log_start":"358'6727","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":6062,"log_dups_size":0,"ondisk_log_size":6062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115933184,"num_objects":1769,"num_object_clones":0,"num_object_copies":5307,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12789,"num_write_kb":465856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"375'34","reported_seq":480,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.934653+0000","last_change":"2024-09-20T19:04:20.988428+0000","last_active":"2024-09-20T19:05:30.934653+0000","last_peered":"2024-09-20T19:05:30.934653+0000","last_clean":"2024-09-20T19:05:30.934653+0000","last_became_active":"2024-09-20T19:04:20.928727+0000","last_became_peered":"2024-09-20T19:04:20.928727+0000","last_unstale":"2024-09-20T19:05:30.934653+0000","last_undegraded":"2024-09-20T19:05:30.934653+0000","last_fullsized":"2024-09-20T19:05:30.934653+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"308'32","last_scrub_stamp":"2024-09-20T19:02:25.699337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:25.699337+0000","objects_scrubbed":8,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":462,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":752,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.506670+0000","last_change":"2024-09-20T19:05:07.665281+0000","last_active":"2024-09-20T19:05:30.506670+0000","last_peered":"2024-09-20T19:05:30.506670+0000","last_clean":"2024-09-20T19:05:30.506670+0000","last_became_active":"2024-09-20T18:59:05.079111+0000","last_became_peered":"2024-09-20T18:59:05.079111+0000","last_unstale":"2024-09-20T19:05:30.506670+0000","last_undegraded":"2024-09-20T19:05:30.506670+0000","last_fullsized":"2024-09-20T19:05:30.506670+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:07.665167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:07.665167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:16.643707+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":801,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.888470+0000","last_change":"2024-09-20T19:03:14.368338+0000","last_active":"2024-09-20T19:05:30.888470+0000","last_peered":"2024-09-20T19:05:30.888470+0000","last_clean":"2024-09-20T19:05:30.888470+0000","last_became_active":"2024-09-20T18:59:05.086103+0000","last_became_peered":"2024-09-20T18:59:05.086103+0000","last_unstale":"2024-09-20T19:05:30.888470+0000","last_undegraded":"2024-09-20T19:05:30.888470+0000","last_fullsized":"2024-09-20T19:05:30.888470+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:03:14.368279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:03:14.368279+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":717,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.889155+0000","last_change":"2024-09-20T19:04:35.533709+0000","last_active":"2024-09-20T19:05:30.889155+0000","last_peered":"2024-09-20T19:05:30.889155+0000","last_clean":"2024-09-20T19:05:30.889155+0000","last_became_active":"2024-09-20T18:49:49.956128+0000","last_became_peered":"2024-09-20T18:49:49.956128+0000","last_unstale":"2024-09-20T19:05:30.889155+0000","last_undegraded":"2024-09-20T19:05:30.889155+0000","last_fullsized":"2024-09-20T19:05:30.889155+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:35.533633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:35.533633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:59.293116+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":751,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.509742+0000","last_change":"2024-09-20T19:04:20.899776+0000","last_active":"2024-09-20T19:05:30.509742+0000","last_peered":"2024-09-20T19:05:30.509742+0000","last_clean":"2024-09-20T19:05:30.509742+0000","last_became_active":"2024-09-20T19:04:20.899321+0000","last_became_peered":"2024-09-20T19:04:20.899321+0000","last_unstale":"2024-09-20T19:05:30.509742+0000","last_undegraded":"2024-09-20T19:05:30.509742+0000","last_fullsized":"2024-09-20T19:05:30.509742+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:30.557368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:30.557368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.0","version":"401'12825","reported_seq":22231,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:31.872173+0000","last_change":"2024-09-20T19:05:21.939972+0000","last_active":"2024-09-20T19:05:31.872173+0000","last_peered":"2024-09-20T19:05:31.872173+0000","last_clean":"2024-09-20T19:05:31.872173+0000","last_became_active":"2024-09-20T19:05:21.905829+0000","last_became_peered":"2024-09-20T19:05:21.905829+0000","last_unstale":"2024-09-20T19:05:31.872173+0000","last_undegraded":"2024-09-20T19:05:31.872173+0000","last_fullsized":"2024-09-20T19:05:31.872173+0000","mapping_epoch":394,"log_start":"358'6739","ondisk_log_start":"358'6739","created":325,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":6086,"log_dups_size":0,"ondisk_log_size":6086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117637120,"num_objects":1795,"num_object_clones":0,"num_object_copies":5385,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12825,"num_write_kb":467840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4613,"num_bytes_recovered":200736768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"274'19","reported_seq":74,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.499738+0000","last_change":"2024-09-20T19:04:21.040511+0000","last_active":"2024-09-20T19:05:30.499738+0000","last_peered":"2024-09-20T19:05:30.499738+0000","last_clean":"2024-09-20T19:05:30.499738+0000","last_became_active":"2024-09-20T19:04:20.897175+0000","last_became_peered":"2024-09-20T19:04:20.897175+0000","last_unstale":"2024-09-20T19:05:30.499738+0000","last_undegraded":"2024-09-20T19:05:30.499738+0000","last_fullsized":"2024-09-20T19:05:30.499738+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:58:24.616580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:24.616580+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":62,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.498222+0000","last_change":"2024-09-20T19:04:20.887455+0000","last_active":"2024-09-20T19:05:30.498222+0000","last_peered":"2024-09-20T19:05:30.498222+0000","last_clean":"2024-09-20T19:05:30.498222+0000","last_became_active":"2024-09-20T19:04:20.887102+0000","last_became_peered":"2024-09-20T19:04:20.887102+0000","last_unstale":"2024-09-20T19:05:30.498222+0000","last_undegraded":"2024-09-20T19:05:30.498222+0000","last_fullsized":"2024-09-20T19:05:30.498222+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.693993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.693993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":768,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.509620+0000","last_change":"2024-09-20T19:04:20.899830+0000","last_active":"2024-09-20T19:05:30.509620+0000","last_peered":"2024-09-20T19:05:30.509620+0000","last_clean":"2024-09-20T19:05:30.509620+0000","last_became_active":"2024-09-20T19:04:20.899331+0000","last_became_peered":"2024-09-20T19:04:20.899331+0000","last_unstale":"2024-09-20T19:05:30.509620+0000","last_undegraded":"2024-09-20T19:05:30.509620+0000","last_fullsized":"2024-09-20T19:05:30.509620+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:38.463404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:38.463404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:26.726228+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":731,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.508259+0000","last_change":"2024-09-20T19:04:35.529672+0000","last_active":"2024-09-20T19:05:30.508259+0000","last_peered":"2024-09-20T19:05:30.508259+0000","last_clean":"2024-09-20T19:05:30.508259+0000","last_became_active":"2024-09-20T18:59:05.079897+0000","last_became_peered":"2024-09-20T18:59:05.079897+0000","last_unstale":"2024-09-20T19:05:30.508259+0000","last_undegraded":"2024-09-20T19:05:30.508259+0000","last_fullsized":"2024-09-20T19:05:30.508259+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:35.529584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:04:35.529584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:57.166916+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"401'12995","reported_seq":13273,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:31.578894+0000","last_change":"2024-09-20T19:05:17.450180+0000","last_active":"2024-09-20T19:05:31.578894+0000","last_peered":"2024-09-20T19:05:31.578894+0000","last_clean":"2024-09-20T19:05:31.578894+0000","last_became_active":"2024-09-20T19:02:11.825011+0000","last_became_peered":"2024-09-20T19:02:11.825011+0000","last_unstale":"2024-09-20T19:05:31.578894+0000","last_undegraded":"2024-09-20T19:05:31.578894+0000","last_fullsized":"2024-09-20T19:05:31.578894+0000","mapping_epoch":325,"log_start":"355'6428","ondisk_log_start":"355'6428","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"392'12286","last_scrub_stamp":"2024-09-20T19:05:17.450066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:05:17.450066+0000","objects_scrubbed":2624,"log_size":6567,"log_dups_size":0,"ondisk_log_size":6567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:20.515552+0000","scrub_duration":996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123273216,"num_objects":1881,"num_object_clones":0,"num_object_copies":5643,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12995,"num_write_kb":476032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"274'31","reported_seq":80,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.499514+0000","last_change":"2024-09-20T19:04:20.949305+0000","last_active":"2024-09-20T19:05:30.499514+0000","last_peered":"2024-09-20T19:05:30.499514+0000","last_clean":"2024-09-20T19:05:30.499514+0000","last_became_active":"2024-09-20T19:04:20.883306+0000","last_became_peered":"2024-09-20T19:04:20.883306+0000","last_unstale":"2024-09-20T19:05:30.499514+0000","last_undegraded":"2024-09-20T19:05:30.499514+0000","last_fullsized":"2024-09-20T19:05:30.499514+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'31","last_scrub_stamp":"2024-09-20T19:03:12.196851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:12.196851+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":49,"num_read_kb":39,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":327,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.933951+0000","last_change":"2024-09-20T19:04:20.899712+0000","last_active":"2024-09-20T19:05:30.933951+0000","last_peered":"2024-09-20T19:05:30.933951+0000","last_clean":"2024-09-20T19:05:30.933951+0000","last_became_active":"2024-09-20T19:04:20.899415+0000","last_became_peered":"2024-09-20T19:04:20.899415+0000","last_unstale":"2024-09-20T19:05:30.933951+0000","last_undegraded":"2024-09-20T19:05:30.933951+0000","last_fullsized":"2024-09-20T19:05:30.933951+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:42.479489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:42.479489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":748,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.915462+0000","last_change":"2024-09-20T19:04:20.892646+0000","last_active":"2024-09-20T19:05:30.915462+0000","last_peered":"2024-09-20T19:05:30.915462+0000","last_clean":"2024-09-20T19:05:30.915462+0000","last_became_active":"2024-09-20T19:04:20.891772+0000","last_became_peered":"2024-09-20T19:04:20.891772+0000","last_unstale":"2024-09-20T19:05:30.915462+0000","last_undegraded":"2024-09-20T19:05:30.915462+0000","last_fullsized":"2024-09-20T19:05:30.915462+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:32.856856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:32.856856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":747,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.921866+0000","last_change":"2024-09-20T19:04:20.897937+0000","last_active":"2024-09-20T19:05:30.921866+0000","last_peered":"2024-09-20T19:05:30.921866+0000","last_clean":"2024-09-20T19:05:30.921866+0000","last_became_active":"2024-09-20T19:04:20.897569+0000","last_became_peered":"2024-09-20T19:04:20.897569+0000","last_unstale":"2024-09-20T19:05:30.921866+0000","last_undegraded":"2024-09-20T19:05:30.921866+0000","last_fullsized":"2024-09-20T19:05:30.921866+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:31.687188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:31.687188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:24.424476+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2","version":"401'12857","reported_seq":14369,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:31.943471+0000","last_change":"2024-09-20T19:04:21.974196+0000","last_active":"2024-09-20T19:05:31.943471+0000","last_peered":"2024-09-20T19:05:31.943471+0000","last_clean":"2024-09-20T19:05:31.943471+0000","last_became_active":"2024-09-20T19:04:21.956633+0000","last_became_peered":"2024-09-20T19:04:21.956633+0000","last_unstale":"2024-09-20T19:05:31.943471+0000","last_undegraded":"2024-09-20T19:05:31.943471+0000","last_fullsized":"2024-09-20T19:05:31.943471+0000","mapping_epoch":375,"log_start":"366'7682","ondisk_log_start":"366'7682","created":325,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":5175,"log_dups_size":0,"ondisk_log_size":5175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:28.169249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118685696,"num_objects":1811,"num_object_clones":0,"num_object_copies":5433,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12857,"num_write_kb":469376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":665,"num_bytes_recovered":43581440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"273'20","reported_seq":74,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.499105+0000","last_change":"2024-09-20T19:04:20.945143+0000","last_active":"2024-09-20T19:05:30.499105+0000","last_peered":"2024-09-20T19:05:30.499105+0000","last_clean":"2024-09-20T19:05:30.499105+0000","last_became_active":"2024-09-20T19:04:20.896483+0000","last_became_peered":"2024-09-20T19:04:20.896483+0000","last_unstale":"2024-09-20T19:05:30.499105+0000","last_undegraded":"2024-09-20T19:05:30.499105+0000","last_fullsized":"2024-09-20T19:05:30.499105+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'20","last_scrub_stamp":"2024-09-20T19:03:04.282800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:04.282800+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":755,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.506555+0000","last_change":"2024-09-20T19:04:20.893263+0000","last_active":"2024-09-20T19:05:30.506555+0000","last_peered":"2024-09-20T19:05:30.506555+0000","last_clean":"2024-09-20T19:05:30.506555+0000","last_became_active":"2024-09-20T19:04:20.892951+0000","last_became_peered":"2024-09-20T19:04:20.892951+0000","last_unstale":"2024-09-20T19:05:30.506555+0000","last_undegraded":"2024-09-20T19:05:30.506555+0000","last_fullsized":"2024-09-20T19:05:30.506555+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:34.985163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:34.985163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":940,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.922151+0000","last_change":"2024-09-20T19:05:08.406307+0000","last_active":"2024-09-20T19:05:30.922151+0000","last_peered":"2024-09-20T19:05:30.922151+0000","last_clean":"2024-09-20T19:05:30.922151+0000","last_became_active":"2024-09-20T18:49:49.957005+0000","last_became_peered":"2024-09-20T18:49:49.957005+0000","last_unstale":"2024-09-20T19:05:30.922151+0000","last_undegraded":"2024-09-20T19:05:30.922151+0000","last_fullsized":"2024-09-20T19:05:30.922151+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T19:05:08.406231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:05:08.406231+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:20.576797+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":62,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.499827+0000","last_change":"2024-09-20T19:04:20.885294+0000","last_active":"2024-09-20T19:05:30.499827+0000","last_peered":"2024-09-20T19:05:30.499827+0000","last_clean":"2024-09-20T19:05:30.499827+0000","last_became_active":"2024-09-20T19:04:20.884994+0000","last_became_peered":"2024-09-20T19:04:20.884994+0000","last_unstale":"2024-09-20T19:05:30.499827+0000","last_undegraded":"2024-09-20T19:05:30.499827+0000","last_fullsized":"2024-09-20T19:05:30.499827+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:11.207994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:03:11.207994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.5","version":"401'12722","reported_seq":12962,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:31.582158+0000","last_change":"2024-09-20T19:04:04.390108+0000","last_active":"2024-09-20T19:05:31.582158+0000","last_peered":"2024-09-20T19:05:31.582158+0000","last_clean":"2024-09-20T19:05:31.582158+0000","last_became_active":"2024-09-20T19:02:11.828162+0000","last_became_peered":"2024-09-20T19:02:11.828162+0000","last_unstale":"2024-09-20T19:05:31.582158+0000","last_undegraded":"2024-09-20T19:05:31.582158+0000","last_fullsized":"2024-09-20T19:05:31.582158+0000","mapping_epoch":325,"log_start":"355'6168","ondisk_log_start":"355'6168","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"367'7875","last_scrub_stamp":"2024-09-20T19:04:04.389838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:04:04.389838+0000","objects_scrubbed":6698,"log_size":6554,"log_dups_size":0,"ondisk_log_size":6554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"queued for scrub","scrub_duration":2035,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117702656,"num_objects":1796,"num_object_clones":0,"num_object_copies":5388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12722,"num_write_kb":464576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"274'15","reported_seq":792,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-20T19:05:25.909953+0000","last_change":"2024-09-20T19:04:20.978525+0000","last_active":"2024-09-20T19:05:25.909953+0000","last_peered":"2024-09-20T19:05:25.909953+0000","last_clean":"2024-09-20T19:05:25.909953+0000","last_became_active":"2024-09-20T19:04:20.906834+0000","last_became_peered":"2024-09-20T19:04:20.906834+0000","last_unstale":"2024-09-20T19:05:25.909953+0000","last_undegraded":"2024-09-20T19:05:25.909953+0000","last_fullsized":"2024-09-20T19:05:25.909953+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'15","last_scrub_stamp":"2024-09-20T19:01:27.802100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:27.802100+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":284,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.920799+0000","last_change":"2024-09-20T19:04:16.384877+0000","last_active":"2024-09-20T19:05:30.920799+0000","last_peered":"2024-09-20T19:05:30.920799+0000","last_clean":"2024-09-20T19:05:30.920799+0000","last_became_active":"2024-09-20T19:00:01.190875+0000","last_became_peered":"2024-09-20T19:00:01.190875+0000","last_unstale":"2024-09-20T19:05:30.920799+0000","last_undegraded":"2024-09-20T19:05:30.920799+0000","last_fullsized":"2024-09-20T19:05:30.920799+0000","mapping_epoch":277,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":278,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:16.384809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:04:16.384809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:44.880751+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":760,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.920770+0000","last_change":"2024-09-20T19:04:20.889614+0000","last_active":"2024-09-20T19:05:30.920770+0000","last_peered":"2024-09-20T19:05:30.920770+0000","last_clean":"2024-09-20T19:05:30.920770+0000","last_became_active":"2024-09-20T19:04:20.889273+0000","last_became_peered":"2024-09-20T19:04:20.889273+0000","last_unstale":"2024-09-20T19:05:30.920770+0000","last_undegraded":"2024-09-20T19:05:30.920770+0000","last_fullsized":"2024-09-20T19:05:30.920770+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:03.389611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:03.389611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:26.422333+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":715,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-20T19:05:25.905204+0000","last_change":"2024-09-20T19:02:31.120809+0000","last_active":"2024-09-20T19:05:25.905204+0000","last_peered":"2024-09-20T19:05:25.905204+0000","last_clean":"2024-09-20T19:05:25.905204+0000","last_became_active":"2024-09-20T18:49:51.964098+0000","last_became_peered":"2024-09-20T18:49:51.964098+0000","last_unstale":"2024-09-20T19:05:25.905204+0000","last_undegraded":"2024-09-20T19:05:25.905204+0000","last_fullsized":"2024-09-20T19:05:25.905204+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:31.120623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:31.120623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"398'12424","reported_seq":12603,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-20T19:05:27.520433+0000","last_change":"2024-09-20T19:04:21.973100+0000","last_active":"2024-09-20T19:05:27.520433+0000","last_peered":"2024-09-20T19:05:27.520433+0000","last_clean":"2024-09-20T19:05:27.520433+0000","last_became_active":"2024-09-20T19:04:21.960863+0000","last_became_peered":"2024-09-20T19:04:21.960863+0000","last_unstale":"2024-09-20T19:05:27.520433+0000","last_undegraded":"2024-09-20T19:05:27.520433+0000","last_fullsized":"2024-09-20T19:05:27.520433+0000","mapping_epoch":375,"log_start":"361'6997","ondisk_log_start":"361'6997","created":325,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":5427,"log_dups_size":0,"ondisk_log_size":5427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128188416,"num_objects":1956,"num_object_clones":0,"num_object_copies":5868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1956,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12424,"num_write_kb":460160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"274'35","reported_seq":354,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.933724+0000","last_change":"2024-09-20T19:03:09.841596+0000","last_active":"2024-09-20T19:05:30.933724+0000","last_peered":"2024-09-20T19:05:30.933724+0000","last_clean":"2024-09-20T19:05:30.933724+0000","last_became_active":"2024-09-20T18:59:05.087391+0000","last_became_peered":"2024-09-20T18:59:05.087391+0000","last_unstale":"2024-09-20T19:05:30.933724+0000","last_undegraded":"2024-09-20T19:05:30.933724+0000","last_fullsized":"2024-09-20T19:05:30.933724+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'35","last_scrub_stamp":"2024-09-20T19:03:09.841361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:09.841361+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":746,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.889441+0000","last_change":"2024-09-20T19:02:05.440745+0000","last_active":"2024-09-20T19:05:30.889441+0000","last_peered":"2024-09-20T19:05:30.889441+0000","last_clean":"2024-09-20T19:05:30.889441+0000","last_became_active":"2024-09-20T18:59:05.086670+0000","last_became_peered":"2024-09-20T18:59:05.086670+0000","last_unstale":"2024-09-20T19:05:30.889441+0000","last_undegraded":"2024-09-20T19:05:30.889441+0000","last_fullsized":"2024-09-20T19:05:30.889441+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:05.440677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:05.440677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":743,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.921122+0000","last_change":"2024-09-20T19:05:11.444197+0000","last_active":"2024-09-20T19:05:30.921122+0000","last_peered":"2024-09-20T19:05:30.921122+0000","last_clean":"2024-09-20T19:05:30.921122+0000","last_became_active":"2024-09-20T18:49:49.957558+0000","last_became_peered":"2024-09-20T18:49:49.957558+0000","last_unstale":"2024-09-20T19:05:30.921122+0000","last_undegraded":"2024-09-20T19:05:30.921122+0000","last_fullsized":"2024-09-20T19:05:30.921122+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:11.444143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:05:11.444143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:23.300683+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":713,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.888351+0000","last_change":"2024-09-20T18:53:47.370152+0000","last_active":"2024-09-20T19:05:30.888351+0000","last_peered":"2024-09-20T19:05:30.888351+0000","last_clean":"2024-09-20T19:05:30.888351+0000","last_became_active":"2024-09-20T18:49:51.970798+0000","last_became_peered":"2024-09-20T18:49:51.970798+0000","last_unstale":"2024-09-20T19:05:30.888351+0000","last_undegraded":"2024-09-20T19:05:30.888351+0000","last_fullsized":"2024-09-20T19:05:30.888351+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:47.370108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:47.370108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:30.728805+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.7","version":"401'12923","reported_seq":13088,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:31.902268+0000","last_change":"2024-09-20T19:02:11.827681+0000","last_active":"2024-09-20T19:05:31.902268+0000","last_peered":"2024-09-20T19:05:31.902268+0000","last_clean":"2024-09-20T19:05:31.902268+0000","last_became_active":"2024-09-20T19:02:11.827360+0000","last_became_peered":"2024-09-20T19:02:11.827360+0000","last_unstale":"2024-09-20T19:05:31.902268+0000","last_undegraded":"2024-09-20T19:05:31.902268+0000","last_fullsized":"2024-09-20T19:05:31.902268+0000","mapping_epoch":325,"log_start":"359'6907","ondisk_log_start":"359'6907","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":6016,"log_dups_size":0,"ondisk_log_size":6016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118030336,"num_objects":1801,"num_object_clones":0,"num_object_copies":5403,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1801,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12923,"num_write_kb":471168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"273'18","reported_seq":74,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.498139+0000","last_change":"2024-09-20T19:04:20.936465+0000","last_active":"2024-09-20T19:05:30.498139+0000","last_peered":"2024-09-20T19:05:30.498139+0000","last_clean":"2024-09-20T19:05:30.498139+0000","last_became_active":"2024-09-20T19:04:20.888905+0000","last_became_peered":"2024-09-20T19:04:20.888905+0000","last_unstale":"2024-09-20T19:05:30.498139+0000","last_undegraded":"2024-09-20T19:05:30.498139+0000","last_fullsized":"2024-09-20T19:05:30.498139+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'18","last_scrub_stamp":"2024-09-20T19:03:13.201592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:13.201592+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":765,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.889432+0000","last_change":"2024-09-20T19:04:50.680303+0000","last_active":"2024-09-20T19:05:30.889432+0000","last_peered":"2024-09-20T19:05:30.889432+0000","last_clean":"2024-09-20T19:05:30.889432+0000","last_became_active":"2024-09-20T19:04:20.892429+0000","last_became_peered":"2024-09-20T19:04:20.892429+0000","last_unstale":"2024-09-20T19:05:30.889432+0000","last_undegraded":"2024-09-20T19:05:30.889432+0000","last_fullsized":"2024-09-20T19:05:30.889432+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:50.680222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:04:50.680222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:57.765584+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":735,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.921701+0000","last_change":"2024-09-20T19:04:48.406445+0000","last_active":"2024-09-20T19:05:30.921701+0000","last_peered":"2024-09-20T19:05:30.921701+0000","last_clean":"2024-09-20T19:05:30.921701+0000","last_became_active":"2024-09-20T18:49:49.954944+0000","last_became_peered":"2024-09-20T18:49:49.954944+0000","last_unstale":"2024-09-20T19:05:30.921701+0000","last_undegraded":"2024-09-20T19:05:30.921701+0000","last_fullsized":"2024-09-20T19:05:30.921701+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:48.406373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:48.406373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:58.629290+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":717,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.506083+0000","last_change":"2024-09-20T19:02:35.968178+0000","last_active":"2024-09-20T19:05:30.506083+0000","last_peered":"2024-09-20T19:05:30.506083+0000","last_clean":"2024-09-20T19:05:30.506083+0000","last_became_active":"2024-09-20T18:49:51.972970+0000","last_became_peered":"2024-09-20T18:49:51.972970+0000","last_unstale":"2024-09-20T19:05:30.506083+0000","last_undegraded":"2024-09-20T19:05:30.506083+0000","last_fullsized":"2024-09-20T19:05:30.506083+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:35.968052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:35.968052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.6","version":"400'12922","reported_seq":22392,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.729167+0000","last_change":"2024-09-20T19:05:21.938456+0000","last_active":"2024-09-20T19:05:30.729167+0000","last_peered":"2024-09-20T19:05:30.729167+0000","last_clean":"2024-09-20T19:05:30.729167+0000","last_became_active":"2024-09-20T19:05:21.906044+0000","last_became_peered":"2024-09-20T19:05:21.906044+0000","last_unstale":"2024-09-20T19:05:30.729167+0000","last_undegraded":"2024-09-20T19:05:30.729167+0000","last_fullsized":"2024-09-20T19:05:30.729167+0000","mapping_epoch":394,"log_start":"361'7162","ondisk_log_start":"361'7162","created":325,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":5760,"log_dups_size":0,"ondisk_log_size":5760,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":122683392,"num_objects":1872,"num_object_clones":0,"num_object_copies":5616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1872,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12922,"num_write_kb":473408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4647,"num_bytes_recovered":203751424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"273'27","reported_seq":823,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.508239+0000","last_change":"2024-09-20T19:02:39.709344+0000","last_active":"2024-09-20T19:05:30.508239+0000","last_peered":"2024-09-20T19:05:30.508239+0000","last_clean":"2024-09-20T19:05:30.508239+0000","last_became_active":"2024-09-20T18:59:05.083726+0000","last_became_peered":"2024-09-20T18:59:05.083726+0000","last_unstale":"2024-09-20T19:05:30.508239+0000","last_undegraded":"2024-09-20T19:05:30.508239+0000","last_fullsized":"2024-09-20T19:05:30.508239+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'27","last_scrub_stamp":"2024-09-20T19:02:39.709269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:39.709269+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":797,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.889485+0000","last_change":"2024-09-20T19:04:46.691255+0000","last_active":"2024-09-20T19:05:30.889485+0000","last_peered":"2024-09-20T19:05:30.889485+0000","last_clean":"2024-09-20T19:05:30.889485+0000","last_became_active":"2024-09-20T19:04:20.884854+0000","last_became_peered":"2024-09-20T19:04:20.884854+0000","last_unstale":"2024-09-20T19:05:30.889485+0000","last_undegraded":"2024-09-20T19:05:30.889485+0000","last_fullsized":"2024-09-20T19:05:30.889485+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-20T19:04:46.691171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:04:46.691171+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:02.158207+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"375'14","reported_seq":972,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.888725+0000","last_change":"2024-09-20T19:04:39.564069+0000","last_active":"2024-09-20T19:05:30.888725+0000","last_peered":"2024-09-20T19:05:30.888725+0000","last_clean":"2024-09-20T19:05:30.888725+0000","last_became_active":"2024-09-20T19:04:20.892956+0000","last_became_peered":"2024-09-20T19:04:20.892956+0000","last_unstale":"2024-09-20T19:05:30.888725+0000","last_undegraded":"2024-09-20T19:05:30.888725+0000","last_fullsized":"2024-09-20T19:05:30.888725+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"375'14","last_scrub_stamp":"2024-09-20T19:04:39.563917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:39.563917+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:49.606578+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":740,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.508718+0000","last_change":"2024-09-20T19:03:06.888620+0000","last_active":"2024-09-20T19:05:30.508718+0000","last_peered":"2024-09-20T19:05:30.508718+0000","last_clean":"2024-09-20T19:05:30.508718+0000","last_became_active":"2024-09-20T18:59:05.087005+0000","last_became_peered":"2024-09-20T18:59:05.087005+0000","last_unstale":"2024-09-20T19:05:30.508718+0000","last_undegraded":"2024-09-20T19:05:30.508718+0000","last_fullsized":"2024-09-20T19:05:30.508718+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:06.888563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:03:06.888563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":302,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.933572+0000","last_change":"2024-09-20T19:01:16.542398+0000","last_active":"2024-09-20T19:05:30.933572+0000","last_peered":"2024-09-20T19:05:30.933572+0000","last_clean":"2024-09-20T19:05:30.933572+0000","last_became_active":"2024-09-20T18:59:05.085427+0000","last_became_peered":"2024-09-20T18:59:05.085427+0000","last_unstale":"2024-09-20T19:05:30.933572+0000","last_undegraded":"2024-09-20T19:05:30.933572+0000","last_fullsized":"2024-09-20T19:05:30.933572+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:16.542335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:16.542335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:29.408351+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"273'12","reported_seq":751,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.511805+0000","last_change":"2024-09-20T19:04:48.570718+0000","last_active":"2024-09-20T19:05:30.511805+0000","last_peered":"2024-09-20T19:05:30.511805+0000","last_clean":"2024-09-20T19:05:30.511805+0000","last_became_active":"2024-09-20T18:49:48.500662+0000","last_became_peered":"2024-09-20T18:49:48.500662+0000","last_unstale":"2024-09-20T19:05:30.511805+0000","last_undegraded":"2024-09-20T19:05:30.511805+0000","last_fullsized":"2024-09-20T19:05:30.511805+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'12","last_scrub_stamp":"2024-09-20T19:04:48.570639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:04:48.570639+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:50.389558+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":729,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.511292+0000","last_change":"2024-09-20T19:05:22.605885+0000","last_active":"2024-09-20T19:05:30.511292+0000","last_peered":"2024-09-20T19:05:30.511292+0000","last_clean":"2024-09-20T19:05:30.511292+0000","last_became_active":"2024-09-20T18:49:49.956749+0000","last_became_peered":"2024-09-20T18:49:49.956749+0000","last_unstale":"2024-09-20T19:05:30.511292+0000","last_undegraded":"2024-09-20T19:05:30.511292+0000","last_fullsized":"2024-09-20T19:05:30.511292+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:22.605839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:05:22.605839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:49.241766+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":759,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.889122+0000","last_change":"2024-09-20T19:04:20.914426+0000","last_active":"2024-09-20T19:05:30.889122+0000","last_peered":"2024-09-20T19:05:30.889122+0000","last_clean":"2024-09-20T19:05:30.889122+0000","last_became_active":"2024-09-20T19:04:20.914115+0000","last_became_peered":"2024-09-20T19:04:20.914115+0000","last_unstale":"2024-09-20T19:05:30.889122+0000","last_undegraded":"2024-09-20T19:05:30.889122+0000","last_fullsized":"2024-09-20T19:05:30.889122+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:03.509903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:03.509903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"273'15","reported_seq":739,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-20T19:05:25.906137+0000","last_change":"2024-09-20T19:01:15.875779+0000","last_active":"2024-09-20T19:05:25.906137+0000","last_peered":"2024-09-20T19:05:25.906137+0000","last_clean":"2024-09-20T19:05:25.906137+0000","last_became_active":"2024-09-20T18:49:48.500803+0000","last_became_peered":"2024-09-20T18:49:48.500803+0000","last_unstale":"2024-09-20T19:05:25.906137+0000","last_undegraded":"2024-09-20T19:05:25.906137+0000","last_fullsized":"2024-09-20T19:05:25.906137+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:01:15.875693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:15.875693+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":771,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.888802+0000","last_change":"2024-09-20T19:04:47.713764+0000","last_active":"2024-09-20T19:05:30.888802+0000","last_peered":"2024-09-20T19:05:30.888802+0000","last_clean":"2024-09-20T19:05:30.888802+0000","last_became_active":"2024-09-20T19:04:20.893250+0000","last_became_peered":"2024-09-20T19:04:20.893250+0000","last_unstale":"2024-09-20T19:05:30.888802+0000","last_undegraded":"2024-09-20T19:05:30.888802+0000","last_fullsized":"2024-09-20T19:05:30.888802+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:47.713585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:04:47.713585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:49.334341+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":729,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.511680+0000","last_change":"2024-09-20T19:04:50.573619+0000","last_active":"2024-09-20T19:05:30.511680+0000","last_peered":"2024-09-20T19:05:30.511680+0000","last_clean":"2024-09-20T19:05:30.511680+0000","last_became_active":"2024-09-20T18:49:49.961947+0000","last_became_peered":"2024-09-20T18:49:49.961947+0000","last_unstale":"2024-09-20T19:05:30.511680+0000","last_undegraded":"2024-09-20T19:05:30.511680+0000","last_fullsized":"2024-09-20T19:05:30.511680+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:50.573550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:50.573550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:02.174525+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"66'1","reported_seq":772,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.920894+0000","last_change":"2024-09-20T19:04:20.913449+0000","last_active":"2024-09-20T19:05:30.920894+0000","last_peered":"2024-09-20T19:05:30.920894+0000","last_clean":"2024-09-20T19:05:30.920894+0000","last_became_active":"2024-09-20T19:04:20.892236+0000","last_became_peered":"2024-09-20T19:04:20.892236+0000","last_unstale":"2024-09-20T19:05:30.920894+0000","last_undegraded":"2024-09-20T19:05:30.920894+0000","last_fullsized":"2024-09-20T19:05:30.920894+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T19:03:12.499684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:03:12.499684+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"274'15","reported_seq":742,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.509537+0000","last_change":"2024-09-20T19:01:36.532486+0000","last_active":"2024-09-20T19:05:30.509537+0000","last_peered":"2024-09-20T19:05:30.509537+0000","last_clean":"2024-09-20T19:05:30.509537+0000","last_became_active":"2024-09-20T18:49:48.501011+0000","last_became_peered":"2024-09-20T18:49:48.501011+0000","last_unstale":"2024-09-20T19:05:30.509537+0000","last_undegraded":"2024-09-20T19:05:30.509537+0000","last_fullsized":"2024-09-20T19:05:30.509537+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'15","last_scrub_stamp":"2024-09-20T19:01:36.532419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:36.532419+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":726,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-20T19:05:25.905284+0000","last_change":"2024-09-20T19:01:30.864866+0000","last_active":"2024-09-20T19:05:25.905284+0000","last_peered":"2024-09-20T19:05:25.905284+0000","last_clean":"2024-09-20T19:05:25.905284+0000","last_became_active":"2024-09-20T18:49:45.938511+0000","last_became_peered":"2024-09-20T18:49:45.938511+0000","last_unstale":"2024-09-20T19:05:25.905284+0000","last_undegraded":"2024-09-20T19:05:25.905284+0000","last_fullsized":"2024-09-20T19:05:25.905284+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:30.864794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:30.864794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":926,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.508418+0000","last_change":"2024-09-20T19:04:41.601208+0000","last_active":"2024-09-20T19:05:30.508418+0000","last_peered":"2024-09-20T19:05:30.508418+0000","last_clean":"2024-09-20T19:05:30.508418+0000","last_became_active":"2024-09-20T18:49:49.960550+0000","last_became_peered":"2024-09-20T18:49:49.960550+0000","last_unstale":"2024-09-20T19:05:30.508418+0000","last_undegraded":"2024-09-20T19:05:30.508418+0000","last_fullsized":"2024-09-20T19:05:30.508418+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T19:04:41.601073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:41.601073+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:03.890101+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":62,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.499471+0000","last_change":"2024-09-20T19:04:20.884397+0000","last_active":"2024-09-20T19:05:30.499471+0000","last_peered":"2024-09-20T19:05:30.499471+0000","last_clean":"2024-09-20T19:05:30.499471+0000","last_became_active":"2024-09-20T19:04:20.884100+0000","last_became_peered":"2024-09-20T19:04:20.884100+0000","last_unstale":"2024-09-20T19:05:30.499471+0000","last_undegraded":"2024-09-20T19:05:30.499471+0000","last_fullsized":"2024-09-20T19:05:30.499471+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:59.497810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:59:59.497810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"274'20","reported_seq":758,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.508580+0000","last_change":"2024-09-20T18:57:20.701592+0000","last_active":"2024-09-20T19:05:30.508580+0000","last_peered":"2024-09-20T19:05:30.508580+0000","last_clean":"2024-09-20T19:05:30.508580+0000","last_became_active":"2024-09-20T18:49:58.997494+0000","last_became_peered":"2024-09-20T18:49:58.997494+0000","last_unstale":"2024-09-20T19:05:30.508580+0000","last_undegraded":"2024-09-20T19:05:30.508580+0000","last_fullsized":"2024-09-20T19:05:30.508580+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:57:20.701543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:20.701543+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:23.577077+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":749,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.889056+0000","last_change":"2024-09-20T19:04:20.884940+0000","last_active":"2024-09-20T19:05:30.889056+0000","last_peered":"2024-09-20T19:05:30.889056+0000","last_clean":"2024-09-20T19:05:30.889056+0000","last_became_active":"2024-09-20T19:04:20.884582+0000","last_became_peered":"2024-09-20T19:04:20.884582+0000","last_unstale":"2024-09-20T19:05:30.889056+0000","last_undegraded":"2024-09-20T19:05:30.889056+0000","last_fullsized":"2024-09-20T19:05:30.889056+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.632380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.632380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:32.500437+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":918,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.509351+0000","last_change":"2024-09-20T19:01:10.601374+0000","last_active":"2024-09-20T19:05:30.509351+0000","last_peered":"2024-09-20T19:05:30.509351+0000","last_clean":"2024-09-20T19:05:30.509351+0000","last_became_active":"2024-09-20T18:49:49.962243+0000","last_became_peered":"2024-09-20T18:49:49.962243+0000","last_unstale":"2024-09-20T19:05:30.509351+0000","last_undegraded":"2024-09-20T19:05:30.509351+0000","last_fullsized":"2024-09-20T19:05:30.509351+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T19:01:10.601300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:10.601300+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:23.615982+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":722,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-20T19:05:25.905666+0000","last_change":"2024-09-20T19:01:03.821977+0000","last_active":"2024-09-20T19:05:25.905666+0000","last_peered":"2024-09-20T19:05:25.905666+0000","last_clean":"2024-09-20T19:05:25.905666+0000","last_became_active":"2024-09-20T18:49:51.969093+0000","last_became_peered":"2024-09-20T18:49:51.969093+0000","last_unstale":"2024-09-20T19:05:25.905666+0000","last_undegraded":"2024-09-20T19:05:25.905666+0000","last_fullsized":"2024-09-20T19:05:25.905666+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:03.821908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:03.821908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"273'19","reported_seq":74,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.498337+0000","last_change":"2024-09-20T19:04:20.970137+0000","last_active":"2024-09-20T19:05:30.498337+0000","last_peered":"2024-09-20T19:05:30.498337+0000","last_clean":"2024-09-20T19:05:30.498337+0000","last_became_active":"2024-09-20T19:04:20.897330+0000","last_became_peered":"2024-09-20T19:04:20.897330+0000","last_unstale":"2024-09-20T19:05:30.498337+0000","last_undegraded":"2024-09-20T19:05:30.498337+0000","last_fullsized":"2024-09-20T19:05:30.498337+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:57:50.899514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:50.899514+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":62,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.499211+0000","last_change":"2024-09-20T19:04:20.896732+0000","last_active":"2024-09-20T19:05:30.499211+0000","last_peered":"2024-09-20T19:05:30.499211+0000","last_clean":"2024-09-20T19:05:30.499211+0000","last_became_active":"2024-09-20T19:04:20.896007+0000","last_became_peered":"2024-09-20T19:04:20.896007+0000","last_unstale":"2024-09-20T19:05:30.499211+0000","last_undegraded":"2024-09-20T19:05:30.499211+0000","last_fullsized":"2024-09-20T19:05:30.499211+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:06.310027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:59:06.310027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":723,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.916108+0000","last_change":"2024-09-20T19:02:38.906352+0000","last_active":"2024-09-20T19:05:30.916108+0000","last_peered":"2024-09-20T19:05:30.916108+0000","last_clean":"2024-09-20T19:05:30.916108+0000","last_became_active":"2024-09-20T18:49:49.957067+0000","last_became_peered":"2024-09-20T18:49:49.957067+0000","last_unstale":"2024-09-20T19:05:30.916108+0000","last_undegraded":"2024-09-20T19:05:30.916108+0000","last_fullsized":"2024-09-20T19:05:30.916108+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:38.906161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:38.906161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":701,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.505886+0000","last_change":"2024-09-20T18:57:16.789412+0000","last_active":"2024-09-20T19:05:30.505886+0000","last_peered":"2024-09-20T19:05:30.505886+0000","last_clean":"2024-09-20T19:05:30.505886+0000","last_became_active":"2024-09-20T18:49:51.966915+0000","last_became_peered":"2024-09-20T18:49:51.966915+0000","last_unstale":"2024-09-20T19:05:30.505886+0000","last_undegraded":"2024-09-20T19:05:30.505886+0000","last_fullsized":"2024-09-20T19:05:30.505886+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:16.789095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:16.789095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"273'15","reported_seq":794,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.916237+0000","last_change":"2024-09-20T19:02:30.816061+0000","last_active":"2024-09-20T19:05:30.916237+0000","last_peered":"2024-09-20T19:05:30.916237+0000","last_clean":"2024-09-20T19:05:30.916237+0000","last_became_active":"2024-09-20T18:59:05.085230+0000","last_became_peered":"2024-09-20T18:59:05.085230+0000","last_unstale":"2024-09-20T19:05:30.916237+0000","last_undegraded":"2024-09-20T19:05:30.916237+0000","last_fullsized":"2024-09-20T19:05:30.916237+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:02:30.815747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:30.815747+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":749,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.889316+0000","last_change":"2024-09-20T19:04:20.885127+0000","last_active":"2024-09-20T19:05:30.889316+0000","last_peered":"2024-09-20T19:05:30.889316+0000","last_clean":"2024-09-20T19:05:30.889316+0000","last_became_active":"2024-09-20T19:04:20.884660+0000","last_became_peered":"2024-09-20T19:04:20.884660+0000","last_unstale":"2024-09-20T19:05:30.889316+0000","last_undegraded":"2024-09-20T19:05:30.889316+0000","last_fullsized":"2024-09-20T19:05:30.889316+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:07.408197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:03:07.408197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":729,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.506310+0000","last_change":"2024-09-20T19:04:44.606497+0000","last_active":"2024-09-20T19:05:30.506310+0000","last_peered":"2024-09-20T19:05:30.506310+0000","last_clean":"2024-09-20T19:05:30.506310+0000","last_became_active":"2024-09-20T18:49:49.962523+0000","last_became_peered":"2024-09-20T18:49:49.962523+0000","last_unstale":"2024-09-20T19:05:30.506310+0000","last_undegraded":"2024-09-20T19:05:30.506310+0000","last_fullsized":"2024-09-20T19:05:30.506310+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:44.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:44.606343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:53.560145+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":714,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-20T19:05:25.904825+0000","last_change":"2024-09-20T19:01:11.847792+0000","last_active":"2024-09-20T19:05:25.904825+0000","last_peered":"2024-09-20T19:05:25.904825+0000","last_clean":"2024-09-20T19:05:25.904825+0000","last_became_active":"2024-09-20T18:49:51.967313+0000","last_became_peered":"2024-09-20T18:49:51.967313+0000","last_unstale":"2024-09-20T19:05:25.904825+0000","last_undegraded":"2024-09-20T19:05:25.904825+0000","last_fullsized":"2024-09-20T19:05:25.904825+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:11.847716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:11.847716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"274'19","reported_seq":794,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.506018+0000","last_change":"2024-09-20T19:01:05.495603+0000","last_active":"2024-09-20T19:05:30.506018+0000","last_peered":"2024-09-20T19:05:30.506018+0000","last_clean":"2024-09-20T19:05:30.506018+0000","last_became_active":"2024-09-20T18:59:05.075390+0000","last_became_peered":"2024-09-20T18:59:05.075390+0000","last_unstale":"2024-09-20T19:05:30.506018+0000","last_undegraded":"2024-09-20T19:05:30.506018+0000","last_fullsized":"2024-09-20T19:05:30.506018+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'19","last_scrub_stamp":"2024-09-20T19:01:05.495478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:05.495478+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:56.633007+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":737,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.511389+0000","last_change":"2024-09-20T19:01:47.521858+0000","last_active":"2024-09-20T19:05:30.511389+0000","last_peered":"2024-09-20T19:05:30.511389+0000","last_clean":"2024-09-20T19:05:30.511389+0000","last_became_active":"2024-09-20T18:49:45.942105+0000","last_became_peered":"2024-09-20T18:49:45.942105+0000","last_unstale":"2024-09-20T19:05:30.511389+0000","last_undegraded":"2024-09-20T19:05:30.511389+0000","last_fullsized":"2024-09-20T19:05:30.511389+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:47.521789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:47.521789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":763,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.888144+0000","last_change":"2024-09-20T19:04:20.912120+0000","last_active":"2024-09-20T19:05:30.888144+0000","last_peered":"2024-09-20T19:05:30.888144+0000","last_clean":"2024-09-20T19:05:30.888144+0000","last_became_active":"2024-09-20T19:04:20.911292+0000","last_became_peered":"2024-09-20T19:04:20.911292+0000","last_unstale":"2024-09-20T19:05:30.888144+0000","last_undegraded":"2024-09-20T19:05:30.888144+0000","last_fullsized":"2024-09-20T19:05:30.888144+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:27.267254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:27.267254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":723,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.506436+0000","last_change":"2024-09-20T19:02:38.950217+0000","last_active":"2024-09-20T19:05:30.506436+0000","last_peered":"2024-09-20T19:05:30.506436+0000","last_clean":"2024-09-20T19:05:30.506436+0000","last_became_active":"2024-09-20T18:59:05.081179+0000","last_became_peered":"2024-09-20T18:59:05.081179+0000","last_unstale":"2024-09-20T19:05:30.506436+0000","last_undegraded":"2024-09-20T19:05:30.506436+0000","last_fullsized":"2024-09-20T19:05:30.506436+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:38.950017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:38.950017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"273'8","reported_seq":771,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.505833+0000","last_change":"2024-09-20T19:04:40.559947+0000","last_active":"2024-09-20T19:05:30.505833+0000","last_peered":"2024-09-20T19:05:30.505833+0000","last_clean":"2024-09-20T19:05:30.505833+0000","last_became_active":"2024-09-20T18:59:05.076365+0000","last_became_peered":"2024-09-20T18:59:05.076365+0000","last_unstale":"2024-09-20T19:05:30.505833+0000","last_undegraded":"2024-09-20T19:05:30.505833+0000","last_fullsized":"2024-09-20T19:05:30.505833+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'8","last_scrub_stamp":"2024-09-20T19:04:40.559877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:04:40.559877+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:53.505030+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":734,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-20T19:05:25.906335+0000","last_change":"2024-09-20T19:02:39.184249+0000","last_active":"2024-09-20T19:05:25.906335+0000","last_peered":"2024-09-20T19:05:25.906335+0000","last_clean":"2024-09-20T19:05:25.906335+0000","last_became_active":"2024-09-20T18:49:45.940988+0000","last_became_peered":"2024-09-20T18:49:45.940988+0000","last_unstale":"2024-09-20T19:05:25.906335+0000","last_undegraded":"2024-09-20T19:05:25.906335+0000","last_fullsized":"2024-09-20T19:05:25.906335+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:39.184150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:39.184150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":327,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.933637+0000","last_change":"2024-09-20T19:04:20.900567+0000","last_active":"2024-09-20T19:05:30.933637+0000","last_peered":"2024-09-20T19:05:30.933637+0000","last_clean":"2024-09-20T19:05:30.933637+0000","last_became_active":"2024-09-20T19:04:20.900214+0000","last_became_peered":"2024-09-20T19:04:20.900214+0000","last_unstale":"2024-09-20T19:05:30.933637+0000","last_undegraded":"2024-09-20T19:05:30.933637+0000","last_fullsized":"2024-09-20T19:05:30.933637+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:41.497115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:41.497115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":737,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.888637+0000","last_change":"2024-09-20T19:04:20.886208+0000","last_active":"2024-09-20T19:05:30.888637+0000","last_peered":"2024-09-20T19:05:30.888637+0000","last_clean":"2024-09-20T19:05:30.888637+0000","last_became_active":"2024-09-20T19:04:20.885211+0000","last_became_peered":"2024-09-20T19:04:20.885211+0000","last_unstale":"2024-09-20T19:05:30.888637+0000","last_undegraded":"2024-09-20T19:05:30.888637+0000","last_fullsized":"2024-09-20T19:05:30.888637+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:29.277916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:29.277916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":720,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.915373+0000","last_change":"2024-09-20T19:04:20.890533+0000","last_active":"2024-09-20T19:05:30.915373+0000","last_peered":"2024-09-20T19:05:30.915373+0000","last_clean":"2024-09-20T19:05:30.915373+0000","last_became_active":"2024-09-20T19:04:20.890101+0000","last_became_peered":"2024-09-20T19:04:20.890101+0000","last_unstale":"2024-09-20T19:05:30.915373+0000","last_undegraded":"2024-09-20T19:05:30.915373+0000","last_fullsized":"2024-09-20T19:05:30.915373+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:10.970779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:10.970779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:24.205999+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":721,"reported_epoch":400,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.506070+0000","last_change":"2024-09-20T18:58:36.077005+0000","last_active":"2024-09-20T19:05:30.506070+0000","last_peered":"2024-09-20T19:05:30.506070+0000","last_clean":"2024-09-20T19:05:30.506070+0000","last_became_active":"2024-09-20T18:49:45.937992+0000","last_became_peered":"2024-09-20T18:49:45.937992+0000","last_unstale":"2024-09-20T19:05:30.506070+0000","last_undegraded":"2024-09-20T19:05:30.506070+0000","last_fullsized":"2024-09-20T19:05:30.506070+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:36.076937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:36.076937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:34.560802+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"274'19","reported_seq":327,"reported_epoch":401,"state":"active+clean","last_fresh":"2024-09-20T19:05:30.934736+0000","last_change":"2024-09-20T19:01:15.544760+0000","last_active":"2024-09-20T19:05:30.934736+0000","last_peered":"2024-09-20T19:05:30.934736+0000","last_clean":"2024-09-20T19:05:30.934736+0000","last_became_active":"2024-09-20T18:59:05.089294+0000","last_became_peered":"2024-09-20T18:59:05.089294+0000","last_unstale":"2024-09-20T19:05:30.934736+0000","last_undegraded":"2024-09-20T19:05:30.934736+0000","last_fullsized":"2024-09-20T19:05:30.934736+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'19","last_scrub_stamp":"2024-09-20T19:01:15.544663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:15.544663+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:24.387863+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":711,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-20T19:05:25.905829+0000","last_change":"2024-09-20T19:01:40.961449+0000","last_active":"2024-09-20T19:05:25.905829+0000","last_peered":"2024-09-20T19:05:25.905829+0000","last_clean":"2024-09-20T19:05:25.905829+0000","last_became_active":"2024-09-20T18:49:49.957582+0000","last_became_peered":"2024-09-20T18:49:49.957582+0000","last_unstale":"2024-09-20T19:05:25.905829+0000","last_undegraded":"2024-09-20T19:05:25.905829+0000","last_fullsized":"2024-09-20T19:05:25.905829+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:40.961363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:40.961363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":1958281240,"num_objects":29882,"num_object_clones":0,"num_object_copies":89646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":758,"num_objects_unfound":0,"num_objects_dirty":29882,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":203838,"num_write_kb":7478977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31723,"num_bytes_recovered":1768292376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6237138944,"data_stored":6237126728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92060,"ondisk_log_size":92060,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1185,"internal_metadata":0},"log_size":87,"ondisk_log_size":87,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1114,"num_read_kb":871,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":257,"num_bytes_recovered":4890,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":686,"ondisk_log_size":686,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":3105,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":938,"num_read_kb":933,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266881,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1048252,"kb_used_data":485356,"kb_used_omap":1,"kb_used_meta":562878,"kb_avail":92696900,"statfs":{"total":95995035648,"available":94921625600,"internally_reserved":0,"allocated":497004544,"data_stored":496553275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":576387530},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528520,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1229824,"kb_used_data":823720,"kb_used_omap":1,"kb_used_meta":406014,"kb_avail":92515328,"statfs":{"total":95995035648,"available":94735695872,"internally_reserved":0,"allocated":843489280,"data_stored":843048490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":415758790},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757453,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1187072,"kb_used_data":601960,"kb_used_omap":1,"kb_used_meta":585086,"kb_avail":92558080,"statfs":{"total":95995035648,"available":94779473920,"internally_reserved":0,"allocated":616407040,"data_stored":615959378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":599128517},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":304,"seq":1305670058035,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1194564,"kb_used_data":943472,"kb_used_omap":5,"kb_used_meta":251066,"kb_avail":92550588,"statfs":{"total":95995035648,"available":94771802112,"internally_reserved":0,"allocated":966115328,"data_stored":965653350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5633,"internal_metadata":257092095},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182617,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1415864,"kb_used_data":712740,"kb_used_omap":1,"kb_used_meta":703102,"kb_avail":92329288,"statfs":{"total":95995035648,"available":94545190912,"internally_reserved":0,"allocated":729845760,"data_stored":729401574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":719976903},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411550,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1306176,"kb_used_data":707176,"kb_used_omap":1,"kb_used_meta":598974,"kb_avail":92438976,"statfs":{"total":95995035648,"available":94657511424,"internally_reserved":0,"allocated":724148224,"data_stored":723707804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":613349829},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575076,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1171820,"kb_used_data":558936,"kb_used_omap":1,"kb_used_meta":612862,"kb_avail":92573332,"statfs":{"total":95995035648,"available":94795091968,"internally_reserved":0,"allocated":572350464,"data_stored":571862076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":627571141},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738601,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1980716,"kb_used_data":1317720,"kb_used_omap":1,"kb_used_meta":662846,"kb_avail":91764436,"statfs":{"total":95995035648,"available":93966782464,"internally_reserved":0,"allocated":1349345280,"data_stored":1348900115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":678754757},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1185,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1346899968,"data_stored":1346895896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":554958848,"data_stored":554958848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":722075648,"data_stored":722075648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":726859776,"data_stored":726859776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":958664704,"data_stored":958660632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":615120896,"data_stored":615120896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":825561088,"data_stored":825557016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":486998016,"data_stored":486998016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T19:05:33.805 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T19:05:33.880 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:33.923 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:33.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:33.925 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:34.012 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:33 smithi142 ceph-mon[32508]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:05:34.012 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:33 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T19:05:34.012 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:33 smithi142 ceph-mon[32508]: osdmap e402: 8 total, 8 up, 8 in 2024-09-20T19:05:34.012 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:33 smithi142 ceph-mon[32508]: osdmap e403: 8 total, 8 up, 8 in 2024-09-20T19:05:34.025 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:05:34.041 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T19:05:34.042 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-20T19:05:34.042 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:34.042 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.25775.0:206205 10.1 e403/402 10:82117eb8:::benchmark_data_smithi083_148100_object89343:head v 403'12711, mlcod=403'12708)", 2024-09-20T19:05:34.042 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:05:34.031455+0000", 2024-09-20T19:05:34.042 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0048658549999999997, 2024-09-20T19:05:34.042 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0048922949999999996, 2024-09-20T19:05:34.042 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:05:34.042 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:05:34.042 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T19:05:34.042 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:05:34.042 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:34.043 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:05:34.043 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:34.031455+0000", 2024-09-20T19:05:34.043 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:05:34.043 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:34.043 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:34.043 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:05:34.043 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:34.031455+0000", 2024-09-20T19:05:34.043 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:05:34.043 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:34.043 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:34.043 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:05:34.043 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:34.031456+0000", 2024-09-20T19:05:34.043 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.3099999999999996e-07 2024-09-20T19:05:34.043 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:34.044 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:34.044 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:05:34.044 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:34.031466+0000", 2024-09-20T19:05:34.044 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8479999999999996e-06 2024-09-20T19:05:34.044 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:34.044 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:34.044 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:05:34.044 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:34.031510+0000", 2024-09-20T19:05:34.044 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4804e-05 2024-09-20T19:05:34.044 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:34.044 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:34.044 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:05:34.044 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:34.031519+0000", 2024-09-20T19:05:34.044 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.259e-06 2024-09-20T19:05:34.045 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:34.045 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:34.045 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:34.045 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:34.031652+0000", 2024-09-20T19:05:34.045 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000133566 2024-09-20T19:05:34.045 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:34.045 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:34.045 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:05:34.045 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:34.031670+0000", 2024-09-20T19:05:34.045 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7399000000000001e-05 2024-09-20T19:05:34.046 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:34.046 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:05:34.046 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:34.046 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:34.046 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:34.046 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.25775.0:206206 10.1 e403/402)", 2024-09-20T19:05:34.046 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:05:34.031539+0000", 2024-09-20T19:05:34.046 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0047820709999999997, 2024-09-20T19:05:34.046 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0048574899999999999, 2024-09-20T19:05:34.046 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:05:34.046 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:05:34.046 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "queued for pg", 2024-09-20T19:05:34.046 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:05:34.046 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:34.047 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:05:34.047 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:34.031539+0000", 2024-09-20T19:05:34.047 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:05:34.047 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:34.047 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:34.047 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:05:34.047 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:34.031539+0000", 2024-09-20T19:05:34.047 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:05:34.047 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:34.047 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:34.047 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:05:34.047 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:34.031539+0000", 2024-09-20T19:05:34.047 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5600000000000001e-07 2024-09-20T19:05:34.047 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:34.048 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:34.048 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:05:34.048 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:34.031543+0000", 2024-09-20T19:05:34.048 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0949999999999998e-06 2024-09-20T19:05:34.048 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:34.048 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:34.048 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:05:34.048 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:34.031550+0000", 2024-09-20T19:05:34.048 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1829999999999999e-06 2024-09-20T19:05:34.048 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:34.048 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:34.048 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:05:34.048 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:34.031554+0000", 2024-09-20T19:05:34.048 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3359999999999997e-06 2024-09-20T19:05:34.049 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:34.049 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:05:34.049 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:34.049 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:34.049 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:34.049 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25775.0:206217 10.4 10:2548c970:::benchmark_data_smithi083_148100_object89355:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e403)", 2024-09-20T19:05:34.049 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:05:34.035201+0000", 2024-09-20T19:05:34.049 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.001120329, 2024-09-20T19:05:34.049 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0012392709999999999, 2024-09-20T19:05:34.050 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:05:34.050 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:05:34.050 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "waiting for sub ops", 2024-09-20T19:05:34.050 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:05:34.050 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25775", 2024-09-20T19:05:34.050 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:05:34.050 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 206217 2024-09-20T19:05:34.050 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:34.050 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:05:34.050 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:34.050 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:05:34.050 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:34.035201+0000", 2024-09-20T19:05:34.050 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:05:34.051 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:34.051 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:34.051 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:05:34.051 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:34.035201+0000", 2024-09-20T19:05:34.051 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:05:34.051 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:34.051 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:34.051 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:05:34.051 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:34.035203+0000", 2024-09-20T19:05:34.051 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.013e-06 2024-09-20T19:05:34.051 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:34.051 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:34.051 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:05:34.051 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:34.035208+0000", 2024-09-20T19:05:34.052 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8259999999999998e-06 2024-09-20T19:05:34.052 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:34.052 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:34.052 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:05:34.052 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:34.035225+0000", 2024-09-20T19:05:34.052 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6908000000000001e-05 2024-09-20T19:05:34.052 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:34.052 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:34.052 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:05:34.052 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:34.035234+0000", 2024-09-20T19:05:34.052 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1519999999999998e-06 2024-09-20T19:05:34.052 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:34.052 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:34.052 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:34.053 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:34.035321+0000", 2024-09-20T19:05:34.053 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6763999999999998e-05 2024-09-20T19:05:34.053 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:34.053 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:34.053 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:05:34.053 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:34.035438+0000", 2024-09-20T19:05:34.053 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000116666 2024-09-20T19:05:34.053 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:34.053 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:34.053 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3,4", 2024-09-20T19:05:34.053 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:34.035546+0000", 2024-09-20T19:05:34.053 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000108374 2024-09-20T19:05:34.053 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:34.053 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:05:34.053 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:34.054 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:34.054 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T19:05:34.054 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 3 2024-09-20T19:05:34.054 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T19:05:34.056 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:34.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:34.057 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:05:34.157 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:34.206 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:34.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:34.207 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:34.308 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:34.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:34.340 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:34.341 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:34.351 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T19:05:34.441 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:34.470 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:34.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:34.471 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:34.572 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:34.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:34.621 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:34.621 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:34.722 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:34.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:34.762 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:34.763 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:34.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:34 smithi083 ceph-mon[29421]: from='client.16296 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:05:34.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:34 smithi083 ceph-mon[29421]: pgmap v790: 148 pgs: 1 active+remapped+backfilling, 147 active+clean; 1.8 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 954 op/s; 758/88398 objects misplaced (0.857%); 10 MiB/s, 159 objects/s recovering 2024-09-20T19:05:34.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:34 smithi083 ceph-mon[36260]: from='client.16296 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:05:34.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:34 smithi083 ceph-mon[36260]: pgmap v790: 148 pgs: 1 active+remapped+backfilling, 147 active+clean; 1.8 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 954 op/s; 758/88398 objects misplaced (0.857%); 10 MiB/s, 159 objects/s recovering 2024-09-20T19:05:34.863 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:34.892 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:34.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:34.893 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:34.994 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:35.012 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:34 smithi142 ceph-mon[32508]: from='client.16296 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:05:35.012 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:34 smithi142 ceph-mon[32508]: pgmap v790: 148 pgs: 1 active+remapped+backfilling, 147 active+clean; 1.8 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 954 op/s; 758/88398 objects misplaced (0.857%); 10 MiB/s, 159 objects/s recovering 2024-09-20T19:05:35.033 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:35.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:35.033 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:35.134 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:35.166 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:35.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:35.167 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:35.268 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:35.301 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:35.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:35.302 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:35.402 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:35.433 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_blocked_ops 2024-09-20T19:05:35.444 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:05:35.451 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:35.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:35.452 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:35.459 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:05:35.553 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:35.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:35.597 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:35.597 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:35.698 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:35.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:35.768 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:35.768 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:35.868 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:35.909 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:35.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:35.910 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:36.010 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:36.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:36.056 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:36.056 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:36.157 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:36.191 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:36.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:36.192 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:36.293 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:36.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:36.327 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:36.327 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:36.428 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:05:36.467 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:36.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:36.469 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:05:36.569 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:36.599 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:36.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:36.600 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:36.614 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T19:05:36.701 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:36.745 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:36.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:36.746 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:36.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:36 smithi083 ceph-mon[29421]: pgmap v792: 148 pgs: 1 active+remapped+backfilling, 147 active+clean; 1.8 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 961 op/s; 758/88398 objects misplaced (0.857%); 10 MiB/s, 160 objects/s recovering 2024-09-20T19:05:36.792 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:36 smithi083 ceph-mon[36260]: pgmap v792: 148 pgs: 1 active+remapped+backfilling, 147 active+clean; 1.8 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 961 op/s; 758/88398 objects misplaced (0.857%); 10 MiB/s, 160 objects/s recovering 2024-09-20T19:05:36.847 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:36.882 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:36.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:36.883 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:36.984 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:37.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:36 smithi142 ceph-mon[32508]: pgmap v792: 148 pgs: 1 active+remapped+backfilling, 147 active+clean; 1.8 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 961 op/s; 758/88398 objects misplaced (0.857%); 10 MiB/s, 160 objects/s recovering 2024-09-20T19:05:37.014 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:37.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:37.015 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:37.116 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:37.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:37.150 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:37.150 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:37.250 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:05:37.281 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:37.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:37.282 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:05:37.383 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:37.417 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:37.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:37.418 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:37.518 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:37.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:37.549 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:37.550 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:37.650 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:37.684 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:37.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:37.685 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:37.762 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:05:37 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:05:37.785 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:37.811 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:37.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:37.812 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:37.913 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:37.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:37.950 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:37.950 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:38.051 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:38.080 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:05:38.080 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:05:38.086 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:38.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:38.087 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:38.115 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:05:38.115 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:05:38.157 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:05:38.157 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:05:38.188 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:38.193 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:05:38.193 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:05:38.219 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:38.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:38.220 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:38.243 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:05:38.243 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:05:38.290 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:05:38.290 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:05:38.293 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:05:38.304 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:05:38.320 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:38.334 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:05:38.335 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:05:38.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:38.375 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:38.376 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:38.384 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:05:38.384 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:05:38.477 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:38.513 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:38.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:38.514 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:38.615 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:38.636 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T19:05:38.640 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-20T19:05:38.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:38.663 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:38.663 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:38.764 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:38.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:38 smithi083 ceph-mon[29421]: pgmap v793: 148 pgs: 148 active+clean; 1.5 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1.25k op/s; 14 MiB/s, 232 objects/s recovering 2024-09-20T19:05:38.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:38 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:05:38.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:38 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:05:38.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:38 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:05:38.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:38 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/970770444' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:05:38.792 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:38 smithi083 ceph-mon[36260]: pgmap v793: 148 pgs: 148 active+clean; 1.5 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1.25k op/s; 14 MiB/s, 232 objects/s recovering 2024-09-20T19:05:38.792 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:38 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:05:38.792 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:38 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:05:38.792 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:38 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:05:38.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:38 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/970770444' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:05:38.798 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:38.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:38.799 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:38.900 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:38.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:38.934 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:38.934 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:39.012 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:38 smithi142 ceph-mon[32508]: pgmap v793: 148 pgs: 148 active+clean; 1.5 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1.25k op/s; 14 MiB/s, 232 objects/s recovering 2024-09-20T19:05:39.012 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:38 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:05:39.012 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:38 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:05:39.012 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:38 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:05:39.012 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:38 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/970770444' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:05:39.036 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:39.073 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:39.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:39.074 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:39.175 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:39.207 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:39.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:39.209 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:39.309 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:39.342 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:39.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:39.343 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:39.445 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:39.482 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:39.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:39.483 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:39.583 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:39.619 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:39.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:39.620 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:39.644 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T19:05:39.644 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:05:39.644 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:39 smithi083 ceph-mon[29421]: osdmap e404: 8 total, 8 up, 8 in 2024-09-20T19:05:39.644 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:39 smithi083 ceph-mon[29421]: from='client.26153 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:05:39.645 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:39 smithi083 ceph-mon[36260]: osdmap e404: 8 total, 8 up, 8 in 2024-09-20T19:05:39.645 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:39 smithi083 ceph-mon[36260]: from='client.26153 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:05:39.721 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:05:39.757 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:39.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:39.759 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:05:39.859 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:39.893 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:39.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:39.894 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:39.995 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:40.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:39 smithi142 ceph-mon[32508]: osdmap e404: 8 total, 8 up, 8 in 2024-09-20T19:05:40.012 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:39 smithi142 ceph-mon[32508]: from='client.26153 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:05:40.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:40.030 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:40.030 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:40.132 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:40.168 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:40.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:40.169 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:40.270 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:40.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:40.303 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:40.304 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:40.405 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:05:40.437 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:40.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:40.438 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:05:40.538 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:05:40.572 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:40.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:40.573 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:05:40.623 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_historic_ops 2024-09-20T19:05:40.674 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:40.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:40.735 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:40.736 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:40.836 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:40.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:40.877 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:40.877 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:40.958 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:05:40 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:05:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:05:40.959 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:40 smithi083 ceph-mon[36260]: pgmap v795: 148 pgs: 148 active+clean; 1.5 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 772 B/s rd, 0 B/s wr, 832 op/s; 6.7 MiB/s, 106 objects/s recovering 2024-09-20T19:05:40.960 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:40 smithi083 ceph-mon[29421]: pgmap v795: 148 pgs: 148 active+clean; 1.5 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 772 B/s rd, 0 B/s wr, 832 op/s; 6.7 MiB/s, 106 objects/s recovering 2024-09-20T19:05:40.978 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:05:41.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:41.010 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:41.011 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:05:41.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:40 smithi142 ceph-mon[32508]: pgmap v795: 148 pgs: 148 active+clean; 1.5 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 772 B/s rd, 0 B/s wr, 832 op/s; 6.7 MiB/s, 106 objects/s recovering 2024-09-20T19:05:41.111 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:41.148 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:41.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:41.149 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:41.249 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:41.284 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:41.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:41.285 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:41.386 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:41.408 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T19:05:41.425 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:41.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:41.426 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:41.463 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":794,"stamp":"2024-09-20T19:05:38.558400+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1597577779,"num_objects":24568,"num_object_clones":0,"num_object_copies":73704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24568,"num_whiteouts":0,"num_read":2201,"num_read_kb":2058,"num_write":210186,"num_write_kb":7480650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32723,"num_bytes_recovered":1816339955,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92325,"ondisk_log_size":92325,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":446,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9397584,"kb_used_data":4998984,"kb_used_omap":16,"kb_used_meta":4398127,"kb_avail":740563632,"statfs":{"total":767960285184,"available":758337159168,"internally_reserved":0,"allocated":5118959616,"data_stored":5115213065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16790,"internal_metadata":4503682666},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,5,6,5,9],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-552337408,"num_objects":-8428,"num_object_clones":0,"num_object_copies":-25284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-1548,"num_objects_unfound":0,"num_objects_dirty":-8428,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":8428,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1454,"num_bytes_recovered":95223808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.409714"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":737,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.525473+0000","last_change":"2024-09-20T19:01:31.817967+0000","last_active":"2024-09-20T19:05:33.525473+0000","last_peered":"2024-09-20T19:05:33.525473+0000","last_clean":"2024-09-20T19:05:33.525473+0000","last_became_active":"2024-09-20T18:49:45.943201+0000","last_became_peered":"2024-09-20T18:49:45.943201+0000","last_unstale":"2024-09-20T19:05:33.525473+0000","last_undegraded":"2024-09-20T19:05:33.525473+0000","last_fullsized":"2024-09-20T19:05:33.525473+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:31.817909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:31.817909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"273'15","reported_seq":78,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.526260+0000","last_change":"2024-09-20T19:04:21.040978+0000","last_active":"2024-09-20T19:05:33.526260+0000","last_peered":"2024-09-20T19:05:33.526260+0000","last_clean":"2024-09-20T19:05:33.526260+0000","last_became_active":"2024-09-20T19:04:20.925853+0000","last_became_peered":"2024-09-20T19:04:20.925853+0000","last_unstale":"2024-09-20T19:05:33.526260+0000","last_undegraded":"2024-09-20T19:05:33.526260+0000","last_fullsized":"2024-09-20T19:05:33.526260+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:03:10.214154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:10.214154+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":727,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.533777+0000","last_change":"2024-09-20T18:57:49.016078+0000","last_active":"2024-09-20T19:05:33.533777+0000","last_peered":"2024-09-20T19:05:33.533777+0000","last_clean":"2024-09-20T19:05:33.533777+0000","last_became_active":"2024-09-20T18:49:49.959514+0000","last_became_peered":"2024-09-20T18:49:49.959514+0000","last_unstale":"2024-09-20T19:05:33.533777+0000","last_undegraded":"2024-09-20T19:05:33.533777+0000","last_fullsized":"2024-09-20T19:05:33.533777+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:49.016006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:49.016006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:25.184023+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":68,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.526654+0000","last_change":"2024-09-20T19:04:20.881918+0000","last_active":"2024-09-20T19:05:33.526654+0000","last_peered":"2024-09-20T19:05:33.526654+0000","last_clean":"2024-09-20T19:05:33.526654+0000","last_became_active":"2024-09-20T19:04:20.880905+0000","last_became_peered":"2024-09-20T19:04:20.880905+0000","last_unstale":"2024-09-20T19:05:33.526654+0000","last_undegraded":"2024-09-20T19:05:33.526654+0000","last_fullsized":"2024-09-20T19:05:33.526654+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:02.084375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:02.084375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":737,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.592162+0000","last_change":"2024-09-20T19:02:02.441633+0000","last_active":"2024-09-20T19:05:33.592162+0000","last_peered":"2024-09-20T19:05:33.592162+0000","last_clean":"2024-09-20T19:05:33.592162+0000","last_became_active":"2024-09-20T18:59:05.081081+0000","last_became_peered":"2024-09-20T18:59:05.081081+0000","last_unstale":"2024-09-20T19:05:33.592162+0000","last_undegraded":"2024-09-20T19:05:33.592162+0000","last_fullsized":"2024-09-20T19:05:33.592162+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:02.441577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:02.441577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:58.610226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":750,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.592717+0000","last_change":"2024-09-20T19:04:37.540824+0000","last_active":"2024-09-20T19:05:33.592717+0000","last_peered":"2024-09-20T19:05:33.592717+0000","last_clean":"2024-09-20T19:05:33.592717+0000","last_became_active":"2024-09-20T18:59:05.072675+0000","last_became_peered":"2024-09-20T18:59:05.072675+0000","last_unstale":"2024-09-20T19:05:33.592717+0000","last_undegraded":"2024-09-20T19:05:33.592717+0000","last_fullsized":"2024-09-20T19:05:33.592717+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:37.540760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:04:37.540760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:55.234054+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"274'27","reported_seq":774,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.593212+0000","last_change":"2024-09-20T19:02:40.899215+0000","last_active":"2024-09-20T19:05:33.593212+0000","last_peered":"2024-09-20T19:05:33.593212+0000","last_clean":"2024-09-20T19:05:33.593212+0000","last_became_active":"2024-09-20T18:49:47.952150+0000","last_became_peered":"2024-09-20T18:49:47.952150+0000","last_unstale":"2024-09-20T19:05:33.593212+0000","last_undegraded":"2024-09-20T19:05:33.593212+0000","last_fullsized":"2024-09-20T19:05:33.593212+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'27","last_scrub_stamp":"2024-09-20T19:02:40.899139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:40.899139+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":68,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.526270+0000","last_change":"2024-09-20T19:04:20.926234+0000","last_active":"2024-09-20T19:05:33.526270+0000","last_peered":"2024-09-20T19:05:33.526270+0000","last_clean":"2024-09-20T19:05:33.526270+0000","last_became_active":"2024-09-20T19:04:20.925394+0000","last_became_peered":"2024-09-20T19:04:20.925394+0000","last_unstale":"2024-09-20T19:05:33.526270+0000","last_undegraded":"2024-09-20T19:05:33.526270+0000","last_fullsized":"2024-09-20T19:05:33.526270+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:23.982813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:23.982813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"59'1","reported_seq":812,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.588481+0000","last_change":"2024-09-20T19:04:20.963496+0000","last_active":"2024-09-20T19:05:33.588481+0000","last_peered":"2024-09-20T19:05:33.588481+0000","last_clean":"2024-09-20T19:05:33.588481+0000","last_became_active":"2024-09-20T19:04:20.914145+0000","last_became_peered":"2024-09-20T19:04:20.914145+0000","last_unstale":"2024-09-20T19:05:33.588481+0000","last_undegraded":"2024-09-20T19:05:33.588481+0000","last_fullsized":"2024-09-20T19:05:33.588481+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:01:28.592653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:28.592653+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:29.742562+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"273'15","reported_seq":78,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.526660+0000","last_change":"2024-09-20T19:04:20.984223+0000","last_active":"2024-09-20T19:05:33.526660+0000","last_peered":"2024-09-20T19:05:33.526660+0000","last_clean":"2024-09-20T19:05:33.526660+0000","last_became_active":"2024-09-20T19:04:20.898235+0000","last_became_peered":"2024-09-20T19:04:20.898235+0000","last_unstale":"2024-09-20T19:05:33.526660+0000","last_undegraded":"2024-09-20T19:05:33.526660+0000","last_fullsized":"2024-09-20T19:05:33.526660+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:02:42.114331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:42.114331+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":735,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.587603+0000","last_change":"2024-09-20T19:01:51.564693+0000","last_active":"2024-09-20T19:05:33.587603+0000","last_peered":"2024-09-20T19:05:33.587603+0000","last_clean":"2024-09-20T19:05:33.587603+0000","last_became_active":"2024-09-20T18:49:49.961188+0000","last_became_peered":"2024-09-20T18:49:49.961188+0000","last_unstale":"2024-09-20T19:05:33.587603+0000","last_undegraded":"2024-09-20T19:05:33.587603+0000","last_fullsized":"2024-09-20T19:05:33.587603+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:51.564624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:51.564624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":716,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.524140+0000","last_change":"2024-09-20T19:03:09.963421+0000","last_active":"2024-09-20T19:05:33.524140+0000","last_peered":"2024-09-20T19:05:33.524140+0000","last_clean":"2024-09-20T19:05:33.524140+0000","last_became_active":"2024-09-20T18:49:51.966715+0000","last_became_peered":"2024-09-20T18:49:51.966715+0000","last_unstale":"2024-09-20T19:05:33.524140+0000","last_undegraded":"2024-09-20T19:05:33.524140+0000","last_fullsized":"2024-09-20T19:05:33.524140+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:09.963298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:03:09.963298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":760,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.588861+0000","last_change":"2024-09-20T19:04:20.879227+0000","last_active":"2024-09-20T19:05:33.588861+0000","last_peered":"2024-09-20T19:05:33.588861+0000","last_clean":"2024-09-20T19:05:33.588861+0000","last_became_active":"2024-09-20T19:04:20.878905+0000","last_became_peered":"2024-09-20T19:04:20.878905+0000","last_unstale":"2024-09-20T19:05:33.588861+0000","last_undegraded":"2024-09-20T19:05:33.588861+0000","last_fullsized":"2024-09-20T19:05:33.588861+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:48.520017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:48.520017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"273'7","reported_seq":74,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.526823+0000","last_change":"2024-09-20T19:04:20.907713+0000","last_active":"2024-09-20T19:05:33.526823+0000","last_peered":"2024-09-20T19:05:33.526823+0000","last_clean":"2024-09-20T19:05:33.526823+0000","last_became_active":"2024-09-20T19:04:20.888572+0000","last_became_peered":"2024-09-20T19:04:20.888572+0000","last_unstale":"2024-09-20T19:05:33.526823+0000","last_undegraded":"2024-09-20T19:05:33.526823+0000","last_fullsized":"2024-09-20T19:05:33.526823+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-20T18:58:33.660531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:33.660531+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":753,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.529021+0000","last_change":"2024-09-20T19:04:20.885873+0000","last_active":"2024-09-20T19:05:33.529021+0000","last_peered":"2024-09-20T19:05:33.529021+0000","last_clean":"2024-09-20T19:05:33.529021+0000","last_became_active":"2024-09-20T19:04:20.885417+0000","last_became_peered":"2024-09-20T19:04:20.885417+0000","last_unstale":"2024-09-20T19:05:33.529021+0000","last_undegraded":"2024-09-20T19:05:33.529021+0000","last_fullsized":"2024-09-20T19:05:33.529021+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:33.203312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:33.203312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":731,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.587257+0000","last_change":"2024-09-20T19:05:21.625378+0000","last_active":"2024-09-20T19:05:33.587257+0000","last_peered":"2024-09-20T19:05:33.587257+0000","last_clean":"2024-09-20T19:05:33.587257+0000","last_became_active":"2024-09-20T18:49:51.970663+0000","last_became_peered":"2024-09-20T18:49:51.970663+0000","last_unstale":"2024-09-20T19:05:33.587257+0000","last_undegraded":"2024-09-20T19:05:33.587257+0000","last_fullsized":"2024-09-20T19:05:33.587257+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:21.625311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:05:21.625311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:25.835298+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":68,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.527364+0000","last_change":"2024-09-20T19:04:20.907531+0000","last_active":"2024-09-20T19:05:33.527364+0000","last_peered":"2024-09-20T19:05:33.527364+0000","last_clean":"2024-09-20T19:05:33.527364+0000","last_became_active":"2024-09-20T19:04:20.906109+0000","last_became_peered":"2024-09-20T19:04:20.906109+0000","last_unstale":"2024-09-20T19:05:33.527364+0000","last_undegraded":"2024-09-20T19:05:33.527364+0000","last_fullsized":"2024-09-20T19:05:33.527364+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:00.906652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:00.906652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":761,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.524936+0000","last_change":"2024-09-20T19:04:20.907057+0000","last_active":"2024-09-20T19:05:33.524936+0000","last_peered":"2024-09-20T19:05:33.524936+0000","last_clean":"2024-09-20T19:05:33.524936+0000","last_became_active":"2024-09-20T19:04:20.906691+0000","last_became_peered":"2024-09-20T19:04:20.906691+0000","last_unstale":"2024-09-20T19:05:33.524936+0000","last_undegraded":"2024-09-20T19:05:33.524936+0000","last_fullsized":"2024-09-20T19:05:33.524936+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:44.998147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:44.998147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"273'27","reported_seq":785,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.524883+0000","last_change":"2024-09-20T19:03:09.456720+0000","last_active":"2024-09-20T19:05:33.524883+0000","last_peered":"2024-09-20T19:05:33.524883+0000","last_clean":"2024-09-20T19:05:33.524883+0000","last_became_active":"2024-09-20T18:49:47.950939+0000","last_became_peered":"2024-09-20T18:49:47.950939+0000","last_unstale":"2024-09-20T19:05:33.524883+0000","last_undegraded":"2024-09-20T19:05:33.524883+0000","last_fullsized":"2024-09-20T19:05:33.524883+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'27","last_scrub_stamp":"2024-09-20T19:03:09.456631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:09.456631+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":744,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.529445+0000","last_change":"2024-09-20T19:04:20.890659+0000","last_active":"2024-09-20T19:05:33.529445+0000","last_peered":"2024-09-20T19:05:33.529445+0000","last_clean":"2024-09-20T19:05:33.529445+0000","last_became_active":"2024-09-20T19:04:20.890370+0000","last_became_peered":"2024-09-20T19:04:20.890370+0000","last_unstale":"2024-09-20T19:05:33.529445+0000","last_undegraded":"2024-09-20T19:05:33.529445+0000","last_fullsized":"2024-09-20T19:05:33.529445+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:05.888974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:05.888974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:26.204424+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":738,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.592682+0000","last_change":"2024-09-20T19:02:28.934597+0000","last_active":"2024-09-20T19:05:33.592682+0000","last_peered":"2024-09-20T19:05:33.592682+0000","last_clean":"2024-09-20T19:05:33.592682+0000","last_became_active":"2024-09-20T18:59:05.084496+0000","last_became_peered":"2024-09-20T18:59:05.084496+0000","last_unstale":"2024-09-20T19:05:33.592682+0000","last_undegraded":"2024-09-20T19:05:33.592682+0000","last_fullsized":"2024-09-20T19:05:33.592682+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:28.934264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:28.934264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:05.678324+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"59'1","reported_seq":72,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.526535+0000","last_change":"2024-09-20T19:04:20.905233+0000","last_active":"2024-09-20T19:05:33.526535+0000","last_peered":"2024-09-20T19:05:33.526535+0000","last_clean":"2024-09-20T19:05:33.526535+0000","last_became_active":"2024-09-20T19:04:20.887062+0000","last_became_peered":"2024-09-20T19:04:20.887062+0000","last_unstale":"2024-09-20T19:05:33.526535+0000","last_undegraded":"2024-09-20T19:05:33.526535+0000","last_fullsized":"2024-09-20T19:05:33.526535+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:03:08.218812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:03:08.218812+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":993,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"274'18","reported_seq":788,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.592365+0000","last_change":"2024-09-20T19:04:20.925690+0000","last_active":"2024-09-20T19:05:33.592365+0000","last_peered":"2024-09-20T19:05:33.592365+0000","last_clean":"2024-09-20T19:05:33.592365+0000","last_became_active":"2024-09-20T19:04:20.896343+0000","last_became_peered":"2024-09-20T19:04:20.896343+0000","last_unstale":"2024-09-20T19:05:33.592365+0000","last_undegraded":"2024-09-20T19:05:33.592365+0000","last_fullsized":"2024-09-20T19:05:33.592365+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:57:09.818999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:09.818999+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:29.636168+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":68,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.528092+0000","last_change":"2024-09-20T19:04:20.889786+0000","last_active":"2024-09-20T19:05:33.528092+0000","last_peered":"2024-09-20T19:05:33.528092+0000","last_clean":"2024-09-20T19:05:33.528092+0000","last_became_active":"2024-09-20T19:04:20.889430+0000","last_became_peered":"2024-09-20T19:04:20.889430+0000","last_unstale":"2024-09-20T19:05:33.528092+0000","last_undegraded":"2024-09-20T19:05:33.528092+0000","last_fullsized":"2024-09-20T19:05:33.528092+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:02.094473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:02.094473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":725,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.529758+0000","last_change":"2024-09-20T19:02:28.294390+0000","last_active":"2024-09-20T19:05:33.529758+0000","last_peered":"2024-09-20T19:05:33.529758+0000","last_clean":"2024-09-20T19:05:33.529758+0000","last_became_active":"2024-09-20T18:49:51.970058+0000","last_became_peered":"2024-09-20T18:49:51.970058+0000","last_unstale":"2024-09-20T19:05:33.529758+0000","last_undegraded":"2024-09-20T19:05:33.529758+0000","last_fullsized":"2024-09-20T19:05:33.529758+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T19:02:28.294292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:28.294292+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":761,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.533890+0000","last_change":"2024-09-20T19:05:22.202134+0000","last_active":"2024-09-20T19:05:33.533890+0000","last_peered":"2024-09-20T19:05:33.533890+0000","last_clean":"2024-09-20T19:05:33.533890+0000","last_became_active":"2024-09-20T19:04:20.881466+0000","last_became_peered":"2024-09-20T19:04:20.881466+0000","last_unstale":"2024-09-20T19:05:33.533890+0000","last_undegraded":"2024-09-20T19:05:33.533890+0000","last_fullsized":"2024-09-20T19:05:33.533890+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:05:22.201836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:22.201836+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:45.062354+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"274'20","reported_seq":798,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.525645+0000","last_change":"2024-09-20T19:03:16.982915+0000","last_active":"2024-09-20T19:05:33.525645+0000","last_peered":"2024-09-20T19:05:33.525645+0000","last_clean":"2024-09-20T19:05:33.525645+0000","last_became_active":"2024-09-20T19:03:16.968636+0000","last_became_peered":"2024-09-20T19:03:16.968636+0000","last_unstale":"2024-09-20T19:05:33.525645+0000","last_undegraded":"2024-09-20T19:05:33.525645+0000","last_fullsized":"2024-09-20T19:05:33.525645+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'20","last_scrub_stamp":"2024-09-20T19:02:35.096979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:35.096979+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"375'14","reported_seq":433,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.582339+0000","last_change":"2024-09-20T19:04:20.956210+0000","last_active":"2024-09-20T19:05:33.582339+0000","last_peered":"2024-09-20T19:05:33.582339+0000","last_clean":"2024-09-20T19:05:33.582339+0000","last_became_active":"2024-09-20T19:04:20.900782+0000","last_became_peered":"2024-09-20T19:04:20.900782+0000","last_unstale":"2024-09-20T19:05:33.582339+0000","last_undegraded":"2024-09-20T19:05:33.582339+0000","last_fullsized":"2024-09-20T19:05:33.582339+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"308'12","last_scrub_stamp":"2024-09-20T19:01:44.541156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:44.541156+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:35.235276+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"403'12856","reported_seq":30493,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:35.679930+0000","last_change":"2024-09-20T19:05:33.617599+0000","last_active":"2024-09-20T19:05:35.679930+0000","last_peered":"2024-09-20T19:05:35.679930+0000","last_clean":"2024-09-20T19:05:35.679930+0000","last_became_active":"2024-09-20T19:05:33.530382+0000","last_became_peered":"2024-09-20T19:05:33.530382+0000","last_unstale":"2024-09-20T19:05:35.679930+0000","last_undegraded":"2024-09-20T19:05:35.679930+0000","last_fullsized":"2024-09-20T19:05:35.679930+0000","mapping_epoch":402,"log_start":"370'8216","ondisk_log_start":"370'8216","created":325,"last_epoch_clean":403,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4640,"log_dups_size":0,"ondisk_log_size":4640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":105512960,"num_objects":1610,"num_object_clones":0,"num_object_copies":4830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1610,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12856,"num_write_kb":462912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8715,"num_bytes_recovered":571080704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"274'27","reported_seq":817,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.525239+0000","last_change":"2024-09-20T19:04:20.920811+0000","last_active":"2024-09-20T19:05:33.525239+0000","last_peered":"2024-09-20T19:05:33.525239+0000","last_clean":"2024-09-20T19:05:33.525239+0000","last_became_active":"2024-09-20T19:04:20.897049+0000","last_became_peered":"2024-09-20T19:04:20.897049+0000","last_unstale":"2024-09-20T19:05:33.525239+0000","last_undegraded":"2024-09-20T19:05:33.525239+0000","last_fullsized":"2024-09-20T19:05:33.525239+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'27","last_scrub_stamp":"2024-09-20T19:02:27.897138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:27.897138+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":736,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.592130+0000","last_change":"2024-09-20T19:04:39.537914+0000","last_active":"2024-09-20T19:05:33.592130+0000","last_peered":"2024-09-20T19:05:33.592130+0000","last_clean":"2024-09-20T19:05:33.592130+0000","last_became_active":"2024-09-20T18:49:45.936104+0000","last_became_peered":"2024-09-20T18:49:45.936104+0000","last_unstale":"2024-09-20T19:05:33.592130+0000","last_undegraded":"2024-09-20T19:05:33.592130+0000","last_fullsized":"2024-09-20T19:05:33.592130+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:39.537818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:04:39.537818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:44.147654+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"375'12","reported_seq":87,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.527562+0000","last_change":"2024-09-20T19:04:21.030980+0000","last_active":"2024-09-20T19:05:33.527562+0000","last_peered":"2024-09-20T19:05:33.527562+0000","last_clean":"2024-09-20T19:05:33.527562+0000","last_became_active":"2024-09-20T19:04:20.907228+0000","last_became_peered":"2024-09-20T19:04:20.907228+0000","last_unstale":"2024-09-20T19:05:33.527562+0000","last_undegraded":"2024-09-20T19:05:33.527562+0000","last_fullsized":"2024-09-20T19:05:33.527562+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"308'10","last_scrub_stamp":"2024-09-20T19:02:23.017626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:23.017626+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":724,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.588109+0000","last_change":"2024-09-20T19:02:22.736879+0000","last_active":"2024-09-20T19:05:33.588109+0000","last_peered":"2024-09-20T19:05:33.588109+0000","last_clean":"2024-09-20T19:05:33.588109+0000","last_became_active":"2024-09-20T18:49:51.971399+0000","last_became_peered":"2024-09-20T18:49:51.971399+0000","last_unstale":"2024-09-20T19:05:33.588109+0000","last_undegraded":"2024-09-20T19:05:33.588109+0000","last_fullsized":"2024-09-20T19:05:33.588109+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:22.736825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:22.736825+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.e","version":"403'13190","reported_seq":14280,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:35.705090+0000","last_change":"2024-09-20T19:04:21.970634+0000","last_active":"2024-09-20T19:05:35.705090+0000","last_peered":"2024-09-20T19:05:35.705090+0000","last_clean":"2024-09-20T19:05:35.705090+0000","last_became_active":"2024-09-20T19:04:21.956145+0000","last_became_peered":"2024-09-20T19:04:21.956145+0000","last_unstale":"2024-09-20T19:05:35.705090+0000","last_undegraded":"2024-09-20T19:05:35.705090+0000","last_fullsized":"2024-09-20T19:05:35.705090+0000","mapping_epoch":375,"log_start":"372'8628","ondisk_log_start":"372'8628","created":325,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4562,"log_dups_size":0,"ondisk_log_size":4562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":105250816,"num_objects":1606,"num_object_clones":0,"num_object_copies":4818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13190,"num_write_kb":473472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":453,"num_bytes_recovered":29687808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"273'30","reported_seq":790,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.591929+0000","last_change":"2024-09-20T19:03:06.038303+0000","last_active":"2024-09-20T19:05:33.591929+0000","last_peered":"2024-09-20T19:05:33.591929+0000","last_clean":"2024-09-20T19:05:33.591929+0000","last_became_active":"2024-09-20T18:49:48.499189+0000","last_became_peered":"2024-09-20T18:49:48.499189+0000","last_unstale":"2024-09-20T19:05:33.591929+0000","last_undegraded":"2024-09-20T19:05:33.591929+0000","last_fullsized":"2024-09-20T19:05:33.591929+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'30","last_scrub_stamp":"2024-09-20T19:03:06.038213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:06.038213+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":728,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.593058+0000","last_change":"2024-09-20T19:02:22.806587+0000","last_active":"2024-09-20T19:05:33.593058+0000","last_peered":"2024-09-20T19:05:33.593058+0000","last_clean":"2024-09-20T19:05:33.593058+0000","last_became_active":"2024-09-20T18:49:45.940544+0000","last_became_peered":"2024-09-20T18:49:45.940544+0000","last_unstale":"2024-09-20T19:05:33.593058+0000","last_undegraded":"2024-09-20T19:05:33.593058+0000","last_fullsized":"2024-09-20T19:05:33.593058+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:22.806509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:22.806509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:02.674388+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":68,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.527539+0000","last_change":"2024-09-20T19:04:20.889280+0000","last_active":"2024-09-20T19:05:33.527539+0000","last_peered":"2024-09-20T19:05:33.527539+0000","last_clean":"2024-09-20T19:05:33.527539+0000","last_became_active":"2024-09-20T19:04:20.888422+0000","last_became_peered":"2024-09-20T19:04:20.888422+0000","last_unstale":"2024-09-20T19:05:33.527539+0000","last_undegraded":"2024-09-20T19:05:33.527539+0000","last_fullsized":"2024-09-20T19:05:33.527539+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.229201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.229201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:33.231712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":68,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.526782+0000","last_change":"2024-09-20T19:04:20.890045+0000","last_active":"2024-09-20T19:05:33.526782+0000","last_peered":"2024-09-20T19:05:33.526782+0000","last_clean":"2024-09-20T19:05:33.526782+0000","last_became_active":"2024-09-20T19:04:20.889418+0000","last_became_peered":"2024-09-20T19:04:20.889418+0000","last_unstale":"2024-09-20T19:05:33.526782+0000","last_undegraded":"2024-09-20T19:05:33.526782+0000","last_fullsized":"2024-09-20T19:05:33.526782+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:27.704520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:27.704520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"403'12913","reported_seq":13082,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:36.896901+0000","last_change":"2024-09-20T19:02:11.828279+0000","last_active":"2024-09-20T19:05:36.896901+0000","last_peered":"2024-09-20T19:05:36.896901+0000","last_clean":"2024-09-20T19:05:36.896901+0000","last_became_active":"2024-09-20T19:02:11.827468+0000","last_became_peered":"2024-09-20T19:02:11.827468+0000","last_unstale":"2024-09-20T19:05:36.896901+0000","last_undegraded":"2024-09-20T19:05:36.896901+0000","last_fullsized":"2024-09-20T19:05:36.896901+0000","mapping_epoch":325,"log_start":"360'6855","ondisk_log_start":"360'6855","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":6058,"log_dups_size":0,"ondisk_log_size":6058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96796672,"num_objects":1477,"num_object_clones":0,"num_object_copies":4431,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12913,"num_write_kb":460480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"274'28","reported_seq":84,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.527508+0000","last_change":"2024-09-20T19:04:20.976682+0000","last_active":"2024-09-20T19:05:33.527508+0000","last_peered":"2024-09-20T19:05:33.527508+0000","last_clean":"2024-09-20T19:05:33.527508+0000","last_became_active":"2024-09-20T19:04:20.897444+0000","last_became_peered":"2024-09-20T19:04:20.897444+0000","last_unstale":"2024-09-20T19:05:33.527508+0000","last_undegraded":"2024-09-20T19:05:33.527508+0000","last_fullsized":"2024-09-20T19:05:33.527508+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'28","last_scrub_stamp":"2024-09-20T19:03:06.269623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:06.269623+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":41,"num_read_kb":34,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":763,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.525912+0000","last_change":"2024-09-20T19:04:21.031015+0000","last_active":"2024-09-20T19:05:33.525912+0000","last_peered":"2024-09-20T19:05:33.525912+0000","last_clean":"2024-09-20T19:05:33.525912+0000","last_became_active":"2024-09-20T19:04:20.906790+0000","last_became_peered":"2024-09-20T19:04:20.906790+0000","last_unstale":"2024-09-20T19:05:33.525912+0000","last_undegraded":"2024-09-20T19:05:33.525912+0000","last_fullsized":"2024-09-20T19:05:33.525912+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:00:29.890681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:00:29.890681+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"375'13","reported_seq":973,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.591775+0000","last_change":"2024-09-20T19:04:20.916872+0000","last_active":"2024-09-20T19:05:33.591775+0000","last_peered":"2024-09-20T19:05:33.591775+0000","last_clean":"2024-09-20T19:05:33.591775+0000","last_became_active":"2024-09-20T19:04:20.896424+0000","last_became_peered":"2024-09-20T19:04:20.896424+0000","last_unstale":"2024-09-20T19:05:33.591775+0000","last_undegraded":"2024-09-20T19:05:33.591775+0000","last_fullsized":"2024-09-20T19:05:33.591775+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:58:23.114371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:23.114371+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":731,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.529065+0000","last_change":"2024-09-20T18:59:05.082117+0000","last_active":"2024-09-20T19:05:33.529065+0000","last_peered":"2024-09-20T19:05:33.529065+0000","last_clean":"2024-09-20T19:05:33.529065+0000","last_became_active":"2024-09-20T18:59:05.081814+0000","last_became_peered":"2024-09-20T18:59:05.081814+0000","last_unstale":"2024-09-20T19:05:33.529065+0000","last_undegraded":"2024-09-20T19:05:33.529065+0000","last_fullsized":"2024-09-20T19:05:33.529065+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:29.530001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:29.530001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:34.493661+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"403'12926","reported_seq":30728,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:36.333688+0000","last_change":"2024-09-20T19:05:30.912897+0000","last_active":"2024-09-20T19:05:36.333688+0000","last_peered":"2024-09-20T19:05:36.333688+0000","last_clean":"2024-09-20T19:05:36.333688+0000","last_became_active":"2024-09-20T19:05:30.889717+0000","last_became_peered":"2024-09-20T19:05:30.889717+0000","last_unstale":"2024-09-20T19:05:36.333688+0000","last_undegraded":"2024-09-20T19:05:36.333688+0000","last_fullsized":"2024-09-20T19:05:36.333688+0000","mapping_epoch":400,"log_start":"367'7755","ondisk_log_start":"367'7755","created":325,"last_epoch_clean":401,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":5171,"log_dups_size":0,"ondisk_log_size":5171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":99876864,"num_objects":1524,"num_object_clones":0,"num_object_copies":4572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12926,"num_write_kb":462400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8799,"num_bytes_recovered":576651264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"274'20","reported_seq":574,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.587492+0000","last_change":"2024-09-20T19:05:20.497269+0000","last_active":"2024-09-20T19:05:33.587492+0000","last_peered":"2024-09-20T19:05:33.587492+0000","last_clean":"2024-09-20T19:05:33.587492+0000","last_became_active":"2024-09-20T19:04:49.419140+0000","last_became_peered":"2024-09-20T19:04:49.419140+0000","last_unstale":"2024-09-20T19:05:33.587492+0000","last_undegraded":"2024-09-20T19:05:33.587492+0000","last_fullsized":"2024-09-20T19:05:33.587492+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'20","last_scrub_stamp":"2024-09-20T19:02:35.930910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:35.930910+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":752,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.588561+0000","last_change":"2024-09-20T19:02:30.686939+0000","last_active":"2024-09-20T19:05:33.588561+0000","last_peered":"2024-09-20T19:05:33.588561+0000","last_clean":"2024-09-20T19:05:33.588561+0000","last_became_active":"2024-09-20T18:49:45.942496+0000","last_became_peered":"2024-09-20T18:49:45.942496+0000","last_unstale":"2024-09-20T19:05:33.588561+0000","last_undegraded":"2024-09-20T19:05:33.588561+0000","last_fullsized":"2024-09-20T19:05:33.588561+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:30.686872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:30.686872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"262'10","reported_seq":950,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.524510+0000","last_change":"2024-09-20T19:02:04.994109+0000","last_active":"2024-09-20T19:05:33.524510+0000","last_peered":"2024-09-20T19:05:33.524510+0000","last_clean":"2024-09-20T19:05:33.524510+0000","last_became_active":"2024-09-20T18:59:05.080183+0000","last_became_peered":"2024-09-20T18:59:05.080183+0000","last_unstale":"2024-09-20T19:05:33.524510+0000","last_undegraded":"2024-09-20T19:05:33.524510+0000","last_fullsized":"2024-09-20T19:05:33.524510+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'10","last_scrub_stamp":"2024-09-20T19:02:04.994047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:04.994047+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":709,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.524343+0000","last_change":"2024-09-20T19:01:37.900410+0000","last_active":"2024-09-20T19:05:33.524343+0000","last_peered":"2024-09-20T19:05:33.524343+0000","last_clean":"2024-09-20T19:05:33.524343+0000","last_became_active":"2024-09-20T18:49:51.964780+0000","last_became_peered":"2024-09-20T18:49:51.964780+0000","last_unstale":"2024-09-20T19:05:33.524343+0000","last_undegraded":"2024-09-20T19:05:33.524343+0000","last_fullsized":"2024-09-20T19:05:33.524343+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:37.900328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:37.900328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.d","version":"403'13464","reported_seq":22723,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:36.917992+0000","last_change":"2024-09-20T19:05:23.028287+0000","last_active":"2024-09-20T19:05:36.917992+0000","last_peered":"2024-09-20T19:05:36.917992+0000","last_clean":"2024-09-20T19:05:36.917992+0000","last_became_active":"2024-09-20T19:05:22.993748+0000","last_became_peered":"2024-09-20T19:05:22.993748+0000","last_unstale":"2024-09-20T19:05:36.917992+0000","last_undegraded":"2024-09-20T19:05:36.917992+0000","last_fullsized":"2024-09-20T19:05:36.917992+0000","mapping_epoch":395,"log_start":"362'7417","ondisk_log_start":"362'7417","created":325,"last_epoch_clean":396,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":6047,"log_dups_size":0,"ondisk_log_size":6047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":97189912,"num_objects":1484,"num_object_clones":0,"num_object_copies":4452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1484,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13464,"num_write_kb":478273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4538,"num_bytes_recovered":189071384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"274'29","reported_seq":776,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.524652+0000","last_change":"2024-09-20T19:00:04.810907+0000","last_active":"2024-09-20T19:05:33.524652+0000","last_peered":"2024-09-20T19:05:33.524652+0000","last_clean":"2024-09-20T19:05:33.524652+0000","last_became_active":"2024-09-20T18:49:48.497736+0000","last_became_peered":"2024-09-20T18:49:48.497736+0000","last_unstale":"2024-09-20T19:05:33.524652+0000","last_undegraded":"2024-09-20T19:05:33.524652+0000","last_fullsized":"2024-09-20T19:05:33.524652+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'29","last_scrub_stamp":"2024-09-20T19:00:04.810831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:00:04.810831+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":728,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.529318+0000","last_change":"2024-09-20T19:02:41.859231+0000","last_active":"2024-09-20T19:05:33.529318+0000","last_peered":"2024-09-20T19:05:33.529318+0000","last_clean":"2024-09-20T19:05:33.529318+0000","last_became_active":"2024-09-20T18:49:45.942253+0000","last_became_peered":"2024-09-20T18:49:45.942253+0000","last_unstale":"2024-09-20T19:05:33.529318+0000","last_undegraded":"2024-09-20T19:05:33.529318+0000","last_fullsized":"2024-09-20T19:05:33.529318+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:41.859168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:41.859168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":315,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.582013+0000","last_change":"2024-09-20T19:02:27.639446+0000","last_active":"2024-09-20T19:05:33.582013+0000","last_peered":"2024-09-20T19:05:33.582013+0000","last_clean":"2024-09-20T19:05:33.582013+0000","last_became_active":"2024-09-20T18:59:05.085636+0000","last_became_peered":"2024-09-20T18:59:05.085636+0000","last_unstale":"2024-09-20T19:05:33.582013+0000","last_undegraded":"2024-09-20T19:05:33.582013+0000","last_fullsized":"2024-09-20T19:05:33.582013+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:27.639384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:27.639384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":746,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.533769+0000","last_change":"2024-09-20T19:02:23.886386+0000","last_active":"2024-09-20T19:05:33.533769+0000","last_peered":"2024-09-20T19:05:33.533769+0000","last_clean":"2024-09-20T19:05:33.533769+0000","last_became_active":"2024-09-20T18:59:05.085131+0000","last_became_peered":"2024-09-20T18:59:05.085131+0000","last_unstale":"2024-09-20T19:05:33.533769+0000","last_undegraded":"2024-09-20T19:05:33.533769+0000","last_fullsized":"2024-09-20T19:05:33.533769+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:23.886308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:23.886308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"403'12853","reported_seq":13086,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:36.898145+0000","last_change":"2024-09-20T19:03:15.459244+0000","last_active":"2024-09-20T19:05:36.898145+0000","last_peered":"2024-09-20T19:05:36.898145+0000","last_clean":"2024-09-20T19:05:36.898145+0000","last_became_active":"2024-09-20T19:02:11.827305+0000","last_became_peered":"2024-09-20T19:02:11.827305+0000","last_unstale":"2024-09-20T19:05:36.898145+0000","last_undegraded":"2024-09-20T19:05:36.898145+0000","last_fullsized":"2024-09-20T19:05:36.898145+0000","mapping_epoch":325,"log_start":"360'6802","ondisk_log_start":"360'6802","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"350'5605","last_scrub_stamp":"2024-09-20T19:03:15.459174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:03:15.459174+0000","objects_scrubbed":5605,"log_size":6051,"log_dups_size":0,"ondisk_log_size":6051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":598,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":99155968,"num_objects":1513,"num_object_clones":0,"num_object_copies":4539,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1513,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12853,"num_write_kb":459712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"353'41","reported_seq":1026,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.529210+0000","last_change":"2024-09-20T19:03:17.024385+0000","last_active":"2024-09-20T19:05:33.529210+0000","last_peered":"2024-09-20T19:05:33.529210+0000","last_clean":"2024-09-20T19:05:33.529210+0000","last_became_active":"2024-09-20T19:03:16.976916+0000","last_became_peered":"2024-09-20T19:03:16.976916+0000","last_unstale":"2024-09-20T19:05:33.529210+0000","last_undegraded":"2024-09-20T19:05:33.529210+0000","last_fullsized":"2024-09-20T19:05:33.529210+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'40","last_scrub_stamp":"2024-09-20T19:02:36.900916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:36.900916+0000","objects_scrubbed":10,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":68,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.526939+0000","last_change":"2024-09-20T19:04:20.899630+0000","last_active":"2024-09-20T19:05:33.526939+0000","last_peered":"2024-09-20T19:05:33.526939+0000","last_clean":"2024-09-20T19:05:33.526939+0000","last_became_active":"2024-09-20T19:04:20.899352+0000","last_became_peered":"2024-09-20T19:04:20.899352+0000","last_unstale":"2024-09-20T19:05:33.526939+0000","last_undegraded":"2024-09-20T19:05:33.526939+0000","last_fullsized":"2024-09-20T19:05:33.526939+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:00:31.630361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:00:31.630361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":792,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.587694+0000","last_change":"2024-09-20T19:03:11.910684+0000","last_active":"2024-09-20T19:05:33.587694+0000","last_peered":"2024-09-20T19:05:33.587694+0000","last_clean":"2024-09-20T19:05:33.587694+0000","last_became_active":"2024-09-20T18:59:05.086786+0000","last_became_peered":"2024-09-20T18:59:05.086786+0000","last_unstale":"2024-09-20T19:05:33.587694+0000","last_undegraded":"2024-09-20T19:05:33.587694+0000","last_fullsized":"2024-09-20T19:05:33.587694+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-20T19:03:11.909394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:03:11.909394+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0.0012781159999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":762,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.525175+0000","last_change":"2024-09-20T19:04:20.907534+0000","last_active":"2024-09-20T19:05:33.525175+0000","last_peered":"2024-09-20T19:05:33.525175+0000","last_clean":"2024-09-20T19:05:33.525175+0000","last_became_active":"2024-09-20T19:04:20.907038+0000","last_became_peered":"2024-09-20T19:04:20.907038+0000","last_unstale":"2024-09-20T19:05:33.525175+0000","last_undegraded":"2024-09-20T19:05:33.525175+0000","last_fullsized":"2024-09-20T19:05:33.525175+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:15.034340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:03:15.034340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":733,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.592539+0000","last_change":"2024-09-20T19:04:20.896616+0000","last_active":"2024-09-20T19:05:33.592539+0000","last_peered":"2024-09-20T19:05:33.592539+0000","last_clean":"2024-09-20T19:05:33.592539+0000","last_became_active":"2024-09-20T19:04:20.896281+0000","last_became_peered":"2024-09-20T19:04:20.896281+0000","last_unstale":"2024-09-20T19:05:33.592539+0000","last_undegraded":"2024-09-20T19:05:33.592539+0000","last_fullsized":"2024-09-20T19:05:33.592539+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:04.526683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:04.526683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:00.655405+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"403'13127","reported_seq":13296,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:35.729741+0000","last_change":"2024-09-20T19:02:11.825132+0000","last_active":"2024-09-20T19:05:35.729741+0000","last_peered":"2024-09-20T19:05:35.729741+0000","last_clean":"2024-09-20T19:05:35.729741+0000","last_became_active":"2024-09-20T19:02:11.824846+0000","last_became_peered":"2024-09-20T19:02:11.824846+0000","last_unstale":"2024-09-20T19:05:35.729741+0000","last_undegraded":"2024-09-20T19:05:35.729741+0000","last_fullsized":"2024-09-20T19:05:35.729741+0000","mapping_epoch":325,"log_start":"363'7427","ondisk_log_start":"363'7427","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":5700,"log_dups_size":0,"ondisk_log_size":5700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":105709568,"num_objects":1613,"num_object_clones":0,"num_object_copies":4839,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1613,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13127,"num_write_kb":471680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"273'17","reported_seq":773,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.529429+0000","last_change":"2024-09-20T19:02:24.853473+0000","last_active":"2024-09-20T19:05:33.529429+0000","last_peered":"2024-09-20T19:05:33.529429+0000","last_clean":"2024-09-20T19:05:33.529429+0000","last_became_active":"2024-09-20T18:49:47.950904+0000","last_became_peered":"2024-09-20T18:49:47.950904+0000","last_unstale":"2024-09-20T19:05:33.529429+0000","last_undegraded":"2024-09-20T19:05:33.529429+0000","last_fullsized":"2024-09-20T19:05:33.529429+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'17","last_scrub_stamp":"2024-09-20T19:02:24.853374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:24.853374+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":36,"num_read_kb":25,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":758,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.529031+0000","last_change":"2024-09-20T19:02:29.840761+0000","last_active":"2024-09-20T19:05:33.529031+0000","last_peered":"2024-09-20T19:05:33.529031+0000","last_clean":"2024-09-20T19:05:33.529031+0000","last_became_active":"2024-09-20T18:59:05.079789+0000","last_became_peered":"2024-09-20T18:59:05.079789+0000","last_unstale":"2024-09-20T19:05:33.529031+0000","last_undegraded":"2024-09-20T19:05:33.529031+0000","last_fullsized":"2024-09-20T19:05:33.529031+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:29.840601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:29.840601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:33.076198+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":740,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.524248+0000","last_change":"2024-09-20T19:01:58.789304+0000","last_active":"2024-09-20T19:05:33.524248+0000","last_peered":"2024-09-20T19:05:33.524248+0000","last_clean":"2024-09-20T19:05:33.524248+0000","last_became_active":"2024-09-20T18:49:44.937135+0000","last_became_peered":"2024-09-20T18:49:44.937135+0000","last_unstale":"2024-09-20T19:05:33.524248+0000","last_undegraded":"2024-09-20T19:05:33.524248+0000","last_fullsized":"2024-09-20T19:05:33.524248+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:01:58.789251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:01:58.789251+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:05.405452+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00095558800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":715,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.525312+0000","last_change":"2024-09-20T19:01:17.747452+0000","last_active":"2024-09-20T19:05:33.525312+0000","last_peered":"2024-09-20T19:05:33.525312+0000","last_clean":"2024-09-20T19:05:33.525312+0000","last_became_active":"2024-09-20T18:49:49.954424+0000","last_became_peered":"2024-09-20T18:49:49.954424+0000","last_unstale":"2024-09-20T19:05:33.525312+0000","last_undegraded":"2024-09-20T19:05:33.525312+0000","last_fullsized":"2024-09-20T19:05:33.525312+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:17.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:17.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:58.423341+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":726,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.529662+0000","last_change":"2024-09-20T19:01:15.203591+0000","last_active":"2024-09-20T19:05:33.529662+0000","last_peered":"2024-09-20T19:05:33.529662+0000","last_clean":"2024-09-20T19:05:33.529662+0000","last_became_active":"2024-09-20T18:59:05.081459+0000","last_became_peered":"2024-09-20T18:59:05.081459+0000","last_unstale":"2024-09-20T19:05:33.529662+0000","last_undegraded":"2024-09-20T19:05:33.529662+0000","last_fullsized":"2024-09-20T19:05:33.529662+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:15.203529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:15.203529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"403'12933","reported_seq":13102,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:37.585952+0000","last_change":"2024-09-20T19:02:11.828477+0000","last_active":"2024-09-20T19:05:37.585952+0000","last_peered":"2024-09-20T19:05:37.585952+0000","last_clean":"2024-09-20T19:05:37.585952+0000","last_became_active":"2024-09-20T19:02:11.827996+0000","last_became_peered":"2024-09-20T19:02:11.827996+0000","last_unstale":"2024-09-20T19:05:37.585952+0000","last_undegraded":"2024-09-20T19:05:37.585952+0000","last_fullsized":"2024-09-20T19:05:37.585952+0000","mapping_epoch":325,"log_start":"366'7484","ondisk_log_start":"366'7484","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":5449,"log_dups_size":0,"ondisk_log_size":5449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":97845248,"num_objects":1493,"num_object_clones":0,"num_object_copies":4479,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12933,"num_write_kb":461632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"273'24","reported_seq":777,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.525730+0000","last_change":"2024-09-20T18:59:06.022021+0000","last_active":"2024-09-20T19:05:33.525730+0000","last_peered":"2024-09-20T19:05:33.525730+0000","last_clean":"2024-09-20T19:05:33.525730+0000","last_became_active":"2024-09-20T18:49:47.949230+0000","last_became_peered":"2024-09-20T18:49:47.949230+0000","last_unstale":"2024-09-20T19:05:33.525730+0000","last_undegraded":"2024-09-20T19:05:33.525730+0000","last_fullsized":"2024-09-20T19:05:33.525730+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:59:06.021962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:59:06.021962+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":762,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.524762+0000","last_change":"2024-09-20T19:04:20.907250+0000","last_active":"2024-09-20T19:05:33.524762+0000","last_peered":"2024-09-20T19:05:33.524762+0000","last_clean":"2024-09-20T19:05:33.524762+0000","last_became_active":"2024-09-20T19:04:20.906969+0000","last_became_peered":"2024-09-20T19:04:20.906969+0000","last_unstale":"2024-09-20T19:05:33.524762+0000","last_undegraded":"2024-09-20T19:05:33.524762+0000","last_fullsized":"2024-09-20T19:05:33.524762+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:24.984022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:24.984022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":1691,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:34.640413+0000","last_change":"2024-09-20T19:02:02.494344+0000","last_active":"2024-09-20T19:05:34.640413+0000","last_peered":"2024-09-20T19:05:34.640413+0000","last_clean":"2024-09-20T19:05:34.640413+0000","last_became_active":"2024-09-20T18:49:44.936819+0000","last_became_peered":"2024-09-20T18:49:44.936819+0000","last_unstale":"2024-09-20T19:05:34.640413+0000","last_undegraded":"2024-09-20T19:05:34.640413+0000","last_fullsized":"2024-09-20T19:05:34.640413+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-20T19:02:02.494291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:02:02.494291+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0010996199999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":943,"num_read_kb":938,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":736,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.589675+0000","last_change":"2024-09-20T19:02:27.785566+0000","last_active":"2024-09-20T19:05:33.589675+0000","last_peered":"2024-09-20T19:05:33.589675+0000","last_clean":"2024-09-20T19:05:33.589675+0000","last_became_active":"2024-09-20T18:49:49.961066+0000","last_became_peered":"2024-09-20T18:49:49.961066+0000","last_unstale":"2024-09-20T19:05:33.589675+0000","last_undegraded":"2024-09-20T19:05:33.589675+0000","last_fullsized":"2024-09-20T19:05:33.589675+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:27.785524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:27.785524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":732,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.528974+0000","last_change":"2024-09-20T19:02:26.861253+0000","last_active":"2024-09-20T19:05:33.528974+0000","last_peered":"2024-09-20T19:05:33.528974+0000","last_clean":"2024-09-20T19:05:33.528974+0000","last_became_active":"2024-09-20T18:49:51.970343+0000","last_became_peered":"2024-09-20T18:49:51.970343+0000","last_unstale":"2024-09-20T19:05:33.528974+0000","last_undegraded":"2024-09-20T19:05:33.528974+0000","last_fullsized":"2024-09-20T19:05:33.528974+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:26.861185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:26.861185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:32.099095+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"403'13074","reported_seq":13243,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:36.895339+0000","last_change":"2024-09-20T19:02:11.828065+0000","last_active":"2024-09-20T19:05:36.895339+0000","last_peered":"2024-09-20T19:05:36.895339+0000","last_clean":"2024-09-20T19:05:36.895339+0000","last_became_active":"2024-09-20T19:02:11.827628+0000","last_became_peered":"2024-09-20T19:02:11.827628+0000","last_unstale":"2024-09-20T19:05:36.895339+0000","last_undegraded":"2024-09-20T19:05:36.895339+0000","last_fullsized":"2024-09-20T19:05:36.895339+0000","mapping_epoch":325,"log_start":"361'7027","ondisk_log_start":"361'7027","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":6047,"log_dups_size":0,"ondisk_log_size":6047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":97255424,"num_objects":1484,"num_object_clones":0,"num_object_copies":4452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1484,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13074,"num_write_kb":465856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"375'34","reported_seq":485,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.581835+0000","last_change":"2024-09-20T19:04:20.988428+0000","last_active":"2024-09-20T19:05:33.581835+0000","last_peered":"2024-09-20T19:05:33.581835+0000","last_clean":"2024-09-20T19:05:33.581835+0000","last_became_active":"2024-09-20T19:04:20.928727+0000","last_became_peered":"2024-09-20T19:04:20.928727+0000","last_unstale":"2024-09-20T19:05:33.581835+0000","last_undegraded":"2024-09-20T19:05:33.581835+0000","last_fullsized":"2024-09-20T19:05:33.581835+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"308'32","last_scrub_stamp":"2024-09-20T19:02:25.699337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:25.699337+0000","objects_scrubbed":8,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":462,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":758,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.592398+0000","last_change":"2024-09-20T19:05:07.665281+0000","last_active":"2024-09-20T19:05:33.592398+0000","last_peered":"2024-09-20T19:05:33.592398+0000","last_clean":"2024-09-20T19:05:33.592398+0000","last_became_active":"2024-09-20T18:59:05.079111+0000","last_became_peered":"2024-09-20T18:59:05.079111+0000","last_unstale":"2024-09-20T19:05:33.592398+0000","last_undegraded":"2024-09-20T19:05:33.592398+0000","last_fullsized":"2024-09-20T19:05:33.592398+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:07.665167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:07.665167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:16.643707+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":805,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.529243+0000","last_change":"2024-09-20T19:03:14.368338+0000","last_active":"2024-09-20T19:05:33.529243+0000","last_peered":"2024-09-20T19:05:33.529243+0000","last_clean":"2024-09-20T19:05:33.529243+0000","last_became_active":"2024-09-20T18:59:05.086103+0000","last_became_peered":"2024-09-20T18:59:05.086103+0000","last_unstale":"2024-09-20T19:05:33.529243+0000","last_undegraded":"2024-09-20T19:05:33.529243+0000","last_fullsized":"2024-09-20T19:05:33.529243+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:03:14.368279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:03:14.368279+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":721,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.530185+0000","last_change":"2024-09-20T19:04:35.533709+0000","last_active":"2024-09-20T19:05:33.530185+0000","last_peered":"2024-09-20T19:05:33.530185+0000","last_clean":"2024-09-20T19:05:33.530185+0000","last_became_active":"2024-09-20T18:49:49.956128+0000","last_became_peered":"2024-09-20T18:49:49.956128+0000","last_unstale":"2024-09-20T19:05:33.530185+0000","last_undegraded":"2024-09-20T19:05:33.530185+0000","last_fullsized":"2024-09-20T19:05:33.530185+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:35.533633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:35.533633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:59.293116+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":757,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.589426+0000","last_change":"2024-09-20T19:04:20.899776+0000","last_active":"2024-09-20T19:05:33.589426+0000","last_peered":"2024-09-20T19:05:33.589426+0000","last_clean":"2024-09-20T19:05:33.589426+0000","last_became_active":"2024-09-20T19:04:20.899321+0000","last_became_peered":"2024-09-20T19:04:20.899321+0000","last_unstale":"2024-09-20T19:05:33.589426+0000","last_undegraded":"2024-09-20T19:05:33.589426+0000","last_fullsized":"2024-09-20T19:05:33.589426+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:30.557368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:30.557368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.0","version":"403'13120","reported_seq":22527,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:36.911029+0000","last_change":"2024-09-20T19:05:21.939972+0000","last_active":"2024-09-20T19:05:36.911029+0000","last_peered":"2024-09-20T19:05:36.911029+0000","last_clean":"2024-09-20T19:05:36.911029+0000","last_became_active":"2024-09-20T19:05:21.905829+0000","last_became_peered":"2024-09-20T19:05:21.905829+0000","last_unstale":"2024-09-20T19:05:36.911029+0000","last_undegraded":"2024-09-20T19:05:36.911029+0000","last_fullsized":"2024-09-20T19:05:36.911029+0000","mapping_epoch":394,"log_start":"360'7039","ondisk_log_start":"360'7039","created":325,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":6081,"log_dups_size":0,"ondisk_log_size":6081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":98304000,"num_objects":1500,"num_object_clones":0,"num_object_copies":4500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13120,"num_write_kb":467840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4613,"num_bytes_recovered":200736768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"274'19","reported_seq":80,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.527216+0000","last_change":"2024-09-20T19:04:21.040511+0000","last_active":"2024-09-20T19:05:33.527216+0000","last_peered":"2024-09-20T19:05:33.527216+0000","last_clean":"2024-09-20T19:05:33.527216+0000","last_became_active":"2024-09-20T19:04:20.897175+0000","last_became_peered":"2024-09-20T19:04:20.897175+0000","last_unstale":"2024-09-20T19:05:33.527216+0000","last_undegraded":"2024-09-20T19:05:33.527216+0000","last_fullsized":"2024-09-20T19:05:33.527216+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:58:24.616580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:24.616580+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":68,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.527639+0000","last_change":"2024-09-20T19:04:20.887455+0000","last_active":"2024-09-20T19:05:33.527639+0000","last_peered":"2024-09-20T19:05:33.527639+0000","last_clean":"2024-09-20T19:05:33.527639+0000","last_became_active":"2024-09-20T19:04:20.887102+0000","last_became_peered":"2024-09-20T19:04:20.887102+0000","last_unstale":"2024-09-20T19:05:33.527639+0000","last_undegraded":"2024-09-20T19:05:33.527639+0000","last_fullsized":"2024-09-20T19:05:33.527639+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.693993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.693993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":774,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.589290+0000","last_change":"2024-09-20T19:04:20.899830+0000","last_active":"2024-09-20T19:05:33.589290+0000","last_peered":"2024-09-20T19:05:33.589290+0000","last_clean":"2024-09-20T19:05:33.589290+0000","last_became_active":"2024-09-20T19:04:20.899331+0000","last_became_peered":"2024-09-20T19:04:20.899331+0000","last_unstale":"2024-09-20T19:05:33.589290+0000","last_undegraded":"2024-09-20T19:05:33.589290+0000","last_fullsized":"2024-09-20T19:05:33.589290+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:38.463404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:38.463404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:26.726228+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":738,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.593996+0000","last_change":"2024-09-20T19:04:35.529672+0000","last_active":"2024-09-20T19:05:33.593996+0000","last_peered":"2024-09-20T19:05:33.593996+0000","last_clean":"2024-09-20T19:05:33.593996+0000","last_became_active":"2024-09-20T18:59:05.079897+0000","last_became_peered":"2024-09-20T18:59:05.079897+0000","last_unstale":"2024-09-20T19:05:33.593996+0000","last_undegraded":"2024-09-20T19:05:33.593996+0000","last_fullsized":"2024-09-20T19:05:33.593996+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:35.529584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:04:35.529584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:57.166916+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"403'13328","reported_seq":13610,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:36.583689+0000","last_change":"2024-09-20T19:05:17.450180+0000","last_active":"2024-09-20T19:05:36.583689+0000","last_peered":"2024-09-20T19:05:36.583689+0000","last_clean":"2024-09-20T19:05:36.583689+0000","last_became_active":"2024-09-20T19:02:11.825011+0000","last_became_peered":"2024-09-20T19:02:11.825011+0000","last_unstale":"2024-09-20T19:05:36.583689+0000","last_undegraded":"2024-09-20T19:05:36.583689+0000","last_fullsized":"2024-09-20T19:05:36.583689+0000","mapping_epoch":325,"log_start":"357'6728","ondisk_log_start":"357'6728","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"392'12286","last_scrub_stamp":"2024-09-20T19:05:17.450066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:05:17.450066+0000","objects_scrubbed":2624,"log_size":6600,"log_dups_size":0,"ondisk_log_size":6600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:20.515552+0000","scrub_duration":996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":101449728,"num_objects":1548,"num_object_clones":0,"num_object_copies":4644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1548,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13328,"num_write_kb":476032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"274'31","reported_seq":86,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.527449+0000","last_change":"2024-09-20T19:04:20.949305+0000","last_active":"2024-09-20T19:05:33.527449+0000","last_peered":"2024-09-20T19:05:33.527449+0000","last_clean":"2024-09-20T19:05:33.527449+0000","last_became_active":"2024-09-20T19:04:20.883306+0000","last_became_peered":"2024-09-20T19:04:20.883306+0000","last_unstale":"2024-09-20T19:05:33.527449+0000","last_undegraded":"2024-09-20T19:05:33.527449+0000","last_fullsized":"2024-09-20T19:05:33.527449+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'31","last_scrub_stamp":"2024-09-20T19:03:12.196851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:12.196851+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":49,"num_read_kb":39,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":332,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.582145+0000","last_change":"2024-09-20T19:04:20.899712+0000","last_active":"2024-09-20T19:05:33.582145+0000","last_peered":"2024-09-20T19:05:33.582145+0000","last_clean":"2024-09-20T19:05:33.582145+0000","last_became_active":"2024-09-20T19:04:20.899415+0000","last_became_peered":"2024-09-20T19:04:20.899415+0000","last_unstale":"2024-09-20T19:05:33.582145+0000","last_undegraded":"2024-09-20T19:05:33.582145+0000","last_fullsized":"2024-09-20T19:05:33.582145+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:42.479489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:42.479489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:36.220255+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":753,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.529388+0000","last_change":"2024-09-20T19:04:20.892646+0000","last_active":"2024-09-20T19:05:33.529388+0000","last_peered":"2024-09-20T19:05:33.529388+0000","last_clean":"2024-09-20T19:05:33.529388+0000","last_became_active":"2024-09-20T19:04:20.891772+0000","last_became_peered":"2024-09-20T19:04:20.891772+0000","last_unstale":"2024-09-20T19:05:33.529388+0000","last_undegraded":"2024-09-20T19:05:33.529388+0000","last_fullsized":"2024-09-20T19:05:33.529388+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:32.856856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:32.856856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":751,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.525329+0000","last_change":"2024-09-20T19:04:20.897937+0000","last_active":"2024-09-20T19:05:33.525329+0000","last_peered":"2024-09-20T19:05:33.525329+0000","last_clean":"2024-09-20T19:05:33.525329+0000","last_became_active":"2024-09-20T19:04:20.897569+0000","last_became_peered":"2024-09-20T19:04:20.897569+0000","last_unstale":"2024-09-20T19:05:33.525329+0000","last_undegraded":"2024-09-20T19:05:33.525329+0000","last_fullsized":"2024-09-20T19:05:33.525329+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:31.687188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:31.687188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:24.424476+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2","version":"403'13129","reported_seq":14645,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:36.947593+0000","last_change":"2024-09-20T19:04:21.974196+0000","last_active":"2024-09-20T19:05:36.947593+0000","last_peered":"2024-09-20T19:05:36.947593+0000","last_clean":"2024-09-20T19:05:36.947593+0000","last_became_active":"2024-09-20T19:04:21.956633+0000","last_became_peered":"2024-09-20T19:04:21.956633+0000","last_unstale":"2024-09-20T19:05:36.947593+0000","last_undegraded":"2024-09-20T19:05:36.947593+0000","last_fullsized":"2024-09-20T19:05:36.947593+0000","mapping_epoch":375,"log_start":"367'7882","ondisk_log_start":"367'7882","created":325,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":5247,"log_dups_size":0,"ondisk_log_size":5247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:28.169249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100859904,"num_objects":1539,"num_object_clones":0,"num_object_copies":4617,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1539,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13129,"num_write_kb":469376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":665,"num_bytes_recovered":43581440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"273'20","reported_seq":80,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.528009+0000","last_change":"2024-09-20T19:04:20.945143+0000","last_active":"2024-09-20T19:05:33.528009+0000","last_peered":"2024-09-20T19:05:33.528009+0000","last_clean":"2024-09-20T19:05:33.528009+0000","last_became_active":"2024-09-20T19:04:20.896483+0000","last_became_peered":"2024-09-20T19:04:20.896483+0000","last_unstale":"2024-09-20T19:05:33.528009+0000","last_undegraded":"2024-09-20T19:05:33.528009+0000","last_fullsized":"2024-09-20T19:05:33.528009+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'20","last_scrub_stamp":"2024-09-20T19:03:04.282800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:04.282800+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":761,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.593649+0000","last_change":"2024-09-20T19:04:20.893263+0000","last_active":"2024-09-20T19:05:33.593649+0000","last_peered":"2024-09-20T19:05:33.593649+0000","last_clean":"2024-09-20T19:05:33.593649+0000","last_became_active":"2024-09-20T19:04:20.892951+0000","last_became_peered":"2024-09-20T19:04:20.892951+0000","last_unstale":"2024-09-20T19:05:33.593649+0000","last_undegraded":"2024-09-20T19:05:33.593649+0000","last_fullsized":"2024-09-20T19:05:33.593649+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:34.985163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:34.985163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":946,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:36.190674+0000","last_change":"2024-09-20T19:05:08.406307+0000","last_active":"2024-09-20T19:05:36.190674+0000","last_peered":"2024-09-20T19:05:36.190674+0000","last_clean":"2024-09-20T19:05:36.190674+0000","last_became_active":"2024-09-20T18:49:49.957005+0000","last_became_peered":"2024-09-20T18:49:49.957005+0000","last_unstale":"2024-09-20T19:05:36.190674+0000","last_undegraded":"2024-09-20T19:05:36.190674+0000","last_fullsized":"2024-09-20T19:05:36.190674+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T19:05:08.406231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:05:08.406231+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:20.576797+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":68,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.527318+0000","last_change":"2024-09-20T19:04:20.885294+0000","last_active":"2024-09-20T19:05:33.527318+0000","last_peered":"2024-09-20T19:05:33.527318+0000","last_clean":"2024-09-20T19:05:33.527318+0000","last_became_active":"2024-09-20T19:04:20.884994+0000","last_became_peered":"2024-09-20T19:04:20.884994+0000","last_unstale":"2024-09-20T19:05:33.527318+0000","last_undegraded":"2024-09-20T19:05:33.527318+0000","last_fullsized":"2024-09-20T19:05:33.527318+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:11.207994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:03:11.207994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.5","version":"403'12986","reported_seq":13230,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:36.554142+0000","last_change":"2024-09-20T19:04:04.390108+0000","last_active":"2024-09-20T19:05:36.554142+0000","last_peered":"2024-09-20T19:05:36.554142+0000","last_clean":"2024-09-20T19:05:36.554142+0000","last_became_active":"2024-09-20T19:02:11.828162+0000","last_became_peered":"2024-09-20T19:02:11.828162+0000","last_unstale":"2024-09-20T19:05:36.554142+0000","last_undegraded":"2024-09-20T19:05:36.554142+0000","last_fullsized":"2024-09-20T19:05:36.554142+0000","mapping_epoch":325,"log_start":"357'6368","ondisk_log_start":"357'6368","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"367'7875","last_scrub_stamp":"2024-09-20T19:04:04.389838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:04:04.389838+0000","objects_scrubbed":6698,"log_size":6618,"log_dups_size":0,"ondisk_log_size":6618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"queued for scrub","scrub_duration":2035,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100401152,"num_objects":1532,"num_object_clones":0,"num_object_copies":4596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1532,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12986,"num_write_kb":464576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"274'15","reported_seq":803,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.524604+0000","last_change":"2024-09-20T19:04:20.978525+0000","last_active":"2024-09-20T19:05:33.524604+0000","last_peered":"2024-09-20T19:05:33.524604+0000","last_clean":"2024-09-20T19:05:33.524604+0000","last_became_active":"2024-09-20T19:04:20.906834+0000","last_became_peered":"2024-09-20T19:04:20.906834+0000","last_unstale":"2024-09-20T19:05:33.524604+0000","last_undegraded":"2024-09-20T19:05:33.524604+0000","last_fullsized":"2024-09-20T19:05:33.524604+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'15","last_scrub_stamp":"2024-09-20T19:01:27.802100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:27.802100+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":288,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.524410+0000","last_change":"2024-09-20T19:04:16.384877+0000","last_active":"2024-09-20T19:05:33.524410+0000","last_peered":"2024-09-20T19:05:33.524410+0000","last_clean":"2024-09-20T19:05:33.524410+0000","last_became_active":"2024-09-20T19:00:01.190875+0000","last_became_peered":"2024-09-20T19:00:01.190875+0000","last_unstale":"2024-09-20T19:05:33.524410+0000","last_undegraded":"2024-09-20T19:05:33.524410+0000","last_fullsized":"2024-09-20T19:05:33.524410+0000","mapping_epoch":277,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":278,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:16.384809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:04:16.384809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:44.880751+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":764,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.523947+0000","last_change":"2024-09-20T19:04:20.889614+0000","last_active":"2024-09-20T19:05:33.523947+0000","last_peered":"2024-09-20T19:05:33.523947+0000","last_clean":"2024-09-20T19:05:33.523947+0000","last_became_active":"2024-09-20T19:04:20.889273+0000","last_became_peered":"2024-09-20T19:04:20.889273+0000","last_unstale":"2024-09-20T19:05:33.523947+0000","last_undegraded":"2024-09-20T19:05:33.523947+0000","last_fullsized":"2024-09-20T19:05:33.523947+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:03.389611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:03.389611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:26.422333+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":725,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.524756+0000","last_change":"2024-09-20T19:02:31.120809+0000","last_active":"2024-09-20T19:05:33.524756+0000","last_peered":"2024-09-20T19:05:33.524756+0000","last_clean":"2024-09-20T19:05:33.524756+0000","last_became_active":"2024-09-20T18:49:51.964098+0000","last_became_peered":"2024-09-20T18:49:51.964098+0000","last_unstale":"2024-09-20T19:05:33.524756+0000","last_undegraded":"2024-09-20T19:05:33.524756+0000","last_fullsized":"2024-09-20T19:05:33.524756+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:31.120623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:31.120623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"403'13031","reported_seq":13220,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:37.577009+0000","last_change":"2024-09-20T19:04:21.973100+0000","last_active":"2024-09-20T19:05:37.577009+0000","last_peered":"2024-09-20T19:05:37.577009+0000","last_clean":"2024-09-20T19:05:37.577009+0000","last_became_active":"2024-09-20T19:04:21.960863+0000","last_became_peered":"2024-09-20T19:04:21.960863+0000","last_unstale":"2024-09-20T19:05:37.577009+0000","last_undegraded":"2024-09-20T19:05:37.577009+0000","last_fullsized":"2024-09-20T19:05:37.577009+0000","mapping_epoch":375,"log_start":"366'7597","ondisk_log_start":"366'7597","created":325,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":5434,"log_dups_size":0,"ondisk_log_size":5434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88408064,"num_objects":1349,"num_object_clones":0,"num_object_copies":4047,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1349,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13031,"num_write_kb":460160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"274'35","reported_seq":359,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.582005+0000","last_change":"2024-09-20T19:03:09.841596+0000","last_active":"2024-09-20T19:05:33.582005+0000","last_peered":"2024-09-20T19:05:33.582005+0000","last_clean":"2024-09-20T19:05:33.582005+0000","last_became_active":"2024-09-20T18:59:05.087391+0000","last_became_peered":"2024-09-20T18:59:05.087391+0000","last_unstale":"2024-09-20T19:05:33.582005+0000","last_undegraded":"2024-09-20T19:05:33.582005+0000","last_fullsized":"2024-09-20T19:05:33.582005+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'35","last_scrub_stamp":"2024-09-20T19:03:09.841361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:09.841361+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":751,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.530147+0000","last_change":"2024-09-20T19:02:05.440745+0000","last_active":"2024-09-20T19:05:33.530147+0000","last_peered":"2024-09-20T19:05:33.530147+0000","last_clean":"2024-09-20T19:05:33.530147+0000","last_became_active":"2024-09-20T18:59:05.086670+0000","last_became_peered":"2024-09-20T18:59:05.086670+0000","last_unstale":"2024-09-20T19:05:33.530147+0000","last_undegraded":"2024-09-20T19:05:33.530147+0000","last_fullsized":"2024-09-20T19:05:33.530147+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:05.440677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:05.440677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":747,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.524290+0000","last_change":"2024-09-20T19:05:11.444197+0000","last_active":"2024-09-20T19:05:33.524290+0000","last_peered":"2024-09-20T19:05:33.524290+0000","last_clean":"2024-09-20T19:05:33.524290+0000","last_became_active":"2024-09-20T18:49:49.957558+0000","last_became_peered":"2024-09-20T18:49:49.957558+0000","last_unstale":"2024-09-20T19:05:33.524290+0000","last_undegraded":"2024-09-20T19:05:33.524290+0000","last_fullsized":"2024-09-20T19:05:33.524290+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:11.444143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:05:11.444143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:23.300683+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":717,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.529228+0000","last_change":"2024-09-20T18:53:47.370152+0000","last_active":"2024-09-20T19:05:33.529228+0000","last_peered":"2024-09-20T19:05:33.529228+0000","last_clean":"2024-09-20T19:05:33.529228+0000","last_became_active":"2024-09-20T18:49:51.970798+0000","last_became_peered":"2024-09-20T18:49:51.970798+0000","last_unstale":"2024-09-20T19:05:33.529228+0000","last_undegraded":"2024-09-20T19:05:33.529228+0000","last_fullsized":"2024-09-20T19:05:33.529228+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:47.370108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:47.370108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:30.728805+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.7","version":"403'13237","reported_seq":13406,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:36.882930+0000","last_change":"2024-09-20T19:02:11.827681+0000","last_active":"2024-09-20T19:05:36.882930+0000","last_peered":"2024-09-20T19:05:36.882930+0000","last_clean":"2024-09-20T19:05:36.882930+0000","last_became_active":"2024-09-20T19:02:11.827360+0000","last_became_peered":"2024-09-20T19:02:11.827360+0000","last_unstale":"2024-09-20T19:05:36.882930+0000","last_undegraded":"2024-09-20T19:05:36.882930+0000","last_fullsized":"2024-09-20T19:05:36.882930+0000","mapping_epoch":325,"log_start":"361'7207","ondisk_log_start":"361'7207","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":6030,"log_dups_size":0,"ondisk_log_size":6030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":97452032,"num_objects":1487,"num_object_clones":0,"num_object_copies":4461,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1487,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13237,"num_write_kb":471168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"273'18","reported_seq":80,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.527734+0000","last_change":"2024-09-20T19:04:20.936465+0000","last_active":"2024-09-20T19:05:33.527734+0000","last_peered":"2024-09-20T19:05:33.527734+0000","last_clean":"2024-09-20T19:05:33.527734+0000","last_became_active":"2024-09-20T19:04:20.888905+0000","last_became_peered":"2024-09-20T19:04:20.888905+0000","last_unstale":"2024-09-20T19:05:33.527734+0000","last_undegraded":"2024-09-20T19:05:33.527734+0000","last_fullsized":"2024-09-20T19:05:33.527734+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'18","last_scrub_stamp":"2024-09-20T19:03:13.201592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:13.201592+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":770,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.530162+0000","last_change":"2024-09-20T19:04:50.680303+0000","last_active":"2024-09-20T19:05:33.530162+0000","last_peered":"2024-09-20T19:05:33.530162+0000","last_clean":"2024-09-20T19:05:33.530162+0000","last_became_active":"2024-09-20T19:04:20.892429+0000","last_became_peered":"2024-09-20T19:04:20.892429+0000","last_unstale":"2024-09-20T19:05:33.530162+0000","last_undegraded":"2024-09-20T19:05:33.530162+0000","last_fullsized":"2024-09-20T19:05:33.530162+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:50.680222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:04:50.680222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:57.765584+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":739,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.523908+0000","last_change":"2024-09-20T19:04:48.406445+0000","last_active":"2024-09-20T19:05:33.523908+0000","last_peered":"2024-09-20T19:05:33.523908+0000","last_clean":"2024-09-20T19:05:33.523908+0000","last_became_active":"2024-09-20T18:49:49.954944+0000","last_became_peered":"2024-09-20T18:49:49.954944+0000","last_unstale":"2024-09-20T19:05:33.523908+0000","last_undegraded":"2024-09-20T19:05:33.523908+0000","last_fullsized":"2024-09-20T19:05:33.523908+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:48.406373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:48.406373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:58.629290+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":724,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.593259+0000","last_change":"2024-09-20T19:02:35.968178+0000","last_active":"2024-09-20T19:05:33.593259+0000","last_peered":"2024-09-20T19:05:33.593259+0000","last_clean":"2024-09-20T19:05:33.593259+0000","last_became_active":"2024-09-20T18:49:51.972970+0000","last_became_peered":"2024-09-20T18:49:51.972970+0000","last_unstale":"2024-09-20T19:05:33.593259+0000","last_undegraded":"2024-09-20T19:05:33.593259+0000","last_fullsized":"2024-09-20T19:05:33.593259+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:35.968052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:35.968052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.6","version":"403'13184","reported_seq":22660,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:35.692670+0000","last_change":"2024-09-20T19:05:21.938456+0000","last_active":"2024-09-20T19:05:35.692670+0000","last_peered":"2024-09-20T19:05:35.692670+0000","last_clean":"2024-09-20T19:05:35.692670+0000","last_became_active":"2024-09-20T19:05:21.906044+0000","last_became_peered":"2024-09-20T19:05:21.906044+0000","last_unstale":"2024-09-20T19:05:35.692670+0000","last_undegraded":"2024-09-20T19:05:35.692670+0000","last_fullsized":"2024-09-20T19:05:35.692670+0000","mapping_epoch":394,"log_start":"363'7462","ondisk_log_start":"363'7462","created":325,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":5722,"log_dups_size":0,"ondisk_log_size":5722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":105512960,"num_objects":1610,"num_object_clones":0,"num_object_copies":4830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1610,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13184,"num_write_kb":473408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4647,"num_bytes_recovered":203751424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"273'27","reported_seq":829,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.587100+0000","last_change":"2024-09-20T19:02:39.709344+0000","last_active":"2024-09-20T19:05:33.587100+0000","last_peered":"2024-09-20T19:05:33.587100+0000","last_clean":"2024-09-20T19:05:33.587100+0000","last_became_active":"2024-09-20T18:59:05.083726+0000","last_became_peered":"2024-09-20T18:59:05.083726+0000","last_unstale":"2024-09-20T19:05:33.587100+0000","last_undegraded":"2024-09-20T19:05:33.587100+0000","last_fullsized":"2024-09-20T19:05:33.587100+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'27","last_scrub_stamp":"2024-09-20T19:02:39.709269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:39.709269+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":802,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.530240+0000","last_change":"2024-09-20T19:04:46.691255+0000","last_active":"2024-09-20T19:05:33.530240+0000","last_peered":"2024-09-20T19:05:33.530240+0000","last_clean":"2024-09-20T19:05:33.530240+0000","last_became_active":"2024-09-20T19:04:20.884854+0000","last_became_peered":"2024-09-20T19:04:20.884854+0000","last_unstale":"2024-09-20T19:05:33.530240+0000","last_undegraded":"2024-09-20T19:05:33.530240+0000","last_fullsized":"2024-09-20T19:05:33.530240+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-20T19:04:46.691171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:04:46.691171+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:02.158207+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"375'14","reported_seq":977,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.529675+0000","last_change":"2024-09-20T19:04:39.564069+0000","last_active":"2024-09-20T19:05:33.529675+0000","last_peered":"2024-09-20T19:05:33.529675+0000","last_clean":"2024-09-20T19:05:33.529675+0000","last_became_active":"2024-09-20T19:04:20.892956+0000","last_became_peered":"2024-09-20T19:04:20.892956+0000","last_unstale":"2024-09-20T19:05:33.529675+0000","last_undegraded":"2024-09-20T19:05:33.529675+0000","last_fullsized":"2024-09-20T19:05:33.529675+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"375'14","last_scrub_stamp":"2024-09-20T19:04:39.563917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:39.563917+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:49.606578+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":746,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.587304+0000","last_change":"2024-09-20T19:03:06.888620+0000","last_active":"2024-09-20T19:05:33.587304+0000","last_peered":"2024-09-20T19:05:33.587304+0000","last_clean":"2024-09-20T19:05:33.587304+0000","last_became_active":"2024-09-20T18:59:05.087005+0000","last_became_peered":"2024-09-20T18:59:05.087005+0000","last_unstale":"2024-09-20T19:05:33.587304+0000","last_undegraded":"2024-09-20T19:05:33.587304+0000","last_fullsized":"2024-09-20T19:05:33.587304+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:06.888563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:03:06.888563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":306,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.581174+0000","last_change":"2024-09-20T19:01:16.542398+0000","last_active":"2024-09-20T19:05:33.581174+0000","last_peered":"2024-09-20T19:05:33.581174+0000","last_clean":"2024-09-20T19:05:33.581174+0000","last_became_active":"2024-09-20T18:59:05.085427+0000","last_became_peered":"2024-09-20T18:59:05.085427+0000","last_unstale":"2024-09-20T19:05:33.581174+0000","last_undegraded":"2024-09-20T19:05:33.581174+0000","last_fullsized":"2024-09-20T19:05:33.581174+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:16.542335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:16.542335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:29.408351+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"273'12","reported_seq":757,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.592936+0000","last_change":"2024-09-20T19:04:48.570718+0000","last_active":"2024-09-20T19:05:33.592936+0000","last_peered":"2024-09-20T19:05:33.592936+0000","last_clean":"2024-09-20T19:05:33.592936+0000","last_became_active":"2024-09-20T18:49:48.500662+0000","last_became_peered":"2024-09-20T18:49:48.500662+0000","last_unstale":"2024-09-20T19:05:33.592936+0000","last_undegraded":"2024-09-20T19:05:33.592936+0000","last_fullsized":"2024-09-20T19:05:33.592936+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'12","last_scrub_stamp":"2024-09-20T19:04:48.570639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:04:48.570639+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:50.389558+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":735,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.589519+0000","last_change":"2024-09-20T19:05:22.605885+0000","last_active":"2024-09-20T19:05:33.589519+0000","last_peered":"2024-09-20T19:05:33.589519+0000","last_clean":"2024-09-20T19:05:33.589519+0000","last_became_active":"2024-09-20T18:49:49.956749+0000","last_became_peered":"2024-09-20T18:49:49.956749+0000","last_unstale":"2024-09-20T19:05:33.589519+0000","last_undegraded":"2024-09-20T19:05:33.589519+0000","last_fullsized":"2024-09-20T19:05:33.589519+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:22.605839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:05:22.605839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:49.241766+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":763,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.529642+0000","last_change":"2024-09-20T19:04:20.914426+0000","last_active":"2024-09-20T19:05:33.529642+0000","last_peered":"2024-09-20T19:05:33.529642+0000","last_clean":"2024-09-20T19:05:33.529642+0000","last_became_active":"2024-09-20T19:04:20.914115+0000","last_became_peered":"2024-09-20T19:04:20.914115+0000","last_unstale":"2024-09-20T19:05:33.529642+0000","last_undegraded":"2024-09-20T19:05:33.529642+0000","last_fullsized":"2024-09-20T19:05:33.529642+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:03.509903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:03.509903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"273'15","reported_seq":749,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.525814+0000","last_change":"2024-09-20T19:01:15.875779+0000","last_active":"2024-09-20T19:05:33.525814+0000","last_peered":"2024-09-20T19:05:33.525814+0000","last_clean":"2024-09-20T19:05:33.525814+0000","last_became_active":"2024-09-20T18:49:48.500803+0000","last_became_peered":"2024-09-20T18:49:48.500803+0000","last_unstale":"2024-09-20T19:05:33.525814+0000","last_undegraded":"2024-09-20T19:05:33.525814+0000","last_fullsized":"2024-09-20T19:05:33.525814+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:01:15.875693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:15.875693+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":775,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.529762+0000","last_change":"2024-09-20T19:04:47.713764+0000","last_active":"2024-09-20T19:05:33.529762+0000","last_peered":"2024-09-20T19:05:33.529762+0000","last_clean":"2024-09-20T19:05:33.529762+0000","last_became_active":"2024-09-20T19:04:20.893250+0000","last_became_peered":"2024-09-20T19:04:20.893250+0000","last_unstale":"2024-09-20T19:05:33.529762+0000","last_undegraded":"2024-09-20T19:05:33.529762+0000","last_fullsized":"2024-09-20T19:05:33.529762+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:47.713585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:04:47.713585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:49.334341+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":735,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.592806+0000","last_change":"2024-09-20T19:04:50.573619+0000","last_active":"2024-09-20T19:05:33.592806+0000","last_peered":"2024-09-20T19:05:33.592806+0000","last_clean":"2024-09-20T19:05:33.592806+0000","last_became_active":"2024-09-20T18:49:49.961947+0000","last_became_peered":"2024-09-20T18:49:49.961947+0000","last_unstale":"2024-09-20T19:05:33.592806+0000","last_undegraded":"2024-09-20T19:05:33.592806+0000","last_fullsized":"2024-09-20T19:05:33.592806+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:50.573550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:50.573550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:02.174525+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"66'1","reported_seq":776,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.524500+0000","last_change":"2024-09-20T19:04:20.913449+0000","last_active":"2024-09-20T19:05:33.524500+0000","last_peered":"2024-09-20T19:05:33.524500+0000","last_clean":"2024-09-20T19:05:33.524500+0000","last_became_active":"2024-09-20T19:04:20.892236+0000","last_became_peered":"2024-09-20T19:04:20.892236+0000","last_unstale":"2024-09-20T19:05:33.524500+0000","last_undegraded":"2024-09-20T19:05:33.524500+0000","last_fullsized":"2024-09-20T19:05:33.524500+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T19:03:12.499684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:03:12.499684+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"274'15","reported_seq":748,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.588386+0000","last_change":"2024-09-20T19:01:36.532486+0000","last_active":"2024-09-20T19:05:33.588386+0000","last_peered":"2024-09-20T19:05:33.588386+0000","last_clean":"2024-09-20T19:05:33.588386+0000","last_became_active":"2024-09-20T18:49:48.501011+0000","last_became_peered":"2024-09-20T18:49:48.501011+0000","last_unstale":"2024-09-20T19:05:33.588386+0000","last_undegraded":"2024-09-20T19:05:33.588386+0000","last_fullsized":"2024-09-20T19:05:33.588386+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'15","last_scrub_stamp":"2024-09-20T19:01:36.532419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:36.532419+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":736,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.525559+0000","last_change":"2024-09-20T19:01:30.864866+0000","last_active":"2024-09-20T19:05:33.525559+0000","last_peered":"2024-09-20T19:05:33.525559+0000","last_clean":"2024-09-20T19:05:33.525559+0000","last_became_active":"2024-09-20T18:49:45.938511+0000","last_became_peered":"2024-09-20T18:49:45.938511+0000","last_unstale":"2024-09-20T19:05:33.525559+0000","last_undegraded":"2024-09-20T19:05:33.525559+0000","last_fullsized":"2024-09-20T19:05:33.525559+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:30.864794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:30.864794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":933,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.594044+0000","last_change":"2024-09-20T19:04:41.601208+0000","last_active":"2024-09-20T19:05:33.594044+0000","last_peered":"2024-09-20T19:05:33.594044+0000","last_clean":"2024-09-20T19:05:33.594044+0000","last_became_active":"2024-09-20T18:49:49.960550+0000","last_became_peered":"2024-09-20T18:49:49.960550+0000","last_unstale":"2024-09-20T19:05:33.594044+0000","last_undegraded":"2024-09-20T19:05:33.594044+0000","last_fullsized":"2024-09-20T19:05:33.594044+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T19:04:41.601073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:41.601073+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:03.890101+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":68,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.527486+0000","last_change":"2024-09-20T19:04:20.884397+0000","last_active":"2024-09-20T19:05:33.527486+0000","last_peered":"2024-09-20T19:05:33.527486+0000","last_clean":"2024-09-20T19:05:33.527486+0000","last_became_active":"2024-09-20T19:04:20.884100+0000","last_became_peered":"2024-09-20T19:04:20.884100+0000","last_unstale":"2024-09-20T19:05:33.527486+0000","last_undegraded":"2024-09-20T19:05:33.527486+0000","last_fullsized":"2024-09-20T19:05:33.527486+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:59.497810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:59:59.497810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"274'20","reported_seq":764,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.594125+0000","last_change":"2024-09-20T18:57:20.701592+0000","last_active":"2024-09-20T19:05:33.594125+0000","last_peered":"2024-09-20T19:05:33.594125+0000","last_clean":"2024-09-20T19:05:33.594125+0000","last_became_active":"2024-09-20T18:49:58.997494+0000","last_became_peered":"2024-09-20T18:49:58.997494+0000","last_unstale":"2024-09-20T19:05:33.594125+0000","last_undegraded":"2024-09-20T19:05:33.594125+0000","last_fullsized":"2024-09-20T19:05:33.594125+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:57:20.701543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:20.701543+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:23.577077+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":753,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.529437+0000","last_change":"2024-09-20T19:04:20.884940+0000","last_active":"2024-09-20T19:05:33.529437+0000","last_peered":"2024-09-20T19:05:33.529437+0000","last_clean":"2024-09-20T19:05:33.529437+0000","last_became_active":"2024-09-20T19:04:20.884582+0000","last_became_peered":"2024-09-20T19:04:20.884582+0000","last_unstale":"2024-09-20T19:05:33.529437+0000","last_undegraded":"2024-09-20T19:05:33.529437+0000","last_fullsized":"2024-09-20T19:05:33.529437+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.632380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.632380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":925,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.588210+0000","last_change":"2024-09-20T19:01:10.601374+0000","last_active":"2024-09-20T19:05:33.588210+0000","last_peered":"2024-09-20T19:05:33.588210+0000","last_clean":"2024-09-20T19:05:33.588210+0000","last_became_active":"2024-09-20T18:49:49.962243+0000","last_became_peered":"2024-09-20T18:49:49.962243+0000","last_unstale":"2024-09-20T19:05:33.588210+0000","last_undegraded":"2024-09-20T19:05:33.588210+0000","last_fullsized":"2024-09-20T19:05:33.588210+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T19:01:10.601300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:10.601300+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:23.615982+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":732,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.525182+0000","last_change":"2024-09-20T19:01:03.821977+0000","last_active":"2024-09-20T19:05:33.525182+0000","last_peered":"2024-09-20T19:05:33.525182+0000","last_clean":"2024-09-20T19:05:33.525182+0000","last_became_active":"2024-09-20T18:49:51.969093+0000","last_became_peered":"2024-09-20T18:49:51.969093+0000","last_unstale":"2024-09-20T19:05:33.525182+0000","last_undegraded":"2024-09-20T19:05:33.525182+0000","last_fullsized":"2024-09-20T19:05:33.525182+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:03.821908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:03.821908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"273'19","reported_seq":80,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.526901+0000","last_change":"2024-09-20T19:04:20.970137+0000","last_active":"2024-09-20T19:05:33.526901+0000","last_peered":"2024-09-20T19:05:33.526901+0000","last_clean":"2024-09-20T19:05:33.526901+0000","last_became_active":"2024-09-20T19:04:20.897330+0000","last_became_peered":"2024-09-20T19:04:20.897330+0000","last_unstale":"2024-09-20T19:05:33.526901+0000","last_undegraded":"2024-09-20T19:05:33.526901+0000","last_fullsized":"2024-09-20T19:05:33.526901+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:57:50.899514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:50.899514+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":68,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.528073+0000","last_change":"2024-09-20T19:04:20.896732+0000","last_active":"2024-09-20T19:05:33.528073+0000","last_peered":"2024-09-20T19:05:33.528073+0000","last_clean":"2024-09-20T19:05:33.528073+0000","last_became_active":"2024-09-20T19:04:20.896007+0000","last_became_peered":"2024-09-20T19:04:20.896007+0000","last_unstale":"2024-09-20T19:05:33.528073+0000","last_undegraded":"2024-09-20T19:05:33.528073+0000","last_fullsized":"2024-09-20T19:05:33.528073+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:06.310027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:59:06.310027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":727,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.528964+0000","last_change":"2024-09-20T19:02:38.906352+0000","last_active":"2024-09-20T19:05:33.528964+0000","last_peered":"2024-09-20T19:05:33.528964+0000","last_clean":"2024-09-20T19:05:33.528964+0000","last_became_active":"2024-09-20T18:49:49.957067+0000","last_became_peered":"2024-09-20T18:49:49.957067+0000","last_unstale":"2024-09-20T19:05:33.528964+0000","last_undegraded":"2024-09-20T19:05:33.528964+0000","last_fullsized":"2024-09-20T19:05:33.528964+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:38.906161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:38.906161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":707,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.592865+0000","last_change":"2024-09-20T18:57:16.789412+0000","last_active":"2024-09-20T19:05:33.592865+0000","last_peered":"2024-09-20T19:05:33.592865+0000","last_clean":"2024-09-20T19:05:33.592865+0000","last_became_active":"2024-09-20T18:49:51.966915+0000","last_became_peered":"2024-09-20T18:49:51.966915+0000","last_unstale":"2024-09-20T19:05:33.592865+0000","last_undegraded":"2024-09-20T19:05:33.592865+0000","last_fullsized":"2024-09-20T19:05:33.592865+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:16.789095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:16.789095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"273'15","reported_seq":798,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.529116+0000","last_change":"2024-09-20T19:02:30.816061+0000","last_active":"2024-09-20T19:05:33.529116+0000","last_peered":"2024-09-20T19:05:33.529116+0000","last_clean":"2024-09-20T19:05:33.529116+0000","last_became_active":"2024-09-20T18:59:05.085230+0000","last_became_peered":"2024-09-20T18:59:05.085230+0000","last_unstale":"2024-09-20T19:05:33.529116+0000","last_undegraded":"2024-09-20T19:05:33.529116+0000","last_fullsized":"2024-09-20T19:05:33.529116+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:02:30.815747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:30.815747+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":753,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.529923+0000","last_change":"2024-09-20T19:04:20.885127+0000","last_active":"2024-09-20T19:05:33.529923+0000","last_peered":"2024-09-20T19:05:33.529923+0000","last_clean":"2024-09-20T19:05:33.529923+0000","last_became_active":"2024-09-20T19:04:20.884660+0000","last_became_peered":"2024-09-20T19:04:20.884660+0000","last_unstale":"2024-09-20T19:05:33.529923+0000","last_undegraded":"2024-09-20T19:05:33.529923+0000","last_fullsized":"2024-09-20T19:05:33.529923+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:07.408197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:03:07.408197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":735,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.593351+0000","last_change":"2024-09-20T19:04:44.606497+0000","last_active":"2024-09-20T19:05:33.593351+0000","last_peered":"2024-09-20T19:05:33.593351+0000","last_clean":"2024-09-20T19:05:33.593351+0000","last_became_active":"2024-09-20T18:49:49.962523+0000","last_became_peered":"2024-09-20T18:49:49.962523+0000","last_unstale":"2024-09-20T19:05:33.593351+0000","last_undegraded":"2024-09-20T19:05:33.593351+0000","last_fullsized":"2024-09-20T19:05:33.593351+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:44.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:44.606343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:53.560145+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":724,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.523994+0000","last_change":"2024-09-20T19:01:11.847792+0000","last_active":"2024-09-20T19:05:33.523994+0000","last_peered":"2024-09-20T19:05:33.523994+0000","last_clean":"2024-09-20T19:05:33.523994+0000","last_became_active":"2024-09-20T18:49:51.967313+0000","last_became_peered":"2024-09-20T18:49:51.967313+0000","last_unstale":"2024-09-20T19:05:33.523994+0000","last_undegraded":"2024-09-20T19:05:33.523994+0000","last_fullsized":"2024-09-20T19:05:33.523994+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:11.847716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:11.847716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"274'19","reported_seq":800,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.592986+0000","last_change":"2024-09-20T19:01:05.495603+0000","last_active":"2024-09-20T19:05:33.592986+0000","last_peered":"2024-09-20T19:05:33.592986+0000","last_clean":"2024-09-20T19:05:33.592986+0000","last_became_active":"2024-09-20T18:59:05.075390+0000","last_became_peered":"2024-09-20T18:59:05.075390+0000","last_unstale":"2024-09-20T19:05:33.592986+0000","last_undegraded":"2024-09-20T19:05:33.592986+0000","last_fullsized":"2024-09-20T19:05:33.592986+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'19","last_scrub_stamp":"2024-09-20T19:01:05.495478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:05.495478+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:56.633007+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":743,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.589582+0000","last_change":"2024-09-20T19:01:47.521858+0000","last_active":"2024-09-20T19:05:33.589582+0000","last_peered":"2024-09-20T19:05:33.589582+0000","last_clean":"2024-09-20T19:05:33.589582+0000","last_became_active":"2024-09-20T18:49:45.942105+0000","last_became_peered":"2024-09-20T18:49:45.942105+0000","last_unstale":"2024-09-20T19:05:33.589582+0000","last_undegraded":"2024-09-20T19:05:33.589582+0000","last_fullsized":"2024-09-20T19:05:33.589582+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:47.521789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:47.521789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":767,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.528718+0000","last_change":"2024-09-20T19:04:20.912120+0000","last_active":"2024-09-20T19:05:33.528718+0000","last_peered":"2024-09-20T19:05:33.528718+0000","last_clean":"2024-09-20T19:05:33.528718+0000","last_became_active":"2024-09-20T19:04:20.911292+0000","last_became_peered":"2024-09-20T19:04:20.911292+0000","last_unstale":"2024-09-20T19:05:33.528718+0000","last_undegraded":"2024-09-20T19:05:33.528718+0000","last_fullsized":"2024-09-20T19:05:33.528718+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:27.267254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:27.267254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":729,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.593467+0000","last_change":"2024-09-20T19:02:38.950217+0000","last_active":"2024-09-20T19:05:33.593467+0000","last_peered":"2024-09-20T19:05:33.593467+0000","last_clean":"2024-09-20T19:05:33.593467+0000","last_became_active":"2024-09-20T18:59:05.081179+0000","last_became_peered":"2024-09-20T18:59:05.081179+0000","last_unstale":"2024-09-20T19:05:33.593467+0000","last_undegraded":"2024-09-20T19:05:33.593467+0000","last_fullsized":"2024-09-20T19:05:33.593467+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:38.950017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:38.950017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"273'8","reported_seq":777,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.591823+0000","last_change":"2024-09-20T19:04:40.559947+0000","last_active":"2024-09-20T19:05:33.591823+0000","last_peered":"2024-09-20T19:05:33.591823+0000","last_clean":"2024-09-20T19:05:33.591823+0000","last_became_active":"2024-09-20T18:59:05.076365+0000","last_became_peered":"2024-09-20T18:59:05.076365+0000","last_unstale":"2024-09-20T19:05:33.591823+0000","last_undegraded":"2024-09-20T19:05:33.591823+0000","last_fullsized":"2024-09-20T19:05:33.591823+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'8","last_scrub_stamp":"2024-09-20T19:04:40.559877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:04:40.559877+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:53.505030+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":744,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.525504+0000","last_change":"2024-09-20T19:02:39.184249+0000","last_active":"2024-09-20T19:05:33.525504+0000","last_peered":"2024-09-20T19:05:33.525504+0000","last_clean":"2024-09-20T19:05:33.525504+0000","last_became_active":"2024-09-20T18:49:45.940988+0000","last_became_peered":"2024-09-20T18:49:45.940988+0000","last_unstale":"2024-09-20T19:05:33.525504+0000","last_undegraded":"2024-09-20T19:05:33.525504+0000","last_fullsized":"2024-09-20T19:05:33.525504+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:39.184150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:39.184150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":331,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.581357+0000","last_change":"2024-09-20T19:04:20.900567+0000","last_active":"2024-09-20T19:05:33.581357+0000","last_peered":"2024-09-20T19:05:33.581357+0000","last_clean":"2024-09-20T19:05:33.581357+0000","last_became_active":"2024-09-20T19:04:20.900214+0000","last_became_peered":"2024-09-20T19:04:20.900214+0000","last_unstale":"2024-09-20T19:05:33.581357+0000","last_undegraded":"2024-09-20T19:05:33.581357+0000","last_fullsized":"2024-09-20T19:05:33.581357+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:41.497115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:41.497115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:33.282430+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":741,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.528928+0000","last_change":"2024-09-20T19:04:20.886208+0000","last_active":"2024-09-20T19:05:33.528928+0000","last_peered":"2024-09-20T19:05:33.528928+0000","last_clean":"2024-09-20T19:05:33.528928+0000","last_became_active":"2024-09-20T19:04:20.885211+0000","last_became_peered":"2024-09-20T19:04:20.885211+0000","last_unstale":"2024-09-20T19:05:33.528928+0000","last_undegraded":"2024-09-20T19:05:33.528928+0000","last_fullsized":"2024-09-20T19:05:33.528928+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:29.277916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:29.277916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":724,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.529136+0000","last_change":"2024-09-20T19:04:20.890533+0000","last_active":"2024-09-20T19:05:33.529136+0000","last_peered":"2024-09-20T19:05:33.529136+0000","last_clean":"2024-09-20T19:05:33.529136+0000","last_became_active":"2024-09-20T19:04:20.890101+0000","last_became_peered":"2024-09-20T19:04:20.890101+0000","last_unstale":"2024-09-20T19:05:33.529136+0000","last_undegraded":"2024-09-20T19:05:33.529136+0000","last_fullsized":"2024-09-20T19:05:33.529136+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:10.970779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:10.970779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:24.205999+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":727,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.592056+0000","last_change":"2024-09-20T18:58:36.077005+0000","last_active":"2024-09-20T19:05:33.592056+0000","last_peered":"2024-09-20T19:05:33.592056+0000","last_clean":"2024-09-20T19:05:33.592056+0000","last_became_active":"2024-09-20T18:49:45.937992+0000","last_became_peered":"2024-09-20T18:49:45.937992+0000","last_unstale":"2024-09-20T19:05:33.592056+0000","last_undegraded":"2024-09-20T19:05:33.592056+0000","last_fullsized":"2024-09-20T19:05:33.592056+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:36.076937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:36.076937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:34.560802+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"274'19","reported_seq":331,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.582368+0000","last_change":"2024-09-20T19:01:15.544760+0000","last_active":"2024-09-20T19:05:33.582368+0000","last_peered":"2024-09-20T19:05:33.582368+0000","last_clean":"2024-09-20T19:05:33.582368+0000","last_became_active":"2024-09-20T18:59:05.089294+0000","last_became_peered":"2024-09-20T18:59:05.089294+0000","last_unstale":"2024-09-20T19:05:33.582368+0000","last_undegraded":"2024-09-20T19:05:33.582368+0000","last_fullsized":"2024-09-20T19:05:33.582368+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'19","last_scrub_stamp":"2024-09-20T19:01:15.544663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:15.544663+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:24.387863+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":721,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-09-20T19:05:33.525561+0000","last_change":"2024-09-20T19:01:40.961449+0000","last_active":"2024-09-20T19:05:33.525561+0000","last_peered":"2024-09-20T19:05:33.525561+0000","last_clean":"2024-09-20T19:05:33.525561+0000","last_became_active":"2024-09-20T18:49:49.957582+0000","last_became_peered":"2024-09-20T18:49:49.957582+0000","last_unstale":"2024-09-20T19:05:33.525561+0000","last_undegraded":"2024-09-20T19:05:33.525561+0000","last_fullsized":"2024-09-20T19:05:33.525561+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:40.961363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:40.961363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":1596981272,"num_objects":24369,"num_object_clones":0,"num_object_copies":73107,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24369,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":209351,"num_write_kb":7478977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32430,"num_bytes_recovered":1814560792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4999098368,"data_stored":4999086152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91457,"ondisk_log_size":91457,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1185,"internal_metadata":0},"log_size":87,"ondisk_log_size":87,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1114,"num_read_kb":871,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":257,"num_bytes_recovered":4890,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":686,"ondisk_log_size":686,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":3105,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":943,"num_read_kb":938,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266882,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":962660,"kb_used_data":412176,"kb_used_omap":1,"kb_used_meta":550398,"kb_avail":92782492,"statfs":{"total":95995035648,"available":95009271808,"internally_reserved":0,"allocated":422068224,"data_stored":421601109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":563608010},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528521,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1109204,"kb_used_data":694784,"kb_used_omap":1,"kb_used_meta":414334,"kb_avail":92635948,"statfs":{"total":95995035648,"available":94859210752,"internally_reserved":0,"allocated":711458816,"data_stored":711007447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":424278470},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757454,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1088348,"kb_used_data":515724,"kb_used_omap":1,"kb_used_meta":572606,"kb_avail":92656804,"statfs":{"total":95995035648,"available":94880567296,"internally_reserved":0,"allocated":528101376,"data_stored":527637868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":586348997},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":304,"seq":1305670058036,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1112880,"kb_used_data":833888,"kb_used_omap":5,"kb_used_meta":278842,"kb_avail":92632272,"statfs":{"total":95995035648,"available":94855446528,"internally_reserved":0,"allocated":853901312,"data_stored":853418242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5633,"internal_metadata":285534719},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182618,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1210840,"kb_used_data":503560,"kb_used_omap":1,"kb_used_meta":707262,"kb_avail":92534312,"statfs":{"total":95995035648,"available":94755135488,"internally_reserved":0,"allocated":515645440,"data_stored":515185408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":724236743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,2,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411551,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1185168,"kb_used_data":594496,"kb_used_omap":1,"kb_used_meta":590654,"kb_avail":92559984,"statfs":{"total":95995035648,"available":94781423616,"internally_reserved":0,"allocated":608763904,"data_stored":608312905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":604830149},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575077,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1088256,"kb_used_data":471216,"kb_used_omap":1,"kb_used_meta":617022,"kb_avail":92656896,"statfs":{"total":95995035648,"available":94880661504,"internally_reserved":0,"allocated":482525184,"data_stored":482026217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":631830981},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738603,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1640228,"kb_used_data":973140,"kb_used_omap":1,"kb_used_meta":667006,"kb_avail":92104924,"statfs":{"total":95995035648,"available":94315442176,"internally_reserved":0,"allocated":996495360,"data_stored":996023869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":683014597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,1,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1185,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":947851264,"data_stored":947847192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":479068160,"data_stored":479068160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":605618176,"data_stored":605618176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":512032768,"data_stored":512032768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":833556480,"data_stored":833552408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":526647296,"data_stored":526647296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":685772800,"data_stored":685768728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":408551424,"data_stored":408551424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T19:05:41.469 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-09-20T19:05:41.470 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T19:05:41.470 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-09-20T19:05:41.470 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-20T19:05:41.470 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-20T19:05:41.470 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.4 config set filestore_inject_stall 3 2024-09-20T19:05:41.527 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:41.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:41.568 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:41.568 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:41.669 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:41.700 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:41.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:41.701 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:41.743 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:05:41.801 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:05:41.837 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:41.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:41.839 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:05:41.903 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:05:41.924 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:41 smithi083 ceph-mon[29421]: osdmap e405: 8 total, 8 up, 8 in 2024-09-20T19:05:41.924 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:41 smithi083 ceph-mon[29421]: pgmap v797: 148 pgs: 148 active+clean; 1.4 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 671 B/s rd, 0 B/s wr, 837 op/s; 5.8 MiB/s, 92 objects/s recovering 2024-09-20T19:05:41.925 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:41 smithi083 ceph-mon[36260]: osdmap e405: 8 total, 8 up, 8 in 2024-09-20T19:05:41.925 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:41 smithi083 ceph-mon[36260]: pgmap v797: 148 pgs: 148 active+clean; 1.4 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 671 B/s rd, 0 B/s wr, 837 op/s; 5.8 MiB/s, 92 objects/s recovering 2024-09-20T19:05:41.939 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:41.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:41.981 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:41.981 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:42.082 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:42.121 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:42.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:42.121 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:42.195 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:41 smithi142 ceph-mon[32508]: osdmap e405: 8 total, 8 up, 8 in 2024-09-20T19:05:42.195 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:41 smithi142 ceph-mon[32508]: pgmap v797: 148 pgs: 148 active+clean; 1.4 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 671 B/s rd, 0 B/s wr, 837 op/s; 5.8 MiB/s, 92 objects/s recovering 2024-09-20T19:05:42.222 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:42.264 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:42.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:42.266 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:42.366 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:42.398 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:42.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:42.399 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:42.440 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T19:05:42.440 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:05:42.497 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T19:05:42.497 INFO:teuthology.orchestra.run.smithi142.stdout: "success": "" 2024-09-20T19:05:42.497 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T19:05:42.500 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:42.543 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:42.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:42.544 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:42.645 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:42.679 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:42.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:42.680 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:42.780 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:42.812 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:42.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:42.813 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:42.914 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:42.923 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-20T19:05:42.949 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:42.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:42.950 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:43.051 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:43.081 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:43.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:43.082 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:43.183 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:43.195 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:42 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2041694344' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:05:43.195 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:42 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:05:43.195 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:42 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2041694344' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:05:43.195 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:42 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:05:43.219 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:43.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:43.220 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:43.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:42 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2041694344' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:05:43.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:42 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:05:43.321 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:43.361 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:43.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:43.362 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:43.462 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:43.500 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:43.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:43.501 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:43.602 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:43.629 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:43.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:43.630 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:43.645 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-20T19:05:43.731 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:05:43.760 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:43.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:43.761 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:05:43.862 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:43.904 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:05:43.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:43.908 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:43.908 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:44.009 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:44.043 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:44.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:44.043 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:44.144 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:44.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:44.182 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:44.182 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:44.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:43 smithi142 ceph-mon[32508]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:05:44.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:43 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T19:05:44.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:43 smithi142 ceph-mon[32508]: osdmap e406: 8 total, 8 up, 8 in 2024-09-20T19:05:44.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:43 smithi142 ceph-mon[32508]: pgmap v799: 148 pgs: 148 active+clean; 1.1 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.09k op/s 2024-09-20T19:05:44.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:43 smithi142 ceph-mon[32508]: 10.d scrub starts 2024-09-20T19:05:44.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:43 smithi142 ceph-mon[32508]: 10.d scrub ok 2024-09-20T19:05:44.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:43 smithi142 ceph-mon[32508]: 6.b scrub starts 2024-09-20T19:05:44.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:43 smithi142 ceph-mon[32508]: 6.b scrub ok 2024-09-20T19:05:44.283 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:44.294 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:43 smithi083 ceph-mon[29421]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:05:44.294 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:43 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T19:05:44.294 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:43 smithi083 ceph-mon[29421]: osdmap e406: 8 total, 8 up, 8 in 2024-09-20T19:05:44.294 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:43 smithi083 ceph-mon[29421]: pgmap v799: 148 pgs: 148 active+clean; 1.1 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.09k op/s 2024-09-20T19:05:44.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:43 smithi083 ceph-mon[29421]: 10.d scrub starts 2024-09-20T19:05:44.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:43 smithi083 ceph-mon[29421]: 10.d scrub ok 2024-09-20T19:05:44.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:43 smithi083 ceph-mon[29421]: 6.b scrub starts 2024-09-20T19:05:44.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:43 smithi083 ceph-mon[29421]: 6.b scrub ok 2024-09-20T19:05:44.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:43 smithi083 ceph-mon[36260]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:05:44.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:43 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T19:05:44.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:43 smithi083 ceph-mon[36260]: osdmap e406: 8 total, 8 up, 8 in 2024-09-20T19:05:44.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:43 smithi083 ceph-mon[36260]: pgmap v799: 148 pgs: 148 active+clean; 1.1 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.09k op/s 2024-09-20T19:05:44.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:43 smithi083 ceph-mon[36260]: 10.d scrub starts 2024-09-20T19:05:44.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:43 smithi083 ceph-mon[36260]: 10.d scrub ok 2024-09-20T19:05:44.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:43 smithi083 ceph-mon[36260]: 6.b scrub starts 2024-09-20T19:05:44.296 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:43 smithi083 ceph-mon[36260]: 6.b scrub ok 2024-09-20T19:05:44.333 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:44.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:44.334 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:44.435 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:44.477 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:44.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:44.479 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:44.580 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:44.618 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:44.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:44.619 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:44.720 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:44.752 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:44.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:44.753 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:44.853 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:44.900 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:44.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:44.902 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:44.917 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:05:44.995 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:05:44.995 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:05:45.003 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:45.030 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:05:45.030 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:05:45.052 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:45.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:45.053 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:45.063 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:05:45.063 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:05:45.096 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:05:45.097 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:05:45.131 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:05:45.132 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:05:45.154 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:45.165 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:05:45.165 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:05:45.166 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:44 smithi083 ceph-mon[29421]: 4.3 scrub starts 2024-09-20T19:05:45.166 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:44 smithi083 ceph-mon[29421]: 4.3 scrub ok 2024-09-20T19:05:45.166 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:44 smithi083 ceph-mon[29421]: 3.16 scrub starts 2024-09-20T19:05:45.166 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:44 smithi083 ceph-mon[29421]: 3.16 scrub ok 2024-09-20T19:05:45.166 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:44 smithi083 ceph-mon[29421]: 4.7 scrub starts 2024-09-20T19:05:45.166 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:44 smithi083 ceph-mon[29421]: 4.7 scrub ok 2024-09-20T19:05:45.166 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:44 smithi083 ceph-mon[36260]: 4.3 scrub starts 2024-09-20T19:05:45.167 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:44 smithi083 ceph-mon[36260]: 4.3 scrub ok 2024-09-20T19:05:45.167 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:44 smithi083 ceph-mon[36260]: 3.16 scrub starts 2024-09-20T19:05:45.167 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:44 smithi083 ceph-mon[36260]: 3.16 scrub ok 2024-09-20T19:05:45.167 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:44 smithi083 ceph-mon[36260]: 4.7 scrub starts 2024-09-20T19:05:45.167 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:44 smithi083 ceph-mon[36260]: 4.7 scrub ok 2024-09-20T19:05:45.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:45.198 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:45.198 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:45.206 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:05:45.206 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:05:45.240 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:05:45.240 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:05:45.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:44 smithi142 ceph-mon[32508]: 4.3 scrub starts 2024-09-20T19:05:45.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:44 smithi142 ceph-mon[32508]: 4.3 scrub ok 2024-09-20T19:05:45.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:44 smithi142 ceph-mon[32508]: 3.16 scrub starts 2024-09-20T19:05:45.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:44 smithi142 ceph-mon[32508]: 3.16 scrub ok 2024-09-20T19:05:45.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:44 smithi142 ceph-mon[32508]: 4.7 scrub starts 2024-09-20T19:05:45.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:44 smithi142 ceph-mon[32508]: 4.7 scrub ok 2024-09-20T19:05:45.299 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:45.332 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:45.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:45.334 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:45.435 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:45.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:45.470 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:45.470 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:45.571 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:45.614 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:45.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:45.615 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:45.716 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:45.748 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:45.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:45.749 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:45.850 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:45.889 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:45.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:45.890 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:45.912 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T19:05:45.912 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-20T19:05:45.912 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:45.913 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25775.0:220421 10.2 10.323e7c12 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e407)", 2024-09-20T19:05:45.913 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:05:45.904834+0000", 2024-09-20T19:05:45.913 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0017769680000000001, 2024-09-20T19:05:45.913 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0017990759999999999, 2024-09-20T19:05:45.913 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:05:45.913 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:05:45.913 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "op must wait for map", 2024-09-20T19:05:45.913 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:05:45.913 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25775", 2024-09-20T19:05:45.913 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:05:45.913 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 220421 2024-09-20T19:05:45.913 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:45.913 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:05:45.913 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:45.914 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:05:45.914 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:45.904834+0000", 2024-09-20T19:05:45.914 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:05:45.914 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:45.914 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:45.914 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:05:45.914 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:45.904834+0000", 2024-09-20T19:05:45.914 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:05:45.914 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:45.914 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:45.914 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:05:45.914 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:45.904835+0000", 2024-09-20T19:05:45.914 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6479999999999999e-06 2024-09-20T19:05:45.914 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:45.914 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:45.915 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:05:45.915 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:45.904842+0000", 2024-09-20T19:05:45.915 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6510000000000004e-06 2024-09-20T19:05:45.915 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:45.915 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:45.915 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:05:45.915 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:45.904865+0000", 2024-09-20T19:05:45.915 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2673000000000001e-05 2024-09-20T19:05:45.915 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:45.915 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:45.915 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:05:45.915 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:45.904873+0000", 2024-09-20T19:05:45.915 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4549999999999995e-06 2024-09-20T19:05:45.915 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:45.915 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:45.916 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:45.916 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:45.904927+0000", 2024-09-20T19:05:45.916 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3516000000000001e-05 2024-09-20T19:05:45.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:45.916 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:45.916 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op must wait for map", 2024-09-20T19:05:45.916 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:45.904932+0000", 2024-09-20T19:05:45.916 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7610000000000004e-06 2024-09-20T19:05:45.916 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:45.916 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:05:45.916 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:45.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:45.916 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:45.916 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25775.0:220423 10.2 10.c0115cc2 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e407)", 2024-09-20T19:05:45.916 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:05:45.906194+0000", 2024-09-20T19:05:45.917 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.000416301, 2024-09-20T19:05:45.917 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00050753799999999998, 2024-09-20T19:05:45.917 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:05:45.917 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:05:45.917 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "waiting_for_map not empty", 2024-09-20T19:05:45.917 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:05:45.917 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25775", 2024-09-20T19:05:45.917 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:05:45.917 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 220423 2024-09-20T19:05:45.917 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:45.917 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:05:45.917 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:45.917 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:05:45.917 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:45.906194+0000", 2024-09-20T19:05:45.918 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:05:45.918 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:45.918 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:45.918 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:05:45.918 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:45.906194+0000", 2024-09-20T19:05:45.918 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:05:45.918 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:45.918 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:45.918 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:05:45.918 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:45.906195+0000", 2024-09-20T19:05:45.918 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1069999999999999e-06 2024-09-20T19:05:45.918 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:45.919 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:45.919 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:05:45.919 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:45.906201+0000", 2024-09-20T19:05:45.919 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1320000000000002e-06 2024-09-20T19:05:45.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:45.919 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:45.919 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:05:45.919 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:45.906213+0000", 2024-09-20T19:05:45.919 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2782999999999999e-05 2024-09-20T19:05:45.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:45.919 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:45.919 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:05:45.919 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:45.906221+0000", 2024-09-20T19:05:45.920 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7170000000000008e-06 2024-09-20T19:05:45.920 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:45.920 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:45.920 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:45.920 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:45.906269+0000", 2024-09-20T19:05:45.920 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7861999999999997e-05 2024-09-20T19:05:45.920 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:45.920 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:45.920 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting_for_map not empty", 2024-09-20T19:05:45.920 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:45.906275+0000", 2024-09-20T19:05:45.920 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6710000000000004e-06 2024-09-20T19:05:45.920 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:45.920 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:05:45.921 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:45.921 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:45.921 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T19:05:45.921 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 2 2024-09-20T19:05:45.921 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T19:05:45.992 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:46.024 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:46.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:46.025 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:46.042 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:45 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/277925144' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:05:46.042 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:45 smithi083 ceph-mon[29421]: pgmap v800: 148 pgs: 148 active+clean; 1.1 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 776 B/s rd, 0 B/s wr, 902 op/s 2024-09-20T19:05:46.042 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:45 smithi083 ceph-mon[29421]: osdmap e407: 8 total, 8 up, 8 in 2024-09-20T19:05:46.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:45 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/277925144' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:05:46.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:45 smithi083 ceph-mon[36260]: pgmap v800: 148 pgs: 148 active+clean; 1.1 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 776 B/s rd, 0 B/s wr, 902 op/s 2024-09-20T19:05:46.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:45 smithi083 ceph-mon[36260]: osdmap e407: 8 total, 8 up, 8 in 2024-09-20T19:05:46.126 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:46.154 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:46.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:46.155 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:46.256 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:46.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:45 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/277925144' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:05:46.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:45 smithi142 ceph-mon[32508]: pgmap v800: 148 pgs: 148 active+clean; 1.1 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 776 B/s rd, 0 B/s wr, 902 op/s 2024-09-20T19:05:46.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:45 smithi142 ceph-mon[32508]: osdmap e407: 8 total, 8 up, 8 in 2024-09-20T19:05:46.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:46.298 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:46.298 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:46.354 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T19:05:46.398 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:46.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:46.438 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:46.439 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:46.539 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:46.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:46.579 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:46.579 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:46.680 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:46.715 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:46.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:46.716 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:46.816 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:46.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:46.859 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:46.859 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:46.960 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:46.994 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:46.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:46.995 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:47.030 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:46 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:05:47.030 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:46 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:05:47.078 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_blocked_ops 2024-09-20T19:05:47.096 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:47.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:47.176 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:47.176 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:47.239 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:05:47.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:46 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:05:47.278 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:47.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:47.334 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:47.335 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:47.435 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:47.470 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:47.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:47.471 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:47.572 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:47.599 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:47.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:47.600 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:47.701 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:47.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:47.741 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:47.741 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:47.761 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:05:47 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:05:47.842 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:47.863 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:05:47.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:47.877 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:47.877 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:47.978 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:48.004 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:48.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:48.005 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:48.105 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:48.144 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:48.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:48.145 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:48.246 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:48.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:47 smithi142 ceph-mon[32508]: 10.9 scrub starts 2024-09-20T19:05:48.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:47 smithi142 ceph-mon[32508]: 5.c scrub starts 2024-09-20T19:05:48.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:47 smithi142 ceph-mon[32508]: 5.c scrub ok 2024-09-20T19:05:48.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:47 smithi142 ceph-mon[32508]: 10.9 scrub ok 2024-09-20T19:05:48.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:47 smithi142 ceph-mon[32508]: 6.2 scrub starts 2024-09-20T19:05:48.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:47 smithi142 ceph-mon[32508]: 6.2 scrub ok 2024-09-20T19:05:48.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:47 smithi142 ceph-mon[32508]: pgmap v802: 148 pgs: 148 active+clean; 821 MiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1.56k op/s 2024-09-20T19:05:48.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:47 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:05:48.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:48.276 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:48.276 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:48.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:47 smithi083 ceph-mon[29421]: 10.9 scrub starts 2024-09-20T19:05:48.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:47 smithi083 ceph-mon[29421]: 5.c scrub starts 2024-09-20T19:05:48.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:47 smithi083 ceph-mon[29421]: 5.c scrub ok 2024-09-20T19:05:48.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:47 smithi083 ceph-mon[29421]: 10.9 scrub ok 2024-09-20T19:05:48.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:47 smithi083 ceph-mon[29421]: 6.2 scrub starts 2024-09-20T19:05:48.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:47 smithi083 ceph-mon[29421]: 6.2 scrub ok 2024-09-20T19:05:48.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:47 smithi083 ceph-mon[29421]: pgmap v802: 148 pgs: 148 active+clean; 821 MiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1.56k op/s 2024-09-20T19:05:48.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:47 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:05:48.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:47 smithi083 ceph-mon[36260]: 10.9 scrub starts 2024-09-20T19:05:48.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:47 smithi083 ceph-mon[36260]: 5.c scrub starts 2024-09-20T19:05:48.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:47 smithi083 ceph-mon[36260]: 5.c scrub ok 2024-09-20T19:05:48.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:47 smithi083 ceph-mon[36260]: 10.9 scrub ok 2024-09-20T19:05:48.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:47 smithi083 ceph-mon[36260]: 6.2 scrub starts 2024-09-20T19:05:48.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:47 smithi083 ceph-mon[36260]: 6.2 scrub ok 2024-09-20T19:05:48.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:47 smithi083 ceph-mon[36260]: pgmap v802: 148 pgs: 148 active+clean; 821 MiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1.56k op/s 2024-09-20T19:05:48.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:47 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:05:48.318 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 7, 6, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 5, 6, 7, 4] 2024-09-20T19:05:48.318 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T19:05:48.319 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_3 2024-09-20T19:05:48.319 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T19:05:48.365 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T19:05:48.376 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:48.404 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:48.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:48.405 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:48.506 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:48.528 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:48.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:48.529 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:48.630 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:48.669 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:48.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:48.670 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:48.771 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:05:48.810 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:48.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:48.811 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:05:48.912 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:48.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:48.951 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:48.951 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:48.986 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-20T19:05:49.052 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:49.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:49.098 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:49.098 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:49.134 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:48 smithi083 ceph-mon[29421]: 3.6 scrub starts 2024-09-20T19:05:49.134 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:48 smithi083 ceph-mon[29421]: 3.6 scrub ok 2024-09-20T19:05:49.134 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:48 smithi083 ceph-mon[29421]: 10.6 scrub starts 2024-09-20T19:05:49.134 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:48 smithi083 ceph-mon[29421]: 10.c scrub starts 2024-09-20T19:05:49.134 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:48 smithi083 ceph-mon[29421]: 3.4 scrub starts 2024-09-20T19:05:49.134 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:48 smithi083 ceph-mon[29421]: 3.4 scrub ok 2024-09-20T19:05:49.134 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:48 smithi083 ceph-mon[29421]: 10.6 scrub ok 2024-09-20T19:05:49.134 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:48 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2939710471' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:05:49.134 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:48 smithi083 ceph-mon[29421]: 5.2 scrub starts 2024-09-20T19:05:49.134 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:48 smithi083 ceph-mon[29421]: 5.2 scrub ok 2024-09-20T19:05:49.135 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:48 smithi083 ceph-mon[29421]: 3.b scrub starts 2024-09-20T19:05:49.135 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:48 smithi083 ceph-mon[29421]: 3.b scrub ok 2024-09-20T19:05:49.135 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:48 smithi083 ceph-mon[36260]: 3.6 scrub starts 2024-09-20T19:05:49.135 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:48 smithi083 ceph-mon[36260]: 3.6 scrub ok 2024-09-20T19:05:49.135 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:48 smithi083 ceph-mon[36260]: 10.6 scrub starts 2024-09-20T19:05:49.135 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:48 smithi083 ceph-mon[36260]: 10.c scrub starts 2024-09-20T19:05:49.135 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:48 smithi083 ceph-mon[36260]: 3.4 scrub starts 2024-09-20T19:05:49.135 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:48 smithi083 ceph-mon[36260]: 3.4 scrub ok 2024-09-20T19:05:49.135 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:48 smithi083 ceph-mon[36260]: 10.6 scrub ok 2024-09-20T19:05:49.135 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:48 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2939710471' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:05:49.136 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:48 smithi083 ceph-mon[36260]: 5.2 scrub starts 2024-09-20T19:05:49.136 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:48 smithi083 ceph-mon[36260]: 5.2 scrub ok 2024-09-20T19:05:49.136 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:48 smithi083 ceph-mon[36260]: 3.b scrub starts 2024-09-20T19:05:49.136 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:48 smithi083 ceph-mon[36260]: 3.b scrub ok 2024-09-20T19:05:49.199 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:49.231 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:49.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:49.232 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:49.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:48 smithi142 ceph-mon[32508]: 3.6 scrub starts 2024-09-20T19:05:49.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:48 smithi142 ceph-mon[32508]: 3.6 scrub ok 2024-09-20T19:05:49.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:48 smithi142 ceph-mon[32508]: 10.6 scrub starts 2024-09-20T19:05:49.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:48 smithi142 ceph-mon[32508]: 10.c scrub starts 2024-09-20T19:05:49.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:48 smithi142 ceph-mon[32508]: 3.4 scrub starts 2024-09-20T19:05:49.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:48 smithi142 ceph-mon[32508]: 3.4 scrub ok 2024-09-20T19:05:49.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:48 smithi142 ceph-mon[32508]: 10.6 scrub ok 2024-09-20T19:05:49.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:48 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2939710471' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:05:49.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:48 smithi142 ceph-mon[32508]: 5.2 scrub starts 2024-09-20T19:05:49.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:48 smithi142 ceph-mon[32508]: 5.2 scrub ok 2024-09-20T19:05:49.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:48 smithi142 ceph-mon[32508]: 3.b scrub starts 2024-09-20T19:05:49.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:48 smithi142 ceph-mon[32508]: 3.b scrub ok 2024-09-20T19:05:49.294 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T19:05:49.294 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-20T19:05:49.294 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-20T19:05:49.294 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-20T19:05:49.294 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T19:05:49.333 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:49.370 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:49.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:49.371 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:49.472 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:49.510 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:49.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:49.511 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:49.612 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:49.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:49.647 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:49.647 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:49.748 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:49.785 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:49.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:49.786 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:49.886 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:05:49.922 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:49.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:49.923 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:05:50.024 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:50.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:50.061 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:50.061 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:50.162 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:50.201 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:50.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:50.203 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:50.219 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:49 smithi083 ceph-mon[29421]: 10.c scrub ok 2024-09-20T19:05:50.219 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:49 smithi083 ceph-mon[29421]: 4.2 scrub starts 2024-09-20T19:05:50.219 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:49 smithi083 ceph-mon[29421]: 4.2 scrub ok 2024-09-20T19:05:50.219 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:49 smithi083 ceph-mon[29421]: 4.4 scrub starts 2024-09-20T19:05:50.219 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:49 smithi083 ceph-mon[29421]: 4.4 scrub ok 2024-09-20T19:05:50.219 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:49 smithi083 ceph-mon[29421]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T19:05:50.219 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:49 smithi083 ceph-mon[29421]: Cluster is now healthy 2024-09-20T19:05:50.219 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:49 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2939710471' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:05:50.219 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:49 smithi083 ceph-mon[29421]: osdmap e408: 8 total, 8 up, 8 in 2024-09-20T19:05:50.219 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:49 smithi083 ceph-mon[29421]: pgmap v804: 148 pgs: 148 active+clean; 774 MiB data, 6.7 GiB used, 708 GiB / 715 GiB avail; 820 B/s rd, 0 B/s wr, 968 op/s 2024-09-20T19:05:50.219 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:49 smithi083 ceph-mon[29421]: 4.1c scrub starts 2024-09-20T19:05:50.220 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:49 smithi083 ceph-mon[29421]: 4.1c scrub ok 2024-09-20T19:05:50.220 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:49 smithi083 ceph-mon[36260]: 10.c scrub ok 2024-09-20T19:05:50.220 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:49 smithi083 ceph-mon[36260]: 4.2 scrub starts 2024-09-20T19:05:50.220 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:49 smithi083 ceph-mon[36260]: 4.2 scrub ok 2024-09-20T19:05:50.220 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:49 smithi083 ceph-mon[36260]: 4.4 scrub starts 2024-09-20T19:05:50.220 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:49 smithi083 ceph-mon[36260]: 4.4 scrub ok 2024-09-20T19:05:50.220 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:49 smithi083 ceph-mon[36260]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T19:05:50.220 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:49 smithi083 ceph-mon[36260]: Cluster is now healthy 2024-09-20T19:05:50.220 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:49 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2939710471' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:05:50.220 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:49 smithi083 ceph-mon[36260]: osdmap e408: 8 total, 8 up, 8 in 2024-09-20T19:05:50.220 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:49 smithi083 ceph-mon[36260]: pgmap v804: 148 pgs: 148 active+clean; 774 MiB data, 6.7 GiB used, 708 GiB / 715 GiB avail; 820 B/s rd, 0 B/s wr, 968 op/s 2024-09-20T19:05:50.220 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:49 smithi083 ceph-mon[36260]: 4.1c scrub starts 2024-09-20T19:05:50.220 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:49 smithi083 ceph-mon[36260]: 4.1c scrub ok 2024-09-20T19:05:50.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:49 smithi142 ceph-mon[32508]: 10.c scrub ok 2024-09-20T19:05:50.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:49 smithi142 ceph-mon[32508]: 4.2 scrub starts 2024-09-20T19:05:50.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:49 smithi142 ceph-mon[32508]: 4.2 scrub ok 2024-09-20T19:05:50.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:49 smithi142 ceph-mon[32508]: 4.4 scrub starts 2024-09-20T19:05:50.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:49 smithi142 ceph-mon[32508]: 4.4 scrub ok 2024-09-20T19:05:50.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:49 smithi142 ceph-mon[32508]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T19:05:50.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:49 smithi142 ceph-mon[32508]: Cluster is now healthy 2024-09-20T19:05:50.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:49 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2939710471' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:05:50.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:49 smithi142 ceph-mon[32508]: osdmap e408: 8 total, 8 up, 8 in 2024-09-20T19:05:50.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:49 smithi142 ceph-mon[32508]: pgmap v804: 148 pgs: 148 active+clean; 774 MiB data, 6.7 GiB used, 708 GiB / 715 GiB avail; 820 B/s rd, 0 B/s wr, 968 op/s 2024-09-20T19:05:50.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:49 smithi142 ceph-mon[32508]: 4.1c scrub starts 2024-09-20T19:05:50.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:49 smithi142 ceph-mon[32508]: 4.1c scrub ok 2024-09-20T19:05:50.271 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_historic_ops 2024-09-20T19:05:50.304 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:50.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:50.337 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:50.337 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:50.438 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:50.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:50.500 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:50.501 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:50.602 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:50.648 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:50.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:50.649 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:50.750 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:50.808 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:50.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:50.809 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:50.841 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:05:50.907 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:05:50.910 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:50.953 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:05:50.971 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:50.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:50.973 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:50.991 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:05:50 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:05:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:05:51.074 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:51.114 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:51.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:51.115 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:51.216 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:51.246 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:51.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:51.247 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:51.262 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 19:05:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=sqlstore.transactions t=2024-09-20T19:05:51.185836418Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-20T19:05:51.262 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 19:05:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=sqlstore.transactions t=2024-09-20T19:05:51.196455831Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=1 code="database is locked" 2024-09-20T19:05:51.262 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 19:05:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=sqlstore.transactions t=2024-09-20T19:05:51.207697605Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=2 code="database is locked" 2024-09-20T19:05:51.262 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 19:05:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=sqlstore.transactions t=2024-09-20T19:05:51.2187958Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=3 code="database is locked" 2024-09-20T19:05:51.262 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 19:05:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=sqlstore.transactions t=2024-09-20T19:05:51.230389414Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=4 code="database is locked" 2024-09-20T19:05:51.262 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 19:05:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=ngalert.notifier.alertmanager org=1 t=2024-09-20T19:05:51.230989955Z level=error msg="Running maintenance failed" err="database is locked" 2024-09-20T19:05:51.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:50 smithi142 ceph-mon[32508]: 3.d scrub starts 2024-09-20T19:05:51.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:50 smithi142 ceph-mon[32508]: 3.d scrub ok 2024-09-20T19:05:51.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:50 smithi142 ceph-mon[32508]: 4.8 scrub starts 2024-09-20T19:05:51.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:50 smithi142 ceph-mon[32508]: 4.8 scrub ok 2024-09-20T19:05:51.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:50 smithi142 ceph-mon[32508]: 4.e scrub starts 2024-09-20T19:05:51.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:50 smithi142 ceph-mon[32508]: 4.e scrub ok 2024-09-20T19:05:51.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:50 smithi142 ceph-mon[32508]: 10.f scrub starts 2024-09-20T19:05:51.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:50 smithi142 ceph-mon[32508]: 10.f scrub ok 2024-09-20T19:05:51.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:50 smithi142 ceph-mon[32508]: 4.19 scrub starts 2024-09-20T19:05:51.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:50 smithi142 ceph-mon[32508]: 4.19 scrub ok 2024-09-20T19:05:51.263 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:50 smithi142 ceph-mon[32508]: osdmap e409: 8 total, 8 up, 8 in 2024-09-20T19:05:51.288 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:50 smithi083 ceph-mon[29421]: 3.d scrub starts 2024-09-20T19:05:51.289 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:50 smithi083 ceph-mon[29421]: 3.d scrub ok 2024-09-20T19:05:51.289 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:50 smithi083 ceph-mon[29421]: 4.8 scrub starts 2024-09-20T19:05:51.289 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:50 smithi083 ceph-mon[29421]: 4.8 scrub ok 2024-09-20T19:05:51.289 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:50 smithi083 ceph-mon[29421]: 4.e scrub starts 2024-09-20T19:05:51.289 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:50 smithi083 ceph-mon[29421]: 4.e scrub ok 2024-09-20T19:05:51.289 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:50 smithi083 ceph-mon[29421]: 10.f scrub starts 2024-09-20T19:05:51.289 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:50 smithi083 ceph-mon[29421]: 10.f scrub ok 2024-09-20T19:05:51.289 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:50 smithi083 ceph-mon[29421]: 4.19 scrub starts 2024-09-20T19:05:51.289 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:50 smithi083 ceph-mon[29421]: 4.19 scrub ok 2024-09-20T19:05:51.289 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:50 smithi083 ceph-mon[29421]: osdmap e409: 8 total, 8 up, 8 in 2024-09-20T19:05:51.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:50 smithi083 ceph-mon[36260]: 3.d scrub starts 2024-09-20T19:05:51.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:50 smithi083 ceph-mon[36260]: 3.d scrub ok 2024-09-20T19:05:51.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:50 smithi083 ceph-mon[36260]: 4.8 scrub starts 2024-09-20T19:05:51.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:50 smithi083 ceph-mon[36260]: 4.8 scrub ok 2024-09-20T19:05:51.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:50 smithi083 ceph-mon[36260]: 4.e scrub starts 2024-09-20T19:05:51.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:50 smithi083 ceph-mon[36260]: 4.e scrub ok 2024-09-20T19:05:51.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:50 smithi083 ceph-mon[36260]: 10.f scrub starts 2024-09-20T19:05:51.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:50 smithi083 ceph-mon[36260]: 10.f scrub ok 2024-09-20T19:05:51.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:50 smithi083 ceph-mon[36260]: 4.19 scrub starts 2024-09-20T19:05:51.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:50 smithi083 ceph-mon[36260]: 4.19 scrub ok 2024-09-20T19:05:51.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:50 smithi083 ceph-mon[36260]: osdmap e409: 8 total, 8 up, 8 in 2024-09-20T19:05:51.348 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:51.381 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:51.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:51.382 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:51.482 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:51.510 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:51.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:51.511 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:51.611 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:51.643 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:51.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:51.644 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:51.745 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:51.782 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:51.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:51.783 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:51.884 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:05:51.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:51.913 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:51.914 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:05:51.938 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T19:05:51.938 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-20T19:05:51.939 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-20T19:05:51.939 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-20T19:05:51.939 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.939 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157429 8.7 8:ee8a5b93:::benchmark_data_smithi083_97422_object41338:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e212)", 2024-09-20T19:05:51.939 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.048178+0000", 2024-09-20T19:05:51.939 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 532.88047350199997, 2024-09-20T19:05:51.939 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.034527355, 2024-09-20T19:05:51.939 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:05:51.939 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:05:51.939 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-20T19:05:51.939 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:05:51.939 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T19:05:51.939 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T19:05:51.940 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157429 2024-09-20T19:05:51.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.940 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:05:51.940 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.940 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:05:51.940 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.048178+0000", 2024-09-20T19:05:51.940 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:05:51.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.940 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.940 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:05:51.940 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.048178+0000", 2024-09-20T19:05:51.940 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:05:51.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.940 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.941 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:05:51.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.048180+0000", 2024-09-20T19:05:51.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.618e-06 2024-09-20T19:05:51.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.941 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.941 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:05:51.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.048187+0000", 2024-09-20T19:05:51.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8240000000000002e-06 2024-09-20T19:05:51.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.941 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.941 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:05:51.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.048205+0000", 2024-09-20T19:05:51.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8717000000000001e-05 2024-09-20T19:05:51.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.942 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.942 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:05:51.942 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.048215+0000", 2024-09-20T19:05:51.942 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.022e-05 2024-09-20T19:05:51.942 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.942 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.942 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:51.942 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.048294+0000", 2024-09-20T19:05:51.942 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8088000000000002e-05 2024-09-20T19:05:51.942 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.942 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.942 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:05:51.942 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.048409+0000", 2024-09-20T19:05:51.943 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115146 2024-09-20T19:05:51.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.943 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.943 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3,6", 2024-09-20T19:05:51.943 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.048517+0000", 2024-09-20T19:05:51.943 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010874099999999999 2024-09-20T19:05:51.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.943 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.943 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:05:51.943 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.059282+0000", 2024-09-20T19:05:51.943 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010764078 2024-09-20T19:05:51.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.943 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.943 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:05:51.944 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.059297+0000", 2024-09-20T19:05:51.944 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5773e-05 2024-09-20T19:05:51.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.944 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.944 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:05:51.944 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.059343+0000", 2024-09-20T19:05:51.944 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5816000000000003e-05 2024-09-20T19:05:51.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.944 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.944 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:51.944 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.070249+0000", 2024-09-20T19:05:51.944 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010906381999999999 2024-09-20T19:05:51.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.944 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.945 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:05:51.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.070260+0000", 2024-09-20T19:05:51.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0404000000000001e-05 2024-09-20T19:05:51.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.945 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:51.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.082592+0000", 2024-09-20T19:05:51.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0123322640000001 2024-09-20T19:05:51.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.945 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:05:51.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.082705+0000", 2024-09-20T19:05:51.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011328399999999999 2024-09-20T19:05:51.946 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:51.946 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:05:51.946 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:51.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.946 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157435 8.7 8:e8101dbf:::benchmark_data_smithi083_97422_object41344:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e212)", 2024-09-20T19:05:51.946 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.053885+0000", 2024-09-20T19:05:51.946 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 532.87476627199999, 2024-09-20T19:05:51.946 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0294991419999999, 2024-09-20T19:05:51.946 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:05:51.946 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:05:51.946 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-20T19:05:51.946 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:05:51.947 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T19:05:51.947 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T19:05:51.947 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157435 2024-09-20T19:05:51.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.947 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:05:51.947 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.947 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:05:51.947 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053885+0000", 2024-09-20T19:05:51.947 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:05:51.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.947 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.947 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:05:51.947 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053885+0000", 2024-09-20T19:05:51.947 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:05:51.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.948 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:05:51.948 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053887+0000", 2024-09-20T19:05:51.948 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.68e-06 2024-09-20T19:05:51.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.948 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:05:51.948 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053894+0000", 2024-09-20T19:05:51.948 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5939999999999999e-06 2024-09-20T19:05:51.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.948 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:05:51.948 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053907+0000", 2024-09-20T19:05:51.949 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3599e-05 2024-09-20T19:05:51.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.949 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:05:51.949 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053916+0000", 2024-09-20T19:05:51.949 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2029999999999998e-06 2024-09-20T19:05:51.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.949 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:51.949 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.054004+0000", 2024-09-20T19:05:51.949 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8026000000000001e-05 2024-09-20T19:05:51.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.949 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:05:51.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.054123+0000", 2024-09-20T19:05:51.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000119046 2024-09-20T19:05:51.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3,6", 2024-09-20T19:05:51.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.056711+0000", 2024-09-20T19:05:51.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0025878049999999999 2024-09-20T19:05:51.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:05:51.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.059354+0000", 2024-09-20T19:05:51.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0026431969999999999 2024-09-20T19:05:51.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.951 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.951 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:51.951 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.070287+0000", 2024-09-20T19:05:51.951 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010932261 2024-09-20T19:05:51.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.951 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.951 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:05:51.951 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.070294+0000", 2024-09-20T19:05:51.951 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0029999999999997e-06 2024-09-20T19:05:51.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.951 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.952 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:51.952 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.083330+0000", 2024-09-20T19:05:51.952 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.013036592 2024-09-20T19:05:51.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.952 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.952 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:05:51.952 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.083384+0000", 2024-09-20T19:05:51.952 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4135999999999998e-05 2024-09-20T19:05:51.952 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:51.952 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:05:51.952 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:51.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.952 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.953 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157436 8.7 8:e2843ea0:::benchmark_data_smithi083_97422_object41345:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e212)", 2024-09-20T19:05:51.953 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.053939+0000", 2024-09-20T19:05:51.953 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 532.87471274100005, 2024-09-20T19:05:51.953 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.029517123, 2024-09-20T19:05:51.953 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:05:51.953 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:05:51.953 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-20T19:05:51.953 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:05:51.953 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T19:05:51.953 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T19:05:51.953 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157436 2024-09-20T19:05:51.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.953 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:05:51.953 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:05:51.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053939+0000", 2024-09-20T19:05:51.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:05:51.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:05:51.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053939+0000", 2024-09-20T19:05:51.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:05:51.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:05:51.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053939+0000", 2024-09-20T19:05:51.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9100000000000004e-07 2024-09-20T19:05:51.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:05:51.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053942+0000", 2024-09-20T19:05:51.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7980000000000001e-06 2024-09-20T19:05:51.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:05:51.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053949+0000", 2024-09-20T19:05:51.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8580000000000002e-06 2024-09-20T19:05:51.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:05:51.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.053955+0000", 2024-09-20T19:05:51.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1449999999999996e-06 2024-09-20T19:05:51.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.956 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:51.956 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.060057+0000", 2024-09-20T19:05:51.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0061021729999999998 2024-09-20T19:05:51.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.956 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:05:51.956 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.060177+0000", 2024-09-20T19:05:51.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000120013 2024-09-20T19:05:51.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.956 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3,6", 2024-09-20T19:05:51.957 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.060294+0000", 2024-09-20T19:05:51.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000117056 2024-09-20T19:05:51.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.957 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.957 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:51.957 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.070317+0000", 2024-09-20T19:05:51.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010022778 2024-09-20T19:05:51.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.957 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.957 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:05:51.957 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.070324+0000", 2024-09-20T19:05:51.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4300000000000003e-06 2024-09-20T19:05:51.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.957 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.958 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:51.958 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.083413+0000", 2024-09-20T19:05:51.958 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0130890960000001 2024-09-20T19:05:51.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.958 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:05:51.958 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.083456+0000", 2024-09-20T19:05:51.958 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3185000000000001e-05 2024-09-20T19:05:51.958 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:51.958 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:05:51.958 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:51.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.958 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157449 8.7 8:e0d43d9e:::benchmark_data_smithi083_97422_object41358:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e213)", 2024-09-20T19:05:51.959 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.075612+0000", 2024-09-20T19:05:51.959 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 532.85303989199997, 2024-09-20T19:05:51.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.016308008, 2024-09-20T19:05:51.959 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:05:51.959 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:05:51.959 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:05:51.959 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:05:51.959 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T19:05:51.959 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T19:05:51.959 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157449 2024-09-20T19:05:51.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.959 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:05:51.959 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.959 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:05:51.960 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.075612+0000", 2024-09-20T19:05:51.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:05:51.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.960 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:05:51.960 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.075612+0000", 2024-09-20T19:05:51.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:05:51.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.960 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:05:51.960 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.075613+0000", 2024-09-20T19:05:51.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1570000000000001e-06 2024-09-20T19:05:51.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.961 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:05:51.961 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.075618+0000", 2024-09-20T19:05:51.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7899999999999999e-06 2024-09-20T19:05:51.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.961 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:05:51.961 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.075627+0000", 2024-09-20T19:05:51.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2569999999999996e-06 2024-09-20T19:05:51.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.961 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:05:51.961 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.075662+0000", 2024-09-20T19:05:51.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5439999999999999e-05 2024-09-20T19:05:51.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.961 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:51.962 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.078479+0000", 2024-09-20T19:05:51.962 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0028168910000000002 2024-09-20T19:05:51.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.962 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.962 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:05:51.968 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.078488+0000", 2024-09-20T19:05:51.968 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4069999999999993e-06 2024-09-20T19:05:51.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.968 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.968 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:51.969 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.083476+0000", 2024-09-20T19:05:51.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.004987893 2024-09-20T19:05:51.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.969 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:05:51.969 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.083567+0000", 2024-09-20T19:05:51.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1598e-05 2024-09-20T19:05:51.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.969 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3", 2024-09-20T19:05:51.969 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.083675+0000", 2024-09-20T19:05:51.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000107701 2024-09-20T19:05:51.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.970 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:05:51.970 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.091482+0000", 2024-09-20T19:05:51.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0078073639999999998 2024-09-20T19:05:51.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.970 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:05:51.970 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.091668+0000", 2024-09-20T19:05:51.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018606100000000001 2024-09-20T19:05:51.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.970 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:05:51.970 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.091718+0000", 2024-09-20T19:05:51.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9830000000000002e-05 2024-09-20T19:05:51.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.971 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.971 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:05:51.971 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.091878+0000", 2024-09-20T19:05:51.971 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000160166 2024-09-20T19:05:51.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.971 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.971 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:05:51.971 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.091920+0000", 2024-09-20T19:05:51.971 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1452999999999998e-05 2024-09-20T19:05:51.971 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:51.971 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:05:51.971 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:51.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.972 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.972 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157451 8.7 8:ea555b5d:::benchmark_data_smithi083_97422_object41360:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e213)", 2024-09-20T19:05:51.972 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.080725+0000", 2024-09-20T19:05:51.972 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 532.84792692500002, 2024-09-20T19:05:51.972 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0181950740000001, 2024-09-20T19:05:51.972 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:05:51.972 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:05:51.972 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:05:51.972 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:05:51.972 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T19:05:51.972 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T19:05:51.972 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157451 2024-09-20T19:05:51.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.972 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:05:51.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.973 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:05:51.973 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.080725+0000", 2024-09-20T19:05:51.973 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:05:51.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.974 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:05:51.974 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.080725+0000", 2024-09-20T19:05:51.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:05:51.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.974 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:05:51.974 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.080726+0000", 2024-09-20T19:05:51.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3459999999999999e-06 2024-09-20T19:05:51.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.974 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:05:51.975 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.080731+0000", 2024-09-20T19:05:51.975 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3619999999999997e-06 2024-09-20T19:05:51.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.975 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.975 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:05:51.975 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.080742+0000", 2024-09-20T19:05:51.975 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0302000000000001e-05 2024-09-20T19:05:51.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.975 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.975 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:05:51.975 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.080757+0000", 2024-09-20T19:05:51.975 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4955e-05 2024-09-20T19:05:51.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.976 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:51.976 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.080819+0000", 2024-09-20T19:05:51.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2310999999999993e-05 2024-09-20T19:05:51.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.976 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:05:51.976 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.080826+0000", 2024-09-20T19:05:51.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1069999999999999e-06 2024-09-20T19:05:51.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.976 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:51.976 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.085855+0000", 2024-09-20T19:05:51.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0050288679999999 2024-09-20T19:05:51.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:05:51.977 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.085947+0000", 2024-09-20T19:05:51.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1911999999999997e-05 2024-09-20T19:05:51.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3", 2024-09-20T19:05:51.977 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.086042+0000", 2024-09-20T19:05:51.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5234000000000003e-05 2024-09-20T19:05:51.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:05:51.977 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.091673+0000", 2024-09-20T19:05:51.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0056306079999999996 2024-09-20T19:05:51.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.978 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:05:51.978 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.098793+0000", 2024-09-20T19:05:51.978 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.007120637 2024-09-20T19:05:51.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.978 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:05:51.978 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.098828+0000", 2024-09-20T19:05:51.978 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4405e-05 2024-09-20T19:05:51.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.978 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:05:51.979 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.098882+0000", 2024-09-20T19:05:51.979 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4481000000000002e-05 2024-09-20T19:05:51.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.979 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.979 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:05:51.979 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.098920+0000", 2024-09-20T19:05:51.979 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7546000000000003e-05 2024-09-20T19:05:51.979 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:51.979 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:05:51.979 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:51.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.979 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.979 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157481 8.7 8:ebfa0664:::benchmark_data_smithi083_97422_object41390:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e213)", 2024-09-20T19:05:51.979 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.116650+0000", 2024-09-20T19:05:51.980 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 532.81200141199997, 2024-09-20T19:05:51.980 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.98244276600000002, 2024-09-20T19:05:51.980 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:05:51.980 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:05:51.980 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:05:51.980 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:05:51.980 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T19:05:51.980 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T19:05:51.980 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157481 2024-09-20T19:05:51.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.980 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:05:51.980 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.980 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:05:51.980 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.116650+0000", 2024-09-20T19:05:51.981 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:05:51.981 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.981 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.981 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:05:51.981 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.116650+0000", 2024-09-20T19:05:51.981 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:05:51.981 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.981 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.981 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:05:51.981 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.116652+0000", 2024-09-20T19:05:51.981 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5680000000000001e-06 2024-09-20T19:05:51.981 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.981 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.981 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:05:51.982 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.116658+0000", 2024-09-20T19:05:51.982 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4309999999999999e-06 2024-09-20T19:05:51.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.982 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.982 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:05:51.982 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.116671+0000", 2024-09-20T19:05:51.982 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2384e-05 2024-09-20T19:05:51.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.982 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.982 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:05:51.982 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.116678+0000", 2024-09-20T19:05:51.982 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8690000000000004e-06 2024-09-20T19:05:51.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.982 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.982 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:51.983 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.116734+0000", 2024-09-20T19:05:51.983 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6041000000000001e-05 2024-09-20T19:05:51.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.983 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.983 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:05:51.983 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.116742+0000", 2024-09-20T19:05:51.983 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8189999999999992e-06 2024-09-20T19:05:51.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.983 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.983 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:51.983 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.088569+0000", 2024-09-20T19:05:51.983 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97182719100000003 2024-09-20T19:05:51.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.983 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.984 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:05:51.984 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.088666+0000", 2024-09-20T19:05:51.984 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6897000000000004e-05 2024-09-20T19:05:51.984 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.984 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.984 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3", 2024-09-20T19:05:51.984 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.088770+0000", 2024-09-20T19:05:51.984 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010320299999999999 2024-09-20T19:05:51.984 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.984 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.984 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:05:51.984 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.091677+0000", 2024-09-20T19:05:51.984 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0029076900000000001 2024-09-20T19:05:51.984 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.985 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.985 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:05:51.985 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.098961+0000", 2024-09-20T19:05:51.985 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0072835080000000002 2024-09-20T19:05:51.985 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.985 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.985 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:05:51.985 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.099000+0000", 2024-09-20T19:05:51.985 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.892e-05 2024-09-20T19:05:51.985 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.985 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.985 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:05:51.985 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.099055+0000", 2024-09-20T19:05:51.985 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5554999999999999e-05 2024-09-20T19:05:51.986 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.986 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.986 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:05:51.986 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.099093+0000", 2024-09-20T19:05:51.986 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.769e-05 2024-09-20T19:05:51.986 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:51.986 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:05:51.986 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:51.986 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.986 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.986 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157496 8.7 8:efb55f87:::benchmark_data_smithi083_97422_object41405:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e213)", 2024-09-20T19:05:51.986 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.130402+0000", 2024-09-20T19:05:51.986 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 532.79824969399999, 2024-09-20T19:05:51.986 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.99286820799999997, 2024-09-20T19:05:51.987 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:05:51.987 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:05:51.987 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:05:51.987 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:05:51.987 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T19:05:51.987 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T19:05:51.987 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157496 2024-09-20T19:05:51.987 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.987 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:05:51.987 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.987 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:05:51.987 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.130402+0000", 2024-09-20T19:05:51.987 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:05:51.988 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.988 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.988 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:05:51.988 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.130402+0000", 2024-09-20T19:05:51.988 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:05:51.988 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.988 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.988 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:05:51.988 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.130404+0000", 2024-09-20T19:05:51.988 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.748e-06 2024-09-20T19:05:51.988 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.988 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.988 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:05:51.988 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.130410+0000", 2024-09-20T19:05:51.989 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2720000000000003e-06 2024-09-20T19:05:51.989 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.989 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.989 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:05:51.989 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.130426+0000", 2024-09-20T19:05:51.989 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6557000000000001e-05 2024-09-20T19:05:51.989 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.989 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.989 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:05:51.989 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.130436+0000", 2024-09-20T19:05:51.989 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4320000000000005e-06 2024-09-20T19:05:51.989 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.989 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.989 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:51.990 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.130499+0000", 2024-09-20T19:05:51.990 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3149000000000001e-05 2024-09-20T19:05:51.990 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.990 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.990 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:05:51.990 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.130508+0000", 2024-09-20T19:05:51.990 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1619999999999997e-06 2024-09-20T19:05:51.990 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.990 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.990 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:51.990 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.091952+0000", 2024-09-20T19:05:51.990 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.96144401999999995 2024-09-20T19:05:51.990 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.990 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.990 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:05:51.991 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.092057+0000", 2024-09-20T19:05:51.991 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000104268 2024-09-20T19:05:51.991 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.991 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.991 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3", 2024-09-20T19:05:51.991 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.092163+0000", 2024-09-20T19:05:51.991 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106545 2024-09-20T19:05:51.991 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.991 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.991 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:05:51.991 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.099133+0000", 2024-09-20T19:05:51.991 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0069702990000000001 2024-09-20T19:05:51.991 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.992 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.992 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:05:51.992 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.104449+0000", 2024-09-20T19:05:51.992 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0053152920000000001 2024-09-20T19:05:51.992 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.992 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.992 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:05:51.992 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.123091+0000", 2024-09-20T19:05:51.992 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.018642757999999999 2024-09-20T19:05:51.992 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.992 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.992 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:05:51.992 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.123225+0000", 2024-09-20T19:05:51.992 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000133881 2024-09-20T19:05:51.993 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.993 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.993 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:05:51.993 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.123270+0000", 2024-09-20T19:05:51.993 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4824999999999999e-05 2024-09-20T19:05:51.993 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:51.993 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:05:51.993 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:51.993 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.993 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.993 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:157528 8.7 8:e4768593:::benchmark_data_smithi083_97422_object41437:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e213)", 2024-09-20T19:05:51.993 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:56:59.165392+0000", 2024-09-20T19:05:51.993 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 532.76325939599997, 2024-09-20T19:05:51.994 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.958042846, 2024-09-20T19:05:51.994 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:05:51.994 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:05:51.994 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:05:51.994 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:05:51.994 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T19:05:51.994 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T19:05:51.994 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 157528 2024-09-20T19:05:51.994 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.994 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:05:51.994 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.994 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:05:51.994 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.165392+0000", 2024-09-20T19:05:51.994 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:05:51.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.995 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.995 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:05:51.995 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.165392+0000", 2024-09-20T19:05:51.995 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:05:51.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.995 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.995 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:05:51.995 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.165394+0000", 2024-09-20T19:05:51.995 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.77e-06 2024-09-20T19:05:51.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.995 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.995 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:05:51.995 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.165400+0000", 2024-09-20T19:05:51.996 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2099999999999998e-06 2024-09-20T19:05:51.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.996 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.996 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:05:51.996 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.165419+0000", 2024-09-20T19:05:51.996 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9046999999999999e-05 2024-09-20T19:05:51.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.996 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.996 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:05:51.996 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.165428+0000", 2024-09-20T19:05:51.996 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1030000000000008e-06 2024-09-20T19:05:51.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.996 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.996 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:51.996 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.166013+0000", 2024-09-20T19:05:51.997 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000584868 2024-09-20T19:05:51.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.997 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.997 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:05:51.997 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:56:59.166022+0000", 2024-09-20T19:05:51.997 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7090000000000008e-06 2024-09-20T19:05:51.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.997 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.997 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:51.997 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.095256+0000", 2024-09-20T19:05:51.997 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.92923407700000005 2024-09-20T19:05:51.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.997 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.997 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:05:51.997 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.095349+0000", 2024-09-20T19:05:51.997 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2724000000000002e-05 2024-09-20T19:05:51.998 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.998 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.998 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,3", 2024-09-20T19:05:51.998 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.095688+0000", 2024-09-20T19:05:51.998 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000339379 2024-09-20T19:05:51.998 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.998 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.998 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:05:51.998 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.104489+0000", 2024-09-20T19:05:51.998 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0088013359999999999 2024-09-20T19:05:51.998 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.998 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.998 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:05:51.999 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.106738+0000", 2024-09-20T19:05:51.999 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0022487029999999999 2024-09-20T19:05:51.999 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.999 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.999 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:05:51.999 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.123280+0000", 2024-09-20T19:05:51.999 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.016541613 2024-09-20T19:05:51.999 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.999 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:51.999 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:05:51.999 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.123398+0000", 2024-09-20T19:05:51.999 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011812299999999999 2024-09-20T19:05:51.999 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:51.999 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.000 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:05:52.000 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:57:00.123435+0000", 2024-09-20T19:05:52.000 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7184000000000002e-05 2024-09-20T19:05:52.000 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:52.000 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:05:52.000 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:52.000 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.000 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.000 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:50392 9.0 9:015f775a:::benchmark_data_smithi083_124552_object50391:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e259)", 2024-09-20T19:05:52.000 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:59:04.061560+0000", 2024-09-20T19:05:52.000 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 407.86709148800003, 2024-09-20T19:05:52.000 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1500109090000001, 2024-09-20T19:05:52.000 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:05:52.000 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:05:52.001 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-20T19:05:52.001 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:05:52.001 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:05:52.001 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:05:52.001 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 50392 2024-09-20T19:05:52.001 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.001 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:05:52.001 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.001 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:05:52.001 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.061560+0000", 2024-09-20T19:05:52.001 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:05:52.001 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.001 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.001 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:05:52.001 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.061560+0000", 2024-09-20T19:05:52.002 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:05:52.002 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.002 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.002 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:05:52.002 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.061561+0000", 2024-09-20T19:05:52.002 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.252e-06 2024-09-20T19:05:52.002 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.002 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.002 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:05:52.002 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.061585+0000", 2024-09-20T19:05:52.002 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3275e-05 2024-09-20T19:05:52.002 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.002 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.003 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:05:52.003 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.061596+0000", 2024-09-20T19:05:52.003 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0963e-05 2024-09-20T19:05:52.003 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.003 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.003 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:05:52.003 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.061604+0000", 2024-09-20T19:05:52.003 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2549999999999997e-06 2024-09-20T19:05:52.003 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.003 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.003 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:52.003 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.063037+0000", 2024-09-20T19:05:52.003 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001432699 2024-09-20T19:05:52.003 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.003 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.004 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:05:52.004 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.063196+0000", 2024-09-20T19:05:52.004 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015964199999999999 2024-09-20T19:05:52.004 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.004 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.004 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,2", 2024-09-20T19:05:52.004 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.063344+0000", 2024-09-20T19:05:52.004 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014738699999999999 2024-09-20T19:05:52.004 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.004 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.004 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:05:52.004 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.068011+0000", 2024-09-20T19:05:52.005 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0046678240000000001 2024-09-20T19:05:52.005 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.005 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.005 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:52.005 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.098231+0000", 2024-09-20T19:05:52.005 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.030219817999999999 2024-09-20T19:05:52.005 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.005 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.005 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:05:52.005 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.098244+0000", 2024-09-20T19:05:52.005 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2381000000000001e-05 2024-09-20T19:05:52.005 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.005 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.005 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:52.006 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.074475+0000", 2024-09-20T19:05:52.006 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97623147099999996 2024-09-20T19:05:52.006 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.006 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.006 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:05:52.006 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.074562+0000", 2024-09-20T19:05:52.006 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7317000000000004e-05 2024-09-20T19:05:52.006 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.006 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.006 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:52.006 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.201219+0000", 2024-09-20T19:05:52.006 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1266569 2024-09-20T19:05:52.006 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.006 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.007 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for degraded object", 2024-09-20T19:05:52.007 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.201984+0000", 2024-09-20T19:05:52.007 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00076430200000000002 2024-09-20T19:05:52.007 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.007 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.007 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:52.007 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.211479+0000", 2024-09-20T19:05:52.007 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0094952230000000006 2024-09-20T19:05:52.007 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.007 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.007 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:05:52.007 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.211571+0000", 2024-09-20T19:05:52.008 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2200000000000005e-05 2024-09-20T19:05:52.008 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:52.008 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:05:52.008 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:52.008 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.008 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.008 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:50400 9.0 9:0091def0:::benchmark_data_smithi083_124552_object50399:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-20T19:05:52.008 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:59:04.066245+0000", 2024-09-20T19:05:52.008 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 407.86240678799999, 2024-09-20T19:05:52.008 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1471234040000002, 2024-09-20T19:05:52.008 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:05:52.008 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:05:52.008 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:05:52.008 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:05:52.008 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:05:52.009 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:05:52.009 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 50400 2024-09-20T19:05:52.009 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.009 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:05:52.009 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.009 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:05:52.009 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.066245+0000", 2024-09-20T19:05:52.009 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:05:52.009 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.009 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.009 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:05:52.009 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.066245+0000", 2024-09-20T19:05:52.009 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:05:52.009 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.010 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.010 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:05:52.010 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.066247+0000", 2024-09-20T19:05:52.010 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.73e-06 2024-09-20T19:05:52.010 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.010 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.010 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:05:52.010 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.066271+0000", 2024-09-20T19:05:52.010 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4793000000000001e-05 2024-09-20T19:05:52.010 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.010 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.010 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:05:52.010 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.066286+0000", 2024-09-20T19:05:52.010 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4515000000000001e-05 2024-09-20T19:05:52.011 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.011 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.011 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:05:52.011 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.066295+0000", 2024-09-20T19:05:52.011 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7360000000000007e-06 2024-09-20T19:05:52.011 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.011 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.011 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:52.011 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.067890+0000", 2024-09-20T19:05:52.011 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001595791 2024-09-20T19:05:52.011 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.011 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.011 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op must wait for map", 2024-09-20T19:05:52.011 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.067896+0000", 2024-09-20T19:05:52.012 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.446e-06 2024-09-20T19:05:52.012 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.012 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.012 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:52.012 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.099590+0000", 2024-09-20T19:05:52.012 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.031694211999999999 2024-09-20T19:05:52.012 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.012 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.012 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:05:52.012 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.099600+0000", 2024-09-20T19:05:52.012 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.91e-06 2024-09-20T19:05:52.012 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.012 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.012 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:52.013 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.075887+0000", 2024-09-20T19:05:52.013 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.976287401 2024-09-20T19:05:52.013 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.013 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.013 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:05:52.013 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.075898+0000", 2024-09-20T19:05:52.013 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0294e-05 2024-09-20T19:05:52.013 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.013 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.013 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:52.013 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.203092+0000", 2024-09-20T19:05:52.013 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1271948649999999 2024-09-20T19:05:52.013 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.013 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.014 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:05:52.014 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.203244+0000", 2024-09-20T19:05:52.014 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015116500000000001 2024-09-20T19:05:52.014 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.014 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.014 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,6", 2024-09-20T19:05:52.014 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.203418+0000", 2024-09-20T19:05:52.014 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017446200000000001 2024-09-20T19:05:52.014 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.014 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.014 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:05:52.014 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.208270+0000", 2024-09-20T19:05:52.015 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:52.016 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0048516799999999997 2024-09-20T19:05:52.016 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.016 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.016 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:05:52.016 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.208385+0000", 2024-09-20T19:05:52.016 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000114954 2024-09-20T19:05:52.016 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.017 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.017 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:05:52.017 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.211663+0000", 2024-09-20T19:05:52.017 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0032777679999999999 2024-09-20T19:05:52.017 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.017 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.017 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:05:52.017 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.211716+0000", 2024-09-20T19:05:52.017 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3748000000000001e-05 2024-09-20T19:05:52.017 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.017 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.017 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:05:52.017 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.213368+0000", 2024-09-20T19:05:52.017 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0016519340000000001 2024-09-20T19:05:52.018 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:52.018 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:05:52.018 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:52.018 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.018 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.018 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:50401 9.0 9:08f1b92c:::benchmark_data_smithi083_124552_object50400:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-20T19:05:52.018 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:59:04.067487+0000", 2024-09-20T19:05:52.018 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 407.86116483799998, 2024-09-20T19:05:52.018 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.149403511, 2024-09-20T19:05:52.018 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:05:52.018 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:05:52.018 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:05:52.018 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:05:52.019 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:05:52.019 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:05:52.019 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 50401 2024-09-20T19:05:52.019 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.019 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:05:52.019 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.019 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:05:52.019 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.067487+0000", 2024-09-20T19:05:52.019 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:05:52.019 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.019 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.019 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:05:52.019 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.067487+0000", 2024-09-20T19:05:52.020 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:05:52.020 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.020 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.020 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:05:52.020 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.067488+0000", 2024-09-20T19:05:52.020 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4670000000000001e-06 2024-09-20T19:05:52.020 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.020 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.020 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:05:52.020 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.067512+0000", 2024-09-20T19:05:52.020 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3561999999999999e-05 2024-09-20T19:05:52.020 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.020 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.020 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:05:52.021 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.067523+0000", 2024-09-20T19:05:52.021 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.148e-05 2024-09-20T19:05:52.021 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.021 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.021 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:05:52.021 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.067531+0000", 2024-09-20T19:05:52.021 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2400000000000001e-06 2024-09-20T19:05:52.021 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.021 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.021 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:52.021 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.068001+0000", 2024-09-20T19:05:52.021 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00047044099999999999 2024-09-20T19:05:52.021 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.021 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.021 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting_for_map not empty", 2024-09-20T19:05:52.022 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.068005+0000", 2024-09-20T19:05:52.022 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3490000000000002e-06 2024-09-20T19:05:52.022 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.022 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.022 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:52.022 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.099629+0000", 2024-09-20T19:05:52.022 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.031623658999999998 2024-09-20T19:05:52.022 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.022 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.022 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:05:52.022 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.099637+0000", 2024-09-20T19:05:52.022 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5750000000000003e-06 2024-09-20T19:05:52.022 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.022 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.023 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:52.023 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.075922+0000", 2024-09-20T19:05:52.023 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97628498500000005 2024-09-20T19:05:52.023 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.023 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.023 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:05:52.023 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.075929+0000", 2024-09-20T19:05:52.023 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2389999999999996e-06 2024-09-20T19:05:52.023 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.023 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.023 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:52.023 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.204955+0000", 2024-09-20T19:05:52.023 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.129026353 2024-09-20T19:05:52.023 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.023 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.024 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:05:52.024 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.205053+0000", 2024-09-20T19:05:52.024 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7659999999999997e-05 2024-09-20T19:05:52.024 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.024 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.024 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,6", 2024-09-20T19:05:52.024 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.205157+0000", 2024-09-20T19:05:52.024 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000104237 2024-09-20T19:05:52.024 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.024 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.024 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:05:52.024 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.208403+0000", 2024-09-20T19:05:52.024 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0032460050000000002 2024-09-20T19:05:52.024 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.024 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.025 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:05:52.025 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.213804+0000", 2024-09-20T19:05:52.025 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0054012280000000001 2024-09-20T19:05:52.025 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.025 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.025 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:05:52.025 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.215227+0000", 2024-09-20T19:05:52.025 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014227039999999999 2024-09-20T19:05:52.025 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.025 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.025 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:05:52.025 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.215288+0000", 2024-09-20T19:05:52.025 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0730999999999998e-05 2024-09-20T19:05:52.025 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.026 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.026 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:05:52.026 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.216890+0000", 2024-09-20T19:05:52.026 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001602596 2024-09-20T19:05:52.026 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:52.026 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:05:52.026 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:52.026 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.026 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.026 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:50420 9.0 9:04fafae8:::benchmark_data_smithi083_124552_object50419:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-20T19:05:52.026 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:59:04.091011+0000", 2024-09-20T19:05:52.026 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 407.83764059999999, 2024-09-20T19:05:52.026 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.143183112, 2024-09-20T19:05:52.026 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:05:52.027 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:05:52.027 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:05:52.027 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:05:52.027 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:05:52.027 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:05:52.027 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 50420 2024-09-20T19:05:52.027 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.027 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:05:52.027 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.027 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:05:52.027 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.091011+0000", 2024-09-20T19:05:52.027 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:05:52.027 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.027 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.028 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:05:52.028 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.091011+0000", 2024-09-20T19:05:52.028 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:05:52.028 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.028 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.028 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:05:52.028 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.091013+0000", 2024-09-20T19:05:52.028 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6440000000000001e-06 2024-09-20T19:05:52.028 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.028 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.028 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:05:52.028 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.091069+0000", 2024-09-20T19:05:52.028 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6008000000000003e-05 2024-09-20T19:05:52.028 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.029 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.029 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:05:52.029 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.091080+0000", 2024-09-20T19:05:52.029 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1449e-05 2024-09-20T19:05:52.029 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.029 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.029 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:05:52.029 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.091087+0000", 2024-09-20T19:05:52.029 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3769999999999998e-06 2024-09-20T19:05:52.029 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.029 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.029 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:52.029 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.099660+0000", 2024-09-20T19:05:52.029 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0085721310000000002 2024-09-20T19:05:52.030 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.030 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.030 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:05:52.030 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.099667+0000", 2024-09-20T19:05:52.030 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9439999999999999e-06 2024-09-20T19:05:52.030 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.030 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.030 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:52.030 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.075951+0000", 2024-09-20T19:05:52.030 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97628466400000002 2024-09-20T19:05:52.030 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.030 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.030 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:05:52.031 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.075958+0000", 2024-09-20T19:05:52.031 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1019999999999999e-06 2024-09-20T19:05:52.031 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.031 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.031 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:52.031 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.216952+0000", 2024-09-20T19:05:52.031 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1409938340000001 2024-09-20T19:05:52.031 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.031 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.031 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:05:52.031 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.217094+0000", 2024-09-20T19:05:52.031 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014154900000000001 2024-09-20T19:05:52.031 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.031 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.031 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,6", 2024-09-20T19:05:52.032 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.217258+0000", 2024-09-20T19:05:52.032 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016462399999999999 2024-09-20T19:05:52.032 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.032 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.032 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:05:52.032 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.223577+0000", 2024-09-20T19:05:52.032 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063191740000000003 2024-09-20T19:05:52.032 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.032 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.032 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:05:52.032 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.223593+0000", 2024-09-20T19:05:52.032 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5588999999999999e-05 2024-09-20T19:05:52.032 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.032 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.033 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:05:52.033 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.230738+0000", 2024-09-20T19:05:52.033 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0071453879999999999 2024-09-20T19:05:52.033 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.033 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.033 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:05:52.033 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.230856+0000", 2024-09-20T19:05:52.033 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000117713 2024-09-20T19:05:52.033 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.033 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.033 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:05:52.033 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.234194+0000", 2024-09-20T19:05:52.033 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0033379220000000001 2024-09-20T19:05:52.033 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:52.034 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:05:52.034 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:52.034 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.034 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.034 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:50490 9.0 9:061fbcaa:::benchmark_data_smithi083_124552_object50489:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-20T19:05:52.034 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:59:04.144098+0000", 2024-09-20T19:05:52.034 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 407.78455399799998, 2024-09-20T19:05:52.034 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.093487444, 2024-09-20T19:05:52.034 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:05:52.034 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:05:52.034 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:05:52.034 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:05:52.034 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:05:52.034 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:05:52.035 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 50490 2024-09-20T19:05:52.035 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.035 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:05:52.035 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.035 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:05:52.035 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.144098+0000", 2024-09-20T19:05:52.035 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:05:52.035 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.035 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.035 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:05:52.035 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.144098+0000", 2024-09-20T19:05:52.035 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:05:52.035 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.036 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.036 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:05:52.036 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.144099+0000", 2024-09-20T19:05:52.036 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.412e-06 2024-09-20T19:05:52.036 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.036 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.036 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:05:52.036 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.144124+0000", 2024-09-20T19:05:52.036 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4788000000000001e-05 2024-09-20T19:05:52.036 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.036 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.036 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:05:52.036 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.144138+0000", 2024-09-20T19:05:52.036 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3735e-05 2024-09-20T19:05:52.037 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.037 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.037 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:05:52.037 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.144146+0000", 2024-09-20T19:05:52.037 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.1689999999999992e-06 2024-09-20T19:05:52.037 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.037 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.037 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:52.037 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.144218+0000", 2024-09-20T19:05:52.037 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1945000000000003e-05 2024-09-20T19:05:52.037 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.037 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.037 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:05:52.037 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.144227+0000", 2024-09-20T19:05:52.038 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6090000000000007e-06 2024-09-20T19:05:52.038 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.038 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.038 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:52.038 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.075982+0000", 2024-09-20T19:05:52.038 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.93175522499999996 2024-09-20T19:05:52.038 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.038 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.038 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:05:52.038 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.075990+0000", 2024-09-20T19:05:52.038 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4789999999999999e-06 2024-09-20T19:05:52.038 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.038 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.039 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:52.039 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.218974+0000", 2024-09-20T19:05:52.039 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1429841510000001 2024-09-20T19:05:52.039 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.039 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.039 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:05:52.039 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.219119+0000", 2024-09-20T19:05:52.039 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014451399999999999 2024-09-20T19:05:52.039 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.039 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.039 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,6", 2024-09-20T19:05:52.039 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.219402+0000", 2024-09-20T19:05:52.039 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00028311699999999999 2024-09-20T19:05:52.039 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.039 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.040 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:05:52.040 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.223642+0000", 2024-09-20T19:05:52.040 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0042407119999999998 2024-09-20T19:05:52.040 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.040 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.040 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:05:52.040 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.230612+0000", 2024-09-20T19:05:52.040 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0069695499999999997 2024-09-20T19:05:52.040 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.040 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.040 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:05:52.040 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.234261+0000", 2024-09-20T19:05:52.040 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0036485459999999999 2024-09-20T19:05:52.040 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.040 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.041 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:05:52.041 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.234386+0000", 2024-09-20T19:05:52.041 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012551199999999999 2024-09-20T19:05:52.041 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.041 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.041 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:05:52.041 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.237585+0000", 2024-09-20T19:05:52.041 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0031989800000000001 2024-09-20T19:05:52.041 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:52.041 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:05:52.041 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:52.041 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.041 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.041 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:50503 9.0 9:0e691601:::benchmark_data_smithi083_124552_object50502:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-20T19:05:52.042 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:59:04.153655+0000", 2024-09-20T19:05:52.042 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 407.77499660400002, 2024-09-20T19:05:52.042 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0864925420000002, 2024-09-20T19:05:52.042 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:05:52.042 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:05:52.042 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:05:52.042 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:05:52.042 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:05:52.042 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:05:52.042 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 50503 2024-09-20T19:05:52.042 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.042 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:05:52.042 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.042 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:05:52.043 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.153655+0000", 2024-09-20T19:05:52.043 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:05:52.043 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.043 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.043 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:05:52.043 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.153655+0000", 2024-09-20T19:05:52.043 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:05:52.043 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.043 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.043 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:05:52.043 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.153656+0000", 2024-09-20T19:05:52.043 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2099999999999996e-07 2024-09-20T19:05:52.043 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.043 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.043 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:05:52.044 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.153675+0000", 2024-09-20T19:05:52.044 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9873000000000001e-05 2024-09-20T19:05:52.044 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.044 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.044 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:05:52.044 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.153683+0000", 2024-09-20T19:05:52.044 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5469999999999999e-06 2024-09-20T19:05:52.044 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.044 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.044 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:05:52.044 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.153689+0000", 2024-09-20T19:05:52.044 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1129999999999997e-06 2024-09-20T19:05:52.044 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.044 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.045 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:52.045 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.153774+0000", 2024-09-20T19:05:52.045 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4806000000000001e-05 2024-09-20T19:05:52.045 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.045 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.045 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:05:52.045 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.153783+0000", 2024-09-20T19:05:52.045 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8890000000000008e-06 2024-09-20T19:05:52.045 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.045 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.045 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:52.045 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.076016+0000", 2024-09-20T19:05:52.045 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.92223312199999996 2024-09-20T19:05:52.045 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.045 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.046 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:05:52.046 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.076025+0000", 2024-09-20T19:05:52.046 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0480000000000004e-06 2024-09-20T19:05:52.046 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.046 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.046 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:52.046 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.221175+0000", 2024-09-20T19:05:52.046 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1451503329999999 2024-09-20T19:05:52.046 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.046 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.046 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:05:52.046 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.221314+0000", 2024-09-20T19:05:52.046 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013887599999999999 2024-09-20T19:05:52.046 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.047 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.047 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,6", 2024-09-20T19:05:52.047 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.221482+0000", 2024-09-20T19:05:52.047 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016769800000000001 2024-09-20T19:05:52.047 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.047 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.047 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:05:52.047 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.230668+0000", 2024-09-20T19:05:52.047 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0091861119999999997 2024-09-20T19:05:52.047 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.047 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.047 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:05:52.047 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.234213+0000", 2024-09-20T19:05:52.048 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0035446620000000001 2024-09-20T19:05:52.048 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.048 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.048 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:05:52.048 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.237649+0000", 2024-09-20T19:05:52.048 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003436555 2024-09-20T19:05:52.048 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.048 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.048 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:05:52.048 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.237809+0000", 2024-09-20T19:05:52.048 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015965099999999999 2024-09-20T19:05:52.048 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.048 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.048 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:05:52.049 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.240148+0000", 2024-09-20T19:05:52.049 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0023386359999999998 2024-09-20T19:05:52.049 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:52.049 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:05:52.049 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:52.049 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.049 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.049 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:142920 9.4 9:275bd396:::benchmark_data_smithi083_124552_object28773:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e291)", 2024-09-20T19:05:52.049 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:35.173575+0000", 2024-09-20T19:05:52.049 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 316.75507679999998, 2024-09-20T19:05:52.049 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0532428149999999, 2024-09-20T19:05:52.049 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:05:52.049 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:05:52.049 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-20T19:05:52.049 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:05:52.050 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:05:52.050 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:05:52.050 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 142920 2024-09-20T19:05:52.050 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.050 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:05:52.050 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.050 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:05:52.050 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.173575+0000", 2024-09-20T19:05:52.050 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:05:52.050 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.050 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.050 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:05:52.050 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.173575+0000", 2024-09-20T19:05:52.050 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:05:52.050 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.051 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.051 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:05:52.051 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.173576+0000", 2024-09-20T19:05:52.051 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6899999999999999e-06 2024-09-20T19:05:52.051 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.051 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.051 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:05:52.051 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.173583+0000", 2024-09-20T19:05:52.051 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0889999999999996e-06 2024-09-20T19:05:52.051 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.051 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.051 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:05:52.051 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.173598+0000", 2024-09-20T19:05:52.051 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5917e-05 2024-09-20T19:05:52.052 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.052 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.052 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:05:52.052 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.173606+0000", 2024-09-20T19:05:52.052 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.362e-06 2024-09-20T19:05:52.052 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.052 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.052 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:52.052 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.173664+0000", 2024-09-20T19:05:52.052 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7725e-05 2024-09-20T19:05:52.052 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.052 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.052 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:05:52.052 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.173770+0000", 2024-09-20T19:05:52.052 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106901 2024-09-20T19:05:52.053 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.053 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.053 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3,4", 2024-09-20T19:05:52.053 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.173894+0000", 2024-09-20T19:05:52.053 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000123139 2024-09-20T19:05:52.053 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.053 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.053 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:05:52.053 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.181240+0000", 2024-09-20T19:05:52.053 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0073461539999999997 2024-09-20T19:05:52.053 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.053 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.053 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:05:52.053 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.181260+0000", 2024-09-20T19:05:52.053 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9987e-05 2024-09-20T19:05:52.053 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.054 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.054 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:52.054 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.200929+0000", 2024-09-20T19:05:52.054 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.019669494999999999 2024-09-20T19:05:52.054 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.054 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.054 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:05:52.054 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.200940+0000", 2024-09-20T19:05:52.054 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0864e-05 2024-09-20T19:05:52.054 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.054 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.054 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:52.054 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.226746+0000", 2024-09-20T19:05:52.055 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.025806225 2024-09-20T19:05:52.055 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.055 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.055 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:05:52.055 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.226818+0000", 2024-09-20T19:05:52.055 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1266999999999994e-05 2024-09-20T19:05:52.055 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:52.055 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:05:52.055 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:52.055 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.055 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.055 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:142948 9.4 9:23ac88ff:::benchmark_data_smithi083_124552_object28801:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e292)", 2024-09-20T19:05:52.055 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:35.220440+0000", 2024-09-20T19:05:52.055 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 316.708211936, 2024-09-20T19:05:52.055 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0177205739999999, 2024-09-20T19:05:52.056 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:05:52.056 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:05:52.056 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:05:52.056 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:05:52.056 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:05:52.056 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:05:52.056 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 142948 2024-09-20T19:05:52.056 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.056 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:05:52.056 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.056 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:05:52.056 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.220440+0000", 2024-09-20T19:05:52.056 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:05:52.056 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.056 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.057 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:05:52.057 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.220440+0000", 2024-09-20T19:05:52.057 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:05:52.057 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.057 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.057 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:05:52.057 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.220441+0000", 2024-09-20T19:05:52.057 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1999999999999998e-07 2024-09-20T19:05:52.057 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.057 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.057 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:05:52.057 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.220445+0000", 2024-09-20T19:05:52.057 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1219999999999997e-06 2024-09-20T19:05:52.057 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.057 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.058 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:05:52.058 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.220453+0000", 2024-09-20T19:05:52.058 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.5359999999999992e-06 2024-09-20T19:05:52.058 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.058 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.058 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:05:52.058 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.220459+0000", 2024-09-20T19:05:52.058 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0929999999999999e-06 2024-09-20T19:05:52.058 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.058 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.058 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:52.058 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.220510+0000", 2024-09-20T19:05:52.058 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0853000000000003e-05 2024-09-20T19:05:52.058 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.059 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.059 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:05:52.059 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.220518+0000", 2024-09-20T19:05:52.059 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4039999999999998e-06 2024-09-20T19:05:52.059 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.059 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.059 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:52.059 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.227191+0000", 2024-09-20T19:05:52.059 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0066732009999999 2024-09-20T19:05:52.059 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.059 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.059 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:05:52.059 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.227287+0000", 2024-09-20T19:05:52.059 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6204000000000003e-05 2024-09-20T19:05:52.059 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.060 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.060 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-20T19:05:52.060 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.227405+0000", 2024-09-20T19:05:52.060 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000118386 2024-09-20T19:05:52.060 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.060 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.060 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:05:52.060 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.237785+0000", 2024-09-20T19:05:52.060 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010379578 2024-09-20T19:05:52.060 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.060 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.060 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:05:52.060 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.238035+0000", 2024-09-20T19:05:52.060 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00024981 2024-09-20T19:05:52.060 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.060 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.061 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:05:52.061 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.238080+0000", 2024-09-20T19:05:52.061 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5015000000000002e-05 2024-09-20T19:05:52.061 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.061 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.061 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:05:52.061 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.238160+0000", 2024-09-20T19:05:52.061 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0451999999999995e-05 2024-09-20T19:05:52.061 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:52.061 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:05:52.061 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:52.061 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.061 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.061 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:142954 9.4 9:221e2aa5:::benchmark_data_smithi083_124552_object28807:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e292)", 2024-09-20T19:05:52.061 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:35.228555+0000", 2024-09-20T19:05:52.062 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 316.70009693999998, 2024-09-20T19:05:52.062 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0097211580000001, 2024-09-20T19:05:52.062 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:05:52.062 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:05:52.062 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:05:52.062 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:05:52.062 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:05:52.062 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:05:52.062 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 142954 2024-09-20T19:05:52.062 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.062 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:05:52.062 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.062 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:05:52.062 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.228555+0000", 2024-09-20T19:05:52.063 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:05:52.063 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.063 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.063 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:05:52.063 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.228555+0000", 2024-09-20T19:05:52.063 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:05:52.063 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.063 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.063 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:05:52.063 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.228556+0000", 2024-09-20T19:05:52.063 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.533e-06 2024-09-20T19:05:52.063 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.063 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.063 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:05:52.063 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.228563+0000", 2024-09-20T19:05:52.064 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6619999999999999e-06 2024-09-20T19:05:52.064 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.064 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.064 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:05:52.064 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.228587+0000", 2024-09-20T19:05:52.064 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.444e-05 2024-09-20T19:05:52.064 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.064 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.064 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:05:52.064 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.228596+0000", 2024-09-20T19:05:52.064 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6060000000000003e-06 2024-09-20T19:05:52.064 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.064 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.064 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:52.065 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.228659+0000", 2024-09-20T19:05:52.065 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2677999999999999e-05 2024-09-20T19:05:52.065 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.065 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.065 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:05:52.065 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.228668+0000", 2024-09-20T19:05:52.065 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2220000000000008e-06 2024-09-20T19:05:52.065 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.065 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.065 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:52.065 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.229869+0000", 2024-09-20T19:05:52.065 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.00120093 2024-09-20T19:05:52.065 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.065 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.066 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:05:52.066 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.229964+0000", 2024-09-20T19:05:52.066 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5072e-05 2024-09-20T19:05:52.066 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.066 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.066 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-20T19:05:52.066 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.230066+0000", 2024-09-20T19:05:52.066 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000102558 2024-09-20T19:05:52.066 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.066 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.066 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:05:52.066 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.237878+0000", 2024-09-20T19:05:52.066 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0078119929999999997 2024-09-20T19:05:52.067 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.067 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.067 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:05:52.067 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.238170+0000", 2024-09-20T19:05:52.067 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00029176400000000002 2024-09-20T19:05:52.067 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.067 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.067 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:05:52.067 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.238208+0000", 2024-09-20T19:05:52.067 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7512999999999998e-05 2024-09-20T19:05:52.067 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.067 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.067 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:05:52.067 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.238276+0000", 2024-09-20T19:05:52.068 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8187000000000003e-05 2024-09-20T19:05:52.068 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:52.068 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:05:52.068 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:52.068 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.068 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.068 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:142972 9.4 9:2d2af5ad:::benchmark_data_smithi083_124552_object28825:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e292)", 2024-09-20T19:05:52.068 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:35.245464+0000", 2024-09-20T19:05:52.068 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 316.68318726500002, 2024-09-20T19:05:52.068 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.99292613500000004, 2024-09-20T19:05:52.068 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:05:52.068 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:05:52.068 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:05:52.068 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:05:52.069 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:05:52.069 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:05:52.069 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 142972 2024-09-20T19:05:52.069 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.069 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:05:52.069 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.069 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:05:52.069 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.245464+0000", 2024-09-20T19:05:52.070 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:05:52.070 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.070 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.070 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:05:52.070 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.245464+0000", 2024-09-20T19:05:52.070 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:05:52.070 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.070 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.070 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:05:52.070 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.245466+0000", 2024-09-20T19:05:52.070 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.869e-06 2024-09-20T19:05:52.070 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.070 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.071 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:05:52.071 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.245473+0000", 2024-09-20T19:05:52.071 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.776e-06 2024-09-20T19:05:52.071 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.071 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.071 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:05:52.071 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.245485+0000", 2024-09-20T19:05:52.071 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.202e-05 2024-09-20T19:05:52.071 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.071 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.071 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:05:52.071 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.245494+0000", 2024-09-20T19:05:52.071 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0920000000000004e-06 2024-09-20T19:05:52.071 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.072 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.072 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:52.072 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.245551+0000", 2024-09-20T19:05:52.072 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7138000000000001e-05 2024-09-20T19:05:52.072 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.072 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.072 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:05:52.072 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.245559+0000", 2024-09-20T19:05:52.072 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8509999999999999e-06 2024-09-20T19:05:52.072 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.072 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.072 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:52.072 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.232633+0000", 2024-09-20T19:05:52.073 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.98707439200000002 2024-09-20T19:05:52.073 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.073 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.073 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:05:52.073 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.232734+0000", 2024-09-20T19:05:52.073 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000100424 2024-09-20T19:05:52.073 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.073 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.073 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-20T19:05:52.073 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.232845+0000", 2024-09-20T19:05:52.073 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000110645 2024-09-20T19:05:52.073 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.073 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.074 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:05:52.074 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.238021+0000", 2024-09-20T19:05:52.074 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0051761410000000004 2024-09-20T19:05:52.074 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.074 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.074 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:05:52.074 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.238283+0000", 2024-09-20T19:05:52.074 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000262339 2024-09-20T19:05:52.074 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.074 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.074 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:05:52.074 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.238322+0000", 2024-09-20T19:05:52.074 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9017000000000003e-05 2024-09-20T19:05:52.074 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.075 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.075 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:05:52.075 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.238390+0000", 2024-09-20T19:05:52.075 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8430999999999996e-05 2024-09-20T19:05:52.075 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:52.075 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:05:52.075 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:52.075 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.075 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.075 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:142980 9.4 9:2b9b75e3:::benchmark_data_smithi083_124552_object28833:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e292)", 2024-09-20T19:05:52.075 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:35.254040+0000", 2024-09-20T19:05:52.075 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 316.67461112000001, 2024-09-20T19:05:52.075 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.99718591300000003, 2024-09-20T19:05:52.075 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:05:52.076 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:05:52.076 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:05:52.076 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:05:52.076 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:05:52.076 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:05:52.076 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 142980 2024-09-20T19:05:52.076 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.076 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:05:52.076 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.076 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:05:52.076 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.254040+0000", 2024-09-20T19:05:52.076 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:05:52.076 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.076 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.076 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:05:52.077 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.254040+0000", 2024-09-20T19:05:52.077 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:05:52.077 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.077 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.077 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:05:52.077 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.254043+0000", 2024-09-20T19:05:52.077 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1100000000000001e-06 2024-09-20T19:05:52.077 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.077 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.077 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:05:52.077 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.254049+0000", 2024-09-20T19:05:52.077 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2949999999999999e-06 2024-09-20T19:05:52.077 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.078 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.078 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:05:52.078 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.254067+0000", 2024-09-20T19:05:52.078 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7912999999999999e-05 2024-09-20T19:05:52.078 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.078 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.078 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:05:52.078 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.254075+0000", 2024-09-20T19:05:52.078 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4889999999999995e-06 2024-09-20T19:05:52.078 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.078 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.078 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:52.078 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.254143+0000", 2024-09-20T19:05:52.078 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8182999999999994e-05 2024-09-20T19:05:52.079 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.079 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.079 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:05:52.079 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.254154+0000", 2024-09-20T19:05:52.079 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0227000000000001e-05 2024-09-20T19:05:52.079 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.079 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.079 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:52.079 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.235381+0000", 2024-09-20T19:05:52.079 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.98122688400000002 2024-09-20T19:05:52.079 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.079 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.079 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:05:52.079 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.235478+0000", 2024-09-20T19:05:52.080 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7344999999999997e-05 2024-09-20T19:05:52.080 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.080 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.080 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-20T19:05:52.080 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.235580+0000", 2024-09-20T19:05:52.080 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010220700000000001 2024-09-20T19:05:52.080 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.080 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.080 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:05:52.080 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.250981+0000", 2024-09-20T19:05:52.080 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.015400713999999999 2024-09-20T19:05:52.080 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.080 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.080 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:05:52.080 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.251017+0000", 2024-09-20T19:05:52.081 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6649000000000001e-05 2024-09-20T19:05:52.081 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.081 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.081 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:05:52.081 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.251114+0000", 2024-09-20T19:05:52.081 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6287999999999997e-05 2024-09-20T19:05:52.081 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.081 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.081 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:05:52.081 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.251226+0000", 2024-09-20T19:05:52.081 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011260900000000001 2024-09-20T19:05:52.081 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:52.081 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:05:52.082 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:52.082 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.082 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.082 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:142984 9.4 9:2e403320:::benchmark_data_smithi083_124552_object28837:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e292)", 2024-09-20T19:05:52.082 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:35.256799+0000", 2024-09-20T19:05:52.082 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 316.671852441, 2024-09-20T19:05:52.082 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.99530053100000004, 2024-09-20T19:05:52.082 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:05:52.082 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:05:52.082 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:05:52.082 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:05:52.082 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:05:52.083 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:05:52.083 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 142984 2024-09-20T19:05:52.083 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.083 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:05:52.083 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.083 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:05:52.083 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.256799+0000", 2024-09-20T19:05:52.083 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:05:52.083 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.083 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.083 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:05:52.083 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.256799+0000", 2024-09-20T19:05:52.083 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:05:52.083 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.084 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.084 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:05:52.084 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.256801+0000", 2024-09-20T19:05:52.084 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9199999999999998e-06 2024-09-20T19:05:52.084 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.084 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.084 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:05:52.084 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.256808+0000", 2024-09-20T19:05:52.084 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.601e-06 2024-09-20T19:05:52.084 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.084 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.084 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:05:52.084 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.256825+0000", 2024-09-20T19:05:52.084 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7189000000000001e-05 2024-09-20T19:05:52.084 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.085 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.085 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:05:52.085 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.256834+0000", 2024-09-20T19:05:52.085 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2669999999999995e-06 2024-09-20T19:05:52.085 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.085 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.085 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:52.085 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.256916+0000", 2024-09-20T19:05:52.085 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2342000000000005e-05 2024-09-20T19:05:52.085 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.085 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.085 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:05:52.085 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.256926+0000", 2024-09-20T19:05:52.085 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6220000000000004e-06 2024-09-20T19:05:52.085 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.086 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.086 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:05:52.086 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.238429+0000", 2024-09-20T19:05:52.086 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.981503126 2024-09-20T19:05:52.086 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.086 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.086 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:05:52.086 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.238613+0000", 2024-09-20T19:05:52.086 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018369299999999999 2024-09-20T19:05:52.086 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.086 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.086 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3", 2024-09-20T19:05:52.086 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.238721+0000", 2024-09-20T19:05:52.086 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000108472 2024-09-20T19:05:52.087 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.087 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.087 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:05:52.087 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.251237+0000", 2024-09-20T19:05:52.087 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.012515584 2024-09-20T19:05:52.087 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.087 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.087 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:05:52.087 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.251956+0000", 2024-09-20T19:05:52.087 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00071889299999999996 2024-09-20T19:05:52.087 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.087 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.087 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:05:52.087 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.252024+0000", 2024-09-20T19:05:52.087 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8477000000000002e-05 2024-09-20T19:05:52.088 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:05:52.088 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:05:52.088 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:05:52.088 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.252100+0000", 2024-09-20T19:05:52.088 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5345000000000005e-05 2024-09-20T19:05:52.088 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:52.088 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:05:52.088 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:52.088 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:05:52.088 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:05:52.088 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T19:05:52.099 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T19:05:52.101 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-20T19:05:52.102 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:52.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:52.103 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:52.119 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:51 smithi083 ceph-mon[29421]: pgmap v806: 148 pgs: 148 active+clean; 686 MiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.24k op/s 2024-09-20T19:05:52.120 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:52 smithi083 ceph-mon[29421]: 10.5 scrub starts 2024-09-20T19:05:52.120 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:52 smithi083 ceph-mon[29421]: 6.13 scrub starts 2024-09-20T19:05:52.120 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:52 smithi083 ceph-mon[29421]: 6.13 scrub ok 2024-09-20T19:05:52.120 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:52 smithi083 ceph-mon[29421]: 10.5 scrub ok 2024-09-20T19:05:52.120 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:52 smithi083 ceph-mon[36260]: pgmap v806: 148 pgs: 148 active+clean; 686 MiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.24k op/s 2024-09-20T19:05:52.120 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:52 smithi083 ceph-mon[36260]: 10.5 scrub starts 2024-09-20T19:05:52.120 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:52 smithi083 ceph-mon[36260]: 6.13 scrub starts 2024-09-20T19:05:52.120 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:52 smithi083 ceph-mon[36260]: 6.13 scrub ok 2024-09-20T19:05:52.121 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:52 smithi083 ceph-mon[36260]: 10.5 scrub ok 2024-09-20T19:05:52.204 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:52.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:52.244 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:52.244 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:52.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:51 smithi142 ceph-mon[32508]: pgmap v806: 148 pgs: 148 active+clean; 686 MiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.24k op/s 2024-09-20T19:05:52.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:51 smithi142 ceph-mon[32508]: 10.5 scrub starts 2024-09-20T19:05:52.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:51 smithi142 ceph-mon[32508]: 6.13 scrub starts 2024-09-20T19:05:52.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:51 smithi142 ceph-mon[32508]: 6.13 scrub ok 2024-09-20T19:05:52.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:51 smithi142 ceph-mon[32508]: 10.5 scrub ok 2024-09-20T19:05:52.344 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:05:52.384 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:52.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:52.385 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:05:52.486 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:52.522 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:52.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:52.523 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:52.624 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:52.659 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:52.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:52.660 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:52.761 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:52.797 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:52.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:52.798 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:52.813 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T19:05:52.899 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:52.935 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:52.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:52.936 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:53.036 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:53.074 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:53.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:53.076 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:53.176 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:53.223 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:53.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:53.225 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:53.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:52 smithi142 ceph-mon[32508]: 6.10 scrub starts 2024-09-20T19:05:53.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:52 smithi142 ceph-mon[32508]: 6.10 scrub ok 2024-09-20T19:05:53.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:52 smithi142 ceph-mon[32508]: 5.7 scrub starts 2024-09-20T19:05:53.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:52 smithi142 ceph-mon[32508]: 5.7 scrub ok 2024-09-20T19:05:53.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:52 smithi142 ceph-mon[32508]: from='client.26168 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:05:53.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:52 smithi142 ceph-mon[32508]: 3.c scrub starts 2024-09-20T19:05:53.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:52 smithi142 ceph-mon[32508]: 3.c scrub ok 2024-09-20T19:05:53.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:52 smithi083 ceph-mon[29421]: 6.10 scrub starts 2024-09-20T19:05:53.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:52 smithi083 ceph-mon[29421]: 6.10 scrub ok 2024-09-20T19:05:53.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:52 smithi083 ceph-mon[29421]: 5.7 scrub starts 2024-09-20T19:05:53.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:52 smithi083 ceph-mon[29421]: 5.7 scrub ok 2024-09-20T19:05:53.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:52 smithi083 ceph-mon[29421]: from='client.26168 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:05:53.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:52 smithi083 ceph-mon[29421]: 3.c scrub starts 2024-09-20T19:05:53.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:52 smithi083 ceph-mon[29421]: 3.c scrub ok 2024-09-20T19:05:53.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:53 smithi083 ceph-mon[36260]: 6.10 scrub starts 2024-09-20T19:05:53.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:53 smithi083 ceph-mon[36260]: 6.10 scrub ok 2024-09-20T19:05:53.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:53 smithi083 ceph-mon[36260]: 5.7 scrub starts 2024-09-20T19:05:53.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:53 smithi083 ceph-mon[36260]: 5.7 scrub ok 2024-09-20T19:05:53.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:53 smithi083 ceph-mon[36260]: from='client.26168 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:05:53.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:53 smithi083 ceph-mon[36260]: 3.c scrub starts 2024-09-20T19:05:53.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:53 smithi083 ceph-mon[36260]: 3.c scrub ok 2024-09-20T19:05:53.325 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:53.359 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-20T19:05:53.367 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:53.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:53.368 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:53.469 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:53.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:53.520 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:53.521 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:53.621 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:53.649 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:53.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:53.650 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:53.750 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:53.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:53.791 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:53.791 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:53.892 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:53.929 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:53.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:53.930 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:54.030 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:05:54.065 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:54.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:54.066 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:05:54.167 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:54.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:54.205 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:54.205 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:54.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:54 smithi142 ceph-mon[32508]: 6.16 scrub starts 2024-09-20T19:05:54.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:54 smithi142 ceph-mon[32508]: 6.16 scrub ok 2024-09-20T19:05:54.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:54 smithi142 ceph-mon[32508]: pgmap v807: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 423 MiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.40k op/s 2024-09-20T19:05:54.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:54 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/4613320' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:05:54.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:54 smithi083 ceph-mon[29421]: 6.16 scrub starts 2024-09-20T19:05:54.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:54 smithi083 ceph-mon[29421]: 6.16 scrub ok 2024-09-20T19:05:54.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:54 smithi083 ceph-mon[29421]: pgmap v807: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 423 MiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.40k op/s 2024-09-20T19:05:54.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:54 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/4613320' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:05:54.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:54 smithi083 ceph-mon[36260]: 6.16 scrub starts 2024-09-20T19:05:54.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:54 smithi083 ceph-mon[36260]: 6.16 scrub ok 2024-09-20T19:05:54.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:54 smithi083 ceph-mon[36260]: pgmap v807: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 423 MiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.40k op/s 2024-09-20T19:05:54.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:54 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/4613320' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:05:54.306 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:54.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:54.338 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:54.339 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:54.439 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:54.474 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:54.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:54.475 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:54.576 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:54.594 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:05:54.594 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:05:54.594 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:05:54.594 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:05:54.595 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:05:54.595 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:05:54.595 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:05:54.595 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:05:54.595 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:05:54.595 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:05:54.595 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:05:54.595 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:05:54.595 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:05:54.595 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:05:54.595 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:05:54.595 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:05:54.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:54.615 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:54.615 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:54.716 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:54.753 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:54.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:54.755 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:54.856 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:54.888 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:54.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:54.889 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:54.990 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:55.024 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:55.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:55.025 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:55.126 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:55.148 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:55.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:55.150 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:55.251 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:55.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:55.285 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:55.286 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:55.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:55 smithi083 ceph-mon[29421]: 4.1b scrub starts 2024-09-20T19:05:55.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:55 smithi083 ceph-mon[29421]: 4.1b scrub ok 2024-09-20T19:05:55.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:55 smithi083 ceph-mon[29421]: 4.5 scrub starts 2024-09-20T19:05:55.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:55 smithi083 ceph-mon[29421]: 4.5 scrub ok 2024-09-20T19:05:55.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:55 smithi083 ceph-mon[29421]: 4.0 scrub starts 2024-09-20T19:05:55.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:55 smithi083 ceph-mon[29421]: 4.0 scrub ok 2024-09-20T19:05:55.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:55 smithi083 ceph-mon[36260]: 4.1b scrub starts 2024-09-20T19:05:55.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:55 smithi083 ceph-mon[36260]: 4.1b scrub ok 2024-09-20T19:05:55.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:55 smithi083 ceph-mon[36260]: 4.5 scrub starts 2024-09-20T19:05:55.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:55 smithi083 ceph-mon[36260]: 4.5 scrub ok 2024-09-20T19:05:55.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:55 smithi083 ceph-mon[36260]: 4.0 scrub starts 2024-09-20T19:05:55.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:55 smithi083 ceph-mon[36260]: 4.0 scrub ok 2024-09-20T19:05:55.386 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:55.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:55.422 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:55.422 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:55.512 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:55 smithi142 ceph-mon[32508]: 4.1b scrub starts 2024-09-20T19:05:55.512 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:55 smithi142 ceph-mon[32508]: 4.1b scrub ok 2024-09-20T19:05:55.512 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:55 smithi142 ceph-mon[32508]: 4.5 scrub starts 2024-09-20T19:05:55.512 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:55 smithi142 ceph-mon[32508]: 4.5 scrub ok 2024-09-20T19:05:55.512 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:55 smithi142 ceph-mon[32508]: 4.0 scrub starts 2024-09-20T19:05:55.512 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:55 smithi142 ceph-mon[32508]: 4.0 scrub ok 2024-09-20T19:05:55.523 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:05:55.559 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:55.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:55.560 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:05:55.661 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:55.682 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:55.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:55.683 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:55.784 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:55.819 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:55.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:55.820 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:55.921 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:05:55.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:55.951 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:55.952 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:05:56.052 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:56.083 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:56.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:56.085 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:56.185 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:56.237 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:56.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:56.238 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:56.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:56 smithi083 ceph-mon[29421]: pgmap v808: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 423 MiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 796 op/s 2024-09-20T19:05:56.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:56 smithi083 ceph-mon[29421]: 3.8 scrub starts 2024-09-20T19:05:56.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:56 smithi083 ceph-mon[29421]: 3.8 scrub ok 2024-09-20T19:05:56.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:56 smithi083 ceph-mon[36260]: pgmap v808: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 423 MiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 796 op/s 2024-09-20T19:05:56.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:56 smithi083 ceph-mon[36260]: 3.8 scrub starts 2024-09-20T19:05:56.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:56 smithi083 ceph-mon[36260]: 3.8 scrub ok 2024-09-20T19:05:56.340 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:56.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:56.375 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:56.375 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:56.476 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:56.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:56 smithi142 ceph-mon[32508]: pgmap v808: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 423 MiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 796 op/s 2024-09-20T19:05:56.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:56 smithi142 ceph-mon[32508]: 3.8 scrub starts 2024-09-20T19:05:56.515 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:56 smithi142 ceph-mon[32508]: 3.8 scrub ok 2024-09-20T19:05:56.517 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:56.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:56.518 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:56.618 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:56.651 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:56.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:56.651 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:56.752 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:56.780 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:56.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:56.781 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:56.882 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:56.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:56.921 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:56.921 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:57.022 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:57.057 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:57.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:57.058 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:57.159 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:57.192 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:57.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:57.193 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:57.224 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:05:57.286 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:57 smithi083 ceph-mon[29421]: 3.9 scrub starts 2024-09-20T19:05:57.286 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:57 smithi083 ceph-mon[29421]: 3.9 scrub ok 2024-09-20T19:05:57.286 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:57 smithi083 ceph-mon[29421]: 6.15 scrub starts 2024-09-20T19:05:57.287 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:57 smithi083 ceph-mon[29421]: 6.15 scrub ok 2024-09-20T19:05:57.287 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:57 smithi083 ceph-mon[36260]: 3.9 scrub starts 2024-09-20T19:05:57.287 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:57 smithi083 ceph-mon[36260]: 3.9 scrub ok 2024-09-20T19:05:57.287 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:57 smithi083 ceph-mon[36260]: 6.15 scrub starts 2024-09-20T19:05:57.287 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:57 smithi083 ceph-mon[36260]: 6.15 scrub ok 2024-09-20T19:05:57.291 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":806,"stamp":"2024-09-20T19:05:51.153908+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":719264307,"num_objects":11166,"num_object_clones":0,"num_object_copies":33498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11166,"num_whiteouts":0,"num_read":2211,"num_read_kb":2068,"num_write":223588,"num_write_kb":7480650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32723,"num_bytes_recovered":1816339955,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92127,"ondisk_log_size":92127,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":446,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":6643820,"kb_used_data":2213344,"kb_used_omap":16,"kb_used_meta":4430063,"kb_avail":743317396,"statfs":{"total":767960285184,"available":761157013504,"internally_reserved":0,"allocated":2266464256,"data_stored":2262537633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16790,"internal_metadata":4536385130},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,4,6,7,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-488439808,"num_objects":-7453,"num_object_clones":0,"num_object_copies":-22359,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7453,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":7453,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001903"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":745,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.905007+0000","last_change":"2024-09-20T19:01:31.817967+0000","last_active":"2024-09-20T19:05:45.905007+0000","last_peered":"2024-09-20T19:05:45.905007+0000","last_clean":"2024-09-20T19:05:45.905007+0000","last_became_active":"2024-09-20T18:49:45.943201+0000","last_became_peered":"2024-09-20T18:49:45.943201+0000","last_unstale":"2024-09-20T19:05:45.905007+0000","last_undegraded":"2024-09-20T19:05:45.905007+0000","last_fullsized":"2024-09-20T19:05:45.905007+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:31.817909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:31.817909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"273'15","reported_seq":88,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:48.982490+0000","last_change":"2024-09-20T19:04:21.040978+0000","last_active":"2024-09-20T19:05:48.982490+0000","last_peered":"2024-09-20T19:05:48.982490+0000","last_clean":"2024-09-20T19:05:48.982490+0000","last_became_active":"2024-09-20T19:04:20.925853+0000","last_became_peered":"2024-09-20T19:04:20.925853+0000","last_unstale":"2024-09-20T19:05:48.982490+0000","last_undegraded":"2024-09-20T19:05:48.982490+0000","last_fullsized":"2024-09-20T19:05:48.982490+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:03:10.214154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:10.214154+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":735,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.907622+0000","last_change":"2024-09-20T18:57:49.016078+0000","last_active":"2024-09-20T19:05:45.907622+0000","last_peered":"2024-09-20T19:05:45.907622+0000","last_clean":"2024-09-20T19:05:45.907622+0000","last_became_active":"2024-09-20T18:49:49.959514+0000","last_became_peered":"2024-09-20T18:49:49.959514+0000","last_unstale":"2024-09-20T19:05:45.907622+0000","last_undegraded":"2024-09-20T19:05:45.907622+0000","last_fullsized":"2024-09-20T19:05:45.907622+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:49.016006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:49.016006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:25.184023+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":78,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:48.982866+0000","last_change":"2024-09-20T19:04:20.881918+0000","last_active":"2024-09-20T19:05:48.982866+0000","last_peered":"2024-09-20T19:05:48.982866+0000","last_clean":"2024-09-20T19:05:48.982866+0000","last_became_active":"2024-09-20T19:04:20.880905+0000","last_became_peered":"2024-09-20T19:04:20.880905+0000","last_unstale":"2024-09-20T19:05:48.982866+0000","last_undegraded":"2024-09-20T19:05:48.982866+0000","last_fullsized":"2024-09-20T19:05:48.982866+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:02.084375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:02.084375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:37.200857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":747,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:49.000916+0000","last_change":"2024-09-20T19:02:02.441633+0000","last_active":"2024-09-20T19:05:49.000916+0000","last_peered":"2024-09-20T19:05:49.000916+0000","last_clean":"2024-09-20T19:05:49.000916+0000","last_became_active":"2024-09-20T18:59:05.081081+0000","last_became_peered":"2024-09-20T18:59:05.081081+0000","last_unstale":"2024-09-20T19:05:49.000916+0000","last_undegraded":"2024-09-20T19:05:49.000916+0000","last_fullsized":"2024-09-20T19:05:49.000916+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:02.441577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:02.441577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:58.610226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":760,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:48.999060+0000","last_change":"2024-09-20T19:04:37.540824+0000","last_active":"2024-09-20T19:05:48.999060+0000","last_peered":"2024-09-20T19:05:48.999060+0000","last_clean":"2024-09-20T19:05:48.999060+0000","last_became_active":"2024-09-20T18:59:05.072675+0000","last_became_peered":"2024-09-20T18:59:05.072675+0000","last_unstale":"2024-09-20T19:05:48.999060+0000","last_undegraded":"2024-09-20T19:05:48.999060+0000","last_fullsized":"2024-09-20T19:05:48.999060+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:37.540760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:04:37.540760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:55.234054+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"274'27","reported_seq":792,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:50.750134+0000","last_change":"2024-09-20T19:05:50.750134+0000","last_active":"2024-09-20T19:05:50.750134+0000","last_peered":"2024-09-20T19:05:50.750134+0000","last_clean":"2024-09-20T19:05:50.750134+0000","last_became_active":"2024-09-20T18:49:47.952150+0000","last_became_peered":"2024-09-20T18:49:47.952150+0000","last_unstale":"2024-09-20T19:05:50.750134+0000","last_undegraded":"2024-09-20T19:05:50.750134+0000","last_fullsized":"2024-09-20T19:05:50.750134+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'27","last_scrub_stamp":"2024-09-20T19:05:50.750050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:50.750050+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:13.513790+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":78,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:48.982500+0000","last_change":"2024-09-20T19:04:20.926234+0000","last_active":"2024-09-20T19:05:48.982500+0000","last_peered":"2024-09-20T19:05:48.982500+0000","last_clean":"2024-09-20T19:05:48.982500+0000","last_became_active":"2024-09-20T19:04:20.925394+0000","last_became_peered":"2024-09-20T19:04:20.925394+0000","last_unstale":"2024-09-20T19:05:48.982500+0000","last_undegraded":"2024-09-20T19:05:48.982500+0000","last_fullsized":"2024-09-20T19:05:48.982500+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:23.982813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:23.982813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"59'1","reported_seq":822,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:48.986942+0000","last_change":"2024-09-20T19:04:20.963496+0000","last_active":"2024-09-20T19:05:48.986942+0000","last_peered":"2024-09-20T19:05:48.986942+0000","last_clean":"2024-09-20T19:05:48.986942+0000","last_became_active":"2024-09-20T19:04:20.914145+0000","last_became_peered":"2024-09-20T19:04:20.914145+0000","last_unstale":"2024-09-20T19:05:48.986942+0000","last_undegraded":"2024-09-20T19:05:48.986942+0000","last_fullsized":"2024-09-20T19:05:48.986942+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:01:28.592653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:28.592653+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:29.742562+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"273'15","reported_seq":88,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:48.982958+0000","last_change":"2024-09-20T19:04:20.984223+0000","last_active":"2024-09-20T19:05:48.982958+0000","last_peered":"2024-09-20T19:05:48.982958+0000","last_clean":"2024-09-20T19:05:48.982958+0000","last_became_active":"2024-09-20T19:04:20.898235+0000","last_became_peered":"2024-09-20T19:04:20.898235+0000","last_unstale":"2024-09-20T19:05:48.982958+0000","last_undegraded":"2024-09-20T19:05:48.982958+0000","last_fullsized":"2024-09-20T19:05:48.982958+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:02:42.114331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:42.114331+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":745,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:48.986437+0000","last_change":"2024-09-20T19:01:51.564693+0000","last_active":"2024-09-20T19:05:48.986437+0000","last_peered":"2024-09-20T19:05:48.986437+0000","last_clean":"2024-09-20T19:05:48.986437+0000","last_became_active":"2024-09-20T18:49:49.961188+0000","last_became_peered":"2024-09-20T18:49:49.961188+0000","last_unstale":"2024-09-20T19:05:48.986437+0000","last_undegraded":"2024-09-20T19:05:48.986437+0000","last_fullsized":"2024-09-20T19:05:48.986437+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:51.564624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:51.564624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:41.667335+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":724,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.905162+0000","last_change":"2024-09-20T19:03:09.963421+0000","last_active":"2024-09-20T19:05:45.905162+0000","last_peered":"2024-09-20T19:05:45.905162+0000","last_clean":"2024-09-20T19:05:45.905162+0000","last_became_active":"2024-09-20T18:49:51.966715+0000","last_became_peered":"2024-09-20T18:49:51.966715+0000","last_unstale":"2024-09-20T19:05:45.905162+0000","last_undegraded":"2024-09-20T19:05:45.905162+0000","last_fullsized":"2024-09-20T19:05:45.905162+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:09.963298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:03:09.963298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":770,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:48.986642+0000","last_change":"2024-09-20T19:04:20.879227+0000","last_active":"2024-09-20T19:05:48.986642+0000","last_peered":"2024-09-20T19:05:48.986642+0000","last_clean":"2024-09-20T19:05:48.986642+0000","last_became_active":"2024-09-20T19:04:20.878905+0000","last_became_peered":"2024-09-20T19:04:20.878905+0000","last_unstale":"2024-09-20T19:05:48.986642+0000","last_undegraded":"2024-09-20T19:05:48.986642+0000","last_fullsized":"2024-09-20T19:05:48.986642+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:48.520017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:48.520017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:42.653960+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"273'7","reported_seq":84,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:48.982818+0000","last_change":"2024-09-20T19:04:20.907713+0000","last_active":"2024-09-20T19:05:48.982818+0000","last_peered":"2024-09-20T19:05:48.982818+0000","last_clean":"2024-09-20T19:05:48.982818+0000","last_became_active":"2024-09-20T19:04:20.888572+0000","last_became_peered":"2024-09-20T19:04:20.888572+0000","last_unstale":"2024-09-20T19:05:48.982818+0000","last_undegraded":"2024-09-20T19:05:48.982818+0000","last_fullsized":"2024-09-20T19:05:48.982818+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-20T18:58:33.660531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:58:33.660531+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":761,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.906238+0000","last_change":"2024-09-20T19:04:20.885873+0000","last_active":"2024-09-20T19:05:45.906238+0000","last_peered":"2024-09-20T19:05:45.906238+0000","last_clean":"2024-09-20T19:05:45.906238+0000","last_became_active":"2024-09-20T19:04:20.885417+0000","last_became_peered":"2024-09-20T19:04:20.885417+0000","last_unstale":"2024-09-20T19:05:45.906238+0000","last_undegraded":"2024-09-20T19:05:45.906238+0000","last_fullsized":"2024-09-20T19:05:45.906238+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:33.203312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:33.203312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":741,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:48.989859+0000","last_change":"2024-09-20T19:05:21.625378+0000","last_active":"2024-09-20T19:05:48.989859+0000","last_peered":"2024-09-20T19:05:48.989859+0000","last_clean":"2024-09-20T19:05:48.989859+0000","last_became_active":"2024-09-20T18:49:51.970663+0000","last_became_peered":"2024-09-20T18:49:51.970663+0000","last_unstale":"2024-09-20T19:05:48.989859+0000","last_undegraded":"2024-09-20T19:05:48.989859+0000","last_fullsized":"2024-09-20T19:05:48.989859+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:21.625311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:05:21.625311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:25.835298+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":78,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:48.983410+0000","last_change":"2024-09-20T19:04:20.907531+0000","last_active":"2024-09-20T19:05:48.983410+0000","last_peered":"2024-09-20T19:05:48.983410+0000","last_clean":"2024-09-20T19:05:48.983410+0000","last_became_active":"2024-09-20T19:04:20.906109+0000","last_became_peered":"2024-09-20T19:04:20.906109+0000","last_unstale":"2024-09-20T19:05:48.983410+0000","last_undegraded":"2024-09-20T19:05:48.983410+0000","last_fullsized":"2024-09-20T19:05:48.983410+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:00.906652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:00.906652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:39.140988+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":769,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.905640+0000","last_change":"2024-09-20T19:04:20.907057+0000","last_active":"2024-09-20T19:05:45.905640+0000","last_peered":"2024-09-20T19:05:45.905640+0000","last_clean":"2024-09-20T19:05:45.905640+0000","last_became_active":"2024-09-20T19:04:20.906691+0000","last_became_peered":"2024-09-20T19:04:20.906691+0000","last_unstale":"2024-09-20T19:05:45.905640+0000","last_undegraded":"2024-09-20T19:05:45.905640+0000","last_fullsized":"2024-09-20T19:05:45.905640+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:44.998147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:44.998147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"273'27","reported_seq":793,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.900312+0000","last_change":"2024-09-20T19:03:09.456720+0000","last_active":"2024-09-20T19:05:45.900312+0000","last_peered":"2024-09-20T19:05:45.900312+0000","last_clean":"2024-09-20T19:05:45.900312+0000","last_became_active":"2024-09-20T18:49:47.950939+0000","last_became_peered":"2024-09-20T18:49:47.950939+0000","last_unstale":"2024-09-20T19:05:45.900312+0000","last_undegraded":"2024-09-20T19:05:45.900312+0000","last_fullsized":"2024-09-20T19:05:45.900312+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'27","last_scrub_stamp":"2024-09-20T19:03:09.456631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:09.456631+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":752,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.908873+0000","last_change":"2024-09-20T19:04:20.890659+0000","last_active":"2024-09-20T19:05:45.908873+0000","last_peered":"2024-09-20T19:05:45.908873+0000","last_clean":"2024-09-20T19:05:45.908873+0000","last_became_active":"2024-09-20T19:04:20.890370+0000","last_became_peered":"2024-09-20T19:04:20.890370+0000","last_unstale":"2024-09-20T19:05:45.908873+0000","last_undegraded":"2024-09-20T19:05:45.908873+0000","last_fullsized":"2024-09-20T19:05:45.908873+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:05.888974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:05.888974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:26.204424+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":748,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:49.004190+0000","last_change":"2024-09-20T19:02:28.934597+0000","last_active":"2024-09-20T19:05:49.004190+0000","last_peered":"2024-09-20T19:05:49.004190+0000","last_clean":"2024-09-20T19:05:49.004190+0000","last_became_active":"2024-09-20T18:59:05.084496+0000","last_became_peered":"2024-09-20T18:59:05.084496+0000","last_unstale":"2024-09-20T19:05:49.004190+0000","last_undegraded":"2024-09-20T19:05:49.004190+0000","last_fullsized":"2024-09-20T19:05:49.004190+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:28.934264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:28.934264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:05.678324+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"59'1","reported_seq":82,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:48.982734+0000","last_change":"2024-09-20T19:04:20.905233+0000","last_active":"2024-09-20T19:05:48.982734+0000","last_peered":"2024-09-20T19:05:48.982734+0000","last_clean":"2024-09-20T19:05:48.982734+0000","last_became_active":"2024-09-20T19:04:20.887062+0000","last_became_peered":"2024-09-20T19:04:20.887062+0000","last_unstale":"2024-09-20T19:05:48.982734+0000","last_undegraded":"2024-09-20T19:05:48.982734+0000","last_fullsized":"2024-09-20T19:05:48.982734+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:03:08.218812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:03:08.218812+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":993,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"274'18","reported_seq":798,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:48.999290+0000","last_change":"2024-09-20T19:04:20.925690+0000","last_active":"2024-09-20T19:05:48.999290+0000","last_peered":"2024-09-20T19:05:48.999290+0000","last_clean":"2024-09-20T19:05:48.999290+0000","last_became_active":"2024-09-20T19:04:20.896343+0000","last_became_peered":"2024-09-20T19:04:20.896343+0000","last_unstale":"2024-09-20T19:05:48.999290+0000","last_undegraded":"2024-09-20T19:05:48.999290+0000","last_fullsized":"2024-09-20T19:05:48.999290+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:57:09.818999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:09.818999+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:29.636168+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":78,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:48.983980+0000","last_change":"2024-09-20T19:04:20.889786+0000","last_active":"2024-09-20T19:05:48.983980+0000","last_peered":"2024-09-20T19:05:48.983980+0000","last_clean":"2024-09-20T19:05:48.983980+0000","last_became_active":"2024-09-20T19:04:20.889430+0000","last_became_peered":"2024-09-20T19:04:20.889430+0000","last_unstale":"2024-09-20T19:05:48.983980+0000","last_undegraded":"2024-09-20T19:05:48.983980+0000","last_fullsized":"2024-09-20T19:05:48.983980+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:02.094473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:02.094473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":733,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.907145+0000","last_change":"2024-09-20T19:02:28.294390+0000","last_active":"2024-09-20T19:05:45.907145+0000","last_peered":"2024-09-20T19:05:45.907145+0000","last_clean":"2024-09-20T19:05:45.907145+0000","last_became_active":"2024-09-20T18:49:51.970058+0000","last_became_peered":"2024-09-20T18:49:51.970058+0000","last_unstale":"2024-09-20T19:05:45.907145+0000","last_undegraded":"2024-09-20T19:05:45.907145+0000","last_fullsized":"2024-09-20T19:05:45.907145+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T19:02:28.294292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:28.294292+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":769,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.907765+0000","last_change":"2024-09-20T19:05:22.202134+0000","last_active":"2024-09-20T19:05:45.907765+0000","last_peered":"2024-09-20T19:05:45.907765+0000","last_clean":"2024-09-20T19:05:45.907765+0000","last_became_active":"2024-09-20T19:04:20.881466+0000","last_became_peered":"2024-09-20T19:04:20.881466+0000","last_unstale":"2024-09-20T19:05:45.907765+0000","last_undegraded":"2024-09-20T19:05:45.907765+0000","last_fullsized":"2024-09-20T19:05:45.907765+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:05:22.201836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:22.201836+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:45.062354+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"274'20","reported_seq":806,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.905169+0000","last_change":"2024-09-20T19:03:16.982915+0000","last_active":"2024-09-20T19:05:45.905169+0000","last_peered":"2024-09-20T19:05:45.905169+0000","last_clean":"2024-09-20T19:05:45.905169+0000","last_became_active":"2024-09-20T19:03:16.968636+0000","last_became_peered":"2024-09-20T19:03:16.968636+0000","last_unstale":"2024-09-20T19:05:45.905169+0000","last_undegraded":"2024-09-20T19:05:45.905169+0000","last_fullsized":"2024-09-20T19:05:45.905169+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'20","last_scrub_stamp":"2024-09-20T19:02:35.096979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:35.096979+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"375'14","reported_seq":443,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.892823+0000","last_change":"2024-09-20T19:04:20.956210+0000","last_active":"2024-09-20T19:05:45.892823+0000","last_peered":"2024-09-20T19:05:45.892823+0000","last_clean":"2024-09-20T19:05:45.892823+0000","last_became_active":"2024-09-20T19:04:20.900782+0000","last_became_peered":"2024-09-20T19:04:20.900782+0000","last_unstale":"2024-09-20T19:05:45.892823+0000","last_undegraded":"2024-09-20T19:05:45.892823+0000","last_fullsized":"2024-09-20T19:05:45.892823+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"308'12","last_scrub_stamp":"2024-09-20T19:01:44.541156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:44.541156+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:35.235276+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"408'13978","reported_seq":31625,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:50.708230+0000","last_change":"2024-09-20T19:05:33.617599+0000","last_active":"2024-09-20T19:05:50.708230+0000","last_peered":"2024-09-20T19:05:50.708230+0000","last_clean":"2024-09-20T19:05:50.708230+0000","last_became_active":"2024-09-20T19:05:33.530382+0000","last_became_peered":"2024-09-20T19:05:33.530382+0000","last_unstale":"2024-09-20T19:05:50.708230+0000","last_undegraded":"2024-09-20T19:05:50.708230+0000","last_fullsized":"2024-09-20T19:05:50.708230+0000","mapping_epoch":402,"log_start":"377'9416","ondisk_log_start":"377'9416","created":325,"last_epoch_clean":403,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4562,"log_dups_size":0,"ondisk_log_size":4562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31981568,"num_objects":488,"num_object_clones":0,"num_object_copies":1464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":488,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13978,"num_write_kb":462912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8715,"num_bytes_recovered":571080704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"274'27","reported_seq":826,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.899920+0000","last_change":"2024-09-20T19:04:20.920811+0000","last_active":"2024-09-20T19:05:45.899920+0000","last_peered":"2024-09-20T19:05:45.899920+0000","last_clean":"2024-09-20T19:05:45.899920+0000","last_became_active":"2024-09-20T19:04:20.897049+0000","last_became_peered":"2024-09-20T19:04:20.897049+0000","last_unstale":"2024-09-20T19:05:45.899920+0000","last_undegraded":"2024-09-20T19:05:45.899920+0000","last_fullsized":"2024-09-20T19:05:45.899920+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'27","last_scrub_stamp":"2024-09-20T19:02:27.897138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:27.897138+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:33.546677+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":746,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:48.999780+0000","last_change":"2024-09-20T19:04:39.537914+0000","last_active":"2024-09-20T19:05:48.999780+0000","last_peered":"2024-09-20T19:05:48.999780+0000","last_clean":"2024-09-20T19:05:48.999780+0000","last_became_active":"2024-09-20T18:49:45.936104+0000","last_became_peered":"2024-09-20T18:49:45.936104+0000","last_unstale":"2024-09-20T19:05:48.999780+0000","last_undegraded":"2024-09-20T19:05:48.999780+0000","last_fullsized":"2024-09-20T19:05:48.999780+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:39.537818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:04:39.537818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"375'12","reported_seq":100,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:48.983501+0000","last_change":"2024-09-20T19:04:21.030980+0000","last_active":"2024-09-20T19:05:48.983501+0000","last_peered":"2024-09-20T19:05:48.983501+0000","last_clean":"2024-09-20T19:05:48.983501+0000","last_became_active":"2024-09-20T19:04:20.907228+0000","last_became_peered":"2024-09-20T19:04:20.907228+0000","last_unstale":"2024-09-20T19:05:48.983501+0000","last_undegraded":"2024-09-20T19:05:48.983501+0000","last_fullsized":"2024-09-20T19:05:48.983501+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"308'10","last_scrub_stamp":"2024-09-20T19:02:23.017626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:23.017626+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":734,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:48.986393+0000","last_change":"2024-09-20T19:02:22.736879+0000","last_active":"2024-09-20T19:05:48.986393+0000","last_peered":"2024-09-20T19:05:48.986393+0000","last_clean":"2024-09-20T19:05:48.986393+0000","last_became_active":"2024-09-20T18:49:51.971399+0000","last_became_peered":"2024-09-20T18:49:51.971399+0000","last_unstale":"2024-09-20T19:05:48.986393+0000","last_undegraded":"2024-09-20T19:05:48.986393+0000","last_fullsized":"2024-09-20T19:05:48.986393+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:22.736825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:22.736825+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.e","version":"408'14268","reported_seq":15368,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:50.697339+0000","last_change":"2024-09-20T19:04:21.970634+0000","last_active":"2024-09-20T19:05:50.697339+0000","last_peered":"2024-09-20T19:05:50.697339+0000","last_clean":"2024-09-20T19:05:50.697339+0000","last_became_active":"2024-09-20T19:04:21.956145+0000","last_became_peered":"2024-09-20T19:04:21.956145+0000","last_unstale":"2024-09-20T19:05:50.697339+0000","last_undegraded":"2024-09-20T19:05:50.697339+0000","last_fullsized":"2024-09-20T19:05:50.697339+0000","mapping_epoch":375,"log_start":"377'9628","ondisk_log_start":"377'9628","created":325,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4640,"log_dups_size":0,"ondisk_log_size":4640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34603008,"num_objects":528,"num_object_clones":0,"num_object_copies":1584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14268,"num_write_kb":473472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":453,"num_bytes_recovered":29687808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"273'30","reported_seq":800,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:48.999558+0000","last_change":"2024-09-20T19:03:06.038303+0000","last_active":"2024-09-20T19:05:48.999558+0000","last_peered":"2024-09-20T19:05:48.999558+0000","last_clean":"2024-09-20T19:05:48.999558+0000","last_became_active":"2024-09-20T18:49:48.499189+0000","last_became_peered":"2024-09-20T18:49:48.499189+0000","last_unstale":"2024-09-20T19:05:48.999558+0000","last_undegraded":"2024-09-20T19:05:48.999558+0000","last_fullsized":"2024-09-20T19:05:48.999558+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'30","last_scrub_stamp":"2024-09-20T19:03:06.038213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:06.038213+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":738,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:49.004628+0000","last_change":"2024-09-20T19:02:22.806587+0000","last_active":"2024-09-20T19:05:49.004628+0000","last_peered":"2024-09-20T19:05:49.004628+0000","last_clean":"2024-09-20T19:05:49.004628+0000","last_became_active":"2024-09-20T18:49:45.940544+0000","last_became_peered":"2024-09-20T18:49:45.940544+0000","last_unstale":"2024-09-20T19:05:49.004628+0000","last_undegraded":"2024-09-20T19:05:49.004628+0000","last_fullsized":"2024-09-20T19:05:49.004628+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:22.806509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:22.806509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:02.674388+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":78,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:48.983071+0000","last_change":"2024-09-20T19:04:20.889280+0000","last_active":"2024-09-20T19:05:48.983071+0000","last_peered":"2024-09-20T19:05:48.983071+0000","last_clean":"2024-09-20T19:05:48.983071+0000","last_became_active":"2024-09-20T19:04:20.888422+0000","last_became_peered":"2024-09-20T19:04:20.888422+0000","last_unstale":"2024-09-20T19:05:48.983071+0000","last_undegraded":"2024-09-20T19:05:48.983071+0000","last_fullsized":"2024-09-20T19:05:48.983071+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.229201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.229201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:33.231712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":86,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:48.983050+0000","last_change":"2024-09-20T19:05:47.121946+0000","last_active":"2024-09-20T19:05:48.983050+0000","last_peered":"2024-09-20T19:05:48.983050+0000","last_clean":"2024-09-20T19:05:48.983050+0000","last_became_active":"2024-09-20T19:04:20.889418+0000","last_became_peered":"2024-09-20T19:04:20.889418+0000","last_unstale":"2024-09-20T19:05:48.983050+0000","last_undegraded":"2024-09-20T19:05:48.983050+0000","last_fullsized":"2024-09-20T19:05:48.983050+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:47.121842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:05:47.121842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:15.049722+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"407'13689","reported_seq":13866,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:46.932980+0000","last_change":"2024-09-20T19:02:11.828279+0000","last_active":"2024-09-20T19:05:46.932980+0000","last_peered":"2024-09-20T19:05:46.932980+0000","last_clean":"2024-09-20T19:05:46.932980+0000","last_became_active":"2024-09-20T19:02:11.827468+0000","last_became_peered":"2024-09-20T19:02:11.827468+0000","last_unstale":"2024-09-20T19:05:46.932980+0000","last_undegraded":"2024-09-20T19:05:46.932980+0000","last_fullsized":"2024-09-20T19:05:46.932980+0000","mapping_epoch":325,"log_start":"366'7655","ondisk_log_start":"366'7655","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":6034,"log_dups_size":0,"ondisk_log_size":6034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45940736,"num_objects":701,"num_object_clones":0,"num_object_copies":2103,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":701,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13689,"num_write_kb":460480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"274'28","reported_seq":94,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:48.982972+0000","last_change":"2024-09-20T19:04:20.976682+0000","last_active":"2024-09-20T19:05:48.982972+0000","last_peered":"2024-09-20T19:05:48.982972+0000","last_clean":"2024-09-20T19:05:48.982972+0000","last_became_active":"2024-09-20T19:04:20.897444+0000","last_became_peered":"2024-09-20T19:04:20.897444+0000","last_unstale":"2024-09-20T19:05:48.982972+0000","last_undegraded":"2024-09-20T19:05:48.982972+0000","last_fullsized":"2024-09-20T19:05:48.982972+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'28","last_scrub_stamp":"2024-09-20T19:03:06.269623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:06.269623+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":41,"num_read_kb":34,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":779,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:47.629956+0000","last_change":"2024-09-20T19:05:47.629956+0000","last_active":"2024-09-20T19:05:47.629956+0000","last_peered":"2024-09-20T19:05:47.629956+0000","last_clean":"2024-09-20T19:05:47.629956+0000","last_became_active":"2024-09-20T19:04:20.906790+0000","last_became_peered":"2024-09-20T19:04:20.906790+0000","last_unstale":"2024-09-20T19:05:47.629956+0000","last_undegraded":"2024-09-20T19:05:47.629956+0000","last_fullsized":"2024-09-20T19:05:47.629956+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:05:47.629892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:47.629892+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:06.773486+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"375'13","reported_seq":986,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:48.999369+0000","last_change":"2024-09-20T19:04:20.916872+0000","last_active":"2024-09-20T19:05:48.999369+0000","last_peered":"2024-09-20T19:05:48.999369+0000","last_clean":"2024-09-20T19:05:48.999369+0000","last_became_active":"2024-09-20T19:04:20.896424+0000","last_became_peered":"2024-09-20T19:04:20.896424+0000","last_unstale":"2024-09-20T19:05:48.999369+0000","last_undegraded":"2024-09-20T19:05:48.999369+0000","last_fullsized":"2024-09-20T19:05:48.999369+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:58:23.114371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:23.114371+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:38.642422+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":739,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.906490+0000","last_change":"2024-09-20T18:59:05.082117+0000","last_active":"2024-09-20T19:05:45.906490+0000","last_peered":"2024-09-20T19:05:45.906490+0000","last_clean":"2024-09-20T19:05:45.906490+0000","last_became_active":"2024-09-20T18:59:05.081814+0000","last_became_peered":"2024-09-20T18:59:05.081814+0000","last_unstale":"2024-09-20T19:05:45.906490+0000","last_undegraded":"2024-09-20T19:05:45.906490+0000","last_fullsized":"2024-09-20T19:05:45.906490+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:29.530001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:29.530001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:36.792476+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"407'13669","reported_seq":31479,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:46.354166+0000","last_change":"2024-09-20T19:05:30.912897+0000","last_active":"2024-09-20T19:05:46.354166+0000","last_peered":"2024-09-20T19:05:46.354166+0000","last_clean":"2024-09-20T19:05:46.354166+0000","last_became_active":"2024-09-20T19:05:30.889717+0000","last_became_peered":"2024-09-20T19:05:30.889717+0000","last_unstale":"2024-09-20T19:05:46.354166+0000","last_undegraded":"2024-09-20T19:05:46.354166+0000","last_fullsized":"2024-09-20T19:05:46.354166+0000","mapping_epoch":400,"log_start":"372'8555","ondisk_log_start":"372'8555","created":325,"last_epoch_clean":401,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":5114,"log_dups_size":0,"ondisk_log_size":5114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51183616,"num_objects":781,"num_object_clones":0,"num_object_copies":2343,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13669,"num_write_kb":462400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8799,"num_bytes_recovered":576651264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"274'20","reported_seq":592,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:48.986848+0000","last_change":"2024-09-20T19:05:48.741237+0000","last_active":"2024-09-20T19:05:48.986848+0000","last_peered":"2024-09-20T19:05:48.986848+0000","last_clean":"2024-09-20T19:05:48.986848+0000","last_became_active":"2024-09-20T19:04:49.419140+0000","last_became_peered":"2024-09-20T19:04:49.419140+0000","last_unstale":"2024-09-20T19:05:48.986848+0000","last_undegraded":"2024-09-20T19:05:48.986848+0000","last_fullsized":"2024-09-20T19:05:48.986848+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'20","last_scrub_stamp":"2024-09-20T19:05:48.741059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:48.741059+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:13.202074+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":762,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:48.987037+0000","last_change":"2024-09-20T19:02:30.686939+0000","last_active":"2024-09-20T19:05:48.987037+0000","last_peered":"2024-09-20T19:05:48.987037+0000","last_clean":"2024-09-20T19:05:48.987037+0000","last_became_active":"2024-09-20T18:49:45.942496+0000","last_became_peered":"2024-09-20T18:49:45.942496+0000","last_unstale":"2024-09-20T19:05:48.987037+0000","last_undegraded":"2024-09-20T19:05:48.987037+0000","last_fullsized":"2024-09-20T19:05:48.987037+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:30.686872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:30.686872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"262'10","reported_seq":960,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.905324+0000","last_change":"2024-09-20T19:02:04.994109+0000","last_active":"2024-09-20T19:05:45.905324+0000","last_peered":"2024-09-20T19:05:45.905324+0000","last_clean":"2024-09-20T19:05:45.905324+0000","last_became_active":"2024-09-20T18:59:05.080183+0000","last_became_peered":"2024-09-20T18:59:05.080183+0000","last_unstale":"2024-09-20T19:05:45.905324+0000","last_undegraded":"2024-09-20T19:05:45.905324+0000","last_fullsized":"2024-09-20T19:05:45.905324+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'10","last_scrub_stamp":"2024-09-20T19:02:04.994047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:04.994047+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":717,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.905304+0000","last_change":"2024-09-20T19:01:37.900410+0000","last_active":"2024-09-20T19:05:45.905304+0000","last_peered":"2024-09-20T19:05:45.905304+0000","last_clean":"2024-09-20T19:05:45.905304+0000","last_became_active":"2024-09-20T18:49:51.964780+0000","last_became_peered":"2024-09-20T18:49:51.964780+0000","last_unstale":"2024-09-20T19:05:45.905304+0000","last_undegraded":"2024-09-20T19:05:45.905304+0000","last_fullsized":"2024-09-20T19:05:45.905304+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:37.900328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:37.900328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.d","version":"407'14203","reported_seq":23487,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:46.913847+0000","last_change":"2024-09-20T19:05:43.612046+0000","last_active":"2024-09-20T19:05:46.913847+0000","last_peered":"2024-09-20T19:05:46.913847+0000","last_clean":"2024-09-20T19:05:46.913847+0000","last_became_active":"2024-09-20T19:05:22.993748+0000","last_became_peered":"2024-09-20T19:05:22.993748+0000","last_unstale":"2024-09-20T19:05:46.913847+0000","last_undegraded":"2024-09-20T19:05:46.913847+0000","last_fullsized":"2024-09-20T19:05:46.913847+0000","mapping_epoch":395,"log_start":"367'8117","ondisk_log_start":"367'8117","created":325,"last_epoch_clean":396,"parent":"0.0","parent_split_bits":0,"last_scrub":"406'13977","last_scrub_stamp":"2024-09-20T19:05:43.611752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:05:43.611752+0000","objects_scrubbed":985,"log_size":6086,"log_dups_size":0,"ondisk_log_size":6086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:51.444455+0000","scrub_duration":433,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48758808,"num_objects":745,"num_object_clones":0,"num_object_copies":2235,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":745,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14203,"num_write_kb":478273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4538,"num_bytes_recovered":189071384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"274'29","reported_seq":792,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.905462+0000","last_change":"2024-09-20T19:05:43.631990+0000","last_active":"2024-09-20T19:05:45.905462+0000","last_peered":"2024-09-20T19:05:45.905462+0000","last_clean":"2024-09-20T19:05:45.905462+0000","last_became_active":"2024-09-20T18:49:48.497736+0000","last_became_peered":"2024-09-20T18:49:48.497736+0000","last_unstale":"2024-09-20T19:05:45.905462+0000","last_undegraded":"2024-09-20T19:05:45.905462+0000","last_fullsized":"2024-09-20T19:05:45.905462+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'29","last_scrub_stamp":"2024-09-20T19:05:43.631845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:43.631845+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:56.642378+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":736,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.908548+0000","last_change":"2024-09-20T19:02:41.859231+0000","last_active":"2024-09-20T19:05:45.908548+0000","last_peered":"2024-09-20T19:05:45.908548+0000","last_clean":"2024-09-20T19:05:45.908548+0000","last_became_active":"2024-09-20T18:49:45.942253+0000","last_became_peered":"2024-09-20T18:49:45.942253+0000","last_unstale":"2024-09-20T19:05:45.908548+0000","last_undegraded":"2024-09-20T19:05:45.908548+0000","last_fullsized":"2024-09-20T19:05:45.908548+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:41.859168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:41.859168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":323,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.892230+0000","last_change":"2024-09-20T19:02:27.639446+0000","last_active":"2024-09-20T19:05:45.892230+0000","last_peered":"2024-09-20T19:05:45.892230+0000","last_clean":"2024-09-20T19:05:45.892230+0000","last_became_active":"2024-09-20T18:59:05.085636+0000","last_became_peered":"2024-09-20T18:59:05.085636+0000","last_unstale":"2024-09-20T19:05:45.892230+0000","last_undegraded":"2024-09-20T19:05:45.892230+0000","last_fullsized":"2024-09-20T19:05:45.892230+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:27.639384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:27.639384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":754,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.907527+0000","last_change":"2024-09-20T19:02:23.886386+0000","last_active":"2024-09-20T19:05:45.907527+0000","last_peered":"2024-09-20T19:05:45.907527+0000","last_clean":"2024-09-20T19:05:45.907527+0000","last_became_active":"2024-09-20T18:59:05.085131+0000","last_became_peered":"2024-09-20T18:59:05.085131+0000","last_unstale":"2024-09-20T19:05:45.907527+0000","last_undegraded":"2024-09-20T19:05:45.907527+0000","last_fullsized":"2024-09-20T19:05:45.907527+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:23.886308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:23.886308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:38.961464+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"407'13608","reported_seq":13849,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:46.895931+0000","last_change":"2024-09-20T19:03:15.459244+0000","last_active":"2024-09-20T19:05:46.895931+0000","last_peered":"2024-09-20T19:05:46.895931+0000","last_clean":"2024-09-20T19:05:46.895931+0000","last_became_active":"2024-09-20T19:02:11.827305+0000","last_became_peered":"2024-09-20T19:02:11.827305+0000","last_unstale":"2024-09-20T19:05:46.895931+0000","last_undegraded":"2024-09-20T19:05:46.895931+0000","last_fullsized":"2024-09-20T19:05:46.895931+0000","mapping_epoch":325,"log_start":"366'7602","ondisk_log_start":"366'7602","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"350'5605","last_scrub_stamp":"2024-09-20T19:03:15.459174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:03:15.459174+0000","objects_scrubbed":5605,"log_size":6006,"log_dups_size":0,"ondisk_log_size":6006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":598,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49676288,"num_objects":758,"num_object_clones":0,"num_object_copies":2274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":758,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13608,"num_write_kb":459712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"353'41","reported_seq":1036,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.907875+0000","last_change":"2024-09-20T19:03:17.024385+0000","last_active":"2024-09-20T19:05:45.907875+0000","last_peered":"2024-09-20T19:05:45.907875+0000","last_clean":"2024-09-20T19:05:45.907875+0000","last_became_active":"2024-09-20T19:03:16.976916+0000","last_became_peered":"2024-09-20T19:03:16.976916+0000","last_unstale":"2024-09-20T19:05:45.907875+0000","last_undegraded":"2024-09-20T19:05:45.907875+0000","last_fullsized":"2024-09-20T19:05:45.907875+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'40","last_scrub_stamp":"2024-09-20T19:02:36.900916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:36.900916+0000","objects_scrubbed":10,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":78,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:48.983001+0000","last_change":"2024-09-20T19:04:20.899630+0000","last_active":"2024-09-20T19:05:48.983001+0000","last_peered":"2024-09-20T19:05:48.983001+0000","last_clean":"2024-09-20T19:05:48.983001+0000","last_became_active":"2024-09-20T19:04:20.899352+0000","last_became_peered":"2024-09-20T19:04:20.899352+0000","last_unstale":"2024-09-20T19:05:48.983001+0000","last_undegraded":"2024-09-20T19:05:48.983001+0000","last_fullsized":"2024-09-20T19:05:48.983001+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:00:31.630361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:00:31.630361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":802,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:48.987053+0000","last_change":"2024-09-20T19:03:11.910684+0000","last_active":"2024-09-20T19:05:48.987053+0000","last_peered":"2024-09-20T19:05:48.987053+0000","last_clean":"2024-09-20T19:05:48.987053+0000","last_became_active":"2024-09-20T18:59:05.086786+0000","last_became_peered":"2024-09-20T18:59:05.086786+0000","last_unstale":"2024-09-20T19:05:48.987053+0000","last_undegraded":"2024-09-20T19:05:48.987053+0000","last_fullsized":"2024-09-20T19:05:48.987053+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-20T19:03:11.909394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:03:11.909394+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0.0012781159999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":770,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.906540+0000","last_change":"2024-09-20T19:04:20.907534+0000","last_active":"2024-09-20T19:05:45.906540+0000","last_peered":"2024-09-20T19:05:45.906540+0000","last_clean":"2024-09-20T19:05:45.906540+0000","last_became_active":"2024-09-20T19:04:20.907038+0000","last_became_peered":"2024-09-20T19:04:20.907038+0000","last_unstale":"2024-09-20T19:05:45.906540+0000","last_undegraded":"2024-09-20T19:05:45.906540+0000","last_fullsized":"2024-09-20T19:05:45.906540+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:15.034340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:03:15.034340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":743,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:48.998886+0000","last_change":"2024-09-20T19:04:20.896616+0000","last_active":"2024-09-20T19:05:48.998886+0000","last_peered":"2024-09-20T19:05:48.998886+0000","last_clean":"2024-09-20T19:05:48.998886+0000","last_became_active":"2024-09-20T19:04:20.896281+0000","last_became_peered":"2024-09-20T19:04:20.896281+0000","last_unstale":"2024-09-20T19:05:48.998886+0000","last_undegraded":"2024-09-20T19:05:48.998886+0000","last_fullsized":"2024-09-20T19:05:48.998886+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:04.526683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:04.526683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:00.655405+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"408'14213","reported_seq":14408,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:50.750648+0000","last_change":"2024-09-20T19:05:46.982910+0000","last_active":"2024-09-20T19:05:50.750648+0000","last_peered":"2024-09-20T19:05:50.750648+0000","last_clean":"2024-09-20T19:05:50.750648+0000","last_became_active":"2024-09-20T19:02:11.824846+0000","last_became_peered":"2024-09-20T19:02:11.824846+0000","last_unstale":"2024-09-20T19:05:50.750648+0000","last_undegraded":"2024-09-20T19:05:50.750648+0000","last_fullsized":"2024-09-20T19:05:50.750648+0000","mapping_epoch":325,"log_start":"372'8527","ondisk_log_start":"372'8527","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"407'13951","last_scrub_stamp":"2024-09-20T19:05:46.982716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:05:46.982716+0000","objects_scrubbed":795,"log_size":5686,"log_dups_size":0,"ondisk_log_size":5686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:15.410021+0000","scrub_duration":218,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34537472,"num_objects":527,"num_object_clones":0,"num_object_copies":1581,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":527,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14213,"num_write_kb":471680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"273'17","reported_seq":789,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.909340+0000","last_change":"2024-09-20T19:05:43.974098+0000","last_active":"2024-09-20T19:05:45.909340+0000","last_peered":"2024-09-20T19:05:45.909340+0000","last_clean":"2024-09-20T19:05:45.909340+0000","last_became_active":"2024-09-20T18:49:47.950904+0000","last_became_peered":"2024-09-20T18:49:47.950904+0000","last_unstale":"2024-09-20T19:05:45.909340+0000","last_undegraded":"2024-09-20T19:05:45.909340+0000","last_fullsized":"2024-09-20T19:05:45.909340+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'17","last_scrub_stamp":"2024-09-20T19:05:43.974013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:43.974013+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:01.950625+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":36,"num_read_kb":25,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":766,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.910239+0000","last_change":"2024-09-20T19:02:29.840761+0000","last_active":"2024-09-20T19:05:45.910239+0000","last_peered":"2024-09-20T19:05:45.910239+0000","last_clean":"2024-09-20T19:05:45.910239+0000","last_became_active":"2024-09-20T18:59:05.079789+0000","last_became_peered":"2024-09-20T18:59:05.079789+0000","last_unstale":"2024-09-20T19:05:45.910239+0000","last_undegraded":"2024-09-20T19:05:45.910239+0000","last_fullsized":"2024-09-20T19:05:45.910239+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:29.840601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:29.840601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:33.076198+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":748,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.901012+0000","last_change":"2024-09-20T19:01:58.789304+0000","last_active":"2024-09-20T19:05:45.901012+0000","last_peered":"2024-09-20T19:05:45.901012+0000","last_clean":"2024-09-20T19:05:45.901012+0000","last_became_active":"2024-09-20T18:49:44.937135+0000","last_became_peered":"2024-09-20T18:49:44.937135+0000","last_unstale":"2024-09-20T19:05:45.901012+0000","last_undegraded":"2024-09-20T19:05:45.901012+0000","last_fullsized":"2024-09-20T19:05:45.901012+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:01:58.789251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:01:58.789251+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:05.405452+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00095558800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":723,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.907087+0000","last_change":"2024-09-20T19:01:17.747452+0000","last_active":"2024-09-20T19:05:45.907087+0000","last_peered":"2024-09-20T19:05:45.907087+0000","last_clean":"2024-09-20T19:05:45.907087+0000","last_became_active":"2024-09-20T18:49:49.954424+0000","last_became_peered":"2024-09-20T18:49:49.954424+0000","last_unstale":"2024-09-20T19:05:45.907087+0000","last_undegraded":"2024-09-20T19:05:45.907087+0000","last_fullsized":"2024-09-20T19:05:45.907087+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:17.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:17.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:58.423341+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":734,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.907041+0000","last_change":"2024-09-20T19:01:15.203591+0000","last_active":"2024-09-20T19:05:45.907041+0000","last_peered":"2024-09-20T19:05:45.907041+0000","last_clean":"2024-09-20T19:05:45.907041+0000","last_became_active":"2024-09-20T18:59:05.081459+0000","last_became_peered":"2024-09-20T18:59:05.081459+0000","last_unstale":"2024-09-20T19:05:45.907041+0000","last_undegraded":"2024-09-20T19:05:45.907041+0000","last_fullsized":"2024-09-20T19:05:45.907041+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:15.203529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:15.203529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:37.777620+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"407'13725","reported_seq":13902,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:47.633437+0000","last_change":"2024-09-20T19:02:11.828477+0000","last_active":"2024-09-20T19:05:47.633437+0000","last_peered":"2024-09-20T19:05:47.633437+0000","last_clean":"2024-09-20T19:05:47.633437+0000","last_became_active":"2024-09-20T19:02:11.827996+0000","last_became_peered":"2024-09-20T19:02:11.827996+0000","last_unstale":"2024-09-20T19:05:47.633437+0000","last_undegraded":"2024-09-20T19:05:47.633437+0000","last_fullsized":"2024-09-20T19:05:47.633437+0000","mapping_epoch":325,"log_start":"370'8284","ondisk_log_start":"370'8284","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":5441,"log_dups_size":0,"ondisk_log_size":5441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45940736,"num_objects":701,"num_object_clones":0,"num_object_copies":2103,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":701,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13725,"num_write_kb":461632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"273'24","reported_seq":785,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.905270+0000","last_change":"2024-09-20T18:59:06.022021+0000","last_active":"2024-09-20T19:05:45.905270+0000","last_peered":"2024-09-20T19:05:45.905270+0000","last_clean":"2024-09-20T19:05:45.905270+0000","last_became_active":"2024-09-20T18:49:47.949230+0000","last_became_peered":"2024-09-20T18:49:47.949230+0000","last_unstale":"2024-09-20T19:05:45.905270+0000","last_undegraded":"2024-09-20T19:05:45.905270+0000","last_fullsized":"2024-09-20T19:05:45.905270+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:59:06.021962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:59:06.021962+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:34.733375+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":770,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.904782+0000","last_change":"2024-09-20T19:04:20.907250+0000","last_active":"2024-09-20T19:05:45.904782+0000","last_peered":"2024-09-20T19:05:45.904782+0000","last_clean":"2024-09-20T19:05:45.904782+0000","last_became_active":"2024-09-20T19:04:20.906969+0000","last_became_peered":"2024-09-20T19:04:20.906969+0000","last_unstale":"2024-09-20T19:05:45.904782+0000","last_undegraded":"2024-09-20T19:05:45.904782+0000","last_fullsized":"2024-09-20T19:05:45.904782+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:24.984022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:24.984022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":1710,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.906135+0000","last_change":"2024-09-20T19:02:02.494344+0000","last_active":"2024-09-20T19:05:45.906135+0000","last_peered":"2024-09-20T19:05:45.906135+0000","last_clean":"2024-09-20T19:05:45.906135+0000","last_became_active":"2024-09-20T18:49:44.936819+0000","last_became_peered":"2024-09-20T18:49:44.936819+0000","last_unstale":"2024-09-20T19:05:45.906135+0000","last_undegraded":"2024-09-20T19:05:45.906135+0000","last_fullsized":"2024-09-20T19:05:45.906135+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-20T19:02:02.494291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:02:02.494291+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:39.801016+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0010996199999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":953,"num_read_kb":948,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":746,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:48.987703+0000","last_change":"2024-09-20T19:02:27.785566+0000","last_active":"2024-09-20T19:05:48.987703+0000","last_peered":"2024-09-20T19:05:48.987703+0000","last_clean":"2024-09-20T19:05:48.987703+0000","last_became_active":"2024-09-20T18:49:49.961066+0000","last_became_peered":"2024-09-20T18:49:49.961066+0000","last_unstale":"2024-09-20T19:05:48.987703+0000","last_undegraded":"2024-09-20T19:05:48.987703+0000","last_fullsized":"2024-09-20T19:05:48.987703+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:27.785524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:27.785524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":740,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.907574+0000","last_change":"2024-09-20T19:02:26.861253+0000","last_active":"2024-09-20T19:05:45.907574+0000","last_peered":"2024-09-20T19:05:45.907574+0000","last_clean":"2024-09-20T19:05:45.907574+0000","last_became_active":"2024-09-20T18:49:51.970343+0000","last_became_peered":"2024-09-20T18:49:51.970343+0000","last_unstale":"2024-09-20T19:05:45.907574+0000","last_undegraded":"2024-09-20T19:05:45.907574+0000","last_fullsized":"2024-09-20T19:05:45.907574+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:26.861185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:26.861185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:32.099095+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"407'13757","reported_seq":13934,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:46.932160+0000","last_change":"2024-09-20T19:02:11.828065+0000","last_active":"2024-09-20T19:05:46.932160+0000","last_peered":"2024-09-20T19:05:46.932160+0000","last_clean":"2024-09-20T19:05:46.932160+0000","last_became_active":"2024-09-20T19:02:11.827628+0000","last_became_peered":"2024-09-20T19:02:11.827628+0000","last_unstale":"2024-09-20T19:05:46.932160+0000","last_undegraded":"2024-09-20T19:05:46.932160+0000","last_fullsized":"2024-09-20T19:05:46.932160+0000","mapping_epoch":325,"log_start":"366'7727","ondisk_log_start":"366'7727","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":6030,"log_dups_size":0,"ondisk_log_size":6030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:40.226338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52494336,"num_objects":801,"num_object_clones":0,"num_object_copies":2403,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":801,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13757,"num_write_kb":465856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"375'34","reported_seq":495,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.893178+0000","last_change":"2024-09-20T19:04:20.988428+0000","last_active":"2024-09-20T19:05:45.893178+0000","last_peered":"2024-09-20T19:05:45.893178+0000","last_clean":"2024-09-20T19:05:45.893178+0000","last_became_active":"2024-09-20T19:04:20.928727+0000","last_became_peered":"2024-09-20T19:04:20.928727+0000","last_unstale":"2024-09-20T19:05:45.893178+0000","last_undegraded":"2024-09-20T19:05:45.893178+0000","last_fullsized":"2024-09-20T19:05:45.893178+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"308'32","last_scrub_stamp":"2024-09-20T19:02:25.699337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:25.699337+0000","objects_scrubbed":8,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":462,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":768,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:48.999203+0000","last_change":"2024-09-20T19:05:07.665281+0000","last_active":"2024-09-20T19:05:48.999203+0000","last_peered":"2024-09-20T19:05:48.999203+0000","last_clean":"2024-09-20T19:05:48.999203+0000","last_became_active":"2024-09-20T18:59:05.079111+0000","last_became_peered":"2024-09-20T18:59:05.079111+0000","last_unstale":"2024-09-20T19:05:48.999203+0000","last_undegraded":"2024-09-20T19:05:48.999203+0000","last_fullsized":"2024-09-20T19:05:48.999203+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:07.665167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:07.665167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:16.643707+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":814,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.906190+0000","last_change":"2024-09-20T19:03:14.368338+0000","last_active":"2024-09-20T19:05:45.906190+0000","last_peered":"2024-09-20T19:05:45.906190+0000","last_clean":"2024-09-20T19:05:45.906190+0000","last_became_active":"2024-09-20T18:59:05.086103+0000","last_became_peered":"2024-09-20T18:59:05.086103+0000","last_unstale":"2024-09-20T19:05:45.906190+0000","last_undegraded":"2024-09-20T19:05:45.906190+0000","last_fullsized":"2024-09-20T19:05:45.906190+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:03:14.368279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:03:14.368279+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":729,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.907226+0000","last_change":"2024-09-20T19:04:35.533709+0000","last_active":"2024-09-20T19:05:45.907226+0000","last_peered":"2024-09-20T19:05:45.907226+0000","last_clean":"2024-09-20T19:05:45.907226+0000","last_became_active":"2024-09-20T18:49:49.956128+0000","last_became_peered":"2024-09-20T18:49:49.956128+0000","last_unstale":"2024-09-20T19:05:45.907226+0000","last_undegraded":"2024-09-20T19:05:45.907226+0000","last_fullsized":"2024-09-20T19:05:45.907226+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:35.533633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:35.533633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:59.293116+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":767,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:48.987510+0000","last_change":"2024-09-20T19:04:20.899776+0000","last_active":"2024-09-20T19:05:48.987510+0000","last_peered":"2024-09-20T19:05:48.987510+0000","last_clean":"2024-09-20T19:05:48.987510+0000","last_became_active":"2024-09-20T19:04:20.899321+0000","last_became_peered":"2024-09-20T19:04:20.899321+0000","last_unstale":"2024-09-20T19:05:48.987510+0000","last_undegraded":"2024-09-20T19:05:48.987510+0000","last_fullsized":"2024-09-20T19:05:48.987510+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:30.557368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:30.557368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:39.737278+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.0","version":"407'13879","reported_seq":23297,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:46.921091+0000","last_change":"2024-09-20T19:05:21.939972+0000","last_active":"2024-09-20T19:05:46.921091+0000","last_peered":"2024-09-20T19:05:46.921091+0000","last_clean":"2024-09-20T19:05:46.921091+0000","last_became_active":"2024-09-20T19:05:21.905829+0000","last_became_peered":"2024-09-20T19:05:21.905829+0000","last_unstale":"2024-09-20T19:05:46.921091+0000","last_undegraded":"2024-09-20T19:05:46.921091+0000","last_fullsized":"2024-09-20T19:05:46.921091+0000","mapping_epoch":394,"log_start":"367'7839","ondisk_log_start":"367'7839","created":325,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":6040,"log_dups_size":0,"ondisk_log_size":6040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:42.201962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48562176,"num_objects":741,"num_object_clones":0,"num_object_copies":2223,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13879,"num_write_kb":467840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4613,"num_bytes_recovered":200736768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"274'19","reported_seq":98,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:50.066533+0000","last_change":"2024-09-20T19:05:50.066533+0000","last_active":"2024-09-20T19:05:50.066533+0000","last_peered":"2024-09-20T19:05:50.066533+0000","last_clean":"2024-09-20T19:05:50.066533+0000","last_became_active":"2024-09-20T19:04:20.897175+0000","last_became_peered":"2024-09-20T19:04:20.897175+0000","last_unstale":"2024-09-20T19:05:50.066533+0000","last_undegraded":"2024-09-20T19:05:50.066533+0000","last_fullsized":"2024-09-20T19:05:50.066533+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'19","last_scrub_stamp":"2024-09-20T19:05:50.066359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:50.066359+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:56.712506+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":78,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:48.983152+0000","last_change":"2024-09-20T19:04:20.887455+0000","last_active":"2024-09-20T19:05:48.983152+0000","last_peered":"2024-09-20T19:05:48.983152+0000","last_clean":"2024-09-20T19:05:48.983152+0000","last_became_active":"2024-09-20T19:04:20.887102+0000","last_became_peered":"2024-09-20T19:04:20.887102+0000","last_unstale":"2024-09-20T19:05:48.983152+0000","last_undegraded":"2024-09-20T19:05:48.983152+0000","last_fullsized":"2024-09-20T19:05:48.983152+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:32.693993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:32.693993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":785,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:48.987376+0000","last_change":"2024-09-20T19:04:20.899830+0000","last_active":"2024-09-20T19:05:48.987376+0000","last_peered":"2024-09-20T19:05:48.987376+0000","last_clean":"2024-09-20T19:05:48.987376+0000","last_became_active":"2024-09-20T19:04:20.899331+0000","last_became_peered":"2024-09-20T19:04:20.899331+0000","last_unstale":"2024-09-20T19:05:48.987376+0000","last_undegraded":"2024-09-20T19:05:48.987376+0000","last_fullsized":"2024-09-20T19:05:48.987376+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:38.463404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:38.463404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:26.726228+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":748,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:49.002112+0000","last_change":"2024-09-20T19:04:35.529672+0000","last_active":"2024-09-20T19:05:49.002112+0000","last_peered":"2024-09-20T19:05:49.002112+0000","last_clean":"2024-09-20T19:05:49.002112+0000","last_became_active":"2024-09-20T18:59:05.079897+0000","last_became_peered":"2024-09-20T18:59:05.079897+0000","last_unstale":"2024-09-20T19:05:49.002112+0000","last_undegraded":"2024-09-20T19:05:49.002112+0000","last_fullsized":"2024-09-20T19:05:49.002112+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:35.529584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:04:35.529584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:57.166916+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"407'14102","reported_seq":14390,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:46.607317+0000","last_change":"2024-09-20T19:05:17.450180+0000","last_active":"2024-09-20T19:05:46.607317+0000","last_peered":"2024-09-20T19:05:46.607317+0000","last_clean":"2024-09-20T19:05:46.607317+0000","last_became_active":"2024-09-20T19:02:11.825011+0000","last_became_peered":"2024-09-20T19:02:11.825011+0000","last_unstale":"2024-09-20T19:05:46.607317+0000","last_undegraded":"2024-09-20T19:05:46.607317+0000","last_fullsized":"2024-09-20T19:05:46.607317+0000","mapping_epoch":325,"log_start":"363'7528","ondisk_log_start":"363'7528","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"392'12286","last_scrub_stamp":"2024-09-20T19:05:17.450066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:05:17.450066+0000","objects_scrubbed":2624,"log_size":6574,"log_dups_size":0,"ondisk_log_size":6574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:20.515552+0000","scrub_duration":996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50724864,"num_objects":774,"num_object_clones":0,"num_object_copies":2322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14102,"num_write_kb":476032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"274'31","reported_seq":96,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:48.983495+0000","last_change":"2024-09-20T19:04:20.949305+0000","last_active":"2024-09-20T19:05:48.983495+0000","last_peered":"2024-09-20T19:05:48.983495+0000","last_clean":"2024-09-20T19:05:48.983495+0000","last_became_active":"2024-09-20T19:04:20.883306+0000","last_became_peered":"2024-09-20T19:04:20.883306+0000","last_unstale":"2024-09-20T19:05:48.983495+0000","last_undegraded":"2024-09-20T19:05:48.983495+0000","last_fullsized":"2024-09-20T19:05:48.983495+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'31","last_scrub_stamp":"2024-09-20T19:03:12.196851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:12.196851+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":49,"num_read_kb":39,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":340,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.892366+0000","last_change":"2024-09-20T19:04:20.899712+0000","last_active":"2024-09-20T19:05:45.892366+0000","last_peered":"2024-09-20T19:05:45.892366+0000","last_clean":"2024-09-20T19:05:45.892366+0000","last_became_active":"2024-09-20T19:04:20.899415+0000","last_became_peered":"2024-09-20T19:04:20.899415+0000","last_unstale":"2024-09-20T19:05:45.892366+0000","last_undegraded":"2024-09-20T19:05:45.892366+0000","last_fullsized":"2024-09-20T19:05:45.892366+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:42.479489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:42.479489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:36.220255+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":769,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:46.866958+0000","last_change":"2024-09-20T19:05:46.866958+0000","last_active":"2024-09-20T19:05:46.866958+0000","last_peered":"2024-09-20T19:05:46.866958+0000","last_clean":"2024-09-20T19:05:46.866958+0000","last_became_active":"2024-09-20T19:04:20.891772+0000","last_became_peered":"2024-09-20T19:04:20.891772+0000","last_unstale":"2024-09-20T19:05:46.866958+0000","last_undegraded":"2024-09-20T19:05:46.866958+0000","last_fullsized":"2024-09-20T19:05:46.866958+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:46.866777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:05:46.866777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:55.257561+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":760,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.900038+0000","last_change":"2024-09-20T19:04:20.897937+0000","last_active":"2024-09-20T19:05:45.900038+0000","last_peered":"2024-09-20T19:05:45.900038+0000","last_clean":"2024-09-20T19:05:45.900038+0000","last_became_active":"2024-09-20T19:04:20.897569+0000","last_became_peered":"2024-09-20T19:04:20.897569+0000","last_unstale":"2024-09-20T19:05:45.900038+0000","last_undegraded":"2024-09-20T19:05:45.900038+0000","last_fullsized":"2024-09-20T19:05:45.900038+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:31.687188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:31.687188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:24.424476+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2","version":"407'13919","reported_seq":15443,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:46.990870+0000","last_change":"2024-09-20T19:04:21.974196+0000","last_active":"2024-09-20T19:05:46.990870+0000","last_peered":"2024-09-20T19:05:46.990870+0000","last_clean":"2024-09-20T19:05:46.990870+0000","last_became_active":"2024-09-20T19:04:21.956633+0000","last_became_peered":"2024-09-20T19:04:21.956633+0000","last_unstale":"2024-09-20T19:05:46.990870+0000","last_undegraded":"2024-09-20T19:05:46.990870+0000","last_fullsized":"2024-09-20T19:05:46.990870+0000","mapping_epoch":375,"log_start":"372'8682","ondisk_log_start":"372'8682","created":325,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":5237,"log_dups_size":0,"ondisk_log_size":5237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:28.169249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49086464,"num_objects":749,"num_object_clones":0,"num_object_copies":2247,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13919,"num_write_kb":469376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":665,"num_bytes_recovered":43581440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"273'20","reported_seq":90,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:48.983831+0000","last_change":"2024-09-20T19:04:20.945143+0000","last_active":"2024-09-20T19:05:48.983831+0000","last_peered":"2024-09-20T19:05:48.983831+0000","last_clean":"2024-09-20T19:05:48.983831+0000","last_became_active":"2024-09-20T19:04:20.896483+0000","last_became_peered":"2024-09-20T19:04:20.896483+0000","last_unstale":"2024-09-20T19:05:48.983831+0000","last_undegraded":"2024-09-20T19:05:48.983831+0000","last_fullsized":"2024-09-20T19:05:48.983831+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'20","last_scrub_stamp":"2024-09-20T19:03:04.282800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:04.282800+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":779,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:49.001372+0000","last_change":"2024-09-20T19:05:48.678438+0000","last_active":"2024-09-20T19:05:49.001372+0000","last_peered":"2024-09-20T19:05:49.001372+0000","last_clean":"2024-09-20T19:05:49.001372+0000","last_became_active":"2024-09-20T19:04:20.892951+0000","last_became_peered":"2024-09-20T19:04:20.892951+0000","last_unstale":"2024-09-20T19:05:49.001372+0000","last_undegraded":"2024-09-20T19:05:49.001372+0000","last_fullsized":"2024-09-20T19:05:49.001372+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:48.678361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:48.678361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:02.488299+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":956,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.905058+0000","last_change":"2024-09-20T19:05:08.406307+0000","last_active":"2024-09-20T19:05:45.905058+0000","last_peered":"2024-09-20T19:05:45.905058+0000","last_clean":"2024-09-20T19:05:45.905058+0000","last_became_active":"2024-09-20T18:49:49.957005+0000","last_became_peered":"2024-09-20T18:49:49.957005+0000","last_unstale":"2024-09-20T19:05:45.905058+0000","last_undegraded":"2024-09-20T19:05:45.905058+0000","last_fullsized":"2024-09-20T19:05:45.905058+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T19:05:08.406231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:05:08.406231+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:20.576797+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":78,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:48.983333+0000","last_change":"2024-09-20T19:04:20.885294+0000","last_active":"2024-09-20T19:05:48.983333+0000","last_peered":"2024-09-20T19:05:48.983333+0000","last_clean":"2024-09-20T19:05:48.983333+0000","last_became_active":"2024-09-20T19:04:20.884994+0000","last_became_peered":"2024-09-20T19:04:20.884994+0000","last_unstale":"2024-09-20T19:05:48.983333+0000","last_undegraded":"2024-09-20T19:05:48.983333+0000","last_fullsized":"2024-09-20T19:05:48.983333+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:11.207994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:03:11.207994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.5","version":"407'13725","reported_seq":13977,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:46.527017+0000","last_change":"2024-09-20T19:04:04.390108+0000","last_active":"2024-09-20T19:05:46.527017+0000","last_peered":"2024-09-20T19:05:46.527017+0000","last_clean":"2024-09-20T19:05:46.527017+0000","last_became_active":"2024-09-20T19:02:11.828162+0000","last_became_peered":"2024-09-20T19:02:11.828162+0000","last_unstale":"2024-09-20T19:05:46.527017+0000","last_undegraded":"2024-09-20T19:05:46.527017+0000","last_fullsized":"2024-09-20T19:05:46.527017+0000","mapping_epoch":325,"log_start":"362'7168","ondisk_log_start":"362'7168","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"367'7875","last_scrub_stamp":"2024-09-20T19:04:04.389838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:04:04.389838+0000","objects_scrubbed":6698,"log_size":6557,"log_dups_size":0,"ondisk_log_size":6557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"queued for scrub","scrub_duration":2035,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51970048,"num_objects":793,"num_object_clones":0,"num_object_copies":2379,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13725,"num_write_kb":464576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"274'15","reported_seq":811,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.905374+0000","last_change":"2024-09-20T19:04:20.978525+0000","last_active":"2024-09-20T19:05:45.905374+0000","last_peered":"2024-09-20T19:05:45.905374+0000","last_clean":"2024-09-20T19:05:45.905374+0000","last_became_active":"2024-09-20T19:04:20.906834+0000","last_became_peered":"2024-09-20T19:04:20.906834+0000","last_unstale":"2024-09-20T19:05:45.905374+0000","last_undegraded":"2024-09-20T19:05:45.905374+0000","last_fullsized":"2024-09-20T19:05:45.905374+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'15","last_scrub_stamp":"2024-09-20T19:01:27.802100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:27.802100+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":296,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.899880+0000","last_change":"2024-09-20T19:04:16.384877+0000","last_active":"2024-09-20T19:05:45.899880+0000","last_peered":"2024-09-20T19:05:45.899880+0000","last_clean":"2024-09-20T19:05:45.899880+0000","last_became_active":"2024-09-20T19:00:01.190875+0000","last_became_peered":"2024-09-20T19:00:01.190875+0000","last_unstale":"2024-09-20T19:05:45.899880+0000","last_undegraded":"2024-09-20T19:05:45.899880+0000","last_fullsized":"2024-09-20T19:05:45.899880+0000","mapping_epoch":277,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":278,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:16.384809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:04:16.384809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":772,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.899654+0000","last_change":"2024-09-20T19:04:20.889614+0000","last_active":"2024-09-20T19:05:45.899654+0000","last_peered":"2024-09-20T19:05:45.899654+0000","last_clean":"2024-09-20T19:05:45.899654+0000","last_became_active":"2024-09-20T19:04:20.889273+0000","last_became_peered":"2024-09-20T19:04:20.889273+0000","last_unstale":"2024-09-20T19:05:45.899654+0000","last_undegraded":"2024-09-20T19:05:45.899654+0000","last_fullsized":"2024-09-20T19:05:45.899654+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:03.389611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:03.389611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:26.422333+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":733,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.904685+0000","last_change":"2024-09-20T19:02:31.120809+0000","last_active":"2024-09-20T19:05:45.904685+0000","last_peered":"2024-09-20T19:05:45.904685+0000","last_clean":"2024-09-20T19:05:45.904685+0000","last_became_active":"2024-09-20T18:49:51.964098+0000","last_became_peered":"2024-09-20T18:49:51.964098+0000","last_unstale":"2024-09-20T19:05:45.904685+0000","last_undegraded":"2024-09-20T19:05:45.904685+0000","last_fullsized":"2024-09-20T19:05:45.904685+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:31.120623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:31.120623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"407'13710","reported_seq":13907,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:47.609455+0000","last_change":"2024-09-20T19:04:21.973100+0000","last_active":"2024-09-20T19:05:47.609455+0000","last_peered":"2024-09-20T19:05:47.609455+0000","last_clean":"2024-09-20T19:05:47.609455+0000","last_became_active":"2024-09-20T19:04:21.960863+0000","last_became_peered":"2024-09-20T19:04:21.960863+0000","last_unstale":"2024-09-20T19:05:47.609455+0000","last_undegraded":"2024-09-20T19:05:47.609455+0000","last_fullsized":"2024-09-20T19:05:47.609455+0000","mapping_epoch":375,"log_start":"371'8297","ondisk_log_start":"371'8297","created":325,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":5413,"log_dups_size":0,"ondisk_log_size":5413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43909120,"num_objects":670,"num_object_clones":0,"num_object_copies":2010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13710,"num_write_kb":460160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"274'35","reported_seq":367,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.892148+0000","last_change":"2024-09-20T19:03:09.841596+0000","last_active":"2024-09-20T19:05:45.892148+0000","last_peered":"2024-09-20T19:05:45.892148+0000","last_clean":"2024-09-20T19:05:45.892148+0000","last_became_active":"2024-09-20T18:59:05.087391+0000","last_became_peered":"2024-09-20T18:59:05.087391+0000","last_unstale":"2024-09-20T19:05:45.892148+0000","last_undegraded":"2024-09-20T19:05:45.892148+0000","last_fullsized":"2024-09-20T19:05:45.892148+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'35","last_scrub_stamp":"2024-09-20T19:03:09.841361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:09.841361+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":759,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.907246+0000","last_change":"2024-09-20T19:02:05.440745+0000","last_active":"2024-09-20T19:05:45.907246+0000","last_peered":"2024-09-20T19:05:45.907246+0000","last_clean":"2024-09-20T19:05:45.907246+0000","last_became_active":"2024-09-20T18:59:05.086670+0000","last_became_peered":"2024-09-20T18:59:05.086670+0000","last_unstale":"2024-09-20T19:05:45.907246+0000","last_undegraded":"2024-09-20T19:05:45.907246+0000","last_fullsized":"2024-09-20T19:05:45.907246+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:05.440677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:05.440677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":755,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.899780+0000","last_change":"2024-09-20T19:05:11.444197+0000","last_active":"2024-09-20T19:05:45.899780+0000","last_peered":"2024-09-20T19:05:45.899780+0000","last_clean":"2024-09-20T19:05:45.899780+0000","last_became_active":"2024-09-20T18:49:49.957558+0000","last_became_peered":"2024-09-20T18:49:49.957558+0000","last_unstale":"2024-09-20T19:05:45.899780+0000","last_undegraded":"2024-09-20T19:05:45.899780+0000","last_fullsized":"2024-09-20T19:05:45.899780+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:11.444143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:05:11.444143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:23.300683+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":725,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.906130+0000","last_change":"2024-09-20T18:53:47.370152+0000","last_active":"2024-09-20T19:05:45.906130+0000","last_peered":"2024-09-20T19:05:45.906130+0000","last_clean":"2024-09-20T19:05:45.906130+0000","last_became_active":"2024-09-20T18:49:51.970798+0000","last_became_peered":"2024-09-20T18:49:51.970798+0000","last_unstale":"2024-09-20T19:05:45.906130+0000","last_undegraded":"2024-09-20T19:05:45.906130+0000","last_fullsized":"2024-09-20T19:05:45.906130+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:47.370108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:47.370108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:30.728805+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.7","version":"407'13998","reported_seq":14175,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:46.924710+0000","last_change":"2024-09-20T19:02:11.827681+0000","last_active":"2024-09-20T19:05:46.924710+0000","last_peered":"2024-09-20T19:05:46.924710+0000","last_clean":"2024-09-20T19:05:46.924710+0000","last_became_active":"2024-09-20T19:02:11.827360+0000","last_became_peered":"2024-09-20T19:02:11.827360+0000","last_unstale":"2024-09-20T19:05:46.924710+0000","last_undegraded":"2024-09-20T19:05:46.924710+0000","last_fullsized":"2024-09-20T19:05:46.924710+0000","mapping_epoch":325,"log_start":"367'7907","ondisk_log_start":"367'7907","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":6091,"log_dups_size":0,"ondisk_log_size":6091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:39.258857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47579136,"num_objects":726,"num_object_clones":0,"num_object_copies":2178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13998,"num_write_kb":471168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"273'18","reported_seq":90,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:48.983291+0000","last_change":"2024-09-20T19:04:20.936465+0000","last_active":"2024-09-20T19:05:48.983291+0000","last_peered":"2024-09-20T19:05:48.983291+0000","last_clean":"2024-09-20T19:05:48.983291+0000","last_became_active":"2024-09-20T19:04:20.888905+0000","last_became_peered":"2024-09-20T19:04:20.888905+0000","last_unstale":"2024-09-20T19:05:48.983291+0000","last_undegraded":"2024-09-20T19:05:48.983291+0000","last_fullsized":"2024-09-20T19:05:48.983291+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'18","last_scrub_stamp":"2024-09-20T19:03:13.201592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:13.201592+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":778,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.907702+0000","last_change":"2024-09-20T19:04:50.680303+0000","last_active":"2024-09-20T19:05:45.907702+0000","last_peered":"2024-09-20T19:05:45.907702+0000","last_clean":"2024-09-20T19:05:45.907702+0000","last_became_active":"2024-09-20T19:04:20.892429+0000","last_became_peered":"2024-09-20T19:04:20.892429+0000","last_unstale":"2024-09-20T19:05:45.907702+0000","last_undegraded":"2024-09-20T19:05:45.907702+0000","last_fullsized":"2024-09-20T19:05:45.907702+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:50.680222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:04:50.680222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:57.765584+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":747,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.900781+0000","last_change":"2024-09-20T19:04:48.406445+0000","last_active":"2024-09-20T19:05:45.900781+0000","last_peered":"2024-09-20T19:05:45.900781+0000","last_clean":"2024-09-20T19:05:45.900781+0000","last_became_active":"2024-09-20T18:49:49.954944+0000","last_became_peered":"2024-09-20T18:49:49.954944+0000","last_unstale":"2024-09-20T19:05:45.900781+0000","last_undegraded":"2024-09-20T19:05:45.900781+0000","last_fullsized":"2024-09-20T19:05:45.900781+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:48.406373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:48.406373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:58.629290+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":742,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:49.001017+0000","last_change":"2024-09-20T19:05:43.657860+0000","last_active":"2024-09-20T19:05:49.001017+0000","last_peered":"2024-09-20T19:05:49.001017+0000","last_clean":"2024-09-20T19:05:49.001017+0000","last_became_active":"2024-09-20T18:49:51.972970+0000","last_became_peered":"2024-09-20T18:49:51.972970+0000","last_unstale":"2024-09-20T19:05:49.001017+0000","last_undegraded":"2024-09-20T19:05:49.001017+0000","last_fullsized":"2024-09-20T19:05:49.001017+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:43.657780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:05:43.657780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:05.178691+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.6","version":"408'14310","reported_seq":23811,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:50.760787+0000","last_change":"2024-09-20T19:05:47.942165+0000","last_active":"2024-09-20T19:05:50.760787+0000","last_peered":"2024-09-20T19:05:50.760787+0000","last_clean":"2024-09-20T19:05:50.760787+0000","last_became_active":"2024-09-20T19:05:21.906044+0000","last_became_peered":"2024-09-20T19:05:21.906044+0000","last_unstale":"2024-09-20T19:05:50.760787+0000","last_undegraded":"2024-09-20T19:05:50.760787+0000","last_fullsized":"2024-09-20T19:05:50.760787+0000","mapping_epoch":394,"log_start":"372'8562","ondisk_log_start":"372'8562","created":325,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"407'14117","last_scrub_stamp":"2024-09-20T19:05:47.941785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:05:47.941785+0000","objects_scrubbed":682,"log_size":5748,"log_dups_size":0,"ondisk_log_size":5748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:08.053533+0000","scrub_duration":191,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31719424,"num_objects":484,"num_object_clones":0,"num_object_copies":1452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":484,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14310,"num_write_kb":473408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4647,"num_bytes_recovered":203751424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"273'27","reported_seq":847,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:49.777488+0000","last_change":"2024-09-20T19:05:49.777488+0000","last_active":"2024-09-20T19:05:49.777488+0000","last_peered":"2024-09-20T19:05:49.777488+0000","last_clean":"2024-09-20T19:05:49.777488+0000","last_became_active":"2024-09-20T18:59:05.083726+0000","last_became_peered":"2024-09-20T18:59:05.083726+0000","last_unstale":"2024-09-20T19:05:49.777488+0000","last_undegraded":"2024-09-20T19:05:49.777488+0000","last_fullsized":"2024-09-20T19:05:49.777488+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'27","last_scrub_stamp":"2024-09-20T19:05:49.777414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:49.777414+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:12.438786+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":810,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.907704+0000","last_change":"2024-09-20T19:04:46.691255+0000","last_active":"2024-09-20T19:05:45.907704+0000","last_peered":"2024-09-20T19:05:45.907704+0000","last_clean":"2024-09-20T19:05:45.907704+0000","last_became_active":"2024-09-20T19:04:20.884854+0000","last_became_peered":"2024-09-20T19:04:20.884854+0000","last_unstale":"2024-09-20T19:05:45.907704+0000","last_undegraded":"2024-09-20T19:05:45.907704+0000","last_fullsized":"2024-09-20T19:05:45.907704+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-20T19:04:46.691171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:04:46.691171+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:02.158207+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"375'14","reported_seq":987,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.906393+0000","last_change":"2024-09-20T19:04:39.564069+0000","last_active":"2024-09-20T19:05:45.906393+0000","last_peered":"2024-09-20T19:05:45.906393+0000","last_clean":"2024-09-20T19:05:45.906393+0000","last_became_active":"2024-09-20T19:04:20.892956+0000","last_became_peered":"2024-09-20T19:04:20.892956+0000","last_unstale":"2024-09-20T19:05:45.906393+0000","last_undegraded":"2024-09-20T19:05:45.906393+0000","last_fullsized":"2024-09-20T19:05:45.906393+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"375'14","last_scrub_stamp":"2024-09-20T19:04:39.563917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:39.563917+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:49.606578+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":756,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:48.986646+0000","last_change":"2024-09-20T19:03:06.888620+0000","last_active":"2024-09-20T19:05:48.986646+0000","last_peered":"2024-09-20T19:05:48.986646+0000","last_clean":"2024-09-20T19:05:48.986646+0000","last_became_active":"2024-09-20T18:59:05.087005+0000","last_became_peered":"2024-09-20T18:59:05.087005+0000","last_unstale":"2024-09-20T19:05:48.986646+0000","last_undegraded":"2024-09-20T19:05:48.986646+0000","last_fullsized":"2024-09-20T19:05:48.986646+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:06.888563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:03:06.888563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":314,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.891696+0000","last_change":"2024-09-20T19:01:16.542398+0000","last_active":"2024-09-20T19:05:45.891696+0000","last_peered":"2024-09-20T19:05:45.891696+0000","last_clean":"2024-09-20T19:05:45.891696+0000","last_became_active":"2024-09-20T18:59:05.085427+0000","last_became_peered":"2024-09-20T18:59:05.085427+0000","last_unstale":"2024-09-20T19:05:45.891696+0000","last_undegraded":"2024-09-20T19:05:45.891696+0000","last_fullsized":"2024-09-20T19:05:45.891696+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:16.542335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:16.542335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:29.408351+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"273'12","reported_seq":767,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:49.004477+0000","last_change":"2024-09-20T19:04:48.570718+0000","last_active":"2024-09-20T19:05:49.004477+0000","last_peered":"2024-09-20T19:05:49.004477+0000","last_clean":"2024-09-20T19:05:49.004477+0000","last_became_active":"2024-09-20T18:49:48.500662+0000","last_became_peered":"2024-09-20T18:49:48.500662+0000","last_unstale":"2024-09-20T19:05:49.004477+0000","last_undegraded":"2024-09-20T19:05:49.004477+0000","last_fullsized":"2024-09-20T19:05:49.004477+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'12","last_scrub_stamp":"2024-09-20T19:04:48.570639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:04:48.570639+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:50.389558+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":745,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:48.988994+0000","last_change":"2024-09-20T19:05:22.605885+0000","last_active":"2024-09-20T19:05:48.988994+0000","last_peered":"2024-09-20T19:05:48.988994+0000","last_clean":"2024-09-20T19:05:48.988994+0000","last_became_active":"2024-09-20T18:49:49.956749+0000","last_became_peered":"2024-09-20T18:49:49.956749+0000","last_unstale":"2024-09-20T19:05:48.988994+0000","last_undegraded":"2024-09-20T19:05:48.988994+0000","last_fullsized":"2024-09-20T19:05:48.988994+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:22.605839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:05:22.605839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:49.241766+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":771,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.906968+0000","last_change":"2024-09-20T19:04:20.914426+0000","last_active":"2024-09-20T19:05:45.906968+0000","last_peered":"2024-09-20T19:05:45.906968+0000","last_clean":"2024-09-20T19:05:45.906968+0000","last_became_active":"2024-09-20T19:04:20.914115+0000","last_became_peered":"2024-09-20T19:04:20.914115+0000","last_unstale":"2024-09-20T19:05:45.906968+0000","last_undegraded":"2024-09-20T19:05:45.906968+0000","last_fullsized":"2024-09-20T19:05:45.906968+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:03.509903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:03.509903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"273'15","reported_seq":757,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.905388+0000","last_change":"2024-09-20T19:01:15.875779+0000","last_active":"2024-09-20T19:05:45.905388+0000","last_peered":"2024-09-20T19:05:45.905388+0000","last_clean":"2024-09-20T19:05:45.905388+0000","last_became_active":"2024-09-20T18:49:48.500803+0000","last_became_peered":"2024-09-20T18:49:48.500803+0000","last_unstale":"2024-09-20T19:05:45.905388+0000","last_undegraded":"2024-09-20T19:05:45.905388+0000","last_fullsized":"2024-09-20T19:05:45.905388+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:01:15.875693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:15.875693+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":783,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.906657+0000","last_change":"2024-09-20T19:04:47.713764+0000","last_active":"2024-09-20T19:05:45.906657+0000","last_peered":"2024-09-20T19:05:45.906657+0000","last_clean":"2024-09-20T19:05:45.906657+0000","last_became_active":"2024-09-20T19:04:20.893250+0000","last_became_peered":"2024-09-20T19:04:20.893250+0000","last_unstale":"2024-09-20T19:05:45.906657+0000","last_undegraded":"2024-09-20T19:05:45.906657+0000","last_fullsized":"2024-09-20T19:05:45.906657+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:47.713585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:04:47.713585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:49.334341+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":745,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:49.004338+0000","last_change":"2024-09-20T19:04:50.573619+0000","last_active":"2024-09-20T19:05:49.004338+0000","last_peered":"2024-09-20T19:05:49.004338+0000","last_clean":"2024-09-20T19:05:49.004338+0000","last_became_active":"2024-09-20T18:49:49.961947+0000","last_became_peered":"2024-09-20T18:49:49.961947+0000","last_unstale":"2024-09-20T19:05:49.004338+0000","last_undegraded":"2024-09-20T19:05:49.004338+0000","last_fullsized":"2024-09-20T19:05:49.004338+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:50.573550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:50.573550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:02.174525+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"66'1","reported_seq":784,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.899968+0000","last_change":"2024-09-20T19:04:20.913449+0000","last_active":"2024-09-20T19:05:45.899968+0000","last_peered":"2024-09-20T19:05:45.899968+0000","last_clean":"2024-09-20T19:05:45.899968+0000","last_became_active":"2024-09-20T19:04:20.892236+0000","last_became_peered":"2024-09-20T19:04:20.892236+0000","last_unstale":"2024-09-20T19:05:45.899968+0000","last_undegraded":"2024-09-20T19:05:45.899968+0000","last_fullsized":"2024-09-20T19:05:45.899968+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T19:03:12.499684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:03:12.499684+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:37.501578+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"274'15","reported_seq":758,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:48.986850+0000","last_change":"2024-09-20T19:01:36.532486+0000","last_active":"2024-09-20T19:05:48.986850+0000","last_peered":"2024-09-20T19:05:48.986850+0000","last_clean":"2024-09-20T19:05:48.986850+0000","last_became_active":"2024-09-20T18:49:48.501011+0000","last_became_peered":"2024-09-20T18:49:48.501011+0000","last_unstale":"2024-09-20T19:05:48.986850+0000","last_undegraded":"2024-09-20T19:05:48.986850+0000","last_fullsized":"2024-09-20T19:05:48.986850+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'15","last_scrub_stamp":"2024-09-20T19:01:36.532419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:36.532419+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:38.766268+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":744,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.905957+0000","last_change":"2024-09-20T19:01:30.864866+0000","last_active":"2024-09-20T19:05:45.905957+0000","last_peered":"2024-09-20T19:05:45.905957+0000","last_clean":"2024-09-20T19:05:45.905957+0000","last_became_active":"2024-09-20T18:49:45.938511+0000","last_became_peered":"2024-09-20T18:49:45.938511+0000","last_unstale":"2024-09-20T19:05:45.905957+0000","last_undegraded":"2024-09-20T19:05:45.905957+0000","last_fullsized":"2024-09-20T19:05:45.905957+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:30.864794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:30.864794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":946,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:49.002237+0000","last_change":"2024-09-20T19:04:41.601208+0000","last_active":"2024-09-20T19:05:49.002237+0000","last_peered":"2024-09-20T19:05:49.002237+0000","last_clean":"2024-09-20T19:05:49.002237+0000","last_became_active":"2024-09-20T18:49:49.960550+0000","last_became_peered":"2024-09-20T18:49:49.960550+0000","last_unstale":"2024-09-20T19:05:49.002237+0000","last_undegraded":"2024-09-20T19:05:49.002237+0000","last_fullsized":"2024-09-20T19:05:49.002237+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T19:04:41.601073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:41.601073+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:03.890101+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":78,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:48.983210+0000","last_change":"2024-09-20T19:04:20.884397+0000","last_active":"2024-09-20T19:05:48.983210+0000","last_peered":"2024-09-20T19:05:48.983210+0000","last_clean":"2024-09-20T19:05:48.983210+0000","last_became_active":"2024-09-20T19:04:20.884100+0000","last_became_peered":"2024-09-20T19:04:20.884100+0000","last_unstale":"2024-09-20T19:05:48.983210+0000","last_undegraded":"2024-09-20T19:05:48.983210+0000","last_fullsized":"2024-09-20T19:05:48.983210+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:59.497810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:59:59.497810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"274'20","reported_seq":774,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:49.002348+0000","last_change":"2024-09-20T18:57:20.701592+0000","last_active":"2024-09-20T19:05:49.002348+0000","last_peered":"2024-09-20T19:05:49.002348+0000","last_clean":"2024-09-20T19:05:49.002348+0000","last_became_active":"2024-09-20T18:49:58.997494+0000","last_became_peered":"2024-09-20T18:49:58.997494+0000","last_unstale":"2024-09-20T19:05:49.002348+0000","last_undegraded":"2024-09-20T19:05:49.002348+0000","last_fullsized":"2024-09-20T19:05:49.002348+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:57:20.701543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:20.701543+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:23.577077+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":761,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.906675+0000","last_change":"2024-09-20T19:04:20.884940+0000","last_active":"2024-09-20T19:05:45.906675+0000","last_peered":"2024-09-20T19:05:45.906675+0000","last_clean":"2024-09-20T19:05:45.906675+0000","last_became_active":"2024-09-20T19:04:20.884582+0000","last_became_peered":"2024-09-20T19:04:20.884582+0000","last_unstale":"2024-09-20T19:05:45.906675+0000","last_undegraded":"2024-09-20T19:05:45.906675+0000","last_fullsized":"2024-09-20T19:05:45.906675+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.632380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.632380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:35.779165+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":938,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:48.986647+0000","last_change":"2024-09-20T19:01:10.601374+0000","last_active":"2024-09-20T19:05:48.986647+0000","last_peered":"2024-09-20T19:05:48.986647+0000","last_clean":"2024-09-20T19:05:48.986647+0000","last_became_active":"2024-09-20T18:49:49.962243+0000","last_became_peered":"2024-09-20T18:49:49.962243+0000","last_unstale":"2024-09-20T19:05:48.986647+0000","last_undegraded":"2024-09-20T19:05:48.986647+0000","last_fullsized":"2024-09-20T19:05:48.986647+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T19:01:10.601300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:10.601300+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:23.615982+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":740,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.904752+0000","last_change":"2024-09-20T19:01:03.821977+0000","last_active":"2024-09-20T19:05:45.904752+0000","last_peered":"2024-09-20T19:05:45.904752+0000","last_clean":"2024-09-20T19:05:45.904752+0000","last_became_active":"2024-09-20T18:49:51.969093+0000","last_became_peered":"2024-09-20T18:49:51.969093+0000","last_unstale":"2024-09-20T19:05:45.904752+0000","last_undegraded":"2024-09-20T19:05:45.904752+0000","last_fullsized":"2024-09-20T19:05:45.904752+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:03.821908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:03.821908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"273'19","reported_seq":90,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:48.982920+0000","last_change":"2024-09-20T19:04:20.970137+0000","last_active":"2024-09-20T19:05:48.982920+0000","last_peered":"2024-09-20T19:05:48.982920+0000","last_clean":"2024-09-20T19:05:48.982920+0000","last_became_active":"2024-09-20T19:04:20.897330+0000","last_became_peered":"2024-09-20T19:04:20.897330+0000","last_unstale":"2024-09-20T19:05:48.982920+0000","last_undegraded":"2024-09-20T19:05:48.982920+0000","last_fullsized":"2024-09-20T19:05:48.982920+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:57:50.899514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:50.899514+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:38.174326+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":78,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:48.983909+0000","last_change":"2024-09-20T19:04:20.896732+0000","last_active":"2024-09-20T19:05:48.983909+0000","last_peered":"2024-09-20T19:05:48.983909+0000","last_clean":"2024-09-20T19:05:48.983909+0000","last_became_active":"2024-09-20T19:04:20.896007+0000","last_became_peered":"2024-09-20T19:04:20.896007+0000","last_unstale":"2024-09-20T19:05:48.983909+0000","last_undegraded":"2024-09-20T19:05:48.983909+0000","last_fullsized":"2024-09-20T19:05:48.983909+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:06.310027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:59:06.310027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":735,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.908413+0000","last_change":"2024-09-20T19:02:38.906352+0000","last_active":"2024-09-20T19:05:45.908413+0000","last_peered":"2024-09-20T19:05:45.908413+0000","last_clean":"2024-09-20T19:05:45.908413+0000","last_became_active":"2024-09-20T18:49:49.957067+0000","last_became_peered":"2024-09-20T18:49:49.957067+0000","last_unstale":"2024-09-20T19:05:45.908413+0000","last_undegraded":"2024-09-20T19:05:45.908413+0000","last_fullsized":"2024-09-20T19:05:45.908413+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:38.906161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:38.906161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:36.012448+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":717,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:49.013250+0000","last_change":"2024-09-20T18:57:16.789412+0000","last_active":"2024-09-20T19:05:49.013250+0000","last_peered":"2024-09-20T19:05:49.013250+0000","last_clean":"2024-09-20T19:05:49.013250+0000","last_became_active":"2024-09-20T18:49:51.966915+0000","last_became_peered":"2024-09-20T18:49:51.966915+0000","last_unstale":"2024-09-20T19:05:49.013250+0000","last_undegraded":"2024-09-20T19:05:49.013250+0000","last_fullsized":"2024-09-20T19:05:49.013250+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:16.789095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:16.789095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:35.677144+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"273'15","reported_seq":806,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.908476+0000","last_change":"2024-09-20T19:02:30.816061+0000","last_active":"2024-09-20T19:05:45.908476+0000","last_peered":"2024-09-20T19:05:45.908476+0000","last_clean":"2024-09-20T19:05:45.908476+0000","last_became_active":"2024-09-20T18:59:05.085230+0000","last_became_peered":"2024-09-20T18:59:05.085230+0000","last_unstale":"2024-09-20T19:05:45.908476+0000","last_undegraded":"2024-09-20T19:05:45.908476+0000","last_fullsized":"2024-09-20T19:05:45.908476+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:02:30.815747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:30.815747+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:39.985336+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":761,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.907155+0000","last_change":"2024-09-20T19:04:20.885127+0000","last_active":"2024-09-20T19:05:45.907155+0000","last_peered":"2024-09-20T19:05:45.907155+0000","last_clean":"2024-09-20T19:05:45.907155+0000","last_became_active":"2024-09-20T19:04:20.884660+0000","last_became_peered":"2024-09-20T19:04:20.884660+0000","last_unstale":"2024-09-20T19:05:45.907155+0000","last_undegraded":"2024-09-20T19:05:45.907155+0000","last_fullsized":"2024-09-20T19:05:45.907155+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:07.408197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:03:07.408197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":745,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:49.001155+0000","last_change":"2024-09-20T19:04:44.606497+0000","last_active":"2024-09-20T19:05:49.001155+0000","last_peered":"2024-09-20T19:05:49.001155+0000","last_clean":"2024-09-20T19:05:49.001155+0000","last_became_active":"2024-09-20T18:49:49.962523+0000","last_became_peered":"2024-09-20T18:49:49.962523+0000","last_unstale":"2024-09-20T19:05:49.001155+0000","last_undegraded":"2024-09-20T19:05:49.001155+0000","last_fullsized":"2024-09-20T19:05:49.001155+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:44.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:44.606343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:53.560145+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":732,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.904836+0000","last_change":"2024-09-20T19:01:11.847792+0000","last_active":"2024-09-20T19:05:45.904836+0000","last_peered":"2024-09-20T19:05:45.904836+0000","last_clean":"2024-09-20T19:05:45.904836+0000","last_became_active":"2024-09-20T18:49:51.967313+0000","last_became_peered":"2024-09-20T18:49:51.967313+0000","last_unstale":"2024-09-20T19:05:45.904836+0000","last_undegraded":"2024-09-20T19:05:45.904836+0000","last_fullsized":"2024-09-20T19:05:45.904836+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:11.847716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:11.847716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"274'19","reported_seq":810,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:49.013389+0000","last_change":"2024-09-20T19:01:05.495603+0000","last_active":"2024-09-20T19:05:49.013389+0000","last_peered":"2024-09-20T19:05:49.013389+0000","last_clean":"2024-09-20T19:05:49.013389+0000","last_became_active":"2024-09-20T18:59:05.075390+0000","last_became_peered":"2024-09-20T18:59:05.075390+0000","last_unstale":"2024-09-20T19:05:49.013389+0000","last_undegraded":"2024-09-20T19:05:49.013389+0000","last_fullsized":"2024-09-20T19:05:49.013389+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'19","last_scrub_stamp":"2024-09-20T19:01:05.495478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:05.495478+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:56.633007+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":761,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:48.988969+0000","last_change":"2024-09-20T19:05:43.686902+0000","last_active":"2024-09-20T19:05:48.988969+0000","last_peered":"2024-09-20T19:05:48.988969+0000","last_clean":"2024-09-20T19:05:48.988969+0000","last_became_active":"2024-09-20T18:49:45.942105+0000","last_became_peered":"2024-09-20T18:49:45.942105+0000","last_unstale":"2024-09-20T19:05:48.988969+0000","last_undegraded":"2024-09-20T19:05:48.988969+0000","last_fullsized":"2024-09-20T19:05:48.988969+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:43.686842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:43.686842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:01.625104+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":775,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.905885+0000","last_change":"2024-09-20T19:04:20.912120+0000","last_active":"2024-09-20T19:05:45.905885+0000","last_peered":"2024-09-20T19:05:45.905885+0000","last_clean":"2024-09-20T19:05:45.905885+0000","last_became_active":"2024-09-20T19:04:20.911292+0000","last_became_peered":"2024-09-20T19:04:20.911292+0000","last_unstale":"2024-09-20T19:05:45.905885+0000","last_undegraded":"2024-09-20T19:05:45.905885+0000","last_fullsized":"2024-09-20T19:05:45.905885+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:27.267254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:27.267254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":739,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:49.001290+0000","last_change":"2024-09-20T19:02:38.950217+0000","last_active":"2024-09-20T19:05:49.001290+0000","last_peered":"2024-09-20T19:05:49.001290+0000","last_clean":"2024-09-20T19:05:49.001290+0000","last_became_active":"2024-09-20T18:59:05.081179+0000","last_became_peered":"2024-09-20T18:59:05.081179+0000","last_unstale":"2024-09-20T19:05:49.001290+0000","last_undegraded":"2024-09-20T19:05:49.001290+0000","last_fullsized":"2024-09-20T19:05:49.001290+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:38.950017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:38.950017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"273'8","reported_seq":787,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:48.999467+0000","last_change":"2024-09-20T19:04:40.559947+0000","last_active":"2024-09-20T19:05:48.999467+0000","last_peered":"2024-09-20T19:05:48.999467+0000","last_clean":"2024-09-20T19:05:48.999467+0000","last_became_active":"2024-09-20T18:59:05.076365+0000","last_became_peered":"2024-09-20T18:59:05.076365+0000","last_unstale":"2024-09-20T19:05:48.999467+0000","last_undegraded":"2024-09-20T19:05:48.999467+0000","last_fullsized":"2024-09-20T19:05:48.999467+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'8","last_scrub_stamp":"2024-09-20T19:04:40.559877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:04:40.559877+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:53.505030+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":752,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.905098+0000","last_change":"2024-09-20T19:02:39.184249+0000","last_active":"2024-09-20T19:05:45.905098+0000","last_peered":"2024-09-20T19:05:45.905098+0000","last_clean":"2024-09-20T19:05:45.905098+0000","last_became_active":"2024-09-20T18:49:45.940988+0000","last_became_peered":"2024-09-20T18:49:45.940988+0000","last_unstale":"2024-09-20T19:05:45.905098+0000","last_undegraded":"2024-09-20T19:05:45.905098+0000","last_fullsized":"2024-09-20T19:05:45.905098+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:39.184150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:39.184150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":339,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.891858+0000","last_change":"2024-09-20T19:04:20.900567+0000","last_active":"2024-09-20T19:05:45.891858+0000","last_peered":"2024-09-20T19:05:45.891858+0000","last_clean":"2024-09-20T19:05:45.891858+0000","last_became_active":"2024-09-20T19:04:20.900214+0000","last_became_peered":"2024-09-20T19:04:20.900214+0000","last_unstale":"2024-09-20T19:05:45.891858+0000","last_undegraded":"2024-09-20T19:05:45.891858+0000","last_fullsized":"2024-09-20T19:05:45.891858+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:41.497115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:41.497115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:33.282430+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":749,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.906149+0000","last_change":"2024-09-20T19:04:20.886208+0000","last_active":"2024-09-20T19:05:45.906149+0000","last_peered":"2024-09-20T19:05:45.906149+0000","last_clean":"2024-09-20T19:05:45.906149+0000","last_became_active":"2024-09-20T19:04:20.885211+0000","last_became_peered":"2024-09-20T19:04:20.885211+0000","last_unstale":"2024-09-20T19:05:45.906149+0000","last_undegraded":"2024-09-20T19:05:45.906149+0000","last_fullsized":"2024-09-20T19:05:45.906149+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:29.277916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:29.277916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":732,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.908612+0000","last_change":"2024-09-20T19:04:20.890533+0000","last_active":"2024-09-20T19:05:45.908612+0000","last_peered":"2024-09-20T19:05:45.908612+0000","last_clean":"2024-09-20T19:05:45.908612+0000","last_became_active":"2024-09-20T19:04:20.890101+0000","last_became_peered":"2024-09-20T19:04:20.890101+0000","last_unstale":"2024-09-20T19:05:45.908612+0000","last_undegraded":"2024-09-20T19:05:45.908612+0000","last_fullsized":"2024-09-20T19:05:45.908612+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:10.970779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:10.970779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:24.205999+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":737,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T19:05:48.999700+0000","last_change":"2024-09-20T18:58:36.077005+0000","last_active":"2024-09-20T19:05:48.999700+0000","last_peered":"2024-09-20T19:05:48.999700+0000","last_clean":"2024-09-20T19:05:48.999700+0000","last_became_active":"2024-09-20T18:49:45.937992+0000","last_became_peered":"2024-09-20T18:49:45.937992+0000","last_unstale":"2024-09-20T19:05:48.999700+0000","last_undegraded":"2024-09-20T19:05:48.999700+0000","last_fullsized":"2024-09-20T19:05:48.999700+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:36.076937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:36.076937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:42.609713+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"274'19","reported_seq":339,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.892872+0000","last_change":"2024-09-20T19:01:15.544760+0000","last_active":"2024-09-20T19:05:45.892872+0000","last_peered":"2024-09-20T19:05:45.892872+0000","last_clean":"2024-09-20T19:05:45.892872+0000","last_became_active":"2024-09-20T18:59:05.089294+0000","last_became_peered":"2024-09-20T18:59:05.089294+0000","last_unstale":"2024-09-20T19:05:45.892872+0000","last_undegraded":"2024-09-20T19:05:45.892872+0000","last_fullsized":"2024-09-20T19:05:45.892872+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'19","last_scrub_stamp":"2024-09-20T19:01:15.544663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:15.544663+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:24.387863+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":729,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T19:05:45.905079+0000","last_change":"2024-09-20T19:01:40.961449+0000","last_active":"2024-09-20T19:05:45.905079+0000","last_peered":"2024-09-20T19:05:45.905079+0000","last_clean":"2024-09-20T19:05:45.905079+0000","last_became_active":"2024-09-20T18:49:49.957582+0000","last_became_peered":"2024-09-20T18:49:49.957582+0000","last_unstale":"2024-09-20T19:05:45.905079+0000","last_undegraded":"2024-09-20T19:05:45.905079+0000","last_fullsized":"2024-09-20T19:05:45.905079+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:40.961363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:40.961363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":718667800,"num_objects":10967,"num_object_clones":0,"num_object_copies":32901,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10967,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":222753,"num_write_kb":7478977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32430,"num_bytes_recovered":1814560792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2188128256,"data_stored":2188116040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91259,"ondisk_log_size":91259,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1185,"internal_metadata":0},"log_size":87,"ondisk_log_size":87,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1114,"num_read_kb":871,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":257,"num_bytes_recovered":4890,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":686,"ondisk_log_size":686,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":3105,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":953,"num_read_kb":948,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266884,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":797380,"kb_used_data":216436,"kb_used_omap":1,"kb_used_meta":580926,"kb_avail":92947772,"statfs":{"total":95995035648,"available":95178518528,"internally_reserved":0,"allocated":221630464,"data_stored":221147475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":594868682},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528523,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":798532,"kb_used_data":361968,"kb_used_omap":1,"kb_used_meta":436542,"kb_avail":92946620,"statfs":{"total":95995035648,"available":95177338880,"internally_reserved":0,"allocated":370655232,"data_stored":370182681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":447019462},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757457,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":735192,"kb_used_data":158408,"kb_used_omap":1,"kb_used_meta":576766,"kb_avail":93009960,"statfs":{"total":95995035648,"available":95242199040,"internally_reserved":0,"allocated":162209792,"data_stored":161719794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":590608837},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":304,"seq":1305670058039,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":552176,"kb_used_data":273308,"kb_used_omap":5,"kb_used_meta":278842,"kb_avail":93192976,"statfs":{"total":95995035648,"available":95429607424,"internally_reserved":0,"allocated":279867392,"data_stored":279357832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5633,"internal_metadata":285534719},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182621,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":851284,"kb_used_data":152196,"kb_used_omap":1,"kb_used_meta":698942,"kb_avail":92893868,"statfs":{"total":95995035648,"available":95123320832,"internally_reserved":0,"allocated":155848704,"data_stored":155362182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":715717063},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411553,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":896708,"kb_used_data":305904,"kb_used_omap":1,"kb_used_meta":590654,"kb_avail":92848444,"statfs":{"total":95995035648,"available":95076806656,"internally_reserved":0,"allocated":313245696,"data_stored":312773515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":604830149},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575079,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":900656,"kb_used_data":291936,"kb_used_omap":1,"kb_used_meta":608702,"kb_avail":92844496,"statfs":{"total":95995035648,"available":95072763904,"internally_reserved":0,"allocated":298942464,"data_stored":298422315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":623311301},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738605,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1111892,"kb_used_data":453188,"kb_used_omap":1,"kb_used_meta":658686,"kb_avail":92633260,"statfs":{"total":95995035648,"available":94856458240,"internally_reserved":0,"allocated":464064512,"data_stored":463571839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":674494917},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,1,4,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1185,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":461377536,"data_stored":461373464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":295501824,"data_stored":295501824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":308412416,"data_stored":308412416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":154009600,"data_stored":154009600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":253956096,"data_stored":253952024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":160235520,"data_stored":160235520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":344461312,"data_stored":344457240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":210173952,"data_stored":210173952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T19:05:57.297 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2024-09-20T19:05:57.297 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd pool set unique_pool_3 pg_num 26 2024-09-20T19:05:57.297 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:57.309 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Removed 116859 objects 2024-09-20T19:05:57.309 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Clean up completed and total clean up time :126.409 2024-09-20T19:05:57.324 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:57.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:57.325 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:57.383 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:57 smithi142 ceph-mon[32508]: 3.9 scrub starts 2024-09-20T19:05:57.383 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:57 smithi142 ceph-mon[32508]: 3.9 scrub ok 2024-09-20T19:05:57.383 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:57 smithi142 ceph-mon[32508]: 6.15 scrub starts 2024-09-20T19:05:57.383 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:57 smithi142 ceph-mon[32508]: 6.15 scrub ok 2024-09-20T19:05:57.426 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:57.447 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:57.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:57.448 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:57.549 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:57.571 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:57.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:57.572 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:57.673 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:57.694 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:57.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:57.695 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:57.750 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:05:57.761 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:05:57 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:05:57.787 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:05:57.796 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:57.817 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:57.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:57.818 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:57.919 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:57.941 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:57.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:57.942 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:58.033 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-20T19:05:58.043 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:58.065 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:58.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:58.066 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:58.125 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:58 smithi083 ceph-mon[29421]: 3.5 scrub starts 2024-09-20T19:05:58.126 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:58 smithi083 ceph-mon[29421]: 3.5 scrub ok 2024-09-20T19:05:58.126 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:58 smithi083 ceph-mon[29421]: pgmap v809: 148 pgs: 148 active+clean; 97 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.33k op/s 2024-09-20T19:05:58.126 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:58 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:05:58.126 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:58 smithi083 ceph-mon[29421]: 6.12 scrub starts 2024-09-20T19:05:58.126 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:58 smithi083 ceph-mon[29421]: 6.12 scrub ok 2024-09-20T19:05:58.126 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:58 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1312366828' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:05:58.126 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:58 smithi083 ceph-mon[36260]: 3.5 scrub starts 2024-09-20T19:05:58.126 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:58 smithi083 ceph-mon[36260]: 3.5 scrub ok 2024-09-20T19:05:58.126 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:58 smithi083 ceph-mon[36260]: pgmap v809: 148 pgs: 148 active+clean; 97 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.33k op/s 2024-09-20T19:05:58.126 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:58 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:05:58.127 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:58 smithi083 ceph-mon[36260]: 6.12 scrub starts 2024-09-20T19:05:58.127 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:58 smithi083 ceph-mon[36260]: 6.12 scrub ok 2024-09-20T19:05:58.127 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:58 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1312366828' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:05:58.167 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:58.189 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:58.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:58.190 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:58.291 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:05:58.312 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:58.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:58.313 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:05:58.340 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T19:05:58.340 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:05:58.413 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:58.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:58.435 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:58.435 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:58.512 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:58 smithi142 ceph-mon[32508]: 3.5 scrub starts 2024-09-20T19:05:58.512 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:58 smithi142 ceph-mon[32508]: 3.5 scrub ok 2024-09-20T19:05:58.512 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:58 smithi142 ceph-mon[32508]: pgmap v809: 148 pgs: 148 active+clean; 97 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.33k op/s 2024-09-20T19:05:58.512 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:58 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:05:58.512 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:58 smithi142 ceph-mon[32508]: 6.12 scrub starts 2024-09-20T19:05:58.512 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:58 smithi142 ceph-mon[32508]: 6.12 scrub ok 2024-09-20T19:05:58.512 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:58 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1312366828' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:05:58.536 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:58.558 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:58.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:58.559 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:58.659 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:58.681 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:58.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:58.682 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:58.782 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:05:58.803 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:58.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:58.804 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:05:58.810 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T19:05:58.906 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:58.927 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:58.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:58.928 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:58.974 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_blocked_ops 2024-09-20T19:05:59.029 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:59.051 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:59.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:59.052 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:59.153 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:59.175 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:59.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:59.176 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:59.277 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:05:59.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:59 smithi083 ceph-mon[36260]: 4.b scrub starts 2024-09-20T19:05:59.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:59 smithi083 ceph-mon[36260]: 4.b scrub ok 2024-09-20T19:05:59.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:59 smithi083 ceph-mon[36260]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:05:59.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:59 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1312366828' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T19:05:59.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:05:59 smithi083 ceph-mon[36260]: osdmap e410: 8 total, 8 up, 8 in 2024-09-20T19:05:59.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:59 smithi083 ceph-mon[29421]: 4.b scrub starts 2024-09-20T19:05:59.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:59 smithi083 ceph-mon[29421]: 4.b scrub ok 2024-09-20T19:05:59.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:59 smithi083 ceph-mon[29421]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:05:59.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:59 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1312366828' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T19:05:59.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:05:59 smithi083 ceph-mon[29421]: osdmap e410: 8 total, 8 up, 8 in 2024-09-20T19:05:59.298 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:59.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:59.299 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:05:59.400 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:59.421 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:59.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:59.422 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:05:59.512 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:59 smithi142 ceph-mon[32508]: 4.b scrub starts 2024-09-20T19:05:59.512 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:59 smithi142 ceph-mon[32508]: 4.b scrub ok 2024-09-20T19:05:59.512 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:59 smithi142 ceph-mon[32508]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:05:59.512 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:59 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1312366828' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T19:05:59.512 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:05:59 smithi142 ceph-mon[32508]: osdmap e410: 8 total, 8 up, 8 in 2024-09-20T19:05:59.524 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:59.550 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:59.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:59.551 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:59.652 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:05:59.677 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:05:59.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:59.678 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:05:59.779 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:05:59.801 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:59.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:59.802 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:05:59.902 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:05:59.923 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:05:59.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:05:59.924 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:00.025 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:00.051 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:00.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:00.052 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:00.077 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:06:00.080 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:06:00.153 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:00.174 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:00.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:00.175 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:00.276 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:00.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:00 smithi083 ceph-mon[29421]: pgmap v811: 148 pgs: 148 active+clean; 53 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.23k op/s 2024-09-20T19:06:00.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:00 smithi083 ceph-mon[36260]: pgmap v811: 148 pgs: 148 active+clean; 53 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.23k op/s 2024-09-20T19:06:00.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:00.303 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:00.303 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:00.404 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:00.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:00.432 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:00.432 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:00.512 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:00 smithi142 ceph-mon[32508]: pgmap v811: 148 pgs: 148 active+clean; 53 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.23k op/s 2024-09-20T19:06:00.532 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:00.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:00.560 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:00.560 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:00.661 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:00.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:00.683 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:00.683 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:00.784 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:00.810 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:00.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:00.811 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:00.865 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T19:06:00.865 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:06:00.903 INFO:teuthology.orchestra.run.smithi083.stderr:set pool 10 pg_num to 26 2024-09-20T19:06:00.903 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:06:00 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:06:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:06:00.913 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:00.937 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:00.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:00.938 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:01.039 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:01.062 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:01.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:01.063 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:01.164 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:01.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:01.189 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:01.190 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:01.290 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:06:01.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:01 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3142799633' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num", "val": "26"} : dispatch 2024-09-20T19:06:01.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:01 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num", "val": "26"} : dispatch 2024-09-20T19:06:01.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:01 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num", "val": "26"}]': finished 2024-09-20T19:06:01.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:01 smithi083 ceph-mon[29421]: osdmap e411: 8 total, 8 up, 8 in 2024-09-20T19:06:01.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:01 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3142799633' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num", "val": "26"} : dispatch 2024-09-20T19:06:01.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:01 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num", "val": "26"} : dispatch 2024-09-20T19:06:01.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:01 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num", "val": "26"}]': finished 2024-09-20T19:06:01.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:01 smithi083 ceph-mon[36260]: osdmap e411: 8 total, 8 up, 8 in 2024-09-20T19:06:01.314 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:01.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:01.315 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:06:01.416 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:01.437 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:01.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:01.439 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:01.512 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:01 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3142799633' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num", "val": "26"} : dispatch 2024-09-20T19:06:01.512 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:01 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num", "val": "26"} : dispatch 2024-09-20T19:06:01.512 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:01 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num", "val": "26"}]': finished 2024-09-20T19:06:01.512 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:01 smithi142 ceph-mon[32508]: osdmap e411: 8 total, 8 up, 8 in 2024-09-20T19:06:01.539 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:01.566 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:01.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:01.567 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:01.574 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_historic_ops 2024-09-20T19:06:01.668 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:01.694 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:01.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:01.695 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:01.776 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:06:01.776 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:06:01.788 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T19:06:01.793 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:06:01.793 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:06:01.795 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:01.809 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:06:01.810 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:06:01.820 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:01.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:01.821 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:01.827 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:06:01.827 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:06:01.860 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:06:01.860 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:06:01.877 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:06:01.877 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:06:01.893 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:06:01.894 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:06:01.922 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:01.926 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:06:01.926 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:06:01.945 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:01.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:01.947 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:02.047 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:06:02.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:02.115 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:02.115 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:06:02.216 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:02.269 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:02.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:02.271 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:02.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:02 smithi083 ceph-mon[29421]: pgmap v813: 148 pgs: 148 active+clean; 20 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 806 op/s 2024-09-20T19:06:02.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:02 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num_actual", "val": "26"} : dispatch 2024-09-20T19:06:02.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:02 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:06:02.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:02 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2832492582' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:06:02.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:02 smithi083 ceph-mon[36260]: pgmap v813: 148 pgs: 148 active+clean; 20 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 806 op/s 2024-09-20T19:06:02.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:02 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num_actual", "val": "26"} : dispatch 2024-09-20T19:06:02.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:02 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:06:02.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:02 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2832492582' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:06:02.371 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:02.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:02.394 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:02.394 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:02.495 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:02.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:02 smithi142 ceph-mon[32508]: pgmap v813: 148 pgs: 148 active+clean; 20 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 806 op/s 2024-09-20T19:06:02.512 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:02 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num_actual", "val": "26"} : dispatch 2024-09-20T19:06:02.512 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:02 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:06:02.512 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:02 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2832492582' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:06:02.516 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:02.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:02.517 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:02.618 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:02.639 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:02.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:02.640 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:02.741 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:02.763 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:02.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:02.764 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:02.865 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:02.886 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:02.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:02.887 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:02.988 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:03.009 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:03.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:03.010 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:03.111 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:06:03.139 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:03.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:03.140 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:06:03.232 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:03 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:06:03.232 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:03 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:06:03.232 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:03 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num_actual", "val": "26"}]': finished 2024-09-20T19:06:03.232 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:03 smithi083 ceph-mon[29421]: osdmap e412: 8 total, 8 up, 8 in 2024-09-20T19:06:03.232 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:03 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:06:03.232 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:03 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:06:03.232 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:03 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num_actual", "val": "26"}]': finished 2024-09-20T19:06:03.233 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:03 smithi083 ceph-mon[36260]: osdmap e412: 8 total, 8 up, 8 in 2024-09-20T19:06:03.241 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:03.265 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:03.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:03.266 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:03.367 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:03.389 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:03.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:03.390 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:03.490 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:03.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:03 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:06:03.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:03 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:06:03.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:03 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num_actual", "val": "26"}]': finished 2024-09-20T19:06:03.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:03 smithi142 ceph-mon[32508]: osdmap e412: 8 total, 8 up, 8 in 2024-09-20T19:06:03.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:03.512 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:03.513 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:03.613 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:03.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:03.636 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:03.636 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:03.737 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:06:03.761 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:03.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:03.762 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:06:03.774 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_3 2024-09-20T19:06:03.775 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd pool rm unique_pool_3 unique_pool_3 --yes-i-really-really-mean-it 2024-09-20T19:06:03.849 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:06:03.855 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:06:03.863 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:03.887 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:03.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:03.888 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:03.989 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:06:04.011 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:04.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:04.012 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:06:04.113 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:04.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:04.140 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:04.140 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:04.240 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:06:04.261 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:04.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:04.262 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:06:04.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:04 smithi083 ceph-mon[29421]: osdmap e413: 8 total, 8 up, 8 in 2024-09-20T19:06:04.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:04 smithi083 ceph-mon[29421]: pgmap v816: 158 pgs: 2 peering, 10 unknown, 146 active+clean; 583 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 998 B/s rd, 0 B/s wr, 163 op/s 2024-09-20T19:06:04.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:04 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pgp_num_actual", "val": "26"} : dispatch 2024-09-20T19:06:04.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:04 smithi083 ceph-mon[36260]: osdmap e413: 8 total, 8 up, 8 in 2024-09-20T19:06:04.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:04 smithi083 ceph-mon[36260]: pgmap v816: 158 pgs: 2 peering, 10 unknown, 146 active+clean; 583 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 998 B/s rd, 0 B/s wr, 163 op/s 2024-09-20T19:06:04.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:04 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pgp_num_actual", "val": "26"} : dispatch 2024-09-20T19:06:04.363 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:04.384 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:04.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:04.385 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:04.486 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:04.508 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:04.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:04.510 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:04.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:04 smithi142 ceph-mon[32508]: osdmap e413: 8 total, 8 up, 8 in 2024-09-20T19:06:04.512 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:04 smithi142 ceph-mon[32508]: pgmap v816: 158 pgs: 2 peering, 10 unknown, 146 active+clean; 583 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 998 B/s rd, 0 B/s wr, 163 op/s 2024-09-20T19:06:04.512 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:04 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pgp_num_actual", "val": "26"} : dispatch 2024-09-20T19:06:04.610 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:04.631 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:04.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:04.632 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:04.733 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:04.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:04.755 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:04.755 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:04.856 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:04.876 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:04.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:04.877 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:04.959 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T19:06:04.960 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-20T19:06:04.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-20T19:06:04.960 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-20T19:06:04.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.960 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.25484.0:124578 9.2 e289/246 9:422abb8a:::benchmark_data_smithi083_124552_object10431:head v 289'7686, mlcod=289'7682)", 2024-09-20T19:06:04.960 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:20.951824+0000", 2024-09-20T19:06:04.960 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 343.99575346099999, 2024-09-20T19:06:04.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.32891357100000002, 2024-09-20T19:06:04.960 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:06:04.960 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:06:04.961 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:04.961 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:06:04.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.961 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:06:04.961 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.951824+0000", 2024-09-20T19:06:04.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:04.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.961 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:06:04.961 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.951824+0000", 2024-09-20T19:06:04.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:04.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.961 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:06:04.962 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.951825+0000", 2024-09-20T19:06:04.962 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0470000000000001e-06 2024-09-20T19:06:04.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.962 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.962 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:06:04.962 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.951831+0000", 2024-09-20T19:06:04.962 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2999999999999998e-06 2024-09-20T19:06:04.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.962 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.962 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:06:04.962 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.951841+0000", 2024-09-20T19:06:04.962 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0234999999999999e-05 2024-09-20T19:06:04.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:06:04.963 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.951854+0000", 2024-09-20T19:06:04.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.287e-05 2024-09-20T19:06:04.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:04.963 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.951926+0000", 2024-09-20T19:06:04.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1932000000000002e-05 2024-09-20T19:06:04.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:06:04.963 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.951943+0000", 2024-09-20T19:06:04.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7033000000000001e-05 2024-09-20T19:06:04.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.964 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.964 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:06:04.964 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.280700+0000", 2024-09-20T19:06:04.964 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.328756979 2024-09-20T19:06:04.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.964 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.964 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:06:04.964 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.280737+0000", 2024-09-20T19:06:04.964 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7175000000000002e-05 2024-09-20T19:06:04.964 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:04.964 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:04.964 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:04.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.965 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.965 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.25484.0:124584 9.0 e289/284 9:06443903:::benchmark_data_smithi083_124552_object10437:head v 289'8049, mlcod=289'8046)", 2024-09-20T19:06:04.965 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:20.953013+0000", 2024-09-20T19:06:04.965 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 343.99456414799999, 2024-09-20T19:06:04.965 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.323892601, 2024-09-20T19:06:04.965 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:06:04.965 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:06:04.965 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:04.965 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:06:04.965 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.966 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:06:04.966 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.953013+0000", 2024-09-20T19:06:04.966 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:04.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.966 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.966 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:06:04.966 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.953013+0000", 2024-09-20T19:06:04.966 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:04.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.966 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.966 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:06:04.966 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.953014+0000", 2024-09-20T19:06:04.967 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0999999999999997e-07 2024-09-20T19:06:04.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.967 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.967 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:06:04.967 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.953020+0000", 2024-09-20T19:06:04.967 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8390000000000003e-06 2024-09-20T19:06:04.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.967 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.967 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:06:04.967 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.953030+0000", 2024-09-20T19:06:04.967 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8530000000000004e-06 2024-09-20T19:06:04.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.967 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.967 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:06:04.968 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.953036+0000", 2024-09-20T19:06:04.968 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1510000000000001e-06 2024-09-20T19:06:04.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.968 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.968 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:04.968 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.953099+0000", 2024-09-20T19:06:04.968 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2959000000000005e-05 2024-09-20T19:06:04.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.968 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.968 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:06:04.968 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.953117+0000", 2024-09-20T19:06:04.968 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8559999999999998e-05 2024-09-20T19:06:04.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.968 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.969 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:06:04.969 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.959129+0000", 2024-09-20T19:06:04.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0060115500000000001 2024-09-20T19:06:04.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.969 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:06:04.969 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.276906+0000", 2024-09-20T19:06:04.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.31777687900000001 2024-09-20T19:06:04.969 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:04.969 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:04.969 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:04.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.969 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.25484.0:124590 9.6 e289/246 9:6ce021c9:::benchmark_data_smithi083_124552_object10443:head v 289'7744, mlcod=289'7742)", 2024-09-20T19:06:04.970 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:20.956000+0000", 2024-09-20T19:06:04.970 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 343.99157690800001, 2024-09-20T19:06:04.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.32402082500000001, 2024-09-20T19:06:04.970 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:06:04.970 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:06:04.970 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:04.970 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:06:04.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.970 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:06:04.970 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.956000+0000", 2024-09-20T19:06:04.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:04.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.970 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:06:04.971 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.956000+0000", 2024-09-20T19:06:04.971 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:04.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.971 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.971 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:06:04.971 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.956001+0000", 2024-09-20T19:06:04.971 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9600000000000008e-07 2024-09-20T19:06:04.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.971 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.971 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:06:04.971 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.956007+0000", 2024-09-20T19:06:04.971 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3199999999999999e-06 2024-09-20T19:06:04.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.971 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.971 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:06:04.972 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.956014+0000", 2024-09-20T19:06:04.972 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7880000000000007e-06 2024-09-20T19:06:04.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.972 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.972 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:06:04.972 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.956019+0000", 2024-09-20T19:06:04.972 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5050000000000001e-06 2024-09-20T19:06:04.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.972 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.972 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:04.972 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.956069+0000", 2024-09-20T19:06:04.972 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0467000000000003e-05 2024-09-20T19:06:04.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.973 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:06:04.973 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.956089+0000", 2024-09-20T19:06:04.973 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.948e-05 2024-09-20T19:06:04.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.973 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:06:04.973 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.279983+0000", 2024-09-20T19:06:04.973 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.32389452099999999 2024-09-20T19:06:04.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.973 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:06:04.974 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.280021+0000", 2024-09-20T19:06:04.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7747999999999998e-05 2024-09-20T19:06:04.974 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:04.974 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:04.974 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:04.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.974 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.25484.0:124594 9.6 e289/246 9:67fbc174:::benchmark_data_smithi083_124552_object10447:head v 289'7745, mlcod=289'7742)", 2024-09-20T19:06:04.974 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:20.960552+0000", 2024-09-20T19:06:04.974 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 343.98702506699999, 2024-09-20T19:06:04.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.32917961699999998, 2024-09-20T19:06:04.974 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:06:04.974 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:06:04.975 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:04.975 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:06:04.975 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.975 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:06:04.975 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.960552+0000", 2024-09-20T19:06:04.975 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:04.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.975 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.975 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:06:04.975 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.960552+0000", 2024-09-20T19:06:04.975 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:04.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.975 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.976 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:06:04.976 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.960553+0000", 2024-09-20T19:06:04.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0610000000000001e-06 2024-09-20T19:06:04.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.976 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:06:04.976 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.960562+0000", 2024-09-20T19:06:04.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0580000000000004e-06 2024-09-20T19:06:04.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.976 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:06:04.976 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.276862+0000", 2024-09-20T19:06:04.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.31629938200000002 2024-09-20T19:06:04.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:06:04.977 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.276872+0000", 2024-09-20T19:06:04.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0614e-05 2024-09-20T19:06:04.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:04.977 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.276943+0000", 2024-09-20T19:06:04.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0933999999999995e-05 2024-09-20T19:06:04.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:06:04.977 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.276962+0000", 2024-09-20T19:06:04.978 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9293999999999999e-05 2024-09-20T19:06:04.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.978 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:06:04.978 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.289671+0000", 2024-09-20T19:06:04.978 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.012708129 2024-09-20T19:06:04.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.978 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:06:04.978 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.289732+0000", 2024-09-20T19:06:04.978 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1144999999999998e-05 2024-09-20T19:06:04.978 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:04.978 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:04.979 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:04.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.979 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.979 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.25484.0:124596 9.2 e289/246 9:42d01fa2:::benchmark_data_smithi083_124552_object10449:head v 289'7687, mlcod=289'7684)", 2024-09-20T19:06:04.979 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:20.963075+0000", 2024-09-20T19:06:04.979 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 343.98450231200002, 2024-09-20T19:06:04.979 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.32681833599999999, 2024-09-20T19:06:04.979 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:06:04.979 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:06:04.979 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:04.979 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:06:04.979 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.979 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:06:04.980 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.963075+0000", 2024-09-20T19:06:04.980 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:04.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.980 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.980 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:06:04.980 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.963075+0000", 2024-09-20T19:06:04.980 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:04.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.980 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.980 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:06:04.980 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.963076+0000", 2024-09-20T19:06:04.980 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6600000000000005e-07 2024-09-20T19:06:04.981 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.981 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.981 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:06:04.981 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.963083+0000", 2024-09-20T19:06:04.981 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9140000000000002e-06 2024-09-20T19:06:04.981 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.981 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.981 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:06:04.981 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.276845+0000", 2024-09-20T19:06:04.981 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.31376241599999999 2024-09-20T19:06:04.981 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.981 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.981 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:06:04.981 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.276871+0000", 2024-09-20T19:06:04.982 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6123999999999999e-05 2024-09-20T19:06:04.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.982 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.982 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:04.982 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.276934+0000", 2024-09-20T19:06:04.982 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3147999999999999e-05 2024-09-20T19:06:04.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.982 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.982 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:06:04.982 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.276960+0000", 2024-09-20T19:06:04.982 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5939e-05 2024-09-20T19:06:04.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.982 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.983 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:06:04.983 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.289643+0000", 2024-09-20T19:06:04.983 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.012682236 2024-09-20T19:06:04.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.983 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.983 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:06:04.983 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.289893+0000", 2024-09-20T19:06:04.983 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00025069300000000002 2024-09-20T19:06:04.983 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:04.983 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:04.983 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:04.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.983 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.984 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.25484.0:210426 9.2 e312/246 9:4c6f61c8:::benchmark_data_smithi083_124552_object96279:head v 312'12986, mlcod=312'12983)", 2024-09-20T19:06:04.984 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:01:34.555445+0000", 2024-09-20T19:06:04.984 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 270.39213212599998, 2024-09-20T19:06:04.984 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.196926772, 2024-09-20T19:06:04.984 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:06:04.984 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:06:04.984 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:04.984 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:06:04.984 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.984 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:06:04.984 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:01:34.555445+0000", 2024-09-20T19:06:04.984 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:04.984 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.984 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.985 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:06:04.985 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:01:34.555445+0000", 2024-09-20T19:06:04.985 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:04.985 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.985 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.985 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:06:04.985 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:01:34.555446+0000", 2024-09-20T19:06:04.985 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.023e-06 2024-09-20T19:06:04.985 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.985 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.985 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:06:04.985 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:01:34.555453+0000", 2024-09-20T19:06:04.985 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4910000000000002e-06 2024-09-20T19:06:04.985 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.985 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.986 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:06:04.986 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:01:34.741524+0000", 2024-09-20T19:06:04.986 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.18607137300000001 2024-09-20T19:06:04.986 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.986 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.986 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:06:04.986 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:01:34.741534+0000", 2024-09-20T19:06:04.986 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0219e-05 2024-09-20T19:06:04.986 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.986 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.986 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:04.986 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:01:34.742048+0000", 2024-09-20T19:06:04.986 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000514267 2024-09-20T19:06:04.987 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.987 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.987 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:06:04.987 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:01:34.742068+0000", 2024-09-20T19:06:04.987 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9916999999999999e-05 2024-09-20T19:06:04.987 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.987 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.987 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:06:04.987 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:01:34.752173+0000", 2024-09-20T19:06:04.987 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.01010463 2024-09-20T19:06:04.987 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.987 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.988 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:06:04.988 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:01:34.752372+0000", 2024-09-20T19:06:04.988 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019885200000000001 2024-09-20T19:06:04.988 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:04.988 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:04.988 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:04.988 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.988 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.988 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25775.0:51921 10.3 10:c83df5a9:::benchmark_data_smithi083_148100_object51920:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e340)", 2024-09-20T19:06:04.988 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:02:42.777109+0000", 2024-09-20T19:06:04.988 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 202.17046851999999, 2024-09-20T19:06:04.988 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.118903313, 2024-09-20T19:06:04.989 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:06:04.989 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:06:04.989 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:04.989 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:06:04.989 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25775", 2024-09-20T19:06:04.989 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:06:04.989 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 51921 2024-09-20T19:06:04.989 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.989 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:06:04.989 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.989 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:06:04.989 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.777109+0000", 2024-09-20T19:06:04.989 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:04.989 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.989 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.990 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:06:04.990 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.777109+0000", 2024-09-20T19:06:04.990 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:04.990 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.990 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.990 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:06:04.990 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.777110+0000", 2024-09-20T19:06:04.990 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4300000000000001e-07 2024-09-20T19:06:04.990 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.990 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.990 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:06:04.990 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.777133+0000", 2024-09-20T19:06:04.991 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3312999999999999e-05 2024-09-20T19:06:04.991 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.991 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.991 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:06:04.991 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.777143+0000", 2024-09-20T19:06:04.991 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0111e-05 2024-09-20T19:06:04.991 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.991 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.991 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:06:04.991 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.777150+0000", 2024-09-20T19:06:04.991 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7889999999999997e-06 2024-09-20T19:06:04.991 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.991 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.991 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:04.991 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.777197+0000", 2024-09-20T19:06:04.992 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7525000000000003e-05 2024-09-20T19:06:04.992 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.992 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.992 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:06:04.992 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.777304+0000", 2024-09-20T19:06:04.992 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000106548 2024-09-20T19:06:04.992 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.992 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.992 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3,5", 2024-09-20T19:06:04.992 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.777571+0000", 2024-09-20T19:06:04.992 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00026748100000000002 2024-09-20T19:06:04.992 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.992 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.993 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:04.993 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.780123+0000", 2024-09-20T19:06:04.993 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002551813 2024-09-20T19:06:04.993 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.993 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.993 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:06:04.993 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.781319+0000", 2024-09-20T19:06:04.993 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001195619 2024-09-20T19:06:04.993 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.993 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.993 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:04.993 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.895858+0000", 2024-09-20T19:06:04.993 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.114538906 2024-09-20T19:06:04.993 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.994 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.994 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:06:04.994 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.895962+0000", 2024-09-20T19:06:04.994 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000104503 2024-09-20T19:06:04.994 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.994 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.994 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:06:04.994 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:02:42.896012+0000", 2024-09-20T19:06:04.994 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9762000000000002e-05 2024-09-20T19:06:04.994 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:04.994 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:04.995 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:04.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.995 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.995 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.25775.0:114741 10.8 e361/325 10:1a525e15:::benchmark_data_smithi083_148100_object114740:head v 361'7078, mlcod=361'7072)", 2024-09-20T19:06:04.995 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:03:47.220020+0000", 2024-09-20T19:06:04.995 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 137.72755722599999, 2024-09-20T19:06:04.995 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.11739464400000001, 2024-09-20T19:06:04.995 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:06:04.995 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:06:04.995 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:04.995 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:06:04.995 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.995 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:06:04.996 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.220020+0000", 2024-09-20T19:06:04.996 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:04.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.996 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.996 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:06:04.996 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.220020+0000", 2024-09-20T19:06:04.996 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:04.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.996 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.996 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:06:04.996 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.220021+0000", 2024-09-20T19:06:04.996 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9400000000000005e-07 2024-09-20T19:06:04.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.996 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.997 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:06:04.997 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.220043+0000", 2024-09-20T19:06:04.997 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1846999999999999e-05 2024-09-20T19:06:04.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.997 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.997 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:06:04.997 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.220051+0000", 2024-09-20T19:06:04.997 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9020000000000005e-06 2024-09-20T19:06:04.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.997 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.997 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:06:04.997 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.220058+0000", 2024-09-20T19:06:04.997 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5350000000000001e-06 2024-09-20T19:06:04.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.998 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.998 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:04.998 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.220102+0000", 2024-09-20T19:06:04.998 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3554999999999999e-05 2024-09-20T19:06:04.998 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.998 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.998 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:06:04.998 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.220118+0000", 2024-09-20T19:06:04.998 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6895999999999999e-05 2024-09-20T19:06:04.998 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.998 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.998 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:06:04.998 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.222643+0000", 2024-09-20T19:06:04.998 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0025243190000000001 2024-09-20T19:06:04.999 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.999 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.999 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:06:04.999 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.337415+0000", 2024-09-20T19:06:04.999 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.114771896 2024-09-20T19:06:04.999 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:04.999 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:04.999 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:04.999 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:04.999 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:04.999 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25775.0:114742 10.5 10:a8d0d8fc:::benchmark_data_smithi083_148100_object114741:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e361)", 2024-09-20T19:06:04.999 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:03:47.222659+0000", 2024-09-20T19:06:04.999 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 137.72491793, 2024-09-20T19:06:05.000 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.13039831599999999, 2024-09-20T19:06:05.000 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:06:05.000 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:06:05.000 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:05.000 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:06:05.000 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25775", 2024-09-20T19:06:05.000 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:06:05.000 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 114742 2024-09-20T19:06:05.000 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.000 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:06:05.000 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.000 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:06:05.000 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.222659+0000", 2024-09-20T19:06:05.000 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:05.001 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.001 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.001 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:06:05.001 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.222659+0000", 2024-09-20T19:06:05.001 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:05.001 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.001 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.001 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:06:05.001 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.222661+0000", 2024-09-20T19:06:05.001 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5510000000000001e-06 2024-09-20T19:06:05.001 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.001 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.001 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:06:05.002 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.222684+0000", 2024-09-20T19:06:05.002 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2759999999999999e-05 2024-09-20T19:06:05.002 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.002 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.002 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:06:05.002 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.337385+0000", 2024-09-20T19:06:05.002 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.114700989 2024-09-20T19:06:05.002 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.002 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.002 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:06:05.002 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.337402+0000", 2024-09-20T19:06:05.002 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7537000000000002e-05 2024-09-20T19:06:05.002 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.003 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.003 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:05.003 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.337469+0000", 2024-09-20T19:06:05.003 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6610000000000001e-05 2024-09-20T19:06:05.003 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.003 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.003 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:06:05.003 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.337616+0000", 2024-09-20T19:06:05.003 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000147274 2024-09-20T19:06:05.003 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:05.005 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.005 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.005 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,3", 2024-09-20T19:06:05.005 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.337807+0000", 2024-09-20T19:06:05.005 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019082800000000001 2024-09-20T19:06:05.005 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.005 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.005 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:06:05.005 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.340737+0000", 2024-09-20T19:06:05.005 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0029298409999999999 2024-09-20T19:06:05.005 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.006 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.006 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:05.006 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.347617+0000", 2024-09-20T19:06:05.006 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0068798410000000003 2024-09-20T19:06:05.006 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.006 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.006 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:05.006 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.352900+0000", 2024-09-20T19:06:05.006 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.005283444 2024-09-20T19:06:05.006 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.006 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.006 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:06:05.006 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.352958+0000", 2024-09-20T19:06:05.007 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8208999999999998e-05 2024-09-20T19:06:05.007 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.007 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.007 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:06:05.007 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.353058+0000", 2024-09-20T19:06:05.007 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9432000000000006e-05 2024-09-20T19:06:05.007 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:05.007 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:05.007 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:05.008 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.008 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.008 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.25775.0:114753 10.9 e361/325 10:9b143418:::benchmark_data_smithi083_148100_object114752:head v 361'7224, mlcod=361'7222)", 2024-09-20T19:06:05.008 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:03:47.234595+0000", 2024-09-20T19:06:05.008 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 137.71298222499999, 2024-09-20T19:06:05.008 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.122020085, 2024-09-20T19:06:05.008 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:06:05.008 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:06:05.008 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:05.008 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:06:05.008 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.008 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:06:05.009 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.234595+0000", 2024-09-20T19:06:05.009 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:05.009 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.009 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.009 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:06:05.009 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.234595+0000", 2024-09-20T19:06:05.009 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:05.009 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.009 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.009 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:06:05.009 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.234596+0000", 2024-09-20T19:06:05.009 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1999999999999999e-06 2024-09-20T19:06:05.009 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.010 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.010 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:06:05.010 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.234667+0000", 2024-09-20T19:06:05.010 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1254999999999995e-05 2024-09-20T19:06:05.010 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.010 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.010 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:06:05.010 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.337440+0000", 2024-09-20T19:06:05.010 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.10277243599999999 2024-09-20T19:06:05.010 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.010 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.010 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:06:05.010 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.337447+0000", 2024-09-20T19:06:05.010 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2099999999999996e-06 2024-09-20T19:06:05.011 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.011 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.011 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:05.011 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.337509+0000", 2024-09-20T19:06:05.011 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2257999999999999e-05 2024-09-20T19:06:05.011 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.011 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.011 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:06:05.011 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.337531+0000", 2024-09-20T19:06:05.011 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1491000000000001e-05 2024-09-20T19:06:05.011 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.011 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.012 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:06:05.012 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.356376+0000", 2024-09-20T19:06:05.012 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.018845641 2024-09-20T19:06:05.012 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.012 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.012 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:06:05.012 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:03:47.356615+0000", 2024-09-20T19:06:05.012 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00023859400000000001 2024-09-20T19:06:05.012 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:05.012 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:05.012 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:05.012 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.012 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.012 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.25775.0:177215 10.f e384/325 10:ff96534a:::benchmark_data_smithi083_148100_object60353:head v 384'10956, mlcod=384'10954)", 2024-09-20T19:06:05.012 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:05:00.561926+0000", 2024-09-20T19:06:05.013 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 64.385650964000007, 2024-09-20T19:06:05.013 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.117835574, 2024-09-20T19:06:05.013 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:06:05.013 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:06:05.013 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:05.013 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:06:05.013 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.013 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:06:05.013 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:00.561926+0000", 2024-09-20T19:06:05.013 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:05.013 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.013 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.013 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:06:05.013 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:00.561926+0000", 2024-09-20T19:06:05.014 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:05.014 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.014 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.014 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:06:05.014 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:00.561927+0000", 2024-09-20T19:06:05.014 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7400000000000002e-07 2024-09-20T19:06:05.014 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.014 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.014 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:06:05.014 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:00.561936+0000", 2024-09-20T19:06:05.014 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8349999999999993e-06 2024-09-20T19:06:05.014 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.014 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.015 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:06:05.015 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:00.561945+0000", 2024-09-20T19:06:05.015 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6250000000000002e-06 2024-09-20T19:06:05.015 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.015 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.015 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:06:05.015 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:00.561952+0000", 2024-09-20T19:06:05.015 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1140000000000002e-06 2024-09-20T19:06:05.015 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.015 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.015 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:05.016 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:00.561993+0000", 2024-09-20T19:06:05.016 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1878999999999997e-05 2024-09-20T19:06:05.016 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.016 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.016 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:06:05.016 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:00.562006+0000", 2024-09-20T19:06:05.016 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2478e-05 2024-09-20T19:06:05.016 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.016 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.016 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:06:05.016 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:00.679707+0000", 2024-09-20T19:06:05.016 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.11770117500000001 2024-09-20T19:06:05.016 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.017 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.017 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:06:05.017 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:00.679762+0000", 2024-09-20T19:06:05.017 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4694000000000002e-05 2024-09-20T19:06:05.017 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:05.017 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:05.017 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:05.017 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.017 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.017 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.25775.0:177220 10.8 e384/325 10:193f2977:::benchmark_data_smithi083_148100_object60358:head v 384'10885, mlcod=384'10883)", 2024-09-20T19:06:05.017 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:05:00.562036+0000", 2024-09-20T19:06:05.017 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 64.385540770000006, 2024-09-20T19:06:05.017 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.11795604699999999, 2024-09-20T19:06:05.017 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:06:05.017 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:06:05.018 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:05.018 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:06:05.018 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.018 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:06:05.018 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:00.562036+0000", 2024-09-20T19:06:05.018 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:05.018 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.018 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.018 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:06:05.018 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:00.562036+0000", 2024-09-20T19:06:05.018 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:05.018 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.018 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.019 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:06:05.019 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:00.562037+0000", 2024-09-20T19:06:05.019 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2200000000000003e-07 2024-09-20T19:06:05.019 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.019 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.019 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:06:05.019 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:00.562043+0000", 2024-09-20T19:06:05.019 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3759999999999999e-06 2024-09-20T19:06:05.019 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.019 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.019 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:06:05.019 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:00.562053+0000", 2024-09-20T19:06:05.019 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0341e-05 2024-09-20T19:06:05.020 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.020 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.020 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:06:05.020 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:00.562059+0000", 2024-09-20T19:06:05.020 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2210000000000002e-06 2024-09-20T19:06:05.020 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.020 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.020 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:05.020 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:00.672526+0000", 2024-09-20T19:06:05.020 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.110466731 2024-09-20T19:06:05.020 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.020 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.020 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:06:05.020 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:00.672548+0000", 2024-09-20T19:06:05.021 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2115000000000001e-05 2024-09-20T19:06:05.021 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.021 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.021 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:06:05.021 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:00.679952+0000", 2024-09-20T19:06:05.021 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.007404323 2024-09-20T19:06:05.021 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.021 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.021 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:06:05.021 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:00.679992+0000", 2024-09-20T19:06:05.021 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0218e-05 2024-09-20T19:06:05.021 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:05.022 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:05.022 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:05.022 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.022 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.022 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.25775.0:177226 10.9 e384/325 10:99c0ea68:::benchmark_data_smithi083_148100_object60364:head v 384'11193, mlcod=384'11191)", 2024-09-20T19:06:05.022 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:05:00.575562+0000", 2024-09-20T19:06:05.022 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 64.372015486999999, 2024-09-20T19:06:05.022 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.123393567, 2024-09-20T19:06:05.022 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:06:05.022 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:06:05.022 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:05.023 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:06:05.023 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.023 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:06:05.023 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:00.575562+0000", 2024-09-20T19:06:05.023 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:05.023 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.023 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.023 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:06:05.023 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:00.575562+0000", 2024-09-20T19:06:05.023 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:05.023 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.023 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.023 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:06:05.024 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:00.575563+0000", 2024-09-20T19:06:05.024 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2589999999999999e-06 2024-09-20T19:06:05.024 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.024 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.024 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:06:05.024 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:00.575571+0000", 2024-09-20T19:06:05.024 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.6979999999999998e-06 2024-09-20T19:06:05.024 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.024 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.024 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:06:05.024 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:00.669542+0000", 2024-09-20T19:06:05.024 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.093971599000000003 2024-09-20T19:06:05.024 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.024 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.025 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:06:05.025 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:00.669558+0000", 2024-09-20T19:06:05.025 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6010000000000001e-05 2024-09-20T19:06:05.025 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.025 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.025 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:05.025 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:00.669879+0000", 2024-09-20T19:06:05.025 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00032046700000000001 2024-09-20T19:06:05.025 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.025 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.025 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:06:05.025 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:00.669900+0000", 2024-09-20T19:06:05.025 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0944000000000001e-05 2024-09-20T19:06:05.026 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.026 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.026 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:06:05.026 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:00.698919+0000", 2024-09-20T19:06:05.026 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.029019053 2024-09-20T19:06:05.026 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.026 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.026 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:06:05.026 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:00.698955+0000", 2024-09-20T19:06:05.026 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6536999999999999e-05 2024-09-20T19:06:05.026 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:05.026 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:05.026 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:05.027 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.027 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.027 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25775.0:178609 10.5 10:a2a0ff3b:::benchmark_data_smithi083_148100_object61747:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e385)", 2024-09-20T19:06:05.027 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:05:02.057106+0000", 2024-09-20T19:06:05.027 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 62.890471181999999, 2024-09-20T19:06:05.027 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.131067458, 2024-09-20T19:06:05.027 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:06:05.027 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:06:05.027 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:05.027 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:06:05.027 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25775", 2024-09-20T19:06:05.027 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:06:05.027 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 178609 2024-09-20T19:06:05.027 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.028 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:06:05.028 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.028 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:06:05.028 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.057106+0000", 2024-09-20T19:06:05.028 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:05.028 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.028 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.028 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:06:05.028 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.057106+0000", 2024-09-20T19:06:05.028 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:05.028 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.028 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.028 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:06:05.029 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.057106+0000", 2024-09-20T19:06:05.029 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.51e-07 2024-09-20T19:06:05.029 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.029 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.029 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:06:05.029 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.057109+0000", 2024-09-20T19:06:05.029 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3259999999999998e-06 2024-09-20T19:06:05.029 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.029 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.029 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:06:05.029 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.057114+0000", 2024-09-20T19:06:05.030 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4360000000000001e-06 2024-09-20T19:06:05.030 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.030 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.030 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:06:05.030 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.057119+0000", 2024-09-20T19:06:05.030 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7299999999999996e-06 2024-09-20T19:06:05.030 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.030 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.030 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:05.030 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.061428+0000", 2024-09-20T19:06:05.030 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0043094420000000001 2024-09-20T19:06:05.030 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.031 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.031 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:06:05.031 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.061535+0000", 2024-09-20T19:06:05.031 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010623 2024-09-20T19:06:05.031 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.031 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.031 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,3", 2024-09-20T19:06:05.031 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.061603+0000", 2024-09-20T19:06:05.031 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7891000000000004e-05 2024-09-20T19:06:05.031 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.031 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.031 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:06:05.031 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.067955+0000", 2024-09-20T19:06:05.032 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063527369999999998 2024-09-20T19:06:05.032 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.032 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.032 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:05.032 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.068419+0000", 2024-09-20T19:06:05.032 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00046344300000000001 2024-09-20T19:06:05.032 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.032 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.032 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:05.032 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.188080+0000", 2024-09-20T19:06:05.032 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.11966094300000001 2024-09-20T19:06:05.032 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.032 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.033 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:06:05.033 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.188130+0000", 2024-09-20T19:06:05.033 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0263999999999999e-05 2024-09-20T19:06:05.033 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.033 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.033 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:06:05.033 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.188173+0000", 2024-09-20T19:06:05.033 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3565e-05 2024-09-20T19:06:05.033 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:05.033 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:05.033 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:05.033 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.033 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.033 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25775.0:178612 10.3 10:c295b8d2:::benchmark_data_smithi083_148100_object61750:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e385)", 2024-09-20T19:06:05.034 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:05:02.061223+0000", 2024-09-20T19:06:05.034 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 62.886354640999997, 2024-09-20T19:06:05.034 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.136060128, 2024-09-20T19:06:05.034 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:06:05.034 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:06:05.034 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:05.034 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:06:05.034 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25775", 2024-09-20T19:06:05.034 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:06:05.034 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 178612 2024-09-20T19:06:05.034 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.034 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:06:05.034 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.035 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:06:05.035 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.061223+0000", 2024-09-20T19:06:05.035 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:05.035 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.035 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.035 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:06:05.035 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.061223+0000", 2024-09-20T19:06:05.035 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:05.035 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.035 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.035 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:06:05.035 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.061224+0000", 2024-09-20T19:06:05.036 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.052e-06 2024-09-20T19:06:05.036 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.036 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.036 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:06:05.036 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.061229+0000", 2024-09-20T19:06:05.036 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9099999999999996e-06 2024-09-20T19:06:05.036 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.036 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.036 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:06:05.036 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.061241+0000", 2024-09-20T19:06:05.036 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2369e-05 2024-09-20T19:06:05.036 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.036 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.037 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:06:05.037 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.061248+0000", 2024-09-20T19:06:05.037 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2060000000000001e-06 2024-09-20T19:06:05.037 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.037 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.037 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:05.037 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.062030+0000", 2024-09-20T19:06:05.037 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00078174699999999995 2024-09-20T19:06:05.037 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.037 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.037 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:06:05.037 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.062145+0000", 2024-09-20T19:06:05.037 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011499 2024-09-20T19:06:05.038 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.038 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.038 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3,5", 2024-09-20T19:06:05.038 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.062556+0000", 2024-09-20T19:06:05.038 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00041131499999999999 2024-09-20T19:06:05.038 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.038 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.038 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:06:05.038 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.066224+0000", 2024-09-20T19:06:05.038 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0036681449999999998 2024-09-20T19:06:05.038 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.038 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.038 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:05.039 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.082037+0000", 2024-09-20T19:06:05.039 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.015813092000000001 2024-09-20T19:06:05.039 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.039 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.039 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:05.039 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.197098+0000", 2024-09-20T19:06:05.039 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.115060783 2024-09-20T19:06:05.039 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.039 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.039 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:06:05.039 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.197208+0000", 2024-09-20T19:06:05.039 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000110139 2024-09-20T19:06:05.039 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.039 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.040 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:06:05.040 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.197283+0000", 2024-09-20T19:06:05.040 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4380000000000003e-05 2024-09-20T19:06:05.040 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:05.040 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:05.040 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:05.040 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.040 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.040 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25775.0:178614 10.5 10:a8ffc50b:::benchmark_data_smithi083_148100_object61752:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e385)", 2024-09-20T19:06:05.040 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:05:02.061485+0000", 2024-09-20T19:06:05.040 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 62.886091897, 2024-09-20T19:06:05.040 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.14234475399999999, 2024-09-20T19:06:05.040 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:06:05.041 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:06:05.041 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:05.041 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:06:05.041 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25775", 2024-09-20T19:06:05.041 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:06:05.041 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 178614 2024-09-20T19:06:05.041 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.041 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:06:05.041 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.041 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:06:05.041 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.061485+0000", 2024-09-20T19:06:05.041 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:05.041 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.041 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.042 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:06:05.042 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.061485+0000", 2024-09-20T19:06:05.042 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:05.042 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.042 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.042 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:06:05.042 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.061486+0000", 2024-09-20T19:06:05.042 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7800000000000001e-07 2024-09-20T19:06:05.042 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.042 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.042 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:06:05.042 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.061490+0000", 2024-09-20T19:06:05.042 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6389999999999998e-06 2024-09-20T19:06:05.043 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.043 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.043 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:06:05.043 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.061497+0000", 2024-09-20T19:06:05.043 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2910000000000002e-06 2024-09-20T19:06:05.043 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.043 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.043 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:06:05.043 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.061503+0000", 2024-09-20T19:06:05.043 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3010000000000003e-06 2024-09-20T19:06:05.043 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.043 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.044 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:05.044 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.064738+0000", 2024-09-20T19:06:05.044 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0032344679999999999 2024-09-20T19:06:05.044 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.044 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.044 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:06:05.044 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.064874+0000", 2024-09-20T19:06:05.044 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000136508 2024-09-20T19:06:05.044 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.044 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.044 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,3", 2024-09-20T19:06:05.044 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.064996+0000", 2024-09-20T19:06:05.044 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000121697 2024-09-20T19:06:05.044 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.045 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.045 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:06:05.045 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.068071+0000", 2024-09-20T19:06:05.045 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003074874 2024-09-20T19:06:05.045 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.045 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.045 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:05.045 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.068614+0000", 2024-09-20T19:06:05.045 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00054337099999999996 2024-09-20T19:06:05.045 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.045 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.045 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:05.045 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.203674+0000", 2024-09-20T19:06:05.045 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.13506006200000001 2024-09-20T19:06:05.046 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.046 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.046 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:06:05.046 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.203780+0000", 2024-09-20T19:06:05.046 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010582300000000001 2024-09-20T19:06:05.046 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.046 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.046 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:06:05.046 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.203830+0000", 2024-09-20T19:06:05.046 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9941999999999997e-05 2024-09-20T19:06:05.046 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:05.046 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:05.046 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:05.047 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.047 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.047 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25775.0:178633 10.5 10:aed3b378:::benchmark_data_smithi083_148100_object61771:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e385)", 2024-09-20T19:06:05.047 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:05:02.087430+0000", 2024-09-20T19:06:05.047 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 62.860147112999996, 2024-09-20T19:06:05.047 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.116569122, 2024-09-20T19:06:05.047 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:06:05.047 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:06:05.047 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:05.047 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:06:05.047 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25775", 2024-09-20T19:06:05.047 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:06:05.047 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 178633 2024-09-20T19:06:05.048 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.048 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:06:05.048 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.048 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:06:05.048 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.087430+0000", 2024-09-20T19:06:05.048 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:05.048 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.048 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.048 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:06:05.048 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.087430+0000", 2024-09-20T19:06:05.048 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:05.048 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.048 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.048 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:06:05.048 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.087431+0000", 2024-09-20T19:06:05.049 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.068e-06 2024-09-20T19:06:05.049 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.049 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.049 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:06:05.049 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.087436+0000", 2024-09-20T19:06:05.049 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9150000000000004e-06 2024-09-20T19:06:05.049 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.049 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.049 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:06:05.049 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.087449+0000", 2024-09-20T19:06:05.049 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2986000000000001e-05 2024-09-20T19:06:05.049 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.049 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.049 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:06:05.050 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.087457+0000", 2024-09-20T19:06:05.050 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2020000000000004e-06 2024-09-20T19:06:05.050 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.050 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.050 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:05.050 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.087514+0000", 2024-09-20T19:06:05.050 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6505999999999997e-05 2024-09-20T19:06:05.050 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.050 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.050 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:06:05.050 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.087630+0000", 2024-09-20T19:06:05.050 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000116494 2024-09-20T19:06:05.050 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.051 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.051 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,3", 2024-09-20T19:06:05.051 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.087697+0000", 2024-09-20T19:06:05.051 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7105000000000002e-05 2024-09-20T19:06:05.051 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.051 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.051 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:06:05.051 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.090806+0000", 2024-09-20T19:06:05.051 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0031086920000000001 2024-09-20T19:06:05.051 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.051 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.052 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:05.052 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.091395+0000", 2024-09-20T19:06:05.052 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00058860900000000003 2024-09-20T19:06:05.052 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.052 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.052 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:05.052 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.203885+0000", 2024-09-20T19:06:05.052 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.112490048 2024-09-20T19:06:05.052 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.052 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.052 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:06:05.052 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.203964+0000", 2024-09-20T19:06:05.052 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8991000000000003e-05 2024-09-20T19:06:05.052 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.053 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.053 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:06:05.053 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:02.203999+0000", 2024-09-20T19:06:05.053 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5506000000000002e-05 2024-09-20T19:06:05.053 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:05.053 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:05.053 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:05.053 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.053 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.053 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25775.0:182428 10.3 10:cacecc61:::benchmark_data_smithi083_148100_object65566:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e386)", 2024-09-20T19:06:05.053 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:05:06.494093+0000", 2024-09-20T19:06:05.053 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 58.453484437999997, 2024-09-20T19:06:05.053 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.132971596, 2024-09-20T19:06:05.054 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:06:05.054 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:06:05.054 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:05.054 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:06:05.054 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25775", 2024-09-20T19:06:05.054 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:06:05.054 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 182428 2024-09-20T19:06:05.054 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.054 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:06:05.054 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.054 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:06:05.054 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:06.494093+0000", 2024-09-20T19:06:05.054 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:05.055 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.055 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.055 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:06:05.055 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:06.494093+0000", 2024-09-20T19:06:05.055 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:05.055 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.055 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.055 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:06:05.055 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:06.494094+0000", 2024-09-20T19:06:05.055 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6270000000000001e-06 2024-09-20T19:06:05.055 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.055 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.055 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:06:05.056 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:06.494100+0000", 2024-09-20T19:06:05.056 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8069999999999996e-06 2024-09-20T19:06:05.056 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.056 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.056 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:06:05.056 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:06.494115+0000", 2024-09-20T19:06:05.056 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4688999999999999e-05 2024-09-20T19:06:05.056 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.056 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.056 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:06:05.056 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:06.494122+0000", 2024-09-20T19:06:05.056 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4719999999999998e-06 2024-09-20T19:06:05.056 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.056 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.057 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:05.057 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:06.494199+0000", 2024-09-20T19:06:05.057 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.6657999999999997e-05 2024-09-20T19:06:05.057 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.057 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.057 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:06:05.057 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:06.494433+0000", 2024-09-20T19:06:05.057 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00023409200000000001 2024-09-20T19:06:05.057 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.057 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.057 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3,5", 2024-09-20T19:06:05.057 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:06.494532+0000", 2024-09-20T19:06:05.057 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8972e-05 2024-09-20T19:06:05.057 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.058 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.058 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:06:05.058 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:06.498751+0000", 2024-09-20T19:06:05.058 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0042191779999999996 2024-09-20T19:06:05.058 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.058 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.058 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:05.058 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:06.500959+0000", 2024-09-20T19:06:05.058 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0022078879999999999 2024-09-20T19:06:05.058 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.058 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.059 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:05.059 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:06.626966+0000", 2024-09-20T19:06:05.059 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.126007227 2024-09-20T19:06:05.059 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.059 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.059 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:06:05.059 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:06.627012+0000", 2024-09-20T19:06:05.059 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5467999999999999e-05 2024-09-20T19:06:05.059 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.059 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.059 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:06:05.059 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:06.627064+0000", 2024-09-20T19:06:05.059 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2518000000000001e-05 2024-09-20T19:06:05.059 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:05.060 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:05.060 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:05.060 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.060 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.060 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25775.0:182450 10.3 10:c09d6732:::benchmark_data_smithi083_148100_object65588:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e386)", 2024-09-20T19:06:05.060 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:05:06.514181+0000", 2024-09-20T19:06:05.060 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 58.433396354000003, 2024-09-20T19:06:05.060 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.12122742, 2024-09-20T19:06:05.060 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:06:05.060 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:06:05.060 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:05.060 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:06:05.060 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25775", 2024-09-20T19:06:05.060 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:06:05.061 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 182450 2024-09-20T19:06:05.061 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.061 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:06:05.061 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.061 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:06:05.061 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:06.514181+0000", 2024-09-20T19:06:05.061 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:05.061 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.061 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.061 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:06:05.061 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:06.514181+0000", 2024-09-20T19:06:05.061 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:05.061 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.061 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.062 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:06:05.062 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:06.514182+0000", 2024-09-20T19:06:05.062 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5659999999999999e-06 2024-09-20T19:06:05.062 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.062 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.062 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:06:05.062 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:06.514187+0000", 2024-09-20T19:06:05.062 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7910000000000004e-06 2024-09-20T19:06:05.062 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.062 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.062 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:06:05.062 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:06.514201+0000", 2024-09-20T19:06:05.062 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4184e-05 2024-09-20T19:06:05.062 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.063 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.063 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:06:05.063 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:06.514209+0000", 2024-09-20T19:06:05.063 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.345e-06 2024-09-20T19:06:05.063 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.063 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.066 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:05.066 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:06.514298+0000", 2024-09-20T19:06:05.066 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9528000000000001e-05 2024-09-20T19:06:05.066 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.066 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.066 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:06:05.066 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:06.514409+0000", 2024-09-20T19:06:05.066 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000110967 2024-09-20T19:06:05.066 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.066 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.066 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3,5", 2024-09-20T19:06:05.067 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:06.514515+0000", 2024-09-20T19:06:05.067 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010602700000000001 2024-09-20T19:06:05.067 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.067 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.067 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:05.067 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:06.519454+0000", 2024-09-20T19:06:05.067 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.004938876 2024-09-20T19:06:05.067 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.067 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.067 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:06:05.067 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:06.519539+0000", 2024-09-20T19:06:05.067 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4765000000000006e-05 2024-09-20T19:06:05.067 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.068 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.068 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:05.068 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:06.635330+0000", 2024-09-20T19:06:05.068 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.11579107299999999 2024-09-20T19:06:05.068 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.068 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.068 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:06:05.068 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:06.635372+0000", 2024-09-20T19:06:05.068 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2162000000000001e-05 2024-09-20T19:06:05.068 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.068 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.068 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:06:05.068 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:06.635408+0000", 2024-09-20T19:06:05.069 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6136000000000001e-05 2024-09-20T19:06:05.069 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:05.069 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:05.069 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:05.069 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.069 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.069 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25775.0:182454 10.3 10:c6e575d4:::benchmark_data_smithi083_148100_object65592:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e386)", 2024-09-20T19:06:05.069 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:05:06.521034+0000", 2024-09-20T19:06:05.069 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 58.426542967000003, 2024-09-20T19:06:05.069 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.117551088, 2024-09-20T19:06:05.069 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:06:05.069 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:06:05.069 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:05.070 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:06:05.070 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25775", 2024-09-20T19:06:05.070 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:06:05.070 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 182454 2024-09-20T19:06:05.070 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.070 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:06:05.070 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.070 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:06:05.070 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:06.521034+0000", 2024-09-20T19:06:05.070 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:05.070 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.070 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.070 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:06:05.071 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:06.521034+0000", 2024-09-20T19:06:05.071 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:05.071 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.071 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.071 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:06:05.071 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:06.521036+0000", 2024-09-20T19:06:05.071 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.883e-06 2024-09-20T19:06:05.071 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.071 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.071 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:06:05.071 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:06.521042+0000", 2024-09-20T19:06:05.071 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6119999999999998e-06 2024-09-20T19:06:05.071 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.071 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.072 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:06:05.072 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:06.521060+0000", 2024-09-20T19:06:05.072 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8108e-05 2024-09-20T19:06:05.072 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.072 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.072 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:06:05.072 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:06.521067+0000", 2024-09-20T19:06:05.072 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3320000000000003e-06 2024-09-20T19:06:05.072 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.072 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.072 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:05.073 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:06.521120+0000", 2024-09-20T19:06:05.073 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2434e-05 2024-09-20T19:06:05.073 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.073 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.073 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:06:05.073 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:06.521235+0000", 2024-09-20T19:06:05.073 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000115263 2024-09-20T19:06:05.073 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.073 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.073 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3,5", 2024-09-20T19:06:05.073 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:06.521411+0000", 2024-09-20T19:06:05.073 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017640200000000001 2024-09-20T19:06:05.073 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.073 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.074 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:06:05.074 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:06.530166+0000", 2024-09-20T19:06:05.074 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0087551710000000008 2024-09-20T19:06:05.074 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.074 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.074 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:05.074 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:06.532030+0000", 2024-09-20T19:06:05.074 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001863951 2024-09-20T19:06:05.074 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.074 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.074 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:05.074 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:06.638486+0000", 2024-09-20T19:06:05.074 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.106455856 2024-09-20T19:06:05.075 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.075 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.075 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:06:05.075 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:06.638538+0000", 2024-09-20T19:06:05.075 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2187e-05 2024-09-20T19:06:05.075 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:05.075 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:05.075 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:06:05.075 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:05:06.638585+0000", 2024-09-20T19:06:05.075 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6888999999999998e-05 2024-09-20T19:06:05.075 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:05.075 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:05.075 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:05.075 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:05.076 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:05.076 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T19:06:05.076 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-20T19:06:05.099 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:05.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:05.100 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:05.201 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:05.224 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:05.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:05.226 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:05.264 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T19:06:05.280 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:05 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pgp_num_actual", "val": "26"}]': finished 2024-09-20T19:06:05.280 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:05 smithi083 ceph-mon[29421]: osdmap e414: 8 total, 8 up, 8 in 2024-09-20T19:06:05.280 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:05 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/78215351' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:06:05.280 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:05 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pgp_num_actual", "val": "26"}]': finished 2024-09-20T19:06:05.281 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:05 smithi083 ceph-mon[36260]: osdmap e414: 8 total, 8 up, 8 in 2024-09-20T19:06:05.281 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:05 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/78215351' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:06:05.326 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:05.347 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:05.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:05.348 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:05.449 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:05.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:05.471 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:05.472 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:05.512 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:05 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pgp_num_actual", "val": "26"}]': finished 2024-09-20T19:06:05.512 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:05 smithi142 ceph-mon[32508]: osdmap e414: 8 total, 8 up, 8 in 2024-09-20T19:06:05.512 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:05 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/78215351' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:06:05.572 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:05.593 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:05.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:05.594 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:05.695 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:05.720 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:05.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:05.721 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:05.821 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:06:05.842 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:05.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:05.843 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:06:05.913 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-20T19:06:05.944 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:06:05.977 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:05.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:05.978 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:06:06.078 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:06.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:06.131 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:06.131 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:06.232 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:06.253 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:06.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:06.255 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:06.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:06 smithi083 ceph-mon[29421]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:06:06.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:06 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/78215351' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T19:06:06.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:06 smithi083 ceph-mon[29421]: osdmap e415: 8 total, 8 up, 8 in 2024-09-20T19:06:06.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:06 smithi083 ceph-mon[29421]: pgmap v819: 158 pgs: 2 peering, 10 unknown, 146 active+clean; 583 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 80 op/s 2024-09-20T19:06:06.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:06 smithi083 ceph-mon[29421]: osdmap e416: 8 total, 8 up, 8 in 2024-09-20T19:06:06.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:06 smithi083 ceph-mon[36260]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:06:06.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:06 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/78215351' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T19:06:06.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:06 smithi083 ceph-mon[36260]: osdmap e415: 8 total, 8 up, 8 in 2024-09-20T19:06:06.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:06 smithi083 ceph-mon[36260]: pgmap v819: 158 pgs: 2 peering, 10 unknown, 146 active+clean; 583 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 80 op/s 2024-09-20T19:06:06.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:06 smithi083 ceph-mon[36260]: osdmap e416: 8 total, 8 up, 8 in 2024-09-20T19:06:06.355 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:06.376 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:06.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:06.377 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:06.478 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:06.499 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:06.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:06.500 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:06.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:06 smithi142 ceph-mon[32508]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:06:06.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:06 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/78215351' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T19:06:06.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:06 smithi142 ceph-mon[32508]: osdmap e415: 8 total, 8 up, 8 in 2024-09-20T19:06:06.512 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:06 smithi142 ceph-mon[32508]: pgmap v819: 158 pgs: 2 peering, 10 unknown, 146 active+clean; 583 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 80 op/s 2024-09-20T19:06:06.512 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:06 smithi142 ceph-mon[32508]: osdmap e416: 8 total, 8 up, 8 in 2024-09-20T19:06:06.601 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:06.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:06.623 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:06.623 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:06.724 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:06.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:06.746 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:06.746 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:06.747 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:06:06.784 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:06:06.847 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:06.868 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:06.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:06.869 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:06.969 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:06.994 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:06.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:06.995 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:07.095 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:07.117 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:07.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:07.118 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:07.219 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:07.240 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:07.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:07.241 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:07.268 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:07 smithi083 ceph-mon[29421]: 10.19 starting backfill to osd.1 from (0'0,0'0] MAX to 409'14740 2024-09-20T19:06:07.268 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:07 smithi083 ceph-mon[29421]: 10.10 starting backfill to osd.7 from (0'0,0'0] MAX to 409'14620 2024-09-20T19:06:07.268 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:07 smithi083 ceph-mon[29421]: 10.18 starting backfill to osd.1 from (0'0,0'0] MAX to 409'14426 2024-09-20T19:06:07.268 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:07 smithi083 ceph-mon[29421]: 10.12 starting backfill to osd.2 from (0'0,0'0] MAX to 409'14668 2024-09-20T19:06:07.268 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:07 smithi083 ceph-mon[29421]: 10.14 starting backfill to osd.1 from (0'0,0'0] MAX to 409'14380 2024-09-20T19:06:07.268 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:07 smithi083 ceph-mon[29421]: 10.16 starting backfill to osd.1 from (342'3762,373'8935] MIN to 409'14794 2024-09-20T19:06:07.268 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:07 smithi083 ceph-mon[29421]: 10.13 starting backfill to osd.1 from (0'0,0'0] MAX to 409'14876 2024-09-20T19:06:07.269 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:07 smithi083 ceph-mon[29421]: 10.11 starting backfill to osd.5 from (0'0,0'0] MAX to 409'14466 2024-09-20T19:06:07.269 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:07 smithi083 ceph-mon[29421]: 10.17 starting backfill to osd.4 from (0'0,0'0] MAX to 409'14724 2024-09-20T19:06:07.269 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:07 smithi083 ceph-mon[29421]: 10.15 starting backfill to osd.1 from (0'0,0'0] MAX to 409'14518 2024-09-20T19:06:07.269 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:07 smithi083 ceph-mon[29421]: 10.14 starting backfill to osd.2 from (0'0,0'0] MAX to 409'14380 2024-09-20T19:06:07.269 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:07 smithi083 ceph-mon[29421]: 10.19 starting backfill to osd.3 from (0'0,0'0] MAX to 409'14740 2024-09-20T19:06:07.269 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:07 smithi083 ceph-mon[29421]: 10.13 starting backfill to osd.7 from (0'0,0'0] MAX to 409'14876 2024-09-20T19:06:07.269 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:07 smithi083 ceph-mon[29421]: 10.18 starting backfill to osd.3 from (0'0,0'0] MAX to 409'14426 2024-09-20T19:06:07.269 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:07 smithi083 ceph-mon[29421]: 10.15 starting backfill to osd.5 from (0'0,0'0] MAX to 409'14518 2024-09-20T19:06:07.269 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:07 smithi083 ceph-mon[29421]: 10.16 starting backfill to osd.6 from (0'0,0'0] MAX to 409'14794 2024-09-20T19:06:07.269 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:07 smithi083 ceph-mon[29421]: 10.12 starting backfill to osd.5 from (0'0,0'0] MAX to 409'14668 2024-09-20T19:06:07.269 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:07 smithi083 ceph-mon[29421]: 10.19 starting backfill to osd.4 from (0'0,0'0] MAX to 409'14740 2024-09-20T19:06:07.269 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:07 smithi083 ceph-mon[29421]: 10.12 starting backfill to osd.6 from (0'0,0'0] MAX to 409'14668 2024-09-20T19:06:07.269 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:07 smithi083 ceph-mon[36260]: 10.19 starting backfill to osd.1 from (0'0,0'0] MAX to 409'14740 2024-09-20T19:06:07.270 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:07 smithi083 ceph-mon[36260]: 10.10 starting backfill to osd.7 from (0'0,0'0] MAX to 409'14620 2024-09-20T19:06:07.270 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:07 smithi083 ceph-mon[36260]: 10.18 starting backfill to osd.1 from (0'0,0'0] MAX to 409'14426 2024-09-20T19:06:07.270 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:07 smithi083 ceph-mon[36260]: 10.12 starting backfill to osd.2 from (0'0,0'0] MAX to 409'14668 2024-09-20T19:06:07.270 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:07 smithi083 ceph-mon[36260]: 10.14 starting backfill to osd.1 from (0'0,0'0] MAX to 409'14380 2024-09-20T19:06:07.270 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:07 smithi083 ceph-mon[36260]: 10.16 starting backfill to osd.1 from (342'3762,373'8935] MIN to 409'14794 2024-09-20T19:06:07.270 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:07 smithi083 ceph-mon[36260]: 10.13 starting backfill to osd.1 from (0'0,0'0] MAX to 409'14876 2024-09-20T19:06:07.270 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:07 smithi083 ceph-mon[36260]: 10.11 starting backfill to osd.5 from (0'0,0'0] MAX to 409'14466 2024-09-20T19:06:07.270 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:07 smithi083 ceph-mon[36260]: 10.17 starting backfill to osd.4 from (0'0,0'0] MAX to 409'14724 2024-09-20T19:06:07.270 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:07 smithi083 ceph-mon[36260]: 10.15 starting backfill to osd.1 from (0'0,0'0] MAX to 409'14518 2024-09-20T19:06:07.270 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:07 smithi083 ceph-mon[36260]: 10.14 starting backfill to osd.2 from (0'0,0'0] MAX to 409'14380 2024-09-20T19:06:07.270 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:07 smithi083 ceph-mon[36260]: 10.19 starting backfill to osd.3 from (0'0,0'0] MAX to 409'14740 2024-09-20T19:06:07.270 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:07 smithi083 ceph-mon[36260]: 10.13 starting backfill to osd.7 from (0'0,0'0] MAX to 409'14876 2024-09-20T19:06:07.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:07 smithi083 ceph-mon[36260]: 10.18 starting backfill to osd.3 from (0'0,0'0] MAX to 409'14426 2024-09-20T19:06:07.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:07 smithi083 ceph-mon[36260]: 10.15 starting backfill to osd.5 from (0'0,0'0] MAX to 409'14518 2024-09-20T19:06:07.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:07 smithi083 ceph-mon[36260]: 10.16 starting backfill to osd.6 from (0'0,0'0] MAX to 409'14794 2024-09-20T19:06:07.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:07 smithi083 ceph-mon[36260]: 10.12 starting backfill to osd.5 from (0'0,0'0] MAX to 409'14668 2024-09-20T19:06:07.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:07 smithi083 ceph-mon[36260]: 10.19 starting backfill to osd.4 from (0'0,0'0] MAX to 409'14740 2024-09-20T19:06:07.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:07 smithi083 ceph-mon[36260]: 10.12 starting backfill to osd.6 from (0'0,0'0] MAX to 409'14668 2024-09-20T19:06:07.342 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:07.363 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:07.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:07.364 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:07.393 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:07 smithi142 ceph-mon[32508]: 10.19 starting backfill to osd.1 from (0'0,0'0] MAX to 409'14740 2024-09-20T19:06:07.393 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:07 smithi142 ceph-mon[32508]: 10.10 starting backfill to osd.7 from (0'0,0'0] MAX to 409'14620 2024-09-20T19:06:07.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:07 smithi142 ceph-mon[32508]: 10.18 starting backfill to osd.1 from (0'0,0'0] MAX to 409'14426 2024-09-20T19:06:07.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:07 smithi142 ceph-mon[32508]: 10.12 starting backfill to osd.2 from (0'0,0'0] MAX to 409'14668 2024-09-20T19:06:07.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:07 smithi142 ceph-mon[32508]: 10.14 starting backfill to osd.1 from (0'0,0'0] MAX to 409'14380 2024-09-20T19:06:07.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:07 smithi142 ceph-mon[32508]: 10.16 starting backfill to osd.1 from (342'3762,373'8935] MIN to 409'14794 2024-09-20T19:06:07.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:07 smithi142 ceph-mon[32508]: 10.13 starting backfill to osd.1 from (0'0,0'0] MAX to 409'14876 2024-09-20T19:06:07.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:07 smithi142 ceph-mon[32508]: 10.11 starting backfill to osd.5 from (0'0,0'0] MAX to 409'14466 2024-09-20T19:06:07.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:07 smithi142 ceph-mon[32508]: 10.17 starting backfill to osd.4 from (0'0,0'0] MAX to 409'14724 2024-09-20T19:06:07.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:07 smithi142 ceph-mon[32508]: 10.15 starting backfill to osd.1 from (0'0,0'0] MAX to 409'14518 2024-09-20T19:06:07.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:07 smithi142 ceph-mon[32508]: 10.14 starting backfill to osd.2 from (0'0,0'0] MAX to 409'14380 2024-09-20T19:06:07.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:07 smithi142 ceph-mon[32508]: 10.19 starting backfill to osd.3 from (0'0,0'0] MAX to 409'14740 2024-09-20T19:06:07.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:07 smithi142 ceph-mon[32508]: 10.13 starting backfill to osd.7 from (0'0,0'0] MAX to 409'14876 2024-09-20T19:06:07.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:07 smithi142 ceph-mon[32508]: 10.18 starting backfill to osd.3 from (0'0,0'0] MAX to 409'14426 2024-09-20T19:06:07.394 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:07 smithi142 ceph-mon[32508]: 10.15 starting backfill to osd.5 from (0'0,0'0] MAX to 409'14518 2024-09-20T19:06:07.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:07 smithi142 ceph-mon[32508]: 10.16 starting backfill to osd.6 from (0'0,0'0] MAX to 409'14794 2024-09-20T19:06:07.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:07 smithi142 ceph-mon[32508]: 10.12 starting backfill to osd.5 from (0'0,0'0] MAX to 409'14668 2024-09-20T19:06:07.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:07 smithi142 ceph-mon[32508]: 10.19 starting backfill to osd.4 from (0'0,0'0] MAX to 409'14740 2024-09-20T19:06:07.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:07 smithi142 ceph-mon[32508]: 10.12 starting backfill to osd.6 from (0'0,0'0] MAX to 409'14668 2024-09-20T19:06:07.465 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:07.486 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:07.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:07.487 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:07.588 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:07.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:07.610 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:07.610 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:07.625 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T19:06:07.625 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:06:07.711 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:07.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:07.735 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:07.735 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:07.761 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:06:07 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:06:07.836 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:07.856 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:07.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:07.857 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:07.958 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:07.978 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:07.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:07.979 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:08.080 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:08.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:08.106 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:08.106 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:08.207 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:08.231 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:08.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:08.232 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:08.332 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:08.357 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:08.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:08.358 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:08.458 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:08.481 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:08.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:08.482 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:08.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:08 smithi142 ceph-mon[32508]: osdmap e417: 8 total, 8 up, 8 in 2024-09-20T19:06:08.512 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:08 smithi142 ceph-mon[32508]: pgmap v822: 158 pgs: 2 peering, 3 remapped+peering, 4 unknown, 3 active+remapped+backfill_toofull, 146 active+clean; 583 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T19:06:08.512 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:08 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:06:08.512 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:08 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3537378875' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:06:08.542 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:08 smithi083 ceph-mon[36260]: osdmap e417: 8 total, 8 up, 8 in 2024-09-20T19:06:08.542 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:08 smithi083 ceph-mon[36260]: pgmap v822: 158 pgs: 2 peering, 3 remapped+peering, 4 unknown, 3 active+remapped+backfill_toofull, 146 active+clean; 583 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T19:06:08.542 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:08 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:06:08.542 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:08 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3537378875' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:06:08.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:08 smithi083 ceph-mon[29421]: osdmap e417: 8 total, 8 up, 8 in 2024-09-20T19:06:08.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:08 smithi083 ceph-mon[29421]: pgmap v822: 158 pgs: 2 peering, 3 remapped+peering, 4 unknown, 3 active+remapped+backfill_toofull, 146 active+clean; 583 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T19:06:08.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:08 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:06:08.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:08 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3537378875' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:06:08.583 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:08.608 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:08.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:08.609 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:08.710 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:08.736 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:08.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:08.737 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:08.773 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T19:06:08.775 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 7, 6, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 5, 6, 7, 4] 2024-09-20T19:06:08.775 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T19:06:08.775 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T19:06:08.838 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:08.862 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:08.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:08.863 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:08.909 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:06:08.909 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:06:08.909 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:06:08.910 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:06:08.910 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:06:08.910 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:06:08.910 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:06:08.910 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:06:08.910 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:06:08.910 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:06:08.910 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:06:08.910 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:06:08.911 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:06:08.911 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:06:08.911 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:06:08.911 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:06:08.964 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:08.987 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:08.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:08.988 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:09.089 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:09.113 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:09.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:09.114 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:09.214 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:06:09.236 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:09.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:09.237 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:06:09.338 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:09.361 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:09.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:09.362 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:09.397 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:09 smithi083 ceph-mon[29421]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T19:06:09.397 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:09 smithi083 ceph-mon[29421]: osdmap e418: 8 total, 8 up, 8 in 2024-09-20T19:06:09.397 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:09 smithi083 ceph-mon[36260]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T19:06:09.397 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:09 smithi083 ceph-mon[36260]: osdmap e418: 8 total, 8 up, 8 in 2024-09-20T19:06:09.463 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:09.485 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:09.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:09.486 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:09.512 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:09 smithi142 ceph-mon[32508]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T19:06:09.512 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:09 smithi142 ceph-mon[32508]: osdmap e418: 8 total, 8 up, 8 in 2024-09-20T19:06:09.586 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:09.612 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:09.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:09.612 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:09.713 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:09.735 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:09.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:09.736 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:09.837 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:09.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:09.864 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:09.865 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:09.965 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:06:09.987 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:09.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:09.988 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:06:10.089 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:10.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:10.117 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:10.117 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:10.218 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:10.243 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:10.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:10.244 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:10.345 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:10.351 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:10 smithi083 ceph-mon[29421]: pgmap v824: 158 pgs: 3 remapped+peering, 1 unknown, 6 active+remapped+backfill_toofull, 148 active+clean; 583 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T19:06:10.351 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:10 smithi083 ceph-mon[36260]: pgmap v824: 158 pgs: 3 remapped+peering, 1 unknown, 6 active+remapped+backfill_toofull, 148 active+clean; 583 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T19:06:10.370 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:10.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:10.371 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:10.374 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:06:10.387 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_blocked_ops 2024-09-20T19:06:10.413 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:06:10.471 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:10.496 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:10.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:10.497 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:10.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:10 smithi142 ceph-mon[32508]: pgmap v824: 158 pgs: 3 remapped+peering, 1 unknown, 6 active+remapped+backfill_toofull, 148 active+clean; 583 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T19:06:10.599 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:10.648 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:10.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:10.649 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:10.749 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:10.775 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:10.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:10.776 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:10.877 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:10.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:10.906 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:10.906 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:10.930 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:06:10 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:06:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:06:10.965 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:06:11.007 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:11.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:11.029 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:11.029 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:11.130 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:11.155 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:11.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:11.156 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:11.257 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:11.282 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:11.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:11.283 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:11.383 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:11.409 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:11.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:11.410 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:11.511 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:06:11.533 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:11.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:11.534 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:06:11.635 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:11.661 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:11.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:11.662 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:11.763 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:11.787 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:11.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:11.788 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:11.802 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T19:06:11.879 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T19:06:11.883 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-20T19:06:11.889 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:06:11.914 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'unique_pool_3' removed 2024-09-20T19:06:11.915 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:11.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:11.916 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:06:12.017 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:12.042 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:12.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:12.043 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:12.144 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:12.168 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:12.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:12.169 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:12.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:11 smithi142 ceph-mon[32508]: osdmap e419: 8 total, 8 up, 8 in 2024-09-20T19:06:12.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:11 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/156640843' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true} : dispatch 2024-09-20T19:06:12.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:11 smithi142 ceph-mon[32508]: pgmap v826: 158 pgs: 1 unknown, 7 active+remapped+backfill_toofull, 150 active+clean; 583 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1003 B/s rd, 0 op/s; 35 MiB/s, 402 objects/s recovering 2024-09-20T19:06:12.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:11 smithi142 ceph-mon[32508]: from='client.26213 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:06:12.269 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:12.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:11 smithi083 ceph-mon[29421]: osdmap e419: 8 total, 8 up, 8 in 2024-09-20T19:06:12.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:11 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/156640843' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true} : dispatch 2024-09-20T19:06:12.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:11 smithi083 ceph-mon[29421]: pgmap v826: 158 pgs: 1 unknown, 7 active+remapped+backfill_toofull, 150 active+clean; 583 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1003 B/s rd, 0 op/s; 35 MiB/s, 402 objects/s recovering 2024-09-20T19:06:12.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:11 smithi083 ceph-mon[29421]: from='client.26213 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:06:12.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:11 smithi083 ceph-mon[36260]: osdmap e419: 8 total, 8 up, 8 in 2024-09-20T19:06:12.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:11 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/156640843' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true} : dispatch 2024-09-20T19:06:12.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:11 smithi083 ceph-mon[36260]: pgmap v826: 158 pgs: 1 unknown, 7 active+remapped+backfill_toofull, 150 active+clean; 583 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1003 B/s rd, 0 op/s; 35 MiB/s, 402 objects/s recovering 2024-09-20T19:06:12.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:11 smithi083 ceph-mon[36260]: from='client.26213 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:06:12.293 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:12.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:12.294 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:12.395 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:12.417 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:12.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:12.418 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:12.519 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:12.540 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:12.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:12.541 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:12.593 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-20T19:06:12.594 INFO:tasks.radosbench:Beginning radosbench... 2024-09-20T19:06:12.595 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_4 2024-09-20T19:06:12.595 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd pool create unique_pool_4 16 2024-09-20T19:06:12.641 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:12.662 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:12.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:12.663 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:12.764 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:06:12.788 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:12.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:12.789 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:06:12.889 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:12.911 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:12.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:12.912 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:12.929 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-20T19:06:13.013 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:06:13.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:13.035 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:13.035 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:06:13.136 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:13.157 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:13.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:13.158 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:13.259 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:13.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:12 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/156640843' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]': finished 2024-09-20T19:06:13.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:12 smithi142 ceph-mon[32508]: osdmap e420: 8 total, 8 up, 8 in 2024-09-20T19:06:13.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:12 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1551374573' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:06:13.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:12 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:06:13.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:12 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T19:06:13.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:12 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:06:13.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:12 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:06:13.280 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:13.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:13.281 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:13.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:12 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/156640843' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]': finished 2024-09-20T19:06:13.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:12 smithi083 ceph-mon[36260]: osdmap e420: 8 total, 8 up, 8 in 2024-09-20T19:06:13.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:12 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1551374573' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:06:13.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:12 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:06:13.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:12 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T19:06:13.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:12 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:06:13.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:12 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:06:13.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:12 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/156640843' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]': finished 2024-09-20T19:06:13.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:12 smithi083 ceph-mon[29421]: osdmap e420: 8 total, 8 up, 8 in 2024-09-20T19:06:13.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:12 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1551374573' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:06:13.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:12 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:06:13.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:12 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T19:06:13.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:12 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:06:13.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:12 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:06:13.339 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":826,"stamp":"2024-09-20T19:06:11.159680+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":2231,"num_read_kb":2088,"num_write":227158,"num_write_kb":7243946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30399,"num_bytes_recovered":1714464243,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143364,"ondisk_log_size":143364,"up":471,"acting":471,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":486,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":4615248,"kb_used_data":68376,"kb_used_omap":16,"kb_used_meta":4546671,"kb_avail":745345968,"statfs":{"total":767960285184,"available":763234271232,"internally_reserved":0,"allocated":70017024,"data_stored":65723374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16790,"internal_metadata":4655791722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":-7328,"num_write_kb":-234496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2051,"num_bytes_recovered":185171968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.100577"},"pg_stats":[{"pgid":"10.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-20T19:06:02.036094+0000","last_change":"2024-09-20T19:06:02.036094+0000","last_active":"2024-09-20T19:06:02.036094+0000","last_peered":"2024-09-20T19:06:02.036094+0000","last_clean":"2024-09-20T19:06:02.036094+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-20T19:06:02.036094+0000","last_undegraded":"2024-09-20T19:06:02.036094+0000","last_fullsized":"2024-09-20T19:06:02.036094+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:02.036094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:02.036094+0000","last_clean_scrub_stamp":"2024-09-20T19:06:02.036094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":765,"reported_epoch":417,"state":"active+clean","last_fresh":"2024-09-20T19:06:07.087710+0000","last_change":"2024-09-20T19:01:31.817967+0000","last_active":"2024-09-20T19:06:07.087710+0000","last_peered":"2024-09-20T19:06:07.087710+0000","last_clean":"2024-09-20T19:06:07.087710+0000","last_became_active":"2024-09-20T18:49:45.943201+0000","last_became_peered":"2024-09-20T18:49:45.943201+0000","last_unstale":"2024-09-20T19:06:07.087710+0000","last_undegraded":"2024-09-20T19:06:07.087710+0000","last_fullsized":"2024-09-20T19:06:07.087710+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:31.817909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:31.817909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:00.513933+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"273'15","reported_seq":107,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.090445+0000","last_change":"2024-09-20T19:04:21.040978+0000","last_active":"2024-09-20T19:06:08.090445+0000","last_peered":"2024-09-20T19:06:08.090445+0000","last_clean":"2024-09-20T19:06:08.090445+0000","last_became_active":"2024-09-20T19:04:20.925853+0000","last_became_peered":"2024-09-20T19:04:20.925853+0000","last_unstale":"2024-09-20T19:06:08.090445+0000","last_undegraded":"2024-09-20T19:06:08.090445+0000","last_fullsized":"2024-09-20T19:06:08.090445+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:03:10.214154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:10.214154+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":753,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.071813+0000","last_change":"2024-09-20T18:57:49.016078+0000","last_active":"2024-09-20T19:06:06.071813+0000","last_peered":"2024-09-20T19:06:06.071813+0000","last_clean":"2024-09-20T19:06:06.071813+0000","last_became_active":"2024-09-20T18:49:49.959514+0000","last_became_peered":"2024-09-20T18:49:49.959514+0000","last_unstale":"2024-09-20T19:06:06.071813+0000","last_undegraded":"2024-09-20T19:06:06.071813+0000","last_fullsized":"2024-09-20T19:06:06.071813+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:49.016006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:49.016006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:25.184023+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":97,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.091207+0000","last_change":"2024-09-20T19:04:20.881918+0000","last_active":"2024-09-20T19:06:08.091207+0000","last_peered":"2024-09-20T19:06:08.091207+0000","last_clean":"2024-09-20T19:06:08.091207+0000","last_became_active":"2024-09-20T19:04:20.880905+0000","last_became_peered":"2024-09-20T19:04:20.880905+0000","last_unstale":"2024-09-20T19:06:08.091207+0000","last_undegraded":"2024-09-20T19:06:08.091207+0000","last_fullsized":"2024-09-20T19:06:08.091207+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:02.084375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:02.084375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:37.200857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.17","version":"409'14724","reported_seq":14928,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.112458+0000","last_change":"2024-09-20T19:06:08.111940+0000","last_active":"2024-09-20T19:06:08.112458+0000","last_peered":"2024-09-20T19:06:08.112458+0000","last_clean":"2024-09-20T19:06:08.112458+0000","last_became_active":"2024-09-20T19:06:08.094909+0000","last_became_peered":"2024-09-20T19:06:08.094909+0000","last_unstale":"2024-09-20T19:06:08.112458+0000","last_undegraded":"2024-09-20T19:06:08.112458+0000","last_fullsized":"2024-09-20T19:06:08.112458+0000","mapping_epoch":417,"log_start":"372'8707","ondisk_log_start":"372'8707","created":412,"last_epoch_clean":418,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":6017,"log_dups_size":0,"ondisk_log_size":6017,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-20T19:07:09.019075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7362,"num_write_kb":235584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":767,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.092736+0000","last_change":"2024-09-20T19:02:02.441633+0000","last_active":"2024-09-20T19:06:08.092736+0000","last_peered":"2024-09-20T19:06:08.092736+0000","last_clean":"2024-09-20T19:06:08.092736+0000","last_became_active":"2024-09-20T18:59:05.081081+0000","last_became_peered":"2024-09-20T18:59:05.081081+0000","last_unstale":"2024-09-20T19:06:08.092736+0000","last_undegraded":"2024-09-20T19:06:08.092736+0000","last_fullsized":"2024-09-20T19:06:08.092736+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:02.441577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:02.441577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":781,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.092253+0000","last_change":"2024-09-20T19:04:37.540824+0000","last_active":"2024-09-20T19:06:08.092253+0000","last_peered":"2024-09-20T19:06:08.092253+0000","last_clean":"2024-09-20T19:06:08.092253+0000","last_became_active":"2024-09-20T18:59:05.072675+0000","last_became_peered":"2024-09-20T18:59:05.072675+0000","last_unstale":"2024-09-20T19:06:08.092253+0000","last_undegraded":"2024-09-20T19:06:08.092253+0000","last_fullsized":"2024-09-20T19:06:08.092253+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:37.540760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:04:37.540760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"274'27","reported_seq":812,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.092751+0000","last_change":"2024-09-20T19:05:50.750134+0000","last_active":"2024-09-20T19:06:08.092751+0000","last_peered":"2024-09-20T19:06:08.092751+0000","last_clean":"2024-09-20T19:06:08.092751+0000","last_became_active":"2024-09-20T18:49:47.952150+0000","last_became_peered":"2024-09-20T18:49:47.952150+0000","last_unstale":"2024-09-20T19:06:08.092751+0000","last_undegraded":"2024-09-20T19:06:08.092751+0000","last_fullsized":"2024-09-20T19:06:08.092751+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'27","last_scrub_stamp":"2024-09-20T19:05:50.750050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:50.750050+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:13.513790+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":97,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.090501+0000","last_change":"2024-09-20T19:04:20.926234+0000","last_active":"2024-09-20T19:06:08.090501+0000","last_peered":"2024-09-20T19:06:08.090501+0000","last_clean":"2024-09-20T19:06:08.090501+0000","last_became_active":"2024-09-20T19:04:20.925394+0000","last_became_peered":"2024-09-20T19:04:20.925394+0000","last_unstale":"2024-09-20T19:06:08.090501+0000","last_undegraded":"2024-09-20T19:06:08.090501+0000","last_fullsized":"2024-09-20T19:06:08.090501+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:23.982813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:23.982813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.14","version":"409'14380","reported_seq":14613,"reported_epoch":417,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-20T19:06:07.086368+0000","last_change":"2024-09-20T19:06:06.220714+0000","last_active":"2024-09-20T19:06:07.086368+0000","last_peered":"2024-09-20T19:06:07.086368+0000","last_clean":"2024-09-20T19:06:03.123348+0000","last_became_active":"2024-09-20T19:06:06.186278+0000","last_became_peered":"2024-09-20T19:06:06.186278+0000","last_unstale":"2024-09-20T19:06:07.086368+0000","last_undegraded":"2024-09-20T19:06:07.086368+0000","last_fullsized":"2024-09-20T19:06:07.086368+0000","mapping_epoch":415,"log_start":"376'8997","ondisk_log_start":"376'8997","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":5383,"log_dups_size":0,"ondisk_log_size":5383,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7190,"num_write_kb":230080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,4],"acting":[0,4,3],"avail_no_missing":["0","1","2","3","4"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"59'1","reported_seq":842,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.094662+0000","last_change":"2024-09-20T19:04:20.963496+0000","last_active":"2024-09-20T19:06:08.094662+0000","last_peered":"2024-09-20T19:06:08.094662+0000","last_clean":"2024-09-20T19:06:08.094662+0000","last_became_active":"2024-09-20T19:04:20.914145+0000","last_became_peered":"2024-09-20T19:04:20.914145+0000","last_unstale":"2024-09-20T19:06:08.094662+0000","last_undegraded":"2024-09-20T19:06:08.094662+0000","last_fullsized":"2024-09-20T19:06:08.094662+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:01:28.592653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:28.592653+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:29.742562+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"273'15","reported_seq":107,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.091207+0000","last_change":"2024-09-20T19:04:20.984223+0000","last_active":"2024-09-20T19:06:08.091207+0000","last_peered":"2024-09-20T19:06:08.091207+0000","last_clean":"2024-09-20T19:06:08.091207+0000","last_became_active":"2024-09-20T19:04:20.898235+0000","last_became_peered":"2024-09-20T19:04:20.898235+0000","last_unstale":"2024-09-20T19:06:08.091207+0000","last_undegraded":"2024-09-20T19:06:08.091207+0000","last_fullsized":"2024-09-20T19:06:08.091207+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:02:42.114331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:42.114331+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":765,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.094453+0000","last_change":"2024-09-20T19:01:51.564693+0000","last_active":"2024-09-20T19:06:08.094453+0000","last_peered":"2024-09-20T19:06:08.094453+0000","last_clean":"2024-09-20T19:06:08.094453+0000","last_became_active":"2024-09-20T18:49:49.961188+0000","last_became_peered":"2024-09-20T18:49:49.961188+0000","last_unstale":"2024-09-20T19:06:08.094453+0000","last_undegraded":"2024-09-20T19:06:08.094453+0000","last_fullsized":"2024-09-20T19:06:08.094453+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:51.564624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:51.564624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:41.667335+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":744,"reported_epoch":417,"state":"active+clean","last_fresh":"2024-09-20T19:06:07.086647+0000","last_change":"2024-09-20T19:03:09.963421+0000","last_active":"2024-09-20T19:06:07.086647+0000","last_peered":"2024-09-20T19:06:07.086647+0000","last_clean":"2024-09-20T19:06:07.086647+0000","last_became_active":"2024-09-20T18:49:51.966715+0000","last_became_peered":"2024-09-20T18:49:51.966715+0000","last_unstale":"2024-09-20T19:06:07.086647+0000","last_undegraded":"2024-09-20T19:06:07.086647+0000","last_fullsized":"2024-09-20T19:06:07.086647+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:09.963298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:03:09.963298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.15","version":"409'14518","reported_seq":14806,"reported_epoch":418,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-20T19:06:08.093273+0000","last_change":"2024-09-20T19:06:06.228529+0000","last_active":"2024-09-20T19:06:08.093273+0000","last_peered":"2024-09-20T19:06:08.093273+0000","last_clean":"2024-09-20T19:05:57.287324+0000","last_became_active":"2024-09-20T19:06:06.191359+0000","last_became_peered":"2024-09-20T19:06:06.191359+0000","last_unstale":"2024-09-20T19:06:08.093273+0000","last_undegraded":"2024-09-20T19:06:08.093273+0000","last_fullsized":"2024-09-20T19:06:08.093273+0000","mapping_epoch":415,"log_start":"369'7968","ondisk_log_start":"369'7968","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":5,"last_scrub":"409'14097","last_scrub_stamp":"2024-09-20T19:05:51.722107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:05:51.722107+0000","objects_scrubbed":430,"log_size":6550,"log_dups_size":0,"ondisk_log_size":6550,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":273,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7259,"num_write_kb":232288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,0,2],"avail_no_missing":["3","0","1","2","5"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":790,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.093954+0000","last_change":"2024-09-20T19:04:20.879227+0000","last_active":"2024-09-20T19:06:08.093954+0000","last_peered":"2024-09-20T19:06:08.093954+0000","last_clean":"2024-09-20T19:06:08.093954+0000","last_became_active":"2024-09-20T19:04:20.878905+0000","last_became_peered":"2024-09-20T19:04:20.878905+0000","last_unstale":"2024-09-20T19:06:08.093954+0000","last_undegraded":"2024-09-20T19:06:08.093954+0000","last_fullsized":"2024-09-20T19:06:08.093954+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:48.520017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:48.520017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:42.653960+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"273'7","reported_seq":111,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.090367+0000","last_change":"2024-09-20T19:05:53.980848+0000","last_active":"2024-09-20T19:06:08.090367+0000","last_peered":"2024-09-20T19:06:08.090367+0000","last_clean":"2024-09-20T19:06:08.090367+0000","last_became_active":"2024-09-20T19:04:20.888572+0000","last_became_peered":"2024-09-20T19:04:20.888572+0000","last_unstale":"2024-09-20T19:06:08.090367+0000","last_undegraded":"2024-09-20T19:06:08.090367+0000","last_fullsized":"2024-09-20T19:06:08.090367+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'7","last_scrub_stamp":"2024-09-20T19:05:53.980772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:53.980772+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:54.857238+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":779,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.097628+0000","last_change":"2024-09-20T19:04:20.885873+0000","last_active":"2024-09-20T19:06:06.097628+0000","last_peered":"2024-09-20T19:06:06.097628+0000","last_clean":"2024-09-20T19:06:06.097628+0000","last_became_active":"2024-09-20T19:04:20.885417+0000","last_became_peered":"2024-09-20T19:04:20.885417+0000","last_unstale":"2024-09-20T19:06:06.097628+0000","last_undegraded":"2024-09-20T19:06:06.097628+0000","last_fullsized":"2024-09-20T19:06:06.097628+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:33.203312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:33.203312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:05.802150+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":761,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.093818+0000","last_change":"2024-09-20T19:05:21.625378+0000","last_active":"2024-09-20T19:06:08.093818+0000","last_peered":"2024-09-20T19:06:08.093818+0000","last_clean":"2024-09-20T19:06:08.093818+0000","last_became_active":"2024-09-20T18:49:51.970663+0000","last_became_peered":"2024-09-20T18:49:51.970663+0000","last_unstale":"2024-09-20T19:06:08.093818+0000","last_undegraded":"2024-09-20T19:06:08.093818+0000","last_fullsized":"2024-09-20T19:06:08.093818+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:21.625311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:05:21.625311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:25.835298+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"409'14668","reported_seq":16212,"reported_epoch":417,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-20T19:06:07.086525+0000","last_change":"2024-09-20T19:06:06.218306+0000","last_active":"2024-09-20T19:06:07.086525+0000","last_peered":"2024-09-20T19:06:07.086525+0000","last_clean":"2024-09-20T19:05:57.283695+0000","last_became_active":"2024-09-20T19:06:06.185045+0000","last_became_peered":"2024-09-20T19:06:06.185045+0000","last_unstale":"2024-09-20T19:06:07.086525+0000","last_undegraded":"2024-09-20T19:06:07.086525+0000","last_fullsized":"2024-09-20T19:06:07.086525+0000","mapping_epoch":415,"log_start":"377'9482","ondisk_log_start":"377'9482","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":5186,"log_dups_size":0,"ondisk_log_size":5186,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7334,"num_write_kb":234688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":333,"num_bytes_recovered":21790720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,5],"acting":[0,1,4],"avail_no_missing":["0","1","2","4","5","6"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":97,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.092307+0000","last_change":"2024-09-20T19:04:20.907531+0000","last_active":"2024-09-20T19:06:08.092307+0000","last_peered":"2024-09-20T19:06:08.092307+0000","last_clean":"2024-09-20T19:06:08.092307+0000","last_became_active":"2024-09-20T19:04:20.906109+0000","last_became_peered":"2024-09-20T19:04:20.906109+0000","last_unstale":"2024-09-20T19:06:08.092307+0000","last_undegraded":"2024-09-20T19:06:08.092307+0000","last_fullsized":"2024-09-20T19:06:08.092307+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:00.906652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:00.906652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:39.140988+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":789,"reported_epoch":417,"state":"active+clean","last_fresh":"2024-09-20T19:06:07.087103+0000","last_change":"2024-09-20T19:04:20.907057+0000","last_active":"2024-09-20T19:06:07.087103+0000","last_peered":"2024-09-20T19:06:07.087103+0000","last_clean":"2024-09-20T19:06:07.087103+0000","last_became_active":"2024-09-20T19:04:20.906691+0000","last_became_peered":"2024-09-20T19:04:20.906691+0000","last_unstale":"2024-09-20T19:06:07.087103+0000","last_undegraded":"2024-09-20T19:06:07.087103+0000","last_fullsized":"2024-09-20T19:06:07.087103+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:44.998147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:44.998147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"273'27","reported_seq":819,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.061467+0000","last_change":"2024-09-20T19:05:49.452928+0000","last_active":"2024-09-20T19:06:06.061467+0000","last_peered":"2024-09-20T19:06:06.061467+0000","last_clean":"2024-09-20T19:06:06.061467+0000","last_became_active":"2024-09-20T18:49:47.950939+0000","last_became_peered":"2024-09-20T18:49:47.950939+0000","last_unstale":"2024-09-20T19:06:06.061467+0000","last_undegraded":"2024-09-20T19:06:06.061467+0000","last_fullsized":"2024-09-20T19:06:06.061467+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'27","last_scrub_stamp":"2024-09-20T19:05:49.452861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:49.452861+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:02.753272+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":770,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.070528+0000","last_change":"2024-09-20T19:04:20.890659+0000","last_active":"2024-09-20T19:06:06.070528+0000","last_peered":"2024-09-20T19:06:06.070528+0000","last_clean":"2024-09-20T19:06:06.070528+0000","last_became_active":"2024-09-20T19:04:20.890370+0000","last_became_peered":"2024-09-20T19:04:20.890370+0000","last_unstale":"2024-09-20T19:06:06.070528+0000","last_undegraded":"2024-09-20T19:06:06.070528+0000","last_fullsized":"2024-09-20T19:06:06.070528+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:05.888974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:05.888974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:26.204424+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.13","version":"409'14876","reported_seq":15200,"reported_epoch":416,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-20T19:06:06.226647+0000","last_change":"2024-09-20T19:06:06.226532+0000","last_active":"2024-09-20T19:06:06.226647+0000","last_peered":"2024-09-20T19:06:06.226647+0000","last_clean":"2024-09-20T19:06:03.130898+0000","last_became_active":"2024-09-20T19:06:06.187273+0000","last_became_peered":"2024-09-20T19:06:06.187273+0000","last_unstale":"2024-09-20T19:06:06.226647+0000","last_undegraded":"2024-09-20T19:06:06.226647+0000","last_fullsized":"2024-09-20T19:06:06.226647+0000","mapping_epoch":415,"log_start":"369'8328","ondisk_log_start":"369'8328","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":5,"last_scrub":"392'12286","last_scrub_stamp":"2024-09-20T19:05:17.450066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:05:17.450066+0000","objects_scrubbed":2624,"log_size":6548,"log_dups_size":0,"ondisk_log_size":6548,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7438,"num_write_kb":238016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[2,3,5],"avail_no_missing":["2","1","3","5","7"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":768,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.092393+0000","last_change":"2024-09-20T19:02:28.934597+0000","last_active":"2024-09-20T19:06:08.092393+0000","last_peered":"2024-09-20T19:06:08.092393+0000","last_clean":"2024-09-20T19:06:08.092393+0000","last_became_active":"2024-09-20T18:59:05.084496+0000","last_became_peered":"2024-09-20T18:59:05.084496+0000","last_unstale":"2024-09-20T19:06:08.092393+0000","last_undegraded":"2024-09-20T19:06:08.092393+0000","last_fullsized":"2024-09-20T19:06:08.092393+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:28.934264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:28.934264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"59'1","reported_seq":101,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.091006+0000","last_change":"2024-09-20T19:04:20.905233+0000","last_active":"2024-09-20T19:06:08.091006+0000","last_peered":"2024-09-20T19:06:08.091006+0000","last_clean":"2024-09-20T19:06:08.091006+0000","last_became_active":"2024-09-20T19:04:20.887062+0000","last_became_peered":"2024-09-20T19:04:20.887062+0000","last_unstale":"2024-09-20T19:06:08.091006+0000","last_undegraded":"2024-09-20T19:06:08.091006+0000","last_fullsized":"2024-09-20T19:06:08.091006+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:03:08.218812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:03:08.218812+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":993,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"274'18","reported_seq":818,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.093482+0000","last_change":"2024-09-20T19:04:20.925690+0000","last_active":"2024-09-20T19:06:08.093482+0000","last_peered":"2024-09-20T19:06:08.093482+0000","last_clean":"2024-09-20T19:06:08.093482+0000","last_became_active":"2024-09-20T19:04:20.896343+0000","last_became_peered":"2024-09-20T19:04:20.896343+0000","last_unstale":"2024-09-20T19:06:08.093482+0000","last_undegraded":"2024-09-20T19:06:08.093482+0000","last_fullsized":"2024-09-20T19:06:08.093482+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:57:09.818999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:09.818999+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:29.636168+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":97,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.092389+0000","last_change":"2024-09-20T19:04:20.889786+0000","last_active":"2024-09-20T19:06:08.092389+0000","last_peered":"2024-09-20T19:06:08.092389+0000","last_clean":"2024-09-20T19:06:08.092389+0000","last_became_active":"2024-09-20T19:04:20.889430+0000","last_became_peered":"2024-09-20T19:04:20.889430+0000","last_unstale":"2024-09-20T19:06:08.092389+0000","last_undegraded":"2024-09-20T19:06:08.092389+0000","last_fullsized":"2024-09-20T19:06:08.092389+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:02.094473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:02.094473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.10","version":"409'14620","reported_seq":24069,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.114426+0000","last_change":"2024-09-20T19:06:08.114361+0000","last_active":"2024-09-20T19:06:08.114426+0000","last_peered":"2024-09-20T19:06:08.114426+0000","last_clean":"2024-09-20T19:06:08.114426+0000","last_became_active":"2024-09-20T19:06:08.095755+0000","last_became_peered":"2024-09-20T19:06:08.095755+0000","last_unstale":"2024-09-20T19:06:08.114426+0000","last_undegraded":"2024-09-20T19:06:08.114426+0000","last_fullsized":"2024-09-20T19:06:08.114426+0000","mapping_epoch":417,"log_start":"372'8539","ondisk_log_start":"372'8539","created":412,"last_epoch_clean":418,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":6081,"log_dups_size":0,"ondisk_log_size":6081,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7310,"num_write_kb":233920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2307,"num_bytes_recovered":100368384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":751,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.098144+0000","last_change":"2024-09-20T19:02:28.294390+0000","last_active":"2024-09-20T19:06:06.098144+0000","last_peered":"2024-09-20T19:06:06.098144+0000","last_clean":"2024-09-20T19:06:06.098144+0000","last_became_active":"2024-09-20T18:49:51.970058+0000","last_became_peered":"2024-09-20T18:49:51.970058+0000","last_unstale":"2024-09-20T19:06:06.098144+0000","last_undegraded":"2024-09-20T19:06:06.098144+0000","last_fullsized":"2024-09-20T19:06:06.098144+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T19:02:28.294292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:28.294292+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:58.767339+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":787,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.072001+0000","last_change":"2024-09-20T19:05:22.202134+0000","last_active":"2024-09-20T19:06:06.072001+0000","last_peered":"2024-09-20T19:06:06.072001+0000","last_clean":"2024-09-20T19:06:06.072001+0000","last_became_active":"2024-09-20T19:04:20.881466+0000","last_became_peered":"2024-09-20T19:04:20.881466+0000","last_unstale":"2024-09-20T19:06:06.072001+0000","last_undegraded":"2024-09-20T19:06:06.072001+0000","last_fullsized":"2024-09-20T19:06:06.072001+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:05:22.201836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:22.201836+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:45.062354+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"274'20","reported_seq":827,"reported_epoch":417,"state":"active+clean","last_fresh":"2024-09-20T19:06:07.087438+0000","last_change":"2024-09-20T19:03:16.982915+0000","last_active":"2024-09-20T19:06:07.087438+0000","last_peered":"2024-09-20T19:06:07.087438+0000","last_clean":"2024-09-20T19:06:07.087438+0000","last_became_active":"2024-09-20T19:03:16.968636+0000","last_became_peered":"2024-09-20T19:03:16.968636+0000","last_unstale":"2024-09-20T19:06:07.087438+0000","last_undegraded":"2024-09-20T19:06:07.087438+0000","last_fullsized":"2024-09-20T19:06:07.087438+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'20","last_scrub_stamp":"2024-09-20T19:02:35.096979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:35.096979+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"375'14","reported_seq":465,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.070465+0000","last_change":"2024-09-20T19:04:20.956210+0000","last_active":"2024-09-20T19:06:06.070465+0000","last_peered":"2024-09-20T19:06:06.070465+0000","last_clean":"2024-09-20T19:06:06.070465+0000","last_became_active":"2024-09-20T19:04:20.900782+0000","last_became_peered":"2024-09-20T19:04:20.900782+0000","last_unstale":"2024-09-20T19:06:06.070465+0000","last_undegraded":"2024-09-20T19:06:06.070465+0000","last_fullsized":"2024-09-20T19:06:06.070465+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"308'12","last_scrub_stamp":"2024-09-20T19:01:44.541156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:44.541156+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:35.235276+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"409'14466","reported_seq":32141,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.090679+0000","last_change":"2024-09-20T19:06:03.083017+0000","last_active":"2024-09-20T19:06:08.090679+0000","last_peered":"2024-09-20T19:06:08.090679+0000","last_clean":"2024-09-20T19:06:08.090679+0000","last_became_active":"2024-09-20T19:06:03.060869+0000","last_became_peered":"2024-09-20T19:06:03.060869+0000","last_unstale":"2024-09-20T19:06:08.090679+0000","last_undegraded":"2024-09-20T19:06:08.090679+0000","last_fullsized":"2024-09-20T19:06:08.090679+0000","mapping_epoch":402,"log_start":"378'9916","ondisk_log_start":"378'9916","created":325,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4550,"log_dups_size":0,"ondisk_log_size":4550,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-20T19:07:08.060156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7233,"num_write_kb":231456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4357,"num_bytes_recovered":285540352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"274'27","reported_seq":844,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.065821+0000","last_change":"2024-09-20T19:04:20.920811+0000","last_active":"2024-09-20T19:06:06.065821+0000","last_peered":"2024-09-20T19:06:06.065821+0000","last_clean":"2024-09-20T19:06:06.065821+0000","last_became_active":"2024-09-20T19:04:20.897049+0000","last_became_peered":"2024-09-20T19:04:20.897049+0000","last_unstale":"2024-09-20T19:06:06.065821+0000","last_undegraded":"2024-09-20T19:06:06.065821+0000","last_fullsized":"2024-09-20T19:06:06.065821+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'27","last_scrub_stamp":"2024-09-20T19:02:27.897138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:27.897138+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:33.546677+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":774,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.092392+0000","last_change":"2024-09-20T19:05:55.742165+0000","last_active":"2024-09-20T19:06:08.092392+0000","last_peered":"2024-09-20T19:06:08.092392+0000","last_clean":"2024-09-20T19:06:08.092392+0000","last_became_active":"2024-09-20T18:49:45.936104+0000","last_became_peered":"2024-09-20T18:49:45.936104+0000","last_unstale":"2024-09-20T19:06:08.092392+0000","last_undegraded":"2024-09-20T19:06:08.092392+0000","last_fullsized":"2024-09-20T19:06:08.092392+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:55.742106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:55.742106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:16.985762+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"375'12","reported_seq":123,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.092412+0000","last_change":"2024-09-20T19:04:21.030980+0000","last_active":"2024-09-20T19:06:08.092412+0000","last_peered":"2024-09-20T19:06:08.092412+0000","last_clean":"2024-09-20T19:06:08.092412+0000","last_became_active":"2024-09-20T19:04:20.907228+0000","last_became_peered":"2024-09-20T19:04:20.907228+0000","last_unstale":"2024-09-20T19:06:08.092412+0000","last_undegraded":"2024-09-20T19:06:08.092412+0000","last_fullsized":"2024-09-20T19:06:08.092412+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"308'10","last_scrub_stamp":"2024-09-20T19:02:23.017626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:23.017626+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":754,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.093525+0000","last_change":"2024-09-20T19:02:22.736879+0000","last_active":"2024-09-20T19:06:08.093525+0000","last_peered":"2024-09-20T19:06:08.093525+0000","last_clean":"2024-09-20T19:06:08.093525+0000","last_became_active":"2024-09-20T18:49:51.971399+0000","last_became_peered":"2024-09-20T18:49:51.971399+0000","last_unstale":"2024-09-20T19:06:08.093525+0000","last_undegraded":"2024-09-20T19:06:08.093525+0000","last_fullsized":"2024-09-20T19:06:08.093525+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:22.736825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:22.736825+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:02.798693+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.e","version":"409'14796","reported_seq":15917,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.092511+0000","last_change":"2024-09-20T19:04:21.970634+0000","last_active":"2024-09-20T19:06:08.092511+0000","last_peered":"2024-09-20T19:06:08.092511+0000","last_clean":"2024-09-20T19:06:08.092511+0000","last_became_active":"2024-09-20T19:04:21.956145+0000","last_became_peered":"2024-09-20T19:04:21.956145+0000","last_unstale":"2024-09-20T19:06:08.092511+0000","last_undegraded":"2024-09-20T19:06:08.092511+0000","last_fullsized":"2024-09-20T19:06:08.092511+0000","mapping_epoch":375,"log_start":"378'10228","ondisk_log_start":"378'10228","created":325,"last_epoch_clean":376,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4568,"log_dups_size":0,"ondisk_log_size":4568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14796,"num_write_kb":473472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":453,"num_bytes_recovered":29687808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"273'30","reported_seq":829,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.092260+0000","last_change":"2024-09-20T19:05:54.717703+0000","last_active":"2024-09-20T19:06:08.092260+0000","last_peered":"2024-09-20T19:06:08.092260+0000","last_clean":"2024-09-20T19:06:08.092260+0000","last_became_active":"2024-09-20T18:49:48.499189+0000","last_became_peered":"2024-09-20T18:49:48.499189+0000","last_unstale":"2024-09-20T19:06:08.092260+0000","last_undegraded":"2024-09-20T19:06:08.092260+0000","last_fullsized":"2024-09-20T19:06:08.092260+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'30","last_scrub_stamp":"2024-09-20T19:05:54.717609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:54.717609+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:54.790802+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":758,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.092692+0000","last_change":"2024-09-20T19:02:22.806587+0000","last_active":"2024-09-20T19:06:08.092692+0000","last_peered":"2024-09-20T19:06:08.092692+0000","last_clean":"2024-09-20T19:06:08.092692+0000","last_became_active":"2024-09-20T18:49:45.940544+0000","last_became_peered":"2024-09-20T18:49:45.940544+0000","last_unstale":"2024-09-20T19:06:08.092692+0000","last_undegraded":"2024-09-20T19:06:08.092692+0000","last_fullsized":"2024-09-20T19:06:08.092692+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:22.806509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:22.806509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":97,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.090936+0000","last_change":"2024-09-20T19:04:20.889280+0000","last_active":"2024-09-20T19:06:08.090936+0000","last_peered":"2024-09-20T19:06:08.090936+0000","last_clean":"2024-09-20T19:06:08.090936+0000","last_became_active":"2024-09-20T19:04:20.888422+0000","last_became_peered":"2024-09-20T19:04:20.888422+0000","last_unstale":"2024-09-20T19:06:08.090936+0000","last_undegraded":"2024-09-20T19:06:08.090936+0000","last_fullsized":"2024-09-20T19:06:08.090936+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.229201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.229201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:33.231712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":105,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.091306+0000","last_change":"2024-09-20T19:05:47.121946+0000","last_active":"2024-09-20T19:06:08.091306+0000","last_peered":"2024-09-20T19:06:08.091306+0000","last_clean":"2024-09-20T19:06:08.091306+0000","last_became_active":"2024-09-20T19:04:20.889418+0000","last_became_peered":"2024-09-20T19:04:20.889418+0000","last_unstale":"2024-09-20T19:06:08.091306+0000","last_undegraded":"2024-09-20T19:06:08.091306+0000","last_fullsized":"2024-09-20T19:06:08.091306+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:47.121842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:05:47.121842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:15.049722+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"409'14390","reported_seq":14601,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.070165+0000","last_change":"2024-09-20T19:05:50.399647+0000","last_active":"2024-09-20T19:06:06.070165+0000","last_peered":"2024-09-20T19:06:06.070165+0000","last_clean":"2024-09-20T19:06:06.070165+0000","last_became_active":"2024-09-20T19:02:11.827468+0000","last_became_peered":"2024-09-20T19:02:11.827468+0000","last_unstale":"2024-09-20T19:06:06.070165+0000","last_undegraded":"2024-09-20T19:06:06.070165+0000","last_fullsized":"2024-09-20T19:06:06.070165+0000","mapping_epoch":325,"log_start":"372'8355","ondisk_log_start":"372'8355","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"408'13932","last_scrub_stamp":"2024-09-20T19:05:50.399156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:05:50.399156+0000","objects_scrubbed":465,"log_size":6035,"log_dups_size":0,"ondisk_log_size":6035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:50.441485+0000","scrub_duration":209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14390,"num_write_kb":460480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"274'28","reported_seq":113,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.090837+0000","last_change":"2024-09-20T19:04:20.976682+0000","last_active":"2024-09-20T19:06:08.090837+0000","last_peered":"2024-09-20T19:06:08.090837+0000","last_clean":"2024-09-20T19:06:08.090837+0000","last_became_active":"2024-09-20T19:04:20.897444+0000","last_became_peered":"2024-09-20T19:04:20.897444+0000","last_unstale":"2024-09-20T19:06:08.090837+0000","last_undegraded":"2024-09-20T19:06:08.090837+0000","last_fullsized":"2024-09-20T19:06:08.090837+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'28","last_scrub_stamp":"2024-09-20T19:03:06.269623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:06.269623+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":41,"num_read_kb":34,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":799,"reported_epoch":417,"state":"active+clean","last_fresh":"2024-09-20T19:06:07.087594+0000","last_change":"2024-09-20T19:05:47.629956+0000","last_active":"2024-09-20T19:06:07.087594+0000","last_peered":"2024-09-20T19:06:07.087594+0000","last_clean":"2024-09-20T19:06:07.087594+0000","last_became_active":"2024-09-20T19:04:20.906790+0000","last_became_peered":"2024-09-20T19:04:20.906790+0000","last_unstale":"2024-09-20T19:06:07.087594+0000","last_undegraded":"2024-09-20T19:06:07.087594+0000","last_fullsized":"2024-09-20T19:06:07.087594+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:05:47.629892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:47.629892+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:06.773486+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"375'13","reported_seq":1011,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.092124+0000","last_change":"2024-09-20T19:04:20.916872+0000","last_active":"2024-09-20T19:06:08.092124+0000","last_peered":"2024-09-20T19:06:08.092124+0000","last_clean":"2024-09-20T19:06:08.092124+0000","last_became_active":"2024-09-20T19:04:20.896424+0000","last_became_peered":"2024-09-20T19:04:20.896424+0000","last_unstale":"2024-09-20T19:06:08.092124+0000","last_undegraded":"2024-09-20T19:06:08.092124+0000","last_fullsized":"2024-09-20T19:06:08.092124+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:58:23.114371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:23.114371+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:38.642422+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":757,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.097770+0000","last_change":"2024-09-20T18:59:05.082117+0000","last_active":"2024-09-20T19:06:06.097770+0000","last_peered":"2024-09-20T19:06:06.097770+0000","last_clean":"2024-09-20T19:06:06.097770+0000","last_became_active":"2024-09-20T18:59:05.081814+0000","last_became_peered":"2024-09-20T18:59:05.081814+0000","last_unstale":"2024-09-20T19:06:06.097770+0000","last_undegraded":"2024-09-20T19:06:06.097770+0000","last_fullsized":"2024-09-20T19:06:06.097770+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:29.530001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:29.530001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:36.792476+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"409'14450","reported_seq":32297,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.097873+0000","last_change":"2024-09-20T19:05:48.249345+0000","last_active":"2024-09-20T19:06:06.097873+0000","last_peered":"2024-09-20T19:06:06.097873+0000","last_clean":"2024-09-20T19:06:06.097873+0000","last_became_active":"2024-09-20T19:05:30.889717+0000","last_became_peered":"2024-09-20T19:05:30.889717+0000","last_unstale":"2024-09-20T19:06:06.097873+0000","last_undegraded":"2024-09-20T19:06:06.097873+0000","last_fullsized":"2024-09-20T19:06:06.097873+0000","mapping_epoch":400,"log_start":"377'9355","ondisk_log_start":"377'9355","created":325,"last_epoch_clean":401,"parent":"0.0","parent_split_bits":0,"last_scrub":"407'13792","last_scrub_stamp":"2024-09-20T19:05:48.249177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:05:48.249177+0000","objects_scrubbed":671,"log_size":5095,"log_dups_size":0,"ondisk_log_size":5095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:59.965171+0000","scrub_duration":423,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14450,"num_write_kb":462400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8799,"num_bytes_recovered":576651264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"274'20","reported_seq":612,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.094066+0000","last_change":"2024-09-20T19:05:48.741237+0000","last_active":"2024-09-20T19:06:08.094066+0000","last_peered":"2024-09-20T19:06:08.094066+0000","last_clean":"2024-09-20T19:06:08.094066+0000","last_became_active":"2024-09-20T19:04:49.419140+0000","last_became_peered":"2024-09-20T19:04:49.419140+0000","last_unstale":"2024-09-20T19:06:08.094066+0000","last_undegraded":"2024-09-20T19:06:08.094066+0000","last_fullsized":"2024-09-20T19:06:08.094066+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'20","last_scrub_stamp":"2024-09-20T19:05:48.741059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:48.741059+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:13.202074+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":790,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.094684+0000","last_change":"2024-09-20T19:05:56.820141+0000","last_active":"2024-09-20T19:06:08.094684+0000","last_peered":"2024-09-20T19:06:08.094684+0000","last_clean":"2024-09-20T19:06:08.094684+0000","last_became_active":"2024-09-20T18:49:45.942496+0000","last_became_peered":"2024-09-20T18:49:45.942496+0000","last_unstale":"2024-09-20T19:06:08.094684+0000","last_undegraded":"2024-09-20T19:06:08.094684+0000","last_fullsized":"2024-09-20T19:06:08.094684+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:56.820079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:56.820079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:16.442542+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"262'10","reported_seq":984,"reported_epoch":417,"state":"active+clean","last_fresh":"2024-09-20T19:06:07.086893+0000","last_change":"2024-09-20T19:02:04.994109+0000","last_active":"2024-09-20T19:06:07.086893+0000","last_peered":"2024-09-20T19:06:07.086893+0000","last_clean":"2024-09-20T19:06:07.086893+0000","last_became_active":"2024-09-20T18:59:05.080183+0000","last_became_peered":"2024-09-20T18:59:05.080183+0000","last_unstale":"2024-09-20T19:06:07.086893+0000","last_undegraded":"2024-09-20T19:06:07.086893+0000","last_fullsized":"2024-09-20T19:06:07.086893+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'10","last_scrub_stamp":"2024-09-20T19:02:04.994047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:04.994047+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":738,"reported_epoch":417,"state":"active+clean","last_fresh":"2024-09-20T19:06:07.086395+0000","last_change":"2024-09-20T19:01:37.900410+0000","last_active":"2024-09-20T19:06:07.086395+0000","last_peered":"2024-09-20T19:06:07.086395+0000","last_clean":"2024-09-20T19:06:07.086395+0000","last_became_active":"2024-09-20T18:49:51.964780+0000","last_became_peered":"2024-09-20T18:49:51.964780+0000","last_unstale":"2024-09-20T19:06:07.086395+0000","last_undegraded":"2024-09-20T19:06:07.086395+0000","last_fullsized":"2024-09-20T19:06:07.086395+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:37.900328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:37.900328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.d","version":"409'14948","reported_seq":24254,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.069184+0000","last_change":"2024-09-20T19:05:43.612046+0000","last_active":"2024-09-20T19:06:06.069184+0000","last_peered":"2024-09-20T19:06:06.069184+0000","last_clean":"2024-09-20T19:06:06.069184+0000","last_became_active":"2024-09-20T19:05:22.993748+0000","last_became_peered":"2024-09-20T19:05:22.993748+0000","last_unstale":"2024-09-20T19:06:06.069184+0000","last_undegraded":"2024-09-20T19:06:06.069184+0000","last_fullsized":"2024-09-20T19:06:06.069184+0000","mapping_epoch":395,"log_start":"372'8917","ondisk_log_start":"372'8917","created":325,"last_epoch_clean":396,"parent":"0.0","parent_split_bits":0,"last_scrub":"406'13977","last_scrub_stamp":"2024-09-20T19:05:43.611752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:05:43.611752+0000","objects_scrubbed":985,"log_size":6031,"log_dups_size":0,"ondisk_log_size":6031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:47.423649+0000","scrub_duration":433,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14948,"num_write_kb":478273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4538,"num_bytes_recovered":189071384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"274'29","reported_seq":812,"reported_epoch":417,"state":"active+clean","last_fresh":"2024-09-20T19:06:07.086994+0000","last_change":"2024-09-20T19:05:43.631990+0000","last_active":"2024-09-20T19:06:07.086994+0000","last_peered":"2024-09-20T19:06:07.086994+0000","last_clean":"2024-09-20T19:06:07.086994+0000","last_became_active":"2024-09-20T18:49:48.497736+0000","last_became_peered":"2024-09-20T18:49:48.497736+0000","last_unstale":"2024-09-20T19:06:07.086994+0000","last_undegraded":"2024-09-20T19:06:07.086994+0000","last_fullsized":"2024-09-20T19:06:07.086994+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'29","last_scrub_stamp":"2024-09-20T19:05:43.631845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:43.631845+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:56.642378+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":762,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.072223+0000","last_change":"2024-09-20T19:05:47.837587+0000","last_active":"2024-09-20T19:06:06.072223+0000","last_peered":"2024-09-20T19:06:06.072223+0000","last_clean":"2024-09-20T19:06:06.072223+0000","last_became_active":"2024-09-20T18:49:45.942253+0000","last_became_peered":"2024-09-20T18:49:45.942253+0000","last_unstale":"2024-09-20T19:06:06.072223+0000","last_undegraded":"2024-09-20T19:06:06.072223+0000","last_fullsized":"2024-09-20T19:06:06.072223+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:47.837520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:47.837520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:15.032955+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":349,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.069550+0000","last_change":"2024-09-20T19:05:48.203244+0000","last_active":"2024-09-20T19:06:06.069550+0000","last_peered":"2024-09-20T19:06:06.069550+0000","last_clean":"2024-09-20T19:06:06.069550+0000","last_became_active":"2024-09-20T18:59:05.085636+0000","last_became_peered":"2024-09-20T18:59:05.085636+0000","last_unstale":"2024-09-20T19:06:06.069550+0000","last_undegraded":"2024-09-20T19:06:06.069550+0000","last_fullsized":"2024-09-20T19:06:06.069550+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:48.203182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:05:48.203182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:58.736758+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":772,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.071723+0000","last_change":"2024-09-20T19:02:23.886386+0000","last_active":"2024-09-20T19:06:06.071723+0000","last_peered":"2024-09-20T19:06:06.071723+0000","last_clean":"2024-09-20T19:06:06.071723+0000","last_became_active":"2024-09-20T18:59:05.085131+0000","last_became_peered":"2024-09-20T18:59:05.085131+0000","last_unstale":"2024-09-20T19:06:06.071723+0000","last_undegraded":"2024-09-20T19:06:06.071723+0000","last_fullsized":"2024-09-20T19:06:06.071723+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:23.886308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:23.886308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:38.961464+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"409'14366","reported_seq":14628,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.069153+0000","last_change":"2024-09-20T19:03:15.459244+0000","last_active":"2024-09-20T19:06:06.069153+0000","last_peered":"2024-09-20T19:06:06.069153+0000","last_clean":"2024-09-20T19:06:06.069153+0000","last_became_active":"2024-09-20T19:02:11.827305+0000","last_became_peered":"2024-09-20T19:02:11.827305+0000","last_unstale":"2024-09-20T19:06:06.069153+0000","last_undegraded":"2024-09-20T19:06:06.069153+0000","last_fullsized":"2024-09-20T19:06:06.069153+0000","mapping_epoch":325,"log_start":"371'8302","ondisk_log_start":"371'8302","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"350'5605","last_scrub_stamp":"2024-09-20T19:03:15.459174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:03:15.459174+0000","objects_scrubbed":5605,"log_size":6064,"log_dups_size":0,"ondisk_log_size":6064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":598,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14366,"num_write_kb":459712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"353'41","reported_seq":1066,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.072112+0000","last_change":"2024-09-20T19:05:48.855983+0000","last_active":"2024-09-20T19:06:06.072112+0000","last_peered":"2024-09-20T19:06:06.072112+0000","last_clean":"2024-09-20T19:06:06.072112+0000","last_became_active":"2024-09-20T19:03:16.976916+0000","last_became_peered":"2024-09-20T19:03:16.976916+0000","last_unstale":"2024-09-20T19:06:06.072112+0000","last_undegraded":"2024-09-20T19:06:06.072112+0000","last_fullsized":"2024-09-20T19:06:06.072112+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"353'41","last_scrub_stamp":"2024-09-20T19:05:48.855876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:48.855876+0000","objects_scrubbed":10,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:09.142116+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":97,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.090583+0000","last_change":"2024-09-20T19:04:20.899630+0000","last_active":"2024-09-20T19:06:08.090583+0000","last_peered":"2024-09-20T19:06:08.090583+0000","last_clean":"2024-09-20T19:06:08.090583+0000","last_became_active":"2024-09-20T19:04:20.899352+0000","last_became_peered":"2024-09-20T19:04:20.899352+0000","last_unstale":"2024-09-20T19:06:08.090583+0000","last_undegraded":"2024-09-20T19:06:08.090583+0000","last_fullsized":"2024-09-20T19:06:08.090583+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:00:31.630361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:00:31.630361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":822,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.094189+0000","last_change":"2024-09-20T19:03:11.910684+0000","last_active":"2024-09-20T19:06:08.094189+0000","last_peered":"2024-09-20T19:06:08.094189+0000","last_clean":"2024-09-20T19:06:08.094189+0000","last_became_active":"2024-09-20T18:59:05.086786+0000","last_became_peered":"2024-09-20T18:59:05.086786+0000","last_unstale":"2024-09-20T19:06:08.094189+0000","last_undegraded":"2024-09-20T19:06:08.094189+0000","last_fullsized":"2024-09-20T19:06:08.094189+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-20T19:03:11.909394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:03:11.909394+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0.0012781159999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":790,"reported_epoch":417,"state":"active+clean","last_fresh":"2024-09-20T19:06:07.087539+0000","last_change":"2024-09-20T19:04:20.907534+0000","last_active":"2024-09-20T19:06:07.087539+0000","last_peered":"2024-09-20T19:06:07.087539+0000","last_clean":"2024-09-20T19:06:07.087539+0000","last_became_active":"2024-09-20T19:04:20.907038+0000","last_became_peered":"2024-09-20T19:04:20.907038+0000","last_unstale":"2024-09-20T19:06:07.087539+0000","last_undegraded":"2024-09-20T19:06:07.087539+0000","last_fullsized":"2024-09-20T19:06:07.087539+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:15.034340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:03:15.034340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":763,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.092055+0000","last_change":"2024-09-20T19:04:20.896616+0000","last_active":"2024-09-20T19:06:08.092055+0000","last_peered":"2024-09-20T19:06:08.092055+0000","last_clean":"2024-09-20T19:06:08.092055+0000","last_became_active":"2024-09-20T19:04:20.896281+0000","last_became_peered":"2024-09-20T19:04:20.896281+0000","last_unstale":"2024-09-20T19:06:08.092055+0000","last_undegraded":"2024-09-20T19:06:08.092055+0000","last_fullsized":"2024-09-20T19:06:08.092055+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:04.526683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:04.526683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"409'14740","reported_seq":14965,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.093700+0000","last_change":"2024-09-20T19:06:03.078240+0000","last_active":"2024-09-20T19:06:08.093700+0000","last_peered":"2024-09-20T19:06:08.093700+0000","last_clean":"2024-09-20T19:06:08.093700+0000","last_became_active":"2024-09-20T19:06:03.054810+0000","last_became_peered":"2024-09-20T19:06:03.054810+0000","last_unstale":"2024-09-20T19:06:08.093700+0000","last_undegraded":"2024-09-20T19:06:08.093700+0000","last_fullsized":"2024-09-20T19:06:08.093700+0000","mapping_epoch":325,"log_start":"376'9027","ondisk_log_start":"376'9027","created":325,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"407'13951","last_scrub_stamp":"2024-09-20T19:05:46.982716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:05:46.982716+0000","objects_scrubbed":795,"log_size":5713,"log_dups_size":0,"ondisk_log_size":5713,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-20T19:07:04.811244+0000","scrub_duration":218,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7370,"num_write_kb":235840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"273'17","reported_seq":807,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.070125+0000","last_change":"2024-09-20T19:05:43.974098+0000","last_active":"2024-09-20T19:06:06.070125+0000","last_peered":"2024-09-20T19:06:06.070125+0000","last_clean":"2024-09-20T19:06:06.070125+0000","last_became_active":"2024-09-20T18:49:47.950904+0000","last_became_peered":"2024-09-20T18:49:47.950904+0000","last_unstale":"2024-09-20T19:06:06.070125+0000","last_undegraded":"2024-09-20T19:06:06.070125+0000","last_fullsized":"2024-09-20T19:06:06.070125+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'17","last_scrub_stamp":"2024-09-20T19:05:43.974013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:43.974013+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:01.950625+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":36,"num_read_kb":25,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":785,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.070974+0000","last_change":"2024-09-20T19:02:29.840761+0000","last_active":"2024-09-20T19:06:06.070974+0000","last_peered":"2024-09-20T19:06:06.070974+0000","last_clean":"2024-09-20T19:06:06.070974+0000","last_became_active":"2024-09-20T18:59:05.079789+0000","last_became_peered":"2024-09-20T18:59:05.079789+0000","last_unstale":"2024-09-20T19:06:06.070974+0000","last_undegraded":"2024-09-20T19:06:06.070974+0000","last_fullsized":"2024-09-20T19:06:06.070974+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:29.840601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:29.840601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:33.076198+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":766,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.061746+0000","last_change":"2024-09-20T19:01:58.789304+0000","last_active":"2024-09-20T19:06:06.061746+0000","last_peered":"2024-09-20T19:06:06.061746+0000","last_clean":"2024-09-20T19:06:06.061746+0000","last_became_active":"2024-09-20T18:49:44.937135+0000","last_became_peered":"2024-09-20T18:49:44.937135+0000","last_unstale":"2024-09-20T19:06:06.061746+0000","last_undegraded":"2024-09-20T19:06:06.061746+0000","last_fullsized":"2024-09-20T19:06:06.061746+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:01:58.789251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:01:58.789251+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00095558800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":741,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.060735+0000","last_change":"2024-09-20T19:01:17.747452+0000","last_active":"2024-09-20T19:06:06.060735+0000","last_peered":"2024-09-20T19:06:06.060735+0000","last_clean":"2024-09-20T19:06:06.060735+0000","last_became_active":"2024-09-20T18:49:49.954424+0000","last_became_peered":"2024-09-20T18:49:49.954424+0000","last_unstale":"2024-09-20T19:06:06.060735+0000","last_undegraded":"2024-09-20T19:06:06.060735+0000","last_fullsized":"2024-09-20T19:06:06.060735+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:17.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:17.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:58.515630+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":752,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.099060+0000","last_change":"2024-09-20T19:01:15.203591+0000","last_active":"2024-09-20T19:06:06.099060+0000","last_peered":"2024-09-20T19:06:06.099060+0000","last_clean":"2024-09-20T19:06:06.099060+0000","last_became_active":"2024-09-20T18:59:05.081459+0000","last_became_peered":"2024-09-20T18:59:05.081459+0000","last_unstale":"2024-09-20T19:06:06.099060+0000","last_undegraded":"2024-09-20T19:06:06.099060+0000","last_fullsized":"2024-09-20T19:06:06.099060+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:15.203529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:15.203529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:37.777620+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"409'14426","reported_seq":14633,"reported_epoch":417,"state":"active+clean","last_fresh":"2024-09-20T19:06:07.086548+0000","last_change":"2024-09-20T19:06:03.080284+0000","last_active":"2024-09-20T19:06:07.086548+0000","last_peered":"2024-09-20T19:06:07.086548+0000","last_clean":"2024-09-20T19:06:07.086548+0000","last_became_active":"2024-09-20T19:06:03.060383+0000","last_became_peered":"2024-09-20T19:06:03.060383+0000","last_unstale":"2024-09-20T19:06:07.086548+0000","last_undegraded":"2024-09-20T19:06:07.086548+0000","last_fullsized":"2024-09-20T19:06:07.086548+0000","mapping_epoch":325,"log_start":"376'8984","ondisk_log_start":"376'8984","created":325,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":5442,"log_dups_size":0,"ondisk_log_size":5442,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7213,"num_write_kb":230816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"273'24","reported_seq":805,"reported_epoch":417,"state":"active+clean","last_fresh":"2024-09-20T19:06:07.087506+0000","last_change":"2024-09-20T18:59:06.022021+0000","last_active":"2024-09-20T19:06:07.087506+0000","last_peered":"2024-09-20T19:06:07.087506+0000","last_clean":"2024-09-20T19:06:07.087506+0000","last_became_active":"2024-09-20T18:49:47.949230+0000","last_became_peered":"2024-09-20T18:49:47.949230+0000","last_unstale":"2024-09-20T19:06:07.087506+0000","last_undegraded":"2024-09-20T19:06:07.087506+0000","last_fullsized":"2024-09-20T19:06:07.087506+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:59:06.021962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:59:06.021962+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:34.733375+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":790,"reported_epoch":417,"state":"active+clean","last_fresh":"2024-09-20T19:06:07.086630+0000","last_change":"2024-09-20T19:04:20.907250+0000","last_active":"2024-09-20T19:06:07.086630+0000","last_peered":"2024-09-20T19:06:07.086630+0000","last_clean":"2024-09-20T19:06:07.086630+0000","last_became_active":"2024-09-20T19:04:20.906969+0000","last_became_peered":"2024-09-20T19:04:20.906969+0000","last_unstale":"2024-09-20T19:06:07.086630+0000","last_undegraded":"2024-09-20T19:06:07.086630+0000","last_fullsized":"2024-09-20T19:06:07.086630+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:24.984022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:24.984022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":1748,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.140278+0000","last_change":"2024-09-20T19:02:02.494344+0000","last_active":"2024-09-20T19:06:06.140278+0000","last_peered":"2024-09-20T19:06:06.140278+0000","last_clean":"2024-09-20T19:06:06.140278+0000","last_became_active":"2024-09-20T18:49:44.936819+0000","last_became_peered":"2024-09-20T18:49:44.936819+0000","last_unstale":"2024-09-20T19:06:06.140278+0000","last_undegraded":"2024-09-20T19:06:06.140278+0000","last_fullsized":"2024-09-20T19:06:06.140278+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-20T19:02:02.494291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:02:02.494291+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:39.801016+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0010996199999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":973,"num_read_kb":968,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":774,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.094943+0000","last_change":"2024-09-20T19:05:51.743485+0000","last_active":"2024-09-20T19:06:08.094943+0000","last_peered":"2024-09-20T19:06:08.094943+0000","last_clean":"2024-09-20T19:06:08.094943+0000","last_became_active":"2024-09-20T18:49:49.961066+0000","last_became_peered":"2024-09-20T18:49:49.961066+0000","last_unstale":"2024-09-20T19:06:08.094943+0000","last_undegraded":"2024-09-20T19:06:08.094943+0000","last_fullsized":"2024-09-20T19:06:08.094943+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:51.743363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:05:51.743363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:06.075322+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":758,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.071828+0000","last_change":"2024-09-20T19:02:26.861253+0000","last_active":"2024-09-20T19:06:06.071828+0000","last_peered":"2024-09-20T19:06:06.071828+0000","last_clean":"2024-09-20T19:06:06.071828+0000","last_became_active":"2024-09-20T18:49:51.970343+0000","last_became_peered":"2024-09-20T18:49:51.970343+0000","last_unstale":"2024-09-20T19:06:06.071828+0000","last_undegraded":"2024-09-20T19:06:06.071828+0000","last_fullsized":"2024-09-20T19:06:06.071828+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:26.861185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:26.861185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:32.099095+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"409'14558","reported_seq":14756,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.069050+0000","last_change":"2024-09-20T19:02:11.828065+0000","last_active":"2024-09-20T19:06:06.069050+0000","last_peered":"2024-09-20T19:06:06.069050+0000","last_clean":"2024-09-20T19:06:06.069050+0000","last_became_active":"2024-09-20T19:02:11.827628+0000","last_became_peered":"2024-09-20T19:02:11.827628+0000","last_unstale":"2024-09-20T19:06:06.069050+0000","last_undegraded":"2024-09-20T19:06:06.069050+0000","last_fullsized":"2024-09-20T19:06:06.069050+0000","mapping_epoch":325,"log_start":"372'8527","ondisk_log_start":"372'8527","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":6031,"log_dups_size":0,"ondisk_log_size":6031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14558,"num_write_kb":465856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"375'34","reported_seq":525,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.069993+0000","last_change":"2024-09-20T19:05:54.115428+0000","last_active":"2024-09-20T19:06:06.069993+0000","last_peered":"2024-09-20T19:06:06.069993+0000","last_clean":"2024-09-20T19:06:06.069993+0000","last_became_active":"2024-09-20T19:04:20.928727+0000","last_became_peered":"2024-09-20T19:04:20.928727+0000","last_unstale":"2024-09-20T19:06:06.069993+0000","last_undegraded":"2024-09-20T19:06:06.069993+0000","last_fullsized":"2024-09-20T19:06:06.069993+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"375'34","last_scrub_stamp":"2024-09-20T19:05:54.115376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:54.115376+0000","objects_scrubbed":8,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:59.905707+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":462,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":788,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.092604+0000","last_change":"2024-09-20T19:05:07.665281+0000","last_active":"2024-09-20T19:06:08.092604+0000","last_peered":"2024-09-20T19:06:08.092604+0000","last_clean":"2024-09-20T19:06:08.092604+0000","last_became_active":"2024-09-20T18:59:05.079111+0000","last_became_peered":"2024-09-20T18:59:05.079111+0000","last_unstale":"2024-09-20T19:06:08.092604+0000","last_undegraded":"2024-09-20T19:06:08.092604+0000","last_fullsized":"2024-09-20T19:06:08.092604+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:07.665167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:07.665167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:16.643707+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":832,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.140463+0000","last_change":"2024-09-20T19:03:14.368338+0000","last_active":"2024-09-20T19:06:06.140463+0000","last_peered":"2024-09-20T19:06:06.140463+0000","last_clean":"2024-09-20T19:06:06.140463+0000","last_became_active":"2024-09-20T18:59:05.086103+0000","last_became_peered":"2024-09-20T18:59:05.086103+0000","last_unstale":"2024-09-20T19:06:06.140463+0000","last_undegraded":"2024-09-20T19:06:06.140463+0000","last_fullsized":"2024-09-20T19:06:06.140463+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:03:14.368279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:03:14.368279+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":747,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.098290+0000","last_change":"2024-09-20T19:04:35.533709+0000","last_active":"2024-09-20T19:06:06.098290+0000","last_peered":"2024-09-20T19:06:06.098290+0000","last_clean":"2024-09-20T19:06:06.098290+0000","last_became_active":"2024-09-20T18:49:49.956128+0000","last_became_peered":"2024-09-20T18:49:49.956128+0000","last_unstale":"2024-09-20T19:06:06.098290+0000","last_undegraded":"2024-09-20T19:06:06.098290+0000","last_fullsized":"2024-09-20T19:06:06.098290+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:35.533633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:35.533633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":787,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.094695+0000","last_change":"2024-09-20T19:04:20.899776+0000","last_active":"2024-09-20T19:06:08.094695+0000","last_peered":"2024-09-20T19:06:08.094695+0000","last_clean":"2024-09-20T19:06:08.094695+0000","last_became_active":"2024-09-20T19:04:20.899321+0000","last_became_peered":"2024-09-20T19:04:20.899321+0000","last_unstale":"2024-09-20T19:06:08.094695+0000","last_undegraded":"2024-09-20T19:06:08.094695+0000","last_fullsized":"2024-09-20T19:06:08.094695+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:30.557368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:30.557368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:39.737278+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.0","version":"409'14620","reported_seq":24066,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.069028+0000","last_change":"2024-09-20T19:06:03.098595+0000","last_active":"2024-09-20T19:06:06.069028+0000","last_peered":"2024-09-20T19:06:06.069028+0000","last_clean":"2024-09-20T19:06:06.069028+0000","last_became_active":"2024-09-20T19:06:03.071767+0000","last_became_peered":"2024-09-20T19:06:03.071767+0000","last_unstale":"2024-09-20T19:06:06.069028+0000","last_undegraded":"2024-09-20T19:06:06.069028+0000","last_fullsized":"2024-09-20T19:06:06.069028+0000","mapping_epoch":394,"log_start":"372'8539","ondisk_log_start":"372'8539","created":325,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":6081,"log_dups_size":0,"ondisk_log_size":6081,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7310,"num_write_kb":233920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2306,"num_bytes_recovered":100368384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"274'19","reported_seq":117,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.092109+0000","last_change":"2024-09-20T19:05:50.066533+0000","last_active":"2024-09-20T19:06:08.092109+0000","last_peered":"2024-09-20T19:06:08.092109+0000","last_clean":"2024-09-20T19:06:08.092109+0000","last_became_active":"2024-09-20T19:04:20.897175+0000","last_became_peered":"2024-09-20T19:04:20.897175+0000","last_unstale":"2024-09-20T19:06:08.092109+0000","last_undegraded":"2024-09-20T19:06:08.092109+0000","last_fullsized":"2024-09-20T19:06:08.092109+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'19","last_scrub_stamp":"2024-09-20T19:05:50.066359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:50.066359+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:56.712506+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":105,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.090935+0000","last_change":"2024-09-20T19:05:56.027187+0000","last_active":"2024-09-20T19:06:08.090935+0000","last_peered":"2024-09-20T19:06:08.090935+0000","last_clean":"2024-09-20T19:06:08.090935+0000","last_became_active":"2024-09-20T19:04:20.887102+0000","last_became_peered":"2024-09-20T19:04:20.887102+0000","last_unstale":"2024-09-20T19:06:08.090935+0000","last_undegraded":"2024-09-20T19:06:08.090935+0000","last_fullsized":"2024-09-20T19:06:08.090935+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:56.027123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:56.027123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:15.919213+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":805,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.094575+0000","last_change":"2024-09-20T19:04:20.899830+0000","last_active":"2024-09-20T19:06:08.094575+0000","last_peered":"2024-09-20T19:06:08.094575+0000","last_clean":"2024-09-20T19:06:08.094575+0000","last_became_active":"2024-09-20T19:04:20.899331+0000","last_became_peered":"2024-09-20T19:04:20.899331+0000","last_unstale":"2024-09-20T19:06:08.094575+0000","last_undegraded":"2024-09-20T19:06:08.094575+0000","last_fullsized":"2024-09-20T19:06:08.094575+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:38.463404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:38.463404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:26.726228+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":768,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.092796+0000","last_change":"2024-09-20T19:04:35.529672+0000","last_active":"2024-09-20T19:06:08.092796+0000","last_peered":"2024-09-20T19:06:08.092796+0000","last_clean":"2024-09-20T19:06:08.092796+0000","last_became_active":"2024-09-20T18:59:05.079897+0000","last_became_peered":"2024-09-20T18:59:05.079897+0000","last_unstale":"2024-09-20T19:06:08.092796+0000","last_undegraded":"2024-09-20T19:06:08.092796+0000","last_fullsized":"2024-09-20T19:06:08.092796+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:35.529584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:04:35.529584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"409'14876","reported_seq":15194,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.065971+0000","last_change":"2024-09-20T19:06:03.095462+0000","last_active":"2024-09-20T19:06:06.065971+0000","last_peered":"2024-09-20T19:06:06.065971+0000","last_clean":"2024-09-20T19:06:06.065971+0000","last_became_active":"2024-09-20T19:06:03.071949+0000","last_became_peered":"2024-09-20T19:06:03.071949+0000","last_unstale":"2024-09-20T19:06:06.065971+0000","last_undegraded":"2024-09-20T19:06:06.065971+0000","last_fullsized":"2024-09-20T19:06:06.065971+0000","mapping_epoch":325,"log_start":"369'8328","ondisk_log_start":"369'8328","created":325,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"392'12286","last_scrub_stamp":"2024-09-20T19:05:17.450066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:05:17.450066+0000","objects_scrubbed":2624,"log_size":6548,"log_dups_size":0,"ondisk_log_size":6548,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7438,"num_write_kb":238016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"274'31","reported_seq":115,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.092234+0000","last_change":"2024-09-20T19:04:20.949305+0000","last_active":"2024-09-20T19:06:08.092234+0000","last_peered":"2024-09-20T19:06:08.092234+0000","last_clean":"2024-09-20T19:06:08.092234+0000","last_became_active":"2024-09-20T19:04:20.883306+0000","last_became_peered":"2024-09-20T19:04:20.883306+0000","last_unstale":"2024-09-20T19:06:08.092234+0000","last_undegraded":"2024-09-20T19:06:08.092234+0000","last_fullsized":"2024-09-20T19:06:08.092234+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'31","last_scrub_stamp":"2024-09-20T19:03:12.196851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:12.196851+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":49,"num_read_kb":39,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":358,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.069624+0000","last_change":"2024-09-20T19:04:20.899712+0000","last_active":"2024-09-20T19:06:06.069624+0000","last_peered":"2024-09-20T19:06:06.069624+0000","last_clean":"2024-09-20T19:06:06.069624+0000","last_became_active":"2024-09-20T19:04:20.899415+0000","last_became_peered":"2024-09-20T19:04:20.899415+0000","last_unstale":"2024-09-20T19:06:06.069624+0000","last_undegraded":"2024-09-20T19:06:06.069624+0000","last_fullsized":"2024-09-20T19:06:06.069624+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:42.479489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:42.479489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:36.220255+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":787,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.071945+0000","last_change":"2024-09-20T19:05:46.866958+0000","last_active":"2024-09-20T19:06:06.071945+0000","last_peered":"2024-09-20T19:06:06.071945+0000","last_clean":"2024-09-20T19:06:06.071945+0000","last_became_active":"2024-09-20T19:04:20.891772+0000","last_became_peered":"2024-09-20T19:04:20.891772+0000","last_unstale":"2024-09-20T19:06:06.071945+0000","last_undegraded":"2024-09-20T19:06:06.071945+0000","last_fullsized":"2024-09-20T19:06:06.071945+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:46.866777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:05:46.866777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:55.257561+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":778,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.067465+0000","last_change":"2024-09-20T19:04:20.897937+0000","last_active":"2024-09-20T19:06:06.067465+0000","last_peered":"2024-09-20T19:06:06.067465+0000","last_clean":"2024-09-20T19:06:06.067465+0000","last_became_active":"2024-09-20T19:04:20.897569+0000","last_became_peered":"2024-09-20T19:04:20.897569+0000","last_unstale":"2024-09-20T19:06:06.067465+0000","last_undegraded":"2024-09-20T19:06:06.067465+0000","last_fullsized":"2024-09-20T19:06:06.067465+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:31.687188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:31.687188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:24.424476+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2","version":"409'14668","reported_seq":16220,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.072239+0000","last_change":"2024-09-20T19:06:03.099619+0000","last_active":"2024-09-20T19:06:06.072239+0000","last_peered":"2024-09-20T19:06:06.072239+0000","last_clean":"2024-09-20T19:06:06.072239+0000","last_became_active":"2024-09-20T19:06:03.080843+0000","last_became_peered":"2024-09-20T19:06:03.080843+0000","last_unstale":"2024-09-20T19:06:06.072239+0000","last_undegraded":"2024-09-20T19:06:06.072239+0000","last_fullsized":"2024-09-20T19:06:06.072239+0000","mapping_epoch":375,"log_start":"377'9482","ondisk_log_start":"377'9482","created":325,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":5186,"log_dups_size":0,"ondisk_log_size":5186,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7334,"num_write_kb":234688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":332,"num_bytes_recovered":21790720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"273'20","reported_seq":109,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.091978+0000","last_change":"2024-09-20T19:04:20.945143+0000","last_active":"2024-09-20T19:06:08.091978+0000","last_peered":"2024-09-20T19:06:08.091978+0000","last_clean":"2024-09-20T19:06:08.091978+0000","last_became_active":"2024-09-20T19:04:20.896483+0000","last_became_peered":"2024-09-20T19:04:20.896483+0000","last_unstale":"2024-09-20T19:06:08.091978+0000","last_undegraded":"2024-09-20T19:06:08.091978+0000","last_fullsized":"2024-09-20T19:06:08.091978+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'20","last_scrub_stamp":"2024-09-20T19:03:04.282800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:04.282800+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":799,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.093092+0000","last_change":"2024-09-20T19:05:48.678438+0000","last_active":"2024-09-20T19:06:08.093092+0000","last_peered":"2024-09-20T19:06:08.093092+0000","last_clean":"2024-09-20T19:06:08.093092+0000","last_became_active":"2024-09-20T19:04:20.892951+0000","last_became_peered":"2024-09-20T19:04:20.892951+0000","last_unstale":"2024-09-20T19:06:08.093092+0000","last_undegraded":"2024-09-20T19:06:08.093092+0000","last_fullsized":"2024-09-20T19:06:08.093092+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:48.678361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:48.678361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:02.488299+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":978,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.065160+0000","last_change":"2024-09-20T19:05:08.406307+0000","last_active":"2024-09-20T19:06:06.065160+0000","last_peered":"2024-09-20T19:06:06.065160+0000","last_clean":"2024-09-20T19:06:06.065160+0000","last_became_active":"2024-09-20T18:49:49.957005+0000","last_became_peered":"2024-09-20T18:49:49.957005+0000","last_unstale":"2024-09-20T19:06:06.065160+0000","last_undegraded":"2024-09-20T19:06:06.065160+0000","last_fullsized":"2024-09-20T19:06:06.065160+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T19:05:08.406231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:05:08.406231+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:20.576797+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":97,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.092216+0000","last_change":"2024-09-20T19:04:20.885294+0000","last_active":"2024-09-20T19:06:08.092216+0000","last_peered":"2024-09-20T19:06:08.092216+0000","last_clean":"2024-09-20T19:06:08.092216+0000","last_became_active":"2024-09-20T19:04:20.884994+0000","last_became_peered":"2024-09-20T19:04:20.884994+0000","last_unstale":"2024-09-20T19:06:08.092216+0000","last_undegraded":"2024-09-20T19:06:08.092216+0000","last_fullsized":"2024-09-20T19:06:08.092216+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:11.207994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:03:11.207994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.5","version":"409'14518","reported_seq":14812,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.061431+0000","last_change":"2024-09-20T19:06:03.101176+0000","last_active":"2024-09-20T19:06:06.061431+0000","last_peered":"2024-09-20T19:06:06.061431+0000","last_clean":"2024-09-20T19:06:06.061431+0000","last_became_active":"2024-09-20T19:06:03.076968+0000","last_became_peered":"2024-09-20T19:06:03.076968+0000","last_unstale":"2024-09-20T19:06:06.061431+0000","last_undegraded":"2024-09-20T19:06:06.061431+0000","last_fullsized":"2024-09-20T19:06:06.061431+0000","mapping_epoch":325,"log_start":"369'7968","ondisk_log_start":"369'7968","created":325,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"409'14097","last_scrub_stamp":"2024-09-20T19:05:51.722107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:05:51.722107+0000","objects_scrubbed":430,"log_size":6550,"log_dups_size":0,"ondisk_log_size":6550,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":273,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7259,"num_write_kb":232288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"274'15","reported_seq":839,"reported_epoch":417,"state":"active+clean","last_fresh":"2024-09-20T19:06:07.086942+0000","last_change":"2024-09-20T19:05:57.548758+0000","last_active":"2024-09-20T19:06:07.086942+0000","last_peered":"2024-09-20T19:06:07.086942+0000","last_clean":"2024-09-20T19:06:07.086942+0000","last_became_active":"2024-09-20T19:04:20.906834+0000","last_became_peered":"2024-09-20T19:04:20.906834+0000","last_unstale":"2024-09-20T19:06:07.086942+0000","last_undegraded":"2024-09-20T19:06:07.086942+0000","last_fullsized":"2024-09-20T19:06:07.086942+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'15","last_scrub_stamp":"2024-09-20T19:05:57.548683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:57.548683+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:13.922548+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":322,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.060768+0000","last_change":"2024-09-20T19:05:52.464362+0000","last_active":"2024-09-20T19:06:06.060768+0000","last_peered":"2024-09-20T19:06:06.060768+0000","last_clean":"2024-09-20T19:06:06.060768+0000","last_became_active":"2024-09-20T19:00:01.190875+0000","last_became_peered":"2024-09-20T19:00:01.190875+0000","last_unstale":"2024-09-20T19:06:06.060768+0000","last_undegraded":"2024-09-20T19:06:06.060768+0000","last_fullsized":"2024-09-20T19:06:06.060768+0000","mapping_epoch":277,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":278,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:52.464261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:52.464261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:59.446086+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":790,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.060740+0000","last_change":"2024-09-20T19:04:20.889614+0000","last_active":"2024-09-20T19:06:06.060740+0000","last_peered":"2024-09-20T19:06:06.060740+0000","last_clean":"2024-09-20T19:06:06.060740+0000","last_became_active":"2024-09-20T19:04:20.889273+0000","last_became_peered":"2024-09-20T19:04:20.889273+0000","last_unstale":"2024-09-20T19:06:06.060740+0000","last_undegraded":"2024-09-20T19:06:06.060740+0000","last_fullsized":"2024-09-20T19:06:06.060740+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:03.389611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:03.389611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:26.422333+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":753,"reported_epoch":417,"state":"active+clean","last_fresh":"2024-09-20T19:06:07.086534+0000","last_change":"2024-09-20T19:02:31.120809+0000","last_active":"2024-09-20T19:06:07.086534+0000","last_peered":"2024-09-20T19:06:07.086534+0000","last_clean":"2024-09-20T19:06:07.086534+0000","last_became_active":"2024-09-20T18:49:51.964098+0000","last_became_peered":"2024-09-20T18:49:51.964098+0000","last_unstale":"2024-09-20T19:06:07.086534+0000","last_undegraded":"2024-09-20T19:06:07.086534+0000","last_fullsized":"2024-09-20T19:06:07.086534+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:31.120623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:31.120623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"409'14380","reported_seq":14607,"reported_epoch":417,"state":"active+clean","last_fresh":"2024-09-20T19:06:07.086800+0000","last_change":"2024-09-20T19:06:03.095557+0000","last_active":"2024-09-20T19:06:07.086800+0000","last_peered":"2024-09-20T19:06:07.086800+0000","last_clean":"2024-09-20T19:06:07.086800+0000","last_became_active":"2024-09-20T19:06:03.072863+0000","last_became_peered":"2024-09-20T19:06:03.072863+0000","last_unstale":"2024-09-20T19:06:07.086800+0000","last_undegraded":"2024-09-20T19:06:07.086800+0000","last_fullsized":"2024-09-20T19:06:07.086800+0000","mapping_epoch":375,"log_start":"376'8997","ondisk_log_start":"376'8997","created":325,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":5383,"log_dups_size":0,"ondisk_log_size":5383,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-20T19:07:05.549934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7190,"num_write_kb":230080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"274'35","reported_seq":385,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.069422+0000","last_change":"2024-09-20T19:03:09.841596+0000","last_active":"2024-09-20T19:06:06.069422+0000","last_peered":"2024-09-20T19:06:06.069422+0000","last_clean":"2024-09-20T19:06:06.069422+0000","last_became_active":"2024-09-20T18:59:05.087391+0000","last_became_peered":"2024-09-20T18:59:05.087391+0000","last_unstale":"2024-09-20T19:06:06.069422+0000","last_undegraded":"2024-09-20T19:06:06.069422+0000","last_fullsized":"2024-09-20T19:06:06.069422+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'35","last_scrub_stamp":"2024-09-20T19:03:09.841361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:09.841361+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":785,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.098321+0000","last_change":"2024-09-20T19:05:49.763478+0000","last_active":"2024-09-20T19:06:06.098321+0000","last_peered":"2024-09-20T19:06:06.098321+0000","last_clean":"2024-09-20T19:06:06.098321+0000","last_became_active":"2024-09-20T18:59:05.086670+0000","last_became_peered":"2024-09-20T18:59:05.086670+0000","last_unstale":"2024-09-20T19:06:06.098321+0000","last_undegraded":"2024-09-20T19:06:06.098321+0000","last_fullsized":"2024-09-20T19:06:06.098321+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:49.763407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:49.763407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:09.527897+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":773,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.083810+0000","last_change":"2024-09-20T19:05:11.444197+0000","last_active":"2024-09-20T19:06:06.083810+0000","last_peered":"2024-09-20T19:06:06.083810+0000","last_clean":"2024-09-20T19:06:06.083810+0000","last_became_active":"2024-09-20T18:49:49.957558+0000","last_became_peered":"2024-09-20T18:49:49.957558+0000","last_unstale":"2024-09-20T19:06:06.083810+0000","last_undegraded":"2024-09-20T19:06:06.083810+0000","last_fullsized":"2024-09-20T19:06:06.083810+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:11.444143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:05:11.444143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:23.300683+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":743,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.140369+0000","last_change":"2024-09-20T18:53:47.370152+0000","last_active":"2024-09-20T19:06:06.140369+0000","last_peered":"2024-09-20T19:06:06.140369+0000","last_clean":"2024-09-20T19:06:06.140369+0000","last_became_active":"2024-09-20T18:49:51.970798+0000","last_became_peered":"2024-09-20T18:49:51.970798+0000","last_unstale":"2024-09-20T19:06:06.140369+0000","last_undegraded":"2024-09-20T19:06:06.140369+0000","last_fullsized":"2024-09-20T19:06:06.140369+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:47.370108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:47.370108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:30.728805+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.7","version":"409'14724","reported_seq":14929,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.070571+0000","last_change":"2024-09-20T19:06:03.092744+0000","last_active":"2024-09-20T19:06:06.070571+0000","last_peered":"2024-09-20T19:06:06.070571+0000","last_clean":"2024-09-20T19:06:06.070571+0000","last_became_active":"2024-09-20T19:06:03.072123+0000","last_became_peered":"2024-09-20T19:06:03.072123+0000","last_unstale":"2024-09-20T19:06:06.070571+0000","last_undegraded":"2024-09-20T19:06:06.070571+0000","last_fullsized":"2024-09-20T19:06:06.070571+0000","mapping_epoch":325,"log_start":"372'8707","ondisk_log_start":"372'8707","created":325,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":6017,"log_dups_size":0,"ondisk_log_size":6017,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7362,"num_write_kb":235584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"273'18","reported_seq":109,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.091067+0000","last_change":"2024-09-20T19:04:20.936465+0000","last_active":"2024-09-20T19:06:08.091067+0000","last_peered":"2024-09-20T19:06:08.091067+0000","last_clean":"2024-09-20T19:06:08.091067+0000","last_became_active":"2024-09-20T19:04:20.888905+0000","last_became_peered":"2024-09-20T19:04:20.888905+0000","last_unstale":"2024-09-20T19:06:08.091067+0000","last_undegraded":"2024-09-20T19:06:08.091067+0000","last_fullsized":"2024-09-20T19:06:08.091067+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'18","last_scrub_stamp":"2024-09-20T19:03:13.201592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:13.201592+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":796,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.098804+0000","last_change":"2024-09-20T19:04:50.680303+0000","last_active":"2024-09-20T19:06:06.098804+0000","last_peered":"2024-09-20T19:06:06.098804+0000","last_clean":"2024-09-20T19:06:06.098804+0000","last_became_active":"2024-09-20T19:04:20.892429+0000","last_became_peered":"2024-09-20T19:04:20.892429+0000","last_unstale":"2024-09-20T19:06:06.098804+0000","last_undegraded":"2024-09-20T19:06:06.098804+0000","last_fullsized":"2024-09-20T19:06:06.098804+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:50.680222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:04:50.680222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":765,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.060533+0000","last_change":"2024-09-20T19:04:48.406445+0000","last_active":"2024-09-20T19:06:06.060533+0000","last_peered":"2024-09-20T19:06:06.060533+0000","last_clean":"2024-09-20T19:06:06.060533+0000","last_became_active":"2024-09-20T18:49:49.954944+0000","last_became_peered":"2024-09-20T18:49:49.954944+0000","last_unstale":"2024-09-20T19:06:06.060533+0000","last_undegraded":"2024-09-20T19:06:06.060533+0000","last_fullsized":"2024-09-20T19:06:06.060533+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:48.406373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:48.406373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:59.483627+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":762,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.092816+0000","last_change":"2024-09-20T19:05:43.657860+0000","last_active":"2024-09-20T19:06:08.092816+0000","last_peered":"2024-09-20T19:06:08.092816+0000","last_clean":"2024-09-20T19:06:08.092816+0000","last_became_active":"2024-09-20T18:49:51.972970+0000","last_became_peered":"2024-09-20T18:49:51.972970+0000","last_unstale":"2024-09-20T19:06:08.092816+0000","last_undegraded":"2024-09-20T19:06:08.092816+0000","last_fullsized":"2024-09-20T19:06:08.092816+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:43.657780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:05:43.657780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:05.178691+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.6","version":"409'14794","reported_seq":24325,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.093216+0000","last_change":"2024-09-20T19:06:03.100013+0000","last_active":"2024-09-20T19:06:08.093216+0000","last_peered":"2024-09-20T19:06:08.093216+0000","last_clean":"2024-09-20T19:06:08.093216+0000","last_became_active":"2024-09-20T19:06:03.075507+0000","last_became_peered":"2024-09-20T19:06:03.075507+0000","last_unstale":"2024-09-20T19:06:08.093216+0000","last_undegraded":"2024-09-20T19:06:08.093216+0000","last_fullsized":"2024-09-20T19:06:08.093216+0000","mapping_epoch":394,"log_start":"376'9062","ondisk_log_start":"376'9062","created":325,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"407'14117","last_scrub_stamp":"2024-09-20T19:05:47.941785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:05:47.941785+0000","objects_scrubbed":682,"log_size":5732,"log_dups_size":0,"ondisk_log_size":5732,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-20T19:07:05.856206+0000","scrub_duration":191,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7397,"num_write_kb":236704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2323,"num_bytes_recovered":101875712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"273'27","reported_seq":867,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.093264+0000","last_change":"2024-09-20T19:05:49.777488+0000","last_active":"2024-09-20T19:06:08.093264+0000","last_peered":"2024-09-20T19:06:08.093264+0000","last_clean":"2024-09-20T19:06:08.093264+0000","last_became_active":"2024-09-20T18:59:05.083726+0000","last_became_peered":"2024-09-20T18:59:05.083726+0000","last_unstale":"2024-09-20T19:06:08.093264+0000","last_undegraded":"2024-09-20T19:06:08.093264+0000","last_fullsized":"2024-09-20T19:06:08.093264+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'27","last_scrub_stamp":"2024-09-20T19:05:49.777414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:49.777414+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:12.438786+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":828,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.099328+0000","last_change":"2024-09-20T19:04:46.691255+0000","last_active":"2024-09-20T19:06:06.099328+0000","last_peered":"2024-09-20T19:06:06.099328+0000","last_clean":"2024-09-20T19:06:06.099328+0000","last_became_active":"2024-09-20T19:04:20.884854+0000","last_became_peered":"2024-09-20T19:04:20.884854+0000","last_unstale":"2024-09-20T19:06:06.099328+0000","last_undegraded":"2024-09-20T19:06:06.099328+0000","last_fullsized":"2024-09-20T19:06:06.099328+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-20T19:04:46.691171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:04:46.691171+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"375'14","reported_seq":1009,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.098053+0000","last_change":"2024-09-20T19:04:39.564069+0000","last_active":"2024-09-20T19:06:06.098053+0000","last_peered":"2024-09-20T19:06:06.098053+0000","last_clean":"2024-09-20T19:06:06.098053+0000","last_became_active":"2024-09-20T19:04:20.892956+0000","last_became_peered":"2024-09-20T19:04:20.892956+0000","last_unstale":"2024-09-20T19:06:06.098053+0000","last_undegraded":"2024-09-20T19:06:06.098053+0000","last_fullsized":"2024-09-20T19:06:06.098053+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"375'14","last_scrub_stamp":"2024-09-20T19:04:39.563917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:39.563917+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":776,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.093932+0000","last_change":"2024-09-20T19:03:06.888620+0000","last_active":"2024-09-20T19:06:08.093932+0000","last_peered":"2024-09-20T19:06:08.093932+0000","last_clean":"2024-09-20T19:06:08.093932+0000","last_became_active":"2024-09-20T18:59:05.087005+0000","last_became_peered":"2024-09-20T18:59:05.087005+0000","last_unstale":"2024-09-20T19:06:08.093932+0000","last_undegraded":"2024-09-20T19:06:08.093932+0000","last_fullsized":"2024-09-20T19:06:08.093932+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:06.888563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:03:06.888563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.19","version":"409'14740","reported_seq":14948,"reported_epoch":416,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-20T19:06:06.239980+0000","last_change":"2024-09-20T19:06:06.239861+0000","last_active":"2024-09-20T19:06:06.239980+0000","last_peered":"2024-09-20T19:06:06.239980+0000","last_clean":"2024-09-20T19:05:57.257853+0000","last_became_active":"2024-09-20T19:06:06.211770+0000","last_became_peered":"2024-09-20T19:06:06.211770+0000","last_unstale":"2024-09-20T19:06:06.239980+0000","last_undegraded":"2024-09-20T19:06:06.239980+0000","last_fullsized":"2024-09-20T19:06:06.239980+0000","mapping_epoch":415,"log_start":"376'9027","ondisk_log_start":"376'9027","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":5,"last_scrub":"407'13951","last_scrub_stamp":"2024-09-20T19:05:46.982716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:05:46.982716+0000","objects_scrubbed":795,"log_size":5713,"log_dups_size":0,"ondisk_log_size":5713,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":218,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7370,"num_write_kb":235840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[2,5,7],"avail_no_missing":["2","1","3","4","5","7"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":333,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.069807+0000","last_change":"2024-09-20T19:01:16.542398+0000","last_active":"2024-09-20T19:06:06.069807+0000","last_peered":"2024-09-20T19:06:06.069807+0000","last_clean":"2024-09-20T19:06:06.069807+0000","last_became_active":"2024-09-20T18:59:05.085427+0000","last_became_peered":"2024-09-20T18:59:05.085427+0000","last_unstale":"2024-09-20T19:06:06.069807+0000","last_undegraded":"2024-09-20T19:06:06.069807+0000","last_fullsized":"2024-09-20T19:06:06.069807+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:16.542335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:16.542335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:29.408351+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"273'12","reported_seq":787,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.092580+0000","last_change":"2024-09-20T19:04:48.570718+0000","last_active":"2024-09-20T19:06:08.092580+0000","last_peered":"2024-09-20T19:06:08.092580+0000","last_clean":"2024-09-20T19:06:08.092580+0000","last_became_active":"2024-09-20T18:49:48.500662+0000","last_became_peered":"2024-09-20T18:49:48.500662+0000","last_unstale":"2024-09-20T19:06:08.092580+0000","last_undegraded":"2024-09-20T19:06:08.092580+0000","last_fullsized":"2024-09-20T19:06:08.092580+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'12","last_scrub_stamp":"2024-09-20T19:04:48.570639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:04:48.570639+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":765,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.093444+0000","last_change":"2024-09-20T19:05:22.605885+0000","last_active":"2024-09-20T19:06:08.093444+0000","last_peered":"2024-09-20T19:06:08.093444+0000","last_clean":"2024-09-20T19:06:08.093444+0000","last_became_active":"2024-09-20T18:49:49.956749+0000","last_became_peered":"2024-09-20T18:49:49.956749+0000","last_unstale":"2024-09-20T19:06:08.093444+0000","last_undegraded":"2024-09-20T19:06:08.093444+0000","last_fullsized":"2024-09-20T19:06:08.093444+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:22.605839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:05:22.605839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:49.241766+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":797,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.097889+0000","last_change":"2024-09-20T19:05:56.761494+0000","last_active":"2024-09-20T19:06:06.097889+0000","last_peered":"2024-09-20T19:06:06.097889+0000","last_clean":"2024-09-20T19:06:06.097889+0000","last_became_active":"2024-09-20T19:04:20.914115+0000","last_became_peered":"2024-09-20T19:04:20.914115+0000","last_unstale":"2024-09-20T19:06:06.097889+0000","last_undegraded":"2024-09-20T19:06:06.097889+0000","last_fullsized":"2024-09-20T19:06:06.097889+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:56.761425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:05:56.761425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:15.725837+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.18","version":"409'14426","reported_seq":14621,"reported_epoch":416,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-20T19:06:06.230480+0000","last_change":"2024-09-20T19:06:06.230424+0000","last_active":"2024-09-20T19:06:06.230480+0000","last_peered":"2024-09-20T19:06:06.230480+0000","last_clean":"2024-09-20T19:05:57.288785+0000","last_became_active":"2024-09-20T19:06:06.198017+0000","last_became_peered":"2024-09-20T19:06:06.198017+0000","last_unstale":"2024-09-20T19:06:06.230480+0000","last_undegraded":"2024-09-20T19:06:06.230480+0000","last_fullsized":"2024-09-20T19:06:06.230480+0000","mapping_epoch":415,"log_start":"376'8984","ondisk_log_start":"376'8984","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":5442,"log_dups_size":0,"ondisk_log_size":5442,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7213,"num_write_kb":230816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,0,7],"avail_no_missing":["2","0","1","3","7"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"273'15","reported_seq":777,"reported_epoch":417,"state":"active+clean","last_fresh":"2024-09-20T19:06:07.087536+0000","last_change":"2024-09-20T19:01:15.875779+0000","last_active":"2024-09-20T19:06:07.087536+0000","last_peered":"2024-09-20T19:06:07.087536+0000","last_clean":"2024-09-20T19:06:07.087536+0000","last_became_active":"2024-09-20T18:49:48.500803+0000","last_became_peered":"2024-09-20T18:49:48.500803+0000","last_unstale":"2024-09-20T19:06:07.087536+0000","last_undegraded":"2024-09-20T19:06:07.087536+0000","last_fullsized":"2024-09-20T19:06:07.087536+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:01:15.875693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:15.875693+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:58.535131+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":801,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.098075+0000","last_change":"2024-09-20T19:04:47.713764+0000","last_active":"2024-09-20T19:06:06.098075+0000","last_peered":"2024-09-20T19:06:06.098075+0000","last_clean":"2024-09-20T19:06:06.098075+0000","last_became_active":"2024-09-20T19:04:20.893250+0000","last_became_peered":"2024-09-20T19:04:20.893250+0000","last_unstale":"2024-09-20T19:06:06.098075+0000","last_undegraded":"2024-09-20T19:06:06.098075+0000","last_fullsized":"2024-09-20T19:06:06.098075+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:47.713585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:04:47.713585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":765,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.092495+0000","last_change":"2024-09-20T19:04:50.573619+0000","last_active":"2024-09-20T19:06:08.092495+0000","last_peered":"2024-09-20T19:06:08.092495+0000","last_clean":"2024-09-20T19:06:08.092495+0000","last_became_active":"2024-09-20T18:49:49.961947+0000","last_became_peered":"2024-09-20T18:49:49.961947+0000","last_unstale":"2024-09-20T19:06:08.092495+0000","last_undegraded":"2024-09-20T19:06:08.092495+0000","last_fullsized":"2024-09-20T19:06:08.092495+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:50.573550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:50.573550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"66'1","reported_seq":802,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.060930+0000","last_change":"2024-09-20T19:04:20.913449+0000","last_active":"2024-09-20T19:06:06.060930+0000","last_peered":"2024-09-20T19:06:06.060930+0000","last_clean":"2024-09-20T19:06:06.060930+0000","last_became_active":"2024-09-20T19:04:20.892236+0000","last_became_peered":"2024-09-20T19:04:20.892236+0000","last_unstale":"2024-09-20T19:06:06.060930+0000","last_undegraded":"2024-09-20T19:06:06.060930+0000","last_fullsized":"2024-09-20T19:06:06.060930+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T19:03:12.499684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:03:12.499684+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:37.501578+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"274'15","reported_seq":778,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.094581+0000","last_change":"2024-09-20T19:01:36.532486+0000","last_active":"2024-09-20T19:06:08.094581+0000","last_peered":"2024-09-20T19:06:08.094581+0000","last_clean":"2024-09-20T19:06:08.094581+0000","last_became_active":"2024-09-20T18:49:48.501011+0000","last_became_peered":"2024-09-20T18:49:48.501011+0000","last_unstale":"2024-09-20T19:06:08.094581+0000","last_undegraded":"2024-09-20T19:06:08.094581+0000","last_fullsized":"2024-09-20T19:06:08.094581+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'15","last_scrub_stamp":"2024-09-20T19:01:36.532419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:36.532419+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:38.766268+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":764,"reported_epoch":417,"state":"active+clean","last_fresh":"2024-09-20T19:06:07.087063+0000","last_change":"2024-09-20T19:01:30.864866+0000","last_active":"2024-09-20T19:06:07.087063+0000","last_peered":"2024-09-20T19:06:07.087063+0000","last_clean":"2024-09-20T19:06:07.087063+0000","last_became_active":"2024-09-20T18:49:45.938511+0000","last_became_peered":"2024-09-20T18:49:45.938511+0000","last_unstale":"2024-09-20T19:06:07.087063+0000","last_undegraded":"2024-09-20T19:06:07.087063+0000","last_fullsized":"2024-09-20T19:06:07.087063+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:30.864794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:30.864794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":970,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.092989+0000","last_change":"2024-09-20T19:04:41.601208+0000","last_active":"2024-09-20T19:06:08.092989+0000","last_peered":"2024-09-20T19:06:08.092989+0000","last_clean":"2024-09-20T19:06:08.092989+0000","last_became_active":"2024-09-20T18:49:49.960550+0000","last_became_peered":"2024-09-20T18:49:49.960550+0000","last_unstale":"2024-09-20T19:06:08.092989+0000","last_undegraded":"2024-09-20T19:06:08.092989+0000","last_fullsized":"2024-09-20T19:06:08.092989+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T19:04:41.601073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:41.601073+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":97,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.092295+0000","last_change":"2024-09-20T19:04:20.884397+0000","last_active":"2024-09-20T19:06:08.092295+0000","last_peered":"2024-09-20T19:06:08.092295+0000","last_clean":"2024-09-20T19:06:08.092295+0000","last_became_active":"2024-09-20T19:04:20.884100+0000","last_became_peered":"2024-09-20T19:04:20.884100+0000","last_unstale":"2024-09-20T19:06:08.092295+0000","last_undegraded":"2024-09-20T19:06:08.092295+0000","last_fullsized":"2024-09-20T19:06:08.092295+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:59.497810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:59:59.497810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:02.999076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"274'20","reported_seq":794,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.093127+0000","last_change":"2024-09-20T18:57:20.701592+0000","last_active":"2024-09-20T19:06:08.093127+0000","last_peered":"2024-09-20T19:06:08.093127+0000","last_clean":"2024-09-20T19:06:08.093127+0000","last_became_active":"2024-09-20T18:49:58.997494+0000","last_became_peered":"2024-09-20T18:49:58.997494+0000","last_unstale":"2024-09-20T19:06:08.093127+0000","last_undegraded":"2024-09-20T19:06:08.093127+0000","last_fullsized":"2024-09-20T19:06:08.093127+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:57:20.701543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:20.701543+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:23.577077+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":779,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.098932+0000","last_change":"2024-09-20T19:04:20.884940+0000","last_active":"2024-09-20T19:06:06.098932+0000","last_peered":"2024-09-20T19:06:06.098932+0000","last_clean":"2024-09-20T19:06:06.098932+0000","last_became_active":"2024-09-20T19:04:20.884582+0000","last_became_peered":"2024-09-20T19:04:20.884582+0000","last_unstale":"2024-09-20T19:06:06.098932+0000","last_undegraded":"2024-09-20T19:06:06.098932+0000","last_fullsized":"2024-09-20T19:06:06.098932+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.632380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.632380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:35.779165+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":962,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.094217+0000","last_change":"2024-09-20T19:01:10.601374+0000","last_active":"2024-09-20T19:06:08.094217+0000","last_peered":"2024-09-20T19:06:08.094217+0000","last_clean":"2024-09-20T19:06:08.094217+0000","last_became_active":"2024-09-20T18:49:49.962243+0000","last_became_peered":"2024-09-20T18:49:49.962243+0000","last_unstale":"2024-09-20T19:06:08.094217+0000","last_undegraded":"2024-09-20T19:06:08.094217+0000","last_fullsized":"2024-09-20T19:06:08.094217+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T19:01:10.601300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:10.601300+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:23.615982+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":768,"reported_epoch":417,"state":"active+clean","last_fresh":"2024-09-20T19:06:07.087342+0000","last_change":"2024-09-20T19:05:52.578042+0000","last_active":"2024-09-20T19:06:07.087342+0000","last_peered":"2024-09-20T19:06:07.087342+0000","last_clean":"2024-09-20T19:06:07.087342+0000","last_became_active":"2024-09-20T18:49:51.969093+0000","last_became_peered":"2024-09-20T18:49:51.969093+0000","last_unstale":"2024-09-20T19:06:07.087342+0000","last_undegraded":"2024-09-20T19:06:07.087342+0000","last_fullsized":"2024-09-20T19:06:07.087342+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:52.577979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:05:52.577979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:16.347110+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"273'19","reported_seq":109,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.090478+0000","last_change":"2024-09-20T19:04:20.970137+0000","last_active":"2024-09-20T19:06:08.090478+0000","last_peered":"2024-09-20T19:06:08.090478+0000","last_clean":"2024-09-20T19:06:08.090478+0000","last_became_active":"2024-09-20T19:04:20.897330+0000","last_became_peered":"2024-09-20T19:04:20.897330+0000","last_unstale":"2024-09-20T19:06:08.090478+0000","last_undegraded":"2024-09-20T19:06:08.090478+0000","last_fullsized":"2024-09-20T19:06:08.090478+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:57:50.899514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:50.899514+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:38.174326+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":97,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.092086+0000","last_change":"2024-09-20T19:04:20.896732+0000","last_active":"2024-09-20T19:06:08.092086+0000","last_peered":"2024-09-20T19:06:08.092086+0000","last_clean":"2024-09-20T19:06:08.092086+0000","last_became_active":"2024-09-20T19:04:20.896007+0000","last_became_peered":"2024-09-20T19:04:20.896007+0000","last_unstale":"2024-09-20T19:06:08.092086+0000","last_undegraded":"2024-09-20T19:06:08.092086+0000","last_fullsized":"2024-09-20T19:06:08.092086+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:06.310027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:59:06.310027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:02.046888+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":753,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.072457+0000","last_change":"2024-09-20T19:02:38.906352+0000","last_active":"2024-09-20T19:06:06.072457+0000","last_peered":"2024-09-20T19:06:06.072457+0000","last_clean":"2024-09-20T19:06:06.072457+0000","last_became_active":"2024-09-20T18:49:49.957067+0000","last_became_peered":"2024-09-20T18:49:49.957067+0000","last_unstale":"2024-09-20T19:06:06.072457+0000","last_undegraded":"2024-09-20T19:06:06.072457+0000","last_fullsized":"2024-09-20T19:06:06.072457+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:38.906161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:38.906161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:36.012448+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":738,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.092551+0000","last_change":"2024-09-20T18:57:16.789412+0000","last_active":"2024-09-20T19:06:08.092551+0000","last_peered":"2024-09-20T19:06:08.092551+0000","last_clean":"2024-09-20T19:06:08.092551+0000","last_became_active":"2024-09-20T18:49:51.966915+0000","last_became_peered":"2024-09-20T18:49:51.966915+0000","last_unstale":"2024-09-20T19:06:08.092551+0000","last_undegraded":"2024-09-20T19:06:08.092551+0000","last_fullsized":"2024-09-20T19:06:08.092551+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:16.789095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:16.789095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:35.677144+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"273'15","reported_seq":824,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.073899+0000","last_change":"2024-09-20T19:02:30.816061+0000","last_active":"2024-09-20T19:06:06.073899+0000","last_peered":"2024-09-20T19:06:06.073899+0000","last_clean":"2024-09-20T19:06:06.073899+0000","last_became_active":"2024-09-20T18:59:05.085230+0000","last_became_peered":"2024-09-20T18:59:05.085230+0000","last_unstale":"2024-09-20T19:06:06.073899+0000","last_undegraded":"2024-09-20T19:06:06.073899+0000","last_fullsized":"2024-09-20T19:06:06.073899+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:02:30.815747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:30.815747+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:39.985336+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":779,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.098152+0000","last_change":"2024-09-20T19:04:20.885127+0000","last_active":"2024-09-20T19:06:06.098152+0000","last_peered":"2024-09-20T19:06:06.098152+0000","last_clean":"2024-09-20T19:06:06.098152+0000","last_became_active":"2024-09-20T19:04:20.884660+0000","last_became_peered":"2024-09-20T19:04:20.884660+0000","last_unstale":"2024-09-20T19:06:06.098152+0000","last_undegraded":"2024-09-20T19:06:06.098152+0000","last_fullsized":"2024-09-20T19:06:06.098152+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:07.408197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:03:07.408197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":765,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.092913+0000","last_change":"2024-09-20T19:04:44.606497+0000","last_active":"2024-09-20T19:06:08.092913+0000","last_peered":"2024-09-20T19:06:08.092913+0000","last_clean":"2024-09-20T19:06:08.092913+0000","last_became_active":"2024-09-20T18:49:49.962523+0000","last_became_peered":"2024-09-20T18:49:49.962523+0000","last_unstale":"2024-09-20T19:06:08.092913+0000","last_undegraded":"2024-09-20T19:06:08.092913+0000","last_fullsized":"2024-09-20T19:06:08.092913+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:44.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:44.606343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":760,"reported_epoch":417,"state":"active+clean","last_fresh":"2024-09-20T19:06:07.086161+0000","last_change":"2024-09-20T19:05:51.626774+0000","last_active":"2024-09-20T19:06:07.086161+0000","last_peered":"2024-09-20T19:06:07.086161+0000","last_clean":"2024-09-20T19:06:07.086161+0000","last_became_active":"2024-09-20T18:49:51.967313+0000","last_became_peered":"2024-09-20T18:49:51.967313+0000","last_unstale":"2024-09-20T19:06:07.086161+0000","last_undegraded":"2024-09-20T19:06:07.086161+0000","last_fullsized":"2024-09-20T19:06:07.086161+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:51.626697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:05:51.626697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:13.789020+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"274'19","reported_seq":831,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.092590+0000","last_change":"2024-09-20T19:01:05.495603+0000","last_active":"2024-09-20T19:06:08.092590+0000","last_peered":"2024-09-20T19:06:08.092590+0000","last_clean":"2024-09-20T19:06:08.092590+0000","last_became_active":"2024-09-20T18:59:05.075390+0000","last_became_peered":"2024-09-20T18:59:05.075390+0000","last_unstale":"2024-09-20T19:06:08.092590+0000","last_undegraded":"2024-09-20T19:06:08.092590+0000","last_fullsized":"2024-09-20T19:06:08.092590+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'19","last_scrub_stamp":"2024-09-20T19:01:05.495478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:05.495478+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:04.804811+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":781,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.093526+0000","last_change":"2024-09-20T19:05:43.686902+0000","last_active":"2024-09-20T19:06:08.093526+0000","last_peered":"2024-09-20T19:06:08.093526+0000","last_clean":"2024-09-20T19:06:08.093526+0000","last_became_active":"2024-09-20T18:49:45.942105+0000","last_became_peered":"2024-09-20T18:49:45.942105+0000","last_unstale":"2024-09-20T19:06:08.093526+0000","last_undegraded":"2024-09-20T19:06:08.093526+0000","last_fullsized":"2024-09-20T19:06:08.093526+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:43.686842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:43.686842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:01.625104+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":793,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.139808+0000","last_change":"2024-09-20T19:04:20.912120+0000","last_active":"2024-09-20T19:06:06.139808+0000","last_peered":"2024-09-20T19:06:06.139808+0000","last_clean":"2024-09-20T19:06:06.139808+0000","last_became_active":"2024-09-20T19:04:20.911292+0000","last_became_peered":"2024-09-20T19:04:20.911292+0000","last_unstale":"2024-09-20T19:06:06.139808+0000","last_undegraded":"2024-09-20T19:06:06.139808+0000","last_fullsized":"2024-09-20T19:06:06.139808+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:27.267254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:27.267254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:59.781503+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":767,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.093008+0000","last_change":"2024-09-20T19:05:51.714712+0000","last_active":"2024-09-20T19:06:08.093008+0000","last_peered":"2024-09-20T19:06:08.093008+0000","last_clean":"2024-09-20T19:06:08.093008+0000","last_became_active":"2024-09-20T18:59:05.081179+0000","last_became_peered":"2024-09-20T18:59:05.081179+0000","last_unstale":"2024-09-20T19:06:08.093008+0000","last_undegraded":"2024-09-20T19:06:08.093008+0000","last_fullsized":"2024-09-20T19:06:08.093008+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:51.714641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:05:51.714641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:58.160366+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"273'8","reported_seq":808,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.092236+0000","last_change":"2024-09-20T19:04:40.559947+0000","last_active":"2024-09-20T19:06:08.092236+0000","last_peered":"2024-09-20T19:06:08.092236+0000","last_clean":"2024-09-20T19:06:08.092236+0000","last_became_active":"2024-09-20T18:59:05.076365+0000","last_became_peered":"2024-09-20T18:59:05.076365+0000","last_unstale":"2024-09-20T19:06:08.092236+0000","last_undegraded":"2024-09-20T19:06:08.092236+0000","last_fullsized":"2024-09-20T19:06:08.092236+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'8","last_scrub_stamp":"2024-09-20T19:04:40.559877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:04:40.559877+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":772,"reported_epoch":417,"state":"active+clean","last_fresh":"2024-09-20T19:06:07.087752+0000","last_change":"2024-09-20T19:02:39.184249+0000","last_active":"2024-09-20T19:06:07.087752+0000","last_peered":"2024-09-20T19:06:07.087752+0000","last_clean":"2024-09-20T19:06:07.087752+0000","last_became_active":"2024-09-20T18:49:45.940988+0000","last_became_peered":"2024-09-20T18:49:45.940988+0000","last_unstale":"2024-09-20T19:06:07.087752+0000","last_undegraded":"2024-09-20T19:06:07.087752+0000","last_fullsized":"2024-09-20T19:06:07.087752+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:39.184150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:39.184150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":358,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.069184+0000","last_change":"2024-09-20T19:04:20.900567+0000","last_active":"2024-09-20T19:06:06.069184+0000","last_peered":"2024-09-20T19:06:06.069184+0000","last_clean":"2024-09-20T19:06:06.069184+0000","last_became_active":"2024-09-20T19:04:20.900214+0000","last_became_peered":"2024-09-20T19:04:20.900214+0000","last_unstale":"2024-09-20T19:06:06.069184+0000","last_undegraded":"2024-09-20T19:06:06.069184+0000","last_fullsized":"2024-09-20T19:06:06.069184+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:41.497115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:41.497115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:33.282430+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":775,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.097594+0000","last_change":"2024-09-20T19:05:57.789513+0000","last_active":"2024-09-20T19:06:06.097594+0000","last_peered":"2024-09-20T19:06:06.097594+0000","last_clean":"2024-09-20T19:06:06.097594+0000","last_became_active":"2024-09-20T19:04:20.885211+0000","last_became_peered":"2024-09-20T19:04:20.885211+0000","last_unstale":"2024-09-20T19:06:06.097594+0000","last_undegraded":"2024-09-20T19:06:06.097594+0000","last_fullsized":"2024-09-20T19:06:06.097594+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:57.789465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:05:57.789465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:08.966824+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"409'14466","reported_seq":32127,"reported_epoch":417,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-20T19:06:07.085834+0000","last_change":"2024-09-20T19:06:06.161986+0000","last_active":"2024-09-20T19:06:07.085834+0000","last_peered":"2024-09-20T19:06:07.085834+0000","last_clean":"2024-09-20T19:05:57.287699+0000","last_became_active":"2024-09-20T19:06:06.131059+0000","last_became_peered":"2024-09-20T19:06:06.131059+0000","last_unstale":"2024-09-20T19:06:07.085834+0000","last_undegraded":"2024-09-20T19:06:07.085834+0000","last_fullsized":"2024-09-20T19:06:07.085834+0000","mapping_epoch":415,"log_start":"378'9916","ondisk_log_start":"378'9916","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:02:10.811693+0000","last_clean_scrub_stamp":"2024-09-20T19:02:10.811693+0000","objects_scrubbed":0,"log_size":4550,"log_dups_size":0,"ondisk_log_size":4550,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7233,"num_write_kb":231456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4358,"num_bytes_recovered":285540352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[0,7,4],"avail_no_missing":["0","1","4","5","7"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":750,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.070231+0000","last_change":"2024-09-20T19:04:20.890533+0000","last_active":"2024-09-20T19:06:06.070231+0000","last_peered":"2024-09-20T19:06:06.070231+0000","last_clean":"2024-09-20T19:06:06.070231+0000","last_became_active":"2024-09-20T19:04:20.890101+0000","last_became_peered":"2024-09-20T19:04:20.890101+0000","last_unstale":"2024-09-20T19:06:06.070231+0000","last_undegraded":"2024-09-20T19:06:06.070231+0000","last_fullsized":"2024-09-20T19:06:06.070231+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:10.970779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:10.970779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:24.205999+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":757,"reported_epoch":418,"state":"active+clean","last_fresh":"2024-09-20T19:06:08.092359+0000","last_change":"2024-09-20T18:58:36.077005+0000","last_active":"2024-09-20T19:06:08.092359+0000","last_peered":"2024-09-20T19:06:08.092359+0000","last_clean":"2024-09-20T19:06:08.092359+0000","last_became_active":"2024-09-20T18:49:45.937992+0000","last_became_peered":"2024-09-20T18:49:45.937992+0000","last_unstale":"2024-09-20T19:06:08.092359+0000","last_undegraded":"2024-09-20T19:06:08.092359+0000","last_fullsized":"2024-09-20T19:06:08.092359+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:36.076937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:36.076937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:42.609713+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"274'19","reported_seq":357,"reported_epoch":416,"state":"active+clean","last_fresh":"2024-09-20T19:06:06.070578+0000","last_change":"2024-09-20T19:01:15.544760+0000","last_active":"2024-09-20T19:06:06.070578+0000","last_peered":"2024-09-20T19:06:06.070578+0000","last_clean":"2024-09-20T19:06:06.070578+0000","last_became_active":"2024-09-20T18:59:05.089294+0000","last_became_peered":"2024-09-20T18:59:05.089294+0000","last_unstale":"2024-09-20T19:06:06.070578+0000","last_undegraded":"2024-09-20T19:06:06.070578+0000","last_fullsized":"2024-09-20T19:06:06.070578+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'19","last_scrub_stamp":"2024-09-20T19:01:15.544663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:15.544663+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:24.387863+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":750,"reported_epoch":417,"state":"active+clean","last_fresh":"2024-09-20T19:06:07.087408+0000","last_change":"2024-09-20T19:01:40.961449+0000","last_active":"2024-09-20T19:06:07.087408+0000","last_peered":"2024-09-20T19:06:07.087408+0000","last_clean":"2024-09-20T19:06:07.087408+0000","last_became_active":"2024-09-20T18:49:49.957582+0000","last_became_peered":"2024-09-20T18:49:49.957582+0000","last_unstale":"2024-09-20T19:06:07.087408+0000","last_undegraded":"2024-09-20T19:06:07.087408+0000","last_fullsized":"2024-09-20T19:06:07.087408+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:40.961363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:40.961363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":26,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":226323,"num_write_kb":7242273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30106,"num_bytes_recovered":1712685080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":58916864,"data_stored":58916864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142496,"ondisk_log_size":142496,"up":75,"acting":75,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1185,"internal_metadata":0},"log_size":87,"ondisk_log_size":87,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1114,"num_read_kb":871,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":257,"num_bytes_recovered":4890,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":686,"ondisk_log_size":686,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":3105,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":973,"num_read_kb":968,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266888,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":578488,"kb_used_data":1696,"kb_used_omap":1,"kb_used_meta":576766,"kb_avail":93166664,"statfs":{"total":95995035648,"available":95402663936,"internally_reserved":0,"allocated":1736704,"data_stored":1207777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":590608842},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528527,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":425764,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":424062,"kb_avail":93319388,"statfs":{"total":95995035648,"available":95559053312,"internally_reserved":0,"allocated":1716224,"data_stored":1207115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":434239942},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757461,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":591828,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":590654,"kb_avail":93153324,"statfs":{"total":95995035648,"available":95389003776,"internally_reserved":0,"allocated":1179648,"data_stored":639331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":604830149},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":304,"seq":1305670058043,"num_pgs":72,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298088,"kb_used_data":1168,"kb_used_omap":5,"kb_used_meta":296890,"kb_avail":93447064,"statfs":{"total":95995035648,"available":95689793536,"internally_reserved":0,"allocated":1196032,"data_stored":640225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5633,"internal_metadata":304015871},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182625,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":726480,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":725310,"kb_avail":93018672,"statfs":{"total":95995035648,"available":95251120128,"internally_reserved":0,"allocated":1175552,"data_stored":638711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":742717895},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411557,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":609840,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":608702,"kb_avail":93135312,"statfs":{"total":95995035648,"available":95370559488,"internally_reserved":0,"allocated":1142784,"data_stored":625027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":623311301},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575083,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":697956,"kb_used_data":58700,"kb_used_omap":1,"kb_used_meta":639230,"kb_avail":93047196,"statfs":{"total":95995035648,"available":95280328704,"internally_reserved":0,"allocated":60108800,"data_stored":59543075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":654571973},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738609,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":686804,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":685054,"kb_avail":93058348,"statfs":{"total":95995035648,"available":95291748352,"internally_reserved":0,"allocated":1761280,"data_stored":1222113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":701495749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1185,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":58916864,"data_stored":58916864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T19:06:13.382 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:13.404 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:13.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:13.405 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:13.505 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:06:13.527 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:13.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:13.528 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:06:13.629 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:13.650 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:13.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:13.651 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:13.752 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:13.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:13.774 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:13.774 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:13.875 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:13.896 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:13.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:13.897 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:13.953 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:13 smithi083 ceph-mon[29421]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:06:13.953 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:13 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T19:06:13.953 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:13 smithi083 ceph-mon[29421]: osdmap e421: 8 total, 8 up, 8 in 2024-09-20T19:06:13.953 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:13 smithi083 ceph-mon[29421]: pgmap v829: 132 pgs: 132 active+clean; 583 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 807 B/s rd, 0 op/s 2024-09-20T19:06:13.953 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:13 smithi083 ceph-mon[29421]: 2.1 scrub starts 2024-09-20T19:06:13.953 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:13 smithi083 ceph-mon[29421]: 2.1 scrub ok 2024-09-20T19:06:13.953 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:13 smithi083 ceph-mon[36260]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:06:13.953 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:13 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T19:06:13.954 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:13 smithi083 ceph-mon[36260]: osdmap e421: 8 total, 8 up, 8 in 2024-09-20T19:06:13.954 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:13 smithi083 ceph-mon[36260]: pgmap v829: 132 pgs: 132 active+clean; 583 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 807 B/s rd, 0 op/s 2024-09-20T19:06:13.954 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:13 smithi083 ceph-mon[36260]: 2.1 scrub starts 2024-09-20T19:06:13.954 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:13 smithi083 ceph-mon[36260]: 2.1 scrub ok 2024-09-20T19:06:13.989 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:06:13.992 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:06:13.998 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:14.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:14.026 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:14.026 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:14.127 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:06:14.148 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:14.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:14.150 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:06:14.250 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:14.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:13 smithi142 ceph-mon[32508]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:06:14.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:13 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T19:06:14.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:13 smithi142 ceph-mon[32508]: osdmap e421: 8 total, 8 up, 8 in 2024-09-20T19:06:14.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:13 smithi142 ceph-mon[32508]: pgmap v829: 132 pgs: 132 active+clean; 583 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 807 B/s rd, 0 op/s 2024-09-20T19:06:14.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:13 smithi142 ceph-mon[32508]: 2.1 scrub starts 2024-09-20T19:06:14.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:13 smithi142 ceph-mon[32508]: 2.1 scrub ok 2024-09-20T19:06:14.273 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:14.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:14.278 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:14.379 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:14.405 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:14.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:14.406 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:14.507 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:06:14.527 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:14.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:14.528 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:06:14.629 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:06:14.650 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:14.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:14.651 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:06:14.689 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T19:06:14.689 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:06:14.694 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:06:14.752 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:14.774 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:14.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:14.775 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:14.876 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:14.901 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:14.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:14.902 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:15.003 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:15.028 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:15.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:15.029 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:15.129 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:15.150 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:15.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:15.151 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:15.252 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:15.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:14 smithi142 ceph-mon[32508]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 7 pgs backfill_toofull) 2024-09-20T19:06:15.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:14 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:06:15.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:14 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:06:15.276 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:15.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:15.277 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:15.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:14 smithi083 ceph-mon[29421]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 7 pgs backfill_toofull) 2024-09-20T19:06:15.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:14 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:06:15.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:14 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:06:15.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:14 smithi083 ceph-mon[36260]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 7 pgs backfill_toofull) 2024-09-20T19:06:15.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:14 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:06:15.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:14 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:06:15.378 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:15.403 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:15.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:15.404 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:15.418 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_historic_ops 2024-09-20T19:06:15.505 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:15.528 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:15.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:15.529 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:15.630 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:15.653 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:15.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:15.654 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:15.755 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:15.778 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:15.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:15.779 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:15.879 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:15.901 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:15.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:15.902 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:15.981 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T19:06:16.004 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:16.025 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:16.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:16.026 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:16.030 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:15 smithi083 ceph-mon[29421]: 5.9 scrub starts 2024-09-20T19:06:16.030 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:15 smithi083 ceph-mon[29421]: 5.9 scrub ok 2024-09-20T19:06:16.030 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:15 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T19:06:16.030 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:15 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T19:06:16.030 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:15 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:06:16.030 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:15 smithi083 ceph-mon[29421]: pgmap v830: 132 pgs: 132 active+clean; 583 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-20T19:06:16.030 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:15 smithi083 ceph-mon[36260]: 5.9 scrub starts 2024-09-20T19:06:16.030 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:15 smithi083 ceph-mon[36260]: 5.9 scrub ok 2024-09-20T19:06:16.030 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:15 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T19:06:16.030 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:15 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T19:06:16.031 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:15 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:06:16.031 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:15 smithi083 ceph-mon[36260]: pgmap v830: 132 pgs: 132 active+clean; 583 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-20T19:06:16.071 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:06:16.071 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:06:16.078 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:06:16.091 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:06:16.091 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:06:16.114 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:06:16.114 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:06:16.127 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:16.132 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:06:16.132 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:06:16.148 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:16.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:16.149 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:16.149 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:06:16.149 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:06:16.166 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:06:16.167 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:06:16.183 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:06:16.184 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:06:16.201 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:06:16.201 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:06:16.249 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:16.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:15 smithi142 ceph-mon[32508]: 5.9 scrub starts 2024-09-20T19:06:16.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:15 smithi142 ceph-mon[32508]: 5.9 scrub ok 2024-09-20T19:06:16.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:15 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T19:06:16.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:15 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T19:06:16.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:15 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:06:16.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:15 smithi142 ceph-mon[32508]: pgmap v830: 132 pgs: 132 active+clean; 583 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-20T19:06:16.271 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:16.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:16.272 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:16.373 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:16.395 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:16.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:16.396 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:16.497 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:16.519 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:16.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:16.520 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:16.621 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:16.642 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:16.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:16.644 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:16.744 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:16.767 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:16.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:16.768 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:16.868 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:16.891 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:16.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:16.892 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:16.992 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:17.014 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:17.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:17.016 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:17.041 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'unique_pool_4' created 2024-09-20T19:06:17.117 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:17.139 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:17.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:17.140 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:17.180 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:16 smithi083 ceph-mon[29421]: 3.3 scrub starts 2024-09-20T19:06:17.181 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:16 smithi083 ceph-mon[29421]: 3.3 scrub ok 2024-09-20T19:06:17.181 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:16 smithi083 ceph-mon[29421]: 6.0 scrub starts 2024-09-20T19:06:17.181 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:16 smithi083 ceph-mon[29421]: 6.0 scrub ok 2024-09-20T19:06:17.181 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:16 smithi083 ceph-mon[29421]: 3.e scrub starts 2024-09-20T19:06:17.181 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:16 smithi083 ceph-mon[29421]: 3.e scrub ok 2024-09-20T19:06:17.181 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:16 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1567882367' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:06:17.181 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:16 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1806455295' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16} : dispatch 2024-09-20T19:06:17.181 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:16 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:06:17.181 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:16 smithi083 ceph-mon[36260]: 3.3 scrub starts 2024-09-20T19:06:17.182 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:16 smithi083 ceph-mon[36260]: 3.3 scrub ok 2024-09-20T19:06:17.182 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:16 smithi083 ceph-mon[36260]: 6.0 scrub starts 2024-09-20T19:06:17.182 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:16 smithi083 ceph-mon[36260]: 6.0 scrub ok 2024-09-20T19:06:17.182 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:16 smithi083 ceph-mon[36260]: 3.e scrub starts 2024-09-20T19:06:17.182 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:16 smithi083 ceph-mon[36260]: 3.e scrub ok 2024-09-20T19:06:17.182 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:16 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1567882367' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:06:17.182 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:16 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1806455295' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16} : dispatch 2024-09-20T19:06:17.182 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:16 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:06:17.206 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T19:06:17.207 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-20T19:06:17.207 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-20T19:06:17.207 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-20T19:06:17.207 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.207 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.24955.0:218173 8.f 8:fc626e9a:::benchmark_data_smithi083_97422_object102082:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-20T19:06:17.207 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:58:01.879894+0000", 2024-09-20T19:06:17.208 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 495.32059515700001, 2024-09-20T19:06:17.208 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.22879112400000001, 2024-09-20T19:06:17.208 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:06:17.208 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:06:17.208 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:17.208 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:06:17.209 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.24955", 2024-09-20T19:06:17.209 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2023720115", 2024-09-20T19:06:17.209 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 218173 2024-09-20T19:06:17.209 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.209 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:06:17.210 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.210 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:06:17.210 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.879894+0000", 2024-09-20T19:06:17.210 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:17.210 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.210 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.210 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:06:17.211 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.879894+0000", 2024-09-20T19:06:17.211 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:17.211 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.211 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.211 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:06:17.211 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.879895+0000", 2024-09-20T19:06:17.212 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3600000000000003e-07 2024-09-20T19:06:17.212 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.212 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.212 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:06:17.212 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.879899+0000", 2024-09-20T19:06:17.212 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3500000000000001e-06 2024-09-20T19:06:17.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.213 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.213 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:06:17.213 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.879907+0000", 2024-09-20T19:06:17.213 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.943e-06 2024-09-20T19:06:17.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.213 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.213 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:06:17.213 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:01.879913+0000", 2024-09-20T19:06:17.213 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3910000000000003e-06 2024-09-20T19:06:17.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.213 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.213 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:17.214 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.095128+0000", 2024-09-20T19:06:17.214 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.21521499199999999 2024-09-20T19:06:17.214 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.214 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.214 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:06:17.214 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.095243+0000", 2024-09-20T19:06:17.214 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011465599999999999 2024-09-20T19:06:17.214 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.214 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.214 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:06:17.214 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.095357+0000", 2024-09-20T19:06:17.214 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000114868 2024-09-20T19:06:17.215 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.215 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.215 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:17.215 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.104639+0000", 2024-09-20T19:06:17.215 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0092811200000000003 2024-09-20T19:06:17.215 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.215 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.215 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:06:17.215 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.105329+0000", 2024-09-20T19:06:17.215 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000690195 2024-09-20T19:06:17.215 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.215 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.215 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:17.216 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.108593+0000", 2024-09-20T19:06:17.216 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003263967 2024-09-20T19:06:17.216 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.216 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.216 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:06:17.216 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.108646+0000", 2024-09-20T19:06:17.216 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3495999999999997e-05 2024-09-20T19:06:17.216 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.216 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.216 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:06:17.216 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:58:02.108686+0000", 2024-09-20T19:06:17.216 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9409999999999997e-05 2024-09-20T19:06:17.216 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:17.216 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:17.217 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:17.217 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.217 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.217 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:50423 9.1 9:8787b2bd:::benchmark_data_smithi083_124552_object50422:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-20T19:06:17.217 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:59:04.093296+0000", 2024-09-20T19:06:17.217 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 433.10719389799999, 2024-09-20T19:06:17.217 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.136410235, 2024-09-20T19:06:17.217 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:06:17.217 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:06:17.217 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:17.217 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:06:17.217 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:06:17.217 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:06:17.217 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 50423 2024-09-20T19:06:17.218 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.218 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:06:17.218 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.218 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:06:17.218 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.093296+0000", 2024-09-20T19:06:17.218 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:17.218 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.218 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.218 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:06:17.218 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.093296+0000", 2024-09-20T19:06:17.218 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:17.218 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.218 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.219 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:06:17.219 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.093297+0000", 2024-09-20T19:06:17.219 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.37e-06 2024-09-20T19:06:17.219 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.219 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.219 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:06:17.219 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.093316+0000", 2024-09-20T19:06:17.219 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8660000000000001e-05 2024-09-20T19:06:17.219 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.219 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.219 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:06:17.219 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.093342+0000", 2024-09-20T19:06:17.219 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6103999999999999e-05 2024-09-20T19:06:17.219 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.220 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.220 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:06:17.220 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.093347+0000", 2024-09-20T19:06:17.220 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0490000000000003e-06 2024-09-20T19:06:17.220 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.220 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.220 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:17.220 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.093482+0000", 2024-09-20T19:06:17.220 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000135449 2024-09-20T19:06:17.220 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.220 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.220 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:06:17.221 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.093490+0000", 2024-09-20T19:06:17.221 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0469999999999994e-06 2024-09-20T19:06:17.221 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.221 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.221 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:17.221 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.067142+0000", 2024-09-20T19:06:17.221 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97365191799999995 2024-09-20T19:06:17.221 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.221 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.221 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:06:17.221 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.067153+0000", 2024-09-20T19:06:17.221 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0559e-05 2024-09-20T19:06:17.221 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.222 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.222 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:17.222 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.215644+0000", 2024-09-20T19:06:17.222 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.148490633 2024-09-20T19:06:17.222 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.222 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.222 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:06:17.222 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.215772+0000", 2024-09-20T19:06:17.222 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012840799999999999 2024-09-20T19:06:17.222 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.222 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.222 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 6,7", 2024-09-20T19:06:17.222 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.215927+0000", 2024-09-20T19:06:17.223 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000155315 2024-09-20T19:06:17.223 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.223 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.223 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:17.223 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.223137+0000", 2024-09-20T19:06:17.223 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0072102049999999999 2024-09-20T19:06:17.223 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.223 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.223 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:17.223 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.223182+0000", 2024-09-20T19:06:17.223 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4158e-05 2024-09-20T19:06:17.223 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.223 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.223 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:06:17.224 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.227203+0000", 2024-09-20T19:06:17.224 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0040211530000000004 2024-09-20T19:06:17.224 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.224 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.224 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:06:17.224 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.227258+0000", 2024-09-20T19:06:17.224 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5050999999999999e-05 2024-09-20T19:06:17.224 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.224 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.224 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:06:17.224 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.229706+0000", 2024-09-20T19:06:17.224 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0024481559999999999 2024-09-20T19:06:17.224 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:17.224 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:17.225 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:17.225 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.225 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.225 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:50474 9.1 9:8f4be195:::benchmark_data_smithi083_124552_object50473:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-20T19:06:17.225 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:59:04.131531+0000", 2024-09-20T19:06:17.225 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 433.06895898099998, 2024-09-20T19:06:17.225 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.101374029, 2024-09-20T19:06:17.225 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:06:17.225 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:06:17.225 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:17.225 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:06:17.225 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:06:17.226 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:06:17.226 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 50474 2024-09-20T19:06:17.226 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.226 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:06:17.226 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.226 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:06:17.226 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.131531+0000", 2024-09-20T19:06:17.226 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:17.226 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.226 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.226 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:06:17.226 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.131531+0000", 2024-09-20T19:06:17.227 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:17.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.227 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.227 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:06:17.227 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.131532+0000", 2024-09-20T19:06:17.227 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5489999999999999e-06 2024-09-20T19:06:17.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.227 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.227 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:06:17.227 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.131552+0000", 2024-09-20T19:06:17.228 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9630999999999999e-05 2024-09-20T19:06:17.228 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.228 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.228 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:06:17.228 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.131568+0000", 2024-09-20T19:06:17.228 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6606999999999999e-05 2024-09-20T19:06:17.228 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.228 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.228 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:06:17.228 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.131577+0000", 2024-09-20T19:06:17.229 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7879999999999995e-06 2024-09-20T19:06:17.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.229 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.229 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:17.229 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.131644+0000", 2024-09-20T19:06:17.229 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7180000000000004e-05 2024-09-20T19:06:17.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.229 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.229 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:06:17.229 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.131654+0000", 2024-09-20T19:06:17.229 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2390000000000008e-06 2024-09-20T19:06:17.230 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.230 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.230 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:17.230 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.067311+0000", 2024-09-20T19:06:17.230 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.935657501 2024-09-20T19:06:17.230 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.230 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.230 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:06:17.230 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.067320+0000", 2024-09-20T19:06:17.230 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9779999999999994e-06 2024-09-20T19:06:17.230 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.230 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.231 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:17.231 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.220228+0000", 2024-09-20T19:06:17.231 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1529083090000001 2024-09-20T19:06:17.231 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.231 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.231 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:06:17.231 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.220362+0000", 2024-09-20T19:06:17.231 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013401 2024-09-20T19:06:17.231 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.231 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.231 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 6,7", 2024-09-20T19:06:17.232 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.220523+0000", 2024-09-20T19:06:17.232 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000160247 2024-09-20T19:06:17.232 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.232 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.232 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:06:17.232 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.229718+0000", 2024-09-20T19:06:17.232 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0091949860000000005 2024-09-20T19:06:17.232 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.232 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.232 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:17.232 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.229797+0000", 2024-09-20T19:06:17.232 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9443999999999994e-05 2024-09-20T19:06:17.232 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.232 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.233 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:17.233 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.230036+0000", 2024-09-20T19:06:17.233 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000239072 2024-09-20T19:06:17.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.233 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.233 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:06:17.233 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.230111+0000", 2024-09-20T19:06:17.233 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4411000000000003e-05 2024-09-20T19:06:17.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.233 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.233 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:06:17.233 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.232905+0000", 2024-09-20T19:06:17.233 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0027940769999999998 2024-09-20T19:06:17.234 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:17.234 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:17.234 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:17.234 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.234 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.234 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:50506 9.1 9:80ae6680:::benchmark_data_smithi083_124552_object50505:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-20T19:06:17.234 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:59:04.155840+0000", 2024-09-20T19:06:17.234 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 433.04464917799999, 2024-09-20T19:06:17.234 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.082518146, 2024-09-20T19:06:17.234 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:06:17.234 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:06:17.234 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:17.234 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:06:17.235 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:06:17.235 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:06:17.235 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 50506 2024-09-20T19:06:17.235 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.235 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:06:17.235 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.235 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:06:17.235 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.155840+0000", 2024-09-20T19:06:17.235 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:17.235 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.235 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.235 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:06:17.235 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.155840+0000", 2024-09-20T19:06:17.235 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:17.236 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.236 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.236 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:06:17.236 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.155842+0000", 2024-09-20T19:06:17.236 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1090000000000001e-06 2024-09-20T19:06:17.236 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.236 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.236 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:06:17.236 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.155869+0000", 2024-09-20T19:06:17.236 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7801999999999999e-05 2024-09-20T19:06:17.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.237 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.237 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:06:17.237 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.155880+0000", 2024-09-20T19:06:17.237 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1059e-05 2024-09-20T19:06:17.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.237 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.237 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:06:17.237 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.155888+0000", 2024-09-20T19:06:17.237 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7559999999999999e-06 2024-09-20T19:06:17.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.237 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.237 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:17.237 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.155955+0000", 2024-09-20T19:06:17.237 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7022999999999998e-05 2024-09-20T19:06:17.238 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.238 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.238 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:06:17.238 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.155965+0000", 2024-09-20T19:06:17.238 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3710000000000006e-06 2024-09-20T19:06:17.238 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.238 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.238 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:17.238 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.067356+0000", 2024-09-20T19:06:17.238 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.91139177299999996 2024-09-20T19:06:17.238 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.238 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.238 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:06:17.238 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.067364+0000", 2024-09-20T19:06:17.239 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.6000000000000001e-06 2024-09-20T19:06:17.239 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.239 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.239 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:17.239 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.223316+0000", 2024-09-20T19:06:17.239 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1559515810000001 2024-09-20T19:06:17.239 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.239 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.239 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:06:17.239 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.223488+0000", 2024-09-20T19:06:17.239 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017242399999999999 2024-09-20T19:06:17.241 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.241 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.241 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 6,7", 2024-09-20T19:06:17.241 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.223628+0000", 2024-09-20T19:06:17.241 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013994100000000001 2024-09-20T19:06:17.241 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.241 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.242 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:17.242 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.229891+0000", 2024-09-20T19:06:17.242 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0062633239999999998 2024-09-20T19:06:17.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.242 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.242 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:06:17.242 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.229911+0000", 2024-09-20T19:06:17.242 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9910000000000001e-05 2024-09-20T19:06:17.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.242 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.242 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:17.242 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.235269+0000", 2024-09-20T19:06:17.242 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0053577429999999999 2024-09-20T19:06:17.243 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.243 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.243 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:06:17.243 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.235358+0000", 2024-09-20T19:06:17.243 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9536999999999993e-05 2024-09-20T19:06:17.243 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.243 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.243 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:06:17.243 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.238359+0000", 2024-09-20T19:06:17.243 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003000193 2024-09-20T19:06:17.243 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:17.243 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:17.243 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:17.243 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.244 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.244 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:50517 9.1 9:8bbc6b9c:::benchmark_data_smithi083_124552_object50516:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-20T19:06:17.244 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T18:59:04.167833+0000", 2024-09-20T19:06:17.244 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 433.03265674099998, 2024-09-20T19:06:17.244 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0735188230000001, 2024-09-20T19:06:17.244 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:06:17.244 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:06:17.244 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:17.244 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:06:17.244 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:06:17.244 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:06:17.244 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 50517 2024-09-20T19:06:17.244 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.244 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:06:17.245 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.245 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:06:17.245 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.167833+0000", 2024-09-20T19:06:17.245 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:17.245 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.245 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.245 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:06:17.245 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.167833+0000", 2024-09-20T19:06:17.245 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:17.245 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.245 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.245 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:06:17.245 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.167835+0000", 2024-09-20T19:06:17.246 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.821e-06 2024-09-20T19:06:17.246 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.246 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.246 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:06:17.246 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.167864+0000", 2024-09-20T19:06:17.246 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9714e-05 2024-09-20T19:06:17.246 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.246 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.246 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:06:17.246 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.167883+0000", 2024-09-20T19:06:17.247 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8788999999999999e-05 2024-09-20T19:06:17.247 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.247 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.247 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:06:17.247 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.167891+0000", 2024-09-20T19:06:17.247 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.1669999999999999e-06 2024-09-20T19:06:17.248 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.248 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.248 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:17.248 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.167959+0000", 2024-09-20T19:06:17.248 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7779000000000003e-05 2024-09-20T19:06:17.248 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.248 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.249 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:06:17.249 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:04.167968+0000", 2024-09-20T19:06:17.249 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9630000000000004e-06 2024-09-20T19:06:17.249 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.249 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.249 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:17.249 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.068252+0000", 2024-09-20T19:06:17.250 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.90028381000000002 2024-09-20T19:06:17.250 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.250 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.250 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:06:17.250 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:05.068262+0000", 2024-09-20T19:06:17.250 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0278000000000001e-05 2024-09-20T19:06:17.250 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.250 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.250 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:17.251 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.225517+0000", 2024-09-20T19:06:17.251 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.157254279 2024-09-20T19:06:17.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.251 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.251 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:06:17.251 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.225656+0000", 2024-09-20T19:06:17.252 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013903900000000001 2024-09-20T19:06:17.252 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.252 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.252 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 6,7", 2024-09-20T19:06:17.252 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.225821+0000", 2024-09-20T19:06:17.252 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016562099999999999 2024-09-20T19:06:17.252 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.253 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.253 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:06:17.253 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.229921+0000", 2024-09-20T19:06:17.253 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.004099742 2024-09-20T19:06:17.253 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.253 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.253 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:17.254 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.229977+0000", 2024-09-20T19:06:17.254 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6523000000000001e-05 2024-09-20T19:06:17.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.254 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.254 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:17.254 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.238409+0000", 2024-09-20T19:06:17.254 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0084314219999999992 2024-09-20T19:06:17.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.254 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.254 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:06:17.254 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.238463+0000", 2024-09-20T19:06:17.254 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4308000000000002e-05 2024-09-20T19:06:17.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.255 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.255 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:06:17.255 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T18:59:06.241352+0000", 2024-09-20T19:06:17.255 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0028885680000000002 2024-09-20T19:06:17.255 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:17.255 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:17.255 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:17.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.255 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.255 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:118977 9.8 9:18fffd38:::benchmark_data_smithi083_124552_object4830:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e286)", 2024-09-20T19:06:17.255 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:16.045469+0000", 2024-09-20T19:06:17.255 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 361.15502110800003, 2024-09-20T19:06:17.256 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.233874096, 2024-09-20T19:06:17.256 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:06:17.256 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:06:17.256 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:17.256 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:06:17.256 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:06:17.256 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:06:17.256 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 118977 2024-09-20T19:06:17.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.256 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:06:17.256 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.256 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:06:17.256 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.045469+0000", 2024-09-20T19:06:17.256 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:17.257 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.257 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.257 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:06:17.257 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.045469+0000", 2024-09-20T19:06:17.257 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:17.257 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.257 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.257 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:06:17.257 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.045470+0000", 2024-09-20T19:06:17.257 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.359e-06 2024-09-20T19:06:17.257 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.257 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.258 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:06:17.258 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.045474+0000", 2024-09-20T19:06:17.258 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0609999999999997e-06 2024-09-20T19:06:17.258 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.258 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.258 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:06:17.258 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.045484+0000", 2024-09-20T19:06:17.258 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0332999999999999e-05 2024-09-20T19:06:17.258 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.258 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.258 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:06:17.258 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.045489+0000", 2024-09-20T19:06:17.258 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1869999999999996e-06 2024-09-20T19:06:17.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.259 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:17.259 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.045547+0000", 2024-09-20T19:06:17.259 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7046999999999998e-05 2024-09-20T19:06:17.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.259 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:06:17.259 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.045681+0000", 2024-09-20T19:06:17.259 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00013437200000000001 2024-09-20T19:06:17.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.259 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T19:06:17.260 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.045767+0000", 2024-09-20T19:06:17.260 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6248000000000005e-05 2024-09-20T19:06:17.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.260 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.260 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:06:17.260 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.047811+0000", 2024-09-20T19:06:17.260 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0020440440000000001 2024-09-20T19:06:17.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.260 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.260 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:17.260 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.048095+0000", 2024-09-20T19:06:17.260 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00028369700000000002 2024-09-20T19:06:17.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.261 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.261 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:17.261 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.279270+0000", 2024-09-20T19:06:17.261 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.23117499899999999 2024-09-20T19:06:17.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.261 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.261 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:06:17.261 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.279311+0000", 2024-09-20T19:06:17.261 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0840000000000002e-05 2024-09-20T19:06:17.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.261 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.261 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:06:17.261 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.279343+0000", 2024-09-20T19:06:17.262 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1909000000000002e-05 2024-09-20T19:06:17.262 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:17.262 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:17.262 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:17.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.262 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.262 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:118986 9.8 9:12412698:::benchmark_data_smithi083_124552_object4839:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e286)", 2024-09-20T19:06:17.262 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:16.058237+0000", 2024-09-20T19:06:17.262 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 361.14225290799999, 2024-09-20T19:06:17.262 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.221293453, 2024-09-20T19:06:17.262 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:06:17.262 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:06:17.262 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:17.262 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:06:17.263 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:06:17.263 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:06:17.263 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 118986 2024-09-20T19:06:17.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.263 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:06:17.263 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.263 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:06:17.263 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.058237+0000", 2024-09-20T19:06:17.263 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:17.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.263 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.263 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:06:17.263 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.058237+0000", 2024-09-20T19:06:17.264 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:17.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.264 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:06:17.264 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.058238+0000", 2024-09-20T19:06:17.264 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1799999999999999e-06 2024-09-20T19:06:17.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.264 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:06:17.264 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.058242+0000", 2024-09-20T19:06:17.264 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3880000000000002e-06 2024-09-20T19:06:17.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.264 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:06:17.265 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.058251+0000", 2024-09-20T19:06:17.265 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0420000000000008e-06 2024-09-20T19:06:17.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.265 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:06:17.265 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.058256+0000", 2024-09-20T19:06:17.265 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8960000000000002e-06 2024-09-20T19:06:17.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.265 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:17.265 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.058309+0000", 2024-09-20T19:06:17.265 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2902999999999998e-05 2024-09-20T19:06:17.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.266 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:06:17.266 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.058410+0000", 2024-09-20T19:06:17.266 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010079200000000001 2024-09-20T19:06:17.266 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.266 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.266 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,5", 2024-09-20T19:06:17.266 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.058476+0000", 2024-09-20T19:06:17.266 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6050000000000006e-05 2024-09-20T19:06:17.266 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.266 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.266 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:06:17.266 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.062309+0000", 2024-09-20T19:06:17.266 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003832701 2024-09-20T19:06:17.266 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.267 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.267 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:17.267 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.062336+0000", 2024-09-20T19:06:17.267 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7426000000000001e-05 2024-09-20T19:06:17.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.267 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.267 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:17.267 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.279409+0000", 2024-09-20T19:06:17.267 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.21707326399999999 2024-09-20T19:06:17.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.267 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.268 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:06:17.268 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.279490+0000", 2024-09-20T19:06:17.268 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0500999999999994e-05 2024-09-20T19:06:17.268 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.268 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.268 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:06:17.268 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:16.279530+0000", 2024-09-20T19:06:17.268 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0309999999999999e-05 2024-09-20T19:06:17.268 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:17.268 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:17.268 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:17.268 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.268 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.268 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:124578 9.2 9:422abb8a:::benchmark_data_smithi083_124552_object10431:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e289)", 2024-09-20T19:06:17.268 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:20.946829+0000", 2024-09-20T19:06:17.269 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 356.25366076199998, 2024-09-20T19:06:17.269 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.33414796600000002, 2024-09-20T19:06:17.269 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:06:17.269 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:06:17.269 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:17.269 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:06:17.269 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:06:17.269 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:06:17.269 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 124578 2024-09-20T19:06:17.269 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.269 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:06:17.269 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.269 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:06:17.269 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.946829+0000", 2024-09-20T19:06:17.269 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:17.269 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.270 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.270 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:06:17.270 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.946829+0000", 2024-09-20T19:06:17.270 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:17.270 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.270 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.270 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:06:17.270 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.946830+0000", 2024-09-20T19:06:17.270 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2299999999999999e-07 2024-09-20T19:06:17.270 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.270 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.270 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:06:17.270 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.946834+0000", 2024-09-20T19:06:17.271 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1189999999999999e-06 2024-09-20T19:06:17.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.271 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.271 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:06:17.271 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.946854+0000", 2024-09-20T19:06:17.271 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9754000000000001e-05 2024-09-20T19:06:17.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.271 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.271 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:06:17.271 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.946861+0000", 2024-09-20T19:06:17.271 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7719999999999994e-06 2024-09-20T19:06:17.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.271 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.271 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:17.271 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.951540+0000", 2024-09-20T19:06:17.271 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.004678354 2024-09-20T19:06:17.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.272 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.272 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:06:17.272 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.951640+0000", 2024-09-20T19:06:17.272 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000100055 2024-09-20T19:06:17.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.272 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.272 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:06:17.272 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.951741+0000", 2024-09-20T19:06:17.272 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010086600000000001 2024-09-20T19:06:17.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.272 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.272 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:06:17.272 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.954721+0000", 2024-09-20T19:06:17.272 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0029804609999999998 2024-09-20T19:06:17.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.273 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.273 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:17.273 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.956254+0000", 2024-09-20T19:06:17.273 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001532683 2024-09-20T19:06:17.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.273 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.273 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:17.273 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.280892+0000", 2024-09-20T19:06:17.273 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.32463842100000001 2024-09-20T19:06:17.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.273 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.273 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:06:17.273 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.280935+0000", 2024-09-20T19:06:17.274 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2592999999999997e-05 2024-09-20T19:06:17.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.274 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.274 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:06:17.274 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.280977+0000", 2024-09-20T19:06:17.274 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1965000000000002e-05 2024-09-20T19:06:17.274 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:17.274 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:17.274 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:17.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.274 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.274 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:124596 9.2 9:42d01fa2:::benchmark_data_smithi083_124552_object10449:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e289)", 2024-09-20T19:06:17.274 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:20.962657+0000", 2024-09-20T19:06:17.274 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 356.23783261400001, 2024-09-20T19:06:17.275 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.32748537700000002, 2024-09-20T19:06:17.275 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:06:17.275 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:06:17.275 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:17.275 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:06:17.275 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:06:17.275 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:06:17.275 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 124596 2024-09-20T19:06:17.275 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.275 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:06:17.275 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.275 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:06:17.275 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.962657+0000", 2024-09-20T19:06:17.275 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:17.276 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.276 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.276 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:06:17.276 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.962657+0000", 2024-09-20T19:06:17.276 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:17.276 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.276 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.276 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:06:17.276 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.962658+0000", 2024-09-20T19:06:17.276 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4020000000000001e-06 2024-09-20T19:06:17.276 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.276 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.276 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:06:17.276 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.962663+0000", 2024-09-20T19:06:17.276 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0590000000000002e-06 2024-09-20T19:06:17.276 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.277 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.277 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:06:17.277 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.962678+0000", 2024-09-20T19:06:17.277 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4469e-05 2024-09-20T19:06:17.277 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.277 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.277 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:06:17.277 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.962685+0000", 2024-09-20T19:06:17.277 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3549999999999999e-06 2024-09-20T19:06:17.277 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.277 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.277 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:17.277 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.962732+0000", 2024-09-20T19:06:17.277 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6502999999999998e-05 2024-09-20T19:06:17.278 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.278 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.278 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:06:17.278 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.962831+0000", 2024-09-20T19:06:17.278 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9489999999999998e-05 2024-09-20T19:06:17.278 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.278 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.278 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:06:17.278 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.962910+0000", 2024-09-20T19:06:17.278 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8527000000000002e-05 2024-09-20T19:06:17.278 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.278 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.278 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:06:17.279 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.973808+0000", 2024-09-20T19:06:17.279 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010898134 2024-09-20T19:06:17.279 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.279 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.279 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:17.279 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.973889+0000", 2024-09-20T19:06:17.279 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0641000000000003e-05 2024-09-20T19:06:17.279 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.279 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.279 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:17.279 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.290050+0000", 2024-09-20T19:06:17.279 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.31616166899999998 2024-09-20T19:06:17.279 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.279 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.279 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:06:17.279 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.290100+0000", 2024-09-20T19:06:17.280 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0179999999999997e-05 2024-09-20T19:06:17.280 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.280 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.280 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:06:17.280 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.290142+0000", 2024-09-20T19:06:17.280 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1947999999999999e-05 2024-09-20T19:06:17.280 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:17.280 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:17.280 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:17.280 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.280 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.280 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:124603 9.2 9:4d3bc5dd:::benchmark_data_smithi083_124552_object10456:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e289)", 2024-09-20T19:06:17.280 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:20.964654+0000", 2024-09-20T19:06:17.280 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 356.23583533300001, 2024-09-20T19:06:17.280 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.32571787499999999, 2024-09-20T19:06:17.281 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:06:17.281 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:06:17.281 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:17.281 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:06:17.281 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:06:17.281 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:06:17.281 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 124603 2024-09-20T19:06:17.281 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.281 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:06:17.281 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.281 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:06:17.281 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.964654+0000", 2024-09-20T19:06:17.281 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:17.282 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.282 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.282 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:06:17.282 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.964654+0000", 2024-09-20T19:06:17.282 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:17.282 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.282 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.282 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:06:17.282 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.964655+0000", 2024-09-20T19:06:17.282 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6099999999999999e-07 2024-09-20T19:06:17.282 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.282 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.282 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:06:17.282 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.964698+0000", 2024-09-20T19:06:17.282 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2778999999999998e-05 2024-09-20T19:06:17.283 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.283 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.283 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:06:17.283 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.964707+0000", 2024-09-20T19:06:17.283 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0869999999999996e-06 2024-09-20T19:06:17.283 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.283 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.283 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:06:17.283 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.964713+0000", 2024-09-20T19:06:17.283 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3339999999999998e-06 2024-09-20T19:06:17.283 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.283 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.283 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:17.283 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.965135+0000", 2024-09-20T19:06:17.284 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00042182599999999998 2024-09-20T19:06:17.284 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.284 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.284 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:06:17.284 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.965229+0000", 2024-09-20T19:06:17.284 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4297999999999997e-05 2024-09-20T19:06:17.284 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.284 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.284 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:06:17.284 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.965326+0000", 2024-09-20T19:06:17.284 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6902000000000002e-05 2024-09-20T19:06:17.284 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.284 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.284 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:06:17.285 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.973814+0000", 2024-09-20T19:06:17.285 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0084871779999999997 2024-09-20T19:06:17.285 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.285 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.285 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:17.285 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.973930+0000", 2024-09-20T19:06:17.285 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011664599999999999 2024-09-20T19:06:17.285 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.285 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.285 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:17.285 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.290288+0000", 2024-09-20T19:06:17.285 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.31635755799999998 2024-09-20T19:06:17.285 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.285 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.285 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:06:17.286 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.290337+0000", 2024-09-20T19:06:17.286 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8948e-05 2024-09-20T19:06:17.286 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.286 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.286 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:06:17.286 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.290372+0000", 2024-09-20T19:06:17.286 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5457999999999998e-05 2024-09-20T19:06:17.286 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:17.286 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:17.286 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:17.286 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.286 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.286 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:124604 9.2 9:40e573dc:::benchmark_data_smithi083_124552_object10457:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e289)", 2024-09-20T19:06:17.286 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:20.964729+0000", 2024-09-20T19:06:17.287 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 356.23576080800001, 2024-09-20T19:06:17.287 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.32575654300000001, 2024-09-20T19:06:17.287 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:06:17.287 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:06:17.287 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:17.287 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:06:17.287 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:06:17.287 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:06:17.287 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 124604 2024-09-20T19:06:17.287 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.287 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:06:17.287 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.287 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:06:17.287 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.964729+0000", 2024-09-20T19:06:17.288 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:17.288 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.288 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.288 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:06:17.288 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.964729+0000", 2024-09-20T19:06:17.288 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:17.288 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.288 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.288 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:06:17.288 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.964729+0000", 2024-09-20T19:06:17.288 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6599999999999996e-07 2024-09-20T19:06:17.288 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.289 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.289 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:06:17.289 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.964732+0000", 2024-09-20T19:06:17.289 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.9679999999999998e-06 2024-09-20T19:06:17.289 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.289 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.289 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:06:17.289 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.964738+0000", 2024-09-20T19:06:17.289 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6910000000000002e-06 2024-09-20T19:06:17.289 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.289 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.289 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:06:17.289 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.964744+0000", 2024-09-20T19:06:17.289 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5899999999999998e-06 2024-09-20T19:06:17.289 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.289 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.290 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:17.290 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.967815+0000", 2024-09-20T19:06:17.290 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0030715209999999998 2024-09-20T19:06:17.290 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.290 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.290 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:06:17.290 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.967939+0000", 2024-09-20T19:06:17.290 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012392100000000001 2024-09-20T19:06:17.290 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.290 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.290 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:06:17.290 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.968041+0000", 2024-09-20T19:06:17.290 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000102166 2024-09-20T19:06:17.291 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.291 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.291 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:17.291 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.973986+0000", 2024-09-20T19:06:17.291 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059442610000000002 2024-09-20T19:06:17.291 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.291 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.291 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:06:17.291 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.973997+0000", 2024-09-20T19:06:17.291 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1313e-05 2024-09-20T19:06:17.291 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.291 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.291 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:17.291 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.290407+0000", 2024-09-20T19:06:17.292 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.31641063699999999 2024-09-20T19:06:17.292 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.292 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.292 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:06:17.292 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.290452+0000", 2024-09-20T19:06:17.292 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4753999999999999e-05 2024-09-20T19:06:17.292 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.292 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.292 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:06:17.292 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.290485+0000", 2024-09-20T19:06:17.292 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3154999999999997e-05 2024-09-20T19:06:17.292 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:17.292 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:17.292 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:17.292 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.292 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.292 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:124610 9.2 9:4e613fc5:::benchmark_data_smithi083_124552_object10463:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e289)", 2024-09-20T19:06:17.293 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:20.970344+0000", 2024-09-20T19:06:17.293 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 356.23014558800003, 2024-09-20T19:06:17.293 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.32025382099999999, 2024-09-20T19:06:17.293 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:06:17.293 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:06:17.293 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:17.293 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:06:17.293 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:06:17.293 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:06:17.293 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 124610 2024-09-20T19:06:17.293 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.293 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:06:17.293 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.293 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:06:17.294 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.970344+0000", 2024-09-20T19:06:17.294 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:17.294 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.294 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.294 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:06:17.294 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.970344+0000", 2024-09-20T19:06:17.294 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:17.294 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.294 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.294 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:06:17.294 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.970345+0000", 2024-09-20T19:06:17.294 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5000000000000003e-07 2024-09-20T19:06:17.295 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.295 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.295 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:06:17.295 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.970347+0000", 2024-09-20T19:06:17.295 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6769999999999999e-06 2024-09-20T19:06:17.295 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.295 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.295 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:06:17.295 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.970354+0000", 2024-09-20T19:06:17.295 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6100000000000002e-06 2024-09-20T19:06:17.295 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.295 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.295 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:06:17.295 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.970359+0000", 2024-09-20T19:06:17.296 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5990000000000001e-06 2024-09-20T19:06:17.296 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.296 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.296 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:17.296 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.971408+0000", 2024-09-20T19:06:17.296 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0010481220000000001 2024-09-20T19:06:17.296 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.296 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.296 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:06:17.296 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.971499+0000", 2024-09-20T19:06:17.296 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1843999999999997e-05 2024-09-20T19:06:17.296 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.296 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.296 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:06:17.297 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.971597+0000", 2024-09-20T19:06:17.297 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8011999999999996e-05 2024-09-20T19:06:17.297 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.297 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.297 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:06:17.297 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.974098+0000", 2024-09-20T19:06:17.297 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00250016 2024-09-20T19:06:17.297 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.297 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.297 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:17.297 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.978935+0000", 2024-09-20T19:06:17.297 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.004836974 2024-09-20T19:06:17.297 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.297 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.297 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:17.297 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.290520+0000", 2024-09-20T19:06:17.298 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.31158548699999999 2024-09-20T19:06:17.298 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.298 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.298 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:06:17.298 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.290566+0000", 2024-09-20T19:06:17.298 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5519000000000002e-05 2024-09-20T19:06:17.298 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.298 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.298 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:06:17.298 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.290598+0000", 2024-09-20T19:06:17.298 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2267000000000001e-05 2024-09-20T19:06:17.299 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:17.299 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:17.299 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:17.299 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.299 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.299 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:124618 9.2 9:4c60c683:::benchmark_data_smithi083_124552_object10471:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e289)", 2024-09-20T19:06:17.299 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:20.974705+0000", 2024-09-20T19:06:17.299 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 356.22578428899999, 2024-09-20T19:06:17.299 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.31613734999999998, 2024-09-20T19:06:17.299 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:06:17.299 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:06:17.299 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:17.299 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:06:17.299 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:06:17.299 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:06:17.299 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 124618 2024-09-20T19:06:17.300 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.300 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:06:17.300 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.300 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:06:17.300 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.974705+0000", 2024-09-20T19:06:17.300 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:17.300 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.300 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.300 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:06:17.300 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.974705+0000", 2024-09-20T19:06:17.300 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:17.300 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.300 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.300 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:06:17.300 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.974707+0000", 2024-09-20T19:06:17.301 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4109999999999999e-06 2024-09-20T19:06:17.301 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.301 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.301 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:06:17.301 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.974711+0000", 2024-09-20T19:06:17.301 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5229999999999997e-06 2024-09-20T19:06:17.301 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.301 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.301 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:06:17.301 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.974720+0000", 2024-09-20T19:06:17.301 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.3829999999999992e-06 2024-09-20T19:06:17.301 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.301 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.301 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:06:17.301 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.974726+0000", 2024-09-20T19:06:17.302 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7039999999999997e-06 2024-09-20T19:06:17.302 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.302 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.302 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:17.302 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.974777+0000", 2024-09-20T19:06:17.302 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0785999999999998e-05 2024-09-20T19:06:17.302 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.302 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.302 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:06:17.302 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.974886+0000", 2024-09-20T19:06:17.302 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010864199999999999 2024-09-20T19:06:17.303 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.303 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.303 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:06:17.303 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.974983+0000", 2024-09-20T19:06:17.303 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7293000000000005e-05 2024-09-20T19:06:17.303 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.303 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.303 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:06:17.303 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.977705+0000", 2024-09-20T19:06:17.303 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0027215339999999998 2024-09-20T19:06:17.303 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.303 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.303 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:17.304 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.979236+0000", 2024-09-20T19:06:17.304 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0015310079999999999 2024-09-20T19:06:17.304 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.304 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.304 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:17.304 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.290703+0000", 2024-09-20T19:06:17.304 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.311467775 2024-09-20T19:06:17.304 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.304 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.304 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:06:17.304 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.290802+0000", 2024-09-20T19:06:17.304 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8282999999999994e-05 2024-09-20T19:06:17.304 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.304 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.305 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:06:17.305 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.290843+0000", 2024-09-20T19:06:17.305 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1007999999999998e-05 2024-09-20T19:06:17.305 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:17.305 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:17.305 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:17.305 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.305 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.305 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:124636 9.2 9:45855acf:::benchmark_data_smithi083_124552_object10489:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e289)", 2024-09-20T19:06:17.305 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:20.997591+0000", 2024-09-20T19:06:17.305 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 356.202898694, 2024-09-20T19:06:17.305 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.29766285199999998, 2024-09-20T19:06:17.305 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:06:17.305 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:06:17.305 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:17.305 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:06:17.306 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:06:17.306 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:06:17.306 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 124636 2024-09-20T19:06:17.306 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.306 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:06:17.306 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.306 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:06:17.306 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.997591+0000", 2024-09-20T19:06:17.306 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:17.306 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.306 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.306 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:06:17.306 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.997591+0000", 2024-09-20T19:06:17.306 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:17.307 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.307 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.307 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:06:17.307 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.997592+0000", 2024-09-20T19:06:17.307 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.6000000000000003e-07 2024-09-20T19:06:17.307 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.307 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.307 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:06:17.307 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.997595+0000", 2024-09-20T19:06:17.307 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6449999999999998e-06 2024-09-20T19:06:17.307 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.307 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.307 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:06:17.307 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.997604+0000", 2024-09-20T19:06:17.307 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2090000000000005e-06 2024-09-20T19:06:17.307 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.308 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.308 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:06:17.308 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.997608+0000", 2024-09-20T19:06:17.308 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8960000000000002e-06 2024-09-20T19:06:17.308 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.308 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.308 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:17.308 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.997646+0000", 2024-09-20T19:06:17.308 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7184000000000002e-05 2024-09-20T19:06:17.308 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.308 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.308 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:06:17.308 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.997718+0000", 2024-09-20T19:06:17.308 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1884000000000005e-05 2024-09-20T19:06:17.309 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.309 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.309 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:06:17.309 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.997784+0000", 2024-09-20T19:06:17.309 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6018000000000004e-05 2024-09-20T19:06:17.309 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.309 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.309 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:06:17.309 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.999669+0000", 2024-09-20T19:06:17.309 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001885954 2024-09-20T19:06:17.309 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.309 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.309 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:17.309 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.999896+0000", 2024-09-20T19:06:17.309 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022635999999999999 2024-09-20T19:06:17.310 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.310 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.310 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:17.310 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.294934+0000", 2024-09-20T19:06:17.310 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.29503834600000001 2024-09-20T19:06:17.310 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.310 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.310 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:06:17.310 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.295205+0000", 2024-09-20T19:06:17.310 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00027075200000000003 2024-09-20T19:06:17.310 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.310 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.310 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:06:17.310 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.295254+0000", 2024-09-20T19:06:17.310 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8844000000000002e-05 2024-09-20T19:06:17.310 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:17.311 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:17.311 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:17.311 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.311 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.311 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:142913 9.c 9:3d798970:::benchmark_data_smithi083_124552_object28766:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e291)", 2024-09-20T19:06:17.311 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:35.168460+0000", 2024-09-20T19:06:17.311 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 342.032029475, 2024-09-20T19:06:17.311 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.040925943, 2024-09-20T19:06:17.311 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:06:17.311 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:06:17.311 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-20T19:06:17.311 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:06:17.311 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:06:17.311 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:06:17.312 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 142913 2024-09-20T19:06:17.312 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.312 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:06:17.312 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.312 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:06:17.312 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.168460+0000", 2024-09-20T19:06:17.312 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:17.312 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.312 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.312 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:06:17.312 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.168460+0000", 2024-09-20T19:06:17.312 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:17.312 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.312 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.312 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:06:17.313 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.168462+0000", 2024-09-20T19:06:17.313 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9520000000000001e-06 2024-09-20T19:06:17.313 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.313 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.313 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:06:17.313 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.168469+0000", 2024-09-20T19:06:17.313 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6309999999999997e-06 2024-09-20T19:06:17.313 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.313 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.313 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:06:17.313 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.168485+0000", 2024-09-20T19:06:17.313 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5798e-05 2024-09-20T19:06:17.313 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.314 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.314 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:06:17.314 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.168492+0000", 2024-09-20T19:06:17.314 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9179999999999994e-06 2024-09-20T19:06:17.314 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.314 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.314 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:17.314 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.168833+0000", 2024-09-20T19:06:17.314 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00034100200000000003 2024-09-20T19:06:17.314 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.314 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.314 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:06:17.314 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.169144+0000", 2024-09-20T19:06:17.315 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00031083499999999997 2024-09-20T19:06:17.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.315 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.315 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4,7", 2024-09-20T19:06:17.315 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.169258+0000", 2024-09-20T19:06:17.315 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011411 2024-09-20T19:06:17.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.315 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.315 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:06:17.315 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.172245+0000", 2024-09-20T19:06:17.315 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002987048 2024-09-20T19:06:17.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.315 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.315 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:17.315 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.203777+0000", 2024-09-20T19:06:17.316 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.031531525999999997 2024-09-20T19:06:17.316 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.316 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.316 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:17.316 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.213031+0000", 2024-09-20T19:06:17.316 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0092539930000000003 2024-09-20T19:06:17.316 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.316 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.316 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:06:17.316 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.213039+0000", 2024-09-20T19:06:17.316 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4090000000000003e-06 2024-09-20T19:06:17.316 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.316 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.316 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:17.316 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.209326+0000", 2024-09-20T19:06:17.317 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.99628679899999995 2024-09-20T19:06:17.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.317 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.317 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:06:17.317 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.209386+0000", 2024-09-20T19:06:17.317 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9922e-05 2024-09-20T19:06:17.317 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:17.317 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:17.317 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:17.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.317 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.317 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:142917 9.c 9:32f2df36:::benchmark_data_smithi083_124552_object28770:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e291)", 2024-09-20T19:06:17.317 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:35.172691+0000", 2024-09-20T19:06:17.317 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 342.02779910599997, 2024-09-20T19:06:17.317 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0372109220000001, 2024-09-20T19:06:17.317 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:06:17.317 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:06:17.318 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-20T19:06:17.318 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:06:17.318 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:06:17.318 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:06:17.318 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 142917 2024-09-20T19:06:17.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.318 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:06:17.318 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.318 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:06:17.318 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.172691+0000", 2024-09-20T19:06:17.318 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:17.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.318 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.318 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:06:17.319 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.172691+0000", 2024-09-20T19:06:17.319 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:17.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.319 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.319 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:06:17.319 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.172692+0000", 2024-09-20T19:06:17.319 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.95e-06 2024-09-20T19:06:17.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.319 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.319 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:06:17.319 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.172701+0000", 2024-09-20T19:06:17.320 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0630000000000006e-06 2024-09-20T19:06:17.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.320 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.320 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:06:17.320 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.172714+0000", 2024-09-20T19:06:17.320 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3303e-05 2024-09-20T19:06:17.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.320 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.320 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:06:17.320 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.172723+0000", 2024-09-20T19:06:17.320 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6160000000000008e-06 2024-09-20T19:06:17.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.320 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.321 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:17.321 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.172857+0000", 2024-09-20T19:06:17.321 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000133829 2024-09-20T19:06:17.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.321 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.321 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:06:17.321 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.172975+0000", 2024-09-20T19:06:17.321 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000117224 2024-09-20T19:06:17.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.321 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.321 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4,7", 2024-09-20T19:06:17.321 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.173088+0000", 2024-09-20T19:06:17.321 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000113551 2024-09-20T19:06:17.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.321 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.321 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:06:17.322 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.202597+0000", 2024-09-20T19:06:17.322 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.029508818999999999 2024-09-20T19:06:17.322 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.322 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.322 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:17.322 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.203835+0000", 2024-09-20T19:06:17.322 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001238444 2024-09-20T19:06:17.322 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.322 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.322 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:17.322 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.215654+0000", 2024-09-20T19:06:17.322 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011818403999999999 2024-09-20T19:06:17.322 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.322 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.323 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:06:17.323 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.215699+0000", 2024-09-20T19:06:17.323 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5084999999999999e-05 2024-09-20T19:06:17.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.323 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.323 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:17.323 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.209843+0000", 2024-09-20T19:06:17.323 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.99414379500000005 2024-09-20T19:06:17.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.323 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.323 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:06:17.323 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.209901+0000", 2024-09-20T19:06:17.323 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8838999999999997e-05 2024-09-20T19:06:17.323 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:17.323 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:17.323 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:17.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.324 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.324 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:142919 9.c 9:3aa96e52:::benchmark_data_smithi083_124552_object28772:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e291)", 2024-09-20T19:06:17.324 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:35.175226+0000", 2024-09-20T19:06:17.324 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 342.02526386099998, 2024-09-20T19:06:17.324 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.044694204, 2024-09-20T19:06:17.324 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:06:17.324 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:06:17.324 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:17.324 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:06:17.324 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:06:17.324 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:06:17.324 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 142919 2024-09-20T19:06:17.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.324 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:06:17.324 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.325 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:06:17.325 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.175226+0000", 2024-09-20T19:06:17.325 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:17.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.325 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.325 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:06:17.325 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.175226+0000", 2024-09-20T19:06:17.325 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:17.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.325 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.325 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:06:17.325 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.175228+0000", 2024-09-20T19:06:17.325 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.88e-06 2024-09-20T19:06:17.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.325 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.325 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:06:17.326 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.175235+0000", 2024-09-20T19:06:17.326 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8419999999999999e-06 2024-09-20T19:06:17.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.326 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.326 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:06:17.326 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.175253+0000", 2024-09-20T19:06:17.326 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8995e-05 2024-09-20T19:06:17.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.326 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.326 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:06:17.326 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.175262+0000", 2024-09-20T19:06:17.326 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.884e-06 2024-09-20T19:06:17.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.326 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.326 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:17.327 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.215752+0000", 2024-09-20T19:06:17.327 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.040489565999999998 2024-09-20T19:06:17.327 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.327 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.327 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:06:17.327 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.215759+0000", 2024-09-20T19:06:17.327 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1380000000000001e-06 2024-09-20T19:06:17.327 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.327 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.327 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:17.327 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.209925+0000", 2024-09-20T19:06:17.327 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.99416557900000002 2024-09-20T19:06:17.328 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.328 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.328 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:06:17.328 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.210025+0000", 2024-09-20T19:06:17.328 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010050700000000001 2024-09-20T19:06:17.328 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.328 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.328 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 7", 2024-09-20T19:06:17.328 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.210147+0000", 2024-09-20T19:06:17.328 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000122309 2024-09-20T19:06:17.328 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.328 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.329 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:17.329 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.219740+0000", 2024-09-20T19:06:17.329 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0095921180000000002 2024-09-20T19:06:17.329 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.329 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.329 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:06:17.329 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.219768+0000", 2024-09-20T19:06:17.329 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8391e-05 2024-09-20T19:06:17.329 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.329 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.329 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:06:17.330 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.219815+0000", 2024-09-20T19:06:17.330 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7197999999999998e-05 2024-09-20T19:06:17.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.330 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.330 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:06:17.330 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.219920+0000", 2024-09-20T19:06:17.330 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010479700000000001 2024-09-20T19:06:17.330 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:17.330 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:17.330 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:17.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.331 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.331 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14460.0:2698 5.0 5:05bf5b68:::notify.1:head [watch reconnect cookie 94056120167808 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e292)", 2024-09-20T19:06:17.331 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:35.178092+0000", 2024-09-20T19:06:17.331 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 342.02239777699998, 2024-09-20T19:06:17.331 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.021665381, 2024-09-20T19:06:17.331 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:06:17.331 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:06:17.332 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T19:06:17.332 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:06:17.332 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14460", 2024-09-20T19:06:17.332 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1555339426", 2024-09-20T19:06:17.332 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2698 2024-09-20T19:06:17.332 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.332 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:06:17.332 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.332 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:06:17.333 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.178092+0000", 2024-09-20T19:06:17.333 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:17.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.333 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:06:17.333 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.178092+0000", 2024-09-20T19:06:17.333 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:17.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.333 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:06:17.333 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.178093+0000", 2024-09-20T19:06:17.333 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2359999999999999e-06 2024-09-20T19:06:17.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.334 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:06:17.334 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.178098+0000", 2024-09-20T19:06:17.334 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1410000000000004e-06 2024-09-20T19:06:17.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.334 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.334 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:06:17.334 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.178113+0000", 2024-09-20T19:06:17.334 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4262e-05 2024-09-20T19:06:17.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.334 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.334 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:06:17.334 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.178120+0000", 2024-09-20T19:06:17.334 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.6210000000000004e-06 2024-09-20T19:06:17.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.335 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.335 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:17.335 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.180022+0000", 2024-09-20T19:06:17.335 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001901668 2024-09-20T19:06:17.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.335 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.335 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op must wait for map", 2024-09-20T19:06:17.335 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.180028+0000", 2024-09-20T19:06:17.335 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9359999999999996e-06 2024-09-20T19:06:17.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.335 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.335 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:17.335 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.195134+0000", 2024-09-20T19:06:17.335 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.015105831 2024-09-20T19:06:17.336 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.336 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.336 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:06:17.336 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.195142+0000", 2024-09-20T19:06:17.336 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8659999999999995e-06 2024-09-20T19:06:17.336 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.336 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.336 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:17.336 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.192667+0000", 2024-09-20T19:06:17.336 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.99752498700000003 2024-09-20T19:06:17.336 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.336 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.336 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:06:17.336 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.192972+0000", 2024-09-20T19:06:17.336 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000304677 2024-09-20T19:06:17.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.337 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.337 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:06:17.337 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.199757+0000", 2024-09-20T19:06:17.337 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0067851559999999997 2024-09-20T19:06:17.337 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:17.337 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:17.337 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:17.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.337 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.337 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:142931 9.c 9:3971d803:::benchmark_data_smithi083_124552_object28784:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e292)", 2024-09-20T19:06:17.337 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:35.206532+0000", 2024-09-20T19:06:17.337 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 341.99395810800002, 2024-09-20T19:06:17.337 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.016374055, 2024-09-20T19:06:17.337 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:06:17.338 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:06:17.338 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:17.338 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:06:17.338 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:06:17.338 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:06:17.338 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 142931 2024-09-20T19:06:17.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.338 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:06:17.338 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.338 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:06:17.338 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.206532+0000", 2024-09-20T19:06:17.338 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:17.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.338 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.338 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:06:17.339 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.206532+0000", 2024-09-20T19:06:17.339 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:17.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.339 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.339 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:06:17.339 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.206532+0000", 2024-09-20T19:06:17.339 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9500000000000001e-07 2024-09-20T19:06:17.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.339 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.339 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:06:17.339 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.206537+0000", 2024-09-20T19:06:17.340 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5600000000000004e-06 2024-09-20T19:06:17.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.340 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:06:17.340 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.206546+0000", 2024-09-20T19:06:17.340 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9479999999999997e-06 2024-09-20T19:06:17.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.340 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:06:17.340 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.206553+0000", 2024-09-20T19:06:17.340 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0119999999999999e-06 2024-09-20T19:06:17.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.341 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:17.341 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.215793+0000", 2024-09-20T19:06:17.341 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0092399519999999992 2024-09-20T19:06:17.341 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.341 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.341 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:06:17.341 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:17.342 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:16 smithi142 ceph-mon[32508]: 3.3 scrub starts 2024-09-20T19:06:17.342 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:16 smithi142 ceph-mon[32508]: 3.3 scrub ok 2024-09-20T19:06:17.342 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:16 smithi142 ceph-mon[32508]: 6.0 scrub starts 2024-09-20T19:06:17.343 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:16 smithi142 ceph-mon[32508]: 6.0 scrub ok 2024-09-20T19:06:17.343 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:16 smithi142 ceph-mon[32508]: 3.e scrub starts 2024-09-20T19:06:17.343 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:16 smithi142 ceph-mon[32508]: 3.e scrub ok 2024-09-20T19:06:17.343 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:16 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1567882367' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:06:17.343 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:16 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1806455295' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16} : dispatch 2024-09-20T19:06:17.343 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:16 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:06:17.343 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.215799+0000", 2024-09-20T19:06:17.343 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7389999999999996e-06 2024-09-20T19:06:17.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.343 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.344 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:17.344 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.212804+0000", 2024-09-20T19:06:17.344 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.99700544700000004 2024-09-20T19:06:17.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.344 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.344 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:06:17.344 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.213100+0000", 2024-09-20T19:06:17.344 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00029572499999999999 2024-09-20T19:06:17.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.344 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.344 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 7", 2024-09-20T19:06:17.344 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.213492+0000", 2024-09-20T19:06:17.344 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000391939 2024-09-20T19:06:17.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.345 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.345 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:06:17.345 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.219928+0000", 2024-09-20T19:06:17.345 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0064358100000000001 2024-09-20T19:06:17.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.345 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.345 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:17.345 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.222687+0000", 2024-09-20T19:06:17.345 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0027591949999999999 2024-09-20T19:06:17.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.346 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.346 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:06:17.346 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.222797+0000", 2024-09-20T19:06:17.346 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000110713 2024-09-20T19:06:17.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.346 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.346 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:06:17.346 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.222906+0000", 2024-09-20T19:06:17.346 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010812 2024-09-20T19:06:17.346 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:17.346 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:17.347 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:17.347 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.347 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.347 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:142996 9.c 9:3cdc78e2:::benchmark_data_smithi083_124552_object28849:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e292)", 2024-09-20T19:06:17.347 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:35.284730+0000", 2024-09-20T19:06:17.347 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 341.915759939, 2024-09-20T19:06:17.347 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.93861088699999995, 2024-09-20T19:06:17.347 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:06:17.347 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:06:17.347 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:17.347 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:06:17.347 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:06:17.347 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:06:17.347 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 142996 2024-09-20T19:06:17.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.348 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:06:17.348 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.348 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:06:17.348 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.284730+0000", 2024-09-20T19:06:17.348 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:17.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.348 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.348 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:06:17.348 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.284730+0000", 2024-09-20T19:06:17.348 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:17.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.348 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.348 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:06:17.349 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.284731+0000", 2024-09-20T19:06:17.349 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.376e-06 2024-09-20T19:06:17.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.349 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.349 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:06:17.349 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.284736+0000", 2024-09-20T19:06:17.349 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1680000000000003e-06 2024-09-20T19:06:17.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.349 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.349 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:06:17.349 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.284747+0000", 2024-09-20T19:06:17.349 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1077000000000001e-05 2024-09-20T19:06:17.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.349 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.349 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:06:17.350 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.284753+0000", 2024-09-20T19:06:17.350 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.874e-06 2024-09-20T19:06:17.350 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.350 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.350 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:17.350 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.284810+0000", 2024-09-20T19:06:17.350 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6573000000000002e-05 2024-09-20T19:06:17.350 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.350 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.350 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:06:17.351 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.284816+0000", 2024-09-20T19:06:17.351 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4840000000000001e-06 2024-09-20T19:06:17.351 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.351 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.351 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:17.351 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.216111+0000", 2024-09-20T19:06:17.351 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.93129438899999994 2024-09-20T19:06:17.351 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.351 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.351 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:06:17.351 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.216258+0000", 2024-09-20T19:06:17.351 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014760700000000001 2024-09-20T19:06:17.351 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.351 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.352 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 7", 2024-09-20T19:06:17.352 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.217033+0000", 2024-09-20T19:06:17.352 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00077462600000000005 2024-09-20T19:06:17.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.352 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.352 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:17.352 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.223055+0000", 2024-09-20T19:06:17.352 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0060222510000000002 2024-09-20T19:06:17.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.352 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.352 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:06:17.352 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.223067+0000", 2024-09-20T19:06:17.352 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2087e-05 2024-09-20T19:06:17.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.353 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.353 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:06:17.353 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.223265+0000", 2024-09-20T19:06:17.353 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019747800000000001 2024-09-20T19:06:17.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:17.353 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:17.353 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:06:17.353 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.223341+0000", 2024-09-20T19:06:17.353 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5896999999999995e-05 2024-09-20T19:06:17.353 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:17.353 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:17.354 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:17.354 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:17.354 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:17.354 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T19:06:17.365 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:17.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:17.366 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:17.467 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:17.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:17.490 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:17.490 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:17.591 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:17.613 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:17.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:17.614 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:17.714 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:17.737 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:17.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:17.738 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:17.761 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:06:17 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:06:17.839 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:17.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:17.863 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:17.863 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:17.957 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd pool application enable unique_pool_4 rados --yes-i-really-mean-it || true 2024-09-20T19:06:17.964 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:17.990 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:17.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:17.991 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:18.092 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:18.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:18.114 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:18.114 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:18.215 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:18.241 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:18.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:18.241 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:18.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:17 smithi142 ceph-mon[32508]: 5.1c scrub starts 2024-09-20T19:06:18.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:17 smithi142 ceph-mon[32508]: 5.1c scrub ok 2024-09-20T19:06:18.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:17 smithi142 ceph-mon[32508]: 3.7 scrub starts 2024-09-20T19:06:18.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:17 smithi142 ceph-mon[32508]: 3.7 scrub ok 2024-09-20T19:06:18.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:17 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1806455295' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]': finished 2024-09-20T19:06:18.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:17 smithi142 ceph-mon[32508]: osdmap e422: 8 total, 8 up, 8 in 2024-09-20T19:06:18.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:17 smithi142 ceph-mon[32508]: pgmap v832: 148 pgs: 16 unknown, 132 active+clean; 583 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-20T19:06:18.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:17 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:06:18.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:17 smithi083 ceph-mon[29421]: 5.1c scrub starts 2024-09-20T19:06:18.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:17 smithi083 ceph-mon[29421]: 5.1c scrub ok 2024-09-20T19:06:18.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:17 smithi083 ceph-mon[29421]: 3.7 scrub starts 2024-09-20T19:06:18.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:17 smithi083 ceph-mon[29421]: 3.7 scrub ok 2024-09-20T19:06:18.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:17 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1806455295' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]': finished 2024-09-20T19:06:18.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:17 smithi083 ceph-mon[29421]: osdmap e422: 8 total, 8 up, 8 in 2024-09-20T19:06:18.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:17 smithi083 ceph-mon[29421]: pgmap v832: 148 pgs: 16 unknown, 132 active+clean; 583 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-20T19:06:18.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:17 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:06:18.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:17 smithi083 ceph-mon[36260]: 5.1c scrub starts 2024-09-20T19:06:18.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:17 smithi083 ceph-mon[36260]: 5.1c scrub ok 2024-09-20T19:06:18.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:17 smithi083 ceph-mon[36260]: 3.7 scrub starts 2024-09-20T19:06:18.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:17 smithi083 ceph-mon[36260]: 3.7 scrub ok 2024-09-20T19:06:18.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:17 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1806455295' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]': finished 2024-09-20T19:06:18.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:17 smithi083 ceph-mon[36260]: osdmap e422: 8 total, 8 up, 8 in 2024-09-20T19:06:18.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:17 smithi083 ceph-mon[36260]: pgmap v832: 148 pgs: 16 unknown, 132 active+clean; 583 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-20T19:06:18.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:17 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:06:18.331 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:06:18.343 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:18.347 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 7, 6, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 5, 6, 7, 4] 2024-09-20T19:06:18.347 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T19:06:18.347 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-20T19:06:18.347 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd dump -f json-pretty 2024-09-20T19:06:18.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:18.370 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:18.371 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:18.471 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:18.493 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:18.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:18.494 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:18.595 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:18.617 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:18.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:18.618 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:18.719 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:18.744 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:18.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:18.745 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:18.846 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:18.871 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:18.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:18.872 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:18.973 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:18.975 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T19:06:18.997 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:18.998 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:18 smithi083 ceph-mon[29421]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-20T19:06:18.998 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:18 smithi083 ceph-mon[29421]: osdmap e423: 8 total, 8 up, 8 in 2024-09-20T19:06:18.999 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:18 smithi083 ceph-mon[36260]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-20T19:06:18.999 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:18 smithi083 ceph-mon[36260]: osdmap e423: 8 total, 8 up, 8 in 2024-09-20T19:06:18.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:18.999 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:19.100 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:19.116 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-20T19:06:19.124 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:19.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:19.126 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:19.227 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:19.248 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:19.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:19.249 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:19.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:18 smithi142 ceph-mon[32508]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-20T19:06:19.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:18 smithi142 ceph-mon[32508]: osdmap e423: 8 total, 8 up, 8 in 2024-09-20T19:06:19.350 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:19.371 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:19.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:19.372 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:19.379 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:06:19.473 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:19.480 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:06:19.501 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:19.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:19.502 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:19.519 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:06:19.603 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:19.626 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:19.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:19.627 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:19.728 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:19.750 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:19.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:19.751 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:19.852 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:19.874 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:19.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:19.875 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:19.911 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T19:06:19.911 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-09-20T19:06:19.911 INFO:teuthology.orchestra.run.smithi142.stdout: "num_ops": 0 2024-09-20T19:06:19.911 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T19:06:19.976 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:19.997 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:19.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:19.998 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:20.021 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:19 smithi083 ceph-mon[29421]: 6.1f scrub starts 2024-09-20T19:06:20.022 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:19 smithi083 ceph-mon[29421]: 6.1f scrub ok 2024-09-20T19:06:20.022 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:19 smithi083 ceph-mon[29421]: osdmap e424: 8 total, 8 up, 8 in 2024-09-20T19:06:20.022 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:19 smithi083 ceph-mon[29421]: pgmap v835: 148 pgs: 1 creating+peering, 15 unknown, 132 active+clean; 583 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T19:06:20.022 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:19 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/4123872888' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:06:20.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:19 smithi083 ceph-mon[36260]: 6.1f scrub starts 2024-09-20T19:06:20.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:19 smithi083 ceph-mon[36260]: 6.1f scrub ok 2024-09-20T19:06:20.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:19 smithi083 ceph-mon[36260]: osdmap e424: 8 total, 8 up, 8 in 2024-09-20T19:06:20.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:19 smithi083 ceph-mon[36260]: pgmap v835: 148 pgs: 1 creating+peering, 15 unknown, 132 active+clean; 583 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T19:06:20.023 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:19 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/4123872888' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:06:20.061 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-20T19:06:20.099 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:20.120 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:20.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:20.121 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:20.222 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:20.243 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:20.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:20.244 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:20.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:19 smithi142 ceph-mon[32508]: 6.1f scrub starts 2024-09-20T19:06:20.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:19 smithi142 ceph-mon[32508]: 6.1f scrub ok 2024-09-20T19:06:20.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:19 smithi142 ceph-mon[32508]: osdmap e424: 8 total, 8 up, 8 in 2024-09-20T19:06:20.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:19 smithi142 ceph-mon[32508]: pgmap v835: 148 pgs: 1 creating+peering, 15 unknown, 132 active+clean; 583 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T19:06:20.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:19 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/4123872888' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:06:20.344 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:20.365 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:20.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:20.366 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:20.467 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:20.489 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:20.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:20.490 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:20.591 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:20.612 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:20.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:20.613 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:20.666 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_blocked_ops 2024-09-20T19:06:20.714 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:20.736 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:20.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:20.736 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:20.837 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:20.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:20.860 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:20.861 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:20.922 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:06:20.961 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:20.983 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:20.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:20.984 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:21.041 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:06:20 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:06:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:06:21.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:20 smithi083 ceph-mon[36260]: 4.a scrub starts 2024-09-20T19:06:21.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:20 smithi083 ceph-mon[36260]: 4.a scrub ok 2024-09-20T19:06:21.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:20 smithi083 ceph-mon[36260]: 4.17 scrub starts 2024-09-20T19:06:21.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:20 smithi083 ceph-mon[36260]: 4.17 scrub ok 2024-09-20T19:06:21.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:20 smithi083 ceph-mon[36260]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T19:06:21.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:20 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/4123872888' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:06:21.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:20 smithi083 ceph-mon[36260]: osdmap e425: 8 total, 8 up, 8 in 2024-09-20T19:06:21.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:20 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1024888968' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-20T19:06:21.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:20 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-20T19:06:21.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:20 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:06:21.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:20 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:06:21.043 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:20 smithi083 ceph-mon[29421]: 4.a scrub starts 2024-09-20T19:06:21.043 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:20 smithi083 ceph-mon[29421]: 4.a scrub ok 2024-09-20T19:06:21.043 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:20 smithi083 ceph-mon[29421]: 4.17 scrub starts 2024-09-20T19:06:21.043 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:20 smithi083 ceph-mon[29421]: 4.17 scrub ok 2024-09-20T19:06:21.043 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:20 smithi083 ceph-mon[29421]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T19:06:21.043 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:20 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/4123872888' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:06:21.043 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:20 smithi083 ceph-mon[29421]: osdmap e425: 8 total, 8 up, 8 in 2024-09-20T19:06:21.043 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:20 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1024888968' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-20T19:06:21.043 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:20 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-20T19:06:21.043 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:20 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:06:21.043 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:20 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:06:21.071 INFO:teuthology.orchestra.run.smithi083.stderr:enabled application 'rados' on pool 'unique_pool_4' 2024-09-20T19:06:21.085 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:21.107 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:21.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:21.108 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:21.208 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:21.229 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:21.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:21.230 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:21.244 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T19:06:21.244 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T19:06:21.244 INFO:teuthology.orchestra.run.smithi083.stdout: "epoch": 426, 2024-09-20T19:06:21.244 INFO:teuthology.orchestra.run.smithi083.stdout: "fsid": "1ed5a50a-7780-11ef-baf5-efdc52797490", 2024-09-20T19:06:21.244 INFO:teuthology.orchestra.run.smithi083.stdout: "created": "2024-09-20T18:44:10.399800+0000", 2024-09-20T19:06:21.244 INFO:teuthology.orchestra.run.smithi083.stdout: "modified": "2024-09-20T19:06:21.057032+0000", 2024-09-20T19:06:21.244 INFO:teuthology.orchestra.run.smithi083.stdout: "last_up_change": "2024-09-20T19:01:21.809732+0000", 2024-09-20T19:06:21.244 INFO:teuthology.orchestra.run.smithi083.stdout: "last_in_change": "2024-09-20T19:04:20.299488+0000", 2024-09-20T19:06:21.244 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-20T19:06:21.244 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_num": 5799936, 2024-09-20T19:06:21.245 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_set": [ 2024-09-20T19:06:21.245 INFO:teuthology.orchestra.run.smithi083.stdout: "pglog_hardlimit", 2024-09-20T19:06:21.245 INFO:teuthology.orchestra.run.smithi083.stdout: "purged_snapdirs", 2024-09-20T19:06:21.245 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_deletes", 2024-09-20T19:06:21.245 INFO:teuthology.orchestra.run.smithi083.stdout: "sortbitwise" 2024-09-20T19:06:21.245 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T19:06:21.245 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_version": 19, 2024-09-20T19:06:21.245 INFO:teuthology.orchestra.run.smithi083.stdout: "full_ratio": 0.94999998807907104, 2024-09-20T19:06:21.245 INFO:teuthology.orchestra.run.smithi083.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-20T19:06:21.245 INFO:teuthology.orchestra.run.smithi083.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-20T19:06:21.245 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_snapshot": "", 2024-09-20T19:06:21.245 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_max": 11, 2024-09-20T19:06:21.245 INFO:teuthology.orchestra.run.smithi083.stdout: "max_osd": 8, 2024-09-20T19:06:21.245 INFO:teuthology.orchestra.run.smithi083.stdout: "require_min_compat_client": "luminous", 2024-09-20T19:06:21.245 INFO:teuthology.orchestra.run.smithi083.stdout: "min_compat_client": "luminous", 2024-09-20T19:06:21.246 INFO:teuthology.orchestra.run.smithi083.stdout: "require_osd_release": "squid", 2024-09-20T19:06:21.246 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_crimson": false, 2024-09-20T19:06:21.246 INFO:teuthology.orchestra.run.smithi083.stdout: "pools": [ 2024-09-20T19:06:21.246 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.246 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 1, 2024-09-20T19:06:21.246 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": ".mgr", 2024-09-20T19:06:21.246 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-20T18:47:15.285795+0000", 2024-09-20T19:06:21.246 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-20T19:06:21.246 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool", 2024-09-20T19:06:21.246 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-20T19:06:21.246 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 3, 2024-09-20T19:06:21.246 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 2, 2024-09-20T19:06:21.246 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-20T19:06:21.246 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-20T19:06:21.246 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-20T19:06:21.247 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T19:06:21.247 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T19:06:21.247 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-20T19:06:21.247 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-20T19:06:21.247 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-20T19:06:21.247 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 1, 2024-09-20T19:06:21.247 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 1, 2024-09-20T19:06:21.247 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 1, 2024-09-20T19:06:21.247 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 1, 2024-09-20T19:06:21.247 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 1, 2024-09-20T19:06:21.247 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-20T19:06:21.247 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-20T19:06:21.247 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-20T19:06:21.247 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-20T19:06:21.248 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-20T19:06:21.248 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-20T19:06:21.248 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-20T19:06:21.248 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.248 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "361", 2024-09-20T19:06:21.248 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-20T19:06:21.248 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T19:06:21.248 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T19:06:21.248 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-20T19:06:21.248 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-20T19:06:21.248 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 0, 2024-09-20T19:06:21.249 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 0, 2024-09-20T19:06:21.249 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-20T19:06:21.249 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-20T19:06:21.249 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-20T19:06:21.249 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-20T19:06:21.249 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-20T19:06:21.249 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-20T19:06:21.249 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-20T19:06:21.249 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-20T19:06:21.249 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-20T19:06:21.249 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-20T19:06:21.249 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-20T19:06:21.249 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T19:06:21.249 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T19:06:21.249 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T19:06:21.250 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-20T19:06:21.250 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-20T19:06:21.250 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-20T19:06:21.250 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-20T19:06:21.250 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-20T19:06:21.250 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.250 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-20T19:06:21.250 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-20T19:06:21.250 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-20T19:06:21.250 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-20T19:06:21.250 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-20T19:06:21.251 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T19:06:21.251 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-20T19:06:21.251 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-20T19:06:21.251 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-20T19:06:21.251 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-20T19:06:21.251 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-20T19:06:21.251 INFO:teuthology.orchestra.run.smithi083.stdout: "options": { 2024-09-20T19:06:21.251 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_max": 32, 2024-09-20T19:06:21.251 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_min": 1 2024-09-20T19:06:21.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.251 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-20T19:06:21.251 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": {} 2024-09-20T19:06:21.252 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.252 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-20T19:06:21.252 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-20T19:06:21.252 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 7.8899998664855957, 2024-09-20T19:06:21.252 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 7.8899998664855957, 2024-09-20T19:06:21.252 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 0.37999999523162842, 2024-09-20T19:06:21.252 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 3, 2024-09-20T19:06:21.252 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 3, 2024-09-20T19:06:21.252 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 1, 2024-09-20T19:06:21.252 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 1, 2024-09-20T19:06:21.252 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 1 2024-09-20T19:06:21.252 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.252 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.252 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.253 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 2, 2024-09-20T19:06:21.253 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": "datapool", 2024-09-20T19:06:21.253 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-20T18:49:43.452984+0000", 2024-09-20T19:06:21.253 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 8193, 2024-09-20T19:06:21.253 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-20T19:06:21.253 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-20T19:06:21.253 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 3, 2024-09-20T19:06:21.253 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 2, 2024-09-20T19:06:21.253 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-20T19:06:21.253 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-20T19:06:21.253 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-20T19:06:21.253 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T19:06:21.253 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T19:06:21.253 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-20T19:06:21.253 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-20T19:06:21.254 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-20T19:06:21.254 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 3, 2024-09-20T19:06:21.254 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 3, 2024-09-20T19:06:21.254 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 3, 2024-09-20T19:06:21.254 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 3, 2024-09-20T19:06:21.254 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 3, 2024-09-20T19:06:21.254 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-20T19:06:21.254 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-20T19:06:21.254 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-20T19:06:21.254 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-20T19:06:21.254 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-20T19:06:21.254 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-20T19:06:21.254 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-20T19:06:21.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.255 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "61", 2024-09-20T19:06:21.255 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-20T19:06:21.255 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T19:06:21.255 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T19:06:21.255 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-20T19:06:21.255 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-20T19:06:21.255 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 2, 2024-09-20T19:06:21.255 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 61, 2024-09-20T19:06:21.255 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-20T19:06:21.255 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-20T19:06:21.255 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-20T19:06:21.255 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-20T19:06:21.256 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-20T19:06:21.256 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-20T19:06:21.256 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-20T19:06:21.256 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-20T19:06:21.256 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-20T19:06:21.256 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-20T19:06:21.256 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-20T19:06:21.256 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T19:06:21.256 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T19:06:21.256 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T19:06:21.256 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-20T19:06:21.256 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-20T19:06:21.256 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-20T19:06:21.256 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-20T19:06:21.256 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-20T19:06:21.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.257 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-20T19:06:21.257 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-20T19:06:21.257 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-20T19:06:21.257 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-20T19:06:21.257 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-20T19:06:21.257 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T19:06:21.257 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-20T19:06:21.257 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-20T19:06:21.257 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-20T19:06:21.257 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-20T19:06:21.257 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-20T19:06:21.257 INFO:teuthology.orchestra.run.smithi083.stdout: "options": {}, 2024-09-20T19:06:21.257 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-20T19:06:21.257 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd": {} 2024-09-20T19:06:21.257 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.257 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-20T19:06:21.258 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-20T19:06:21.258 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 2.6500000953674316, 2024-09-20T19:06:21.258 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 2.6500000953674316, 2024-09-20T19:06:21.258 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 0.87999999523162842, 2024-09-20T19:06:21.258 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 2.3299999237060547, 2024-09-20T19:06:21.258 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 2.3299999237060547, 2024-09-20T19:06:21.258 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 1, 2024-09-20T19:06:21.258 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 1, 2024-09-20T19:06:21.258 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 1 2024-09-20T19:06:21.258 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.258 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.258 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.258 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 3, 2024-09-20T19:06:21.259 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": ".rgw.root", 2024-09-20T19:06:21.259 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-20T18:49:44.217011+0000", 2024-09-20T19:06:21.259 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-20T19:06:21.259 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool", 2024-09-20T19:06:21.259 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-20T19:06:21.259 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 3, 2024-09-20T19:06:21.259 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 2, 2024-09-20T19:06:21.259 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-20T19:06:21.259 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-20T19:06:21.259 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-20T19:06:21.259 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T19:06:21.260 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T19:06:21.260 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-20T19:06:21.260 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-20T19:06:21.260 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-20T19:06:21.260 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 32, 2024-09-20T19:06:21.260 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 32, 2024-09-20T19:06:21.260 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 32, 2024-09-20T19:06:21.260 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 32, 2024-09-20T19:06:21.260 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 32, 2024-09-20T19:06:21.260 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-20T19:06:21.260 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-20T19:06:21.260 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-20T19:06:21.260 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-20T19:06:21.260 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-20T19:06:21.261 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-20T19:06:21.261 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-20T19:06:21.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.261 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "60", 2024-09-20T19:06:21.261 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-20T19:06:21.262 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T19:06:21.262 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T19:06:21.262 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-20T19:06:21.262 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-20T19:06:21.262 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 0, 2024-09-20T19:06:21.262 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 0, 2024-09-20T19:06:21.262 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-20T19:06:21.262 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-20T19:06:21.262 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-20T19:06:21.262 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-20T19:06:21.262 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-20T19:06:21.262 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-20T19:06:21.262 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-20T19:06:21.263 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-20T19:06:21.263 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-20T19:06:21.263 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-20T19:06:21.263 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-20T19:06:21.263 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T19:06:21.263 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T19:06:21.263 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T19:06:21.263 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-20T19:06:21.263 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-20T19:06:21.263 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-20T19:06:21.263 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-20T19:06:21.263 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-20T19:06:21.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.263 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-20T19:06:21.264 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-20T19:06:21.264 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-20T19:06:21.264 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-20T19:06:21.264 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-20T19:06:21.264 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T19:06:21.264 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-20T19:06:21.264 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-20T19:06:21.264 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-20T19:06:21.264 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-20T19:06:21.264 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-20T19:06:21.264 INFO:teuthology.orchestra.run.smithi083.stdout: "options": {}, 2024-09-20T19:06:21.264 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-20T19:06:21.265 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw": {} 2024-09-20T19:06:21.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.265 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-20T19:06:21.265 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-20T19:06:21.265 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 1.5, 2024-09-20T19:06:21.265 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 1.5, 2024-09-20T19:06:21.265 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 1, 2024-09-20T19:06:21.265 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 1.5, 2024-09-20T19:06:21.265 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 1.5, 2024-09-20T19:06:21.265 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 1, 2024-09-20T19:06:21.265 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 1, 2024-09-20T19:06:21.265 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 1 2024-09-20T19:06:21.265 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.265 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 4, 2024-09-20T19:06:21.266 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": "default.rgw.log", 2024-09-20T19:06:21.266 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-20T18:49:46.007275+0000", 2024-09-20T19:06:21.266 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-20T19:06:21.266 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool", 2024-09-20T19:06:21.266 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-20T19:06:21.266 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 3, 2024-09-20T19:06:21.266 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 2, 2024-09-20T19:06:21.266 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-20T19:06:21.266 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-20T19:06:21.266 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-20T19:06:21.266 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T19:06:21.266 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T19:06:21.266 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-20T19:06:21.266 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-20T19:06:21.266 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-20T19:06:21.267 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 32, 2024-09-20T19:06:21.267 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 32, 2024-09-20T19:06:21.267 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 32, 2024-09-20T19:06:21.267 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 32, 2024-09-20T19:06:21.267 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 32, 2024-09-20T19:06:21.267 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-20T19:06:21.267 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-20T19:06:21.267 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-20T19:06:21.267 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-20T19:06:21.267 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-20T19:06:21.267 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-20T19:06:21.267 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-20T19:06:21.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.267 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "62", 2024-09-20T19:06:21.267 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-20T19:06:21.268 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T19:06:21.268 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T19:06:21.268 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-20T19:06:21.268 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-20T19:06:21.268 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 0, 2024-09-20T19:06:21.268 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 0, 2024-09-20T19:06:21.268 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-20T19:06:21.268 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-20T19:06:21.268 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-20T19:06:21.268 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-20T19:06:21.268 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-20T19:06:21.268 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-20T19:06:21.268 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-20T19:06:21.268 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-20T19:06:21.268 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-20T19:06:21.269 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-20T19:06:21.269 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-20T19:06:21.269 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T19:06:21.269 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T19:06:21.269 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T19:06:21.269 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-20T19:06:21.269 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-20T19:06:21.269 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-20T19:06:21.269 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-20T19:06:21.269 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-20T19:06:21.269 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.270 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-20T19:06:21.270 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-20T19:06:21.270 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-20T19:06:21.270 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-20T19:06:21.270 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-20T19:06:21.270 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T19:06:21.270 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-20T19:06:21.270 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-20T19:06:21.270 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-20T19:06:21.270 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-20T19:06:21.270 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-20T19:06:21.270 INFO:teuthology.orchestra.run.smithi083.stdout: "options": {}, 2024-09-20T19:06:21.270 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-20T19:06:21.271 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw": {} 2024-09-20T19:06:21.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.271 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-20T19:06:21.271 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-20T19:06:21.271 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 2.25, 2024-09-20T19:06:21.271 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 2.25, 2024-09-20T19:06:21.271 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 1, 2024-09-20T19:06:21.271 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 2.25, 2024-09-20T19:06:21.271 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 2.25, 2024-09-20T19:06:21.271 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 1, 2024-09-20T19:06:21.271 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 1, 2024-09-20T19:06:21.271 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 1 2024-09-20T19:06:21.271 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.271 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.272 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 5, 2024-09-20T19:06:21.272 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": "default.rgw.control", 2024-09-20T19:06:21.272 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-20T18:49:47.961258+0000", 2024-09-20T19:06:21.272 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-20T19:06:21.272 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool", 2024-09-20T19:06:21.272 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-20T19:06:21.272 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 3, 2024-09-20T19:06:21.272 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 2, 2024-09-20T19:06:21.272 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-20T19:06:21.272 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-20T19:06:21.272 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-20T19:06:21.272 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T19:06:21.272 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T19:06:21.273 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-20T19:06:21.273 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-20T19:06:21.273 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-20T19:06:21.273 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 32, 2024-09-20T19:06:21.273 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 32, 2024-09-20T19:06:21.273 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 32, 2024-09-20T19:06:21.273 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 32, 2024-09-20T19:06:21.273 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 32, 2024-09-20T19:06:21.273 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-20T19:06:21.273 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-20T19:06:21.273 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-20T19:06:21.273 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-20T19:06:21.273 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-20T19:06:21.273 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-20T19:06:21.274 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-20T19:06:21.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.274 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "64", 2024-09-20T19:06:21.274 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-20T19:06:21.274 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T19:06:21.274 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T19:06:21.274 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-20T19:06:21.274 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-20T19:06:21.274 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 0, 2024-09-20T19:06:21.274 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 0, 2024-09-20T19:06:21.275 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-20T19:06:21.275 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-20T19:06:21.275 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-20T19:06:21.275 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-20T19:06:21.275 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-20T19:06:21.275 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-20T19:06:21.275 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-20T19:06:21.276 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-20T19:06:21.276 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-20T19:06:21.276 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-20T19:06:21.276 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-20T19:06:21.276 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T19:06:21.276 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T19:06:21.276 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T19:06:21.277 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-20T19:06:21.277 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-20T19:06:21.277 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-20T19:06:21.277 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-20T19:06:21.277 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-20T19:06:21.277 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.277 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-20T19:06:21.278 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-20T19:06:21.278 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-20T19:06:21.278 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-20T19:06:21.278 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-20T19:06:21.278 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T19:06:21.278 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-20T19:06:21.278 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-20T19:06:21.279 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-20T19:06:21.279 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-20T19:06:21.279 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-20T19:06:21.279 INFO:teuthology.orchestra.run.smithi083.stdout: "options": {}, 2024-09-20T19:06:21.279 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-20T19:06:21.279 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw": {} 2024-09-20T19:06:21.280 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.280 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-20T19:06:21.280 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-20T19:06:21.280 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 1.25, 2024-09-20T19:06:21.280 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 1.25, 2024-09-20T19:06:21.280 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 1, 2024-09-20T19:06:21.281 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 1.25, 2024-09-20T19:06:21.281 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 1.25, 2024-09-20T19:06:21.281 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 1, 2024-09-20T19:06:21.281 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 1, 2024-09-20T19:06:21.281 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 1 2024-09-20T19:06:21.281 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.281 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.282 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.282 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 6, 2024-09-20T19:06:21.282 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": "default.rgw.meta", 2024-09-20T19:06:21.282 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-20T18:49:50.011276+0000", 2024-09-20T19:06:21.282 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-20T19:06:21.282 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool", 2024-09-20T19:06:21.283 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-20T19:06:21.283 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 3, 2024-09-20T19:06:21.283 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 2, 2024-09-20T19:06:21.283 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-20T19:06:21.283 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-20T19:06:21.283 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-20T19:06:21.284 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T19:06:21.284 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T19:06:21.284 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-20T19:06:21.284 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-20T19:06:21.284 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-20T19:06:21.284 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 32, 2024-09-20T19:06:21.284 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 32, 2024-09-20T19:06:21.285 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 32, 2024-09-20T19:06:21.285 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 32, 2024-09-20T19:06:21.285 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 32, 2024-09-20T19:06:21.285 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-20T19:06:21.285 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-20T19:06:21.285 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-20T19:06:21.286 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-20T19:06:21.286 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-20T19:06:21.286 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-20T19:06:21.286 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-20T19:06:21.286 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.286 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "66", 2024-09-20T19:06:21.286 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-20T19:06:21.287 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T19:06:21.287 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T19:06:21.287 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-20T19:06:21.287 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-20T19:06:21.287 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 0, 2024-09-20T19:06:21.287 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 0, 2024-09-20T19:06:21.288 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-20T19:06:21.288 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-20T19:06:21.288 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-20T19:06:21.288 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-20T19:06:21.288 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-20T19:06:21.288 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-20T19:06:21.288 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-20T19:06:21.288 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-20T19:06:21.289 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-20T19:06:21.289 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-20T19:06:21.289 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-20T19:06:21.289 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T19:06:21.289 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T19:06:21.289 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T19:06:21.289 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-20T19:06:21.290 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-20T19:06:21.290 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-20T19:06:21.290 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-20T19:06:21.290 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-20T19:06:21.290 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.290 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-20T19:06:21.290 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-20T19:06:21.290 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-20T19:06:21.291 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-20T19:06:21.291 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-20T19:06:21.291 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T19:06:21.291 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-20T19:06:21.291 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-20T19:06:21.291 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-20T19:06:21.291 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-20T19:06:21.291 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-20T19:06:21.291 INFO:teuthology.orchestra.run.smithi083.stdout: "options": { 2024-09-20T19:06:21.291 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_bias": 4 2024-09-20T19:06:21.291 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.291 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-20T19:06:21.291 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw": {} 2024-09-20T19:06:21.291 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.292 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-20T19:06:21.292 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-20T19:06:21.292 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 1.75, 2024-09-20T19:06:21.292 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 1.75, 2024-09-20T19:06:21.292 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 1, 2024-09-20T19:06:21.292 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 1.75, 2024-09-20T19:06:21.292 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 1.75, 2024-09-20T19:06:21.292 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 1, 2024-09-20T19:06:21.292 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 1, 2024-09-20T19:06:21.292 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 1 2024-09-20T19:06:21.292 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.292 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.292 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.292 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 11, 2024-09-20T19:06:21.292 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": "unique_pool_4", 2024-09-20T19:06:21.293 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-20T19:06:16.062904+0000", 2024-09-20T19:06:21.293 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-20T19:06:21.293 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool", 2024-09-20T19:06:21.293 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-20T19:06:21.293 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 3, 2024-09-20T19:06:21.293 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 2, 2024-09-20T19:06:21.293 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-20T19:06:21.293 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-20T19:06:21.293 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-20T19:06:21.293 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T19:06:21.293 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T19:06:21.293 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-20T19:06:21.293 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-20T19:06:21.293 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-20T19:06:21.293 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 16, 2024-09-20T19:06:21.293 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 16, 2024-09-20T19:06:21.294 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 16, 2024-09-20T19:06:21.294 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 16, 2024-09-20T19:06:21.294 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 16, 2024-09-20T19:06:21.294 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-20T19:06:21.294 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-20T19:06:21.294 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-20T19:06:21.294 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-20T19:06:21.294 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-20T19:06:21.294 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-20T19:06:21.294 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-20T19:06:21.294 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.294 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "426", 2024-09-20T19:06:21.294 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-20T19:06:21.295 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T19:06:21.295 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T19:06:21.295 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-20T19:06:21.295 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-20T19:06:21.295 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 0, 2024-09-20T19:06:21.295 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 0, 2024-09-20T19:06:21.295 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-20T19:06:21.295 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-20T19:06:21.295 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-20T19:06:21.295 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-20T19:06:21.295 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-20T19:06:21.295 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-20T19:06:21.295 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-20T19:06:21.295 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-20T19:06:21.295 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-20T19:06:21.296 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-20T19:06:21.296 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-20T19:06:21.296 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T19:06:21.296 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T19:06:21.296 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T19:06:21.296 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-20T19:06:21.296 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-20T19:06:21.296 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-20T19:06:21.296 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-20T19:06:21.296 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-20T19:06:21.296 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.296 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-20T19:06:21.296 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-20T19:06:21.296 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-20T19:06:21.297 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-20T19:06:21.297 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-20T19:06:21.297 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T19:06:21.297 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-20T19:06:21.297 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-20T19:06:21.297 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-20T19:06:21.297 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-20T19:06:21.297 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-20T19:06:21.297 INFO:teuthology.orchestra.run.smithi083.stdout: "options": {}, 2024-09-20T19:06:21.297 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-20T19:06:21.297 INFO:teuthology.orchestra.run.smithi083.stdout: "rados": {} 2024-09-20T19:06:21.297 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.297 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-20T19:06:21.297 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-20T19:06:21.298 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 2, 2024-09-20T19:06:21.298 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 2, 2024-09-20T19:06:21.298 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 1, 2024-09-20T19:06:21.298 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 2, 2024-09-20T19:06:21.298 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 2, 2024-09-20T19:06:21.298 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 1, 2024-09-20T19:06:21.298 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 1, 2024-09-20T19:06:21.298 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 1 2024-09-20T19:06:21.298 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.298 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.298 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T19:06:21.298 INFO:teuthology.orchestra.run.smithi083.stdout: "osds": [ 2024-09-20T19:06:21.298 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.299 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 0, 2024-09-20T19:06:21.299 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "c714e1ec-f0c3-4054-9792-61b84e53eaa8", 2024-09-20T19:06:21.299 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T19:06:21.299 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T19:06:21.299 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T19:06:21.299 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T19:06:21.299 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T19:06:21.299 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T19:06:21.299 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 8, 2024-09-20T19:06:21.299 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 422, 2024-09-20T19:06:21.299 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T19:06:21.299 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T19:06:21.299 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T19:06:21.299 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:06:21.300 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.300 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:06:21.300 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6801", 2024-09-20T19:06:21.300 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2574084049 2024-09-20T19:06:21.300 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.300 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:21.300 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.300 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T19:06:21.300 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:06:21.300 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.300 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:06:21.300 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6802", 2024-09-20T19:06:21.300 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2574084049 2024-09-20T19:06:21.300 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.301 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:21.301 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.301 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T19:06:21.301 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:06:21.301 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.301 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:06:21.301 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6804", 2024-09-20T19:06:21.301 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2574084049 2024-09-20T19:06:21.301 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.301 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:21.301 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.301 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T19:06:21.301 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:06:21.301 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.301 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:06:21.301 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6803", 2024-09-20T19:06:21.302 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2574084049 2024-09-20T19:06:21.302 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.302 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:21.302 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.302 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.83:6801/2574084049", 2024-09-20T19:06:21.302 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.83:6802/2574084049", 2024-09-20T19:06:21.302 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.83:6804/2574084049", 2024-09-20T19:06:21.302 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.83:6803/2574084049", 2024-09-20T19:06:21.302 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T19:06:21.302 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T19:06:21.302 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T19:06:21.302 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:21.302 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.303 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.303 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 1, 2024-09-20T19:06:21.303 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "9dc0f2a8-bafa-4a42-88f5-0301dd58f78d", 2024-09-20T19:06:21.303 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T19:06:21.303 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T19:06:21.303 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T19:06:21.303 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T19:06:21.303 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T19:06:21.303 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T19:06:21.303 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 13, 2024-09-20T19:06:21.303 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 422, 2024-09-20T19:06:21.303 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T19:06:21.303 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T19:06:21.304 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T19:06:21.304 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:06:21.304 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.304 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:06:21.304 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6805", 2024-09-20T19:06:21.304 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 4213285798 2024-09-20T19:06:21.304 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.304 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:21.304 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.304 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T19:06:21.304 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:06:21.304 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.304 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:06:21.304 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6806", 2024-09-20T19:06:21.305 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 4213285798 2024-09-20T19:06:21.305 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.305 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:21.305 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.305 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T19:06:21.305 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:06:21.305 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.305 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:06:21.305 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6808", 2024-09-20T19:06:21.305 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 4213285798 2024-09-20T19:06:21.305 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.305 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:21.305 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.306 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T19:06:21.306 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:06:21.306 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.306 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:06:21.306 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6807", 2024-09-20T19:06:21.306 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 4213285798 2024-09-20T19:06:21.306 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.306 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:21.306 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.306 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.83:6805/4213285798", 2024-09-20T19:06:21.306 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.83:6806/4213285798", 2024-09-20T19:06:21.307 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.83:6808/4213285798", 2024-09-20T19:06:21.307 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.83:6807/4213285798", 2024-09-20T19:06:21.307 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T19:06:21.307 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T19:06:21.307 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T19:06:21.307 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:21.307 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.307 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.307 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 2, 2024-09-20T19:06:21.307 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "448ab692-ec41-44dc-b2d5-dbefd255b3fd", 2024-09-20T19:06:21.307 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T19:06:21.307 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T19:06:21.308 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T19:06:21.308 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T19:06:21.308 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T19:06:21.308 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T19:06:21.308 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 18, 2024-09-20T19:06:21.308 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 422, 2024-09-20T19:06:21.308 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T19:06:21.308 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T19:06:21.308 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T19:06:21.308 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:06:21.308 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.308 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:06:21.308 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6809", 2024-09-20T19:06:21.309 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 805947571 2024-09-20T19:06:21.309 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.309 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:21.309 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.309 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T19:06:21.309 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:06:21.309 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.309 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:06:21.309 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6810", 2024-09-20T19:06:21.309 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 805947571 2024-09-20T19:06:21.309 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.309 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:21.309 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.309 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T19:06:21.310 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:06:21.310 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.310 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:06:21.310 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6812", 2024-09-20T19:06:21.310 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 805947571 2024-09-20T19:06:21.310 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.310 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:21.310 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.310 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T19:06:21.310 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:06:21.310 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.310 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:06:21.311 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6811", 2024-09-20T19:06:21.311 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 805947571 2024-09-20T19:06:21.311 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.311 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:21.311 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.311 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.83:6809/805947571", 2024-09-20T19:06:21.311 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.83:6810/805947571", 2024-09-20T19:06:21.312 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.83:6812/805947571", 2024-09-20T19:06:21.312 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.83:6811/805947571", 2024-09-20T19:06:21.312 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T19:06:21.312 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T19:06:21.312 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T19:06:21.312 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:21.312 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.312 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.312 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 3, 2024-09-20T19:06:21.312 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "7cda5e50-e22e-4d4a-823b-53c8f5061193", 2024-09-20T19:06:21.312 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T19:06:21.312 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T19:06:21.313 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T19:06:21.313 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T19:06:21.313 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T19:06:21.313 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T19:06:21.313 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 25, 2024-09-20T19:06:21.313 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 422, 2024-09-20T19:06:21.313 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T19:06:21.313 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T19:06:21.313 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T19:06:21.313 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:06:21.313 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.313 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:06:21.313 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6813", 2024-09-20T19:06:21.313 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 987237318 2024-09-20T19:06:21.313 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.314 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:21.314 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.314 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T19:06:21.314 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:06:21.314 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.314 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:06:21.314 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6814", 2024-09-20T19:06:21.314 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 987237318 2024-09-20T19:06:21.314 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.314 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:21.314 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.315 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T19:06:21.315 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:06:21.315 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.315 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:06:21.315 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6816", 2024-09-20T19:06:21.315 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 987237318 2024-09-20T19:06:21.315 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.315 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:21.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.315 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T19:06:21.315 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:06:21.315 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.315 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:06:21.315 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6815", 2024-09-20T19:06:21.316 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 987237318 2024-09-20T19:06:21.316 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.316 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:21.316 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.316 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.83:6813/987237318", 2024-09-20T19:06:21.316 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.83:6814/987237318", 2024-09-20T19:06:21.316 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.83:6816/987237318", 2024-09-20T19:06:21.316 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.83:6815/987237318", 2024-09-20T19:06:21.316 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T19:06:21.317 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T19:06:21.317 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T19:06:21.317 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:21.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.317 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.317 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 4, 2024-09-20T19:06:21.317 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "3d62654d-d31d-4310-9934-49e694dd17d9", 2024-09-20T19:06:21.318 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T19:06:21.318 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T19:06:21.318 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T19:06:21.318 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T19:06:21.318 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 31, 2024-09-20T19:06:21.318 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 291, 2024-09-20T19:06:21.318 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 304, 2024-09-20T19:06:21.318 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 422, 2024-09-20T19:06:21.318 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 292, 2024-09-20T19:06:21.318 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T19:06:21.319 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T19:06:21.319 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:06:21.319 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.319 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:06:21.319 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6800", 2024-09-20T19:06:21.319 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3322919402 2024-09-20T19:06:21.319 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.319 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:21.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.319 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T19:06:21.319 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:06:21.319 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.319 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:06:21.319 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6801", 2024-09-20T19:06:21.320 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3322919402 2024-09-20T19:06:21.320 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.320 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:21.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.320 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T19:06:21.320 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:06:21.320 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.320 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:06:21.320 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6803", 2024-09-20T19:06:21.320 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3322919402 2024-09-20T19:06:21.320 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.320 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:21.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.320 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T19:06:21.320 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:06:21.321 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.321 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:06:21.321 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6802", 2024-09-20T19:06:21.321 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3322919402 2024-09-20T19:06:21.321 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.321 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:21.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.321 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.142:6800/3322919402", 2024-09-20T19:06:21.321 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.142:6801/3322919402", 2024-09-20T19:06:21.321 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.142:6803/3322919402", 2024-09-20T19:06:21.321 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.142:6802/3322919402", 2024-09-20T19:06:21.321 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T19:06:21.321 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T19:06:21.321 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T19:06:21.322 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:21.322 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.322 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.322 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 5, 2024-09-20T19:06:21.322 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "b3e4e21b-639f-4d25-8bbf-ea8922a9222a", 2024-09-20T19:06:21.322 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T19:06:21.322 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T19:06:21.322 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T19:06:21.322 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T19:06:21.322 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T19:06:21.323 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T19:06:21.323 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 38, 2024-09-20T19:06:21.323 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 422, 2024-09-20T19:06:21.323 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T19:06:21.323 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T19:06:21.323 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T19:06:21.323 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:06:21.323 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.323 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:06:21.323 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6804", 2024-09-20T19:06:21.323 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2500466067 2024-09-20T19:06:21.323 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.323 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:21.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.324 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T19:06:21.324 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:06:21.324 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.324 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:06:21.324 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6805", 2024-09-20T19:06:21.324 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2500466067 2024-09-20T19:06:21.324 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.324 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:21.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.324 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T19:06:21.324 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:06:21.324 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.324 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:06:21.324 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6807", 2024-09-20T19:06:21.325 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2500466067 2024-09-20T19:06:21.325 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:20 smithi142 ceph-mon[32508]: 4.a scrub starts 2024-09-20T19:06:21.325 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:20 smithi142 ceph-mon[32508]: 4.a scrub ok 2024-09-20T19:06:21.325 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:20 smithi142 ceph-mon[32508]: 4.17 scrub starts 2024-09-20T19:06:21.326 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:20 smithi142 ceph-mon[32508]: 4.17 scrub ok 2024-09-20T19:06:21.326 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:20 smithi142 ceph-mon[32508]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T19:06:21.326 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:20 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/4123872888' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:06:21.326 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:20 smithi142 ceph-mon[32508]: osdmap e425: 8 total, 8 up, 8 in 2024-09-20T19:06:21.326 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:20 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1024888968' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-20T19:06:21.326 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:20 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-20T19:06:21.326 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:20 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:06:21.326 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:20 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:06:21.326 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.326 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:21.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.327 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T19:06:21.327 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:06:21.327 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.327 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:06:21.327 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6806", 2024-09-20T19:06:21.327 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2500466067 2024-09-20T19:06:21.327 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.327 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:21.327 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.327 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.142:6804/2500466067", 2024-09-20T19:06:21.327 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.142:6805/2500466067", 2024-09-20T19:06:21.327 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.142:6807/2500466067", 2024-09-20T19:06:21.327 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.142:6806/2500466067", 2024-09-20T19:06:21.328 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T19:06:21.328 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T19:06:21.328 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T19:06:21.328 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:21.328 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.328 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.328 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 6, 2024-09-20T19:06:21.328 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "c5db7107-ccb4-4ae2-9e63-f444897942e3", 2024-09-20T19:06:21.328 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T19:06:21.328 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T19:06:21.328 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T19:06:21.328 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T19:06:21.328 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T19:06:21.329 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T19:06:21.329 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 45, 2024-09-20T19:06:21.329 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 422, 2024-09-20T19:06:21.329 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T19:06:21.329 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T19:06:21.329 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T19:06:21.329 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:06:21.329 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.329 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:06:21.329 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6808", 2024-09-20T19:06:21.329 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1688186226 2024-09-20T19:06:21.329 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.329 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:21.329 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.330 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T19:06:21.330 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:06:21.330 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.330 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:06:21.330 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6809", 2024-09-20T19:06:21.330 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1688186226 2024-09-20T19:06:21.330 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.330 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:21.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.330 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T19:06:21.330 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:06:21.330 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.330 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:06:21.331 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6811", 2024-09-20T19:06:21.331 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1688186226 2024-09-20T19:06:21.331 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.331 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:21.331 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.331 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T19:06:21.331 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:06:21.331 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.331 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:06:21.331 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6810", 2024-09-20T19:06:21.331 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1688186226 2024-09-20T19:06:21.331 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.331 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:21.331 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.332 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.142:6808/1688186226", 2024-09-20T19:06:21.332 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.142:6809/1688186226", 2024-09-20T19:06:21.332 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.142:6811/1688186226", 2024-09-20T19:06:21.332 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.142:6810/1688186226", 2024-09-20T19:06:21.332 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T19:06:21.332 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T19:06:21.332 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T19:06:21.332 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:21.332 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.332 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.332 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 7, 2024-09-20T19:06:21.333 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "07d16806-e2c8-4acb-b284-d75701e61622", 2024-09-20T19:06:21.333 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T19:06:21.333 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T19:06:21.333 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T19:06:21.333 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T19:06:21.333 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T19:06:21.333 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T19:06:21.333 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 53, 2024-09-20T19:06:21.333 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 422, 2024-09-20T19:06:21.333 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T19:06:21.333 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T19:06:21.333 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T19:06:21.333 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:06:21.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.334 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:06:21.334 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6812", 2024-09-20T19:06:21.334 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3679770984 2024-09-20T19:06:21.334 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.334 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:21.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.334 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T19:06:21.334 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:06:21.334 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.334 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:06:21.334 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6813", 2024-09-20T19:06:21.334 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3679770984 2024-09-20T19:06:21.335 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.335 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:21.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.335 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T19:06:21.335 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:06:21.335 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.335 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:06:21.335 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6815", 2024-09-20T19:06:21.335 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3679770984 2024-09-20T19:06:21.335 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.335 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:21.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.335 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T19:06:21.335 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:06:21.336 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.336 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:06:21.336 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6814", 2024-09-20T19:06:21.336 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3679770984 2024-09-20T19:06:21.336 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.336 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:21.336 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.336 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.142:6812/3679770984", 2024-09-20T19:06:21.336 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.142:6813/3679770984", 2024-09-20T19:06:21.336 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.142:6815/3679770984", 2024-09-20T19:06:21.336 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.142:6814/3679770984", 2024-09-20T19:06:21.336 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T19:06:21.336 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T19:06:21.336 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T19:06:21.336 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:21.337 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.337 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T19:06:21.337 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_xinfo": [ 2024-09-20T19:06:21.337 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.337 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 0, 2024-09-20T19:06:21.337 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T19:06:21.337 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T19:06:21.337 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T19:06:21.337 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T19:06:21.337 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T19:06:21.337 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T19:05:14.862999+0000", 2024-09-20T19:06:21.337 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T19:06:21.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.337 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.338 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 1, 2024-09-20T19:06:21.338 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T19:06:21.338 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T19:06:21.338 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T19:06:21.338 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T19:06:21.338 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T19:06:21.338 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T19:06:20.624438+0000", 2024-09-20T19:06:21.338 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T19:06:21.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.338 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.338 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 2, 2024-09-20T19:06:21.338 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T19:06:21.338 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T19:06:21.339 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T19:06:21.339 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T19:06:21.339 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T19:06:21.339 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T19:04:57.447245+0000", 2024-09-20T19:06:21.339 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T19:06:21.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.339 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.339 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 3, 2024-09-20T19:06:21.339 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T19:06:21.339 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T19:06:21.339 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T19:06:21.339 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T19:06:21.339 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T19:06:21.339 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T19:05:37.721563+0000", 2024-09-20T19:06:21.339 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T19:06:21.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.340 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 4, 2024-09-20T19:06:21.340 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T19:06:21.340 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T19:06:21.340 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T19:06:21.340 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T19:06:21.340 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T19:06:21.340 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T19:05:10.575535+0000", 2024-09-20T19:06:21.340 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 291 2024-09-20T19:06:21.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.340 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 5, 2024-09-20T19:06:21.340 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T19:06:21.340 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T19:06:21.341 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T19:06:21.341 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T19:06:21.341 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T19:06:21.341 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T19:06:01.752083+0000", 2024-09-20T19:06:21.341 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T19:06:21.341 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.341 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.341 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 6, 2024-09-20T19:06:21.341 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T19:06:21.341 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T19:06:21.341 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T19:06:21.341 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T19:06:21.341 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T19:06:21.341 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T19:05:08.941679+0000", 2024-09-20T19:06:21.341 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T19:06:21.342 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.342 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.342 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 7, 2024-09-20T19:06:21.342 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T19:06:21.342 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T19:06:21.342 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T19:06:21.342 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T19:06:21.342 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T19:06:21.342 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T19:06:20.160210+0000", 2024-09-20T19:06:21.342 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T19:06:21.342 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.342 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T19:06:21.342 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_upmap": [ 2024-09-20T19:06:21.343 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.343 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "3.c", 2024-09-20T19:06:21.343 INFO:teuthology.orchestra.run.smithi083.stdout: "osds": [ 2024-09-20T19:06:21.343 INFO:teuthology.orchestra.run.smithi083.stdout: 2, 2024-09-20T19:06:21.343 INFO:teuthology.orchestra.run.smithi083.stdout: 7, 2024-09-20T19:06:21.343 INFO:teuthology.orchestra.run.smithi083.stdout: 6 2024-09-20T19:06:21.343 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:21.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.343 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.343 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "4.2", 2024-09-20T19:06:21.343 INFO:teuthology.orchestra.run.smithi083.stdout: "osds": [ 2024-09-20T19:06:21.344 INFO:teuthology.orchestra.run.smithi083.stdout: 5, 2024-09-20T19:06:21.344 INFO:teuthology.orchestra.run.smithi083.stdout: 7, 2024-09-20T19:06:21.344 INFO:teuthology.orchestra.run.smithi083.stdout: 4 2024-09-20T19:06:21.344 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:21.344 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.344 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T19:06:21.344 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_upmap_items": [ 2024-09-20T19:06:21.344 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.344 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "4.4", 2024-09-20T19:06:21.344 INFO:teuthology.orchestra.run.smithi083.stdout: "mappings": [ 2024-09-20T19:06:21.344 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.344 INFO:teuthology.orchestra.run.smithi083.stdout: "from": 3, 2024-09-20T19:06:21.344 INFO:teuthology.orchestra.run.smithi083.stdout: "to": 5 2024-09-20T19:06:21.344 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.344 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:21.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.345 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.345 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "4.f", 2024-09-20T19:06:21.345 INFO:teuthology.orchestra.run.smithi083.stdout: "mappings": [ 2024-09-20T19:06:21.345 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.345 INFO:teuthology.orchestra.run.smithi083.stdout: "from": 1, 2024-09-20T19:06:21.345 INFO:teuthology.orchestra.run.smithi083.stdout: "to": 2 2024-09-20T19:06:21.345 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.345 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:21.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.345 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.345 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "4.14", 2024-09-20T19:06:21.345 INFO:teuthology.orchestra.run.smithi083.stdout: "mappings": [ 2024-09-20T19:06:21.345 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.345 INFO:teuthology.orchestra.run.smithi083.stdout: "from": 1, 2024-09-20T19:06:21.346 INFO:teuthology.orchestra.run.smithi083.stdout: "to": 5 2024-09-20T19:06:21.346 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.346 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:21.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.346 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.346 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "4.18", 2024-09-20T19:06:21.346 INFO:teuthology.orchestra.run.smithi083.stdout: "mappings": [ 2024-09-20T19:06:21.346 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.346 INFO:teuthology.orchestra.run.smithi083.stdout: "from": 1, 2024-09-20T19:06:21.346 INFO:teuthology.orchestra.run.smithi083.stdout: "to": 2 2024-09-20T19:06:21.346 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.346 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:21.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.346 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.347 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "4.1e", 2024-09-20T19:06:21.347 INFO:teuthology.orchestra.run.smithi083.stdout: "mappings": [ 2024-09-20T19:06:21.347 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:21.347 INFO:teuthology.orchestra.run.smithi083.stdout: "from": 3, 2024-09-20T19:06:21.347 INFO:teuthology.orchestra.run.smithi083.stdout: "to": 2 2024-09-20T19:06:21.347 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.347 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:21.347 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.347 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T19:06:21.347 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_upmap_primaries": [], 2024-09-20T19:06:21.347 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_temp": [], 2024-09-20T19:06:21.347 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_temp": [], 2024-09-20T19:06:21.347 INFO:teuthology.orchestra.run.smithi083.stdout: "blocklist": { 2024-09-20T19:06:21.347 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/3474253603": "2024-09-21T18:44:32.434180+0000", 2024-09-20T19:06:21.348 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:6800/3525856829": "2024-09-21T18:44:32.434180+0000", 2024-09-20T19:06:21.348 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/220746284": "2024-09-21T18:44:32.434180+0000", 2024-09-20T19:06:21.348 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/3875445393": "2024-09-21T18:44:56.985620+0000", 2024-09-20T19:06:21.348 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/3009092807": "2024-09-21T18:44:56.985620+0000", 2024-09-20T19:06:21.348 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/4267132131": "2024-09-21T18:44:56.985620+0000", 2024-09-20T19:06:21.348 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/3684382972": "2024-09-21T18:50:14.729526+0000", 2024-09-20T19:06:21.348 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/3132933178": "2024-09-21T18:50:14.729526+0000", 2024-09-20T19:06:21.348 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:6800/1007846146": "2024-09-21T18:50:14.729526+0000", 2024-09-20T19:06:21.348 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:6800/2985596191": "2024-09-21T18:44:56.985620+0000", 2024-09-20T19:06:21.348 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/2410768464": "2024-09-21T18:50:14.729526+0000", 2024-09-20T19:06:21.348 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/1868694097": "2024-09-21T18:50:14.729526+0000", 2024-09-20T19:06:21.348 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/3777331122": "2024-09-21T18:44:32.434180+0000", 2024-09-20T19:06:21.348 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/2161728215": "2024-09-21T18:50:14.729526+0000" 2024-09-20T19:06:21.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.348 INFO:teuthology.orchestra.run.smithi083.stdout: "range_blocklist": {}, 2024-09-20T19:06:21.349 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profiles": { 2024-09-20T19:06:21.349 INFO:teuthology.orchestra.run.smithi083.stdout: "default": { 2024-09-20T19:06:21.349 INFO:teuthology.orchestra.run.smithi083.stdout: "crush-failure-domain": "osd", 2024-09-20T19:06:21.349 INFO:teuthology.orchestra.run.smithi083.stdout: "k": "2", 2024-09-20T19:06:21.349 INFO:teuthology.orchestra.run.smithi083.stdout: "m": "1", 2024-09-20T19:06:21.349 INFO:teuthology.orchestra.run.smithi083.stdout: "plugin": "jerasure", 2024-09-20T19:06:21.349 INFO:teuthology.orchestra.run.smithi083.stdout: "technique": "reed_sol_van" 2024-09-20T19:06:21.349 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:21.349 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps_queue": [], 2024-09-20T19:06:21.349 INFO:teuthology.orchestra.run.smithi083.stdout: "new_removed_snaps": [], 2024-09-20T19:06:21.349 INFO:teuthology.orchestra.run.smithi083.stdout: "new_purged_snaps": [], 2024-09-20T19:06:21.349 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_node_flags": {}, 2024-09-20T19:06:21.349 INFO:teuthology.orchestra.run.smithi083.stdout: "device_class_flags": {}, 2024-09-20T19:06:21.349 INFO:teuthology.orchestra.run.smithi083.stdout: "stretch_mode": { 2024-09-20T19:06:21.350 INFO:teuthology.orchestra.run.smithi083.stdout: "stretch_mode_enabled": false, 2024-09-20T19:06:21.350 INFO:teuthology.orchestra.run.smithi083.stdout: "stretch_bucket_count": 0, 2024-09-20T19:06:21.350 INFO:teuthology.orchestra.run.smithi083.stdout: "degraded_stretch_mode": 0, 2024-09-20T19:06:21.350 INFO:teuthology.orchestra.run.smithi083.stdout: "recovering_stretch_mode": 0, 2024-09-20T19:06:21.350 INFO:teuthology.orchestra.run.smithi083.stdout: "stretch_mode_bucket": 0 2024-09-20T19:06:21.350 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:21.350 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T19:06:21.350 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:21.372 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:21.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:21.373 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:21.418 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T19:06:21.418 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-09-20T19:06:21.418 INFO:teuthology.orchestra.run.smithi142.stdout: "complaint_time": 30, 2024-09-20T19:06:21.418 INFO:teuthology.orchestra.run.smithi142.stdout: "num_blocked_ops": 0 2024-09-20T19:06:21.419 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T19:06:21.474 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:06:21.495 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:21.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:21.496 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:06:21.597 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:21.618 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:21.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:21.620 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:21.721 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:21.743 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:21.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:21.744 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:21.845 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:06:21.866 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:21.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:21.867 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:06:21.968 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:21.989 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:21.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:21.990 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:22.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:21 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:06:22.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:21 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:06:22.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:21 smithi142 ceph-mon[32508]: 5.e scrub starts 2024-09-20T19:06:22.012 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:21 smithi142 ceph-mon[32508]: 5.e scrub ok 2024-09-20T19:06:22.012 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:21 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-20T19:06:22.012 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:21 smithi142 ceph-mon[32508]: osdmap e426: 8 total, 8 up, 8 in 2024-09-20T19:06:22.012 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:21 smithi142 ceph-mon[32508]: pgmap v838: 148 pgs: 1 creating+peering, 9 unknown, 138 active+clean; 583 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail 2024-09-20T19:06:22.012 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:21 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/4116997950' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-20T19:06:22.092 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:22.102 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_historic_ops 2024-09-20T19:06:22.113 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:22.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:22.114 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:22.215 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:22.237 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:22.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:22.238 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:22.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:21 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:06:22.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:21 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:06:22.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:21 smithi083 ceph-mon[29421]: 5.e scrub starts 2024-09-20T19:06:22.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:21 smithi083 ceph-mon[29421]: 5.e scrub ok 2024-09-20T19:06:22.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:21 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-20T19:06:22.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:21 smithi083 ceph-mon[29421]: osdmap e426: 8 total, 8 up, 8 in 2024-09-20T19:06:22.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:21 smithi083 ceph-mon[29421]: pgmap v838: 148 pgs: 1 creating+peering, 9 unknown, 138 active+clean; 583 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail 2024-09-20T19:06:22.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:21 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/4116997950' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-20T19:06:22.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:21 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:06:22.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:21 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:06:22.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:21 smithi083 ceph-mon[36260]: 5.e scrub starts 2024-09-20T19:06:22.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:21 smithi083 ceph-mon[36260]: 5.e scrub ok 2024-09-20T19:06:22.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:21 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-20T19:06:22.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:21 smithi083 ceph-mon[36260]: osdmap e426: 8 total, 8 up, 8 in 2024-09-20T19:06:22.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:21 smithi083 ceph-mon[36260]: pgmap v838: 148 pgs: 1 creating+peering, 9 unknown, 138 active+clean; 583 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail 2024-09-20T19:06:22.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:21 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/4116997950' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-20T19:06:22.338 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:22.355 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:06:22.359 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:22.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:22.360 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:22.461 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:22.482 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:22.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:22.483 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:22.502 INFO:tasks.thrashosds.thrasher:j is {'epoch': 426, 'fsid': '1ed5a50a-7780-11ef-baf5-efdc52797490', 'created': '2024-09-20T18:44:10.399800+0000', 'modified': '2024-09-20T19:06:21.057032+0000', 'last_up_change': '2024-09-20T19:01:21.809732+0000', 'last_in_change': '2024-09-20T19:04:20.299488+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 19, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 11, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-20T18:47:15.285795+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '361', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'datapool', 'create_time': '2024-09-20T18:49:43.452984+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 61, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.6500000953674316, 'score_stable': 2.6500000953674316, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 2.3299999237060547, 'raw_score_stable': 2.3299999237060547, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': '.rgw.root', 'create_time': '2024-09-20T18:49:44.217011+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-20T18:49:46.007275+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-20T18:49:47.961258+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-20T18:49:50.011276+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'unique_pool_4', 'create_time': '2024-09-20T19:06:16.062904+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '426', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': 'c714e1ec-f0c3-4054-9792-61b84e53eaa8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 422, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6801', 'nonce': 2574084049}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6802', 'nonce': 2574084049}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6804', 'nonce': 2574084049}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6803', 'nonce': 2574084049}]}, 'public_addr': '172.21.15.83:6801/2574084049', 'cluster_addr': '172.21.15.83:6802/2574084049', 'heartbeat_back_addr': '172.21.15.83:6804/2574084049', 'heartbeat_front_addr': '172.21.15.83:6803/2574084049', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '9dc0f2a8-bafa-4a42-88f5-0301dd58f78d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 422, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6805', 'nonce': 4213285798}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6806', 'nonce': 4213285798}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6808', 'nonce': 4213285798}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6807', 'nonce': 4213285798}]}, 'public_addr': '172.21.15.83:6805/4213285798', 'cluster_addr': '172.21.15.83:6806/4213285798', 'heartbeat_back_addr': '172.21.15.83:6808/4213285798', 'heartbeat_front_addr': '172.21.15.83:6807/4213285798', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '448ab692-ec41-44dc-b2d5-dbefd255b3fd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 422, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6809', 'nonce': 805947571}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6810', 'nonce': 805947571}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6812', 'nonce': 805947571}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6811', 'nonce': 805947571}]}, 'public_addr': '172.21.15.83:6809/805947571', 'cluster_addr': '172.21.15.83:6810/805947571', 'heartbeat_back_addr': '172.21.15.83:6812/805947571', 'heartbeat_front_addr': '172.21.15.83:6811/805947571', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '7cda5e50-e22e-4d4a-823b-53c8f5061193', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 25, 'up_thru': 422, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6813', 'nonce': 987237318}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6814', 'nonce': 987237318}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6816', 'nonce': 987237318}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6815', 'nonce': 987237318}]}, 'public_addr': '172.21.15.83:6813/987237318', 'cluster_addr': '172.21.15.83:6814/987237318', 'heartbeat_back_addr': '172.21.15.83:6816/987237318', 'heartbeat_front_addr': '172.21.15.83:6815/987237318', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '3d62654d-d31d-4310-9934-49e694dd17d9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 31, 'last_clean_end': 291, 'up_from': 304, 'up_thru': 422, 'down_at': 292, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6800', 'nonce': 3322919402}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6801', 'nonce': 3322919402}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6803', 'nonce': 3322919402}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6802', 'nonce': 3322919402}]}, 'public_addr': '172.21.15.142:6800/3322919402', 'cluster_addr': '172.21.15.142:6801/3322919402', 'heartbeat_back_addr': '172.21.15.142:6803/3322919402', 'heartbeat_front_addr': '172.21.15.142:6802/3322919402', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'b3e4e21b-639f-4d25-8bbf-ea8922a9222a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 38, 'up_thru': 422, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6804', 'nonce': 2500466067}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6805', 'nonce': 2500466067}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6807', 'nonce': 2500466067}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6806', 'nonce': 2500466067}]}, 'public_addr': '172.21.15.142:6804/2500466067', 'cluster_addr': '172.21.15.142:6805/2500466067', 'heartbeat_back_addr': '172.21.15.142:6807/2500466067', 'heartbeat_front_addr': '172.21.15.142:6806/2500466067', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'c5db7107-ccb4-4ae2-9e63-f444897942e3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 45, 'up_thru': 422, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6808', 'nonce': 1688186226}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6809', 'nonce': 1688186226}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6811', 'nonce': 1688186226}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6810', 'nonce': 1688186226}]}, 'public_addr': '172.21.15.142:6808/1688186226', 'cluster_addr': '172.21.15.142:6809/1688186226', 'heartbeat_back_addr': '172.21.15.142:6811/1688186226', 'heartbeat_front_addr': '172.21.15.142:6810/1688186226', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '07d16806-e2c8-4acb-b284-d75701e61622', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 53, 'up_thru': 422, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6812', 'nonce': 3679770984}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6813', 'nonce': 3679770984}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6815', 'nonce': 3679770984}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6814', 'nonce': 3679770984}]}, 'public_addr': '172.21.15.142:6812/3679770984', 'cluster_addr': '172.21.15.142:6813/3679770984', 'heartbeat_back_addr': '172.21.15.142:6815/3679770984', 'heartbeat_front_addr': '172.21.15.142:6814/3679770984', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T19:05:14.862999+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T19:06:20.624438+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T19:04:57.447245+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T19:05:37.721563+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T19:05:10.575535+0000', 'dead_epoch': 291}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T19:06:01.752083+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T19:05:08.941679+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T19:06:20.160210+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.c', 'osds': [2, 7, 6]}, {'pgid': '4.2', 'osds': [5, 7, 4]}], 'pg_upmap_items': [{'pgid': '4.4', 'mappings': [{'from': 3, 'to': 5}]}, {'pgid': '4.f', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.14', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.18', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.1e', 'mappings': [{'from': 3, 'to': 2}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {'172.21.15.83:0/3474253603': '2024-09-21T18:44:32.434180+0000', '172.21.15.83:6800/3525856829': '2024-09-21T18:44:32.434180+0000', '172.21.15.83:0/220746284': '2024-09-21T18:44:32.434180+0000', '172.21.15.83:0/3875445393': '2024-09-21T18:44:56.985620+0000', '172.21.15.83:0/3009092807': '2024-09-21T18:44:56.985620+0000', '172.21.15.83:0/4267132131': '2024-09-21T18:44:56.985620+0000', '172.21.15.83:0/3684382972': '2024-09-21T18:50:14.729526+0000', '172.21.15.83:0/3132933178': '2024-09-21T18:50:14.729526+0000', '172.21.15.83:6800/1007846146': '2024-09-21T18:50:14.729526+0000', '172.21.15.83:6800/2985596191': '2024-09-21T18:44:56.985620+0000', '172.21.15.83:0/2410768464': '2024-09-21T18:50:14.729526+0000', '172.21.15.83:0/1868694097': '2024-09-21T18:50:14.729526+0000', '172.21.15.83:0/3777331122': '2024-09-21T18:44:32.434180+0000', '172.21.15.83:0/2161728215': '2024-09-21T18:50:14.729526+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-20T19:06:22.502 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T19:06:22.554 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:06:22.584 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:22.606 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:22.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:22.607 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:22.707 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:06:22.728 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:22.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:22.729 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:06:22.819 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T19:06:22.819 INFO:teuthology.orchestra.run.smithi142.stdout: "size": 20, 2024-09-20T19:06:22.819 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 600, 2024-09-20T19:06:22.819 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-09-20T19:06:22.819 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.819 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:124590 9.6 9:6ce021c9:::benchmark_data_smithi083_124552_object10443:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e289)", 2024-09-20T19:06:22.820 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:00:20.955292+0000", 2024-09-20T19:06:22.820 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 361.84972337599999, 2024-09-20T19:06:22.820 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.324912592, 2024-09-20T19:06:22.820 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:06:22.820 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:06:22.820 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:22.820 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:06:22.820 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:06:22.820 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:06:22.820 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 124590 2024-09-20T19:06:22.820 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.820 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:06:22.820 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.820 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:06:22.820 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.955292+0000", 2024-09-20T19:06:22.821 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:22.821 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.821 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.821 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:06:22.821 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.955292+0000", 2024-09-20T19:06:22.821 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:22.821 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.821 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.821 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:06:22.821 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.955293+0000", 2024-09-20T19:06:22.821 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.8899999999999998e-07 2024-09-20T19:06:22.821 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.821 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.821 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:06:22.822 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.955297+0000", 2024-09-20T19:06:22.822 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6370000000000001e-06 2024-09-20T19:06:22.822 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.822 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.822 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:06:22.822 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.955309+0000", 2024-09-20T19:06:22.822 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.205e-05 2024-09-20T19:06:22.822 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.822 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.822 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:06:22.822 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.955316+0000", 2024-09-20T19:06:22.822 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.7789999999999998e-06 2024-09-20T19:06:22.822 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.822 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.823 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:22.823 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.955398+0000", 2024-09-20T19:06:22.823 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.1893999999999999e-05 2024-09-20T19:06:22.823 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.823 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.823 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:06:22.823 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.955512+0000", 2024-09-20T19:06:22.823 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000114036 2024-09-20T19:06:22.823 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.823 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.823 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:06:22.823 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.955606+0000", 2024-09-20T19:06:22.823 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.4199999999999999e-05 2024-09-20T19:06:22.823 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.823 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.824 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:06:22.824 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.958828+0000", 2024-09-20T19:06:22.824 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.003221912 2024-09-20T19:06:22.824 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.824 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.824 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:22.824 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.963128+0000", 2024-09-20T19:06:22.824 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.004300379 2024-09-20T19:06:22.824 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.824 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.824 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:22.824 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:21.280133+0000", 2024-09-20T19:06:22.824 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.317005388 2024-09-20T19:06:22.824 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.825 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.825 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:06:22.825 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:21.280172+0000", 2024-09-20T19:06:22.825 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9014000000000003e-05 2024-09-20T19:06:22.825 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.825 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.825 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:06:22.825 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:21.280205+0000", 2024-09-20T19:06:22.825 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.2514000000000001e-05 2024-09-20T19:06:22.825 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:22.825 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:06:22.825 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:22.825 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.825 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.825 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:124594 9.6 9:67fbc174:::benchmark_data_smithi083_124552_object10447:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e289)", 2024-09-20T19:06:22.826 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:00:20.960066+0000", 2024-09-20T19:06:22.826 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 361.84494963899999, 2024-09-20T19:06:22.826 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.32985013699999999, 2024-09-20T19:06:22.826 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:06:22.826 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:06:22.826 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:22.826 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:06:22.826 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:06:22.826 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:06:22.826 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 124594 2024-09-20T19:06:22.826 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.826 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:06:22.826 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.826 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:06:22.827 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.960066+0000", 2024-09-20T19:06:22.827 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:22.827 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.827 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.827 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:06:22.827 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.960066+0000", 2024-09-20T19:06:22.827 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:22.827 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.827 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.827 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:06:22.827 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.960068+0000", 2024-09-20T19:06:22.827 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.514e-06 2024-09-20T19:06:22.827 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.827 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.828 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:06:22.828 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.960073+0000", 2024-09-20T19:06:22.828 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3040000000000004e-06 2024-09-20T19:06:22.828 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.828 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.828 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:06:22.828 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.960082+0000", 2024-09-20T19:06:22.828 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.4590000000000004e-06 2024-09-20T19:06:22.828 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.828 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.828 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:06:22.828 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.960089+0000", 2024-09-20T19:06:22.828 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.3419999999999996e-06 2024-09-20T19:06:22.828 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.828 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.829 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:22.829 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.960147+0000", 2024-09-20T19:06:22.829 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8289999999999999e-05 2024-09-20T19:06:22.829 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.829 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.829 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:06:22.829 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.960255+0000", 2024-09-20T19:06:22.829 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000107666 2024-09-20T19:06:22.829 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.829 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.829 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:06:22.829 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.960363+0000", 2024-09-20T19:06:22.829 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000108186 2024-09-20T19:06:22.829 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.830 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.830 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:06:22.830 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.963253+0000", 2024-09-20T19:06:22.830 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0028899799999999999 2024-09-20T19:06:22.830 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.830 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.830 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:22.830 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.966302+0000", 2024-09-20T19:06:22.830 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.003049169 2024-09-20T19:06:22.830 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.830 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.830 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:22.830 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:21.289803+0000", 2024-09-20T19:06:22.830 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.32350114299999999 2024-09-20T19:06:22.830 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.831 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.831 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:06:22.831 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:21.289861+0000", 2024-09-20T19:06:22.831 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8034e-05 2024-09-20T19:06:22.831 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.831 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.831 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:06:22.831 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:21.289916+0000", 2024-09-20T19:06:22.831 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5050000000000003e-05 2024-09-20T19:06:22.831 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:22.831 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:06:22.831 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:22.831 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.831 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.831 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:124626 9.6 9:6930ec60:::benchmark_data_smithi083_124552_object10479:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e289)", 2024-09-20T19:06:22.832 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:00:20.983128+0000", 2024-09-20T19:06:22.832 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 361.821887699, 2024-09-20T19:06:22.832 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30694355299999998, 2024-09-20T19:06:22.832 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:06:22.832 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:06:22.832 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:22.832 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:06:22.832 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:06:22.832 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:06:22.832 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 124626 2024-09-20T19:06:22.832 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.832 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:06:22.832 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.832 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:06:22.833 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.983128+0000", 2024-09-20T19:06:22.833 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:22.833 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.833 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.833 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:06:22.833 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.983128+0000", 2024-09-20T19:06:22.833 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:22.833 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.833 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.833 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:06:22.833 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.983130+0000", 2024-09-20T19:06:22.833 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.426e-06 2024-09-20T19:06:22.833 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.833 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.834 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:06:22.834 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.983134+0000", 2024-09-20T19:06:22.834 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.4309999999999996e-06 2024-09-20T19:06:22.834 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.834 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.834 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:06:22.834 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.983144+0000", 2024-09-20T19:06:22.834 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0023e-05 2024-09-20T19:06:22.834 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.834 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.834 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:06:22.834 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.983149+0000", 2024-09-20T19:06:22.834 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.9949999999999996e-06 2024-09-20T19:06:22.834 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.834 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.835 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:22.835 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.983201+0000", 2024-09-20T19:06:22.835 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.2312999999999999e-05 2024-09-20T19:06:22.835 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.835 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.835 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:06:22.835 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.983303+0000", 2024-09-20T19:06:22.835 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000101433 2024-09-20T19:06:22.835 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.835 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.835 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:06:22.835 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.983418+0000", 2024-09-20T19:06:22.835 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000114851 2024-09-20T19:06:22.835 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.836 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.836 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:06:22.836 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.987483+0000", 2024-09-20T19:06:22.836 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0040654990000000002 2024-09-20T19:06:22.836 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.836 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.836 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:22.836 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:20.989475+0000", 2024-09-20T19:06:22.836 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001992204 2024-09-20T19:06:22.836 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.836 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.836 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:22.836 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:21.289975+0000", 2024-09-20T19:06:22.836 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30049981100000001 2024-09-20T19:06:22.837 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.837 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.837 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:06:22.837 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:21.290025+0000", 2024-09-20T19:06:22.837 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0009000000000002e-05 2024-09-20T19:06:22.837 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.837 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.837 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:06:22.837 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:21.290072+0000", 2024-09-20T19:06:22.837 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.6557999999999997e-05 2024-09-20T19:06:22.837 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:22.837 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:06:22.837 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:22.837 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.837 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.838 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14460.0:2699 5.e 5:7759931f:::notify.3:head [watch reconnect cookie 94056120170112 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e292)", 2024-09-20T19:06:22.838 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:00:35.180070+0000", 2024-09-20T19:06:22.838 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 347.62494583699998, 2024-09-20T19:06:22.838 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0139281419999999, 2024-09-20T19:06:22.838 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:06:22.838 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:06:22.838 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-09-20T19:06:22.838 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:06:22.838 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14460", 2024-09-20T19:06:22.838 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1555339426", 2024-09-20T19:06:22.838 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 2699 2024-09-20T19:06:22.838 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.838 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:06:22.838 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.839 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:06:22.839 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.180070+0000", 2024-09-20T19:06:22.839 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:22.839 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.839 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.839 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:06:22.839 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.180070+0000", 2024-09-20T19:06:22.839 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:22.839 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.839 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.839 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:06:22.839 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.180071+0000", 2024-09-20T19:06:22.839 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.093e-06 2024-09-20T19:06:22.839 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.840 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.840 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:06:22.840 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.180077+0000", 2024-09-20T19:06:22.840 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5079999999999996e-06 2024-09-20T19:06:22.840 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.840 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.840 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:06:22.840 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.180086+0000", 2024-09-20T19:06:22.840 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.9499999999999996e-06 2024-09-20T19:06:22.840 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.840 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.840 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:06:22.840 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.180094+0000", 2024-09-20T19:06:22.840 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5780000000000001e-06 2024-09-20T19:06:22.840 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.841 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.841 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:22.841 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.180159+0000", 2024-09-20T19:06:22.841 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5139000000000001e-05 2024-09-20T19:06:22.841 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.841 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.841 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op must wait for map", 2024-09-20T19:06:22.841 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.180165+0000", 2024-09-20T19:06:22.841 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.7629999999999997e-06 2024-09-20T19:06:22.841 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.841 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.841 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:22.841 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.182672+0000", 2024-09-20T19:06:22.841 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0025072710000000002 2024-09-20T19:06:22.842 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.842 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.842 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:06:22.842 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.182681+0000", 2024-09-20T19:06:22.842 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.4780000000000008e-06 2024-09-20T19:06:22.842 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.842 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.842 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:22.842 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.191896+0000", 2024-09-20T19:06:22.842 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.009215755 2024-09-20T19:06:22.842 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.842 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.842 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:06:22.843 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.191989+0000", 2024-09-20T19:06:22.843 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.2868000000000006e-05 2024-09-20T19:06:22.843 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.843 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.843 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:06:22.843 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.193998+0000", 2024-09-20T19:06:22.843 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0020087389999999998 2024-09-20T19:06:22.843 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:22.843 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:06:22.843 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:22.843 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.843 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.843 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:147626 10.9 10:98e2f0b0:::benchmark_data_smithi083_148100_object30764:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e376)", 2024-09-20T19:06:22.843 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:04:28.118535+0000", 2024-09-20T19:06:22.844 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 114.686480616, 2024-09-20T19:06:22.844 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.40251810599999999, 2024-09-20T19:06:22.844 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:06:22.844 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:06:22.844 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:22.844 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:06:22.844 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:06:22.844 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:06:22.844 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 147626 2024-09-20T19:06:22.844 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.844 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:06:22.844 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.844 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:06:22.844 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.118535+0000", 2024-09-20T19:06:22.844 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:22.845 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.845 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.845 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:06:22.845 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.118535+0000", 2024-09-20T19:06:22.845 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:22.845 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.845 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.845 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:06:22.845 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.118537+0000", 2024-09-20T19:06:22.845 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7910000000000001e-06 2024-09-20T19:06:22.845 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.845 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.845 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:06:22.845 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.118542+0000", 2024-09-20T19:06:22.845 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4049999999999999e-06 2024-09-20T19:06:22.846 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.846 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.846 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:06:22.846 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.118556+0000", 2024-09-20T19:06:22.846 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3988999999999999e-05 2024-09-20T19:06:22.846 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.846 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.846 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:06:22.846 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.118564+0000", 2024-09-20T19:06:22.846 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.0979999999999994e-06 2024-09-20T19:06:22.846 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.846 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.846 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:22.846 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.118622+0000", 2024-09-20T19:06:22.847 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.7796e-05 2024-09-20T19:06:22.847 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.847 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.847 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:06:22.847 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.118757+0000", 2024-09-20T19:06:22.847 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013468599999999999 2024-09-20T19:06:22.847 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.847 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.847 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 2,7", 2024-09-20T19:06:22.847 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.118877+0000", 2024-09-20T19:06:22.847 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000120103 2024-09-20T19:06:22.847 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.847 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.847 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:06:22.848 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.121979+0000", 2024-09-20T19:06:22.848 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0031019569999999998 2024-09-20T19:06:22.848 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.848 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.848 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:22.848 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.122589+0000", 2024-09-20T19:06:22.848 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00061044199999999999 2024-09-20T19:06:22.848 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.848 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.848 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:22.848 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.520939+0000", 2024-09-20T19:06:22.848 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.39834966599999999 2024-09-20T19:06:22.848 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.848 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.849 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:06:22.849 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.520995+0000", 2024-09-20T19:06:22.849 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5405999999999998e-05 2024-09-20T19:06:22.849 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.849 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.849 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:06:22.849 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.521053+0000", 2024-09-20T19:06:22.849 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8767000000000002e-05 2024-09-20T19:06:22.849 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:22.849 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:06:22.849 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:22.849 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.849 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.849 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:147635 10.9 10:95751c44:::benchmark_data_smithi083_148100_object30773:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e376)", 2024-09-20T19:06:22.850 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:04:28.131221+0000", 2024-09-20T19:06:22.850 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 114.673795016, 2024-09-20T19:06:22.850 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.398357515, 2024-09-20T19:06:22.850 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:06:22.850 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:06:22.850 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:22.850 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:06:22.850 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:06:22.850 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:06:22.850 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 147635 2024-09-20T19:06:22.850 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.850 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:06:22.850 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.850 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:06:22.850 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.131221+0000", 2024-09-20T19:06:22.851 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:22.851 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.851 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.851 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:06:22.851 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.131221+0000", 2024-09-20T19:06:22.851 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:22.851 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.851 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.851 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:06:22.851 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.131222+0000", 2024-09-20T19:06:22.851 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.575e-06 2024-09-20T19:06:22.851 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.851 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.851 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:06:22.852 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.131229+0000", 2024-09-20T19:06:22.852 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2740000000000004e-06 2024-09-20T19:06:22.852 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.852 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.852 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:06:22.852 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.131244+0000", 2024-09-20T19:06:22.852 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5362e-05 2024-09-20T19:06:22.852 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.852 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.852 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:06:22.852 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.131252+0000", 2024-09-20T19:06:22.852 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.3040000000000005e-06 2024-09-20T19:06:22.852 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.852 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.852 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:22.853 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.131312+0000", 2024-09-20T19:06:22.853 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9703000000000001e-05 2024-09-20T19:06:22.853 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.853 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.853 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:06:22.853 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.131421+0000", 2024-09-20T19:06:22.853 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00010873400000000001 2024-09-20T19:06:22.853 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.853 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.853 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 2,7", 2024-09-20T19:06:22.853 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.131541+0000", 2024-09-20T19:06:22.853 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000120625 2024-09-20T19:06:22.853 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.853 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.854 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:06:22.854 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.134390+0000", 2024-09-20T19:06:22.854 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0028481959999999999 2024-09-20T19:06:22.854 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.854 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.854 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:22.854 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.135079+0000", 2024-09-20T19:06:22.854 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00068968800000000004 2024-09-20T19:06:22.854 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.854 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.854 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:22.854 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.529488+0000", 2024-09-20T19:06:22.854 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.39440882500000002 2024-09-20T19:06:22.854 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.855 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.855 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:06:22.855 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.529539+0000", 2024-09-20T19:06:22.855 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.1038000000000002e-05 2024-09-20T19:06:22.855 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.855 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.855 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:06:22.855 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.529578+0000", 2024-09-20T19:06:22.855 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9190999999999998e-05 2024-09-20T19:06:22.855 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:22.855 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:06:22.855 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:22.855 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.855 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.856 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:147639 10.9 10:96d58120:::benchmark_data_smithi083_148100_object30777:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e376)", 2024-09-20T19:06:22.856 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:04:28.138921+0000", 2024-09-20T19:06:22.856 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 114.666094973, 2024-09-20T19:06:22.856 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.39076443700000002, 2024-09-20T19:06:22.856 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:06:22.856 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:06:22.856 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:22.856 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:06:22.856 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:06:22.856 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:06:22.856 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 147639 2024-09-20T19:06:22.856 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.856 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:06:22.856 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.857 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:06:22.857 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.138921+0000", 2024-09-20T19:06:22.857 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:22.857 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.857 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.857 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:06:22.857 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.138921+0000", 2024-09-20T19:06:22.857 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:22.857 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.857 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.857 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:06:22.857 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.138922+0000", 2024-09-20T19:06:22.857 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.598e-06 2024-09-20T19:06:22.857 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.858 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.858 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:06:22.858 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.138929+0000", 2024-09-20T19:06:22.858 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2890000000000003e-06 2024-09-20T19:06:22.858 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.858 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.858 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:06:22.858 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.138940+0000", 2024-09-20T19:06:22.858 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1292e-05 2024-09-20T19:06:22.858 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.858 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.858 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:06:22.858 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.138948+0000", 2024-09-20T19:06:22.858 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.7190000000000001e-06 2024-09-20T19:06:22.859 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.859 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.859 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:22.859 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.139013+0000", 2024-09-20T19:06:22.859 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5686999999999996e-05 2024-09-20T19:06:22.859 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.859 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.859 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:06:22.859 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.139133+0000", 2024-09-20T19:06:22.859 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000119709 2024-09-20T19:06:22.859 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.859 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.859 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 2,7", 2024-09-20T19:06:22.859 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.139232+0000", 2024-09-20T19:06:22.860 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.9275000000000001e-05 2024-09-20T19:06:22.860 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.860 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.860 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:06:22.860 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.142295+0000", 2024-09-20T19:06:22.860 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0030624210000000001 2024-09-20T19:06:22.860 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.860 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.860 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:22.860 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.142766+0000", 2024-09-20T19:06:22.860 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00047071800000000002 2024-09-20T19:06:22.860 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.860 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.860 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:22.860 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.529610+0000", 2024-09-20T19:06:22.861 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.38684496499999999 2024-09-20T19:06:22.861 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.861 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.861 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:06:22.861 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.529654+0000", 2024-09-20T19:06:22.861 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3988e-05 2024-09-20T19:06:22.861 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.861 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.861 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:06:22.861 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.529685+0000", 2024-09-20T19:06:22.861 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.0775999999999998e-05 2024-09-20T19:06:22.861 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:22.861 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:06:22.861 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:22.862 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.862 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.862 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:147641 10.9 10:957d4f1f:::benchmark_data_smithi083_148100_object30779:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e376)", 2024-09-20T19:06:22.862 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:04:28.142745+0000", 2024-09-20T19:06:22.862 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 114.662271206, 2024-09-20T19:06:22.862 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.38705916899999998, 2024-09-20T19:06:22.862 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:06:22.862 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:06:22.862 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:22.862 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:06:22.862 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:06:22.862 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:06:22.862 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 147641 2024-09-20T19:06:22.863 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.863 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:06:22.863 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.863 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:06:22.863 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.142745+0000", 2024-09-20T19:06:22.863 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:22.863 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.863 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.863 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:06:22.863 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.142745+0000", 2024-09-20T19:06:22.863 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:22.863 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.863 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.864 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:06:22.864 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.142746+0000", 2024-09-20T19:06:22.864 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7680000000000001e-06 2024-09-20T19:06:22.864 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.864 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.864 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:06:22.864 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.142752+0000", 2024-09-20T19:06:22.864 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.7019999999999998e-06 2024-09-20T19:06:22.864 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.864 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.864 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:06:22.864 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.142761+0000", 2024-09-20T19:06:22.864 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.4190000000000008e-06 2024-09-20T19:06:22.864 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.865 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.865 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:06:22.865 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.142768+0000", 2024-09-20T19:06:22.865 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.8009999999999998e-06 2024-09-20T19:06:22.865 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.865 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.865 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:22.865 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.142829+0000", 2024-09-20T19:06:22.865 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0537e-05 2024-09-20T19:06:22.865 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.865 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.865 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:06:22.865 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.142951+0000", 2024-09-20T19:06:22.865 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000122619 2024-09-20T19:06:22.865 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.866 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.866 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 2,7", 2024-09-20T19:06:22.866 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.143050+0000", 2024-09-20T19:06:22.866 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.8121999999999993e-05 2024-09-20T19:06:22.866 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.866 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.866 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:22.866 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.145907+0000", 2024-09-20T19:06:22.866 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0028578789999999998 2024-09-20T19:06:22.866 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.866 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.866 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:06:22.866 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.145926+0000", 2024-09-20T19:06:22.866 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8272000000000001e-05 2024-09-20T19:06:22.867 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.867 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.867 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:22.867 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.529726+0000", 2024-09-20T19:06:22.867 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.38380014200000001 2024-09-20T19:06:22.867 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.867 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.867 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:06:22.867 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.529774+0000", 2024-09-20T19:06:22.867 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8279999999999999e-05 2024-09-20T19:06:22.867 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.867 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.867 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:06:22.867 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.529804+0000", 2024-09-20T19:06:22.868 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9628000000000002e-05 2024-09-20T19:06:22.868 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:22.868 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:06:22.868 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:22.868 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.868 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.868 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:147647 10.9 10:988ad65f:::benchmark_data_smithi083_148100_object30785:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e376)", 2024-09-20T19:06:22.868 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:04:28.151581+0000", 2024-09-20T19:06:22.868 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 114.6534345, 2024-09-20T19:06:22.868 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.37836669499999998, 2024-09-20T19:06:22.868 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:06:22.868 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:06:22.868 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:22.868 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:06:22.869 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:06:22.869 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:06:22.869 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 147647 2024-09-20T19:06:22.869 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.869 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:06:22.869 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.869 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:06:22.869 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.151581+0000", 2024-09-20T19:06:22.869 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:22.869 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.869 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.869 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:06:22.869 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.151581+0000", 2024-09-20T19:06:22.869 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:22.869 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.870 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.870 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:06:22.870 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.151584+0000", 2024-09-20T19:06:22.870 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.5349999999999999e-06 2024-09-20T19:06:22.870 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.870 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.870 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:06:22.870 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.151590+0000", 2024-09-20T19:06:22.870 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9560000000000002e-06 2024-09-20T19:06:22.870 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.870 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.870 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:06:22.870 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.151605+0000", 2024-09-20T19:06:22.871 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5553000000000002e-05 2024-09-20T19:06:22.871 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.871 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.871 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:06:22.871 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.151615+0000", 2024-09-20T19:06:22.871 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.6160000000000008e-06 2024-09-20T19:06:22.871 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.871 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.871 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:22.871 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.151678+0000", 2024-09-20T19:06:22.871 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2650000000000005e-05 2024-09-20T19:06:22.872 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.872 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.872 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:06:22.872 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.151816+0000", 2024-09-20T19:06:22.872 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013797399999999999 2024-09-20T19:06:22.872 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.872 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.872 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 2,7", 2024-09-20T19:06:22.872 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.151936+0000", 2024-09-20T19:06:22.872 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012091699999999999 2024-09-20T19:06:22.872 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.872 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.872 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:22.872 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.154743+0000", 2024-09-20T19:06:22.873 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0028064359999999998 2024-09-20T19:06:22.873 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.873 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.873 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:06:22.873 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.154877+0000", 2024-09-20T19:06:22.873 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000133893 2024-09-20T19:06:22.873 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.873 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.873 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:22.875 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:22.875 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.529857+0000", 2024-09-20T19:06:22.875 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.37498010799999998 2024-09-20T19:06:22.875 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.875 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.875 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:06:22.875 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.529911+0000", 2024-09-20T19:06:22.876 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3785000000000001e-05 2024-09-20T19:06:22.876 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.876 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.876 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:06:22.876 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.529948+0000", 2024-09-20T19:06:22.876 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.7271999999999998e-05 2024-09-20T19:06:22.876 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:22.876 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:06:22.876 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:22.876 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.876 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.876 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:196171 10.6 10:6296a24c:::benchmark_data_smithi083_148100_object79309:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e393)", 2024-09-20T19:06:22.876 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:20.884625+0000", 2024-09-20T19:06:22.876 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 61.920390283000003, 2024-09-20T19:06:22.877 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0535514020000001, 2024-09-20T19:06:22.877 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:06:22.877 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:06:22.877 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-09-20T19:06:22.877 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:06:22.877 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:06:22.877 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:06:22.877 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 196171 2024-09-20T19:06:22.877 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.877 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:06:22.877 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.877 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:06:22.877 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.884625+0000", 2024-09-20T19:06:22.877 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:22.877 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.878 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.878 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:06:22.878 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.884625+0000", 2024-09-20T19:06:22.878 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:22.878 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.878 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.878 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:06:22.878 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.884627+0000", 2024-09-20T19:06:22.878 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.339e-06 2024-09-20T19:06:22.878 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.878 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.878 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:06:22.878 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.884631+0000", 2024-09-20T19:06:22.878 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.437e-06 2024-09-20T19:06:22.879 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.879 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.879 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:06:22.879 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.884642+0000", 2024-09-20T19:06:22.879 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0514e-05 2024-09-20T19:06:22.879 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.879 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.879 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:06:22.879 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.884649+0000", 2024-09-20T19:06:22.879 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.7159999999999997e-06 2024-09-20T19:06:22.879 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.879 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.879 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:22.879 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.884690+0000", 2024-09-20T19:06:22.880 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.1452000000000002e-05 2024-09-20T19:06:22.880 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.880 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.880 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:06:22.880 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.884763+0000", 2024-09-20T19:06:22.880 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.2774999999999994e-05 2024-09-20T19:06:22.880 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.880 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.880 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,4", 2024-09-20T19:06:22.880 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.884833+0000", 2024-09-20T19:06:22.880 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.0147000000000004e-05 2024-09-20T19:06:22.880 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.880 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.880 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:06:22.880 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.887337+0000", 2024-09-20T19:06:22.880 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0025039810000000002 2024-09-20T19:06:22.881 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.881 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.881 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:22.881 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.919111+0000", 2024-09-20T19:06:22.881 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.031774167999999998 2024-09-20T19:06:22.881 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.881 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.881 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:06:22.881 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.919127+0000", 2024-09-20T19:06:22.881 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5934999999999999e-05 2024-09-20T19:06:22.881 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.881 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.881 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:22.881 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.938106+0000", 2024-09-20T19:06:22.882 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.018978588 2024-09-20T19:06:22.882 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.882 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.882 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:06:22.882 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.938177+0000", 2024-09-20T19:06:22.882 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.135e-05 2024-09-20T19:06:22.882 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:22.882 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:06:22.882 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:22.882 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.882 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.882 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:196176 10.6 10:6158460d:::benchmark_data_smithi083_148100_object79314:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e393)", 2024-09-20T19:06:22.882 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:20.893880+0000", 2024-09-20T19:06:22.882 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 61.911136163999998, 2024-09-20T19:06:22.883 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0516040250000001, 2024-09-20T19:06:22.883 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:06:22.883 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:06:22.883 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:22.883 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:06:22.883 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:06:22.883 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:06:22.883 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 196176 2024-09-20T19:06:22.883 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.883 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:06:22.883 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.883 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:06:22.883 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.893880+0000", 2024-09-20T19:06:22.883 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:22.884 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.884 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.884 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:06:22.884 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.893880+0000", 2024-09-20T19:06:22.884 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:22.884 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.884 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.884 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:06:22.884 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.893881+0000", 2024-09-20T19:06:22.884 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.47e-07 2024-09-20T19:06:22.884 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.884 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.884 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:06:22.884 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.893885+0000", 2024-09-20T19:06:22.884 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3930000000000001e-06 2024-09-20T19:06:22.885 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.885 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.885 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:06:22.885 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.915561+0000", 2024-09-20T19:06:22.885 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.021675638000000001 2024-09-20T19:06:22.885 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.885 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.885 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:06:22.885 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.915571+0000", 2024-09-20T19:06:22.885 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0362e-05 2024-09-20T19:06:22.885 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.885 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.885 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:22.885 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.934415+0000", 2024-09-20T19:06:22.885 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0188439 2024-09-20T19:06:22.885 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.886 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.886 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:06:22.886 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.934425+0000", 2024-09-20T19:06:22.886 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0052e-05 2024-09-20T19:06:22.886 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.886 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.886 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:22.886 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.938653+0000", 2024-09-20T19:06:22.886 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0042276919999999 2024-09-20T19:06:22.886 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.886 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.886 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:06:22.886 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.938749+0000", 2024-09-20T19:06:22.886 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.6009999999999997e-05 2024-09-20T19:06:22.887 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.887 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.887 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,4", 2024-09-20T19:06:22.887 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.938860+0000", 2024-09-20T19:06:22.887 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000111764 2024-09-20T19:06:22.887 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.887 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.887 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:22.887 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.945361+0000", 2024-09-20T19:06:22.887 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0065005610000000002 2024-09-20T19:06:22.887 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.887 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.887 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:06:22.887 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.945372+0000", 2024-09-20T19:06:22.887 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1056000000000001e-05 2024-09-20T19:06:22.888 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.888 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.888 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:22.888 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.945401+0000", 2024-09-20T19:06:22.888 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9167999999999999e-05 2024-09-20T19:06:22.888 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.888 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.888 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:06:22.888 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.945449+0000", 2024-09-20T19:06:22.888 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8205999999999999e-05 2024-09-20T19:06:22.888 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.888 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.888 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:06:22.888 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.945484+0000", 2024-09-20T19:06:22.888 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4276000000000002e-05 2024-09-20T19:06:22.889 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:22.889 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:06:22.889 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:22.889 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.889 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.889 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:196198 10.6 10:6893dd7d:::benchmark_data_smithi083_148100_object79336:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e394)", 2024-09-20T19:06:22.889 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:20.948044+0000", 2024-09-20T19:06:22.889 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 61.856971926, 2024-09-20T19:06:22.889 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0041771209999999, 2024-09-20T19:06:22.889 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:06:22.889 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:06:22.889 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:22.889 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:06:22.890 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:06:22.890 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:06:22.890 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 196198 2024-09-20T19:06:22.890 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.890 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:06:22.890 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.890 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:06:22.890 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.948044+0000", 2024-09-20T19:06:22.890 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:22.890 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.890 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.890 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:06:22.890 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.948044+0000", 2024-09-20T19:06:22.890 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:22.891 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.891 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.891 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:06:22.891 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.948045+0000", 2024-09-20T19:06:22.891 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.8800000000000002e-07 2024-09-20T19:06:22.891 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.891 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.891 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:06:22.891 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.948049+0000", 2024-09-20T19:06:22.891 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0629999999999999e-06 2024-09-20T19:06:22.891 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.891 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.891 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:06:22.891 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.948059+0000", 2024-09-20T19:06:22.891 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0519e-05 2024-09-20T19:06:22.892 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.892 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.892 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:06:22.892 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.948066+0000", 2024-09-20T19:06:22.892 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.6229999999999999e-06 2024-09-20T19:06:22.892 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.892 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.892 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:22.892 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.948129+0000", 2024-09-20T19:06:22.892 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.3423000000000006e-05 2024-09-20T19:06:22.892 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.892 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.892 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:06:22.892 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.948948+0000", 2024-09-20T19:06:22.892 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00081862199999999997 2024-09-20T19:06:22.892 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.893 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.893 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:22.893 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.941881+0000", 2024-09-20T19:06:22.893 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.99293267299999999 2024-09-20T19:06:22.893 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.893 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.893 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:06:22.893 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.941984+0000", 2024-09-20T19:06:22.893 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000103571 2024-09-20T19:06:22.893 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.893 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.893 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,4", 2024-09-20T19:06:22.893 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.942099+0000", 2024-09-20T19:06:22.893 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000115085 2024-09-20T19:06:22.894 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.894 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.894 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:22.894 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.945520+0000", 2024-09-20T19:06:22.894 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0034205419999999999 2024-09-20T19:06:22.894 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.894 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.894 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:06:22.894 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.945530+0000", 2024-09-20T19:06:22.894 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.8079999999999999e-06 2024-09-20T19:06:22.894 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.894 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.894 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:22.895 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.952124+0000", 2024-09-20T19:06:22.895 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0065947929999999998 2024-09-20T19:06:22.895 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.895 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.895 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:06:22.895 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.952176+0000", 2024-09-20T19:06:22.895 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.1548000000000002e-05 2024-09-20T19:06:22.895 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.895 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.895 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:06:22.895 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.952221+0000", 2024-09-20T19:06:22.895 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.5062999999999999e-05 2024-09-20T19:06:22.895 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:22.895 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:06:22.895 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:22.896 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.896 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.896 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:196199 10.6 10:64c368fa:::benchmark_data_smithi083_148100_object79337:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e394)", 2024-09-20T19:06:22.896 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:20.948119+0000", 2024-09-20T19:06:22.896 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 61.856897054999997, 2024-09-20T19:06:22.896 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0137902990000001, 2024-09-20T19:06:22.896 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:06:22.896 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:06:22.896 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:22.896 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:06:22.896 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:06:22.896 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:06:22.896 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 196199 2024-09-20T19:06:22.896 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.896 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:06:22.897 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.897 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:06:22.897 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.948119+0000", 2024-09-20T19:06:22.897 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:22.897 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.897 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.897 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:06:22.897 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.948119+0000", 2024-09-20T19:06:22.897 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:22.897 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.897 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.897 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:06:22.897 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.948120+0000", 2024-09-20T19:06:22.897 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.78e-07 2024-09-20T19:06:22.898 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.898 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.898 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:06:22.898 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.948124+0000", 2024-09-20T19:06:22.898 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7530000000000001e-06 2024-09-20T19:06:22.898 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.898 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.898 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:06:22.898 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.948135+0000", 2024-09-20T19:06:22.898 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0614999999999999e-05 2024-09-20T19:06:22.898 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.898 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.898 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:06:22.898 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.948150+0000", 2024-09-20T19:06:22.899 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4684999999999999e-05 2024-09-20T19:06:22.899 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.899 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.899 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:22.899 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.949184+0000", 2024-09-20T19:06:22.899 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0010337479999999999 2024-09-20T19:06:22.899 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.899 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.899 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:06:22.899 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.949209+0000", 2024-09-20T19:06:22.899 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.5122000000000001e-05 2024-09-20T19:06:22.899 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.899 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.899 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:22.899 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.945551+0000", 2024-09-20T19:06:22.900 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.99634271399999996 2024-09-20T19:06:22.900 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.900 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.900 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:06:22.900 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.945655+0000", 2024-09-20T19:06:22.900 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000103538 2024-09-20T19:06:22.900 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.900 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.900 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,4", 2024-09-20T19:06:22.900 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.945752+0000", 2024-09-20T19:06:22.900 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.6932e-05 2024-09-20T19:06:22.900 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.900 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.900 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:22.900 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.952257+0000", 2024-09-20T19:06:22.901 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0065051639999999999 2024-09-20T19:06:22.901 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.901 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.901 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:22.901 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.952290+0000", 2024-09-20T19:06:22.901 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.3149999999999999e-05 2024-09-20T19:06:22.901 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.901 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.901 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:06:22.901 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.961822+0000", 2024-09-20T19:06:22.901 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.009532067 2024-09-20T19:06:22.901 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.901 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.901 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:06:22.901 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.961870+0000", 2024-09-20T19:06:22.902 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7515000000000002e-05 2024-09-20T19:06:22.902 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.902 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.902 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:06:22.902 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.961909+0000", 2024-09-20T19:06:22.902 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9317999999999999e-05 2024-09-20T19:06:22.902 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:22.902 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:06:22.902 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:22.902 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.902 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.902 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:196203 10.6 10:60330bdb:::benchmark_data_smithi083_148100_object79341:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e394)", 2024-09-20T19:06:22.902 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:20.952749+0000", 2024-09-20T19:06:22.902 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 61.852266448000002, 2024-09-20T19:06:22.903 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0092730700000001, 2024-09-20T19:06:22.903 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:06:22.903 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:06:22.903 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:22.903 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:06:22.903 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:06:22.903 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:06:22.903 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 196203 2024-09-20T19:06:22.903 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.903 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:06:22.903 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.903 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:06:22.903 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.952749+0000", 2024-09-20T19:06:22.903 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:22.904 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.904 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.904 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:06:22.904 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.952749+0000", 2024-09-20T19:06:22.904 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:22.904 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.904 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.904 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:06:22.904 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.952751+0000", 2024-09-20T19:06:22.904 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6619999999999999e-06 2024-09-20T19:06:22.904 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.904 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.904 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:06:22.904 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.952757+0000", 2024-09-20T19:06:22.904 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.3090000000000001e-06 2024-09-20T19:06:22.905 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.905 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.905 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:06:22.905 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.952769+0000", 2024-09-20T19:06:22.905 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1769e-05 2024-09-20T19:06:22.905 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.905 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.905 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:06:22.905 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.952785+0000", 2024-09-20T19:06:22.905 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.554e-05 2024-09-20T19:06:22.905 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.905 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.905 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:22.905 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.952854+0000", 2024-09-20T19:06:22.906 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.9089000000000002e-05 2024-09-20T19:06:22.906 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.906 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.906 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:06:22.906 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.952865+0000", 2024-09-20T19:06:22.906 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1211e-05 2024-09-20T19:06:22.906 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.906 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.906 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:22.906 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.948741+0000", 2024-09-20T19:06:22.906 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.99587615399999996 2024-09-20T19:06:22.906 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.906 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.906 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:06:22.907 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.948950+0000", 2024-09-20T19:06:22.907 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00020904800000000001 2024-09-20T19:06:22.907 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.907 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.907 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,4", 2024-09-20T19:06:22.907 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.949070+0000", 2024-09-20T19:06:22.907 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000120241 2024-09-20T19:06:22.907 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.907 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.907 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:22.907 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.952325+0000", 2024-09-20T19:06:22.907 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0032544190000000001 2024-09-20T19:06:22.907 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.907 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.907 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:06:22.908 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.961916+0000", 2024-09-20T19:06:22.908 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0095912749999999998 2024-09-20T19:06:22.908 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.908 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.908 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:22.908 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.961956+0000", 2024-09-20T19:06:22.908 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9975999999999998e-05 2024-09-20T19:06:22.908 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.908 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.908 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:06:22.908 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.961993+0000", 2024-09-20T19:06:22.908 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.7076000000000002e-05 2024-09-20T19:06:22.908 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.908 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.909 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:06:22.909 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.962022+0000", 2024-09-20T19:06:22.909 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9300999999999999e-05 2024-09-20T19:06:22.909 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:22.909 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:06:22.909 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:22.909 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.909 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.909 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:196222 10.6 10:61e1d2b6:::benchmark_data_smithi083_148100_object79360:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e394)", 2024-09-20T19:06:22.909 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:20.969258+0000", 2024-09-20T19:06:22.909 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 61.835758046000002, 2024-09-20T19:06:22.909 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.99987098600000002, 2024-09-20T19:06:22.909 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:06:22.909 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:06:22.909 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:22.910 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:06:22.910 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:06:22.910 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:06:22.910 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 196222 2024-09-20T19:06:22.910 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.910 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:06:22.910 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.910 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:06:22.910 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.969258+0000", 2024-09-20T19:06:22.910 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:22.910 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.910 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.910 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:06:22.911 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.969258+0000", 2024-09-20T19:06:22.911 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:22.911 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.911 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.911 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:06:22.911 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.969259+0000", 2024-09-20T19:06:22.911 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.491e-06 2024-09-20T19:06:22.911 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.911 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.911 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:06:22.911 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.969264+0000", 2024-09-20T19:06:22.911 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3540000000000002e-06 2024-09-20T19:06:22.911 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.911 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.911 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:06:22.912 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.969280+0000", 2024-09-20T19:06:22.912 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6056e-05 2024-09-20T19:06:22.912 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.912 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.912 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:06:22.912 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.969289+0000", 2024-09-20T19:06:22.912 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.9469999999999992e-06 2024-09-20T19:06:22.912 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.912 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.912 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:22.912 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.969627+0000", 2024-09-20T19:06:22.912 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00033818500000000002 2024-09-20T19:06:22.912 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.912 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.912 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:06:22.913 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.969650+0000", 2024-09-20T19:06:22.913 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2878999999999999e-05 2024-09-20T19:06:22.913 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.913 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.913 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:22.913 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.952352+0000", 2024-09-20T19:06:22.913 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.98270262399999997 2024-09-20T19:06:22.913 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.913 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.913 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:06:22.913 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.952449+0000", 2024-09-20T19:06:22.913 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.6273000000000005e-05 2024-09-20T19:06:22.913 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.913 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.914 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,4", 2024-09-20T19:06:22.914 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.952555+0000", 2024-09-20T19:06:22.914 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000106038 2024-09-20T19:06:22.914 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.914 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.914 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:06:22.914 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.961921+0000", 2024-09-20T19:06:22.914 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0093659710000000007 2024-09-20T19:06:22.914 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.914 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.914 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:22.914 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.962078+0000", 2024-09-20T19:06:22.914 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000157458 2024-09-20T19:06:22.914 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.915 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.915 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:22.915 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.968986+0000", 2024-09-20T19:06:22.915 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0069078810000000003 2024-09-20T19:06:22.915 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.915 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.915 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:06:22.915 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.969081+0000", 2024-09-20T19:06:22.915 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.5468000000000001e-05 2024-09-20T19:06:22.915 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.915 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.915 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:06:22.915 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.969129+0000", 2024-09-20T19:06:22.915 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7361000000000003e-05 2024-09-20T19:06:22.916 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:22.916 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:06:22.916 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:22.916 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.916 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.916 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:196225 10.6 10:6fb9381d:::benchmark_data_smithi083_148100_object79363:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e394)", 2024-09-20T19:06:22.916 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:20.970076+0000", 2024-09-20T19:06:22.916 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 61.834940060999998, 2024-09-20T19:06:22.916 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.99929758400000002, 2024-09-20T19:06:22.916 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:06:22.916 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:06:22.916 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:22.916 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:06:22.917 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:06:22.917 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:06:22.917 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 196225 2024-09-20T19:06:22.917 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.917 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:06:22.917 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.917 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:06:22.917 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.970076+0000", 2024-09-20T19:06:22.917 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:22.917 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.917 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.917 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:06:22.917 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.970076+0000", 2024-09-20T19:06:22.917 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:22.917 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.918 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.918 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:06:22.918 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.970085+0000", 2024-09-20T19:06:22.918 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.2269999999999999e-06 2024-09-20T19:06:22.918 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.918 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.918 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:06:22.918 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.970090+0000", 2024-09-20T19:06:22.918 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7040000000000002e-06 2024-09-20T19:06:22.918 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.918 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.918 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:06:22.918 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.970101+0000", 2024-09-20T19:06:22.918 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1779e-05 2024-09-20T19:06:22.919 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.919 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.919 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:06:22.919 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.970109+0000", 2024-09-20T19:06:22.919 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.4680000000000003e-06 2024-09-20T19:06:22.919 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.919 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.919 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:22.919 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.970171+0000", 2024-09-20T19:06:22.919 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.1754000000000005e-05 2024-09-20T19:06:22.919 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.919 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.919 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:06:22.919 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.970179+0000", 2024-09-20T19:06:22.920 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.3280000000000006e-06 2024-09-20T19:06:22.920 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.920 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.920 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:22.920 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.955348+0000", 2024-09-20T19:06:22.920 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.98516891799999995 2024-09-20T19:06:22.920 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.920 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.920 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:06:22.920 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.955467+0000", 2024-09-20T19:06:22.920 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011861799999999999 2024-09-20T19:06:22.920 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.920 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.920 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,4", 2024-09-20T19:06:22.920 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.955581+0000", 2024-09-20T19:06:22.921 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011460599999999999 2024-09-20T19:06:22.921 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.921 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.921 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:06:22.921 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.962030+0000", 2024-09-20T19:06:22.921 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0064484900000000003 2024-09-20T19:06:22.921 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.921 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.921 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:22.921 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.969205+0000", 2024-09-20T19:06:22.921 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0071753449999999996 2024-09-20T19:06:22.921 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.921 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.921 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:22.922 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.969268+0000", 2024-09-20T19:06:22.922 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2830000000000007e-05 2024-09-20T19:06:22.922 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.922 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.922 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:06:22.922 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.969336+0000", 2024-09-20T19:06:22.922 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.7737999999999994e-05 2024-09-20T19:06:22.922 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.922 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.922 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:06:22.922 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.969373+0000", 2024-09-20T19:06:22.922 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.7778999999999999e-05 2024-09-20T19:06:22.922 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:22.922 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:06:22.923 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:22.923 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.923 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.923 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:196235 10.6 10:667fa116:::benchmark_data_smithi083_148100_object79373:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e394)", 2024-09-20T19:06:22.923 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:20.981304+0000", 2024-09-20T19:06:22.923 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 61.823711891000002, 2024-09-20T19:06:22.923 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.98825587299999995, 2024-09-20T19:06:22.923 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:06:22.923 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:06:22.923 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:22.923 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:06:22.923 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:06:22.923 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:06:22.923 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 196235 2024-09-20T19:06:22.923 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.924 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:06:22.924 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.924 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:06:22.924 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.981304+0000", 2024-09-20T19:06:22.924 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:22.924 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.924 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.924 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:06:22.924 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.981304+0000", 2024-09-20T19:06:22.924 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:22.924 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.924 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.924 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:06:22.925 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.981306+0000", 2024-09-20T19:06:22.925 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8029999999999999e-06 2024-09-20T19:06:22.925 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.925 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.925 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:06:22.925 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.981311+0000", 2024-09-20T19:06:22.925 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.7239999999999997e-06 2024-09-20T19:06:22.925 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.925 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.925 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:06:22.925 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.981327+0000", 2024-09-20T19:06:22.925 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5954999999999999e-05 2024-09-20T19:06:22.925 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.925 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.925 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:06:22.926 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.981336+0000", 2024-09-20T19:06:22.926 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.5599999999999994e-06 2024-09-20T19:06:22.926 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.926 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.926 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:22.926 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.981434+0000", 2024-09-20T19:06:22.926 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.8542000000000006e-05 2024-09-20T19:06:22.926 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.926 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.926 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:06:22.926 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.981444+0000", 2024-09-20T19:06:22.926 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.7480000000000005e-06 2024-09-20T19:06:22.926 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.926 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.927 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:22.927 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.958453+0000", 2024-09-20T19:06:22.927 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.97700898800000002 2024-09-20T19:06:22.927 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.927 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.927 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:06:22.927 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.958579+0000", 2024-09-20T19:06:22.927 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000125683 2024-09-20T19:06:22.927 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.927 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.927 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,4", 2024-09-20T19:06:22.927 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.958697+0000", 2024-09-20T19:06:22.927 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011854 2024-09-20T19:06:22.927 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.927 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.928 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:06:22.928 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.969142+0000", 2024-09-20T19:06:22.928 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.010444248 2024-09-20T19:06:22.928 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.928 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.928 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:22.928 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.969425+0000", 2024-09-20T19:06:22.928 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00028332200000000002 2024-09-20T19:06:22.928 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.928 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.928 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:22.928 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.969476+0000", 2024-09-20T19:06:22.928 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0565999999999997e-05 2024-09-20T19:06:22.929 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.929 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.929 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:06:22.929 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.969522+0000", 2024-09-20T19:06:22.929 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.6745000000000001e-05 2024-09-20T19:06:22.929 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.929 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.929 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:06:22.929 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.969560+0000", 2024-09-20T19:06:22.929 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.7449e-05 2024-09-20T19:06:22.929 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:22.929 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:06:22.929 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:22.929 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.930 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.930 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:196237 10.6 10:62e94daf:::benchmark_data_smithi083_148100_object79375:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e394)", 2024-09-20T19:06:22.930 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:20.982454+0000", 2024-09-20T19:06:22.930 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 61.822562114, 2024-09-20T19:06:22.930 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.987291221, 2024-09-20T19:06:22.930 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:06:22.930 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:06:22.930 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:22.930 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:06:22.930 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:06:22.930 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:06:22.930 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 196237 2024-09-20T19:06:22.930 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.930 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:06:22.931 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.931 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:06:22.931 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.982454+0000", 2024-09-20T19:06:22.931 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:22.931 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.931 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.931 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:06:22.931 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.982454+0000", 2024-09-20T19:06:22.931 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:22.931 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.931 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.931 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:06:22.931 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.982456+0000", 2024-09-20T19:06:22.932 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8589999999999999e-06 2024-09-20T19:06:22.932 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.932 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.932 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:06:22.932 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.982461+0000", 2024-09-20T19:06:22.932 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0300000000000001e-06 2024-09-20T19:06:22.932 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.932 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.932 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:06:22.932 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.982475+0000", 2024-09-20T19:06:22.932 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3986e-05 2024-09-20T19:06:22.932 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.932 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.932 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:06:22.932 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.982483+0000", 2024-09-20T19:06:22.933 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.3410000000000003e-06 2024-09-20T19:06:22.933 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.933 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.933 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:22.933 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.982573+0000", 2024-09-20T19:06:22.933 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.0099000000000006e-05 2024-09-20T19:06:22.933 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.933 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.933 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:06:22.933 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.982583+0000", 2024-09-20T19:06:22.933 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.5189999999999998e-06 2024-09-20T19:06:22.933 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.933 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.933 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:22.934 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.962104+0000", 2024-09-20T19:06:22.934 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.97952148100000003 2024-09-20T19:06:22.934 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.934 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.934 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:06:22.934 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.962278+0000", 2024-09-20T19:06:22.934 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00017353299999999999 2024-09-20T19:06:22.934 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.934 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.934 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,4", 2024-09-20T19:06:22.934 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.962386+0000", 2024-09-20T19:06:22.934 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000108365 2024-09-20T19:06:22.934 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.934 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.935 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:06:22.935 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.969149+0000", 2024-09-20T19:06:22.935 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00676342 2024-09-20T19:06:22.935 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.935 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.935 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:22.935 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.969612+0000", 2024-09-20T19:06:22.935 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00046227099999999999 2024-09-20T19:06:22.935 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.935 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.935 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:22.935 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.969665+0000", 2024-09-20T19:06:22.935 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.2988999999999997e-05 2024-09-20T19:06:22.935 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.935 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.936 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:06:22.936 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.969710+0000", 2024-09-20T19:06:22.936 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.4975000000000002e-05 2024-09-20T19:06:22.936 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.936 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.936 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:06:22.936 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.969745+0000", 2024-09-20T19:06:22.936 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.5352999999999998e-05 2024-09-20T19:06:22.936 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:22.936 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:06:22.936 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:22.936 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.936 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.936 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:196260 10.6 10:683aac99:::benchmark_data_smithi083_148100_object79398:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e394)", 2024-09-20T19:06:22.936 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:21.007238+0000", 2024-09-20T19:06:22.937 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 61.797777815000003, 2024-09-20T19:06:22.937 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.96912988799999999, 2024-09-20T19:06:22.937 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:06:22.937 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:06:22.937 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:22.937 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:06:22.937 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:06:22.937 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:06:22.937 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 196260 2024-09-20T19:06:22.937 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.937 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:06:22.937 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.937 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:06:22.937 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.007238+0000", 2024-09-20T19:06:22.937 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:22.938 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.938 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.938 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:06:22.938 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.007238+0000", 2024-09-20T19:06:22.938 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:22.938 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.938 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.938 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:06:22.938 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.007240+0000", 2024-09-20T19:06:22.938 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3769999999999999e-06 2024-09-20T19:06:22.938 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.938 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.938 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:06:22.938 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.007248+0000", 2024-09-20T19:06:22.939 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5939999999999996e-06 2024-09-20T19:06:22.939 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.939 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.939 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:06:22.939 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.007267+0000", 2024-09-20T19:06:22.939 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8885999999999998e-05 2024-09-20T19:06:22.939 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.939 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.939 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:06:22.939 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.007277+0000", 2024-09-20T19:06:22.939 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0519999999999999e-05 2024-09-20T19:06:22.939 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.939 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.939 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:22.939 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.007358+0000", 2024-09-20T19:06:22.940 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.0189000000000001e-05 2024-09-20T19:06:22.940 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.940 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.940 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:06:22.940 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.007369+0000", 2024-09-20T19:06:22.940 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1046000000000001e-05 2024-09-20T19:06:22.940 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.940 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.940 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:22.940 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.965540+0000", 2024-09-20T19:06:22.940 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.95817183100000003 2024-09-20T19:06:22.940 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.940 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.940 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:06:22.940 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.965695+0000", 2024-09-20T19:06:22.941 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000154482 2024-09-20T19:06:22.941 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.941 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.941 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,4", 2024-09-20T19:06:22.941 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.965812+0000", 2024-09-20T19:06:22.941 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000117409 2024-09-20T19:06:22.941 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.941 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.941 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:06:22.941 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.969389+0000", 2024-09-20T19:06:22.941 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0035766169999999998 2024-09-20T19:06:22.941 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.941 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.941 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:22.942 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.969811+0000", 2024-09-20T19:06:22.942 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00042253000000000001 2024-09-20T19:06:22.942 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.942 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.942 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:22.942 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.976270+0000", 2024-09-20T19:06:22.942 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0064581120000000002 2024-09-20T19:06:22.942 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.942 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.942 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:06:22.942 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.976327+0000", 2024-09-20T19:06:22.942 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.7305000000000001e-05 2024-09-20T19:06:22.942 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.942 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.943 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:06:22.943 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.976368+0000", 2024-09-20T19:06:22.943 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0989999999999999e-05 2024-09-20T19:06:22.943 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:22.943 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:06:22.943 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:22.943 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.943 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.943 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:196295 10.6 10:605954dd:::benchmark_data_smithi083_148100_object79433:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e394)", 2024-09-20T19:06:22.943 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:21.077278+0000", 2024-09-20T19:06:22.943 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 61.727738127000002, 2024-09-20T19:06:22.943 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.90257779000000005, 2024-09-20T19:06:22.943 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:06:22.943 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:06:22.944 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:22.944 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:06:22.944 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:06:22.944 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:06:22.944 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 196295 2024-09-20T19:06:22.944 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.944 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:06:22.944 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.944 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:06:22.944 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.077278+0000", 2024-09-20T19:06:22.944 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:22.944 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.944 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.944 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:06:22.944 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.077278+0000", 2024-09-20T19:06:22.945 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:22.945 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.945 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.945 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:06:22.945 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.077280+0000", 2024-09-20T19:06:22.945 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.88e-06 2024-09-20T19:06:22.945 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.945 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.945 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:06:22.945 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.077287+0000", 2024-09-20T19:06:22.945 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.1439999999999997e-06 2024-09-20T19:06:22.945 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.945 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.945 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:06:22.946 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.077302+0000", 2024-09-20T19:06:22.946 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4888e-05 2024-09-20T19:06:22.946 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.946 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.946 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:06:22.946 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.077308+0000", 2024-09-20T19:06:22.946 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.8000000000000001e-06 2024-09-20T19:06:22.946 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.946 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.946 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:22.946 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.077365+0000", 2024-09-20T19:06:22.946 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.7058000000000002e-05 2024-09-20T19:06:22.946 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.946 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.947 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:06:22.947 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.077372+0000", 2024-09-20T19:06:22.947 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.9709999999999998e-06 2024-09-20T19:06:22.947 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.947 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.947 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:22.947 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.969853+0000", 2024-09-20T19:06:22.947 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.89248060600000001 2024-09-20T19:06:22.947 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.947 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.947 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:06:22.947 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.969955+0000", 2024-09-20T19:06:22.947 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000101699 2024-09-20T19:06:22.947 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.947 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.948 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,4", 2024-09-20T19:06:22.948 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.970067+0000", 2024-09-20T19:06:22.948 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011228800000000001 2024-09-20T19:06:22.948 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.948 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.948 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:06:22.948 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.979653+0000", 2024-09-20T19:06:22.948 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0095861339999999996 2024-09-20T19:06:22.948 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.948 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.948 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:22.948 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.979713+0000", 2024-09-20T19:06:22.948 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0353999999999998e-05 2024-09-20T19:06:22.948 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.949 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.949 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:22.949 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.979758+0000", 2024-09-20T19:06:22.949 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.4894000000000001e-05 2024-09-20T19:06:22.949 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.949 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.949 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:06:22.949 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.979812+0000", 2024-09-20T19:06:22.949 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3164999999999997e-05 2024-09-20T19:06:22.949 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:22.949 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:22.949 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:06:22.949 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.979855+0000", 2024-09-20T19:06:22.949 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3909000000000003e-05 2024-09-20T19:06:22.949 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:22.950 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:06:22.950 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:22.950 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:22.950 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:06:22.950 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T19:06:22.950 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T19:06:22.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:22.952 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:22.952 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:22.986 DEBUG:teuthology.orchestra.run.smithi083:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_4 bench 90 write' 2024-09-20T19:06:22.996 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-20T19:06:22.996 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-20T19:06:23.053 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:23.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:23.078 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:23.078 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:23.110 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:hints = 1 2024-09-20T19:06:23.110 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-20T19:06:23.110 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Object prefix: benchmark_data_smithi083_177491 2024-09-20T19:06:23.111 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-20T19:06:23.111 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 0 0 0 0 0 0 - 0 2024-09-20T19:06:23.146 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:06:23.146 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:06:23.146 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:06:23.146 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:06:23.156 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:22 smithi142 ceph-mon[32508]: 5.4 scrub starts 2024-09-20T19:06:23.156 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:22 smithi142 ceph-mon[32508]: 5.4 scrub ok 2024-09-20T19:06:23.157 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:22 smithi142 ceph-mon[32508]: 4.10 scrub starts 2024-09-20T19:06:23.157 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:22 smithi142 ceph-mon[32508]: 4.10 scrub ok 2024-09-20T19:06:23.157 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:22 smithi142 ceph-mon[32508]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-20T19:06:23.157 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:22 smithi142 ceph-mon[32508]: Cluster is now healthy 2024-09-20T19:06:23.179 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:23.180 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:06:23.180 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:06:23.218 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:06:23.219 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:06:23.219 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:22 smithi083 ceph-mon[29421]: 5.4 scrub starts 2024-09-20T19:06:23.219 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:22 smithi083 ceph-mon[29421]: 5.4 scrub ok 2024-09-20T19:06:23.219 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:22 smithi083 ceph-mon[29421]: 4.10 scrub starts 2024-09-20T19:06:23.219 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:22 smithi083 ceph-mon[29421]: 4.10 scrub ok 2024-09-20T19:06:23.219 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:22 smithi083 ceph-mon[29421]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-20T19:06:23.219 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:22 smithi083 ceph-mon[29421]: Cluster is now healthy 2024-09-20T19:06:23.220 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:22 smithi083 ceph-mon[36260]: 5.4 scrub starts 2024-09-20T19:06:23.220 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:22 smithi083 ceph-mon[36260]: 5.4 scrub ok 2024-09-20T19:06:23.220 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:22 smithi083 ceph-mon[36260]: 4.10 scrub starts 2024-09-20T19:06:23.220 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:22 smithi083 ceph-mon[36260]: 4.10 scrub ok 2024-09-20T19:06:23.220 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:22 smithi083 ceph-mon[36260]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-20T19:06:23.220 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:22 smithi083 ceph-mon[36260]: Cluster is now healthy 2024-09-20T19:06:23.223 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:23.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:23.225 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:23.249 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:06:23.250 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:06:23.273 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:06:23.284 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:06:23.284 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:06:23.323 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:06:23.324 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:06:23.326 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:23.359 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:06:23.360 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:06:23.366 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:23.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:23.369 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:23.470 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:23.504 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:23.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:23.505 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:23.605 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:06:23.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:23.665 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:23.665 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:06:23.766 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:06:23.804 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:23.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:23.806 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:06:23.906 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:23.943 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-20T19:06:23.944 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:23.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:23.945 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:24.046 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:24.088 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:24.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:24.089 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:24.111 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 1 16 3106 3090 192.96 193.125 0.00912632 0.00515082 2024-09-20T19:06:24.190 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:24.220 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:24.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:24.221 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:24.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:23 smithi142 ceph-mon[32508]: 5.13 scrub starts 2024-09-20T19:06:24.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:23 smithi142 ceph-mon[32508]: 5.13 scrub ok 2024-09-20T19:06:24.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:23 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3469255610' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:06:24.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:23 smithi142 ceph-mon[32508]: pgmap v839: 148 pgs: 148 active+clean; 583 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1000 B/s rd, 0 op/s 2024-09-20T19:06:24.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:23 smithi083 ceph-mon[29421]: 5.13 scrub starts 2024-09-20T19:06:24.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:23 smithi083 ceph-mon[29421]: 5.13 scrub ok 2024-09-20T19:06:24.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:23 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3469255610' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:06:24.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:23 smithi083 ceph-mon[29421]: pgmap v839: 148 pgs: 148 active+clean; 583 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1000 B/s rd, 0 op/s 2024-09-20T19:06:24.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:23 smithi083 ceph-mon[36260]: 5.13 scrub starts 2024-09-20T19:06:24.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:23 smithi083 ceph-mon[36260]: 5.13 scrub ok 2024-09-20T19:06:24.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:23 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3469255610' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:06:24.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:23 smithi083 ceph-mon[36260]: pgmap v839: 148 pgs: 148 active+clean; 583 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1000 B/s rd, 0 op/s 2024-09-20T19:06:24.323 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:24.355 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:06:24.360 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:24.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:24.361 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:24.462 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:24.501 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:24.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:24.503 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:24.543 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T19:06:24.545 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-20T19:06:24.603 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:24.640 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:24.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:24.641 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:24.741 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:24.777 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:24.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:24.778 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:24.879 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:24.916 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:24.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:24.917 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:25.001 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:06:25.009 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T19:06:25.009 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:06:25.018 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:06:25.024 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:24 smithi142 ceph-mon[32508]: from='client.16422 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:06:25.066 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:25.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:25.067 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:06:25.112 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 2 15 6133 6118 191.096 189.25 0.00346371 0.00522483 2024-09-20T19:06:25.139 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:24 smithi083 ceph-mon[29421]: from='client.16422 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:06:25.139 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:24 smithi083 ceph-mon[36260]: from='client.16422 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:06:25.169 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:25.201 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:25.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:25.202 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:25.303 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:25.337 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:25.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:25.338 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:25.438 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:25.479 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:25.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:25.480 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:25.581 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:25.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:25.619 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:25.619 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:25.720 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:25.731 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":839,"stamp":"2024-09-20T19:06:23.161724+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":2245,"num_read_kb":2102,"num_write":835,"num_write_kb":1673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":1779163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":868,"ondisk_log_size":868,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":441,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":4580304,"kb_used_data":50076,"kb_used_omap":16,"kb_used_meta":4530031,"kb_avail":745380912,"statfs":{"total":767960285184,"available":763270053888,"internally_reserved":0,"allocated":51277824,"data_stored":46606669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16790,"internal_metadata":4638752362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.118109"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":783,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.400309+0000","last_change":"2024-09-20T19:01:31.817967+0000","last_active":"2024-09-20T19:06:21.400309+0000","last_peered":"2024-09-20T19:06:21.400309+0000","last_clean":"2024-09-20T19:06:21.400309+0000","last_became_active":"2024-09-20T18:49:45.943201+0000","last_became_peered":"2024-09-20T18:49:45.943201+0000","last_unstale":"2024-09-20T19:06:21.400309+0000","last_undegraded":"2024-09-20T19:06:21.400309+0000","last_fullsized":"2024-09-20T19:06:21.400309+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:31.817909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:31.817909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:00.513933+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"273'15","reported_seq":121,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:20.172659+0000","last_change":"2024-09-20T19:04:21.040978+0000","last_active":"2024-09-20T19:06:20.172659+0000","last_peered":"2024-09-20T19:06:20.172659+0000","last_clean":"2024-09-20T19:06:20.172659+0000","last_became_active":"2024-09-20T19:04:20.925853+0000","last_became_peered":"2024-09-20T19:04:20.925853+0000","last_unstale":"2024-09-20T19:06:20.172659+0000","last_undegraded":"2024-09-20T19:06:20.172659+0000","last_fullsized":"2024-09-20T19:06:20.172659+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:03:10.214154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:10.214154+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":773,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.065742+0000","last_change":"2024-09-20T18:57:49.016078+0000","last_active":"2024-09-20T19:06:21.065742+0000","last_peered":"2024-09-20T19:06:21.065742+0000","last_clean":"2024-09-20T19:06:21.065742+0000","last_became_active":"2024-09-20T18:49:49.959514+0000","last_became_peered":"2024-09-20T18:49:49.959514+0000","last_unstale":"2024-09-20T19:06:21.065742+0000","last_undegraded":"2024-09-20T19:06:21.065742+0000","last_fullsized":"2024-09-20T19:06:21.065742+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:49.016006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:49.016006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:25.184023+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":111,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:20.172888+0000","last_change":"2024-09-20T19:04:20.881918+0000","last_active":"2024-09-20T19:06:20.172888+0000","last_peered":"2024-09-20T19:06:20.172888+0000","last_clean":"2024-09-20T19:06:20.172888+0000","last_became_active":"2024-09-20T19:04:20.880905+0000","last_became_peered":"2024-09-20T19:04:20.880905+0000","last_unstale":"2024-09-20T19:06:20.172888+0000","last_undegraded":"2024-09-20T19:06:20.172888+0000","last_fullsized":"2024-09-20T19:06:20.172888+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:02.084375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:02.084375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:37.200857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":779,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T19:06:19.389580+0000","last_change":"2024-09-20T19:02:02.441633+0000","last_active":"2024-09-20T19:06:19.389580+0000","last_peered":"2024-09-20T19:06:19.389580+0000","last_clean":"2024-09-20T19:06:19.389580+0000","last_became_active":"2024-09-20T18:59:05.081081+0000","last_became_peered":"2024-09-20T18:59:05.081081+0000","last_unstale":"2024-09-20T19:06:19.389580+0000","last_undegraded":"2024-09-20T19:06:19.389580+0000","last_fullsized":"2024-09-20T19:06:19.389580+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:02.441577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:02.441577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:11.774126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":793,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T19:06:19.390140+0000","last_change":"2024-09-20T19:04:37.540824+0000","last_active":"2024-09-20T19:06:19.390140+0000","last_peered":"2024-09-20T19:06:19.390140+0000","last_clean":"2024-09-20T19:06:19.390140+0000","last_became_active":"2024-09-20T18:59:05.072675+0000","last_became_peered":"2024-09-20T18:59:05.072675+0000","last_unstale":"2024-09-20T19:06:19.390140+0000","last_undegraded":"2024-09-20T19:06:19.390140+0000","last_fullsized":"2024-09-20T19:06:19.390140+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:37.540760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:04:37.540760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"274'27","reported_seq":824,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T19:06:19.389997+0000","last_change":"2024-09-20T19:05:50.750134+0000","last_active":"2024-09-20T19:06:19.389997+0000","last_peered":"2024-09-20T19:06:19.389997+0000","last_clean":"2024-09-20T19:06:19.389997+0000","last_became_active":"2024-09-20T18:49:47.952150+0000","last_became_peered":"2024-09-20T18:49:47.952150+0000","last_unstale":"2024-09-20T19:06:19.389997+0000","last_undegraded":"2024-09-20T19:06:19.389997+0000","last_fullsized":"2024-09-20T19:06:19.389997+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'27","last_scrub_stamp":"2024-09-20T19:05:50.750050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:50.750050+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:13.513790+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":111,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:20.172654+0000","last_change":"2024-09-20T19:04:20.926234+0000","last_active":"2024-09-20T19:06:20.172654+0000","last_peered":"2024-09-20T19:06:20.172654+0000","last_clean":"2024-09-20T19:06:20.172654+0000","last_became_active":"2024-09-20T19:04:20.925394+0000","last_became_peered":"2024-09-20T19:04:20.925394+0000","last_unstale":"2024-09-20T19:06:20.172654+0000","last_undegraded":"2024-09-20T19:06:20.172654+0000","last_fullsized":"2024-09-20T19:06:20.172654+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:23.982813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:23.982813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"59'1","reported_seq":856,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:20.059680+0000","last_change":"2024-09-20T19:04:20.963496+0000","last_active":"2024-09-20T19:06:20.059680+0000","last_peered":"2024-09-20T19:06:20.059680+0000","last_clean":"2024-09-20T19:06:20.059680+0000","last_became_active":"2024-09-20T19:04:20.914145+0000","last_became_peered":"2024-09-20T19:04:20.914145+0000","last_unstale":"2024-09-20T19:06:20.059680+0000","last_undegraded":"2024-09-20T19:06:20.059680+0000","last_fullsized":"2024-09-20T19:06:20.059680+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:01:28.592653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:28.592653+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:29.742562+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"273'15","reported_seq":121,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:20.172921+0000","last_change":"2024-09-20T19:04:20.984223+0000","last_active":"2024-09-20T19:06:20.172921+0000","last_peered":"2024-09-20T19:06:20.172921+0000","last_clean":"2024-09-20T19:06:20.172921+0000","last_became_active":"2024-09-20T19:04:20.898235+0000","last_became_peered":"2024-09-20T19:04:20.898235+0000","last_unstale":"2024-09-20T19:06:20.172921+0000","last_undegraded":"2024-09-20T19:06:20.172921+0000","last_fullsized":"2024-09-20T19:06:20.172921+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:02:42.114331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:42.114331+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":779,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:20.059323+0000","last_change":"2024-09-20T19:01:51.564693+0000","last_active":"2024-09-20T19:06:20.059323+0000","last_peered":"2024-09-20T19:06:20.059323+0000","last_clean":"2024-09-20T19:06:20.059323+0000","last_became_active":"2024-09-20T18:49:49.961188+0000","last_became_peered":"2024-09-20T18:49:49.961188+0000","last_unstale":"2024-09-20T19:06:20.059323+0000","last_undegraded":"2024-09-20T19:06:20.059323+0000","last_fullsized":"2024-09-20T19:06:20.059323+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:51.564624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:51.564624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:41.667335+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":762,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.399708+0000","last_change":"2024-09-20T19:03:09.963421+0000","last_active":"2024-09-20T19:06:21.399708+0000","last_peered":"2024-09-20T19:06:21.399708+0000","last_clean":"2024-09-20T19:06:21.399708+0000","last_became_active":"2024-09-20T18:49:51.966715+0000","last_became_peered":"2024-09-20T18:49:51.966715+0000","last_unstale":"2024-09-20T19:06:21.399708+0000","last_undegraded":"2024-09-20T19:06:21.399708+0000","last_fullsized":"2024-09-20T19:06:21.399708+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:09.963298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:03:09.963298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":804,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:20.059460+0000","last_change":"2024-09-20T19:04:20.879227+0000","last_active":"2024-09-20T19:06:20.059460+0000","last_peered":"2024-09-20T19:06:20.059460+0000","last_clean":"2024-09-20T19:06:20.059460+0000","last_became_active":"2024-09-20T19:04:20.878905+0000","last_became_peered":"2024-09-20T19:04:20.878905+0000","last_unstale":"2024-09-20T19:06:20.059460+0000","last_undegraded":"2024-09-20T19:06:20.059460+0000","last_fullsized":"2024-09-20T19:06:20.059460+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:48.520017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:48.520017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:42.653960+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"273'7","reported_seq":125,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:20.173098+0000","last_change":"2024-09-20T19:05:53.980848+0000","last_active":"2024-09-20T19:06:20.173098+0000","last_peered":"2024-09-20T19:06:20.173098+0000","last_clean":"2024-09-20T19:06:20.173098+0000","last_became_active":"2024-09-20T19:04:20.888572+0000","last_became_peered":"2024-09-20T19:04:20.888572+0000","last_unstale":"2024-09-20T19:06:20.173098+0000","last_undegraded":"2024-09-20T19:06:20.173098+0000","last_fullsized":"2024-09-20T19:06:20.173098+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'7","last_scrub_stamp":"2024-09-20T19:05:53.980772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:53.980772+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:54.857238+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":797,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.140408+0000","last_change":"2024-09-20T19:04:20.885873+0000","last_active":"2024-09-20T19:06:21.140408+0000","last_peered":"2024-09-20T19:06:21.140408+0000","last_clean":"2024-09-20T19:06:21.140408+0000","last_became_active":"2024-09-20T19:04:20.885417+0000","last_became_peered":"2024-09-20T19:04:20.885417+0000","last_unstale":"2024-09-20T19:06:21.140408+0000","last_undegraded":"2024-09-20T19:06:21.140408+0000","last_fullsized":"2024-09-20T19:06:21.140408+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:33.203312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:33.203312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:05.802150+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":775,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:20.059082+0000","last_change":"2024-09-20T19:05:21.625378+0000","last_active":"2024-09-20T19:06:20.059082+0000","last_peered":"2024-09-20T19:06:20.059082+0000","last_clean":"2024-09-20T19:06:20.059082+0000","last_became_active":"2024-09-20T18:49:51.970663+0000","last_became_peered":"2024-09-20T18:49:51.970663+0000","last_unstale":"2024-09-20T19:06:20.059082+0000","last_undegraded":"2024-09-20T19:06:20.059082+0000","last_fullsized":"2024-09-20T19:06:20.059082+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:21.625311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:05:21.625311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:25.835298+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":112,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:20.173811+0000","last_change":"2024-09-20T19:04:20.907531+0000","last_active":"2024-09-20T19:06:20.173811+0000","last_peered":"2024-09-20T19:06:20.173811+0000","last_clean":"2024-09-20T19:06:20.173811+0000","last_became_active":"2024-09-20T19:04:20.906109+0000","last_became_peered":"2024-09-20T19:04:20.906109+0000","last_unstale":"2024-09-20T19:06:20.173811+0000","last_undegraded":"2024-09-20T19:06:20.173811+0000","last_fullsized":"2024-09-20T19:06:20.173811+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:00.906652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:00.906652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:39.140988+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":807,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.400219+0000","last_change":"2024-09-20T19:04:20.907057+0000","last_active":"2024-09-20T19:06:21.400219+0000","last_peered":"2024-09-20T19:06:21.400219+0000","last_clean":"2024-09-20T19:06:21.400219+0000","last_became_active":"2024-09-20T19:04:20.906691+0000","last_became_peered":"2024-09-20T19:04:20.906691+0000","last_unstale":"2024-09-20T19:06:21.400219+0000","last_undegraded":"2024-09-20T19:06:21.400219+0000","last_fullsized":"2024-09-20T19:06:21.400219+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:44.998147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:44.998147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"273'27","reported_seq":839,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.141119+0000","last_change":"2024-09-20T19:05:49.452928+0000","last_active":"2024-09-20T19:06:21.141119+0000","last_peered":"2024-09-20T19:06:21.141119+0000","last_clean":"2024-09-20T19:06:21.141119+0000","last_became_active":"2024-09-20T18:49:47.950939+0000","last_became_peered":"2024-09-20T18:49:47.950939+0000","last_unstale":"2024-09-20T19:06:21.141119+0000","last_undegraded":"2024-09-20T19:06:21.141119+0000","last_fullsized":"2024-09-20T19:06:21.141119+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'27","last_scrub_stamp":"2024-09-20T19:05:49.452861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:49.452861+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:02.753272+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":790,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.066325+0000","last_change":"2024-09-20T19:04:20.890659+0000","last_active":"2024-09-20T19:06:21.066325+0000","last_peered":"2024-09-20T19:06:21.066325+0000","last_clean":"2024-09-20T19:06:21.066325+0000","last_became_active":"2024-09-20T19:04:20.890370+0000","last_became_peered":"2024-09-20T19:04:20.890370+0000","last_unstale":"2024-09-20T19:06:21.066325+0000","last_undegraded":"2024-09-20T19:06:21.066325+0000","last_fullsized":"2024-09-20T19:06:21.066325+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:05.888974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:05.888974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:26.204424+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":788,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T19:06:19.390248+0000","last_change":"2024-09-20T19:06:17.937954+0000","last_active":"2024-09-20T19:06:19.390248+0000","last_peered":"2024-09-20T19:06:19.390248+0000","last_clean":"2024-09-20T19:06:19.390248+0000","last_became_active":"2024-09-20T18:59:05.084496+0000","last_became_peered":"2024-09-20T18:59:05.084496+0000","last_unstale":"2024-09-20T19:06:19.390248+0000","last_undegraded":"2024-09-20T19:06:19.390248+0000","last_fullsized":"2024-09-20T19:06:19.390248+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.937896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.937896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:37.574460+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"59'1","reported_seq":115,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:20.172814+0000","last_change":"2024-09-20T19:04:20.905233+0000","last_active":"2024-09-20T19:06:20.172814+0000","last_peered":"2024-09-20T19:06:20.172814+0000","last_clean":"2024-09-20T19:06:20.172814+0000","last_became_active":"2024-09-20T19:04:20.887062+0000","last_became_peered":"2024-09-20T19:04:20.887062+0000","last_unstale":"2024-09-20T19:06:20.172814+0000","last_undegraded":"2024-09-20T19:06:20.172814+0000","last_fullsized":"2024-09-20T19:06:20.172814+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:03:08.218812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:03:08.218812+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":993,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"274'18","reported_seq":830,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T19:06:19.389686+0000","last_change":"2024-09-20T19:04:20.925690+0000","last_active":"2024-09-20T19:06:19.389686+0000","last_peered":"2024-09-20T19:06:19.389686+0000","last_clean":"2024-09-20T19:06:19.389686+0000","last_became_active":"2024-09-20T19:04:20.896343+0000","last_became_peered":"2024-09-20T19:04:20.896343+0000","last_unstale":"2024-09-20T19:06:19.389686+0000","last_undegraded":"2024-09-20T19:06:19.389686+0000","last_fullsized":"2024-09-20T19:06:19.389686+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:57:09.818999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:09.818999+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:29.636168+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":119,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:20.173461+0000","last_change":"2024-09-20T19:06:16.148081+0000","last_active":"2024-09-20T19:06:20.173461+0000","last_peered":"2024-09-20T19:06:20.173461+0000","last_clean":"2024-09-20T19:06:20.173461+0000","last_became_active":"2024-09-20T19:04:20.889430+0000","last_became_peered":"2024-09-20T19:04:20.889430+0000","last_unstale":"2024-09-20T19:06:20.173461+0000","last_undegraded":"2024-09-20T19:06:20.173461+0000","last_fullsized":"2024-09-20T19:06:20.173461+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:16.148022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:06:16.148022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:27.803843+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":769,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.141530+0000","last_change":"2024-09-20T19:02:28.294390+0000","last_active":"2024-09-20T19:06:21.141530+0000","last_peered":"2024-09-20T19:06:21.141530+0000","last_clean":"2024-09-20T19:06:21.141530+0000","last_became_active":"2024-09-20T18:49:51.970058+0000","last_became_peered":"2024-09-20T18:49:51.970058+0000","last_unstale":"2024-09-20T19:06:21.141530+0000","last_undegraded":"2024-09-20T19:06:21.141530+0000","last_fullsized":"2024-09-20T19:06:21.141530+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T19:02:28.294292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:28.294292+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:58.767339+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":807,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.065890+0000","last_change":"2024-09-20T19:05:22.202134+0000","last_active":"2024-09-20T19:06:21.065890+0000","last_peered":"2024-09-20T19:06:21.065890+0000","last_clean":"2024-09-20T19:06:21.065890+0000","last_became_active":"2024-09-20T19:04:20.881466+0000","last_became_peered":"2024-09-20T19:04:20.881466+0000","last_unstale":"2024-09-20T19:06:21.065890+0000","last_undegraded":"2024-09-20T19:06:21.065890+0000","last_fullsized":"2024-09-20T19:06:21.065890+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:05:22.201836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:22.201836+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:45.062354+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"274'20","reported_seq":845,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.400182+0000","last_change":"2024-09-20T19:03:16.982915+0000","last_active":"2024-09-20T19:06:21.400182+0000","last_peered":"2024-09-20T19:06:21.400182+0000","last_clean":"2024-09-20T19:06:21.400182+0000","last_became_active":"2024-09-20T19:03:16.968636+0000","last_became_peered":"2024-09-20T19:03:16.968636+0000","last_unstale":"2024-09-20T19:06:21.400182+0000","last_undegraded":"2024-09-20T19:06:21.400182+0000","last_fullsized":"2024-09-20T19:06:21.400182+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'20","last_scrub_stamp":"2024-09-20T19:02:35.096979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:35.096979+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"375'14","reported_seq":488,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.400808+0000","last_change":"2024-09-20T19:04:20.956210+0000","last_active":"2024-09-20T19:06:21.400808+0000","last_peered":"2024-09-20T19:06:21.400808+0000","last_clean":"2024-09-20T19:06:21.400808+0000","last_became_active":"2024-09-20T19:04:20.900782+0000","last_became_peered":"2024-09-20T19:04:20.900782+0000","last_unstale":"2024-09-20T19:06:21.400808+0000","last_undegraded":"2024-09-20T19:06:21.400808+0000","last_fullsized":"2024-09-20T19:06:21.400808+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"308'12","last_scrub_stamp":"2024-09-20T19:01:44.541156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:44.541156+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:35.235276+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":19,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.065483+0000","last_change":"2024-09-20T19:06:18.053333+0000","last_active":"2024-09-20T19:06:21.065483+0000","last_peered":"2024-09-20T19:06:21.065483+0000","last_clean":"2024-09-20T19:06:21.065483+0000","last_became_active":"2024-09-20T19:06:18.053079+0000","last_became_peered":"2024-09-20T19:06:18.053079+0000","last_unstale":"2024-09-20T19:06:21.065483+0000","last_undegraded":"2024-09-20T19:06:21.065483+0000","last_fullsized":"2024-09-20T19:06:21.065483+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:38.562225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"274'27","reported_seq":864,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.141109+0000","last_change":"2024-09-20T19:04:20.920811+0000","last_active":"2024-09-20T19:06:21.141109+0000","last_peered":"2024-09-20T19:06:21.141109+0000","last_clean":"2024-09-20T19:06:21.141109+0000","last_became_active":"2024-09-20T19:04:20.897049+0000","last_became_peered":"2024-09-20T19:04:20.897049+0000","last_unstale":"2024-09-20T19:06:21.141109+0000","last_undegraded":"2024-09-20T19:06:21.141109+0000","last_fullsized":"2024-09-20T19:06:21.141109+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'27","last_scrub_stamp":"2024-09-20T19:02:27.897138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:27.897138+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:33.546677+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":786,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T19:06:19.388513+0000","last_change":"2024-09-20T19:05:55.742165+0000","last_active":"2024-09-20T19:06:19.388513+0000","last_peered":"2024-09-20T19:06:19.388513+0000","last_clean":"2024-09-20T19:06:19.388513+0000","last_became_active":"2024-09-20T18:49:45.936104+0000","last_became_peered":"2024-09-20T18:49:45.936104+0000","last_unstale":"2024-09-20T19:06:19.388513+0000","last_undegraded":"2024-09-20T19:06:19.388513+0000","last_fullsized":"2024-09-20T19:06:19.388513+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:55.742106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:55.742106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:16.985762+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"375'12","reported_seq":147,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:20.174206+0000","last_change":"2024-09-20T19:06:20.174206+0000","last_active":"2024-09-20T19:06:20.174206+0000","last_peered":"2024-09-20T19:06:20.174206+0000","last_clean":"2024-09-20T19:06:20.174206+0000","last_became_active":"2024-09-20T19:04:20.907228+0000","last_became_peered":"2024-09-20T19:04:20.907228+0000","last_unstale":"2024-09-20T19:06:20.174206+0000","last_undegraded":"2024-09-20T19:06:20.174206+0000","last_fullsized":"2024-09-20T19:06:20.174206+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"375'12","last_scrub_stamp":"2024-09-20T19:06:20.174160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:06:20.174160+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:49.543973+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":768,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:20.059350+0000","last_change":"2024-09-20T19:02:22.736879+0000","last_active":"2024-09-20T19:06:20.059350+0000","last_peered":"2024-09-20T19:06:20.059350+0000","last_clean":"2024-09-20T19:06:20.059350+0000","last_became_active":"2024-09-20T18:49:51.971399+0000","last_became_peered":"2024-09-20T18:49:51.971399+0000","last_unstale":"2024-09-20T19:06:20.059350+0000","last_undegraded":"2024-09-20T19:06:20.059350+0000","last_fullsized":"2024-09-20T19:06:20.059350+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:22.736825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:22.736825+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:02.798693+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":16,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.141518+0000","last_change":"2024-09-20T19:06:18.056190+0000","last_active":"2024-09-20T19:06:21.141518+0000","last_peered":"2024-09-20T19:06:21.141518+0000","last_clean":"2024-09-20T19:06:21.141518+0000","last_became_active":"2024-09-20T19:06:18.055847+0000","last_became_peered":"2024-09-20T19:06:18.055847+0000","last_unstale":"2024-09-20T19:06:21.141518+0000","last_undegraded":"2024-09-20T19:06:21.141518+0000","last_fullsized":"2024-09-20T19:06:21.141518+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:18.429594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"273'30","reported_seq":841,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T19:06:19.388346+0000","last_change":"2024-09-20T19:05:54.717703+0000","last_active":"2024-09-20T19:06:19.388346+0000","last_peered":"2024-09-20T19:06:19.388346+0000","last_clean":"2024-09-20T19:06:19.388346+0000","last_became_active":"2024-09-20T18:49:48.499189+0000","last_became_peered":"2024-09-20T18:49:48.499189+0000","last_unstale":"2024-09-20T19:06:19.388346+0000","last_undegraded":"2024-09-20T19:06:19.388346+0000","last_fullsized":"2024-09-20T19:06:19.388346+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'30","last_scrub_stamp":"2024-09-20T19:05:54.717609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:54.717609+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:54.790802+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":778,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T19:06:19.390443+0000","last_change":"2024-09-20T19:06:16.897724+0000","last_active":"2024-09-20T19:06:19.390443+0000","last_peered":"2024-09-20T19:06:19.390443+0000","last_clean":"2024-09-20T19:06:19.390443+0000","last_became_active":"2024-09-20T18:49:45.940544+0000","last_became_peered":"2024-09-20T18:49:45.940544+0000","last_unstale":"2024-09-20T19:06:19.390443+0000","last_undegraded":"2024-09-20T19:06:19.390443+0000","last_fullsized":"2024-09-20T19:06:19.390443+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:16.897677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:06:16.897677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:27.308342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":111,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:20.172723+0000","last_change":"2024-09-20T19:04:20.889280+0000","last_active":"2024-09-20T19:06:20.172723+0000","last_peered":"2024-09-20T19:06:20.172723+0000","last_clean":"2024-09-20T19:06:20.172723+0000","last_became_active":"2024-09-20T19:04:20.888422+0000","last_became_peered":"2024-09-20T19:04:20.888422+0000","last_unstale":"2024-09-20T19:06:20.172723+0000","last_undegraded":"2024-09-20T19:06:20.172723+0000","last_fullsized":"2024-09-20T19:06:20.172723+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.229201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.229201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:33.231712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":119,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:20.172939+0000","last_change":"2024-09-20T19:05:47.121946+0000","last_active":"2024-09-20T19:06:20.172939+0000","last_peered":"2024-09-20T19:06:20.172939+0000","last_clean":"2024-09-20T19:06:20.172939+0000","last_became_active":"2024-09-20T19:04:20.889418+0000","last_became_peered":"2024-09-20T19:04:20.889418+0000","last_unstale":"2024-09-20T19:06:20.172939+0000","last_undegraded":"2024-09-20T19:06:20.172939+0000","last_fullsized":"2024-09-20T19:06:20.172939+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:47.121842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:05:47.121842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:15.049722+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":16,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:20.173281+0000","last_change":"2024-09-20T19:06:18.054020+0000","last_active":"2024-09-20T19:06:20.173281+0000","last_peered":"2024-09-20T19:06:20.173281+0000","last_clean":"2024-09-20T19:06:20.173281+0000","last_became_active":"2024-09-20T19:06:18.053343+0000","last_became_peered":"2024-09-20T19:06:18.053343+0000","last_unstale":"2024-09-20T19:06:20.173281+0000","last_undegraded":"2024-09-20T19:06:20.173281+0000","last_fullsized":"2024-09-20T19:06:20.173281+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:31.127025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"274'28","reported_seq":127,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:20.172697+0000","last_change":"2024-09-20T19:04:20.976682+0000","last_active":"2024-09-20T19:06:20.172697+0000","last_peered":"2024-09-20T19:06:20.172697+0000","last_clean":"2024-09-20T19:06:20.172697+0000","last_became_active":"2024-09-20T19:04:20.897444+0000","last_became_peered":"2024-09-20T19:04:20.897444+0000","last_unstale":"2024-09-20T19:06:20.172697+0000","last_undegraded":"2024-09-20T19:06:20.172697+0000","last_fullsized":"2024-09-20T19:06:20.172697+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'28","last_scrub_stamp":"2024-09-20T19:03:06.269623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:06.269623+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":41,"num_read_kb":34,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":817,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.400385+0000","last_change":"2024-09-20T19:05:47.629956+0000","last_active":"2024-09-20T19:06:21.400385+0000","last_peered":"2024-09-20T19:06:21.400385+0000","last_clean":"2024-09-20T19:06:21.400385+0000","last_became_active":"2024-09-20T19:04:20.906790+0000","last_became_peered":"2024-09-20T19:04:20.906790+0000","last_unstale":"2024-09-20T19:06:21.400385+0000","last_undegraded":"2024-09-20T19:06:21.400385+0000","last_fullsized":"2024-09-20T19:06:21.400385+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:05:47.629892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:47.629892+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:06.773486+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"375'13","reported_seq":1025,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T19:06:19.388213+0000","last_change":"2024-09-20T19:04:20.916872+0000","last_active":"2024-09-20T19:06:19.388213+0000","last_peered":"2024-09-20T19:06:19.388213+0000","last_clean":"2024-09-20T19:06:19.388213+0000","last_became_active":"2024-09-20T19:04:20.896424+0000","last_became_peered":"2024-09-20T19:04:20.896424+0000","last_unstale":"2024-09-20T19:06:19.388213+0000","last_undegraded":"2024-09-20T19:06:19.388213+0000","last_fullsized":"2024-09-20T19:06:19.388213+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:58:23.114371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:23.114371+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:38.642422+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":775,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.141218+0000","last_change":"2024-09-20T18:59:05.082117+0000","last_active":"2024-09-20T19:06:21.141218+0000","last_peered":"2024-09-20T19:06:21.141218+0000","last_clean":"2024-09-20T19:06:21.141218+0000","last_became_active":"2024-09-20T18:59:05.081814+0000","last_became_peered":"2024-09-20T18:59:05.081814+0000","last_unstale":"2024-09-20T19:06:21.141218+0000","last_undegraded":"2024-09-20T19:06:21.141218+0000","last_fullsized":"2024-09-20T19:06:21.141218+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:29.530001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:29.530001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:36.792476+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":19,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.141273+0000","last_change":"2024-09-20T19:06:18.053626+0000","last_active":"2024-09-20T19:06:21.141273+0000","last_peered":"2024-09-20T19:06:21.141273+0000","last_clean":"2024-09-20T19:06:21.141273+0000","last_became_active":"2024-09-20T19:06:18.053336+0000","last_became_peered":"2024-09-20T19:06:18.053336+0000","last_unstale":"2024-09-20T19:06:21.141273+0000","last_undegraded":"2024-09-20T19:06:21.141273+0000","last_fullsized":"2024-09-20T19:06:21.141273+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:26.656015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"274'20","reported_seq":626,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:20.059286+0000","last_change":"2024-09-20T19:05:48.741237+0000","last_active":"2024-09-20T19:06:20.059286+0000","last_peered":"2024-09-20T19:06:20.059286+0000","last_clean":"2024-09-20T19:06:20.059286+0000","last_became_active":"2024-09-20T19:04:49.419140+0000","last_became_peered":"2024-09-20T19:04:49.419140+0000","last_unstale":"2024-09-20T19:06:20.059286+0000","last_undegraded":"2024-09-20T19:06:20.059286+0000","last_fullsized":"2024-09-20T19:06:20.059286+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'20","last_scrub_stamp":"2024-09-20T19:05:48.741059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:48.741059+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:13.202074+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":804,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:20.059727+0000","last_change":"2024-09-20T19:05:56.820141+0000","last_active":"2024-09-20T19:06:20.059727+0000","last_peered":"2024-09-20T19:06:20.059727+0000","last_clean":"2024-09-20T19:06:20.059727+0000","last_became_active":"2024-09-20T18:49:45.942496+0000","last_became_peered":"2024-09-20T18:49:45.942496+0000","last_unstale":"2024-09-20T19:06:20.059727+0000","last_undegraded":"2024-09-20T19:06:20.059727+0000","last_fullsized":"2024-09-20T19:06:20.059727+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:56.820079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:56.820079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:16.442542+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"262'10","reported_seq":1005,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.399980+0000","last_change":"2024-09-20T19:02:04.994109+0000","last_active":"2024-09-20T19:06:21.399980+0000","last_peered":"2024-09-20T19:06:21.399980+0000","last_clean":"2024-09-20T19:06:21.399980+0000","last_became_active":"2024-09-20T18:59:05.080183+0000","last_became_peered":"2024-09-20T18:59:05.080183+0000","last_unstale":"2024-09-20T19:06:21.399980+0000","last_undegraded":"2024-09-20T19:06:21.399980+0000","last_fullsized":"2024-09-20T19:06:21.399980+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'10","last_scrub_stamp":"2024-09-20T19:02:04.994047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:04.994047+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":764,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.399981+0000","last_change":"2024-09-20T19:06:15.671414+0000","last_active":"2024-09-20T19:06:21.399981+0000","last_peered":"2024-09-20T19:06:21.399981+0000","last_clean":"2024-09-20T19:06:21.399981+0000","last_became_active":"2024-09-20T18:49:51.964780+0000","last_became_peered":"2024-09-20T18:49:51.964780+0000","last_unstale":"2024-09-20T19:06:21.399981+0000","last_undegraded":"2024-09-20T19:06:21.399981+0000","last_fullsized":"2024-09-20T19:06:21.399981+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:15.671352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:06:15.671352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:25.558790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":19,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.066226+0000","last_change":"2024-09-20T19:06:18.050185+0000","last_active":"2024-09-20T19:06:21.066226+0000","last_peered":"2024-09-20T19:06:21.066226+0000","last_clean":"2024-09-20T19:06:21.066226+0000","last_became_active":"2024-09-20T19:06:18.049949+0000","last_became_peered":"2024-09-20T19:06:18.049949+0000","last_unstale":"2024-09-20T19:06:21.066226+0000","last_undegraded":"2024-09-20T19:06:21.066226+0000","last_fullsized":"2024-09-20T19:06:21.066226+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:27.811048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,4],"acting":[1,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"274'29","reported_seq":830,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.400110+0000","last_change":"2024-09-20T19:05:43.631990+0000","last_active":"2024-09-20T19:06:21.400110+0000","last_peered":"2024-09-20T19:06:21.400110+0000","last_clean":"2024-09-20T19:06:21.400110+0000","last_became_active":"2024-09-20T18:49:48.497736+0000","last_became_peered":"2024-09-20T18:49:48.497736+0000","last_unstale":"2024-09-20T19:06:21.400110+0000","last_undegraded":"2024-09-20T19:06:21.400110+0000","last_fullsized":"2024-09-20T19:06:21.400110+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'29","last_scrub_stamp":"2024-09-20T19:05:43.631845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:43.631845+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:56.642378+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":782,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.066278+0000","last_change":"2024-09-20T19:05:47.837587+0000","last_active":"2024-09-20T19:06:21.066278+0000","last_peered":"2024-09-20T19:06:21.066278+0000","last_clean":"2024-09-20T19:06:21.066278+0000","last_became_active":"2024-09-20T18:49:45.942253+0000","last_became_peered":"2024-09-20T18:49:45.942253+0000","last_unstale":"2024-09-20T19:06:21.066278+0000","last_undegraded":"2024-09-20T19:06:21.066278+0000","last_fullsized":"2024-09-20T19:06:21.066278+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:47.837520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:47.837520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:15.032955+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":369,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.401116+0000","last_change":"2024-09-20T19:05:48.203244+0000","last_active":"2024-09-20T19:06:21.401116+0000","last_peered":"2024-09-20T19:06:21.401116+0000","last_clean":"2024-09-20T19:06:21.401116+0000","last_became_active":"2024-09-20T18:59:05.085636+0000","last_became_peered":"2024-09-20T18:59:05.085636+0000","last_unstale":"2024-09-20T19:06:21.401116+0000","last_undegraded":"2024-09-20T19:06:21.401116+0000","last_fullsized":"2024-09-20T19:06:21.401116+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:48.203182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:05:48.203182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:58.736758+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":792,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.065650+0000","last_change":"2024-09-20T19:02:23.886386+0000","last_active":"2024-09-20T19:06:21.065650+0000","last_peered":"2024-09-20T19:06:21.065650+0000","last_clean":"2024-09-20T19:06:21.065650+0000","last_became_active":"2024-09-20T18:59:05.085131+0000","last_became_peered":"2024-09-20T18:59:05.085131+0000","last_unstale":"2024-09-20T19:06:21.065650+0000","last_undegraded":"2024-09-20T19:06:21.065650+0000","last_fullsized":"2024-09-20T19:06:21.065650+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:23.886308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:23.886308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:38.961464+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":14,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T19:06:19.389813+0000","last_change":"2024-09-20T19:06:18.057411+0000","last_active":"2024-09-20T19:06:19.389813+0000","last_peered":"2024-09-20T19:06:19.389813+0000","last_clean":"2024-09-20T19:06:19.389813+0000","last_became_active":"2024-09-20T19:06:18.057129+0000","last_became_peered":"2024-09-20T19:06:18.057129+0000","last_unstale":"2024-09-20T19:06:19.389813+0000","last_undegraded":"2024-09-20T19:06:19.389813+0000","last_fullsized":"2024-09-20T19:06:19.389813+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:18.121262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"353'41","reported_seq":1089,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.066004+0000","last_change":"2024-09-20T19:05:48.855983+0000","last_active":"2024-09-20T19:06:21.066004+0000","last_peered":"2024-09-20T19:06:21.066004+0000","last_clean":"2024-09-20T19:06:21.066004+0000","last_became_active":"2024-09-20T19:03:16.976916+0000","last_became_peered":"2024-09-20T19:03:16.976916+0000","last_unstale":"2024-09-20T19:06:21.066004+0000","last_undegraded":"2024-09-20T19:06:21.066004+0000","last_fullsized":"2024-09-20T19:06:21.066004+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"353'41","last_scrub_stamp":"2024-09-20T19:05:48.855876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:48.855876+0000","objects_scrubbed":10,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:09.142116+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":119,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:20.173212+0000","last_change":"2024-09-20T19:06:15.168347+0000","last_active":"2024-09-20T19:06:20.173212+0000","last_peered":"2024-09-20T19:06:20.173212+0000","last_clean":"2024-09-20T19:06:20.173212+0000","last_became_active":"2024-09-20T19:04:20.899352+0000","last_became_peered":"2024-09-20T19:04:20.899352+0000","last_unstale":"2024-09-20T19:06:20.173212+0000","last_undegraded":"2024-09-20T19:06:20.173212+0000","last_fullsized":"2024-09-20T19:06:20.173212+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:15.168293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:06:15.168293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:21.325884+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":836,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:20.059408+0000","last_change":"2024-09-20T19:03:11.910684+0000","last_active":"2024-09-20T19:06:20.059408+0000","last_peered":"2024-09-20T19:06:20.059408+0000","last_clean":"2024-09-20T19:06:20.059408+0000","last_became_active":"2024-09-20T18:59:05.086786+0000","last_became_peered":"2024-09-20T18:59:05.086786+0000","last_unstale":"2024-09-20T19:06:20.059408+0000","last_undegraded":"2024-09-20T19:06:20.059408+0000","last_fullsized":"2024-09-20T19:06:20.059408+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-20T19:03:11.909394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:03:11.909394+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:12.871941+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0.0012781159999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":808,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.400179+0000","last_change":"2024-09-20T19:04:20.907534+0000","last_active":"2024-09-20T19:06:21.400179+0000","last_peered":"2024-09-20T19:06:21.400179+0000","last_clean":"2024-09-20T19:06:21.400179+0000","last_became_active":"2024-09-20T19:04:20.907038+0000","last_became_peered":"2024-09-20T19:04:20.907038+0000","last_unstale":"2024-09-20T19:06:21.400179+0000","last_undegraded":"2024-09-20T19:06:21.400179+0000","last_fullsized":"2024-09-20T19:06:21.400179+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:15.034340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:03:15.034340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":775,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T19:06:19.389940+0000","last_change":"2024-09-20T19:04:20.896616+0000","last_active":"2024-09-20T19:06:19.389940+0000","last_peered":"2024-09-20T19:06:19.389940+0000","last_clean":"2024-09-20T19:06:19.389940+0000","last_became_active":"2024-09-20T19:04:20.896281+0000","last_became_peered":"2024-09-20T19:04:20.896281+0000","last_unstale":"2024-09-20T19:06:19.389940+0000","last_undegraded":"2024-09-20T19:06:19.389940+0000","last_fullsized":"2024-09-20T19:06:19.389940+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:04.526683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:04.526683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:12.801724+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":14,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T19:06:19.388292+0000","last_change":"2024-09-20T19:06:18.054306+0000","last_active":"2024-09-20T19:06:19.388292+0000","last_peered":"2024-09-20T19:06:19.388292+0000","last_clean":"2024-09-20T19:06:19.388292+0000","last_became_active":"2024-09-20T19:06:18.054031+0000","last_became_peered":"2024-09-20T19:06:18.054031+0000","last_unstale":"2024-09-20T19:06:19.388292+0000","last_undegraded":"2024-09-20T19:06:19.388292+0000","last_fullsized":"2024-09-20T19:06:19.388292+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:34.385514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"273'17","reported_seq":827,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.066768+0000","last_change":"2024-09-20T19:05:43.974098+0000","last_active":"2024-09-20T19:06:21.066768+0000","last_peered":"2024-09-20T19:06:21.066768+0000","last_clean":"2024-09-20T19:06:21.066768+0000","last_became_active":"2024-09-20T18:49:47.950904+0000","last_became_peered":"2024-09-20T18:49:47.950904+0000","last_unstale":"2024-09-20T19:06:21.066768+0000","last_undegraded":"2024-09-20T19:06:21.066768+0000","last_fullsized":"2024-09-20T19:06:21.066768+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'17","last_scrub_stamp":"2024-09-20T19:05:43.974013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:43.974013+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:01.950625+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":36,"num_read_kb":25,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":805,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.065588+0000","last_change":"2024-09-20T19:02:29.840761+0000","last_active":"2024-09-20T19:06:21.065588+0000","last_peered":"2024-09-20T19:06:21.065588+0000","last_clean":"2024-09-20T19:06:21.065588+0000","last_became_active":"2024-09-20T18:59:05.079789+0000","last_became_peered":"2024-09-20T18:59:05.079789+0000","last_unstale":"2024-09-20T19:06:21.065588+0000","last_undegraded":"2024-09-20T19:06:21.065588+0000","last_fullsized":"2024-09-20T19:06:21.065588+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:29.840601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:29.840601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:33.076198+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":795,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.140707+0000","last_change":"2024-09-20T19:06:13.599431+0000","last_active":"2024-09-20T19:06:21.140707+0000","last_peered":"2024-09-20T19:06:21.140707+0000","last_clean":"2024-09-20T19:06:21.140707+0000","last_became_active":"2024-09-20T18:49:44.937135+0000","last_became_peered":"2024-09-20T18:49:44.937135+0000","last_unstale":"2024-09-20T19:06:21.140707+0000","last_undegraded":"2024-09-20T19:06:21.140707+0000","last_fullsized":"2024-09-20T19:06:21.140707+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:06:13.599376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:06:13.599376+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:27.051567+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00095558800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":761,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.141005+0000","last_change":"2024-09-20T19:01:17.747452+0000","last_active":"2024-09-20T19:06:21.141005+0000","last_peered":"2024-09-20T19:06:21.141005+0000","last_clean":"2024-09-20T19:06:21.141005+0000","last_became_active":"2024-09-20T18:49:49.954424+0000","last_became_peered":"2024-09-20T18:49:49.954424+0000","last_unstale":"2024-09-20T19:06:21.141005+0000","last_undegraded":"2024-09-20T19:06:21.141005+0000","last_fullsized":"2024-09-20T19:06:21.141005+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:17.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:17.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:58.515630+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":770,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.141583+0000","last_change":"2024-09-20T19:01:15.203591+0000","last_active":"2024-09-20T19:06:21.141583+0000","last_peered":"2024-09-20T19:06:21.141583+0000","last_clean":"2024-09-20T19:06:21.141583+0000","last_became_active":"2024-09-20T18:59:05.081459+0000","last_became_peered":"2024-09-20T18:59:05.081459+0000","last_unstale":"2024-09-20T19:06:21.141583+0000","last_undegraded":"2024-09-20T19:06:21.141583+0000","last_fullsized":"2024-09-20T19:06:21.141583+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:15.203529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:15.203529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:37.777620+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":14,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T19:06:19.389938+0000","last_change":"2024-09-20T19:06:18.055443+0000","last_active":"2024-09-20T19:06:19.389938+0000","last_peered":"2024-09-20T19:06:19.389938+0000","last_clean":"2024-09-20T19:06:19.389938+0000","last_became_active":"2024-09-20T19:06:18.055137+0000","last_became_peered":"2024-09-20T19:06:18.055137+0000","last_unstale":"2024-09-20T19:06:19.389938+0000","last_undegraded":"2024-09-20T19:06:19.389938+0000","last_fullsized":"2024-09-20T19:06:19.389938+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:21.014567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"273'24","reported_seq":823,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.400274+0000","last_change":"2024-09-20T18:59:06.022021+0000","last_active":"2024-09-20T19:06:21.400274+0000","last_peered":"2024-09-20T19:06:21.400274+0000","last_clean":"2024-09-20T19:06:21.400274+0000","last_became_active":"2024-09-20T18:49:47.949230+0000","last_became_peered":"2024-09-20T18:49:47.949230+0000","last_unstale":"2024-09-20T19:06:21.400274+0000","last_undegraded":"2024-09-20T19:06:21.400274+0000","last_fullsized":"2024-09-20T19:06:21.400274+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:59:06.021962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:59:06.021962+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:34.733375+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":808,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.399830+0000","last_change":"2024-09-20T19:04:20.907250+0000","last_active":"2024-09-20T19:06:21.399830+0000","last_peered":"2024-09-20T19:06:21.399830+0000","last_clean":"2024-09-20T19:06:21.399830+0000","last_became_active":"2024-09-20T19:04:20.906969+0000","last_became_peered":"2024-09-20T19:04:20.906969+0000","last_unstale":"2024-09-20T19:06:21.399830+0000","last_undegraded":"2024-09-20T19:06:21.399830+0000","last_fullsized":"2024-09-20T19:06:21.399830+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:24.984022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:24.984022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":1781,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.141757+0000","last_change":"2024-09-20T19:02:02.494344+0000","last_active":"2024-09-20T19:06:21.141757+0000","last_peered":"2024-09-20T19:06:21.141757+0000","last_clean":"2024-09-20T19:06:21.141757+0000","last_became_active":"2024-09-20T18:49:44.936819+0000","last_became_peered":"2024-09-20T18:49:44.936819+0000","last_unstale":"2024-09-20T19:06:21.141757+0000","last_undegraded":"2024-09-20T19:06:21.141757+0000","last_fullsized":"2024-09-20T19:06:21.141757+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-20T19:02:02.494291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:02:02.494291+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:39.801016+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0010996199999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":988,"num_read_kb":983,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":788,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:20.059845+0000","last_change":"2024-09-20T19:05:51.743485+0000","last_active":"2024-09-20T19:06:20.059845+0000","last_peered":"2024-09-20T19:06:20.059845+0000","last_clean":"2024-09-20T19:06:20.059845+0000","last_became_active":"2024-09-20T18:49:49.961066+0000","last_became_peered":"2024-09-20T18:49:49.961066+0000","last_unstale":"2024-09-20T19:06:20.059845+0000","last_undegraded":"2024-09-20T19:06:20.059845+0000","last_fullsized":"2024-09-20T19:06:20.059845+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:51.743363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:05:51.743363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:06.075322+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":778,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.066642+0000","last_change":"2024-09-20T19:02:26.861253+0000","last_active":"2024-09-20T19:06:21.066642+0000","last_peered":"2024-09-20T19:06:21.066642+0000","last_clean":"2024-09-20T19:06:21.066642+0000","last_became_active":"2024-09-20T18:49:51.970343+0000","last_became_peered":"2024-09-20T18:49:51.970343+0000","last_unstale":"2024-09-20T19:06:21.066642+0000","last_undegraded":"2024-09-20T19:06:21.066642+0000","last_fullsized":"2024-09-20T19:06:21.066642+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:26.861185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:26.861185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:32.099095+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":14,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T19:06:19.389590+0000","last_change":"2024-09-20T19:06:18.055447+0000","last_active":"2024-09-20T19:06:19.389590+0000","last_peered":"2024-09-20T19:06:19.389590+0000","last_clean":"2024-09-20T19:06:19.389590+0000","last_became_active":"2024-09-20T19:06:18.055189+0000","last_became_peered":"2024-09-20T19:06:18.055189+0000","last_unstale":"2024-09-20T19:06:19.389590+0000","last_undegraded":"2024-09-20T19:06:19.389590+0000","last_fullsized":"2024-09-20T19:06:19.389590+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:35.116345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"375'34","reported_seq":548,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.401005+0000","last_change":"2024-09-20T19:05:54.115428+0000","last_active":"2024-09-20T19:06:21.401005+0000","last_peered":"2024-09-20T19:06:21.401005+0000","last_clean":"2024-09-20T19:06:21.401005+0000","last_became_active":"2024-09-20T19:04:20.928727+0000","last_became_peered":"2024-09-20T19:04:20.928727+0000","last_unstale":"2024-09-20T19:06:21.401005+0000","last_undegraded":"2024-09-20T19:06:21.401005+0000","last_fullsized":"2024-09-20T19:06:21.401005+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"375'34","last_scrub_stamp":"2024-09-20T19:05:54.115376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:54.115376+0000","objects_scrubbed":8,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:59.905707+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":462,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":800,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T19:06:19.389571+0000","last_change":"2024-09-20T19:05:07.665281+0000","last_active":"2024-09-20T19:06:19.389571+0000","last_peered":"2024-09-20T19:06:19.389571+0000","last_clean":"2024-09-20T19:06:19.389571+0000","last_became_active":"2024-09-20T18:59:05.079111+0000","last_became_peered":"2024-09-20T18:59:05.079111+0000","last_unstale":"2024-09-20T19:06:19.389571+0000","last_undegraded":"2024-09-20T19:06:19.389571+0000","last_fullsized":"2024-09-20T19:06:19.389571+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:07.665167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:07.665167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":850,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.141861+0000","last_change":"2024-09-20T19:03:14.368338+0000","last_active":"2024-09-20T19:06:21.141861+0000","last_peered":"2024-09-20T19:06:21.141861+0000","last_clean":"2024-09-20T19:06:21.141861+0000","last_became_active":"2024-09-20T18:59:05.086103+0000","last_became_peered":"2024-09-20T18:59:05.086103+0000","last_unstale":"2024-09-20T19:06:21.141861+0000","last_undegraded":"2024-09-20T19:06:21.141861+0000","last_fullsized":"2024-09-20T19:06:21.141861+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:03:14.368279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:03:14.368279+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:09.743853+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":765,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.141638+0000","last_change":"2024-09-20T19:04:35.533709+0000","last_active":"2024-09-20T19:06:21.141638+0000","last_peered":"2024-09-20T19:06:21.141638+0000","last_clean":"2024-09-20T19:06:21.141638+0000","last_became_active":"2024-09-20T18:49:49.956128+0000","last_became_peered":"2024-09-20T18:49:49.956128+0000","last_unstale":"2024-09-20T19:06:21.141638+0000","last_undegraded":"2024-09-20T19:06:21.141638+0000","last_fullsized":"2024-09-20T19:06:21.141638+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:35.533633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:35.533633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":801,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:20.059771+0000","last_change":"2024-09-20T19:04:20.899776+0000","last_active":"2024-09-20T19:06:20.059771+0000","last_peered":"2024-09-20T19:06:20.059771+0000","last_clean":"2024-09-20T19:06:20.059771+0000","last_became_active":"2024-09-20T19:04:20.899321+0000","last_became_peered":"2024-09-20T19:04:20.899321+0000","last_unstale":"2024-09-20T19:06:20.059771+0000","last_undegraded":"2024-09-20T19:06:20.059771+0000","last_fullsized":"2024-09-20T19:06:20.059771+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:30.557368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:30.557368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:39.737278+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":19,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.400075+0000","last_change":"2024-09-20T19:06:18.056631+0000","last_active":"2024-09-20T19:06:21.400075+0000","last_peered":"2024-09-20T19:06:21.400075+0000","last_clean":"2024-09-20T19:06:21.400075+0000","last_became_active":"2024-09-20T19:06:18.056332+0000","last_became_peered":"2024-09-20T19:06:18.056332+0000","last_unstale":"2024-09-20T19:06:21.400075+0000","last_undegraded":"2024-09-20T19:06:21.400075+0000","last_fullsized":"2024-09-20T19:06:21.400075+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:30.224100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"274'19","reported_seq":131,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:20.173665+0000","last_change":"2024-09-20T19:05:50.066533+0000","last_active":"2024-09-20T19:06:20.173665+0000","last_peered":"2024-09-20T19:06:20.173665+0000","last_clean":"2024-09-20T19:06:20.173665+0000","last_became_active":"2024-09-20T19:04:20.897175+0000","last_became_peered":"2024-09-20T19:04:20.897175+0000","last_unstale":"2024-09-20T19:06:20.173665+0000","last_undegraded":"2024-09-20T19:06:20.173665+0000","last_fullsized":"2024-09-20T19:06:20.173665+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'19","last_scrub_stamp":"2024-09-20T19:05:50.066359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:50.066359+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:56.712506+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":119,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:20.172784+0000","last_change":"2024-09-20T19:05:56.027187+0000","last_active":"2024-09-20T19:06:20.172784+0000","last_peered":"2024-09-20T19:06:20.172784+0000","last_clean":"2024-09-20T19:06:20.172784+0000","last_became_active":"2024-09-20T19:04:20.887102+0000","last_became_peered":"2024-09-20T19:04:20.887102+0000","last_unstale":"2024-09-20T19:06:20.172784+0000","last_undegraded":"2024-09-20T19:06:20.172784+0000","last_fullsized":"2024-09-20T19:06:20.172784+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:56.027123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:56.027123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:15.919213+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":819,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:20.059669+0000","last_change":"2024-09-20T19:04:20.899830+0000","last_active":"2024-09-20T19:06:20.059669+0000","last_peered":"2024-09-20T19:06:20.059669+0000","last_clean":"2024-09-20T19:06:20.059669+0000","last_became_active":"2024-09-20T19:04:20.899331+0000","last_became_peered":"2024-09-20T19:04:20.899331+0000","last_unstale":"2024-09-20T19:06:20.059669+0000","last_undegraded":"2024-09-20T19:06:20.059669+0000","last_fullsized":"2024-09-20T19:06:20.059669+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:38.463404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:38.463404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:26.726228+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":780,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T19:06:19.388717+0000","last_change":"2024-09-20T19:04:35.529672+0000","last_active":"2024-09-20T19:06:19.388717+0000","last_peered":"2024-09-20T19:06:19.388717+0000","last_clean":"2024-09-20T19:06:19.388717+0000","last_became_active":"2024-09-20T18:59:05.079897+0000","last_became_peered":"2024-09-20T18:59:05.079897+0000","last_unstale":"2024-09-20T19:06:19.388717+0000","last_undegraded":"2024-09-20T19:06:19.388717+0000","last_fullsized":"2024-09-20T19:06:19.388717+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:35.529584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:04:35.529584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":19,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.140976+0000","last_change":"2024-09-20T19:06:18.054677+0000","last_active":"2024-09-20T19:06:21.140976+0000","last_peered":"2024-09-20T19:06:21.140976+0000","last_clean":"2024-09-20T19:06:21.140976+0000","last_became_active":"2024-09-20T19:06:18.054421+0000","last_became_peered":"2024-09-20T19:06:18.054421+0000","last_unstale":"2024-09-20T19:06:21.140976+0000","last_undegraded":"2024-09-20T19:06:21.140976+0000","last_fullsized":"2024-09-20T19:06:21.140976+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:26.682986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"274'31","reported_seq":129,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:20.173879+0000","last_change":"2024-09-20T19:04:20.949305+0000","last_active":"2024-09-20T19:06:20.173879+0000","last_peered":"2024-09-20T19:06:20.173879+0000","last_clean":"2024-09-20T19:06:20.173879+0000","last_became_active":"2024-09-20T19:04:20.883306+0000","last_became_peered":"2024-09-20T19:04:20.883306+0000","last_unstale":"2024-09-20T19:06:20.173879+0000","last_undegraded":"2024-09-20T19:06:20.173879+0000","last_fullsized":"2024-09-20T19:06:20.173879+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'31","last_scrub_stamp":"2024-09-20T19:03:12.196851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:12.196851+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":49,"num_read_kb":39,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":378,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.401261+0000","last_change":"2024-09-20T19:04:20.899712+0000","last_active":"2024-09-20T19:06:21.401261+0000","last_peered":"2024-09-20T19:06:21.401261+0000","last_clean":"2024-09-20T19:06:21.401261+0000","last_became_active":"2024-09-20T19:04:20.899415+0000","last_became_peered":"2024-09-20T19:04:20.899415+0000","last_unstale":"2024-09-20T19:06:21.401261+0000","last_undegraded":"2024-09-20T19:06:21.401261+0000","last_fullsized":"2024-09-20T19:06:21.401261+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:42.479489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:42.479489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:36.220255+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":807,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.065790+0000","last_change":"2024-09-20T19:05:46.866958+0000","last_active":"2024-09-20T19:06:21.065790+0000","last_peered":"2024-09-20T19:06:21.065790+0000","last_clean":"2024-09-20T19:06:21.065790+0000","last_became_active":"2024-09-20T19:04:20.891772+0000","last_became_peered":"2024-09-20T19:04:20.891772+0000","last_unstale":"2024-09-20T19:06:21.065790+0000","last_undegraded":"2024-09-20T19:06:21.065790+0000","last_fullsized":"2024-09-20T19:06:21.065790+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:46.866777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:05:46.866777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:55.257561+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":798,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.141214+0000","last_change":"2024-09-20T19:04:20.897937+0000","last_active":"2024-09-20T19:06:21.141214+0000","last_peered":"2024-09-20T19:06:21.141214+0000","last_clean":"2024-09-20T19:06:21.141214+0000","last_became_active":"2024-09-20T19:04:20.897569+0000","last_became_peered":"2024-09-20T19:04:20.897569+0000","last_unstale":"2024-09-20T19:06:21.141214+0000","last_undegraded":"2024-09-20T19:06:21.141214+0000","last_fullsized":"2024-09-20T19:06:21.141214+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:31.687188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:31.687188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:24.424476+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":19,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.400513+0000","last_change":"2024-09-20T19:06:18.056510+0000","last_active":"2024-09-20T19:06:21.400513+0000","last_peered":"2024-09-20T19:06:21.400513+0000","last_clean":"2024-09-20T19:06:21.400513+0000","last_became_active":"2024-09-20T19:06:18.056187+0000","last_became_peered":"2024-09-20T19:06:18.056187+0000","last_unstale":"2024-09-20T19:06:21.400513+0000","last_undegraded":"2024-09-20T19:06:21.400513+0000","last_fullsized":"2024-09-20T19:06:21.400513+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:27.399713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1],"acting":[6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"273'20","reported_seq":123,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:20.173390+0000","last_change":"2024-09-20T19:04:20.945143+0000","last_active":"2024-09-20T19:06:20.173390+0000","last_peered":"2024-09-20T19:06:20.173390+0000","last_clean":"2024-09-20T19:06:20.173390+0000","last_became_active":"2024-09-20T19:04:20.896483+0000","last_became_peered":"2024-09-20T19:04:20.896483+0000","last_unstale":"2024-09-20T19:06:20.173390+0000","last_undegraded":"2024-09-20T19:06:20.173390+0000","last_fullsized":"2024-09-20T19:06:20.173390+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'20","last_scrub_stamp":"2024-09-20T19:03:04.282800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:04.282800+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":811,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T19:06:19.390030+0000","last_change":"2024-09-20T19:05:48.678438+0000","last_active":"2024-09-20T19:06:19.390030+0000","last_peered":"2024-09-20T19:06:19.390030+0000","last_clean":"2024-09-20T19:06:19.390030+0000","last_became_active":"2024-09-20T19:04:20.892951+0000","last_became_peered":"2024-09-20T19:04:20.892951+0000","last_unstale":"2024-09-20T19:06:19.390030+0000","last_undegraded":"2024-09-20T19:06:19.390030+0000","last_fullsized":"2024-09-20T19:06:19.390030+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:48.678361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:48.678361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:02.488299+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":1001,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.141118+0000","last_change":"2024-09-20T19:05:08.406307+0000","last_active":"2024-09-20T19:06:21.141118+0000","last_peered":"2024-09-20T19:06:21.141118+0000","last_clean":"2024-09-20T19:06:21.141118+0000","last_became_active":"2024-09-20T18:49:49.957005+0000","last_became_peered":"2024-09-20T18:49:49.957005+0000","last_unstale":"2024-09-20T19:06:21.141118+0000","last_undegraded":"2024-09-20T19:06:21.141118+0000","last_fullsized":"2024-09-20T19:06:21.141118+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T19:05:08.406231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:05:08.406231+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":111,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:20.173685+0000","last_change":"2024-09-20T19:04:20.885294+0000","last_active":"2024-09-20T19:06:20.173685+0000","last_peered":"2024-09-20T19:06:20.173685+0000","last_clean":"2024-09-20T19:06:20.173685+0000","last_became_active":"2024-09-20T19:04:20.884994+0000","last_became_peered":"2024-09-20T19:04:20.884994+0000","last_unstale":"2024-09-20T19:06:20.173685+0000","last_undegraded":"2024-09-20T19:06:20.173685+0000","last_fullsized":"2024-09-20T19:06:20.173685+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:11.207994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:03:11.207994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":16,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.141385+0000","last_change":"2024-09-20T19:06:18.055511+0000","last_active":"2024-09-20T19:06:21.141385+0000","last_peered":"2024-09-20T19:06:21.141385+0000","last_clean":"2024-09-20T19:06:21.141385+0000","last_became_active":"2024-09-20T19:06:18.055241+0000","last_became_peered":"2024-09-20T19:06:18.055241+0000","last_unstale":"2024-09-20T19:06:21.141385+0000","last_undegraded":"2024-09-20T19:06:21.141385+0000","last_fullsized":"2024-09-20T19:06:21.141385+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:26.463118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"274'15","reported_seq":857,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.400026+0000","last_change":"2024-09-20T19:05:57.548758+0000","last_active":"2024-09-20T19:06:21.400026+0000","last_peered":"2024-09-20T19:06:21.400026+0000","last_clean":"2024-09-20T19:06:21.400026+0000","last_became_active":"2024-09-20T19:04:20.906834+0000","last_became_peered":"2024-09-20T19:04:20.906834+0000","last_unstale":"2024-09-20T19:06:21.400026+0000","last_undegraded":"2024-09-20T19:06:21.400026+0000","last_fullsized":"2024-09-20T19:06:21.400026+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'15","last_scrub_stamp":"2024-09-20T19:05:57.548683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:57.548683+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:13.922548+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":342,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.140753+0000","last_change":"2024-09-20T19:05:52.464362+0000","last_active":"2024-09-20T19:06:21.140753+0000","last_peered":"2024-09-20T19:06:21.140753+0000","last_clean":"2024-09-20T19:06:21.140753+0000","last_became_active":"2024-09-20T19:00:01.190875+0000","last_became_peered":"2024-09-20T19:00:01.190875+0000","last_unstale":"2024-09-20T19:06:21.140753+0000","last_undegraded":"2024-09-20T19:06:21.140753+0000","last_fullsized":"2024-09-20T19:06:21.140753+0000","mapping_epoch":277,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":278,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:52.464261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:52.464261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:59.446086+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":810,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.140693+0000","last_change":"2024-09-20T19:04:20.889614+0000","last_active":"2024-09-20T19:06:21.140693+0000","last_peered":"2024-09-20T19:06:21.140693+0000","last_clean":"2024-09-20T19:06:21.140693+0000","last_became_active":"2024-09-20T19:04:20.889273+0000","last_became_peered":"2024-09-20T19:04:20.889273+0000","last_unstale":"2024-09-20T19:06:21.140693+0000","last_undegraded":"2024-09-20T19:06:21.140693+0000","last_fullsized":"2024-09-20T19:06:21.140693+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:03.389611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:03.389611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:26.422333+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":771,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.399774+0000","last_change":"2024-09-20T19:02:31.120809+0000","last_active":"2024-09-20T19:06:21.399774+0000","last_peered":"2024-09-20T19:06:21.399774+0000","last_clean":"2024-09-20T19:06:21.399774+0000","last_became_active":"2024-09-20T18:49:51.964098+0000","last_became_peered":"2024-09-20T18:49:51.964098+0000","last_unstale":"2024-09-20T19:06:21.399774+0000","last_undegraded":"2024-09-20T19:06:21.399774+0000","last_fullsized":"2024-09-20T19:06:21.399774+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:31.120623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:31.120623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":16,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.141980+0000","last_change":"2024-09-20T19:06:18.053945+0000","last_active":"2024-09-20T19:06:21.141980+0000","last_peered":"2024-09-20T19:06:21.141980+0000","last_clean":"2024-09-20T19:06:21.141980+0000","last_became_active":"2024-09-20T19:06:18.053641+0000","last_became_peered":"2024-09-20T19:06:18.053641+0000","last_unstale":"2024-09-20T19:06:21.141980+0000","last_undegraded":"2024-09-20T19:06:21.141980+0000","last_fullsized":"2024-09-20T19:06:21.141980+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:37.951457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"274'35","reported_seq":413,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.401008+0000","last_change":"2024-09-20T19:06:19.915841+0000","last_active":"2024-09-20T19:06:21.401008+0000","last_peered":"2024-09-20T19:06:21.401008+0000","last_clean":"2024-09-20T19:06:21.401008+0000","last_became_active":"2024-09-20T18:59:05.087391+0000","last_became_peered":"2024-09-20T18:59:05.087391+0000","last_unstale":"2024-09-20T19:06:21.401008+0000","last_undegraded":"2024-09-20T19:06:21.401008+0000","last_fullsized":"2024-09-20T19:06:21.401008+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'35","last_scrub_stamp":"2024-09-20T19:06:19.915767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:06:19.915767+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:46.153655+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":803,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.141812+0000","last_change":"2024-09-20T19:05:49.763478+0000","last_active":"2024-09-20T19:06:21.141812+0000","last_peered":"2024-09-20T19:06:21.141812+0000","last_clean":"2024-09-20T19:06:21.141812+0000","last_became_active":"2024-09-20T18:59:05.086670+0000","last_became_peered":"2024-09-20T18:59:05.086670+0000","last_unstale":"2024-09-20T19:06:21.141812+0000","last_undegraded":"2024-09-20T19:06:21.141812+0000","last_fullsized":"2024-09-20T19:06:21.141812+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:49.763407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:49.763407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:09.527897+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":793,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.140726+0000","last_change":"2024-09-20T19:05:11.444197+0000","last_active":"2024-09-20T19:06:21.140726+0000","last_peered":"2024-09-20T19:06:21.140726+0000","last_clean":"2024-09-20T19:06:21.140726+0000","last_became_active":"2024-09-20T18:49:49.957558+0000","last_became_peered":"2024-09-20T18:49:49.957558+0000","last_unstale":"2024-09-20T19:06:21.140726+0000","last_undegraded":"2024-09-20T19:06:21.140726+0000","last_fullsized":"2024-09-20T19:06:21.140726+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:11.444143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:05:11.444143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:23.300683+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":761,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.141785+0000","last_change":"2024-09-20T18:53:47.370152+0000","last_active":"2024-09-20T19:06:21.141785+0000","last_peered":"2024-09-20T19:06:21.141785+0000","last_clean":"2024-09-20T19:06:21.141785+0000","last_became_active":"2024-09-20T18:49:51.970798+0000","last_became_peered":"2024-09-20T18:49:51.970798+0000","last_unstale":"2024-09-20T19:06:21.141785+0000","last_undegraded":"2024-09-20T19:06:21.141785+0000","last_fullsized":"2024-09-20T19:06:21.141785+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:47.370108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:47.370108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:30.728805+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":19,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.400153+0000","last_change":"2024-09-20T19:06:18.055118+0000","last_active":"2024-09-20T19:06:21.400153+0000","last_peered":"2024-09-20T19:06:21.400153+0000","last_clean":"2024-09-20T19:06:21.400153+0000","last_became_active":"2024-09-20T19:06:18.054843+0000","last_became_peered":"2024-09-20T19:06:18.054843+0000","last_unstale":"2024-09-20T19:06:21.400153+0000","last_undegraded":"2024-09-20T19:06:21.400153+0000","last_fullsized":"2024-09-20T19:06:21.400153+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:21.237812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"273'18","reported_seq":123,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:20.172848+0000","last_change":"2024-09-20T19:04:20.936465+0000","last_active":"2024-09-20T19:06:20.172848+0000","last_peered":"2024-09-20T19:06:20.172848+0000","last_clean":"2024-09-20T19:06:20.172848+0000","last_became_active":"2024-09-20T19:04:20.888905+0000","last_became_peered":"2024-09-20T19:04:20.888905+0000","last_unstale":"2024-09-20T19:06:20.172848+0000","last_undegraded":"2024-09-20T19:06:20.172848+0000","last_fullsized":"2024-09-20T19:06:20.172848+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'18","last_scrub_stamp":"2024-09-20T19:03:13.201592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:13.201592+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":822,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.140531+0000","last_change":"2024-09-20T19:06:15.692558+0000","last_active":"2024-09-20T19:06:21.140531+0000","last_peered":"2024-09-20T19:06:21.140531+0000","last_clean":"2024-09-20T19:06:21.140531+0000","last_became_active":"2024-09-20T19:04:20.892429+0000","last_became_peered":"2024-09-20T19:04:20.892429+0000","last_unstale":"2024-09-20T19:06:21.140531+0000","last_undegraded":"2024-09-20T19:06:21.140531+0000","last_fullsized":"2024-09-20T19:06:21.140531+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:15.692508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:06:15.692508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:27.149889+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":785,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.140540+0000","last_change":"2024-09-20T19:04:48.406445+0000","last_active":"2024-09-20T19:06:21.140540+0000","last_peered":"2024-09-20T19:06:21.140540+0000","last_clean":"2024-09-20T19:06:21.140540+0000","last_became_active":"2024-09-20T18:49:49.954944+0000","last_became_peered":"2024-09-20T18:49:49.954944+0000","last_unstale":"2024-09-20T19:06:21.140540+0000","last_undegraded":"2024-09-20T19:06:21.140540+0000","last_fullsized":"2024-09-20T19:06:21.140540+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:48.406373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:48.406373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:59.483627+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":774,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T19:06:19.389680+0000","last_change":"2024-09-20T19:05:43.657860+0000","last_active":"2024-09-20T19:06:19.389680+0000","last_peered":"2024-09-20T19:06:19.389680+0000","last_clean":"2024-09-20T19:06:19.389680+0000","last_became_active":"2024-09-20T18:49:51.972970+0000","last_became_peered":"2024-09-20T18:49:51.972970+0000","last_unstale":"2024-09-20T19:06:19.389680+0000","last_undegraded":"2024-09-20T19:06:19.389680+0000","last_fullsized":"2024-09-20T19:06:19.389680+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:43.657780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:05:43.657780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:05.178691+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":16,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:20.059602+0000","last_change":"2024-09-20T19:06:18.053735+0000","last_active":"2024-09-20T19:06:20.059602+0000","last_peered":"2024-09-20T19:06:20.059602+0000","last_clean":"2024-09-20T19:06:20.059602+0000","last_became_active":"2024-09-20T19:06:18.053317+0000","last_became_peered":"2024-09-20T19:06:18.053317+0000","last_unstale":"2024-09-20T19:06:20.059602+0000","last_undegraded":"2024-09-20T19:06:20.059602+0000","last_fullsized":"2024-09-20T19:06:20.059602+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:24.608030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"273'27","reported_seq":881,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:20.059077+0000","last_change":"2024-09-20T19:05:49.777488+0000","last_active":"2024-09-20T19:06:20.059077+0000","last_peered":"2024-09-20T19:06:20.059077+0000","last_clean":"2024-09-20T19:06:20.059077+0000","last_became_active":"2024-09-20T18:59:05.083726+0000","last_became_peered":"2024-09-20T18:59:05.083726+0000","last_unstale":"2024-09-20T19:06:20.059077+0000","last_undegraded":"2024-09-20T19:06:20.059077+0000","last_fullsized":"2024-09-20T19:06:20.059077+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'27","last_scrub_stamp":"2024-09-20T19:05:49.777414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:49.777414+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:12.438786+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":846,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.140794+0000","last_change":"2024-09-20T19:04:46.691255+0000","last_active":"2024-09-20T19:06:21.140794+0000","last_peered":"2024-09-20T19:06:21.140794+0000","last_clean":"2024-09-20T19:06:21.140794+0000","last_became_active":"2024-09-20T19:04:20.884854+0000","last_became_peered":"2024-09-20T19:04:20.884854+0000","last_unstale":"2024-09-20T19:06:21.140794+0000","last_undegraded":"2024-09-20T19:06:21.140794+0000","last_fullsized":"2024-09-20T19:06:21.140794+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-20T19:04:46.691171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:04:46.691171+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"375'14","reported_seq":1038,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.141100+0000","last_change":"2024-09-20T19:06:14.693176+0000","last_active":"2024-09-20T19:06:21.141100+0000","last_peered":"2024-09-20T19:06:21.141100+0000","last_clean":"2024-09-20T19:06:21.141100+0000","last_became_active":"2024-09-20T19:04:20.892956+0000","last_became_peered":"2024-09-20T19:04:20.892956+0000","last_unstale":"2024-09-20T19:06:21.141100+0000","last_undegraded":"2024-09-20T19:06:21.141100+0000","last_fullsized":"2024-09-20T19:06:21.141100+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"375'14","last_scrub_stamp":"2024-09-20T19:06:14.693111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:06:14.693111+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:44.526972+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":790,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:20.059089+0000","last_change":"2024-09-20T19:03:06.888620+0000","last_active":"2024-09-20T19:06:20.059089+0000","last_peered":"2024-09-20T19:06:20.059089+0000","last_clean":"2024-09-20T19:06:20.059089+0000","last_became_active":"2024-09-20T18:59:05.087005+0000","last_became_peered":"2024-09-20T18:59:05.087005+0000","last_unstale":"2024-09-20T19:06:20.059089+0000","last_undegraded":"2024-09-20T19:06:20.059089+0000","last_fullsized":"2024-09-20T19:06:20.059089+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:06.888563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:03:06.888563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:09.776326+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":353,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.400000+0000","last_change":"2024-09-20T19:01:16.542398+0000","last_active":"2024-09-20T19:06:21.400000+0000","last_peered":"2024-09-20T19:06:21.400000+0000","last_clean":"2024-09-20T19:06:21.400000+0000","last_became_active":"2024-09-20T18:59:05.085427+0000","last_became_peered":"2024-09-20T18:59:05.085427+0000","last_unstale":"2024-09-20T19:06:21.400000+0000","last_undegraded":"2024-09-20T19:06:21.400000+0000","last_fullsized":"2024-09-20T19:06:21.400000+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:16.542335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:16.542335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:29.408351+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"273'12","reported_seq":807,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T19:06:19.973831+0000","last_change":"2024-09-20T19:06:19.973831+0000","last_active":"2024-09-20T19:06:19.973831+0000","last_peered":"2024-09-20T19:06:19.973831+0000","last_clean":"2024-09-20T19:06:19.973831+0000","last_became_active":"2024-09-20T18:49:48.500662+0000","last_became_peered":"2024-09-20T18:49:48.500662+0000","last_unstale":"2024-09-20T19:06:19.973831+0000","last_undegraded":"2024-09-20T19:06:19.973831+0000","last_fullsized":"2024-09-20T19:06:19.973831+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'12","last_scrub_stamp":"2024-09-20T19:06:19.973765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:06:19.973765+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:34.482020+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":779,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:20.059636+0000","last_change":"2024-09-20T19:05:22.605885+0000","last_active":"2024-09-20T19:06:20.059636+0000","last_peered":"2024-09-20T19:06:20.059636+0000","last_clean":"2024-09-20T19:06:20.059636+0000","last_became_active":"2024-09-20T18:49:49.956749+0000","last_became_peered":"2024-09-20T18:49:49.956749+0000","last_unstale":"2024-09-20T19:06:20.059636+0000","last_undegraded":"2024-09-20T19:06:20.059636+0000","last_fullsized":"2024-09-20T19:06:20.059636+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:22.605839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:05:22.605839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:49.241766+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":815,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.140834+0000","last_change":"2024-09-20T19:05:56.761494+0000","last_active":"2024-09-20T19:06:21.140834+0000","last_peered":"2024-09-20T19:06:21.140834+0000","last_clean":"2024-09-20T19:06:21.140834+0000","last_became_active":"2024-09-20T19:04:20.914115+0000","last_became_peered":"2024-09-20T19:04:20.914115+0000","last_unstale":"2024-09-20T19:06:21.140834+0000","last_undegraded":"2024-09-20T19:06:21.140834+0000","last_fullsized":"2024-09-20T19:06:21.140834+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:56.761425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:05:56.761425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:15.725837+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"273'15","reported_seq":795,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.400316+0000","last_change":"2024-09-20T19:01:15.875779+0000","last_active":"2024-09-20T19:06:21.400316+0000","last_peered":"2024-09-20T19:06:21.400316+0000","last_clean":"2024-09-20T19:06:21.400316+0000","last_became_active":"2024-09-20T18:49:48.500803+0000","last_became_peered":"2024-09-20T18:49:48.500803+0000","last_unstale":"2024-09-20T19:06:21.400316+0000","last_undegraded":"2024-09-20T19:06:21.400316+0000","last_fullsized":"2024-09-20T19:06:21.400316+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:01:15.875693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:15.875693+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:58.535131+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":820,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.140388+0000","last_change":"2024-09-20T19:04:47.713764+0000","last_active":"2024-09-20T19:06:21.140388+0000","last_peered":"2024-09-20T19:06:21.140388+0000","last_clean":"2024-09-20T19:06:21.140388+0000","last_became_active":"2024-09-20T19:04:20.893250+0000","last_became_peered":"2024-09-20T19:04:20.893250+0000","last_unstale":"2024-09-20T19:06:21.140388+0000","last_undegraded":"2024-09-20T19:06:21.140388+0000","last_fullsized":"2024-09-20T19:06:21.140388+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:47.713585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:04:47.713585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:10.723942+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":777,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T19:06:19.390252+0000","last_change":"2024-09-20T19:04:50.573619+0000","last_active":"2024-09-20T19:06:19.390252+0000","last_peered":"2024-09-20T19:06:19.390252+0000","last_clean":"2024-09-20T19:06:19.390252+0000","last_became_active":"2024-09-20T18:49:49.961947+0000","last_became_peered":"2024-09-20T18:49:49.961947+0000","last_unstale":"2024-09-20T19:06:19.390252+0000","last_undegraded":"2024-09-20T19:06:19.390252+0000","last_fullsized":"2024-09-20T19:06:19.390252+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:50.573550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:50.573550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"66'1","reported_seq":823,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.140861+0000","last_change":"2024-09-20T19:04:20.913449+0000","last_active":"2024-09-20T19:06:21.140861+0000","last_peered":"2024-09-20T19:06:21.140861+0000","last_clean":"2024-09-20T19:06:21.140861+0000","last_became_active":"2024-09-20T19:04:20.892236+0000","last_became_peered":"2024-09-20T19:04:20.892236+0000","last_unstale":"2024-09-20T19:06:21.140861+0000","last_undegraded":"2024-09-20T19:06:21.140861+0000","last_fullsized":"2024-09-20T19:06:21.140861+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T19:03:12.499684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:03:12.499684+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:37.501578+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"274'15","reported_seq":792,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:20.059612+0000","last_change":"2024-09-20T19:01:36.532486+0000","last_active":"2024-09-20T19:06:20.059612+0000","last_peered":"2024-09-20T19:06:20.059612+0000","last_clean":"2024-09-20T19:06:20.059612+0000","last_became_active":"2024-09-20T18:49:48.501011+0000","last_became_peered":"2024-09-20T18:49:48.501011+0000","last_unstale":"2024-09-20T19:06:20.059612+0000","last_undegraded":"2024-09-20T19:06:20.059612+0000","last_fullsized":"2024-09-20T19:06:20.059612+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'15","last_scrub_stamp":"2024-09-20T19:01:36.532419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:36.532419+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:38.766268+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":783,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.399887+0000","last_change":"2024-09-20T19:01:30.864866+0000","last_active":"2024-09-20T19:06:21.399887+0000","last_peered":"2024-09-20T19:06:21.399887+0000","last_clean":"2024-09-20T19:06:21.399887+0000","last_became_active":"2024-09-20T18:49:45.938511+0000","last_became_peered":"2024-09-20T18:49:45.938511+0000","last_unstale":"2024-09-20T19:06:21.399887+0000","last_undegraded":"2024-09-20T19:06:21.399887+0000","last_fullsized":"2024-09-20T19:06:21.399887+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:30.864794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:30.864794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:11.643706+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":985,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T19:06:19.388903+0000","last_change":"2024-09-20T19:04:41.601208+0000","last_active":"2024-09-20T19:06:19.388903+0000","last_peered":"2024-09-20T19:06:19.388903+0000","last_clean":"2024-09-20T19:06:19.388903+0000","last_became_active":"2024-09-20T18:49:49.960550+0000","last_became_peered":"2024-09-20T18:49:49.960550+0000","last_unstale":"2024-09-20T19:06:19.388903+0000","last_undegraded":"2024-09-20T19:06:19.388903+0000","last_fullsized":"2024-09-20T19:06:19.388903+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T19:04:41.601073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:41.601073+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":111,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:20.173577+0000","last_change":"2024-09-20T19:04:20.884397+0000","last_active":"2024-09-20T19:06:20.173577+0000","last_peered":"2024-09-20T19:06:20.173577+0000","last_clean":"2024-09-20T19:06:20.173577+0000","last_became_active":"2024-09-20T19:04:20.884100+0000","last_became_peered":"2024-09-20T19:04:20.884100+0000","last_unstale":"2024-09-20T19:06:20.173577+0000","last_undegraded":"2024-09-20T19:06:20.173577+0000","last_fullsized":"2024-09-20T19:06:20.173577+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:59.497810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:59:59.497810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:02.999076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"274'20","reported_seq":807,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T19:06:19.389004+0000","last_change":"2024-09-20T18:57:20.701592+0000","last_active":"2024-09-20T19:06:19.389004+0000","last_peered":"2024-09-20T19:06:19.389004+0000","last_clean":"2024-09-20T19:06:19.389004+0000","last_became_active":"2024-09-20T18:49:58.997494+0000","last_became_peered":"2024-09-20T18:49:58.997494+0000","last_unstale":"2024-09-20T19:06:19.389004+0000","last_undegraded":"2024-09-20T19:06:19.389004+0000","last_fullsized":"2024-09-20T19:06:19.389004+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:57:20.701543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:20.701543+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:23.577077+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":798,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.140568+0000","last_change":"2024-09-20T19:04:20.884940+0000","last_active":"2024-09-20T19:06:21.140568+0000","last_peered":"2024-09-20T19:06:21.140568+0000","last_clean":"2024-09-20T19:06:21.140568+0000","last_became_active":"2024-09-20T19:04:20.884582+0000","last_became_peered":"2024-09-20T19:04:20.884582+0000","last_unstale":"2024-09-20T19:06:21.140568+0000","last_undegraded":"2024-09-20T19:06:21.140568+0000","last_fullsized":"2024-09-20T19:06:21.140568+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.632380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.632380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:35.779165+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":978,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:20.059457+0000","last_change":"2024-09-20T19:01:10.601374+0000","last_active":"2024-09-20T19:06:20.059457+0000","last_peered":"2024-09-20T19:06:20.059457+0000","last_clean":"2024-09-20T19:06:20.059457+0000","last_became_active":"2024-09-20T18:49:49.962243+0000","last_became_peered":"2024-09-20T18:49:49.962243+0000","last_unstale":"2024-09-20T19:06:20.059457+0000","last_undegraded":"2024-09-20T19:06:20.059457+0000","last_fullsized":"2024-09-20T19:06:20.059457+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T19:01:10.601300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:10.601300+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:23.615982+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":786,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.399986+0000","last_change":"2024-09-20T19:05:52.578042+0000","last_active":"2024-09-20T19:06:21.399986+0000","last_peered":"2024-09-20T19:06:21.399986+0000","last_clean":"2024-09-20T19:06:21.399986+0000","last_became_active":"2024-09-20T18:49:51.969093+0000","last_became_peered":"2024-09-20T18:49:51.969093+0000","last_unstale":"2024-09-20T19:06:21.399986+0000","last_undegraded":"2024-09-20T19:06:21.399986+0000","last_fullsized":"2024-09-20T19:06:21.399986+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:52.577979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:05:52.577979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:16.347110+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"273'19","reported_seq":123,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:20.173132+0000","last_change":"2024-09-20T19:04:20.970137+0000","last_active":"2024-09-20T19:06:20.173132+0000","last_peered":"2024-09-20T19:06:20.173132+0000","last_clean":"2024-09-20T19:06:20.173132+0000","last_became_active":"2024-09-20T19:04:20.897330+0000","last_became_peered":"2024-09-20T19:04:20.897330+0000","last_unstale":"2024-09-20T19:06:20.173132+0000","last_undegraded":"2024-09-20T19:06:20.173132+0000","last_fullsized":"2024-09-20T19:06:20.173132+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-20T18:57:50.899514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:50.899514+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:38.174326+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":111,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:20.173399+0000","last_change":"2024-09-20T19:04:20.896732+0000","last_active":"2024-09-20T19:06:20.173399+0000","last_peered":"2024-09-20T19:06:20.173399+0000","last_clean":"2024-09-20T19:06:20.173399+0000","last_became_active":"2024-09-20T19:04:20.896007+0000","last_became_peered":"2024-09-20T19:04:20.896007+0000","last_unstale":"2024-09-20T19:06:20.173399+0000","last_undegraded":"2024-09-20T19:06:20.173399+0000","last_fullsized":"2024-09-20T19:06:20.173399+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:06.310027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:59:06.310027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:02.046888+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":774,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.066045+0000","last_change":"2024-09-20T19:02:38.906352+0000","last_active":"2024-09-20T19:06:21.066045+0000","last_peered":"2024-09-20T19:06:21.066045+0000","last_clean":"2024-09-20T19:06:21.066045+0000","last_became_active":"2024-09-20T18:49:49.957067+0000","last_became_peered":"2024-09-20T18:49:49.957067+0000","last_unstale":"2024-09-20T19:06:21.066045+0000","last_undegraded":"2024-09-20T19:06:21.066045+0000","last_fullsized":"2024-09-20T19:06:21.066045+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:38.906161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:38.906161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:36.012448+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":750,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T19:06:19.389629+0000","last_change":"2024-09-20T18:57:16.789412+0000","last_active":"2024-09-20T19:06:19.389629+0000","last_peered":"2024-09-20T19:06:19.389629+0000","last_clean":"2024-09-20T19:06:19.389629+0000","last_became_active":"2024-09-20T18:49:51.966915+0000","last_became_peered":"2024-09-20T18:49:51.966915+0000","last_unstale":"2024-09-20T19:06:19.389629+0000","last_undegraded":"2024-09-20T19:06:19.389629+0000","last_fullsized":"2024-09-20T19:06:19.389629+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:16.789095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:16.789095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:35.677144+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"273'15","reported_seq":845,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.066186+0000","last_change":"2024-09-20T19:02:30.816061+0000","last_active":"2024-09-20T19:06:21.066186+0000","last_peered":"2024-09-20T19:06:21.066186+0000","last_clean":"2024-09-20T19:06:21.066186+0000","last_became_active":"2024-09-20T18:59:05.085230+0000","last_became_peered":"2024-09-20T18:59:05.085230+0000","last_unstale":"2024-09-20T19:06:21.066186+0000","last_undegraded":"2024-09-20T19:06:21.066186+0000","last_fullsized":"2024-09-20T19:06:21.066186+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:02:30.815747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:30.815747+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:39.985336+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":797,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.140676+0000","last_change":"2024-09-20T19:04:20.885127+0000","last_active":"2024-09-20T19:06:21.140676+0000","last_peered":"2024-09-20T19:06:21.140676+0000","last_clean":"2024-09-20T19:06:21.140676+0000","last_became_active":"2024-09-20T19:04:20.884660+0000","last_became_peered":"2024-09-20T19:04:20.884660+0000","last_unstale":"2024-09-20T19:06:21.140676+0000","last_undegraded":"2024-09-20T19:06:21.140676+0000","last_fullsized":"2024-09-20T19:06:21.140676+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:07.408197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:03:07.408197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:07.715846+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":778,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T19:06:19.390099+0000","last_change":"2024-09-20T19:04:44.606497+0000","last_active":"2024-09-20T19:06:19.390099+0000","last_peered":"2024-09-20T19:06:19.390099+0000","last_clean":"2024-09-20T19:06:19.390099+0000","last_became_active":"2024-09-20T18:49:49.962523+0000","last_became_peered":"2024-09-20T18:49:49.962523+0000","last_unstale":"2024-09-20T19:06:19.390099+0000","last_undegraded":"2024-09-20T19:06:19.390099+0000","last_fullsized":"2024-09-20T19:06:19.390099+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:44.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:44.606343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":779,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.399631+0000","last_change":"2024-09-20T19:05:51.626774+0000","last_active":"2024-09-20T19:06:21.399631+0000","last_peered":"2024-09-20T19:06:21.399631+0000","last_clean":"2024-09-20T19:06:21.399631+0000","last_became_active":"2024-09-20T18:49:51.967313+0000","last_became_peered":"2024-09-20T18:49:51.967313+0000","last_unstale":"2024-09-20T19:06:21.399631+0000","last_undegraded":"2024-09-20T19:06:21.399631+0000","last_fullsized":"2024-09-20T19:06:21.399631+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:51.626697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:05:51.626697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:13.789020+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"274'19","reported_seq":843,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T19:06:19.389725+0000","last_change":"2024-09-20T19:01:05.495603+0000","last_active":"2024-09-20T19:06:19.389725+0000","last_peered":"2024-09-20T19:06:19.389725+0000","last_clean":"2024-09-20T19:06:19.389725+0000","last_became_active":"2024-09-20T18:59:05.075390+0000","last_became_peered":"2024-09-20T18:59:05.075390+0000","last_unstale":"2024-09-20T19:06:19.389725+0000","last_undegraded":"2024-09-20T19:06:19.389725+0000","last_fullsized":"2024-09-20T19:06:19.389725+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'19","last_scrub_stamp":"2024-09-20T19:01:05.495478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:05.495478+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:04.804811+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":795,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:20.059651+0000","last_change":"2024-09-20T19:05:43.686902+0000","last_active":"2024-09-20T19:06:20.059651+0000","last_peered":"2024-09-20T19:06:20.059651+0000","last_clean":"2024-09-20T19:06:20.059651+0000","last_became_active":"2024-09-20T18:49:45.942105+0000","last_became_peered":"2024-09-20T18:49:45.942105+0000","last_unstale":"2024-09-20T19:06:20.059651+0000","last_undegraded":"2024-09-20T19:06:20.059651+0000","last_fullsized":"2024-09-20T19:06:20.059651+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:43.686842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:43.686842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:01.625104+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":812,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.140386+0000","last_change":"2024-09-20T19:04:20.912120+0000","last_active":"2024-09-20T19:06:21.140386+0000","last_peered":"2024-09-20T19:06:21.140386+0000","last_clean":"2024-09-20T19:06:21.140386+0000","last_became_active":"2024-09-20T19:04:20.911292+0000","last_became_peered":"2024-09-20T19:04:20.911292+0000","last_unstale":"2024-09-20T19:06:21.140386+0000","last_undegraded":"2024-09-20T19:06:21.140386+0000","last_fullsized":"2024-09-20T19:06:21.140386+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:27.267254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:27.267254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:59.781503+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":780,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T19:06:19.389985+0000","last_change":"2024-09-20T19:05:51.714712+0000","last_active":"2024-09-20T19:06:19.389985+0000","last_peered":"2024-09-20T19:06:19.389985+0000","last_clean":"2024-09-20T19:06:19.389985+0000","last_became_active":"2024-09-20T18:59:05.081179+0000","last_became_peered":"2024-09-20T18:59:05.081179+0000","last_unstale":"2024-09-20T19:06:19.389985+0000","last_undegraded":"2024-09-20T19:06:19.389985+0000","last_fullsized":"2024-09-20T19:06:19.389985+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:51.714641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:05:51.714641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:58.160366+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"273'8","reported_seq":820,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T19:06:19.388311+0000","last_change":"2024-09-20T19:04:40.559947+0000","last_active":"2024-09-20T19:06:19.388311+0000","last_peered":"2024-09-20T19:06:19.388311+0000","last_clean":"2024-09-20T19:06:19.388311+0000","last_became_active":"2024-09-20T18:59:05.076365+0000","last_became_peered":"2024-09-20T18:59:05.076365+0000","last_unstale":"2024-09-20T19:06:19.388311+0000","last_undegraded":"2024-09-20T19:06:19.388311+0000","last_fullsized":"2024-09-20T19:06:19.388311+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'8","last_scrub_stamp":"2024-09-20T19:04:40.559877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:04:40.559877+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":790,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.400310+0000","last_change":"2024-09-20T19:02:39.184249+0000","last_active":"2024-09-20T19:06:21.400310+0000","last_peered":"2024-09-20T19:06:21.400310+0000","last_clean":"2024-09-20T19:06:21.400310+0000","last_became_active":"2024-09-20T18:49:45.940988+0000","last_became_peered":"2024-09-20T18:49:45.940988+0000","last_unstale":"2024-09-20T19:06:21.400310+0000","last_undegraded":"2024-09-20T19:06:21.400310+0000","last_fullsized":"2024-09-20T19:06:21.400310+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:39.184150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:39.184150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":378,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.400374+0000","last_change":"2024-09-20T19:04:20.900567+0000","last_active":"2024-09-20T19:06:21.400374+0000","last_peered":"2024-09-20T19:06:21.400374+0000","last_clean":"2024-09-20T19:06:21.400374+0000","last_became_active":"2024-09-20T19:04:20.900214+0000","last_became_peered":"2024-09-20T19:04:20.900214+0000","last_unstale":"2024-09-20T19:06:21.400374+0000","last_undegraded":"2024-09-20T19:06:21.400374+0000","last_fullsized":"2024-09-20T19:06:21.400374+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:41.497115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:41.497115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:33.282430+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":794,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.140524+0000","last_change":"2024-09-20T19:05:57.789513+0000","last_active":"2024-09-20T19:06:21.140524+0000","last_peered":"2024-09-20T19:06:21.140524+0000","last_clean":"2024-09-20T19:06:21.140524+0000","last_became_active":"2024-09-20T19:04:20.885211+0000","last_became_peered":"2024-09-20T19:04:20.885211+0000","last_unstale":"2024-09-20T19:06:21.140524+0000","last_undegraded":"2024-09-20T19:06:21.140524+0000","last_fullsized":"2024-09-20T19:06:21.140524+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:57.789465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:05:57.789465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:08.966824+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":770,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.066075+0000","last_change":"2024-09-20T19:04:20.890533+0000","last_active":"2024-09-20T19:06:21.066075+0000","last_peered":"2024-09-20T19:06:21.066075+0000","last_clean":"2024-09-20T19:06:21.066075+0000","last_became_active":"2024-09-20T19:04:20.890101+0000","last_became_peered":"2024-09-20T19:04:20.890101+0000","last_unstale":"2024-09-20T19:06:21.066075+0000","last_undegraded":"2024-09-20T19:06:21.066075+0000","last_fullsized":"2024-09-20T19:06:21.066075+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:10.970779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:10.970779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:24.205999+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":769,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T19:06:19.388432+0000","last_change":"2024-09-20T18:58:36.077005+0000","last_active":"2024-09-20T19:06:19.388432+0000","last_peered":"2024-09-20T19:06:19.388432+0000","last_clean":"2024-09-20T19:06:19.388432+0000","last_became_active":"2024-09-20T18:49:45.937992+0000","last_became_peered":"2024-09-20T18:49:45.937992+0000","last_unstale":"2024-09-20T19:06:19.388432+0000","last_undegraded":"2024-09-20T19:06:19.388432+0000","last_fullsized":"2024-09-20T19:06:19.388432+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:36.076937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:58:36.076937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:42.609713+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"274'19","reported_seq":377,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.400910+0000","last_change":"2024-09-20T19:01:15.544760+0000","last_active":"2024-09-20T19:06:21.400910+0000","last_peered":"2024-09-20T19:06:21.400910+0000","last_clean":"2024-09-20T19:06:21.400910+0000","last_became_active":"2024-09-20T18:59:05.089294+0000","last_became_peered":"2024-09-20T18:59:05.089294+0000","last_unstale":"2024-09-20T19:06:21.400910+0000","last_undegraded":"2024-09-20T19:06:21.400910+0000","last_fullsized":"2024-09-20T19:06:21.400910+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'19","last_scrub_stamp":"2024-09-20T19:01:15.544663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:15.544663+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:24.387863+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":768,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-20T19:06:21.400086+0000","last_change":"2024-09-20T19:01:40.961449+0000","last_active":"2024-09-20T19:06:21.400086+0000","last_peered":"2024-09-20T19:06:21.400086+0000","last_clean":"2024-09-20T19:06:21.400086+0000","last_became_active":"2024-09-20T18:49:49.957582+0000","last_became_peered":"2024-09-20T18:49:49.957582+0000","last_unstale":"2024-09-20T19:06:21.400086+0000","last_undegraded":"2024-09-20T19:06:21.400086+0000","last_fullsized":"2024-09-20T19:06:21.400086+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:40.961363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:40.961363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:08.603698+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1185,"internal_metadata":0},"log_size":87,"ondisk_log_size":87,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1114,"num_read_kb":871,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":257,"num_bytes_recovered":4890,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":686,"ondisk_log_size":686,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":3105,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":988,"num_read_kb":983,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266891,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":578596,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":576766,"kb_avail":93166556,"statfs":{"total":95995035648,"available":95402553344,"internally_reserved":0,"allocated":1851392,"data_stored":1272269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":590608842},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528530,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":425904,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":424062,"kb_avail":93319248,"statfs":{"total":95995035648,"available":95558909952,"internally_reserved":0,"allocated":1855488,"data_stored":1285994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":434239942},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757463,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":591904,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":590654,"kb_avail":93153248,"statfs":{"total":95995035648,"available":95388925952,"internally_reserved":0,"allocated":1257472,"data_stored":681956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":604830149},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":304,"seq":1305670058045,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298160,"kb_used_data":1244,"kb_used_omap":5,"kb_used_meta":296890,"kb_avail":93446992,"statfs":{"total":95995035648,"available":95689719808,"internally_reserved":0,"allocated":1273856,"data_stored":682850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5633,"internal_metadata":304015871},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182627,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":726560,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":725310,"kb_avail":93018592,"statfs":{"total":95995035648,"available":95251038208,"internally_reserved":0,"allocated":1253376,"data_stored":681336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":742717895},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411560,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":609964,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":608702,"kb_avail":93135188,"statfs":{"total":95995035648,"available":95370432512,"internally_reserved":0,"allocated":1269760,"data_stored":696564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":623311301},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575086,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":662308,"kb_used_data":39688,"kb_used_omap":1,"kb_used_meta":622590,"kb_avail":93082844,"statfs":{"total":95995035648,"available":95316832256,"internally_reserved":0,"allocated":40640512,"data_stored":40019348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":637532613},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738612,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":686908,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":685054,"kb_avail":93058244,"statfs":{"total":95995035648,"available":95291641856,"internally_reserved":0,"allocated":1875968,"data_stored":1286352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":701495749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1185,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T19:06:25.737 INFO:tasks.thrashosds.thrasher:Setting 5.1c to [1, 5, 7, 2, 4, 3] 2024-09-20T19:06:25.737 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '5.1c', '1', '5', '7', '2', '4', '3'] 2024-09-20T19:06:25.737 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd pg-upmap-items 5.1c 1 5 7 2 4 3 2024-09-20T19:06:25.750 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:25.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:25.751 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:25.852 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:06:25.866 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_blocked_ops 2024-09-20T19:06:25.898 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:25.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:25.899 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:06:26.000 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:26.037 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:26.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:26.038 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:26.112 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 3 16 9008 8992 187.264 179.625 0.00188298 0.00532821 2024-09-20T19:06:26.138 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:26.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:26.178 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:26.178 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:26.225 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:25 smithi083 ceph-mon[29421]: 6.1d scrub starts 2024-09-20T19:06:26.225 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:25 smithi083 ceph-mon[29421]: 6.1d scrub ok 2024-09-20T19:06:26.225 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:25 smithi083 ceph-mon[29421]: pgmap v840: 148 pgs: 148 active+clean; 583 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 837 B/s rd, 0 op/s 2024-09-20T19:06:26.225 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:25 smithi083 ceph-mon[29421]: 6.f scrub starts 2024-09-20T19:06:26.225 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:25 smithi083 ceph-mon[29421]: 6.f scrub ok 2024-09-20T19:06:26.226 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:25 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:06:26.226 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:25 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:06:26.226 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:25 smithi083 ceph-mon[29421]: osdmap e427: 8 total, 8 up, 8 in 2024-09-20T19:06:26.226 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:25 smithi083 ceph-mon[36260]: 6.1d scrub starts 2024-09-20T19:06:26.226 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:25 smithi083 ceph-mon[36260]: 6.1d scrub ok 2024-09-20T19:06:26.226 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:25 smithi083 ceph-mon[36260]: pgmap v840: 148 pgs: 148 active+clean; 583 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 837 B/s rd, 0 op/s 2024-09-20T19:06:26.226 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:25 smithi083 ceph-mon[36260]: 6.f scrub starts 2024-09-20T19:06:26.226 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:25 smithi083 ceph-mon[36260]: 6.f scrub ok 2024-09-20T19:06:26.226 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:25 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:06:26.227 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:25 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:06:26.227 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:25 smithi083 ceph-mon[36260]: osdmap e427: 8 total, 8 up, 8 in 2024-09-20T19:06:26.232 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:06:26.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:25 smithi142 ceph-mon[32508]: 6.1d scrub starts 2024-09-20T19:06:26.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:25 smithi142 ceph-mon[32508]: 6.1d scrub ok 2024-09-20T19:06:26.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:25 smithi142 ceph-mon[32508]: pgmap v840: 148 pgs: 148 active+clean; 583 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 837 B/s rd, 0 op/s 2024-09-20T19:06:26.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:25 smithi142 ceph-mon[32508]: 6.f scrub starts 2024-09-20T19:06:26.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:25 smithi142 ceph-mon[32508]: 6.f scrub ok 2024-09-20T19:06:26.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:25 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:06:26.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:25 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:06:26.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:25 smithi142 ceph-mon[32508]: osdmap e427: 8 total, 8 up, 8 in 2024-09-20T19:06:26.279 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:26.307 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:26.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:26.308 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:26.338 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:06:26.408 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:06:26.447 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:26.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:26.448 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:06:26.549 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:26.585 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:26.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:26.586 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:26.686 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:26.714 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:26.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:26.715 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:26.816 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:26.846 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T19:06:26.852 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:26.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:26.853 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:26.916 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T19:06:26.917 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:06:26.953 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:26.983 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-20T19:06:26.996 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:26.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:26.997 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:27.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:26 smithi142 ceph-mon[32508]: 5.18 scrub starts 2024-09-20T19:06:27.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:26 smithi142 ceph-mon[32508]: 5.18 scrub ok 2024-09-20T19:06:27.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:26 smithi142 ceph-mon[32508]: 3.f scrub starts 2024-09-20T19:06:27.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:26 smithi142 ceph-mon[32508]: 3.f scrub ok 2024-09-20T19:06:27.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:26 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1116933003' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:06:27.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:26 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:06:27.098 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:27.112 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 4 16 11274 11258 175.853 141.625 0.00298815 0.00567419 2024-09-20T19:06:27.136 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:27.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:27.137 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:27.238 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:27.291 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:27.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:26 smithi083 ceph-mon[29421]: 5.18 scrub starts 2024-09-20T19:06:27.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:26 smithi083 ceph-mon[29421]: 5.18 scrub ok 2024-09-20T19:06:27.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:26 smithi083 ceph-mon[29421]: 3.f scrub starts 2024-09-20T19:06:27.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:26 smithi083 ceph-mon[29421]: 3.f scrub ok 2024-09-20T19:06:27.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:26 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1116933003' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:06:27.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:26 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:06:27.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:26 smithi083 ceph-mon[36260]: 5.18 scrub starts 2024-09-20T19:06:27.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:26 smithi083 ceph-mon[36260]: 5.18 scrub ok 2024-09-20T19:06:27.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:26 smithi083 ceph-mon[36260]: 3.f scrub starts 2024-09-20T19:06:27.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:26 smithi083 ceph-mon[36260]: 3.f scrub ok 2024-09-20T19:06:27.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:26 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1116933003' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:06:27.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:26 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:06:27.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:27.293 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:27.394 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:27.433 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:27.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:27.434 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:27.535 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:27.571 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:27.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:27.572 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:27.672 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:27.699 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:06:27 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:06:27.700 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_historic_ops 2024-09-20T19:06:27.714 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:27.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:27.715 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:27.815 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:27.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:27.858 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:27.858 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:27.958 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:27.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:27.994 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:27.994 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:27.997 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:06:28.014 INFO:teuthology.orchestra.run.smithi083.stderr:set 5.1c pg_upmap_items mapping to [1->5,7->2,4->3] 2024-09-20T19:06:28.095 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:28.112 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 5 16 13888 13872 173.355 163.375 0.00359851 0.00576031 2024-09-20T19:06:28.120 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:06:28.132 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:28.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:28.133 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:28.234 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:28.256 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:27 smithi142 ceph-mon[32508]: 5.15 scrub starts 2024-09-20T19:06:28.256 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:27 smithi142 ceph-mon[32508]: 5.15 scrub ok 2024-09-20T19:06:28.256 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:27 smithi142 ceph-mon[32508]: 4.1f scrub starts 2024-09-20T19:06:28.256 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:27 smithi142 ceph-mon[32508]: 4.1f scrub ok 2024-09-20T19:06:28.256 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:27 smithi142 ceph-mon[32508]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:06:28.256 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:27 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T19:06:28.256 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:27 smithi142 ceph-mon[32508]: osdmap e428: 8 total, 8 up, 8 in 2024-09-20T19:06:28.256 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:27 smithi142 ceph-mon[32508]: pgmap v843: 148 pgs: 148 active+clean; 560 MiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 1.6 KiB/s rd, 92 MiB/s wr, 1.47k op/s 2024-09-20T19:06:28.256 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:27 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:06:28.256 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:27 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1640088259' entity='client.admin' cmd={"prefix": "osd pg-upmap-items", "pgid": "5.1c", "id": [1, 5, 7, 2, 4, 3]} : dispatch 2024-09-20T19:06:28.256 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:27 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pg-upmap-items", "pgid": "5.1c", "id": [1, 5, 7, 2, 4, 3]} : dispatch 2024-09-20T19:06:28.263 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:28.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:28.264 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:28.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:27 smithi083 ceph-mon[29421]: 5.15 scrub starts 2024-09-20T19:06:28.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:27 smithi083 ceph-mon[29421]: 5.15 scrub ok 2024-09-20T19:06:28.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:27 smithi083 ceph-mon[29421]: 4.1f scrub starts 2024-09-20T19:06:28.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:27 smithi083 ceph-mon[29421]: 4.1f scrub ok 2024-09-20T19:06:28.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:27 smithi083 ceph-mon[29421]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:06:28.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:27 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T19:06:28.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:27 smithi083 ceph-mon[29421]: osdmap e428: 8 total, 8 up, 8 in 2024-09-20T19:06:28.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:27 smithi083 ceph-mon[29421]: pgmap v843: 148 pgs: 148 active+clean; 560 MiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 1.6 KiB/s rd, 92 MiB/s wr, 1.47k op/s 2024-09-20T19:06:28.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:27 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:06:28.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:27 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1640088259' entity='client.admin' cmd={"prefix": "osd pg-upmap-items", "pgid": "5.1c", "id": [1, 5, 7, 2, 4, 3]} : dispatch 2024-09-20T19:06:28.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:27 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pg-upmap-items", "pgid": "5.1c", "id": [1, 5, 7, 2, 4, 3]} : dispatch 2024-09-20T19:06:28.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:27 smithi083 ceph-mon[36260]: 5.15 scrub starts 2024-09-20T19:06:28.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:27 smithi083 ceph-mon[36260]: 5.15 scrub ok 2024-09-20T19:06:28.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:28 smithi083 ceph-mon[36260]: 4.1f scrub starts 2024-09-20T19:06:28.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:28 smithi083 ceph-mon[36260]: 4.1f scrub ok 2024-09-20T19:06:28.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:28 smithi083 ceph-mon[36260]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:06:28.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:28 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T19:06:28.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:28 smithi083 ceph-mon[36260]: osdmap e428: 8 total, 8 up, 8 in 2024-09-20T19:06:28.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:28 smithi083 ceph-mon[36260]: pgmap v843: 148 pgs: 148 active+clean; 560 MiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 1.6 KiB/s rd, 92 MiB/s wr, 1.47k op/s 2024-09-20T19:06:28.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:28 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:06:28.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:28 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1640088259' entity='client.admin' cmd={"prefix": "osd pg-upmap-items", "pgid": "5.1c", "id": [1, 5, 7, 2, 4, 3]} : dispatch 2024-09-20T19:06:28.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:28 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pg-upmap-items", "pgid": "5.1c", "id": [1, 5, 7, 2, 4, 3]} : dispatch 2024-09-20T19:06:28.364 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:28.400 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:28.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:28.402 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:28.503 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:28.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:28.531 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:28.531 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:28.632 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:28.668 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:28.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:28.669 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:28.770 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:28.794 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:28.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:28.795 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:28.895 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:28.933 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:28.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:28.935 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:29.035 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:29.056 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:29.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:29.057 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:29.112 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 6 15 16261 16246 169.191 148.375 0.00887857 0.00590393 2024-09-20T19:06:29.158 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:29.193 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:29.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:29.195 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:29.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:29 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:06:29.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:29 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:06:29.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:29 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "5.1c", "id": [1, 5, 7, 2, 4, 3]}]': finished 2024-09-20T19:06:29.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:29 smithi142 ceph-mon[32508]: osdmap e429: 8 total, 8 up, 8 in 2024-09-20T19:06:29.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:29 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:06:29.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:29 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:06:29.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:29 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "5.1c", "id": [1, 5, 7, 2, 4, 3]}]': finished 2024-09-20T19:06:29.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:29 smithi083 ceph-mon[29421]: osdmap e429: 8 total, 8 up, 8 in 2024-09-20T19:06:29.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:29 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:06:29.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:29 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:06:29.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:29 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "5.1c", "id": [1, 5, 7, 2, 4, 3]}]': finished 2024-09-20T19:06:29.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:29 smithi083 ceph-mon[36260]: osdmap e429: 8 total, 8 up, 8 in 2024-09-20T19:06:29.295 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:29.330 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:29.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:29.331 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:29.431 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:29.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:29.466 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:29.466 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:29.567 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:29.599 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:29.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:29.600 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:29.701 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:29.722 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:29.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:29.723 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:29.824 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:29.857 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:29.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:29.858 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:29.879 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T19:06:29.959 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:29.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:29.998 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:29.998 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:30.099 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:30.113 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 7 16 18653 18637 166.367 149.438 0.00760186 0.00600356 2024-09-20T19:06:30.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:30.146 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:30.147 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:30.199 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T19:06:30.199 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:06:30.248 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:30.270 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:30.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:30.271 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:30.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:30 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:06:30.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:30 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:06:30.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:30 smithi083 ceph-mon[29421]: osdmap e430: 8 total, 8 up, 8 in 2024-09-20T19:06:30.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:30 smithi083 ceph-mon[29421]: pgmap v846: 148 pgs: 148 active+clean; 610 MiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 152 MiB/s wr, 2.44k op/s 2024-09-20T19:06:30.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:30 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:06:30.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:30 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:06:30.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:30 smithi083 ceph-mon[36260]: osdmap e430: 8 total, 8 up, 8 in 2024-09-20T19:06:30.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:30 smithi083 ceph-mon[36260]: pgmap v846: 148 pgs: 148 active+clean; 610 MiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 152 MiB/s wr, 2.44k op/s 2024-09-20T19:06:30.371 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:30.402 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:30.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:30.403 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:30.504 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:30.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:30 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:06:30.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:30 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:06:30.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:30 smithi142 ceph-mon[32508]: osdmap e430: 8 total, 8 up, 8 in 2024-09-20T19:06:30.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:30 smithi142 ceph-mon[32508]: pgmap v846: 148 pgs: 148 active+clean; 610 MiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 152 MiB/s wr, 2.44k op/s 2024-09-20T19:06:30.525 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:30.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:30.526 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:30.627 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:30.665 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:30.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:30.666 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:30.767 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:30.791 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:30.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:30.792 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:30.893 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:30.944 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:30.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:30.945 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:31.042 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:06:30 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:06:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:06:31.046 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:31.058 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:06:31.060 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:06:31.060 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:06:31.075 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:31.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:31.076 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:31.095 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:06:31.095 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:06:31.112 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 8 16 20589 20573 160.69 121 0.00484193 0.00621617 2024-09-20T19:06:31.132 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:06:31.132 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:06:31.168 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:06:31.169 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:06:31.176 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:31.208 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:06:31.208 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:06:31.218 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:31.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:31.219 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:31.241 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:06:31.242 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:06:31.274 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:06:31.274 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:06:31.308 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:06:31.308 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:06:31.320 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:31.352 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:31.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:31.353 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:31.445 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-20T19:06:31.453 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:31.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:31.491 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:31.491 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:31.591 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:31.625 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:31.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:31.626 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:31.727 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:31.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:31.768 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:31.768 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:31.827 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:06:31.869 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:31.904 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:31.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:31.905 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:32.006 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:32.044 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:32.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:32.045 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:32.112 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 9 16 22855 22839 158.57 141.625 0.00769248 0.00630145 2024-09-20T19:06:32.146 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:32.186 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:32.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:32.187 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:32.192 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:31 smithi083 ceph-mon[29421]: osdmap e431: 8 total, 8 up, 8 in 2024-09-20T19:06:32.192 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:31 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3185732530' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:06:32.192 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:31 smithi083 ceph-mon[29421]: pgmap v848: 148 pgs: 148 active+clean; 882 MiB data, 6.7 GiB used, 709 GiB / 715 GiB avail; 98 MiB/s wr, 1.56k op/s 2024-09-20T19:06:32.192 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:31 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:06:32.193 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:31 smithi083 ceph-mon[36260]: osdmap e431: 8 total, 8 up, 8 in 2024-09-20T19:06:32.193 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:31 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3185732530' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:06:32.193 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:31 smithi083 ceph-mon[36260]: pgmap v848: 148 pgs: 148 active+clean; 882 MiB data, 6.7 GiB used, 709 GiB / 715 GiB avail; 98 MiB/s wr, 1.56k op/s 2024-09-20T19:06:32.193 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:31 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:06:32.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:31 smithi142 ceph-mon[32508]: osdmap e431: 8 total, 8 up, 8 in 2024-09-20T19:06:32.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:31 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3185732530' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:06:32.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:31 smithi142 ceph-mon[32508]: pgmap v848: 148 pgs: 148 active+clean; 882 MiB data, 6.7 GiB used, 709 GiB / 715 GiB avail; 98 MiB/s wr, 1.56k op/s 2024-09-20T19:06:32.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:31 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:06:32.288 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:06:32.327 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:32.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:32.328 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:06:32.428 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:32.430 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T19:06:32.430 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-09-20T19:06:32.430 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:32.430 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.26264.0:23549 11.f 11:f1213bf1:::benchmark_data_smithi083_177491_object23548:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e431)", 2024-09-20T19:06:32.430 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:06:32.417995+0000", 2024-09-20T19:06:32.430 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.0058850389999999999, 2024-09-20T19:06:32.430 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0059174350000000004, 2024-09-20T19:06:32.431 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:06:32.431 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:06:32.431 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "waiting for sub ops", 2024-09-20T19:06:32.431 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:06:32.431 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.26264", 2024-09-20T19:06:32.431 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2608780316", 2024-09-20T19:06:32.431 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 23549 2024-09-20T19:06:32.431 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:32.431 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:06:32.431 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:32.431 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:06:32.432 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:32.417995+0000", 2024-09-20T19:06:32.432 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:32.432 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:32.432 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:32.432 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:06:32.432 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:32.417995+0000", 2024-09-20T19:06:32.432 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:32.432 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:32.432 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:32.432 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:06:32.433 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:32.417996+0000", 2024-09-20T19:06:32.433 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.446e-06 2024-09-20T19:06:32.433 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:32.433 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:32.433 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:06:32.433 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:32.418046+0000", 2024-09-20T19:06:32.433 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0158999999999999e-05 2024-09-20T19:06:32.433 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:32.433 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:32.433 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:06:32.433 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:32.418060+0000", 2024-09-20T19:06:32.434 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3823e-05 2024-09-20T19:06:32.434 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:32.434 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:32.434 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:06:32.434 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:32.418068+0000", 2024-09-20T19:06:32.434 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.9570000000000002e-06 2024-09-20T19:06:32.434 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:32.434 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:32.434 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:32.434 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:32.418132+0000", 2024-09-20T19:06:32.434 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.3338999999999998e-05 2024-09-20T19:06:32.434 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:32.435 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:32.435 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:06:32.435 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:32.418256+0000", 2024-09-20T19:06:32.435 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012458799999999999 2024-09-20T19:06:32.435 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:32.435 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:32.435 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:06:32.435 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:32.418385+0000", 2024-09-20T19:06:32.435 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012924400000000001 2024-09-20T19:06:32.435 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:32.435 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:32.435 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:06:32.436 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:32.419730+0000", 2024-09-20T19:06:32.436 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0013445499999999999 2024-09-20T19:06:32.436 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:32.436 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:06:32.436 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:32.436 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:32.436 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:32.436 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.26264.0:23555 11.f 11:f6ce7bb9:::benchmark_data_smithi083_177491_object23554:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e431)", 2024-09-20T19:06:32.436 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:06:32.420376+0000", 2024-09-20T19:06:32.436 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.0035041479999999999, 2024-09-20T19:06:32.436 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.003619386, 2024-09-20T19:06:32.436 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:06:32.437 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:06:32.437 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "waiting for sub ops", 2024-09-20T19:06:32.437 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:06:32.437 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.26264", 2024-09-20T19:06:32.437 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2608780316", 2024-09-20T19:06:32.437 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 23555 2024-09-20T19:06:32.437 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:32.437 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:06:32.437 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:32.437 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:06:32.437 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:32.420376+0000", 2024-09-20T19:06:32.437 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:32.437 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:32.438 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:32.438 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:06:32.438 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:32.420376+0000", 2024-09-20T19:06:32.438 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:32.438 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:32.438 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:32.438 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:06:32.438 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:32.420377+0000", 2024-09-20T19:06:32.438 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.2399999999999996e-07 2024-09-20T19:06:32.438 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:32.438 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:32.438 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:06:32.439 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:32.420447+0000", 2024-09-20T19:06:32.439 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.9983999999999999e-05 2024-09-20T19:06:32.439 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:32.439 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:32.439 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:06:32.439 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:32.421114+0000", 2024-09-20T19:06:32.439 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00066754399999999995 2024-09-20T19:06:32.439 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:32.439 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:32.439 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:06:32.439 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:32.421123+0000", 2024-09-20T19:06:32.439 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.0690000000000008e-06 2024-09-20T19:06:32.440 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:32.440 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:32.440 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:32.440 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:32.421295+0000", 2024-09-20T19:06:32.440 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000171483 2024-09-20T19:06:32.440 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:32.440 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:32.440 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:06:32.440 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:32.421409+0000", 2024-09-20T19:06:32.440 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011395099999999999 2024-09-20T19:06:32.440 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:32.440 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:32.441 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:06:32.441 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:32.421523+0000", 2024-09-20T19:06:32.441 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011442600000000001 2024-09-20T19:06:32.441 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:32.441 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:32.441 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:06:32.441 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:32.422568+0000", 2024-09-20T19:06:32.441 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001044817 2024-09-20T19:06:32.441 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:32.441 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:06:32.441 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:32.441 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:32.442 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-09-20T19:06:32.442 INFO:teuthology.orchestra.run.smithi142.stdout: "num_ops": 2 2024-09-20T19:06:32.442 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T19:06:32.469 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:32.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:32.470 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:32.571 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:32.607 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:32.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:32.608 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:32.709 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:32.744 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:32.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:32.745 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:32.846 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:32.882 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:32.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:32.883 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:32.947 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-20T19:06:32.984 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:33.020 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:33.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:33.021 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:33.112 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 10 16 25239 25223 157.612 149 0.00604178 0.0063391 2024-09-20T19:06:33.122 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:33.161 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:33.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:33.162 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:33.202 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:32 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/601132435' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:06:33.202 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:32 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:06:33.203 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:32 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/601132435' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:06:33.203 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:32 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:06:33.210 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:32 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/601132435' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:06:33.210 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:32 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:06:33.263 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_blocked_ops 2024-09-20T19:06:33.263 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:33.294 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:33.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:33.295 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:33.396 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:33.430 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:33.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:33.432 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:33.532 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:33.569 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:33.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:33.570 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:33.671 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:06:33.679 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:06:33.708 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:33.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:33.709 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:06:33.810 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:33.845 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 7, 6, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 5, 6, 7, 4] 2024-09-20T19:06:33.845 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T19:06:33.845 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-20T19:06:33.845 INFO:tasks.thrashosds.thrasher:Setting osd 1 primary_affinity to 0.888283 2024-09-20T19:06:33.845 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd primary-affinity 1 0.8882825953359879 2024-09-20T19:06:33.850 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:33.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:33.852 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:33.953 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:33.988 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:33.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:33.989 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:34.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:33 smithi083 ceph-mon[29421]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:06:34.042 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:33 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T19:06:34.042 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:33 smithi083 ceph-mon[29421]: osdmap e432: 8 total, 8 up, 8 in 2024-09-20T19:06:34.042 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:33 smithi083 ceph-mon[29421]: pgmap v850: 148 pgs: 148 active+clean; 1.3 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 990 B/s rd, 140 MiB/s wr, 2.24k op/s 2024-09-20T19:06:34.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:33 smithi083 ceph-mon[36260]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:06:34.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:33 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T19:06:34.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:33 smithi083 ceph-mon[36260]: osdmap e432: 8 total, 8 up, 8 in 2024-09-20T19:06:34.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:33 smithi083 ceph-mon[36260]: pgmap v850: 148 pgs: 148 active+clean; 1.3 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 990 B/s rd, 140 MiB/s wr, 2.24k op/s 2024-09-20T19:06:34.090 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:34.113 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 11 16 27449 27433 155.839 138.125 0.00834493 0.00640932 2024-09-20T19:06:34.132 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:34.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:34.133 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:34.194 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T19:06:34.235 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:34.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:33 smithi142 ceph-mon[32508]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:06:34.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:33 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T19:06:34.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:33 smithi142 ceph-mon[32508]: osdmap e432: 8 total, 8 up, 8 in 2024-09-20T19:06:34.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:33 smithi142 ceph-mon[32508]: pgmap v850: 148 pgs: 148 active+clean; 1.3 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 990 B/s rd, 140 MiB/s wr, 2.24k op/s 2024-09-20T19:06:34.263 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:34.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:34.264 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:34.327 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:06:34.365 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:34.399 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:34.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:34.400 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:34.501 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:34.528 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:34.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:34.529 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:34.630 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:34.672 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:34.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:34.673 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:34.774 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:34.794 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:34.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:34.795 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:34.896 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:34.929 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:34.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:34.930 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:35.030 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:35.064 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:35.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:35.065 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:35.113 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 12 16 29382 29366 152.92 120.812 0.00709935 0.00653375 2024-09-20T19:06:35.166 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:35.202 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:35.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:35.203 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:35.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:34 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:06:35.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:34 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:06:35.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:34 smithi142 ceph-mon[32508]: osdmap e433: 8 total, 8 up, 8 in 2024-09-20T19:06:35.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:34 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:06:35.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:34 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:06:35.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:34 smithi083 ceph-mon[29421]: osdmap e433: 8 total, 8 up, 8 in 2024-09-20T19:06:35.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:34 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:06:35.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:34 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:06:35.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:34 smithi083 ceph-mon[36260]: osdmap e433: 8 total, 8 up, 8 in 2024-09-20T19:06:35.304 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:35.340 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:35.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:35.341 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:35.442 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:35.496 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:35.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:35.498 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:35.598 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:35.623 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:35.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:35.624 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:35.662 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T19:06:35.662 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-09-20T19:06:35.662 INFO:teuthology.orchestra.run.smithi142.stdout: "complaint_time": 30, 2024-09-20T19:06:35.662 INFO:teuthology.orchestra.run.smithi142.stdout: "num_blocked_ops": 0 2024-09-20T19:06:35.662 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T19:06:35.724 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:35.760 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:35.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:35.761 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:35.848 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T19:06:35.862 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:35.894 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:35.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:35.896 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:35.996 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:06:36.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:36.047 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:36.047 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:06:36.113 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 13 16 31230 31214 150.041 115.5 0.00895752 0.00665886 2024-09-20T19:06:36.148 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:36.175 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:36.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:36.176 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:36.187 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:35 smithi083 ceph-mon[29421]: pgmap v852: 148 pgs: 148 active+clean; 1.3 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 853 B/s rd, 121 MiB/s wr, 1.93k op/s 2024-09-20T19:06:36.187 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:35 smithi083 ceph-mon[36260]: pgmap v852: 148 pgs: 148 active+clean; 1.3 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 853 B/s rd, 121 MiB/s wr, 1.93k op/s 2024-09-20T19:06:36.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:35 smithi142 ceph-mon[32508]: pgmap v852: 148 pgs: 148 active+clean; 1.3 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 853 B/s rd, 121 MiB/s wr, 1.93k op/s 2024-09-20T19:06:36.277 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:36.313 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:36.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:36.314 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:36.402 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_historic_ops 2024-09-20T19:06:36.414 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:36.481 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:36.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:36.482 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:36.583 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:36.617 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:36.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:36.618 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:36.719 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:36.769 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:36.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:36.770 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:36.786 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:06:36.870 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:36.912 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:36.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:36.913 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:37.014 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:37.035 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:37.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:37.036 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:37.113 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 14 16 32956 32940 147.028 107.875 0.00207339 0.00674511 2024-09-20T19:06:37.136 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:37.162 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:37.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:37.163 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:37.264 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:37.295 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:37.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:37.296 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:37.376 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T19:06:37.376 INFO:teuthology.orchestra.run.smithi142.stdout: "size": 20, 2024-09-20T19:06:37.376 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 600, 2024-09-20T19:06:37.377 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-09-20T19:06:37.377 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.377 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:84060 9.e 9:73c1c2b3:::benchmark_data_smithi083_124552_object84059:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e268)", 2024-09-20T19:06:37.377 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:38.588091+0000", 2024-09-20T19:06:37.377 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 418.779548406, 2024-09-20T19:06:37.377 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13688319800000001, 2024-09-20T19:06:37.377 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:06:37.377 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:06:37.377 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:37.377 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:06:37.377 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:06:37.377 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:06:37.377 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 84060 2024-09-20T19:06:37.377 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.378 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:06:37.378 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.378 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:06:37.378 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.588091+0000", 2024-09-20T19:06:37.378 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:37.378 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.378 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.378 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:06:37.378 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.588091+0000", 2024-09-20T19:06:37.378 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:37.378 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.378 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.378 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:06:37.378 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.588092+0000", 2024-09-20T19:06:37.378 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.694e-06 2024-09-20T19:06:37.379 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.379 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.379 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:06:37.379 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.588232+0000", 2024-09-20T19:06:37.379 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013927899999999999 2024-09-20T19:06:37.379 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.379 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.379 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:06:37.379 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.588247+0000", 2024-09-20T19:06:37.379 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5670000000000001e-05 2024-09-20T19:06:37.379 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.379 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.379 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:06:37.380 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.588255+0000", 2024-09-20T19:06:37.380 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.0630000000000006e-06 2024-09-20T19:06:37.380 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.380 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.380 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:37.380 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.588320+0000", 2024-09-20T19:06:37.380 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5049999999999996e-05 2024-09-20T19:06:37.380 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.380 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.380 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:06:37.380 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.588440+0000", 2024-09-20T19:06:37.380 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011967 2024-09-20T19:06:37.380 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.380 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.381 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,4", 2024-09-20T19:06:37.381 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.588596+0000", 2024-09-20T19:06:37.381 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015572200000000001 2024-09-20T19:06:37.381 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.381 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.381 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:06:37.381 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.591386+0000", 2024-09-20T19:06:37.381 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.002789962 2024-09-20T19:06:37.381 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.381 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.381 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:37.381 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.591731+0000", 2024-09-20T19:06:37.381 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00034524700000000002 2024-09-20T19:06:37.381 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.382 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.382 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:37.382 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.724872+0000", 2024-09-20T19:06:37.382 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13314100300000001 2024-09-20T19:06:37.382 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.382 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.382 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:06:37.382 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.724930+0000", 2024-09-20T19:06:37.382 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.7472000000000001e-05 2024-09-20T19:06:37.382 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.382 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.382 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:06:37.383 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:38.724974+0000", 2024-09-20T19:06:37.383 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.4366000000000002e-05 2024-09-20T19:06:37.383 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:37.383 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:06:37.383 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:37.383 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.383 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.383 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:101846 9.d 9:bb37c834:::benchmark_data_smithi083_124552_object101845:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e275)", 2024-09-20T19:06:37.383 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T18:59:57.497943+0000", 2024-09-20T19:06:37.383 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 399.86969592499997, 2024-09-20T19:06:37.383 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.22483731800000001, 2024-09-20T19:06:37.383 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:06:37.383 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:06:37.383 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:37.384 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:06:37.384 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:06:37.384 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:06:37.384 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 101846 2024-09-20T19:06:37.384 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.384 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:06:37.384 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.384 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:06:37.384 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.497943+0000", 2024-09-20T19:06:37.384 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:37.384 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.384 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.384 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:06:37.385 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.497943+0000", 2024-09-20T19:06:37.385 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:37.385 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.385 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.385 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:06:37.385 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.497945+0000", 2024-09-20T19:06:37.385 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6029999999999999e-06 2024-09-20T19:06:37.385 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.385 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.385 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:06:37.385 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.497989+0000", 2024-09-20T19:06:37.385 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3755999999999999e-05 2024-09-20T19:06:37.385 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.385 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.386 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:06:37.386 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.498001+0000", 2024-09-20T19:06:37.386 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2615e-05 2024-09-20T19:06:37.386 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.386 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.386 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:06:37.386 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.498010+0000", 2024-09-20T19:06:37.386 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.3599999999999996e-06 2024-09-20T19:06:37.386 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.386 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.386 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:37.386 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.498066+0000", 2024-09-20T19:06:37.386 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6923999999999999e-05 2024-09-20T19:06:37.387 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.387 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.387 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:06:37.387 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.498167+0000", 2024-09-20T19:06:37.387 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000100721 2024-09-20T19:06:37.387 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.387 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.387 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,5", 2024-09-20T19:06:37.387 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.498268+0000", 2024-09-20T19:06:37.387 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000100488 2024-09-20T19:06:37.387 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.387 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.388 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:06:37.388 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.500395+0000", 2024-09-20T19:06:37.388 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0021269259999999999 2024-09-20T19:06:37.388 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.388 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.388 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:37.388 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.500711+0000", 2024-09-20T19:06:37.388 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00031641299999999998 2024-09-20T19:06:37.388 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.388 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.388 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:37.388 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.722681+0000", 2024-09-20T19:06:37.388 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.22197028999999999 2024-09-20T19:06:37.388 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.389 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.389 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:06:37.389 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.722735+0000", 2024-09-20T19:06:37.389 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3879999999999999e-05 2024-09-20T19:06:37.389 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.389 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.389 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:06:37.389 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T18:59:57.722780+0000", 2024-09-20T19:06:37.389 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.5342000000000001e-05 2024-09-20T19:06:37.389 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:37.389 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:06:37.389 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:37.389 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.390 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.390 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:119689 9.d 9:bd867cd9:::benchmark_data_smithi083_124552_object5542:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e287)", 2024-09-20T19:06:37.390 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:00:16.841487+0000", 2024-09-20T19:06:37.390 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 380.52615168599999, 2024-09-20T19:06:37.390 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13893808099999999, 2024-09-20T19:06:37.390 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:06:37.390 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:06:37.390 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:37.390 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:06:37.390 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:06:37.390 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:06:37.390 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 119689 2024-09-20T19:06:37.390 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.391 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:06:37.391 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.391 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:06:37.391 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841487+0000", 2024-09-20T19:06:37.391 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:37.391 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.391 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.391 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:06:37.391 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841487+0000", 2024-09-20T19:06:37.391 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:37.391 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.391 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.391 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:06:37.391 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841489+0000", 2024-09-20T19:06:37.392 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2309999999999999e-06 2024-09-20T19:06:37.392 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.392 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.392 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:06:37.392 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841493+0000", 2024-09-20T19:06:37.392 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.374e-06 2024-09-20T19:06:37.392 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.392 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.392 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:06:37.392 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841502+0000", 2024-09-20T19:06:37.392 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.3649999999999993e-06 2024-09-20T19:06:37.392 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.392 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.392 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:06:37.393 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841509+0000", 2024-09-20T19:06:37.393 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.0999999999999998e-06 2024-09-20T19:06:37.393 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.393 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.393 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:37.393 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841561+0000", 2024-09-20T19:06:37.393 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.1968000000000001e-05 2024-09-20T19:06:37.393 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.393 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.393 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:06:37.393 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841684+0000", 2024-09-20T19:06:37.393 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012302 2024-09-20T19:06:37.393 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.393 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.394 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,5", 2024-09-20T19:06:37.394 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841798+0000", 2024-09-20T19:06:37.394 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011330799999999999 2024-09-20T19:06:37.394 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.394 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.394 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:37.394 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.845340+0000", 2024-09-20T19:06:37.394 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0035421129999999999 2024-09-20T19:06:37.394 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.394 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.394 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:06:37.394 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.849468+0000", 2024-09-20T19:06:37.394 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0041283489999999999 2024-09-20T19:06:37.395 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.395 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.395 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:37.395 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.980284+0000", 2024-09-20T19:06:37.395 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13081616800000001 2024-09-20T19:06:37.395 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.395 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.395 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:06:37.395 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.980351+0000", 2024-09-20T19:06:37.395 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.6928999999999996e-05 2024-09-20T19:06:37.395 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.395 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.395 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:06:37.395 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.980425+0000", 2024-09-20T19:06:37.395 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.4155999999999999e-05 2024-09-20T19:06:37.396 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:37.396 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:06:37.396 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:37.396 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.396 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.396 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:119690 9.3 9:cd5c9a6c:::benchmark_data_smithi083_124552_object5543:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e287)", 2024-09-20T19:06:37.396 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:00:16.841539+0000", 2024-09-20T19:06:37.396 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 380.52610047600001, 2024-09-20T19:06:37.396 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.138853324, 2024-09-20T19:06:37.396 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:06:37.396 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:06:37.396 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:37.396 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:06:37.397 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:06:37.397 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:06:37.397 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 119690 2024-09-20T19:06:37.397 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.397 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:06:37.397 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.397 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:06:37.397 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841539+0000", 2024-09-20T19:06:37.397 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:37.397 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.397 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.397 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:06:37.397 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841539+0000", 2024-09-20T19:06:37.397 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:37.398 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.398 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.398 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:06:37.398 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841539+0000", 2024-09-20T19:06:37.398 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4499999999999997e-07 2024-09-20T19:06:37.398 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.398 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.398 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:06:37.398 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841542+0000", 2024-09-20T19:06:37.398 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.128e-06 2024-09-20T19:06:37.398 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.398 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.398 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:06:37.399 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841548+0000", 2024-09-20T19:06:37.399 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8220000000000003e-06 2024-09-20T19:06:37.399 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.399 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.399 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:06:37.399 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841553+0000", 2024-09-20T19:06:37.399 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.967e-06 2024-09-20T19:06:37.399 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.399 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.399 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:37.399 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841603+0000", 2024-09-20T19:06:37.399 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.9636999999999999e-05 2024-09-20T19:06:37.399 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.399 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.399 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:06:37.400 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841706+0000", 2024-09-20T19:06:37.400 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000103609 2024-09-20T19:06:37.400 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.400 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.400 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,3", 2024-09-20T19:06:37.400 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.841803+0000", 2024-09-20T19:06:37.400 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.6217000000000004e-05 2024-09-20T19:06:37.400 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.400 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.400 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:37.400 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.845333+0000", 2024-09-20T19:06:37.400 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0035301540000000002 2024-09-20T19:06:37.400 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.400 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.401 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:06:37.401 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.846476+0000", 2024-09-20T19:06:37.401 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0011436619999999999 2024-09-20T19:06:37.401 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.401 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.401 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:37.401 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.980293+0000", 2024-09-20T19:06:37.401 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13381654500000001 2024-09-20T19:06:37.401 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.401 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.401 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:06:37.401 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.980328+0000", 2024-09-20T19:06:37.401 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.5255e-05 2024-09-20T19:06:37.401 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.401 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.402 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:06:37.402 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.980392+0000", 2024-09-20T19:06:37.402 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.3782999999999996e-05 2024-09-20T19:06:37.402 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:37.402 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:06:37.402 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:37.402 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.402 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.402 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:119697 9.d 9:b589807c:::benchmark_data_smithi083_124552_object5550:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e287)", 2024-09-20T19:06:37.402 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:00:16.846849+0000", 2024-09-20T19:06:37.402 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 380.520790519, 2024-09-20T19:06:37.402 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13367747499999999, 2024-09-20T19:06:37.402 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:06:37.402 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:06:37.402 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:37.403 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:06:37.403 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:06:37.403 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:06:37.403 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 119697 2024-09-20T19:06:37.403 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.403 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:06:37.403 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.403 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:06:37.403 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.846849+0000", 2024-09-20T19:06:37.403 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:37.403 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.403 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.403 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:06:37.403 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.846849+0000", 2024-09-20T19:06:37.403 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:37.404 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.404 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.404 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:06:37.404 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.846850+0000", 2024-09-20T19:06:37.404 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:37.405 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5829999999999999e-06 2024-09-20T19:06:37.405 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.405 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.405 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:06:37.405 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.846856+0000", 2024-09-20T19:06:37.405 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6570000000000002e-06 2024-09-20T19:06:37.405 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.406 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.406 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:06:37.406 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.846867+0000", 2024-09-20T19:06:37.406 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1321999999999999e-05 2024-09-20T19:06:37.406 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.406 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.406 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:06:37.406 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.846875+0000", 2024-09-20T19:06:37.406 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.9270000000000005e-06 2024-09-20T19:06:37.406 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.406 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.406 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:37.406 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.846926+0000", 2024-09-20T19:06:37.407 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0759e-05 2024-09-20T19:06:37.407 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.407 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.407 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:06:37.407 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.847050+0000", 2024-09-20T19:06:37.407 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012434300000000001 2024-09-20T19:06:37.407 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.407 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.407 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,5", 2024-09-20T19:06:37.407 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.847172+0000", 2024-09-20T19:06:37.407 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000121586 2024-09-20T19:06:37.407 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.407 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.408 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:06:37.408 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.849552+0000", 2024-09-20T19:06:37.408 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0023799810000000002 2024-09-20T19:06:37.408 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.408 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.408 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:37.408 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.849792+0000", 2024-09-20T19:06:37.408 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000239965 2024-09-20T19:06:37.408 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.408 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.408 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:37.408 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.980470+0000", 2024-09-20T19:06:37.408 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13067801600000001 2024-09-20T19:06:37.409 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.409 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.409 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:06:37.409 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.980505+0000", 2024-09-20T19:06:37.409 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.5627000000000003e-05 2024-09-20T19:06:37.409 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.409 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.409 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:06:37.409 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:16.980526+0000", 2024-09-20T19:06:37.409 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0709000000000001e-05 2024-09-20T19:06:37.409 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:37.409 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:06:37.409 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:37.409 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.409 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.409 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14460.0:2702 5.9 5:93e5b521:::notify.7:head [watch reconnect cookie 94056120174720 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e292)", 2024-09-20T19:06:37.410 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:00:35.178267+0000", 2024-09-20T19:06:37.410 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 362.18937190899999, 2024-09-20T19:06:37.410 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.011898462, 2024-09-20T19:06:37.410 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:06:37.410 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:06:37.410 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-09-20T19:06:37.410 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:06:37.410 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14460", 2024-09-20T19:06:37.410 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1555339426", 2024-09-20T19:06:37.410 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 2702 2024-09-20T19:06:37.410 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.410 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:06:37.410 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.410 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:06:37.411 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.178267+0000", 2024-09-20T19:06:37.411 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:37.411 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.411 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.411 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:06:37.411 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.178267+0000", 2024-09-20T19:06:37.411 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:37.411 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.411 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.411 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:06:37.411 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.178269+0000", 2024-09-20T19:06:37.411 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.703e-06 2024-09-20T19:06:37.411 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.412 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.412 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:06:37.412 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.178274+0000", 2024-09-20T19:06:37.412 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3909999999999997e-06 2024-09-20T19:06:37.412 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.412 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.412 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:06:37.412 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.178292+0000", 2024-09-20T19:06:37.412 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7601e-05 2024-09-20T19:06:37.412 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.412 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.412 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:06:37.412 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.178301+0000", 2024-09-20T19:06:37.412 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.6950000000000006e-06 2024-09-20T19:06:37.413 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.413 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.413 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:37.413 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.178364+0000", 2024-09-20T19:06:37.413 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.3304000000000002e-05 2024-09-20T19:06:37.413 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.413 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.413 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op must wait for map", 2024-09-20T19:06:37.413 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.178370+0000", 2024-09-20T19:06:37.413 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2979999999999997e-06 2024-09-20T19:06:37.413 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.413 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.413 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:37.413 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.191158+0000", 2024-09-20T19:06:37.413 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.012787822000000001 2024-09-20T19:06:37.414 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.414 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.414 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:06:37.414 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.191166+0000", 2024-09-20T19:06:37.414 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.9959999999999993e-06 2024-09-20T19:06:37.414 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.414 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.414 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:37.414 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.188716+0000", 2024-09-20T19:06:37.414 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.997550198 2024-09-20T19:06:37.414 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.414 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.414 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:06:37.415 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.188848+0000", 2024-09-20T19:06:37.415 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013202400000000001 2024-09-20T19:06:37.415 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.415 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.415 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:06:37.415 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.190166+0000", 2024-09-20T19:06:37.415 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0013174300000000001 2024-09-20T19:06:37.415 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:37.415 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:06:37.415 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:37.415 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.415 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.415 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:142953 9.e 9:785c93fe:::benchmark_data_smithi083_124552_object28806:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e292)", 2024-09-20T19:06:37.415 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:00:35.227055+0000", 2024-09-20T19:06:37.416 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 362.14058422800002, 2024-09-20T19:06:37.416 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.985725662, 2024-09-20T19:06:37.416 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:06:37.416 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:06:37.416 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:37.416 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:06:37.416 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:06:37.416 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:06:37.416 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 142953 2024-09-20T19:06:37.416 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.416 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:06:37.416 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.416 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:06:37.416 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.227055+0000", 2024-09-20T19:06:37.416 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:37.417 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.417 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.417 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:06:37.417 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.227055+0000", 2024-09-20T19:06:37.417 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:37.417 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.417 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.417 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:06:37.417 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.227056+0000", 2024-09-20T19:06:37.417 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1170000000000001e-06 2024-09-20T19:06:37.417 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.417 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.417 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:06:37.417 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.227060+0000", 2024-09-20T19:06:37.417 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.1320000000000004e-06 2024-09-20T19:06:37.418 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.418 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.418 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:06:37.418 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.227070+0000", 2024-09-20T19:06:37.418 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.8179999999999999e-06 2024-09-20T19:06:37.418 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.418 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.418 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:06:37.418 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.227075+0000", 2024-09-20T19:06:37.418 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5289999999999999e-06 2024-09-20T19:06:37.418 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.418 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.418 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:37.418 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.227131+0000", 2024-09-20T19:06:37.419 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5482999999999997e-05 2024-09-20T19:06:37.419 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.419 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.419 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:06:37.419 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.227140+0000", 2024-09-20T19:06:37.419 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.4159999999999993e-06 2024-09-20T19:06:37.419 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.419 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.419 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:37.419 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.205610+0000", 2024-09-20T19:06:37.419 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.97846971500000002 2024-09-20T19:06:37.419 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.419 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.419 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:06:37.420 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.205744+0000", 2024-09-20T19:06:37.420 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013422099999999999 2024-09-20T19:06:37.420 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.420 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.420 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1", 2024-09-20T19:06:37.420 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.205878+0000", 2024-09-20T19:06:37.420 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000134108 2024-09-20T19:06:37.420 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.420 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.420 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:37.420 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.212569+0000", 2024-09-20T19:06:37.420 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0066901629999999998 2024-09-20T19:06:37.420 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.421 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.421 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:06:37.421 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.212643+0000", 2024-09-20T19:06:37.421 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.4120000000000002e-05 2024-09-20T19:06:37.421 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.421 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.421 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:06:37.421 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.212692+0000", 2024-09-20T19:06:37.421 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.922e-05 2024-09-20T19:06:37.421 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.421 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.421 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:06:37.421 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.212781+0000", 2024-09-20T19:06:37.421 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.8620000000000002e-05 2024-09-20T19:06:37.421 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:37.422 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:06:37.422 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:37.422 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.422 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.422 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:142961 9.e 9:70c9c31e:::benchmark_data_smithi083_124552_object28814:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e292)", 2024-09-20T19:06:37.422 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:00:35.233023+0000", 2024-09-20T19:06:37.422 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 362.13461579300002, 2024-09-20T19:06:37.422 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.98462836899999995, 2024-09-20T19:06:37.422 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:06:37.422 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:06:37.422 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:37.422 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:06:37.422 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:06:37.422 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:06:37.423 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 142961 2024-09-20T19:06:37.423 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.423 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:06:37.423 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.423 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:06:37.423 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.233023+0000", 2024-09-20T19:06:37.423 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:37.423 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.423 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.423 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:06:37.423 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.233023+0000", 2024-09-20T19:06:37.423 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:37.423 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.423 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.424 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:06:37.424 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.233025+0000", 2024-09-20T19:06:37.424 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.252e-06 2024-09-20T19:06:37.424 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.424 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.424 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:06:37.424 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.233029+0000", 2024-09-20T19:06:37.424 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.442e-06 2024-09-20T19:06:37.424 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.424 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.424 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:06:37.424 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.233048+0000", 2024-09-20T19:06:37.424 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9096000000000001e-05 2024-09-20T19:06:37.425 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.425 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.425 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:06:37.425 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.233055+0000", 2024-09-20T19:06:37.425 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.0729999999999999e-06 2024-09-20T19:06:37.425 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.425 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.425 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:37.425 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.233109+0000", 2024-09-20T19:06:37.425 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4049999999999999e-05 2024-09-20T19:06:37.425 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.425 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.425 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:06:37.425 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.233117+0000", 2024-09-20T19:06:37.426 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.379e-06 2024-09-20T19:06:37.426 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.426 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.426 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:37.426 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.208136+0000", 2024-09-20T19:06:37.426 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.97501899599999997 2024-09-20T19:06:37.426 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.426 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.426 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:06:37.426 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.208257+0000", 2024-09-20T19:06:37.426 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012119 2024-09-20T19:06:37.426 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.426 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.427 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1", 2024-09-20T19:06:37.427 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.208383+0000", 2024-09-20T19:06:37.427 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000126484 2024-09-20T19:06:37.427 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.427 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.427 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:37.427 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.212629+0000", 2024-09-20T19:06:37.427 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0042453329999999996 2024-09-20T19:06:37.427 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.427 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.427 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:06:37.427 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.217490+0000", 2024-09-20T19:06:37.427 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0048616470000000002 2024-09-20T19:06:37.427 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.427 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.428 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:06:37.428 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.217551+0000", 2024-09-20T19:06:37.428 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0797000000000001e-05 2024-09-20T19:06:37.428 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.428 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.428 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:06:37.428 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.217652+0000", 2024-09-20T19:06:37.428 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00010063 2024-09-20T19:06:37.428 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:37.428 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:06:37.428 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:37.428 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.428 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.428 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:142969 9.e 9:74a6f6aa:::benchmark_data_smithi083_124552_object28822:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e292)", 2024-09-20T19:06:37.428 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:00:35.238908+0000", 2024-09-20T19:06:37.429 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 362.12873116999998, 2024-09-20T19:06:37.429 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.98019590400000001, 2024-09-20T19:06:37.429 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:06:37.429 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:06:37.429 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:37.429 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:06:37.429 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:06:37.429 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:06:37.429 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 142969 2024-09-20T19:06:37.429 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.429 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:06:37.429 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.429 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:06:37.429 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.238908+0000", 2024-09-20T19:06:37.429 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:37.429 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.430 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.430 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:06:37.430 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.238908+0000", 2024-09-20T19:06:37.430 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:37.430 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.430 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.430 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:06:37.430 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.238909+0000", 2024-09-20T19:06:37.430 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.7800000000000001e-07 2024-09-20T19:06:37.430 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.430 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.430 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:06:37.430 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.238912+0000", 2024-09-20T19:06:37.430 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.3919999999999999e-06 2024-09-20T19:06:37.431 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.431 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.431 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:06:37.431 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.238919+0000", 2024-09-20T19:06:37.431 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.0679999999999999e-06 2024-09-20T19:06:37.431 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.431 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.431 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:06:37.431 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.238925+0000", 2024-09-20T19:06:37.431 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.1369999999999999e-06 2024-09-20T19:06:37.431 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.431 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.431 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:37.431 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.238981+0000", 2024-09-20T19:06:37.432 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6067999999999999e-05 2024-09-20T19:06:37.432 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.432 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.432 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:06:37.432 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.238990+0000", 2024-09-20T19:06:37.432 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.2600000000000005e-06 2024-09-20T19:06:37.432 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.432 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.432 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:37.432 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.210313+0000", 2024-09-20T19:06:37.432 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.97132380500000004 2024-09-20T19:06:37.432 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.432 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.432 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:06:37.433 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.210428+0000", 2024-09-20T19:06:37.433 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000114369 2024-09-20T19:06:37.433 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.433 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.433 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1", 2024-09-20T19:06:37.433 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.210549+0000", 2024-09-20T19:06:37.433 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000121558 2024-09-20T19:06:37.433 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.433 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.433 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:06:37.433 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.217664+0000", 2024-09-20T19:06:37.433 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0071143090000000001 2024-09-20T19:06:37.433 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.433 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.434 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:37.434 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.218937+0000", 2024-09-20T19:06:37.434 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0012730599999999999 2024-09-20T19:06:37.434 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.434 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.434 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:06:37.434 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.218991+0000", 2024-09-20T19:06:37.434 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4722999999999998e-05 2024-09-20T19:06:37.434 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.434 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.434 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:06:37.434 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.219104+0000", 2024-09-20T19:06:37.434 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000112477 2024-09-20T19:06:37.435 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:37.435 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:06:37.435 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:37.435 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.435 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.435 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:142981 9.e 9:7a27425f:::benchmark_data_smithi083_124552_object28834:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e292)", 2024-09-20T19:06:37.435 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:00:35.254008+0000", 2024-09-20T19:06:37.435 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 362.11363067899998, 2024-09-20T19:06:37.435 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.966262548, 2024-09-20T19:06:37.435 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:06:37.435 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:06:37.435 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:37.435 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:06:37.436 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:06:37.436 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:06:37.436 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 142981 2024-09-20T19:06:37.436 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.436 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:06:37.436 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.436 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:06:37.436 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.254008+0000", 2024-09-20T19:06:37.436 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:37.436 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.436 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.436 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:06:37.436 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.254008+0000", 2024-09-20T19:06:37.436 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:37.437 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.437 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.437 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:06:37.437 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.254010+0000", 2024-09-20T19:06:37.437 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.443e-06 2024-09-20T19:06:37.437 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.437 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.437 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:06:37.437 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.254014+0000", 2024-09-20T19:06:37.437 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9260000000000002e-06 2024-09-20T19:06:37.437 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.437 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.438 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:06:37.438 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.254024+0000", 2024-09-20T19:06:37.438 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.9839999999999996e-06 2024-09-20T19:06:37.438 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.438 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.438 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:06:37.438 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.254029+0000", 2024-09-20T19:06:37.438 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6869999999999999e-06 2024-09-20T19:06:37.438 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.438 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.438 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:37.438 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.254089+0000", 2024-09-20T19:06:37.438 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9435000000000003e-05 2024-09-20T19:06:37.438 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.439 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.439 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:06:37.439 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.254094+0000", 2024-09-20T19:06:37.439 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4399999999999996e-06 2024-09-20T19:06:37.439 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.439 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.439 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:37.439 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.212819+0000", 2024-09-20T19:06:37.439 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.95872477300000003 2024-09-20T19:06:37.439 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.439 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.439 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:06:37.439 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.212953+0000", 2024-09-20T19:06:37.440 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013434699999999999 2024-09-20T19:06:37.440 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.440 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.440 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1", 2024-09-20T19:06:37.440 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.213056+0000", 2024-09-20T19:06:37.440 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000102675 2024-09-20T19:06:37.440 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.440 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.440 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:06:37.440 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.217670+0000", 2024-09-20T19:06:37.440 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0046140879999999997 2024-09-20T19:06:37.440 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.440 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.441 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:37.441 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.219977+0000", 2024-09-20T19:06:37.441 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0023067230000000001 2024-09-20T19:06:37.441 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.441 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.441 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:06:37.441 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.220160+0000", 2024-09-20T19:06:37.441 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00018349800000000001 2024-09-20T19:06:37.441 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.441 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.441 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:06:37.441 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.220271+0000", 2024-09-20T19:06:37.441 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000110529 2024-09-20T19:06:37.442 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:37.442 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:06:37.442 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:37.442 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.442 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.442 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25484.0:142991 9.e 9:75f432de:::benchmark_data_smithi083_124552_object28844:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e292)", 2024-09-20T19:06:37.442 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:00:35.271404+0000", 2024-09-20T19:06:37.442 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 362.09623531800003, 2024-09-20T19:06:37.442 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.94906818900000001, 2024-09-20T19:06:37.442 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:06:37.442 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:06:37.442 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:37.442 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:06:37.443 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25484", 2024-09-20T19:06:37.443 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:06:37.443 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 142991 2024-09-20T19:06:37.443 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.443 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:06:37.443 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.443 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:06:37.443 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.271404+0000", 2024-09-20T19:06:37.443 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:37.443 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.443 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.443 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:06:37.443 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.271404+0000", 2024-09-20T19:06:37.443 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:37.444 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.444 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.444 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:06:37.444 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.271405+0000", 2024-09-20T19:06:37.444 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.569999999999999e-07 2024-09-20T19:06:37.444 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.444 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.444 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:06:37.444 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.271408+0000", 2024-09-20T19:06:37.444 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.647e-06 2024-09-20T19:06:37.444 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.444 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.444 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:06:37.444 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.271418+0000", 2024-09-20T19:06:37.445 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.6360000000000006e-06 2024-09-20T19:06:37.445 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.445 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.445 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:06:37.445 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.271424+0000", 2024-09-20T19:06:37.445 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6080000000000003e-06 2024-09-20T19:06:37.445 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.445 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.445 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:37.445 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.271468+0000", 2024-09-20T19:06:37.445 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3958999999999997e-05 2024-09-20T19:06:37.445 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.445 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.446 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:06:37.446 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.271473+0000", 2024-09-20T19:06:37.446 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0710000000000002e-06 2024-09-20T19:06:37.446 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.446 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.446 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:37.446 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.214908+0000", 2024-09-20T19:06:37.446 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.94343566999999995 2024-09-20T19:06:37.446 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.446 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.446 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:06:37.446 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.215018+0000", 2024-09-20T19:06:37.446 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000109641 2024-09-20T19:06:37.446 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.447 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.448 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1", 2024-09-20T19:06:37.448 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.215110+0000", 2024-09-20T19:06:37.448 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.2324000000000006e-05 2024-09-20T19:06:37.448 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.448 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.448 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:06:37.448 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.220290+0000", 2024-09-20T19:06:37.448 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0051801019999999998 2024-09-20T19:06:37.448 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.448 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.449 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:37.449 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.220324+0000", 2024-09-20T19:06:37.449 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4020000000000003e-05 2024-09-20T19:06:37.449 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.449 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.449 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:06:37.449 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.220389+0000", 2024-09-20T19:06:37.449 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5048999999999993e-05 2024-09-20T19:06:37.449 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.449 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.449 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:06:37.449 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.220472+0000", 2024-09-20T19:06:37.449 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.2504999999999997e-05 2024-09-20T19:06:37.449 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:37.449 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:06:37.450 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:37.450 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.450 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.450 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:147604 10.c 10:335682df:::benchmark_data_smithi083_148100_object30742:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e376)", 2024-09-20T19:06:37.450 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:04:28.093070+0000", 2024-09-20T19:06:37.450 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 129.27456916, 2024-09-20T19:06:37.450 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.31573811600000001, 2024-09-20T19:06:37.450 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:06:37.450 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:06:37.450 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:37.450 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:06:37.450 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:06:37.451 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:06:37.451 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 147604 2024-09-20T19:06:37.451 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.451 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:06:37.451 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.451 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:06:37.451 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.093070+0000", 2024-09-20T19:06:37.451 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:37.451 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.451 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.451 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:06:37.451 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.093070+0000", 2024-09-20T19:06:37.451 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:37.451 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.452 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.452 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:06:37.452 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.093071+0000", 2024-09-20T19:06:37.452 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.53e-06 2024-09-20T19:06:37.452 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.452 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.452 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:06:37.452 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.093076+0000", 2024-09-20T19:06:37.452 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.4499999999999997e-06 2024-09-20T19:06:37.452 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.452 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.452 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:06:37.452 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.093085+0000", 2024-09-20T19:06:37.452 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.7299999999999994e-06 2024-09-20T19:06:37.453 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.453 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.453 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:06:37.453 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.093091+0000", 2024-09-20T19:06:37.453 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8969999999999996e-06 2024-09-20T19:06:37.453 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.453 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.453 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:37.453 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.093143+0000", 2024-09-20T19:06:37.453 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.2042000000000001e-05 2024-09-20T19:06:37.453 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.453 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.453 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:06:37.453 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.093255+0000", 2024-09-20T19:06:37.454 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000111956 2024-09-20T19:06:37.454 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.454 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.454 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,3,4", 2024-09-20T19:06:37.454 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.093369+0000", 2024-09-20T19:06:37.454 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000114407 2024-09-20T19:06:37.454 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.454 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.454 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:06:37.454 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.096935+0000", 2024-09-20T19:06:37.454 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0035662459999999999 2024-09-20T19:06:37.454 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.454 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.454 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:37.455 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.098150+0000", 2024-09-20T19:06:37.455 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001215278 2024-09-20T19:06:37.455 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.455 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.455 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:37.455 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.100206+0000", 2024-09-20T19:06:37.455 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0020550389999999998 2024-09-20T19:06:37.455 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.455 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.455 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:37.455 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.408586+0000", 2024-09-20T19:06:37.455 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30838020599999999 2024-09-20T19:06:37.455 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.456 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.456 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:06:37.456 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.408695+0000", 2024-09-20T19:06:37.456 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000109114 2024-09-20T19:06:37.456 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.456 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.456 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:06:37.456 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.408808+0000", 2024-09-20T19:06:37.456 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011322100000000001 2024-09-20T19:06:37.456 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:37.456 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:06:37.456 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:37.456 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.456 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.457 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:147615 10.c 10:3d45ec03:::benchmark_data_smithi083_148100_object30753:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e376)", 2024-09-20T19:06:37.457 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:04:28.104908+0000", 2024-09-20T19:06:37.457 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 129.262731038, 2024-09-20T19:06:37.457 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.309806101, 2024-09-20T19:06:37.457 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:06:37.457 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:06:37.457 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:37.457 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:06:37.457 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:06:37.457 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:06:37.457 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 147615 2024-09-20T19:06:37.457 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.457 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:06:37.457 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.457 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:06:37.457 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.104908+0000", 2024-09-20T19:06:37.458 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:37.458 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.458 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.458 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:06:37.458 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.104908+0000", 2024-09-20T19:06:37.458 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:37.458 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.458 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.458 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:06:37.458 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.104917+0000", 2024-09-20T19:06:37.458 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.7960000000000001e-06 2024-09-20T19:06:37.458 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.458 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.458 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:06:37.459 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.104923+0000", 2024-09-20T19:06:37.459 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0889999999999996e-06 2024-09-20T19:06:37.459 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.459 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.459 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:06:37.459 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.104937+0000", 2024-09-20T19:06:37.459 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4161999999999999e-05 2024-09-20T19:06:37.459 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.459 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.459 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:06:37.459 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.104945+0000", 2024-09-20T19:06:37.459 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.2479999999999996e-06 2024-09-20T19:06:37.459 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.459 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.460 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:37.460 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.105011+0000", 2024-09-20T19:06:37.460 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5505000000000004e-05 2024-09-20T19:06:37.460 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.460 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.460 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:06:37.460 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.406010+0000", 2024-09-20T19:06:37.460 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30099882 2024-09-20T19:06:37.460 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.460 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.460 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,3,4", 2024-09-20T19:06:37.460 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.406176+0000", 2024-09-20T19:06:37.460 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00016597899999999999 2024-09-20T19:06:37.460 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.461 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.461 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:37.461 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.411469+0000", 2024-09-20T19:06:37.461 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.005293145 2024-09-20T19:06:37.461 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.461 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.461 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:37.461 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.411536+0000", 2024-09-20T19:06:37.461 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.6879999999999997e-05 2024-09-20T19:06:37.461 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.461 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.461 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:06:37.461 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.411581+0000", 2024-09-20T19:06:37.461 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.5354e-05 2024-09-20T19:06:37.461 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.462 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.462 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:37.462 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.414564+0000", 2024-09-20T19:06:37.462 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0029832230000000001 2024-09-20T19:06:37.462 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.462 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.462 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:06:37.462 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.414615+0000", 2024-09-20T19:06:37.462 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0874000000000001e-05 2024-09-20T19:06:37.462 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.462 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.462 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:06:37.462 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.414714+0000", 2024-09-20T19:06:37.463 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.9025999999999997e-05 2024-09-20T19:06:37.463 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:37.463 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:06:37.463 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:37.463 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.463 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.463 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25775.0:147612 10.1 e376/375 10:845b71ed:::benchmark_data_smithi083_148100_object30750:head v 376'9140, mlcod=376'9137)", 2024-09-20T19:06:37.463 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:04:28.105562+0000", 2024-09-20T19:06:37.463 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 129.26207736000001, 2024-09-20T19:06:37.463 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.31529038700000001, 2024-09-20T19:06:37.463 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:06:37.463 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:06:37.463 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:37.463 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:06:37.463 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.464 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:06:37.464 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.105562+0000", 2024-09-20T19:06:37.464 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:37.464 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.464 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.464 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:06:37.464 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.105562+0000", 2024-09-20T19:06:37.464 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:37.464 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.464 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.464 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:06:37.464 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.105563+0000", 2024-09-20T19:06:37.464 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.6000000000000002e-07 2024-09-20T19:06:37.465 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.465 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.465 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:06:37.465 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.105568+0000", 2024-09-20T19:06:37.465 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.2129999999999999e-06 2024-09-20T19:06:37.465 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.465 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.465 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:06:37.465 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.405775+0000", 2024-09-20T19:06:37.465 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.300207479 2024-09-20T19:06:37.465 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.465 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.465 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:06:37.466 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.405792+0000", 2024-09-20T19:06:37.466 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6310000000000001e-05 2024-09-20T19:06:37.466 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.466 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.466 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:37.466 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.406175+0000", 2024-09-20T19:06:37.466 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00038306899999999999 2024-09-20T19:06:37.466 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.466 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.466 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:06:37.466 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.406196+0000", 2024-09-20T19:06:37.466 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1667000000000001e-05 2024-09-20T19:06:37.466 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.466 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.466 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:06:37.467 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.420814+0000", 2024-09-20T19:06:37.467 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.014617170000000001 2024-09-20T19:06:37.467 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.467 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.467 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:06:37.467 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.420852+0000", 2024-09-20T19:06:37.467 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.8618999999999997e-05 2024-09-20T19:06:37.467 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:37.467 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:06:37.467 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:37.467 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.467 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.467 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.25775.0:147626 10.9 e376/325 10:98e2f0b0:::benchmark_data_smithi083_148100_object30764:head v 376'9323, mlcod=376'9321)", 2024-09-20T19:06:37.468 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:04:28.118980+0000", 2024-09-20T19:06:37.468 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 129.24865914899999, 2024-09-20T19:06:37.468 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.400823608, 2024-09-20T19:06:37.468 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:06:37.468 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:06:37.468 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:37.468 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:06:37.468 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.468 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:06:37.468 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.118980+0000", 2024-09-20T19:06:37.468 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:37.468 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.468 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.468 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:06:37.469 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.118980+0000", 2024-09-20T19:06:37.469 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:37.469 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.469 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.469 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:06:37.469 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.118981+0000", 2024-09-20T19:06:37.469 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.207e-06 2024-09-20T19:06:37.469 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.469 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.469 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:06:37.469 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.118989+0000", 2024-09-20T19:06:37.469 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5240000000000003e-06 2024-09-20T19:06:37.469 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.469 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.470 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:06:37.470 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.406133+0000", 2024-09-20T19:06:37.470 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.28714446799999999 2024-09-20T19:06:37.470 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.470 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.470 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:06:37.470 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.406141+0000", 2024-09-20T19:06:37.470 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.6249999999999998e-06 2024-09-20T19:06:37.470 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.470 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.470 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:37.470 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.416048+0000", 2024-09-20T19:06:37.470 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0099072339999999991 2024-09-20T19:06:37.471 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.471 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.471 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:06:37.471 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.416065+0000", 2024-09-20T19:06:37.471 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7399999999999999e-05 2024-09-20T19:06:37.471 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.471 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.471 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:06:37.471 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.519526+0000", 2024-09-20T19:06:37.471 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10346095700000001 2024-09-20T19:06:37.471 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.471 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.471 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:06:37.471 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.519804+0000", 2024-09-20T19:06:37.472 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00027719300000000001 2024-09-20T19:06:37.472 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:37.472 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:06:37.472 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:37.472 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.472 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.472 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.24404.0:870 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch (1) in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e376)", 2024-09-20T19:06:37.472 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:04:28.123191+0000", 2024-09-20T19:06:37.472 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 129.24444829500001, 2024-09-20T19:06:37.472 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.28388144100000001, 2024-09-20T19:06:37.472 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:06:37.472 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:06:37.472 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-09-20T19:06:37.472 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:06:37.472 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.24404", 2024-09-20T19:06:37.473 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/3429674210", 2024-09-20T19:06:37.473 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 870 2024-09-20T19:06:37.473 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.473 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:06:37.473 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.473 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:06:37.473 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.123191+0000", 2024-09-20T19:06:37.473 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:37.473 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.473 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.473 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:06:37.473 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.123191+0000", 2024-09-20T19:06:37.473 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:37.473 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.474 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.474 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:06:37.474 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.123193+0000", 2024-09-20T19:06:37.474 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1900000000000002e-06 2024-09-20T19:06:37.474 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.474 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.474 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:06:37.474 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.123202+0000", 2024-09-20T19:06:37.474 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.8049999999999996e-06 2024-09-20T19:06:37.474 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.474 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.474 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:06:37.474 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.405816+0000", 2024-09-20T19:06:37.475 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.28261431100000001 2024-09-20T19:06:37.475 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.475 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.475 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:06:37.475 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.405828+0000", 2024-09-20T19:06:37.475 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1911e-05 2024-09-20T19:06:37.475 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.475 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.475 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:37.475 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.405891+0000", 2024-09-20T19:06:37.475 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.3318999999999995e-05 2024-09-20T19:06:37.475 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.475 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.476 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:06:37.476 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.405977+0000", 2024-09-20T19:06:37.476 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.5560000000000001e-05 2024-09-20T19:06:37.476 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.476 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.476 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:06:37.476 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.407072+0000", 2024-09-20T19:06:37.476 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001095345 2024-09-20T19:06:37.476 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:37.476 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:06:37.476 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:37.476 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.476 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.476 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:222835 10.c 10:3dc02ffd:::benchmark_data_smithi083_148100_object105973:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e407)", 2024-09-20T19:06:37.477 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:47.959106+0000", 2024-09-20T19:06:37.477 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 49.408532844, 2024-09-20T19:06:37.477 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.14504133399999999, 2024-09-20T19:06:37.477 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:06:37.477 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:06:37.477 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:37.477 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:06:37.477 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:06:37.477 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:06:37.477 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 222835 2024-09-20T19:06:37.477 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.477 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:06:37.477 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.477 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:06:37.478 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:47.959106+0000", 2024-09-20T19:06:37.478 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:37.478 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.478 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.478 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:06:37.478 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:47.959106+0000", 2024-09-20T19:06:37.478 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:37.478 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.478 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.478 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:06:37.478 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:47.959108+0000", 2024-09-20T19:06:37.478 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8300000000000001e-06 2024-09-20T19:06:37.478 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.478 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.479 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:06:37.479 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:47.959113+0000", 2024-09-20T19:06:37.479 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3380000000000004e-06 2024-09-20T19:06:37.479 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.479 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.479 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:06:37.479 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:47.959126+0000", 2024-09-20T19:06:37.479 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2293000000000001e-05 2024-09-20T19:06:37.479 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.479 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.479 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:06:37.479 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:47.959134+0000", 2024-09-20T19:06:37.479 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.9999999999999996e-06 2024-09-20T19:06:37.479 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.480 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.480 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:37.480 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:47.965994+0000", 2024-09-20T19:06:37.480 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0068606309999999998 2024-09-20T19:06:37.480 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.480 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.480 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:06:37.480 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:47.966115+0000", 2024-09-20T19:06:37.480 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000120665 2024-09-20T19:06:37.480 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.480 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.480 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,4", 2024-09-20T19:06:37.480 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:47.966237+0000", 2024-09-20T19:06:37.480 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000121758 2024-09-20T19:06:37.480 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.481 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.481 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:06:37.481 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:47.971799+0000", 2024-09-20T19:06:37.481 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0055626460000000001 2024-09-20T19:06:37.481 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.481 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.481 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:37.481 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:47.975021+0000", 2024-09-20T19:06:37.481 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0032211399999999999 2024-09-20T19:06:37.481 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.481 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.481 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:37.481 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:48.104005+0000", 2024-09-20T19:06:37.481 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.128984456 2024-09-20T19:06:37.481 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.482 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.482 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:06:37.482 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:48.104087+0000", 2024-09-20T19:06:37.482 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.1729000000000003e-05 2024-09-20T19:06:37.482 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.482 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.482 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:06:37.482 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:48.104148+0000", 2024-09-20T19:06:37.482 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0847999999999997e-05 2024-09-20T19:06:37.482 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:37.482 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:06:37.482 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:37.482 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.482 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.483 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.26264.0:28743 11.f 11:f0384e23:::benchmark_data_smithi083_177491_object28742:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e433)", 2024-09-20T19:06:37.483 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:06:34.749312+0000", 2024-09-20T19:06:37.483 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 2.6183266029999999, 2024-09-20T19:06:37.483 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.096600372000000004, 2024-09-20T19:06:37.483 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:06:37.483 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:06:37.483 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:37.483 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:06:37.483 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.26264", 2024-09-20T19:06:37.483 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2608780316", 2024-09-20T19:06:37.483 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 28743 2024-09-20T19:06:37.483 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.483 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:06:37.483 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.483 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:06:37.484 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:34.749312+0000", 2024-09-20T19:06:37.484 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:37.484 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.484 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.484 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:06:37.484 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:34.749312+0000", 2024-09-20T19:06:37.484 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:37.484 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.484 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.484 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:06:37.484 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:34.749314+0000", 2024-09-20T19:06:37.484 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6270000000000001e-06 2024-09-20T19:06:37.484 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.484 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.485 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:06:37.485 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:34.749393+0000", 2024-09-20T19:06:37.485 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.8799000000000002e-05 2024-09-20T19:06:37.485 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.485 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.485 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:06:37.485 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:34.749403+0000", 2024-09-20T19:06:37.485 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0147e-05 2024-09-20T19:06:37.485 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.485 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.485 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:06:37.485 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:34.749410+0000", 2024-09-20T19:06:37.485 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.3339999999999996e-06 2024-09-20T19:06:37.485 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.485 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.486 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:37.486 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:34.749462+0000", 2024-09-20T19:06:37.486 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.1761999999999997e-05 2024-09-20T19:06:37.486 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.486 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.486 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:06:37.486 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:34.749546+0000", 2024-09-20T19:06:37.486 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.3823e-05 2024-09-20T19:06:37.486 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.486 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.486 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:06:37.486 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:34.749621+0000", 2024-09-20T19:06:37.486 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5339999999999994e-05 2024-09-20T19:06:37.486 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.487 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.487 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:06:37.487 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:34.750594+0000", 2024-09-20T19:06:37.487 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00097224599999999996 2024-09-20T19:06:37.487 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.487 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.487 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:37.487 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:34.753573+0000", 2024-09-20T19:06:37.487 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0029789249999999999 2024-09-20T19:06:37.487 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.487 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.487 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:37.487 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:34.845736+0000", 2024-09-20T19:06:37.487 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.092163253000000001 2024-09-20T19:06:37.487 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.488 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.488 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:06:37.488 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:34.845791+0000", 2024-09-20T19:06:37.488 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5392000000000001e-05 2024-09-20T19:06:37.488 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.488 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.488 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:06:37.488 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:34.845913+0000", 2024-09-20T19:06:37.488 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000121724 2024-09-20T19:06:37.488 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:37.488 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:06:37.488 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:37.488 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.488 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.488 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.26264.0:28748 11.f 11:f3e28581:::benchmark_data_smithi083_177491_object28747:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e433)", 2024-09-20T19:06:37.489 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:06:34.752440+0000", 2024-09-20T19:06:37.489 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 2.6151990469999999, 2024-09-20T19:06:37.489 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.093741174999999996, 2024-09-20T19:06:37.489 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:06:37.489 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:06:37.489 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:37.489 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:06:37.489 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.26264", 2024-09-20T19:06:37.489 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2608780316", 2024-09-20T19:06:37.489 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 28748 2024-09-20T19:06:37.489 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.489 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:06:37.489 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.489 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:06:37.490 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:34.752440+0000", 2024-09-20T19:06:37.490 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:37.490 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.490 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.490 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:06:37.490 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:34.752440+0000", 2024-09-20T19:06:37.490 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:37.490 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.490 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.490 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:06:37.490 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:34.752442+0000", 2024-09-20T19:06:37.490 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7489999999999999e-06 2024-09-20T19:06:37.490 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.490 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.491 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:06:37.491 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:34.752521+0000", 2024-09-20T19:06:37.491 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.9435000000000001e-05 2024-09-20T19:06:37.491 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.491 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.491 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:06:37.491 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:34.752531+0000", 2024-09-20T19:06:37.491 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.4439999999999997e-06 2024-09-20T19:06:37.491 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.491 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.491 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:06:37.491 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:34.752536+0000", 2024-09-20T19:06:37.491 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3929999999999999e-06 2024-09-20T19:06:37.491 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.492 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.492 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:37.492 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:34.752578+0000", 2024-09-20T19:06:37.492 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.1641999999999999e-05 2024-09-20T19:06:37.492 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.492 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.492 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:06:37.492 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:34.752663+0000", 2024-09-20T19:06:37.492 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.5582999999999997e-05 2024-09-20T19:06:37.492 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.492 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.492 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:06:37.492 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:34.752740+0000", 2024-09-20T19:06:37.492 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.6470000000000005e-05 2024-09-20T19:06:37.493 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.493 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.493 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:06:37.493 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:34.753663+0000", 2024-09-20T19:06:37.493 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00092321600000000003 2024-09-20T19:06:37.493 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.493 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.493 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:37.493 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:34.755435+0000", 2024-09-20T19:06:37.493 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0017722510000000001 2024-09-20T19:06:37.493 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.493 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.493 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:37.493 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:34.846095+0000", 2024-09-20T19:06:37.494 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.090659703999999994 2024-09-20T19:06:37.494 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.494 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.494 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:06:37.494 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:34.846143+0000", 2024-09-20T19:06:37.494 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7715e-05 2024-09-20T19:06:37.494 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.494 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.494 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:06:37.494 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:34.846181+0000", 2024-09-20T19:06:37.494 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.8572999999999998e-05 2024-09-20T19:06:37.494 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:37.494 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:06:37.494 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:37.494 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.495 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.495 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.26264.0:28753 11.f 11:f1756b08:::benchmark_data_smithi083_177491_object28752:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e433)", 2024-09-20T19:06:37.495 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:06:34.755455+0000", 2024-09-20T19:06:37.495 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 2.6121840240000003, 2024-09-20T19:06:37.495 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.093749387000000003, 2024-09-20T19:06:37.495 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:06:37.495 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:06:37.495 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:06:37.495 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:06:37.495 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.26264", 2024-09-20T19:06:37.495 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2608780316", 2024-09-20T19:06:37.495 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 28753 2024-09-20T19:06:37.495 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.495 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:06:37.496 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.496 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:06:37.496 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:34.755455+0000", 2024-09-20T19:06:37.496 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:37.496 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.496 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.496 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:06:37.496 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:34.755455+0000", 2024-09-20T19:06:37.496 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:37.496 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.496 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.496 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:06:37.496 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:34.755456+0000", 2024-09-20T19:06:37.496 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1209999999999999e-06 2024-09-20T19:06:37.496 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.496 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.497 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:06:37.497 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:34.755511+0000", 2024-09-20T19:06:37.497 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4648999999999998e-05 2024-09-20T19:06:37.497 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.497 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.497 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:06:37.497 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:34.755519+0000", 2024-09-20T19:06:37.497 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.4619999999999996e-06 2024-09-20T19:06:37.497 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.497 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.497 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:06:37.497 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:34.755526+0000", 2024-09-20T19:06:37.497 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.8279999999999997e-06 2024-09-20T19:06:37.497 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.498 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.498 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:37.498 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:34.755564+0000", 2024-09-20T19:06:37.498 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.7790000000000002e-05 2024-09-20T19:06:37.498 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.498 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.498 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:06:37.498 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:34.755643+0000", 2024-09-20T19:06:37.498 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.9367000000000001e-05 2024-09-20T19:06:37.498 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.498 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.498 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:06:37.498 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:34.844443+0000", 2024-09-20T19:06:37.498 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.088799439999999993 2024-09-20T19:06:37.499 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.499 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.499 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:06:37.499 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:34.846195+0000", 2024-09-20T19:06:37.499 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001752551 2024-09-20T19:06:37.499 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.499 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.499 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:37.499 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:34.849067+0000", 2024-09-20T19:06:37.499 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0028718289999999998 2024-09-20T19:06:37.499 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.499 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.499 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:37.499 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:34.849107+0000", 2024-09-20T19:06:37.499 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9872e-05 2024-09-20T19:06:37.500 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.500 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.500 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:06:37.500 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:34.849170+0000", 2024-09-20T19:06:37.500 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.3503000000000005e-05 2024-09-20T19:06:37.500 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:37.500 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:37.500 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:06:37.500 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:34.849204+0000", 2024-09-20T19:06:37.500 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.3974999999999999e-05 2024-09-20T19:06:37.500 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:37.500 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:06:37.500 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:37.500 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:37.500 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:06:37.500 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T19:06:37.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:37.502 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:37.502 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:37.511 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:06:37 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:06:37.603 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:37.640 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:37.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:37.641 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:37.742 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:37.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:37.780 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:37.780 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:37.881 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:37.915 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:37.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:37.916 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:38.017 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:38.041 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:38.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:38.042 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:38.113 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 15 16 34757 34741 144.73 112.562 0.0123587 0.00690352 2024-09-20T19:06:38.143 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:06:38.176 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:38.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:38.177 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:06:38.278 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:38.324 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:38.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:38.326 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:38.341 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:38 smithi142 ceph-mon[32508]: pgmap v853: 148 pgs: 148 active+clean; 1.9 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 1.6 KiB/s rd, 170 MiB/s wr, 2.73k op/s 2024-09-20T19:06:38.341 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:38 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:06:38.380 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-20T19:06:38.427 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:38.450 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:38.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:38.451 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:38.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:38 smithi083 ceph-mon[29421]: pgmap v853: 148 pgs: 148 active+clean; 1.9 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 1.6 KiB/s rd, 170 MiB/s wr, 2.73k op/s 2024-09-20T19:06:38.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:38 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:06:38.542 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:38 smithi083 ceph-mon[36260]: pgmap v853: 148 pgs: 148 active+clean; 1.9 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 1.6 KiB/s rd, 170 MiB/s wr, 2.73k op/s 2024-09-20T19:06:38.542 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:38 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:06:38.552 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:38.590 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:38.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:38.591 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:38.692 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:38.710 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:06:38.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:38.715 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:38.715 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:38.816 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:38.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:38.852 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:38.852 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:38.953 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:38.978 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:38.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:38.979 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:39.080 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:39.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:39.102 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:39.102 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:39.113 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 16 16 36036 36020 140.681 79.9375 0.0021047 0.00704487 2024-09-20T19:06:39.202 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:39.226 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:39.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:39.227 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:39.327 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:39.364 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:39.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:39.365 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:39.465 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:39.500 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:39.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:39.501 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:39.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:39 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/819491327' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 1, "weight": 0.8882825953359879} : dispatch 2024-09-20T19:06:39.523 INFO:teuthology.orchestra.run.smithi083.stderr:set osd.1 primary-affinity to 0.888283 (8582142) 2024-09-20T19:06:39.524 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:39 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/819491327' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 1, "weight": 0.8882825953359879} : dispatch 2024-09-20T19:06:39.524 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:39 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/819491327' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 1, "weight": 0.8882825953359879} : dispatch 2024-09-20T19:06:39.602 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:39.631 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:39.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:39.632 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:39.733 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:06:39.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:39.768 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:39.768 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:06:39.869 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:39.912 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:39.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:39.912 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:40.013 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:40.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:40.046 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:40.047 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:40.113 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 17 16 37446 37430 137.589 88.125 0.00321314 0.00726153 2024-09-20T19:06:40.147 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:40.181 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:40.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:40.182 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:40.283 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:40.311 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:40.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:40.312 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:40.387 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:06:40.388 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:06:40.412 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:40.442 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:40.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:40.443 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:40.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:40 smithi083 ceph-mon[36260]: pgmap v854: 148 pgs: 148 active+clean; 1.9 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 138 MiB/s wr, 2.21k op/s 2024-09-20T19:06:40.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:40 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/819491327' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 0.8882825953359879}]': finished 2024-09-20T19:06:40.542 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:40 smithi083 ceph-mon[36260]: osdmap e434: 8 total, 8 up, 8 in 2024-09-20T19:06:40.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:40 smithi083 ceph-mon[29421]: pgmap v854: 148 pgs: 148 active+clean; 1.9 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 138 MiB/s wr, 2.21k op/s 2024-09-20T19:06:40.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:40 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/819491327' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 0.8882825953359879}]': finished 2024-09-20T19:06:40.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:40 smithi083 ceph-mon[29421]: osdmap e434: 8 total, 8 up, 8 in 2024-09-20T19:06:40.543 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:06:40.580 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:40.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:40.581 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:06:40.682 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:40.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:40.716 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:40.716 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:40.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:40 smithi142 ceph-mon[32508]: pgmap v854: 148 pgs: 148 active+clean; 1.9 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 138 MiB/s wr, 2.21k op/s 2024-09-20T19:06:40.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:40 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/819491327' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 0.8882825953359879}]': finished 2024-09-20T19:06:40.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:40 smithi142 ceph-mon[32508]: osdmap e434: 8 total, 8 up, 8 in 2024-09-20T19:06:40.817 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:40.857 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:40.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:40.858 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:40.913 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:06:40 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:06:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:06:40.959 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:41.005 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:41.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:41.006 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:41.106 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:41.113 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 18 16 39120 39104 135.757 104.625 0.00640182 0.00736057 2024-09-20T19:06:41.134 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:41.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:41.135 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:41.236 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:41.273 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:41.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:41.320 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:41.421 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:41.453 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:41.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:41.454 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:41.464 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T19:06:41.464 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-09-20T19:06:41.464 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:41.464 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.26264.0:39560 11.7 e435/422 11:e949edcd:::benchmark_data_smithi083_177491_object39559:head v 435'2493, mlcod=435'2488)", 2024-09-20T19:06:41.464 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:06:41.381203+0000", 2024-09-20T19:06:41.464 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.0051342860000000001, 2024-09-20T19:06:41.464 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0051534470000000002, 2024-09-20T19:06:41.464 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:06:41.464 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:06:41.464 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-09-20T19:06:41.464 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:06:41.464 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:41.464 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:06:41.465 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:41.381203+0000", 2024-09-20T19:06:41.465 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:41.465 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:41.465 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:41.465 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:06:41.465 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:41.381203+0000", 2024-09-20T19:06:41.465 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:41.465 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:41.465 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:41.465 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:06:41.465 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:41.381204+0000", 2024-09-20T19:06:41.465 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.3499999999999996e-07 2024-09-20T19:06:41.465 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:41.465 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:41.465 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:06:41.466 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:41.381226+0000", 2024-09-20T19:06:41.466 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2101000000000001e-05 2024-09-20T19:06:41.466 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:41.466 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:41.466 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:06:41.466 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:41.381235+0000", 2024-09-20T19:06:41.466 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.1589999999999999e-06 2024-09-20T19:06:41.466 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:41.466 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:41.466 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:06:41.466 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:41.381242+0000", 2024-09-20T19:06:41.466 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.455e-06 2024-09-20T19:06:41.466 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:41.466 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:41.466 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:41.467 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:41.383735+0000", 2024-09-20T19:06:41.467 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0024932370000000001 2024-09-20T19:06:41.467 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:41.467 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:41.467 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:06:41.467 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:41.383755+0000", 2024-09-20T19:06:41.467 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0273999999999999e-05 2024-09-20T19:06:41.467 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:41.467 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:06:41.467 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:41.467 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:41.467 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:41.467 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.26264.0:39564 11.7 e435/422 11:ee965057:::benchmark_data_smithi083_177491_object39563:head v 435'2494, mlcod=435'2488)", 2024-09-20T19:06:41.467 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:06:41.383620+0000", 2024-09-20T19:06:41.467 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.0027174130000000001, 2024-09-20T19:06:41.468 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0027711609999999999, 2024-09-20T19:06:41.468 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:06:41.468 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:06:41.468 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-09-20T19:06:41.468 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:06:41.468 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:41.468 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:06:41.468 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:41.383620+0000", 2024-09-20T19:06:41.468 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:41.468 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:41.468 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:41.468 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:06:41.468 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:41.383620+0000", 2024-09-20T19:06:41.468 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:06:41.468 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:41.468 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:41.469 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:06:41.469 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:41.383621+0000", 2024-09-20T19:06:41.469 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.4499999999999996e-07 2024-09-20T19:06:41.469 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:41.469 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:41.469 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:06:41.469 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:41.383654+0000", 2024-09-20T19:06:41.469 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.2545000000000001e-05 2024-09-20T19:06:41.469 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:41.469 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:41.469 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:06:41.469 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:41.383663+0000", 2024-09-20T19:06:41.469 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.3989999999999993e-06 2024-09-20T19:06:41.469 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:41.469 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:41.469 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:06:41.470 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:41.383670+0000", 2024-09-20T19:06:41.470 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.6800000000000004e-06 2024-09-20T19:06:41.470 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:41.470 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:41.470 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:06:41.470 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:41.385432+0000", 2024-09-20T19:06:41.470 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001761989 2024-09-20T19:06:41.470 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:06:41.470 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:06:41.470 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:06:41.470 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:41.385452+0000", 2024-09-20T19:06:41.470 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9913999999999999e-05 2024-09-20T19:06:41.471 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:41.471 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:06:41.471 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:41.471 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:06:41.471 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-09-20T19:06:41.471 INFO:teuthology.orchestra.run.smithi142.stdout: "num_ops": 2 2024-09-20T19:06:41.471 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T19:06:41.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:41 smithi142 ceph-mon[32508]: osdmap e435: 8 total, 8 up, 8 in 2024-09-20T19:06:41.555 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:41.574 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:41 smithi083 ceph-mon[29421]: osdmap e435: 8 total, 8 up, 8 in 2024-09-20T19:06:41.574 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:41 smithi083 ceph-mon[36260]: osdmap e435: 8 total, 8 up, 8 in 2024-09-20T19:06:41.588 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:41.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:41.589 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:41.690 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:41.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:41.721 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:41.721 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:41.822 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:41.855 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:41.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:41.857 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:41.957 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:41.990 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:41.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:41.991 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:42.091 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:42.112 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:42.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:42.113 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:42.114 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 19 16 40756 40740 133.993 102.25 0.00263851 0.00744117 2024-09-20T19:06:42.206 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_blocked_ops 2024-09-20T19:06:42.215 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:06:42.250 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:42.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:42.252 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:06:42.268 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:06:42.268 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:06:42.268 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:06:42.268 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:06:42.268 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:06:42.268 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:06:42.268 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:06:42.268 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:06:42.269 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:06:42.269 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:06:42.269 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:06:42.269 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:06:42.269 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:06:42.269 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:06:42.269 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:06:42.269 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:06:42.323 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-20T19:06:42.352 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:42.389 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:42.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:42.390 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:42.491 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:42.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:42 smithi142 ceph-mon[32508]: pgmap v857: 148 pgs: 148 active+clean; 2.1 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 710 B/s rd, 117 MiB/s wr, 1.87k op/s 2024-09-20T19:06:42.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:42 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3282944716' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:06:42.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:42 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:06:42.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:42 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:06:42.513 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:42 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3212674220' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:06:42.538 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:42.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:42.540 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:42.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:42 smithi083 ceph-mon[29421]: pgmap v857: 148 pgs: 148 active+clean; 2.1 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 710 B/s rd, 117 MiB/s wr, 1.87k op/s 2024-09-20T19:06:42.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:42 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3282944716' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:06:42.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:42 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:06:42.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:42 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:06:42.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:42 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3212674220' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:06:42.542 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:42 smithi083 ceph-mon[36260]: pgmap v857: 148 pgs: 148 active+clean; 2.1 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 710 B/s rd, 117 MiB/s wr, 1.87k op/s 2024-09-20T19:06:42.542 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:42 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3282944716' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:06:42.542 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:42 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:06:42.542 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:42 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:06:42.542 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:42 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3212674220' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:06:42.600 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:06:42.640 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:42.661 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:42.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:42.662 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:42.763 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:42.794 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:42.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:42.795 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:42.896 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:06:42.923 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:42.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:42.925 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:06:43.025 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:43.061 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:43.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:43.062 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:43.113 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:2024-09-20T19:06:43.110182+0000 min lat: 0.00109192 max lat: 0.332044 avg lat: 0.00756437 2024-09-20T19:06:43.113 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-20T19:06:43.113 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 20 16 42298 42282 132.112 96.375 0.00753802 0.00756437 2024-09-20T19:06:43.163 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:43.182 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T19:06:43.182 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:06:43.199 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:43.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:43.200 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:43.301 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:43.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:43.342 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:43.342 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:43.443 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:43.475 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:43.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:43.476 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:43.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:43 smithi083 ceph-mon[29421]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:06:43.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:43 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3282944716' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T19:06:43.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:43 smithi083 ceph-mon[29421]: osdmap e436: 8 total, 8 up, 8 in 2024-09-20T19:06:43.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:43 smithi083 ceph-mon[29421]: 4.1e scrub starts 2024-09-20T19:06:43.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:43 smithi083 ceph-mon[29421]: 4.1e scrub ok 2024-09-20T19:06:43.542 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:43 smithi083 ceph-mon[36260]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:06:43.542 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:43 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3282944716' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T19:06:43.542 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:43 smithi083 ceph-mon[36260]: osdmap e436: 8 total, 8 up, 8 in 2024-09-20T19:06:43.542 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:43 smithi083 ceph-mon[36260]: 4.1e scrub starts 2024-09-20T19:06:43.542 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:43 smithi083 ceph-mon[36260]: 4.1e scrub ok 2024-09-20T19:06:43.577 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:43.609 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:43.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:43.610 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:43.710 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:43.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:43.751 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:43.751 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:43.751 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:43 smithi142 ceph-mon[32508]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:06:43.752 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:43 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3282944716' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T19:06:43.752 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:43 smithi142 ceph-mon[32508]: osdmap e436: 8 total, 8 up, 8 in 2024-09-20T19:06:43.752 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:43 smithi142 ceph-mon[32508]: 4.1e scrub starts 2024-09-20T19:06:43.752 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:43 smithi142 ceph-mon[32508]: 4.1e scrub ok 2024-09-20T19:06:43.851 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:43.895 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:43.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:43.896 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:43.924 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_historic_ops 2024-09-20T19:06:43.997 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:44.044 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:44.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:44.045 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:44.113 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 21 16 44207 44191 131.502 119.312 0.00732346 0.00759918 2024-09-20T19:06:44.146 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:44.184 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:44.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:44.185 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:44.286 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:44.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:44.320 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:44.320 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:44.363 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:06:44.420 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:44.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:44.457 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:44.458 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:44.558 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:44.580 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:44.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:44.581 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:44.682 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:44.714 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:44.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:44.715 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:44.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:44 smithi142 ceph-mon[32508]: 4.15 scrub starts 2024-09-20T19:06:44.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:44 smithi142 ceph-mon[32508]: 4.15 scrub ok 2024-09-20T19:06:44.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:44 smithi142 ceph-mon[32508]: pgmap v859: 148 pgs: 148 active+clean; 2.4 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 853 B/s rd, 88 MiB/s wr, 1.42k op/s 2024-09-20T19:06:44.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:44 smithi142 ceph-mon[32508]: 2.0 scrub starts 2024-09-20T19:06:44.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:44 smithi142 ceph-mon[32508]: 2.0 scrub ok 2024-09-20T19:06:44.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:44 smithi083 ceph-mon[29421]: 4.15 scrub starts 2024-09-20T19:06:44.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:44 smithi083 ceph-mon[29421]: 4.15 scrub ok 2024-09-20T19:06:44.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:44 smithi083 ceph-mon[29421]: pgmap v859: 148 pgs: 148 active+clean; 2.4 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 853 B/s rd, 88 MiB/s wr, 1.42k op/s 2024-09-20T19:06:44.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:44 smithi083 ceph-mon[29421]: 2.0 scrub starts 2024-09-20T19:06:44.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:44 smithi083 ceph-mon[29421]: 2.0 scrub ok 2024-09-20T19:06:44.792 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:44 smithi083 ceph-mon[36260]: 4.15 scrub starts 2024-09-20T19:06:44.792 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:44 smithi083 ceph-mon[36260]: 4.15 scrub ok 2024-09-20T19:06:44.792 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:44 smithi083 ceph-mon[36260]: pgmap v859: 148 pgs: 148 active+clean; 2.4 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 853 B/s rd, 88 MiB/s wr, 1.42k op/s 2024-09-20T19:06:44.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:44 smithi083 ceph-mon[36260]: 2.0 scrub starts 2024-09-20T19:06:44.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:44 smithi083 ceph-mon[36260]: 2.0 scrub ok 2024-09-20T19:06:44.816 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:44.847 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:44.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:44.847 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:44.948 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:44.980 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:44.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:44.981 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:45.082 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:45.113 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 22 16 45801 45785 130.053 99.625 0.00807882 0.0076841 2024-09-20T19:06:45.115 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:45.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:45.116 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:45.217 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:45.248 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:45.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:45.249 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:45.345 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 7, 6, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 5, 6, 7, 4] 2024-09-20T19:06:45.345 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T19:06:45.345 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_4 2024-09-20T19:06:45.345 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T19:06:45.349 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:45.433 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:45.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:45.434 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:45.535 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:45.585 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:45.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:45.586 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:45.684 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:45 smithi142 ceph-mon[32508]: 6.1 scrub starts 2024-09-20T19:06:45.684 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:45 smithi142 ceph-mon[32508]: 6.1 scrub ok 2024-09-20T19:06:45.684 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:45 smithi142 ceph-mon[32508]: 6.7 scrub starts 2024-09-20T19:06:45.684 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:45 smithi142 ceph-mon[32508]: 6.7 scrub ok 2024-09-20T19:06:45.684 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:45 smithi142 ceph-mon[32508]: 4.13 scrub starts 2024-09-20T19:06:45.684 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:45 smithi142 ceph-mon[32508]: 4.13 scrub ok 2024-09-20T19:06:45.684 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:45 smithi142 ceph-mon[32508]: 3.18 scrub starts 2024-09-20T19:06:45.684 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:45 smithi142 ceph-mon[32508]: 3.18 scrub ok 2024-09-20T19:06:45.686 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:45.723 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:45.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:45.724 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:45.781 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:06:45.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:45 smithi083 ceph-mon[29421]: 6.1 scrub starts 2024-09-20T19:06:45.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:45 smithi083 ceph-mon[29421]: 6.1 scrub ok 2024-09-20T19:06:45.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:45 smithi083 ceph-mon[29421]: 6.7 scrub starts 2024-09-20T19:06:45.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:45 smithi083 ceph-mon[29421]: 6.7 scrub ok 2024-09-20T19:06:45.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:45 smithi083 ceph-mon[29421]: 4.13 scrub starts 2024-09-20T19:06:45.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:45 smithi083 ceph-mon[29421]: 4.13 scrub ok 2024-09-20T19:06:45.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:45 smithi083 ceph-mon[29421]: 3.18 scrub starts 2024-09-20T19:06:45.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:45 smithi083 ceph-mon[29421]: 3.18 scrub ok 2024-09-20T19:06:45.792 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:45 smithi083 ceph-mon[36260]: 6.1 scrub starts 2024-09-20T19:06:45.792 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:45 smithi083 ceph-mon[36260]: 6.1 scrub ok 2024-09-20T19:06:45.792 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:45 smithi083 ceph-mon[36260]: 6.7 scrub starts 2024-09-20T19:06:45.792 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:45 smithi083 ceph-mon[36260]: 6.7 scrub ok 2024-09-20T19:06:45.792 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:45 smithi083 ceph-mon[36260]: 4.13 scrub starts 2024-09-20T19:06:45.792 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:45 smithi083 ceph-mon[36260]: 4.13 scrub ok 2024-09-20T19:06:45.792 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:45 smithi083 ceph-mon[36260]: 3.18 scrub starts 2024-09-20T19:06:45.792 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:45 smithi083 ceph-mon[36260]: 3.18 scrub ok 2024-09-20T19:06:45.825 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:45.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:45.858 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:45.858 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:45.959 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:45.985 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T19:06:45.985 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:06:45.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:45.997 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:45.997 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:46.098 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:46.114 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 23 16 47462 47446 128.911 103.812 0.0086684 0.00775161 2024-09-20T19:06:46.123 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:46.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:46.125 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:46.226 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:06:46.263 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T19:06:46.266 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:46.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:46.267 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:06:46.359 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T19:06:46.368 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:06:46.443 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:46.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:46.444 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:06:46.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:46 smithi083 ceph-mon[29421]: pgmap v860: 148 pgs: 148 active+clean; 2.4 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 853 B/s rd, 82 MiB/s wr, 1.31k op/s 2024-09-20T19:06:46.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:46 smithi083 ceph-mon[29421]: 6.1e scrub starts 2024-09-20T19:06:46.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:46 smithi083 ceph-mon[29421]: 6.1e scrub ok 2024-09-20T19:06:46.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:46 smithi083 ceph-mon[29421]: 3.17 scrub starts 2024-09-20T19:06:46.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:46 smithi083 ceph-mon[29421]: 3.17 scrub ok 2024-09-20T19:06:46.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:46 smithi083 ceph-mon[29421]: osdmap e437: 8 total, 8 up, 8 in 2024-09-20T19:06:46.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:46 smithi083 ceph-mon[36260]: pgmap v860: 148 pgs: 148 active+clean; 2.4 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 853 B/s rd, 82 MiB/s wr, 1.31k op/s 2024-09-20T19:06:46.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:46 smithi083 ceph-mon[36260]: 6.1e scrub starts 2024-09-20T19:06:46.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:46 smithi083 ceph-mon[36260]: 6.1e scrub ok 2024-09-20T19:06:46.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:46 smithi083 ceph-mon[36260]: 3.17 scrub starts 2024-09-20T19:06:46.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:46 smithi083 ceph-mon[36260]: 3.17 scrub ok 2024-09-20T19:06:46.543 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:46 smithi083 ceph-mon[36260]: osdmap e437: 8 total, 8 up, 8 in 2024-09-20T19:06:46.545 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:46.572 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:46.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:46.573 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:46.659 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:46 smithi142 ceph-mon[32508]: pgmap v860: 148 pgs: 148 active+clean; 2.4 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 853 B/s rd, 82 MiB/s wr, 1.31k op/s 2024-09-20T19:06:46.659 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:46 smithi142 ceph-mon[32508]: 6.1e scrub starts 2024-09-20T19:06:46.659 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:46 smithi142 ceph-mon[32508]: 6.1e scrub ok 2024-09-20T19:06:46.659 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:46 smithi142 ceph-mon[32508]: 3.17 scrub starts 2024-09-20T19:06:46.659 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:46 smithi142 ceph-mon[32508]: 3.17 scrub ok 2024-09-20T19:06:46.659 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:46 smithi142 ceph-mon[32508]: osdmap e437: 8 total, 8 up, 8 in 2024-09-20T19:06:46.674 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:46.707 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:46.707 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-20T19:06:46.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:46.708 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:46.772 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:06:46.810 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:46.834 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:46.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:46.835 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:46.839 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T19:06:46.841 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-20T19:06:46.935 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:06:46.979 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:46.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:46.980 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:06:47.081 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:47.111 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:47.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:47.112 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:47.116 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 24 16 48740 48724 126.861 79.875 0.0051481 0.00787744 2024-09-20T19:06:47.213 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:47.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:47.257 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:47.257 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:47.358 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:47.400 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:47.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:47.401 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:47.502 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:47.539 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:47.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:47.540 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:47.640 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:47.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:47.676 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:47.676 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:47.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:47 smithi142 ceph-mon[32508]: 4.18 scrub starts 2024-09-20T19:06:47.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:47 smithi142 ceph-mon[32508]: 4.18 scrub ok 2024-09-20T19:06:47.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:47 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:06:47.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:47 smithi142 ceph-mon[32508]: from='client.16455 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:06:47.762 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:06:47 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:06:47.777 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:47.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:47 smithi083 ceph-mon[29421]: 4.18 scrub starts 2024-09-20T19:06:47.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:47 smithi083 ceph-mon[29421]: 4.18 scrub ok 2024-09-20T19:06:47.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:47 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:06:47.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:47 smithi083 ceph-mon[29421]: from='client.16455 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:06:47.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:47 smithi083 ceph-mon[36260]: 4.18 scrub starts 2024-09-20T19:06:47.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:47 smithi083 ceph-mon[36260]: 4.18 scrub ok 2024-09-20T19:06:47.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:47 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:06:47.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:47 smithi083 ceph-mon[36260]: from='client.16455 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:06:47.816 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:47.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:47.817 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:47.918 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:06:47.959 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:47.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:47.960 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:06:48.061 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:48.090 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:48.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:48.091 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:48.115 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 25 16 50505 50489 126.199 110.312 0.0177438 0.00791827 2024-09-20T19:06:48.192 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:48.230 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:48.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:48.231 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:48.332 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:48.366 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:48.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:48.367 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:48.468 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:48.494 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:48.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:48.495 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:48.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:48 smithi083 ceph-mon[29421]: pgmap v862: 148 pgs: 148 active+clean; 2.9 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.5 KiB/s rd, 141 MiB/s wr, 2.26k op/s 2024-09-20T19:06:48.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:48 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:06:48.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:48 smithi083 ceph-mon[29421]: 3.2 scrub starts 2024-09-20T19:06:48.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:48 smithi083 ceph-mon[29421]: 3.2 scrub ok 2024-09-20T19:06:48.542 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:48 smithi083 ceph-mon[36260]: pgmap v862: 148 pgs: 148 active+clean; 2.9 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.5 KiB/s rd, 141 MiB/s wr, 2.26k op/s 2024-09-20T19:06:48.542 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:48 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:06:48.542 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:48 smithi083 ceph-mon[36260]: 3.2 scrub starts 2024-09-20T19:06:48.542 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:48 smithi083 ceph-mon[36260]: 3.2 scrub ok 2024-09-20T19:06:48.596 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:48.627 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:48.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:48.628 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:48.729 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:48.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:48 smithi142 ceph-mon[32508]: pgmap v862: 148 pgs: 148 active+clean; 2.9 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.5 KiB/s rd, 141 MiB/s wr, 2.26k op/s 2024-09-20T19:06:48.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:48 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:06:48.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:48 smithi142 ceph-mon[32508]: 3.2 scrub starts 2024-09-20T19:06:48.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:48 smithi142 ceph-mon[32508]: 3.2 scrub ok 2024-09-20T19:06:48.763 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:48.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:48.764 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:48.865 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:48.898 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:48.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:48.899 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:49.000 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:49.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:49.030 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:49.030 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:49.115 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 26 16 52096 52080 125.169 99.4375 0.0088776 0.00798435 2024-09-20T19:06:49.131 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:49.166 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:49.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:49.167 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:49.203 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:06:49.223 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":861,"stamp":"2024-09-20T19:06:45.911862+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2805209627,"num_objects":42994,"num_object_clones":0,"num_object_copies":128982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42994,"num_whiteouts":0,"num_read":2265,"num_read_kb":2122,"num_write":43630,"num_write_kb":2740553,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":1779163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43663,"ondisk_log_size":43663,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":445,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":12807952,"kb_used_data":8005632,"kb_used_omap":16,"kb_used_meta":4802095,"kb_avail":737153264,"statfs":{"total":767960285184,"available":754844942336,"internally_reserved":0,"allocated":8197767168,"data_stored":8192660978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16790,"internal_metadata":4917345898},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,2,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":720175104,"num_objects":10989,"num_object_clones":0,"num_object_copies":32967,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10989,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":10989,"num_write_kb":703296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.621235"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":804,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.333949+0000","last_change":"2024-09-20T19:01:31.817967+0000","last_active":"2024-09-20T19:06:42.333949+0000","last_peered":"2024-09-20T19:06:42.333949+0000","last_clean":"2024-09-20T19:06:42.333949+0000","last_became_active":"2024-09-20T18:49:45.943201+0000","last_became_peered":"2024-09-20T18:49:45.943201+0000","last_unstale":"2024-09-20T19:06:42.333949+0000","last_undegraded":"2024-09-20T19:06:42.333949+0000","last_fullsized":"2024-09-20T19:06:42.333949+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:31.817909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:31.817909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:00.513933+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"273'15","reported_seq":144,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.322393+0000","last_change":"2024-09-20T19:04:21.040978+0000","last_active":"2024-09-20T19:06:42.322393+0000","last_peered":"2024-09-20T19:06:42.322393+0000","last_clean":"2024-09-20T19:06:42.322393+0000","last_became_active":"2024-09-20T19:04:20.925853+0000","last_became_peered":"2024-09-20T19:04:20.925853+0000","last_unstale":"2024-09-20T19:06:42.322393+0000","last_undegraded":"2024-09-20T19:06:42.322393+0000","last_fullsized":"2024-09-20T19:06:42.322393+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:03:10.214154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:10.214154+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":16,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.332542+0000","last_change":"2024-09-20T19:06:40.320839+0000","last_active":"2024-09-20T19:06:42.332542+0000","last_peered":"2024-09-20T19:06:42.332542+0000","last_clean":"2024-09-20T19:06:42.332542+0000","last_became_active":"2024-09-20T19:06:40.320505+0000","last_became_peered":"2024-09-20T19:06:40.320505+0000","last_unstale":"2024-09-20T19:06:42.332542+0000","last_undegraded":"2024-09-20T19:06:42.332542+0000","last_fullsized":"2024-09-20T19:06:42.332542+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:49.016006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:49.016006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:40.896471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":133,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.324123+0000","last_change":"2024-09-20T19:04:20.881918+0000","last_active":"2024-09-20T19:06:42.324123+0000","last_peered":"2024-09-20T19:06:42.324123+0000","last_clean":"2024-09-20T19:06:42.324123+0000","last_became_active":"2024-09-20T19:04:20.880905+0000","last_became_peered":"2024-09-20T19:04:20.880905+0000","last_unstale":"2024-09-20T19:06:42.324123+0000","last_undegraded":"2024-09-20T19:06:42.324123+0000","last_fullsized":"2024-09-20T19:06:42.324123+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:02.084375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:02.084375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:39.438073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":803,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.334111+0000","last_change":"2024-09-20T19:02:02.441633+0000","last_active":"2024-09-20T19:06:42.334111+0000","last_peered":"2024-09-20T19:06:42.334111+0000","last_clean":"2024-09-20T19:06:42.334111+0000","last_became_active":"2024-09-20T18:59:05.081081+0000","last_became_peered":"2024-09-20T18:59:05.081081+0000","last_unstale":"2024-09-20T19:06:42.334111+0000","last_undegraded":"2024-09-20T19:06:42.334111+0000","last_fullsized":"2024-09-20T19:06:42.334111+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:02.441577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:02.441577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:11.774126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":817,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.334776+0000","last_change":"2024-09-20T19:04:37.540824+0000","last_active":"2024-09-20T19:06:42.334776+0000","last_peered":"2024-09-20T19:06:42.334776+0000","last_clean":"2024-09-20T19:06:42.334776+0000","last_became_active":"2024-09-20T18:59:05.072675+0000","last_became_peered":"2024-09-20T18:59:05.072675+0000","last_unstale":"2024-09-20T19:06:42.334776+0000","last_undegraded":"2024-09-20T19:06:42.334776+0000","last_fullsized":"2024-09-20T19:06:42.334776+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:37.540760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:04:37.540760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:30.957798+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"274'27","reported_seq":848,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.330723+0000","last_change":"2024-09-20T19:05:50.750134+0000","last_active":"2024-09-20T19:06:42.330723+0000","last_peered":"2024-09-20T19:06:42.330723+0000","last_clean":"2024-09-20T19:06:42.330723+0000","last_became_active":"2024-09-20T18:49:47.952150+0000","last_became_peered":"2024-09-20T18:49:47.952150+0000","last_unstale":"2024-09-20T19:06:42.330723+0000","last_undegraded":"2024-09-20T19:06:42.330723+0000","last_fullsized":"2024-09-20T19:06:42.330723+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'27","last_scrub_stamp":"2024-09-20T19:05:50.750050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:50.750050+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:13.513790+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":142,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.322488+0000","last_change":"2024-09-20T19:06:25.218221+0000","last_active":"2024-09-20T19:06:42.322488+0000","last_peered":"2024-09-20T19:06:42.322488+0000","last_clean":"2024-09-20T19:06:42.322488+0000","last_became_active":"2024-09-20T19:04:20.925394+0000","last_became_peered":"2024-09-20T19:04:20.925394+0000","last_unstale":"2024-09-20T19:06:42.322488+0000","last_undegraded":"2024-09-20T19:06:42.322488+0000","last_fullsized":"2024-09-20T19:06:42.322488+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:25.218150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:06:25.218150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:38.749309+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"59'1","reported_seq":878,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.333941+0000","last_change":"2024-09-20T19:04:20.963496+0000","last_active":"2024-09-20T19:06:42.333941+0000","last_peered":"2024-09-20T19:06:42.333941+0000","last_clean":"2024-09-20T19:06:42.333941+0000","last_became_active":"2024-09-20T19:04:20.914145+0000","last_became_peered":"2024-09-20T19:04:20.914145+0000","last_unstale":"2024-09-20T19:06:42.333941+0000","last_undegraded":"2024-09-20T19:06:42.333941+0000","last_fullsized":"2024-09-20T19:06:42.333941+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:01:28.592653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:28.592653+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:33.945469+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"273'15","reported_seq":143,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.324184+0000","last_change":"2024-09-20T19:04:20.984223+0000","last_active":"2024-09-20T19:06:42.324184+0000","last_peered":"2024-09-20T19:06:42.324184+0000","last_clean":"2024-09-20T19:06:42.324184+0000","last_became_active":"2024-09-20T19:04:20.898235+0000","last_became_peered":"2024-09-20T19:04:20.898235+0000","last_unstale":"2024-09-20T19:06:42.324184+0000","last_undegraded":"2024-09-20T19:06:42.324184+0000","last_fullsized":"2024-09-20T19:06:42.324184+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:02:42.114331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:42.114331+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:28.178800+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":801,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.332698+0000","last_change":"2024-09-20T19:01:51.564693+0000","last_active":"2024-09-20T19:06:42.332698+0000","last_peered":"2024-09-20T19:06:42.332698+0000","last_clean":"2024-09-20T19:06:42.332698+0000","last_became_active":"2024-09-20T18:49:49.961188+0000","last_became_peered":"2024-09-20T18:49:49.961188+0000","last_unstale":"2024-09-20T19:06:42.332698+0000","last_undegraded":"2024-09-20T19:06:42.332698+0000","last_fullsized":"2024-09-20T19:06:42.332698+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:51.564624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:51.564624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":782,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.332890+0000","last_change":"2024-09-20T19:03:09.963421+0000","last_active":"2024-09-20T19:06:42.332890+0000","last_peered":"2024-09-20T19:06:42.332890+0000","last_clean":"2024-09-20T19:06:42.332890+0000","last_became_active":"2024-09-20T18:49:51.966715+0000","last_became_peered":"2024-09-20T18:49:51.966715+0000","last_unstale":"2024-09-20T19:06:42.332890+0000","last_undegraded":"2024-09-20T19:06:42.332890+0000","last_fullsized":"2024-09-20T19:06:42.332890+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:09.963298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:03:09.963298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":826,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.333201+0000","last_change":"2024-09-20T19:04:20.879227+0000","last_active":"2024-09-20T19:06:42.333201+0000","last_peered":"2024-09-20T19:06:42.333201+0000","last_clean":"2024-09-20T19:06:42.333201+0000","last_became_active":"2024-09-20T19:04:20.878905+0000","last_became_peered":"2024-09-20T19:04:20.878905+0000","last_unstale":"2024-09-20T19:06:42.333201+0000","last_undegraded":"2024-09-20T19:06:42.333201+0000","last_fullsized":"2024-09-20T19:06:42.333201+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:48.520017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:48.520017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:42.653960+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"273'7","reported_seq":147,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.324535+0000","last_change":"2024-09-20T19:05:53.980848+0000","last_active":"2024-09-20T19:06:42.324535+0000","last_peered":"2024-09-20T19:06:42.324535+0000","last_clean":"2024-09-20T19:06:42.324535+0000","last_became_active":"2024-09-20T19:04:20.888572+0000","last_became_peered":"2024-09-20T19:04:20.888572+0000","last_unstale":"2024-09-20T19:06:42.324535+0000","last_undegraded":"2024-09-20T19:06:42.324535+0000","last_fullsized":"2024-09-20T19:06:42.324535+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'7","last_scrub_stamp":"2024-09-20T19:05:53.980772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:53.980772+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:54.857238+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":817,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:40.314817+0000","last_change":"2024-09-20T19:04:20.885873+0000","last_active":"2024-09-20T19:06:40.314817+0000","last_peered":"2024-09-20T19:06:40.314817+0000","last_clean":"2024-09-20T19:06:40.314817+0000","last_became_active":"2024-09-20T19:04:20.885417+0000","last_became_peered":"2024-09-20T19:04:20.885417+0000","last_unstale":"2024-09-20T19:06:40.314817+0000","last_undegraded":"2024-09-20T19:06:40.314817+0000","last_fullsized":"2024-09-20T19:06:40.314817+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:33.203312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:33.203312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:05.802150+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":797,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.333382+0000","last_change":"2024-09-20T19:05:21.625378+0000","last_active":"2024-09-20T19:06:42.333382+0000","last_peered":"2024-09-20T19:06:42.333382+0000","last_clean":"2024-09-20T19:06:42.333382+0000","last_became_active":"2024-09-20T18:49:51.970663+0000","last_became_peered":"2024-09-20T18:49:51.970663+0000","last_unstale":"2024-09-20T19:06:42.333382+0000","last_undegraded":"2024-09-20T19:06:42.333382+0000","last_fullsized":"2024-09-20T19:06:42.333382+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:21.625311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:05:21.625311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:35.920771+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":142,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:45.481243+0000","last_change":"2024-09-20T19:06:45.481243+0000","last_active":"2024-09-20T19:06:45.481243+0000","last_peered":"2024-09-20T19:06:45.481243+0000","last_clean":"2024-09-20T19:06:45.481243+0000","last_became_active":"2024-09-20T19:04:20.906109+0000","last_became_peered":"2024-09-20T19:04:20.906109+0000","last_unstale":"2024-09-20T19:06:45.481243+0000","last_undegraded":"2024-09-20T19:06:45.481243+0000","last_fullsized":"2024-09-20T19:06:45.481243+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:45.479545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:06:45.479545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:08:14.852920+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":827,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.333844+0000","last_change":"2024-09-20T19:04:20.907057+0000","last_active":"2024-09-20T19:06:42.333844+0000","last_peered":"2024-09-20T19:06:42.333844+0000","last_clean":"2024-09-20T19:06:42.333844+0000","last_became_active":"2024-09-20T19:04:20.906691+0000","last_became_peered":"2024-09-20T19:04:20.906691+0000","last_unstale":"2024-09-20T19:06:42.333844+0000","last_undegraded":"2024-09-20T19:06:42.333844+0000","last_fullsized":"2024-09-20T19:06:42.333844+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:44.998147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:44.998147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:28.750471+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"273'27","reported_seq":857,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:40.314505+0000","last_change":"2024-09-20T19:05:49.452928+0000","last_active":"2024-09-20T19:06:40.314505+0000","last_peered":"2024-09-20T19:06:40.314505+0000","last_clean":"2024-09-20T19:06:40.314505+0000","last_became_active":"2024-09-20T18:49:47.950939+0000","last_became_peered":"2024-09-20T18:49:47.950939+0000","last_unstale":"2024-09-20T19:06:40.314505+0000","last_undegraded":"2024-09-20T19:06:40.314505+0000","last_fullsized":"2024-09-20T19:06:40.314505+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'27","last_scrub_stamp":"2024-09-20T19:05:49.452861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:49.452861+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:02.753272+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":808,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:40.312748+0000","last_change":"2024-09-20T19:04:20.890659+0000","last_active":"2024-09-20T19:06:40.312748+0000","last_peered":"2024-09-20T19:06:40.312748+0000","last_clean":"2024-09-20T19:06:40.312748+0000","last_became_active":"2024-09-20T19:04:20.890370+0000","last_became_peered":"2024-09-20T19:04:20.890370+0000","last_unstale":"2024-09-20T19:06:40.312748+0000","last_undegraded":"2024-09-20T19:06:40.312748+0000","last_fullsized":"2024-09-20T19:06:40.312748+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:05.888974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:05.888974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:29.757611+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":813,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.335177+0000","last_change":"2024-09-20T19:06:17.937954+0000","last_active":"2024-09-20T19:06:42.335177+0000","last_peered":"2024-09-20T19:06:42.335177+0000","last_clean":"2024-09-20T19:06:42.335177+0000","last_became_active":"2024-09-20T18:59:05.084496+0000","last_became_peered":"2024-09-20T18:59:05.084496+0000","last_unstale":"2024-09-20T19:06:42.335177+0000","last_undegraded":"2024-09-20T19:06:42.335177+0000","last_fullsized":"2024-09-20T19:06:42.335177+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.937896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.937896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:37.574460+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"59'1","reported_seq":137,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.323974+0000","last_change":"2024-09-20T19:04:20.905233+0000","last_active":"2024-09-20T19:06:42.323974+0000","last_peered":"2024-09-20T19:06:42.323974+0000","last_clean":"2024-09-20T19:06:42.323974+0000","last_became_active":"2024-09-20T19:04:20.887062+0000","last_became_peered":"2024-09-20T19:04:20.887062+0000","last_unstale":"2024-09-20T19:06:42.323974+0000","last_undegraded":"2024-09-20T19:06:42.323974+0000","last_fullsized":"2024-09-20T19:06:42.323974+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:03:08.218812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:03:08.218812+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":993,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"274'18","reported_seq":854,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.335605+0000","last_change":"2024-09-20T19:04:20.925690+0000","last_active":"2024-09-20T19:06:42.335605+0000","last_peered":"2024-09-20T19:06:42.335605+0000","last_clean":"2024-09-20T19:06:42.335605+0000","last_became_active":"2024-09-20T19:04:20.896343+0000","last_became_peered":"2024-09-20T19:04:20.896343+0000","last_unstale":"2024-09-20T19:06:42.335605+0000","last_undegraded":"2024-09-20T19:06:42.335605+0000","last_fullsized":"2024-09-20T19:06:42.335605+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:57:09.818999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:09.818999+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:29.919360+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":141,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.321880+0000","last_change":"2024-09-20T19:06:16.148081+0000","last_active":"2024-09-20T19:06:42.321880+0000","last_peered":"2024-09-20T19:06:42.321880+0000","last_clean":"2024-09-20T19:06:42.321880+0000","last_became_active":"2024-09-20T19:04:20.889430+0000","last_became_peered":"2024-09-20T19:04:20.889430+0000","last_unstale":"2024-09-20T19:06:42.321880+0000","last_undegraded":"2024-09-20T19:06:42.321880+0000","last_fullsized":"2024-09-20T19:06:42.321880+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:16.148022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:06:16.148022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:27.803843+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":789,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:40.315664+0000","last_change":"2024-09-20T19:02:28.294390+0000","last_active":"2024-09-20T19:06:40.315664+0000","last_peered":"2024-09-20T19:06:40.315664+0000","last_clean":"2024-09-20T19:06:40.315664+0000","last_became_active":"2024-09-20T18:49:51.970058+0000","last_became_peered":"2024-09-20T18:49:51.970058+0000","last_unstale":"2024-09-20T19:06:40.315664+0000","last_undegraded":"2024-09-20T19:06:40.315664+0000","last_fullsized":"2024-09-20T19:06:40.315664+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T19:02:28.294292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:28.294292+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:58.767339+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":825,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:40.313630+0000","last_change":"2024-09-20T19:05:22.202134+0000","last_active":"2024-09-20T19:06:40.313630+0000","last_peered":"2024-09-20T19:06:40.313630+0000","last_clean":"2024-09-20T19:06:40.313630+0000","last_became_active":"2024-09-20T19:04:20.881466+0000","last_became_peered":"2024-09-20T19:04:20.881466+0000","last_unstale":"2024-09-20T19:06:40.313630+0000","last_undegraded":"2024-09-20T19:06:40.313630+0000","last_fullsized":"2024-09-20T19:06:40.313630+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:05:22.201836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:22.201836+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:45.062354+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"274'20","reported_seq":865,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.334257+0000","last_change":"2024-09-20T19:03:16.982915+0000","last_active":"2024-09-20T19:06:42.334257+0000","last_peered":"2024-09-20T19:06:42.334257+0000","last_clean":"2024-09-20T19:06:42.334257+0000","last_became_active":"2024-09-20T19:03:16.968636+0000","last_became_peered":"2024-09-20T19:03:16.968636+0000","last_unstale":"2024-09-20T19:06:42.334257+0000","last_undegraded":"2024-09-20T19:06:42.334257+0000","last_fullsized":"2024-09-20T19:06:42.334257+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'20","last_scrub_stamp":"2024-09-20T19:02:35.096979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:35.096979+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"375'14","reported_seq":510,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:40.321029+0000","last_change":"2024-09-20T19:04:20.956210+0000","last_active":"2024-09-20T19:06:40.321029+0000","last_peered":"2024-09-20T19:06:40.321029+0000","last_clean":"2024-09-20T19:06:40.321029+0000","last_became_active":"2024-09-20T19:04:20.900782+0000","last_became_peered":"2024-09-20T19:04:20.900782+0000","last_unstale":"2024-09-20T19:06:40.321029+0000","last_undegraded":"2024-09-20T19:06:40.321029+0000","last_fullsized":"2024-09-20T19:06:40.321029+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"308'12","last_scrub_stamp":"2024-09-20T19:01:44.541156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:44.541156+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:39.843812+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.0","version":"435'2558","reported_seq":2595,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.060787+0000","last_change":"2024-09-20T19:06:18.053333+0000","last_active":"2024-09-20T19:06:42.060787+0000","last_peered":"2024-09-20T19:06:42.060787+0000","last_clean":"2024-09-20T19:06:42.060787+0000","last_became_active":"2024-09-20T19:06:18.053079+0000","last_became_peered":"2024-09-20T19:06:18.053079+0000","last_unstale":"2024-09-20T19:06:42.060787+0000","last_undegraded":"2024-09-20T19:06:42.060787+0000","last_fullsized":"2024-09-20T19:06:42.060787+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":2558,"log_dups_size":0,"ondisk_log_size":2558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:38.562225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":167641088,"num_objects":2558,"num_object_clones":0,"num_object_copies":7674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2558,"num_write_kb":163712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"274'27","reported_seq":882,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:40.313790+0000","last_change":"2024-09-20T19:04:20.920811+0000","last_active":"2024-09-20T19:06:40.313790+0000","last_peered":"2024-09-20T19:06:40.313790+0000","last_clean":"2024-09-20T19:06:40.313790+0000","last_became_active":"2024-09-20T19:04:20.897049+0000","last_became_peered":"2024-09-20T19:04:20.897049+0000","last_unstale":"2024-09-20T19:06:40.313790+0000","last_undegraded":"2024-09-20T19:06:40.313790+0000","last_fullsized":"2024-09-20T19:06:40.313790+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'27","last_scrub_stamp":"2024-09-20T19:02:27.897138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:27.897138+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:33.671747+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":810,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.328926+0000","last_change":"2024-09-20T19:05:55.742165+0000","last_active":"2024-09-20T19:06:42.328926+0000","last_peered":"2024-09-20T19:06:42.328926+0000","last_clean":"2024-09-20T19:06:42.328926+0000","last_became_active":"2024-09-20T18:49:45.936104+0000","last_became_peered":"2024-09-20T18:49:45.936104+0000","last_unstale":"2024-09-20T19:06:42.328926+0000","last_undegraded":"2024-09-20T19:06:42.328926+0000","last_fullsized":"2024-09-20T19:06:42.328926+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:55.742106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:55.742106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:16.985762+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"375'12","reported_seq":175,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.322941+0000","last_change":"2024-09-20T19:06:20.174206+0000","last_active":"2024-09-20T19:06:42.322941+0000","last_peered":"2024-09-20T19:06:42.322941+0000","last_clean":"2024-09-20T19:06:42.322941+0000","last_became_active":"2024-09-20T19:04:20.907228+0000","last_became_peered":"2024-09-20T19:04:20.907228+0000","last_unstale":"2024-09-20T19:06:42.322941+0000","last_undegraded":"2024-09-20T19:06:42.322941+0000","last_fullsized":"2024-09-20T19:06:42.322941+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"375'12","last_scrub_stamp":"2024-09-20T19:06:20.174160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:06:20.174160+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:49.543973+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":790,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.332538+0000","last_change":"2024-09-20T19:02:22.736879+0000","last_active":"2024-09-20T19:06:42.332538+0000","last_peered":"2024-09-20T19:06:42.332538+0000","last_clean":"2024-09-20T19:06:42.332538+0000","last_became_active":"2024-09-20T18:49:51.971399+0000","last_became_peered":"2024-09-20T18:49:51.971399+0000","last_unstale":"2024-09-20T19:06:42.332538+0000","last_undegraded":"2024-09-20T19:06:42.332538+0000","last_fullsized":"2024-09-20T19:06:42.332538+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:22.736825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:22.736825+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:02.798693+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.f","version":"435'2547","reported_seq":2580,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:41.360285+0000","last_change":"2024-09-20T19:06:18.056190+0000","last_active":"2024-09-20T19:06:41.360285+0000","last_peered":"2024-09-20T19:06:41.360285+0000","last_clean":"2024-09-20T19:06:41.360285+0000","last_became_active":"2024-09-20T19:06:18.055847+0000","last_became_peered":"2024-09-20T19:06:18.055847+0000","last_unstale":"2024-09-20T19:06:41.360285+0000","last_undegraded":"2024-09-20T19:06:41.360285+0000","last_fullsized":"2024-09-20T19:06:41.360285+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":2547,"log_dups_size":0,"ondisk_log_size":2547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:43.410607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":166920192,"num_objects":2547,"num_object_clones":0,"num_object_copies":7641,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2547,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2547,"num_write_kb":163008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"273'30","reported_seq":865,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.328673+0000","last_change":"2024-09-20T19:05:54.717703+0000","last_active":"2024-09-20T19:06:42.328673+0000","last_peered":"2024-09-20T19:06:42.328673+0000","last_clean":"2024-09-20T19:06:42.328673+0000","last_became_active":"2024-09-20T18:49:48.499189+0000","last_became_peered":"2024-09-20T18:49:48.499189+0000","last_unstale":"2024-09-20T19:06:42.328673+0000","last_undegraded":"2024-09-20T19:06:42.328673+0000","last_fullsized":"2024-09-20T19:06:42.328673+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'30","last_scrub_stamp":"2024-09-20T19:05:54.717609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:54.717609+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:54.790802+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":802,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.335527+0000","last_change":"2024-09-20T19:06:16.897724+0000","last_active":"2024-09-20T19:06:42.335527+0000","last_peered":"2024-09-20T19:06:42.335527+0000","last_clean":"2024-09-20T19:06:42.335527+0000","last_became_active":"2024-09-20T18:49:45.940544+0000","last_became_peered":"2024-09-20T18:49:45.940544+0000","last_unstale":"2024-09-20T19:06:42.335527+0000","last_undegraded":"2024-09-20T19:06:42.335527+0000","last_fullsized":"2024-09-20T19:06:42.335527+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:16.897677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:06:16.897677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:27.308342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":134,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.321289+0000","last_change":"2024-09-20T19:04:20.889280+0000","last_active":"2024-09-20T19:06:42.321289+0000","last_peered":"2024-09-20T19:06:42.321289+0000","last_clean":"2024-09-20T19:06:42.321289+0000","last_became_active":"2024-09-20T19:04:20.888422+0000","last_became_peered":"2024-09-20T19:04:20.888422+0000","last_unstale":"2024-09-20T19:06:42.321289+0000","last_undegraded":"2024-09-20T19:06:42.321289+0000","last_fullsized":"2024-09-20T19:06:42.321289+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.229201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.229201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:33.311119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":142,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.324210+0000","last_change":"2024-09-20T19:05:47.121946+0000","last_active":"2024-09-20T19:06:42.324210+0000","last_peered":"2024-09-20T19:06:42.324210+0000","last_clean":"2024-09-20T19:06:42.324210+0000","last_became_active":"2024-09-20T19:04:20.889418+0000","last_became_peered":"2024-09-20T19:04:20.889418+0000","last_unstale":"2024-09-20T19:06:42.324210+0000","last_undegraded":"2024-09-20T19:06:42.324210+0000","last_fullsized":"2024-09-20T19:06:42.324210+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:47.121842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:05:47.121842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:15.049722+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"436'2963","reported_seq":3002,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:45.698314+0000","last_change":"2024-09-20T19:06:18.054020+0000","last_active":"2024-09-20T19:06:45.698314+0000","last_peered":"2024-09-20T19:06:45.698314+0000","last_clean":"2024-09-20T19:06:45.698314+0000","last_became_active":"2024-09-20T19:06:18.053343+0000","last_became_peered":"2024-09-20T19:06:18.053343+0000","last_unstale":"2024-09-20T19:06:45.698314+0000","last_undegraded":"2024-09-20T19:06:45.698314+0000","last_fullsized":"2024-09-20T19:06:45.698314+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":2963,"log_dups_size":0,"ondisk_log_size":2963,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:33.217188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":194183168,"num_objects":2963,"num_object_clones":0,"num_object_copies":8889,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2963,"num_write_kb":189632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"274'28","reported_seq":150,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.321187+0000","last_change":"2024-09-20T19:04:20.976682+0000","last_active":"2024-09-20T19:06:42.321187+0000","last_peered":"2024-09-20T19:06:42.321187+0000","last_clean":"2024-09-20T19:06:42.321187+0000","last_became_active":"2024-09-20T19:04:20.897444+0000","last_became_peered":"2024-09-20T19:04:20.897444+0000","last_unstale":"2024-09-20T19:06:42.321187+0000","last_undegraded":"2024-09-20T19:06:42.321187+0000","last_fullsized":"2024-09-20T19:06:42.321187+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'28","last_scrub_stamp":"2024-09-20T19:03:06.269623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:06.269623+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":41,"num_read_kb":34,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":837,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.334480+0000","last_change":"2024-09-20T19:05:47.629956+0000","last_active":"2024-09-20T19:06:42.334480+0000","last_peered":"2024-09-20T19:06:42.334480+0000","last_clean":"2024-09-20T19:06:42.334480+0000","last_became_active":"2024-09-20T19:04:20.906790+0000","last_became_peered":"2024-09-20T19:04:20.906790+0000","last_unstale":"2024-09-20T19:06:42.334480+0000","last_undegraded":"2024-09-20T19:06:42.334480+0000","last_fullsized":"2024-09-20T19:06:42.334480+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:05:47.629892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:47.629892+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:06.773486+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"375'13","reported_seq":1054,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.328508+0000","last_change":"2024-09-20T19:04:20.916872+0000","last_active":"2024-09-20T19:06:42.328508+0000","last_peered":"2024-09-20T19:06:42.328508+0000","last_clean":"2024-09-20T19:06:42.328508+0000","last_became_active":"2024-09-20T19:04:20.896424+0000","last_became_peered":"2024-09-20T19:04:20.896424+0000","last_unstale":"2024-09-20T19:06:42.328508+0000","last_undegraded":"2024-09-20T19:06:42.328508+0000","last_fullsized":"2024-09-20T19:06:42.328508+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:58:23.114371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:23.114371+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:39.902774+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":796,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:40.315059+0000","last_change":"2024-09-20T18:59:05.082117+0000","last_active":"2024-09-20T19:06:40.315059+0000","last_peered":"2024-09-20T19:06:40.315059+0000","last_clean":"2024-09-20T19:06:40.315059+0000","last_became_active":"2024-09-20T18:59:05.081814+0000","last_became_peered":"2024-09-20T18:59:05.081814+0000","last_unstale":"2024-09-20T19:06:40.315059+0000","last_undegraded":"2024-09-20T19:06:40.315059+0000","last_fullsized":"2024-09-20T19:06:40.315059+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:29.530001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:29.530001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:38.835871+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.d","version":"435'2500","reported_seq":2535,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:41.666700+0000","last_change":"2024-09-20T19:06:18.053626+0000","last_active":"2024-09-20T19:06:41.666700+0000","last_peered":"2024-09-20T19:06:41.666700+0000","last_clean":"2024-09-20T19:06:41.666700+0000","last_became_active":"2024-09-20T19:06:18.053336+0000","last_became_peered":"2024-09-20T19:06:18.053336+0000","last_unstale":"2024-09-20T19:06:41.666700+0000","last_undegraded":"2024-09-20T19:06:41.666700+0000","last_fullsized":"2024-09-20T19:06:41.666700+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":2500,"log_dups_size":0,"ondisk_log_size":2500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:26.656015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163840000,"num_objects":2500,"num_object_clones":0,"num_object_copies":7500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2500,"num_write_kb":160000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"274'20","reported_seq":648,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.333078+0000","last_change":"2024-09-20T19:05:48.741237+0000","last_active":"2024-09-20T19:06:42.333078+0000","last_peered":"2024-09-20T19:06:42.333078+0000","last_clean":"2024-09-20T19:06:42.333078+0000","last_became_active":"2024-09-20T19:04:49.419140+0000","last_became_peered":"2024-09-20T19:04:49.419140+0000","last_unstale":"2024-09-20T19:06:42.333078+0000","last_undegraded":"2024-09-20T19:06:42.333078+0000","last_fullsized":"2024-09-20T19:06:42.333078+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'20","last_scrub_stamp":"2024-09-20T19:05:48.741059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:48.741059+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:13.202074+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":827,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.334049+0000","last_change":"2024-09-20T19:05:56.820141+0000","last_active":"2024-09-20T19:06:42.334049+0000","last_peered":"2024-09-20T19:06:42.334049+0000","last_clean":"2024-09-20T19:06:42.334049+0000","last_became_active":"2024-09-20T18:49:45.942496+0000","last_became_peered":"2024-09-20T18:49:45.942496+0000","last_unstale":"2024-09-20T19:06:42.334049+0000","last_undegraded":"2024-09-20T19:06:42.334049+0000","last_fullsized":"2024-09-20T19:06:42.334049+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:56.820079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:56.820079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:16.442542+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"262'10","reported_seq":1030,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.333437+0000","last_change":"2024-09-20T19:02:04.994109+0000","last_active":"2024-09-20T19:06:42.333437+0000","last_peered":"2024-09-20T19:06:42.333437+0000","last_clean":"2024-09-20T19:06:42.333437+0000","last_became_active":"2024-09-20T18:59:05.080183+0000","last_became_peered":"2024-09-20T18:59:05.080183+0000","last_unstale":"2024-09-20T19:06:42.333437+0000","last_undegraded":"2024-09-20T19:06:42.333437+0000","last_fullsized":"2024-09-20T19:06:42.333437+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'10","last_scrub_stamp":"2024-09-20T19:02:04.994047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:04.994047+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:32.731999+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":784,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.333000+0000","last_change":"2024-09-20T19:06:15.671414+0000","last_active":"2024-09-20T19:06:42.333000+0000","last_peered":"2024-09-20T19:06:42.333000+0000","last_clean":"2024-09-20T19:06:42.333000+0000","last_became_active":"2024-09-20T18:49:51.964780+0000","last_became_peered":"2024-09-20T18:49:51.964780+0000","last_unstale":"2024-09-20T19:06:42.333000+0000","last_undegraded":"2024-09-20T19:06:42.333000+0000","last_fullsized":"2024-09-20T19:06:42.333000+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:15.671352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:06:15.671352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:25.558790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"435'2550","reported_seq":2587,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.054544+0000","last_change":"2024-09-20T19:06:18.050185+0000","last_active":"2024-09-20T19:06:42.054544+0000","last_peered":"2024-09-20T19:06:42.054544+0000","last_clean":"2024-09-20T19:06:42.054544+0000","last_became_active":"2024-09-20T19:06:18.049949+0000","last_became_peered":"2024-09-20T19:06:18.049949+0000","last_unstale":"2024-09-20T19:06:42.054544+0000","last_undegraded":"2024-09-20T19:06:42.054544+0000","last_fullsized":"2024-09-20T19:06:42.054544+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":2550,"log_dups_size":0,"ondisk_log_size":2550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:27.811048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":167116800,"num_objects":2550,"num_object_clones":0,"num_object_copies":7650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2550,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2550,"num_write_kb":163200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,4],"acting":[1,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"274'29","reported_seq":851,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.333619+0000","last_change":"2024-09-20T19:05:43.631990+0000","last_active":"2024-09-20T19:06:42.333619+0000","last_peered":"2024-09-20T19:06:42.333619+0000","last_clean":"2024-09-20T19:06:42.333619+0000","last_became_active":"2024-09-20T18:49:48.497736+0000","last_became_peered":"2024-09-20T18:49:48.497736+0000","last_unstale":"2024-09-20T19:06:42.333619+0000","last_undegraded":"2024-09-20T19:06:42.333619+0000","last_fullsized":"2024-09-20T19:06:42.333619+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'29","last_scrub_stamp":"2024-09-20T19:05:43.631845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:43.631845+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:56.642378+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":801,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:40.314866+0000","last_change":"2024-09-20T19:05:47.837587+0000","last_active":"2024-09-20T19:06:40.314866+0000","last_peered":"2024-09-20T19:06:40.314866+0000","last_clean":"2024-09-20T19:06:40.314866+0000","last_became_active":"2024-09-20T18:49:45.942253+0000","last_became_peered":"2024-09-20T18:49:45.942253+0000","last_unstale":"2024-09-20T19:06:40.314866+0000","last_undegraded":"2024-09-20T19:06:40.314866+0000","last_fullsized":"2024-09-20T19:06:40.314866+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:47.837520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:47.837520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:15.032955+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":387,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:40.320729+0000","last_change":"2024-09-20T19:05:48.203244+0000","last_active":"2024-09-20T19:06:40.320729+0000","last_peered":"2024-09-20T19:06:40.320729+0000","last_clean":"2024-09-20T19:06:40.320729+0000","last_became_active":"2024-09-20T18:59:05.085636+0000","last_became_peered":"2024-09-20T18:59:05.085636+0000","last_unstale":"2024-09-20T19:06:40.320729+0000","last_undegraded":"2024-09-20T19:06:40.320729+0000","last_fullsized":"2024-09-20T19:06:40.320729+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:48.203182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:05:48.203182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:58.736758+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":811,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:40.313723+0000","last_change":"2024-09-20T19:02:23.886386+0000","last_active":"2024-09-20T19:06:40.313723+0000","last_peered":"2024-09-20T19:06:40.313723+0000","last_clean":"2024-09-20T19:06:40.313723+0000","last_became_active":"2024-09-20T18:59:05.085131+0000","last_became_peered":"2024-09-20T18:59:05.085131+0000","last_unstale":"2024-09-20T19:06:40.313723+0000","last_undegraded":"2024-09-20T19:06:40.313723+0000","last_fullsized":"2024-09-20T19:06:40.313723+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:23.886308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:23.886308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.b","version":"436'2845","reported_seq":2884,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:45.789062+0000","last_change":"2024-09-20T19:06:18.057411+0000","last_active":"2024-09-20T19:06:45.789062+0000","last_peered":"2024-09-20T19:06:45.789062+0000","last_clean":"2024-09-20T19:06:45.789062+0000","last_became_active":"2024-09-20T19:06:18.057129+0000","last_became_peered":"2024-09-20T19:06:18.057129+0000","last_unstale":"2024-09-20T19:06:45.789062+0000","last_undegraded":"2024-09-20T19:06:45.789062+0000","last_fullsized":"2024-09-20T19:06:45.789062+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":2845,"log_dups_size":0,"ondisk_log_size":2845,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:39.374025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":186449920,"num_objects":2845,"num_object_clones":0,"num_object_copies":8535,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2845,"num_write_kb":182080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"353'41","reported_seq":1112,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:40.314688+0000","last_change":"2024-09-20T19:05:48.855983+0000","last_active":"2024-09-20T19:06:40.314688+0000","last_peered":"2024-09-20T19:06:40.314688+0000","last_clean":"2024-09-20T19:06:40.314688+0000","last_became_active":"2024-09-20T19:03:16.976916+0000","last_became_peered":"2024-09-20T19:03:16.976916+0000","last_unstale":"2024-09-20T19:06:40.314688+0000","last_undegraded":"2024-09-20T19:06:40.314688+0000","last_fullsized":"2024-09-20T19:06:40.314688+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"353'41","last_scrub_stamp":"2024-09-20T19:05:48.855876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:48.855876+0000","objects_scrubbed":10,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:09.142116+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":142,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.324721+0000","last_change":"2024-09-20T19:06:15.168347+0000","last_active":"2024-09-20T19:06:42.324721+0000","last_peered":"2024-09-20T19:06:42.324721+0000","last_clean":"2024-09-20T19:06:42.324721+0000","last_became_active":"2024-09-20T19:04:20.899352+0000","last_became_peered":"2024-09-20T19:04:20.899352+0000","last_unstale":"2024-09-20T19:06:42.324721+0000","last_undegraded":"2024-09-20T19:06:42.324721+0000","last_fullsized":"2024-09-20T19:06:42.324721+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:15.168293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:06:15.168293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:21.325884+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":859,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.334064+0000","last_change":"2024-09-20T19:03:11.910684+0000","last_active":"2024-09-20T19:06:42.334064+0000","last_peered":"2024-09-20T19:06:42.334064+0000","last_clean":"2024-09-20T19:06:42.334064+0000","last_became_active":"2024-09-20T18:59:05.086786+0000","last_became_peered":"2024-09-20T18:59:05.086786+0000","last_unstale":"2024-09-20T19:06:42.334064+0000","last_undegraded":"2024-09-20T19:06:42.334064+0000","last_fullsized":"2024-09-20T19:06:42.334064+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-20T19:03:11.909394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:03:11.909394+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:12.871941+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0.0012781159999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":829,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.333630+0000","last_change":"2024-09-20T19:04:20.907534+0000","last_active":"2024-09-20T19:06:42.333630+0000","last_peered":"2024-09-20T19:06:42.333630+0000","last_clean":"2024-09-20T19:06:42.333630+0000","last_became_active":"2024-09-20T19:04:20.907038+0000","last_became_peered":"2024-09-20T19:04:20.907038+0000","last_unstale":"2024-09-20T19:06:42.333630+0000","last_undegraded":"2024-09-20T19:06:42.333630+0000","last_fullsized":"2024-09-20T19:06:42.333630+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:15.034340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:03:15.034340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":799,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.334594+0000","last_change":"2024-09-20T19:04:20.896616+0000","last_active":"2024-09-20T19:06:42.334594+0000","last_peered":"2024-09-20T19:06:42.334594+0000","last_clean":"2024-09-20T19:06:42.334594+0000","last_became_active":"2024-09-20T19:04:20.896281+0000","last_became_peered":"2024-09-20T19:04:20.896281+0000","last_unstale":"2024-09-20T19:06:42.334594+0000","last_undegraded":"2024-09-20T19:06:42.334594+0000","last_fullsized":"2024-09-20T19:06:42.334594+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:04.526683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:04.526683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:12.801724+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"436'2836","reported_seq":2875,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:45.767308+0000","last_change":"2024-09-20T19:06:18.054306+0000","last_active":"2024-09-20T19:06:45.767308+0000","last_peered":"2024-09-20T19:06:45.767308+0000","last_clean":"2024-09-20T19:06:45.767308+0000","last_became_active":"2024-09-20T19:06:18.054031+0000","last_became_peered":"2024-09-20T19:06:18.054031+0000","last_unstale":"2024-09-20T19:06:45.767308+0000","last_undegraded":"2024-09-20T19:06:45.767308+0000","last_fullsized":"2024-09-20T19:06:45.767308+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":2836,"log_dups_size":0,"ondisk_log_size":2836,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:32.510710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":185860096,"num_objects":2836,"num_object_clones":0,"num_object_copies":8508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2836,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2836,"num_write_kb":181504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"273'17","reported_seq":845,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:40.315641+0000","last_change":"2024-09-20T19:05:43.974098+0000","last_active":"2024-09-20T19:06:40.315641+0000","last_peered":"2024-09-20T19:06:40.315641+0000","last_clean":"2024-09-20T19:06:40.315641+0000","last_became_active":"2024-09-20T18:49:47.950904+0000","last_became_peered":"2024-09-20T18:49:47.950904+0000","last_unstale":"2024-09-20T19:06:40.315641+0000","last_undegraded":"2024-09-20T19:06:40.315641+0000","last_fullsized":"2024-09-20T19:06:40.315641+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'17","last_scrub_stamp":"2024-09-20T19:05:43.974013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:43.974013+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:01.950625+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":36,"num_read_kb":25,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":823,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:40.313979+0000","last_change":"2024-09-20T19:02:29.840761+0000","last_active":"2024-09-20T19:06:40.313979+0000","last_peered":"2024-09-20T19:06:40.313979+0000","last_clean":"2024-09-20T19:06:40.313979+0000","last_became_active":"2024-09-20T18:59:05.079789+0000","last_became_peered":"2024-09-20T18:59:05.079789+0000","last_unstale":"2024-09-20T19:06:40.313979+0000","last_undegraded":"2024-09-20T19:06:40.313979+0000","last_fullsized":"2024-09-20T19:06:40.313979+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:29.840601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:29.840601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:35.837533+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":813,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:40.312435+0000","last_change":"2024-09-20T19:06:13.599431+0000","last_active":"2024-09-20T19:06:40.312435+0000","last_peered":"2024-09-20T19:06:40.312435+0000","last_clean":"2024-09-20T19:06:40.312435+0000","last_became_active":"2024-09-20T18:49:44.937135+0000","last_became_peered":"2024-09-20T18:49:44.937135+0000","last_unstale":"2024-09-20T19:06:40.312435+0000","last_undegraded":"2024-09-20T19:06:40.312435+0000","last_fullsized":"2024-09-20T19:06:40.312435+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:06:13.599376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:06:13.599376+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:27.051567+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00095558800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":779,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:40.313181+0000","last_change":"2024-09-20T19:01:17.747452+0000","last_active":"2024-09-20T19:06:40.313181+0000","last_peered":"2024-09-20T19:06:40.313181+0000","last_clean":"2024-09-20T19:06:40.313181+0000","last_became_active":"2024-09-20T18:49:49.954424+0000","last_became_peered":"2024-09-20T18:49:49.954424+0000","last_unstale":"2024-09-20T19:06:40.313181+0000","last_undegraded":"2024-09-20T19:06:40.313181+0000","last_fullsized":"2024-09-20T19:06:40.313181+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:17.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:17.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:58.515630+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":791,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:40.315378+0000","last_change":"2024-09-20T19:01:15.203591+0000","last_active":"2024-09-20T19:06:40.315378+0000","last_peered":"2024-09-20T19:06:40.315378+0000","last_clean":"2024-09-20T19:06:40.315378+0000","last_became_active":"2024-09-20T18:59:05.081459+0000","last_became_peered":"2024-09-20T18:59:05.081459+0000","last_unstale":"2024-09-20T19:06:40.315378+0000","last_undegraded":"2024-09-20T19:06:40.315378+0000","last_fullsized":"2024-09-20T19:06:40.315378+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:15.203529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:15.203529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"436'2978","reported_seq":3017,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:45.798963+0000","last_change":"2024-09-20T19:06:18.055443+0000","last_active":"2024-09-20T19:06:45.798963+0000","last_peered":"2024-09-20T19:06:45.798963+0000","last_clean":"2024-09-20T19:06:45.798963+0000","last_became_active":"2024-09-20T19:06:18.055137+0000","last_became_peered":"2024-09-20T19:06:18.055137+0000","last_unstale":"2024-09-20T19:06:45.798963+0000","last_undegraded":"2024-09-20T19:06:45.798963+0000","last_fullsized":"2024-09-20T19:06:45.798963+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":2978,"log_dups_size":0,"ondisk_log_size":2978,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:32.593377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":195166208,"num_objects":2978,"num_object_clones":0,"num_object_copies":8934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2978,"num_write_kb":190592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"273'24","reported_seq":843,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.334332+0000","last_change":"2024-09-20T18:59:06.022021+0000","last_active":"2024-09-20T19:06:42.334332+0000","last_peered":"2024-09-20T19:06:42.334332+0000","last_clean":"2024-09-20T19:06:42.334332+0000","last_became_active":"2024-09-20T18:49:47.949230+0000","last_became_peered":"2024-09-20T18:49:47.949230+0000","last_unstale":"2024-09-20T19:06:42.334332+0000","last_undegraded":"2024-09-20T19:06:42.334332+0000","last_fullsized":"2024-09-20T19:06:42.334332+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:59:06.021962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:59:06.021962+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:35.720209+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":829,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.336245+0000","last_change":"2024-09-20T19:04:20.907250+0000","last_active":"2024-09-20T19:06:42.336245+0000","last_peered":"2024-09-20T19:06:42.336245+0000","last_clean":"2024-09-20T19:06:42.336245+0000","last_became_active":"2024-09-20T19:04:20.906969+0000","last_became_peered":"2024-09-20T19:04:20.906969+0000","last_unstale":"2024-09-20T19:06:42.336245+0000","last_undegraded":"2024-09-20T19:06:42.336245+0000","last_fullsized":"2024-09-20T19:06:42.336245+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:24.984022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:24.984022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:30.762279+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":1821,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:40.314946+0000","last_change":"2024-09-20T19:02:02.494344+0000","last_active":"2024-09-20T19:06:40.314946+0000","last_peered":"2024-09-20T19:06:40.314946+0000","last_clean":"2024-09-20T19:06:40.314946+0000","last_became_active":"2024-09-20T18:49:44.936819+0000","last_became_peered":"2024-09-20T18:49:44.936819+0000","last_unstale":"2024-09-20T19:06:40.314946+0000","last_undegraded":"2024-09-20T19:06:40.314946+0000","last_fullsized":"2024-09-20T19:06:40.314946+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-20T19:02:02.494291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:02:02.494291+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0010996199999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1008,"num_read_kb":1003,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":810,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.334757+0000","last_change":"2024-09-20T19:05:51.743485+0000","last_active":"2024-09-20T19:06:42.334757+0000","last_peered":"2024-09-20T19:06:42.334757+0000","last_clean":"2024-09-20T19:06:42.334757+0000","last_became_active":"2024-09-20T18:49:49.961066+0000","last_became_peered":"2024-09-20T18:49:49.961066+0000","last_unstale":"2024-09-20T19:06:42.334757+0000","last_undegraded":"2024-09-20T19:06:42.334757+0000","last_fullsized":"2024-09-20T19:06:42.334757+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:51.743363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:05:51.743363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:06.075322+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":797,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:40.314424+0000","last_change":"2024-09-20T19:02:26.861253+0000","last_active":"2024-09-20T19:06:40.314424+0000","last_peered":"2024-09-20T19:06:40.314424+0000","last_clean":"2024-09-20T19:06:40.314424+0000","last_became_active":"2024-09-20T18:49:51.970343+0000","last_became_peered":"2024-09-20T18:49:51.970343+0000","last_unstale":"2024-09-20T19:06:40.314424+0000","last_undegraded":"2024-09-20T19:06:40.314424+0000","last_fullsized":"2024-09-20T19:06:40.314424+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:26.861185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:26.861185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:32.805240+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.a","version":"436'2902","reported_seq":2941,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:45.796384+0000","last_change":"2024-09-20T19:06:18.055447+0000","last_active":"2024-09-20T19:06:45.796384+0000","last_peered":"2024-09-20T19:06:45.796384+0000","last_clean":"2024-09-20T19:06:45.796384+0000","last_became_active":"2024-09-20T19:06:18.055189+0000","last_became_peered":"2024-09-20T19:06:18.055189+0000","last_unstale":"2024-09-20T19:06:45.796384+0000","last_undegraded":"2024-09-20T19:06:45.796384+0000","last_fullsized":"2024-09-20T19:06:45.796384+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":2902,"log_dups_size":0,"ondisk_log_size":2902,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:39.851662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":190185472,"num_objects":2902,"num_object_clones":0,"num_object_copies":8706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2902,"num_write_kb":185728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"375'34","reported_seq":570,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:40.320864+0000","last_change":"2024-09-20T19:05:54.115428+0000","last_active":"2024-09-20T19:06:40.320864+0000","last_peered":"2024-09-20T19:06:40.320864+0000","last_clean":"2024-09-20T19:06:40.320864+0000","last_became_active":"2024-09-20T19:04:20.928727+0000","last_became_peered":"2024-09-20T19:04:20.928727+0000","last_unstale":"2024-09-20T19:06:40.320864+0000","last_undegraded":"2024-09-20T19:06:40.320864+0000","last_fullsized":"2024-09-20T19:06:40.320864+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"375'34","last_scrub_stamp":"2024-09-20T19:05:54.115376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:54.115376+0000","objects_scrubbed":8,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:59.905707+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":462,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":825,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.335675+0000","last_change":"2024-09-20T19:05:07.665281+0000","last_active":"2024-09-20T19:06:42.335675+0000","last_peered":"2024-09-20T19:06:42.335675+0000","last_clean":"2024-09-20T19:06:42.335675+0000","last_became_active":"2024-09-20T18:59:05.079111+0000","last_became_peered":"2024-09-20T18:59:05.079111+0000","last_unstale":"2024-09-20T19:06:42.335675+0000","last_undegraded":"2024-09-20T19:06:42.335675+0000","last_fullsized":"2024-09-20T19:06:42.335675+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:07.665167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:07.665167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":870,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:40.315079+0000","last_change":"2024-09-20T19:03:14.368338+0000","last_active":"2024-09-20T19:06:40.315079+0000","last_peered":"2024-09-20T19:06:40.315079+0000","last_clean":"2024-09-20T19:06:40.315079+0000","last_became_active":"2024-09-20T18:59:05.086103+0000","last_became_peered":"2024-09-20T18:59:05.086103+0000","last_unstale":"2024-09-20T19:06:40.315079+0000","last_undegraded":"2024-09-20T19:06:40.315079+0000","last_fullsized":"2024-09-20T19:06:40.315079+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:03:14.368279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:03:14.368279+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:09.743853+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":794,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:40.315816+0000","last_change":"2024-09-20T19:06:21.584094+0000","last_active":"2024-09-20T19:06:40.315816+0000","last_peered":"2024-09-20T19:06:40.315816+0000","last_clean":"2024-09-20T19:06:40.315816+0000","last_became_active":"2024-09-20T18:49:49.956128+0000","last_became_peered":"2024-09-20T18:49:49.956128+0000","last_unstale":"2024-09-20T19:06:40.315816+0000","last_undegraded":"2024-09-20T19:06:40.315816+0000","last_fullsized":"2024-09-20T19:06:40.315816+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:21.584027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:06:21.584027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:29.362759+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":831,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:43.994634+0000","last_change":"2024-09-20T19:06:43.994634+0000","last_active":"2024-09-20T19:06:43.994634+0000","last_peered":"2024-09-20T19:06:43.994634+0000","last_clean":"2024-09-20T19:06:43.994634+0000","last_became_active":"2024-09-20T19:04:20.899321+0000","last_became_peered":"2024-09-20T19:04:20.899321+0000","last_unstale":"2024-09-20T19:06:43.994634+0000","last_undegraded":"2024-09-20T19:06:43.994634+0000","last_fullsized":"2024-09-20T19:06:43.994634+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:43.994470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:06:43.994470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:45.047924+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1","version":"435'2567","reported_seq":2604,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:41.950196+0000","last_change":"2024-09-20T19:06:18.056631+0000","last_active":"2024-09-20T19:06:41.950196+0000","last_peered":"2024-09-20T19:06:41.950196+0000","last_clean":"2024-09-20T19:06:41.950196+0000","last_became_active":"2024-09-20T19:06:18.056332+0000","last_became_peered":"2024-09-20T19:06:18.056332+0000","last_unstale":"2024-09-20T19:06:41.950196+0000","last_undegraded":"2024-09-20T19:06:41.950196+0000","last_fullsized":"2024-09-20T19:06:41.950196+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":2567,"log_dups_size":0,"ondisk_log_size":2567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:30.224100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":168230912,"num_objects":2567,"num_object_clones":0,"num_object_copies":7701,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2567,"num_write_kb":164288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"274'19","reported_seq":154,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.322366+0000","last_change":"2024-09-20T19:05:50.066533+0000","last_active":"2024-09-20T19:06:42.322366+0000","last_peered":"2024-09-20T19:06:42.322366+0000","last_clean":"2024-09-20T19:06:42.322366+0000","last_became_active":"2024-09-20T19:04:20.897175+0000","last_became_peered":"2024-09-20T19:04:20.897175+0000","last_unstale":"2024-09-20T19:06:42.322366+0000","last_undegraded":"2024-09-20T19:06:42.322366+0000","last_fullsized":"2024-09-20T19:06:42.322366+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'19","last_scrub_stamp":"2024-09-20T19:05:50.066359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:50.066359+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:56.712506+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":142,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.321382+0000","last_change":"2024-09-20T19:05:56.027187+0000","last_active":"2024-09-20T19:06:42.321382+0000","last_peered":"2024-09-20T19:06:42.321382+0000","last_clean":"2024-09-20T19:06:42.321382+0000","last_became_active":"2024-09-20T19:04:20.887102+0000","last_became_peered":"2024-09-20T19:04:20.887102+0000","last_unstale":"2024-09-20T19:06:42.321382+0000","last_undegraded":"2024-09-20T19:06:42.321382+0000","last_fullsized":"2024-09-20T19:06:42.321382+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:56.027123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:56.027123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:15.919213+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":841,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.334269+0000","last_change":"2024-09-20T19:04:20.899830+0000","last_active":"2024-09-20T19:06:42.334269+0000","last_peered":"2024-09-20T19:06:42.334269+0000","last_clean":"2024-09-20T19:06:42.334269+0000","last_became_active":"2024-09-20T19:04:20.899331+0000","last_became_peered":"2024-09-20T19:04:20.899331+0000","last_unstale":"2024-09-20T19:06:42.334269+0000","last_undegraded":"2024-09-20T19:06:42.334269+0000","last_fullsized":"2024-09-20T19:06:42.334269+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:38.463404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:38.463404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:30.881046+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":804,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.335272+0000","last_change":"2024-09-20T19:04:35.529672+0000","last_active":"2024-09-20T19:06:42.335272+0000","last_peered":"2024-09-20T19:06:42.335272+0000","last_clean":"2024-09-20T19:06:42.335272+0000","last_became_active":"2024-09-20T18:59:05.079897+0000","last_became_peered":"2024-09-20T18:59:05.079897+0000","last_unstale":"2024-09-20T19:06:42.335272+0000","last_undegraded":"2024-09-20T19:06:42.335272+0000","last_fullsized":"2024-09-20T19:06:42.335272+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:35.529584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:04:35.529584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:31.985657+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2","version":"435'2452","reported_seq":2489,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:41.665842+0000","last_change":"2024-09-20T19:06:18.054677+0000","last_active":"2024-09-20T19:06:41.665842+0000","last_peered":"2024-09-20T19:06:41.665842+0000","last_clean":"2024-09-20T19:06:41.665842+0000","last_became_active":"2024-09-20T19:06:18.054421+0000","last_became_peered":"2024-09-20T19:06:18.054421+0000","last_unstale":"2024-09-20T19:06:41.665842+0000","last_undegraded":"2024-09-20T19:06:41.665842+0000","last_fullsized":"2024-09-20T19:06:41.665842+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":2452,"log_dups_size":0,"ondisk_log_size":2452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:26.682986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":160694272,"num_objects":2452,"num_object_clones":0,"num_object_copies":7356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2452,"num_write_kb":156928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"274'31","reported_seq":152,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.325257+0000","last_change":"2024-09-20T19:04:20.949305+0000","last_active":"2024-09-20T19:06:42.325257+0000","last_peered":"2024-09-20T19:06:42.325257+0000","last_clean":"2024-09-20T19:06:42.325257+0000","last_became_active":"2024-09-20T19:04:20.883306+0000","last_became_peered":"2024-09-20T19:04:20.883306+0000","last_unstale":"2024-09-20T19:06:42.325257+0000","last_undegraded":"2024-09-20T19:06:42.325257+0000","last_fullsized":"2024-09-20T19:06:42.325257+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'31","last_scrub_stamp":"2024-09-20T19:03:12.196851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:12.196851+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":49,"num_read_kb":39,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":396,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:40.320867+0000","last_change":"2024-09-20T19:04:20.899712+0000","last_active":"2024-09-20T19:06:40.320867+0000","last_peered":"2024-09-20T19:06:40.320867+0000","last_clean":"2024-09-20T19:06:40.320867+0000","last_became_active":"2024-09-20T19:04:20.899415+0000","last_became_peered":"2024-09-20T19:04:20.899415+0000","last_unstale":"2024-09-20T19:06:40.320867+0000","last_undegraded":"2024-09-20T19:06:40.320867+0000","last_fullsized":"2024-09-20T19:06:40.320867+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:42.479489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:42.479489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":825,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:40.314517+0000","last_change":"2024-09-20T19:05:46.866958+0000","last_active":"2024-09-20T19:06:40.314517+0000","last_peered":"2024-09-20T19:06:40.314517+0000","last_clean":"2024-09-20T19:06:40.314517+0000","last_became_active":"2024-09-20T19:04:20.891772+0000","last_became_peered":"2024-09-20T19:04:20.891772+0000","last_unstale":"2024-09-20T19:06:40.314517+0000","last_undegraded":"2024-09-20T19:06:40.314517+0000","last_fullsized":"2024-09-20T19:06:40.314517+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:46.866777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:05:46.866777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:55.257561+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":824,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:40.318783+0000","last_change":"2024-09-20T19:06:25.628102+0000","last_active":"2024-09-20T19:06:40.318783+0000","last_peered":"2024-09-20T19:06:40.318783+0000","last_clean":"2024-09-20T19:06:40.318783+0000","last_became_active":"2024-09-20T19:04:20.897569+0000","last_became_peered":"2024-09-20T19:04:20.897569+0000","last_unstale":"2024-09-20T19:06:40.318783+0000","last_undegraded":"2024-09-20T19:06:40.318783+0000","last_fullsized":"2024-09-20T19:06:40.318783+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:25.628041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:06:25.628041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:36.219393+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"435'2574","reported_seq":2611,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:41.951149+0000","last_change":"2024-09-20T19:06:18.056510+0000","last_active":"2024-09-20T19:06:41.951149+0000","last_peered":"2024-09-20T19:06:41.951149+0000","last_clean":"2024-09-20T19:06:41.951149+0000","last_became_active":"2024-09-20T19:06:18.056187+0000","last_became_peered":"2024-09-20T19:06:18.056187+0000","last_unstale":"2024-09-20T19:06:41.951149+0000","last_undegraded":"2024-09-20T19:06:41.951149+0000","last_fullsized":"2024-09-20T19:06:41.951149+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":2574,"log_dups_size":0,"ondisk_log_size":2574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:27.399713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":168689664,"num_objects":2574,"num_object_clones":0,"num_object_copies":7722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2574,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2574,"num_write_kb":164736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1],"acting":[6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"273'20","reported_seq":146,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.321710+0000","last_change":"2024-09-20T19:04:20.945143+0000","last_active":"2024-09-20T19:06:42.321710+0000","last_peered":"2024-09-20T19:06:42.321710+0000","last_clean":"2024-09-20T19:06:42.321710+0000","last_became_active":"2024-09-20T19:04:20.896483+0000","last_became_peered":"2024-09-20T19:04:20.896483+0000","last_unstale":"2024-09-20T19:06:42.321710+0000","last_undegraded":"2024-09-20T19:06:42.321710+0000","last_fullsized":"2024-09-20T19:06:42.321710+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'20","last_scrub_stamp":"2024-09-20T19:03:04.282800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:04.282800+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":835,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.334770+0000","last_change":"2024-09-20T19:05:48.678438+0000","last_active":"2024-09-20T19:06:42.334770+0000","last_peered":"2024-09-20T19:06:42.334770+0000","last_clean":"2024-09-20T19:06:42.334770+0000","last_became_active":"2024-09-20T19:04:20.892951+0000","last_became_peered":"2024-09-20T19:04:20.892951+0000","last_unstale":"2024-09-20T19:06:42.334770+0000","last_undegraded":"2024-09-20T19:06:42.334770+0000","last_fullsized":"2024-09-20T19:06:42.334770+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:48.678361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:48.678361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:02.488299+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":1023,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:40.312388+0000","last_change":"2024-09-20T19:05:08.406307+0000","last_active":"2024-09-20T19:06:40.312388+0000","last_peered":"2024-09-20T19:06:40.312388+0000","last_clean":"2024-09-20T19:06:40.312388+0000","last_became_active":"2024-09-20T18:49:49.957005+0000","last_became_peered":"2024-09-20T18:49:49.957005+0000","last_unstale":"2024-09-20T19:06:40.312388+0000","last_undegraded":"2024-09-20T19:06:40.312388+0000","last_fullsized":"2024-09-20T19:06:40.312388+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T19:05:08.406231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:05:08.406231+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:32.637035+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":134,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.322446+0000","last_change":"2024-09-20T19:04:20.885294+0000","last_active":"2024-09-20T19:06:42.322446+0000","last_peered":"2024-09-20T19:06:42.322446+0000","last_clean":"2024-09-20T19:06:42.322446+0000","last_became_active":"2024-09-20T19:04:20.884994+0000","last_became_peered":"2024-09-20T19:04:20.884994+0000","last_unstale":"2024-09-20T19:06:42.322446+0000","last_undegraded":"2024-09-20T19:06:42.322446+0000","last_fullsized":"2024-09-20T19:06:42.322446+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:11.207994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:03:11.207994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.4","version":"435'2498","reported_seq":2535,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:41.373796+0000","last_change":"2024-09-20T19:06:18.055511+0000","last_active":"2024-09-20T19:06:41.373796+0000","last_peered":"2024-09-20T19:06:41.373796+0000","last_clean":"2024-09-20T19:06:41.373796+0000","last_became_active":"2024-09-20T19:06:18.055241+0000","last_became_peered":"2024-09-20T19:06:18.055241+0000","last_unstale":"2024-09-20T19:06:41.373796+0000","last_undegraded":"2024-09-20T19:06:41.373796+0000","last_fullsized":"2024-09-20T19:06:41.373796+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":2498,"log_dups_size":0,"ondisk_log_size":2498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:36.548677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163708928,"num_objects":2498,"num_object_clones":0,"num_object_copies":7494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2498,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2498,"num_write_kb":159872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"274'15","reported_seq":877,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.333531+0000","last_change":"2024-09-20T19:05:57.548758+0000","last_active":"2024-09-20T19:06:42.333531+0000","last_peered":"2024-09-20T19:06:42.333531+0000","last_clean":"2024-09-20T19:06:42.333531+0000","last_became_active":"2024-09-20T19:04:20.906834+0000","last_became_peered":"2024-09-20T19:04:20.906834+0000","last_unstale":"2024-09-20T19:06:42.333531+0000","last_undegraded":"2024-09-20T19:06:42.333531+0000","last_fullsized":"2024-09-20T19:06:42.333531+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'15","last_scrub_stamp":"2024-09-20T19:05:57.548683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:57.548683+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:13.922548+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":360,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:40.313523+0000","last_change":"2024-09-20T19:05:52.464362+0000","last_active":"2024-09-20T19:06:40.313523+0000","last_peered":"2024-09-20T19:06:40.313523+0000","last_clean":"2024-09-20T19:06:40.313523+0000","last_became_active":"2024-09-20T19:00:01.190875+0000","last_became_peered":"2024-09-20T19:00:01.190875+0000","last_unstale":"2024-09-20T19:06:40.313523+0000","last_undegraded":"2024-09-20T19:06:40.313523+0000","last_fullsized":"2024-09-20T19:06:40.313523+0000","mapping_epoch":277,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":278,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:52.464261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:52.464261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:59.446086+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":828,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:40.312649+0000","last_change":"2024-09-20T19:04:20.889614+0000","last_active":"2024-09-20T19:06:40.312649+0000","last_peered":"2024-09-20T19:06:40.312649+0000","last_clean":"2024-09-20T19:06:40.312649+0000","last_became_active":"2024-09-20T19:04:20.889273+0000","last_became_peered":"2024-09-20T19:04:20.889273+0000","last_unstale":"2024-09-20T19:06:40.312649+0000","last_undegraded":"2024-09-20T19:06:40.312649+0000","last_fullsized":"2024-09-20T19:06:40.312649+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:03.389611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:03.389611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:31.592842+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":791,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.336147+0000","last_change":"2024-09-20T19:02:31.120809+0000","last_active":"2024-09-20T19:06:42.336147+0000","last_peered":"2024-09-20T19:06:42.336147+0000","last_clean":"2024-09-20T19:06:42.336147+0000","last_became_active":"2024-09-20T18:49:51.964098+0000","last_became_peered":"2024-09-20T18:49:51.964098+0000","last_unstale":"2024-09-20T19:06:42.336147+0000","last_undegraded":"2024-09-20T19:06:42.336147+0000","last_fullsized":"2024-09-20T19:06:42.336147+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:31.120623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:31.120623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:40.760081+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.5","version":"435'2441","reported_seq":2477,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:41.399873+0000","last_change":"2024-09-20T19:06:18.053945+0000","last_active":"2024-09-20T19:06:41.399873+0000","last_peered":"2024-09-20T19:06:41.399873+0000","last_clean":"2024-09-20T19:06:41.399873+0000","last_became_active":"2024-09-20T19:06:18.053641+0000","last_became_peered":"2024-09-20T19:06:18.053641+0000","last_unstale":"2024-09-20T19:06:41.399873+0000","last_undegraded":"2024-09-20T19:06:41.399873+0000","last_fullsized":"2024-09-20T19:06:41.399873+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":2441,"log_dups_size":0,"ondisk_log_size":2441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:41.948840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":159973376,"num_objects":2441,"num_object_clones":0,"num_object_copies":7323,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2441,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2441,"num_write_kb":156224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"274'35","reported_seq":431,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:40.320635+0000","last_change":"2024-09-20T19:06:19.915841+0000","last_active":"2024-09-20T19:06:40.320635+0000","last_peered":"2024-09-20T19:06:40.320635+0000","last_clean":"2024-09-20T19:06:40.320635+0000","last_became_active":"2024-09-20T18:59:05.087391+0000","last_became_peered":"2024-09-20T18:59:05.087391+0000","last_unstale":"2024-09-20T19:06:40.320635+0000","last_undegraded":"2024-09-20T19:06:40.320635+0000","last_fullsized":"2024-09-20T19:06:40.320635+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'35","last_scrub_stamp":"2024-09-20T19:06:19.915767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:06:19.915767+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:46.153655+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":823,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:40.315883+0000","last_change":"2024-09-20T19:05:49.763478+0000","last_active":"2024-09-20T19:06:40.315883+0000","last_peered":"2024-09-20T19:06:40.315883+0000","last_clean":"2024-09-20T19:06:40.315883+0000","last_became_active":"2024-09-20T18:59:05.086670+0000","last_became_peered":"2024-09-20T18:59:05.086670+0000","last_unstale":"2024-09-20T19:06:40.315883+0000","last_undegraded":"2024-09-20T19:06:40.315883+0000","last_fullsized":"2024-09-20T19:06:40.315883+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:49.763407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:49.763407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:09.527897+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":811,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:40.313050+0000","last_change":"2024-09-20T19:05:11.444197+0000","last_active":"2024-09-20T19:06:40.313050+0000","last_peered":"2024-09-20T19:06:40.313050+0000","last_clean":"2024-09-20T19:06:40.313050+0000","last_became_active":"2024-09-20T18:49:49.957558+0000","last_became_peered":"2024-09-20T18:49:49.957558+0000","last_unstale":"2024-09-20T19:06:40.313050+0000","last_undegraded":"2024-09-20T19:06:40.313050+0000","last_fullsized":"2024-09-20T19:06:40.313050+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:11.444143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:05:11.444143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:36.686941+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":781,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:40.314925+0000","last_change":"2024-09-20T18:53:47.370152+0000","last_active":"2024-09-20T19:06:40.314925+0000","last_peered":"2024-09-20T19:06:40.314925+0000","last_clean":"2024-09-20T19:06:40.314925+0000","last_became_active":"2024-09-20T18:49:51.970798+0000","last_became_peered":"2024-09-20T18:49:51.970798+0000","last_unstale":"2024-09-20T19:06:40.314925+0000","last_undegraded":"2024-09-20T19:06:40.314925+0000","last_fullsized":"2024-09-20T19:06:40.314925+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:47.370108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:47.370108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:30.810770+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.6","version":"436'2612","reported_seq":2651,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.560018+0000","last_change":"2024-09-20T19:06:18.055118+0000","last_active":"2024-09-20T19:06:42.560018+0000","last_peered":"2024-09-20T19:06:42.560018+0000","last_clean":"2024-09-20T19:06:42.560018+0000","last_became_active":"2024-09-20T19:06:18.054843+0000","last_became_peered":"2024-09-20T19:06:18.054843+0000","last_unstale":"2024-09-20T19:06:42.560018+0000","last_undegraded":"2024-09-20T19:06:42.560018+0000","last_fullsized":"2024-09-20T19:06:42.560018+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":2612,"log_dups_size":0,"ondisk_log_size":2612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:21.237812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":171180032,"num_objects":2612,"num_object_clones":0,"num_object_copies":7836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2612,"num_write_kb":167168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"273'18","reported_seq":146,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.321569+0000","last_change":"2024-09-20T19:04:20.936465+0000","last_active":"2024-09-20T19:06:42.321569+0000","last_peered":"2024-09-20T19:06:42.321569+0000","last_clean":"2024-09-20T19:06:42.321569+0000","last_became_active":"2024-09-20T19:04:20.888905+0000","last_became_peered":"2024-09-20T19:04:20.888905+0000","last_unstale":"2024-09-20T19:06:42.321569+0000","last_undegraded":"2024-09-20T19:06:42.321569+0000","last_fullsized":"2024-09-20T19:06:42.321569+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'18","last_scrub_stamp":"2024-09-20T19:03:13.201592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:13.201592+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:29.171477+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":842,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:40.315554+0000","last_change":"2024-09-20T19:06:15.692558+0000","last_active":"2024-09-20T19:06:40.315554+0000","last_peered":"2024-09-20T19:06:40.315554+0000","last_clean":"2024-09-20T19:06:40.315554+0000","last_became_active":"2024-09-20T19:04:20.892429+0000","last_became_peered":"2024-09-20T19:04:20.892429+0000","last_unstale":"2024-09-20T19:06:40.315554+0000","last_undegraded":"2024-09-20T19:06:40.315554+0000","last_fullsized":"2024-09-20T19:06:40.315554+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:15.692508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:06:15.692508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:27.149889+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":803,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:40.312119+0000","last_change":"2024-09-20T19:04:48.406445+0000","last_active":"2024-09-20T19:06:40.312119+0000","last_peered":"2024-09-20T19:06:40.312119+0000","last_clean":"2024-09-20T19:06:40.312119+0000","last_became_active":"2024-09-20T18:49:49.954944+0000","last_became_peered":"2024-09-20T18:49:49.954944+0000","last_unstale":"2024-09-20T19:06:40.312119+0000","last_undegraded":"2024-09-20T19:06:40.312119+0000","last_fullsized":"2024-09-20T19:06:40.312119+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:48.406373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:48.406373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:59.483627+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":798,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.334326+0000","last_change":"2024-09-20T19:05:43.657860+0000","last_active":"2024-09-20T19:06:42.334326+0000","last_peered":"2024-09-20T19:06:42.334326+0000","last_clean":"2024-09-20T19:06:42.334326+0000","last_became_active":"2024-09-20T18:49:51.972970+0000","last_became_peered":"2024-09-20T18:49:51.972970+0000","last_unstale":"2024-09-20T19:06:42.334326+0000","last_undegraded":"2024-09-20T19:06:42.334326+0000","last_fullsized":"2024-09-20T19:06:42.334326+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:43.657780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:05:43.657780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:05.178691+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"436'2972","reported_seq":3011,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:45.789343+0000","last_change":"2024-09-20T19:06:18.053735+0000","last_active":"2024-09-20T19:06:45.789343+0000","last_peered":"2024-09-20T19:06:45.789343+0000","last_clean":"2024-09-20T19:06:45.789343+0000","last_became_active":"2024-09-20T19:06:18.053317+0000","last_became_peered":"2024-09-20T19:06:18.053317+0000","last_unstale":"2024-09-20T19:06:45.789343+0000","last_undegraded":"2024-09-20T19:06:45.789343+0000","last_fullsized":"2024-09-20T19:06:45.789343+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":2972,"log_dups_size":0,"ondisk_log_size":2972,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:21.054313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":194772992,"num_objects":2972,"num_object_clones":0,"num_object_copies":8916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2972,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2972,"num_write_kb":190208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"273'27","reported_seq":903,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.333365+0000","last_change":"2024-09-20T19:05:49.777488+0000","last_active":"2024-09-20T19:06:42.333365+0000","last_peered":"2024-09-20T19:06:42.333365+0000","last_clean":"2024-09-20T19:06:42.333365+0000","last_became_active":"2024-09-20T18:59:05.083726+0000","last_became_peered":"2024-09-20T18:59:05.083726+0000","last_unstale":"2024-09-20T19:06:42.333365+0000","last_undegraded":"2024-09-20T19:06:42.333365+0000","last_fullsized":"2024-09-20T19:06:42.333365+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'27","last_scrub_stamp":"2024-09-20T19:05:49.777414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:49.777414+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:12.438786+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":874,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:40.315621+0000","last_change":"2024-09-20T19:06:25.718817+0000","last_active":"2024-09-20T19:06:40.315621+0000","last_peered":"2024-09-20T19:06:40.315621+0000","last_clean":"2024-09-20T19:06:40.315621+0000","last_became_active":"2024-09-20T19:04:20.884854+0000","last_became_peered":"2024-09-20T19:04:20.884854+0000","last_unstale":"2024-09-20T19:06:40.315621+0000","last_undegraded":"2024-09-20T19:06:40.315621+0000","last_fullsized":"2024-09-20T19:06:40.315621+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-20T19:06:25.718731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:06:25.718731+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:49.878149+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"375'14","reported_seq":1062,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:40.314918+0000","last_change":"2024-09-20T19:06:14.693176+0000","last_active":"2024-09-20T19:06:40.314918+0000","last_peered":"2024-09-20T19:06:40.314918+0000","last_clean":"2024-09-20T19:06:40.314918+0000","last_became_active":"2024-09-20T19:04:20.892956+0000","last_became_peered":"2024-09-20T19:04:20.892956+0000","last_unstale":"2024-09-20T19:06:40.314918+0000","last_undegraded":"2024-09-20T19:06:40.314918+0000","last_fullsized":"2024-09-20T19:06:40.314918+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"375'14","last_scrub_stamp":"2024-09-20T19:06:14.693111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:06:14.693111+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:44.526972+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":812,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.332895+0000","last_change":"2024-09-20T19:03:06.888620+0000","last_active":"2024-09-20T19:06:42.332895+0000","last_peered":"2024-09-20T19:06:42.332895+0000","last_clean":"2024-09-20T19:06:42.332895+0000","last_became_active":"2024-09-20T18:59:05.087005+0000","last_became_peered":"2024-09-20T18:59:05.087005+0000","last_unstale":"2024-09-20T19:06:42.332895+0000","last_undegraded":"2024-09-20T19:06:42.332895+0000","last_fullsized":"2024-09-20T19:06:42.332895+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:06.888563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:03:06.888563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:09.776326+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":371,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:40.319983+0000","last_change":"2024-09-20T19:01:16.542398+0000","last_active":"2024-09-20T19:06:40.319983+0000","last_peered":"2024-09-20T19:06:40.319983+0000","last_clean":"2024-09-20T19:06:40.319983+0000","last_became_active":"2024-09-20T18:59:05.085427+0000","last_became_peered":"2024-09-20T18:59:05.085427+0000","last_unstale":"2024-09-20T19:06:40.319983+0000","last_undegraded":"2024-09-20T19:06:40.319983+0000","last_fullsized":"2024-09-20T19:06:40.319983+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:16.542335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:16.542335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:33.842644+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"273'12","reported_seq":832,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.335618+0000","last_change":"2024-09-20T19:06:19.973831+0000","last_active":"2024-09-20T19:06:42.335618+0000","last_peered":"2024-09-20T19:06:42.335618+0000","last_clean":"2024-09-20T19:06:42.335618+0000","last_became_active":"2024-09-20T18:49:48.500662+0000","last_became_peered":"2024-09-20T18:49:48.500662+0000","last_unstale":"2024-09-20T19:06:42.335618+0000","last_undegraded":"2024-09-20T19:06:42.335618+0000","last_fullsized":"2024-09-20T19:06:42.335618+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'12","last_scrub_stamp":"2024-09-20T19:06:19.973765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:06:19.973765+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:34.482020+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":802,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.333881+0000","last_change":"2024-09-20T19:05:22.605885+0000","last_active":"2024-09-20T19:06:42.333881+0000","last_peered":"2024-09-20T19:06:42.333881+0000","last_clean":"2024-09-20T19:06:42.333881+0000","last_became_active":"2024-09-20T18:49:49.956749+0000","last_became_peered":"2024-09-20T18:49:49.956749+0000","last_unstale":"2024-09-20T19:06:42.333881+0000","last_undegraded":"2024-09-20T19:06:42.333881+0000","last_fullsized":"2024-09-20T19:06:42.333881+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:22.605839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:05:22.605839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:49.241766+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":835,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:40.315339+0000","last_change":"2024-09-20T19:05:56.761494+0000","last_active":"2024-09-20T19:06:40.315339+0000","last_peered":"2024-09-20T19:06:40.315339+0000","last_clean":"2024-09-20T19:06:40.315339+0000","last_became_active":"2024-09-20T19:04:20.914115+0000","last_became_peered":"2024-09-20T19:04:20.914115+0000","last_unstale":"2024-09-20T19:06:40.315339+0000","last_undegraded":"2024-09-20T19:06:40.315339+0000","last_fullsized":"2024-09-20T19:06:40.315339+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:56.761425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:05:56.761425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:15.725837+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"273'15","reported_seq":815,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.334406+0000","last_change":"2024-09-20T19:01:15.875779+0000","last_active":"2024-09-20T19:06:42.334406+0000","last_peered":"2024-09-20T19:06:42.334406+0000","last_clean":"2024-09-20T19:06:42.334406+0000","last_became_active":"2024-09-20T18:49:48.500803+0000","last_became_peered":"2024-09-20T18:49:48.500803+0000","last_unstale":"2024-09-20T19:06:42.334406+0000","last_undegraded":"2024-09-20T19:06:42.334406+0000","last_fullsized":"2024-09-20T19:06:42.334406+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:01:15.875693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:15.875693+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:58.535131+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":840,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:40.315082+0000","last_change":"2024-09-20T19:04:47.713764+0000","last_active":"2024-09-20T19:06:40.315082+0000","last_peered":"2024-09-20T19:06:40.315082+0000","last_clean":"2024-09-20T19:06:40.315082+0000","last_became_active":"2024-09-20T19:04:20.893250+0000","last_became_peered":"2024-09-20T19:04:20.893250+0000","last_unstale":"2024-09-20T19:06:40.315082+0000","last_undegraded":"2024-09-20T19:06:40.315082+0000","last_fullsized":"2024-09-20T19:06:40.315082+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:47.713585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:04:47.713585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:10.723942+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":801,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.335302+0000","last_change":"2024-09-20T19:04:50.573619+0000","last_active":"2024-09-20T19:06:42.335302+0000","last_peered":"2024-09-20T19:06:42.335302+0000","last_clean":"2024-09-20T19:06:42.335302+0000","last_became_active":"2024-09-20T18:49:49.961947+0000","last_became_peered":"2024-09-20T18:49:49.961947+0000","last_unstale":"2024-09-20T19:06:42.335302+0000","last_undegraded":"2024-09-20T19:06:42.335302+0000","last_fullsized":"2024-09-20T19:06:42.335302+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:50.573550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:50.573550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:33.984555+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"66'1","reported_seq":841,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:40.313851+0000","last_change":"2024-09-20T19:04:20.913449+0000","last_active":"2024-09-20T19:06:40.313851+0000","last_peered":"2024-09-20T19:06:40.313851+0000","last_clean":"2024-09-20T19:06:40.313851+0000","last_became_active":"2024-09-20T19:04:20.892236+0000","last_became_peered":"2024-09-20T19:04:20.892236+0000","last_unstale":"2024-09-20T19:06:40.313851+0000","last_undegraded":"2024-09-20T19:06:40.313851+0000","last_fullsized":"2024-09-20T19:06:40.313851+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T19:03:12.499684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:03:12.499684+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"274'15","reported_seq":823,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.967192+0000","last_change":"2024-09-20T19:06:42.967192+0000","last_active":"2024-09-20T19:06:42.967192+0000","last_peered":"2024-09-20T19:06:42.967192+0000","last_clean":"2024-09-20T19:06:42.967192+0000","last_became_active":"2024-09-20T18:49:48.501011+0000","last_became_peered":"2024-09-20T18:49:48.501011+0000","last_unstale":"2024-09-20T19:06:42.967192+0000","last_undegraded":"2024-09-20T19:06:42.967192+0000","last_fullsized":"2024-09-20T19:06:42.967192+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'15","last_scrub_stamp":"2024-09-20T19:06:42.967003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:06:42.967003+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:59.075700+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":803,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.333584+0000","last_change":"2024-09-20T19:01:30.864866+0000","last_active":"2024-09-20T19:06:42.333584+0000","last_peered":"2024-09-20T19:06:42.333584+0000","last_clean":"2024-09-20T19:06:42.333584+0000","last_became_active":"2024-09-20T18:49:45.938511+0000","last_became_peered":"2024-09-20T18:49:45.938511+0000","last_unstale":"2024-09-20T19:06:42.333584+0000","last_undegraded":"2024-09-20T19:06:42.333584+0000","last_fullsized":"2024-09-20T19:06:42.333584+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:30.864794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:30.864794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:11.643706+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":1014,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.335349+0000","last_change":"2024-09-20T19:04:41.601208+0000","last_active":"2024-09-20T19:06:42.335349+0000","last_peered":"2024-09-20T19:06:42.335349+0000","last_clean":"2024-09-20T19:06:42.335349+0000","last_became_active":"2024-09-20T18:49:49.960550+0000","last_became_peered":"2024-09-20T18:49:49.960550+0000","last_unstale":"2024-09-20T19:06:42.335349+0000","last_undegraded":"2024-09-20T19:06:42.335349+0000","last_fullsized":"2024-09-20T19:06:42.335349+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T19:04:41.601073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:41.601073+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:34.939420+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":134,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.322078+0000","last_change":"2024-09-20T19:04:20.884397+0000","last_active":"2024-09-20T19:06:42.322078+0000","last_peered":"2024-09-20T19:06:42.322078+0000","last_clean":"2024-09-20T19:06:42.322078+0000","last_became_active":"2024-09-20T19:04:20.884100+0000","last_became_peered":"2024-09-20T19:04:20.884100+0000","last_unstale":"2024-09-20T19:06:42.322078+0000","last_undegraded":"2024-09-20T19:06:42.322078+0000","last_fullsized":"2024-09-20T19:06:42.322078+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:59.497810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:59:59.497810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:02.999076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"274'20","reported_seq":831,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.335601+0000","last_change":"2024-09-20T18:57:20.701592+0000","last_active":"2024-09-20T19:06:42.335601+0000","last_peered":"2024-09-20T19:06:42.335601+0000","last_clean":"2024-09-20T19:06:42.335601+0000","last_became_active":"2024-09-20T18:49:58.997494+0000","last_became_peered":"2024-09-20T18:49:58.997494+0000","last_unstale":"2024-09-20T19:06:42.335601+0000","last_undegraded":"2024-09-20T19:06:42.335601+0000","last_fullsized":"2024-09-20T19:06:42.335601+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:57:20.701543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:20.701543+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:28.869204+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":818,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:40.315314+0000","last_change":"2024-09-20T19:04:20.884940+0000","last_active":"2024-09-20T19:06:40.315314+0000","last_peered":"2024-09-20T19:06:40.315314+0000","last_clean":"2024-09-20T19:06:40.315314+0000","last_became_active":"2024-09-20T19:04:20.884582+0000","last_became_peered":"2024-09-20T19:04:20.884582+0000","last_unstale":"2024-09-20T19:06:40.315314+0000","last_undegraded":"2024-09-20T19:06:40.315314+0000","last_fullsized":"2024-09-20T19:06:40.315314+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.632380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.632380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:36.764394+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":1014,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.333632+0000","last_change":"2024-09-20T19:06:26.893183+0000","last_active":"2024-09-20T19:06:42.333632+0000","last_peered":"2024-09-20T19:06:42.333632+0000","last_clean":"2024-09-20T19:06:42.333632+0000","last_became_active":"2024-09-20T18:49:49.962243+0000","last_became_peered":"2024-09-20T18:49:49.962243+0000","last_unstale":"2024-09-20T19:06:42.333632+0000","last_undegraded":"2024-09-20T19:06:42.333632+0000","last_fullsized":"2024-09-20T19:06:42.333632+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T19:06:26.893109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:06:26.893109+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:34.296447+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":806,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.333975+0000","last_change":"2024-09-20T19:05:52.578042+0000","last_active":"2024-09-20T19:06:42.333975+0000","last_peered":"2024-09-20T19:06:42.333975+0000","last_clean":"2024-09-20T19:06:42.333975+0000","last_became_active":"2024-09-20T18:49:51.969093+0000","last_became_peered":"2024-09-20T18:49:51.969093+0000","last_unstale":"2024-09-20T19:06:42.333975+0000","last_undegraded":"2024-09-20T19:06:42.333975+0000","last_fullsized":"2024-09-20T19:06:42.333975+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:52.577979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:05:52.577979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:16.347110+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"273'19","reported_seq":154,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:44.432152+0000","last_change":"2024-09-20T19:06:44.432152+0000","last_active":"2024-09-20T19:06:44.432152+0000","last_peered":"2024-09-20T19:06:44.432152+0000","last_clean":"2024-09-20T19:06:44.432152+0000","last_became_active":"2024-09-20T19:04:20.897330+0000","last_became_peered":"2024-09-20T19:04:20.897330+0000","last_unstale":"2024-09-20T19:06:44.432152+0000","last_undegraded":"2024-09-20T19:06:44.432152+0000","last_fullsized":"2024-09-20T19:06:44.432152+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'19","last_scrub_stamp":"2024-09-20T19:06:44.432058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:06:44.432058+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:44.463553+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":134,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.321807+0000","last_change":"2024-09-20T19:04:20.896732+0000","last_active":"2024-09-20T19:06:42.321807+0000","last_peered":"2024-09-20T19:06:42.321807+0000","last_clean":"2024-09-20T19:06:42.321807+0000","last_became_active":"2024-09-20T19:04:20.896007+0000","last_became_peered":"2024-09-20T19:04:20.896007+0000","last_unstale":"2024-09-20T19:06:42.321807+0000","last_undegraded":"2024-09-20T19:06:42.321807+0000","last_fullsized":"2024-09-20T19:06:42.321807+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:06.310027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:59:06.310027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:02.046888+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":792,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:40.312022+0000","last_change":"2024-09-20T19:02:38.906352+0000","last_active":"2024-09-20T19:06:40.312022+0000","last_peered":"2024-09-20T19:06:40.312022+0000","last_clean":"2024-09-20T19:06:40.312022+0000","last_became_active":"2024-09-20T18:49:49.957067+0000","last_became_peered":"2024-09-20T18:49:49.957067+0000","last_unstale":"2024-09-20T19:06:40.312022+0000","last_undegraded":"2024-09-20T19:06:40.312022+0000","last_fullsized":"2024-09-20T19:06:40.312022+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:38.906161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:38.906161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:37.858190+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":774,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.330235+0000","last_change":"2024-09-20T18:57:16.789412+0000","last_active":"2024-09-20T19:06:42.330235+0000","last_peered":"2024-09-20T19:06:42.330235+0000","last_clean":"2024-09-20T19:06:42.330235+0000","last_became_active":"2024-09-20T18:49:51.966915+0000","last_became_peered":"2024-09-20T18:49:51.966915+0000","last_unstale":"2024-09-20T19:06:42.330235+0000","last_undegraded":"2024-09-20T19:06:42.330235+0000","last_fullsized":"2024-09-20T19:06:42.330235+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:16.789095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:16.789095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:37.856372+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"273'15","reported_seq":863,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:40.312187+0000","last_change":"2024-09-20T19:02:30.816061+0000","last_active":"2024-09-20T19:06:40.312187+0000","last_peered":"2024-09-20T19:06:40.312187+0000","last_clean":"2024-09-20T19:06:40.312187+0000","last_became_active":"2024-09-20T18:59:05.085230+0000","last_became_peered":"2024-09-20T18:59:05.085230+0000","last_unstale":"2024-09-20T19:06:40.312187+0000","last_undegraded":"2024-09-20T19:06:40.312187+0000","last_fullsized":"2024-09-20T19:06:40.312187+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:02:30.815747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:30.815747+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":817,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:40.315734+0000","last_change":"2024-09-20T19:04:20.885127+0000","last_active":"2024-09-20T19:06:40.315734+0000","last_peered":"2024-09-20T19:06:40.315734+0000","last_clean":"2024-09-20T19:06:40.315734+0000","last_became_active":"2024-09-20T19:04:20.884660+0000","last_became_peered":"2024-09-20T19:04:20.884660+0000","last_unstale":"2024-09-20T19:06:40.315734+0000","last_undegraded":"2024-09-20T19:06:40.315734+0000","last_fullsized":"2024-09-20T19:06:40.315734+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:07.408197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:03:07.408197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:07.715846+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":810,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.334404+0000","last_change":"2024-09-20T19:06:22.930775+0000","last_active":"2024-09-20T19:06:42.334404+0000","last_peered":"2024-09-20T19:06:42.334404+0000","last_clean":"2024-09-20T19:06:42.334404+0000","last_became_active":"2024-09-20T18:49:49.962523+0000","last_became_peered":"2024-09-20T18:49:49.962523+0000","last_unstale":"2024-09-20T19:06:42.334404+0000","last_undegraded":"2024-09-20T19:06:42.334404+0000","last_fullsized":"2024-09-20T19:06:42.334404+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:22.930701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:06:22.930701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:28.969493+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":799,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.332760+0000","last_change":"2024-09-20T19:05:51.626774+0000","last_active":"2024-09-20T19:06:42.332760+0000","last_peered":"2024-09-20T19:06:42.332760+0000","last_clean":"2024-09-20T19:06:42.332760+0000","last_became_active":"2024-09-20T18:49:51.967313+0000","last_became_peered":"2024-09-20T18:49:51.967313+0000","last_unstale":"2024-09-20T19:06:42.332760+0000","last_undegraded":"2024-09-20T19:06:42.332760+0000","last_fullsized":"2024-09-20T19:06:42.332760+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:51.626697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:05:51.626697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:13.789020+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"274'19","reported_seq":867,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.330361+0000","last_change":"2024-09-20T19:01:05.495603+0000","last_active":"2024-09-20T19:06:42.330361+0000","last_peered":"2024-09-20T19:06:42.330361+0000","last_clean":"2024-09-20T19:06:42.330361+0000","last_became_active":"2024-09-20T18:59:05.075390+0000","last_became_peered":"2024-09-20T18:59:05.075390+0000","last_unstale":"2024-09-20T19:06:42.330361+0000","last_undegraded":"2024-09-20T19:06:42.330361+0000","last_fullsized":"2024-09-20T19:06:42.330361+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'19","last_scrub_stamp":"2024-09-20T19:01:05.495478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:05.495478+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:04.804811+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":818,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.333967+0000","last_change":"2024-09-20T19:05:43.686902+0000","last_active":"2024-09-20T19:06:42.333967+0000","last_peered":"2024-09-20T19:06:42.333967+0000","last_clean":"2024-09-20T19:06:42.333967+0000","last_became_active":"2024-09-20T18:49:45.942105+0000","last_became_peered":"2024-09-20T18:49:45.942105+0000","last_unstale":"2024-09-20T19:06:42.333967+0000","last_undegraded":"2024-09-20T19:06:42.333967+0000","last_fullsized":"2024-09-20T19:06:42.333967+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:43.686842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:43.686842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:01.625104+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":832,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:40.314603+0000","last_change":"2024-09-20T19:04:20.912120+0000","last_active":"2024-09-20T19:06:40.314603+0000","last_peered":"2024-09-20T19:06:40.314603+0000","last_clean":"2024-09-20T19:06:40.314603+0000","last_became_active":"2024-09-20T19:04:20.911292+0000","last_became_peered":"2024-09-20T19:04:20.911292+0000","last_unstale":"2024-09-20T19:06:40.314603+0000","last_undegraded":"2024-09-20T19:06:40.314603+0000","last_fullsized":"2024-09-20T19:06:40.314603+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:27.267254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:27.267254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:59.781503+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":804,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.334677+0000","last_change":"2024-09-20T19:05:51.714712+0000","last_active":"2024-09-20T19:06:42.334677+0000","last_peered":"2024-09-20T19:06:42.334677+0000","last_clean":"2024-09-20T19:06:42.334677+0000","last_became_active":"2024-09-20T18:59:05.081179+0000","last_became_peered":"2024-09-20T18:59:05.081179+0000","last_unstale":"2024-09-20T19:06:42.334677+0000","last_undegraded":"2024-09-20T19:06:42.334677+0000","last_fullsized":"2024-09-20T19:06:42.334677+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:51.714641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:05:51.714641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:58.160366+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"273'8","reported_seq":852,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.328596+0000","last_change":"2024-09-20T19:06:21.974151+0000","last_active":"2024-09-20T19:06:42.328596+0000","last_peered":"2024-09-20T19:06:42.328596+0000","last_clean":"2024-09-20T19:06:42.328596+0000","last_became_active":"2024-09-20T18:59:05.076365+0000","last_became_peered":"2024-09-20T18:59:05.076365+0000","last_unstale":"2024-09-20T19:06:42.328596+0000","last_undegraded":"2024-09-20T19:06:42.328596+0000","last_fullsized":"2024-09-20T19:06:42.328596+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'8","last_scrub_stamp":"2024-09-20T19:06:21.974084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:06:21.974084+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:38.613762+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":810,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.334031+0000","last_change":"2024-09-20T19:02:39.184249+0000","last_active":"2024-09-20T19:06:42.334031+0000","last_peered":"2024-09-20T19:06:42.334031+0000","last_clean":"2024-09-20T19:06:42.334031+0000","last_became_active":"2024-09-20T18:49:45.940988+0000","last_became_peered":"2024-09-20T18:49:45.940988+0000","last_unstale":"2024-09-20T19:06:42.334031+0000","last_undegraded":"2024-09-20T19:06:42.334031+0000","last_fullsized":"2024-09-20T19:06:42.334031+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:39.184150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:39.184150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":396,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:40.320291+0000","last_change":"2024-09-20T19:04:20.900567+0000","last_active":"2024-09-20T19:06:40.320291+0000","last_peered":"2024-09-20T19:06:40.320291+0000","last_clean":"2024-09-20T19:06:40.320291+0000","last_became_active":"2024-09-20T19:04:20.900214+0000","last_became_peered":"2024-09-20T19:04:20.900214+0000","last_unstale":"2024-09-20T19:06:40.320291+0000","last_undegraded":"2024-09-20T19:06:40.320291+0000","last_fullsized":"2024-09-20T19:06:40.320291+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:41.497115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:41.497115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:38.822793+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":814,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:40.314735+0000","last_change":"2024-09-20T19:05:57.789513+0000","last_active":"2024-09-20T19:06:40.314735+0000","last_peered":"2024-09-20T19:06:40.314735+0000","last_clean":"2024-09-20T19:06:40.314735+0000","last_became_active":"2024-09-20T19:04:20.885211+0000","last_became_peered":"2024-09-20T19:04:20.885211+0000","last_unstale":"2024-09-20T19:06:40.314735+0000","last_undegraded":"2024-09-20T19:06:40.314735+0000","last_fullsized":"2024-09-20T19:06:40.314735+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:57.789465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:05:57.789465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:08.966824+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":796,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:40.312488+0000","last_change":"2024-09-20T19:06:24.854187+0000","last_active":"2024-09-20T19:06:40.312488+0000","last_peered":"2024-09-20T19:06:40.312488+0000","last_clean":"2024-09-20T19:06:40.312488+0000","last_became_active":"2024-09-20T19:04:20.890101+0000","last_became_peered":"2024-09-20T19:04:20.890101+0000","last_unstale":"2024-09-20T19:06:40.312488+0000","last_undegraded":"2024-09-20T19:06:40.312488+0000","last_fullsized":"2024-09-20T19:06:40.312488+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:24.854124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:06:24.854124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:46.101065+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":801,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:44.934346+0000","last_change":"2024-09-20T19:06:44.934346+0000","last_active":"2024-09-20T19:06:44.934346+0000","last_peered":"2024-09-20T19:06:44.934346+0000","last_clean":"2024-09-20T19:06:44.934346+0000","last_became_active":"2024-09-20T18:49:45.937992+0000","last_became_peered":"2024-09-20T18:49:45.937992+0000","last_unstale":"2024-09-20T19:06:44.934346+0000","last_undegraded":"2024-09-20T19:06:44.934346+0000","last_fullsized":"2024-09-20T19:06:44.934346+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:44.934143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:06:44.934143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:47.788190+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"274'19","reported_seq":403,"reported_epoch":435,"state":"active+clean","last_fresh":"2024-09-20T19:06:40.321174+0000","last_change":"2024-09-20T19:06:26.909675+0000","last_active":"2024-09-20T19:06:40.321174+0000","last_peered":"2024-09-20T19:06:40.321174+0000","last_clean":"2024-09-20T19:06:40.321174+0000","last_became_active":"2024-09-20T18:59:05.089294+0000","last_became_peered":"2024-09-20T18:59:05.089294+0000","last_unstale":"2024-09-20T19:06:40.321174+0000","last_undegraded":"2024-09-20T19:06:40.321174+0000","last_fullsized":"2024-09-20T19:06:40.321174+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'19","last_scrub_stamp":"2024-09-20T19:06:26.909593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:06:26.909593+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:54.942533+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":788,"reported_epoch":436,"state":"active+clean","last_fresh":"2024-09-20T19:06:42.334105+0000","last_change":"2024-09-20T19:01:40.961449+0000","last_active":"2024-09-20T19:06:42.334105+0000","last_peered":"2024-09-20T19:06:42.334105+0000","last_clean":"2024-09-20T19:06:42.334105+0000","last_became_active":"2024-09-20T18:49:49.957582+0000","last_became_peered":"2024-09-20T18:49:49.957582+0000","last_unstale":"2024-09-20T19:06:42.334105+0000","last_undegraded":"2024-09-20T19:06:42.334105+0000","last_fullsized":"2024-09-20T19:06:42.334105+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:40.961363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:40.961363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:08.603698+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":16,"stat_sum":{"num_bytes":2804613120,"num_objects":42795,"num_object_clones":0,"num_object_copies":128385,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42795,"num_write_kb":2738880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8351776768,"data_stored":8351776768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42795,"ondisk_log_size":42795,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1185,"internal_metadata":0},"log_size":87,"ondisk_log_size":87,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1114,"num_read_kb":871,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":257,"num_bytes_recovered":4890,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":686,"ondisk_log_size":686,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":3105,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1008,"num_read_kb":1003,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266895,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1554984,"kb_used_data":937940,"kb_used_omap":1,"kb_used_meta":617022,"kb_avail":92190168,"statfs":{"total":95995035648,"available":94402732032,"internally_reserved":0,"allocated":960450560,"data_stored":959813924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":631830986},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528534,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1738068,"kb_used_data":1291776,"kb_used_omap":1,"kb_used_meta":446270,"kb_avail":92007084,"statfs":{"total":95995035648,"available":94215254016,"internally_reserved":0,"allocated":1322778624,"data_stored":1322161806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":456980934},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757468,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1340652,"kb_used_data":723612,"kb_used_omap":1,"kb_used_meta":617022,"kb_avail":92404500,"statfs":{"total":95995035648,"available":94622208000,"internally_reserved":0,"allocated":740978688,"data_stored":740340430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":631830981},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":304,"seq":1305670058050,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1093628,"kb_used_data":748140,"kb_used_omap":5,"kb_used_meta":345466,"kb_avail":92651524,"statfs":{"total":95995035648,"available":94875160576,"internally_reserved":0,"allocated":766095360,"data_stored":765441612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5633,"internal_metadata":353757695},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182632,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2041320,"kb_used_data":1261848,"kb_used_omap":1,"kb_used_meta":779454,"kb_avail":91703832,"statfs":{"total":95995035648,"available":93904723968,"internally_reserved":0,"allocated":1292132352,"data_stored":1291497570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":798161351},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411564,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1557276,"kb_used_data":930500,"kb_used_omap":1,"kb_used_meta":626750,"kb_avail":92187876,"statfs":{"total":95995035648,"available":94400385024,"internally_reserved":0,"allocated":952832000,"data_stored":952211480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":641792453},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575090,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1976644,"kb_used_data":1305460,"kb_used_omap":1,"kb_used_meta":671166,"kb_avail":91768508,"statfs":{"total":95995035648,"available":93970952192,"internally_reserved":0,"allocated":1336791040,"data_stored":1336122552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":687274437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,2,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738616,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1505380,"kb_used_data":806356,"kb_used_omap":1,"kb_used_meta":698942,"kb_avail":92239772,"statfs":{"total":95995035648,"available":94453526528,"internally_reserved":0,"allocated":825708544,"data_stored":825071604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":715717061},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1185,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":848035840,"data_stored":848035840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1343422464,"data_stored":1343422464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":988807168,"data_stored":988807168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1333067776,"data_stored":1333067776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":771489792,"data_stored":771489792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":759824384,"data_stored":759824384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1325793280,"data_stored":1325793280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":981336064,"data_stored":981336064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T19:06:49.230 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2024-09-20T19:06:49.230 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd pool set unique_pool_4 pg_num 26 2024-09-20T19:06:49.268 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:49.296 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:49.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:49.297 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:49.398 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:49.426 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:49.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:49.427 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:49.471 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:49 smithi083 ceph-mon[29421]: 5.1b scrub starts 2024-09-20T19:06:49.471 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:49 smithi083 ceph-mon[29421]: 5.1b scrub ok 2024-09-20T19:06:49.471 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:49 smithi083 ceph-mon[36260]: 5.1b scrub starts 2024-09-20T19:06:49.471 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:49 smithi083 ceph-mon[36260]: 5.1b scrub ok 2024-09-20T19:06:49.527 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:49.557 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:49.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:49.558 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:49.659 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:49.685 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:49.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:49.687 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:49.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:49 smithi142 ceph-mon[32508]: 5.1b scrub starts 2024-09-20T19:06:49.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:49 smithi142 ceph-mon[32508]: 5.1b scrub ok 2024-09-20T19:06:49.788 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:49.825 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:49.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:49.828 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:49.928 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:49.959 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:49.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:49.960 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:50.061 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:06:50.099 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:50.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:50.100 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:06:50.115 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 27 16 53494 53478 123.769 87.375 0.00557903 0.00807393 2024-09-20T19:06:50.201 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:06:50.228 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:50.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:50.229 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:06:50.330 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:50.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:50.369 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:50.369 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:50.398 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-20T19:06:50.470 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:50.506 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:50.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:50.507 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:50.608 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:50.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:50.665 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:50.665 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:50.715 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:50 smithi083 ceph-mon[29421]: pgmap v863: 148 pgs: 148 active+clean; 2.9 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 102 MiB/s wr, 1.64k op/s 2024-09-20T19:06:50.715 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:50 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/9291081' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:06:50.715 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:50 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:06:50.716 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:50 smithi083 ceph-mon[36260]: pgmap v863: 148 pgs: 148 active+clean; 2.9 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 102 MiB/s wr, 1.64k op/s 2024-09-20T19:06:50.716 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:50 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/9291081' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:06:50.716 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:50 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:06:50.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:50 smithi142 ceph-mon[32508]: pgmap v863: 148 pgs: 148 active+clean; 2.9 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 102 MiB/s wr, 1.64k op/s 2024-09-20T19:06:50.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:50 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/9291081' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:06:50.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:50 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:06:50.766 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:50.794 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:06:50.794 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:06:50.804 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:50.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:50.805 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:50.830 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:06:50.830 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:06:50.871 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:06:50.872 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:06:50.905 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:06:50.905 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:06:50.906 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:50.936 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:50.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:50.937 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:50.942 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:06:50.942 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:06:50.967 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:06:50.977 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:06:50.977 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:06:50.978 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:06:50 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:06:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:06:51.012 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:06:51.012 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:06:51.039 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:51.046 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:06:51.047 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:06:51.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:51.082 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:51.082 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:51.115 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 28 16 54946 54930 122.59 90.75 0.00964867 0.0081518 2024-09-20T19:06:51.183 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:51.217 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:51.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:51.218 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:51.319 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:51.355 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:51.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:51.355 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:51.456 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:51.482 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:51.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:51.483 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:51.584 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:51.617 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:51.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:51.618 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:51.700 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:51 smithi083 ceph-mon[29421]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T19:06:51.700 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:51 smithi083 ceph-mon[29421]: Cluster is now healthy 2024-09-20T19:06:51.700 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:51 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:06:51.700 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:51 smithi083 ceph-mon[29421]: osdmap e438: 8 total, 8 up, 8 in 2024-09-20T19:06:51.700 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:51 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/113536473' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:06:51.700 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:51 smithi083 ceph-mon[29421]: osdmap e439: 8 total, 8 up, 8 in 2024-09-20T19:06:51.701 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:51 smithi083 ceph-mon[36260]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T19:06:51.701 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:51 smithi083 ceph-mon[36260]: Cluster is now healthy 2024-09-20T19:06:51.701 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:51 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:06:51.701 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:51 smithi083 ceph-mon[36260]: osdmap e438: 8 total, 8 up, 8 in 2024-09-20T19:06:51.701 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:51 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/113536473' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:06:51.701 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:51 smithi083 ceph-mon[36260]: osdmap e439: 8 total, 8 up, 8 in 2024-09-20T19:06:51.718 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:51.752 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:51.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:51.753 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:51.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:51 smithi142 ceph-mon[32508]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T19:06:51.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:51 smithi142 ceph-mon[32508]: Cluster is now healthy 2024-09-20T19:06:51.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:51 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:06:51.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:51 smithi142 ceph-mon[32508]: osdmap e438: 8 total, 8 up, 8 in 2024-09-20T19:06:51.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:51 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/113536473' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:06:51.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:51 smithi142 ceph-mon[32508]: osdmap e439: 8 total, 8 up, 8 in 2024-09-20T19:06:51.854 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:06:51.895 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:06:51.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:51.896 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:51.896 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:06:51.997 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:52.029 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:52.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:52.030 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:52.115 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 29 16 56511 56495 121.735 97.8125 0.0115698 0.00820957 2024-09-20T19:06:52.131 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:52.158 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:52.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:52.159 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:52.192 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T19:06:52.192 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-20T19:06:52.192 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:52.192 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.26264.0:56593 11.6 11:6f257950:::benchmark_data_smithi083_177491_object56592:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e439)", 2024-09-20T19:06:52.193 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:06:52.175602+0000", 2024-09-20T19:06:52.193 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.011064991, 2024-09-20T19:06:52.193 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011097793, 2024-09-20T19:06:52.193 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:06:52.193 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:06:52.193 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "waiting for sub ops", 2024-09-20T19:06:52.193 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:06:52.193 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.26264", 2024-09-20T19:06:52.193 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2608780316", 2024-09-20T19:06:52.193 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 56593 2024-09-20T19:06:52.193 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:52.193 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:06:52.193 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:52.194 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:06:52.194 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:52.175602+0000", 2024-09-20T19:06:52.194 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:52.194 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:52.194 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:52.194 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:06:52.194 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:52.175602+0000", 2024-09-20T19:06:52.194 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:52.194 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:52.194 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:52.194 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:06:52.194 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:52.175604+0000", 2024-09-20T19:06:52.194 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8819999999999999e-06 2024-09-20T19:06:52.195 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:52.195 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:52.195 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:06:52.195 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:52.175626+0000", 2024-09-20T19:06:52.195 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2538e-05 2024-09-20T19:06:52.195 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:52.195 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:52.195 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:06:52.195 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:52.175639+0000", 2024-09-20T19:06:52.195 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.243e-05 2024-09-20T19:06:52.195 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:52.195 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:52.195 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:06:52.195 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:52.175648+0000", 2024-09-20T19:06:52.196 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2669999999999995e-06 2024-09-20T19:06:52.196 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:52.196 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:52.196 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:52.196 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:52.175838+0000", 2024-09-20T19:06:52.196 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000189808 2024-09-20T19:06:52.196 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:52.196 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:52.196 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:06:52.196 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:52.176974+0000", 2024-09-20T19:06:52.196 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001135832 2024-09-20T19:06:52.196 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:52.196 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:52.196 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:06:52.197 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:52.177099+0000", 2024-09-20T19:06:52.197 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000125074 2024-09-20T19:06:52.197 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:52.197 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:52.197 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:06:52.197 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:52.179148+0000", 2024-09-20T19:06:52.197 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0020490830000000002 2024-09-20T19:06:52.197 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:52.197 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:52.197 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:06:52.197 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:52.181368+0000", 2024-09-20T19:06:52.197 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0022205649999999999 2024-09-20T19:06:52.198 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:52.198 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:52.198 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:52.198 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:52.198 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:52.198 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.26264.0:56607 11.6 11:6328e792:::benchmark_data_smithi083_177491_object56606:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e439)", 2024-09-20T19:06:52.198 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:06:52.181626+0000", 2024-09-20T19:06:52.198 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0050408400000000004, 2024-09-20T19:06:52.198 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0051593309999999996, 2024-09-20T19:06:52.199 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:06:52.199 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:06:52.199 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "waiting for sub ops", 2024-09-20T19:06:52.199 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:06:52.199 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.26264", 2024-09-20T19:06:52.199 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2608780316", 2024-09-20T19:06:52.200 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 56607 2024-09-20T19:06:52.200 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:52.200 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:06:52.200 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:52.200 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:06:52.200 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:52.181626+0000", 2024-09-20T19:06:52.200 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:52.200 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:52.200 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:52.200 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:06:52.200 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:52.181626+0000", 2024-09-20T19:06:52.200 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:52.200 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:52.201 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:52.201 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:06:52.201 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:52.181628+0000", 2024-09-20T19:06:52.201 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.513e-06 2024-09-20T19:06:52.201 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:52.201 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:52.201 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:06:52.201 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:52.181650+0000", 2024-09-20T19:06:52.201 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2595e-05 2024-09-20T19:06:52.201 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:52.201 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:52.201 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:06:52.201 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:52.181660+0000", 2024-09-20T19:06:52.202 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.713e-06 2024-09-20T19:06:52.202 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:52.202 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:52.202 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:06:52.202 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:52.181667+0000", 2024-09-20T19:06:52.202 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2470000000000002e-06 2024-09-20T19:06:52.202 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:52.202 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:52.202 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:52.202 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:52.181720+0000", 2024-09-20T19:06:52.202 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3426e-05 2024-09-20T19:06:52.203 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:52.203 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:52.203 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:06:52.203 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:52.181882+0000", 2024-09-20T19:06:52.203 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016157899999999999 2024-09-20T19:06:52.203 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:52.203 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:52.203 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,2", 2024-09-20T19:06:52.203 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:52.182033+0000", 2024-09-20T19:06:52.203 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015096900000000001 2024-09-20T19:06:52.203 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:52.204 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:52.204 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:06:52.204 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:52.184279+0000", 2024-09-20T19:06:52.204 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0022454720000000001 2024-09-20T19:06:52.204 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:52.204 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:52.204 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:52.204 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:52.204 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:52.204 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.26264.0:56598 11.2 e439/422 11:48f99436:::benchmark_data_smithi083_177491_object56597:head v 439'3465, mlcod=439'3463)", 2024-09-20T19:06:52.204 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:06:52.182256+0000", 2024-09-20T19:06:52.204 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0044112409999999998, 2024-09-20T19:06:52.204 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0046233309999999996, 2024-09-20T19:06:52.204 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:06:52.205 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:06:52.205 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T19:06:52.205 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:06:52.205 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:52.205 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:06:52.205 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:52.182256+0000", 2024-09-20T19:06:52.205 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:52.205 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:52.205 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:52.205 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:06:52.205 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:52.182256+0000", 2024-09-20T19:06:52.205 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:52.205 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:52.206 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:52.206 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:06:52.206 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:52.182256+0000", 2024-09-20T19:06:52.206 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2600000000000002e-07 2024-09-20T19:06:52.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:52.206 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:52.206 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:06:52.206 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:52.182280+0000", 2024-09-20T19:06:52.206 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3989999999999999e-05 2024-09-20T19:06:52.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:52.206 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:52.206 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:06:52.206 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:52.182288+0000", 2024-09-20T19:06:52.206 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5000000000000002e-06 2024-09-20T19:06:52.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:52.207 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:52.207 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:06:52.207 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:52.182294+0000", 2024-09-20T19:06:52.207 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4999999999999996e-06 2024-09-20T19:06:52.207 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:52.207 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:52.207 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:52.207 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:52.182477+0000", 2024-09-20T19:06:52.207 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00018278000000000001 2024-09-20T19:06:52.207 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:52.207 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:52.207 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:06:52.207 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:52.182496+0000", 2024-09-20T19:06:52.207 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8898000000000001e-05 2024-09-20T19:06:52.208 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:52.208 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:52.208 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:52.208 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:52.208 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:52.208 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.26264.0:56603 11.b e439/422 11:d369c42e:::benchmark_data_smithi083_177491_object56602:head v 439'3421, mlcod=439'3418)", 2024-09-20T19:06:52.208 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:06:52.184884+0000", 2024-09-20T19:06:52.208 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0017827209999999999, 2024-09-20T19:06:52.208 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002044499, 2024-09-20T19:06:52.208 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:06:52.208 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:06:52.208 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T19:06:52.208 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:06:52.208 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:52.209 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:06:52.209 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:52.184884+0000", 2024-09-20T19:06:52.209 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:52.209 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:52.209 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:52.209 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:06:52.209 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:52.184884+0000", 2024-09-20T19:06:52.209 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:06:52.209 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:52.209 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:52.209 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:06:52.209 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:52.184885+0000", 2024-09-20T19:06:52.209 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0699999999999999e-06 2024-09-20T19:06:52.209 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:52.209 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:52.210 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:06:52.210 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:52.184906+0000", 2024-09-20T19:06:52.210 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1192999999999999e-05 2024-09-20T19:06:52.210 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:52.210 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:52.210 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:06:52.210 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:52.184916+0000", 2024-09-20T19:06:52.210 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7580000000000004e-06 2024-09-20T19:06:52.210 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:52.210 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:52.210 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:06:52.210 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:52.184923+0000", 2024-09-20T19:06:52.210 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6800000000000004e-06 2024-09-20T19:06:52.210 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:52.211 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:52.211 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:06:52.211 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:52.185019+0000", 2024-09-20T19:06:52.211 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.5706999999999997e-05 2024-09-20T19:06:52.211 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:06:52.211 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:06:52.211 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:06:52.211 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:52.185036+0000", 2024-09-20T19:06:52.211 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7717999999999999e-05 2024-09-20T19:06:52.211 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:52.211 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:06:52.211 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:52.211 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:06:52.212 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T19:06:52.212 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 4 2024-09-20T19:06:52.212 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T19:06:52.259 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:52.296 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:52.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:52.297 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:52.398 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:52.430 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:52.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:52.431 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:52.533 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:52.562 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:52.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:52.563 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:52.664 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:52.702 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:52.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:52.703 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:52.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:52 smithi142 ceph-mon[32508]: 4.12 scrub starts 2024-09-20T19:06:52.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:52 smithi142 ceph-mon[32508]: 4.12 scrub ok 2024-09-20T19:06:52.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:52 smithi142 ceph-mon[32508]: pgmap v866: 148 pgs: 148 active+clean; 3.1 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 852 B/s rd, 115 MiB/s wr, 1.83k op/s 2024-09-20T19:06:52.779 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:52 smithi083 ceph-mon[29421]: 4.12 scrub starts 2024-09-20T19:06:52.779 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:52 smithi083 ceph-mon[29421]: 4.12 scrub ok 2024-09-20T19:06:52.779 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:52 smithi083 ceph-mon[29421]: pgmap v866: 148 pgs: 148 active+clean; 3.1 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 852 B/s rd, 115 MiB/s wr, 1.83k op/s 2024-09-20T19:06:52.779 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:52 smithi083 ceph-mon[36260]: 4.12 scrub starts 2024-09-20T19:06:52.779 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:52 smithi083 ceph-mon[36260]: 4.12 scrub ok 2024-09-20T19:06:52.779 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:52 smithi083 ceph-mon[36260]: pgmap v866: 148 pgs: 148 active+clean; 3.1 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 852 B/s rd, 115 MiB/s wr, 1.83k op/s 2024-09-20T19:06:52.803 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:52.849 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:52.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:52.850 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:52.950 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:52.982 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:52.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:52.983 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:53.084 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:53.117 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 30 16 57958 57942 120.687 90.4375 0.0207023 0.00828034 2024-09-20T19:06:53.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:53.131 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:53.131 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:53.232 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:53.267 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:53.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:53.268 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:53.369 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:53.401 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:53.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:53.402 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:53.426 INFO:teuthology.orchestra.run.smithi083.stderr:set pool 11 pg_num to 26 2024-09-20T19:06:53.495 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T19:06:53.503 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:53.532 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:53.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:53.533 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:53.634 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:53.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:53.675 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:53.675 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:53.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:53 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1920362080' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num", "val": "26"} : dispatch 2024-09-20T19:06:53.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:53 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num", "val": "26"} : dispatch 2024-09-20T19:06:53.775 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:53.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:53 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1920362080' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num", "val": "26"} : dispatch 2024-09-20T19:06:53.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:53 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num", "val": "26"} : dispatch 2024-09-20T19:06:53.792 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:53 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1920362080' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num", "val": "26"} : dispatch 2024-09-20T19:06:53.792 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:53 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num", "val": "26"} : dispatch 2024-09-20T19:06:53.812 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:53.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:53.813 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:53.914 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:53.940 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:53.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:53.941 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:54.041 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:54.075 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:54.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:54.076 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:54.117 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 31 16 59278 59262 119.455 82.5 0.00870262 0.0083664 2024-09-20T19:06:54.176 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:54.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:54.209 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:54.209 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:54.310 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:54.347 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:54.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:54.348 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:54.448 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:54.484 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:54.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:54.485 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:54.586 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:54.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:54.622 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:54.622 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:54.723 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:54.755 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:54.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:54.756 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:54.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:54 smithi142 ceph-mon[32508]: 3.19 scrub starts 2024-09-20T19:06:54.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:54 smithi142 ceph-mon[32508]: 3.19 scrub ok 2024-09-20T19:06:54.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:54 smithi142 ceph-mon[32508]: pgmap v867: 148 pgs: 148 active+clean; 3.4 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.4 KiB/s rd, 109 MiB/s wr, 1.75k op/s 2024-09-20T19:06:54.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:54 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num", "val": "26"}]': finished 2024-09-20T19:06:54.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:54 smithi142 ceph-mon[32508]: osdmap e440: 8 total, 8 up, 8 in 2024-09-20T19:06:54.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:54 smithi142 ceph-mon[32508]: 6.18 scrub starts 2024-09-20T19:06:54.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:54 smithi142 ceph-mon[32508]: 6.18 scrub ok 2024-09-20T19:06:54.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:54 smithi083 ceph-mon[29421]: 3.19 scrub starts 2024-09-20T19:06:54.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:54 smithi083 ceph-mon[29421]: 3.19 scrub ok 2024-09-20T19:06:54.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:54 smithi083 ceph-mon[29421]: pgmap v867: 148 pgs: 148 active+clean; 3.4 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.4 KiB/s rd, 109 MiB/s wr, 1.75k op/s 2024-09-20T19:06:54.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:54 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num", "val": "26"}]': finished 2024-09-20T19:06:54.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:54 smithi083 ceph-mon[29421]: osdmap e440: 8 total, 8 up, 8 in 2024-09-20T19:06:54.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:54 smithi083 ceph-mon[29421]: 6.18 scrub starts 2024-09-20T19:06:54.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:54 smithi083 ceph-mon[29421]: 6.18 scrub ok 2024-09-20T19:06:54.792 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:54 smithi083 ceph-mon[36260]: 3.19 scrub starts 2024-09-20T19:06:54.792 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:54 smithi083 ceph-mon[36260]: 3.19 scrub ok 2024-09-20T19:06:54.792 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:54 smithi083 ceph-mon[36260]: pgmap v867: 148 pgs: 148 active+clean; 3.4 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.4 KiB/s rd, 109 MiB/s wr, 1.75k op/s 2024-09-20T19:06:54.792 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:54 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num", "val": "26"}]': finished 2024-09-20T19:06:54.792 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:54 smithi083 ceph-mon[36260]: osdmap e440: 8 total, 8 up, 8 in 2024-09-20T19:06:54.792 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:54 smithi083 ceph-mon[36260]: 6.18 scrub starts 2024-09-20T19:06:54.792 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:54 smithi083 ceph-mon[36260]: 6.18 scrub ok 2024-09-20T19:06:54.857 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:54.878 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:54.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:54.879 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:54.980 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:55.001 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:55.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:55.002 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:55.103 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:06:55.117 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 32 16 60263 60247 117.646 61.5625 0.00473159 0.00843603 2024-09-20T19:06:55.124 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:55.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:55.125 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:06:55.226 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:55.254 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:55.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:55.255 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:55.356 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:55.391 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:55.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:55.392 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:55.493 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:55.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:55 smithi142 ceph-mon[32508]: 4.0 scrub starts 2024-09-20T19:06:55.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:55 smithi142 ceph-mon[32508]: 4.0 scrub ok 2024-09-20T19:06:55.514 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:55 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num_actual", "val": "26"} : dispatch 2024-09-20T19:06:55.558 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:55.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:55.559 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:55.660 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:55.682 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:55.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:55.683 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:55.783 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:55.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:55 smithi083 ceph-mon[29421]: 4.0 scrub starts 2024-09-20T19:06:55.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:55 smithi083 ceph-mon[29421]: 4.0 scrub ok 2024-09-20T19:06:55.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:55 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num_actual", "val": "26"} : dispatch 2024-09-20T19:06:55.791 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:55 smithi083 ceph-mon[36260]: 4.0 scrub starts 2024-09-20T19:06:55.791 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:55 smithi083 ceph-mon[36260]: 4.0 scrub ok 2024-09-20T19:06:55.792 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:55 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num_actual", "val": "26"} : dispatch 2024-09-20T19:06:55.804 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:55.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:55.805 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:55.906 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:55.928 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:55.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:55.929 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:56.030 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:56.051 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:56.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:56.052 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:56.117 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 33 16 60574 60558 114.67 19.4375 0.214403 0.00855669 2024-09-20T19:06:56.152 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:56.174 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:56.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:56.175 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:56.275 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:56.296 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:56.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:56.297 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:56.398 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:56.419 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:56.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:56.419 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:56.520 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:56.554 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:56.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:56.555 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:56.656 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:56.693 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:56.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:56.693 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:56.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:56 smithi142 ceph-mon[32508]: pgmap v869: 148 pgs: 148 active+clean; 3.4 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 853 B/s rd, 79 MiB/s wr, 1.27k op/s 2024-09-20T19:06:56.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:56 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num_actual", "val": "26"}]': finished 2024-09-20T19:06:56.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:56 smithi142 ceph-mon[32508]: osdmap e441: 8 total, 8 up, 8 in 2024-09-20T19:06:56.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:56 smithi142 ceph-mon[32508]: 5.5 scrub starts 2024-09-20T19:06:56.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:56 smithi142 ceph-mon[32508]: 5.5 scrub ok 2024-09-20T19:06:56.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:56 smithi083 ceph-mon[29421]: pgmap v869: 148 pgs: 148 active+clean; 3.4 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 853 B/s rd, 79 MiB/s wr, 1.27k op/s 2024-09-20T19:06:56.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:56 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num_actual", "val": "26"}]': finished 2024-09-20T19:06:56.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:56 smithi083 ceph-mon[29421]: osdmap e441: 8 total, 8 up, 8 in 2024-09-20T19:06:56.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:56 smithi083 ceph-mon[29421]: 5.5 scrub starts 2024-09-20T19:06:56.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:56 smithi083 ceph-mon[29421]: 5.5 scrub ok 2024-09-20T19:06:56.791 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:56 smithi083 ceph-mon[36260]: pgmap v869: 148 pgs: 148 active+clean; 3.4 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 853 B/s rd, 79 MiB/s wr, 1.27k op/s 2024-09-20T19:06:56.792 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:56 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num_actual", "val": "26"}]': finished 2024-09-20T19:06:56.792 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:56 smithi083 ceph-mon[36260]: osdmap e441: 8 total, 8 up, 8 in 2024-09-20T19:06:56.792 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:56 smithi083 ceph-mon[36260]: 5.5 scrub starts 2024-09-20T19:06:56.792 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:56 smithi083 ceph-mon[36260]: 5.5 scrub ok 2024-09-20T19:06:56.794 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:56.832 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:56.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:56.833 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:56.934 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:56.960 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:56.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:56.960 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:57.061 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:57.087 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:57.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:57.088 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:57.117 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 34 15 61468 61453 112.943 55.9375 0.00906716 0.0088494 2024-09-20T19:06:57.189 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:57.224 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:57.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:57.225 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:57.326 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:06:57.365 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:57.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:57.366 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:06:57.467 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:06:57.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:57.500 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:57.500 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:06:57.601 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:57.623 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:57.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:57.624 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:57.725 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:57.746 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:57.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:57.747 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:57.761 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:06:57 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:06:57.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:57 smithi142 ceph-mon[32508]: osdmap e442: 8 total, 8 up, 8 in 2024-09-20T19:06:57.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:57 smithi142 ceph-mon[32508]: 6.e scrub starts 2024-09-20T19:06:57.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:57 smithi142 ceph-mon[32508]: 6.e scrub ok 2024-09-20T19:06:57.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:57 smithi142 ceph-mon[32508]: 11.5 scrub starts 2024-09-20T19:06:57.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:57 smithi142 ceph-mon[32508]: 11.9 scrub starts 2024-09-20T19:06:57.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:57 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pgp_num_actual", "val": "26"} : dispatch 2024-09-20T19:06:57.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:57 smithi083 ceph-mon[29421]: osdmap e442: 8 total, 8 up, 8 in 2024-09-20T19:06:57.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:57 smithi083 ceph-mon[29421]: 6.e scrub starts 2024-09-20T19:06:57.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:57 smithi083 ceph-mon[29421]: 6.e scrub ok 2024-09-20T19:06:57.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:57 smithi083 ceph-mon[29421]: 11.5 scrub starts 2024-09-20T19:06:57.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:57 smithi083 ceph-mon[29421]: 11.9 scrub starts 2024-09-20T19:06:57.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:57 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pgp_num_actual", "val": "26"} : dispatch 2024-09-20T19:06:57.792 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:57 smithi083 ceph-mon[36260]: osdmap e442: 8 total, 8 up, 8 in 2024-09-20T19:06:57.792 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:57 smithi083 ceph-mon[36260]: 6.e scrub starts 2024-09-20T19:06:57.792 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:57 smithi083 ceph-mon[36260]: 6.e scrub ok 2024-09-20T19:06:57.792 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:57 smithi083 ceph-mon[36260]: 11.5 scrub starts 2024-09-20T19:06:57.792 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:57 smithi083 ceph-mon[36260]: 11.9 scrub starts 2024-09-20T19:06:57.792 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:57 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pgp_num_actual", "val": "26"} : dispatch 2024-09-20T19:06:57.848 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:57.869 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:57.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:57.870 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:57.971 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:57.992 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:57.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:57.993 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:58.094 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:58.116 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:58.117 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 35 16 62122 62106 110.882 40.8125 0.00146386 0.00885186 2024-09-20T19:06:58.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:58.118 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:58.218 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:06:58.240 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:58.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:58.241 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:06:58.341 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:06:58.363 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:58.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:58.364 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:06:58.464 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:58.485 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:58.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:58.486 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:58.587 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:06:58.608 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:58.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:58.609 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:06:58.709 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:58.730 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:58.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:58.731 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:58.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:58 smithi142 ceph-mon[32508]: 11.7 scrub starts 2024-09-20T19:06:58.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:58 smithi142 ceph-mon[32508]: 11.5 scrub ok 2024-09-20T19:06:58.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:58 smithi142 ceph-mon[32508]: pgmap v872: 158 pgs: 5 peering, 6 unknown, 147 active+clean; 3.7 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.7 KiB/s rd, 104 MiB/s wr, 1.67k op/s; 11 KiB/s, 0 objects/s recovering 2024-09-20T19:06:58.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:58 smithi142 ceph-mon[32508]: 11.7 scrub ok 2024-09-20T19:06:58.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:58 smithi142 ceph-mon[32508]: 11.9 scrub ok 2024-09-20T19:06:58.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:58 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:06:58.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:58 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pgp_num_actual", "val": "26"}]': finished 2024-09-20T19:06:58.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:58 smithi142 ceph-mon[32508]: osdmap e443: 8 total, 8 up, 8 in 2024-09-20T19:06:58.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:58 smithi142 ceph-mon[32508]: 4.d scrub starts 2024-09-20T19:06:58.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:58 smithi142 ceph-mon[32508]: 4.d scrub ok 2024-09-20T19:06:58.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:58 smithi142 ceph-mon[32508]: 11.8 scrub starts 2024-09-20T19:06:58.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:58 smithi142 ceph-mon[32508]: 11.6 scrub starts 2024-09-20T19:06:58.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:58 smithi142 ceph-mon[32508]: 11.6 scrub ok 2024-09-20T19:06:58.762 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:58 smithi142 ceph-mon[32508]: osdmap e444: 8 total, 8 up, 8 in 2024-09-20T19:06:58.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:58 smithi083 ceph-mon[29421]: 11.7 scrub starts 2024-09-20T19:06:58.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:58 smithi083 ceph-mon[29421]: 11.5 scrub ok 2024-09-20T19:06:58.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:58 smithi083 ceph-mon[29421]: pgmap v872: 158 pgs: 5 peering, 6 unknown, 147 active+clean; 3.7 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.7 KiB/s rd, 104 MiB/s wr, 1.67k op/s; 11 KiB/s, 0 objects/s recovering 2024-09-20T19:06:58.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:58 smithi083 ceph-mon[29421]: 11.7 scrub ok 2024-09-20T19:06:58.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:58 smithi083 ceph-mon[29421]: 11.9 scrub ok 2024-09-20T19:06:58.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:58 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:06:58.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:58 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pgp_num_actual", "val": "26"}]': finished 2024-09-20T19:06:58.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:58 smithi083 ceph-mon[29421]: osdmap e443: 8 total, 8 up, 8 in 2024-09-20T19:06:58.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:58 smithi083 ceph-mon[29421]: 4.d scrub starts 2024-09-20T19:06:58.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:58 smithi083 ceph-mon[29421]: 4.d scrub ok 2024-09-20T19:06:58.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:58 smithi083 ceph-mon[29421]: 11.8 scrub starts 2024-09-20T19:06:58.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:58 smithi083 ceph-mon[29421]: 11.6 scrub starts 2024-09-20T19:06:58.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:58 smithi083 ceph-mon[29421]: 11.6 scrub ok 2024-09-20T19:06:58.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:58 smithi083 ceph-mon[29421]: osdmap e444: 8 total, 8 up, 8 in 2024-09-20T19:06:58.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:58 smithi083 ceph-mon[36260]: 11.7 scrub starts 2024-09-20T19:06:58.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:58 smithi083 ceph-mon[36260]: 11.5 scrub ok 2024-09-20T19:06:58.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:58 smithi083 ceph-mon[36260]: pgmap v872: 158 pgs: 5 peering, 6 unknown, 147 active+clean; 3.7 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.7 KiB/s rd, 104 MiB/s wr, 1.67k op/s; 11 KiB/s, 0 objects/s recovering 2024-09-20T19:06:58.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:58 smithi083 ceph-mon[36260]: 11.7 scrub ok 2024-09-20T19:06:58.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:58 smithi083 ceph-mon[36260]: 11.9 scrub ok 2024-09-20T19:06:58.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:58 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:06:58.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:58 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pgp_num_actual", "val": "26"}]': finished 2024-09-20T19:06:58.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:58 smithi083 ceph-mon[36260]: osdmap e443: 8 total, 8 up, 8 in 2024-09-20T19:06:58.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:58 smithi083 ceph-mon[36260]: 4.d scrub starts 2024-09-20T19:06:58.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:58 smithi083 ceph-mon[36260]: 4.d scrub ok 2024-09-20T19:06:58.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:58 smithi083 ceph-mon[36260]: 11.8 scrub starts 2024-09-20T19:06:58.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:58 smithi083 ceph-mon[36260]: 11.6 scrub starts 2024-09-20T19:06:58.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:58 smithi083 ceph-mon[36260]: 11.6 scrub ok 2024-09-20T19:06:58.793 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:58 smithi083 ceph-mon[36260]: osdmap e444: 8 total, 8 up, 8 in 2024-09-20T19:06:58.832 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:06:58.853 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:58.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:58.854 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:06:58.955 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:58.972 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_blocked_ops 2024-09-20T19:06:58.976 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:58.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:58.977 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:59.077 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:06:59.098 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:59.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:59.099 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:06:59.117 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 36 16 62122 62106 107.803 0 - 0.00885186 2024-09-20T19:06:59.200 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:06:59.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:59.223 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:59.223 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:06:59.324 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:59.344 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:59.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:59.345 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:59.446 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:06:59.478 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:59 smithi083 ceph-mon[36260]: 11.8 scrub ok 2024-09-20T19:06:59.478 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:59 smithi083 ceph-mon[36260]: 11.0 scrub starts 2024-09-20T19:06:59.478 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:59 smithi083 ceph-mon[36260]: 11.0 scrub ok 2024-09-20T19:06:59.478 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:59 smithi083 ceph-mon[36260]: 11.2 scrub starts 2024-09-20T19:06:59.478 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:59 smithi083 ceph-mon[36260]: 11.2 scrub ok 2024-09-20T19:06:59.479 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:59 smithi083 ceph-mon[36260]: 6.13 scrub starts 2024-09-20T19:06:59.479 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:59 smithi083 ceph-mon[36260]: 6.13 scrub ok 2024-09-20T19:06:59.479 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:06:59 smithi083 ceph-mon[36260]: osdmap e445: 8 total, 8 up, 8 in 2024-09-20T19:06:59.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:59.479 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:06:59.480 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:06:59.557 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:06:59.581 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:59.613 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:59.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:59.614 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:59.715 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:06:59.741 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:59.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:59.742 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:06:59.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:59 smithi142 ceph-mon[32508]: 11.8 scrub ok 2024-09-20T19:06:59.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:59 smithi142 ceph-mon[32508]: 11.0 scrub starts 2024-09-20T19:06:59.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:59 smithi142 ceph-mon[32508]: 11.0 scrub ok 2024-09-20T19:06:59.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:59 smithi142 ceph-mon[32508]: 11.2 scrub starts 2024-09-20T19:06:59.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:59 smithi142 ceph-mon[32508]: 11.2 scrub ok 2024-09-20T19:06:59.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:59 smithi142 ceph-mon[32508]: 6.13 scrub starts 2024-09-20T19:06:59.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:59 smithi142 ceph-mon[32508]: 6.13 scrub ok 2024-09-20T19:06:59.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:06:59 smithi142 ceph-mon[32508]: osdmap e445: 8 total, 8 up, 8 in 2024-09-20T19:06:59.782 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:59 smithi083 ceph-mon[29421]: 11.8 scrub ok 2024-09-20T19:06:59.782 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:59 smithi083 ceph-mon[29421]: 11.0 scrub starts 2024-09-20T19:06:59.782 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:59 smithi083 ceph-mon[29421]: 11.0 scrub ok 2024-09-20T19:06:59.783 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:59 smithi083 ceph-mon[29421]: 11.2 scrub starts 2024-09-20T19:06:59.783 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:59 smithi083 ceph-mon[29421]: 11.2 scrub ok 2024-09-20T19:06:59.783 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:59 smithi083 ceph-mon[29421]: 6.13 scrub starts 2024-09-20T19:06:59.783 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:59 smithi083 ceph-mon[29421]: 6.13 scrub ok 2024-09-20T19:06:59.783 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:06:59 smithi083 ceph-mon[29421]: osdmap e445: 8 total, 8 up, 8 in 2024-09-20T19:06:59.842 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:06:59.873 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:06:59.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:06:59.874 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:06:59.974 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:00.003 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:00.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:00.004 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:00.105 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:00.118 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 37 16 62416 62400 105.384 9.1875 0.00894971 0.00948208 2024-09-20T19:07:00.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:00.155 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:00.155 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:07:00.256 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:00.298 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:00.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:00.299 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:00.358 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:07:00.399 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:00.456 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:00.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:00.457 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:00.558 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:00.591 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:00.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:00.592 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:00.693 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:00.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:00.739 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:00.739 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:00.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:00 smithi142 ceph-mon[32508]: pgmap v875: 158 pgs: 5 peering, 7 unknown, 146 active+clean; 3.5 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 25 MiB/s wr, 394 op/s; 16 KiB/s, 0 objects/s recovering 2024-09-20T19:07:00.762 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:00 smithi083 ceph-mon[29421]: pgmap v875: 158 pgs: 5 peering, 7 unknown, 146 active+clean; 3.5 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 25 MiB/s wr, 394 op/s; 16 KiB/s, 0 objects/s recovering 2024-09-20T19:07:00.762 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:07:00 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:07:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:07:00.762 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:00 smithi083 ceph-mon[36260]: pgmap v875: 158 pgs: 5 peering, 7 unknown, 146 active+clean; 3.5 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 25 MiB/s wr, 394 op/s; 16 KiB/s, 0 objects/s recovering 2024-09-20T19:07:00.841 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:00.857 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T19:07:00.880 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:00.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:00.881 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:00.981 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:01.036 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:01.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:01.037 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:01.118 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 38 16 63172 63156 103.854 47.25 0.0116551 0.00962329 2024-09-20T19:07:01.138 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:01.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:01.193 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:01.193 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:01.294 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:01.324 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:01.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:01.325 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:07:01.426 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:01.465 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:01.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:01.466 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:07:01.566 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:01.589 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:01.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:01.590 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:01.691 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:01.732 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:01.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:01.733 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:01.834 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:01.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:01.892 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:01.892 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:01.937 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-20T19:07:01.993 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:02.020 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:02.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:02.021 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:02.118 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 39 16 64046 64030 102.592 54.625 0.0107045 0.00974135 2024-09-20T19:07:02.121 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:02.147 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:02.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:02.149 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:02.226 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T19:07:02.226 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-20T19:07:02.226 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-20T19:07:02.226 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-20T19:07:02.227 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T19:07:02.227 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:01 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1926531685' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:07:02.227 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:01 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:07:02.227 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:01 smithi083 ceph-mon[29421]: pgmap v877: 158 pgs: 2 active+recovering+undersized+remapped, 2 peering, 5 unknown, 149 active+clean; 3.5 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 9090/174384 objects misplaced (5.213%); 1.1 MiB/s, 18 objects/s recovering 2024-09-20T19:07:02.227 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:01 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:07:02.227 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:01 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:07:02.228 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:01 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1926531685' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:07:02.228 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:01 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:07:02.228 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:01 smithi083 ceph-mon[36260]: pgmap v877: 158 pgs: 2 active+recovering+undersized+remapped, 2 peering, 5 unknown, 149 active+clean; 3.5 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 9090/174384 objects misplaced (5.213%); 1.1 MiB/s, 18 objects/s recovering 2024-09-20T19:07:02.228 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:01 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:07:02.228 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:01 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:07:02.249 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:07:02.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:01 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1926531685' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:07:02.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:01 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:07:02.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:01 smithi142 ceph-mon[32508]: pgmap v877: 158 pgs: 2 active+recovering+undersized+remapped, 2 peering, 5 unknown, 149 active+clean; 3.5 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 9090/174384 objects misplaced (5.213%); 1.1 MiB/s, 18 objects/s recovering 2024-09-20T19:07:02.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:01 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:07:02.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:01 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:07:02.280 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:02.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:02.281 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:07:02.381 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:02.416 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:02.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:02.417 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:02.518 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:07:02.551 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:02.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:02.552 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:07:02.653 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:02.691 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:02.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:02.693 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:02.793 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:02.830 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:02.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:02.831 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:02.932 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:02.958 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:02.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:02.959 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:03.060 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:03.102 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:03.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:03.103 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:03.118 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:2024-09-20T19:07:03.115210+0000 min lat: 0.00109192 max lat: 2.36333 avg lat: 0.00988745 2024-09-20T19:07:03.118 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-20T19:07:03.118 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 40 16 64722 64706 101.083 42.25 0.0138676 0.00988745 2024-09-20T19:07:03.204 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:03.233 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:03.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:03.234 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:03.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:02 smithi142 ceph-mon[32508]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:07:03.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:02 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T19:07:03.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:02 smithi142 ceph-mon[32508]: osdmap e446: 8 total, 8 up, 8 in 2024-09-20T19:07:03.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:02 smithi083 ceph-mon[29421]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:07:03.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:02 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T19:07:03.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:02 smithi083 ceph-mon[29421]: osdmap e446: 8 total, 8 up, 8 in 2024-09-20T19:07:03.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:02 smithi083 ceph-mon[36260]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:07:03.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:02 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T19:07:03.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:02 smithi083 ceph-mon[36260]: osdmap e446: 8 total, 8 up, 8 in 2024-09-20T19:07:03.335 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:03.375 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:03.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:03.376 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:03.477 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:07:03.512 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:03.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:03.513 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:07:03.614 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:03.635 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:03.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:03.636 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:03.737 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:03.783 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:03.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:03.784 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:03.885 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:03.905 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:03.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:03.907 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:04.007 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:04.041 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:04.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:04.042 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:04.118 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 41 16 65504 65488 99.8096 48.875 0.014706 0.0100125 2024-09-20T19:07:04.143 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:04.164 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:04.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:04.165 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:07:04.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:03 smithi142 ceph-mon[32508]: pgmap v879: 158 pgs: 4 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 148 active+clean; 3.9 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 893 B/s rd, 0 op/s; 13750/191922 objects degraded (7.164%); 29962/191922 objects misplaced (15.612%); 5.4 MiB/s, 86 objects/s recovering 2024-09-20T19:07:04.265 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:04.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:03 smithi083 ceph-mon[29421]: pgmap v879: 158 pgs: 4 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 148 active+clean; 3.9 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 893 B/s rd, 0 op/s; 13750/191922 objects degraded (7.164%); 29962/191922 objects misplaced (15.612%); 5.4 MiB/s, 86 objects/s recovering 2024-09-20T19:07:04.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:03 smithi083 ceph-mon[36260]: pgmap v879: 158 pgs: 4 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 148 active+clean; 3.9 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 893 B/s rd, 0 op/s; 13750/191922 objects degraded (7.164%); 29962/191922 objects misplaced (15.612%); 5.4 MiB/s, 86 objects/s recovering 2024-09-20T19:07:04.309 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:04.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:04.310 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:04.411 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:07:04.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:04.444 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:04.444 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:07:04.545 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:04.595 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:04.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:04.596 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:04.626 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 7, 6, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 5, 6, 7, 4] 2024-09-20T19:07:04.626 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T19:07:04.626 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-09-20T19:07:04.626 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-20T19:07:04.626 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-20T19:07:04.626 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.7 config set filestore_inject_stall 3 2024-09-20T19:07:04.697 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:04.732 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:04.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:04.733 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:04.834 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:07:04.859 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:04.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:04.860 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:07:04.960 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:05.011 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:05.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:05.012 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:05.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:04 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:07:05.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:04 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:07:05.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:04 smithi083 ceph-mon[29421]: Health check failed: Degraded data redundancy: 13750/191922 objects degraded (7.164%), 4 pgs degraded (PG_DEGRADED) 2024-09-20T19:07:05.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:04 smithi083 ceph-mon[29421]: osdmap e447: 8 total, 8 up, 8 in 2024-09-20T19:07:05.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:04 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:07:05.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:04 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:07:05.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:04 smithi083 ceph-mon[36260]: Health check failed: Degraded data redundancy: 13750/191922 objects degraded (7.164%), 4 pgs degraded (PG_DEGRADED) 2024-09-20T19:07:05.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:04 smithi083 ceph-mon[36260]: osdmap e447: 8 total, 8 up, 8 in 2024-09-20T19:07:05.044 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:07:05.113 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:07:05.118 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 42 16 66402 66386 98.7695 56.125 0.0223271 0.0101193 2024-09-20T19:07:05.145 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:05.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:05.146 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:07:05.236 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:04 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:07:05.237 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:04 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:07:05.237 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:04 smithi142 ceph-mon[32508]: Health check failed: Degraded data redundancy: 13750/191922 objects degraded (7.164%), 4 pgs degraded (PG_DEGRADED) 2024-09-20T19:07:05.237 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:04 smithi142 ceph-mon[32508]: osdmap e447: 8 total, 8 up, 8 in 2024-09-20T19:07:05.247 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:07:05.268 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:05.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:05.269 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:07:05.370 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:05.412 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:05.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:05.413 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:05.514 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:05.553 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:05.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:05.555 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:05.655 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:07:05.673 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T19:07:05.673 INFO:teuthology.orchestra.run.smithi142.stdout: "success": "" 2024-09-20T19:07:05.673 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T19:07:05.691 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:05.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:05.692 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:07:05.793 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:05.835 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:05.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:05.836 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:05.937 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:05.966 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:05.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:05.967 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:06.068 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:06.101 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:06.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:06.102 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:06.119 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 43 16 67143 67127 97.5496 46.3125 0.00243052 0.0102366 2024-09-20T19:07:06.202 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:06.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:06.229 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:06.229 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:06.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:05 smithi142 ceph-mon[32508]: pgmap v881: 158 pgs: 4 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 148 active+clean; 3.9 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 13750/191922 objects degraded (7.164%); 29962/191922 objects misplaced (15.612%); 5.2 MiB/s, 82 objects/s recovering 2024-09-20T19:07:06.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:05 smithi083 ceph-mon[29421]: pgmap v881: 158 pgs: 4 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 148 active+clean; 3.9 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 13750/191922 objects degraded (7.164%); 29962/191922 objects misplaced (15.612%); 5.2 MiB/s, 82 objects/s recovering 2024-09-20T19:07:06.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:05 smithi083 ceph-mon[36260]: pgmap v881: 158 pgs: 4 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 148 active+clean; 3.9 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 13750/191922 objects degraded (7.164%); 29962/191922 objects misplaced (15.612%); 5.2 MiB/s, 82 objects/s recovering 2024-09-20T19:07:06.330 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:06.355 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:06.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:06.356 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:06.457 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:06.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:06.493 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:06.494 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:07:06.594 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:06.620 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:06.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:06.620 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:06.721 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:06.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:06.745 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:06.745 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:06.846 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:06.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:06.887 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:06.888 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:07:06.988 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:07.010 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:07.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:07.010 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:07:07.111 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:07.119 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 44 16 67615 67599 96.0031 29.5 0.00423866 0.0103939 2024-09-20T19:07:07.136 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:07.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:07.137 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:07.237 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:07.261 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:07.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:07.262 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:07.362 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:07.394 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:07.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:07.395 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:07.496 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:07.546 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:07.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:07.548 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:07.648 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:07:07.670 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:07.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:07.671 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:07:07.761 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:07:07 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:07:07.772 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:07.806 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:07.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:07.807 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:07.908 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:07.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:07.953 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:07.953 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:07:08.054 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:07:08.085 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:08.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:08.086 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:07:08.118 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 45 16 68426 68410 94.9961 50.6875 0.00488976 0.0105201 2024-09-20T19:07:08.187 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:08.212 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:08.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:08.213 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:08.314 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:08.360 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:08.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:08.360 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:08.461 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:08.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:08.495 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:08.496 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:08.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:08 smithi142 ceph-mon[32508]: pgmap v882: 158 pgs: 4 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 148 active+clean; 4.0 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s; 13750/199482 objects degraded (6.893%); 28596/199482 objects misplaced (14.335%); 8.8 MiB/s, 140 objects/s recovering 2024-09-20T19:07:08.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:08 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:07:08.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:08 smithi083 ceph-mon[29421]: pgmap v882: 158 pgs: 4 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 148 active+clean; 4.0 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s; 13750/199482 objects degraded (6.893%); 28596/199482 objects misplaced (14.335%); 8.8 MiB/s, 140 objects/s recovering 2024-09-20T19:07:08.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:08 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:07:08.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:08 smithi083 ceph-mon[36260]: pgmap v882: 158 pgs: 4 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 148 active+clean; 4.0 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s; 13750/199482 objects degraded (6.893%); 28596/199482 objects misplaced (14.335%); 8.8 MiB/s, 140 objects/s recovering 2024-09-20T19:07:08.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:08 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:07:08.597 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:07:08.631 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:08.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:08.632 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:07:08.733 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:08.780 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:08.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:08.781 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:07:08.882 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:08.921 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:08.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:08.922 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:09.023 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:07:09.050 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:09.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:09.051 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:07:09.119 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 46 16 69382 69366 94.2298 59.75 0.00630249 0.0106055 2024-09-20T19:07:09.151 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:09.201 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:09.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:09.202 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:09.303 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:09.341 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:09.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:09.342 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:09.443 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:09.476 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:09.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:09.477 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:09.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:09 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 13750/203064 objects degraded (6.771%), 4 pgs degraded (PG_DEGRADED) 2024-09-20T19:07:09.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:09 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 13750/203064 objects degraded (6.771%), 4 pgs degraded (PG_DEGRADED) 2024-09-20T19:07:09.577 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:09.624 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:09.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:09.625 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:07:09.725 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:09.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:09 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 13750/203064 objects degraded (6.771%), 4 pgs degraded (PG_DEGRADED) 2024-09-20T19:07:09.765 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:09.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:09.766 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:09.866 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:09.888 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:09.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:09.889 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:09.989 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:10.030 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:10.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:10.031 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:07:10.127 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 47 16 70275 70259 93.3971 55.8125 0.0302176 0.0107005 2024-09-20T19:07:10.132 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:10.161 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:10.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:10.161 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:10.262 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:07:10.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:10.303 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:10.303 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:07:10.403 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:10.442 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:10.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:10.443 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:10.544 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:10.581 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:10.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:10.582 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:10.683 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:10.725 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:10 smithi083 ceph-mon[29421]: pgmap v883: 158 pgs: 4 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 148 active+clean; 4.1 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 14 MiB/s wr, 218 op/s; 13750/203064 objects degraded (6.771%); 27425/203064 objects misplaced (13.506%); 13 MiB/s, 211 objects/s recovering 2024-09-20T19:07:10.725 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:10 smithi083 ceph-mon[36260]: pgmap v883: 158 pgs: 4 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 148 active+clean; 4.1 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 14 MiB/s wr, 218 op/s; 13750/203064 objects degraded (6.771%); 27425/203064 objects misplaced (13.506%); 13 MiB/s, 211 objects/s recovering 2024-09-20T19:07:10.730 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:10.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:10.731 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:10.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:10 smithi142 ceph-mon[32508]: pgmap v883: 158 pgs: 4 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 148 active+clean; 4.1 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 14 MiB/s wr, 218 op/s; 13750/203064 objects degraded (6.771%); 27425/203064 objects misplaced (13.506%); 13 MiB/s, 211 objects/s recovering 2024-09-20T19:07:10.832 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:10.872 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:10.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:10.873 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:07:10.974 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:07:11.003 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:11.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:11.004 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:07:11.041 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:07:10 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:07:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:07:11.104 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:11.127 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 48 16 71210 71194 92.6689 58.4375 0.00649063 0.0107839 2024-09-20T19:07:11.139 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:11.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:11.140 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:11.241 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:11.276 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:11.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:11.277 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:11.377 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:11.415 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:11.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:11.416 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:11.517 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:11.555 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:11.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:11.557 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:11.657 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:11.690 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:11.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:11.691 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:11.742 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 7, 6, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 5, 6, 7, 4] 2024-09-20T19:07:11.742 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T19:07:11.742 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2024-09-20T19:07:11.742 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T19:07:11.792 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:11.861 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:11.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:11.862 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:11.963 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:11.995 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:12.043 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:11 smithi083 ceph-mon[29421]: pgmap v884: 158 pgs: 4 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 148 active+clean; 4.2 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 553 B/s rd, 43 MiB/s wr, 683 op/s; 13750/207231 objects degraded (6.635%); 26500/207231 objects misplaced (12.788%); 14 MiB/s, 222 objects/s recovering 2024-09-20T19:07:12.043 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:11 smithi083 ceph-mon[36260]: pgmap v884: 158 pgs: 4 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 148 active+clean; 4.2 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 553 B/s rd, 43 MiB/s wr, 683 op/s; 13750/207231 objects degraded (6.635%); 26500/207231 objects misplaced (12.788%); 14 MiB/s, 222 objects/s recovering 2024-09-20T19:07:12.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:12.044 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:12.130 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 49 16 72115 72099 91.9306 56.5625 0.0222047 0.0108726 2024-09-20T19:07:12.144 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:07:12.171 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:12.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:12.172 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:07:12.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:11 smithi142 ceph-mon[32508]: pgmap v884: 158 pgs: 4 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 148 active+clean; 4.2 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 553 B/s rd, 43 MiB/s wr, 683 op/s; 13750/207231 objects degraded (6.635%); 26500/207231 objects misplaced (12.788%); 14 MiB/s, 222 objects/s recovering 2024-09-20T19:07:12.273 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:07:12.298 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:12.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:12.299 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:07:12.399 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:12.432 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:12.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:12.433 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:12.534 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:12.575 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:12.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:12.576 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:12.676 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:12.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:12.717 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:12.717 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:12.818 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:12.864 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:12.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:12.865 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:12.965 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:07:13.000 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:13.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:13.001 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:07:13.102 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:13.127 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 50 16 73003 72987 91.2018 55.5 0.02149 0.0109588 2024-09-20T19:07:13.143 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:13.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:13.144 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:13.245 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:13.290 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:13.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:13.291 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:13.392 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:13.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:13.430 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:13.430 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:07:13.531 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:13.570 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:13.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:13.571 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:13.672 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:13.689 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_historic_ops 2024-09-20T19:07:13.699 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:13.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:13.700 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:13.790 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:07:13.790 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:07:13.801 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:13.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:13.883 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:13.883 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:07:13.984 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:14.015 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:14.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:14.016 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:14.117 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:14.129 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 51 16 73854 73838 90.4547 53.1875 0.0144163 0.0110495 2024-09-20T19:07:14.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:14.171 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:14.171 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:14.272 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:14.289 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:14 smithi083 ceph-mon[29421]: pgmap v885: 158 pgs: 4 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 148 active+clean; 4.4 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1023 B/s rd, 50 MiB/s wr, 805 op/s; 13750/216066 objects degraded (6.364%); 24764/216066 objects misplaced (11.461%); 16 MiB/s, 261 objects/s recovering 2024-09-20T19:07:14.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:14 smithi083 ceph-mon[36260]: pgmap v885: 158 pgs: 4 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 148 active+clean; 4.4 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1023 B/s rd, 50 MiB/s wr, 805 op/s; 13750/216066 objects degraded (6.364%); 24764/216066 objects misplaced (11.461%); 16 MiB/s, 261 objects/s recovering 2024-09-20T19:07:14.329 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:14.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:14.330 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:14.431 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:07:14.463 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:14.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:14.465 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:07:14.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:14 smithi142 ceph-mon[32508]: pgmap v885: 158 pgs: 4 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 148 active+clean; 4.4 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1023 B/s rd, 50 MiB/s wr, 805 op/s; 13750/216066 objects degraded (6.364%); 24764/216066 objects misplaced (11.461%); 16 MiB/s, 261 objects/s recovering 2024-09-20T19:07:14.566 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:14.603 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:14.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:14.604 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:14.705 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:14.738 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:14.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:14.739 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:14.840 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:14.883 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:14.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:14.883 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:14.984 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:07:15.011 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:15.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:15.012 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:07:15.055 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:07:15.055 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:07:15.100 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:07:15.100 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:07:15.112 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:15.130 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 52 15 74752 74737 89.7942 56.1875 0.0064574 0.01113 2024-09-20T19:07:15.138 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:15.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:15.139 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:15.152 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:07:15.161 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:07:15.162 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:07:15.217 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:07:15.217 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:07:15.239 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:15.257 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:07:15.257 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:07:15.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:15.284 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:15.284 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:15.293 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:07:15.293 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:07:15.346 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:07:15.346 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:07:15.385 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:15.402 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:07:15.403 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:07:15.432 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T19:07:15.437 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:15.438 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-20T19:07:15.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:15.439 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:15.499 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:15 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/488158026' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:07:15.499 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:15 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/488158026' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:07:15.503 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T19:07:15.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:15 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/488158026' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:07:15.540 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:15.576 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:15.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:15.577 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:15.678 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:15.731 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:15.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:15.732 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:15.833 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:07:15.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:15.862 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:15.862 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:07:15.963 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:16.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:16.034 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:16.034 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:16.131 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 53 16 75439 75423 88.9077 42.875 0.0148824 0.0112412 2024-09-20T19:07:16.135 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:07:16.164 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:16.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:16.165 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:07:16.266 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:07:16.311 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:16.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:16.312 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:07:16.412 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:16.442 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:16.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:16.443 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:16.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:16 smithi142 ceph-mon[32508]: pgmap v886: 158 pgs: 4 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 148 active+clean; 4.4 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 910 B/s rd, 45 MiB/s wr, 716 op/s; 13750/216066 objects degraded (6.364%); 24764/216066 objects misplaced (11.461%); 15 MiB/s, 232 objects/s recovering 2024-09-20T19:07:16.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:16 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T19:07:16.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:16 smithi142 ceph-mon[32508]: from='client.16488 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:07:16.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:16 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 13750/216066 objects degraded (6.364%), 4 pgs degraded (PG_DEGRADED) 2024-09-20T19:07:16.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:16 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T19:07:16.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:16 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T19:07:16.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:16 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:07:16.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:16 smithi083 ceph-mon[29421]: pgmap v886: 158 pgs: 4 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 148 active+clean; 4.4 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 910 B/s rd, 45 MiB/s wr, 716 op/s; 13750/216066 objects degraded (6.364%); 24764/216066 objects misplaced (11.461%); 15 MiB/s, 232 objects/s recovering 2024-09-20T19:07:16.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:16 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T19:07:16.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:16 smithi083 ceph-mon[29421]: from='client.16488 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:07:16.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:16 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 13750/216066 objects degraded (6.364%), 4 pgs degraded (PG_DEGRADED) 2024-09-20T19:07:16.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:16 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T19:07:16.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:16 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T19:07:16.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:16 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:07:16.542 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:16 smithi083 ceph-mon[36260]: pgmap v886: 158 pgs: 4 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 148 active+clean; 4.4 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 910 B/s rd, 45 MiB/s wr, 716 op/s; 13750/216066 objects degraded (6.364%); 24764/216066 objects misplaced (11.461%); 15 MiB/s, 232 objects/s recovering 2024-09-20T19:07:16.542 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:16 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T19:07:16.542 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:16 smithi083 ceph-mon[36260]: from='client.16488 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:07:16.542 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:16 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 13750/216066 objects degraded (6.364%), 4 pgs degraded (PG_DEGRADED) 2024-09-20T19:07:16.542 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:16 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T19:07:16.542 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:16 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T19:07:16.542 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:16 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:07:16.544 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:16.594 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:16.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:16.595 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:16.682 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":886,"stamp":"2024-09-20T19:07:15.180450+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4707457563,"num_objects":72022,"num_object_clones":0,"num_object_copies":216066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13750,"num_objects_misplaced":24764,"num_objects_unfound":0,"num_objects_dirty":72022,"num_whiteouts":0,"num_read":2295,"num_read_kb":2152,"num_write":59035,"num_write_kb":3726345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3402,"num_bytes_recovered":205530587,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110287,"ondisk_log_size":110287,"up":474,"acting":464,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":497,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":19318748,"kb_used_data":14181916,"kb_used_omap":16,"kb_used_meta":5136687,"kb_avail":730642468,"statfs":{"total":767960285184,"available":748177887232,"internally_reserved":0,"allocated":14522281984,"data_stored":14516706409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16790,"internal_metadata":5259968106},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[8,6,6,4,5,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":527433728,"num_objects":8048,"num_object_clones":0,"num_object_copies":24144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-5198,"num_objects_unfound":0,"num_objects_dirty":8048,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":8048,"num_write_kb":515072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2613,"num_bytes_recovered":171245568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.245954"},"pg_stats":[{"pgid":"11.17","version":"447'4152","reported_seq":5600,"reported_epoch":447,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-20T19:07:12.273253+0000","last_change":"2024-09-20T19:06:59.820580+0000","last_active":"2024-09-20T19:07:12.273253+0000","last_peered":"2024-09-20T19:07:12.273253+0000","last_clean":"2024-09-20T19:06:57.409724+0000","last_became_active":"2024-09-20T19:06:59.725195+0000","last_became_peered":"2024-09-20T19:06:59.725195+0000","last_unstale":"2024-09-20T19:07:12.273253+0000","last_undegraded":"2024-09-20T19:07:12.273253+0000","last_fullsized":"2024-09-20T19:06:59.451494+0000","mapping_epoch":444,"log_start":"0'0","ondisk_log_start":"0'0","created":441,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":4152,"log_dups_size":0,"ondisk_log_size":4152,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":147292160,"num_objects":2248,"num_object_clones":0,"num_object_copies":6744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3024,"num_objects_unfound":0,"num_objects_dirty":2248,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2248,"num_write_kb":143840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":460,"num_bytes_recovered":30146560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,5],"acting":[1,5],"avail_no_missing":["1","4","5"],"object_location_counts":[{"shards":"1,4,5","objects":1512},{"shards":"1,5","objects":736}],"blocked_by":[],"up_primary":7,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":826,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.965605+0000","last_change":"2024-09-20T19:01:31.817967+0000","last_active":"2024-09-20T19:07:03.965605+0000","last_peered":"2024-09-20T19:07:03.965605+0000","last_clean":"2024-09-20T19:07:03.965605+0000","last_became_active":"2024-09-20T18:49:45.943201+0000","last_became_peered":"2024-09-20T18:49:45.943201+0000","last_unstale":"2024-09-20T19:07:03.965605+0000","last_undegraded":"2024-09-20T19:07:03.965605+0000","last_fullsized":"2024-09-20T19:07:03.965605+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:31.817909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:31.817909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"273'15","reported_seq":174,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.942101+0000","last_change":"2024-09-20T19:06:46.500942+0000","last_active":"2024-09-20T19:07:03.942101+0000","last_peered":"2024-09-20T19:07:03.942101+0000","last_clean":"2024-09-20T19:07:03.942101+0000","last_became_active":"2024-09-20T19:04:20.925853+0000","last_became_peered":"2024-09-20T19:04:20.925853+0000","last_unstale":"2024-09-20T19:07:03.942101+0000","last_undegraded":"2024-09-20T19:07:03.942101+0000","last_fullsized":"2024-09-20T19:07:03.942101+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:06:46.500881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:06:46.500881+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:58.004473+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":38,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.963484+0000","last_change":"2024-09-20T19:06:40.320839+0000","last_active":"2024-09-20T19:07:03.963484+0000","last_peered":"2024-09-20T19:07:03.963484+0000","last_clean":"2024-09-20T19:07:03.963484+0000","last_became_active":"2024-09-20T19:06:40.320505+0000","last_became_peered":"2024-09-20T19:06:40.320505+0000","last_unstale":"2024-09-20T19:07:03.963484+0000","last_undegraded":"2024-09-20T19:07:03.963484+0000","last_fullsized":"2024-09-20T19:07:03.963484+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:49.016006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:49.016006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:40.896471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":156,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.942560+0000","last_change":"2024-09-20T19:04:20.881918+0000","last_active":"2024-09-20T19:07:03.942560+0000","last_peered":"2024-09-20T19:07:03.942560+0000","last_clean":"2024-09-20T19:07:03.942560+0000","last_became_active":"2024-09-20T19:04:20.880905+0000","last_became_peered":"2024-09-20T19:04:20.880905+0000","last_unstale":"2024-09-20T19:07:03.942560+0000","last_undegraded":"2024-09-20T19:07:03.942560+0000","last_fullsized":"2024-09-20T19:07:03.942560+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:02.084375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:02.084375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:39.438073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"447'4241","reported_seq":5826,"reported_epoch":447,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-20T19:07:12.762716+0000","last_change":"2024-09-20T19:06:59.879404+0000","last_active":"2024-09-20T19:07:12.762716+0000","last_peered":"2024-09-20T19:07:12.762716+0000","last_clean":"2024-09-20T19:06:57.451557+0000","last_became_active":"2024-09-20T19:06:59.765746+0000","last_became_peered":"2024-09-20T19:06:59.765746+0000","last_unstale":"2024-09-20T19:07:12.762716+0000","last_undegraded":"2024-09-20T19:07:12.762716+0000","last_fullsized":"2024-09-20T19:06:59.454904+0000","mapping_epoch":444,"log_start":"0'0","ondisk_log_start":"0'0","created":441,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":4241,"log_dups_size":0,"ondisk_log_size":4241,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":152371200,"num_objects":2325,"num_object_clones":0,"num_object_copies":6975,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4782,"num_objects_unfound":0,"num_objects_dirty":2325,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2325,"num_write_kb":148800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":378,"num_bytes_recovered":24772608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[0,3],"avail_no_missing":["0","1","2"],"object_location_counts":[{"shards":"0,1,2","objects":1594},{"shards":"0,3","objects":731}],"blocked_by":[],"up_primary":5,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":825,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.958039+0000","last_change":"2024-09-20T19:02:02.441633+0000","last_active":"2024-09-20T19:07:03.958039+0000","last_peered":"2024-09-20T19:07:03.958039+0000","last_clean":"2024-09-20T19:07:03.958039+0000","last_became_active":"2024-09-20T18:59:05.081081+0000","last_became_peered":"2024-09-20T18:59:05.081081+0000","last_unstale":"2024-09-20T19:07:03.958039+0000","last_undegraded":"2024-09-20T19:07:03.958039+0000","last_fullsized":"2024-09-20T19:07:03.958039+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:02.441577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:02.441577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:11.774126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":839,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.963812+0000","last_change":"2024-09-20T19:04:37.540824+0000","last_active":"2024-09-20T19:07:03.963812+0000","last_peered":"2024-09-20T19:07:03.963812+0000","last_clean":"2024-09-20T19:07:03.963812+0000","last_became_active":"2024-09-20T18:59:05.072675+0000","last_became_peered":"2024-09-20T18:59:05.072675+0000","last_unstale":"2024-09-20T19:07:03.963812+0000","last_undegraded":"2024-09-20T19:07:03.963812+0000","last_fullsized":"2024-09-20T19:07:03.963812+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:37.540760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:04:37.540760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:30.957798+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"274'27","reported_seq":871,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.963989+0000","last_change":"2024-09-20T19:05:50.750134+0000","last_active":"2024-09-20T19:07:03.963989+0000","last_peered":"2024-09-20T19:07:03.963989+0000","last_clean":"2024-09-20T19:07:03.963989+0000","last_became_active":"2024-09-20T18:49:47.952150+0000","last_became_peered":"2024-09-20T18:49:47.952150+0000","last_unstale":"2024-09-20T19:07:03.963989+0000","last_undegraded":"2024-09-20T19:07:03.963989+0000","last_fullsized":"2024-09-20T19:07:03.963989+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'27","last_scrub_stamp":"2024-09-20T19:05:50.750050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:50.750050+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:13.513790+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":164,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.942119+0000","last_change":"2024-09-20T19:06:25.218221+0000","last_active":"2024-09-20T19:07:03.942119+0000","last_peered":"2024-09-20T19:07:03.942119+0000","last_clean":"2024-09-20T19:07:03.942119+0000","last_became_active":"2024-09-20T19:04:20.925394+0000","last_became_peered":"2024-09-20T19:04:20.925394+0000","last_unstale":"2024-09-20T19:07:03.942119+0000","last_undegraded":"2024-09-20T19:07:03.942119+0000","last_fullsized":"2024-09-20T19:07:03.942119+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:25.218150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:06:25.218150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:38.749309+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"447'4121","reported_seq":5881,"reported_epoch":447,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-20T19:07:10.794727+0000","last_change":"2024-09-20T19:06:59.701369+0000","last_active":"2024-09-20T19:07:10.794727+0000","last_peered":"2024-09-20T19:07:10.794727+0000","last_clean":"2024-09-20T19:06:57.439089+0000","last_became_active":"2024-09-20T19:06:59.678574+0000","last_became_peered":"2024-09-20T19:06:59.678574+0000","last_unstale":"2024-09-20T19:07:10.794727+0000","last_undegraded":"2024-09-20T19:07:10.794727+0000","last_fullsized":"2024-09-20T19:06:59.455742+0000","mapping_epoch":444,"log_start":"0'0","ondisk_log_start":"0'0","created":441,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":4121,"log_dups_size":0,"ondisk_log_size":4121,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":145424384,"num_objects":2219,"num_object_clones":0,"num_object_copies":6657,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4376,"num_objects_unfound":0,"num_objects_dirty":2219,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2219,"num_write_kb":142016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":421,"num_bytes_recovered":27590656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[5,1],"avail_no_missing":["5","6","7"],"object_location_counts":[{"shards":"1,5","objects":760},{"shards":"4,5,6,7","objects":1},{"shards":"5,6,7","objects":1458}],"blocked_by":[],"up_primary":2,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"59'1","reported_seq":900,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.971304+0000","last_change":"2024-09-20T19:04:20.963496+0000","last_active":"2024-09-20T19:07:03.971304+0000","last_peered":"2024-09-20T19:07:03.971304+0000","last_clean":"2024-09-20T19:07:03.971304+0000","last_became_active":"2024-09-20T19:04:20.914145+0000","last_became_peered":"2024-09-20T19:04:20.914145+0000","last_unstale":"2024-09-20T19:07:03.971304+0000","last_undegraded":"2024-09-20T19:07:03.971304+0000","last_fullsized":"2024-09-20T19:07:03.971304+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:01:28.592653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:28.592653+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:33.945469+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"273'15","reported_seq":166,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.942671+0000","last_change":"2024-09-20T19:04:20.984223+0000","last_active":"2024-09-20T19:07:03.942671+0000","last_peered":"2024-09-20T19:07:03.942671+0000","last_clean":"2024-09-20T19:07:03.942671+0000","last_became_active":"2024-09-20T19:04:20.898235+0000","last_became_peered":"2024-09-20T19:04:20.898235+0000","last_unstale":"2024-09-20T19:07:03.942671+0000","last_undegraded":"2024-09-20T19:07:03.942671+0000","last_fullsized":"2024-09-20T19:07:03.942671+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:02:42.114331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:42.114331+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:28.178800+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":831,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.963672+0000","last_change":"2024-09-20T19:06:47.942245+0000","last_active":"2024-09-20T19:07:03.963672+0000","last_peered":"2024-09-20T19:07:03.963672+0000","last_clean":"2024-09-20T19:07:03.963672+0000","last_became_active":"2024-09-20T18:49:49.961188+0000","last_became_peered":"2024-09-20T18:49:49.961188+0000","last_unstale":"2024-09-20T19:07:03.963672+0000","last_undegraded":"2024-09-20T19:07:03.963672+0000","last_fullsized":"2024-09-20T19:07:03.963672+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:47.942182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:06:47.942182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:08:14.235108+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":813,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.965793+0000","last_change":"2024-09-20T19:06:53.956421+0000","last_active":"2024-09-20T19:07:03.965793+0000","last_peered":"2024-09-20T19:07:03.965793+0000","last_clean":"2024-09-20T19:07:03.965793+0000","last_became_active":"2024-09-20T18:49:51.966715+0000","last_became_peered":"2024-09-20T18:49:51.966715+0000","last_unstale":"2024-09-20T19:07:03.965793+0000","last_undegraded":"2024-09-20T19:07:03.965793+0000","last_fullsized":"2024-09-20T19:07:03.965793+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:53.956349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:06:53.956349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:08:21.845549+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"447'4149","reported_seq":6018,"reported_epoch":447,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-20T19:07:11.967877+0000","last_change":"2024-09-20T19:06:59.702232+0000","last_active":"2024-09-20T19:07:11.967877+0000","last_peered":"2024-09-20T19:07:11.967877+0000","last_clean":"2024-09-20T19:06:57.450979+0000","last_became_active":"2024-09-20T19:06:59.669793+0000","last_became_peered":"2024-09-20T19:06:59.669793+0000","last_unstale":"2024-09-20T19:07:11.967877+0000","last_undegraded":"2024-09-20T19:07:11.967877+0000","last_fullsized":"2024-09-20T19:06:59.454891+0000","mapping_epoch":444,"log_start":"0'0","ondisk_log_start":"0'0","created":441,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":4149,"log_dups_size":0,"ondisk_log_size":4149,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":147914752,"num_objects":2257,"num_object_clones":0,"num_object_copies":6771,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2730,"num_objects_unfound":0,"num_objects_dirty":2257,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2257,"num_write_kb":144448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":599,"num_bytes_recovered":39256064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,1],"avail_no_missing":["6","2","7"],"object_location_counts":[{"shards":"1,6","objects":892},{"shards":"2,6,7","objects":1365}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":848,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.963938+0000","last_change":"2024-09-20T19:04:20.879227+0000","last_active":"2024-09-20T19:07:03.963938+0000","last_peered":"2024-09-20T19:07:03.963938+0000","last_clean":"2024-09-20T19:07:03.963938+0000","last_became_active":"2024-09-20T19:04:20.878905+0000","last_became_peered":"2024-09-20T19:04:20.878905+0000","last_unstale":"2024-09-20T19:07:03.963938+0000","last_undegraded":"2024-09-20T19:07:03.963938+0000","last_fullsized":"2024-09-20T19:07:03.963938+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:48.520017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:48.520017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"273'7","reported_seq":170,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.942426+0000","last_change":"2024-09-20T19:05:53.980848+0000","last_active":"2024-09-20T19:07:03.942426+0000","last_peered":"2024-09-20T19:07:03.942426+0000","last_clean":"2024-09-20T19:07:03.942426+0000","last_became_active":"2024-09-20T19:04:20.888572+0000","last_became_peered":"2024-09-20T19:04:20.888572+0000","last_unstale":"2024-09-20T19:07:03.942426+0000","last_undegraded":"2024-09-20T19:07:03.942426+0000","last_fullsized":"2024-09-20T19:07:03.942426+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'7","last_scrub_stamp":"2024-09-20T19:05:53.980772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:53.980772+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":841,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.961943+0000","last_change":"2024-09-20T19:04:20.885873+0000","last_active":"2024-09-20T19:07:03.961943+0000","last_peered":"2024-09-20T19:07:03.961943+0000","last_clean":"2024-09-20T19:07:03.961943+0000","last_became_active":"2024-09-20T19:04:20.885417+0000","last_became_peered":"2024-09-20T19:04:20.885417+0000","last_unstale":"2024-09-20T19:07:03.961943+0000","last_undegraded":"2024-09-20T19:07:03.961943+0000","last_fullsized":"2024-09-20T19:07:03.961943+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:33.203312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:33.203312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:05.802150+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":820,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.963564+0000","last_change":"2024-09-20T19:05:21.625378+0000","last_active":"2024-09-20T19:07:03.963564+0000","last_peered":"2024-09-20T19:07:03.963564+0000","last_clean":"2024-09-20T19:07:03.963564+0000","last_became_active":"2024-09-20T18:49:51.970663+0000","last_became_peered":"2024-09-20T18:49:51.970663+0000","last_unstale":"2024-09-20T19:07:03.963564+0000","last_undegraded":"2024-09-20T19:07:03.963564+0000","last_fullsized":"2024-09-20T19:07:03.963564+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:21.625311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:05:21.625311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:35.920771+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"447'4209","reported_seq":4279,"reported_epoch":447,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-20T19:07:12.252321+0000","last_change":"2024-09-20T19:06:59.757437+0000","last_active":"2024-09-20T19:07:12.252321+0000","last_peered":"2024-09-20T19:07:12.252321+0000","last_clean":"2024-09-20T19:06:57.409541+0000","last_became_active":"2024-09-20T19:06:59.724211+0000","last_became_peered":"2024-09-20T19:06:59.724211+0000","last_unstale":"2024-09-20T19:07:12.252321+0000","last_undegraded":"2024-09-20T19:06:59.717720+0000","last_fullsized":"2024-09-20T19:06:59.451727+0000","mapping_epoch":444,"log_start":"0'0","ondisk_log_start":"0'0","created":441,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":4209,"log_dups_size":0,"ondisk_log_size":4209,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":149880832,"num_objects":2287,"num_object_clones":0,"num_object_copies":6861,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4092,"num_objects_misplaced":2046,"num_objects_unfound":0,"num_objects_dirty":2287,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2287,"num_write_kb":146368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[1,0],"avail_no_missing":["1","6","7"],"object_location_counts":[{"shards":"0,1","objects":241},{"shards":"1,6,7","objects":2046}],"blocked_by":[],"up_primary":5,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":164,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.942975+0000","last_change":"2024-09-20T19:06:45.481243+0000","last_active":"2024-09-20T19:07:03.942975+0000","last_peered":"2024-09-20T19:07:03.942975+0000","last_clean":"2024-09-20T19:07:03.942975+0000","last_became_active":"2024-09-20T19:04:20.906109+0000","last_became_peered":"2024-09-20T19:04:20.906109+0000","last_unstale":"2024-09-20T19:07:03.942975+0000","last_undegraded":"2024-09-20T19:07:03.942975+0000","last_fullsized":"2024-09-20T19:07:03.942975+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:45.479545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:06:45.479545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:08:14.852920+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":849,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.966761+0000","last_change":"2024-09-20T19:04:20.907057+0000","last_active":"2024-09-20T19:07:03.966761+0000","last_peered":"2024-09-20T19:07:03.966761+0000","last_clean":"2024-09-20T19:07:03.966761+0000","last_became_active":"2024-09-20T19:04:20.906691+0000","last_became_peered":"2024-09-20T19:04:20.906691+0000","last_unstale":"2024-09-20T19:07:03.966761+0000","last_undegraded":"2024-09-20T19:07:03.966761+0000","last_fullsized":"2024-09-20T19:07:03.966761+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:44.998147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:44.998147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:28.750471+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"273'27","reported_seq":881,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.962517+0000","last_change":"2024-09-20T19:05:49.452928+0000","last_active":"2024-09-20T19:07:03.962517+0000","last_peered":"2024-09-20T19:07:03.962517+0000","last_clean":"2024-09-20T19:07:03.962517+0000","last_became_active":"2024-09-20T18:49:47.950939+0000","last_became_peered":"2024-09-20T18:49:47.950939+0000","last_unstale":"2024-09-20T19:07:03.962517+0000","last_undegraded":"2024-09-20T19:07:03.962517+0000","last_fullsized":"2024-09-20T19:07:03.962517+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'27","last_scrub_stamp":"2024-09-20T19:05:49.452861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:49.452861+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":832,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.978942+0000","last_change":"2024-09-20T19:04:20.890659+0000","last_active":"2024-09-20T19:07:03.978942+0000","last_peered":"2024-09-20T19:07:03.978942+0000","last_clean":"2024-09-20T19:07:03.978942+0000","last_became_active":"2024-09-20T19:04:20.890370+0000","last_became_peered":"2024-09-20T19:04:20.890370+0000","last_unstale":"2024-09-20T19:07:03.978942+0000","last_undegraded":"2024-09-20T19:07:03.978942+0000","last_fullsized":"2024-09-20T19:07:03.978942+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:05.888974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:05.888974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:29.757611+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"447'4116","reported_seq":5919,"reported_epoch":447,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-20T19:07:12.769761+0000","last_change":"2024-09-20T19:06:59.717605+0000","last_active":"2024-09-20T19:07:12.769761+0000","last_peered":"2024-09-20T19:07:12.769761+0000","last_clean":"2024-09-20T19:06:57.380352+0000","last_became_active":"2024-09-20T19:06:59.683397+0000","last_became_peered":"2024-09-20T19:06:59.683397+0000","last_unstale":"2024-09-20T19:07:12.769761+0000","last_undegraded":"2024-09-20T19:07:12.769761+0000","last_fullsized":"2024-09-20T19:06:59.478033+0000","mapping_epoch":444,"log_start":"0'0","ondisk_log_start":"0'0","created":441,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":4116,"log_dups_size":0,"ondisk_log_size":4116,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":148373504,"num_objects":2264,"num_object_clones":0,"num_object_copies":6792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1044,"num_objects_unfound":0,"num_objects_dirty":2264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2264,"num_write_kb":144896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":865,"num_bytes_recovered":56688640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[0,2],"avail_no_missing":["0","2","3"],"object_location_counts":[{"shards":"0,2","objects":1220},{"shards":"0,2,3","objects":1044}],"blocked_by":[],"up_primary":7,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":835,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.962800+0000","last_change":"2024-09-20T19:06:17.937954+0000","last_active":"2024-09-20T19:07:03.962800+0000","last_peered":"2024-09-20T19:07:03.962800+0000","last_clean":"2024-09-20T19:07:03.962800+0000","last_became_active":"2024-09-20T18:59:05.084496+0000","last_became_peered":"2024-09-20T18:59:05.084496+0000","last_unstale":"2024-09-20T19:07:03.962800+0000","last_undegraded":"2024-09-20T19:07:03.962800+0000","last_fullsized":"2024-09-20T19:07:03.962800+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.937896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.937896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:37.574460+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"59'1","reported_seq":160,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.942345+0000","last_change":"2024-09-20T19:04:20.905233+0000","last_active":"2024-09-20T19:07:03.942345+0000","last_peered":"2024-09-20T19:07:03.942345+0000","last_clean":"2024-09-20T19:07:03.942345+0000","last_became_active":"2024-09-20T19:04:20.887062+0000","last_became_peered":"2024-09-20T19:04:20.887062+0000","last_unstale":"2024-09-20T19:07:03.942345+0000","last_undegraded":"2024-09-20T19:07:03.942345+0000","last_fullsized":"2024-09-20T19:07:03.942345+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:03:08.218812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:03:08.218812+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":993,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"274'18","reported_seq":876,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.963302+0000","last_change":"2024-09-20T19:04:20.925690+0000","last_active":"2024-09-20T19:07:03.963302+0000","last_peered":"2024-09-20T19:07:03.963302+0000","last_clean":"2024-09-20T19:07:03.963302+0000","last_became_active":"2024-09-20T19:04:20.896343+0000","last_became_peered":"2024-09-20T19:04:20.896343+0000","last_unstale":"2024-09-20T19:07:03.963302+0000","last_undegraded":"2024-09-20T19:07:03.963302+0000","last_fullsized":"2024-09-20T19:07:03.963302+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:57:09.818999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:09.818999+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:29.919360+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":164,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.942847+0000","last_change":"2024-09-20T19:06:16.148081+0000","last_active":"2024-09-20T19:07:03.942847+0000","last_peered":"2024-09-20T19:07:03.942847+0000","last_clean":"2024-09-20T19:07:03.942847+0000","last_became_active":"2024-09-20T19:04:20.889430+0000","last_became_peered":"2024-09-20T19:04:20.889430+0000","last_unstale":"2024-09-20T19:07:03.942847+0000","last_undegraded":"2024-09-20T19:07:03.942847+0000","last_fullsized":"2024-09-20T19:07:03.942847+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:16.148022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:06:16.148022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:27.803843+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.11","version":"447'4233","reported_seq":4306,"reported_epoch":447,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-20T19:07:12.764650+0000","last_change":"2024-09-20T19:06:59.825120+0000","last_active":"2024-09-20T19:07:12.764650+0000","last_peered":"2024-09-20T19:07:12.764650+0000","last_clean":"2024-09-20T19:06:57.417320+0000","last_became_active":"2024-09-20T19:06:59.817960+0000","last_became_peered":"2024-09-20T19:06:59.817960+0000","last_unstale":"2024-09-20T19:07:12.764650+0000","last_undegraded":"2024-09-20T19:06:59.745995+0000","last_fullsized":"2024-09-20T19:06:59.454366+0000","mapping_epoch":444,"log_start":"0'0","ondisk_log_start":"0'0","created":441,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":4233,"log_dups_size":0,"ondisk_log_size":4233,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":151715840,"num_objects":2315,"num_object_clones":0,"num_object_copies":6945,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1950,"num_objects_misplaced":1950,"num_objects_unfound":0,"num_objects_dirty":2315,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2315,"num_write_kb":148160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[0,3],"avail_no_missing":["0","3","6"],"object_location_counts":[{"shards":"0,3","objects":365},{"shards":"0,3,6","objects":1950}],"blocked_by":[],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":813,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.961860+0000","last_change":"2024-09-20T19:02:28.294390+0000","last_active":"2024-09-20T19:07:03.961860+0000","last_peered":"2024-09-20T19:07:03.961860+0000","last_clean":"2024-09-20T19:07:03.961860+0000","last_became_active":"2024-09-20T18:49:51.970058+0000","last_became_peered":"2024-09-20T18:49:51.970058+0000","last_unstale":"2024-09-20T19:07:03.961860+0000","last_undegraded":"2024-09-20T19:07:03.961860+0000","last_fullsized":"2024-09-20T19:07:03.961860+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T19:02:28.294292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:28.294292+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":858,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.977651+0000","last_change":"2024-09-20T19:06:53.021433+0000","last_active":"2024-09-20T19:07:03.977651+0000","last_peered":"2024-09-20T19:07:03.977651+0000","last_clean":"2024-09-20T19:07:03.977651+0000","last_became_active":"2024-09-20T19:04:20.881466+0000","last_became_peered":"2024-09-20T19:04:20.881466+0000","last_unstale":"2024-09-20T19:07:03.977651+0000","last_undegraded":"2024-09-20T19:07:03.977651+0000","last_fullsized":"2024-09-20T19:07:03.977651+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:06:53.021363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:06:53.021363+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:08:12.405490+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"274'20","reported_seq":895,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.973943+0000","last_change":"2024-09-20T19:06:42.777378+0000","last_active":"2024-09-20T19:07:03.973943+0000","last_peered":"2024-09-20T19:07:03.973943+0000","last_clean":"2024-09-20T19:07:03.973943+0000","last_became_active":"2024-09-20T19:03:16.968636+0000","last_became_peered":"2024-09-20T19:03:16.968636+0000","last_unstale":"2024-09-20T19:07:03.973943+0000","last_undegraded":"2024-09-20T19:07:03.973943+0000","last_fullsized":"2024-09-20T19:07:03.973943+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'20","last_scrub_stamp":"2024-09-20T19:06:42.777179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:06:42.777179+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:08:01.213453+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"375'14","reported_seq":540,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:06.204553+0000","last_change":"2024-09-20T19:04:20.956210+0000","last_active":"2024-09-20T19:07:06.204553+0000","last_peered":"2024-09-20T19:07:06.204553+0000","last_clean":"2024-09-20T19:07:06.204553+0000","last_became_active":"2024-09-20T19:04:20.900782+0000","last_became_peered":"2024-09-20T19:04:20.900782+0000","last_unstale":"2024-09-20T19:07:06.204553+0000","last_undegraded":"2024-09-20T19:07:06.204553+0000","last_fullsized":"2024-09-20T19:07:06.204553+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"308'12","last_scrub_stamp":"2024-09-20T19:01:44.541156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:44.541156+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:39.843812+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.0","version":"447'4170","reported_seq":4269,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:12.092832+0000","last_change":"2024-09-20T19:06:58.386274+0000","last_active":"2024-09-20T19:07:12.092832+0000","last_peered":"2024-09-20T19:07:12.092832+0000","last_clean":"2024-09-20T19:07:12.092832+0000","last_became_active":"2024-09-20T19:06:56.482140+0000","last_became_peered":"2024-09-20T19:06:56.482140+0000","last_unstale":"2024-09-20T19:07:12.092832+0000","last_undegraded":"2024-09-20T19:07:12.092832+0000","last_fullsized":"2024-09-20T19:07:12.092832+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"443'3875","last_scrub_stamp":"2024-09-20T19:06:58.386214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:58.386214+0000","objects_scrubbed":2003,"log_size":4170,"log_dups_size":0,"ondisk_log_size":4170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:08:21.450248+0000","scrub_duration":191,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":150601728,"num_objects":2298,"num_object_clones":0,"num_object_copies":6894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":295,"num_write_kb":18880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"274'27","reported_seq":906,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.962238+0000","last_change":"2024-09-20T19:04:20.920811+0000","last_active":"2024-09-20T19:07:03.962238+0000","last_peered":"2024-09-20T19:07:03.962238+0000","last_clean":"2024-09-20T19:07:03.962238+0000","last_became_active":"2024-09-20T19:04:20.897049+0000","last_became_peered":"2024-09-20T19:04:20.897049+0000","last_unstale":"2024-09-20T19:07:03.962238+0000","last_undegraded":"2024-09-20T19:07:03.962238+0000","last_fullsized":"2024-09-20T19:07:03.962238+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'27","last_scrub_stamp":"2024-09-20T19:02:27.897138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:27.897138+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:33.671747+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":833,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.980715+0000","last_change":"2024-09-20T19:05:55.742165+0000","last_active":"2024-09-20T19:07:03.980715+0000","last_peered":"2024-09-20T19:07:03.980715+0000","last_clean":"2024-09-20T19:07:03.980715+0000","last_became_active":"2024-09-20T18:49:45.936104+0000","last_became_peered":"2024-09-20T18:49:45.936104+0000","last_unstale":"2024-09-20T19:07:03.980715+0000","last_undegraded":"2024-09-20T19:07:03.980715+0000","last_fullsized":"2024-09-20T19:07:03.980715+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:55.742106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:55.742106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:16.985762+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"375'12","reported_seq":202,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.943004+0000","last_change":"2024-09-20T19:06:20.174206+0000","last_active":"2024-09-20T19:07:03.943004+0000","last_peered":"2024-09-20T19:07:03.943004+0000","last_clean":"2024-09-20T19:07:03.943004+0000","last_became_active":"2024-09-20T19:04:20.907228+0000","last_became_peered":"2024-09-20T19:04:20.907228+0000","last_unstale":"2024-09-20T19:07:03.943004+0000","last_undegraded":"2024-09-20T19:07:03.943004+0000","last_fullsized":"2024-09-20T19:07:03.943004+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"375'12","last_scrub_stamp":"2024-09-20T19:06:20.174160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:06:20.174160+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:49.543973+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":812,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.970891+0000","last_change":"2024-09-20T19:02:22.736879+0000","last_active":"2024-09-20T19:07:03.970891+0000","last_peered":"2024-09-20T19:07:03.970891+0000","last_clean":"2024-09-20T19:07:03.970891+0000","last_became_active":"2024-09-20T18:49:51.971399+0000","last_became_peered":"2024-09-20T18:49:51.971399+0000","last_unstale":"2024-09-20T19:07:03.970891+0000","last_undegraded":"2024-09-20T19:07:03.970891+0000","last_fullsized":"2024-09-20T19:07:03.970891+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:22.736825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:22.736825+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.f","version":"447'4536","reported_seq":4600,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:11.358324+0000","last_change":"2024-09-20T19:06:18.056190+0000","last_active":"2024-09-20T19:07:11.358324+0000","last_peered":"2024-09-20T19:07:11.358324+0000","last_clean":"2024-09-20T19:07:11.358324+0000","last_became_active":"2024-09-20T19:06:18.055847+0000","last_became_peered":"2024-09-20T19:06:18.055847+0000","last_unstale":"2024-09-20T19:07:11.358324+0000","last_undegraded":"2024-09-20T19:07:11.358324+0000","last_fullsized":"2024-09-20T19:07:11.358324+0000","mapping_epoch":422,"log_start":"426'200","ondisk_log_start":"426'200","created":422,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":4336,"log_dups_size":0,"ondisk_log_size":4336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:43.947068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":297271296,"num_objects":4536,"num_object_clones":0,"num_object_copies":13608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4536,"num_write_kb":290304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"273'30","reported_seq":895,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.989076+0000","last_change":"2024-09-20T19:06:54.927673+0000","last_active":"2024-09-20T19:07:03.989076+0000","last_peered":"2024-09-20T19:07:03.989076+0000","last_clean":"2024-09-20T19:07:03.989076+0000","last_became_active":"2024-09-20T18:49:48.499189+0000","last_became_peered":"2024-09-20T18:49:48.499189+0000","last_unstale":"2024-09-20T19:07:03.989076+0000","last_undegraded":"2024-09-20T19:07:03.989076+0000","last_fullsized":"2024-09-20T19:07:03.989076+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'30","last_scrub_stamp":"2024-09-20T19:06:54.927614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:06:54.927614+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:08:10.287625+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":825,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.963178+0000","last_change":"2024-09-20T19:06:16.897724+0000","last_active":"2024-09-20T19:07:03.963178+0000","last_peered":"2024-09-20T19:07:03.963178+0000","last_clean":"2024-09-20T19:07:03.963178+0000","last_became_active":"2024-09-20T18:49:45.940544+0000","last_became_peered":"2024-09-20T18:49:45.940544+0000","last_unstale":"2024-09-20T19:07:03.963178+0000","last_undegraded":"2024-09-20T19:07:03.963178+0000","last_fullsized":"2024-09-20T19:07:03.963178+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:16.897677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:06:16.897677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:27.308342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":156,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.942215+0000","last_change":"2024-09-20T19:04:20.889280+0000","last_active":"2024-09-20T19:07:03.942215+0000","last_peered":"2024-09-20T19:07:03.942215+0000","last_clean":"2024-09-20T19:07:03.942215+0000","last_became_active":"2024-09-20T19:04:20.888422+0000","last_became_peered":"2024-09-20T19:04:20.888422+0000","last_unstale":"2024-09-20T19:07:03.942215+0000","last_undegraded":"2024-09-20T19:07:03.942215+0000","last_fullsized":"2024-09-20T19:07:03.942215+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.229201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.229201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:33.311119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":164,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.942721+0000","last_change":"2024-09-20T19:05:47.121946+0000","last_active":"2024-09-20T19:07:03.942721+0000","last_peered":"2024-09-20T19:07:03.942721+0000","last_clean":"2024-09-20T19:07:03.942721+0000","last_became_active":"2024-09-20T19:04:20.889418+0000","last_became_peered":"2024-09-20T19:04:20.889418+0000","last_unstale":"2024-09-20T19:07:03.942721+0000","last_undegraded":"2024-09-20T19:07:03.942721+0000","last_fullsized":"2024-09-20T19:07:03.942721+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:47.121842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:05:47.121842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:15.049722+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"447'4466","reported_seq":4530,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:10.726380+0000","last_change":"2024-09-20T19:06:18.054020+0000","last_active":"2024-09-20T19:07:10.726380+0000","last_peered":"2024-09-20T19:07:10.726380+0000","last_clean":"2024-09-20T19:07:10.726380+0000","last_became_active":"2024-09-20T19:06:18.053343+0000","last_became_peered":"2024-09-20T19:06:18.053343+0000","last_unstale":"2024-09-20T19:07:10.726380+0000","last_undegraded":"2024-09-20T19:07:10.726380+0000","last_fullsized":"2024-09-20T19:07:10.726380+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":4466,"log_dups_size":0,"ondisk_log_size":4466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:38.809608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":292683776,"num_objects":4466,"num_object_clones":0,"num_object_copies":13398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4466,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4466,"num_write_kb":285824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"274'28","reported_seq":172,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.942188+0000","last_change":"2024-09-20T19:04:20.976682+0000","last_active":"2024-09-20T19:07:03.942188+0000","last_peered":"2024-09-20T19:07:03.942188+0000","last_clean":"2024-09-20T19:07:03.942188+0000","last_became_active":"2024-09-20T19:04:20.897444+0000","last_became_peered":"2024-09-20T19:04:20.897444+0000","last_unstale":"2024-09-20T19:07:03.942188+0000","last_undegraded":"2024-09-20T19:07:03.942188+0000","last_fullsized":"2024-09-20T19:07:03.942188+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'28","last_scrub_stamp":"2024-09-20T19:03:06.269623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:06.269623+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":41,"num_read_kb":34,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":860,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.974284+0000","last_change":"2024-09-20T19:05:47.629956+0000","last_active":"2024-09-20T19:07:03.974284+0000","last_peered":"2024-09-20T19:07:03.974284+0000","last_clean":"2024-09-20T19:07:03.974284+0000","last_became_active":"2024-09-20T19:04:20.906790+0000","last_became_peered":"2024-09-20T19:04:20.906790+0000","last_unstale":"2024-09-20T19:07:03.974284+0000","last_undegraded":"2024-09-20T19:07:03.974284+0000","last_fullsized":"2024-09-20T19:07:03.974284+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:05:47.629892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:47.629892+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"375'13","reported_seq":1081,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.974738+0000","last_change":"2024-09-20T19:04:20.916872+0000","last_active":"2024-09-20T19:07:03.974738+0000","last_peered":"2024-09-20T19:07:03.974738+0000","last_clean":"2024-09-20T19:07:03.974738+0000","last_became_active":"2024-09-20T19:04:20.896424+0000","last_became_peered":"2024-09-20T19:04:20.896424+0000","last_unstale":"2024-09-20T19:07:03.974738+0000","last_undegraded":"2024-09-20T19:07:03.974738+0000","last_fullsized":"2024-09-20T19:07:03.974738+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:58:23.114371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:23.114371+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:39.902774+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":820,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.961226+0000","last_change":"2024-09-20T18:59:05.082117+0000","last_active":"2024-09-20T19:07:03.961226+0000","last_peered":"2024-09-20T19:07:03.961226+0000","last_clean":"2024-09-20T19:07:03.961226+0000","last_became_active":"2024-09-20T18:59:05.081814+0000","last_became_peered":"2024-09-20T18:59:05.081814+0000","last_unstale":"2024-09-20T19:07:03.961226+0000","last_undegraded":"2024-09-20T19:07:03.961226+0000","last_fullsized":"2024-09-20T19:07:03.961226+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:29.530001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:29.530001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:38.835871+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.d","version":"447'4460","reported_seq":4524,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:11.686694+0000","last_change":"2024-09-20T19:06:18.053626+0000","last_active":"2024-09-20T19:07:11.686694+0000","last_peered":"2024-09-20T19:07:11.686694+0000","last_clean":"2024-09-20T19:07:11.686694+0000","last_became_active":"2024-09-20T19:06:18.053336+0000","last_became_peered":"2024-09-20T19:06:18.053336+0000","last_unstale":"2024-09-20T19:07:11.686694+0000","last_undegraded":"2024-09-20T19:07:11.686694+0000","last_fullsized":"2024-09-20T19:07:11.686694+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":4460,"log_dups_size":0,"ondisk_log_size":4460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:29.122276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":292290560,"num_objects":4460,"num_object_clones":0,"num_object_copies":13380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4460,"num_write_kb":285440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"274'20","reported_seq":671,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.964336+0000","last_change":"2024-09-20T19:05:48.741237+0000","last_active":"2024-09-20T19:07:03.964336+0000","last_peered":"2024-09-20T19:07:03.964336+0000","last_clean":"2024-09-20T19:07:03.964336+0000","last_became_active":"2024-09-20T19:04:49.419140+0000","last_became_peered":"2024-09-20T19:04:49.419140+0000","last_unstale":"2024-09-20T19:07:03.964336+0000","last_undegraded":"2024-09-20T19:07:03.964336+0000","last_fullsized":"2024-09-20T19:07:03.964336+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'20","last_scrub_stamp":"2024-09-20T19:05:48.741059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:48.741059+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:13.202074+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":849,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.971375+0000","last_change":"2024-09-20T19:05:56.820141+0000","last_active":"2024-09-20T19:07:03.971375+0000","last_peered":"2024-09-20T19:07:03.971375+0000","last_clean":"2024-09-20T19:07:03.971375+0000","last_became_active":"2024-09-20T18:49:45.942496+0000","last_became_peered":"2024-09-20T18:49:45.942496+0000","last_unstale":"2024-09-20T19:07:03.971375+0000","last_undegraded":"2024-09-20T19:07:03.971375+0000","last_fullsized":"2024-09-20T19:07:03.971375+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:56.820079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:56.820079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:16.442542+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"262'10","reported_seq":1058,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:06.204385+0000","last_change":"2024-09-20T19:02:04.994109+0000","last_active":"2024-09-20T19:07:06.204385+0000","last_peered":"2024-09-20T19:07:06.204385+0000","last_clean":"2024-09-20T19:07:06.204385+0000","last_became_active":"2024-09-20T18:59:05.080183+0000","last_became_peered":"2024-09-20T18:59:05.080183+0000","last_unstale":"2024-09-20T19:07:06.204385+0000","last_undegraded":"2024-09-20T19:07:06.204385+0000","last_fullsized":"2024-09-20T19:07:06.204385+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'10","last_scrub_stamp":"2024-09-20T19:02:04.994047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:04.994047+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:32.731999+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":806,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.965939+0000","last_change":"2024-09-20T19:06:15.671414+0000","last_active":"2024-09-20T19:07:03.965939+0000","last_peered":"2024-09-20T19:07:03.965939+0000","last_clean":"2024-09-20T19:07:03.965939+0000","last_became_active":"2024-09-20T18:49:51.964780+0000","last_became_peered":"2024-09-20T18:49:51.964780+0000","last_unstale":"2024-09-20T19:07:03.965939+0000","last_undegraded":"2024-09-20T19:07:03.965939+0000","last_fullsized":"2024-09-20T19:07:03.965939+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:15.671352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:06:15.671352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:25.558790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"447'4494","reported_seq":4558,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:12.140594+0000","last_change":"2024-09-20T19:06:18.050185+0000","last_active":"2024-09-20T19:07:12.140594+0000","last_peered":"2024-09-20T19:07:12.140594+0000","last_clean":"2024-09-20T19:07:12.140594+0000","last_became_active":"2024-09-20T19:06:18.049949+0000","last_became_peered":"2024-09-20T19:06:18.049949+0000","last_unstale":"2024-09-20T19:07:12.140594+0000","last_undegraded":"2024-09-20T19:07:12.140594+0000","last_fullsized":"2024-09-20T19:07:12.140594+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":4494,"log_dups_size":0,"ondisk_log_size":4494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:45.048224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":294518784,"num_objects":4494,"num_object_clones":0,"num_object_copies":13482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4494,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4494,"num_write_kb":287616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,4],"acting":[1,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"274'29","reported_seq":873,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.965807+0000","last_change":"2024-09-20T19:05:43.631990+0000","last_active":"2024-09-20T19:07:03.965807+0000","last_peered":"2024-09-20T19:07:03.965807+0000","last_clean":"2024-09-20T19:07:03.965807+0000","last_became_active":"2024-09-20T18:49:48.497736+0000","last_became_peered":"2024-09-20T18:49:48.497736+0000","last_unstale":"2024-09-20T19:07:03.965807+0000","last_undegraded":"2024-09-20T19:07:03.965807+0000","last_fullsized":"2024-09-20T19:07:03.965807+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'29","last_scrub_stamp":"2024-09-20T19:05:43.631845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:43.631845+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":825,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.979657+0000","last_change":"2024-09-20T19:05:47.837587+0000","last_active":"2024-09-20T19:07:03.979657+0000","last_peered":"2024-09-20T19:07:03.979657+0000","last_clean":"2024-09-20T19:07:03.979657+0000","last_became_active":"2024-09-20T18:49:45.942253+0000","last_became_peered":"2024-09-20T18:49:45.942253+0000","last_unstale":"2024-09-20T19:07:03.979657+0000","last_undegraded":"2024-09-20T19:07:03.979657+0000","last_fullsized":"2024-09-20T19:07:03.979657+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:47.837520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:47.837520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:15.032955+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":411,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.964844+0000","last_change":"2024-09-20T19:05:48.203244+0000","last_active":"2024-09-20T19:07:03.964844+0000","last_peered":"2024-09-20T19:07:03.964844+0000","last_clean":"2024-09-20T19:07:03.964844+0000","last_became_active":"2024-09-20T18:59:05.085636+0000","last_became_peered":"2024-09-20T18:59:05.085636+0000","last_unstale":"2024-09-20T19:07:03.964844+0000","last_undegraded":"2024-09-20T19:07:03.964844+0000","last_fullsized":"2024-09-20T19:07:03.964844+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:48.203182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:05:48.203182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":843,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.977445+0000","last_change":"2024-09-20T19:06:43.932713+0000","last_active":"2024-09-20T19:07:03.977445+0000","last_peered":"2024-09-20T19:07:03.977445+0000","last_clean":"2024-09-20T19:07:03.977445+0000","last_became_active":"2024-09-20T18:59:05.085131+0000","last_became_peered":"2024-09-20T18:59:05.085131+0000","last_unstale":"2024-09-20T19:07:03.977445+0000","last_undegraded":"2024-09-20T19:07:03.977445+0000","last_fullsized":"2024-09-20T19:07:03.977445+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:43.932649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:06:43.932649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:46.429239+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.b","version":"447'4297","reported_seq":4361,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:10.807556+0000","last_change":"2024-09-20T19:06:18.057411+0000","last_active":"2024-09-20T19:07:10.807556+0000","last_peered":"2024-09-20T19:07:10.807556+0000","last_clean":"2024-09-20T19:07:10.807556+0000","last_became_active":"2024-09-20T19:06:18.057129+0000","last_became_peered":"2024-09-20T19:06:18.057129+0000","last_unstale":"2024-09-20T19:07:10.807556+0000","last_undegraded":"2024-09-20T19:07:10.807556+0000","last_fullsized":"2024-09-20T19:07:10.807556+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":4297,"log_dups_size":0,"ondisk_log_size":4297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:37.528784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":281608192,"num_objects":4297,"num_object_clones":0,"num_object_copies":12891,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4297,"num_write_kb":275008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"353'41","reported_seq":1142,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:06.205013+0000","last_change":"2024-09-20T19:05:48.855983+0000","last_active":"2024-09-20T19:07:06.205013+0000","last_peered":"2024-09-20T19:07:06.205013+0000","last_clean":"2024-09-20T19:07:06.205013+0000","last_became_active":"2024-09-20T19:03:16.976916+0000","last_became_peered":"2024-09-20T19:03:16.976916+0000","last_unstale":"2024-09-20T19:07:06.205013+0000","last_undegraded":"2024-09-20T19:07:06.205013+0000","last_fullsized":"2024-09-20T19:07:06.205013+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"353'41","last_scrub_stamp":"2024-09-20T19:05:48.855876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:48.855876+0000","objects_scrubbed":10,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":164,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.942535+0000","last_change":"2024-09-20T19:06:15.168347+0000","last_active":"2024-09-20T19:07:03.942535+0000","last_peered":"2024-09-20T19:07:03.942535+0000","last_clean":"2024-09-20T19:07:03.942535+0000","last_became_active":"2024-09-20T19:04:20.899352+0000","last_became_peered":"2024-09-20T19:04:20.899352+0000","last_unstale":"2024-09-20T19:07:03.942535+0000","last_undegraded":"2024-09-20T19:07:03.942535+0000","last_fullsized":"2024-09-20T19:07:03.942535+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:15.168293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:06:15.168293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:21.325884+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":881,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.964489+0000","last_change":"2024-09-20T19:03:11.910684+0000","last_active":"2024-09-20T19:07:03.964489+0000","last_peered":"2024-09-20T19:07:03.964489+0000","last_clean":"2024-09-20T19:07:03.964489+0000","last_became_active":"2024-09-20T18:59:05.086786+0000","last_became_peered":"2024-09-20T18:59:05.086786+0000","last_unstale":"2024-09-20T19:07:03.964489+0000","last_undegraded":"2024-09-20T19:07:03.964489+0000","last_fullsized":"2024-09-20T19:07:03.964489+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-20T19:03:11.909394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:03:11.909394+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:12.871941+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0.0012781159999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":859,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.965512+0000","last_change":"2024-09-20T19:06:55.997516+0000","last_active":"2024-09-20T19:07:03.965512+0000","last_peered":"2024-09-20T19:07:03.965512+0000","last_clean":"2024-09-20T19:07:03.965512+0000","last_became_active":"2024-09-20T19:04:20.907038+0000","last_became_peered":"2024-09-20T19:04:20.907038+0000","last_unstale":"2024-09-20T19:07:03.965512+0000","last_undegraded":"2024-09-20T19:07:03.965512+0000","last_fullsized":"2024-09-20T19:07:03.965512+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:55.997469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:06:55.997469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:08:16.409939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":822,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.963587+0000","last_change":"2024-09-20T19:04:20.896616+0000","last_active":"2024-09-20T19:07:03.963587+0000","last_peered":"2024-09-20T19:07:03.963587+0000","last_clean":"2024-09-20T19:07:03.963587+0000","last_became_active":"2024-09-20T19:04:20.896281+0000","last_became_peered":"2024-09-20T19:04:20.896281+0000","last_unstale":"2024-09-20T19:07:03.963587+0000","last_undegraded":"2024-09-20T19:07:03.963587+0000","last_fullsized":"2024-09-20T19:07:03.963587+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:04.526683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:04.526683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:12.801724+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"447'3997","reported_seq":4095,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:10.811265+0000","last_change":"2024-09-20T19:06:58.108560+0000","last_active":"2024-09-20T19:07:10.811265+0000","last_peered":"2024-09-20T19:07:10.811265+0000","last_clean":"2024-09-20T19:07:10.811265+0000","last_became_active":"2024-09-20T19:06:56.463634+0000","last_became_peered":"2024-09-20T19:06:56.463634+0000","last_unstale":"2024-09-20T19:07:10.811265+0000","last_undegraded":"2024-09-20T19:07:10.811265+0000","last_fullsized":"2024-09-20T19:07:10.811265+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"443'3725","last_scrub_stamp":"2024-09-20T19:06:58.108433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:58.108433+0000","objects_scrubbed":1931,"log_size":3997,"log_dups_size":0,"ondisk_log_size":3997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:08:01.384707+0000","scrub_duration":242,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":144375808,"num_objects":2203,"num_object_clones":0,"num_object_copies":6609,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":272,"num_write_kb":17408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"273'17","reported_seq":870,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.984897+0000","last_change":"2024-09-20T19:05:43.974098+0000","last_active":"2024-09-20T19:07:03.984897+0000","last_peered":"2024-09-20T19:07:03.984897+0000","last_clean":"2024-09-20T19:07:03.984897+0000","last_became_active":"2024-09-20T18:49:47.950904+0000","last_became_peered":"2024-09-20T18:49:47.950904+0000","last_unstale":"2024-09-20T19:07:03.984897+0000","last_undegraded":"2024-09-20T19:07:03.984897+0000","last_fullsized":"2024-09-20T19:07:03.984897+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'17","last_scrub_stamp":"2024-09-20T19:05:43.974013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:43.974013+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":36,"num_read_kb":25,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":847,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.979916+0000","last_change":"2024-09-20T19:02:29.840761+0000","last_active":"2024-09-20T19:07:03.979916+0000","last_peered":"2024-09-20T19:07:03.979916+0000","last_clean":"2024-09-20T19:07:03.979916+0000","last_became_active":"2024-09-20T18:59:05.079789+0000","last_became_peered":"2024-09-20T18:59:05.079789+0000","last_unstale":"2024-09-20T19:07:03.979916+0000","last_undegraded":"2024-09-20T19:07:03.979916+0000","last_fullsized":"2024-09-20T19:07:03.979916+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:29.840601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:29.840601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:35.837533+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":837,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.964147+0000","last_change":"2024-09-20T19:06:13.599431+0000","last_active":"2024-09-20T19:07:03.964147+0000","last_peered":"2024-09-20T19:07:03.964147+0000","last_clean":"2024-09-20T19:07:03.964147+0000","last_became_active":"2024-09-20T18:49:44.937135+0000","last_became_peered":"2024-09-20T18:49:44.937135+0000","last_unstale":"2024-09-20T19:07:03.964147+0000","last_undegraded":"2024-09-20T19:07:03.964147+0000","last_fullsized":"2024-09-20T19:07:03.964147+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:06:13.599376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:06:13.599376+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:27.051567+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00095558800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":804,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.964215+0000","last_change":"2024-09-20T19:01:17.747452+0000","last_active":"2024-09-20T19:07:03.964215+0000","last_peered":"2024-09-20T19:07:03.964215+0000","last_clean":"2024-09-20T19:07:03.964215+0000","last_became_active":"2024-09-20T18:49:49.954424+0000","last_became_peered":"2024-09-20T18:49:49.954424+0000","last_unstale":"2024-09-20T19:07:03.964215+0000","last_undegraded":"2024-09-20T19:07:03.964215+0000","last_fullsized":"2024-09-20T19:07:03.964215+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:17.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:17.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":815,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.961901+0000","last_change":"2024-09-20T19:01:15.203591+0000","last_active":"2024-09-20T19:07:03.961901+0000","last_peered":"2024-09-20T19:07:03.961901+0000","last_clean":"2024-09-20T19:07:03.961901+0000","last_became_active":"2024-09-20T18:59:05.081459+0000","last_became_peered":"2024-09-20T18:59:05.081459+0000","last_unstale":"2024-09-20T19:07:03.961901+0000","last_undegraded":"2024-09-20T19:07:03.961901+0000","last_fullsized":"2024-09-20T19:07:03.961901+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:15.203529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:15.203529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:41.883469+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"447'4157","reported_seq":4256,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:10.810418+0000","last_change":"2024-09-20T19:06:57.306900+0000","last_active":"2024-09-20T19:07:10.810418+0000","last_peered":"2024-09-20T19:07:10.810418+0000","last_clean":"2024-09-20T19:07:10.810418+0000","last_became_active":"2024-09-20T19:06:56.466304+0000","last_became_peered":"2024-09-20T19:06:56.466304+0000","last_unstale":"2024-09-20T19:07:10.810418+0000","last_undegraded":"2024-09-20T19:07:10.810418+0000","last_fullsized":"2024-09-20T19:07:10.810418+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"442'3861","last_scrub_stamp":"2024-09-20T19:06:57.306685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:57.306685+0000","objects_scrubbed":1958,"log_size":4157,"log_dups_size":0,"ondisk_log_size":4157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:08:12.322706+0000","scrub_duration":402,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":148373504,"num_objects":2264,"num_object_clones":0,"num_object_copies":6792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":306,"num_write_kb":19584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"273'24","reported_seq":866,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.974091+0000","last_change":"2024-09-20T18:59:06.022021+0000","last_active":"2024-09-20T19:07:03.974091+0000","last_peered":"2024-09-20T19:07:03.974091+0000","last_clean":"2024-09-20T19:07:03.974091+0000","last_became_active":"2024-09-20T18:49:47.949230+0000","last_became_peered":"2024-09-20T18:49:47.949230+0000","last_unstale":"2024-09-20T19:07:03.974091+0000","last_undegraded":"2024-09-20T19:07:03.974091+0000","last_fullsized":"2024-09-20T19:07:03.974091+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:59:06.021962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:59:06.021962+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:35.720209+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":851,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.977314+0000","last_change":"2024-09-20T19:04:20.907250+0000","last_active":"2024-09-20T19:07:03.977314+0000","last_peered":"2024-09-20T19:07:03.977314+0000","last_clean":"2024-09-20T19:07:03.977314+0000","last_became_active":"2024-09-20T19:04:20.906969+0000","last_became_peered":"2024-09-20T19:04:20.906969+0000","last_unstale":"2024-09-20T19:07:03.977314+0000","last_undegraded":"2024-09-20T19:07:03.977314+0000","last_fullsized":"2024-09-20T19:07:03.977314+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:24.984022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:24.984022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:30.762279+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":1883,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:09.773416+0000","last_change":"2024-09-20T19:06:43.885193+0000","last_active":"2024-09-20T19:07:09.773416+0000","last_peered":"2024-09-20T19:07:09.773416+0000","last_clean":"2024-09-20T19:07:09.773416+0000","last_became_active":"2024-09-20T18:49:44.936819+0000","last_became_peered":"2024-09-20T18:49:44.936819+0000","last_unstale":"2024-09-20T19:07:09.773416+0000","last_undegraded":"2024-09-20T19:07:09.773416+0000","last_fullsized":"2024-09-20T19:07:09.773416+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-20T19:06:43.883404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:06:43.883404+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:47.147460+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.0010996199999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1038,"num_read_kb":1033,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":833,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.964300+0000","last_change":"2024-09-20T19:05:51.743485+0000","last_active":"2024-09-20T19:07:03.964300+0000","last_peered":"2024-09-20T19:07:03.964300+0000","last_clean":"2024-09-20T19:07:03.964300+0000","last_became_active":"2024-09-20T18:49:49.961066+0000","last_became_peered":"2024-09-20T18:49:49.961066+0000","last_unstale":"2024-09-20T19:07:03.964300+0000","last_undegraded":"2024-09-20T19:07:03.964300+0000","last_fullsized":"2024-09-20T19:07:03.964300+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:51.743363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:05:51.743363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":821,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.979119+0000","last_change":"2024-09-20T19:02:26.861253+0000","last_active":"2024-09-20T19:07:03.979119+0000","last_peered":"2024-09-20T19:07:03.979119+0000","last_clean":"2024-09-20T19:07:03.979119+0000","last_became_active":"2024-09-20T18:49:51.970343+0000","last_became_peered":"2024-09-20T18:49:51.970343+0000","last_unstale":"2024-09-20T19:07:03.979119+0000","last_undegraded":"2024-09-20T19:07:03.979119+0000","last_fullsized":"2024-09-20T19:07:03.979119+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:26.861185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:26.861185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:32.805240+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.a","version":"447'4435","reported_seq":4497,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:10.848056+0000","last_change":"2024-09-20T19:06:18.055447+0000","last_active":"2024-09-20T19:07:10.848056+0000","last_peered":"2024-09-20T19:07:10.848056+0000","last_clean":"2024-09-20T19:07:10.848056+0000","last_became_active":"2024-09-20T19:06:18.055189+0000","last_became_peered":"2024-09-20T19:06:18.055189+0000","last_unstale":"2024-09-20T19:07:10.848056+0000","last_undegraded":"2024-09-20T19:07:10.848056+0000","last_fullsized":"2024-09-20T19:07:10.848056+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":4435,"log_dups_size":0,"ondisk_log_size":4435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:38.823767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":290652160,"num_objects":4435,"num_object_clones":0,"num_object_copies":13305,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4435,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4435,"num_write_kb":283840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"375'34","reported_seq":600,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:06.204445+0000","last_change":"2024-09-20T19:05:54.115428+0000","last_active":"2024-09-20T19:07:06.204445+0000","last_peered":"2024-09-20T19:07:06.204445+0000","last_clean":"2024-09-20T19:07:06.204445+0000","last_became_active":"2024-09-20T19:04:20.928727+0000","last_became_peered":"2024-09-20T19:04:20.928727+0000","last_unstale":"2024-09-20T19:07:06.204445+0000","last_undegraded":"2024-09-20T19:07:06.204445+0000","last_fullsized":"2024-09-20T19:07:06.204445+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"375'34","last_scrub_stamp":"2024-09-20T19:05:54.115376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:54.115376+0000","objects_scrubbed":8,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":462,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":855,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.958194+0000","last_change":"2024-09-20T19:06:47.914880+0000","last_active":"2024-09-20T19:07:03.958194+0000","last_peered":"2024-09-20T19:07:03.958194+0000","last_clean":"2024-09-20T19:07:03.958194+0000","last_became_active":"2024-09-20T18:59:05.079111+0000","last_became_peered":"2024-09-20T18:59:05.079111+0000","last_unstale":"2024-09-20T19:07:03.958194+0000","last_undegraded":"2024-09-20T19:07:03.958194+0000","last_fullsized":"2024-09-20T19:07:03.958194+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:47.914630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:06:47.914630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:08:09.959497+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":894,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.961177+0000","last_change":"2024-09-20T19:03:14.368338+0000","last_active":"2024-09-20T19:07:03.961177+0000","last_peered":"2024-09-20T19:07:03.961177+0000","last_clean":"2024-09-20T19:07:03.961177+0000","last_became_active":"2024-09-20T18:59:05.086103+0000","last_became_peered":"2024-09-20T18:59:05.086103+0000","last_unstale":"2024-09-20T19:07:03.961177+0000","last_undegraded":"2024-09-20T19:07:03.961177+0000","last_fullsized":"2024-09-20T19:07:03.961177+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:03:14.368279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:03:14.368279+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:09.743853+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":818,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.961955+0000","last_change":"2024-09-20T19:06:21.584094+0000","last_active":"2024-09-20T19:07:03.961955+0000","last_peered":"2024-09-20T19:07:03.961955+0000","last_clean":"2024-09-20T19:07:03.961955+0000","last_became_active":"2024-09-20T18:49:49.956128+0000","last_became_peered":"2024-09-20T18:49:49.956128+0000","last_unstale":"2024-09-20T19:07:03.961955+0000","last_undegraded":"2024-09-20T19:07:03.961955+0000","last_fullsized":"2024-09-20T19:07:03.961955+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:21.584027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:06:21.584027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:29.362759+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":854,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.964143+0000","last_change":"2024-09-20T19:06:43.994634+0000","last_active":"2024-09-20T19:07:03.964143+0000","last_peered":"2024-09-20T19:07:03.964143+0000","last_clean":"2024-09-20T19:07:03.964143+0000","last_became_active":"2024-09-20T19:04:20.899321+0000","last_became_peered":"2024-09-20T19:04:20.899321+0000","last_unstale":"2024-09-20T19:07:03.964143+0000","last_undegraded":"2024-09-20T19:07:03.964143+0000","last_fullsized":"2024-09-20T19:07:03.964143+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:43.994470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:06:43.994470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:45.047924+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1","version":"447'4186","reported_seq":4257,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:11.900919+0000","last_change":"2024-09-20T19:06:56.473065+0000","last_active":"2024-09-20T19:07:11.900919+0000","last_peered":"2024-09-20T19:07:11.900919+0000","last_clean":"2024-09-20T19:07:11.900919+0000","last_became_active":"2024-09-20T19:06:56.471168+0000","last_became_peered":"2024-09-20T19:06:56.471168+0000","last_unstale":"2024-09-20T19:07:11.900919+0000","last_undegraded":"2024-09-20T19:07:11.900919+0000","last_fullsized":"2024-09-20T19:07:11.900919+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":4186,"log_dups_size":0,"ondisk_log_size":4186,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":148635648,"num_objects":2268,"num_object_clones":0,"num_object_copies":6804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2268,"num_write_kb":145152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"274'19","reported_seq":176,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.942906+0000","last_change":"2024-09-20T19:05:50.066533+0000","last_active":"2024-09-20T19:07:03.942906+0000","last_peered":"2024-09-20T19:07:03.942906+0000","last_clean":"2024-09-20T19:07:03.942906+0000","last_became_active":"2024-09-20T19:04:20.897175+0000","last_became_peered":"2024-09-20T19:04:20.897175+0000","last_unstale":"2024-09-20T19:07:03.942906+0000","last_undegraded":"2024-09-20T19:07:03.942906+0000","last_fullsized":"2024-09-20T19:07:03.942906+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'19","last_scrub_stamp":"2024-09-20T19:05:50.066359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:50.066359+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":164,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.942284+0000","last_change":"2024-09-20T19:05:56.027187+0000","last_active":"2024-09-20T19:07:03.942284+0000","last_peered":"2024-09-20T19:07:03.942284+0000","last_clean":"2024-09-20T19:07:03.942284+0000","last_became_active":"2024-09-20T19:04:20.887102+0000","last_became_peered":"2024-09-20T19:04:20.887102+0000","last_unstale":"2024-09-20T19:07:03.942284+0000","last_undegraded":"2024-09-20T19:07:03.942284+0000","last_fullsized":"2024-09-20T19:07:03.942284+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:56.027123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:56.027123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:15.919213+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":863,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.963923+0000","last_change":"2024-09-20T19:04:20.899830+0000","last_active":"2024-09-20T19:07:03.963923+0000","last_peered":"2024-09-20T19:07:03.963923+0000","last_clean":"2024-09-20T19:07:03.963923+0000","last_became_active":"2024-09-20T19:04:20.899331+0000","last_became_peered":"2024-09-20T19:04:20.899331+0000","last_unstale":"2024-09-20T19:07:03.963923+0000","last_undegraded":"2024-09-20T19:07:03.963923+0000","last_fullsized":"2024-09-20T19:07:03.963923+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:38.463404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:38.463404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:30.881046+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":826,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.958127+0000","last_change":"2024-09-20T19:04:35.529672+0000","last_active":"2024-09-20T19:07:03.958127+0000","last_peered":"2024-09-20T19:07:03.958127+0000","last_clean":"2024-09-20T19:07:03.958127+0000","last_became_active":"2024-09-20T18:59:05.079897+0000","last_became_peered":"2024-09-20T18:59:05.079897+0000","last_unstale":"2024-09-20T19:07:03.958127+0000","last_undegraded":"2024-09-20T19:07:03.958127+0000","last_fullsized":"2024-09-20T19:07:03.958127+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:35.529584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:04:35.529584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:31.985657+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2","version":"447'4038","reported_seq":4135,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:11.732371+0000","last_change":"2024-09-20T19:06:58.880519+0000","last_active":"2024-09-20T19:07:11.732371+0000","last_peered":"2024-09-20T19:07:11.732371+0000","last_clean":"2024-09-20T19:07:11.732371+0000","last_became_active":"2024-09-20T19:06:56.454466+0000","last_became_peered":"2024-09-20T19:06:56.454466+0000","last_unstale":"2024-09-20T19:07:11.732371+0000","last_undegraded":"2024-09-20T19:07:11.732371+0000","last_fullsized":"2024-09-20T19:07:11.732371+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"443'3744","last_scrub_stamp":"2024-09-20T19:06:58.880419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:58.880419+0000","objects_scrubbed":1883,"log_size":4038,"log_dups_size":0,"ondisk_log_size":4038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:08:04.683990+0000","scrub_duration":189,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":142671872,"num_objects":2177,"num_object_clones":0,"num_object_copies":6531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":294,"num_write_kb":18816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"274'31","reported_seq":182,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.942913+0000","last_change":"2024-09-20T19:06:57.528138+0000","last_active":"2024-09-20T19:07:03.942913+0000","last_peered":"2024-09-20T19:07:03.942913+0000","last_clean":"2024-09-20T19:07:03.942913+0000","last_became_active":"2024-09-20T19:04:20.883306+0000","last_became_peered":"2024-09-20T19:04:20.883306+0000","last_unstale":"2024-09-20T19:07:03.942913+0000","last_undegraded":"2024-09-20T19:07:03.942913+0000","last_fullsized":"2024-09-20T19:07:03.942913+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'31","last_scrub_stamp":"2024-09-20T19:06:57.528084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:06:57.528084+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:08:09.636293+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":49,"num_read_kb":39,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":420,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.965175+0000","last_change":"2024-09-20T19:04:20.899712+0000","last_active":"2024-09-20T19:07:03.965175+0000","last_peered":"2024-09-20T19:07:03.965175+0000","last_clean":"2024-09-20T19:07:03.965175+0000","last_became_active":"2024-09-20T19:04:20.899415+0000","last_became_peered":"2024-09-20T19:04:20.899415+0000","last_unstale":"2024-09-20T19:07:03.965175+0000","last_undegraded":"2024-09-20T19:07:03.965175+0000","last_fullsized":"2024-09-20T19:07:03.965175+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:42.479489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:42.479489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:41.877949+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":849,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.979151+0000","last_change":"2024-09-20T19:05:46.866958+0000","last_active":"2024-09-20T19:07:03.979151+0000","last_peered":"2024-09-20T19:07:03.979151+0000","last_clean":"2024-09-20T19:07:03.979151+0000","last_became_active":"2024-09-20T19:04:20.891772+0000","last_became_peered":"2024-09-20T19:04:20.891772+0000","last_unstale":"2024-09-20T19:07:03.979151+0000","last_undegraded":"2024-09-20T19:07:03.979151+0000","last_fullsized":"2024-09-20T19:07:03.979151+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:46.866777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:05:46.866777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":848,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.962447+0000","last_change":"2024-09-20T19:06:25.628102+0000","last_active":"2024-09-20T19:07:03.962447+0000","last_peered":"2024-09-20T19:07:03.962447+0000","last_clean":"2024-09-20T19:07:03.962447+0000","last_became_active":"2024-09-20T19:04:20.897569+0000","last_became_peered":"2024-09-20T19:04:20.897569+0000","last_unstale":"2024-09-20T19:07:03.962447+0000","last_undegraded":"2024-09-20T19:07:03.962447+0000","last_fullsized":"2024-09-20T19:07:03.962447+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:25.628041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:06:25.628041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:36.219393+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"447'4213","reported_seq":4287,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:11.899947+0000","last_change":"2024-09-20T19:06:56.492093+0000","last_active":"2024-09-20T19:07:11.899947+0000","last_peered":"2024-09-20T19:07:11.899947+0000","last_clean":"2024-09-20T19:07:11.899947+0000","last_became_active":"2024-09-20T19:06:56.480422+0000","last_became_peered":"2024-09-20T19:06:56.480422+0000","last_unstale":"2024-09-20T19:07:11.899947+0000","last_undegraded":"2024-09-20T19:07:11.899947+0000","last_fullsized":"2024-09-20T19:07:11.899947+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":4213,"log_dups_size":0,"ondisk_log_size":4213,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":150142976,"num_objects":2291,"num_object_clones":0,"num_object_copies":6873,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2291,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2291,"num_write_kb":146624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1],"acting":[6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"273'20","reported_seq":168,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.942695+0000","last_change":"2024-09-20T19:04:20.945143+0000","last_active":"2024-09-20T19:07:03.942695+0000","last_peered":"2024-09-20T19:07:03.942695+0000","last_clean":"2024-09-20T19:07:03.942695+0000","last_became_active":"2024-09-20T19:04:20.896483+0000","last_became_peered":"2024-09-20T19:04:20.896483+0000","last_unstale":"2024-09-20T19:07:03.942695+0000","last_undegraded":"2024-09-20T19:07:03.942695+0000","last_fullsized":"2024-09-20T19:07:03.942695+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'20","last_scrub_stamp":"2024-09-20T19:03:04.282800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:04.282800+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":857,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.962751+0000","last_change":"2024-09-20T19:05:48.678438+0000","last_active":"2024-09-20T19:07:03.962751+0000","last_peered":"2024-09-20T19:07:03.962751+0000","last_clean":"2024-09-20T19:07:03.962751+0000","last_became_active":"2024-09-20T19:04:20.892951+0000","last_became_peered":"2024-09-20T19:04:20.892951+0000","last_unstale":"2024-09-20T19:07:03.962751+0000","last_undegraded":"2024-09-20T19:07:03.962751+0000","last_fullsized":"2024-09-20T19:07:03.962751+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:48.678361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:48.678361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":1053,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:06.353695+0000","last_change":"2024-09-20T19:05:08.406307+0000","last_active":"2024-09-20T19:07:06.353695+0000","last_peered":"2024-09-20T19:07:06.353695+0000","last_clean":"2024-09-20T19:07:06.353695+0000","last_became_active":"2024-09-20T18:49:49.957005+0000","last_became_peered":"2024-09-20T18:49:49.957005+0000","last_unstale":"2024-09-20T19:07:06.353695+0000","last_undegraded":"2024-09-20T19:07:06.353695+0000","last_fullsized":"2024-09-20T19:07:06.353695+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T19:05:08.406231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:05:08.406231+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:32.637035+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":164,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.942910+0000","last_change":"2024-09-20T19:06:56.546180+0000","last_active":"2024-09-20T19:07:03.942910+0000","last_peered":"2024-09-20T19:07:03.942910+0000","last_clean":"2024-09-20T19:07:03.942910+0000","last_became_active":"2024-09-20T19:04:20.884994+0000","last_became_peered":"2024-09-20T19:04:20.884994+0000","last_unstale":"2024-09-20T19:07:03.942910+0000","last_undegraded":"2024-09-20T19:07:03.942910+0000","last_fullsized":"2024-09-20T19:07:03.942910+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:56.546114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:06:56.546114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:08:12.373966+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.4","version":"447'4134","reported_seq":4203,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:11.351341+0000","last_change":"2024-09-20T19:06:56.464683+0000","last_active":"2024-09-20T19:07:11.351341+0000","last_peered":"2024-09-20T19:07:11.351341+0000","last_clean":"2024-09-20T19:07:11.351341+0000","last_became_active":"2024-09-20T19:06:56.462890+0000","last_became_peered":"2024-09-20T19:06:56.462890+0000","last_unstale":"2024-09-20T19:07:11.351341+0000","last_undegraded":"2024-09-20T19:07:11.351341+0000","last_fullsized":"2024-09-20T19:07:11.351341+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":4134,"log_dups_size":0,"ondisk_log_size":4134,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":146931712,"num_objects":2242,"num_object_clones":0,"num_object_copies":6726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2242,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2242,"num_write_kb":143488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"274'15","reported_seq":899,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.965718+0000","last_change":"2024-09-20T19:05:57.548758+0000","last_active":"2024-09-20T19:07:03.965718+0000","last_peered":"2024-09-20T19:07:03.965718+0000","last_clean":"2024-09-20T19:07:03.965718+0000","last_became_active":"2024-09-20T19:04:20.906834+0000","last_became_peered":"2024-09-20T19:04:20.906834+0000","last_unstale":"2024-09-20T19:07:03.965718+0000","last_undegraded":"2024-09-20T19:07:03.965718+0000","last_fullsized":"2024-09-20T19:07:03.965718+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'15","last_scrub_stamp":"2024-09-20T19:05:57.548683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:57.548683+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:13.922548+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":385,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.962068+0000","last_change":"2024-09-20T19:05:52.464362+0000","last_active":"2024-09-20T19:07:03.962068+0000","last_peered":"2024-09-20T19:07:03.962068+0000","last_clean":"2024-09-20T19:07:03.962068+0000","last_became_active":"2024-09-20T19:00:01.190875+0000","last_became_peered":"2024-09-20T19:00:01.190875+0000","last_unstale":"2024-09-20T19:07:03.962068+0000","last_undegraded":"2024-09-20T19:07:03.962068+0000","last_fullsized":"2024-09-20T19:07:03.962068+0000","mapping_epoch":277,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":278,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:52.464261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:52.464261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":853,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.963332+0000","last_change":"2024-09-20T19:04:20.889614+0000","last_active":"2024-09-20T19:07:03.963332+0000","last_peered":"2024-09-20T19:07:03.963332+0000","last_clean":"2024-09-20T19:07:03.963332+0000","last_became_active":"2024-09-20T19:04:20.889273+0000","last_became_peered":"2024-09-20T19:04:20.889273+0000","last_unstale":"2024-09-20T19:07:03.963332+0000","last_undegraded":"2024-09-20T19:07:03.963332+0000","last_fullsized":"2024-09-20T19:07:03.963332+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:03.389611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:03.389611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:31.592842+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":813,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.977223+0000","last_change":"2024-09-20T19:02:31.120809+0000","last_active":"2024-09-20T19:07:03.977223+0000","last_peered":"2024-09-20T19:07:03.977223+0000","last_clean":"2024-09-20T19:07:03.977223+0000","last_became_active":"2024-09-20T18:49:51.964098+0000","last_became_peered":"2024-09-20T18:49:51.964098+0000","last_unstale":"2024-09-20T19:07:03.977223+0000","last_undegraded":"2024-09-20T19:07:03.977223+0000","last_fullsized":"2024-09-20T19:07:03.977223+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:31.120623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:31.120623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:40.760081+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.5","version":"447'4150","reported_seq":4249,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:11.355448+0000","last_change":"2024-09-20T19:06:57.022784+0000","last_active":"2024-09-20T19:07:11.355448+0000","last_peered":"2024-09-20T19:07:11.355448+0000","last_clean":"2024-09-20T19:07:11.355448+0000","last_became_active":"2024-09-20T19:06:56.460858+0000","last_became_peered":"2024-09-20T19:06:56.460858+0000","last_unstale":"2024-09-20T19:07:11.355448+0000","last_undegraded":"2024-09-20T19:07:11.355448+0000","last_fullsized":"2024-09-20T19:07:11.355448+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"442'3824","last_scrub_stamp":"2024-09-20T19:06:57.022671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:57.022671+0000","objects_scrubbed":1982,"log_size":4150,"log_dups_size":0,"ondisk_log_size":4150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:08:20.848977+0000","scrub_duration":299,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":151650304,"num_objects":2314,"num_object_clones":0,"num_object_copies":6942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2314,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":332,"num_write_kb":21248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"274'35","reported_seq":455,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.964746+0000","last_change":"2024-09-20T19:06:19.915841+0000","last_active":"2024-09-20T19:07:03.964746+0000","last_peered":"2024-09-20T19:07:03.964746+0000","last_clean":"2024-09-20T19:07:03.964746+0000","last_became_active":"2024-09-20T18:59:05.087391+0000","last_became_peered":"2024-09-20T18:59:05.087391+0000","last_unstale":"2024-09-20T19:07:03.964746+0000","last_undegraded":"2024-09-20T19:07:03.964746+0000","last_fullsized":"2024-09-20T19:07:03.964746+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'35","last_scrub_stamp":"2024-09-20T19:06:19.915767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:06:19.915767+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:46.153655+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":847,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.962216+0000","last_change":"2024-09-20T19:05:49.763478+0000","last_active":"2024-09-20T19:07:03.962216+0000","last_peered":"2024-09-20T19:07:03.962216+0000","last_clean":"2024-09-20T19:07:03.962216+0000","last_became_active":"2024-09-20T18:59:05.086670+0000","last_became_peered":"2024-09-20T18:59:05.086670+0000","last_unstale":"2024-09-20T19:07:03.962216+0000","last_undegraded":"2024-09-20T19:07:03.962216+0000","last_fullsized":"2024-09-20T19:07:03.962216+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:49.763407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:49.763407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:09.527897+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":835,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.962082+0000","last_change":"2024-09-20T19:05:11.444197+0000","last_active":"2024-09-20T19:07:03.962082+0000","last_peered":"2024-09-20T19:07:03.962082+0000","last_clean":"2024-09-20T19:07:03.962082+0000","last_became_active":"2024-09-20T18:49:49.957558+0000","last_became_peered":"2024-09-20T18:49:49.957558+0000","last_unstale":"2024-09-20T19:07:03.962082+0000","last_undegraded":"2024-09-20T19:07:03.962082+0000","last_fullsized":"2024-09-20T19:07:03.962082+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:11.444143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:05:11.444143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:36.686941+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":805,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.961101+0000","last_change":"2024-09-20T18:53:47.370152+0000","last_active":"2024-09-20T19:07:03.961101+0000","last_peered":"2024-09-20T19:07:03.961101+0000","last_clean":"2024-09-20T19:07:03.961101+0000","last_became_active":"2024-09-20T18:49:51.970798+0000","last_became_peered":"2024-09-20T18:49:51.970798+0000","last_unstale":"2024-09-20T19:07:03.961101+0000","last_undegraded":"2024-09-20T19:07:03.961101+0000","last_fullsized":"2024-09-20T19:07:03.961101+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:47.370108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:47.370108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:30.810770+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.6","version":"447'4160","reported_seq":4257,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:12.690251+0000","last_change":"2024-09-20T19:06:58.182942+0000","last_active":"2024-09-20T19:07:12.690251+0000","last_peered":"2024-09-20T19:07:12.690251+0000","last_clean":"2024-09-20T19:07:12.690251+0000","last_became_active":"2024-09-20T19:06:56.472642+0000","last_became_peered":"2024-09-20T19:06:56.472642+0000","last_unstale":"2024-09-20T19:07:12.690251+0000","last_undegraded":"2024-09-20T19:07:12.690251+0000","last_fullsized":"2024-09-20T19:07:12.690251+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"443'3869","last_scrub_stamp":"2024-09-20T19:06:58.182865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:58.182865+0000","objects_scrubbed":1953,"log_size":4160,"log_dups_size":0,"ondisk_log_size":4160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:08:21.446176+0000","scrub_duration":217,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":147062784,"num_objects":2244,"num_object_clones":0,"num_object_copies":6732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2244,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":291,"num_write_kb":18624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"273'18","reported_seq":168,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.942427+0000","last_change":"2024-09-20T19:04:20.936465+0000","last_active":"2024-09-20T19:07:03.942427+0000","last_peered":"2024-09-20T19:07:03.942427+0000","last_clean":"2024-09-20T19:07:03.942427+0000","last_became_active":"2024-09-20T19:04:20.888905+0000","last_became_peered":"2024-09-20T19:04:20.888905+0000","last_unstale":"2024-09-20T19:07:03.942427+0000","last_undegraded":"2024-09-20T19:07:03.942427+0000","last_fullsized":"2024-09-20T19:07:03.942427+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'18","last_scrub_stamp":"2024-09-20T19:03:13.201592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:13.201592+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:29.171477+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":866,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.963182+0000","last_change":"2024-09-20T19:06:15.692558+0000","last_active":"2024-09-20T19:07:03.963182+0000","last_peered":"2024-09-20T19:07:03.963182+0000","last_clean":"2024-09-20T19:07:03.963182+0000","last_became_active":"2024-09-20T19:04:20.892429+0000","last_became_peered":"2024-09-20T19:04:20.892429+0000","last_unstale":"2024-09-20T19:07:03.963182+0000","last_undegraded":"2024-09-20T19:07:03.963182+0000","last_fullsized":"2024-09-20T19:07:03.963182+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:15.692508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:06:15.692508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:27.149889+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":828,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.964308+0000","last_change":"2024-09-20T19:04:48.406445+0000","last_active":"2024-09-20T19:07:03.964308+0000","last_peered":"2024-09-20T19:07:03.964308+0000","last_clean":"2024-09-20T19:07:03.964308+0000","last_became_active":"2024-09-20T18:49:49.954944+0000","last_became_peered":"2024-09-20T18:49:49.954944+0000","last_unstale":"2024-09-20T19:07:03.964308+0000","last_undegraded":"2024-09-20T19:07:03.964308+0000","last_fullsized":"2024-09-20T19:07:03.964308+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:48.406373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:48.406373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":820,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.967941+0000","last_change":"2024-09-20T19:05:43.657860+0000","last_active":"2024-09-20T19:07:03.967941+0000","last_peered":"2024-09-20T19:07:03.967941+0000","last_clean":"2024-09-20T19:07:03.967941+0000","last_became_active":"2024-09-20T18:49:51.972970+0000","last_became_peered":"2024-09-20T18:49:51.972970+0000","last_unstale":"2024-09-20T19:07:03.967941+0000","last_undegraded":"2024-09-20T19:07:03.967941+0000","last_fullsized":"2024-09-20T19:07:03.967941+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:43.657780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:05:43.657780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:05.178691+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"447'4136","reported_seq":4239,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:10.771206+0000","last_change":"2024-09-20T19:06:57.243186+0000","last_active":"2024-09-20T19:07:10.771206+0000","last_peered":"2024-09-20T19:07:10.771206+0000","last_clean":"2024-09-20T19:07:10.771206+0000","last_became_active":"2024-09-20T19:06:56.479387+0000","last_became_peered":"2024-09-20T19:06:56.479387+0000","last_unstale":"2024-09-20T19:07:10.771206+0000","last_undegraded":"2024-09-20T19:07:10.771206+0000","last_fullsized":"2024-09-20T19:07:10.771206+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"442'3847","last_scrub_stamp":"2024-09-20T19:06:57.243081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:57.243081+0000","objects_scrubbed":1913,"log_size":4136,"log_dups_size":0,"ondisk_log_size":4136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:08:07.284534+0000","scrub_duration":380,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":144769024,"num_objects":2209,"num_object_clones":0,"num_object_copies":6627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":296,"num_write_kb":18944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"273'27","reported_seq":926,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.963506+0000","last_change":"2024-09-20T19:05:49.777488+0000","last_active":"2024-09-20T19:07:03.963506+0000","last_peered":"2024-09-20T19:07:03.963506+0000","last_clean":"2024-09-20T19:07:03.963506+0000","last_became_active":"2024-09-20T18:59:05.083726+0000","last_became_peered":"2024-09-20T18:59:05.083726+0000","last_unstale":"2024-09-20T19:07:03.963506+0000","last_undegraded":"2024-09-20T19:07:03.963506+0000","last_fullsized":"2024-09-20T19:07:03.963506+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'27","last_scrub_stamp":"2024-09-20T19:05:49.777414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:49.777414+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:12.438786+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":898,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.962654+0000","last_change":"2024-09-20T19:06:25.718817+0000","last_active":"2024-09-20T19:07:03.962654+0000","last_peered":"2024-09-20T19:07:03.962654+0000","last_clean":"2024-09-20T19:07:03.962654+0000","last_became_active":"2024-09-20T19:04:20.884854+0000","last_became_peered":"2024-09-20T19:04:20.884854+0000","last_unstale":"2024-09-20T19:07:03.962654+0000","last_undegraded":"2024-09-20T19:07:03.962654+0000","last_fullsized":"2024-09-20T19:07:03.962654+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-20T19:06:25.718731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:06:25.718731+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:49.878149+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"375'14","reported_seq":1093,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:11.205723+0000","last_change":"2024-09-20T19:06:14.693176+0000","last_active":"2024-09-20T19:07:11.205723+0000","last_peered":"2024-09-20T19:07:11.205723+0000","last_clean":"2024-09-20T19:07:11.205723+0000","last_became_active":"2024-09-20T19:04:20.892956+0000","last_became_peered":"2024-09-20T19:04:20.892956+0000","last_unstale":"2024-09-20T19:07:11.205723+0000","last_undegraded":"2024-09-20T19:07:11.205723+0000","last_fullsized":"2024-09-20T19:07:11.205723+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"375'14","last_scrub_stamp":"2024-09-20T19:06:14.693111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:06:14.693111+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:44.526972+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":834,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.964068+0000","last_change":"2024-09-20T19:03:06.888620+0000","last_active":"2024-09-20T19:07:03.964068+0000","last_peered":"2024-09-20T19:07:03.964068+0000","last_clean":"2024-09-20T19:07:03.964068+0000","last_became_active":"2024-09-20T18:59:05.087005+0000","last_became_peered":"2024-09-20T18:59:05.087005+0000","last_unstale":"2024-09-20T19:07:03.964068+0000","last_undegraded":"2024-09-20T19:07:03.964068+0000","last_fullsized":"2024-09-20T19:07:03.964068+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:06.888563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:03:06.888563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:09.776326+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.18","version":"447'3999","reported_seq":5225,"reported_epoch":447,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-20T19:07:10.828722+0000","last_change":"2024-09-20T19:06:59.708684+0000","last_active":"2024-09-20T19:07:10.828722+0000","last_peered":"2024-09-20T19:07:10.828722+0000","last_clean":"2024-09-20T19:06:57.442513+0000","last_became_active":"2024-09-20T19:06:59.662137+0000","last_became_peered":"2024-09-20T19:06:59.662137+0000","last_unstale":"2024-09-20T19:07:10.828722+0000","last_undegraded":"2024-09-20T19:07:10.828722+0000","last_fullsized":"2024-09-20T19:06:59.456805+0000","mapping_epoch":444,"log_start":"0'0","ondisk_log_start":"0'0","created":441,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":3999,"log_dups_size":0,"ondisk_log_size":3999,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":141656064,"num_objects":2162,"num_object_clones":0,"num_object_copies":6486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2896,"num_objects_unfound":0,"num_objects_dirty":2162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2162,"num_write_kb":138336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":384,"num_bytes_recovered":25165824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[3,6],"avail_no_missing":["3","5","6"],"object_location_counts":[{"shards":"3,5,6","objects":1448},{"shards":"3,6","objects":714}],"blocked_by":[],"up_primary":2,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":395,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.964094+0000","last_change":"2024-09-20T19:01:16.542398+0000","last_active":"2024-09-20T19:07:03.964094+0000","last_peered":"2024-09-20T19:07:03.964094+0000","last_clean":"2024-09-20T19:07:03.964094+0000","last_became_active":"2024-09-20T18:59:05.085427+0000","last_became_peered":"2024-09-20T18:59:05.085427+0000","last_unstale":"2024-09-20T19:07:03.964094+0000","last_undegraded":"2024-09-20T19:07:03.964094+0000","last_fullsized":"2024-09-20T19:07:03.964094+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:16.542335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:16.542335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:33.842644+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"273'12","reported_seq":854,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.963021+0000","last_change":"2024-09-20T19:06:19.973831+0000","last_active":"2024-09-20T19:07:03.963021+0000","last_peered":"2024-09-20T19:07:03.963021+0000","last_clean":"2024-09-20T19:07:03.963021+0000","last_became_active":"2024-09-20T18:49:48.500662+0000","last_became_peered":"2024-09-20T18:49:48.500662+0000","last_unstale":"2024-09-20T19:07:03.963021+0000","last_undegraded":"2024-09-20T19:07:03.963021+0000","last_fullsized":"2024-09-20T19:07:03.963021+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'12","last_scrub_stamp":"2024-09-20T19:06:19.973765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:06:19.973765+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:34.482020+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":824,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.964636+0000","last_change":"2024-09-20T19:05:22.605885+0000","last_active":"2024-09-20T19:07:03.964636+0000","last_peered":"2024-09-20T19:07:03.964636+0000","last_clean":"2024-09-20T19:07:03.964636+0000","last_became_active":"2024-09-20T18:49:49.956749+0000","last_became_peered":"2024-09-20T18:49:49.956749+0000","last_unstale":"2024-09-20T19:07:03.964636+0000","last_undegraded":"2024-09-20T19:07:03.964636+0000","last_fullsized":"2024-09-20T19:07:03.964636+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:22.605839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:05:22.605839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":860,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.961782+0000","last_change":"2024-09-20T19:05:56.761494+0000","last_active":"2024-09-20T19:07:03.961782+0000","last_peered":"2024-09-20T19:07:03.961782+0000","last_clean":"2024-09-20T19:07:03.961782+0000","last_became_active":"2024-09-20T19:04:20.914115+0000","last_became_peered":"2024-09-20T19:04:20.914115+0000","last_unstale":"2024-09-20T19:07:03.961782+0000","last_undegraded":"2024-09-20T19:07:03.961782+0000","last_fullsized":"2024-09-20T19:07:03.961782+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:56.761425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:05:56.761425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:15.725837+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.19","version":"447'4184","reported_seq":4257,"reported_epoch":447,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-20T19:07:12.764153+0000","last_change":"2024-09-20T19:06:59.823450+0000","last_active":"2024-09-20T19:07:12.764153+0000","last_peered":"2024-09-20T19:07:12.764153+0000","last_clean":"2024-09-20T19:06:57.404714+0000","last_became_active":"2024-09-20T19:06:59.810412+0000","last_became_peered":"2024-09-20T19:06:59.810412+0000","last_unstale":"2024-09-20T19:07:12.764153+0000","last_undegraded":"2024-09-20T19:06:59.703127+0000","last_fullsized":"2024-09-20T19:06:59.454934+0000","mapping_epoch":444,"log_start":"0'0","ondisk_log_start":"0'0","created":441,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":4184,"log_dups_size":0,"ondisk_log_size":4184,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":148996096,"num_objects":2274,"num_object_clones":0,"num_object_copies":6822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3876,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2274,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2274,"num_write_kb":145504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,2],"avail_no_missing":["0","3","6"],"object_location_counts":[{"shards":"0,2","objects":336},{"shards":"0,3,6","objects":1938}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"273'15","reported_seq":838,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.974144+0000","last_change":"2024-09-20T19:01:15.875779+0000","last_active":"2024-09-20T19:07:03.974144+0000","last_peered":"2024-09-20T19:07:03.974144+0000","last_clean":"2024-09-20T19:07:03.974144+0000","last_became_active":"2024-09-20T18:49:48.500803+0000","last_became_peered":"2024-09-20T18:49:48.500803+0000","last_unstale":"2024-09-20T19:07:03.974144+0000","last_undegraded":"2024-09-20T19:07:03.974144+0000","last_fullsized":"2024-09-20T19:07:03.974144+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:01:15.875693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:15.875693+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":864,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.961319+0000","last_change":"2024-09-20T19:04:47.713764+0000","last_active":"2024-09-20T19:07:03.961319+0000","last_peered":"2024-09-20T19:07:03.961319+0000","last_clean":"2024-09-20T19:07:03.961319+0000","last_became_active":"2024-09-20T19:04:20.893250+0000","last_became_peered":"2024-09-20T19:04:20.893250+0000","last_unstale":"2024-09-20T19:07:03.961319+0000","last_undegraded":"2024-09-20T19:07:03.961319+0000","last_fullsized":"2024-09-20T19:07:03.961319+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:47.713585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:04:47.713585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:10.723942+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":824,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.962884+0000","last_change":"2024-09-20T19:04:50.573619+0000","last_active":"2024-09-20T19:07:03.962884+0000","last_peered":"2024-09-20T19:07:03.962884+0000","last_clean":"2024-09-20T19:07:03.962884+0000","last_became_active":"2024-09-20T18:49:49.961947+0000","last_became_peered":"2024-09-20T18:49:49.961947+0000","last_unstale":"2024-09-20T19:07:03.962884+0000","last_undegraded":"2024-09-20T19:07:03.962884+0000","last_fullsized":"2024-09-20T19:07:03.962884+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:50.573550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:50.573550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:33.984555+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"66'1","reported_seq":865,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.962204+0000","last_change":"2024-09-20T19:04:20.913449+0000","last_active":"2024-09-20T19:07:03.962204+0000","last_peered":"2024-09-20T19:07:03.962204+0000","last_clean":"2024-09-20T19:07:03.962204+0000","last_became_active":"2024-09-20T19:04:20.892236+0000","last_became_peered":"2024-09-20T19:04:20.892236+0000","last_unstale":"2024-09-20T19:07:03.962204+0000","last_undegraded":"2024-09-20T19:07:03.962204+0000","last_fullsized":"2024-09-20T19:07:03.962204+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T19:03:12.499684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:03:12.499684+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:40.712118+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"274'15","reported_seq":845,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.971216+0000","last_change":"2024-09-20T19:06:42.967192+0000","last_active":"2024-09-20T19:07:03.971216+0000","last_peered":"2024-09-20T19:07:03.971216+0000","last_clean":"2024-09-20T19:07:03.971216+0000","last_became_active":"2024-09-20T18:49:48.501011+0000","last_became_peered":"2024-09-20T18:49:48.501011+0000","last_unstale":"2024-09-20T19:07:03.971216+0000","last_undegraded":"2024-09-20T19:07:03.971216+0000","last_fullsized":"2024-09-20T19:07:03.971216+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'15","last_scrub_stamp":"2024-09-20T19:06:42.967003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:06:42.967003+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:59.075700+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":825,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.972053+0000","last_change":"2024-09-20T19:01:30.864866+0000","last_active":"2024-09-20T19:07:03.972053+0000","last_peered":"2024-09-20T19:07:03.972053+0000","last_clean":"2024-09-20T19:07:03.972053+0000","last_became_active":"2024-09-20T18:49:45.938511+0000","last_became_peered":"2024-09-20T18:49:45.938511+0000","last_unstale":"2024-09-20T19:07:03.972053+0000","last_undegraded":"2024-09-20T19:07:03.972053+0000","last_fullsized":"2024-09-20T19:07:03.972053+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:30.864794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:30.864794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:11.643706+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":1041,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.962232+0000","last_change":"2024-09-20T19:04:41.601208+0000","last_active":"2024-09-20T19:07:03.962232+0000","last_peered":"2024-09-20T19:07:03.962232+0000","last_clean":"2024-09-20T19:07:03.962232+0000","last_became_active":"2024-09-20T18:49:49.960550+0000","last_became_peered":"2024-09-20T18:49:49.960550+0000","last_unstale":"2024-09-20T19:07:03.962232+0000","last_undegraded":"2024-09-20T19:07:03.962232+0000","last_fullsized":"2024-09-20T19:07:03.962232+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T19:04:41.601073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:41.601073+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:34.939420+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":156,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.942855+0000","last_change":"2024-09-20T19:04:20.884397+0000","last_active":"2024-09-20T19:07:03.942855+0000","last_peered":"2024-09-20T19:07:03.942855+0000","last_clean":"2024-09-20T19:07:03.942855+0000","last_became_active":"2024-09-20T19:04:20.884100+0000","last_became_peered":"2024-09-20T19:04:20.884100+0000","last_unstale":"2024-09-20T19:07:03.942855+0000","last_undegraded":"2024-09-20T19:07:03.942855+0000","last_fullsized":"2024-09-20T19:07:03.942855+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:59.497810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:59:59.497810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"274'20","reported_seq":853,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.962335+0000","last_change":"2024-09-20T18:57:20.701592+0000","last_active":"2024-09-20T19:07:03.962335+0000","last_peered":"2024-09-20T19:07:03.962335+0000","last_clean":"2024-09-20T19:07:03.962335+0000","last_became_active":"2024-09-20T18:49:58.997494+0000","last_became_peered":"2024-09-20T18:49:58.997494+0000","last_unstale":"2024-09-20T19:07:03.962335+0000","last_undegraded":"2024-09-20T19:07:03.962335+0000","last_fullsized":"2024-09-20T19:07:03.962335+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:57:20.701543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:20.701543+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:28.869204+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":842,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.961577+0000","last_change":"2024-09-20T19:04:20.884940+0000","last_active":"2024-09-20T19:07:03.961577+0000","last_peered":"2024-09-20T19:07:03.961577+0000","last_clean":"2024-09-20T19:07:03.961577+0000","last_became_active":"2024-09-20T19:04:20.884582+0000","last_became_peered":"2024-09-20T19:04:20.884582+0000","last_unstale":"2024-09-20T19:07:03.961577+0000","last_undegraded":"2024-09-20T19:07:03.961577+0000","last_fullsized":"2024-09-20T19:07:03.961577+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.632380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.632380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:36.764394+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":1041,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.971059+0000","last_change":"2024-09-20T19:06:26.893183+0000","last_active":"2024-09-20T19:07:03.971059+0000","last_peered":"2024-09-20T19:07:03.971059+0000","last_clean":"2024-09-20T19:07:03.971059+0000","last_became_active":"2024-09-20T18:49:49.962243+0000","last_became_peered":"2024-09-20T18:49:49.962243+0000","last_unstale":"2024-09-20T19:07:03.971059+0000","last_undegraded":"2024-09-20T19:07:03.971059+0000","last_fullsized":"2024-09-20T19:07:03.971059+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T19:06:26.893109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:06:26.893109+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:34.296447+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":829,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.970027+0000","last_change":"2024-09-20T19:05:52.578042+0000","last_active":"2024-09-20T19:07:03.970027+0000","last_peered":"2024-09-20T19:07:03.970027+0000","last_clean":"2024-09-20T19:07:03.970027+0000","last_became_active":"2024-09-20T18:49:51.969093+0000","last_became_peered":"2024-09-20T18:49:51.969093+0000","last_unstale":"2024-09-20T19:07:03.970027+0000","last_undegraded":"2024-09-20T19:07:03.970027+0000","last_fullsized":"2024-09-20T19:07:03.970027+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:52.577979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:05:52.577979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:16.347110+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"273'19","reported_seq":176,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.942475+0000","last_change":"2024-09-20T19:06:44.432152+0000","last_active":"2024-09-20T19:07:03.942475+0000","last_peered":"2024-09-20T19:07:03.942475+0000","last_clean":"2024-09-20T19:07:03.942475+0000","last_became_active":"2024-09-20T19:04:20.897330+0000","last_became_peered":"2024-09-20T19:04:20.897330+0000","last_unstale":"2024-09-20T19:07:03.942475+0000","last_undegraded":"2024-09-20T19:07:03.942475+0000","last_fullsized":"2024-09-20T19:07:03.942475+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'19","last_scrub_stamp":"2024-09-20T19:06:44.432058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:06:44.432058+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:44.463553+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":156,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.942746+0000","last_change":"2024-09-20T19:04:20.896732+0000","last_active":"2024-09-20T19:07:03.942746+0000","last_peered":"2024-09-20T19:07:03.942746+0000","last_clean":"2024-09-20T19:07:03.942746+0000","last_became_active":"2024-09-20T19:04:20.896007+0000","last_became_peered":"2024-09-20T19:04:20.896007+0000","last_unstale":"2024-09-20T19:07:03.942746+0000","last_undegraded":"2024-09-20T19:07:03.942746+0000","last_fullsized":"2024-09-20T19:07:03.942746+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:06.310027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:59:06.310027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":816,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.977693+0000","last_change":"2024-09-20T19:02:38.906352+0000","last_active":"2024-09-20T19:07:03.977693+0000","last_peered":"2024-09-20T19:07:03.977693+0000","last_clean":"2024-09-20T19:07:03.977693+0000","last_became_active":"2024-09-20T18:49:49.957067+0000","last_became_peered":"2024-09-20T18:49:49.957067+0000","last_unstale":"2024-09-20T19:07:03.977693+0000","last_undegraded":"2024-09-20T19:07:03.977693+0000","last_fullsized":"2024-09-20T19:07:03.977693+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:38.906161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:38.906161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:37.858190+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":796,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.957919+0000","last_change":"2024-09-20T18:57:16.789412+0000","last_active":"2024-09-20T19:07:03.957919+0000","last_peered":"2024-09-20T19:07:03.957919+0000","last_clean":"2024-09-20T19:07:03.957919+0000","last_became_active":"2024-09-20T18:49:51.966915+0000","last_became_peered":"2024-09-20T18:49:51.966915+0000","last_unstale":"2024-09-20T19:07:03.957919+0000","last_undegraded":"2024-09-20T19:07:03.957919+0000","last_fullsized":"2024-09-20T19:07:03.957919+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:16.789095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:16.789095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:37.856372+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"273'15","reported_seq":895,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.978301+0000","last_change":"2024-09-20T19:06:51.012046+0000","last_active":"2024-09-20T19:07:03.978301+0000","last_peered":"2024-09-20T19:07:03.978301+0000","last_clean":"2024-09-20T19:07:03.978301+0000","last_became_active":"2024-09-20T18:59:05.085230+0000","last_became_peered":"2024-09-20T18:59:05.085230+0000","last_unstale":"2024-09-20T19:07:03.978301+0000","last_undegraded":"2024-09-20T19:07:03.978301+0000","last_fullsized":"2024-09-20T19:07:03.978301+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:06:51.011969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:06:51.011969+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:57.317016+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":842,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.962095+0000","last_change":"2024-09-20T19:04:20.885127+0000","last_active":"2024-09-20T19:07:03.962095+0000","last_peered":"2024-09-20T19:07:03.962095+0000","last_clean":"2024-09-20T19:07:03.962095+0000","last_became_active":"2024-09-20T19:04:20.884660+0000","last_became_peered":"2024-09-20T19:04:20.884660+0000","last_unstale":"2024-09-20T19:07:03.962095+0000","last_undegraded":"2024-09-20T19:07:03.962095+0000","last_fullsized":"2024-09-20T19:07:03.962095+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:07.408197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:03:07.408197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:07.715846+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":832,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.958216+0000","last_change":"2024-09-20T19:06:22.930775+0000","last_active":"2024-09-20T19:07:03.958216+0000","last_peered":"2024-09-20T19:07:03.958216+0000","last_clean":"2024-09-20T19:07:03.958216+0000","last_became_active":"2024-09-20T18:49:49.962523+0000","last_became_peered":"2024-09-20T18:49:49.962523+0000","last_unstale":"2024-09-20T19:07:03.958216+0000","last_undegraded":"2024-09-20T19:07:03.958216+0000","last_fullsized":"2024-09-20T19:07:03.958216+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:22.930701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:06:22.930701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:28.969493+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":821,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.965656+0000","last_change":"2024-09-20T19:05:51.626774+0000","last_active":"2024-09-20T19:07:03.965656+0000","last_peered":"2024-09-20T19:07:03.965656+0000","last_clean":"2024-09-20T19:07:03.965656+0000","last_became_active":"2024-09-20T18:49:51.967313+0000","last_became_peered":"2024-09-20T18:49:51.967313+0000","last_unstale":"2024-09-20T19:07:03.965656+0000","last_undegraded":"2024-09-20T19:07:03.965656+0000","last_fullsized":"2024-09-20T19:07:03.965656+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:51.626697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:05:51.626697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:13.789020+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"274'19","reported_seq":889,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.957988+0000","last_change":"2024-09-20T19:01:05.495603+0000","last_active":"2024-09-20T19:07:03.957988+0000","last_peered":"2024-09-20T19:07:03.957988+0000","last_clean":"2024-09-20T19:07:03.957988+0000","last_became_active":"2024-09-20T18:59:05.075390+0000","last_became_peered":"2024-09-20T18:59:05.075390+0000","last_unstale":"2024-09-20T19:07:03.957988+0000","last_undegraded":"2024-09-20T19:07:03.957988+0000","last_fullsized":"2024-09-20T19:07:03.957988+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'19","last_scrub_stamp":"2024-09-20T19:01:05.495478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:05.495478+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:04.804811+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":840,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.964716+0000","last_change":"2024-09-20T19:05:43.686902+0000","last_active":"2024-09-20T19:07:03.964716+0000","last_peered":"2024-09-20T19:07:03.964716+0000","last_clean":"2024-09-20T19:07:03.964716+0000","last_became_active":"2024-09-20T18:49:45.942105+0000","last_became_peered":"2024-09-20T18:49:45.942105+0000","last_unstale":"2024-09-20T19:07:03.964716+0000","last_undegraded":"2024-09-20T19:07:03.964716+0000","last_fullsized":"2024-09-20T19:07:03.964716+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:43.686842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:43.686842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":856,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.960628+0000","last_change":"2024-09-20T19:04:20.912120+0000","last_active":"2024-09-20T19:07:03.960628+0000","last_peered":"2024-09-20T19:07:03.960628+0000","last_clean":"2024-09-20T19:07:03.960628+0000","last_became_active":"2024-09-20T19:04:20.911292+0000","last_became_peered":"2024-09-20T19:04:20.911292+0000","last_unstale":"2024-09-20T19:07:03.960628+0000","last_undegraded":"2024-09-20T19:07:03.960628+0000","last_fullsized":"2024-09-20T19:07:03.960628+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:27.267254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:27.267254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":834,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.962667+0000","last_change":"2024-09-20T19:06:58.896885+0000","last_active":"2024-09-20T19:07:03.962667+0000","last_peered":"2024-09-20T19:07:03.962667+0000","last_clean":"2024-09-20T19:07:03.962667+0000","last_became_active":"2024-09-20T18:59:05.081179+0000","last_became_peered":"2024-09-20T18:59:05.081179+0000","last_unstale":"2024-09-20T19:07:03.962667+0000","last_undegraded":"2024-09-20T19:07:03.962667+0000","last_fullsized":"2024-09-20T19:07:03.962667+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:58.896826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:06:58.896826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:08:23.832233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"273'8","reported_seq":874,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.974787+0000","last_change":"2024-09-20T19:06:21.974151+0000","last_active":"2024-09-20T19:07:03.974787+0000","last_peered":"2024-09-20T19:07:03.974787+0000","last_clean":"2024-09-20T19:07:03.974787+0000","last_became_active":"2024-09-20T18:59:05.076365+0000","last_became_peered":"2024-09-20T18:59:05.076365+0000","last_unstale":"2024-09-20T19:07:03.974787+0000","last_undegraded":"2024-09-20T19:07:03.974787+0000","last_fullsized":"2024-09-20T19:07:03.974787+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'8","last_scrub_stamp":"2024-09-20T19:06:21.974084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:06:21.974084+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:38.613762+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":841,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.965683+0000","last_change":"2024-09-20T19:06:45.833934+0000","last_active":"2024-09-20T19:07:03.965683+0000","last_peered":"2024-09-20T19:07:03.965683+0000","last_clean":"2024-09-20T19:07:03.965683+0000","last_became_active":"2024-09-20T18:49:45.940988+0000","last_became_peered":"2024-09-20T18:49:45.940988+0000","last_unstale":"2024-09-20T19:07:03.965683+0000","last_undegraded":"2024-09-20T19:07:03.965683+0000","last_fullsized":"2024-09-20T19:07:03.965683+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:45.833872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:06:45.833872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:08:13.366576+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":420,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.964617+0000","last_change":"2024-09-20T19:04:20.900567+0000","last_active":"2024-09-20T19:07:03.964617+0000","last_peered":"2024-09-20T19:07:03.964617+0000","last_clean":"2024-09-20T19:07:03.964617+0000","last_became_active":"2024-09-20T19:04:20.900214+0000","last_became_peered":"2024-09-20T19:04:20.900214+0000","last_unstale":"2024-09-20T19:07:03.964617+0000","last_undegraded":"2024-09-20T19:07:03.964617+0000","last_fullsized":"2024-09-20T19:07:03.964617+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:41.497115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:41.497115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:38.822793+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":838,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.961855+0000","last_change":"2024-09-20T19:05:57.789513+0000","last_active":"2024-09-20T19:07:03.961855+0000","last_peered":"2024-09-20T19:07:03.961855+0000","last_clean":"2024-09-20T19:07:03.961855+0000","last_became_active":"2024-09-20T19:04:20.885211+0000","last_became_peered":"2024-09-20T19:04:20.885211+0000","last_unstale":"2024-09-20T19:07:03.961855+0000","last_undegraded":"2024-09-20T19:07:03.961855+0000","last_fullsized":"2024-09-20T19:07:03.961855+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:57.789465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:05:57.789465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:08.966824+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.10","version":"447'4186","reported_seq":4262,"reported_epoch":447,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-20T19:07:12.115903+0000","last_change":"2024-09-20T19:06:59.721489+0000","last_active":"2024-09-20T19:07:12.115903+0000","last_peered":"2024-09-20T19:07:12.115903+0000","last_clean":"2024-09-20T19:06:57.424433+0000","last_became_active":"2024-09-20T19:06:59.675453+0000","last_became_peered":"2024-09-20T19:06:59.675453+0000","last_unstale":"2024-09-20T19:07:12.115903+0000","last_undegraded":"2024-09-20T19:06:59.660821+0000","last_fullsized":"2024-09-20T19:06:59.456598+0000","mapping_epoch":444,"log_start":"0'0","ondisk_log_start":"0'0","created":441,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":4186,"log_dups_size":0,"ondisk_log_size":4186,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":148996096,"num_objects":2274,"num_object_clones":0,"num_object_copies":6822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3832,"num_objects_misplaced":1916,"num_objects_unfound":0,"num_objects_dirty":2274,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2274,"num_write_kb":145504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[1,0],"avail_no_missing":["1","3","7"],"object_location_counts":[{"shards":"0,1","objects":358},{"shards":"1,3,7","objects":1916}],"blocked_by":[],"up_primary":0,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":820,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.978692+0000","last_change":"2024-09-20T19:06:24.854187+0000","last_active":"2024-09-20T19:07:03.978692+0000","last_peered":"2024-09-20T19:07:03.978692+0000","last_clean":"2024-09-20T19:07:03.978692+0000","last_became_active":"2024-09-20T19:04:20.890101+0000","last_became_peered":"2024-09-20T19:04:20.890101+0000","last_unstale":"2024-09-20T19:07:03.978692+0000","last_undegraded":"2024-09-20T19:07:03.978692+0000","last_fullsized":"2024-09-20T19:07:03.978692+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:24.854124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:06:24.854124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:46.101065+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":824,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.980550+0000","last_change":"2024-09-20T19:06:44.934346+0000","last_active":"2024-09-20T19:07:03.980550+0000","last_peered":"2024-09-20T19:07:03.980550+0000","last_clean":"2024-09-20T19:07:03.980550+0000","last_became_active":"2024-09-20T18:49:45.937992+0000","last_became_peered":"2024-09-20T18:49:45.937992+0000","last_unstale":"2024-09-20T19:07:03.980550+0000","last_undegraded":"2024-09-20T19:07:03.980550+0000","last_fullsized":"2024-09-20T19:07:03.980550+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:44.934143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:06:44.934143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:47.788190+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"274'19","reported_seq":427,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.965906+0000","last_change":"2024-09-20T19:06:26.909675+0000","last_active":"2024-09-20T19:07:03.965906+0000","last_peered":"2024-09-20T19:07:03.965906+0000","last_clean":"2024-09-20T19:07:03.965906+0000","last_became_active":"2024-09-20T18:59:05.089294+0000","last_became_peered":"2024-09-20T18:59:05.089294+0000","last_unstale":"2024-09-20T19:07:03.965906+0000","last_undegraded":"2024-09-20T19:07:03.965906+0000","last_fullsized":"2024-09-20T19:07:03.965906+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'19","last_scrub_stamp":"2024-09-20T19:06:26.909593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:06:26.909593+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:54.942533+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":810,"reported_epoch":447,"state":"active+clean","last_fresh":"2024-09-20T19:07:03.970182+0000","last_change":"2024-09-20T19:01:40.961449+0000","last_active":"2024-09-20T19:07:03.970182+0000","last_peered":"2024-09-20T19:07:03.970182+0000","last_clean":"2024-09-20T19:07:03.970182+0000","last_became_active":"2024-09-20T18:49:49.957582+0000","last_became_peered":"2024-09-20T18:49:49.957582+0000","last_unstale":"2024-09-20T19:07:03.970182+0000","last_undegraded":"2024-09-20T19:07:03.970182+0000","last_fullsized":"2024-09-20T19:07:03.970182+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:40.961363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:40.961363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:08.603698+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":26,"stat_sum":{"num_bytes":4706861056,"num_objects":71823,"num_object_clones":0,"num_object_copies":215469,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13750,"num_objects_misplaced":24764,"num_objects_unfound":0,"num_objects_dirty":71823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58200,"num_write_kb":3724672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3109,"num_bytes_recovered":203751424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":14619115520,"data_stored":14619115520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109419,"ondisk_log_size":109419,"up":78,"acting":68,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1185,"internal_metadata":0},"log_size":87,"ondisk_log_size":87,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1114,"num_read_kb":871,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":257,"num_bytes_recovered":4890,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":686,"ondisk_log_size":686,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":3105,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1038,"num_read_kb":1033,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266901,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2470388,"kb_used_data":1829732,"kb_used_omap":1,"kb_used_meta":640638,"kb_avail":91274764,"statfs":{"total":95995035648,"available":93465358336,"internally_reserved":0,"allocated":1873645568,"data_stored":1872948298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":656013770},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528540,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2825184,"kb_used_data":2298384,"kb_used_omap":1,"kb_used_meta":526782,"kb_avail":90919968,"statfs":{"total":95995035648,"available":93102047232,"internally_reserved":0,"allocated":2353545216,"data_stored":2352867764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":539425222},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757473,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1798576,"kb_used_data":1167648,"kb_used_omap":1,"kb_used_meta":630910,"kb_avail":91946576,"statfs":{"total":95995035648,"available":94153293824,"internally_reserved":0,"allocated":1195671552,"data_stored":1194977889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":646052293},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":304,"seq":1305670058055,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1647168,"kb_used_data":1265584,"kb_used_omap":5,"kb_used_meta":381562,"kb_avail":92097984,"statfs":{"total":95995035648,"available":94308335616,"internally_reserved":0,"allocated":1295958016,"data_stored":1295248863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5633,"internal_metadata":390719999},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182637,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2738028,"kb_used_data":1936348,"kb_used_omap":1,"kb_used_meta":801662,"kb_avail":91007124,"statfs":{"total":95995035648,"available":93191294976,"internally_reserved":0,"allocated":1982820352,"data_stored":1982130165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":820902343},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411570,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2491556,"kb_used_data":1810644,"kb_used_omap":1,"kb_used_meta":680894,"kb_avail":91253596,"statfs":{"total":95995035648,"available":93443682304,"internally_reserved":0,"allocated":1854099456,"data_stored":1853418302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":697235909},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575096,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3095252,"kb_used_data":2396292,"kb_used_omap":1,"kb_used_meta":698942,"kb_avail":90649900,"statfs":{"total":95995035648,"available":92825497600,"internally_reserved":0,"allocated":2453803008,"data_stored":2453073886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":715717061},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,3,0,1,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738622,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2252596,"kb_used_data":1477284,"kb_used_omap":1,"kb_used_meta":775294,"kb_avail":91492556,"statfs":{"total":95995035648,"available":93688377344,"internally_reserved":0,"allocated":1512738816,"data_stored":1512041242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":793901509},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1185,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1526464512,"data_stored":1526464512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2460024832,"data_stored":2460024832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1881276416,"data_stored":1881276416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2005008384,"data_stored":2005008384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1303707648,"data_stored":1303707648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1196097536,"data_stored":1196097536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2352676864,"data_stored":2352676864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1893859328,"data_stored":1893859328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T19:07:16.689 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2024-09-20T19:07:16.689 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd pool set .mgr pg_num 11 2024-09-20T19:07:16.696 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:16.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:16.731 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:16.731 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:16.800 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T19:07:16.800 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:07:16.832 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:07:16.864 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:16.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:16.865 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:07:16.966 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:07:16.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:16.997 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:16.997 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:07:17.097 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:17.132 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 54 16 76343 76327 88.3056 56.5 0.0239573 0.0113185 2024-09-20T19:07:17.145 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:17.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:17.146 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:17.246 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:17.280 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:17.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:17.281 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:17.382 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:07:17.411 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:17.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:17.412 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:07:17.513 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:17.558 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:17.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:17.560 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:17.660 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:17.693 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:17.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:17.694 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:17.761 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:07:17 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:07:17.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:17 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:07:17.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:17 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:07:17.795 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:17.827 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:17.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:17.829 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:17.929 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:17.973 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:17.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:17.975 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:18.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:17 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:07:18.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:17 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:07:18.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:17 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:07:18.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:17 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:07:18.076 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:18.108 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:18.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:18.109 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:18.132 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 55 16 77304 77288 87.7921 60.0625 0.010606 0.0113832 2024-09-20T19:07:18.209 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:18.247 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:18.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:18.248 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:07:18.348 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:07:18.377 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:18.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:18.378 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:07:18.406 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-20T19:07:18.479 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:18.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:18.539 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:18.539 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:18.640 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:18.672 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:18.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:18.673 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:18.774 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:18.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:18 smithi083 ceph-mon[29421]: pgmap v887: 158 pgs: 4 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 148 active+clean; 4.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 56 MiB/s wr, 902 op/s; 13750/224388 objects degraded (6.128%); 23181/224388 objects misplaced (10.331%); 17 MiB/s, 274 objects/s recovering 2024-09-20T19:07:18.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:18 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:07:18.791 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:18 smithi083 ceph-mon[36260]: pgmap v887: 158 pgs: 4 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 148 active+clean; 4.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 56 MiB/s wr, 902 op/s; 13750/224388 objects degraded (6.128%); 23181/224388 objects misplaced (10.331%); 17 MiB/s, 274 objects/s recovering 2024-09-20T19:07:18.791 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:18 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:07:18.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:18.827 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:18.827 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:18.928 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:18.964 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:18.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:18.965 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:19.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:18 smithi142 ceph-mon[32508]: pgmap v887: 158 pgs: 4 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 148 active+clean; 4.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 56 MiB/s wr, 902 op/s; 13750/224388 objects degraded (6.128%); 23181/224388 objects misplaced (10.331%); 17 MiB/s, 274 objects/s recovering 2024-09-20T19:07:19.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:18 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:07:19.066 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:19.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:19.106 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:19.106 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:19.132 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 56 16 78097 78081 87.1096 49.5625 0.00951762 0.0114747 2024-09-20T19:07:19.207 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:19.227 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:19.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:19.228 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:19.329 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:19.369 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:19.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:19.370 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:19.471 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:19.520 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:19.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:19.521 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:19.621 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:19.641 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:19.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:19.642 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:19.743 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:19.785 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:19.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:19.786 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:19.886 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:19.929 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:19.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:19.930 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:20.031 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:20.075 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:20.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:20.076 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:20.132 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 57 16 78798 78782 86.3501 43.8125 0.00241866 0.0115617 2024-09-20T19:07:20.177 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:20.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:20.202 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:20.203 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:20.303 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:20.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:20.344 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:20.344 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:20.445 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:20.479 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:20.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:20.480 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:20.581 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:07:20.609 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:20.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:20.610 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:07:20.711 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:20.747 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:20.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:20.748 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:07:20.849 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:20.869 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:20.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:20.870 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:20.971 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:21.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:20 smithi142 ceph-mon[32508]: pgmap v888: 158 pgs: 4 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 148 active+clean; 4.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 852 B/s rd, 51 MiB/s wr, 818 op/s; 13750/228927 objects degraded (6.006%); 21984/228927 objects misplaced (9.603%); 18 MiB/s, 283 objects/s recovering 2024-09-20T19:07:21.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:21.019 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:21.019 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:21.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:20 smithi083 ceph-mon[29421]: pgmap v888: 158 pgs: 4 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 148 active+clean; 4.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 852 B/s rd, 51 MiB/s wr, 818 op/s; 13750/228927 objects degraded (6.006%); 21984/228927 objects misplaced (9.603%); 18 MiB/s, 283 objects/s recovering 2024-09-20T19:07:21.042 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:07:20 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:07:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:07:21.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:20 smithi083 ceph-mon[36260]: pgmap v888: 158 pgs: 4 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 148 active+clean; 4.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 852 B/s rd, 51 MiB/s wr, 818 op/s; 13750/228927 objects degraded (6.006%); 21984/228927 objects misplaced (9.603%); 18 MiB/s, 283 objects/s recovering 2024-09-20T19:07:21.120 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:21.132 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 58 16 79548 79532 85.6697 46.875 0.0115072 0.0116669 2024-09-20T19:07:21.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:21.147 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:21.148 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:21.249 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:21.285 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:21.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:21.287 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:21.388 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:21.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:21.426 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:21.427 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:21.527 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:21.552 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:21.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:21.553 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:21.654 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:21.692 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:21.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:21.693 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:21.793 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:21.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:21.834 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:21.834 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:07:21.935 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:21.969 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:21.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:21.970 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:22.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:21 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 13750/228927 objects degraded (6.006%), 4 pgs degraded (PG_DEGRADED) 2024-09-20T19:07:22.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:21 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 13750/228927 objects degraded (6.006%), 4 pgs degraded (PG_DEGRADED) 2024-09-20T19:07:22.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:21 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 13750/228927 objects degraded (6.006%), 4 pgs degraded (PG_DEGRADED) 2024-09-20T19:07:22.071 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:07:22.107 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:22.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:22.108 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:07:22.132 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 59 16 80301 80285 85.0154 47.0625 0.0105688 0.0117568 2024-09-20T19:07:22.208 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:22.246 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:22.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:22.247 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:07:22.347 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:22.392 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:22.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:22.393 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:22.494 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:07:22.524 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:22.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:22.526 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:07:22.626 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:22.648 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:22.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:22.649 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:22.750 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:22.790 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:22.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:22.791 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:22.892 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:22.924 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:22.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:22.926 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:22.942 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:22 smithi083 ceph-mon[29421]: pgmap v889: 158 pgs: 4 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 148 active+clean; 4.7 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 852 B/s rd, 52 MiB/s wr, 836 op/s; 13750/233145 objects degraded (5.898%); 20889/233145 objects misplaced (8.960%); 16 MiB/s, 262 objects/s recovering 2024-09-20T19:07:22.942 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:22 smithi083 ceph-mon[36260]: pgmap v889: 158 pgs: 4 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 148 active+clean; 4.7 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 852 B/s rd, 52 MiB/s wr, 836 op/s; 13750/233145 objects degraded (5.898%); 20889/233145 objects misplaced (8.960%); 16 MiB/s, 262 objects/s recovering 2024-09-20T19:07:23.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:22 smithi142 ceph-mon[32508]: pgmap v889: 158 pgs: 4 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 148 active+clean; 4.7 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 852 B/s rd, 52 MiB/s wr, 836 op/s; 13750/233145 objects degraded (5.898%); 20889/233145 objects misplaced (8.960%); 16 MiB/s, 262 objects/s recovering 2024-09-20T19:07:23.026 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:07:23.055 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:23.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:23.057 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:07:23.133 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:2024-09-20T19:07:23.129690+0000 min lat: 0.00109192 max lat: 2.36333 avg lat: 0.0118227 2024-09-20T19:07:23.133 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-20T19:07:23.133 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 60 16 81211 81195 84.5465 56.875 0.0260726 0.0118227 2024-09-20T19:07:23.157 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:23.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:23.201 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:23.201 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:23.302 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:23.330 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:23.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:23.331 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:23.431 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:23.463 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:23.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:23.464 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:23.565 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:23.589 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:23.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:23.590 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:23.691 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:07:23.721 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:23.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:23.722 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:07:23.823 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:23.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:23.859 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:23.859 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:23.960 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:23.991 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:23.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:23.992 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:24.093 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:24.132 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:24.133 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 61 16 81975 81959 83.9434 47.75 0.00822034 0.0119072 2024-09-20T19:07:24.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:24.133 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:24.234 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:24.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:24.281 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:24.281 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:24.382 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:07:24.413 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:24.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:24.414 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:07:24.514 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:07:24.547 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:24.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:24.548 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:07:24.649 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:07:24.681 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:24.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:24.682 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:07:24.783 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:24.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:24.815 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:24.815 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:24.916 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:24.959 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:24.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:24.960 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:07:25.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:24 smithi142 ceph-mon[32508]: pgmap v890: 158 pgs: 4 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 148 active+clean; 4.9 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 58 MiB/s wr, 937 op/s; 13750/240939 objects degraded (5.707%); 19035/240939 objects misplaced (7.900%); 20 MiB/s, 321 objects/s recovering 2024-09-20T19:07:25.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:24 smithi083 ceph-mon[29421]: pgmap v890: 158 pgs: 4 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 148 active+clean; 4.9 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 58 MiB/s wr, 937 op/s; 13750/240939 objects degraded (5.707%); 19035/240939 objects misplaced (7.900%); 20 MiB/s, 321 objects/s recovering 2024-09-20T19:07:25.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:24 smithi083 ceph-mon[36260]: pgmap v890: 158 pgs: 4 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 148 active+clean; 4.9 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 58 MiB/s wr, 937 op/s; 13750/240939 objects degraded (5.707%); 19035/240939 objects misplaced (7.900%); 20 MiB/s, 321 objects/s recovering 2024-09-20T19:07:25.061 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:25.092 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:25.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:25.093 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:25.133 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 62 15 82909 82894 83.532 58.4375 0.0101164 0.0119663 2024-09-20T19:07:25.193 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:25.214 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:25.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:25.215 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:25.316 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:25.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:25.350 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:25.350 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:25.451 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:25.482 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:25.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:25.483 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:25.584 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:25.622 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:25.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:25.623 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:25.724 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:25.765 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:25.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:25.767 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:25.832 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:07:25.867 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:25.927 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:25.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:25.928 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:25.960 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:07:26.029 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:07:26.055 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:26.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:26.057 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:07:26.133 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 63 16 83613 83597 82.9036 43.9375 0.00822411 0.0120573 2024-09-20T19:07:26.157 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:07:26.183 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:26.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:26.184 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:07:26.285 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:26.321 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:26.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:26.323 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:26.424 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:26.466 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:26.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:26.467 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:26.567 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:26.599 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:26.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:26.600 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:26.701 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:07:26.722 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:26.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:26.722 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:07:26.823 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:26.852 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:26.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:26.853 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:26.954 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:07:26.980 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:26.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:26.981 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:07:27.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:26 smithi142 ceph-mon[32508]: pgmap v891: 158 pgs: 4 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 148 active+clean; 4.9 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 852 B/s rd, 43 MiB/s wr, 691 op/s; 13750/240939 objects degraded (5.707%); 19035/240939 objects misplaced (7.900%); 15 MiB/s, 240 objects/s recovering 2024-09-20T19:07:27.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:26 smithi142 ceph-mon[32508]: osdmap e448: 8 total, 8 up, 8 in 2024-09-20T19:07:27.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:26 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 13750/240939 objects degraded (5.707%), 4 pgs degraded (PG_DEGRADED) 2024-09-20T19:07:27.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:26 smithi142 ceph-mon[32508]: osdmap e449: 8 total, 8 up, 8 in 2024-09-20T19:07:27.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:26 smithi083 ceph-mon[29421]: pgmap v891: 158 pgs: 4 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 148 active+clean; 4.9 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 852 B/s rd, 43 MiB/s wr, 691 op/s; 13750/240939 objects degraded (5.707%); 19035/240939 objects misplaced (7.900%); 15 MiB/s, 240 objects/s recovering 2024-09-20T19:07:27.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:26 smithi083 ceph-mon[29421]: osdmap e448: 8 total, 8 up, 8 in 2024-09-20T19:07:27.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:26 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 13750/240939 objects degraded (5.707%), 4 pgs degraded (PG_DEGRADED) 2024-09-20T19:07:27.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:26 smithi083 ceph-mon[29421]: osdmap e449: 8 total, 8 up, 8 in 2024-09-20T19:07:27.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:26 smithi083 ceph-mon[36260]: pgmap v891: 158 pgs: 4 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 148 active+clean; 4.9 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 852 B/s rd, 43 MiB/s wr, 691 op/s; 13750/240939 objects degraded (5.707%); 19035/240939 objects misplaced (7.900%); 15 MiB/s, 240 objects/s recovering 2024-09-20T19:07:27.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:26 smithi083 ceph-mon[36260]: osdmap e448: 8 total, 8 up, 8 in 2024-09-20T19:07:27.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:26 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 13750/240939 objects degraded (5.707%), 4 pgs degraded (PG_DEGRADED) 2024-09-20T19:07:27.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:26 smithi083 ceph-mon[36260]: osdmap e449: 8 total, 8 up, 8 in 2024-09-20T19:07:27.082 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:27.117 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:27.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:27.118 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:27.133 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 64 16 84516 84500 82.4901 56.4375 0.0209853 0.0121177 2024-09-20T19:07:27.219 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:07:27.247 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:27.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:27.248 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:07:27.349 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:27.382 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:27.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:27.383 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:27.484 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:27.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:27.507 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:27.507 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:27.608 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:27.636 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:27.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:27.637 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:27.738 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:07:27.759 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:27.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:27.760 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:07:27.760 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:07:27 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:07:27.860 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:27.862 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T19:07:27.914 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:27.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:27.915 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:07:28.016 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:28.072 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:28.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:28.074 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:28.133 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 65 16 85300 85284 81.975 49 0.0117097 0.012193 2024-09-20T19:07:28.175 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:28.196 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:28.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:28.197 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:28.297 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:28.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:28.331 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:28.331 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:28.432 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:28.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:28.470 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:28.470 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:28.571 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:28.600 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:28.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:28.601 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:28.702 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:28.732 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:28.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:28.733 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:28.833 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:28.866 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:28.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:28.867 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:07:28.968 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:29.002 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:29.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:29.003 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:29.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:28 smithi142 ceph-mon[32508]: pgmap v894: 158 pgs: 4 active+recovery_wait+undersized+degraded+remapped, 5 active+recovering+undersized+remapped, 149 active+clean; 5.1 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 53 MiB/s wr, 846 op/s; 13750/249219 objects degraded (5.517%); 17116/249219 objects misplaced (6.868%); 19 MiB/s, 308 objects/s recovering 2024-09-20T19:07:29.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:28 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:07:29.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:28 smithi083 ceph-mon[29421]: pgmap v894: 158 pgs: 4 active+recovery_wait+undersized+degraded+remapped, 5 active+recovering+undersized+remapped, 149 active+clean; 5.1 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 53 MiB/s wr, 846 op/s; 13750/249219 objects degraded (5.517%); 17116/249219 objects misplaced (6.868%); 19 MiB/s, 308 objects/s recovering 2024-09-20T19:07:29.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:28 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:07:29.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:28 smithi083 ceph-mon[36260]: pgmap v894: 158 pgs: 4 active+recovery_wait+undersized+degraded+remapped, 5 active+recovering+undersized+remapped, 149 active+clean; 5.1 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 53 MiB/s wr, 846 op/s; 13750/249219 objects degraded (5.517%); 17116/249219 objects misplaced (6.868%); 19 MiB/s, 308 objects/s recovering 2024-09-20T19:07:29.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:28 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:07:29.104 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:29.133 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 66 15 86038 86023 81.4328 46.1875 0.0130628 0.0122744 2024-09-20T19:07:29.144 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:29.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:29.145 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:29.245 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:29.276 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:29.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:29.277 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:07:29.377 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:29.420 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:29.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:29.421 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:29.522 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:29.565 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:29.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:29.566 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:29.667 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:29.710 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:29.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:29.711 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:07:29.812 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:29.850 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:29.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:29.851 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:29.952 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:29.985 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:29.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:29.986 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:30.086 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:07:30.113 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:30.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:30.114 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:07:30.136 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 67 16 86833 86817 80.9553 49.625 0.0200898 0.0123472 2024-09-20T19:07:30.215 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:30.235 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:30.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:30.236 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:30.337 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:07:30.367 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:30.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:30.368 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:07:30.469 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:30.512 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:30.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:30.514 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:30.614 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:30.645 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:30.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:30.646 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:30.746 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:07:30.775 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:30.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:30.776 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:07:30.877 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:07:30.916 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:30.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:30.917 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:07:31.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:30 smithi142 ceph-mon[32508]: pgmap v895: 158 pgs: 3 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 149 active+clean; 5.1 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 52 MiB/s wr, 837 op/s; 9918/253233 objects degraded (3.917%); 19900/253233 objects misplaced (7.858%); 19 MiB/s, 305 objects/s recovering 2024-09-20T19:07:31.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:30 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:07:31.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:30 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:07:31.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:30 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/4006639945' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:07:31.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:30 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:07:31.017 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:07:31.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:30 smithi083 ceph-mon[29421]: pgmap v895: 158 pgs: 3 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 149 active+clean; 5.1 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 52 MiB/s wr, 837 op/s; 9918/253233 objects degraded (3.917%); 19900/253233 objects misplaced (7.858%); 19 MiB/s, 305 objects/s recovering 2024-09-20T19:07:31.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:30 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:07:31.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:30 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:07:31.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:30 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/4006639945' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:07:31.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:30 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:07:31.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:30 smithi083 ceph-mon[36260]: pgmap v895: 158 pgs: 3 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 149 active+clean; 5.1 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 52 MiB/s wr, 837 op/s; 9918/253233 objects degraded (3.917%); 19900/253233 objects misplaced (7.858%); 19 MiB/s, 305 objects/s recovering 2024-09-20T19:07:31.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:30 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:07:31.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:30 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:07:31.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:30 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/4006639945' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:07:31.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:30 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:07:31.042 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:07:30 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:07:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:07:31.048 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:31.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:31.049 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:07:31.136 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 68 16 87549 87533 80.4229 44.75 0.0251297 0.0124295 2024-09-20T19:07:31.150 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:31.182 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:31.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:31.183 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:31.281 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-20T19:07:31.283 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:31.313 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:31.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:31.314 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:31.415 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:07:31.436 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:31.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:31.437 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:07:31.538 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:07:31.572 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:31.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:31.573 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:07:31.674 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:31.702 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:31.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:31.703 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:31.804 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:07:31.825 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:31.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:31.826 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:07:31.927 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:07:31.957 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:31.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:31.958 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:07:32.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:31 smithi142 ceph-mon[32508]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:07:32.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:31 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T19:07:32.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:31 smithi142 ceph-mon[32508]: osdmap e450: 8 total, 8 up, 8 in 2024-09-20T19:07:32.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:31 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 9918/253233 objects degraded (3.917%), 3 pgs degraded (PG_DEGRADED) 2024-09-20T19:07:32.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:31 smithi142 ceph-mon[32508]: osdmap e451: 8 total, 8 up, 8 in 2024-09-20T19:07:32.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:31 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:07:32.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:31 smithi083 ceph-mon[29421]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:07:32.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:31 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T19:07:32.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:31 smithi083 ceph-mon[29421]: osdmap e450: 8 total, 8 up, 8 in 2024-09-20T19:07:32.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:31 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 9918/253233 objects degraded (3.917%), 3 pgs degraded (PG_DEGRADED) 2024-09-20T19:07:32.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:31 smithi083 ceph-mon[29421]: osdmap e451: 8 total, 8 up, 8 in 2024-09-20T19:07:32.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:31 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:07:32.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:31 smithi083 ceph-mon[36260]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:07:32.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:31 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T19:07:32.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:31 smithi083 ceph-mon[36260]: osdmap e450: 8 total, 8 up, 8 in 2024-09-20T19:07:32.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:31 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 9918/253233 objects degraded (3.917%), 3 pgs degraded (PG_DEGRADED) 2024-09-20T19:07:32.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:31 smithi083 ceph-mon[36260]: osdmap e451: 8 total, 8 up, 8 in 2024-09-20T19:07:32.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:31 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:07:32.060 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:32.085 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:32.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:32.086 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:07:32.136 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 69 16 88122 88106 79.7765 35.8125 0.00509291 0.0124931 2024-09-20T19:07:32.187 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:32.216 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:32.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:32.217 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:32.318 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:32.346 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:32.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:32.347 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:32.447 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:07:32.483 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:32.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:32.484 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:07:32.585 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:32.615 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:32.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:32.616 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:32.717 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:07:32.745 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:32.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:32.746 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:07:32.846 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:07:32.867 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:32.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:32.868 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:07:32.969 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:32.998 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:33.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:33.000 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:33.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:32 smithi142 ceph-mon[32508]: pgmap v898: 158 pgs: 3 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 149 active+clean; 5.2 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 917 B/s rd, 61 MiB/s wr, 980 op/s; 9918/257325 objects degraded (3.854%); 18738/257325 objects misplaced (7.282%); 21 MiB/s, 329 objects/s recovering 2024-09-20T19:07:33.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:32 smithi083 ceph-mon[29421]: pgmap v898: 158 pgs: 3 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 149 active+clean; 5.2 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 917 B/s rd, 61 MiB/s wr, 980 op/s; 9918/257325 objects degraded (3.854%); 18738/257325 objects misplaced (7.282%); 21 MiB/s, 329 objects/s recovering 2024-09-20T19:07:33.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:32 smithi083 ceph-mon[36260]: pgmap v898: 158 pgs: 3 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 149 active+clean; 5.2 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 917 B/s rd, 61 MiB/s wr, 980 op/s; 9918/257325 objects degraded (3.854%); 18738/257325 objects misplaced (7.282%); 21 MiB/s, 329 objects/s recovering 2024-09-20T19:07:33.101 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:07:33.122 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:33.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:33.123 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:07:33.136 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 70 16 88889 88873 79.3217 47.9375 0.00630186 0.0125854 2024-09-20T19:07:33.223 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:07:33.258 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:33.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:33.259 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:07:33.360 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:33.402 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:33.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:33.403 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:33.504 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:33.553 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:33.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:33.554 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:07:33.655 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:33.683 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:33.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:33.684 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:33.784 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:33.827 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:33.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:33.828 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:33.929 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:33.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:33.968 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:33.968 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:34.069 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:34.112 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:34.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:34.113 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:07:34.136 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 71 16 89825 89809 79.0285 58.5 0.0104442 0.0126485 2024-09-20T19:07:34.214 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:34.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:34.237 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:34.237 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:34.338 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:34.382 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:34.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:34.384 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:34.485 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:34.513 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:34.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:34.514 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:34.615 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:34.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:34.644 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:34.644 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:34.745 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:07:34.776 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:34.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:34.777 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:07:34.878 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:34.904 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:34.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:34.905 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:35.006 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:07:35.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:34 smithi142 ceph-mon[32508]: pgmap v899: 158 pgs: 3 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 149 active+clean; 5.4 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.4 KiB/s rd, 64 MiB/s wr, 1.02k op/s; 9918/264822 objects degraded (3.745%); 16843/264822 objects misplaced (6.360%); 21 MiB/s, 331 objects/s recovering 2024-09-20T19:07:35.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:35.039 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:35.039 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:07:35.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:34 smithi083 ceph-mon[29421]: pgmap v899: 158 pgs: 3 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 149 active+clean; 5.4 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.4 KiB/s rd, 64 MiB/s wr, 1.02k op/s; 9918/264822 objects degraded (3.745%); 16843/264822 objects misplaced (6.360%); 21 MiB/s, 331 objects/s recovering 2024-09-20T19:07:35.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:34 smithi083 ceph-mon[36260]: pgmap v899: 158 pgs: 3 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 149 active+clean; 5.4 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.4 KiB/s rd, 64 MiB/s wr, 1.02k op/s; 9918/264822 objects degraded (3.745%); 16843/264822 objects misplaced (6.360%); 21 MiB/s, 331 objects/s recovering 2024-09-20T19:07:35.136 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 72 16 90631 90615 78.6306 50.375 0.00327349 0.0127122 2024-09-20T19:07:35.140 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:35.178 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:35.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:35.179 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:07:35.280 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:35.314 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:35.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:35.315 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:35.416 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:35.453 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:35.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:35.454 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:35.555 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:35.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:35.590 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:35.590 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:35.691 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:35.731 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:35.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:35.732 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:35.833 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:35.875 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:07:35.906 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:35.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:35.915 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:36.015 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:36.045 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:36.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:36.047 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:36.136 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 73 16 91416 91400 78.2256 49.0625 0.0230855 0.0127783 2024-09-20T19:07:36.147 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:07:36.179 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:36.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:36.180 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:07:36.281 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:36.307 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:36.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:36.308 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:36.409 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:07:36.438 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:36.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:36.439 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:07:36.540 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:36.578 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:36.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:36.579 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:07:36.680 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:36.708 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:36.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:36.710 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:36.811 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:36.849 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:36.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:36.850 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:36.951 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:36.972 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:36 smithi083 ceph-mon[29421]: pgmap v900: 158 pgs: 3 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 149 active+clean; 5.4 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 639 B/s rd, 41 MiB/s wr, 650 op/s; 9918/264822 objects degraded (3.745%); 16843/264822 objects misplaced (6.360%); 13 MiB/s, 209 objects/s recovering 2024-09-20T19:07:36.972 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:36 smithi083 ceph-mon[29421]: osdmap e452: 8 total, 8 up, 8 in 2024-09-20T19:07:36.972 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:36 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 9918/264822 objects degraded (3.745%), 3 pgs degraded (PG_DEGRADED) 2024-09-20T19:07:36.972 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:36 smithi083 ceph-mon[29421]: osdmap e453: 8 total, 8 up, 8 in 2024-09-20T19:07:36.972 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:36 smithi083 ceph-mon[36260]: pgmap v900: 158 pgs: 3 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 149 active+clean; 5.4 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 639 B/s rd, 41 MiB/s wr, 650 op/s; 9918/264822 objects degraded (3.745%); 16843/264822 objects misplaced (6.360%); 13 MiB/s, 209 objects/s recovering 2024-09-20T19:07:36.973 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:36 smithi083 ceph-mon[36260]: osdmap e452: 8 total, 8 up, 8 in 2024-09-20T19:07:36.973 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:36 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 9918/264822 objects degraded (3.745%), 3 pgs degraded (PG_DEGRADED) 2024-09-20T19:07:36.973 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:36 smithi083 ceph-mon[36260]: osdmap e453: 8 total, 8 up, 8 in 2024-09-20T19:07:36.986 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:36.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:36.987 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:37.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:36 smithi142 ceph-mon[32508]: pgmap v900: 158 pgs: 3 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 149 active+clean; 5.4 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 639 B/s rd, 41 MiB/s wr, 650 op/s; 9918/264822 objects degraded (3.745%); 16843/264822 objects misplaced (6.360%); 13 MiB/s, 209 objects/s recovering 2024-09-20T19:07:37.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:36 smithi142 ceph-mon[32508]: osdmap e452: 8 total, 8 up, 8 in 2024-09-20T19:07:37.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:36 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 9918/264822 objects degraded (3.745%), 3 pgs degraded (PG_DEGRADED) 2024-09-20T19:07:37.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:36 smithi142 ceph-mon[32508]: osdmap e453: 8 total, 8 up, 8 in 2024-09-20T19:07:37.089 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:07:37.119 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:37.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:37.120 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:07:37.136 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 74 16 92177 92161 77.8113 47.5625 0.0343278 0.0128455 2024-09-20T19:07:37.220 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:07:37.243 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:37.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:37.244 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:07:37.344 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:07:37.372 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:37.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:37.373 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:07:37.474 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:37.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:37.507 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:37.507 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:37.608 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:37.653 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:37.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:37.654 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:37.755 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:37.760 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:07:37 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:07:37.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:37 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/96669752' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"} : dispatch 2024-09-20T19:07:37.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:37 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"} : dispatch 2024-09-20T19:07:37.785 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:37.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:37.786 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:37.887 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:37.915 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:37.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:37.916 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:38.017 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:07:38.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:37 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/96669752' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"} : dispatch 2024-09-20T19:07:38.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:37 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"} : dispatch 2024-09-20T19:07:38.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:37 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/96669752' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"} : dispatch 2024-09-20T19:07:38.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:37 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"} : dispatch 2024-09-20T19:07:38.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:38.050 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:38.050 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:07:38.136 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 75 16 92864 92848 77.3464 42.9375 0.0180712 0.012923 2024-09-20T19:07:38.151 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:38.177 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:38.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:38.178 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:38.223 INFO:teuthology.orchestra.run.smithi083.stderr:set pool 1 pg_num to 11 2024-09-20T19:07:38.279 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:38.300 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:38.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:38.301 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:38.402 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:38.430 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:38.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:38.431 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:07:38.532 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:38.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:38.563 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:38.563 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:38.664 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:38.684 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:38.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:38.685 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:38.786 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:38.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:38 smithi083 ceph-mon[29421]: pgmap v903: 158 pgs: 3 active+recovery_wait+undersized+degraded+remapped, 5 active+recovering+undersized+remapped, 150 active+clean; 5.5 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.6 KiB/s rd, 60 MiB/s wr, 964 op/s; 9918/272799 objects degraded (3.636%); 15023/272799 objects misplaced (5.507%); 18 MiB/s, 287 objects/s recovering 2024-09-20T19:07:38.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:38 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:07:38.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:38 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"}]': finished 2024-09-20T19:07:38.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:38 smithi083 ceph-mon[29421]: osdmap e454: 8 total, 8 up, 8 in 2024-09-20T19:07:38.791 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:38 smithi083 ceph-mon[36260]: pgmap v903: 158 pgs: 3 active+recovery_wait+undersized+degraded+remapped, 5 active+recovering+undersized+remapped, 150 active+clean; 5.5 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.6 KiB/s rd, 60 MiB/s wr, 964 op/s; 9918/272799 objects degraded (3.636%); 15023/272799 objects misplaced (5.507%); 18 MiB/s, 287 objects/s recovering 2024-09-20T19:07:38.791 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:38 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:07:38.791 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:38 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"}]': finished 2024-09-20T19:07:38.791 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:38 smithi083 ceph-mon[36260]: osdmap e454: 8 total, 8 up, 8 in 2024-09-20T19:07:38.826 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:38.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:38.827 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:38.927 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:38.977 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:38.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:38.978 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:39.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:38 smithi142 ceph-mon[32508]: pgmap v903: 158 pgs: 3 active+recovery_wait+undersized+degraded+remapped, 5 active+recovering+undersized+remapped, 150 active+clean; 5.5 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.6 KiB/s rd, 60 MiB/s wr, 964 op/s; 9918/272799 objects degraded (3.636%); 15023/272799 objects misplaced (5.507%); 18 MiB/s, 287 objects/s recovering 2024-09-20T19:07:39.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:38 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:07:39.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:38 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"}]': finished 2024-09-20T19:07:39.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:38 smithi142 ceph-mon[32508]: osdmap e454: 8 total, 8 up, 8 in 2024-09-20T19:07:39.078 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:39.126 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:39.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:39.127 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:07:39.136 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 76 16 93393 93377 76.7638 33.0625 0.0454917 0.0130214 2024-09-20T19:07:39.229 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:39.257 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:39.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:39.258 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:39.359 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:39.402 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:39.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:39.403 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:39.504 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:39.525 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:39.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:39.526 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:39.627 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:39.650 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:39.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:39.651 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:39.751 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:39.789 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:39.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:39.790 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:39.890 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:39.916 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:39.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:39.917 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:40.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:39 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"} : dispatch 2024-09-20T19:07:40.018 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:40.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:39 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"} : dispatch 2024-09-20T19:07:40.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:39 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"} : dispatch 2024-09-20T19:07:40.057 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:40.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:40.058 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:40.136 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 77 16 93992 93976 76.2531 37.4375 0.00542154 0.0130795 2024-09-20T19:07:40.160 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:40.195 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:40.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:40.196 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:40.297 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:40.321 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:40.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:40.322 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:40.423 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:40.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:40.465 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:40.465 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:40.566 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:40.601 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:40.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:40.603 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:40.703 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:40.744 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:40.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:40.745 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:40.846 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:40.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:40.891 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:40.892 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:40.992 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:07:41.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:40 smithi142 ceph-mon[32508]: pgmap v905: 158 pgs: 2 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 150 active+clean; 5.6 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 853 B/s rd, 41 MiB/s wr, 650 op/s; 5826/276516 objects degraded (2.107%); 17627/276516 objects misplaced (6.375%); 14 MiB/s, 225 objects/s recovering 2024-09-20T19:07:41.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:40 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"}]': finished 2024-09-20T19:07:41.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:40 smithi142 ceph-mon[32508]: osdmap e455: 8 total, 8 up, 8 in 2024-09-20T19:07:41.021 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:41.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:41.023 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:07:41.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:40 smithi083 ceph-mon[29421]: pgmap v905: 158 pgs: 2 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 150 active+clean; 5.6 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 853 B/s rd, 41 MiB/s wr, 650 op/s; 5826/276516 objects degraded (2.107%); 17627/276516 objects misplaced (6.375%); 14 MiB/s, 225 objects/s recovering 2024-09-20T19:07:41.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:40 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"}]': finished 2024-09-20T19:07:41.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:40 smithi083 ceph-mon[29421]: osdmap e455: 8 total, 8 up, 8 in 2024-09-20T19:07:41.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:40 smithi083 ceph-mon[36260]: pgmap v905: 158 pgs: 2 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 150 active+clean; 5.6 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 853 B/s rd, 41 MiB/s wr, 650 op/s; 5826/276516 objects degraded (2.107%); 17627/276516 objects misplaced (6.375%); 14 MiB/s, 225 objects/s recovering 2024-09-20T19:07:41.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:40 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"}]': finished 2024-09-20T19:07:41.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:40 smithi083 ceph-mon[36260]: osdmap e455: 8 total, 8 up, 8 in 2024-09-20T19:07:41.042 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:07:40 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:07:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:07:41.124 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:41.136 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 78 16 94476 94460 75.6635 30.25 0.00956536 0.013211 2024-09-20T19:07:41.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:41.164 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:41.164 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:41.265 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:07:41.297 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:41.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:41.298 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:07:41.399 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:41.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:41.422 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:41.423 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:41.523 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:41.554 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:41.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:41.556 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:41.656 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:41.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:41.678 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:41.678 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:07:41.779 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:41.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:41.816 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:41.816 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:41.917 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:41.945 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:41.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:41.946 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:42.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:41 smithi142 ceph-mon[32508]: osdmap e456: 8 total, 8 up, 8 in 2024-09-20T19:07:42.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:41 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 5826/276516 objects degraded (2.107%), 2 pgs degraded (PG_DEGRADED) 2024-09-20T19:07:42.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:41 smithi142 ceph-mon[32508]: osdmap e457: 8 total, 8 up, 8 in 2024-09-20T19:07:42.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:41 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"} : dispatch 2024-09-20T19:07:42.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:41 smithi083 ceph-mon[29421]: osdmap e456: 8 total, 8 up, 8 in 2024-09-20T19:07:42.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:41 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 5826/276516 objects degraded (2.107%), 2 pgs degraded (PG_DEGRADED) 2024-09-20T19:07:42.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:41 smithi083 ceph-mon[29421]: osdmap e457: 8 total, 8 up, 8 in 2024-09-20T19:07:42.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:41 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"} : dispatch 2024-09-20T19:07:42.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:41 smithi083 ceph-mon[36260]: osdmap e456: 8 total, 8 up, 8 in 2024-09-20T19:07:42.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:41 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 5826/276516 objects degraded (2.107%), 2 pgs degraded (PG_DEGRADED) 2024-09-20T19:07:42.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:41 smithi083 ceph-mon[36260]: osdmap e457: 8 total, 8 up, 8 in 2024-09-20T19:07:42.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:41 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"} : dispatch 2024-09-20T19:07:42.047 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:07:42.072 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:42.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:42.073 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:07:42.136 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 79 16 95155 95139 75.2429 42.4375 0.00564158 0.0132812 2024-09-20T19:07:42.174 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:42.196 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:42.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:42.198 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:42.298 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:42.324 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:42.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:42.325 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:42.426 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:07:42.448 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:42.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:42.449 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:07:42.550 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:42.582 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:42.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:42.583 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:42.684 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:07:42.708 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:42.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:42.782 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:07:42.883 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:42.911 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:42.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:42.912 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:43.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:42 smithi142 ceph-mon[32508]: pgmap v909: 168 pgs: 1 active+undersized+remapped, 10 unknown, 2 active+recovery_wait+undersized+degraded+remapped, 5 active+recovering+undersized+remapped, 150 active+clean; 5.7 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 34 MiB/s wr, 547 op/s; 5826/279372 objects degraded (2.085%); 16921/279372 objects misplaced (6.057%); 14 MiB/s, 231 objects/s recovering 2024-09-20T19:07:43.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:42 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"}]': finished 2024-09-20T19:07:43.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:42 smithi142 ceph-mon[32508]: osdmap e458: 8 total, 8 up, 8 in 2024-09-20T19:07:43.012 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:43.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:42 smithi083 ceph-mon[29421]: pgmap v909: 168 pgs: 1 active+undersized+remapped, 10 unknown, 2 active+recovery_wait+undersized+degraded+remapped, 5 active+recovering+undersized+remapped, 150 active+clean; 5.7 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 34 MiB/s wr, 547 op/s; 5826/279372 objects degraded (2.085%); 16921/279372 objects misplaced (6.057%); 14 MiB/s, 231 objects/s recovering 2024-09-20T19:07:43.042 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:42 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"}]': finished 2024-09-20T19:07:43.042 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:42 smithi083 ceph-mon[29421]: osdmap e458: 8 total, 8 up, 8 in 2024-09-20T19:07:43.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:42 smithi083 ceph-mon[36260]: pgmap v909: 168 pgs: 1 active+undersized+remapped, 10 unknown, 2 active+recovery_wait+undersized+degraded+remapped, 5 active+recovering+undersized+remapped, 150 active+clean; 5.7 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 34 MiB/s wr, 547 op/s; 5826/279372 objects degraded (2.085%); 16921/279372 objects misplaced (6.057%); 14 MiB/s, 231 objects/s recovering 2024-09-20T19:07:43.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:42 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"}]': finished 2024-09-20T19:07:43.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:42 smithi083 ceph-mon[36260]: osdmap e458: 8 total, 8 up, 8 in 2024-09-20T19:07:43.051 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:43.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:43.052 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:43.137 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:2024-09-20T19:07:43.133866+0000 min lat: 0.00109192 max lat: 2.36333 avg lat: 0.0134008 2024-09-20T19:07:43.137 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-20T19:07:43.137 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 80 16 95499 95483 74.5713 21.5 0.00799369 0.0134008 2024-09-20T19:07:43.153 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:43.193 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:43.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:43.194 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:07:43.295 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:43.337 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:43.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:43.338 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:43.439 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:43.478 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:43.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:43.479 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:43.579 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:07:43.603 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:43.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:43.604 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:07:43.704 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:07:43.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:43.728 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:43.728 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:07:43.829 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:43.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:43.862 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:43.862 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:43.963 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:43.989 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:43.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:43.989 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:07:44.090 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:07:44.118 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:44.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:44.119 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:07:44.137 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 81 16 96036 96020 74.0651 33.5625 0.027018 0.0134962 2024-09-20T19:07:44.219 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:07:44.243 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:44.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:44.244 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:07:44.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:43 smithi142 ceph-mon[32508]: osdmap e459: 8 total, 8 up, 8 in 2024-09-20T19:07:44.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:43 smithi142 ceph-mon[32508]: pgmap v912: 168 pgs: 1 peering, 1 active+undersized+remapped, 6 active+recovering+undersized+remapped, 160 active+clean; 5.8 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.1 KiB/s rd, 54 MiB/s wr, 868 op/s; 18694/285609 objects misplaced (6.545%); 34 MiB/s, 549 objects/s recovering 2024-09-20T19:07:44.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:43 smithi083 ceph-mon[29421]: osdmap e459: 8 total, 8 up, 8 in 2024-09-20T19:07:44.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:43 smithi083 ceph-mon[29421]: pgmap v912: 168 pgs: 1 peering, 1 active+undersized+remapped, 6 active+recovering+undersized+remapped, 160 active+clean; 5.8 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.1 KiB/s rd, 54 MiB/s wr, 868 op/s; 18694/285609 objects misplaced (6.545%); 34 MiB/s, 549 objects/s recovering 2024-09-20T19:07:44.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:43 smithi083 ceph-mon[36260]: osdmap e459: 8 total, 8 up, 8 in 2024-09-20T19:07:44.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:43 smithi083 ceph-mon[36260]: pgmap v912: 168 pgs: 1 peering, 1 active+undersized+remapped, 6 active+recovering+undersized+remapped, 160 active+clean; 5.8 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.1 KiB/s rd, 54 MiB/s wr, 868 op/s; 18694/285609 objects misplaced (6.545%); 34 MiB/s, 549 objects/s recovering 2024-09-20T19:07:44.345 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:44.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:44.384 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:44.384 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:44.485 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:07:44.507 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:44.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:44.507 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:07:44.608 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:07:44.637 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:44.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:44.638 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:07:44.739 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:07:44.767 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:44.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:44.768 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:07:44.868 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:44.889 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:44.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:44.890 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:44.991 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:07:45.016 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:45.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:45.017 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:07:45.117 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:45.137 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 82 16 96882 96866 73.8065 52.875 0.0239406 0.0135433 2024-09-20T19:07:45.154 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:45.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:45.155 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:45.256 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:07:45.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:44 smithi142 ceph-mon[32508]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-20T19:07:45.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:44 smithi142 ceph-mon[32508]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 5826/279372 objects degraded (2.085%), 2 pgs degraded) 2024-09-20T19:07:45.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:44 smithi142 ceph-mon[32508]: osdmap e460: 8 total, 8 up, 8 in 2024-09-20T19:07:45.285 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:45.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:45.286 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:07:45.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:44 smithi083 ceph-mon[29421]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-20T19:07:45.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:44 smithi083 ceph-mon[29421]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 5826/279372 objects degraded (2.085%), 2 pgs degraded) 2024-09-20T19:07:45.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:44 smithi083 ceph-mon[29421]: osdmap e460: 8 total, 8 up, 8 in 2024-09-20T19:07:45.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:44 smithi083 ceph-mon[36260]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-20T19:07:45.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:44 smithi083 ceph-mon[36260]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 5826/279372 objects degraded (2.085%), 2 pgs degraded) 2024-09-20T19:07:45.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:44 smithi083 ceph-mon[36260]: osdmap e460: 8 total, 8 up, 8 in 2024-09-20T19:07:45.386 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:45.413 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:45.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:45.414 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:45.514 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:45.558 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:45.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:45.559 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:45.660 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:07:45.683 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:45.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:45.684 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:07:45.785 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:45.831 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:45.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:45.832 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:45.933 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:45.975 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:45.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:45.976 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:46.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:45 smithi083 ceph-mon[29421]: pgmap v914: 168 pgs: 1 peering, 1 active+undersized+remapped, 6 active+recovering+undersized+remapped, 160 active+clean; 5.8 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 962 B/s rd, 40 MiB/s wr, 648 op/s; 18694/285609 objects misplaced (6.545%); 28 MiB/s, 450 objects/s recovering 2024-09-20T19:07:46.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:45 smithi083 ceph-mon[29421]: osdmap e461: 8 total, 8 up, 8 in 2024-09-20T19:07:46.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:45 smithi083 ceph-mon[36260]: pgmap v914: 168 pgs: 1 peering, 1 active+undersized+remapped, 6 active+recovering+undersized+remapped, 160 active+clean; 5.8 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 962 B/s rd, 40 MiB/s wr, 648 op/s; 18694/285609 objects misplaced (6.545%); 28 MiB/s, 450 objects/s recovering 2024-09-20T19:07:46.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:45 smithi083 ceph-mon[36260]: osdmap e461: 8 total, 8 up, 8 in 2024-09-20T19:07:46.077 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:46.104 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:46.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:46.106 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:46.137 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 83 16 97716 97700 73.5453 52.125 0.0129924 0.0135913 2024-09-20T19:07:46.207 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:07:46.238 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:46.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:46.239 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:07:46.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:45 smithi142 ceph-mon[32508]: pgmap v914: 168 pgs: 1 peering, 1 active+undersized+remapped, 6 active+recovering+undersized+remapped, 160 active+clean; 5.8 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 962 B/s rd, 40 MiB/s wr, 648 op/s; 18694/285609 objects misplaced (6.545%); 28 MiB/s, 450 objects/s recovering 2024-09-20T19:07:46.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:45 smithi142 ceph-mon[32508]: osdmap e461: 8 total, 8 up, 8 in 2024-09-20T19:07:46.341 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:46.371 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:46.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:46.372 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:46.473 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:46.515 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:46.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:46.516 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:07:46.617 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:07:46.642 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:46.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:46.643 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:07:46.743 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:46.790 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:46.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:46.791 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:46.892 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:07:46.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:46.923 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:46.923 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:07:47.024 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:47.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:47.070 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:47.070 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:47.137 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 84 16 98697 98681 73.3996 61.3125 0.00826481 0.013618 2024-09-20T19:07:47.171 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:47.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:47.217 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:47.217 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:47.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:46 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:07:47.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:46 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:07:47.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:46 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:07:47.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:46 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:07:47.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:46 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:07:47.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:46 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:07:47.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:46 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:07:47.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:46 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:07:47.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:46 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:07:47.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:46 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:07:47.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:46 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:07:47.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:46 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:07:47.318 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:47.345 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:47.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:47.346 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:47.447 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:47.485 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:47.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:47.486 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:47.587 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:47.641 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:07:47.660 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:47.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:47.661 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:47.760 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:07:47 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:07:47.762 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:07:47.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:47.785 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:47.785 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:07:47.886 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:47.922 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:47.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:47.923 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:07:48.024 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:48.067 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:48.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:48.068 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:07:48.137 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 85 16 99556 99540 73.1677 53.6875 0.0116934 0.0136621 2024-09-20T19:07:48.170 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:48.190 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:48.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:48.191 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:48.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:47 smithi142 ceph-mon[32508]: pgmap v916: 168 pgs: 2 active+recovery_wait+degraded, 1 peering, 5 active+recovering+undersized+remapped, 160 active+clean; 5.9 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 42 MiB/s wr, 672 op/s; 5/292650 objects degraded (0.002%); 18658/292650 objects misplaced (6.376%); 16 MiB/s, 249 objects/s recovering 2024-09-20T19:07:48.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:47 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:07:48.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:47 smithi083 ceph-mon[29421]: pgmap v916: 168 pgs: 2 active+recovery_wait+degraded, 1 peering, 5 active+recovering+undersized+remapped, 160 active+clean; 5.9 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 42 MiB/s wr, 672 op/s; 5/292650 objects degraded (0.002%); 18658/292650 objects misplaced (6.376%); 16 MiB/s, 249 objects/s recovering 2024-09-20T19:07:48.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:47 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:07:48.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:47 smithi083 ceph-mon[36260]: pgmap v916: 168 pgs: 2 active+recovery_wait+degraded, 1 peering, 5 active+recovering+undersized+remapped, 160 active+clean; 5.9 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 42 MiB/s wr, 672 op/s; 5/292650 objects degraded (0.002%); 18658/292650 objects misplaced (6.376%); 16 MiB/s, 249 objects/s recovering 2024-09-20T19:07:48.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:47 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:07:48.292 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:48.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:48.332 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:48.332 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:48.433 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:48.464 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:48.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:48.465 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:48.566 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:07:48.593 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:48.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:48.594 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:07:48.695 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:48.737 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:48.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:48.738 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:48.839 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:48.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:48.877 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:48.878 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:48.978 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:49.014 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:49.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:49.015 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:07:49.116 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:49.138 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 86 16 100488 100472 72.9937 58.25 0.0100192 0.0136941 2024-09-20T19:07:49.151 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:49.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:49.153 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:49.253 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:49.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:49 smithi142 ceph-mon[32508]: Health check failed: Degraded data redundancy: 5/292650 objects degraded (0.002%), 2 pgs degraded (PG_DEGRADED) 2024-09-20T19:07:49.275 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:49.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:49.275 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:49.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:49 smithi083 ceph-mon[29421]: Health check failed: Degraded data redundancy: 5/292650 objects degraded (0.002%), 2 pgs degraded (PG_DEGRADED) 2024-09-20T19:07:49.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:49 smithi083 ceph-mon[36260]: Health check failed: Degraded data redundancy: 5/292650 objects degraded (0.002%), 2 pgs degraded (PG_DEGRADED) 2024-09-20T19:07:49.376 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:49.405 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:49.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:49.406 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:49.507 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:49.529 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:49.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:49.530 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:49.618 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T19:07:49.631 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:49.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:49.704 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:49.705 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:49.805 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:07:49.827 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:49.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:49.827 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:07:49.928 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:49.952 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:49.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:49.954 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:50.054 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:07:50.080 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:50.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:50.081 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:07:50.138 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 87 16 101116 101100 72.6058 39.25 0.00315846 0.0137652 2024-09-20T19:07:50.182 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:07:50.208 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:50.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:50.209 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:07:50.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:50 smithi083 ceph-mon[29421]: osdmap e462: 8 total, 8 up, 8 in 2024-09-20T19:07:50.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:50 smithi083 ceph-mon[29421]: pgmap v918: 168 pgs: 2 active+recovery_wait+degraded, 5 active+recovering+undersized+remapped, 161 active+clean; 6.0 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 36 MiB/s wr, 572 op/s; 5/295914 objects degraded (0.002%); 16032/295914 objects misplaced (5.418%); 10 MiB/s, 163 objects/s recovering 2024-09-20T19:07:50.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:50 smithi083 ceph-mon[36260]: osdmap e462: 8 total, 8 up, 8 in 2024-09-20T19:07:50.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:50 smithi083 ceph-mon[36260]: pgmap v918: 168 pgs: 2 active+recovery_wait+degraded, 5 active+recovering+undersized+remapped, 161 active+clean; 6.0 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 36 MiB/s wr, 572 op/s; 5/295914 objects degraded (0.002%); 16032/295914 objects misplaced (5.418%); 10 MiB/s, 163 objects/s recovering 2024-09-20T19:07:50.310 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:07:50.337 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:50.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:50.339 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:07:50.439 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:50.482 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:50.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:50.483 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:50.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:50 smithi142 ceph-mon[32508]: osdmap e462: 8 total, 8 up, 8 in 2024-09-20T19:07:50.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:50 smithi142 ceph-mon[32508]: pgmap v918: 168 pgs: 2 active+recovery_wait+degraded, 5 active+recovering+undersized+remapped, 161 active+clean; 6.0 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 36 MiB/s wr, 572 op/s; 5/295914 objects degraded (0.002%); 16032/295914 objects misplaced (5.418%); 10 MiB/s, 163 objects/s recovering 2024-09-20T19:07:50.584 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:07:50.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:50.614 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:50.614 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:07:50.715 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:50.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:50.757 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:50.757 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:07:50.858 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:50.879 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:50.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:50.881 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:50.982 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:51.020 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:07:50 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:07:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:07:51.023 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:51.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:51.024 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:51.125 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:51.138 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 88 16 102006 101990 72.4129 55.625 0.0249952 0.0138042 2024-09-20T19:07:51.151 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:51.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:51.152 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:51.253 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:51.282 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:51.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:51.283 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:51.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:51 smithi083 ceph-mon[29421]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-20T19:07:51.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:51 smithi083 ceph-mon[29421]: osdmap e463: 8 total, 8 up, 8 in 2024-09-20T19:07:51.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:51 smithi083 ceph-mon[36260]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-20T19:07:51.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:51 smithi083 ceph-mon[36260]: osdmap e463: 8 total, 8 up, 8 in 2024-09-20T19:07:51.383 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:07:51.411 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:51.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:51.412 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:07:51.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:51 smithi142 ceph-mon[32508]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-20T19:07:51.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:51 smithi142 ceph-mon[32508]: osdmap e463: 8 total, 8 up, 8 in 2024-09-20T19:07:51.513 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:51.552 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:51.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:51.554 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:07:51.654 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:07:51.675 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:51.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:51.676 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:07:51.776 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:51.812 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:51.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:51.813 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:51.914 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:51.946 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:51.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:51.947 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:52.048 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:07:52.075 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:52.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:52.076 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:07:52.138 INFO:tasks.radosbench.radosbench.0.smithi083.stdout: 89 16 102975 102959 72.2797 60.5625 0.0174112 0.0138295 2024-09-20T19:07:52.176 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:07:52.205 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:52.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:52.207 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:07:52.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:52 smithi083 ceph-mon[29421]: pgmap v920: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 6.1 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 49 MiB/s wr, 783 op/s; 3/299724 objects degraded (0.001%); 15925/299724 objects misplaced (5.313%); 11 MiB/s, 169 objects/s recovering 2024-09-20T19:07:52.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:52 smithi083 ceph-mon[36260]: pgmap v920: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 6.1 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 49 MiB/s wr, 783 op/s; 3/299724 objects degraded (0.001%); 15925/299724 objects misplaced (5.313%); 11 MiB/s, 169 objects/s recovering 2024-09-20T19:07:52.307 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:52.335 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:52.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:52.336 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:52.437 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:52.458 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:52.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:52.459 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:52.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:52 smithi142 ceph-mon[32508]: pgmap v920: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 6.1 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 49 MiB/s wr, 783 op/s; 3/299724 objects degraded (0.001%); 15925/299724 objects misplaced (5.313%); 11 MiB/s, 169 objects/s recovering 2024-09-20T19:07:52.560 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:52.580 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:52.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:52.581 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:52.682 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:52.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:52.717 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:52.717 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:52.818 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:52.853 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:52.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:52.854 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:07:52.955 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:52.976 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:52.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:52.977 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:53.078 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:53.118 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:53.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:53.119 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:53.138 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Total time run: 90.0161 2024-09-20T19:07:53.138 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Total writes made: 103439 2024-09-20T19:07:53.138 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Write size: 65536 2024-09-20T19:07:53.138 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Object size: 65536 2024-09-20T19:07:53.138 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Bandwidth (MB/sec): 71.8198 2024-09-20T19:07:53.139 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Stddev Bandwidth: 40.7477 2024-09-20T19:07:53.139 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Max bandwidth (MB/sec): 193.125 2024-09-20T19:07:53.139 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Min bandwidth (MB/sec): 0 2024-09-20T19:07:53.139 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Average IOPS: 1149 2024-09-20T19:07:53.139 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Stddev IOPS: 651.963 2024-09-20T19:07:53.139 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Max IOPS: 3090 2024-09-20T19:07:53.139 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Min IOPS: 0 2024-09-20T19:07:53.139 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Average Latency(s): 0.0139184 2024-09-20T19:07:53.139 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Stddev Latency(s): 0.0389355 2024-09-20T19:07:53.139 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Max latency(s): 2.36333 2024-09-20T19:07:53.139 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Min latency(s): 0.00109192 2024-09-20T19:07:53.143 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Cleaning up (deleting benchmark objects) 2024-09-20T19:07:53.220 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:53.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:53.247 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:53.248 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:53.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:53 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:07:53.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:53 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:07:53.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:53 smithi083 ceph-mon[29421]: osdmap e464: 8 total, 8 up, 8 in 2024-09-20T19:07:53.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:53 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:07:53.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:53 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:07:53.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:53 smithi083 ceph-mon[36260]: osdmap e464: 8 total, 8 up, 8 in 2024-09-20T19:07:53.349 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:53.389 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:53.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:53.390 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:53.491 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:53.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:53 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:07:53.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:53 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:07:53.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:53 smithi142 ceph-mon[32508]: osdmap e464: 8 total, 8 up, 8 in 2024-09-20T19:07:53.530 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:53.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:53.531 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:53.632 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:53.665 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:53.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:53.667 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:53.767 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:53.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:53.809 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:53.810 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:07:53.910 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:53.949 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:53.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:53.950 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:54.051 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:54.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:54.084 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:54.084 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:07:54.184 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:54.205 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:54.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:54.206 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:07:54.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:54 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:07:54.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:54 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:07:54.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:54 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:07:54.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:54 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:07:54.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:54 smithi083 ceph-mon[29421]: pgmap v922: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 6.3 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 853 B/s rd, 55 MiB/s wr, 883 op/s; 3/308544 objects degraded (0.001%); 14595/308544 objects misplaced (4.730%); 16 MiB/s, 255 objects/s recovering 2024-09-20T19:07:54.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:54 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:07:54.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:54 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:07:54.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:54 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:07:54.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:54 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:07:54.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:54 smithi083 ceph-mon[36260]: pgmap v922: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 6.3 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 853 B/s rd, 55 MiB/s wr, 883 op/s; 3/308544 objects degraded (0.001%); 14595/308544 objects misplaced (4.730%); 16 MiB/s, 255 objects/s recovering 2024-09-20T19:07:54.307 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:54.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:54.337 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:54.337 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:54.411 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:54 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:07:54.411 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:54 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:07:54.411 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:54 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:07:54.411 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:54 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:07:54.411 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:54 smithi142 ceph-mon[32508]: pgmap v922: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 6.3 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 853 B/s rd, 55 MiB/s wr, 883 op/s; 3/308544 objects degraded (0.001%); 14595/308544 objects misplaced (4.730%); 16 MiB/s, 255 objects/s recovering 2024-09-20T19:07:54.438 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:54.466 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:54.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:54.468 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:07:54.568 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:54.595 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:54.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:54.595 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:54.696 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:54.718 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:54.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:54.719 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:54.819 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:54.865 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:54.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:54.866 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:07:54.966 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:54.989 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:54.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:54.990 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:55.091 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:07:55.119 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:55.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:55.120 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:07:55.221 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:55.250 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:55.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:55.251 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:55.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:55 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 3/308544 objects degraded (0.001%), 1 pg degraded (PG_DEGRADED) 2024-09-20T19:07:55.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:55 smithi083 ceph-mon[29421]: osdmap e465: 8 total, 8 up, 8 in 2024-09-20T19:07:55.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:55 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 3/308544 objects degraded (0.001%), 1 pg degraded (PG_DEGRADED) 2024-09-20T19:07:55.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:55 smithi083 ceph-mon[36260]: osdmap e465: 8 total, 8 up, 8 in 2024-09-20T19:07:55.352 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:55.391 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:55.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:55.392 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:55.493 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:55.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:55 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 3/308544 objects degraded (0.001%), 1 pg degraded (PG_DEGRADED) 2024-09-20T19:07:55.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:55 smithi142 ceph-mon[32508]: osdmap e465: 8 total, 8 up, 8 in 2024-09-20T19:07:55.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:55.549 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:55.549 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:55.650 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:55.679 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:55.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:55.680 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:55.781 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:55.802 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:55.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:55.803 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:55.903 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:55.933 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:55.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:55.934 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:56.035 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:07:56.066 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:56.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:56.067 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:07:56.168 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:56.205 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:56.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:56.206 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:07:56.307 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:56.353 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:56.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:56.354 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:07:56.415 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:56 smithi083 ceph-mon[29421]: pgmap v924: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 6.3 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 853 B/s rd, 44 MiB/s wr, 702 op/s; 3/308544 objects degraded (0.001%); 14595/308544 objects misplaced (4.730%); 5.9 MiB/s, 94 objects/s recovering 2024-09-20T19:07:56.415 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:56 smithi083 ceph-mon[36260]: pgmap v924: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 6.3 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 853 B/s rd, 44 MiB/s wr, 702 op/s; 3/308544 objects degraded (0.001%); 14595/308544 objects misplaced (4.730%); 5.9 MiB/s, 94 objects/s recovering 2024-09-20T19:07:56.455 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:56.483 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:56.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:56.484 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:56.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:56 smithi142 ceph-mon[32508]: pgmap v924: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 6.3 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 853 B/s rd, 44 MiB/s wr, 702 op/s; 3/308544 objects degraded (0.001%); 14595/308544 objects misplaced (4.730%); 5.9 MiB/s, 94 objects/s recovering 2024-09-20T19:07:56.585 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:56.619 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:56.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:56.620 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:07:56.721 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:56.758 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:56.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:56.759 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:07:56.860 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:56.887 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:56.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:56.888 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:56.989 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:57.016 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:57.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:57.028 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:57.129 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:07:57.150 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:57.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:57.152 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:07:57.252 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:57.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:57.282 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:57.283 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:57.383 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:57.413 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:57.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:57.414 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:57.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:57 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:07:57.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:57 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:07:57.515 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:07:57.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:57 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:07:57.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:57 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:07:57.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:57 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:07:57.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:57 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:07:57.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:57.546 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:57.546 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:07:57.647 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:57.676 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:57.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:57.677 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:57.778 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:57.807 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:57.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:57.808 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:57.908 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:07:57.934 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:57.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:57.935 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:07:58.010 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:07:57 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:07:58.036 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:58.061 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:58.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:58.062 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:58.163 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:58.191 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:58.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:58.192 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:58.293 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:58.317 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:58.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:58.319 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:58.419 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:07:58.461 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:58.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:58.462 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:07:58.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:58 smithi083 ceph-mon[29421]: osdmap e466: 8 total, 8 up, 8 in 2024-09-20T19:07:58.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:58 smithi083 ceph-mon[29421]: pgmap v926: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 6.2 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.8 KiB/s rd, 40 MiB/s wr, 981 op/s; 3/304998 objects degraded (0.001%); 14595/304998 objects misplaced (4.785%); 5.5 MiB/s, 87 objects/s recovering 2024-09-20T19:07:58.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:58 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:07:58.507 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:58 smithi083 ceph-mon[36260]: osdmap e466: 8 total, 8 up, 8 in 2024-09-20T19:07:58.507 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:58 smithi083 ceph-mon[36260]: pgmap v926: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 6.2 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.8 KiB/s rd, 40 MiB/s wr, 981 op/s; 3/304998 objects degraded (0.001%); 14595/304998 objects misplaced (4.785%); 5.5 MiB/s, 87 objects/s recovering 2024-09-20T19:07:58.507 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:58 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:07:58.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:58 smithi142 ceph-mon[32508]: osdmap e466: 8 total, 8 up, 8 in 2024-09-20T19:07:58.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:58 smithi142 ceph-mon[32508]: pgmap v926: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 6.2 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.8 KiB/s rd, 40 MiB/s wr, 981 op/s; 3/304998 objects degraded (0.001%); 14595/304998 objects misplaced (4.785%); 5.5 MiB/s, 87 objects/s recovering 2024-09-20T19:07:58.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:58 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:07:58.563 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:58.605 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:58.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:58.606 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:58.707 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:58.742 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:58.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:58.743 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:58.843 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:58.873 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:58.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:58.874 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:58.975 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:07:59.007 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:59.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:59.008 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:07:59.109 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:07:59.140 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:59.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:59.141 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:07:59.242 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:59.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:59.279 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:59.279 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:07:59.380 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:07:59.407 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:07:59.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:59.408 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:07:59.509 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:07:59.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:07:59 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 3/302004 objects degraded (0.001%), 1 pg degraded (PG_DEGRADED) 2024-09-20T19:07:59.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:07:59 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 3/302004 objects degraded (0.001%), 1 pg degraded (PG_DEGRADED) 2024-09-20T19:07:59.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:59.553 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:59.553 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:07:59.654 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:59.688 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:59.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:59.689 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:59.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:07:59 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 3/302004 objects degraded (0.001%), 1 pg degraded (PG_DEGRADED) 2024-09-20T19:07:59.779 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T19:07:59.780 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:07:59.790 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:07:59.831 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:59.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:59.832 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:07:59.933 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:07:59.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:07:59.973 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:07:59.973 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:08:00.074 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:00.101 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:00.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:00.102 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:00.203 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:08:00.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:00.237 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:00.238 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:08:00.338 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:00.360 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:00.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:00.361 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:00.462 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:00.475 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 7, 6, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 5, 6, 7, 4] 2024-09-20T19:08:00.475 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T19:08:00.475 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-20T19:08:00.475 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd dump -f json-pretty 2024-09-20T19:08:00.483 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:00.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:00.484 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:00.584 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:00.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:00.612 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:00.612 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:00.713 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:00.725 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:00 smithi083 ceph-mon[29421]: pgmap v927: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 6.1 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 857 B/s rd, 17 MiB/s wr, 709 op/s; 3/302004 objects degraded (0.001%); 12913/302004 objects misplaced (4.276%); 11 MiB/s, 168 objects/s recovering 2024-09-20T19:08:00.726 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:00 smithi083 ceph-mon[36260]: pgmap v927: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 6.1 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 857 B/s rd, 17 MiB/s wr, 709 op/s; 3/302004 objects degraded (0.001%); 12913/302004 objects misplaced (4.276%); 11 MiB/s, 168 objects/s recovering 2024-09-20T19:08:00.727 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:08:00 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:08:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:08:00.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:00 smithi142 ceph-mon[32508]: pgmap v927: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 6.1 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 857 B/s rd, 17 MiB/s wr, 709 op/s; 3/302004 objects degraded (0.001%); 12913/302004 objects misplaced (4.276%); 11 MiB/s, 168 objects/s recovering 2024-09-20T19:08:00.774 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:00.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:00.775 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:00.876 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:00.900 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:00.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:00.900 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:01.001 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:01.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:01.050 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:01.050 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:01.151 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:01.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:01.192 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:01.193 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:01.294 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:08:01.314 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:01.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:01.315 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:08:01.416 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:01.447 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:01.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:01.448 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:01.549 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:01.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:01.580 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:01.580 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:01.680 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:01.701 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:01.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:01.702 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:01.803 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:08:01.841 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:01.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:01.842 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:08:01.943 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:01.966 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:01.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:01.967 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:02.067 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:02.096 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:02.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:02.097 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:02.198 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:02.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:02.230 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:02.230 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:02.237 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:01 smithi083 ceph-mon[29421]: osdmap e467: 8 total, 8 up, 8 in 2024-09-20T19:08:02.237 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:02 smithi083 ceph-mon[29421]: pgmap v929: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 6.0 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 857 B/s rd, 8.1 MiB/s wr, 778 op/s; 3/297405 objects degraded (0.001%); 12913/297405 objects misplaced (4.342%); 5.9 MiB/s, 94 objects/s recovering 2024-09-20T19:08:02.237 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:02 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:08:02.237 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:02 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:08:02.237 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:02 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:08:02.237 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:02 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:08:02.238 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:01 smithi083 ceph-mon[36260]: osdmap e467: 8 total, 8 up, 8 in 2024-09-20T19:08:02.238 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:02 smithi083 ceph-mon[36260]: pgmap v929: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 6.0 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 857 B/s rd, 8.1 MiB/s wr, 778 op/s; 3/297405 objects degraded (0.001%); 12913/297405 objects misplaced (4.342%); 5.9 MiB/s, 94 objects/s recovering 2024-09-20T19:08:02.238 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:02 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:08:02.238 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:02 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:08:02.238 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:02 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:08:02.238 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:02 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:08:02.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:01 smithi142 ceph-mon[32508]: osdmap e467: 8 total, 8 up, 8 in 2024-09-20T19:08:02.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:01 smithi142 ceph-mon[32508]: pgmap v929: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 6.0 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 857 B/s rd, 8.1 MiB/s wr, 778 op/s; 3/297405 objects degraded (0.001%); 12913/297405 objects misplaced (4.342%); 5.9 MiB/s, 94 objects/s recovering 2024-09-20T19:08:02.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:01 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:08:02.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:01 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:08:02.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:01 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:08:02.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:01 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:08:02.331 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:02.355 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:02.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:02.356 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:02.457 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:02.480 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:02.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:02.481 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:02.582 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:02.623 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:02.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:02.624 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:02.725 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:02.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:02.753 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:02.753 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:02.854 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:02.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:02.883 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:02.883 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:02.983 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:03.018 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:03.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:03.019 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:03.120 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:03.156 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:03.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:03.157 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:03.257 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:03.290 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:03.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:03.291 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:03.391 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:03.418 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:03.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:03.420 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:03.476 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:03 smithi083 ceph-mon[36260]: osdmap e468: 8 total, 8 up, 8 in 2024-09-20T19:08:03.476 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:03 smithi083 ceph-mon[29421]: osdmap e468: 8 total, 8 up, 8 in 2024-09-20T19:08:03.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:03 smithi142 ceph-mon[32508]: osdmap e468: 8 total, 8 up, 8 in 2024-09-20T19:08:03.521 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:03.545 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:03.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:03.546 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:03.647 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:03.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:03.674 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:03.674 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:03.775 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:03.823 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:03.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:03.824 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:03.924 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:03.952 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:03.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:03.953 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:04.054 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:04.083 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:04.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:04.084 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:04.096 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:08:04.096 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:08:04.096 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:08:04.096 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:08:04.097 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:08:04.097 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:08:04.097 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:08:04.097 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:08:04.097 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:08:04.097 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:08:04.097 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:08:04.097 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:08:04.097 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:08:04.097 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:08:04.097 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:08:04.097 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:08:04.185 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:04.213 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:04.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:04.214 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:04.315 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:08:04.356 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:04.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:04.357 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:08:04.458 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:04.496 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:04.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:04.497 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:04.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:04 smithi142 ceph-mon[32508]: pgmap v931: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 5.8 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 835 B/s rd, 533 KiB/s wr, 955 op/s; 3/287745 objects degraded (0.001%); 11390/287745 objects misplaced (3.958%); 13 MiB/s, 209 objects/s recovering 2024-09-20T19:08:04.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:04 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3643443034' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:08:04.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:04 smithi083 ceph-mon[29421]: pgmap v931: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 5.8 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 835 B/s rd, 533 KiB/s wr, 955 op/s; 3/287745 objects degraded (0.001%); 11390/287745 objects misplaced (3.958%); 13 MiB/s, 209 objects/s recovering 2024-09-20T19:08:04.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:04 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3643443034' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:08:04.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:04 smithi083 ceph-mon[36260]: pgmap v931: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 5.8 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 835 B/s rd, 533 KiB/s wr, 955 op/s; 3/287745 objects degraded (0.001%); 11390/287745 objects misplaced (3.958%); 13 MiB/s, 209 objects/s recovering 2024-09-20T19:08:04.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:04 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3643443034' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:08:04.598 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:08:04.639 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:04.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:04.640 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:08:04.740 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:04.779 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:04.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:04.780 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:04.880 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:04.910 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:04.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:04.911 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:05.012 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:05.039 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:05.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:05.040 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:05.141 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:05.161 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:05.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:05.162 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:05.263 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:05.293 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:05.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:05.294 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:05.394 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:05.434 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:05.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:05.435 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:05.536 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:08:05.573 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:05.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:05.575 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:08:05.675 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:05.701 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:05.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:05.702 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:05.803 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:05.830 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:05.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:05.831 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:05.932 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:05.954 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:05.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:05.955 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:06.055 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:06.077 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:06.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:06.078 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:06.178 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:06.202 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:06.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:06.203 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:06.304 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:06.341 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:06.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:06.342 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:06.443 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:08:06.483 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:06.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:06.484 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:08:06.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:06 smithi142 ceph-mon[32508]: pgmap v932: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 5.8 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 639 B/s rd, 408 KiB/s wr, 732 op/s; 3/287745 objects degraded (0.001%); 11390/287745 objects misplaced (3.958%); 10 MiB/s, 160 objects/s recovering 2024-09-20T19:08:06.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:06 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 3/287745 objects degraded (0.001%), 1 pg degraded, 4 pgs undersized (PG_DEGRADED) 2024-09-20T19:08:06.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:06 smithi142 ceph-mon[32508]: osdmap e469: 8 total, 8 up, 8 in 2024-09-20T19:08:06.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:06 smithi083 ceph-mon[29421]: pgmap v932: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 5.8 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 639 B/s rd, 408 KiB/s wr, 732 op/s; 3/287745 objects degraded (0.001%); 11390/287745 objects misplaced (3.958%); 10 MiB/s, 160 objects/s recovering 2024-09-20T19:08:06.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:06 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 3/287745 objects degraded (0.001%), 1 pg degraded, 4 pgs undersized (PG_DEGRADED) 2024-09-20T19:08:06.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:06 smithi083 ceph-mon[29421]: osdmap e469: 8 total, 8 up, 8 in 2024-09-20T19:08:06.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:06 smithi083 ceph-mon[36260]: pgmap v932: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 5.8 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 639 B/s rd, 408 KiB/s wr, 732 op/s; 3/287745 objects degraded (0.001%); 11390/287745 objects misplaced (3.958%); 10 MiB/s, 160 objects/s recovering 2024-09-20T19:08:06.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:06 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 3/287745 objects degraded (0.001%), 1 pg degraded, 4 pgs undersized (PG_DEGRADED) 2024-09-20T19:08:06.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:06 smithi083 ceph-mon[36260]: osdmap e469: 8 total, 8 up, 8 in 2024-09-20T19:08:06.585 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:06.615 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:06.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:06.616 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:06.717 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:06.746 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:06.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:06.747 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:06.848 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:06.876 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:06.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:06.877 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:06.978 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:07.006 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:07.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:07.007 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:07.108 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:07.138 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:07.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:07.139 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:07.240 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:07.266 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:07.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:07.267 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:07.368 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:07.396 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:07.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:07.397 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:07.497 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:07.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:07.527 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:07.528 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:07.628 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:07.654 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:07.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:07.655 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:07.756 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:07.784 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:07.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:07.785 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:07.886 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:08:07.926 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:07.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:07.927 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:08:08.010 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:08:07 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:08:08.028 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:08.056 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:08.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:08.057 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:08.158 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:08:08.199 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:08.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:08.200 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:08:08.301 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:08.326 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:08.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:08.327 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:08.427 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:08:08.464 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:08.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:08.466 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:08:08.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:08 smithi142 ceph-mon[32508]: pgmap v934: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 5.6 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1.21k op/s; 3/277443 objects degraded (0.001%); 11390/277443 objects misplaced (4.105%); 6.0 MiB/s, 96 objects/s recovering 2024-09-20T19:08:08.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:08 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:08:08.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:08 smithi083 ceph-mon[29421]: pgmap v934: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 5.6 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1.21k op/s; 3/277443 objects degraded (0.001%); 11390/277443 objects misplaced (4.105%); 6.0 MiB/s, 96 objects/s recovering 2024-09-20T19:08:08.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:08 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:08:08.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:08 smithi083 ceph-mon[36260]: pgmap v934: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 5.6 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1.21k op/s; 3/277443 objects degraded (0.001%); 11390/277443 objects misplaced (4.105%); 6.0 MiB/s, 96 objects/s recovering 2024-09-20T19:08:08.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:08 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:08:08.567 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:08:08.600 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:08.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:08.601 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:08:08.701 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:08.722 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:08.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:08.723 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:08.824 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:08.851 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:08.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:08.852 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:08.953 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:08:08.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:08.986 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:08.986 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:08:09.087 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:09.126 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:09.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:09.127 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:09.227 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:09.252 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:09.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:09.253 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:09.354 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:09.380 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:09.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:09.381 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:09.482 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:09.522 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:09.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:09.523 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:09.624 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:09.666 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:09.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:09.666 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:09.767 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:09.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:09.797 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:09.797 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:09.898 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:09.942 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:09.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:09.943 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:10.043 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:10.071 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:10.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:10.072 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:10.153 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:08:10.173 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:08:10.235 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:10.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:10.237 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:08:10.338 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:10.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:10.383 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:10.383 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:10.484 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:10.518 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:10.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:10.519 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:10.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:10 smithi083 ceph-mon[29421]: pgmap v935: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 5.6 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 988 op/s; 3/273699 objects degraded (0.001%); 9738/273699 objects misplaced (3.558%); 9.8 MiB/s, 156 objects/s recovering 2024-09-20T19:08:10.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:10 smithi083 ceph-mon[36260]: pgmap v935: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 5.6 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 988 op/s; 3/273699 objects degraded (0.001%); 9738/273699 objects misplaced (3.558%); 9.8 MiB/s, 156 objects/s recovering 2024-09-20T19:08:10.619 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:10.648 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:10.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:10.649 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:10.750 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:10.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:10 smithi142 ceph-mon[32508]: pgmap v935: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 5.6 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 988 op/s; 3/273699 objects degraded (0.001%); 9738/273699 objects misplaced (3.558%); 9.8 MiB/s, 156 objects/s recovering 2024-09-20T19:08:10.783 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:10.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:10.784 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:10.885 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:10.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:10.915 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:10.916 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:11.016 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:11.035 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:08:10 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:08:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:08:11.052 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:11.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:11.053 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:11.154 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:11.184 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:11.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:11.186 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:11.287 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:11.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:11 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 3/273699 objects degraded (0.001%), 1 pg degraded, 4 pgs undersized (PG_DEGRADED) 2024-09-20T19:08:11.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:11 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 3/273699 objects degraded (0.001%), 1 pg degraded, 4 pgs undersized (PG_DEGRADED) 2024-09-20T19:08:11.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:11.320 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:11.321 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:11.422 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:11.449 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:11.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:11.451 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:11.552 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:11.583 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:11.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:11.584 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:11.685 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:11.716 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:11.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:11.717 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:11.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:11 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 3/273699 objects degraded (0.001%), 1 pg degraded, 4 pgs undersized (PG_DEGRADED) 2024-09-20T19:08:11.818 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:11.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:11.863 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:11.864 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:11.964 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:11.990 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:11.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:11.991 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:12.092 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:12.119 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:12.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:12.120 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:12.201 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T19:08:12.221 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:12.248 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:12.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:12.250 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:12.350 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:08:12.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:12.391 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:12.391 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:08:12.491 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:12.520 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:12.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:12.521 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:12.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:12 smithi083 ceph-mon[29421]: pgmap v936: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 5.5 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 558 B/s rd, 0 B/s wr, 809 op/s; 3/269007 objects degraded (0.001%); 9738/269007 objects misplaced (3.620%); 8.5 MiB/s, 136 objects/s recovering 2024-09-20T19:08:12.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:12 smithi083 ceph-mon[36260]: pgmap v936: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 5.5 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 558 B/s rd, 0 B/s wr, 809 op/s; 3/269007 objects degraded (0.001%); 9738/269007 objects misplaced (3.620%); 8.5 MiB/s, 136 objects/s recovering 2024-09-20T19:08:12.622 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:12.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:12.648 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:12.648 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:12.749 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:12.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:12 smithi142 ceph-mon[32508]: pgmap v936: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 5.5 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 558 B/s rd, 0 B/s wr, 809 op/s; 3/269007 objects degraded (0.001%); 9738/269007 objects misplaced (3.620%); 8.5 MiB/s, 136 objects/s recovering 2024-09-20T19:08:12.777 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:12.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:12.778 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:12.879 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:08:12.921 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:12.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:12.922 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:08:13.023 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:08:13.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:13.064 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:13.065 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:08:13.165 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:13.210 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:13.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:13.211 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:13.312 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:13.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:13.352 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:13.353 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:13.453 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:13.490 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:13.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:13.491 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:13.592 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:08:13.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:13.632 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:13.632 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:08:13.733 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:13.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:13.768 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:13.768 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:13.869 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:13.895 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:13.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:13.896 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:13.997 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:14.032 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:14.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:14.033 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:14.134 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:08:14.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:14.168 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:14.169 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:08:14.269 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:14.296 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:14.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:14.297 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:14.398 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:14.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:14.430 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:14.430 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:14.531 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:14.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:14 smithi083 ceph-mon[29421]: pgmap v937: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 5.2 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 985 op/s; 3/258189 objects degraded (0.001%); 8017/258189 objects misplaced (3.105%); 8.3 MiB/s, 133 objects/s recovering 2024-09-20T19:08:14.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:14 smithi083 ceph-mon[36260]: pgmap v937: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 5.2 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 985 op/s; 3/258189 objects degraded (0.001%); 8017/258189 objects misplaced (3.105%); 8.3 MiB/s, 133 objects/s recovering 2024-09-20T19:08:14.558 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:14.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:14.559 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:14.660 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:08:14.707 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:14.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:14.708 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:08:14.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:14 smithi142 ceph-mon[32508]: pgmap v937: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 5.2 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 985 op/s; 3/258189 objects degraded (0.001%); 8017/258189 objects misplaced (3.105%); 8.3 MiB/s, 133 objects/s recovering 2024-09-20T19:08:14.809 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:14.833 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:14.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:14.835 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:14.936 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:14.964 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:14.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:14.965 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:15.066 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:15.093 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:15.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:15.094 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:15.194 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:15.223 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:15.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:15.224 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:15.325 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:15.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:15.356 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:15.356 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:15.457 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:15.491 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:15.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:15.492 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:15.593 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:15.620 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:15.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:15.621 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:15.722 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:15.751 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:15.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:15.752 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:15.852 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:15.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:15.892 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:15.893 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:15.993 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:08:16.039 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:16.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:16.040 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:08:16.141 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:08:16.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:16.183 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:16.183 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:08:16.284 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:08:16.324 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:16.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:16.325 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:08:16.426 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:16.452 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:16.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:16.453 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:16.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:16 smithi083 ceph-mon[29421]: pgmap v938: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 5.2 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 985 op/s; 3/258189 objects degraded (0.001%); 8017/258189 objects misplaced (3.105%); 8.3 MiB/s, 133 objects/s recovering 2024-09-20T19:08:16.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:16 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 3/258189 objects degraded (0.001%), 1 pg degraded, 4 pgs undersized (PG_DEGRADED) 2024-09-20T19:08:16.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:16 smithi083 ceph-mon[36260]: pgmap v938: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 5.2 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 985 op/s; 3/258189 objects degraded (0.001%); 8017/258189 objects misplaced (3.105%); 8.3 MiB/s, 133 objects/s recovering 2024-09-20T19:08:16.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:16 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 3/258189 objects degraded (0.001%), 1 pg degraded, 4 pgs undersized (PG_DEGRADED) 2024-09-20T19:08:16.554 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:16.593 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:16.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:16.594 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:16.694 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:16.716 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:16.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:16.717 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:16.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:16 smithi142 ceph-mon[32508]: pgmap v938: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 5.2 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 985 op/s; 3/258189 objects degraded (0.001%); 8017/258189 objects misplaced (3.105%); 8.3 MiB/s, 133 objects/s recovering 2024-09-20T19:08:16.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:16 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 3/258189 objects degraded (0.001%), 1 pg degraded, 4 pgs undersized (PG_DEGRADED) 2024-09-20T19:08:16.818 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:16.851 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:16.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:16.852 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:16.952 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:08:16.974 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:16.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:16.975 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:08:17.075 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:17.105 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:17.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:17.106 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:17.207 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:17.236 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:17.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:17.237 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:17.338 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:17.361 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:17.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:17.362 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:17.462 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:17.494 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:17.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:17.495 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:17.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:17 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:08:17.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:17 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T19:08:17.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:17 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:08:17.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:17 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T19:08:17.596 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:08:17.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:17.619 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:17.619 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:08:17.720 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:17.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:17 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:08:17.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:17 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T19:08:17.761 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:08:17 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:08:17.766 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:17.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:17.767 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:17.867 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:17.902 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:17.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:17.903 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:18.004 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:18.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:18.032 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:18.032 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:18.133 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:18.160 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:18.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:18.161 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:18.262 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:18.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:18.291 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:18.291 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:18.392 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:18.417 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:18.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:18.418 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:18.519 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:18.561 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:18.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:18.562 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:18.662 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:08:18.706 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:18.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:18.707 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:08:18.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:18 smithi142 ceph-mon[32508]: pgmap v939: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 5.0 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.16k op/s; 3/246783 objects degraded (0.001%); 8017/246783 objects misplaced (3.249%); 7.4 MiB/s, 118 objects/s recovering 2024-09-20T19:08:18.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:18 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:08:18.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:18 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T19:08:18.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:18 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T19:08:18.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:18 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:08:18.790 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:18 smithi083 ceph-mon[29421]: pgmap v939: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 5.0 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.16k op/s; 3/246783 objects degraded (0.001%); 8017/246783 objects misplaced (3.249%); 7.4 MiB/s, 118 objects/s recovering 2024-09-20T19:08:18.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:18 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:08:18.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:18 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T19:08:18.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:18 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T19:08:18.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:18 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:08:18.791 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:18 smithi083 ceph-mon[36260]: pgmap v939: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 5.0 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.16k op/s; 3/246783 objects degraded (0.001%); 8017/246783 objects misplaced (3.249%); 7.4 MiB/s, 118 objects/s recovering 2024-09-20T19:08:18.791 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:18 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:08:18.791 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:18 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T19:08:18.791 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:18 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T19:08:18.791 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:18 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:08:18.808 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:18.843 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:18.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:18.844 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:18.945 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:18.957 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_blocked_ops 2024-09-20T19:08:18.980 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:18.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:18.981 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:19.082 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:19.104 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:19.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:19.105 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:19.206 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:08:19.227 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:19.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:19.227 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:08:19.328 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:08:19.379 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:19.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:19.380 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:08:19.481 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:08:19.502 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:19.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:19.503 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:08:19.603 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:19.636 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:19.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:19.637 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:19.738 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:19.770 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:19.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:19.771 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:19.872 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:08:19.896 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:19.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:19.897 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:08:19.998 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:20.027 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:20.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:20.028 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:20.128 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:08:20.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:20.173 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:20.173 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:08:20.274 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:20.301 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:20.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:20.302 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:20.403 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:20.444 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:20.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:20.446 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:20.546 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:20.575 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:20.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:20.576 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:20.677 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:08:20.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:20.723 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:20.723 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:08:20.724 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:20 smithi083 ceph-mon[29421]: pgmap v940: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 4.9 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 957 op/s; 3/242973 objects degraded (0.001%); 6163/242973 objects misplaced (2.536%); 11 MiB/s, 172 objects/s recovering 2024-09-20T19:08:20.724 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:20 smithi083 ceph-mon[36260]: pgmap v940: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 4.9 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 957 op/s; 3/242973 objects degraded (0.001%); 6163/242973 objects misplaced (2.536%); 11 MiB/s, 172 objects/s recovering 2024-09-20T19:08:20.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:20 smithi142 ceph-mon[32508]: pgmap v940: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 4.9 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 957 op/s; 3/242973 objects degraded (0.001%); 6163/242973 objects misplaced (2.536%); 11 MiB/s, 172 objects/s recovering 2024-09-20T19:08:20.817 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:08:20.824 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:08:20.859 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:08:20.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:20.923 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:20.923 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:08:20.980 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:08:20 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:08:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:08:21.024 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:21.048 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:21.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:21.049 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:21.150 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:21.176 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:21.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:21.177 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:21.278 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:08:21.304 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-20T19:08:21.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:21.355 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:21.355 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:08:21.456 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:21.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:21.495 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:21.495 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:21.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:21 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2622726265' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:08:21.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:21 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:08:21.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:21 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 3/242973 objects degraded (0.001%), 1 pg degraded, 4 pgs undersized (PG_DEGRADED) 2024-09-20T19:08:21.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:21 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2622726265' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:08:21.542 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:21 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:08:21.542 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:21 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 3/242973 objects degraded (0.001%), 1 pg degraded, 4 pgs undersized (PG_DEGRADED) 2024-09-20T19:08:21.597 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:08:21.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:21.636 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:21.636 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:08:21.736 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:08:21.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:21 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2622726265' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:08:21.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:21 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:08:21.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:21 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 3/242973 objects degraded (0.001%), 1 pg degraded, 4 pgs undersized (PG_DEGRADED) 2024-09-20T19:08:21.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:21.777 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:21.777 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:08:21.877 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:21.903 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:21.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:21.904 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:22.005 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:22.045 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:22.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:22.046 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:22.146 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:22.175 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:22.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:22.176 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:22.277 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:22.314 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:22.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:22.315 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:22.417 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:22.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:22.460 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:22.460 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:22.562 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:22.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:22.591 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:22.592 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:22.632 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:22 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:08:22.632 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:22 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:08:22.633 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:22 smithi083 ceph-mon[29421]: pgmap v941: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 4.8 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 966 op/s; 3/238914 objects degraded (0.001%); 6163/238914 objects misplaced (2.580%); 7.4 MiB/s, 118 objects/s recovering 2024-09-20T19:08:22.633 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:22 smithi083 ceph-mon[29421]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:08:22.633 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:22 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T19:08:22.633 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:22 smithi083 ceph-mon[29421]: osdmap e470: 8 total, 8 up, 8 in 2024-09-20T19:08:22.633 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:22 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/381534363' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-20T19:08:22.633 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:22 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:08:22.633 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:22 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:08:22.633 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:22 smithi083 ceph-mon[36260]: pgmap v941: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 4.8 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 966 op/s; 3/238914 objects degraded (0.001%); 6163/238914 objects misplaced (2.580%); 7.4 MiB/s, 118 objects/s recovering 2024-09-20T19:08:22.634 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:22 smithi083 ceph-mon[36260]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:08:22.634 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:22 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T19:08:22.634 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:22 smithi083 ceph-mon[36260]: osdmap e470: 8 total, 8 up, 8 in 2024-09-20T19:08:22.634 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:22 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/381534363' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-20T19:08:22.635 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T19:08:22.635 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T19:08:22.635 INFO:teuthology.orchestra.run.smithi083.stdout: "epoch": 470, 2024-09-20T19:08:22.635 INFO:teuthology.orchestra.run.smithi083.stdout: "fsid": "1ed5a50a-7780-11ef-baf5-efdc52797490", 2024-09-20T19:08:22.635 INFO:teuthology.orchestra.run.smithi083.stdout: "created": "2024-09-20T18:44:10.399800+0000", 2024-09-20T19:08:22.635 INFO:teuthology.orchestra.run.smithi083.stdout: "modified": "2024-09-20T19:08:21.291027+0000", 2024-09-20T19:08:22.635 INFO:teuthology.orchestra.run.smithi083.stdout: "last_up_change": "2024-09-20T19:01:21.809732+0000", 2024-09-20T19:08:22.635 INFO:teuthology.orchestra.run.smithi083.stdout: "last_in_change": "2024-09-20T19:04:20.299488+0000", 2024-09-20T19:08:22.635 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-20T19:08:22.635 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_num": 5804032, 2024-09-20T19:08:22.635 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_set": [ 2024-09-20T19:08:22.635 INFO:teuthology.orchestra.run.smithi083.stdout: "nodeep-scrub", 2024-09-20T19:08:22.636 INFO:teuthology.orchestra.run.smithi083.stdout: "pglog_hardlimit", 2024-09-20T19:08:22.636 INFO:teuthology.orchestra.run.smithi083.stdout: "purged_snapdirs", 2024-09-20T19:08:22.636 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_deletes", 2024-09-20T19:08:22.636 INFO:teuthology.orchestra.run.smithi083.stdout: "sortbitwise" 2024-09-20T19:08:22.636 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T19:08:22.636 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_version": 19, 2024-09-20T19:08:22.636 INFO:teuthology.orchestra.run.smithi083.stdout: "full_ratio": 0.94999998807907104, 2024-09-20T19:08:22.636 INFO:teuthology.orchestra.run.smithi083.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-20T19:08:22.636 INFO:teuthology.orchestra.run.smithi083.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-20T19:08:22.636 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_snapshot": "", 2024-09-20T19:08:22.636 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_max": 11, 2024-09-20T19:08:22.636 INFO:teuthology.orchestra.run.smithi083.stdout: "max_osd": 8, 2024-09-20T19:08:22.636 INFO:teuthology.orchestra.run.smithi083.stdout: "require_min_compat_client": "luminous", 2024-09-20T19:08:22.637 INFO:teuthology.orchestra.run.smithi083.stdout: "min_compat_client": "luminous", 2024-09-20T19:08:22.637 INFO:teuthology.orchestra.run.smithi083.stdout: "require_osd_release": "squid", 2024-09-20T19:08:22.637 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_crimson": false, 2024-09-20T19:08:22.637 INFO:teuthology.orchestra.run.smithi083.stdout: "pools": [ 2024-09-20T19:08:22.637 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.637 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 1, 2024-09-20T19:08:22.637 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": ".mgr", 2024-09-20T19:08:22.637 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-20T18:47:15.285795+0000", 2024-09-20T19:08:22.637 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-20T19:08:22.637 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool", 2024-09-20T19:08:22.637 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-20T19:08:22.637 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 3, 2024-09-20T19:08:22.637 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 2, 2024-09-20T19:08:22.637 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-20T19:08:22.638 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-20T19:08:22.638 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-20T19:08:22.638 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T19:08:22.638 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T19:08:22.638 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-20T19:08:22.638 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-20T19:08:22.638 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-20T19:08:22.638 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 11, 2024-09-20T19:08:22.638 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 11, 2024-09-20T19:08:22.638 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 11, 2024-09-20T19:08:22.638 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 11, 2024-09-20T19:08:22.638 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 11, 2024-09-20T19:08:22.639 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-20T19:08:22.639 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-20T19:08:22.639 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-20T19:08:22.639 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-20T19:08:22.639 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-20T19:08:22.639 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-20T19:08:22.639 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-20T19:08:22.639 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.639 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "458", 2024-09-20T19:08:22.639 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-20T19:08:22.639 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T19:08:22.639 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "455", 2024-09-20T19:08:22.639 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-20T19:08:22.640 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-20T19:08:22.640 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 0, 2024-09-20T19:08:22.640 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 0, 2024-09-20T19:08:22.640 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-20T19:08:22.640 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-20T19:08:22.640 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-20T19:08:22.640 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-20T19:08:22.640 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-20T19:08:22.640 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-20T19:08:22.640 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-20T19:08:22.640 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-20T19:08:22.640 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-20T19:08:22.641 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-20T19:08:22.641 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-20T19:08:22.641 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T19:08:22.641 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T19:08:22.641 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T19:08:22.641 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-20T19:08:22.641 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-20T19:08:22.641 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-20T19:08:22.641 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-20T19:08:22.641 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-20T19:08:22.641 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.641 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-20T19:08:22.641 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-20T19:08:22.641 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-20T19:08:22.642 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-20T19:08:22.642 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-20T19:08:22.642 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T19:08:22.642 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-20T19:08:22.642 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-20T19:08:22.642 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-20T19:08:22.642 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-20T19:08:22.642 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-20T19:08:22.642 INFO:teuthology.orchestra.run.smithi083.stdout: "options": { 2024-09-20T19:08:22.642 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_max": 32, 2024-09-20T19:08:22.642 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_min": 1 2024-09-20T19:08:22.642 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.643 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-20T19:08:22.643 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": {} 2024-09-20T19:08:22.643 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.643 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-20T19:08:22.643 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-20T19:08:22.643 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 2.1800000667572021, 2024-09-20T19:08:22.643 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 2.1800000667572021, 2024-09-20T19:08:22.643 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 1, 2024-09-20T19:08:22.643 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 2.1800000667572021, 2024-09-20T19:08:22.643 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 2.1800000667572021, 2024-09-20T19:08:22.643 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 0.98603439331054688, 2024-09-20T19:08:22.643 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 0.99000000953674316, 2024-09-20T19:08:22.643 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 0.99000000953674316 2024-09-20T19:08:22.643 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.644 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.644 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.644 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 2, 2024-09-20T19:08:22.644 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": "datapool", 2024-09-20T19:08:22.644 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-20T18:49:43.452984+0000", 2024-09-20T19:08:22.644 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 8193, 2024-09-20T19:08:22.644 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-20T19:08:22.644 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-20T19:08:22.644 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 3, 2024-09-20T19:08:22.644 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 2, 2024-09-20T19:08:22.644 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-20T19:08:22.644 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-20T19:08:22.644 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-20T19:08:22.644 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T19:08:22.645 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T19:08:22.645 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-20T19:08:22.645 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-20T19:08:22.645 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-20T19:08:22.645 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 3, 2024-09-20T19:08:22.645 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 3, 2024-09-20T19:08:22.645 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 3, 2024-09-20T19:08:22.645 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 3, 2024-09-20T19:08:22.645 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 3, 2024-09-20T19:08:22.645 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-20T19:08:22.645 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-20T19:08:22.645 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-20T19:08:22.645 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-20T19:08:22.646 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-20T19:08:22.646 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-20T19:08:22.646 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-20T19:08:22.646 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.646 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "61", 2024-09-20T19:08:22.646 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-20T19:08:22.646 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T19:08:22.646 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T19:08:22.646 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-20T19:08:22.646 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-20T19:08:22.646 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 2, 2024-09-20T19:08:22.646 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 61, 2024-09-20T19:08:22.646 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-20T19:08:22.647 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-20T19:08:22.647 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-20T19:08:22.647 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-20T19:08:22.647 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-20T19:08:22.647 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-20T19:08:22.647 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-20T19:08:22.647 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-20T19:08:22.647 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-20T19:08:22.647 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-20T19:08:22.647 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-20T19:08:22.647 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T19:08:22.647 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T19:08:22.647 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T19:08:22.647 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-20T19:08:22.648 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-20T19:08:22.648 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-20T19:08:22.648 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-20T19:08:22.648 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-20T19:08:22.648 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.648 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-20T19:08:22.648 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-20T19:08:22.648 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-20T19:08:22.648 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-20T19:08:22.648 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-20T19:08:22.648 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T19:08:22.648 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-20T19:08:22.648 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-20T19:08:22.648 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-20T19:08:22.649 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-20T19:08:22.649 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-20T19:08:22.649 INFO:teuthology.orchestra.run.smithi083.stdout: "options": {}, 2024-09-20T19:08:22.649 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-20T19:08:22.649 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd": {} 2024-09-20T19:08:22.649 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.649 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-20T19:08:22.649 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-20T19:08:22.649 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 2.6500000953674316, 2024-09-20T19:08:22.649 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 2.6500000953674316, 2024-09-20T19:08:22.649 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 0.87999999523162842, 2024-09-20T19:08:22.649 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 2.3299999237060547, 2024-09-20T19:08:22.649 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 2.3299999237060547, 2024-09-20T19:08:22.649 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 0.98603439331054688, 2024-09-20T19:08:22.649 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 0.99000000953674316, 2024-09-20T19:08:22.650 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 0.99000000953674316 2024-09-20T19:08:22.650 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.650 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.650 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.650 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 3, 2024-09-20T19:08:22.650 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": ".rgw.root", 2024-09-20T19:08:22.650 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-20T18:49:44.217011+0000", 2024-09-20T19:08:22.650 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-20T19:08:22.650 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool", 2024-09-20T19:08:22.650 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-20T19:08:22.650 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 3, 2024-09-20T19:08:22.650 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 2, 2024-09-20T19:08:22.650 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-20T19:08:22.651 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-20T19:08:22.651 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-20T19:08:22.651 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T19:08:22.651 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T19:08:22.651 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-20T19:08:22.651 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-20T19:08:22.651 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-20T19:08:22.651 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 32, 2024-09-20T19:08:22.651 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 32, 2024-09-20T19:08:22.651 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 32, 2024-09-20T19:08:22.651 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 32, 2024-09-20T19:08:22.651 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 32, 2024-09-20T19:08:22.651 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-20T19:08:22.651 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-20T19:08:22.652 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-20T19:08:22.652 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-20T19:08:22.652 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-20T19:08:22.652 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-20T19:08:22.652 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-20T19:08:22.652 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.652 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "60", 2024-09-20T19:08:22.652 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-20T19:08:22.652 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T19:08:22.652 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T19:08:22.652 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-20T19:08:22.652 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-20T19:08:22.652 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 0, 2024-09-20T19:08:22.652 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 0, 2024-09-20T19:08:22.653 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-20T19:08:22.653 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-20T19:08:22.653 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-20T19:08:22.653 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-20T19:08:22.653 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-20T19:08:22.653 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-20T19:08:22.653 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-20T19:08:22.653 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-20T19:08:22.653 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-20T19:08:22.653 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-20T19:08:22.653 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-20T19:08:22.653 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T19:08:22.653 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T19:08:22.654 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T19:08:22.654 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-20T19:08:22.654 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-20T19:08:22.654 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-20T19:08:22.654 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-20T19:08:22.654 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-20T19:08:22.654 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.654 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-20T19:08:22.654 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-20T19:08:22.654 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-20T19:08:22.654 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-20T19:08:22.654 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-20T19:08:22.654 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T19:08:22.654 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-20T19:08:22.655 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-20T19:08:22.655 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-20T19:08:22.655 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-20T19:08:22.655 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-20T19:08:22.655 INFO:teuthology.orchestra.run.smithi083.stdout: "options": {}, 2024-09-20T19:08:22.655 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-20T19:08:22.655 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw": {} 2024-09-20T19:08:22.655 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.655 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-20T19:08:22.655 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-20T19:08:22.655 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 1.5, 2024-09-20T19:08:22.655 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 1.5, 2024-09-20T19:08:22.655 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 1, 2024-09-20T19:08:22.656 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 1.5, 2024-09-20T19:08:22.656 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 1.5, 2024-09-20T19:08:22.656 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 0.98603439331054688, 2024-09-20T19:08:22.656 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 0.99000000953674316, 2024-09-20T19:08:22.656 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 0.99000000953674316 2024-09-20T19:08:22.656 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.656 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.656 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 4, 2024-09-20T19:08:22.656 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": "default.rgw.log", 2024-09-20T19:08:22.656 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-20T18:49:46.007275+0000", 2024-09-20T19:08:22.656 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-20T19:08:22.656 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool", 2024-09-20T19:08:22.656 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-20T19:08:22.657 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 3, 2024-09-20T19:08:22.657 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 2, 2024-09-20T19:08:22.657 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-20T19:08:22.657 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-20T19:08:22.657 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-20T19:08:22.657 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T19:08:22.657 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T19:08:22.657 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-20T19:08:22.657 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-20T19:08:22.657 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-20T19:08:22.657 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 32, 2024-09-20T19:08:22.657 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 32, 2024-09-20T19:08:22.657 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 32, 2024-09-20T19:08:22.657 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 32, 2024-09-20T19:08:22.657 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 32, 2024-09-20T19:08:22.658 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-20T19:08:22.658 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-20T19:08:22.658 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-20T19:08:22.658 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-20T19:08:22.658 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-20T19:08:22.658 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-20T19:08:22.658 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-20T19:08:22.658 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.658 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "62", 2024-09-20T19:08:22.658 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-20T19:08:22.658 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T19:08:22.658 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T19:08:22.658 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-20T19:08:22.658 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-20T19:08:22.659 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 0, 2024-09-20T19:08:22.659 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 0, 2024-09-20T19:08:22.659 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-20T19:08:22.659 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-20T19:08:22.659 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-20T19:08:22.659 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-20T19:08:22.659 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-20T19:08:22.659 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-20T19:08:22.659 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-20T19:08:22.659 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-20T19:08:22.659 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-20T19:08:22.659 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-20T19:08:22.659 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-20T19:08:22.660 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T19:08:22.660 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T19:08:22.660 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T19:08:22.660 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-20T19:08:22.660 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-20T19:08:22.660 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-20T19:08:22.660 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-20T19:08:22.660 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-20T19:08:22.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.660 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-20T19:08:22.660 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-20T19:08:22.660 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-20T19:08:22.660 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-20T19:08:22.660 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-20T19:08:22.660 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T19:08:22.661 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-20T19:08:22.661 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-20T19:08:22.661 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-20T19:08:22.661 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-20T19:08:22.661 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-20T19:08:22.661 INFO:teuthology.orchestra.run.smithi083.stdout: "options": {}, 2024-09-20T19:08:22.661 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-20T19:08:22.661 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw": {} 2024-09-20T19:08:22.661 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.661 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-20T19:08:22.661 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-20T19:08:22.661 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 2.25, 2024-09-20T19:08:22.661 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 2.25, 2024-09-20T19:08:22.661 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 1, 2024-09-20T19:08:22.661 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 2.25, 2024-09-20T19:08:22.662 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 2.25, 2024-09-20T19:08:22.662 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 0.98603439331054688, 2024-09-20T19:08:22.662 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 0.99000000953674316, 2024-09-20T19:08:22.662 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 0.99000000953674316 2024-09-20T19:08:22.662 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.662 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.662 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.662 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 5, 2024-09-20T19:08:22.662 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": "default.rgw.control", 2024-09-20T19:08:22.662 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-20T18:49:47.961258+0000", 2024-09-20T19:08:22.662 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-20T19:08:22.662 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool", 2024-09-20T19:08:22.662 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-20T19:08:22.662 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 3, 2024-09-20T19:08:22.663 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 2, 2024-09-20T19:08:22.663 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-20T19:08:22.663 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-20T19:08:22.663 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-20T19:08:22.663 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T19:08:22.663 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T19:08:22.663 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-20T19:08:22.663 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-20T19:08:22.663 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-20T19:08:22.663 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 32, 2024-09-20T19:08:22.663 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 32, 2024-09-20T19:08:22.663 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 32, 2024-09-20T19:08:22.663 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 32, 2024-09-20T19:08:22.663 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 32, 2024-09-20T19:08:22.664 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-20T19:08:22.664 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-20T19:08:22.664 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-20T19:08:22.664 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-20T19:08:22.664 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-20T19:08:22.664 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-20T19:08:22.664 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-20T19:08:22.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.664 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "64", 2024-09-20T19:08:22.664 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-20T19:08:22.664 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T19:08:22.664 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T19:08:22.664 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-20T19:08:22.664 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-20T19:08:22.665 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 0, 2024-09-20T19:08:22.665 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 0, 2024-09-20T19:08:22.665 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-20T19:08:22.665 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-20T19:08:22.665 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-20T19:08:22.665 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-20T19:08:22.665 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-20T19:08:22.665 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-20T19:08:22.665 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-20T19:08:22.665 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-20T19:08:22.665 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-20T19:08:22.665 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-20T19:08:22.665 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-20T19:08:22.665 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T19:08:22.666 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T19:08:22.666 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T19:08:22.666 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-20T19:08:22.666 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-20T19:08:22.666 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-20T19:08:22.666 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-20T19:08:22.666 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-20T19:08:22.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.666 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-20T19:08:22.666 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-20T19:08:22.666 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-20T19:08:22.666 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-20T19:08:22.666 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-20T19:08:22.667 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T19:08:22.667 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-20T19:08:22.667 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-20T19:08:22.667 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-20T19:08:22.667 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-20T19:08:22.667 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-20T19:08:22.667 INFO:teuthology.orchestra.run.smithi083.stdout: "options": {}, 2024-09-20T19:08:22.667 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-20T19:08:22.667 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw": {} 2024-09-20T19:08:22.667 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.667 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-20T19:08:22.667 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-20T19:08:22.667 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 1.5, 2024-09-20T19:08:22.667 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 1.5, 2024-09-20T19:08:22.667 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 1, 2024-09-20T19:08:22.668 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 1.5, 2024-09-20T19:08:22.668 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 1.5, 2024-09-20T19:08:22.668 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 0.98603439331054688, 2024-09-20T19:08:22.668 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 0.99000000953674316, 2024-09-20T19:08:22.668 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 0.99000000953674316 2024-09-20T19:08:22.668 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.668 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.668 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.668 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 6, 2024-09-20T19:08:22.668 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": "default.rgw.meta", 2024-09-20T19:08:22.668 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-20T18:49:50.011276+0000", 2024-09-20T19:08:22.668 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-20T19:08:22.668 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool", 2024-09-20T19:08:22.668 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-20T19:08:22.669 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 3, 2024-09-20T19:08:22.669 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 2, 2024-09-20T19:08:22.669 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-20T19:08:22.669 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-20T19:08:22.669 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-20T19:08:22.669 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T19:08:22.669 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T19:08:22.669 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-20T19:08:22.669 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-20T19:08:22.669 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-20T19:08:22.669 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 32, 2024-09-20T19:08:22.669 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 32, 2024-09-20T19:08:22.669 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 32, 2024-09-20T19:08:22.669 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 32, 2024-09-20T19:08:22.670 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 32, 2024-09-20T19:08:22.670 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-20T19:08:22.670 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-20T19:08:22.670 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-20T19:08:22.670 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-20T19:08:22.670 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-20T19:08:22.670 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-20T19:08:22.670 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-20T19:08:22.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.670 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "66", 2024-09-20T19:08:22.670 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-20T19:08:22.670 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T19:08:22.670 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T19:08:22.670 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-20T19:08:22.671 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-20T19:08:22.671 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 0, 2024-09-20T19:08:22.671 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 0, 2024-09-20T19:08:22.671 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-20T19:08:22.671 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-20T19:08:22.671 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-20T19:08:22.671 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-20T19:08:22.671 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-20T19:08:22.671 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-20T19:08:22.671 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-20T19:08:22.671 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-20T19:08:22.671 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-20T19:08:22.671 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-20T19:08:22.671 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-20T19:08:22.672 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T19:08:22.672 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T19:08:22.672 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T19:08:22.672 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-20T19:08:22.672 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-20T19:08:22.672 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-20T19:08:22.672 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-20T19:08:22.672 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-20T19:08:22.672 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.672 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-20T19:08:22.672 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-20T19:08:22.672 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-20T19:08:22.672 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-20T19:08:22.672 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-20T19:08:22.672 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T19:08:22.673 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-20T19:08:22.673 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-20T19:08:22.673 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-20T19:08:22.673 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-20T19:08:22.673 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-20T19:08:22.673 INFO:teuthology.orchestra.run.smithi083.stdout: "options": { 2024-09-20T19:08:22.673 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_bias": 4 2024-09-20T19:08:22.673 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.673 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-20T19:08:22.673 INFO:teuthology.orchestra.run.smithi083.stdout: "rgw": {} 2024-09-20T19:08:22.673 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.673 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-20T19:08:22.673 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-20T19:08:22.673 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 1.75, 2024-09-20T19:08:22.673 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 1.75, 2024-09-20T19:08:22.674 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 1, 2024-09-20T19:08:22.674 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 1.75, 2024-09-20T19:08:22.674 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 1.75, 2024-09-20T19:08:22.674 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 0.98603439331054688, 2024-09-20T19:08:22.674 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 0.99000000953674316, 2024-09-20T19:08:22.674 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 0.99000000953674316 2024-09-20T19:08:22.674 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.674 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.674 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 11, 2024-09-20T19:08:22.674 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": "unique_pool_4", 2024-09-20T19:08:22.674 INFO:teuthology.orchestra.run.smithi083.stdout: "create_time": "2024-09-20T19:06:16.062904+0000", 2024-09-20T19:08:22.674 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-09-20T19:08:22.675 INFO:teuthology.orchestra.run.smithi083.stdout: "flags_names": "hashpspool", 2024-09-20T19:08:22.675 INFO:teuthology.orchestra.run.smithi083.stdout: "type": 1, 2024-09-20T19:08:22.675 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 3, 2024-09-20T19:08:22.675 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": 2, 2024-09-20T19:08:22.675 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_rule": 0, 2024-09-20T19:08:22.675 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_count": 0, 2024-09-20T19:08:22.675 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_target": 0, 2024-09-20T19:08:22.675 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T19:08:22.675 INFO:teuthology.orchestra.run.smithi083.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T19:08:22.675 INFO:teuthology.orchestra.run.smithi083.stdout: "is_stretch_pool": false, 2024-09-20T19:08:22.675 INFO:teuthology.orchestra.run.smithi083.stdout: "object_hash": 2, 2024-09-20T19:08:22.675 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscale_mode": "off", 2024-09-20T19:08:22.676 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": 26, 2024-09-20T19:08:22.676 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num": 26, 2024-09-20T19:08:22.676 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_placement_num_target": 26, 2024-09-20T19:08:22.676 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_target": 26, 2024-09-20T19:08:22.676 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num_pending": 26, 2024-09-20T19:08:22.676 INFO:teuthology.orchestra.run.smithi083.stdout: "last_pg_merge_meta": { 2024-09-20T19:08:22.676 INFO:teuthology.orchestra.run.smithi083.stdout: "source_pgid": "0.0", 2024-09-20T19:08:22.676 INFO:teuthology.orchestra.run.smithi083.stdout: "ready_epoch": 0, 2024-09-20T19:08:22.676 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_started": 0, 2024-09-20T19:08:22.676 INFO:teuthology.orchestra.run.smithi083.stdout: "last_epoch_clean": 0, 2024-09-20T19:08:22.676 INFO:teuthology.orchestra.run.smithi083.stdout: "source_version": "0'0", 2024-09-20T19:08:22.676 INFO:teuthology.orchestra.run.smithi083.stdout: "target_version": "0'0" 2024-09-20T19:08:22.676 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.676 INFO:teuthology.orchestra.run.smithi083.stdout: "last_change": "443", 2024-09-20T19:08:22.677 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend": "0", 2024-09-20T19:08:22.677 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T19:08:22.677 INFO:teuthology.orchestra.run.smithi083.stdout: "last_force_op_resend_preluminous": "441", 2024-09-20T19:08:22.677 INFO:teuthology.orchestra.run.smithi083.stdout: "auid": 0, 2024-09-20T19:08:22.677 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_mode": "selfmanaged", 2024-09-20T19:08:22.677 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_seq": 0, 2024-09-20T19:08:22.677 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_epoch": 0, 2024-09-20T19:08:22.677 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_snaps": [], 2024-09-20T19:08:22.677 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps": "[]", 2024-09-20T19:08:22.677 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_bytes": 0, 2024-09-20T19:08:22.677 INFO:teuthology.orchestra.run.smithi083.stdout: "quota_max_objects": 0, 2024-09-20T19:08:22.678 INFO:teuthology.orchestra.run.smithi083.stdout: "tiers": [], 2024-09-20T19:08:22.678 INFO:teuthology.orchestra.run.smithi083.stdout: "tier_of": -1, 2024-09-20T19:08:22.678 INFO:teuthology.orchestra.run.smithi083.stdout: "read_tier": -1, 2024-09-20T19:08:22.678 INFO:teuthology.orchestra.run.smithi083.stdout: "write_tier": -1, 2024-09-20T19:08:22.678 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_mode": "none", 2024-09-20T19:08:22.678 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_bytes": 0, 2024-09-20T19:08:22.678 INFO:teuthology.orchestra.run.smithi083.stdout: "target_max_objects": 0, 2024-09-20T19:08:22.678 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T19:08:22.678 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T19:08:22.678 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T19:08:22.678 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_flush_age": 0, 2024-09-20T19:08:22.678 INFO:teuthology.orchestra.run.smithi083.stdout: "cache_min_evict_age": 0, 2024-09-20T19:08:22.678 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profile": "", 2024-09-20T19:08:22.678 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_params": { 2024-09-20T19:08:22.679 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "none" 2024-09-20T19:08:22.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.679 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_period": 0, 2024-09-20T19:08:22.679 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_count": 0, 2024-09-20T19:08:22.679 INFO:teuthology.orchestra.run.smithi083.stdout: "use_gmt_hitset": true, 2024-09-20T19:08:22.679 INFO:teuthology.orchestra.run.smithi083.stdout: "min_read_recency_for_promote": 0, 2024-09-20T19:08:22.679 INFO:teuthology.orchestra.run.smithi083.stdout: "min_write_recency_for_promote": 0, 2024-09-20T19:08:22.679 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T19:08:22.679 INFO:teuthology.orchestra.run.smithi083.stdout: "hit_set_search_last_n": 0, 2024-09-20T19:08:22.679 INFO:teuthology.orchestra.run.smithi083.stdout: "grade_table": [], 2024-09-20T19:08:22.679 INFO:teuthology.orchestra.run.smithi083.stdout: "stripe_width": 0, 2024-09-20T19:08:22.679 INFO:teuthology.orchestra.run.smithi083.stdout: "expected_num_objects": 0, 2024-09-20T19:08:22.679 INFO:teuthology.orchestra.run.smithi083.stdout: "fast_read": false, 2024-09-20T19:08:22.680 INFO:teuthology.orchestra.run.smithi083.stdout: "options": {}, 2024-09-20T19:08:22.680 INFO:teuthology.orchestra.run.smithi083.stdout: "application_metadata": { 2024-09-20T19:08:22.680 INFO:teuthology.orchestra.run.smithi083.stdout: "rados": {} 2024-09-20T19:08:22.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.680 INFO:teuthology.orchestra.run.smithi083.stdout: "read_balance": { 2024-09-20T19:08:22.680 INFO:teuthology.orchestra.run.smithi083.stdout: "score_type": "Fair distribution", 2024-09-20T19:08:22.680 INFO:teuthology.orchestra.run.smithi083.stdout: "score_acting": 1.5399999618530273, 2024-09-20T19:08:22.680 INFO:teuthology.orchestra.run.smithi083.stdout: "score_stable": 1.5399999618530273, 2024-09-20T19:08:22.680 INFO:teuthology.orchestra.run.smithi083.stdout: "optimal_score": 1, 2024-09-20T19:08:22.680 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_acting": 1.5399999618530273, 2024-09-20T19:08:22.680 INFO:teuthology.orchestra.run.smithi083.stdout: "raw_score_stable": 1.5399999618530273, 2024-09-20T19:08:22.680 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity_weighted": 0.98603439331054688, 2024-09-20T19:08:22.680 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity": 0.99000000953674316, 2024-09-20T19:08:22.680 INFO:teuthology.orchestra.run.smithi083.stdout: "average_primary_affinity_weighted": 0.99000000953674316 2024-09-20T19:08:22.681 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.681 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.681 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T19:08:22.681 INFO:teuthology.orchestra.run.smithi083.stdout: "osds": [ 2024-09-20T19:08:22.681 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.681 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 0, 2024-09-20T19:08:22.681 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "c714e1ec-f0c3-4054-9792-61b84e53eaa8", 2024-09-20T19:08:22.681 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T19:08:22.681 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T19:08:22.681 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T19:08:22.681 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T19:08:22.681 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T19:08:22.682 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T19:08:22.682 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 8, 2024-09-20T19:08:22.682 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 461, 2024-09-20T19:08:22.682 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T19:08:22.682 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T19:08:22.682 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T19:08:22.682 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:08:22.682 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.682 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:08:22.682 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6801", 2024-09-20T19:08:22.682 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2574084049 2024-09-20T19:08:22.682 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.682 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.683 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T19:08:22.683 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:08:22.683 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.683 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:08:22.683 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6802", 2024-09-20T19:08:22.683 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2574084049 2024-09-20T19:08:22.683 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.683 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.683 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T19:08:22.683 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:08:22.683 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.683 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:08:22.684 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6804", 2024-09-20T19:08:22.684 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2574084049 2024-09-20T19:08:22.684 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.684 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.684 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.684 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T19:08:22.684 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:08:22.684 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.684 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:08:22.684 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6803", 2024-09-20T19:08:22.684 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2574084049 2024-09-20T19:08:22.685 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.685 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.685 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.685 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.83:6801/2574084049", 2024-09-20T19:08:22.685 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.83:6802/2574084049", 2024-09-20T19:08:22.685 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.83:6804/2574084049", 2024-09-20T19:08:22.685 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.83:6803/2574084049", 2024-09-20T19:08:22.685 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T19:08:22.685 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T19:08:22.685 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T19:08:22.685 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.685 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.686 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.686 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 1, 2024-09-20T19:08:22.686 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "9dc0f2a8-bafa-4a42-88f5-0301dd58f78d", 2024-09-20T19:08:22.686 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T19:08:22.686 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T19:08:22.686 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T19:08:22.686 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 0.888275146484375, 2024-09-20T19:08:22.686 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T19:08:22.686 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T19:08:22.686 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 13, 2024-09-20T19:08:22.686 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 458, 2024-09-20T19:08:22.686 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T19:08:22.686 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T19:08:22.687 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T19:08:22.687 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:08:22.687 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.687 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:08:22.687 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6805", 2024-09-20T19:08:22.687 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 4213285798 2024-09-20T19:08:22.687 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.687 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.687 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.687 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T19:08:22.687 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:08:22.687 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.687 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:08:22.687 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6806", 2024-09-20T19:08:22.687 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 4213285798 2024-09-20T19:08:22.688 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.688 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.688 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.688 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T19:08:22.688 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:08:22.688 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.688 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:08:22.688 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6808", 2024-09-20T19:08:22.688 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 4213285798 2024-09-20T19:08:22.688 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.688 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.688 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.688 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T19:08:22.688 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:08:22.689 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.689 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:08:22.689 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6807", 2024-09-20T19:08:22.689 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 4213285798 2024-09-20T19:08:22.689 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.689 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.689 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.689 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.83:6805/4213285798", 2024-09-20T19:08:22.689 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.83:6806/4213285798", 2024-09-20T19:08:22.689 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.83:6808/4213285798", 2024-09-20T19:08:22.689 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.83:6807/4213285798", 2024-09-20T19:08:22.689 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T19:08:22.689 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T19:08:22.689 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T19:08:22.689 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.690 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.690 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 2, 2024-09-20T19:08:22.690 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "448ab692-ec41-44dc-b2d5-dbefd255b3fd", 2024-09-20T19:08:22.690 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T19:08:22.690 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T19:08:22.690 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T19:08:22.690 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T19:08:22.690 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T19:08:22.690 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T19:08:22.690 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 18, 2024-09-20T19:08:22.690 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 458, 2024-09-20T19:08:22.690 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T19:08:22.691 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T19:08:22.691 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T19:08:22.691 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:08:22.691 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.691 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:08:22.691 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6809", 2024-09-20T19:08:22.691 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 805947571 2024-09-20T19:08:22.691 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.691 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.691 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.691 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T19:08:22.691 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:08:22.691 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.691 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:08:22.691 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6810", 2024-09-20T19:08:22.692 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 805947571 2024-09-20T19:08:22.692 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.692 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.692 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T19:08:22.692 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:08:22.692 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.692 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:08:22.692 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6812", 2024-09-20T19:08:22.692 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 805947571 2024-09-20T19:08:22.692 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.692 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.693 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T19:08:22.693 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:08:22.693 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.693 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:08:22.693 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6811", 2024-09-20T19:08:22.693 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 805947571 2024-09-20T19:08:22.693 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.693 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.693 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.693 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.83:6809/805947571", 2024-09-20T19:08:22.693 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.83:6810/805947571", 2024-09-20T19:08:22.693 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.83:6812/805947571", 2024-09-20T19:08:22.693 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.83:6811/805947571", 2024-09-20T19:08:22.693 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T19:08:22.693 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T19:08:22.694 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T19:08:22.694 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.694 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.694 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 3, 2024-09-20T19:08:22.694 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "7cda5e50-e22e-4d4a-823b-53c8f5061193", 2024-09-20T19:08:22.694 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T19:08:22.694 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T19:08:22.694 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T19:08:22.694 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T19:08:22.694 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T19:08:22.694 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T19:08:22.694 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 25, 2024-09-20T19:08:22.695 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 458, 2024-09-20T19:08:22.695 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T19:08:22.695 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T19:08:22.695 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T19:08:22.695 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:08:22.695 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.695 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:08:22.695 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6813", 2024-09-20T19:08:22.695 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 987237318 2024-09-20T19:08:22.695 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.695 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.695 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.695 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T19:08:22.695 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:08:22.695 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.696 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:08:22.696 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6814", 2024-09-20T19:08:22.696 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 987237318 2024-09-20T19:08:22.696 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.696 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.696 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T19:08:22.696 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:08:22.696 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.696 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:08:22.696 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6816", 2024-09-20T19:08:22.696 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 987237318 2024-09-20T19:08:22.697 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.697 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.697 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T19:08:22.697 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:08:22.697 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.697 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:08:22.697 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6815", 2024-09-20T19:08:22.697 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 987237318 2024-09-20T19:08:22.697 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.697 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.697 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.83:6813/987237318", 2024-09-20T19:08:22.697 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.83:6814/987237318", 2024-09-20T19:08:22.698 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.83:6816/987237318", 2024-09-20T19:08:22.698 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.83:6815/987237318", 2024-09-20T19:08:22.698 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T19:08:22.698 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T19:08:22.698 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T19:08:22.698 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.698 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.698 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.698 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 4, 2024-09-20T19:08:22.698 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "3d62654d-d31d-4310-9934-49e694dd17d9", 2024-09-20T19:08:22.698 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T19:08:22.698 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T19:08:22.698 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T19:08:22.699 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T19:08:22.699 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 31, 2024-09-20T19:08:22.699 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 291, 2024-09-20T19:08:22.699 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 304, 2024-09-20T19:08:22.699 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 458, 2024-09-20T19:08:22.699 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 292, 2024-09-20T19:08:22.699 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T19:08:22.699 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T19:08:22.699 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:08:22.699 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.699 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:08:22.699 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6800", 2024-09-20T19:08:22.699 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3322919402 2024-09-20T19:08:22.699 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.700 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.700 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T19:08:22.700 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:08:22.700 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.700 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:08:22.700 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6801", 2024-09-20T19:08:22.700 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3322919402 2024-09-20T19:08:22.700 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.700 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.700 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T19:08:22.700 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:08:22.700 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.701 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:08:22.701 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6803", 2024-09-20T19:08:22.701 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3322919402 2024-09-20T19:08:22.701 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.701 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.701 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T19:08:22.701 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:08:22.701 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.701 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:08:22.701 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6802", 2024-09-20T19:08:22.701 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3322919402 2024-09-20T19:08:22.701 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.701 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.702 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.702 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.142:6800/3322919402", 2024-09-20T19:08:22.702 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.142:6801/3322919402", 2024-09-20T19:08:22.702 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.142:6803/3322919402", 2024-09-20T19:08:22.702 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.142:6802/3322919402", 2024-09-20T19:08:22.702 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T19:08:22.702 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T19:08:22.702 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T19:08:22.702 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.702 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.702 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.702 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 5, 2024-09-20T19:08:22.702 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "b3e4e21b-639f-4d25-8bbf-ea8922a9222a", 2024-09-20T19:08:22.702 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T19:08:22.702 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T19:08:22.703 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T19:08:22.703 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T19:08:22.703 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T19:08:22.703 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T19:08:22.703 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 38, 2024-09-20T19:08:22.703 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 462, 2024-09-20T19:08:22.703 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T19:08:22.703 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T19:08:22.703 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T19:08:22.703 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:08:22.703 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.703 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:08:22.703 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6804", 2024-09-20T19:08:22.703 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2500466067 2024-09-20T19:08:22.704 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.704 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.704 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T19:08:22.704 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:08:22.704 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.704 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:08:22.704 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6805", 2024-09-20T19:08:22.704 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2500466067 2024-09-20T19:08:22.704 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.704 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.704 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T19:08:22.705 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:08:22.705 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.705 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:08:22.705 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6807", 2024-09-20T19:08:22.705 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2500466067 2024-09-20T19:08:22.705 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.705 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.705 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T19:08:22.705 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:08:22.705 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.705 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:08:22.705 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6806", 2024-09-20T19:08:22.705 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2500466067 2024-09-20T19:08:22.705 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.706 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.706 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.706 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.142:6804/2500466067", 2024-09-20T19:08:22.706 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.142:6805/2500466067", 2024-09-20T19:08:22.706 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.142:6807/2500466067", 2024-09-20T19:08:22.706 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.142:6806/2500466067", 2024-09-20T19:08:22.706 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T19:08:22.706 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T19:08:22.706 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T19:08:22.706 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.706 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.706 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.707 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 6, 2024-09-20T19:08:22.707 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "c5db7107-ccb4-4ae2-9e63-f444897942e3", 2024-09-20T19:08:22.707 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T19:08:22.707 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T19:08:22.707 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T19:08:22.707 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T19:08:22.707 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T19:08:22.707 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T19:08:22.707 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 45, 2024-09-20T19:08:22.707 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 458, 2024-09-20T19:08:22.707 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T19:08:22.707 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T19:08:22.707 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T19:08:22.707 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:08:22.707 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.708 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:08:22.708 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6808", 2024-09-20T19:08:22.708 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1688186226 2024-09-20T19:08:22.708 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.708 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.708 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T19:08:22.708 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:08:22.708 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.708 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:08:22.708 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6809", 2024-09-20T19:08:22.708 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1688186226 2024-09-20T19:08:22.708 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.708 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.709 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T19:08:22.709 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:08:22.709 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.709 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:08:22.709 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6811", 2024-09-20T19:08:22.709 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1688186226 2024-09-20T19:08:22.709 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.709 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.709 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T19:08:22.709 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:08:22.709 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.710 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:08:22.710 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6810", 2024-09-20T19:08:22.710 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1688186226 2024-09-20T19:08:22.710 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.710 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.710 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.142:6808/1688186226", 2024-09-20T19:08:22.710 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.142:6809/1688186226", 2024-09-20T19:08:22.710 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.142:6811/1688186226", 2024-09-20T19:08:22.710 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.142:6810/1688186226", 2024-09-20T19:08:22.710 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T19:08:22.710 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T19:08:22.710 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T19:08:22.710 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.711 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.711 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 7, 2024-09-20T19:08:22.711 INFO:teuthology.orchestra.run.smithi083.stdout: "uuid": "07d16806-e2c8-4acb-b284-d75701e61622", 2024-09-20T19:08:22.711 INFO:teuthology.orchestra.run.smithi083.stdout: "up": 1, 2024-09-20T19:08:22.711 INFO:teuthology.orchestra.run.smithi083.stdout: "in": 1, 2024-09-20T19:08:22.711 INFO:teuthology.orchestra.run.smithi083.stdout: "weight": 1, 2024-09-20T19:08:22.711 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_affinity": 1, 2024-09-20T19:08:22.711 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_begin": 0, 2024-09-20T19:08:22.711 INFO:teuthology.orchestra.run.smithi083.stdout: "last_clean_end": 0, 2024-09-20T19:08:22.711 INFO:teuthology.orchestra.run.smithi083.stdout: "up_from": 53, 2024-09-20T19:08:22.711 INFO:teuthology.orchestra.run.smithi083.stdout: "up_thru": 459, 2024-09-20T19:08:22.711 INFO:teuthology.orchestra.run.smithi083.stdout: "down_at": 0, 2024-09-20T19:08:22.712 INFO:teuthology.orchestra.run.smithi083.stdout: "lost_at": 0, 2024-09-20T19:08:22.712 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addrs": { 2024-09-20T19:08:22.712 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:08:22.712 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.712 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:08:22.712 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6812", 2024-09-20T19:08:22.712 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3679770984 2024-09-20T19:08:22.712 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.712 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.712 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addrs": { 2024-09-20T19:08:22.712 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:08:22.712 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.712 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:08:22.712 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6813", 2024-09-20T19:08:22.713 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3679770984 2024-09-20T19:08:22.713 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.713 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.713 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addrs": { 2024-09-20T19:08:22.713 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:08:22.713 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.713 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:08:22.713 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6815", 2024-09-20T19:08:22.713 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3679770984 2024-09-20T19:08:22.713 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.713 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.713 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addrs": { 2024-09-20T19:08:22.714 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-09-20T19:08:22.714 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.714 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-09-20T19:08:22.714 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.142:6814", 2024-09-20T19:08:22.714 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3679770984 2024-09-20T19:08:22.714 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.714 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.714 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.714 INFO:teuthology.orchestra.run.smithi083.stdout: "public_addr": "172.21.15.142:6812/3679770984", 2024-09-20T19:08:22.714 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_addr": "172.21.15.142:6813/3679770984", 2024-09-20T19:08:22.714 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_back_addr": "172.21.15.142:6815/3679770984", 2024-09-20T19:08:22.714 INFO:teuthology.orchestra.run.smithi083.stdout: "heartbeat_front_addr": "172.21.15.142:6814/3679770984", 2024-09-20T19:08:22.714 INFO:teuthology.orchestra.run.smithi083.stdout: "state": [ 2024-09-20T19:08:22.714 INFO:teuthology.orchestra.run.smithi083.stdout: "exists", 2024-09-20T19:08:22.715 INFO:teuthology.orchestra.run.smithi083.stdout: "up" 2024-09-20T19:08:22.715 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.715 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.715 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T19:08:22.715 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_xinfo": [ 2024-09-20T19:08:22.715 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.715 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 0, 2024-09-20T19:08:22.715 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T19:08:22.715 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T19:08:22.715 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T19:08:22.715 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T19:08:22.715 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T19:08:22.715 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T19:08:01.536016+0000", 2024-09-20T19:08:22.715 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T19:08:22.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.716 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.716 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 1, 2024-09-20T19:08:22.716 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T19:08:22.716 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T19:08:22.716 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T19:08:22.716 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T19:08:22.716 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T19:08:22.716 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T19:07:29.884045+0000", 2024-09-20T19:08:22.716 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T19:08:22.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.716 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.717 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 2, 2024-09-20T19:08:22.717 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T19:08:22.717 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T19:08:22.717 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T19:08:22.717 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T19:08:22.717 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T19:08:22.717 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T19:07:51.419802+0000", 2024-09-20T19:08:22.717 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T19:08:22.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.717 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.717 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 3, 2024-09-20T19:08:22.717 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T19:08:22.717 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T19:08:22.717 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T19:08:22.717 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T19:08:22.718 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T19:08:22.718 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T19:07:53.111767+0000", 2024-09-20T19:08:22.718 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T19:08:22.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.718 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.718 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 4, 2024-09-20T19:08:22.718 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T19:08:22.718 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T19:08:22.718 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T19:08:22.718 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T19:08:22.718 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T19:08:22.718 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T19:07:53.146125+0000", 2024-09-20T19:08:22.718 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 291 2024-09-20T19:08:22.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.719 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.719 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 5, 2024-09-20T19:08:22.719 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T19:08:22.719 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T19:08:22.719 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T19:08:22.719 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T19:08:22.719 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T19:08:22.719 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T19:08:20.812960+0000", 2024-09-20T19:08:22.719 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T19:08:22.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.719 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.719 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 6, 2024-09-20T19:08:22.719 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T19:08:22.719 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T19:08:22.720 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T19:08:22.720 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T19:08:22.720 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T19:08:22.720 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T19:07:56.073055+0000", 2024-09-20T19:08:22.720 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T19:08:22.720 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.720 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.720 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": 7, 2024-09-20T19:08:22.720 INFO:teuthology.orchestra.run.smithi083.stdout: "down_stamp": "0.000000", 2024-09-20T19:08:22.720 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_probability": 0, 2024-09-20T19:08:22.720 INFO:teuthology.orchestra.run.smithi083.stdout: "laggy_interval": 0, 2024-09-20T19:08:22.720 INFO:teuthology.orchestra.run.smithi083.stdout: "features": 4540719139924082687, 2024-09-20T19:08:22.720 INFO:teuthology.orchestra.run.smithi083.stdout: "old_weight": 0, 2024-09-20T19:08:22.721 INFO:teuthology.orchestra.run.smithi083.stdout: "last_purged_snaps_scrub": "2024-09-20T19:07:45.313116+0000", 2024-09-20T19:08:22.721 INFO:teuthology.orchestra.run.smithi083.stdout: "dead_epoch": 0 2024-09-20T19:08:22.721 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.721 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T19:08:22.721 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_upmap": [ 2024-09-20T19:08:22.721 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.721 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "3.c", 2024-09-20T19:08:22.721 INFO:teuthology.orchestra.run.smithi083.stdout: "osds": [ 2024-09-20T19:08:22.721 INFO:teuthology.orchestra.run.smithi083.stdout: 2, 2024-09-20T19:08:22.721 INFO:teuthology.orchestra.run.smithi083.stdout: 7, 2024-09-20T19:08:22.721 INFO:teuthology.orchestra.run.smithi083.stdout: 6 2024-09-20T19:08:22.721 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.721 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.721 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.722 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "4.2", 2024-09-20T19:08:22.722 INFO:teuthology.orchestra.run.smithi083.stdout: "osds": [ 2024-09-20T19:08:22.722 INFO:teuthology.orchestra.run.smithi083.stdout: 5, 2024-09-20T19:08:22.722 INFO:teuthology.orchestra.run.smithi083.stdout: 7, 2024-09-20T19:08:22.722 INFO:teuthology.orchestra.run.smithi083.stdout: 4 2024-09-20T19:08:22.722 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.722 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.722 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T19:08:22.722 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_upmap_items": [ 2024-09-20T19:08:22.722 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.722 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "4.4", 2024-09-20T19:08:22.722 INFO:teuthology.orchestra.run.smithi083.stdout: "mappings": [ 2024-09-20T19:08:22.722 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.723 INFO:teuthology.orchestra.run.smithi083.stdout: "from": 3, 2024-09-20T19:08:22.723 INFO:teuthology.orchestra.run.smithi083.stdout: "to": 5 2024-09-20T19:08:22.723 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.723 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.723 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.723 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "4.f", 2024-09-20T19:08:22.723 INFO:teuthology.orchestra.run.smithi083.stdout: "mappings": [ 2024-09-20T19:08:22.723 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.723 INFO:teuthology.orchestra.run.smithi083.stdout: "from": 1, 2024-09-20T19:08:22.723 INFO:teuthology.orchestra.run.smithi083.stdout: "to": 2 2024-09-20T19:08:22.723 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.723 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.724 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.724 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "4.14", 2024-09-20T19:08:22.724 INFO:teuthology.orchestra.run.smithi083.stdout: "mappings": [ 2024-09-20T19:08:22.724 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.724 INFO:teuthology.orchestra.run.smithi083.stdout: "from": 1, 2024-09-20T19:08:22.724 INFO:teuthology.orchestra.run.smithi083.stdout: "to": 5 2024-09-20T19:08:22.724 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.724 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.724 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.724 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "4.18", 2024-09-20T19:08:22.724 INFO:teuthology.orchestra.run.smithi083.stdout: "mappings": [ 2024-09-20T19:08:22.724 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.724 INFO:teuthology.orchestra.run.smithi083.stdout: "from": 1, 2024-09-20T19:08:22.725 INFO:teuthology.orchestra.run.smithi083.stdout: "to": 2 2024-09-20T19:08:22.725 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.725 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.725 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.725 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.725 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "4.1e", 2024-09-20T19:08:22.725 INFO:teuthology.orchestra.run.smithi083.stdout: "mappings": [ 2024-09-20T19:08:22.725 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.725 INFO:teuthology.orchestra.run.smithi083.stdout: "from": 3, 2024-09-20T19:08:22.725 INFO:teuthology.orchestra.run.smithi083.stdout: "to": 2 2024-09-20T19:08:22.725 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.725 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.725 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.725 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.725 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "5.1c", 2024-09-20T19:08:22.726 INFO:teuthology.orchestra.run.smithi083.stdout: "mappings": [ 2024-09-20T19:08:22.726 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.726 INFO:teuthology.orchestra.run.smithi083.stdout: "from": 4, 2024-09-20T19:08:22.726 INFO:teuthology.orchestra.run.smithi083.stdout: "to": 3 2024-09-20T19:08:22.726 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.726 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.726 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.726 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T19:08:22.726 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_upmap_primaries": [], 2024-09-20T19:08:22.726 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_temp": [ 2024-09-20T19:08:22.726 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.726 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "11.10", 2024-09-20T19:08:22.726 INFO:teuthology.orchestra.run.smithi083.stdout: "osds": [ 2024-09-20T19:08:22.726 INFO:teuthology.orchestra.run.smithi083.stdout: 1, 2024-09-20T19:08:22.727 INFO:teuthology.orchestra.run.smithi083.stdout: 0 2024-09-20T19:08:22.727 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.727 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.727 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "11.11", 2024-09-20T19:08:22.727 INFO:teuthology.orchestra.run.smithi083.stdout: "osds": [ 2024-09-20T19:08:22.727 INFO:teuthology.orchestra.run.smithi083.stdout: 0, 2024-09-20T19:08:22.727 INFO:teuthology.orchestra.run.smithi083.stdout: 3 2024-09-20T19:08:22.727 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.727 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.727 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "11.13", 2024-09-20T19:08:22.727 INFO:teuthology.orchestra.run.smithi083.stdout: "osds": [ 2024-09-20T19:08:22.727 INFO:teuthology.orchestra.run.smithi083.stdout: 1, 2024-09-20T19:08:22.728 INFO:teuthology.orchestra.run.smithi083.stdout: 0 2024-09-20T19:08:22.728 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.728 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:22.728 INFO:teuthology.orchestra.run.smithi083.stdout: "pgid": "11.19", 2024-09-20T19:08:22.728 INFO:teuthology.orchestra.run.smithi083.stdout: "osds": [ 2024-09-20T19:08:22.728 INFO:teuthology.orchestra.run.smithi083.stdout: 0, 2024-09-20T19:08:22.728 INFO:teuthology.orchestra.run.smithi083.stdout: 2 2024-09-20T19:08:22.728 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:22.728 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.728 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T19:08:22.728 INFO:teuthology.orchestra.run.smithi083.stdout: "primary_temp": [], 2024-09-20T19:08:22.728 INFO:teuthology.orchestra.run.smithi083.stdout: "blocklist": { 2024-09-20T19:08:22.728 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/3474253603": "2024-09-21T18:44:32.434180+0000", 2024-09-20T19:08:22.729 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:6800/3525856829": "2024-09-21T18:44:32.434180+0000", 2024-09-20T19:08:22.729 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/220746284": "2024-09-21T18:44:32.434180+0000", 2024-09-20T19:08:22.729 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/3875445393": "2024-09-21T18:44:56.985620+0000", 2024-09-20T19:08:22.729 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/3009092807": "2024-09-21T18:44:56.985620+0000", 2024-09-20T19:08:22.729 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/4267132131": "2024-09-21T18:44:56.985620+0000", 2024-09-20T19:08:22.729 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/3684382972": "2024-09-21T18:50:14.729526+0000", 2024-09-20T19:08:22.729 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/3132933178": "2024-09-21T18:50:14.729526+0000", 2024-09-20T19:08:22.729 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:6800/1007846146": "2024-09-21T18:50:14.729526+0000", 2024-09-20T19:08:22.729 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:6800/2985596191": "2024-09-21T18:44:56.985620+0000", 2024-09-20T19:08:22.729 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/2410768464": "2024-09-21T18:50:14.729526+0000", 2024-09-20T19:08:22.729 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/1868694097": "2024-09-21T18:50:14.729526+0000", 2024-09-20T19:08:22.729 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/3777331122": "2024-09-21T18:44:32.434180+0000", 2024-09-20T19:08:22.729 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:0/2161728215": "2024-09-21T18:50:14.729526+0000" 2024-09-20T19:08:22.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.729 INFO:teuthology.orchestra.run.smithi083.stdout: "range_blocklist": {}, 2024-09-20T19:08:22.730 INFO:teuthology.orchestra.run.smithi083.stdout: "erasure_code_profiles": { 2024-09-20T19:08:22.730 INFO:teuthology.orchestra.run.smithi083.stdout: "default": { 2024-09-20T19:08:22.730 INFO:teuthology.orchestra.run.smithi083.stdout: "crush-failure-domain": "osd", 2024-09-20T19:08:22.730 INFO:teuthology.orchestra.run.smithi083.stdout: "k": "2", 2024-09-20T19:08:22.730 INFO:teuthology.orchestra.run.smithi083.stdout: "m": "1", 2024-09-20T19:08:22.730 INFO:teuthology.orchestra.run.smithi083.stdout: "plugin": "jerasure", 2024-09-20T19:08:22.730 INFO:teuthology.orchestra.run.smithi083.stdout: "technique": "reed_sol_van" 2024-09-20T19:08:22.730 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.730 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:22.730 INFO:teuthology.orchestra.run.smithi083.stdout: "removed_snaps_queue": [], 2024-09-20T19:08:22.730 INFO:teuthology.orchestra.run.smithi083.stdout: "new_removed_snaps": [], 2024-09-20T19:08:22.730 INFO:teuthology.orchestra.run.smithi083.stdout: "new_purged_snaps": [], 2024-09-20T19:08:22.730 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_node_flags": {}, 2024-09-20T19:08:22.730 INFO:teuthology.orchestra.run.smithi083.stdout: "device_class_flags": {}, 2024-09-20T19:08:22.731 INFO:teuthology.orchestra.run.smithi083.stdout: "stretch_mode": { 2024-09-20T19:08:22.731 INFO:teuthology.orchestra.run.smithi083.stdout: "stretch_mode_enabled": false, 2024-09-20T19:08:22.731 INFO:teuthology.orchestra.run.smithi083.stdout: "stretch_bucket_count": 0, 2024-09-20T19:08:22.731 INFO:teuthology.orchestra.run.smithi083.stdout: "degraded_stretch_mode": 0, 2024-09-20T19:08:22.731 INFO:teuthology.orchestra.run.smithi083.stdout: "recovering_stretch_mode": 0, 2024-09-20T19:08:22.731 INFO:teuthology.orchestra.run.smithi083.stdout: "stretch_mode_bucket": 0 2024-09-20T19:08:22.731 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:22.731 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T19:08:22.731 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:22.757 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:22.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:22.758 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:22.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:22 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:08:22.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:22 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:08:22.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:22 smithi142 ceph-mon[32508]: pgmap v941: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 4.8 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 966 op/s; 3/238914 objects degraded (0.001%); 6163/238914 objects misplaced (2.580%); 7.4 MiB/s, 118 objects/s recovering 2024-09-20T19:08:22.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:22 smithi142 ceph-mon[32508]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:08:22.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:22 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T19:08:22.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:22 smithi142 ceph-mon[32508]: osdmap e470: 8 total, 8 up, 8 in 2024-09-20T19:08:22.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:22 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/381534363' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-20T19:08:22.859 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:22.888 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:22.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:22.889 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:22.978 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:08:22.990 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:23.038 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:23.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:23.039 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:23.140 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:23.169 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:23.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:23.170 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:23.271 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:23.298 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:23.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:23.299 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:23.399 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:23.439 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:23.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:23.441 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:23.541 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:23.572 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:23.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:23.573 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:23.674 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:23.701 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:23.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:23.702 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:23.773 INFO:tasks.thrashosds.thrasher:j is {'epoch': 470, 'fsid': '1ed5a50a-7780-11ef-baf5-efdc52797490', 'created': '2024-09-20T18:44:10.399800+0000', 'modified': '2024-09-20T19:08:21.291027+0000', 'last_up_change': '2024-09-20T19:01:21.809732+0000', 'last_in_change': '2024-09-20T19:04:20.299488+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 19, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 11, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-20T18:47:15.285795+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 11, 'pg_placement_num': 11, 'pg_placement_num_target': 11, 'pg_num_target': 11, 'pg_num_pending': 11, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '458', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '455', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.180000066757202, 'score_stable': 2.180000066757202, 'optimal_score': 1, 'raw_score_acting': 2.180000066757202, 'raw_score_stable': 2.180000066757202, 'primary_affinity_weighted': 0.9860343933105469, 'average_primary_affinity': 0.9900000095367432, 'average_primary_affinity_weighted': 0.9900000095367432}}, {'pool': 2, 'pool_name': 'datapool', 'create_time': '2024-09-20T18:49:43.452984+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 61, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.6500000953674316, 'score_stable': 2.6500000953674316, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 2.3299999237060547, 'raw_score_stable': 2.3299999237060547, 'primary_affinity_weighted': 0.9860343933105469, 'average_primary_affinity': 0.9900000095367432, 'average_primary_affinity_weighted': 0.9900000095367432}}, {'pool': 3, 'pool_name': '.rgw.root', 'create_time': '2024-09-20T18:49:44.217011+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9860343933105469, 'average_primary_affinity': 0.9900000095367432, 'average_primary_affinity_weighted': 0.9900000095367432}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-20T18:49:46.007275+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9860343933105469, 'average_primary_affinity': 0.9900000095367432, 'average_primary_affinity_weighted': 0.9900000095367432}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-20T18:49:47.961258+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9860343933105469, 'average_primary_affinity': 0.9900000095367432, 'average_primary_affinity_weighted': 0.9900000095367432}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-20T18:49:50.011276+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.9860343933105469, 'average_primary_affinity': 0.9900000095367432, 'average_primary_affinity_weighted': 0.9900000095367432}}, {'pool': 11, 'pool_name': 'unique_pool_4', 'create_time': '2024-09-20T19:06:16.062904+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 26, 'pg_placement_num': 26, 'pg_placement_num_target': 26, 'pg_num_target': 26, 'pg_num_pending': 26, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '443', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '441', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5399999618530273, 'score_stable': 1.5399999618530273, 'optimal_score': 1, 'raw_score_acting': 1.5399999618530273, 'raw_score_stable': 1.5399999618530273, 'primary_affinity_weighted': 0.9860343933105469, 'average_primary_affinity': 0.9900000095367432, 'average_primary_affinity_weighted': 0.9900000095367432}}], 'osds': [{'osd': 0, 'uuid': 'c714e1ec-f0c3-4054-9792-61b84e53eaa8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 461, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6801', 'nonce': 2574084049}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6802', 'nonce': 2574084049}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6804', 'nonce': 2574084049}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6803', 'nonce': 2574084049}]}, 'public_addr': '172.21.15.83:6801/2574084049', 'cluster_addr': '172.21.15.83:6802/2574084049', 'heartbeat_back_addr': '172.21.15.83:6804/2574084049', 'heartbeat_front_addr': '172.21.15.83:6803/2574084049', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '9dc0f2a8-bafa-4a42-88f5-0301dd58f78d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.888275146484375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 458, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6805', 'nonce': 4213285798}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6806', 'nonce': 4213285798}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6808', 'nonce': 4213285798}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6807', 'nonce': 4213285798}]}, 'public_addr': '172.21.15.83:6805/4213285798', 'cluster_addr': '172.21.15.83:6806/4213285798', 'heartbeat_back_addr': '172.21.15.83:6808/4213285798', 'heartbeat_front_addr': '172.21.15.83:6807/4213285798', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '448ab692-ec41-44dc-b2d5-dbefd255b3fd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 458, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6809', 'nonce': 805947571}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6810', 'nonce': 805947571}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6812', 'nonce': 805947571}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6811', 'nonce': 805947571}]}, 'public_addr': '172.21.15.83:6809/805947571', 'cluster_addr': '172.21.15.83:6810/805947571', 'heartbeat_back_addr': '172.21.15.83:6812/805947571', 'heartbeat_front_addr': '172.21.15.83:6811/805947571', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '7cda5e50-e22e-4d4a-823b-53c8f5061193', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 25, 'up_thru': 458, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6813', 'nonce': 987237318}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6814', 'nonce': 987237318}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6816', 'nonce': 987237318}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6815', 'nonce': 987237318}]}, 'public_addr': '172.21.15.83:6813/987237318', 'cluster_addr': '172.21.15.83:6814/987237318', 'heartbeat_back_addr': '172.21.15.83:6816/987237318', 'heartbeat_front_addr': '172.21.15.83:6815/987237318', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '3d62654d-d31d-4310-9934-49e694dd17d9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 31, 'last_clean_end': 291, 'up_from': 304, 'up_thru': 458, 'down_at': 292, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6800', 'nonce': 3322919402}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6801', 'nonce': 3322919402}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6803', 'nonce': 3322919402}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6802', 'nonce': 3322919402}]}, 'public_addr': '172.21.15.142:6800/3322919402', 'cluster_addr': '172.21.15.142:6801/3322919402', 'heartbeat_back_addr': '172.21.15.142:6803/3322919402', 'heartbeat_front_addr': '172.21.15.142:6802/3322919402', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'b3e4e21b-639f-4d25-8bbf-ea8922a9222a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 38, 'up_thru': 462, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6804', 'nonce': 2500466067}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6805', 'nonce': 2500466067}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6807', 'nonce': 2500466067}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6806', 'nonce': 2500466067}]}, 'public_addr': '172.21.15.142:6804/2500466067', 'cluster_addr': '172.21.15.142:6805/2500466067', 'heartbeat_back_addr': '172.21.15.142:6807/2500466067', 'heartbeat_front_addr': '172.21.15.142:6806/2500466067', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'c5db7107-ccb4-4ae2-9e63-f444897942e3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 45, 'up_thru': 458, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6808', 'nonce': 1688186226}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6809', 'nonce': 1688186226}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6811', 'nonce': 1688186226}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6810', 'nonce': 1688186226}]}, 'public_addr': '172.21.15.142:6808/1688186226', 'cluster_addr': '172.21.15.142:6809/1688186226', 'heartbeat_back_addr': '172.21.15.142:6811/1688186226', 'heartbeat_front_addr': '172.21.15.142:6810/1688186226', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '07d16806-e2c8-4acb-b284-d75701e61622', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 53, 'up_thru': 459, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6812', 'nonce': 3679770984}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6813', 'nonce': 3679770984}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6815', 'nonce': 3679770984}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6814', 'nonce': 3679770984}]}, 'public_addr': '172.21.15.142:6812/3679770984', 'cluster_addr': '172.21.15.142:6813/3679770984', 'heartbeat_back_addr': '172.21.15.142:6815/3679770984', 'heartbeat_front_addr': '172.21.15.142:6814/3679770984', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T19:08:01.536016+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T19:07:29.884045+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T19:07:51.419802+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T19:07:53.111767+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T19:07:53.146125+0000', 'dead_epoch': 291}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T19:08:20.812960+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T19:07:56.073055+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T19:07:45.313116+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.c', 'osds': [2, 7, 6]}, {'pgid': '4.2', 'osds': [5, 7, 4]}], 'pg_upmap_items': [{'pgid': '4.4', 'mappings': [{'from': 3, 'to': 5}]}, {'pgid': '4.f', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.14', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.18', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.1e', 'mappings': [{'from': 3, 'to': 2}]}, {'pgid': '5.1c', 'mappings': [{'from': 4, 'to': 3}]}], 'pg_upmap_primaries': [], 'pg_temp': [{'pgid': '11.10', 'osds': [1, 0]}, {'pgid': '11.11', 'osds': [0, 3]}, {'pgid': '11.13', 'osds': [1, 0]}, {'pgid': '11.19', 'osds': [0, 2]}], 'primary_temp': [], 'blocklist': {'172.21.15.83:0/3474253603': '2024-09-21T18:44:32.434180+0000', '172.21.15.83:6800/3525856829': '2024-09-21T18:44:32.434180+0000', '172.21.15.83:0/220746284': '2024-09-21T18:44:32.434180+0000', '172.21.15.83:0/3875445393': '2024-09-21T18:44:56.985620+0000', '172.21.15.83:0/3009092807': '2024-09-21T18:44:56.985620+0000', '172.21.15.83:0/4267132131': '2024-09-21T18:44:56.985620+0000', '172.21.15.83:0/3684382972': '2024-09-21T18:50:14.729526+0000', '172.21.15.83:0/3132933178': '2024-09-21T18:50:14.729526+0000', '172.21.15.83:6800/1007846146': '2024-09-21T18:50:14.729526+0000', '172.21.15.83:6800/2985596191': '2024-09-21T18:44:56.985620+0000', '172.21.15.83:0/2410768464': '2024-09-21T18:50:14.729526+0000', '172.21.15.83:0/1868694097': '2024-09-21T18:50:14.729526+0000', '172.21.15.83:0/3777331122': '2024-09-21T18:44:32.434180+0000', '172.21.15.83:0/2161728215': '2024-09-21T18:50:14.729526+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-20T19:08:23.773 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T19:08:23.802 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:23.828 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:23.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:23.829 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:23.930 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:08:23.962 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:23.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:23.963 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:08:24.064 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:24.090 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:24.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:24.091 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:24.192 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:24.214 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:24.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:24.215 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:24.316 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:24.339 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:24.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:24.340 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:24.441 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:08:24.462 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T19:08:24.483 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:24.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:24.485 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:08:24.525 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:24 smithi083 ceph-mon[29421]: pgmap v943: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 4.7 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 947 op/s; 3/229776 objects degraded (0.001%); 4012/229776 objects misplaced (1.746%); 9.9 MiB/s, 157 objects/s recovering 2024-09-20T19:08:24.525 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:24 smithi083 ceph-mon[29421]: osdmap e471: 8 total, 8 up, 8 in 2024-09-20T19:08:24.526 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:24 smithi083 ceph-mon[29421]: 11.1 scrub starts 2024-09-20T19:08:24.526 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:24 smithi083 ceph-mon[36260]: pgmap v943: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 4.7 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 947 op/s; 3/229776 objects degraded (0.001%); 4012/229776 objects misplaced (1.746%); 9.9 MiB/s, 157 objects/s recovering 2024-09-20T19:08:24.526 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:24 smithi083 ceph-mon[36260]: osdmap e471: 8 total, 8 up, 8 in 2024-09-20T19:08:24.526 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:24 smithi083 ceph-mon[36260]: 11.1 scrub starts 2024-09-20T19:08:24.587 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:24.648 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T19:08:24.648 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:08:24.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:24.672 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:24.672 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:24.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:24 smithi142 ceph-mon[32508]: pgmap v943: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 4.7 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 947 op/s; 3/229776 objects degraded (0.001%); 4012/229776 objects misplaced (1.746%); 9.9 MiB/s, 157 objects/s recovering 2024-09-20T19:08:24.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:24 smithi142 ceph-mon[32508]: osdmap e471: 8 total, 8 up, 8 in 2024-09-20T19:08:24.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:24 smithi142 ceph-mon[32508]: 11.1 scrub starts 2024-09-20T19:08:24.773 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:24.812 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:24.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:24.813 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:24.814 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:08:24.814 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:08:24.814 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:08:24.815 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:08:24.815 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:08:24.815 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:08:24.815 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:08:24.815 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:08:24.815 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:08:24.815 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:08:24.815 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:08:24.815 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:08:24.843 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:08:24.843 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:08:24.881 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:08:24.881 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:08:24.915 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:24.951 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:24.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:24.952 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:25.053 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:25.097 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:25.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:25.098 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:25.198 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:08:25.239 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:25.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:25.240 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:08:25.341 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:08:25.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:25.384 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:25.384 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:08:25.485 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:25.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:25.515 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:25.515 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:25.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:25 smithi083 ceph-mon[29421]: 11.1 scrub ok 2024-09-20T19:08:25.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:25 smithi083 ceph-mon[29421]: osdmap e472: 8 total, 8 up, 8 in 2024-09-20T19:08:25.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:25 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/4165751361' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:08:25.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:25 smithi083 ceph-mon[29421]: 11.3 scrub starts 2024-09-20T19:08:25.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:25 smithi083 ceph-mon[36260]: 11.1 scrub ok 2024-09-20T19:08:25.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:25 smithi083 ceph-mon[36260]: osdmap e472: 8 total, 8 up, 8 in 2024-09-20T19:08:25.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:25 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/4165751361' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:08:25.542 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:25 smithi083 ceph-mon[36260]: 11.3 scrub starts 2024-09-20T19:08:25.616 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:25.625 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_historic_ops 2024-09-20T19:08:25.647 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:25.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:25.648 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:25.749 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:25.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:25 smithi142 ceph-mon[32508]: 11.1 scrub ok 2024-09-20T19:08:25.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:25 smithi142 ceph-mon[32508]: osdmap e472: 8 total, 8 up, 8 in 2024-09-20T19:08:25.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:25 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/4165751361' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:08:25.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:25 smithi142 ceph-mon[32508]: 11.3 scrub starts 2024-09-20T19:08:25.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:25.817 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:25.818 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:25.918 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:25.978 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:25.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:25.979 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:26.080 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:26.117 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:26.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:26.118 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:26.218 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:26.254 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:26.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:26.255 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:26.356 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:26.386 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:26.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:26.387 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:26.428 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:08:26.475 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:08:26.487 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:26.513 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:26.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:26.513 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:26.614 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:26.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:26.649 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:26.649 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:26.711 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:26 smithi083 ceph-mon[29421]: pgmap v946: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 4.7 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 733 op/s; 3/229776 objects degraded (0.001%); 4012/229776 objects misplaced (1.746%); 8.8 MiB/s, 140 objects/s recovering 2024-09-20T19:08:26.711 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:26 smithi083 ceph-mon[29421]: 11.3 scrub ok 2024-09-20T19:08:26.711 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:26 smithi083 ceph-mon[29421]: 11.14 scrub starts 2024-09-20T19:08:26.711 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:26 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 3/229776 objects degraded (0.001%), 1 pg degraded, 4 pgs undersized (PG_DEGRADED) 2024-09-20T19:08:26.711 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:26 smithi083 ceph-mon[36260]: pgmap v946: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 4.7 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 733 op/s; 3/229776 objects degraded (0.001%); 4012/229776 objects misplaced (1.746%); 8.8 MiB/s, 140 objects/s recovering 2024-09-20T19:08:26.711 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:26 smithi083 ceph-mon[36260]: 11.3 scrub ok 2024-09-20T19:08:26.711 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:26 smithi083 ceph-mon[36260]: 11.14 scrub starts 2024-09-20T19:08:26.711 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:26 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 3/229776 objects degraded (0.001%), 1 pg degraded, 4 pgs undersized (PG_DEGRADED) 2024-09-20T19:08:26.750 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:08:26.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:26 smithi142 ceph-mon[32508]: pgmap v946: 168 pgs: 1 active+recovery_wait+degraded, 4 active+recovering+undersized+remapped, 163 active+clean; 4.7 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 733 op/s; 3/229776 objects degraded (0.001%); 4012/229776 objects misplaced (1.746%); 8.8 MiB/s, 140 objects/s recovering 2024-09-20T19:08:26.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:26 smithi142 ceph-mon[32508]: 11.3 scrub ok 2024-09-20T19:08:26.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:26 smithi142 ceph-mon[32508]: 11.14 scrub starts 2024-09-20T19:08:26.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:26 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 3/229776 objects degraded (0.001%), 1 pg degraded, 4 pgs undersized (PG_DEGRADED) 2024-09-20T19:08:26.793 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:26.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:26.794 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:08:26.894 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:08:26.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:26.944 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:26.944 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:08:27.045 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:27.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:27.081 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:27.081 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:27.182 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:27.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:27.211 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:27.211 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:27.312 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:27.342 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:27.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:27.343 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:27.443 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:27.484 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:27.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:27.485 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:27.586 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:27.609 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:27.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:27.610 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:27.711 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:27.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:27.743 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:27.743 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:27.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:27 smithi142 ceph-mon[32508]: 11.14 scrub ok 2024-09-20T19:08:27.760 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:08:27 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:08:27.790 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:27 smithi083 ceph-mon[29421]: 11.14 scrub ok 2024-09-20T19:08:27.790 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:27 smithi083 ceph-mon[36260]: 11.14 scrub ok 2024-09-20T19:08:27.844 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:27.873 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:27.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:27.874 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:27.975 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:28.009 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:28.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:28.011 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:28.112 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:28.140 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:28.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:28.141 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:28.242 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:28.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:28.275 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:28.275 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:28.376 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:28.404 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-20T19:08:28.410 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:28.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:28.412 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:28.456 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T19:08:28.512 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:28.540 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:28.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:28.541 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:28.641 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:08:28.687 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:28 smithi083 ceph-mon[29421]: pgmap v947: 168 pgs: 4 active+recovering+undersized+remapped, 164 active+clean; 4.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s; 4012/218538 objects misplaced (1.836%) 2024-09-20T19:08:28.687 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:28 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:08:28.687 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:28 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3479245972' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:08:28.687 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:28 smithi083 ceph-mon[36260]: pgmap v947: 168 pgs: 4 active+recovering+undersized+remapped, 164 active+clean; 4.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s; 4012/218538 objects misplaced (1.836%) 2024-09-20T19:08:28.687 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:28 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:08:28.687 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:28 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3479245972' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:08:28.708 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:28.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:28.709 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:08:28.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:28 smithi142 ceph-mon[32508]: pgmap v947: 168 pgs: 4 active+recovering+undersized+remapped, 164 active+clean; 4.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s; 4012/218538 objects misplaced (1.836%) 2024-09-20T19:08:28.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:28 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:08:28.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:28 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3479245972' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:08:28.809 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:28.835 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:28.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:28.836 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:28.938 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:28.963 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:28.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:28.964 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:29.065 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:29.094 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:29.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:29.095 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:29.196 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:08:29.240 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:29.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:29.241 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:08:29.342 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:08:29.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:29.390 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:29.390 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:08:29.479 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T19:08:29.480 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:29 smithi083 ceph-mon[29421]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T19:08:29.480 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:29 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3479245972' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:08:29.480 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:29 smithi083 ceph-mon[29421]: osdmap e473: 8 total, 8 up, 8 in 2024-09-20T19:08:29.481 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:29 smithi083 ceph-mon[29421]: 1.7 scrub starts 2024-09-20T19:08:29.481 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:29 smithi083 ceph-mon[29421]: 1.7 scrub ok 2024-09-20T19:08:29.481 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:29 smithi083 ceph-mon[36260]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T19:08:29.481 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:29 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3479245972' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:08:29.482 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:29 smithi083 ceph-mon[36260]: osdmap e473: 8 total, 8 up, 8 in 2024-09-20T19:08:29.482 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:29 smithi083 ceph-mon[36260]: 1.7 scrub starts 2024-09-20T19:08:29.482 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:29 smithi083 ceph-mon[36260]: 1.7 scrub ok 2024-09-20T19:08:29.491 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:29.494 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-20T19:08:29.494 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:08:29.545 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:29.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:29.546 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:29.646 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:08:29.686 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:29.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:29.687 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:08:29.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:29 smithi142 ceph-mon[32508]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T19:08:29.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:29 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3479245972' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:08:29.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:29 smithi142 ceph-mon[32508]: osdmap e473: 8 total, 8 up, 8 in 2024-09-20T19:08:29.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:29 smithi142 ceph-mon[32508]: 1.7 scrub starts 2024-09-20T19:08:29.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:29 smithi142 ceph-mon[32508]: 1.7 scrub ok 2024-09-20T19:08:29.788 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:29.828 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:29.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:29.829 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:29.930 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:29.956 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:29.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:29.957 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:30.058 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:30.084 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:30.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:30.086 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:30.186 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:30.222 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:30.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:30.223 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:30.324 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:30.353 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:30.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:30.355 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:30.455 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:08:30.489 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:30.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:30.491 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:08:30.592 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:08:30.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:30.617 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:30.617 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:08:30.717 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:30.725 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:30 smithi083 ceph-mon[29421]: from='client.26378 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:08:30.725 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:30 smithi083 ceph-mon[29421]: pgmap v949: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 4.4 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 852 B/s rd, 0 op/s; 1910/214797 objects misplaced (0.889%) 2024-09-20T19:08:30.725 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:30 smithi083 ceph-mon[36260]: from='client.26378 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:08:30.725 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:30 smithi083 ceph-mon[36260]: pgmap v949: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 4.4 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 852 B/s rd, 0 op/s; 1910/214797 objects misplaced (0.889%) 2024-09-20T19:08:30.754 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:30.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:30.755 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:30.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:30 smithi142 ceph-mon[32508]: from='client.26378 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:08:30.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:30 smithi142 ceph-mon[32508]: pgmap v949: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 4.4 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 852 B/s rd, 0 op/s; 1910/214797 objects misplaced (0.889%) 2024-09-20T19:08:30.856 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:30.890 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:30.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:30.891 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:30.991 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:31.027 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:31.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:31.028 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:31.040 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:08:30 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:08:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:08:31.130 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:31.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:31.165 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:31.165 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:31.266 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:31.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:31.296 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:31.297 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:31.397 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:31.428 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:31.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:31.429 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:31.530 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:08:31.578 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:31.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:31.579 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:08:31.680 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:31.708 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:31.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:31.709 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:31.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:31 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 3 pgs undersized (PG_DEGRADED) 2024-09-20T19:08:31.790 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:31 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 3 pgs undersized (PG_DEGRADED) 2024-09-20T19:08:31.791 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:31 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 3 pgs undersized (PG_DEGRADED) 2024-09-20T19:08:31.810 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:31.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:31.839 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:31.840 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:31.940 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:31.978 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:31.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:31.979 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:32.080 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:32.115 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:32.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:32.117 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:32.217 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:32.238 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:32.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:32.239 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:32.340 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:32.371 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:32.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:32.372 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:32.473 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:08:32.504 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:32.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:32.505 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:08:32.606 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:32.643 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:32.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:32.644 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:32.745 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:32.771 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:32.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:32.772 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:32.873 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:32.895 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:32.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:32.895 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:32.996 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:33.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:32 smithi142 ceph-mon[32508]: pgmap v950: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 4.2 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 649 B/s rd, 0 op/s; 1910/209349 objects misplaced (0.912%) 2024-09-20T19:08:33.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:32 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:08:33.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:32 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:08:33.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:32 smithi142 ceph-mon[32508]: 1.3 scrub starts 2024-09-20T19:08:33.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:32 smithi142 ceph-mon[32508]: 1.3 scrub ok 2024-09-20T19:08:33.017 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:33.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:33.018 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:33.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:32 smithi083 ceph-mon[29421]: pgmap v950: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 4.2 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 649 B/s rd, 0 op/s; 1910/209349 objects misplaced (0.912%) 2024-09-20T19:08:33.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:32 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:08:33.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:32 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:08:33.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:32 smithi083 ceph-mon[29421]: 1.3 scrub starts 2024-09-20T19:08:33.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:32 smithi083 ceph-mon[29421]: 1.3 scrub ok 2024-09-20T19:08:33.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:32 smithi083 ceph-mon[36260]: pgmap v950: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 4.2 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 649 B/s rd, 0 op/s; 1910/209349 objects misplaced (0.912%) 2024-09-20T19:08:33.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:32 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:08:33.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:32 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:08:33.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:32 smithi083 ceph-mon[36260]: 1.3 scrub starts 2024-09-20T19:08:33.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:32 smithi083 ceph-mon[36260]: 1.3 scrub ok 2024-09-20T19:08:33.119 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:33.140 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:33.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:33.141 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:33.242 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:33.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:33.265 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:33.265 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:33.366 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:33.388 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:33.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:33.389 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:33.489 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:33.512 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:33.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:33.513 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:33.614 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:33.636 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:33.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:33.637 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:33.738 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:33.774 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:33.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:33.775 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:33.876 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:33.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:33.899 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:33.899 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:34.000 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:34.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:33 smithi142 ceph-mon[32508]: osdmap e474: 8 total, 8 up, 8 in 2024-09-20T19:08:34.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:33 smithi142 ceph-mon[32508]: 1.a scrub starts 2024-09-20T19:08:34.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:33 smithi142 ceph-mon[32508]: 1.a scrub ok 2024-09-20T19:08:34.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:33 smithi142 ceph-mon[32508]: 1.6 scrub starts 2024-09-20T19:08:34.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:33 smithi142 ceph-mon[32508]: 1.6 scrub ok 2024-09-20T19:08:34.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:33 smithi142 ceph-mon[32508]: 1.5 scrub starts 2024-09-20T19:08:34.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:33 smithi142 ceph-mon[32508]: 1.5 scrub ok 2024-09-20T19:08:34.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:33 smithi142 ceph-mon[32508]: 11.10 scrub starts 2024-09-20T19:08:34.029 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:34.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:34.029 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:34.040 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:33 smithi083 ceph-mon[36260]: osdmap e474: 8 total, 8 up, 8 in 2024-09-20T19:08:34.040 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:33 smithi083 ceph-mon[36260]: 1.a scrub starts 2024-09-20T19:08:34.040 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:33 smithi083 ceph-mon[36260]: 1.a scrub ok 2024-09-20T19:08:34.040 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:33 smithi083 ceph-mon[36260]: 1.6 scrub starts 2024-09-20T19:08:34.040 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:33 smithi083 ceph-mon[36260]: 1.6 scrub ok 2024-09-20T19:08:34.040 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:33 smithi083 ceph-mon[36260]: 1.5 scrub starts 2024-09-20T19:08:34.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:33 smithi083 ceph-mon[36260]: 1.5 scrub ok 2024-09-20T19:08:34.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:33 smithi083 ceph-mon[36260]: 11.10 scrub starts 2024-09-20T19:08:34.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:33 smithi083 ceph-mon[29421]: osdmap e474: 8 total, 8 up, 8 in 2024-09-20T19:08:34.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:33 smithi083 ceph-mon[29421]: 1.a scrub starts 2024-09-20T19:08:34.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:33 smithi083 ceph-mon[29421]: 1.a scrub ok 2024-09-20T19:08:34.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:33 smithi083 ceph-mon[29421]: 1.6 scrub starts 2024-09-20T19:08:34.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:33 smithi083 ceph-mon[29421]: 1.6 scrub ok 2024-09-20T19:08:34.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:33 smithi083 ceph-mon[29421]: 1.5 scrub starts 2024-09-20T19:08:34.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:33 smithi083 ceph-mon[29421]: 1.5 scrub ok 2024-09-20T19:08:34.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:33 smithi083 ceph-mon[29421]: 11.10 scrub starts 2024-09-20T19:08:34.130 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:34.156 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:34.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:34.157 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:34.258 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:08:34.278 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:34.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:34.279 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:08:34.380 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:34.402 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:34.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:34.403 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:34.504 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:34.525 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:34.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:34.526 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:34.627 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:08:34.648 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:34.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:34.649 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:08:34.750 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:34.779 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:34.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:34.780 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:34.881 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:08:34.923 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:34.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:34.924 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:08:35.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:34 smithi142 ceph-mon[32508]: pgmap v952: 168 pgs: 1 peering, 2 active+recovering+undersized+remapped, 165 active+clean; 4.0 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 728/197805 objects misplaced (0.368%) 2024-09-20T19:08:35.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:34 smithi142 ceph-mon[32508]: osdmap e475: 8 total, 8 up, 8 in 2024-09-20T19:08:35.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:34 smithi142 ceph-mon[32508]: 11.10 scrub ok 2024-09-20T19:08:35.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:34 smithi142 ceph-mon[32508]: 5.1 scrub starts 2024-09-20T19:08:35.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:34 smithi142 ceph-mon[32508]: 5.1 scrub ok 2024-09-20T19:08:35.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:34 smithi142 ceph-mon[32508]: 11.15 scrub starts 2024-09-20T19:08:35.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:34 smithi142 ceph-mon[32508]: 5.1d scrub starts 2024-09-20T19:08:35.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:34 smithi142 ceph-mon[32508]: 1.4 scrub starts 2024-09-20T19:08:35.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:34 smithi142 ceph-mon[32508]: 5.1d scrub ok 2024-09-20T19:08:35.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:34 smithi142 ceph-mon[32508]: 1.4 scrub ok 2024-09-20T19:08:35.024 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:35.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:34 smithi083 ceph-mon[29421]: pgmap v952: 168 pgs: 1 peering, 2 active+recovering+undersized+remapped, 165 active+clean; 4.0 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 728/197805 objects misplaced (0.368%) 2024-09-20T19:08:35.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:34 smithi083 ceph-mon[29421]: osdmap e475: 8 total, 8 up, 8 in 2024-09-20T19:08:35.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:34 smithi083 ceph-mon[29421]: 11.10 scrub ok 2024-09-20T19:08:35.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:34 smithi083 ceph-mon[29421]: 5.1 scrub starts 2024-09-20T19:08:35.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:34 smithi083 ceph-mon[29421]: 5.1 scrub ok 2024-09-20T19:08:35.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:34 smithi083 ceph-mon[29421]: 11.15 scrub starts 2024-09-20T19:08:35.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:34 smithi083 ceph-mon[29421]: 5.1d scrub starts 2024-09-20T19:08:35.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:34 smithi083 ceph-mon[29421]: 1.4 scrub starts 2024-09-20T19:08:35.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:34 smithi083 ceph-mon[29421]: 5.1d scrub ok 2024-09-20T19:08:35.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:34 smithi083 ceph-mon[29421]: 1.4 scrub ok 2024-09-20T19:08:35.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:34 smithi083 ceph-mon[36260]: pgmap v952: 168 pgs: 1 peering, 2 active+recovering+undersized+remapped, 165 active+clean; 4.0 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 728/197805 objects misplaced (0.368%) 2024-09-20T19:08:35.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:34 smithi083 ceph-mon[36260]: osdmap e475: 8 total, 8 up, 8 in 2024-09-20T19:08:35.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:34 smithi083 ceph-mon[36260]: 11.10 scrub ok 2024-09-20T19:08:35.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:34 smithi083 ceph-mon[36260]: 5.1 scrub starts 2024-09-20T19:08:35.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:34 smithi083 ceph-mon[36260]: 5.1 scrub ok 2024-09-20T19:08:35.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:34 smithi083 ceph-mon[36260]: 11.15 scrub starts 2024-09-20T19:08:35.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:34 smithi083 ceph-mon[36260]: 5.1d scrub starts 2024-09-20T19:08:35.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:34 smithi083 ceph-mon[36260]: 1.4 scrub starts 2024-09-20T19:08:35.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:34 smithi083 ceph-mon[36260]: 5.1d scrub ok 2024-09-20T19:08:35.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:34 smithi083 ceph-mon[36260]: 1.4 scrub ok 2024-09-20T19:08:35.057 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:35.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:35.058 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:35.159 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:35.195 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:35.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:35.196 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:35.297 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:35.332 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:35.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:35.333 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:35.433 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:35.471 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:35.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:35.472 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:35.572 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:35.605 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:35.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:35.606 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:35.707 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:08:35.744 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:35.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:35.745 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:08:35.845 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:35.884 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:35.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:35.885 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:35.986 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:36.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:35 smithi142 ceph-mon[32508]: 11.15 scrub ok 2024-09-20T19:08:36.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:35 smithi142 ceph-mon[32508]: osdmap e476: 8 total, 8 up, 8 in 2024-09-20T19:08:36.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:35 smithi142 ceph-mon[32508]: 1.1 scrub starts 2024-09-20T19:08:36.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:35 smithi142 ceph-mon[32508]: 1.1 scrub ok 2024-09-20T19:08:36.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:35 smithi142 ceph-mon[32508]: 11.4 scrub starts 2024-09-20T19:08:36.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:35 smithi142 ceph-mon[32508]: 11.19 scrub starts 2024-09-20T19:08:36.022 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:36.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:36.023 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:36.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:35 smithi083 ceph-mon[29421]: 11.15 scrub ok 2024-09-20T19:08:36.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:35 smithi083 ceph-mon[29421]: osdmap e476: 8 total, 8 up, 8 in 2024-09-20T19:08:36.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:35 smithi083 ceph-mon[29421]: 1.1 scrub starts 2024-09-20T19:08:36.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:35 smithi083 ceph-mon[29421]: 1.1 scrub ok 2024-09-20T19:08:36.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:35 smithi083 ceph-mon[29421]: 11.4 scrub starts 2024-09-20T19:08:36.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:35 smithi083 ceph-mon[29421]: 11.19 scrub starts 2024-09-20T19:08:36.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:35 smithi083 ceph-mon[36260]: 11.15 scrub ok 2024-09-20T19:08:36.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:35 smithi083 ceph-mon[36260]: osdmap e476: 8 total, 8 up, 8 in 2024-09-20T19:08:36.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:35 smithi083 ceph-mon[36260]: 1.1 scrub starts 2024-09-20T19:08:36.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:35 smithi083 ceph-mon[36260]: 1.1 scrub ok 2024-09-20T19:08:36.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:35 smithi083 ceph-mon[36260]: 11.4 scrub starts 2024-09-20T19:08:36.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:35 smithi083 ceph-mon[36260]: 11.19 scrub starts 2024-09-20T19:08:36.124 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:08:36.162 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:36.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:36.163 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:08:36.264 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:36.297 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:36.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:36.298 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:36.361 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:08:36.387 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T19:08:36.387 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:08:36.399 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:36.427 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:36.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:36.428 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:36.529 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:36.560 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:36.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:36.561 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:36.661 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:36.701 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:36.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:36.702 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:36.803 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:36.842 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:36.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:36.843 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:36.944 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:08:36.977 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:36.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:36.979 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:08:37.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:36 smithi142 ceph-mon[32508]: pgmap v955: 168 pgs: 1 peering, 2 active+recovering+undersized+remapped, 165 active+clean; 4.0 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 0 B/s wr, 940 op/s; 728/197805 objects misplaced (0.368%); 6.1 MiB/s, 98 objects/s recovering 2024-09-20T19:08:37.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:36 smithi142 ceph-mon[32508]: 11.4 scrub ok 2024-09-20T19:08:37.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:36 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 2 pgs undersized (PG_DEGRADED) 2024-09-20T19:08:37.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:36 smithi142 ceph-mon[32508]: 11.19 scrub ok 2024-09-20T19:08:37.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:36 smithi142 ceph-mon[32508]: 11.12 scrub starts 2024-09-20T19:08:37.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:36 smithi142 ceph-mon[32508]: 11.18 scrub starts 2024-09-20T19:08:37.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:36 smithi142 ceph-mon[32508]: 4.1d scrub starts 2024-09-20T19:08:37.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:36 smithi142 ceph-mon[32508]: 4.1d scrub ok 2024-09-20T19:08:37.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:36 smithi142 ceph-mon[32508]: 4.6 scrub starts 2024-09-20T19:08:37.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:36 smithi142 ceph-mon[32508]: 4.6 scrub ok 2024-09-20T19:08:37.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:36 smithi083 ceph-mon[29421]: pgmap v955: 168 pgs: 1 peering, 2 active+recovering+undersized+remapped, 165 active+clean; 4.0 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 0 B/s wr, 940 op/s; 728/197805 objects misplaced (0.368%); 6.1 MiB/s, 98 objects/s recovering 2024-09-20T19:08:37.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:36 smithi083 ceph-mon[29421]: 11.4 scrub ok 2024-09-20T19:08:37.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:36 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 2 pgs undersized (PG_DEGRADED) 2024-09-20T19:08:37.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:36 smithi083 ceph-mon[29421]: 11.19 scrub ok 2024-09-20T19:08:37.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:36 smithi083 ceph-mon[29421]: 11.12 scrub starts 2024-09-20T19:08:37.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:36 smithi083 ceph-mon[29421]: 11.18 scrub starts 2024-09-20T19:08:37.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:36 smithi083 ceph-mon[29421]: 4.1d scrub starts 2024-09-20T19:08:37.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:36 smithi083 ceph-mon[29421]: 4.1d scrub ok 2024-09-20T19:08:37.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:36 smithi083 ceph-mon[29421]: 4.6 scrub starts 2024-09-20T19:08:37.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:36 smithi083 ceph-mon[29421]: 4.6 scrub ok 2024-09-20T19:08:37.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:36 smithi083 ceph-mon[36260]: pgmap v955: 168 pgs: 1 peering, 2 active+recovering+undersized+remapped, 165 active+clean; 4.0 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 0 B/s wr, 940 op/s; 728/197805 objects misplaced (0.368%); 6.1 MiB/s, 98 objects/s recovering 2024-09-20T19:08:37.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:36 smithi083 ceph-mon[36260]: 11.4 scrub ok 2024-09-20T19:08:37.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:36 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 2 pgs undersized (PG_DEGRADED) 2024-09-20T19:08:37.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:36 smithi083 ceph-mon[36260]: 11.19 scrub ok 2024-09-20T19:08:37.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:36 smithi083 ceph-mon[36260]: 11.12 scrub starts 2024-09-20T19:08:37.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:36 smithi083 ceph-mon[36260]: 11.18 scrub starts 2024-09-20T19:08:37.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:36 smithi083 ceph-mon[36260]: 4.1d scrub starts 2024-09-20T19:08:37.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:36 smithi083 ceph-mon[36260]: 4.1d scrub ok 2024-09-20T19:08:37.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:36 smithi083 ceph-mon[36260]: 4.6 scrub starts 2024-09-20T19:08:37.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:36 smithi083 ceph-mon[36260]: 4.6 scrub ok 2024-09-20T19:08:37.079 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:08:37.114 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:37.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:37.115 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:08:37.216 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:37.236 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":948,"stamp":"2024-09-20T19:08:28.400781+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4679670323,"num_objects":71599,"num_object_clones":0,"num_object_copies":214797,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1910,"num_objects_unfound":0,"num_objects_dirty":71599,"num_whiteouts":0,"num_read":2362,"num_read_kb":2222,"num_write":110315,"num_write_kb":5135306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16870,"num_bytes_recovered":1088139498,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151660,"ondisk_log_size":151660,"up":504,"acting":501,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":517,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":20519352,"kb_used_data":14918568,"kb_used_omap":16,"kb_used_meta":5599023,"kb_avail":729441864,"statfs":{"total":767960285184,"available":746948468736,"internally_reserved":0,"allocated":15276613632,"data_stored":15271871217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16790,"internal_metadata":5733400170},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,9,4,6],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-526843904,"num_objects":-8039,"num_object_clones":0,"num_object_copies":-24117,"num_objects_missing_on_primary":-1,"num_objects_missing":-1,"num_objects_degraded":-3,"num_objects_misplaced":-4253,"num_objects_unfound":0,"num_objects_dirty":-8039,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":-4614,"num_write_kb":-623296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-225,"num_bytes_recovered":-14548448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.106332"},"pg_stats":[{"pgid":"11.17","version":"472'6167","reported_seq":12215,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:26.423870+0000","last_change":"2024-09-20T19:07:43.964356+0000","last_active":"2024-09-20T19:08:26.423870+0000","last_peered":"2024-09-20T19:08:26.423870+0000","last_clean":"2024-09-20T19:08:26.423870+0000","last_became_active":"2024-09-20T19:07:43.959947+0000","last_became_peered":"2024-09-20T19:07:43.959947+0000","last_unstale":"2024-09-20T19:08:26.423870+0000","last_undegraded":"2024-09-20T19:08:26.423870+0000","last_fullsized":"2024-09-20T19:08:26.423870+0000","mapping_epoch":459,"log_start":"426'110","ondisk_log_start":"426'110","created":441,"last_epoch_clean":460,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":6057,"log_dups_size":0,"ondisk_log_size":6057,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":145653760,"num_objects":2223,"num_object_clones":0,"num_object_copies":6669,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2223,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4263,"num_write_kb":207520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1972,"num_bytes_recovered":129236992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,5],"acting":[7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":876,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.336947+0000","last_change":"2024-09-20T19:01:31.817967+0000","last_active":"2024-09-20T19:08:24.336947+0000","last_peered":"2024-09-20T19:08:24.336947+0000","last_clean":"2024-09-20T19:08:24.336947+0000","last_became_active":"2024-09-20T18:49:45.943201+0000","last_became_peered":"2024-09-20T18:49:45.943201+0000","last_unstale":"2024-09-20T19:08:24.336947+0000","last_undegraded":"2024-09-20T19:08:24.336947+0000","last_fullsized":"2024-09-20T19:08:24.336947+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:31.817909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:31.817909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"273'15","reported_seq":224,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.328973+0000","last_change":"2024-09-20T19:06:46.500942+0000","last_active":"2024-09-20T19:08:24.328973+0000","last_peered":"2024-09-20T19:08:24.328973+0000","last_clean":"2024-09-20T19:08:24.328973+0000","last_became_active":"2024-09-20T19:04:20.925853+0000","last_became_peered":"2024-09-20T19:04:20.925853+0000","last_unstale":"2024-09-20T19:08:24.328973+0000","last_undegraded":"2024-09-20T19:08:24.328973+0000","last_fullsized":"2024-09-20T19:08:24.328973+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:06:46.500881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:06:46.500881+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":88,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.343504+0000","last_change":"2024-09-20T19:06:40.320839+0000","last_active":"2024-09-20T19:08:24.343504+0000","last_peered":"2024-09-20T19:08:24.343504+0000","last_clean":"2024-09-20T19:08:24.343504+0000","last_became_active":"2024-09-20T19:06:40.320505+0000","last_became_peered":"2024-09-20T19:06:40.320505+0000","last_unstale":"2024-09-20T19:08:24.343504+0000","last_undegraded":"2024-09-20T19:08:24.343504+0000","last_fullsized":"2024-09-20T19:08:24.343504+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:49.016006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:49.016006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":206,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.329392+0000","last_change":"2024-09-20T19:04:20.881918+0000","last_active":"2024-09-20T19:08:24.329392+0000","last_peered":"2024-09-20T19:08:24.329392+0000","last_clean":"2024-09-20T19:08:24.329392+0000","last_became_active":"2024-09-20T19:04:20.880905+0000","last_became_peered":"2024-09-20T19:04:20.880905+0000","last_unstale":"2024-09-20T19:08:24.329392+0000","last_undegraded":"2024-09-20T19:08:24.329392+0000","last_fullsized":"2024-09-20T19:08:24.329392+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:02.084375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:02.084375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"472'6179","reported_seq":14201,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:25.790161+0000","last_change":"2024-09-20T19:07:50.027667+0000","last_active":"2024-09-20T19:08:25.790161+0000","last_peered":"2024-09-20T19:08:25.790161+0000","last_clean":"2024-09-20T19:08:25.790161+0000","last_became_active":"2024-09-20T19:07:50.022839+0000","last_became_peered":"2024-09-20T19:07:50.022839+0000","last_unstale":"2024-09-20T19:08:25.790161+0000","last_undegraded":"2024-09-20T19:08:25.790161+0000","last_fullsized":"2024-09-20T19:08:25.790161+0000","mapping_epoch":462,"log_start":"0'0","ondisk_log_start":"0'0","created":441,"last_epoch_clean":463,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":6179,"log_dups_size":0,"ondisk_log_size":6179,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":150929408,"num_objects":2303,"num_object_clones":0,"num_object_copies":6909,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4263,"num_write_kb":210112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1972,"num_bytes_recovered":129236992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":876,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.333290+0000","last_change":"2024-09-20T19:02:02.441633+0000","last_active":"2024-09-20T19:08:24.333290+0000","last_peered":"2024-09-20T19:08:24.333290+0000","last_clean":"2024-09-20T19:08:24.333290+0000","last_became_active":"2024-09-20T18:59:05.081081+0000","last_became_peered":"2024-09-20T18:59:05.081081+0000","last_unstale":"2024-09-20T19:08:24.333290+0000","last_undegraded":"2024-09-20T19:08:24.333290+0000","last_fullsized":"2024-09-20T19:08:24.333290+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:02.441577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:02.441577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":890,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.333590+0000","last_change":"2024-09-20T19:04:37.540824+0000","last_active":"2024-09-20T19:08:24.333590+0000","last_peered":"2024-09-20T19:08:24.333590+0000","last_clean":"2024-09-20T19:08:24.333590+0000","last_became_active":"2024-09-20T18:59:05.072675+0000","last_became_peered":"2024-09-20T18:59:05.072675+0000","last_unstale":"2024-09-20T19:08:24.333590+0000","last_undegraded":"2024-09-20T19:08:24.333590+0000","last_fullsized":"2024-09-20T19:08:24.333590+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:37.540760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:04:37.540760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"274'27","reported_seq":921,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.332576+0000","last_change":"2024-09-20T19:05:50.750134+0000","last_active":"2024-09-20T19:08:24.332576+0000","last_peered":"2024-09-20T19:08:24.332576+0000","last_clean":"2024-09-20T19:08:24.332576+0000","last_became_active":"2024-09-20T18:49:47.952150+0000","last_became_peered":"2024-09-20T18:49:47.952150+0000","last_unstale":"2024-09-20T19:08:24.332576+0000","last_undegraded":"2024-09-20T19:08:24.332576+0000","last_fullsized":"2024-09-20T19:08:24.332576+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'27","last_scrub_stamp":"2024-09-20T19:05:50.750050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:50.750050+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":214,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.329001+0000","last_change":"2024-09-20T19:06:25.218221+0000","last_active":"2024-09-20T19:08:24.329001+0000","last_peered":"2024-09-20T19:08:24.329001+0000","last_clean":"2024-09-20T19:08:24.329001+0000","last_became_active":"2024-09-20T19:04:20.925394+0000","last_became_peered":"2024-09-20T19:04:20.925394+0000","last_unstale":"2024-09-20T19:08:24.329001+0000","last_undegraded":"2024-09-20T19:08:24.329001+0000","last_fullsized":"2024-09-20T19:08:24.329001+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:25.218150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:06:25.218150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"472'6110","reported_seq":13764,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:26.792528+0000","last_change":"2024-09-20T19:07:40.725604+0000","last_active":"2024-09-20T19:08:26.792528+0000","last_peered":"2024-09-20T19:08:26.792528+0000","last_clean":"2024-09-20T19:08:26.792528+0000","last_became_active":"2024-09-20T19:07:40.719995+0000","last_became_peered":"2024-09-20T19:07:40.719995+0000","last_unstale":"2024-09-20T19:08:26.792528+0000","last_undegraded":"2024-09-20T19:08:26.792528+0000","last_fullsized":"2024-09-20T19:08:26.792528+0000","mapping_epoch":455,"log_start":"0'0","ondisk_log_start":"0'0","created":441,"last_epoch_clean":456,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":6110,"log_dups_size":0,"ondisk_log_size":6110,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":147980288,"num_objects":2258,"num_object_clones":0,"num_object_copies":6774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4208,"num_write_kb":206912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1880,"num_bytes_recovered":123207680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"59'1","reported_seq":951,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.344311+0000","last_change":"2024-09-20T19:04:20.963496+0000","last_active":"2024-09-20T19:08:24.344311+0000","last_peered":"2024-09-20T19:08:24.344311+0000","last_clean":"2024-09-20T19:08:24.344311+0000","last_became_active":"2024-09-20T19:04:20.914145+0000","last_became_peered":"2024-09-20T19:04:20.914145+0000","last_unstale":"2024-09-20T19:08:24.344311+0000","last_undegraded":"2024-09-20T19:08:24.344311+0000","last_fullsized":"2024-09-20T19:08:24.344311+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:01:28.592653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:28.592653+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"273'15","reported_seq":216,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.329502+0000","last_change":"2024-09-20T19:04:20.984223+0000","last_active":"2024-09-20T19:08:24.329502+0000","last_peered":"2024-09-20T19:08:24.329502+0000","last_clean":"2024-09-20T19:08:24.329502+0000","last_became_active":"2024-09-20T19:04:20.898235+0000","last_became_peered":"2024-09-20T19:04:20.898235+0000","last_unstale":"2024-09-20T19:08:24.329502+0000","last_undegraded":"2024-09-20T19:08:24.329502+0000","last_fullsized":"2024-09-20T19:08:24.329502+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:02:42.114331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:42.114331+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":882,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.344619+0000","last_change":"2024-09-20T19:06:47.942245+0000","last_active":"2024-09-20T19:08:24.344619+0000","last_peered":"2024-09-20T19:08:24.344619+0000","last_clean":"2024-09-20T19:08:24.344619+0000","last_became_active":"2024-09-20T18:49:49.961188+0000","last_became_peered":"2024-09-20T18:49:49.961188+0000","last_unstale":"2024-09-20T19:08:24.344619+0000","last_undegraded":"2024-09-20T19:08:24.344619+0000","last_fullsized":"2024-09-20T19:08:24.344619+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:47.942182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:06:47.942182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":863,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.336591+0000","last_change":"2024-09-20T19:06:53.956421+0000","last_active":"2024-09-20T19:08:24.336591+0000","last_peered":"2024-09-20T19:08:24.336591+0000","last_clean":"2024-09-20T19:08:24.336591+0000","last_became_active":"2024-09-20T18:49:51.966715+0000","last_became_peered":"2024-09-20T18:49:51.966715+0000","last_unstale":"2024-09-20T19:08:24.336591+0000","last_undegraded":"2024-09-20T19:08:24.336591+0000","last_fullsized":"2024-09-20T19:08:24.336591+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:53.956349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:06:53.956349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"472'6196","reported_seq":12249,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:26.990986+0000","last_change":"2024-09-20T19:08:26.337018+0000","last_active":"2024-09-20T19:08:26.990986+0000","last_peered":"2024-09-20T19:08:26.990986+0000","last_clean":"2024-09-20T19:08:26.990986+0000","last_became_active":"2024-09-20T19:07:36.015594+0000","last_became_peered":"2024-09-20T19:07:36.015594+0000","last_unstale":"2024-09-20T19:08:26.990986+0000","last_undegraded":"2024-09-20T19:08:26.990986+0000","last_fullsized":"2024-09-20T19:08:26.990986+0000","mapping_epoch":452,"log_start":"0'0","ondisk_log_start":"0'0","created":441,"last_epoch_clean":453,"parent":"0.0","parent_split_bits":5,"last_scrub":"472'6170","last_scrub_stamp":"2024-09-20T19:08:26.336764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:08:26.336764+0000","objects_scrubbed":2280,"log_size":6196,"log_dups_size":0,"ondisk_log_size":6196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:09:39.950097+0000","scrub_duration":519,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":146997248,"num_objects":2243,"num_object_clones":0,"num_object_copies":6729,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2243,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":899,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.344066+0000","last_change":"2024-09-20T19:04:20.879227+0000","last_active":"2024-09-20T19:08:24.344066+0000","last_peered":"2024-09-20T19:08:24.344066+0000","last_clean":"2024-09-20T19:08:24.344066+0000","last_became_active":"2024-09-20T19:04:20.878905+0000","last_became_peered":"2024-09-20T19:04:20.878905+0000","last_unstale":"2024-09-20T19:08:24.344066+0000","last_undegraded":"2024-09-20T19:08:24.344066+0000","last_fullsized":"2024-09-20T19:08:24.344066+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:48.520017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:48.520017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"273'7","reported_seq":220,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.329568+0000","last_change":"2024-09-20T19:05:53.980848+0000","last_active":"2024-09-20T19:08:24.329568+0000","last_peered":"2024-09-20T19:08:24.329568+0000","last_clean":"2024-09-20T19:08:24.329568+0000","last_became_active":"2024-09-20T19:04:20.888572+0000","last_became_peered":"2024-09-20T19:04:20.888572+0000","last_unstale":"2024-09-20T19:08:24.329568+0000","last_undegraded":"2024-09-20T19:08:24.329568+0000","last_fullsized":"2024-09-20T19:08:24.329568+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'7","last_scrub_stamp":"2024-09-20T19:05:53.980772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:53.980772+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":892,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.340744+0000","last_change":"2024-09-20T19:04:20.885873+0000","last_active":"2024-09-20T19:08:24.340744+0000","last_peered":"2024-09-20T19:08:24.340744+0000","last_clean":"2024-09-20T19:08:24.340744+0000","last_became_active":"2024-09-20T19:04:20.885417+0000","last_became_peered":"2024-09-20T19:04:20.885417+0000","last_unstale":"2024-09-20T19:08:24.340744+0000","last_undegraded":"2024-09-20T19:08:24.340744+0000","last_fullsized":"2024-09-20T19:08:24.340744+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:33.203312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:33.203312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":870,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.343510+0000","last_change":"2024-09-20T19:05:21.625378+0000","last_active":"2024-09-20T19:08:24.343510+0000","last_peered":"2024-09-20T19:08:24.343510+0000","last_clean":"2024-09-20T19:08:24.343510+0000","last_became_active":"2024-09-20T18:49:51.970663+0000","last_became_peered":"2024-09-20T18:49:51.970663+0000","last_unstale":"2024-09-20T19:08:24.343510+0000","last_undegraded":"2024-09-20T19:08:24.343510+0000","last_fullsized":"2024-09-20T19:08:24.343510+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:21.625311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:05:21.625311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"472'6312","reported_seq":13682,"reported_epoch":472,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-20T19:08:27.689699+0000","last_change":"2024-09-20T19:07:35.667329+0000","last_active":"2024-09-20T19:08:27.689699+0000","last_peered":"2024-09-20T19:08:27.689699+0000","last_clean":"2024-09-20T19:06:57.409541+0000","last_became_active":"2024-09-20T19:06:59.724211+0000","last_became_peered":"2024-09-20T19:06:59.724211+0000","last_unstale":"2024-09-20T19:08:27.689699+0000","last_undegraded":"2024-09-20T19:08:27.689699+0000","last_fullsized":"2024-09-20T19:06:59.451727+0000","mapping_epoch":444,"log_start":"426'98","ondisk_log_start":"426'98","created":441,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":6214,"log_dups_size":0,"ondisk_log_size":6214,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":140640256,"num_objects":2146,"num_object_clones":0,"num_object_copies":6438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":702,"num_objects_unfound":0,"num_objects_dirty":2146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4390,"num_write_kb":209152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1812,"num_bytes_recovered":118751232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[1,0],"avail_no_missing":["1","6","7"],"object_location_counts":[{"shards":"0,1","objects":1912},{"shards":"1,6,7","objects":234}],"blocked_by":[],"up_primary":5,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":215,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.329956+0000","last_change":"2024-09-20T19:06:45.481243+0000","last_active":"2024-09-20T19:08:24.329956+0000","last_peered":"2024-09-20T19:08:24.329956+0000","last_clean":"2024-09-20T19:08:24.329956+0000","last_became_active":"2024-09-20T19:04:20.906109+0000","last_became_peered":"2024-09-20T19:04:20.906109+0000","last_unstale":"2024-09-20T19:08:24.329956+0000","last_undegraded":"2024-09-20T19:08:24.329956+0000","last_fullsized":"2024-09-20T19:08:24.329956+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:45.479545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:06:45.479545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":900,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.336874+0000","last_change":"2024-09-20T19:04:20.907057+0000","last_active":"2024-09-20T19:08:24.336874+0000","last_peered":"2024-09-20T19:08:24.336874+0000","last_clean":"2024-09-20T19:08:24.336874+0000","last_became_active":"2024-09-20T19:04:20.906691+0000","last_became_peered":"2024-09-20T19:04:20.906691+0000","last_unstale":"2024-09-20T19:08:24.336874+0000","last_undegraded":"2024-09-20T19:08:24.336874+0000","last_fullsized":"2024-09-20T19:08:24.336874+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:44.998147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:44.998147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"273'27","reported_seq":932,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.340354+0000","last_change":"2024-09-20T19:05:49.452928+0000","last_active":"2024-09-20T19:08:24.340354+0000","last_peered":"2024-09-20T19:08:24.340354+0000","last_clean":"2024-09-20T19:08:24.340354+0000","last_became_active":"2024-09-20T18:49:47.950939+0000","last_became_peered":"2024-09-20T18:49:47.950939+0000","last_unstale":"2024-09-20T19:08:24.340354+0000","last_undegraded":"2024-09-20T19:08:24.340354+0000","last_fullsized":"2024-09-20T19:08:24.340354+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'27","last_scrub_stamp":"2024-09-20T19:05:49.452861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:49.452861+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":883,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.334402+0000","last_change":"2024-09-20T19:04:20.890659+0000","last_active":"2024-09-20T19:08:24.334402+0000","last_peered":"2024-09-20T19:08:24.334402+0000","last_clean":"2024-09-20T19:08:24.334402+0000","last_became_active":"2024-09-20T19:04:20.890370+0000","last_became_peered":"2024-09-20T19:04:20.890370+0000","last_unstale":"2024-09-20T19:08:24.334402+0000","last_undegraded":"2024-09-20T19:08:24.334402+0000","last_fullsized":"2024-09-20T19:08:24.334402+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:05.888974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:05.888974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"472'6050","reported_seq":10001,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:26.319384+0000","last_change":"2024-09-20T19:07:25.940462+0000","last_active":"2024-09-20T19:08:26.319384+0000","last_peered":"2024-09-20T19:08:26.319384+0000","last_clean":"2024-09-20T19:08:26.319384+0000","last_became_active":"2024-09-20T19:07:25.936833+0000","last_became_peered":"2024-09-20T19:07:25.936833+0000","last_unstale":"2024-09-20T19:08:26.319384+0000","last_undegraded":"2024-09-20T19:08:26.319384+0000","last_fullsized":"2024-09-20T19:08:26.319384+0000","mapping_epoch":448,"log_start":"0'0","ondisk_log_start":"0'0","created":441,"last_epoch_clean":449,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":6050,"log_dups_size":0,"ondisk_log_size":6050,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":149946368,"num_objects":2288,"num_object_clones":0,"num_object_copies":6864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2288,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4198,"num_write_kb":207552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1909,"num_bytes_recovered":125108224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":885,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.333913+0000","last_change":"2024-09-20T19:06:17.937954+0000","last_active":"2024-09-20T19:08:24.333913+0000","last_peered":"2024-09-20T19:08:24.333913+0000","last_clean":"2024-09-20T19:08:24.333913+0000","last_became_active":"2024-09-20T18:59:05.084496+0000","last_became_peered":"2024-09-20T18:59:05.084496+0000","last_unstale":"2024-09-20T19:08:24.333913+0000","last_undegraded":"2024-09-20T19:08:24.333913+0000","last_fullsized":"2024-09-20T19:08:24.333913+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.937896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.937896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"59'1","reported_seq":210,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.329301+0000","last_change":"2024-09-20T19:04:20.905233+0000","last_active":"2024-09-20T19:08:24.329301+0000","last_peered":"2024-09-20T19:08:24.329301+0000","last_clean":"2024-09-20T19:08:24.329301+0000","last_became_active":"2024-09-20T19:04:20.887062+0000","last_became_peered":"2024-09-20T19:04:20.887062+0000","last_unstale":"2024-09-20T19:08:24.329301+0000","last_undegraded":"2024-09-20T19:08:24.329301+0000","last_fullsized":"2024-09-20T19:08:24.329301+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:03:08.218812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:03:08.218812+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":993,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"274'18","reported_seq":927,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.333608+0000","last_change":"2024-09-20T19:04:20.925690+0000","last_active":"2024-09-20T19:08:24.333608+0000","last_peered":"2024-09-20T19:08:24.333608+0000","last_clean":"2024-09-20T19:08:24.333608+0000","last_became_active":"2024-09-20T19:04:20.896343+0000","last_became_peered":"2024-09-20T19:04:20.896343+0000","last_unstale":"2024-09-20T19:08:24.333608+0000","last_undegraded":"2024-09-20T19:08:24.333608+0000","last_fullsized":"2024-09-20T19:08:24.333608+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:57:09.818999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:09.818999+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":214,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.329398+0000","last_change":"2024-09-20T19:06:16.148081+0000","last_active":"2024-09-20T19:08:24.329398+0000","last_peered":"2024-09-20T19:08:24.329398+0000","last_clean":"2024-09-20T19:08:24.329398+0000","last_became_active":"2024-09-20T19:04:20.889430+0000","last_became_peered":"2024-09-20T19:04:20.889430+0000","last_unstale":"2024-09-20T19:08:24.329398+0000","last_undegraded":"2024-09-20T19:08:24.329398+0000","last_fullsized":"2024-09-20T19:08:24.329398+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:16.148022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:06:16.148022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.11","version":"472'6270","reported_seq":11110,"reported_epoch":472,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-20T19:08:28.138759+0000","last_change":"2024-09-20T19:07:42.588580+0000","last_active":"2024-09-20T19:08:28.138759+0000","last_peered":"2024-09-20T19:08:28.138759+0000","last_clean":"2024-09-20T19:06:57.417320+0000","last_became_active":"2024-09-20T19:06:59.817960+0000","last_became_peered":"2024-09-20T19:06:59.817960+0000","last_unstale":"2024-09-20T19:08:28.138759+0000","last_undegraded":"2024-09-20T19:08:28.138759+0000","last_fullsized":"2024-09-20T19:06:59.454366+0000","mapping_epoch":444,"log_start":"0'0","ondisk_log_start":"0'0","created":441,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":6270,"log_dups_size":0,"ondisk_log_size":6270,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":143785984,"num_objects":2194,"num_object_clones":0,"num_object_copies":6582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":756,"num_objects_unfound":0,"num_objects_dirty":2194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4352,"num_write_kb":209472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1572,"num_bytes_recovered":103022592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[0,3],"avail_no_missing":["0","3","6"],"object_location_counts":[{"shards":"0,3","objects":1816},{"shards":"0,3,6","objects":378}],"blocked_by":[],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":864,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.340953+0000","last_change":"2024-09-20T19:02:28.294390+0000","last_active":"2024-09-20T19:08:24.340953+0000","last_peered":"2024-09-20T19:08:24.340953+0000","last_clean":"2024-09-20T19:08:24.340953+0000","last_became_active":"2024-09-20T18:49:51.970058+0000","last_became_peered":"2024-09-20T18:49:51.970058+0000","last_unstale":"2024-09-20T19:08:24.340953+0000","last_undegraded":"2024-09-20T19:08:24.340953+0000","last_fullsized":"2024-09-20T19:08:24.340953+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T19:02:28.294292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:28.294292+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":908,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.333464+0000","last_change":"2024-09-20T19:06:53.021433+0000","last_active":"2024-09-20T19:08:24.333464+0000","last_peered":"2024-09-20T19:08:24.333464+0000","last_clean":"2024-09-20T19:08:24.333464+0000","last_became_active":"2024-09-20T19:04:20.881466+0000","last_became_peered":"2024-09-20T19:04:20.881466+0000","last_unstale":"2024-09-20T19:08:24.333464+0000","last_undegraded":"2024-09-20T19:08:24.333464+0000","last_fullsized":"2024-09-20T19:08:24.333464+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:06:53.021363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:06:53.021363+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"274'20","reported_seq":946,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.337584+0000","last_change":"2024-09-20T19:06:42.777378+0000","last_active":"2024-09-20T19:08:24.337584+0000","last_peered":"2024-09-20T19:08:24.337584+0000","last_clean":"2024-09-20T19:08:24.337584+0000","last_became_active":"2024-09-20T19:03:16.968636+0000","last_became_peered":"2024-09-20T19:03:16.968636+0000","last_unstale":"2024-09-20T19:08:24.337584+0000","last_undegraded":"2024-09-20T19:08:24.337584+0000","last_fullsized":"2024-09-20T19:08:24.337584+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'20","last_scrub_stamp":"2024-09-20T19:06:42.777179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:06:42.777179+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"375'14","reported_seq":606,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.339187+0000","last_change":"2024-09-20T19:04:20.956210+0000","last_active":"2024-09-20T19:08:24.339187+0000","last_peered":"2024-09-20T19:08:24.339187+0000","last_clean":"2024-09-20T19:08:24.339187+0000","last_became_active":"2024-09-20T19:04:20.900782+0000","last_became_peered":"2024-09-20T19:04:20.900782+0000","last_unstale":"2024-09-20T19:08:24.339187+0000","last_undegraded":"2024-09-20T19:08:24.339187+0000","last_fullsized":"2024-09-20T19:08:24.339187+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"308'12","last_scrub_stamp":"2024-09-20T19:01:44.541156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:44.541156+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.a","version":"74'79","reported_seq":38,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.343578+0000","last_change":"2024-09-20T19:07:42.980723+0000","last_active":"2024-09-20T19:08:24.343578+0000","last_peered":"2024-09-20T19:08:24.343578+0000","last_clean":"2024-09-20T19:08:24.343578+0000","last_became_active":"2024-09-20T19:07:42.980316+0000","last_became_peered":"2024-09-20T19:07:42.980316+0000","last_unstale":"2024-09-20T19:08:24.343578+0000","last_undegraded":"2024-09-20T19:08:24.343578+0000","last_fullsized":"2024-09-20T19:08:24.343578+0000","mapping_epoch":458,"log_start":"0'0","ondisk_log_start":"0'0","created":455,"last_epoch_clean":459,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:03:14.368279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:03:14.368279+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53669,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":20,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":53669,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"472'6214","reported_seq":6364,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:27.607803+0000","last_change":"2024-09-20T19:06:58.386274+0000","last_active":"2024-09-20T19:08:27.607803+0000","last_peered":"2024-09-20T19:08:27.607803+0000","last_clean":"2024-09-20T19:08:27.607803+0000","last_became_active":"2024-09-20T19:06:56.482140+0000","last_became_peered":"2024-09-20T19:06:56.482140+0000","last_unstale":"2024-09-20T19:08:27.607803+0000","last_undegraded":"2024-09-20T19:08:27.607803+0000","last_fullsized":"2024-09-20T19:08:27.607803+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"443'3875","last_scrub_stamp":"2024-09-20T19:06:58.386214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:58.386214+0000","objects_scrubbed":2003,"log_size":6214,"log_dups_size":0,"ondisk_log_size":6214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":191,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":147193856,"num_objects":2246,"num_object_clones":0,"num_object_copies":6738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2246,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2339,"num_write_kb":82624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"274'27","reported_seq":957,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.340332+0000","last_change":"2024-09-20T19:04:20.920811+0000","last_active":"2024-09-20T19:08:24.340332+0000","last_peered":"2024-09-20T19:08:24.340332+0000","last_clean":"2024-09-20T19:08:24.340332+0000","last_became_active":"2024-09-20T19:04:20.897049+0000","last_became_peered":"2024-09-20T19:04:20.897049+0000","last_unstale":"2024-09-20T19:08:24.340332+0000","last_undegraded":"2024-09-20T19:08:24.340332+0000","last_fullsized":"2024-09-20T19:08:24.340332+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'27","last_scrub_stamp":"2024-09-20T19:02:27.897138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:27.897138+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":883,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.334058+0000","last_change":"2024-09-20T19:05:55.742165+0000","last_active":"2024-09-20T19:08:24.334058+0000","last_peered":"2024-09-20T19:08:24.334058+0000","last_clean":"2024-09-20T19:08:24.334058+0000","last_became_active":"2024-09-20T18:49:45.936104+0000","last_became_peered":"2024-09-20T18:49:45.936104+0000","last_unstale":"2024-09-20T19:08:24.334058+0000","last_undegraded":"2024-09-20T19:08:24.334058+0000","last_fullsized":"2024-09-20T19:08:24.334058+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:55.742106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:55.742106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"375'12","reported_seq":267,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.330025+0000","last_change":"2024-09-20T19:06:20.174206+0000","last_active":"2024-09-20T19:08:24.330025+0000","last_peered":"2024-09-20T19:08:24.330025+0000","last_clean":"2024-09-20T19:08:24.330025+0000","last_became_active":"2024-09-20T19:04:20.907228+0000","last_became_peered":"2024-09-20T19:04:20.907228+0000","last_unstale":"2024-09-20T19:08:24.330025+0000","last_undegraded":"2024-09-20T19:08:24.330025+0000","last_fullsized":"2024-09-20T19:08:24.330025+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"375'12","last_scrub_stamp":"2024-09-20T19:06:20.174160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:06:20.174160+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":863,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.344100+0000","last_change":"2024-09-20T19:02:22.736879+0000","last_active":"2024-09-20T19:08:24.344100+0000","last_peered":"2024-09-20T19:08:24.344100+0000","last_clean":"2024-09-20T19:08:24.344100+0000","last_became_active":"2024-09-20T18:49:51.971399+0000","last_became_peered":"2024-09-20T18:49:51.971399+0000","last_unstale":"2024-09-20T19:08:24.344100+0000","last_undegraded":"2024-09-20T19:08:24.344100+0000","last_fullsized":"2024-09-20T19:08:24.344100+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:22.736825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:22.736825+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"74'79","reported_seq":38,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.329721+0000","last_change":"2024-09-20T19:07:42.981112+0000","last_active":"2024-09-20T19:08:24.329721+0000","last_peered":"2024-09-20T19:08:24.329721+0000","last_clean":"2024-09-20T19:08:24.329721+0000","last_became_active":"2024-09-20T19:07:42.980650+0000","last_became_peered":"2024-09-20T19:07:42.980650+0000","last_unstale":"2024-09-20T19:08:24.329721+0000","last_undegraded":"2024-09-20T19:08:24.329721+0000","last_fullsized":"2024-09-20T19:08:24.329721+0000","mapping_epoch":458,"log_start":"0'0","ondisk_log_start":"0'0","created":455,"last_epoch_clean":459,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:03:14.368279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:03:14.368279+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":20,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":53670,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.f","version":"472'8492","reported_seq":8606,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:26.414284+0000","last_change":"2024-09-20T19:06:18.056190+0000","last_active":"2024-09-20T19:08:26.414284+0000","last_peered":"2024-09-20T19:08:26.414284+0000","last_clean":"2024-09-20T19:08:26.414284+0000","last_became_active":"2024-09-20T19:06:18.055847+0000","last_became_peered":"2024-09-20T19:06:18.055847+0000","last_unstale":"2024-09-20T19:08:26.414284+0000","last_undegraded":"2024-09-20T19:08:26.414284+0000","last_fullsized":"2024-09-20T19:08:26.414284+0000","mapping_epoch":422,"log_start":"447'4211","ondisk_log_start":"447'4211","created":422,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":4281,"log_dups_size":0,"ondisk_log_size":4281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":289406976,"num_objects":4416,"num_object_clones":0,"num_object_copies":13248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4416,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8492,"num_write_kb":413056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"273'30","reported_seq":946,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.332273+0000","last_change":"2024-09-20T19:06:54.927673+0000","last_active":"2024-09-20T19:08:24.332273+0000","last_peered":"2024-09-20T19:08:24.332273+0000","last_clean":"2024-09-20T19:08:24.332273+0000","last_became_active":"2024-09-20T18:49:48.499189+0000","last_became_peered":"2024-09-20T18:49:48.499189+0000","last_unstale":"2024-09-20T19:08:24.332273+0000","last_undegraded":"2024-09-20T19:08:24.332273+0000","last_fullsized":"2024-09-20T19:08:24.332273+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'30","last_scrub_stamp":"2024-09-20T19:06:54.927614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:06:54.927614+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":875,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.334216+0000","last_change":"2024-09-20T19:06:16.897724+0000","last_active":"2024-09-20T19:08:24.334216+0000","last_peered":"2024-09-20T19:08:24.334216+0000","last_clean":"2024-09-20T19:08:24.334216+0000","last_became_active":"2024-09-20T18:49:45.940544+0000","last_became_peered":"2024-09-20T18:49:45.940544+0000","last_unstale":"2024-09-20T19:08:24.334216+0000","last_undegraded":"2024-09-20T19:08:24.334216+0000","last_fullsized":"2024-09-20T19:08:24.334216+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:16.897677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:06:16.897677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":206,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.329200+0000","last_change":"2024-09-20T19:04:20.889280+0000","last_active":"2024-09-20T19:08:24.329200+0000","last_peered":"2024-09-20T19:08:24.329200+0000","last_clean":"2024-09-20T19:08:24.329200+0000","last_became_active":"2024-09-20T19:04:20.888422+0000","last_became_peered":"2024-09-20T19:04:20.888422+0000","last_unstale":"2024-09-20T19:08:24.329200+0000","last_undegraded":"2024-09-20T19:08:24.329200+0000","last_fullsized":"2024-09-20T19:08:24.329200+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:51.229201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:54:51.229201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":214,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.329506+0000","last_change":"2024-09-20T19:05:47.121946+0000","last_active":"2024-09-20T19:08:24.329506+0000","last_peered":"2024-09-20T19:08:24.329506+0000","last_clean":"2024-09-20T19:08:24.329506+0000","last_became_active":"2024-09-20T19:04:20.889418+0000","last_became_peered":"2024-09-20T19:04:20.889418+0000","last_unstale":"2024-09-20T19:08:24.329506+0000","last_undegraded":"2024-09-20T19:08:24.329506+0000","last_fullsized":"2024-09-20T19:08:24.329506+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:47.121842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:05:47.121842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"74'79","reported_seq":43,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.333202+0000","last_change":"2024-09-20T19:07:48.592259+0000","last_active":"2024-09-20T19:08:24.333202+0000","last_peered":"2024-09-20T19:08:24.333202+0000","last_clean":"2024-09-20T19:08:24.333202+0000","last_became_active":"2024-09-20T19:07:42.987705+0000","last_became_peered":"2024-09-20T19:07:42.987705+0000","last_unstale":"2024-09-20T19:08:24.333202+0000","last_undegraded":"2024-09-20T19:08:24.333202+0000","last_fullsized":"2024-09-20T19:08:24.333202+0000","mapping_epoch":458,"log_start":"0'0","ondisk_log_start":"0'0","created":455,"last_epoch_clean":461,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:03:14.368279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:03:14.368279+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":20,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":381350,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.e","version":"472'8453","reported_seq":8567,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:25.727324+0000","last_change":"2024-09-20T19:06:18.054020+0000","last_active":"2024-09-20T19:08:25.727324+0000","last_peered":"2024-09-20T19:08:25.727324+0000","last_clean":"2024-09-20T19:08:25.727324+0000","last_became_active":"2024-09-20T19:06:18.053343+0000","last_became_peered":"2024-09-20T19:06:18.053343+0000","last_unstale":"2024-09-20T19:08:25.727324+0000","last_undegraded":"2024-09-20T19:08:25.727324+0000","last_fullsized":"2024-09-20T19:08:25.727324+0000","mapping_epoch":422,"log_start":"445'4014","ondisk_log_start":"445'4014","created":422,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":4439,"log_dups_size":0,"ondisk_log_size":4439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":297336832,"num_objects":4537,"num_object_clones":0,"num_object_copies":13611,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4537,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8453,"num_write_kb":415680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"274'28","reported_seq":222,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.329098+0000","last_change":"2024-09-20T19:04:20.976682+0000","last_active":"2024-09-20T19:08:24.329098+0000","last_peered":"2024-09-20T19:08:24.329098+0000","last_clean":"2024-09-20T19:08:24.329098+0000","last_became_active":"2024-09-20T19:04:20.897444+0000","last_became_peered":"2024-09-20T19:04:20.897444+0000","last_unstale":"2024-09-20T19:08:24.329098+0000","last_undegraded":"2024-09-20T19:08:24.329098+0000","last_fullsized":"2024-09-20T19:08:24.329098+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'28","last_scrub_stamp":"2024-09-20T19:03:06.269623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:06.269623+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":41,"num_read_kb":34,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":910,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.337862+0000","last_change":"2024-09-20T19:05:47.629956+0000","last_active":"2024-09-20T19:08:24.337862+0000","last_peered":"2024-09-20T19:08:24.337862+0000","last_clean":"2024-09-20T19:08:24.337862+0000","last_became_active":"2024-09-20T19:04:20.906790+0000","last_became_peered":"2024-09-20T19:04:20.906790+0000","last_unstale":"2024-09-20T19:08:24.337862+0000","last_undegraded":"2024-09-20T19:08:24.337862+0000","last_fullsized":"2024-09-20T19:08:24.337862+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:05:47.629892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:47.629892+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"375'13","reported_seq":1147,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.332105+0000","last_change":"2024-09-20T19:04:20.916872+0000","last_active":"2024-09-20T19:08:24.332105+0000","last_peered":"2024-09-20T19:08:24.332105+0000","last_clean":"2024-09-20T19:08:24.332105+0000","last_became_active":"2024-09-20T19:04:20.896424+0000","last_became_peered":"2024-09-20T19:04:20.896424+0000","last_unstale":"2024-09-20T19:08:24.332105+0000","last_undegraded":"2024-09-20T19:08:24.332105+0000","last_fullsized":"2024-09-20T19:08:24.332105+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:58:23.114371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:23.114371+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":870,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.341013+0000","last_change":"2024-09-20T18:59:05.082117+0000","last_active":"2024-09-20T19:08:24.341013+0000","last_peered":"2024-09-20T19:08:24.341013+0000","last_clean":"2024-09-20T19:08:24.341013+0000","last_became_active":"2024-09-20T18:59:05.081814+0000","last_became_peered":"2024-09-20T18:59:05.081814+0000","last_unstale":"2024-09-20T19:08:24.341013+0000","last_undegraded":"2024-09-20T19:08:24.341013+0000","last_fullsized":"2024-09-20T19:08:24.341013+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:29.530001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:29.530001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"74'79","reported_seq":532,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.338861+0000","last_change":"2024-09-20T19:07:42.989292+0000","last_active":"2024-09-20T19:08:24.338861+0000","last_peered":"2024-09-20T19:08:24.338861+0000","last_clean":"2024-09-20T19:08:24.338861+0000","last_became_active":"2024-09-20T19:07:42.988925+0000","last_became_peered":"2024-09-20T19:07:42.988925+0000","last_unstale":"2024-09-20T19:08:24.338861+0000","last_undegraded":"2024-09-20T19:08:24.338861+0000","last_fullsized":"2024-09-20T19:08:24.338861+0000","mapping_epoch":458,"log_start":"0'0","ondisk_log_start":"0'0","created":455,"last_epoch_clean":459,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:03:14.368279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:03:14.368279+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":21,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":107340,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"472'8475","reported_seq":8588,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:26.751163+0000","last_change":"2024-09-20T19:06:18.053626+0000","last_active":"2024-09-20T19:08:26.751163+0000","last_peered":"2024-09-20T19:08:26.751163+0000","last_clean":"2024-09-20T19:08:26.751163+0000","last_became_active":"2024-09-20T19:06:18.053336+0000","last_became_peered":"2024-09-20T19:06:18.053336+0000","last_unstale":"2024-09-20T19:08:26.751163+0000","last_undegraded":"2024-09-20T19:08:26.751163+0000","last_fullsized":"2024-09-20T19:08:26.751163+0000","mapping_epoch":422,"log_start":"437'3166","ondisk_log_start":"437'3166","created":422,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":5309,"log_dups_size":0,"ondisk_log_size":5309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":293994520,"num_objects":4487,"num_object_clones":0,"num_object_copies":13461,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4487,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8475,"num_write_kb":414721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"274'20","reported_seq":721,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.343739+0000","last_change":"2024-09-20T19:05:48.741237+0000","last_active":"2024-09-20T19:08:24.343739+0000","last_peered":"2024-09-20T19:08:24.343739+0000","last_clean":"2024-09-20T19:08:24.343739+0000","last_became_active":"2024-09-20T19:04:49.419140+0000","last_became_peered":"2024-09-20T19:04:49.419140+0000","last_unstale":"2024-09-20T19:08:24.343739+0000","last_undegraded":"2024-09-20T19:08:24.343739+0000","last_fullsized":"2024-09-20T19:08:24.343739+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'20","last_scrub_stamp":"2024-09-20T19:05:48.741059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:48.741059+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":899,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.344377+0000","last_change":"2024-09-20T19:05:56.820141+0000","last_active":"2024-09-20T19:08:24.344377+0000","last_peered":"2024-09-20T19:08:24.344377+0000","last_clean":"2024-09-20T19:08:24.344377+0000","last_became_active":"2024-09-20T18:49:45.942496+0000","last_became_peered":"2024-09-20T18:49:45.942496+0000","last_unstale":"2024-09-20T19:08:24.344377+0000","last_undegraded":"2024-09-20T19:08:24.344377+0000","last_fullsized":"2024-09-20T19:08:24.344377+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:56.820079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:56.820079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"262'10","reported_seq":1123,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.336482+0000","last_change":"2024-09-20T19:02:04.994109+0000","last_active":"2024-09-20T19:08:24.336482+0000","last_peered":"2024-09-20T19:08:24.336482+0000","last_clean":"2024-09-20T19:08:24.336482+0000","last_became_active":"2024-09-20T18:59:05.080183+0000","last_became_peered":"2024-09-20T18:59:05.080183+0000","last_unstale":"2024-09-20T19:08:24.336482+0000","last_undegraded":"2024-09-20T19:08:24.336482+0000","last_fullsized":"2024-09-20T19:08:24.336482+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'10","last_scrub_stamp":"2024-09-20T19:02:04.994047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:04.994047+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":857,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.336718+0000","last_change":"2024-09-20T19:06:15.671414+0000","last_active":"2024-09-20T19:08:24.336718+0000","last_peered":"2024-09-20T19:08:24.336718+0000","last_clean":"2024-09-20T19:08:24.336718+0000","last_became_active":"2024-09-20T18:49:51.964780+0000","last_became_peered":"2024-09-20T18:49:51.964780+0000","last_unstale":"2024-09-20T19:08:24.336718+0000","last_undegraded":"2024-09-20T19:08:24.336718+0000","last_fullsized":"2024-09-20T19:08:24.336718+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:15.671352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:06:15.671352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"74'79","reported_seq":532,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.339040+0000","last_change":"2024-09-20T19:07:42.994751+0000","last_active":"2024-09-20T19:08:24.339040+0000","last_peered":"2024-09-20T19:08:24.339040+0000","last_clean":"2024-09-20T19:08:24.339040+0000","last_became_active":"2024-09-20T19:07:42.994388+0000","last_became_peered":"2024-09-20T19:07:42.994388+0000","last_unstale":"2024-09-20T19:08:24.339040+0000","last_undegraded":"2024-09-20T19:08:24.339040+0000","last_fullsized":"2024-09-20T19:08:24.339040+0000","mapping_epoch":458,"log_start":"0'0","ondisk_log_start":"0'0","created":455,"last_epoch_clean":459,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:03:14.368279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:03:14.368279+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":21,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":107340,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.c","version":"472'8515","reported_seq":8629,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:27.667315+0000","last_change":"2024-09-20T19:06:18.050185+0000","last_active":"2024-09-20T19:08:27.667315+0000","last_peered":"2024-09-20T19:08:27.667315+0000","last_clean":"2024-09-20T19:08:27.667315+0000","last_became_active":"2024-09-20T19:06:18.049949+0000","last_became_peered":"2024-09-20T19:06:18.049949+0000","last_unstale":"2024-09-20T19:08:27.667315+0000","last_undegraded":"2024-09-20T19:08:27.667315+0000","last_fullsized":"2024-09-20T19:08:27.667315+0000","mapping_epoch":422,"log_start":"447'4100","ondisk_log_start":"447'4100","created":422,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":4415,"log_dups_size":0,"ondisk_log_size":4415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":284753920,"num_objects":4345,"num_object_clones":0,"num_object_copies":13035,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8515,"num_write_kb":411520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,4],"acting":[1,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"274'29","reported_seq":923,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.336626+0000","last_change":"2024-09-20T19:05:43.631990+0000","last_active":"2024-09-20T19:08:24.336626+0000","last_peered":"2024-09-20T19:08:24.336626+0000","last_clean":"2024-09-20T19:08:24.336626+0000","last_became_active":"2024-09-20T18:49:48.497736+0000","last_became_peered":"2024-09-20T18:49:48.497736+0000","last_unstale":"2024-09-20T19:08:24.336626+0000","last_undegraded":"2024-09-20T19:08:24.336626+0000","last_fullsized":"2024-09-20T19:08:24.336626+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'29","last_scrub_stamp":"2024-09-20T19:05:43.631845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:43.631845+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":875,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.333544+0000","last_change":"2024-09-20T19:05:47.837587+0000","last_active":"2024-09-20T19:08:24.333544+0000","last_peered":"2024-09-20T19:08:24.333544+0000","last_clean":"2024-09-20T19:08:24.333544+0000","last_became_active":"2024-09-20T18:49:45.942253+0000","last_became_peered":"2024-09-20T18:49:45.942253+0000","last_unstale":"2024-09-20T19:08:24.333544+0000","last_undegraded":"2024-09-20T19:08:24.333544+0000","last_fullsized":"2024-09-20T19:08:24.333544+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:47.837520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:47.837520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":462,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.338753+0000","last_change":"2024-09-20T19:05:48.203244+0000","last_active":"2024-09-20T19:08:24.338753+0000","last_peered":"2024-09-20T19:08:24.338753+0000","last_clean":"2024-09-20T19:08:24.338753+0000","last_became_active":"2024-09-20T18:59:05.085636+0000","last_became_peered":"2024-09-20T18:59:05.085636+0000","last_unstale":"2024-09-20T19:08:24.338753+0000","last_undegraded":"2024-09-20T19:08:24.338753+0000","last_fullsized":"2024-09-20T19:08:24.338753+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:48.203182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:05:48.203182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":893,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.333260+0000","last_change":"2024-09-20T19:06:43.932713+0000","last_active":"2024-09-20T19:08:24.333260+0000","last_peered":"2024-09-20T19:08:24.333260+0000","last_clean":"2024-09-20T19:08:24.333260+0000","last_became_active":"2024-09-20T18:59:05.085131+0000","last_became_peered":"2024-09-20T18:59:05.085131+0000","last_unstale":"2024-09-20T19:08:24.333260+0000","last_undegraded":"2024-09-20T19:08:24.333260+0000","last_fullsized":"2024-09-20T19:08:24.333260+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:43.932649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:06:43.932649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"74'79","reported_seq":38,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.343395+0000","last_change":"2024-09-20T19:07:42.995596+0000","last_active":"2024-09-20T19:08:24.343395+0000","last_peered":"2024-09-20T19:08:24.343395+0000","last_clean":"2024-09-20T19:08:24.343395+0000","last_became_active":"2024-09-20T19:07:42.995299+0000","last_became_peered":"2024-09-20T19:07:42.995299+0000","last_unstale":"2024-09-20T19:08:24.343395+0000","last_undegraded":"2024-09-20T19:08:24.343395+0000","last_fullsized":"2024-09-20T19:08:24.343395+0000","mapping_epoch":458,"log_start":"0'0","ondisk_log_start":"0'0","created":455,"last_epoch_clean":459,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:03:14.368279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:03:14.368279+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":11,"num_read_kb":20,"num_write":11,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":53670,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.b","version":"472'8176","reported_seq":8290,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:26.061779+0000","last_change":"2024-09-20T19:06:18.057411+0000","last_active":"2024-09-20T19:08:26.061779+0000","last_peered":"2024-09-20T19:08:26.061779+0000","last_clean":"2024-09-20T19:08:26.061779+0000","last_became_active":"2024-09-20T19:06:18.057129+0000","last_became_peered":"2024-09-20T19:06:18.057129+0000","last_unstale":"2024-09-20T19:08:26.061779+0000","last_undegraded":"2024-09-20T19:08:26.061779+0000","last_fullsized":"2024-09-20T19:08:26.061779+0000","mapping_epoch":422,"log_start":"440'3673","ondisk_log_start":"440'3673","created":422,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":4503,"log_dups_size":0,"ondisk_log_size":4503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":283770880,"num_objects":4330,"num_object_clones":0,"num_object_copies":12990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4330,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8176,"num_write_kb":400192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"353'41","reported_seq":1207,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.333473+0000","last_change":"2024-09-20T19:05:48.855983+0000","last_active":"2024-09-20T19:08:24.333473+0000","last_peered":"2024-09-20T19:08:24.333473+0000","last_clean":"2024-09-20T19:08:24.333473+0000","last_became_active":"2024-09-20T19:03:16.976916+0000","last_became_peered":"2024-09-20T19:03:16.976916+0000","last_unstale":"2024-09-20T19:08:24.333473+0000","last_undegraded":"2024-09-20T19:08:24.333473+0000","last_fullsized":"2024-09-20T19:08:24.333473+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"353'41","last_scrub_stamp":"2024-09-20T19:05:48.855876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:48.855876+0000","objects_scrubbed":10,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":214,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.329643+0000","last_change":"2024-09-20T19:06:15.168347+0000","last_active":"2024-09-20T19:08:24.329643+0000","last_peered":"2024-09-20T19:08:24.329643+0000","last_clean":"2024-09-20T19:08:24.329643+0000","last_became_active":"2024-09-20T19:04:20.899352+0000","last_became_peered":"2024-09-20T19:04:20.899352+0000","last_unstale":"2024-09-20T19:08:24.329643+0000","last_undegraded":"2024-09-20T19:08:24.329643+0000","last_fullsized":"2024-09-20T19:08:24.329643+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:15.168293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:06:15.168293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":931,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.343834+0000","last_change":"2024-09-20T19:03:11.910684+0000","last_active":"2024-09-20T19:08:24.343834+0000","last_peered":"2024-09-20T19:08:24.343834+0000","last_clean":"2024-09-20T19:08:24.343834+0000","last_became_active":"2024-09-20T18:59:05.086786+0000","last_became_peered":"2024-09-20T18:59:05.086786+0000","last_unstale":"2024-09-20T19:08:24.343834+0000","last_undegraded":"2024-09-20T19:08:24.343834+0000","last_fullsized":"2024-09-20T19:08:24.343834+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-20T19:03:11.909394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:03:11.909394+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0.0012781159999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":909,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.337157+0000","last_change":"2024-09-20T19:06:55.997516+0000","last_active":"2024-09-20T19:08:24.337157+0000","last_peered":"2024-09-20T19:08:24.337157+0000","last_clean":"2024-09-20T19:08:24.337157+0000","last_became_active":"2024-09-20T19:04:20.907038+0000","last_became_peered":"2024-09-20T19:04:20.907038+0000","last_unstale":"2024-09-20T19:08:24.337157+0000","last_undegraded":"2024-09-20T19:08:24.337157+0000","last_fullsized":"2024-09-20T19:08:24.337157+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:55.997469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:06:55.997469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":872,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.333289+0000","last_change":"2024-09-20T19:04:20.896616+0000","last_active":"2024-09-20T19:08:24.333289+0000","last_peered":"2024-09-20T19:08:24.333289+0000","last_clean":"2024-09-20T19:08:24.333289+0000","last_became_active":"2024-09-20T19:04:20.896281+0000","last_became_peered":"2024-09-20T19:04:20.896281+0000","last_unstale":"2024-09-20T19:08:24.333289+0000","last_undegraded":"2024-09-20T19:08:24.333289+0000","last_fullsized":"2024-09-20T19:08:24.333289+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:04.526683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:04.526683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"74'79","reported_seq":960,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.340530+0000","last_change":"2024-09-20T19:07:42.987494+0000","last_active":"2024-09-20T19:08:24.340530+0000","last_peered":"2024-09-20T19:08:24.340530+0000","last_clean":"2024-09-20T19:08:24.340530+0000","last_became_active":"2024-09-20T19:07:42.987148+0000","last_became_peered":"2024-09-20T19:07:42.987148+0000","last_unstale":"2024-09-20T19:08:24.340530+0000","last_undegraded":"2024-09-20T19:08:24.340530+0000","last_fullsized":"2024-09-20T19:08:24.340530+0000","mapping_epoch":458,"log_start":"0'0","ondisk_log_start":"0'0","created":455,"last_epoch_clean":459,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:03:14.368279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:03:14.368279+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":21,"num_write":11,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":161010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.8","version":"472'6056","reported_seq":6204,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:26.081243+0000","last_change":"2024-09-20T19:06:58.108560+0000","last_active":"2024-09-20T19:08:26.081243+0000","last_peered":"2024-09-20T19:08:26.081243+0000","last_clean":"2024-09-20T19:08:26.081243+0000","last_became_active":"2024-09-20T19:06:56.463634+0000","last_became_peered":"2024-09-20T19:06:56.463634+0000","last_unstale":"2024-09-20T19:08:26.081243+0000","last_undegraded":"2024-09-20T19:08:26.081243+0000","last_fullsized":"2024-09-20T19:08:26.081243+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"443'3725","last_scrub_stamp":"2024-09-20T19:06:58.108433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:58.108433+0000","objects_scrubbed":1931,"log_size":6056,"log_dups_size":0,"ondisk_log_size":6056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":242,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":148635648,"num_objects":2268,"num_object_clones":0,"num_object_copies":6804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2331,"num_write_kb":85376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"273'17","reported_seq":920,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.334833+0000","last_change":"2024-09-20T19:05:43.974098+0000","last_active":"2024-09-20T19:08:24.334833+0000","last_peered":"2024-09-20T19:08:24.334833+0000","last_clean":"2024-09-20T19:08:24.334833+0000","last_became_active":"2024-09-20T18:49:47.950904+0000","last_became_peered":"2024-09-20T18:49:47.950904+0000","last_unstale":"2024-09-20T19:08:24.334833+0000","last_undegraded":"2024-09-20T19:08:24.334833+0000","last_fullsized":"2024-09-20T19:08:24.334833+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'17","last_scrub_stamp":"2024-09-20T19:05:43.974013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:43.974013+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":36,"num_read_kb":25,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":898,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.333124+0000","last_change":"2024-09-20T19:02:29.840761+0000","last_active":"2024-09-20T19:08:24.333124+0000","last_peered":"2024-09-20T19:08:24.333124+0000","last_clean":"2024-09-20T19:08:24.333124+0000","last_became_active":"2024-09-20T18:59:05.079789+0000","last_became_peered":"2024-09-20T18:59:05.079789+0000","last_unstale":"2024-09-20T19:08:24.333124+0000","last_undegraded":"2024-09-20T19:08:24.333124+0000","last_fullsized":"2024-09-20T19:08:24.333124+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:29.840601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:29.840601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":888,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.340129+0000","last_change":"2024-09-20T19:06:13.599431+0000","last_active":"2024-09-20T19:08:24.340129+0000","last_peered":"2024-09-20T19:08:24.340129+0000","last_clean":"2024-09-20T19:08:24.340129+0000","last_became_active":"2024-09-20T18:49:44.937135+0000","last_became_peered":"2024-09-20T18:49:44.937135+0000","last_unstale":"2024-09-20T19:08:24.340129+0000","last_undegraded":"2024-09-20T19:08:24.340129+0000","last_fullsized":"2024-09-20T19:08:24.340129+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:06:13.599376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:06:13.599376+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00095558800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":854,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.340438+0000","last_change":"2024-09-20T19:01:17.747452+0000","last_active":"2024-09-20T19:08:24.340438+0000","last_peered":"2024-09-20T19:08:24.340438+0000","last_clean":"2024-09-20T19:08:24.340438+0000","last_became_active":"2024-09-20T18:49:49.954424+0000","last_became_peered":"2024-09-20T18:49:49.954424+0000","last_unstale":"2024-09-20T19:08:24.340438+0000","last_undegraded":"2024-09-20T19:08:24.340438+0000","last_fullsized":"2024-09-20T19:08:24.340438+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:17.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:17.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":865,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.340855+0000","last_change":"2024-09-20T19:01:15.203591+0000","last_active":"2024-09-20T19:08:24.340855+0000","last_peered":"2024-09-20T19:08:24.340855+0000","last_clean":"2024-09-20T19:08:24.340855+0000","last_became_active":"2024-09-20T18:59:05.081459+0000","last_became_peered":"2024-09-20T18:59:05.081459+0000","last_unstale":"2024-09-20T19:08:24.340855+0000","last_undegraded":"2024-09-20T19:08:24.340855+0000","last_fullsized":"2024-09-20T19:08:24.340855+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:15.203529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:15.203529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"74'79","reported_seq":38,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.337985+0000","last_change":"2024-09-20T19:07:42.992213+0000","last_active":"2024-09-20T19:08:24.337985+0000","last_peered":"2024-09-20T19:08:24.337985+0000","last_clean":"2024-09-20T19:08:24.337985+0000","last_became_active":"2024-09-20T19:07:42.991822+0000","last_became_peered":"2024-09-20T19:07:42.991822+0000","last_unstale":"2024-09-20T19:08:24.337985+0000","last_undegraded":"2024-09-20T19:08:24.337985+0000","last_fullsized":"2024-09-20T19:08:24.337985+0000","mapping_epoch":458,"log_start":"0'0","ondisk_log_start":"0'0","created":455,"last_epoch_clean":459,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:03:14.368279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:03:14.368279+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":20,"num_write":11,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":53670,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.9","version":"472'6197","reported_seq":6346,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:26.022031+0000","last_change":"2024-09-20T19:06:57.306900+0000","last_active":"2024-09-20T19:08:26.022031+0000","last_peered":"2024-09-20T19:08:26.022031+0000","last_clean":"2024-09-20T19:08:26.022031+0000","last_became_active":"2024-09-20T19:06:56.466304+0000","last_became_peered":"2024-09-20T19:06:56.466304+0000","last_unstale":"2024-09-20T19:08:26.022031+0000","last_undegraded":"2024-09-20T19:08:26.022031+0000","last_fullsized":"2024-09-20T19:08:26.022031+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"442'3861","last_scrub_stamp":"2024-09-20T19:06:57.306685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:57.306685+0000","objects_scrubbed":1958,"log_size":6197,"log_dups_size":0,"ondisk_log_size":6197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":402,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":152305664,"num_objects":2324,"num_object_clones":0,"num_object_copies":6972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2346,"num_write_kb":86784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"273'24","reported_seq":916,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.338705+0000","last_change":"2024-09-20T18:59:06.022021+0000","last_active":"2024-09-20T19:08:24.338705+0000","last_peered":"2024-09-20T19:08:24.338705+0000","last_clean":"2024-09-20T19:08:24.338705+0000","last_became_active":"2024-09-20T18:49:47.949230+0000","last_became_peered":"2024-09-20T18:49:47.949230+0000","last_unstale":"2024-09-20T19:08:24.338705+0000","last_undegraded":"2024-09-20T19:08:24.338705+0000","last_fullsized":"2024-09-20T19:08:24.338705+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-20T18:59:06.021962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:59:06.021962+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":901,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.337742+0000","last_change":"2024-09-20T19:04:20.907250+0000","last_active":"2024-09-20T19:08:24.337742+0000","last_peered":"2024-09-20T19:08:24.337742+0000","last_clean":"2024-09-20T19:08:24.337742+0000","last_became_active":"2024-09-20T19:04:20.906969+0000","last_became_peered":"2024-09-20T19:04:20.906969+0000","last_unstale":"2024-09-20T19:08:24.337742+0000","last_undegraded":"2024-09-20T19:08:24.337742+0000","last_fullsized":"2024-09-20T19:08:24.337742+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:24.984022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:24.984022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":2009,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.863454+0000","last_change":"2024-09-20T19:06:43.885193+0000","last_active":"2024-09-20T19:08:24.863454+0000","last_peered":"2024-09-20T19:08:24.863454+0000","last_clean":"2024-09-20T19:08:24.863454+0000","last_became_active":"2024-09-20T18:49:44.936819+0000","last_became_peered":"2024-09-20T18:49:44.936819+0000","last_unstale":"2024-09-20T19:08:24.863454+0000","last_undegraded":"2024-09-20T19:08:24.863454+0000","last_fullsized":"2024-09-20T19:08:24.863454+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-20T19:06:43.883404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:06:43.883404+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.0010996199999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1113,"num_read_kb":1108,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":883,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.344817+0000","last_change":"2024-09-20T19:05:51.743485+0000","last_active":"2024-09-20T19:08:24.344817+0000","last_peered":"2024-09-20T19:08:24.344817+0000","last_clean":"2024-09-20T19:08:24.344817+0000","last_became_active":"2024-09-20T18:49:49.961066+0000","last_became_peered":"2024-09-20T18:49:49.961066+0000","last_unstale":"2024-09-20T19:08:24.344817+0000","last_undegraded":"2024-09-20T19:08:24.344817+0000","last_fullsized":"2024-09-20T19:08:24.344817+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:51.743363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:05:51.743363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":871,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.333224+0000","last_change":"2024-09-20T19:02:26.861253+0000","last_active":"2024-09-20T19:08:24.333224+0000","last_peered":"2024-09-20T19:08:24.333224+0000","last_clean":"2024-09-20T19:08:24.333224+0000","last_became_active":"2024-09-20T18:49:51.970343+0000","last_became_peered":"2024-09-20T18:49:51.970343+0000","last_unstale":"2024-09-20T19:08:24.333224+0000","last_undegraded":"2024-09-20T19:08:24.333224+0000","last_fullsized":"2024-09-20T19:08:24.333224+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:26.861185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:26.861185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.a","version":"472'8354","reported_seq":8468,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:26.086637+0000","last_change":"2024-09-20T19:06:18.055447+0000","last_active":"2024-09-20T19:08:26.086637+0000","last_peered":"2024-09-20T19:08:26.086637+0000","last_clean":"2024-09-20T19:08:26.086637+0000","last_became_active":"2024-09-20T19:06:18.055189+0000","last_became_peered":"2024-09-20T19:06:18.055189+0000","last_unstale":"2024-09-20T19:08:26.086637+0000","last_undegraded":"2024-09-20T19:08:26.086637+0000","last_fullsized":"2024-09-20T19:08:26.086637+0000","mapping_epoch":422,"log_start":"445'3934","ondisk_log_start":"445'3934","created":422,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":4420,"log_dups_size":0,"ondisk_log_size":4420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":293601280,"num_objects":4480,"num_object_clones":0,"num_object_copies":13440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4480,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8354,"num_write_kb":410688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"375'34","reported_seq":666,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.339204+0000","last_change":"2024-09-20T19:05:54.115428+0000","last_active":"2024-09-20T19:08:24.339204+0000","last_peered":"2024-09-20T19:08:24.339204+0000","last_clean":"2024-09-20T19:08:24.339204+0000","last_became_active":"2024-09-20T19:04:20.928727+0000","last_became_peered":"2024-09-20T19:04:20.928727+0000","last_unstale":"2024-09-20T19:08:24.339204+0000","last_undegraded":"2024-09-20T19:08:24.339204+0000","last_fullsized":"2024-09-20T19:08:24.339204+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"375'34","last_scrub_stamp":"2024-09-20T19:05:54.115376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:54.115376+0000","objects_scrubbed":8,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":462,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":905,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.333621+0000","last_change":"2024-09-20T19:06:47.914880+0000","last_active":"2024-09-20T19:08:24.333621+0000","last_peered":"2024-09-20T19:08:24.333621+0000","last_clean":"2024-09-20T19:08:24.333621+0000","last_became_active":"2024-09-20T18:59:05.079111+0000","last_became_peered":"2024-09-20T18:59:05.079111+0000","last_unstale":"2024-09-20T19:08:24.333621+0000","last_undegraded":"2024-09-20T19:08:24.333621+0000","last_fullsized":"2024-09-20T19:08:24.333621+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:47.914630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:06:47.914630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":955,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.340730+0000","last_change":"2024-09-20T19:07:40.718445+0000","last_active":"2024-09-20T19:08:24.340730+0000","last_peered":"2024-09-20T19:08:24.340730+0000","last_clean":"2024-09-20T19:08:24.340730+0000","last_became_active":"2024-09-20T19:07:40.718130+0000","last_became_peered":"2024-09-20T19:07:40.718130+0000","last_unstale":"2024-09-20T19:08:24.340730+0000","last_undegraded":"2024-09-20T19:08:24.340730+0000","last_fullsized":"2024-09-20T19:08:24.340730+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":456,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:03:14.368279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:03:14.368279+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53669,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":21,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":161009,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":868,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.341132+0000","last_change":"2024-09-20T19:06:21.584094+0000","last_active":"2024-09-20T19:08:24.341132+0000","last_peered":"2024-09-20T19:08:24.341132+0000","last_clean":"2024-09-20T19:08:24.341132+0000","last_became_active":"2024-09-20T18:49:49.956128+0000","last_became_peered":"2024-09-20T18:49:49.956128+0000","last_unstale":"2024-09-20T19:08:24.341132+0000","last_undegraded":"2024-09-20T19:08:24.341132+0000","last_fullsized":"2024-09-20T19:08:24.341132+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:21.584027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:06:21.584027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":904,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.344753+0000","last_change":"2024-09-20T19:06:43.994634+0000","last_active":"2024-09-20T19:08:24.344753+0000","last_peered":"2024-09-20T19:08:24.344753+0000","last_clean":"2024-09-20T19:08:24.344753+0000","last_became_active":"2024-09-20T19:04:20.899321+0000","last_became_peered":"2024-09-20T19:04:20.899321+0000","last_unstale":"2024-09-20T19:08:24.344753+0000","last_undegraded":"2024-09-20T19:08:24.344753+0000","last_fullsized":"2024-09-20T19:08:24.344753+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:43.994470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:06:43.994470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1","version":"472'6169","reported_seq":6321,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:26.971009+0000","last_change":"2024-09-20T19:08:24.157661+0000","last_active":"2024-09-20T19:08:26.971009+0000","last_peered":"2024-09-20T19:08:26.971009+0000","last_clean":"2024-09-20T19:08:26.971009+0000","last_became_active":"2024-09-20T19:06:56.471168+0000","last_became_peered":"2024-09-20T19:06:56.471168+0000","last_unstale":"2024-09-20T19:08:26.971009+0000","last_undegraded":"2024-09-20T19:08:26.971009+0000","last_fullsized":"2024-09-20T19:08:26.971009+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'6094","last_scrub_stamp":"2024-09-20T19:08:24.157520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:08:24.157520+0000","objects_scrubbed":2284,"log_size":6169,"log_dups_size":0,"ondisk_log_size":6169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:09:37.562109+0000","scrub_duration":339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":144572416,"num_objects":2206,"num_object_clones":0,"num_object_copies":6618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2206,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":78,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"274'19","reported_seq":226,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.329896+0000","last_change":"2024-09-20T19:05:50.066533+0000","last_active":"2024-09-20T19:08:24.329896+0000","last_peered":"2024-09-20T19:08:24.329896+0000","last_clean":"2024-09-20T19:08:24.329896+0000","last_became_active":"2024-09-20T19:04:20.897175+0000","last_became_peered":"2024-09-20T19:04:20.897175+0000","last_unstale":"2024-09-20T19:08:24.329896+0000","last_undegraded":"2024-09-20T19:08:24.329896+0000","last_fullsized":"2024-09-20T19:08:24.329896+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'19","last_scrub_stamp":"2024-09-20T19:05:50.066359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:50.066359+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":214,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.329283+0000","last_change":"2024-09-20T19:05:56.027187+0000","last_active":"2024-09-20T19:08:24.329283+0000","last_peered":"2024-09-20T19:08:24.329283+0000","last_clean":"2024-09-20T19:08:24.329283+0000","last_became_active":"2024-09-20T19:04:20.887102+0000","last_became_peered":"2024-09-20T19:04:20.887102+0000","last_unstale":"2024-09-20T19:08:24.329283+0000","last_undegraded":"2024-09-20T19:08:24.329283+0000","last_fullsized":"2024-09-20T19:08:24.329283+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:56.027123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:56.027123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":914,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.344672+0000","last_change":"2024-09-20T19:04:20.899830+0000","last_active":"2024-09-20T19:08:24.344672+0000","last_peered":"2024-09-20T19:08:24.344672+0000","last_clean":"2024-09-20T19:08:24.344672+0000","last_became_active":"2024-09-20T19:04:20.899331+0000","last_became_peered":"2024-09-20T19:04:20.899331+0000","last_unstale":"2024-09-20T19:08:24.344672+0000","last_undegraded":"2024-09-20T19:08:24.344672+0000","last_fullsized":"2024-09-20T19:08:24.344672+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:38.463404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:38.463404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":877,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.333291+0000","last_change":"2024-09-20T19:04:35.529672+0000","last_active":"2024-09-20T19:08:24.333291+0000","last_peered":"2024-09-20T19:08:24.333291+0000","last_clean":"2024-09-20T19:08:24.333291+0000","last_became_active":"2024-09-20T18:59:05.079897+0000","last_became_peered":"2024-09-20T18:59:05.079897+0000","last_unstale":"2024-09-20T19:08:24.333291+0000","last_undegraded":"2024-09-20T19:08:24.333291+0000","last_fullsized":"2024-09-20T19:08:24.333291+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:35.529584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:04:35.529584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.8","version":"74'79","reported_seq":45,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.343335+0000","last_change":"2024-09-20T19:08:22.873127+0000","last_active":"2024-09-20T19:08:24.343335+0000","last_peered":"2024-09-20T19:08:24.343335+0000","last_clean":"2024-09-20T19:08:24.343335+0000","last_became_active":"2024-09-20T19:07:42.988175+0000","last_became_peered":"2024-09-20T19:07:42.988175+0000","last_unstale":"2024-09-20T19:08:24.343335+0000","last_undegraded":"2024-09-20T19:08:24.343335+0000","last_fullsized":"2024-09-20T19:08:24.343335+0000","mapping_epoch":458,"log_start":"0'0","ondisk_log_start":"0'0","created":455,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:03:14.368279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:03:14.368279+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":20,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":316358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2","version":"472'5989","reported_seq":6136,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:26.779024+0000","last_change":"2024-09-20T19:06:58.880519+0000","last_active":"2024-09-20T19:08:26.779024+0000","last_peered":"2024-09-20T19:08:26.779024+0000","last_clean":"2024-09-20T19:08:26.779024+0000","last_became_active":"2024-09-20T19:06:56.454466+0000","last_became_peered":"2024-09-20T19:06:56.454466+0000","last_unstale":"2024-09-20T19:08:26.779024+0000","last_undegraded":"2024-09-20T19:08:26.779024+0000","last_fullsized":"2024-09-20T19:08:26.779024+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"443'3744","last_scrub_stamp":"2024-09-20T19:06:58.880419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:58.880419+0000","objects_scrubbed":1883,"log_size":5989,"log_dups_size":0,"ondisk_log_size":5989,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":189,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":141295616,"num_objects":2156,"num_object_clones":0,"num_object_copies":6468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2245,"num_write_kb":80576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"274'31","reported_seq":232,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.329850+0000","last_change":"2024-09-20T19:06:57.528138+0000","last_active":"2024-09-20T19:08:24.329850+0000","last_peered":"2024-09-20T19:08:24.329850+0000","last_clean":"2024-09-20T19:08:24.329850+0000","last_became_active":"2024-09-20T19:04:20.883306+0000","last_became_peered":"2024-09-20T19:04:20.883306+0000","last_unstale":"2024-09-20T19:08:24.329850+0000","last_undegraded":"2024-09-20T19:08:24.329850+0000","last_fullsized":"2024-09-20T19:08:24.329850+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'31","last_scrub_stamp":"2024-09-20T19:06:57.528084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:06:57.528084+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":49,"num_read_kb":39,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":471,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.338982+0000","last_change":"2024-09-20T19:04:20.899712+0000","last_active":"2024-09-20T19:08:24.338982+0000","last_peered":"2024-09-20T19:08:24.338982+0000","last_clean":"2024-09-20T19:08:24.338982+0000","last_became_active":"2024-09-20T19:04:20.899415+0000","last_became_peered":"2024-09-20T19:04:20.899415+0000","last_unstale":"2024-09-20T19:08:24.338982+0000","last_undegraded":"2024-09-20T19:08:24.338982+0000","last_fullsized":"2024-09-20T19:08:24.338982+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:42.479489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:42.479489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":900,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.333333+0000","last_change":"2024-09-20T19:05:46.866958+0000","last_active":"2024-09-20T19:08:24.333333+0000","last_peered":"2024-09-20T19:08:24.333333+0000","last_clean":"2024-09-20T19:08:24.333333+0000","last_became_active":"2024-09-20T19:04:20.891772+0000","last_became_peered":"2024-09-20T19:04:20.891772+0000","last_unstale":"2024-09-20T19:08:24.333333+0000","last_undegraded":"2024-09-20T19:08:24.333333+0000","last_fullsized":"2024-09-20T19:08:24.333333+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:46.866777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:05:46.866777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":899,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.340510+0000","last_change":"2024-09-20T19:06:25.628102+0000","last_active":"2024-09-20T19:08:24.340510+0000","last_peered":"2024-09-20T19:08:24.340510+0000","last_clean":"2024-09-20T19:08:24.340510+0000","last_became_active":"2024-09-20T19:04:20.897569+0000","last_became_peered":"2024-09-20T19:04:20.897569+0000","last_unstale":"2024-09-20T19:08:24.340510+0000","last_undegraded":"2024-09-20T19:08:24.340510+0000","last_fullsized":"2024-09-20T19:08:24.340510+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:25.628041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:06:25.628041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"74'79","reported_seq":960,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.340754+0000","last_change":"2024-09-20T19:07:42.989807+0000","last_active":"2024-09-20T19:08:24.340754+0000","last_peered":"2024-09-20T19:08:24.340754+0000","last_clean":"2024-09-20T19:08:24.340754+0000","last_became_active":"2024-09-20T19:07:42.989434+0000","last_became_peered":"2024-09-20T19:07:42.989434+0000","last_unstale":"2024-09-20T19:08:24.340754+0000","last_undegraded":"2024-09-20T19:08:24.340754+0000","last_fullsized":"2024-09-20T19:08:24.340754+0000","mapping_epoch":458,"log_start":"0'0","ondisk_log_start":"0'0","created":455,"last_epoch_clean":459,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:03:14.368279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:03:14.368279+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":21,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":161009,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3","version":"472'6154","reported_seq":6308,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:26.790359+0000","last_change":"2024-09-20T19:08:25.439042+0000","last_active":"2024-09-20T19:08:26.790359+0000","last_peered":"2024-09-20T19:08:26.790359+0000","last_clean":"2024-09-20T19:08:26.790359+0000","last_became_active":"2024-09-20T19:06:56.480422+0000","last_became_peered":"2024-09-20T19:06:56.480422+0000","last_unstale":"2024-09-20T19:08:26.790359+0000","last_undegraded":"2024-09-20T19:08:26.790359+0000","last_fullsized":"2024-09-20T19:08:26.790359+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"472'6105","last_scrub_stamp":"2024-09-20T19:08:25.438862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:08:25.438862+0000","objects_scrubbed":2261,"log_size":6154,"log_dups_size":0,"ondisk_log_size":6154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:09:37.377223+0000","scrub_duration":582,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":144048128,"num_objects":2198,"num_object_clones":0,"num_object_copies":6594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2198,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":63,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1],"acting":[6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"273'20","reported_seq":218,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.329303+0000","last_change":"2024-09-20T19:04:20.945143+0000","last_active":"2024-09-20T19:08:24.329303+0000","last_peered":"2024-09-20T19:08:24.329303+0000","last_clean":"2024-09-20T19:08:24.329303+0000","last_became_active":"2024-09-20T19:04:20.896483+0000","last_became_peered":"2024-09-20T19:04:20.896483+0000","last_unstale":"2024-09-20T19:08:24.329303+0000","last_undegraded":"2024-09-20T19:08:24.329303+0000","last_fullsized":"2024-09-20T19:08:24.329303+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'20","last_scrub_stamp":"2024-09-20T19:03:04.282800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:04.282800+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":908,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.333868+0000","last_change":"2024-09-20T19:05:48.678438+0000","last_active":"2024-09-20T19:08:24.333868+0000","last_peered":"2024-09-20T19:08:24.333868+0000","last_clean":"2024-09-20T19:08:24.333868+0000","last_became_active":"2024-09-20T19:04:20.892951+0000","last_became_peered":"2024-09-20T19:04:20.892951+0000","last_unstale":"2024-09-20T19:08:24.333868+0000","last_undegraded":"2024-09-20T19:08:24.333868+0000","last_fullsized":"2024-09-20T19:08:24.333868+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:48.678361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:48.678361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":1119,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.340263+0000","last_change":"2024-09-20T19:05:08.406307+0000","last_active":"2024-09-20T19:08:24.340263+0000","last_peered":"2024-09-20T19:08:24.340263+0000","last_clean":"2024-09-20T19:08:24.340263+0000","last_became_active":"2024-09-20T18:49:49.957005+0000","last_became_peered":"2024-09-20T18:49:49.957005+0000","last_unstale":"2024-09-20T19:08:24.340263+0000","last_undegraded":"2024-09-20T19:08:24.340263+0000","last_fullsized":"2024-09-20T19:08:24.340263+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T19:05:08.406231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:05:08.406231+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":214,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.329923+0000","last_change":"2024-09-20T19:06:56.546180+0000","last_active":"2024-09-20T19:08:24.329923+0000","last_peered":"2024-09-20T19:08:24.329923+0000","last_clean":"2024-09-20T19:08:24.329923+0000","last_became_active":"2024-09-20T19:04:20.884994+0000","last_became_peered":"2024-09-20T19:04:20.884994+0000","last_unstale":"2024-09-20T19:08:24.329923+0000","last_undegraded":"2024-09-20T19:08:24.329923+0000","last_fullsized":"2024-09-20T19:08:24.329923+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:56.546114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:06:56.546114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.4","version":"472'6110","reported_seq":6230,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:26.430705+0000","last_change":"2024-09-20T19:06:56.464683+0000","last_active":"2024-09-20T19:08:26.430705+0000","last_peered":"2024-09-20T19:08:26.430705+0000","last_clean":"2024-09-20T19:08:26.430705+0000","last_became_active":"2024-09-20T19:06:56.462890+0000","last_became_peered":"2024-09-20T19:06:56.462890+0000","last_unstale":"2024-09-20T19:08:26.430705+0000","last_undegraded":"2024-09-20T19:08:26.430705+0000","last_fullsized":"2024-09-20T19:08:26.430705+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":6110,"log_dups_size":0,"ondisk_log_size":6110,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":146931712,"num_objects":2242,"num_object_clones":0,"num_object_copies":6726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2242,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4218,"num_write_kb":206720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"274'15","reported_seq":950,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.336537+0000","last_change":"2024-09-20T19:05:57.548758+0000","last_active":"2024-09-20T19:08:24.336537+0000","last_peered":"2024-09-20T19:08:24.336537+0000","last_clean":"2024-09-20T19:08:24.336537+0000","last_became_active":"2024-09-20T19:04:20.906834+0000","last_became_peered":"2024-09-20T19:04:20.906834+0000","last_unstale":"2024-09-20T19:08:24.336537+0000","last_undegraded":"2024-09-20T19:08:24.336537+0000","last_fullsized":"2024-09-20T19:08:24.336537+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'15","last_scrub_stamp":"2024-09-20T19:05:57.548683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:57.548683+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":435,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.339363+0000","last_change":"2024-09-20T19:05:52.464362+0000","last_active":"2024-09-20T19:08:24.339363+0000","last_peered":"2024-09-20T19:08:24.339363+0000","last_clean":"2024-09-20T19:08:24.339363+0000","last_became_active":"2024-09-20T19:00:01.190875+0000","last_became_peered":"2024-09-20T19:00:01.190875+0000","last_unstale":"2024-09-20T19:08:24.339363+0000","last_undegraded":"2024-09-20T19:08:24.339363+0000","last_fullsized":"2024-09-20T19:08:24.339363+0000","mapping_epoch":277,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":278,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:52.464261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:52.464261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":903,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.339439+0000","last_change":"2024-09-20T19:04:20.889614+0000","last_active":"2024-09-20T19:08:24.339439+0000","last_peered":"2024-09-20T19:08:24.339439+0000","last_clean":"2024-09-20T19:08:24.339439+0000","last_became_active":"2024-09-20T19:04:20.889273+0000","last_became_peered":"2024-09-20T19:04:20.889273+0000","last_unstale":"2024-09-20T19:08:24.339439+0000","last_undegraded":"2024-09-20T19:08:24.339439+0000","last_fullsized":"2024-09-20T19:08:24.339439+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:03.389611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:03.389611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":864,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.337667+0000","last_change":"2024-09-20T19:02:31.120809+0000","last_active":"2024-09-20T19:08:24.337667+0000","last_peered":"2024-09-20T19:08:24.337667+0000","last_clean":"2024-09-20T19:08:24.337667+0000","last_became_active":"2024-09-20T18:49:51.964098+0000","last_became_peered":"2024-09-20T18:49:51.964098+0000","last_unstale":"2024-09-20T19:08:24.337667+0000","last_undegraded":"2024-09-20T19:08:24.337667+0000","last_fullsized":"2024-09-20T19:08:24.337667+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:31.120623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:31.120623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.5","version":"472'6141","reported_seq":6290,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:26.412658+0000","last_change":"2024-09-20T19:06:57.022784+0000","last_active":"2024-09-20T19:08:26.412658+0000","last_peered":"2024-09-20T19:08:26.412658+0000","last_clean":"2024-09-20T19:08:26.412658+0000","last_became_active":"2024-09-20T19:06:56.460858+0000","last_became_peered":"2024-09-20T19:06:56.460858+0000","last_unstale":"2024-09-20T19:08:26.412658+0000","last_undegraded":"2024-09-20T19:08:26.412658+0000","last_fullsized":"2024-09-20T19:08:26.412658+0000","mapping_epoch":422,"log_start":"426'200","ondisk_log_start":"426'200","created":422,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"442'3824","last_scrub_stamp":"2024-09-20T19:06:57.022671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:57.022671+0000","objects_scrubbed":1982,"log_size":5941,"log_dups_size":0,"ondisk_log_size":5941,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":299,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":151977984,"num_objects":2319,"num_object_clones":0,"num_object_copies":6957,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2323,"num_write_kb":85120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"274'35","reported_seq":506,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.338654+0000","last_change":"2024-09-20T19:06:19.915841+0000","last_active":"2024-09-20T19:08:24.338654+0000","last_peered":"2024-09-20T19:08:24.338654+0000","last_clean":"2024-09-20T19:08:24.338654+0000","last_became_active":"2024-09-20T18:59:05.087391+0000","last_became_peered":"2024-09-20T18:59:05.087391+0000","last_unstale":"2024-09-20T19:08:24.338654+0000","last_undegraded":"2024-09-20T19:08:24.338654+0000","last_fullsized":"2024-09-20T19:08:24.338654+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'35","last_scrub_stamp":"2024-09-20T19:06:19.915767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:06:19.915767+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":898,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.341224+0000","last_change":"2024-09-20T19:05:49.763478+0000","last_active":"2024-09-20T19:08:24.341224+0000","last_peered":"2024-09-20T19:08:24.341224+0000","last_clean":"2024-09-20T19:08:24.341224+0000","last_became_active":"2024-09-20T18:59:05.086670+0000","last_became_peered":"2024-09-20T18:59:05.086670+0000","last_unstale":"2024-09-20T19:08:24.341224+0000","last_undegraded":"2024-09-20T19:08:24.341224+0000","last_fullsized":"2024-09-20T19:08:24.341224+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:49.763407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:49.763407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":886,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.339370+0000","last_change":"2024-09-20T19:05:11.444197+0000","last_active":"2024-09-20T19:08:24.339370+0000","last_peered":"2024-09-20T19:08:24.339370+0000","last_clean":"2024-09-20T19:08:24.339370+0000","last_became_active":"2024-09-20T18:49:49.957558+0000","last_became_peered":"2024-09-20T18:49:49.957558+0000","last_unstale":"2024-09-20T19:08:24.339370+0000","last_undegraded":"2024-09-20T19:08:24.339370+0000","last_fullsized":"2024-09-20T19:08:24.339370+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:11.444143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:05:11.444143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":856,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.341041+0000","last_change":"2024-09-20T18:53:47.370152+0000","last_active":"2024-09-20T19:08:24.341041+0000","last_peered":"2024-09-20T19:08:24.341041+0000","last_clean":"2024-09-20T19:08:24.341041+0000","last_became_active":"2024-09-20T18:49:51.970798+0000","last_became_peered":"2024-09-20T18:49:51.970798+0000","last_unstale":"2024-09-20T19:08:24.341041+0000","last_undegraded":"2024-09-20T19:08:24.341041+0000","last_fullsized":"2024-09-20T19:08:24.341041+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:47.370108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:47.370108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.6","version":"472'6236","reported_seq":6384,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:28.059522+0000","last_change":"2024-09-20T19:06:58.182942+0000","last_active":"2024-09-20T19:08:28.059522+0000","last_peered":"2024-09-20T19:08:28.059522+0000","last_clean":"2024-09-20T19:08:28.059522+0000","last_became_active":"2024-09-20T19:06:56.472642+0000","last_became_peered":"2024-09-20T19:06:56.472642+0000","last_unstale":"2024-09-20T19:08:28.059522+0000","last_undegraded":"2024-09-20T19:08:28.059522+0000","last_fullsized":"2024-09-20T19:08:28.059522+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"443'3869","last_scrub_stamp":"2024-09-20T19:06:58.182865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:58.182865+0000","objects_scrubbed":1953,"log_size":6236,"log_dups_size":0,"ondisk_log_size":6236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":217,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":141426688,"num_objects":2158,"num_object_clones":0,"num_object_copies":6474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2367,"num_write_kb":82304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"273'18","reported_seq":218,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.329420+0000","last_change":"2024-09-20T19:04:20.936465+0000","last_active":"2024-09-20T19:08:24.329420+0000","last_peered":"2024-09-20T19:08:24.329420+0000","last_clean":"2024-09-20T19:08:24.329420+0000","last_became_active":"2024-09-20T19:04:20.888905+0000","last_became_peered":"2024-09-20T19:04:20.888905+0000","last_unstale":"2024-09-20T19:08:24.329420+0000","last_undegraded":"2024-09-20T19:08:24.329420+0000","last_fullsized":"2024-09-20T19:08:24.329420+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'18","last_scrub_stamp":"2024-09-20T19:03:13.201592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:13.201592+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":917,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.341292+0000","last_change":"2024-09-20T19:06:15.692558+0000","last_active":"2024-09-20T19:08:24.341292+0000","last_peered":"2024-09-20T19:08:24.341292+0000","last_clean":"2024-09-20T19:08:24.341292+0000","last_became_active":"2024-09-20T19:04:20.892429+0000","last_became_peered":"2024-09-20T19:04:20.892429+0000","last_unstale":"2024-09-20T19:08:24.341292+0000","last_undegraded":"2024-09-20T19:08:24.341292+0000","last_fullsized":"2024-09-20T19:08:24.341292+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:15.692508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:06:15.692508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":878,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.339582+0000","last_change":"2024-09-20T19:04:48.406445+0000","last_active":"2024-09-20T19:08:24.339582+0000","last_peered":"2024-09-20T19:08:24.339582+0000","last_clean":"2024-09-20T19:08:24.339582+0000","last_became_active":"2024-09-20T18:49:49.954944+0000","last_became_peered":"2024-09-20T18:49:49.954944+0000","last_unstale":"2024-09-20T19:08:24.339582+0000","last_undegraded":"2024-09-20T19:08:24.339582+0000","last_fullsized":"2024-09-20T19:08:24.339582+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:48.406373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:48.406373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":871,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.333520+0000","last_change":"2024-09-20T19:05:43.657860+0000","last_active":"2024-09-20T19:08:24.333520+0000","last_peered":"2024-09-20T19:08:24.333520+0000","last_clean":"2024-09-20T19:08:24.333520+0000","last_became_active":"2024-09-20T18:49:51.972970+0000","last_became_peered":"2024-09-20T18:49:51.972970+0000","last_unstale":"2024-09-20T19:08:24.333520+0000","last_undegraded":"2024-09-20T19:08:24.333520+0000","last_fullsized":"2024-09-20T19:08:24.333520+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:43.657780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:05:43.657780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"472'6100","reported_seq":6254,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:25.783238+0000","last_change":"2024-09-20T19:06:57.243186+0000","last_active":"2024-09-20T19:08:25.783238+0000","last_peered":"2024-09-20T19:08:25.783238+0000","last_clean":"2024-09-20T19:08:25.783238+0000","last_became_active":"2024-09-20T19:06:56.479387+0000","last_became_peered":"2024-09-20T19:06:56.479387+0000","last_unstale":"2024-09-20T19:08:25.783238+0000","last_undegraded":"2024-09-20T19:08:25.783238+0000","last_fullsized":"2024-09-20T19:08:25.783238+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"442'3847","last_scrub_stamp":"2024-09-20T19:06:57.243081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:57.243081+0000","objects_scrubbed":1913,"log_size":6100,"log_dups_size":0,"ondisk_log_size":6100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":380,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":152502272,"num_objects":2327,"num_object_clones":0,"num_object_copies":6981,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2327,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2260,"num_write_kb":85568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"273'27","reported_seq":976,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.343436+0000","last_change":"2024-09-20T19:05:49.777488+0000","last_active":"2024-09-20T19:08:24.343436+0000","last_peered":"2024-09-20T19:08:24.343436+0000","last_clean":"2024-09-20T19:08:24.343436+0000","last_became_active":"2024-09-20T18:59:05.083726+0000","last_became_peered":"2024-09-20T18:59:05.083726+0000","last_unstale":"2024-09-20T19:08:24.343436+0000","last_undegraded":"2024-09-20T19:08:24.343436+0000","last_fullsized":"2024-09-20T19:08:24.343436+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'27","last_scrub_stamp":"2024-09-20T19:05:49.777414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:49.777414+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":949,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.341448+0000","last_change":"2024-09-20T19:06:25.718817+0000","last_active":"2024-09-20T19:08:24.341448+0000","last_peered":"2024-09-20T19:08:24.341448+0000","last_clean":"2024-09-20T19:08:24.341448+0000","last_became_active":"2024-09-20T19:04:20.884854+0000","last_became_peered":"2024-09-20T19:04:20.884854+0000","last_unstale":"2024-09-20T19:08:24.341448+0000","last_undegraded":"2024-09-20T19:08:24.341448+0000","last_fullsized":"2024-09-20T19:08:24.341448+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-20T19:06:25.718731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:06:25.718731+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"375'14","reported_seq":1158,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.340699+0000","last_change":"2024-09-20T19:06:14.693176+0000","last_active":"2024-09-20T19:08:24.340699+0000","last_peered":"2024-09-20T19:08:24.340699+0000","last_clean":"2024-09-20T19:08:24.340699+0000","last_became_active":"2024-09-20T19:04:20.892956+0000","last_became_peered":"2024-09-20T19:04:20.892956+0000","last_unstale":"2024-09-20T19:08:24.340699+0000","last_undegraded":"2024-09-20T19:08:24.340699+0000","last_fullsized":"2024-09-20T19:08:24.340699+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"375'14","last_scrub_stamp":"2024-09-20T19:06:14.693111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:06:14.693111+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":885,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.343665+0000","last_change":"2024-09-20T19:03:06.888620+0000","last_active":"2024-09-20T19:08:24.343665+0000","last_peered":"2024-09-20T19:08:24.343665+0000","last_clean":"2024-09-20T19:08:24.343665+0000","last_became_active":"2024-09-20T18:59:05.087005+0000","last_became_peered":"2024-09-20T18:59:05.087005+0000","last_unstale":"2024-09-20T19:08:24.343665+0000","last_undegraded":"2024-09-20T19:08:24.343665+0000","last_fullsized":"2024-09-20T19:08:24.343665+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:06.888563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:03:06.888563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.18","version":"472'5959","reported_seq":11593,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:26.805985+0000","last_change":"2024-09-20T19:07:43.036329+0000","last_active":"2024-09-20T19:08:26.805985+0000","last_peered":"2024-09-20T19:08:26.805985+0000","last_clean":"2024-09-20T19:08:26.805985+0000","last_became_active":"2024-09-20T19:07:42.984159+0000","last_became_peered":"2024-09-20T19:07:42.984159+0000","last_unstale":"2024-09-20T19:08:26.805985+0000","last_undegraded":"2024-09-20T19:08:26.805985+0000","last_fullsized":"2024-09-20T19:08:26.805985+0000","mapping_epoch":458,"log_start":"0'0","ondisk_log_start":"0'0","created":441,"last_epoch_clean":459,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":5959,"log_dups_size":0,"ondisk_log_size":5959,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":150175744,"num_objects":2292,"num_object_clones":0,"num_object_copies":6876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2292,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4122,"num_write_kb":205216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1833,"num_bytes_recovered":120127488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":446,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.339518+0000","last_change":"2024-09-20T19:01:16.542398+0000","last_active":"2024-09-20T19:08:24.339518+0000","last_peered":"2024-09-20T19:08:24.339518+0000","last_clean":"2024-09-20T19:08:24.339518+0000","last_became_active":"2024-09-20T18:59:05.085427+0000","last_became_peered":"2024-09-20T18:59:05.085427+0000","last_unstale":"2024-09-20T19:08:24.339518+0000","last_undegraded":"2024-09-20T19:08:24.339518+0000","last_fullsized":"2024-09-20T19:08:24.339518+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:16.542335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:16.542335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"273'12","reported_seq":904,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.334129+0000","last_change":"2024-09-20T19:06:19.973831+0000","last_active":"2024-09-20T19:08:24.334129+0000","last_peered":"2024-09-20T19:08:24.334129+0000","last_clean":"2024-09-20T19:08:24.334129+0000","last_became_active":"2024-09-20T18:49:48.500662+0000","last_became_peered":"2024-09-20T18:49:48.500662+0000","last_unstale":"2024-09-20T19:08:24.334129+0000","last_undegraded":"2024-09-20T19:08:24.334129+0000","last_fullsized":"2024-09-20T19:08:24.334129+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'12","last_scrub_stamp":"2024-09-20T19:06:19.973765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:06:19.973765+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":874,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.344533+0000","last_change":"2024-09-20T19:05:22.605885+0000","last_active":"2024-09-20T19:08:24.344533+0000","last_peered":"2024-09-20T19:08:24.344533+0000","last_clean":"2024-09-20T19:08:24.344533+0000","last_became_active":"2024-09-20T18:49:49.956749+0000","last_became_peered":"2024-09-20T18:49:49.956749+0000","last_unstale":"2024-09-20T19:08:24.344533+0000","last_undegraded":"2024-09-20T19:08:24.344533+0000","last_fullsized":"2024-09-20T19:08:24.344533+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:22.605839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:05:22.605839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":910,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.340727+0000","last_change":"2024-09-20T19:05:56.761494+0000","last_active":"2024-09-20T19:08:24.340727+0000","last_peered":"2024-09-20T19:08:24.340727+0000","last_clean":"2024-09-20T19:08:24.340727+0000","last_became_active":"2024-09-20T19:04:20.914115+0000","last_became_peered":"2024-09-20T19:04:20.914115+0000","last_unstale":"2024-09-20T19:08:24.340727+0000","last_undegraded":"2024-09-20T19:08:24.340727+0000","last_fullsized":"2024-09-20T19:08:24.340727+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:56.761425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:05:56.761425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.19","version":"472'6244","reported_seq":11503,"reported_epoch":472,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-20T19:08:28.134311+0000","last_change":"2024-09-20T19:07:39.432946+0000","last_active":"2024-09-20T19:08:28.134311+0000","last_peered":"2024-09-20T19:08:28.134311+0000","last_clean":"2024-09-20T19:06:57.404714+0000","last_became_active":"2024-09-20T19:06:59.810412+0000","last_became_peered":"2024-09-20T19:06:59.810412+0000","last_unstale":"2024-09-20T19:08:28.134311+0000","last_undegraded":"2024-09-20T19:08:28.134311+0000","last_fullsized":"2024-09-20T19:06:59.454934+0000","mapping_epoch":444,"log_start":"0'0","ondisk_log_start":"0'0","created":441,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":6244,"log_dups_size":0,"ondisk_log_size":6244,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":144277504,"num_objects":2202,"num_object_clones":0,"num_object_copies":6606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":452,"num_objects_unfound":0,"num_objects_dirty":2202,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4334,"num_write_kb":209120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1712,"num_bytes_recovered":112197632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,2],"avail_no_missing":["0","3","6"],"object_location_counts":[{"shards":"0,2","objects":1976},{"shards":"0,3,6","objects":226}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"273'15","reported_seq":888,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.337726+0000","last_change":"2024-09-20T19:01:15.875779+0000","last_active":"2024-09-20T19:08:24.337726+0000","last_peered":"2024-09-20T19:08:24.337726+0000","last_clean":"2024-09-20T19:08:24.337726+0000","last_became_active":"2024-09-20T18:49:48.500803+0000","last_became_peered":"2024-09-20T18:49:48.500803+0000","last_unstale":"2024-09-20T19:08:24.337726+0000","last_undegraded":"2024-09-20T19:08:24.337726+0000","last_fullsized":"2024-09-20T19:08:24.337726+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:01:15.875693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:15.875693+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":914,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.340941+0000","last_change":"2024-09-20T19:04:47.713764+0000","last_active":"2024-09-20T19:08:24.340941+0000","last_peered":"2024-09-20T19:08:24.340941+0000","last_clean":"2024-09-20T19:08:24.340941+0000","last_became_active":"2024-09-20T19:04:20.893250+0000","last_became_peered":"2024-09-20T19:04:20.893250+0000","last_unstale":"2024-09-20T19:08:24.340941+0000","last_undegraded":"2024-09-20T19:08:24.340941+0000","last_fullsized":"2024-09-20T19:08:24.340941+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:47.713585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:04:47.713585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":874,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.333995+0000","last_change":"2024-09-20T19:04:50.573619+0000","last_active":"2024-09-20T19:08:24.333995+0000","last_peered":"2024-09-20T19:08:24.333995+0000","last_clean":"2024-09-20T19:08:24.333995+0000","last_became_active":"2024-09-20T18:49:49.961947+0000","last_became_peered":"2024-09-20T18:49:49.961947+0000","last_unstale":"2024-09-20T19:08:24.333995+0000","last_undegraded":"2024-09-20T19:08:24.333995+0000","last_fullsized":"2024-09-20T19:08:24.333995+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:50.573550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:50.573550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"66'1","reported_seq":915,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.340072+0000","last_change":"2024-09-20T19:04:20.913449+0000","last_active":"2024-09-20T19:08:24.340072+0000","last_peered":"2024-09-20T19:08:24.340072+0000","last_clean":"2024-09-20T19:08:24.340072+0000","last_became_active":"2024-09-20T19:04:20.892236+0000","last_became_peered":"2024-09-20T19:04:20.892236+0000","last_unstale":"2024-09-20T19:08:24.340072+0000","last_undegraded":"2024-09-20T19:08:24.340072+0000","last_fullsized":"2024-09-20T19:08:24.340072+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T19:03:12.499684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:03:12.499684+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"274'15","reported_seq":895,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.344294+0000","last_change":"2024-09-20T19:06:42.967192+0000","last_active":"2024-09-20T19:08:24.344294+0000","last_peered":"2024-09-20T19:08:24.344294+0000","last_clean":"2024-09-20T19:08:24.344294+0000","last_became_active":"2024-09-20T18:49:48.501011+0000","last_became_peered":"2024-09-20T18:49:48.501011+0000","last_unstale":"2024-09-20T19:08:24.344294+0000","last_undegraded":"2024-09-20T19:08:24.344294+0000","last_fullsized":"2024-09-20T19:08:24.344294+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'15","last_scrub_stamp":"2024-09-20T19:06:42.967003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:06:42.967003+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":875,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.336479+0000","last_change":"2024-09-20T19:01:30.864866+0000","last_active":"2024-09-20T19:08:24.336479+0000","last_peered":"2024-09-20T19:08:24.336479+0000","last_clean":"2024-09-20T19:08:24.336479+0000","last_became_active":"2024-09-20T18:49:45.938511+0000","last_became_peered":"2024-09-20T18:49:45.938511+0000","last_unstale":"2024-09-20T19:08:24.336479+0000","last_undegraded":"2024-09-20T19:08:24.336479+0000","last_fullsized":"2024-09-20T19:08:24.336479+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:30.864794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:30.864794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":1106,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.333388+0000","last_change":"2024-09-20T19:04:41.601208+0000","last_active":"2024-09-20T19:08:24.333388+0000","last_peered":"2024-09-20T19:08:24.333388+0000","last_clean":"2024-09-20T19:08:24.333388+0000","last_became_active":"2024-09-20T18:49:49.960550+0000","last_became_peered":"2024-09-20T18:49:49.960550+0000","last_unstale":"2024-09-20T19:08:24.333388+0000","last_undegraded":"2024-09-20T19:08:24.333388+0000","last_fullsized":"2024-09-20T19:08:24.333388+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T19:04:41.601073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:41.601073+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":206,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.330150+0000","last_change":"2024-09-20T19:04:20.884397+0000","last_active":"2024-09-20T19:08:24.330150+0000","last_peered":"2024-09-20T19:08:24.330150+0000","last_clean":"2024-09-20T19:08:24.330150+0000","last_became_active":"2024-09-20T19:04:20.884100+0000","last_became_peered":"2024-09-20T19:04:20.884100+0000","last_unstale":"2024-09-20T19:08:24.330150+0000","last_undegraded":"2024-09-20T19:08:24.330150+0000","last_fullsized":"2024-09-20T19:08:24.330150+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:59.497810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:59:59.497810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"274'20","reported_seq":903,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.334231+0000","last_change":"2024-09-20T18:57:20.701592+0000","last_active":"2024-09-20T19:08:24.334231+0000","last_peered":"2024-09-20T19:08:24.334231+0000","last_clean":"2024-09-20T19:08:24.334231+0000","last_became_active":"2024-09-20T18:49:58.997494+0000","last_became_peered":"2024-09-20T18:49:58.997494+0000","last_unstale":"2024-09-20T19:08:24.334231+0000","last_undegraded":"2024-09-20T19:08:24.334231+0000","last_fullsized":"2024-09-20T19:08:24.334231+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:57:20.701543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:20.701543+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":892,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.341219+0000","last_change":"2024-09-20T19:04:20.884940+0000","last_active":"2024-09-20T19:08:24.341219+0000","last_peered":"2024-09-20T19:08:24.341219+0000","last_clean":"2024-09-20T19:08:24.341219+0000","last_became_active":"2024-09-20T19:04:20.884582+0000","last_became_peered":"2024-09-20T19:04:20.884582+0000","last_unstale":"2024-09-20T19:08:24.341219+0000","last_undegraded":"2024-09-20T19:08:24.341219+0000","last_fullsized":"2024-09-20T19:08:24.341219+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.632380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.632380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":1106,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.344183+0000","last_change":"2024-09-20T19:06:26.893183+0000","last_active":"2024-09-20T19:08:24.344183+0000","last_peered":"2024-09-20T19:08:24.344183+0000","last_clean":"2024-09-20T19:08:24.344183+0000","last_became_active":"2024-09-20T18:49:49.962243+0000","last_became_peered":"2024-09-20T18:49:49.962243+0000","last_unstale":"2024-09-20T19:08:24.344183+0000","last_undegraded":"2024-09-20T19:08:24.344183+0000","last_fullsized":"2024-09-20T19:08:24.344183+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T19:06:26.893109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:06:26.893109+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":879,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.337299+0000","last_change":"2024-09-20T19:05:52.578042+0000","last_active":"2024-09-20T19:08:24.337299+0000","last_peered":"2024-09-20T19:08:24.337299+0000","last_clean":"2024-09-20T19:08:24.337299+0000","last_became_active":"2024-09-20T18:49:51.969093+0000","last_became_peered":"2024-09-20T18:49:51.969093+0000","last_unstale":"2024-09-20T19:08:24.337299+0000","last_undegraded":"2024-09-20T19:08:24.337299+0000","last_fullsized":"2024-09-20T19:08:24.337299+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:52.577979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:05:52.577979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"273'19","reported_seq":226,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.329602+0000","last_change":"2024-09-20T19:06:44.432152+0000","last_active":"2024-09-20T19:08:24.329602+0000","last_peered":"2024-09-20T19:08:24.329602+0000","last_clean":"2024-09-20T19:08:24.329602+0000","last_became_active":"2024-09-20T19:04:20.897330+0000","last_became_peered":"2024-09-20T19:04:20.897330+0000","last_unstale":"2024-09-20T19:08:24.329602+0000","last_undegraded":"2024-09-20T19:08:24.329602+0000","last_fullsized":"2024-09-20T19:08:24.329602+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'19","last_scrub_stamp":"2024-09-20T19:06:44.432058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:06:44.432058+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":206,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.329345+0000","last_change":"2024-09-20T19:04:20.896732+0000","last_active":"2024-09-20T19:08:24.329345+0000","last_peered":"2024-09-20T19:08:24.329345+0000","last_clean":"2024-09-20T19:08:24.329345+0000","last_became_active":"2024-09-20T19:04:20.896007+0000","last_became_peered":"2024-09-20T19:04:20.896007+0000","last_unstale":"2024-09-20T19:08:24.329345+0000","last_undegraded":"2024-09-20T19:08:24.329345+0000","last_fullsized":"2024-09-20T19:08:24.329345+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:59:06.310027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:59:06.310027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":866,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.333675+0000","last_change":"2024-09-20T19:02:38.906352+0000","last_active":"2024-09-20T19:08:24.333675+0000","last_peered":"2024-09-20T19:08:24.333675+0000","last_clean":"2024-09-20T19:08:24.333675+0000","last_became_active":"2024-09-20T18:49:49.957067+0000","last_became_peered":"2024-09-20T18:49:49.957067+0000","last_unstale":"2024-09-20T19:08:24.333675+0000","last_undegraded":"2024-09-20T19:08:24.333675+0000","last_fullsized":"2024-09-20T19:08:24.333675+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:38.906161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:38.906161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":846,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.332996+0000","last_change":"2024-09-20T18:57:16.789412+0000","last_active":"2024-09-20T19:08:24.332996+0000","last_peered":"2024-09-20T19:08:24.332996+0000","last_clean":"2024-09-20T19:08:24.332996+0000","last_became_active":"2024-09-20T18:49:51.966915+0000","last_became_peered":"2024-09-20T18:49:51.966915+0000","last_unstale":"2024-09-20T19:08:24.332996+0000","last_undegraded":"2024-09-20T19:08:24.332996+0000","last_fullsized":"2024-09-20T19:08:24.332996+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:16.789095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:16.789095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"273'15","reported_seq":945,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.333814+0000","last_change":"2024-09-20T19:06:51.012046+0000","last_active":"2024-09-20T19:08:24.333814+0000","last_peered":"2024-09-20T19:08:24.333814+0000","last_clean":"2024-09-20T19:08:24.333814+0000","last_became_active":"2024-09-20T18:59:05.085230+0000","last_became_peered":"2024-09-20T18:59:05.085230+0000","last_unstale":"2024-09-20T19:08:24.333814+0000","last_undegraded":"2024-09-20T19:08:24.333814+0000","last_fullsized":"2024-09-20T19:08:24.333814+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:06:51.011969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:06:51.011969+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":892,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.340966+0000","last_change":"2024-09-20T19:04:20.885127+0000","last_active":"2024-09-20T19:08:24.340966+0000","last_peered":"2024-09-20T19:08:24.340966+0000","last_clean":"2024-09-20T19:08:24.340966+0000","last_became_active":"2024-09-20T19:04:20.884660+0000","last_became_peered":"2024-09-20T19:04:20.884660+0000","last_unstale":"2024-09-20T19:08:24.340966+0000","last_undegraded":"2024-09-20T19:08:24.340966+0000","last_fullsized":"2024-09-20T19:08:24.340966+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:07.408197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:03:07.408197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":882,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.333610+0000","last_change":"2024-09-20T19:06:22.930775+0000","last_active":"2024-09-20T19:08:24.333610+0000","last_peered":"2024-09-20T19:08:24.333610+0000","last_clean":"2024-09-20T19:08:24.333610+0000","last_became_active":"2024-09-20T18:49:49.962523+0000","last_became_peered":"2024-09-20T18:49:49.962523+0000","last_unstale":"2024-09-20T19:08:24.333610+0000","last_undegraded":"2024-09-20T19:08:24.333610+0000","last_fullsized":"2024-09-20T19:08:24.333610+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:22.930701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:06:22.930701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":871,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.336461+0000","last_change":"2024-09-20T19:05:51.626774+0000","last_active":"2024-09-20T19:08:24.336461+0000","last_peered":"2024-09-20T19:08:24.336461+0000","last_clean":"2024-09-20T19:08:24.336461+0000","last_became_active":"2024-09-20T18:49:51.967313+0000","last_became_peered":"2024-09-20T18:49:51.967313+0000","last_unstale":"2024-09-20T19:08:24.336461+0000","last_undegraded":"2024-09-20T19:08:24.336461+0000","last_fullsized":"2024-09-20T19:08:24.336461+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:51.626697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:05:51.626697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"274'19","reported_seq":940,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.332899+0000","last_change":"2024-09-20T19:01:05.495603+0000","last_active":"2024-09-20T19:08:24.332899+0000","last_peered":"2024-09-20T19:08:24.332899+0000","last_clean":"2024-09-20T19:08:24.332899+0000","last_became_active":"2024-09-20T18:59:05.075390+0000","last_became_peered":"2024-09-20T18:59:05.075390+0000","last_unstale":"2024-09-20T19:08:24.332899+0000","last_undegraded":"2024-09-20T19:08:24.332899+0000","last_fullsized":"2024-09-20T19:08:24.332899+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'19","last_scrub_stamp":"2024-09-20T19:01:05.495478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:05.495478+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":890,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.344612+0000","last_change":"2024-09-20T19:05:43.686902+0000","last_active":"2024-09-20T19:08:24.344612+0000","last_peered":"2024-09-20T19:08:24.344612+0000","last_clean":"2024-09-20T19:08:24.344612+0000","last_became_active":"2024-09-20T18:49:45.942105+0000","last_became_peered":"2024-09-20T18:49:45.942105+0000","last_unstale":"2024-09-20T19:08:24.344612+0000","last_undegraded":"2024-09-20T19:08:24.344612+0000","last_fullsized":"2024-09-20T19:08:24.344612+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:43.686842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:43.686842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":906,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.340354+0000","last_change":"2024-09-20T19:04:20.912120+0000","last_active":"2024-09-20T19:08:24.340354+0000","last_peered":"2024-09-20T19:08:24.340354+0000","last_clean":"2024-09-20T19:08:24.340354+0000","last_became_active":"2024-09-20T19:04:20.911292+0000","last_became_peered":"2024-09-20T19:04:20.911292+0000","last_unstale":"2024-09-20T19:08:24.340354+0000","last_undegraded":"2024-09-20T19:08:24.340354+0000","last_fullsized":"2024-09-20T19:08:24.340354+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:27.267254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:27.267254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":884,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.333769+0000","last_change":"2024-09-20T19:06:58.896885+0000","last_active":"2024-09-20T19:08:24.333769+0000","last_peered":"2024-09-20T19:08:24.333769+0000","last_clean":"2024-09-20T19:08:24.333769+0000","last_became_active":"2024-09-20T18:59:05.081179+0000","last_became_peered":"2024-09-20T18:59:05.081179+0000","last_unstale":"2024-09-20T19:08:24.333769+0000","last_undegraded":"2024-09-20T19:08:24.333769+0000","last_fullsized":"2024-09-20T19:08:24.333769+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:58.896826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:06:58.896826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"273'8","reported_seq":925,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.332232+0000","last_change":"2024-09-20T19:06:21.974151+0000","last_active":"2024-09-20T19:08:24.332232+0000","last_peered":"2024-09-20T19:08:24.332232+0000","last_clean":"2024-09-20T19:08:24.332232+0000","last_became_active":"2024-09-20T18:59:05.076365+0000","last_became_peered":"2024-09-20T18:59:05.076365+0000","last_unstale":"2024-09-20T19:08:24.332232+0000","last_undegraded":"2024-09-20T19:08:24.332232+0000","last_fullsized":"2024-09-20T19:08:24.332232+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'8","last_scrub_stamp":"2024-09-20T19:06:21.974084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:06:21.974084+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":891,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.337027+0000","last_change":"2024-09-20T19:06:45.833934+0000","last_active":"2024-09-20T19:08:24.337027+0000","last_peered":"2024-09-20T19:08:24.337027+0000","last_clean":"2024-09-20T19:08:24.337027+0000","last_became_active":"2024-09-20T18:49:45.940988+0000","last_became_peered":"2024-09-20T18:49:45.940988+0000","last_unstale":"2024-09-20T19:08:24.337027+0000","last_undegraded":"2024-09-20T19:08:24.337027+0000","last_fullsized":"2024-09-20T19:08:24.337027+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:45.833872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:06:45.833872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":471,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.339171+0000","last_change":"2024-09-20T19:04:20.900567+0000","last_active":"2024-09-20T19:08:24.339171+0000","last_peered":"2024-09-20T19:08:24.339171+0000","last_clean":"2024-09-20T19:08:24.339171+0000","last_became_active":"2024-09-20T19:04:20.900214+0000","last_became_peered":"2024-09-20T19:04:20.900214+0000","last_unstale":"2024-09-20T19:08:24.339171+0000","last_undegraded":"2024-09-20T19:08:24.339171+0000","last_fullsized":"2024-09-20T19:08:24.339171+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:41.497115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:41.497115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":888,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.340693+0000","last_change":"2024-09-20T19:05:57.789513+0000","last_active":"2024-09-20T19:08:24.340693+0000","last_peered":"2024-09-20T19:08:24.340693+0000","last_clean":"2024-09-20T19:08:24.340693+0000","last_became_active":"2024-09-20T19:04:20.885211+0000","last_became_peered":"2024-09-20T19:04:20.885211+0000","last_unstale":"2024-09-20T19:08:24.340693+0000","last_undegraded":"2024-09-20T19:08:24.340693+0000","last_fullsized":"2024-09-20T19:08:24.340693+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:57.789465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:05:57.789465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.10","version":"472'6190","reported_seq":13988,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:28.151717+0000","last_change":"2024-09-20T19:08:24.353465+0000","last_active":"2024-09-20T19:08:28.151717+0000","last_peered":"2024-09-20T19:08:28.151717+0000","last_clean":"2024-09-20T19:08:28.151717+0000","last_became_active":"2024-09-20T19:08:24.341564+0000","last_became_peered":"2024-09-20T19:08:24.341564+0000","last_unstale":"2024-09-20T19:08:28.151717+0000","last_undegraded":"2024-09-20T19:08:28.151717+0000","last_fullsized":"2024-09-20T19:08:28.151717+0000","mapping_epoch":471,"log_start":"0'0","ondisk_log_start":"0'0","created":441,"last_epoch_clean":472,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":6190,"log_dups_size":0,"ondisk_log_size":6190,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":144932864,"num_objects":2212,"num_object_clones":0,"num_object_copies":6636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4278,"num_write_kb":207648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1917,"num_bytes_recovered":125632512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":871,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.334198+0000","last_change":"2024-09-20T19:06:24.854187+0000","last_active":"2024-09-20T19:08:24.334198+0000","last_peered":"2024-09-20T19:08:24.334198+0000","last_clean":"2024-09-20T19:08:24.334198+0000","last_became_active":"2024-09-20T19:04:20.890101+0000","last_became_peered":"2024-09-20T19:04:20.890101+0000","last_unstale":"2024-09-20T19:08:24.334198+0000","last_undegraded":"2024-09-20T19:08:24.334198+0000","last_fullsized":"2024-09-20T19:08:24.334198+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:24.854124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:06:24.854124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":874,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.332702+0000","last_change":"2024-09-20T19:06:44.934346+0000","last_active":"2024-09-20T19:08:24.332702+0000","last_peered":"2024-09-20T19:08:24.332702+0000","last_clean":"2024-09-20T19:08:24.332702+0000","last_became_active":"2024-09-20T18:49:45.937992+0000","last_became_peered":"2024-09-20T18:49:45.937992+0000","last_unstale":"2024-09-20T19:08:24.332702+0000","last_undegraded":"2024-09-20T19:08:24.332702+0000","last_fullsized":"2024-09-20T19:08:24.332702+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:44.934143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:06:44.934143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"274'19","reported_seq":478,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.339235+0000","last_change":"2024-09-20T19:06:26.909675+0000","last_active":"2024-09-20T19:08:24.339235+0000","last_peered":"2024-09-20T19:08:24.339235+0000","last_clean":"2024-09-20T19:08:24.339235+0000","last_became_active":"2024-09-20T18:59:05.089294+0000","last_became_peered":"2024-09-20T18:59:05.089294+0000","last_unstale":"2024-09-20T19:08:24.339235+0000","last_undegraded":"2024-09-20T19:08:24.339235+0000","last_fullsized":"2024-09-20T19:08:24.339235+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'19","last_scrub_stamp":"2024-09-20T19:06:26.909593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:06:26.909593+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":861,"reported_epoch":472,"state":"active+clean","last_fresh":"2024-09-20T19:08:24.337406+0000","last_change":"2024-09-20T19:01:40.961449+0000","last_active":"2024-09-20T19:08:24.337406+0000","last_peered":"2024-09-20T19:08:24.337406+0000","last_clean":"2024-09-20T19:08:24.337406+0000","last_became_active":"2024-09-20T18:49:49.957582+0000","last_became_peered":"2024-09-20T18:49:49.957582+0000","last_unstale":"2024-09-20T19:08:24.337406+0000","last_undegraded":"2024-09-20T19:08:24.337406+0000","last_fullsized":"2024-09-20T19:08:24.337406+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:40.961363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:40.961363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":26,"stat_sum":{"num_bytes":4679073816,"num_objects":71400,"num_object_clones":0,"num_object_copies":214200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1910,"num_objects_unfound":0,"num_objects_dirty":71400,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":109480,"num_write_kb":5133633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16579,"num_bytes_recovered":1086521344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":15141253120,"data_stored":15141240904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150002,"ondisk_log_size":150002,"up":78,"acting":75,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1185,"internal_metadata":0},"log_size":87,"ondisk_log_size":87,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1114,"num_read_kb":871,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":257,"num_bytes_recovered":4890,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":686,"ondisk_log_size":686,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":3105,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1113,"num_read_kb":1108,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":225,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1610095,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":869,"ondisk_log_size":869,"up":33,"acting":33,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266916,"num_pgs":72,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2694464,"kb_used_data":1969072,"kb_used_omap":1,"kb_used_meta":725310,"kb_avail":91050688,"statfs":{"total":95995035648,"available":93235904512,"internally_reserved":0,"allocated":2016329728,"data_stored":2015741682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":742717898},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528555,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3006064,"kb_used_data":2415392,"kb_used_omap":1,"kb_used_meta":590654,"kb_avail":90739088,"statfs":{"total":95995035648,"available":92916826112,"internally_reserved":0,"allocated":2473361408,"data_stored":2472789108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":604830150},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757488,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1817460,"kb_used_data":1154100,"kb_used_omap":1,"kb_used_meta":662846,"kb_avail":91927692,"statfs":{"total":95995035648,"available":94133956608,"internally_reserved":0,"allocated":1181798400,"data_stored":1181206857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":678754757},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":304,"seq":1305670058070,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2005508,"kb_used_data":1595780,"kb_used_omap":5,"kb_used_meta":409338,"kb_avail":91739644,"statfs":{"total":95995035648,"available":93941395456,"internally_reserved":0,"allocated":1634078720,"data_stored":1633471167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5633,"internal_metadata":419162623},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182652,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2551856,"kb_used_data":1726064,"kb_used_omap":1,"kb_used_meta":825278,"kb_avail":91193296,"statfs":{"total":95995035648,"available":93381935104,"internally_reserved":0,"allocated":1767489536,"data_stored":1766901469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":845085127},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,3,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411585,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2868924,"kb_used_data":2107500,"kb_used_omap":1,"kb_used_meta":761406,"kb_avail":90876228,"statfs":{"total":95995035648,"available":93057257472,"internally_reserved":0,"allocated":2158080000,"data_stored":2157496894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":779680197},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,0,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575111,"num_pgs":68,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3223524,"kb_used_data":2420308,"kb_used_omap":1,"kb_used_meta":803070,"kb_avail":90521628,"statfs":{"total":95995035648,"available":92694147072,"internally_reserved":0,"allocated":2478395392,"data_stored":2477771942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":822344133},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738637,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2351552,"kb_used_data":1530352,"kb_used_omap":1,"kb_used_meta":821118,"kb_avail":91393600,"statfs":{"total":95995035648,"available":93587046400,"internally_reserved":0,"allocated":1567080448,"data_stored":1566492098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":840825285},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1185,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1546911744,"data_stored":1546911744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2453602304,"data_stored":2453602304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2132414464,"data_stored":2132410392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1743585280,"data_stored":1743585280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1617104896,"data_stored":1617100824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1178533888,"data_stored":1178533888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2466320384,"data_stored":2466316312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2002780160,"data_stored":2002780160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T19:08:37.243 INFO:tasks.thrashosds.thrasher:Setting 1.5 to [1, 7, 0] 2024-09-20T19:08:37.243 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '1.5', '1', '7', '0'] 2024-09-20T19:08:37.244 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd pg-upmap 1.5 1 7 0 2024-09-20T19:08:37.261 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:37.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:37.262 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:37.363 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:37.391 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:37.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:37.392 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:37.492 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:37.523 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:37.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:37.524 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:37.625 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:08:37.664 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:37.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:37.665 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:08:37.766 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:08:37.809 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:37.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:37.809 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:08:37.812 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T19:08:37.843 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-20T19:08:37.911 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:37.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:37.975 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:37.975 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:38.010 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:08:37 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:08:38.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:37 smithi142 ceph-mon[32508]: 11.12 scrub ok 2024-09-20T19:08:38.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:37 smithi142 ceph-mon[32508]: 11.18 scrub ok 2024-09-20T19:08:38.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:37 smithi142 ceph-mon[32508]: 6.4 scrub starts 2024-09-20T19:08:38.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:37 smithi142 ceph-mon[32508]: 6.4 scrub ok 2024-09-20T19:08:38.042 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:37 smithi083 ceph-mon[29421]: 11.12 scrub ok 2024-09-20T19:08:38.042 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:37 smithi083 ceph-mon[29421]: 11.18 scrub ok 2024-09-20T19:08:38.042 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:37 smithi083 ceph-mon[29421]: 6.4 scrub starts 2024-09-20T19:08:38.042 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:37 smithi083 ceph-mon[29421]: 6.4 scrub ok 2024-09-20T19:08:38.043 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:37 smithi083 ceph-mon[36260]: 11.12 scrub ok 2024-09-20T19:08:38.043 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:37 smithi083 ceph-mon[36260]: 11.18 scrub ok 2024-09-20T19:08:38.043 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:37 smithi083 ceph-mon[36260]: 6.4 scrub starts 2024-09-20T19:08:38.043 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:37 smithi083 ceph-mon[36260]: 6.4 scrub ok 2024-09-20T19:08:38.076 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:38.124 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:38.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:38.140 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:38.241 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:08:38.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:38.288 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:38.288 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:08:38.389 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:08:38.416 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:08:38.427 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:08:38.427 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:08:38.451 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:08:38.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:38.452 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:38.452 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:08:38.453 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:08:38.467 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:08:38.467 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:08:38.506 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:08:38.506 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:08:38.538 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:08:38.538 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:08:38.553 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:38.575 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:08:38.576 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:08:38.603 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:38.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:38.605 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:38.607 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:08:38.607 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:08:38.641 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:08:38.641 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:08:38.676 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:08:38.676 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:08:38.706 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:38.736 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:38.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:38.737 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:38.837 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:38.868 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:38.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:38.869 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:38.970 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:39.009 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:39.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:39.010 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:39.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:38 smithi142 ceph-mon[32508]: pgmap v956: 168 pgs: 2 active+clean+scrubbing, 1 peering, 165 active+clean; 3.8 GiB data, 18 GiB used, 697 GiB / 715 GiB avail 2024-09-20T19:08:39.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:38 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:08:39.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:38 smithi142 ceph-mon[32508]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2 pgs undersized) 2024-09-20T19:08:39.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:38 smithi142 ceph-mon[32508]: Cluster is now healthy 2024-09-20T19:08:39.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:38 smithi142 ceph-mon[32508]: 6.1a scrub starts 2024-09-20T19:08:39.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:38 smithi142 ceph-mon[32508]: 6.1a scrub ok 2024-09-20T19:08:39.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:38 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2793997831' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:08:39.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:38 smithi142 ceph-mon[32508]: 4.16 scrub starts 2024-09-20T19:08:39.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:38 smithi142 ceph-mon[32508]: 4.16 scrub ok 2024-09-20T19:08:39.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:38 smithi083 ceph-mon[29421]: pgmap v956: 168 pgs: 2 active+clean+scrubbing, 1 peering, 165 active+clean; 3.8 GiB data, 18 GiB used, 697 GiB / 715 GiB avail 2024-09-20T19:08:39.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:38 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:08:39.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:38 smithi083 ceph-mon[29421]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2 pgs undersized) 2024-09-20T19:08:39.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:38 smithi083 ceph-mon[29421]: Cluster is now healthy 2024-09-20T19:08:39.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:38 smithi083 ceph-mon[29421]: 6.1a scrub starts 2024-09-20T19:08:39.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:38 smithi083 ceph-mon[29421]: 6.1a scrub ok 2024-09-20T19:08:39.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:38 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2793997831' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:08:39.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:38 smithi083 ceph-mon[29421]: 4.16 scrub starts 2024-09-20T19:08:39.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:38 smithi083 ceph-mon[29421]: 4.16 scrub ok 2024-09-20T19:08:39.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:38 smithi083 ceph-mon[36260]: pgmap v956: 168 pgs: 2 active+clean+scrubbing, 1 peering, 165 active+clean; 3.8 GiB data, 18 GiB used, 697 GiB / 715 GiB avail 2024-09-20T19:08:39.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:38 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:08:39.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:38 smithi083 ceph-mon[36260]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2 pgs undersized) 2024-09-20T19:08:39.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:38 smithi083 ceph-mon[36260]: Cluster is now healthy 2024-09-20T19:08:39.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:38 smithi083 ceph-mon[36260]: 6.1a scrub starts 2024-09-20T19:08:39.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:38 smithi083 ceph-mon[36260]: 6.1a scrub ok 2024-09-20T19:08:39.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:38 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2793997831' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:08:39.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:38 smithi083 ceph-mon[36260]: 4.16 scrub starts 2024-09-20T19:08:39.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:38 smithi083 ceph-mon[36260]: 4.16 scrub ok 2024-09-20T19:08:39.111 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:39.146 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:39.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:39.147 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:39.247 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:08:39.292 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:39.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:39.293 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:08:39.393 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:39.429 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:39.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:39.430 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:39.531 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:39.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:39.571 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:39.571 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:39.672 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:39.693 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:39.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:39.694 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:39.732 INFO:teuthology.orchestra.run.smithi083.stderr:set 1.5 pg_upmap mapping to [1,7,0] 2024-09-20T19:08:39.795 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:39.829 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:39.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:39.830 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:39.930 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:39.965 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:39.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:39.966 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:40.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:39 smithi142 ceph-mon[32508]: 3.10 scrub starts 2024-09-20T19:08:40.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:39 smithi142 ceph-mon[32508]: 3.10 scrub ok 2024-09-20T19:08:40.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:39 smithi142 ceph-mon[32508]: 1.0 scrub starts 2024-09-20T19:08:40.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:39 smithi142 ceph-mon[32508]: 1.0 scrub ok 2024-09-20T19:08:40.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:39 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2526668401' entity='client.admin' cmd={"prefix": "osd pg-upmap", "pgid": "1.5", "id": [1, 7, 0]} : dispatch 2024-09-20T19:08:40.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:39 smithi083 ceph-mon[29421]: 3.10 scrub starts 2024-09-20T19:08:40.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:39 smithi083 ceph-mon[29421]: 3.10 scrub ok 2024-09-20T19:08:40.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:39 smithi083 ceph-mon[29421]: 1.0 scrub starts 2024-09-20T19:08:40.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:39 smithi083 ceph-mon[29421]: 1.0 scrub ok 2024-09-20T19:08:40.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:39 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2526668401' entity='client.admin' cmd={"prefix": "osd pg-upmap", "pgid": "1.5", "id": [1, 7, 0]} : dispatch 2024-09-20T19:08:40.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:39 smithi083 ceph-mon[36260]: 3.10 scrub starts 2024-09-20T19:08:40.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:39 smithi083 ceph-mon[36260]: 3.10 scrub ok 2024-09-20T19:08:40.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:39 smithi083 ceph-mon[36260]: 1.0 scrub starts 2024-09-20T19:08:40.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:39 smithi083 ceph-mon[36260]: 1.0 scrub ok 2024-09-20T19:08:40.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:39 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2526668401' entity='client.admin' cmd={"prefix": "osd pg-upmap", "pgid": "1.5", "id": [1, 7, 0]} : dispatch 2024-09-20T19:08:40.067 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:40.096 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:40.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:40.097 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:40.197 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:40.227 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:40.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:40.229 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:40.329 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:40.356 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:40.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:40.357 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:40.458 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:40.500 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:40.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:40.501 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:40.602 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:40.632 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:40.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:40.633 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:40.734 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:40.742 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:08:40 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:08:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:08:40.743 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:40 smithi083 ceph-mon[29421]: pgmap v957: 168 pgs: 2 active+clean+scrubbing, 166 active+clean; 3.7 GiB data, 17 GiB used, 698 GiB / 715 GiB avail 2024-09-20T19:08:40.779 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:40.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:40.780 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:40.881 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:40.918 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:40.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:40.919 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:41.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:40 smithi142 ceph-mon[32508]: pgmap v957: 168 pgs: 2 active+clean+scrubbing, 166 active+clean; 3.7 GiB data, 17 GiB used, 698 GiB / 715 GiB avail 2024-09-20T19:08:41.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:40 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2526668401' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "1.5", "id": [1, 7, 0]}]': finished 2024-09-20T19:08:41.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:40 smithi142 ceph-mon[32508]: osdmap e477: 8 total, 8 up, 8 in 2024-09-20T19:08:41.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:40 smithi142 ceph-mon[32508]: 3.14 scrub starts 2024-09-20T19:08:41.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:40 smithi142 ceph-mon[32508]: 3.14 scrub ok 2024-09-20T19:08:41.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:40 smithi142 ceph-mon[32508]: 1.2 scrub starts 2024-09-20T19:08:41.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:40 smithi142 ceph-mon[32508]: 1.2 scrub ok 2024-09-20T19:08:41.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:40 smithi142 ceph-mon[32508]: 3.0 scrub starts 2024-09-20T19:08:41.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:40 smithi142 ceph-mon[32508]: 3.0 scrub ok 2024-09-20T19:08:41.020 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:41.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:40 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2526668401' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "1.5", "id": [1, 7, 0]}]': finished 2024-09-20T19:08:41.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:40 smithi083 ceph-mon[29421]: osdmap e477: 8 total, 8 up, 8 in 2024-09-20T19:08:41.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:40 smithi083 ceph-mon[29421]: 3.14 scrub starts 2024-09-20T19:08:41.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:40 smithi083 ceph-mon[29421]: 3.14 scrub ok 2024-09-20T19:08:41.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:40 smithi083 ceph-mon[29421]: 1.2 scrub starts 2024-09-20T19:08:41.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:40 smithi083 ceph-mon[29421]: 1.2 scrub ok 2024-09-20T19:08:41.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:40 smithi083 ceph-mon[29421]: 3.0 scrub starts 2024-09-20T19:08:41.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:40 smithi083 ceph-mon[29421]: 3.0 scrub ok 2024-09-20T19:08:41.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:40 smithi083 ceph-mon[36260]: pgmap v957: 168 pgs: 2 active+clean+scrubbing, 166 active+clean; 3.7 GiB data, 17 GiB used, 698 GiB / 715 GiB avail 2024-09-20T19:08:41.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:40 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2526668401' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "1.5", "id": [1, 7, 0]}]': finished 2024-09-20T19:08:41.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:40 smithi083 ceph-mon[36260]: osdmap e477: 8 total, 8 up, 8 in 2024-09-20T19:08:41.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:40 smithi083 ceph-mon[36260]: 3.14 scrub starts 2024-09-20T19:08:41.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:40 smithi083 ceph-mon[36260]: 3.14 scrub ok 2024-09-20T19:08:41.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:40 smithi083 ceph-mon[36260]: 1.2 scrub starts 2024-09-20T19:08:41.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:40 smithi083 ceph-mon[36260]: 1.2 scrub ok 2024-09-20T19:08:41.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:40 smithi083 ceph-mon[36260]: 3.0 scrub starts 2024-09-20T19:08:41.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:40 smithi083 ceph-mon[36260]: 3.0 scrub ok 2024-09-20T19:08:41.057 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:41.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:41.058 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:41.159 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:41.185 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:41.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:41.186 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:41.287 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:41.312 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:41.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:41.313 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:41.413 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:41.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:41.450 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:41.450 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:41.550 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:41.577 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:41.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:41.578 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:41.678 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:08:41.716 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:41.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:41.716 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:08:41.817 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:41.857 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:41.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:41.858 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:41.959 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:41.974 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-20T19:08:41.996 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:41.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:41.997 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:42.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:41 smithi142 ceph-mon[32508]: 5.11 scrub starts 2024-09-20T19:08:42.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:41 smithi142 ceph-mon[32508]: 5.11 scrub ok 2024-09-20T19:08:42.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:41 smithi142 ceph-mon[32508]: 11.16 scrub starts 2024-09-20T19:08:42.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:41 smithi142 ceph-mon[32508]: osdmap e478: 8 total, 8 up, 8 in 2024-09-20T19:08:42.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:41 smithi142 ceph-mon[32508]: 11.16 scrub ok 2024-09-20T19:08:42.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:41 smithi142 ceph-mon[32508]: 1.5 scrub starts 2024-09-20T19:08:42.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:41 smithi142 ceph-mon[32508]: 1.5 scrub ok 2024-09-20T19:08:42.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:41 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/295221288' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:08:42.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:41 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:08:42.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:41 smithi142 ceph-mon[32508]: 3.1f scrub starts 2024-09-20T19:08:42.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:41 smithi142 ceph-mon[32508]: 3.1f scrub ok 2024-09-20T19:08:42.025 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:41 smithi083 ceph-mon[29421]: 5.11 scrub starts 2024-09-20T19:08:42.025 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:41 smithi083 ceph-mon[29421]: 5.11 scrub ok 2024-09-20T19:08:42.025 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:41 smithi083 ceph-mon[29421]: 11.16 scrub starts 2024-09-20T19:08:42.025 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:41 smithi083 ceph-mon[29421]: osdmap e478: 8 total, 8 up, 8 in 2024-09-20T19:08:42.025 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:41 smithi083 ceph-mon[29421]: 11.16 scrub ok 2024-09-20T19:08:42.026 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:41 smithi083 ceph-mon[29421]: 1.5 scrub starts 2024-09-20T19:08:42.026 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:41 smithi083 ceph-mon[29421]: 1.5 scrub ok 2024-09-20T19:08:42.026 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:41 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/295221288' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:08:42.026 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:41 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:08:42.026 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:41 smithi083 ceph-mon[29421]: 3.1f scrub starts 2024-09-20T19:08:42.026 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:41 smithi083 ceph-mon[29421]: 3.1f scrub ok 2024-09-20T19:08:42.026 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:41 smithi083 ceph-mon[36260]: 5.11 scrub starts 2024-09-20T19:08:42.026 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:41 smithi083 ceph-mon[36260]: 5.11 scrub ok 2024-09-20T19:08:42.026 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:41 smithi083 ceph-mon[36260]: 11.16 scrub starts 2024-09-20T19:08:42.027 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:41 smithi083 ceph-mon[36260]: osdmap e478: 8 total, 8 up, 8 in 2024-09-20T19:08:42.027 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:41 smithi083 ceph-mon[36260]: 11.16 scrub ok 2024-09-20T19:08:42.027 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:41 smithi083 ceph-mon[36260]: 1.5 scrub starts 2024-09-20T19:08:42.027 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:41 smithi083 ceph-mon[36260]: 1.5 scrub ok 2024-09-20T19:08:42.027 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:41 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/295221288' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:08:42.027 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:41 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:08:42.027 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:41 smithi083 ceph-mon[36260]: 3.1f scrub starts 2024-09-20T19:08:42.027 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:41 smithi083 ceph-mon[36260]: 3.1f scrub ok 2024-09-20T19:08:42.098 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:42.133 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:42.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:42.135 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:42.235 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:08:42.273 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:42.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:42.275 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:08:42.375 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:42.412 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:42.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:42.413 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:42.514 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:08:42.543 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T19:08:42.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:42.556 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:42.556 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:08:42.657 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:42.688 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:42.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:42.689 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:42.790 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:42.824 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:42.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:42.825 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:42.913 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T19:08:42.913 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-20T19:08:42.913 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.913 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.26264.0:151621 11.11 11:8eba783c:::benchmark_data_smithi083_177491_object48179:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e478)", 2024-09-20T19:08:42.913 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:08:41.246341+0000", 2024-09-20T19:08:42.913 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0098065730000000007, 2024-09-20T19:08:42.913 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0098363099999999992, 2024-09-20T19:08:42.913 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:08:42.914 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:08:42.914 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "waiting for sub ops", 2024-09-20T19:08:42.914 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:08:42.914 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.26264", 2024-09-20T19:08:42.914 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2608780316", 2024-09-20T19:08:42.914 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 151621 2024-09-20T19:08:42.914 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.914 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:08:42.914 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.915 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:08:42.915 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.246341+0000", 2024-09-20T19:08:42.915 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:42.915 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.915 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.915 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:08:42.915 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.246341+0000", 2024-09-20T19:08:42.915 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:42.915 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.916 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.916 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:08:42.916 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.246342+0000", 2024-09-20T19:08:42.916 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.263e-06 2024-09-20T19:08:42.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.916 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.916 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:08:42.916 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.246347+0000", 2024-09-20T19:08:42.916 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9370000000000003e-06 2024-09-20T19:08:42.917 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.917 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.917 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:08:42.917 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.246356+0000", 2024-09-20T19:08:42.917 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6829999999999997e-06 2024-09-20T19:08:42.917 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.917 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.917 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:08:42.917 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.246362+0000", 2024-09-20T19:08:42.918 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1480000000000003e-06 2024-09-20T19:08:42.918 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.918 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.918 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:08:42.918 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.246920+0000", 2024-09-20T19:08:42.918 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00055839599999999996 2024-09-20T19:08:42.918 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.918 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.918 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:08:42.918 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.247021+0000", 2024-09-20T19:08:42.919 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010027900000000001 2024-09-20T19:08:42.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.919 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.919 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3,7", 2024-09-20T19:08:42.919 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.247384+0000", 2024-09-20T19:08:42.919 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00036348700000000001 2024-09-20T19:08:42.919 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:42.919 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:42.919 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:42.920 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.920 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.920 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.26264.0:151622 11.d 11:b46e5efd:::benchmark_data_smithi083_177491_object48180:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e478)", 2024-09-20T19:08:42.920 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:08:41.248220+0000", 2024-09-20T19:08:42.920 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0079279380000000007, 2024-09-20T19:08:42.920 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0080370909999999997, 2024-09-20T19:08:42.920 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:08:42.920 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:08:42.920 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "waiting for sub ops", 2024-09-20T19:08:42.920 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:08:42.921 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.26264", 2024-09-20T19:08:42.921 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2608780316", 2024-09-20T19:08:42.921 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 151622 2024-09-20T19:08:42.921 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.921 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:08:42.921 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.921 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:08:42.921 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.248220+0000", 2024-09-20T19:08:42.921 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:42.921 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.922 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.922 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:08:42.922 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.248220+0000", 2024-09-20T19:08:42.922 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:42.922 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.922 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.922 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:08:42.922 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.248381+0000", 2024-09-20T19:08:42.922 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016153700000000001 2024-09-20T19:08:42.923 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.923 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.923 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:08:42.923 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.248387+0000", 2024-09-20T19:08:42.923 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7370000000000003e-06 2024-09-20T19:08:42.923 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.923 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.923 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:08:42.923 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.248400+0000", 2024-09-20T19:08:42.924 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.346e-05 2024-09-20T19:08:42.924 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.924 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.924 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:08:42.924 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.248408+0000", 2024-09-20T19:08:42.924 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9119999999999998e-06 2024-09-20T19:08:42.924 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.924 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.924 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:08:42.925 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.250317+0000", 2024-09-20T19:08:42.925 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001908578 2024-09-20T19:08:42.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.925 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.925 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:08:42.925 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.250497+0000", 2024-09-20T19:08:42.925 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017988100000000001 2024-09-20T19:08:42.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.925 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.926 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4,6", 2024-09-20T19:08:42.926 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.250601+0000", 2024-09-20T19:08:42.926 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000103841 2024-09-20T19:08:42.926 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:42.926 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:42.926 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:42.926 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.926 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.926 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.26264.0:151623 11.15 11:ada3c851:::benchmark_data_smithi083_177491_object48181:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e478)", 2024-09-20T19:08:42.927 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:08:41.248423+0000", 2024-09-20T19:08:42.927 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0077249659999999998, 2024-09-20T19:08:42.927 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0079018040000000001, 2024-09-20T19:08:42.927 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:08:42.927 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:08:42.927 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "waiting for sub ops", 2024-09-20T19:08:42.927 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:08:42.927 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.26264", 2024-09-20T19:08:42.927 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2608780316", 2024-09-20T19:08:42.927 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 151623 2024-09-20T19:08:42.928 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.928 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:08:42.928 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.928 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:08:42.928 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.248423+0000", 2024-09-20T19:08:42.928 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:42.928 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.928 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.928 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:08:42.929 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.248423+0000", 2024-09-20T19:08:42.929 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:42.929 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.929 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.929 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:08:42.929 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.248423+0000", 2024-09-20T19:08:42.929 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.44e-07 2024-09-20T19:08:42.929 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.929 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.930 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:08:42.930 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.248426+0000", 2024-09-20T19:08:42.930 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.666e-06 2024-09-20T19:08:42.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.930 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.930 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:08:42.930 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.248431+0000", 2024-09-20T19:08:42.930 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.575e-06 2024-09-20T19:08:42.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.931 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.931 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:08:42.931 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.248436+0000", 2024-09-20T19:08:42.931 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6759999999999998e-06 2024-09-20T19:08:42.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.931 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.931 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:08:42.931 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.250007+0000", 2024-09-20T19:08:42.931 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00157088 2024-09-20T19:08:42.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.932 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.932 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:08:42.932 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.250104+0000", 2024-09-20T19:08:42.932 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7162000000000002e-05 2024-09-20T19:08:42.932 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.932 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.932 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,4", 2024-09-20T19:08:42.932 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.253778+0000", 2024-09-20T19:08:42.932 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003673674 2024-09-20T19:08:42.933 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:42.933 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:42.933 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:42.933 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.933 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.933 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.26264.0:151612 11.d e478/422)", 2024-09-20T19:08:42.933 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:08:41.250016+0000", 2024-09-20T19:08:42.933 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.006131526, 2024-09-20T19:08:42.933 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063697620000000002, 2024-09-20T19:08:42.933 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:08:42.934 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:08:42.934 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "queued for pg", 2024-09-20T19:08:42.934 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:08:42.934 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.934 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:08:42.934 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.250016+0000", 2024-09-20T19:08:42.934 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:42.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.934 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.935 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:08:42.935 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.250016+0000", 2024-09-20T19:08:42.935 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:42.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.935 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:08:42.935 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.250017+0000", 2024-09-20T19:08:42.935 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5000000000000002e-07 2024-09-20T19:08:42.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.936 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.936 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:08:42.936 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.250020+0000", 2024-09-20T19:08:42.936 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3680000000000002e-06 2024-09-20T19:08:42.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.936 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.936 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:08:42.936 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.250028+0000", 2024-09-20T19:08:42.936 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8849999999999999e-06 2024-09-20T19:08:42.937 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.937 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.937 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:08:42.937 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.250034+0000", 2024-09-20T19:08:42.937 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7239999999999997e-06 2024-09-20T19:08:42.937 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:42.937 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:42.937 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:42.937 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.937 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.938 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.26264.0:151613 11.d e478/422)", 2024-09-20T19:08:42.938 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:08:41.250252+0000", 2024-09-20T19:08:42.938 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0058960460000000003, 2024-09-20T19:08:42.938 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0061745869999999996, 2024-09-20T19:08:42.938 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:08:42.938 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:08:42.938 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "queued for pg", 2024-09-20T19:08:42.938 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:08:42.938 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.939 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:08:42.939 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.250252+0000", 2024-09-20T19:08:42.939 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:42.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.939 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.939 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:08:42.939 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.250252+0000", 2024-09-20T19:08:42.939 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:42.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.939 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.940 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:08:42.940 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.250252+0000", 2024-09-20T19:08:42.940 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8400000000000004e-07 2024-09-20T19:08:42.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.940 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.940 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:08:42.940 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.250256+0000", 2024-09-20T19:08:42.940 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7589999999999999e-06 2024-09-20T19:08:42.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.941 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.941 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:08:42.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.250265+0000", 2024-09-20T19:08:42.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9090000000000006e-06 2024-09-20T19:08:42.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.941 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.941 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:08:42.941 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.250271+0000", 2024-09-20T19:08:42.941 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.942e-06 2024-09-20T19:08:42.941 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:42.942 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:42.942 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:42.942 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.942 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.942 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.26264.0:151629 11.2 11:46fef60b:::benchmark_data_smithi083_177491_object48187:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e478)", 2024-09-20T19:08:42.942 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:08:41.250376+0000", 2024-09-20T19:08:42.942 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0057718780000000003, 2024-09-20T19:08:42.942 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0060944329999999998, 2024-09-20T19:08:42.942 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:08:42.943 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:08:42.943 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "waiting for sub ops", 2024-09-20T19:08:42.943 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:08:42.943 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.26264", 2024-09-20T19:08:42.943 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2608780316", 2024-09-20T19:08:42.943 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 151629 2024-09-20T19:08:42.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.943 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:08:42.943 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.944 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:08:42.944 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.250376+0000", 2024-09-20T19:08:42.944 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:42.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.944 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.944 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:08:42.944 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.250376+0000", 2024-09-20T19:08:42.944 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:42.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.945 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:08:42.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.250377+0000", 2024-09-20T19:08:42.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1500000000000003e-07 2024-09-20T19:08:42.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.945 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.945 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:08:42.945 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.250381+0000", 2024-09-20T19:08:42.945 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9469999999999996e-06 2024-09-20T19:08:42.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.946 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:08:42.946 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.250391+0000", 2024-09-20T19:08:42.946 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0417999999999999e-05 2024-09-20T19:08:42.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.946 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:08:42.946 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.250398+0000", 2024-09-20T19:08:42.946 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.883e-06 2024-09-20T19:08:42.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.947 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.947 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:08:42.947 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.250473+0000", 2024-09-20T19:08:42.947 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5498000000000001e-05 2024-09-20T19:08:42.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.947 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.947 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:08:42.947 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.250577+0000", 2024-09-20T19:08:42.947 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000103514 2024-09-20T19:08:42.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.948 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.948 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,3", 2024-09-20T19:08:42.948 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.251032+0000", 2024-09-20T19:08:42.948 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00045511 2024-09-20T19:08:42.948 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:42.948 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:42.948 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:42.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.949 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.26264.0:151617 11.15 e478/455)", 2024-09-20T19:08:42.949 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:08:41.250631+0000", 2024-09-20T19:08:42.949 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.005517129, 2024-09-20T19:08:42.949 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0058994709999999999, 2024-09-20T19:08:42.949 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:08:42.949 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:08:42.949 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "queued for pg", 2024-09-20T19:08:42.949 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:08:42.949 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:08:42.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.250631+0000", 2024-09-20T19:08:42.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:42.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.950 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.950 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:08:42.950 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.250631+0000", 2024-09-20T19:08:42.950 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:42.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.951 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.951 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:08:42.951 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.250631+0000", 2024-09-20T19:08:42.951 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5899999999999996e-07 2024-09-20T19:08:42.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.951 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.951 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:08:42.951 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.250634+0000", 2024-09-20T19:08:42.951 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1140000000000002e-06 2024-09-20T19:08:42.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.952 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.952 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:08:42.952 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.250643+0000", 2024-09-20T19:08:42.952 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0969999999999995e-06 2024-09-20T19:08:42.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.952 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.952 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:08:42.952 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.250649+0000", 2024-09-20T19:08:42.952 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8479999999999997e-06 2024-09-20T19:08:42.953 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:42.953 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:42.953 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:42.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.953 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.953 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.26264.0:151626 11.c e478/422 11:3a87e08f:::benchmark_data_smithi083_177491_object48184:head v 478'9456, mlcod=478'9454)", 2024-09-20T19:08:42.953 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:08:41.250660+0000", 2024-09-20T19:08:42.953 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0054873049999999996, 2024-09-20T19:08:42.953 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059139960000000004, 2024-09-20T19:08:42.953 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:08:42.954 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:08:42.954 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T19:08:42.954 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:08:42.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.954 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:08:42.954 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.250660+0000", 2024-09-20T19:08:42.954 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:42.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:08:42.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.250660+0000", 2024-09-20T19:08:42.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:42.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.955 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:08:42.955 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.250661+0000", 2024-09-20T19:08:42.955 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9000000000000002e-07 2024-09-20T19:08:42.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.956 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:08:42.956 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.250664+0000", 2024-09-20T19:08:42.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6500000000000002e-06 2024-09-20T19:08:42.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.956 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:08:42.956 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.250670+0000", 2024-09-20T19:08:42.956 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.451e-06 2024-09-20T19:08:42.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.956 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.957 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:08:42.957 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.250675+0000", 2024-09-20T19:08:42.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8980000000000004e-06 2024-09-20T19:08:42.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.957 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.957 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:08:42.957 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.253694+0000", 2024-09-20T19:08:42.957 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0030190989999999999 2024-09-20T19:08:42.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.958 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:08:42.958 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.254705+0000", 2024-09-20T19:08:42.958 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0010113990000000001 2024-09-20T19:08:42.958 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:42.958 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:42.958 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:42.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.958 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.959 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.26264.0:151621 11.11 e478/475)", 2024-09-20T19:08:42.959 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:08:41.250720+0000", 2024-09-20T19:08:42.959 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.005427421, 2024-09-20T19:08:42.959 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0059060390000000001, 2024-09-20T19:08:42.959 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:08:42.959 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:08:42.959 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-20T19:08:42.959 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:08:42.959 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.959 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:08:42.960 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.250720+0000", 2024-09-20T19:08:42.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:42.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.960 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:08:42.960 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.250720+0000", 2024-09-20T19:08:42.960 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:42.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.960 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.960 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:08:42.961 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.250720+0000", 2024-09-20T19:08:42.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.86e-07 2024-09-20T19:08:42.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.961 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:08:42.961 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.250722+0000", 2024-09-20T19:08:42.961 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.694e-06 2024-09-20T19:08:42.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.961 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.962 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:08:42.962 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.250726+0000", 2024-09-20T19:08:42.962 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0029999999999996e-06 2024-09-20T19:08:42.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.962 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.962 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:08:42.962 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.250730+0000", 2024-09-20T19:08:42.962 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7469999999999999e-06 2024-09-20T19:08:42.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.963 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:08:42.963 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.253643+0000", 2024-09-20T19:08:42.963 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002913325 2024-09-20T19:08:42.963 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:42.963 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:42.963 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:42.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.963 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.964 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.26264.0:151629 11.2 e478/441)", 2024-09-20T19:08:42.964 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:08:41.253723+0000", 2024-09-20T19:08:42.964 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.002424762, 2024-09-20T19:08:42.964 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0029463530000000001, 2024-09-20T19:08:42.964 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:08:42.964 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:08:42.964 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "queued for pg", 2024-09-20T19:08:42.964 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:08:42.964 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.964 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:08:42.965 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.253723+0000", 2024-09-20T19:08:42.965 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:42.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.965 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.965 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:08:42.965 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.253723+0000", 2024-09-20T19:08:42.965 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:42.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.965 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.965 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:08:42.966 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.253723+0000", 2024-09-20T19:08:42.966 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7999999999999995e-07 2024-09-20T19:08:42.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.966 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.966 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:08:42.966 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.253727+0000", 2024-09-20T19:08:42.966 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.721e-06 2024-09-20T19:08:42.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.966 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.967 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:08:42.967 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.253735+0000", 2024-09-20T19:08:42.967 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7829999999999999e-06 2024-09-20T19:08:42.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.967 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.967 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:08:42.967 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.253758+0000", 2024-09-20T19:08:42.967 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2682000000000001e-05 2024-09-20T19:08:42.967 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:42.968 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:42.968 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:42.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.968 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.968 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.26264.0:151624 11.10 e478/471)", 2024-09-20T19:08:42.968 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:08:41.252583+0000", 2024-09-20T19:08:42.968 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0035648580000000002, 2024-09-20T19:08:42.968 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0041245769999999999, 2024-09-20T19:08:42.968 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:08:42.968 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:08:42.969 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-20T19:08:42.969 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:08:42.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.969 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:08:42.969 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.252583+0000", 2024-09-20T19:08:42.969 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:42.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.969 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:08:42.970 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.252583+0000", 2024-09-20T19:08:42.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:42.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.970 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:08:42.970 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.252584+0000", 2024-09-20T19:08:42.970 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9600000000000008e-07 2024-09-20T19:08:42.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.970 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.971 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:08:42.971 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.252591+0000", 2024-09-20T19:08:42.971 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0489999999999998e-06 2024-09-20T19:08:42.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.971 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.971 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:08:42.971 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.253799+0000", 2024-09-20T19:08:42.971 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0012084929999999999 2024-09-20T19:08:42.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.971 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.972 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:08:42.972 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.253806+0000", 2024-09-20T19:08:42.972 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1999999999999999e-06 2024-09-20T19:08:42.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.972 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.972 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:08:42.972 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.253888+0000", 2024-09-20T19:08:42.972 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2575000000000001e-05 2024-09-20T19:08:42.972 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:42.972 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:42.973 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:42.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.973 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.26264.0:151622 11.d e478/422)", 2024-09-20T19:08:42.973 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:08:41.253526+0000", 2024-09-20T19:08:42.973 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0026220649999999998, 2024-09-20T19:08:42.973 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0032244510000000001, 2024-09-20T19:08:42.973 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:08:42.973 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:08:42.973 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "queued for pg", 2024-09-20T19:08:42.974 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:08:42.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.974 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:08:42.974 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.253526+0000", 2024-09-20T19:08:42.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:42.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.974 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:08:42.974 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.253526+0000", 2024-09-20T19:08:42.974 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:42.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.975 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.975 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:08:42.975 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.253526+0000", 2024-09-20T19:08:42.975 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3900000000000004e-07 2024-09-20T19:08:42.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.975 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.975 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:08:42.975 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.253530+0000", 2024-09-20T19:08:42.975 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1350000000000002e-06 2024-09-20T19:08:42.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.975 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.975 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:08:42.975 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.255203+0000", 2024-09-20T19:08:42.975 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001672443 2024-09-20T19:08:42.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.976 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:08:42.976 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.255220+0000", 2024-09-20T19:08:42.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7180000000000002e-05 2024-09-20T19:08:42.976 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:42.976 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:42.976 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:42.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.976 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.26264.0:151609 11.11 11:897f5411:::benchmark_data_smithi083_177491_object48167:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e478)", 2024-09-20T19:08:42.976 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:08:41.240137+0000", 2024-09-20T19:08:42.976 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.016010984999999998, 2024-09-20T19:08:42.976 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.016657882999999998, 2024-09-20T19:08:42.977 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:08:42.977 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:08:42.977 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "waiting for sub ops", 2024-09-20T19:08:42.977 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:08:42.977 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.26264", 2024-09-20T19:08:42.977 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2608780316", 2024-09-20T19:08:42.977 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 151609 2024-09-20T19:08:42.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.977 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:08:42.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.977 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:08:42.977 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.240137+0000", 2024-09-20T19:08:42.977 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:42.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.977 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.978 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:08:42.978 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.240137+0000", 2024-09-20T19:08:42.978 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:42.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.978 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:08:42.978 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.240138+0000", 2024-09-20T19:08:42.978 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3570000000000001e-06 2024-09-20T19:08:42.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.978 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:08:42.978 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.240142+0000", 2024-09-20T19:08:42.978 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7009999999999998e-06 2024-09-20T19:08:42.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.979 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.979 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:08:42.979 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.240150+0000", 2024-09-20T19:08:42.979 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.8769999999999993e-06 2024-09-20T19:08:42.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.979 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.979 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:08:42.979 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.240156+0000", 2024-09-20T19:08:42.979 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4320000000000004e-06 2024-09-20T19:08:42.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.979 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.979 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:08:42.979 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.240329+0000", 2024-09-20T19:08:42.979 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017317 2024-09-20T19:08:42.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.980 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.980 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:08:42.980 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.240479+0000", 2024-09-20T19:08:42.980 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000150197 2024-09-20T19:08:42.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.980 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.980 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 3,7", 2024-09-20T19:08:42.980 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.240868+0000", 2024-09-20T19:08:42.980 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00038823200000000002 2024-09-20T19:08:42.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.980 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.980 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:08:42.980 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.243856+0000", 2024-09-20T19:08:42.980 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0029888710000000001 2024-09-20T19:08:42.981 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.981 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.981 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:08:42.981 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.246870+0000", 2024-09-20T19:08:42.981 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0030134789999999999 2024-09-20T19:08:42.981 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:42.981 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:42.981 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:42.981 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.981 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.981 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop_reply(client.26264.0:151622 11.d e478/422)", 2024-09-20T19:08:42.981 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:08:41.255250+0000", 2024-09-20T19:08:42.981 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.000897213, 2024-09-20T19:08:42.981 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001621138, 2024-09-20T19:08:42.982 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:08:42.982 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:08:42.982 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "queued for pg", 2024-09-20T19:08:42.982 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:08:42.982 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.982 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:08:42.982 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.255250+0000", 2024-09-20T19:08:42.982 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:42.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.982 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.982 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:08:42.982 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.255250+0000", 2024-09-20T19:08:42.982 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:42.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.983 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.983 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:08:42.983 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.255251+0000", 2024-09-20T19:08:42.983 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0100000000000005e-07 2024-09-20T19:08:42.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.983 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.983 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:08:42.983 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.255255+0000", 2024-09-20T19:08:42.983 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7110000000000001e-06 2024-09-20T19:08:42.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.983 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.983 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:08:42.983 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.255262+0000", 2024-09-20T19:08:42.983 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3390000000000004e-06 2024-09-20T19:08:42.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.984 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.984 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:08:42.984 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.255267+0000", 2024-09-20T19:08:42.984 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6419999999999998e-06 2024-09-20T19:08:42.984 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:42.984 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:42.984 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:42.984 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.984 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.984 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.26264.0:151612 11.d 11:b72c7ca2:::benchmark_data_smithi083_177491_object48170:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e478)", 2024-09-20T19:08:42.984 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:08:41.240218+0000", 2024-09-20T19:08:42.984 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.015929641000000001, 2024-09-20T19:08:42.984 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.016698339999999999, 2024-09-20T19:08:42.984 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:08:42.984 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:08:42.985 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "waiting for sub ops", 2024-09-20T19:08:42.985 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:08:42.985 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.26264", 2024-09-20T19:08:42.985 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2608780316", 2024-09-20T19:08:42.985 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 151612 2024-09-20T19:08:42.985 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.985 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:08:42.985 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.985 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:08:42.985 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.240218+0000", 2024-09-20T19:08:42.985 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:42.985 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.985 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.985 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:08:42.986 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.240218+0000", 2024-09-20T19:08:42.986 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:42.986 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.986 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.986 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:08:42.986 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.240219+0000", 2024-09-20T19:08:42.986 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2600000000000002e-07 2024-09-20T19:08:42.986 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.986 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.986 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:08:42.986 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.240221+0000", 2024-09-20T19:08:42.986 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3140000000000002e-06 2024-09-20T19:08:42.986 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.986 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.987 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:08:42.987 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.240226+0000", 2024-09-20T19:08:42.987 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2050000000000001e-06 2024-09-20T19:08:42.987 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.987 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.987 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:08:42.987 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.240231+0000", 2024-09-20T19:08:42.987 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0159999999999999e-06 2024-09-20T19:08:42.987 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.987 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.987 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:08:42.987 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.243346+0000", 2024-09-20T19:08:42.987 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003114571 2024-09-20T19:08:42.987 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.987 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.988 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:08:42.988 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.243449+0000", 2024-09-20T19:08:42.988 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010299800000000001 2024-09-20T19:08:42.988 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.988 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.988 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4,6", 2024-09-20T19:08:42.988 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.243540+0000", 2024-09-20T19:08:42.988 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.111e-05 2024-09-20T19:08:42.988 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.988 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.988 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:08:42.988 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.249928+0000", 2024-09-20T19:08:42.988 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0063882660000000001 2024-09-20T19:08:42.988 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:42.988 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:42.989 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:42.989 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.989 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.989 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.26264.0:151613 11.d 11:bbd538cd:::benchmark_data_smithi083_177491_object48171:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e478)", 2024-09-20T19:08:42.989 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:08:41.240238+0000", 2024-09-20T19:08:42.989 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.015909710000000001, 2024-09-20T19:08:42.989 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.016751742, 2024-09-20T19:08:42.989 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:08:42.989 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:08:42.989 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "waiting for sub ops", 2024-09-20T19:08:42.989 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:08:42.989 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.26264", 2024-09-20T19:08:42.989 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2608780316", 2024-09-20T19:08:42.989 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 151613 2024-09-20T19:08:42.990 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.990 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:08:42.990 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.990 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:08:42.990 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.240238+0000", 2024-09-20T19:08:42.990 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:42.990 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.990 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.990 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:08:42.990 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.240238+0000", 2024-09-20T19:08:42.990 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:42.990 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.990 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.990 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:08:42.991 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.240238+0000", 2024-09-20T19:08:42.991 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.01e-07 2024-09-20T19:08:42.991 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.991 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.991 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:08:42.991 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.240241+0000", 2024-09-20T19:08:42.991 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2239999999999998e-06 2024-09-20T19:08:42.991 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.991 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.991 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:08:42.991 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.240246+0000", 2024-09-20T19:08:42.991 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7189999999999998e-06 2024-09-20T19:08:42.991 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.991 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.991 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:08:42.992 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.240251+0000", 2024-09-20T19:08:42.992 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9690000000000002e-06 2024-09-20T19:08:42.992 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.992 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.992 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:08:42.992 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.246372+0000", 2024-09-20T19:08:42.992 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0061212230000000003 2024-09-20T19:08:42.992 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.992 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.992 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:08:42.992 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.246469+0000", 2024-09-20T19:08:42.992 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6063999999999994e-05 2024-09-20T19:08:42.992 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.992 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.993 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4,6", 2024-09-20T19:08:42.993 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.246565+0000", 2024-09-20T19:08:42.993 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6296000000000001e-05 2024-09-20T19:08:42.993 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.993 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.993 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:08:42.993 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.249979+0000", 2024-09-20T19:08:42.993 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0034140559999999999 2024-09-20T19:08:42.993 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:42.993 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:42.993 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:42.993 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.993 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.993 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.26264.0:151617 11.15 11:af6e300f:::benchmark_data_smithi083_177491_object48175:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e478)", 2024-09-20T19:08:42.993 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:08:41.240262+0000", 2024-09-20T19:08:42.994 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.015886121999999999, 2024-09-20T19:08:42.994 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.016796972, 2024-09-20T19:08:42.994 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:08:42.994 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:08:42.994 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "waiting for sub ops", 2024-09-20T19:08:42.994 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:08:42.994 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.26264", 2024-09-20T19:08:42.994 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2608780316", 2024-09-20T19:08:42.994 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 151617 2024-09-20T19:08:42.994 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.994 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:08:42.994 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.994 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:08:42.994 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.240262+0000", 2024-09-20T19:08:42.994 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:42.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.995 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.995 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:08:42.995 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.240262+0000", 2024-09-20T19:08:42.995 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:42.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.995 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.995 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:08:42.995 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.240262+0000", 2024-09-20T19:08:42.995 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1600000000000002e-07 2024-09-20T19:08:42.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.995 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.995 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:08:42.995 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.240265+0000", 2024-09-20T19:08:42.996 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7489999999999999e-06 2024-09-20T19:08:42.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.996 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.996 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:08:42.996 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.240271+0000", 2024-09-20T19:08:42.996 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8470000000000001e-06 2024-09-20T19:08:42.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.996 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.996 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:08:42.996 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.240276+0000", 2024-09-20T19:08:42.996 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0180000000000001e-06 2024-09-20T19:08:42.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.996 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.996 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:08:42.997 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.246218+0000", 2024-09-20T19:08:42.997 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.005942208 2024-09-20T19:08:42.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.997 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.997 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:08:42.997 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.246324+0000", 2024-09-20T19:08:42.997 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000105901 2024-09-20T19:08:42.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.997 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.997 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,4", 2024-09-20T19:08:42.997 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.246682+0000", 2024-09-20T19:08:42.997 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00035829500000000001 2024-09-20T19:08:42.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:42.997 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:42.997 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:08:42.997 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:41.249889+0000", 2024-09-20T19:08:42.998 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0032067189999999998 2024-09-20T19:08:42.998 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:42.998 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:42.998 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:42.998 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:42.998 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-20T19:08:42.998 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 17 2024-09-20T19:08:42.998 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T19:08:42.998 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:43.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:42 smithi142 ceph-mon[32508]: 5.1f scrub starts 2024-09-20T19:08:43.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:42 smithi142 ceph-mon[32508]: 5.1f scrub ok 2024-09-20T19:08:43.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:42 smithi142 ceph-mon[32508]: pgmap v960: 168 pgs: 2 active+clean+scrubbing, 166 active+clean; 3.6 GiB data, 17 GiB used, 698 GiB / 715 GiB avail 2024-09-20T19:08:43.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:42 smithi142 ceph-mon[32508]: 6.17 scrub starts 2024-09-20T19:08:43.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:42 smithi142 ceph-mon[32508]: 6.17 scrub ok 2024-09-20T19:08:43.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:42 smithi142 ceph-mon[32508]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:08:43.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:42 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T19:08:43.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:42 smithi142 ceph-mon[32508]: osdmap e479: 8 total, 8 up, 8 in 2024-09-20T19:08:43.031 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:43.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:43.032 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:43.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:42 smithi083 ceph-mon[29421]: 5.1f scrub starts 2024-09-20T19:08:43.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:42 smithi083 ceph-mon[29421]: 5.1f scrub ok 2024-09-20T19:08:43.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:42 smithi083 ceph-mon[29421]: pgmap v960: 168 pgs: 2 active+clean+scrubbing, 166 active+clean; 3.6 GiB data, 17 GiB used, 698 GiB / 715 GiB avail 2024-09-20T19:08:43.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:42 smithi083 ceph-mon[29421]: 6.17 scrub starts 2024-09-20T19:08:43.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:42 smithi083 ceph-mon[29421]: 6.17 scrub ok 2024-09-20T19:08:43.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:42 smithi083 ceph-mon[29421]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:08:43.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:42 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T19:08:43.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:42 smithi083 ceph-mon[29421]: osdmap e479: 8 total, 8 up, 8 in 2024-09-20T19:08:43.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:42 smithi083 ceph-mon[36260]: 5.1f scrub starts 2024-09-20T19:08:43.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:42 smithi083 ceph-mon[36260]: 5.1f scrub ok 2024-09-20T19:08:43.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:42 smithi083 ceph-mon[36260]: pgmap v960: 168 pgs: 2 active+clean+scrubbing, 166 active+clean; 3.6 GiB data, 17 GiB used, 698 GiB / 715 GiB avail 2024-09-20T19:08:43.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:42 smithi083 ceph-mon[36260]: 6.17 scrub starts 2024-09-20T19:08:43.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:42 smithi083 ceph-mon[36260]: 6.17 scrub ok 2024-09-20T19:08:43.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:42 smithi083 ceph-mon[36260]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:08:43.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:42 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T19:08:43.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:42 smithi083 ceph-mon[36260]: osdmap e479: 8 total, 8 up, 8 in 2024-09-20T19:08:43.133 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:43.174 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:43.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:43.176 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:43.276 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:43.309 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:43.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:43.310 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:43.411 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:43.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:43.445 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:43.445 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:43.546 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:08:43.580 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:43.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:43.581 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:08:43.682 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:43.684 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_blocked_ops 2024-09-20T19:08:43.709 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:43.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:43.710 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:43.811 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:43.829 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:08:43.875 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:43.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:43.876 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:43.977 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:08:44.016 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:44.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:44.017 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:08:44.118 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:44.148 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:44.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:44.149 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:44.249 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:44.282 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:44.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:44.283 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:44.383 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:44.409 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:44.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:44.410 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:44.447 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:08:44.447 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:08:44.466 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:08:44.485 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:08:44.486 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:08:44.510 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:44.523 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:08:44.524 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:08:44.538 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:44.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:44.539 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:44.561 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:08:44.561 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:08:44.592 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:08:44.592 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:08:44.625 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:08:44.625 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:08:44.639 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:44.656 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:08:44.656 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:08:44.666 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:44.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:44.667 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:44.691 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:08:44.691 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:08:44.768 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:08:44.803 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:44.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:44.804 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:08:44.811 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:44 smithi083 ceph-mon[29421]: pgmap v962: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 3.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail 2024-09-20T19:08:44.811 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:44 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3768785149' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:08:44.812 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:44 smithi083 ceph-mon[36260]: pgmap v962: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 3.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail 2024-09-20T19:08:44.812 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:44 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3768785149' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:08:44.861 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T19:08:44.905 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:44.933 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:44.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:44.933 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:45.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:44 smithi142 ceph-mon[32508]: pgmap v962: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 3.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail 2024-09-20T19:08:45.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:44 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3768785149' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:08:45.035 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:45.081 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:45.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:45.082 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:45.183 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:45.213 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:45.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:45.214 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:45.315 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:45.347 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:45.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:45.348 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:45.449 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:08:45.492 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:45.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:45.493 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:08:45.594 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:45.626 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:45.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:45.627 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:45.728 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:45.738 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T19:08:45.738 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:08:45.753 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:45.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:45.754 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:45.855 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:45.890 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:45.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:45.891 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:45.991 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:46.026 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:46.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:46.027 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:46.127 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:46.169 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:46.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:46.170 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:46.271 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:46.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:46.304 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:46.304 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:46.405 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:46.431 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:46.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:46.432 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:46.533 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:46.568 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:46.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:46.569 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:46.669 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:08:46.691 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:46.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:46.692 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:08:46.793 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:46.826 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:46.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:46.827 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:46.928 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:46.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:46.965 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:46.965 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:47.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:46 smithi083 ceph-mon[29421]: pgmap v963: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 3.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 0 B/s wr, 313 op/s 2024-09-20T19:08:47.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:46 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:08:47.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:46 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:08:47.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:46 smithi083 ceph-mon[36260]: pgmap v963: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 3.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 0 B/s wr, 313 op/s 2024-09-20T19:08:47.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:46 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:08:47.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:46 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:08:47.066 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:47.077 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 7, 6, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 5, 6, 7, 4] 2024-09-20T19:08:47.077 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T19:08:47.077 INFO:tasks.thrashosds.thrasher:Shrinking pool unique_pool_4 2024-09-20T19:08:47.077 INFO:tasks.cephadm.ceph_manager.ceph:contract_pool unique_pool_4 by 10 min 8 2024-09-20T19:08:47.078 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T19:08:47.101 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:47.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:47.102 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:47.203 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:47.250 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:47.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:47.251 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:47.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:46 smithi142 ceph-mon[32508]: pgmap v963: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 3.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 0 B/s wr, 313 op/s 2024-09-20T19:08:47.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:46 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:08:47.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:46 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:08:47.351 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:47.387 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:47.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:47.388 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:47.468 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T19:08:47.489 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:47.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:47.563 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:47.563 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:47.664 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:47.695 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:47.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:47.696 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:47.797 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:47.827 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:47.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:47.828 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:47.928 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:08:47.971 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:47.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:47.972 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:08:48.010 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:08:47 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:08:48.073 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:48.103 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:48.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:48.104 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:48.204 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:48.225 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:48.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:48.226 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:48.327 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:08:48.364 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:48.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:48.365 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:08:48.466 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:48.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:48.498 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:48.499 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:48.599 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:48.634 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:48.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:48.635 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:48.735 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:48.770 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:48.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:48.771 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:48.872 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:48.907 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:48.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:48.908 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:49.009 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:49.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:49.036 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:49.036 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:49.039 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:48 smithi083 ceph-mon[29421]: pgmap v964: 168 pgs: 168 active+clean; 3.0 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 0 B/s wr, 342 op/s 2024-09-20T19:08:49.039 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:48 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:08:49.039 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:48 smithi083 ceph-mon[36260]: pgmap v964: 168 pgs: 168 active+clean; 3.0 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 0 B/s wr, 342 op/s 2024-09-20T19:08:49.039 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:48 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:08:49.040 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_historic_ops 2024-09-20T19:08:49.137 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:49.141 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:08:49.165 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:49.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:49.166 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:49.167 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:08:49.206 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:08:49.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:48 smithi142 ceph-mon[32508]: pgmap v964: 168 pgs: 168 active+clean; 3.0 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 0 B/s wr, 342 op/s 2024-09-20T19:08:49.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:48 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:08:49.268 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:49.362 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:49.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:49.363 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:49.464 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:49.492 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:49.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:49.493 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:49.594 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:49.623 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:49.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:49.624 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:49.724 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:49.773 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:49.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:49.775 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:49.875 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:49.906 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:49.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:49.907 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:50.008 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:50.035 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:50.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:50.036 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:50.137 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:50.163 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:50.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:50.164 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:50.265 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:50.292 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:50.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:50.293 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:50.394 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:08:50.447 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:50.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:50.448 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:08:50.522 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T19:08:50.525 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-20T19:08:50.549 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:50.573 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:50.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:50.574 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:50.675 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:50.704 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:50.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:50.705 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:50.781 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-20T19:08:50.806 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:50.836 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:50.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:50.837 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:50.938 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:50.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:50.973 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:50.973 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:51.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:50 smithi083 ceph-mon[29421]: pgmap v965: 168 pgs: 168 active+clean; 2.9 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 0 B/s wr, 450 op/s 2024-09-20T19:08:51.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:50 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2371335159' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:08:51.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:50 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:08:51.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:50 smithi083 ceph-mon[36260]: pgmap v965: 168 pgs: 168 active+clean; 2.9 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 0 B/s wr, 450 op/s 2024-09-20T19:08:51.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:50 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2371335159' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:08:51.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:50 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:08:51.041 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:08:50 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:08:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:08:51.074 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:51.113 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:51.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:51.114 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:51.214 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:51.242 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:51.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:51.243 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:51.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:50 smithi142 ceph-mon[32508]: pgmap v965: 168 pgs: 168 active+clean; 2.9 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 0 B/s wr, 450 op/s 2024-09-20T19:08:51.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:50 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2371335159' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:08:51.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:50 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:08:51.344 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:08:51.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:51.377 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:51.377 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:08:51.478 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:51.507 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:51.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:51.508 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:51.609 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:08:51.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:51.631 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:51.631 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:08:51.732 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:51.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:51.770 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:51.771 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:51.872 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:08:51.904 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:51.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:51.905 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:08:52.006 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:52.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:51 smithi083 ceph-mon[29421]: from='client.26095 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:08:52.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:51 smithi083 ceph-mon[29421]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:08:52.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:51 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T19:08:52.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:51 smithi083 ceph-mon[29421]: osdmap e480: 8 total, 8 up, 8 in 2024-09-20T19:08:52.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:51 smithi083 ceph-mon[36260]: from='client.26095 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:08:52.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:51 smithi083 ceph-mon[36260]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:08:52.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:51 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T19:08:52.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:51 smithi083 ceph-mon[36260]: osdmap e480: 8 total, 8 up, 8 in 2024-09-20T19:08:52.045 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:52.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:52.047 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:52.147 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:52.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:52.175 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:52.176 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:52.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:51 smithi142 ceph-mon[32508]: from='client.26095 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:08:52.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:51 smithi142 ceph-mon[32508]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:08:52.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:51 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T19:08:52.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:51 smithi142 ceph-mon[32508]: osdmap e480: 8 total, 8 up, 8 in 2024-09-20T19:08:52.278 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:52.304 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:52.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:52.305 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:52.406 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:52.446 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:52.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:52.447 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:52.548 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:52.574 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:52.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:52.575 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:52.676 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:52.711 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:52.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:52.713 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:52.813 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:52.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:52.847 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:52.847 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:52.948 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:52.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:52.985 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:52.985 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:53.086 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:53.114 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:53.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:53.115 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:53.216 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:53.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:53.248 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:53.248 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:53.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:52 smithi142 ceph-mon[32508]: pgmap v967: 168 pgs: 168 active+clean; 2.8 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 553 B/s rd, 0 B/s wr, 641 op/s 2024-09-20T19:08:53.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:52 smithi083 ceph-mon[29421]: pgmap v967: 168 pgs: 168 active+clean; 2.8 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 553 B/s rd, 0 B/s wr, 641 op/s 2024-09-20T19:08:53.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:52 smithi083 ceph-mon[36260]: pgmap v967: 168 pgs: 168 active+clean; 2.8 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 553 B/s rd, 0 B/s wr, 641 op/s 2024-09-20T19:08:53.350 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:53.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:53.374 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:53.374 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:53.475 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:53.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:53.498 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:53.498 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:53.599 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:53.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:53.620 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:53.621 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:53.721 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:53.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:53.754 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:53.754 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:53.855 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:53.876 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:53.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:53.877 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:53.978 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:54.002 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:54.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:54.003 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:54.104 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:54.125 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:54.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:54.126 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:54.227 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:54.257 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:54.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:54.258 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:54.358 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:54.392 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:54.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:54.393 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:54.494 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:54.535 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:54.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:54.536 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:54.637 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:54.667 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:54.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:54.668 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:54.769 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:54.809 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:54.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:54.810 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:54.911 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:08:54.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:54.951 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:54.951 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:08:55.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:54 smithi083 ceph-mon[29421]: pgmap v968: 168 pgs: 168 active+clean; 2.5 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 819 op/s 2024-09-20T19:08:55.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:54 smithi083 ceph-mon[36260]: pgmap v968: 168 pgs: 168 active+clean; 2.5 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 819 op/s 2024-09-20T19:08:55.052 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:08:55.061 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:54 smithi142 ceph-mon[32508]: pgmap v968: 168 pgs: 168 active+clean; 2.5 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 819 op/s 2024-09-20T19:08:55.090 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:55.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:55.091 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:08:55.192 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:55.235 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:55.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:55.236 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:55.337 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:08:55.374 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:55.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:55.375 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:08:55.476 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:55.515 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:55.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:55.516 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:55.617 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:55.628 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":965,"stamp":"2024-09-20T19:08:49.219413+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3131652959,"num_objects":47978,"num_object_clones":0,"num_object_copies":143934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47978,"num_whiteouts":0,"num_read":2286,"num_read_kb":2058,"num_write":101953,"num_write_kb":3680694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6257,"num_bytes_recovered":391604274,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155802,"ondisk_log_size":155802,"up":504,"acting":504,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":511,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":15681224,"kb_used_data":9933448,"kb_used_omap":16,"kb_used_meta":5747567,"kb_avail":734279992,"statfs":{"total":767960285184,"available":751902711808,"internally_reserved":0,"allocated":10171850752,"data_stored":10168629057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16790,"internal_metadata":5885509226},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-864330571,"num_objects":-13188,"num_object_clones":0,"num_object_copies":-39564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-13188,"num_whiteouts":0,"num_read":-13,"num_read_kb":-32,"num_write":3825,"num_write_kb":-417960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-3882,"num_bytes_recovered":-254667235,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.488676"},"pg_stats":[{"pgid":"11.17","version":"479'6904","reported_seq":12966,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:46.439203+0000","last_change":"2024-09-20T19:07:43.964356+0000","last_active":"2024-09-20T19:08:46.439203+0000","last_peered":"2024-09-20T19:08:46.439203+0000","last_clean":"2024-09-20T19:08:46.439203+0000","last_became_active":"2024-09-20T19:07:43.959947+0000","last_became_peered":"2024-09-20T19:07:43.959947+0000","last_unstale":"2024-09-20T19:08:46.439203+0000","last_undegraded":"2024-09-20T19:08:46.439203+0000","last_fullsized":"2024-09-20T19:08:46.439203+0000","mapping_epoch":459,"log_start":"432'1560","ondisk_log_start":"432'1560","created":441,"last_epoch_clean":460,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":5344,"log_dups_size":0,"ondisk_log_size":5344,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-20T19:09:43.009165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":97353728,"num_objects":1486,"num_object_clones":0,"num_object_copies":4458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1486,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5000,"num_write_kb":207520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1972,"num_bytes_recovered":129236992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,5],"acting":[7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":899,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.964421+0000","last_change":"2024-09-20T19:08:41.521624+0000","last_active":"2024-09-20T19:08:41.964421+0000","last_peered":"2024-09-20T19:08:41.964421+0000","last_clean":"2024-09-20T19:08:41.964421+0000","last_became_active":"2024-09-20T18:49:45.943201+0000","last_became_peered":"2024-09-20T18:49:45.943201+0000","last_unstale":"2024-09-20T19:08:41.964421+0000","last_undegraded":"2024-09-20T19:08:41.964421+0000","last_fullsized":"2024-09-20T19:08:41.964421+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:08:41.521562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:08:41.521562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:09:57.765947+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"273'15","reported_seq":238,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.974932+0000","last_change":"2024-09-20T19:06:46.500942+0000","last_active":"2024-09-20T19:08:41.974932+0000","last_peered":"2024-09-20T19:08:41.974932+0000","last_clean":"2024-09-20T19:08:41.974932+0000","last_became_active":"2024-09-20T19:04:20.925853+0000","last_became_peered":"2024-09-20T19:04:20.925853+0000","last_unstale":"2024-09-20T19:08:41.974932+0000","last_undegraded":"2024-09-20T19:08:41.974932+0000","last_fullsized":"2024-09-20T19:08:41.974932+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:06:46.500881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:06:46.500881+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":102,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.977431+0000","last_change":"2024-09-20T19:06:40.320839+0000","last_active":"2024-09-20T19:08:41.977431+0000","last_peered":"2024-09-20T19:08:41.977431+0000","last_clean":"2024-09-20T19:08:41.977431+0000","last_became_active":"2024-09-20T19:06:40.320505+0000","last_became_peered":"2024-09-20T19:06:40.320505+0000","last_unstale":"2024-09-20T19:08:41.977431+0000","last_undegraded":"2024-09-20T19:08:41.977431+0000","last_fullsized":"2024-09-20T19:08:41.977431+0000","mapping_epoch":434,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:49.016006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:57:49.016006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":228,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.977481+0000","last_change":"2024-09-20T19:08:37.950394+0000","last_active":"2024-09-20T19:08:41.977481+0000","last_peered":"2024-09-20T19:08:41.977481+0000","last_clean":"2024-09-20T19:08:41.977481+0000","last_became_active":"2024-09-20T19:04:20.880905+0000","last_became_peered":"2024-09-20T19:04:20.880905+0000","last_unstale":"2024-09-20T19:08:41.977481+0000","last_undegraded":"2024-09-20T19:08:41.977481+0000","last_fullsized":"2024-09-20T19:08:41.977481+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:08:37.950341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:08:37.950341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:09:56.495637+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"479'6923","reported_seq":14986,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:45.801404+0000","last_change":"2024-09-20T19:08:41.119160+0000","last_active":"2024-09-20T19:08:45.801404+0000","last_peered":"2024-09-20T19:08:45.801404+0000","last_clean":"2024-09-20T19:08:45.801404+0000","last_became_active":"2024-09-20T19:07:50.022839+0000","last_became_peered":"2024-09-20T19:07:50.022839+0000","last_unstale":"2024-09-20T19:08:45.801404+0000","last_undegraded":"2024-09-20T19:08:45.801404+0000","last_fullsized":"2024-09-20T19:08:45.801404+0000","mapping_epoch":462,"log_start":"0'0","ondisk_log_start":"0'0","created":441,"last_epoch_clean":463,"parent":"0.0","parent_split_bits":5,"last_scrub":"478'6749","last_scrub_stamp":"2024-09-20T19:08:41.119102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:08:41.119102+0000","objects_scrubbed":1738,"log_size":6923,"log_dups_size":0,"ondisk_log_size":6923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:09:46.725231+0000","scrub_duration":391,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":102170624,"num_objects":1559,"num_object_clones":0,"num_object_copies":4677,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1559,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":179,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":890,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.980490+0000","last_change":"2024-09-20T19:02:02.441633+0000","last_active":"2024-09-20T19:08:41.980490+0000","last_peered":"2024-09-20T19:08:41.980490+0000","last_clean":"2024-09-20T19:08:41.980490+0000","last_became_active":"2024-09-20T18:59:05.081081+0000","last_became_peered":"2024-09-20T18:59:05.081081+0000","last_unstale":"2024-09-20T19:08:41.980490+0000","last_undegraded":"2024-09-20T19:08:41.980490+0000","last_fullsized":"2024-09-20T19:08:41.980490+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:02.441577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:02.441577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":904,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.981326+0000","last_change":"2024-09-20T19:04:37.540824+0000","last_active":"2024-09-20T19:08:41.981326+0000","last_peered":"2024-09-20T19:08:41.981326+0000","last_clean":"2024-09-20T19:08:41.981326+0000","last_became_active":"2024-09-20T18:59:05.072675+0000","last_became_peered":"2024-09-20T18:59:05.072675+0000","last_unstale":"2024-09-20T19:08:41.981326+0000","last_undegraded":"2024-09-20T19:08:41.981326+0000","last_fullsized":"2024-09-20T19:08:41.981326+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:37.540760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:04:37.540760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"274'27","reported_seq":935,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.981582+0000","last_change":"2024-09-20T19:05:50.750134+0000","last_active":"2024-09-20T19:08:41.981582+0000","last_peered":"2024-09-20T19:08:41.981582+0000","last_clean":"2024-09-20T19:08:41.981582+0000","last_became_active":"2024-09-20T18:49:47.952150+0000","last_became_peered":"2024-09-20T18:49:47.952150+0000","last_unstale":"2024-09-20T19:08:41.981582+0000","last_undegraded":"2024-09-20T19:08:41.981582+0000","last_fullsized":"2024-09-20T19:08:41.981582+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'27","last_scrub_stamp":"2024-09-20T19:05:50.750050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:50.750050+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":228,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.975027+0000","last_change":"2024-09-20T19:06:25.218221+0000","last_active":"2024-09-20T19:08:41.975027+0000","last_peered":"2024-09-20T19:08:41.975027+0000","last_clean":"2024-09-20T19:08:41.975027+0000","last_became_active":"2024-09-20T19:04:20.925394+0000","last_became_peered":"2024-09-20T19:04:20.925394+0000","last_unstale":"2024-09-20T19:08:41.975027+0000","last_undegraded":"2024-09-20T19:08:41.975027+0000","last_fullsized":"2024-09-20T19:08:41.975027+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:25.218150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:06:25.218150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"479'6827","reported_seq":14522,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:46.817487+0000","last_change":"2024-09-20T19:08:34.394041+0000","last_active":"2024-09-20T19:08:46.817487+0000","last_peered":"2024-09-20T19:08:46.817487+0000","last_clean":"2024-09-20T19:08:46.817487+0000","last_became_active":"2024-09-20T19:07:40.719995+0000","last_became_peered":"2024-09-20T19:07:40.719995+0000","last_unstale":"2024-09-20T19:08:46.817487+0000","last_undegraded":"2024-09-20T19:08:46.817487+0000","last_fullsized":"2024-09-20T19:08:46.817487+0000","mapping_epoch":455,"log_start":"0'0","ondisk_log_start":"0'0","created":441,"last_epoch_clean":456,"parent":"0.0","parent_split_bits":5,"last_scrub":"475'6312","last_scrub_stamp":"2024-09-20T19:08:34.393876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:08:34.393876+0000","objects_scrubbed":1992,"log_size":6827,"log_dups_size":0,"ondisk_log_size":6827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:09:39.912909+0000","scrub_duration":301,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96665600,"num_objects":1475,"num_object_clones":0,"num_object_copies":4425,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":517,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"59'1","reported_seq":965,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.979719+0000","last_change":"2024-09-20T19:04:20.963496+0000","last_active":"2024-09-20T19:08:41.979719+0000","last_peered":"2024-09-20T19:08:41.979719+0000","last_clean":"2024-09-20T19:08:41.979719+0000","last_became_active":"2024-09-20T19:04:20.914145+0000","last_became_peered":"2024-09-20T19:04:20.914145+0000","last_unstale":"2024-09-20T19:08:41.979719+0000","last_undegraded":"2024-09-20T19:08:41.979719+0000","last_fullsized":"2024-09-20T19:08:41.979719+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:01:28.592653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:28.592653+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"273'15","reported_seq":230,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.977585+0000","last_change":"2024-09-20T19:04:20.984223+0000","last_active":"2024-09-20T19:08:41.977585+0000","last_peered":"2024-09-20T19:08:41.977585+0000","last_clean":"2024-09-20T19:08:41.977585+0000","last_became_active":"2024-09-20T19:04:20.898235+0000","last_became_peered":"2024-09-20T19:04:20.898235+0000","last_unstale":"2024-09-20T19:08:41.977585+0000","last_undegraded":"2024-09-20T19:08:41.977585+0000","last_fullsized":"2024-09-20T19:08:41.977585+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:02:42.114331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:42.114331+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":896,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.978324+0000","last_change":"2024-09-20T19:06:47.942245+0000","last_active":"2024-09-20T19:08:41.978324+0000","last_peered":"2024-09-20T19:08:41.978324+0000","last_clean":"2024-09-20T19:08:41.978324+0000","last_became_active":"2024-09-20T18:49:49.961188+0000","last_became_peered":"2024-09-20T18:49:49.961188+0000","last_unstale":"2024-09-20T19:08:41.978324+0000","last_undegraded":"2024-09-20T19:08:41.978324+0000","last_fullsized":"2024-09-20T19:08:41.978324+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:47.942182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:06:47.942182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":877,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.964583+0000","last_change":"2024-09-20T19:06:53.956421+0000","last_active":"2024-09-20T19:08:41.964583+0000","last_peered":"2024-09-20T19:08:41.964583+0000","last_clean":"2024-09-20T19:08:41.964583+0000","last_became_active":"2024-09-20T18:49:51.966715+0000","last_became_peered":"2024-09-20T18:49:51.966715+0000","last_unstale":"2024-09-20T19:08:41.964583+0000","last_undegraded":"2024-09-20T19:08:41.964583+0000","last_fullsized":"2024-09-20T19:08:41.964583+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:53.956349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:06:53.956349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"479'6936","reported_seq":13003,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:46.924970+0000","last_change":"2024-09-20T19:08:26.337018+0000","last_active":"2024-09-20T19:08:46.924970+0000","last_peered":"2024-09-20T19:08:46.924970+0000","last_clean":"2024-09-20T19:08:46.924970+0000","last_became_active":"2024-09-20T19:07:36.015594+0000","last_became_peered":"2024-09-20T19:07:36.015594+0000","last_unstale":"2024-09-20T19:08:46.924970+0000","last_undegraded":"2024-09-20T19:08:46.924970+0000","last_fullsized":"2024-09-20T19:08:46.924970+0000","mapping_epoch":452,"log_start":"426'430","ondisk_log_start":"426'430","created":441,"last_epoch_clean":453,"parent":"0.0","parent_split_bits":5,"last_scrub":"472'6170","last_scrub_stamp":"2024-09-20T19:08:26.336764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:08:26.336764+0000","objects_scrubbed":2280,"log_size":6506,"log_dups_size":0,"ondisk_log_size":6506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:09:39.950097+0000","scrub_duration":519,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":98500608,"num_objects":1503,"num_object_clones":0,"num_object_copies":4509,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1503,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":777,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":913,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.978482+0000","last_change":"2024-09-20T19:04:20.879227+0000","last_active":"2024-09-20T19:08:41.978482+0000","last_peered":"2024-09-20T19:08:41.978482+0000","last_clean":"2024-09-20T19:08:41.978482+0000","last_became_active":"2024-09-20T19:04:20.878905+0000","last_became_peered":"2024-09-20T19:04:20.878905+0000","last_unstale":"2024-09-20T19:08:41.978482+0000","last_undegraded":"2024-09-20T19:08:41.978482+0000","last_fullsized":"2024-09-20T19:08:41.978482+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:48.520017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:48.520017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"273'7","reported_seq":234,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.977727+0000","last_change":"2024-09-20T19:05:53.980848+0000","last_active":"2024-09-20T19:08:41.977727+0000","last_peered":"2024-09-20T19:08:41.977727+0000","last_clean":"2024-09-20T19:08:41.977727+0000","last_became_active":"2024-09-20T19:04:20.888572+0000","last_became_peered":"2024-09-20T19:04:20.888572+0000","last_unstale":"2024-09-20T19:08:41.977727+0000","last_undegraded":"2024-09-20T19:08:41.977727+0000","last_fullsized":"2024-09-20T19:08:41.977727+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'7","last_scrub_stamp":"2024-09-20T19:05:53.980772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:53.980772+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":906,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.973307+0000","last_change":"2024-09-20T19:04:20.885873+0000","last_active":"2024-09-20T19:08:41.973307+0000","last_peered":"2024-09-20T19:08:41.973307+0000","last_clean":"2024-09-20T19:08:41.973307+0000","last_became_active":"2024-09-20T19:04:20.885417+0000","last_became_peered":"2024-09-20T19:04:20.885417+0000","last_unstale":"2024-09-20T19:08:41.973307+0000","last_undegraded":"2024-09-20T19:08:41.973307+0000","last_fullsized":"2024-09-20T19:08:41.973307+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:33.203312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:33.203312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":884,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.977521+0000","last_change":"2024-09-20T19:05:21.625378+0000","last_active":"2024-09-20T19:08:41.977521+0000","last_peered":"2024-09-20T19:08:41.977521+0000","last_clean":"2024-09-20T19:08:41.977521+0000","last_became_active":"2024-09-20T18:49:51.970663+0000","last_became_peered":"2024-09-20T18:49:51.970663+0000","last_unstale":"2024-09-20T19:08:41.977521+0000","last_undegraded":"2024-09-20T19:08:41.977521+0000","last_fullsized":"2024-09-20T19:08:41.977521+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:21.625311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:05:21.625311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"479'6999","reported_seq":15335,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:45.760571+0000","last_change":"2024-09-20T19:08:33.730663+0000","last_active":"2024-09-20T19:08:45.760571+0000","last_peered":"2024-09-20T19:08:45.760571+0000","last_clean":"2024-09-20T19:08:45.760571+0000","last_became_active":"2024-09-20T19:08:33.706319+0000","last_became_peered":"2024-09-20T19:08:33.706319+0000","last_unstale":"2024-09-20T19:08:45.760571+0000","last_undegraded":"2024-09-20T19:08:45.760571+0000","last_fullsized":"2024-09-20T19:08:45.760571+0000","mapping_epoch":474,"log_start":"426'456","ondisk_log_start":"426'456","created":441,"last_epoch_clean":475,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":6543,"log_dups_size":0,"ondisk_log_size":6543,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":95617024,"num_objects":1459,"num_object_clones":0,"num_object_copies":4377,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1459,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5077,"num_write_kb":209152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2047,"num_bytes_recovered":134086656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":229,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.975881+0000","last_change":"2024-09-20T19:06:45.481243+0000","last_active":"2024-09-20T19:08:41.975881+0000","last_peered":"2024-09-20T19:08:41.975881+0000","last_clean":"2024-09-20T19:08:41.975881+0000","last_became_active":"2024-09-20T19:04:20.906109+0000","last_became_peered":"2024-09-20T19:04:20.906109+0000","last_unstale":"2024-09-20T19:08:41.975881+0000","last_undegraded":"2024-09-20T19:08:41.975881+0000","last_fullsized":"2024-09-20T19:08:41.975881+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:45.479545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:06:45.479545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":914,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.968081+0000","last_change":"2024-09-20T19:04:20.907057+0000","last_active":"2024-09-20T19:08:41.968081+0000","last_peered":"2024-09-20T19:08:41.968081+0000","last_clean":"2024-09-20T19:08:41.968081+0000","last_became_active":"2024-09-20T19:04:20.906691+0000","last_became_peered":"2024-09-20T19:04:20.906691+0000","last_unstale":"2024-09-20T19:08:41.968081+0000","last_undegraded":"2024-09-20T19:08:41.968081+0000","last_fullsized":"2024-09-20T19:08:41.968081+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:44.998147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:44.998147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"273'27","reported_seq":946,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.972648+0000","last_change":"2024-09-20T19:05:49.452928+0000","last_active":"2024-09-20T19:08:41.972648+0000","last_peered":"2024-09-20T19:08:41.972648+0000","last_clean":"2024-09-20T19:08:41.972648+0000","last_became_active":"2024-09-20T18:49:47.950939+0000","last_became_peered":"2024-09-20T18:49:47.950939+0000","last_unstale":"2024-09-20T19:08:41.972648+0000","last_undegraded":"2024-09-20T19:08:41.972648+0000","last_fullsized":"2024-09-20T19:08:41.972648+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'27","last_scrub_stamp":"2024-09-20T19:05:49.452861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:49.452861+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":905,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.975246+0000","last_change":"2024-09-20T19:08:34.106350+0000","last_active":"2024-09-20T19:08:41.975246+0000","last_peered":"2024-09-20T19:08:41.975246+0000","last_clean":"2024-09-20T19:08:41.975246+0000","last_became_active":"2024-09-20T19:04:20.890370+0000","last_became_peered":"2024-09-20T19:04:20.890370+0000","last_unstale":"2024-09-20T19:08:41.975246+0000","last_undegraded":"2024-09-20T19:08:41.975246+0000","last_fullsized":"2024-09-20T19:08:41.975246+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:08:34.106277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:08:34.106277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:09:41.685448+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"479'6790","reported_seq":10783,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:46.438529+0000","last_change":"2024-09-20T19:08:36.464247+0000","last_active":"2024-09-20T19:08:46.438529+0000","last_peered":"2024-09-20T19:08:46.438529+0000","last_clean":"2024-09-20T19:08:46.438529+0000","last_became_active":"2024-09-20T19:07:25.936833+0000","last_became_peered":"2024-09-20T19:07:25.936833+0000","last_unstale":"2024-09-20T19:08:46.438529+0000","last_undegraded":"2024-09-20T19:08:46.438529+0000","last_fullsized":"2024-09-20T19:08:46.438529+0000","mapping_epoch":448,"log_start":"431'1489","ondisk_log_start":"431'1489","created":441,"last_epoch_clean":449,"parent":"0.0","parent_split_bits":5,"last_scrub":"476'6378","last_scrub_stamp":"2024-09-20T19:08:36.464045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:08:36.464045+0000","objects_scrubbed":1973,"log_size":5301,"log_dups_size":0,"ondisk_log_size":5301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:09:44.388358+0000","scrub_duration":470,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":102039552,"num_objects":1557,"num_object_clones":0,"num_object_copies":4671,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1557,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":416,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":900,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.981441+0000","last_change":"2024-09-20T19:06:17.937954+0000","last_active":"2024-09-20T19:08:41.981441+0000","last_peered":"2024-09-20T19:08:41.981441+0000","last_clean":"2024-09-20T19:08:41.981441+0000","last_became_active":"2024-09-20T18:59:05.084496+0000","last_became_peered":"2024-09-20T18:59:05.084496+0000","last_unstale":"2024-09-20T19:08:41.981441+0000","last_undegraded":"2024-09-20T19:08:41.981441+0000","last_fullsized":"2024-09-20T19:08:41.981441+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.937896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.937896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"59'1","reported_seq":224,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.976847+0000","last_change":"2024-09-20T19:04:20.905233+0000","last_active":"2024-09-20T19:08:41.976847+0000","last_peered":"2024-09-20T19:08:41.976847+0000","last_clean":"2024-09-20T19:08:41.976847+0000","last_became_active":"2024-09-20T19:04:20.887062+0000","last_became_peered":"2024-09-20T19:04:20.887062+0000","last_unstale":"2024-09-20T19:08:41.976847+0000","last_undegraded":"2024-09-20T19:08:41.976847+0000","last_fullsized":"2024-09-20T19:08:41.976847+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:03:08.218812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:03:08.218812+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":993,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"274'18","reported_seq":950,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.981109+0000","last_change":"2024-09-20T19:08:36.123919+0000","last_active":"2024-09-20T19:08:41.981109+0000","last_peered":"2024-09-20T19:08:41.981109+0000","last_clean":"2024-09-20T19:08:41.981109+0000","last_became_active":"2024-09-20T19:04:20.896343+0000","last_became_peered":"2024-09-20T19:04:20.896343+0000","last_unstale":"2024-09-20T19:08:41.981109+0000","last_undegraded":"2024-09-20T19:08:41.981109+0000","last_fullsized":"2024-09-20T19:08:41.981109+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'18","last_scrub_stamp":"2024-09-20T19:08:36.123833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:08:36.123833+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:10:01.483394+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":228,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.978678+0000","last_change":"2024-09-20T19:06:16.148081+0000","last_active":"2024-09-20T19:08:41.978678+0000","last_peered":"2024-09-20T19:08:41.978678+0000","last_clean":"2024-09-20T19:08:41.978678+0000","last_became_active":"2024-09-20T19:04:20.889430+0000","last_became_peered":"2024-09-20T19:04:20.889430+0000","last_unstale":"2024-09-20T19:08:41.978678+0000","last_undegraded":"2024-09-20T19:08:41.978678+0000","last_fullsized":"2024-09-20T19:08:41.978678+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:16.148022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:06:16.148022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.11","version":"479'6928","reported_seq":12508,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:46.662437+0000","last_change":"2024-09-20T19:08:34.743968+0000","last_active":"2024-09-20T19:08:46.662437+0000","last_peered":"2024-09-20T19:08:46.662437+0000","last_clean":"2024-09-20T19:08:46.662437+0000","last_became_active":"2024-09-20T19:08:34.735149+0000","last_became_peered":"2024-09-20T19:08:34.735149+0000","last_unstale":"2024-09-20T19:08:46.662437+0000","last_undegraded":"2024-09-20T19:08:46.662437+0000","last_fullsized":"2024-09-20T19:08:46.662437+0000","mapping_epoch":475,"log_start":"0'0","ondisk_log_start":"0'0","created":441,"last_epoch_clean":476,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":6928,"log_dups_size":0,"ondisk_log_size":6928,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-20T19:09:42.073774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":1536,"num_object_clones":0,"num_object_copies":4608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5010,"num_write_kb":209472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1950,"num_bytes_recovered":127795200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":878,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.972237+0000","last_change":"2024-09-20T19:02:28.294390+0000","last_active":"2024-09-20T19:08:41.972237+0000","last_peered":"2024-09-20T19:08:41.972237+0000","last_clean":"2024-09-20T19:08:41.972237+0000","last_became_active":"2024-09-20T18:49:51.970058+0000","last_became_peered":"2024-09-20T18:49:51.970058+0000","last_unstale":"2024-09-20T19:08:41.972237+0000","last_undegraded":"2024-09-20T19:08:41.972237+0000","last_fullsized":"2024-09-20T19:08:41.972237+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T19:02:28.294292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:28.294292+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":922,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.973999+0000","last_change":"2024-09-20T19:06:53.021433+0000","last_active":"2024-09-20T19:08:41.973999+0000","last_peered":"2024-09-20T19:08:41.973999+0000","last_clean":"2024-09-20T19:08:41.973999+0000","last_became_active":"2024-09-20T19:04:20.881466+0000","last_became_peered":"2024-09-20T19:04:20.881466+0000","last_unstale":"2024-09-20T19:08:41.973999+0000","last_undegraded":"2024-09-20T19:08:41.973999+0000","last_fullsized":"2024-09-20T19:08:41.973999+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:06:53.021363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:06:53.021363+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"274'20","reported_seq":960,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.964995+0000","last_change":"2024-09-20T19:06:42.777378+0000","last_active":"2024-09-20T19:08:41.964995+0000","last_peered":"2024-09-20T19:08:41.964995+0000","last_clean":"2024-09-20T19:08:41.964995+0000","last_became_active":"2024-09-20T19:03:16.968636+0000","last_became_peered":"2024-09-20T19:03:16.968636+0000","last_unstale":"2024-09-20T19:08:41.964995+0000","last_undegraded":"2024-09-20T19:08:41.964995+0000","last_fullsized":"2024-09-20T19:08:41.964995+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'20","last_scrub_stamp":"2024-09-20T19:06:42.777179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:06:42.777179+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"375'14","reported_seq":632,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.966498+0000","last_change":"2024-09-20T19:08:40.945609+0000","last_active":"2024-09-20T19:08:41.966498+0000","last_peered":"2024-09-20T19:08:41.966498+0000","last_clean":"2024-09-20T19:08:41.966498+0000","last_became_active":"2024-09-20T19:04:20.900782+0000","last_became_peered":"2024-09-20T19:04:20.900782+0000","last_unstale":"2024-09-20T19:08:41.966498+0000","last_undegraded":"2024-09-20T19:08:41.966498+0000","last_fullsized":"2024-09-20T19:08:41.966498+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"375'14","last_scrub_stamp":"2024-09-20T19:08:40.945560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:08:40.945560+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:09:46.550525+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.a","version":"74'79","reported_seq":60,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.977318+0000","last_change":"2024-09-20T19:08:32.844712+0000","last_active":"2024-09-20T19:08:41.977318+0000","last_peered":"2024-09-20T19:08:41.977318+0000","last_clean":"2024-09-20T19:08:41.977318+0000","last_became_active":"2024-09-20T19:07:42.980316+0000","last_became_peered":"2024-09-20T19:07:42.980316+0000","last_unstale":"2024-09-20T19:08:41.977318+0000","last_undegraded":"2024-09-20T19:08:41.977318+0000","last_fullsized":"2024-09-20T19:08:41.977318+0000","mapping_epoch":458,"log_start":"0'0","ondisk_log_start":"0'0","created":455,"last_epoch_clean":459,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:08:32.844657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:08:32.844657+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:09:40.566205+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"479'6978","reported_seq":7142,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:47.624666+0000","last_change":"2024-09-20T19:06:58.386274+0000","last_active":"2024-09-20T19:08:47.624666+0000","last_peered":"2024-09-20T19:08:47.624666+0000","last_clean":"2024-09-20T19:08:47.624666+0000","last_became_active":"2024-09-20T19:06:56.482140+0000","last_became_peered":"2024-09-20T19:06:56.482140+0000","last_unstale":"2024-09-20T19:08:47.624666+0000","last_undegraded":"2024-09-20T19:08:47.624666+0000","last_fullsized":"2024-09-20T19:08:47.624666+0000","mapping_epoch":422,"log_start":"431'1492","ondisk_log_start":"431'1492","created":422,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"443'3875","last_scrub_stamp":"2024-09-20T19:06:58.386214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:58.386214+0000","objects_scrubbed":2003,"log_size":5486,"log_dups_size":0,"ondisk_log_size":5486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":191,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":97124352,"num_objects":1482,"num_object_clones":0,"num_object_copies":4446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3103,"num_write_kb":82624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"274'27","reported_seq":971,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.971897+0000","last_change":"2024-09-20T19:04:20.920811+0000","last_active":"2024-09-20T19:08:41.971897+0000","last_peered":"2024-09-20T19:08:41.971897+0000","last_clean":"2024-09-20T19:08:41.971897+0000","last_became_active":"2024-09-20T19:04:20.897049+0000","last_became_peered":"2024-09-20T19:04:20.897049+0000","last_unstale":"2024-09-20T19:08:41.971897+0000","last_undegraded":"2024-09-20T19:08:41.971897+0000","last_fullsized":"2024-09-20T19:08:41.971897+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'27","last_scrub_stamp":"2024-09-20T19:02:27.897138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:02:27.897138+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":897,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.981764+0000","last_change":"2024-09-20T19:05:55.742165+0000","last_active":"2024-09-20T19:08:41.981764+0000","last_peered":"2024-09-20T19:08:41.981764+0000","last_clean":"2024-09-20T19:08:41.981764+0000","last_became_active":"2024-09-20T18:49:45.936104+0000","last_became_peered":"2024-09-20T18:49:45.936104+0000","last_unstale":"2024-09-20T19:08:41.981764+0000","last_undegraded":"2024-09-20T19:08:41.981764+0000","last_fullsized":"2024-09-20T19:08:41.981764+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:55.742106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:55.742106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"375'12","reported_seq":285,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.975964+0000","last_change":"2024-09-20T19:06:20.174206+0000","last_active":"2024-09-20T19:08:41.975964+0000","last_peered":"2024-09-20T19:08:41.975964+0000","last_clean":"2024-09-20T19:08:41.975964+0000","last_became_active":"2024-09-20T19:04:20.907228+0000","last_became_peered":"2024-09-20T19:04:20.907228+0000","last_unstale":"2024-09-20T19:08:41.975964+0000","last_undegraded":"2024-09-20T19:08:41.975964+0000","last_fullsized":"2024-09-20T19:08:41.975964+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"375'12","last_scrub_stamp":"2024-09-20T19:06:20.174160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:06:20.174160+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":877,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.979218+0000","last_change":"2024-09-20T19:02:22.736879+0000","last_active":"2024-09-20T19:08:41.979218+0000","last_peered":"2024-09-20T19:08:41.979218+0000","last_clean":"2024-09-20T19:08:41.979218+0000","last_became_active":"2024-09-20T18:49:51.971399+0000","last_became_peered":"2024-09-20T18:49:51.971399+0000","last_unstale":"2024-09-20T19:08:41.979218+0000","last_undegraded":"2024-09-20T19:08:41.979218+0000","last_fullsized":"2024-09-20T19:08:41.979218+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:22.736825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:22.736825+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"74'79","reported_seq":22,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.974904+0000","last_change":"2024-09-20T19:08:41.216678+0000","last_active":"2024-09-20T19:08:41.974904+0000","last_peered":"2024-09-20T19:08:41.974904+0000","last_clean":"2024-09-20T19:08:41.974904+0000","last_became_active":"2024-09-20T19:08:40.751519+0000","last_became_peered":"2024-09-20T19:08:40.751519+0000","last_unstale":"2024-09-20T19:08:41.974904+0000","last_undegraded":"2024-09-20T19:08:41.974904+0000","last_fullsized":"2024-09-20T19:08:41.974904+0000","mapping_epoch":477,"log_start":"0'0","ondisk_log_start":"0'0","created":455,"last_epoch_clean":478,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:08:41.216542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:08:41.216542+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:09:44.715593+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.f","version":"479'9994","reported_seq":10120,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:46.447567+0000","last_change":"2024-09-20T19:06:18.056190+0000","last_active":"2024-09-20T19:08:46.447567+0000","last_peered":"2024-09-20T19:08:46.447567+0000","last_clean":"2024-09-20T19:08:46.447567+0000","last_became_active":"2024-09-20T19:06:18.055847+0000","last_became_peered":"2024-09-20T19:06:18.055847+0000","last_unstale":"2024-09-20T19:08:46.447567+0000","last_undegraded":"2024-09-20T19:08:46.447567+0000","last_fullsized":"2024-09-20T19:08:46.447567+0000","mapping_epoch":422,"log_start":"453'5763","ondisk_log_start":"453'5763","created":422,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":4231,"log_dups_size":0,"ondisk_log_size":4231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":190971904,"num_objects":2914,"num_object_clones":0,"num_object_copies":8742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9994,"num_write_kb":413056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"273'30","reported_seq":960,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.980493+0000","last_change":"2024-09-20T19:06:54.927673+0000","last_active":"2024-09-20T19:08:41.980493+0000","last_peered":"2024-09-20T19:08:41.980493+0000","last_clean":"2024-09-20T19:08:41.980493+0000","last_became_active":"2024-09-20T18:49:48.499189+0000","last_became_peered":"2024-09-20T18:49:48.499189+0000","last_unstale":"2024-09-20T19:08:41.980493+0000","last_undegraded":"2024-09-20T19:08:41.980493+0000","last_fullsized":"2024-09-20T19:08:41.980493+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'30","last_scrub_stamp":"2024-09-20T19:06:54.927614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:06:54.927614+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":889,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.981681+0000","last_change":"2024-09-20T19:06:16.897724+0000","last_active":"2024-09-20T19:08:41.981681+0000","last_peered":"2024-09-20T19:08:41.981681+0000","last_clean":"2024-09-20T19:08:41.981681+0000","last_became_active":"2024-09-20T18:49:45.940544+0000","last_became_peered":"2024-09-20T18:49:45.940544+0000","last_unstale":"2024-09-20T19:08:41.981681+0000","last_undegraded":"2024-09-20T19:08:41.981681+0000","last_fullsized":"2024-09-20T19:08:41.981681+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:16.897677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:06:16.897677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":229,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.975459+0000","last_change":"2024-09-20T19:08:33.976475+0000","last_active":"2024-09-20T19:08:41.975459+0000","last_peered":"2024-09-20T19:08:41.975459+0000","last_clean":"2024-09-20T19:08:41.975459+0000","last_became_active":"2024-09-20T19:04:20.888422+0000","last_became_peered":"2024-09-20T19:04:20.888422+0000","last_unstale":"2024-09-20T19:08:41.975459+0000","last_undegraded":"2024-09-20T19:08:41.975459+0000","last_fullsized":"2024-09-20T19:08:41.975459+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:08:33.976404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:08:33.976404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:09:50.829457+0000","scrub_duration":72,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":229,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.977670+0000","last_change":"2024-09-20T19:05:47.121946+0000","last_active":"2024-09-20T19:08:41.977670+0000","last_peered":"2024-09-20T19:08:41.977670+0000","last_clean":"2024-09-20T19:08:41.977670+0000","last_became_active":"2024-09-20T19:04:20.889418+0000","last_became_peered":"2024-09-20T19:04:20.889418+0000","last_unstale":"2024-09-20T19:08:41.977670+0000","last_undegraded":"2024-09-20T19:08:41.977670+0000","last_fullsized":"2024-09-20T19:08:41.977670+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:47.121842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:05:47.121842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"74'79","reported_seq":65,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.980654+0000","last_change":"2024-09-20T19:08:34.108484+0000","last_active":"2024-09-20T19:08:41.980654+0000","last_peered":"2024-09-20T19:08:41.980654+0000","last_clean":"2024-09-20T19:08:41.980654+0000","last_became_active":"2024-09-20T19:07:42.987705+0000","last_became_peered":"2024-09-20T19:07:42.987705+0000","last_unstale":"2024-09-20T19:08:41.980654+0000","last_undegraded":"2024-09-20T19:08:41.980654+0000","last_fullsized":"2024-09-20T19:08:41.980654+0000","mapping_epoch":458,"log_start":"0'0","ondisk_log_start":"0'0","created":455,"last_epoch_clean":461,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:08:34.108411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:08:34.108411+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:09:59.257263+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.e","version":"479'9897","reported_seq":10026,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:45.698112+0000","last_change":"2024-09-20T19:06:18.054020+0000","last_active":"2024-09-20T19:08:45.698112+0000","last_peered":"2024-09-20T19:08:45.698112+0000","last_clean":"2024-09-20T19:08:45.698112+0000","last_became_active":"2024-09-20T19:06:18.053343+0000","last_became_peered":"2024-09-20T19:06:18.053343+0000","last_unstale":"2024-09-20T19:08:45.698112+0000","last_undegraded":"2024-09-20T19:08:45.698112+0000","last_fullsized":"2024-09-20T19:08:45.698112+0000","mapping_epoch":422,"log_start":"449'5414","ondisk_log_start":"449'5414","created":422,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":4483,"log_dups_size":0,"ondisk_log_size":4483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":202702848,"num_objects":3093,"num_object_clones":0,"num_object_copies":9279,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9897,"num_write_kb":415680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"274'28","reported_seq":237,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.975361+0000","last_change":"2024-09-20T19:04:20.976682+0000","last_active":"2024-09-20T19:08:41.975361+0000","last_peered":"2024-09-20T19:08:41.975361+0000","last_clean":"2024-09-20T19:08:41.975361+0000","last_became_active":"2024-09-20T19:04:20.897444+0000","last_became_peered":"2024-09-20T19:04:20.897444+0000","last_unstale":"2024-09-20T19:08:41.975361+0000","last_undegraded":"2024-09-20T19:08:41.975361+0000","last_fullsized":"2024-09-20T19:08:41.975361+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'28","last_scrub_stamp":"2024-09-20T19:03:06.269623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:06.269623+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":41,"num_read_kb":34,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":924,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.965198+0000","last_change":"2024-09-20T19:05:47.629956+0000","last_active":"2024-09-20T19:08:41.965198+0000","last_peered":"2024-09-20T19:08:41.965198+0000","last_clean":"2024-09-20T19:08:41.965198+0000","last_became_active":"2024-09-20T19:04:20.906790+0000","last_became_peered":"2024-09-20T19:04:20.906790+0000","last_unstale":"2024-09-20T19:08:41.965198+0000","last_undegraded":"2024-09-20T19:08:41.965198+0000","last_fullsized":"2024-09-20T19:08:41.965198+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:05:47.629892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:47.629892+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"375'13","reported_seq":1165,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.980399+0000","last_change":"2024-09-20T19:04:20.916872+0000","last_active":"2024-09-20T19:08:41.980399+0000","last_peered":"2024-09-20T19:08:41.980399+0000","last_clean":"2024-09-20T19:08:41.980399+0000","last_became_active":"2024-09-20T19:04:20.896424+0000","last_became_peered":"2024-09-20T19:04:20.896424+0000","last_unstale":"2024-09-20T19:08:41.980399+0000","last_undegraded":"2024-09-20T19:08:41.980399+0000","last_fullsized":"2024-09-20T19:08:41.980399+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T18:58:23.114371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T18:58:23.114371+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":885,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.972639+0000","last_change":"2024-09-20T18:59:05.082117+0000","last_active":"2024-09-20T19:08:41.972639+0000","last_peered":"2024-09-20T19:08:41.972639+0000","last_clean":"2024-09-20T19:08:41.972639+0000","last_became_active":"2024-09-20T18:59:05.081814+0000","last_became_peered":"2024-09-20T18:59:05.081814+0000","last_unstale":"2024-09-20T19:08:41.972639+0000","last_undegraded":"2024-09-20T19:08:41.972639+0000","last_fullsized":"2024-09-20T19:08:41.972639+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:58:29.530001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:58:29.530001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"74'79","reported_seq":554,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.966096+0000","last_change":"2024-09-20T19:08:28.837854+0000","last_active":"2024-09-20T19:08:41.966096+0000","last_peered":"2024-09-20T19:08:41.966096+0000","last_clean":"2024-09-20T19:08:41.966096+0000","last_became_active":"2024-09-20T19:07:42.988925+0000","last_became_peered":"2024-09-20T19:07:42.988925+0000","last_unstale":"2024-09-20T19:08:41.966096+0000","last_undegraded":"2024-09-20T19:08:41.966096+0000","last_fullsized":"2024-09-20T19:08:41.966096+0000","mapping_epoch":458,"log_start":"0'0","ondisk_log_start":"0'0","created":455,"last_epoch_clean":459,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:08:28.837791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:08:28.837791+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:09:44.409164+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"479'9936","reported_seq":10066,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:46.847013+0000","last_change":"2024-09-20T19:06:18.053626+0000","last_active":"2024-09-20T19:08:46.847013+0000","last_peered":"2024-09-20T19:08:46.847013+0000","last_clean":"2024-09-20T19:08:46.847013+0000","last_became_active":"2024-09-20T19:06:18.053336+0000","last_became_peered":"2024-09-20T19:06:18.053336+0000","last_unstale":"2024-09-20T19:08:46.847013+0000","last_undegraded":"2024-09-20T19:08:46.847013+0000","last_fullsized":"2024-09-20T19:08:46.847013+0000","mapping_epoch":422,"log_start":"447'4566","ondisk_log_start":"447'4566","created":422,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":5370,"log_dups_size":0,"ondisk_log_size":5370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":198246424,"num_objects":3026,"num_object_clones":0,"num_object_copies":9078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3026,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9936,"num_write_kb":414721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"274'20","reported_seq":735,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.977909+0000","last_change":"2024-09-20T19:05:48.741237+0000","last_active":"2024-09-20T19:08:41.977909+0000","last_peered":"2024-09-20T19:08:41.977909+0000","last_clean":"2024-09-20T19:08:41.977909+0000","last_became_active":"2024-09-20T19:04:49.419140+0000","last_became_peered":"2024-09-20T19:04:49.419140+0000","last_unstale":"2024-09-20T19:08:41.977909+0000","last_undegraded":"2024-09-20T19:08:41.977909+0000","last_fullsized":"2024-09-20T19:08:41.977909+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'20","last_scrub_stamp":"2024-09-20T19:05:48.741059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:48.741059+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":914,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.979798+0000","last_change":"2024-09-20T19:05:56.820141+0000","last_active":"2024-09-20T19:08:41.979798+0000","last_peered":"2024-09-20T19:08:41.979798+0000","last_clean":"2024-09-20T19:08:41.979798+0000","last_became_active":"2024-09-20T18:49:45.942496+0000","last_became_peered":"2024-09-20T18:49:45.942496+0000","last_unstale":"2024-09-20T19:08:41.979798+0000","last_undegraded":"2024-09-20T19:08:41.979798+0000","last_fullsized":"2024-09-20T19:08:41.979798+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:56.820079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:56.820079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"262'10","reported_seq":1142,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.967595+0000","last_change":"2024-09-20T19:02:04.994109+0000","last_active":"2024-09-20T19:08:41.967595+0000","last_peered":"2024-09-20T19:08:41.967595+0000","last_clean":"2024-09-20T19:08:41.967595+0000","last_became_active":"2024-09-20T18:59:05.080183+0000","last_became_peered":"2024-09-20T18:59:05.080183+0000","last_unstale":"2024-09-20T19:08:41.967595+0000","last_undegraded":"2024-09-20T19:08:41.967595+0000","last_fullsized":"2024-09-20T19:08:41.967595+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'10","last_scrub_stamp":"2024-09-20T19:02:04.994047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:04.994047+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":871,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.965068+0000","last_change":"2024-09-20T19:06:15.671414+0000","last_active":"2024-09-20T19:08:41.965068+0000","last_peered":"2024-09-20T19:08:41.965068+0000","last_clean":"2024-09-20T19:08:41.965068+0000","last_became_active":"2024-09-20T18:49:51.964780+0000","last_became_peered":"2024-09-20T18:49:51.964780+0000","last_unstale":"2024-09-20T19:08:41.965068+0000","last_undegraded":"2024-09-20T19:08:41.965068+0000","last_fullsized":"2024-09-20T19:08:41.965068+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:15.671352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:06:15.671352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"74'79","reported_seq":554,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.966100+0000","last_change":"2024-09-20T19:08:32.863345+0000","last_active":"2024-09-20T19:08:41.966100+0000","last_peered":"2024-09-20T19:08:41.966100+0000","last_clean":"2024-09-20T19:08:41.966100+0000","last_became_active":"2024-09-20T19:07:42.994388+0000","last_became_peered":"2024-09-20T19:07:42.994388+0000","last_unstale":"2024-09-20T19:08:41.966100+0000","last_undegraded":"2024-09-20T19:08:41.966100+0000","last_fullsized":"2024-09-20T19:08:41.966100+0000","mapping_epoch":458,"log_start":"0'0","ondisk_log_start":"0'0","created":455,"last_epoch_clean":459,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:08:32.863301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:08:32.863301+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:10:02.430951+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.c","version":"479'9995","reported_seq":10124,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:47.685875+0000","last_change":"2024-09-20T19:06:18.050185+0000","last_active":"2024-09-20T19:08:47.685875+0000","last_peered":"2024-09-20T19:08:47.685875+0000","last_clean":"2024-09-20T19:08:47.685875+0000","last_became_active":"2024-09-20T19:06:18.049949+0000","last_became_peered":"2024-09-20T19:06:18.049949+0000","last_unstale":"2024-09-20T19:08:47.685875+0000","last_undegraded":"2024-09-20T19:08:47.685875+0000","last_fullsized":"2024-09-20T19:08:47.685875+0000","mapping_epoch":422,"log_start":"451'5600","ondisk_log_start":"451'5600","created":422,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":4395,"log_dups_size":0,"ondisk_log_size":4395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":187760640,"num_objects":2865,"num_object_clones":0,"num_object_copies":8595,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9995,"num_write_kb":411520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,4],"acting":[1,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"274'29","reported_seq":938,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.967742+0000","last_change":"2024-09-20T19:05:43.631990+0000","last_active":"2024-09-20T19:08:41.967742+0000","last_peered":"2024-09-20T19:08:41.967742+0000","last_clean":"2024-09-20T19:08:41.967742+0000","last_became_active":"2024-09-20T18:49:48.497736+0000","last_became_peered":"2024-09-20T18:49:48.497736+0000","last_unstale":"2024-09-20T19:08:41.967742+0000","last_undegraded":"2024-09-20T19:08:41.967742+0000","last_fullsized":"2024-09-20T19:08:41.967742+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'29","last_scrub_stamp":"2024-09-20T19:05:43.631845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:43.631845+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":890,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.977124+0000","last_change":"2024-09-20T19:05:47.837587+0000","last_active":"2024-09-20T19:08:41.977124+0000","last_peered":"2024-09-20T19:08:41.977124+0000","last_clean":"2024-09-20T19:08:41.977124+0000","last_became_active":"2024-09-20T18:49:45.942253+0000","last_became_peered":"2024-09-20T18:49:45.942253+0000","last_unstale":"2024-09-20T19:08:41.977124+0000","last_undegraded":"2024-09-20T19:08:41.977124+0000","last_fullsized":"2024-09-20T19:08:41.977124+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:47.837520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:47.837520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":476,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.970891+0000","last_change":"2024-09-20T19:05:48.203244+0000","last_active":"2024-09-20T19:08:41.970891+0000","last_peered":"2024-09-20T19:08:41.970891+0000","last_clean":"2024-09-20T19:08:41.970891+0000","last_became_active":"2024-09-20T18:59:05.085636+0000","last_became_peered":"2024-09-20T18:59:05.085636+0000","last_unstale":"2024-09-20T19:08:41.970891+0000","last_undegraded":"2024-09-20T19:08:41.970891+0000","last_fullsized":"2024-09-20T19:08:41.970891+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:48.203182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:05:48.203182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":908,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.973806+0000","last_change":"2024-09-20T19:06:43.932713+0000","last_active":"2024-09-20T19:08:41.973806+0000","last_peered":"2024-09-20T19:08:41.973806+0000","last_clean":"2024-09-20T19:08:41.973806+0000","last_became_active":"2024-09-20T18:59:05.085131+0000","last_became_peered":"2024-09-20T18:59:05.085131+0000","last_unstale":"2024-09-20T19:08:41.973806+0000","last_undegraded":"2024-09-20T19:08:41.973806+0000","last_fullsized":"2024-09-20T19:08:41.973806+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:43.932649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:06:43.932649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"74'79","reported_seq":60,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.977330+0000","last_change":"2024-09-20T19:08:34.853946+0000","last_active":"2024-09-20T19:08:41.977330+0000","last_peered":"2024-09-20T19:08:41.977330+0000","last_clean":"2024-09-20T19:08:41.977330+0000","last_became_active":"2024-09-20T19:07:42.995299+0000","last_became_peered":"2024-09-20T19:07:42.995299+0000","last_unstale":"2024-09-20T19:08:41.977330+0000","last_undegraded":"2024-09-20T19:08:41.977330+0000","last_fullsized":"2024-09-20T19:08:41.977330+0000","mapping_epoch":458,"log_start":"0'0","ondisk_log_start":"0'0","created":455,"last_epoch_clean":459,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:08:34.853872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:08:34.853872+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:09:56.511535+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.b","version":"479'9576","reported_seq":9705,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:46.075448+0000","last_change":"2024-09-20T19:06:18.057411+0000","last_active":"2024-09-20T19:08:46.075448+0000","last_peered":"2024-09-20T19:08:46.075448+0000","last_clean":"2024-09-20T19:08:46.075448+0000","last_became_active":"2024-09-20T19:06:18.057129+0000","last_became_peered":"2024-09-20T19:06:18.057129+0000","last_unstale":"2024-09-20T19:08:46.075448+0000","last_undegraded":"2024-09-20T19:08:46.075448+0000","last_fullsized":"2024-09-20T19:08:46.075448+0000","mapping_epoch":422,"log_start":"449'5073","ondisk_log_start":"449'5073","created":422,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":4503,"log_dups_size":0,"ondisk_log_size":4503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":192020480,"num_objects":2930,"num_object_clones":0,"num_object_copies":8790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9576,"num_write_kb":400192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"353'41","reported_seq":1226,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.976865+0000","last_change":"2024-09-20T19:05:48.855983+0000","last_active":"2024-09-20T19:08:41.976865+0000","last_peered":"2024-09-20T19:08:41.976865+0000","last_clean":"2024-09-20T19:08:41.976865+0000","last_became_active":"2024-09-20T19:03:16.976916+0000","last_became_peered":"2024-09-20T19:03:16.976916+0000","last_unstale":"2024-09-20T19:08:41.976865+0000","last_undegraded":"2024-09-20T19:08:41.976865+0000","last_fullsized":"2024-09-20T19:08:41.976865+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"353'41","last_scrub_stamp":"2024-09-20T19:05:48.855876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:48.855876+0000","objects_scrubbed":10,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":229,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.977908+0000","last_change":"2024-09-20T19:06:15.168347+0000","last_active":"2024-09-20T19:08:41.977908+0000","last_peered":"2024-09-20T19:08:41.977908+0000","last_clean":"2024-09-20T19:08:41.977908+0000","last_became_active":"2024-09-20T19:04:20.899352+0000","last_became_peered":"2024-09-20T19:04:20.899352+0000","last_unstale":"2024-09-20T19:08:41.977908+0000","last_undegraded":"2024-09-20T19:08:41.977908+0000","last_fullsized":"2024-09-20T19:08:41.977908+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:15.168293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:06:15.168293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":946,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.977997+0000","last_change":"2024-09-20T19:03:11.910684+0000","last_active":"2024-09-20T19:08:41.977997+0000","last_peered":"2024-09-20T19:08:41.977997+0000","last_clean":"2024-09-20T19:08:41.977997+0000","last_became_active":"2024-09-20T18:59:05.086786+0000","last_became_peered":"2024-09-20T18:59:05.086786+0000","last_unstale":"2024-09-20T19:08:41.977997+0000","last_undegraded":"2024-09-20T19:08:41.977997+0000","last_fullsized":"2024-09-20T19:08:41.977997+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-20T19:03:11.909394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:03:11.909394+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0.0012781159999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":924,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.964345+0000","last_change":"2024-09-20T19:06:55.997516+0000","last_active":"2024-09-20T19:08:41.964345+0000","last_peered":"2024-09-20T19:08:41.964345+0000","last_clean":"2024-09-20T19:08:41.964345+0000","last_became_active":"2024-09-20T19:04:20.907038+0000","last_became_peered":"2024-09-20T19:04:20.907038+0000","last_unstale":"2024-09-20T19:08:41.964345+0000","last_undegraded":"2024-09-20T19:08:41.964345+0000","last_fullsized":"2024-09-20T19:08:41.964345+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:55.997469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:06:55.997469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":886,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.981215+0000","last_change":"2024-09-20T19:04:20.896616+0000","last_active":"2024-09-20T19:08:41.981215+0000","last_peered":"2024-09-20T19:08:41.981215+0000","last_clean":"2024-09-20T19:08:41.981215+0000","last_became_active":"2024-09-20T19:04:20.896281+0000","last_became_peered":"2024-09-20T19:04:20.896281+0000","last_unstale":"2024-09-20T19:08:41.981215+0000","last_undegraded":"2024-09-20T19:08:41.981215+0000","last_fullsized":"2024-09-20T19:08:41.981215+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:04.526683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:04.526683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"74'79","reported_seq":982,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.971861+0000","last_change":"2024-09-20T19:08:39.956541+0000","last_active":"2024-09-20T19:08:41.971861+0000","last_peered":"2024-09-20T19:08:41.971861+0000","last_clean":"2024-09-20T19:08:41.971861+0000","last_became_active":"2024-09-20T19:07:42.987148+0000","last_became_peered":"2024-09-20T19:07:42.987148+0000","last_unstale":"2024-09-20T19:08:41.971861+0000","last_undegraded":"2024-09-20T19:08:41.971861+0000","last_fullsized":"2024-09-20T19:08:41.971861+0000","mapping_epoch":458,"log_start":"0'0","ondisk_log_start":"0'0","created":455,"last_epoch_clean":459,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:08:39.956371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:08:39.956371+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:09:42.603382+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.8","version":"479'6765","reported_seq":6928,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:46.092826+0000","last_change":"2024-09-20T19:06:58.108560+0000","last_active":"2024-09-20T19:08:46.092826+0000","last_peered":"2024-09-20T19:08:46.092826+0000","last_clean":"2024-09-20T19:08:46.092826+0000","last_became_active":"2024-09-20T19:06:56.463634+0000","last_became_peered":"2024-09-20T19:06:56.463634+0000","last_unstale":"2024-09-20T19:08:46.092826+0000","last_undegraded":"2024-09-20T19:08:46.092826+0000","last_fullsized":"2024-09-20T19:08:46.092826+0000","mapping_epoch":422,"log_start":"429'950","ondisk_log_start":"429'950","created":422,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"443'3725","last_scrub_stamp":"2024-09-20T19:06:58.108433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:58.108433+0000","objects_scrubbed":1931,"log_size":5815,"log_dups_size":0,"ondisk_log_size":5815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":242,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":102170624,"num_objects":1559,"num_object_clones":0,"num_object_copies":4677,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1559,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3040,"num_write_kb":85376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"273'17","reported_seq":934,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.975926+0000","last_change":"2024-09-20T19:05:43.974098+0000","last_active":"2024-09-20T19:08:41.975926+0000","last_peered":"2024-09-20T19:08:41.975926+0000","last_clean":"2024-09-20T19:08:41.975926+0000","last_became_active":"2024-09-20T18:49:47.950904+0000","last_became_peered":"2024-09-20T18:49:47.950904+0000","last_unstale":"2024-09-20T19:08:41.975926+0000","last_undegraded":"2024-09-20T19:08:41.975926+0000","last_fullsized":"2024-09-20T19:08:41.975926+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'17","last_scrub_stamp":"2024-09-20T19:05:43.974013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:43.974013+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":36,"num_read_kb":25,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":920,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.977456+0000","last_change":"2024-09-20T19:08:40.199763+0000","last_active":"2024-09-20T19:08:41.977456+0000","last_peered":"2024-09-20T19:08:41.977456+0000","last_clean":"2024-09-20T19:08:41.977456+0000","last_became_active":"2024-09-20T18:59:05.079789+0000","last_became_peered":"2024-09-20T18:59:05.079789+0000","last_unstale":"2024-09-20T19:08:41.977456+0000","last_undegraded":"2024-09-20T19:08:41.977456+0000","last_fullsized":"2024-09-20T19:08:41.977456+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:08:40.199695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:08:40.199695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:09:43.615230+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":902,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.968724+0000","last_change":"2024-09-20T19:06:13.599431+0000","last_active":"2024-09-20T19:08:41.968724+0000","last_peered":"2024-09-20T19:08:41.968724+0000","last_clean":"2024-09-20T19:08:41.968724+0000","last_became_active":"2024-09-20T18:49:44.937135+0000","last_became_peered":"2024-09-20T18:49:44.937135+0000","last_unstale":"2024-09-20T19:08:41.968724+0000","last_undegraded":"2024-09-20T19:08:41.968724+0000","last_fullsized":"2024-09-20T19:08:41.968724+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:06:13.599376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:06:13.599376+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00095558800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":868,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.968462+0000","last_change":"2024-09-20T19:01:17.747452+0000","last_active":"2024-09-20T19:08:41.968462+0000","last_peered":"2024-09-20T19:08:41.968462+0000","last_clean":"2024-09-20T19:08:41.968462+0000","last_became_active":"2024-09-20T18:49:49.954424+0000","last_became_peered":"2024-09-20T18:49:49.954424+0000","last_unstale":"2024-09-20T19:08:41.968462+0000","last_undegraded":"2024-09-20T19:08:41.968462+0000","last_fullsized":"2024-09-20T19:08:41.968462+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:17.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:17.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":880,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.972010+0000","last_change":"2024-09-20T19:01:15.203591+0000","last_active":"2024-09-20T19:08:41.972010+0000","last_peered":"2024-09-20T19:08:41.972010+0000","last_clean":"2024-09-20T19:08:41.972010+0000","last_became_active":"2024-09-20T18:59:05.081459+0000","last_became_peered":"2024-09-20T18:59:05.081459+0000","last_unstale":"2024-09-20T19:08:41.972010+0000","last_undegraded":"2024-09-20T19:08:41.972010+0000","last_fullsized":"2024-09-20T19:08:41.972010+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:15.203529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:01:15.203529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"74'79","reported_seq":60,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.974941+0000","last_change":"2024-09-20T19:08:32.123403+0000","last_active":"2024-09-20T19:08:41.974941+0000","last_peered":"2024-09-20T19:08:41.974941+0000","last_clean":"2024-09-20T19:08:41.974941+0000","last_became_active":"2024-09-20T19:07:42.991822+0000","last_became_peered":"2024-09-20T19:07:42.991822+0000","last_unstale":"2024-09-20T19:08:41.974941+0000","last_undegraded":"2024-09-20T19:08:41.974941+0000","last_fullsized":"2024-09-20T19:08:41.974941+0000","mapping_epoch":458,"log_start":"0'0","ondisk_log_start":"0'0","created":455,"last_epoch_clean":459,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:08:32.123325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:08:32.123325+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:09:36.642961+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.9","version":"479'6940","reported_seq":7104,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:46.007488+0000","last_change":"2024-09-20T19:06:57.306900+0000","last_active":"2024-09-20T19:08:46.007488+0000","last_peered":"2024-09-20T19:08:46.007488+0000","last_clean":"2024-09-20T19:08:46.007488+0000","last_became_active":"2024-09-20T19:06:56.466304+0000","last_became_peered":"2024-09-20T19:06:56.466304+0000","last_unstale":"2024-09-20T19:08:46.007488+0000","last_undegraded":"2024-09-20T19:08:46.007488+0000","last_fullsized":"2024-09-20T19:08:46.007488+0000","mapping_epoch":422,"log_start":"430'1114","ondisk_log_start":"430'1114","created":422,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"442'3861","last_scrub_stamp":"2024-09-20T19:06:57.306685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:57.306685+0000","objects_scrubbed":1958,"log_size":5826,"log_dups_size":0,"ondisk_log_size":5826,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":402,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":103612416,"num_objects":1581,"num_object_clones":0,"num_object_copies":4743,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3089,"num_write_kb":86784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"273'24","reported_seq":938,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.965102+0000","last_change":"2024-09-20T19:08:36.505778+0000","last_active":"2024-09-20T19:08:41.965102+0000","last_peered":"2024-09-20T19:08:41.965102+0000","last_clean":"2024-09-20T19:08:41.965102+0000","last_became_active":"2024-09-20T18:49:47.949230+0000","last_became_peered":"2024-09-20T18:49:47.949230+0000","last_unstale":"2024-09-20T19:08:41.965102+0000","last_undegraded":"2024-09-20T19:08:41.965102+0000","last_fullsized":"2024-09-20T19:08:41.965102+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'24","last_scrub_stamp":"2024-09-20T19:08:36.505700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:08:36.505700+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:09:40.434737+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":916,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.965412+0000","last_change":"2024-09-20T19:04:20.907250+0000","last_active":"2024-09-20T19:08:41.965412+0000","last_peered":"2024-09-20T19:08:41.965412+0000","last_clean":"2024-09-20T19:08:41.965412+0000","last_became_active":"2024-09-20T19:04:20.906969+0000","last_became_peered":"2024-09-20T19:04:20.906969+0000","last_unstale":"2024-09-20T19:08:41.965412+0000","last_undegraded":"2024-09-20T19:08:41.965412+0000","last_fullsized":"2024-09-20T19:08:41.965412+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:24.984022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:02:24.984022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":2043,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:44.889352+0000","last_change":"2024-09-20T19:06:43.885193+0000","last_active":"2024-09-20T19:08:44.889352+0000","last_peered":"2024-09-20T19:08:44.889352+0000","last_clean":"2024-09-20T19:08:44.889352+0000","last_became_active":"2024-09-20T18:49:44.936819+0000","last_became_peered":"2024-09-20T18:49:44.936819+0000","last_unstale":"2024-09-20T19:08:44.889352+0000","last_undegraded":"2024-09-20T19:08:44.889352+0000","last_fullsized":"2024-09-20T19:08:44.889352+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-20T19:06:43.883404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:06:43.883404+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.0010996199999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1133,"num_read_kb":1128,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":897,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.981635+0000","last_change":"2024-09-20T19:05:51.743485+0000","last_active":"2024-09-20T19:08:41.981635+0000","last_peered":"2024-09-20T19:08:41.981635+0000","last_clean":"2024-09-20T19:08:41.981635+0000","last_became_active":"2024-09-20T18:49:49.961066+0000","last_became_peered":"2024-09-20T18:49:49.961066+0000","last_unstale":"2024-09-20T19:08:41.981635+0000","last_undegraded":"2024-09-20T19:08:41.981635+0000","last_fullsized":"2024-09-20T19:08:41.981635+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:51.743363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:05:51.743363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":894,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.977068+0000","last_change":"2024-09-20T19:08:37.176006+0000","last_active":"2024-09-20T19:08:41.977068+0000","last_peered":"2024-09-20T19:08:41.977068+0000","last_clean":"2024-09-20T19:08:41.977068+0000","last_became_active":"2024-09-20T18:49:51.970343+0000","last_became_peered":"2024-09-20T18:49:51.970343+0000","last_unstale":"2024-09-20T19:08:41.977068+0000","last_undegraded":"2024-09-20T19:08:41.977068+0000","last_fullsized":"2024-09-20T19:08:41.977068+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:08:37.175807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:08:37.175807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:09:47.982253+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.a","version":"479'9831","reported_seq":9960,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:46.119113+0000","last_change":"2024-09-20T19:06:18.055447+0000","last_active":"2024-09-20T19:08:46.119113+0000","last_peered":"2024-09-20T19:08:46.119113+0000","last_clean":"2024-09-20T19:08:46.119113+0000","last_became_active":"2024-09-20T19:06:18.055189+0000","last_became_peered":"2024-09-20T19:06:18.055189+0000","last_unstale":"2024-09-20T19:08:46.119113+0000","last_undegraded":"2024-09-20T19:08:46.119113+0000","last_fullsized":"2024-09-20T19:08:46.119113+0000","mapping_epoch":422,"log_start":"449'5334","ondisk_log_start":"449'5334","created":422,"last_epoch_clean":423,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:17.033924+0000","objects_scrubbed":0,"log_size":4497,"log_dups_size":0,"ondisk_log_size":4497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":196804608,"num_objects":3003,"num_object_clones":0,"num_object_copies":9009,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9831,"num_write_kb":410688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"375'34","reported_seq":684,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.971150+0000","last_change":"2024-09-20T19:05:54.115428+0000","last_active":"2024-09-20T19:08:41.971150+0000","last_peered":"2024-09-20T19:08:41.971150+0000","last_clean":"2024-09-20T19:08:41.971150+0000","last_became_active":"2024-09-20T19:04:20.928727+0000","last_became_peered":"2024-09-20T19:04:20.928727+0000","last_unstale":"2024-09-20T19:08:41.971150+0000","last_undegraded":"2024-09-20T19:08:41.971150+0000","last_fullsized":"2024-09-20T19:08:41.971150+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"375'34","last_scrub_stamp":"2024-09-20T19:05:54.115376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:54.115376+0000","objects_scrubbed":8,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":462,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":920,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.983468+0000","last_change":"2024-09-20T19:06:47.914880+0000","last_active":"2024-09-20T19:08:41.983468+0000","last_peered":"2024-09-20T19:08:41.983468+0000","last_clean":"2024-09-20T19:08:41.983468+0000","last_became_active":"2024-09-20T18:59:05.079111+0000","last_became_peered":"2024-09-20T18:59:05.079111+0000","last_unstale":"2024-09-20T19:08:41.983468+0000","last_undegraded":"2024-09-20T19:08:41.983468+0000","last_fullsized":"2024-09-20T19:08:41.983468+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:47.914630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:06:47.914630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":977,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.972903+0000","last_change":"2024-09-20T19:08:38.989365+0000","last_active":"2024-09-20T19:08:41.972903+0000","last_peered":"2024-09-20T19:08:41.972903+0000","last_clean":"2024-09-20T19:08:41.972903+0000","last_became_active":"2024-09-20T19:07:40.718130+0000","last_became_peered":"2024-09-20T19:07:40.718130+0000","last_unstale":"2024-09-20T19:08:41.972903+0000","last_undegraded":"2024-09-20T19:08:41.972903+0000","last_fullsized":"2024-09-20T19:08:41.972903+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":456,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:08:38.989294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:08:38.989294+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:09:52.027427+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":883,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.972406+0000","last_change":"2024-09-20T19:06:21.584094+0000","last_active":"2024-09-20T19:08:41.972406+0000","last_peered":"2024-09-20T19:08:41.972406+0000","last_clean":"2024-09-20T19:08:41.972406+0000","last_became_active":"2024-09-20T18:49:49.956128+0000","last_became_peered":"2024-09-20T18:49:49.956128+0000","last_unstale":"2024-09-20T19:08:41.972406+0000","last_undegraded":"2024-09-20T19:08:41.972406+0000","last_fullsized":"2024-09-20T19:08:41.972406+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:21.584027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:06:21.584027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":918,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.981520+0000","last_change":"2024-09-20T19:06:43.994634+0000","last_active":"2024-09-20T19:08:41.981520+0000","last_peered":"2024-09-20T19:08:41.981520+0000","last_clean":"2024-09-20T19:08:41.981520+0000","last_became_active":"2024-09-20T19:04:20.899321+0000","last_became_peered":"2024-09-20T19:04:20.899321+0000","last_unstale":"2024-09-20T19:08:41.981520+0000","last_undegraded":"2024-09-20T19:08:41.981520+0000","last_fullsized":"2024-09-20T19:08:41.981520+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:43.994470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:06:43.994470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1","version":"479'6943","reported_seq":7109,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:47.010567+0000","last_change":"2024-09-20T19:08:24.157661+0000","last_active":"2024-09-20T19:08:47.010567+0000","last_peered":"2024-09-20T19:08:47.010567+0000","last_clean":"2024-09-20T19:08:47.010567+0000","last_became_active":"2024-09-20T19:06:56.471168+0000","last_became_peered":"2024-09-20T19:06:56.471168+0000","last_unstale":"2024-09-20T19:08:47.010567+0000","last_undegraded":"2024-09-20T19:08:47.010567+0000","last_fullsized":"2024-09-20T19:08:47.010567+0000","mapping_epoch":422,"log_start":"426'399","ondisk_log_start":"426'399","created":422,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'6094","last_scrub_stamp":"2024-09-20T19:08:24.157520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:08:24.157520+0000","objects_scrubbed":2284,"log_size":6544,"log_dups_size":0,"ondisk_log_size":6544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:09:37.562109+0000","scrub_duration":339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":93847552,"num_objects":1432,"num_object_clones":0,"num_object_copies":4296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":852,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"274'19","reported_seq":240,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.975703+0000","last_change":"2024-09-20T19:05:50.066533+0000","last_active":"2024-09-20T19:08:41.975703+0000","last_peered":"2024-09-20T19:08:41.975703+0000","last_clean":"2024-09-20T19:08:41.975703+0000","last_became_active":"2024-09-20T19:04:20.897175+0000","last_became_peered":"2024-09-20T19:04:20.897175+0000","last_unstale":"2024-09-20T19:08:41.975703+0000","last_undegraded":"2024-09-20T19:08:41.975703+0000","last_fullsized":"2024-09-20T19:08:41.975703+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'19","last_scrub_stamp":"2024-09-20T19:05:50.066359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:50.066359+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":229,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.975499+0000","last_change":"2024-09-20T19:05:56.027187+0000","last_active":"2024-09-20T19:08:41.975499+0000","last_peered":"2024-09-20T19:08:41.975499+0000","last_clean":"2024-09-20T19:08:41.975499+0000","last_became_active":"2024-09-20T19:04:20.887102+0000","last_became_peered":"2024-09-20T19:04:20.887102+0000","last_unstale":"2024-09-20T19:08:41.975499+0000","last_undegraded":"2024-09-20T19:08:41.975499+0000","last_fullsized":"2024-09-20T19:08:41.975499+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:56.027123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:56.027123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":928,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.981289+0000","last_change":"2024-09-20T19:04:20.899830+0000","last_active":"2024-09-20T19:08:41.981289+0000","last_peered":"2024-09-20T19:08:41.981289+0000","last_clean":"2024-09-20T19:08:41.981289+0000","last_became_active":"2024-09-20T19:04:20.899331+0000","last_became_peered":"2024-09-20T19:04:20.899331+0000","last_unstale":"2024-09-20T19:08:41.981289+0000","last_undegraded":"2024-09-20T19:08:41.981289+0000","last_fullsized":"2024-09-20T19:08:41.981289+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:38.463404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:38.463404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":891,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.980744+0000","last_change":"2024-09-20T19:04:35.529672+0000","last_active":"2024-09-20T19:08:41.980744+0000","last_peered":"2024-09-20T19:08:41.980744+0000","last_clean":"2024-09-20T19:08:41.980744+0000","last_became_active":"2024-09-20T18:59:05.079897+0000","last_became_peered":"2024-09-20T18:59:05.079897+0000","last_unstale":"2024-09-20T19:08:41.980744+0000","last_undegraded":"2024-09-20T19:08:41.980744+0000","last_fullsized":"2024-09-20T19:08:41.980744+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:35.529584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:04:35.529584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.8","version":"74'79","reported_seq":59,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.977814+0000","last_change":"2024-09-20T19:08:22.873127+0000","last_active":"2024-09-20T19:08:41.977814+0000","last_peered":"2024-09-20T19:08:41.977814+0000","last_clean":"2024-09-20T19:08:41.977814+0000","last_became_active":"2024-09-20T19:07:42.988175+0000","last_became_peered":"2024-09-20T19:07:42.988175+0000","last_unstale":"2024-09-20T19:08:41.977814+0000","last_undegraded":"2024-09-20T19:08:41.977814+0000","last_fullsized":"2024-09-20T19:08:41.977814+0000","mapping_epoch":458,"log_start":"0'0","ondisk_log_start":"0'0","created":455,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:03:14.368279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:03:14.368279+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":20,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":316358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2","version":"479'6700","reported_seq":6862,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:46.854577+0000","last_change":"2024-09-20T19:06:58.880519+0000","last_active":"2024-09-20T19:08:46.854577+0000","last_peered":"2024-09-20T19:08:46.854577+0000","last_clean":"2024-09-20T19:08:46.854577+0000","last_became_active":"2024-09-20T19:06:56.454466+0000","last_became_peered":"2024-09-20T19:06:56.454466+0000","last_unstale":"2024-09-20T19:08:46.854577+0000","last_undegraded":"2024-09-20T19:08:46.854577+0000","last_fullsized":"2024-09-20T19:08:46.854577+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"443'3744","last_scrub_stamp":"2024-09-20T19:06:58.880419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:58.880419+0000","objects_scrubbed":1883,"log_size":6700,"log_dups_size":0,"ondisk_log_size":6700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":189,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":94699520,"num_objects":1445,"num_object_clones":0,"num_object_copies":4335,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2956,"num_write_kb":80576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"274'31","reported_seq":246,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.978481+0000","last_change":"2024-09-20T19:06:57.528138+0000","last_active":"2024-09-20T19:08:41.978481+0000","last_peered":"2024-09-20T19:08:41.978481+0000","last_clean":"2024-09-20T19:08:41.978481+0000","last_became_active":"2024-09-20T19:04:20.883306+0000","last_became_peered":"2024-09-20T19:04:20.883306+0000","last_unstale":"2024-09-20T19:08:41.978481+0000","last_undegraded":"2024-09-20T19:08:41.978481+0000","last_fullsized":"2024-09-20T19:08:41.978481+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'31","last_scrub_stamp":"2024-09-20T19:06:57.528084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:06:57.528084+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":49,"num_read_kb":39,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":485,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.971058+0000","last_change":"2024-09-20T19:04:20.899712+0000","last_active":"2024-09-20T19:08:41.971058+0000","last_peered":"2024-09-20T19:08:41.971058+0000","last_clean":"2024-09-20T19:08:41.971058+0000","last_became_active":"2024-09-20T19:04:20.899415+0000","last_became_peered":"2024-09-20T19:04:20.899415+0000","last_unstale":"2024-09-20T19:08:41.971058+0000","last_undegraded":"2024-09-20T19:08:41.971058+0000","last_fullsized":"2024-09-20T19:08:41.971058+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:42.479489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:42.479489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":914,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.976685+0000","last_change":"2024-09-20T19:05:46.866958+0000","last_active":"2024-09-20T19:08:41.976685+0000","last_peered":"2024-09-20T19:08:41.976685+0000","last_clean":"2024-09-20T19:08:41.976685+0000","last_became_active":"2024-09-20T19:04:20.891772+0000","last_became_peered":"2024-09-20T19:04:20.891772+0000","last_unstale":"2024-09-20T19:08:41.976685+0000","last_undegraded":"2024-09-20T19:08:41.976685+0000","last_fullsized":"2024-09-20T19:08:41.976685+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:46.866777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:05:46.866777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":913,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.972086+0000","last_change":"2024-09-20T19:06:25.628102+0000","last_active":"2024-09-20T19:08:41.972086+0000","last_peered":"2024-09-20T19:08:41.972086+0000","last_clean":"2024-09-20T19:08:41.972086+0000","last_became_active":"2024-09-20T19:04:20.897569+0000","last_became_peered":"2024-09-20T19:04:20.897569+0000","last_unstale":"2024-09-20T19:08:41.972086+0000","last_undegraded":"2024-09-20T19:08:41.972086+0000","last_fullsized":"2024-09-20T19:08:41.972086+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:25.628041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:06:25.628041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"74'79","reported_seq":974,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.973406+0000","last_change":"2024-09-20T19:07:42.989807+0000","last_active":"2024-09-20T19:08:41.973406+0000","last_peered":"2024-09-20T19:08:41.973406+0000","last_clean":"2024-09-20T19:08:41.973406+0000","last_became_active":"2024-09-20T19:07:42.989434+0000","last_became_peered":"2024-09-20T19:07:42.989434+0000","last_unstale":"2024-09-20T19:08:41.973406+0000","last_undegraded":"2024-09-20T19:08:41.973406+0000","last_fullsized":"2024-09-20T19:08:41.973406+0000","mapping_epoch":458,"log_start":"0'0","ondisk_log_start":"0'0","created":455,"last_epoch_clean":459,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:03:14.368279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:03:14.368279+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":21,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":161009,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3","version":"479'6873","reported_seq":7042,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:47.017556+0000","last_change":"2024-09-20T19:08:25.439042+0000","last_active":"2024-09-20T19:08:47.017556+0000","last_peered":"2024-09-20T19:08:47.017556+0000","last_clean":"2024-09-20T19:08:47.017556+0000","last_became_active":"2024-09-20T19:06:56.480422+0000","last_became_peered":"2024-09-20T19:06:56.480422+0000","last_unstale":"2024-09-20T19:08:47.017556+0000","last_undegraded":"2024-09-20T19:08:47.017556+0000","last_fullsized":"2024-09-20T19:08:47.017556+0000","mapping_epoch":422,"log_start":"426'80","ondisk_log_start":"426'80","created":422,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"472'6105","last_scrub_stamp":"2024-09-20T19:08:25.438862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:08:25.438862+0000","objects_scrubbed":2261,"log_size":6793,"log_dups_size":0,"ondisk_log_size":6793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:09:37.377223+0000","scrub_duration":582,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96927744,"num_objects":1479,"num_object_clones":0,"num_object_copies":4437,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1479,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":782,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1],"acting":[6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"273'20","reported_seq":232,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.978493+0000","last_change":"2024-09-20T19:04:20.945143+0000","last_active":"2024-09-20T19:08:41.978493+0000","last_peered":"2024-09-20T19:08:41.978493+0000","last_clean":"2024-09-20T19:08:41.978493+0000","last_became_active":"2024-09-20T19:04:20.896483+0000","last_became_peered":"2024-09-20T19:04:20.896483+0000","last_unstale":"2024-09-20T19:08:41.978493+0000","last_undegraded":"2024-09-20T19:08:41.978493+0000","last_fullsized":"2024-09-20T19:08:41.978493+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'20","last_scrub_stamp":"2024-09-20T19:03:04.282800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:04.282800+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":922,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.980906+0000","last_change":"2024-09-20T19:05:48.678438+0000","last_active":"2024-09-20T19:08:41.980906+0000","last_peered":"2024-09-20T19:08:41.980906+0000","last_clean":"2024-09-20T19:08:41.980906+0000","last_became_active":"2024-09-20T19:04:20.892951+0000","last_became_peered":"2024-09-20T19:04:20.892951+0000","last_unstale":"2024-09-20T19:08:41.980906+0000","last_undegraded":"2024-09-20T19:08:41.980906+0000","last_fullsized":"2024-09-20T19:08:41.980906+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:48.678361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:48.678361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":1137,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.968696+0000","last_change":"2024-09-20T19:05:08.406307+0000","last_active":"2024-09-20T19:08:41.968696+0000","last_peered":"2024-09-20T19:08:41.968696+0000","last_clean":"2024-09-20T19:08:41.968696+0000","last_became_active":"2024-09-20T18:49:49.957005+0000","last_became_peered":"2024-09-20T18:49:49.957005+0000","last_unstale":"2024-09-20T19:08:41.968696+0000","last_undegraded":"2024-09-20T19:08:41.968696+0000","last_fullsized":"2024-09-20T19:08:41.968696+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T19:05:08.406231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:05:08.406231+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":228,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.975794+0000","last_change":"2024-09-20T19:06:56.546180+0000","last_active":"2024-09-20T19:08:41.975794+0000","last_peered":"2024-09-20T19:08:41.975794+0000","last_clean":"2024-09-20T19:08:41.975794+0000","last_became_active":"2024-09-20T19:04:20.884994+0000","last_became_peered":"2024-09-20T19:04:20.884994+0000","last_unstale":"2024-09-20T19:08:41.975794+0000","last_undegraded":"2024-09-20T19:08:41.975794+0000","last_fullsized":"2024-09-20T19:08:41.975794+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:56.546114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:06:56.546114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.4","version":"479'6849","reported_seq":7010,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:46.416926+0000","last_change":"2024-09-20T19:08:35.508989+0000","last_active":"2024-09-20T19:08:46.416926+0000","last_peered":"2024-09-20T19:08:46.416926+0000","last_clean":"2024-09-20T19:08:46.416926+0000","last_became_active":"2024-09-20T19:06:56.462890+0000","last_became_peered":"2024-09-20T19:06:56.462890+0000","last_unstale":"2024-09-20T19:08:46.416926+0000","last_undegraded":"2024-09-20T19:08:46.416926+0000","last_fullsized":"2024-09-20T19:08:46.416926+0000","mapping_epoch":422,"log_start":"431'1406","ondisk_log_start":"431'1406","created":422,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"476'6395","last_scrub_stamp":"2024-09-20T19:08:35.508814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:08:35.508814+0000","objects_scrubbed":1953,"log_size":5443,"log_dups_size":0,"ondisk_log_size":5443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:10:05.298422+0000","scrub_duration":536,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":97386496,"num_objects":1486,"num_object_clones":0,"num_object_copies":4458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1486,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":467,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"274'15","reported_seq":964,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.967651+0000","last_change":"2024-09-20T19:05:57.548758+0000","last_active":"2024-09-20T19:08:41.967651+0000","last_peered":"2024-09-20T19:08:41.967651+0000","last_clean":"2024-09-20T19:08:41.967651+0000","last_became_active":"2024-09-20T19:04:20.906834+0000","last_became_peered":"2024-09-20T19:04:20.906834+0000","last_unstale":"2024-09-20T19:08:41.967651+0000","last_undegraded":"2024-09-20T19:08:41.967651+0000","last_fullsized":"2024-09-20T19:08:41.967651+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'15","last_scrub_stamp":"2024-09-20T19:05:57.548683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:57.548683+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":449,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.972097+0000","last_change":"2024-09-20T19:05:52.464362+0000","last_active":"2024-09-20T19:08:41.972097+0000","last_peered":"2024-09-20T19:08:41.972097+0000","last_clean":"2024-09-20T19:08:41.972097+0000","last_became_active":"2024-09-20T19:00:01.190875+0000","last_became_peered":"2024-09-20T19:00:01.190875+0000","last_unstale":"2024-09-20T19:08:41.972097+0000","last_undegraded":"2024-09-20T19:08:41.972097+0000","last_fullsized":"2024-09-20T19:08:41.972097+0000","mapping_epoch":277,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":278,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:52.464261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:52.464261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":917,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.972459+0000","last_change":"2024-09-20T19:04:20.889614+0000","last_active":"2024-09-20T19:08:41.972459+0000","last_peered":"2024-09-20T19:08:41.972459+0000","last_clean":"2024-09-20T19:08:41.972459+0000","last_became_active":"2024-09-20T19:04:20.889273+0000","last_became_peered":"2024-09-20T19:04:20.889273+0000","last_unstale":"2024-09-20T19:08:41.972459+0000","last_undegraded":"2024-09-20T19:08:41.972459+0000","last_fullsized":"2024-09-20T19:08:41.972459+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:03.389611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:03.389611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":878,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.965339+0000","last_change":"2024-09-20T19:02:31.120809+0000","last_active":"2024-09-20T19:08:41.965339+0000","last_peered":"2024-09-20T19:08:41.965339+0000","last_clean":"2024-09-20T19:08:41.965339+0000","last_became_active":"2024-09-20T18:49:51.964098+0000","last_became_peered":"2024-09-20T18:49:51.964098+0000","last_unstale":"2024-09-20T19:08:41.965339+0000","last_undegraded":"2024-09-20T19:08:41.965339+0000","last_fullsized":"2024-09-20T19:08:41.965339+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:31.120623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:02:31.120623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.5","version":"479'6924","reported_seq":7088,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:46.438286+0000","last_change":"2024-09-20T19:06:57.022784+0000","last_active":"2024-09-20T19:08:46.438286+0000","last_peered":"2024-09-20T19:08:46.438286+0000","last_clean":"2024-09-20T19:08:46.438286+0000","last_became_active":"2024-09-20T19:06:56.460858+0000","last_became_peered":"2024-09-20T19:06:56.460858+0000","last_unstale":"2024-09-20T19:08:46.438286+0000","last_undegraded":"2024-09-20T19:08:46.438286+0000","last_fullsized":"2024-09-20T19:08:46.438286+0000","mapping_epoch":422,"log_start":"433'1681","ondisk_log_start":"433'1681","created":422,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"442'3824","last_scrub_stamp":"2024-09-20T19:06:57.022671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:57.022671+0000","objects_scrubbed":1982,"log_size":5243,"log_dups_size":0,"ondisk_log_size":5243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":299,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":1536,"num_object_clones":0,"num_object_copies":4608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3106,"num_write_kb":85120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"274'35","reported_seq":520,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.966507+0000","last_change":"2024-09-20T19:06:19.915841+0000","last_active":"2024-09-20T19:08:41.966507+0000","last_peered":"2024-09-20T19:08:41.966507+0000","last_clean":"2024-09-20T19:08:41.966507+0000","last_became_active":"2024-09-20T18:59:05.087391+0000","last_became_peered":"2024-09-20T18:59:05.087391+0000","last_unstale":"2024-09-20T19:08:41.966507+0000","last_undegraded":"2024-09-20T19:08:41.966507+0000","last_fullsized":"2024-09-20T19:08:41.966507+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'35","last_scrub_stamp":"2024-09-20T19:06:19.915767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:06:19.915767+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":912,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.972532+0000","last_change":"2024-09-20T19:05:49.763478+0000","last_active":"2024-09-20T19:08:41.972532+0000","last_peered":"2024-09-20T19:08:41.972532+0000","last_clean":"2024-09-20T19:08:41.972532+0000","last_became_active":"2024-09-20T18:59:05.086670+0000","last_became_peered":"2024-09-20T18:59:05.086670+0000","last_unstale":"2024-09-20T19:08:41.972532+0000","last_undegraded":"2024-09-20T19:08:41.972532+0000","last_fullsized":"2024-09-20T19:08:41.972532+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:49.763407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:49.763407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":900,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.970058+0000","last_change":"2024-09-20T19:05:11.444197+0000","last_active":"2024-09-20T19:08:41.970058+0000","last_peered":"2024-09-20T19:08:41.970058+0000","last_clean":"2024-09-20T19:08:41.970058+0000","last_became_active":"2024-09-20T18:49:49.957558+0000","last_became_peered":"2024-09-20T18:49:49.957558+0000","last_unstale":"2024-09-20T19:08:41.970058+0000","last_undegraded":"2024-09-20T19:08:41.970058+0000","last_fullsized":"2024-09-20T19:08:41.970058+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:11.444143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:05:11.444143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":870,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.972801+0000","last_change":"2024-09-20T18:53:47.370152+0000","last_active":"2024-09-20T19:08:41.972801+0000","last_peered":"2024-09-20T19:08:41.972801+0000","last_clean":"2024-09-20T19:08:41.972801+0000","last_became_active":"2024-09-20T18:49:51.970798+0000","last_became_peered":"2024-09-20T18:49:51.970798+0000","last_unstale":"2024-09-20T19:08:41.972801+0000","last_undegraded":"2024-09-20T19:08:41.972801+0000","last_fullsized":"2024-09-20T19:08:41.972801+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:53:47.370108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:53:47.370108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.6","version":"479'6978","reported_seq":7140,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:48.081746+0000","last_change":"2024-09-20T19:06:58.182942+0000","last_active":"2024-09-20T19:08:48.081746+0000","last_peered":"2024-09-20T19:08:48.081746+0000","last_clean":"2024-09-20T19:08:48.081746+0000","last_became_active":"2024-09-20T19:06:56.472642+0000","last_became_peered":"2024-09-20T19:06:56.472642+0000","last_unstale":"2024-09-20T19:08:48.081746+0000","last_undegraded":"2024-09-20T19:08:48.081746+0000","last_fullsized":"2024-09-20T19:08:48.081746+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"443'3869","last_scrub_stamp":"2024-09-20T19:06:58.182865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:58.182865+0000","objects_scrubbed":1953,"log_size":6978,"log_dups_size":0,"ondisk_log_size":6978,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":217,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92798976,"num_objects":1416,"num_object_clones":0,"num_object_copies":4248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1416,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3109,"num_write_kb":82304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"273'18","reported_seq":233,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.975639+0000","last_change":"2024-09-20T19:04:20.936465+0000","last_active":"2024-09-20T19:08:41.975639+0000","last_peered":"2024-09-20T19:08:41.975639+0000","last_clean":"2024-09-20T19:08:41.975639+0000","last_became_active":"2024-09-20T19:04:20.888905+0000","last_became_peered":"2024-09-20T19:04:20.888905+0000","last_unstale":"2024-09-20T19:08:41.975639+0000","last_undegraded":"2024-09-20T19:08:41.975639+0000","last_fullsized":"2024-09-20T19:08:41.975639+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'18","last_scrub_stamp":"2024-09-20T19:03:13.201592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:03:13.201592+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":931,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.975971+0000","last_change":"2024-09-20T19:06:15.692558+0000","last_active":"2024-09-20T19:08:41.975971+0000","last_peered":"2024-09-20T19:08:41.975971+0000","last_clean":"2024-09-20T19:08:41.975971+0000","last_became_active":"2024-09-20T19:04:20.892429+0000","last_became_peered":"2024-09-20T19:04:20.892429+0000","last_unstale":"2024-09-20T19:08:41.975971+0000","last_undegraded":"2024-09-20T19:08:41.975971+0000","last_fullsized":"2024-09-20T19:08:41.975971+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:15.692508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:06:15.692508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":892,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.968345+0000","last_change":"2024-09-20T19:04:48.406445+0000","last_active":"2024-09-20T19:08:41.968345+0000","last_peered":"2024-09-20T19:08:41.968345+0000","last_clean":"2024-09-20T19:08:41.968345+0000","last_became_active":"2024-09-20T18:49:49.954944+0000","last_became_peered":"2024-09-20T18:49:49.954944+0000","last_unstale":"2024-09-20T19:08:41.968345+0000","last_undegraded":"2024-09-20T19:08:41.968345+0000","last_fullsized":"2024-09-20T19:08:41.968345+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:48.406373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:48.406373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":885,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.980596+0000","last_change":"2024-09-20T19:05:43.657860+0000","last_active":"2024-09-20T19:08:41.980596+0000","last_peered":"2024-09-20T19:08:41.980596+0000","last_clean":"2024-09-20T19:08:41.980596+0000","last_became_active":"2024-09-20T18:49:51.972970+0000","last_became_peered":"2024-09-20T18:49:51.972970+0000","last_unstale":"2024-09-20T19:08:41.980596+0000","last_undegraded":"2024-09-20T19:08:41.980596+0000","last_fullsized":"2024-09-20T19:08:41.980596+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:43.657780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:05:43.657780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"479'6852","reported_seq":7021,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:45.758963+0000","last_change":"2024-09-20T19:06:57.243186+0000","last_active":"2024-09-20T19:08:45.758963+0000","last_peered":"2024-09-20T19:08:45.758963+0000","last_clean":"2024-09-20T19:08:45.758963+0000","last_became_active":"2024-09-20T19:06:56.479387+0000","last_became_peered":"2024-09-20T19:06:56.479387+0000","last_unstale":"2024-09-20T19:08:45.758963+0000","last_undegraded":"2024-09-20T19:08:45.758963+0000","last_fullsized":"2024-09-20T19:08:45.758963+0000","mapping_epoch":422,"log_start":"0'0","ondisk_log_start":"0'0","created":422,"last_epoch_clean":442,"parent":"0.0","parent_split_bits":0,"last_scrub":"442'3847","last_scrub_stamp":"2024-09-20T19:06:57.243081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:06:57.243081+0000","objects_scrubbed":1913,"log_size":6852,"log_dups_size":0,"ondisk_log_size":6852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":380,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":103219200,"num_objects":1575,"num_object_clones":0,"num_object_copies":4725,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3012,"num_write_kb":85568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"273'27","reported_seq":990,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.977382+0000","last_change":"2024-09-20T19:05:49.777488+0000","last_active":"2024-09-20T19:08:41.977382+0000","last_peered":"2024-09-20T19:08:41.977382+0000","last_clean":"2024-09-20T19:08:41.977382+0000","last_became_active":"2024-09-20T18:59:05.083726+0000","last_became_peered":"2024-09-20T18:59:05.083726+0000","last_unstale":"2024-09-20T19:08:41.977382+0000","last_undegraded":"2024-09-20T19:08:41.977382+0000","last_fullsized":"2024-09-20T19:08:41.977382+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'27","last_scrub_stamp":"2024-09-20T19:05:49.777414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:05:49.777414+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":963,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.974191+0000","last_change":"2024-09-20T19:06:25.718817+0000","last_active":"2024-09-20T19:08:41.974191+0000","last_peered":"2024-09-20T19:08:41.974191+0000","last_clean":"2024-09-20T19:08:41.974191+0000","last_became_active":"2024-09-20T19:04:20.884854+0000","last_became_peered":"2024-09-20T19:04:20.884854+0000","last_unstale":"2024-09-20T19:08:41.974191+0000","last_undegraded":"2024-09-20T19:08:41.974191+0000","last_fullsized":"2024-09-20T19:08:41.974191+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-20T19:06:25.718731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:06:25.718731+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"375'14","reported_seq":1176,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.973353+0000","last_change":"2024-09-20T19:06:14.693176+0000","last_active":"2024-09-20T19:08:41.973353+0000","last_peered":"2024-09-20T19:08:41.973353+0000","last_clean":"2024-09-20T19:08:41.973353+0000","last_became_active":"2024-09-20T19:04:20.892956+0000","last_became_peered":"2024-09-20T19:04:20.892956+0000","last_unstale":"2024-09-20T19:08:41.973353+0000","last_undegraded":"2024-09-20T19:08:41.973353+0000","last_fullsized":"2024-09-20T19:08:41.973353+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"375'14","last_scrub_stamp":"2024-09-20T19:06:14.693111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:06:14.693111+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":899,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.977737+0000","last_change":"2024-09-20T19:03:06.888620+0000","last_active":"2024-09-20T19:08:41.977737+0000","last_peered":"2024-09-20T19:08:41.977737+0000","last_clean":"2024-09-20T19:08:41.977737+0000","last_became_active":"2024-09-20T18:59:05.087005+0000","last_became_peered":"2024-09-20T18:59:05.087005+0000","last_unstale":"2024-09-20T19:08:41.977737+0000","last_undegraded":"2024-09-20T19:08:41.977737+0000","last_fullsized":"2024-09-20T19:08:41.977737+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:06.888563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:03:06.888563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.18","version":"479'6686","reported_seq":12361,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:46.853179+0000","last_change":"2024-09-20T19:08:36.601045+0000","last_active":"2024-09-20T19:08:46.853179+0000","last_peered":"2024-09-20T19:08:46.853179+0000","last_clean":"2024-09-20T19:08:46.853179+0000","last_became_active":"2024-09-20T19:07:42.984159+0000","last_became_peered":"2024-09-20T19:07:42.984159+0000","last_unstale":"2024-09-20T19:08:46.853179+0000","last_undegraded":"2024-09-20T19:08:46.853179+0000","last_fullsized":"2024-09-20T19:08:46.853179+0000","mapping_epoch":458,"log_start":"0'0","ondisk_log_start":"0'0","created":441,"last_epoch_clean":459,"parent":"0.0","parent_split_bits":5,"last_scrub":"476'6268","last_scrub_stamp":"2024-09-20T19:08:36.600754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:08:36.600754+0000","objects_scrubbed":1946,"log_size":6686,"log_dups_size":0,"ondisk_log_size":6686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:09:50.593188+0000","scrub_duration":496,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":99680256,"num_objects":1521,"num_object_clones":0,"num_object_copies":4563,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":425,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":468,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.971529+0000","last_change":"2024-09-20T19:08:37.897300+0000","last_active":"2024-09-20T19:08:41.971529+0000","last_peered":"2024-09-20T19:08:41.971529+0000","last_clean":"2024-09-20T19:08:41.971529+0000","last_became_active":"2024-09-20T18:59:05.085427+0000","last_became_peered":"2024-09-20T18:59:05.085427+0000","last_unstale":"2024-09-20T19:08:41.971529+0000","last_undegraded":"2024-09-20T19:08:41.971529+0000","last_fullsized":"2024-09-20T19:08:41.971529+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:08:37.897242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:08:37.897242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:09:47.145217+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"273'12","reported_seq":918,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.981595+0000","last_change":"2024-09-20T19:06:19.973831+0000","last_active":"2024-09-20T19:08:41.981595+0000","last_peered":"2024-09-20T19:08:41.981595+0000","last_clean":"2024-09-20T19:08:41.981595+0000","last_became_active":"2024-09-20T18:49:48.500662+0000","last_became_peered":"2024-09-20T18:49:48.500662+0000","last_unstale":"2024-09-20T19:08:41.981595+0000","last_undegraded":"2024-09-20T19:08:41.981595+0000","last_fullsized":"2024-09-20T19:08:41.981595+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'12","last_scrub_stamp":"2024-09-20T19:06:19.973765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:06:19.973765+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":888,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.979057+0000","last_change":"2024-09-20T19:05:22.605885+0000","last_active":"2024-09-20T19:08:41.979057+0000","last_peered":"2024-09-20T19:08:41.979057+0000","last_clean":"2024-09-20T19:08:41.979057+0000","last_became_active":"2024-09-20T18:49:49.956749+0000","last_became_peered":"2024-09-20T18:49:49.956749+0000","last_unstale":"2024-09-20T19:08:41.979057+0000","last_undegraded":"2024-09-20T19:08:41.979057+0000","last_fullsized":"2024-09-20T19:08:41.979057+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:22.605839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:05:22.605839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":924,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.971902+0000","last_change":"2024-09-20T19:05:56.761494+0000","last_active":"2024-09-20T19:08:41.971902+0000","last_peered":"2024-09-20T19:08:41.971902+0000","last_clean":"2024-09-20T19:08:41.971902+0000","last_became_active":"2024-09-20T19:04:20.914115+0000","last_became_peered":"2024-09-20T19:04:20.914115+0000","last_unstale":"2024-09-20T19:08:41.971902+0000","last_undegraded":"2024-09-20T19:08:41.971902+0000","last_fullsized":"2024-09-20T19:08:41.971902+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:56.761425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:05:56.761425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.19","version":"479'6989","reported_seq":12980,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:48.110007+0000","last_change":"2024-09-20T19:08:35.994471+0000","last_active":"2024-09-20T19:08:48.110007+0000","last_peered":"2024-09-20T19:08:48.110007+0000","last_clean":"2024-09-20T19:08:48.110007+0000","last_became_active":"2024-09-20T19:08:33.707474+0000","last_became_peered":"2024-09-20T19:08:33.707474+0000","last_unstale":"2024-09-20T19:08:48.110007+0000","last_undegraded":"2024-09-20T19:08:48.110007+0000","last_fullsized":"2024-09-20T19:08:48.110007+0000","mapping_epoch":474,"log_start":"0'0","ondisk_log_start":"0'0","created":441,"last_epoch_clean":475,"parent":"0.0","parent_split_bits":5,"last_scrub":"476'6501","last_scrub_stamp":"2024-09-20T19:08:35.994184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:08:35.994184+0000","objects_scrubbed":1935,"log_size":6989,"log_dups_size":0,"ondisk_log_size":6989,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:09:55.533316+0000","scrub_duration":530,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":94306304,"num_objects":1439,"num_object_clones":0,"num_object_copies":4317,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1439,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":496,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"273'15","reported_seq":910,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.965145+0000","last_change":"2024-09-20T19:08:38.529367+0000","last_active":"2024-09-20T19:08:41.965145+0000","last_peered":"2024-09-20T19:08:41.965145+0000","last_clean":"2024-09-20T19:08:41.965145+0000","last_became_active":"2024-09-20T18:49:48.500803+0000","last_became_peered":"2024-09-20T18:49:48.500803+0000","last_unstale":"2024-09-20T19:08:41.965145+0000","last_undegraded":"2024-09-20T19:08:41.965145+0000","last_fullsized":"2024-09-20T19:08:41.965145+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:08:38.529296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:08:38.529296+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:10:07.524513+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":929,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.973666+0000","last_change":"2024-09-20T19:04:47.713764+0000","last_active":"2024-09-20T19:08:41.973666+0000","last_peered":"2024-09-20T19:08:41.973666+0000","last_clean":"2024-09-20T19:08:41.973666+0000","last_became_active":"2024-09-20T19:04:20.893250+0000","last_became_peered":"2024-09-20T19:04:20.893250+0000","last_unstale":"2024-09-20T19:08:41.973666+0000","last_undegraded":"2024-09-20T19:08:41.973666+0000","last_fullsized":"2024-09-20T19:08:41.973666+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:47.713585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:04:47.713585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":888,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.981518+0000","last_change":"2024-09-20T19:04:50.573619+0000","last_active":"2024-09-20T19:08:41.981518+0000","last_peered":"2024-09-20T19:08:41.981518+0000","last_clean":"2024-09-20T19:08:41.981518+0000","last_became_active":"2024-09-20T18:49:49.961947+0000","last_became_peered":"2024-09-20T18:49:49.961947+0000","last_unstale":"2024-09-20T19:08:41.981518+0000","last_undegraded":"2024-09-20T19:08:41.981518+0000","last_fullsized":"2024-09-20T19:08:41.981518+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:04:50.573550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:50.573550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"66'1","reported_seq":930,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.972186+0000","last_change":"2024-09-20T19:04:20.913449+0000","last_active":"2024-09-20T19:08:41.972186+0000","last_peered":"2024-09-20T19:08:41.972186+0000","last_clean":"2024-09-20T19:08:41.972186+0000","last_became_active":"2024-09-20T19:04:20.892236+0000","last_became_peered":"2024-09-20T19:04:20.892236+0000","last_unstale":"2024-09-20T19:08:41.972186+0000","last_undegraded":"2024-09-20T19:08:41.972186+0000","last_fullsized":"2024-09-20T19:08:41.972186+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T19:03:12.499684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:03:12.499684+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"274'15","reported_seq":909,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.979620+0000","last_change":"2024-09-20T19:06:42.967192+0000","last_active":"2024-09-20T19:08:41.979620+0000","last_peered":"2024-09-20T19:08:41.979620+0000","last_clean":"2024-09-20T19:08:41.979620+0000","last_became_active":"2024-09-20T18:49:48.501011+0000","last_became_peered":"2024-09-20T18:49:48.501011+0000","last_unstale":"2024-09-20T19:08:41.979620+0000","last_undegraded":"2024-09-20T19:08:41.979620+0000","last_fullsized":"2024-09-20T19:08:41.979620+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'15","last_scrub_stamp":"2024-09-20T19:06:42.967003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:06:42.967003+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":890,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.965483+0000","last_change":"2024-09-20T19:01:30.864866+0000","last_active":"2024-09-20T19:08:41.965483+0000","last_peered":"2024-09-20T19:08:41.965483+0000","last_clean":"2024-09-20T19:08:41.965483+0000","last_became_active":"2024-09-20T18:49:45.938511+0000","last_became_peered":"2024-09-20T18:49:45.938511+0000","last_unstale":"2024-09-20T19:08:41.965483+0000","last_undegraded":"2024-09-20T19:08:41.965483+0000","last_fullsized":"2024-09-20T19:08:41.965483+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:30.864794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:01:30.864794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":1125,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.980890+0000","last_change":"2024-09-20T19:04:41.601208+0000","last_active":"2024-09-20T19:08:41.980890+0000","last_peered":"2024-09-20T19:08:41.980890+0000","last_clean":"2024-09-20T19:08:41.980890+0000","last_became_active":"2024-09-20T18:49:49.960550+0000","last_became_peered":"2024-09-20T18:49:49.960550+0000","last_unstale":"2024-09-20T19:08:41.980890+0000","last_undegraded":"2024-09-20T19:08:41.980890+0000","last_fullsized":"2024-09-20T19:08:41.980890+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T19:04:41.601073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:04:41.601073+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":228,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.978098+0000","last_change":"2024-09-20T19:08:41.857818+0000","last_active":"2024-09-20T19:08:41.978098+0000","last_peered":"2024-09-20T19:08:41.978098+0000","last_clean":"2024-09-20T19:08:41.978098+0000","last_became_active":"2024-09-20T19:04:20.884100+0000","last_became_peered":"2024-09-20T19:04:20.884100+0000","last_unstale":"2024-09-20T19:08:41.978098+0000","last_undegraded":"2024-09-20T19:08:41.978098+0000","last_fullsized":"2024-09-20T19:08:41.978098+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:08:41.857755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:08:41.857755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:09:53.622385+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"274'20","reported_seq":918,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.980946+0000","last_change":"2024-09-20T18:57:20.701592+0000","last_active":"2024-09-20T19:08:41.980946+0000","last_peered":"2024-09-20T19:08:41.980946+0000","last_clean":"2024-09-20T19:08:41.980946+0000","last_became_active":"2024-09-20T18:49:58.997494+0000","last_became_peered":"2024-09-20T18:49:58.997494+0000","last_unstale":"2024-09-20T19:08:41.980946+0000","last_undegraded":"2024-09-20T19:08:41.980946+0000","last_fullsized":"2024-09-20T19:08:41.980946+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-20T18:57:20.701543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T18:57:20.701543+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":907,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.972300+0000","last_change":"2024-09-20T19:04:20.884940+0000","last_active":"2024-09-20T19:08:41.972300+0000","last_peered":"2024-09-20T19:08:41.972300+0000","last_clean":"2024-09-20T19:08:41.972300+0000","last_became_active":"2024-09-20T19:04:20.884582+0000","last_became_peered":"2024-09-20T19:04:20.884582+0000","last_unstale":"2024-09-20T19:08:41.972300+0000","last_undegraded":"2024-09-20T19:08:41.972300+0000","last_fullsized":"2024-09-20T19:08:41.972300+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:54:57.632380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T18:54:57.632380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":1124,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.979354+0000","last_change":"2024-09-20T19:06:26.893183+0000","last_active":"2024-09-20T19:08:41.979354+0000","last_peered":"2024-09-20T19:08:41.979354+0000","last_clean":"2024-09-20T19:08:41.979354+0000","last_became_active":"2024-09-20T18:49:49.962243+0000","last_became_peered":"2024-09-20T18:49:49.962243+0000","last_unstale":"2024-09-20T19:08:41.979354+0000","last_undegraded":"2024-09-20T19:08:41.979354+0000","last_fullsized":"2024-09-20T19:08:41.979354+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T19:06:26.893109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:06:26.893109+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":893,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.964685+0000","last_change":"2024-09-20T19:05:52.578042+0000","last_active":"2024-09-20T19:08:41.964685+0000","last_peered":"2024-09-20T19:08:41.964685+0000","last_clean":"2024-09-20T19:08:41.964685+0000","last_became_active":"2024-09-20T18:49:51.969093+0000","last_became_peered":"2024-09-20T18:49:51.969093+0000","last_unstale":"2024-09-20T19:08:41.964685+0000","last_undegraded":"2024-09-20T19:08:41.964685+0000","last_fullsized":"2024-09-20T19:08:41.964685+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:52.577979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:05:52.577979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"273'19","reported_seq":240,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.977819+0000","last_change":"2024-09-20T19:06:44.432152+0000","last_active":"2024-09-20T19:08:41.977819+0000","last_peered":"2024-09-20T19:08:41.977819+0000","last_clean":"2024-09-20T19:08:41.977819+0000","last_became_active":"2024-09-20T19:04:20.897330+0000","last_became_peered":"2024-09-20T19:04:20.897330+0000","last_unstale":"2024-09-20T19:08:41.977819+0000","last_undegraded":"2024-09-20T19:08:41.977819+0000","last_fullsized":"2024-09-20T19:08:41.977819+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'19","last_scrub_stamp":"2024-09-20T19:06:44.432058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:06:44.432058+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":228,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.978588+0000","last_change":"2024-09-20T19:08:39.923229+0000","last_active":"2024-09-20T19:08:41.978588+0000","last_peered":"2024-09-20T19:08:41.978588+0000","last_clean":"2024-09-20T19:08:41.978588+0000","last_became_active":"2024-09-20T19:04:20.896007+0000","last_became_peered":"2024-09-20T19:04:20.896007+0000","last_unstale":"2024-09-20T19:08:41.978588+0000","last_undegraded":"2024-09-20T19:08:41.978588+0000","last_fullsized":"2024-09-20T19:08:41.978588+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:08:39.923078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:08:39.923078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:10:08.391342+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":881,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.974176+0000","last_change":"2024-09-20T19:02:38.906352+0000","last_active":"2024-09-20T19:08:41.974176+0000","last_peered":"2024-09-20T19:08:41.974176+0000","last_clean":"2024-09-20T19:08:41.974176+0000","last_became_active":"2024-09-20T18:49:49.957067+0000","last_became_peered":"2024-09-20T18:49:49.957067+0000","last_unstale":"2024-09-20T19:08:41.974176+0000","last_undegraded":"2024-09-20T19:08:41.974176+0000","last_fullsized":"2024-09-20T19:08:41.974176+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:38.906161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:38.906161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":861,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.981929+0000","last_change":"2024-09-20T18:57:16.789412+0000","last_active":"2024-09-20T19:08:41.981929+0000","last_peered":"2024-09-20T19:08:41.981929+0000","last_clean":"2024-09-20T19:08:41.981929+0000","last_became_active":"2024-09-20T18:49:51.966915+0000","last_became_peered":"2024-09-20T18:49:51.966915+0000","last_unstale":"2024-09-20T19:08:41.981929+0000","last_undegraded":"2024-09-20T19:08:41.981929+0000","last_fullsized":"2024-09-20T19:08:41.981929+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T18:57:16.789095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T18:57:16.789095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"273'15","reported_seq":960,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.974332+0000","last_change":"2024-09-20T19:06:51.012046+0000","last_active":"2024-09-20T19:08:41.974332+0000","last_peered":"2024-09-20T19:08:41.974332+0000","last_clean":"2024-09-20T19:08:41.974332+0000","last_became_active":"2024-09-20T18:59:05.085230+0000","last_became_peered":"2024-09-20T18:59:05.085230+0000","last_unstale":"2024-09-20T19:08:41.974332+0000","last_undegraded":"2024-09-20T19:08:41.974332+0000","last_fullsized":"2024-09-20T19:08:41.974332+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'15","last_scrub_stamp":"2024-09-20T19:06:51.011969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:06:51.011969+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":906,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.972292+0000","last_change":"2024-09-20T19:04:20.885127+0000","last_active":"2024-09-20T19:08:41.972292+0000","last_peered":"2024-09-20T19:08:41.972292+0000","last_clean":"2024-09-20T19:08:41.972292+0000","last_became_active":"2024-09-20T19:04:20.884660+0000","last_became_peered":"2024-09-20T19:04:20.884660+0000","last_unstale":"2024-09-20T19:08:41.972292+0000","last_undegraded":"2024-09-20T19:08:41.972292+0000","last_fullsized":"2024-09-20T19:08:41.972292+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:03:07.408197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:03:07.408197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":897,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.980679+0000","last_change":"2024-09-20T19:06:22.930775+0000","last_active":"2024-09-20T19:08:41.980679+0000","last_peered":"2024-09-20T19:08:41.980679+0000","last_clean":"2024-09-20T19:08:41.980679+0000","last_became_active":"2024-09-20T18:49:49.962523+0000","last_became_peered":"2024-09-20T18:49:49.962523+0000","last_unstale":"2024-09-20T19:08:41.980679+0000","last_undegraded":"2024-09-20T19:08:41.980679+0000","last_fullsized":"2024-09-20T19:08:41.980679+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:22.930701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:06:22.930701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":886,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.964433+0000","last_change":"2024-09-20T19:05:51.626774+0000","last_active":"2024-09-20T19:08:41.964433+0000","last_peered":"2024-09-20T19:08:41.964433+0000","last_clean":"2024-09-20T19:08:41.964433+0000","last_became_active":"2024-09-20T18:49:51.967313+0000","last_became_peered":"2024-09-20T18:49:51.967313+0000","last_unstale":"2024-09-20T19:08:41.964433+0000","last_undegraded":"2024-09-20T19:08:41.964433+0000","last_fullsized":"2024-09-20T19:08:41.964433+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:51.626697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:05:51.626697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"274'19","reported_seq":954,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.981829+0000","last_change":"2024-09-20T19:01:05.495603+0000","last_active":"2024-09-20T19:08:41.981829+0000","last_peered":"2024-09-20T19:08:41.981829+0000","last_clean":"2024-09-20T19:08:41.981829+0000","last_became_active":"2024-09-20T18:59:05.075390+0000","last_became_peered":"2024-09-20T18:59:05.075390+0000","last_unstale":"2024-09-20T19:08:41.981829+0000","last_undegraded":"2024-09-20T19:08:41.981829+0000","last_fullsized":"2024-09-20T19:08:41.981829+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'19","last_scrub_stamp":"2024-09-20T19:01:05.495478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:01:05.495478+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":904,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.979131+0000","last_change":"2024-09-20T19:05:43.686902+0000","last_active":"2024-09-20T19:08:41.979131+0000","last_peered":"2024-09-20T19:08:41.979131+0000","last_clean":"2024-09-20T19:08:41.979131+0000","last_became_active":"2024-09-20T18:49:45.942105+0000","last_became_peered":"2024-09-20T18:49:45.942105+0000","last_unstale":"2024-09-20T19:08:41.979131+0000","last_undegraded":"2024-09-20T19:08:41.979131+0000","last_fullsized":"2024-09-20T19:08:41.979131+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:43.686842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:05:43.686842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":921,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.971787+0000","last_change":"2024-09-20T19:04:20.912120+0000","last_active":"2024-09-20T19:08:41.971787+0000","last_peered":"2024-09-20T19:08:41.971787+0000","last_clean":"2024-09-20T19:08:41.971787+0000","last_became_active":"2024-09-20T19:04:20.911292+0000","last_became_peered":"2024-09-20T19:04:20.911292+0000","last_unstale":"2024-09-20T19:08:41.971787+0000","last_undegraded":"2024-09-20T19:08:41.971787+0000","last_fullsized":"2024-09-20T19:08:41.971787+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:02:27.267254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:02:27.267254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":899,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.980812+0000","last_change":"2024-09-20T19:06:58.896885+0000","last_active":"2024-09-20T19:08:41.980812+0000","last_peered":"2024-09-20T19:08:41.980812+0000","last_clean":"2024-09-20T19:08:41.980812+0000","last_became_active":"2024-09-20T18:59:05.081179+0000","last_became_peered":"2024-09-20T18:59:05.081179+0000","last_unstale":"2024-09-20T19:08:41.980812+0000","last_undegraded":"2024-09-20T19:08:41.980812+0000","last_fullsized":"2024-09-20T19:08:41.980812+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:58.896826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:06:58.896826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"273'8","reported_seq":939,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.980409+0000","last_change":"2024-09-20T19:06:21.974151+0000","last_active":"2024-09-20T19:08:41.980409+0000","last_peered":"2024-09-20T19:08:41.980409+0000","last_clean":"2024-09-20T19:08:41.980409+0000","last_became_active":"2024-09-20T18:59:05.076365+0000","last_became_peered":"2024-09-20T18:59:05.076365+0000","last_unstale":"2024-09-20T19:08:41.980409+0000","last_undegraded":"2024-09-20T19:08:41.980409+0000","last_fullsized":"2024-09-20T19:08:41.980409+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'8","last_scrub_stamp":"2024-09-20T19:06:21.974084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:06:21.974084+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":905,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.964510+0000","last_change":"2024-09-20T19:06:45.833934+0000","last_active":"2024-09-20T19:08:41.964510+0000","last_peered":"2024-09-20T19:08:41.964510+0000","last_clean":"2024-09-20T19:08:41.964510+0000","last_became_active":"2024-09-20T18:49:45.940988+0000","last_became_peered":"2024-09-20T18:49:45.940988+0000","last_unstale":"2024-09-20T19:08:41.964510+0000","last_undegraded":"2024-09-20T19:08:41.964510+0000","last_fullsized":"2024-09-20T19:08:41.964510+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:45.833872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:06:45.833872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":493,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.971268+0000","last_change":"2024-09-20T19:08:39.923310+0000","last_active":"2024-09-20T19:08:41.971268+0000","last_peered":"2024-09-20T19:08:41.971268+0000","last_clean":"2024-09-20T19:08:41.971268+0000","last_became_active":"2024-09-20T19:04:20.900214+0000","last_became_peered":"2024-09-20T19:04:20.900214+0000","last_unstale":"2024-09-20T19:08:41.971268+0000","last_undegraded":"2024-09-20T19:08:41.971268+0000","last_fullsized":"2024-09-20T19:08:41.971268+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:08:39.923163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:08:39.923163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:10:04.377752+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":903,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.973216+0000","last_change":"2024-09-20T19:05:57.789513+0000","last_active":"2024-09-20T19:08:41.973216+0000","last_peered":"2024-09-20T19:08:41.973216+0000","last_clean":"2024-09-20T19:08:41.973216+0000","last_became_active":"2024-09-20T19:04:20.885211+0000","last_became_peered":"2024-09-20T19:04:20.885211+0000","last_unstale":"2024-09-20T19:08:41.973216+0000","last_undegraded":"2024-09-20T19:08:41.973216+0000","last_fullsized":"2024-09-20T19:08:41.973216+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:05:57.789465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:05:57.789465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.10","version":"479'6938","reported_seq":14777,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:48.115427+0000","last_change":"2024-09-20T19:08:33.756228+0000","last_active":"2024-09-20T19:08:48.115427+0000","last_peered":"2024-09-20T19:08:48.115427+0000","last_clean":"2024-09-20T19:08:48.115427+0000","last_became_active":"2024-09-20T19:08:24.341564+0000","last_became_peered":"2024-09-20T19:08:24.341564+0000","last_unstale":"2024-09-20T19:08:48.115427+0000","last_undegraded":"2024-09-20T19:08:48.115427+0000","last_fullsized":"2024-09-20T19:08:48.115427+0000","mapping_epoch":471,"log_start":"0'0","ondisk_log_start":"0'0","created":441,"last_epoch_clean":472,"parent":"0.0","parent_split_bits":5,"last_scrub":"475'6370","last_scrub_stamp":"2024-09-20T19:08:33.755937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T19:06:17.033924+0000","last_clean_scrub_stamp":"2024-09-20T19:08:33.755937+0000","objects_scrubbed":1992,"log_size":6938,"log_dups_size":0,"ondisk_log_size":6938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:09:35.881668+0000","scrub_duration":282,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":93257728,"num_objects":1423,"num_object_clones":0,"num_object_copies":4269,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1423,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":569,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":885,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.975013+0000","last_change":"2024-09-20T19:06:24.854187+0000","last_active":"2024-09-20T19:08:41.975013+0000","last_peered":"2024-09-20T19:08:41.975013+0000","last_clean":"2024-09-20T19:08:41.975013+0000","last_became_active":"2024-09-20T19:04:20.890101+0000","last_became_peered":"2024-09-20T19:04:20.890101+0000","last_unstale":"2024-09-20T19:08:41.975013+0000","last_undegraded":"2024-09-20T19:08:41.975013+0000","last_fullsized":"2024-09-20T19:08:41.975013+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:24.854124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:06:24.854124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":888,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.981617+0000","last_change":"2024-09-20T19:06:44.934346+0000","last_active":"2024-09-20T19:08:41.981617+0000","last_peered":"2024-09-20T19:08:41.981617+0000","last_clean":"2024-09-20T19:08:41.981617+0000","last_became_active":"2024-09-20T18:49:45.937992+0000","last_became_peered":"2024-09-20T18:49:45.937992+0000","last_unstale":"2024-09-20T19:08:41.981617+0000","last_undegraded":"2024-09-20T19:08:41.981617+0000","last_fullsized":"2024-09-20T19:08:41.981617+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:06:44.934143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:06:44.934143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"274'19","reported_seq":492,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.966375+0000","last_change":"2024-09-20T19:06:26.909675+0000","last_active":"2024-09-20T19:08:41.966375+0000","last_peered":"2024-09-20T19:08:41.966375+0000","last_clean":"2024-09-20T19:08:41.966375+0000","last_became_active":"2024-09-20T18:59:05.089294+0000","last_became_peered":"2024-09-20T18:59:05.089294+0000","last_unstale":"2024-09-20T19:08:41.966375+0000","last_undegraded":"2024-09-20T19:08:41.966375+0000","last_fullsized":"2024-09-20T19:08:41.966375+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'19","last_scrub_stamp":"2024-09-20T19:06:26.909593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:06:26.909593+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":875,"reported_epoch":479,"state":"active+clean","last_fresh":"2024-09-20T19:08:41.964820+0000","last_change":"2024-09-20T19:01:40.961449+0000","last_active":"2024-09-20T19:08:41.964820+0000","last_peered":"2024-09-20T19:08:41.964820+0000","last_clean":"2024-09-20T19:08:41.964820+0000","last_became_active":"2024-09-20T18:49:49.957582+0000","last_became_peered":"2024-09-20T18:49:49.957582+0000","last_unstale":"2024-09-20T19:08:41.964820+0000","last_undegraded":"2024-09-20T19:08:41.964820+0000","last_fullsized":"2024-09-20T19:08:41.964820+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:01:40.961363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:01:40.961363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":26,"stat_sum":{"num_bytes":3131211800,"num_objects":47780,"num_object_clones":0,"num_object_copies":143340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47780,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":101211,"num_write_kb":3680353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5969,"num_bytes_recovered":391118848,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":9990320128,"data_stored":9990307912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154144,"ondisk_log_size":154144,"up":78,"acting":78,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1185,"internal_metadata":0},"log_size":87,"ondisk_log_size":87,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1114,"num_read_kb":871,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":257,"num_bytes_recovered":4890,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":686,"ondisk_log_size":686,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":3105,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1133,"num_read_kb":1128,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":435020,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":21,"num_read_kb":41,"num_write":20,"num_write_kb":296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":477367,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":869,"ondisk_log_size":869,"up":33,"acting":33,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266920,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1991940,"kb_used_data":1274932,"kb_used_omap":1,"kb_used_meta":716990,"kb_avail":91753212,"statfs":{"total":95995035648,"available":93955289088,"internally_reserved":0,"allocated":1305530368,"data_stored":1305132412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":734198218},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528559,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2277556,"kb_used_data":1682724,"kb_used_omap":1,"kb_used_meta":594814,"kb_avail":91467596,"statfs":{"total":95995035648,"available":93662818304,"internally_reserved":0,"allocated":1723109376,"data_stored":1722727166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":609089990},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757492,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1503496,"kb_used_data":796216,"kb_used_omap":1,"kb_used_meta":707262,"kb_avail":92241656,"statfs":{"total":95995035648,"available":94455455744,"internally_reserved":0,"allocated":815325184,"data_stored":814923731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":724236741},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":304,"seq":1305670058074,"num_pgs":68,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1560152,"kb_used_data":1114696,"kb_used_omap":5,"kb_used_meta":445434,"kb_avail":92185000,"statfs":{"total":95995035648,"available":94397440000,"internally_reserved":0,"allocated":1141448704,"data_stored":1141031241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5633,"internal_metadata":456124927},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182656,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1927300,"kb_used_data":1070068,"kb_used_omap":1,"kb_used_meta":857214,"kb_avail":91817852,"statfs":{"total":95995035648,"available":94021480448,"internally_reserved":0,"allocated":1095749632,"data_stored":1095351655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":877787591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411589,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2206784,"kb_used_data":1457840,"kb_used_omap":1,"kb_used_meta":748926,"kb_avail":91538368,"statfs":{"total":95995035648,"available":93735288832,"internally_reserved":0,"allocated":1492828160,"data_stored":1492435144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":766900677},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575115,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2413352,"kb_used_data":1606040,"kb_used_omap":1,"kb_used_meta":807230,"kb_avail":91331800,"statfs":{"total":95995035648,"available":93523763200,"internally_reserved":0,"allocated":1644584960,"data_stored":1644151600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":826603973},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738641,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1800644,"kb_used_data":930932,"kb_used_omap":1,"kb_used_meta":869694,"kb_avail":91944508,"statfs":{"total":95995035648,"available":94151176192,"internally_reserved":0,"allocated":953274368,"data_stored":952876108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":890567109},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1185,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":933232640,"data_stored":933232640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1621360640,"data_stored":1621360640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1457131520,"data_stored":1457127448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1091502080,"data_stored":1091502080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1114509312,"data_stored":1114505240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":811663360,"data_stored":811663360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1674973184,"data_stored":1674969112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1285947392,"data_stored":1285947392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T19:08:55.636 INFO:tasks.cephadm.ceph_manager.ceph:decrease pool size by 10 2024-09-20T19:08:55.636 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd pool set unique_pool_4 pg_num 16 2024-09-20T19:08:55.657 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:55.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:55.658 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:55.759 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:55.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:55.786 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:55.786 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:55.887 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:55.898 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:08:55.898 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:08:55.933 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:08:55.933 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:08:55.943 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:55.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:55.944 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:55.972 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:08:55.972 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:08:56.008 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:08:56.008 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:08:56.044 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:08:56.044 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:56.045 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:08:56.075 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:56.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:56.076 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:56.077 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:08:56.077 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:08:56.112 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:08:56.113 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:08:56.113 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:55 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:08:56.113 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:55 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:08:56.113 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:55 smithi083 ceph-mon[29421]: osdmap e481: 8 total, 8 up, 8 in 2024-09-20T19:08:56.114 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:55 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:08:56.114 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:55 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:08:56.114 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:55 smithi083 ceph-mon[36260]: osdmap e481: 8 total, 8 up, 8 in 2024-09-20T19:08:56.146 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:08:56.147 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:08:56.177 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:08:56.218 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:56.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:56.219 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:08:56.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:55 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:08:56.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:55 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:08:56.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:55 smithi142 ceph-mon[32508]: osdmap e481: 8 total, 8 up, 8 in 2024-09-20T19:08:56.320 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:08:56.351 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:56.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:56.352 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:08:56.379 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:08:56.453 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:56.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:56.506 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:56.507 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:56.608 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:56.646 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:56.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:56.647 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:08:56.748 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:56.778 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:56.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:56.779 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:56.880 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:56.916 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:56.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:56.917 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:56.978 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:56 smithi083 ceph-mon[29421]: pgmap v970: 168 pgs: 168 active+clean; 2.5 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 938 op/s 2024-09-20T19:08:56.978 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:56 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2293529322' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:08:56.978 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:56 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:08:56.978 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:56 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:08:56.979 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:56 smithi083 ceph-mon[36260]: pgmap v970: 168 pgs: 168 active+clean; 2.5 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 938 op/s 2024-09-20T19:08:56.979 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:56 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2293529322' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:08:56.979 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:56 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:08:56.979 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:56 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:08:57.019 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:57.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:57.052 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:57.053 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:57.153 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:57.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:57.179 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:57.180 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:57.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:56 smithi142 ceph-mon[32508]: pgmap v970: 168 pgs: 168 active+clean; 2.5 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 938 op/s 2024-09-20T19:08:57.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:56 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2293529322' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:08:57.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:56 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:08:57.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:56 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:08:57.280 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:57.310 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:57.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:57.311 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:57.412 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:57.438 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:57.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:57.440 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:57.540 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:57.576 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:57.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:57.577 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:57.677 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:08:57.715 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:57.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:57.716 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:08:57.722 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:08:57.768 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T19:08:57.768 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-20T19:08:57.768 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-20T19:08:57.769 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-20T19:08:57.769 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.769 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.25484.0:124578 9.2 e289/246 9:422abb8a:::benchmark_data_smithi083_124552_object10431:head v 289'7686, mlcod=289'7682)", 2024-09-20T19:08:57.769 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:20.951824+0000", 2024-09-20T19:08:57.769 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 516.80856109499996, 2024-09-20T19:08:57.769 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.32891357100000002, 2024-09-20T19:08:57.769 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:08:57.769 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:08:57.769 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:08:57.769 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:08:57.769 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.769 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:08:57.769 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.951824+0000", 2024-09-20T19:08:57.769 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:57.770 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.770 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.770 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:08:57.770 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.951824+0000", 2024-09-20T19:08:57.770 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:57.770 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.770 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.770 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:08:57.770 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.951825+0000", 2024-09-20T19:08:57.770 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0470000000000001e-06 2024-09-20T19:08:57.770 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.770 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.770 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:08:57.770 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.951831+0000", 2024-09-20T19:08:57.771 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2999999999999998e-06 2024-09-20T19:08:57.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.771 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:08:57.771 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.951841+0000", 2024-09-20T19:08:57.771 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0234999999999999e-05 2024-09-20T19:08:57.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.771 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:08:57.771 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.951854+0000", 2024-09-20T19:08:57.771 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.287e-05 2024-09-20T19:08:57.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.771 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.771 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:08:57.772 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.951926+0000", 2024-09-20T19:08:57.772 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1932000000000002e-05 2024-09-20T19:08:57.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.772 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.772 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:08:57.772 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.951943+0000", 2024-09-20T19:08:57.772 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7033000000000001e-05 2024-09-20T19:08:57.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.772 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.772 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:08:57.772 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.280700+0000", 2024-09-20T19:08:57.772 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.328756979 2024-09-20T19:08:57.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.772 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.772 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:08:57.773 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.280737+0000", 2024-09-20T19:08:57.773 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7175000000000002e-05 2024-09-20T19:08:57.773 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:57.773 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:57.773 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:57.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.773 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.773 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.25484.0:124584 9.0 e289/284 9:06443903:::benchmark_data_smithi083_124552_object10437:head v 289'8049, mlcod=289'8046)", 2024-09-20T19:08:57.773 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:20.953013+0000", 2024-09-20T19:08:57.773 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 516.80737178200002, 2024-09-20T19:08:57.773 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.323892601, 2024-09-20T19:08:57.773 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:08:57.773 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:08:57.773 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:08:57.774 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:08:57.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.774 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:08:57.774 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.953013+0000", 2024-09-20T19:08:57.774 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:57.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.774 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:08:57.774 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.953013+0000", 2024-09-20T19:08:57.774 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:57.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.774 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.774 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:08:57.774 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.953014+0000", 2024-09-20T19:08:57.775 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0999999999999997e-07 2024-09-20T19:08:57.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.775 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:08:57.775 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.953020+0000", 2024-09-20T19:08:57.775 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8390000000000003e-06 2024-09-20T19:08:57.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.775 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:08:57.775 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.953030+0000", 2024-09-20T19:08:57.775 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8530000000000004e-06 2024-09-20T19:08:57.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.775 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:08:57.775 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.953036+0000", 2024-09-20T19:08:57.776 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1510000000000001e-06 2024-09-20T19:08:57.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.776 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.776 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:08:57.776 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.953099+0000", 2024-09-20T19:08:57.776 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2959000000000005e-05 2024-09-20T19:08:57.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.776 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.776 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:08:57.776 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.953117+0000", 2024-09-20T19:08:57.776 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8559999999999998e-05 2024-09-20T19:08:57.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.776 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.776 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:08:57.777 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.959129+0000", 2024-09-20T19:08:57.777 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0060115500000000001 2024-09-20T19:08:57.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.777 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.777 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:08:57.777 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.276906+0000", 2024-09-20T19:08:57.777 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.31777687900000001 2024-09-20T19:08:57.777 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:57.777 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:57.777 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:57.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.777 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.777 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.25484.0:124590 9.6 e289/246 9:6ce021c9:::benchmark_data_smithi083_124552_object10443:head v 289'7744, mlcod=289'7742)", 2024-09-20T19:08:57.777 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:20.956000+0000", 2024-09-20T19:08:57.777 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 516.80438454199998, 2024-09-20T19:08:57.778 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.32402082500000001, 2024-09-20T19:08:57.778 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:08:57.778 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:08:57.778 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:08:57.778 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:08:57.778 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.778 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:08:57.778 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.956000+0000", 2024-09-20T19:08:57.778 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:57.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.778 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.778 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:08:57.778 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.956000+0000", 2024-09-20T19:08:57.779 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:57.779 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.779 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.779 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:08:57.779 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.956001+0000", 2024-09-20T19:08:57.779 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9600000000000008e-07 2024-09-20T19:08:57.779 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.779 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.779 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:08:57.779 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.956007+0000", 2024-09-20T19:08:57.779 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3199999999999999e-06 2024-09-20T19:08:57.779 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.779 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.779 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:08:57.779 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.956014+0000", 2024-09-20T19:08:57.780 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7880000000000007e-06 2024-09-20T19:08:57.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.780 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.780 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:08:57.780 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.956019+0000", 2024-09-20T19:08:57.780 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5050000000000001e-06 2024-09-20T19:08:57.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.780 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.780 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:08:57.780 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.956069+0000", 2024-09-20T19:08:57.780 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0467000000000003e-05 2024-09-20T19:08:57.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.780 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.780 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:08:57.781 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.956089+0000", 2024-09-20T19:08:57.781 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.948e-05 2024-09-20T19:08:57.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.781 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:08:57.781 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.279983+0000", 2024-09-20T19:08:57.781 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.32389452099999999 2024-09-20T19:08:57.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.781 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:08:57.781 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.280021+0000", 2024-09-20T19:08:57.781 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.7747999999999998e-05 2024-09-20T19:08:57.781 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:57.781 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:57.782 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:57.782 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.782 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.782 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.25484.0:124594 9.6 e289/246 9:67fbc174:::benchmark_data_smithi083_124552_object10447:head v 289'7745, mlcod=289'7742)", 2024-09-20T19:08:57.782 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:20.960552+0000", 2024-09-20T19:08:57.782 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 516.79983270100001, 2024-09-20T19:08:57.782 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.32917961699999998, 2024-09-20T19:08:57.782 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:08:57.782 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:08:57.782 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:08:57.782 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:08:57.782 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.782 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:08:57.782 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.960552+0000", 2024-09-20T19:08:57.783 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:57.783 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.783 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.783 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:08:57.783 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.960552+0000", 2024-09-20T19:08:57.783 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:57.783 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.783 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.783 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:08:57.783 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.960553+0000", 2024-09-20T19:08:57.783 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0610000000000001e-06 2024-09-20T19:08:57.783 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.783 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.783 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:08:57.784 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.960562+0000", 2024-09-20T19:08:57.784 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0580000000000004e-06 2024-09-20T19:08:57.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.784 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.784 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:08:57.784 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.276862+0000", 2024-09-20T19:08:57.784 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.31629938200000002 2024-09-20T19:08:57.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.784 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.784 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:08:57.784 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.276872+0000", 2024-09-20T19:08:57.784 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0614e-05 2024-09-20T19:08:57.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.784 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.784 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:08:57.785 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.276943+0000", 2024-09-20T19:08:57.785 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0933999999999995e-05 2024-09-20T19:08:57.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.785 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:08:57.785 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.276962+0000", 2024-09-20T19:08:57.785 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9293999999999999e-05 2024-09-20T19:08:57.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.785 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:08:57.785 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.289671+0000", 2024-09-20T19:08:57.785 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.012708129 2024-09-20T19:08:57.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.785 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:08:57.786 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.289732+0000", 2024-09-20T19:08:57.786 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1144999999999998e-05 2024-09-20T19:08:57.786 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:57.786 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:57.786 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:57.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.786 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.786 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.25484.0:124596 9.2 e289/246 9:42d01fa2:::benchmark_data_smithi083_124552_object10449:head v 289'7687, mlcod=289'7684)", 2024-09-20T19:08:57.786 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:20.963075+0000", 2024-09-20T19:08:57.786 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 516.79730994600004, 2024-09-20T19:08:57.786 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.32681833599999999, 2024-09-20T19:08:57.786 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:08:57.786 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:08:57.786 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:08:57.787 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:08:57.787 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.787 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:08:57.787 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.963075+0000", 2024-09-20T19:08:57.787 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:57.787 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.787 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.787 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:08:57.787 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.963075+0000", 2024-09-20T19:08:57.787 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:57.787 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.787 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.787 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:08:57.787 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.963076+0000", 2024-09-20T19:08:57.788 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6600000000000005e-07 2024-09-20T19:08:57.788 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.788 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.788 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:08:57.788 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:20.963083+0000", 2024-09-20T19:08:57.788 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9140000000000002e-06 2024-09-20T19:08:57.788 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.788 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.788 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:08:57.788 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.276845+0000", 2024-09-20T19:08:57.788 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.31376241599999999 2024-09-20T19:08:57.788 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.788 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.788 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:08:57.789 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.276871+0000", 2024-09-20T19:08:57.789 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6123999999999999e-05 2024-09-20T19:08:57.789 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.789 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.789 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:08:57.789 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.276934+0000", 2024-09-20T19:08:57.789 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3147999999999999e-05 2024-09-20T19:08:57.789 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.789 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.789 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:08:57.789 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.276960+0000", 2024-09-20T19:08:57.789 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.5939e-05 2024-09-20T19:08:57.789 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.789 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.789 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:08:57.790 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.289643+0000", 2024-09-20T19:08:57.790 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.012682236 2024-09-20T19:08:57.790 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.790 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.790 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:08:57.790 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:21.289893+0000", 2024-09-20T19:08:57.790 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00025069300000000002 2024-09-20T19:08:57.790 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:57.790 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:57.790 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:57.790 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.790 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.790 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.26264.0:60564 11.12 11:4d967e86:::benchmark_data_smithi083_177491_object60563:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e441)", 2024-09-20T19:08:57.790 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:06:55.439386+0000", 2024-09-20T19:08:57.791 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 122.320998715, 2024-09-20T19:08:57.791 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0464370300000001, 2024-09-20T19:08:57.791 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:08:57.791 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:08:57.791 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:08:57.791 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:08:57.791 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.26264", 2024-09-20T19:08:57.791 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2608780316", 2024-09-20T19:08:57.791 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 60564 2024-09-20T19:08:57.791 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.791 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:08:57.791 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.791 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:08:57.791 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:55.439386+0000", 2024-09-20T19:08:57.792 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:57.792 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.792 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.792 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:08:57.792 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:55.439386+0000", 2024-09-20T19:08:57.792 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:57.792 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.792 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.792 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:08:57.792 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:55.439388+0000", 2024-09-20T19:08:57.792 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4500000000000001e-06 2024-09-20T19:08:57.792 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.792 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.792 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:08:57.792 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:55.439412+0000", 2024-09-20T19:08:57.793 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4766e-05 2024-09-20T19:08:57.793 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.793 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.793 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:08:57.793 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:55.440274+0000", 2024-09-20T19:08:57.793 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00086140299999999995 2024-09-20T19:08:57.793 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.793 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.793 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:08:57.793 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:55.440286+0000", 2024-09-20T19:08:57.793 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2398000000000001e-05 2024-09-20T19:08:57.793 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.793 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.793 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:08:57.793 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:55.585045+0000", 2024-09-20T19:08:57.794 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.144758838 2024-09-20T19:08:57.794 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.794 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.794 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:08:57.794 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:55.585054+0000", 2024-09-20T19:08:57.794 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9719999999999998e-06 2024-09-20T19:08:57.794 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.794 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.794 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:08:57.794 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:56.479223+0000", 2024-09-20T19:08:57.794 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.89416917100000004 2024-09-20T19:08:57.794 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.794 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.794 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:08:57.794 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:56.479365+0000", 2024-09-20T19:08:57.795 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014197299999999999 2024-09-20T19:08:57.795 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.795 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.795 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,3", 2024-09-20T19:08:57.795 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:56.479517+0000", 2024-09-20T19:08:57.795 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015203699999999999 2024-09-20T19:08:57.795 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.795 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.795 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:08:57.796 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:56.481062+0000", 2024-09-20T19:08:57.796 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001545325 2024-09-20T19:08:57.796 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.796 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.796 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:08:57.796 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:56.483372+0000", 2024-09-20T19:08:57.797 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0023094539999999998 2024-09-20T19:08:57.797 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.797 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.797 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:08:57.797 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:56.485713+0000", 2024-09-20T19:08:57.797 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002340778 2024-09-20T19:08:57.797 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.797 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.797 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:08:57.797 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:56.485778+0000", 2024-09-20T19:08:57.797 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5146000000000003e-05 2024-09-20T19:08:57.797 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.797 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.798 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:08:57.798 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:56.485823+0000", 2024-09-20T19:08:57.798 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5318999999999998e-05 2024-09-20T19:08:57.798 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:57.798 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:57.798 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:57.798 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.798 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.798 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.26264.0:60568 11.2 11:45e00d5d:::benchmark_data_smithi083_177491_object60567:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e441)", 2024-09-20T19:08:57.798 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:06:55.515916+0000", 2024-09-20T19:08:57.798 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 122.24446931, 2024-09-20T19:08:57.798 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.95516527799999995, 2024-09-20T19:08:57.798 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:08:57.798 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:08:57.799 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:08:57.799 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:08:57.799 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.26264", 2024-09-20T19:08:57.799 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2608780316", 2024-09-20T19:08:57.799 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 60568 2024-09-20T19:08:57.799 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.799 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:08:57.799 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.799 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:08:57.799 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:55.515916+0000", 2024-09-20T19:08:57.799 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:57.799 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.799 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.799 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:08:57.800 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:55.515916+0000", 2024-09-20T19:08:57.800 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:57.800 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.800 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.800 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:08:57.800 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:55.515918+0000", 2024-09-20T19:08:57.800 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.204e-06 2024-09-20T19:08:57.800 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.800 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.800 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:08:57.800 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:55.515957+0000", 2024-09-20T19:08:57.800 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9750999999999999e-05 2024-09-20T19:08:57.800 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.800 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.801 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:08:57.801 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:55.515978+0000", 2024-09-20T19:08:57.801 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0557999999999999e-05 2024-09-20T19:08:57.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.801 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.801 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:08:57.801 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:55.515991+0000", 2024-09-20T19:08:57.801 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3314e-05 2024-09-20T19:08:57.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.801 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.801 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:08:57.801 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:55.584126+0000", 2024-09-20T19:08:57.801 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.068134923999999999 2024-09-20T19:08:57.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.802 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.802 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:08:57.802 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:55.584146+0000", 2024-09-20T19:08:57.802 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0166999999999999e-05 2024-09-20T19:08:57.802 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.802 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.802 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:08:57.802 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:56.456714+0000", 2024-09-20T19:08:57.802 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.872567753 2024-09-20T19:08:57.802 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.802 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.802 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:08:57.802 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:56.456894+0000", 2024-09-20T19:08:57.802 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000179388 2024-09-20T19:08:57.803 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.803 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.803 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,3", 2024-09-20T19:08:57.803 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:56.457041+0000", 2024-09-20T19:08:57.803 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000147498 2024-09-20T19:08:57.803 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.803 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.803 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:08:57.803 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:56.459673+0000", 2024-09-20T19:08:57.803 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0026316339999999999 2024-09-20T19:08:57.803 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.803 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.803 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:08:57.803 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:56.466500+0000", 2024-09-20T19:08:57.804 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0068273860000000004 2024-09-20T19:08:57.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.804 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.804 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:08:57.804 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:56.470960+0000", 2024-09-20T19:08:57.804 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0044596269999999999 2024-09-20T19:08:57.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.804 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.804 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:08:57.804 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:56.471032+0000", 2024-09-20T19:08:57.804 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2552999999999995e-05 2024-09-20T19:08:57.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.804 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.804 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:08:57.805 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:56.471081+0000", 2024-09-20T19:08:57.805 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8520999999999998e-05 2024-09-20T19:08:57.805 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:57.805 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:57.805 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:57.805 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.805 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.805 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.26264.0:62064 11.18 11.8cbaaf58 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e443)", 2024-09-20T19:08:57.805 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:06:57.461898+0000", 2024-09-20T19:08:57.805 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 120.298486986, 2024-09-20T19:08:57.805 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.99147923299999996, 2024-09-20T19:08:57.805 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:08:57.805 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:08:57.805 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-20T19:08:57.805 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:08:57.806 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.26264", 2024-09-20T19:08:57.806 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2608780316", 2024-09-20T19:08:57.806 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 62064 2024-09-20T19:08:57.806 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.806 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:08:57.806 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.806 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:08:57.806 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:57.461898+0000", 2024-09-20T19:08:57.806 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:57.806 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.806 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.806 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:08:57.806 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:57.461898+0000", 2024-09-20T19:08:57.806 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:57.806 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.807 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.807 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:08:57.807 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:57.461899+0000", 2024-09-20T19:08:57.807 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.341e-06 2024-09-20T19:08:57.807 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.807 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.807 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:08:57.807 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:57.461923+0000", 2024-09-20T19:08:57.807 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.3332e-05 2024-09-20T19:08:57.807 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.807 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.807 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:08:57.807 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:57.461930+0000", 2024-09-20T19:08:57.807 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.6279999999999996e-06 2024-09-20T19:08:57.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.808 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.808 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:08:57.808 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:57.461936+0000", 2024-09-20T19:08:57.808 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0610000000000001e-06 2024-09-20T19:08:57.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.808 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.808 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:08:57.808 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:57.461998+0000", 2024-09-20T19:08:57.808 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1709000000000002e-05 2024-09-20T19:08:57.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.808 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.808 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:08:57.808 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:57.462012+0000", 2024-09-20T19:08:57.808 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4082e-05 2024-09-20T19:08:57.809 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.809 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.809 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:08:57.809 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:58.453364+0000", 2024-09-20T19:08:57.809 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.99135248200000003 2024-09-20T19:08:57.809 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.809 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.809 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:08:57.809 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:58.453377+0000", 2024-09-20T19:08:57.809 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2598e-05 2024-09-20T19:08:57.809 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:57.809 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:57.809 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:57.809 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.809 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.810 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.26264.0:62111 11.18 11.aa224e58 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e443)", 2024-09-20T19:08:57.810 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:06:57.493065+0000", 2024-09-20T19:08:57.810 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 120.26731934199999, 2024-09-20T19:08:57.810 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.96228124500000001, 2024-09-20T19:08:57.810 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:08:57.810 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:08:57.810 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-20T19:08:57.810 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:08:57.810 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.26264", 2024-09-20T19:08:57.810 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2608780316", 2024-09-20T19:08:57.810 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 62111 2024-09-20T19:08:57.810 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.810 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:08:57.810 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.811 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:08:57.811 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:57.493065+0000", 2024-09-20T19:08:57.811 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:57.811 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.811 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.811 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:08:57.811 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:57.493065+0000", 2024-09-20T19:08:57.811 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:57.811 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.811 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.811 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:08:57.811 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:57.493067+0000", 2024-09-20T19:08:57.811 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5e-06 2024-09-20T19:08:57.811 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.812 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.812 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:08:57.812 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:57.493089+0000", 2024-09-20T19:08:57.812 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1908000000000001e-05 2024-09-20T19:08:57.812 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.812 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.812 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:08:57.812 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:57.493100+0000", 2024-09-20T19:08:57.812 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0857e-05 2024-09-20T19:08:57.812 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.812 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.812 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:08:57.812 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:57.493107+0000", 2024-09-20T19:08:57.812 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2189999999999998e-06 2024-09-20T19:08:57.812 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.813 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.813 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:08:57.813 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:57.493168+0000", 2024-09-20T19:08:57.813 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1179999999999994e-05 2024-09-20T19:08:57.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.813 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.813 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:08:57.813 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:57.493176+0000", 2024-09-20T19:08:57.813 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2830000000000001e-06 2024-09-20T19:08:57.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.813 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.813 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:08:57.813 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:58.453396+0000", 2024-09-20T19:08:57.813 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.96022001000000001 2024-09-20T19:08:57.814 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.814 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.814 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:08:57.814 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:58.455347+0000", 2024-09-20T19:08:57.814 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0019502880000000001 2024-09-20T19:08:57.814 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:57.814 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:57.814 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:57.814 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.814 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.814 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.26264.0:62112 11.18 11.84949458 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e443)", 2024-09-20T19:08:57.814 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:06:57.493799+0000", 2024-09-20T19:08:57.814 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 120.2665858, 2024-09-20T19:08:57.814 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.96174308900000005, 2024-09-20T19:08:57.815 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:08:57.815 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:08:57.815 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-20T19:08:57.815 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:08:57.815 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.26264", 2024-09-20T19:08:57.815 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2608780316", 2024-09-20T19:08:57.815 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 62112 2024-09-20T19:08:57.815 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.815 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:08:57.815 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.815 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:08:57.815 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:57.493799+0000", 2024-09-20T19:08:57.815 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:57.815 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.816 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.816 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:08:57.816 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:57.493799+0000", 2024-09-20T19:08:57.816 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:57.816 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.816 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.816 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:08:57.816 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:57.493800+0000", 2024-09-20T19:08:57.816 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.540000000000001e-07 2024-09-20T19:08:57.816 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.816 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.816 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:08:57.816 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:57.493821+0000", 2024-09-20T19:08:57.816 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0788000000000002e-05 2024-09-20T19:08:57.817 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.817 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.817 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:08:57.817 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:57.493833+0000", 2024-09-20T19:08:57.817 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2398999999999999e-05 2024-09-20T19:08:57.817 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.817 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.817 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:08:57.817 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:57.493841+0000", 2024-09-20T19:08:57.817 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7400000000000004e-06 2024-09-20T19:08:57.817 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.817 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.817 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:08:57.817 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:57.493903+0000", 2024-09-20T19:08:57.817 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2057000000000006e-05 2024-09-20T19:08:57.818 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.818 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.818 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:08:57.818 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:57.493908+0000", 2024-09-20T19:08:57.818 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5639999999999998e-06 2024-09-20T19:08:57.818 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.818 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.818 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:08:57.818 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:58.455532+0000", 2024-09-20T19:08:57.818 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.96162445900000004 2024-09-20T19:08:57.818 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.818 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.818 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:08:57.818 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:58.455542+0000", 2024-09-20T19:08:57.818 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0128e-05 2024-09-20T19:08:57.819 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:57.819 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:57.819 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:57.819 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.819 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.819 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.26264.0:62113 11.11 11.9a6659f1 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e443)", 2024-09-20T19:08:57.819 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:06:57.493932+0000", 2024-09-20T19:08:57.819 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 120.266452624, 2024-09-20T19:08:57.819 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.96075666900000001, 2024-09-20T19:08:57.819 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:08:57.819 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:08:57.819 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-20T19:08:57.819 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:08:57.819 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.26264", 2024-09-20T19:08:57.820 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2608780316", 2024-09-20T19:08:57.820 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 62113 2024-09-20T19:08:57.820 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.820 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:08:57.820 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.820 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:08:57.820 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:57.493932+0000", 2024-09-20T19:08:57.820 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:57.820 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.820 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.820 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:08:57.820 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:57.493932+0000", 2024-09-20T19:08:57.820 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:57.820 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.820 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.821 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:08:57.821 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:57.493933+0000", 2024-09-20T19:08:57.821 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2200000000000003e-07 2024-09-20T19:08:57.821 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.821 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.821 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:08:57.821 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:57.493954+0000", 2024-09-20T19:08:57.821 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1100000000000001e-05 2024-09-20T19:08:57.821 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.821 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.821 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:08:57.821 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:57.493963+0000", 2024-09-20T19:08:57.821 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9279999999999999e-06 2024-09-20T19:08:57.821 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.821 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.822 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:08:57.822 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:57.493970+0000", 2024-09-20T19:08:57.822 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.3900000000000004e-06 2024-09-20T19:08:57.822 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.822 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.822 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:08:57.822 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:57.494028+0000", 2024-09-20T19:08:57.822 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7742999999999999e-05 2024-09-20T19:08:57.822 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.822 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.822 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:08:57.822 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:57.494036+0000", 2024-09-20T19:08:57.822 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.6179999999999997e-06 2024-09-20T19:08:57.822 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.823 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.823 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:08:57.823 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:58.454274+0000", 2024-09-20T19:08:57.823 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.96023781799999997 2024-09-20T19:08:57.823 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.823 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.823 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:08:57.823 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:58.454689+0000", 2024-09-20T19:08:57.823 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00041524999999999998 2024-09-20T19:08:57.823 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:57.823 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:57.823 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:57.823 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.823 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.824 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.26264.0:115387 11.c e468/422 11:3f09ca3c:::benchmark_data_smithi083_177491_object11945:head v 468'7198, mlcod=468'7194)", 2024-09-20T19:08:57.824 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:08:05.893748+0000", 2024-09-20T19:08:57.824 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.866636374999999, 2024-09-20T19:08:57.824 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.39787827599999998, 2024-09-20T19:08:57.824 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:08:57.824 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:08:57.824 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:08:57.824 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:08:57.824 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.824 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:08:57.824 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.893748+0000", 2024-09-20T19:08:57.824 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:57.824 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.824 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.825 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:08:57.825 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.893748+0000", 2024-09-20T19:08:57.825 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:57.825 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.825 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.825 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:08:57.825 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.893749+0000", 2024-09-20T19:08:57.825 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0400000000000005e-07 2024-09-20T19:08:57.825 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.825 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.825 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:08:57.825 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.893755+0000", 2024-09-20T19:08:57.825 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.1759999999999998e-06 2024-09-20T19:08:57.825 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.825 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.826 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:08:57.826 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.893768+0000", 2024-09-20T19:08:57.826 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2564e-05 2024-09-20T19:08:57.826 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.826 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.826 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:08:57.826 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.893774+0000", 2024-09-20T19:08:57.826 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4169999999999997e-06 2024-09-20T19:08:57.826 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.826 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.826 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:08:57.826 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.906193+0000", 2024-09-20T19:08:57.826 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.012418544 2024-09-20T19:08:57.827 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.827 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.827 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:08:57.827 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.906281+0000", 2024-09-20T19:08:57.827 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.7776999999999997e-05 2024-09-20T19:08:57.827 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.827 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.827 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:08:57.827 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.999092+0000", 2024-09-20T19:08:57.827 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.092811088999999999 2024-09-20T19:08:57.827 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.827 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.827 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:08:57.827 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:06.291627+0000", 2024-09-20T19:08:57.827 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.29253490500000001 2024-09-20T19:08:57.828 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:57.828 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:57.828 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:57.828 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.828 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.828 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.26264.0:115396 11.c e468/422 11:3a3880c1:::benchmark_data_smithi083_177491_object11954:head v 468'7199, mlcod=468'7194)", 2024-09-20T19:08:57.828 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:08:05.903685+0000", 2024-09-20T19:08:57.828 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.856699714999998, 2024-09-20T19:08:57.828 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.38805428199999997, 2024-09-20T19:08:57.828 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:08:57.828 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:08:57.828 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:08:57.828 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:08:57.828 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.829 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:08:57.829 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.903685+0000", 2024-09-20T19:08:57.829 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:57.829 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.829 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.829 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:08:57.829 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.903685+0000", 2024-09-20T19:08:57.829 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:57.829 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.829 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.829 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:08:57.829 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.903686+0000", 2024-09-20T19:08:57.829 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.3600000000000002e-07 2024-09-20T19:08:57.829 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.829 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.830 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:08:57.830 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.903693+0000", 2024-09-20T19:08:57.830 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.5930000000000002e-06 2024-09-20T19:08:57.830 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.830 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.830 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:08:57.830 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.903703+0000", 2024-09-20T19:08:57.830 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9869999999999994e-06 2024-09-20T19:08:57.830 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.830 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.830 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:08:57.830 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.903709+0000", 2024-09-20T19:08:57.830 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7530000000000004e-06 2024-09-20T19:08:57.830 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.831 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.831 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:08:57.831 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.909180+0000", 2024-09-20T19:08:57.831 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0054703729999999997 2024-09-20T19:08:57.831 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.831 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.831 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:08:57.831 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.909193+0000", 2024-09-20T19:08:57.831 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2857000000000001e-05 2024-09-20T19:08:57.831 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.831 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.831 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:08:57.831 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:06.291711+0000", 2024-09-20T19:08:57.831 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.38251810000000003 2024-09-20T19:08:57.832 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.832 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.832 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:08:57.832 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:06.291739+0000", 2024-09-20T19:08:57.832 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8683e-05 2024-09-20T19:08:57.832 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:57.832 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:57.832 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:57.832 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.832 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.832 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.26264.0:115402 11.4 e468/441 11:243413da:::benchmark_data_smithi083_177491_object11960:head v 468'5509, mlcod=468'5505)", 2024-09-20T19:08:57.832 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:08:05.909366+0000", 2024-09-20T19:08:57.832 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.851018760999999, 2024-09-20T19:08:57.832 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.38240226999999999, 2024-09-20T19:08:57.832 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:08:57.833 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:08:57.833 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:08:57.833 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:08:57.833 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.833 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:08:57.833 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.909366+0000", 2024-09-20T19:08:57.833 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:57.833 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.833 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.833 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:08:57.833 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.909366+0000", 2024-09-20T19:08:57.833 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:57.833 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.833 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.834 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:08:57.834 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.909367+0000", 2024-09-20T19:08:57.834 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.92e-07 2024-09-20T19:08:57.834 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.834 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.834 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:08:57.834 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.909371+0000", 2024-09-20T19:08:57.834 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0090000000000001e-06 2024-09-20T19:08:57.834 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.834 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.834 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:08:57.834 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.909378+0000", 2024-09-20T19:08:57.834 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0690000000000004e-06 2024-09-20T19:08:57.834 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.835 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.835 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:08:57.835 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.909382+0000", 2024-09-20T19:08:57.835 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3529999999999997e-06 2024-09-20T19:08:57.835 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.835 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.835 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:08:57.835 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.994289+0000", 2024-09-20T19:08:57.835 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.084906563000000004 2024-09-20T19:08:57.835 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.835 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.835 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:08:57.835 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.994307+0000", 2024-09-20T19:08:57.835 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.7770000000000001e-05 2024-09-20T19:08:57.835 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.836 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.836 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:08:57.836 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:06.291742+0000", 2024-09-20T19:08:57.836 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.29743537799999997 2024-09-20T19:08:57.836 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.836 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.836 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:08:57.836 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:06.291768+0000", 2024-09-20T19:08:57.836 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6336e-05 2024-09-20T19:08:57.836 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:57.836 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:57.836 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:57.836 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.836 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.837 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.26264.0:115409 11.12 e468/448 11:4afd55ba:::benchmark_data_smithi083_177491_object11967:head v 468'5414, mlcod=468'5412)", 2024-09-20T19:08:57.837 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:08:05.911802+0000", 2024-09-20T19:08:57.837 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.848582833999998, 2024-09-20T19:08:57.837 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.38005981900000002, 2024-09-20T19:08:57.837 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:57.838 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:08:57.838 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:08:57.838 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:08:57.838 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:08:57.838 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.838 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:08:57.839 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.911802+0000", 2024-09-20T19:08:57.839 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:57.839 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.839 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.839 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:08:57.839 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.911802+0000", 2024-09-20T19:08:57.839 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:57.839 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.839 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.839 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:08:57.839 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.911803+0000", 2024-09-20T19:08:57.839 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.99e-07 2024-09-20T19:08:57.839 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.839 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.840 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:08:57.840 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.911807+0000", 2024-09-20T19:08:57.840 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.9700000000000001e-06 2024-09-20T19:08:57.840 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.840 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.840 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:08:57.840 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.994184+0000", 2024-09-20T19:08:57.840 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.082377132000000006 2024-09-20T19:08:57.840 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.840 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.840 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:08:57.840 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.994199+0000", 2024-09-20T19:08:57.840 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5356e-05 2024-09-20T19:08:57.840 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.841 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.841 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:08:57.841 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.994267+0000", 2024-09-20T19:08:57.841 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.7776999999999998e-05 2024-09-20T19:08:57.841 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.841 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.841 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:08:57.841 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.994293+0000", 2024-09-20T19:08:57.841 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6162000000000001e-05 2024-09-20T19:08:57.841 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.841 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.841 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:08:57.841 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:06.291829+0000", 2024-09-20T19:08:57.841 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.29753624699999998 2024-09-20T19:08:57.842 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.842 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.842 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:08:57.842 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:06.291862+0000", 2024-09-20T19:08:57.842 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.2576000000000001e-05 2024-09-20T19:08:57.842 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:57.842 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:57.842 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:57.842 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.842 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.842 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.26264.0:115411 11.2 11:43179233:::benchmark_data_smithi083_177491_object11969:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e468)", 2024-09-20T19:08:57.842 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:08:05.912323+0000", 2024-09-20T19:08:57.842 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.848061991000002, 2024-09-20T19:08:57.843 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.39330036400000001, 2024-09-20T19:08:57.843 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:08:57.843 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:08:57.843 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:08:57.843 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:08:57.843 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.26264", 2024-09-20T19:08:57.843 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2608780316", 2024-09-20T19:08:57.843 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 115411 2024-09-20T19:08:57.843 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.843 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:08:57.843 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.843 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:08:57.843 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.912323+0000", 2024-09-20T19:08:57.843 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:57.843 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.844 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.844 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:08:57.844 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.912323+0000", 2024-09-20T19:08:57.844 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:57.844 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.844 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.844 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:08:57.844 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.912324+0000", 2024-09-20T19:08:57.844 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2839999999999999e-06 2024-09-20T19:08:57.844 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.844 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.844 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:08:57.844 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.912328+0000", 2024-09-20T19:08:57.844 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1829999999999996e-06 2024-09-20T19:08:57.845 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.845 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.845 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:08:57.845 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.994962+0000", 2024-09-20T19:08:57.845 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.082634087999999994 2024-09-20T19:08:57.845 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.845 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.845 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:08:57.845 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.994973+0000", 2024-09-20T19:08:57.845 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1e-05 2024-09-20T19:08:57.845 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.845 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.845 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:08:57.845 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:06.292014+0000", 2024-09-20T19:08:57.846 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.29704023099999999 2024-09-20T19:08:57.846 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.846 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.846 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:08:57.846 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:06.292116+0000", 2024-09-20T19:08:57.846 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010279400000000001 2024-09-20T19:08:57.846 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.846 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.846 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,3", 2024-09-20T19:08:57.846 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:06.292509+0000", 2024-09-20T19:08:57.846 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00039233300000000003 2024-09-20T19:08:57.846 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.846 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.846 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:08:57.847 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:06.296362+0000", 2024-09-20T19:08:57.847 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0038531110000000002 2024-09-20T19:08:57.847 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.847 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.847 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:08:57.847 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:06.296781+0000", 2024-09-20T19:08:57.847 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00041945099999999999 2024-09-20T19:08:57.847 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.847 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.847 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:08:57.847 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:06.305527+0000", 2024-09-20T19:08:57.847 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0087458099999999997 2024-09-20T19:08:57.847 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.847 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.847 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:08:57.848 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:06.305580+0000", 2024-09-20T19:08:57.848 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2655000000000003e-05 2024-09-20T19:08:57.848 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.848 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.848 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:08:57.848 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:06.305623+0000", 2024-09-20T19:08:57.848 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3424000000000003e-05 2024-09-20T19:08:57.848 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:57.848 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:57.848 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:57.848 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.848 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.848 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.26264.0:115404 11.c e468/422 11:3d2e93ac:::benchmark_data_smithi083_177491_object11962:head v 468'7200, mlcod=468'7196)", 2024-09-20T19:08:57.848 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:08:05.998683+0000", 2024-09-20T19:08:57.849 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.761701774000002, 2024-09-20T19:08:57.849 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.306639999, 2024-09-20T19:08:57.849 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:08:57.849 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:08:57.849 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:08:57.849 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:08:57.849 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.849 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:08:57.849 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.998683+0000", 2024-09-20T19:08:57.849 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:57.849 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.849 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.849 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:08:57.849 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.998683+0000", 2024-09-20T19:08:57.849 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:57.850 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.850 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.850 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:08:57.850 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.998684+0000", 2024-09-20T19:08:57.850 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.2099999999999995e-07 2024-09-20T19:08:57.850 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.850 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.850 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:08:57.850 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.998690+0000", 2024-09-20T19:08:57.850 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2700000000000001e-06 2024-09-20T19:08:57.850 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.850 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.850 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:08:57.850 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.998700+0000", 2024-09-20T19:08:57.851 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.8430000000000005e-06 2024-09-20T19:08:57.851 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.851 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.851 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:08:57.851 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.998708+0000", 2024-09-20T19:08:57.851 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.3180000000000007e-06 2024-09-20T19:08:57.851 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.851 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.851 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:08:57.851 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:06.294683+0000", 2024-09-20T19:08:57.851 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.295974189 2024-09-20T19:08:57.851 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.851 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.851 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:08:57.851 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:06.294701+0000", 2024-09-20T19:08:57.852 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8947999999999999e-05 2024-09-20T19:08:57.852 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.852 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.852 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:08:57.852 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:06.305197+0000", 2024-09-20T19:08:57.852 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.010495489 2024-09-20T19:08:57.852 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.852 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.852 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:08:57.852 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:06.305323+0000", 2024-09-20T19:08:57.852 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00012612100000000001 2024-09-20T19:08:57.852 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:57.852 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:57.852 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:57.853 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.853 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.853 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.26264.0:115416 11.15 11:afbec06a:::benchmark_data_smithi083_177491_object11974:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e468)", 2024-09-20T19:08:57.853 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:08:05.998740+0000", 2024-09-20T19:08:57.853 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.761644654000001, 2024-09-20T19:08:57.853 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.30282773600000001, 2024-09-20T19:08:57.853 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:08:57.853 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:08:57.853 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:08:57.853 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:08:57.853 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.26264", 2024-09-20T19:08:57.853 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2608780316", 2024-09-20T19:08:57.853 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 115416 2024-09-20T19:08:57.853 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.853 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:08:57.854 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.854 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:08:57.854 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.998740+0000", 2024-09-20T19:08:57.854 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:57.854 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.854 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.854 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:08:57.854 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.998740+0000", 2024-09-20T19:08:57.854 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:57.854 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.854 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.854 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:08:57.854 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.998741+0000", 2024-09-20T19:08:57.854 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2589999999999999e-06 2024-09-20T19:08:57.855 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.855 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.855 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:08:57.855 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.998746+0000", 2024-09-20T19:08:57.855 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0100000000000003e-06 2024-09-20T19:08:57.855 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.855 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.855 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:08:57.855 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.998756+0000", 2024-09-20T19:08:57.855 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6679999999999996e-06 2024-09-20T19:08:57.855 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.855 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.855 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:08:57.855 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.998762+0000", 2024-09-20T19:08:57.855 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8449999999999999e-06 2024-09-20T19:08:57.856 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.856 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.856 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:08:57.856 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:06.291419+0000", 2024-09-20T19:08:57.856 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.292657167 2024-09-20T19:08:57.856 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.856 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.856 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:08:57.856 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:06.291648+0000", 2024-09-20T19:08:57.856 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022887499999999999 2024-09-20T19:08:57.856 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.856 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.856 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 1,4", 2024-09-20T19:08:57.856 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:06.292040+0000", 2024-09-20T19:08:57.857 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00039167200000000003 2024-09-20T19:08:57.857 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.857 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.857 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:08:57.857 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:06.296404+0000", 2024-09-20T19:08:57.857 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0043641560000000001 2024-09-20T19:08:57.857 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.857 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.857 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:08:57.857 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:06.298525+0000", 2024-09-20T19:08:57.857 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0021213450000000002 2024-09-20T19:08:57.857 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.857 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.857 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:08:57.858 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:06.301442+0000", 2024-09-20T19:08:57.858 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002916891 2024-09-20T19:08:57.858 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.858 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.858 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:08:57.858 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:06.301497+0000", 2024-09-20T19:08:57.858 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5028999999999998e-05 2024-09-20T19:08:57.858 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.858 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.858 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:08:57.858 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:06.301568+0000", 2024-09-20T19:08:57.858 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0819000000000001e-05 2024-09-20T19:08:57.858 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:57.858 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:57.859 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:57.859 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.859 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.859 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "MOSDPGPush(11.11 468/444 [PushOp(11:8ef90696:::benchmark_data_smithi083_177491_object22054:head, version: 431'1405, data_included: [0~65536], data_size: 65536, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(11:8ef90696:::benchmark_data_smithi083_177491_object22054:head@431'1405, size: 65536, copy_subset: [(0, 65536)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 65536, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-20T19:08:57.859 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:08:05.999290+0000", 2024-09-20T19:08:57.859 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.761095263999998, 2024-09-20T19:08:57.859 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.302668202, 2024-09-20T19:08:57.859 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:08:57.859 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:08:57.859 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T19:08:57.859 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:08:57.859 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.859 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:08:57.859 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.999290+0000", 2024-09-20T19:08:57.860 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:57.860 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.860 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.860 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:08:57.860 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.999290+0000", 2024-09-20T19:08:57.860 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:57.860 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.860 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.860 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:08:57.860 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.999290+0000", 2024-09-20T19:08:57.860 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6700000000000002e-07 2024-09-20T19:08:57.860 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.860 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.860 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:08:57.860 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:05.999326+0000", 2024-09-20T19:08:57.861 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.5150000000000001e-05 2024-09-20T19:08:57.861 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.861 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.861 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:08:57.861 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:06.291619+0000", 2024-09-20T19:08:57.861 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.29229354600000002 2024-09-20T19:08:57.861 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.861 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.861 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:08:57.861 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:06.291648+0000", 2024-09-20T19:08:57.861 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.849e-05 2024-09-20T19:08:57.861 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.861 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.861 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:08:57.861 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:06.299971+0000", 2024-09-20T19:08:57.862 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0083229329999999994 2024-09-20T19:08:57.862 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.862 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.862 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:08:57.862 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:06.299983+0000", 2024-09-20T19:08:57.862 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2468e-05 2024-09-20T19:08:57.862 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.862 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.862 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:08:57.862 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:06.301958+0000", 2024-09-20T19:08:57.862 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0019747480000000001 2024-09-20T19:08:57.862 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:57.862 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:57.862 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:57.863 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.863 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.863 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.26264.0:115408 11.c e468/422 11:3ef90db1:::benchmark_data_smithi083_177491_object11966:head v 468'7201, mlcod=468'7196)", 2024-09-20T19:08:57.863 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:08:06.001922+0000", 2024-09-20T19:08:57.863 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 51.758462571999999, 2024-09-20T19:08:57.863 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.30346746800000002, 2024-09-20T19:08:57.863 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:08:57.863 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:08:57.863 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:08:57.863 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:08:57.863 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.863 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:08:57.863 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:06.001922+0000", 2024-09-20T19:08:57.863 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:57.864 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.864 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.864 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:08:57.864 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:06.001922+0000", 2024-09-20T19:08:57.864 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:08:57.864 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.864 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.864 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:08:57.864 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:06.001923+0000", 2024-09-20T19:08:57.864 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0109999999999999e-06 2024-09-20T19:08:57.864 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.864 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.864 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:08:57.864 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:06.001931+0000", 2024-09-20T19:08:57.865 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.4410000000000004e-06 2024-09-20T19:08:57.865 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.865 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.865 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:08:57.865 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:06.291736+0000", 2024-09-20T19:08:57.865 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.28980531100000001 2024-09-20T19:08:57.865 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.865 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.865 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:08:57.865 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:06.291744+0000", 2024-09-20T19:08:57.865 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.1489999999999994e-06 2024-09-20T19:08:57.865 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.865 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.865 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:08:57.865 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:06.298284+0000", 2024-09-20T19:08:57.866 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.006539352 2024-09-20T19:08:57.866 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.866 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.866 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:08:57.866 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:06.298302+0000", 2024-09-20T19:08:57.866 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8029e-05 2024-09-20T19:08:57.866 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.866 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.866 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:08:57.866 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:06.305326+0000", 2024-09-20T19:08:57.866 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0070243889999999998 2024-09-20T19:08:57.866 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:08:57.866 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:08:57.866 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:08:57.867 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:08:06.305390+0000", 2024-09-20T19:08:57.867 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3786000000000002e-05 2024-09-20T19:08:57.867 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:57.867 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:57.867 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:57.867 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:08:57.867 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:08:57.867 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T19:08:57.868 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:08:57 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:08:57.894 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:57.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:57.895 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:57.995 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:08:58.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:58.038 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:58.038 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:08:58.139 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:58.140 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:57 smithi083 ceph-mon[29421]: osdmap e482: 8 total, 8 up, 8 in 2024-09-20T19:08:58.140 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:57 smithi083 ceph-mon[36260]: osdmap e482: 8 total, 8 up, 8 in 2024-09-20T19:08:58.169 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:58.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:58.170 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:58.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:57 smithi142 ceph-mon[32508]: osdmap e482: 8 total, 8 up, 8 in 2024-09-20T19:08:58.270 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:58.303 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:58.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:58.304 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:58.405 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:08:58.420 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T19:08:58.449 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:58.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:58.450 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:08:58.551 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:58.577 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:58.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:58.578 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:58.679 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:08:58.702 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:58.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:58.703 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:08:58.803 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:58.834 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:58.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:58.835 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:58.936 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:08:58.969 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:58.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:58.970 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:08:59.071 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:08:59.101 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:59.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:59.102 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:08:59.163 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T19:08:59.198 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:58 smithi083 ceph-mon[29421]: pgmap v972: 168 pgs: 168 active+clean; 2.3 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1.68k op/s 2024-09-20T19:08:59.198 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:58 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:08:59.199 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:58 smithi083 ceph-mon[36260]: pgmap v972: 168 pgs: 168 active+clean; 2.3 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1.68k op/s 2024-09-20T19:08:59.199 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:58 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:08:59.204 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:08:59.235 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:59.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:59.236 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:08:59.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:58 smithi142 ceph-mon[32508]: pgmap v972: 168 pgs: 168 active+clean; 2.3 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1.68k op/s 2024-09-20T19:08:59.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:58 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:08:59.337 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:08:59.370 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:59.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:59.371 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:08:59.472 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:08:59.515 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:08:59.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:59.516 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:08:59.616 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:08:59.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:59.652 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:59.652 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:08:59.752 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:08:59.786 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:59.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:59.787 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:08:59.888 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:08:59.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:08:59.927 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:08:59.928 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:09:00.029 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:09:00.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:00.058 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:00.059 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:09:00.159 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:09:00.183 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:00.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:00.184 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:09:00.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:59 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1080932248' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num", "val": "16"} : dispatch 2024-09-20T19:09:00.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:08:59 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num", "val": "16"} : dispatch 2024-09-20T19:09:00.285 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:09:00.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:59 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1080932248' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num", "val": "16"} : dispatch 2024-09-20T19:09:00.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:08:59 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num", "val": "16"} : dispatch 2024-09-20T19:09:00.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:59 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1080932248' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num", "val": "16"} : dispatch 2024-09-20T19:09:00.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:08:59 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num", "val": "16"} : dispatch 2024-09-20T19:09:00.321 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:00.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:00.322 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:09:00.422 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:09:00.464 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:00.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:00.465 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:09:00.566 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:09:00.596 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:00.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:00.597 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:09:00.697 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:09:00.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:00.721 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:00.721 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:09:00.822 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:09:00.860 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:00.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:00.861 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:09:00.962 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:09:00.993 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:00.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:00.994 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:09:01.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:00 smithi083 ceph-mon[29421]: pgmap v973: 168 pgs: 168 active+clean; 2.2 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.21k op/s 2024-09-20T19:09:01.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:00 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num", "val": "16"}]': finished 2024-09-20T19:09:01.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:00 smithi083 ceph-mon[29421]: osdmap e483: 8 total, 8 up, 8 in 2024-09-20T19:09:01.041 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:09:00 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:09:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:09:01.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:00 smithi083 ceph-mon[36260]: pgmap v973: 168 pgs: 168 active+clean; 2.2 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.21k op/s 2024-09-20T19:09:01.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:00 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num", "val": "16"}]': finished 2024-09-20T19:09:01.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:00 smithi083 ceph-mon[36260]: osdmap e483: 8 total, 8 up, 8 in 2024-09-20T19:09:01.095 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:09:01.127 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:01.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:01.128 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:09:01.229 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:09:01.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:00 smithi142 ceph-mon[32508]: pgmap v973: 168 pgs: 168 active+clean; 2.2 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.21k op/s 2024-09-20T19:09:01.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:00 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num", "val": "16"}]': finished 2024-09-20T19:09:01.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:00 smithi142 ceph-mon[32508]: osdmap e483: 8 total, 8 up, 8 in 2024-09-20T19:09:01.268 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:01.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:01.269 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:09:01.347 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-20T19:09:01.370 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:09:01.398 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:01.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:01.399 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:09:01.440 INFO:teuthology.orchestra.run.smithi083.stderr:set pool 11 pg_num to 16 2024-09-20T19:09:01.500 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:09:01.526 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:01.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:01.527 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:09:01.628 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:09:01.661 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:01.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:01.662 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:09:01.763 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:09:01.801 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:01.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:01.802 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:09:01.903 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:09:01.938 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:01.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:01.939 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:09:02.040 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:09:02.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:02.062 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:02.062 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:09:02.163 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:09:02.185 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:02.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:02.186 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:09:02.205 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:01 smithi083 ceph-mon[29421]: pgmap v975: 168 pgs: 168 active+clean; 2.1 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 798 B/s rd, 0 B/s wr, 1.21k op/s 2024-09-20T19:09:02.206 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:01 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pgp_num_actual", "val": "16"} : dispatch 2024-09-20T19:09:02.206 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:01 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:09:02.206 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:01 smithi083 ceph-mon[36260]: pgmap v975: 168 pgs: 168 active+clean; 2.1 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 798 B/s rd, 0 B/s wr, 1.21k op/s 2024-09-20T19:09:02.206 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:01 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pgp_num_actual", "val": "16"} : dispatch 2024-09-20T19:09:02.206 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:01 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:09:02.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:01 smithi142 ceph-mon[32508]: pgmap v975: 168 pgs: 168 active+clean; 2.1 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 798 B/s rd, 0 B/s wr, 1.21k op/s 2024-09-20T19:09:02.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:01 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pgp_num_actual", "val": "16"} : dispatch 2024-09-20T19:09:02.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:01 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:09:02.286 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:09:02.287 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:09:02.291 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:09:02.301 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:09:02.308 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:02.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:02.309 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:09:02.410 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:09:02.431 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:02.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:02.432 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:09:02.533 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:09:02.555 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:02.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:02.556 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:09:02.657 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:09:02.678 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:02.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:02.679 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:09:02.780 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:09:02.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:02.803 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:02.803 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:09:02.904 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:09:02.926 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:02.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:02.927 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:09:02.960 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:09:02.960 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:09:02.988 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:09:02.988 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:09:03.008 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:09:03.008 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:09:03.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:02 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pgp_num_actual", "val": "16"}]': finished 2024-09-20T19:09:03.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:02 smithi083 ceph-mon[29421]: osdmap e484: 8 total, 8 up, 8 in 2024-09-20T19:09:03.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:02 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3075875958' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:09:03.009 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:02 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pgp_num_actual", "val": "16"}]': finished 2024-09-20T19:09:03.009 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:02 smithi083 ceph-mon[36260]: osdmap e484: 8 total, 8 up, 8 in 2024-09-20T19:09:03.009 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:02 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3075875958' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:09:03.024 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:09:03.024 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:09:03.027 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:09:03.040 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:09:03.040 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:09:03.049 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:03.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:03.050 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:09:03.057 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:09:03.057 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:09:03.074 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:09:03.075 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:09:03.091 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:09:03.091 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:09:03.151 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:09:03.172 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:03.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:03.173 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:09:03.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:02 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pgp_num_actual", "val": "16"}]': finished 2024-09-20T19:09:03.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:02 smithi142 ceph-mon[32508]: osdmap e484: 8 total, 8 up, 8 in 2024-09-20T19:09:03.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:02 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3075875958' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:09:03.274 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:09:03.295 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:03.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:03.296 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:09:03.397 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:09:03.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:03.420 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:03.420 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:09:03.520 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:09:03.542 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:03.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:03.543 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:09:03.644 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:09:03.665 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:03.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:03.666 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:09:03.766 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:09:03.788 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:03.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:03.789 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:09:03.889 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:09:03.916 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:03.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:03.917 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:09:03.975 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-20T19:09:04.017 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:09:04.041 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:04.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:04.042 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:09:04.142 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:09:04.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:04.181 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:04.181 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:09:04.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:03 smithi142 ceph-mon[32508]: osdmap e485: 8 total, 8 up, 8 in 2024-09-20T19:09:04.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:03 smithi142 ceph-mon[32508]: pgmap v978: 168 pgs: 2 remapped+peering, 2 peering, 164 active+clean; 2.0 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 853 B/s rd, 20 MiB/s wr, 1.05k op/s 2024-09-20T19:09:04.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:03 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1928366783' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:09:04.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:03 smithi142 ceph-mon[32508]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:09:04.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:03 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1928366783' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T19:09:04.282 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:09:04.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:03 smithi083 ceph-mon[29421]: osdmap e485: 8 total, 8 up, 8 in 2024-09-20T19:09:04.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:03 smithi083 ceph-mon[29421]: pgmap v978: 168 pgs: 2 remapped+peering, 2 peering, 164 active+clean; 2.0 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 853 B/s rd, 20 MiB/s wr, 1.05k op/s 2024-09-20T19:09:04.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:03 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1928366783' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:09:04.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:03 smithi083 ceph-mon[29421]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:09:04.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:03 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1928366783' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T19:09:04.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:03 smithi083 ceph-mon[36260]: osdmap e485: 8 total, 8 up, 8 in 2024-09-20T19:09:04.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:03 smithi083 ceph-mon[36260]: pgmap v978: 168 pgs: 2 remapped+peering, 2 peering, 164 active+clean; 2.0 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 853 B/s rd, 20 MiB/s wr, 1.05k op/s 2024-09-20T19:09:04.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:03 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1928366783' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:09:04.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:03 smithi083 ceph-mon[36260]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:09:04.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:03 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1928366783' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T19:09:04.313 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:04.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:04.314 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:09:04.415 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:09:04.448 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:04.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:04.449 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:09:04.550 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:09:04.591 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:04.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:04.593 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:09:04.693 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:09:04.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:04.728 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:04.728 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:09:04.829 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:09:04.872 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:04.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:04.873 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:09:04.973 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:09:04.998 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:04.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:04.999 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:09:05.100 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:09:05.124 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:05.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:05.125 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:09:05.226 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:09:05.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:04 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:09:05.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:04 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:09:05.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:04 smithi142 ceph-mon[32508]: 11.15 starting backfill to osd.5 from (0'0,0'0] MAX to 483'7370 2024-09-20T19:09:05.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:04 smithi142 ceph-mon[32508]: 11.14 starting backfill to osd.2 from (0'0,0'0] MAX to 483'7494 2024-09-20T19:09:05.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:04 smithi142 ceph-mon[32508]: 11.16 starting backfill to osd.0 from (0'0,0'0] MAX to 483'7565 2024-09-20T19:09:05.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:04 smithi142 ceph-mon[32508]: osdmap e486: 8 total, 8 up, 8 in 2024-09-20T19:09:05.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:04 smithi142 ceph-mon[32508]: 11.10 starting backfill to osd.1 from (0'0,470'6052] MIN to 483'7454 2024-09-20T19:09:05.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:04 smithi142 ceph-mon[32508]: 11.17 starting backfill to osd.1 from (0'0,458'4892] MIN to 483'7477 2024-09-20T19:09:05.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:04 smithi142 ceph-mon[32508]: 11.13 starting backfill to osd.1 from (426'459,473'6502] MIN to 483'7611 2024-09-20T19:09:05.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:04 smithi142 ceph-mon[32508]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-20T19:09:05.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:04 smithi142 ceph-mon[32508]: 11.18 starting backfill to osd.3 from (0'0,0'0] MAX to 483'7263 2024-09-20T19:09:05.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:04 smithi142 ceph-mon[32508]: 11.11 starting backfill to osd.0 from (0'0,0'0] MAX to 483'7528 2024-09-20T19:09:05.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:04 smithi142 ceph-mon[32508]: 11.12 starting backfill to osd.3 from (0'0,0'0] MAX to 483'7413 2024-09-20T19:09:05.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:04 smithi142 ceph-mon[32508]: 11.17 starting backfill to osd.4 from (0'0,0'0] MAX to 483'7477 2024-09-20T19:09:05.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:04 smithi142 ceph-mon[32508]: 11.16 starting backfill to osd.1 from (0'0,0'0] MAX to 483'7565 2024-09-20T19:09:05.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:04 smithi142 ceph-mon[32508]: 11.15 starting backfill to osd.6 from (0'0,0'0] MAX to 483'7370 2024-09-20T19:09:05.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:04 smithi142 ceph-mon[32508]: 11.10 starting backfill to osd.3 from (0'0,0'0] MAX to 483'7454 2024-09-20T19:09:05.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:04 smithi142 ceph-mon[32508]: 11.16 starting backfill to osd.2 from (0'0,0'0] MAX to 483'7565 2024-09-20T19:09:05.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:04 smithi142 ceph-mon[32508]: 11.13 starting backfill to osd.6 from (0'0,442'3886] MIN to 483'7611 2024-09-20T19:09:05.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:04 smithi142 ceph-mon[32508]: 11.19 starting backfill to osd.3 from (0'0,0'0] MAX to 483'7480 2024-09-20T19:09:05.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:04 smithi142 ceph-mon[32508]: 11.18 starting backfill to osd.5 from (0'0,0'0] MAX to 483'7263 2024-09-20T19:09:05.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:04 smithi142 ceph-mon[32508]: 11.15 starting backfill to osd.7 from (0'0,0'0] MAX to 483'7370 2024-09-20T19:09:05.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:04 smithi142 ceph-mon[32508]: 11.11 starting backfill to osd.6 from (0'0,442'3889] MIN to 483'7528 2024-09-20T19:09:05.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:04 smithi142 ceph-mon[32508]: 11.19 starting backfill to osd.6 from (0'0,442'3866] MIN to 483'7480 2024-09-20T19:09:05.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:04 smithi142 ceph-mon[32508]: 11.14 starting backfill to osd.7 from (0'0,0'0] MAX to 483'7494 2024-09-20T19:09:05.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:04 smithi142 ceph-mon[32508]: 11.10 starting backfill to osd.7 from (0'0,0'0] MAX to 483'7454 2024-09-20T19:09:05.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:04 smithi142 ceph-mon[32508]: 11.13 starting backfill to osd.7 from (0'0,0'0] MAX to 483'7611 2024-09-20T19:09:05.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:04 smithi142 ceph-mon[32508]: 6.1 scrub starts 2024-09-20T19:09:05.262 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:04 smithi142 ceph-mon[32508]: 6.1 scrub ok 2024-09-20T19:09:05.268 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:05.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:05.269 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:09:05.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:09:05.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:09:05.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[29421]: 11.15 starting backfill to osd.5 from (0'0,0'0] MAX to 483'7370 2024-09-20T19:09:05.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[29421]: 11.14 starting backfill to osd.2 from (0'0,0'0] MAX to 483'7494 2024-09-20T19:09:05.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[29421]: 11.16 starting backfill to osd.0 from (0'0,0'0] MAX to 483'7565 2024-09-20T19:09:05.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[29421]: osdmap e486: 8 total, 8 up, 8 in 2024-09-20T19:09:05.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[29421]: 11.10 starting backfill to osd.1 from (0'0,470'6052] MIN to 483'7454 2024-09-20T19:09:05.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[29421]: 11.17 starting backfill to osd.1 from (0'0,458'4892] MIN to 483'7477 2024-09-20T19:09:05.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[29421]: 11.13 starting backfill to osd.1 from (426'459,473'6502] MIN to 483'7611 2024-09-20T19:09:05.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[29421]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-20T19:09:05.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[29421]: 11.18 starting backfill to osd.3 from (0'0,0'0] MAX to 483'7263 2024-09-20T19:09:05.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[29421]: 11.11 starting backfill to osd.0 from (0'0,0'0] MAX to 483'7528 2024-09-20T19:09:05.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[29421]: 11.12 starting backfill to osd.3 from (0'0,0'0] MAX to 483'7413 2024-09-20T19:09:05.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[29421]: 11.17 starting backfill to osd.4 from (0'0,0'0] MAX to 483'7477 2024-09-20T19:09:05.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[29421]: 11.16 starting backfill to osd.1 from (0'0,0'0] MAX to 483'7565 2024-09-20T19:09:05.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[29421]: 11.15 starting backfill to osd.6 from (0'0,0'0] MAX to 483'7370 2024-09-20T19:09:05.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[29421]: 11.10 starting backfill to osd.3 from (0'0,0'0] MAX to 483'7454 2024-09-20T19:09:05.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[29421]: 11.16 starting backfill to osd.2 from (0'0,0'0] MAX to 483'7565 2024-09-20T19:09:05.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[29421]: 11.13 starting backfill to osd.6 from (0'0,442'3886] MIN to 483'7611 2024-09-20T19:09:05.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[29421]: 11.19 starting backfill to osd.3 from (0'0,0'0] MAX to 483'7480 2024-09-20T19:09:05.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[29421]: 11.18 starting backfill to osd.5 from (0'0,0'0] MAX to 483'7263 2024-09-20T19:09:05.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[29421]: 11.15 starting backfill to osd.7 from (0'0,0'0] MAX to 483'7370 2024-09-20T19:09:05.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[29421]: 11.11 starting backfill to osd.6 from (0'0,442'3889] MIN to 483'7528 2024-09-20T19:09:05.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[29421]: 11.19 starting backfill to osd.6 from (0'0,442'3866] MIN to 483'7480 2024-09-20T19:09:05.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[29421]: 11.14 starting backfill to osd.7 from (0'0,0'0] MAX to 483'7494 2024-09-20T19:09:05.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[29421]: 11.10 starting backfill to osd.7 from (0'0,0'0] MAX to 483'7454 2024-09-20T19:09:05.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[29421]: 11.13 starting backfill to osd.7 from (0'0,0'0] MAX to 483'7611 2024-09-20T19:09:05.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[29421]: 6.1 scrub starts 2024-09-20T19:09:05.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[29421]: 6.1 scrub ok 2024-09-20T19:09:05.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:09:05.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:09:05.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[36260]: 11.15 starting backfill to osd.5 from (0'0,0'0] MAX to 483'7370 2024-09-20T19:09:05.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[36260]: 11.14 starting backfill to osd.2 from (0'0,0'0] MAX to 483'7494 2024-09-20T19:09:05.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[36260]: 11.16 starting backfill to osd.0 from (0'0,0'0] MAX to 483'7565 2024-09-20T19:09:05.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[36260]: osdmap e486: 8 total, 8 up, 8 in 2024-09-20T19:09:05.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[36260]: 11.10 starting backfill to osd.1 from (0'0,470'6052] MIN to 483'7454 2024-09-20T19:09:05.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[36260]: 11.17 starting backfill to osd.1 from (0'0,458'4892] MIN to 483'7477 2024-09-20T19:09:05.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[36260]: 11.13 starting backfill to osd.1 from (426'459,473'6502] MIN to 483'7611 2024-09-20T19:09:05.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[36260]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-20T19:09:05.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[36260]: 11.18 starting backfill to osd.3 from (0'0,0'0] MAX to 483'7263 2024-09-20T19:09:05.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[36260]: 11.11 starting backfill to osd.0 from (0'0,0'0] MAX to 483'7528 2024-09-20T19:09:05.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[36260]: 11.12 starting backfill to osd.3 from (0'0,0'0] MAX to 483'7413 2024-09-20T19:09:05.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[36260]: 11.17 starting backfill to osd.4 from (0'0,0'0] MAX to 483'7477 2024-09-20T19:09:05.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[36260]: 11.16 starting backfill to osd.1 from (0'0,0'0] MAX to 483'7565 2024-09-20T19:09:05.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[36260]: 11.15 starting backfill to osd.6 from (0'0,0'0] MAX to 483'7370 2024-09-20T19:09:05.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[36260]: 11.10 starting backfill to osd.3 from (0'0,0'0] MAX to 483'7454 2024-09-20T19:09:05.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[36260]: 11.16 starting backfill to osd.2 from (0'0,0'0] MAX to 483'7565 2024-09-20T19:09:05.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[36260]: 11.13 starting backfill to osd.6 from (0'0,442'3886] MIN to 483'7611 2024-09-20T19:09:05.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[36260]: 11.19 starting backfill to osd.3 from (0'0,0'0] MAX to 483'7480 2024-09-20T19:09:05.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[36260]: 11.18 starting backfill to osd.5 from (0'0,0'0] MAX to 483'7263 2024-09-20T19:09:05.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[36260]: 11.15 starting backfill to osd.7 from (0'0,0'0] MAX to 483'7370 2024-09-20T19:09:05.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[36260]: 11.11 starting backfill to osd.6 from (0'0,442'3889] MIN to 483'7528 2024-09-20T19:09:05.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[36260]: 11.19 starting backfill to osd.6 from (0'0,442'3866] MIN to 483'7480 2024-09-20T19:09:05.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[36260]: 11.14 starting backfill to osd.7 from (0'0,0'0] MAX to 483'7494 2024-09-20T19:09:05.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[36260]: 11.10 starting backfill to osd.7 from (0'0,0'0] MAX to 483'7454 2024-09-20T19:09:05.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[36260]: 11.13 starting backfill to osd.7 from (0'0,0'0] MAX to 483'7611 2024-09-20T19:09:05.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[36260]: 6.1 scrub starts 2024-09-20T19:09:05.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:04 smithi083 ceph-mon[36260]: 6.1 scrub ok 2024-09-20T19:09:05.371 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:09:05.402 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:05.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:05.403 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:09:05.504 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:09:05.551 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:05.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:05.552 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:09:05.652 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:09:05.681 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:05.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:05.682 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:09:05.783 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:09:05.821 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:05.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:05.822 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:09:05.910 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T19:09:05.923 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:09:05.955 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:05.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:05.956 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:09:06.057 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:09:06.085 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:06.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:06.086 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:09:06.187 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:09:06.238 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:06.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:06.239 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:09:06.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:06 smithi083 ceph-mon[29421]: pgmap v980: 168 pgs: 2 remapped+peering, 2 peering, 164 active+clean; 2.0 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 957 B/s rd, 23 MiB/s wr, 987 op/s 2024-09-20T19:09:06.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:06 smithi083 ceph-mon[29421]: 3.6 scrub starts 2024-09-20T19:09:06.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:06 smithi083 ceph-mon[29421]: 3.6 scrub ok 2024-09-20T19:09:06.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:06 smithi083 ceph-mon[36260]: pgmap v980: 168 pgs: 2 remapped+peering, 2 peering, 164 active+clean; 2.0 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 957 B/s rd, 23 MiB/s wr, 987 op/s 2024-09-20T19:09:06.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:06 smithi083 ceph-mon[36260]: 3.6 scrub starts 2024-09-20T19:09:06.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:06 smithi083 ceph-mon[36260]: 3.6 scrub ok 2024-09-20T19:09:06.340 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:09:06.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:06.381 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:06.381 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:09:06.482 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:09:06.509 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:06.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:06.510 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:09:06.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:06 smithi142 ceph-mon[32508]: pgmap v980: 168 pgs: 2 remapped+peering, 2 peering, 164 active+clean; 2.0 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 957 B/s rd, 23 MiB/s wr, 987 op/s 2024-09-20T19:09:06.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:06 smithi142 ceph-mon[32508]: 3.6 scrub starts 2024-09-20T19:09:06.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:06 smithi142 ceph-mon[32508]: 3.6 scrub ok 2024-09-20T19:09:06.611 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:09:06.657 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T19:09:06.657 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:09:06.658 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:06.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:06.659 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:09:06.760 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:09:06.805 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:06.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:06.807 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:09:06.908 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:09:06.949 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:06.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:06.950 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:09:07.051 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:09:07.080 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:07.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:07.081 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:09:07.182 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:09:07.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:07.227 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:07.227 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:09:07.233 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 7, 6, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 5, 6, 7, 4] 2024-09-20T19:09:07.233 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T19:09:07.233 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-09-20T19:09:07.233 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-20T19:09:07.233 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-20T19:09:07.233 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.4 config set filestore_inject_stall 3 2024-09-20T19:09:07.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:07 smithi083 ceph-mon[29421]: 6.10 scrub starts 2024-09-20T19:09:07.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:07 smithi083 ceph-mon[29421]: 6.10 scrub ok 2024-09-20T19:09:07.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:07 smithi083 ceph-mon[36260]: 6.10 scrub starts 2024-09-20T19:09:07.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:07 smithi083 ceph-mon[36260]: 6.10 scrub ok 2024-09-20T19:09:07.329 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:09:07.366 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:07.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:07.367 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:09:07.463 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:09:07.468 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:09:07.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:07 smithi142 ceph-mon[32508]: 6.10 scrub starts 2024-09-20T19:09:07.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:07 smithi142 ceph-mon[32508]: 6.10 scrub ok 2024-09-20T19:09:07.512 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_blocked_ops 2024-09-20T19:09:07.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:07.514 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:07.514 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:09:07.615 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:09:07.618 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:09:07.649 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:07.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:07.650 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:09:07.751 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:09:07.781 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:07.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:07.783 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:09:07.884 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:09:07.929 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:07.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:07.930 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:09:07.941 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:09:07 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:09:08.031 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:09:08.059 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:08.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:08.060 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:09:08.119 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:09:08.120 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:09:08.120 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:08 smithi083 ceph-mon[36260]: pgmap v981: 168 pgs: 2 active+remapped+backfilling, 2 remapped+peering, 1 peering, 3 active+remapped+backfill_toofull, 160 active+clean; 1.8 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.7 KiB/s rd, 34 MiB/s wr, 1.45k op/s; 8054/89382 objects misplaced (9.011%); 25 MiB/s, 401 objects/s recovering 2024-09-20T19:09:08.120 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:08 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:09:08.120 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:08 smithi083 ceph-mon[29421]: pgmap v981: 168 pgs: 2 active+remapped+backfilling, 2 remapped+peering, 1 peering, 3 active+remapped+backfill_toofull, 160 active+clean; 1.8 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.7 KiB/s rd, 34 MiB/s wr, 1.45k op/s; 8054/89382 objects misplaced (9.011%); 25 MiB/s, 401 objects/s recovering 2024-09-20T19:09:08.120 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:08 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:09:08.161 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:09:08.162 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:09:08.162 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:09:08.202 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:09:08.203 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:09:08.203 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:09:08.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:08.219 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:08.219 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:09:08.246 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:09:08.246 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:09:08.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:08 smithi142 ceph-mon[32508]: pgmap v981: 168 pgs: 2 active+remapped+backfilling, 2 remapped+peering, 1 peering, 3 active+remapped+backfill_toofull, 160 active+clean; 1.8 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.7 KiB/s rd, 34 MiB/s wr, 1.45k op/s; 8054/89382 objects misplaced (9.011%); 25 MiB/s, 401 objects/s recovering 2024-09-20T19:09:08.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:08 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:09:08.276 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:09:08.276 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:09:08.309 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:09:08.309 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:09:08.320 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:09:08.344 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:09:08.344 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:09:08.361 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:08.361 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T19:09:08.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:08.362 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:09:08.382 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:09:08.382 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:09:08.416 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T19:09:08.416 INFO:teuthology.orchestra.run.smithi142.stdout: "success": "" 2024-09-20T19:09:08.416 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T19:09:08.463 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:09:08.516 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:08.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:08.517 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:09:08.618 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:09:08.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:08.656 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:08.657 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:09:08.757 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:09:08.813 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:08.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:08.814 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:09:08.914 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:09:08.948 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:08.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:08.949 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:09:09.018 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T19:09:09.018 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-20T19:09:09.018 INFO:teuthology.orchestra.run.smithi083.stdout: "complaint_time": 30, 2024-09-20T19:09:09.018 INFO:teuthology.orchestra.run.smithi083.stdout: "num_blocked_ops": 0 2024-09-20T19:09:09.018 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T19:09:09.050 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:09:09.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:09.090 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:09.090 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:09:09.191 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:09:09.219 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:09.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:09.220 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:09:09.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:09 smithi142 ceph-mon[32508]: 4.1b scrub starts 2024-09-20T19:09:09.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:09 smithi142 ceph-mon[32508]: 4.1b scrub ok 2024-09-20T19:09:09.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:09 smithi142 ceph-mon[32508]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T19:09:09.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:09 smithi142 ceph-mon[32508]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-20T19:09:09.261 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:09 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2206235018' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:09:09.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:09 smithi083 ceph-mon[29421]: 4.1b scrub starts 2024-09-20T19:09:09.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:09 smithi083 ceph-mon[29421]: 4.1b scrub ok 2024-09-20T19:09:09.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:09 smithi083 ceph-mon[29421]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T19:09:09.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:09 smithi083 ceph-mon[29421]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-20T19:09:09.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:09 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2206235018' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:09:09.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:09 smithi083 ceph-mon[36260]: 4.1b scrub starts 2024-09-20T19:09:09.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:09 smithi083 ceph-mon[36260]: 4.1b scrub ok 2024-09-20T19:09:09.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:09 smithi083 ceph-mon[36260]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T19:09:09.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:09 smithi083 ceph-mon[36260]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-20T19:09:09.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:09 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2206235018' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:09:09.322 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:09:09.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:09.366 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:09.367 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:09:09.468 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:09:09.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:09.502 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:09.502 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:09:09.603 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:09:09.641 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:09.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:09.642 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:09:09.742 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:09:09.776 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:09.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:09.824 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:09:09.925 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:09:09.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:09.961 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:09.962 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:09:10.062 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:09:10.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:10.096 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:10.096 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:09:10.171 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:09:10.186 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_historic_ops 2024-09-20T19:09:10.196 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:09:10.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:10.292 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:10.293 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:09:10.322 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:10 smithi083 ceph-mon[29421]: 4.e scrub starts 2024-09-20T19:09:10.322 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:10 smithi083 ceph-mon[29421]: 4.e scrub ok 2024-09-20T19:09:10.322 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:10 smithi083 ceph-mon[29421]: pgmap v982: 168 pgs: 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 158 active+clean; 1.6 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 705 B/s rd, 0 B/s wr, 617 op/s; 16867/81543 objects misplaced (20.685%); 9.6 MiB/s, 153 objects/s recovering 2024-09-20T19:09:10.323 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:10 smithi083 ceph-mon[36260]: 4.e scrub starts 2024-09-20T19:09:10.323 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:10 smithi083 ceph-mon[36260]: 4.e scrub ok 2024-09-20T19:09:10.323 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:10 smithi083 ceph-mon[36260]: pgmap v982: 168 pgs: 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 158 active+clean; 1.6 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 705 B/s rd, 0 B/s wr, 617 op/s; 16867/81543 objects misplaced (20.685%); 9.6 MiB/s, 153 objects/s recovering 2024-09-20T19:09:10.393 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:09:10.425 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:10.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:10.427 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:09:10.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:10 smithi142 ceph-mon[32508]: 4.e scrub starts 2024-09-20T19:09:10.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:10 smithi142 ceph-mon[32508]: 4.e scrub ok 2024-09-20T19:09:10.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:10 smithi142 ceph-mon[32508]: pgmap v982: 168 pgs: 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 158 active+clean; 1.6 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 705 B/s rd, 0 B/s wr, 617 op/s; 16867/81543 objects misplaced (20.685%); 9.6 MiB/s, 153 objects/s recovering 2024-09-20T19:09:10.528 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:09:10.556 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:10.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:10.558 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:09:10.658 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:09:10.704 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:10.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:10.705 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:09:10.806 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:09:10.844 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:10.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:10.845 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:09:10.946 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:09:10.972 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:10.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:10.974 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:09:11.003 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:09:10 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:09:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:09:11.074 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:09:11.111 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:11.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:11.112 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:09:11.213 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:09:11.252 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:11.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:11.253 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:09:11.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:11 smithi083 ceph-mon[29421]: 6.2 scrub starts 2024-09-20T19:09:11.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:11 smithi083 ceph-mon[29421]: 6.2 scrub ok 2024-09-20T19:09:11.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:11 smithi083 ceph-mon[29421]: 4.b scrub starts 2024-09-20T19:09:11.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:11 smithi083 ceph-mon[29421]: 4.b scrub ok 2024-09-20T19:09:11.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:11 smithi083 ceph-mon[36260]: 6.2 scrub starts 2024-09-20T19:09:11.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:11 smithi083 ceph-mon[36260]: 6.2 scrub ok 2024-09-20T19:09:11.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:11 smithi083 ceph-mon[36260]: 4.b scrub starts 2024-09-20T19:09:11.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:11 smithi083 ceph-mon[36260]: 4.b scrub ok 2024-09-20T19:09:11.354 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:09:11.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:11.405 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:11.405 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:09:11.467 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T19:09:11.506 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:09:11.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:11 smithi142 ceph-mon[32508]: 6.2 scrub starts 2024-09-20T19:09:11.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:11 smithi142 ceph-mon[32508]: 6.2 scrub ok 2024-09-20T19:09:11.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:11 smithi142 ceph-mon[32508]: 4.b scrub starts 2024-09-20T19:09:11.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:11 smithi142 ceph-mon[32508]: 4.b scrub ok 2024-09-20T19:09:11.520 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:09:11.569 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:11.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:11.570 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:09:11.671 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:09:11.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:11.721 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:11.721 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:09:11.822 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:09:11.856 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:11.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:11.857 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:09:11.957 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:09:11.995 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:11.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:11.996 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:09:12.062 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-20T19:09:12.097 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:09:12.124 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:12.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:12.125 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:09:12.173 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:12 smithi083 ceph-mon[29421]: pgmap v983: 168 pgs: 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 158 active+clean; 1.5 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 620 B/s rd, 0 B/s wr, 1.13k op/s; 16075/76635 objects misplaced (20.976%); 24 MiB/s, 386 objects/s recovering 2024-09-20T19:09:12.173 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:12 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:09:12.173 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:12 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:09:12.173 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:12 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2194647180' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:09:12.174 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:12 smithi083 ceph-mon[29421]: 6.16 scrub starts 2024-09-20T19:09:12.174 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:12 smithi083 ceph-mon[29421]: 6.16 scrub ok 2024-09-20T19:09:12.174 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:12 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:09:12.174 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:12 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:09:12.174 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:12 smithi083 ceph-mon[36260]: pgmap v983: 168 pgs: 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 158 active+clean; 1.5 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 620 B/s rd, 0 B/s wr, 1.13k op/s; 16075/76635 objects misplaced (20.976%); 24 MiB/s, 386 objects/s recovering 2024-09-20T19:09:12.174 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:12 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:09:12.174 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:12 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:09:12.174 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:12 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2194647180' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:09:12.174 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:12 smithi083 ceph-mon[36260]: 6.16 scrub starts 2024-09-20T19:09:12.174 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:12 smithi083 ceph-mon[36260]: 6.16 scrub ok 2024-09-20T19:09:12.174 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:12 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:09:12.175 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:12 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:09:12.224 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T19:09:12.225 INFO:teuthology.orchestra.run.smithi083.stdout: "size": 20, 2024-09-20T19:09:12.225 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 600, 2024-09-20T19:09:12.225 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-20T19:09:12.225 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.225 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:142913 9.c 9:3d798970:::benchmark_data_smithi083_124552_object28766:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e291)", 2024-09-20T19:09:12.225 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:35.168460+0000", 2024-09-20T19:09:12.225 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 517.05078007999998, 2024-09-20T19:09:12.225 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.040925943, 2024-09-20T19:09:12.225 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:09:12.225 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:09:12.225 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-20T19:09:12.225 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:09:12.225 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:09:12.226 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:09:12.226 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 142913 2024-09-20T19:09:12.226 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.226 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:09:12.226 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.226 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:09:12.226 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.168460+0000", 2024-09-20T19:09:12.226 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:09:12.226 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.226 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.226 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:09:12.226 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.168460+0000", 2024-09-20T19:09:12.226 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:09:12.226 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.227 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.227 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:09:12.227 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.168462+0000", 2024-09-20T19:09:12.227 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9520000000000001e-06 2024-09-20T19:09:12.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.227 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.227 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:09:12.227 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.168469+0000", 2024-09-20T19:09:12.227 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6309999999999997e-06 2024-09-20T19:09:12.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.227 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.227 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:09:12.227 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.168485+0000", 2024-09-20T19:09:12.227 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5798e-05 2024-09-20T19:09:12.228 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.228 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.228 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:09:12.228 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.168492+0000", 2024-09-20T19:09:12.228 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.9179999999999994e-06 2024-09-20T19:09:12.228 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.228 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.228 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:09:12.228 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.168833+0000", 2024-09-20T19:09:12.228 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00034100200000000003 2024-09-20T19:09:12.228 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.228 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.228 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:09:12.229 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.169144+0000", 2024-09-20T19:09:12.229 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00031083499999999997 2024-09-20T19:09:12.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.229 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.229 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4,7", 2024-09-20T19:09:12.229 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.169258+0000", 2024-09-20T19:09:12.229 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00011411 2024-09-20T19:09:12.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.229 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.229 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:09:12.229 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.172245+0000", 2024-09-20T19:09:12.229 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002987048 2024-09-20T19:09:12.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.230 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.230 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:12.230 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.203777+0000", 2024-09-20T19:09:12.230 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.031531525999999997 2024-09-20T19:09:12.230 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.230 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.230 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:09:12.230 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.213031+0000", 2024-09-20T19:09:12.230 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0092539930000000003 2024-09-20T19:09:12.230 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.230 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.230 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:09:12.230 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.213039+0000", 2024-09-20T19:09:12.230 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4090000000000003e-06 2024-09-20T19:09:12.230 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.231 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.231 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:09:12.231 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.209326+0000", 2024-09-20T19:09:12.231 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.99628679899999995 2024-09-20T19:09:12.231 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.231 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.231 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:09:12.231 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.209386+0000", 2024-09-20T19:09:12.231 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9922e-05 2024-09-20T19:09:12.231 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:09:12.231 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:09:12.231 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:09:12.231 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.231 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.231 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:142917 9.c 9:32f2df36:::benchmark_data_smithi083_124552_object28770:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e291)", 2024-09-20T19:09:12.231 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:35.172691+0000", 2024-09-20T19:09:12.232 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 517.04654971100001, 2024-09-20T19:09:12.232 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0372109220000001, 2024-09-20T19:09:12.232 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:09:12.232 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:09:12.232 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-20T19:09:12.232 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:09:12.232 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:09:12.232 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:09:12.232 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 142917 2024-09-20T19:09:12.232 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.232 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:09:12.232 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.232 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:09:12.232 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.172691+0000", 2024-09-20T19:09:12.232 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:09:12.232 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.233 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.233 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:09:12.233 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.172691+0000", 2024-09-20T19:09:12.233 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:09:12.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.233 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.233 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:09:12.233 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.172692+0000", 2024-09-20T19:09:12.233 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.95e-06 2024-09-20T19:09:12.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.233 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.233 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:09:12.233 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.172701+0000", 2024-09-20T19:09:12.233 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0630000000000006e-06 2024-09-20T19:09:12.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.233 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.234 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:09:12.234 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.172714+0000", 2024-09-20T19:09:12.234 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3303e-05 2024-09-20T19:09:12.234 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.234 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.234 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:09:12.234 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.172723+0000", 2024-09-20T19:09:12.234 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.6160000000000008e-06 2024-09-20T19:09:12.234 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.234 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.234 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:09:12.234 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.172857+0000", 2024-09-20T19:09:12.234 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000133829 2024-09-20T19:09:12.234 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.234 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.235 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:09:12.235 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.172975+0000", 2024-09-20T19:09:12.235 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000117224 2024-09-20T19:09:12.235 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.235 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.235 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 4,7", 2024-09-20T19:09:12.235 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.173088+0000", 2024-09-20T19:09:12.235 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000113551 2024-09-20T19:09:12.235 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.235 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.235 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:09:12.235 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.202597+0000", 2024-09-20T19:09:12.235 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.029508818999999999 2024-09-20T19:09:12.235 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.235 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.235 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:12.235 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.203835+0000", 2024-09-20T19:09:12.236 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001238444 2024-09-20T19:09:12.236 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.236 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.236 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:09:12.236 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.215654+0000", 2024-09-20T19:09:12.236 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011818403999999999 2024-09-20T19:09:12.236 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.236 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.236 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:09:12.236 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.215699+0000", 2024-09-20T19:09:12.236 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5084999999999999e-05 2024-09-20T19:09:12.236 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.236 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.236 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:09:12.236 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.209843+0000", 2024-09-20T19:09:12.236 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.99414379500000005 2024-09-20T19:09:12.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.237 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.237 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:09:12.237 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.209901+0000", 2024-09-20T19:09:12.237 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8838999999999997e-05 2024-09-20T19:09:12.237 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:09:12.237 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:09:12.237 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:09:12.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.237 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.237 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:142919 9.c 9:3aa96e52:::benchmark_data_smithi083_124552_object28772:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e291)", 2024-09-20T19:09:12.237 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:35.175226+0000", 2024-09-20T19:09:12.237 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 517.04401446600002, 2024-09-20T19:09:12.238 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:09:12.239 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.044694204, 2024-09-20T19:09:12.239 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:09:12.239 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:09:12.239 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:09:12.239 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:09:12.239 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:09:12.239 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:09:12.239 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 142919 2024-09-20T19:09:12.239 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.239 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:09:12.239 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.240 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:09:12.240 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.175226+0000", 2024-09-20T19:09:12.240 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:09:12.240 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.240 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.240 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:09:12.240 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.175226+0000", 2024-09-20T19:09:12.240 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:09:12.240 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.240 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.240 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:09:12.240 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.175228+0000", 2024-09-20T19:09:12.240 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.88e-06 2024-09-20T19:09:12.240 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.240 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.240 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:09:12.240 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.175235+0000", 2024-09-20T19:09:12.241 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.8419999999999999e-06 2024-09-20T19:09:12.241 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.241 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.241 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:09:12.241 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.175253+0000", 2024-09-20T19:09:12.241 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.8995e-05 2024-09-20T19:09:12.241 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.241 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.241 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:09:12.241 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.175262+0000", 2024-09-20T19:09:12.241 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.884e-06 2024-09-20T19:09:12.241 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.241 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.241 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:09:12.241 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.215752+0000", 2024-09-20T19:09:12.242 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.040489565999999998 2024-09-20T19:09:12.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.242 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.242 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:09:12.242 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.215759+0000", 2024-09-20T19:09:12.242 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.1380000000000001e-06 2024-09-20T19:09:12.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.242 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.242 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:09:12.242 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.209925+0000", 2024-09-20T19:09:12.242 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.99416557900000002 2024-09-20T19:09:12.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.242 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.242 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:09:12.243 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.210025+0000", 2024-09-20T19:09:12.243 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010050700000000001 2024-09-20T19:09:12.243 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.243 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.243 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 7", 2024-09-20T19:09:12.243 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.210147+0000", 2024-09-20T19:09:12.243 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000122309 2024-09-20T19:09:12.243 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.243 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.243 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:12.243 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.219740+0000", 2024-09-20T19:09:12.243 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0095921180000000002 2024-09-20T19:09:12.243 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.243 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.243 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:09:12.244 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.219768+0000", 2024-09-20T19:09:12.244 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8391e-05 2024-09-20T19:09:12.244 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.244 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.244 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:09:12.244 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.219815+0000", 2024-09-20T19:09:12.244 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7197999999999998e-05 2024-09-20T19:09:12.244 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.244 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.244 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:09:12.244 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.219920+0000", 2024-09-20T19:09:12.244 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010479700000000001 2024-09-20T19:09:12.244 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:09:12.244 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:09:12.244 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:09:12.245 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.245 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.245 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.14460.0:2698 5.0 5:05bf5b68:::notify.1:head [watch reconnect cookie 94056120167808 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e292)", 2024-09-20T19:09:12.245 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:35.178092+0000", 2024-09-20T19:09:12.245 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 517.04114838199996, 2024-09-20T19:09:12.245 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.021665381, 2024-09-20T19:09:12.245 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:09:12.245 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:09:12.245 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-20T19:09:12.245 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:09:12.245 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.14460", 2024-09-20T19:09:12.245 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1555339426", 2024-09-20T19:09:12.245 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 2698 2024-09-20T19:09:12.245 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.245 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:09:12.246 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.246 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:09:12.246 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.178092+0000", 2024-09-20T19:09:12.246 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:09:12.246 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.246 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.246 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:09:12.246 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.178092+0000", 2024-09-20T19:09:12.246 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:09:12.246 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.246 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.246 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:09:12.246 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.178093+0000", 2024-09-20T19:09:12.246 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2359999999999999e-06 2024-09-20T19:09:12.247 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.247 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.247 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:09:12.247 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.178098+0000", 2024-09-20T19:09:12.247 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1410000000000004e-06 2024-09-20T19:09:12.247 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.247 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.247 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:09:12.247 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.178113+0000", 2024-09-20T19:09:12.247 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4262e-05 2024-09-20T19:09:12.247 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.247 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.247 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:09:12.247 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.178120+0000", 2024-09-20T19:09:12.247 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.6210000000000004e-06 2024-09-20T19:09:12.247 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.248 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.248 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:09:12.248 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.180022+0000", 2024-09-20T19:09:12.248 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001901668 2024-09-20T19:09:12.248 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.248 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.248 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op must wait for map", 2024-09-20T19:09:12.248 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.180028+0000", 2024-09-20T19:09:12.248 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9359999999999996e-06 2024-09-20T19:09:12.248 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.248 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.248 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:09:12.248 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.195134+0000", 2024-09-20T19:09:12.248 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.015105831 2024-09-20T19:09:12.249 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.249 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.249 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:09:12.249 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.195142+0000", 2024-09-20T19:09:12.249 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8659999999999995e-06 2024-09-20T19:09:12.249 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.249 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.249 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:09:12.249 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.192667+0000", 2024-09-20T19:09:12.249 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.99752498700000003 2024-09-20T19:09:12.249 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.249 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.249 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:09:12.249 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.192972+0000", 2024-09-20T19:09:12.249 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000304677 2024-09-20T19:09:12.250 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.250 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.250 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:09:12.250 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.199757+0000", 2024-09-20T19:09:12.250 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0067851559999999997 2024-09-20T19:09:12.250 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:09:12.250 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:09:12.250 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:09:12.250 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.250 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.250 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:142931 9.c 9:3971d803:::benchmark_data_smithi083_124552_object28784:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e292)", 2024-09-20T19:09:12.250 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:35.206532+0000", 2024-09-20T19:09:12.250 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 517.01270871300005, 2024-09-20T19:09:12.250 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.016374055, 2024-09-20T19:09:12.250 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:09:12.251 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:09:12.251 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:09:12.251 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:09:12.251 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:09:12.251 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:09:12.251 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 142931 2024-09-20T19:09:12.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.251 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:09:12.251 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.251 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:09:12.251 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.206532+0000", 2024-09-20T19:09:12.251 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:09:12.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.251 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.252 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:09:12.252 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.206532+0000", 2024-09-20T19:09:12.252 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:09:12.252 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.252 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.252 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:09:12.252 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.206532+0000", 2024-09-20T19:09:12.252 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9500000000000001e-07 2024-09-20T19:09:12.252 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.252 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.252 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:09:12.252 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.206537+0000", 2024-09-20T19:09:12.252 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5600000000000004e-06 2024-09-20T19:09:12.253 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.253 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.253 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:09:12.253 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.206546+0000", 2024-09-20T19:09:12.253 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9479999999999997e-06 2024-09-20T19:09:12.253 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.253 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.253 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:09:12.253 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.206553+0000", 2024-09-20T19:09:12.253 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0119999999999999e-06 2024-09-20T19:09:12.253 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.253 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.253 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:09:12.253 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.215793+0000", 2024-09-20T19:09:12.253 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0092399519999999992 2024-09-20T19:09:12.253 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.253 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.254 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:09:12.254 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.215799+0000", 2024-09-20T19:09:12.254 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7389999999999996e-06 2024-09-20T19:09:12.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.254 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.254 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:09:12.254 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.212804+0000", 2024-09-20T19:09:12.254 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.99700544700000004 2024-09-20T19:09:12.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.254 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.254 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:09:12.254 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.213100+0000", 2024-09-20T19:09:12.254 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00029572499999999999 2024-09-20T19:09:12.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.254 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.255 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 7", 2024-09-20T19:09:12.255 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.213492+0000", 2024-09-20T19:09:12.255 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000391939 2024-09-20T19:09:12.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.255 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.255 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:09:12.255 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.219928+0000", 2024-09-20T19:09:12.255 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0064358100000000001 2024-09-20T19:09:12.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.255 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.255 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:12.255 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.222687+0000", 2024-09-20T19:09:12.255 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0027591949999999999 2024-09-20T19:09:12.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.255 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.256 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:09:12.256 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.222797+0000", 2024-09-20T19:09:12.256 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000110713 2024-09-20T19:09:12.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.256 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.256 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:09:12.256 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.222906+0000", 2024-09-20T19:09:12.256 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00010812 2024-09-20T19:09:12.256 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:09:12.256 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:09:12.256 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:09:12.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.256 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.256 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.25484.0:142996 9.c 9:3cdc78e2:::benchmark_data_smithi083_124552_object28849:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e292)", 2024-09-20T19:09:12.256 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:00:35.284730+0000", 2024-09-20T19:09:12.256 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 516.93451054399998, 2024-09-20T19:09:12.256 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.93861088699999995, 2024-09-20T19:09:12.257 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:09:12.257 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:09:12.257 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:09:12.257 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:09:12.257 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.25484", 2024-09-20T19:09:12.257 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/1107247030", 2024-09-20T19:09:12.257 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 142996 2024-09-20T19:09:12.257 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.257 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:09:12.257 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.257 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:09:12.257 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.284730+0000", 2024-09-20T19:09:12.257 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:09:12.257 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.258 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.258 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:09:12.258 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.284730+0000", 2024-09-20T19:09:12.258 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:09:12.258 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.258 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.258 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:09:12.258 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.284731+0000", 2024-09-20T19:09:12.258 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.376e-06 2024-09-20T19:09:12.258 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.258 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.258 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:09:12.258 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.284736+0000", 2024-09-20T19:09:12.259 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1680000000000003e-06 2024-09-20T19:09:12.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.259 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:09:12.259 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.284747+0000", 2024-09-20T19:09:12.259 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1077000000000001e-05 2024-09-20T19:09:12.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.259 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:09:12.259 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.284753+0000", 2024-09-20T19:09:12.259 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.874e-06 2024-09-20T19:09:12.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.259 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:09:12.260 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.284810+0000", 2024-09-20T19:09:12.260 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6573000000000002e-05 2024-09-20T19:09:12.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.260 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.260 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:09:12.260 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:35.284816+0000", 2024-09-20T19:09:12.260 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4840000000000001e-06 2024-09-20T19:09:12.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.260 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.260 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:09:12.260 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.216111+0000", 2024-09-20T19:09:12.260 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.93129438899999994 2024-09-20T19:09:12.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.260 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.260 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:09:12.260 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.216258+0000", 2024-09-20T19:09:12.261 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014760700000000001 2024-09-20T19:09:12.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.261 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.261 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 7", 2024-09-20T19:09:12.261 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.217033+0000", 2024-09-20T19:09:12.261 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00077462600000000005 2024-09-20T19:09:12.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.261 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.261 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:12.261 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.223055+0000", 2024-09-20T19:09:12.261 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0060222510000000002 2024-09-20T19:09:12.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.261 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.261 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:09:12.261 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.223067+0000", 2024-09-20T19:09:12.261 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2087e-05 2024-09-20T19:09:12.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.262 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.262 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:09:12.262 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.223265+0000", 2024-09-20T19:09:12.262 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00019747800000000001 2024-09-20T19:09:12.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.262 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.262 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:09:12.262 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:00:36.223341+0000", 2024-09-20T19:09:12.262 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5896999999999995e-05 2024-09-20T19:09:12.262 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:09:12.262 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:09:12.262 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:09:12.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.262 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.262 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.26264.0:60242 11.b 11:d0afd171:::benchmark_data_smithi083_177491_object60241:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e440)", 2024-09-20T19:09:12.263 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:06:54.885920+0000", 2024-09-20T19:09:12.263 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 137.333320391, 2024-09-20T19:09:12.263 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.34507496999999998, 2024-09-20T19:09:12.263 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:09:12.263 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:09:12.263 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:09:12.263 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:09:12.263 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.26264", 2024-09-20T19:09:12.263 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2608780316", 2024-09-20T19:09:12.263 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 60242 2024-09-20T19:09:12.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.263 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:09:12.263 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.263 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:09:12.263 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:54.885920+0000", 2024-09-20T19:09:12.263 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:09:12.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.264 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:09:12.264 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:54.885920+0000", 2024-09-20T19:09:12.264 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:09:12.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.264 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:09:12.264 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:54.885921+0000", 2024-09-20T19:09:12.264 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2800000000000005e-07 2024-09-20T19:09:12.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.264 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.266 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:09:12.266 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:54.885946+0000", 2024-09-20T19:09:12.266 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4783e-05 2024-09-20T19:09:12.266 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.266 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.266 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:09:12.266 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:54.885956+0000", 2024-09-20T19:09:12.266 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0437e-05 2024-09-20T19:09:12.266 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.266 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.266 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:09:12.267 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:54.885963+0000", 2024-09-20T19:09:12.267 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2219999999999996e-06 2024-09-20T19:09:12.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.267 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.267 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:09:12.267 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:54.886021+0000", 2024-09-20T19:09:12.267 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8193999999999999e-05 2024-09-20T19:09:12.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.267 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.267 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:09:12.267 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:54.886174+0000", 2024-09-20T19:09:12.267 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00015244799999999999 2024-09-20T19:09:12.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.268 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.268 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,1", 2024-09-20T19:09:12.268 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:54.886310+0000", 2024-09-20T19:09:12.268 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000136627 2024-09-20T19:09:12.268 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.268 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.268 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:09:12.268 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:54.891528+0000", 2024-09-20T19:09:12.268 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0052171520000000001 2024-09-20T19:09:12.268 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.268 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.268 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:12.268 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:54.892025+0000", 2024-09-20T19:09:12.268 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000496997 2024-09-20T19:09:12.268 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.269 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.269 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:12.269 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:55.230889+0000", 2024-09-20T19:09:12.269 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.33886417499999999 2024-09-20T19:09:12.269 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.269 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.269 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:09:12.269 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:55.230948+0000", 2024-09-20T19:09:12.269 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9388999999999997e-05 2024-09-20T19:09:12.269 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.269 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.269 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:09:12.269 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:55.230995+0000", 2024-09-20T19:09:12.270 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.6618e-05 2024-09-20T19:09:12.270 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:09:12.270 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:09:12.270 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:09:12.270 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.270 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.270 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.26264.0:60551 11.19 11:9ce1c71f:::benchmark_data_smithi083_177491_object60550:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e441)", 2024-09-20T19:09:12.270 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:06:55.432029+0000", 2024-09-20T19:09:12.270 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 136.78721152899999, 2024-09-20T19:09:12.270 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0551587469999999, 2024-09-20T19:09:12.270 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:09:12.270 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:09:12.270 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-20T19:09:12.270 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:09:12.271 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.26264", 2024-09-20T19:09:12.271 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2608780316", 2024-09-20T19:09:12.271 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 60551 2024-09-20T19:09:12.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.271 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:09:12.271 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.271 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:09:12.271 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:55.432029+0000", 2024-09-20T19:09:12.271 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:09:12.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.271 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.271 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:09:12.271 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:55.432029+0000", 2024-09-20T19:09:12.271 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:09:12.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.272 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.272 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:09:12.272 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:55.432031+0000", 2024-09-20T19:09:12.272 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.351e-06 2024-09-20T19:09:12.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.272 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.272 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:09:12.272 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:55.432060+0000", 2024-09-20T19:09:12.272 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8540000000000001e-05 2024-09-20T19:09:12.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.272 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.272 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:09:12.272 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:55.432081+0000", 2024-09-20T19:09:12.272 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1557999999999999e-05 2024-09-20T19:09:12.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.273 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.273 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:09:12.273 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:55.432093+0000", 2024-09-20T19:09:12.273 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1555e-05 2024-09-20T19:09:12.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.273 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.273 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:09:12.273 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:55.589623+0000", 2024-09-20T19:09:12.273 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.15753077500000001 2024-09-20T19:09:12.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.273 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.273 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:09:12.273 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:55.589634+0000", 2024-09-20T19:09:12.273 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0214000000000001e-05 2024-09-20T19:09:12.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.274 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.274 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:09:12.274 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:56.487084+0000", 2024-09-20T19:09:12.274 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.89744985799999999 2024-09-20T19:09:12.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.274 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.274 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:09:12.274 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:56.487187+0000", 2024-09-20T19:09:12.274 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000103896 2024-09-20T19:09:12.274 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:09:12.274 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:09:12.274 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:09:12.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.275 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.275 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.26264.0:60565 11.19 11:9fcd43bb:::benchmark_data_smithi083_177491_object60564:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e441)", 2024-09-20T19:09:12.275 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:06:55.435636+0000", 2024-09-20T19:09:12.275 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 136.78360383200001, 2024-09-20T19:09:12.275 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.056329088, 2024-09-20T19:09:12.275 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:09:12.275 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:09:12.275 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:09:12.275 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:09:12.275 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.26264", 2024-09-20T19:09:12.275 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2608780316", 2024-09-20T19:09:12.275 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 60565 2024-09-20T19:09:12.275 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.275 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:09:12.275 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.276 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:09:12.276 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:55.435636+0000", 2024-09-20T19:09:12.276 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:09:12.276 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.276 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.276 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:09:12.276 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:55.435636+0000", 2024-09-20T19:09:12.276 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:09:12.276 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.276 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.276 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:09:12.276 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:55.435638+0000", 2024-09-20T19:09:12.276 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4640000000000001e-06 2024-09-20T19:09:12.276 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.277 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.277 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:09:12.277 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:55.435666+0000", 2024-09-20T19:09:12.277 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7923e-05 2024-09-20T19:09:12.277 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.277 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.277 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:09:12.277 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:55.435683+0000", 2024-09-20T19:09:12.277 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6888000000000002e-05 2024-09-20T19:09:12.277 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.277 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.277 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:09:12.277 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:55.435692+0000", 2024-09-20T19:09:12.277 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.1579999999999994e-06 2024-09-20T19:09:12.277 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.277 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.278 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:09:12.278 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:55.589660+0000", 2024-09-20T19:09:12.278 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.15396837599999999 2024-09-20T19:09:12.278 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.278 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.278 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:09:12.278 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:55.589666+0000", 2024-09-20T19:09:12.278 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0170000000000002e-06 2024-09-20T19:09:12.278 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.278 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.278 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:09:12.278 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:56.487581+0000", 2024-09-20T19:09:12.278 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.89791522199999996 2024-09-20T19:09:12.278 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.278 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.279 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:09:12.279 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:56.487721+0000", 2024-09-20T19:09:12.279 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000139123 2024-09-20T19:09:12.279 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.279 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.279 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,6", 2024-09-20T19:09:12.279 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:56.487862+0000", 2024-09-20T19:09:12.279 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00014189699999999999 2024-09-20T19:09:12.279 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.279 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.279 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:09:12.279 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:56.490522+0000", 2024-09-20T19:09:12.279 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0026595220000000001 2024-09-20T19:09:12.279 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.280 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.280 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:12.280 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:56.491109+0000", 2024-09-20T19:09:12.280 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00058726099999999995 2024-09-20T19:09:12.280 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.280 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.280 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:12.280 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:56.491656+0000", 2024-09-20T19:09:12.280 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00054673799999999995 2024-09-20T19:09:12.280 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.280 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.280 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:09:12.280 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:56.491923+0000", 2024-09-20T19:09:12.280 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00026705499999999998 2024-09-20T19:09:12.280 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.281 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.281 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:09:12.281 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:56.491966+0000", 2024-09-20T19:09:12.281 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2444000000000002e-05 2024-09-20T19:09:12.281 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:09:12.281 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:09:12.281 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:09:12.281 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.281 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.281 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.26264.0:62064 11.18 11:1af55d31:::benchmark_data_smithi083_177491_object62063:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e444)", 2024-09-20T19:09:12.281 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:06:58.449422+0000", 2024-09-20T19:09:12.281 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 133.769817999, 2024-09-20T19:09:12.281 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2400463319999999, 2024-09-20T19:09:12.281 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:09:12.282 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:09:12.282 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:09:12.282 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:09:12.282 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.26264", 2024-09-20T19:09:12.282 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2608780316", 2024-09-20T19:09:12.282 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 62064 2024-09-20T19:09:12.282 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.282 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:09:12.282 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.282 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:09:12.282 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:58.449422+0000", 2024-09-20T19:09:12.282 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:09:12.282 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.282 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.283 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:09:12.283 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:58.449422+0000", 2024-09-20T19:09:12.283 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:09:12.283 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.283 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.283 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:09:12.283 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:58.449424+0000", 2024-09-20T19:09:12.283 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.9530000000000002e-06 2024-09-20T19:09:12.283 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.283 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.283 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:09:12.283 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:58.449488+0000", 2024-09-20T19:09:12.283 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3467999999999996e-05 2024-09-20T19:09:12.283 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.283 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.283 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:09:12.284 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:58.449510+0000", 2024-09-20T19:09:12.284 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2575999999999999e-05 2024-09-20T19:09:12.284 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.284 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.284 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:09:12.284 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:58.449520+0000", 2024-09-20T19:09:12.284 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.9750000000000002e-06 2024-09-20T19:09:12.284 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.284 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.284 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:09:12.284 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:58.449591+0000", 2024-09-20T19:09:12.284 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0576999999999999e-05 2024-09-20T19:09:12.284 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.284 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.284 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op must wait for map", 2024-09-20T19:09:12.284 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:58.449598+0000", 2024-09-20T19:09:12.285 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.9460000000000001e-06 2024-09-20T19:09:12.285 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.285 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.285 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:09:12.285 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:58.459127+0000", 2024-09-20T19:09:12.285 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0095287800000000006 2024-09-20T19:09:12.285 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.285 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.285 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:09:12.285 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:58.459136+0000", 2024-09-20T19:09:12.285 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7310000000000005e-06 2024-09-20T19:09:12.285 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.285 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.285 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:09:12.286 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:59.666423+0000", 2024-09-20T19:09:12.286 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2072863620000001 2024-09-20T19:09:12.286 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.286 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.286 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:09:12.286 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:59.666612+0000", 2024-09-20T19:09:12.286 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000189197 2024-09-20T19:09:12.286 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.286 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.286 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,4,6", 2024-09-20T19:09:12.286 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:59.666800+0000", 2024-09-20T19:09:12.286 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000187813 2024-09-20T19:09:12.286 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.287 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.287 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:12.287 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:59.670074+0000", 2024-09-20T19:09:12.287 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0032747499999999999 2024-09-20T19:09:12.287 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.287 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.287 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:09:12.287 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:59.670091+0000", 2024-09-20T19:09:12.287 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.624e-05 2024-09-20T19:09:12.287 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.287 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.287 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:12.287 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:59.670125+0000", 2024-09-20T19:09:12.287 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4858999999999999e-05 2024-09-20T19:09:12.288 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.288 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.288 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:12.288 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:59.687452+0000", 2024-09-20T19:09:12.288 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.017326946999999999 2024-09-20T19:09:12.288 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.288 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.288 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:09:12.288 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:59.687664+0000", 2024-09-20T19:09:12.288 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00021162000000000001 2024-09-20T19:09:12.288 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.288 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.288 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:09:12.288 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:59.689469+0000", 2024-09-20T19:09:12.288 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0018045380000000001 2024-09-20T19:09:12.289 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:09:12.289 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:09:12.289 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:09:12.289 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.289 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.289 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.26264.0:62111 11.18 11:1a724455:::benchmark_data_smithi083_177491_object62110:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e444)", 2024-09-20T19:09:12.289 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:06:58.449552+0000", 2024-09-20T19:09:12.289 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 133.76968795100001, 2024-09-20T19:09:12.289 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2420059270000001, 2024-09-20T19:09:12.289 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:09:12.289 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:09:12.289 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:09:12.289 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:09:12.290 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.26264", 2024-09-20T19:09:12.290 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2608780316", 2024-09-20T19:09:12.290 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 62111 2024-09-20T19:09:12.290 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.290 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:09:12.290 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.290 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:09:12.290 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:58.449552+0000", 2024-09-20T19:09:12.290 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:09:12.290 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.290 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.290 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:09:12.290 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:58.449552+0000", 2024-09-20T19:09:12.291 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:09:12.291 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.291 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.291 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:09:12.291 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:58.449553+0000", 2024-09-20T19:09:12.291 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.5799999999999999e-07 2024-09-20T19:09:12.291 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.291 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.291 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:09:12.291 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:58.449598+0000", 2024-09-20T19:09:12.291 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4672000000000002e-05 2024-09-20T19:09:12.291 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.291 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.292 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:09:12.292 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:58.449604+0000", 2024-09-20T19:09:12.292 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6279999999999999e-06 2024-09-20T19:09:12.292 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.292 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.292 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:09:12.292 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:58.449610+0000", 2024-09-20T19:09:12.292 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7590000000000003e-06 2024-09-20T19:09:12.292 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.292 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.292 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:09:12.292 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:58.449641+0000", 2024-09-20T19:09:12.292 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.0877999999999998e-05 2024-09-20T19:09:12.292 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.293 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.293 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting_for_map not empty", 2024-09-20T19:09:12.293 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:58.449646+0000", 2024-09-20T19:09:12.293 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9960000000000001e-06 2024-09-20T19:09:12.293 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.293 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.293 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:09:12.293 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:58.461156+0000", 2024-09-20T19:09:12.293 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011510408999999999 2024-09-20T19:09:12.293 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.293 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.293 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:09:12.293 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:58.461161+0000", 2024-09-20T19:09:12.293 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2870000000000004e-06 2024-09-20T19:09:12.293 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.294 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.294 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:09:12.294 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:59.670338+0000", 2024-09-20T19:09:12.294 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2091770129999999 2024-09-20T19:09:12.294 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.294 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.294 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:09:12.294 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:59.670505+0000", 2024-09-20T19:09:12.294 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016696199999999999 2024-09-20T19:09:12.294 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.294 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.294 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,4,6", 2024-09-20T19:09:12.294 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:59.670673+0000", 2024-09-20T19:09:12.294 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00016786899999999999 2024-09-20T19:09:12.294 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.295 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.295 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:12.295 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:59.673972+0000", 2024-09-20T19:09:12.295 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.003298559 2024-09-20T19:09:12.295 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.295 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.295 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:09:12.295 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:59.673993+0000", 2024-09-20T19:09:12.295 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0868000000000001e-05 2024-09-20T19:09:12.295 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.295 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.295 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:12.295 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:59.676570+0000", 2024-09-20T19:09:12.295 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0025773739999999999 2024-09-20T19:09:12.295 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.296 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.296 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:12.296 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:59.689680+0000", 2024-09-20T19:09:12.296 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.013109961999999999 2024-09-20T19:09:12.296 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.296 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.296 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:09:12.296 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:59.689927+0000", 2024-09-20T19:09:12.296 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000247304 2024-09-20T19:09:12.296 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.296 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.296 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:09:12.296 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:59.691558+0000", 2024-09-20T19:09:12.296 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0016308290000000001 2024-09-20T19:09:12.296 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:09:12.296 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:09:12.297 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:09:12.297 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.297 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.297 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.26264.0:62112 11.18 11:1a292921:::benchmark_data_smithi083_177491_object62111:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e444)", 2024-09-20T19:09:12.297 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:06:58.449620+0000", 2024-09-20T19:09:12.297 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 133.76961989099999, 2024-09-20T19:09:12.297 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2439026989999999, 2024-09-20T19:09:12.297 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:09:12.297 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:09:12.297 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:09:12.297 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:09:12.297 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.26264", 2024-09-20T19:09:12.297 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2608780316", 2024-09-20T19:09:12.297 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 62112 2024-09-20T19:09:12.297 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.298 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:09:12.298 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.298 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:09:12.298 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:58.449620+0000", 2024-09-20T19:09:12.298 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:09:12.298 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.298 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.298 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:09:12.298 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:58.449620+0000", 2024-09-20T19:09:12.298 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:09:12.298 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.298 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.298 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:09:12.298 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:58.449621+0000", 2024-09-20T19:09:12.299 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1799999999999995e-07 2024-09-20T19:09:12.299 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.299 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.299 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:09:12.299 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:58.449664+0000", 2024-09-20T19:09:12.299 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3501000000000002e-05 2024-09-20T19:09:12.299 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.299 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.299 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:09:12.299 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:58.449671+0000", 2024-09-20T19:09:12.299 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.4099999999999996e-06 2024-09-20T19:09:12.299 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.299 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.299 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:09:12.299 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:58.449677+0000", 2024-09-20T19:09:12.300 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8969999999999996e-06 2024-09-20T19:09:12.300 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.300 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.300 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:09:12.300 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:58.449736+0000", 2024-09-20T19:09:12.300 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9474e-05 2024-09-20T19:09:12.300 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.300 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.300 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting_for_map not empty", 2024-09-20T19:09:12.300 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:58.449740+0000", 2024-09-20T19:09:12.300 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3500000000000001e-06 2024-09-20T19:09:12.300 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.300 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.300 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:09:12.300 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:58.461177+0000", 2024-09-20T19:09:12.301 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.011437533 2024-09-20T19:09:12.301 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.301 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.301 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:09:12.301 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:58.461182+0000", 2024-09-20T19:09:12.301 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7339999999999999e-06 2024-09-20T19:09:12.301 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.301 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.301 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:09:12.301 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:59.672473+0000", 2024-09-20T19:09:12.301 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2112915719999999 2024-09-20T19:09:12.301 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.301 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.301 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:09:12.301 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:59.672641+0000", 2024-09-20T19:09:12.301 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000167255 2024-09-20T19:09:12.302 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.302 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.302 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 2,4,6", 2024-09-20T19:09:12.302 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:59.672812+0000", 2024-09-20T19:09:12.302 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00017158900000000001 2024-09-20T19:09:12.302 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.302 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.302 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:12.302 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:59.675340+0000", 2024-09-20T19:09:12.302 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0025282400000000002 2024-09-20T19:09:12.302 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.302 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.302 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:09:12.303 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:59.675433+0000", 2024-09-20T19:09:12.303 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2615999999999996e-05 2024-09-20T19:09:12.303 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.303 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.303 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:12.303 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:59.676730+0000", 2024-09-20T19:09:12.303 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001296908 2024-09-20T19:09:12.303 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.303 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.303 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:12.303 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:59.691934+0000", 2024-09-20T19:09:12.303 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.015204159 2024-09-20T19:09:12.303 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.303 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.304 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:09:12.304 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:59.692170+0000", 2024-09-20T19:09:12.304 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000236283 2024-09-20T19:09:12.304 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.304 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.304 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:09:12.304 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:06:59.693523+0000", 2024-09-20T19:09:12.304 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0013526600000000001 2024-09-20T19:09:12.304 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:09:12.304 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:09:12.304 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:09:12.304 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.304 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.304 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.26264.0:177455 11.19 11.53640759 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e484)", 2024-09-20T19:09:12.305 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:09:01.981011+0000", 2024-09-20T19:09:12.305 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 10.238229039, 2024-09-20T19:09:12.305 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0005661180000001, 2024-09-20T19:09:12.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:12.306 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:09:12.306 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:09:12.306 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-20T19:09:12.306 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:09:12.307 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.26264", 2024-09-20T19:09:12.307 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2608780316", 2024-09-20T19:09:12.307 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 177455 2024-09-20T19:09:12.307 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.307 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:09:12.307 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.307 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:09:12.307 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:01.981011+0000", 2024-09-20T19:09:12.307 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:09:12.307 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.307 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.307 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:09:12.307 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:01.981011+0000", 2024-09-20T19:09:12.307 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:09:12.308 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.308 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.308 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:09:12.308 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:01.981013+0000", 2024-09-20T19:09:12.308 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6750000000000001e-06 2024-09-20T19:09:12.308 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.308 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.308 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:09:12.308 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:01.981019+0000", 2024-09-20T19:09:12.308 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.049e-06 2024-09-20T19:09:12.308 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.308 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.308 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:09:12.309 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:01.981034+0000", 2024-09-20T19:09:12.309 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4603000000000001e-05 2024-09-20T19:09:12.309 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.309 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.309 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:09:12.309 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:01.981042+0000", 2024-09-20T19:09:12.309 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.0239999999999997e-06 2024-09-20T19:09:12.309 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.309 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.309 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:09:12.309 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:01.995301+0000", 2024-09-20T19:09:12.309 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.014259340000000001 2024-09-20T19:09:12.309 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.310 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.310 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:09:12.310 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:01.995316+0000", 2024-09-20T19:09:12.310 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.5162e-05 2024-09-20T19:09:12.310 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.310 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.310 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:09:12.310 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:02.981564+0000", 2024-09-20T19:09:12.310 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.98624774599999998 2024-09-20T19:09:12.310 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.310 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.310 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:09:12.310 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:02.981577+0000", 2024-09-20T19:09:12.310 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3519e-05 2024-09-20T19:09:12.311 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:09:12.311 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:09:12.311 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:09:12.311 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.311 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.311 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.26264.0:177462 11.19 11.7093d159 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e484)", 2024-09-20T19:09:12.311 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:09:01.981067+0000", 2024-09-20T19:09:12.311 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 10.238173505000001, 2024-09-20T19:09:12.311 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0005452349999999, 2024-09-20T19:09:12.311 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:09:12.311 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:09:12.311 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-20T19:09:12.311 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:09:12.312 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.26264", 2024-09-20T19:09:12.312 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2608780316", 2024-09-20T19:09:12.312 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 177462 2024-09-20T19:09:12.312 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.312 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:09:12.312 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.312 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:09:12.312 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:01.981067+0000", 2024-09-20T19:09:12.312 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:09:12.312 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.312 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.312 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:09:12.312 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:01.981067+0000", 2024-09-20T19:09:12.312 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:09:12.312 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.313 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.313 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:09:12.313 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:01.981067+0000", 2024-09-20T19:09:12.313 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.4499999999999997e-07 2024-09-20T19:09:12.313 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.313 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.313 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:09:12.313 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:01.981070+0000", 2024-09-20T19:09:12.313 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7750000000000001e-06 2024-09-20T19:09:12.313 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.313 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.313 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:09:12.313 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:01.981079+0000", 2024-09-20T19:09:12.313 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.8929999999999994e-06 2024-09-20T19:09:12.313 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.313 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.314 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:09:12.314 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:01.981085+0000", 2024-09-20T19:09:12.314 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9000000000000003e-06 2024-09-20T19:09:12.314 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.314 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.314 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:09:12.314 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:01.998157+0000", 2024-09-20T19:09:12.314 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.017071662000000001 2024-09-20T19:09:12.314 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.314 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.314 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:09:12.314 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:01.998247+0000", 2024-09-20T19:09:12.314 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.0536000000000002e-05 2024-09-20T19:09:12.314 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.315 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.315 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:09:12.315 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:02.981605+0000", 2024-09-20T19:09:12.315 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.98335763300000001 2024-09-20T19:09:12.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.315 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.315 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:09:12.315 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:02.981612+0000", 2024-09-20T19:09:12.315 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2910000000000002e-06 2024-09-20T19:09:12.315 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:09:12.315 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:09:12.315 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:09:12.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.315 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.315 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.26264.0:177474 11.19 11.a3accdf9 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e484)", 2024-09-20T19:09:12.316 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:09:01.996914+0000", 2024-09-20T19:09:12.316 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 10.222325873000001, 2024-09-20T19:09:12.316 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.98472428300000003, 2024-09-20T19:09:12.316 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:09:12.316 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:09:12.316 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-20T19:09:12.316 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:09:12.316 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.26264", 2024-09-20T19:09:12.316 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2608780316", 2024-09-20T19:09:12.316 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 177474 2024-09-20T19:09:12.316 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.316 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:09:12.316 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.317 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:09:12.317 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:01.996914+0000", 2024-09-20T19:09:12.317 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:09:12.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.317 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.317 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:09:12.317 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:01.996914+0000", 2024-09-20T19:09:12.317 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:09:12.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.317 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.317 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:09:12.317 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:01.996916+0000", 2024-09-20T19:09:12.317 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.75e-06 2024-09-20T19:09:12.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.317 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.317 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:09:12.317 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:01.996921+0000", 2024-09-20T19:09:12.318 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.0730000000000004e-06 2024-09-20T19:09:12.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.318 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.318 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:09:12.318 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:01.996932+0000", 2024-09-20T19:09:12.318 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1049e-05 2024-09-20T19:09:12.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.318 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.318 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:09:12.318 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:01.996940+0000", 2024-09-20T19:09:12.318 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.5700000000000004e-06 2024-09-20T19:09:12.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.318 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.318 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:09:12.318 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:01.998271+0000", 2024-09-20T19:09:12.318 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001331512 2024-09-20T19:09:12.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.319 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.319 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:09:12.319 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:01.998276+0000", 2024-09-20T19:09:12.319 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9540000000000003e-06 2024-09-20T19:09:12.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.319 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.319 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:09:12.319 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:02.981632+0000", 2024-09-20T19:09:12.319 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.98335529300000002 2024-09-20T19:09:12.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.319 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.319 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:09:12.319 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:02.981639+0000", 2024-09-20T19:09:12.320 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.0820000000000001e-06 2024-09-20T19:09:12.320 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:09:12.320 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:09:12.320 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:09:12.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.320 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.320 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.26264.0:177485 11.19 11.471ea439 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e484)", 2024-09-20T19:09:12.320 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:09:02.006313+0000", 2024-09-20T19:09:12.320 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 10.212927669000001, 2024-09-20T19:09:12.320 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97536030500000004, 2024-09-20T19:09:12.320 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:09:12.320 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:09:12.320 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "reached pg", 2024-09-20T19:09:12.320 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:09:12.320 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.26264", 2024-09-20T19:09:12.320 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2608780316", 2024-09-20T19:09:12.321 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 177485 2024-09-20T19:09:12.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.321 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:09:12.321 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.321 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:09:12.321 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:02.006313+0000", 2024-09-20T19:09:12.321 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:09:12.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.321 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.321 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:09:12.321 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:02.006313+0000", 2024-09-20T19:09:12.321 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:09:12.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.321 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.321 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:09:12.321 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:02.006313+0000", 2024-09-20T19:09:12.322 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6600000000000005e-07 2024-09-20T19:09:12.322 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.322 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.322 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:09:12.322 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:02.006318+0000", 2024-09-20T19:09:12.322 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4449999999999998e-06 2024-09-20T19:09:12.322 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.322 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.322 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:09:12.322 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:02.006332+0000", 2024-09-20T19:09:12.322 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.4011e-05 2024-09-20T19:09:12.322 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.322 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.322 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:09:12.322 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:02.006339+0000", 2024-09-20T19:09:12.322 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2409999999999998e-06 2024-09-20T19:09:12.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.323 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.323 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:09:12.323 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:02.006382+0000", 2024-09-20T19:09:12.323 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.3228e-05 2024-09-20T19:09:12.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.323 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.323 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for peered", 2024-09-20T19:09:12.323 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:02.006387+0000", 2024-09-20T19:09:12.323 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7690000000000004e-06 2024-09-20T19:09:12.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.323 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.323 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:09:12.323 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:02.981665+0000", 2024-09-20T19:09:12.323 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.97527809700000001 2024-09-20T19:09:12.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.324 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.324 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:09:12.324 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:02.981673+0000", 2024-09-20T19:09:12.324 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.6480000000000003e-06 2024-09-20T19:09:12.324 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:09:12.324 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:09:12.324 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:09:12.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.324 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.324 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.26264.0:181816 11.b 11:db20fce0:::benchmark_data_smithi083_177491_object78374:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e486)", 2024-09-20T19:09:12.324 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:09:09.011813+0000", 2024-09-20T19:09:12.324 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 3.20742686, 2024-09-20T19:09:12.324 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.040756524000000002, 2024-09-20T19:09:12.324 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:09:12.324 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:09:12.325 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:09:12.325 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:09:12.325 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.26264", 2024-09-20T19:09:12.325 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2608780316", 2024-09-20T19:09:12.325 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 181816 2024-09-20T19:09:12.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.325 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:09:12.325 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.325 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:09:12.325 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:09.011813+0000", 2024-09-20T19:09:12.325 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:09:12.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.325 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.325 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:09:12.325 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:09.011813+0000", 2024-09-20T19:09:12.325 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:09:12.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.326 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.326 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:09:12.326 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:09.011814+0000", 2024-09-20T19:09:12.326 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3799999999999997e-07 2024-09-20T19:09:12.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.326 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.326 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:09:12.326 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:09.011817+0000", 2024-09-20T19:09:12.326 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7420000000000002e-06 2024-09-20T19:09:12.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.326 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.326 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:09:12.326 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:09.011823+0000", 2024-09-20T19:09:12.326 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.6089999999999997e-06 2024-09-20T19:09:12.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.327 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.327 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:09:12.327 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:09.011829+0000", 2024-09-20T19:09:12.327 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.6019999999999999e-06 2024-09-20T19:09:12.327 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.327 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.327 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:09:12.327 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:09.011886+0000", 2024-09-20T19:09:12.327 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.7139999999999998e-05 2024-09-20T19:09:12.327 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.327 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.327 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:09:12.327 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:09.011980+0000", 2024-09-20T19:09:12.327 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.4115000000000003e-05 2024-09-20T19:09:12.327 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.327 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.328 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,1", 2024-09-20T19:09:12.328 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:09.012067+0000", 2024-09-20T19:09:12.328 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.6938e-05 2024-09-20T19:09:12.328 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.328 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.328 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:09:12.328 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:09.017259+0000", 2024-09-20T19:09:12.328 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0051916280000000002 2024-09-20T19:09:12.328 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.328 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.328 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:12.328 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:09.023426+0000", 2024-09-20T19:09:12.328 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0061669790000000004 2024-09-20T19:09:12.328 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.328 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.329 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:12.329 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:09.052475+0000", 2024-09-20T19:09:12.329 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.029049405 2024-09-20T19:09:12.329 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.329 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.329 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:09:12.329 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:09.052525+0000", 2024-09-20T19:09:12.329 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9450999999999998e-05 2024-09-20T19:09:12.329 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.329 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.329 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:09:12.329 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:09.052570+0000", 2024-09-20T19:09:12.329 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5377000000000003e-05 2024-09-20T19:09:12.329 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:09:12.330 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:09:12.330 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:09:12.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.330 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.330 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.26264.0:183744 11.a 11:530a5d18:::benchmark_data_smithi083_177491_object80302:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e486)", 2024-09-20T19:09:12.330 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:09:10.923837+0000", 2024-09-20T19:09:12.330 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 1.295403241, 2024-09-20T19:09:12.330 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.12089446, 2024-09-20T19:09:12.330 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:09:12.330 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:09:12.330 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:09:12.330 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:09:12.330 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.26264", 2024-09-20T19:09:12.330 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2608780316", 2024-09-20T19:09:12.330 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 183744 2024-09-20T19:09:12.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.331 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:09:12.331 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.331 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:09:12.331 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:10.923837+0000", 2024-09-20T19:09:12.331 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:09:12.331 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.331 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.331 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:09:12.331 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:10.923837+0000", 2024-09-20T19:09:12.331 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:09:12.331 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.331 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.331 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:09:12.331 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:10.923838+0000", 2024-09-20T19:09:12.331 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.204e-06 2024-09-20T19:09:12.331 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.331 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.332 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:09:12.332 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:10.923843+0000", 2024-09-20T19:09:12.332 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1200000000000001e-06 2024-09-20T19:09:12.332 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.332 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.332 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:09:12.332 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:10.923859+0000", 2024-09-20T19:09:12.332 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.6035000000000001e-05 2024-09-20T19:09:12.332 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.332 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.332 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:09:12.332 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:10.923867+0000", 2024-09-20T19:09:12.332 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2779999999999996e-06 2024-09-20T19:09:12.332 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.332 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.333 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:09:12.333 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:10.924089+0000", 2024-09-20T19:09:12.333 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00022245899999999999 2024-09-20T19:09:12.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.333 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:09:12.333 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:10.924227+0000", 2024-09-20T19:09:12.333 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000137613 2024-09-20T19:09:12.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.333 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,6", 2024-09-20T19:09:12.333 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:10.924345+0000", 2024-09-20T19:09:12.333 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000118718 2024-09-20T19:09:12.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.333 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:12.334 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:10.928321+0000", 2024-09-20T19:09:12.334 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0039754259999999998 2024-09-20T19:09:12.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.334 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.334 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:09:12.334 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:10.929368+0000", 2024-09-20T19:09:12.334 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.001046936 2024-09-20T19:09:12.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.334 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.334 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:12.334 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:11.044667+0000", 2024-09-20T19:09:12.334 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.115298818 2024-09-20T19:09:12.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.334 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.334 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:09:12.335 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:11.044703+0000", 2024-09-20T19:09:12.335 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.6049e-05 2024-09-20T19:09:12.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.335 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.335 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:09:12.335 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:11.044731+0000", 2024-09-20T19:09:12.335 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8804000000000001e-05 2024-09-20T19:09:12.335 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:09:12.335 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:09:12.335 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:09:12.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.335 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.335 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.26264.0:184123 11.b 11:d6b90893:::benchmark_data_smithi083_177491_object80681:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e486)", 2024-09-20T19:09:12.335 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:09:11.450310+0000", 2024-09-20T19:09:12.335 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.768930322, 2024-09-20T19:09:12.335 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.043455494999999997, 2024-09-20T19:09:12.336 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:09:12.336 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:09:12.336 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:09:12.336 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:09:12.336 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.26264", 2024-09-20T19:09:12.336 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2608780316", 2024-09-20T19:09:12.336 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 184123 2024-09-20T19:09:12.336 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.336 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:09:12.336 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.336 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:09:12.336 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:11.450310+0000", 2024-09-20T19:09:12.336 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:09:12.336 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.336 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.336 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:09:12.337 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:11.450310+0000", 2024-09-20T19:09:12.337 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:09:12.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.337 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.337 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:09:12.337 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:11.450310+0000", 2024-09-20T19:09:12.337 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.1399999999999997e-07 2024-09-20T19:09:12.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.337 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.337 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:09:12.337 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:11.450313+0000", 2024-09-20T19:09:12.337 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.711e-06 2024-09-20T19:09:12.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.338 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.338 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:09:12.338 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:11.450318+0000", 2024-09-20T19:09:12.338 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.3839999999999996e-06 2024-09-20T19:09:12.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.338 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.338 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:09:12.338 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:11.450323+0000", 2024-09-20T19:09:12.338 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.831e-06 2024-09-20T19:09:12.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.338 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.338 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:09:12.338 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:11.454236+0000", 2024-09-20T19:09:12.338 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0039123480000000004 2024-09-20T19:09:12.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.339 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.339 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:09:12.339 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:11.454340+0000", 2024-09-20T19:09:12.339 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000103998 2024-09-20T19:09:12.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.339 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.339 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 0,1", 2024-09-20T19:09:12.339 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:11.454444+0000", 2024-09-20T19:09:12.339 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.000104417 2024-09-20T19:09:12.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.339 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.339 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:09:12.339 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:11.458038+0000", 2024-09-20T19:09:12.339 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0035938300000000001 2024-09-20T19:09:12.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.339 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.340 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:12.340 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:11.480372+0000", 2024-09-20T19:09:12.340 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.022334471000000002 2024-09-20T19:09:12.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.340 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:12.340 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:11.493655+0000", 2024-09-20T19:09:12.340 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.013282403 2024-09-20T19:09:12.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.340 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:09:12.340 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:11.493716+0000", 2024-09-20T19:09:12.340 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.0958000000000001e-05 2024-09-20T19:09:12.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.340 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.341 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:09:12.341 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:11.493765+0000", 2024-09-20T19:09:12.341 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.973e-05 2024-09-20T19:09:12.341 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:09:12.341 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:09:12.341 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:09:12.341 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.341 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.341 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_op(client.26264.0:184534 11.a 11:5c844c87:::benchmark_data_smithi083_177491_object81092:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e486)", 2024-09-20T19:09:12.341 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-20T19:09:11.921511+0000", 2024-09-20T19:09:12.341 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.29772963899999999, 2024-09-20T19:09:12.341 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.041395427999999998, 2024-09-20T19:09:12.341 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-20T19:09:12.341 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-20T19:09:12.341 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:09:12.341 INFO:teuthology.orchestra.run.smithi083.stdout: "client_info": { 2024-09-20T19:09:12.341 INFO:teuthology.orchestra.run.smithi083.stdout: "client": "client.26264", 2024-09-20T19:09:12.342 INFO:teuthology.orchestra.run.smithi083.stdout: "client_addr": "172.21.15.83:0/2608780316", 2024-09-20T19:09:12.342 INFO:teuthology.orchestra.run.smithi083.stdout: "tid": 184534 2024-09-20T19:09:12.342 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.342 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-20T19:09:12.342 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.342 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-20T19:09:12.342 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:11.921511+0000", 2024-09-20T19:09:12.342 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:09:12.342 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.342 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.342 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-20T19:09:12.342 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:11.921511+0000", 2024-09-20T19:09:12.342 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-20T19:09:12.342 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.342 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.343 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-20T19:09:12.343 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:11.921511+0000", 2024-09-20T19:09:12.343 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.3799999999999998e-07 2024-09-20T19:09:12.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.343 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.343 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-20T19:09:12.343 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:11.921514+0000", 2024-09-20T19:09:12.343 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.7760000000000002e-06 2024-09-20T19:09:12.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.343 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.343 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-20T19:09:12.343 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:11.921518+0000", 2024-09-20T19:09:12.343 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5750000000000002e-06 2024-09-20T19:09:12.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.343 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.344 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-20T19:09:12.344 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:11.921523+0000", 2024-09-20T19:09:12.344 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4560000000000002e-06 2024-09-20T19:09:12.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.344 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.344 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-20T19:09:12.344 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:11.924470+0000", 2024-09-20T19:09:12.344 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.002946812 2024-09-20T19:09:12.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.344 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.344 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-20T19:09:12.344 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:11.924559+0000", 2024-09-20T19:09:12.344 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9522000000000001e-05 2024-09-20T19:09:12.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.344 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.345 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "waiting for subops from 5,6", 2024-09-20T19:09:12.345 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:11.924649+0000", 2024-09-20T19:09:12.345 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.9767999999999998e-05 2024-09-20T19:09:12.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.345 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.345 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "op_commit", 2024-09-20T19:09:12.345 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:11.930031+0000", 2024-09-20T19:09:12.345 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0053819280000000002 2024-09-20T19:09:12.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.345 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.345 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:12.345 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:11.933741+0000", 2024-09-20T19:09:12.345 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0037099020000000002 2024-09-20T19:09:12.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.345 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.346 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:12.346 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:11.962802+0000", 2024-09-20T19:09:12.346 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.029061653 2024-09-20T19:09:12.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.346 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.346 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "commit_sent", 2024-09-20T19:09:12.346 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:11.962862+0000", 2024-09-20T19:09:12.346 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.9589000000000002e-05 2024-09-20T19:09:12.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-20T19:09:12.346 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-20T19:09:12.346 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "done", 2024-09-20T19:09:12.346 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-20T19:09:11.962906+0000", 2024-09-20T19:09:12.346 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.4109000000000001e-05 2024-09-20T19:09:12.346 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:09:12.346 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:09:12.346 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:09:12.347 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-20T19:09:12.347 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-20T19:09:12.347 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T19:09:12.347 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:12.347 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:09:12.357 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:12 smithi142 ceph-mon[32508]: pgmap v983: 168 pgs: 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 158 active+clean; 1.5 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 620 B/s rd, 0 B/s wr, 1.13k op/s; 16075/76635 objects misplaced (20.976%); 24 MiB/s, 386 objects/s recovering 2024-09-20T19:09:12.357 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:12 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:09:12.357 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:12 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:09:12.357 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:12 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2194647180' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:09:12.357 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:12 smithi142 ceph-mon[32508]: 6.16 scrub starts 2024-09-20T19:09:12.357 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:12 smithi142 ceph-mon[32508]: 6.16 scrub ok 2024-09-20T19:09:12.357 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:12 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:09:12.357 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:12 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:09:12.448 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:09:12.484 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:12.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:12.485 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:09:12.585 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:09:12.621 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:12.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:12.622 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:09:12.723 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:09:12.748 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:12.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:12.750 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:09:12.850 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:09:12.878 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:12.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:12.879 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:09:12.980 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:09:13.007 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:13.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:13.008 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:09:13.108 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:09:13.137 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:13.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:13.138 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:09:13.239 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:09:13.270 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:13.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:13.271 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:09:13.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:13 smithi083 ceph-mon[29421]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T19:09:13.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:13 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2194647180' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:09:13.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:13 smithi083 ceph-mon[29421]: osdmap e487: 8 total, 8 up, 8 in 2024-09-20T19:09:13.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:13 smithi083 ceph-mon[36260]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T19:09:13.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:13 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2194647180' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:09:13.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:13 smithi083 ceph-mon[36260]: osdmap e487: 8 total, 8 up, 8 in 2024-09-20T19:09:13.372 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:09:13.406 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:13.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:13.407 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:09:13.508 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:09:13.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:13 smithi142 ceph-mon[32508]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T19:09:13.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:13 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2194647180' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:09:13.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:13 smithi142 ceph-mon[32508]: osdmap e487: 8 total, 8 up, 8 in 2024-09-20T19:09:13.530 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:13.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:13.531 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:09:13.544 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:09:13.572 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-20T19:09:13.632 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:09:13.653 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:13.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:13.654 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:09:13.755 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:09:13.776 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:13.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:13.777 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:09:13.820 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:09:13.878 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:09:13.900 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:13.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:13.901 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:09:14.001 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:09:14.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:14.023 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:14.023 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:09:14.124 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:09:14.150 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:14.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:14.151 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:09:14.223 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:14 smithi142 ceph-mon[32508]: osdmap e488: 8 total, 8 up, 8 in 2024-09-20T19:09:14.223 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:14 smithi142 ceph-mon[32508]: pgmap v986: 168 pgs: 1 active+clean+remapped, 3 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 158 active+clean; 1.4 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.05k op/s; 10823/67365 objects misplaced (16.066%); 21 MiB/s, 343 objects/s recovering 2024-09-20T19:09:14.223 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:14 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1306657742' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:09:14.252 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:09:14.280 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:14.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:14.281 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:09:14.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:14 smithi083 ceph-mon[29421]: osdmap e488: 8 total, 8 up, 8 in 2024-09-20T19:09:14.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:14 smithi083 ceph-mon[29421]: pgmap v986: 168 pgs: 1 active+clean+remapped, 3 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 158 active+clean; 1.4 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.05k op/s; 10823/67365 objects misplaced (16.066%); 21 MiB/s, 343 objects/s recovering 2024-09-20T19:09:14.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:14 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1306657742' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:09:14.318 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:14 smithi083 ceph-mon[36260]: osdmap e488: 8 total, 8 up, 8 in 2024-09-20T19:09:14.318 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:14 smithi083 ceph-mon[36260]: pgmap v986: 168 pgs: 1 active+clean+remapped, 3 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 158 active+clean; 1.4 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.05k op/s; 10823/67365 objects misplaced (16.066%); 21 MiB/s, 343 objects/s recovering 2024-09-20T19:09:14.318 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:14 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1306657742' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:09:14.338 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 7, 6, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 5, 6, 7, 4] 2024-09-20T19:09:14.338 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T19:09:14.338 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-09-20T19:09:14.338 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-20T19:09:14.338 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-20T19:09:14.338 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.7 config set filestore_inject_stall 3 2024-09-20T19:09:14.359 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T19:09:14.359 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:09:14.383 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:09:14.407 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:14.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:14.408 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:09:14.452 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:09:14.452 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:09:14.452 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:09:14.453 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:09:14.453 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:09:14.453 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:09:14.453 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:09:14.453 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:09:14.453 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:09:14.453 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:09:14.453 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:09:14.453 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:09:14.453 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:09:14.453 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:09:14.453 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:09:14.454 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:09:14.509 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:09:14.530 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:14.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:14.531 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:09:14.632 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:09:14.653 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:14.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:14.654 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:09:14.754 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:09:14.776 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:14.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:14.777 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:09:14.878 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:09:14.901 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:14.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:14.902 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:09:15.003 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:09:15.026 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:15.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:15.027 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:09:15.127 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:09:15.157 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:15.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:15.158 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:09:15.178 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_blocked_ops 2024-09-20T19:09:15.259 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:09:15.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:15.293 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:15.294 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:09:15.301 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:09:15.394 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:09:15.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:15.432 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:15.432 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:09:15.442 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:15 smithi142 ceph-mon[32508]: osdmap e489: 8 total, 8 up, 8 in 2024-09-20T19:09:15.443 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:15 smithi142 ceph-mon[32508]: 4.12 scrub starts 2024-09-20T19:09:15.443 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:15 smithi142 ceph-mon[32508]: 4.12 scrub ok 2024-09-20T19:09:15.443 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:15 smithi142 ceph-mon[32508]: 4.19 scrub starts 2024-09-20T19:09:15.443 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:15 smithi142 ceph-mon[32508]: 4.19 scrub ok 2024-09-20T19:09:15.443 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:15 smithi142 ceph-mon[32508]: 5.1a scrub starts 2024-09-20T19:09:15.443 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:15 smithi142 ceph-mon[32508]: 5.1a scrub ok 2024-09-20T19:09:15.534 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:09:15.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:15 smithi083 ceph-mon[29421]: osdmap e489: 8 total, 8 up, 8 in 2024-09-20T19:09:15.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:15 smithi083 ceph-mon[29421]: 4.12 scrub starts 2024-09-20T19:09:15.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:15 smithi083 ceph-mon[29421]: 4.12 scrub ok 2024-09-20T19:09:15.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:15 smithi083 ceph-mon[29421]: 4.19 scrub starts 2024-09-20T19:09:15.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:15 smithi083 ceph-mon[29421]: 4.19 scrub ok 2024-09-20T19:09:15.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:15 smithi083 ceph-mon[29421]: 5.1a scrub starts 2024-09-20T19:09:15.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:15 smithi083 ceph-mon[29421]: 5.1a scrub ok 2024-09-20T19:09:15.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:15 smithi083 ceph-mon[36260]: osdmap e489: 8 total, 8 up, 8 in 2024-09-20T19:09:15.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:15 smithi083 ceph-mon[36260]: 4.12 scrub starts 2024-09-20T19:09:15.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:15 smithi083 ceph-mon[36260]: 4.12 scrub ok 2024-09-20T19:09:15.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:15 smithi083 ceph-mon[36260]: 4.19 scrub starts 2024-09-20T19:09:15.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:15 smithi083 ceph-mon[36260]: 4.19 scrub ok 2024-09-20T19:09:15.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:15 smithi083 ceph-mon[36260]: 5.1a scrub starts 2024-09-20T19:09:15.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:15 smithi083 ceph-mon[36260]: 5.1a scrub ok 2024-09-20T19:09:15.561 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:15.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:15.563 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:09:15.664 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:09:15.696 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:15.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:15.697 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:09:15.798 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:09:15.832 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:15.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:15.833 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:09:15.934 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:09:15.959 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T19:09:15.960 INFO:teuthology.orchestra.run.smithi142.stdout: "success": "" 2024-09-20T19:09:15.960 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T19:09:15.969 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:15.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:15.971 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:09:16.071 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:09:16.109 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:16.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:16.110 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:09:16.211 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:09:16.255 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:16.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:16.256 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:09:16.356 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T19:09:16.357 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:09:16.393 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:16 smithi083 ceph-mon[29421]: osdmap e490: 8 total, 8 up, 8 in 2024-09-20T19:09:16.393 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:16 smithi083 ceph-mon[29421]: 11.12 scrub starts 2024-09-20T19:09:16.393 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:16 smithi083 ceph-mon[29421]: pgmap v989: 168 pgs: 1 active+clean+remapped, 3 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 158 active+clean; 1.4 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 773 op/s; 10823/67365 objects misplaced (16.066%); 25 MiB/s, 400 objects/s recovering 2024-09-20T19:09:16.393 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:16 smithi083 ceph-mon[29421]: 11.12 scrub ok 2024-09-20T19:09:16.393 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:16 smithi083 ceph-mon[29421]: 6.0 scrub starts 2024-09-20T19:09:16.393 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:16 smithi083 ceph-mon[29421]: 6.0 scrub ok 2024-09-20T19:09:16.393 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:16 smithi083 ceph-mon[29421]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 6 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T19:09:16.393 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:16 smithi083 ceph-mon[36260]: osdmap e490: 8 total, 8 up, 8 in 2024-09-20T19:09:16.393 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:16 smithi083 ceph-mon[36260]: 11.12 scrub starts 2024-09-20T19:09:16.394 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:16 smithi083 ceph-mon[36260]: pgmap v989: 168 pgs: 1 active+clean+remapped, 3 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 158 active+clean; 1.4 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 773 op/s; 10823/67365 objects misplaced (16.066%); 25 MiB/s, 400 objects/s recovering 2024-09-20T19:09:16.394 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:16 smithi083 ceph-mon[36260]: 11.12 scrub ok 2024-09-20T19:09:16.394 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:16 smithi083 ceph-mon[36260]: 6.0 scrub starts 2024-09-20T19:09:16.394 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:16 smithi083 ceph-mon[36260]: 6.0 scrub ok 2024-09-20T19:09:16.394 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:16 smithi083 ceph-mon[36260]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 6 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T19:09:16.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:16.403 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:16.404 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:09:16.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:16 smithi142 ceph-mon[32508]: osdmap e490: 8 total, 8 up, 8 in 2024-09-20T19:09:16.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:16 smithi142 ceph-mon[32508]: 11.12 scrub starts 2024-09-20T19:09:16.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:16 smithi142 ceph-mon[32508]: pgmap v989: 168 pgs: 1 active+clean+remapped, 3 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 158 active+clean; 1.4 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 773 op/s; 10823/67365 objects misplaced (16.066%); 25 MiB/s, 400 objects/s recovering 2024-09-20T19:09:16.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:16 smithi142 ceph-mon[32508]: 11.12 scrub ok 2024-09-20T19:09:16.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:16 smithi142 ceph-mon[32508]: 6.0 scrub starts 2024-09-20T19:09:16.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:16 smithi142 ceph-mon[32508]: 6.0 scrub ok 2024-09-20T19:09:16.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:16 smithi142 ceph-mon[32508]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 6 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T19:09:16.505 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:09:16.533 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:16.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:16.534 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:09:16.635 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:09:16.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:16.681 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:16.681 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:09:16.781 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:09:16.805 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:09:16.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:16.841 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:16.841 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:09:16.917 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:09:16.941 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:09:16.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:16.996 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:16.996 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:09:17.078 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T19:09:17.097 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:09:17.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:17.156 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:17.156 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:09:17.257 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:09:17.285 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:17.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:17.286 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:09:17.386 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:09:17.415 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:17.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:17.416 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:09:17.516 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:09:17.535 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:09:17.542 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T19:09:17.542 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:09:17.544 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:17.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:17.546 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:09:17.646 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:09:17.679 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:17.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:17.680 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:09:17.760 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:09:17 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:09:17.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:17 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:09:17.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:17 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:09:17.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:17 smithi142 ceph-mon[32508]: 3.c scrub starts 2024-09-20T19:09:17.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:17 smithi142 ceph-mon[32508]: 3.c scrub ok 2024-09-20T19:09:17.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:17 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:09:17.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:17 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:09:17.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:17 smithi142 ceph-mon[32508]: 6.19 scrub starts 2024-09-20T19:09:17.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:17 smithi142 ceph-mon[32508]: 6.19 scrub ok 2024-09-20T19:09:17.781 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:09:17.809 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:17.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:17.810 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:09:17.910 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:09:17.942 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:17.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:17.943 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:09:17.967 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:17 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:09:17.967 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:17 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:09:17.967 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:17 smithi083 ceph-mon[29421]: 3.c scrub starts 2024-09-20T19:09:17.967 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:17 smithi083 ceph-mon[29421]: 3.c scrub ok 2024-09-20T19:09:17.967 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:17 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:09:17.967 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:17 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:09:17.967 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:17 smithi083 ceph-mon[29421]: 6.19 scrub starts 2024-09-20T19:09:17.968 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:17 smithi083 ceph-mon[29421]: 6.19 scrub ok 2024-09-20T19:09:17.968 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:17 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:09:17.968 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:17 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:09:17.968 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:17 smithi083 ceph-mon[36260]: 3.c scrub starts 2024-09-20T19:09:17.968 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:17 smithi083 ceph-mon[36260]: 3.c scrub ok 2024-09-20T19:09:17.968 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:17 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:09:17.968 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:17 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:09:17.968 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:17 smithi083 ceph-mon[36260]: 6.19 scrub starts 2024-09-20T19:09:17.968 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:17 smithi083 ceph-mon[36260]: 6.19 scrub ok 2024-09-20T19:09:18.044 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:09:18.084 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:18.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:18.086 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:09:18.150 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:09:18.150 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:09:18.184 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:09:18.185 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:09:18.186 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:09:18.221 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:09:18.221 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:09:18.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:18.242 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:18.242 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:09:18.258 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:09:18.258 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:09:18.297 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:09:18.297 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:09:18.331 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:09:18.331 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:09:18.343 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:09:18.369 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:09:18.369 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:09:18.371 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:18.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:18.373 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:09:18.403 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:09:18.404 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:09:18.440 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_historic_ops 2024-09-20T19:09:18.473 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:09:18.512 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:18.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:18.514 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:09:18.614 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:09:18.655 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:18.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:18.656 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:09:18.753 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:18 smithi142 ceph-mon[32508]: pgmap v990: 168 pgs: 6 active+remapped+backfill_toofull, 162 active+clean; 1.2 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 990 B/s rd, 0 B/s wr, 814 op/s; 9124/58911 objects misplaced (15.488%); 15 MiB/s, 242 objects/s recovering 2024-09-20T19:09:18.753 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:18 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:09:18.753 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:18 smithi142 ceph-mon[32508]: osdmap e491: 8 total, 8 up, 8 in 2024-09-20T19:09:18.753 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:18 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2433826328' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:09:18.753 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:18 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:09:18.753 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:18 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T19:09:18.753 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:18 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3776184476' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:09:18.753 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:18 smithi142 ceph-mon[32508]: 4.1c scrub starts 2024-09-20T19:09:18.753 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:18 smithi142 ceph-mon[32508]: 4.1c scrub ok 2024-09-20T19:09:18.753 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:18 smithi142 ceph-mon[32508]: 3.19 scrub starts 2024-09-20T19:09:18.754 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:18 smithi142 ceph-mon[32508]: 3.19 scrub ok 2024-09-20T19:09:18.754 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:18 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T19:09:18.754 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:18 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T19:09:18.754 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:18 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:09:18.757 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:09:18.774 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is set 2024-09-20T19:09:18.807 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:18.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:18.808 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:09:18.816 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:09:18.908 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:09:18.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:18.947 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:18.947 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:09:19.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:18 smithi083 ceph-mon[29421]: pgmap v990: 168 pgs: 6 active+remapped+backfill_toofull, 162 active+clean; 1.2 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 990 B/s rd, 0 B/s wr, 814 op/s; 9124/58911 objects misplaced (15.488%); 15 MiB/s, 242 objects/s recovering 2024-09-20T19:09:19.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:18 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:09:19.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:18 smithi083 ceph-mon[29421]: osdmap e491: 8 total, 8 up, 8 in 2024-09-20T19:09:19.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:18 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2433826328' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:09:19.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:18 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:09:19.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:18 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T19:09:19.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:18 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3776184476' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:09:19.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:18 smithi083 ceph-mon[29421]: 4.1c scrub starts 2024-09-20T19:09:19.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:18 smithi083 ceph-mon[29421]: 4.1c scrub ok 2024-09-20T19:09:19.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:18 smithi083 ceph-mon[29421]: 3.19 scrub starts 2024-09-20T19:09:19.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:18 smithi083 ceph-mon[29421]: 3.19 scrub ok 2024-09-20T19:09:19.042 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:18 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T19:09:19.042 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:18 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T19:09:19.042 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:18 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:09:19.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:18 smithi083 ceph-mon[36260]: pgmap v990: 168 pgs: 6 active+remapped+backfill_toofull, 162 active+clean; 1.2 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 990 B/s rd, 0 B/s wr, 814 op/s; 9124/58911 objects misplaced (15.488%); 15 MiB/s, 242 objects/s recovering 2024-09-20T19:09:19.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:18 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:09:19.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:18 smithi083 ceph-mon[36260]: osdmap e491: 8 total, 8 up, 8 in 2024-09-20T19:09:19.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:18 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2433826328' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:09:19.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:18 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-20T19:09:19.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:18 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T19:09:19.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:18 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3776184476' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:09:19.042 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:18 smithi083 ceph-mon[36260]: 4.1c scrub starts 2024-09-20T19:09:19.043 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:18 smithi083 ceph-mon[36260]: 4.1c scrub ok 2024-09-20T19:09:19.043 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:18 smithi083 ceph-mon[36260]: 3.19 scrub starts 2024-09-20T19:09:19.043 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:18 smithi083 ceph-mon[36260]: 3.19 scrub ok 2024-09-20T19:09:19.043 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:18 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T19:09:19.043 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:18 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T19:09:19.043 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:18 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:09:19.048 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:09:19.088 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:19.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:19.089 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:09:19.190 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:09:19.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:19.229 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:19.229 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:09:19.330 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:09:19.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:19.371 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:19.372 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:09:19.399 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T19:09:19.399 INFO:teuthology.orchestra.run.smithi142.stdout: "size": 20, 2024-09-20T19:09:19.400 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 600, 2024-09-20T19:09:19.400 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-09-20T19:09:19.400 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.400 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.14460.0:2699 5.e 5:7759931f:::notify.3:head [watch reconnect cookie 94056120170112 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e292)", 2024-09-20T19:09:19.400 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:00:35.180070+0000", 2024-09-20T19:09:19.400 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 524.21376431500005, 2024-09-20T19:09:19.400 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0139281419999999, 2024-09-20T19:09:19.400 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:09:19.400 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:09:19.400 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-09-20T19:09:19.400 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:09:19.400 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.14460", 2024-09-20T19:09:19.401 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/1555339426", 2024-09-20T19:09:19.401 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 2699 2024-09-20T19:09:19.401 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.401 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:09:19.401 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.401 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:09:19.401 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.180070+0000", 2024-09-20T19:09:19.401 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:19.401 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.401 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.401 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:09:19.401 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.180070+0000", 2024-09-20T19:09:19.401 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:19.402 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.402 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.402 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:09:19.402 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.180071+0000", 2024-09-20T19:09:19.402 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.093e-06 2024-09-20T19:09:19.402 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.402 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.402 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:09:19.402 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.180077+0000", 2024-09-20T19:09:19.402 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5079999999999996e-06 2024-09-20T19:09:19.402 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.402 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.402 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:09:19.403 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.180086+0000", 2024-09-20T19:09:19.403 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.9499999999999996e-06 2024-09-20T19:09:19.403 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.403 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.403 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:09:19.403 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.180094+0000", 2024-09-20T19:09:19.403 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5780000000000001e-06 2024-09-20T19:09:19.403 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.403 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.403 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:19.403 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.180159+0000", 2024-09-20T19:09:19.403 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5139000000000001e-05 2024-09-20T19:09:19.403 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.403 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.403 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op must wait for map", 2024-09-20T19:09:19.404 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.180165+0000", 2024-09-20T19:09:19.404 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.7629999999999997e-06 2024-09-20T19:09:19.404 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.404 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.404 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:19.404 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.182672+0000", 2024-09-20T19:09:19.404 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0025072710000000002 2024-09-20T19:09:19.404 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.404 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.404 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:09:19.404 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:35.182681+0000", 2024-09-20T19:09:19.404 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.4780000000000008e-06 2024-09-20T19:09:19.404 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.404 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.404 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:19.405 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.191896+0000", 2024-09-20T19:09:19.405 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.009215755 2024-09-20T19:09:19.405 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.405 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.405 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:09:19.405 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.191989+0000", 2024-09-20T19:09:19.405 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.2868000000000006e-05 2024-09-20T19:09:19.405 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.405 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.405 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:09:19.405 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:00:36.193998+0000", 2024-09-20T19:09:19.405 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0020087389999999998 2024-09-20T19:09:19.405 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:19.405 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:09:19.405 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:19.405 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.406 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.406 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:147626 10.9 10:98e2f0b0:::benchmark_data_smithi083_148100_object30764:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e376)", 2024-09-20T19:09:19.406 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:04:28.118535+0000", 2024-09-20T19:09:19.406 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 291.27529909399999, 2024-09-20T19:09:19.406 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.40251810599999999, 2024-09-20T19:09:19.406 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:09:19.406 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:09:19.406 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:09:19.406 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:09:19.406 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:09:19.406 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:09:19.406 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 147626 2024-09-20T19:09:19.406 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.406 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:09:19.407 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.407 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:09:19.407 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.118535+0000", 2024-09-20T19:09:19.407 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:19.407 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.407 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.407 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:09:19.407 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.118535+0000", 2024-09-20T19:09:19.407 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:19.407 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.407 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.407 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:09:19.407 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.118537+0000", 2024-09-20T19:09:19.407 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7910000000000001e-06 2024-09-20T19:09:19.408 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.408 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.408 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:09:19.408 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.118542+0000", 2024-09-20T19:09:19.408 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4049999999999999e-06 2024-09-20T19:09:19.408 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.408 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.408 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:09:19.408 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.118556+0000", 2024-09-20T19:09:19.408 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3988999999999999e-05 2024-09-20T19:09:19.408 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.408 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.408 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:09:19.408 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.118564+0000", 2024-09-20T19:09:19.408 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.0979999999999994e-06 2024-09-20T19:09:19.409 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.409 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.409 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:19.409 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.118622+0000", 2024-09-20T19:09:19.409 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.7796e-05 2024-09-20T19:09:19.409 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.409 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.409 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:09:19.409 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.118757+0000", 2024-09-20T19:09:19.409 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013468599999999999 2024-09-20T19:09:19.409 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.409 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.409 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 2,7", 2024-09-20T19:09:19.409 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.118877+0000", 2024-09-20T19:09:19.409 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000120103 2024-09-20T19:09:19.410 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.410 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.410 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:09:19.410 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.121979+0000", 2024-09-20T19:09:19.410 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0031019569999999998 2024-09-20T19:09:19.410 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.410 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.410 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:19.410 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.122589+0000", 2024-09-20T19:09:19.410 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00061044199999999999 2024-09-20T19:09:19.410 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.410 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.411 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:19.411 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.520939+0000", 2024-09-20T19:09:19.411 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.39834966599999999 2024-09-20T19:09:19.411 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.411 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.411 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:09:19.411 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.520995+0000", 2024-09-20T19:09:19.411 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5405999999999998e-05 2024-09-20T19:09:19.411 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.411 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.411 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:09:19.411 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.521053+0000", 2024-09-20T19:09:19.411 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8767000000000002e-05 2024-09-20T19:09:19.411 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:19.411 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:09:19.412 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:19.412 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.412 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.412 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:147635 10.9 10:95751c44:::benchmark_data_smithi083_148100_object30773:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e376)", 2024-09-20T19:09:19.412 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:04:28.131221+0000", 2024-09-20T19:09:19.412 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 291.26261349399999, 2024-09-20T19:09:19.412 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.398357515, 2024-09-20T19:09:19.412 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:09:19.412 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:09:19.412 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:09:19.412 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:09:19.412 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:09:19.412 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:09:19.412 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 147635 2024-09-20T19:09:19.413 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.413 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:09:19.413 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.413 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:09:19.413 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.131221+0000", 2024-09-20T19:09:19.413 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:19.413 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.413 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.413 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:09:19.413 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.131221+0000", 2024-09-20T19:09:19.413 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:19.413 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.413 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.414 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:09:19.414 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.131222+0000", 2024-09-20T19:09:19.414 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.575e-06 2024-09-20T19:09:19.414 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.414 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.414 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:09:19.414 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.131229+0000", 2024-09-20T19:09:19.414 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2740000000000004e-06 2024-09-20T19:09:19.414 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.414 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.414 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:09:19.414 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.131244+0000", 2024-09-20T19:09:19.414 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5362e-05 2024-09-20T19:09:19.415 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.415 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.415 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:09:19.415 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.131252+0000", 2024-09-20T19:09:19.415 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.3040000000000005e-06 2024-09-20T19:09:19.415 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.415 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.415 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:19.415 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.131312+0000", 2024-09-20T19:09:19.415 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9703000000000001e-05 2024-09-20T19:09:19.415 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.415 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.415 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:09:19.415 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.131421+0000", 2024-09-20T19:09:19.416 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00010873400000000001 2024-09-20T19:09:19.416 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.416 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.416 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 2,7", 2024-09-20T19:09:19.416 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.131541+0000", 2024-09-20T19:09:19.416 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000120625 2024-09-20T19:09:19.416 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.416 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.416 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:09:19.416 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.134390+0000", 2024-09-20T19:09:19.416 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0028481959999999999 2024-09-20T19:09:19.416 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.416 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.417 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:19.417 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.135079+0000", 2024-09-20T19:09:19.417 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00068968800000000004 2024-09-20T19:09:19.417 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.417 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.417 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:19.417 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.529488+0000", 2024-09-20T19:09:19.417 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.39440882500000002 2024-09-20T19:09:19.417 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.417 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.417 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:09:19.417 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.529539+0000", 2024-09-20T19:09:19.417 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.1038000000000002e-05 2024-09-20T19:09:19.417 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.417 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.418 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:09:19.418 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.529578+0000", 2024-09-20T19:09:19.418 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9190999999999998e-05 2024-09-20T19:09:19.418 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:19.418 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:09:19.418 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:19.418 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.418 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.418 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:147639 10.9 10:96d58120:::benchmark_data_smithi083_148100_object30777:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e376)", 2024-09-20T19:09:19.418 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:04:28.138921+0000", 2024-09-20T19:09:19.418 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 291.25491345099999, 2024-09-20T19:09:19.418 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.39076443700000002, 2024-09-20T19:09:19.418 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:09:19.419 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:09:19.419 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:09:19.419 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:09:19.419 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:09:19.419 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:09:19.419 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 147639 2024-09-20T19:09:19.419 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.419 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:09:19.419 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.419 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:09:19.419 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.138921+0000", 2024-09-20T19:09:19.419 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:19.419 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.419 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.420 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:09:19.420 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.138921+0000", 2024-09-20T19:09:19.420 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:19.420 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.420 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.420 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:09:19.420 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.138922+0000", 2024-09-20T19:09:19.420 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.598e-06 2024-09-20T19:09:19.420 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.420 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.420 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:09:19.420 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.138929+0000", 2024-09-20T19:09:19.420 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2890000000000003e-06 2024-09-20T19:09:19.420 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.420 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.421 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:09:19.421 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.138940+0000", 2024-09-20T19:09:19.421 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1292e-05 2024-09-20T19:09:19.421 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.421 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.421 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:09:19.421 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.138948+0000", 2024-09-20T19:09:19.421 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.7190000000000001e-06 2024-09-20T19:09:19.421 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.421 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.421 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:19.421 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.139013+0000", 2024-09-20T19:09:19.421 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5686999999999996e-05 2024-09-20T19:09:19.421 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.422 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.422 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:09:19.422 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.139133+0000", 2024-09-20T19:09:19.422 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000119709 2024-09-20T19:09:19.422 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.422 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.422 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 2,7", 2024-09-20T19:09:19.422 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.139232+0000", 2024-09-20T19:09:19.422 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.9275000000000001e-05 2024-09-20T19:09:19.422 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.422 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.422 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:09:19.422 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.142295+0000", 2024-09-20T19:09:19.422 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0030624210000000001 2024-09-20T19:09:19.422 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.423 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.423 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:19.423 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.142766+0000", 2024-09-20T19:09:19.423 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00047071800000000002 2024-09-20T19:09:19.423 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.423 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.423 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:19.423 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.529610+0000", 2024-09-20T19:09:19.423 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.38684496499999999 2024-09-20T19:09:19.423 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.423 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.423 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:09:19.423 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.529654+0000", 2024-09-20T19:09:19.423 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3988e-05 2024-09-20T19:09:19.424 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.424 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.424 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:09:19.424 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.529685+0000", 2024-09-20T19:09:19.424 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.0775999999999998e-05 2024-09-20T19:09:19.424 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:19.424 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:09:19.424 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:19.424 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.424 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.424 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:147641 10.9 10:957d4f1f:::benchmark_data_smithi083_148100_object30779:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e376)", 2024-09-20T19:09:19.424 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:04:28.142745+0000", 2024-09-20T19:09:19.424 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 291.25108968400002, 2024-09-20T19:09:19.424 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.38705916899999998, 2024-09-20T19:09:19.424 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:09:19.424 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:09:19.425 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:09:19.425 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:09:19.425 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:09:19.425 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:09:19.425 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 147641 2024-09-20T19:09:19.425 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.425 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:09:19.425 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.425 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:09:19.425 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.142745+0000", 2024-09-20T19:09:19.425 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:19.426 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.426 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.426 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:09:19.426 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.142745+0000", 2024-09-20T19:09:19.426 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:19.426 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.426 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.426 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:09:19.426 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.142746+0000", 2024-09-20T19:09:19.426 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7680000000000001e-06 2024-09-20T19:09:19.426 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.426 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.426 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:09:19.426 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.142752+0000", 2024-09-20T19:09:19.426 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.7019999999999998e-06 2024-09-20T19:09:19.426 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.427 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.427 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:09:19.427 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.142761+0000", 2024-09-20T19:09:19.427 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.4190000000000008e-06 2024-09-20T19:09:19.428 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.428 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.428 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:09:19.428 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.142768+0000", 2024-09-20T19:09:19.428 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.8009999999999998e-06 2024-09-20T19:09:19.428 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.428 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.428 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:19.428 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.142829+0000", 2024-09-20T19:09:19.428 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0537e-05 2024-09-20T19:09:19.428 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.428 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.428 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:09:19.429 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.142951+0000", 2024-09-20T19:09:19.429 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000122619 2024-09-20T19:09:19.429 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.429 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.429 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 2,7", 2024-09-20T19:09:19.429 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.143050+0000", 2024-09-20T19:09:19.429 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.8121999999999993e-05 2024-09-20T19:09:19.429 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.429 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.429 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:19.429 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.145907+0000", 2024-09-20T19:09:19.429 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0028578789999999998 2024-09-20T19:09:19.429 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.429 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.429 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:09:19.430 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.145926+0000", 2024-09-20T19:09:19.430 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8272000000000001e-05 2024-09-20T19:09:19.430 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.430 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.430 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:19.430 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.529726+0000", 2024-09-20T19:09:19.430 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.38380014200000001 2024-09-20T19:09:19.430 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.430 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.430 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:09:19.430 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.529774+0000", 2024-09-20T19:09:19.430 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8279999999999999e-05 2024-09-20T19:09:19.430 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.430 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.431 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:09:19.431 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:04:28.529804+0000", 2024-09-20T19:09:19.431 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9628000000000002e-05 2024-09-20T19:09:19.431 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:19.431 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:09:19.431 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:19.431 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.431 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.431 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:196171 10.6 10:6296a24c:::benchmark_data_smithi083_148100_object79309:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e393)", 2024-09-20T19:09:19.431 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:20.884625+0000", 2024-09-20T19:09:19.431 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 238.509208761, 2024-09-20T19:09:19.431 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0535514020000001, 2024-09-20T19:09:19.431 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:09:19.431 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:09:19.431 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-09-20T19:09:19.432 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:09:19.432 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:09:19.432 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:09:19.432 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 196171 2024-09-20T19:09:19.432 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.432 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:09:19.432 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.432 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:09:19.432 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.884625+0000", 2024-09-20T19:09:19.432 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:19.432 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.432 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.432 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:09:19.433 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.884625+0000", 2024-09-20T19:09:19.433 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:19.433 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.433 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.433 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:09:19.433 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.884627+0000", 2024-09-20T19:09:19.433 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.339e-06 2024-09-20T19:09:19.433 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.433 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.433 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:09:19.433 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.884631+0000", 2024-09-20T19:09:19.433 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.437e-06 2024-09-20T19:09:19.433 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.433 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.433 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:09:19.434 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.884642+0000", 2024-09-20T19:09:19.434 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0514e-05 2024-09-20T19:09:19.434 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.434 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.434 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:09:19.434 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.884649+0000", 2024-09-20T19:09:19.434 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.7159999999999997e-06 2024-09-20T19:09:19.434 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.434 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.434 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:19.434 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.884690+0000", 2024-09-20T19:09:19.434 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.1452000000000002e-05 2024-09-20T19:09:19.434 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.435 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.435 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:09:19.435 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.884763+0000", 2024-09-20T19:09:19.435 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.2774999999999994e-05 2024-09-20T19:09:19.435 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.435 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.435 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,4", 2024-09-20T19:09:19.435 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.884833+0000", 2024-09-20T19:09:19.435 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.0147000000000004e-05 2024-09-20T19:09:19.435 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.435 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.435 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:09:19.435 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.887337+0000", 2024-09-20T19:09:19.435 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0025039810000000002 2024-09-20T19:09:19.435 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.436 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.436 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:19.436 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.919111+0000", 2024-09-20T19:09:19.436 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.031774167999999998 2024-09-20T19:09:19.436 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.436 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.436 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:09:19.436 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.919127+0000", 2024-09-20T19:09:19.436 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5934999999999999e-05 2024-09-20T19:09:19.436 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.436 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.436 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:19.436 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.938106+0000", 2024-09-20T19:09:19.436 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.018978588 2024-09-20T19:09:19.436 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.437 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.437 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:09:19.437 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.938177+0000", 2024-09-20T19:09:19.437 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.135e-05 2024-09-20T19:09:19.437 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:19.437 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:09:19.437 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:19.437 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.437 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.437 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:196176 10.6 10:6158460d:::benchmark_data_smithi083_148100_object79314:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e393)", 2024-09-20T19:09:19.437 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:20.893880+0000", 2024-09-20T19:09:19.437 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 238.49995464200001, 2024-09-20T19:09:19.437 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0516040250000001, 2024-09-20T19:09:19.437 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:09:19.438 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:09:19.438 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:09:19.438 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:09:19.438 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:09:19.438 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:09:19.438 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 196176 2024-09-20T19:09:19.438 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.438 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:09:19.438 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.438 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:09:19.438 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.893880+0000", 2024-09-20T19:09:19.438 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:19.438 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.439 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.439 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:09:19.439 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.893880+0000", 2024-09-20T19:09:19.439 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:19.439 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.439 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.439 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:09:19.439 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.893881+0000", 2024-09-20T19:09:19.439 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.47e-07 2024-09-20T19:09:19.439 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.439 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.439 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:09:19.439 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.893885+0000", 2024-09-20T19:09:19.439 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3930000000000001e-06 2024-09-20T19:09:19.439 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.439 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.440 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:09:19.440 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.915561+0000", 2024-09-20T19:09:19.440 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.021675638000000001 2024-09-20T19:09:19.440 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.440 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.440 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:09:19.440 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.915571+0000", 2024-09-20T19:09:19.440 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0362e-05 2024-09-20T19:09:19.440 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.440 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.440 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:19.440 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.934415+0000", 2024-09-20T19:09:19.440 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0188439 2024-09-20T19:09:19.440 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.441 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.441 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:09:19.441 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.934425+0000", 2024-09-20T19:09:19.441 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0052e-05 2024-09-20T19:09:19.441 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.441 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.441 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:19.441 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.938653+0000", 2024-09-20T19:09:19.441 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0042276919999999 2024-09-20T19:09:19.441 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.441 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.441 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:09:19.441 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.938749+0000", 2024-09-20T19:09:19.441 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.6009999999999997e-05 2024-09-20T19:09:19.442 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.442 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.442 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,4", 2024-09-20T19:09:19.442 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.938860+0000", 2024-09-20T19:09:19.442 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000111764 2024-09-20T19:09:19.442 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.442 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.442 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:19.442 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.945361+0000", 2024-09-20T19:09:19.442 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0065005610000000002 2024-09-20T19:09:19.442 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.442 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.442 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:09:19.442 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.945372+0000", 2024-09-20T19:09:19.443 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1056000000000001e-05 2024-09-20T19:09:19.443 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.443 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.443 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:19.443 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.945401+0000", 2024-09-20T19:09:19.443 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9167999999999999e-05 2024-09-20T19:09:19.443 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.443 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.443 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:09:19.443 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.945449+0000", 2024-09-20T19:09:19.443 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8205999999999999e-05 2024-09-20T19:09:19.443 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.443 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.443 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:09:19.443 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.945484+0000", 2024-09-20T19:09:19.444 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4276000000000002e-05 2024-09-20T19:09:19.444 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:19.444 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:09:19.444 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:19.444 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.444 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.444 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:196198 10.6 10:6893dd7d:::benchmark_data_smithi083_148100_object79336:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e394)", 2024-09-20T19:09:19.444 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:20.948044+0000", 2024-09-20T19:09:19.444 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 238.44579040400001, 2024-09-20T19:09:19.444 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0041771209999999, 2024-09-20T19:09:19.444 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:09:19.444 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:09:19.444 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:09:19.444 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:09:19.444 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:09:19.445 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:09:19.445 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 196198 2024-09-20T19:09:19.445 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.445 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:09:19.445 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.445 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:09:19.445 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.948044+0000", 2024-09-20T19:09:19.445 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:19.445 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.445 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.445 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:09:19.445 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.948044+0000", 2024-09-20T19:09:19.446 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:19.446 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.446 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.446 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:09:19.446 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.948045+0000", 2024-09-20T19:09:19.446 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.8800000000000002e-07 2024-09-20T19:09:19.446 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.446 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.446 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:09:19.446 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.948049+0000", 2024-09-20T19:09:19.446 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0629999999999999e-06 2024-09-20T19:09:19.446 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.446 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.446 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:09:19.447 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.948059+0000", 2024-09-20T19:09:19.447 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0519e-05 2024-09-20T19:09:19.447 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.447 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.447 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:09:19.447 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.948066+0000", 2024-09-20T19:09:19.447 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.6229999999999999e-06 2024-09-20T19:09:19.447 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.447 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.447 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:19.447 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.948129+0000", 2024-09-20T19:09:19.447 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.3423000000000006e-05 2024-09-20T19:09:19.447 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.447 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.447 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:09:19.448 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.948948+0000", 2024-09-20T19:09:19.448 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00081862199999999997 2024-09-20T19:09:19.448 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.448 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.448 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:19.448 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.941881+0000", 2024-09-20T19:09:19.448 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.99293267299999999 2024-09-20T19:09:19.448 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.448 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.448 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:09:19.448 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.941984+0000", 2024-09-20T19:09:19.448 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000103571 2024-09-20T19:09:19.448 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.448 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.448 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,4", 2024-09-20T19:09:19.449 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.942099+0000", 2024-09-20T19:09:19.449 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000115085 2024-09-20T19:09:19.449 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.449 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.449 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:19.449 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.945520+0000", 2024-09-20T19:09:19.449 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0034205419999999999 2024-09-20T19:09:19.449 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.449 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.449 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:09:19.449 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.945530+0000", 2024-09-20T19:09:19.449 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.8079999999999999e-06 2024-09-20T19:09:19.449 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.449 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.449 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:19.450 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.952124+0000", 2024-09-20T19:09:19.450 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0065947929999999998 2024-09-20T19:09:19.450 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.450 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.450 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:09:19.450 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.952176+0000", 2024-09-20T19:09:19.450 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.1548000000000002e-05 2024-09-20T19:09:19.450 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.450 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.450 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:09:19.450 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.952221+0000", 2024-09-20T19:09:19.450 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.5062999999999999e-05 2024-09-20T19:09:19.450 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:19.451 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:09:19.451 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:19.451 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.451 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.451 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:196199 10.6 10:64c368fa:::benchmark_data_smithi083_148100_object79337:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e394)", 2024-09-20T19:09:19.451 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:20.948119+0000", 2024-09-20T19:09:19.451 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 238.445715533, 2024-09-20T19:09:19.451 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0137902990000001, 2024-09-20T19:09:19.451 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:09:19.451 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:09:19.451 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:09:19.451 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:09:19.451 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:09:19.451 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:09:19.451 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 196199 2024-09-20T19:09:19.451 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.452 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:09:19.452 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.452 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:09:19.452 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.948119+0000", 2024-09-20T19:09:19.452 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:19.452 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.452 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.452 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:09:19.452 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.948119+0000", 2024-09-20T19:09:19.452 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:19.452 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.452 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.452 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:09:19.452 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.948120+0000", 2024-09-20T19:09:19.452 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.78e-07 2024-09-20T19:09:19.452 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.453 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.453 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:09:19.453 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.948124+0000", 2024-09-20T19:09:19.453 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7530000000000001e-06 2024-09-20T19:09:19.453 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.453 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.453 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:09:19.453 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.948135+0000", 2024-09-20T19:09:19.453 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0614999999999999e-05 2024-09-20T19:09:19.453 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.453 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.453 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:09:19.453 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.948150+0000", 2024-09-20T19:09:19.453 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4684999999999999e-05 2024-09-20T19:09:19.453 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.453 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.454 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:19.454 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.949184+0000", 2024-09-20T19:09:19.454 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0010337479999999999 2024-09-20T19:09:19.454 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.454 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.454 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:09:19.454 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.949209+0000", 2024-09-20T19:09:19.454 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.5122000000000001e-05 2024-09-20T19:09:19.454 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.454 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.454 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:19.454 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.945551+0000", 2024-09-20T19:09:19.454 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.99634271399999996 2024-09-20T19:09:19.454 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.454 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.455 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:09:19.455 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.945655+0000", 2024-09-20T19:09:19.455 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000103538 2024-09-20T19:09:19.455 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.455 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.455 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,4", 2024-09-20T19:09:19.455 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.945752+0000", 2024-09-20T19:09:19.455 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.6932e-05 2024-09-20T19:09:19.455 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.455 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.455 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:19.455 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.952257+0000", 2024-09-20T19:09:19.455 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0065051639999999999 2024-09-20T19:09:19.455 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.455 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.456 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:19.456 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.952290+0000", 2024-09-20T19:09:19.456 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.3149999999999999e-05 2024-09-20T19:09:19.456 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.456 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.456 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:09:19.456 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.961822+0000", 2024-09-20T19:09:19.456 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.009532067 2024-09-20T19:09:19.456 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.456 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.456 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:09:19.456 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.961870+0000", 2024-09-20T19:09:19.456 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7515000000000002e-05 2024-09-20T19:09:19.456 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.456 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.456 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:09:19.456 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.961909+0000", 2024-09-20T19:09:19.457 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9317999999999999e-05 2024-09-20T19:09:19.457 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:19.457 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:09:19.457 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:19.457 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.457 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.457 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:196203 10.6 10:60330bdb:::benchmark_data_smithi083_148100_object79341:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e394)", 2024-09-20T19:09:19.457 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:20.952749+0000", 2024-09-20T19:09:19.457 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 238.441084926, 2024-09-20T19:09:19.457 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0092730700000001, 2024-09-20T19:09:19.457 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:09:19.457 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:09:19.457 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:09:19.457 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:09:19.458 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:09:19.458 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:09:19.458 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 196203 2024-09-20T19:09:19.458 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.458 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:09:19.458 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.458 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:09:19.458 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.952749+0000", 2024-09-20T19:09:19.458 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:19.458 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.458 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.458 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:09:19.458 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.952749+0000", 2024-09-20T19:09:19.458 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:19.458 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.458 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.459 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:09:19.459 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.952751+0000", 2024-09-20T19:09:19.459 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6619999999999999e-06 2024-09-20T19:09:19.459 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.459 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.459 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:09:19.459 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.952757+0000", 2024-09-20T19:09:19.459 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.3090000000000001e-06 2024-09-20T19:09:19.459 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.459 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.459 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:09:19.459 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.952769+0000", 2024-09-20T19:09:19.460 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1769e-05 2024-09-20T19:09:19.460 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.460 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.460 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:09:19.460 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.952785+0000", 2024-09-20T19:09:19.460 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.554e-05 2024-09-20T19:09:19.460 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.460 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.460 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:19.460 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.952854+0000", 2024-09-20T19:09:19.460 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.9089000000000002e-05 2024-09-20T19:09:19.460 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.461 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.461 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:09:19.461 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.952865+0000", 2024-09-20T19:09:19.461 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1211e-05 2024-09-20T19:09:19.461 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.461 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.461 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:19.461 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.948741+0000", 2024-09-20T19:09:19.461 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.99587615399999996 2024-09-20T19:09:19.461 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.461 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.461 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:09:19.461 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.948950+0000", 2024-09-20T19:09:19.462 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00020904800000000001 2024-09-20T19:09:19.462 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.462 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.462 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,4", 2024-09-20T19:09:19.462 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.949070+0000", 2024-09-20T19:09:19.462 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000120241 2024-09-20T19:09:19.462 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.462 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.462 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:19.462 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.952325+0000", 2024-09-20T19:09:19.462 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0032544190000000001 2024-09-20T19:09:19.462 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.462 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.462 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:09:19.462 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.961916+0000", 2024-09-20T19:09:19.463 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0095912749999999998 2024-09-20T19:09:19.463 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.463 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.463 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:19.463 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.961956+0000", 2024-09-20T19:09:19.463 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9975999999999998e-05 2024-09-20T19:09:19.463 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.463 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.463 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:09:19.463 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.961993+0000", 2024-09-20T19:09:19.463 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.7076000000000002e-05 2024-09-20T19:09:19.463 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.463 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.463 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:09:19.463 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.962022+0000", 2024-09-20T19:09:19.464 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9300999999999999e-05 2024-09-20T19:09:19.464 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:19.464 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:09:19.464 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:19.464 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.464 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.464 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:196222 10.6 10:61e1d2b6:::benchmark_data_smithi083_148100_object79360:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e394)", 2024-09-20T19:09:19.464 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:20.969258+0000", 2024-09-20T19:09:19.464 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 238.424576524, 2024-09-20T19:09:19.464 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.99987098600000002, 2024-09-20T19:09:19.464 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:09:19.464 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:09:19.464 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:09:19.465 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:09:19.465 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:09:19.465 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:09:19.465 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 196222 2024-09-20T19:09:19.465 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.465 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:09:19.465 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.465 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:09:19.465 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.969258+0000", 2024-09-20T19:09:19.465 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:19.465 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.465 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.465 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:09:19.465 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.969258+0000", 2024-09-20T19:09:19.465 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:19.466 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.466 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.466 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:09:19.466 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.969259+0000", 2024-09-20T19:09:19.466 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.491e-06 2024-09-20T19:09:19.466 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.466 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.466 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:09:19.466 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.969264+0000", 2024-09-20T19:09:19.466 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3540000000000002e-06 2024-09-20T19:09:19.466 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.466 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.466 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:09:19.466 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.969280+0000", 2024-09-20T19:09:19.466 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6056e-05 2024-09-20T19:09:19.467 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.467 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.467 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:09:19.467 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.969289+0000", 2024-09-20T19:09:19.467 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.9469999999999992e-06 2024-09-20T19:09:19.467 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.467 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.467 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:19.468 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.969627+0000", 2024-09-20T19:09:19.468 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00033818500000000002 2024-09-20T19:09:19.468 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.468 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.468 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:09:19.468 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.969650+0000", 2024-09-20T19:09:19.468 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2878999999999999e-05 2024-09-20T19:09:19.468 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.468 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.468 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:19.468 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.952352+0000", 2024-09-20T19:09:19.468 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.98270262399999997 2024-09-20T19:09:19.468 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.469 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.469 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:09:19.469 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.952449+0000", 2024-09-20T19:09:19.469 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.6273000000000005e-05 2024-09-20T19:09:19.469 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.469 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.469 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,4", 2024-09-20T19:09:19.469 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.952555+0000", 2024-09-20T19:09:19.469 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000106038 2024-09-20T19:09:19.469 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.469 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.469 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:09:19.469 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.961921+0000", 2024-09-20T19:09:19.469 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0093659710000000007 2024-09-20T19:09:19.470 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.470 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.470 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:19.470 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.962078+0000", 2024-09-20T19:09:19.470 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000157458 2024-09-20T19:09:19.470 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.470 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.470 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:19.470 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.968986+0000", 2024-09-20T19:09:19.470 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0069078810000000003 2024-09-20T19:09:19.470 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.470 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.470 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:09:19.470 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.969081+0000", 2024-09-20T19:09:19.471 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.5468000000000001e-05 2024-09-20T19:09:19.471 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.471 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.471 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:09:19.471 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.969129+0000", 2024-09-20T19:09:19.471 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7361000000000003e-05 2024-09-20T19:09:19.471 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:19.471 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:09:19.471 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:19.471 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.471 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.471 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:196225 10.6 10:6fb9381d:::benchmark_data_smithi083_148100_object79363:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e394)", 2024-09-20T19:09:19.472 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:20.970076+0000", 2024-09-20T19:09:19.472 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 238.423758539, 2024-09-20T19:09:19.472 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.99929758400000002, 2024-09-20T19:09:19.472 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:09:19.472 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:09:19.472 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:09:19.472 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:09:19.472 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:09:19.472 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:09:19.472 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 196225 2024-09-20T19:09:19.472 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.472 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:09:19.472 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.472 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:09:19.472 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.970076+0000", 2024-09-20T19:09:19.473 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:19.473 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.473 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.473 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:09:19.473 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.970076+0000", 2024-09-20T19:09:19.473 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:19.473 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.473 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.473 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:09:19.473 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.970085+0000", 2024-09-20T19:09:19.473 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.2269999999999999e-06 2024-09-20T19:09:19.473 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.473 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.473 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:09:19.473 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.970090+0000", 2024-09-20T19:09:19.473 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7040000000000002e-06 2024-09-20T19:09:19.474 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.474 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.474 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:09:19.474 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.970101+0000", 2024-09-20T19:09:19.474 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1779e-05 2024-09-20T19:09:19.474 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.474 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.474 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:09:19.474 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.970109+0000", 2024-09-20T19:09:19.474 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.4680000000000003e-06 2024-09-20T19:09:19.474 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.474 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.474 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:19.474 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.970171+0000", 2024-09-20T19:09:19.474 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.1754000000000005e-05 2024-09-20T19:09:19.475 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.475 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.475 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:09:19.475 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.970179+0000", 2024-09-20T19:09:19.475 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.3280000000000006e-06 2024-09-20T19:09:19.475 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.475 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.475 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:19.475 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.955348+0000", 2024-09-20T19:09:19.475 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.98516891799999995 2024-09-20T19:09:19.475 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.475 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.475 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:09:19.475 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.955467+0000", 2024-09-20T19:09:19.475 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011861799999999999 2024-09-20T19:09:19.476 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.476 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.476 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,4", 2024-09-20T19:09:19.476 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.955581+0000", 2024-09-20T19:09:19.476 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011460599999999999 2024-09-20T19:09:19.476 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.476 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.476 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:09:19.476 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.962030+0000", 2024-09-20T19:09:19.476 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0064484900000000003 2024-09-20T19:09:19.476 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.476 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.476 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:19.476 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.969205+0000", 2024-09-20T19:09:19.476 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0071753449999999996 2024-09-20T19:09:19.476 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.476 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.477 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:19.477 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.969268+0000", 2024-09-20T19:09:19.477 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2830000000000007e-05 2024-09-20T19:09:19.477 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.477 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.477 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:09:19.477 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.969336+0000", 2024-09-20T19:09:19.477 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.7737999999999994e-05 2024-09-20T19:09:19.477 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.477 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.477 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:09:19.477 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.969373+0000", 2024-09-20T19:09:19.477 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.7778999999999999e-05 2024-09-20T19:09:19.477 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:19.477 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:09:19.478 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:19.478 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.478 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.478 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:196235 10.6 10:667fa116:::benchmark_data_smithi083_148100_object79373:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e394)", 2024-09-20T19:09:19.478 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:20.981304+0000", 2024-09-20T19:09:19.478 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 238.412530369, 2024-09-20T19:09:19.478 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.98825587299999995, 2024-09-20T19:09:19.478 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:09:19.478 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:09:19.478 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:09:19.478 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:09:19.478 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:09:19.478 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:09:19.479 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 196235 2024-09-20T19:09:19.479 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.479 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:09:19.479 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.479 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:09:19.479 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.981304+0000", 2024-09-20T19:09:19.479 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:19.479 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.479 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.479 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:09:19.479 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.981304+0000", 2024-09-20T19:09:19.479 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:19.479 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.479 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.479 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:09:19.480 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.981306+0000", 2024-09-20T19:09:19.480 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8029999999999999e-06 2024-09-20T19:09:19.480 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.480 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.480 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:09:19.480 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.981311+0000", 2024-09-20T19:09:19.480 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.7239999999999997e-06 2024-09-20T19:09:19.480 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.480 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.480 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:09:19.480 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.981327+0000", 2024-09-20T19:09:19.481 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5954999999999999e-05 2024-09-20T19:09:19.481 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.481 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.481 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:09:19.481 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.981336+0000", 2024-09-20T19:09:19.481 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.5599999999999994e-06 2024-09-20T19:09:19.481 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.481 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.481 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:19.481 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.981434+0000", 2024-09-20T19:09:19.481 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.8542000000000006e-05 2024-09-20T19:09:19.481 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.481 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.482 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:09:19.482 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.981444+0000", 2024-09-20T19:09:19.482 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.7480000000000005e-06 2024-09-20T19:09:19.482 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.482 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.482 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:19.482 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.958453+0000", 2024-09-20T19:09:19.482 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.97700898800000002 2024-09-20T19:09:19.482 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.482 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.482 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:09:19.482 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.958579+0000", 2024-09-20T19:09:19.482 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000125683 2024-09-20T19:09:19.482 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.482 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.483 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,4", 2024-09-20T19:09:19.483 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.958697+0000", 2024-09-20T19:09:19.483 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011854 2024-09-20T19:09:19.483 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.483 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.483 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:09:19.483 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.969142+0000", 2024-09-20T19:09:19.483 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.010444248 2024-09-20T19:09:19.483 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.483 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.483 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:19.483 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.969425+0000", 2024-09-20T19:09:19.483 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00028332200000000002 2024-09-20T19:09:19.483 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.484 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.484 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:19.484 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.969476+0000", 2024-09-20T19:09:19.484 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0565999999999997e-05 2024-09-20T19:09:19.484 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.484 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.484 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:09:19.484 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.969522+0000", 2024-09-20T19:09:19.484 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.6745000000000001e-05 2024-09-20T19:09:19.484 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.484 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.484 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:09:19.484 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.969560+0000", 2024-09-20T19:09:19.484 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.7449e-05 2024-09-20T19:09:19.484 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:19.484 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:09:19.484 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:19.485 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.485 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.485 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:196237 10.6 10:62e94daf:::benchmark_data_smithi083_148100_object79375:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e394)", 2024-09-20T19:09:19.485 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:20.982454+0000", 2024-09-20T19:09:19.485 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 238.411380592, 2024-09-20T19:09:19.485 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.987291221, 2024-09-20T19:09:19.485 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:09:19.485 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:09:19.485 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:09:19.485 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:09:19.485 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:09:19.485 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:09:19.485 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 196237 2024-09-20T19:09:19.485 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.485 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:09:19.485 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.486 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:09:19.486 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.982454+0000", 2024-09-20T19:09:19.486 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:19.486 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.486 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.486 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:09:19.486 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.982454+0000", 2024-09-20T19:09:19.486 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:19.486 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.486 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.486 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:09:19.486 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.982456+0000", 2024-09-20T19:09:19.486 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8589999999999999e-06 2024-09-20T19:09:19.486 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.486 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.487 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:09:19.487 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.982461+0000", 2024-09-20T19:09:19.487 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0300000000000001e-06 2024-09-20T19:09:19.487 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.487 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.487 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:09:19.487 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.982475+0000", 2024-09-20T19:09:19.487 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3986e-05 2024-09-20T19:09:19.487 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.487 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.487 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:09:19.487 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.982483+0000", 2024-09-20T19:09:19.487 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.3410000000000003e-06 2024-09-20T19:09:19.487 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.487 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.487 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:19.488 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.982573+0000", 2024-09-20T19:09:19.488 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.0099000000000006e-05 2024-09-20T19:09:19.488 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.488 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.488 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:09:19.488 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:20.982583+0000", 2024-09-20T19:09:19.488 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.5189999999999998e-06 2024-09-20T19:09:19.488 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.488 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.488 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:19.488 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.962104+0000", 2024-09-20T19:09:19.488 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.97952148100000003 2024-09-20T19:09:19.488 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.488 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.488 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:09:19.488 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.962278+0000", 2024-09-20T19:09:19.489 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00017353299999999999 2024-09-20T19:09:19.489 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.489 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.489 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,4", 2024-09-20T19:09:19.489 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.962386+0000", 2024-09-20T19:09:19.489 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000108365 2024-09-20T19:09:19.489 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.489 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.489 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:09:19.489 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.969149+0000", 2024-09-20T19:09:19.489 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00676342 2024-09-20T19:09:19.489 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.489 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.489 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:19.490 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.969612+0000", 2024-09-20T19:09:19.490 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00046227099999999999 2024-09-20T19:09:19.490 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.490 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.490 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:19.490 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.969665+0000", 2024-09-20T19:09:19.490 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.2988999999999997e-05 2024-09-20T19:09:19.490 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.490 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.490 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:09:19.490 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.969710+0000", 2024-09-20T19:09:19.490 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.4975000000000002e-05 2024-09-20T19:09:19.490 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.490 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.491 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:09:19.491 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.969745+0000", 2024-09-20T19:09:19.491 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.5352999999999998e-05 2024-09-20T19:09:19.491 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:19.491 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:09:19.491 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:19.491 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.491 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.491 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:196260 10.6 10:683aac99:::benchmark_data_smithi083_148100_object79398:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e394)", 2024-09-20T19:09:19.491 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:21.007238+0000", 2024-09-20T19:09:19.491 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 238.386596293, 2024-09-20T19:09:19.491 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.96912988799999999, 2024-09-20T19:09:19.491 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:09:19.491 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:09:19.492 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:09:19.492 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:09:19.492 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:09:19.492 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:09:19.492 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 196260 2024-09-20T19:09:19.492 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.492 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:09:19.492 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.492 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:09:19.492 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.007238+0000", 2024-09-20T19:09:19.492 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:19.492 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.492 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.492 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:09:19.492 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.007238+0000", 2024-09-20T19:09:19.493 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:19.493 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.493 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.493 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:09:19.493 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.007240+0000", 2024-09-20T19:09:19.493 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3769999999999999e-06 2024-09-20T19:09:19.493 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.493 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.493 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:09:19.493 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.007248+0000", 2024-09-20T19:09:19.493 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5939999999999996e-06 2024-09-20T19:09:19.493 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.493 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.493 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:09:19.493 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.007267+0000", 2024-09-20T19:09:19.494 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8885999999999998e-05 2024-09-20T19:09:19.494 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.494 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.494 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:09:19.494 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.007277+0000", 2024-09-20T19:09:19.494 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0519999999999999e-05 2024-09-20T19:09:19.494 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.494 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.494 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:19.494 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.007358+0000", 2024-09-20T19:09:19.494 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.0189000000000001e-05 2024-09-20T19:09:19.494 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.494 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.494 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:09:19.494 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.007369+0000", 2024-09-20T19:09:19.495 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1046000000000001e-05 2024-09-20T19:09:19.495 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.495 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.495 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:19.495 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.965540+0000", 2024-09-20T19:09:19.495 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.95817183100000003 2024-09-20T19:09:19.495 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.495 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.495 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:09:19.495 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.965695+0000", 2024-09-20T19:09:19.495 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000154482 2024-09-20T19:09:19.495 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.495 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.495 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,4", 2024-09-20T19:09:19.496 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.965812+0000", 2024-09-20T19:09:19.496 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000117409 2024-09-20T19:09:19.496 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.496 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.496 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:09:19.496 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.969389+0000", 2024-09-20T19:09:19.496 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0035766169999999998 2024-09-20T19:09:19.496 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.496 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.496 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:19.496 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.969811+0000", 2024-09-20T19:09:19.496 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00042253000000000001 2024-09-20T19:09:19.496 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.496 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.496 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:19.496 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.976270+0000", 2024-09-20T19:09:19.497 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0064581120000000002 2024-09-20T19:09:19.497 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.497 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.497 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:09:19.497 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.976327+0000", 2024-09-20T19:09:19.497 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.7305000000000001e-05 2024-09-20T19:09:19.497 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.497 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.497 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:09:19.497 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.976368+0000", 2024-09-20T19:09:19.497 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0989999999999999e-05 2024-09-20T19:09:19.497 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:19.497 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:09:19.497 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:19.498 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.498 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.498 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:196295 10.6 10:605954dd:::benchmark_data_smithi083_148100_object79433:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e394)", 2024-09-20T19:09:19.498 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:21.077278+0000", 2024-09-20T19:09:19.498 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 238.31655660499999, 2024-09-20T19:09:19.498 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.90257779000000005, 2024-09-20T19:09:19.498 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:09:19.498 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:09:19.498 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:09:19.498 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:09:19.498 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:09:19.498 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:09:19.498 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 196295 2024-09-20T19:09:19.499 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.499 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:09:19.499 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.499 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:09:19.499 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.077278+0000", 2024-09-20T19:09:19.499 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:19.499 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.499 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.499 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:09:19.499 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.077278+0000", 2024-09-20T19:09:19.499 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:19.499 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.499 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.499 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:09:19.500 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.077280+0000", 2024-09-20T19:09:19.500 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.88e-06 2024-09-20T19:09:19.500 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.500 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.500 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:09:19.500 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.077287+0000", 2024-09-20T19:09:19.500 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.1439999999999997e-06 2024-09-20T19:09:19.500 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.500 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.500 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:09:19.500 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.077302+0000", 2024-09-20T19:09:19.500 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4888e-05 2024-09-20T19:09:19.500 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.500 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.500 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:09:19.500 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.077308+0000", 2024-09-20T19:09:19.501 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.8000000000000001e-06 2024-09-20T19:09:19.501 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.501 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.501 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:19.501 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.077365+0000", 2024-09-20T19:09:19.501 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.7058000000000002e-05 2024-09-20T19:09:19.501 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.501 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.501 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:09:19.501 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.077372+0000", 2024-09-20T19:09:19.501 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.9709999999999998e-06 2024-09-20T19:09:19.501 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.501 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.501 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:19.501 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.969853+0000", 2024-09-20T19:09:19.501 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.89248060600000001 2024-09-20T19:09:19.501 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.502 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.502 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:09:19.502 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.969955+0000", 2024-09-20T19:09:19.502 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000101699 2024-09-20T19:09:19.502 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.502 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.502 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,4", 2024-09-20T19:09:19.502 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.970067+0000", 2024-09-20T19:09:19.502 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011228800000000001 2024-09-20T19:09:19.502 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.502 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.502 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:09:19.502 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.979653+0000", 2024-09-20T19:09:19.502 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0095861339999999996 2024-09-20T19:09:19.502 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.502 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.503 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:19.503 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.979713+0000", 2024-09-20T19:09:19.503 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0353999999999998e-05 2024-09-20T19:09:19.503 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.503 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.503 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:19.503 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.979758+0000", 2024-09-20T19:09:19.503 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.4894000000000001e-05 2024-09-20T19:09:19.503 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.503 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.503 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:09:19.503 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.979812+0000", 2024-09-20T19:09:19.503 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3164999999999997e-05 2024-09-20T19:09:19.503 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.503 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.503 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:09:19.504 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:21.979855+0000", 2024-09-20T19:09:19.504 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3909000000000003e-05 2024-09-20T19:09:19.504 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:19.504 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:09:19.504 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:19.504 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.504 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.504 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.26264.0:60562 11.7 11:e684034d:::benchmark_data_smithi083_177491_object60561:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e441)", 2024-09-20T19:09:19.504 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:06:55.431841+0000", 2024-09-20T19:09:19.504 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 143.96199368000001, 2024-09-20T19:09:19.504 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.060064095, 2024-09-20T19:09:19.504 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:09:19.504 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:09:19.504 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-09-20T19:09:19.504 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:09:19.504 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.26264", 2024-09-20T19:09:19.505 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2608780316", 2024-09-20T19:09:19.505 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 60562 2024-09-20T19:09:19.505 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.505 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:09:19.505 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.505 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:09:19.505 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:55.431841+0000", 2024-09-20T19:09:19.505 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:19.505 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.505 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.505 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:09:19.505 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:55.431841+0000", 2024-09-20T19:09:19.505 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:19.505 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.505 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.505 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:09:19.505 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:55.431849+0000", 2024-09-20T19:09:19.506 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.4989999999999994e-06 2024-09-20T19:09:19.506 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.506 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.506 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:09:19.506 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:55.431907+0000", 2024-09-20T19:09:19.506 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:09:19.507 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.7775999999999997e-05 2024-09-20T19:09:19.507 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.507 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.507 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:09:19.507 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:55.431930+0000", 2024-09-20T19:09:19.507 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2891000000000002e-05 2024-09-20T19:09:19.507 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.507 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.507 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:09:19.507 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:55.431940+0000", 2024-09-20T19:09:19.507 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.7759999999999992e-06 2024-09-20T19:09:19.507 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.508 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.508 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:19.508 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:55.432509+0000", 2024-09-20T19:09:19.508 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00056960500000000005 2024-09-20T19:09:19.508 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.508 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.508 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op must wait for map", 2024-09-20T19:09:19.508 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:55.432514+0000", 2024-09-20T19:09:19.508 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.118e-06 2024-09-20T19:09:19.508 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.508 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.508 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:19.508 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:55.551905+0000", 2024-09-20T19:09:19.508 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.119390828 2024-09-20T19:09:19.508 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.509 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.509 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:09:19.509 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:55.551915+0000", 2024-09-20T19:09:19.509 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.6709999999999994e-06 2024-09-20T19:09:19.509 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.509 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.509 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:19.509 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:56.481370+0000", 2024-09-20T19:09:19.509 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.92945523299999999 2024-09-20T19:09:19.509 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.509 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.509 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for degraded object", 2024-09-20T19:09:19.509 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:56.482836+0000", 2024-09-20T19:09:19.509 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0014662 2024-09-20T19:09:19.509 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.509 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.510 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:19.510 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:56.491782+0000", 2024-09-20T19:09:19.510 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0089462789999999997 2024-09-20T19:09:19.510 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.510 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.510 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:09:19.510 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:56.491905+0000", 2024-09-20T19:09:19.510 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012221899999999999 2024-09-20T19:09:19.510 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:19.510 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:09:19.510 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:19.510 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.510 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.510 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.26264.0:60563 11.7 11:e24a71dd:::benchmark_data_smithi083_177491_object60562:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e441)", 2024-09-20T19:09:19.510 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:06:55.431963+0000", 2024-09-20T19:09:19.511 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 143.961871188, 2024-09-20T19:09:19.511 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.060358132, 2024-09-20T19:09:19.511 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:09:19.511 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:09:19.511 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-09-20T19:09:19.511 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:09:19.511 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.26264", 2024-09-20T19:09:19.511 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2608780316", 2024-09-20T19:09:19.511 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 60563 2024-09-20T19:09:19.511 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.511 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:09:19.511 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.511 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:09:19.511 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:55.431963+0000", 2024-09-20T19:09:19.511 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:19.512 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.512 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.512 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:09:19.512 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:55.431963+0000", 2024-09-20T19:09:19.512 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:19.512 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.512 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.512 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:09:19.512 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:55.431964+0000", 2024-09-20T19:09:19.512 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9100000000000004e-07 2024-09-20T19:09:19.512 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.512 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.512 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:09:19.512 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:55.431993+0000", 2024-09-20T19:09:19.512 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9164000000000001e-05 2024-09-20T19:09:19.512 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.513 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.513 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:09:19.513 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:55.431999+0000", 2024-09-20T19:09:19.513 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4350000000000002e-06 2024-09-20T19:09:19.513 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.513 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.513 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:09:19.513 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:55.432005+0000", 2024-09-20T19:09:19.513 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.7119999999999997e-06 2024-09-20T19:09:19.513 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.513 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.513 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:19.513 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:55.432562+0000", 2024-09-20T19:09:19.513 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00055732799999999995 2024-09-20T19:09:19.513 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.513 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.514 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting_for_map not empty", 2024-09-20T19:09:19.514 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:55.432566+0000", 2024-09-20T19:09:19.514 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9169999999999999e-06 2024-09-20T19:09:19.514 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.514 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.514 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:19.514 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:55.551938+0000", 2024-09-20T19:09:19.514 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.119371803 2024-09-20T19:09:19.514 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.514 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.514 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:09:19.514 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:55.551945+0000", 2024-09-20T19:09:19.514 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.8390000000000001e-06 2024-09-20T19:09:19.514 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.514 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.515 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:19.515 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:56.483282+0000", 2024-09-20T19:09:19.515 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.93133691799999996 2024-09-20T19:09:19.515 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.515 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.515 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for degraded object", 2024-09-20T19:09:19.515 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:56.485117+0000", 2024-09-20T19:09:19.515 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0018347929999999999 2024-09-20T19:09:19.515 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.515 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.515 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:19.515 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:56.492208+0000", 2024-09-20T19:09:19.515 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0070911899999999998 2024-09-20T19:09:19.515 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.516 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.516 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:09:19.516 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:56.492321+0000", 2024-09-20T19:09:19.516 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000113442 2024-09-20T19:09:19.516 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:19.516 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:09:19.516 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:19.516 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.516 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.516 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.26264.0:62080 11.16 11.639f7d16 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e443)", 2024-09-20T19:09:19.516 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:06:57.474982+0000", 2024-09-20T19:09:19.516 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 141.918851872, 2024-09-20T19:09:19.516 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.97798127700000004, 2024-09-20T19:09:19.516 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:09:19.517 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:09:19.517 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-09-20T19:09:19.517 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:09:19.517 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.26264", 2024-09-20T19:09:19.517 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2608780316", 2024-09-20T19:09:19.517 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 62080 2024-09-20T19:09:19.517 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.517 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:09:19.517 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.517 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:09:19.517 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:57.474982+0000", 2024-09-20T19:09:19.517 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:19.517 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.517 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.517 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:09:19.517 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:57.474982+0000", 2024-09-20T19:09:19.518 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:19.518 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.518 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.518 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:09:19.518 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:57.474984+0000", 2024-09-20T19:09:19.518 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.699e-06 2024-09-20T19:09:19.518 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.518 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.518 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:09:19.518 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:57.475057+0000", 2024-09-20T19:09:19.518 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.2861000000000006e-05 2024-09-20T19:09:19.518 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.518 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.518 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:09:19.518 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:57.475066+0000", 2024-09-20T19:09:19.519 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.9809999999999992e-06 2024-09-20T19:09:19.519 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.519 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.519 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:09:19.519 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:57.475072+0000", 2024-09-20T19:09:19.519 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5629999999999997e-06 2024-09-20T19:09:19.519 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.519 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.519 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:19.519 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:57.475137+0000", 2024-09-20T19:09:19.519 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4264000000000007e-05 2024-09-20T19:09:19.519 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.519 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.519 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:09:19.520 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:57.475145+0000", 2024-09-20T19:09:19.520 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.3100000000000001e-06 2024-09-20T19:09:19.520 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.520 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.520 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:19.520 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:58.452954+0000", 2024-09-20T19:09:19.520 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.97780911400000003 2024-09-20T19:09:19.520 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.520 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.520 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:09:19.520 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:58.452964+0000", 2024-09-20T19:09:19.520 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.4849999999999998e-06 2024-09-20T19:09:19.521 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:19.521 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:09:19.521 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:19.521 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.521 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.521 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.26264.0:62081 11.13 11.7c6096d3 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e443)", 2024-09-20T19:09:19.521 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:06:57.475341+0000", 2024-09-20T19:09:19.521 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 141.918493447, 2024-09-20T19:09:19.521 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.97561901600000001, 2024-09-20T19:09:19.521 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:09:19.521 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:09:19.521 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-09-20T19:09:19.521 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:09:19.521 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.26264", 2024-09-20T19:09:19.522 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2608780316", 2024-09-20T19:09:19.522 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 62081 2024-09-20T19:09:19.522 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.522 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:09:19.522 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.522 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:09:19.522 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:57.475341+0000", 2024-09-20T19:09:19.522 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:19.522 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.522 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.522 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:09:19.522 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:57.475341+0000", 2024-09-20T19:09:19.522 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:19.522 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.523 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.523 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:09:19.523 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:57.475342+0000", 2024-09-20T19:09:19.523 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.7899999999999997e-07 2024-09-20T19:09:19.523 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.523 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.523 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:09:19.523 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:57.475405+0000", 2024-09-20T19:09:19.523 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.3192000000000001e-05 2024-09-20T19:09:19.523 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.523 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.523 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:09:19.523 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:57.475419+0000", 2024-09-20T19:09:19.523 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.438e-05 2024-09-20T19:09:19.523 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.524 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.524 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:09:19.524 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:57.475427+0000", 2024-09-20T19:09:19.524 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.396e-06 2024-09-20T19:09:19.524 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.524 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.524 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:19.524 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:57.475523+0000", 2024-09-20T19:09:19.524 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.6265000000000001e-05 2024-09-20T19:09:19.524 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.524 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.524 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:09:19.524 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:57.475528+0000", 2024-09-20T19:09:19.524 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8149999999999996e-06 2024-09-20T19:09:19.524 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.525 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.525 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:19.525 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:58.450949+0000", 2024-09-20T19:09:19.525 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.97542104600000001 2024-09-20T19:09:19.525 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:19.525 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:19.525 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:09:19.525 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:58.450960+0000", 2024-09-20T19:09:19.525 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1043000000000001e-05 2024-09-20T19:09:19.525 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:19.525 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:09:19.525 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:19.525 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:19.525 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:09:19.525 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T19:09:19.567 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:19.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:19.568 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:09:19.669 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:09:19.706 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:19.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:19.707 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:09:19.808 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:09:19.846 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:19.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:19.847 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:09:19.948 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:09:19.988 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:19.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:19.989 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:09:20.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:19 smithi142 ceph-mon[32508]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:09:20.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:19 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T19:09:20.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:19 smithi142 ceph-mon[32508]: osdmap e492: 8 total, 8 up, 8 in 2024-09-20T19:09:20.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:19 smithi083 ceph-mon[29421]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:09:20.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:19 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T19:09:20.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:19 smithi083 ceph-mon[29421]: osdmap e492: 8 total, 8 up, 8 in 2024-09-20T19:09:20.040 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:19 smithi083 ceph-mon[36260]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:09:20.040 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:19 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T19:09:20.040 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:19 smithi083 ceph-mon[36260]: osdmap e492: 8 total, 8 up, 8 in 2024-09-20T19:09:20.090 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:09:20.135 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:20.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:20.136 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:09:20.236 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:09:20.281 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:20.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:20.282 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:09:20.383 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:09:20.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:20.419 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:20.419 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:09:20.520 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:09:20.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:20.554 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:20.554 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:09:20.655 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:09:20.688 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:20.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:20.689 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:09:20.790 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:09:20.824 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:20.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:20.825 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:09:20.904 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-20T19:09:20.926 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:09:20.978 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:20.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:20.979 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:09:21.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:20 smithi083 ceph-mon[29421]: pgmap v993: 168 pgs: 6 active+remapped+backfill_toofull, 162 active+clean; 1.1 GiB data, 9.6 GiB used, 706 GiB / 715 GiB avail; 993 B/s rd, 0 B/s wr, 708 op/s; 8181/56424 objects misplaced (14.499%); 5.6 MiB/s, 89 objects/s recovering 2024-09-20T19:09:21.040 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:09:20 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:09:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:09:21.041 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:20 smithi083 ceph-mon[36260]: pgmap v993: 168 pgs: 6 active+remapped+backfill_toofull, 162 active+clean; 1.1 GiB data, 9.6 GiB used, 706 GiB / 715 GiB avail; 993 B/s rd, 0 B/s wr, 708 op/s; 8181/56424 objects misplaced (14.499%); 5.6 MiB/s, 89 objects/s recovering 2024-09-20T19:09:21.079 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:09:21.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:21.122 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:21.123 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:09:21.223 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:09:21.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:20 smithi142 ceph-mon[32508]: pgmap v993: 168 pgs: 6 active+remapped+backfill_toofull, 162 active+clean; 1.1 GiB data, 9.6 GiB used, 706 GiB / 715 GiB avail; 993 B/s rd, 0 B/s wr, 708 op/s; 8181/56424 objects misplaced (14.499%); 5.6 MiB/s, 89 objects/s recovering 2024-09-20T19:09:21.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:21.270 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:21.270 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:09:21.302 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T19:09:21.325 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:09:21.370 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:09:21.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:21.407 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:21.407 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:09:21.508 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:09:21.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:21.546 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:21.546 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:09:21.647 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0 2024-09-20T19:09:21.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:21.703 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:21.703 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.0' 2024-09-20T19:09:21.741 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:09:21.805 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:09:21.840 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:21.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:21.842 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:09:21.925 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 7, 6, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 5, 6, 7, 4] 2024-09-20T19:09:21.925 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T19:09:21.926 INFO:tasks.thrashosds.thrasher:Killing osd 0, live_osds are [0, 1, 2, 3, 5, 6, 7, 4] 2024-09-20T19:09:21.926 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-20T19:09:21.926 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@osd.0 2024-09-20T19:09:21.943 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:09:22.009 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:22.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:22.010 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:09:22.111 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:09:22.125 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T19:09:22.155 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:22.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:22.157 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:09:22.159 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:21 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:09:22.160 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:21 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:09:22.160 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:21 smithi142 ceph-mon[32508]: osdmap e493: 8 total, 8 up, 8 in 2024-09-20T19:09:22.192 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:21 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:09:22.193 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:21 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:09:22.193 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:21 smithi083 ceph-mon[29421]: osdmap e493: 8 total, 8 up, 8 in 2024-09-20T19:09:22.193 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:21 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:09:22.193 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:21 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:09:22.193 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:21 smithi083 ceph-mon[36260]: osdmap e493: 8 total, 8 up, 8 in 2024-09-20T19:09:22.193 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:09:21 smithi083 systemd[1]: Stopping Ceph osd.0 for 1ed5a50a-7780-11ef-baf5-efdc52797490... 2024-09-20T19:09:22.258 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:09:22.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:22.296 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:22.296 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:09:22.301 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T19:09:22.302 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-09-20T19:09:22.302 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:22.302 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.26264.0:194659 11.1 11:843e0cc9:::benchmark_data_smithi083_177491_object91217:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e493)", 2024-09-20T19:09:22.302 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:09:22.289785+0000", 2024-09-20T19:09:22.302 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.0081999429999999995, 2024-09-20T19:09:22.302 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0082265689999999995, 2024-09-20T19:09:22.302 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:09:22.302 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:09:22.302 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "waiting for sub ops", 2024-09-20T19:09:22.302 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:09:22.302 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.26264", 2024-09-20T19:09:22.302 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2608780316", 2024-09-20T19:09:22.302 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 194659 2024-09-20T19:09:22.303 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:22.303 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:09:22.303 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:22.303 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:09:22.303 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:22.289785+0000", 2024-09-20T19:09:22.303 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:22.303 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:22.303 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:22.303 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:09:22.303 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:22.289785+0000", 2024-09-20T19:09:22.303 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:22.303 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:22.303 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:22.303 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:09:22.304 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:22.289786+0000", 2024-09-20T19:09:22.304 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.421e-06 2024-09-20T19:09:22.304 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:22.304 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:22.304 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:09:22.304 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:22.289791+0000", 2024-09-20T19:09:22.304 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.5170000000000001e-06 2024-09-20T19:09:22.304 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:22.304 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:22.304 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:09:22.304 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:22.289806+0000", 2024-09-20T19:09:22.304 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5798999999999999e-05 2024-09-20T19:09:22.304 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:22.304 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:22.305 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:09:22.305 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:22.289814+0000", 2024-09-20T19:09:22.305 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.6839999999999996e-06 2024-09-20T19:09:22.305 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:22.305 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:22.305 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:22.305 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:22.289867+0000", 2024-09-20T19:09:22.305 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.2655000000000003e-05 2024-09-20T19:09:22.305 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:22.305 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:22.305 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:09:22.305 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:22.290065+0000", 2024-09-20T19:09:22.305 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00019783100000000001 2024-09-20T19:09:22.305 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:22.305 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:22.305 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,3", 2024-09-20T19:09:22.306 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:22.291337+0000", 2024-09-20T19:09:22.306 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0012723400000000001 2024-09-20T19:09:22.306 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:22.306 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:22.306 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:09:22.306 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:22.293727+0000", 2024-09-20T19:09:22.306 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0023900079999999999 2024-09-20T19:09:22.306 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:22.306 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:09:22.306 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:22.306 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:22.306 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:22.306 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_repop(client.26264.0:194664 11.4 e493/441 11:20fe8e08:::benchmark_data_smithi083_177491_object91222:head v 493'7922, mlcod=493'7919)", 2024-09-20T19:09:22.307 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:09:22.297163+0000", 2024-09-20T19:09:22.307 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.00082132899999999998, 2024-09-20T19:09:22.307 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00092479900000000002, 2024-09-20T19:09:22.307 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:09:22.307 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:09:22.307 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-09-20T19:09:22.307 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:09:22.307 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:22.307 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:09:22.307 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:22.297163+0000", 2024-09-20T19:09:22.307 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:22.307 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:22.307 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:22.308 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:09:22.308 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:22.297163+0000", 2024-09-20T19:09:22.308 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:22.308 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:22.308 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:22.308 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:09:22.308 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:22.297164+0000", 2024-09-20T19:09:22.308 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.5300000000000002e-07 2024-09-20T19:09:22.308 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:22.308 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:22.308 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:09:22.308 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:22.297170+0000", 2024-09-20T19:09:22.309 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9159999999999998e-06 2024-09-20T19:09:22.309 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:22.309 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:22.309 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:09:22.309 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:22.297181+0000", 2024-09-20T19:09:22.309 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1123e-05 2024-09-20T19:09:22.309 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:22.309 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:22.309 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:09:22.309 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:22.297188+0000", 2024-09-20T19:09:22.309 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.9890000000000003e-06 2024-09-20T19:09:22.309 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:22.309 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:22.310 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:22.310 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:22.297259+0000", 2024-09-20T19:09:22.310 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.0837e-05 2024-09-20T19:09:22.310 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:22.310 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:22.310 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:09:22.310 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:22.297274+0000", 2024-09-20T19:09:22.310 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4822e-05 2024-09-20T19:09:22.310 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:22.310 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:09:22.310 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:22.310 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:22.310 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-09-20T19:09:22.310 INFO:teuthology.orchestra.run.smithi142.stdout: "num_ops": 2 2024-09-20T19:09:22.311 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T19:09:22.397 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:09:22.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:22.440 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:22.440 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:09:22.541 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:09:22.573 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:22.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:22.574 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:09:22.675 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:09:22.711 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:22.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:22.714 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:09:22.814 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:09:22.843 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:22.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:22.844 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:09:22.945 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:09:22.977 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:22.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:22.978 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:09:23.051 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:22 smithi142 ceph-mon[32508]: pgmap v995: 168 pgs: 6 active+remapped+backfill_toofull, 162 active+clean; 1.0 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 974 op/s; 7265/49836 objects misplaced (14.578%); 4.8 MiB/s, 76 objects/s recovering 2024-09-20T19:09:23.051 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:22 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:09:23.051 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:22 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:09:23.052 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_blocked_ops 2024-09-20T19:09:23.079 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:09:23.125 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:23.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:23.126 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:09:23.227 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:09:23.248 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:23.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:23.249 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:09:23.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:22 smithi083 ceph-mon[29421]: pgmap v995: 168 pgs: 6 active+remapped+backfill_toofull, 162 active+clean; 1.0 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 974 op/s; 7265/49836 objects misplaced (14.578%); 4.8 MiB/s, 76 objects/s recovering 2024-09-20T19:09:23.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:22 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:09:23.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:22 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:09:23.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:22 smithi083 ceph-mon[36260]: pgmap v995: 168 pgs: 6 active+remapped+backfill_toofull, 162 active+clean; 1.0 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 974 op/s; 7265/49836 objects misplaced (14.578%); 4.8 MiB/s, 76 objects/s recovering 2024-09-20T19:09:23.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:22 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:09:23.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:22 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:09:23.313 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:09:23.350 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:09:23.371 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:23.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:23.372 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:09:23.473 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:09:23.506 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:23.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:23.507 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:09:23.608 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:09:23.644 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:23.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:23.645 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:09:23.746 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:09:23.778 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:23.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:23.779 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:09:23.879 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:09:23.919 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:23.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:23.921 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:09:24.021 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:09:24.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:23 smithi083 ceph-mon[29421]: osdmap e494: 8 total, 8 up, 8 in 2024-09-20T19:09:24.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:23 smithi083 ceph-mon[29421]: pgmap v997: 168 pgs: 6 active+remapped+backfill_toofull, 162 active+clean; 799 MiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 933 B/s rd, 0 B/s wr, 1.15k op/s; 5591/38919 objects misplaced (14.366%) 2024-09-20T19:09:24.040 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:23 smithi083 ceph-mon[36260]: osdmap e494: 8 total, 8 up, 8 in 2024-09-20T19:09:24.040 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:23 smithi083 ceph-mon[36260]: pgmap v997: 168 pgs: 6 active+remapped+backfill_toofull, 162 active+clean; 799 MiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 933 B/s rd, 0 B/s wr, 1.15k op/s; 5591/38919 objects misplaced (14.366%) 2024-09-20T19:09:24.059 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:24.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:24.060 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:09:24.061 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T19:09:24.061 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-09-20T19:09:24.061 INFO:teuthology.orchestra.run.smithi142.stdout: "complaint_time": 30, 2024-09-20T19:09:24.061 INFO:teuthology.orchestra.run.smithi142.stdout: "num_blocked_ops": 0 2024-09-20T19:09:24.061 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T19:09:24.128 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0: {} 2024-09-20T19:09:24.128 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T19:09:24.161 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:09:24.171 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:09:24.172 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:09:24.186 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:24.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:24.187 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:09:24.208 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:09:24.208 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:09:24.246 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:09:24.247 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:09:24.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:23 smithi142 ceph-mon[32508]: osdmap e494: 8 total, 8 up, 8 in 2024-09-20T19:09:24.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:23 smithi142 ceph-mon[32508]: pgmap v997: 168 pgs: 6 active+remapped+backfill_toofull, 162 active+clean; 799 MiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 933 B/s rd, 0 B/s wr, 1.15k op/s; 5591/38919 objects misplaced (14.366%) 2024-09-20T19:09:24.284 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:09:24.284 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:09:24.288 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:09:24.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:24.317 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:24.317 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:09:24.325 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:09:24.325 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:09:24.356 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:09:24.356 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:09:24.391 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:09:24.392 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:09:24.418 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:09:24.419 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:09:24.461 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:24.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:24.462 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:09:24.563 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:09:24.596 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:24.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:24.597 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:09:24.697 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:09:24.723 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:24.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:24.724 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:09:24.790 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:09:24 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-0[39573]: 2024-09-20T19:09:24.389+0000 7f75c305a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-20T19:09:24.790 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:09:24 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-0[39573]: 2024-09-20T19:09:24.389+0000 7f75c305a640 -1 osd.0 494 *** Got signal Terminated *** 2024-09-20T19:09:24.790 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:09:24 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-0[39573]: 2024-09-20T19:09:24.389+0000 7f75c305a640 -1 osd.0 494 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-20T19:09:24.800 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_historic_ops 2024-09-20T19:09:24.825 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:09:24.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:24.875 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:24.876 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:09:24.976 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:09:25.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:25.016 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:25.017 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:09:25.117 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:09:25.175 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:09:25.195 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:25.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:25.196 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:09:25.233 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:25 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1305197677' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:09:25.233 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:25 smithi083 ceph-mon[29421]: osd.0 marked itself down and dead 2024-09-20T19:09:25.234 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:25 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1305197677' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:09:25.234 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:25 smithi083 ceph-mon[36260]: osd.0 marked itself down and dead 2024-09-20T19:09:25.297 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:09:25.321 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:25.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:25.322 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:09:25.423 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:09:25.445 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:25.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:25.446 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:09:25.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:25 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1305197677' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:09:25.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:25 smithi142 ceph-mon[32508]: osd.0 marked itself down and dead 2024-09-20T19:09:25.546 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:09:25.568 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:25.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:25.569 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:09:25.669 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:09:25.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:25.692 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:25.692 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:09:25.793 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:09:25.814 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:25.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:25.815 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:09:25.916 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:09:25.955 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:25.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:25.956 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:09:26.057 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:09:26.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:26.085 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:26.085 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:09:26.186 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:09:26.222 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:26.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:26.223 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:09:26.267 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:26 smithi083 ceph-mon[29421]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-20T19:09:26.267 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:26 smithi083 ceph-mon[29421]: osdmap e495: 8 total, 7 up, 8 in 2024-09-20T19:09:26.267 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:26 smithi083 ceph-mon[29421]: pgmap v999: 168 pgs: 3 stale+active+remapped+backfill_toofull, 20 stale+active+clean, 3 active+remapped+backfill_toofull, 142 active+clean; 799 MiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 973 op/s; 5591/38919 objects misplaced (14.366%) 2024-09-20T19:09:26.267 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:26 smithi083 ceph-mon[29421]: osdmap e496: 8 total, 7 up, 8 in 2024-09-20T19:09:26.268 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:09:25 smithi083 podman[193726]: 2024-09-20 19:09:25.937338568 +0000 UTC m=+1.735249608 container died d827380a84ebbbddee866a1e704b4f4e8695adacb684d9af079f0377c7d70544 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-0, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True) 2024-09-20T19:09:26.268 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:26 smithi083 ceph-mon[36260]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-20T19:09:26.268 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:26 smithi083 ceph-mon[36260]: osdmap e495: 8 total, 7 up, 8 in 2024-09-20T19:09:26.268 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:26 smithi083 ceph-mon[36260]: pgmap v999: 168 pgs: 3 stale+active+remapped+backfill_toofull, 20 stale+active+clean, 3 active+remapped+backfill_toofull, 142 active+clean; 799 MiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 973 op/s; 5591/38919 objects misplaced (14.366%) 2024-09-20T19:09:26.268 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:26 smithi083 ceph-mon[36260]: osdmap e496: 8 total, 7 up, 8 in 2024-09-20T19:09:26.323 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:09:26.359 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:26.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:26.360 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:09:26.459 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:26 smithi142 ceph-mon[32508]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-20T19:09:26.460 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:26 smithi142 ceph-mon[32508]: osdmap e495: 8 total, 7 up, 8 in 2024-09-20T19:09:26.460 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:26 smithi142 ceph-mon[32508]: pgmap v999: 168 pgs: 3 stale+active+remapped+backfill_toofull, 20 stale+active+clean, 3 active+remapped+backfill_toofull, 142 active+clean; 799 MiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 973 op/s; 5591/38919 objects misplaced (14.366%) 2024-09-20T19:09:26.460 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:26 smithi142 ceph-mon[32508]: osdmap e496: 8 total, 7 up, 8 in 2024-09-20T19:09:26.461 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:09:26.482 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:26.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:26.483 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:09:26.540 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:09:26 smithi083 podman[193726]: 2024-09-20 19:09:26.460033032 +0000 UTC m=+2.257944073 container cleanup d827380a84ebbbddee866a1e704b4f4e8695adacb684d9af079f0377c7d70544 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-20T19:09:26.540 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:09:26 smithi083 bash[193726]: ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-0 2024-09-20T19:09:26.584 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:09:26.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:26.620 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:26.621 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:09:26.722 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:09:26.755 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:26.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:26.756 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:09:26.857 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:09:26.900 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:26.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:26.902 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:09:27.002 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:09:27.027 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:09:26 smithi083 podman[193866]: 2024-09-20 19:09:26.770731917 +0000 UTC m=+1.011125876 container remove d827380a84ebbbddee866a1e704b4f4e8695adacb684d9af079f0377c7d70544 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-20T19:09:27.038 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:27.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:27.039 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:09:27.140 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:09:27.197 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:27.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:27.198 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:09:27.269 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T19:09:27.270 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:09:27.282 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:27 smithi142 ceph-mon[32508]: 11.19 continuing backfill to osd.3 from (435'2525,494'8106] MIN to 494'8106 2024-09-20T19:09:27.282 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:27 smithi142 ceph-mon[32508]: 11.19 continuing backfill to osd.6 from (435'2525,494'8106] MIN to 494'8106 2024-09-20T19:09:27.282 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:27 smithi142 ceph-mon[32508]: 11.11 continuing backfill to osd.6 from (432'1672,494'8129] MIN to 494'8129 2024-09-20T19:09:27.282 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:27 smithi142 ceph-mon[32508]: 11.13 continuing backfill to osd.1 from (436'2788,494'8231] MIN to 494'8231 2024-09-20T19:09:27.282 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:27 smithi142 ceph-mon[32508]: 11.10 continuing backfill to osd.1 from (435'2439,494'8071] MIN to 494'8071 2024-09-20T19:09:27.283 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:27 smithi142 ceph-mon[32508]: 11.13 continuing backfill to osd.6 from (436'2788,494'8231] MIN to 494'8231 2024-09-20T19:09:27.283 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:27 smithi142 ceph-mon[32508]: 11.10 continuing backfill to osd.3 from (435'2439,494'8071] MIN to 494'8071 2024-09-20T19:09:27.283 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:27 smithi142 ceph-mon[32508]: 11.13 continuing backfill to osd.7 from (436'2788,494'8231] MIN to 494'8231 2024-09-20T19:09:27.283 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:27 smithi142 ceph-mon[32508]: 11.10 continuing backfill to osd.7 from (435'2439,494'8071] MIN to 494'8071 2024-09-20T19:09:27.283 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:27 smithi142 ceph-mon[32508]: 11.16 continuing backfill to osd.1 from (445'3948,494'8196] MIN to 494'8196 2024-09-20T19:09:27.283 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:27 smithi142 ceph-mon[32508]: 11.16 starting backfill to osd.2 from (445'3948,494'8195] MIN to 494'8196 2024-09-20T19:09:27.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:27 smithi083 ceph-mon[29421]: 11.19 continuing backfill to osd.3 from (435'2525,494'8106] MIN to 494'8106 2024-09-20T19:09:27.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:27 smithi083 ceph-mon[29421]: 11.19 continuing backfill to osd.6 from (435'2525,494'8106] MIN to 494'8106 2024-09-20T19:09:27.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:27 smithi083 ceph-mon[29421]: 11.11 continuing backfill to osd.6 from (432'1672,494'8129] MIN to 494'8129 2024-09-20T19:09:27.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:27 smithi083 ceph-mon[29421]: 11.13 continuing backfill to osd.1 from (436'2788,494'8231] MIN to 494'8231 2024-09-20T19:09:27.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:27 smithi083 ceph-mon[29421]: 11.10 continuing backfill to osd.1 from (435'2439,494'8071] MIN to 494'8071 2024-09-20T19:09:27.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:27 smithi083 ceph-mon[29421]: 11.13 continuing backfill to osd.6 from (436'2788,494'8231] MIN to 494'8231 2024-09-20T19:09:27.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:27 smithi083 ceph-mon[29421]: 11.10 continuing backfill to osd.3 from (435'2439,494'8071] MIN to 494'8071 2024-09-20T19:09:27.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:27 smithi083 ceph-mon[29421]: 11.13 continuing backfill to osd.7 from (436'2788,494'8231] MIN to 494'8231 2024-09-20T19:09:27.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:27 smithi083 ceph-mon[29421]: 11.10 continuing backfill to osd.7 from (435'2439,494'8071] MIN to 494'8071 2024-09-20T19:09:27.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:27 smithi083 ceph-mon[29421]: 11.16 continuing backfill to osd.1 from (445'3948,494'8196] MIN to 494'8196 2024-09-20T19:09:27.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:27 smithi083 ceph-mon[29421]: 11.16 starting backfill to osd.2 from (445'3948,494'8195] MIN to 494'8196 2024-09-20T19:09:27.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:27 smithi083 ceph-mon[36260]: 11.19 continuing backfill to osd.3 from (435'2525,494'8106] MIN to 494'8106 2024-09-20T19:09:27.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:27 smithi083 ceph-mon[36260]: 11.19 continuing backfill to osd.6 from (435'2525,494'8106] MIN to 494'8106 2024-09-20T19:09:27.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:27 smithi083 ceph-mon[36260]: 11.11 continuing backfill to osd.6 from (432'1672,494'8129] MIN to 494'8129 2024-09-20T19:09:27.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:27 smithi083 ceph-mon[36260]: 11.13 continuing backfill to osd.1 from (436'2788,494'8231] MIN to 494'8231 2024-09-20T19:09:27.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:27 smithi083 ceph-mon[36260]: 11.10 continuing backfill to osd.1 from (435'2439,494'8071] MIN to 494'8071 2024-09-20T19:09:27.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:27 smithi083 ceph-mon[36260]: 11.13 continuing backfill to osd.6 from (436'2788,494'8231] MIN to 494'8231 2024-09-20T19:09:27.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:27 smithi083 ceph-mon[36260]: 11.10 continuing backfill to osd.3 from (435'2439,494'8071] MIN to 494'8071 2024-09-20T19:09:27.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:27 smithi083 ceph-mon[36260]: 11.13 continuing backfill to osd.7 from (436'2788,494'8231] MIN to 494'8231 2024-09-20T19:09:27.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:27 smithi083 ceph-mon[36260]: 11.10 continuing backfill to osd.7 from (435'2439,494'8071] MIN to 494'8071 2024-09-20T19:09:27.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:27 smithi083 ceph-mon[36260]: 11.16 continuing backfill to osd.1 from (445'3948,494'8196] MIN to 494'8196 2024-09-20T19:09:27.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:27 smithi083 ceph-mon[36260]: 11.16 starting backfill to osd.2 from (445'3948,494'8195] MIN to 494'8196 2024-09-20T19:09:27.299 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:09:27.332 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:27.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:27.333 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:09:27.434 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:09:27.469 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:27.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:27.470 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:09:27.571 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:09:27.610 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:27.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:27.611 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:09:27.712 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:09:27.745 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:27.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:27.746 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:09:27.847 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:09:27.880 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:27.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:27.881 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:09:27.889 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:09:27 smithi083 podman[194054]: 2024-09-20 19:09:27.607011761 +0000 UTC m=+0.034218373 image pull 441f9a3fe51528ef653b21f42e9d884db6124a0c7d482e865382bdfdb9ec979c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2 2024-09-20T19:09:27.889 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:09:27 smithi083 podman[194054]: 2024-09-20 19:09:27.734994442 +0000 UTC m=+0.162201048 container create ac8f3160459292303d475950287a7ff3dcf9e279984327b63f56c1816625f4e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.schema-version=1.0, ceph=True) 2024-09-20T19:09:27.956 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:09:27 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:09:27.983 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:09:27.983 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T19:09:28.015 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:28.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:28.016 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:09:28.046 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is set 2024-09-20T19:09:28.117 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:09:28.169 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:28.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:28.170 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:09:28.183 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:28 smithi083 ceph-mon[29421]: pgmap v1001: 168 pgs: 1 active+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_toofull, 1 active+remapped+backfill_wait, 18 active+undersized, 1 remapped+peering, 15 peering, 4 stale+active+clean, 15 active+undersized+degraded, 1 active+remapped+backfill_toofull, 110 active+clean; 578 MiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 682 B/s rd, 0 B/s wr, 1.20k op/s; 2501/28308 objects degraded (8.835%); 3025/28308 objects misplaced (10.686%); 992 KiB/s, 14 objects/s recovering 2024-09-20T19:09:28.183 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:28 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:09:28.183 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:28 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3673472647' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:09:28.184 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:28 smithi083 ceph-mon[36260]: pgmap v1001: 168 pgs: 1 active+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_toofull, 1 active+remapped+backfill_wait, 18 active+undersized, 1 remapped+peering, 15 peering, 4 stale+active+clean, 15 active+undersized+degraded, 1 active+remapped+backfill_toofull, 110 active+clean; 578 MiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 682 B/s rd, 0 B/s wr, 1.20k op/s; 2501/28308 objects degraded (8.835%); 3025/28308 objects misplaced (10.686%); 992 KiB/s, 14 objects/s recovering 2024-09-20T19:09:28.184 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:28 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:09:28.184 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:28 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3673472647' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:09:28.184 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:09:28 smithi083 podman[194054]: 2024-09-20 19:09:28.013476302 +0000 UTC m=+0.440682913 container init ac8f3160459292303d475950287a7ff3dcf9e279984327b63f56c1816625f4e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-20T19:09:28.184 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:09:28 smithi083 podman[194054]: 2024-09-20 19:09:28.019544816 +0000 UTC m=+0.446751421 container start ac8f3160459292303d475950287a7ff3dcf9e279984327b63f56c1816625f4e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-20T19:09:28.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:28 smithi142 ceph-mon[32508]: pgmap v1001: 168 pgs: 1 active+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_toofull, 1 active+remapped+backfill_wait, 18 active+undersized, 1 remapped+peering, 15 peering, 4 stale+active+clean, 15 active+undersized+degraded, 1 active+remapped+backfill_toofull, 110 active+clean; 578 MiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 682 B/s rd, 0 B/s wr, 1.20k op/s; 2501/28308 objects degraded (8.835%); 3025/28308 objects misplaced (10.686%); 992 KiB/s, 14 objects/s recovering 2024-09-20T19:09:28.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:28 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:09:28.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:28 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3673472647' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:09:28.271 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:09:28.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:28.303 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:28.303 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:09:28.404 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:09:28.418 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-20T19:09:28.443 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:28.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:28.444 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:09:28.540 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:09:28 smithi083 podman[194054]: 2024-09-20 19:09:28.180681551 +0000 UTC m=+0.607888159 container attach ac8f3160459292303d475950287a7ff3dcf9e279984327b63f56c1816625f4e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-20T19:09:28.545 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:09:28.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:28.583 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:28.583 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:09:28.684 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:09:28.716 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:28.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:28.717 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:09:28.818 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:09:28.852 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:28.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:28.853 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:09:28.858 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:09:28.923 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:09:28 smithi083 conmon[194098]: conmon ac8f3160459292303d47 : Failed to open cgroups file: /sys/fs/cgroup/machine.slice/libpod-ac8f3160459292303d475950287a7ff3dcf9e279984327b63f56c1816625f4e2.scope/container/memory.events 2024-09-20T19:09:28.954 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:09:28.989 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:28.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:28.991 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:09:29.091 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:09:29.131 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:29.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:29.132 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:09:29.233 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:09:29.235 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:29 smithi083 ceph-mon[29421]: Health check failed: Reduced data availability: 5 pgs inactive, 6 pgs peering (PG_AVAILABILITY) 2024-09-20T19:09:29.235 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:29 smithi083 ceph-mon[29421]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T19:09:29.235 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:29 smithi083 ceph-mon[29421]: Health check failed: Degraded data redundancy: 2501/28308 objects degraded (8.835%), 17 pgs degraded (PG_DEGRADED) 2024-09-20T19:09:29.235 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:29 smithi083 ceph-mon[29421]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:09:29.235 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:29 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3673472647' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T19:09:29.235 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:29 smithi083 ceph-mon[29421]: osdmap e497: 8 total, 7 up, 8 in 2024-09-20T19:09:29.235 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:29 smithi083 ceph-mon[36260]: Health check failed: Reduced data availability: 5 pgs inactive, 6 pgs peering (PG_AVAILABILITY) 2024-09-20T19:09:29.235 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:29 smithi083 ceph-mon[36260]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T19:09:29.235 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:29 smithi083 ceph-mon[36260]: Health check failed: Degraded data redundancy: 2501/28308 objects degraded (8.835%), 17 pgs degraded (PG_DEGRADED) 2024-09-20T19:09:29.235 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:29 smithi083 ceph-mon[36260]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:09:29.236 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:29 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3673472647' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T19:09:29.236 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:29 smithi083 ceph-mon[36260]: osdmap e497: 8 total, 7 up, 8 in 2024-09-20T19:09:29.236 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:09:28 smithi083 podman[194054]: 2024-09-20 19:09:28.982994005 +0000 UTC m=+1.410200618 container died ac8f3160459292303d475950287a7ff3dcf9e279984327b63f56c1816625f4e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=) 2024-09-20T19:09:29.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:29 smithi142 ceph-mon[32508]: Health check failed: Reduced data availability: 5 pgs inactive, 6 pgs peering (PG_AVAILABILITY) 2024-09-20T19:09:29.265 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:29 smithi142 ceph-mon[32508]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T19:09:29.266 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:29 smithi142 ceph-mon[32508]: Health check failed: Degraded data redundancy: 2501/28308 objects degraded (8.835%), 17 pgs degraded (PG_DEGRADED) 2024-09-20T19:09:29.266 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:29 smithi142 ceph-mon[32508]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:09:29.266 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:29 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3673472647' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T19:09:29.266 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:29 smithi142 ceph-mon[32508]: osdmap e497: 8 total, 7 up, 8 in 2024-09-20T19:09:29.268 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:29.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:29.269 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:09:29.370 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:09:29.409 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:29.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:29.410 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:09:29.511 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:09:29.550 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:29.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:29.551 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:09:29.652 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:09:29.671 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T19:09:29.671 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:09:29.688 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:29.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:29.689 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:09:29.722 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@osd.0.service' 2024-09-20T19:09:29.790 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:09:29.795 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-20T19:09:29.796 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-20T19:09:29.796 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.0 2024-09-20T19:09:29.796 INFO:tasks.thrashosds.thrasher:remote for osd 0 is ubuntu@smithi083.front.sepia.ceph.com 2024-09-20T19:09:29.796 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell -n osd.0 --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --op list-pgs 2024-09-20T19:09:29.801 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:09:29.821 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:29.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:29.822 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:09:29.923 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:09:29.954 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:29.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:29.955 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:09:30.056 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:09:30.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:30.097 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:30.097 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:09:30.198 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:09:30.232 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:30.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:30.233 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:09:30.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:30 smithi083 ceph-mon[29421]: pgmap v1003: 168 pgs: 1 active+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_toofull, 1 active+remapped+backfill_wait, 23 active+undersized, 1 remapped+peering, 15 peering, 18 active+undersized+degraded, 1 active+remapped+backfill_toofull, 106 active+clean; 555 MiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 0 B/s wr, 650 op/s; 2511/27198 objects degraded (9.232%); 3025/27198 objects misplaced (11.122%); 992 KiB/s, 12 objects/s recovering 2024-09-20T19:09:30.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:30 smithi083 ceph-mon[36260]: pgmap v1003: 168 pgs: 1 active+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_toofull, 1 active+remapped+backfill_wait, 23 active+undersized, 1 remapped+peering, 15 peering, 18 active+undersized+degraded, 1 active+remapped+backfill_toofull, 106 active+clean; 555 MiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 0 B/s wr, 650 op/s; 2511/27198 objects degraded (9.232%); 3025/27198 objects misplaced (11.122%); 992 KiB/s, 12 objects/s recovering 2024-09-20T19:09:30.333 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:09:30.361 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/osd.0/config 2024-09-20T19:09:30.362 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:30.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:30.366 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:09:30.428 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:30 smithi142 ceph-mon[32508]: pgmap v1003: 168 pgs: 1 active+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_toofull, 1 active+remapped+backfill_wait, 23 active+undersized, 1 remapped+peering, 15 peering, 18 active+undersized+degraded, 1 active+remapped+backfill_toofull, 106 active+clean; 555 MiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 0 B/s wr, 650 op/s; 2511/27198 objects degraded (9.232%); 3025/27198 objects misplaced (11.122%); 992 KiB/s, 12 objects/s recovering 2024-09-20T19:09:30.465 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_blocked_ops 2024-09-20T19:09:30.467 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:09:30.494 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:30.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:30.496 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:09:30.597 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:09:30.618 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:30.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:30.619 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:09:30.720 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:09:30.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:30.742 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:30.742 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:09:30.746 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:09:30.843 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:09:30.864 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:30.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:30.865 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:09:30.966 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:09:30.976 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T19:09:30.991 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:30.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:30.992 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:09:31.012 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:09:30 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:09:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:09:31.093 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:09:31.122 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:31.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:31.123 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:09:31.224 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:09:31.258 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:31.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:31.259 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:09:31.259 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:31 smithi142 ceph-mon[32508]: osdmap e498: 8 total, 7 up, 8 in 2024-09-20T19:09:31.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:31 smithi083 ceph-mon[29421]: osdmap e498: 8 total, 7 up, 8 in 2024-09-20T19:09:31.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:31 smithi083 ceph-mon[36260]: osdmap e498: 8 total, 7 up, 8 in 2024-09-20T19:09:31.360 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:09:31.398 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:31.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:31.399 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:09:31.500 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:09:31.538 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:31.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:31.539 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:09:31.640 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:09:31.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:31.675 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:31.675 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:09:31.776 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:09:31.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:31.819 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:31.819 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:09:31.920 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:09:31.931 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T19:09:31.932 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-09-20T19:09:31.932 INFO:teuthology.orchestra.run.smithi142.stdout: "complaint_time": 30, 2024-09-20T19:09:31.932 INFO:teuthology.orchestra.run.smithi142.stdout: "num_blocked_ops": 0 2024-09-20T19:09:31.932 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T19:09:31.942 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:31.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:31.943 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:09:32.044 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:09:32.065 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:32.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:32.066 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:09:32.167 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:09:32.189 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:32.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:32.190 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:09:32.291 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:09:32.328 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:32.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:32.329 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:09:32.430 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:09:32.442 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:32 smithi142 ceph-mon[32508]: osdmap e499: 8 total, 7 up, 8 in 2024-09-20T19:09:32.442 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:32 smithi142 ceph-mon[32508]: pgmap v1006: 168 pgs: 1 active+remapped+backfilling, 3 active+undersized+degraded+remapped+backfill_toofull, 34 active+undersized, 2 active+remapped+backfill_toofull, 22 active+undersized+degraded, 106 active+clean; 412 MiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 0 B/s wr, 1.07k op/s; 2057/20355 objects degraded (10.106%); 2280/20355 objects misplaced (11.201%); 1.1 MiB/s, 14 objects/s recovering 2024-09-20T19:09:32.442 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:32 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1910888977' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:09:32.442 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:32 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:09:32.442 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:32 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:09:32.464 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:32.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:32.465 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:09:32.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:32 smithi083 ceph-mon[29421]: osdmap e499: 8 total, 7 up, 8 in 2024-09-20T19:09:32.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:32 smithi083 ceph-mon[29421]: pgmap v1006: 168 pgs: 1 active+remapped+backfilling, 3 active+undersized+degraded+remapped+backfill_toofull, 34 active+undersized, 2 active+remapped+backfill_toofull, 22 active+undersized+degraded, 106 active+clean; 412 MiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 0 B/s wr, 1.07k op/s; 2057/20355 objects degraded (10.106%); 2280/20355 objects misplaced (11.201%); 1.1 MiB/s, 14 objects/s recovering 2024-09-20T19:09:32.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:32 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1910888977' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:09:32.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:32 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:09:32.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:32 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:09:32.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:32 smithi083 ceph-mon[36260]: osdmap e499: 8 total, 7 up, 8 in 2024-09-20T19:09:32.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:32 smithi083 ceph-mon[36260]: pgmap v1006: 168 pgs: 1 active+remapped+backfilling, 3 active+undersized+degraded+remapped+backfill_toofull, 34 active+undersized, 2 active+remapped+backfill_toofull, 22 active+undersized+degraded, 106 active+clean; 412 MiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 0 B/s wr, 1.07k op/s; 2057/20355 objects degraded (10.106%); 2280/20355 objects misplaced (11.201%); 1.1 MiB/s, 14 objects/s recovering 2024-09-20T19:09:32.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:32 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1910888977' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:09:32.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:32 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:09:32.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:32 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:09:32.565 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:09:32.593 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:32.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:32.594 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:09:32.639 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_historic_ops 2024-09-20T19:09:32.694 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:09:32.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:32.730 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:32.730 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:09:32.787 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:09:32.787 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:09:32.787 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:09:32.787 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:09:32.787 INFO:teuthology.orchestra.run.smithi083.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-20T19:09:32.787 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:09:32.788 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:09:32.788 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:09:32.788 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:09:32.788 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:09:32.788 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:09:32.788 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:09:32.788 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:09:32.789 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:09:32.789 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:09:32.831 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:09:32.880 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:32.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:32.881 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:09:32.982 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:09:33.021 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:33.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:33.022 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:09:33.058 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:09:33.124 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:09:33.163 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:33.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:33.163 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:09:33.264 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:09:33.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:33.294 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:33.294 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:09:33.395 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:09:33.419 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:33.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:33.420 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:09:33.507 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:33 smithi142 ceph-mon[32508]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 5 pgs inactive, 6 pgs peering) 2024-09-20T19:09:33.521 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:09:33.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:33 smithi083 ceph-mon[29421]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 5 pgs inactive, 6 pgs peering) 2024-09-20T19:09:33.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:33 smithi083 ceph-mon[36260]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 5 pgs inactive, 6 pgs peering) 2024-09-20T19:09:33.555 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:33.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:33.556 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:09:33.657 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:09:33.690 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:33.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:33.691 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:09:33.792 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:09:33.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:33.837 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:33.837 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:09:33.937 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:09:33.977 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:33.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:33.978 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:09:33.981 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-20T19:09:33.981 INFO:teuthology.orchestra.run.smithi142.stdout: "size": 20, 2024-09-20T19:09:33.981 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 600, 2024-09-20T19:09:33.981 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-09-20T19:09:33.981 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:33.981 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:205605 10.1 10:85efc907:::benchmark_data_smithi083_148100_object88743:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e402)", 2024-09-20T19:09:33.981 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:32.531622+0000", 2024-09-20T19:09:33.981 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 241.44438384200001, 2024-09-20T19:09:33.981 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.047547566, 2024-09-20T19:09:33.982 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:09:33.982 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:09:33.982 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-09-20T19:09:33.982 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:09:33.982 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:09:33.982 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:09:33.982 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 205605 2024-09-20T19:09:33.982 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:33.982 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:09:33.982 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:33.982 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:09:33.982 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.531622+0000", 2024-09-20T19:09:33.982 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:33.982 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:33.983 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:33.983 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:09:33.983 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.531622+0000", 2024-09-20T19:09:33.983 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:33.983 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:33.983 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:33.983 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:09:33.983 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.531631+0000", 2024-09-20T19:09:33.983 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.6989999999999992e-06 2024-09-20T19:09:33.983 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:33.983 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:33.983 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:09:33.984 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.531637+0000", 2024-09-20T19:09:33.984 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6550000000000001e-06 2024-09-20T19:09:33.984 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:33.984 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:33.984 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:09:33.984 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.531649+0000", 2024-09-20T19:09:33.984 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2621e-05 2024-09-20T19:09:33.984 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:33.984 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:33.984 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:09:33.984 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.531658+0000", 2024-09-20T19:09:33.984 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.3809999999999999e-06 2024-09-20T19:09:33.985 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:33.985 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:33.985 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:33.985 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.531749+0000", 2024-09-20T19:09:33.985 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.0933000000000005e-05 2024-09-20T19:09:33.985 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:33.985 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:33.985 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:09:33.985 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.531757+0000", 2024-09-20T19:09:33.985 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.2239999999999995e-06 2024-09-20T19:09:33.985 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:33.985 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:33.985 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:33.985 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.568872+0000", 2024-09-20T19:09:33.986 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.037115467 2024-09-20T19:09:33.986 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:33.986 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:33.986 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for missing object", 2024-09-20T19:09:33.986 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.568963+0000", 2024-09-20T19:09:33.986 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.0613999999999997e-05 2024-09-20T19:09:33.986 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:33.986 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:33.986 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:33.986 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.579116+0000", 2024-09-20T19:09:33.986 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.010153387 2024-09-20T19:09:33.986 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:33.987 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:33.987 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:09:33.987 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.579170+0000", 2024-09-20T19:09:33.987 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3585000000000003e-05 2024-09-20T19:09:33.987 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:33.987 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:09:33.987 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:33.987 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:33.987 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:33.987 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:205614 10.1 10:8d1cab49:::benchmark_data_smithi083_148100_object88752:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e402)", 2024-09-20T19:09:33.987 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:32.536063+0000", 2024-09-20T19:09:33.987 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 241.43994323999999, 2024-09-20T19:09:33.987 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0492143700000001, 2024-09-20T19:09:33.988 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:09:33.988 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:09:33.988 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:09:33.988 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:09:33.988 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:09:33.988 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:09:33.988 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 205614 2024-09-20T19:09:33.988 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:33.988 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:09:33.988 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:33.988 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:09:33.988 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.536063+0000", 2024-09-20T19:09:33.988 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:33.988 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:33.988 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:33.989 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:09:33.989 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.536063+0000", 2024-09-20T19:09:33.989 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:33.989 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:33.989 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:33.989 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:09:33.989 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.536065+0000", 2024-09-20T19:09:33.989 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6649999999999999e-06 2024-09-20T19:09:33.989 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:33.989 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:33.989 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:09:33.989 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.536071+0000", 2024-09-20T19:09:33.989 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4799999999999998e-06 2024-09-20T19:09:33.990 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:33.990 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:33.990 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:09:33.990 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.536088+0000", 2024-09-20T19:09:33.990 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7070000000000001e-05 2024-09-20T19:09:33.990 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:33.990 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:33.990 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:09:33.990 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.536097+0000", 2024-09-20T19:09:33.990 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.4109999999999996e-06 2024-09-20T19:09:33.990 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:33.990 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:33.990 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:33.990 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.536146+0000", 2024-09-20T19:09:33.991 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.9475999999999999e-05 2024-09-20T19:09:33.991 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:33.991 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:33.991 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:09:33.991 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.536152+0000", 2024-09-20T19:09:33.991 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.105e-06 2024-09-20T19:09:33.991 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:33.991 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:33.991 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:33.991 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.569308+0000", 2024-09-20T19:09:33.991 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0331554300000001 2024-09-20T19:09:33.991 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:33.991 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:33.991 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:09:33.991 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.569399+0000", 2024-09-20T19:09:33.992 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.0871999999999993e-05 2024-09-20T19:09:33.992 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:33.992 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:33.992 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,7", 2024-09-20T19:09:33.992 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.569821+0000", 2024-09-20T19:09:33.992 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00042259099999999999 2024-09-20T19:09:33.992 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:33.992 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:33.992 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:09:33.992 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.576110+0000", 2024-09-20T19:09:33.992 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0062885049999999998 2024-09-20T19:09:33.992 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:33.992 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:33.992 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:33.992 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.576171+0000", 2024-09-20T19:09:33.993 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0900999999999998e-05 2024-09-20T19:09:33.993 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:33.993 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:33.993 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:33.993 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.585141+0000", 2024-09-20T19:09:33.993 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0089703590000000007 2024-09-20T19:09:33.993 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:33.993 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:33.993 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:09:33.993 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.585192+0000", 2024-09-20T19:09:33.993 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0982000000000001e-05 2024-09-20T19:09:33.993 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:33.993 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:33.993 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:09:33.994 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.585277+0000", 2024-09-20T19:09:33.994 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.5523000000000001e-05 2024-09-20T19:09:33.994 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:33.994 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:09:33.994 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:33.994 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:33.994 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:33.994 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:205625 10.1 10:882102ec:::benchmark_data_smithi083_148100_object88763:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e402)", 2024-09-20T19:09:33.994 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:32.551241+0000", 2024-09-20T19:09:33.994 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 241.42476560899999, 2024-09-20T19:09:33.994 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0342664909999999, 2024-09-20T19:09:33.994 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:09:33.994 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:09:33.995 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:09:33.995 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:09:33.995 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:09:33.995 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:09:33.995 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 205625 2024-09-20T19:09:33.995 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:33.995 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:09:33.995 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:33.995 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:09:33.995 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.551241+0000", 2024-09-20T19:09:33.995 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:33.995 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:33.995 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:33.995 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:09:33.995 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.551241+0000", 2024-09-20T19:09:33.996 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:33.996 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:33.996 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:33.996 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:09:33.996 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.551242+0000", 2024-09-20T19:09:33.996 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1000000000000001e-06 2024-09-20T19:09:33.996 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:33.996 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:33.996 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:09:33.996 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.551246+0000", 2024-09-20T19:09:33.996 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9740000000000004e-06 2024-09-20T19:09:33.996 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:33.996 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:33.996 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:09:33.996 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.551255+0000", 2024-09-20T19:09:33.997 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.1050000000000001e-06 2024-09-20T19:09:33.997 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:33.997 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:33.997 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:09:33.997 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.551263+0000", 2024-09-20T19:09:33.997 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.6159999999999996e-06 2024-09-20T19:09:33.997 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:33.997 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:33.997 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:33.997 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.551320+0000", 2024-09-20T19:09:33.997 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.7170999999999999e-05 2024-09-20T19:09:33.997 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:33.997 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:33.997 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:09:33.998 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.551328+0000", 2024-09-20T19:09:33.998 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.67e-06 2024-09-20T19:09:33.998 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:33.998 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:33.998 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:33.998 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.576215+0000", 2024-09-20T19:09:33.998 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.024886865 2024-09-20T19:09:33.998 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:33.998 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:33.998 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:09:33.998 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.576302+0000", 2024-09-20T19:09:33.998 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.6860999999999994e-05 2024-09-20T19:09:33.998 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:33.998 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:33.999 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,7", 2024-09-20T19:09:33.999 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.576386+0000", 2024-09-20T19:09:33.999 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.3749000000000001e-05 2024-09-20T19:09:33.999 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:33.999 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:33.999 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:33.999 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.579206+0000", 2024-09-20T19:09:33.999 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0028204110000000001 2024-09-20T19:09:33.999 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:33.999 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:33.999 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:09:33.999 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.579216+0000", 2024-09-20T19:09:33.999 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0241000000000001e-05 2024-09-20T19:09:33.999 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.000 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.000 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:34.000 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.585353+0000", 2024-09-20T19:09:34.000 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0061364130000000003 2024-09-20T19:09:34.000 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.000 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.000 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:09:34.000 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.585407+0000", 2024-09-20T19:09:34.000 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3899e-05 2024-09-20T19:09:34.000 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.000 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.000 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:09:34.000 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.585507+0000", 2024-09-20T19:09:34.001 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000100416 2024-09-20T19:09:34.001 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:34.001 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:09:34.001 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:34.001 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.001 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.001 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:205665 10.1 10:81984290:::benchmark_data_smithi083_148100_object88803:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e402)", 2024-09-20T19:09:34.001 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:32.597237+0000", 2024-09-20T19:09:34.001 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 241.378769533, 2024-09-20T19:09:34.001 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.98843897400000003, 2024-09-20T19:09:34.001 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:09:34.001 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:09:34.001 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:09:34.001 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:09:34.001 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:09:34.002 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:09:34.002 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 205665 2024-09-20T19:09:34.002 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.002 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:09:34.002 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.002 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:09:34.002 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.597237+0000", 2024-09-20T19:09:34.002 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:34.002 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.002 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.002 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:09:34.002 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.597237+0000", 2024-09-20T19:09:34.002 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:34.003 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.003 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.003 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:09:34.003 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.597238+0000", 2024-09-20T19:09:34.003 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.44e-06 2024-09-20T19:09:34.003 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.003 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.003 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:09:34.003 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.597243+0000", 2024-09-20T19:09:34.003 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.5709999999999999e-06 2024-09-20T19:09:34.003 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.003 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.003 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:09:34.003 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.597255+0000", 2024-09-20T19:09:34.003 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.222e-05 2024-09-20T19:09:34.004 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.004 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.004 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:09:34.004 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.597262+0000", 2024-09-20T19:09:34.004 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.7349999999999999e-06 2024-09-20T19:09:34.004 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.004 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.004 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:34.004 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.597324+0000", 2024-09-20T19:09:34.004 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2370000000000001e-05 2024-09-20T19:09:34.004 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.004 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.004 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:09:34.004 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.597333+0000", 2024-09-20T19:09:34.005 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.9199999999999993e-06 2024-09-20T19:09:34.005 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.005 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.005 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:34.005 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.577429+0000", 2024-09-20T19:09:34.005 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.98009612899999998 2024-09-20T19:09:34.005 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.005 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.005 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:09:34.005 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.577505+0000", 2024-09-20T19:09:34.005 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.6052999999999998e-05 2024-09-20T19:09:34.005 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.005 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.005 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,7", 2024-09-20T19:09:34.005 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.577606+0000", 2024-09-20T19:09:34.006 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00010052699999999999 2024-09-20T19:09:34.006 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.006 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.006 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:09:34.006 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.579224+0000", 2024-09-20T19:09:34.006 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001617889 2024-09-20T19:09:34.006 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.006 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.006 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:34.006 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.579255+0000", 2024-09-20T19:09:34.006 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.1303000000000002e-05 2024-09-20T19:09:34.006 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.006 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.006 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:34.007 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.585554+0000", 2024-09-20T19:09:34.007 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0062987700000000004 2024-09-20T19:09:34.007 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.007 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.007 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:09:34.007 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.585600+0000", 2024-09-20T19:09:34.007 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.6272e-05 2024-09-20T19:09:34.007 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.007 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.007 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:09:34.007 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.585676+0000", 2024-09-20T19:09:34.007 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5774999999999999e-05 2024-09-20T19:09:34.008 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:34.008 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:09:34.008 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:34.008 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.008 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.008 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:205673 10.1 10:842f84a5:::benchmark_data_smithi083_148100_object88811:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e402)", 2024-09-20T19:09:34.008 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:32.607094+0000", 2024-09-20T19:09:34.008 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 241.368912037, 2024-09-20T19:09:34.008 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.97897194300000001, 2024-09-20T19:09:34.008 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:09:34.008 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:09:34.008 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:09:34.009 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:09:34.009 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:09:34.009 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:09:34.009 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 205673 2024-09-20T19:09:34.009 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.009 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:09:34.009 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.009 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:09:34.010 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.607094+0000", 2024-09-20T19:09:34.010 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:34.010 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.010 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.010 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:09:34.010 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.607094+0000", 2024-09-20T19:09:34.010 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:34.010 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.010 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.010 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:09:34.010 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.607097+0000", 2024-09-20T19:09:34.011 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2409999999999998e-06 2024-09-20T19:09:34.011 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.011 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.011 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:09:34.011 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.607103+0000", 2024-09-20T19:09:34.011 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5509999999999996e-06 2024-09-20T19:09:34.011 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.011 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.011 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:09:34.011 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.607117+0000", 2024-09-20T19:09:34.011 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4158999999999999e-05 2024-09-20T19:09:34.011 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.011 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.011 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:09:34.012 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.607126+0000", 2024-09-20T19:09:34.012 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.833e-06 2024-09-20T19:09:34.012 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.012 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.012 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:34.012 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.607189+0000", 2024-09-20T19:09:34.012 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2844999999999999e-05 2024-09-20T19:09:34.012 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.012 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.012 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:09:34.012 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.607198+0000", 2024-09-20T19:09:34.012 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.4970000000000007e-06 2024-09-20T19:09:34.012 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.012 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.013 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:34.013 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.579280+0000", 2024-09-20T19:09:34.013 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.97208206799999997 2024-09-20T19:09:34.013 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.013 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.013 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:09:34.013 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.579345+0000", 2024-09-20T19:09:34.013 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4623999999999997e-05 2024-09-20T19:09:34.013 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.013 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.013 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,7", 2024-09-20T19:09:34.013 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.579415+0000", 2024-09-20T19:09:34.013 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.9593999999999998e-05 2024-09-20T19:09:34.013 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.013 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.014 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:34.014 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.585009+0000", 2024-09-20T19:09:34.014 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0055947790000000002 2024-09-20T19:09:34.014 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.014 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.014 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:09:34.014 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.585020+0000", 2024-09-20T19:09:34.014 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.03e-05 2024-09-20T19:09:34.014 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.014 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.014 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:34.014 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.585934+0000", 2024-09-20T19:09:34.014 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00091444499999999995 2024-09-20T19:09:34.014 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.015 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.015 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:09:34.015 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.585975+0000", 2024-09-20T19:09:34.015 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0535999999999999e-05 2024-09-20T19:09:34.015 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.015 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.015 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:09:34.015 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.586066+0000", 2024-09-20T19:09:34.015 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.1471000000000006e-05 2024-09-20T19:09:34.015 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:34.015 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:09:34.015 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:34.015 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.015 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.016 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:205677 10.1 10:8a3f436e:::benchmark_data_smithi083_148100_object88815:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e402)", 2024-09-20T19:09:34.016 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:32.607945+0000", 2024-09-20T19:09:34.016 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 241.36806151499999, 2024-09-20T19:09:34.016 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.98557370600000005, 2024-09-20T19:09:34.016 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:09:34.016 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:09:34.016 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:09:34.016 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:09:34.016 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:09:34.016 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:09:34.016 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 205677 2024-09-20T19:09:34.016 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.016 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:09:34.016 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.017 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:09:34.017 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.607945+0000", 2024-09-20T19:09:34.017 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:34.017 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.017 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.017 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:09:34.017 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.607945+0000", 2024-09-20T19:09:34.017 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:34.017 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.017 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.017 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:09:34.017 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.607946+0000", 2024-09-20T19:09:34.017 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3799999999999999e-06 2024-09-20T19:09:34.017 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.017 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.018 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:09:34.018 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.607951+0000", 2024-09-20T19:09:34.018 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.2619999999999998e-06 2024-09-20T19:09:34.018 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.018 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.018 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:09:34.018 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.607967+0000", 2024-09-20T19:09:34.018 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5189e-05 2024-09-20T19:09:34.018 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.018 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.018 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:09:34.018 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.607974+0000", 2024-09-20T19:09:34.018 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.6930000000000007e-06 2024-09-20T19:09:34.018 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.019 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.019 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:34.019 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.608043+0000", 2024-09-20T19:09:34.019 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.9105999999999999e-05 2024-09-20T19:09:34.019 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.019 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.019 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:09:34.019 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.608057+0000", 2024-09-20T19:09:34.019 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3971999999999999e-05 2024-09-20T19:09:34.019 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.019 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.019 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:34.019 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.580677+0000", 2024-09-20T19:09:34.019 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.97261913700000002 2024-09-20T19:09:34.020 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.020 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.020 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:09:34.020 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.580813+0000", 2024-09-20T19:09:34.020 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000136644 2024-09-20T19:09:34.020 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.020 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.020 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,7", 2024-09-20T19:09:34.020 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.580942+0000", 2024-09-20T19:09:34.020 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000128818 2024-09-20T19:09:34.020 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.020 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.020 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:34.020 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.585081+0000", 2024-09-20T19:09:34.021 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0041393000000000003 2024-09-20T19:09:34.021 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.021 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.021 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:09:34.021 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.585295+0000", 2024-09-20T19:09:34.021 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000213604 2024-09-20T19:09:34.021 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.021 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.021 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:34.021 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.593134+0000", 2024-09-20T19:09:34.021 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0078393830000000001 2024-09-20T19:09:34.021 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.021 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.021 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:09:34.021 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.593410+0000", 2024-09-20T19:09:34.022 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00027596899999999999 2024-09-20T19:09:34.022 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.022 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.022 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:09:34.022 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.593519+0000", 2024-09-20T19:09:34.022 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000108249 2024-09-20T19:09:34.022 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:34.022 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:09:34.022 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:34.022 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.022 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.022 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:205680 10.1 10:88a7c9ce:::benchmark_data_smithi083_148100_object88818:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e402)", 2024-09-20T19:09:34.022 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:32.612025+0000", 2024-09-20T19:09:34.023 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 241.36398091699999, 2024-09-20T19:09:34.023 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.98176313299999995, 2024-09-20T19:09:34.023 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:09:34.023 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:09:34.023 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:09:34.023 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:09:34.023 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:09:34.023 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:09:34.023 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 205680 2024-09-20T19:09:34.023 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.023 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:09:34.023 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.023 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:09:34.023 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.612025+0000", 2024-09-20T19:09:34.024 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:34.024 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.024 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.024 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:09:34.024 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.612025+0000", 2024-09-20T19:09:34.024 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:34.024 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.024 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.024 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:09:34.024 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.612027+0000", 2024-09-20T19:09:34.024 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7489999999999999e-06 2024-09-20T19:09:34.024 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.024 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.024 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:09:34.024 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.612040+0000", 2024-09-20T19:09:34.025 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3032e-05 2024-09-20T19:09:34.025 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.025 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.025 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:09:34.025 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.612056+0000", 2024-09-20T19:09:34.025 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5628999999999999e-05 2024-09-20T19:09:34.025 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.025 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.025 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:09:34.025 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.612062+0000", 2024-09-20T19:09:34.025 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4389999999999997e-06 2024-09-20T19:09:34.025 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.025 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.025 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:34.025 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.612139+0000", 2024-09-20T19:09:34.026 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.6618000000000004e-05 2024-09-20T19:09:34.026 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.026 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.026 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:09:34.026 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.612148+0000", 2024-09-20T19:09:34.026 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.5359999999999998e-06 2024-09-20T19:09:34.026 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.026 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.026 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:34.026 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.582881+0000", 2024-09-20T19:09:34.026 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.97073284800000004 2024-09-20T19:09:34.026 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.026 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.026 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:09:34.027 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.582977+0000", 2024-09-20T19:09:34.027 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.6219999999999997e-05 2024-09-20T19:09:34.027 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.027 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.027 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,7", 2024-09-20T19:09:34.027 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.583134+0000", 2024-09-20T19:09:34.027 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015651 2024-09-20T19:09:34.027 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.027 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.027 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:09:34.027 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.585688+0000", 2024-09-20T19:09:34.027 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0025544280000000001 2024-09-20T19:09:34.027 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.027 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.027 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:34.028 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.585751+0000", 2024-09-20T19:09:34.028 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2176999999999998e-05 2024-09-20T19:09:34.028 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.028 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.028 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:34.028 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.593629+0000", 2024-09-20T19:09:34.028 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0078781679999999996 2024-09-20T19:09:34.028 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.028 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.028 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:09:34.028 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.593693+0000", 2024-09-20T19:09:34.028 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.3904000000000003e-05 2024-09-20T19:09:34.028 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.028 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.028 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:09:34.029 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.593789+0000", 2024-09-20T19:09:34.029 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.5874999999999999e-05 2024-09-20T19:09:34.029 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:34.029 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:09:34.029 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:34.029 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.029 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.029 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:205685 10.1 10:81582fda:::benchmark_data_smithi083_148100_object88823:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e402)", 2024-09-20T19:09:34.029 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:32.615641+0000", 2024-09-20T19:09:34.029 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 241.360365823, 2024-09-20T19:09:34.029 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.97834197300000003, 2024-09-20T19:09:34.029 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:09:34.029 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:09:34.029 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:09:34.030 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:09:34.030 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:09:34.030 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:09:34.030 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 205685 2024-09-20T19:09:34.030 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.030 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:09:34.030 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.030 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:09:34.030 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.615641+0000", 2024-09-20T19:09:34.030 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:34.030 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.030 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.030 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:09:34.030 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.615641+0000", 2024-09-20T19:09:34.031 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:34.031 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.031 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.031 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:09:34.031 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.615642+0000", 2024-09-20T19:09:34.031 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.328e-06 2024-09-20T19:09:34.031 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.031 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.031 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:09:34.031 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.615646+0000", 2024-09-20T19:09:34.031 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.2209999999999999e-06 2024-09-20T19:09:34.031 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.031 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.032 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:09:34.032 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.615657+0000", 2024-09-20T19:09:34.032 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0868e-05 2024-09-20T19:09:34.032 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.032 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.032 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:09:34.032 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.615662+0000", 2024-09-20T19:09:34.032 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.1200000000000001e-06 2024-09-20T19:09:34.032 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.032 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.032 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:34.032 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.615717+0000", 2024-09-20T19:09:34.032 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4642999999999998e-05 2024-09-20T19:09:34.033 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.033 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.033 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:09:34.033 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.615729+0000", 2024-09-20T19:09:34.033 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.189e-05 2024-09-20T19:09:34.033 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.033 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.033 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:34.033 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.586114+0000", 2024-09-20T19:09:34.033 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.97038504999999997 2024-09-20T19:09:34.033 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.033 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.033 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:09:34.033 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.586193+0000", 2024-09-20T19:09:34.034 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.9775000000000001e-05 2024-09-20T19:09:34.034 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.034 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.034 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,7", 2024-09-20T19:09:34.034 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.586269+0000", 2024-09-20T19:09:34.034 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5494999999999995e-05 2024-09-20T19:09:34.034 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.034 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.034 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:34.034 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.593085+0000", 2024-09-20T19:09:34.034 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.006816491 2024-09-20T19:09:34.034 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.034 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.034 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:09:34.035 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.593535+0000", 2024-09-20T19:09:34.035 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00044982300000000002 2024-09-20T19:09:34.035 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.035 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.035 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:34.035 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.593843+0000", 2024-09-20T19:09:34.035 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00030740599999999998 2024-09-20T19:09:34.035 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.035 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.035 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:09:34.035 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.593891+0000", 2024-09-20T19:09:34.035 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8356999999999998e-05 2024-09-20T19:09:34.035 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.035 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.036 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:09:34.036 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.593982+0000", 2024-09-20T19:09:34.036 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.1506000000000001e-05 2024-09-20T19:09:34.036 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:34.036 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:09:34.036 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:34.036 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.036 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.037 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:205708 10.1 10:814c8b67:::benchmark_data_smithi083_148100_object88846:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e402)", 2024-09-20T19:09:34.037 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:32.638626+0000", 2024-09-20T19:09:34.037 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 241.33738000299999, 2024-09-20T19:09:34.037 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.95558198500000002, 2024-09-20T19:09:34.037 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:09:34.037 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:09:34.037 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:09:34.037 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:09:34.038 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:09:34.038 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:09:34.038 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 205708 2024-09-20T19:09:34.038 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.038 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:09:34.038 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.038 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:09:34.038 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.638626+0000", 2024-09-20T19:09:34.038 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:34.038 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.038 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.038 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:09:34.038 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.638626+0000", 2024-09-20T19:09:34.039 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:34.039 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.039 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.039 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:09:34.039 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.638628+0000", 2024-09-20T19:09:34.039 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.42e-06 2024-09-20T19:09:34.039 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.039 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.039 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:09:34.039 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.638633+0000", 2024-09-20T19:09:34.039 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.967e-06 2024-09-20T19:09:34.039 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.039 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.039 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:09:34.040 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.638646+0000", 2024-09-20T19:09:34.040 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3019e-05 2024-09-20T19:09:34.040 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.040 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.040 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:09:34.040 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.638653+0000", 2024-09-20T19:09:34.040 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.6930000000000007e-06 2024-09-20T19:09:34.040 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.040 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.040 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:34.040 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.638718+0000", 2024-09-20T19:09:34.040 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4233000000000007e-05 2024-09-20T19:09:34.040 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.040 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.040 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:09:34.041 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.638727+0000", 2024-09-20T19:09:34.041 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.4339999999999998e-06 2024-09-20T19:09:34.041 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.041 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.041 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:34.041 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.588452+0000", 2024-09-20T19:09:34.041 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.94972512399999998 2024-09-20T19:09:34.041 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.041 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.041 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:09:34.041 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.588554+0000", 2024-09-20T19:09:34.041 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00010216400000000001 2024-09-20T19:09:34.042 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.042 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.042 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,7", 2024-09-20T19:09:34.042 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.588674+0000", 2024-09-20T19:09:34.042 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011972900000000001 2024-09-20T19:09:34.042 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.042 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.042 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:09:34.042 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.593541+0000", 2024-09-20T19:09:34.042 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0048667520000000002 2024-09-20T19:09:34.042 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.042 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.042 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:34.042 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.594027+0000", 2024-09-20T19:09:34.043 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00048586199999999999 2024-09-20T19:09:34.043 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.043 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.043 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:34.043 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.594080+0000", 2024-09-20T19:09:34.043 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3154e-05 2024-09-20T19:09:34.043 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.043 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.043 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:09:34.043 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.594125+0000", 2024-09-20T19:09:34.043 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.4835e-05 2024-09-20T19:09:34.043 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.044 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.044 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:09:34.044 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.594208+0000", 2024-09-20T19:09:34.044 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.3598999999999997e-05 2024-09-20T19:09:34.044 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:34.044 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:09:34.044 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:34.044 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.044 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.044 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:205719 10.1 10:86f49b31:::benchmark_data_smithi083_148100_object88857:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e402)", 2024-09-20T19:09:34.044 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:32.648180+0000", 2024-09-20T19:09:34.044 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 241.327826593, 2024-09-20T19:09:34.044 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.96688380100000004, 2024-09-20T19:09:34.045 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:09:34.045 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:09:34.045 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:09:34.045 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:09:34.045 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:09:34.045 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:09:34.045 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 205719 2024-09-20T19:09:34.045 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.045 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:09:34.045 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.045 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:09:34.045 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.648180+0000", 2024-09-20T19:09:34.045 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:34.046 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.046 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.046 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:09:34.046 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.648180+0000", 2024-09-20T19:09:34.046 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:34.046 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.046 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.046 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:09:34.046 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.648181+0000", 2024-09-20T19:09:34.046 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1230000000000001e-06 2024-09-20T19:09:34.046 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.046 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.046 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:09:34.047 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.648185+0000", 2024-09-20T19:09:34.047 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.7790000000000002e-06 2024-09-20T19:09:34.047 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.047 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.047 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:09:34.047 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.648195+0000", 2024-09-20T19:09:34.047 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0516999999999999e-05 2024-09-20T19:09:34.047 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.047 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.047 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:09:34.047 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.648201+0000", 2024-09-20T19:09:34.047 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.57e-06 2024-09-20T19:09:34.047 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.048 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.048 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:34.048 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.648262+0000", 2024-09-20T19:09:34.048 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0897000000000003e-05 2024-09-20T19:09:34.048 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.048 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.048 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:09:34.048 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.648270+0000", 2024-09-20T19:09:34.048 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.96e-06 2024-09-20T19:09:34.048 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.048 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.048 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:34.049 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.590682+0000", 2024-09-20T19:09:34.049 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.94241219300000001 2024-09-20T19:09:34.049 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.049 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.049 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:09:34.049 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.590815+0000", 2024-09-20T19:09:34.049 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013299000000000001 2024-09-20T19:09:34.049 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.049 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.049 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,7", 2024-09-20T19:09:34.049 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.590926+0000", 2024-09-20T19:09:34.049 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000111524 2024-09-20T19:09:34.049 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.050 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.050 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:34.050 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.594279+0000", 2024-09-20T19:09:34.050 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0033527510000000002 2024-09-20T19:09:34.050 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.050 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.050 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:34.050 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.602137+0000", 2024-09-20T19:09:34.050 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0078584109999999992 2024-09-20T19:09:34.050 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.050 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.050 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:09:34.050 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.614874+0000", 2024-09-20T19:09:34.050 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.012736855 2024-09-20T19:09:34.051 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.051 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.051 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:09:34.051 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.614936+0000", 2024-09-20T19:09:34.051 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.1848999999999997e-05 2024-09-20T19:09:34.051 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.051 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.051 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:09:34.051 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.615064+0000", 2024-09-20T19:09:34.051 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012738199999999999 2024-09-20T19:09:34.051 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:34.051 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:09:34.051 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:34.051 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.052 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.052 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:205723 10.1 10:863f61c6:::benchmark_data_smithi083_148100_object88861:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e402)", 2024-09-20T19:09:34.052 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:32.651977+0000", 2024-09-20T19:09:34.052 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 241.32402954299999, 2024-09-20T19:09:34.052 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.96330407100000004, 2024-09-20T19:09:34.052 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:09:34.052 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:09:34.052 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:09:34.052 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:09:34.052 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:09:34.052 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:09:34.052 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 205723 2024-09-20T19:09:34.052 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.052 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:09:34.053 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.053 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:09:34.053 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.651977+0000", 2024-09-20T19:09:34.053 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:34.053 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.053 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.053 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:09:34.053 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.651977+0000", 2024-09-20T19:09:34.053 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:34.053 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.053 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.053 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:09:34.053 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.651978+0000", 2024-09-20T19:09:34.053 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3629999999999999e-06 2024-09-20T19:09:34.054 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.054 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.054 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:09:34.054 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.651982+0000", 2024-09-20T19:09:34.054 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.1439999999999996e-06 2024-09-20T19:09:34.054 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.054 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.054 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:09:34.054 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.651996+0000", 2024-09-20T19:09:34.054 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3570999999999999e-05 2024-09-20T19:09:34.054 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.054 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.054 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:09:34.055 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.652003+0000", 2024-09-20T19:09:34.055 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.0330000000000003e-06 2024-09-20T19:09:34.055 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.055 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.055 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:34.055 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.652083+0000", 2024-09-20T19:09:34.055 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.0519000000000006e-05 2024-09-20T19:09:34.055 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.055 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.055 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:09:34.055 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.652092+0000", 2024-09-20T19:09:34.055 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.6510000000000007e-06 2024-09-20T19:09:34.055 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.056 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.056 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:34.056 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.594315+0000", 2024-09-20T19:09:34.056 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.94222261900000004 2024-09-20T19:09:34.056 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.056 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.056 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:09:34.056 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.594445+0000", 2024-09-20T19:09:34.056 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013034799999999999 2024-09-20T19:09:34.056 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.056 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.056 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,7", 2024-09-20T19:09:34.056 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.594561+0000", 2024-09-20T19:09:34.056 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011615 2024-09-20T19:09:34.056 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.057 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.057 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:34.057 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.602095+0000", 2024-09-20T19:09:34.057 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0075340620000000002 2024-09-20T19:09:34.057 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.057 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.057 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:34.057 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.602178+0000", 2024-09-20T19:09:34.057 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.2545000000000003e-05 2024-09-20T19:09:34.057 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.057 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.057 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:09:34.057 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.615117+0000", 2024-09-20T19:09:34.058 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.01293937 2024-09-20T19:09:34.058 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.058 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.058 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:09:34.058 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.615197+0000", 2024-09-20T19:09:34.058 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.9394999999999995e-05 2024-09-20T19:09:34.058 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.058 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.058 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:09:34.058 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.615281+0000", 2024-09-20T19:09:34.058 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.4301000000000005e-05 2024-09-20T19:09:34.058 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:34.058 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:09:34.058 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:34.059 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.059 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.059 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:205730 10.1 10:8d4073c3:::benchmark_data_smithi083_148100_object88868:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e402)", 2024-09-20T19:09:34.059 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:32.656915+0000", 2024-09-20T19:09:34.059 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 241.319091273, 2024-09-20T19:09:34.059 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.95860854799999995, 2024-09-20T19:09:34.059 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:09:34.059 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:09:34.059 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:09:34.059 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:09:34.059 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:09:34.059 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:09:34.059 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 205730 2024-09-20T19:09:34.060 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.060 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:09:34.060 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.060 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:09:34.060 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.656915+0000", 2024-09-20T19:09:34.060 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:34.060 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.060 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.060 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:09:34.060 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.656915+0000", 2024-09-20T19:09:34.060 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:34.060 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.060 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.060 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:09:34.060 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.656915+0000", 2024-09-20T19:09:34.060 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.4400000000000001e-07 2024-09-20T19:09:34.061 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.061 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.061 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:09:34.061 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.656918+0000", 2024-09-20T19:09:34.061 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2299999999999998e-06 2024-09-20T19:09:34.061 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.061 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.061 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:09:34.061 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.656923+0000", 2024-09-20T19:09:34.061 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.1959999999999999e-06 2024-09-20T19:09:34.061 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.061 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.061 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:09:34.061 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.656928+0000", 2024-09-20T19:09:34.062 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.9810000000000003e-06 2024-09-20T19:09:34.062 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.062 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.062 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:34.062 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.656976+0000", 2024-09-20T19:09:34.062 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8124000000000002e-05 2024-09-20T19:09:34.062 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.062 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.062 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:09:34.062 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.656984+0000", 2024-09-20T19:09:34.062 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.6699999999999994e-06 2024-09-20T19:09:34.062 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.062 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.063 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:34.066 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.597077+0000", 2024-09-20T19:09:34.066 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.94009377500000002 2024-09-20T19:09:34.066 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.066 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.066 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:09:34.067 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.597187+0000", 2024-09-20T19:09:34.067 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000109634 2024-09-20T19:09:34.067 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.067 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.067 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,7", 2024-09-20T19:09:34.067 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.597329+0000", 2024-09-20T19:09:34.067 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00014175900000000001 2024-09-20T19:09:34.067 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.067 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.067 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:34.067 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.602215+0000", 2024-09-20T19:09:34.067 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0048856699999999999 2024-09-20T19:09:34.067 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.067 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.068 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:34.068 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.607168+0000", 2024-09-20T19:09:34.068 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0049536320000000003 2024-09-20T19:09:34.068 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.068 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.068 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:09:34.068 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.615291+0000", 2024-09-20T19:09:34.068 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0081227369999999997 2024-09-20T19:09:34.068 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.068 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.068 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:09:34.068 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.615367+0000", 2024-09-20T19:09:34.068 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5866999999999997e-05 2024-09-20T19:09:34.068 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.069 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.069 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:09:34.069 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.615524+0000", 2024-09-20T19:09:34.069 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015682900000000001 2024-09-20T19:09:34.069 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:34.069 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:09:34.069 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:34.069 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.069 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.069 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:205764 10.1 10:815dd112:::benchmark_data_smithi083_148100_object88902:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e402)", 2024-09-20T19:09:34.069 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:32.693962+0000", 2024-09-20T19:09:34.069 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 241.28204443000001, 2024-09-20T19:09:34.069 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.92202304099999999, 2024-09-20T19:09:34.070 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:09:34.070 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:09:34.070 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:09:34.070 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:09:34.070 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:09:34.070 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:09:34.070 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 205764 2024-09-20T19:09:34.070 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.070 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:09:34.070 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.070 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:09:34.070 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.693962+0000", 2024-09-20T19:09:34.070 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:34.071 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.071 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.071 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:09:34.071 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.693962+0000", 2024-09-20T19:09:34.071 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:34.071 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.071 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.071 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:09:34.071 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.693963+0000", 2024-09-20T19:09:34.071 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.3600000000000002e-07 2024-09-20T19:09:34.071 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.071 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.071 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:09:34.071 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.693966+0000", 2024-09-20T19:09:34.072 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4929999999999998e-06 2024-09-20T19:09:34.072 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.072 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.072 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:09:34.072 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.693975+0000", 2024-09-20T19:09:34.072 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.5609999999999999e-06 2024-09-20T19:09:34.072 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.072 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.073 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:09:34.073 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.693980+0000", 2024-09-20T19:09:34.073 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.1889999999999998e-06 2024-09-20T19:09:34.073 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.073 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.073 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:34.073 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.694089+0000", 2024-09-20T19:09:34.073 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00010870999999999999 2024-09-20T19:09:34.073 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.073 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.073 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:09:34.073 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.694098+0000", 2024-09-20T19:09:34.073 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.2280000000000004e-06 2024-09-20T19:09:34.073 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.073 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.074 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:34.074 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.599637+0000", 2024-09-20T19:09:34.074 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.90553884799999995 2024-09-20T19:09:34.074 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.074 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.074 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:09:34.074 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.599736+0000", 2024-09-20T19:09:34.074 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.8777000000000007e-05 2024-09-20T19:09:34.074 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.074 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.075 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,7", 2024-09-20T19:09:34.075 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.599843+0000", 2024-09-20T19:09:34.075 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000106981 2024-09-20T19:09:34.075 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.075 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.075 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:34.075 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.602255+0000", 2024-09-20T19:09:34.075 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0024121809999999998 2024-09-20T19:09:34.075 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.075 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.075 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:09:34.075 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.615534+0000", 2024-09-20T19:09:34.075 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.013278853 2024-09-20T19:09:34.075 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.076 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.076 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:34.076 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.615677+0000", 2024-09-20T19:09:34.076 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00014370499999999999 2024-09-20T19:09:34.076 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.076 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.076 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:09:34.076 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.615884+0000", 2024-09-20T19:09:34.076 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00020712899999999999 2024-09-20T19:09:34.076 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.076 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.076 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:09:34.076 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.615985+0000", 2024-09-20T19:09:34.076 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00010045 2024-09-20T19:09:34.076 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:34.077 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:09:34.077 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:34.077 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.077 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.077 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:205767 10.1 10:818a5cd4:::benchmark_data_smithi083_148100_object88905:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e402)", 2024-09-20T19:09:34.077 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:32.697165+0000", 2024-09-20T19:09:34.077 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 241.278841301, 2024-09-20T19:09:34.077 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.919069044, 2024-09-20T19:09:34.077 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:09:34.077 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:09:34.077 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:09:34.077 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:09:34.077 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:09:34.077 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:09:34.078 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 205767 2024-09-20T19:09:34.078 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.078 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:09:34.078 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.078 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:09:34.078 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.697165+0000", 2024-09-20T19:09:34.078 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:34.078 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.078 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.078 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:09:34.078 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.697165+0000", 2024-09-20T19:09:34.078 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:34.078 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.078 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.079 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:09:34.079 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.697166+0000", 2024-09-20T19:09:34.079 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.269e-06 2024-09-20T19:09:34.079 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.079 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.079 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:09:34.079 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.697171+0000", 2024-09-20T19:09:34.079 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.5260000000000004e-06 2024-09-20T19:09:34.079 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.079 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.079 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:09:34.079 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.697183+0000", 2024-09-20T19:09:34.079 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1691999999999999e-05 2024-09-20T19:09:34.079 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.080 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.080 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:09:34.080 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.697189+0000", 2024-09-20T19:09:34.080 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5880000000000003e-06 2024-09-20T19:09:34.080 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.080 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.080 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:34.080 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.697298+0000", 2024-09-20T19:09:34.080 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000108582 2024-09-20T19:09:34.080 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.080 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.080 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:09:34.080 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.697307+0000", 2024-09-20T19:09:34.080 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.8480000000000007e-06 2024-09-20T19:09:34.080 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.081 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.081 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:34.081 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.602282+0000", 2024-09-20T19:09:34.081 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.90497590000000006 2024-09-20T19:09:34.081 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.081 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.081 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:09:34.081 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.602412+0000", 2024-09-20T19:09:34.081 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000129677 2024-09-20T19:09:34.081 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.081 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.081 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,7", 2024-09-20T19:09:34.081 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.602519+0000", 2024-09-20T19:09:34.081 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000107104 2024-09-20T19:09:34.082 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.082 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.082 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:34.082 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.607208+0000", 2024-09-20T19:09:34.082 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0046892269999999998 2024-09-20T19:09:34.082 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.082 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.082 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:09:34.082 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.615539+0000", 2024-09-20T19:09:34.082 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0083305319999999999 2024-09-20T19:09:34.082 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.082 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.082 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:34.082 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.616043+0000", 2024-09-20T19:09:34.083 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00050444400000000005 2024-09-20T19:09:34.083 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.083 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.083 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:09:34.083 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.616125+0000", 2024-09-20T19:09:34.083 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.1205999999999995e-05 2024-09-20T19:09:34.083 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.083 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.083 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:09:34.083 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.616234+0000", 2024-09-20T19:09:34.083 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000109449 2024-09-20T19:09:34.083 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:34.083 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:09:34.083 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:34.084 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.084 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.084 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:205781 10.1 10:80f99c73:::benchmark_data_smithi083_148100_object88919:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e402)", 2024-09-20T19:09:34.084 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:32.721084+0000", 2024-09-20T19:09:34.084 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 241.254921964, 2024-09-20T19:09:34.084 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.89539797499999996, 2024-09-20T19:09:34.084 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:09:34.084 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:09:34.084 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:09:34.084 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:09:34.084 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:09:34.084 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:09:34.084 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 205781 2024-09-20T19:09:34.084 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.084 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:09:34.085 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.085 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:09:34.085 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.721084+0000", 2024-09-20T19:09:34.085 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:34.085 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.085 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.085 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:09:34.085 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.721084+0000", 2024-09-20T19:09:34.085 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:34.085 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.085 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.085 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:09:34.085 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.721086+0000", 2024-09-20T19:09:34.085 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5260000000000001e-06 2024-09-20T19:09:34.085 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.086 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.086 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:09:34.086 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.721090+0000", 2024-09-20T19:09:34.086 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9009999999999996e-06 2024-09-20T19:09:34.086 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.086 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.086 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:09:34.086 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.721099+0000", 2024-09-20T19:09:34.086 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.6479999999999998e-06 2024-09-20T19:09:34.086 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.086 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.086 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:09:34.086 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.721105+0000", 2024-09-20T19:09:34.086 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.1340000000000003e-06 2024-09-20T19:09:34.087 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.087 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.087 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:34.087 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.721152+0000", 2024-09-20T19:09:34.087 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7840000000000003e-05 2024-09-20T19:09:34.087 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.087 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.087 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:09:34.087 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.721159+0000", 2024-09-20T19:09:34.087 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2199999999999997e-06 2024-09-20T19:09:34.087 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.087 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.087 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:34.087 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.604676+0000", 2024-09-20T19:09:34.087 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.88351720700000003 2024-09-20T19:09:34.088 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.088 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.088 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:09:34.088 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.604781+0000", 2024-09-20T19:09:34.088 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000104929 2024-09-20T19:09:34.088 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.088 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.088 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,7", 2024-09-20T19:09:34.088 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.604885+0000", 2024-09-20T19:09:34.088 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000104182 2024-09-20T19:09:34.088 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.088 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.088 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:34.089 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.615105+0000", 2024-09-20T19:09:34.089 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.010220167 2024-09-20T19:09:34.089 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.089 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.090 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:09:34.090 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.615544+0000", 2024-09-20T19:09:34.090 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00043874199999999999 2024-09-20T19:09:34.090 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.090 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.090 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:34.090 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.616316+0000", 2024-09-20T19:09:34.090 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00077201899999999998 2024-09-20T19:09:34.090 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.090 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.090 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:09:34.090 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.616404+0000", 2024-09-20T19:09:34.090 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.8182000000000004e-05 2024-09-20T19:09:34.091 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.091 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.091 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:09:34.091 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.616482+0000", 2024-09-20T19:09:34.091 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.8277999999999998e-05 2024-09-20T19:09:34.091 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:34.091 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:09:34.091 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:34.091 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.091 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.091 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.25775.0:205782 10.1 10:8533dcb4:::benchmark_data_smithi083_148100_object88920:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e402)", 2024-09-20T19:09:34.091 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:05:32.721158+0000", 2024-09-20T19:09:34.091 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 241.254848646, 2024-09-20T19:09:34.091 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.89551617299999997, 2024-09-20T19:09:34.091 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:09:34.092 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:09:34.092 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:09:34.092 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:09:34.092 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.25775", 2024-09-20T19:09:34.092 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2156658160", 2024-09-20T19:09:34.092 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 205782 2024-09-20T19:09:34.092 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.092 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:09:34.092 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.092 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:09:34.092 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.721158+0000", 2024-09-20T19:09:34.092 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:34.092 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.092 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.093 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:09:34.093 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.721158+0000", 2024-09-20T19:09:34.093 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:34.093 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.093 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.093 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:09:34.093 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.721158+0000", 2024-09-20T19:09:34.093 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.08e-07 2024-09-20T19:09:34.093 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.093 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.093 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:09:34.093 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.721160+0000", 2024-09-20T19:09:34.093 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0559999999999999e-06 2024-09-20T19:09:34.094 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.094 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.094 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:09:34.094 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.721165+0000", 2024-09-20T19:09:34.094 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.6959999999999996e-06 2024-09-20T19:09:34.094 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.094 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.094 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:09:34.094 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.721169+0000", 2024-09-20T19:09:34.094 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.6059999999999996e-06 2024-09-20T19:09:34.094 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.094 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.094 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:34.094 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.721197+0000", 2024-09-20T19:09:34.095 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.7588000000000001e-05 2024-09-20T19:09:34.095 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.095 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.095 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for peered", 2024-09-20T19:09:34.095 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:32.721201+0000", 2024-09-20T19:09:34.095 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4740000000000001e-06 2024-09-20T19:09:34.095 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.095 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.095 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:34.095 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.607242+0000", 2024-09-20T19:09:34.095 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.88604142500000005 2024-09-20T19:09:34.095 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.096 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.096 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:09:34.096 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.607364+0000", 2024-09-20T19:09:34.096 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012256699999999999 2024-09-20T19:09:34.096 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.096 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.096 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 0,7", 2024-09-20T19:09:34.097 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.607585+0000", 2024-09-20T19:09:34.097 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000220724 2024-09-20T19:09:34.097 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.097 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.097 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:09:34.097 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.615549+0000", 2024-09-20T19:09:34.097 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0079635890000000001 2024-09-20T19:09:34.097 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.097 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.097 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:34.097 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.615592+0000", 2024-09-20T19:09:34.097 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3353999999999999e-05 2024-09-20T19:09:34.097 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.097 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.098 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:34.098 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.616519+0000", 2024-09-20T19:09:34.098 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:09:34.099 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00092725000000000001 2024-09-20T19:09:34.099 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.099 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.099 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:09:34.099 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.616593+0000", 2024-09-20T19:09:34.099 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.3176000000000005e-05 2024-09-20T19:09:34.099 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.099 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.100 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:09:34.100 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:05:33.616674+0000", 2024-09-20T19:09:34.100 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.1260000000000005e-05 2024-09-20T19:09:34.100 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:34.100 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:09:34.100 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:34.100 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.100 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.100 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.26264.0:60550 11.e 11:704e4ac7:::benchmark_data_smithi083_177491_object60549:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e440)", 2024-09-20T19:09:34.100 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:06:55.419747+0000", 2024-09-20T19:09:34.100 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 158.55625974, 2024-09-20T19:09:34.100 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.38041948799999997, 2024-09-20T19:09:34.100 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:09:34.100 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:09:34.101 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:09:34.101 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:09:34.101 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.26264", 2024-09-20T19:09:34.101 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2608780316", 2024-09-20T19:09:34.101 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 60550 2024-09-20T19:09:34.101 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.101 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:09:34.101 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.101 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:09:34.101 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:55.419747+0000", 2024-09-20T19:09:34.101 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:34.101 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.101 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.101 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:09:34.101 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:55.419747+0000", 2024-09-20T19:09:34.102 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:34.102 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.102 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.102 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:09:34.102 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:55.419748+0000", 2024-09-20T19:09:34.102 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.519e-06 2024-09-20T19:09:34.102 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.102 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.102 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:09:34.102 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:55.419790+0000", 2024-09-20T19:09:34.102 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.1907999999999999e-05 2024-09-20T19:09:34.103 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.103 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.103 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:09:34.103 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:55.419805+0000", 2024-09-20T19:09:34.103 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4581e-05 2024-09-20T19:09:34.104 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.104 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.104 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:09:34.104 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:55.419813+0000", 2024-09-20T19:09:34.104 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.3399999999999998e-06 2024-09-20T19:09:34.104 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.104 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.104 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:34.104 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:55.419864+0000", 2024-09-20T19:09:34.104 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.1181999999999999e-05 2024-09-20T19:09:34.104 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.104 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.104 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:09:34.104 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:55.420009+0000", 2024-09-20T19:09:34.105 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000144723 2024-09-20T19:09:34.105 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.105 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.105 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,7", 2024-09-20T19:09:34.105 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:55.420154+0000", 2024-09-20T19:09:34.105 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000145117 2024-09-20T19:09:34.105 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.105 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.105 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:09:34.105 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:55.424484+0000", 2024-09-20T19:09:34.105 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0043300860000000004 2024-09-20T19:09:34.105 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.105 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.105 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:34.105 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:55.425237+0000", 2024-09-20T19:09:34.106 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00075254900000000003 2024-09-20T19:09:34.106 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.106 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.106 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:34.106 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:55.800061+0000", 2024-09-20T19:09:34.106 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.37482443300000001 2024-09-20T19:09:34.106 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.106 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.106 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:09:34.106 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:55.800124+0000", 2024-09-20T19:09:34.106 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.3184999999999999e-05 2024-09-20T19:09:34.106 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.106 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.106 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:09:34.107 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:06:55.800166+0000", 2024-09-20T19:09:34.107 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.1865e-05 2024-09-20T19:09:34.107 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:34.107 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:09:34.107 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:34.107 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.107 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.107 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.26264.0:195574 11.e 11:7c27cc84:::benchmark_data_smithi083_177491_object92132:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e494)", 2024-09-20T19:09:34.107 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:09:23.120398+0000", 2024-09-20T19:09:34.107 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 10.855608543000001, 2024-09-20T19:09:34.107 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.27825100600000002, 2024-09-20T19:09:34.107 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:09:34.108 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:09:34.108 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:09:34.108 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:09:34.108 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.26264", 2024-09-20T19:09:34.108 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2608780316", 2024-09-20T19:09:34.108 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 195574 2024-09-20T19:09:34.108 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.108 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:09:34.108 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.108 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:09:34.108 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:23.120398+0000", 2024-09-20T19:09:34.108 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:34.108 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.108 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.109 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:09:34.109 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:23.120398+0000", 2024-09-20T19:09:34.109 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:34.109 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.109 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.109 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:09:34.109 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:23.120400+0000", 2024-09-20T19:09:34.109 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.781e-06 2024-09-20T19:09:34.109 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.109 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.109 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:09:34.109 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:23.120406+0000", 2024-09-20T19:09:34.110 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5989999999999998e-06 2024-09-20T19:09:34.110 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.110 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.110 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:09:34.110 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:23.120423+0000", 2024-09-20T19:09:34.110 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6623999999999999e-05 2024-09-20T19:09:34.111 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.111 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.111 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:09:34.111 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:23.120432+0000", 2024-09-20T19:09:34.111 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.0969999999999995e-06 2024-09-20T19:09:34.111 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.111 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.111 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:34.111 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:23.120497+0000", 2024-09-20T19:09:34.111 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4857000000000006e-05 2024-09-20T19:09:34.111 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.111 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.111 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:09:34.111 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:23.120621+0000", 2024-09-20T19:09:34.111 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012463 2024-09-20T19:09:34.112 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.112 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.112 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,7", 2024-09-20T19:09:34.112 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:23.120743+0000", 2024-09-20T19:09:34.112 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012183100000000001 2024-09-20T19:09:34.112 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.112 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.112 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:09:34.112 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:23.123407+0000", 2024-09-20T19:09:34.112 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0026639480000000002 2024-09-20T19:09:34.112 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.112 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.112 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:34.112 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:23.123860+0000", 2024-09-20T19:09:34.113 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00045281899999999998 2024-09-20T19:09:34.113 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.113 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.113 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:34.113 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:23.398510+0000", 2024-09-20T19:09:34.113 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.274649698 2024-09-20T19:09:34.113 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.113 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.113 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:09:34.113 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:23.398603+0000", 2024-09-20T19:09:34.113 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.3288999999999994e-05 2024-09-20T19:09:34.113 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.113 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.113 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:09:34.114 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:23.398649+0000", 2024-09-20T19:09:34.114 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.5833e-05 2024-09-20T19:09:34.114 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:34.114 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:09:34.114 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:34.114 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.114 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.114 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.26264.0:203926 11.e 11:72d583bd:::benchmark_data_smithi083_177491_object100484:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e499)", 2024-09-20T19:09:34.114 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:09:31.991662+0000", 2024-09-20T19:09:34.114 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 1.9843445, 2024-09-20T19:09:34.114 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.22287606300000001, 2024-09-20T19:09:34.114 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:09:34.114 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:09:34.114 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:09:34.115 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:09:34.115 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.26264", 2024-09-20T19:09:34.115 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2608780316", 2024-09-20T19:09:34.115 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 203926 2024-09-20T19:09:34.115 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.115 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:09:34.115 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.115 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:09:34.115 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:31.991662+0000", 2024-09-20T19:09:34.115 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:34.115 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.115 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.115 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:09:34.116 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:31.991662+0000", 2024-09-20T19:09:34.116 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:34.116 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.116 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.116 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:09:34.116 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:31.991664+0000", 2024-09-20T19:09:34.116 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.905e-06 2024-09-20T19:09:34.116 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.116 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.116 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:09:34.116 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:31.991670+0000", 2024-09-20T19:09:34.116 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.1109999999999996e-06 2024-09-20T19:09:34.116 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.117 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.117 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:09:34.117 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:31.991683+0000", 2024-09-20T19:09:34.117 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2819e-05 2024-09-20T19:09:34.117 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.118 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.118 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:09:34.118 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:31.991690+0000", 2024-09-20T19:09:34.118 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.99e-06 2024-09-20T19:09:34.118 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.118 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.118 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:34.118 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:31.991744+0000", 2024-09-20T19:09:34.118 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4438000000000003e-05 2024-09-20T19:09:34.118 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.118 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.118 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:09:34.118 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:31.991856+0000", 2024-09-20T19:09:34.119 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011154000000000001 2024-09-20T19:09:34.119 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.119 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.119 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,7", 2024-09-20T19:09:34.119 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:31.991962+0000", 2024-09-20T19:09:34.119 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000106778 2024-09-20T19:09:34.119 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.119 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.119 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:09:34.119 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:31.995633+0000", 2024-09-20T19:09:34.119 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.003670881 2024-09-20T19:09:34.119 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.119 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.119 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:34.119 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:32.003277+0000", 2024-09-20T19:09:34.120 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0076439120000000001 2024-09-20T19:09:34.120 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.120 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.120 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:34.120 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:32.214473+0000", 2024-09-20T19:09:34.120 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.21119617700000001 2024-09-20T19:09:34.120 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.120 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.120 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:09:34.120 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:32.214509+0000", 2024-09-20T19:09:34.120 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.5620000000000001e-05 2024-09-20T19:09:34.120 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.120 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.121 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:09:34.121 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:32.214538+0000", 2024-09-20T19:09:34.121 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.8892000000000001e-05 2024-09-20T19:09:34.121 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:34.121 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:09:34.121 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:34.121 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.121 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.121 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.26264.0:203935 11.e 11:70cc1fea:::benchmark_data_smithi083_177491_object100493:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e499)", 2024-09-20T19:09:34.121 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-09-20T19:09:31.996512+0000", 2024-09-20T19:09:34.121 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 1.9794943890000001, 2024-09-20T19:09:34.121 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.21845092999999999, 2024-09-20T19:09:34.121 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-09-20T19:09:34.121 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-09-20T19:09:34.122 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T19:09:34.122 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-09-20T19:09:34.122 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.26264", 2024-09-20T19:09:34.122 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.83:0/2608780316", 2024-09-20T19:09:34.122 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 203935 2024-09-20T19:09:34.122 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.122 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-09-20T19:09:34.122 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.122 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-09-20T19:09:34.122 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:31.996512+0000", 2024-09-20T19:09:34.122 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:34.122 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.122 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.122 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-09-20T19:09:34.122 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:31.996512+0000", 2024-09-20T19:09:34.123 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-09-20T19:09:34.123 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.123 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.123 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-09-20T19:09:34.123 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:31.996513+0000", 2024-09-20T19:09:34.123 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.6000000000000003e-07 2024-09-20T19:09:34.123 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.123 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.123 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-09-20T19:09:34.123 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:31.996516+0000", 2024-09-20T19:09:34.123 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.027e-06 2024-09-20T19:09:34.123 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.124 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.124 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-09-20T19:09:34.124 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:31.996525+0000", 2024-09-20T19:09:34.124 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.6129999999999993e-06 2024-09-20T19:09:34.124 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.124 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.124 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-09-20T19:09:34.124 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:31.996532+0000", 2024-09-20T19:09:34.124 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4069999999999998e-06 2024-09-20T19:09:34.125 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.125 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.125 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-09-20T19:09:34.125 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:31.999982+0000", 2024-09-20T19:09:34.125 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0034499399999999999 2024-09-20T19:09:34.125 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.125 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.125 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-09-20T19:09:34.125 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:32.000291+0000", 2024-09-20T19:09:34.125 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00030936000000000001 2024-09-20T19:09:34.125 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.125 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.125 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for subops from 1,7", 2024-09-20T19:09:34.125 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:32.000620+0000", 2024-09-20T19:09:34.125 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00032911600000000001 2024-09-20T19:09:34.126 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.126 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.126 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:34.126 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:32.004015+0000", 2024-09-20T19:09:34.126 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0033947999999999999 2024-09-20T19:09:34.126 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.126 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.126 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op_commit", 2024-09-20T19:09:34.126 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:32.004185+0000", 2024-09-20T19:09:34.126 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000170089 2024-09-20T19:09:34.126 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.126 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.127 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_commit_rec", 2024-09-20T19:09:34.127 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:32.214695+0000", 2024-09-20T19:09:34.127 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.21051020300000001 2024-09-20T19:09:34.127 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.127 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.127 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-09-20T19:09:34.127 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:32.214905+0000", 2024-09-20T19:09:34.127 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00021005000000000001 2024-09-20T19:09:34.127 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-20T19:09:34.127 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-09-20T19:09:34.127 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-09-20T19:09:34.127 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-09-20T19:09:32.214963+0000", 2024-09-20T19:09:34.127 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.7664999999999998e-05 2024-09-20T19:09:34.127 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:34.127 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:09:34.128 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:34.128 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-20T19:09:34.128 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-20T19:09:34.128 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-20T19:09:34.162 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:34.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:34.163 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:09:34.264 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:09:34.299 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:34.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:34.301 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:09:34.370 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Removed 103439 objects 2024-09-20T19:09:34.370 INFO:tasks.radosbench.radosbench.0.smithi083.stdout:Clean up completed and total clean up time :101.227 2024-09-20T19:09:34.401 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:09:34.423 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:34.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:34.425 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:09:34.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:34 smithi142 ceph-mon[32508]: pgmap v1007: 168 pgs: 1 active+clean+remapped, 3 active+undersized+degraded+remapped+backfill_toofull, 34 active+undersized, 2 active+remapped+backfill_toofull, 22 active+undersized+degraded, 106 active+clean; 210 MiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 982 op/s; 1057/10635 objects degraded (9.939%); 1177/10635 objects misplaced (11.067%); 1.8 MiB/s, 26 objects/s recovering 2024-09-20T19:09:34.525 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:09:34.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:34 smithi083 ceph-mon[29421]: pgmap v1007: 168 pgs: 1 active+clean+remapped, 3 active+undersized+degraded+remapped+backfill_toofull, 34 active+undersized, 2 active+remapped+backfill_toofull, 22 active+undersized+degraded, 106 active+clean; 210 MiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 982 op/s; 1057/10635 objects degraded (9.939%); 1177/10635 objects misplaced (11.067%); 1.8 MiB/s, 26 objects/s recovering 2024-09-20T19:09:34.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:34 smithi083 ceph-mon[36260]: pgmap v1007: 168 pgs: 1 active+clean+remapped, 3 active+undersized+degraded+remapped+backfill_toofull, 34 active+undersized, 2 active+remapped+backfill_toofull, 22 active+undersized+degraded, 106 active+clean; 210 MiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 982 op/s; 1057/10635 objects degraded (9.939%); 1177/10635 objects misplaced (11.067%); 1.8 MiB/s, 26 objects/s recovering 2024-09-20T19:09:34.545 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:34.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:34.546 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:09:34.647 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:09:34.669 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:34.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:34.670 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:09:34.770 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:09:34.791 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:34.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:34.792 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:09:34.892 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:09:34.918 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:34.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:34.919 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:09:35.020 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:09:35.044 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:35.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:35.045 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:09:35.146 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:09:35.168 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:35.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:35.169 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:09:35.238 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-20T19:09:35.270 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:09:35.304 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:35.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:35.305 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:09:35.406 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:09:35.427 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:35.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:35.428 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:09:35.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:35 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 1057/10635 objects degraded (9.939%), 25 pgs degraded (PG_DEGRADED) 2024-09-20T19:09:35.529 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:09:35.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:35 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 1057/10635 objects degraded (9.939%), 25 pgs degraded (PG_DEGRADED) 2024-09-20T19:09:35.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:35 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 1057/10635 objects degraded (9.939%), 25 pgs degraded (PG_DEGRADED) 2024-09-20T19:09:35.552 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:35.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:35.553 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:09:35.654 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:09:35.674 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:35.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:35.675 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:09:35.776 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:09:35.798 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:35.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:35.799 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:09:35.900 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:09:35.924 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:35.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:35.925 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:09:36.025 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:09:36.047 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:36.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:36.048 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:09:36.148 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:09:36.174 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:36.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:36.175 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:09:36.276 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:09:36.299 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:36.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:36.300 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:09:36.401 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:09:36.423 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:36.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:36.424 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:09:36.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:36 smithi142 ceph-mon[32508]: pgmap v1008: 168 pgs: 1 active+clean+remapped, 3 active+undersized+degraded+remapped+backfill_toofull, 34 active+undersized, 2 active+remapped+backfill_toofull, 22 active+undersized+degraded, 106 active+clean; 210 MiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 712 B/s rd, 0 B/s wr, 768 op/s; 1057/10635 objects degraded (9.939%); 1177/10635 objects misplaced (11.067%); 1.5 MiB/s, 23 objects/s recovering 2024-09-20T19:09:36.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:36 smithi142 ceph-mon[32508]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 5 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T19:09:36.525 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:09:36.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:36 smithi083 ceph-mon[29421]: pgmap v1008: 168 pgs: 1 active+clean+remapped, 3 active+undersized+degraded+remapped+backfill_toofull, 34 active+undersized, 2 active+remapped+backfill_toofull, 22 active+undersized+degraded, 106 active+clean; 210 MiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 712 B/s rd, 0 B/s wr, 768 op/s; 1057/10635 objects degraded (9.939%); 1177/10635 objects misplaced (11.067%); 1.5 MiB/s, 23 objects/s recovering 2024-09-20T19:09:36.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:36 smithi083 ceph-mon[29421]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 5 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T19:09:36.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:36 smithi083 ceph-mon[36260]: pgmap v1008: 168 pgs: 1 active+clean+remapped, 3 active+undersized+degraded+remapped+backfill_toofull, 34 active+undersized, 2 active+remapped+backfill_toofull, 22 active+undersized+degraded, 106 active+clean; 210 MiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 712 B/s rd, 0 B/s wr, 768 op/s; 1057/10635 objects degraded (9.939%); 1177/10635 objects misplaced (11.067%); 1.5 MiB/s, 23 objects/s recovering 2024-09-20T19:09:36.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:36 smithi083 ceph-mon[36260]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 5 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T19:09:36.551 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:36.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:36.552 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:09:36.653 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:09:36.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:36.675 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:36.675 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:09:36.776 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:09:36.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:36.802 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:36.802 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:09:36.903 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:09:36.924 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:36.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:36.925 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:09:37.026 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:09:37.052 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:37.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:37.054 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:09:37.154 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:09:37.178 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:37.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:37.180 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:09:37.280 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:09:37.304 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:37.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:37.305 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:09:37.406 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:09:37.429 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:37.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:37.430 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:09:37.531 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:09:37.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:37.558 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:37.558 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:09:37.631 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:09:37.634 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-20T19:09:37.635 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-20T19:09:37.642 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:09:37.659 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:09:37.681 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:37.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:37.682 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:09:37.783 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:09:37.810 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:37.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:37.811 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:09:37.912 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:09:37.933 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:37.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:37.934 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:09:38.010 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:09:37 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:09:38.035 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:09:38.061 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:38.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:38.062 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:09:38.162 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:09:38.187 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:38.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:38.188 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:09:38.259 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_4 2024-09-20T19:09:38.259 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd pool rm unique_pool_4 unique_pool_4 --yes-i-really-really-mean-it 2024-09-20T19:09:38.289 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:09:38.323 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:38.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:38.324 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:09:38.425 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:09:38.445 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:38 smithi083 ceph-mon[29421]: pgmap v1009: 168 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 1 active+clean+remapped, 2 active+undersized+remapped+backfill_toofull, 36 active+undersized, 2 active+remapped+backfill_toofull, 20 active+undersized+degraded, 106 active+clean; 14 MiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.09k op/s; 59/1269 objects degraded (4.649%); 21/1269 objects misplaced (1.655%); 1.3 MiB/s, 21 objects/s recovering 2024-09-20T19:09:38.445 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:38 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:09:38.445 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:38 smithi083 ceph-mon[36260]: pgmap v1009: 168 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 1 active+clean+remapped, 2 active+undersized+remapped+backfill_toofull, 36 active+undersized, 2 active+remapped+backfill_toofull, 20 active+undersized+degraded, 106 active+clean; 14 MiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.09k op/s; 59/1269 objects degraded (4.649%); 21/1269 objects misplaced (1.655%); 1.3 MiB/s, 21 objects/s recovering 2024-09-20T19:09:38.445 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:38 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:09:38.447 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:38.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:38.448 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:09:38.549 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:09:38.562 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T19:09:38.562 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-20T19:09:38.562 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-20T19:09:38.562 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T19:09:38.575 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:38.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:38.576 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:09:38.676 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:09:38.701 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:38.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:38.702 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:09:38.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:38 smithi142 ceph-mon[32508]: pgmap v1009: 168 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 1 active+clean+remapped, 2 active+undersized+remapped+backfill_toofull, 36 active+undersized, 2 active+remapped+backfill_toofull, 20 active+undersized+degraded, 106 active+clean; 14 MiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.09k op/s; 59/1269 objects degraded (4.649%); 21/1269 objects misplaced (1.655%); 1.3 MiB/s, 21 objects/s recovering 2024-09-20T19:09:38.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:38 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:09:38.803 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:09:38.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:38.826 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:38.826 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:09:38.929 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:09:38.956 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:38.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:38.957 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:09:39.057 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:09:39.080 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:39.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:39.081 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:09:39.182 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:09:39.203 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:39.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:39.204 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:09:39.301 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-20T19:09:39.305 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:09:39.331 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:39.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:39.333 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:09:39.433 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:09:39.455 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:39.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:39.456 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:09:39.496 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:39 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/4040114718' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:09:39.496 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:39 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/4040114718' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:09:39.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:39 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/4040114718' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:09:39.556 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:09:39.578 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:39.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:39.579 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:09:39.636 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T19:09:39.680 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:09:39.705 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:39.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:39.706 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:09:39.806 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:09:39.828 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:39.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:39.829 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:09:39.930 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:09:39.951 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:39.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:39.952 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:09:40.052 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:09:40.079 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:40.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:40.080 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:09:40.181 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:09:40.205 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:40.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:40.206 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:09:40.307 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:09:40.331 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:40.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:40.332 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:09:40.433 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:09:40.458 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:40.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:40.459 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:09:40.516 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:40 smithi083 ceph-mon[29421]: pgmap v1010: 168 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 1 active+clean+remapped, 2 active+undersized+remapped+backfill_toofull, 36 active+undersized, 2 active+remapped+backfill_toofull, 20 active+undersized+degraded, 106 active+clean; 1.7 MiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 976 op/s; 59/657 objects degraded (8.980%); 21/657 objects misplaced (3.196%); 1.2 MiB/s, 18 objects/s recovering 2024-09-20T19:09:40.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:40 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 59/657 objects degraded (8.980%), 21 pgs degraded (PG_DEGRADED) 2024-09-20T19:09:40.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:40 smithi083 ceph-mon[29421]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:09:40.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:40 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/4040114718' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T19:09:40.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:40 smithi083 ceph-mon[29421]: osdmap e500: 8 total, 7 up, 8 in 2024-09-20T19:09:40.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:40 smithi083 ceph-mon[29421]: 1.3 scrub starts 2024-09-20T19:09:40.545 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:40 smithi083 ceph-mon[29421]: 1.3 scrub ok 2024-09-20T19:09:40.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:40 smithi083 ceph-mon[36260]: pgmap v1010: 168 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 1 active+clean+remapped, 2 active+undersized+remapped+backfill_toofull, 36 active+undersized, 2 active+remapped+backfill_toofull, 20 active+undersized+degraded, 106 active+clean; 1.7 MiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 976 op/s; 59/657 objects degraded (8.980%); 21/657 objects misplaced (3.196%); 1.2 MiB/s, 18 objects/s recovering 2024-09-20T19:09:40.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:40 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 59/657 objects degraded (8.980%), 21 pgs degraded (PG_DEGRADED) 2024-09-20T19:09:40.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:40 smithi083 ceph-mon[36260]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:09:40.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:40 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/4040114718' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T19:09:40.545 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:40 smithi083 ceph-mon[36260]: osdmap e500: 8 total, 7 up, 8 in 2024-09-20T19:09:40.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:40 smithi083 ceph-mon[36260]: 1.3 scrub starts 2024-09-20T19:09:40.546 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:40 smithi083 ceph-mon[36260]: 1.3 scrub ok 2024-09-20T19:09:40.547 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_blocked_ops 2024-09-20T19:09:40.559 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:09:40.566 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:09:40.590 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:40.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:40.592 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:09:40.692 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:09:40.714 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:40.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:40.715 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:09:40.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:40 smithi142 ceph-mon[32508]: pgmap v1010: 168 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 1 active+clean+remapped, 2 active+undersized+remapped+backfill_toofull, 36 active+undersized, 2 active+remapped+backfill_toofull, 20 active+undersized+degraded, 106 active+clean; 1.7 MiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 976 op/s; 59/657 objects degraded (8.980%); 21/657 objects misplaced (3.196%); 1.2 MiB/s, 18 objects/s recovering 2024-09-20T19:09:40.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:40 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 59/657 objects degraded (8.980%), 21 pgs degraded (PG_DEGRADED) 2024-09-20T19:09:40.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:40 smithi142 ceph-mon[32508]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T19:09:40.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:40 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/4040114718' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T19:09:40.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:40 smithi142 ceph-mon[32508]: osdmap e500: 8 total, 7 up, 8 in 2024-09-20T19:09:40.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:40 smithi142 ceph-mon[32508]: 1.3 scrub starts 2024-09-20T19:09:40.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:40 smithi142 ceph-mon[32508]: 1.3 scrub ok 2024-09-20T19:09:40.790 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:09:40 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:09:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:09:40.815 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:09:40.842 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:40.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:40.843 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:09:40.946 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:09:40.970 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:40.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:40.971 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:09:41.071 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:09:41.094 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:41.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:41.095 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:09:41.196 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:09:41.218 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:41.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:41.219 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:09:41.320 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:09:41.342 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:41.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:41.343 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:09:41.443 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:09:41.454 INFO:teuthology.orchestra.run.smithi083.stderr:Creating an OSD daemon form without an OSD FSID value 2024-09-20T19:09:41.467 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:41.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:41.468 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:09:41.568 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:09:41.591 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:41.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:41.592 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:09:41.597 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:09:41.693 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:09:41.715 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:41.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:41.716 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:09:41.817 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:09:41.840 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:41.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:41.841 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:09:41.942 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3 2024-09-20T19:09:41.963 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:41.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:41.964 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.3' 2024-09-20T19:09:41.984 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'unique_pool_4' removed 2024-09-20T19:09:42.065 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:09:42.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:42.088 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:42.088 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:09:42.175 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T19:09:42.189 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:09:42.240 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:42.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:42.241 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:09:42.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:41 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2690078088' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true} : dispatch 2024-09-20T19:09:42.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:41 smithi142 ceph-mon[32508]: pgmap v1012: 168 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 1 active+clean+remapped, 2 active+undersized+remapped+backfill_toofull, 38 active+undersized, 2 active+remapped+backfill_toofull, 18 active+undersized+degraded, 106 active+clean; 1.7 MiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 667 op/s; 59/657 objects degraded (8.980%); 21/657 objects misplaced (3.196%); 1.1 MiB/s, 16 objects/s recovering 2024-09-20T19:09:42.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:41 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2690078088' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true} : dispatch 2024-09-20T19:09:42.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:41 smithi083 ceph-mon[29421]: pgmap v1012: 168 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 1 active+clean+remapped, 2 active+undersized+remapped+backfill_toofull, 38 active+undersized, 2 active+remapped+backfill_toofull, 18 active+undersized+degraded, 106 active+clean; 1.7 MiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 667 op/s; 59/657 objects degraded (8.980%); 21/657 objects misplaced (3.196%); 1.1 MiB/s, 16 objects/s recovering 2024-09-20T19:09:42.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:41 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2690078088' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true} : dispatch 2024-09-20T19:09:42.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:41 smithi083 ceph-mon[36260]: pgmap v1012: 168 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 1 active+clean+remapped, 2 active+undersized+remapped+backfill_toofull, 38 active+undersized, 2 active+remapped+backfill_toofull, 18 active+undersized+degraded, 106 active+clean; 1.7 MiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 667 op/s; 59/657 objects degraded (8.980%); 21/657 objects misplaced (3.196%); 1.1 MiB/s, 16 objects/s recovering 2024-09-20T19:09:42.342 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:09:42.365 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:42.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:42.366 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:09:42.467 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7 2024-09-20T19:09:42.495 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:42.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:42.496 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.7' 2024-09-20T19:09:42.597 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1 2024-09-20T19:09:42.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:42.621 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:42.621 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.1' 2024-09-20T19:09:42.722 DEBUG:teuthology.orchestra.run.smithi083:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2 2024-09-20T19:09:42.743 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: docker: command not found 2024-09-20T19:09:42.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:42.744 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi083 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.2' 2024-09-20T19:09:42.845 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:09:42.866 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:42.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:42.867 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:09:42.968 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5 2024-09-20T19:09:42.990 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:42.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:42.991 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.5' 2024-09-20T19:09:43.092 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4 2024-09-20T19:09:43.113 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:43.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:43.114 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.4' 2024-09-20T19:09:43.215 DEBUG:teuthology.orchestra.run.smithi142:> sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6 2024-09-20T19:09:43.219 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:42 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2690078088' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]': finished 2024-09-20T19:09:43.219 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:42 smithi083 ceph-mon[29421]: osdmap e501: 8 total, 7 up, 8 in 2024-09-20T19:09:43.219 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:42 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2690078088' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]': finished 2024-09-20T19:09:43.219 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:42 smithi083 ceph-mon[36260]: osdmap e501: 8 total, 7 up, 8 in 2024-09-20T19:09:43.236 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: docker: command not found 2024-09-20T19:09:43.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:09:43.237 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi142 with status 1: 'sudo docker kill -s 1 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd.6' 2024-09-20T19:09:43.246 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-20T19:09:43.258 INFO:tasks.thrashosds:joining thrashosds 2024-09-20T19:09:43.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:42 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2690078088' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]': finished 2024-09-20T19:09:43.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:42 smithi142 ceph-mon[32508]: osdmap e501: 8 total, 7 up, 8 in 2024-09-20T19:09:43.334 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:09:43.758 INFO:teuthology.orchestra.run.smithi083.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-20T19:09:43.776 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1: {} 2024-09-20T19:09:43.776 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T19:09:43.794 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2: {} 2024-09-20T19:09:43.794 INFO:teuthology.orchestra.run.smithi083.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T19:09:43.810 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3: {} 2024-09-20T19:09:43.811 INFO:teuthology.orchestra.run.smithi083.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T19:09:43.840 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4: {} 2024-09-20T19:09:43.841 INFO:teuthology.orchestra.run.smithi083.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T19:09:43.870 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5: {} 2024-09-20T19:09:43.871 INFO:teuthology.orchestra.run.smithi083.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T19:09:43.887 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6: {} 2024-09-20T19:09:43.887 INFO:teuthology.orchestra.run.smithi083.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T19:09:43.912 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7: {} 2024-09-20T19:09:43.912 INFO:teuthology.orchestra.run.smithi083.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T19:09:44.016 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T19:09:44.016 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:09:44.243 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:43 smithi083 ceph-mon[29421]: 1.9 scrub starts 2024-09-20T19:09:44.244 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:43 smithi083 ceph-mon[29421]: 1.9 scrub ok 2024-09-20T19:09:44.244 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:43 smithi083 ceph-mon[29421]: pgmap v1014: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 431 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 59/594 objects degraded (9.933%) 2024-09-20T19:09:44.244 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:43 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/203864733' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:09:44.244 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:43 smithi083 ceph-mon[36260]: 1.9 scrub starts 2024-09-20T19:09:44.244 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:43 smithi083 ceph-mon[36260]: 1.9 scrub ok 2024-09-20T19:09:44.244 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:43 smithi083 ceph-mon[36260]: pgmap v1014: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 431 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 59/594 objects degraded (9.933%) 2024-09-20T19:09:44.244 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:43 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/203864733' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:09:44.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:43 smithi142 ceph-mon[32508]: 1.9 scrub starts 2024-09-20T19:09:44.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:43 smithi142 ceph-mon[32508]: 1.9 scrub ok 2024-09-20T19:09:44.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:43 smithi142 ceph-mon[32508]: pgmap v1014: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 431 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 59/594 objects degraded (9.933%) 2024-09-20T19:09:44.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:43 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/203864733' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-20T19:09:44.439 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:09:44.978 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_historic_ops 2024-09-20T19:09:45.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:44 smithi142 ceph-mon[32508]: 4.14 scrub starts 2024-09-20T19:09:45.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:44 smithi142 ceph-mon[32508]: 4.14 scrub ok 2024-09-20T19:09:45.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:44 smithi142 ceph-mon[32508]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 5 pgs backfill_toofull) 2024-09-20T19:09:45.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:44 smithi083 ceph-mon[29421]: 4.14 scrub starts 2024-09-20T19:09:45.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:44 smithi083 ceph-mon[29421]: 4.14 scrub ok 2024-09-20T19:09:45.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:44 smithi083 ceph-mon[29421]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 5 pgs backfill_toofull) 2024-09-20T19:09:45.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:44 smithi083 ceph-mon[36260]: 4.14 scrub starts 2024-09-20T19:09:45.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:44 smithi083 ceph-mon[36260]: 4.14 scrub ok 2024-09-20T19:09:45.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:44 smithi083 ceph-mon[36260]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 5 pgs backfill_toofull) 2024-09-20T19:09:45.552 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-20T19:09:45.553 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-20T19:09:46.072 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:46 smithi083 ceph-mon[29421]: pgmap v1015: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 431 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 511 B/s rd, 0 op/s; 59/594 objects degraded (9.933%) 2024-09-20T19:09:46.072 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:46 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 59/594 objects degraded (9.933%), 16 pgs degraded (PG_DEGRADED) 2024-09-20T19:09:46.072 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:46 smithi083 ceph-mon[36260]: pgmap v1015: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 431 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 511 B/s rd, 0 op/s; 59/594 objects degraded (9.933%) 2024-09-20T19:09:46.072 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:46 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 59/594 objects degraded (9.933%), 16 pgs degraded (PG_DEGRADED) 2024-09-20T19:09:46.111 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:09:46.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:46 smithi142 ceph-mon[32508]: pgmap v1015: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 431 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 511 B/s rd, 0 op/s; 59/594 objects degraded (9.933%) 2024-09-20T19:09:46.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:46 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 59/594 objects degraded (9.933%), 16 pgs degraded (PG_DEGRADED) 2024-09-20T19:09:46.651 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T19:09:46.651 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:09:46.931 INFO:teuthology.orchestra.run.smithi083.stdout:6.d 2024-09-20T19:09:46.931 INFO:teuthology.orchestra.run.smithi083.stdout:6.a 2024-09-20T19:09:46.931 INFO:teuthology.orchestra.run.smithi083.stdout:3.7 2024-09-20T19:09:46.931 INFO:teuthology.orchestra.run.smithi083.stdout:1.6 2024-09-20T19:09:46.931 INFO:teuthology.orchestra.run.smithi083.stdout:3.6 2024-09-20T19:09:46.931 INFO:teuthology.orchestra.run.smithi083.stdout:1.0 2024-09-20T19:09:46.931 INFO:teuthology.orchestra.run.smithi083.stdout:3.12 2024-09-20T19:09:46.931 INFO:teuthology.orchestra.run.smithi083.stdout:6.11 2024-09-20T19:09:46.931 INFO:teuthology.orchestra.run.smithi083.stdout:3.1b 2024-09-20T19:09:46.931 INFO:teuthology.orchestra.run.smithi083.stdout:2.0 2024-09-20T19:09:46.931 INFO:teuthology.orchestra.run.smithi083.stdout:5.13 2024-09-20T19:09:46.932 INFO:teuthology.orchestra.run.smithi083.stdout:4.b 2024-09-20T19:09:46.932 INFO:teuthology.orchestra.run.smithi083.stdout:5.3 2024-09-20T19:09:46.932 INFO:teuthology.orchestra.run.smithi083.stdout:1.1 2024-09-20T19:09:46.932 INFO:teuthology.orchestra.run.smithi083.stdout:11.1 2024-09-20T19:09:46.932 INFO:teuthology.orchestra.run.smithi083.stdout:3.18 2024-09-20T19:09:46.932 INFO:teuthology.orchestra.run.smithi083.stdout:3.3 2024-09-20T19:09:46.932 INFO:teuthology.orchestra.run.smithi083.stdout:5.2 2024-09-20T19:09:46.932 INFO:teuthology.orchestra.run.smithi083.stdout:1.5 2024-09-20T19:09:46.932 INFO:teuthology.orchestra.run.smithi083.stdout:3.17 2024-09-20T19:09:46.932 INFO:teuthology.orchestra.run.smithi083.stdout:11.10 2024-09-20T19:09:46.932 INFO:teuthology.orchestra.run.smithi083.stdout:11.b 2024-09-20T19:09:46.932 INFO:teuthology.orchestra.run.smithi083.stdout:3.1f 2024-09-20T19:09:46.932 INFO:teuthology.orchestra.run.smithi083.stdout:6.9 2024-09-20T19:09:46.932 INFO:teuthology.orchestra.run.smithi083.stdout:4.0 2024-09-20T19:09:46.932 INFO:teuthology.orchestra.run.smithi083.stdout:3.f 2024-09-20T19:09:46.933 INFO:teuthology.orchestra.run.smithi083.stdout:11.16 2024-09-20T19:09:46.933 INFO:teuthology.orchestra.run.smithi083.stdout:11.13 2024-09-20T19:09:46.933 INFO:teuthology.orchestra.run.smithi083.stdout:11.9 2024-09-20T19:09:46.933 INFO:teuthology.orchestra.run.smithi083.stdout:3.10 2024-09-20T19:09:46.933 INFO:teuthology.orchestra.run.smithi083.stdout:6.0 2024-09-20T19:09:46.933 INFO:teuthology.orchestra.run.smithi083.stdout:1.2 2024-09-20T19:09:46.933 INFO:teuthology.orchestra.run.smithi083.stdout:2.1 2024-09-20T19:09:46.933 INFO:teuthology.orchestra.run.smithi083.stdout:11.19 2024-09-20T19:09:46.933 INFO:teuthology.orchestra.run.smithi083.stdout:3.b 2024-09-20T19:09:46.933 INFO:teuthology.orchestra.run.smithi083.stdout:11.2 2024-09-20T19:09:46.933 INFO:teuthology.orchestra.run.smithi083.stdout:11.11 2024-09-20T19:09:46.933 INFO:teuthology.orchestra.run.smithi083.stdout:6.16 2024-09-20T19:09:46.933 INFO:teuthology.orchestra.run.smithi083.stdout:4.19 2024-09-20T19:09:46.933 INFO:teuthology.orchestra.run.smithi083.stdout:4.16 2024-09-20T19:09:46.933 INFO:teuthology.orchestra.run.smithi083.stdout:4.1a 2024-09-20T19:09:46.934 INFO:teuthology.orchestra.run.smithi083.stdout:4.1e 2024-09-20T19:09:46.934 INFO:teuthology.orchestra.run.smithi083.stdout:4.3 2024-09-20T19:09:46.934 INFO:teuthology.orchestra.run.smithi083.stdout:6.13 2024-09-20T19:09:46.934 INFO:teuthology.orchestra.run.smithi083.stdout:4.6 2024-09-20T19:09:46.934 INFO:teuthology.orchestra.run.smithi083.stdout:4.7 2024-09-20T19:09:46.934 INFO:teuthology.orchestra.run.smithi083.stdout:5.15 2024-09-20T19:09:46.934 INFO:teuthology.orchestra.run.smithi083.stdout:5.7 2024-09-20T19:09:46.934 INFO:teuthology.orchestra.run.smithi083.stdout:3.1 2024-09-20T19:09:46.934 INFO:teuthology.orchestra.run.smithi083.stdout:5.1b 2024-09-20T19:09:46.934 INFO:teuthology.orchestra.run.smithi083.stdout:11.6 2024-09-20T19:09:46.934 INFO:teuthology.orchestra.run.smithi083.stdout:5.1d 2024-09-20T19:09:46.934 INFO:teuthology.orchestra.run.smithi083.stdout:5.b 2024-09-20T19:09:46.934 INFO:teuthology.orchestra.run.smithi083.stdout:5.1e 2024-09-20T19:09:46.934 INFO:teuthology.orchestra.run.smithi083.stdout:11.12 2024-09-20T19:09:46.934 INFO:teuthology.orchestra.run.smithi083.stdout:5.5 2024-09-20T19:09:46.935 INFO:teuthology.orchestra.run.smithi083.stdout:5.8 2024-09-20T19:09:46.935 INFO:teuthology.orchestra.run.smithi083.stdout:5.c 2024-09-20T19:09:46.935 INFO:teuthology.orchestra.run.smithi083.stdout:5.e 2024-09-20T19:09:46.935 INFO:teuthology.orchestra.run.smithi083.stdout:6.10 2024-09-20T19:09:46.935 INFO:teuthology.orchestra.run.smithi083.stdout:6.18 2024-09-20T19:09:47.024 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-20T19:09:47.253 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:47 smithi083 ceph-mon[29421]: 6.15 scrub starts 2024-09-20T19:09:47.253 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:47 smithi083 ceph-mon[29421]: 6.15 scrub ok 2024-09-20T19:09:47.253 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:47 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1631048786' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:09:47.253 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:47 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:09:47.253 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:47 smithi083 ceph-mon[29421]: 3.a scrub starts 2024-09-20T19:09:47.254 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:47 smithi083 ceph-mon[29421]: 3.a scrub ok 2024-09-20T19:09:47.254 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:47 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:09:47.254 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:47 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:09:47.254 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:47 smithi083 ceph-mon[36260]: 6.15 scrub starts 2024-09-20T19:09:47.254 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:47 smithi083 ceph-mon[36260]: 6.15 scrub ok 2024-09-20T19:09:47.254 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:47 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1631048786' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:09:47.254 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:47 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:09:47.254 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:47 smithi083 ceph-mon[36260]: 3.a scrub starts 2024-09-20T19:09:47.254 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:47 smithi083 ceph-mon[36260]: 3.a scrub ok 2024-09-20T19:09:47.254 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:47 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:09:47.254 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:47 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:09:47.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:47 smithi142 ceph-mon[32508]: 6.15 scrub starts 2024-09-20T19:09:47.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:47 smithi142 ceph-mon[32508]: 6.15 scrub ok 2024-09-20T19:09:47.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:47 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1631048786' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:09:47.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:47 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:09:47.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:47 smithi142 ceph-mon[32508]: 3.a scrub starts 2024-09-20T19:09:47.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:47 smithi142 ceph-mon[32508]: 3.a scrub ok 2024-09-20T19:09:47.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:47 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:09:47.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:47 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:09:47.442 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-20T19:09:47.889 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:09:48.010 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:09:47 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:09:48.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:48 smithi083 ceph-mon[29421]: 5.0 scrub starts 2024-09-20T19:09:48.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:48 smithi083 ceph-mon[29421]: 5.0 scrub ok 2024-09-20T19:09:48.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:48 smithi083 ceph-mon[29421]: 5.6 scrub starts 2024-09-20T19:09:48.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:48 smithi083 ceph-mon[29421]: 5.6 scrub ok 2024-09-20T19:09:48.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:48 smithi083 ceph-mon[29421]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T19:09:48.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:48 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:09:48.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:48 smithi083 ceph-mon[29421]: osdmap e502: 8 total, 7 up, 8 in 2024-09-20T19:09:48.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:48 smithi083 ceph-mon[29421]: pgmap v1017: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 378 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 59/594 objects degraded (9.933%) 2024-09-20T19:09:48.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:48 smithi083 ceph-mon[29421]: 3.0 scrub starts 2024-09-20T19:09:48.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:48 smithi083 ceph-mon[29421]: 3.0 scrub ok 2024-09-20T19:09:48.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:48 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:09:48.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:48 smithi083 ceph-mon[29421]: 1.7 scrub starts 2024-09-20T19:09:48.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:48 smithi083 ceph-mon[29421]: 1.7 scrub ok 2024-09-20T19:09:48.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:48 smithi083 ceph-mon[36260]: 5.0 scrub starts 2024-09-20T19:09:48.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:48 smithi083 ceph-mon[36260]: 5.0 scrub ok 2024-09-20T19:09:48.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:48 smithi083 ceph-mon[36260]: 5.6 scrub starts 2024-09-20T19:09:48.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:48 smithi083 ceph-mon[36260]: 5.6 scrub ok 2024-09-20T19:09:48.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:48 smithi083 ceph-mon[36260]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T19:09:48.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:48 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:09:48.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:48 smithi083 ceph-mon[36260]: osdmap e502: 8 total, 7 up, 8 in 2024-09-20T19:09:48.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:48 smithi083 ceph-mon[36260]: pgmap v1017: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 378 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 59/594 objects degraded (9.933%) 2024-09-20T19:09:48.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:48 smithi083 ceph-mon[36260]: 3.0 scrub starts 2024-09-20T19:09:48.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:48 smithi083 ceph-mon[36260]: 3.0 scrub ok 2024-09-20T19:09:48.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:48 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:09:48.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:48 smithi083 ceph-mon[36260]: 1.7 scrub starts 2024-09-20T19:09:48.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:48 smithi083 ceph-mon[36260]: 1.7 scrub ok 2024-09-20T19:09:48.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:48 smithi142 ceph-mon[32508]: 5.0 scrub starts 2024-09-20T19:09:48.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:48 smithi142 ceph-mon[32508]: 5.0 scrub ok 2024-09-20T19:09:48.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:48 smithi142 ceph-mon[32508]: 5.6 scrub starts 2024-09-20T19:09:48.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:48 smithi142 ceph-mon[32508]: 5.6 scrub ok 2024-09-20T19:09:48.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:48 smithi142 ceph-mon[32508]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T19:09:48.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:48 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:09:48.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:48 smithi142 ceph-mon[32508]: osdmap e502: 8 total, 7 up, 8 in 2024-09-20T19:09:48.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:48 smithi142 ceph-mon[32508]: pgmap v1017: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 378 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 59/594 objects degraded (9.933%) 2024-09-20T19:09:48.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:48 smithi142 ceph-mon[32508]: 3.0 scrub starts 2024-09-20T19:09:48.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:48 smithi142 ceph-mon[32508]: 3.0 scrub ok 2024-09-20T19:09:48.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:48 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:09:48.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:48 smithi142 ceph-mon[32508]: 1.7 scrub starts 2024-09-20T19:09:48.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:48 smithi142 ceph-mon[32508]: 1.7 scrub ok 2024-09-20T19:09:48.884 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T19:09:48.885 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:09:49.040 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:49 smithi083 ceph-mon[36260]: 6.8 scrub starts 2024-09-20T19:09:49.040 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:49 smithi083 ceph-mon[36260]: 6.8 scrub ok 2024-09-20T19:09:49.040 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:49 smithi083 ceph-mon[36260]: 4.11 scrub starts 2024-09-20T19:09:49.040 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:49 smithi083 ceph-mon[36260]: 4.11 scrub ok 2024-09-20T19:09:49.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:49 smithi083 ceph-mon[29421]: 6.8 scrub starts 2024-09-20T19:09:49.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:49 smithi083 ceph-mon[29421]: 6.8 scrub ok 2024-09-20T19:09:49.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:49 smithi083 ceph-mon[29421]: 4.11 scrub starts 2024-09-20T19:09:49.041 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:49 smithi083 ceph-mon[29421]: 4.11 scrub ok 2024-09-20T19:09:49.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:49 smithi142 ceph-mon[32508]: 6.8 scrub starts 2024-09-20T19:09:49.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:49 smithi142 ceph-mon[32508]: 6.8 scrub ok 2024-09-20T19:09:49.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:49 smithi142 ceph-mon[32508]: 4.11 scrub starts 2024-09-20T19:09:49.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:49 smithi142 ceph-mon[32508]: 4.11 scrub ok 2024-09-20T19:09:49.523 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell -n osd.0 --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --op export --pgid 1.2 --file /var/log/ceph/exp.1.2.0 2024-09-20T19:09:49.967 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_blocked_ops 2024-09-20T19:09:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/osd.0/config 2024-09-20T19:09:50.275 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:50 smithi083 ceph-mon[29421]: pgmap v1018: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 378 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 59/594 objects degraded (9.933%) 2024-09-20T19:09:50.275 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:50 smithi083 ceph-mon[29421]: 5.12 scrub starts 2024-09-20T19:09:50.275 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:50 smithi083 ceph-mon[29421]: 5.12 scrub ok 2024-09-20T19:09:50.276 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:50 smithi083 ceph-mon[36260]: pgmap v1018: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 378 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 59/594 objects degraded (9.933%) 2024-09-20T19:09:50.276 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:50 smithi083 ceph-mon[36260]: 5.12 scrub starts 2024-09-20T19:09:50.276 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:50 smithi083 ceph-mon[36260]: 5.12 scrub ok 2024-09-20T19:09:50.441 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T19:09:50.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:50 smithi142 ceph-mon[32508]: pgmap v1018: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 378 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 59/594 objects degraded (9.933%) 2024-09-20T19:09:50.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:50 smithi142 ceph-mon[32508]: 5.12 scrub starts 2024-09-20T19:09:50.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:50 smithi142 ceph-mon[32508]: 5.12 scrub ok 2024-09-20T19:09:50.708 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:09:50.977 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:09:50 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:09:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:09:51.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:51 smithi083 ceph-mon[36260]: 3.13 scrub starts 2024-09-20T19:09:51.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:51 smithi083 ceph-mon[36260]: 3.13 scrub ok 2024-09-20T19:09:51.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:51 smithi083 ceph-mon[36260]: 6.1b scrub starts 2024-09-20T19:09:51.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:51 smithi083 ceph-mon[36260]: 6.1b scrub ok 2024-09-20T19:09:51.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:51 smithi083 ceph-mon[36260]: 1.8 scrub starts 2024-09-20T19:09:51.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:51 smithi083 ceph-mon[36260]: 1.8 scrub ok 2024-09-20T19:09:51.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:51 smithi083 ceph-mon[29421]: 3.13 scrub starts 2024-09-20T19:09:51.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:51 smithi083 ceph-mon[29421]: 3.13 scrub ok 2024-09-20T19:09:51.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:51 smithi083 ceph-mon[29421]: 6.1b scrub starts 2024-09-20T19:09:51.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:51 smithi083 ceph-mon[29421]: 6.1b scrub ok 2024-09-20T19:09:51.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:51 smithi083 ceph-mon[29421]: 1.8 scrub starts 2024-09-20T19:09:51.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:51 smithi083 ceph-mon[29421]: 1.8 scrub ok 2024-09-20T19:09:51.475 INFO:teuthology.orchestra.run.smithi083.stderr:Creating an OSD daemon form without an OSD FSID value 2024-09-20T19:09:51.478 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:09:51.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:51 smithi142 ceph-mon[32508]: 3.13 scrub starts 2024-09-20T19:09:51.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:51 smithi142 ceph-mon[32508]: 3.13 scrub ok 2024-09-20T19:09:51.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:51 smithi142 ceph-mon[32508]: 6.1b scrub starts 2024-09-20T19:09:51.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:51 smithi142 ceph-mon[32508]: 6.1b scrub ok 2024-09-20T19:09:51.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:51 smithi142 ceph-mon[32508]: 1.8 scrub starts 2024-09-20T19:09:51.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:51 smithi142 ceph-mon[32508]: 1.8 scrub ok 2024-09-20T19:09:51.702 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T19:09:51.703 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:09:52.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:52 smithi083 ceph-mon[29421]: 3.e scrub starts 2024-09-20T19:09:52.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:52 smithi083 ceph-mon[29421]: 3.e scrub ok 2024-09-20T19:09:52.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:52 smithi083 ceph-mon[29421]: pgmap v1019: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 59/597 objects degraded (9.883%) 2024-09-20T19:09:52.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:52 smithi083 ceph-mon[36260]: 3.e scrub starts 2024-09-20T19:09:52.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:52 smithi083 ceph-mon[36260]: 3.e scrub ok 2024-09-20T19:09:52.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:52 smithi083 ceph-mon[36260]: pgmap v1019: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 59/597 objects degraded (9.883%) 2024-09-20T19:09:52.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:52 smithi142 ceph-mon[32508]: 3.e scrub starts 2024-09-20T19:09:52.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:52 smithi142 ceph-mon[32508]: 3.e scrub ok 2024-09-20T19:09:52.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:52 smithi142 ceph-mon[32508]: pgmap v1019: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 59/597 objects degraded (9.883%) 2024-09-20T19:09:53.047 INFO:teuthology.orchestra.run.smithi083.stderr:noscrub is unset 2024-09-20T19:09:53.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:53 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 59/597 objects degraded (9.883%), 16 pgs degraded (PG_DEGRADED) 2024-09-20T19:09:53.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:53 smithi083 ceph-mon[29421]: 5.19 scrub starts 2024-09-20T19:09:53.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:53 smithi083 ceph-mon[29421]: 5.19 scrub ok 2024-09-20T19:09:53.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:53 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/985077291' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:09:53.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:53 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:09:53.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:53 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 59/597 objects degraded (9.883%), 16 pgs degraded (PG_DEGRADED) 2024-09-20T19:09:53.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:53 smithi083 ceph-mon[36260]: 5.19 scrub starts 2024-09-20T19:09:53.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:53 smithi083 ceph-mon[36260]: 5.19 scrub ok 2024-09-20T19:09:53.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:53 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/985077291' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:09:53.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:53 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:09:53.328 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_historic_ops 2024-09-20T19:09:53.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:53 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 59/597 objects degraded (9.883%), 16 pgs degraded (PG_DEGRADED) 2024-09-20T19:09:53.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:53 smithi142 ceph-mon[32508]: 5.19 scrub starts 2024-09-20T19:09:53.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:53 smithi142 ceph-mon[32508]: 5.19 scrub ok 2024-09-20T19:09:53.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:53 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/985077291' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:09:53.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:53 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-20T19:09:54.090 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T19:09:54.143 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:09:54.300 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:54 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T19:09:54.300 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:54 smithi083 ceph-mon[29421]: 4.f scrub starts 2024-09-20T19:09:54.300 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:54 smithi083 ceph-mon[29421]: 4.f scrub ok 2024-09-20T19:09:54.300 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:54 smithi083 ceph-mon[29421]: osdmap e503: 8 total, 7 up, 8 in 2024-09-20T19:09:54.300 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:54 smithi083 ceph-mon[29421]: pgmap v1021: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 59/597 objects degraded (9.883%) 2024-09-20T19:09:54.301 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:54 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T19:09:54.301 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:54 smithi083 ceph-mon[36260]: 4.f scrub starts 2024-09-20T19:09:54.301 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:54 smithi083 ceph-mon[36260]: 4.f scrub ok 2024-09-20T19:09:54.301 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:54 smithi083 ceph-mon[36260]: osdmap e503: 8 total, 7 up, 8 in 2024-09-20T19:09:54.301 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:54 smithi083 ceph-mon[36260]: pgmap v1021: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 59/597 objects degraded (9.883%) 2024-09-20T19:09:54.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:54 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T19:09:54.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:54 smithi142 ceph-mon[32508]: 4.f scrub starts 2024-09-20T19:09:54.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:54 smithi142 ceph-mon[32508]: 4.f scrub ok 2024-09-20T19:09:54.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:54 smithi142 ceph-mon[32508]: osdmap e503: 8 total, 7 up, 8 in 2024-09-20T19:09:54.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:54 smithi142 ceph-mon[32508]: pgmap v1021: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 59/597 objects degraded (9.883%) 2024-09-20T19:09:54.680 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T19:09:54.680 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:09:54.883 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:09:55.301 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:55 smithi083 ceph-mon[29421]: 6.3 scrub starts 2024-09-20T19:09:55.301 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:55 smithi083 ceph-mon[29421]: 6.3 scrub ok 2024-09-20T19:09:55.301 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:55 smithi083 ceph-mon[29421]: 5.1c scrub starts 2024-09-20T19:09:55.301 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:55 smithi083 ceph-mon[29421]: 5.1c scrub ok 2024-09-20T19:09:55.301 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:55 smithi083 ceph-mon[29421]: 6.6 scrub starts 2024-09-20T19:09:55.301 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:55 smithi083 ceph-mon[29421]: 6.6 scrub ok 2024-09-20T19:09:55.301 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:55 smithi083 ceph-mon[29421]: 6.14 scrub starts 2024-09-20T19:09:55.301 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:55 smithi083 ceph-mon[29421]: 6.14 scrub ok 2024-09-20T19:09:55.301 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:55 smithi083 ceph-mon[29421]: 5.f scrub starts 2024-09-20T19:09:55.301 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:55 smithi083 ceph-mon[29421]: 5.f scrub ok 2024-09-20T19:09:55.302 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:55 smithi083 ceph-mon[36260]: 6.3 scrub starts 2024-09-20T19:09:55.302 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:55 smithi083 ceph-mon[36260]: 6.3 scrub ok 2024-09-20T19:09:55.302 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:55 smithi083 ceph-mon[36260]: 5.1c scrub starts 2024-09-20T19:09:55.302 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:55 smithi083 ceph-mon[36260]: 5.1c scrub ok 2024-09-20T19:09:55.302 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:55 smithi083 ceph-mon[36260]: 6.6 scrub starts 2024-09-20T19:09:55.302 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:55 smithi083 ceph-mon[36260]: 6.6 scrub ok 2024-09-20T19:09:55.302 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:55 smithi083 ceph-mon[36260]: 6.14 scrub starts 2024-09-20T19:09:55.302 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:55 smithi083 ceph-mon[36260]: 6.14 scrub ok 2024-09-20T19:09:55.302 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:55 smithi083 ceph-mon[36260]: 5.f scrub starts 2024-09-20T19:09:55.302 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:55 smithi083 ceph-mon[36260]: 5.f scrub ok 2024-09-20T19:09:55.394 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-20T19:09:55.411 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:55 smithi142 ceph-mon[32508]: 6.3 scrub starts 2024-09-20T19:09:55.412 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:55 smithi142 ceph-mon[32508]: 6.3 scrub ok 2024-09-20T19:09:55.412 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:55 smithi142 ceph-mon[32508]: 5.1c scrub starts 2024-09-20T19:09:55.412 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:55 smithi142 ceph-mon[32508]: 5.1c scrub ok 2024-09-20T19:09:55.412 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:55 smithi142 ceph-mon[32508]: 6.6 scrub starts 2024-09-20T19:09:55.412 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:55 smithi142 ceph-mon[32508]: 6.6 scrub ok 2024-09-20T19:09:55.412 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:55 smithi142 ceph-mon[32508]: 6.14 scrub starts 2024-09-20T19:09:55.412 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:55 smithi142 ceph-mon[32508]: 6.14 scrub ok 2024-09-20T19:09:55.412 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:55 smithi142 ceph-mon[32508]: 5.f scrub starts 2024-09-20T19:09:55.412 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:55 smithi142 ceph-mon[32508]: 5.f scrub ok 2024-09-20T19:09:55.946 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:09:56.021 INFO:teuthology.orchestra.run.smithi083.stderr:Exporting 1.2 info 1.2( v 74'79 (0'0,74'79] local-lis/les=458/459 n=1 ec=455/19 lis/c=458/458 les/c/f=459/459/0 sis=458) 2024-09-20T19:09:56.021 INFO:teuthology.orchestra.run.smithi083.stderr:Export successful 2024-09-20T19:09:56.087 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:56 smithi083 ceph-mon[29421]: 3.1e scrub starts 2024-09-20T19:09:56.087 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:56 smithi083 ceph-mon[29421]: 3.1e scrub ok 2024-09-20T19:09:56.087 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:56 smithi083 ceph-mon[29421]: 5.4 scrub starts 2024-09-20T19:09:56.087 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:56 smithi083 ceph-mon[29421]: 5.4 scrub ok 2024-09-20T19:09:56.088 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:56 smithi083 ceph-mon[29421]: pgmap v1022: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 59/597 objects degraded (9.883%) 2024-09-20T19:09:56.088 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:56 smithi083 ceph-mon[29421]: 4.4 scrub starts 2024-09-20T19:09:56.088 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:56 smithi083 ceph-mon[29421]: 4.4 scrub ok 2024-09-20T19:09:56.088 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:56 smithi083 ceph-mon[29421]: 4.5 scrub starts 2024-09-20T19:09:56.088 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:56 smithi083 ceph-mon[29421]: 4.5 scrub ok 2024-09-20T19:09:56.089 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:56 smithi083 ceph-mon[36260]: 3.1e scrub starts 2024-09-20T19:09:56.089 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:56 smithi083 ceph-mon[36260]: 3.1e scrub ok 2024-09-20T19:09:56.089 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:56 smithi083 ceph-mon[36260]: 5.4 scrub starts 2024-09-20T19:09:56.089 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:56 smithi083 ceph-mon[36260]: 5.4 scrub ok 2024-09-20T19:09:56.089 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:56 smithi083 ceph-mon[36260]: pgmap v1022: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 59/597 objects degraded (9.883%) 2024-09-20T19:09:56.089 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:56 smithi083 ceph-mon[36260]: 4.4 scrub starts 2024-09-20T19:09:56.089 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:56 smithi083 ceph-mon[36260]: 4.4 scrub ok 2024-09-20T19:09:56.089 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:56 smithi083 ceph-mon[36260]: 4.5 scrub starts 2024-09-20T19:09:56.089 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:56 smithi083 ceph-mon[36260]: 4.5 scrub ok 2024-09-20T19:09:56.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:56 smithi142 ceph-mon[32508]: 3.1e scrub starts 2024-09-20T19:09:56.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:56 smithi142 ceph-mon[32508]: 3.1e scrub ok 2024-09-20T19:09:56.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:56 smithi142 ceph-mon[32508]: 5.4 scrub starts 2024-09-20T19:09:56.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:56 smithi142 ceph-mon[32508]: 5.4 scrub ok 2024-09-20T19:09:56.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:56 smithi142 ceph-mon[32508]: pgmap v1022: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 59/597 objects degraded (9.883%) 2024-09-20T19:09:56.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:56 smithi142 ceph-mon[32508]: 4.4 scrub starts 2024-09-20T19:09:56.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:56 smithi142 ceph-mon[32508]: 4.4 scrub ok 2024-09-20T19:09:56.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:56 smithi142 ceph-mon[32508]: 4.5 scrub starts 2024-09-20T19:09:56.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:56 smithi142 ceph-mon[32508]: 4.5 scrub ok 2024-09-20T19:09:56.456 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T19:09:56.456 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:09:57.059 INFO:teuthology.orchestra.run.smithi083.stderr:nodeep-scrub is unset 2024-09-20T19:09:57.072 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:57 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2541017563' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:09:57.072 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:57 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:09:57.072 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:57 smithi083 ceph-mon[29421]: 3.4 scrub starts 2024-09-20T19:09:57.072 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:57 smithi083 ceph-mon[29421]: 3.4 scrub ok 2024-09-20T19:09:57.072 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:57 smithi083 ceph-mon[29421]: 3.1d scrub starts 2024-09-20T19:09:57.072 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:57 smithi083 ceph-mon[29421]: 3.1d scrub ok 2024-09-20T19:09:57.072 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:57 smithi083 ceph-mon[29421]: 4.a scrub starts 2024-09-20T19:09:57.072 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:57 smithi083 ceph-mon[29421]: 4.a scrub ok 2024-09-20T19:09:57.072 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:57 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2541017563' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:09:57.073 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:57 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:09:57.073 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:57 smithi083 ceph-mon[36260]: 3.4 scrub starts 2024-09-20T19:09:57.073 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:57 smithi083 ceph-mon[36260]: 3.4 scrub ok 2024-09-20T19:09:57.073 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:57 smithi083 ceph-mon[36260]: 3.1d scrub starts 2024-09-20T19:09:57.073 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:57 smithi083 ceph-mon[36260]: 3.1d scrub ok 2024-09-20T19:09:57.073 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:57 smithi083 ceph-mon[36260]: 4.a scrub starts 2024-09-20T19:09:57.073 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:57 smithi083 ceph-mon[36260]: 4.a scrub ok 2024-09-20T19:09:57.148 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_blocked_ops 2024-09-20T19:09:57.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:57 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2541017563' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:09:57.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:57 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-20T19:09:57.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:57 smithi142 ceph-mon[32508]: 3.4 scrub starts 2024-09-20T19:09:57.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:57 smithi142 ceph-mon[32508]: 3.4 scrub ok 2024-09-20T19:09:57.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:57 smithi142 ceph-mon[32508]: 3.1d scrub starts 2024-09-20T19:09:57.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:57 smithi142 ceph-mon[32508]: 3.1d scrub ok 2024-09-20T19:09:57.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:57 smithi142 ceph-mon[32508]: 4.a scrub starts 2024-09-20T19:09:57.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:57 smithi142 ceph-mon[32508]: 4.a scrub ok 2024-09-20T19:09:58.009 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:09:57 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:09:58.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:58 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:09:58.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:58 smithi083 ceph-mon[29421]: osdmap e504: 8 total, 7 up, 8 in 2024-09-20T19:09:58.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:58 smithi083 ceph-mon[29421]: pgmap v1024: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 47 KiB/s rd, 0 B/s wr, 79 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:09:58.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:58 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:09:58.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:58 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:09:58.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:58 smithi083 ceph-mon[36260]: osdmap e504: 8 total, 7 up, 8 in 2024-09-20T19:09:58.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:58 smithi083 ceph-mon[36260]: pgmap v1024: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 47 KiB/s rd, 0 B/s wr, 79 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:09:58.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:58 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:09:58.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:58 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T19:09:58.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:58 smithi142 ceph-mon[32508]: osdmap e504: 8 total, 7 up, 8 in 2024-09-20T19:09:58.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:58 smithi142 ceph-mon[32508]: pgmap v1024: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 47 KiB/s rd, 0 B/s wr, 79 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:09:58.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:58 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:09:58.804 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:09:58.833 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell -n osd.0 --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --force --op remove --pgid 1.2 2024-09-20T19:09:59.043 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/osd.0/config 2024-09-20T19:09:59.327 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:59 smithi083 ceph-mon[29421]: 4.1f scrub starts 2024-09-20T19:09:59.327 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:09:59 smithi083 ceph-mon[29421]: 4.1f scrub ok 2024-09-20T19:09:59.327 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:59 smithi083 ceph-mon[36260]: 4.1f scrub starts 2024-09-20T19:09:59.327 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:09:59 smithi083 ceph-mon[36260]: 4.1f scrub ok 2024-09-20T19:09:59.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:59 smithi142 ceph-mon[32508]: 4.1f scrub starts 2024-09-20T19:09:59.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:09:59 smithi142 ceph-mon[32508]: 4.1f scrub ok 2024-09-20T19:09:59.542 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T19:09:59.542 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:10:00.174 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:00 smithi083 ceph-mon[29421]: pgmap v1025: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 55 KiB/s rd, 0 B/s wr, 90 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:00.174 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:00 smithi083 ceph-mon[29421]: Health detail: HEALTH_WARN 1 osds down; Degraded data redundancy: 59/597 objects degraded (9.883%), 16 pgs degraded 2024-09-20T19:10:00.174 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:00 smithi083 ceph-mon[29421]: [WRN] OSD_DOWN: 1 osds down 2024-09-20T19:10:00.174 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:00 smithi083 ceph-mon[29421]: osd.0 (root=default,host=smithi083) is down 2024-09-20T19:10:00.174 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:00 smithi083 ceph-mon[29421]: [WRN] PG_DEGRADED: Degraded data redundancy: 59/597 objects degraded (9.883%), 16 pgs degraded 2024-09-20T19:10:00.174 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:00 smithi083 ceph-mon[29421]: pg 2.0 is active+undersized+degraded, acting [7,1] 2024-09-20T19:10:00.174 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:00 smithi083 ceph-mon[29421]: pg 2.1 is active+undersized+degraded, acting [2,3] 2024-09-20T19:10:00.174 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:00 smithi083 ceph-mon[29421]: pg 3.6 is active+undersized+degraded, acting [1,4] 2024-09-20T19:10:00.174 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:00 smithi083 ceph-mon[29421]: pg 3.f is active+undersized+degraded, acting [7,4] 2024-09-20T19:10:00.174 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:00 smithi083 ceph-mon[29421]: pg 4.0 is active+undersized+degraded, acting [3,7] 2024-09-20T19:10:00.174 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:00 smithi083 ceph-mon[29421]: pg 4.3 is active+undersized+degraded, acting [5,7] 2024-09-20T19:10:00.175 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:00 smithi083 ceph-mon[29421]: pg 4.6 is active+undersized+degraded, acting [2,1] 2024-09-20T19:10:00.175 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:00 smithi083 ceph-mon[29421]: pg 4.7 is active+undersized+degraded, acting [1,5] 2024-09-20T19:10:00.175 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:00 smithi083 ceph-mon[29421]: pg 4.b is active+undersized+degraded, acting [1,4] 2024-09-20T19:10:00.175 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:00 smithi083 ceph-mon[29421]: pg 4.16 is active+undersized+degraded, acting [3,7] 2024-09-20T19:10:00.175 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:00 smithi083 ceph-mon[29421]: pg 4.19 is active+undersized+degraded, acting [3,2] 2024-09-20T19:10:00.175 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:00 smithi083 ceph-mon[29421]: pg 4.1a is active+undersized+degraded, acting [4,3] 2024-09-20T19:10:00.175 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:00 smithi083 ceph-mon[29421]: pg 4.1e is active+undersized+degraded, acting [7,2] 2024-09-20T19:10:00.175 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:00 smithi083 ceph-mon[29421]: pg 5.3 is active+undersized+degraded, acting [6,5] 2024-09-20T19:10:00.175 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:00 smithi083 ceph-mon[29421]: pg 5.e is active+undersized+degraded, acting [4,5] 2024-09-20T19:10:00.175 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:00 smithi083 ceph-mon[29421]: pg 5.15 is active+undersized+degraded, acting [5,1] 2024-09-20T19:10:00.175 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:00 smithi083 ceph-mon[36260]: pgmap v1025: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 55 KiB/s rd, 0 B/s wr, 90 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:00.175 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:00 smithi083 ceph-mon[36260]: Health detail: HEALTH_WARN 1 osds down; Degraded data redundancy: 59/597 objects degraded (9.883%), 16 pgs degraded 2024-09-20T19:10:00.176 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:00 smithi083 ceph-mon[36260]: [WRN] OSD_DOWN: 1 osds down 2024-09-20T19:10:00.176 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:00 smithi083 ceph-mon[36260]: osd.0 (root=default,host=smithi083) is down 2024-09-20T19:10:00.176 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:00 smithi083 ceph-mon[36260]: [WRN] PG_DEGRADED: Degraded data redundancy: 59/597 objects degraded (9.883%), 16 pgs degraded 2024-09-20T19:10:00.176 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:00 smithi083 ceph-mon[36260]: pg 2.0 is active+undersized+degraded, acting [7,1] 2024-09-20T19:10:00.176 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:00 smithi083 ceph-mon[36260]: pg 2.1 is active+undersized+degraded, acting [2,3] 2024-09-20T19:10:00.176 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:00 smithi083 ceph-mon[36260]: pg 3.6 is active+undersized+degraded, acting [1,4] 2024-09-20T19:10:00.176 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:00 smithi083 ceph-mon[36260]: pg 3.f is active+undersized+degraded, acting [7,4] 2024-09-20T19:10:00.176 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:00 smithi083 ceph-mon[36260]: pg 4.0 is active+undersized+degraded, acting [3,7] 2024-09-20T19:10:00.176 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:00 smithi083 ceph-mon[36260]: pg 4.3 is active+undersized+degraded, acting [5,7] 2024-09-20T19:10:00.176 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:00 smithi083 ceph-mon[36260]: pg 4.6 is active+undersized+degraded, acting [2,1] 2024-09-20T19:10:00.176 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:00 smithi083 ceph-mon[36260]: pg 4.7 is active+undersized+degraded, acting [1,5] 2024-09-20T19:10:00.176 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:00 smithi083 ceph-mon[36260]: pg 4.b is active+undersized+degraded, acting [1,4] 2024-09-20T19:10:00.176 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:00 smithi083 ceph-mon[36260]: pg 4.16 is active+undersized+degraded, acting [3,7] 2024-09-20T19:10:00.177 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:00 smithi083 ceph-mon[36260]: pg 4.19 is active+undersized+degraded, acting [3,2] 2024-09-20T19:10:00.177 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:00 smithi083 ceph-mon[36260]: pg 4.1a is active+undersized+degraded, acting [4,3] 2024-09-20T19:10:00.177 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:00 smithi083 ceph-mon[36260]: pg 4.1e is active+undersized+degraded, acting [7,2] 2024-09-20T19:10:00.177 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:00 smithi083 ceph-mon[36260]: pg 5.3 is active+undersized+degraded, acting [6,5] 2024-09-20T19:10:00.177 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:00 smithi083 ceph-mon[36260]: pg 5.e is active+undersized+degraded, acting [4,5] 2024-09-20T19:10:00.177 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:00 smithi083 ceph-mon[36260]: pg 5.15 is active+undersized+degraded, acting [5,1] 2024-09-20T19:10:00.203 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_historic_ops 2024-09-20T19:10:00.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:00 smithi142 ceph-mon[32508]: pgmap v1025: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 55 KiB/s rd, 0 B/s wr, 90 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:00.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:00 smithi142 ceph-mon[32508]: Health detail: HEALTH_WARN 1 osds down; Degraded data redundancy: 59/597 objects degraded (9.883%), 16 pgs degraded 2024-09-20T19:10:00.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:00 smithi142 ceph-mon[32508]: [WRN] OSD_DOWN: 1 osds down 2024-09-20T19:10:00.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:00 smithi142 ceph-mon[32508]: osd.0 (root=default,host=smithi083) is down 2024-09-20T19:10:00.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:00 smithi142 ceph-mon[32508]: [WRN] PG_DEGRADED: Degraded data redundancy: 59/597 objects degraded (9.883%), 16 pgs degraded 2024-09-20T19:10:00.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:00 smithi142 ceph-mon[32508]: pg 2.0 is active+undersized+degraded, acting [7,1] 2024-09-20T19:10:00.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:00 smithi142 ceph-mon[32508]: pg 2.1 is active+undersized+degraded, acting [2,3] 2024-09-20T19:10:00.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:00 smithi142 ceph-mon[32508]: pg 3.6 is active+undersized+degraded, acting [1,4] 2024-09-20T19:10:00.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:00 smithi142 ceph-mon[32508]: pg 3.f is active+undersized+degraded, acting [7,4] 2024-09-20T19:10:00.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:00 smithi142 ceph-mon[32508]: pg 4.0 is active+undersized+degraded, acting [3,7] 2024-09-20T19:10:00.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:00 smithi142 ceph-mon[32508]: pg 4.3 is active+undersized+degraded, acting [5,7] 2024-09-20T19:10:00.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:00 smithi142 ceph-mon[32508]: pg 4.6 is active+undersized+degraded, acting [2,1] 2024-09-20T19:10:00.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:00 smithi142 ceph-mon[32508]: pg 4.7 is active+undersized+degraded, acting [1,5] 2024-09-20T19:10:00.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:00 smithi142 ceph-mon[32508]: pg 4.b is active+undersized+degraded, acting [1,4] 2024-09-20T19:10:00.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:00 smithi142 ceph-mon[32508]: pg 4.16 is active+undersized+degraded, acting [3,7] 2024-09-20T19:10:00.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:00 smithi142 ceph-mon[32508]: pg 4.19 is active+undersized+degraded, acting [3,2] 2024-09-20T19:10:00.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:00 smithi142 ceph-mon[32508]: pg 4.1a is active+undersized+degraded, acting [4,3] 2024-09-20T19:10:00.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:00 smithi142 ceph-mon[32508]: pg 4.1e is active+undersized+degraded, acting [7,2] 2024-09-20T19:10:00.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:00 smithi142 ceph-mon[32508]: pg 5.3 is active+undersized+degraded, acting [6,5] 2024-09-20T19:10:00.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:00 smithi142 ceph-mon[32508]: pg 5.e is active+undersized+degraded, acting [4,5] 2024-09-20T19:10:00.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:00 smithi142 ceph-mon[32508]: pg 5.15 is active+undersized+degraded, acting [5,1] 2024-09-20T19:10:00.790 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:10:00 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:10:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:10:01.280 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:01 smithi083 ceph-mon[29421]: 3.1c scrub starts 2024-09-20T19:10:01.280 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:01 smithi083 ceph-mon[29421]: 3.1c scrub ok 2024-09-20T19:10:01.280 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:01 smithi083 ceph-mon[29421]: 5.a scrub starts 2024-09-20T19:10:01.280 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:01 smithi083 ceph-mon[29421]: 5.a scrub ok 2024-09-20T19:10:01.280 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:01 smithi083 ceph-mon[36260]: 3.1c scrub starts 2024-09-20T19:10:01.280 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:01 smithi083 ceph-mon[36260]: 3.1c scrub ok 2024-09-20T19:10:01.280 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:01 smithi083 ceph-mon[36260]: 5.a scrub starts 2024-09-20T19:10:01.280 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:01 smithi083 ceph-mon[36260]: 5.a scrub ok 2024-09-20T19:10:01.324 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:10:01.329 INFO:teuthology.orchestra.run.smithi083.stderr:Creating an OSD daemon form without an OSD FSID value 2024-09-20T19:10:01.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:01 smithi142 ceph-mon[32508]: 3.1c scrub starts 2024-09-20T19:10:01.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:01 smithi142 ceph-mon[32508]: 3.1c scrub ok 2024-09-20T19:10:01.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:01 smithi142 ceph-mon[32508]: 5.a scrub starts 2024-09-20T19:10:01.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:01 smithi142 ceph-mon[32508]: 5.a scrub ok 2024-09-20T19:10:01.994 INFO:teuthology.orchestra.run.smithi083.stdout:ERROR: (22) Invalid argument 2024-09-20T19:10:01.995 INFO:teuthology.orchestra.run.smithi083.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:10:02.467 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:02 smithi083 ceph-mon[29421]: 6.c scrub starts 2024-09-20T19:10:02.467 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:02 smithi083 ceph-mon[29421]: 6.c scrub ok 2024-09-20T19:10:02.467 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:02 smithi083 ceph-mon[29421]: 5.d scrub starts 2024-09-20T19:10:02.467 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:02 smithi083 ceph-mon[29421]: 5.d scrub ok 2024-09-20T19:10:02.467 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:02 smithi083 ceph-mon[29421]: pgmap v1026: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 53 KiB/s rd, 0 B/s wr, 87 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:02.468 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:02 smithi083 ceph-mon[29421]: 6.1d scrub starts 2024-09-20T19:10:02.468 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:02 smithi083 ceph-mon[29421]: 6.1d scrub ok 2024-09-20T19:10:02.468 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:02 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:10:02.468 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:02 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:10:02.468 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:02 smithi083 ceph-mon[36260]: 6.c scrub starts 2024-09-20T19:10:02.468 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:02 smithi083 ceph-mon[36260]: 6.c scrub ok 2024-09-20T19:10:02.468 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:02 smithi083 ceph-mon[36260]: 5.d scrub starts 2024-09-20T19:10:02.468 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:02 smithi083 ceph-mon[36260]: 5.d scrub ok 2024-09-20T19:10:02.468 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:02 smithi083 ceph-mon[36260]: pgmap v1026: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 53 KiB/s rd, 0 B/s wr, 87 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:02.468 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:02 smithi083 ceph-mon[36260]: 6.1d scrub starts 2024-09-20T19:10:02.468 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:02 smithi083 ceph-mon[36260]: 6.1d scrub ok 2024-09-20T19:10:02.468 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:02 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:10:02.469 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:02 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:10:02.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:02 smithi142 ceph-mon[32508]: 6.c scrub starts 2024-09-20T19:10:02.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:02 smithi142 ceph-mon[32508]: 6.c scrub ok 2024-09-20T19:10:02.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:02 smithi142 ceph-mon[32508]: 5.d scrub starts 2024-09-20T19:10:02.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:02 smithi142 ceph-mon[32508]: 5.d scrub ok 2024-09-20T19:10:02.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:02 smithi142 ceph-mon[32508]: pgmap v1026: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 53 KiB/s rd, 0 B/s wr, 87 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:02.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:02 smithi142 ceph-mon[32508]: 6.1d scrub starts 2024-09-20T19:10:02.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:02 smithi142 ceph-mon[32508]: 6.1d scrub ok 2024-09-20T19:10:02.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:02 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:10:02.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:02 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:10:02.657 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-20T19:10:02.925 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:10:03.331 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:03 smithi142 ceph-mon[32508]: 4.8 scrub starts 2024-09-20T19:10:03.331 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:03 smithi142 ceph-mon[32508]: 4.8 scrub ok 2024-09-20T19:10:03.331 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:03 smithi142 ceph-mon[32508]: 4.9 scrub starts 2024-09-20T19:10:03.331 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:03 smithi142 ceph-mon[32508]: 4.9 scrub ok 2024-09-20T19:10:03.331 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:03 smithi142 ceph-mon[32508]: 5.1f scrub starts 2024-09-20T19:10:03.331 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:03 smithi142 ceph-mon[32508]: 5.1f scrub ok 2024-09-20T19:10:03.438 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T19:10:03.439 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:10:03.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:03 smithi083 ceph-mon[29421]: 4.8 scrub starts 2024-09-20T19:10:03.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:03 smithi083 ceph-mon[29421]: 4.8 scrub ok 2024-09-20T19:10:03.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:03 smithi083 ceph-mon[29421]: 4.9 scrub starts 2024-09-20T19:10:03.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:03 smithi083 ceph-mon[29421]: 4.9 scrub ok 2024-09-20T19:10:03.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:03 smithi083 ceph-mon[29421]: 5.1f scrub starts 2024-09-20T19:10:03.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:03 smithi083 ceph-mon[29421]: 5.1f scrub ok 2024-09-20T19:10:03.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:03 smithi083 ceph-mon[36260]: 4.8 scrub starts 2024-09-20T19:10:03.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:03 smithi083 ceph-mon[36260]: 4.8 scrub ok 2024-09-20T19:10:03.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:03 smithi083 ceph-mon[36260]: 4.9 scrub starts 2024-09-20T19:10:03.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:03 smithi083 ceph-mon[36260]: 4.9 scrub ok 2024-09-20T19:10:03.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:03 smithi083 ceph-mon[36260]: 5.1f scrub starts 2024-09-20T19:10:03.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:03 smithi083 ceph-mon[36260]: 5.1f scrub ok 2024-09-20T19:10:04.148 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_blocked_ops 2024-09-20T19:10:04.396 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:10:04.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:04 smithi142 ceph-mon[32508]: 4.2 scrub starts 2024-09-20T19:10:04.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:04 smithi142 ceph-mon[32508]: 4.2 scrub ok 2024-09-20T19:10:04.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:04 smithi142 ceph-mon[32508]: 3.1a scrub starts 2024-09-20T19:10:04.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:04 smithi142 ceph-mon[32508]: 3.1a scrub ok 2024-09-20T19:10:04.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:04 smithi142 ceph-mon[32508]: 6.5 scrub starts 2024-09-20T19:10:04.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:04 smithi142 ceph-mon[32508]: 6.5 scrub ok 2024-09-20T19:10:04.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:04 smithi142 ceph-mon[32508]: pgmap v1027: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 44 KiB/s rd, 0 B/s wr, 72 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:04.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:04 smithi083 ceph-mon[29421]: 4.2 scrub starts 2024-09-20T19:10:04.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:04 smithi083 ceph-mon[29421]: 4.2 scrub ok 2024-09-20T19:10:04.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:04 smithi083 ceph-mon[29421]: 3.1a scrub starts 2024-09-20T19:10:04.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:04 smithi083 ceph-mon[29421]: 3.1a scrub ok 2024-09-20T19:10:04.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:04 smithi083 ceph-mon[29421]: 6.5 scrub starts 2024-09-20T19:10:04.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:04 smithi083 ceph-mon[29421]: 6.5 scrub ok 2024-09-20T19:10:04.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:04 smithi083 ceph-mon[29421]: pgmap v1027: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 44 KiB/s rd, 0 B/s wr, 72 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:04.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:04 smithi083 ceph-mon[36260]: 4.2 scrub starts 2024-09-20T19:10:04.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:04 smithi083 ceph-mon[36260]: 4.2 scrub ok 2024-09-20T19:10:04.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:04 smithi083 ceph-mon[36260]: 3.1a scrub starts 2024-09-20T19:10:04.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:04 smithi083 ceph-mon[36260]: 3.1a scrub ok 2024-09-20T19:10:04.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:04 smithi083 ceph-mon[36260]: 6.5 scrub starts 2024-09-20T19:10:04.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:04 smithi083 ceph-mon[36260]: 6.5 scrub ok 2024-09-20T19:10:04.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:04 smithi083 ceph-mon[36260]: pgmap v1027: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 44 KiB/s rd, 0 B/s wr, 72 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:04.873 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T19:10:04.873 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:10:05.434 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:05 smithi083 ceph-mon[29421]: 6.4 scrub starts 2024-09-20T19:10:05.434 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:05 smithi083 ceph-mon[29421]: 6.4 scrub ok 2024-09-20T19:10:05.435 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:05 smithi083 ceph-mon[36260]: 6.4 scrub starts 2024-09-20T19:10:05.435 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:05 smithi083 ceph-mon[36260]: 6.4 scrub ok 2024-09-20T19:10:05.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:05 smithi142 ceph-mon[32508]: 6.4 scrub starts 2024-09-20T19:10:05.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:05 smithi142 ceph-mon[32508]: 6.4 scrub ok 2024-09-20T19:10:05.605 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_historic_ops 2024-09-20T19:10:05.853 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:10:06.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:06 smithi142 ceph-mon[32508]: 5.17 scrub starts 2024-09-20T19:10:06.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:06 smithi142 ceph-mon[32508]: 5.17 scrub ok 2024-09-20T19:10:06.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:06 smithi142 ceph-mon[32508]: 6.1c scrub starts 2024-09-20T19:10:06.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:06 smithi142 ceph-mon[32508]: 6.1c scrub ok 2024-09-20T19:10:06.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:06 smithi142 ceph-mon[32508]: pgmap v1028: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 44 KiB/s rd, 0 B/s wr, 72 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:06.363 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T19:10:06.364 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:10:06.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:06 smithi083 ceph-mon[29421]: 5.17 scrub starts 2024-09-20T19:10:06.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:06 smithi083 ceph-mon[29421]: 5.17 scrub ok 2024-09-20T19:10:06.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:06 smithi083 ceph-mon[29421]: 6.1c scrub starts 2024-09-20T19:10:06.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:06 smithi083 ceph-mon[29421]: 6.1c scrub ok 2024-09-20T19:10:06.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:06 smithi083 ceph-mon[29421]: pgmap v1028: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 44 KiB/s rd, 0 B/s wr, 72 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:06.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:06 smithi083 ceph-mon[36260]: 5.17 scrub starts 2024-09-20T19:10:06.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:06 smithi083 ceph-mon[36260]: 5.17 scrub ok 2024-09-20T19:10:06.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:06 smithi083 ceph-mon[36260]: 6.1c scrub starts 2024-09-20T19:10:06.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:06 smithi083 ceph-mon[36260]: 6.1c scrub ok 2024-09-20T19:10:06.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:06 smithi083 ceph-mon[36260]: pgmap v1028: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 44 KiB/s rd, 0 B/s wr, 72 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:07.223 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-20T19:10:07.468 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:10:07.509 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:07 smithi142 ceph-mon[32508]: 2.2 scrub starts 2024-09-20T19:10:07.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:07 smithi142 ceph-mon[32508]: 2.2 scrub ok 2024-09-20T19:10:07.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:07 smithi083 ceph-mon[29421]: 2.2 scrub starts 2024-09-20T19:10:07.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:07 smithi083 ceph-mon[29421]: 2.2 scrub ok 2024-09-20T19:10:07.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:07 smithi083 ceph-mon[36260]: 2.2 scrub starts 2024-09-20T19:10:07.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:07 smithi083 ceph-mon[36260]: 2.2 scrub ok 2024-09-20T19:10:07.880 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:10:07 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:10:07.982 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T19:10:07.982 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:10:08.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:08 smithi142 ceph-mon[32508]: 4.c scrub starts 2024-09-20T19:10:08.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:08 smithi142 ceph-mon[32508]: 4.c scrub ok 2024-09-20T19:10:08.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:08 smithi142 ceph-mon[32508]: 5.14 scrub starts 2024-09-20T19:10:08.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:08 smithi142 ceph-mon[32508]: 5.14 scrub ok 2024-09-20T19:10:08.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:08 smithi142 ceph-mon[32508]: pgmap v1029: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 10 KiB/s rd, 0 B/s wr, 16 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:08.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:08 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:10:08.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:08 smithi083 ceph-mon[29421]: 4.c scrub starts 2024-09-20T19:10:08.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:08 smithi083 ceph-mon[29421]: 4.c scrub ok 2024-09-20T19:10:08.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:08 smithi083 ceph-mon[29421]: 5.14 scrub starts 2024-09-20T19:10:08.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:08 smithi083 ceph-mon[29421]: 5.14 scrub ok 2024-09-20T19:10:08.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:08 smithi083 ceph-mon[29421]: pgmap v1029: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 10 KiB/s rd, 0 B/s wr, 16 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:08.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:08 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:10:08.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:08 smithi083 ceph-mon[36260]: 4.c scrub starts 2024-09-20T19:10:08.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:08 smithi083 ceph-mon[36260]: 4.c scrub ok 2024-09-20T19:10:08.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:08 smithi083 ceph-mon[36260]: 5.14 scrub starts 2024-09-20T19:10:08.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:08 smithi083 ceph-mon[36260]: 5.14 scrub ok 2024-09-20T19:10:08.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:08 smithi083 ceph-mon[36260]: pgmap v1029: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 10 KiB/s rd, 0 B/s wr, 16 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:08.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:08 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:10:08.780 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_blocked_ops 2024-09-20T19:10:09.026 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:10:09.208 INFO:teuthology.orchestra.run.smithi083.stdout: marking collection for removal 2024-09-20T19:10:09.209 INFO:teuthology.orchestra.run.smithi083.stdout:setting '_remove' omap key 2024-09-20T19:10:09.209 INFO:teuthology.orchestra.run.smithi083.stdout:finish_remove_pgs 1.2_head removing 1.2 2024-09-20T19:10:09.226 INFO:teuthology.orchestra.run.smithi083.stdout:Remove successful 2024-09-20T19:10:09.404 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:09 smithi142 ceph-mon[32508]: 4.1 scrub starts 2024-09-20T19:10:09.404 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:09 smithi142 ceph-mon[32508]: 4.1 scrub ok 2024-09-20T19:10:09.405 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:09 smithi142 ceph-mon[32508]: 6.1 scrub starts 2024-09-20T19:10:09.405 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:09 smithi142 ceph-mon[32508]: 6.1 scrub ok 2024-09-20T19:10:09.405 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:09 smithi142 ceph-mon[32508]: 5.11 scrub starts 2024-09-20T19:10:09.405 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:09 smithi142 ceph-mon[32508]: 5.11 scrub ok 2024-09-20T19:10:09.505 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T19:10:09.505 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:10:09.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:09 smithi083 ceph-mon[29421]: 4.1 scrub starts 2024-09-20T19:10:09.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:09 smithi083 ceph-mon[29421]: 4.1 scrub ok 2024-09-20T19:10:09.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:09 smithi083 ceph-mon[29421]: 6.1 scrub starts 2024-09-20T19:10:09.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:09 smithi083 ceph-mon[29421]: 6.1 scrub ok 2024-09-20T19:10:09.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:09 smithi083 ceph-mon[29421]: 5.11 scrub starts 2024-09-20T19:10:09.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:09 smithi083 ceph-mon[29421]: 5.11 scrub ok 2024-09-20T19:10:09.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:09 smithi083 ceph-mon[36260]: 4.1 scrub starts 2024-09-20T19:10:09.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:09 smithi083 ceph-mon[36260]: 4.1 scrub ok 2024-09-20T19:10:09.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:09 smithi083 ceph-mon[36260]: 6.1 scrub starts 2024-09-20T19:10:09.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:09 smithi083 ceph-mon[36260]: 6.1 scrub ok 2024-09-20T19:10:09.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:09 smithi083 ceph-mon[36260]: 5.11 scrub starts 2024-09-20T19:10:09.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:09 smithi083 ceph-mon[36260]: 5.11 scrub ok 2024-09-20T19:10:10.160 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_historic_ops 2024-09-20T19:10:10.407 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:10:10.488 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:10 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:10:10.489 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:10 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:10:10.489 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:10 smithi083 ceph-mon[29421]: 6.b scrub starts 2024-09-20T19:10:10.489 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:10 smithi083 ceph-mon[29421]: 6.b scrub ok 2024-09-20T19:10:10.489 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:10 smithi083 ceph-mon[29421]: osdmap e505: 8 total, 7 up, 8 in 2024-09-20T19:10:10.489 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:10 smithi083 ceph-mon[29421]: pgmap v1031: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:10.489 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:10 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:10:10.489 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:10 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:10:10.489 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:10 smithi083 ceph-mon[36260]: 6.b scrub starts 2024-09-20T19:10:10.489 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:10 smithi083 ceph-mon[36260]: 6.b scrub ok 2024-09-20T19:10:10.489 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:10 smithi083 ceph-mon[36260]: osdmap e505: 8 total, 7 up, 8 in 2024-09-20T19:10:10.489 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:10 smithi083 ceph-mon[36260]: pgmap v1031: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:10.509 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:10 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:10:10.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:10 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:10:10.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:10 smithi142 ceph-mon[32508]: 6.b scrub starts 2024-09-20T19:10:10.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:10 smithi142 ceph-mon[32508]: 6.b scrub ok 2024-09-20T19:10:10.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:10 smithi142 ceph-mon[32508]: osdmap e505: 8 total, 7 up, 8 in 2024-09-20T19:10:10.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:10 smithi142 ceph-mon[32508]: pgmap v1031: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:10.520 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell -n osd.0 --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.1.2.0 2024-09-20T19:10:10.713 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/osd.0/config 2024-09-20T19:10:10.790 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:10:10 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:10:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:10:10.894 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T19:10:10.894 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:10:11.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:11 smithi142 ceph-mon[32508]: 5.10 scrub starts 2024-09-20T19:10:11.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:11 smithi142 ceph-mon[32508]: 5.10 scrub ok 2024-09-20T19:10:11.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:11 smithi083 ceph-mon[29421]: 5.10 scrub starts 2024-09-20T19:10:11.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:11 smithi083 ceph-mon[29421]: 5.10 scrub ok 2024-09-20T19:10:11.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:11 smithi083 ceph-mon[36260]: 5.10 scrub starts 2024-09-20T19:10:11.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:11 smithi083 ceph-mon[36260]: 5.10 scrub ok 2024-09-20T19:10:11.589 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-20T19:10:11.836 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:10:11.988 INFO:teuthology.orchestra.run.smithi083.stderr:Creating an OSD daemon form without an OSD FSID value 2024-09-20T19:10:12.227 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:12 smithi142 ceph-mon[32508]: 4.17 scrub starts 2024-09-20T19:10:12.227 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:12 smithi142 ceph-mon[32508]: 4.17 scrub ok 2024-09-20T19:10:12.228 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:12 smithi142 ceph-mon[32508]: 5.9 scrub starts 2024-09-20T19:10:12.228 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:12 smithi142 ceph-mon[32508]: 5.9 scrub ok 2024-09-20T19:10:12.228 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:12 smithi142 ceph-mon[32508]: pgmap v1032: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:12.228 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:12 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:10:12.228 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:12 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:10:12.232 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:12 smithi083 ceph-mon[29421]: 4.17 scrub starts 2024-09-20T19:10:12.232 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:12 smithi083 ceph-mon[29421]: 4.17 scrub ok 2024-09-20T19:10:12.232 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:12 smithi083 ceph-mon[29421]: 5.9 scrub starts 2024-09-20T19:10:12.232 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:12 smithi083 ceph-mon[29421]: 5.9 scrub ok 2024-09-20T19:10:12.232 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:12 smithi083 ceph-mon[29421]: pgmap v1032: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:12.232 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:12 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:10:12.233 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:12 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:10:12.233 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:12 smithi083 ceph-mon[36260]: 4.17 scrub starts 2024-09-20T19:10:12.233 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:12 smithi083 ceph-mon[36260]: 4.17 scrub ok 2024-09-20T19:10:12.233 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:12 smithi083 ceph-mon[36260]: 5.9 scrub starts 2024-09-20T19:10:12.233 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:12 smithi083 ceph-mon[36260]: 5.9 scrub ok 2024-09-20T19:10:12.233 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:12 smithi083 ceph-mon[36260]: pgmap v1032: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:12.233 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:12 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:10:12.233 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:12 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:10:12.502 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T19:10:12.502 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:10:12.977 INFO:teuthology.orchestra.run.smithi083.stdout:get_pg_num_history pg_num_history pg_num_history(e494 pg_nums {1={19=1,455=11},2={57=3},3={58=32},4={60=32},5={62=32},6={64=32},11={422=16,441=26}} deleted_pools ) 2024-09-20T19:10:12.977 INFO:teuthology.orchestra.run.smithi083.stdout:Importing pgid 1.2 2024-09-20T19:10:12.977 INFO:teuthology.orchestra.run.smithi083.stdout:write_pg epoch 477 info 1.2( v 74'79 (0'0,74'79] local-lis/les=458/459 n=1 ec=455/19 lis/c=458/458 les/c/f=459/459/0 sis=458) 2024-09-20T19:10:12.977 INFO:teuthology.orchestra.run.smithi083.stdout:Import successful 2024-09-20T19:10:13.168 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_blocked_ops 2024-09-20T19:10:13.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:13 smithi083 ceph-mon[29421]: osdmap e506: 8 total, 7 up, 8 in 2024-09-20T19:10:13.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:13 smithi083 ceph-mon[36260]: osdmap e506: 8 total, 7 up, 8 in 2024-09-20T19:10:13.386 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:13 smithi142 ceph-mon[32508]: osdmap e506: 8 total, 7 up, 8 in 2024-09-20T19:10:13.425 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:10:13.901 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T19:10:13.902 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:10:14.155 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/exp.1.2.0 2024-09-20T19:10:14.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:14 smithi142 ceph-mon[32508]: 5.18 scrub starts 2024-09-20T19:10:14.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:14 smithi142 ceph-mon[32508]: 5.18 scrub ok 2024-09-20T19:10:14.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:14 smithi142 ceph-mon[32508]: 3.5 scrub starts 2024-09-20T19:10:14.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:14 smithi142 ceph-mon[32508]: 3.5 scrub ok 2024-09-20T19:10:14.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:14 smithi142 ceph-mon[32508]: 6.1f scrub starts 2024-09-20T19:10:14.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:14 smithi142 ceph-mon[32508]: 6.1f scrub ok 2024-09-20T19:10:14.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:14 smithi142 ceph-mon[32508]: pgmap v1034: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:14.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:14 smithi083 ceph-mon[29421]: 5.18 scrub starts 2024-09-20T19:10:14.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:14 smithi083 ceph-mon[29421]: 5.18 scrub ok 2024-09-20T19:10:14.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:14 smithi083 ceph-mon[29421]: 3.5 scrub starts 2024-09-20T19:10:14.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:14 smithi083 ceph-mon[29421]: 3.5 scrub ok 2024-09-20T19:10:14.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:14 smithi083 ceph-mon[29421]: 6.1f scrub starts 2024-09-20T19:10:14.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:14 smithi083 ceph-mon[29421]: 6.1f scrub ok 2024-09-20T19:10:14.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:14 smithi083 ceph-mon[29421]: pgmap v1034: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:14.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:14 smithi083 ceph-mon[36260]: 5.18 scrub starts 2024-09-20T19:10:14.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:14 smithi083 ceph-mon[36260]: 5.18 scrub ok 2024-09-20T19:10:14.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:14 smithi083 ceph-mon[36260]: 3.5 scrub starts 2024-09-20T19:10:14.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:14 smithi083 ceph-mon[36260]: 3.5 scrub ok 2024-09-20T19:10:14.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:14 smithi083 ceph-mon[36260]: 6.1f scrub starts 2024-09-20T19:10:14.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:14 smithi083 ceph-mon[36260]: 6.1f scrub ok 2024-09-20T19:10:14.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:14 smithi083 ceph-mon[36260]: pgmap v1034: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:14.719 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_historic_ops 2024-09-20T19:10:14.963 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:10:15.385 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:15 smithi142 ceph-mon[32508]: 4.10 scrub starts 2024-09-20T19:10:15.385 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:15 smithi142 ceph-mon[32508]: 4.10 scrub ok 2024-09-20T19:10:15.485 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T19:10:15.485 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:10:15.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:15 smithi083 ceph-mon[29421]: 4.10 scrub starts 2024-09-20T19:10:15.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:15 smithi083 ceph-mon[29421]: 4.10 scrub ok 2024-09-20T19:10:15.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:15 smithi083 ceph-mon[36260]: 4.10 scrub starts 2024-09-20T19:10:15.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:15 smithi083 ceph-mon[36260]: 4.10 scrub ok 2024-09-20T19:10:16.103 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-20T19:10:16.352 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:10:16.473 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:16 smithi142 ceph-mon[32508]: pgmap v1035: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:16.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:16 smithi083 ceph-mon[29421]: pgmap v1035: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:16.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:16 smithi083 ceph-mon[36260]: pgmap v1035: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:16.808 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T19:10:16.808 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:10:17.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:17 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:10:17.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:17 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:10:17.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:17 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:10:17.584 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_blocked_ops 2024-09-20T19:10:17.831 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:10:17.952 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:10:17 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:10:18.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:18 smithi142 ceph-mon[32508]: pgmap v1036: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:18.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:18 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:10:18.494 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T19:10:18.494 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:10:18.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:18 smithi083 ceph-mon[29421]: pgmap v1036: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:18.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:18 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:10:18.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:18 smithi083 ceph-mon[36260]: pgmap v1036: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:18.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:18 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:10:19.131 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_historic_ops 2024-09-20T19:10:19.186 INFO:tasks.thrashosds.thrasher:reviving osd 2024-09-20T19:10:19.186 INFO:tasks.thrashosds.thrasher:Reviving osd 0 2024-09-20T19:10:19.187 INFO:tasks.cephadm.osd.0:Restarting osd.0 (starting--it wasn't running)... 2024-09-20T19:10:19.187 DEBUG:teuthology.orchestra.run.smithi083:osd.0> sudo journalctl -f -n 0 -u ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@osd.0.service 2024-09-20T19:10:19.189 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl start ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@osd.0 2024-09-20T19:10:19.387 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T19:10:19.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:19 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:10:19.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:19 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:10:19.510 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:19 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:10:19.510 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:19 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:10:19.511 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:19 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:10:19.511 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:19 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:10:19.511 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:10:19 smithi083 systemd[1]: Starting Ceph osd.0 for 1ed5a50a-7780-11ef-baf5-efdc52797490... 2024-09-20T19:10:19.790 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:10:19 smithi083 podman[198345]: 2024-09-20 19:10:19.41073039 +0000 UTC m=+0.015978980 image pull 441f9a3fe51528ef653b21f42e9d884db6124a0c7d482e865382bdfdb9ec979c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2 2024-09-20T19:10:19.790 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:10:19 smithi083 podman[198345]: 2024-09-20 19:10:19.545574213 +0000 UTC m=+0.150822781 container create 3de53dc383d7b8cd8e386ee0b0b1fc9e6c53ae00112adc1b34d906dbcefb9f74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-0-activate, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-20T19:10:19.859 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-09-20T19:10:19.859 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T19:10:20.141 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:10:19 smithi083 podman[198345]: 2024-09-20 19:10:19.815210642 +0000 UTC m=+0.420459224 container init 3de53dc383d7b8cd8e386ee0b0b1fc9e6c53ae00112adc1b34d906dbcefb9f74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-0-activate, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-20T19:10:20.141 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:10:19 smithi083 podman[198345]: 2024-09-20 19:10:19.818670352 +0000 UTC m=+0.423918922 container start 3de53dc383d7b8cd8e386ee0b0b1fc9e6c53ae00112adc1b34d906dbcefb9f74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-0-activate, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-20T19:10:20.141 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:10:19 smithi083 podman[198345]: 2024-09-20 19:10:19.915131006 +0000 UTC m=+0.520379587 container attach 3de53dc383d7b8cd8e386ee0b0b1fc9e6c53ae00112adc1b34d906dbcefb9f74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-0-activate, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.vendor=CentOS) 2024-09-20T19:10:20.141 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:10:19 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-0-activate[198393]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-20T19:10:20.141 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:10:19 smithi083 bash[198345]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-20T19:10:20.141 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:10:19 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-0-activate[198393]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-20T19:10:20.141 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:10:19 smithi083 bash[198345]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-20T19:10:20.488 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:20 smithi142 ceph-mon[32508]: 6.7 scrub starts 2024-09-20T19:10:20.488 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:20 smithi142 ceph-mon[32508]: 6.7 scrub ok 2024-09-20T19:10:20.488 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:20 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T19:10:20.488 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:20 smithi142 ceph-mon[32508]: osdmap e507: 8 total, 7 up, 8 in 2024-09-20T19:10:20.488 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:20 smithi142 ceph-mon[32508]: pgmap v1038: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:20.488 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:20 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T19:10:20.488 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:20 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T19:10:20.488 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:20 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:10:20.488 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:20 smithi142 ceph-mon[32508]: 3.15 scrub starts 2024-09-20T19:10:20.488 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:20 smithi142 ceph-mon[32508]: 3.15 scrub ok 2024-09-20T19:10:20.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:20 smithi083 ceph-mon[29421]: 6.7 scrub starts 2024-09-20T19:10:20.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:20 smithi083 ceph-mon[29421]: 6.7 scrub ok 2024-09-20T19:10:20.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:20 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T19:10:20.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:20 smithi083 ceph-mon[29421]: osdmap e507: 8 total, 7 up, 8 in 2024-09-20T19:10:20.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:20 smithi083 ceph-mon[29421]: pgmap v1038: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:20.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:20 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T19:10:20.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:20 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T19:10:20.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:20 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:10:20.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:20 smithi083 ceph-mon[29421]: 3.15 scrub starts 2024-09-20T19:10:20.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:20 smithi083 ceph-mon[29421]: 3.15 scrub ok 2024-09-20T19:10:20.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:20 smithi083 ceph-mon[36260]: 6.7 scrub starts 2024-09-20T19:10:20.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:20 smithi083 ceph-mon[36260]: 6.7 scrub ok 2024-09-20T19:10:20.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:20 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T19:10:20.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:20 smithi083 ceph-mon[36260]: osdmap e507: 8 total, 7 up, 8 in 2024-09-20T19:10:20.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:20 smithi083 ceph-mon[36260]: pgmap v1038: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:20.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:20 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T19:10:20.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:20 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T19:10:20.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:20 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:10:20.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:20 smithi083 ceph-mon[36260]: 3.15 scrub starts 2024-09-20T19:10:20.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:20 smithi083 ceph-mon[36260]: 3.15 scrub ok 2024-09-20T19:10:21.039 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:10:20 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:10:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:10:22.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:21 smithi142 ceph-mon[32508]: 3.9 scrub starts 2024-09-20T19:10:22.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:21 smithi142 ceph-mon[32508]: 3.9 scrub ok 2024-09-20T19:10:22.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:21 smithi142 ceph-mon[32508]: 3.2 scrub starts 2024-09-20T19:10:22.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:21 smithi142 ceph-mon[32508]: 3.2 scrub ok 2024-09-20T19:10:22.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:21 smithi142 ceph-mon[32508]: pgmap v1039: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 560 B/s rd, 0 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:22.289 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:21 smithi083 ceph-mon[29421]: 3.9 scrub starts 2024-09-20T19:10:22.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:21 smithi083 ceph-mon[29421]: 3.9 scrub ok 2024-09-20T19:10:22.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:21 smithi083 ceph-mon[29421]: 3.2 scrub starts 2024-09-20T19:10:22.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:21 smithi083 ceph-mon[29421]: 3.2 scrub ok 2024-09-20T19:10:22.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:21 smithi083 ceph-mon[29421]: pgmap v1039: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 560 B/s rd, 0 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:22.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:21 smithi083 ceph-mon[36260]: 3.9 scrub starts 2024-09-20T19:10:22.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:21 smithi083 ceph-mon[36260]: 3.9 scrub ok 2024-09-20T19:10:22.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:21 smithi083 ceph-mon[36260]: 3.2 scrub starts 2024-09-20T19:10:22.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:21 smithi083 ceph-mon[36260]: 3.2 scrub ok 2024-09-20T19:10:22.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:21 smithi083 ceph-mon[36260]: pgmap v1039: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 560 B/s rd, 0 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:23.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:22 smithi142 ceph-mon[32508]: 3.11 scrub starts 2024-09-20T19:10:23.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:22 smithi142 ceph-mon[32508]: 3.11 scrub ok 2024-09-20T19:10:23.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:22 smithi142 ceph-mon[32508]: 3.8 scrub starts 2024-09-20T19:10:23.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:22 smithi142 ceph-mon[32508]: 3.8 scrub ok 2024-09-20T19:10:23.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:22 smithi083 ceph-mon[29421]: 3.11 scrub starts 2024-09-20T19:10:23.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:22 smithi083 ceph-mon[29421]: 3.11 scrub ok 2024-09-20T19:10:23.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:22 smithi083 ceph-mon[29421]: 3.8 scrub starts 2024-09-20T19:10:23.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:22 smithi083 ceph-mon[29421]: 3.8 scrub ok 2024-09-20T19:10:23.290 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:10:23 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-0-activate[198393]: --> Failed to activate via raw: did not find any matching OSD to activate 2024-09-20T19:10:23.291 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:10:23 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-0-activate[198393]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-20T19:10:23.291 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:10:23 smithi083 bash[198345]: --> Failed to activate via raw: did not find any matching OSD to activate 2024-09-20T19:10:23.291 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:10:23 smithi083 bash[198345]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-20T19:10:23.291 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:10:23 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-0-activate[198393]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-20T19:10:23.291 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:10:23 smithi083 bash[198345]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-20T19:10:23.291 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:10:23 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-0-activate[198393]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-20T19:10:23.291 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:10:23 smithi083 bash[198345]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-20T19:10:23.291 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:10:23 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-0-activate[198393]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_4 --path /var/lib/ceph/osd/ceph-0 --no-mon-config 2024-09-20T19:10:23.291 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:10:23 smithi083 bash[198345]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_4 --path /var/lib/ceph/osd/ceph-0 --no-mon-config 2024-09-20T19:10:23.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:22 smithi083 ceph-mon[36260]: 3.11 scrub starts 2024-09-20T19:10:23.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:22 smithi083 ceph-mon[36260]: 3.11 scrub ok 2024-09-20T19:10:23.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:22 smithi083 ceph-mon[36260]: 3.8 scrub starts 2024-09-20T19:10:23.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:22 smithi083 ceph-mon[36260]: 3.8 scrub ok 2024-09-20T19:10:23.790 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:10:23 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-0-activate[198393]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-0/block 2024-09-20T19:10:23.790 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:10:23 smithi083 bash[198345]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-0/block 2024-09-20T19:10:23.790 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:10:23 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-0-activate[198393]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-0/block 2024-09-20T19:10:23.790 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:10:23 smithi083 bash[198345]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-0/block 2024-09-20T19:10:23.790 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:10:23 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-0-activate[198393]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-09-20T19:10:23.790 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:10:23 smithi083 bash[198345]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-09-20T19:10:23.790 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:10:23 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-0-activate[198393]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-20T19:10:23.791 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:10:23 smithi083 bash[198345]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-20T19:10:23.791 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:10:23 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-0-activate[198393]: --> ceph-volume lvm activate successful for osd ID: 0 2024-09-20T19:10:23.791 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:10:23 smithi083 bash[198345]: --> ceph-volume lvm activate successful for osd ID: 0 2024-09-20T19:10:23.791 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:10:23 smithi083 podman[198345]: 2024-09-20 19:10:23.444503247 +0000 UTC m=+4.049751832 container died 3de53dc383d7b8cd8e386ee0b0b1fc9e6c53ae00112adc1b34d906dbcefb9f74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-0-activate, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-20-1028-88deadb, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-20T19:10:24.172 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:23 smithi083 ceph-mon[29421]: 4.13 scrub starts 2024-09-20T19:10:24.172 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:23 smithi083 ceph-mon[29421]: 4.13 scrub ok 2024-09-20T19:10:24.172 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:23 smithi083 ceph-mon[29421]: 5.16 scrub starts 2024-09-20T19:10:24.172 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:23 smithi083 ceph-mon[29421]: 5.16 scrub ok 2024-09-20T19:10:24.172 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:23 smithi083 ceph-mon[29421]: pgmap v1040: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:24.172 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:23 smithi083 ceph-mon[36260]: 4.13 scrub starts 2024-09-20T19:10:24.173 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:23 smithi083 ceph-mon[36260]: 4.13 scrub ok 2024-09-20T19:10:24.173 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:23 smithi083 ceph-mon[36260]: 5.16 scrub starts 2024-09-20T19:10:24.173 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:23 smithi083 ceph-mon[36260]: 5.16 scrub ok 2024-09-20T19:10:24.173 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:23 smithi083 ceph-mon[36260]: pgmap v1040: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:24.259 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:23 smithi142 ceph-mon[32508]: 4.13 scrub starts 2024-09-20T19:10:24.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:23 smithi142 ceph-mon[32508]: 4.13 scrub ok 2024-09-20T19:10:24.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:23 smithi142 ceph-mon[32508]: 5.16 scrub starts 2024-09-20T19:10:24.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:23 smithi142 ceph-mon[32508]: 5.16 scrub ok 2024-09-20T19:10:24.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:23 smithi142 ceph-mon[32508]: pgmap v1040: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:24.507 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:10:24 smithi083 podman[198613]: 2024-09-20 19:10:24.169766316 +0000 UTC m=+0.715568094 container remove 3de53dc383d7b8cd8e386ee0b0b1fc9e6c53ae00112adc1b34d906dbcefb9f74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-0-activate, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True) 2024-09-20T19:10:24.507 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:10:24 smithi083 podman[198676]: 2024-09-20 19:10:24.281415674 +0000 UTC m=+0.016342330 image pull 441f9a3fe51528ef653b21f42e9d884db6124a0c7d482e865382bdfdb9ec979c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2 2024-09-20T19:10:24.507 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:10:24 smithi083 podman[198676]: 2024-09-20 19:10:24.398226553 +0000 UTC m=+0.133153209 container create 19fa3e95aff40c22d8d336802cbaa910fbacdbe5538beae132a28290823ce37a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731) 2024-09-20T19:10:24.758 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-20T19:10:24.782 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:10:24 smithi083 podman[198676]: 2024-09-20 19:10:24.609011157 +0000 UTC m=+0.343937821 container init 19fa3e95aff40c22d8d336802cbaa910fbacdbe5538beae132a28290823ce37a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-0, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-20-1028-88deadb, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-20T19:10:24.782 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:10:24 smithi083 podman[198676]: 2024-09-20 19:10:24.612896534 +0000 UTC m=+0.347823191 container start 19fa3e95aff40c22d8d336802cbaa910fbacdbe5538beae132a28290823ce37a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-20T19:10:24.782 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:10:24 smithi083 ceph-osd[198696]: -- 172.21.15.83:0/4085351232 <== mon.0 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5624952dda40 con 0x5624960ac000 2024-09-20T19:10:24.782 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:10:24 smithi083 bash[198676]: 19fa3e95aff40c22d8d336802cbaa910fbacdbe5538beae132a28290823ce37a 2024-09-20T19:10:24.782 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:10:24 smithi083 systemd[1]: Started Ceph osd.0 for 1ed5a50a-7780-11ef-baf5-efdc52797490. 2024-09-20T19:10:25.013 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:10:25.039 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:24 smithi083 ceph-mon[36260]: 6.12 scrub starts 2024-09-20T19:10:25.040 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:24 smithi083 ceph-mon[36260]: 6.12 scrub ok 2024-09-20T19:10:25.040 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:24 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:10:25.040 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:24 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:10:25.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:24 smithi083 ceph-mon[29421]: 6.12 scrub starts 2024-09-20T19:10:25.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:24 smithi083 ceph-mon[29421]: 6.12 scrub ok 2024-09-20T19:10:25.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:24 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:10:25.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:24 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:10:25.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:24 smithi142 ceph-mon[32508]: 6.12 scrub starts 2024-09-20T19:10:25.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:24 smithi142 ceph-mon[32508]: 6.12 scrub ok 2024-09-20T19:10:25.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:24 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:10:25.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:24 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:10:25.506 INFO:teuthology.orchestra.run.smithi083.stderr:no valid command found; 10 closest matches: 2024-09-20T19:10:25.506 INFO:teuthology.orchestra.run.smithi083.stderr:0 2024-09-20T19:10:25.507 INFO:teuthology.orchestra.run.smithi083.stderr:1 2024-09-20T19:10:25.507 INFO:teuthology.orchestra.run.smithi083.stderr:2 2024-09-20T19:10:25.507 INFO:teuthology.orchestra.run.smithi083.stderr:abort 2024-09-20T19:10:25.507 INFO:teuthology.orchestra.run.smithi083.stderr:assert 2024-09-20T19:10:25.507 INFO:teuthology.orchestra.run.smithi083.stderr:config diff 2024-09-20T19:10:25.507 INFO:teuthology.orchestra.run.smithi083.stderr:config diff get 2024-09-20T19:10:25.507 INFO:teuthology.orchestra.run.smithi083.stderr:config get 2024-09-20T19:10:25.507 INFO:teuthology.orchestra.run.smithi083.stderr:config help [] 2024-09-20T19:10:25.507 INFO:teuthology.orchestra.run.smithi083.stderr:config set ... 2024-09-20T19:10:25.507 INFO:teuthology.orchestra.run.smithi083.stderr:admin_socket: invalid command 2024-09-20T19:10:26.270 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:26 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:10:26.270 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:26 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:10:26.271 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:26 smithi083 ceph-mon[29421]: 4.18 scrub starts 2024-09-20T19:10:26.271 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:26 smithi083 ceph-mon[29421]: 4.18 scrub ok 2024-09-20T19:10:26.271 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:26 smithi083 ceph-mon[29421]: 3.16 scrub starts 2024-09-20T19:10:26.271 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:26 smithi083 ceph-mon[29421]: 3.16 scrub ok 2024-09-20T19:10:26.271 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:26 smithi083 ceph-mon[29421]: osdmap e508: 8 total, 7 up, 8 in 2024-09-20T19:10:26.271 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:26 smithi083 ceph-mon[29421]: pgmap v1042: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:26.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:26 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:10:26.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:26 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:10:26.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:26 smithi083 ceph-mon[36260]: 4.18 scrub starts 2024-09-20T19:10:26.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:26 smithi083 ceph-mon[36260]: 4.18 scrub ok 2024-09-20T19:10:26.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:26 smithi083 ceph-mon[36260]: 3.16 scrub starts 2024-09-20T19:10:26.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:26 smithi083 ceph-mon[36260]: 3.16 scrub ok 2024-09-20T19:10:26.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:26 smithi083 ceph-mon[36260]: osdmap e508: 8 total, 7 up, 8 in 2024-09-20T19:10:26.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:26 smithi083 ceph-mon[36260]: pgmap v1042: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:26.300 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-20T19:10:26.300 INFO:tasks.cephadm.ceph_manager.ceph:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-20T19:10:26.434 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:26 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:10:26.434 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:26 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:10:26.435 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:26 smithi142 ceph-mon[32508]: 4.18 scrub starts 2024-09-20T19:10:26.435 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:26 smithi142 ceph-mon[32508]: 4.18 scrub ok 2024-09-20T19:10:26.435 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:26 smithi142 ceph-mon[32508]: 3.16 scrub starts 2024-09-20T19:10:26.435 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:26 smithi142 ceph-mon[32508]: 3.16 scrub ok 2024-09-20T19:10:26.435 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:26 smithi142 ceph-mon[32508]: osdmap e508: 8 total, 7 up, 8 in 2024-09-20T19:10:26.435 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:26 smithi142 ceph-mon[32508]: pgmap v1042: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:26.539 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:10:26 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-0[198692]: 2024-09-20T19:10:26.339+0000 7ff82170b640 -1 Falling back to public interface 2024-09-20T19:10:27.289 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:27 smithi083 ceph-mon[29421]: 4.15 scrub starts 2024-09-20T19:10:27.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:27 smithi083 ceph-mon[29421]: 4.15 scrub ok 2024-09-20T19:10:27.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:27 smithi083 ceph-mon[29421]: 6.f scrub starts 2024-09-20T19:10:27.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:27 smithi083 ceph-mon[29421]: 6.f scrub ok 2024-09-20T19:10:27.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:27 smithi083 ceph-mon[36260]: 4.15 scrub starts 2024-09-20T19:10:27.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:27 smithi083 ceph-mon[36260]: 4.15 scrub ok 2024-09-20T19:10:27.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:27 smithi083 ceph-mon[36260]: 6.f scrub starts 2024-09-20T19:10:27.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:27 smithi083 ceph-mon[36260]: 6.f scrub ok 2024-09-20T19:10:27.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:27 smithi142 ceph-mon[32508]: 4.15 scrub starts 2024-09-20T19:10:27.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:27 smithi142 ceph-mon[32508]: 4.15 scrub ok 2024-09-20T19:10:27.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:27 smithi142 ceph-mon[32508]: 6.f scrub starts 2024-09-20T19:10:27.395 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:27 smithi142 ceph-mon[32508]: 6.f scrub ok 2024-09-20T19:10:28.009 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:10:27 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:10:28.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:28 smithi083 ceph-mon[29421]: 3.c scrub starts 2024-09-20T19:10:28.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:28 smithi083 ceph-mon[29421]: 3.c scrub ok 2024-09-20T19:10:28.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:28 smithi083 ceph-mon[29421]: pgmap v1043: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:28.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:28 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:10:28.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:28 smithi083 ceph-mon[36260]: 3.c scrub starts 2024-09-20T19:10:28.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:28 smithi083 ceph-mon[36260]: 3.c scrub ok 2024-09-20T19:10:28.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:28 smithi083 ceph-mon[36260]: pgmap v1043: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:28.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:28 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:10:28.509 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:28 smithi142 ceph-mon[32508]: 3.c scrub starts 2024-09-20T19:10:28.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:28 smithi142 ceph-mon[32508]: 3.c scrub ok 2024-09-20T19:10:28.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:28 smithi142 ceph-mon[32508]: pgmap v1043: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:28.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:28 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:10:29.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:29 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 59/597 objects degraded (9.883%), 16 pgs degraded, 50 pgs undersized (PG_DEGRADED) 2024-09-20T19:10:29.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:29 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 59/597 objects degraded (9.883%), 16 pgs degraded, 50 pgs undersized (PG_DEGRADED) 2024-09-20T19:10:29.509 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:29 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 59/597 objects degraded (9.883%), 16 pgs degraded, 50 pgs undersized (PG_DEGRADED) 2024-09-20T19:10:30.289 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:30 smithi083 ceph-mon[29421]: pgmap v1044: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:30.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:30 smithi083 ceph-mon[36260]: pgmap v1044: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:30.509 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:30 smithi142 ceph-mon[32508]: pgmap v1044: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:30.722 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:10:30 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-0[198692]: 2024-09-20T19:10:30.340+0000 7ff82170b640 -1 osd.0 494 log_to_monitors true 2024-09-20T19:10:31.035 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:10:30 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:10:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:10:31.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:31 smithi083 ceph-mon[29421]: from='osd.0 v2:172.21.15.83:6801/444027075' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-20T19:10:31.290 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:10:31 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-0[198692]: 2024-09-20T19:10:31.054+0000 7ff8194b5640 -1 osd.0 494 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-20T19:10:31.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:31 smithi083 ceph-mon[36260]: from='osd.0 v2:172.21.15.83:6801/444027075' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-20T19:10:31.301 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-20T19:10:31.509 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:31 smithi142 ceph-mon[32508]: from='osd.0 v2:172.21.15.83:6801/444027075' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-20T19:10:31.559 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:10:32.066 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-20T19:10:32.066 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-20T19:10:32.066 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-20T19:10:32.066 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-20T19:10:32.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:32 smithi083 ceph-mon[29421]: from='osd.0 v2:172.21.15.83:6801/444027075' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-20T19:10:32.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:32 smithi083 ceph-mon[29421]: osdmap e509: 8 total, 7 up, 8 in 2024-09-20T19:10:32.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:32 smithi083 ceph-mon[29421]: from='osd.0 v2:172.21.15.83:6801/444027075' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]} : dispatch 2024-09-20T19:10:32.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:32 smithi083 ceph-mon[29421]: pgmap v1046: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:32.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:32 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:10:32.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:32 smithi083 ceph-mon[36260]: from='osd.0 v2:172.21.15.83:6801/444027075' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-20T19:10:32.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:32 smithi083 ceph-mon[36260]: osdmap e509: 8 total, 7 up, 8 in 2024-09-20T19:10:32.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:32 smithi083 ceph-mon[36260]: from='osd.0 v2:172.21.15.83:6801/444027075' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]} : dispatch 2024-09-20T19:10:32.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:32 smithi083 ceph-mon[36260]: pgmap v1046: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:32.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:32 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:10:32.509 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:32 smithi142 ceph-mon[32508]: from='osd.0 v2:172.21.15.83:6801/444027075' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-20T19:10:32.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:32 smithi142 ceph-mon[32508]: osdmap e509: 8 total, 7 up, 8 in 2024-09-20T19:10:32.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:32 smithi142 ceph-mon[32508]: from='osd.0 v2:172.21.15.83:6801/444027075' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]} : dispatch 2024-09-20T19:10:32.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:32 smithi142 ceph-mon[32508]: pgmap v1046: 142 pgs: 34 active+undersized, 16 active+undersized+degraded, 92 active+clean; 583 KiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 59/597 objects degraded (9.883%) 2024-09-20T19:10:32.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:32 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:10:32.788 INFO:tasks.thrashosds.thrasher:fixing pg num pool .mgr 2024-09-20T19:10:32.788 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd pool set .mgr pgp_num 11 2024-09-20T19:10:33.039 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:10:33.311 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:33 smithi083 ceph-mon[29421]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-20T19:10:33.311 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:33 smithi083 ceph-mon[29421]: osd.0 v2:172.21.15.83:6801/444027075 boot 2024-09-20T19:10:33.311 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:33 smithi083 ceph-mon[29421]: osdmap e510: 8 total, 8 up, 8 in 2024-09-20T19:10:33.311 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:33 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-20T19:10:33.311 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:33 smithi083 ceph-mon[29421]: osdmap e511: 8 total, 8 up, 8 in 2024-09-20T19:10:33.311 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:33 smithi083 ceph-mon[36260]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-20T19:10:33.312 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:33 smithi083 ceph-mon[36260]: osd.0 v2:172.21.15.83:6801/444027075 boot 2024-09-20T19:10:33.312 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:33 smithi083 ceph-mon[36260]: osdmap e510: 8 total, 8 up, 8 in 2024-09-20T19:10:33.312 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:33 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-20T19:10:33.312 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:33 smithi083 ceph-mon[36260]: osdmap e511: 8 total, 8 up, 8 in 2024-09-20T19:10:33.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:33 smithi142 ceph-mon[32508]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-20T19:10:33.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:33 smithi142 ceph-mon[32508]: osd.0 v2:172.21.15.83:6801/444027075 boot 2024-09-20T19:10:33.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:33 smithi142 ceph-mon[32508]: osdmap e510: 8 total, 8 up, 8 in 2024-09-20T19:10:33.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:33 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-20T19:10:33.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:33 smithi142 ceph-mon[32508]: osdmap e511: 8 total, 8 up, 8 in 2024-09-20T19:10:34.067 INFO:teuthology.orchestra.run.smithi083.stderr:set pool 1 pgp_num to 11 2024-09-20T19:10:34.412 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:34 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:10:34.413 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:34 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:10:34.413 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:34 smithi083 ceph-mon[29421]: pgmap v1049: 142 pgs: 7 peering, 28 active+undersized, 15 active+undersized+degraded, 92 active+clean; 583 KiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 852 B/s rd, 0 op/s; 54/597 objects degraded (9.045%) 2024-09-20T19:10:34.413 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:34 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1081818318' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "11"} : dispatch 2024-09-20T19:10:34.413 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:34 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:10:34.413 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:34 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:10:34.413 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:34 smithi083 ceph-mon[36260]: pgmap v1049: 142 pgs: 7 peering, 28 active+undersized, 15 active+undersized+degraded, 92 active+clean; 583 KiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 852 B/s rd, 0 op/s; 54/597 objects degraded (9.045%) 2024-09-20T19:10:34.413 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:34 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1081818318' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "11"} : dispatch 2024-09-20T19:10:34.509 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:34 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:10:34.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:34 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:10:34.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:34 smithi142 ceph-mon[32508]: pgmap v1049: 142 pgs: 7 peering, 28 active+undersized, 15 active+undersized+degraded, 92 active+clean; 583 KiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 852 B/s rd, 0 op/s; 54/597 objects degraded (9.045%) 2024-09-20T19:10:34.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:34 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1081818318' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "11"} : dispatch 2024-09-20T19:10:34.826 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-20T19:10:35.082 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:10:35.325 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:35 smithi083 ceph-mon[36260]: 6.d scrub starts 2024-09-20T19:10:35.325 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:35 smithi083 ceph-mon[36260]: 6.d scrub ok 2024-09-20T19:10:35.325 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:35 smithi083 ceph-mon[36260]: 6.11 scrub starts 2024-09-20T19:10:35.325 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:35 smithi083 ceph-mon[36260]: 6.11 scrub ok 2024-09-20T19:10:35.325 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:35 smithi083 ceph-mon[36260]: Health check update: Degraded data redundancy: 54/597 objects degraded (9.045%), 15 pgs degraded, 43 pgs undersized (PG_DEGRADED) 2024-09-20T19:10:35.326 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:35 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1081818318' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "11"}]': finished 2024-09-20T19:10:35.326 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:35 smithi083 ceph-mon[36260]: osdmap e512: 8 total, 8 up, 8 in 2024-09-20T19:10:35.326 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:35 smithi083 ceph-mon[36260]: 5.2 scrub starts 2024-09-20T19:10:35.326 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:35 smithi083 ceph-mon[36260]: 5.2 scrub ok 2024-09-20T19:10:35.326 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:35 smithi083 ceph-mon[29421]: 6.d scrub starts 2024-09-20T19:10:35.326 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:35 smithi083 ceph-mon[29421]: 6.d scrub ok 2024-09-20T19:10:35.326 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:35 smithi083 ceph-mon[29421]: 6.11 scrub starts 2024-09-20T19:10:35.326 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:35 smithi083 ceph-mon[29421]: 6.11 scrub ok 2024-09-20T19:10:35.326 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:35 smithi083 ceph-mon[29421]: Health check update: Degraded data redundancy: 54/597 objects degraded (9.045%), 15 pgs degraded, 43 pgs undersized (PG_DEGRADED) 2024-09-20T19:10:35.326 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:35 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1081818318' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "11"}]': finished 2024-09-20T19:10:35.326 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:35 smithi083 ceph-mon[29421]: osdmap e512: 8 total, 8 up, 8 in 2024-09-20T19:10:35.327 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:35 smithi083 ceph-mon[29421]: 5.2 scrub starts 2024-09-20T19:10:35.327 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:35 smithi083 ceph-mon[29421]: 5.2 scrub ok 2024-09-20T19:10:35.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:35 smithi142 ceph-mon[32508]: 6.d scrub starts 2024-09-20T19:10:35.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:35 smithi142 ceph-mon[32508]: 6.d scrub ok 2024-09-20T19:10:35.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:35 smithi142 ceph-mon[32508]: 6.11 scrub starts 2024-09-20T19:10:35.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:35 smithi142 ceph-mon[32508]: 6.11 scrub ok 2024-09-20T19:10:35.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:35 smithi142 ceph-mon[32508]: Health check update: Degraded data redundancy: 54/597 objects degraded (9.045%), 15 pgs degraded, 43 pgs undersized (PG_DEGRADED) 2024-09-20T19:10:35.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:35 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1081818318' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "11"}]': finished 2024-09-20T19:10:35.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:35 smithi142 ceph-mon[32508]: osdmap e512: 8 total, 8 up, 8 in 2024-09-20T19:10:35.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:35 smithi142 ceph-mon[32508]: 5.2 scrub starts 2024-09-20T19:10:35.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:35 smithi142 ceph-mon[32508]: 5.2 scrub ok 2024-09-20T19:10:35.542 INFO:teuthology.orchestra.run.smithi083.stdout:mon.a: {} 2024-09-20T19:10:35.542 INFO:teuthology.orchestra.run.smithi083.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-20T19:10:35.551 INFO:teuthology.orchestra.run.smithi083.stdout:mon.b: {} 2024-09-20T19:10:35.551 INFO:teuthology.orchestra.run.smithi083.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-20T19:10:35.560 INFO:teuthology.orchestra.run.smithi083.stdout:mon.c: {} 2024-09-20T19:10:35.560 INFO:teuthology.orchestra.run.smithi083.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-20T19:10:36.163 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:36 smithi083 ceph-mon[29421]: pgmap v1051: 142 pgs: 7 peering, 28 active+undersized, 15 active+undersized+degraded, 92 active+clean; 583 KiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 54/597 objects degraded (9.045%) 2024-09-20T19:10:36.164 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:36 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1423884765' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-20T19:10:36.164 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:36 smithi083 ceph-mon[36260]: pgmap v1051: 142 pgs: 7 peering, 28 active+undersized, 15 active+undersized+degraded, 92 active+clean; 583 KiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 54/597 objects degraded (9.045%) 2024-09-20T19:10:36.164 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:36 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1423884765' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-20T19:10:36.191 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd reweight 1 1 2024-09-20T19:10:36.439 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:10:36.509 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:36 smithi142 ceph-mon[32508]: pgmap v1051: 142 pgs: 7 peering, 28 active+undersized, 15 active+undersized+degraded, 92 active+clean; 583 KiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 54/597 objects degraded (9.045%) 2024-09-20T19:10:36.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:36 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1423884765' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-20T19:10:37.259 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:37 smithi142 ceph-mon[32508]: 3.12 scrub starts 2024-09-20T19:10:37.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:37 smithi142 ceph-mon[32508]: 3.12 scrub ok 2024-09-20T19:10:37.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:37 smithi142 ceph-mon[32508]: 4.1a scrub starts 2024-09-20T19:10:37.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:37 smithi142 ceph-mon[32508]: 4.1a scrub ok 2024-09-20T19:10:37.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:37 smithi142 ceph-mon[32508]: 3.b scrub starts 2024-09-20T19:10:37.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:37 smithi142 ceph-mon[32508]: 3.b scrub ok 2024-09-20T19:10:37.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:37 smithi142 ceph-mon[32508]: 4.7 scrub starts 2024-09-20T19:10:37.260 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:37 smithi142 ceph-mon[32508]: 4.7 scrub ok 2024-09-20T19:10:37.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:37 smithi083 ceph-mon[29421]: 3.12 scrub starts 2024-09-20T19:10:37.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:37 smithi083 ceph-mon[29421]: 3.12 scrub ok 2024-09-20T19:10:37.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:37 smithi083 ceph-mon[29421]: 4.1a scrub starts 2024-09-20T19:10:37.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:37 smithi083 ceph-mon[29421]: 4.1a scrub ok 2024-09-20T19:10:37.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:37 smithi083 ceph-mon[29421]: 3.b scrub starts 2024-09-20T19:10:37.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:37 smithi083 ceph-mon[29421]: 3.b scrub ok 2024-09-20T19:10:37.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:37 smithi083 ceph-mon[29421]: 4.7 scrub starts 2024-09-20T19:10:37.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:37 smithi083 ceph-mon[29421]: 4.7 scrub ok 2024-09-20T19:10:37.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:37 smithi083 ceph-mon[36260]: 3.12 scrub starts 2024-09-20T19:10:37.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:37 smithi083 ceph-mon[36260]: 3.12 scrub ok 2024-09-20T19:10:37.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:37 smithi083 ceph-mon[36260]: 4.1a scrub starts 2024-09-20T19:10:37.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:37 smithi083 ceph-mon[36260]: 4.1a scrub ok 2024-09-20T19:10:37.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:37 smithi083 ceph-mon[36260]: 3.b scrub starts 2024-09-20T19:10:37.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:37 smithi083 ceph-mon[36260]: 3.b scrub ok 2024-09-20T19:10:37.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:37 smithi083 ceph-mon[36260]: 4.7 scrub starts 2024-09-20T19:10:37.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:37 smithi083 ceph-mon[36260]: 4.7 scrub ok 2024-09-20T19:10:37.759 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:10:37 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:10:38.087 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.1 to 1 (10000) 2024-09-20T19:10:38.480 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:38 smithi083 ceph-mon[29421]: 3.7 scrub starts 2024-09-20T19:10:38.480 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:38 smithi083 ceph-mon[29421]: 3.7 scrub ok 2024-09-20T19:10:38.480 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:38 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2530935002' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-20T19:10:38.480 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:38 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-20T19:10:38.480 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:38 smithi083 ceph-mon[29421]: pgmap v1052: 142 pgs: 4 peering, 138 active+clean; 583 KiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 0 B/s, 5 objects/s recovering 2024-09-20T19:10:38.481 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:38 smithi083 ceph-mon[29421]: 5.c scrub starts 2024-09-20T19:10:38.481 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:38 smithi083 ceph-mon[29421]: 5.c scrub ok 2024-09-20T19:10:38.481 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:38 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:10:38.481 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:38 smithi083 ceph-mon[29421]: 1.6 scrub starts 2024-09-20T19:10:38.481 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:38 smithi083 ceph-mon[29421]: 1.6 scrub ok 2024-09-20T19:10:38.481 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:38 smithi083 ceph-mon[36260]: 3.7 scrub starts 2024-09-20T19:10:38.481 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:38 smithi083 ceph-mon[36260]: 3.7 scrub ok 2024-09-20T19:10:38.481 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:38 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2530935002' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-20T19:10:38.481 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:38 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-20T19:10:38.481 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:38 smithi083 ceph-mon[36260]: pgmap v1052: 142 pgs: 4 peering, 138 active+clean; 583 KiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 0 B/s, 5 objects/s recovering 2024-09-20T19:10:38.481 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:38 smithi083 ceph-mon[36260]: 5.c scrub starts 2024-09-20T19:10:38.482 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:38 smithi083 ceph-mon[36260]: 5.c scrub ok 2024-09-20T19:10:38.482 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:38 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:10:38.482 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:38 smithi083 ceph-mon[36260]: 1.6 scrub starts 2024-09-20T19:10:38.482 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:38 smithi083 ceph-mon[36260]: 1.6 scrub ok 2024-09-20T19:10:38.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:38 smithi142 ceph-mon[32508]: 3.7 scrub starts 2024-09-20T19:10:38.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:38 smithi142 ceph-mon[32508]: 3.7 scrub ok 2024-09-20T19:10:38.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:38 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2530935002' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-20T19:10:38.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:38 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-20T19:10:38.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:38 smithi142 ceph-mon[32508]: pgmap v1052: 142 pgs: 4 peering, 138 active+clean; 583 KiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 0 B/s, 5 objects/s recovering 2024-09-20T19:10:38.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:38 smithi142 ceph-mon[32508]: 5.c scrub starts 2024-09-20T19:10:38.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:38 smithi142 ceph-mon[32508]: 5.c scrub ok 2024-09-20T19:10:38.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:38 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:10:38.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:38 smithi142 ceph-mon[32508]: 1.6 scrub starts 2024-09-20T19:10:38.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:38 smithi142 ceph-mon[32508]: 1.6 scrub ok 2024-09-20T19:10:38.855 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd primary-affinity 1 1 2024-09-20T19:10:39.107 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:10:39.370 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:39 smithi083 ceph-mon[29421]: 5.8 scrub starts 2024-09-20T19:10:39.370 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:39 smithi083 ceph-mon[29421]: 5.8 scrub ok 2024-09-20T19:10:39.370 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:39 smithi083 ceph-mon[29421]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 54/597 objects degraded (9.045%), 15 pgs degraded, 43 pgs undersized) 2024-09-20T19:10:39.370 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:39 smithi083 ceph-mon[29421]: Cluster is now healthy 2024-09-20T19:10:39.370 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:39 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-20T19:10:39.370 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:39 smithi083 ceph-mon[29421]: 5.13 scrub starts 2024-09-20T19:10:39.370 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:39 smithi083 ceph-mon[29421]: osdmap e513: 8 total, 8 up, 8 in 2024-09-20T19:10:39.370 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:39 smithi083 ceph-mon[29421]: 5.13 scrub ok 2024-09-20T19:10:39.371 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:39 smithi083 ceph-mon[36260]: 5.8 scrub starts 2024-09-20T19:10:39.371 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:39 smithi083 ceph-mon[36260]: 5.8 scrub ok 2024-09-20T19:10:39.371 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:39 smithi083 ceph-mon[36260]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 54/597 objects degraded (9.045%), 15 pgs degraded, 43 pgs undersized) 2024-09-20T19:10:39.371 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:39 smithi083 ceph-mon[36260]: Cluster is now healthy 2024-09-20T19:10:39.371 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:39 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-20T19:10:39.371 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:39 smithi083 ceph-mon[36260]: 5.13 scrub starts 2024-09-20T19:10:39.371 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:39 smithi083 ceph-mon[36260]: osdmap e513: 8 total, 8 up, 8 in 2024-09-20T19:10:39.371 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:39 smithi083 ceph-mon[36260]: 5.13 scrub ok 2024-09-20T19:10:39.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:39 smithi142 ceph-mon[32508]: 5.8 scrub starts 2024-09-20T19:10:39.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:39 smithi142 ceph-mon[32508]: 5.8 scrub ok 2024-09-20T19:10:39.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:39 smithi142 ceph-mon[32508]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 54/597 objects degraded (9.045%), 15 pgs degraded, 43 pgs undersized) 2024-09-20T19:10:39.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:39 smithi142 ceph-mon[32508]: Cluster is now healthy 2024-09-20T19:10:39.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:39 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-20T19:10:39.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:39 smithi142 ceph-mon[32508]: 5.13 scrub starts 2024-09-20T19:10:39.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:39 smithi142 ceph-mon[32508]: osdmap e513: 8 total, 8 up, 8 in 2024-09-20T19:10:39.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:39 smithi142 ceph-mon[32508]: 5.13 scrub ok 2024-09-20T19:10:40.091 INFO:teuthology.orchestra.run.smithi083.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-20T19:10:40.456 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:40 smithi083 ceph-mon[29421]: 5.b scrub starts 2024-09-20T19:10:40.456 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:40 smithi083 ceph-mon[29421]: 5.b scrub ok 2024-09-20T19:10:40.456 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:40 smithi083 ceph-mon[29421]: 3.d scrub starts 2024-09-20T19:10:40.456 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:40 smithi083 ceph-mon[29421]: 3.d scrub ok 2024-09-20T19:10:40.456 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:40 smithi083 ceph-mon[29421]: pgmap v1054: 142 pgs: 142 active+clean; 583 KiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 329 B/s rd, 0 op/s; 0 B/s, 6 objects/s recovering 2024-09-20T19:10:40.456 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:40 smithi083 ceph-mon[29421]: 1.5 scrub starts 2024-09-20T19:10:40.456 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:40 smithi083 ceph-mon[29421]: 1.5 scrub ok 2024-09-20T19:10:40.456 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:40 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1638809138' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 1, "weight": 1.0} : dispatch 2024-09-20T19:10:40.456 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:40 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 1, "weight": 1.0} : dispatch 2024-09-20T19:10:40.457 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:40 smithi083 ceph-mon[36260]: 5.b scrub starts 2024-09-20T19:10:40.457 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:40 smithi083 ceph-mon[36260]: 5.b scrub ok 2024-09-20T19:10:40.457 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:40 smithi083 ceph-mon[36260]: 3.d scrub starts 2024-09-20T19:10:40.457 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:40 smithi083 ceph-mon[36260]: 3.d scrub ok 2024-09-20T19:10:40.457 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:40 smithi083 ceph-mon[36260]: pgmap v1054: 142 pgs: 142 active+clean; 583 KiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 329 B/s rd, 0 op/s; 0 B/s, 6 objects/s recovering 2024-09-20T19:10:40.457 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:40 smithi083 ceph-mon[36260]: 1.5 scrub starts 2024-09-20T19:10:40.457 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:40 smithi083 ceph-mon[36260]: 1.5 scrub ok 2024-09-20T19:10:40.457 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:40 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1638809138' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 1, "weight": 1.0} : dispatch 2024-09-20T19:10:40.457 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:40 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 1, "weight": 1.0} : dispatch 2024-09-20T19:10:40.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:40 smithi142 ceph-mon[32508]: 5.b scrub starts 2024-09-20T19:10:40.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:40 smithi142 ceph-mon[32508]: 5.b scrub ok 2024-09-20T19:10:40.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:40 smithi142 ceph-mon[32508]: 3.d scrub starts 2024-09-20T19:10:40.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:40 smithi142 ceph-mon[32508]: 3.d scrub ok 2024-09-20T19:10:40.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:40 smithi142 ceph-mon[32508]: pgmap v1054: 142 pgs: 142 active+clean; 583 KiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 329 B/s rd, 0 op/s; 0 B/s, 6 objects/s recovering 2024-09-20T19:10:40.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:40 smithi142 ceph-mon[32508]: 1.5 scrub starts 2024-09-20T19:10:40.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:40 smithi142 ceph-mon[32508]: 1.5 scrub ok 2024-09-20T19:10:40.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:40 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1638809138' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 1, "weight": 1.0} : dispatch 2024-09-20T19:10:40.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:40 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 1, "weight": 1.0} : dispatch 2024-09-20T19:10:40.855 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd reweight 2 1 2024-09-20T19:10:41.039 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:10:40 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:10:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:10:41.116 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:10:41.429 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:41 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]': finished 2024-09-20T19:10:41.429 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:41 smithi083 ceph-mon[29421]: osdmap e514: 8 total, 8 up, 8 in 2024-09-20T19:10:41.429 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:41 smithi083 ceph-mon[29421]: 3.1b scrub starts 2024-09-20T19:10:41.430 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:41 smithi083 ceph-mon[29421]: 3.1b scrub ok 2024-09-20T19:10:41.430 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:41 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]': finished 2024-09-20T19:10:41.430 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:41 smithi083 ceph-mon[36260]: osdmap e514: 8 total, 8 up, 8 in 2024-09-20T19:10:41.430 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:41 smithi083 ceph-mon[36260]: 3.1b scrub starts 2024-09-20T19:10:41.430 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:41 smithi083 ceph-mon[36260]: 3.1b scrub ok 2024-09-20T19:10:41.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:41 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]': finished 2024-09-20T19:10:41.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:41 smithi142 ceph-mon[32508]: osdmap e514: 8 total, 8 up, 8 in 2024-09-20T19:10:41.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:41 smithi142 ceph-mon[32508]: 3.1b scrub starts 2024-09-20T19:10:41.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:41 smithi142 ceph-mon[32508]: 3.1b scrub ok 2024-09-20T19:10:42.094 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.2 to 1 (10000) 2024-09-20T19:10:42.473 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:42 smithi083 ceph-mon[36260]: 2.1 scrub starts 2024-09-20T19:10:42.473 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:42 smithi083 ceph-mon[36260]: 2.1 scrub ok 2024-09-20T19:10:42.473 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:42 smithi083 ceph-mon[36260]: osdmap e515: 8 total, 8 up, 8 in 2024-09-20T19:10:42.473 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:42 smithi083 ceph-mon[36260]: pgmap v1057: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 341 B/s rd, 0 op/s; 0 B/s, 6 objects/s recovering 2024-09-20T19:10:42.474 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:42 smithi083 ceph-mon[36260]: 5.1d scrub starts 2024-09-20T19:10:42.474 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:42 smithi083 ceph-mon[36260]: 5.1d scrub ok 2024-09-20T19:10:42.474 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:42 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/744926285' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-20T19:10:42.474 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:42 smithi083 ceph-mon[36260]: 3.10 scrub starts 2024-09-20T19:10:42.474 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:42 smithi083 ceph-mon[36260]: 3.10 scrub ok 2024-09-20T19:10:42.474 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:42 smithi083 ceph-mon[29421]: 2.1 scrub starts 2024-09-20T19:10:42.474 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:42 smithi083 ceph-mon[29421]: 2.1 scrub ok 2024-09-20T19:10:42.474 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:42 smithi083 ceph-mon[29421]: osdmap e515: 8 total, 8 up, 8 in 2024-09-20T19:10:42.474 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:42 smithi083 ceph-mon[29421]: pgmap v1057: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 341 B/s rd, 0 op/s; 0 B/s, 6 objects/s recovering 2024-09-20T19:10:42.474 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:42 smithi083 ceph-mon[29421]: 5.1d scrub starts 2024-09-20T19:10:42.474 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:42 smithi083 ceph-mon[29421]: 5.1d scrub ok 2024-09-20T19:10:42.475 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:42 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/744926285' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-20T19:10:42.475 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:42 smithi083 ceph-mon[29421]: 3.10 scrub starts 2024-09-20T19:10:42.475 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:42 smithi083 ceph-mon[29421]: 3.10 scrub ok 2024-09-20T19:10:42.509 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:42 smithi142 ceph-mon[32508]: 2.1 scrub starts 2024-09-20T19:10:42.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:42 smithi142 ceph-mon[32508]: 2.1 scrub ok 2024-09-20T19:10:42.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:42 smithi142 ceph-mon[32508]: osdmap e515: 8 total, 8 up, 8 in 2024-09-20T19:10:42.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:42 smithi142 ceph-mon[32508]: pgmap v1057: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 341 B/s rd, 0 op/s; 0 B/s, 6 objects/s recovering 2024-09-20T19:10:42.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:42 smithi142 ceph-mon[32508]: 5.1d scrub starts 2024-09-20T19:10:42.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:42 smithi142 ceph-mon[32508]: 5.1d scrub ok 2024-09-20T19:10:42.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:42 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/744926285' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-20T19:10:42.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:42 smithi142 ceph-mon[32508]: 3.10 scrub starts 2024-09-20T19:10:42.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:42 smithi142 ceph-mon[32508]: 3.10 scrub ok 2024-09-20T19:10:42.870 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd primary-affinity 2 1 2024-09-20T19:10:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:10:43.368 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:43 smithi083 ceph-mon[29421]: 4.1c scrub starts 2024-09-20T19:10:43.368 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:43 smithi083 ceph-mon[29421]: 4.1c scrub ok 2024-09-20T19:10:43.368 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:43 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/744926285' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-20T19:10:43.368 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:43 smithi083 ceph-mon[29421]: osdmap e516: 8 total, 8 up, 8 in 2024-09-20T19:10:43.368 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:43 smithi083 ceph-mon[29421]: 4.12 scrub starts 2024-09-20T19:10:43.368 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:43 smithi083 ceph-mon[29421]: 4.12 scrub ok 2024-09-20T19:10:43.368 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:43 smithi083 ceph-mon[36260]: 4.1c scrub starts 2024-09-20T19:10:43.369 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:43 smithi083 ceph-mon[36260]: 4.1c scrub ok 2024-09-20T19:10:43.369 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:43 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/744926285' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-20T19:10:43.369 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:43 smithi083 ceph-mon[36260]: osdmap e516: 8 total, 8 up, 8 in 2024-09-20T19:10:43.369 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:43 smithi083 ceph-mon[36260]: 4.12 scrub starts 2024-09-20T19:10:43.369 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:43 smithi083 ceph-mon[36260]: 4.12 scrub ok 2024-09-20T19:10:43.509 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:43 smithi142 ceph-mon[32508]: 4.1c scrub starts 2024-09-20T19:10:43.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:43 smithi142 ceph-mon[32508]: 4.1c scrub ok 2024-09-20T19:10:43.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:43 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/744926285' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-20T19:10:43.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:43 smithi142 ceph-mon[32508]: osdmap e516: 8 total, 8 up, 8 in 2024-09-20T19:10:43.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:43 smithi142 ceph-mon[32508]: 4.12 scrub starts 2024-09-20T19:10:43.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:43 smithi142 ceph-mon[32508]: 4.12 scrub ok 2024-09-20T19:10:44.107 INFO:teuthology.orchestra.run.smithi083.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-20T19:10:44.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:44 smithi142 ceph-mon[32508]: 6.a scrub starts 2024-09-20T19:10:44.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:44 smithi142 ceph-mon[32508]: 6.a scrub ok 2024-09-20T19:10:44.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:44 smithi142 ceph-mon[32508]: 3.f scrub starts 2024-09-20T19:10:44.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:44 smithi142 ceph-mon[32508]: 3.f scrub ok 2024-09-20T19:10:44.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:44 smithi142 ceph-mon[32508]: pgmap v1059: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 988 B/s rd, 0 op/s 2024-09-20T19:10:44.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:44 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2409253097' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 2, "weight": 1.0} : dispatch 2024-09-20T19:10:44.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:44 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 2, "weight": 1.0} : dispatch 2024-09-20T19:10:44.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:44 smithi083 ceph-mon[29421]: 6.a scrub starts 2024-09-20T19:10:44.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:44 smithi083 ceph-mon[29421]: 6.a scrub ok 2024-09-20T19:10:44.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:44 smithi083 ceph-mon[29421]: 3.f scrub starts 2024-09-20T19:10:44.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:44 smithi083 ceph-mon[29421]: 3.f scrub ok 2024-09-20T19:10:44.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:44 smithi083 ceph-mon[29421]: pgmap v1059: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 988 B/s rd, 0 op/s 2024-09-20T19:10:44.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:44 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2409253097' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 2, "weight": 1.0} : dispatch 2024-09-20T19:10:44.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:44 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 2, "weight": 1.0} : dispatch 2024-09-20T19:10:44.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:44 smithi083 ceph-mon[36260]: 6.a scrub starts 2024-09-20T19:10:44.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:44 smithi083 ceph-mon[36260]: 6.a scrub ok 2024-09-20T19:10:44.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:44 smithi083 ceph-mon[36260]: 3.f scrub starts 2024-09-20T19:10:44.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:44 smithi083 ceph-mon[36260]: 3.f scrub ok 2024-09-20T19:10:44.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:44 smithi083 ceph-mon[36260]: pgmap v1059: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 988 B/s rd, 0 op/s 2024-09-20T19:10:44.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:44 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2409253097' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 2, "weight": 1.0} : dispatch 2024-09-20T19:10:44.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:44 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 2, "weight": 1.0} : dispatch 2024-09-20T19:10:44.990 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd reweight 3 1 2024-09-20T19:10:45.247 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:10:45.289 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:45 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:10:45.289 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:45 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:10:45.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:45 smithi083 ceph-mon[29421]: 2.0 scrub starts 2024-09-20T19:10:45.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:45 smithi083 ceph-mon[29421]: 2.0 scrub ok 2024-09-20T19:10:45.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:45 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]': finished 2024-09-20T19:10:45.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:45 smithi083 ceph-mon[29421]: osdmap e517: 8 total, 8 up, 8 in 2024-09-20T19:10:45.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:45 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:10:45.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:45 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:10:45.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:45 smithi083 ceph-mon[36260]: 2.0 scrub starts 2024-09-20T19:10:45.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:45 smithi083 ceph-mon[36260]: 2.0 scrub ok 2024-09-20T19:10:45.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:45 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]': finished 2024-09-20T19:10:45.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:45 smithi083 ceph-mon[36260]: osdmap e517: 8 total, 8 up, 8 in 2024-09-20T19:10:45.509 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:45 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:10:45.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:45 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:10:45.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:45 smithi142 ceph-mon[32508]: 2.0 scrub starts 2024-09-20T19:10:45.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:45 smithi142 ceph-mon[32508]: 2.0 scrub ok 2024-09-20T19:10:45.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:45 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]': finished 2024-09-20T19:10:45.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:45 smithi142 ceph-mon[32508]: osdmap e517: 8 total, 8 up, 8 in 2024-09-20T19:10:46.289 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:46 smithi083 ceph-mon[29421]: 1.0 scrub starts 2024-09-20T19:10:46.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:46 smithi083 ceph-mon[29421]: 1.0 scrub ok 2024-09-20T19:10:46.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:46 smithi083 ceph-mon[29421]: pgmap v1061: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 989 B/s rd, 0 op/s 2024-09-20T19:10:46.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:46 smithi083 ceph-mon[29421]: 5.1e scrub starts 2024-09-20T19:10:46.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:46 smithi083 ceph-mon[29421]: 5.1e scrub ok 2024-09-20T19:10:46.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:46 smithi083 ceph-mon[36260]: 1.0 scrub starts 2024-09-20T19:10:46.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:46 smithi083 ceph-mon[36260]: 1.0 scrub ok 2024-09-20T19:10:46.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:46 smithi083 ceph-mon[36260]: pgmap v1061: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 989 B/s rd, 0 op/s 2024-09-20T19:10:46.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:46 smithi083 ceph-mon[36260]: 5.1e scrub starts 2024-09-20T19:10:46.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:46 smithi083 ceph-mon[36260]: 5.1e scrub ok 2024-09-20T19:10:46.509 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:46 smithi142 ceph-mon[32508]: 1.0 scrub starts 2024-09-20T19:10:46.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:46 smithi142 ceph-mon[32508]: 1.0 scrub ok 2024-09-20T19:10:46.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:46 smithi142 ceph-mon[32508]: pgmap v1061: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 989 B/s rd, 0 op/s 2024-09-20T19:10:46.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:46 smithi142 ceph-mon[32508]: 5.1e scrub starts 2024-09-20T19:10:46.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:46 smithi142 ceph-mon[32508]: 5.1e scrub ok 2024-09-20T19:10:47.124 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.3 to 1 (10000) 2024-09-20T19:10:47.492 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:47 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/235295321' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-20T19:10:47.492 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:47 smithi083 ceph-mon[29421]: 3.19 scrub starts 2024-09-20T19:10:47.492 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:47 smithi083 ceph-mon[29421]: 3.19 scrub ok 2024-09-20T19:10:47.492 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:47 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:10:47.492 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:47 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:10:47.493 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:47 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/235295321' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-20T19:10:47.493 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:47 smithi083 ceph-mon[36260]: 3.19 scrub starts 2024-09-20T19:10:47.493 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:47 smithi083 ceph-mon[36260]: 3.19 scrub ok 2024-09-20T19:10:47.493 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:47 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:10:47.493 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:47 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:10:47.509 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:47 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/235295321' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-20T19:10:47.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:47 smithi142 ceph-mon[32508]: 3.19 scrub starts 2024-09-20T19:10:47.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:47 smithi142 ceph-mon[32508]: 3.19 scrub ok 2024-09-20T19:10:47.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:47 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:10:47.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:47 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:10:47.879 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd primary-affinity 3 1 2024-09-20T19:10:48.009 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:10:47 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:10:48.140 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:10:48.397 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:48 smithi083 ceph-mon[29421]: 1.2 scrub starts 2024-09-20T19:10:48.397 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:48 smithi083 ceph-mon[29421]: 1.2 scrub ok 2024-09-20T19:10:48.397 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:48 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/235295321' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-20T19:10:48.397 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:48 smithi083 ceph-mon[29421]: osdmap e518: 8 total, 8 up, 8 in 2024-09-20T19:10:48.397 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:48 smithi083 ceph-mon[29421]: pgmap v1063: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-20T19:10:48.397 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:48 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:10:48.398 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:48 smithi083 ceph-mon[36260]: 1.2 scrub starts 2024-09-20T19:10:48.398 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:48 smithi083 ceph-mon[36260]: 1.2 scrub ok 2024-09-20T19:10:48.398 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:48 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/235295321' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-20T19:10:48.398 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:48 smithi083 ceph-mon[36260]: osdmap e518: 8 total, 8 up, 8 in 2024-09-20T19:10:48.398 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:48 smithi083 ceph-mon[36260]: pgmap v1063: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-20T19:10:48.398 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:48 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:10:48.509 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:48 smithi142 ceph-mon[32508]: 1.2 scrub starts 2024-09-20T19:10:48.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:48 smithi142 ceph-mon[32508]: 1.2 scrub ok 2024-09-20T19:10:48.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:48 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/235295321' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-20T19:10:48.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:48 smithi142 ceph-mon[32508]: osdmap e518: 8 total, 8 up, 8 in 2024-09-20T19:10:48.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:48 smithi142 ceph-mon[32508]: pgmap v1063: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-20T19:10:48.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:48 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:10:49.130 INFO:teuthology.orchestra.run.smithi083.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-20T19:10:49.475 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:49 smithi083 ceph-mon[29421]: 5.e scrub starts 2024-09-20T19:10:49.475 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:49 smithi083 ceph-mon[29421]: 5.e scrub ok 2024-09-20T19:10:49.475 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:49 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:10:49.475 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:49 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:10:49.475 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:49 smithi083 ceph-mon[29421]: 4.0 scrub starts 2024-09-20T19:10:49.475 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:49 smithi083 ceph-mon[29421]: 4.0 scrub ok 2024-09-20T19:10:49.476 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:49 smithi083 ceph-mon[29421]: osdmap e519: 8 total, 8 up, 8 in 2024-09-20T19:10:49.476 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:49 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3433677629' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 3, "weight": 1.0} : dispatch 2024-09-20T19:10:49.476 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:49 smithi083 ceph-mon[36260]: 5.e scrub starts 2024-09-20T19:10:49.476 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:49 smithi083 ceph-mon[36260]: 5.e scrub ok 2024-09-20T19:10:49.476 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:49 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:10:49.476 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:49 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:10:49.476 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:49 smithi083 ceph-mon[36260]: 4.0 scrub starts 2024-09-20T19:10:49.476 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:49 smithi083 ceph-mon[36260]: 4.0 scrub ok 2024-09-20T19:10:49.476 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:49 smithi083 ceph-mon[36260]: osdmap e519: 8 total, 8 up, 8 in 2024-09-20T19:10:49.476 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:49 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3433677629' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 3, "weight": 1.0} : dispatch 2024-09-20T19:10:49.509 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:49 smithi142 ceph-mon[32508]: 5.e scrub starts 2024-09-20T19:10:49.509 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:49 smithi142 ceph-mon[32508]: 5.e scrub ok 2024-09-20T19:10:49.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:49 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:10:49.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:49 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:10:49.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:49 smithi142 ceph-mon[32508]: 4.0 scrub starts 2024-09-20T19:10:49.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:49 smithi142 ceph-mon[32508]: 4.0 scrub ok 2024-09-20T19:10:49.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:49 smithi142 ceph-mon[32508]: osdmap e519: 8 total, 8 up, 8 in 2024-09-20T19:10:49.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:49 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3433677629' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 3, "weight": 1.0} : dispatch 2024-09-20T19:10:49.883 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd reweight 5 1 2024-09-20T19:10:50.134 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:10:50.408 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:50 smithi083 ceph-mon[29421]: 3.3 scrub starts 2024-09-20T19:10:50.408 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:50 smithi083 ceph-mon[29421]: 3.3 scrub ok 2024-09-20T19:10:50.409 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:50 smithi083 ceph-mon[29421]: 5.7 scrub starts 2024-09-20T19:10:50.409 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:50 smithi083 ceph-mon[29421]: 5.7 scrub ok 2024-09-20T19:10:50.409 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:50 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3433677629' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]': finished 2024-09-20T19:10:50.409 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:50 smithi083 ceph-mon[29421]: osdmap e520: 8 total, 8 up, 8 in 2024-09-20T19:10:50.409 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:50 smithi083 ceph-mon[29421]: pgmap v1066: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 992 B/s rd, 0 op/s 2024-09-20T19:10:50.409 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:50 smithi083 ceph-mon[36260]: 3.3 scrub starts 2024-09-20T19:10:50.409 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:50 smithi083 ceph-mon[36260]: 3.3 scrub ok 2024-09-20T19:10:50.409 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:50 smithi083 ceph-mon[36260]: 5.7 scrub starts 2024-09-20T19:10:50.409 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:50 smithi083 ceph-mon[36260]: 5.7 scrub ok 2024-09-20T19:10:50.409 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:50 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3433677629' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]': finished 2024-09-20T19:10:50.409 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:50 smithi083 ceph-mon[36260]: osdmap e520: 8 total, 8 up, 8 in 2024-09-20T19:10:50.410 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:50 smithi083 ceph-mon[36260]: pgmap v1066: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 992 B/s rd, 0 op/s 2024-09-20T19:10:50.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:50 smithi142 ceph-mon[32508]: 3.3 scrub starts 2024-09-20T19:10:50.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:50 smithi142 ceph-mon[32508]: 3.3 scrub ok 2024-09-20T19:10:50.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:50 smithi142 ceph-mon[32508]: 5.7 scrub starts 2024-09-20T19:10:50.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:50 smithi142 ceph-mon[32508]: 5.7 scrub ok 2024-09-20T19:10:50.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:50 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3433677629' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]': finished 2024-09-20T19:10:50.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:50 smithi142 ceph-mon[32508]: osdmap e520: 8 total, 8 up, 8 in 2024-09-20T19:10:50.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:50 smithi142 ceph-mon[32508]: pgmap v1066: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 992 B/s rd, 0 op/s 2024-09-20T19:10:50.790 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:10:50 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:10:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:10:51.142 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.5 to 1 (10000) 2024-09-20T19:10:51.469 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:51 smithi083 ceph-mon[29421]: 5.3 scrub starts 2024-09-20T19:10:51.469 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:51 smithi083 ceph-mon[29421]: 5.3 scrub ok 2024-09-20T19:10:51.469 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:51 smithi083 ceph-mon[29421]: 3.18 scrub starts 2024-09-20T19:10:51.469 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:51 smithi083 ceph-mon[29421]: 3.18 scrub ok 2024-09-20T19:10:51.469 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:51 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2782491420' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-20T19:10:51.469 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:51 smithi083 ceph-mon[29421]: 5.1a scrub starts 2024-09-20T19:10:51.469 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:51 smithi083 ceph-mon[29421]: 5.1a scrub ok 2024-09-20T19:10:51.470 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:51 smithi083 ceph-mon[36260]: 5.3 scrub starts 2024-09-20T19:10:51.470 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:51 smithi083 ceph-mon[36260]: 5.3 scrub ok 2024-09-20T19:10:51.470 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:51 smithi083 ceph-mon[36260]: 3.18 scrub starts 2024-09-20T19:10:51.470 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:51 smithi083 ceph-mon[36260]: 3.18 scrub ok 2024-09-20T19:10:51.470 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:51 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2782491420' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-20T19:10:51.470 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:51 smithi083 ceph-mon[36260]: 5.1a scrub starts 2024-09-20T19:10:51.470 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:51 smithi083 ceph-mon[36260]: 5.1a scrub ok 2024-09-20T19:10:51.510 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 19:10:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=cleanup t=2024-09-20T19:10:51.217545261Z level=info msg="Completed cleanup jobs" duration=3.414625ms 2024-09-20T19:10:51.510 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 19:10:51 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=plugins.update.checker t=2024-09-20T19:10:51.370174474Z level=info msg="Update check succeeded" duration=64.148037ms 2024-09-20T19:10:51.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:51 smithi142 ceph-mon[32508]: 5.3 scrub starts 2024-09-20T19:10:51.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:51 smithi142 ceph-mon[32508]: 5.3 scrub ok 2024-09-20T19:10:51.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:51 smithi142 ceph-mon[32508]: 3.18 scrub starts 2024-09-20T19:10:51.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:51 smithi142 ceph-mon[32508]: 3.18 scrub ok 2024-09-20T19:10:51.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:51 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2782491420' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-20T19:10:51.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:51 smithi142 ceph-mon[32508]: 5.1a scrub starts 2024-09-20T19:10:51.511 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:51 smithi142 ceph-mon[32508]: 5.1a scrub ok 2024-09-20T19:10:51.823 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd primary-affinity 5 1 2024-09-20T19:10:52.067 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:10:52.439 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:52 smithi083 ceph-mon[29421]: 3.1 scrub starts 2024-09-20T19:10:52.439 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:52 smithi083 ceph-mon[29421]: 3.1 scrub ok 2024-09-20T19:10:52.439 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:52 smithi083 ceph-mon[29421]: 5.15 scrub starts 2024-09-20T19:10:52.439 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:52 smithi083 ceph-mon[29421]: 5.15 scrub ok 2024-09-20T19:10:52.439 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:52 smithi083 ceph-mon[29421]: 6.13 scrub starts 2024-09-20T19:10:52.440 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:52 smithi083 ceph-mon[29421]: 6.13 scrub ok 2024-09-20T19:10:52.440 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:52 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2782491420' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-20T19:10:52.440 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:52 smithi083 ceph-mon[29421]: osdmap e521: 8 total, 8 up, 8 in 2024-09-20T19:10:52.440 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:52 smithi083 ceph-mon[29421]: pgmap v1068: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail 2024-09-20T19:10:52.440 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:52 smithi083 ceph-mon[29421]: 6.15 scrub starts 2024-09-20T19:10:52.440 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:52 smithi083 ceph-mon[29421]: 6.15 scrub ok 2024-09-20T19:10:52.440 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:52 smithi083 ceph-mon[36260]: 3.1 scrub starts 2024-09-20T19:10:52.440 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:52 smithi083 ceph-mon[36260]: 3.1 scrub ok 2024-09-20T19:10:52.440 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:52 smithi083 ceph-mon[36260]: 5.15 scrub starts 2024-09-20T19:10:52.440 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:52 smithi083 ceph-mon[36260]: 5.15 scrub ok 2024-09-20T19:10:52.440 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:52 smithi083 ceph-mon[36260]: 6.13 scrub starts 2024-09-20T19:10:52.440 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:52 smithi083 ceph-mon[36260]: 6.13 scrub ok 2024-09-20T19:10:52.441 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:52 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2782491420' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-20T19:10:52.441 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:52 smithi083 ceph-mon[36260]: osdmap e521: 8 total, 8 up, 8 in 2024-09-20T19:10:52.441 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:52 smithi083 ceph-mon[36260]: pgmap v1068: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail 2024-09-20T19:10:52.441 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:52 smithi083 ceph-mon[36260]: 6.15 scrub starts 2024-09-20T19:10:52.441 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:52 smithi083 ceph-mon[36260]: 6.15 scrub ok 2024-09-20T19:10:52.509 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:52 smithi142 ceph-mon[32508]: 3.1 scrub starts 2024-09-20T19:10:52.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:52 smithi142 ceph-mon[32508]: 3.1 scrub ok 2024-09-20T19:10:52.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:52 smithi142 ceph-mon[32508]: 5.15 scrub starts 2024-09-20T19:10:52.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:52 smithi142 ceph-mon[32508]: 5.15 scrub ok 2024-09-20T19:10:52.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:52 smithi142 ceph-mon[32508]: 6.13 scrub starts 2024-09-20T19:10:52.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:52 smithi142 ceph-mon[32508]: 6.13 scrub ok 2024-09-20T19:10:52.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:52 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2782491420' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-20T19:10:52.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:52 smithi142 ceph-mon[32508]: osdmap e521: 8 total, 8 up, 8 in 2024-09-20T19:10:52.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:52 smithi142 ceph-mon[32508]: pgmap v1068: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail 2024-09-20T19:10:52.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:52 smithi142 ceph-mon[32508]: 6.15 scrub starts 2024-09-20T19:10:52.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:52 smithi142 ceph-mon[32508]: 6.15 scrub ok 2024-09-20T19:10:53.155 INFO:teuthology.orchestra.run.smithi083.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-20T19:10:53.445 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:53 smithi083 ceph-mon[29421]: 6.9 scrub starts 2024-09-20T19:10:53.445 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:53 smithi083 ceph-mon[29421]: 6.9 scrub ok 2024-09-20T19:10:53.445 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:53 smithi083 ceph-mon[29421]: 4.d scrub starts 2024-09-20T19:10:53.445 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:53 smithi083 ceph-mon[29421]: 4.d scrub ok 2024-09-20T19:10:53.445 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:53 smithi083 ceph-mon[29421]: 5.1b scrub starts 2024-09-20T19:10:53.445 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:53 smithi083 ceph-mon[29421]: 5.1b scrub ok 2024-09-20T19:10:53.445 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:53 smithi083 ceph-mon[29421]: 1.4 scrub starts 2024-09-20T19:10:53.445 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:53 smithi083 ceph-mon[29421]: 1.4 scrub ok 2024-09-20T19:10:53.445 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:53 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1042584849' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 5, "weight": 1.0} : dispatch 2024-09-20T19:10:53.446 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:53 smithi083 ceph-mon[36260]: 6.9 scrub starts 2024-09-20T19:10:53.446 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:53 smithi083 ceph-mon[36260]: 6.9 scrub ok 2024-09-20T19:10:53.446 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:53 smithi083 ceph-mon[36260]: 4.d scrub starts 2024-09-20T19:10:53.446 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:53 smithi083 ceph-mon[36260]: 4.d scrub ok 2024-09-20T19:10:53.446 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:53 smithi083 ceph-mon[36260]: 5.1b scrub starts 2024-09-20T19:10:53.446 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:53 smithi083 ceph-mon[36260]: 5.1b scrub ok 2024-09-20T19:10:53.446 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:53 smithi083 ceph-mon[36260]: 1.4 scrub starts 2024-09-20T19:10:53.446 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:53 smithi083 ceph-mon[36260]: 1.4 scrub ok 2024-09-20T19:10:53.446 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:53 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1042584849' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 5, "weight": 1.0} : dispatch 2024-09-20T19:10:53.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:53 smithi142 ceph-mon[32508]: 6.9 scrub starts 2024-09-20T19:10:53.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:53 smithi142 ceph-mon[32508]: 6.9 scrub ok 2024-09-20T19:10:53.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:53 smithi142 ceph-mon[32508]: 4.d scrub starts 2024-09-20T19:10:53.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:53 smithi142 ceph-mon[32508]: 4.d scrub ok 2024-09-20T19:10:53.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:53 smithi142 ceph-mon[32508]: 5.1b scrub starts 2024-09-20T19:10:53.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:53 smithi142 ceph-mon[32508]: 5.1b scrub ok 2024-09-20T19:10:53.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:53 smithi142 ceph-mon[32508]: 1.4 scrub starts 2024-09-20T19:10:53.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:53 smithi142 ceph-mon[32508]: 1.4 scrub ok 2024-09-20T19:10:53.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:53 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1042584849' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 5, "weight": 1.0} : dispatch 2024-09-20T19:10:53.949 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd reweight 6 1 2024-09-20T19:10:54.207 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:10:54.289 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:54 smithi083 ceph-mon[29421]: 1.1 scrub starts 2024-09-20T19:10:54.289 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:54 smithi083 ceph-mon[29421]: 1.1 scrub ok 2024-09-20T19:10:54.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:54 smithi083 ceph-mon[29421]: 4.1d scrub starts 2024-09-20T19:10:54.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:54 smithi083 ceph-mon[29421]: 4.1d scrub ok 2024-09-20T19:10:54.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:54 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1042584849' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]': finished 2024-09-20T19:10:54.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:54 smithi083 ceph-mon[29421]: osdmap e522: 8 total, 8 up, 8 in 2024-09-20T19:10:54.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:54 smithi083 ceph-mon[29421]: pgmap v1070: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 997 B/s rd, 0 op/s 2024-09-20T19:10:54.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:54 smithi083 ceph-mon[36260]: 1.1 scrub starts 2024-09-20T19:10:54.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:54 smithi083 ceph-mon[36260]: 1.1 scrub ok 2024-09-20T19:10:54.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:54 smithi083 ceph-mon[36260]: 4.1d scrub starts 2024-09-20T19:10:54.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:54 smithi083 ceph-mon[36260]: 4.1d scrub ok 2024-09-20T19:10:54.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:54 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1042584849' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]': finished 2024-09-20T19:10:54.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:54 smithi083 ceph-mon[36260]: osdmap e522: 8 total, 8 up, 8 in 2024-09-20T19:10:54.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:54 smithi083 ceph-mon[36260]: pgmap v1070: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 997 B/s rd, 0 op/s 2024-09-20T19:10:54.509 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:54 smithi142 ceph-mon[32508]: 1.1 scrub starts 2024-09-20T19:10:54.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:54 smithi142 ceph-mon[32508]: 1.1 scrub ok 2024-09-20T19:10:54.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:54 smithi142 ceph-mon[32508]: 4.1d scrub starts 2024-09-20T19:10:54.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:54 smithi142 ceph-mon[32508]: 4.1d scrub ok 2024-09-20T19:10:54.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:54 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1042584849' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]': finished 2024-09-20T19:10:54.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:54 smithi142 ceph-mon[32508]: osdmap e522: 8 total, 8 up, 8 in 2024-09-20T19:10:54.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:54 smithi142 ceph-mon[32508]: pgmap v1070: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 997 B/s rd, 0 op/s 2024-09-20T19:10:55.173 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.6 to 1 (10000) 2024-09-20T19:10:55.424 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:55 smithi142 ceph-mon[32508]: 4.3 scrub starts 2024-09-20T19:10:55.424 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:55 smithi142 ceph-mon[32508]: 4.3 scrub ok 2024-09-20T19:10:55.425 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:55 smithi142 ceph-mon[32508]: 1.a scrub starts 2024-09-20T19:10:55.425 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:55 smithi142 ceph-mon[32508]: 1.a scrub ok 2024-09-20T19:10:55.425 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:55 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/517262941' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-20T19:10:55.425 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:55 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-20T19:10:55.478 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:55 smithi083 ceph-mon[29421]: 4.3 scrub starts 2024-09-20T19:10:55.478 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:55 smithi083 ceph-mon[29421]: 4.3 scrub ok 2024-09-20T19:10:55.478 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:55 smithi083 ceph-mon[29421]: 1.a scrub starts 2024-09-20T19:10:55.478 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:55 smithi083 ceph-mon[29421]: 1.a scrub ok 2024-09-20T19:10:55.478 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:55 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/517262941' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-20T19:10:55.478 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:55 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-20T19:10:55.478 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:55 smithi083 ceph-mon[36260]: 4.3 scrub starts 2024-09-20T19:10:55.479 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:55 smithi083 ceph-mon[36260]: 4.3 scrub ok 2024-09-20T19:10:55.479 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:55 smithi083 ceph-mon[36260]: 1.a scrub starts 2024-09-20T19:10:55.479 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:55 smithi083 ceph-mon[36260]: 1.a scrub ok 2024-09-20T19:10:55.479 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:55 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/517262941' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-20T19:10:55.479 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:55 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-20T19:10:55.898 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd primary-affinity 6 1 2024-09-20T19:10:56.146 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:10:56.437 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:56 smithi083 ceph-mon[29421]: 4.19 scrub starts 2024-09-20T19:10:56.438 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:56 smithi083 ceph-mon[29421]: 4.19 scrub ok 2024-09-20T19:10:56.438 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:56 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-20T19:10:56.438 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:56 smithi083 ceph-mon[29421]: osdmap e523: 8 total, 8 up, 8 in 2024-09-20T19:10:56.438 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:56 smithi083 ceph-mon[29421]: pgmap v1072: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T19:10:56.438 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:56 smithi083 ceph-mon[36260]: 4.19 scrub starts 2024-09-20T19:10:56.438 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:56 smithi083 ceph-mon[36260]: 4.19 scrub ok 2024-09-20T19:10:56.438 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:56 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-20T19:10:56.439 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:56 smithi083 ceph-mon[36260]: osdmap e523: 8 total, 8 up, 8 in 2024-09-20T19:10:56.439 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:56 smithi083 ceph-mon[36260]: pgmap v1072: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T19:10:56.509 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:56 smithi142 ceph-mon[32508]: 4.19 scrub starts 2024-09-20T19:10:56.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:56 smithi142 ceph-mon[32508]: 4.19 scrub ok 2024-09-20T19:10:56.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:56 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-20T19:10:56.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:56 smithi142 ceph-mon[32508]: osdmap e523: 8 total, 8 up, 8 in 2024-09-20T19:10:56.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:56 smithi142 ceph-mon[32508]: pgmap v1072: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T19:10:57.188 INFO:teuthology.orchestra.run.smithi083.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-20T19:10:57.509 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:57 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2811536100' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 6, "weight": 1.0} : dispatch 2024-09-20T19:10:57.520 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:57 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2811536100' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 6, "weight": 1.0} : dispatch 2024-09-20T19:10:57.520 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:57 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2811536100' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 6, "weight": 1.0} : dispatch 2024-09-20T19:10:57.849 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd reweight 7 1 2024-09-20T19:10:58.009 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:10:57 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:10:58.096 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:10:58.474 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:58 smithi083 ceph-mon[29421]: 4.14 scrub starts 2024-09-20T19:10:58.474 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:58 smithi083 ceph-mon[29421]: 4.14 scrub ok 2024-09-20T19:10:58.474 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:58 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2811536100' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]': finished 2024-09-20T19:10:58.475 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:58 smithi083 ceph-mon[29421]: osdmap e524: 8 total, 8 up, 8 in 2024-09-20T19:10:58.475 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:58 smithi083 ceph-mon[29421]: pgmap v1074: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-20T19:10:58.475 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:58 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:10:58.475 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:58 smithi083 ceph-mon[36260]: 4.14 scrub starts 2024-09-20T19:10:58.475 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:58 smithi083 ceph-mon[36260]: 4.14 scrub ok 2024-09-20T19:10:58.475 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:58 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2811536100' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]': finished 2024-09-20T19:10:58.475 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:58 smithi083 ceph-mon[36260]: osdmap e524: 8 total, 8 up, 8 in 2024-09-20T19:10:58.475 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:58 smithi083 ceph-mon[36260]: pgmap v1074: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-20T19:10:58.475 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:58 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:10:58.509 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:58 smithi142 ceph-mon[32508]: 4.14 scrub starts 2024-09-20T19:10:58.509 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:58 smithi142 ceph-mon[32508]: 4.14 scrub ok 2024-09-20T19:10:58.509 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:58 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2811536100' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]': finished 2024-09-20T19:10:58.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:58 smithi142 ceph-mon[32508]: osdmap e524: 8 total, 8 up, 8 in 2024-09-20T19:10:58.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:58 smithi142 ceph-mon[32508]: pgmap v1074: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-20T19:10:58.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:58 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:10:59.210 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.7 to 1 (10000) 2024-09-20T19:10:59.509 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:10:59 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/948292947' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-20T19:10:59.536 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:10:59 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/948292947' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-20T19:10:59.537 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:10:59 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/948292947' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-20T19:10:59.968 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd primary-affinity 7 1 2024-09-20T19:11:00.218 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:11:00.469 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:00 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/948292947' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-20T19:11:00.469 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:00 smithi083 ceph-mon[29421]: osdmap e525: 8 total, 8 up, 8 in 2024-09-20T19:11:00.469 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:00 smithi083 ceph-mon[29421]: pgmap v1076: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T19:11:00.470 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:00 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/948292947' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-20T19:11:00.470 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:00 smithi083 ceph-mon[36260]: osdmap e525: 8 total, 8 up, 8 in 2024-09-20T19:11:00.470 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:00 smithi083 ceph-mon[36260]: pgmap v1076: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T19:11:00.509 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:00 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/948292947' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-20T19:11:00.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:00 smithi142 ceph-mon[32508]: osdmap e525: 8 total, 8 up, 8 in 2024-09-20T19:11:00.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:00 smithi142 ceph-mon[32508]: pgmap v1076: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T19:11:01.000 INFO:teuthology.orchestra.run.smithi083.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-20T19:11:01.000 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:11:00 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:11:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:11:01.289 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:01 smithi083 ceph-mon[29421]: 3.e scrub starts 2024-09-20T19:11:01.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:01 smithi083 ceph-mon[29421]: 3.e scrub ok 2024-09-20T19:11:01.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:01 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1827705186' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 7, "weight": 1.0} : dispatch 2024-09-20T19:11:01.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:01 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 7, "weight": 1.0} : dispatch 2024-09-20T19:11:01.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:01 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]': finished 2024-09-20T19:11:01.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:01 smithi083 ceph-mon[29421]: osdmap e526: 8 total, 8 up, 8 in 2024-09-20T19:11:01.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:01 smithi083 ceph-mon[36260]: 3.e scrub starts 2024-09-20T19:11:01.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:01 smithi083 ceph-mon[36260]: 3.e scrub ok 2024-09-20T19:11:01.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:01 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1827705186' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 7, "weight": 1.0} : dispatch 2024-09-20T19:11:01.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:01 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 7, "weight": 1.0} : dispatch 2024-09-20T19:11:01.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:01 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]': finished 2024-09-20T19:11:01.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:01 smithi083 ceph-mon[36260]: osdmap e526: 8 total, 8 up, 8 in 2024-09-20T19:11:01.509 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:01 smithi142 ceph-mon[32508]: 3.e scrub starts 2024-09-20T19:11:01.509 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:01 smithi142 ceph-mon[32508]: 3.e scrub ok 2024-09-20T19:11:01.509 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:01 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1827705186' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 7, "weight": 1.0} : dispatch 2024-09-20T19:11:01.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:01 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 7, "weight": 1.0} : dispatch 2024-09-20T19:11:01.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:01 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]': finished 2024-09-20T19:11:01.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:01 smithi142 ceph-mon[32508]: osdmap e526: 8 total, 8 up, 8 in 2024-09-20T19:11:01.825 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd reweight 4 1 2024-09-20T19:11:02.070 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:11:02.459 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:02 smithi083 ceph-mon[29421]: 5.6 scrub starts 2024-09-20T19:11:02.460 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:02 smithi083 ceph-mon[29421]: 5.6 scrub ok 2024-09-20T19:11:02.461 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:02 smithi083 ceph-mon[29421]: pgmap v1078: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T19:11:02.461 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:02 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:11:02.461 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:02 smithi083 ceph-mon[36260]: 5.6 scrub starts 2024-09-20T19:11:02.461 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:02 smithi083 ceph-mon[36260]: 5.6 scrub ok 2024-09-20T19:11:02.461 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:02 smithi083 ceph-mon[36260]: pgmap v1078: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T19:11:02.461 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:02 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:11:02.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:02 smithi142 ceph-mon[32508]: 5.6 scrub starts 2024-09-20T19:11:02.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:02 smithi142 ceph-mon[32508]: 5.6 scrub ok 2024-09-20T19:11:02.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:02 smithi142 ceph-mon[32508]: pgmap v1078: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T19:11:02.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:02 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:11:03.234 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.4 to 1 (10000) 2024-09-20T19:11:03.509 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:03 smithi142 ceph-mon[32508]: 4.1e scrub starts 2024-09-20T19:11:03.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:03 smithi142 ceph-mon[32508]: 4.1e scrub ok 2024-09-20T19:11:03.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:03 smithi142 ceph-mon[32508]: 5.d scrub starts 2024-09-20T19:11:03.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:03 smithi142 ceph-mon[32508]: 5.d scrub ok 2024-09-20T19:11:03.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:03 smithi142 ceph-mon[32508]: 6.1b scrub starts 2024-09-20T19:11:03.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:03 smithi142 ceph-mon[32508]: 6.1b scrub ok 2024-09-20T19:11:03.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:03 smithi142 ceph-mon[32508]: 5.19 scrub starts 2024-09-20T19:11:03.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:03 smithi142 ceph-mon[32508]: 5.19 scrub ok 2024-09-20T19:11:03.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:03 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/681177528' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-20T19:11:03.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:03 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-20T19:11:03.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:03 smithi083 ceph-mon[29421]: 4.1e scrub starts 2024-09-20T19:11:03.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:03 smithi083 ceph-mon[29421]: 4.1e scrub ok 2024-09-20T19:11:03.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:03 smithi083 ceph-mon[29421]: 5.d scrub starts 2024-09-20T19:11:03.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:03 smithi083 ceph-mon[29421]: 5.d scrub ok 2024-09-20T19:11:03.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:03 smithi083 ceph-mon[29421]: 6.1b scrub starts 2024-09-20T19:11:03.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:03 smithi083 ceph-mon[29421]: 6.1b scrub ok 2024-09-20T19:11:03.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:03 smithi083 ceph-mon[29421]: 5.19 scrub starts 2024-09-20T19:11:03.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:03 smithi083 ceph-mon[29421]: 5.19 scrub ok 2024-09-20T19:11:03.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:03 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/681177528' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-20T19:11:03.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:03 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-20T19:11:03.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:03 smithi083 ceph-mon[36260]: 4.1e scrub starts 2024-09-20T19:11:03.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:03 smithi083 ceph-mon[36260]: 4.1e scrub ok 2024-09-20T19:11:03.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:03 smithi083 ceph-mon[36260]: 5.d scrub starts 2024-09-20T19:11:03.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:03 smithi083 ceph-mon[36260]: 5.d scrub ok 2024-09-20T19:11:03.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:03 smithi083 ceph-mon[36260]: 6.1b scrub starts 2024-09-20T19:11:03.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:03 smithi083 ceph-mon[36260]: 6.1b scrub ok 2024-09-20T19:11:03.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:03 smithi083 ceph-mon[36260]: 5.19 scrub starts 2024-09-20T19:11:03.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:03 smithi083 ceph-mon[36260]: 5.19 scrub ok 2024-09-20T19:11:03.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:03 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/681177528' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-20T19:11:03.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:03 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-20T19:11:03.947 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd primary-affinity 4 1 2024-09-20T19:11:04.200 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:11:04.509 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:04 smithi142 ceph-mon[32508]: 6.18 scrub starts 2024-09-20T19:11:04.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:04 smithi142 ceph-mon[32508]: 6.18 scrub ok 2024-09-20T19:11:04.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:04 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-20T19:11:04.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:04 smithi142 ceph-mon[32508]: osdmap e527: 8 total, 8 up, 8 in 2024-09-20T19:11:04.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:04 smithi142 ceph-mon[32508]: pgmap v1080: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T19:11:04.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:04 smithi142 ceph-mon[32508]: 6.e scrub starts 2024-09-20T19:11:04.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:04 smithi142 ceph-mon[32508]: 6.e scrub ok 2024-09-20T19:11:04.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:04 smithi083 ceph-mon[29421]: 6.18 scrub starts 2024-09-20T19:11:04.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:04 smithi083 ceph-mon[29421]: 6.18 scrub ok 2024-09-20T19:11:04.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:04 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-20T19:11:04.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:04 smithi083 ceph-mon[29421]: osdmap e527: 8 total, 8 up, 8 in 2024-09-20T19:11:04.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:04 smithi083 ceph-mon[29421]: pgmap v1080: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T19:11:04.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:04 smithi083 ceph-mon[29421]: 6.e scrub starts 2024-09-20T19:11:04.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:04 smithi083 ceph-mon[29421]: 6.e scrub ok 2024-09-20T19:11:04.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:04 smithi083 ceph-mon[36260]: 6.18 scrub starts 2024-09-20T19:11:04.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:04 smithi083 ceph-mon[36260]: 6.18 scrub ok 2024-09-20T19:11:04.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:04 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-20T19:11:04.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:04 smithi083 ceph-mon[36260]: osdmap e527: 8 total, 8 up, 8 in 2024-09-20T19:11:04.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:04 smithi083 ceph-mon[36260]: pgmap v1080: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T19:11:04.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:04 smithi083 ceph-mon[36260]: 6.e scrub starts 2024-09-20T19:11:04.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:04 smithi083 ceph-mon[36260]: 6.e scrub ok 2024-09-20T19:11:05.254 INFO:teuthology.orchestra.run.smithi083.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-20T19:11:05.509 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:05 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/4171721851' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 4, "weight": 1.0} : dispatch 2024-09-20T19:11:05.509 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:05 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 4, "weight": 1.0} : dispatch 2024-09-20T19:11:05.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:05 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/4171721851' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 4, "weight": 1.0} : dispatch 2024-09-20T19:11:05.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:05 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 4, "weight": 1.0} : dispatch 2024-09-20T19:11:05.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:05 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/4171721851' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 4, "weight": 1.0} : dispatch 2024-09-20T19:11:05.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:05 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 4, "weight": 1.0} : dispatch 2024-09-20T19:11:06.015 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd reweight 0 1 2024-09-20T19:11:06.266 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:11:06.509 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:06 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]': finished 2024-09-20T19:11:06.509 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:06 smithi142 ceph-mon[32508]: osdmap e528: 8 total, 8 up, 8 in 2024-09-20T19:11:06.509 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:06 smithi142 ceph-mon[32508]: pgmap v1082: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T19:11:06.510 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:06 smithi142 ceph-mon[32508]: osdmap e529: 8 total, 8 up, 8 in 2024-09-20T19:11:06.526 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:06 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]': finished 2024-09-20T19:11:06.526 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:06 smithi083 ceph-mon[29421]: osdmap e528: 8 total, 8 up, 8 in 2024-09-20T19:11:06.526 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:06 smithi083 ceph-mon[29421]: pgmap v1082: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T19:11:06.526 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:06 smithi083 ceph-mon[29421]: osdmap e529: 8 total, 8 up, 8 in 2024-09-20T19:11:06.526 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:06 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]': finished 2024-09-20T19:11:06.526 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:06 smithi083 ceph-mon[36260]: osdmap e528: 8 total, 8 up, 8 in 2024-09-20T19:11:06.527 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:06 smithi083 ceph-mon[36260]: pgmap v1082: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T19:11:06.527 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:06 smithi083 ceph-mon[36260]: osdmap e529: 8 total, 8 up, 8 in 2024-09-20T19:11:07.255 INFO:teuthology.orchestra.run.smithi083.stderr:reweighted osd.0 to 1 (10000) 2024-09-20T19:11:07.539 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:07 smithi083 ceph-mon[36260]: 3.17 scrub starts 2024-09-20T19:11:07.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:07 smithi083 ceph-mon[36260]: 3.17 scrub ok 2024-09-20T19:11:07.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:07 smithi083 ceph-mon[36260]: 6.19 scrub starts 2024-09-20T19:11:07.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:07 smithi083 ceph-mon[36260]: 6.19 scrub ok 2024-09-20T19:11:07.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:07 smithi083 ceph-mon[36260]: 6.c scrub starts 2024-09-20T19:11:07.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:07 smithi083 ceph-mon[36260]: 6.c scrub ok 2024-09-20T19:11:07.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:07 smithi083 ceph-mon[36260]: 1.3 scrub starts 2024-09-20T19:11:07.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:07 smithi083 ceph-mon[36260]: 1.3 scrub ok 2024-09-20T19:11:07.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:07 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/218265002' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-20T19:11:07.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:07 smithi083 ceph-mon[29421]: 3.17 scrub starts 2024-09-20T19:11:07.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:07 smithi083 ceph-mon[29421]: 3.17 scrub ok 2024-09-20T19:11:07.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:07 smithi083 ceph-mon[29421]: 6.19 scrub starts 2024-09-20T19:11:07.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:07 smithi083 ceph-mon[29421]: 6.19 scrub ok 2024-09-20T19:11:07.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:07 smithi083 ceph-mon[29421]: 6.c scrub starts 2024-09-20T19:11:07.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:07 smithi083 ceph-mon[29421]: 6.c scrub ok 2024-09-20T19:11:07.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:07 smithi083 ceph-mon[29421]: 1.3 scrub starts 2024-09-20T19:11:07.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:07 smithi083 ceph-mon[29421]: 1.3 scrub ok 2024-09-20T19:11:07.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:07 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/218265002' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-20T19:11:07.723 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:07 smithi142 ceph-mon[32508]: 3.17 scrub starts 2024-09-20T19:11:07.724 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:07 smithi142 ceph-mon[32508]: 3.17 scrub ok 2024-09-20T19:11:07.724 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:07 smithi142 ceph-mon[32508]: 6.19 scrub starts 2024-09-20T19:11:07.724 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:07 smithi142 ceph-mon[32508]: 6.19 scrub ok 2024-09-20T19:11:07.724 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:07 smithi142 ceph-mon[32508]: 6.c scrub starts 2024-09-20T19:11:07.724 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:07 smithi142 ceph-mon[32508]: 6.c scrub ok 2024-09-20T19:11:07.724 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:07 smithi142 ceph-mon[32508]: 1.3 scrub starts 2024-09-20T19:11:07.724 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:07 smithi142 ceph-mon[32508]: 1.3 scrub ok 2024-09-20T19:11:07.724 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:07 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/218265002' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-20T19:11:07.991 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd primary-affinity 0 1 2024-09-20T19:11:08.010 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:11:07 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:11:08.243 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:11:08.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:08 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/218265002' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-20T19:11:08.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:08 smithi083 ceph-mon[29421]: osdmap e530: 8 total, 8 up, 8 in 2024-09-20T19:11:08.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:08 smithi083 ceph-mon[29421]: pgmap v1085: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T19:11:08.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:08 smithi083 ceph-mon[29421]: 1.7 scrub starts 2024-09-20T19:11:08.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:08 smithi083 ceph-mon[29421]: 1.7 scrub ok 2024-09-20T19:11:08.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:08 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:11:08.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:08 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/218265002' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-20T19:11:08.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:08 smithi083 ceph-mon[36260]: osdmap e530: 8 total, 8 up, 8 in 2024-09-20T19:11:08.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:08 smithi083 ceph-mon[36260]: pgmap v1085: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T19:11:08.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:08 smithi083 ceph-mon[36260]: 1.7 scrub starts 2024-09-20T19:11:08.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:08 smithi083 ceph-mon[36260]: 1.7 scrub ok 2024-09-20T19:11:08.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:08 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:11:08.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:08 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/218265002' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-20T19:11:08.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:08 smithi142 ceph-mon[32508]: osdmap e530: 8 total, 8 up, 8 in 2024-09-20T19:11:08.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:08 smithi142 ceph-mon[32508]: pgmap v1085: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T19:11:08.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:08 smithi142 ceph-mon[32508]: 1.7 scrub starts 2024-09-20T19:11:08.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:08 smithi142 ceph-mon[32508]: 1.7 scrub ok 2024-09-20T19:11:08.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:08 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:11:09.268 INFO:teuthology.orchestra.run.smithi083.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-20T19:11:09.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:09 smithi083 ceph-mon[29421]: 1.8 scrub starts 2024-09-20T19:11:09.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:09 smithi083 ceph-mon[29421]: 1.8 scrub ok 2024-09-20T19:11:09.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:09 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2405390785' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 0, "weight": 1.0} : dispatch 2024-09-20T19:11:09.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:09 smithi083 ceph-mon[36260]: 1.8 scrub starts 2024-09-20T19:11:09.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:09 smithi083 ceph-mon[36260]: 1.8 scrub ok 2024-09-20T19:11:09.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:09 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2405390785' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 0, "weight": 1.0} : dispatch 2024-09-20T19:11:09.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:09 smithi142 ceph-mon[32508]: 1.8 scrub starts 2024-09-20T19:11:09.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:09 smithi142 ceph-mon[32508]: 1.8 scrub ok 2024-09-20T19:11:09.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:09 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2405390785' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 0, "weight": 1.0} : dispatch 2024-09-20T19:11:10.004 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-20T19:11:10.004 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-20T19:11:10.004 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-20T19:11:10.004 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-20T19:11:10.004 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-20T19:11:10.005 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-20T19:11:10.251 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:11:10.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:10 smithi083 ceph-mon[29421]: 5.5 scrub starts 2024-09-20T19:11:10.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:10 smithi083 ceph-mon[29421]: 5.5 scrub ok 2024-09-20T19:11:10.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:10 smithi083 ceph-mon[29421]: pgmap v1086: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T19:11:10.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:10 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2405390785' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]': finished 2024-09-20T19:11:10.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:10 smithi083 ceph-mon[29421]: osdmap e531: 8 total, 8 up, 8 in 2024-09-20T19:11:10.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:10 smithi083 ceph-mon[29421]: 6.1e scrub starts 2024-09-20T19:11:10.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:10 smithi083 ceph-mon[29421]: 6.1e scrub ok 2024-09-20T19:11:10.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:10 smithi083 ceph-mon[36260]: 5.5 scrub starts 2024-09-20T19:11:10.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:10 smithi083 ceph-mon[36260]: 5.5 scrub ok 2024-09-20T19:11:10.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:10 smithi083 ceph-mon[36260]: pgmap v1086: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T19:11:10.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:10 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2405390785' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]': finished 2024-09-20T19:11:10.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:10 smithi083 ceph-mon[36260]: osdmap e531: 8 total, 8 up, 8 in 2024-09-20T19:11:10.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:10 smithi083 ceph-mon[36260]: 6.1e scrub starts 2024-09-20T19:11:10.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:10 smithi083 ceph-mon[36260]: 6.1e scrub ok 2024-09-20T19:11:10.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:10 smithi142 ceph-mon[32508]: 5.5 scrub starts 2024-09-20T19:11:10.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:10 smithi142 ceph-mon[32508]: 5.5 scrub ok 2024-09-20T19:11:10.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:10 smithi142 ceph-mon[32508]: pgmap v1086: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T19:11:10.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:10 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2405390785' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]': finished 2024-09-20T19:11:10.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:10 smithi142 ceph-mon[32508]: osdmap e531: 8 total, 8 up, 8 in 2024-09-20T19:11:10.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:10 smithi142 ceph-mon[32508]: 6.1e scrub starts 2024-09-20T19:11:10.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:10 smithi142 ceph-mon[32508]: 6.1e scrub ok 2024-09-20T19:11:10.904 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T19:11:10.904 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":531,"fsid":"1ed5a50a-7780-11ef-baf5-efdc52797490","created":"2024-09-20T18:44:10.399800+0000","modified":"2024-09-20T19:11:09.264521+0000","last_up_change":"2024-09-20T19:10:32.038029+0000","last_in_change":"2024-09-20T19:04:20.299488+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-20T18:47:15.285795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":11,"pg_placement_num":11,"pg_placement_num_target":11,"pg_num_target":11,"pg_num_pending":11,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"512","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"455","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.1800000667572021,"score_stable":2.1800000667572021,"optimal_score":1,"raw_score_acting":2.1800000667572021,"raw_score_stable":2.1800000667572021,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"datapool","create_time":"2024-09-20T18:49:43.452984+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":61,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-09-20T18:49:44.217011+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-20T18:49:46.007275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-20T18:49:47.961258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-20T18:49:50.011276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c714e1ec-f0c3-4054-9792-61b84e53eaa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":494,"up_from":510,"up_thru":510,"down_at":495,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6801","nonce":444027075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":444027075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":444027075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6803","nonce":444027075}]},"public_addr":"172.21.15.83:6801/444027075","cluster_addr":"172.21.15.83:6802/444027075","heartbeat_back_addr":"172.21.15.83:6804/444027075","heartbeat_front_addr":"172.21.15.83:6803/444027075","state":["exists","up"]},{"osd":1,"uuid":"9dc0f2a8-bafa-4a42-88f5-0301dd58f78d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":514,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6805","nonce":4213285798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":4213285798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":4213285798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6807","nonce":4213285798}]},"public_addr":"172.21.15.83:6805/4213285798","cluster_addr":"172.21.15.83:6806/4213285798","heartbeat_back_addr":"172.21.15.83:6808/4213285798","heartbeat_front_addr":"172.21.15.83:6807/4213285798","state":["exists","up"]},{"osd":2,"uuid":"448ab692-ec41-44dc-b2d5-dbefd255b3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":510,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":805947571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":805947571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":805947571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6811","nonce":805947571}]},"public_addr":"172.21.15.83:6809/805947571","cluster_addr":"172.21.15.83:6810/805947571","heartbeat_back_addr":"172.21.15.83:6812/805947571","heartbeat_front_addr":"172.21.15.83:6811/805947571","state":["exists","up"]},{"osd":3,"uuid":"7cda5e50-e22e-4d4a-823b-53c8f5061193","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":510,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":987237318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":987237318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":987237318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":987237318}]},"public_addr":"172.21.15.83:6813/987237318","cluster_addr":"172.21.15.83:6814/987237318","heartbeat_back_addr":"172.21.15.83:6816/987237318","heartbeat_front_addr":"172.21.15.83:6815/987237318","state":["exists","up"]},{"osd":4,"uuid":"3d62654d-d31d-4310-9934-49e694dd17d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":31,"last_clean_end":291,"up_from":304,"up_thru":510,"down_at":292,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":3322919402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":3322919402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6803","nonce":3322919402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":3322919402}]},"public_addr":"172.21.15.142:6800/3322919402","cluster_addr":"172.21.15.142:6801/3322919402","heartbeat_back_addr":"172.21.15.142:6803/3322919402","heartbeat_front_addr":"172.21.15.142:6802/3322919402","state":["exists","up"]},{"osd":5,"uuid":"b3e4e21b-639f-4d25-8bbf-ea8922a9222a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":510,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":2500466067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6805","nonce":2500466067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":2500466067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":2500466067}]},"public_addr":"172.21.15.142:6804/2500466067","cluster_addr":"172.21.15.142:6805/2500466067","heartbeat_back_addr":"172.21.15.142:6807/2500466067","heartbeat_front_addr":"172.21.15.142:6806/2500466067","state":["exists","up"]},{"osd":6,"uuid":"c5db7107-ccb4-4ae2-9e63-f444897942e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":510,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1688186226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6809","nonce":1688186226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6811","nonce":1688186226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1688186226}]},"public_addr":"172.21.15.142:6808/1688186226","cluster_addr":"172.21.15.142:6809/1688186226","heartbeat_back_addr":"172.21.15.142:6811/1688186226","heartbeat_front_addr":"172.21.15.142:6810/1688186226","state":["exists","up"]},{"osd":7,"uuid":"07d16806-e2c8-4acb-b284-d75701e61622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":510,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3679770984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6813","nonce":3679770984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6815","nonce":3679770984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3679770984}]},"public_addr":"172.21.15.142:6812/3679770984","cluster_addr":"172.21.15.142:6813/3679770984","heartbeat_back_addr":"172.21.15.142:6815/3679770984","heartbeat_front_addr":"172.21.15.142:6814/3679770984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T19:10:31.364895+0000","dead_epoch":494},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T19:10:09.025543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T19:10:24.748740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T19:10:43.931526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T19:10:47.821744+0000","dead_epoch":291},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T19:10:24.450009+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T19:10:11.847732+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T19:10:18.812475+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"1.5","osds":[1,7,0]},{"pgid":"3.c","osds":[2,7,6]},{"pgid":"4.2","osds":[5,7,4]}],"pg_upmap_items":[{"pgid":"4.4","mappings":[{"from":3,"to":5}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]},{"pgid":"4.1e","mappings":[{"from":3,"to":2}]},{"pgid":"5.1c","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/3474253603":"2024-09-21T18:44:32.434180+0000","172.21.15.83:6800/3525856829":"2024-09-21T18:44:32.434180+0000","172.21.15.83:0/220746284":"2024-09-21T18:44:32.434180+0000","172.21.15.83:0/3875445393":"2024-09-21T18:44:56.985620+0000","172.21.15.83:0/3009092807":"2024-09-21T18:44:56.985620+0000","172.21.15.83:0/4267132131":"2024-09-21T18:44:56.985620+0000","172.21.15.83:0/3684382972":"2024-09-21T18:50:14.729526+0000","172.21.15.83:0/3132933178":"2024-09-21T18:50:14.729526+0000","172.21.15.83:6800/1007846146":"2024-09-21T18:50:14.729526+0000","172.21.15.83:6800/2985596191":"2024-09-21T18:44:56.985620+0000","172.21.15.83:0/2410768464":"2024-09-21T18:50:14.729526+0000","172.21.15.83:0/1868694097":"2024-09-21T18:50:14.729526+0000","172.21.15.83:0/3777331122":"2024-09-21T18:44:32.434180+0000","172.21.15.83:0/2161728215":"2024-09-21T18:50:14.729526+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-20T19:11:10.905 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:11:10 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:11:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:11:11.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:11 smithi083 ceph-mon[29421]: 4.a scrub starts 2024-09-20T19:11:11.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:11 smithi083 ceph-mon[29421]: 4.a scrub ok 2024-09-20T19:11:11.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:11 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2756945147' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-20T19:11:11.539 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:11 smithi083 ceph-mon[36260]: 4.a scrub starts 2024-09-20T19:11:11.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:11 smithi083 ceph-mon[36260]: 4.a scrub ok 2024-09-20T19:11:11.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:11 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2756945147' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-20T19:11:11.713 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-20T19:11:11.713 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-20T19:11:11.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:11 smithi142 ceph-mon[32508]: 4.a scrub starts 2024-09-20T19:11:11.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:11 smithi142 ceph-mon[32508]: 4.a scrub ok 2024-09-20T19:11:11.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:11 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2756945147' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-20T19:11:11.961 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:11:12.386 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:12 smithi083 ceph-mon[29421]: 4.f scrub starts 2024-09-20T19:11:12.387 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:12 smithi083 ceph-mon[29421]: 4.f scrub ok 2024-09-20T19:11:12.387 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:12 smithi083 ceph-mon[29421]: 3.1d scrub starts 2024-09-20T19:11:12.387 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:12 smithi083 ceph-mon[29421]: 3.1d scrub ok 2024-09-20T19:11:12.387 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:12 smithi083 ceph-mon[29421]: pgmap v1088: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T19:11:12.387 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:12 smithi083 ceph-mon[29421]: 5.12 scrub starts 2024-09-20T19:11:12.387 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:12 smithi083 ceph-mon[29421]: 5.12 scrub ok 2024-09-20T19:11:12.387 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:12 smithi083 ceph-mon[36260]: 4.f scrub starts 2024-09-20T19:11:12.388 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:12 smithi083 ceph-mon[36260]: 4.f scrub ok 2024-09-20T19:11:12.388 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:12 smithi083 ceph-mon[36260]: 3.1d scrub starts 2024-09-20T19:11:12.388 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:12 smithi083 ceph-mon[36260]: 3.1d scrub ok 2024-09-20T19:11:12.388 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:12 smithi083 ceph-mon[36260]: pgmap v1088: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T19:11:12.388 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:12 smithi083 ceph-mon[36260]: 5.12 scrub starts 2024-09-20T19:11:12.388 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:12 smithi083 ceph-mon[36260]: 5.12 scrub ok 2024-09-20T19:11:12.657 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T19:11:12.657 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":531,"fsid":"1ed5a50a-7780-11ef-baf5-efdc52797490","created":"2024-09-20T18:44:10.399800+0000","modified":"2024-09-20T19:11:09.264521+0000","last_up_change":"2024-09-20T19:10:32.038029+0000","last_in_change":"2024-09-20T19:04:20.299488+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-20T18:47:15.285795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":11,"pg_placement_num":11,"pg_placement_num_target":11,"pg_num_target":11,"pg_num_pending":11,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"512","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"455","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.1800000667572021,"score_stable":2.1800000667572021,"optimal_score":1,"raw_score_acting":2.1800000667572021,"raw_score_stable":2.1800000667572021,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"datapool","create_time":"2024-09-20T18:49:43.452984+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":61,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-09-20T18:49:44.217011+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-20T18:49:46.007275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-20T18:49:47.961258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-20T18:49:50.011276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c714e1ec-f0c3-4054-9792-61b84e53eaa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":494,"up_from":510,"up_thru":510,"down_at":495,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6801","nonce":444027075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":444027075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":444027075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6803","nonce":444027075}]},"public_addr":"172.21.15.83:6801/444027075","cluster_addr":"172.21.15.83:6802/444027075","heartbeat_back_addr":"172.21.15.83:6804/444027075","heartbeat_front_addr":"172.21.15.83:6803/444027075","state":["exists","up"]},{"osd":1,"uuid":"9dc0f2a8-bafa-4a42-88f5-0301dd58f78d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":514,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6805","nonce":4213285798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":4213285798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":4213285798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6807","nonce":4213285798}]},"public_addr":"172.21.15.83:6805/4213285798","cluster_addr":"172.21.15.83:6806/4213285798","heartbeat_back_addr":"172.21.15.83:6808/4213285798","heartbeat_front_addr":"172.21.15.83:6807/4213285798","state":["exists","up"]},{"osd":2,"uuid":"448ab692-ec41-44dc-b2d5-dbefd255b3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":510,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":805947571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":805947571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":805947571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6811","nonce":805947571}]},"public_addr":"172.21.15.83:6809/805947571","cluster_addr":"172.21.15.83:6810/805947571","heartbeat_back_addr":"172.21.15.83:6812/805947571","heartbeat_front_addr":"172.21.15.83:6811/805947571","state":["exists","up"]},{"osd":3,"uuid":"7cda5e50-e22e-4d4a-823b-53c8f5061193","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":510,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6813","nonce":987237318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":987237318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":987237318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6815","nonce":987237318}]},"public_addr":"172.21.15.83:6813/987237318","cluster_addr":"172.21.15.83:6814/987237318","heartbeat_back_addr":"172.21.15.83:6816/987237318","heartbeat_front_addr":"172.21.15.83:6815/987237318","state":["exists","up"]},{"osd":4,"uuid":"3d62654d-d31d-4310-9934-49e694dd17d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":31,"last_clean_end":291,"up_from":304,"up_thru":510,"down_at":292,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":3322919402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6801","nonce":3322919402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6803","nonce":3322919402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":3322919402}]},"public_addr":"172.21.15.142:6800/3322919402","cluster_addr":"172.21.15.142:6801/3322919402","heartbeat_back_addr":"172.21.15.142:6803/3322919402","heartbeat_front_addr":"172.21.15.142:6802/3322919402","state":["exists","up"]},{"osd":5,"uuid":"b3e4e21b-639f-4d25-8bbf-ea8922a9222a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":510,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":2500466067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6805","nonce":2500466067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6807","nonce":2500466067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":2500466067}]},"public_addr":"172.21.15.142:6804/2500466067","cluster_addr":"172.21.15.142:6805/2500466067","heartbeat_back_addr":"172.21.15.142:6807/2500466067","heartbeat_front_addr":"172.21.15.142:6806/2500466067","state":["exists","up"]},{"osd":6,"uuid":"c5db7107-ccb4-4ae2-9e63-f444897942e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":510,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1688186226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6809","nonce":1688186226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6811","nonce":1688186226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1688186226}]},"public_addr":"172.21.15.142:6808/1688186226","cluster_addr":"172.21.15.142:6809/1688186226","heartbeat_back_addr":"172.21.15.142:6811/1688186226","heartbeat_front_addr":"172.21.15.142:6810/1688186226","state":["exists","up"]},{"osd":7,"uuid":"07d16806-e2c8-4acb-b284-d75701e61622","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":510,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3679770984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6813","nonce":3679770984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6815","nonce":3679770984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3679770984}]},"public_addr":"172.21.15.142:6812/3679770984","cluster_addr":"172.21.15.142:6813/3679770984","heartbeat_back_addr":"172.21.15.142:6815/3679770984","heartbeat_front_addr":"172.21.15.142:6814/3679770984","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T19:10:31.364895+0000","dead_epoch":494},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T19:10:09.025543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T19:10:24.748740+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T19:10:43.931526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T19:10:47.821744+0000","dead_epoch":291},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T19:10:24.450009+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T19:10:11.847732+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T19:10:18.812475+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"1.5","osds":[1,7,0]},{"pgid":"3.c","osds":[2,7,6]},{"pgid":"4.2","osds":[5,7,4]}],"pg_upmap_items":[{"pgid":"4.4","mappings":[{"from":3,"to":5}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]},{"pgid":"4.1e","mappings":[{"from":3,"to":2}]},{"pgid":"5.1c","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/3474253603":"2024-09-21T18:44:32.434180+0000","172.21.15.83:6800/3525856829":"2024-09-21T18:44:32.434180+0000","172.21.15.83:0/220746284":"2024-09-21T18:44:32.434180+0000","172.21.15.83:0/3875445393":"2024-09-21T18:44:56.985620+0000","172.21.15.83:0/3009092807":"2024-09-21T18:44:56.985620+0000","172.21.15.83:0/4267132131":"2024-09-21T18:44:56.985620+0000","172.21.15.83:0/3684382972":"2024-09-21T18:50:14.729526+0000","172.21.15.83:0/3132933178":"2024-09-21T18:50:14.729526+0000","172.21.15.83:6800/1007846146":"2024-09-21T18:50:14.729526+0000","172.21.15.83:6800/2985596191":"2024-09-21T18:44:56.985620+0000","172.21.15.83:0/2410768464":"2024-09-21T18:50:14.729526+0000","172.21.15.83:0/1868694097":"2024-09-21T18:50:14.729526+0000","172.21.15.83:0/3777331122":"2024-09-21T18:44:32.434180+0000","172.21.15.83:0/2161728215":"2024-09-21T18:50:14.729526+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-20T19:11:12.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:12 smithi142 ceph-mon[32508]: 4.f scrub starts 2024-09-20T19:11:12.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:12 smithi142 ceph-mon[32508]: 4.f scrub ok 2024-09-20T19:11:12.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:12 smithi142 ceph-mon[32508]: 3.1d scrub starts 2024-09-20T19:11:12.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:12 smithi142 ceph-mon[32508]: 3.1d scrub ok 2024-09-20T19:11:12.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:12 smithi142 ceph-mon[32508]: pgmap v1088: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T19:11:12.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:12 smithi142 ceph-mon[32508]: 5.12 scrub starts 2024-09-20T19:11:12.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:12 smithi142 ceph-mon[32508]: 5.12 scrub ok 2024-09-20T19:11:13.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:13 smithi083 ceph-mon[29421]: 3.1c scrub starts 2024-09-20T19:11:13.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:13 smithi083 ceph-mon[29421]: 3.1c scrub ok 2024-09-20T19:11:13.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:13 smithi083 ceph-mon[29421]: 4.4 scrub starts 2024-09-20T19:11:13.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:13 smithi083 ceph-mon[29421]: 4.4 scrub ok 2024-09-20T19:11:13.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:13 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2429672767' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-20T19:11:13.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:13 smithi083 ceph-mon[29421]: 5.1 scrub starts 2024-09-20T19:11:13.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:13 smithi083 ceph-mon[29421]: 5.1 scrub ok 2024-09-20T19:11:13.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:13 smithi083 ceph-mon[36260]: 3.1c scrub starts 2024-09-20T19:11:13.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:13 smithi083 ceph-mon[36260]: 3.1c scrub ok 2024-09-20T19:11:13.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:13 smithi083 ceph-mon[36260]: 4.4 scrub starts 2024-09-20T19:11:13.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:13 smithi083 ceph-mon[36260]: 4.4 scrub ok 2024-09-20T19:11:13.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:13 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2429672767' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-20T19:11:13.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:13 smithi083 ceph-mon[36260]: 5.1 scrub starts 2024-09-20T19:11:13.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:13 smithi083 ceph-mon[36260]: 5.1 scrub ok 2024-09-20T19:11:13.675 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph tell osd.0 flush_pg_stats 2024-09-20T19:11:13.676 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph tell osd.1 flush_pg_stats 2024-09-20T19:11:13.676 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph tell osd.2 flush_pg_stats 2024-09-20T19:11:13.676 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph tell osd.3 flush_pg_stats 2024-09-20T19:11:13.677 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph tell osd.4 flush_pg_stats 2024-09-20T19:11:13.677 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph tell osd.5 flush_pg_stats 2024-09-20T19:11:13.677 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph tell osd.6 flush_pg_stats 2024-09-20T19:11:13.678 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph tell osd.7 flush_pg_stats 2024-09-20T19:11:13.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:13 smithi142 ceph-mon[32508]: 3.1c scrub starts 2024-09-20T19:11:13.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:13 smithi142 ceph-mon[32508]: 3.1c scrub ok 2024-09-20T19:11:13.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:13 smithi142 ceph-mon[32508]: 4.4 scrub starts 2024-09-20T19:11:13.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:13 smithi142 ceph-mon[32508]: 4.4 scrub ok 2024-09-20T19:11:13.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:13 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2429672767' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-20T19:11:13.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:13 smithi142 ceph-mon[32508]: 5.1 scrub starts 2024-09-20T19:11:13.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:13 smithi142 ceph-mon[32508]: 5.1 scrub ok 2024-09-20T19:11:14.205 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:11:14.205 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:11:14.206 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:11:14.206 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:11:14.210 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:11:14.211 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:11:14.212 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:11:14.212 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:11:14.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:14 smithi083 ceph-mon[29421]: pgmap v1089: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-20T19:11:14.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:14 smithi083 ceph-mon[29421]: 5.1f scrub starts 2024-09-20T19:11:14.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:14 smithi083 ceph-mon[29421]: 5.1f scrub ok 2024-09-20T19:11:14.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:14 smithi083 ceph-mon[36260]: pgmap v1089: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-20T19:11:14.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:14 smithi083 ceph-mon[36260]: 5.1f scrub starts 2024-09-20T19:11:14.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:14 smithi083 ceph-mon[36260]: 5.1f scrub ok 2024-09-20T19:11:14.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:14 smithi142 ceph-mon[32508]: pgmap v1089: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-20T19:11:14.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:14 smithi142 ceph-mon[32508]: 5.1f scrub starts 2024-09-20T19:11:14.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:14 smithi142 ceph-mon[32508]: 5.1f scrub ok 2024-09-20T19:11:15.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:15 smithi083 ceph-mon[29421]: 4.2 scrub starts 2024-09-20T19:11:15.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:15 smithi083 ceph-mon[29421]: 4.2 scrub ok 2024-09-20T19:11:15.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:15 smithi083 ceph-mon[29421]: 6.6 scrub starts 2024-09-20T19:11:15.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:15 smithi083 ceph-mon[29421]: 6.6 scrub ok 2024-09-20T19:11:15.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:15 smithi083 ceph-mon[29421]: 4.5 scrub starts 2024-09-20T19:11:15.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:15 smithi083 ceph-mon[29421]: 4.5 scrub ok 2024-09-20T19:11:15.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:15 smithi083 ceph-mon[29421]: 6.17 scrub starts 2024-09-20T19:11:15.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:15 smithi083 ceph-mon[29421]: 6.17 scrub ok 2024-09-20T19:11:15.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:15 smithi083 ceph-mon[36260]: 4.2 scrub starts 2024-09-20T19:11:15.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:15 smithi083 ceph-mon[36260]: 4.2 scrub ok 2024-09-20T19:11:15.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:15 smithi083 ceph-mon[36260]: 6.6 scrub starts 2024-09-20T19:11:15.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:15 smithi083 ceph-mon[36260]: 6.6 scrub ok 2024-09-20T19:11:15.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:15 smithi083 ceph-mon[36260]: 4.5 scrub starts 2024-09-20T19:11:15.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:15 smithi083 ceph-mon[36260]: 4.5 scrub ok 2024-09-20T19:11:15.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:15 smithi083 ceph-mon[36260]: 6.17 scrub starts 2024-09-20T19:11:15.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:15 smithi083 ceph-mon[36260]: 6.17 scrub ok 2024-09-20T19:11:15.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:15 smithi142 ceph-mon[32508]: 4.2 scrub starts 2024-09-20T19:11:15.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:15 smithi142 ceph-mon[32508]: 4.2 scrub ok 2024-09-20T19:11:15.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:15 smithi142 ceph-mon[32508]: 6.6 scrub starts 2024-09-20T19:11:15.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:15 smithi142 ceph-mon[32508]: 6.6 scrub ok 2024-09-20T19:11:15.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:15 smithi142 ceph-mon[32508]: 4.5 scrub starts 2024-09-20T19:11:15.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:15 smithi142 ceph-mon[32508]: 4.5 scrub ok 2024-09-20T19:11:15.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:15 smithi142 ceph-mon[32508]: 6.17 scrub starts 2024-09-20T19:11:15.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:15 smithi142 ceph-mon[32508]: 6.17 scrub ok 2024-09-20T19:11:16.690 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:16 smithi083 ceph-mon[29421]: 5.f scrub starts 2024-09-20T19:11:16.690 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:16 smithi083 ceph-mon[29421]: 5.f scrub ok 2024-09-20T19:11:16.690 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:16 smithi083 ceph-mon[29421]: 3.1e scrub starts 2024-09-20T19:11:16.690 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:16 smithi083 ceph-mon[29421]: 3.1e scrub ok 2024-09-20T19:11:16.690 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:16 smithi083 ceph-mon[29421]: pgmap v1090: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-20T19:11:16.690 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:16 smithi083 ceph-mon[29421]: 3.a scrub starts 2024-09-20T19:11:16.690 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:16 smithi083 ceph-mon[29421]: 3.a scrub ok 2024-09-20T19:11:16.691 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:16 smithi083 ceph-mon[36260]: 5.f scrub starts 2024-09-20T19:11:16.691 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:16 smithi083 ceph-mon[36260]: 5.f scrub ok 2024-09-20T19:11:16.691 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:16 smithi083 ceph-mon[36260]: 3.1e scrub starts 2024-09-20T19:11:16.691 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:16 smithi083 ceph-mon[36260]: 3.1e scrub ok 2024-09-20T19:11:16.691 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:16 smithi083 ceph-mon[36260]: pgmap v1090: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-20T19:11:16.691 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:16 smithi083 ceph-mon[36260]: 3.a scrub starts 2024-09-20T19:11:16.691 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:16 smithi083 ceph-mon[36260]: 3.a scrub ok 2024-09-20T19:11:16.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:16 smithi142 ceph-mon[32508]: 5.f scrub starts 2024-09-20T19:11:16.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:16 smithi142 ceph-mon[32508]: 5.f scrub ok 2024-09-20T19:11:16.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:16 smithi142 ceph-mon[32508]: 3.1e scrub starts 2024-09-20T19:11:16.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:16 smithi142 ceph-mon[32508]: 3.1e scrub ok 2024-09-20T19:11:16.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:16 smithi142 ceph-mon[32508]: pgmap v1090: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-20T19:11:16.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:16 smithi142 ceph-mon[32508]: 3.a scrub starts 2024-09-20T19:11:16.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:16 smithi142 ceph-mon[32508]: 3.a scrub ok 2024-09-20T19:11:17.734 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:17 smithi142 ceph-mon[32508]: 6.1d scrub starts 2024-09-20T19:11:17.734 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:17 smithi142 ceph-mon[32508]: 6.1d scrub ok 2024-09-20T19:11:17.734 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:17 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:11:17.789 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:17 smithi083 ceph-mon[36260]: 6.1d scrub starts 2024-09-20T19:11:17.789 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:17 smithi083 ceph-mon[36260]: 6.1d scrub ok 2024-09-20T19:11:17.789 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:17 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:11:17.790 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:17 smithi083 ceph-mon[29421]: 6.1d scrub starts 2024-09-20T19:11:17.790 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:17 smithi083 ceph-mon[29421]: 6.1d scrub ok 2024-09-20T19:11:17.790 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:17 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:11:18.009 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:11:17 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:11:18.230 INFO:teuthology.orchestra.run.smithi083.stdout:227633266951 2024-09-20T19:11:18.231 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.7 2024-09-20T19:11:18.506 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:18 smithi083 ceph-mon[36260]: pgmap v1091: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-20T19:11:18.506 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:18 smithi083 ceph-mon[36260]: 5.0 scrub starts 2024-09-20T19:11:18.506 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:18 smithi083 ceph-mon[36260]: 5.0 scrub ok 2024-09-20T19:11:18.506 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:18 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:11:18.506 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:18 smithi083 ceph-mon[36260]: 6.1a scrub starts 2024-09-20T19:11:18.506 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:18 smithi083 ceph-mon[36260]: 6.1a scrub ok 2024-09-20T19:11:18.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:18 smithi083 ceph-mon[29421]: pgmap v1091: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-20T19:11:18.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:18 smithi083 ceph-mon[29421]: 5.0 scrub starts 2024-09-20T19:11:18.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:18 smithi083 ceph-mon[29421]: 5.0 scrub ok 2024-09-20T19:11:18.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:18 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:11:18.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:18 smithi083 ceph-mon[29421]: 6.1a scrub starts 2024-09-20T19:11:18.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:18 smithi083 ceph-mon[29421]: 6.1a scrub ok 2024-09-20T19:11:18.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:18 smithi142 ceph-mon[32508]: pgmap v1091: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-20T19:11:18.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:18 smithi142 ceph-mon[32508]: 5.0 scrub starts 2024-09-20T19:11:18.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:18 smithi142 ceph-mon[32508]: 5.0 scrub ok 2024-09-20T19:11:18.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:18 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:11:18.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:18 smithi142 ceph-mon[32508]: 6.1a scrub starts 2024-09-20T19:11:18.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:18 smithi142 ceph-mon[32508]: 6.1a scrub ok 2024-09-20T19:11:18.997 INFO:teuthology.orchestra.run.smithi083.stdout:1305670058105 2024-09-20T19:11:18.998 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.4 2024-09-20T19:11:19.564 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:19 smithi083 ceph-mon[29421]: 4.11 scrub starts 2024-09-20T19:11:19.565 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:19 smithi083 ceph-mon[29421]: 4.11 scrub ok 2024-09-20T19:11:19.565 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:19 smithi083 ceph-mon[29421]: 3.0 scrub starts 2024-09-20T19:11:19.565 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:19 smithi083 ceph-mon[29421]: 3.0 scrub ok 2024-09-20T19:11:19.565 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:19 smithi083 ceph-mon[36260]: 4.11 scrub starts 2024-09-20T19:11:19.565 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:19 smithi083 ceph-mon[36260]: 4.11 scrub ok 2024-09-20T19:11:19.565 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:19 smithi083 ceph-mon[36260]: 3.0 scrub starts 2024-09-20T19:11:19.565 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:19 smithi083 ceph-mon[36260]: 3.0 scrub ok 2024-09-20T19:11:19.566 INFO:teuthology.orchestra.run.smithi083.stdout:163208757523 2024-09-20T19:11:19.566 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.5 2024-09-20T19:11:19.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:19 smithi142 ceph-mon[32508]: 4.11 scrub starts 2024-09-20T19:11:19.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:19 smithi142 ceph-mon[32508]: 4.11 scrub ok 2024-09-20T19:11:19.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:19 smithi142 ceph-mon[32508]: 3.0 scrub starts 2024-09-20T19:11:19.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:19 smithi142 ceph-mon[32508]: 3.0 scrub ok 2024-09-20T19:11:19.966 INFO:teuthology.orchestra.run.smithi083.stdout:107374182686 2024-09-20T19:11:19.966 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.3 2024-09-20T19:11:20.567 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:20 smithi083 ceph-mon[29421]: pgmap v1092: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-20T19:11:20.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:20 smithi083 ceph-mon[29421]: 5.14 scrub starts 2024-09-20T19:11:20.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:20 smithi083 ceph-mon[29421]: 5.14 scrub ok 2024-09-20T19:11:20.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:20 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T19:11:20.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:20 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T19:11:20.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:20 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T19:11:20.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:20 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:11:20.568 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:20 smithi083 ceph-mon[36260]: pgmap v1092: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-20T19:11:20.568 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:20 smithi083 ceph-mon[36260]: 5.14 scrub starts 2024-09-20T19:11:20.569 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:20 smithi083 ceph-mon[36260]: 5.14 scrub ok 2024-09-20T19:11:20.569 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:20 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T19:11:20.569 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:20 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T19:11:20.569 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:20 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T19:11:20.569 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:20 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:11:20.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:20 smithi142 ceph-mon[32508]: pgmap v1092: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-20T19:11:20.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:20 smithi142 ceph-mon[32508]: 5.14 scrub starts 2024-09-20T19:11:20.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:20 smithi142 ceph-mon[32508]: 5.14 scrub ok 2024-09-20T19:11:20.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:20 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T19:11:20.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:20 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T19:11:20.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:20 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T19:11:20.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:20 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:11:20.880 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:11:20 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:11:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:11:21.302 INFO:teuthology.orchestra.run.smithi083.stdout:77309411620 2024-09-20T19:11:21.303 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.2 2024-09-20T19:11:21.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:21 smithi083 ceph-mon[29421]: 6.1f scrub starts 2024-09-20T19:11:21.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:21 smithi083 ceph-mon[29421]: 6.1f scrub ok 2024-09-20T19:11:21.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:21 smithi083 ceph-mon[29421]: 3.4 scrub starts 2024-09-20T19:11:21.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:21 smithi083 ceph-mon[29421]: 3.4 scrub ok 2024-09-20T19:11:21.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:21 smithi083 ceph-mon[29421]: 3.14 scrub starts 2024-09-20T19:11:21.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:21 smithi083 ceph-mon[29421]: 3.14 scrub ok 2024-09-20T19:11:21.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:21 smithi083 ceph-mon[36260]: 6.1f scrub starts 2024-09-20T19:11:21.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:21 smithi083 ceph-mon[36260]: 6.1f scrub ok 2024-09-20T19:11:21.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:21 smithi083 ceph-mon[36260]: 3.4 scrub starts 2024-09-20T19:11:21.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:21 smithi083 ceph-mon[36260]: 3.4 scrub ok 2024-09-20T19:11:21.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:21 smithi083 ceph-mon[36260]: 3.14 scrub starts 2024-09-20T19:11:21.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:21 smithi083 ceph-mon[36260]: 3.14 scrub ok 2024-09-20T19:11:21.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:21 smithi142 ceph-mon[32508]: 6.1f scrub starts 2024-09-20T19:11:21.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:21 smithi142 ceph-mon[32508]: 6.1f scrub ok 2024-09-20T19:11:21.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:21 smithi142 ceph-mon[32508]: 3.4 scrub starts 2024-09-20T19:11:21.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:21 smithi142 ceph-mon[32508]: 3.4 scrub ok 2024-09-20T19:11:21.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:21 smithi142 ceph-mon[32508]: 3.14 scrub starts 2024-09-20T19:11:21.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:21 smithi142 ceph-mon[32508]: 3.14 scrub ok 2024-09-20T19:11:22.128 INFO:teuthology.orchestra.run.smithi083.stdout:55834575145 2024-09-20T19:11:22.128 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.1 2024-09-20T19:11:22.613 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:22 smithi083 ceph-mon[29421]: pgmap v1093: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 938 B/s rd, 0 op/s 2024-09-20T19:11:22.613 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:22 smithi083 ceph-mon[29421]: 6.b scrub starts 2024-09-20T19:11:22.613 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:22 smithi083 ceph-mon[29421]: 6.b scrub ok 2024-09-20T19:11:22.613 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:22 smithi083 ceph-mon[29421]: 6.3 scrub starts 2024-09-20T19:11:22.613 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:22 smithi083 ceph-mon[29421]: 6.3 scrub ok 2024-09-20T19:11:22.613 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:22 smithi083 ceph-mon[36260]: pgmap v1093: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 938 B/s rd, 0 op/s 2024-09-20T19:11:22.613 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:22 smithi083 ceph-mon[36260]: 6.b scrub starts 2024-09-20T19:11:22.613 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:22 smithi083 ceph-mon[36260]: 6.b scrub ok 2024-09-20T19:11:22.613 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:22 smithi083 ceph-mon[36260]: 6.3 scrub starts 2024-09-20T19:11:22.613 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:22 smithi083 ceph-mon[36260]: 6.3 scrub ok 2024-09-20T19:11:22.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:22 smithi142 ceph-mon[32508]: pgmap v1093: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 938 B/s rd, 0 op/s 2024-09-20T19:11:22.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:22 smithi142 ceph-mon[32508]: 6.b scrub starts 2024-09-20T19:11:22.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:22 smithi142 ceph-mon[32508]: 6.b scrub ok 2024-09-20T19:11:22.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:22 smithi142 ceph-mon[32508]: 6.3 scrub starts 2024-09-20T19:11:22.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:22 smithi142 ceph-mon[32508]: 6.3 scrub ok 2024-09-20T19:11:22.872 INFO:teuthology.orchestra.run.smithi083.stdout:193273528590 2024-09-20T19:11:22.872 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.6 2024-09-20T19:11:23.521 INFO:teuthology.orchestra.run.smithi083.stdout:2190433320971 2024-09-20T19:11:23.522 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.0 2024-09-20T19:11:23.526 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:11:23.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:23 smithi083 ceph-mon[29421]: 4.6 scrub starts 2024-09-20T19:11:23.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:23 smithi083 ceph-mon[29421]: 4.6 scrub ok 2024-09-20T19:11:23.658 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:23 smithi083 ceph-mon[36260]: 4.6 scrub starts 2024-09-20T19:11:23.658 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:23 smithi083 ceph-mon[36260]: 4.6 scrub ok 2024-09-20T19:11:23.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:23 smithi142 ceph-mon[32508]: 4.6 scrub starts 2024-09-20T19:11:23.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:23 smithi142 ceph-mon[32508]: 4.6 scrub ok 2024-09-20T19:11:24.109 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:11:24.109 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:11:24.110 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:11:24.112 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:11:24.113 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:11:24.259 INFO:teuthology.orchestra.run.smithi083.stdout:55834575147 2024-09-20T19:11:24.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:24 smithi083 ceph-mon[29421]: 5.a scrub starts 2024-09-20T19:11:24.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:24 smithi083 ceph-mon[29421]: 5.a scrub ok 2024-09-20T19:11:24.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:24 smithi083 ceph-mon[29421]: pgmap v1094: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T19:11:24.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:24 smithi083 ceph-mon[29421]: 6.8 scrub starts 2024-09-20T19:11:24.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:24 smithi083 ceph-mon[29421]: 6.8 scrub ok 2024-09-20T19:11:24.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:24 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/632540770' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-20T19:11:24.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:24 smithi083 ceph-mon[36260]: 5.a scrub starts 2024-09-20T19:11:24.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:24 smithi083 ceph-mon[36260]: 5.a scrub ok 2024-09-20T19:11:24.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:24 smithi083 ceph-mon[36260]: pgmap v1094: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T19:11:24.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:24 smithi083 ceph-mon[36260]: 6.8 scrub starts 2024-09-20T19:11:24.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:24 smithi083 ceph-mon[36260]: 6.8 scrub ok 2024-09-20T19:11:24.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:24 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/632540770' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-20T19:11:24.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:24 smithi142 ceph-mon[32508]: 5.a scrub starts 2024-09-20T19:11:24.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:24 smithi142 ceph-mon[32508]: 5.a scrub ok 2024-09-20T19:11:24.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:24 smithi142 ceph-mon[32508]: pgmap v1094: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T19:11:24.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:24 smithi142 ceph-mon[32508]: 6.8 scrub starts 2024-09-20T19:11:24.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:24 smithi142 ceph-mon[32508]: 6.8 scrub ok 2024-09-20T19:11:24.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:24 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/632540770' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-20T19:11:25.099 INFO:teuthology.orchestra.run.smithi083.stdout:1305670058106 2024-09-20T19:11:25.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:25 smithi083 ceph-mon[36260]: 4.16 scrub starts 2024-09-20T19:11:25.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:25 smithi083 ceph-mon[36260]: 4.16 scrub ok 2024-09-20T19:11:25.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:25 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:11:25.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:25 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:11:25.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:25 smithi083 ceph-mon[36260]: osdmap e532: 8 total, 8 up, 8 in 2024-09-20T19:11:25.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:25 smithi083 ceph-mon[36260]: 4.1f scrub starts 2024-09-20T19:11:25.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:25 smithi083 ceph-mon[36260]: 4.1f scrub ok 2024-09-20T19:11:25.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:25 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/4081613423' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-20T19:11:25.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:25 smithi083 ceph-mon[29421]: 4.16 scrub starts 2024-09-20T19:11:25.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:25 smithi083 ceph-mon[29421]: 4.16 scrub ok 2024-09-20T19:11:25.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:25 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:11:25.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:25 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:11:25.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:25 smithi083 ceph-mon[29421]: osdmap e532: 8 total, 8 up, 8 in 2024-09-20T19:11:25.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:25 smithi083 ceph-mon[29421]: 4.1f scrub starts 2024-09-20T19:11:25.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:25 smithi083 ceph-mon[29421]: 4.1f scrub ok 2024-09-20T19:11:25.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:25 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/4081613423' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-20T19:11:25.661 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:25 smithi142 ceph-mon[32508]: 4.16 scrub starts 2024-09-20T19:11:25.661 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:25 smithi142 ceph-mon[32508]: 4.16 scrub ok 2024-09-20T19:11:25.661 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:25 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:11:25.661 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:25 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:11:25.661 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:25 smithi142 ceph-mon[32508]: osdmap e532: 8 total, 8 up, 8 in 2024-09-20T19:11:25.661 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:25 smithi142 ceph-mon[32508]: 4.1f scrub starts 2024-09-20T19:11:25.661 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:25 smithi142 ceph-mon[32508]: 4.1f scrub ok 2024-09-20T19:11:25.661 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:25 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/4081613423' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-20T19:11:25.896 INFO:tasks.cephadm.ceph_manager.ceph:need seq 1305670058105 got 1305670058106 for osd.4 2024-09-20T19:11:25.896 DEBUG:teuthology.parallel:result is None 2024-09-20T19:11:26.640 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:26 smithi083 ceph-mon[29421]: 3.1f scrub starts 2024-09-20T19:11:26.641 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:26 smithi083 ceph-mon[29421]: 3.1f scrub ok 2024-09-20T19:11:26.641 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:26 smithi083 ceph-mon[29421]: 5.17 scrub starts 2024-09-20T19:11:26.641 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:26 smithi083 ceph-mon[29421]: 5.17 scrub ok 2024-09-20T19:11:26.641 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:26 smithi083 ceph-mon[29421]: pgmap v1096: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-20T19:11:26.641 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:26 smithi083 ceph-mon[29421]: 6.14 scrub starts 2024-09-20T19:11:26.641 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:26 smithi083 ceph-mon[29421]: 6.14 scrub ok 2024-09-20T19:11:26.641 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:26 smithi083 ceph-mon[29421]: osdmap e533: 8 total, 8 up, 8 in 2024-09-20T19:11:26.641 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:26 smithi083 ceph-mon[36260]: 3.1f scrub starts 2024-09-20T19:11:26.641 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:26 smithi083 ceph-mon[36260]: 3.1f scrub ok 2024-09-20T19:11:26.642 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:26 smithi083 ceph-mon[36260]: 5.17 scrub starts 2024-09-20T19:11:26.642 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:26 smithi083 ceph-mon[36260]: 5.17 scrub ok 2024-09-20T19:11:26.642 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:26 smithi083 ceph-mon[36260]: pgmap v1096: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-20T19:11:26.642 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:26 smithi083 ceph-mon[36260]: 6.14 scrub starts 2024-09-20T19:11:26.642 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:26 smithi083 ceph-mon[36260]: 6.14 scrub ok 2024-09-20T19:11:26.642 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:26 smithi083 ceph-mon[36260]: osdmap e533: 8 total, 8 up, 8 in 2024-09-20T19:11:26.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:26 smithi142 ceph-mon[32508]: 3.1f scrub starts 2024-09-20T19:11:26.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:26 smithi142 ceph-mon[32508]: 3.1f scrub ok 2024-09-20T19:11:26.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:26 smithi142 ceph-mon[32508]: 5.17 scrub starts 2024-09-20T19:11:26.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:26 smithi142 ceph-mon[32508]: 5.17 scrub ok 2024-09-20T19:11:26.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:26 smithi142 ceph-mon[32508]: pgmap v1096: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-20T19:11:26.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:26 smithi142 ceph-mon[32508]: 6.14 scrub starts 2024-09-20T19:11:26.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:26 smithi142 ceph-mon[32508]: 6.14 scrub ok 2024-09-20T19:11:26.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:26 smithi142 ceph-mon[32508]: osdmap e533: 8 total, 8 up, 8 in 2024-09-20T19:11:26.917 INFO:teuthology.orchestra.run.smithi083.stdout:227633266952 2024-09-20T19:11:27.159 INFO:teuthology.orchestra.run.smithi083.stdout:163208757524 2024-09-20T19:11:27.382 INFO:teuthology.orchestra.run.smithi083.stdout:107374182689 2024-09-20T19:11:27.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:27 smithi083 ceph-mon[29421]: 6.1 scrub starts 2024-09-20T19:11:27.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:27 smithi083 ceph-mon[29421]: 6.1 scrub ok 2024-09-20T19:11:27.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:27 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2639080467' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-20T19:11:27.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:27 smithi083 ceph-mon[29421]: 6.2 scrub starts 2024-09-20T19:11:27.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:27 smithi083 ceph-mon[29421]: 6.2 scrub ok 2024-09-20T19:11:27.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:27 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/2435834308' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-20T19:11:27.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:27 smithi083 ceph-mon[36260]: 6.1 scrub starts 2024-09-20T19:11:27.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:27 smithi083 ceph-mon[36260]: 6.1 scrub ok 2024-09-20T19:11:27.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:27 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2639080467' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-20T19:11:27.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:27 smithi083 ceph-mon[36260]: 6.2 scrub starts 2024-09-20T19:11:27.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:27 smithi083 ceph-mon[36260]: 6.2 scrub ok 2024-09-20T19:11:27.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:27 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/2435834308' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-20T19:11:27.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:27 smithi142 ceph-mon[32508]: 6.1 scrub starts 2024-09-20T19:11:27.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:27 smithi142 ceph-mon[32508]: 6.1 scrub ok 2024-09-20T19:11:27.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:27 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2639080467' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-20T19:11:27.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:27 smithi142 ceph-mon[32508]: 6.2 scrub starts 2024-09-20T19:11:27.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:27 smithi142 ceph-mon[32508]: 6.2 scrub ok 2024-09-20T19:11:27.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:27 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/2435834308' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-20T19:11:27.910 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757523 got 163208757524 for osd.5 2024-09-20T19:11:27.910 DEBUG:teuthology.parallel:result is None 2024-09-20T19:11:28.009 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:11:27 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:11:28.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:28 smithi083 ceph-mon[29421]: pgmap v1098: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-20T19:11:28.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:28 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/1208435450' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-20T19:11:28.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:28 smithi083 ceph-mon[29421]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:11:28.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:28 smithi083 ceph-mon[36260]: pgmap v1098: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-20T19:11:28.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:28 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/1208435450' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-20T19:11:28.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:28 smithi083 ceph-mon[36260]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:11:28.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:28 smithi142 ceph-mon[32508]: pgmap v1098: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-20T19:11:28.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:28 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/1208435450' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-20T19:11:28.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:28 smithi142 ceph-mon[32508]: from='client.14526 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T19:11:28.955 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:11:28.955 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:11:28.977 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266951 got 227633266952 for osd.7 2024-09-20T19:11:28.977 DEBUG:teuthology.parallel:result is None 2024-09-20T19:11:29.398 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834575145 got 55834575147 for osd.1 2024-09-20T19:11:29.398 DEBUG:teuthology.parallel:result is None 2024-09-20T19:11:29.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:29 smithi142 ceph-mon[32508]: 4.8 scrub starts 2024-09-20T19:11:29.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:29 smithi142 ceph-mon[32508]: 4.8 scrub ok 2024-09-20T19:11:29.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:29 smithi142 ceph-mon[32508]: 3.2 scrub starts 2024-09-20T19:11:29.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:29 smithi142 ceph-mon[32508]: 3.2 scrub ok 2024-09-20T19:11:29.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:29 smithi142 ceph-mon[32508]: 1.9 scrub starts 2024-09-20T19:11:29.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:29 smithi142 ceph-mon[32508]: 1.9 scrub ok 2024-09-20T19:11:29.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:29 smithi142 ceph-mon[32508]: 6.f scrub starts 2024-09-20T19:11:29.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:29 smithi142 ceph-mon[32508]: 6.f scrub ok 2024-09-20T19:11:29.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:29 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/3199091238' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-20T19:11:29.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:29 smithi142 ceph-mon[32508]: 4.1b scrub starts 2024-09-20T19:11:29.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:29 smithi142 ceph-mon[32508]: 4.1b scrub ok 2024-09-20T19:11:29.778 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:29 smithi083 ceph-mon[29421]: 4.8 scrub starts 2024-09-20T19:11:29.779 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:29 smithi083 ceph-mon[29421]: 4.8 scrub ok 2024-09-20T19:11:29.779 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:29 smithi083 ceph-mon[29421]: 3.2 scrub starts 2024-09-20T19:11:29.779 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:29 smithi083 ceph-mon[29421]: 3.2 scrub ok 2024-09-20T19:11:29.779 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:29 smithi083 ceph-mon[29421]: 1.9 scrub starts 2024-09-20T19:11:29.779 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:29 smithi083 ceph-mon[29421]: 1.9 scrub ok 2024-09-20T19:11:29.779 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:29 smithi083 ceph-mon[29421]: 6.f scrub starts 2024-09-20T19:11:29.779 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:29 smithi083 ceph-mon[29421]: 6.f scrub ok 2024-09-20T19:11:29.779 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:29 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/3199091238' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-20T19:11:29.779 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:29 smithi083 ceph-mon[29421]: 4.1b scrub starts 2024-09-20T19:11:29.779 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:29 smithi083 ceph-mon[29421]: 4.1b scrub ok 2024-09-20T19:11:29.779 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:29 smithi083 ceph-mon[36260]: 4.8 scrub starts 2024-09-20T19:11:29.779 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:29 smithi083 ceph-mon[36260]: 4.8 scrub ok 2024-09-20T19:11:29.780 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:29 smithi083 ceph-mon[36260]: 3.2 scrub starts 2024-09-20T19:11:29.780 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:29 smithi083 ceph-mon[36260]: 3.2 scrub ok 2024-09-20T19:11:29.780 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:29 smithi083 ceph-mon[36260]: 1.9 scrub starts 2024-09-20T19:11:29.780 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:29 smithi083 ceph-mon[36260]: 1.9 scrub ok 2024-09-20T19:11:29.780 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:29 smithi083 ceph-mon[36260]: 6.f scrub starts 2024-09-20T19:11:29.780 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:29 smithi083 ceph-mon[36260]: 6.f scrub ok 2024-09-20T19:11:29.780 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:29 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/3199091238' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-20T19:11:29.780 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:29 smithi083 ceph-mon[36260]: 4.1b scrub starts 2024-09-20T19:11:29.780 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:29 smithi083 ceph-mon[36260]: 4.1b scrub ok 2024-09-20T19:11:29.810 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182686 got 107374182689 for osd.3 2024-09-20T19:11:29.810 DEBUG:teuthology.parallel:result is None 2024-09-20T19:11:29.895 INFO:teuthology.orchestra.run.smithi083.stdout:77309411622 2024-09-20T19:11:30.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:30 smithi083 ceph-mon[29421]: pgmap v1099: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-20T19:11:30.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:30 smithi083 ceph-mon[36260]: pgmap v1099: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-20T19:11:30.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:30 smithi142 ceph-mon[32508]: pgmap v1099: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-20T19:11:31.030 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:11:30 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:11:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:11:31.059 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411620 got 77309411622 for osd.2 2024-09-20T19:11:31.059 DEBUG:teuthology.parallel:result is None 2024-09-20T19:11:31.417 INFO:teuthology.orchestra.run.smithi083.stdout:193273528592 2024-09-20T19:11:31.660 INFO:teuthology.orchestra.run.smithi083.stdout:2190433320973 2024-09-20T19:11:31.661 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:31 smithi083 ceph-mon[29421]: 6.7 scrub starts 2024-09-20T19:11:31.661 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:31 smithi083 ceph-mon[29421]: 6.7 scrub ok 2024-09-20T19:11:31.661 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:31 smithi083 ceph-mon[29421]: 5.9 scrub starts 2024-09-20T19:11:31.661 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:31 smithi083 ceph-mon[29421]: 5.9 scrub ok 2024-09-20T19:11:31.661 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:31 smithi083 ceph-mon[29421]: 6.4 scrub starts 2024-09-20T19:11:31.661 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:31 smithi083 ceph-mon[29421]: 6.4 scrub ok 2024-09-20T19:11:31.661 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:31 smithi083 ceph-mon[36260]: 6.7 scrub starts 2024-09-20T19:11:31.661 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:31 smithi083 ceph-mon[36260]: 6.7 scrub ok 2024-09-20T19:11:31.661 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:31 smithi083 ceph-mon[36260]: 5.9 scrub starts 2024-09-20T19:11:31.662 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:31 smithi083 ceph-mon[36260]: 5.9 scrub ok 2024-09-20T19:11:31.662 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:31 smithi083 ceph-mon[36260]: 6.4 scrub starts 2024-09-20T19:11:31.662 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:31 smithi083 ceph-mon[36260]: 6.4 scrub ok 2024-09-20T19:11:31.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:31 smithi142 ceph-mon[32508]: 6.7 scrub starts 2024-09-20T19:11:31.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:31 smithi142 ceph-mon[32508]: 6.7 scrub ok 2024-09-20T19:11:31.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:31 smithi142 ceph-mon[32508]: 5.9 scrub starts 2024-09-20T19:11:31.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:31 smithi142 ceph-mon[32508]: 5.9 scrub ok 2024-09-20T19:11:31.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:31 smithi142 ceph-mon[32508]: 6.4 scrub starts 2024-09-20T19:11:31.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:31 smithi142 ceph-mon[32508]: 6.4 scrub ok 2024-09-20T19:11:32.383 INFO:tasks.cephadm.ceph_manager.ceph:need seq 193273528590 got 193273528592 for osd.6 2024-09-20T19:11:32.384 DEBUG:teuthology.parallel:result is None 2024-09-20T19:11:32.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:32 smithi142 ceph-mon[32508]: 3.6 scrub starts 2024-09-20T19:11:32.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:32 smithi142 ceph-mon[32508]: 3.6 scrub ok 2024-09-20T19:11:32.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:32 smithi142 ceph-mon[32508]: purged_snaps scrub starts 2024-09-20T19:11:32.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:32 smithi142 ceph-mon[32508]: purged_snaps scrub ok 2024-09-20T19:11:32.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:32 smithi142 ceph-mon[32508]: pgmap v1100: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-20T19:11:32.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:32 smithi142 ceph-mon[32508]: 5.10 scrub starts 2024-09-20T19:11:32.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:32 smithi142 ceph-mon[32508]: 5.10 scrub ok 2024-09-20T19:11:32.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:32 smithi142 ceph-mon[32508]: osdmap e534: 8 total, 8 up, 8 in 2024-09-20T19:11:32.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:32 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/810983245' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-20T19:11:32.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:32 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/146576027' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-20T19:11:32.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:32 smithi142 ceph-mon[32508]: 5.11 scrub starts 2024-09-20T19:11:32.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:32 smithi142 ceph-mon[32508]: 5.11 scrub ok 2024-09-20T19:11:32.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:32 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:11:32.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:32 smithi142 ceph-mon[32508]: 4.e scrub starts 2024-09-20T19:11:32.761 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:32 smithi142 ceph-mon[32508]: 4.e scrub ok 2024-09-20T19:11:32.790 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:32 smithi083 ceph-mon[29421]: 3.6 scrub starts 2024-09-20T19:11:32.790 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:32 smithi083 ceph-mon[29421]: 3.6 scrub ok 2024-09-20T19:11:32.790 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:32 smithi083 ceph-mon[29421]: purged_snaps scrub starts 2024-09-20T19:11:32.790 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:32 smithi083 ceph-mon[29421]: purged_snaps scrub ok 2024-09-20T19:11:32.790 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:32 smithi083 ceph-mon[29421]: pgmap v1100: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-20T19:11:32.790 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:32 smithi083 ceph-mon[29421]: 5.10 scrub starts 2024-09-20T19:11:32.790 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:32 smithi083 ceph-mon[29421]: 5.10 scrub ok 2024-09-20T19:11:32.790 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:32 smithi083 ceph-mon[29421]: osdmap e534: 8 total, 8 up, 8 in 2024-09-20T19:11:32.790 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:32 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/810983245' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-20T19:11:32.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:32 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/146576027' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-20T19:11:32.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:32 smithi083 ceph-mon[29421]: 5.11 scrub starts 2024-09-20T19:11:32.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:32 smithi083 ceph-mon[29421]: 5.11 scrub ok 2024-09-20T19:11:32.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:32 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:11:32.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:32 smithi083 ceph-mon[29421]: 4.e scrub starts 2024-09-20T19:11:32.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:32 smithi083 ceph-mon[29421]: 4.e scrub ok 2024-09-20T19:11:32.794 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:32 smithi083 ceph-mon[36260]: 3.6 scrub starts 2024-09-20T19:11:32.794 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:32 smithi083 ceph-mon[36260]: 3.6 scrub ok 2024-09-20T19:11:32.794 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:32 smithi083 ceph-mon[36260]: purged_snaps scrub starts 2024-09-20T19:11:32.794 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:32 smithi083 ceph-mon[36260]: purged_snaps scrub ok 2024-09-20T19:11:32.794 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:32 smithi083 ceph-mon[36260]: pgmap v1100: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-20T19:11:32.794 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:32 smithi083 ceph-mon[36260]: 5.10 scrub starts 2024-09-20T19:11:32.795 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:32 smithi083 ceph-mon[36260]: 5.10 scrub ok 2024-09-20T19:11:32.795 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:32 smithi083 ceph-mon[36260]: osdmap e534: 8 total, 8 up, 8 in 2024-09-20T19:11:32.795 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:32 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/810983245' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-20T19:11:32.795 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:32 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/146576027' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-20T19:11:32.795 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:32 smithi083 ceph-mon[36260]: 5.11 scrub starts 2024-09-20T19:11:32.795 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:32 smithi083 ceph-mon[36260]: 5.11 scrub ok 2024-09-20T19:11:32.795 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:32 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-20T19:11:32.795 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:32 smithi083 ceph-mon[36260]: 4.e scrub starts 2024-09-20T19:11:32.795 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:32 smithi083 ceph-mon[36260]: 4.e scrub ok 2024-09-20T19:11:32.852 INFO:tasks.cephadm.ceph_manager.ceph:need seq 2190433320971 got 2190433320973 for osd.0 2024-09-20T19:11:32.852 DEBUG:teuthology.parallel:result is None 2024-09-20T19:11:32.852 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-09-20T19:11:32.853 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T19:11:33.112 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:11:33.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:33 smithi083 ceph-mon[29421]: 2.2 scrub starts 2024-09-20T19:11:33.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:33 smithi083 ceph-mon[29421]: 2.2 scrub ok 2024-09-20T19:11:33.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:33 smithi083 ceph-mon[36260]: 2.2 scrub starts 2024-09-20T19:11:33.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:33 smithi083 ceph-mon[36260]: 2.2 scrub ok 2024-09-20T19:11:33.732 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T19:11:33.734 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-20T19:11:33.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:33 smithi142 ceph-mon[32508]: 2.2 scrub starts 2024-09-20T19:11:33.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:33 smithi142 ceph-mon[32508]: 2.2 scrub ok 2024-09-20T19:11:34.330 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":1102,"stamp":"2024-09-20T19:11:33.268741+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":2849,"num_read_kb":2601,"num_write":1008,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":7747,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1950,"ondisk_log_size":1950,"up":426,"acting":426,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":426,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":5241528,"kb_used_data":9632,"kb_used_omap":19,"kb_used_meta":5231724,"kb_avail":744719688,"statfs":{"total":767960285184,"available":762592960512,"internally_reserved":0,"allocated":9863168,"data_stored":5996726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19640,"internal_metadata":5357286216},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001937"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":962,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800244+0000","last_change":"2024-09-20T19:11:24.538639+0000","last_active":"2024-09-20T19:11:26.800244+0000","last_peered":"2024-09-20T19:11:26.800244+0000","last_clean":"2024-09-20T19:11:26.800244+0000","last_became_active":"2024-09-20T19:10:33.066631+0000","last_became_peered":"2024-09-20T19:10:33.066631+0000","last_unstale":"2024-09-20T19:11:26.800244+0000","last_undegraded":"2024-09-20T19:11:26.800244+0000","last_fullsized":"2024-09-20T19:11:26.800244+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:24.538587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:11:24.538587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:39.722056+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"503'21","reported_seq":364,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.002798+0000","last_change":"2024-09-20T19:10:24.850308+0000","last_active":"2024-09-20T19:11:26.002798+0000","last_peered":"2024-09-20T19:11:26.002798+0000","last_clean":"2024-09-20T19:11:26.002798+0000","last_became_active":"2024-09-20T19:04:20.925853+0000","last_became_peered":"2024-09-20T19:04:20.925853+0000","last_unstale":"2024-09-20T19:11:26.002798+0000","last_undegraded":"2024-09-20T19:11:26.002798+0000","last_fullsized":"2024-09-20T19:11:26.002798+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'21","last_scrub_stamp":"2024-09-20T19:10:24.850234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:10:24.850234+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:33.407527+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":845,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.351268+0000","last_change":"2024-09-20T19:11:02.485373+0000","last_active":"2024-09-20T19:11:31.351268+0000","last_peered":"2024-09-20T19:11:31.351268+0000","last_clean":"2024-09-20T19:11:31.351268+0000","last_became_active":"2024-09-20T19:10:41.802004+0000","last_became_peered":"2024-09-20T19:10:41.802004+0000","last_unstale":"2024-09-20T19:11:31.351268+0000","last_undegraded":"2024-09-20T19:11:31.351268+0000","last_fullsized":"2024-09-20T19:11:31.351268+0000","mapping_epoch":514,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":515,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:02.485325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:11:02.485325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:03.674166+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":345,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.003237+0000","last_change":"2024-09-20T19:11:18.075615+0000","last_active":"2024-09-20T19:11:26.003237+0000","last_peered":"2024-09-20T19:11:26.003237+0000","last_clean":"2024-09-20T19:11:26.003237+0000","last_became_active":"2024-09-20T19:04:20.880905+0000","last_became_peered":"2024-09-20T19:04:20.880905+0000","last_unstale":"2024-09-20T19:11:26.003237+0000","last_undegraded":"2024-09-20T19:11:26.003237+0000","last_fullsized":"2024-09-20T19:11:26.003237+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:18.075542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:11:18.075542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:43.868766+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1015,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.799951+0000","last_change":"2024-09-20T19:11:02.212586+0000","last_active":"2024-09-20T19:11:26.799951+0000","last_peered":"2024-09-20T19:11:26.799951+0000","last_clean":"2024-09-20T19:11:26.799951+0000","last_became_active":"2024-09-20T18:59:05.081081+0000","last_became_peered":"2024-09-20T18:59:05.081081+0000","last_unstale":"2024-09-20T19:11:26.799951+0000","last_undegraded":"2024-09-20T19:11:26.799951+0000","last_fullsized":"2024-09-20T19:11:26.799951+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:02.212529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:11:02.212529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:14.661634+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1029,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800663+0000","last_change":"2024-09-20T19:11:15.173377+0000","last_active":"2024-09-20T19:11:26.800663+0000","last_peered":"2024-09-20T19:11:26.800663+0000","last_clean":"2024-09-20T19:11:26.800663+0000","last_became_active":"2024-09-20T18:59:05.072675+0000","last_became_peered":"2024-09-20T18:59:05.072675+0000","last_unstale":"2024-09-20T19:11:26.800663+0000","last_undegraded":"2024-09-20T19:11:26.800663+0000","last_fullsized":"2024-09-20T19:11:26.800663+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:15.173333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:11:15.173333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:28.036486+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"503'39","reported_seq":1104,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800971+0000","last_change":"2024-09-20T19:10:55.119441+0000","last_active":"2024-09-20T19:11:26.800971+0000","last_peered":"2024-09-20T19:11:26.800971+0000","last_clean":"2024-09-20T19:11:26.800971+0000","last_became_active":"2024-09-20T19:10:33.069997+0000","last_became_peered":"2024-09-20T19:10:33.069997+0000","last_unstale":"2024-09-20T19:11:26.800971+0000","last_undegraded":"2024-09-20T19:11:26.800971+0000","last_fullsized":"2024-09-20T19:11:26.800971+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'39","last_scrub_stamp":"2024-09-20T19:10:55.119374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:10:55.119374+0000","objects_scrubbed":7,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:10.666604+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":345,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.002815+0000","last_change":"2024-09-20T19:10:12.758720+0000","last_active":"2024-09-20T19:11:26.002815+0000","last_peered":"2024-09-20T19:11:26.002815+0000","last_clean":"2024-09-20T19:11:26.002815+0000","last_became_active":"2024-09-20T19:04:20.925394+0000","last_became_peered":"2024-09-20T19:04:20.925394+0000","last_unstale":"2024-09-20T19:11:26.002815+0000","last_undegraded":"2024-09-20T19:11:26.002815+0000","last_fullsized":"2024-09-20T19:11:26.002815+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:12.758650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:10:12.758650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"59'1","reported_seq":1090,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800550+0000","last_change":"2024-09-20T19:11:11.016408+0000","last_active":"2024-09-20T19:11:26.800550+0000","last_peered":"2024-09-20T19:11:26.800550+0000","last_clean":"2024-09-20T19:11:26.800550+0000","last_became_active":"2024-09-20T19:04:20.914145+0000","last_became_peered":"2024-09-20T19:04:20.914145+0000","last_unstale":"2024-09-20T19:11:26.800550+0000","last_undegraded":"2024-09-20T19:11:26.800550+0000","last_fullsized":"2024-09-20T19:11:26.800550+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:11:11.016347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:11:11.016347+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:24.620595+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"503'21","reported_seq":376,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.003317+0000","last_change":"2024-09-20T19:10:35.823487+0000","last_active":"2024-09-20T19:11:26.003317+0000","last_peered":"2024-09-20T19:11:26.003317+0000","last_clean":"2024-09-20T19:11:26.003317+0000","last_became_active":"2024-09-20T19:10:33.058094+0000","last_became_peered":"2024-09-20T19:10:33.058094+0000","last_unstale":"2024-09-20T19:11:26.003317+0000","last_undegraded":"2024-09-20T19:11:26.003317+0000","last_fullsized":"2024-09-20T19:11:26.003317+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'21","last_scrub_stamp":"2024-09-20T19:10:35.823388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:10:35.823388+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:42.981473+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1024,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800054+0000","last_change":"2024-09-20T19:10:52.005533+0000","last_active":"2024-09-20T19:11:26.800054+0000","last_peered":"2024-09-20T19:11:26.800054+0000","last_clean":"2024-09-20T19:11:26.800054+0000","last_became_active":"2024-09-20T19:10:33.054613+0000","last_became_peered":"2024-09-20T19:10:33.054613+0000","last_unstale":"2024-09-20T19:11:26.800054+0000","last_undegraded":"2024-09-20T19:11:26.800054+0000","last_fullsized":"2024-09-20T19:11:26.800054+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:52.005475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:10:52.005475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:04.756633+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":948,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.799401+0000","last_change":"2024-09-20T19:11:02.450255+0000","last_active":"2024-09-20T19:11:26.799401+0000","last_peered":"2024-09-20T19:11:26.799401+0000","last_clean":"2024-09-20T19:11:26.799401+0000","last_became_active":"2024-09-20T19:10:33.061000+0000","last_became_peered":"2024-09-20T19:10:33.061000+0000","last_unstale":"2024-09-20T19:11:26.799401+0000","last_undegraded":"2024-09-20T19:11:26.799401+0000","last_fullsized":"2024-09-20T19:11:26.799401+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:02.450209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:11:02.450209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:04.893685+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1038,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800315+0000","last_change":"2024-09-20T19:11:11.993880+0000","last_active":"2024-09-20T19:11:26.800315+0000","last_peered":"2024-09-20T19:11:26.800315+0000","last_clean":"2024-09-20T19:11:26.800315+0000","last_became_active":"2024-09-20T19:04:20.878905+0000","last_became_peered":"2024-09-20T19:04:20.878905+0000","last_unstale":"2024-09-20T19:11:26.800315+0000","last_undegraded":"2024-09-20T19:11:26.800315+0000","last_fullsized":"2024-09-20T19:11:26.800315+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:11.993819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:11:11.993819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:34.760402+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"503'9","reported_seq":362,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:29.028487+0000","last_change":"2024-09-20T19:11:29.028487+0000","last_active":"2024-09-20T19:11:29.028487+0000","last_peered":"2024-09-20T19:11:29.028487+0000","last_clean":"2024-09-20T19:11:29.028487+0000","last_became_active":"2024-09-20T19:04:20.888572+0000","last_became_peered":"2024-09-20T19:04:20.888572+0000","last_unstale":"2024-09-20T19:11:29.028487+0000","last_undegraded":"2024-09-20T19:11:29.028487+0000","last_fullsized":"2024-09-20T19:11:29.028487+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'9","last_scrub_stamp":"2024-09-20T19:11:29.028427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:11:29.028427+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:54.155717+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":12,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1031,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.197147+0000","last_change":"2024-09-20T19:10:51.022419+0000","last_active":"2024-09-20T19:11:26.197147+0000","last_peered":"2024-09-20T19:11:26.197147+0000","last_clean":"2024-09-20T19:11:26.197147+0000","last_became_active":"2024-09-20T19:04:20.885417+0000","last_became_peered":"2024-09-20T19:04:20.885417+0000","last_unstale":"2024-09-20T19:11:26.197147+0000","last_undegraded":"2024-09-20T19:11:26.197147+0000","last_fullsized":"2024-09-20T19:11:26.197147+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:51.022353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:10:51.022353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:03.028955+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1009,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.799873+0000","last_change":"2024-09-20T19:11:06.015570+0000","last_active":"2024-09-20T19:11:26.799873+0000","last_peered":"2024-09-20T19:11:26.799873+0000","last_clean":"2024-09-20T19:11:26.799873+0000","last_became_active":"2024-09-20T18:49:51.970663+0000","last_became_peered":"2024-09-20T18:49:51.970663+0000","last_unstale":"2024-09-20T19:11:26.799873+0000","last_undegraded":"2024-09-20T19:11:26.799873+0000","last_fullsized":"2024-09-20T19:11:26.799873+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:06.015482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:11:06.015482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:21.398187+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":346,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.003914+0000","last_change":"2024-09-20T19:11:09.987842+0000","last_active":"2024-09-20T19:11:26.003914+0000","last_peered":"2024-09-20T19:11:26.003914+0000","last_clean":"2024-09-20T19:11:26.003914+0000","last_became_active":"2024-09-20T19:04:20.906109+0000","last_became_peered":"2024-09-20T19:04:20.906109+0000","last_unstale":"2024-09-20T19:11:26.003914+0000","last_undegraded":"2024-09-20T19:11:26.003914+0000","last_fullsized":"2024-09-20T19:11:26.003914+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:09.987779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:11:09.987779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:31.306115+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":937,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.801085+0000","last_change":"2024-09-20T19:10:40.313128+0000","last_active":"2024-09-20T19:11:26.801085+0000","last_peered":"2024-09-20T19:11:26.801085+0000","last_clean":"2024-09-20T19:11:26.801085+0000","last_became_active":"2024-09-20T19:10:33.058426+0000","last_became_peered":"2024-09-20T19:10:33.058426+0000","last_unstale":"2024-09-20T19:11:26.801085+0000","last_undegraded":"2024-09-20T19:11:26.801085+0000","last_fullsized":"2024-09-20T19:11:26.801085+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:40.313080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:10:40.313080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:40.699676+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"503'39","reported_seq":1091,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.350618+0000","last_change":"2024-09-20T19:10:42.046992+0000","last_active":"2024-09-20T19:11:31.350618+0000","last_peered":"2024-09-20T19:11:31.350618+0000","last_clean":"2024-09-20T19:11:31.350618+0000","last_became_active":"2024-09-20T18:49:47.950939+0000","last_became_peered":"2024-09-20T18:49:47.950939+0000","last_unstale":"2024-09-20T19:11:31.350618+0000","last_undegraded":"2024-09-20T19:11:31.350618+0000","last_fullsized":"2024-09-20T19:11:31.350618+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'39","last_scrub_stamp":"2024-09-20T19:10:42.046937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:10:42.046937+0000","objects_scrubbed":7,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:03.180119+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1035,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.352252+0000","last_change":"2024-09-20T19:10:41.407482+0000","last_active":"2024-09-20T19:11:31.352252+0000","last_peered":"2024-09-20T19:11:31.352252+0000","last_clean":"2024-09-20T19:11:31.352252+0000","last_became_active":"2024-09-20T19:10:33.053415+0000","last_became_peered":"2024-09-20T19:10:33.053415+0000","last_unstale":"2024-09-20T19:11:31.352252+0000","last_undegraded":"2024-09-20T19:11:31.352252+0000","last_fullsized":"2024-09-20T19:11:31.352252+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:41.407434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:10:41.407434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:46.861200+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1025,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800591+0000","last_change":"2024-09-20T19:11:20.252773+0000","last_active":"2024-09-20T19:11:26.800591+0000","last_peered":"2024-09-20T19:11:26.800591+0000","last_clean":"2024-09-20T19:11:26.800591+0000","last_became_active":"2024-09-20T18:59:05.084496+0000","last_became_peered":"2024-09-20T18:59:05.084496+0000","last_unstale":"2024-09-20T19:11:26.800591+0000","last_undegraded":"2024-09-20T19:11:26.800591+0000","last_fullsized":"2024-09-20T19:11:26.800591+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:20.252722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:11:20.252722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:20.802803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"59'1","reported_seq":341,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.003080+0000","last_change":"2024-09-20T19:10:02.651674+0000","last_active":"2024-09-20T19:11:26.003080+0000","last_peered":"2024-09-20T19:11:26.003080+0000","last_clean":"2024-09-20T19:11:26.003080+0000","last_became_active":"2024-09-20T19:04:20.887062+0000","last_became_peered":"2024-09-20T19:04:20.887062+0000","last_unstale":"2024-09-20T19:11:26.003080+0000","last_undegraded":"2024-09-20T19:11:26.003080+0000","last_fullsized":"2024-09-20T19:11:26.003080+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:10:02.651608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:10:02.651608+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":993,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"503'24","reported_seq":1076,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800243+0000","last_change":"2024-09-20T19:10:53.092558+0000","last_active":"2024-09-20T19:11:26.800243+0000","last_peered":"2024-09-20T19:11:26.800243+0000","last_clean":"2024-09-20T19:11:26.800243+0000","last_became_active":"2024-09-20T19:04:20.896343+0000","last_became_peered":"2024-09-20T19:04:20.896343+0000","last_unstale":"2024-09-20T19:11:26.800243+0000","last_undegraded":"2024-09-20T19:11:26.800243+0000","last_fullsized":"2024-09-20T19:11:26.800243+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'24","last_scrub_stamp":"2024-09-20T19:10:53.092470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:10:53.092470+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:15.748746+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":345,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.003614+0000","last_change":"2024-09-20T19:09:53.818967+0000","last_active":"2024-09-20T19:11:26.003614+0000","last_peered":"2024-09-20T19:11:26.003614+0000","last_clean":"2024-09-20T19:11:26.003614+0000","last_became_active":"2024-09-20T19:04:20.889430+0000","last_became_peered":"2024-09-20T19:04:20.889430+0000","last_unstale":"2024-09-20T19:11:26.003614+0000","last_undegraded":"2024-09-20T19:11:26.003614+0000","last_fullsized":"2024-09-20T19:11:26.003614+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:09:53.818919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:09:53.818919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":995,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.197585+0000","last_change":"2024-09-20T19:10:05.135865+0000","last_active":"2024-09-20T19:11:26.197585+0000","last_peered":"2024-09-20T19:11:26.197585+0000","last_clean":"2024-09-20T19:11:26.197585+0000","last_became_active":"2024-09-20T18:49:51.970058+0000","last_became_peered":"2024-09-20T18:49:51.970058+0000","last_unstale":"2024-09-20T19:11:26.197585+0000","last_undegraded":"2024-09-20T19:11:26.197585+0000","last_fullsized":"2024-09-20T19:11:26.197585+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T19:10:05.135811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:10:05.135811+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:27.893747+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":1049,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.351355+0000","last_change":"2024-09-20T19:10:46.507557+0000","last_active":"2024-09-20T19:11:31.351355+0000","last_peered":"2024-09-20T19:11:31.351355+0000","last_clean":"2024-09-20T19:11:31.351355+0000","last_became_active":"2024-09-20T19:04:20.881466+0000","last_became_peered":"2024-09-20T19:04:20.881466+0000","last_unstale":"2024-09-20T19:11:31.351355+0000","last_undegraded":"2024-09-20T19:11:31.351355+0000","last_fullsized":"2024-09-20T19:11:31.351355+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:10:46.507497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:10:46.507497+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:55.995372+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"503'30","reported_seq":1035,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.801297+0000","last_change":"2024-09-20T19:11:01.424998+0000","last_active":"2024-09-20T19:11:26.801297+0000","last_peered":"2024-09-20T19:11:26.801297+0000","last_clean":"2024-09-20T19:11:26.801297+0000","last_became_active":"2024-09-20T19:10:33.069139+0000","last_became_peered":"2024-09-20T19:10:33.069139+0000","last_unstale":"2024-09-20T19:11:26.801297+0000","last_undegraded":"2024-09-20T19:11:26.801297+0000","last_fullsized":"2024-09-20T19:11:26.801297+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'30","last_scrub_stamp":"2024-09-20T19:11:01.424946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:11:01.424946+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:23.139948+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":38,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"375'14","reported_seq":792,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.502745+0000","last_change":"2024-09-20T19:11:13.583596+0000","last_active":"2024-09-20T19:11:31.502745+0000","last_peered":"2024-09-20T19:11:31.502745+0000","last_clean":"2024-09-20T19:11:31.502745+0000","last_became_active":"2024-09-20T19:04:20.900782+0000","last_became_peered":"2024-09-20T19:04:20.900782+0000","last_unstale":"2024-09-20T19:11:31.502745+0000","last_undegraded":"2024-09-20T19:11:31.502745+0000","last_fullsized":"2024-09-20T19:11:31.502745+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"375'14","last_scrub_stamp":"2024-09-20T19:11:13.583506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:11:13.583506+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:35.102554+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.a","version":"74'79","reported_seq":178,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.799843+0000","last_change":"2024-09-20T19:10:53.978356+0000","last_active":"2024-09-20T19:11:26.799843+0000","last_peered":"2024-09-20T19:11:26.799843+0000","last_clean":"2024-09-20T19:11:26.799843+0000","last_became_active":"2024-09-20T19:07:42.980316+0000","last_became_peered":"2024-09-20T19:07:42.980316+0000","last_unstale":"2024-09-20T19:11:26.799843+0000","last_undegraded":"2024-09-20T19:11:26.799843+0000","last_fullsized":"2024-09-20T19:11:26.799843+0000","mapping_epoch":458,"log_start":"0'0","ondisk_log_start":"0'0","created":455,"last_epoch_clean":459,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:10:53.978296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:10:53.978296+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:15.764358+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"503'39","reported_seq":1116,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.350883+0000","last_change":"2024-09-20T19:11:10.749016+0000","last_active":"2024-09-20T19:11:31.350883+0000","last_peered":"2024-09-20T19:11:31.350883+0000","last_clean":"2024-09-20T19:11:31.350883+0000","last_became_active":"2024-09-20T19:04:20.897049+0000","last_became_peered":"2024-09-20T19:04:20.897049+0000","last_unstale":"2024-09-20T19:11:31.350883+0000","last_undegraded":"2024-09-20T19:11:31.350883+0000","last_fullsized":"2024-09-20T19:11:31.350883+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'39","last_scrub_stamp":"2024-09-20T19:11:10.748955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:11:10.748955+0000","objects_scrubbed":7,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:37.687162+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1015,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.801018+0000","last_change":"2024-09-20T19:10:22.117116+0000","last_active":"2024-09-20T19:11:26.801018+0000","last_peered":"2024-09-20T19:11:26.801018+0000","last_clean":"2024-09-20T19:11:26.801018+0000","last_became_active":"2024-09-20T18:49:45.936104+0000","last_became_peered":"2024-09-20T18:49:45.936104+0000","last_unstale":"2024-09-20T19:11:26.801018+0000","last_undegraded":"2024-09-20T19:11:26.801018+0000","last_fullsized":"2024-09-20T19:11:26.801018+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:22.117067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:10:22.117067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:37.715930+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"511'14","reported_seq":450,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.003999+0000","last_change":"2024-09-20T19:10:47.817685+0000","last_active":"2024-09-20T19:11:26.003999+0000","last_peered":"2024-09-20T19:11:26.003999+0000","last_clean":"2024-09-20T19:11:26.003999+0000","last_became_active":"2024-09-20T19:10:33.065459+0000","last_became_peered":"2024-09-20T19:10:33.065459+0000","last_unstale":"2024-09-20T19:11:26.003999+0000","last_undegraded":"2024-09-20T19:11:26.003999+0000","last_fullsized":"2024-09-20T19:11:26.003999+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"511'14","last_scrub_stamp":"2024-09-20T19:10:47.817618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:10:47.817618+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:15.033915+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1005,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800217+0000","last_change":"2024-09-20T19:10:33.864924+0000","last_active":"2024-09-20T19:11:26.800217+0000","last_peered":"2024-09-20T19:11:26.800217+0000","last_clean":"2024-09-20T19:11:26.800217+0000","last_became_active":"2024-09-20T19:10:33.055508+0000","last_became_peered":"2024-09-20T19:10:33.055508+0000","last_unstale":"2024-09-20T19:11:26.800217+0000","last_undegraded":"2024-09-20T19:11:26.800217+0000","last_fullsized":"2024-09-20T19:11:26.800217+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:33.864864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:10:33.864864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:59.266093+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"74'79","reported_seq":153,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.351996+0000","last_change":"2024-09-20T19:10:39.432942+0000","last_active":"2024-09-20T19:11:31.351996+0000","last_peered":"2024-09-20T19:11:31.351996+0000","last_clean":"2024-09-20T19:11:31.351996+0000","last_became_active":"2024-09-20T19:10:33.062664+0000","last_became_peered":"2024-09-20T19:10:33.062664+0000","last_unstale":"2024-09-20T19:11:31.351996+0000","last_undegraded":"2024-09-20T19:11:31.351996+0000","last_fullsized":"2024-09-20T19:11:31.351996+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":455,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:10:39.432884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:10:39.432884+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:41.077221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"503'42","reported_seq":1121,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800737+0000","last_change":"2024-09-20T19:10:48.089587+0000","last_active":"2024-09-20T19:11:26.800737+0000","last_peered":"2024-09-20T19:11:26.800737+0000","last_clean":"2024-09-20T19:11:26.800737+0000","last_became_active":"2024-09-20T19:10:33.072974+0000","last_became_peered":"2024-09-20T19:10:33.072974+0000","last_unstale":"2024-09-20T19:11:26.800737+0000","last_undegraded":"2024-09-20T19:11:26.800737+0000","last_fullsized":"2024-09-20T19:11:26.800737+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'42","last_scrub_stamp":"2024-09-20T19:10:48.089522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:10:48.089522+0000","objects_scrubbed":8,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:54.892638+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":70,"num_read_kb":58,"num_write":44,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1018,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800855+0000","last_change":"2024-09-20T19:10:37.066650+0000","last_active":"2024-09-20T19:11:26.800855+0000","last_peered":"2024-09-20T19:11:26.800855+0000","last_clean":"2024-09-20T19:11:26.800855+0000","last_became_active":"2024-09-20T19:10:33.072653+0000","last_became_peered":"2024-09-20T19:10:33.072653+0000","last_unstale":"2024-09-20T19:11:26.800855+0000","last_undegraded":"2024-09-20T19:11:26.800855+0000","last_fullsized":"2024-09-20T19:11:26.800855+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:37.066599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:10:37.066599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:59.273296+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":346,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.003023+0000","last_change":"2024-09-20T19:11:13.047031+0000","last_active":"2024-09-20T19:11:26.003023+0000","last_peered":"2024-09-20T19:11:26.003023+0000","last_clean":"2024-09-20T19:11:26.003023+0000","last_became_active":"2024-09-20T19:04:20.888422+0000","last_became_peered":"2024-09-20T19:04:20.888422+0000","last_unstale":"2024-09-20T19:11:26.003023+0000","last_undegraded":"2024-09-20T19:11:26.003023+0000","last_fullsized":"2024-09-20T19:11:26.003023+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:13.046968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:11:13.046968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:35.748751+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":354,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:27.044192+0000","last_change":"2024-09-20T19:11:27.044192+0000","last_active":"2024-09-20T19:11:27.044192+0000","last_peered":"2024-09-20T19:11:27.044192+0000","last_clean":"2024-09-20T19:11:27.044192+0000","last_became_active":"2024-09-20T19:04:20.889418+0000","last_became_peered":"2024-09-20T19:04:20.889418+0000","last_unstale":"2024-09-20T19:11:27.044192+0000","last_undegraded":"2024-09-20T19:11:27.044192+0000","last_fullsized":"2024-09-20T19:11:27.044192+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:27.044134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:11:27.044134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:34.841052+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"74'79","reported_seq":183,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.799881+0000","last_change":"2024-09-20T19:10:52.068206+0000","last_active":"2024-09-20T19:11:26.799881+0000","last_peered":"2024-09-20T19:11:26.799881+0000","last_clean":"2024-09-20T19:11:26.799881+0000","last_became_active":"2024-09-20T19:07:42.987705+0000","last_became_peered":"2024-09-20T19:07:42.987705+0000","last_unstale":"2024-09-20T19:11:26.799881+0000","last_undegraded":"2024-09-20T19:11:26.799881+0000","last_fullsized":"2024-09-20T19:11:26.799881+0000","mapping_epoch":458,"log_start":"0'0","ondisk_log_start":"0'0","created":455,"last_epoch_clean":461,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:10:52.068154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:10:52.068154+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:07.002653+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"503'42","reported_seq":375,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.002928+0000","last_change":"2024-09-20T19:10:07.661213+0000","last_active":"2024-09-20T19:11:26.002928+0000","last_peered":"2024-09-20T19:11:26.002928+0000","last_clean":"2024-09-20T19:11:26.002928+0000","last_became_active":"2024-09-20T19:04:20.897444+0000","last_became_peered":"2024-09-20T19:04:20.897444+0000","last_unstale":"2024-09-20T19:11:26.002928+0000","last_undegraded":"2024-09-20T19:11:26.002928+0000","last_fullsized":"2024-09-20T19:11:26.002928+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'42","last_scrub_stamp":"2024-09-20T19:10:07.661154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:10:07.661154+0000","objects_scrubbed":7,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:30.338276+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":55,"num_write":42,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":1011,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:30.464758+0000","last_change":"2024-09-20T19:11:30.464758+0000","last_active":"2024-09-20T19:11:30.464758+0000","last_peered":"2024-09-20T19:11:30.464758+0000","last_clean":"2024-09-20T19:11:30.464758+0000","last_became_active":"2024-09-20T19:10:33.058108+0000","last_became_peered":"2024-09-20T19:10:33.058108+0000","last_unstale":"2024-09-20T19:11:30.464758+0000","last_undegraded":"2024-09-20T19:11:30.464758+0000","last_fullsized":"2024-09-20T19:11:30.464758+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:11:30.464706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:11:30.464706+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:39.225544+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"375'13","reported_seq":1323,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800557+0000","last_change":"2024-09-20T19:11:17.271628+0000","last_active":"2024-09-20T19:11:26.800557+0000","last_peered":"2024-09-20T19:11:26.800557+0000","last_clean":"2024-09-20T19:11:26.800557+0000","last_became_active":"2024-09-20T19:04:20.896424+0000","last_became_peered":"2024-09-20T19:04:20.896424+0000","last_unstale":"2024-09-20T19:11:26.800557+0000","last_undegraded":"2024-09-20T19:11:26.800557+0000","last_fullsized":"2024-09-20T19:11:26.800557+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"375'13","last_scrub_stamp":"2024-09-20T19:11:17.271568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:11:17.271568+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:36.937837+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1010,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.197179+0000","last_change":"2024-09-20T19:11:21.324384+0000","last_active":"2024-09-20T19:11:26.197179+0000","last_peered":"2024-09-20T19:11:26.197179+0000","last_clean":"2024-09-20T19:11:26.197179+0000","last_became_active":"2024-09-20T18:59:05.081814+0000","last_became_peered":"2024-09-20T18:59:05.081814+0000","last_unstale":"2024-09-20T19:11:26.197179+0000","last_undegraded":"2024-09-20T19:11:26.197179+0000","last_fullsized":"2024-09-20T19:11:26.197179+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:21.324328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:11:21.324328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:50.932404+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"74'79","reported_seq":682,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.502622+0000","last_change":"2024-09-20T19:11:07.691872+0000","last_active":"2024-09-20T19:11:31.502622+0000","last_peered":"2024-09-20T19:11:31.502622+0000","last_clean":"2024-09-20T19:11:31.502622+0000","last_became_active":"2024-09-20T19:07:42.988925+0000","last_became_peered":"2024-09-20T19:07:42.988925+0000","last_unstale":"2024-09-20T19:11:31.502622+0000","last_undegraded":"2024-09-20T19:11:31.502622+0000","last_fullsized":"2024-09-20T19:11:31.502622+0000","mapping_epoch":458,"log_start":"0'0","ondisk_log_start":"0'0","created":455,"last_epoch_clean":459,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:11:07.691802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:11:07.691802+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:11.842150+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"503'30","reported_seq":876,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800066+0000","last_change":"2024-09-20T19:11:13.960675+0000","last_active":"2024-09-20T19:11:26.800066+0000","last_peered":"2024-09-20T19:11:26.800066+0000","last_clean":"2024-09-20T19:11:26.800066+0000","last_became_active":"2024-09-20T19:04:49.419140+0000","last_became_peered":"2024-09-20T19:04:49.419140+0000","last_unstale":"2024-09-20T19:11:26.800066+0000","last_undegraded":"2024-09-20T19:11:26.800066+0000","last_fullsized":"2024-09-20T19:11:26.800066+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'30","last_scrub_stamp":"2024-09-20T19:11:13.960591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:11:13.960591+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:16.572086+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":39,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1031,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800630+0000","last_change":"2024-09-20T19:10:12.783104+0000","last_active":"2024-09-20T19:11:26.800630+0000","last_peered":"2024-09-20T19:11:26.800630+0000","last_clean":"2024-09-20T19:11:26.800630+0000","last_became_active":"2024-09-20T18:49:45.942496+0000","last_became_peered":"2024-09-20T18:49:45.942496+0000","last_unstale":"2024-09-20T19:11:26.800630+0000","last_undegraded":"2024-09-20T19:11:26.800630+0000","last_fullsized":"2024-09-20T19:11:26.800630+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:12.783056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:10:12.783056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:38.264049+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"511'12","reported_seq":1238,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800824+0000","last_change":"2024-09-20T19:10:49.456915+0000","last_active":"2024-09-20T19:11:26.800824+0000","last_peered":"2024-09-20T19:11:26.800824+0000","last_clean":"2024-09-20T19:11:26.800824+0000","last_became_active":"2024-09-20T19:10:33.070526+0000","last_became_peered":"2024-09-20T19:10:33.070526+0000","last_unstale":"2024-09-20T19:11:26.800824+0000","last_undegraded":"2024-09-20T19:11:26.800824+0000","last_fullsized":"2024-09-20T19:11:26.800824+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"511'12","last_scrub_stamp":"2024-09-20T19:10:49.456843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:10:49.456843+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:11.998844+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":958,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.799313+0000","last_change":"2024-09-20T19:10:33.070955+0000","last_active":"2024-09-20T19:11:26.799313+0000","last_peered":"2024-09-20T19:11:26.799313+0000","last_clean":"2024-09-20T19:11:26.799313+0000","last_became_active":"2024-09-20T19:10:33.068671+0000","last_became_peered":"2024-09-20T19:10:33.068671+0000","last_unstale":"2024-09-20T19:11:26.799313+0000","last_undegraded":"2024-09-20T19:11:26.799313+0000","last_fullsized":"2024-09-20T19:11:26.799313+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:09:15.568243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:09:15.568243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"74'79","reported_seq":685,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.502003+0000","last_change":"2024-09-20T19:10:37.817529+0000","last_active":"2024-09-20T19:11:31.502003+0000","last_peered":"2024-09-20T19:11:31.502003+0000","last_clean":"2024-09-20T19:11:31.502003+0000","last_became_active":"2024-09-20T19:10:33.065986+0000","last_became_peered":"2024-09-20T19:10:33.065986+0000","last_unstale":"2024-09-20T19:11:31.502003+0000","last_undegraded":"2024-09-20T19:11:31.502003+0000","last_fullsized":"2024-09-20T19:11:31.502003+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":455,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:10:37.817466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:10:37.817466+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:59.851352+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"503'39","reported_seq":1035,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800570+0000","last_change":"2024-09-20T19:10:53.372308+0000","last_active":"2024-09-20T19:11:26.800570+0000","last_peered":"2024-09-20T19:11:26.800570+0000","last_clean":"2024-09-20T19:11:26.800570+0000","last_became_active":"2024-09-20T19:10:33.061190+0000","last_became_peered":"2024-09-20T19:10:33.061190+0000","last_unstale":"2024-09-20T19:11:26.800570+0000","last_undegraded":"2024-09-20T19:11:26.800570+0000","last_fullsized":"2024-09-20T19:11:26.800570+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'39","last_scrub_stamp":"2024-09-20T19:10:53.372238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:10:53.372238+0000","objects_scrubbed":8,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:04.218975+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":68,"num_read_kb":54,"num_write":42,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1017,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.352314+0000","last_change":"2024-09-20T19:11:20.582502+0000","last_active":"2024-09-20T19:11:31.352314+0000","last_peered":"2024-09-20T19:11:31.352314+0000","last_clean":"2024-09-20T19:11:31.352314+0000","last_became_active":"2024-09-20T18:49:45.942253+0000","last_became_peered":"2024-09-20T18:49:45.942253+0000","last_unstale":"2024-09-20T19:11:31.352314+0000","last_undegraded":"2024-09-20T19:11:31.352314+0000","last_fullsized":"2024-09-20T19:11:31.352314+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:20.582454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:11:20.582454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:34.232784+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":606,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.502507+0000","last_change":"2024-09-20T19:10:34.806471+0000","last_active":"2024-09-20T19:11:31.502507+0000","last_peered":"2024-09-20T19:11:31.502507+0000","last_clean":"2024-09-20T19:11:31.502507+0000","last_became_active":"2024-09-20T19:10:33.065396+0000","last_became_peered":"2024-09-20T19:10:33.065396+0000","last_unstale":"2024-09-20T19:11:31.502507+0000","last_undegraded":"2024-09-20T19:11:31.502507+0000","last_fullsized":"2024-09-20T19:11:31.502507+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:34.806385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:10:34.806385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:03.745685+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1043,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.351011+0000","last_change":"2024-09-20T19:11:26.605786+0000","last_active":"2024-09-20T19:11:31.351011+0000","last_peered":"2024-09-20T19:11:31.351011+0000","last_clean":"2024-09-20T19:11:31.351011+0000","last_became_active":"2024-09-20T18:59:05.085131+0000","last_became_peered":"2024-09-20T18:59:05.085131+0000","last_unstale":"2024-09-20T19:11:31.351011+0000","last_undegraded":"2024-09-20T19:11:31.351011+0000","last_fullsized":"2024-09-20T19:11:31.351011+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:26.605739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:11:26.605739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:35.075195+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"74'79","reported_seq":189,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.799741+0000","last_change":"2024-09-20T19:10:52.958927+0000","last_active":"2024-09-20T19:11:26.799741+0000","last_peered":"2024-09-20T19:11:26.799741+0000","last_clean":"2024-09-20T19:11:26.799741+0000","last_became_active":"2024-09-20T19:10:33.059058+0000","last_became_peered":"2024-09-20T19:10:33.059058+0000","last_unstale":"2024-09-20T19:11:26.799741+0000","last_undegraded":"2024-09-20T19:11:26.799741+0000","last_fullsized":"2024-09-20T19:11:26.799741+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":455,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:10:52.958863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:10:52.958863+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:04.902192+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"503'53","reported_seq":1404,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.352244+0000","last_change":"2024-09-20T19:11:12.468531+0000","last_active":"2024-09-20T19:11:31.352244+0000","last_peered":"2024-09-20T19:11:31.352244+0000","last_clean":"2024-09-20T19:11:31.352244+0000","last_became_active":"2024-09-20T19:03:16.976916+0000","last_became_peered":"2024-09-20T19:03:16.976916+0000","last_unstale":"2024-09-20T19:11:31.352244+0000","last_undegraded":"2024-09-20T19:11:31.352244+0000","last_fullsized":"2024-09-20T19:11:31.352244+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'53","last_scrub_stamp":"2024-09-20T19:11:12.468447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:11:12.468447+0000","objects_scrubbed":10,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:34.327889+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":84,"num_read_kb":69,"num_write":50,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":357,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.003441+0000","last_change":"2024-09-20T19:10:48.828570+0000","last_active":"2024-09-20T19:11:26.003441+0000","last_peered":"2024-09-20T19:11:26.003441+0000","last_clean":"2024-09-20T19:11:26.003441+0000","last_became_active":"2024-09-20T19:10:33.055871+0000","last_became_peered":"2024-09-20T19:10:33.055871+0000","last_unstale":"2024-09-20T19:11:26.003441+0000","last_undegraded":"2024-09-20T19:11:26.003441+0000","last_fullsized":"2024-09-20T19:11:26.003441+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:48.828503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:10:48.828503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:53.094861+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":1063,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800185+0000","last_change":"2024-09-20T19:10:05.653339+0000","last_active":"2024-09-20T19:11:26.800185+0000","last_peered":"2024-09-20T19:11:26.800185+0000","last_clean":"2024-09-20T19:11:26.800185+0000","last_became_active":"2024-09-20T18:59:05.086786+0000","last_became_peered":"2024-09-20T18:59:05.086786+0000","last_unstale":"2024-09-20T19:11:26.800185+0000","last_undegraded":"2024-09-20T19:11:26.800185+0000","last_fullsized":"2024-09-20T19:11:26.800185+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-20T19:10:05.653292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:10:05.653292+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:28.948889+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0012781159999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":997,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.799757+0000","last_change":"2024-09-20T19:11:08.471903+0000","last_active":"2024-09-20T19:11:26.799757+0000","last_peered":"2024-09-20T19:11:26.799757+0000","last_clean":"2024-09-20T19:11:26.799757+0000","last_became_active":"2024-09-20T19:10:33.055631+0000","last_became_peered":"2024-09-20T19:10:33.055631+0000","last_unstale":"2024-09-20T19:11:26.799757+0000","last_undegraded":"2024-09-20T19:11:26.799757+0000","last_fullsized":"2024-09-20T19:11:26.799757+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:08.471839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:11:08.471839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:24.310778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1012,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800503+0000","last_change":"2024-09-20T19:11:14.129151+0000","last_active":"2024-09-20T19:11:26.800503+0000","last_peered":"2024-09-20T19:11:26.800503+0000","last_clean":"2024-09-20T19:11:26.800503+0000","last_became_active":"2024-09-20T19:04:20.896281+0000","last_became_peered":"2024-09-20T19:04:20.896281+0000","last_unstale":"2024-09-20T19:11:26.800503+0000","last_undegraded":"2024-09-20T19:11:26.800503+0000","last_fullsized":"2024-09-20T19:11:26.800503+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:14.129057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:11:14.129057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:28.639340+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"74'79","reported_seq":1111,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.196990+0000","last_change":"2024-09-20T19:10:47.068872+0000","last_active":"2024-09-20T19:11:26.196990+0000","last_peered":"2024-09-20T19:11:26.196990+0000","last_clean":"2024-09-20T19:11:26.196990+0000","last_became_active":"2024-09-20T19:10:33.056170+0000","last_became_peered":"2024-09-20T19:10:33.056170+0000","last_unstale":"2024-09-20T19:11:26.196990+0000","last_undegraded":"2024-09-20T19:11:26.196990+0000","last_fullsized":"2024-09-20T19:11:26.196990+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":455,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:10:47.068800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:10:47.068800+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:47.880145+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"503'21","reported_seq":1078,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.351916+0000","last_change":"2024-09-20T19:10:36.475204+0000","last_active":"2024-09-20T19:11:31.351916+0000","last_peered":"2024-09-20T19:11:31.351916+0000","last_clean":"2024-09-20T19:11:31.351916+0000","last_became_active":"2024-09-20T19:10:33.062421+0000","last_became_peered":"2024-09-20T19:10:33.062421+0000","last_unstale":"2024-09-20T19:11:31.351916+0000","last_undegraded":"2024-09-20T19:11:31.351916+0000","last_fullsized":"2024-09-20T19:11:31.351916+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'21","last_scrub_stamp":"2024-09-20T19:10:36.475150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:10:36.475150+0000","objects_scrubbed":5,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:41.186787+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":42,"num_read_kb":31,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1047,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.351223+0000","last_change":"2024-09-20T19:11:18.569751+0000","last_active":"2024-09-20T19:11:31.351223+0000","last_peered":"2024-09-20T19:11:31.351223+0000","last_clean":"2024-09-20T19:11:31.351223+0000","last_became_active":"2024-09-20T18:59:05.079789+0000","last_became_peered":"2024-09-20T18:59:05.079789+0000","last_unstale":"2024-09-20T19:11:31.351223+0000","last_undegraded":"2024-09-20T19:11:31.351223+0000","last_fullsized":"2024-09-20T19:11:31.351223+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:18.569691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:11:18.569691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:21.202936+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":1032,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.349912+0000","last_change":"2024-09-20T19:10:41.015564+0000","last_active":"2024-09-20T19:11:31.349912+0000","last_peered":"2024-09-20T19:11:31.349912+0000","last_clean":"2024-09-20T19:11:31.349912+0000","last_became_active":"2024-09-20T19:10:33.070316+0000","last_became_peered":"2024-09-20T19:10:33.070316+0000","last_unstale":"2024-09-20T19:11:31.349912+0000","last_undegraded":"2024-09-20T19:11:31.349912+0000","last_fullsized":"2024-09-20T19:11:31.349912+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:10:41.015471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:10:41.015471+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:04.761830+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00095558800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":996,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.351125+0000","last_change":"2024-09-20T19:11:00.752002+0000","last_active":"2024-09-20T19:11:31.351125+0000","last_peered":"2024-09-20T19:11:31.351125+0000","last_clean":"2024-09-20T19:11:31.351125+0000","last_became_active":"2024-09-20T18:49:49.954424+0000","last_became_peered":"2024-09-20T18:49:49.954424+0000","last_unstale":"2024-09-20T19:11:31.351125+0000","last_undegraded":"2024-09-20T19:11:31.351125+0000","last_fullsized":"2024-09-20T19:11:31.351125+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:00.751949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:11:00.751949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:29.709540+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":997,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.197490+0000","last_change":"2024-09-20T19:10:03.177269+0000","last_active":"2024-09-20T19:11:26.197490+0000","last_peered":"2024-09-20T19:11:26.197490+0000","last_clean":"2024-09-20T19:11:26.197490+0000","last_became_active":"2024-09-20T18:59:05.081459+0000","last_became_peered":"2024-09-20T18:59:05.081459+0000","last_unstale":"2024-09-20T19:11:26.197490+0000","last_undegraded":"2024-09-20T19:11:26.197490+0000","last_fullsized":"2024-09-20T19:11:26.197490+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:03.177222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:10:03.177222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:32.875230+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"74'79","reported_seq":188,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.351799+0000","last_change":"2024-09-20T19:11:06.496487+0000","last_active":"2024-09-20T19:11:31.351799+0000","last_peered":"2024-09-20T19:11:31.351799+0000","last_clean":"2024-09-20T19:11:31.351799+0000","last_became_active":"2024-09-20T19:07:42.991822+0000","last_became_peered":"2024-09-20T19:07:42.991822+0000","last_unstale":"2024-09-20T19:11:31.351799+0000","last_undegraded":"2024-09-20T19:11:31.351799+0000","last_fullsized":"2024-09-20T19:11:31.351799+0000","mapping_epoch":458,"log_start":"0'0","ondisk_log_start":"0'0","created":455,"last_epoch_clean":459,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:11:06.496437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:11:06.496437+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:11.805852+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"503'36","reported_seq":1011,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.801229+0000","last_change":"2024-09-20T19:11:21.505648+0000","last_active":"2024-09-20T19:11:26.801229+0000","last_peered":"2024-09-20T19:11:26.801229+0000","last_clean":"2024-09-20T19:11:26.801229+0000","last_became_active":"2024-09-20T19:10:33.066863+0000","last_became_peered":"2024-09-20T19:10:33.066863+0000","last_unstale":"2024-09-20T19:11:26.801229+0000","last_undegraded":"2024-09-20T19:11:26.801229+0000","last_fullsized":"2024-09-20T19:11:26.801229+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'36","last_scrub_stamp":"2024-09-20T19:11:21.505584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:11:21.505584+0000","objects_scrubbed":6,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:39.637463+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":52,"num_read_kb":46,"num_write":36,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":938,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.799802+0000","last_change":"2024-09-20T19:10:50.457309+0000","last_active":"2024-09-20T19:11:26.799802+0000","last_peered":"2024-09-20T19:11:26.799802+0000","last_clean":"2024-09-20T19:11:26.799802+0000","last_became_active":"2024-09-20T19:10:33.067036+0000","last_became_peered":"2024-09-20T19:10:33.067036+0000","last_unstale":"2024-09-20T19:11:26.799802+0000","last_undegraded":"2024-09-20T19:11:26.799802+0000","last_fullsized":"2024-09-20T19:11:26.799802+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:50.457253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:10:50.457253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:11.294614+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":2335,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:30.148101+0000","last_change":"2024-09-20T19:10:44.087064+0000","last_active":"2024-09-20T19:11:30.148101+0000","last_peered":"2024-09-20T19:11:30.148101+0000","last_clean":"2024-09-20T19:11:30.148101+0000","last_became_active":"2024-09-20T19:10:33.054628+0000","last_became_peered":"2024-09-20T19:10:33.054628+0000","last_unstale":"2024-09-20T19:11:30.148101+0000","last_undegraded":"2024-09-20T19:11:30.148101+0000","last_fullsized":"2024-09-20T19:11:30.148101+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-20T19:10:44.086956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:10:44.086956+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:06.195464+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0010996199999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1295,"num_read_kb":1290,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1026,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800703+0000","last_change":"2024-09-20T19:10:48.916403+0000","last_active":"2024-09-20T19:11:26.800703+0000","last_peered":"2024-09-20T19:11:26.800703+0000","last_clean":"2024-09-20T19:11:26.800703+0000","last_became_active":"2024-09-20T19:10:33.055822+0000","last_became_peered":"2024-09-20T19:10:33.055822+0000","last_unstale":"2024-09-20T19:11:26.800703+0000","last_undegraded":"2024-09-20T19:11:26.800703+0000","last_fullsized":"2024-09-20T19:11:26.800703+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:48.916355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:10:48.916355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:09.089843+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1021,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.352064+0000","last_change":"2024-09-20T19:11:30.578866+0000","last_active":"2024-09-20T19:11:31.352064+0000","last_peered":"2024-09-20T19:11:31.352064+0000","last_clean":"2024-09-20T19:11:31.352064+0000","last_became_active":"2024-09-20T18:49:51.970343+0000","last_became_peered":"2024-09-20T18:49:51.970343+0000","last_unstale":"2024-09-20T19:11:31.352064+0000","last_undegraded":"2024-09-20T19:11:31.352064+0000","last_fullsized":"2024-09-20T19:11:31.352064+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:30.578798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:11:30.578798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:48.536129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"503'46","reported_seq":862,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.502389+0000","last_change":"2024-09-20T19:11:14.575075+0000","last_active":"2024-09-20T19:11:31.502389+0000","last_peered":"2024-09-20T19:11:31.502389+0000","last_clean":"2024-09-20T19:11:31.502389+0000","last_became_active":"2024-09-20T19:04:20.928727+0000","last_became_peered":"2024-09-20T19:04:20.928727+0000","last_unstale":"2024-09-20T19:11:31.502389+0000","last_undegraded":"2024-09-20T19:11:31.502389+0000","last_fullsized":"2024-09-20T19:11:31.502389+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'46","last_scrub_stamp":"2024-09-20T19:11:14.574982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:11:14.574982+0000","objects_scrubbed":8,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:28.154337+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":60,"num_read_kb":50,"num_write":37,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":462,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1045,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:28.223790+0000","last_change":"2024-09-20T19:11:28.223790+0000","last_active":"2024-09-20T19:11:28.223790+0000","last_peered":"2024-09-20T19:11:28.223790+0000","last_clean":"2024-09-20T19:11:28.223790+0000","last_became_active":"2024-09-20T18:59:05.079111+0000","last_became_peered":"2024-09-20T18:59:05.079111+0000","last_unstale":"2024-09-20T19:11:28.223790+0000","last_undegraded":"2024-09-20T19:11:28.223790+0000","last_fullsized":"2024-09-20T19:11:28.223790+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:28.223738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:11:28.223738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:50.811519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":1106,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.197162+0000","last_change":"2024-09-20T19:10:45.068703+0000","last_active":"2024-09-20T19:11:26.197162+0000","last_peered":"2024-09-20T19:11:26.197162+0000","last_clean":"2024-09-20T19:11:26.197162+0000","last_became_active":"2024-09-20T19:10:33.055105+0000","last_became_peered":"2024-09-20T19:10:33.055105+0000","last_unstale":"2024-09-20T19:11:26.197162+0000","last_undegraded":"2024-09-20T19:11:26.197162+0000","last_fullsized":"2024-09-20T19:11:26.197162+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:10:45.068650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:10:45.068650+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:13.176018+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1000,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.196746+0000","last_change":"2024-09-20T19:09:55.125713+0000","last_active":"2024-09-20T19:11:26.196746+0000","last_peered":"2024-09-20T19:11:26.196746+0000","last_clean":"2024-09-20T19:11:26.196746+0000","last_became_active":"2024-09-20T18:49:49.956128+0000","last_became_peered":"2024-09-20T18:49:49.956128+0000","last_unstale":"2024-09-20T19:11:26.196746+0000","last_undegraded":"2024-09-20T19:11:26.196746+0000","last_fullsized":"2024-09-20T19:11:26.196746+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:09:55.125653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:09:55.125653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1044,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:29.976955+0000","last_change":"2024-09-20T19:11:29.976906+0000","last_active":"2024-09-20T19:11:29.976955+0000","last_peered":"2024-09-20T19:11:29.976955+0000","last_clean":"2024-09-20T19:11:29.976955+0000","last_became_active":"2024-09-20T19:04:20.899321+0000","last_became_peered":"2024-09-20T19:04:20.899321+0000","last_unstale":"2024-09-20T19:11:29.976955+0000","last_undegraded":"2024-09-20T19:11:29.976955+0000","last_fullsized":"2024-09-20T19:11:29.976955+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:29.976839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:11:29.976839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:44.848330+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"503'27","reported_seq":370,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.003771+0000","last_change":"2024-09-20T19:09:08.725949+0000","last_active":"2024-09-20T19:11:26.003771+0000","last_peered":"2024-09-20T19:11:26.003771+0000","last_clean":"2024-09-20T19:11:26.003771+0000","last_became_active":"2024-09-20T19:04:20.897175+0000","last_became_peered":"2024-09-20T19:04:20.897175+0000","last_unstale":"2024-09-20T19:11:26.003771+0000","last_undegraded":"2024-09-20T19:11:26.003771+0000","last_fullsized":"2024-09-20T19:11:26.003771+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'19","last_scrub_stamp":"2024-09-20T19:09:08.725879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:09:08.725879+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":36,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":346,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.003073+0000","last_change":"2024-09-20T19:10:20.816540+0000","last_active":"2024-09-20T19:11:26.003073+0000","last_peered":"2024-09-20T19:11:26.003073+0000","last_clean":"2024-09-20T19:11:26.003073+0000","last_became_active":"2024-09-20T19:04:20.887102+0000","last_became_peered":"2024-09-20T19:04:20.887102+0000","last_unstale":"2024-09-20T19:11:26.003073+0000","last_undegraded":"2024-09-20T19:11:26.003073+0000","last_fullsized":"2024-09-20T19:11:26.003073+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:20.816485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:10:20.816485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:45.763638+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1053,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800516+0000","last_change":"2024-09-20T19:11:15.002216+0000","last_active":"2024-09-20T19:11:26.800516+0000","last_peered":"2024-09-20T19:11:26.800516+0000","last_clean":"2024-09-20T19:11:26.800516+0000","last_became_active":"2024-09-20T19:04:20.899331+0000","last_became_peered":"2024-09-20T19:04:20.899331+0000","last_unstale":"2024-09-20T19:11:26.800516+0000","last_undegraded":"2024-09-20T19:11:26.800516+0000","last_fullsized":"2024-09-20T19:11:26.800516+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:15.002153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:11:15.002153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:26.273304+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1016,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.799957+0000","last_change":"2024-09-20T19:11:06.275569+0000","last_active":"2024-09-20T19:11:26.799957+0000","last_peered":"2024-09-20T19:11:26.799957+0000","last_clean":"2024-09-20T19:11:26.799957+0000","last_became_active":"2024-09-20T18:59:05.079897+0000","last_became_peered":"2024-09-20T18:59:05.079897+0000","last_unstale":"2024-09-20T19:11:26.799957+0000","last_undegraded":"2024-09-20T19:11:26.799957+0000","last_fullsized":"2024-09-20T19:11:26.799957+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:06.275512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:11:06.275512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:27.646469+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.8","version":"74'79","reported_seq":185,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.799630+0000","last_change":"2024-09-20T19:11:08.032372+0000","last_active":"2024-09-20T19:11:26.799630+0000","last_peered":"2024-09-20T19:11:26.799630+0000","last_clean":"2024-09-20T19:11:26.799630+0000","last_became_active":"2024-09-20T19:07:42.988175+0000","last_became_peered":"2024-09-20T19:07:42.988175+0000","last_unstale":"2024-09-20T19:11:26.799630+0000","last_undegraded":"2024-09-20T19:11:26.799630+0000","last_fullsized":"2024-09-20T19:11:26.799630+0000","mapping_epoch":458,"log_start":"0'0","ondisk_log_start":"0'0","created":455,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:11:08.032295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:11:08.032295+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:30.843530+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"503'45","reported_seq":385,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.003827+0000","last_change":"2024-09-20T19:10:51.809954+0000","last_active":"2024-09-20T19:11:26.003827+0000","last_peered":"2024-09-20T19:11:26.003827+0000","last_clean":"2024-09-20T19:11:26.003827+0000","last_became_active":"2024-09-20T19:04:20.883306+0000","last_became_peered":"2024-09-20T19:04:20.883306+0000","last_unstale":"2024-09-20T19:11:26.003827+0000","last_undegraded":"2024-09-20T19:11:26.003827+0000","last_fullsized":"2024-09-20T19:11:26.003827+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'45","last_scrub_stamp":"2024-09-20T19:10:51.809866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:10:51.809866+0000","objects_scrubbed":8,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:59.563385+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":70,"num_read_kb":60,"num_write":46,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":612,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.502689+0000","last_change":"2024-09-20T19:11:15.534315+0000","last_active":"2024-09-20T19:11:31.502689+0000","last_peered":"2024-09-20T19:11:31.502689+0000","last_clean":"2024-09-20T19:11:31.502689+0000","last_became_active":"2024-09-20T19:04:20.899415+0000","last_became_peered":"2024-09-20T19:04:20.899415+0000","last_unstale":"2024-09-20T19:11:31.502689+0000","last_undegraded":"2024-09-20T19:11:31.502689+0000","last_fullsized":"2024-09-20T19:11:31.502689+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:15.534261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:11:15.534261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:41.287064+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1044,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.352165+0000","last_change":"2024-09-20T19:10:37.482578+0000","last_active":"2024-09-20T19:11:31.352165+0000","last_peered":"2024-09-20T19:11:31.352165+0000","last_clean":"2024-09-20T19:11:31.352165+0000","last_became_active":"2024-09-20T19:10:33.061076+0000","last_became_peered":"2024-09-20T19:10:33.061076+0000","last_unstale":"2024-09-20T19:11:31.352165+0000","last_undegraded":"2024-09-20T19:11:31.352165+0000","last_fullsized":"2024-09-20T19:11:31.352165+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:37.482528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:10:37.482528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:54.510997+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1040,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.351400+0000","last_change":"2024-09-20T19:11:28.421417+0000","last_active":"2024-09-20T19:11:31.351400+0000","last_peered":"2024-09-20T19:11:31.351400+0000","last_clean":"2024-09-20T19:11:31.351400+0000","last_became_active":"2024-09-20T19:04:20.897569+0000","last_became_peered":"2024-09-20T19:04:20.897569+0000","last_unstale":"2024-09-20T19:11:31.351400+0000","last_undegraded":"2024-09-20T19:11:31.351400+0000","last_fullsized":"2024-09-20T19:11:31.351400+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:28.421366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:11:28.421366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:32.852109+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"74'79","reported_seq":1100,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:28.231862+0000","last_change":"2024-09-20T19:11:28.231862+0000","last_active":"2024-09-20T19:11:28.231862+0000","last_peered":"2024-09-20T19:11:28.231862+0000","last_clean":"2024-09-20T19:11:28.231862+0000","last_became_active":"2024-09-20T19:07:42.989434+0000","last_became_peered":"2024-09-20T19:07:42.989434+0000","last_unstale":"2024-09-20T19:11:28.231862+0000","last_undegraded":"2024-09-20T19:11:28.231862+0000","last_fullsized":"2024-09-20T19:11:28.231862+0000","mapping_epoch":458,"log_start":"0'0","ondisk_log_start":"0'0","created":455,"last_epoch_clean":459,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:11:28.231803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:11:28.231803+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:51.068626+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"503'30","reported_seq":365,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.003460+0000","last_change":"2024-09-20T19:10:06.663245+0000","last_active":"2024-09-20T19:11:26.003460+0000","last_peered":"2024-09-20T19:11:26.003460+0000","last_clean":"2024-09-20T19:11:26.003460+0000","last_became_active":"2024-09-20T19:04:20.896483+0000","last_became_peered":"2024-09-20T19:04:20.896483+0000","last_unstale":"2024-09-20T19:11:26.003460+0000","last_undegraded":"2024-09-20T19:11:26.003460+0000","last_fullsized":"2024-09-20T19:11:26.003460+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'30","last_scrub_stamp":"2024-09-20T19:10:06.663192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:10:06.663192+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:33.785131+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":39,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1050,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800290+0000","last_change":"2024-09-20T19:10:36.020262+0000","last_active":"2024-09-20T19:11:26.800290+0000","last_peered":"2024-09-20T19:11:26.800290+0000","last_clean":"2024-09-20T19:11:26.800290+0000","last_became_active":"2024-09-20T19:10:33.066807+0000","last_became_peered":"2024-09-20T19:10:33.066807+0000","last_unstale":"2024-09-20T19:11:26.800290+0000","last_undegraded":"2024-09-20T19:11:26.800290+0000","last_fullsized":"2024-09-20T19:11:26.800290+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:36.020215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:10:36.020215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:52.192006+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":1297,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.350144+0000","last_change":"2024-09-20T19:11:01.758072+0000","last_active":"2024-09-20T19:11:31.350144+0000","last_peered":"2024-09-20T19:11:31.350144+0000","last_clean":"2024-09-20T19:11:31.350144+0000","last_became_active":"2024-09-20T18:49:49.957005+0000","last_became_peered":"2024-09-20T18:49:49.957005+0000","last_unstale":"2024-09-20T19:11:31.350144+0000","last_undegraded":"2024-09-20T19:11:31.350144+0000","last_fullsized":"2024-09-20T19:11:31.350144+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T19:11:01.758014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:11:01.758014+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:05.141869+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":346,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.003837+0000","last_change":"2024-09-20T19:11:03.923503+0000","last_active":"2024-09-20T19:11:26.003837+0000","last_peered":"2024-09-20T19:11:26.003837+0000","last_clean":"2024-09-20T19:11:26.003837+0000","last_became_active":"2024-09-20T19:04:20.884994+0000","last_became_peered":"2024-09-20T19:04:20.884994+0000","last_unstale":"2024-09-20T19:11:26.003837+0000","last_undegraded":"2024-09-20T19:11:26.003837+0000","last_fullsized":"2024-09-20T19:11:26.003837+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:03.923445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:11:03.923445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:26.634146+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"503'21","reported_seq":1051,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800933+0000","last_change":"2024-09-20T19:10:33.080187+0000","last_active":"2024-09-20T19:11:26.800933+0000","last_peered":"2024-09-20T19:11:26.800933+0000","last_clean":"2024-09-20T19:11:26.800933+0000","last_became_active":"2024-09-20T19:10:33.055843+0000","last_became_peered":"2024-09-20T19:10:33.055843+0000","last_unstale":"2024-09-20T19:11:26.800933+0000","last_undegraded":"2024-09-20T19:11:26.800933+0000","last_fullsized":"2024-09-20T19:11:26.800933+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'15","last_scrub_stamp":"2024-09-20T19:09:10.482098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:09:10.482098+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":577,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.350150+0000","last_change":"2024-09-20T19:10:27.013476+0000","last_active":"2024-09-20T19:11:31.350150+0000","last_peered":"2024-09-20T19:11:31.350150+0000","last_clean":"2024-09-20T19:11:31.350150+0000","last_became_active":"2024-09-20T19:00:01.190875+0000","last_became_peered":"2024-09-20T19:00:01.190875+0000","last_unstale":"2024-09-20T19:11:31.350150+0000","last_undegraded":"2024-09-20T19:11:31.350150+0000","last_fullsized":"2024-09-20T19:11:31.350150+0000","mapping_epoch":277,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":278,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:27.013413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:10:27.013413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:38.292329+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1045,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.350065+0000","last_change":"2024-09-20T19:11:22.540975+0000","last_active":"2024-09-20T19:11:31.350065+0000","last_peered":"2024-09-20T19:11:31.350065+0000","last_clean":"2024-09-20T19:11:31.350065+0000","last_became_active":"2024-09-20T19:04:20.889273+0000","last_became_peered":"2024-09-20T19:04:20.889273+0000","last_unstale":"2024-09-20T19:11:31.350065+0000","last_undegraded":"2024-09-20T19:11:31.350065+0000","last_fullsized":"2024-09-20T19:11:31.350065+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:22.540918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:11:22.540918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:50.723472+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":909,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.799901+0000","last_change":"2024-09-20T19:10:51.450775+0000","last_active":"2024-09-20T19:11:26.799901+0000","last_peered":"2024-09-20T19:11:26.799901+0000","last_clean":"2024-09-20T19:11:26.799901+0000","last_became_active":"2024-09-20T19:10:33.065201+0000","last_became_peered":"2024-09-20T19:10:33.065201+0000","last_unstale":"2024-09-20T19:11:26.799901+0000","last_undegraded":"2024-09-20T19:11:26.799901+0000","last_fullsized":"2024-09-20T19:11:26.799901+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:51.450714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:10:51.450714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:55.908573+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"503'51","reported_seq":671,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.502009+0000","last_change":"2024-09-20T19:11:10.634984+0000","last_active":"2024-09-20T19:11:31.502009+0000","last_peered":"2024-09-20T19:11:31.502009+0000","last_clean":"2024-09-20T19:11:31.502009+0000","last_became_active":"2024-09-20T18:59:05.087391+0000","last_became_peered":"2024-09-20T18:59:05.087391+0000","last_unstale":"2024-09-20T19:11:31.502009+0000","last_undegraded":"2024-09-20T19:11:31.502009+0000","last_fullsized":"2024-09-20T19:11:31.502009+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'51","last_scrub_stamp":"2024-09-20T19:11:10.634911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:11:10.634911+0000","objects_scrubbed":9,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:30.842412+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":79,"num_read_kb":68,"num_write":52,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1029,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.197822+0000","last_change":"2024-09-20T19:10:39.057768+0000","last_active":"2024-09-20T19:11:26.197822+0000","last_peered":"2024-09-20T19:11:26.197822+0000","last_clean":"2024-09-20T19:11:26.197822+0000","last_became_active":"2024-09-20T18:59:05.086670+0000","last_became_peered":"2024-09-20T18:59:05.086670+0000","last_unstale":"2024-09-20T19:11:26.197822+0000","last_undegraded":"2024-09-20T19:11:26.197822+0000","last_fullsized":"2024-09-20T19:11:26.197822+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:39.057708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:10:39.057708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:58.262736+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1030,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.350050+0000","last_change":"2024-09-20T19:10:38.994859+0000","last_active":"2024-09-20T19:11:31.350050+0000","last_peered":"2024-09-20T19:11:31.350050+0000","last_clean":"2024-09-20T19:11:31.350050+0000","last_became_active":"2024-09-20T19:10:33.073428+0000","last_became_peered":"2024-09-20T19:10:33.073428+0000","last_unstale":"2024-09-20T19:11:31.350050+0000","last_undegraded":"2024-09-20T19:11:31.350050+0000","last_fullsized":"2024-09-20T19:11:31.350050+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:38.994809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:10:38.994809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:08.197204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":995,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.196973+0000","last_change":"2024-09-20T19:11:23.272980+0000","last_active":"2024-09-20T19:11:26.196973+0000","last_peered":"2024-09-20T19:11:26.196973+0000","last_clean":"2024-09-20T19:11:26.196973+0000","last_became_active":"2024-09-20T18:49:51.970798+0000","last_became_peered":"2024-09-20T18:49:51.970798+0000","last_unstale":"2024-09-20T19:11:26.196973+0000","last_undegraded":"2024-09-20T19:11:26.196973+0000","last_fullsized":"2024-09-20T19:11:26.196973+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:23.272918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:11:23.272918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:34.736526+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"503'24","reported_seq":359,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.003236+0000","last_change":"2024-09-20T19:10:01.667939+0000","last_active":"2024-09-20T19:11:26.003236+0000","last_peered":"2024-09-20T19:11:26.003236+0000","last_clean":"2024-09-20T19:11:26.003236+0000","last_became_active":"2024-09-20T19:04:20.888905+0000","last_became_peered":"2024-09-20T19:04:20.888905+0000","last_unstale":"2024-09-20T19:11:26.003236+0000","last_undegraded":"2024-09-20T19:11:26.003236+0000","last_fullsized":"2024-09-20T19:11:26.003236+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'24","last_scrub_stamp":"2024-09-20T19:10:01.667857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:10:01.667857+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":42,"num_read_kb":33,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1056,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.197868+0000","last_change":"2024-09-20T19:11:00.050521+0000","last_active":"2024-09-20T19:11:26.197868+0000","last_peered":"2024-09-20T19:11:26.197868+0000","last_clean":"2024-09-20T19:11:26.197868+0000","last_became_active":"2024-09-20T19:04:20.892429+0000","last_became_peered":"2024-09-20T19:04:20.892429+0000","last_unstale":"2024-09-20T19:11:26.197868+0000","last_undegraded":"2024-09-20T19:11:26.197868+0000","last_fullsized":"2024-09-20T19:11:26.197868+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:00.050463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:11:00.050463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:13.299758+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1023,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.349767+0000","last_change":"2024-09-20T19:10:37.957926+0000","last_active":"2024-09-20T19:11:31.349767+0000","last_peered":"2024-09-20T19:11:31.349767+0000","last_clean":"2024-09-20T19:11:31.349767+0000","last_became_active":"2024-09-20T19:10:33.071122+0000","last_became_peered":"2024-09-20T19:10:33.071122+0000","last_unstale":"2024-09-20T19:11:31.349767+0000","last_undegraded":"2024-09-20T19:11:31.349767+0000","last_fullsized":"2024-09-20T19:11:31.349767+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:37.957844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:10:37.957844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:42.877703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1010,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.799885+0000","last_change":"2024-09-20T19:11:21.296277+0000","last_active":"2024-09-20T19:11:26.799885+0000","last_peered":"2024-09-20T19:11:26.799885+0000","last_clean":"2024-09-20T19:11:26.799885+0000","last_became_active":"2024-09-20T18:49:51.972970+0000","last_became_peered":"2024-09-20T18:49:51.972970+0000","last_unstale":"2024-09-20T19:11:26.799885+0000","last_undegraded":"2024-09-20T19:11:26.799885+0000","last_fullsized":"2024-09-20T19:11:26.799885+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:21.296228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:11:21.296228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:29.809923+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"503'39","reported_seq":1134,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:27.988380+0000","last_change":"2024-09-20T19:11:27.988380+0000","last_active":"2024-09-20T19:11:27.988380+0000","last_peered":"2024-09-20T19:11:27.988380+0000","last_clean":"2024-09-20T19:11:27.988380+0000","last_became_active":"2024-09-20T18:59:05.083726+0000","last_became_peered":"2024-09-20T18:59:05.083726+0000","last_unstale":"2024-09-20T19:11:27.988380+0000","last_undegraded":"2024-09-20T19:11:27.988380+0000","last_fullsized":"2024-09-20T19:11:27.988380+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'39","last_scrub_stamp":"2024-09-20T19:11:27.988290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:11:27.988290+0000","objects_scrubbed":7,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:55.230857+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":1091,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.197977+0000","last_change":"2024-09-20T19:10:43.059450+0000","last_active":"2024-09-20T19:11:26.197977+0000","last_peered":"2024-09-20T19:11:26.197977+0000","last_clean":"2024-09-20T19:11:26.197977+0000","last_became_active":"2024-09-20T19:10:33.053470+0000","last_became_peered":"2024-09-20T19:10:33.053470+0000","last_unstale":"2024-09-20T19:11:26.197977+0000","last_undegraded":"2024-09-20T19:11:26.197977+0000","last_fullsized":"2024-09-20T19:11:26.197977+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-20T19:10:43.059386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:10:43.059386+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:01.060512+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"375'14","reported_seq":1335,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.258588+0000","last_change":"2024-09-20T19:11:30.224880+0000","last_active":"2024-09-20T19:11:31.258588+0000","last_peered":"2024-09-20T19:11:31.258588+0000","last_clean":"2024-09-20T19:11:31.258588+0000","last_became_active":"2024-09-20T19:04:20.892956+0000","last_became_peered":"2024-09-20T19:04:20.892956+0000","last_unstale":"2024-09-20T19:11:31.258588+0000","last_undegraded":"2024-09-20T19:11:31.258588+0000","last_fullsized":"2024-09-20T19:11:31.258588+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"375'14","last_scrub_stamp":"2024-09-20T19:11:30.224806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:11:30.224806+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:51.715022+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1027,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.799930+0000","last_change":"2024-09-20T19:10:42.828430+0000","last_active":"2024-09-20T19:11:26.799930+0000","last_peered":"2024-09-20T19:11:26.799930+0000","last_clean":"2024-09-20T19:11:26.799930+0000","last_became_active":"2024-09-20T19:10:33.056617+0000","last_became_peered":"2024-09-20T19:10:33.056617+0000","last_unstale":"2024-09-20T19:11:26.799930+0000","last_undegraded":"2024-09-20T19:11:26.799930+0000","last_fullsized":"2024-09-20T19:11:26.799930+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:42.828351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:10:42.828351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:45.665648+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":598,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.500248+0000","last_change":"2024-09-20T19:10:41.884113+0000","last_active":"2024-09-20T19:11:31.500248+0000","last_peered":"2024-09-20T19:11:31.500248+0000","last_clean":"2024-09-20T19:11:31.500248+0000","last_became_active":"2024-09-20T19:10:33.065397+0000","last_became_peered":"2024-09-20T19:10:33.065397+0000","last_unstale":"2024-09-20T19:11:31.500248+0000","last_undegraded":"2024-09-20T19:11:31.500248+0000","last_fullsized":"2024-09-20T19:11:31.500248+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:41.884061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:10:41.884061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:57.894186+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"503'18","reported_seq":1044,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800740+0000","last_change":"2024-09-20T19:10:11.085494+0000","last_active":"2024-09-20T19:11:26.800740+0000","last_peered":"2024-09-20T19:11:26.800740+0000","last_clean":"2024-09-20T19:11:26.800740+0000","last_became_active":"2024-09-20T18:49:48.500662+0000","last_became_peered":"2024-09-20T18:49:48.500662+0000","last_unstale":"2024-09-20T19:11:26.800740+0000","last_undegraded":"2024-09-20T19:11:26.800740+0000","last_fullsized":"2024-09-20T19:11:26.800740+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'18","last_scrub_stamp":"2024-09-20T19:10:11.085439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:10:11.085439+0000","objects_scrubbed":3,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:40.638496+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":24,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1005,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.799554+0000","last_change":"2024-09-20T19:10:22.919792+0000","last_active":"2024-09-20T19:11:26.799554+0000","last_peered":"2024-09-20T19:11:26.799554+0000","last_clean":"2024-09-20T19:11:26.799554+0000","last_became_active":"2024-09-20T18:49:49.956749+0000","last_became_peered":"2024-09-20T18:49:49.956749+0000","last_unstale":"2024-09-20T19:11:26.799554+0000","last_undegraded":"2024-09-20T19:11:26.799554+0000","last_fullsized":"2024-09-20T19:11:26.799554+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:22.919731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:10:22.919731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:30.927422+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1049,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.197453+0000","last_change":"2024-09-20T19:10:51.999987+0000","last_active":"2024-09-20T19:11:26.197453+0000","last_peered":"2024-09-20T19:11:26.197453+0000","last_clean":"2024-09-20T19:11:26.197453+0000","last_became_active":"2024-09-20T19:04:20.914115+0000","last_became_peered":"2024-09-20T19:04:20.914115+0000","last_unstale":"2024-09-20T19:11:26.197453+0000","last_undegraded":"2024-09-20T19:11:26.197453+0000","last_fullsized":"2024-09-20T19:11:26.197453+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:51.999928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:10:51.999928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:09.214826+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"503'21","reported_seq":977,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.801408+0000","last_change":"2024-09-20T19:11:23.540164+0000","last_active":"2024-09-20T19:11:26.801408+0000","last_peered":"2024-09-20T19:11:26.801408+0000","last_clean":"2024-09-20T19:11:26.801408+0000","last_became_active":"2024-09-20T19:10:33.071006+0000","last_became_peered":"2024-09-20T19:10:33.071006+0000","last_unstale":"2024-09-20T19:11:26.801408+0000","last_undegraded":"2024-09-20T19:11:26.801408+0000","last_fullsized":"2024-09-20T19:11:26.801408+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'21","last_scrub_stamp":"2024-09-20T19:11:23.540096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:11:23.540096+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:39.554901+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1046,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.197284+0000","last_change":"2024-09-20T19:10:21.950929+0000","last_active":"2024-09-20T19:11:26.197284+0000","last_peered":"2024-09-20T19:11:26.197284+0000","last_clean":"2024-09-20T19:11:26.197284+0000","last_became_active":"2024-09-20T19:04:20.893250+0000","last_became_peered":"2024-09-20T19:04:20.893250+0000","last_unstale":"2024-09-20T19:11:26.197284+0000","last_undegraded":"2024-09-20T19:11:26.197284+0000","last_fullsized":"2024-09-20T19:11:26.197284+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:21.950869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:10:21.950869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:51.444258+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1013,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800176+0000","last_change":"2024-09-20T19:11:25.266739+0000","last_active":"2024-09-20T19:11:26.800176+0000","last_peered":"2024-09-20T19:11:26.800176+0000","last_clean":"2024-09-20T19:11:26.800176+0000","last_became_active":"2024-09-20T18:49:49.961947+0000","last_became_peered":"2024-09-20T18:49:49.961947+0000","last_unstale":"2024-09-20T19:11:26.800176+0000","last_undegraded":"2024-09-20T19:11:26.800176+0000","last_fullsized":"2024-09-20T19:11:26.800176+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:25.266691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:11:25.266691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:26.413210+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"66'1","reported_seq":1057,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.350244+0000","last_change":"2024-09-20T19:11:25.472705+0000","last_active":"2024-09-20T19:11:31.350244+0000","last_peered":"2024-09-20T19:11:31.350244+0000","last_clean":"2024-09-20T19:11:31.350244+0000","last_became_active":"2024-09-20T19:04:20.892236+0000","last_became_peered":"2024-09-20T19:04:20.892236+0000","last_unstale":"2024-09-20T19:11:31.350244+0000","last_undegraded":"2024-09-20T19:11:31.350244+0000","last_fullsized":"2024-09-20T19:11:31.350244+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T19:11:25.472652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:11:25.472652+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:47.401565+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"503'21","reported_seq":1036,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800573+0000","last_change":"2024-09-20T19:10:25.840456+0000","last_active":"2024-09-20T19:11:26.800573+0000","last_peered":"2024-09-20T19:11:26.800573+0000","last_clean":"2024-09-20T19:11:26.800573+0000","last_became_active":"2024-09-20T18:49:48.501011+0000","last_became_peered":"2024-09-20T18:49:48.501011+0000","last_unstale":"2024-09-20T19:11:26.800573+0000","last_undegraded":"2024-09-20T19:11:26.800573+0000","last_fullsized":"2024-09-20T19:11:26.800573+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'21","last_scrub_stamp":"2024-09-20T19:10:25.840352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:10:25.840352+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:37.655279+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":951,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.799608+0000","last_change":"2024-09-20T19:10:35.394841+0000","last_active":"2024-09-20T19:11:26.799608+0000","last_peered":"2024-09-20T19:11:26.799608+0000","last_clean":"2024-09-20T19:11:26.799608+0000","last_became_active":"2024-09-20T19:10:33.067597+0000","last_became_peered":"2024-09-20T19:10:33.067597+0000","last_unstale":"2024-09-20T19:11:26.799608+0000","last_undegraded":"2024-09-20T19:11:26.799608+0000","last_fullsized":"2024-09-20T19:11:26.799608+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:35.394792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:10:35.394792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:56.263119+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":1283,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.799459+0000","last_change":"2024-09-20T19:11:19.298125+0000","last_active":"2024-09-20T19:11:26.799459+0000","last_peered":"2024-09-20T19:11:26.799459+0000","last_clean":"2024-09-20T19:11:26.799459+0000","last_became_active":"2024-09-20T18:49:49.960550+0000","last_became_peered":"2024-09-20T18:49:49.960550+0000","last_unstale":"2024-09-20T19:11:26.799459+0000","last_undegraded":"2024-09-20T19:11:26.799459+0000","last_fullsized":"2024-09-20T19:11:26.799459+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T19:11:19.298074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:11:19.298074+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:34.913656+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":345,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.004044+0000","last_change":"2024-09-20T19:11:15.056390+0000","last_active":"2024-09-20T19:11:26.004044+0000","last_peered":"2024-09-20T19:11:26.004044+0000","last_clean":"2024-09-20T19:11:26.004044+0000","last_became_active":"2024-09-20T19:04:20.884100+0000","last_became_peered":"2024-09-20T19:04:20.884100+0000","last_unstale":"2024-09-20T19:11:26.004044+0000","last_undegraded":"2024-09-20T19:11:26.004044+0000","last_fullsized":"2024-09-20T19:11:26.004044+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:15.056336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:11:15.056336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:27.612026+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"503'30","reported_seq":1058,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800067+0000","last_change":"2024-09-20T19:10:57.119244+0000","last_active":"2024-09-20T19:11:26.800067+0000","last_peered":"2024-09-20T19:11:26.800067+0000","last_clean":"2024-09-20T19:11:26.800067+0000","last_became_active":"2024-09-20T18:49:58.997494+0000","last_became_peered":"2024-09-20T18:49:58.997494+0000","last_unstale":"2024-09-20T19:11:26.800067+0000","last_undegraded":"2024-09-20T19:11:26.800067+0000","last_fullsized":"2024-09-20T19:11:26.800067+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'30","last_scrub_stamp":"2024-09-20T19:10:57.119185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:10:57.119185+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:18.560447+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1024,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.197454+0000","last_change":"2024-09-20T19:09:49.959358+0000","last_active":"2024-09-20T19:11:26.197454+0000","last_peered":"2024-09-20T19:11:26.197454+0000","last_clean":"2024-09-20T19:11:26.197454+0000","last_became_active":"2024-09-20T19:04:20.884582+0000","last_became_peered":"2024-09-20T19:04:20.884582+0000","last_unstale":"2024-09-20T19:11:26.197454+0000","last_undegraded":"2024-09-20T19:11:26.197454+0000","last_fullsized":"2024-09-20T19:11:26.197454+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:09:49.959277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:09:49.959277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"511'10","reported_seq":1288,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800263+0000","last_change":"2024-09-20T19:10:50.984705+0000","last_active":"2024-09-20T19:11:26.800263+0000","last_peered":"2024-09-20T19:11:26.800263+0000","last_clean":"2024-09-20T19:11:26.800263+0000","last_became_active":"2024-09-20T19:10:33.056096+0000","last_became_peered":"2024-09-20T19:10:33.056096+0000","last_unstale":"2024-09-20T19:11:26.800263+0000","last_undegraded":"2024-09-20T19:11:26.800263+0000","last_fullsized":"2024-09-20T19:11:26.800263+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"511'10","last_scrub_stamp":"2024-09-20T19:10:50.984636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:10:50.984636+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:57.528281+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":972,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.801327+0000","last_change":"2024-09-20T19:10:33.071644+0000","last_active":"2024-09-20T19:11:26.801327+0000","last_peered":"2024-09-20T19:11:26.801327+0000","last_clean":"2024-09-20T19:11:26.801327+0000","last_became_active":"2024-09-20T19:10:33.069634+0000","last_became_peered":"2024-09-20T19:10:33.069634+0000","last_unstale":"2024-09-20T19:11:26.801327+0000","last_undegraded":"2024-09-20T19:11:26.801327+0000","last_fullsized":"2024-09-20T19:11:26.801327+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:09:11.549521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:09:11.549521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"503'27","reported_seq":370,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.003401+0000","last_change":"2024-09-20T19:10:22.810122+0000","last_active":"2024-09-20T19:11:26.003401+0000","last_peered":"2024-09-20T19:11:26.003401+0000","last_clean":"2024-09-20T19:11:26.003401+0000","last_became_active":"2024-09-20T19:04:20.897330+0000","last_became_peered":"2024-09-20T19:04:20.897330+0000","last_unstale":"2024-09-20T19:11:26.003401+0000","last_undegraded":"2024-09-20T19:11:26.003401+0000","last_fullsized":"2024-09-20T19:11:26.003401+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'27","last_scrub_stamp":"2024-09-20T19:10:22.810038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:10:22.810038+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:50.390035+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":36,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":346,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.003504+0000","last_change":"2024-09-20T19:11:21.078463+0000","last_active":"2024-09-20T19:11:26.003504+0000","last_peered":"2024-09-20T19:11:26.003504+0000","last_clean":"2024-09-20T19:11:26.003504+0000","last_became_active":"2024-09-20T19:04:20.896007+0000","last_became_peered":"2024-09-20T19:04:20.896007+0000","last_unstale":"2024-09-20T19:11:26.003504+0000","last_undegraded":"2024-09-20T19:11:26.003504+0000","last_fullsized":"2024-09-20T19:11:26.003504+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:21.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:11:21.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:22.732517+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1008,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.351600+0000","last_change":"2024-09-20T19:11:11.460689+0000","last_active":"2024-09-20T19:11:31.351600+0000","last_peered":"2024-09-20T19:11:31.351600+0000","last_clean":"2024-09-20T19:11:31.351600+0000","last_became_active":"2024-09-20T18:49:49.957067+0000","last_became_peered":"2024-09-20T18:49:49.957067+0000","last_unstale":"2024-09-20T19:11:31.351600+0000","last_undegraded":"2024-09-20T19:11:31.351600+0000","last_fullsized":"2024-09-20T19:11:31.351600+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:11.460643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:11:11.460643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:35.547041+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":989,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.801232+0000","last_change":"2024-09-20T19:10:33.957131+0000","last_active":"2024-09-20T19:11:26.801232+0000","last_peered":"2024-09-20T19:11:26.801232+0000","last_clean":"2024-09-20T19:11:26.801232+0000","last_became_active":"2024-09-20T19:10:33.070199+0000","last_became_peered":"2024-09-20T19:10:33.070199+0000","last_unstale":"2024-09-20T19:11:26.801232+0000","last_undegraded":"2024-09-20T19:11:26.801232+0000","last_fullsized":"2024-09-20T19:11:26.801232+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:33.957082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:10:33.957082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:42.894167+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"503'21","reported_seq":1096,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.351752+0000","last_change":"2024-09-20T19:10:42.453779+0000","last_active":"2024-09-20T19:11:31.351752+0000","last_peered":"2024-09-20T19:11:31.351752+0000","last_clean":"2024-09-20T19:11:31.351752+0000","last_became_active":"2024-09-20T18:59:05.085230+0000","last_became_peered":"2024-09-20T18:59:05.085230+0000","last_unstale":"2024-09-20T19:11:31.351752+0000","last_undegraded":"2024-09-20T19:11:31.351752+0000","last_fullsized":"2024-09-20T19:11:31.351752+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'21","last_scrub_stamp":"2024-09-20T19:10:42.453714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:10:42.453714+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:12.188538+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1023,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.197753+0000","last_change":"2024-09-20T19:10:19.980466+0000","last_active":"2024-09-20T19:11:26.197753+0000","last_peered":"2024-09-20T19:11:26.197753+0000","last_clean":"2024-09-20T19:11:26.197753+0000","last_became_active":"2024-09-20T19:04:20.884660+0000","last_became_peered":"2024-09-20T19:04:20.884660+0000","last_unstale":"2024-09-20T19:11:26.197753+0000","last_undegraded":"2024-09-20T19:11:26.197753+0000","last_fullsized":"2024-09-20T19:11:26.197753+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:19.980413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:10:19.980413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:48.776512+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1025,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800043+0000","last_change":"2024-09-20T19:10:38.084909+0000","last_active":"2024-09-20T19:11:26.800043+0000","last_peered":"2024-09-20T19:11:26.800043+0000","last_clean":"2024-09-20T19:11:26.800043+0000","last_became_active":"2024-09-20T19:10:33.065411+0000","last_became_peered":"2024-09-20T19:10:33.065411+0000","last_unstale":"2024-09-20T19:11:26.800043+0000","last_undegraded":"2024-09-20T19:11:26.800043+0000","last_fullsized":"2024-09-20T19:11:26.800043+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:38.084679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:10:38.084679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:43.367664+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":965,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.799240+0000","last_change":"2024-09-20T19:10:33.064070+0000","last_active":"2024-09-20T19:11:26.799240+0000","last_peered":"2024-09-20T19:11:26.799240+0000","last_clean":"2024-09-20T19:11:26.799240+0000","last_became_active":"2024-09-20T19:10:33.062950+0000","last_became_peered":"2024-09-20T19:10:33.062950+0000","last_unstale":"2024-09-20T19:11:26.799240+0000","last_undegraded":"2024-09-20T19:11:26.799240+0000","last_fullsized":"2024-09-20T19:11:26.799240+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:09:06.410215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:09:06.410215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"503'27","reported_seq":1091,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.801132+0000","last_change":"2024-09-20T19:11:18.259644+0000","last_active":"2024-09-20T19:11:26.801132+0000","last_peered":"2024-09-20T19:11:26.801132+0000","last_clean":"2024-09-20T19:11:26.801132+0000","last_became_active":"2024-09-20T18:59:05.075390+0000","last_became_peered":"2024-09-20T18:59:05.075390+0000","last_unstale":"2024-09-20T19:11:26.801132+0000","last_undegraded":"2024-09-20T19:11:26.801132+0000","last_fullsized":"2024-09-20T19:11:26.801132+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'27","last_scrub_stamp":"2024-09-20T19:11:18.259570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:11:18.259570+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:26.478226+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1022,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800499+0000","last_change":"2024-09-20T19:10:24.887203+0000","last_active":"2024-09-20T19:11:26.800499+0000","last_peered":"2024-09-20T19:11:26.800499+0000","last_clean":"2024-09-20T19:11:26.800499+0000","last_became_active":"2024-09-20T18:49:45.942105+0000","last_became_peered":"2024-09-20T18:49:45.942105+0000","last_unstale":"2024-09-20T19:11:26.800499+0000","last_undegraded":"2024-09-20T19:11:26.800499+0000","last_fullsized":"2024-09-20T19:11:26.800499+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:24.887146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:10:24.887146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:35.744922+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1046,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.272188+0000","last_change":"2024-09-20T19:11:31.272188+0000","last_active":"2024-09-20T19:11:31.272188+0000","last_peered":"2024-09-20T19:11:31.272188+0000","last_clean":"2024-09-20T19:11:31.272188+0000","last_became_active":"2024-09-20T19:04:20.911292+0000","last_became_peered":"2024-09-20T19:04:20.911292+0000","last_unstale":"2024-09-20T19:11:31.272188+0000","last_undegraded":"2024-09-20T19:11:31.272188+0000","last_fullsized":"2024-09-20T19:11:31.272188+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:31.272124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:11:31.272124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:47.689226+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1027,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800191+0000","last_change":"2024-09-20T19:10:51.114754+0000","last_active":"2024-09-20T19:11:26.800191+0000","last_peered":"2024-09-20T19:11:26.800191+0000","last_clean":"2024-09-20T19:11:26.800191+0000","last_became_active":"2024-09-20T19:10:33.066160+0000","last_became_peered":"2024-09-20T19:10:33.066160+0000","last_unstale":"2024-09-20T19:11:26.800191+0000","last_undegraded":"2024-09-20T19:11:26.800191+0000","last_fullsized":"2024-09-20T19:11:26.800191+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:51.114701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:10:51.114701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:58.511628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"503'12","reported_seq":1062,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800634+0000","last_change":"2024-09-20T19:10:14.169742+0000","last_active":"2024-09-20T19:11:26.800634+0000","last_peered":"2024-09-20T19:11:26.800634+0000","last_clean":"2024-09-20T19:11:26.800634+0000","last_became_active":"2024-09-20T18:59:05.076365+0000","last_became_peered":"2024-09-20T18:59:05.076365+0000","last_unstale":"2024-09-20T19:11:26.800634+0000","last_undegraded":"2024-09-20T19:11:26.800634+0000","last_fullsized":"2024-09-20T19:11:26.800634+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'12","last_scrub_stamp":"2024-09-20T19:10:14.169687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:10:14.169687+0000","objects_scrubbed":2,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:29.283499+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":16,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":968,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800323+0000","last_change":"2024-09-20T19:11:05.381449+0000","last_active":"2024-09-20T19:11:26.800323+0000","last_peered":"2024-09-20T19:11:26.800323+0000","last_clean":"2024-09-20T19:11:26.800323+0000","last_became_active":"2024-09-20T19:10:33.067749+0000","last_became_peered":"2024-09-20T19:10:33.067749+0000","last_unstale":"2024-09-20T19:11:26.800323+0000","last_undegraded":"2024-09-20T19:11:26.800323+0000","last_fullsized":"2024-09-20T19:11:26.800323+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:05.381390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:11:05.381390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:10.990453+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":620,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.684272+0000","last_change":"2024-09-20T19:11:31.684272+0000","last_active":"2024-09-20T19:11:31.684272+0000","last_peered":"2024-09-20T19:11:31.684272+0000","last_clean":"2024-09-20T19:11:31.684272+0000","last_became_active":"2024-09-20T19:04:20.900214+0000","last_became_peered":"2024-09-20T19:04:20.900214+0000","last_unstale":"2024-09-20T19:11:31.684272+0000","last_undegraded":"2024-09-20T19:11:31.684272+0000","last_fullsized":"2024-09-20T19:11:31.684272+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:31.684211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:11:31.684211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:45.398104+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1020,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.197150+0000","last_change":"2024-09-20T19:10:23.938694+0000","last_active":"2024-09-20T19:11:26.197150+0000","last_peered":"2024-09-20T19:11:26.197150+0000","last_clean":"2024-09-20T19:11:26.197150+0000","last_became_active":"2024-09-20T19:04:20.885211+0000","last_became_peered":"2024-09-20T19:04:20.885211+0000","last_unstale":"2024-09-20T19:11:26.197150+0000","last_undegraded":"2024-09-20T19:11:26.197150+0000","last_fullsized":"2024-09-20T19:11:26.197150+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:23.938636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:10:23.938636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:28.976214+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1012,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.352091+0000","last_change":"2024-09-20T19:11:16.516786+0000","last_active":"2024-09-20T19:11:31.352091+0000","last_peered":"2024-09-20T19:11:31.352091+0000","last_clean":"2024-09-20T19:11:31.352091+0000","last_became_active":"2024-09-20T19:04:20.890101+0000","last_became_peered":"2024-09-20T19:04:20.890101+0000","last_unstale":"2024-09-20T19:11:31.352091+0000","last_undegraded":"2024-09-20T19:11:31.352091+0000","last_fullsized":"2024-09-20T19:11:31.352091+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:16.516732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:11:16.516732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:25.404643+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1016,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800899+0000","last_change":"2024-09-20T19:10:50.100813+0000","last_active":"2024-09-20T19:11:26.800899+0000","last_peered":"2024-09-20T19:11:26.800899+0000","last_clean":"2024-09-20T19:11:26.800899+0000","last_became_active":"2024-09-20T19:10:33.072534+0000","last_became_peered":"2024-09-20T19:10:33.072534+0000","last_unstale":"2024-09-20T19:11:26.800899+0000","last_undegraded":"2024-09-20T19:11:26.800899+0000","last_fullsized":"2024-09-20T19:11:26.800899+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:50.100761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:10:50.100761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:56.699571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"503'27","reported_seq":631,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.502808+0000","last_change":"2024-09-20T19:11:24.552622+0000","last_active":"2024-09-20T19:11:31.502808+0000","last_peered":"2024-09-20T19:11:31.502808+0000","last_clean":"2024-09-20T19:11:31.502808+0000","last_became_active":"2024-09-20T18:59:05.089294+0000","last_became_peered":"2024-09-20T18:59:05.089294+0000","last_unstale":"2024-09-20T19:11:31.502808+0000","last_undegraded":"2024-09-20T19:11:31.502808+0000","last_fullsized":"2024-09-20T19:11:31.502808+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'27","last_scrub_stamp":"2024-09-20T19:11:24.552545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:11:24.552545+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:31.353939+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":36,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":940,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.801172+0000","last_change":"2024-09-20T19:10:45.301218+0000","last_active":"2024-09-20T19:11:26.801172+0000","last_peered":"2024-09-20T19:11:26.801172+0000","last_clean":"2024-09-20T19:11:26.801172+0000","last_became_active":"2024-09-20T19:10:33.067702+0000","last_became_peered":"2024-09-20T19:10:33.067702+0000","last_unstale":"2024-09-20T19:11:26.801172+0000","last_undegraded":"2024-09-20T19:11:26.801172+0000","last_fullsized":"2024-09-20T19:11:26.801172+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:45.301165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:10:45.301165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:49.666319+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1185,"internal_metadata":0},"log_size":93,"ondisk_log_size":93,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1537,"num_read_kb":1294,"num_write":989,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":283,"num_bytes_recovered":4670,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":972,"ondisk_log_size":972,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":3013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1295,"num_read_kb":1290,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":869,"ondisk_log_size":869,"up":33,"acting":33,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266954,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":768292,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":766974,"kb_avail":92976860,"statfs":{"total":95995035648,"available":95208304640,"internally_reserved":0,"allocated":1331200,"data_stored":853209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":785381834},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528593,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":649952,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":648958,"kb_avail":93095200,"statfs":{"total":95995035648,"available":95329484800,"internally_reserved":0,"allocated":995328,"data_stored":531594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":664533446},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757526,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":726892,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":725310,"kb_avail":93018260,"statfs":{"total":95995035648,"available":95250698240,"internally_reserved":0,"allocated":1597440,"data_stored":1115952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":742717893},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":304,"seq":1305670058108,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":464500,"kb_used_data":996,"kb_used_omap":5,"kb_used_meta":463482,"kb_avail":93280652,"statfs":{"total":95995035648,"available":95519387648,"internally_reserved":0,"allocated":1019904,"data_stored":526478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5633,"internal_metadata":474606079},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182690,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":894888,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":893310,"kb_avail":92850264,"statfs":{"total":95995035648,"available":95078670336,"internally_reserved":0,"allocated":1593344,"data_stored":1115332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":914749895},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411623,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":790444,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":789182,"kb_avail":92954708,"statfs":{"total":95995035648,"available":95185620992,"internally_reserved":0,"allocated":1269760,"data_stored":795220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":808122821},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575149,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":862424,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":861374,"kb_avail":92882728,"statfs":{"total":95995035648,"available":95111913472,"internally_reserved":0,"allocated":1052672,"data_stored":533716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":882047429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":510,"seq":2190433320974,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":84136,"kb_used_data":980,"kb_used_omap":4,"kb_used_meta":83131,"kb_avail":93661016,"statfs":{"total":95995035648,"available":95908880384,"internally_reserved":0,"allocated":1003520,"data_stored":525225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4445,"internal_metadata":85126819},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1185,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T19:11:34.336 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T19:11:34.586 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:11:34.707 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:34 smithi083 ceph-mon[29421]: 5.16 scrub starts 2024-09-20T19:11:34.707 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:34 smithi083 ceph-mon[29421]: 5.16 scrub ok 2024-09-20T19:11:34.707 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:34 smithi083 ceph-mon[29421]: pgmap v1102: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T19:11:34.707 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:34 smithi083 ceph-mon[29421]: from='client.26720 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:11:34.708 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:34 smithi083 ceph-mon[29421]: 5.1c scrub starts 2024-09-20T19:11:34.708 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:34 smithi083 ceph-mon[29421]: 5.1c scrub ok 2024-09-20T19:11:34.708 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:34 smithi083 ceph-mon[36260]: 5.16 scrub starts 2024-09-20T19:11:34.708 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:34 smithi083 ceph-mon[36260]: 5.16 scrub ok 2024-09-20T19:11:34.708 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:34 smithi083 ceph-mon[36260]: pgmap v1102: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T19:11:34.708 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:34 smithi083 ceph-mon[36260]: from='client.26720 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:11:34.708 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:34 smithi083 ceph-mon[36260]: 5.1c scrub starts 2024-09-20T19:11:34.708 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:34 smithi083 ceph-mon[36260]: 5.1c scrub ok 2024-09-20T19:11:34.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:34 smithi142 ceph-mon[32508]: 5.16 scrub starts 2024-09-20T19:11:34.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:34 smithi142 ceph-mon[32508]: 5.16 scrub ok 2024-09-20T19:11:34.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:34 smithi142 ceph-mon[32508]: pgmap v1102: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T19:11:34.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:34 smithi142 ceph-mon[32508]: from='client.26720 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:11:34.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:34 smithi142 ceph-mon[32508]: 5.1c scrub starts 2024-09-20T19:11:34.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:34 smithi142 ceph-mon[32508]: 5.1c scrub ok 2024-09-20T19:11:35.217 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-20T19:11:35.219 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-20T19:11:35.712 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:35 smithi083 ceph-mon[29421]: 4.10 scrub starts 2024-09-20T19:11:35.712 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:35 smithi083 ceph-mon[29421]: 4.10 scrub ok 2024-09-20T19:11:35.712 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:35 smithi083 ceph-mon[36260]: 4.10 scrub starts 2024-09-20T19:11:35.712 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:35 smithi083 ceph-mon[36260]: 4.10 scrub ok 2024-09-20T19:11:35.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:35 smithi142 ceph-mon[32508]: 4.10 scrub starts 2024-09-20T19:11:35.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:35 smithi142 ceph-mon[32508]: 4.10 scrub ok 2024-09-20T19:11:35.888 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":1102,"stamp":"2024-09-20T19:11:33.268741+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":2849,"num_read_kb":2601,"num_write":1008,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":7747,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1950,"ondisk_log_size":1950,"up":426,"acting":426,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":426,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":5241528,"kb_used_data":9632,"kb_used_omap":19,"kb_used_meta":5231724,"kb_avail":744719688,"statfs":{"total":767960285184,"available":762592960512,"internally_reserved":0,"allocated":9863168,"data_stored":5996726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19640,"internal_metadata":5357286216},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001937"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":962,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800244+0000","last_change":"2024-09-20T19:11:24.538639+0000","last_active":"2024-09-20T19:11:26.800244+0000","last_peered":"2024-09-20T19:11:26.800244+0000","last_clean":"2024-09-20T19:11:26.800244+0000","last_became_active":"2024-09-20T19:10:33.066631+0000","last_became_peered":"2024-09-20T19:10:33.066631+0000","last_unstale":"2024-09-20T19:11:26.800244+0000","last_undegraded":"2024-09-20T19:11:26.800244+0000","last_fullsized":"2024-09-20T19:11:26.800244+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:24.538587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:11:24.538587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:39.722056+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"503'21","reported_seq":364,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.002798+0000","last_change":"2024-09-20T19:10:24.850308+0000","last_active":"2024-09-20T19:11:26.002798+0000","last_peered":"2024-09-20T19:11:26.002798+0000","last_clean":"2024-09-20T19:11:26.002798+0000","last_became_active":"2024-09-20T19:04:20.925853+0000","last_became_peered":"2024-09-20T19:04:20.925853+0000","last_unstale":"2024-09-20T19:11:26.002798+0000","last_undegraded":"2024-09-20T19:11:26.002798+0000","last_fullsized":"2024-09-20T19:11:26.002798+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'21","last_scrub_stamp":"2024-09-20T19:10:24.850234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:10:24.850234+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:33.407527+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":845,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.351268+0000","last_change":"2024-09-20T19:11:02.485373+0000","last_active":"2024-09-20T19:11:31.351268+0000","last_peered":"2024-09-20T19:11:31.351268+0000","last_clean":"2024-09-20T19:11:31.351268+0000","last_became_active":"2024-09-20T19:10:41.802004+0000","last_became_peered":"2024-09-20T19:10:41.802004+0000","last_unstale":"2024-09-20T19:11:31.351268+0000","last_undegraded":"2024-09-20T19:11:31.351268+0000","last_fullsized":"2024-09-20T19:11:31.351268+0000","mapping_epoch":514,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":515,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:02.485325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:11:02.485325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:03.674166+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":345,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.003237+0000","last_change":"2024-09-20T19:11:18.075615+0000","last_active":"2024-09-20T19:11:26.003237+0000","last_peered":"2024-09-20T19:11:26.003237+0000","last_clean":"2024-09-20T19:11:26.003237+0000","last_became_active":"2024-09-20T19:04:20.880905+0000","last_became_peered":"2024-09-20T19:04:20.880905+0000","last_unstale":"2024-09-20T19:11:26.003237+0000","last_undegraded":"2024-09-20T19:11:26.003237+0000","last_fullsized":"2024-09-20T19:11:26.003237+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:18.075542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:11:18.075542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:43.868766+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1015,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.799951+0000","last_change":"2024-09-20T19:11:02.212586+0000","last_active":"2024-09-20T19:11:26.799951+0000","last_peered":"2024-09-20T19:11:26.799951+0000","last_clean":"2024-09-20T19:11:26.799951+0000","last_became_active":"2024-09-20T18:59:05.081081+0000","last_became_peered":"2024-09-20T18:59:05.081081+0000","last_unstale":"2024-09-20T19:11:26.799951+0000","last_undegraded":"2024-09-20T19:11:26.799951+0000","last_fullsized":"2024-09-20T19:11:26.799951+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:02.212529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:11:02.212529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:14.661634+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1029,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800663+0000","last_change":"2024-09-20T19:11:15.173377+0000","last_active":"2024-09-20T19:11:26.800663+0000","last_peered":"2024-09-20T19:11:26.800663+0000","last_clean":"2024-09-20T19:11:26.800663+0000","last_became_active":"2024-09-20T18:59:05.072675+0000","last_became_peered":"2024-09-20T18:59:05.072675+0000","last_unstale":"2024-09-20T19:11:26.800663+0000","last_undegraded":"2024-09-20T19:11:26.800663+0000","last_fullsized":"2024-09-20T19:11:26.800663+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:15.173333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:11:15.173333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:28.036486+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"503'39","reported_seq":1104,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800971+0000","last_change":"2024-09-20T19:10:55.119441+0000","last_active":"2024-09-20T19:11:26.800971+0000","last_peered":"2024-09-20T19:11:26.800971+0000","last_clean":"2024-09-20T19:11:26.800971+0000","last_became_active":"2024-09-20T19:10:33.069997+0000","last_became_peered":"2024-09-20T19:10:33.069997+0000","last_unstale":"2024-09-20T19:11:26.800971+0000","last_undegraded":"2024-09-20T19:11:26.800971+0000","last_fullsized":"2024-09-20T19:11:26.800971+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'39","last_scrub_stamp":"2024-09-20T19:10:55.119374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:10:55.119374+0000","objects_scrubbed":7,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:10.666604+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":345,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.002815+0000","last_change":"2024-09-20T19:10:12.758720+0000","last_active":"2024-09-20T19:11:26.002815+0000","last_peered":"2024-09-20T19:11:26.002815+0000","last_clean":"2024-09-20T19:11:26.002815+0000","last_became_active":"2024-09-20T19:04:20.925394+0000","last_became_peered":"2024-09-20T19:04:20.925394+0000","last_unstale":"2024-09-20T19:11:26.002815+0000","last_undegraded":"2024-09-20T19:11:26.002815+0000","last_fullsized":"2024-09-20T19:11:26.002815+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:12.758650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:10:12.758650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"59'1","reported_seq":1090,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800550+0000","last_change":"2024-09-20T19:11:11.016408+0000","last_active":"2024-09-20T19:11:26.800550+0000","last_peered":"2024-09-20T19:11:26.800550+0000","last_clean":"2024-09-20T19:11:26.800550+0000","last_became_active":"2024-09-20T19:04:20.914145+0000","last_became_peered":"2024-09-20T19:04:20.914145+0000","last_unstale":"2024-09-20T19:11:26.800550+0000","last_undegraded":"2024-09-20T19:11:26.800550+0000","last_fullsized":"2024-09-20T19:11:26.800550+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:11:11.016347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:11:11.016347+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:24.620595+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"503'21","reported_seq":376,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.003317+0000","last_change":"2024-09-20T19:10:35.823487+0000","last_active":"2024-09-20T19:11:26.003317+0000","last_peered":"2024-09-20T19:11:26.003317+0000","last_clean":"2024-09-20T19:11:26.003317+0000","last_became_active":"2024-09-20T19:10:33.058094+0000","last_became_peered":"2024-09-20T19:10:33.058094+0000","last_unstale":"2024-09-20T19:11:26.003317+0000","last_undegraded":"2024-09-20T19:11:26.003317+0000","last_fullsized":"2024-09-20T19:11:26.003317+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'21","last_scrub_stamp":"2024-09-20T19:10:35.823388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:10:35.823388+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:42.981473+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1024,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800054+0000","last_change":"2024-09-20T19:10:52.005533+0000","last_active":"2024-09-20T19:11:26.800054+0000","last_peered":"2024-09-20T19:11:26.800054+0000","last_clean":"2024-09-20T19:11:26.800054+0000","last_became_active":"2024-09-20T19:10:33.054613+0000","last_became_peered":"2024-09-20T19:10:33.054613+0000","last_unstale":"2024-09-20T19:11:26.800054+0000","last_undegraded":"2024-09-20T19:11:26.800054+0000","last_fullsized":"2024-09-20T19:11:26.800054+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:52.005475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:10:52.005475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:04.756633+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":948,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.799401+0000","last_change":"2024-09-20T19:11:02.450255+0000","last_active":"2024-09-20T19:11:26.799401+0000","last_peered":"2024-09-20T19:11:26.799401+0000","last_clean":"2024-09-20T19:11:26.799401+0000","last_became_active":"2024-09-20T19:10:33.061000+0000","last_became_peered":"2024-09-20T19:10:33.061000+0000","last_unstale":"2024-09-20T19:11:26.799401+0000","last_undegraded":"2024-09-20T19:11:26.799401+0000","last_fullsized":"2024-09-20T19:11:26.799401+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:02.450209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:11:02.450209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:04.893685+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1038,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800315+0000","last_change":"2024-09-20T19:11:11.993880+0000","last_active":"2024-09-20T19:11:26.800315+0000","last_peered":"2024-09-20T19:11:26.800315+0000","last_clean":"2024-09-20T19:11:26.800315+0000","last_became_active":"2024-09-20T19:04:20.878905+0000","last_became_peered":"2024-09-20T19:04:20.878905+0000","last_unstale":"2024-09-20T19:11:26.800315+0000","last_undegraded":"2024-09-20T19:11:26.800315+0000","last_fullsized":"2024-09-20T19:11:26.800315+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:11.993819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:11:11.993819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:34.760402+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"503'9","reported_seq":362,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:29.028487+0000","last_change":"2024-09-20T19:11:29.028487+0000","last_active":"2024-09-20T19:11:29.028487+0000","last_peered":"2024-09-20T19:11:29.028487+0000","last_clean":"2024-09-20T19:11:29.028487+0000","last_became_active":"2024-09-20T19:04:20.888572+0000","last_became_peered":"2024-09-20T19:04:20.888572+0000","last_unstale":"2024-09-20T19:11:29.028487+0000","last_undegraded":"2024-09-20T19:11:29.028487+0000","last_fullsized":"2024-09-20T19:11:29.028487+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'9","last_scrub_stamp":"2024-09-20T19:11:29.028427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:11:29.028427+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:54.155717+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":12,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1031,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.197147+0000","last_change":"2024-09-20T19:10:51.022419+0000","last_active":"2024-09-20T19:11:26.197147+0000","last_peered":"2024-09-20T19:11:26.197147+0000","last_clean":"2024-09-20T19:11:26.197147+0000","last_became_active":"2024-09-20T19:04:20.885417+0000","last_became_peered":"2024-09-20T19:04:20.885417+0000","last_unstale":"2024-09-20T19:11:26.197147+0000","last_undegraded":"2024-09-20T19:11:26.197147+0000","last_fullsized":"2024-09-20T19:11:26.197147+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:51.022353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:10:51.022353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:03.028955+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1009,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.799873+0000","last_change":"2024-09-20T19:11:06.015570+0000","last_active":"2024-09-20T19:11:26.799873+0000","last_peered":"2024-09-20T19:11:26.799873+0000","last_clean":"2024-09-20T19:11:26.799873+0000","last_became_active":"2024-09-20T18:49:51.970663+0000","last_became_peered":"2024-09-20T18:49:51.970663+0000","last_unstale":"2024-09-20T19:11:26.799873+0000","last_undegraded":"2024-09-20T19:11:26.799873+0000","last_fullsized":"2024-09-20T19:11:26.799873+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:06.015482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:11:06.015482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:21.398187+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":346,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.003914+0000","last_change":"2024-09-20T19:11:09.987842+0000","last_active":"2024-09-20T19:11:26.003914+0000","last_peered":"2024-09-20T19:11:26.003914+0000","last_clean":"2024-09-20T19:11:26.003914+0000","last_became_active":"2024-09-20T19:04:20.906109+0000","last_became_peered":"2024-09-20T19:04:20.906109+0000","last_unstale":"2024-09-20T19:11:26.003914+0000","last_undegraded":"2024-09-20T19:11:26.003914+0000","last_fullsized":"2024-09-20T19:11:26.003914+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:09.987779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:11:09.987779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:31.306115+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":937,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.801085+0000","last_change":"2024-09-20T19:10:40.313128+0000","last_active":"2024-09-20T19:11:26.801085+0000","last_peered":"2024-09-20T19:11:26.801085+0000","last_clean":"2024-09-20T19:11:26.801085+0000","last_became_active":"2024-09-20T19:10:33.058426+0000","last_became_peered":"2024-09-20T19:10:33.058426+0000","last_unstale":"2024-09-20T19:11:26.801085+0000","last_undegraded":"2024-09-20T19:11:26.801085+0000","last_fullsized":"2024-09-20T19:11:26.801085+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:40.313080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:10:40.313080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:40.699676+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"503'39","reported_seq":1091,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.350618+0000","last_change":"2024-09-20T19:10:42.046992+0000","last_active":"2024-09-20T19:11:31.350618+0000","last_peered":"2024-09-20T19:11:31.350618+0000","last_clean":"2024-09-20T19:11:31.350618+0000","last_became_active":"2024-09-20T18:49:47.950939+0000","last_became_peered":"2024-09-20T18:49:47.950939+0000","last_unstale":"2024-09-20T19:11:31.350618+0000","last_undegraded":"2024-09-20T19:11:31.350618+0000","last_fullsized":"2024-09-20T19:11:31.350618+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'39","last_scrub_stamp":"2024-09-20T19:10:42.046937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:10:42.046937+0000","objects_scrubbed":7,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:03.180119+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1035,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.352252+0000","last_change":"2024-09-20T19:10:41.407482+0000","last_active":"2024-09-20T19:11:31.352252+0000","last_peered":"2024-09-20T19:11:31.352252+0000","last_clean":"2024-09-20T19:11:31.352252+0000","last_became_active":"2024-09-20T19:10:33.053415+0000","last_became_peered":"2024-09-20T19:10:33.053415+0000","last_unstale":"2024-09-20T19:11:31.352252+0000","last_undegraded":"2024-09-20T19:11:31.352252+0000","last_fullsized":"2024-09-20T19:11:31.352252+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:41.407434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:10:41.407434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:46.861200+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1025,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800591+0000","last_change":"2024-09-20T19:11:20.252773+0000","last_active":"2024-09-20T19:11:26.800591+0000","last_peered":"2024-09-20T19:11:26.800591+0000","last_clean":"2024-09-20T19:11:26.800591+0000","last_became_active":"2024-09-20T18:59:05.084496+0000","last_became_peered":"2024-09-20T18:59:05.084496+0000","last_unstale":"2024-09-20T19:11:26.800591+0000","last_undegraded":"2024-09-20T19:11:26.800591+0000","last_fullsized":"2024-09-20T19:11:26.800591+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:20.252722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:11:20.252722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:20.802803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"59'1","reported_seq":341,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.003080+0000","last_change":"2024-09-20T19:10:02.651674+0000","last_active":"2024-09-20T19:11:26.003080+0000","last_peered":"2024-09-20T19:11:26.003080+0000","last_clean":"2024-09-20T19:11:26.003080+0000","last_became_active":"2024-09-20T19:04:20.887062+0000","last_became_peered":"2024-09-20T19:04:20.887062+0000","last_unstale":"2024-09-20T19:11:26.003080+0000","last_undegraded":"2024-09-20T19:11:26.003080+0000","last_fullsized":"2024-09-20T19:11:26.003080+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:10:02.651608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:10:02.651608+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":993,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"503'24","reported_seq":1076,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800243+0000","last_change":"2024-09-20T19:10:53.092558+0000","last_active":"2024-09-20T19:11:26.800243+0000","last_peered":"2024-09-20T19:11:26.800243+0000","last_clean":"2024-09-20T19:11:26.800243+0000","last_became_active":"2024-09-20T19:04:20.896343+0000","last_became_peered":"2024-09-20T19:04:20.896343+0000","last_unstale":"2024-09-20T19:11:26.800243+0000","last_undegraded":"2024-09-20T19:11:26.800243+0000","last_fullsized":"2024-09-20T19:11:26.800243+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'24","last_scrub_stamp":"2024-09-20T19:10:53.092470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:10:53.092470+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:15.748746+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":345,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.003614+0000","last_change":"2024-09-20T19:09:53.818967+0000","last_active":"2024-09-20T19:11:26.003614+0000","last_peered":"2024-09-20T19:11:26.003614+0000","last_clean":"2024-09-20T19:11:26.003614+0000","last_became_active":"2024-09-20T19:04:20.889430+0000","last_became_peered":"2024-09-20T19:04:20.889430+0000","last_unstale":"2024-09-20T19:11:26.003614+0000","last_undegraded":"2024-09-20T19:11:26.003614+0000","last_fullsized":"2024-09-20T19:11:26.003614+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:09:53.818919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:09:53.818919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":995,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.197585+0000","last_change":"2024-09-20T19:10:05.135865+0000","last_active":"2024-09-20T19:11:26.197585+0000","last_peered":"2024-09-20T19:11:26.197585+0000","last_clean":"2024-09-20T19:11:26.197585+0000","last_became_active":"2024-09-20T18:49:51.970058+0000","last_became_peered":"2024-09-20T18:49:51.970058+0000","last_unstale":"2024-09-20T19:11:26.197585+0000","last_undegraded":"2024-09-20T19:11:26.197585+0000","last_fullsized":"2024-09-20T19:11:26.197585+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T19:10:05.135811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:10:05.135811+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:27.893747+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":1049,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.351355+0000","last_change":"2024-09-20T19:10:46.507557+0000","last_active":"2024-09-20T19:11:31.351355+0000","last_peered":"2024-09-20T19:11:31.351355+0000","last_clean":"2024-09-20T19:11:31.351355+0000","last_became_active":"2024-09-20T19:04:20.881466+0000","last_became_peered":"2024-09-20T19:04:20.881466+0000","last_unstale":"2024-09-20T19:11:31.351355+0000","last_undegraded":"2024-09-20T19:11:31.351355+0000","last_fullsized":"2024-09-20T19:11:31.351355+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:10:46.507497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:10:46.507497+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:55.995372+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"503'30","reported_seq":1035,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.801297+0000","last_change":"2024-09-20T19:11:01.424998+0000","last_active":"2024-09-20T19:11:26.801297+0000","last_peered":"2024-09-20T19:11:26.801297+0000","last_clean":"2024-09-20T19:11:26.801297+0000","last_became_active":"2024-09-20T19:10:33.069139+0000","last_became_peered":"2024-09-20T19:10:33.069139+0000","last_unstale":"2024-09-20T19:11:26.801297+0000","last_undegraded":"2024-09-20T19:11:26.801297+0000","last_fullsized":"2024-09-20T19:11:26.801297+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'30","last_scrub_stamp":"2024-09-20T19:11:01.424946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:11:01.424946+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:23.139948+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":38,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"375'14","reported_seq":792,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.502745+0000","last_change":"2024-09-20T19:11:13.583596+0000","last_active":"2024-09-20T19:11:31.502745+0000","last_peered":"2024-09-20T19:11:31.502745+0000","last_clean":"2024-09-20T19:11:31.502745+0000","last_became_active":"2024-09-20T19:04:20.900782+0000","last_became_peered":"2024-09-20T19:04:20.900782+0000","last_unstale":"2024-09-20T19:11:31.502745+0000","last_undegraded":"2024-09-20T19:11:31.502745+0000","last_fullsized":"2024-09-20T19:11:31.502745+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"375'14","last_scrub_stamp":"2024-09-20T19:11:13.583506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:11:13.583506+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:35.102554+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.a","version":"74'79","reported_seq":178,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.799843+0000","last_change":"2024-09-20T19:10:53.978356+0000","last_active":"2024-09-20T19:11:26.799843+0000","last_peered":"2024-09-20T19:11:26.799843+0000","last_clean":"2024-09-20T19:11:26.799843+0000","last_became_active":"2024-09-20T19:07:42.980316+0000","last_became_peered":"2024-09-20T19:07:42.980316+0000","last_unstale":"2024-09-20T19:11:26.799843+0000","last_undegraded":"2024-09-20T19:11:26.799843+0000","last_fullsized":"2024-09-20T19:11:26.799843+0000","mapping_epoch":458,"log_start":"0'0","ondisk_log_start":"0'0","created":455,"last_epoch_clean":459,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:10:53.978296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:10:53.978296+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:15.764358+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"503'39","reported_seq":1116,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.350883+0000","last_change":"2024-09-20T19:11:10.749016+0000","last_active":"2024-09-20T19:11:31.350883+0000","last_peered":"2024-09-20T19:11:31.350883+0000","last_clean":"2024-09-20T19:11:31.350883+0000","last_became_active":"2024-09-20T19:04:20.897049+0000","last_became_peered":"2024-09-20T19:04:20.897049+0000","last_unstale":"2024-09-20T19:11:31.350883+0000","last_undegraded":"2024-09-20T19:11:31.350883+0000","last_fullsized":"2024-09-20T19:11:31.350883+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'39","last_scrub_stamp":"2024-09-20T19:11:10.748955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:11:10.748955+0000","objects_scrubbed":7,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:37.687162+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1015,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.801018+0000","last_change":"2024-09-20T19:10:22.117116+0000","last_active":"2024-09-20T19:11:26.801018+0000","last_peered":"2024-09-20T19:11:26.801018+0000","last_clean":"2024-09-20T19:11:26.801018+0000","last_became_active":"2024-09-20T18:49:45.936104+0000","last_became_peered":"2024-09-20T18:49:45.936104+0000","last_unstale":"2024-09-20T19:11:26.801018+0000","last_undegraded":"2024-09-20T19:11:26.801018+0000","last_fullsized":"2024-09-20T19:11:26.801018+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:22.117067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:10:22.117067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:37.715930+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"511'14","reported_seq":450,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.003999+0000","last_change":"2024-09-20T19:10:47.817685+0000","last_active":"2024-09-20T19:11:26.003999+0000","last_peered":"2024-09-20T19:11:26.003999+0000","last_clean":"2024-09-20T19:11:26.003999+0000","last_became_active":"2024-09-20T19:10:33.065459+0000","last_became_peered":"2024-09-20T19:10:33.065459+0000","last_unstale":"2024-09-20T19:11:26.003999+0000","last_undegraded":"2024-09-20T19:11:26.003999+0000","last_fullsized":"2024-09-20T19:11:26.003999+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"511'14","last_scrub_stamp":"2024-09-20T19:10:47.817618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:10:47.817618+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:15.033915+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1005,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800217+0000","last_change":"2024-09-20T19:10:33.864924+0000","last_active":"2024-09-20T19:11:26.800217+0000","last_peered":"2024-09-20T19:11:26.800217+0000","last_clean":"2024-09-20T19:11:26.800217+0000","last_became_active":"2024-09-20T19:10:33.055508+0000","last_became_peered":"2024-09-20T19:10:33.055508+0000","last_unstale":"2024-09-20T19:11:26.800217+0000","last_undegraded":"2024-09-20T19:11:26.800217+0000","last_fullsized":"2024-09-20T19:11:26.800217+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:33.864864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:10:33.864864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:59.266093+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"74'79","reported_seq":153,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.351996+0000","last_change":"2024-09-20T19:10:39.432942+0000","last_active":"2024-09-20T19:11:31.351996+0000","last_peered":"2024-09-20T19:11:31.351996+0000","last_clean":"2024-09-20T19:11:31.351996+0000","last_became_active":"2024-09-20T19:10:33.062664+0000","last_became_peered":"2024-09-20T19:10:33.062664+0000","last_unstale":"2024-09-20T19:11:31.351996+0000","last_undegraded":"2024-09-20T19:11:31.351996+0000","last_fullsized":"2024-09-20T19:11:31.351996+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":455,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:10:39.432884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:10:39.432884+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:41.077221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"503'42","reported_seq":1121,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800737+0000","last_change":"2024-09-20T19:10:48.089587+0000","last_active":"2024-09-20T19:11:26.800737+0000","last_peered":"2024-09-20T19:11:26.800737+0000","last_clean":"2024-09-20T19:11:26.800737+0000","last_became_active":"2024-09-20T19:10:33.072974+0000","last_became_peered":"2024-09-20T19:10:33.072974+0000","last_unstale":"2024-09-20T19:11:26.800737+0000","last_undegraded":"2024-09-20T19:11:26.800737+0000","last_fullsized":"2024-09-20T19:11:26.800737+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'42","last_scrub_stamp":"2024-09-20T19:10:48.089522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:10:48.089522+0000","objects_scrubbed":8,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:54.892638+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":70,"num_read_kb":58,"num_write":44,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1018,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800855+0000","last_change":"2024-09-20T19:10:37.066650+0000","last_active":"2024-09-20T19:11:26.800855+0000","last_peered":"2024-09-20T19:11:26.800855+0000","last_clean":"2024-09-20T19:11:26.800855+0000","last_became_active":"2024-09-20T19:10:33.072653+0000","last_became_peered":"2024-09-20T19:10:33.072653+0000","last_unstale":"2024-09-20T19:11:26.800855+0000","last_undegraded":"2024-09-20T19:11:26.800855+0000","last_fullsized":"2024-09-20T19:11:26.800855+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:37.066599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:10:37.066599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:59.273296+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":346,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.003023+0000","last_change":"2024-09-20T19:11:13.047031+0000","last_active":"2024-09-20T19:11:26.003023+0000","last_peered":"2024-09-20T19:11:26.003023+0000","last_clean":"2024-09-20T19:11:26.003023+0000","last_became_active":"2024-09-20T19:04:20.888422+0000","last_became_peered":"2024-09-20T19:04:20.888422+0000","last_unstale":"2024-09-20T19:11:26.003023+0000","last_undegraded":"2024-09-20T19:11:26.003023+0000","last_fullsized":"2024-09-20T19:11:26.003023+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:13.046968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:11:13.046968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:35.748751+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":354,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:27.044192+0000","last_change":"2024-09-20T19:11:27.044192+0000","last_active":"2024-09-20T19:11:27.044192+0000","last_peered":"2024-09-20T19:11:27.044192+0000","last_clean":"2024-09-20T19:11:27.044192+0000","last_became_active":"2024-09-20T19:04:20.889418+0000","last_became_peered":"2024-09-20T19:04:20.889418+0000","last_unstale":"2024-09-20T19:11:27.044192+0000","last_undegraded":"2024-09-20T19:11:27.044192+0000","last_fullsized":"2024-09-20T19:11:27.044192+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:27.044134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:11:27.044134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:34.841052+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"74'79","reported_seq":183,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.799881+0000","last_change":"2024-09-20T19:10:52.068206+0000","last_active":"2024-09-20T19:11:26.799881+0000","last_peered":"2024-09-20T19:11:26.799881+0000","last_clean":"2024-09-20T19:11:26.799881+0000","last_became_active":"2024-09-20T19:07:42.987705+0000","last_became_peered":"2024-09-20T19:07:42.987705+0000","last_unstale":"2024-09-20T19:11:26.799881+0000","last_undegraded":"2024-09-20T19:11:26.799881+0000","last_fullsized":"2024-09-20T19:11:26.799881+0000","mapping_epoch":458,"log_start":"0'0","ondisk_log_start":"0'0","created":455,"last_epoch_clean":461,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:10:52.068154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:10:52.068154+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:07.002653+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"503'42","reported_seq":375,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.002928+0000","last_change":"2024-09-20T19:10:07.661213+0000","last_active":"2024-09-20T19:11:26.002928+0000","last_peered":"2024-09-20T19:11:26.002928+0000","last_clean":"2024-09-20T19:11:26.002928+0000","last_became_active":"2024-09-20T19:04:20.897444+0000","last_became_peered":"2024-09-20T19:04:20.897444+0000","last_unstale":"2024-09-20T19:11:26.002928+0000","last_undegraded":"2024-09-20T19:11:26.002928+0000","last_fullsized":"2024-09-20T19:11:26.002928+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'42","last_scrub_stamp":"2024-09-20T19:10:07.661154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:10:07.661154+0000","objects_scrubbed":7,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:30.338276+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":55,"num_write":42,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":1011,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:30.464758+0000","last_change":"2024-09-20T19:11:30.464758+0000","last_active":"2024-09-20T19:11:30.464758+0000","last_peered":"2024-09-20T19:11:30.464758+0000","last_clean":"2024-09-20T19:11:30.464758+0000","last_became_active":"2024-09-20T19:10:33.058108+0000","last_became_peered":"2024-09-20T19:10:33.058108+0000","last_unstale":"2024-09-20T19:11:30.464758+0000","last_undegraded":"2024-09-20T19:11:30.464758+0000","last_fullsized":"2024-09-20T19:11:30.464758+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:11:30.464706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:11:30.464706+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:39.225544+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"375'13","reported_seq":1323,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800557+0000","last_change":"2024-09-20T19:11:17.271628+0000","last_active":"2024-09-20T19:11:26.800557+0000","last_peered":"2024-09-20T19:11:26.800557+0000","last_clean":"2024-09-20T19:11:26.800557+0000","last_became_active":"2024-09-20T19:04:20.896424+0000","last_became_peered":"2024-09-20T19:04:20.896424+0000","last_unstale":"2024-09-20T19:11:26.800557+0000","last_undegraded":"2024-09-20T19:11:26.800557+0000","last_fullsized":"2024-09-20T19:11:26.800557+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"375'13","last_scrub_stamp":"2024-09-20T19:11:17.271568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:11:17.271568+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:36.937837+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1010,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.197179+0000","last_change":"2024-09-20T19:11:21.324384+0000","last_active":"2024-09-20T19:11:26.197179+0000","last_peered":"2024-09-20T19:11:26.197179+0000","last_clean":"2024-09-20T19:11:26.197179+0000","last_became_active":"2024-09-20T18:59:05.081814+0000","last_became_peered":"2024-09-20T18:59:05.081814+0000","last_unstale":"2024-09-20T19:11:26.197179+0000","last_undegraded":"2024-09-20T19:11:26.197179+0000","last_fullsized":"2024-09-20T19:11:26.197179+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:21.324328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:11:21.324328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:50.932404+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"74'79","reported_seq":682,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.502622+0000","last_change":"2024-09-20T19:11:07.691872+0000","last_active":"2024-09-20T19:11:31.502622+0000","last_peered":"2024-09-20T19:11:31.502622+0000","last_clean":"2024-09-20T19:11:31.502622+0000","last_became_active":"2024-09-20T19:07:42.988925+0000","last_became_peered":"2024-09-20T19:07:42.988925+0000","last_unstale":"2024-09-20T19:11:31.502622+0000","last_undegraded":"2024-09-20T19:11:31.502622+0000","last_fullsized":"2024-09-20T19:11:31.502622+0000","mapping_epoch":458,"log_start":"0'0","ondisk_log_start":"0'0","created":455,"last_epoch_clean":459,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:11:07.691802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:11:07.691802+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:11.842150+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"503'30","reported_seq":876,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800066+0000","last_change":"2024-09-20T19:11:13.960675+0000","last_active":"2024-09-20T19:11:26.800066+0000","last_peered":"2024-09-20T19:11:26.800066+0000","last_clean":"2024-09-20T19:11:26.800066+0000","last_became_active":"2024-09-20T19:04:49.419140+0000","last_became_peered":"2024-09-20T19:04:49.419140+0000","last_unstale":"2024-09-20T19:11:26.800066+0000","last_undegraded":"2024-09-20T19:11:26.800066+0000","last_fullsized":"2024-09-20T19:11:26.800066+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'30","last_scrub_stamp":"2024-09-20T19:11:13.960591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:11:13.960591+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:16.572086+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":39,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1031,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800630+0000","last_change":"2024-09-20T19:10:12.783104+0000","last_active":"2024-09-20T19:11:26.800630+0000","last_peered":"2024-09-20T19:11:26.800630+0000","last_clean":"2024-09-20T19:11:26.800630+0000","last_became_active":"2024-09-20T18:49:45.942496+0000","last_became_peered":"2024-09-20T18:49:45.942496+0000","last_unstale":"2024-09-20T19:11:26.800630+0000","last_undegraded":"2024-09-20T19:11:26.800630+0000","last_fullsized":"2024-09-20T19:11:26.800630+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:12.783056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:10:12.783056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:38.264049+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"511'12","reported_seq":1238,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800824+0000","last_change":"2024-09-20T19:10:49.456915+0000","last_active":"2024-09-20T19:11:26.800824+0000","last_peered":"2024-09-20T19:11:26.800824+0000","last_clean":"2024-09-20T19:11:26.800824+0000","last_became_active":"2024-09-20T19:10:33.070526+0000","last_became_peered":"2024-09-20T19:10:33.070526+0000","last_unstale":"2024-09-20T19:11:26.800824+0000","last_undegraded":"2024-09-20T19:11:26.800824+0000","last_fullsized":"2024-09-20T19:11:26.800824+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"511'12","last_scrub_stamp":"2024-09-20T19:10:49.456843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:10:49.456843+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:11.998844+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":958,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.799313+0000","last_change":"2024-09-20T19:10:33.070955+0000","last_active":"2024-09-20T19:11:26.799313+0000","last_peered":"2024-09-20T19:11:26.799313+0000","last_clean":"2024-09-20T19:11:26.799313+0000","last_became_active":"2024-09-20T19:10:33.068671+0000","last_became_peered":"2024-09-20T19:10:33.068671+0000","last_unstale":"2024-09-20T19:11:26.799313+0000","last_undegraded":"2024-09-20T19:11:26.799313+0000","last_fullsized":"2024-09-20T19:11:26.799313+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:09:15.568243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:09:15.568243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"74'79","reported_seq":685,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.502003+0000","last_change":"2024-09-20T19:10:37.817529+0000","last_active":"2024-09-20T19:11:31.502003+0000","last_peered":"2024-09-20T19:11:31.502003+0000","last_clean":"2024-09-20T19:11:31.502003+0000","last_became_active":"2024-09-20T19:10:33.065986+0000","last_became_peered":"2024-09-20T19:10:33.065986+0000","last_unstale":"2024-09-20T19:11:31.502003+0000","last_undegraded":"2024-09-20T19:11:31.502003+0000","last_fullsized":"2024-09-20T19:11:31.502003+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":455,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:10:37.817466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:10:37.817466+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:59.851352+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"503'39","reported_seq":1035,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800570+0000","last_change":"2024-09-20T19:10:53.372308+0000","last_active":"2024-09-20T19:11:26.800570+0000","last_peered":"2024-09-20T19:11:26.800570+0000","last_clean":"2024-09-20T19:11:26.800570+0000","last_became_active":"2024-09-20T19:10:33.061190+0000","last_became_peered":"2024-09-20T19:10:33.061190+0000","last_unstale":"2024-09-20T19:11:26.800570+0000","last_undegraded":"2024-09-20T19:11:26.800570+0000","last_fullsized":"2024-09-20T19:11:26.800570+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'39","last_scrub_stamp":"2024-09-20T19:10:53.372238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:10:53.372238+0000","objects_scrubbed":8,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:04.218975+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":68,"num_read_kb":54,"num_write":42,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1017,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.352314+0000","last_change":"2024-09-20T19:11:20.582502+0000","last_active":"2024-09-20T19:11:31.352314+0000","last_peered":"2024-09-20T19:11:31.352314+0000","last_clean":"2024-09-20T19:11:31.352314+0000","last_became_active":"2024-09-20T18:49:45.942253+0000","last_became_peered":"2024-09-20T18:49:45.942253+0000","last_unstale":"2024-09-20T19:11:31.352314+0000","last_undegraded":"2024-09-20T19:11:31.352314+0000","last_fullsized":"2024-09-20T19:11:31.352314+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:20.582454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:11:20.582454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:34.232784+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":606,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.502507+0000","last_change":"2024-09-20T19:10:34.806471+0000","last_active":"2024-09-20T19:11:31.502507+0000","last_peered":"2024-09-20T19:11:31.502507+0000","last_clean":"2024-09-20T19:11:31.502507+0000","last_became_active":"2024-09-20T19:10:33.065396+0000","last_became_peered":"2024-09-20T19:10:33.065396+0000","last_unstale":"2024-09-20T19:11:31.502507+0000","last_undegraded":"2024-09-20T19:11:31.502507+0000","last_fullsized":"2024-09-20T19:11:31.502507+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:34.806385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:10:34.806385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:03.745685+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1043,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.351011+0000","last_change":"2024-09-20T19:11:26.605786+0000","last_active":"2024-09-20T19:11:31.351011+0000","last_peered":"2024-09-20T19:11:31.351011+0000","last_clean":"2024-09-20T19:11:31.351011+0000","last_became_active":"2024-09-20T18:59:05.085131+0000","last_became_peered":"2024-09-20T18:59:05.085131+0000","last_unstale":"2024-09-20T19:11:31.351011+0000","last_undegraded":"2024-09-20T19:11:31.351011+0000","last_fullsized":"2024-09-20T19:11:31.351011+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:26.605739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:11:26.605739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:35.075195+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"74'79","reported_seq":189,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.799741+0000","last_change":"2024-09-20T19:10:52.958927+0000","last_active":"2024-09-20T19:11:26.799741+0000","last_peered":"2024-09-20T19:11:26.799741+0000","last_clean":"2024-09-20T19:11:26.799741+0000","last_became_active":"2024-09-20T19:10:33.059058+0000","last_became_peered":"2024-09-20T19:10:33.059058+0000","last_unstale":"2024-09-20T19:11:26.799741+0000","last_undegraded":"2024-09-20T19:11:26.799741+0000","last_fullsized":"2024-09-20T19:11:26.799741+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":455,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:10:52.958863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:10:52.958863+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:04.902192+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"503'53","reported_seq":1404,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.352244+0000","last_change":"2024-09-20T19:11:12.468531+0000","last_active":"2024-09-20T19:11:31.352244+0000","last_peered":"2024-09-20T19:11:31.352244+0000","last_clean":"2024-09-20T19:11:31.352244+0000","last_became_active":"2024-09-20T19:03:16.976916+0000","last_became_peered":"2024-09-20T19:03:16.976916+0000","last_unstale":"2024-09-20T19:11:31.352244+0000","last_undegraded":"2024-09-20T19:11:31.352244+0000","last_fullsized":"2024-09-20T19:11:31.352244+0000","mapping_epoch":352,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":353,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'53","last_scrub_stamp":"2024-09-20T19:11:12.468447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:11:12.468447+0000","objects_scrubbed":10,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:34.327889+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":84,"num_read_kb":69,"num_write":50,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":357,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.003441+0000","last_change":"2024-09-20T19:10:48.828570+0000","last_active":"2024-09-20T19:11:26.003441+0000","last_peered":"2024-09-20T19:11:26.003441+0000","last_clean":"2024-09-20T19:11:26.003441+0000","last_became_active":"2024-09-20T19:10:33.055871+0000","last_became_peered":"2024-09-20T19:10:33.055871+0000","last_unstale":"2024-09-20T19:11:26.003441+0000","last_undegraded":"2024-09-20T19:11:26.003441+0000","last_fullsized":"2024-09-20T19:11:26.003441+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:48.828503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:10:48.828503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:53.094861+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":1063,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800185+0000","last_change":"2024-09-20T19:10:05.653339+0000","last_active":"2024-09-20T19:11:26.800185+0000","last_peered":"2024-09-20T19:11:26.800185+0000","last_clean":"2024-09-20T19:11:26.800185+0000","last_became_active":"2024-09-20T18:59:05.086786+0000","last_became_peered":"2024-09-20T18:59:05.086786+0000","last_unstale":"2024-09-20T19:11:26.800185+0000","last_undegraded":"2024-09-20T19:11:26.800185+0000","last_fullsized":"2024-09-20T19:11:26.800185+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-20T19:10:05.653292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:10:05.653292+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:28.948889+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0012781159999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":997,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.799757+0000","last_change":"2024-09-20T19:11:08.471903+0000","last_active":"2024-09-20T19:11:26.799757+0000","last_peered":"2024-09-20T19:11:26.799757+0000","last_clean":"2024-09-20T19:11:26.799757+0000","last_became_active":"2024-09-20T19:10:33.055631+0000","last_became_peered":"2024-09-20T19:10:33.055631+0000","last_unstale":"2024-09-20T19:11:26.799757+0000","last_undegraded":"2024-09-20T19:11:26.799757+0000","last_fullsized":"2024-09-20T19:11:26.799757+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:08.471839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:11:08.471839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:24.310778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1012,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800503+0000","last_change":"2024-09-20T19:11:14.129151+0000","last_active":"2024-09-20T19:11:26.800503+0000","last_peered":"2024-09-20T19:11:26.800503+0000","last_clean":"2024-09-20T19:11:26.800503+0000","last_became_active":"2024-09-20T19:04:20.896281+0000","last_became_peered":"2024-09-20T19:04:20.896281+0000","last_unstale":"2024-09-20T19:11:26.800503+0000","last_undegraded":"2024-09-20T19:11:26.800503+0000","last_fullsized":"2024-09-20T19:11:26.800503+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:14.129057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:11:14.129057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:28.639340+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"74'79","reported_seq":1111,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.196990+0000","last_change":"2024-09-20T19:10:47.068872+0000","last_active":"2024-09-20T19:11:26.196990+0000","last_peered":"2024-09-20T19:11:26.196990+0000","last_clean":"2024-09-20T19:11:26.196990+0000","last_became_active":"2024-09-20T19:10:33.056170+0000","last_became_peered":"2024-09-20T19:10:33.056170+0000","last_unstale":"2024-09-20T19:11:26.196990+0000","last_undegraded":"2024-09-20T19:11:26.196990+0000","last_fullsized":"2024-09-20T19:11:26.196990+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":455,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:10:47.068800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:10:47.068800+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:47.880145+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"503'21","reported_seq":1078,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.351916+0000","last_change":"2024-09-20T19:10:36.475204+0000","last_active":"2024-09-20T19:11:31.351916+0000","last_peered":"2024-09-20T19:11:31.351916+0000","last_clean":"2024-09-20T19:11:31.351916+0000","last_became_active":"2024-09-20T19:10:33.062421+0000","last_became_peered":"2024-09-20T19:10:33.062421+0000","last_unstale":"2024-09-20T19:11:31.351916+0000","last_undegraded":"2024-09-20T19:11:31.351916+0000","last_fullsized":"2024-09-20T19:11:31.351916+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'21","last_scrub_stamp":"2024-09-20T19:10:36.475150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:10:36.475150+0000","objects_scrubbed":5,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:41.186787+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":42,"num_read_kb":31,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1047,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.351223+0000","last_change":"2024-09-20T19:11:18.569751+0000","last_active":"2024-09-20T19:11:31.351223+0000","last_peered":"2024-09-20T19:11:31.351223+0000","last_clean":"2024-09-20T19:11:31.351223+0000","last_became_active":"2024-09-20T18:59:05.079789+0000","last_became_peered":"2024-09-20T18:59:05.079789+0000","last_unstale":"2024-09-20T19:11:31.351223+0000","last_undegraded":"2024-09-20T19:11:31.351223+0000","last_fullsized":"2024-09-20T19:11:31.351223+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:18.569691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:11:18.569691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:21.202936+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":1032,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.349912+0000","last_change":"2024-09-20T19:10:41.015564+0000","last_active":"2024-09-20T19:11:31.349912+0000","last_peered":"2024-09-20T19:11:31.349912+0000","last_clean":"2024-09-20T19:11:31.349912+0000","last_became_active":"2024-09-20T19:10:33.070316+0000","last_became_peered":"2024-09-20T19:10:33.070316+0000","last_unstale":"2024-09-20T19:11:31.349912+0000","last_undegraded":"2024-09-20T19:11:31.349912+0000","last_fullsized":"2024-09-20T19:11:31.349912+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-20T19:10:41.015471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:10:41.015471+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:04.761830+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00095558800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":996,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.351125+0000","last_change":"2024-09-20T19:11:00.752002+0000","last_active":"2024-09-20T19:11:31.351125+0000","last_peered":"2024-09-20T19:11:31.351125+0000","last_clean":"2024-09-20T19:11:31.351125+0000","last_became_active":"2024-09-20T18:49:49.954424+0000","last_became_peered":"2024-09-20T18:49:49.954424+0000","last_unstale":"2024-09-20T19:11:31.351125+0000","last_undegraded":"2024-09-20T19:11:31.351125+0000","last_fullsized":"2024-09-20T19:11:31.351125+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:00.751949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:11:00.751949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:29.709540+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":997,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.197490+0000","last_change":"2024-09-20T19:10:03.177269+0000","last_active":"2024-09-20T19:11:26.197490+0000","last_peered":"2024-09-20T19:11:26.197490+0000","last_clean":"2024-09-20T19:11:26.197490+0000","last_became_active":"2024-09-20T18:59:05.081459+0000","last_became_peered":"2024-09-20T18:59:05.081459+0000","last_unstale":"2024-09-20T19:11:26.197490+0000","last_undegraded":"2024-09-20T19:11:26.197490+0000","last_fullsized":"2024-09-20T19:11:26.197490+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:03.177222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:10:03.177222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:32.875230+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"74'79","reported_seq":188,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.351799+0000","last_change":"2024-09-20T19:11:06.496487+0000","last_active":"2024-09-20T19:11:31.351799+0000","last_peered":"2024-09-20T19:11:31.351799+0000","last_clean":"2024-09-20T19:11:31.351799+0000","last_became_active":"2024-09-20T19:07:42.991822+0000","last_became_peered":"2024-09-20T19:07:42.991822+0000","last_unstale":"2024-09-20T19:11:31.351799+0000","last_undegraded":"2024-09-20T19:11:31.351799+0000","last_fullsized":"2024-09-20T19:11:31.351799+0000","mapping_epoch":458,"log_start":"0'0","ondisk_log_start":"0'0","created":455,"last_epoch_clean":459,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:11:06.496437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:11:06.496437+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:11.805852+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"503'36","reported_seq":1011,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.801229+0000","last_change":"2024-09-20T19:11:21.505648+0000","last_active":"2024-09-20T19:11:26.801229+0000","last_peered":"2024-09-20T19:11:26.801229+0000","last_clean":"2024-09-20T19:11:26.801229+0000","last_became_active":"2024-09-20T19:10:33.066863+0000","last_became_peered":"2024-09-20T19:10:33.066863+0000","last_unstale":"2024-09-20T19:11:26.801229+0000","last_undegraded":"2024-09-20T19:11:26.801229+0000","last_fullsized":"2024-09-20T19:11:26.801229+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'36","last_scrub_stamp":"2024-09-20T19:11:21.505584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:11:21.505584+0000","objects_scrubbed":6,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:39.637463+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":52,"num_read_kb":46,"num_write":36,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":938,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.799802+0000","last_change":"2024-09-20T19:10:50.457309+0000","last_active":"2024-09-20T19:11:26.799802+0000","last_peered":"2024-09-20T19:11:26.799802+0000","last_clean":"2024-09-20T19:11:26.799802+0000","last_became_active":"2024-09-20T19:10:33.067036+0000","last_became_peered":"2024-09-20T19:10:33.067036+0000","last_unstale":"2024-09-20T19:11:26.799802+0000","last_undegraded":"2024-09-20T19:11:26.799802+0000","last_fullsized":"2024-09-20T19:11:26.799802+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:50.457253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:10:50.457253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:11.294614+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":2335,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:30.148101+0000","last_change":"2024-09-20T19:10:44.087064+0000","last_active":"2024-09-20T19:11:30.148101+0000","last_peered":"2024-09-20T19:11:30.148101+0000","last_clean":"2024-09-20T19:11:30.148101+0000","last_became_active":"2024-09-20T19:10:33.054628+0000","last_became_peered":"2024-09-20T19:10:33.054628+0000","last_unstale":"2024-09-20T19:11:30.148101+0000","last_undegraded":"2024-09-20T19:11:30.148101+0000","last_fullsized":"2024-09-20T19:11:30.148101+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-20T19:10:44.086956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:43.921002+0000","last_clean_scrub_stamp":"2024-09-20T19:10:44.086956+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:06.195464+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0010996199999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1295,"num_read_kb":1290,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1026,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800703+0000","last_change":"2024-09-20T19:10:48.916403+0000","last_active":"2024-09-20T19:11:26.800703+0000","last_peered":"2024-09-20T19:11:26.800703+0000","last_clean":"2024-09-20T19:11:26.800703+0000","last_became_active":"2024-09-20T19:10:33.055822+0000","last_became_peered":"2024-09-20T19:10:33.055822+0000","last_unstale":"2024-09-20T19:11:26.800703+0000","last_undegraded":"2024-09-20T19:11:26.800703+0000","last_fullsized":"2024-09-20T19:11:26.800703+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:48.916355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:10:48.916355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:09.089843+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1021,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.352064+0000","last_change":"2024-09-20T19:11:30.578866+0000","last_active":"2024-09-20T19:11:31.352064+0000","last_peered":"2024-09-20T19:11:31.352064+0000","last_clean":"2024-09-20T19:11:31.352064+0000","last_became_active":"2024-09-20T18:49:51.970343+0000","last_became_peered":"2024-09-20T18:49:51.970343+0000","last_unstale":"2024-09-20T19:11:31.352064+0000","last_undegraded":"2024-09-20T19:11:31.352064+0000","last_fullsized":"2024-09-20T19:11:31.352064+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:30.578798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:11:30.578798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:48.536129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"503'46","reported_seq":862,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.502389+0000","last_change":"2024-09-20T19:11:14.575075+0000","last_active":"2024-09-20T19:11:31.502389+0000","last_peered":"2024-09-20T19:11:31.502389+0000","last_clean":"2024-09-20T19:11:31.502389+0000","last_became_active":"2024-09-20T19:04:20.928727+0000","last_became_peered":"2024-09-20T19:04:20.928727+0000","last_unstale":"2024-09-20T19:11:31.502389+0000","last_undegraded":"2024-09-20T19:11:31.502389+0000","last_fullsized":"2024-09-20T19:11:31.502389+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'46","last_scrub_stamp":"2024-09-20T19:11:14.574982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:11:14.574982+0000","objects_scrubbed":8,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:28.154337+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":60,"num_read_kb":50,"num_write":37,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":462,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1045,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:28.223790+0000","last_change":"2024-09-20T19:11:28.223790+0000","last_active":"2024-09-20T19:11:28.223790+0000","last_peered":"2024-09-20T19:11:28.223790+0000","last_clean":"2024-09-20T19:11:28.223790+0000","last_became_active":"2024-09-20T18:59:05.079111+0000","last_became_peered":"2024-09-20T18:59:05.079111+0000","last_unstale":"2024-09-20T19:11:28.223790+0000","last_undegraded":"2024-09-20T19:11:28.223790+0000","last_fullsized":"2024-09-20T19:11:28.223790+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:28.223738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:11:28.223738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:50.811519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":1106,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.197162+0000","last_change":"2024-09-20T19:10:45.068703+0000","last_active":"2024-09-20T19:11:26.197162+0000","last_peered":"2024-09-20T19:11:26.197162+0000","last_clean":"2024-09-20T19:11:26.197162+0000","last_became_active":"2024-09-20T19:10:33.055105+0000","last_became_peered":"2024-09-20T19:10:33.055105+0000","last_unstale":"2024-09-20T19:11:26.197162+0000","last_undegraded":"2024-09-20T19:11:26.197162+0000","last_fullsized":"2024-09-20T19:11:26.197162+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:10:45.068650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:10:45.068650+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:13.176018+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1000,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.196746+0000","last_change":"2024-09-20T19:09:55.125713+0000","last_active":"2024-09-20T19:11:26.196746+0000","last_peered":"2024-09-20T19:11:26.196746+0000","last_clean":"2024-09-20T19:11:26.196746+0000","last_became_active":"2024-09-20T18:49:49.956128+0000","last_became_peered":"2024-09-20T18:49:49.956128+0000","last_unstale":"2024-09-20T19:11:26.196746+0000","last_undegraded":"2024-09-20T19:11:26.196746+0000","last_fullsized":"2024-09-20T19:11:26.196746+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:09:55.125653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:09:55.125653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1044,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:29.976955+0000","last_change":"2024-09-20T19:11:29.976906+0000","last_active":"2024-09-20T19:11:29.976955+0000","last_peered":"2024-09-20T19:11:29.976955+0000","last_clean":"2024-09-20T19:11:29.976955+0000","last_became_active":"2024-09-20T19:04:20.899321+0000","last_became_peered":"2024-09-20T19:04:20.899321+0000","last_unstale":"2024-09-20T19:11:29.976955+0000","last_undegraded":"2024-09-20T19:11:29.976955+0000","last_fullsized":"2024-09-20T19:11:29.976955+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:29.976839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:11:29.976839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:44.848330+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"503'27","reported_seq":370,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.003771+0000","last_change":"2024-09-20T19:09:08.725949+0000","last_active":"2024-09-20T19:11:26.003771+0000","last_peered":"2024-09-20T19:11:26.003771+0000","last_clean":"2024-09-20T19:11:26.003771+0000","last_became_active":"2024-09-20T19:04:20.897175+0000","last_became_peered":"2024-09-20T19:04:20.897175+0000","last_unstale":"2024-09-20T19:11:26.003771+0000","last_undegraded":"2024-09-20T19:11:26.003771+0000","last_fullsized":"2024-09-20T19:11:26.003771+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'19","last_scrub_stamp":"2024-09-20T19:09:08.725879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:09:08.725879+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":36,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":346,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.003073+0000","last_change":"2024-09-20T19:10:20.816540+0000","last_active":"2024-09-20T19:11:26.003073+0000","last_peered":"2024-09-20T19:11:26.003073+0000","last_clean":"2024-09-20T19:11:26.003073+0000","last_became_active":"2024-09-20T19:04:20.887102+0000","last_became_peered":"2024-09-20T19:04:20.887102+0000","last_unstale":"2024-09-20T19:11:26.003073+0000","last_undegraded":"2024-09-20T19:11:26.003073+0000","last_fullsized":"2024-09-20T19:11:26.003073+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:20.816485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:10:20.816485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:45.763638+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1053,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800516+0000","last_change":"2024-09-20T19:11:15.002216+0000","last_active":"2024-09-20T19:11:26.800516+0000","last_peered":"2024-09-20T19:11:26.800516+0000","last_clean":"2024-09-20T19:11:26.800516+0000","last_became_active":"2024-09-20T19:04:20.899331+0000","last_became_peered":"2024-09-20T19:04:20.899331+0000","last_unstale":"2024-09-20T19:11:26.800516+0000","last_undegraded":"2024-09-20T19:11:26.800516+0000","last_fullsized":"2024-09-20T19:11:26.800516+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:15.002153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:11:15.002153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:26.273304+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1016,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.799957+0000","last_change":"2024-09-20T19:11:06.275569+0000","last_active":"2024-09-20T19:11:26.799957+0000","last_peered":"2024-09-20T19:11:26.799957+0000","last_clean":"2024-09-20T19:11:26.799957+0000","last_became_active":"2024-09-20T18:59:05.079897+0000","last_became_peered":"2024-09-20T18:59:05.079897+0000","last_unstale":"2024-09-20T19:11:26.799957+0000","last_undegraded":"2024-09-20T19:11:26.799957+0000","last_fullsized":"2024-09-20T19:11:26.799957+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:06.275512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:11:06.275512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:27.646469+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.8","version":"74'79","reported_seq":185,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.799630+0000","last_change":"2024-09-20T19:11:08.032372+0000","last_active":"2024-09-20T19:11:26.799630+0000","last_peered":"2024-09-20T19:11:26.799630+0000","last_clean":"2024-09-20T19:11:26.799630+0000","last_became_active":"2024-09-20T19:07:42.988175+0000","last_became_peered":"2024-09-20T19:07:42.988175+0000","last_unstale":"2024-09-20T19:11:26.799630+0000","last_undegraded":"2024-09-20T19:11:26.799630+0000","last_fullsized":"2024-09-20T19:11:26.799630+0000","mapping_epoch":458,"log_start":"0'0","ondisk_log_start":"0'0","created":455,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:11:08.032295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:11:08.032295+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:30.843530+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"503'45","reported_seq":385,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.003827+0000","last_change":"2024-09-20T19:10:51.809954+0000","last_active":"2024-09-20T19:11:26.003827+0000","last_peered":"2024-09-20T19:11:26.003827+0000","last_clean":"2024-09-20T19:11:26.003827+0000","last_became_active":"2024-09-20T19:04:20.883306+0000","last_became_peered":"2024-09-20T19:04:20.883306+0000","last_unstale":"2024-09-20T19:11:26.003827+0000","last_undegraded":"2024-09-20T19:11:26.003827+0000","last_fullsized":"2024-09-20T19:11:26.003827+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'45","last_scrub_stamp":"2024-09-20T19:10:51.809866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:10:51.809866+0000","objects_scrubbed":8,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:59.563385+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":70,"num_read_kb":60,"num_write":46,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":612,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.502689+0000","last_change":"2024-09-20T19:11:15.534315+0000","last_active":"2024-09-20T19:11:31.502689+0000","last_peered":"2024-09-20T19:11:31.502689+0000","last_clean":"2024-09-20T19:11:31.502689+0000","last_became_active":"2024-09-20T19:04:20.899415+0000","last_became_peered":"2024-09-20T19:04:20.899415+0000","last_unstale":"2024-09-20T19:11:31.502689+0000","last_undegraded":"2024-09-20T19:11:31.502689+0000","last_fullsized":"2024-09-20T19:11:31.502689+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:15.534261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:11:15.534261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:41.287064+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1044,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.352165+0000","last_change":"2024-09-20T19:10:37.482578+0000","last_active":"2024-09-20T19:11:31.352165+0000","last_peered":"2024-09-20T19:11:31.352165+0000","last_clean":"2024-09-20T19:11:31.352165+0000","last_became_active":"2024-09-20T19:10:33.061076+0000","last_became_peered":"2024-09-20T19:10:33.061076+0000","last_unstale":"2024-09-20T19:11:31.352165+0000","last_undegraded":"2024-09-20T19:11:31.352165+0000","last_fullsized":"2024-09-20T19:11:31.352165+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:37.482528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:10:37.482528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:54.510997+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1040,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.351400+0000","last_change":"2024-09-20T19:11:28.421417+0000","last_active":"2024-09-20T19:11:31.351400+0000","last_peered":"2024-09-20T19:11:31.351400+0000","last_clean":"2024-09-20T19:11:31.351400+0000","last_became_active":"2024-09-20T19:04:20.897569+0000","last_became_peered":"2024-09-20T19:04:20.897569+0000","last_unstale":"2024-09-20T19:11:31.351400+0000","last_undegraded":"2024-09-20T19:11:31.351400+0000","last_fullsized":"2024-09-20T19:11:31.351400+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:28.421366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:11:28.421366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:32.852109+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"74'79","reported_seq":1100,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:28.231862+0000","last_change":"2024-09-20T19:11:28.231862+0000","last_active":"2024-09-20T19:11:28.231862+0000","last_peered":"2024-09-20T19:11:28.231862+0000","last_clean":"2024-09-20T19:11:28.231862+0000","last_became_active":"2024-09-20T19:07:42.989434+0000","last_became_peered":"2024-09-20T19:07:42.989434+0000","last_unstale":"2024-09-20T19:11:28.231862+0000","last_undegraded":"2024-09-20T19:11:28.231862+0000","last_fullsized":"2024-09-20T19:11:28.231862+0000","mapping_epoch":458,"log_start":"0'0","ondisk_log_start":"0'0","created":455,"last_epoch_clean":459,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-20T19:11:28.231803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:47:15.691433+0000","last_clean_scrub_stamp":"2024-09-20T19:11:28.231803+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:51.068626+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"503'30","reported_seq":365,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.003460+0000","last_change":"2024-09-20T19:10:06.663245+0000","last_active":"2024-09-20T19:11:26.003460+0000","last_peered":"2024-09-20T19:11:26.003460+0000","last_clean":"2024-09-20T19:11:26.003460+0000","last_became_active":"2024-09-20T19:04:20.896483+0000","last_became_peered":"2024-09-20T19:04:20.896483+0000","last_unstale":"2024-09-20T19:11:26.003460+0000","last_undegraded":"2024-09-20T19:11:26.003460+0000","last_fullsized":"2024-09-20T19:11:26.003460+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'30","last_scrub_stamp":"2024-09-20T19:10:06.663192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:10:06.663192+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:33.785131+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":39,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1050,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800290+0000","last_change":"2024-09-20T19:10:36.020262+0000","last_active":"2024-09-20T19:11:26.800290+0000","last_peered":"2024-09-20T19:11:26.800290+0000","last_clean":"2024-09-20T19:11:26.800290+0000","last_became_active":"2024-09-20T19:10:33.066807+0000","last_became_peered":"2024-09-20T19:10:33.066807+0000","last_unstale":"2024-09-20T19:11:26.800290+0000","last_undegraded":"2024-09-20T19:11:26.800290+0000","last_fullsized":"2024-09-20T19:11:26.800290+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:36.020215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:10:36.020215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:52.192006+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":1297,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.350144+0000","last_change":"2024-09-20T19:11:01.758072+0000","last_active":"2024-09-20T19:11:31.350144+0000","last_peered":"2024-09-20T19:11:31.350144+0000","last_clean":"2024-09-20T19:11:31.350144+0000","last_became_active":"2024-09-20T18:49:49.957005+0000","last_became_peered":"2024-09-20T18:49:49.957005+0000","last_unstale":"2024-09-20T19:11:31.350144+0000","last_undegraded":"2024-09-20T19:11:31.350144+0000","last_fullsized":"2024-09-20T19:11:31.350144+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T19:11:01.758014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:11:01.758014+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:05.141869+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":346,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.003837+0000","last_change":"2024-09-20T19:11:03.923503+0000","last_active":"2024-09-20T19:11:26.003837+0000","last_peered":"2024-09-20T19:11:26.003837+0000","last_clean":"2024-09-20T19:11:26.003837+0000","last_became_active":"2024-09-20T19:04:20.884994+0000","last_became_peered":"2024-09-20T19:04:20.884994+0000","last_unstale":"2024-09-20T19:11:26.003837+0000","last_undegraded":"2024-09-20T19:11:26.003837+0000","last_fullsized":"2024-09-20T19:11:26.003837+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:03.923445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:11:03.923445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:26.634146+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"503'21","reported_seq":1051,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800933+0000","last_change":"2024-09-20T19:10:33.080187+0000","last_active":"2024-09-20T19:11:26.800933+0000","last_peered":"2024-09-20T19:11:26.800933+0000","last_clean":"2024-09-20T19:11:26.800933+0000","last_became_active":"2024-09-20T19:10:33.055843+0000","last_became_peered":"2024-09-20T19:10:33.055843+0000","last_unstale":"2024-09-20T19:11:26.800933+0000","last_undegraded":"2024-09-20T19:11:26.800933+0000","last_fullsized":"2024-09-20T19:11:26.800933+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'15","last_scrub_stamp":"2024-09-20T19:09:10.482098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:09:10.482098+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":577,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.350150+0000","last_change":"2024-09-20T19:10:27.013476+0000","last_active":"2024-09-20T19:11:31.350150+0000","last_peered":"2024-09-20T19:11:31.350150+0000","last_clean":"2024-09-20T19:11:31.350150+0000","last_became_active":"2024-09-20T19:00:01.190875+0000","last_became_peered":"2024-09-20T19:00:01.190875+0000","last_unstale":"2024-09-20T19:11:31.350150+0000","last_undegraded":"2024-09-20T19:11:31.350150+0000","last_fullsized":"2024-09-20T19:11:31.350150+0000","mapping_epoch":277,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":278,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:27.013413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:10:27.013413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:38.292329+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1045,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.350065+0000","last_change":"2024-09-20T19:11:22.540975+0000","last_active":"2024-09-20T19:11:31.350065+0000","last_peered":"2024-09-20T19:11:31.350065+0000","last_clean":"2024-09-20T19:11:31.350065+0000","last_became_active":"2024-09-20T19:04:20.889273+0000","last_became_peered":"2024-09-20T19:04:20.889273+0000","last_unstale":"2024-09-20T19:11:31.350065+0000","last_undegraded":"2024-09-20T19:11:31.350065+0000","last_fullsized":"2024-09-20T19:11:31.350065+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:22.540918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:11:22.540918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:50.723472+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":909,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.799901+0000","last_change":"2024-09-20T19:10:51.450775+0000","last_active":"2024-09-20T19:11:26.799901+0000","last_peered":"2024-09-20T19:11:26.799901+0000","last_clean":"2024-09-20T19:11:26.799901+0000","last_became_active":"2024-09-20T19:10:33.065201+0000","last_became_peered":"2024-09-20T19:10:33.065201+0000","last_unstale":"2024-09-20T19:11:26.799901+0000","last_undegraded":"2024-09-20T19:11:26.799901+0000","last_fullsized":"2024-09-20T19:11:26.799901+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:51.450714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:10:51.450714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:55.908573+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"503'51","reported_seq":671,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.502009+0000","last_change":"2024-09-20T19:11:10.634984+0000","last_active":"2024-09-20T19:11:31.502009+0000","last_peered":"2024-09-20T19:11:31.502009+0000","last_clean":"2024-09-20T19:11:31.502009+0000","last_became_active":"2024-09-20T18:59:05.087391+0000","last_became_peered":"2024-09-20T18:59:05.087391+0000","last_unstale":"2024-09-20T19:11:31.502009+0000","last_undegraded":"2024-09-20T19:11:31.502009+0000","last_fullsized":"2024-09-20T19:11:31.502009+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'51","last_scrub_stamp":"2024-09-20T19:11:10.634911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:11:10.634911+0000","objects_scrubbed":9,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:30.842412+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":79,"num_read_kb":68,"num_write":52,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1029,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.197822+0000","last_change":"2024-09-20T19:10:39.057768+0000","last_active":"2024-09-20T19:11:26.197822+0000","last_peered":"2024-09-20T19:11:26.197822+0000","last_clean":"2024-09-20T19:11:26.197822+0000","last_became_active":"2024-09-20T18:59:05.086670+0000","last_became_peered":"2024-09-20T18:59:05.086670+0000","last_unstale":"2024-09-20T19:11:26.197822+0000","last_undegraded":"2024-09-20T19:11:26.197822+0000","last_fullsized":"2024-09-20T19:11:26.197822+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:39.057708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:10:39.057708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:58.262736+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1030,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.350050+0000","last_change":"2024-09-20T19:10:38.994859+0000","last_active":"2024-09-20T19:11:31.350050+0000","last_peered":"2024-09-20T19:11:31.350050+0000","last_clean":"2024-09-20T19:11:31.350050+0000","last_became_active":"2024-09-20T19:10:33.073428+0000","last_became_peered":"2024-09-20T19:10:33.073428+0000","last_unstale":"2024-09-20T19:11:31.350050+0000","last_undegraded":"2024-09-20T19:11:31.350050+0000","last_fullsized":"2024-09-20T19:11:31.350050+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:38.994809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:10:38.994809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:08.197204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":995,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.196973+0000","last_change":"2024-09-20T19:11:23.272980+0000","last_active":"2024-09-20T19:11:26.196973+0000","last_peered":"2024-09-20T19:11:26.196973+0000","last_clean":"2024-09-20T19:11:26.196973+0000","last_became_active":"2024-09-20T18:49:51.970798+0000","last_became_peered":"2024-09-20T18:49:51.970798+0000","last_unstale":"2024-09-20T19:11:26.196973+0000","last_undegraded":"2024-09-20T19:11:26.196973+0000","last_fullsized":"2024-09-20T19:11:26.196973+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:23.272918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:11:23.272918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:34.736526+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"503'24","reported_seq":359,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.003236+0000","last_change":"2024-09-20T19:10:01.667939+0000","last_active":"2024-09-20T19:11:26.003236+0000","last_peered":"2024-09-20T19:11:26.003236+0000","last_clean":"2024-09-20T19:11:26.003236+0000","last_became_active":"2024-09-20T19:04:20.888905+0000","last_became_peered":"2024-09-20T19:04:20.888905+0000","last_unstale":"2024-09-20T19:11:26.003236+0000","last_undegraded":"2024-09-20T19:11:26.003236+0000","last_fullsized":"2024-09-20T19:11:26.003236+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'24","last_scrub_stamp":"2024-09-20T19:10:01.667857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:10:01.667857+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":42,"num_read_kb":33,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1056,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.197868+0000","last_change":"2024-09-20T19:11:00.050521+0000","last_active":"2024-09-20T19:11:26.197868+0000","last_peered":"2024-09-20T19:11:26.197868+0000","last_clean":"2024-09-20T19:11:26.197868+0000","last_became_active":"2024-09-20T19:04:20.892429+0000","last_became_peered":"2024-09-20T19:04:20.892429+0000","last_unstale":"2024-09-20T19:11:26.197868+0000","last_undegraded":"2024-09-20T19:11:26.197868+0000","last_fullsized":"2024-09-20T19:11:26.197868+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:00.050463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:11:00.050463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:13.299758+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1023,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.349767+0000","last_change":"2024-09-20T19:10:37.957926+0000","last_active":"2024-09-20T19:11:31.349767+0000","last_peered":"2024-09-20T19:11:31.349767+0000","last_clean":"2024-09-20T19:11:31.349767+0000","last_became_active":"2024-09-20T19:10:33.071122+0000","last_became_peered":"2024-09-20T19:10:33.071122+0000","last_unstale":"2024-09-20T19:11:31.349767+0000","last_undegraded":"2024-09-20T19:11:31.349767+0000","last_fullsized":"2024-09-20T19:11:31.349767+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:37.957844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:10:37.957844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:42.877703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1010,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.799885+0000","last_change":"2024-09-20T19:11:21.296277+0000","last_active":"2024-09-20T19:11:26.799885+0000","last_peered":"2024-09-20T19:11:26.799885+0000","last_clean":"2024-09-20T19:11:26.799885+0000","last_became_active":"2024-09-20T18:49:51.972970+0000","last_became_peered":"2024-09-20T18:49:51.972970+0000","last_unstale":"2024-09-20T19:11:26.799885+0000","last_undegraded":"2024-09-20T19:11:26.799885+0000","last_fullsized":"2024-09-20T19:11:26.799885+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:21.296228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:11:21.296228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:29.809923+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"503'39","reported_seq":1134,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:27.988380+0000","last_change":"2024-09-20T19:11:27.988380+0000","last_active":"2024-09-20T19:11:27.988380+0000","last_peered":"2024-09-20T19:11:27.988380+0000","last_clean":"2024-09-20T19:11:27.988380+0000","last_became_active":"2024-09-20T18:59:05.083726+0000","last_became_peered":"2024-09-20T18:59:05.083726+0000","last_unstale":"2024-09-20T19:11:27.988380+0000","last_undegraded":"2024-09-20T19:11:27.988380+0000","last_fullsized":"2024-09-20T19:11:27.988380+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'39","last_scrub_stamp":"2024-09-20T19:11:27.988290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:11:27.988290+0000","objects_scrubbed":7,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:55.230857+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":1091,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.197977+0000","last_change":"2024-09-20T19:10:43.059450+0000","last_active":"2024-09-20T19:11:26.197977+0000","last_peered":"2024-09-20T19:11:26.197977+0000","last_clean":"2024-09-20T19:11:26.197977+0000","last_became_active":"2024-09-20T19:10:33.053470+0000","last_became_peered":"2024-09-20T19:10:33.053470+0000","last_unstale":"2024-09-20T19:11:26.197977+0000","last_undegraded":"2024-09-20T19:11:26.197977+0000","last_fullsized":"2024-09-20T19:11:26.197977+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-20T19:10:43.059386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:10:43.059386+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:01.060512+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"375'14","reported_seq":1335,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.258588+0000","last_change":"2024-09-20T19:11:30.224880+0000","last_active":"2024-09-20T19:11:31.258588+0000","last_peered":"2024-09-20T19:11:31.258588+0000","last_clean":"2024-09-20T19:11:31.258588+0000","last_became_active":"2024-09-20T19:04:20.892956+0000","last_became_peered":"2024-09-20T19:04:20.892956+0000","last_unstale":"2024-09-20T19:11:31.258588+0000","last_undegraded":"2024-09-20T19:11:31.258588+0000","last_fullsized":"2024-09-20T19:11:31.258588+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"375'14","last_scrub_stamp":"2024-09-20T19:11:30.224806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:11:30.224806+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:51.715022+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1027,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.799930+0000","last_change":"2024-09-20T19:10:42.828430+0000","last_active":"2024-09-20T19:11:26.799930+0000","last_peered":"2024-09-20T19:11:26.799930+0000","last_clean":"2024-09-20T19:11:26.799930+0000","last_became_active":"2024-09-20T19:10:33.056617+0000","last_became_peered":"2024-09-20T19:10:33.056617+0000","last_unstale":"2024-09-20T19:11:26.799930+0000","last_undegraded":"2024-09-20T19:11:26.799930+0000","last_fullsized":"2024-09-20T19:11:26.799930+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:42.828351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:10:42.828351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:45.665648+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":598,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.500248+0000","last_change":"2024-09-20T19:10:41.884113+0000","last_active":"2024-09-20T19:11:31.500248+0000","last_peered":"2024-09-20T19:11:31.500248+0000","last_clean":"2024-09-20T19:11:31.500248+0000","last_became_active":"2024-09-20T19:10:33.065397+0000","last_became_peered":"2024-09-20T19:10:33.065397+0000","last_unstale":"2024-09-20T19:11:31.500248+0000","last_undegraded":"2024-09-20T19:11:31.500248+0000","last_fullsized":"2024-09-20T19:11:31.500248+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:41.884061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:10:41.884061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:57.894186+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"503'18","reported_seq":1044,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800740+0000","last_change":"2024-09-20T19:10:11.085494+0000","last_active":"2024-09-20T19:11:26.800740+0000","last_peered":"2024-09-20T19:11:26.800740+0000","last_clean":"2024-09-20T19:11:26.800740+0000","last_became_active":"2024-09-20T18:49:48.500662+0000","last_became_peered":"2024-09-20T18:49:48.500662+0000","last_unstale":"2024-09-20T19:11:26.800740+0000","last_undegraded":"2024-09-20T19:11:26.800740+0000","last_fullsized":"2024-09-20T19:11:26.800740+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'18","last_scrub_stamp":"2024-09-20T19:10:11.085439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:10:11.085439+0000","objects_scrubbed":3,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:40.638496+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":24,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1005,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.799554+0000","last_change":"2024-09-20T19:10:22.919792+0000","last_active":"2024-09-20T19:11:26.799554+0000","last_peered":"2024-09-20T19:11:26.799554+0000","last_clean":"2024-09-20T19:11:26.799554+0000","last_became_active":"2024-09-20T18:49:49.956749+0000","last_became_peered":"2024-09-20T18:49:49.956749+0000","last_unstale":"2024-09-20T19:11:26.799554+0000","last_undegraded":"2024-09-20T19:11:26.799554+0000","last_fullsized":"2024-09-20T19:11:26.799554+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:22.919731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:10:22.919731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:30.927422+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1049,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.197453+0000","last_change":"2024-09-20T19:10:51.999987+0000","last_active":"2024-09-20T19:11:26.197453+0000","last_peered":"2024-09-20T19:11:26.197453+0000","last_clean":"2024-09-20T19:11:26.197453+0000","last_became_active":"2024-09-20T19:04:20.914115+0000","last_became_peered":"2024-09-20T19:04:20.914115+0000","last_unstale":"2024-09-20T19:11:26.197453+0000","last_undegraded":"2024-09-20T19:11:26.197453+0000","last_fullsized":"2024-09-20T19:11:26.197453+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:51.999928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:10:51.999928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:09.214826+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"503'21","reported_seq":977,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.801408+0000","last_change":"2024-09-20T19:11:23.540164+0000","last_active":"2024-09-20T19:11:26.801408+0000","last_peered":"2024-09-20T19:11:26.801408+0000","last_clean":"2024-09-20T19:11:26.801408+0000","last_became_active":"2024-09-20T19:10:33.071006+0000","last_became_peered":"2024-09-20T19:10:33.071006+0000","last_unstale":"2024-09-20T19:11:26.801408+0000","last_undegraded":"2024-09-20T19:11:26.801408+0000","last_fullsized":"2024-09-20T19:11:26.801408+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'21","last_scrub_stamp":"2024-09-20T19:11:23.540096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:11:23.540096+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:39.554901+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1046,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.197284+0000","last_change":"2024-09-20T19:10:21.950929+0000","last_active":"2024-09-20T19:11:26.197284+0000","last_peered":"2024-09-20T19:11:26.197284+0000","last_clean":"2024-09-20T19:11:26.197284+0000","last_became_active":"2024-09-20T19:04:20.893250+0000","last_became_peered":"2024-09-20T19:04:20.893250+0000","last_unstale":"2024-09-20T19:11:26.197284+0000","last_undegraded":"2024-09-20T19:11:26.197284+0000","last_fullsized":"2024-09-20T19:11:26.197284+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:21.950869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:10:21.950869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:51.444258+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1013,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800176+0000","last_change":"2024-09-20T19:11:25.266739+0000","last_active":"2024-09-20T19:11:26.800176+0000","last_peered":"2024-09-20T19:11:26.800176+0000","last_clean":"2024-09-20T19:11:26.800176+0000","last_became_active":"2024-09-20T18:49:49.961947+0000","last_became_peered":"2024-09-20T18:49:49.961947+0000","last_unstale":"2024-09-20T19:11:26.800176+0000","last_undegraded":"2024-09-20T19:11:26.800176+0000","last_fullsized":"2024-09-20T19:11:26.800176+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:25.266691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:11:25.266691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:26.413210+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"66'1","reported_seq":1057,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.350244+0000","last_change":"2024-09-20T19:11:25.472705+0000","last_active":"2024-09-20T19:11:31.350244+0000","last_peered":"2024-09-20T19:11:31.350244+0000","last_clean":"2024-09-20T19:11:31.350244+0000","last_became_active":"2024-09-20T19:04:20.892236+0000","last_became_peered":"2024-09-20T19:04:20.892236+0000","last_unstale":"2024-09-20T19:11:31.350244+0000","last_undegraded":"2024-09-20T19:11:31.350244+0000","last_fullsized":"2024-09-20T19:11:31.350244+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-20T19:11:25.472652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:11:25.472652+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:47.401565+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"503'21","reported_seq":1036,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800573+0000","last_change":"2024-09-20T19:10:25.840456+0000","last_active":"2024-09-20T19:11:26.800573+0000","last_peered":"2024-09-20T19:11:26.800573+0000","last_clean":"2024-09-20T19:11:26.800573+0000","last_became_active":"2024-09-20T18:49:48.501011+0000","last_became_peered":"2024-09-20T18:49:48.501011+0000","last_unstale":"2024-09-20T19:11:26.800573+0000","last_undegraded":"2024-09-20T19:11:26.800573+0000","last_fullsized":"2024-09-20T19:11:26.800573+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'21","last_scrub_stamp":"2024-09-20T19:10:25.840352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:10:25.840352+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:37.655279+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":951,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.799608+0000","last_change":"2024-09-20T19:10:35.394841+0000","last_active":"2024-09-20T19:11:26.799608+0000","last_peered":"2024-09-20T19:11:26.799608+0000","last_clean":"2024-09-20T19:11:26.799608+0000","last_became_active":"2024-09-20T19:10:33.067597+0000","last_became_peered":"2024-09-20T19:10:33.067597+0000","last_unstale":"2024-09-20T19:11:26.799608+0000","last_undegraded":"2024-09-20T19:11:26.799608+0000","last_fullsized":"2024-09-20T19:11:26.799608+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:35.394792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:10:35.394792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:56.263119+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":1283,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.799459+0000","last_change":"2024-09-20T19:11:19.298125+0000","last_active":"2024-09-20T19:11:26.799459+0000","last_peered":"2024-09-20T19:11:26.799459+0000","last_clean":"2024-09-20T19:11:26.799459+0000","last_became_active":"2024-09-20T18:49:49.960550+0000","last_became_peered":"2024-09-20T18:49:49.960550+0000","last_unstale":"2024-09-20T19:11:26.799459+0000","last_undegraded":"2024-09-20T19:11:26.799459+0000","last_fullsized":"2024-09-20T19:11:26.799459+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-20T19:11:19.298074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:11:19.298074+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:34.913656+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":345,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.004044+0000","last_change":"2024-09-20T19:11:15.056390+0000","last_active":"2024-09-20T19:11:26.004044+0000","last_peered":"2024-09-20T19:11:26.004044+0000","last_clean":"2024-09-20T19:11:26.004044+0000","last_became_active":"2024-09-20T19:04:20.884100+0000","last_became_peered":"2024-09-20T19:04:20.884100+0000","last_unstale":"2024-09-20T19:11:26.004044+0000","last_undegraded":"2024-09-20T19:11:26.004044+0000","last_fullsized":"2024-09-20T19:11:26.004044+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:15.056336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:11:15.056336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:27.612026+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"503'30","reported_seq":1058,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800067+0000","last_change":"2024-09-20T19:10:57.119244+0000","last_active":"2024-09-20T19:11:26.800067+0000","last_peered":"2024-09-20T19:11:26.800067+0000","last_clean":"2024-09-20T19:11:26.800067+0000","last_became_active":"2024-09-20T18:49:58.997494+0000","last_became_peered":"2024-09-20T18:49:58.997494+0000","last_unstale":"2024-09-20T19:11:26.800067+0000","last_undegraded":"2024-09-20T19:11:26.800067+0000","last_fullsized":"2024-09-20T19:11:26.800067+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'30","last_scrub_stamp":"2024-09-20T19:10:57.119185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:10:57.119185+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:18.560447+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1024,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.197454+0000","last_change":"2024-09-20T19:09:49.959358+0000","last_active":"2024-09-20T19:11:26.197454+0000","last_peered":"2024-09-20T19:11:26.197454+0000","last_clean":"2024-09-20T19:11:26.197454+0000","last_became_active":"2024-09-20T19:04:20.884582+0000","last_became_peered":"2024-09-20T19:04:20.884582+0000","last_unstale":"2024-09-20T19:11:26.197454+0000","last_undegraded":"2024-09-20T19:11:26.197454+0000","last_fullsized":"2024-09-20T19:11:26.197454+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:09:49.959277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:09:49.959277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"511'10","reported_seq":1288,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800263+0000","last_change":"2024-09-20T19:10:50.984705+0000","last_active":"2024-09-20T19:11:26.800263+0000","last_peered":"2024-09-20T19:11:26.800263+0000","last_clean":"2024-09-20T19:11:26.800263+0000","last_became_active":"2024-09-20T19:10:33.056096+0000","last_became_peered":"2024-09-20T19:10:33.056096+0000","last_unstale":"2024-09-20T19:11:26.800263+0000","last_undegraded":"2024-09-20T19:11:26.800263+0000","last_fullsized":"2024-09-20T19:11:26.800263+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"511'10","last_scrub_stamp":"2024-09-20T19:10:50.984636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:10:50.984636+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:57.528281+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":972,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.801327+0000","last_change":"2024-09-20T19:10:33.071644+0000","last_active":"2024-09-20T19:11:26.801327+0000","last_peered":"2024-09-20T19:11:26.801327+0000","last_clean":"2024-09-20T19:11:26.801327+0000","last_became_active":"2024-09-20T19:10:33.069634+0000","last_became_peered":"2024-09-20T19:10:33.069634+0000","last_unstale":"2024-09-20T19:11:26.801327+0000","last_undegraded":"2024-09-20T19:11:26.801327+0000","last_fullsized":"2024-09-20T19:11:26.801327+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:09:11.549521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:09:11.549521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"503'27","reported_seq":370,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.003401+0000","last_change":"2024-09-20T19:10:22.810122+0000","last_active":"2024-09-20T19:11:26.003401+0000","last_peered":"2024-09-20T19:11:26.003401+0000","last_clean":"2024-09-20T19:11:26.003401+0000","last_became_active":"2024-09-20T19:04:20.897330+0000","last_became_peered":"2024-09-20T19:04:20.897330+0000","last_unstale":"2024-09-20T19:11:26.003401+0000","last_undegraded":"2024-09-20T19:11:26.003401+0000","last_fullsized":"2024-09-20T19:11:26.003401+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'27","last_scrub_stamp":"2024-09-20T19:10:22.810038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:10:22.810038+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:50.390035+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":36,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":346,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.003504+0000","last_change":"2024-09-20T19:11:21.078463+0000","last_active":"2024-09-20T19:11:26.003504+0000","last_peered":"2024-09-20T19:11:26.003504+0000","last_clean":"2024-09-20T19:11:26.003504+0000","last_became_active":"2024-09-20T19:04:20.896007+0000","last_became_peered":"2024-09-20T19:04:20.896007+0000","last_unstale":"2024-09-20T19:11:26.003504+0000","last_undegraded":"2024-09-20T19:11:26.003504+0000","last_fullsized":"2024-09-20T19:11:26.003504+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:21.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:11:21.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:22.732517+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1008,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.351600+0000","last_change":"2024-09-20T19:11:11.460689+0000","last_active":"2024-09-20T19:11:31.351600+0000","last_peered":"2024-09-20T19:11:31.351600+0000","last_clean":"2024-09-20T19:11:31.351600+0000","last_became_active":"2024-09-20T18:49:49.957067+0000","last_became_peered":"2024-09-20T18:49:49.957067+0000","last_unstale":"2024-09-20T19:11:31.351600+0000","last_undegraded":"2024-09-20T19:11:31.351600+0000","last_fullsized":"2024-09-20T19:11:31.351600+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:11.460643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:11:11.460643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:35.547041+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":989,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.801232+0000","last_change":"2024-09-20T19:10:33.957131+0000","last_active":"2024-09-20T19:11:26.801232+0000","last_peered":"2024-09-20T19:11:26.801232+0000","last_clean":"2024-09-20T19:11:26.801232+0000","last_became_active":"2024-09-20T19:10:33.070199+0000","last_became_peered":"2024-09-20T19:10:33.070199+0000","last_unstale":"2024-09-20T19:11:26.801232+0000","last_undegraded":"2024-09-20T19:11:26.801232+0000","last_fullsized":"2024-09-20T19:11:26.801232+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:33.957082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:10:33.957082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:42.894167+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"503'21","reported_seq":1096,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.351752+0000","last_change":"2024-09-20T19:10:42.453779+0000","last_active":"2024-09-20T19:11:31.351752+0000","last_peered":"2024-09-20T19:11:31.351752+0000","last_clean":"2024-09-20T19:11:31.351752+0000","last_became_active":"2024-09-20T18:59:05.085230+0000","last_became_peered":"2024-09-20T18:59:05.085230+0000","last_unstale":"2024-09-20T19:11:31.351752+0000","last_undegraded":"2024-09-20T19:11:31.351752+0000","last_fullsized":"2024-09-20T19:11:31.351752+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'21","last_scrub_stamp":"2024-09-20T19:10:42.453714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:10:42.453714+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:12.188538+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1023,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.197753+0000","last_change":"2024-09-20T19:10:19.980466+0000","last_active":"2024-09-20T19:11:26.197753+0000","last_peered":"2024-09-20T19:11:26.197753+0000","last_clean":"2024-09-20T19:11:26.197753+0000","last_became_active":"2024-09-20T19:04:20.884660+0000","last_became_peered":"2024-09-20T19:04:20.884660+0000","last_unstale":"2024-09-20T19:11:26.197753+0000","last_undegraded":"2024-09-20T19:11:26.197753+0000","last_fullsized":"2024-09-20T19:11:26.197753+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:19.980413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:10:19.980413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:48.776512+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1025,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800043+0000","last_change":"2024-09-20T19:10:38.084909+0000","last_active":"2024-09-20T19:11:26.800043+0000","last_peered":"2024-09-20T19:11:26.800043+0000","last_clean":"2024-09-20T19:11:26.800043+0000","last_became_active":"2024-09-20T19:10:33.065411+0000","last_became_peered":"2024-09-20T19:10:33.065411+0000","last_unstale":"2024-09-20T19:11:26.800043+0000","last_undegraded":"2024-09-20T19:11:26.800043+0000","last_fullsized":"2024-09-20T19:11:26.800043+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:38.084679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:10:38.084679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:43.367664+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":965,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.799240+0000","last_change":"2024-09-20T19:10:33.064070+0000","last_active":"2024-09-20T19:11:26.799240+0000","last_peered":"2024-09-20T19:11:26.799240+0000","last_clean":"2024-09-20T19:11:26.799240+0000","last_became_active":"2024-09-20T19:10:33.062950+0000","last_became_peered":"2024-09-20T19:10:33.062950+0000","last_unstale":"2024-09-20T19:11:26.799240+0000","last_undegraded":"2024-09-20T19:11:26.799240+0000","last_fullsized":"2024-09-20T19:11:26.799240+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:09:06.410215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:09:06.410215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"503'27","reported_seq":1091,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.801132+0000","last_change":"2024-09-20T19:11:18.259644+0000","last_active":"2024-09-20T19:11:26.801132+0000","last_peered":"2024-09-20T19:11:26.801132+0000","last_clean":"2024-09-20T19:11:26.801132+0000","last_became_active":"2024-09-20T18:59:05.075390+0000","last_became_peered":"2024-09-20T18:59:05.075390+0000","last_unstale":"2024-09-20T19:11:26.801132+0000","last_undegraded":"2024-09-20T19:11:26.801132+0000","last_fullsized":"2024-09-20T19:11:26.801132+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'27","last_scrub_stamp":"2024-09-20T19:11:18.259570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:11:18.259570+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:26.478226+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1022,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800499+0000","last_change":"2024-09-20T19:10:24.887203+0000","last_active":"2024-09-20T19:11:26.800499+0000","last_peered":"2024-09-20T19:11:26.800499+0000","last_clean":"2024-09-20T19:11:26.800499+0000","last_became_active":"2024-09-20T18:49:45.942105+0000","last_became_peered":"2024-09-20T18:49:45.942105+0000","last_unstale":"2024-09-20T19:11:26.800499+0000","last_undegraded":"2024-09-20T19:11:26.800499+0000","last_fullsized":"2024-09-20T19:11:26.800499+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:24.887146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:10:24.887146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:35.744922+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1046,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.272188+0000","last_change":"2024-09-20T19:11:31.272188+0000","last_active":"2024-09-20T19:11:31.272188+0000","last_peered":"2024-09-20T19:11:31.272188+0000","last_clean":"2024-09-20T19:11:31.272188+0000","last_became_active":"2024-09-20T19:04:20.911292+0000","last_became_peered":"2024-09-20T19:04:20.911292+0000","last_unstale":"2024-09-20T19:11:31.272188+0000","last_undegraded":"2024-09-20T19:11:31.272188+0000","last_fullsized":"2024-09-20T19:11:31.272188+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:31.272124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:11:31.272124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:47.689226+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1027,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800191+0000","last_change":"2024-09-20T19:10:51.114754+0000","last_active":"2024-09-20T19:11:26.800191+0000","last_peered":"2024-09-20T19:11:26.800191+0000","last_clean":"2024-09-20T19:11:26.800191+0000","last_became_active":"2024-09-20T19:10:33.066160+0000","last_became_peered":"2024-09-20T19:10:33.066160+0000","last_unstale":"2024-09-20T19:11:26.800191+0000","last_undegraded":"2024-09-20T19:11:26.800191+0000","last_fullsized":"2024-09-20T19:11:26.800191+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:51.114701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:10:51.114701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:58.511628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"503'12","reported_seq":1062,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800634+0000","last_change":"2024-09-20T19:10:14.169742+0000","last_active":"2024-09-20T19:11:26.800634+0000","last_peered":"2024-09-20T19:11:26.800634+0000","last_clean":"2024-09-20T19:11:26.800634+0000","last_became_active":"2024-09-20T18:59:05.076365+0000","last_became_peered":"2024-09-20T18:59:05.076365+0000","last_unstale":"2024-09-20T19:11:26.800634+0000","last_undegraded":"2024-09-20T19:11:26.800634+0000","last_fullsized":"2024-09-20T19:11:26.800634+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'12","last_scrub_stamp":"2024-09-20T19:10:14.169687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:10:14.169687+0000","objects_scrubbed":2,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:29.283499+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":16,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":968,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800323+0000","last_change":"2024-09-20T19:11:05.381449+0000","last_active":"2024-09-20T19:11:26.800323+0000","last_peered":"2024-09-20T19:11:26.800323+0000","last_clean":"2024-09-20T19:11:26.800323+0000","last_became_active":"2024-09-20T19:10:33.067749+0000","last_became_peered":"2024-09-20T19:10:33.067749+0000","last_unstale":"2024-09-20T19:11:26.800323+0000","last_undegraded":"2024-09-20T19:11:26.800323+0000","last_fullsized":"2024-09-20T19:11:26.800323+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:05.381390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:11:05.381390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:10.990453+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":620,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.684272+0000","last_change":"2024-09-20T19:11:31.684272+0000","last_active":"2024-09-20T19:11:31.684272+0000","last_peered":"2024-09-20T19:11:31.684272+0000","last_clean":"2024-09-20T19:11:31.684272+0000","last_became_active":"2024-09-20T19:04:20.900214+0000","last_became_peered":"2024-09-20T19:04:20.900214+0000","last_unstale":"2024-09-20T19:11:31.684272+0000","last_undegraded":"2024-09-20T19:11:31.684272+0000","last_fullsized":"2024-09-20T19:11:31.684272+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:31.684211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:11:31.684211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:45.398104+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1020,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.197150+0000","last_change":"2024-09-20T19:10:23.938694+0000","last_active":"2024-09-20T19:11:26.197150+0000","last_peered":"2024-09-20T19:11:26.197150+0000","last_clean":"2024-09-20T19:11:26.197150+0000","last_became_active":"2024-09-20T19:04:20.885211+0000","last_became_peered":"2024-09-20T19:04:20.885211+0000","last_unstale":"2024-09-20T19:11:26.197150+0000","last_undegraded":"2024-09-20T19:11:26.197150+0000","last_fullsized":"2024-09-20T19:11:26.197150+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:23.938636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:10:23.938636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:28.976214+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1012,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.352091+0000","last_change":"2024-09-20T19:11:16.516786+0000","last_active":"2024-09-20T19:11:31.352091+0000","last_peered":"2024-09-20T19:11:31.352091+0000","last_clean":"2024-09-20T19:11:31.352091+0000","last_became_active":"2024-09-20T19:04:20.890101+0000","last_became_peered":"2024-09-20T19:04:20.890101+0000","last_unstale":"2024-09-20T19:11:31.352091+0000","last_undegraded":"2024-09-20T19:11:31.352091+0000","last_fullsized":"2024-09-20T19:11:31.352091+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:11:16.516732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:50.942419+0000","last_clean_scrub_stamp":"2024-09-20T19:11:16.516732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:25.404643+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1016,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.800899+0000","last_change":"2024-09-20T19:10:50.100813+0000","last_active":"2024-09-20T19:11:26.800899+0000","last_peered":"2024-09-20T19:11:26.800899+0000","last_clean":"2024-09-20T19:11:26.800899+0000","last_became_active":"2024-09-20T19:10:33.072534+0000","last_became_peered":"2024-09-20T19:10:33.072534+0000","last_unstale":"2024-09-20T19:11:26.800899+0000","last_undegraded":"2024-09-20T19:11:26.800899+0000","last_fullsized":"2024-09-20T19:11:26.800899+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:50.100761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:44.925938+0000","last_clean_scrub_stamp":"2024-09-20T19:10:50.100761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:56.699571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"503'27","reported_seq":631,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-20T19:11:31.502808+0000","last_change":"2024-09-20T19:11:24.552622+0000","last_active":"2024-09-20T19:11:31.502808+0000","last_peered":"2024-09-20T19:11:31.502808+0000","last_clean":"2024-09-20T19:11:31.502808+0000","last_became_active":"2024-09-20T18:59:05.089294+0000","last_became_peered":"2024-09-20T18:59:05.089294+0000","last_unstale":"2024-09-20T19:11:31.502808+0000","last_undegraded":"2024-09-20T19:11:31.502808+0000","last_fullsized":"2024-09-20T19:11:31.502808+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"503'27","last_scrub_stamp":"2024-09-20T19:11:24.552545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:46.928922+0000","last_clean_scrub_stamp":"2024-09-20T19:11:24.552545+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:31.353939+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":36,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":940,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-20T19:11:26.801172+0000","last_change":"2024-09-20T19:10:45.301218+0000","last_active":"2024-09-20T19:11:26.801172+0000","last_peered":"2024-09-20T19:11:26.801172+0000","last_clean":"2024-09-20T19:11:26.801172+0000","last_became_active":"2024-09-20T19:10:33.067702+0000","last_became_peered":"2024-09-20T19:10:33.067702+0000","last_unstale":"2024-09-20T19:11:26.801172+0000","last_undegraded":"2024-09-20T19:11:26.801172+0000","last_fullsized":"2024-09-20T19:11:26.801172+0000","mapping_epoch":510,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":511,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T19:10:45.301165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T18:49:48.932694+0000","last_clean_scrub_stamp":"2024-09-20T19:10:45.301165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:49.666319+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1185,"internal_metadata":0},"log_size":93,"ondisk_log_size":93,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1537,"num_read_kb":1294,"num_write":989,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":283,"num_bytes_recovered":4670,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":972,"ondisk_log_size":972,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":3013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1295,"num_read_kb":1290,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":869,"ondisk_log_size":869,"up":33,"acting":33,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266954,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":768292,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":766974,"kb_avail":92976860,"statfs":{"total":95995035648,"available":95208304640,"internally_reserved":0,"allocated":1331200,"data_stored":853209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":785381834},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528593,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":649952,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":648958,"kb_avail":93095200,"statfs":{"total":95995035648,"available":95329484800,"internally_reserved":0,"allocated":995328,"data_stored":531594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":664533446},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757526,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":726892,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":725310,"kb_avail":93018260,"statfs":{"total":95995035648,"available":95250698240,"internally_reserved":0,"allocated":1597440,"data_stored":1115952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":742717893},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":304,"seq":1305670058108,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":464500,"kb_used_data":996,"kb_used_omap":5,"kb_used_meta":463482,"kb_avail":93280652,"statfs":{"total":95995035648,"available":95519387648,"internally_reserved":0,"allocated":1019904,"data_stored":526478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5633,"internal_metadata":474606079},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182690,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":894888,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":893310,"kb_avail":92850264,"statfs":{"total":95995035648,"available":95078670336,"internally_reserved":0,"allocated":1593344,"data_stored":1115332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":914749895},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411623,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":790444,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":789182,"kb_avail":92954708,"statfs":{"total":95995035648,"available":95185620992,"internally_reserved":0,"allocated":1269760,"data_stored":795220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":808122821},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575149,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":862424,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":861374,"kb_avail":92882728,"statfs":{"total":95995035648,"available":95111913472,"internally_reserved":0,"allocated":1052672,"data_stored":533716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":882047429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":510,"seq":2190433320974,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":84136,"kb_used_data":980,"kb_used_omap":4,"kb_used_meta":83131,"kb_avail":93661016,"statfs":{"total":95995035648,"available":95908880384,"internally_reserved":0,"allocated":1003520,"data_stored":525225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4445,"internal_metadata":85126819},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1185,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T19:11:35.894 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-09-20T19:11:35.894 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5 shell --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 -- ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-20T19:11:36.145 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T19:11:36.539 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:36 smithi083 ceph-mon[36260]: from='client.16887 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:11:36.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:36 smithi083 ceph-mon[36260]: pgmap v1103: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-20T19:11:36.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:36 smithi083 ceph-mon[36260]: 3.13 scrub starts 2024-09-20T19:11:36.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:36 smithi083 ceph-mon[36260]: 3.13 scrub ok 2024-09-20T19:11:36.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:36 smithi083 ceph-mon[36260]: 6.0 scrub starts 2024-09-20T19:11:36.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:36 smithi083 ceph-mon[36260]: 6.0 scrub ok 2024-09-20T19:11:36.540 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:36 smithi083 ceph-mon[36260]: osdmap e535: 8 total, 8 up, 8 in 2024-09-20T19:11:36.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:36 smithi083 ceph-mon[29421]: from='client.16887 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:11:36.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:36 smithi083 ceph-mon[29421]: pgmap v1103: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-20T19:11:36.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:36 smithi083 ceph-mon[29421]: 3.13 scrub starts 2024-09-20T19:11:36.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:36 smithi083 ceph-mon[29421]: 3.13 scrub ok 2024-09-20T19:11:36.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:36 smithi083 ceph-mon[29421]: 6.0 scrub starts 2024-09-20T19:11:36.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:36 smithi083 ceph-mon[29421]: 6.0 scrub ok 2024-09-20T19:11:36.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:36 smithi083 ceph-mon[29421]: osdmap e535: 8 total, 8 up, 8 in 2024-09-20T19:11:36.759 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:36 smithi142 ceph-mon[32508]: from='client.16887 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T19:11:36.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:36 smithi142 ceph-mon[32508]: pgmap v1103: 142 pgs: 142 active+clean; 583 KiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-20T19:11:36.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:36 smithi142 ceph-mon[32508]: 3.13 scrub starts 2024-09-20T19:11:36.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:36 smithi142 ceph-mon[32508]: 3.13 scrub ok 2024-09-20T19:11:36.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:36 smithi142 ceph-mon[32508]: 6.0 scrub starts 2024-09-20T19:11:36.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:36 smithi142 ceph-mon[32508]: 6.0 scrub ok 2024-09-20T19:11:36.760 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:36 smithi142 ceph-mon[32508]: osdmap e535: 8 total, 8 up, 8 in 2024-09-20T19:11:37.231 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:11:36 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: [20/Sep/2024:19:11:36] ENGINE Bus STOPPING 2024-09-20T19:11:37.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:37 smithi083 ceph-mon[29421]: 3.16 scrub starts 2024-09-20T19:11:37.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:37 smithi083 ceph-mon[29421]: 3.16 scrub ok 2024-09-20T19:11:37.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:37 smithi083 ceph-mon[29421]: 4.b scrub starts 2024-09-20T19:11:37.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:37 smithi083 ceph-mon[29421]: 4.b scrub ok 2024-09-20T19:11:37.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:37 smithi083 ceph-mon[29421]: from='client.? 172.21.15.83:0/551343849' entity='client.admin' cmd={"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"} : dispatch 2024-09-20T19:11:37.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:37 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd={"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"} : dispatch 2024-09-20T19:11:37.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:37 smithi083 ceph-mon[29421]: from='client.? ' entity='client.admin' cmd='[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]': finished 2024-09-20T19:11:37.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:37 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T19:11:37.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:37 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T19:11:37.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:37 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T19:11:37.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:37 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:11:37.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:37 smithi083 ceph-mon[29421]: 5.18 scrub starts 2024-09-20T19:11:37.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:37 smithi083 ceph-mon[29421]: 5.18 scrub ok 2024-09-20T19:11:37.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:37 smithi083 ceph-mon[29421]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-20T19:11:37.541 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:11:37 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: [20/Sep/2024:19:11:37] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-20T19:11:37.541 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:11:37 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: [20/Sep/2024:19:11:37] ENGINE Bus STOPPED 2024-09-20T19:11:37.541 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:11:37 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: [20/Sep/2024:19:11:37] ENGINE Bus STARTING 2024-09-20T19:11:37.541 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:11:37 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: [20/Sep/2024:19:11:37] ENGINE Serving on http://:::9283 2024-09-20T19:11:37.541 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:11:37 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: [20/Sep/2024:19:11:37] ENGINE Bus STARTED 2024-09-20T19:11:37.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:37 smithi083 ceph-mon[36260]: 3.16 scrub starts 2024-09-20T19:11:37.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:37 smithi083 ceph-mon[36260]: 3.16 scrub ok 2024-09-20T19:11:37.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:37 smithi083 ceph-mon[36260]: 4.b scrub starts 2024-09-20T19:11:37.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:37 smithi083 ceph-mon[36260]: 4.b scrub ok 2024-09-20T19:11:37.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:37 smithi083 ceph-mon[36260]: from='client.? 172.21.15.83:0/551343849' entity='client.admin' cmd={"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"} : dispatch 2024-09-20T19:11:37.541 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:37 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd={"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"} : dispatch 2024-09-20T19:11:37.542 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:37 smithi083 ceph-mon[36260]: from='client.? ' entity='client.admin' cmd='[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]': finished 2024-09-20T19:11:37.542 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:37 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T19:11:37.542 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:37 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T19:11:37.542 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:37 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T19:11:37.542 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:37 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:11:37.542 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:37 smithi083 ceph-mon[36260]: 5.18 scrub starts 2024-09-20T19:11:37.542 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:37 smithi083 ceph-mon[36260]: 5.18 scrub ok 2024-09-20T19:11:37.542 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:37 smithi083 ceph-mon[36260]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-20T19:11:37.599 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-20T19:11:37.615 INFO:tasks.cephadm:Teardown begin 2024-09-20T19:11:37.616 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-20T19:11:37.652 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-20T19:11:37.679 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:37 smithi142 ceph-mon[32508]: 3.16 scrub starts 2024-09-20T19:11:37.679 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:37 smithi142 ceph-mon[32508]: 3.16 scrub ok 2024-09-20T19:11:37.679 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:37 smithi142 ceph-mon[32508]: 4.b scrub starts 2024-09-20T19:11:37.679 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:37 smithi142 ceph-mon[32508]: 4.b scrub ok 2024-09-20T19:11:37.679 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:37 smithi142 ceph-mon[32508]: from='client.? 172.21.15.83:0/551343849' entity='client.admin' cmd={"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"} : dispatch 2024-09-20T19:11:37.679 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:37 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd={"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"} : dispatch 2024-09-20T19:11:37.679 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:37 smithi142 ceph-mon[32508]: from='client.? ' entity='client.admin' cmd='[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]': finished 2024-09-20T19:11:37.680 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:37 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-20T19:11:37.680 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:37 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-20T19:11:37.680 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:37 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-20T19:11:37.680 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:37 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' 2024-09-20T19:11:37.680 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:37 smithi142 ceph-mon[32508]: 5.18 scrub starts 2024-09-20T19:11:37.680 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:37 smithi142 ceph-mon[32508]: 5.18 scrub ok 2024-09-20T19:11:37.680 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:37 smithi142 ceph-mon[32508]: from='mgr.24467 172.21.15.83:0/1113101200' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-20T19:11:37.708 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-20T19:11:37.709 DEBUG:teuthology.orchestra.run.smithi083:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-20T19:11:37.743 DEBUG:teuthology.orchestra.run.smithi142:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-20T19:11:37.788 INFO:tasks.cephadm:Stopping all daemons... 2024-09-20T19:11:37.788 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-20T19:11:37.788 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@mon.a 2024-09-20T19:11:38.009 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:11:37 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug there is no tcmu-runner data available 2024-09-20T19:11:38.086 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:37 smithi083 systemd[1]: Stopping Ceph mon.a for 1ed5a50a-7780-11ef-baf5-efdc52797490... 2024-09-20T19:11:38.087 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:38 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mon-a[29417]: 2024-09-20T19:11:38.075+0000 7f6c0e9d1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-20T19:11:38.087 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:38 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mon-a[29417]: 2024-09-20T19:11:38.075+0000 7f6c0e9d1640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-20T19:11:38.339 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:38 smithi083 podman[203966]: 2024-09-20 19:11:38.138194161 +0000 UTC m=+0.164472280 container died 16f5b979aa6993ebd1c12c4cfb286a1b6ee3b50a802cdb718cb26d9d680ebcd8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mon-a, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-20-1028-88deadb) 2024-09-20T19:11:38.340 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:11:38 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: [20/Sep/2024:19:11:38] ENGINE Bus STOPPING 2024-09-20T19:11:38.789 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:38 smithi083 podman[203966]: 2024-09-20 19:11:38.519244026 +0000 UTC m=+0.545522134 container cleanup 16f5b979aa6993ebd1c12c4cfb286a1b6ee3b50a802cdb718cb26d9d680ebcd8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mon-a, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2) 2024-09-20T19:11:38.790 INFO:journalctl@ceph.mon.a.smithi083.stdout:Sep 20 19:11:38 smithi083 bash[203966]: ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mon-a 2024-09-20T19:11:38.790 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:11:38 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: [20/Sep/2024:19:11:38] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-20T19:11:38.790 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:11:38 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: [20/Sep/2024:19:11:38] ENGINE Bus STOPPED 2024-09-20T19:11:38.790 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:11:38 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: [20/Sep/2024:19:11:38] ENGINE Bus STARTING 2024-09-20T19:11:38.790 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:11:38 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: [20/Sep/2024:19:11:38] ENGINE Serving on http://:::9283 2024-09-20T19:11:38.790 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:11:38 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: [20/Sep/2024:19:11:38] ENGINE Bus STARTED 2024-09-20T19:11:38.921 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@mon.a.service' 2024-09-20T19:11:38.962 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-20T19:11:38.962 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-20T19:11:38.962 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-09-20T19:11:38.962 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@mon.c 2024-09-20T19:11:39.289 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:39 smithi083 systemd[1]: Stopping Ceph mon.c for 1ed5a50a-7780-11ef-baf5-efdc52797490... 2024-09-20T19:11:39.702 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:39 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mon-c[36256]: 2024-09-20T19:11:39.290+0000 7f5d05d08640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-20T19:11:39.702 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:39 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mon-c[36256]: 2024-09-20T19:11:39.290+0000 7f5d05d08640 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-09-20T19:11:39.702 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:39 smithi083 podman[204100]: 2024-09-20 19:11:39.433470479 +0000 UTC m=+0.257037951 container died 2837de37301cefc1c8893b4c3061773c68fd77c27d285be424ef5238bfda0e64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mon-c, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, maintainer=Guillaume Abrioux ) 2024-09-20T19:11:40.039 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:39 smithi083 podman[204100]: 2024-09-20 19:11:39.891293011 +0000 UTC m=+0.714860481 container cleanup 2837de37301cefc1c8893b4c3061773c68fd77c27d285be424ef5238bfda0e64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mon-c, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-guits-main-2024-09-20-1028-88deadb) 2024-09-20T19:11:40.039 INFO:journalctl@ceph.mon.c.smithi083.stdout:Sep 20 19:11:39 smithi083 bash[204100]: ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mon-c 2024-09-20T19:11:40.319 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@mon.c.service' 2024-09-20T19:11:40.354 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-20T19:11:40.354 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-09-20T19:11:40.354 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-09-20T19:11:40.354 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@mon.b 2024-09-20T19:11:40.686 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:40 smithi142 systemd[1]: Stopping Ceph mon.b for 1ed5a50a-7780-11ef-baf5-efdc52797490... 2024-09-20T19:11:41.009 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:40 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mon-b[32504]: 2024-09-20T19:11:40.683+0000 7fa270d38640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-20T19:11:41.009 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:40 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mon-b[32504]: 2024-09-20T19:11:40.683+0000 7fa270d38640 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-09-20T19:11:41.009 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:40 smithi142 podman[153430]: 2024-09-20 19:11:40.806234431 +0000 UTC m=+0.243520261 container died 39a1eb744183509762a450d7f8d4dd0244d65c54e86fa5e1e8f0335a4524b998 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2) 2024-09-20T19:11:41.039 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:11:40 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: ::ffff:172.21.15.142 - - [20/Sep/2024:19:11:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-20T19:11:41.471 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:41 smithi142 podman[153430]: 2024-09-20 19:11:41.25044558 +0000 UTC m=+0.687731404 container cleanup 39a1eb744183509762a450d7f8d4dd0244d65c54e86fa5e1e8f0335a4524b998 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-20T19:11:41.471 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:41 smithi142 bash[153430]: ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mon-b 2024-09-20T19:11:41.471 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 20 19:11:41 smithi142 podman[153442]: 2024-09-20 19:11:41.468869298 +0000 UTC m=+0.660053830 container remove 39a1eb744183509762a450d7f8d4dd0244d65c54e86fa5e1e8f0335a4524b998 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mon-b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2) 2024-09-20T19:11:41.685 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@mon.b.service' 2024-09-20T19:11:41.733 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-20T19:11:41.733 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-09-20T19:11:41.733 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-20T19:11:41.734 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@mgr.y 2024-09-20T19:11:42.039 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:11:41 smithi083 systemd[1]: Stopping Ceph mgr.y for 1ed5a50a-7780-11ef-baf5-efdc52797490... 2024-09-20T19:11:42.399 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:11:42 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: 2024-09-20T19:11:42.037+0000 7f1e2626c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-20T19:11:42.399 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:11:42 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y[29646]: 2024-09-20T19:11:42.037+0000 7f1e2626c640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-20T19:11:42.400 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:11:42 smithi083 podman[204222]: 2024-09-20 19:11:42.068793875 +0000 UTC m=+0.150379550 container died 9a996b91b375bfbd641ea6f030c760cb8c69dfb83105193f6093263cfe478f3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-20T19:11:42.789 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:11:42 smithi083 podman[204222]: 2024-09-20 19:11:42.563633097 +0000 UTC m=+0.645218753 container cleanup 9a996b91b375bfbd641ea6f030c760cb8c69dfb83105193f6093263cfe478f3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:88deadb8082f4a33832e228ccefc59cedba158a5, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2) 2024-09-20T19:11:42.789 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Sep 20 19:11:42 smithi083 bash[204222]: ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-y 2024-09-20T19:11:42.993 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@mgr.y.service' 2024-09-20T19:11:43.028 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-20T19:11:43.028 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-20T19:11:43.028 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-20T19:11:43.029 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@mgr.x 2024-09-20T19:11:43.363 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Sep 20 19:11:43 smithi142 systemd[1]: Stopping Ceph mgr.x for 1ed5a50a-7780-11ef-baf5-efdc52797490... 2024-09-20T19:11:43.641 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Sep 20 19:11:43 smithi142 podman[153551]: 2024-09-20 19:11:43.361158243 +0000 UTC m=+0.136422293 container died 43e8111ef62f266b483275ea4d436cb55241141bae96655c111be6e34c313579 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-20-1028-88deadb) 2024-09-20T19:11:44.009 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Sep 20 19:11:43 smithi142 podman[153551]: 2024-09-20 19:11:43.838837699 +0000 UTC m=+0.614101749 container cleanup 43e8111ef62f266b483275ea4d436cb55241141bae96655c111be6e34c313579 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-x, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, ceph=True, GIT_BRANCH=HEAD) 2024-09-20T19:11:44.010 INFO:journalctl@ceph.mgr.x.smithi142.stdout:Sep 20 19:11:43 smithi142 bash[153551]: ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-mgr-x 2024-09-20T19:11:44.276 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@mgr.x.service' 2024-09-20T19:11:44.311 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-20T19:11:44.311 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-20T19:11:44.311 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-20T19:11:44.311 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@osd.0 2024-09-20T19:11:44.629 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:11:44 smithi083 systemd[1]: Stopping Ceph osd.0 for 1ed5a50a-7780-11ef-baf5-efdc52797490... 2024-09-20T19:11:45.039 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:11:44 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-0[198692]: 2024-09-20T19:11:44.626+0000 7ff81e6a3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-20T19:11:45.039 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:11:44 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-0[198692]: 2024-09-20T19:11:44.626+0000 7ff81e6a3640 -1 osd.0 535 *** Got signal Terminated *** 2024-09-20T19:11:45.039 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:11:44 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-0[198692]: 2024-09-20T19:11:44.626+0000 7ff81e6a3640 -1 osd.0 535 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-20T19:11:50.363 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:11:50 smithi083 podman[204343]: 2024-09-20 19:11:50.103769491 +0000 UTC m=+5.601496835 container died 19fa3e95aff40c22d8d336802cbaa910fbacdbe5538beae132a28290823ce37a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-20T19:11:50.739 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:11:50 smithi083 podman[204343]: 2024-09-20 19:11:50.536148747 +0000 UTC m=+6.033876129 container cleanup 19fa3e95aff40c22d8d336802cbaa910fbacdbe5538beae132a28290823ce37a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-20T19:11:50.739 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:11:50 smithi083 bash[204343]: ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-0 2024-09-20T19:11:51.039 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:11:50 smithi083 podman[204361]: 2024-09-20 19:11:50.737610578 +0000 UTC m=+0.623562888 container remove 19fa3e95aff40c22d8d336802cbaa910fbacdbe5538beae132a28290823ce37a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-20T19:11:51.436 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:11:51 smithi083 podman[204449]: 2024-09-20 19:11:51.056642073 +0000 UTC m=+0.015963953 image pull 441f9a3fe51528ef653b21f42e9d884db6124a0c7d482e865382bdfdb9ec979c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2 2024-09-20T19:11:51.436 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:11:51 smithi083 podman[204449]: 2024-09-20 19:11:51.180348607 +0000 UTC m=+0.139670505 container create 4296d34c4fe5b4d3c9cfe24e9c84696e18addb38c485abcc8e56927c9d9d9fb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-20T19:11:51.789 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:11:51 smithi083 podman[204449]: 2024-09-20 19:11:51.433318842 +0000 UTC m=+0.392640698 container init 4296d34c4fe5b4d3c9cfe24e9c84696e18addb38c485abcc8e56927c9d9d9fb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-20T19:11:51.790 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:11:51 smithi083 podman[204449]: 2024-09-20 19:11:51.437455274 +0000 UTC m=+0.396777133 container start 4296d34c4fe5b4d3c9cfe24e9c84696e18addb38c485abcc8e56927c9d9d9fb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True) 2024-09-20T19:11:51.790 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:11:51 smithi083 podman[204449]: 2024-09-20 19:11:51.507825671 +0000 UTC m=+0.467147541 container attach 4296d34c4fe5b4d3c9cfe24e9c84696e18addb38c485abcc8e56927c9d9d9fb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-0-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-20T19:11:51.790 INFO:journalctl@ceph.osd.0.smithi083.stdout:Sep 20 19:11:51 smithi083 podman[204449]: 2024-09-20 19:11:51.621281657 +0000 UTC m=+0.580603526 container died 4296d34c4fe5b4d3c9cfe24e9c84696e18addb38c485abcc8e56927c9d9d9fb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-20T19:11:52.305 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@osd.0.service' 2024-09-20T19:11:52.356 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-20T19:11:52.356 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-20T19:11:52.356 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-20T19:11:52.357 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@osd.1 2024-09-20T19:11:52.539 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 20 19:11:52 smithi083 systemd[1]: Stopping Ceph osd.1 for 1ed5a50a-7780-11ef-baf5-efdc52797490... 2024-09-20T19:11:53.039 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 20 19:11:52 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-1[45160]: 2024-09-20T19:11:52.631+0000 7fa1faee4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-20T19:11:53.039 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 20 19:11:52 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-1[45160]: 2024-09-20T19:11:52.631+0000 7fa1faee4640 -1 osd.1 535 *** Got signal Terminated *** 2024-09-20T19:11:53.039 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 20 19:11:52 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-1[45160]: 2024-09-20T19:11:52.631+0000 7fa1faee4640 -1 osd.1 535 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-20T19:11:58.539 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 20 19:11:58 smithi083 podman[204555]: 2024-09-20 19:11:58.235097408 +0000 UTC m=+5.695313048 container died e849695644d03e8d3125c05664234735fc8ef917651444dab278a3725553e45f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-1, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-20T19:11:59.002 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 20 19:11:58 smithi083 podman[204555]: 2024-09-20 19:11:58.715523536 +0000 UTC m=+6.175739220 container cleanup e849695644d03e8d3125c05664234735fc8ef917651444dab278a3725553e45f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-1, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-20T19:11:59.002 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 20 19:11:58 smithi083 bash[204555]: ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-1 2024-09-20T19:11:59.289 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 20 19:11:59 smithi083 podman[204582]: 2024-09-20 19:11:59.00061488 +0000 UTC m=+0.760402945 container remove e849695644d03e8d3125c05664234735fc8ef917651444dab278a3725553e45f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-1, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-20T19:11:59.767 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 20 19:11:59 smithi083 podman[204673]: 2024-09-20 19:11:59.326606439 +0000 UTC m=+0.108061992 container create aa21b09ba756c823beba021eb7e08d00d4ce20a81a6ea50f093f52dc8fef3fc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-20T19:11:59.767 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 20 19:11:59 smithi083 podman[204673]: 2024-09-20 19:11:59.234636525 +0000 UTC m=+0.016092032 image pull 441f9a3fe51528ef653b21f42e9d884db6124a0c7d482e865382bdfdb9ec979c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2 2024-09-20T19:11:59.768 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 20 19:11:59 smithi083 podman[204673]: 2024-09-20 19:11:59.567756443 +0000 UTC m=+0.349211954 container init aa21b09ba756c823beba021eb7e08d00d4ce20a81a6ea50f093f52dc8fef3fc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-20T19:11:59.768 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 20 19:11:59 smithi083 podman[204673]: 2024-09-20 19:11:59.5711778 +0000 UTC m=+0.352633307 container start aa21b09ba756c823beba021eb7e08d00d4ce20a81a6ea50f093f52dc8fef3fc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-1-deactivate, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-20T19:12:00.039 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 20 19:11:59 smithi083 podman[204673]: 2024-09-20 19:11:59.82102567 +0000 UTC m=+0.602481193 container attach aa21b09ba756c823beba021eb7e08d00d4ce20a81a6ea50f093f52dc8fef3fc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-1-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-20T19:12:00.039 INFO:journalctl@ceph.osd.1.smithi083.stdout:Sep 20 19:11:59 smithi083 podman[204673]: 2024-09-20 19:11:59.821219334 +0000 UTC m=+0.602674839 container died aa21b09ba756c823beba021eb7e08d00d4ce20a81a6ea50f093f52dc8fef3fc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-1-deactivate, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-20T19:12:00.399 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@osd.1.service' 2024-09-20T19:12:00.440 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-20T19:12:00.441 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-20T19:12:00.441 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-20T19:12:00.441 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@osd.2 2024-09-20T19:12:00.772 INFO:journalctl@ceph.osd.2.smithi083.stdout:Sep 20 19:12:00 smithi083 systemd[1]: Stopping Ceph osd.2 for 1ed5a50a-7780-11ef-baf5-efdc52797490... 2024-09-20T19:12:01.039 INFO:journalctl@ceph.osd.2.smithi083.stdout:Sep 20 19:12:00 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-2[50848]: 2024-09-20T19:12:00.768+0000 7fd68ab9e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-20T19:12:01.039 INFO:journalctl@ceph.osd.2.smithi083.stdout:Sep 20 19:12:00 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-2[50848]: 2024-09-20T19:12:00.768+0000 7fd68ab9e640 -1 osd.2 535 *** Got signal Terminated *** 2024-09-20T19:12:01.039 INFO:journalctl@ceph.osd.2.smithi083.stdout:Sep 20 19:12:00 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-2[50848]: 2024-09-20T19:12:00.768+0000 7fd68ab9e640 -1 osd.2 535 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-20T19:12:02.259 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 19:12:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[66989]: ts=2024-09-20T19:12:01.829Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=smb msg="Unable to refresh target groups" err="Get \"http://172.21.15.83:8765/sd/prometheus/sd-config?service=smb\": dial tcp 172.21.15.83:8765: connect: connection refused" 2024-09-20T19:12:02.259 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 19:12:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[66989]: ts=2024-09-20T19:12:01.829Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=nfs msg="Unable to refresh target groups" err="Get \"http://172.21.15.83:8765/sd/prometheus/sd-config?service=nfs\": dial tcp 172.21.15.83:8765: connect: connection refused" 2024-09-20T19:12:02.260 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 19:12:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[66989]: ts=2024-09-20T19:12:01.831Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=nvmeof msg="Unable to refresh target groups" err="Get \"http://172.21.15.83:8765/sd/prometheus/sd-config?service=nvmeof\": dial tcp 172.21.15.83:8765: connect: connection refused" 2024-09-20T19:12:02.260 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 19:12:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[66989]: ts=2024-09-20T19:12:01.833Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=ceph msg="Unable to refresh target groups" err="Get \"http://172.21.15.83:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.83:8765: connect: connection refused" 2024-09-20T19:12:02.260 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 19:12:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[66989]: ts=2024-09-20T19:12:01.833Z caller=refresh.go:90 level=error component="discovery manager notify" discovery=http config=config-0 msg="Unable to refresh target groups" err="Get \"http://172.21.15.83:8765/sd/prometheus/sd-config?service=alertmanager\": dial tcp 172.21.15.83:8765: connect: connection refused" 2024-09-20T19:12:02.260 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 19:12:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[66989]: ts=2024-09-20T19:12:01.835Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=ceph-exporter msg="Unable to refresh target groups" err="Get \"http://172.21.15.83:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.83:8765: connect: connection refused" 2024-09-20T19:12:02.260 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 19:12:01 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[66989]: ts=2024-09-20T19:12:01.835Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=node msg="Unable to refresh target groups" err="Get \"http://172.21.15.83:8765/sd/prometheus/sd-config?service=node-exporter\": dial tcp 172.21.15.83:8765: connect: connection refused" 2024-09-20T19:12:06.479 INFO:journalctl@ceph.osd.2.smithi083.stdout:Sep 20 19:12:06 smithi083 podman[204775]: 2024-09-20 19:12:06.204420705 +0000 UTC m=+5.540855845 container died 147453217d6662eb1c7bb16a7cdd59c17ad830cfd1937c823e1250b5b4c4122b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-20T19:12:06.789 INFO:journalctl@ceph.osd.2.smithi083.stdout:Sep 20 19:12:06 smithi083 podman[204775]: 2024-09-20 19:12:06.644027617 +0000 UTC m=+5.980462754 container cleanup 147453217d6662eb1c7bb16a7cdd59c17ad830cfd1937c823e1250b5b4c4122b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-20T19:12:06.789 INFO:journalctl@ceph.osd.2.smithi083.stdout:Sep 20 19:12:06 smithi083 bash[204775]: ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-2 2024-09-20T19:12:07.235 INFO:journalctl@ceph.osd.2.smithi083.stdout:Sep 20 19:12:06 smithi083 podman[204792]: 2024-09-20 19:12:06.837392141 +0000 UTC m=+0.626746868 container remove 147453217d6662eb1c7bb16a7cdd59c17ad830cfd1937c823e1250b5b4c4122b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-2, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-20T19:12:07.539 INFO:journalctl@ceph.osd.2.smithi083.stdout:Sep 20 19:12:07 smithi083 podman[204878]: 2024-09-20 19:12:07.135395012 +0000 UTC m=+0.015593604 image pull 441f9a3fe51528ef653b21f42e9d884db6124a0c7d482e865382bdfdb9ec979c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2 2024-09-20T19:12:07.540 INFO:journalctl@ceph.osd.2.smithi083.stdout:Sep 20 19:12:07 smithi083 podman[204878]: 2024-09-20 19:12:07.238339222 +0000 UTC m=+0.118537815 container create 0322364d9a52448addb107cdbeb1cc06a02fd3308b77047a49ef3c78911d3209 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2) 2024-09-20T19:12:07.540 INFO:journalctl@ceph.osd.2.smithi083.stdout:Sep 20 19:12:07 smithi083 podman[204878]: 2024-09-20 19:12:07.457296802 +0000 UTC m=+0.337495391 container init 0322364d9a52448addb107cdbeb1cc06a02fd3308b77047a49ef3c78911d3209 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-2-deactivate, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS) 2024-09-20T19:12:07.540 INFO:journalctl@ceph.osd.2.smithi083.stdout:Sep 20 19:12:07 smithi083 podman[204878]: 2024-09-20 19:12:07.460571422 +0000 UTC m=+0.340770005 container start 0322364d9a52448addb107cdbeb1cc06a02fd3308b77047a49ef3c78911d3209 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2) 2024-09-20T19:12:07.987 INFO:journalctl@ceph.osd.2.smithi083.stdout:Sep 20 19:12:07 smithi083 podman[204878]: 2024-09-20 19:12:07.557600986 +0000 UTC m=+0.437799580 container attach 0322364d9a52448addb107cdbeb1cc06a02fd3308b77047a49ef3c78911d3209 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-2-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-20T19:12:07.987 INFO:journalctl@ceph.osd.2.smithi083.stdout:Sep 20 19:12:07 smithi083 podman[204878]: 2024-09-20 19:12:07.647537732 +0000 UTC m=+0.527736319 container died 0322364d9a52448addb107cdbeb1cc06a02fd3308b77047a49ef3c78911d3209 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-2-deactivate, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-20T19:12:08.242 INFO:journalctl@ceph.osd.2.smithi083.stdout:Sep 20 19:12:08 smithi083 podman[204902]: 2024-09-20 19:12:08.227838634 +0000 UTC m=+0.574002744 container remove 0322364d9a52448addb107cdbeb1cc06a02fd3308b77047a49ef3c78911d3209 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-20T19:12:08.245 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@osd.2.service' 2024-09-20T19:12:08.285 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-20T19:12:08.285 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-20T19:12:08.286 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-20T19:12:08.286 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@osd.3 2024-09-20T19:12:08.539 INFO:journalctl@ceph.osd.3.smithi083.stdout:Sep 20 19:12:08 smithi083 systemd[1]: Stopping Ceph osd.3 for 1ed5a50a-7780-11ef-baf5-efdc52797490... 2024-09-20T19:12:09.039 INFO:journalctl@ceph.osd.3.smithi083.stdout:Sep 20 19:12:08 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-3[56873]: 2024-09-20T19:12:08.622+0000 7fd87474a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-20T19:12:09.039 INFO:journalctl@ceph.osd.3.smithi083.stdout:Sep 20 19:12:08 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-3[56873]: 2024-09-20T19:12:08.622+0000 7fd87474a640 -1 osd.3 535 *** Got signal Terminated *** 2024-09-20T19:12:09.039 INFO:journalctl@ceph.osd.3.smithi083.stdout:Sep 20 19:12:08 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-3[56873]: 2024-09-20T19:12:08.622+0000 7fd87474a640 -1 osd.3 535 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-20T19:12:12.009 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:11 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:11.752+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:46.668105+0000 front 2024-09-20T19:11:46.667987+0000 (oldest deadline 2024-09-20T19:12:11.367969+0000) 2024-09-20T19:12:12.289 INFO:journalctl@ceph.osd.3.smithi083.stdout:Sep 20 19:12:11 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-3[56873]: 2024-09-20T19:12:11.994+0000 7fd86cd57640 -1 osd.3 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:46.419329+0000 front 2024-09-20T19:11:46.419563+0000 (oldest deadline 2024-09-20T19:12:11.719474+0000) 2024-09-20T19:12:12.997 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:12 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:12.742+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:46.668105+0000 front 2024-09-20T19:11:46.667987+0000 (oldest deadline 2024-09-20T19:12:11.367969+0000) 2024-09-20T19:12:12.997 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:12:12 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4[112539]: 2024-09-20T19:12:12.664+0000 7f7b51ed6640 -1 osd.4 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:46.275004+0000 front 2024-09-20T19:11:46.275058+0000 (oldest deadline 2024-09-20T19:12:12.174932+0000) 2024-09-20T19:12:13.289 INFO:journalctl@ceph.osd.3.smithi083.stdout:Sep 20 19:12:12 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-3[56873]: 2024-09-20T19:12:12.996+0000 7fd86cd57640 -1 osd.3 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:46.419329+0000 front 2024-09-20T19:11:46.419563+0000 (oldest deadline 2024-09-20T19:12:11.719474+0000) 2024-09-20T19:12:14.009 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:13 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:13.754+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:46.668105+0000 front 2024-09-20T19:11:46.667987+0000 (oldest deadline 2024-09-20T19:12:11.367969+0000) 2024-09-20T19:12:14.009 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:12:13 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4[112539]: 2024-09-20T19:12:13.640+0000 7f7b51ed6640 -1 osd.4 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:46.275004+0000 front 2024-09-20T19:11:46.275058+0000 (oldest deadline 2024-09-20T19:12:12.174932+0000) 2024-09-20T19:12:14.532 INFO:journalctl@ceph.osd.3.smithi083.stdout:Sep 20 19:12:14 smithi083 podman[204981]: 2024-09-20 19:12:14.217477738 +0000 UTC m=+5.709001011 container died 4095afe810fe30b52d0cf15d9af0872c40917487f7a0a3d1ca9870185aba3412 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-3, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.build-date=20240731) 2024-09-20T19:12:14.532 INFO:journalctl@ceph.osd.3.smithi083.stdout:Sep 20 19:12:14 smithi083 podman[204981]: 2024-09-20 19:12:14.530482017 +0000 UTC m=+6.022005279 container cleanup 4095afe810fe30b52d0cf15d9af0872c40917487f7a0a3d1ca9870185aba3412 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-3, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-20T19:12:14.639 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:14 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:14.365+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:49.564682+0000 front 2024-09-20T19:11:49.564736+0000 (oldest deadline 2024-09-20T19:12:14.264791+0000) 2024-09-20T19:12:14.789 INFO:journalctl@ceph.osd.3.smithi083.stdout:Sep 20 19:12:14 smithi083 bash[204981]: ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-3 2024-09-20T19:12:14.789 INFO:journalctl@ceph.osd.3.smithi083.stdout:Sep 20 19:12:14 smithi083 podman[204999]: 2024-09-20 19:12:14.740289949 +0000 UTC m=+0.523277403 container remove 4095afe810fe30b52d0cf15d9af0872c40917487f7a0a3d1ca9870185aba3412 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-3, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-20T19:12:15.010 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:14 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:14.783+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:46.668105+0000 front 2024-09-20T19:11:46.667987+0000 (oldest deadline 2024-09-20T19:12:11.367969+0000) 2024-09-20T19:12:15.010 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:12:14 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4[112539]: 2024-09-20T19:12:14.637+0000 7f7b51ed6640 -1 osd.4 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:46.275004+0000 front 2024-09-20T19:11:46.275058+0000 (oldest deadline 2024-09-20T19:12:12.174932+0000) 2024-09-20T19:12:15.445 INFO:journalctl@ceph.osd.3.smithi083.stdout:Sep 20 19:12:15 smithi083 podman[205088]: 2024-09-20 19:12:15.043506919 +0000 UTC m=+0.015531228 image pull 441f9a3fe51528ef653b21f42e9d884db6124a0c7d482e865382bdfdb9ec979c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2 2024-09-20T19:12:15.445 INFO:journalctl@ceph.osd.3.smithi083.stdout:Sep 20 19:12:15 smithi083 podman[205088]: 2024-09-20 19:12:15.149387638 +0000 UTC m=+0.121411939 container create 4410ff31eacb43bbf090f455c2a87c716dc40c8f9749f8425610df46891e3828 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-3-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-20-1028-88deadb) 2024-09-20T19:12:15.445 INFO:journalctl@ceph.osd.3.smithi083.stdout:Sep 20 19:12:15 smithi083 podman[205088]: 2024-09-20 19:12:15.360080424 +0000 UTC m=+0.332104725 container init 4410ff31eacb43bbf090f455c2a87c716dc40c8f9749f8425610df46891e3828 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-3-deactivate, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-20T19:12:15.445 INFO:journalctl@ceph.osd.3.smithi083.stdout:Sep 20 19:12:15 smithi083 podman[205088]: 2024-09-20 19:12:15.363303355 +0000 UTC m=+0.335327675 container start 4410ff31eacb43bbf090f455c2a87c716dc40c8f9749f8425610df46891e3828 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-3-deactivate, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD) 2024-09-20T19:12:15.445 INFO:journalctl@ceph.osd.3.smithi083.stdout:Sep 20 19:12:15 smithi083 podman[205088]: 2024-09-20 19:12:15.443576323 +0000 UTC m=+0.415600647 container attach 4410ff31eacb43bbf090f455c2a87c716dc40c8f9749f8425610df46891e3828 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-3-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-20T19:12:15.509 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:15 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:15.344+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:49.564682+0000 front 2024-09-20T19:11:49.564736+0000 (oldest deadline 2024-09-20T19:12:14.264791+0000) 2024-09-20T19:12:15.509 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 20 19:12:15 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-5[42297]: 2024-09-20T19:12:15.246+0000 7fdd1fc99640 -1 osd.5 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:49.108256+0000 front 2024-09-20T19:11:49.108302+0000 (oldest deadline 2024-09-20T19:12:15.008424+0000) 2024-09-20T19:12:15.789 INFO:journalctl@ceph.osd.3.smithi083.stdout:Sep 20 19:12:15 smithi083 podman[205088]: 2024-09-20 19:12:15.54961264 +0000 UTC m=+0.521636950 container died 4410ff31eacb43bbf090f455c2a87c716dc40c8f9749f8425610df46891e3828 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-3-deactivate, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-20T19:12:16.009 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:15 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:15.771+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:46.668105+0000 front 2024-09-20T19:11:46.667987+0000 (oldest deadline 2024-09-20T19:12:11.367969+0000) 2024-09-20T19:12:16.009 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:12:15 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4[112539]: 2024-09-20T19:12:15.677+0000 7f7b51ed6640 -1 osd.4 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:46.275004+0000 front 2024-09-20T19:11:46.275058+0000 (oldest deadline 2024-09-20T19:12:12.174932+0000) 2024-09-20T19:12:16.264 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@osd.3.service' 2024-09-20T19:12:16.304 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-20T19:12:16.304 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-20T19:12:16.304 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-20T19:12:16.305 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@osd.4 2024-09-20T19:12:16.602 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:12:16 smithi142 systemd[1]: Stopping Ceph osd.4 for 1ed5a50a-7780-11ef-baf5-efdc52797490... 2024-09-20T19:12:16.602 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 20 19:12:16 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-5[42297]: 2024-09-20T19:12:16.280+0000 7fdd1fc99640 -1 osd.5 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:49.108256+0000 front 2024-09-20T19:11:49.108302+0000 (oldest deadline 2024-09-20T19:12:15.008424+0000) 2024-09-20T19:12:16.602 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:16 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:16.314+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:49.564682+0000 front 2024-09-20T19:11:49.564736+0000 (oldest deadline 2024-09-20T19:12:14.264791+0000) 2024-09-20T19:12:17.009 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:16 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:16.805+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:46.668105+0000 front 2024-09-20T19:11:46.667987+0000 (oldest deadline 2024-09-20T19:12:11.367969+0000) 2024-09-20T19:12:17.009 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:12:16 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4[112539]: 2024-09-20T19:12:16.599+0000 7f7b598c9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-20T19:12:17.009 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:12:16 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4[112539]: 2024-09-20T19:12:16.599+0000 7f7b598c9640 -1 osd.4 535 *** Got signal Terminated *** 2024-09-20T19:12:17.009 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:12:16 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4[112539]: 2024-09-20T19:12:16.599+0000 7f7b598c9640 -1 osd.4 535 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-20T19:12:17.009 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:12:16 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4[112539]: 2024-09-20T19:12:16.716+0000 7f7b51ed6640 -1 osd.4 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:46.275004+0000 front 2024-09-20T19:11:46.275058+0000 (oldest deadline 2024-09-20T19:12:12.174932+0000) 2024-09-20T19:12:17.726 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 20 19:12:17 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-5[42297]: 2024-09-20T19:12:17.280+0000 7fdd1fc99640 -1 osd.5 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:49.108256+0000 front 2024-09-20T19:11:49.108302+0000 (oldest deadline 2024-09-20T19:12:15.008424+0000) 2024-09-20T19:12:17.726 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:17 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:17.270+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:49.564682+0000 front 2024-09-20T19:11:49.564736+0000 (oldest deadline 2024-09-20T19:12:14.264791+0000) 2024-09-20T19:12:18.009 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:17 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:17.773+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:46.668105+0000 front 2024-09-20T19:11:46.667987+0000 (oldest deadline 2024-09-20T19:12:11.367969+0000) 2024-09-20T19:12:18.009 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:12:17 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4[112539]: 2024-09-20T19:12:17.724+0000 7f7b51ed6640 -1 osd.4 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:46.275004+0000 front 2024-09-20T19:11:46.275058+0000 (oldest deadline 2024-09-20T19:12:12.174932+0000) 2024-09-20T19:12:18.509 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 20 19:12:18 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-5[42297]: 2024-09-20T19:12:18.251+0000 7fdd1fc99640 -1 osd.5 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:49.108256+0000 front 2024-09-20T19:11:49.108302+0000 (oldest deadline 2024-09-20T19:12:15.008424+0000) 2024-09-20T19:12:18.509 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:18 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:18.300+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:49.564682+0000 front 2024-09-20T19:11:49.564736+0000 (oldest deadline 2024-09-20T19:12:14.264791+0000) 2024-09-20T19:12:19.009 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:12:18 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4[112539]: 2024-09-20T19:12:18.730+0000 7f7b51ed6640 -1 osd.4 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:46.275004+0000 front 2024-09-20T19:11:46.275058+0000 (oldest deadline 2024-09-20T19:12:12.174932+0000) 2024-09-20T19:12:19.009 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:18 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:18.770+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:46.668105+0000 front 2024-09-20T19:11:46.667987+0000 (oldest deadline 2024-09-20T19:12:11.367969+0000) 2024-09-20T19:12:19.010 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:18 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:18.770+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:53.068542+0000 front 2024-09-20T19:11:53.068570+0000 (oldest deadline 2024-09-20T19:12:18.269068+0000) 2024-09-20T19:12:19.729 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 20 19:12:19 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-5[42297]: 2024-09-20T19:12:19.294+0000 7fdd1fc99640 -1 osd.5 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:49.108256+0000 front 2024-09-20T19:11:49.108302+0000 (oldest deadline 2024-09-20T19:12:15.008424+0000) 2024-09-20T19:12:19.729 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:19 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:19.269+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:49.564682+0000 front 2024-09-20T19:11:49.564736+0000 (oldest deadline 2024-09-20T19:12:14.264791+0000) 2024-09-20T19:12:20.009 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:19 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:19.727+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:46.668105+0000 front 2024-09-20T19:11:46.667987+0000 (oldest deadline 2024-09-20T19:12:11.367969+0000) 2024-09-20T19:12:20.009 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:19 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:19.727+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:53.068542+0000 front 2024-09-20T19:11:53.068570+0000 (oldest deadline 2024-09-20T19:12:18.269068+0000) 2024-09-20T19:12:20.010 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:12:19 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4[112539]: 2024-09-20T19:12:19.758+0000 7f7b51ed6640 -1 osd.4 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:46.275004+0000 front 2024-09-20T19:11:46.275058+0000 (oldest deadline 2024-09-20T19:12:12.174932+0000) 2024-09-20T19:12:20.010 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:12:19 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4[112539]: 2024-09-20T19:12:19.758+0000 7f7b51ed6640 -1 osd.4 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:56.776332+0000 front 2024-09-20T19:11:56.776403+0000 (oldest deadline 2024-09-20T19:12:19.676175+0000) 2024-09-20T19:12:20.731 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 20 19:12:20 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-5[42297]: 2024-09-20T19:12:20.331+0000 7fdd1fc99640 -1 osd.5 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:49.108256+0000 front 2024-09-20T19:11:49.108302+0000 (oldest deadline 2024-09-20T19:12:15.008424+0000) 2024-09-20T19:12:20.731 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:20 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:20.315+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:49.564682+0000 front 2024-09-20T19:11:49.564736+0000 (oldest deadline 2024-09-20T19:12:14.264791+0000) 2024-09-20T19:12:21.009 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:12:20 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4[112539]: 2024-09-20T19:12:20.729+0000 7f7b51ed6640 -1 osd.4 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:46.275004+0000 front 2024-09-20T19:11:46.275058+0000 (oldest deadline 2024-09-20T19:12:12.174932+0000) 2024-09-20T19:12:21.009 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:12:20 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4[112539]: 2024-09-20T19:12:20.729+0000 7f7b51ed6640 -1 osd.4 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:56.776332+0000 front 2024-09-20T19:11:56.776403+0000 (oldest deadline 2024-09-20T19:12:19.676175+0000) 2024-09-20T19:12:21.009 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:20 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:20.772+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:46.668105+0000 front 2024-09-20T19:11:46.667987+0000 (oldest deadline 2024-09-20T19:12:11.367969+0000) 2024-09-20T19:12:21.009 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:20 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:20.772+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:53.068542+0000 front 2024-09-20T19:11:53.068570+0000 (oldest deadline 2024-09-20T19:12:18.269068+0000) 2024-09-20T19:12:21.759 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 20 19:12:21 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-5[42297]: 2024-09-20T19:12:21.355+0000 7fdd1fc99640 -1 osd.5 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:49.108256+0000 front 2024-09-20T19:11:49.108302+0000 (oldest deadline 2024-09-20T19:12:15.008424+0000) 2024-09-20T19:12:21.759 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 20 19:12:21 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-5[42297]: 2024-09-20T19:12:21.355+0000 7fdd1fc99640 -1 osd.5 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:55.008893+0000 front 2024-09-20T19:11:55.009011+0000 (oldest deadline 2024-09-20T19:12:20.908853+0000) 2024-09-20T19:12:21.760 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:21 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:21.304+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:49.564682+0000 front 2024-09-20T19:11:49.564736+0000 (oldest deadline 2024-09-20T19:12:14.264791+0000) 2024-09-20T19:12:21.760 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:21 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:21.304+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:57.165806+0000 front 2024-09-20T19:11:57.165881+0000 (oldest deadline 2024-09-20T19:12:21.265786+0000) 2024-09-20T19:12:22.047 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:21 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:21.772+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:46.668105+0000 front 2024-09-20T19:11:46.667987+0000 (oldest deadline 2024-09-20T19:12:11.367969+0000) 2024-09-20T19:12:22.047 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:21 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:21.772+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:53.068542+0000 front 2024-09-20T19:11:53.068570+0000 (oldest deadline 2024-09-20T19:12:18.269068+0000) 2024-09-20T19:12:22.335 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:12:22 smithi142 podman[153694]: 2024-09-20 19:12:22.045261933 +0000 UTC m=+5.534995432 container died 0e1bf4a72b09a3ebdb20ce95b1bf69f073f5863223b2bfded919824418a56a7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-20T19:12:22.670 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:12:22 smithi142 podman[153694]: 2024-09-20 19:12:22.499864471 +0000 UTC m=+5.989597960 container cleanup 0e1bf4a72b09a3ebdb20ce95b1bf69f073f5863223b2bfded919824418a56a7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-20T19:12:22.670 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:12:22 smithi142 bash[153694]: ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4 2024-09-20T19:12:22.670 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 20 19:12:22 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-5[42297]: 2024-09-20T19:12:22.404+0000 7fdd1fc99640 -1 osd.5 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:49.108256+0000 front 2024-09-20T19:11:49.108302+0000 (oldest deadline 2024-09-20T19:12:15.008424+0000) 2024-09-20T19:12:22.670 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 20 19:12:22 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-5[42297]: 2024-09-20T19:12:22.404+0000 7fdd1fc99640 -1 osd.5 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:55.008893+0000 front 2024-09-20T19:11:55.009011+0000 (oldest deadline 2024-09-20T19:12:20.908853+0000) 2024-09-20T19:12:22.670 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:22 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:22.332+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:49.564682+0000 front 2024-09-20T19:11:49.564736+0000 (oldest deadline 2024-09-20T19:12:14.264791+0000) 2024-09-20T19:12:22.671 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:22 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:22.332+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:57.165806+0000 front 2024-09-20T19:11:57.165881+0000 (oldest deadline 2024-09-20T19:12:21.265786+0000) 2024-09-20T19:12:23.009 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:12:22 smithi142 podman[153709]: 2024-09-20 19:12:22.668010992 +0000 UTC m=+0.577548091 container remove 0e1bf4a72b09a3ebdb20ce95b1bf69f073f5863223b2bfded919824418a56a7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-20-1028-88deadb, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-20T19:12:23.009 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:22 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:22.787+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:46.668105+0000 front 2024-09-20T19:11:46.667987+0000 (oldest deadline 2024-09-20T19:12:11.367969+0000) 2024-09-20T19:12:23.009 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:22 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:22.787+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:53.068542+0000 front 2024-09-20T19:11:53.068570+0000 (oldest deadline 2024-09-20T19:12:18.269068+0000) 2024-09-20T19:12:23.322 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:12:23 smithi142 podman[153795]: 2024-09-20 19:12:22.969848735 +0000 UTC m=+0.016051147 image pull 441f9a3fe51528ef653b21f42e9d884db6124a0c7d482e865382bdfdb9ec979c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2 2024-09-20T19:12:23.323 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:12:23 smithi142 podman[153795]: 2024-09-20 19:12:23.094045247 +0000 UTC m=+0.140247655 container create ed10ef2e9991fd9c08606fea41a273f3bb8e5cc1af3df6e0447c7cced3ed828b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4-deactivate, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-20T19:12:23.323 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:12:23 smithi142 podman[153795]: 2024-09-20 19:12:23.271433409 +0000 UTC m=+0.317635816 container init ed10ef2e9991fd9c08606fea41a273f3bb8e5cc1af3df6e0447c7cced3ed828b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2) 2024-09-20T19:12:23.323 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:12:23 smithi142 podman[153795]: 2024-09-20 19:12:23.274776401 +0000 UTC m=+0.320978811 container start ed10ef2e9991fd9c08606fea41a273f3bb8e5cc1af3df6e0447c7cced3ed828b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, GIT_CLEAN=True) 2024-09-20T19:12:23.573 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:12:23 smithi142 podman[153795]: 2024-09-20 19:12:23.354684215 +0000 UTC m=+0.400886628 container attach ed10ef2e9991fd9c08606fea41a273f3bb8e5cc1af3df6e0447c7cced3ed828b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-20T19:12:23.574 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 20 19:12:23 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-5[42297]: 2024-09-20T19:12:23.414+0000 7fdd1fc99640 -1 osd.5 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:49.108256+0000 front 2024-09-20T19:11:49.108302+0000 (oldest deadline 2024-09-20T19:12:15.008424+0000) 2024-09-20T19:12:23.574 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 20 19:12:23 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-5[42297]: 2024-09-20T19:12:23.414+0000 7fdd1fc99640 -1 osd.5 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:55.008893+0000 front 2024-09-20T19:11:55.009011+0000 (oldest deadline 2024-09-20T19:12:20.908853+0000) 2024-09-20T19:12:23.574 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:23 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:23.320+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:49.564682+0000 front 2024-09-20T19:11:49.564736+0000 (oldest deadline 2024-09-20T19:12:14.264791+0000) 2024-09-20T19:12:23.574 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:23 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:23.320+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:57.165806+0000 front 2024-09-20T19:11:57.165881+0000 (oldest deadline 2024-09-20T19:12:21.265786+0000) 2024-09-20T19:12:23.885 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:12:23 smithi142 podman[153795]: 2024-09-20 19:12:23.572812437 +0000 UTC m=+0.619014844 container died ed10ef2e9991fd9c08606fea41a273f3bb8e5cc1af3df6e0447c7cced3ed828b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-20T19:12:23.885 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:23 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:23.786+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:46.668105+0000 front 2024-09-20T19:11:46.667987+0000 (oldest deadline 2024-09-20T19:12:11.367969+0000) 2024-09-20T19:12:23.885 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:23 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:23.786+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:53.068542+0000 front 2024-09-20T19:11:53.068570+0000 (oldest deadline 2024-09-20T19:12:18.269068+0000) 2024-09-20T19:12:24.140 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 20 19:12:24 smithi142 podman[153818]: 2024-09-20 19:12:24.125113298 +0000 UTC m=+0.542175925 container remove ed10ef2e9991fd9c08606fea41a273f3bb8e5cc1af3df6e0447c7cced3ed828b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-20T19:12:24.146 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@osd.4.service' 2024-09-20T19:12:24.189 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-20T19:12:24.189 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-20T19:12:24.189 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-20T19:12:24.190 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@osd.5 2024-09-20T19:12:24.412 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 20 19:12:24 smithi142 systemd[1]: Stopping Ceph osd.5 for 1ed5a50a-7780-11ef-baf5-efdc52797490... 2024-09-20T19:12:24.412 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:24 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:24.355+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:49.564682+0000 front 2024-09-20T19:11:49.564736+0000 (oldest deadline 2024-09-20T19:12:14.264791+0000) 2024-09-20T19:12:24.412 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:24 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:24.355+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:57.165806+0000 front 2024-09-20T19:11:57.165881+0000 (oldest deadline 2024-09-20T19:12:21.265786+0000) 2024-09-20T19:12:24.759 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 20 19:12:24 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-5[42297]: 2024-09-20T19:12:24.424+0000 7fdd1fc99640 -1 osd.5 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:49.108256+0000 front 2024-09-20T19:11:49.108302+0000 (oldest deadline 2024-09-20T19:12:15.008424+0000) 2024-09-20T19:12:24.760 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 20 19:12:24 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-5[42297]: 2024-09-20T19:12:24.424+0000 7fdd1fc99640 -1 osd.5 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:55.008893+0000 front 2024-09-20T19:11:55.009011+0000 (oldest deadline 2024-09-20T19:12:20.908853+0000) 2024-09-20T19:12:24.760 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 20 19:12:24 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-5[42297]: 2024-09-20T19:12:24.536+0000 7fdd2768c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-20T19:12:24.760 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 20 19:12:24 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-5[42297]: 2024-09-20T19:12:24.536+0000 7fdd2768c640 -1 osd.5 535 *** Got signal Terminated *** 2024-09-20T19:12:24.760 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 20 19:12:24 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-5[42297]: 2024-09-20T19:12:24.536+0000 7fdd2768c640 -1 osd.5 535 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-20T19:12:25.259 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:24 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:24.819+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:46.668105+0000 front 2024-09-20T19:11:46.667987+0000 (oldest deadline 2024-09-20T19:12:11.367969+0000) 2024-09-20T19:12:25.259 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:24 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:24.819+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:53.068542+0000 front 2024-09-20T19:11:53.068570+0000 (oldest deadline 2024-09-20T19:12:18.269068+0000) 2024-09-20T19:12:25.759 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 20 19:12:25 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-5[42297]: 2024-09-20T19:12:25.390+0000 7fdd1fc99640 -1 osd.5 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:49.108256+0000 front 2024-09-20T19:11:49.108302+0000 (oldest deadline 2024-09-20T19:12:15.008424+0000) 2024-09-20T19:12:25.759 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 20 19:12:25 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-5[42297]: 2024-09-20T19:12:25.390+0000 7fdd1fc99640 -1 osd.5 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:55.008893+0000 front 2024-09-20T19:11:55.009011+0000 (oldest deadline 2024-09-20T19:12:20.908853+0000) 2024-09-20T19:12:25.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:25 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:25.393+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:49.564682+0000 front 2024-09-20T19:11:49.564736+0000 (oldest deadline 2024-09-20T19:12:14.264791+0000) 2024-09-20T19:12:25.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:25 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:25.393+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:57.165806+0000 front 2024-09-20T19:11:57.165881+0000 (oldest deadline 2024-09-20T19:12:21.265786+0000) 2024-09-20T19:12:26.259 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:25 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:25.827+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:46.668105+0000 front 2024-09-20T19:11:46.667987+0000 (oldest deadline 2024-09-20T19:12:11.367969+0000) 2024-09-20T19:12:26.259 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:25 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:25.827+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:53.068542+0000 front 2024-09-20T19:11:53.068570+0000 (oldest deadline 2024-09-20T19:12:18.269068+0000) 2024-09-20T19:12:26.759 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 20 19:12:26 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-5[42297]: 2024-09-20T19:12:26.435+0000 7fdd1fc99640 -1 osd.5 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:49.108256+0000 front 2024-09-20T19:11:49.108302+0000 (oldest deadline 2024-09-20T19:12:15.008424+0000) 2024-09-20T19:12:26.759 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 20 19:12:26 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-5[42297]: 2024-09-20T19:12:26.435+0000 7fdd1fc99640 -1 osd.5 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:55.008893+0000 front 2024-09-20T19:11:55.009011+0000 (oldest deadline 2024-09-20T19:12:20.908853+0000) 2024-09-20T19:12:26.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:26 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:26.439+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:49.564682+0000 front 2024-09-20T19:11:49.564736+0000 (oldest deadline 2024-09-20T19:12:14.264791+0000) 2024-09-20T19:12:26.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:26 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:26.439+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:57.165806+0000 front 2024-09-20T19:11:57.165881+0000 (oldest deadline 2024-09-20T19:12:21.265786+0000) 2024-09-20T19:12:27.259 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:26 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:26.804+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:46.668105+0000 front 2024-09-20T19:11:46.667987+0000 (oldest deadline 2024-09-20T19:12:11.367969+0000) 2024-09-20T19:12:27.259 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:26 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:26.804+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:53.068542+0000 front 2024-09-20T19:11:53.068570+0000 (oldest deadline 2024-09-20T19:12:18.269068+0000) 2024-09-20T19:12:27.759 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 20 19:12:27 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-5[42297]: 2024-09-20T19:12:27.483+0000 7fdd1fc99640 -1 osd.5 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:49.108256+0000 front 2024-09-20T19:11:49.108302+0000 (oldest deadline 2024-09-20T19:12:15.008424+0000) 2024-09-20T19:12:27.759 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 20 19:12:27 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-5[42297]: 2024-09-20T19:12:27.483+0000 7fdd1fc99640 -1 osd.5 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:55.008893+0000 front 2024-09-20T19:11:55.009011+0000 (oldest deadline 2024-09-20T19:12:20.908853+0000) 2024-09-20T19:12:27.760 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:27 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:27.417+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:49.564682+0000 front 2024-09-20T19:11:49.564736+0000 (oldest deadline 2024-09-20T19:12:14.264791+0000) 2024-09-20T19:12:27.760 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:27 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:27.417+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:57.165806+0000 front 2024-09-20T19:11:57.165881+0000 (oldest deadline 2024-09-20T19:12:21.265786+0000) 2024-09-20T19:12:27.760 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:27 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:27.417+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6811 osd.2 since back 2024-09-20T19:12:03.566836+0000 front 2024-09-20T19:12:03.566793+0000 (oldest deadline 2024-09-20T19:12:27.066675+0000) 2024-09-20T19:12:28.259 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:27 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:27.788+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:46.668105+0000 front 2024-09-20T19:11:46.667987+0000 (oldest deadline 2024-09-20T19:12:11.367969+0000) 2024-09-20T19:12:28.259 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:27 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:27.788+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:53.068542+0000 front 2024-09-20T19:11:53.068570+0000 (oldest deadline 2024-09-20T19:12:18.269068+0000) 2024-09-20T19:12:28.759 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 20 19:12:28 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-5[42297]: 2024-09-20T19:12:28.497+0000 7fdd1fc99640 -1 osd.5 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:49.108256+0000 front 2024-09-20T19:11:49.108302+0000 (oldest deadline 2024-09-20T19:12:15.008424+0000) 2024-09-20T19:12:28.759 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 20 19:12:28 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-5[42297]: 2024-09-20T19:12:28.497+0000 7fdd1fc99640 -1 osd.5 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:55.008893+0000 front 2024-09-20T19:11:55.009011+0000 (oldest deadline 2024-09-20T19:12:20.908853+0000) 2024-09-20T19:12:28.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:28 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:28.425+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:49.564682+0000 front 2024-09-20T19:11:49.564736+0000 (oldest deadline 2024-09-20T19:12:14.264791+0000) 2024-09-20T19:12:28.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:28 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:28.425+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:57.165806+0000 front 2024-09-20T19:11:57.165881+0000 (oldest deadline 2024-09-20T19:12:21.265786+0000) 2024-09-20T19:12:28.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:28 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:28.425+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6811 osd.2 since back 2024-09-20T19:12:03.566836+0000 front 2024-09-20T19:12:03.566793+0000 (oldest deadline 2024-09-20T19:12:27.066675+0000) 2024-09-20T19:12:29.259 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:28 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:28.816+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:46.668105+0000 front 2024-09-20T19:11:46.667987+0000 (oldest deadline 2024-09-20T19:12:11.367969+0000) 2024-09-20T19:12:29.259 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:28 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:28.816+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:53.068542+0000 front 2024-09-20T19:11:53.068570+0000 (oldest deadline 2024-09-20T19:12:18.269068+0000) 2024-09-20T19:12:29.759 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 20 19:12:29 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-5[42297]: 2024-09-20T19:12:29.510+0000 7fdd1fc99640 -1 osd.5 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:49.108256+0000 front 2024-09-20T19:11:49.108302+0000 (oldest deadline 2024-09-20T19:12:15.008424+0000) 2024-09-20T19:12:29.759 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 20 19:12:29 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-5[42297]: 2024-09-20T19:12:29.510+0000 7fdd1fc99640 -1 osd.5 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:55.008893+0000 front 2024-09-20T19:11:55.009011+0000 (oldest deadline 2024-09-20T19:12:20.908853+0000) 2024-09-20T19:12:29.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:29 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:29.378+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:49.564682+0000 front 2024-09-20T19:11:49.564736+0000 (oldest deadline 2024-09-20T19:12:14.264791+0000) 2024-09-20T19:12:29.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:29 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:29.378+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:57.165806+0000 front 2024-09-20T19:11:57.165881+0000 (oldest deadline 2024-09-20T19:12:21.265786+0000) 2024-09-20T19:12:29.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:29 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:29.378+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6811 osd.2 since back 2024-09-20T19:12:03.566836+0000 front 2024-09-20T19:12:03.566793+0000 (oldest deadline 2024-09-20T19:12:27.066675+0000) 2024-09-20T19:12:30.259 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 20 19:12:30 smithi142 podman[153897]: 2024-09-20 19:12:30.061979843 +0000 UTC m=+5.634260398 container died e305514d3002336bc52e4d1717720967560c3a055300d1a5c93a905c5f452bd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-20T19:12:30.259 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:29 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:29.840+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:46.668105+0000 front 2024-09-20T19:11:46.667987+0000 (oldest deadline 2024-09-20T19:12:11.367969+0000) 2024-09-20T19:12:30.259 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:29 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:29.840+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:53.068542+0000 front 2024-09-20T19:11:53.068570+0000 (oldest deadline 2024-09-20T19:12:18.269068+0000) 2024-09-20T19:12:30.759 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 20 19:12:30 smithi142 podman[153897]: 2024-09-20 19:12:30.428154343 +0000 UTC m=+6.000434903 container cleanup e305514d3002336bc52e4d1717720967560c3a055300d1a5c93a905c5f452bd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True) 2024-09-20T19:12:30.759 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 20 19:12:30 smithi142 bash[153897]: ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-5 2024-09-20T19:12:30.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:30 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:30.369+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:49.564682+0000 front 2024-09-20T19:11:49.564736+0000 (oldest deadline 2024-09-20T19:12:14.264791+0000) 2024-09-20T19:12:30.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:30 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:30.369+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:57.165806+0000 front 2024-09-20T19:11:57.165881+0000 (oldest deadline 2024-09-20T19:12:21.265786+0000) 2024-09-20T19:12:30.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:30 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:30.369+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6811 osd.2 since back 2024-09-20T19:12:03.566836+0000 front 2024-09-20T19:12:03.566793+0000 (oldest deadline 2024-09-20T19:12:27.066675+0000) 2024-09-20T19:12:31.259 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 20 19:12:30 smithi142 podman[153925]: 2024-09-20 19:12:30.921602508 +0000 UTC m=+0.852777871 container remove e305514d3002336bc52e4d1717720967560c3a055300d1a5c93a905c5f452bd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-5, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-20T19:12:31.259 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:30 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:30.868+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:46.668105+0000 front 2024-09-20T19:11:46.667987+0000 (oldest deadline 2024-09-20T19:12:11.367969+0000) 2024-09-20T19:12:31.259 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:30 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:30.868+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:53.068542+0000 front 2024-09-20T19:11:53.068570+0000 (oldest deadline 2024-09-20T19:12:18.269068+0000) 2024-09-20T19:12:31.259 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:30 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:30.868+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6811 osd.2 since back 2024-09-20T19:12:01.169925+0000 front 2024-09-20T19:12:01.169842+0000 (oldest deadline 2024-09-20T19:12:29.970115+0000) 2024-09-20T19:12:31.593 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:31 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:31.377+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:49.564682+0000 front 2024-09-20T19:11:49.564736+0000 (oldest deadline 2024-09-20T19:12:14.264791+0000) 2024-09-20T19:12:31.593 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:31 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:31.377+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:57.165806+0000 front 2024-09-20T19:11:57.165881+0000 (oldest deadline 2024-09-20T19:12:21.265786+0000) 2024-09-20T19:12:31.593 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:31 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:31.377+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6811 osd.2 since back 2024-09-20T19:12:03.566836+0000 front 2024-09-20T19:12:03.566793+0000 (oldest deadline 2024-09-20T19:12:27.066675+0000) 2024-09-20T19:12:31.593 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 20 19:12:31 smithi142 podman[154013]: 2024-09-20 19:12:31.226138318 +0000 UTC m=+0.015891496 image pull 441f9a3fe51528ef653b21f42e9d884db6124a0c7d482e865382bdfdb9ec979c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2 2024-09-20T19:12:31.593 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 20 19:12:31 smithi142 podman[154013]: 2024-09-20 19:12:31.36437215 +0000 UTC m=+0.154125317 container create f4ca22bce60ec72119589dc911fb70a28b9bc87943a0c9ecc740c1397660fa7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-20T19:12:31.593 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 20 19:12:31 smithi142 podman[154013]: 2024-09-20 19:12:31.591726247 +0000 UTC m=+0.381479419 container init f4ca22bce60ec72119589dc911fb70a28b9bc87943a0c9ecc740c1397660fa7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-20-1028-88deadb, ceph=True, org.label-schema.license=GPLv2) 2024-09-20T19:12:31.854 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 20 19:12:31 smithi142 podman[154013]: 2024-09-20 19:12:31.595187965 +0000 UTC m=+0.384941140 container start f4ca22bce60ec72119589dc911fb70a28b9bc87943a0c9ecc740c1397660fa7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-5-deactivate, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux ) 2024-09-20T19:12:31.854 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 20 19:12:31 smithi142 podman[154013]: 2024-09-20 19:12:31.691798645 +0000 UTC m=+0.481551820 container attach f4ca22bce60ec72119589dc911fb70a28b9bc87943a0c9ecc740c1397660fa7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-5-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-20T19:12:31.854 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 20 19:12:31 smithi142 podman[154013]: 2024-09-20 19:12:31.787476346 +0000 UTC m=+0.577229520 container died f4ca22bce60ec72119589dc911fb70a28b9bc87943a0c9ecc740c1397660fa7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-5-deactivate, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-20-1028-88deadb, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-20T19:12:32.259 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:31 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:31.851+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:46.668105+0000 front 2024-09-20T19:11:46.667987+0000 (oldest deadline 2024-09-20T19:12:11.367969+0000) 2024-09-20T19:12:32.259 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:31 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:31.851+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:53.068542+0000 front 2024-09-20T19:11:53.068570+0000 (oldest deadline 2024-09-20T19:12:18.269068+0000) 2024-09-20T19:12:32.259 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:31 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:31.851+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6811 osd.2 since back 2024-09-20T19:12:01.169925+0000 front 2024-09-20T19:12:01.169842+0000 (oldest deadline 2024-09-20T19:12:29.970115+0000) 2024-09-20T19:12:32.379 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@osd.5.service' 2024-09-20T19:12:32.418 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-20T19:12:32.419 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-20T19:12:32.419 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-20T19:12:32.419 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@osd.6 2024-09-20T19:12:32.759 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:32 smithi142 systemd[1]: Stopping Ceph osd.6 for 1ed5a50a-7780-11ef-baf5-efdc52797490... 2024-09-20T19:12:32.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:32 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:32.365+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:49.564682+0000 front 2024-09-20T19:11:49.564736+0000 (oldest deadline 2024-09-20T19:12:14.264791+0000) 2024-09-20T19:12:32.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:32 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:32.365+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:57.165806+0000 front 2024-09-20T19:11:57.165881+0000 (oldest deadline 2024-09-20T19:12:21.265786+0000) 2024-09-20T19:12:32.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:32 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:32.365+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6811 osd.2 since back 2024-09-20T19:12:03.566836+0000 front 2024-09-20T19:12:03.566793+0000 (oldest deadline 2024-09-20T19:12:27.066675+0000) 2024-09-20T19:12:33.259 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:32 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:32.756+0000 7fa1d85e1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-20T19:12:33.259 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:32 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:32.756+0000 7fa1d85e1640 -1 osd.6 535 *** Got signal Terminated *** 2024-09-20T19:12:33.259 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:32 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:32.756+0000 7fa1d85e1640 -1 osd.6 535 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-20T19:12:33.259 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:32 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:32.894+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:46.668105+0000 front 2024-09-20T19:11:46.667987+0000 (oldest deadline 2024-09-20T19:12:11.367969+0000) 2024-09-20T19:12:33.259 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:32 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:32.894+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:53.068542+0000 front 2024-09-20T19:11:53.068570+0000 (oldest deadline 2024-09-20T19:12:18.269068+0000) 2024-09-20T19:12:33.259 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:32 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:32.894+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6811 osd.2 since back 2024-09-20T19:12:01.169925+0000 front 2024-09-20T19:12:01.169842+0000 (oldest deadline 2024-09-20T19:12:29.970115+0000) 2024-09-20T19:12:33.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:33 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:33.330+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:49.564682+0000 front 2024-09-20T19:11:49.564736+0000 (oldest deadline 2024-09-20T19:12:14.264791+0000) 2024-09-20T19:12:33.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:33 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:33.330+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:57.165806+0000 front 2024-09-20T19:11:57.165881+0000 (oldest deadline 2024-09-20T19:12:21.265786+0000) 2024-09-20T19:12:33.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:33 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:33.330+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6811 osd.2 since back 2024-09-20T19:12:03.566836+0000 front 2024-09-20T19:12:03.566793+0000 (oldest deadline 2024-09-20T19:12:27.066675+0000) 2024-09-20T19:12:34.259 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:33 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:33.883+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:46.668105+0000 front 2024-09-20T19:11:46.667987+0000 (oldest deadline 2024-09-20T19:12:11.367969+0000) 2024-09-20T19:12:34.259 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:33 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:33.883+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:53.068542+0000 front 2024-09-20T19:11:53.068570+0000 (oldest deadline 2024-09-20T19:12:18.269068+0000) 2024-09-20T19:12:34.259 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:33 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:33.883+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6811 osd.2 since back 2024-09-20T19:12:01.169925+0000 front 2024-09-20T19:12:01.169842+0000 (oldest deadline 2024-09-20T19:12:29.970115+0000) 2024-09-20T19:12:34.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:34 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:34.299+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:49.564682+0000 front 2024-09-20T19:11:49.564736+0000 (oldest deadline 2024-09-20T19:12:14.264791+0000) 2024-09-20T19:12:34.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:34 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:34.299+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:57.165806+0000 front 2024-09-20T19:11:57.165881+0000 (oldest deadline 2024-09-20T19:12:21.265786+0000) 2024-09-20T19:12:34.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:34 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:34.299+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6811 osd.2 since back 2024-09-20T19:12:03.566836+0000 front 2024-09-20T19:12:03.566793+0000 (oldest deadline 2024-09-20T19:12:27.066675+0000) 2024-09-20T19:12:35.259 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:34 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:34.866+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:46.668105+0000 front 2024-09-20T19:11:46.667987+0000 (oldest deadline 2024-09-20T19:12:11.367969+0000) 2024-09-20T19:12:35.259 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:34 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:34.866+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:53.068542+0000 front 2024-09-20T19:11:53.068570+0000 (oldest deadline 2024-09-20T19:12:18.269068+0000) 2024-09-20T19:12:35.259 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:34 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:34.866+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6811 osd.2 since back 2024-09-20T19:12:01.169925+0000 front 2024-09-20T19:12:01.169842+0000 (oldest deadline 2024-09-20T19:12:29.970115+0000) 2024-09-20T19:12:35.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:35 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:35.335+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:49.564682+0000 front 2024-09-20T19:11:49.564736+0000 (oldest deadline 2024-09-20T19:12:14.264791+0000) 2024-09-20T19:12:35.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:35 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:35.335+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:57.165806+0000 front 2024-09-20T19:11:57.165881+0000 (oldest deadline 2024-09-20T19:12:21.265786+0000) 2024-09-20T19:12:35.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:35 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:35.335+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6811 osd.2 since back 2024-09-20T19:12:03.566836+0000 front 2024-09-20T19:12:03.566793+0000 (oldest deadline 2024-09-20T19:12:27.066675+0000) 2024-09-20T19:12:35.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:35 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:35.335+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6815 osd.3 since back 2024-09-20T19:12:08.767528+0000 front 2024-09-20T19:12:08.767585+0000 (oldest deadline 2024-09-20T19:12:34.667531+0000) 2024-09-20T19:12:36.259 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:35 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:35.885+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:46.668105+0000 front 2024-09-20T19:11:46.667987+0000 (oldest deadline 2024-09-20T19:12:11.367969+0000) 2024-09-20T19:12:36.259 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:35 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:35.885+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:53.068542+0000 front 2024-09-20T19:11:53.068570+0000 (oldest deadline 2024-09-20T19:12:18.269068+0000) 2024-09-20T19:12:36.259 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:35 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:35.885+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6811 osd.2 since back 2024-09-20T19:12:01.169925+0000 front 2024-09-20T19:12:01.169842+0000 (oldest deadline 2024-09-20T19:12:29.970115+0000) 2024-09-20T19:12:36.259 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:35 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:35.885+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6815 osd.3 since back 2024-09-20T19:12:09.970561+0000 front 2024-09-20T19:12:09.970582+0000 (oldest deadline 2024-09-20T19:12:35.270480+0000) 2024-09-20T19:12:36.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:36 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:36.313+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:49.564682+0000 front 2024-09-20T19:11:49.564736+0000 (oldest deadline 2024-09-20T19:12:14.264791+0000) 2024-09-20T19:12:36.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:36 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:36.313+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:57.165806+0000 front 2024-09-20T19:11:57.165881+0000 (oldest deadline 2024-09-20T19:12:21.265786+0000) 2024-09-20T19:12:36.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:36 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:36.313+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6811 osd.2 since back 2024-09-20T19:12:03.566836+0000 front 2024-09-20T19:12:03.566793+0000 (oldest deadline 2024-09-20T19:12:27.066675+0000) 2024-09-20T19:12:36.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:36 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:36.313+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6815 osd.3 since back 2024-09-20T19:12:08.767528+0000 front 2024-09-20T19:12:08.767585+0000 (oldest deadline 2024-09-20T19:12:34.667531+0000) 2024-09-20T19:12:37.259 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:36 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:36.851+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:46.668105+0000 front 2024-09-20T19:11:46.667987+0000 (oldest deadline 2024-09-20T19:12:11.367969+0000) 2024-09-20T19:12:37.259 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:36 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:36.851+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:53.068542+0000 front 2024-09-20T19:11:53.068570+0000 (oldest deadline 2024-09-20T19:12:18.269068+0000) 2024-09-20T19:12:37.259 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:36 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:36.851+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6811 osd.2 since back 2024-09-20T19:12:01.169925+0000 front 2024-09-20T19:12:01.169842+0000 (oldest deadline 2024-09-20T19:12:29.970115+0000) 2024-09-20T19:12:37.259 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:36 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6[47791]: 2024-09-20T19:12:36.851+0000 7fa1d4bf6640 -1 osd.6 535 heartbeat_check: no reply from 172.21.15.83:6815 osd.3 since back 2024-09-20T19:12:09.970561+0000 front 2024-09-20T19:12:09.970582+0000 (oldest deadline 2024-09-20T19:12:35.270480+0000) 2024-09-20T19:12:37.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:37 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:37.298+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:49.564682+0000 front 2024-09-20T19:11:49.564736+0000 (oldest deadline 2024-09-20T19:12:14.264791+0000) 2024-09-20T19:12:37.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:37 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:37.298+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:57.165806+0000 front 2024-09-20T19:11:57.165881+0000 (oldest deadline 2024-09-20T19:12:21.265786+0000) 2024-09-20T19:12:37.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:37 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:37.298+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6811 osd.2 since back 2024-09-20T19:12:03.566836+0000 front 2024-09-20T19:12:03.566793+0000 (oldest deadline 2024-09-20T19:12:27.066675+0000) 2024-09-20T19:12:37.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:37 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:37.298+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6815 osd.3 since back 2024-09-20T19:12:08.767528+0000 front 2024-09-20T19:12:08.767585+0000 (oldest deadline 2024-09-20T19:12:34.667531+0000) 2024-09-20T19:12:38.509 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:38 smithi142 podman[154115]: 2024-09-20 19:12:38.255534197 +0000 UTC m=+5.612916472 container died 08f1b108ac6a5ca62e9f8a04e9fa79df20974d16fd408a1e9e493776863c9fa7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-20T19:12:38.509 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:38 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:38.296+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:49.564682+0000 front 2024-09-20T19:11:49.564736+0000 (oldest deadline 2024-09-20T19:12:14.264791+0000) 2024-09-20T19:12:38.509 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:38 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:38.296+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:57.165806+0000 front 2024-09-20T19:11:57.165881+0000 (oldest deadline 2024-09-20T19:12:21.265786+0000) 2024-09-20T19:12:38.509 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:38 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:38.296+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6811 osd.2 since back 2024-09-20T19:12:03.566836+0000 front 2024-09-20T19:12:03.566793+0000 (oldest deadline 2024-09-20T19:12:27.066675+0000) 2024-09-20T19:12:38.509 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:38 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:38.296+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6815 osd.3 since back 2024-09-20T19:12:08.767528+0000 front 2024-09-20T19:12:08.767585+0000 (oldest deadline 2024-09-20T19:12:34.667531+0000) 2024-09-20T19:12:38.783 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:38 smithi142 podman[154115]: 2024-09-20 19:12:38.589446922 +0000 UTC m=+5.946829177 container cleanup 08f1b108ac6a5ca62e9f8a04e9fa79df20974d16fd408a1e9e493776863c9fa7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-20T19:12:38.783 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:38 smithi142 bash[154115]: ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6 2024-09-20T19:12:39.175 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:38 smithi142 podman[154131]: 2024-09-20 19:12:38.781721945 +0000 UTC m=+0.526238691 container remove 08f1b108ac6a5ca62e9f8a04e9fa79df20974d16fd408a1e9e493776863c9fa7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-20T19:12:39.461 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:39 smithi142 podman[154219]: 2024-09-20 19:12:39.174276894 +0000 UTC m=+0.108478513 container create 9669b6a480f2746b7a904d50fc689d27576247db734c7ec0051625aecbfd4cf7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6-deactivate, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-20T19:12:39.461 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:39 smithi142 podman[154219]: 2024-09-20 19:12:39.082171268 +0000 UTC m=+0.016372841 image pull 441f9a3fe51528ef653b21f42e9d884db6124a0c7d482e865382bdfdb9ec979c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2 2024-09-20T19:12:39.461 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:39 smithi142 podman[154219]: 2024-09-20 19:12:39.385281222 +0000 UTC m=+0.319482804 container init 9669b6a480f2746b7a904d50fc689d27576247db734c7ec0051625aecbfd4cf7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-20T19:12:39.461 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:39 smithi142 podman[154219]: 2024-09-20 19:12:39.389039242 +0000 UTC m=+0.323240819 container start 9669b6a480f2746b7a904d50fc689d27576247db734c7ec0051625aecbfd4cf7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-20T19:12:39.462 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:39 smithi142 podman[154219]: 2024-09-20 19:12:39.459814153 +0000 UTC m=+0.394015732 container attach 9669b6a480f2746b7a904d50fc689d27576247db734c7ec0051625aecbfd4cf7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6-deactivate, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=) 2024-09-20T19:12:39.462 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:39 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:39.304+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:49.564682+0000 front 2024-09-20T19:11:49.564736+0000 (oldest deadline 2024-09-20T19:12:14.264791+0000) 2024-09-20T19:12:39.462 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:39 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:39.304+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:57.165806+0000 front 2024-09-20T19:11:57.165881+0000 (oldest deadline 2024-09-20T19:12:21.265786+0000) 2024-09-20T19:12:39.462 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:39 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:39.304+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6811 osd.2 since back 2024-09-20T19:12:03.566836+0000 front 2024-09-20T19:12:03.566793+0000 (oldest deadline 2024-09-20T19:12:27.066675+0000) 2024-09-20T19:12:39.462 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:39 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:39.304+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6815 osd.3 since back 2024-09-20T19:12:08.767528+0000 front 2024-09-20T19:12:08.767585+0000 (oldest deadline 2024-09-20T19:12:34.667531+0000) 2024-09-20T19:12:39.759 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:39 smithi142 podman[154219]: 2024-09-20 19:12:39.579138751 +0000 UTC m=+0.513340326 container died 9669b6a480f2746b7a904d50fc689d27576247db734c7ec0051625aecbfd4cf7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-20T19:12:40.119 INFO:journalctl@ceph.osd.6.smithi142.stdout:Sep 20 19:12:40 smithi142 podman[154242]: 2024-09-20 19:12:40.096854749 +0000 UTC m=+0.508597171 container remove 9669b6a480f2746b7a904d50fc689d27576247db734c7ec0051625aecbfd4cf7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-6-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-20T19:12:40.125 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@osd.6.service' 2024-09-20T19:12:40.165 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-20T19:12:40.165 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-20T19:12:40.165 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-20T19:12:40.166 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@osd.7 2024-09-20T19:12:40.502 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:40 smithi142 systemd[1]: Stopping Ceph osd.7 for 1ed5a50a-7780-11ef-baf5-efdc52797490... 2024-09-20T19:12:40.502 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:40 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:40.299+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:49.564682+0000 front 2024-09-20T19:11:49.564736+0000 (oldest deadline 2024-09-20T19:12:14.264791+0000) 2024-09-20T19:12:40.502 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:40 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:40.299+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:57.165806+0000 front 2024-09-20T19:11:57.165881+0000 (oldest deadline 2024-09-20T19:12:21.265786+0000) 2024-09-20T19:12:40.502 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:40 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:40.299+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6811 osd.2 since back 2024-09-20T19:12:03.566836+0000 front 2024-09-20T19:12:03.566793+0000 (oldest deadline 2024-09-20T19:12:27.066675+0000) 2024-09-20T19:12:40.502 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:40 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:40.299+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6815 osd.3 since back 2024-09-20T19:12:08.767528+0000 front 2024-09-20T19:12:08.767585+0000 (oldest deadline 2024-09-20T19:12:34.667531+0000) 2024-09-20T19:12:40.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:40 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:40.499+0000 7f3765a4d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-20T19:12:40.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:40 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:40.499+0000 7f3765a4d640 -1 osd.7 535 *** Got signal Terminated *** 2024-09-20T19:12:40.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:40 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:40.499+0000 7f3765a4d640 -1 osd.7 535 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-20T19:12:41.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:41 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:41.328+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:49.564682+0000 front 2024-09-20T19:11:49.564736+0000 (oldest deadline 2024-09-20T19:12:14.264791+0000) 2024-09-20T19:12:41.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:41 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:41.328+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:57.165806+0000 front 2024-09-20T19:11:57.165881+0000 (oldest deadline 2024-09-20T19:12:21.265786+0000) 2024-09-20T19:12:41.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:41 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:41.328+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6811 osd.2 since back 2024-09-20T19:12:03.566836+0000 front 2024-09-20T19:12:03.566793+0000 (oldest deadline 2024-09-20T19:12:27.066675+0000) 2024-09-20T19:12:41.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:41 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:41.328+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6815 osd.3 since back 2024-09-20T19:12:08.767528+0000 front 2024-09-20T19:12:08.767585+0000 (oldest deadline 2024-09-20T19:12:34.667531+0000) 2024-09-20T19:12:42.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:42 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:42.319+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:49.564682+0000 front 2024-09-20T19:11:49.564736+0000 (oldest deadline 2024-09-20T19:12:14.264791+0000) 2024-09-20T19:12:42.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:42 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:42.319+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:57.165806+0000 front 2024-09-20T19:11:57.165881+0000 (oldest deadline 2024-09-20T19:12:21.265786+0000) 2024-09-20T19:12:42.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:42 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:42.319+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6811 osd.2 since back 2024-09-20T19:12:03.566836+0000 front 2024-09-20T19:12:03.566793+0000 (oldest deadline 2024-09-20T19:12:27.066675+0000) 2024-09-20T19:12:42.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:42 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:42.319+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6815 osd.3 since back 2024-09-20T19:12:08.767528+0000 front 2024-09-20T19:12:08.767585+0000 (oldest deadline 2024-09-20T19:12:34.667531+0000) 2024-09-20T19:12:43.567 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:43 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:43.306+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:49.564682+0000 front 2024-09-20T19:11:49.564736+0000 (oldest deadline 2024-09-20T19:12:14.264791+0000) 2024-09-20T19:12:43.567 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:43 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:43.306+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:57.165806+0000 front 2024-09-20T19:11:57.165881+0000 (oldest deadline 2024-09-20T19:12:21.265786+0000) 2024-09-20T19:12:43.567 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:43 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:43.306+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6811 osd.2 since back 2024-09-20T19:12:03.566836+0000 front 2024-09-20T19:12:03.566793+0000 (oldest deadline 2024-09-20T19:12:27.066675+0000) 2024-09-20T19:12:43.567 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:43 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:43.306+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6815 osd.3 since back 2024-09-20T19:12:08.767528+0000 front 2024-09-20T19:12:08.767585+0000 (oldest deadline 2024-09-20T19:12:34.667531+0000) 2024-09-20T19:12:44.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:44 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:44.261+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:49.564682+0000 front 2024-09-20T19:11:49.564736+0000 (oldest deadline 2024-09-20T19:12:14.264791+0000) 2024-09-20T19:12:44.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:44 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:44.261+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:57.165806+0000 front 2024-09-20T19:11:57.165881+0000 (oldest deadline 2024-09-20T19:12:21.265786+0000) 2024-09-20T19:12:44.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:44 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:44.261+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6811 osd.2 since back 2024-09-20T19:12:03.566836+0000 front 2024-09-20T19:12:03.566793+0000 (oldest deadline 2024-09-20T19:12:27.066675+0000) 2024-09-20T19:12:44.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:44 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:44.261+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6815 osd.3 since back 2024-09-20T19:12:08.767528+0000 front 2024-09-20T19:12:08.767585+0000 (oldest deadline 2024-09-20T19:12:34.667531+0000) 2024-09-20T19:12:45.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:45 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:45.287+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6803 osd.0 since back 2024-09-20T19:11:49.564682+0000 front 2024-09-20T19:11:49.564736+0000 (oldest deadline 2024-09-20T19:12:14.264791+0000) 2024-09-20T19:12:45.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:45 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:45.287+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6807 osd.1 since back 2024-09-20T19:11:57.165806+0000 front 2024-09-20T19:11:57.165881+0000 (oldest deadline 2024-09-20T19:12:21.265786+0000) 2024-09-20T19:12:45.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:45 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:45.287+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6811 osd.2 since back 2024-09-20T19:12:03.566836+0000 front 2024-09-20T19:12:03.566793+0000 (oldest deadline 2024-09-20T19:12:27.066675+0000) 2024-09-20T19:12:45.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:45 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:45.287+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.83:6815 osd.3 since back 2024-09-20T19:12:08.767528+0000 front 2024-09-20T19:12:08.767585+0000 (oldest deadline 2024-09-20T19:12:34.667531+0000) 2024-09-20T19:12:45.759 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:45 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7[53553]: 2024-09-20T19:12:45.287+0000 7f375e05a640 -1 osd.7 535 heartbeat_check: no reply from 172.21.15.142:6802 osd.4 since back 2024-09-20T19:12:19.968486+0000 front 2024-09-20T19:12:19.968597+0000 (oldest deadline 2024-09-20T19:12:44.668458+0000) 2024-09-20T19:12:46.311 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:46 smithi142 podman[154321]: 2024-09-20 19:12:46.046583248 +0000 UTC m=+5.664455877 container died 29146fb4e43e0b7ff608c815661826ab8c1df57d8768dd0a031dd8b40309cccb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-20T19:12:46.668 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:46 smithi142 podman[154321]: 2024-09-20 19:12:46.399468613 +0000 UTC m=+6.017341239 container cleanup 29146fb4e43e0b7ff608c815661826ab8c1df57d8768dd0a031dd8b40309cccb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-20T19:12:46.669 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:46 smithi142 bash[154321]: ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7 2024-09-20T19:12:47.004 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:46 smithi142 podman[154338]: 2024-09-20 19:12:46.667211147 +0000 UTC m=+0.623664838 container remove 29146fb4e43e0b7ff608c815661826ab8c1df57d8768dd0a031dd8b40309cccb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-20T19:12:47.259 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:47 smithi142 podman[154425]: 2024-09-20 19:12:46.905005836 +0000 UTC m=+0.016128432 image pull 441f9a3fe51528ef653b21f42e9d884db6124a0c7d482e865382bdfdb9ec979c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2 2024-09-20T19:12:47.259 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:47 smithi142 podman[154425]: 2024-09-20 19:12:47.009850012 +0000 UTC m=+0.120972605 container create e1997ceb0f00c2cc85534b3d3a898b2c44ebfc31dcedcc18108fc6b9ae551a44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2) 2024-09-20T19:12:47.259 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:47 smithi142 podman[154425]: 2024-09-20 19:12:47.229396509 +0000 UTC m=+0.340519112 container init e1997ceb0f00c2cc85534b3d3a898b2c44ebfc31dcedcc18108fc6b9ae551a44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7-deactivate, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-20T19:12:47.259 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:47 smithi142 podman[154425]: 2024-09-20 19:12:47.232828279 +0000 UTC m=+0.343950875 container start e1997ceb0f00c2cc85534b3d3a898b2c44ebfc31dcedcc18108fc6b9ae551a44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7-deactivate, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-20T19:12:47.709 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:47 smithi142 podman[154425]: 2024-09-20 19:12:47.30413948 +0000 UTC m=+0.415262089 container attach e1997ceb0f00c2cc85534b3d3a898b2c44ebfc31dcedcc18108fc6b9ae551a44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-20-1028-88deadb) 2024-09-20T19:12:47.709 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:47 smithi142 podman[154425]: 2024-09-20 19:12:47.421452277 +0000 UTC m=+0.532574870 container died e1997ceb0f00c2cc85534b3d3a898b2c44ebfc31dcedcc18108fc6b9ae551a44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-20T19:12:47.964 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 20 19:12:47 smithi142 podman[154449]: 2024-09-20 19:12:47.940489334 +0000 UTC m=+0.508255030 container remove e1997ceb0f00c2cc85534b3d3a898b2c44ebfc31dcedcc18108fc6b9ae551a44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-osd-7-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.build-date=20240731) 2024-09-20T19:12:47.970 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@osd.7.service' 2024-09-20T19:12:48.007 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-20T19:12:48.007 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-20T19:12:48.008 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-09-20T19:12:48.008 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@rgw.foo.a 2024-09-20T19:12:48.289 INFO:journalctl@ceph.rgw.foo.a.smithi083.stdout:Sep 20 19:12:48 smithi083 systemd[1]: Stopping Ceph rgw.foo.a for 1ed5a50a-7780-11ef-baf5-efdc52797490... 2024-09-20T19:12:48.789 INFO:journalctl@ceph.rgw.foo.a.smithi083.stdout:Sep 20 19:12:48 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-rgw-foo-a[61173]: 2024-09-20T19:12:48.313+0000 7f140b614640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-20T19:12:48.789 INFO:journalctl@ceph.rgw.foo.a.smithi083.stdout:Sep 20 19:12:48 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-rgw-foo-a[61173]: 2024-09-20T19:12:48.313+0000 7f140ee80880 -1 shutting down 2024-09-20T19:12:58.675 INFO:journalctl@ceph.rgw.foo.a.smithi083.stdout:Sep 20 19:12:58 smithi083 bash[205190]: time="2024-09-20T19:12:58Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-09-20T19:12:58.675 INFO:journalctl@ceph.rgw.foo.a.smithi083.stdout:Sep 20 19:12:58 smithi083 podman[205190]: 2024-09-20 19:12:58.346684442 +0000 UTC m=+10.146623597 container died a80fe9a0c509df35e57ee3fd5ba360c96e33d10cc3deea23c869db363a88a656 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-rgw-foo-a, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-20T19:12:59.042 INFO:journalctl@ceph.rgw.foo.a.smithi083.stdout:Sep 20 19:12:58 smithi083 podman[205190]: 2024-09-20 19:12:58.855107358 +0000 UTC m=+10.655046549 container cleanup a80fe9a0c509df35e57ee3fd5ba360c96e33d10cc3deea23c869db363a88a656 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-rgw-foo-a, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-20T19:12:59.042 INFO:journalctl@ceph.rgw.foo.a.smithi083.stdout:Sep 20 19:12:58 smithi083 bash[205190]: ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-rgw-foo-a 2024-09-20T19:12:59.218 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@rgw.foo.a.service' 2024-09-20T19:12:59.254 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-20T19:12:59.255 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-09-20T19:12:59.255 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-09-20T19:12:59.255 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@prometheus.a 2024-09-20T19:12:59.544 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 19:12:59 smithi142 systemd[1]: Stopping Ceph prometheus.a for 1ed5a50a-7780-11ef-baf5-efdc52797490... 2024-09-20T19:12:59.823 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 19:12:59 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[66989]: ts=2024-09-20T19:12:59.542Z caller=main.go:964 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-20T19:12:59.824 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 19:12:59 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[66989]: ts=2024-09-20T19:12:59.542Z caller=main.go:988 level=info msg="Stopping scrape discovery manager..." 2024-09-20T19:12:59.824 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 19:12:59 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[66989]: ts=2024-09-20T19:12:59.542Z caller=main.go:1002 level=info msg="Stopping notify discovery manager..." 2024-09-20T19:12:59.824 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 19:12:59 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[66989]: ts=2024-09-20T19:12:59.542Z caller=manager.go:177 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-20T19:12:59.824 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 19:12:59 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[66989]: ts=2024-09-20T19:12:59.542Z caller=main.go:984 level=info msg="Scrape discovery manager stopped" 2024-09-20T19:12:59.824 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 19:12:59 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[66989]: ts=2024-09-20T19:12:59.542Z caller=manager.go:187 level=info component="rule manager" msg="Rule manager stopped" 2024-09-20T19:12:59.824 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 19:12:59 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[66989]: ts=2024-09-20T19:12:59.542Z caller=main.go:998 level=info msg="Notify discovery manager stopped" 2024-09-20T19:12:59.824 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 19:12:59 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[66989]: ts=2024-09-20T19:12:59.542Z caller=main.go:1039 level=info msg="Stopping scrape manager..." 2024-09-20T19:12:59.824 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 19:12:59 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[66989]: ts=2024-09-20T19:12:59.542Z caller=main.go:1031 level=info msg="Scrape manager stopped" 2024-09-20T19:12:59.824 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 19:12:59 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[66989]: ts=2024-09-20T19:12:59.543Z caller=notifier.go:618 level=info component=notifier msg="Stopping notification manager..." 2024-09-20T19:12:59.824 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 19:12:59 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[66989]: ts=2024-09-20T19:12:59.543Z caller=main.go:1261 level=info msg="Notifier manager stopped" 2024-09-20T19:12:59.824 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 19:12:59 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a[66989]: ts=2024-09-20T19:12:59.543Z caller=main.go:1273 level=info msg="See you next time!" 2024-09-20T19:12:59.824 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 19:12:59 smithi142 podman[154527]: 2024-09-20 19:12:59.552532798 +0000 UTC m=+0.106533917 container died 790d73aeb9d9c2099db6e1fa2bf1b460ea298eb02a70b14f37a146af9298b73a (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-20T19:13:00.194 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 19:12:59 smithi142 podman[154527]: 2024-09-20 19:12:59.933441227 +0000 UTC m=+0.487442338 container cleanup 790d73aeb9d9c2099db6e1fa2bf1b460ea298eb02a70b14f37a146af9298b73a (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-20T19:13:00.195 INFO:journalctl@ceph.prometheus.a.smithi142.stdout:Sep 20 19:12:59 smithi142 bash[154527]: ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-prometheus-a 2024-09-20T19:13:00.349 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@prometheus.a.service' 2024-09-20T19:13:00.385 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-20T19:13:00.385 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-09-20T19:13:00.385 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-20T19:13:00.574 INFO:teuthology.orchestra.run.smithi083.stdout:Deleting cluster with fsid: 1ed5a50a-7780-11ef-baf5-efdc52797490 2024-09-20T19:13:02.478 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 19:13:02 smithi083 systemd[1]: Stopping Ceph alertmanager.a for 1ed5a50a-7780-11ef-baf5-efdc52797490... 2024-09-20T19:13:02.478 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 19:13:02 smithi083 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-alertmanager-a[76723]: ts=2024-09-20T19:13:02.435Z caller=main.go:542 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-20T19:13:02.789 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 19:13:02 smithi083 podman[205696]: 2024-09-20 19:13:02.476709705 +0000 UTC m=+0.148906718 container died b430eb5089f642f0ecfd2acd70578c45654c89ff3fb2b3a944d95b5ed008c92d (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-20T19:13:03.165 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 19:13:02 smithi083 podman[205696]: 2024-09-20 19:13:02.911190965 +0000 UTC m=+0.583387977 container cleanup b430eb5089f642f0ecfd2acd70578c45654c89ff3fb2b3a944d95b5ed008c92d (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-20T19:13:03.165 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 19:13:02 smithi083 bash[205696]: ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-alertmanager-a 2024-09-20T19:13:03.165 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 19:13:03 smithi083 podman[205708]: 2024-09-20 19:13:03.104273337 +0000 UTC m=+0.630622433 container remove b430eb5089f642f0ecfd2acd70578c45654c89ff3fb2b3a944d95b5ed008c92d (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-20T19:13:03.165 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 19:13:03 smithi083 podman[205708]: 2024-09-20 19:13:03.163818101 +0000 UTC m=+0.690167194 volume remove c060cec2a873706de75405ae24ec4a1c6125b8682cbf27b596fb17f273a8b608 2024-09-20T19:13:03.539 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 19:13:03 smithi083 systemd[1]: ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@alertmanager.a.service: Deactivated successfully. 2024-09-20T19:13:03.539 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 19:13:03 smithi083 systemd[1]: Stopped Ceph alertmanager.a for 1ed5a50a-7780-11ef-baf5-efdc52797490. 2024-09-20T19:13:03.539 INFO:journalctl@ceph.alertmanager.a.smithi083.stdout:Sep 20 19:13:03 smithi083 systemd[1]: ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@alertmanager.a.service: Consumed 1.633s CPU time. 2024-09-20T19:13:03.819 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 19:13:03 smithi083 systemd[1]: Stopping Ceph node-exporter.a for 1ed5a50a-7780-11ef-baf5-efdc52797490... 2024-09-20T19:13:04.094 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 19:13:03 smithi083 podman[205824]: 2024-09-20 19:13:03.818517698 +0000 UTC m=+0.113993771 container died 7f83f79ccb492f2ab13d53dac42b62bae921065d3f46d47bf4d42aaa44eee1ee (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-20T19:13:04.411 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 19:13:04 smithi083 podman[205824]: 2024-09-20 19:13:04.258066369 +0000 UTC m=+0.553542442 container cleanup 7f83f79ccb492f2ab13d53dac42b62bae921065d3f46d47bf4d42aaa44eee1ee (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-20T19:13:04.411 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 19:13:04 smithi083 bash[205824]: ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a 2024-09-20T19:13:04.773 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 19:13:04 smithi083 podman[205836]: 2024-09-20 19:13:04.409554616 +0000 UTC m=+0.590563923 container remove 7f83f79ccb492f2ab13d53dac42b62bae921065d3f46d47bf4d42aaa44eee1ee (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-20T19:13:04.773 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 19:13:04 smithi083 systemd[1]: ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-09-20T19:13:04.773 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 19:13:04 smithi083 systemd[1]: ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@node-exporter.a.service: Failed with result 'exit-code'. 2024-09-20T19:13:04.774 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 19:13:04 smithi083 systemd[1]: Stopped Ceph node-exporter.a for 1ed5a50a-7780-11ef-baf5-efdc52797490. 2024-09-20T19:13:04.774 INFO:journalctl@ceph.node-exporter.a.smithi083.stdout:Sep 20 19:13:04 smithi083 systemd[1]: ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@node-exporter.a.service: Consumed 4.520s CPU time. 2024-09-20T19:13:05.281 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-20T19:13:05.482 INFO:teuthology.orchestra.run.smithi142.stdout:Deleting cluster with fsid: 1ed5a50a-7780-11ef-baf5-efdc52797490 2024-09-20T19:13:07.181 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:13:06 smithi142 systemd[1]: Stopping Ceph iscsi.iscsi.a for 1ed5a50a-7780-11ef-baf5-efdc52797490... 2024-09-20T19:13:07.181 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:13:07 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a[58422]: debug Shutdown received 2024-09-20T19:13:17.382 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:13:17 smithi142 bash[154947]: time="2024-09-20T19:13:17Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a in 10 seconds, resorting to SIGKILL" 2024-09-20T19:13:17.382 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:13:17 smithi142 podman[154947]: 2024-09-20 19:13:17.101844062 +0000 UTC m=+10.121849881 container died 0dd854e354b5fcb4bdca32946a07c57472a6088f7af513a1df16796b2e61714d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-20T19:13:17.758 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:13:17 smithi142 podman[154947]: 2024-09-20 19:13:17.453921749 +0000 UTC m=+10.473927558 container cleanup 0dd854e354b5fcb4bdca32946a07c57472a6088f7af513a1df16796b2e61714d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-20T19:13:17.759 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:13:17 smithi142 bash[154947]: ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a 2024-09-20T19:13:18.232 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:13:17 smithi142 podman[154960]: 2024-09-20 19:13:17.863436702 +0000 UTC m=+0.760115827 container remove 0dd854e354b5fcb4bdca32946a07c57472a6088f7af513a1df16796b2e61714d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8a1a6edbc97129e01d4344a6578add33aaab808082c84049bd2e48e1f97b80d2, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-iscsi-iscsi-a, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-20-1028-88deadb, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2) 2024-09-20T19:13:18.233 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:13:17 smithi142 systemd[1]: ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-09-20T19:13:18.233 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:13:18 smithi142 systemd[1]: ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-09-20T19:13:18.233 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:13:18 smithi142 systemd[1]: Stopped Ceph iscsi.iscsi.a for 1ed5a50a-7780-11ef-baf5-efdc52797490. 2024-09-20T19:13:18.233 INFO:journalctl@ceph.iscsi.iscsi.a.smithi142.stdout:Sep 20 19:13:18 smithi142 systemd[1]: ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@iscsi.iscsi.a.service: Consumed 2.691s CPU time. 2024-09-20T19:13:19.759 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 19:13:19 smithi142 systemd[1]: Stopping Ceph grafana.a for 1ed5a50a-7780-11ef-baf5-efdc52797490... 2024-09-20T19:13:20.022 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 19:13:19 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=server t=2024-09-20T19:13:19.76238189Z level=info msg="Shutdown started" reason="System signal: terminated" 2024-09-20T19:13:20.022 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 19:13:19 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=ticker t=2024-09-20T19:13:19.76271977Z level=info msg=stopped last_tick=2024-09-20T19:13:10Z 2024-09-20T19:13:20.022 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 19:13:19 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=tracing t=2024-09-20T19:13:19.762731029Z level=info msg="Closing tracing" 2024-09-20T19:13:20.022 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 19:13:19 smithi142 ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a[61041]: logger=grafana-apiserver t=2024-09-20T19:13:19.76275861Z level=info msg="StorageObjectCountTracker pruner is exiting" 2024-09-20T19:13:20.022 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 19:13:19 smithi142 podman[155226]: 2024-09-20 19:13:19.772625387 +0000 UTC m=+0.104879870 container died d6156fc1566a6dd842aa6ea09bdc43c5b2782008b443904ba3fd7694a4c7b76d (image=quay.io/ceph/grafana:10.4.8, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a, maintainer=Grafana Labs ) 2024-09-20T19:13:20.322 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 19:13:20 smithi142 podman[155226]: 2024-09-20 19:13:20.09536228 +0000 UTC m=+0.427616758 container cleanup d6156fc1566a6dd842aa6ea09bdc43c5b2782008b443904ba3fd7694a4c7b76d (image=quay.io/ceph/grafana:10.4.8, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a, maintainer=Grafana Labs ) 2024-09-20T19:13:20.323 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 19:13:20 smithi142 bash[155226]: ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a 2024-09-20T19:13:20.738 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 19:13:20 smithi142 podman[155239]: 2024-09-20 19:13:20.321503761 +0000 UTC m=+0.545348216 container remove d6156fc1566a6dd842aa6ea09bdc43c5b2782008b443904ba3fd7694a4c7b76d (image=quay.io/ceph/grafana:10.4.8, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-grafana-a, maintainer=Grafana Labs ) 2024-09-20T19:13:20.738 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 19:13:20 smithi142 systemd[1]: ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@grafana.a.service: Deactivated successfully. 2024-09-20T19:13:20.738 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 19:13:20 smithi142 systemd[1]: Stopped Ceph grafana.a for 1ed5a50a-7780-11ef-baf5-efdc52797490. 2024-09-20T19:13:20.739 INFO:journalctl@ceph.grafana.a.smithi142.stdout:Sep 20 19:13:20 smithi142 systemd[1]: ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@grafana.a.service: Consumed 15.490s CPU time. 2024-09-20T19:13:20.995 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 19:13:20 smithi142 systemd[1]: Stopping Ceph node-exporter.b for 1ed5a50a-7780-11ef-baf5-efdc52797490... 2024-09-20T19:13:21.258 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 19:13:20 smithi142 podman[155354]: 2024-09-20 19:13:20.993792681 +0000 UTC m=+0.119749572 container died 04b8bd889b9399d276f21196e1d787b41e4e3ed213a8ffa2cd8adeedf42ab377 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-20T19:13:21.627 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 19:13:21 smithi142 podman[155354]: 2024-09-20 19:13:21.374763828 +0000 UTC m=+0.500720718 container cleanup 04b8bd889b9399d276f21196e1d787b41e4e3ed213a8ffa2cd8adeedf42ab377 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-20T19:13:21.627 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 19:13:21 smithi142 bash[155354]: ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b 2024-09-20T19:13:21.897 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 19:13:21 smithi142 podman[155369]: 2024-09-20 19:13:21.625879806 +0000 UTC m=+0.636135799 container remove 04b8bd889b9399d276f21196e1d787b41e4e3ed213a8ffa2cd8adeedf42ab377 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-1ed5a50a-7780-11ef-baf5-efdc52797490-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-20T19:13:21.897 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 19:13:21 smithi142 systemd[1]: ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-09-20T19:13:21.898 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 19:13:21 smithi142 systemd[1]: ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@node-exporter.b.service: Failed with result 'exit-code'. 2024-09-20T19:13:21.898 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 19:13:21 smithi142 systemd[1]: Stopped Ceph node-exporter.b for 1ed5a50a-7780-11ef-baf5-efdc52797490. 2024-09-20T19:13:21.898 INFO:journalctl@ceph.node-exporter.b.smithi142.stdout:Sep 20 19:13:21 smithi142 systemd[1]: ceph-1ed5a50a-7780-11ef-baf5-efdc52797490@node-exporter.b.service: Consumed 4.561s CPU time. 2024-09-20T19:13:22.506 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-20T19:13:22.540 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-20T19:13:22.567 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-20T19:13:22.569 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/gabrioux-2024-09-20_12:24:58-orch:cephadm-wip-guits-main-2024-09-20-1028-distro-default-smithi/7912715/remote/smithi083/crash 2024-09-20T19:13:22.570 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/crash -- . 2024-09-20T19:13:22.621 INFO:teuthology.orchestra.run.smithi083.stderr:tar: /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-20T19:13:22.621 INFO:teuthology.orchestra.run.smithi083.stderr:tar: Error is not recoverable: exiting now 2024-09-20T19:13:22.623 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/gabrioux-2024-09-20_12:24:58-orch:cephadm-wip-guits-main-2024-09-20-1028-distro-default-smithi/7912715/remote/smithi142/crash 2024-09-20T19:13:22.624 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/crash -- . 2024-09-20T19:13:22.671 INFO:teuthology.orchestra.run.smithi142.stderr:tar: /var/lib/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-20T19:13:22.671 INFO:teuthology.orchestra.run.smithi142.stderr:tar: Error is not recoverable: exiting now 2024-09-20T19:13:22.672 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-20T19:13:22.673 DEBUG:teuthology.orchestra.run.smithi083:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph.log | egrep CEPHADM_ | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v CEPHADM_STRAY_DAEMON | egrep -v CEPHADM_FAILED_DAEMON | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | head -n 1 2024-09-20T19:13:22.756 INFO:tasks.cephadm:Compressing logs... 2024-09-20T19:13:22.756 DEBUG:teuthology.orchestra.run.smithi083:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-20T19:13:22.758 DEBUG:teuthology.orchestra.run.smithi142:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-20T19:13:22.794 INFO:teuthology.orchestra.run.smithi083.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-20T19:13:22.794 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph-osd.0.log 2024-09-20T19:13:22.794 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph-osd.1.log 2024-09-20T19:13:22.794 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/objectstore_tool.2.log 2024-09-20T19:13:22.795 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph-osd.0.log: /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-20T19:13:22.799 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/objectstore_tool.2.log: gzip -5 --verbose -- /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-20T19:13:22.799 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph-mon.c.log 2024-09-20T19:13:22.799 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-20T19:13:22.799 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph-mon.a.log 2024-09-20T19:13:22.799 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph-osd.2.log 2024-09-20T19:13:22.799 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph-mgr.y.log 2024-09-20T19:13:22.799 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph.log 2024-09-20T19:13:22.800 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph-client.rgw.foo.a.log 2024-09-20T19:13:22.801 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph.log: gzip -5 --verbose -- /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph-osd.3.log 2024-09-20T19:13:22.801 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph-client.rgw.foo.a.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-20T19:13:22.801 INFO:teuthology.orchestra.run.smithi083.stderr: 59.1% -- replaced with /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph-client.rgw.foo.a.log.gz 2024-09-20T19:13:22.803 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph-osd.3.log: /var/log/ceph/cephadm.log: 94.0% -- replaced with /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/objectstore_tool.2.log.gz 2024-09-20T19:13:22.806 INFO:teuthology.orchestra.run.smithi142.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose: No such file or directory -- 2024-09-20T19:13:22.806 INFO:teuthology.orchestra.run.smithi142.stderr: /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/tcmu-runner.log 2024-09-20T19:13:22.806 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/objectstore_tool.2.log 2024-09-20T19:13:22.806 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/tcmu-runner.log: gzip -5 --verbose -- /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph-osd.7.log 2024-09-20T19:13:22.806 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/objectstore_tool.2.log: gzip -5 --verbose -- /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-20T19:13:22.806 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph-osd.4.log 2024-09-20T19:13:22.806 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-20T19:13:22.806 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-20T19:13:22.806 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph-osd.6.log 2024-09-20T19:13:22.806 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph-osd.5.log 2024-09-20T19:13:22.806 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph.log 2024-09-20T19:13:22.806 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph-mgr.x.log 2024-09-20T19:13:22.807 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph.log: gzip -5 --verbose -- /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph-mon.b.log 2024-09-20T19:13:22.807 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-20T19:13:22.807 INFO:teuthology.orchestra.run.smithi083.stderr: 91.9% -- replaced with /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-20T19:13:22.807 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph-mon.b.log: /var/log/ceph/cephadm.log: 63.1% -- replaced with /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/tcmu-runner.log.gz 2024-09-20T19:13:22.870 INFO:teuthology.orchestra.run.smithi083.stderr: 89.2% -- replaced with /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-20T19:13:22.891 INFO:teuthology.orchestra.run.smithi083.stderr: 94.0% -- replaced with /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-20T19:13:22.903 INFO:teuthology.orchestra.run.smithi142.stderr: 91.7% -- replaced with /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph-mgr.x.log.gz 2024-09-20T19:13:22.907 INFO:teuthology.orchestra.run.smithi142.stderr: 90.0% -- replaced with /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-20T19:13:22.993 INFO:teuthology.orchestra.run.smithi083.stderr: 91.5% -- replaced with /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-20T19:13:23.033 INFO:teuthology.orchestra.run.smithi142.stderr: 79.8% -- replaced with /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-20T19:13:23.066 INFO:teuthology.orchestra.run.smithi083.stderr: 95.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-20T19:13:23.088 INFO:teuthology.orchestra.run.smithi142.stderr: 85.6% -- replaced with /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-20T19:13:23.094 INFO:teuthology.orchestra.run.smithi142.stderr: 94.3% -- replaced with /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/objectstore_tool.2.log.gz 2024-09-20T19:13:23.107 INFO:teuthology.orchestra.run.smithi142.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-20T19:13:23.231 INFO:teuthology.orchestra.run.smithi142.stderr: 91.4% -- replaced with /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-20T19:13:25.348 INFO:teuthology.orchestra.run.smithi083.stderr: 90.1% -- replaced with /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph-mgr.y.log.gz 2024-09-20T19:13:26.243 INFO:teuthology.orchestra.run.smithi142.stderr: 91.1% -- replaced with /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph-mon.b.log.gz 2024-09-20T19:13:30.620 INFO:teuthology.orchestra.run.smithi083.stderr: 91.0% -- replaced with /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph-mon.c.log.gz 2024-09-20T19:13:52.928 INFO:teuthology.orchestra.run.smithi083.stderr: 90.3% -- replaced with /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph-mon.a.log.gz 2024-09-20T19:23:51.665 INFO:teuthology.orchestra.run.smithi142.stderr: 92.3% -- replaced with /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph-osd.5.log.gz 2024-09-20T19:25:05.729 INFO:teuthology.orchestra.run.smithi083.stderr: 93.8% -- replaced with /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph-osd.2.log.gz 2024-09-20T19:25:33.245 INFO:teuthology.orchestra.run.smithi142.stderr: 92.8% -- replaced with /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph-osd.4.log.gz 2024-09-20T19:27:02.685 INFO:teuthology.orchestra.run.smithi142.stderr: 92.9% -- replaced with /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph-osd.7.log.gz 2024-09-20T19:27:38.056 INFO:teuthology.orchestra.run.smithi142.stderr: 92.4% -- replaced with /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph-osd.6.log.gz 2024-09-20T19:27:38.058 INFO:teuthology.orchestra.run.smithi142.stderr: 2024-09-20T19:27:38.058 INFO:teuthology.orchestra.run.smithi142.stderr:real 14m15.287s 2024-09-20T19:27:38.058 INFO:teuthology.orchestra.run.smithi142.stderr:user 6m16.445s 2024-09-20T19:27:38.058 INFO:teuthology.orchestra.run.smithi142.stderr:sys 0m18.716s 2024-09-20T19:31:18.560 INFO:teuthology.orchestra.run.smithi083.stderr: 92.9% -- replaced with /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph-osd.3.log.gz 2024-09-20T19:33:39.390 INFO:teuthology.orchestra.run.smithi083.stderr: 92.2% -- replaced with /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph-osd.1.log.gz 2024-09-20T19:34:10.799 INFO:teuthology.orchestra.run.smithi083.stderr: 92.4% -- replaced with /var/log/ceph/1ed5a50a-7780-11ef-baf5-efdc52797490/ceph-osd.0.log.gz 2024-09-20T19:34:10.847 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-09-20T19:34:10.847 INFO:teuthology.orchestra.run.smithi083.stderr:real 20m48.077s 2024-09-20T19:34:10.847 INFO:teuthology.orchestra.run.smithi083.stderr:user 7m6.582s 2024-09-20T19:34:10.847 INFO:teuthology.orchestra.run.smithi083.stderr:sys 0m22.114s 2024-09-20T19:34:10.848 INFO:tasks.cephadm:Archiving logs... 2024-09-20T19:34:10.848 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-20_12:24:58-orch:cephadm-wip-guits-main-2024-09-20-1028-distro-default-smithi/7912715/remote/smithi083/log 2024-09-20T19:34:10.849 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-20T19:35:16.799 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-20_12:24:58-orch:cephadm-wip-guits-main-2024-09-20-1028-distro-default-smithi/7912715/remote/smithi142/log 2024-09-20T19:35:16.800 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-20T19:36:16.872 INFO:tasks.cephadm:Removing cluster... 2024-09-20T19:36:16.873 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 --force 2024-09-20T19:36:17.482 INFO:teuthology.orchestra.run.smithi083.stdout:Deleting cluster with fsid: 1ed5a50a-7780-11ef-baf5-efdc52797490 2024-09-20T19:36:18.227 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1ed5a50a-7780-11ef-baf5-efdc52797490 --force 2024-09-20T19:36:18.857 INFO:teuthology.orchestra.run.smithi142.stdout:Deleting cluster with fsid: 1ed5a50a-7780-11ef-baf5-efdc52797490 2024-09-20T19:36:19.603 INFO:tasks.cephadm:Removing cephadm ... 2024-09-20T19:36:19.603 DEBUG:teuthology.orchestra.run.smithi083:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-20T19:36:19.623 DEBUG:teuthology.orchestra.run.smithi142:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-20T19:36:19.638 INFO:tasks.cephadm:Teardown complete 2024-09-20T19:36:19.638 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-20T19:36:19.648 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-20T19:36:19.648 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-20T19:36:19.666 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-20T19:36:19.793 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-20T19:36:19.806 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-20T19:36:19.806 DEBUG:teuthology.orchestra.run.smithi083:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-20T19:36:19.809 DEBUG:teuthology.orchestra.run.smithi142:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-20T19:36:19.824 INFO:teuthology.orchestra.run.smithi083.stderr:bash: line 1: ntpq: command not found 2024-09-20T19:36:19.825 INFO:teuthology.orchestra.run.smithi142.stderr:bash: line 1: ntpq: command not found 2024-09-20T19:36:19.870 INFO:teuthology.orchestra.run.smithi142.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-20T19:36:19.870 INFO:teuthology.orchestra.run.smithi142.stdout:=============================================================================== 2024-09-20T19:36:19.870 INFO:teuthology.orchestra.run.smithi142.stdout:^* hv01.front.sepia.ceph.com 3 6 377 9 +575us[ +602us] +/- 54ms 2024-09-20T19:36:19.870 INFO:teuthology.orchestra.run.smithi142.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 7 +782us[ +782us] +/- 50ms 2024-09-20T19:36:19.870 INFO:teuthology.orchestra.run.smithi142.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 8 -1702us[-1702us] +/- 62ms 2024-09-20T19:36:19.870 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-20T19:36:19.903 INFO:teuthology.orchestra.run.smithi083.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-20T19:36:19.903 INFO:teuthology.orchestra.run.smithi083.stdout:=============================================================================== 2024-09-20T19:36:19.903 INFO:teuthology.orchestra.run.smithi083.stdout:^* hv01.front.sepia.ceph.com 3 6 377 10 +561us[ +567us] +/- 54ms 2024-09-20T19:36:19.903 INFO:teuthology.orchestra.run.smithi083.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 7 +768us[ +768us] +/- 50ms 2024-09-20T19:36:19.904 INFO:teuthology.orchestra.run.smithi083.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 10 -1724us[-1724us] +/- 62ms 2024-09-20T19:36:19.904 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-20T19:36:19.914 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-20T19:36:19.929 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-20T19:36:19.930 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-20T19:36:19.948 DEBUG:teuthology.orchestra.run.smithi083:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-20T19:36:20.054 DEBUG:teuthology.orchestra.run.smithi142:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-20T19:36:20.118 DEBUG:teuthology.orchestra.run.smithi083:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-20T19:36:20.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:36:20.151 DEBUG:teuthology.orchestra.run.smithi142:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-20T19:36:20.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:36:20.178 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-20T19:36:20.190 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-20T19:36:20.258 INFO:teuthology.task.internal:Duration was 3600.720863 seconds 2024-09-20T19:36:20.258 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-20T19:36:20.305 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-20T19:36:20.305 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-20T19:36:20.308 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-20T19:36:20.354 INFO:teuthology.orchestra.run.smithi142.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-20T19:36:20.368 INFO:teuthology.orchestra.run.smithi083.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-20T19:36:20.876 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-20T19:36:20.876 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi083.front.sepia.ceph.com 2024-09-20T19:36:20.877 DEBUG:teuthology.orchestra.run.smithi083:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-20T19:36:20.916 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi142.front.sepia.ceph.com 2024-09-20T19:36:20.917 DEBUG:teuthology.orchestra.run.smithi142:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-20T19:36:20.938 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-20T19:36:20.938 DEBUG:teuthology.orchestra.run.smithi083:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-20T19:36:20.959 DEBUG:teuthology.orchestra.run.smithi142:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-20T19:36:21.017 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-20T19:36:21.017 DEBUG:teuthology.orchestra.run.smithi083:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-20T19:36:21.020 DEBUG:teuthology.orchestra.run.smithi142:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-20T19:36:23.141 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-20T19:36:23.152 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-20T19:36:23.152 DEBUG:teuthology.orchestra.run.smithi083:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-20T19:36:23.183 DEBUG:teuthology.orchestra.run.smithi142:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-20T19:36:23.240 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-20T19:36:23.250 DEBUG:teuthology.orchestra.run.smithi083:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-20T19:36:23.253 DEBUG:teuthology.orchestra.run.smithi142:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-20T19:36:23.283 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern = core 2024-09-20T19:36:23.291 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern = core 2024-09-20T19:36:23.318 DEBUG:teuthology.orchestra.run.smithi083:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-20T19:36:23.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:36:23.355 DEBUG:teuthology.orchestra.run.smithi142:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-20T19:36:23.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T19:36:23.377 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-20T19:36:23.388 INFO:teuthology.task.internal:Transferring archived files... 2024-09-20T19:36:23.389 DEBUG:teuthology.misc:Transferring archived files from smithi083:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-20_12:24:58-orch:cephadm-wip-guits-main-2024-09-20-1028-distro-default-smithi/7912715/remote/smithi083 2024-09-20T19:36:23.389 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-20T19:36:23.468 DEBUG:teuthology.misc:Transferring archived files from smithi142:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-20_12:24:58-orch:cephadm-wip-guits-main-2024-09-20-1028-distro-default-smithi/7912715/remote/smithi142 2024-09-20T19:36:23.469 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-20T19:36:23.514 INFO:teuthology.task.internal:Removing archive directory... 2024-09-20T19:36:23.514 DEBUG:teuthology.orchestra.run.smithi083:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-20T19:36:23.517 DEBUG:teuthology.orchestra.run.smithi142:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-20T19:36:23.565 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-20T19:36:23.575 INFO:teuthology.task.internal:Not uploading archives. 2024-09-20T19:36:23.576 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-20T19:36:23.585 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-20T19:36:23.585 DEBUG:teuthology.orchestra.run.smithi083:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-20T19:36:23.587 DEBUG:teuthology.orchestra.run.smithi142:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-20T19:36:23.600 INFO:teuthology.orchestra.run.smithi083.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 20 19:36 /home/ubuntu/cephtest 2024-09-20T19:36:23.622 INFO:teuthology.orchestra.run.smithi142.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 20 19:36 /home/ubuntu/cephtest 2024-09-20T19:36:23.623 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-20T19:36:23.634 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-20T19:36:23.657 INFO:teuthology.run:Summary data: description: orch:cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root} duration: 3600.7208631038666 flavor: default owner: scheduled_gabrioux@teuthology success: true 2024-09-20T19:36:23.657 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-20T19:36:23.744 INFO:teuthology.run:pass